2024-09-11T22:58:56.874 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-11T22:58:56.902 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-11T22:58:57.009 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901457 branch: squid description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/blogbench}} email: ceph-qa@ceph.com first_in_suite: false job_id: '7901457' kernel: client: sha1: distro kdb: 1 sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules - exec: mon.a: - ceph mgr module enable snap_schedule name: teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: squid ceph: cephfs: max_mds: 5 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_bal_replicate_threshold: 1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 mon warn on pool no app: false osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - online, but wants - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy - slow metadata IO - SLOW_OPS - slow request sha1: b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc subvols: create: 2 subvol_options: --namespace-isolated ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount - mds.dir_split install: ceph: flavor: default sha1: b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=secure - wsync syntax: v2 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: 086c737fa75c22d521a3932c59b31c5e1d1c9ef4 owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 481 sha1: b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc sleep_before_teardown: 0 subset: 57/512 suite: fs suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_086c737fa75c22d521a3932c59b31c5e1d1c9ef4/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: 086c737fa75c22d521a3932c59b31c5e1d1c9ef4 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi026.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKRqBE91AEQTr0p4HGTeWNVs4nfHPCx3cUBLqq4SQcftLTmrGvSAV63xWJdEzN1uFi7CC4+K+tbx/m4pCPeCHrc= smithi076.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAMFp36H2zuwD7j7w13gMdmgnhfzU/LvCmyS3W5uHST0U9HVtr0ITontAajkhE2rNkykIIs/32hBjrOolIiFugo= smithi136.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHURnopKI0U3Ntt8ehPNKFMamlnOR1QCo6k21guEy+Y9Wck6cUUZ1IGqZb8ILiygk0LMjyakHVxyetL+m94TBMg= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate true - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - exec: mon.a: - ceph config set mgr mgr/snap_schedule/log_level debug - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - quiescer: null - workunit: clients: all: - suites/blogbench.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-11_21:24:17 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-11T22:58:57.009 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_086c737fa75c22d521a3932c59b31c5e1d1c9ef4/qa; will attempt to use it 2024-09-11T22:58:57.010 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_086c737fa75c22d521a3932c59b31c5e1d1c9ef4/qa/tasks 2024-09-11T22:58:57.011 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-11T22:58:57.012 INFO:teuthology.task.internal:Checking packages... 2024-09-11T22:58:57.034 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc' 2024-09-11T22:58:57.035 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-11T22:58:57.035 INFO:teuthology.packaging:ref: None 2024-09-11T22:58:57.035 INFO:teuthology.packaging:tag: None 2024-09-11T22:58:57.035 INFO:teuthology.packaging:branch: squid 2024-09-11T22:58:57.035 INFO:teuthology.packaging:sha1: b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-11T22:58:57.035 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2024-09-11T22:58:57.250 INFO:teuthology.task.internal:Found packages for ceph version 19.1.1-314.g086c737f 2024-09-11T22:58:57.252 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-11T22:58:57.258 INFO:teuthology.task.internal:no buildpackages task found 2024-09-11T22:58:57.259 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-11T22:58:57.264 INFO:teuthology.task.internal:Saving configuration 2024-09-11T22:58:57.290 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-11T22:58:57.296 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-11T22:58:57.326 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi026.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901457', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-11 22:53:33.517055', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKRqBE91AEQTr0p4HGTeWNVs4nfHPCx3cUBLqq4SQcftLTmrGvSAV63xWJdEzN1uFi7CC4+K+tbx/m4pCPeCHrc='} 2024-09-11T22:58:57.360 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi076.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901457', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-11 22:53:33.515411', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAMFp36H2zuwD7j7w13gMdmgnhfzU/LvCmyS3W5uHST0U9HVtr0ITontAajkhE2rNkykIIs/32hBjrOolIiFugo='} 2024-09-11T22:58:57.392 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi136.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901457', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-11 22:53:33.519146', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHURnopKI0U3Ntt8ehPNKFMamlnOR1QCo6k21guEy+Y9Wck6cUUZ1IGqZb8ILiygk0LMjyakHVxyetL+m94TBMg='} 2024-09-11T22:58:57.393 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-11T22:58:57.398 INFO:teuthology.task.internal:roles: ubuntu@smithi026.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-09-11T22:58:57.399 INFO:teuthology.task.internal:roles: ubuntu@smithi076.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-09-11T22:58:57.399 INFO:teuthology.task.internal:roles: ubuntu@smithi136.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-09-11T22:58:57.399 INFO:teuthology.run_tasks:Running task console_log... 2024-09-11T22:58:57.551 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fec859535b0>, signals=[15]) 2024-09-11T22:58:57.551 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-11T22:58:57.556 INFO:teuthology.task.internal:Opening connections... 2024-09-11T22:58:57.556 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi026.front.sepia.ceph.com 2024-09-11T22:58:57.557 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi026.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T22:58:57.667 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi076.front.sepia.ceph.com 2024-09-11T22:58:57.668 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi076.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T22:58:57.749 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi136.front.sepia.ceph.com 2024-09-11T22:58:57.750 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi136.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T22:58:57.839 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-11T22:58:57.846 DEBUG:teuthology.orchestra.run.smithi026:> uname -m 2024-09-11T22:58:57.864 INFO:teuthology.orchestra.run.smithi026.stdout:x86_64 2024-09-11T22:58:57.864 DEBUG:teuthology.orchestra.run.smithi026:> cat /etc/os-release 2024-09-11T22:58:57.922 INFO:teuthology.orchestra.run.smithi026.stdout:NAME="CentOS Stream" 2024-09-11T22:58:57.922 INFO:teuthology.orchestra.run.smithi026.stdout:VERSION="9" 2024-09-11T22:58:57.922 INFO:teuthology.orchestra.run.smithi026.stdout:ID="centos" 2024-09-11T22:58:57.922 INFO:teuthology.orchestra.run.smithi026.stdout:ID_LIKE="rhel fedora" 2024-09-11T22:58:57.923 INFO:teuthology.orchestra.run.smithi026.stdout:VERSION_ID="9" 2024-09-11T22:58:57.923 INFO:teuthology.orchestra.run.smithi026.stdout:PLATFORM_ID="platform:el9" 2024-09-11T22:58:57.923 INFO:teuthology.orchestra.run.smithi026.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-11T22:58:57.923 INFO:teuthology.orchestra.run.smithi026.stdout:ANSI_COLOR="0;31" 2024-09-11T22:58:57.923 INFO:teuthology.orchestra.run.smithi026.stdout:LOGO="fedora-logo-icon" 2024-09-11T22:58:57.923 INFO:teuthology.orchestra.run.smithi026.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-11T22:58:57.923 INFO:teuthology.orchestra.run.smithi026.stdout:HOME_URL="https://centos.org/" 2024-09-11T22:58:57.923 INFO:teuthology.orchestra.run.smithi026.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-11T22:58:57.923 INFO:teuthology.orchestra.run.smithi026.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-11T22:58:57.923 INFO:teuthology.orchestra.run.smithi026.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-11T22:58:57.924 INFO:teuthology.lock.ops:Updating smithi026.front.sepia.ceph.com on lock server 2024-09-11T22:58:57.950 DEBUG:teuthology.orchestra.run.smithi076:> uname -m 2024-09-11T22:58:57.969 INFO:teuthology.orchestra.run.smithi076.stdout:x86_64 2024-09-11T22:58:57.970 DEBUG:teuthology.orchestra.run.smithi076:> cat /etc/os-release 2024-09-11T22:58:58.027 INFO:teuthology.orchestra.run.smithi076.stdout:NAME="CentOS Stream" 2024-09-11T22:58:58.028 INFO:teuthology.orchestra.run.smithi076.stdout:VERSION="9" 2024-09-11T22:58:58.028 INFO:teuthology.orchestra.run.smithi076.stdout:ID="centos" 2024-09-11T22:58:58.028 INFO:teuthology.orchestra.run.smithi076.stdout:ID_LIKE="rhel fedora" 2024-09-11T22:58:58.028 INFO:teuthology.orchestra.run.smithi076.stdout:VERSION_ID="9" 2024-09-11T22:58:58.028 INFO:teuthology.orchestra.run.smithi076.stdout:PLATFORM_ID="platform:el9" 2024-09-11T22:58:58.028 INFO:teuthology.orchestra.run.smithi076.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-11T22:58:58.028 INFO:teuthology.orchestra.run.smithi076.stdout:ANSI_COLOR="0;31" 2024-09-11T22:58:58.028 INFO:teuthology.orchestra.run.smithi076.stdout:LOGO="fedora-logo-icon" 2024-09-11T22:58:58.028 INFO:teuthology.orchestra.run.smithi076.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-11T22:58:58.028 INFO:teuthology.orchestra.run.smithi076.stdout:HOME_URL="https://centos.org/" 2024-09-11T22:58:58.028 INFO:teuthology.orchestra.run.smithi076.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-11T22:58:58.028 INFO:teuthology.orchestra.run.smithi076.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-11T22:58:58.029 INFO:teuthology.orchestra.run.smithi076.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-11T22:58:58.029 INFO:teuthology.lock.ops:Updating smithi076.front.sepia.ceph.com on lock server 2024-09-11T22:58:58.058 DEBUG:teuthology.orchestra.run.smithi136:> uname -m 2024-09-11T22:58:58.077 INFO:teuthology.orchestra.run.smithi136.stdout:x86_64 2024-09-11T22:58:58.078 DEBUG:teuthology.orchestra.run.smithi136:> cat /etc/os-release 2024-09-11T22:58:58.136 INFO:teuthology.orchestra.run.smithi136.stdout:NAME="CentOS Stream" 2024-09-11T22:58:58.136 INFO:teuthology.orchestra.run.smithi136.stdout:VERSION="9" 2024-09-11T22:58:58.136 INFO:teuthology.orchestra.run.smithi136.stdout:ID="centos" 2024-09-11T22:58:58.136 INFO:teuthology.orchestra.run.smithi136.stdout:ID_LIKE="rhel fedora" 2024-09-11T22:58:58.136 INFO:teuthology.orchestra.run.smithi136.stdout:VERSION_ID="9" 2024-09-11T22:58:58.136 INFO:teuthology.orchestra.run.smithi136.stdout:PLATFORM_ID="platform:el9" 2024-09-11T22:58:58.136 INFO:teuthology.orchestra.run.smithi136.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-11T22:58:58.136 INFO:teuthology.orchestra.run.smithi136.stdout:ANSI_COLOR="0;31" 2024-09-11T22:58:58.136 INFO:teuthology.orchestra.run.smithi136.stdout:LOGO="fedora-logo-icon" 2024-09-11T22:58:58.137 INFO:teuthology.orchestra.run.smithi136.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-11T22:58:58.137 INFO:teuthology.orchestra.run.smithi136.stdout:HOME_URL="https://centos.org/" 2024-09-11T22:58:58.137 INFO:teuthology.orchestra.run.smithi136.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-11T22:58:58.137 INFO:teuthology.orchestra.run.smithi136.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-11T22:58:58.137 INFO:teuthology.orchestra.run.smithi136.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-11T22:58:58.137 INFO:teuthology.lock.ops:Updating smithi136.front.sepia.ceph.com on lock server 2024-09-11T22:58:58.166 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-11T22:58:58.176 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-11T22:58:58.191 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-11T22:58:58.191 DEBUG:teuthology.orchestra.run.smithi026:> test '!' -e /home/ubuntu/cephtest 2024-09-11T22:58:58.195 DEBUG:teuthology.orchestra.run.smithi076:> test '!' -e /home/ubuntu/cephtest 2024-09-11T22:58:58.197 DEBUG:teuthology.orchestra.run.smithi136:> test '!' -e /home/ubuntu/cephtest 2024-09-11T22:58:58.214 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-11T22:58:58.220 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-11T22:58:58.220 DEBUG:teuthology.orchestra.run.smithi026:> test -z $(ls -A /var/lib/ceph) 2024-09-11T22:58:58.252 DEBUG:teuthology.orchestra.run.smithi076:> test -z $(ls -A /var/lib/ceph) 2024-09-11T22:58:58.255 DEBUG:teuthology.orchestra.run.smithi136:> test -z $(ls -A /var/lib/ceph) 2024-09-11T22:58:58.299 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-11T22:58:58.404 INFO:teuthology.run_tasks:Running task kernel... 2024-09-11T22:58:58.418 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': 1, 'sha1': 'distro'} 2024-09-11T22:58:58.418 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-09-11T22:58:58.418 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-09-11T22:58:58.419 DEBUG:teuthology.orchestra.run.smithi026:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-11T22:58:58.436 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T22:58:58.436 DEBUG:teuthology.orchestra.run.smithi026:> uname -r 2024-09-11T22:58:58.495 INFO:teuthology.orchestra.run.smithi026.stdout:5.14.0-503.el9.x86_64 2024-09-11T22:58:58.495 INFO:teuthology.task.kernel:Running kernel on smithi026: 5.14.0-503.el9.x86_64 2024-09-11T22:58:58.495 DEBUG:teuthology.orchestra.run.smithi026:> sudo yum install -y kernel 2024-09-11T22:59:01.879 INFO:teuthology.orchestra.run.smithi026.stdout:CentOS Stream 9 - BaseOS 5.0 MB/s | 8.2 MB 00:01 2024-09-11T22:59:08.872 INFO:teuthology.orchestra.run.smithi026.stdout:CentOS Stream 9 - AppStream 3.5 MB/s | 20 MB 00:05 2024-09-11T22:59:13.613 INFO:teuthology.orchestra.run.smithi026.stdout:CentOS Stream 9 - CRB 7.8 MB/s | 6.5 MB 00:00 2024-09-11T22:59:15.014 INFO:teuthology.orchestra.run.smithi026.stdout:CentOS Stream 9 - Extras packages 116 kB/s | 19 kB 00:00 2024-09-11T22:59:15.777 INFO:teuthology.orchestra.run.smithi026.stdout:Extra Packages for Enterprise Linux 36 MB/s | 23 MB 00:00 2024-09-11T22:59:21.399 INFO:teuthology.orchestra.run.smithi026.stdout:lab-extras 33 kB/s | 1.7 kB 00:00 2024-09-11T22:59:23.472 INFO:teuthology.orchestra.run.smithi026.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-11T22:59:23.473 INFO:teuthology.orchestra.run.smithi026.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-11T22:59:23.570 INFO:teuthology.orchestra.run.smithi026.stdout:Dependencies resolved. 2024-09-11T22:59:23.575 INFO:teuthology.orchestra.run.smithi026.stdout:================================================================================ 2024-09-11T22:59:23.575 INFO:teuthology.orchestra.run.smithi026.stdout: Package Architecture Version Repository Size 2024-09-11T22:59:23.576 INFO:teuthology.orchestra.run.smithi026.stdout:================================================================================ 2024-09-11T22:59:23.576 INFO:teuthology.orchestra.run.smithi026.stdout:Installing: 2024-09-11T22:59:23.576 INFO:teuthology.orchestra.run.smithi026.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-11T22:59:23.576 INFO:teuthology.orchestra.run.smithi026.stdout:Installing dependencies: 2024-09-11T22:59:23.576 INFO:teuthology.orchestra.run.smithi026.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-11T22:59:23.576 INFO:teuthology.orchestra.run.smithi026.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-11T22:59:23.576 INFO:teuthology.orchestra.run.smithi026.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-11T22:59:23.576 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-11T22:59:23.576 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction Summary 2024-09-11T22:59:23.576 INFO:teuthology.orchestra.run.smithi026.stdout:================================================================================ 2024-09-11T22:59:23.576 INFO:teuthology.orchestra.run.smithi026.stdout:Install 4 Packages 2024-09-11T22:59:23.576 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-11T22:59:23.577 INFO:teuthology.orchestra.run.smithi026.stdout:Total download size: 79 M 2024-09-11T22:59:23.577 INFO:teuthology.orchestra.run.smithi026.stdout:Installed size: 126 M 2024-09-11T22:59:23.577 INFO:teuthology.orchestra.run.smithi026.stdout:Downloading Packages: 2024-09-11T22:59:23.717 INFO:teuthology.orchestra.run.smithi026.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 245 kB/s | 23 kB 00:00 2024-09-11T22:59:26.234 INFO:teuthology.orchestra.run.smithi026.stdout:(2/4): kernel-core-5.14.0-505.el9.x86_64.rpm 6.0 MB/s | 16 MB 00:02 2024-09-11T22:59:27.809 INFO:teuthology.orchestra.run.smithi026.stdout:(3/4): kernel-modules-core-5.14.0-505.el9.x86_6 7.0 MB/s | 29 MB 00:04 2024-09-11T22:59:29.084 INFO:teuthology.orchestra.run.smithi026.stdout:(4/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 6.3 MB/s | 35 MB 00:05 2024-09-11T22:59:29.084 INFO:teuthology.orchestra.run.smithi026.stdout:-------------------------------------------------------------------------------- 2024-09-11T22:59:29.084 INFO:teuthology.orchestra.run.smithi026.stdout:Total 14 MB/s | 79 MB 00:05 2024-09-11T22:59:29.797 INFO:teuthology.orchestra.run.smithi026.stdout:Running transaction check 2024-09-11T22:59:30.216 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction check succeeded. 2024-09-11T22:59:30.216 INFO:teuthology.orchestra.run.smithi026.stdout:Running transaction test 2024-09-11T22:59:31.214 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction test succeeded. 2024-09-11T22:59:31.214 INFO:teuthology.orchestra.run.smithi026.stdout:Running transaction 2024-09-11T22:59:33.793 INFO:teuthology.orchestra.run.smithi026.stdout: Preparing : 1/1 2024-09-11T22:59:34.485 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-11T22:59:34.567 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-11T22:59:37.368 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-11T22:59:37.502 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-11T22:59:42.614 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-11T22:59:42.661 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-11T22:59:47.805 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-11T23:00:26.923 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-11T23:00:26.964 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-11T23:00:27.351 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-11T23:00:27.351 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-11T23:00:27.351 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-11T23:00:27.351 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-11T23:00:28.258 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-11T23:00:28.259 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-11T23:00:28.259 INFO:teuthology.orchestra.run.smithi026.stdout:Installed: 2024-09-11T23:00:28.259 INFO:teuthology.orchestra.run.smithi026.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-11T23:00:28.259 INFO:teuthology.orchestra.run.smithi026.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-11T23:00:28.259 INFO:teuthology.orchestra.run.smithi026.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-11T23:00:28.259 INFO:teuthology.orchestra.run.smithi026.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-11T23:00:28.259 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-11T23:00:28.259 INFO:teuthology.orchestra.run.smithi026.stdout:Complete! 2024-09-11T23:00:28.605 DEBUG:teuthology.orchestra.run.smithi026:> rpm -q kernel | sort -rV | head -n 1 2024-09-11T23:00:28.736 INFO:teuthology.orchestra.run.smithi026.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-11T23:00:28.736 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-11T23:00:28.736 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-11T23:00:28.736 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-11T23:00:28.737 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-11T23:00:28.737 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-09-11T23:00:28.737 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi026.front.sepia.ceph.com, path=None, version=distro) 2024-09-11T23:00:28.737 DEBUG:teuthology.orchestra.run.smithi026:> rpm -q kernel | sort -rV | head -n 1 2024-09-11T23:00:28.795 INFO:teuthology.orchestra.run.smithi026.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-11T23:00:28.796 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-11T23:00:28.796 DEBUG:teuthology.orchestra.run.smithi026:> sudo rpm -qi grub2-tools 2024-09-11T23:00:28.834 INFO:teuthology.orchestra.run.smithi026.stdout:Name : grub2-tools 2024-09-11T23:00:28.834 INFO:teuthology.orchestra.run.smithi026.stdout:Epoch : 1 2024-09-11T23:00:28.834 INFO:teuthology.orchestra.run.smithi026.stdout:Version : 2.06 2024-09-11T23:00:28.834 INFO:teuthology.orchestra.run.smithi026.stdout:Release : 82.el9 2024-09-11T23:00:28.834 INFO:teuthology.orchestra.run.smithi026.stdout:Architecture: x86_64 2024-09-11T23:00:28.834 INFO:teuthology.orchestra.run.smithi026.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-11T23:00:28.834 INFO:teuthology.orchestra.run.smithi026.stdout:Group : Unspecified 2024-09-11T23:00:28.834 INFO:teuthology.orchestra.run.smithi026.stdout:Size : 8274670 2024-09-11T23:00:28.834 INFO:teuthology.orchestra.run.smithi026.stdout:License : GPLv3+ 2024-09-11T23:00:28.834 INFO:teuthology.orchestra.run.smithi026.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-11T23:00:28.834 INFO:teuthology.orchestra.run.smithi026.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-11T23:00:28.834 INFO:teuthology.orchestra.run.smithi026.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-11T23:00:28.835 INFO:teuthology.orchestra.run.smithi026.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-11T23:00:28.835 INFO:teuthology.orchestra.run.smithi026.stdout:Packager : builder@centos.org 2024-09-11T23:00:28.835 INFO:teuthology.orchestra.run.smithi026.stdout:Vendor : CentOS 2024-09-11T23:00:28.835 INFO:teuthology.orchestra.run.smithi026.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-11T23:00:28.835 INFO:teuthology.orchestra.run.smithi026.stdout:Summary : Support tools for GRUB. 2024-09-11T23:00:28.835 INFO:teuthology.orchestra.run.smithi026.stdout:Description : 2024-09-11T23:00:28.835 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-11T23:00:28.835 INFO:teuthology.orchestra.run.smithi026.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-11T23:00:28.835 INFO:teuthology.orchestra.run.smithi026.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-11T23:00:28.835 INFO:teuthology.orchestra.run.smithi026.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-11T23:00:28.835 INFO:teuthology.orchestra.run.smithi026.stdout:hardware devices. 2024-09-11T23:00:28.835 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-11T23:00:28.835 INFO:teuthology.orchestra.run.smithi026.stdout:This subpackage provides tools for support of all platforms. 2024-09-11T23:00:28.836 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-11T23:00:28.836 INFO:teuthology.task.kernel:Updating grub on smithi026 to boot 5.14.0-505.el9.x86_64 2024-09-11T23:00:28.837 DEBUG:teuthology.orchestra.run.smithi026:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-11T23:00:29.564 INFO:teuthology.orchestra.run.smithi026.stderr:Generating grub configuration file ... 2024-09-11T23:00:30.970 INFO:teuthology.orchestra.run.smithi026.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-11T23:00:30.998 INFO:teuthology.orchestra.run.smithi026.stderr:done 2024-09-11T23:00:31.000 DEBUG:teuthology.orchestra.run.smithi026:> mktemp 2024-09-11T23:00:31.015 INFO:teuthology.orchestra.run.smithi026.stdout:/tmp/tmp.I0E8SgdmBf 2024-09-11T23:00:31.016 DEBUG:teuthology.orchestra.run.smithi026:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.I0E8SgdmBf 2024-09-11T23:00:31.082 DEBUG:teuthology.orchestra.run.smithi026:> sudo chmod 0666 /tmp/tmp.I0E8SgdmBf 2024-09-11T23:00:31.228 DEBUG:teuthology.orchestra.remote:smithi026:/tmp/tmp.I0E8SgdmBf is 6KB 2024-09-11T23:00:31.276 DEBUG:teuthology.orchestra.run.smithi026:> rm -fr /tmp/tmp.I0E8SgdmBf 2024-09-11T23:00:31.291 DEBUG:teuthology.orchestra.run.smithi026:> sudo /bin/ls /boot/loader/entries || true 2024-09-11T23:00:31.375 INFO:teuthology.orchestra.run.smithi026.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-11T23:00:31.375 INFO:teuthology.orchestra.run.smithi026.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-11T23:00:31.375 INFO:teuthology.orchestra.run.smithi026.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-11T23:00:31.375 INFO:teuthology.orchestra.run.smithi026.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-11T23:00:31.377 DEBUG:teuthology.orchestra.run.smithi026:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-11T23:00:31.496 DEBUG:teuthology.orchestra.run.smithi026:> sudo shutdown -r now 2024-09-11T23:01:01.500 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-09-11T23:01:01.500 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi026.front.sepia.ceph.com' 2024-09-11T23:01:01.501 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi026.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T23:01:20.061 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.26 2024-09-11T23:01:29.062 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi026.front.sepia.ceph.com' 2024-09-11T23:01:29.063 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi026.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T23:01:47.453 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.26 2024-09-11T23:01:59.454 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi026.front.sepia.ceph.com' 2024-09-11T23:01:59.456 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi026.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T23:01:59.880 DEBUG:teuthology.orchestra.run.smithi026:> true 2024-09-11T23:02:00.383 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi026.front.sepia.ceph.com' 2024-09-11T23:02:00.383 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.14.0-505.el9.x86_64"... 2024-09-11T23:02:00.383 DEBUG:teuthology.orchestra.run.smithi026:> uname -r 2024-09-11T23:02:00.398 INFO:teuthology.orchestra.run.smithi026.stdout:5.14.0-505.el9.x86_64 2024-09-11T23:02:00.398 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-11T23:02:00.398 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-11T23:02:00.399 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-11T23:02:01.400 DEBUG:teuthology.parallel:result is None 2024-09-11T23:02:01.400 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-11T23:02:01.408 INFO:teuthology.task.internal:Creating test directory... 2024-09-11T23:02:01.409 DEBUG:teuthology.orchestra.run.smithi026:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-11T23:02:01.412 DEBUG:teuthology.orchestra.run.smithi076:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-11T23:02:01.415 DEBUG:teuthology.orchestra.run.smithi136:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-11T23:02:01.433 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-11T23:02:01.439 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-11T23:02:01.464 INFO:teuthology.task.internal:Creating archive directory... 2024-09-11T23:02:01.464 DEBUG:teuthology.orchestra.run.smithi026:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-11T23:02:01.469 DEBUG:teuthology.orchestra.run.smithi076:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-11T23:02:01.473 DEBUG:teuthology.orchestra.run.smithi136:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-11T23:02:01.622 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-11T23:02:01.629 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-11T23:02:01.629 DEBUG:teuthology.orchestra.run.smithi076:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-11T23:02:01.646 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T23:02:01.647 DEBUG:teuthology.orchestra.run.smithi136:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-11T23:02:01.664 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T23:02:01.664 DEBUG:teuthology.orchestra.run.smithi026:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-11T23:02:01.666 DEBUG:teuthology.orchestra.run.smithi076:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-11T23:02:01.689 DEBUG:teuthology.orchestra.run.smithi136:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-11T23:02:01.739 INFO:teuthology.orchestra.run.smithi136.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-11T23:02:01.750 INFO:teuthology.orchestra.run.smithi076.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-11T23:02:01.756 INFO:teuthology.orchestra.run.smithi136.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-11T23:02:01.758 INFO:teuthology.orchestra.run.smithi026.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-11T23:02:01.766 INFO:teuthology.orchestra.run.smithi076.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-11T23:02:01.795 INFO:teuthology.orchestra.run.smithi026.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-11T23:02:01.797 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-11T23:02:01.804 INFO:teuthology.task.internal:Configuring sudo... 2024-09-11T23:02:01.805 DEBUG:teuthology.orchestra.run.smithi026:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-11T23:02:01.807 DEBUG:teuthology.orchestra.run.smithi076:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-11T23:02:01.809 DEBUG:teuthology.orchestra.run.smithi136:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-11T23:02:01.839 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-11T23:02:01.848 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-11T23:02:01.848 DEBUG:teuthology.orchestra.run.smithi026:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-11T23:02:01.873 DEBUG:teuthology.orchestra.run.smithi076:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-11T23:02:01.879 DEBUG:teuthology.orchestra.run.smithi136:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-11T23:02:01.897 DEBUG:teuthology.orchestra.run.smithi026:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-11T23:02:01.976 DEBUG:teuthology.orchestra.run.smithi026:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-11T23:02:02.023 DEBUG:teuthology.orchestra.run.smithi026:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-11T23:02:02.110 DEBUG:teuthology.orchestra.run.smithi026:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-11T23:02:02.148 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-09-11T23:02:02.148 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-11T23:02:02.223 DEBUG:teuthology.orchestra.run.smithi076:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-11T23:02:02.277 DEBUG:teuthology.orchestra.run.smithi076:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-11T23:02:02.325 DEBUG:teuthology.orchestra.run.smithi076:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-11T23:02:02.417 DEBUG:teuthology.orchestra.run.smithi076:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-11T23:02:02.452 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-09-11T23:02:02.452 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-11T23:02:02.519 DEBUG:teuthology.orchestra.run.smithi136:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-11T23:02:02.572 DEBUG:teuthology.orchestra.run.smithi136:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-11T23:02:02.623 DEBUG:teuthology.orchestra.run.smithi136:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-11T23:02:02.716 DEBUG:teuthology.orchestra.run.smithi136:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-11T23:02:02.751 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-09-11T23:02:02.751 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-11T23:02:02.816 DEBUG:teuthology.orchestra.run.smithi026:> sudo service rsyslog restart 2024-09-11T23:02:02.819 DEBUG:teuthology.orchestra.run.smithi076:> sudo service rsyslog restart 2024-09-11T23:02:02.821 DEBUG:teuthology.orchestra.run.smithi136:> sudo service rsyslog restart 2024-09-11T23:02:02.860 INFO:teuthology.orchestra.run.smithi076.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-11T23:02:02.861 INFO:teuthology.orchestra.run.smithi026.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-11T23:02:02.891 INFO:teuthology.orchestra.run.smithi136.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-11T23:02:03.375 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-11T23:02:03.383 INFO:teuthology.task.internal:Starting timer... 2024-09-11T23:02:03.383 INFO:teuthology.run_tasks:Running task pcp... 2024-09-11T23:02:03.393 INFO:teuthology.run_tasks:Running task selinux... 2024-09-11T23:02:03.401 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-09-11T23:02:03.401 DEBUG:teuthology.orchestra.run.smithi026:> sudo service auditd rotate 2024-09-11T23:02:03.469 INFO:teuthology.orchestra.run.smithi026.stdout:Rotating logs: 2024-09-11T23:02:03.471 DEBUG:teuthology.orchestra.run.smithi076:> sudo service auditd rotate 2024-09-11T23:02:03.524 INFO:teuthology.orchestra.run.smithi076.stdout:Rotating logs: 2024-09-11T23:02:03.526 DEBUG:teuthology.orchestra.run.smithi136:> sudo service auditd rotate 2024-09-11T23:02:03.569 INFO:teuthology.orchestra.run.smithi136.stdout:Rotating logs: 2024-09-11T23:02:03.571 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-11T23:02:03.571 DEBUG:teuthology.orchestra.run.smithi026:> /usr/sbin/getenforce 2024-09-11T23:02:03.595 INFO:teuthology.orchestra.run.smithi026.stdout:Permissive 2024-09-11T23:02:03.595 DEBUG:teuthology.orchestra.run.smithi076:> /usr/sbin/getenforce 2024-09-11T23:02:03.619 INFO:teuthology.orchestra.run.smithi076.stdout:Permissive 2024-09-11T23:02:03.619 DEBUG:teuthology.orchestra.run.smithi136:> /usr/sbin/getenforce 2024-09-11T23:02:03.650 INFO:teuthology.orchestra.run.smithi136.stdout:Permissive 2024-09-11T23:02:03.650 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi026.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi076.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi136.front.sepia.ceph.com': 'permissive'} 2024-09-11T23:02:03.650 DEBUG:teuthology.orchestra.run.smithi026:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-11T23:02:03.677 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T23:02:03.677 DEBUG:teuthology.orchestra.run.smithi076:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-11T23:02:03.708 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T23:02:03.708 DEBUG:teuthology.orchestra.run.smithi136:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-11T23:02:03.737 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T23:02:03.737 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-11T23:02:03.738 DEBUG:teuthology.orchestra.run.smithi026:> sudo /usr/sbin/setenforce permissive 2024-09-11T23:02:03.779 DEBUG:teuthology.orchestra.run.smithi076:> sudo /usr/sbin/setenforce permissive 2024-09-11T23:02:03.821 DEBUG:teuthology.orchestra.run.smithi136:> sudo /usr/sbin/setenforce permissive 2024-09-11T23:02:03.857 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-11T23:02:03.867 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-11T23:02:03.879 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-11T23:02:03.901 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi026.front.sepia.ceph.com,smithi076.front.sepia.ceph.com,smithi136.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-11T23:06:13.713 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi026.front.sepia.ceph.com'), Remote(name='ubuntu@smithi076.front.sepia.ceph.com'), Remote(name='ubuntu@smithi136.front.sepia.ceph.com')] 2024-09-11T23:06:13.714 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi026.front.sepia.ceph.com' 2024-09-11T23:06:13.716 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi026.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T23:06:13.804 DEBUG:teuthology.orchestra.run.smithi026:> true 2024-09-11T23:06:13.885 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi026.front.sepia.ceph.com' 2024-09-11T23:06:13.885 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi076.front.sepia.ceph.com' 2024-09-11T23:06:13.886 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi076.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T23:06:13.967 DEBUG:teuthology.orchestra.run.smithi076:> true 2024-09-11T23:06:14.045 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi076.front.sepia.ceph.com' 2024-09-11T23:06:14.045 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi136.front.sepia.ceph.com' 2024-09-11T23:06:14.045 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi136.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T23:06:14.132 DEBUG:teuthology.orchestra.run.smithi136:> true 2024-09-11T23:06:14.219 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi136.front.sepia.ceph.com' 2024-09-11T23:06:14.220 INFO:teuthology.run_tasks:Running task clock... 2024-09-11T23:06:14.230 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-11T23:06:14.231 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-11T23:06:14.231 DEBUG:teuthology.orchestra.run.smithi026:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-11T23:06:14.234 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-11T23:06:14.234 DEBUG:teuthology.orchestra.run.smithi076:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-11T23:06:14.236 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-11T23:06:14.236 DEBUG:teuthology.orchestra.run.smithi136:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-11T23:06:14.263 INFO:teuthology.orchestra.run.smithi026.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-11T23:06:14.265 INFO:teuthology.orchestra.run.smithi076.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-11T23:06:14.279 INFO:teuthology.orchestra.run.smithi026.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-11T23:06:14.280 INFO:teuthology.orchestra.run.smithi076.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-11T23:06:14.293 INFO:teuthology.orchestra.run.smithi136.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-11T23:06:14.305 INFO:teuthology.orchestra.run.smithi076.stderr:sudo: ntpd: command not found 2024-09-11T23:06:14.305 INFO:teuthology.orchestra.run.smithi026.stderr:sudo: ntpd: command not found 2024-09-11T23:06:14.308 INFO:teuthology.orchestra.run.smithi136.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-11T23:06:14.316 INFO:teuthology.orchestra.run.smithi076.stdout:506 Cannot talk to daemon 2024-09-11T23:06:14.318 INFO:teuthology.orchestra.run.smithi026.stdout:506 Cannot talk to daemon 2024-09-11T23:06:14.331 INFO:teuthology.orchestra.run.smithi076.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-11T23:06:14.334 INFO:teuthology.orchestra.run.smithi026.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-11T23:06:14.335 INFO:teuthology.orchestra.run.smithi136.stderr:sudo: ntpd: command not found 2024-09-11T23:06:14.345 INFO:teuthology.orchestra.run.smithi076.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-11T23:06:14.347 INFO:teuthology.orchestra.run.smithi136.stdout:506 Cannot talk to daemon 2024-09-11T23:06:14.349 INFO:teuthology.orchestra.run.smithi026.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-11T23:06:14.362 INFO:teuthology.orchestra.run.smithi136.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-11T23:06:14.376 INFO:teuthology.orchestra.run.smithi136.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-11T23:06:14.399 INFO:teuthology.orchestra.run.smithi076.stderr:bash: line 1: ntpq: command not found 2024-09-11T23:06:14.403 INFO:teuthology.orchestra.run.smithi076.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-11T23:06:14.403 INFO:teuthology.orchestra.run.smithi076.stdout:=============================================================================== 2024-09-11T23:06:14.404 INFO:teuthology.orchestra.run.smithi076.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T23:06:14.404 INFO:teuthology.orchestra.run.smithi076.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T23:06:14.404 INFO:teuthology.orchestra.run.smithi076.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T23:06:14.404 INFO:teuthology.orchestra.run.smithi076.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T23:06:14.411 INFO:teuthology.orchestra.run.smithi026.stderr:bash: line 1: ntpq: command not found 2024-09-11T23:06:14.415 INFO:teuthology.orchestra.run.smithi026.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-11T23:06:14.415 INFO:teuthology.orchestra.run.smithi026.stdout:=============================================================================== 2024-09-11T23:06:14.415 INFO:teuthology.orchestra.run.smithi026.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T23:06:14.415 INFO:teuthology.orchestra.run.smithi026.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T23:06:14.415 INFO:teuthology.orchestra.run.smithi026.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T23:06:14.415 INFO:teuthology.orchestra.run.smithi026.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T23:06:14.432 INFO:teuthology.orchestra.run.smithi136.stderr:bash: line 1: ntpq: command not found 2024-09-11T23:06:14.435 INFO:teuthology.orchestra.run.smithi136.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-11T23:06:14.435 INFO:teuthology.orchestra.run.smithi136.stdout:=============================================================================== 2024-09-11T23:06:14.436 INFO:teuthology.orchestra.run.smithi136.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T23:06:14.436 INFO:teuthology.orchestra.run.smithi136.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T23:06:14.436 INFO:teuthology.orchestra.run.smithi136.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T23:06:14.436 INFO:teuthology.orchestra.run.smithi136.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T23:06:14.437 INFO:teuthology.run_tasks:Running task install... 2024-09-11T23:06:14.445 DEBUG:teuthology.task.install:project ceph 2024-09-11T23:06:14.445 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc'}} 2024-09-11T23:06:14.445 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc'} 2024-09-11T23:06:14.445 INFO:teuthology.task.install:Using flavor: default 2024-09-11T23:06:14.450 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-11T23:06:14.451 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-09-11T23:06:14.451 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc', 'tag': None, 'wait_for_package': False} 2024-09-11T23:06:14.452 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-11T23:06:14.454 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc', 'tag': None, 'wait_for_package': False} 2024-09-11T23:06:14.454 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-11T23:06:14.455 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc', 'tag': None, 'wait_for_package': False} 2024-09-11T23:06:14.455 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-11T23:06:14.608 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/ 2024-09-11T23:06:14.608 INFO:teuthology.task.install.rpm:Package version is 19.1.1-298.gb53b6d3e 2024-09-11T23:06:14.626 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/ 2024-09-11T23:06:14.626 INFO:teuthology.task.install.rpm:Package version is 19.1.1-298.gb53b6d3e 2024-09-11T23:06:14.637 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/ 2024-09-11T23:06:14.637 INFO:teuthology.task.install.rpm:Package version is 19.1.1-298.gb53b6d3e 2024-09-11T23:06:14.735 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-11T23:06:14.736 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-09-11T23:06:14.736 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-11T23:06:14.742 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-11T23:06:14.742 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-09-11T23:06:14.742 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-11T23:06:14.750 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-11T23:06:14.750 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-09-11T23:06:14.750 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-11T23:06:14.763 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-11T23:06:14.763 DEBUG:teuthology.orchestra.run.smithi026:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-11T23:06:14.768 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-11T23:06:14.768 DEBUG:teuthology.orchestra.run.smithi076:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-11T23:06:14.776 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-11T23:06:14.777 DEBUG:teuthology.orchestra.run.smithi136:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-11T23:06:14.831 DEBUG:teuthology.orchestra.run.smithi026:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-11T23:06:14.833 DEBUG:teuthology.orchestra.run.smithi076:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-11T23:06:14.844 DEBUG:teuthology.orchestra.run.smithi136:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-11T23:06:14.909 DEBUG:teuthology.orchestra.run.smithi026:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-11T23:06:14.910 DEBUG:teuthology.orchestra.run.smithi076:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-11T23:06:14.920 DEBUG:teuthology.orchestra.run.smithi136:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-11T23:06:14.977 INFO:teuthology.orchestra.run.smithi026.stdout:check_obsoletes = 1 2024-09-11T23:06:14.977 DEBUG:teuthology.orchestra.run.smithi026:> sudo yum clean all 2024-09-11T23:06:14.978 INFO:teuthology.orchestra.run.smithi076.stdout:check_obsoletes = 1 2024-09-11T23:06:14.978 DEBUG:teuthology.orchestra.run.smithi076:> sudo yum clean all 2024-09-11T23:06:14.984 INFO:teuthology.orchestra.run.smithi136.stdout:check_obsoletes = 1 2024-09-11T23:06:14.986 DEBUG:teuthology.orchestra.run.smithi136:> sudo yum clean all 2024-09-11T23:06:15.205 INFO:teuthology.orchestra.run.smithi076.stdout:45 files removed 2024-09-11T23:06:15.211 INFO:teuthology.orchestra.run.smithi136.stdout:45 files removed 2024-09-11T23:06:15.221 INFO:teuthology.orchestra.run.smithi026.stdout:45 files removed 2024-09-11T23:06:15.233 DEBUG:teuthology.orchestra.run.smithi076:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-11T23:06:15.239 DEBUG:teuthology.orchestra.run.smithi136:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-11T23:06:15.248 DEBUG:teuthology.orchestra.run.smithi026:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-11T23:06:15.765 INFO:teuthology.orchestra.run.smithi076.stdout:ceph packages for x86_64 327 kB/s | 84 kB 00:00 2024-09-11T23:06:15.793 INFO:teuthology.orchestra.run.smithi136.stdout:ceph packages for x86_64 301 kB/s | 84 kB 00:00 2024-09-11T23:06:15.815 INFO:teuthology.orchestra.run.smithi026.stdout:ceph packages for x86_64 314 kB/s | 84 kB 00:00 2024-09-11T23:06:16.048 INFO:teuthology.orchestra.run.smithi076.stdout:ceph noarch packages 48 kB/s | 12 kB 00:00 2024-09-11T23:06:16.077 INFO:teuthology.orchestra.run.smithi136.stdout:ceph noarch packages 48 kB/s | 12 kB 00:00 2024-09-11T23:06:16.099 INFO:teuthology.orchestra.run.smithi026.stdout:ceph noarch packages 49 kB/s | 12 kB 00:00 2024-09-11T23:06:16.315 INFO:teuthology.orchestra.run.smithi076.stdout:ceph source packages 8.1 kB/s | 1.9 kB 00:00 2024-09-11T23:06:16.353 INFO:teuthology.orchestra.run.smithi136.stdout:ceph source packages 7.9 kB/s | 1.9 kB 00:00 2024-09-11T23:06:16.366 INFO:teuthology.orchestra.run.smithi026.stdout:ceph source packages 8.3 kB/s | 1.9 kB 00:00 2024-09-11T23:06:16.966 INFO:teuthology.orchestra.run.smithi026.stdout:CentOS Stream 9 - BaseOS 15 MB/s | 8.2 MB 00:00 2024-09-11T23:06:16.977 INFO:teuthology.orchestra.run.smithi136.stdout:CentOS Stream 9 - BaseOS 14 MB/s | 8.2 MB 00:00 2024-09-11T23:06:17.041 INFO:teuthology.orchestra.run.smithi076.stdout:CentOS Stream 9 - BaseOS 12 MB/s | 8.2 MB 00:00 2024-09-11T23:06:18.870 INFO:teuthology.orchestra.run.smithi136.stdout:CentOS Stream 9 - AppStream 26 MB/s | 20 MB 00:00 2024-09-11T23:06:19.225 INFO:teuthology.orchestra.run.smithi076.stdout:CentOS Stream 9 - AppStream 19 MB/s | 20 MB 00:01 2024-09-11T23:06:23.063 INFO:teuthology.orchestra.run.smithi136.stdout:CentOS Stream 9 - CRB 18 MB/s | 6.5 MB 00:00 2024-09-11T23:06:23.192 INFO:teuthology.orchestra.run.smithi026.stdout:CentOS Stream 9 - AppStream 4.0 MB/s | 20 MB 00:05 2024-09-11T23:06:23.568 INFO:teuthology.orchestra.run.smithi076.stdout:CentOS Stream 9 - CRB 16 MB/s | 6.5 MB 00:00 2024-09-11T23:06:24.430 INFO:teuthology.orchestra.run.smithi136.stdout:CentOS Stream 9 - Extras packages 120 kB/s | 19 kB 00:00 2024-09-11T23:06:25.618 INFO:teuthology.orchestra.run.smithi136.stdout:Extra Packages for Enterprise Linux 21 MB/s | 23 MB 00:01 2024-09-11T23:06:26.152 INFO:teuthology.orchestra.run.smithi076.stdout:CentOS Stream 9 - Extras packages 15 kB/s | 19 kB 00:01 2024-09-11T23:06:27.348 INFO:teuthology.orchestra.run.smithi076.stdout:Extra Packages for Enterprise Linux 21 MB/s | 23 MB 00:01 2024-09-11T23:06:28.959 INFO:teuthology.orchestra.run.smithi026.stdout:CentOS Stream 9 - CRB 3.7 MB/s | 6.5 MB 00:01 2024-09-11T23:06:30.535 INFO:teuthology.orchestra.run.smithi026.stdout:CentOS Stream 9 - Extras packages 63 kB/s | 19 kB 00:00 2024-09-11T23:06:31.090 INFO:teuthology.orchestra.run.smithi136.stdout:lab-extras 25 kB/s | 1.7 kB 00:00 2024-09-11T23:06:31.489 INFO:teuthology.orchestra.run.smithi026.stdout:Extra Packages for Enterprise Linux 27 MB/s | 23 MB 00:00 2024-09-11T23:06:32.871 INFO:teuthology.orchestra.run.smithi076.stdout:lab-extras 46 kB/s | 1.7 kB 00:00 2024-09-11T23:06:33.386 INFO:teuthology.orchestra.run.smithi136.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-11T23:06:33.387 INFO:teuthology.orchestra.run.smithi136.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-11T23:06:33.400 INFO:teuthology.orchestra.run.smithi136.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-11T23:06:33.400 INFO:teuthology.orchestra.run.smithi136.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-11T23:06:33.402 INFO:teuthology.orchestra.run.smithi136.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-11T23:06:33.402 INFO:teuthology.orchestra.run.smithi136.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-11T23:06:33.405 INFO:teuthology.orchestra.run.smithi136.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-11T23:06:33.407 INFO:teuthology.orchestra.run.smithi136.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-11T23:06:33.408 INFO:teuthology.orchestra.run.smithi136.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-11T23:06:33.472 INFO:teuthology.orchestra.run.smithi136.stdout:Dependencies resolved. 2024-09-11T23:06:33.494 INFO:teuthology.orchestra.run.smithi136.stdout:====================================================================================== 2024-09-11T23:06:33.494 INFO:teuthology.orchestra.run.smithi136.stdout: Package Arch Version Repository Size 2024-09-11T23:06:33.494 INFO:teuthology.orchestra.run.smithi136.stdout:====================================================================================== 2024-09-11T23:06:33.494 INFO:teuthology.orchestra.run.smithi136.stdout:Installing: 2024-09-11T23:06:33.494 INFO:teuthology.orchestra.run.smithi136.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-11T23:06:33.495 INFO:teuthology.orchestra.run.smithi136.stdout: ceph x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 6.5 k 2024-09-11T23:06:33.495 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-base x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 5.4 M 2024-09-11T23:06:33.495 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-fuse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 897 k 2024-09-11T23:06:33.495 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 145 k 2024-09-11T23:06:33.495 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 1.5 M 2024-09-11T23:06:33.495 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr-cephadm noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 146 k 2024-09-11T23:06:33.495 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr-dashboard noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 3.6 M 2024-09-11T23:06:33.495 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 7.4 M 2024-09-11T23:06:33.495 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr-rook noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 49 k 2024-09-11T23:06:33.495 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-radosgw x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 11 M 2024-09-11T23:06:33.495 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-test x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 49 M 2024-09-11T23:06:33.495 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-volume noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 285 k 2024-09-11T23:06:33.496 INFO:teuthology.orchestra.run.smithi136.stdout: cephadm noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 765 k 2024-09-11T23:06:33.496 INFO:teuthology.orchestra.run.smithi136.stdout: cephfs-mirror x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 228 k 2024-09-11T23:06:33.496 INFO:teuthology.orchestra.run.smithi136.stdout: cephfs-shell x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 27 k 2024-09-11T23:06:33.496 INFO:teuthology.orchestra.run.smithi136.stdout: cephfs-top noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 18 k 2024-09-11T23:06:33.496 INFO:teuthology.orchestra.run.smithi136.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-11T23:06:33.496 INFO:teuthology.orchestra.run.smithi136.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-11T23:06:33.496 INFO:teuthology.orchestra.run.smithi136.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-11T23:06:33.496 INFO:teuthology.orchestra.run.smithi136.stdout: libcephfs-devel x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 32 k 2024-09-11T23:06:33.496 INFO:teuthology.orchestra.run.smithi136.stdout: libcephfs2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 752 k 2024-09-11T23:06:33.496 INFO:teuthology.orchestra.run.smithi136.stdout: librados-devel x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 127 k 2024-09-11T23:06:33.496 INFO:teuthology.orchestra.run.smithi136.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-11T23:06:33.496 INFO:teuthology.orchestra.run.smithi136.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-11T23:06:33.497 INFO:teuthology.orchestra.run.smithi136.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-11T23:06:33.497 INFO:teuthology.orchestra.run.smithi136.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-11T23:06:33.497 INFO:teuthology.orchestra.run.smithi136.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-11T23:06:33.497 INFO:teuthology.orchestra.run.smithi136.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-11T23:06:33.497 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cephfs x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 162 k 2024-09-11T23:06:33.497 INFO:teuthology.orchestra.run.smithi136.stdout: python3-rados x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 321 k 2024-09-11T23:06:33.497 INFO:teuthology.orchestra.run.smithi136.stdout: python3-rbd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 301 k 2024-09-11T23:06:33.497 INFO:teuthology.orchestra.run.smithi136.stdout: python3-rgw x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 100 k 2024-09-11T23:06:33.497 INFO:teuthology.orchestra.run.smithi136.stdout: rbd-fuse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 85 k 2024-09-11T23:06:33.497 INFO:teuthology.orchestra.run.smithi136.stdout: rbd-mirror x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.1 M 2024-09-11T23:06:33.497 INFO:teuthology.orchestra.run.smithi136.stdout: rbd-nbd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 171 k 2024-09-11T23:06:33.498 INFO:teuthology.orchestra.run.smithi136.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-11T23:06:33.498 INFO:teuthology.orchestra.run.smithi136.stdout:Upgrading: 2024-09-11T23:06:33.498 INFO:teuthology.orchestra.run.smithi136.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-11T23:06:33.498 INFO:teuthology.orchestra.run.smithi136.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-11T23:06:33.498 INFO:teuthology.orchestra.run.smithi136.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-11T23:06:33.498 INFO:teuthology.orchestra.run.smithi136.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-11T23:06:33.498 INFO:teuthology.orchestra.run.smithi136.stdout: librados2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.4 M 2024-09-11T23:06:33.498 INFO:teuthology.orchestra.run.smithi136.stdout: librbd1 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.2 M 2024-09-11T23:06:33.498 INFO:teuthology.orchestra.run.smithi136.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-11T23:06:33.498 INFO:teuthology.orchestra.run.smithi136.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-11T23:06:33.499 INFO:teuthology.orchestra.run.smithi136.stdout:Installing dependencies: 2024-09-11T23:06:33.499 INFO:teuthology.orchestra.run.smithi136.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-11T23:06:33.499 INFO:teuthology.orchestra.run.smithi136.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-11T23:06:33.499 INFO:teuthology.orchestra.run.smithi136.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-11T23:06:33.499 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-common x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 21 M 2024-09-11T23:06:33.499 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-grafana-dashboards noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 32 k 2024-09-11T23:06:33.499 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mds x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 2.4 M 2024-09-11T23:06:33.499 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr-modules-core noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 249 k 2024-09-11T23:06:33.499 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mon x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 4.7 M 2024-09-11T23:06:33.499 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-osd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 17 M 2024-09-11T23:06:33.499 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-prometheus-alerts noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 16 k 2024-09-11T23:06:33.499 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-selinux x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 25 k 2024-09-11T23:06:33.500 INFO:teuthology.orchestra.run.smithi136.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-11T23:06:33.500 INFO:teuthology.orchestra.run.smithi136.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-11T23:06:33.500 INFO:teuthology.orchestra.run.smithi136.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-11T23:06:33.500 INFO:teuthology.orchestra.run.smithi136.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-11T23:06:33.500 INFO:teuthology.orchestra.run.smithi136.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-11T23:06:33.500 INFO:teuthology.orchestra.run.smithi136.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-11T23:06:33.500 INFO:teuthology.orchestra.run.smithi136.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-11T23:06:33.500 INFO:teuthology.orchestra.run.smithi136.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-11T23:06:33.500 INFO:teuthology.orchestra.run.smithi136.stdout: libcephsqlite x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 163 k 2024-09-11T23:06:33.500 INFO:teuthology.orchestra.run.smithi136.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-11T23:06:33.500 INFO:teuthology.orchestra.run.smithi136.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-11T23:06:33.500 INFO:teuthology.orchestra.run.smithi136.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-11T23:06:33.501 INFO:teuthology.orchestra.run.smithi136.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-11T23:06:33.501 INFO:teuthology.orchestra.run.smithi136.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-11T23:06:33.501 INFO:teuthology.orchestra.run.smithi136.stdout: libradosstriper1 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 506 k 2024-09-11T23:06:33.501 INFO:teuthology.orchestra.run.smithi136.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-11T23:06:33.501 INFO:teuthology.orchestra.run.smithi136.stdout: librgw2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 5.3 M 2024-09-11T23:06:33.501 INFO:teuthology.orchestra.run.smithi136.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-11T23:06:33.501 INFO:teuthology.orchestra.run.smithi136.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-11T23:06:33.501 INFO:teuthology.orchestra.run.smithi136.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-11T23:06:33.501 INFO:teuthology.orchestra.run.smithi136.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-11T23:06:33.501 INFO:teuthology.orchestra.run.smithi136.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-11T23:06:33.501 INFO:teuthology.orchestra.run.smithi136.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-11T23:06:33.501 INFO:teuthology.orchestra.run.smithi136.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-11T23:06:33.501 INFO:teuthology.orchestra.run.smithi136.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-11T23:06:33.501 INFO:teuthology.orchestra.run.smithi136.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-11T23:06:33.502 INFO:teuthology.orchestra.run.smithi136.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-11T23:06:33.502 INFO:teuthology.orchestra.run.smithi136.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-11T23:06:33.502 INFO:teuthology.orchestra.run.smithi136.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-11T23:06:33.502 INFO:teuthology.orchestra.run.smithi136.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-11T23:06:33.502 INFO:teuthology.orchestra.run.smithi136.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-11T23:06:33.502 INFO:teuthology.orchestra.run.smithi136.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-11T23:06:33.502 INFO:teuthology.orchestra.run.smithi136.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-11T23:06:33.502 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-11T23:06:33.502 INFO:teuthology.orchestra.run.smithi136.stdout: python3-ceph-argparse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 45 k 2024-09-11T23:06:33.503 INFO:teuthology.orchestra.run.smithi136.stdout: python3-ceph-common x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 136 k 2024-09-11T23:06:33.503 INFO:teuthology.orchestra.run.smithi136.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-11T23:06:33.503 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-11T23:06:33.503 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-11T23:06:33.503 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-11T23:06:33.503 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-11T23:06:33.503 INFO:teuthology.orchestra.run.smithi136.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-11T23:06:33.503 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-11T23:06:33.503 INFO:teuthology.orchestra.run.smithi136.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-11T23:06:33.503 INFO:teuthology.orchestra.run.smithi136.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-11T23:06:33.503 INFO:teuthology.orchestra.run.smithi136.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-11T23:06:33.503 INFO:teuthology.orchestra.run.smithi136.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-11T23:06:33.503 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-11T23:06:33.504 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-11T23:06:33.504 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-11T23:06:33.504 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-11T23:06:33.504 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-11T23:06:33.504 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-11T23:06:33.504 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-11T23:06:33.504 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-11T23:06:33.504 INFO:teuthology.orchestra.run.smithi136.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-11T23:06:33.504 INFO:teuthology.orchestra.run.smithi136.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-11T23:06:33.504 INFO:teuthology.orchestra.run.smithi136.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-11T23:06:33.504 INFO:teuthology.orchestra.run.smithi136.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-11T23:06:33.504 INFO:teuthology.orchestra.run.smithi136.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-11T23:06:33.504 INFO:teuthology.orchestra.run.smithi136.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-11T23:06:33.505 INFO:teuthology.orchestra.run.smithi136.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-11T23:06:33.505 INFO:teuthology.orchestra.run.smithi136.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-11T23:06:33.505 INFO:teuthology.orchestra.run.smithi136.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-11T23:06:33.505 INFO:teuthology.orchestra.run.smithi136.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-11T23:06:33.505 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-11T23:06:33.505 INFO:teuthology.orchestra.run.smithi136.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-11T23:06:33.505 INFO:teuthology.orchestra.run.smithi136.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-11T23:06:33.505 INFO:teuthology.orchestra.run.smithi136.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-11T23:06:33.505 INFO:teuthology.orchestra.run.smithi136.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-11T23:06:33.505 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-11T23:06:33.505 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-11T23:06:33.505 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-11T23:06:33.505 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-11T23:06:33.505 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-11T23:06:33.506 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-11T23:06:33.506 INFO:teuthology.orchestra.run.smithi136.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-11T23:06:33.506 INFO:teuthology.orchestra.run.smithi136.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-11T23:06:33.506 INFO:teuthology.orchestra.run.smithi136.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-11T23:06:33.506 INFO:teuthology.orchestra.run.smithi136.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-11T23:06:33.506 INFO:teuthology.orchestra.run.smithi136.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-11T23:06:33.506 INFO:teuthology.orchestra.run.smithi136.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-11T23:06:33.506 INFO:teuthology.orchestra.run.smithi136.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-11T23:06:33.506 INFO:teuthology.orchestra.run.smithi136.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-11T23:06:33.506 INFO:teuthology.orchestra.run.smithi136.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-11T23:06:33.506 INFO:teuthology.orchestra.run.smithi136.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-11T23:06:33.506 INFO:teuthology.orchestra.run.smithi136.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-11T23:06:33.506 INFO:teuthology.orchestra.run.smithi136.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-11T23:06:33.506 INFO:teuthology.orchestra.run.smithi136.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-11T23:06:33.506 INFO:teuthology.orchestra.run.smithi136.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-11T23:06:33.507 INFO:teuthology.orchestra.run.smithi136.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-11T23:06:33.507 INFO:teuthology.orchestra.run.smithi136.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-11T23:06:33.507 INFO:teuthology.orchestra.run.smithi136.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-11T23:06:33.507 INFO:teuthology.orchestra.run.smithi136.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-11T23:06:33.507 INFO:teuthology.orchestra.run.smithi136.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-11T23:06:33.507 INFO:teuthology.orchestra.run.smithi136.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-11T23:06:33.507 INFO:teuthology.orchestra.run.smithi136.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-11T23:06:33.507 INFO:teuthology.orchestra.run.smithi136.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-11T23:06:33.507 INFO:teuthology.orchestra.run.smithi136.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-11T23:06:33.507 INFO:teuthology.orchestra.run.smithi136.stdout:Installing weak dependencies: 2024-09-11T23:06:33.507 INFO:teuthology.orchestra.run.smithi136.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-11T23:06:33.507 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-09-11T23:06:33.507 INFO:teuthology.orchestra.run.smithi136.stdout:Transaction Summary 2024-09-11T23:06:33.507 INFO:teuthology.orchestra.run.smithi136.stdout:====================================================================================== 2024-09-11T23:06:33.508 INFO:teuthology.orchestra.run.smithi136.stdout:Install 148 Packages 2024-09-11T23:06:33.508 INFO:teuthology.orchestra.run.smithi136.stdout:Upgrade 8 Packages 2024-09-11T23:06:33.508 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-09-11T23:06:33.508 INFO:teuthology.orchestra.run.smithi136.stdout:Total download size: 226 M 2024-09-11T23:06:33.508 INFO:teuthology.orchestra.run.smithi136.stdout:Downloading Packages: 2024-09-11T23:06:34.058 INFO:teuthology.orchestra.run.smithi136.stdout:(1/156): ceph-19.1.1-298.gb53b6d3e.el9.x86_64.r 54 kB/s | 6.5 kB 00:00 2024-09-11T23:06:34.261 INFO:teuthology.orchestra.run.smithi136.stdout:(2/156): ceph-fuse-19.1.1-298.gb53b6d3e.el9.x86 4.4 MB/s | 897 kB 00:00 2024-09-11T23:06:34.300 INFO:teuthology.orchestra.run.smithi136.stdout:(3/156): ceph-immutable-object-cache-19.1.1-298 3.4 MB/s | 145 kB 00:00 2024-09-11T23:06:34.642 INFO:teuthology.orchestra.run.smithi136.stdout:(4/156): ceph-mds-19.1.1-298.gb53b6d3e.el9.x86_ 7.0 MB/s | 2.4 MB 00:00 2024-09-11T23:06:34.718 INFO:teuthology.orchestra.run.smithi136.stdout:(5/156): ceph-base-19.1.1-298.gb53b6d3e.el9.x86 6.9 MB/s | 5.4 MB 00:00 2024-09-11T23:06:34.835 INFO:teuthology.orchestra.run.smithi136.stdout:(6/156): ceph-mgr-19.1.1-298.gb53b6d3e.el9.x86_ 7.9 MB/s | 1.5 MB 00:00 2024-09-11T23:06:35.051 INFO:teuthology.orchestra.run.smithi136.stdout:(7/156): ceph-mon-19.1.1-298.gb53b6d3e.el9.x86_ 14 MB/s | 4.7 MB 00:00 2024-09-11T23:06:35.148 INFO:teuthology.orchestra.run.smithi076.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-11T23:06:35.148 INFO:teuthology.orchestra.run.smithi076.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-11T23:06:35.160 INFO:teuthology.orchestra.run.smithi076.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-11T23:06:35.161 INFO:teuthology.orchestra.run.smithi076.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-11T23:06:35.162 INFO:teuthology.orchestra.run.smithi076.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-11T23:06:35.163 INFO:teuthology.orchestra.run.smithi076.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-11T23:06:35.164 INFO:teuthology.orchestra.run.smithi076.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-11T23:06:35.167 INFO:teuthology.orchestra.run.smithi076.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-11T23:06:35.167 INFO:teuthology.orchestra.run.smithi076.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-11T23:06:35.230 INFO:teuthology.orchestra.run.smithi076.stdout:Dependencies resolved. 2024-09-11T23:06:35.246 INFO:teuthology.orchestra.run.smithi076.stdout:====================================================================================== 2024-09-11T23:06:35.246 INFO:teuthology.orchestra.run.smithi076.stdout: Package Arch Version Repository Size 2024-09-11T23:06:35.246 INFO:teuthology.orchestra.run.smithi076.stdout:====================================================================================== 2024-09-11T23:06:35.246 INFO:teuthology.orchestra.run.smithi076.stdout:Installing: 2024-09-11T23:06:35.246 INFO:teuthology.orchestra.run.smithi076.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-11T23:06:35.246 INFO:teuthology.orchestra.run.smithi076.stdout: ceph x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 6.5 k 2024-09-11T23:06:35.246 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-base x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 5.4 M 2024-09-11T23:06:35.246 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-fuse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 897 k 2024-09-11T23:06:35.246 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 145 k 2024-09-11T23:06:35.246 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mgr x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 1.5 M 2024-09-11T23:06:35.246 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mgr-cephadm noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 146 k 2024-09-11T23:06:35.246 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mgr-dashboard noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 3.6 M 2024-09-11T23:06:35.246 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 7.4 M 2024-09-11T23:06:35.247 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mgr-rook noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 49 k 2024-09-11T23:06:35.247 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-radosgw x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 11 M 2024-09-11T23:06:35.247 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-test x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 49 M 2024-09-11T23:06:35.247 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-volume noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 285 k 2024-09-11T23:06:35.247 INFO:teuthology.orchestra.run.smithi076.stdout: cephadm noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 765 k 2024-09-11T23:06:35.247 INFO:teuthology.orchestra.run.smithi076.stdout: cephfs-mirror x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 228 k 2024-09-11T23:06:35.247 INFO:teuthology.orchestra.run.smithi076.stdout: cephfs-shell x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 27 k 2024-09-11T23:06:35.247 INFO:teuthology.orchestra.run.smithi076.stdout: cephfs-top noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 18 k 2024-09-11T23:06:35.247 INFO:teuthology.orchestra.run.smithi076.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-11T23:06:35.247 INFO:teuthology.orchestra.run.smithi076.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-11T23:06:35.247 INFO:teuthology.orchestra.run.smithi076.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-11T23:06:35.247 INFO:teuthology.orchestra.run.smithi076.stdout: libcephfs-devel x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 32 k 2024-09-11T23:06:35.248 INFO:teuthology.orchestra.run.smithi076.stdout: libcephfs2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 752 k 2024-09-11T23:06:35.248 INFO:teuthology.orchestra.run.smithi076.stdout: librados-devel x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 127 k 2024-09-11T23:06:35.248 INFO:teuthology.orchestra.run.smithi076.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-11T23:06:35.248 INFO:teuthology.orchestra.run.smithi076.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-11T23:06:35.248 INFO:teuthology.orchestra.run.smithi076.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-11T23:06:35.248 INFO:teuthology.orchestra.run.smithi076.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-11T23:06:35.248 INFO:teuthology.orchestra.run.smithi076.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-11T23:06:35.248 INFO:teuthology.orchestra.run.smithi076.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-11T23:06:35.248 INFO:teuthology.orchestra.run.smithi076.stdout: python3-cephfs x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 162 k 2024-09-11T23:06:35.248 INFO:teuthology.orchestra.run.smithi076.stdout: python3-rados x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 321 k 2024-09-11T23:06:35.248 INFO:teuthology.orchestra.run.smithi076.stdout: python3-rbd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 301 k 2024-09-11T23:06:35.248 INFO:teuthology.orchestra.run.smithi076.stdout: python3-rgw x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 100 k 2024-09-11T23:06:35.248 INFO:teuthology.orchestra.run.smithi076.stdout: rbd-fuse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 85 k 2024-09-11T23:06:35.249 INFO:teuthology.orchestra.run.smithi076.stdout: rbd-mirror x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.1 M 2024-09-11T23:06:35.249 INFO:teuthology.orchestra.run.smithi076.stdout: rbd-nbd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 171 k 2024-09-11T23:06:35.249 INFO:teuthology.orchestra.run.smithi076.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-11T23:06:35.249 INFO:teuthology.orchestra.run.smithi076.stdout:Upgrading: 2024-09-11T23:06:35.249 INFO:teuthology.orchestra.run.smithi076.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-11T23:06:35.249 INFO:teuthology.orchestra.run.smithi076.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-11T23:06:35.249 INFO:teuthology.orchestra.run.smithi076.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-11T23:06:35.249 INFO:teuthology.orchestra.run.smithi076.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-11T23:06:35.249 INFO:teuthology.orchestra.run.smithi076.stdout: librados2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.4 M 2024-09-11T23:06:35.249 INFO:teuthology.orchestra.run.smithi076.stdout: librbd1 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.2 M 2024-09-11T23:06:35.249 INFO:teuthology.orchestra.run.smithi076.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-11T23:06:35.249 INFO:teuthology.orchestra.run.smithi076.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-11T23:06:35.249 INFO:teuthology.orchestra.run.smithi076.stdout:Installing dependencies: 2024-09-11T23:06:35.250 INFO:teuthology.orchestra.run.smithi076.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-11T23:06:35.250 INFO:teuthology.orchestra.run.smithi076.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-11T23:06:35.250 INFO:teuthology.orchestra.run.smithi076.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-11T23:06:35.250 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-common x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 21 M 2024-09-11T23:06:35.250 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-grafana-dashboards noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 32 k 2024-09-11T23:06:35.250 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mds x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 2.4 M 2024-09-11T23:06:35.250 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mgr-modules-core noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 249 k 2024-09-11T23:06:35.250 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mon x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 4.7 M 2024-09-11T23:06:35.250 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-osd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 17 M 2024-09-11T23:06:35.250 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-prometheus-alerts noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 16 k 2024-09-11T23:06:35.250 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-selinux x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 25 k 2024-09-11T23:06:35.250 INFO:teuthology.orchestra.run.smithi076.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-11T23:06:35.250 INFO:teuthology.orchestra.run.smithi076.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-11T23:06:35.250 INFO:teuthology.orchestra.run.smithi076.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-11T23:06:35.250 INFO:teuthology.orchestra.run.smithi076.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-11T23:06:35.251 INFO:teuthology.orchestra.run.smithi076.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-11T23:06:35.251 INFO:teuthology.orchestra.run.smithi076.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-11T23:06:35.251 INFO:teuthology.orchestra.run.smithi076.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-11T23:06:35.251 INFO:teuthology.orchestra.run.smithi076.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-11T23:06:35.251 INFO:teuthology.orchestra.run.smithi076.stdout: libcephsqlite x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 163 k 2024-09-11T23:06:35.251 INFO:teuthology.orchestra.run.smithi076.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-11T23:06:35.251 INFO:teuthology.orchestra.run.smithi076.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-11T23:06:35.251 INFO:teuthology.orchestra.run.smithi076.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-11T23:06:35.251 INFO:teuthology.orchestra.run.smithi076.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-11T23:06:35.251 INFO:teuthology.orchestra.run.smithi076.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-11T23:06:35.251 INFO:teuthology.orchestra.run.smithi076.stdout: libradosstriper1 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 506 k 2024-09-11T23:06:35.251 INFO:teuthology.orchestra.run.smithi076.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-11T23:06:35.251 INFO:teuthology.orchestra.run.smithi076.stdout: librgw2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 5.3 M 2024-09-11T23:06:35.252 INFO:teuthology.orchestra.run.smithi076.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-11T23:06:35.252 INFO:teuthology.orchestra.run.smithi076.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-11T23:06:35.252 INFO:teuthology.orchestra.run.smithi076.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-11T23:06:35.252 INFO:teuthology.orchestra.run.smithi076.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-11T23:06:35.252 INFO:teuthology.orchestra.run.smithi076.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-11T23:06:35.252 INFO:teuthology.orchestra.run.smithi076.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-11T23:06:35.252 INFO:teuthology.orchestra.run.smithi076.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-11T23:06:35.252 INFO:teuthology.orchestra.run.smithi076.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-11T23:06:35.252 INFO:teuthology.orchestra.run.smithi076.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-11T23:06:35.252 INFO:teuthology.orchestra.run.smithi076.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-11T23:06:35.252 INFO:teuthology.orchestra.run.smithi076.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-11T23:06:35.252 INFO:teuthology.orchestra.run.smithi076.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-11T23:06:35.252 INFO:teuthology.orchestra.run.smithi076.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-11T23:06:35.252 INFO:teuthology.orchestra.run.smithi076.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-11T23:06:35.253 INFO:teuthology.orchestra.run.smithi076.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-11T23:06:35.253 INFO:teuthology.orchestra.run.smithi076.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-11T23:06:35.253 INFO:teuthology.orchestra.run.smithi076.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-11T23:06:35.253 INFO:teuthology.orchestra.run.smithi076.stdout: python3-ceph-argparse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 45 k 2024-09-11T23:06:35.253 INFO:teuthology.orchestra.run.smithi076.stdout: python3-ceph-common x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 136 k 2024-09-11T23:06:35.253 INFO:teuthology.orchestra.run.smithi076.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-11T23:06:35.253 INFO:teuthology.orchestra.run.smithi076.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-11T23:06:35.253 INFO:teuthology.orchestra.run.smithi076.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-11T23:06:35.253 INFO:teuthology.orchestra.run.smithi076.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-11T23:06:35.253 INFO:teuthology.orchestra.run.smithi076.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-11T23:06:35.253 INFO:teuthology.orchestra.run.smithi076.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-11T23:06:35.253 INFO:teuthology.orchestra.run.smithi076.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-11T23:06:35.254 INFO:teuthology.orchestra.run.smithi076.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-11T23:06:35.254 INFO:teuthology.orchestra.run.smithi076.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-11T23:06:35.254 INFO:teuthology.orchestra.run.smithi076.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-11T23:06:35.254 INFO:teuthology.orchestra.run.smithi076.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-11T23:06:35.254 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-11T23:06:35.254 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-11T23:06:35.254 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-11T23:06:35.254 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-11T23:06:35.254 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-11T23:06:35.254 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-11T23:06:35.254 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-11T23:06:35.254 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-11T23:06:35.254 INFO:teuthology.orchestra.run.smithi076.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-11T23:06:35.255 INFO:teuthology.orchestra.run.smithi076.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-11T23:06:35.255 INFO:teuthology.orchestra.run.smithi076.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-11T23:06:35.255 INFO:teuthology.orchestra.run.smithi076.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-11T23:06:35.255 INFO:teuthology.orchestra.run.smithi076.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-11T23:06:35.255 INFO:teuthology.orchestra.run.smithi076.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-11T23:06:35.255 INFO:teuthology.orchestra.run.smithi076.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-11T23:06:35.255 INFO:teuthology.orchestra.run.smithi076.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-11T23:06:35.255 INFO:teuthology.orchestra.run.smithi076.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-11T23:06:35.255 INFO:teuthology.orchestra.run.smithi076.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-11T23:06:35.255 INFO:teuthology.orchestra.run.smithi076.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-11T23:06:35.255 INFO:teuthology.orchestra.run.smithi076.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-11T23:06:35.255 INFO:teuthology.orchestra.run.smithi076.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-11T23:06:35.255 INFO:teuthology.orchestra.run.smithi076.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-11T23:06:35.256 INFO:teuthology.orchestra.run.smithi076.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-11T23:06:35.256 INFO:teuthology.orchestra.run.smithi076.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-11T23:06:35.256 INFO:teuthology.orchestra.run.smithi076.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-11T23:06:35.256 INFO:teuthology.orchestra.run.smithi076.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-11T23:06:35.256 INFO:teuthology.orchestra.run.smithi076.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-11T23:06:35.256 INFO:teuthology.orchestra.run.smithi076.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-11T23:06:35.256 INFO:teuthology.orchestra.run.smithi076.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-11T23:06:35.256 INFO:teuthology.orchestra.run.smithi076.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-11T23:06:35.257 INFO:teuthology.orchestra.run.smithi076.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-11T23:06:35.257 INFO:teuthology.orchestra.run.smithi076.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-11T23:06:35.257 INFO:teuthology.orchestra.run.smithi076.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-11T23:06:35.257 INFO:teuthology.orchestra.run.smithi076.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-11T23:06:35.257 INFO:teuthology.orchestra.run.smithi076.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-11T23:06:35.257 INFO:teuthology.orchestra.run.smithi076.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-11T23:06:35.257 INFO:teuthology.orchestra.run.smithi076.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-11T23:06:35.257 INFO:teuthology.orchestra.run.smithi076.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-11T23:06:35.257 INFO:teuthology.orchestra.run.smithi076.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-11T23:06:35.257 INFO:teuthology.orchestra.run.smithi076.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-11T23:06:35.257 INFO:teuthology.orchestra.run.smithi076.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-11T23:06:35.257 INFO:teuthology.orchestra.run.smithi076.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-11T23:06:35.257 INFO:teuthology.orchestra.run.smithi076.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-11T23:06:35.258 INFO:teuthology.orchestra.run.smithi076.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-11T23:06:35.258 INFO:teuthology.orchestra.run.smithi076.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-11T23:06:35.258 INFO:teuthology.orchestra.run.smithi076.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-11T23:06:35.258 INFO:teuthology.orchestra.run.smithi076.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-11T23:06:35.258 INFO:teuthology.orchestra.run.smithi076.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-11T23:06:35.258 INFO:teuthology.orchestra.run.smithi076.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-11T23:06:35.258 INFO:teuthology.orchestra.run.smithi076.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-11T23:06:35.258 INFO:teuthology.orchestra.run.smithi076.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-11T23:06:35.258 INFO:teuthology.orchestra.run.smithi076.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-11T23:06:35.258 INFO:teuthology.orchestra.run.smithi076.stdout:Installing weak dependencies: 2024-09-11T23:06:35.258 INFO:teuthology.orchestra.run.smithi076.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-11T23:06:35.258 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-11T23:06:35.258 INFO:teuthology.orchestra.run.smithi076.stdout:Transaction Summary 2024-09-11T23:06:35.258 INFO:teuthology.orchestra.run.smithi076.stdout:====================================================================================== 2024-09-11T23:06:35.259 INFO:teuthology.orchestra.run.smithi076.stdout:Install 148 Packages 2024-09-11T23:06:35.259 INFO:teuthology.orchestra.run.smithi076.stdout:Upgrade 8 Packages 2024-09-11T23:06:35.259 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-11T23:06:35.259 INFO:teuthology.orchestra.run.smithi076.stdout:Total download size: 226 M 2024-09-11T23:06:35.259 INFO:teuthology.orchestra.run.smithi076.stdout:Downloading Packages: 2024-09-11T23:06:35.569 INFO:teuthology.orchestra.run.smithi136.stdout:(8/156): ceph-common-19.1.1-298.gb53b6d3e.el9.x 13 MB/s | 21 MB 00:01 2024-09-11T23:06:35.619 INFO:teuthology.orchestra.run.smithi136.stdout:(9/156): ceph-selinux-19.1.1-298.gb53b6d3e.el9. 506 kB/s | 25 kB 00:00 2024-09-11T23:06:35.735 INFO:teuthology.orchestra.run.smithi136.stdout:(10/156): ceph-radosgw-19.1.1-298.gb53b6d3e.el9 16 MB/s | 11 MB 00:00 2024-09-11T23:06:35.747 INFO:teuthology.orchestra.run.smithi076.stdout:(1/156): ceph-19.1.1-298.gb53b6d3e.el9.x86_64.r 41 kB/s | 6.5 kB 00:00 2024-09-11T23:06:35.802 INFO:teuthology.orchestra.run.smithi136.stdout:(11/156): cephfs-mirror-19.1.1-298.gb53b6d3e.el 3.4 MB/s | 228 kB 00:00 2024-09-11T23:06:35.978 INFO:teuthology.orchestra.run.smithi136.stdout:(12/156): ceph-osd-19.1.1-298.gb53b6d3e.el9.x86 15 MB/s | 17 MB 00:01 2024-09-11T23:06:36.003 INFO:teuthology.orchestra.run.smithi136.stdout:(13/156): cephfs-shell-19.1.1-298.gb53b6d3e.el9 135 kB/s | 27 kB 00:00 2024-09-11T23:06:36.006 INFO:teuthology.orchestra.run.smithi076.stdout:(2/156): ceph-fuse-19.1.1-298.gb53b6d3e.el9.x86 3.4 MB/s | 897 kB 00:00 2024-09-11T23:06:36.028 INFO:teuthology.orchestra.run.smithi136.stdout:(14/156): libcephfs-devel-19.1.1-298.gb53b6d3e. 632 kB/s | 32 kB 00:00 2024-09-11T23:06:36.056 INFO:teuthology.orchestra.run.smithi076.stdout:(3/156): ceph-immutable-object-cache-19.1.1-298 2.8 MB/s | 145 kB 00:00 2024-09-11T23:06:36.070 INFO:teuthology.orchestra.run.smithi136.stdout:(15/156): libcephfs2-19.1.1-298.gb53b6d3e.el9.x 11 MB/s | 752 kB 00:00 2024-09-11T23:06:36.095 INFO:teuthology.orchestra.run.smithi136.stdout:(16/156): libcephsqlite-19.1.1-298.gb53b6d3e.el 2.4 MB/s | 163 kB 00:00 2024-09-11T23:06:36.121 INFO:teuthology.orchestra.run.smithi136.stdout:(17/156): librados-devel-19.1.1-298.gb53b6d3e.e 2.5 MB/s | 127 kB 00:00 2024-09-11T23:06:36.146 INFO:teuthology.orchestra.run.smithi136.stdout:(18/156): libradosstriper1-19.1.1-298.gb53b6d3e 9.8 MB/s | 506 kB 00:00 2024-09-11T23:06:36.213 INFO:teuthology.orchestra.run.smithi136.stdout:(19/156): python3-ceph-argparse-19.1.1-298.gb53 672 kB/s | 45 kB 00:00 2024-09-11T23:06:36.313 INFO:teuthology.orchestra.run.smithi136.stdout:(20/156): python3-ceph-common-19.1.1-298.gb53b6 1.3 MB/s | 136 kB 00:00 2024-09-11T23:06:36.414 INFO:teuthology.orchestra.run.smithi136.stdout:(21/156): python3-cephfs-19.1.1-298.gb53b6d3e.e 1.6 MB/s | 162 kB 00:00 2024-09-11T23:06:36.448 INFO:teuthology.orchestra.run.smithi076.stdout:(4/156): ceph-mds-19.1.1-298.gb53b6d3e.el9.x86_ 6.1 MB/s | 2.4 MB 00:00 2024-09-11T23:06:36.489 INFO:teuthology.orchestra.run.smithi136.stdout:(22/156): librgw2-19.1.1-298.gb53b6d3e.el9.x86_ 15 MB/s | 5.3 MB 00:00 2024-09-11T23:06:36.514 INFO:teuthology.orchestra.run.smithi136.stdout:(23/156): python3-rados-19.1.1-298.gb53b6d3e.el 3.1 MB/s | 321 kB 00:00 2024-09-11T23:06:36.548 INFO:teuthology.orchestra.run.smithi136.stdout:(24/156): python3-rbd-19.1.1-298.gb53b6d3e.el9. 5.0 MB/s | 301 kB 00:00 2024-09-11T23:06:36.573 INFO:teuthology.orchestra.run.smithi136.stdout:(25/156): python3-rgw-19.1.1-298.gb53b6d3e.el9. 1.7 MB/s | 100 kB 00:00 2024-09-11T23:06:36.615 INFO:teuthology.orchestra.run.smithi136.stdout:(26/156): rbd-fuse-19.1.1-298.gb53b6d3e.el9.x86 1.3 MB/s | 85 kB 00:00 2024-09-11T23:06:36.707 INFO:teuthology.orchestra.run.smithi076.stdout:(5/156): ceph-mgr-19.1.1-298.gb53b6d3e.el9.x86_ 5.9 MB/s | 1.5 MB 00:00 2024-09-11T23:06:36.707 INFO:teuthology.orchestra.run.smithi136.stdout:(27/156): rbd-nbd-19.1.1-298.gb53b6d3e.el9.x86_ 1.8 MB/s | 171 kB 00:00 2024-09-11T23:06:36.841 INFO:teuthology.orchestra.run.smithi136.stdout:(28/156): rbd-mirror-19.1.1-298.gb53b6d3e.el9.x 12 MB/s | 3.1 MB 00:00 2024-09-11T23:06:36.866 INFO:teuthology.orchestra.run.smithi136.stdout:(29/156): ceph-grafana-dashboards-19.1.1-298.gb 201 kB/s | 32 kB 00:00 2024-09-11T23:06:36.916 INFO:teuthology.orchestra.run.smithi136.stdout:(30/156): ceph-mgr-cephadm-19.1.1-298.gb53b6d3e 1.9 MB/s | 146 kB 00:00 2024-09-11T23:06:37.061 INFO:teuthology.orchestra.run.smithi026.stdout:lab-extras 38 kB/s | 1.7 kB 00:00 2024-09-11T23:06:37.158 INFO:teuthology.orchestra.run.smithi136.stdout:(31/156): ceph-mgr-dashboard-19.1.1-298.gb53b6d 12 MB/s | 3.6 MB 00:00 2024-09-11T23:06:37.217 INFO:teuthology.orchestra.run.smithi136.stdout:(32/156): ceph-mgr-modules-core-19.1.1-298.gb53 4.2 MB/s | 249 kB 00:00 2024-09-11T23:06:37.223 INFO:teuthology.orchestra.run.smithi076.stdout:(6/156): ceph-base-19.1.1-298.gb53b6d3e.el9.x86 3.3 MB/s | 5.4 MB 00:01 2024-09-11T23:06:37.284 INFO:teuthology.orchestra.run.smithi136.stdout:(33/156): ceph-mgr-rook-19.1.1-298.gb53b6d3e.el 739 kB/s | 49 kB 00:00 2024-09-11T23:06:37.359 INFO:teuthology.orchestra.run.smithi136.stdout:(34/156): ceph-prometheus-alerts-19.1.1-298.gb5 219 kB/s | 16 kB 00:00 2024-09-11T23:06:37.451 INFO:teuthology.orchestra.run.smithi136.stdout:(35/156): ceph-volume-19.1.1-298.gb53b6d3e.el9. 3.0 MB/s | 285 kB 00:00 2024-09-11T23:06:37.524 INFO:teuthology.orchestra.run.smithi076.stdout:(7/156): ceph-mon-19.1.1-298.gb53b6d3e.el9.x86_ 5.8 MB/s | 4.7 MB 00:00 2024-09-11T23:06:37.577 INFO:teuthology.orchestra.run.smithi136.stdout:(36/156): cephadm-19.1.1-298.gb53b6d3e.el9.noar 6.0 MB/s | 765 kB 00:00 2024-09-11T23:06:37.635 INFO:teuthology.orchestra.run.smithi136.stdout:(37/156): cephfs-top-19.1.1-298.gb53b6d3e.el9.n 313 kB/s | 18 kB 00:00 2024-09-11T23:06:37.766 INFO:teuthology.orchestra.run.smithi076.stdout:(8/156): ceph-common-19.1.1-298.gb53b6d3e.el9.x 9.7 MB/s | 21 MB 00:02 2024-09-11T23:06:37.807 INFO:teuthology.orchestra.run.smithi076.stdout:(9/156): ceph-selinux-19.1.1-298.gb53b6d3e.el9. 611 kB/s | 25 kB 00:00 2024-09-11T23:06:38.161 INFO:teuthology.orchestra.run.smithi136.stdout:(38/156): ceph-test-19.1.1-298.gb53b6d3e.el9.x8 19 MB/s | 49 MB 00:02 2024-09-11T23:06:38.186 INFO:teuthology.orchestra.run.smithi136.stdout:(39/156): fuse-2.9.9-16.el9.x86_64.rpm 145 kB/s | 80 kB 00:00 2024-09-11T23:06:38.278 INFO:teuthology.orchestra.run.smithi136.stdout:(40/156): ceph-mgr-diskprediction-local-19.1.1- 5.4 MB/s | 7.4 MB 00:01 2024-09-11T23:06:38.303 INFO:teuthology.orchestra.run.smithi136.stdout:(41/156): libquadmath-11.5.0-2.el9.x86_64.rpm 1.6 MB/s | 191 kB 00:00 2024-09-11T23:06:38.462 INFO:teuthology.orchestra.run.smithi136.stdout:(42/156): libgfortran-11.5.0-2.el9.x86_64.rpm 2.6 MB/s | 801 kB 00:00 2024-09-11T23:06:38.512 INFO:teuthology.orchestra.run.smithi136.stdout:(43/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.1 MB/s | 253 kB 00:00 2024-09-11T23:06:38.537 INFO:teuthology.orchestra.run.smithi136.stdout:(44/156): python3-ply-3.11-14.el9.noarch.rpm 1.4 MB/s | 106 kB 00:00 2024-09-11T23:06:38.579 INFO:teuthology.orchestra.run.smithi136.stdout:(45/156): python3-cryptography-36.0.1-4.el9.x86 4.4 MB/s | 1.2 MB 00:00 2024-09-11T23:06:38.604 INFO:teuthology.orchestra.run.smithi136.stdout:(46/156): python3-requests-2.25.1-8.el9.noarch. 1.8 MB/s | 125 kB 00:00 2024-09-11T23:06:38.630 INFO:teuthology.orchestra.run.smithi136.stdout:(47/156): python3-pycparser-2.20-6.el9.noarch.r 1.1 MB/s | 135 kB 00:00 2024-09-11T23:06:38.722 INFO:teuthology.orchestra.run.smithi136.stdout:(48/156): python3-urllib3-1.26.5-6.el9.noarch.r 1.5 MB/s | 215 kB 00:00 2024-09-11T23:06:38.741 INFO:teuthology.orchestra.run.smithi076.stdout:(10/156): ceph-radosgw-19.1.1-298.gb53b6d3e.el9 8.7 MB/s | 11 MB 00:01 2024-09-11T23:06:38.783 INFO:teuthology.orchestra.run.smithi076.stdout:(11/156): cephfs-mirror-19.1.1-298.gb53b6d3e.el 5.4 MB/s | 228 kB 00:00 2024-09-11T23:06:38.892 INFO:teuthology.orchestra.run.smithi076.stdout:(12/156): cephfs-shell-19.1.1-298.gb53b6d3e.el9 249 kB/s | 27 kB 00:00 2024-09-11T23:06:38.942 INFO:teuthology.orchestra.run.smithi076.stdout:(13/156): libcephfs-devel-19.1.1-298.gb53b6d3e. 637 kB/s | 32 kB 00:00 2024-09-11T23:06:39.389 INFO:teuthology.orchestra.run.smithi136.stdout:(49/156): boost-program-options-1.75.0-8.el9.x8 160 kB/s | 107 kB 00:00 2024-09-11T23:06:39.476 INFO:teuthology.orchestra.run.smithi076.stdout:(14/156): ceph-test-19.1.1-298.gb53b6d3e.el9.x8 29 MB/s | 49 MB 00:01 2024-09-11T23:06:39.510 INFO:teuthology.orchestra.run.smithi076.stdout:(15/156): libcephfs2-19.1.1-298.gb53b6d3e.el9.x 1.3 MB/s | 752 kB 00:00 2024-09-11T23:06:39.532 INFO:teuthology.orchestra.run.smithi026.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-11T23:06:39.533 INFO:teuthology.orchestra.run.smithi026.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-11T23:06:39.535 INFO:teuthology.orchestra.run.smithi076.stdout:(16/156): libcephsqlite-19.1.1-298.gb53b6d3e.el 2.7 MB/s | 163 kB 00:00 2024-09-11T23:06:39.546 INFO:teuthology.orchestra.run.smithi026.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-11T23:06:39.547 INFO:teuthology.orchestra.run.smithi026.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-11T23:06:39.548 INFO:teuthology.orchestra.run.smithi026.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-11T23:06:39.549 INFO:teuthology.orchestra.run.smithi026.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-11T23:06:39.551 INFO:teuthology.orchestra.run.smithi026.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-11T23:06:39.554 INFO:teuthology.orchestra.run.smithi026.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-11T23:06:39.555 INFO:teuthology.orchestra.run.smithi026.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-11T23:06:39.577 INFO:teuthology.orchestra.run.smithi076.stdout:(17/156): librados-devel-19.1.1-298.gb53b6d3e.e 1.9 MB/s | 127 kB 00:00 2024-09-11T23:06:39.610 INFO:teuthology.orchestra.run.smithi076.stdout:(18/156): libradosstriper1-19.1.1-298.gb53b6d3e 6.6 MB/s | 506 kB 00:00 2024-09-11T23:06:39.622 INFO:teuthology.orchestra.run.smithi026.stdout:Dependencies resolved. 2024-09-11T23:06:39.636 INFO:teuthology.orchestra.run.smithi136.stdout:(50/156): automake-1.16.2-8.el9.noarch.rpm 668 kB/s | 689 kB 00:01 2024-09-11T23:06:39.642 INFO:teuthology.orchestra.run.smithi026.stdout:====================================================================================== 2024-09-11T23:06:39.642 INFO:teuthology.orchestra.run.smithi026.stdout: Package Arch Version Repository Size 2024-09-11T23:06:39.643 INFO:teuthology.orchestra.run.smithi026.stdout:====================================================================================== 2024-09-11T23:06:39.643 INFO:teuthology.orchestra.run.smithi026.stdout:Installing: 2024-09-11T23:06:39.643 INFO:teuthology.orchestra.run.smithi026.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-11T23:06:39.643 INFO:teuthology.orchestra.run.smithi026.stdout: ceph x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 6.5 k 2024-09-11T23:06:39.643 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-base x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 5.4 M 2024-09-11T23:06:39.643 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-fuse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 897 k 2024-09-11T23:06:39.643 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 145 k 2024-09-11T23:06:39.643 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 1.5 M 2024-09-11T23:06:39.643 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-cephadm noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 146 k 2024-09-11T23:06:39.643 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-dashboard noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 3.6 M 2024-09-11T23:06:39.643 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 7.4 M 2024-09-11T23:06:39.643 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-rook noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 49 k 2024-09-11T23:06:39.643 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-radosgw x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 11 M 2024-09-11T23:06:39.644 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-test x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 49 M 2024-09-11T23:06:39.644 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-volume noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 285 k 2024-09-11T23:06:39.644 INFO:teuthology.orchestra.run.smithi026.stdout: cephadm noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 765 k 2024-09-11T23:06:39.644 INFO:teuthology.orchestra.run.smithi026.stdout: cephfs-mirror x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 228 k 2024-09-11T23:06:39.644 INFO:teuthology.orchestra.run.smithi026.stdout: cephfs-shell x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 27 k 2024-09-11T23:06:39.644 INFO:teuthology.orchestra.run.smithi026.stdout: cephfs-top noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 18 k 2024-09-11T23:06:39.644 INFO:teuthology.orchestra.run.smithi026.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-11T23:06:39.644 INFO:teuthology.orchestra.run.smithi026.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-11T23:06:39.644 INFO:teuthology.orchestra.run.smithi026.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-11T23:06:39.644 INFO:teuthology.orchestra.run.smithi026.stdout: libcephfs-devel x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 32 k 2024-09-11T23:06:39.644 INFO:teuthology.orchestra.run.smithi026.stdout: libcephfs2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 752 k 2024-09-11T23:06:39.644 INFO:teuthology.orchestra.run.smithi026.stdout: librados-devel x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 127 k 2024-09-11T23:06:39.644 INFO:teuthology.orchestra.run.smithi026.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-11T23:06:39.645 INFO:teuthology.orchestra.run.smithi026.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-11T23:06:39.645 INFO:teuthology.orchestra.run.smithi026.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-11T23:06:39.645 INFO:teuthology.orchestra.run.smithi026.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-11T23:06:39.645 INFO:teuthology.orchestra.run.smithi026.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-11T23:06:39.645 INFO:teuthology.orchestra.run.smithi026.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-11T23:06:39.645 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cephfs x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 162 k 2024-09-11T23:06:39.645 INFO:teuthology.orchestra.run.smithi026.stdout: python3-rados x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 321 k 2024-09-11T23:06:39.645 INFO:teuthology.orchestra.run.smithi026.stdout: python3-rbd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 301 k 2024-09-11T23:06:39.645 INFO:teuthology.orchestra.run.smithi026.stdout: python3-rgw x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 100 k 2024-09-11T23:06:39.645 INFO:teuthology.orchestra.run.smithi026.stdout: rbd-fuse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 85 k 2024-09-11T23:06:39.645 INFO:teuthology.orchestra.run.smithi026.stdout: rbd-mirror x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.1 M 2024-09-11T23:06:39.645 INFO:teuthology.orchestra.run.smithi026.stdout: rbd-nbd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 171 k 2024-09-11T23:06:39.646 INFO:teuthology.orchestra.run.smithi026.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-11T23:06:39.646 INFO:teuthology.orchestra.run.smithi026.stdout:Upgrading: 2024-09-11T23:06:39.646 INFO:teuthology.orchestra.run.smithi026.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-11T23:06:39.646 INFO:teuthology.orchestra.run.smithi026.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-11T23:06:39.646 INFO:teuthology.orchestra.run.smithi026.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-11T23:06:39.646 INFO:teuthology.orchestra.run.smithi026.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-11T23:06:39.646 INFO:teuthology.orchestra.run.smithi026.stdout: librados2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.4 M 2024-09-11T23:06:39.646 INFO:teuthology.orchestra.run.smithi026.stdout: librbd1 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.2 M 2024-09-11T23:06:39.646 INFO:teuthology.orchestra.run.smithi026.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-11T23:06:39.646 INFO:teuthology.orchestra.run.smithi026.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-11T23:06:39.646 INFO:teuthology.orchestra.run.smithi026.stdout:Installing dependencies: 2024-09-11T23:06:39.646 INFO:teuthology.orchestra.run.smithi026.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-11T23:06:39.646 INFO:teuthology.orchestra.run.smithi026.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-11T23:06:39.646 INFO:teuthology.orchestra.run.smithi026.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-11T23:06:39.647 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-common x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 21 M 2024-09-11T23:06:39.647 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-grafana-dashboards noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 32 k 2024-09-11T23:06:39.647 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mds x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 2.4 M 2024-09-11T23:06:39.647 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-modules-core noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 249 k 2024-09-11T23:06:39.647 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mon x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 4.7 M 2024-09-11T23:06:39.647 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-osd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 17 M 2024-09-11T23:06:39.647 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-prometheus-alerts noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 16 k 2024-09-11T23:06:39.647 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-selinux x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 25 k 2024-09-11T23:06:39.647 INFO:teuthology.orchestra.run.smithi026.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-11T23:06:39.647 INFO:teuthology.orchestra.run.smithi026.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-11T23:06:39.647 INFO:teuthology.orchestra.run.smithi026.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-11T23:06:39.647 INFO:teuthology.orchestra.run.smithi026.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-11T23:06:39.647 INFO:teuthology.orchestra.run.smithi026.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-11T23:06:39.648 INFO:teuthology.orchestra.run.smithi026.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-11T23:06:39.648 INFO:teuthology.orchestra.run.smithi026.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-11T23:06:39.648 INFO:teuthology.orchestra.run.smithi026.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-11T23:06:39.648 INFO:teuthology.orchestra.run.smithi026.stdout: libcephsqlite x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 163 k 2024-09-11T23:06:39.648 INFO:teuthology.orchestra.run.smithi026.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-11T23:06:39.648 INFO:teuthology.orchestra.run.smithi026.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-11T23:06:39.648 INFO:teuthology.orchestra.run.smithi026.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-11T23:06:39.648 INFO:teuthology.orchestra.run.smithi026.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-11T23:06:39.648 INFO:teuthology.orchestra.run.smithi026.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-11T23:06:39.648 INFO:teuthology.orchestra.run.smithi026.stdout: libradosstriper1 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 506 k 2024-09-11T23:06:39.648 INFO:teuthology.orchestra.run.smithi026.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-11T23:06:39.648 INFO:teuthology.orchestra.run.smithi026.stdout: librgw2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 5.3 M 2024-09-11T23:06:39.648 INFO:teuthology.orchestra.run.smithi026.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-11T23:06:39.649 INFO:teuthology.orchestra.run.smithi026.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-11T23:06:39.649 INFO:teuthology.orchestra.run.smithi026.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-11T23:06:39.649 INFO:teuthology.orchestra.run.smithi026.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-11T23:06:39.649 INFO:teuthology.orchestra.run.smithi026.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-11T23:06:39.649 INFO:teuthology.orchestra.run.smithi026.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-11T23:06:39.649 INFO:teuthology.orchestra.run.smithi026.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-11T23:06:39.649 INFO:teuthology.orchestra.run.smithi026.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-11T23:06:39.649 INFO:teuthology.orchestra.run.smithi026.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-11T23:06:39.649 INFO:teuthology.orchestra.run.smithi026.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-11T23:06:39.649 INFO:teuthology.orchestra.run.smithi026.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-11T23:06:39.649 INFO:teuthology.orchestra.run.smithi026.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-11T23:06:39.649 INFO:teuthology.orchestra.run.smithi026.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-11T23:06:39.650 INFO:teuthology.orchestra.run.smithi026.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-11T23:06:39.650 INFO:teuthology.orchestra.run.smithi026.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-11T23:06:39.650 INFO:teuthology.orchestra.run.smithi026.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-11T23:06:39.650 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-11T23:06:39.650 INFO:teuthology.orchestra.run.smithi026.stdout: python3-ceph-argparse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 45 k 2024-09-11T23:06:39.650 INFO:teuthology.orchestra.run.smithi026.stdout: python3-ceph-common x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 136 k 2024-09-11T23:06:39.650 INFO:teuthology.orchestra.run.smithi026.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-11T23:06:39.650 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-11T23:06:39.650 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-11T23:06:39.650 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-11T23:06:39.650 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-11T23:06:39.650 INFO:teuthology.orchestra.run.smithi026.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-11T23:06:39.650 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-11T23:06:39.650 INFO:teuthology.orchestra.run.smithi026.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-11T23:06:39.651 INFO:teuthology.orchestra.run.smithi026.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-11T23:06:39.651 INFO:teuthology.orchestra.run.smithi026.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-11T23:06:39.651 INFO:teuthology.orchestra.run.smithi026.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-11T23:06:39.651 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-11T23:06:39.651 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-11T23:06:39.651 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-11T23:06:39.651 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-11T23:06:39.651 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-11T23:06:39.651 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-11T23:06:39.651 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-11T23:06:39.651 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-11T23:06:39.651 INFO:teuthology.orchestra.run.smithi026.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-11T23:06:39.651 INFO:teuthology.orchestra.run.smithi026.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-11T23:06:39.652 INFO:teuthology.orchestra.run.smithi026.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-11T23:06:39.652 INFO:teuthology.orchestra.run.smithi026.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-11T23:06:39.652 INFO:teuthology.orchestra.run.smithi026.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-11T23:06:39.652 INFO:teuthology.orchestra.run.smithi026.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-11T23:06:39.652 INFO:teuthology.orchestra.run.smithi026.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-11T23:06:39.652 INFO:teuthology.orchestra.run.smithi026.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-11T23:06:39.652 INFO:teuthology.orchestra.run.smithi026.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-11T23:06:39.652 INFO:teuthology.orchestra.run.smithi026.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-11T23:06:39.652 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-11T23:06:39.652 INFO:teuthology.orchestra.run.smithi026.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-11T23:06:39.652 INFO:teuthology.orchestra.run.smithi026.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-11T23:06:39.652 INFO:teuthology.orchestra.run.smithi026.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-11T23:06:39.652 INFO:teuthology.orchestra.run.smithi026.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-11T23:06:39.653 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-11T23:06:39.653 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-11T23:06:39.653 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-11T23:06:39.653 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-11T23:06:39.653 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-11T23:06:39.653 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-11T23:06:39.653 INFO:teuthology.orchestra.run.smithi026.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-11T23:06:39.653 INFO:teuthology.orchestra.run.smithi026.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-11T23:06:39.653 INFO:teuthology.orchestra.run.smithi026.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-11T23:06:39.653 INFO:teuthology.orchestra.run.smithi026.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-11T23:06:39.653 INFO:teuthology.orchestra.run.smithi026.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-11T23:06:39.653 INFO:teuthology.orchestra.run.smithi026.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-11T23:06:39.653 INFO:teuthology.orchestra.run.smithi026.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-11T23:06:39.653 INFO:teuthology.orchestra.run.smithi026.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-11T23:06:39.654 INFO:teuthology.orchestra.run.smithi026.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-11T23:06:39.654 INFO:teuthology.orchestra.run.smithi026.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-11T23:06:39.654 INFO:teuthology.orchestra.run.smithi026.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-11T23:06:39.654 INFO:teuthology.orchestra.run.smithi026.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-11T23:06:39.654 INFO:teuthology.orchestra.run.smithi026.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-11T23:06:39.654 INFO:teuthology.orchestra.run.smithi026.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-11T23:06:39.654 INFO:teuthology.orchestra.run.smithi026.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-11T23:06:39.654 INFO:teuthology.orchestra.run.smithi026.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-11T23:06:39.654 INFO:teuthology.orchestra.run.smithi026.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-11T23:06:39.654 INFO:teuthology.orchestra.run.smithi026.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-11T23:06:39.654 INFO:teuthology.orchestra.run.smithi026.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-11T23:06:39.654 INFO:teuthology.orchestra.run.smithi026.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-11T23:06:39.655 INFO:teuthology.orchestra.run.smithi026.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-11T23:06:39.655 INFO:teuthology.orchestra.run.smithi026.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-11T23:06:39.655 INFO:teuthology.orchestra.run.smithi026.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-11T23:06:39.655 INFO:teuthology.orchestra.run.smithi026.stdout:Installing weak dependencies: 2024-09-11T23:06:39.655 INFO:teuthology.orchestra.run.smithi026.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-11T23:06:39.655 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-11T23:06:39.655 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction Summary 2024-09-11T23:06:39.655 INFO:teuthology.orchestra.run.smithi026.stdout:====================================================================================== 2024-09-11T23:06:39.655 INFO:teuthology.orchestra.run.smithi026.stdout:Install 148 Packages 2024-09-11T23:06:39.655 INFO:teuthology.orchestra.run.smithi026.stdout:Upgrade 8 Packages 2024-09-11T23:06:39.655 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-11T23:06:39.656 INFO:teuthology.orchestra.run.smithi026.stdout:Total download size: 226 M 2024-09-11T23:06:39.656 INFO:teuthology.orchestra.run.smithi026.stdout:Downloading Packages: 2024-09-11T23:06:39.660 INFO:teuthology.orchestra.run.smithi076.stdout:(19/156): python3-ceph-argparse-19.1.1-298.gb53 902 kB/s | 45 kB 00:00 2024-09-11T23:06:39.685 INFO:teuthology.orchestra.run.smithi136.stdout:(51/156): bison-3.7.4-5.el9.x86_64.rpm 894 kB/s | 943 kB 00:01 2024-09-11T23:06:39.711 INFO:teuthology.orchestra.run.smithi076.stdout:(20/156): python3-ceph-common-19.1.1-298.gb53b6 2.7 MB/s | 136 kB 00:00 2024-09-11T23:06:39.753 INFO:teuthology.orchestra.run.smithi076.stdout:(21/156): python3-cephfs-19.1.1-298.gb53b6d3e.e 3.8 MB/s | 162 kB 00:00 2024-09-11T23:06:39.811 INFO:teuthology.orchestra.run.smithi076.stdout:(22/156): python3-rados-19.1.1-298.gb53b6d3e.el 5.4 MB/s | 321 kB 00:00 2024-09-11T23:06:39.861 INFO:teuthology.orchestra.run.smithi076.stdout:(23/156): python3-rbd-19.1.1-298.gb53b6d3e.el9. 5.9 MB/s | 301 kB 00:00 2024-09-11T23:06:39.912 INFO:teuthology.orchestra.run.smithi076.stdout:(24/156): python3-rgw-19.1.1-298.gb53b6d3e.el9. 1.9 MB/s | 100 kB 00:00 2024-09-11T23:06:39.969 INFO:teuthology.orchestra.run.smithi136.stdout:(52/156): elfutils-libelf-devel-0.191-4.el9.x86 37 kB/s | 22 kB 00:00 2024-09-11T23:06:39.979 INFO:teuthology.orchestra.run.smithi076.stdout:(25/156): librgw2-19.1.1-298.gb53b6d3e.el9.x86_ 13 MB/s | 5.3 MB 00:00 2024-09-11T23:06:40.004 INFO:teuthology.orchestra.run.smithi076.stdout:(26/156): rbd-fuse-19.1.1-298.gb53b6d3e.el9.x86 928 kB/s | 85 kB 00:00 2024-09-11T23:06:40.163 INFO:teuthology.orchestra.run.smithi076.stdout:(27/156): ceph-osd-19.1.1-298.gb53b6d3e.el9.x86 5.7 MB/s | 17 MB 00:02 2024-09-11T23:06:40.165 INFO:teuthology.orchestra.run.smithi026.stdout:(1/156): ceph-19.1.1-298.gb53b6d3e.el9.x86_64.r 50 kB/s | 6.5 kB 00:00 2024-09-11T23:06:40.188 INFO:teuthology.orchestra.run.smithi076.stdout:(28/156): rbd-nbd-19.1.1-298.gb53b6d3e.el9.x86_ 931 kB/s | 171 kB 00:00 2024-09-11T23:06:40.205 INFO:teuthology.orchestra.run.smithi076.stdout:(29/156): ceph-grafana-dashboards-19.1.1-298.gb 769 kB/s | 32 kB 00:00 2024-09-11T23:06:40.239 INFO:teuthology.orchestra.run.smithi076.stdout:(30/156): ceph-mgr-cephadm-19.1.1-298.gb53b6d3e 2.9 MB/s | 146 kB 00:00 2024-09-11T23:06:40.294 INFO:teuthology.orchestra.run.smithi136.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 54 kB/s | 33 kB 00:00 2024-09-11T23:06:40.297 INFO:teuthology.orchestra.run.smithi076.stdout:(31/156): rbd-mirror-19.1.1-298.gb53b6d3e.el9.x 9.8 MB/s | 3.1 MB 00:00 2024-09-11T23:06:40.373 INFO:teuthology.orchestra.run.smithi076.stdout:(32/156): ceph-mgr-modules-core-19.1.1-298.gb53 3.3 MB/s | 249 kB 00:00 2024-09-11T23:06:40.456 INFO:teuthology.orchestra.run.smithi076.stdout:(33/156): ceph-mgr-rook-19.1.1-298.gb53b6d3e.el 594 kB/s | 49 kB 00:00 2024-09-11T23:06:40.490 INFO:teuthology.orchestra.run.smithi026.stdout:(2/156): ceph-fuse-19.1.1-298.gb53b6d3e.el9.x86 2.7 MB/s | 897 kB 00:00 2024-09-11T23:06:40.519 INFO:teuthology.orchestra.run.smithi136.stdout:(54/156): flex-2.6.4-9.el9.x86_64.rpm 358 kB/s | 313 kB 00:00 2024-09-11T23:06:40.548 INFO:teuthology.orchestra.run.smithi076.stdout:(34/156): ceph-mgr-diskprediction-local-19.1.1- 24 MB/s | 7.4 MB 00:00 2024-09-11T23:06:40.549 INFO:teuthology.orchestra.run.smithi026.stdout:(3/156): ceph-immutable-object-cache-19.1.1-298 2.4 MB/s | 145 kB 00:00 2024-09-11T23:06:40.565 INFO:teuthology.orchestra.run.smithi076.stdout:(35/156): ceph-prometheus-alerts-19.1.1-298.gb5 151 kB/s | 16 kB 00:00 2024-09-11T23:06:40.607 INFO:teuthology.orchestra.run.smithi076.stdout:(36/156): ceph-volume-19.1.1-298.gb53b6d3e.el9. 4.8 MB/s | 285 kB 00:00 2024-09-11T23:06:40.666 INFO:teuthology.orchestra.run.smithi076.stdout:(37/156): cephadm-19.1.1-298.gb53b6d3e.el9.noar 7.5 MB/s | 765 kB 00:00 2024-09-11T23:06:40.682 INFO:teuthology.orchestra.run.smithi076.stdout:(38/156): cephfs-top-19.1.1-298.gb53b6d3e.el9.n 243 kB/s | 18 kB 00:00 2024-09-11T23:06:40.741 INFO:teuthology.orchestra.run.smithi076.stdout:(39/156): ceph-mgr-dashboard-19.1.1-298.gb53b6d 6.6 MB/s | 3.6 MB 00:00 2024-09-11T23:06:40.794 INFO:teuthology.orchestra.run.smithi136.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 36 kB/s | 18 kB 00:00 2024-09-11T23:06:40.824 INFO:teuthology.orchestra.run.smithi026.stdout:(4/156): ceph-base-19.1.1-298.gb53b6d3e.el9.x86 6.8 MB/s | 5.4 MB 00:00 2024-09-11T23:06:40.833 INFO:teuthology.orchestra.run.smithi076.stdout:(40/156): fuse-2.9.9-16.el9.x86_64.rpm 478 kB/s | 80 kB 00:00 2024-09-11T23:06:40.874 INFO:teuthology.orchestra.run.smithi026.stdout:(5/156): ceph-mds-19.1.1-298.gb53b6d3e.el9.x86_ 7.3 MB/s | 2.4 MB 00:00 2024-09-11T23:06:40.900 INFO:teuthology.orchestra.run.smithi076.stdout:(41/156): libquadmath-11.5.0-2.el9.x86_64.rpm 1.2 MB/s | 191 kB 00:00 2024-09-11T23:06:40.926 INFO:teuthology.orchestra.run.smithi076.stdout:(42/156): libgfortran-11.5.0-2.el9.x86_64.rpm 3.2 MB/s | 801 kB 00:00 2024-09-11T23:06:40.951 INFO:teuthology.orchestra.run.smithi076.stdout:(43/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.1 MB/s | 253 kB 00:00 2024-09-11T23:06:40.976 INFO:teuthology.orchestra.run.smithi076.stdout:(44/156): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-09-11T23:06:40.983 INFO:teuthology.orchestra.run.smithi026.stdout:(6/156): ceph-mgr-19.1.1-298.gb53b6d3e.el9.x86_ 9.6 MB/s | 1.5 MB 00:00 2024-09-11T23:06:41.001 INFO:teuthology.orchestra.run.smithi076.stdout:(45/156): python3-pycparser-2.20-6.el9.noarch.r 2.6 MB/s | 135 kB 00:00 2024-09-11T23:06:41.028 INFO:teuthology.orchestra.run.smithi136.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 22 kB/s | 11 kB 00:00 2024-09-11T23:06:41.035 INFO:teuthology.orchestra.run.smithi076.stdout:(46/156): python3-requests-2.25.1-8.el9.noarch. 2.1 MB/s | 125 kB 00:00 2024-09-11T23:06:41.060 INFO:teuthology.orchestra.run.smithi076.stdout:(47/156): python3-urllib3-1.26.5-6.el9.noarch.r 3.6 MB/s | 215 kB 00:00 2024-09-11T23:06:41.094 INFO:teuthology.orchestra.run.smithi076.stdout:(48/156): python3-cryptography-36.0.1-4.el9.x86 6.3 MB/s | 1.2 MB 00:00 2024-09-11T23:06:41.162 INFO:teuthology.orchestra.run.smithi136.stdout:(57/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 2.5 MB/s | 3.0 MB 00:01 2024-09-11T23:06:41.227 INFO:teuthology.orchestra.run.smithi076.stdout:(49/156): boost-program-options-1.75.0-8.el9.x8 803 kB/s | 107 kB 00:00 2024-09-11T23:06:41.278 INFO:teuthology.orchestra.run.smithi076.stdout:(50/156): elfutils-libelf-devel-0.191-4.el9.x86 432 kB/s | 22 kB 00:00 2024-09-11T23:06:41.333 INFO:teuthology.orchestra.run.smithi026.stdout:(7/156): ceph-mon-19.1.1-298.gb53b6d3e.el9.x86_ 10 MB/s | 4.7 MB 00:00 2024-09-11T23:06:41.394 INFO:teuthology.orchestra.run.smithi076.stdout:(51/156): automake-1.16.2-8.el9.noarch.rpm 1.9 MB/s | 689 kB 00:00 2024-09-11T23:06:41.420 INFO:teuthology.orchestra.run.smithi076.stdout:(52/156): flex-2.6.4-9.el9.x86_64.rpm 2.2 MB/s | 313 kB 00:00 2024-09-11T23:06:41.445 INFO:teuthology.orchestra.run.smithi076.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 650 kB/s | 33 kB 00:00 2024-09-11T23:06:41.472 INFO:teuthology.orchestra.run.smithi076.stdout:(54/156): bison-3.7.4-5.el9.x86_64.rpm 2.2 MB/s | 943 kB 00:00 2024-09-11T23:06:41.489 INFO:teuthology.orchestra.run.smithi076.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 414 kB/s | 18 kB 00:00 2024-09-11T23:06:41.515 INFO:teuthology.orchestra.run.smithi076.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 267 kB/s | 11 kB 00:00 2024-09-11T23:06:41.565 INFO:teuthology.orchestra.run.smithi076.stdout:(57/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 905 kB/s | 45 kB 00:00 2024-09-11T23:06:41.590 INFO:teuthology.orchestra.run.smithi076.stdout:(58/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.6 MB/s | 160 kB 00:00 2024-09-11T23:06:41.629 INFO:teuthology.orchestra.run.smithi136.stdout:(58/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 192 kB/s | 160 kB 00:00 2024-09-11T23:06:41.662 INFO:teuthology.orchestra.run.smithi136.stdout:(59/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 71 kB/s | 45 kB 00:00 2024-09-11T23:06:41.765 INFO:teuthology.orchestra.run.smithi076.stdout:(59/156): libtool-2.4.6-46.el9.x86_64.rpm 3.2 MB/s | 581 kB 00:00 2024-09-11T23:06:41.767 INFO:teuthology.orchestra.run.smithi026.stdout:(8/156): ceph-common-19.1.1-298.gb53b6d3e.el9.x 12 MB/s | 21 MB 00:01 2024-09-11T23:06:41.791 INFO:teuthology.orchestra.run.smithi076.stdout:(60/156): librdkafka-1.6.1-102.el9.x86_64.rpm 2.9 MB/s | 662 kB 00:00 2024-09-11T23:06:41.808 INFO:teuthology.orchestra.run.smithi076.stdout:(61/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 582 kB/s | 24 kB 00:00 2024-09-11T23:06:41.809 INFO:teuthology.orchestra.run.smithi026.stdout:(9/156): ceph-selinux-19.1.1-298.gb53b6d3e.el9. 610 kB/s | 25 kB 00:00 2024-09-11T23:06:41.833 INFO:teuthology.orchestra.run.smithi076.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 1.1 MB/s | 47 kB 00:00 2024-09-11T23:06:41.917 INFO:teuthology.orchestra.run.smithi076.stdout:(63/156): lua-5.4.4-4.el9.x86_64.rpm 2.2 MB/s | 188 kB 00:00 2024-09-11T23:06:41.942 INFO:teuthology.orchestra.run.smithi076.stdout:(64/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.1 MB/s | 292 kB 00:00 2024-09-11T23:06:41.958 INFO:teuthology.orchestra.run.smithi076.stdout:(65/156): openblas-0.3.26-2.el9.x86_64.rpm 920 kB/s | 38 kB 00:00 2024-09-11T23:06:42.046 INFO:teuthology.orchestra.run.smithi136.stdout:(60/156): librdkafka-1.6.1-102.el9.x86_64.rpm 750 kB/s | 662 kB 00:00 2024-09-11T23:06:42.059 INFO:teuthology.orchestra.run.smithi026.stdout:(10/156): ceph-osd-19.1.1-298.gb53b6d3e.el9.x86 16 MB/s | 17 MB 00:01 2024-09-11T23:06:42.193 INFO:teuthology.orchestra.run.smithi026.stdout:(11/156): ceph-radosgw-19.1.1-298.gb53b6d3e.el9 12 MB/s | 11 MB 00:00 2024-09-11T23:06:42.212 INFO:teuthology.orchestra.run.smithi136.stdout:(61/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 44 kB/s | 24 kB 00:00 2024-09-11T23:06:42.217 INFO:teuthology.orchestra.run.smithi076.stdout:(66/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 3.8 MB/s | 3.0 MB 00:00 2024-09-11T23:06:42.243 INFO:teuthology.orchestra.run.smithi026.stdout:(12/156): cephfs-mirror-19.1.1-298.gb53b6d3e.el 1.2 MB/s | 228 kB 00:00 2024-09-11T23:06:42.268 INFO:teuthology.orchestra.run.smithi026.stdout:(13/156): cephfs-shell-19.1.1-298.gb53b6d3e.el9 360 kB/s | 27 kB 00:00 2024-09-11T23:06:42.293 INFO:teuthology.orchestra.run.smithi026.stdout:(14/156): libcephfs-devel-19.1.1-298.gb53b6d3e. 634 kB/s | 32 kB 00:00 2024-09-11T23:06:42.344 INFO:teuthology.orchestra.run.smithi026.stdout:(15/156): libcephfs2-19.1.1-298.gb53b6d3e.el9.x 9.7 MB/s | 752 kB 00:00 2024-09-11T23:06:42.377 INFO:teuthology.orchestra.run.smithi026.stdout:(16/156): libcephsqlite-19.1.1-298.gb53b6d3e.el 1.9 MB/s | 163 kB 00:00 2024-09-11T23:06:42.402 INFO:teuthology.orchestra.run.smithi026.stdout:(17/156): librados-devel-19.1.1-298.gb53b6d3e.e 2.1 MB/s | 127 kB 00:00 2024-09-11T23:06:42.436 INFO:teuthology.orchestra.run.smithi026.stdout:(18/156): libradosstriper1-19.1.1-298.gb53b6d3e 8.5 MB/s | 506 kB 00:00 2024-09-11T23:06:42.486 INFO:teuthology.orchestra.run.smithi026.stdout:(19/156): python3-ceph-argparse-19.1.1-298.gb53 904 kB/s | 45 kB 00:00 2024-09-11T23:06:42.562 INFO:teuthology.orchestra.run.smithi026.stdout:(20/156): python3-ceph-common-19.1.1-298.gb53b6 1.8 MB/s | 136 kB 00:00 2024-09-11T23:06:42.567 INFO:teuthology.orchestra.run.smithi076.stdout:(67/156): postgresql-13.16-1.el9.x86_64.rpm 4.5 MB/s | 1.6 MB 00:00 2024-09-11T23:06:42.588 INFO:teuthology.orchestra.run.smithi136.stdout:(62/156): libtool-2.4.6-46.el9.x86_64.rpm 606 kB/s | 581 kB 00:00 2024-09-11T23:06:42.679 INFO:teuthology.orchestra.run.smithi026.stdout:(21/156): librgw2-19.1.1-298.gb53b6d3e.el9.x86_ 19 MB/s | 5.3 MB 00:00 2024-09-11T23:06:42.696 INFO:teuthology.orchestra.run.smithi136.stdout:(63/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 73 kB/s | 47 kB 00:00 2024-09-11T23:06:42.704 INFO:teuthology.orchestra.run.smithi026.stdout:(22/156): python3-cephfs-19.1.1-298.gb53b6d3e.e 1.1 MB/s | 162 kB 00:00 2024-09-11T23:06:42.738 INFO:teuthology.orchestra.run.smithi026.stdout:(23/156): python3-rados-19.1.1-298.gb53b6d3e.el 5.4 MB/s | 321 kB 00:00 2024-09-11T23:06:42.771 INFO:teuthology.orchestra.run.smithi026.stdout:(24/156): python3-rbd-19.1.1-298.gb53b6d3e.el9. 4.4 MB/s | 301 kB 00:00 2024-09-11T23:06:42.776 INFO:teuthology.orchestra.run.smithi076.stdout:(68/156): postgresql-contrib-13.16-1.el9.x86_64 4.3 MB/s | 910 kB 00:00 2024-09-11T23:06:42.797 INFO:teuthology.orchestra.run.smithi026.stdout:(25/156): python3-rgw-19.1.1-298.gb53b6d3e.el9. 1.7 MB/s | 100 kB 00:00 2024-09-11T23:06:42.822 INFO:teuthology.orchestra.run.smithi026.stdout:(26/156): rbd-fuse-19.1.1-298.gb53b6d3e.el9.x86 1.7 MB/s | 85 kB 00:00 2024-09-11T23:06:42.844 INFO:teuthology.orchestra.run.smithi076.stdout:(69/156): postgresql-private-libs-13.16-1.el9.x 2.0 MB/s | 137 kB 00:00 2024-09-11T23:06:42.881 INFO:teuthology.orchestra.run.smithi026.stdout:(27/156): rbd-nbd-19.1.1-298.gb53b6d3e.el9.x86_ 2.9 MB/s | 171 kB 00:00 2024-09-11T23:06:42.947 INFO:teuthology.orchestra.run.smithi026.stdout:(28/156): rbd-mirror-19.1.1-298.gb53b6d3e.el9.x 21 MB/s | 3.1 MB 00:00 2024-09-11T23:06:42.968 INFO:teuthology.orchestra.run.smithi076.stdout:(70/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 4.4 MB/s | 4.4 MB 00:01 2024-09-11T23:06:42.972 INFO:teuthology.orchestra.run.smithi026.stdout:(29/156): ceph-grafana-dashboards-19.1.1-298.gb 348 kB/s | 32 kB 00:00 2024-09-11T23:06:42.998 INFO:teuthology.orchestra.run.smithi026.stdout:(30/156): ceph-mgr-cephadm-19.1.1-298.gb53b6d3e 2.8 MB/s | 146 kB 00:00 2024-09-11T23:06:43.072 INFO:teuthology.orchestra.run.smithi136.stdout:(64/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 340 kB/s | 292 kB 00:00 2024-09-11T23:06:43.144 INFO:teuthology.orchestra.run.smithi076.stdout:(71/156): openblas-openmp-0.3.26-2.el9.x86_64.r 4.1 MB/s | 4.9 MB 00:01 2024-09-11T23:06:43.194 INFO:teuthology.orchestra.run.smithi076.stdout:(72/156): python3-attrs-20.3.0-7.el9.noarch.rpm 1.7 MB/s | 87 kB 00:00 2024-09-11T23:06:43.228 INFO:teuthology.orchestra.run.smithi076.stdout:(73/156): protobuf-3.14.0-13.el9.x86_64.rpm 3.9 MB/s | 1.0 MB 00:00 2024-09-11T23:06:43.274 INFO:teuthology.orchestra.run.smithi026.stdout:(31/156): ceph-mgr-dashboard-19.1.1-298.gb53b6d 12 MB/s | 3.6 MB 00:00 2024-09-11T23:06:43.280 INFO:teuthology.orchestra.run.smithi136.stdout:(65/156): openblas-0.3.26-2.el9.x86_64.rpm 66 kB/s | 38 kB 00:00 2024-09-11T23:06:43.286 INFO:teuthology.orchestra.run.smithi076.stdout:(74/156): python3-colorama-0.4.6-3.el9.noarch.r 1.0 MB/s | 61 kB 00:00 2024-09-11T23:06:43.322 INFO:teuthology.orchestra.run.smithi136.stdout:(66/156): lua-5.4.4-4.el9.x86_64.rpm 257 kB/s | 188 kB 00:00 2024-09-11T23:06:43.341 INFO:teuthology.orchestra.run.smithi026.stdout:(32/156): ceph-mgr-modules-core-19.1.1-298.gb53 3.7 MB/s | 249 kB 00:00 2024-09-11T23:06:43.370 INFO:teuthology.orchestra.run.smithi076.stdout:(75/156): python3-devel-3.9.19-8.el9.x86_64.rpm 2.9 MB/s | 245 kB 00:00 2024-09-11T23:06:43.432 INFO:teuthology.orchestra.run.smithi026.stdout:(33/156): ceph-mgr-diskprediction-local-19.1.1- 17 MB/s | 7.4 MB 00:00 2024-09-11T23:06:43.453 INFO:teuthology.orchestra.run.smithi076.stdout:(76/156): python3-jinja2-2.11.3-6.el9.noarch.rp 2.9 MB/s | 248 kB 00:00 2024-09-11T23:06:43.458 INFO:teuthology.orchestra.run.smithi026.stdout:(34/156): ceph-mgr-rook-19.1.1-298.gb53b6d3e.el 421 kB/s | 49 kB 00:00 2024-09-11T23:06:43.483 INFO:teuthology.orchestra.run.smithi026.stdout:(35/156): ceph-prometheus-alerts-19.1.1-298.gb5 327 kB/s | 16 kB 00:00 2024-09-11T23:06:43.503 INFO:teuthology.orchestra.run.smithi076.stdout:(77/156): python3-jsonpatch-1.21-16.el9.noarch. 512 kB/s | 26 kB 00:00 2024-09-11T23:06:43.516 INFO:teuthology.orchestra.run.smithi026.stdout:(36/156): ceph-volume-19.1.1-298.gb53b6d3e.el9. 4.8 MB/s | 285 kB 00:00 2024-09-11T23:06:43.550 INFO:teuthology.orchestra.run.smithi026.stdout:(37/156): cephadm-19.1.1-298.gb53b6d3e.el9.noar 11 MB/s | 765 kB 00:00 2024-09-11T23:06:43.554 INFO:teuthology.orchestra.run.smithi076.stdout:(78/156): python3-jsonpointer-2.0-4.el9.noarch. 383 kB/s | 19 kB 00:00 2024-09-11T23:06:43.576 INFO:teuthology.orchestra.run.smithi026.stdout:(38/156): cephfs-top-19.1.1-298.gb53b6d3e.el9.n 311 kB/s | 18 kB 00:00 2024-09-11T23:06:43.637 INFO:teuthology.orchestra.run.smithi076.stdout:(79/156): python3-mako-1.1.4-6.el9.noarch.rpm 2.0 MB/s | 172 kB 00:00 2024-09-11T23:06:43.696 INFO:teuthology.orchestra.run.smithi076.stdout:(80/156): python3-markupsafe-1.1.1-12.el9.x86_6 596 kB/s | 35 kB 00:00 2024-09-11T23:06:43.751 INFO:teuthology.orchestra.run.smithi026.stdout:(39/156): fuse-2.9.9-16.el9.x86_64.rpm 398 kB/s | 80 kB 00:00 2024-09-11T23:06:43.930 INFO:teuthology.orchestra.run.smithi076.stdout:(81/156): postgresql-server-13.16-1.el9.x86_64. 5.3 MB/s | 5.8 MB 00:01 2024-09-11T23:06:44.013 INFO:teuthology.orchestra.run.smithi076.stdout:(82/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.9 MB/s | 250 kB 00:00 2024-09-11T23:06:44.072 INFO:teuthology.orchestra.run.smithi076.stdout:(83/156): python3-oauthlib-3.1.1-5.el9.noarch.r 3.7 MB/s | 222 kB 00:00 2024-09-11T23:06:44.130 INFO:teuthology.orchestra.run.smithi076.stdout:(84/156): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 77 kB 00:00 2024-09-11T23:06:44.193 INFO:teuthology.orchestra.run.smithi026.stdout:(40/156): ceph-test-19.1.1-298.gb53b6d3e.el9.x8 20 MB/s | 49 MB 00:02 2024-09-11T23:06:44.205 INFO:teuthology.orchestra.run.smithi076.stdout:(85/156): python3-prettytable-0.7.2-27.el9.noar 564 kB/s | 42 kB 00:00 2024-09-11T23:06:44.218 INFO:teuthology.orchestra.run.smithi026.stdout:(41/156): libquadmath-11.5.0-2.el9.x86_64.rpm 409 kB/s | 191 kB 00:00 2024-09-11T23:06:44.252 INFO:teuthology.orchestra.run.smithi026.stdout:(42/156): libgfortran-11.5.0-2.el9.x86_64.rpm 1.2 MB/s | 801 kB 00:00 2024-09-11T23:06:44.289 INFO:teuthology.orchestra.run.smithi076.stdout:(86/156): python3-protobuf-3.14.0-13.el9.noarch 3.2 MB/s | 269 kB 00:00 2024-09-11T23:06:44.310 INFO:teuthology.orchestra.run.smithi026.stdout:(43/156): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 106 kB 00:00 2024-09-11T23:06:44.331 INFO:teuthology.orchestra.run.smithi136.stdout:(67/156): openblas-openmp-0.3.26-2.el9.x86_64.r 3.9 MB/s | 4.9 MB 00:01 2024-09-11T23:06:44.365 INFO:teuthology.orchestra.run.smithi136.stdout:(68/156): postgresql-13.16-1.el9.x86_64.rpm 1.5 MB/s | 1.6 MB 00:01 2024-09-11T23:06:44.377 INFO:teuthology.orchestra.run.smithi026.stdout:(44/156): python3-pycparser-2.20-6.el9.noarch.r 2.0 MB/s | 135 kB 00:00 2024-09-11T23:06:44.389 INFO:teuthology.orchestra.run.smithi076.stdout:(87/156): python3-babel-2.9.1-2.el9.noarch.rpm 5.0 MB/s | 6.0 MB 00:01 2024-09-11T23:06:44.411 INFO:teuthology.orchestra.run.smithi026.stdout:(45/156): python3-cryptography-36.0.1-4.el9.x86 6.4 MB/s | 1.2 MB 00:00 2024-09-11T23:06:44.415 INFO:teuthology.orchestra.run.smithi076.stdout:(88/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.2 MB/s | 159 kB 00:00 2024-09-11T23:06:44.436 INFO:teuthology.orchestra.run.smithi026.stdout:(46/156): python3-requests-2.25.1-8.el9.noarch. 2.1 MB/s | 125 kB 00:00 2024-09-11T23:06:44.461 INFO:teuthology.orchestra.run.smithi026.stdout:(47/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 945 kB/s | 253 kB 00:00 2024-09-11T23:06:44.465 INFO:teuthology.orchestra.run.smithi076.stdout:(89/156): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-09-11T23:06:44.490 INFO:teuthology.orchestra.run.smithi076.stdout:(90/156): python3-pyasn1-modules-0.4.8-6.el9.no 2.7 MB/s | 279 kB 00:00 2024-09-11T23:06:44.495 INFO:teuthology.orchestra.run.smithi026.stdout:(48/156): python3-urllib3-1.26.5-6.el9.noarch.r 2.5 MB/s | 215 kB 00:00 2024-09-11T23:06:44.506 INFO:teuthology.orchestra.run.smithi136.stdout:(69/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 3.6 MB/s | 4.4 MB 00:01 2024-09-11T23:06:44.516 INFO:teuthology.orchestra.run.smithi076.stdout:(91/156): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-09-11T23:06:44.566 INFO:teuthology.orchestra.run.smithi076.stdout:(92/156): python3-toml-0.10.2-6.el9.noarch.rpm 836 kB/s | 42 kB 00:00 2024-09-11T23:06:44.616 INFO:teuthology.orchestra.run.smithi076.stdout:(93/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 884 kB/s | 44 kB 00:00 2024-09-11T23:06:44.637 INFO:teuthology.orchestra.run.smithi026.stdout:(49/156): automake-1.16.2-8.el9.noarch.rpm 3.4 MB/s | 689 kB 00:00 2024-09-11T23:06:44.662 INFO:teuthology.orchestra.run.smithi026.stdout:(50/156): boost-program-options-1.75.0-8.el9.x8 640 kB/s | 107 kB 00:00 2024-09-11T23:06:44.675 INFO:teuthology.orchestra.run.smithi076.stdout:(94/156): qatlib-24.02.0-1.el9.x86_64.rpm 3.7 MB/s | 221 kB 00:00 2024-09-11T23:06:44.704 INFO:teuthology.orchestra.run.smithi026.stdout:(51/156): bison-3.7.4-5.el9.x86_64.rpm 3.8 MB/s | 943 kB 00:00 2024-09-11T23:06:44.721 INFO:teuthology.orchestra.run.smithi026.stdout:(52/156): elfutils-libelf-devel-0.191-4.el9.x86 259 kB/s | 22 kB 00:00 2024-09-11T23:06:44.725 INFO:teuthology.orchestra.run.smithi076.stdout:(95/156): qatlib-service-24.02.0-1.el9.x86_64.r 718 kB/s | 36 kB 00:00 2024-09-11T23:06:44.763 INFO:teuthology.orchestra.run.smithi026.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 557 kB/s | 33 kB 00:00 2024-09-11T23:06:44.775 INFO:teuthology.orchestra.run.smithi076.stdout:(96/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 950 kB/s | 47 kB 00:00 2024-09-11T23:06:44.788 INFO:teuthology.orchestra.run.smithi026.stdout:(54/156): flex-2.6.4-9.el9.x86_64.rpm 2.4 MB/s | 313 kB 00:00 2024-09-11T23:06:44.864 INFO:teuthology.orchestra.run.smithi026.stdout:(55/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 21 MB/s | 3.0 MB 00:00 2024-09-11T23:06:44.876 INFO:teuthology.orchestra.run.smithi076.stdout:(97/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 4.3 MB/s | 5.1 MB 00:01 2024-09-11T23:06:44.880 INFO:teuthology.orchestra.run.smithi026.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 121 kB/s | 11 kB 00:00 2024-09-11T23:06:44.901 INFO:teuthology.orchestra.run.smithi076.stdout:(98/156): socat-1.7.4.1-6.el9.x86_64.rpm 2.4 MB/s | 304 kB 00:00 2024-09-11T23:06:44.906 INFO:teuthology.orchestra.run.smithi026.stdout:(57/156): flexiblas-openblas-openmp-3.0.4-8.el9 128 kB/s | 18 kB 00:00 2024-09-11T23:06:44.918 INFO:teuthology.orchestra.run.smithi076.stdout:(99/156): uuid-1.6.2-55.el9.x86_64.rpm 1.3 MB/s | 58 kB 00:00 2024-09-11T23:06:44.939 INFO:teuthology.orchestra.run.smithi026.stdout:(58/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.1 MB/s | 160 kB 00:00 2024-09-11T23:06:44.952 INFO:teuthology.orchestra.run.smithi076.stdout:(100/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.9 MB/s | 96 kB 00:00 2024-09-11T23:06:44.965 INFO:teuthology.orchestra.run.smithi026.stdout:(59/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 538 kB/s | 45 kB 00:00 2024-09-11T23:06:44.969 INFO:teuthology.orchestra.run.smithi076.stdout:(101/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-09-11T23:06:45.007 INFO:teuthology.orchestra.run.smithi026.stdout:(60/156): librdkafka-1.6.1-102.el9.x86_64.rpm 6.4 MB/s | 662 kB 00:00 2024-09-11T23:06:45.010 INFO:teuthology.orchestra.run.smithi076.stdout:(102/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 767 kB/s | 45 kB 00:00 2024-09-11T23:06:45.040 INFO:teuthology.orchestra.run.smithi026.stdout:(61/156): libtool-2.4.6-46.el9.x86_64.rpm 5.6 MB/s | 581 kB 00:00 2024-09-11T23:06:45.066 INFO:teuthology.orchestra.run.smithi026.stdout:(62/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 243 kB/s | 24 kB 00:00 2024-09-11T23:06:45.073 INFO:teuthology.orchestra.run.smithi136.stdout:(70/156): postgresql-private-libs-13.16-1.el9.x 194 kB/s | 137 kB 00:00 2024-09-11T23:06:45.077 INFO:teuthology.orchestra.run.smithi076.stdout:(103/156): lua-devel-5.4.4-4.el9.x86_64.rpm 205 kB/s | 22 kB 00:00 2024-09-11T23:06:45.116 INFO:teuthology.orchestra.run.smithi026.stdout:(63/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 435 kB/s | 47 kB 00:00 2024-09-11T23:06:45.128 INFO:teuthology.orchestra.run.smithi076.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 11 MB/s | 548 kB 00:00 2024-09-11T23:06:45.141 INFO:teuthology.orchestra.run.smithi026.stdout:(64/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.8 MB/s | 292 kB 00:00 2024-09-11T23:06:45.153 INFO:teuthology.orchestra.run.smithi076.stdout:(105/156): gperftools-libs-2.9.1-2.el9.x86_64.r 12 MB/s | 309 kB 00:00 2024-09-11T23:06:45.166 INFO:teuthology.orchestra.run.smithi026.stdout:(65/156): lua-5.4.4-4.el9.x86_64.rpm 1.8 MB/s | 188 kB 00:00 2024-09-11T23:06:45.170 INFO:teuthology.orchestra.run.smithi076.stdout:(106/156): grpc-data-1.46.7-10.el9.noarch.rpm 1.1 MB/s | 19 kB 00:00 2024-09-11T23:06:45.200 INFO:teuthology.orchestra.run.smithi026.stdout:(66/156): openblas-0.3.26-2.el9.x86_64.rpm 460 kB/s | 38 kB 00:00 2024-09-11T23:06:45.237 INFO:teuthology.orchestra.run.smithi076.stdout:(107/156): protobuf-compiler-3.14.0-13.el9.x86_ 3.7 MB/s | 863 kB 00:00 2024-09-11T23:06:45.257 INFO:teuthology.orchestra.run.smithi136.stdout:(71/156): postgresql-contrib-13.16-1.el9.x86_64 983 kB/s | 910 kB 00:00 2024-09-11T23:06:45.301 INFO:teuthology.orchestra.run.smithi026.stdout:(67/156): openblas-openmp-0.3.26-2.el9.x86_64.r 31 MB/s | 4.9 MB 00:00 2024-09-11T23:06:45.312 INFO:teuthology.orchestra.run.smithi076.stdout:(108/156): libarrow-9.0.0-11.el9.x86_64.rpm 31 MB/s | 4.4 MB 00:00 2024-09-11T23:06:45.329 INFO:teuthology.orchestra.run.smithi076.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 3.3 MB/s | 49 kB 00:00 2024-09-11T23:06:45.342 INFO:teuthology.orchestra.run.smithi026.stdout:(68/156): postgresql-13.16-1.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-09-11T23:06:45.346 INFO:teuthology.orchestra.run.smithi076.stdout:(110/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 239 kB/s | 26 kB 00:00 2024-09-11T23:06:45.371 INFO:teuthology.orchestra.run.smithi076.stdout:(111/156): libunwind-1.6.2-1.el9.x86_64.rpm 1.6 MB/s | 67 kB 00:00 2024-09-11T23:06:45.397 INFO:teuthology.orchestra.run.smithi076.stdout:(112/156): luarocks-3.9.2-5.el9.noarch.rpm 3.0 MB/s | 151 kB 00:00 2024-09-11T23:06:45.409 INFO:teuthology.orchestra.run.smithi026.stdout:(69/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 18 MB/s | 4.4 MB 00:00 2024-09-11T23:06:45.430 INFO:teuthology.orchestra.run.smithi076.stdout:(113/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 14 MB/s | 839 kB 00:00 2024-09-11T23:06:45.443 INFO:teuthology.orchestra.run.smithi026.stdout:(70/156): postgresql-private-libs-13.16-1.el9.x 1.3 MB/s | 137 kB 00:00 2024-09-11T23:06:45.488 INFO:teuthology.orchestra.run.smithi026.stdout:(71/156): postgresql-contrib-13.16-1.el9.x86_64 5.3 MB/s | 910 kB 00:00 2024-09-11T23:06:45.488 INFO:teuthology.orchestra.run.smithi076.stdout:(114/156): python3-asyncssh-2.13.2-5.el9.noarch 9.1 MB/s | 548 kB 00:00 2024-09-11T23:06:45.488 INFO:teuthology.orchestra.run.smithi076.stdout:(115/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 867 kB/s | 43 kB 00:00 2024-09-11T23:06:45.506 INFO:teuthology.orchestra.run.smithi076.stdout:(116/156): python3-certifi-2023.05.07-4.el9.noa 565 kB/s | 14 kB 00:00 2024-09-11T23:06:45.531 INFO:teuthology.orchestra.run.smithi076.stdout:(117/156): python3-cachetools-4.2.4-1.el9.noarc 640 kB/s | 32 kB 00:00 2024-09-11T23:06:45.535 INFO:teuthology.orchestra.run.smithi026.stdout:(72/156): protobuf-3.14.0-13.el9.x86_64.rpm 11 MB/s | 1.0 MB 00:00 2024-09-11T23:06:45.557 INFO:teuthology.orchestra.run.smithi076.stdout:(118/156): python3-cheroot-8.6.0-4.el9.noarch.r 3.3 MB/s | 172 kB 00:00 2024-09-11T23:06:45.560 INFO:teuthology.orchestra.run.smithi026.stdout:(73/156): python3-attrs-20.3.0-7.el9.noarch.rpm 951 kB/s | 87 kB 00:00 2024-09-11T23:06:45.590 INFO:teuthology.orchestra.run.smithi076.stdout:(119/156): python3-cherrypy-18.6.1-2.el9.noarch 6.0 MB/s | 358 kB 00:00 2024-09-11T23:06:45.616 INFO:teuthology.orchestra.run.smithi076.stdout:(120/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.1 MB/s | 303 kB 00:00 2024-09-11T23:06:45.641 INFO:teuthology.orchestra.run.smithi076.stdout:(121/156): python3-google-auth-2.34.0-1.el9.noa 4.3 MB/s | 220 kB 00:00 2024-09-11T23:06:45.657 INFO:teuthology.orchestra.run.smithi136.stdout:(72/156): postgresql-server-13.16-1.el9.x86_64. 5.0 MB/s | 5.8 MB 00:01 2024-09-11T23:06:45.660 INFO:teuthology.orchestra.run.smithi026.stdout:(74/156): postgresql-server-13.16-1.el9.x86_64. 23 MB/s | 5.8 MB 00:00 2024-09-11T23:06:45.675 INFO:teuthology.orchestra.run.smithi076.stdout:(122/156): python3-grpcio-tools-1.46.7-10.el9.x 4.2 MB/s | 144 kB 00:00 2024-09-11T23:06:45.686 INFO:teuthology.orchestra.run.smithi026.stdout:(75/156): python3-colorama-0.4.6-3.el9.noarch.r 488 kB/s | 61 kB 00:00 2024-09-11T23:06:45.708 INFO:teuthology.orchestra.run.smithi076.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 322 kB/s | 11 kB 00:00 2024-09-11T23:06:45.728 INFO:teuthology.orchestra.run.smithi026.stdout:(76/156): python3-devel-3.9.19-8.el9.x86_64.rpm 3.6 MB/s | 245 kB 00:00 2024-09-11T23:06:45.733 INFO:teuthology.orchestra.run.smithi076.stdout:(124/156): python3-jaraco-classes-3.2.1-5.el9.n 704 kB/s | 18 kB 00:00 2024-09-11T23:06:45.759 INFO:teuthology.orchestra.run.smithi076.stdout:(125/156): python3-jaraco-collections-3.0.0-8.e 932 kB/s | 23 kB 00:00 2024-09-11T23:06:45.784 INFO:teuthology.orchestra.run.smithi076.stdout:(126/156): python3-jaraco-functools-3.5.0-2.el9 780 kB/s | 19 kB 00:00 2024-09-11T23:06:45.809 INFO:teuthology.orchestra.run.smithi076.stdout:(127/156): python3-jaraco-text-3.2.0-6.el9.noar 789 kB/s | 20 kB 00:00 2024-09-11T23:06:45.811 INFO:teuthology.orchestra.run.smithi026.stdout:(77/156): python3-babel-2.9.1-2.el9.noarch.rpm 22 MB/s | 6.0 MB 00:00 2024-09-11T23:06:45.828 INFO:teuthology.orchestra.run.smithi026.stdout:(78/156): python3-jsonpatch-1.21-16.el9.noarch. 255 kB/s | 26 kB 00:00 2024-09-11T23:06:45.854 INFO:teuthology.orchestra.run.smithi026.stdout:(79/156): python3-jinja2-2.11.3-6.el9.noarch.rp 1.4 MB/s | 248 kB 00:00 2024-09-11T23:06:45.868 INFO:teuthology.orchestra.run.smithi076.stdout:(128/156): python3-grpcio-1.46.7-10.el9.x86_64. 8.1 MB/s | 2.0 MB 00:00 2024-09-11T23:06:45.879 INFO:teuthology.orchestra.run.smithi026.stdout:(80/156): python3-jsonpointer-2.0-4.el9.noarch. 286 kB/s | 19 kB 00:00 2024-09-11T23:06:45.893 INFO:teuthology.orchestra.run.smithi076.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-11T23:06:45.910 INFO:teuthology.orchestra.run.smithi076.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 4.6 MB/s | 79 kB 00:00 2024-09-11T23:06:45.913 INFO:teuthology.orchestra.run.smithi026.stdout:(81/156): python3-mako-1.1.4-6.el9.noarch.rpm 2.0 MB/s | 172 kB 00:00 2024-09-11T23:06:45.929 INFO:teuthology.orchestra.run.smithi026.stdout:(82/156): python3-markupsafe-1.1.1-12.el9.x86_6 461 kB/s | 35 kB 00:00 2024-09-11T23:06:45.936 INFO:teuthology.orchestra.run.smithi076.stdout:(131/156): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-09-11T23:06:45.969 INFO:teuthology.orchestra.run.smithi076.stdout:(132/156): python3-kubernetes-26.1.0-2.el9.noar 6.4 MB/s | 1.0 MB 00:00 2024-09-11T23:06:45.980 INFO:teuthology.orchestra.run.smithi026.stdout:(83/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 3.7 MB/s | 250 kB 00:00 2024-09-11T23:06:45.995 INFO:teuthology.orchestra.run.smithi076.stdout:(133/156): python3-pecan-1.4.2-3.el9.noarch.rpm 4.6 MB/s | 272 kB 00:00 2024-09-11T23:06:46.011 INFO:teuthology.orchestra.run.smithi076.stdout:(134/156): python3-portend-3.1.0-2.el9.noarch.r 392 kB/s | 16 kB 00:00 2024-09-11T23:06:46.013 INFO:teuthology.orchestra.run.smithi026.stdout:(84/156): python3-oauthlib-3.1.1-5.el9.noarch.r 2.6 MB/s | 222 kB 00:00 2024-09-11T23:06:46.037 INFO:teuthology.orchestra.run.smithi076.stdout:(135/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 2.1 MB/s | 90 kB 00:00 2024-09-11T23:06:46.040 INFO:teuthology.orchestra.run.smithi136.stdout:(73/156): python3-attrs-20.3.0-7.el9.noarch.rpm 112 kB/s | 87 kB 00:00 2024-09-11T23:06:46.054 INFO:teuthology.orchestra.run.smithi076.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 673 kB/s | 28 kB 00:00 2024-09-11T23:06:46.071 INFO:teuthology.orchestra.run.smithi076.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 917 kB/s | 31 kB 00:00 2024-09-11T23:06:46.096 INFO:teuthology.orchestra.run.smithi076.stdout:(138/156): python3-routes-2.5.1-5.el9.noarch.rp 4.4 MB/s | 188 kB 00:00 2024-09-11T23:06:46.097 INFO:teuthology.orchestra.run.smithi026.stdout:(85/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 23 MB/s | 5.1 MB 00:00 2024-09-11T23:06:46.113 INFO:teuthology.orchestra.run.smithi076.stdout:(139/156): python3-rsa-4.9-2.el9.noarch.rpm 1.4 MB/s | 59 kB 00:00 2024-09-11T23:06:46.114 INFO:teuthology.orchestra.run.smithi026.stdout:(86/156): python3-prettytable-0.7.2-27.el9.noar 422 kB/s | 42 kB 00:00 2024-09-11T23:06:46.130 INFO:teuthology.orchestra.run.smithi076.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 1.0 MB/s | 36 kB 00:00 2024-09-11T23:06:46.139 INFO:teuthology.orchestra.run.smithi026.stdout:(87/156): python3-packaging-20.9-5.el9.noarch.r 485 kB/s | 77 kB 00:00 2024-09-11T23:06:46.155 INFO:teuthology.orchestra.run.smithi076.stdout:(141/156): python3-typing-extensions-4.12.2-2.e 1.7 MB/s | 74 kB 00:00 2024-09-11T23:06:46.173 INFO:teuthology.orchestra.run.smithi026.stdout:(88/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.6 MB/s | 159 kB 00:00 2024-09-11T23:06:46.174 INFO:teuthology.orchestra.run.smithi136.stdout:(74/156): protobuf-3.14.0-13.el9.x86_64.rpm 939 kB/s | 1.0 MB 00:01 2024-09-11T23:06:46.181 INFO:teuthology.orchestra.run.smithi076.stdout:(142/156): python3-webob-1.8.8-2.el9.noarch.rpm 4.5 MB/s | 230 kB 00:00 2024-09-11T23:06:46.198 INFO:teuthology.orchestra.run.smithi026.stdout:(89/156): python3-protobuf-3.14.0-13.el9.noarch 2.6 MB/s | 269 kB 00:00 2024-09-11T23:06:46.206 INFO:teuthology.orchestra.run.smithi076.stdout:(143/156): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-09-11T23:06:46.224 INFO:teuthology.orchestra.run.smithi026.stdout:(90/156): python3-pyasn1-modules-0.4.8-6.el9.no 3.2 MB/s | 279 kB 00:00 2024-09-11T23:06:46.240 INFO:teuthology.orchestra.run.smithi076.stdout:(144/156): python3-werkzeug-2.0.3-3.el9.1.noarc 7.1 MB/s | 427 kB 00:00 2024-09-11T23:06:46.249 INFO:teuthology.orchestra.run.smithi026.stdout:(91/156): python3-pytz-2021.1-5.el9.noarch.rpm 672 kB/s | 51 kB 00:00 2024-09-11T23:06:46.256 INFO:teuthology.orchestra.run.smithi076.stdout:(145/156): python3-zc-lockfile-2.0-10.el9.noarc 398 kB/s | 20 kB 00:00 2024-09-11T23:06:46.282 INFO:teuthology.orchestra.run.smithi076.stdout:(146/156): re2-20211101-20.el9.x86_64.rpm 4.5 MB/s | 191 kB 00:00 2024-09-11T23:06:46.291 INFO:teuthology.orchestra.run.smithi026.stdout:(92/156): python3-requests-oauthlib-1.3.0-12.el 582 kB/s | 54 kB 00:00 2024-09-11T23:06:46.324 INFO:teuthology.orchestra.run.smithi026.stdout:(93/156): python3-toml-0.10.2-6.el9.noarch.rpm 554 kB/s | 42 kB 00:00 2024-09-11T23:06:46.357 INFO:teuthology.orchestra.run.smithi076.stdout:(147/156): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-09-11T23:06:46.375 INFO:teuthology.orchestra.run.smithi026.stdout:(94/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 528 kB/s | 44 kB 00:00 2024-09-11T23:06:46.425 INFO:teuthology.orchestra.run.smithi026.stdout:(95/156): qatlib-24.02.0-1.el9.x86_64.rpm 2.1 MB/s | 221 kB 00:00 2024-09-11T23:06:46.450 INFO:teuthology.orchestra.run.smithi026.stdout:(96/156): qatlib-service-24.02.0-1.el9.x86_64.r 477 kB/s | 36 kB 00:00 2024-09-11T23:06:46.492 INFO:teuthology.orchestra.run.smithi026.stdout:(97/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 711 kB/s | 47 kB 00:00 2024-09-11T23:06:46.582 INFO:teuthology.orchestra.run.smithi076.stdout:(148/156): librbd1-19.1.1-298.gb53b6d3e.el9.x86 14 MB/s | 3.2 MB 00:00 2024-09-11T23:06:46.666 INFO:teuthology.orchestra.run.smithi136.stdout:(75/156): python3-colorama-0.4.6-3.el9.noarch.r 98 kB/s | 61 kB 00:00 2024-09-11T23:06:46.668 INFO:teuthology.orchestra.run.smithi026.stdout:(98/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 37 MB/s | 16 MB 00:00 2024-09-11T23:06:46.685 INFO:teuthology.orchestra.run.smithi026.stdout:(99/156): uuid-1.6.2-55.el9.x86_64.rpm 301 kB/s | 58 kB 00:00 2024-09-11T23:06:46.716 INFO:teuthology.orchestra.run.smithi076.stdout:(149/156): librados2-19.1.1-298.gb53b6d3e.el9.x 7.9 MB/s | 3.4 MB 00:00 2024-09-11T23:06:46.718 INFO:teuthology.orchestra.run.smithi026.stdout:(100/156): socat-1.7.4.1-6.el9.x86_64.rpm 1.1 MB/s | 304 kB 00:00 2024-09-11T23:06:46.743 INFO:teuthology.orchestra.run.smithi026.stdout:(101/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.2 MB/s | 96 kB 00:00 2024-09-11T23:06:46.768 INFO:teuthology.orchestra.run.smithi026.stdout:(102/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 761 kB/s | 64 kB 00:00 2024-09-11T23:06:46.800 INFO:teuthology.orchestra.run.smithi076.stdout:(150/156): NetworkManager-1.49.90-1.el9.x86_64. 11 MB/s | 2.3 MB 00:00 2024-09-11T23:06:46.802 INFO:teuthology.orchestra.run.smithi026.stdout:(103/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 535 kB/s | 45 kB 00:00 2024-09-11T23:06:46.833 INFO:teuthology.orchestra.run.smithi076.stdout:(151/156): NetworkManager-team-1.49.90-1.el9.x8 1.2 MB/s | 41 kB 00:00 2024-09-11T23:06:46.875 INFO:teuthology.orchestra.run.smithi076.stdout:(152/156): NetworkManager-tui-1.49.90-1.el9.x86 5.9 MB/s | 249 kB 00:00 2024-09-11T23:06:46.886 INFO:teuthology.orchestra.run.smithi026.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 6.4 MB/s | 548 kB 00:00 2024-09-11T23:06:46.908 INFO:teuthology.orchestra.run.smithi136.stdout:(76/156): python3-babel-2.9.1-2.el9.noarch.rpm 4.8 MB/s | 6.0 MB 00:01 2024-09-11T23:06:46.909 INFO:teuthology.orchestra.run.smithi076.stdout:(153/156): NetworkManager-libnm-1.49.90-1.el9.x 9.4 MB/s | 1.8 MB 00:00 2024-09-11T23:06:46.928 INFO:teuthology.orchestra.run.smithi026.stdout:(105/156): gperftools-libs-2.9.1-2.el9.x86_64.r 7.2 MB/s | 309 kB 00:00 2024-09-11T23:06:46.951 INFO:teuthology.orchestra.run.smithi076.stdout:(154/156): openssl-3.2.2-6.el9.x86_64.rpm 18 MB/s | 1.4 MB 00:00 2024-09-11T23:06:46.953 INFO:teuthology.orchestra.run.smithi026.stdout:(106/156): grpc-data-1.46.7-10.el9.noarch.rpm 784 kB/s | 19 kB 00:00 2024-09-11T23:06:47.009 INFO:teuthology.orchestra.run.smithi076.stdout:(155/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 24 MB/s | 2.4 MB 00:00 2024-09-11T23:06:47.016 INFO:teuthology.orchestra.run.smithi136.stdout:(77/156): python3-devel-3.9.19-8.el9.x86_64.rpm 291 kB/s | 245 kB 00:00 2024-09-11T23:06:47.053 INFO:teuthology.orchestra.run.smithi026.stdout:(107/156): lua-devel-5.4.4-4.el9.x86_64.rpm 72 kB/s | 22 kB 00:00 2024-09-11T23:06:47.103 INFO:teuthology.orchestra.run.smithi026.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 520 kB/s | 26 kB 00:00 2024-09-11T23:06:47.137 INFO:teuthology.orchestra.run.smithi026.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 1.4 MB/s | 49 kB 00:00 2024-09-11T23:06:47.171 INFO:teuthology.orchestra.run.smithi026.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 2.0 MB/s | 67 kB 00:00 2024-09-11T23:06:47.204 INFO:teuthology.orchestra.run.smithi026.stdout:(111/156): luarocks-3.9.2-5.el9.noarch.rpm 4.5 MB/s | 151 kB 00:00 2024-09-11T23:06:47.280 INFO:teuthology.orchestra.run.smithi026.stdout:(112/156): libarrow-9.0.0-11.el9.x86_64.rpm 14 MB/s | 4.4 MB 00:00 2024-09-11T23:06:47.330 INFO:teuthology.orchestra.run.smithi026.stdout:(113/156): python3-asyncssh-2.13.2-5.el9.noarch 11 MB/s | 548 kB 00:00 2024-09-11T23:06:47.351 INFO:teuthology.orchestra.run.smithi076.stdout:(156/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 5.7 MB/s | 16 MB 00:02 2024-09-11T23:06:47.355 INFO:teuthology.orchestra.run.smithi026.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-09-11T23:06:47.356 INFO:teuthology.orchestra.run.smithi076.stdout:-------------------------------------------------------------------------------- 2024-09-11T23:06:47.356 INFO:teuthology.orchestra.run.smithi076.stdout:Total 19 MB/s | 226 MB 00:12 2024-09-11T23:06:47.380 INFO:teuthology.orchestra.run.smithi026.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-09-11T23:06:47.414 INFO:teuthology.orchestra.run.smithi026.stdout:(116/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 3.9 MB/s | 839 kB 00:00 2024-09-11T23:06:47.439 INFO:teuthology.orchestra.run.smithi026.stdout:(117/156): python3-certifi-2023.05.07-4.el9.noa 241 kB/s | 14 kB 00:00 2024-09-11T23:06:47.464 INFO:teuthology.orchestra.run.smithi026.stdout:(118/156): python3-cheroot-8.6.0-4.el9.noarch.r 3.3 MB/s | 172 kB 00:00 2024-09-11T23:06:47.498 INFO:teuthology.orchestra.run.smithi026.stdout:(119/156): python3-cherrypy-18.6.1-2.el9.noarch 5.9 MB/s | 358 kB 00:00 2024-09-11T23:06:47.532 INFO:teuthology.orchestra.run.smithi026.stdout:(120/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 4.4 MB/s | 303 kB 00:00 2024-09-11T23:06:47.533 INFO:teuthology.orchestra.run.smithi136.stdout:(78/156): python3-jsonpatch-1.21-16.el9.noarch. 41 kB/s | 26 kB 00:00 2024-09-11T23:06:47.557 INFO:teuthology.orchestra.run.smithi026.stdout:(121/156): python3-google-auth-2.34.0-1.el9.noa 3.7 MB/s | 220 kB 00:00 2024-09-11T23:06:47.558 INFO:teuthology.orchestra.run.smithi136.stdout:(79/156): python3-jsonpointer-2.0-4.el9.noarch. 35 kB/s | 19 kB 00:00 2024-09-11T23:06:47.583 INFO:teuthology.orchestra.run.smithi026.stdout:(122/156): python3-grpcio-tools-1.46.7-10.el9.x 5.7 MB/s | 144 kB 00:00 2024-09-11T23:06:47.592 INFO:teuthology.orchestra.run.smithi136.stdout:(80/156): python3-jinja2-2.11.3-6.el9.noarch.rp 268 kB/s | 248 kB 00:00 2024-09-11T23:06:47.599 INFO:teuthology.orchestra.run.smithi026.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 649 kB/s | 11 kB 00:00 2024-09-11T23:06:47.642 INFO:teuthology.orchestra.run.smithi026.stdout:(124/156): protobuf-compiler-3.14.0-13.el9.x86_ 989 kB/s | 863 kB 00:00 2024-09-11T23:06:47.658 INFO:teuthology.orchestra.run.smithi026.stdout:(125/156): python3-jaraco-classes-3.2.1-5.el9.n 302 kB/s | 18 kB 00:00 2024-09-11T23:06:47.684 INFO:teuthology.orchestra.run.smithi026.stdout:(126/156): python3-jaraco-functools-3.5.0-2.el9 1.2 MB/s | 19 kB 00:00 2024-09-11T23:06:47.709 INFO:teuthology.orchestra.run.smithi026.stdout:(127/156): python3-jaraco-text-3.2.0-6.el9.noar 843 kB/s | 20 kB 00:00 2024-09-11T23:06:47.726 INFO:teuthology.orchestra.run.smithi026.stdout:(128/156): python3-jaraco-collections-3.0.0-8.e 274 kB/s | 23 kB 00:00 2024-09-11T23:06:47.752 INFO:teuthology.orchestra.run.smithi026.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-11T23:06:47.785 INFO:teuthology.orchestra.run.smithi026.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 2.3 MB/s | 79 kB 00:00 2024-09-11T23:06:47.827 INFO:teuthology.orchestra.run.smithi026.stdout:(131/156): python3-kubernetes-26.1.0-2.el9.noar 8.7 MB/s | 1.0 MB 00:00 2024-09-11T23:06:47.844 INFO:teuthology.orchestra.run.smithi026.stdout:(132/156): python3-natsort-7.1.1-5.el9.noarch.r 985 kB/s | 58 kB 00:00 2024-09-11T23:06:47.895 INFO:teuthology.orchestra.run.smithi026.stdout:(133/156): python3-grpcio-1.46.7-10.el9.x86_64. 5.6 MB/s | 2.0 MB 00:00 2024-09-11T23:06:47.911 INFO:teuthology.orchestra.run.smithi026.stdout:(134/156): python3-portend-3.1.0-2.el9.noarch.r 246 kB/s | 16 kB 00:00 2024-09-11T23:06:47.937 INFO:teuthology.orchestra.run.smithi026.stdout:(135/156): python3-pecan-1.4.2-3.el9.noarch.rpm 2.4 MB/s | 272 kB 00:00 2024-09-11T23:06:47.953 INFO:teuthology.orchestra.run.smithi026.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 677 kB/s | 28 kB 00:00 2024-09-11T23:06:47.970 INFO:teuthology.orchestra.run.smithi026.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 921 kB/s | 31 kB 00:00 2024-09-11T23:06:47.995 INFO:teuthology.orchestra.run.smithi026.stdout:(138/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.0 MB/s | 90 kB 00:00 2024-09-11T23:06:48.013 INFO:teuthology.orchestra.run.smithi026.stdout:(139/156): python3-rsa-4.9-2.el9.noarch.rpm 1.4 MB/s | 59 kB 00:00 2024-09-11T23:06:48.029 INFO:teuthology.orchestra.run.smithi026.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 1.0 MB/s | 36 kB 00:00 2024-09-11T23:06:48.055 INFO:teuthology.orchestra.run.smithi026.stdout:(141/156): python3-routes-2.5.1-5.el9.noarch.rp 1.8 MB/s | 188 kB 00:00 2024-09-11T23:06:48.080 INFO:teuthology.orchestra.run.smithi026.stdout:(142/156): python3-typing-extensions-4.12.2-2.e 1.1 MB/s | 74 kB 00:00 2024-09-11T23:06:48.105 INFO:teuthology.orchestra.run.smithi026.stdout:(143/156): python3-websocket-client-1.2.3-2.el9 3.5 MB/s | 90 kB 00:00 2024-09-11T23:06:48.130 INFO:teuthology.orchestra.run.smithi026.stdout:(144/156): python3-webob-1.8.8-2.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-09-11T23:06:48.134 INFO:teuthology.orchestra.run.smithi136.stdout:(81/156): python3-markupsafe-1.1.1-12.el9.x86_6 60 kB/s | 35 kB 00:00 2024-09-11T23:06:48.156 INFO:teuthology.orchestra.run.smithi026.stdout:(145/156): python3-zc-lockfile-2.0-10.el9.noarc 398 kB/s | 20 kB 00:00 2024-09-11T23:06:48.189 INFO:teuthology.orchestra.run.smithi026.stdout:(146/156): re2-20211101-20.el9.x86_64.rpm 3.2 MB/s | 191 kB 00:00 2024-09-11T23:06:48.215 INFO:teuthology.orchestra.run.smithi026.stdout:(147/156): python3-werkzeug-2.0.3-3.el9.1.noarc 3.1 MB/s | 427 kB 00:00 2024-09-11T23:06:48.273 INFO:teuthology.orchestra.run.smithi026.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 14 MB/s | 1.6 MB 00:00 2024-09-11T23:06:48.368 INFO:teuthology.orchestra.run.smithi136.stdout:(82/156): python3-mako-1.1.4-6.el9.noarch.rpm 207 kB/s | 172 kB 00:00 2024-09-11T23:06:48.440 INFO:teuthology.orchestra.run.smithi026.stdout:(149/156): librados2-19.1.1-298.gb53b6d3e.el9.x 14 MB/s | 3.4 MB 00:00 2024-09-11T23:06:48.585 INFO:teuthology.orchestra.run.smithi076.stdout:Running transaction check 2024-09-11T23:06:48.615 INFO:teuthology.orchestra.run.smithi026.stdout:(150/156): librbd1-19.1.1-298.gb53b6d3e.el9.x86 7.9 MB/s | 3.2 MB 00:00 2024-09-11T23:06:48.732 INFO:teuthology.orchestra.run.smithi026.stdout:(151/156): NetworkManager-team-1.49.90-1.el9.x8 349 kB/s | 41 kB 00:00 2024-09-11T23:06:48.735 INFO:teuthology.orchestra.run.smithi076.stdout:Transaction check succeeded. 2024-09-11T23:06:48.735 INFO:teuthology.orchestra.run.smithi076.stdout:Running transaction test 2024-09-11T23:06:48.783 INFO:teuthology.orchestra.run.smithi026.stdout:(152/156): NetworkManager-1.49.90-1.el9.x86_64. 4.6 MB/s | 2.3 MB 00:00 2024-09-11T23:06:48.850 INFO:teuthology.orchestra.run.smithi026.stdout:(153/156): NetworkManager-tui-1.49.90-1.el9.x86 2.1 MB/s | 249 kB 00:00 2024-09-11T23:06:48.892 INFO:teuthology.orchestra.run.smithi026.stdout:(154/156): NetworkManager-libnm-1.49.90-1.el9.x 4.0 MB/s | 1.8 MB 00:00 2024-09-11T23:06:49.101 INFO:teuthology.orchestra.run.smithi136.stdout:(83/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 259 kB/s | 250 kB 00:00 2024-09-11T23:06:49.134 INFO:teuthology.orchestra.run.smithi026.stdout:(155/156): openssl-3.2.2-6.el9.x86_64.rpm 3.9 MB/s | 1.4 MB 00:00 2024-09-11T23:06:49.243 INFO:teuthology.orchestra.run.smithi136.stdout:(84/156): python3-oauthlib-3.1.1-5.el9.noarch.r 256 kB/s | 222 kB 00:00 2024-09-11T23:06:49.685 INFO:teuthology.orchestra.run.smithi136.stdout:(85/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 2.4 MB/s | 5.1 MB 00:02 2024-09-11T23:06:49.700 INFO:teuthology.orchestra.run.smithi026.stdout:(156/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 2.8 MB/s | 2.4 MB 00:00 2024-09-11T23:06:49.707 INFO:teuthology.orchestra.run.smithi026.stdout:-------------------------------------------------------------------------------- 2024-09-11T23:06:49.707 INFO:teuthology.orchestra.run.smithi026.stdout:Total 22 MB/s | 226 MB 00:10 2024-09-11T23:06:49.819 INFO:teuthology.orchestra.run.smithi136.stdout:(86/156): python3-packaging-20.9-5.el9.noarch.r 108 kB/s | 77 kB 00:00 2024-09-11T23:06:49.894 INFO:teuthology.orchestra.run.smithi136.stdout:(87/156): python3-prettytable-0.7.2-27.el9.noar 65 kB/s | 42 kB 00:00 2024-09-11T23:06:50.539 INFO:teuthology.orchestra.run.smithi076.stdout:Transaction test succeeded. 2024-09-11T23:06:50.540 INFO:teuthology.orchestra.run.smithi076.stdout:Running transaction 2024-09-11T23:06:50.661 INFO:teuthology.orchestra.run.smithi136.stdout:(88/156): python3-protobuf-3.14.0-13.el9.noarch 276 kB/s | 269 kB 00:00 2024-09-11T23:06:50.719 INFO:teuthology.orchestra.run.smithi136.stdout:(89/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 176 kB/s | 159 kB 00:00 2024-09-11T23:06:50.917 INFO:teuthology.orchestra.run.smithi026.stdout:Running transaction check 2024-09-11T23:06:50.978 INFO:teuthology.orchestra.run.smithi136.stdout:(90/156): python3-pyasn1-modules-0.4.8-6.el9.no 257 kB/s | 279 kB 00:01 2024-09-11T23:06:51.083 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction check succeeded. 2024-09-11T23:06:51.083 INFO:teuthology.orchestra.run.smithi026.stdout:Running transaction test 2024-09-11T23:06:51.278 INFO:teuthology.orchestra.run.smithi136.stdout:(91/156): python3-pytz-2021.1-5.el9.noarch.rpm 83 kB/s | 51 kB 00:00 2024-09-11T23:06:51.504 INFO:teuthology.orchestra.run.smithi136.stdout:(92/156): python3-requests-oauthlib-1.3.0-12.el 68 kB/s | 54 kB 00:00 2024-09-11T23:06:51.854 INFO:teuthology.orchestra.run.smithi136.stdout:(93/156): python3-toml-0.10.2-6.el9.noarch.rpm 72 kB/s | 42 kB 00:00 2024-09-11T23:06:52.121 INFO:teuthology.orchestra.run.smithi136.stdout:(94/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 72 kB/s | 44 kB 00:00 2024-09-11T23:06:52.738 INFO:teuthology.orchestra.run.smithi136.stdout:(95/156): qatlib-service-24.02.0-1.el9.x86_64.r 58 kB/s | 36 kB 00:00 2024-09-11T23:06:52.763 INFO:teuthology.orchestra.run.smithi136.stdout:(96/156): qatlib-24.02.0-1.el9.x86_64.rpm 243 kB/s | 221 kB 00:00 2024-09-11T23:06:53.380 INFO:teuthology.orchestra.run.smithi136.stdout:(97/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 74 kB/s | 47 kB 00:00 2024-09-11T23:06:53.411 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction test succeeded. 2024-09-11T23:06:53.411 INFO:teuthology.orchestra.run.smithi026.stdout:Running transaction 2024-09-11T23:06:53.680 INFO:teuthology.orchestra.run.smithi136.stdout:(98/156): socat-1.7.4.1-6.el9.x86_64.rpm 332 kB/s | 304 kB 00:00 2024-09-11T23:06:54.038 INFO:teuthology.orchestra.run.smithi136.stdout:(99/156): uuid-1.6.2-55.el9.x86_64.rpm 88 kB/s | 58 kB 00:00 2024-09-11T23:06:54.439 INFO:teuthology.orchestra.run.smithi136.stdout:(100/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 126 kB/s | 96 kB 00:00 2024-09-11T23:06:54.689 INFO:teuthology.orchestra.run.smithi136.stdout:(101/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 4.4 MB/s | 16 MB 00:03 2024-09-11T23:06:54.706 INFO:teuthology.orchestra.run.smithi136.stdout:(102/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 95 kB/s | 64 kB 00:00 2024-09-11T23:06:54.789 INFO:teuthology.orchestra.run.smithi136.stdout:(103/156): lua-devel-5.4.4-4.el9.x86_64.rpm 224 kB/s | 22 kB 00:00 2024-09-11T23:06:54.840 INFO:teuthology.orchestra.run.smithi136.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 11 MB/s | 548 kB 00:00 2024-09-11T23:06:54.865 INFO:teuthology.orchestra.run.smithi136.stdout:(105/156): gperftools-libs-2.9.1-2.el9.x86_64.r 12 MB/s | 309 kB 00:00 2024-09-11T23:06:54.882 INFO:teuthology.orchestra.run.smithi136.stdout:(106/156): grpc-data-1.46.7-10.el9.noarch.rpm 1.2 MB/s | 19 kB 00:00 2024-09-11T23:06:54.991 INFO:teuthology.orchestra.run.smithi136.stdout:(107/156): libarrow-9.0.0-11.el9.x86_64.rpm 41 MB/s | 4.4 MB 00:00 2024-09-11T23:06:55.016 INFO:teuthology.orchestra.run.smithi136.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-09-11T23:06:55.049 INFO:teuthology.orchestra.run.smithi136.stdout:(109/156): protobuf-compiler-3.14.0-13.el9.x86_ 2.5 MB/s | 863 kB 00:00 2024-09-11T23:06:55.075 INFO:teuthology.orchestra.run.smithi136.stdout:(110/156): liboath-2.6.7-2.el9.x86_64.rpm 827 kB/s | 49 kB 00:00 2024-09-11T23:06:55.108 INFO:teuthology.orchestra.run.smithi136.stdout:(111/156): luarocks-3.9.2-5.el9.noarch.rpm 5.7 MB/s | 151 kB 00:00 2024-09-11T23:06:55.142 INFO:teuthology.orchestra.run.smithi136.stdout:(112/156): libunwind-1.6.2-1.el9.x86_64.rpm 732 kB/s | 67 kB 00:00 2024-09-11T23:06:55.176 INFO:teuthology.orchestra.run.smithi136.stdout:(113/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 13 MB/s | 839 kB 00:00 2024-09-11T23:06:55.201 INFO:teuthology.orchestra.run.smithi136.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-09-11T23:06:55.226 INFO:teuthology.orchestra.run.smithi136.stdout:(115/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 57 kB/s | 45 kB 00:00 2024-09-11T23:06:55.251 INFO:teuthology.orchestra.run.smithi136.stdout:(116/156): python3-cachetools-4.2.4-1.el9.noarc 653 kB/s | 32 kB 00:00 2024-09-11T23:06:55.296 INFO:teuthology.orchestra.run.smithi136.stdout:(117/156): python3-asyncssh-2.13.2-5.el9.noarch 3.5 MB/s | 548 kB 00:00 2024-09-11T23:06:55.319 INFO:teuthology.orchestra.run.smithi136.stdout:(118/156): python3-cheroot-8.6.0-4.el9.noarch.r 2.8 MB/s | 172 kB 00:00 2024-09-11T23:06:55.344 INFO:teuthology.orchestra.run.smithi136.stdout:(119/156): python3-certifi-2023.05.07-4.el9.noa 120 kB/s | 14 kB 00:00 2024-09-11T23:06:55.394 INFO:teuthology.orchestra.run.smithi136.stdout:(120/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.9 MB/s | 303 kB 00:00 2024-09-11T23:06:55.428 INFO:teuthology.orchestra.run.smithi136.stdout:(121/156): python3-cherrypy-18.6.1-2.el9.noarch 2.6 MB/s | 358 kB 00:00 2024-09-11T23:06:55.453 INFO:teuthology.orchestra.run.smithi136.stdout:(122/156): python3-google-auth-2.34.0-1.el9.noa 2.0 MB/s | 220 kB 00:00 2024-09-11T23:06:55.479 INFO:teuthology.orchestra.run.smithi136.stdout:(123/156): python3-grpcio-tools-1.46.7-10.el9.x 2.8 MB/s | 144 kB 00:00 2024-09-11T23:06:55.504 INFO:teuthology.orchestra.run.smithi136.stdout:(124/156): python3-jaraco-8.2.1-3.el9.noarch.rp 213 kB/s | 11 kB 00:00 2024-09-11T23:06:55.529 INFO:teuthology.orchestra.run.smithi136.stdout:(125/156): python3-jaraco-classes-3.2.1-5.el9.n 353 kB/s | 18 kB 00:00 2024-09-11T23:06:55.554 INFO:teuthology.orchestra.run.smithi136.stdout:(126/156): python3-jaraco-collections-3.0.0-8.e 463 kB/s | 23 kB 00:00 2024-09-11T23:06:55.580 INFO:teuthology.orchestra.run.smithi136.stdout:(127/156): python3-jaraco-functools-3.5.0-2.el9 388 kB/s | 19 kB 00:00 2024-09-11T23:06:55.605 INFO:teuthology.orchestra.run.smithi136.stdout:(128/156): python3-jaraco-text-3.2.0-6.el9.noar 391 kB/s | 20 kB 00:00 2024-09-11T23:06:55.630 INFO:teuthology.orchestra.run.smithi136.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-11T23:06:55.664 INFO:teuthology.orchestra.run.smithi136.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 2.3 MB/s | 79 kB 00:00 2024-09-11T23:06:55.689 INFO:teuthology.orchestra.run.smithi136.stdout:(131/156): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-09-11T23:06:55.723 INFO:teuthology.orchestra.run.smithi136.stdout:(132/156): python3-pecan-1.4.2-3.el9.noarch.rpm 8.0 MB/s | 272 kB 00:00 2024-09-11T23:06:55.765 INFO:teuthology.orchestra.run.smithi136.stdout:(133/156): python3-grpcio-1.46.7-10.el9.x86_64. 5.5 MB/s | 2.0 MB 00:00 2024-09-11T23:06:55.798 INFO:teuthology.orchestra.run.smithi136.stdout:(134/156): python3-kubernetes-26.1.0-2.el9.noar 4.7 MB/s | 1.0 MB 00:00 2024-09-11T23:06:55.824 INFO:teuthology.orchestra.run.smithi136.stdout:(135/156): python3-portend-3.1.0-2.el9.noarch.r 164 kB/s | 16 kB 00:00 2024-09-11T23:06:55.849 INFO:teuthology.orchestra.run.smithi136.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 565 kB/s | 28 kB 00:00 2024-09-11T23:06:55.874 INFO:teuthology.orchestra.run.smithi136.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 615 kB/s | 31 kB 00:00 2024-09-11T23:06:55.900 INFO:teuthology.orchestra.run.smithi136.stdout:(138/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 894 kB/s | 90 kB 00:00 2024-09-11T23:06:55.925 INFO:teuthology.orchestra.run.smithi136.stdout:(139/156): python3-rsa-4.9-2.el9.noarch.rpm 1.1 MB/s | 59 kB 00:00 2024-09-11T23:06:55.950 INFO:teuthology.orchestra.run.smithi136.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 715 kB/s | 36 kB 00:00 2024-09-11T23:06:55.975 INFO:teuthology.orchestra.run.smithi136.stdout:(141/156): python3-routes-2.5.1-5.el9.noarch.rp 1.5 MB/s | 188 kB 00:00 2024-09-11T23:06:56.064 INFO:teuthology.orchestra.run.smithi136.stdout:(142/156): python3-typing-extensions-4.12.2-2.e 538 kB/s | 74 kB 00:00 2024-09-11T23:06:56.209 INFO:teuthology.orchestra.run.smithi136.stdout:(143/156): python3-websocket-client-1.2.3-2.el9 384 kB/s | 90 kB 00:00 2024-09-11T23:06:56.226 INFO:teuthology.orchestra.run.smithi136.stdout:(144/156): python3-zc-lockfile-2.0-10.el9.noarc 1.2 MB/s | 20 kB 00:00 2024-09-11T23:06:56.251 INFO:teuthology.orchestra.run.smithi136.stdout:(145/156): python3-webob-1.8.8-2.el9.noarch.rpm 765 kB/s | 230 kB 00:00 2024-09-11T23:06:56.285 INFO:teuthology.orchestra.run.smithi136.stdout:(146/156): re2-20211101-20.el9.x86_64.rpm 3.2 MB/s | 191 kB 00:00 2024-09-11T23:06:56.310 INFO:teuthology.orchestra.run.smithi136.stdout:(147/156): python3-werkzeug-2.0.3-3.el9.1.noarc 1.7 MB/s | 427 kB 00:00 2024-09-11T23:06:56.361 INFO:teuthology.orchestra.run.smithi136.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-09-11T23:06:56.586 INFO:teuthology.orchestra.run.smithi136.stdout:(149/156): librados2-19.1.1-298.gb53b6d3e.el9.x 11 MB/s | 3.4 MB 00:00 2024-09-11T23:06:56.645 INFO:teuthology.orchestra.run.smithi136.stdout:(150/156): librbd1-19.1.1-298.gb53b6d3e.el9.x86 9.4 MB/s | 3.2 MB 00:00 2024-09-11T23:06:56.762 INFO:teuthology.orchestra.run.smithi136.stdout:(151/156): NetworkManager-team-1.49.90-1.el9.x8 349 kB/s | 41 kB 00:00 2024-09-11T23:06:56.812 INFO:teuthology.orchestra.run.smithi136.stdout:(152/156): NetworkManager-1.49.90-1.el9.x86_64. 5.1 MB/s | 2.3 MB 00:00 2024-09-11T23:06:56.887 INFO:teuthology.orchestra.run.smithi136.stdout:(153/156): NetworkManager-libnm-1.49.90-1.el9.x 6.0 MB/s | 1.8 MB 00:00 2024-09-11T23:06:56.913 INFO:teuthology.orchestra.run.smithi136.stdout:(154/156): NetworkManager-tui-1.49.90-1.el9.x86 1.6 MB/s | 249 kB 00:00 2024-09-11T23:06:56.955 INFO:teuthology.orchestra.run.smithi136.stdout:(155/156): openssl-3.2.2-6.el9.x86_64.rpm 9.5 MB/s | 1.4 MB 00:00 2024-09-11T23:06:57.030 INFO:teuthology.orchestra.run.smithi136.stdout:(156/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 17 MB/s | 2.4 MB 00:00 2024-09-11T23:06:57.034 INFO:teuthology.orchestra.run.smithi136.stdout:-------------------------------------------------------------------------------- 2024-09-11T23:06:57.034 INFO:teuthology.orchestra.run.smithi136.stdout:Total 9.6 MB/s | 226 MB 00:23 2024-09-11T23:06:57.072 INFO:teuthology.orchestra.run.smithi076.stdout: Preparing : 1/1 2024-09-11T23:06:57.189 INFO:teuthology.orchestra.run.smithi076.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-11T23:06:57.227 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-11T23:06:57.289 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-11T23:06:57.323 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-11T23:06:57.348 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-11T23:06:57.391 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-11T23:06:57.712 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-11T23:06:57.833 INFO:teuthology.orchestra.run.smithi076.stdout: Upgrading : librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-11T23:06:57.954 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-11T23:06:57.984 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-11T23:06:58.076 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-11T23:06:58.105 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 10/164 2024-09-11T23:06:58.227 INFO:teuthology.orchestra.run.smithi136.stdout:Running transaction check 2024-09-11T23:06:58.254 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-11T23:06:58.310 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-11T23:06:58.338 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-11T23:06:58.376 INFO:teuthology.orchestra.run.smithi136.stdout:Transaction check succeeded. 2024-09-11T23:06:58.377 INFO:teuthology.orchestra.run.smithi136.stdout:Running transaction test 2024-09-11T23:06:58.442 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-11T23:06:58.464 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 14/164 2024-09-11T23:06:58.527 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 14/164 2024-09-11T23:06:58.557 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-11T23:06:58.577 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-11T23:06:58.733 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-11T23:06:58.877 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-11T23:06:58.988 INFO:teuthology.orchestra.run.smithi076.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 17/164 2024-09-11T23:06:59.087 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-11T23:06:59.173 INFO:teuthology.orchestra.run.smithi076.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-11T23:06:59.404 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-11T23:06:59.516 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-11T23:06:59.618 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-11T23:06:59.654 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-11T23:06:59.689 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-11T23:06:59.771 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-11T23:06:59.831 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-11T23:06:59.863 INFO:teuthology.orchestra.run.smithi026.stdout: Preparing : 1/1 2024-09-11T23:06:59.871 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-11T23:06:59.915 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-11T23:06:59.947 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-11T23:06:59.975 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-11T23:06:59.980 INFO:teuthology.orchestra.run.smithi026.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-11T23:07:00.019 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-11T23:07:00.045 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-11T23:07:00.080 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-11T23:07:00.098 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-11T23:07:00.122 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-11T23:07:00.136 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-11T23:07:00.147 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-11T23:07:00.178 INFO:teuthology.orchestra.run.smithi136.stdout:Transaction test succeeded. 2024-09-11T23:07:00.178 INFO:teuthology.orchestra.run.smithi136.stdout:Running transaction 2024-09-11T23:07:00.180 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-11T23:07:00.199 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-11T23:07:00.258 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-11T23:07:00.303 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x 34/164 2024-09-11T23:07:00.354 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9 35/164 2024-09-11T23:07:00.383 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 36/164 2024-09-11T23:07:00.452 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-11T23:07:00.506 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-11T23:07:00.546 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-11T23:07:00.581 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-11T23:07:00.665 INFO:teuthology.orchestra.run.smithi026.stdout: Upgrading : librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-11T23:07:00.713 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-11T23:07:00.767 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-11T23:07:00.790 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-11T23:07:00.816 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-11T23:07:00.908 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-11T23:07:00.937 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 10/164 2024-09-11T23:07:01.139 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-11T23:07:01.201 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-11T23:07:01.228 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-11T23:07:01.301 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-11T23:07:01.334 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-11T23:07:01.363 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 14/164 2024-09-11T23:07:01.380 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-11T23:07:01.409 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-11T23:07:01.439 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 14/164 2024-09-11T23:07:01.451 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-11T23:07:01.463 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-11T23:07:01.489 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 46/164 2024-09-11T23:07:01.490 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-11T23:07:01.540 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-11T23:07:01.692 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-11T23:07:01.705 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-11T23:07:01.759 INFO:teuthology.orchestra.run.smithi076.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-11T23:07:01.822 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-11T23:07:01.866 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-11T23:07:01.895 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-11T23:07:01.927 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-11T23:07:01.963 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-11T23:07:02.003 INFO:teuthology.orchestra.run.smithi026.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 17/164 2024-09-11T23:07:02.009 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-11T23:07:02.062 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-11T23:07:02.105 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-11T23:07:02.129 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-11T23:07:02.147 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-11T23:07:02.213 INFO:teuthology.orchestra.run.smithi026.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-11T23:07:02.284 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-11T23:07:02.328 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-11T23:07:02.370 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-11T23:07:02.460 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-11T23:07:02.473 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-11T23:07:02.503 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-11T23:07:02.543 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-11T23:07:02.573 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-11T23:07:02.577 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-09-11T23:07:02.668 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-11T23:07:02.711 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-11T23:07:02.754 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-11T23:07:02.838 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-11T23:07:02.853 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-11T23:07:02.889 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-11T23:07:02.929 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-11T23:07:02.934 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-11T23:07:02.972 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-11T23:07:03.005 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-11T23:07:03.040 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-11T23:07:03.064 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-11T23:07:03.121 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-11T23:07:03.141 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-11T23:07:03.180 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-11T23:07:03.210 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-11T23:07:03.254 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-11T23:07:03.329 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-11T23:07:03.332 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-11T23:07:03.373 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x 34/164 2024-09-11T23:07:03.417 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9 35/164 2024-09-11T23:07:03.445 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 36/164 2024-09-11T23:07:03.500 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-11T23:07:03.518 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-11T23:07:03.568 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-11T23:07:03.594 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-11T23:07:03.634 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-11T23:07:03.646 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-11T23:07:03.664 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-11T23:07:03.698 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-11T23:07:03.740 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-11T23:07:03.782 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-11T23:07:03.790 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-11T23:07:03.816 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-11T23:07:03.846 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-11T23:07:03.903 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-11T23:07:03.947 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-11T23:07:03.978 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-11T23:07:04.081 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-11T23:07:04.137 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-11T23:07:04.403 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-11T23:07:04.493 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-11T23:07:04.528 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-11T23:07:04.580 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-11T23:07:04.626 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 46/164 2024-09-11T23:07:04.668 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-11T23:07:04.831 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-11T23:07:04.870 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-11T23:07:04.887 INFO:teuthology.orchestra.run.smithi026.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-11T23:07:04.926 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-11T23:07:04.943 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-11T23:07:04.960 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-11T23:07:05.024 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-11T23:07:05.060 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-11T23:07:05.071 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-11T23:07:05.107 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-11T23:07:05.116 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-11T23:07:05.155 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-11T23:07:05.160 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-11T23:07:05.190 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-11T23:07:05.234 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-11T23:07:05.248 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-11T23:07:05.292 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-11T23:07:05.304 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-11T23:07:05.434 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-11T23:07:05.480 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-11T23:07:05.489 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-11T23:07:05.522 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-11T23:07:05.527 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-11T23:07:05.605 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-11T23:07:05.636 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-11T23:07:05.681 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-11T23:07:05.729 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-11T23:07:05.762 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-09-11T23:07:05.787 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-11T23:07:05.888 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-11T23:07:06.018 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-11T23:07:06.250 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-11T23:07:06.253 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-11T23:07:06.376 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-11T23:07:06.424 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-11T23:07:06.465 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-11T23:07:06.515 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-11T23:07:06.562 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-11T23:07:06.591 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-11T23:07:06.626 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-11T23:07:06.659 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-11T23:07:06.702 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-11T23:07:06.735 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-11T23:07:06.764 INFO:teuthology.orchestra.run.smithi136.stdout: Preparing : 1/1 2024-09-11T23:07:06.830 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-11T23:07:06.883 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-11T23:07:06.913 INFO:teuthology.orchestra.run.smithi136.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-11T23:07:06.930 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-11T23:07:06.954 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-11T23:07:07.008 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-11T23:07:07.039 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-11T23:07:07.041 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-11T23:07:07.067 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-11T23:07:07.107 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-11T23:07:07.118 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-11T23:07:07.442 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-11T23:07:07.544 INFO:teuthology.orchestra.run.smithi136.stdout: Upgrading : librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-11T23:07:07.602 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-11T23:07:07.657 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-11T23:07:07.678 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-11T23:07:07.690 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-11T23:07:07.727 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-11T23:07:07.768 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-11T23:07:07.770 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-11T23:07:07.799 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 10/164 2024-09-11T23:07:07.875 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-11T23:07:07.936 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-11T23:07:07.946 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-11T23:07:07.970 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-11T23:07:07.996 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-11T23:07:08.024 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-11T23:07:08.072 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-11T23:07:08.078 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-11T23:07:08.111 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-11T23:07:08.122 INFO:teuthology.orchestra.run.smithi076.stdout: Upgrading : librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 101/164 2024-09-11T23:07:08.125 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-11T23:07:08.133 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 14/164 2024-09-11T23:07:08.170 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-11T23:07:08.187 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 14/164 2024-09-11T23:07:08.205 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 101/164 2024-09-11T23:07:08.218 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-11T23:07:08.244 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-11T23:07:08.260 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-11T23:07:08.314 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-11T23:07:08.336 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 102/164 2024-09-11T23:07:08.372 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-11T23:07:08.394 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-11T23:07:08.505 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-11T23:07:08.535 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-11T23:07:08.537 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-11T23:07:08.637 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-11T23:07:08.650 INFO:teuthology.orchestra.run.smithi136.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 17/164 2024-09-11T23:07:08.750 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-11T23:07:08.835 INFO:teuthology.orchestra.run.smithi136.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-11T23:07:08.853 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-11T23:07:08.901 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 105/164 2024-09-11T23:07:08.972 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 105/164 2024-09-11T23:07:09.011 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 106/164 2024-09-11T23:07:09.062 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-11T23:07:09.179 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-11T23:07:09.271 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-11T23:07:09.307 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-11T23:07:09.341 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-11T23:07:09.350 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-11T23:07:09.424 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-11T23:07:09.455 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-11T23:07:09.467 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-11T23:07:09.499 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-11T23:07:09.543 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-11T23:07:09.576 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-11T23:07:09.604 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-11T23:07:09.673 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-11T23:07:09.726 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-11T23:07:09.757 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-11T23:07:09.792 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-11T23:07:09.861 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-11T23:07:09.899 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x 34/164 2024-09-11T23:07:09.941 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9 35/164 2024-09-11T23:07:09.970 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 36/164 2024-09-11T23:07:10.038 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-11T23:07:10.110 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-11T23:07:10.183 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-11T23:07:10.314 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-11T23:07:10.362 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-11T23:07:10.772 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-11T23:07:10.865 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-11T23:07:10.885 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-11T23:07:10.893 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-11T23:07:10.926 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-11T23:07:10.967 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-11T23:07:10.996 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-11T23:07:11.030 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-11T23:07:11.069 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 46/164 2024-09-11T23:07:11.112 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-11T23:07:11.254 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-11T23:07:11.260 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-11T23:07:11.285 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-11T23:07:11.291 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-11T23:07:11.297 INFO:teuthology.orchestra.run.smithi026.stdout: Upgrading : librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 101/164 2024-09-11T23:07:11.338 INFO:teuthology.orchestra.run.smithi136.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-11T23:07:11.389 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 101/164 2024-09-11T23:07:11.392 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-11T23:07:11.418 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 102/164 2024-09-11T23:07:11.455 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-11T23:07:11.465 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-11T23:07:11.498 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-11T23:07:11.534 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-11T23:07:11.580 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-11T23:07:11.617 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-11T23:07:11.660 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-11T23:07:11.702 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-11T23:07:11.837 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-11T23:07:11.837 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-11T23:07:11.877 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-11T23:07:11.877 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 108/164 2024-09-11T23:07:11.918 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-11T23:07:11.970 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-11T23:07:11.970 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 108/164 2024-09-11T23:07:11.970 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-11T23:07:11.970 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-11T23:07:11.970 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-11T23:07:12.004 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 109/164 2024-09-11T23:07:12.008 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 105/164 2024-09-11T23:07:12.020 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-11T23:07:12.052 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-11T23:07:12.088 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 105/164 2024-09-11T23:07:12.092 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-11T23:07:12.117 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-09-11T23:07:12.118 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 106/164 2024-09-11T23:07:12.142 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-11T23:07:12.216 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-11T23:07:12.342 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-11T23:07:12.375 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-11T23:07:12.403 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-11T23:07:12.437 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-11T23:07:12.476 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-11T23:07:12.508 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-11T23:07:12.538 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-11T23:07:12.572 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-11T23:07:12.606 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-11T23:07:12.647 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-11T23:07:12.682 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-11T23:07:12.769 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-11T23:07:12.812 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-11T23:07:12.860 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-11T23:07:12.963 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-11T23:07:13.011 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-11T23:07:13.726 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-11T23:07:13.776 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-11T23:07:13.809 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-11T23:07:13.906 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-11T23:07:13.973 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-11T23:07:14.017 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-11T23:07:14.096 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-11T23:07:14.129 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-11T23:07:14.313 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-11T23:07:14.343 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-11T23:07:14.421 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-11T23:07:14.912 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-11T23:07:14.948 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-11T23:07:15.058 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-11T23:07:15.175 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-11T23:07:15.536 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-11T23:07:15.568 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 108/164 2024-09-11T23:07:15.664 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 108/164 2024-09-11T23:07:15.664 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-11T23:07:15.664 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-11T23:07:15.664 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-11T23:07:15.703 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 109/164 2024-09-11T23:07:16.391 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-11T23:07:16.465 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-11T23:07:16.493 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-11T23:07:16.526 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-11T23:07:16.837 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-11T23:07:16.881 INFO:teuthology.orchestra.run.smithi136.stdout: Upgrading : librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 101/164 2024-09-11T23:07:16.955 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 101/164 2024-09-11T23:07:16.986 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 102/164 2024-09-11T23:07:17.022 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-11T23:07:17.501 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-11T23:07:17.525 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 105/164 2024-09-11T23:07:17.580 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 105/164 2024-09-11T23:07:17.610 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 106/164 2024-09-11T23:07:19.948 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-11T23:07:19.982 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-11T23:07:20.524 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-11T23:07:20.569 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 108/164 2024-09-11T23:07:20.659 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 108/164 2024-09-11T23:07:20.659 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-11T23:07:20.659 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-11T23:07:20.660 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-09-11T23:07:20.687 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 109/164 2024-09-11T23:07:23.991 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 109/164 2024-09-11T23:07:23.992 INFO:teuthology.orchestra.run.smithi076.stdout:skipping the directory /sys 2024-09-11T23:07:23.992 INFO:teuthology.orchestra.run.smithi076.stdout:skipping the directory /proc 2024-09-11T23:07:23.992 INFO:teuthology.orchestra.run.smithi076.stdout:skipping the directory /mnt 2024-09-11T23:07:23.992 INFO:teuthology.orchestra.run.smithi076.stdout:skipping the directory /var/tmp 2024-09-11T23:07:23.992 INFO:teuthology.orchestra.run.smithi076.stdout:skipping the directory /home 2024-09-11T23:07:23.992 INFO:teuthology.orchestra.run.smithi076.stdout:skipping the directory /root 2024-09-11T23:07:23.992 INFO:teuthology.orchestra.run.smithi076.stdout:skipping the directory /tmp 2024-09-11T23:07:23.992 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-11T23:07:24.191 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 110/164 2024-09-11T23:07:24.209 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 110/164 2024-09-11T23:07:24.209 INFO:teuthology.orchestra.run.smithi076.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T23:07:24.209 INFO:teuthology.orchestra.run.smithi076.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-11T23:07:24.209 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-11T23:07:25.719 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 111/164 2024-09-11T23:07:25.746 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 111/164 2024-09-11T23:07:25.747 INFO:teuthology.orchestra.run.smithi076.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T23:07:25.747 INFO:teuthology.orchestra.run.smithi076.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-11T23:07:25.747 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-11T23:07:25.747 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-11T23:07:25.747 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-11T23:07:25.978 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 112/164 2024-09-11T23:07:26.002 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 112/164 2024-09-11T23:07:26.002 INFO:teuthology.orchestra.run.smithi076.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T23:07:26.002 INFO:teuthology.orchestra.run.smithi076.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-11T23:07:26.002 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-11T23:07:26.002 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-11T23:07:26.002 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-11T23:07:26.446 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 113/164 2024-09-11T23:07:26.470 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 113/164 2024-09-11T23:07:26.470 INFO:teuthology.orchestra.run.smithi076.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T23:07:26.470 INFO:teuthology.orchestra.run.smithi076.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-11T23:07:26.470 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-11T23:07:26.470 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-11T23:07:26.470 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-11T23:07:26.576 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-11T23:07:26.633 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-11T23:07:26.688 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-11T23:07:26.885 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-11T23:07:26.930 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-11T23:07:27.008 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-11T23:07:27.106 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-11T23:07:27.246 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-11T23:07:27.289 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-11T23:07:27.329 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-11T23:07:27.365 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-11T23:07:27.766 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 125/164 2024-09-11T23:07:27.797 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 125/164 2024-09-11T23:07:27.829 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el 126/164 2024-09-11T23:07:27.891 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.e 127/164 2024-09-11T23:07:27.928 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 128/164 2024-09-11T23:07:27.944 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 109/164 2024-09-11T23:07:27.945 INFO:teuthology.orchestra.run.smithi026.stdout:skipping the directory /sys 2024-09-11T23:07:27.945 INFO:teuthology.orchestra.run.smithi026.stdout:skipping the directory /proc 2024-09-11T23:07:27.945 INFO:teuthology.orchestra.run.smithi026.stdout:skipping the directory /mnt 2024-09-11T23:07:27.945 INFO:teuthology.orchestra.run.smithi026.stdout:skipping the directory /var/tmp 2024-09-11T23:07:27.945 INFO:teuthology.orchestra.run.smithi026.stdout:skipping the directory /home 2024-09-11T23:07:27.945 INFO:teuthology.orchestra.run.smithi026.stdout:skipping the directory /root 2024-09-11T23:07:27.945 INFO:teuthology.orchestra.run.smithi026.stdout:skipping the directory /tmp 2024-09-11T23:07:27.945 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-11T23:07:28.124 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 110/164 2024-09-11T23:07:28.141 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 110/164 2024-09-11T23:07:28.141 INFO:teuthology.orchestra.run.smithi026.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T23:07:28.141 INFO:teuthology.orchestra.run.smithi026.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-11T23:07:28.141 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-11T23:07:28.976 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 128/164 2024-09-11T23:07:29.014 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 129/164 2024-09-11T23:07:29.677 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 111/164 2024-09-11T23:07:29.705 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 111/164 2024-09-11T23:07:29.705 INFO:teuthology.orchestra.run.smithi026.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T23:07:29.705 INFO:teuthology.orchestra.run.smithi026.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-11T23:07:29.705 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-11T23:07:29.705 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-11T23:07:29.706 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-11T23:07:29.944 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 112/164 2024-09-11T23:07:29.969 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 112/164 2024-09-11T23:07:29.969 INFO:teuthology.orchestra.run.smithi026.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T23:07:29.969 INFO:teuthology.orchestra.run.smithi026.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-11T23:07:29.969 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-11T23:07:29.969 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-11T23:07:29.969 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-11T23:07:29.978 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 129/164 2024-09-11T23:07:30.006 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 130/164 2024-09-11T23:07:30.110 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 130/164 2024-09-11T23:07:30.272 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9 131/164 2024-09-11T23:07:30.308 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 132/164 2024-09-11T23:07:30.332 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 132/164 2024-09-11T23:07:30.332 INFO:teuthology.orchestra.run.smithi076.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T23:07:30.332 INFO:teuthology.orchestra.run.smithi076.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-11T23:07:30.333 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-11T23:07:30.333 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-11T23:07:30.333 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-11T23:07:30.384 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 133/164 2024-09-11T23:07:30.396 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 133/164 2024-09-11T23:07:30.428 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 113/164 2024-09-11T23:07:30.453 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 113/164 2024-09-11T23:07:30.453 INFO:teuthology.orchestra.run.smithi026.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T23:07:30.453 INFO:teuthology.orchestra.run.smithi026.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-11T23:07:30.453 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-11T23:07:30.453 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-11T23:07:30.453 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-11T23:07:30.493 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 134/164 2024-09-11T23:07:30.548 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 135/164 2024-09-11T23:07:30.559 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-11T23:07:30.617 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-11T23:07:30.689 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-11T23:07:30.902 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-11T23:07:30.954 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-11T23:07:31.035 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-11T23:07:31.133 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-11T23:07:31.284 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-11T23:07:31.338 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-11T23:07:31.369 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-11T23:07:31.397 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-11T23:07:31.477 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-11T23:07:31.520 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 137/164 2024-09-11T23:07:31.544 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 137/164 2024-09-11T23:07:31.544 INFO:teuthology.orchestra.run.smithi076.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T23:07:31.544 INFO:teuthology.orchestra.run.smithi076.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-11T23:07:31.544 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-11T23:07:31.545 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-11T23:07:31.545 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-11T23:07:31.587 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 138/164 2024-09-11T23:07:31.610 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 138/164 2024-09-11T23:07:31.610 INFO:teuthology.orchestra.run.smithi076.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T23:07:31.610 INFO:teuthology.orchestra.run.smithi076.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-11T23:07:31.610 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-11T23:07:31.663 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 139/164 2024-09-11T23:07:31.686 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 139/164 2024-09-11T23:07:31.686 INFO:teuthology.orchestra.run.smithi076.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T23:07:31.686 INFO:teuthology.orchestra.run.smithi076.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-11T23:07:31.686 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-11T23:07:31.686 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-11T23:07:31.687 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-11T23:07:31.858 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 125/164 2024-09-11T23:07:31.887 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 125/164 2024-09-11T23:07:31.919 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el 126/164 2024-09-11T23:07:31.984 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.e 127/164 2024-09-11T23:07:31.997 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 140/164 2024-09-11T23:07:32.018 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 128/164 2024-09-11T23:07:32.020 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 140/164 2024-09-11T23:07:32.021 INFO:teuthology.orchestra.run.smithi076.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T23:07:32.021 INFO:teuthology.orchestra.run.smithi076.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-11T23:07:32.021 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-11T23:07:32.021 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-11T23:07:32.021 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-11T23:07:32.468 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 109/164 2024-09-11T23:07:32.468 INFO:teuthology.orchestra.run.smithi136.stdout:skipping the directory /sys 2024-09-11T23:07:32.468 INFO:teuthology.orchestra.run.smithi136.stdout:skipping the directory /proc 2024-09-11T23:07:32.468 INFO:teuthology.orchestra.run.smithi136.stdout:skipping the directory /mnt 2024-09-11T23:07:32.468 INFO:teuthology.orchestra.run.smithi136.stdout:skipping the directory /var/tmp 2024-09-11T23:07:32.468 INFO:teuthology.orchestra.run.smithi136.stdout:skipping the directory /home 2024-09-11T23:07:32.468 INFO:teuthology.orchestra.run.smithi136.stdout:skipping the directory /root 2024-09-11T23:07:32.469 INFO:teuthology.orchestra.run.smithi136.stdout:skipping the directory /tmp 2024-09-11T23:07:32.469 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-09-11T23:07:32.641 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 110/164 2024-09-11T23:07:32.657 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 110/164 2024-09-11T23:07:32.657 INFO:teuthology.orchestra.run.smithi136.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T23:07:32.657 INFO:teuthology.orchestra.run.smithi136.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-11T23:07:32.657 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-09-11T23:07:33.119 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 128/164 2024-09-11T23:07:33.164 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 129/164 2024-09-11T23:07:34.163 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 111/164 2024-09-11T23:07:34.177 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 129/164 2024-09-11T23:07:34.190 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 111/164 2024-09-11T23:07:34.190 INFO:teuthology.orchestra.run.smithi136.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T23:07:34.190 INFO:teuthology.orchestra.run.smithi136.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-11T23:07:34.190 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-11T23:07:34.190 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-11T23:07:34.190 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-09-11T23:07:34.213 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 130/164 2024-09-11T23:07:34.322 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 130/164 2024-09-11T23:07:34.413 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 112/164 2024-09-11T23:07:34.437 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 112/164 2024-09-11T23:07:34.437 INFO:teuthology.orchestra.run.smithi136.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T23:07:34.437 INFO:teuthology.orchestra.run.smithi136.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-11T23:07:34.437 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-11T23:07:34.437 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-11T23:07:34.437 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-09-11T23:07:34.499 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9 131/164 2024-09-11T23:07:34.531 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 132/164 2024-09-11T23:07:34.557 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 132/164 2024-09-11T23:07:34.557 INFO:teuthology.orchestra.run.smithi026.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T23:07:34.557 INFO:teuthology.orchestra.run.smithi026.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-11T23:07:34.557 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-11T23:07:34.557 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-11T23:07:34.557 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-11T23:07:34.607 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 133/164 2024-09-11T23:07:34.619 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 133/164 2024-09-11T23:07:34.720 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 134/164 2024-09-11T23:07:34.776 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 135/164 2024-09-11T23:07:34.864 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 113/164 2024-09-11T23:07:34.887 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 113/164 2024-09-11T23:07:34.887 INFO:teuthology.orchestra.run.smithi136.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T23:07:34.888 INFO:teuthology.orchestra.run.smithi136.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-11T23:07:34.888 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-11T23:07:34.888 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-11T23:07:34.888 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-09-11T23:07:34.994 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-11T23:07:35.050 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-11T23:07:35.106 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-11T23:07:35.290 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-11T23:07:35.340 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-11T23:07:35.409 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-11T23:07:35.498 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-11T23:07:35.636 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-11T23:07:35.690 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-11T23:07:35.722 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-11T23:07:35.747 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-11T23:07:35.750 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-11T23:07:35.776 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 137/164 2024-09-11T23:07:35.801 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 137/164 2024-09-11T23:07:35.801 INFO:teuthology.orchestra.run.smithi026.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T23:07:35.801 INFO:teuthology.orchestra.run.smithi026.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-11T23:07:35.801 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-11T23:07:35.801 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-11T23:07:35.801 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-11T23:07:35.843 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 138/164 2024-09-11T23:07:35.868 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 138/164 2024-09-11T23:07:35.868 INFO:teuthology.orchestra.run.smithi026.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T23:07:35.868 INFO:teuthology.orchestra.run.smithi026.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-11T23:07:35.868 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-11T23:07:35.919 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 139/164 2024-09-11T23:07:35.943 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 139/164 2024-09-11T23:07:35.943 INFO:teuthology.orchestra.run.smithi026.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T23:07:35.944 INFO:teuthology.orchestra.run.smithi026.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-11T23:07:35.944 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-11T23:07:35.944 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-11T23:07:35.944 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-11T23:07:36.134 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 125/164 2024-09-11T23:07:36.176 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 125/164 2024-09-11T23:07:36.214 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el 126/164 2024-09-11T23:07:36.261 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 140/164 2024-09-11T23:07:36.276 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.e 127/164 2024-09-11T23:07:36.288 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 140/164 2024-09-11T23:07:36.288 INFO:teuthology.orchestra.run.smithi026.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T23:07:36.289 INFO:teuthology.orchestra.run.smithi026.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-11T23:07:36.289 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-11T23:07:36.289 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-11T23:07:36.289 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-11T23:07:36.305 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 128/164 2024-09-11T23:07:36.559 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 141/164 2024-09-11T23:07:36.600 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 142/164 2024-09-11T23:07:36.641 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 143/164 2024-09-11T23:07:36.673 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-11T23:07:36.700 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-11T23:07:36.883 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 146/164 2024-09-11T23:07:36.922 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-11T23:07:36.953 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_6 148/164 2024-09-11T23:07:36.983 INFO:teuthology.orchestra.run.smithi076.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 149/164 2024-09-11T23:07:37.029 INFO:teuthology.orchestra.run.smithi076.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 150/164 2024-09-11T23:07:37.344 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 128/164 2024-09-11T23:07:37.382 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 129/164 2024-09-11T23:07:37.649 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-11T23:07:37.685 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-11T23:07:37.705 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-11T23:07:38.346 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 129/164 2024-09-11T23:07:38.366 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 130/164 2024-09-11T23:07:38.468 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 130/164 2024-09-11T23:07:38.564 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 152/164 2024-09-11T23:07:38.628 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9 131/164 2024-09-11T23:07:38.649 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-11T23:07:38.668 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 132/164 2024-09-11T23:07:38.692 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 132/164 2024-09-11T23:07:38.693 INFO:teuthology.orchestra.run.smithi136.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T23:07:38.693 INFO:teuthology.orchestra.run.smithi136.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-11T23:07:38.693 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-11T23:07:38.693 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-11T23:07:38.693 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-09-11T23:07:38.697 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-11T23:07:38.744 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 133/164 2024-09-11T23:07:38.756 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 133/164 2024-09-11T23:07:38.790 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-11T23:07:38.826 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-11T23:07:38.852 INFO:teuthology.orchestra.run.smithi076.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-11T23:07:38.870 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 134/164 2024-09-11T23:07:38.878 INFO:teuthology.orchestra.run.smithi076.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-11T23:07:38.878 INFO:teuthology.orchestra.run.smithi076.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-11T23:07:38.918 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 135/164 2024-09-11T23:07:38.970 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-11T23:07:38.971 INFO:teuthology.orchestra.run.smithi076.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-11T23:07:39.038 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-11T23:07:39.063 INFO:teuthology.orchestra.run.smithi076.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-11T23:07:39.068 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-11T23:07:39.068 INFO:teuthology.orchestra.run.smithi076.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-11T23:07:39.256 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-11T23:07:39.257 INFO:teuthology.orchestra.run.smithi076.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-11T23:07:39.340 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-11T23:07:39.404 INFO:teuthology.orchestra.run.smithi076.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-11T23:07:39.841 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-11T23:07:39.871 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 137/164 2024-09-11T23:07:39.895 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 137/164 2024-09-11T23:07:39.895 INFO:teuthology.orchestra.run.smithi136.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T23:07:39.896 INFO:teuthology.orchestra.run.smithi136.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-11T23:07:39.896 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-11T23:07:39.896 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-11T23:07:39.896 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-09-11T23:07:39.939 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 138/164 2024-09-11T23:07:39.962 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 138/164 2024-09-11T23:07:39.962 INFO:teuthology.orchestra.run.smithi136.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T23:07:39.962 INFO:teuthology.orchestra.run.smithi136.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-11T23:07:39.962 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-09-11T23:07:40.014 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 139/164 2024-09-11T23:07:40.038 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 139/164 2024-09-11T23:07:40.038 INFO:teuthology.orchestra.run.smithi136.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T23:07:40.038 INFO:teuthology.orchestra.run.smithi136.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-11T23:07:40.038 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-11T23:07:40.039 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-11T23:07:40.039 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-09-11T23:07:40.340 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 140/164 2024-09-11T23:07:40.363 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 140/164 2024-09-11T23:07:40.363 INFO:teuthology.orchestra.run.smithi136.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T23:07:40.364 INFO:teuthology.orchestra.run.smithi136.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-11T23:07:40.364 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-11T23:07:40.364 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-11T23:07:40.364 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-09-11T23:07:41.124 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 141/164 2024-09-11T23:07:41.282 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 142/164 2024-09-11T23:07:41.323 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 143/164 2024-09-11T23:07:41.346 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-11T23:07:41.373 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-11T23:07:41.588 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 146/164 2024-09-11T23:07:41.628 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-11T23:07:41.658 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_6 148/164 2024-09-11T23:07:41.688 INFO:teuthology.orchestra.run.smithi026.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 149/164 2024-09-11T23:07:41.742 INFO:teuthology.orchestra.run.smithi026.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 150/164 2024-09-11T23:07:42.392 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-11T23:07:42.431 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-11T23:07:42.453 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-11T23:07:43.351 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 152/164 2024-09-11T23:07:43.436 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-11T23:07:43.489 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-11T23:07:43.586 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-11T23:07:43.614 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-11T23:07:43.640 INFO:teuthology.orchestra.run.smithi026.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-11T23:07:43.674 INFO:teuthology.orchestra.run.smithi026.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-11T23:07:43.674 INFO:teuthology.orchestra.run.smithi026.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-11T23:07:43.758 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-11T23:07:43.758 INFO:teuthology.orchestra.run.smithi026.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-11T23:07:43.825 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-11T23:07:43.850 INFO:teuthology.orchestra.run.smithi026.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-11T23:07:43.856 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-11T23:07:43.856 INFO:teuthology.orchestra.run.smithi026.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-11T23:07:44.044 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-11T23:07:44.044 INFO:teuthology.orchestra.run.smithi026.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-11T23:07:44.128 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-11T23:07:44.196 INFO:teuthology.orchestra.run.smithi026.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-11T23:07:44.911 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 141/164 2024-09-11T23:07:44.952 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 142/164 2024-09-11T23:07:44.993 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 143/164 2024-09-11T23:07:45.025 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-11T23:07:45.061 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-11T23:07:45.074 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-11T23:07:45.074 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/164 2024-09-11T23:07:45.074 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 2/164 2024-09-11T23:07:45.074 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 3/164 2024-09-11T23:07:45.074 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 4/164 2024-09-11T23:07:45.075 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 5/164 2024-09-11T23:07:45.075 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 6/164 2024-09-11T23:07:45.075 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 7/164 2024-09-11T23:07:45.075 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-11T23:07:45.075 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-11T23:07:45.075 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 10/164 2024-09-11T23:07:45.075 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 11/164 2024-09-11T23:07:45.075 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 12/164 2024-09-11T23:07:45.075 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-11T23:07:45.075 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 14/164 2024-09-11T23:07:45.075 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_6 15/164 2024-09-11T23:07:45.075 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 16/164 2024-09-11T23:07:45.075 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 17/164 2024-09-11T23:07:45.076 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 18/164 2024-09-11T23:07:45.076 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 19/164 2024-09-11T23:07:45.076 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 20/164 2024-09-11T23:07:45.076 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9 21/164 2024-09-11T23:07:45.076 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x 22/164 2024-09-11T23:07:45.076 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 23/164 2024-09-11T23:07:45.076 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 24/164 2024-09-11T23:07:45.076 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 25/164 2024-09-11T23:07:45.076 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 26/164 2024-09-11T23:07:45.076 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 27/164 2024-09-11T23:07:45.076 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 28/164 2024-09-11T23:07:45.076 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 29/164 2024-09-11T23:07:45.077 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.e 30/164 2024-09-11T23:07:45.089 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 31/164 2024-09-11T23:07:45.089 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 32/164 2024-09-11T23:07:45.089 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 33/164 2024-09-11T23:07:45.089 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9 34/164 2024-09-11T23:07:45.089 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 35/164 2024-09-11T23:07:45.089 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el 36/164 2024-09-11T23:07:45.093 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 37/164 2024-09-11T23:07:45.093 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 38/164 2024-09-11T23:07:45.093 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 39/164 2024-09-11T23:07:45.093 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-11T23:07:45.093 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-11T23:07:45.093 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-11T23:07:45.093 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-11T23:07:45.093 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-11T23:07:45.093 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-11T23:07:45.093 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-11T23:07:45.094 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-11T23:07:45.094 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-11T23:07:45.094 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-11T23:07:45.094 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-11T23:07:45.094 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-11T23:07:45.094 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-11T23:07:45.094 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-11T23:07:45.094 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-11T23:07:45.094 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-11T23:07:45.094 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-11T23:07:45.094 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-11T23:07:45.094 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-11T23:07:45.094 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-11T23:07:45.095 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-11T23:07:45.095 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-11T23:07:45.095 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-11T23:07:45.095 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-11T23:07:45.095 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-11T23:07:45.095 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-11T23:07:45.095 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-11T23:07:45.095 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-11T23:07:45.095 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-11T23:07:45.096 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-11T23:07:45.096 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-11T23:07:45.096 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-11T23:07:45.096 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-11T23:07:45.096 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-11T23:07:45.096 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-11T23:07:45.096 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-11T23:07:45.096 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-11T23:07:45.096 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-11T23:07:45.096 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-11T23:07:45.096 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-11T23:07:45.096 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-11T23:07:45.096 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-11T23:07:45.097 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-11T23:07:45.097 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-11T23:07:45.097 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-11T23:07:45.097 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-11T23:07:45.097 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-11T23:07:45.097 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-11T23:07:45.097 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-11T23:07:45.097 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-11T23:07:45.097 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-11T23:07:45.097 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-11T23:07:45.097 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-11T23:07:45.097 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-11T23:07:45.097 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-11T23:07:45.097 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-11T23:07:45.098 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-11T23:07:45.098 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-11T23:07:45.098 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-11T23:07:45.098 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-11T23:07:45.098 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-11T23:07:45.098 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-11T23:07:45.098 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-11T23:07:45.098 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-11T23:07:45.098 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-11T23:07:45.098 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-11T23:07:45.098 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-11T23:07:45.098 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/164 2024-09-11T23:07:45.098 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-11T23:07:45.098 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-11T23:07:45.098 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-11T23:07:45.099 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-11T23:07:45.099 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-11T23:07:45.099 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-11T23:07:45.099 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-11T23:07:45.099 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-11T23:07:45.099 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-11T23:07:45.099 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-11T23:07:45.099 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-11T23:07:45.099 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-11T23:07:45.099 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-11T23:07:45.099 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-11T23:07:45.099 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-11T23:07:45.099 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-11T23:07:45.099 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-11T23:07:45.100 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-11T23:07:45.100 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-11T23:07:45.100 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-11T23:07:45.100 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-11T23:07:45.100 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-11T23:07:45.100 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-11T23:07:45.100 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-11T23:07:45.100 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-11T23:07:45.100 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-11T23:07:45.100 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-11T23:07:45.100 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-11T23:07:45.100 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-11T23:07:45.100 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-11T23:07:45.100 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-11T23:07:45.101 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-11T23:07:45.101 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-11T23:07:45.101 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-11T23:07:45.101 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-11T23:07:45.101 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-11T23:07:45.101 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-11T23:07:45.101 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-11T23:07:45.101 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-11T23:07:45.101 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-11T23:07:45.101 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-11T23:07:45.101 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 149/164 2024-09-11T23:07:45.101 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-11T23:07:45.101 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 151/164 2024-09-11T23:07:45.101 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-11T23:07:45.102 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 153/164 2024-09-11T23:07:45.102 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-11T23:07:45.102 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 155/164 2024-09-11T23:07:45.102 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-11T23:07:45.102 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 157/164 2024-09-11T23:07:45.102 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-11T23:07:45.102 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 159/164 2024-09-11T23:07:45.102 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-11T23:07:45.102 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-11T23:07:45.102 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-11T23:07:45.102 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-11T23:07:45.244 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 146/164 2024-09-11T23:07:45.324 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-11T23:07:45.355 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_6 148/164 2024-09-11T23:07:45.384 INFO:teuthology.orchestra.run.smithi136.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 149/164 2024-09-11T23:07:45.422 INFO:teuthology.orchestra.run.smithi136.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 150/164 2024-09-11T23:07:46.037 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-11T23:07:46.079 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-11T23:07:46.099 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-11T23:07:46.928 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 152/164 2024-09-11T23:07:47.018 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-11T23:07:47.058 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-11T23:07:47.139 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-11T23:07:47.171 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-11T23:07:47.196 INFO:teuthology.orchestra.run.smithi136.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-11T23:07:47.223 INFO:teuthology.orchestra.run.smithi136.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-11T23:07:47.223 INFO:teuthology.orchestra.run.smithi136.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-11T23:07:47.306 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-11T23:07:47.307 INFO:teuthology.orchestra.run.smithi136.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-11T23:07:47.391 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-11T23:07:47.416 INFO:teuthology.orchestra.run.smithi136.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-11T23:07:47.421 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-11T23:07:47.421 INFO:teuthology.orchestra.run.smithi136.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-11T23:07:47.601 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-11T23:07:47.601 INFO:teuthology.orchestra.run.smithi136.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-11T23:07:47.693 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-11T23:07:47.728 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-11T23:07:47.728 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-11T23:07:47.728 INFO:teuthology.orchestra.run.smithi076.stdout:Upgraded: 2024-09-11T23:07:47.728 INFO:teuthology.orchestra.run.smithi076.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-11T23:07:47.728 INFO:teuthology.orchestra.run.smithi076.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-11T23:07:47.728 INFO:teuthology.orchestra.run.smithi076.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-11T23:07:47.728 INFO:teuthology.orchestra.run.smithi076.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-11T23:07:47.728 INFO:teuthology.orchestra.run.smithi076.stdout: librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:07:47.729 INFO:teuthology.orchestra.run.smithi076.stdout: librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:07:47.729 INFO:teuthology.orchestra.run.smithi076.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-11T23:07:47.729 INFO:teuthology.orchestra.run.smithi076.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-11T23:07:47.729 INFO:teuthology.orchestra.run.smithi076.stdout:Installed: 2024-09-11T23:07:47.729 INFO:teuthology.orchestra.run.smithi076.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-11T23:07:47.729 INFO:teuthology.orchestra.run.smithi076.stdout: automake-1.16.2-8.el9.noarch 2024-09-11T23:07:47.729 INFO:teuthology.orchestra.run.smithi076.stdout: bison-3.7.4-5.el9.x86_64 2024-09-11T23:07:47.729 INFO:teuthology.orchestra.run.smithi076.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-11T23:07:47.729 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:07:47.729 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:07:47.729 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:07:47.729 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:07:47.730 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T23:07:47.730 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-immutable-object-cache-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:07:47.730 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:07:47.730 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:07:47.730 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T23:07:47.730 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T23:07:47.730 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T23:07:47.730 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T23:07:47.730 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T23:07:47.730 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:07:47.730 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:07:47.730 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T23:07:47.730 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:07:47.731 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:07:47.731 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:07:47.731 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T23:07:47.731 INFO:teuthology.orchestra.run.smithi076.stdout: cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T23:07:47.731 INFO:teuthology.orchestra.run.smithi076.stdout: cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:07:47.731 INFO:teuthology.orchestra.run.smithi076.stdout: cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:07:47.731 INFO:teuthology.orchestra.run.smithi076.stdout: cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T23:07:47.731 INFO:teuthology.orchestra.run.smithi076.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-11T23:07:47.731 INFO:teuthology.orchestra.run.smithi076.stdout: flex-2.6.4-9.el9.x86_64 2024-09-11T23:07:47.731 INFO:teuthology.orchestra.run.smithi076.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-11T23:07:47.731 INFO:teuthology.orchestra.run.smithi076.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-11T23:07:47.731 INFO:teuthology.orchestra.run.smithi076.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-11T23:07:47.731 INFO:teuthology.orchestra.run.smithi076.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-11T23:07:47.731 INFO:teuthology.orchestra.run.smithi076.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-11T23:07:47.732 INFO:teuthology.orchestra.run.smithi076.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-11T23:07:47.732 INFO:teuthology.orchestra.run.smithi076.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-11T23:07:47.732 INFO:teuthology.orchestra.run.smithi076.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-11T23:07:47.732 INFO:teuthology.orchestra.run.smithi076.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-11T23:07:47.732 INFO:teuthology.orchestra.run.smithi076.stdout: libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:07:47.732 INFO:teuthology.orchestra.run.smithi076.stdout: libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:07:47.732 INFO:teuthology.orchestra.run.smithi076.stdout: libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:07:47.732 INFO:teuthology.orchestra.run.smithi076.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-11T23:07:47.732 INFO:teuthology.orchestra.run.smithi076.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-11T23:07:47.732 INFO:teuthology.orchestra.run.smithi076.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-11T23:07:47.732 INFO:teuthology.orchestra.run.smithi076.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-11T23:07:47.732 INFO:teuthology.orchestra.run.smithi076.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-11T23:07:47.733 INFO:teuthology.orchestra.run.smithi076.stdout: librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:07:47.733 INFO:teuthology.orchestra.run.smithi076.stdout: libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:07:47.733 INFO:teuthology.orchestra.run.smithi076.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-11T23:07:47.733 INFO:teuthology.orchestra.run.smithi076.stdout: librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:07:47.733 INFO:teuthology.orchestra.run.smithi076.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-11T23:07:47.733 INFO:teuthology.orchestra.run.smithi076.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-11T23:07:47.733 INFO:teuthology.orchestra.run.smithi076.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-11T23:07:47.733 INFO:teuthology.orchestra.run.smithi076.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-11T23:07:47.733 INFO:teuthology.orchestra.run.smithi076.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-11T23:07:47.733 INFO:teuthology.orchestra.run.smithi076.stdout: lua-5.4.4-4.el9.x86_64 2024-09-11T23:07:47.733 INFO:teuthology.orchestra.run.smithi076.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-11T23:07:47.733 INFO:teuthology.orchestra.run.smithi076.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-11T23:07:47.733 INFO:teuthology.orchestra.run.smithi076.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-11T23:07:47.733 INFO:teuthology.orchestra.run.smithi076.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-11T23:07:47.734 INFO:teuthology.orchestra.run.smithi076.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-11T23:07:47.734 INFO:teuthology.orchestra.run.smithi076.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-11T23:07:47.734 INFO:teuthology.orchestra.run.smithi076.stdout: postgresql-13.16-1.el9.x86_64 2024-09-11T23:07:47.734 INFO:teuthology.orchestra.run.smithi076.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-11T23:07:47.734 INFO:teuthology.orchestra.run.smithi076.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-11T23:07:47.734 INFO:teuthology.orchestra.run.smithi076.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-11T23:07:47.734 INFO:teuthology.orchestra.run.smithi076.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-11T23:07:47.734 INFO:teuthology.orchestra.run.smithi076.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-11T23:07:47.734 INFO:teuthology.orchestra.run.smithi076.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-11T23:07:47.734 INFO:teuthology.orchestra.run.smithi076.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-11T23:07:47.734 INFO:teuthology.orchestra.run.smithi076.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-11T23:07:47.734 INFO:teuthology.orchestra.run.smithi076.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-11T23:07:47.734 INFO:teuthology.orchestra.run.smithi076.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-11T23:07:47.734 INFO:teuthology.orchestra.run.smithi076.stdout: python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:07:47.734 INFO:teuthology.orchestra.run.smithi076.stdout: python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:07:47.735 INFO:teuthology.orchestra.run.smithi076.stdout: python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:07:47.735 INFO:teuthology.orchestra.run.smithi076.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-11T23:07:47.735 INFO:teuthology.orchestra.run.smithi076.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-11T23:07:47.735 INFO:teuthology.orchestra.run.smithi076.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-11T23:07:47.735 INFO:teuthology.orchestra.run.smithi076.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-11T23:07:47.735 INFO:teuthology.orchestra.run.smithi076.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-11T23:07:47.735 INFO:teuthology.orchestra.run.smithi076.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-11T23:07:47.735 INFO:teuthology.orchestra.run.smithi076.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-11T23:07:47.735 INFO:teuthology.orchestra.run.smithi076.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-11T23:07:47.735 INFO:teuthology.orchestra.run.smithi076.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-11T23:07:47.735 INFO:teuthology.orchestra.run.smithi076.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-11T23:07:47.735 INFO:teuthology.orchestra.run.smithi076.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-11T23:07:47.735 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-11T23:07:47.735 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-11T23:07:47.735 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-11T23:07:47.736 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-11T23:07:47.736 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-11T23:07:47.736 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-11T23:07:47.736 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-11T23:07:47.736 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-11T23:07:47.736 INFO:teuthology.orchestra.run.smithi076.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-11T23:07:47.736 INFO:teuthology.orchestra.run.smithi076.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-11T23:07:47.736 INFO:teuthology.orchestra.run.smithi076.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-11T23:07:47.736 INFO:teuthology.orchestra.run.smithi076.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-11T23:07:47.736 INFO:teuthology.orchestra.run.smithi076.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-11T23:07:47.736 INFO:teuthology.orchestra.run.smithi076.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-11T23:07:47.736 INFO:teuthology.orchestra.run.smithi076.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-11T23:07:47.737 INFO:teuthology.orchestra.run.smithi076.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-11T23:07:47.737 INFO:teuthology.orchestra.run.smithi076.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-11T23:07:47.737 INFO:teuthology.orchestra.run.smithi076.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-11T23:07:47.737 INFO:teuthology.orchestra.run.smithi076.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-11T23:07:47.737 INFO:teuthology.orchestra.run.smithi076.stdout: python3-ply-3.11-14.el9.noarch 2024-09-11T23:07:47.737 INFO:teuthology.orchestra.run.smithi076.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-11T23:07:47.737 INFO:teuthology.orchestra.run.smithi076.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-11T23:07:47.737 INFO:teuthology.orchestra.run.smithi076.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-11T23:07:47.737 INFO:teuthology.orchestra.run.smithi076.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-11T23:07:47.737 INFO:teuthology.orchestra.run.smithi076.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-11T23:07:47.737 INFO:teuthology.orchestra.run.smithi076.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-11T23:07:47.737 INFO:teuthology.orchestra.run.smithi076.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-11T23:07:47.737 INFO:teuthology.orchestra.run.smithi076.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-11T23:07:47.737 INFO:teuthology.orchestra.run.smithi076.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-11T23:07:47.738 INFO:teuthology.orchestra.run.smithi076.stdout: python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:07:47.738 INFO:teuthology.orchestra.run.smithi076.stdout: python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:07:47.738 INFO:teuthology.orchestra.run.smithi076.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-11T23:07:47.738 INFO:teuthology.orchestra.run.smithi076.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-11T23:07:47.738 INFO:teuthology.orchestra.run.smithi076.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-11T23:07:47.738 INFO:teuthology.orchestra.run.smithi076.stdout: python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:07:47.738 INFO:teuthology.orchestra.run.smithi076.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-11T23:07:47.738 INFO:teuthology.orchestra.run.smithi076.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-11T23:07:47.738 INFO:teuthology.orchestra.run.smithi076.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-11T23:07:47.738 INFO:teuthology.orchestra.run.smithi076.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-11T23:07:47.738 INFO:teuthology.orchestra.run.smithi076.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-11T23:07:47.738 INFO:teuthology.orchestra.run.smithi076.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-11T23:07:47.738 INFO:teuthology.orchestra.run.smithi076.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-11T23:07:47.738 INFO:teuthology.orchestra.run.smithi076.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-11T23:07:47.738 INFO:teuthology.orchestra.run.smithi076.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-11T23:07:47.739 INFO:teuthology.orchestra.run.smithi076.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-11T23:07:47.739 INFO:teuthology.orchestra.run.smithi076.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-11T23:07:47.739 INFO:teuthology.orchestra.run.smithi076.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-11T23:07:47.739 INFO:teuthology.orchestra.run.smithi076.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-11T23:07:47.739 INFO:teuthology.orchestra.run.smithi076.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-11T23:07:47.739 INFO:teuthology.orchestra.run.smithi076.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-11T23:07:47.739 INFO:teuthology.orchestra.run.smithi076.stdout: rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:07:47.739 INFO:teuthology.orchestra.run.smithi076.stdout: rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:07:47.739 INFO:teuthology.orchestra.run.smithi076.stdout: rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:07:47.739 INFO:teuthology.orchestra.run.smithi076.stdout: re2-1:20211101-20.el9.x86_64 2024-09-11T23:07:47.739 INFO:teuthology.orchestra.run.smithi076.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-11T23:07:47.739 INFO:teuthology.orchestra.run.smithi076.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-11T23:07:47.739 INFO:teuthology.orchestra.run.smithi076.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-11T23:07:47.739 INFO:teuthology.orchestra.run.smithi076.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-11T23:07:47.740 INFO:teuthology.orchestra.run.smithi076.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-11T23:07:47.740 INFO:teuthology.orchestra.run.smithi076.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-11T23:07:47.740 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-11T23:07:47.740 INFO:teuthology.orchestra.run.smithi076.stdout:Complete! 2024-09-11T23:07:47.756 INFO:teuthology.orchestra.run.smithi136.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-11T23:07:48.139 DEBUG:teuthology.parallel:result is None 2024-09-11T23:07:49.898 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-11T23:07:49.898 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/164 2024-09-11T23:07:49.898 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 2/164 2024-09-11T23:07:49.898 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 3/164 2024-09-11T23:07:49.898 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 4/164 2024-09-11T23:07:49.898 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 5/164 2024-09-11T23:07:49.898 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 6/164 2024-09-11T23:07:49.898 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 7/164 2024-09-11T23:07:49.898 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-11T23:07:49.898 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-11T23:07:49.899 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 10/164 2024-09-11T23:07:49.899 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 11/164 2024-09-11T23:07:49.899 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 12/164 2024-09-11T23:07:49.899 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-11T23:07:49.899 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 14/164 2024-09-11T23:07:49.899 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_6 15/164 2024-09-11T23:07:49.899 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 16/164 2024-09-11T23:07:49.899 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 17/164 2024-09-11T23:07:49.899 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 18/164 2024-09-11T23:07:49.899 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 19/164 2024-09-11T23:07:49.899 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 20/164 2024-09-11T23:07:49.899 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9 21/164 2024-09-11T23:07:49.899 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x 22/164 2024-09-11T23:07:49.900 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 23/164 2024-09-11T23:07:49.900 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 24/164 2024-09-11T23:07:49.900 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 25/164 2024-09-11T23:07:49.900 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 26/164 2024-09-11T23:07:49.900 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 27/164 2024-09-11T23:07:49.900 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 28/164 2024-09-11T23:07:49.900 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 29/164 2024-09-11T23:07:49.912 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.e 30/164 2024-09-11T23:07:49.912 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 31/164 2024-09-11T23:07:49.912 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 32/164 2024-09-11T23:07:49.912 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 33/164 2024-09-11T23:07:49.913 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9 34/164 2024-09-11T23:07:49.913 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 35/164 2024-09-11T23:07:49.913 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el 36/164 2024-09-11T23:07:49.913 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 37/164 2024-09-11T23:07:49.913 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 38/164 2024-09-11T23:07:49.913 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 39/164 2024-09-11T23:07:49.913 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-11T23:07:49.913 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-11T23:07:49.913 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-11T23:07:49.913 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-11T23:07:49.913 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-11T23:07:49.913 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-11T23:07:49.914 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-11T23:07:49.914 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-11T23:07:49.914 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-11T23:07:49.914 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-11T23:07:49.914 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-11T23:07:49.914 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-11T23:07:49.914 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-11T23:07:49.914 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-11T23:07:49.914 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-11T23:07:49.914 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-11T23:07:49.914 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-11T23:07:49.914 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-11T23:07:49.914 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-11T23:07:49.915 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-11T23:07:49.915 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-11T23:07:49.915 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-11T23:07:49.915 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-11T23:07:49.915 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-11T23:07:49.915 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-11T23:07:49.915 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-11T23:07:49.915 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-11T23:07:49.915 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-11T23:07:49.915 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-11T23:07:49.915 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-11T23:07:49.915 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-11T23:07:49.915 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-11T23:07:49.916 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-11T23:07:49.916 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-11T23:07:49.916 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-11T23:07:49.916 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-11T23:07:49.916 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-11T23:07:49.916 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-11T23:07:49.916 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-11T23:07:49.916 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-11T23:07:49.916 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-11T23:07:49.916 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-11T23:07:49.916 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-11T23:07:49.916 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-11T23:07:49.916 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-11T23:07:49.917 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-11T23:07:49.917 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-11T23:07:49.917 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-11T23:07:49.917 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-11T23:07:49.917 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-11T23:07:49.917 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-11T23:07:49.917 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-11T23:07:49.917 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-11T23:07:49.917 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-11T23:07:49.917 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-11T23:07:49.917 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-11T23:07:49.917 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-11T23:07:49.918 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-11T23:07:49.918 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-11T23:07:49.918 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-11T23:07:49.918 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-11T23:07:49.918 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-11T23:07:49.918 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-11T23:07:49.918 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-11T23:07:49.918 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-11T23:07:49.918 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-11T23:07:49.918 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-11T23:07:49.918 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/164 2024-09-11T23:07:49.918 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-11T23:07:49.918 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-11T23:07:49.919 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-11T23:07:49.919 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-11T23:07:49.919 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-11T23:07:49.919 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-11T23:07:49.919 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-11T23:07:49.919 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-11T23:07:49.919 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-11T23:07:49.919 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-11T23:07:49.919 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-11T23:07:49.919 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-11T23:07:49.919 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-11T23:07:49.919 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-11T23:07:49.919 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-11T23:07:49.919 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-11T23:07:49.920 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-11T23:07:49.920 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-11T23:07:49.920 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-11T23:07:49.920 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-11T23:07:49.920 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-11T23:07:49.920 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-11T23:07:49.920 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-11T23:07:49.920 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-11T23:07:49.920 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-11T23:07:49.920 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-11T23:07:49.920 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-11T23:07:49.920 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-11T23:07:49.921 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-11T23:07:49.921 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-11T23:07:49.921 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-11T23:07:49.921 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-11T23:07:49.921 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-11T23:07:49.921 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-11T23:07:49.921 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-11T23:07:49.921 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-11T23:07:49.921 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-11T23:07:49.921 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-11T23:07:49.921 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-11T23:07:49.921 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-11T23:07:49.921 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-11T23:07:49.922 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 149/164 2024-09-11T23:07:49.922 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-11T23:07:49.922 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 151/164 2024-09-11T23:07:49.922 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-11T23:07:49.922 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 153/164 2024-09-11T23:07:49.922 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-11T23:07:49.922 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 155/164 2024-09-11T23:07:49.922 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-11T23:07:49.922 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 157/164 2024-09-11T23:07:49.922 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-11T23:07:49.922 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 159/164 2024-09-11T23:07:49.922 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-11T23:07:49.923 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-11T23:07:49.923 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-11T23:07:49.923 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-11T23:07:52.688 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-11T23:07:52.688 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-11T23:07:52.688 INFO:teuthology.orchestra.run.smithi026.stdout:Upgraded: 2024-09-11T23:07:52.688 INFO:teuthology.orchestra.run.smithi026.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-11T23:07:52.688 INFO:teuthology.orchestra.run.smithi026.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-11T23:07:52.688 INFO:teuthology.orchestra.run.smithi026.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-11T23:07:52.688 INFO:teuthology.orchestra.run.smithi026.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-11T23:07:52.688 INFO:teuthology.orchestra.run.smithi026.stdout: librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:07:52.688 INFO:teuthology.orchestra.run.smithi026.stdout: librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:07:52.689 INFO:teuthology.orchestra.run.smithi026.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-11T23:07:52.689 INFO:teuthology.orchestra.run.smithi026.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-11T23:07:52.689 INFO:teuthology.orchestra.run.smithi026.stdout:Installed: 2024-09-11T23:07:52.689 INFO:teuthology.orchestra.run.smithi026.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-11T23:07:52.689 INFO:teuthology.orchestra.run.smithi026.stdout: automake-1.16.2-8.el9.noarch 2024-09-11T23:07:52.689 INFO:teuthology.orchestra.run.smithi026.stdout: bison-3.7.4-5.el9.x86_64 2024-09-11T23:07:52.689 INFO:teuthology.orchestra.run.smithi026.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-11T23:07:52.689 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:07:52.689 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:07:52.689 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:07:52.689 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:07:52.689 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T23:07:52.689 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-immutable-object-cache-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:07:52.690 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:07:52.690 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:07:52.690 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T23:07:52.690 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T23:07:52.690 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T23:07:52.690 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T23:07:52.690 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T23:07:52.690 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:07:52.690 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:07:52.690 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T23:07:52.690 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:07:52.691 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:07:52.691 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:07:52.691 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T23:07:52.691 INFO:teuthology.orchestra.run.smithi026.stdout: cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T23:07:52.691 INFO:teuthology.orchestra.run.smithi026.stdout: cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:07:52.691 INFO:teuthology.orchestra.run.smithi026.stdout: cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:07:52.691 INFO:teuthology.orchestra.run.smithi026.stdout: cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T23:07:52.691 INFO:teuthology.orchestra.run.smithi026.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-11T23:07:52.691 INFO:teuthology.orchestra.run.smithi026.stdout: flex-2.6.4-9.el9.x86_64 2024-09-11T23:07:52.691 INFO:teuthology.orchestra.run.smithi026.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-11T23:07:52.691 INFO:teuthology.orchestra.run.smithi026.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-11T23:07:52.691 INFO:teuthology.orchestra.run.smithi026.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-11T23:07:52.691 INFO:teuthology.orchestra.run.smithi026.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-11T23:07:52.691 INFO:teuthology.orchestra.run.smithi026.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-11T23:07:52.692 INFO:teuthology.orchestra.run.smithi026.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-11T23:07:52.692 INFO:teuthology.orchestra.run.smithi026.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-11T23:07:52.692 INFO:teuthology.orchestra.run.smithi026.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-11T23:07:52.692 INFO:teuthology.orchestra.run.smithi026.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-11T23:07:52.692 INFO:teuthology.orchestra.run.smithi026.stdout: libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:07:52.692 INFO:teuthology.orchestra.run.smithi026.stdout: libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:07:52.692 INFO:teuthology.orchestra.run.smithi026.stdout: libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:07:52.692 INFO:teuthology.orchestra.run.smithi026.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-11T23:07:52.692 INFO:teuthology.orchestra.run.smithi026.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-11T23:07:52.692 INFO:teuthology.orchestra.run.smithi026.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-11T23:07:52.692 INFO:teuthology.orchestra.run.smithi026.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-11T23:07:52.692 INFO:teuthology.orchestra.run.smithi026.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-11T23:07:52.692 INFO:teuthology.orchestra.run.smithi026.stdout: librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:07:52.693 INFO:teuthology.orchestra.run.smithi026.stdout: libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:07:52.693 INFO:teuthology.orchestra.run.smithi026.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-11T23:07:52.693 INFO:teuthology.orchestra.run.smithi026.stdout: librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:07:52.693 INFO:teuthology.orchestra.run.smithi026.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-11T23:07:52.693 INFO:teuthology.orchestra.run.smithi026.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-11T23:07:52.693 INFO:teuthology.orchestra.run.smithi026.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-11T23:07:52.693 INFO:teuthology.orchestra.run.smithi026.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-11T23:07:52.693 INFO:teuthology.orchestra.run.smithi026.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-11T23:07:52.693 INFO:teuthology.orchestra.run.smithi026.stdout: lua-5.4.4-4.el9.x86_64 2024-09-11T23:07:52.693 INFO:teuthology.orchestra.run.smithi026.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-11T23:07:52.693 INFO:teuthology.orchestra.run.smithi026.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-11T23:07:52.693 INFO:teuthology.orchestra.run.smithi026.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-11T23:07:52.693 INFO:teuthology.orchestra.run.smithi026.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-11T23:07:52.694 INFO:teuthology.orchestra.run.smithi026.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-11T23:07:52.694 INFO:teuthology.orchestra.run.smithi026.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-11T23:07:52.694 INFO:teuthology.orchestra.run.smithi026.stdout: postgresql-13.16-1.el9.x86_64 2024-09-11T23:07:52.694 INFO:teuthology.orchestra.run.smithi026.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-11T23:07:52.694 INFO:teuthology.orchestra.run.smithi026.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-11T23:07:52.694 INFO:teuthology.orchestra.run.smithi026.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-11T23:07:52.694 INFO:teuthology.orchestra.run.smithi026.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-11T23:07:52.694 INFO:teuthology.orchestra.run.smithi026.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-11T23:07:52.694 INFO:teuthology.orchestra.run.smithi026.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-11T23:07:52.694 INFO:teuthology.orchestra.run.smithi026.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-11T23:07:52.694 INFO:teuthology.orchestra.run.smithi026.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-11T23:07:52.694 INFO:teuthology.orchestra.run.smithi026.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-11T23:07:52.694 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-11T23:07:52.695 INFO:teuthology.orchestra.run.smithi026.stdout: python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:07:52.695 INFO:teuthology.orchestra.run.smithi026.stdout: python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:07:52.695 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:07:52.695 INFO:teuthology.orchestra.run.smithi026.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-11T23:07:52.695 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-11T23:07:52.695 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-11T23:07:52.695 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-11T23:07:52.695 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-11T23:07:52.695 INFO:teuthology.orchestra.run.smithi026.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-11T23:07:52.695 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-11T23:07:52.695 INFO:teuthology.orchestra.run.smithi026.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-11T23:07:52.695 INFO:teuthology.orchestra.run.smithi026.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-11T23:07:52.695 INFO:teuthology.orchestra.run.smithi026.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-11T23:07:52.696 INFO:teuthology.orchestra.run.smithi026.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-11T23:07:52.696 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-11T23:07:52.696 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-11T23:07:52.696 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-11T23:07:52.696 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-11T23:07:52.696 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-11T23:07:52.696 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-11T23:07:52.696 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-11T23:07:52.696 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-11T23:07:52.696 INFO:teuthology.orchestra.run.smithi026.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-11T23:07:52.696 INFO:teuthology.orchestra.run.smithi026.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-11T23:07:52.696 INFO:teuthology.orchestra.run.smithi026.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-11T23:07:52.696 INFO:teuthology.orchestra.run.smithi026.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-11T23:07:52.697 INFO:teuthology.orchestra.run.smithi026.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-11T23:07:52.697 INFO:teuthology.orchestra.run.smithi026.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-11T23:07:52.697 INFO:teuthology.orchestra.run.smithi026.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-11T23:07:52.697 INFO:teuthology.orchestra.run.smithi026.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-11T23:07:52.697 INFO:teuthology.orchestra.run.smithi026.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-11T23:07:52.697 INFO:teuthology.orchestra.run.smithi026.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-11T23:07:52.697 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-11T23:07:52.697 INFO:teuthology.orchestra.run.smithi026.stdout: python3-ply-3.11-14.el9.noarch 2024-09-11T23:07:52.697 INFO:teuthology.orchestra.run.smithi026.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-11T23:07:52.697 INFO:teuthology.orchestra.run.smithi026.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-11T23:07:52.697 INFO:teuthology.orchestra.run.smithi026.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-11T23:07:52.697 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-11T23:07:52.698 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-11T23:07:52.698 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-11T23:07:52.698 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-11T23:07:52.698 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-11T23:07:52.698 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-11T23:07:52.698 INFO:teuthology.orchestra.run.smithi026.stdout: python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:07:52.698 INFO:teuthology.orchestra.run.smithi026.stdout: python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:07:52.698 INFO:teuthology.orchestra.run.smithi026.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-11T23:07:52.698 INFO:teuthology.orchestra.run.smithi026.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-11T23:07:52.698 INFO:teuthology.orchestra.run.smithi026.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-11T23:07:52.698 INFO:teuthology.orchestra.run.smithi026.stdout: python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:07:52.698 INFO:teuthology.orchestra.run.smithi026.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-11T23:07:52.698 INFO:teuthology.orchestra.run.smithi026.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-11T23:07:52.698 INFO:teuthology.orchestra.run.smithi026.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-11T23:07:52.699 INFO:teuthology.orchestra.run.smithi026.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-11T23:07:52.699 INFO:teuthology.orchestra.run.smithi026.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-11T23:07:52.699 INFO:teuthology.orchestra.run.smithi026.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-11T23:07:52.699 INFO:teuthology.orchestra.run.smithi026.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-11T23:07:52.699 INFO:teuthology.orchestra.run.smithi026.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-11T23:07:52.699 INFO:teuthology.orchestra.run.smithi026.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-11T23:07:52.699 INFO:teuthology.orchestra.run.smithi026.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-11T23:07:52.699 INFO:teuthology.orchestra.run.smithi026.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-11T23:07:52.699 INFO:teuthology.orchestra.run.smithi026.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-11T23:07:52.699 INFO:teuthology.orchestra.run.smithi026.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-11T23:07:52.699 INFO:teuthology.orchestra.run.smithi026.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-11T23:07:52.699 INFO:teuthology.orchestra.run.smithi026.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-11T23:07:52.700 INFO:teuthology.orchestra.run.smithi026.stdout: rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:07:52.700 INFO:teuthology.orchestra.run.smithi026.stdout: rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:07:52.700 INFO:teuthology.orchestra.run.smithi026.stdout: rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:07:52.700 INFO:teuthology.orchestra.run.smithi026.stdout: re2-1:20211101-20.el9.x86_64 2024-09-11T23:07:52.700 INFO:teuthology.orchestra.run.smithi026.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-11T23:07:52.700 INFO:teuthology.orchestra.run.smithi026.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-11T23:07:52.700 INFO:teuthology.orchestra.run.smithi026.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-11T23:07:52.700 INFO:teuthology.orchestra.run.smithi026.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-11T23:07:52.700 INFO:teuthology.orchestra.run.smithi026.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-11T23:07:52.701 INFO:teuthology.orchestra.run.smithi026.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-11T23:07:52.701 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-11T23:07:52.701 INFO:teuthology.orchestra.run.smithi026.stdout:Complete! 2024-09-11T23:07:53.109 DEBUG:teuthology.parallel:result is None 2024-09-11T23:07:53.378 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-11T23:07:53.378 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/164 2024-09-11T23:07:53.378 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 2/164 2024-09-11T23:07:53.378 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 3/164 2024-09-11T23:07:53.378 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 4/164 2024-09-11T23:07:53.378 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 5/164 2024-09-11T23:07:53.378 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 6/164 2024-09-11T23:07:53.378 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 7/164 2024-09-11T23:07:53.379 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-11T23:07:53.379 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-11T23:07:53.379 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 10/164 2024-09-11T23:07:53.379 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 11/164 2024-09-11T23:07:53.379 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 12/164 2024-09-11T23:07:53.379 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-11T23:07:53.379 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 14/164 2024-09-11T23:07:53.379 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_6 15/164 2024-09-11T23:07:53.379 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 16/164 2024-09-11T23:07:53.379 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 17/164 2024-09-11T23:07:53.379 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 18/164 2024-09-11T23:07:53.379 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 19/164 2024-09-11T23:07:53.379 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 20/164 2024-09-11T23:07:53.380 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9 21/164 2024-09-11T23:07:53.380 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x 22/164 2024-09-11T23:07:53.380 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 23/164 2024-09-11T23:07:53.380 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 24/164 2024-09-11T23:07:53.380 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 25/164 2024-09-11T23:07:53.380 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 26/164 2024-09-11T23:07:53.380 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 27/164 2024-09-11T23:07:53.380 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 28/164 2024-09-11T23:07:53.380 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 29/164 2024-09-11T23:07:53.380 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.e 30/164 2024-09-11T23:07:53.393 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 31/164 2024-09-11T23:07:53.393 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 32/164 2024-09-11T23:07:53.393 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 33/164 2024-09-11T23:07:53.393 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9 34/164 2024-09-11T23:07:53.393 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 35/164 2024-09-11T23:07:53.393 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el 36/164 2024-09-11T23:07:53.393 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 37/164 2024-09-11T23:07:53.393 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 38/164 2024-09-11T23:07:53.393 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 39/164 2024-09-11T23:07:53.394 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-11T23:07:53.394 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-11T23:07:53.394 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-11T23:07:53.394 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-11T23:07:53.394 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-11T23:07:53.394 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-11T23:07:53.394 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-11T23:07:53.394 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-11T23:07:53.394 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-11T23:07:53.394 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-11T23:07:53.394 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-11T23:07:53.394 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-11T23:07:53.395 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-11T23:07:53.395 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-11T23:07:53.395 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-11T23:07:53.395 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-11T23:07:53.395 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-11T23:07:53.395 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-11T23:07:53.395 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-11T23:07:53.395 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-11T23:07:53.395 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-11T23:07:53.395 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-11T23:07:53.395 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-11T23:07:53.395 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-11T23:07:53.395 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-11T23:07:53.395 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-11T23:07:53.396 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-11T23:07:53.396 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-11T23:07:53.396 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-11T23:07:53.396 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-11T23:07:53.396 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-11T23:07:53.396 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-11T23:07:53.396 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-11T23:07:53.396 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-11T23:07:53.396 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-11T23:07:53.396 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-11T23:07:53.396 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-11T23:07:53.396 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-11T23:07:53.396 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-11T23:07:53.397 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-11T23:07:53.397 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-11T23:07:53.397 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-11T23:07:53.397 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-11T23:07:53.397 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-11T23:07:53.397 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-11T23:07:53.397 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-11T23:07:53.397 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-11T23:07:53.397 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-11T23:07:53.397 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-11T23:07:53.397 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-11T23:07:53.397 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-11T23:07:53.398 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-11T23:07:53.398 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-11T23:07:53.398 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-11T23:07:53.398 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-11T23:07:53.398 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-11T23:07:53.398 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-11T23:07:53.398 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-11T23:07:53.398 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-11T23:07:53.398 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-11T23:07:53.398 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-11T23:07:53.398 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-11T23:07:53.398 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-11T23:07:53.398 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-11T23:07:53.399 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-11T23:07:53.399 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-11T23:07:53.399 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-11T23:07:53.399 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/164 2024-09-11T23:07:53.399 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-11T23:07:53.399 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-11T23:07:53.399 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-11T23:07:53.399 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-11T23:07:53.399 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-11T23:07:53.399 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-11T23:07:53.399 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-11T23:07:53.399 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-11T23:07:53.399 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-11T23:07:53.400 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-11T23:07:53.400 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-11T23:07:53.400 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-11T23:07:53.400 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-11T23:07:53.400 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-11T23:07:53.400 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-11T23:07:53.400 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-11T23:07:53.400 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-11T23:07:53.400 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-11T23:07:53.400 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-11T23:07:53.400 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-11T23:07:53.400 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-11T23:07:53.400 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-11T23:07:53.401 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-11T23:07:53.401 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-11T23:07:53.401 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-11T23:07:53.401 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-11T23:07:53.401 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-11T23:07:53.401 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-11T23:07:53.401 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-11T23:07:53.401 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-11T23:07:53.401 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-11T23:07:53.401 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-11T23:07:53.401 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-11T23:07:53.401 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-11T23:07:53.401 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-11T23:07:53.402 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-11T23:07:53.402 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-11T23:07:53.402 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-11T23:07:53.402 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-11T23:07:53.402 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-11T23:07:53.402 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-11T23:07:53.402 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 149/164 2024-09-11T23:07:53.402 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-11T23:07:53.402 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 151/164 2024-09-11T23:07:53.402 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-11T23:07:53.402 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 153/164 2024-09-11T23:07:53.402 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-11T23:07:53.402 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 155/164 2024-09-11T23:07:53.402 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-11T23:07:53.403 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 157/164 2024-09-11T23:07:53.403 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-11T23:07:53.403 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 159/164 2024-09-11T23:07:53.403 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-11T23:07:53.403 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-11T23:07:53.403 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-11T23:07:53.403 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-11T23:07:56.286 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-11T23:07:56.286 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-09-11T23:07:56.286 INFO:teuthology.orchestra.run.smithi136.stdout:Upgraded: 2024-09-11T23:07:56.286 INFO:teuthology.orchestra.run.smithi136.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-11T23:07:56.287 INFO:teuthology.orchestra.run.smithi136.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-11T23:07:56.287 INFO:teuthology.orchestra.run.smithi136.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-11T23:07:56.287 INFO:teuthology.orchestra.run.smithi136.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-11T23:07:56.287 INFO:teuthology.orchestra.run.smithi136.stdout: librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:07:56.287 INFO:teuthology.orchestra.run.smithi136.stdout: librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:07:56.287 INFO:teuthology.orchestra.run.smithi136.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-11T23:07:56.287 INFO:teuthology.orchestra.run.smithi136.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-11T23:07:56.287 INFO:teuthology.orchestra.run.smithi136.stdout:Installed: 2024-09-11T23:07:56.287 INFO:teuthology.orchestra.run.smithi136.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-11T23:07:56.287 INFO:teuthology.orchestra.run.smithi136.stdout: automake-1.16.2-8.el9.noarch 2024-09-11T23:07:56.287 INFO:teuthology.orchestra.run.smithi136.stdout: bison-3.7.4-5.el9.x86_64 2024-09-11T23:07:56.288 INFO:teuthology.orchestra.run.smithi136.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-11T23:07:56.288 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:07:56.288 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:07:56.288 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:07:56.288 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:07:56.288 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T23:07:56.288 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-immutable-object-cache-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:07:56.288 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:07:56.288 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:07:56.288 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T23:07:56.288 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T23:07:56.288 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T23:07:56.288 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T23:07:56.289 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T23:07:56.289 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:07:56.289 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:07:56.289 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T23:07:56.289 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:07:56.289 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:07:56.289 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:07:56.289 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T23:07:56.289 INFO:teuthology.orchestra.run.smithi136.stdout: cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T23:07:56.289 INFO:teuthology.orchestra.run.smithi136.stdout: cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:07:56.289 INFO:teuthology.orchestra.run.smithi136.stdout: cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:07:56.289 INFO:teuthology.orchestra.run.smithi136.stdout: cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T23:07:56.290 INFO:teuthology.orchestra.run.smithi136.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-11T23:07:56.290 INFO:teuthology.orchestra.run.smithi136.stdout: flex-2.6.4-9.el9.x86_64 2024-09-11T23:07:56.290 INFO:teuthology.orchestra.run.smithi136.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-11T23:07:56.290 INFO:teuthology.orchestra.run.smithi136.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-11T23:07:56.290 INFO:teuthology.orchestra.run.smithi136.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-11T23:07:56.290 INFO:teuthology.orchestra.run.smithi136.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-11T23:07:56.290 INFO:teuthology.orchestra.run.smithi136.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-11T23:07:56.290 INFO:teuthology.orchestra.run.smithi136.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-11T23:07:56.290 INFO:teuthology.orchestra.run.smithi136.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-11T23:07:56.290 INFO:teuthology.orchestra.run.smithi136.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-11T23:07:56.290 INFO:teuthology.orchestra.run.smithi136.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-11T23:07:56.290 INFO:teuthology.orchestra.run.smithi136.stdout: libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:07:56.290 INFO:teuthology.orchestra.run.smithi136.stdout: libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:07:56.291 INFO:teuthology.orchestra.run.smithi136.stdout: libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:07:56.291 INFO:teuthology.orchestra.run.smithi136.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-11T23:07:56.291 INFO:teuthology.orchestra.run.smithi136.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-11T23:07:56.291 INFO:teuthology.orchestra.run.smithi136.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-11T23:07:56.291 INFO:teuthology.orchestra.run.smithi136.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-11T23:07:56.291 INFO:teuthology.orchestra.run.smithi136.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-11T23:07:56.291 INFO:teuthology.orchestra.run.smithi136.stdout: librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:07:56.291 INFO:teuthology.orchestra.run.smithi136.stdout: libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:07:56.291 INFO:teuthology.orchestra.run.smithi136.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-11T23:07:56.291 INFO:teuthology.orchestra.run.smithi136.stdout: librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:07:56.292 INFO:teuthology.orchestra.run.smithi136.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-11T23:07:56.292 INFO:teuthology.orchestra.run.smithi136.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-11T23:07:56.292 INFO:teuthology.orchestra.run.smithi136.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-11T23:07:56.292 INFO:teuthology.orchestra.run.smithi136.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-11T23:07:56.292 INFO:teuthology.orchestra.run.smithi136.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-11T23:07:56.292 INFO:teuthology.orchestra.run.smithi136.stdout: lua-5.4.4-4.el9.x86_64 2024-09-11T23:07:56.292 INFO:teuthology.orchestra.run.smithi136.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-11T23:07:56.292 INFO:teuthology.orchestra.run.smithi136.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-11T23:07:56.292 INFO:teuthology.orchestra.run.smithi136.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-11T23:07:56.292 INFO:teuthology.orchestra.run.smithi136.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-11T23:07:56.292 INFO:teuthology.orchestra.run.smithi136.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-11T23:07:56.293 INFO:teuthology.orchestra.run.smithi136.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-11T23:07:56.293 INFO:teuthology.orchestra.run.smithi136.stdout: postgresql-13.16-1.el9.x86_64 2024-09-11T23:07:56.293 INFO:teuthology.orchestra.run.smithi136.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-11T23:07:56.293 INFO:teuthology.orchestra.run.smithi136.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-11T23:07:56.293 INFO:teuthology.orchestra.run.smithi136.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-11T23:07:56.293 INFO:teuthology.orchestra.run.smithi136.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-11T23:07:56.293 INFO:teuthology.orchestra.run.smithi136.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-11T23:07:56.293 INFO:teuthology.orchestra.run.smithi136.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-11T23:07:56.293 INFO:teuthology.orchestra.run.smithi136.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-11T23:07:56.293 INFO:teuthology.orchestra.run.smithi136.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-11T23:07:56.293 INFO:teuthology.orchestra.run.smithi136.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-11T23:07:56.293 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-11T23:07:56.293 INFO:teuthology.orchestra.run.smithi136.stdout: python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:07:56.294 INFO:teuthology.orchestra.run.smithi136.stdout: python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:07:56.294 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:07:56.294 INFO:teuthology.orchestra.run.smithi136.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-11T23:07:56.294 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-11T23:07:56.294 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-11T23:07:56.294 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-11T23:07:56.294 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-11T23:07:56.294 INFO:teuthology.orchestra.run.smithi136.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-11T23:07:56.294 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-11T23:07:56.294 INFO:teuthology.orchestra.run.smithi136.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-11T23:07:56.294 INFO:teuthology.orchestra.run.smithi136.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-11T23:07:56.294 INFO:teuthology.orchestra.run.smithi136.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-11T23:07:56.294 INFO:teuthology.orchestra.run.smithi136.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-11T23:07:56.294 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-11T23:07:56.294 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-11T23:07:56.295 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-11T23:07:56.295 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-11T23:07:56.295 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-11T23:07:56.295 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-11T23:07:56.295 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-11T23:07:56.295 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-11T23:07:56.295 INFO:teuthology.orchestra.run.smithi136.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-11T23:07:56.295 INFO:teuthology.orchestra.run.smithi136.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-11T23:07:56.295 INFO:teuthology.orchestra.run.smithi136.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-11T23:07:56.295 INFO:teuthology.orchestra.run.smithi136.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-11T23:07:56.295 INFO:teuthology.orchestra.run.smithi136.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-11T23:07:56.295 INFO:teuthology.orchestra.run.smithi136.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-11T23:07:56.295 INFO:teuthology.orchestra.run.smithi136.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-11T23:07:56.295 INFO:teuthology.orchestra.run.smithi136.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-11T23:07:56.296 INFO:teuthology.orchestra.run.smithi136.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-11T23:07:56.296 INFO:teuthology.orchestra.run.smithi136.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-11T23:07:56.296 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-11T23:07:56.296 INFO:teuthology.orchestra.run.smithi136.stdout: python3-ply-3.11-14.el9.noarch 2024-09-11T23:07:56.296 INFO:teuthology.orchestra.run.smithi136.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-11T23:07:56.296 INFO:teuthology.orchestra.run.smithi136.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-11T23:07:56.296 INFO:teuthology.orchestra.run.smithi136.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-11T23:07:56.296 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-11T23:07:56.296 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-11T23:07:56.296 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-11T23:07:56.296 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-11T23:07:56.296 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-11T23:07:56.296 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-11T23:07:56.297 INFO:teuthology.orchestra.run.smithi136.stdout: python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:07:56.297 INFO:teuthology.orchestra.run.smithi136.stdout: python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:07:56.297 INFO:teuthology.orchestra.run.smithi136.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-11T23:07:56.297 INFO:teuthology.orchestra.run.smithi136.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-11T23:07:56.297 INFO:teuthology.orchestra.run.smithi136.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-11T23:07:56.297 INFO:teuthology.orchestra.run.smithi136.stdout: python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:07:56.297 INFO:teuthology.orchestra.run.smithi136.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-11T23:07:56.297 INFO:teuthology.orchestra.run.smithi136.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-11T23:07:56.297 INFO:teuthology.orchestra.run.smithi136.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-11T23:07:56.297 INFO:teuthology.orchestra.run.smithi136.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-11T23:07:56.297 INFO:teuthology.orchestra.run.smithi136.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-11T23:07:56.297 INFO:teuthology.orchestra.run.smithi136.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-11T23:07:56.298 INFO:teuthology.orchestra.run.smithi136.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-11T23:07:56.298 INFO:teuthology.orchestra.run.smithi136.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-11T23:07:56.298 INFO:teuthology.orchestra.run.smithi136.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-11T23:07:56.298 INFO:teuthology.orchestra.run.smithi136.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-11T23:07:56.298 INFO:teuthology.orchestra.run.smithi136.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-11T23:07:56.298 INFO:teuthology.orchestra.run.smithi136.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-11T23:07:56.298 INFO:teuthology.orchestra.run.smithi136.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-11T23:07:56.298 INFO:teuthology.orchestra.run.smithi136.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-11T23:07:56.298 INFO:teuthology.orchestra.run.smithi136.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-11T23:07:56.298 INFO:teuthology.orchestra.run.smithi136.stdout: rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:07:56.298 INFO:teuthology.orchestra.run.smithi136.stdout: rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:07:56.298 INFO:teuthology.orchestra.run.smithi136.stdout: rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:07:56.298 INFO:teuthology.orchestra.run.smithi136.stdout: re2-1:20211101-20.el9.x86_64 2024-09-11T23:07:56.298 INFO:teuthology.orchestra.run.smithi136.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-11T23:07:56.299 INFO:teuthology.orchestra.run.smithi136.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-11T23:07:56.299 INFO:teuthology.orchestra.run.smithi136.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-11T23:07:56.299 INFO:teuthology.orchestra.run.smithi136.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-11T23:07:56.299 INFO:teuthology.orchestra.run.smithi136.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-11T23:07:56.299 INFO:teuthology.orchestra.run.smithi136.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-11T23:07:56.299 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-09-11T23:07:56.300 INFO:teuthology.orchestra.run.smithi136.stdout:Complete! 2024-09-11T23:07:56.735 DEBUG:teuthology.parallel:result is None 2024-09-11T23:07:56.736 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-11T23:07:56.891 DEBUG:teuthology.orchestra.run.smithi026:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-11T23:07:56.919 INFO:teuthology.orchestra.run.smithi026.stdout:19.1.1-298.gb53b6d3e.el9 2024-09-11T23:07:56.919 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-298.gb53b6d3e.el9 2024-09-11T23:07:56.920 INFO:teuthology.task.install:The correct ceph version 19.1.1-298.gb53b6d3e is installed. 2024-09-11T23:07:56.921 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-11T23:07:57.072 DEBUG:teuthology.orchestra.run.smithi076:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-11T23:07:57.098 INFO:teuthology.orchestra.run.smithi076.stdout:19.1.1-298.gb53b6d3e.el9 2024-09-11T23:07:57.099 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-298.gb53b6d3e.el9 2024-09-11T23:07:57.099 INFO:teuthology.task.install:The correct ceph version 19.1.1-298.gb53b6d3e is installed. 2024-09-11T23:07:57.101 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-11T23:07:57.251 DEBUG:teuthology.orchestra.run.smithi136:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-11T23:07:57.275 INFO:teuthology.orchestra.run.smithi136.stdout:19.1.1-298.gb53b6d3e.el9 2024-09-11T23:07:57.276 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-298.gb53b6d3e.el9 2024-09-11T23:07:57.276 INFO:teuthology.task.install:The correct ceph version 19.1.1-298.gb53b6d3e is installed. 2024-09-11T23:07:57.278 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-11T23:07:57.278 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-09-11T23:07:57.278 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-11T23:07:57.312 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-09-11T23:07:57.312 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-11T23:07:57.344 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-09-11T23:07:57.345 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-11T23:07:57.375 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-11T23:07:57.376 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-09-11T23:07:57.376 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd of=/usr/bin/daemon-helper 2024-09-11T23:07:57.407 DEBUG:teuthology.orchestra.run.smithi026:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-11T23:07:57.478 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-09-11T23:07:57.478 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd of=/usr/bin/daemon-helper 2024-09-11T23:07:57.508 DEBUG:teuthology.orchestra.run.smithi076:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-11T23:07:57.578 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-09-11T23:07:57.578 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd of=/usr/bin/daemon-helper 2024-09-11T23:07:57.603 DEBUG:teuthology.orchestra.run.smithi136:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-11T23:07:57.669 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-11T23:07:57.669 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-09-11T23:07:57.669 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-11T23:07:57.700 DEBUG:teuthology.orchestra.run.smithi026:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-11T23:07:57.772 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-09-11T23:07:57.772 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-11T23:07:57.800 DEBUG:teuthology.orchestra.run.smithi076:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-11T23:07:57.868 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-09-11T23:07:57.868 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-11T23:07:57.893 DEBUG:teuthology.orchestra.run.smithi136:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-11T23:07:57.958 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-11T23:07:57.958 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-09-11T23:07:57.959 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd of=/usr/bin/stdin-killer 2024-09-11T23:07:57.990 DEBUG:teuthology.orchestra.run.smithi026:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-11T23:07:58.061 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-09-11T23:07:58.061 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd of=/usr/bin/stdin-killer 2024-09-11T23:07:58.090 DEBUG:teuthology.orchestra.run.smithi076:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-11T23:07:58.160 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-09-11T23:07:58.160 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd of=/usr/bin/stdin-killer 2024-09-11T23:07:58.186 DEBUG:teuthology.orchestra.run.smithi136:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-11T23:07:58.251 INFO:teuthology.run_tasks:Running task cephadm... 2024-09-11T23:07:58.360 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-09-11T23:07:58.360 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-11T23:07:58.361 INFO:tasks.cephadm:Cluster fsid is af7e6fc8-7092-11ef-bcea-c7b262605968 2024-09-11T23:07:58.361 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-09-11T23:07:58.361 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.26', 'mon.b': '172.21.15.76', 'mon.c': '172.21.15.136'} 2024-09-11T23:07:58.361 INFO:tasks.cephadm:First mon is mon.a on smithi026 2024-09-11T23:07:58.361 INFO:tasks.cephadm:First mgr is x 2024-09-11T23:07:58.361 INFO:tasks.cephadm:Normalizing hostnames... 2024-09-11T23:07:58.361 DEBUG:teuthology.orchestra.run.smithi026:> sudo hostname $(hostname -s) 2024-09-11T23:07:58.397 DEBUG:teuthology.orchestra.run.smithi076:> sudo hostname $(hostname -s) 2024-09-11T23:07:58.426 DEBUG:teuthology.orchestra.run.smithi136:> sudo hostname $(hostname -s) 2024-09-11T23:07:58.453 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-09-11T23:07:58.453 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-11T23:07:58.610 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/56038/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.1.1-298-gb53b6d3e', 'node_name': '172.21.5.36+adami06', 'job_name': 'ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.1.1-298.gb53b6d3e'}, 'url': 'https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-09-09 17:24:58.599585', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'squid', 'chacra_url': 'https://2.chacra.ceph.com/repos/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-09-11T23:07:58.757 INFO:tasks.util.chacra:got chacra host 2.chacra.ceph.com, ref squid, sha1 b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-11T23:07:58.758 INFO:tasks.cephadm:Discovered cachra url: https://2.chacra.ceph.com/binaries/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/x86_64/flavors/default/cephadm 2024-09-11T23:07:58.759 INFO:tasks.cephadm:Downloading cephadm from url: https://2.chacra.ceph.com/binaries/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/x86_64/flavors/default/cephadm 2024-09-11T23:07:58.759 DEBUG:teuthology.orchestra.run.smithi026:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-11T23:07:59.126 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r--. 1 ubuntu ubuntu 784498 Sep 11 23:07 /home/ubuntu/cephtest/cephadm 2024-09-11T23:07:59.126 DEBUG:teuthology.orchestra.run.smithi076:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-11T23:07:59.537 INFO:teuthology.orchestra.run.smithi076.stdout:-rw-r--r--. 1 ubuntu ubuntu 784498 Sep 11 23:07 /home/ubuntu/cephtest/cephadm 2024-09-11T23:07:59.538 DEBUG:teuthology.orchestra.run.smithi136:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-11T23:07:59.925 INFO:teuthology.orchestra.run.smithi136.stdout:-rw-r--r--. 1 ubuntu ubuntu 784498 Sep 11 23:07 /home/ubuntu/cephtest/cephadm 2024-09-11T23:07:59.925 DEBUG:teuthology.orchestra.run.smithi026:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-11T23:07:59.946 DEBUG:teuthology.orchestra.run.smithi076:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-11T23:07:59.965 DEBUG:teuthology.orchestra.run.smithi136:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-11T23:08:00.018 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc on all hosts... 2024-09-11T23:08:00.019 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc pull 2024-09-11T23:08:00.021 DEBUG:teuthology.orchestra.run.smithi076:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc pull 2024-09-11T23:08:00.024 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc pull 2024-09-11T23:08:01.606 INFO:teuthology.orchestra.run.smithi026.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc... 2024-09-11T23:08:01.655 INFO:teuthology.orchestra.run.smithi076.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc... 2024-09-11T23:08:01.700 INFO:teuthology.orchestra.run.smithi136.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc... 2024-09-11T23:08:28.911 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-11T23:08:28.912 INFO:teuthology.orchestra.run.smithi136.stdout: "ceph_version": "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)", 2024-09-11T23:08:28.912 INFO:teuthology.orchestra.run.smithi136.stdout: "image_id": "54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594", 2024-09-11T23:08:28.912 INFO:teuthology.orchestra.run.smithi136.stdout: "repo_digests": [ 2024-09-11T23:08:28.912 INFO:teuthology.orchestra.run.smithi136.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66" 2024-09-11T23:08:28.912 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-11T23:08:28.912 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-11T23:08:31.430 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-11T23:08:31.431 INFO:teuthology.orchestra.run.smithi076.stdout: "ceph_version": "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)", 2024-09-11T23:08:31.431 INFO:teuthology.orchestra.run.smithi076.stdout: "image_id": "54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594", 2024-09-11T23:08:31.431 INFO:teuthology.orchestra.run.smithi076.stdout: "repo_digests": [ 2024-09-11T23:08:31.431 INFO:teuthology.orchestra.run.smithi076.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66" 2024-09-11T23:08:31.431 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-11T23:08:31.431 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-11T23:08:31.611 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-11T23:08:31.611 INFO:teuthology.orchestra.run.smithi026.stdout: "ceph_version": "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)", 2024-09-11T23:08:31.611 INFO:teuthology.orchestra.run.smithi026.stdout: "image_id": "54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594", 2024-09-11T23:08:31.611 INFO:teuthology.orchestra.run.smithi026.stdout: "repo_digests": [ 2024-09-11T23:08:31.612 INFO:teuthology.orchestra.run.smithi026.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66" 2024-09-11T23:08:31.612 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-11T23:08:31.612 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-11T23:08:31.644 DEBUG:teuthology.orchestra.run.smithi026:> sudo mkdir -p /etc/ceph 2024-09-11T23:08:31.676 DEBUG:teuthology.orchestra.run.smithi076:> sudo mkdir -p /etc/ceph 2024-09-11T23:08:31.704 DEBUG:teuthology.orchestra.run.smithi136:> sudo mkdir -p /etc/ceph 2024-09-11T23:08:31.738 DEBUG:teuthology.orchestra.run.smithi026:> sudo chmod 777 /etc/ceph 2024-09-11T23:08:31.766 DEBUG:teuthology.orchestra.run.smithi076:> sudo chmod 777 /etc/ceph 2024-09-11T23:08:31.791 DEBUG:teuthology.orchestra.run.smithi136:> sudo chmod 777 /etc/ceph 2024-09-11T23:08:31.819 INFO:tasks.cephadm:Writing seed config... 2024-09-11T23:08:31.820 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-09-11T23:08:31.821 INFO:tasks.cephadm: override: [client] debug client = 20 2024-09-11T23:08:31.821 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-09-11T23:08:31.821 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-09-11T23:08:31.821 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-09-11T23:08:31.821 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-09-11T23:08:31.821 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-09-11T23:08:31.821 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-09-11T23:08:31.821 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-09-11T23:08:31.821 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-09-11T23:08:31.821 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-09-11T23:08:31.821 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-09-11T23:08:31.821 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-09-11T23:08:31.822 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-09-11T23:08:31.822 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-09-11T23:08:31.822 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-09-11T23:08:31.822 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-09-11T23:08:31.822 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2024-09-11T23:08:31.822 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-09-11T23:08:31.822 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-09-11T23:08:31.822 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-09-11T23:08:31.822 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-09-11T23:08:31.822 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2024-09-11T23:08:31.822 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-09-11T23:08:31.822 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-09-11T23:08:31.822 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-09-11T23:08:31.822 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-09-11T23:08:31.823 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-09-11T23:08:31.823 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-09-11T23:08:31.823 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-09-11T23:08:31.823 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-09-11T23:08:31.823 INFO:tasks.cephadm: override: [mon] mon warn on pool no app = False 2024-09-11T23:08:31.823 INFO:tasks.cephadm: override: [osd] bdev async discard = True 2024-09-11T23:08:31.823 INFO:tasks.cephadm: override: [osd] bdev enable discard = True 2024-09-11T23:08:31.823 INFO:tasks.cephadm: override: [osd] bluestore allocator = bitmap 2024-09-11T23:08:31.823 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-09-11T23:08:31.823 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-09-11T23:08:31.823 INFO:tasks.cephadm: override: [osd] debug bluefs = 1/20 2024-09-11T23:08:31.823 INFO:tasks.cephadm: override: [osd] debug bluestore = 1/20 2024-09-11T23:08:31.823 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-09-11T23:08:31.824 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-09-11T23:08:31.824 INFO:tasks.cephadm: override: [osd] debug rocksdb = 4/10 2024-09-11T23:08:31.824 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-09-11T23:08:31.824 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-09-11T23:08:31.824 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-09-11T23:08:31.824 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-09-11T23:08:31.824 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-09-11T23:08:31.824 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-09-11T23:08:31.824 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-09-11T23:08:31.824 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-09-11T23:08:31.825 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-09-11T23:08:31.825 DEBUG:teuthology.orchestra.run.smithi026:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-09-11T23:08:31.843 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = af7e6fc8-7092-11ef-bcea-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bdev async discard = True bdev enable discard = True bluestore allocator = bitmap bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 1/20 debug bluestore = 1/20 debug ms = 1 debug osd = 20 debug rocksdb = 4/10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 mon warn on pool no app = False [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_bal_replicate_threshold = 1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-09-11T23:08:31.844 DEBUG:teuthology.orchestra.run.smithi026:mon.a> sudo journalctl -f -n 0 -u ceph-af7e6fc8-7092-11ef-bcea-c7b262605968@mon.a.service 2024-09-11T23:08:31.887 DEBUG:teuthology.orchestra.run.smithi026:mgr.x> sudo journalctl -f -n 0 -u ceph-af7e6fc8-7092-11ef-bcea-c7b262605968@mgr.x.service 2024-09-11T23:08:31.930 INFO:tasks.cephadm:Bootstrapping... 2024-09-11T23:08:31.930 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc -v bootstrap --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.26 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-09-11T23:08:32.145 INFO:teuthology.orchestra.run.smithi026.stdout:-------------------------------------------------------------------------------- 2024-09-11T23:08:32.145 INFO:teuthology.orchestra.run.smithi026.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc', '-v', 'bootstrap', '--fsid', 'af7e6fc8-7092-11ef-bcea-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.26', '--skip-admin-label'] 2024-09-11T23:08:32.145 INFO:teuthology.orchestra.run.smithi026.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-09-11T23:08:32.146 INFO:teuthology.orchestra.run.smithi026.stdout:Verifying podman|docker is present... 2024-09-11T23:08:32.170 INFO:teuthology.orchestra.run.smithi026.stdout:/bin/podman: stdout 5.2.2 2024-09-11T23:08:32.170 INFO:teuthology.orchestra.run.smithi026.stdout:Verifying lvm2 is present... 2024-09-11T23:08:32.170 INFO:teuthology.orchestra.run.smithi026.stdout:Verifying time synchronization is in place... 2024-09-11T23:08:32.178 INFO:teuthology.orchestra.run.smithi026.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-11T23:08:32.178 INFO:teuthology.orchestra.run.smithi026.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-11T23:08:32.184 INFO:teuthology.orchestra.run.smithi026.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-11T23:08:32.184 INFO:teuthology.orchestra.run.smithi026.stdout:systemctl: stdout inactive 2024-09-11T23:08:32.191 INFO:teuthology.orchestra.run.smithi026.stdout:systemctl: stdout enabled 2024-09-11T23:08:32.198 INFO:teuthology.orchestra.run.smithi026.stdout:systemctl: stdout active 2024-09-11T23:08:32.198 INFO:teuthology.orchestra.run.smithi026.stdout:Unit chronyd.service is enabled and running 2024-09-11T23:08:32.198 INFO:teuthology.orchestra.run.smithi026.stdout:Repeating the final host check... 2024-09-11T23:08:32.221 INFO:teuthology.orchestra.run.smithi026.stdout:/bin/podman: stdout 5.2.2 2024-09-11T23:08:32.222 INFO:teuthology.orchestra.run.smithi026.stdout:podman (/bin/podman) version 5.2.2 is present 2024-09-11T23:08:32.222 INFO:teuthology.orchestra.run.smithi026.stdout:systemctl is present 2024-09-11T23:08:32.222 INFO:teuthology.orchestra.run.smithi026.stdout:lvcreate is present 2024-09-11T23:08:32.228 INFO:teuthology.orchestra.run.smithi026.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-11T23:08:32.228 INFO:teuthology.orchestra.run.smithi026.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-11T23:08:32.235 INFO:teuthology.orchestra.run.smithi026.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-11T23:08:32.235 INFO:teuthology.orchestra.run.smithi026.stdout:systemctl: stdout inactive 2024-09-11T23:08:32.241 INFO:teuthology.orchestra.run.smithi026.stdout:systemctl: stdout enabled 2024-09-11T23:08:32.248 INFO:teuthology.orchestra.run.smithi026.stdout:systemctl: stdout active 2024-09-11T23:08:32.248 INFO:teuthology.orchestra.run.smithi026.stdout:Unit chronyd.service is enabled and running 2024-09-11T23:08:32.248 INFO:teuthology.orchestra.run.smithi026.stdout:Host looks OK 2024-09-11T23:08:32.248 INFO:teuthology.orchestra.run.smithi026.stdout:Cluster fsid: af7e6fc8-7092-11ef-bcea-c7b262605968 2024-09-11T23:08:32.249 INFO:teuthology.orchestra.run.smithi026.stdout:Acquiring lock 139921322099712 on /run/cephadm/af7e6fc8-7092-11ef-bcea-c7b262605968.lock 2024-09-11T23:08:32.249 INFO:teuthology.orchestra.run.smithi026.stdout:Lock 139921322099712 acquired on /run/cephadm/af7e6fc8-7092-11ef-bcea-c7b262605968.lock 2024-09-11T23:08:32.249 INFO:teuthology.orchestra.run.smithi026.stdout:Verifying IP 172.21.15.26 port 3300 ... 2024-09-11T23:08:32.249 INFO:teuthology.orchestra.run.smithi026.stdout:Verifying IP 172.21.15.26 port 6789 ... 2024-09-11T23:08:32.249 INFO:teuthology.orchestra.run.smithi026.stdout:Base mon IP(s) is [172.21.15.26:3300, 172.21.15.26:6789], mon addrv is [v2:172.21.15.26:3300,v1:172.21.15.26:6789] 2024-09-11T23:08:32.253 INFO:teuthology.orchestra.run.smithi026.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.26 metric 100 2024-09-11T23:08:32.253 INFO:teuthology.orchestra.run.smithi026.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.26 metric 100 2024-09-11T23:08:32.256 INFO:teuthology.orchestra.run.smithi026.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-09-11T23:08:32.256 INFO:teuthology.orchestra.run.smithi026.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 256 pref medium 2024-09-11T23:08:32.256 INFO:teuthology.orchestra.run.smithi026.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 1024 expires 1797sec hoplimit 64 pref medium 2024-09-11T23:08:32.258 INFO:teuthology.orchestra.run.smithi026.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-09-11T23:08:32.258 INFO:teuthology.orchestra.run.smithi026.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-09-11T23:08:32.258 INFO:teuthology.orchestra.run.smithi026.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-11T23:08:32.259 INFO:teuthology.orchestra.run.smithi026.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-09-11T23:08:32.259 INFO:teuthology.orchestra.run.smithi026.stdout:/sbin/ip: stdout inet6 fe80::ae1f:6bff:fea5:cac0/64 scope link 2024-09-11T23:08:32.259 INFO:teuthology.orchestra.run.smithi026.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-11T23:08:32.259 INFO:teuthology.orchestra.run.smithi026.stdout:Mon IP `172.21.15.26` is in CIDR network `172.21.0.0/20` 2024-09-11T23:08:32.259 INFO:teuthology.orchestra.run.smithi026.stdout:Mon IP `172.21.15.26` is in CIDR network `172.21.0.0/20` 2024-09-11T23:08:32.259 INFO:teuthology.orchestra.run.smithi026.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-09-11T23:08:32.260 INFO:teuthology.orchestra.run.smithi026.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-09-11T23:08:32.261 INFO:teuthology.orchestra.run.smithi026.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc... 2024-09-11T23:08:32.688 INFO:teuthology.orchestra.run.smithi026.stdout:/bin/podman: stdout 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 2024-09-11T23:08:32.688 INFO:teuthology.orchestra.run.smithi026.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc... 2024-09-11T23:08:32.689 INFO:teuthology.orchestra.run.smithi026.stdout:/bin/podman: stderr Getting image source signatures 2024-09-11T23:08:32.689 INFO:teuthology.orchestra.run.smithi026.stdout:/bin/podman: stderr Copying blob sha256:60180723200353b4e13b39fd162aecaf1bc097c859c7c6537324229e97c78567 2024-09-11T23:08:32.689 INFO:teuthology.orchestra.run.smithi026.stdout:/bin/podman: stderr Copying blob sha256:e8b54c863393b7a8216a71737771b73b16a8e43ec7acf927c7faa175c255e551 2024-09-11T23:08:32.689 INFO:teuthology.orchestra.run.smithi026.stdout:/bin/podman: stderr Copying config sha256:54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 2024-09-11T23:08:32.689 INFO:teuthology.orchestra.run.smithi026.stdout:/bin/podman: stderr Writing manifest to image destination 2024-09-11T23:08:33.575 INFO:teuthology.orchestra.run.smithi026.stdout:ceph: stdout ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable) 2024-09-11T23:08:33.576 INFO:teuthology.orchestra.run.smithi026.stdout:Ceph version: ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable) 2024-09-11T23:08:33.576 INFO:teuthology.orchestra.run.smithi026.stdout:Extracting ceph user uid/gid from container image... 2024-09-11T23:08:34.431 INFO:teuthology.orchestra.run.smithi026.stdout:stat: stdout 167 167 2024-09-11T23:08:34.431 INFO:teuthology.orchestra.run.smithi026.stdout:Creating initial keys... 2024-09-11T23:08:35.320 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph-authtool: stdout AQDyIuJmNfYYLRAAbD0cClEPcU4c5HtD/oIC7w== 2024-09-11T23:08:36.242 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph-authtool: stdout AQDzIuJmn84NJhAAT3r9QwWT98XmOI3XYoN7GA== 2024-09-11T23:08:37.139 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph-authtool: stdout AQD0IuJmjArdIRAAq+KsUHu24rRzZ5A+zSRqkA== 2024-09-11T23:08:37.140 INFO:teuthology.orchestra.run.smithi026.stdout:Creating initial monmap... 2024-09-11T23:08:38.078 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-11T23:08:38.078 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-09-11T23:08:38.078 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to af7e6fc8-7092-11ef-bcea-c7b262605968 2024-09-11T23:08:38.079 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-11T23:08:38.079 INFO:teuthology.orchestra.run.smithi026.stdout:monmaptool for a [v2:172.21.15.26:3300,v1:172.21.15.26:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-11T23:08:38.079 INFO:teuthology.orchestra.run.smithi026.stdout:setting min_mon_release = quincy 2024-09-11T23:08:38.079 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/monmaptool: set fsid to af7e6fc8-7092-11ef-bcea-c7b262605968 2024-09-11T23:08:38.079 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-11T23:08:38.079 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-11T23:08:38.079 INFO:teuthology.orchestra.run.smithi026.stdout:Creating mon... 2024-09-11T23:08:38.967 INFO:teuthology.orchestra.run.smithi026.stdout:create mon.a on 2024-09-11T23:08:39.189 INFO:teuthology.orchestra.run.smithi026.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-09-11T23:08:39.350 INFO:teuthology.orchestra.run.smithi026.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-09-11T23:08:39.540 INFO:teuthology.orchestra.run.smithi026.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-af7e6fc8-7092-11ef-bcea-c7b262605968.target → /etc/systemd/system/ceph-af7e6fc8-7092-11ef-bcea-c7b262605968.target. 2024-09-11T23:08:39.540 INFO:teuthology.orchestra.run.smithi026.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-af7e6fc8-7092-11ef-bcea-c7b262605968.target → /etc/systemd/system/ceph-af7e6fc8-7092-11ef-bcea-c7b262605968.target. 2024-09-11T23:08:39.800 INFO:teuthology.orchestra.run.smithi026.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-af7e6fc8-7092-11ef-bcea-c7b262605968@mon.a 2024-09-11T23:08:39.800 INFO:teuthology.orchestra.run.smithi026.stdout:systemctl: stderr Failed to reset failed state of unit ceph-af7e6fc8-7092-11ef-bcea-c7b262605968@mon.a.service: Unit ceph-af7e6fc8-7092-11ef-bcea-c7b262605968@mon.a.service not loaded. 2024-09-11T23:08:39.970 INFO:teuthology.orchestra.run.smithi026.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-af7e6fc8-7092-11ef-bcea-c7b262605968.target.wants/ceph-af7e6fc8-7092-11ef-bcea-c7b262605968@mon.a.service → /etc/systemd/system/ceph-af7e6fc8-7092-11ef-bcea-c7b262605968@.service. 2024-09-11T23:08:40.534 INFO:teuthology.orchestra.run.smithi026.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-11T23:08:40.534 INFO:teuthology.orchestra.run.smithi026.stdout:systemctl: stdout disabled 2024-09-11T23:08:40.541 INFO:teuthology.orchestra.run.smithi026.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-11T23:08:40.542 INFO:teuthology.orchestra.run.smithi026.stdout:systemctl: stdout inactive 2024-09-11T23:08:40.542 INFO:teuthology.orchestra.run.smithi026.stdout:firewalld.service is not enabled 2024-09-11T23:08:40.542 INFO:teuthology.orchestra.run.smithi026.stdout:Not possible to enable service . firewalld.service is not available 2024-09-11T23:08:40.542 INFO:teuthology.orchestra.run.smithi026.stdout:Waiting for mon to start... 2024-09-11T23:08:40.542 INFO:teuthology.orchestra.run.smithi026.stdout:Waiting for mon... 2024-09-11T23:08:40.620 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:40 smithi026 systemd[1]: Started Ceph mon.a for af7e6fc8-7092-11ef-bcea-c7b262605968. 2024-09-11T23:08:41.584 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout cluster: 2024-09-11T23:08:41.584 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout id: af7e6fc8-7092-11ef-bcea-c7b262605968 2024-09-11T23:08:41.584 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-09-11T23:08:41.585 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout 2024-09-11T23:08:41.585 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout services: 2024-09-11T23:08:41.585 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.554983s) 2024-09-11T23:08:41.585 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-09-11T23:08:41.585 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-09-11T23:08:41.585 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout 2024-09-11T23:08:41.585 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout data: 2024-09-11T23:08:41.585 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-09-11T23:08:41.585 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-09-11T23:08:41.585 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-09-11T23:08:41.585 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout pgs: 2024-09-11T23:08:41.585 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout 2024-09-11T23:08:41.585 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:40.968+0000 7fb7e30c5640 1 Processor -- start 2024-09-11T23:08:41.586 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:40.968+0000 7fb7e30c5640 1 -- start start 2024-09-11T23:08:41.586 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:40.968+0000 7fb7e30c5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb7dc07de20 0x7fb7dc07c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:08:41.586 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:40.968+0000 7fb7e30c5640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb7dc07c850 con 0x7fb7dc07de20 2024-09-11T23:08:41.586 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:40.968+0000 7fb7e0e61640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb7dc07de20 0x7fb7dc07c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:08:41.586 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:40.968+0000 7fb7e0e61640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb7dc07de20 0x7fb7dc07c280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43760/0 (socket says 172.21.15.26:43760) 2024-09-11T23:08:41.586 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:40.968+0000 7fb7e0e61640 1 -- 172.21.15.26:0/2131945615 learned_addr learned my addr 172.21.15.26:0/2131945615 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:08:41.586 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:40.969+0000 7fb7e0e61640 1 -- 172.21.15.26:0/2131945615 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7dc07c9d0 con 0x7fb7dc07de20 2024-09-11T23:08:41.586 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:40.971+0000 7fb7e0e61640 1 --2- 172.21.15.26:0/2131945615 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb7dc07de20 0x7fb7dc07c280 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fb7d0009820 tx=0x7fb7d002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7286c9ca1474a65 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:08:41.586 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:40.971+0000 7fb7db7fe640 1 -- 172.21.15.26:0/2131945615 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb7d0037070 con 0x7fb7dc07de20 2024-09-11T23:08:41.586 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:40.971+0000 7fb7db7fe640 1 -- 172.21.15.26:0/2131945615 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7d0032440 con 0x7fb7dc07de20 2024-09-11T23:08:41.586 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:40.971+0000 7fb7db7fe640 1 -- 172.21.15.26:0/2131945615 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb7d0032740 con 0x7fb7dc07de20 2024-09-11T23:08:41.586 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:40.971+0000 7fb7e30c5640 1 -- 172.21.15.26:0/2131945615 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb7dc07de20 msgr2=0x7fb7dc07c280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:08:41.586 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:40.971+0000 7fb7e30c5640 1 --2- 172.21.15.26:0/2131945615 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb7dc07de20 0x7fb7dc07c280 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fb7d0009820 tx=0x7fb7d002a270 comp rx=0 tx=0).stop 2024-09-11T23:08:41.587 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:40.972+0000 7fb7e30c5640 1 -- 172.21.15.26:0/2131945615 shutdown_connections 2024-09-11T23:08:41.587 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:40.972+0000 7fb7e30c5640 1 --2- 172.21.15.26:0/2131945615 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb7dc07de20 0x7fb7dc07c280 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:08:41.587 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:40.972+0000 7fb7e30c5640 1 -- 172.21.15.26:0/2131945615 >> 172.21.15.26:0/2131945615 conn(0x7fb7dc102f60 msgr2=0x7fb7dc1053a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:08:41.587 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:40.972+0000 7fb7e30c5640 1 -- 172.21.15.26:0/2131945615 shutdown_connections 2024-09-11T23:08:41.587 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:40.972+0000 7fb7e30c5640 1 -- 172.21.15.26:0/2131945615 wait complete. 2024-09-11T23:08:41.587 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:40.972+0000 7fb7e30c5640 1 Processor -- start 2024-09-11T23:08:41.587 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:40.972+0000 7fb7e30c5640 1 -- start start 2024-09-11T23:08:41.587 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:40.973+0000 7fb7e30c5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb7dc07de20 0x7fb7dc19d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:08:41.587 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:40.973+0000 7fb7e30c5640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb7dc07cf00 con 0x7fb7dc07de20 2024-09-11T23:08:41.587 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:40.973+0000 7fb7e0e61640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb7dc07de20 0x7fb7dc19d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:08:41.587 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:40.973+0000 7fb7e0e61640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb7dc07de20 0x7fb7dc19d150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43774/0 (socket says 172.21.15.26:43774) 2024-09-11T23:08:41.587 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:40.973+0000 7fb7e0e61640 1 -- 172.21.15.26:0/3893715615 learned_addr learned my addr 172.21.15.26:0/3893715615 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:08:41.587 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:40.973+0000 7fb7e0e61640 1 -- 172.21.15.26:0/3893715615 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7dc19d690 con 0x7fb7dc07de20 2024-09-11T23:08:41.588 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:40.973+0000 7fb7e0e61640 1 --2- 172.21.15.26:0/3893715615 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb7dc07de20 0x7fb7dc19d150 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fb7d00090b0 tx=0x7fb7d00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:08:41.588 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:40.973+0000 7fb7d9ffb640 1 -- 172.21.15.26:0/3893715615 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb7d0041070 con 0x7fb7dc07de20 2024-09-11T23:08:41.588 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:40.973+0000 7fb7d9ffb640 1 -- 172.21.15.26:0/3893715615 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7d003c7a0 con 0x7fb7dc07de20 2024-09-11T23:08:41.588 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:40.973+0000 7fb7d9ffb640 1 -- 172.21.15.26:0/3893715615 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb7d0037040 con 0x7fb7dc07de20 2024-09-11T23:08:41.588 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:40.973+0000 7fb7e30c5640 1 -- 172.21.15.26:0/3893715615 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7dc19d920 con 0x7fb7dc07de20 2024-09-11T23:08:41.588 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:40.974+0000 7fb7e30c5640 1 -- 172.21.15.26:0/3893715615 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7dc1a0610 con 0x7fb7dc07de20 2024-09-11T23:08:41.588 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:40.974+0000 7fb7d9ffb640 1 -- 172.21.15.26:0/3893715615 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fb7d0044070 con 0x7fb7dc07de20 2024-09-11T23:08:41.588 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:40.974+0000 7fb7d9ffb640 1 -- 172.21.15.26:0/3893715615 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fb7d003d3f0 con 0x7fb7dc07de20 2024-09-11T23:08:41.588 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:40.975+0000 7fb7e30c5640 1 -- 172.21.15.26:0/3893715615 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7dc07c300 con 0x7fb7dc07de20 2024-09-11T23:08:41.588 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:40.977+0000 7fb7d9ffb640 1 -- 172.21.15.26:0/3893715615 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fb7d0046450 con 0x7fb7dc07de20 2024-09-11T23:08:41.588 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:41.038+0000 7fb7e30c5640 1 -- 172.21.15.26:0/3893715615 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7fb7dc02d940 con 0x7fb7dc07de20 2024-09-11T23:08:41.588 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:41.039+0000 7fb7d9ffb640 1 -- 172.21.15.26:0/3893715615 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7fb7d0046630 con 0x7fb7dc07de20 2024-09-11T23:08:41.588 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:41.040+0000 7fb7e30c5640 1 -- 172.21.15.26:0/3893715615 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb7dc07de20 msgr2=0x7fb7dc19d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:08:41.589 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:41.040+0000 7fb7e30c5640 1 --2- 172.21.15.26:0/3893715615 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb7dc07de20 0x7fb7dc19d150 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fb7d00090b0 tx=0x7fb7d00046c0 comp rx=0 tx=0).stop 2024-09-11T23:08:41.589 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:41.040+0000 7fb7e30c5640 1 -- 172.21.15.26:0/3893715615 shutdown_connections 2024-09-11T23:08:41.589 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:41.040+0000 7fb7e30c5640 1 --2- 172.21.15.26:0/3893715615 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb7dc07de20 0x7fb7dc19d150 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:08:41.589 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:41.040+0000 7fb7e30c5640 1 -- 172.21.15.26:0/3893715615 >> 172.21.15.26:0/3893715615 conn(0x7fb7dc102f60 msgr2=0x7fb7dc104000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:08:41.589 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:41.040+0000 7fb7e30c5640 1 -- 172.21.15.26:0/3893715615 shutdown_connections 2024-09-11T23:08:41.589 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:41.040+0000 7fb7e30c5640 1 -- 172.21.15.26:0/3893715615 wait complete. 2024-09-11T23:08:41.589 INFO:teuthology.orchestra.run.smithi026.stdout:mon is available 2024-09-11T23:08:41.589 INFO:teuthology.orchestra.run.smithi026.stdout:Assimilating anything we can from ceph.conf... 2024-09-11T23:08:41.701 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:41 smithi026 ceph-mon[30804]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-11T23:08:41.702 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:41 smithi026 ceph-mon[30804]: monmap epoch 1 2024-09-11T23:08:41.702 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:41 smithi026 ceph-mon[30804]: fsid af7e6fc8-7092-11ef-bcea-c7b262605968 2024-09-11T23:08:41.702 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:41 smithi026 ceph-mon[30804]: last_changed 2024-09-11T23:08:37.446176+0000 2024-09-11T23:08:41.702 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:41 smithi026 ceph-mon[30804]: created 2024-09-11T23:08:37.446176+0000 2024-09-11T23:08:41.702 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:41 smithi026 ceph-mon[30804]: min_mon_release 19 (squid) 2024-09-11T23:08:41.702 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:41 smithi026 ceph-mon[30804]: election_strategy: 1 2024-09-11T23:08:41.702 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:41 smithi026 ceph-mon[30804]: 0: [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] mon.a 2024-09-11T23:08:41.702 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:41 smithi026 ceph-mon[30804]: fsmap 2024-09-11T23:08:41.702 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:41 smithi026 ceph-mon[30804]: osdmap e1: 0 total, 0 up, 0 in 2024-09-11T23:08:41.702 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:41 smithi026 ceph-mon[30804]: mgrmap e1: no daemons active 2024-09-11T23:08:41.702 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:41 smithi026 ceph-mon[30804]: from='client.? 172.21.15.26:0/3893715615' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-09-11T23:08:42.640 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout 2024-09-11T23:08:42.640 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout [global] 2024-09-11T23:08:42.640 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout fsid = af7e6fc8-7092-11ef-bcea-c7b262605968 2024-09-11T23:08:42.641 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.26:3300,v1:172.21.15.26:6789] 2024-09-11T23:08:42.641 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-11T23:08:42.641 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-11T23:08:42.641 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-11T23:08:42.641 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-11T23:08:42.641 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout 2024-09-11T23:08:42.641 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-11T23:08:42.641 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-11T23:08:42.641 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout 2024-09-11T23:08:42.641 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout [osd] 2024-09-11T23:08:42.641 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-11T23:08:42.641 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-11T23:08:42.641 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-11T23:08:42.641 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-11T23:08:42.642 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-11T23:08:42.642 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:41.983+0000 7f942e29f640 1 Processor -- start 2024-09-11T23:08:42.642 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:41.984+0000 7f942e29f640 1 -- start start 2024-09-11T23:08:42.642 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:41.984+0000 7f942e29f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94281066d0 0x7f9428106ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:08:42.642 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:41.984+0000 7f942e29f640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f94281070a0 con 0x7f94281066d0 2024-09-11T23:08:42.642 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:41.984+0000 7f9427fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94281066d0 0x7f9428106ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:08:42.642 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:41.984+0000 7f9427fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94281066d0 0x7f9428106ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43790/0 (socket says 172.21.15.26:43790) 2024-09-11T23:08:42.642 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:41.984+0000 7f9427fff640 1 -- 172.21.15.26:0/213155113 learned_addr learned my addr 172.21.15.26:0/213155113 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:08:42.642 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:41.984+0000 7f9427fff640 1 -- 172.21.15.26:0/213155113 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94281078d0 con 0x7f94281066d0 2024-09-11T23:08:42.642 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:41.985+0000 7f9427fff640 1 --2- 172.21.15.26:0/213155113 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94281066d0 0x7f9428106ad0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f9418009820 tx=0x7f941802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12667ddcc408cd69 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:08:42.642 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:41.985+0000 7f94277fe640 1 -- 172.21.15.26:0/213155113 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9418037070 con 0x7f94281066d0 2024-09-11T23:08:42.642 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:41.985+0000 7f94277fe640 1 -- 172.21.15.26:0/213155113 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9418032440 con 0x7f94281066d0 2024-09-11T23:08:42.643 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:41.985+0000 7f94277fe640 1 -- 172.21.15.26:0/213155113 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9418032740 con 0x7f94281066d0 2024-09-11T23:08:42.643 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:41.986+0000 7f942e29f640 1 -- 172.21.15.26:0/213155113 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94281066d0 msgr2=0x7f9428106ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:08:42.643 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:41.986+0000 7f942e29f640 1 --2- 172.21.15.26:0/213155113 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94281066d0 0x7f9428106ad0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f9418009820 tx=0x7f941802a270 comp rx=0 tx=0).stop 2024-09-11T23:08:42.643 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:41.986+0000 7f942e29f640 1 -- 172.21.15.26:0/213155113 shutdown_connections 2024-09-11T23:08:42.643 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:41.986+0000 7f942e29f640 1 --2- 172.21.15.26:0/213155113 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94281066d0 0x7f9428106ad0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:08:42.643 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:41.986+0000 7f942e29f640 1 -- 172.21.15.26:0/213155113 >> 172.21.15.26:0/213155113 conn(0x7f9428101e60 msgr2=0x7f94281042a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:08:42.643 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:41.986+0000 7f942e29f640 1 -- 172.21.15.26:0/213155113 shutdown_connections 2024-09-11T23:08:42.643 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:41.986+0000 7f942e29f640 1 -- 172.21.15.26:0/213155113 wait complete. 2024-09-11T23:08:42.643 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:41.986+0000 7f942e29f640 1 Processor -- start 2024-09-11T23:08:42.643 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:41.986+0000 7f942e29f640 1 -- start start 2024-09-11T23:08:42.643 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:41.987+0000 7f942e29f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94281066d0 0x7f942819af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:08:42.643 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:41.987+0000 7f942e29f640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f9428107e00 con 0x7f94281066d0 2024-09-11T23:08:42.643 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:41.987+0000 7f9427fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94281066d0 0x7f942819af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:08:42.644 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:41.987+0000 7f9427fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94281066d0 0x7f942819af80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43796/0 (socket says 172.21.15.26:43796) 2024-09-11T23:08:42.644 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:41.987+0000 7f9427fff640 1 -- 172.21.15.26:0/2748415062 learned_addr learned my addr 172.21.15.26:0/2748415062 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:08:42.644 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:41.987+0000 7f9427fff640 1 -- 172.21.15.26:0/2748415062 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f942819b4c0 con 0x7f94281066d0 2024-09-11T23:08:42.644 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:41.987+0000 7f9427fff640 1 --2- 172.21.15.26:0/2748415062 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94281066d0 0x7f942819af80 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f94180090b0 tx=0x7f94180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:08:42.644 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:41.987+0000 7f9425ffb640 1 -- 172.21.15.26:0/2748415062 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9418041070 con 0x7f94281066d0 2024-09-11T23:08:42.644 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:41.987+0000 7f9425ffb640 1 -- 172.21.15.26:0/2748415062 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f941803c7a0 con 0x7f94281066d0 2024-09-11T23:08:42.644 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:41.987+0000 7f9425ffb640 1 -- 172.21.15.26:0/2748415062 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9418037040 con 0x7f94281066d0 2024-09-11T23:08:42.644 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:41.987+0000 7f942e29f640 1 -- 172.21.15.26:0/2748415062 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f942819b750 con 0x7f94281066d0 2024-09-11T23:08:42.644 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:41.987+0000 7f942e29f640 1 -- 172.21.15.26:0/2748415062 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f942819e440 con 0x7f94281066d0 2024-09-11T23:08:42.644 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:41.988+0000 7f9425ffb640 1 -- 172.21.15.26:0/2748415062 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f9418044070 con 0x7f94281066d0 2024-09-11T23:08:42.644 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:41.988+0000 7f9425ffb640 1 -- 172.21.15.26:0/2748415062 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f941803d3f0 con 0x7f94281066d0 2024-09-11T23:08:42.644 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:41.989+0000 7f942e29f640 1 -- 172.21.15.26:0/2748415062 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9428108450 con 0x7f94281066d0 2024-09-11T23:08:42.645 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:41.991+0000 7f9425ffb640 1 -- 172.21.15.26:0/2748415062 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f9418046450 con 0x7f94281066d0 2024-09-11T23:08:42.645 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:42.050+0000 7f942e29f640 1 -- 172.21.15.26:0/2748415062 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f942819fb10 con 0x7f94281066d0 2024-09-11T23:08:42.645 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:42.054+0000 7f9425ffb640 1 -- 172.21.15.26:0/2748415062 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+429 (secure 0 0 0) 0x7f9418046630 con 0x7f94281066d0 2024-09-11T23:08:42.645 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:42.054+0000 7f9425ffb640 1 -- 172.21.15.26:0/2748415062 <== mon.0 v2:172.21.15.26:3300/0 8 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f9418056070 con 0x7f94281066d0 2024-09-11T23:08:42.645 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:42.055+0000 7f942e29f640 1 -- 172.21.15.26:0/2748415062 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94281066d0 msgr2=0x7f942819af80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:08:42.645 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:42.055+0000 7f942e29f640 1 --2- 172.21.15.26:0/2748415062 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94281066d0 0x7f942819af80 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f94180090b0 tx=0x7f94180046c0 comp rx=0 tx=0).stop 2024-09-11T23:08:42.645 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:42.055+0000 7f942e29f640 1 -- 172.21.15.26:0/2748415062 shutdown_connections 2024-09-11T23:08:42.645 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:42.055+0000 7f942e29f640 1 --2- 172.21.15.26:0/2748415062 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94281066d0 0x7f942819af80 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:08:42.645 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:42.055+0000 7f942e29f640 1 -- 172.21.15.26:0/2748415062 >> 172.21.15.26:0/2748415062 conn(0x7f9428101e60 msgr2=0x7f942807b810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:08:42.645 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:42.055+0000 7f942e29f640 1 -- 172.21.15.26:0/2748415062 shutdown_connections 2024-09-11T23:08:42.645 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:42.055+0000 7f942e29f640 1 -- 172.21.15.26:0/2748415062 wait complete. 2024-09-11T23:08:42.645 INFO:teuthology.orchestra.run.smithi026.stdout:Generating new minimal ceph.conf... 2024-09-11T23:08:42.749 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:42 smithi026 ceph-mon[30804]: from='client.? 172.21.15.26:0/2748415062' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-11T23:08:42.749 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:42 smithi026 ceph-mon[30804]: from='client.? 172.21.15.26:0/2748415062' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-09-11T23:08:43.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:43 smithi026 ceph-mon[30804]: from='client.? 172.21.15.26:0/3165113467' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:08:43.796 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:43.149+0000 7fb783af0640 1 Processor -- start 2024-09-11T23:08:43.796 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:43.149+0000 7fb783af0640 1 -- start start 2024-09-11T23:08:43.797 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:43.149+0000 7fb783af0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb77c07de00 0x7fb77c07c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:08:43.797 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:43.149+0000 7fb783af0640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb77c07c830 con 0x7fb77c07de00 2024-09-11T23:08:43.797 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:43.149+0000 7fb78188c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb77c07de00 0x7fb77c07c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:08:43.797 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:43.149+0000 7fb78188c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb77c07de00 0x7fb77c07c260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43808/0 (socket says 172.21.15.26:43808) 2024-09-11T23:08:43.797 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:43.149+0000 7fb78188c640 1 -- 172.21.15.26:0/2953561051 learned_addr learned my addr 172.21.15.26:0/2953561051 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:08:43.797 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:43.150+0000 7fb78188c640 1 -- 172.21.15.26:0/2953561051 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb77c07c9b0 con 0x7fb77c07de00 2024-09-11T23:08:43.797 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:43.150+0000 7fb78188c640 1 --2- 172.21.15.26:0/2953561051 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb77c07de00 0x7fb77c07c260 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fb778009820 tx=0x7fb77802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5316051cc8fcad35 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:08:43.797 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:43.150+0000 7fb78088a640 1 -- 172.21.15.26:0/2953561051 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb778037070 con 0x7fb77c07de00 2024-09-11T23:08:43.797 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:43.150+0000 7fb78088a640 1 -- 172.21.15.26:0/2953561051 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb778032900 con 0x7fb77c07de00 2024-09-11T23:08:43.797 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:43.150+0000 7fb78088a640 1 -- 172.21.15.26:0/2953561051 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb778030850 con 0x7fb77c07de00 2024-09-11T23:08:43.797 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:43.151+0000 7fb783af0640 1 -- 172.21.15.26:0/2953561051 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb77c07de00 msgr2=0x7fb77c07c260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:08:43.797 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:43.151+0000 7fb783af0640 1 --2- 172.21.15.26:0/2953561051 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb77c07de00 0x7fb77c07c260 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fb778009820 tx=0x7fb77802a270 comp rx=0 tx=0).stop 2024-09-11T23:08:43.797 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:43.151+0000 7fb783af0640 1 -- 172.21.15.26:0/2953561051 shutdown_connections 2024-09-11T23:08:43.798 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:43.151+0000 7fb783af0640 1 --2- 172.21.15.26:0/2953561051 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb77c07de00 0x7fb77c07c260 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:08:43.798 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:43.151+0000 7fb783af0640 1 -- 172.21.15.26:0/2953561051 >> 172.21.15.26:0/2953561051 conn(0x7fb77c102f60 msgr2=0x7fb77c1053a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:08:43.798 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:43.151+0000 7fb783af0640 1 -- 172.21.15.26:0/2953561051 shutdown_connections 2024-09-11T23:08:43.798 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:43.151+0000 7fb783af0640 1 -- 172.21.15.26:0/2953561051 wait complete. 2024-09-11T23:08:43.798 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:43.152+0000 7fb783af0640 1 Processor -- start 2024-09-11T23:08:43.798 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:43.152+0000 7fb783af0640 1 -- start start 2024-09-11T23:08:43.798 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:43.152+0000 7fb783af0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb77c07de00 0x7fb77c19d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:08:43.798 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:43.152+0000 7fb783af0640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb77c10dda0 con 0x7fb77c07de00 2024-09-11T23:08:43.798 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:43.152+0000 7fb78188c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb77c07de00 0x7fb77c19d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:08:43.798 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:43.152+0000 7fb78188c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb77c07de00 0x7fb77c19d950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43814/0 (socket says 172.21.15.26:43814) 2024-09-11T23:08:43.798 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:43.152+0000 7fb78188c640 1 -- 172.21.15.26:0/3165113467 learned_addr learned my addr 172.21.15.26:0/3165113467 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:08:43.798 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:43.152+0000 7fb78188c640 1 -- 172.21.15.26:0/3165113467 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb77c19de90 con 0x7fb77c07de00 2024-09-11T23:08:43.798 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:43.153+0000 7fb78188c640 1 --2- 172.21.15.26:0/3165113467 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb77c07de00 0x7fb77c19d950 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fb778032ce0 tx=0x7fb778032d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:08:43.799 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:43.153+0000 7fb772ffd640 1 -- 172.21.15.26:0/3165113467 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb778042070 con 0x7fb77c07de00 2024-09-11T23:08:43.799 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:43.153+0000 7fb772ffd640 1 -- 172.21.15.26:0/3165113467 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb7780308c0 con 0x7fb77c07de00 2024-09-11T23:08:43.799 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:43.153+0000 7fb783af0640 1 -- 172.21.15.26:0/3165113467 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb77c19e120 con 0x7fb77c07de00 2024-09-11T23:08:43.799 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:43.153+0000 7fb772ffd640 1 -- 172.21.15.26:0/3165113467 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb778037040 con 0x7fb77c07de00 2024-09-11T23:08:43.799 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:43.153+0000 7fb783af0640 1 -- 172.21.15.26:0/3165113467 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb77c1a0e10 con 0x7fb77c07de00 2024-09-11T23:08:43.799 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:43.153+0000 7fb772ffd640 1 -- 172.21.15.26:0/3165113467 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fb778045070 con 0x7fb77c07de00 2024-09-11T23:08:43.799 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:43.153+0000 7fb772ffd640 1 -- 172.21.15.26:0/3165113467 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fb778047c10 con 0x7fb77c07de00 2024-09-11T23:08:43.799 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:43.154+0000 7fb783af0640 1 -- 172.21.15.26:0/3165113467 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb77c07c2e0 con 0x7fb77c07de00 2024-09-11T23:08:43.799 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:43.156+0000 7fb772ffd640 1 -- 172.21.15.26:0/3165113467 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fb77803c7b0 con 0x7fb77c07de00 2024-09-11T23:08:43.799 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:43.217+0000 7fb783af0640 1 -- 172.21.15.26:0/3165113467 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7fb77c1a0fe0 con 0x7fb77c07de00 2024-09-11T23:08:43.799 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:43.217+0000 7fb772ffd640 1 -- 172.21.15.26:0/3165113467 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+175 (secure 0 0 0) 0x7fb77803c990 con 0x7fb77c07de00 2024-09-11T23:08:43.799 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:43.218+0000 7fb783af0640 1 -- 172.21.15.26:0/3165113467 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb77c07de00 msgr2=0x7fb77c19d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:08:43.799 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:43.218+0000 7fb783af0640 1 --2- 172.21.15.26:0/3165113467 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb77c07de00 0x7fb77c19d950 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fb778032ce0 tx=0x7fb778032d10 comp rx=0 tx=0).stop 2024-09-11T23:08:43.800 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:43.218+0000 7fb783af0640 1 -- 172.21.15.26:0/3165113467 shutdown_connections 2024-09-11T23:08:43.800 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:43.218+0000 7fb783af0640 1 --2- 172.21.15.26:0/3165113467 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb77c07de00 0x7fb77c19d950 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:08:43.800 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:43.218+0000 7fb783af0640 1 -- 172.21.15.26:0/3165113467 >> 172.21.15.26:0/3165113467 conn(0x7fb77c102f60 msgr2=0x7fb77c103b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:08:43.800 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:43.218+0000 7fb783af0640 1 -- 172.21.15.26:0/3165113467 shutdown_connections 2024-09-11T23:08:43.800 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:43.218+0000 7fb783af0640 1 -- 172.21.15.26:0/3165113467 wait complete. 2024-09-11T23:08:43.800 INFO:teuthology.orchestra.run.smithi026.stdout:Restarting the monitor... 2024-09-11T23:08:44.077 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:43 smithi026 systemd[1]: Stopping Ceph mon.a for af7e6fc8-7092-11ef-bcea-c7b262605968... 2024-09-11T23:08:44.442 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:44 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mon-a[30800]: 2024-09-11T23:08:44.074+0000 7f54a51da640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T23:08:44.443 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:44 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mon-a[30800]: 2024-09-11T23:08:44.074+0000 7f54a51da640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-09-11T23:08:44.443 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:44 smithi026 podman[31045]: 2024-09-11 23:08:44.199228534 +0000 UTC m=+0.224544090 container died 320eb2d81f1492e68ed7d769e43c3192466ce7ffe997ab89dc7df37a8c7cd0d1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mon-a, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, ceph=True) 2024-09-11T23:08:44.709 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:44 smithi026 podman[31045]: 2024-09-11 23:08:44.533614739 +0000 UTC m=+0.558930292 container cleanup 320eb2d81f1492e68ed7d769e43c3192466ce7ffe997ab89dc7df37a8c7cd0d1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mon-a, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-09-11T23:08:44.709 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:44 smithi026 bash[31045]: ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mon-a 2024-09-11T23:08:44.960 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:44 smithi026 podman[31059]: 2024-09-11 23:08:44.709504776 +0000 UTC m=+0.507666141 container remove 320eb2d81f1492e68ed7d769e43c3192466ce7ffe997ab89dc7df37a8c7cd0d1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mon-a, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T23:08:44.960 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:44 smithi026 systemd[1]: ceph-af7e6fc8-7092-11ef-bcea-c7b262605968@mon.a.service: Deactivated successfully. 2024-09-11T23:08:44.960 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:44 smithi026 systemd[1]: Stopped Ceph mon.a for af7e6fc8-7092-11ef-bcea-c7b262605968. 2024-09-11T23:08:44.961 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:44 smithi026 systemd[1]: Starting Ceph mon.a for af7e6fc8-7092-11ef-bcea-c7b262605968... 2024-09-11T23:08:45.238 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 podman[31152]: 2024-09-11 23:08:45.152011821 +0000 UTC m=+0.094378972 container create fb7a9fd245e408acfe118ef86b660356ee76bf85b8846bb3e771b8ed7860a140 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mon-a, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-09-11T23:08:45.238 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 podman[31152]: 2024-09-11 23:08:45.074064939 +0000 UTC m=+0.016432058 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-11T23:08:45.425 INFO:teuthology.orchestra.run.smithi026.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-09-11T23:08:45.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 podman[31152]: 2024-09-11 23:08:45.342038804 +0000 UTC m=+0.284405914 container init fb7a9fd245e408acfe118ef86b660356ee76bf85b8846bb3e771b8ed7860a140 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mon-a, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-09-11T23:08:45.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 podman[31152]: 2024-09-11 23:08:45.345261276 +0000 UTC m=+0.287628386 container start fb7a9fd245e408acfe118ef86b660356ee76bf85b8846bb3e771b8ed7860a140 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mon-a, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-09-11T23:08:45.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: set uid:gid to 167:167 (ceph:ceph) 2024-09-11T23:08:45.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable), process ceph-mon, pid 2 2024-09-11T23:08:45.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: pidfile_write: ignore empty --pid-file 2024-09-11T23:08:45.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: load: jerasure load: lrc 2024-09-11T23:08:45.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: RocksDB version: 7.9.2 2024-09-11T23:08:45.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Git sha 0 2024-09-11T23:08:45.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Compile date 2024-09-09 16:23:28 2024-09-11T23:08:45.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: DB SUMMARY 2024-09-11T23:08:45.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: DB Session ID: 17A2PAB1WHMFC4R1TT6O 2024-09-11T23:08:45.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: CURRENT file: CURRENT 2024-09-11T23:08:45.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: IDENTITY file: IDENTITY 2024-09-11T23:08:45.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-09-11T23:08:45.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-09-11T23:08:45.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 106632 ; 2024-09-11T23:08:45.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.error_if_exists: 0 2024-09-11T23:08:45.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.create_if_missing: 0 2024-09-11T23:08:45.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.paranoid_checks: 1 2024-09-11T23:08:45.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-11T23:08:45.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-11T23:08:45.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-11T23:08:45.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.env: 0x56252196d940 2024-09-11T23:08:45.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.fs: PosixFileSystem 2024-09-11T23:08:45.504 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.info_log: 0x5625235a56e0 2024-09-11T23:08:45.504 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.max_file_opening_threads: 16 2024-09-11T23:08:45.504 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.statistics: (nil) 2024-09-11T23:08:45.505 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.use_fsync: 0 2024-09-11T23:08:45.506 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.max_log_file_size: 0 2024-09-11T23:08:45.506 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-11T23:08:45.506 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-11T23:08:45.506 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.keep_log_file_num: 1000 2024-09-11T23:08:45.506 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.recycle_log_file_num: 0 2024-09-11T23:08:45.506 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.allow_fallocate: 1 2024-09-11T23:08:45.506 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.allow_mmap_reads: 0 2024-09-11T23:08:45.507 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.allow_mmap_writes: 0 2024-09-11T23:08:45.507 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.use_direct_reads: 0 2024-09-11T23:08:45.507 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-11T23:08:45.507 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.create_missing_column_families: 0 2024-09-11T23:08:45.507 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.db_log_dir: 2024-09-11T23:08:45.507 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.wal_dir: 2024-09-11T23:08:45.507 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-11T23:08:45.507 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-11T23:08:45.507 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-11T23:08:45.507 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-11T23:08:45.507 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-11T23:08:45.507 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-11T23:08:45.507 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.advise_random_on_open: 1 2024-09-11T23:08:45.508 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.db_write_buffer_size: 0 2024-09-11T23:08:45.508 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.write_buffer_manager: 0x5625235b4dc0 2024-09-11T23:08:45.508 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-11T23:08:45.508 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-11T23:08:45.508 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-11T23:08:45.508 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.rate_limiter: (nil) 2024-09-11T23:08:45.508 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-11T23:08:45.508 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.wal_recovery_mode: 2 2024-09-11T23:08:45.508 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.enable_thread_tracking: 0 2024-09-11T23:08:45.508 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.enable_pipelined_write: 0 2024-09-11T23:08:45.508 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.unordered_write: 0 2024-09-11T23:08:45.508 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-11T23:08:45.508 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-11T23:08:45.509 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-11T23:08:45.509 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-11T23:08:45.509 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.row_cache: None 2024-09-11T23:08:45.509 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.wal_filter: None 2024-09-11T23:08:45.509 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-11T23:08:45.509 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.allow_ingest_behind: 0 2024-09-11T23:08:45.509 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.two_write_queues: 0 2024-09-11T23:08:45.509 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.manual_wal_flush: 0 2024-09-11T23:08:45.509 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.wal_compression: 0 2024-09-11T23:08:45.509 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.atomic_flush: 0 2024-09-11T23:08:45.509 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-11T23:08:45.509 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-11T23:08:45.509 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-11T23:08:45.509 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.log_readahead_size: 0 2024-09-11T23:08:45.510 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-11T23:08:45.510 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.best_efforts_recovery: 0 2024-09-11T23:08:45.510 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-11T23:08:45.510 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-11T23:08:45.510 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.allow_data_in_errors: 0 2024-09-11T23:08:45.510 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.db_host_id: __hostname__ 2024-09-11T23:08:45.510 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-11T23:08:45.510 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.max_background_jobs: 2 2024-09-11T23:08:45.510 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.max_background_compactions: -1 2024-09-11T23:08:45.510 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.max_subcompactions: 1 2024-09-11T23:08:45.510 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-11T23:08:45.510 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-11T23:08:45.510 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-11T23:08:45.510 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.max_total_wal_size: 0 2024-09-11T23:08:45.511 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-11T23:08:45.511 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-11T23:08:45.511 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-11T23:08:45.511 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-11T23:08:45.511 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.max_open_files: -1 2024-09-11T23:08:45.511 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.bytes_per_sync: 0 2024-09-11T23:08:45.511 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-11T23:08:45.511 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-11T23:08:45.511 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.compaction_readahead_size: 0 2024-09-11T23:08:45.511 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.max_background_flushes: -1 2024-09-11T23:08:45.511 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Compression algorithms supported: 2024-09-11T23:08:45.511 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: kZSTD supported: 0 2024-09-11T23:08:45.511 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: kXpressCompression supported: 0 2024-09-11T23:08:45.512 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: kBZip2Compression supported: 0 2024-09-11T23:08:45.512 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-11T23:08:45.512 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: kLZ4Compression supported: 1 2024-09-11T23:08:45.512 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: kZlibCompression supported: 1 2024-09-11T23:08:45.512 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: kLZ4HCCompression supported: 1 2024-09-11T23:08:45.512 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: kSnappyCompression supported: 1 2024-09-11T23:08:45.512 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-11T23:08:45.512 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-11T23:08:45.512 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-09-11T23:08:45.512 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-11T23:08:45.512 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-11T23:08:45.512 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.merge_operator: 2024-09-11T23:08:45.512 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.compaction_filter: None 2024-09-11T23:08:45.512 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.compaction_filter_factory: None 2024-09-11T23:08:45.513 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.sst_partitioner_factory: None 2024-09-11T23:08:45.513 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-11T23:08:45.513 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-11T23:08:45.513 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5625235a5320) 2024-09-11T23:08:45.513 INFO:journalctl@ceph.mon.a.smithi026.stdout: cache_index_and_filter_blocks: 1 2024-09-11T23:08:45.513 INFO:journalctl@ceph.mon.a.smithi026.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-11T23:08:45.513 INFO:journalctl@ceph.mon.a.smithi026.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-11T23:08:45.513 INFO:journalctl@ceph.mon.a.smithi026.stdout: pin_top_level_index_and_filter: 1 2024-09-11T23:08:45.513 INFO:journalctl@ceph.mon.a.smithi026.stdout: index_type: 0 2024-09-11T23:08:45.514 INFO:journalctl@ceph.mon.a.smithi026.stdout: data_block_index_type: 0 2024-09-11T23:08:45.514 INFO:journalctl@ceph.mon.a.smithi026.stdout: index_shortening: 1 2024-09-11T23:08:45.514 INFO:journalctl@ceph.mon.a.smithi026.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-11T23:08:45.514 INFO:journalctl@ceph.mon.a.smithi026.stdout: checksum: 4 2024-09-11T23:08:45.514 INFO:journalctl@ceph.mon.a.smithi026.stdout: no_block_cache: 0 2024-09-11T23:08:45.514 INFO:journalctl@ceph.mon.a.smithi026.stdout: block_cache: 0x5625235991f0 2024-09-11T23:08:45.514 INFO:journalctl@ceph.mon.a.smithi026.stdout: block_cache_name: BinnedLRUCache 2024-09-11T23:08:45.514 INFO:journalctl@ceph.mon.a.smithi026.stdout: block_cache_options: 2024-09-11T23:08:45.514 INFO:journalctl@ceph.mon.a.smithi026.stdout: capacity : 536870912 2024-09-11T23:08:45.514 INFO:journalctl@ceph.mon.a.smithi026.stdout: num_shard_bits : 4 2024-09-11T23:08:45.514 INFO:journalctl@ceph.mon.a.smithi026.stdout: strict_capacity_limit : 0 2024-09-11T23:08:45.514 INFO:journalctl@ceph.mon.a.smithi026.stdout: high_pri_pool_ratio: 0.000 2024-09-11T23:08:45.514 INFO:journalctl@ceph.mon.a.smithi026.stdout: block_cache_compressed: (nil) 2024-09-11T23:08:45.515 INFO:journalctl@ceph.mon.a.smithi026.stdout: persistent_cache: (nil) 2024-09-11T23:08:45.515 INFO:journalctl@ceph.mon.a.smithi026.stdout: block_size: 4096 2024-09-11T23:08:45.515 INFO:journalctl@ceph.mon.a.smithi026.stdout: block_size_deviation: 10 2024-09-11T23:08:45.515 INFO:journalctl@ceph.mon.a.smithi026.stdout: block_restart_interval: 16 2024-09-11T23:08:45.515 INFO:journalctl@ceph.mon.a.smithi026.stdout: index_block_restart_interval: 1 2024-09-11T23:08:45.515 INFO:journalctl@ceph.mon.a.smithi026.stdout: metadata_block_size: 4096 2024-09-11T23:08:45.515 INFO:journalctl@ceph.mon.a.smithi026.stdout: partition_filters: 0 2024-09-11T23:08:45.515 INFO:journalctl@ceph.mon.a.smithi026.stdout: use_delta_encoding: 1 2024-09-11T23:08:45.515 INFO:journalctl@ceph.mon.a.smithi026.stdout: filter_policy: bloomfilter 2024-09-11T23:08:45.515 INFO:journalctl@ceph.mon.a.smithi026.stdout: whole_key_filtering: 1 2024-09-11T23:08:45.515 INFO:journalctl@ceph.mon.a.smithi026.stdout: verify_compression: 0 2024-09-11T23:08:45.515 INFO:journalctl@ceph.mon.a.smithi026.stdout: read_amp_bytes_per_bit: 0 2024-09-11T23:08:45.515 INFO:journalctl@ceph.mon.a.smithi026.stdout: format_version: 5 2024-09-11T23:08:45.515 INFO:journalctl@ceph.mon.a.smithi026.stdout: enable_index_compression: 1 2024-09-11T23:08:45.516 INFO:journalctl@ceph.mon.a.smithi026.stdout: block_align: 0 2024-09-11T23:08:45.516 INFO:journalctl@ceph.mon.a.smithi026.stdout: max_auto_readahead_size: 262144 2024-09-11T23:08:45.516 INFO:journalctl@ceph.mon.a.smithi026.stdout: prepopulate_block_cache: 0 2024-09-11T23:08:45.516 INFO:journalctl@ceph.mon.a.smithi026.stdout: initial_auto_readahead_size: 8192 2024-09-11T23:08:45.516 INFO:journalctl@ceph.mon.a.smithi026.stdout: num_file_reads_for_auto_readahead: 2 2024-09-11T23:08:45.516 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.write_buffer_size: 33554432 2024-09-11T23:08:45.516 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.max_write_buffer_number: 2 2024-09-11T23:08:45.516 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.compression: NoCompression 2024-09-11T23:08:45.516 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.bottommost_compression: Disabled 2024-09-11T23:08:45.516 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.prefix_extractor: nullptr 2024-09-11T23:08:45.516 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-11T23:08:45.516 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.num_levels: 7 2024-09-11T23:08:45.516 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-11T23:08:45.516 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-11T23:08:45.517 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-11T23:08:45.517 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-11T23:08:45.517 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-11T23:08:45.517 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-11T23:08:45.517 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-11T23:08:45.517 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-11T23:08:45.517 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-11T23:08:45.517 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-11T23:08:45.517 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-11T23:08:45.518 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-11T23:08:45.518 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-11T23:08:45.518 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.compression_opts.level: 32767 2024-09-11T23:08:45.518 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.compression_opts.strategy: 0 2024-09-11T23:08:45.518 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-11T23:08:45.518 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-11T23:08:45.518 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-11T23:08:45.518 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-11T23:08:45.518 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.compression_opts.enabled: false 2024-09-11T23:08:45.518 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-11T23:08:45.518 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-11T23:08:45.518 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-11T23:08:45.518 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-11T23:08:45.519 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.target_file_size_base: 67108864 2024-09-11T23:08:45.519 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-11T23:08:45.519 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-11T23:08:45.519 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-11T23:08:45.519 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-11T23:08:45.519 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-11T23:08:45.519 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-11T23:08:45.519 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-11T23:08:45.519 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-11T23:08:45.519 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-11T23:08:45.519 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-11T23:08:45.519 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-11T23:08:45.519 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-11T23:08:45.519 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-11T23:08:45.520 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-11T23:08:45.520 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.arena_block_size: 1048576 2024-09-11T23:08:45.520 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-11T23:08:45.520 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-11T23:08:45.520 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.disable_auto_compactions: 0 2024-09-11T23:08:45.520 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-11T23:08:45.520 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-11T23:08:45.520 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-11T23:08:45.520 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-11T23:08:45.520 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-11T23:08:45.520 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-11T23:08:45.520 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-11T23:08:45.520 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-11T23:08:45.520 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-11T23:08:45.521 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-11T23:08:45.521 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-11T23:08:45.521 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.inplace_update_support: 0 2024-09-11T23:08:45.521 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-11T23:08:45.521 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-11T23:08:45.521 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-11T23:08:45.521 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-11T23:08:45.521 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.bloom_locality: 0 2024-09-11T23:08:45.521 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.max_successive_merges: 0 2024-09-11T23:08:45.521 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-11T23:08:45.521 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.paranoid_file_checks: 0 2024-09-11T23:08:45.521 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.force_consistency_checks: 1 2024-09-11T23:08:45.521 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.report_bg_io_stats: 0 2024-09-11T23:08:45.522 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.ttl: 2592000 2024-09-11T23:08:45.522 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-11T23:08:45.522 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-11T23:08:45.522 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-11T23:08:45.522 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.enable_blob_files: false 2024-09-11T23:08:45.522 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.min_blob_size: 0 2024-09-11T23:08:45.522 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.blob_file_size: 268435456 2024-09-11T23:08:45.522 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-11T23:08:45.522 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-11T23:08:45.522 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-11T23:08:45.522 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-11T23:08:45.522 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-11T23:08:45.522 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.blob_file_starting_level: 0 2024-09-11T23:08:45.522 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-11T23:08:45.523 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-09-11T23:08:45.523 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-09-11T23:08:45.523 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 08ec5c94-12a6-49ed-b579-b9c98393e82d 2024-09-11T23:08:45.523 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726096125372483, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-09-11T23:08:45.523 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-09-11T23:08:45.523 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726096125373590, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 101520, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 334, "table_properties": {"data_size": 99405, "index_size": 307, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14641, "raw_average_key_size": 49, "raw_value_size": 91536, "raw_average_value_size": 312, "num_data_blocks": 12, "num_entries": 293, "num_filter_entries": 293, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726096125, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "08ec5c94-12a6-49ed-b579-b9c98393e82d", "db_session_id": "17A2PAB1WHMFC4R1TT6O", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-09-11T23:08:45.523 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726096125373695, "job": 1, "event": "recovery_finished"} 2024-09-11T23:08:45.523 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-09-11T23:08:45.523 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-11T23:08:45.523 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x5625235c6e00 2024-09-11T23:08:45.523 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: DB pointer 0x5625236c2000 2024-09-11T23:08:45.523 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-11T23:08:45.523 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-11T23:08:45.524 INFO:journalctl@ceph.mon.a.smithi026.stdout: ** DB Stats ** 2024-09-11T23:08:45.524 INFO:journalctl@ceph.mon.a.smithi026.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-11T23:08:45.524 INFO:journalctl@ceph.mon.a.smithi026.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-11T23:08:45.524 INFO:journalctl@ceph.mon.a.smithi026.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-11T23:08:45.524 INFO:journalctl@ceph.mon.a.smithi026.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-11T23:08:45.524 INFO:journalctl@ceph.mon.a.smithi026.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-11T23:08:45.524 INFO:journalctl@ceph.mon.a.smithi026.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-11T23:08:45.524 INFO:journalctl@ceph.mon.a.smithi026.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-11T23:08:45.524 INFO:journalctl@ceph.mon.a.smithi026.stdout: 2024-09-11T23:08:45.524 INFO:journalctl@ceph.mon.a.smithi026.stdout: ** Compaction Stats [default] ** 2024-09-11T23:08:45.524 INFO:journalctl@ceph.mon.a.smithi026.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-11T23:08:45.524 INFO:journalctl@ceph.mon.a.smithi026.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-11T23:08:45.524 INFO:journalctl@ceph.mon.a.smithi026.stdout: L0 2/0 101.00 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 140.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-11T23:08:45.524 INFO:journalctl@ceph.mon.a.smithi026.stdout: Sum 2/0 101.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 140.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-11T23:08:45.525 INFO:journalctl@ceph.mon.a.smithi026.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 140.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-11T23:08:45.525 INFO:journalctl@ceph.mon.a.smithi026.stdout: 2024-09-11T23:08:45.525 INFO:journalctl@ceph.mon.a.smithi026.stdout: ** Compaction Stats [default] ** 2024-09-11T23:08:45.525 INFO:journalctl@ceph.mon.a.smithi026.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-11T23:08:45.525 INFO:journalctl@ceph.mon.a.smithi026.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-11T23:08:45.525 INFO:journalctl@ceph.mon.a.smithi026.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 140.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-11T23:08:45.525 INFO:journalctl@ceph.mon.a.smithi026.stdout: 2024-09-11T23:08:45.525 INFO:journalctl@ceph.mon.a.smithi026.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-11T23:08:45.525 INFO:journalctl@ceph.mon.a.smithi026.stdout: 2024-09-11T23:08:45.525 INFO:journalctl@ceph.mon.a.smithi026.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-11T23:08:45.525 INFO:journalctl@ceph.mon.a.smithi026.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-11T23:08:45.525 INFO:journalctl@ceph.mon.a.smithi026.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-11T23:08:45.525 INFO:journalctl@ceph.mon.a.smithi026.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-11T23:08:45.526 INFO:journalctl@ceph.mon.a.smithi026.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-11T23:08:45.526 INFO:journalctl@ceph.mon.a.smithi026.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-11T23:08:45.526 INFO:journalctl@ceph.mon.a.smithi026.stdout: Cumulative compaction: 0.00 GB write, 29.75 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-11T23:08:45.526 INFO:journalctl@ceph.mon.a.smithi026.stdout: Interval compaction: 0.00 GB write, 29.75 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-11T23:08:45.526 INFO:journalctl@ceph.mon.a.smithi026.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-11T23:08:45.526 INFO:journalctl@ceph.mon.a.smithi026.stdout: Block cache BinnedLRUCache@0x5625235991f0#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 8e-06 secs_since: 0 2024-09-11T23:08:45.526 INFO:journalctl@ceph.mon.a.smithi026.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-09-11T23:08:45.526 INFO:journalctl@ceph.mon.a.smithi026.stdout: 2024-09-11T23:08:45.526 INFO:journalctl@ceph.mon.a.smithi026.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-11T23:08:45.526 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: starting mon.a rank 0 at public addrs [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] at bind addrs [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid af7e6fc8-7092-11ef-bcea-c7b262605968 2024-09-11T23:08:45.526 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: mon.a@-1(???) e1 preinit fsid af7e6fc8-7092-11ef-bcea-c7b262605968 2024-09-11T23:08:45.526 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: mon.a@-1(???).mds e1 new map 2024-09-11T23:08:45.526 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: mon.a@-1(???).mds e1 print_map 2024-09-11T23:08:45.526 INFO:journalctl@ceph.mon.a.smithi026.stdout: e1 2024-09-11T23:08:45.526 INFO:journalctl@ceph.mon.a.smithi026.stdout: btime 2024-09-11T23:08:40:486266+0000 2024-09-11T23:08:45.527 INFO:journalctl@ceph.mon.a.smithi026.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-11T23:08:45.527 INFO:journalctl@ceph.mon.a.smithi026.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-11T23:08:45.527 INFO:journalctl@ceph.mon.a.smithi026.stdout: legacy client fscid: -1 2024-09-11T23:08:45.527 INFO:journalctl@ceph.mon.a.smithi026.stdout: 2024-09-11T23:08:45.527 INFO:journalctl@ceph.mon.a.smithi026.stdout: No filesystems configured 2024-09-11T23:08:45.527 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-11T23:08:45.527 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-11T23:08:45.527 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-11T23:08:45.527 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-11T23:08:45.527 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 ceph-mon[31172]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-09-11T23:08:45.527 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 bash[31152]: fb7a9fd245e408acfe118ef86b660356ee76bf85b8846bb3e771b8ed7860a140 2024-09-11T23:08:45.527 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:45 smithi026 systemd[1]: Started Ceph mon.a for af7e6fc8-7092-11ef-bcea-c7b262605968. 2024-09-11T23:08:46.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:46 smithi026 ceph-mon[31172]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-11T23:08:46.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:46 smithi026 ceph-mon[31172]: monmap epoch 1 2024-09-11T23:08:46.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:46 smithi026 ceph-mon[31172]: fsid af7e6fc8-7092-11ef-bcea-c7b262605968 2024-09-11T23:08:46.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:46 smithi026 ceph-mon[31172]: last_changed 2024-09-11T23:08:37.446176+0000 2024-09-11T23:08:46.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:46 smithi026 ceph-mon[31172]: created 2024-09-11T23:08:37.446176+0000 2024-09-11T23:08:46.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:46 smithi026 ceph-mon[31172]: min_mon_release 19 (squid) 2024-09-11T23:08:46.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:46 smithi026 ceph-mon[31172]: election_strategy: 1 2024-09-11T23:08:46.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:46 smithi026 ceph-mon[31172]: 0: [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] mon.a 2024-09-11T23:08:46.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:46 smithi026 ceph-mon[31172]: fsmap 2024-09-11T23:08:46.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:46 smithi026 ceph-mon[31172]: osdmap e1: 0 total, 0 up, 0 in 2024-09-11T23:08:46.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:46 smithi026 ceph-mon[31172]: mgrmap e1: no daemons active 2024-09-11T23:08:46.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:46 smithi026 ceph-mon[31172]: from='client.? 172.21.15.26:0/1083855875' entity='client.admin' 2024-09-11T23:08:48.568 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:45.864+0000 7ff7aecd0640 1 Processor -- start 2024-09-11T23:08:48.568 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:45.864+0000 7ff7aecd0640 1 -- start start 2024-09-11T23:08:48.568 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:45.865+0000 7ff7aecd0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff7a81082c0 0x7ff7a810a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:08:48.569 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:45.865+0000 7ff7aecd0640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff7a807bdb0 con 0x7ff7a81082c0 2024-09-11T23:08:48.569 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:45.865+0000 7ff7aca6c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff7a81082c0 0x7ff7a810a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:08:48.569 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:45.865+0000 7ff7aca6c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff7a81082c0 0x7ff7a810a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43820/0 (socket says 172.21.15.26:43820) 2024-09-11T23:08:48.569 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:45.865+0000 7ff7aca6c640 1 -- 172.21.15.26:0/4032540157 learned_addr learned my addr 172.21.15.26:0/4032540157 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:08:48.569 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:45.865+0000 7ff7aca6c640 1 -- 172.21.15.26:0/4032540157 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7a810abf0 con 0x7ff7a81082c0 2024-09-11T23:08:48.569 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:45.868+0000 7ff7aca6c640 1 --2- 172.21.15.26:0/4032540157 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff7a81082c0 0x7ff7a810a6b0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7ff7a0009e80 tx=0x7ff7a002a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6382a4fa9e318e3a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:08:48.569 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:45.868+0000 7ff7a77fe640 1 -- 172.21.15.26:0/4032540157 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff7a0036070 con 0x7ff7a81082c0 2024-09-11T23:08:48.573 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:45.868+0000 7ff7a77fe640 1 -- 172.21.15.26:0/4032540157 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff7a00318c0 con 0x7ff7a81082c0 2024-09-11T23:08:48.573 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:45.868+0000 7ff7a77fe640 1 -- 172.21.15.26:0/4032540157 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff7a0007ab0 con 0x7ff7a81082c0 2024-09-11T23:08:48.573 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:45.868+0000 7ff7aecd0640 1 -- 172.21.15.26:0/4032540157 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff7a81082c0 msgr2=0x7ff7a810a6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:08:48.573 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:45.868+0000 7ff7aecd0640 1 --2- 172.21.15.26:0/4032540157 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff7a81082c0 0x7ff7a810a6b0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7ff7a0009e80 tx=0x7ff7a002a7a0 comp rx=0 tx=0).stop 2024-09-11T23:08:48.573 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:45.868+0000 7ff7aecd0640 1 -- 172.21.15.26:0/4032540157 shutdown_connections 2024-09-11T23:08:48.573 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:45.868+0000 7ff7aecd0640 1 --2- 172.21.15.26:0/4032540157 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff7a81082c0 0x7ff7a810a6b0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:08:48.573 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:45.868+0000 7ff7aecd0640 1 -- 172.21.15.26:0/4032540157 >> 172.21.15.26:0/4032540157 conn(0x7ff7a8101e60 msgr2=0x7ff7a81042a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:08:48.574 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:45.868+0000 7ff7aecd0640 1 -- 172.21.15.26:0/4032540157 shutdown_connections 2024-09-11T23:08:48.574 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:45.868+0000 7ff7aecd0640 1 -- 172.21.15.26:0/4032540157 wait complete. 2024-09-11T23:08:48.574 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:45.869+0000 7ff7aecd0640 1 Processor -- start 2024-09-11T23:08:48.574 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:45.869+0000 7ff7aecd0640 1 -- start start 2024-09-11T23:08:48.574 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:45.869+0000 7ff7aecd0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff7a81082c0 0x7ff7a819d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:08:48.574 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:45.869+0000 7ff7aecd0640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff7a810bb60 con 0x7ff7a81082c0 2024-09-11T23:08:48.574 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:45.869+0000 7ff7aca6c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff7a81082c0 0x7ff7a819d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:08:48.574 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:45.869+0000 7ff7aca6c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff7a81082c0 0x7ff7a819d9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43828/0 (socket says 172.21.15.26:43828) 2024-09-11T23:08:48.574 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:45.869+0000 7ff7aca6c640 1 -- 172.21.15.26:0/1083855875 learned_addr learned my addr 172.21.15.26:0/1083855875 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:08:48.574 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:45.870+0000 7ff7aca6c640 1 -- 172.21.15.26:0/1083855875 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7a819dee0 con 0x7ff7a81082c0 2024-09-11T23:08:48.574 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:45.870+0000 7ff7aca6c640 1 --2- 172.21.15.26:0/1083855875 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff7a81082c0 0x7ff7a819d9a0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7ff7a0031ca0 tx=0x7ff7a0031cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:08:48.574 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:45.870+0000 7ff7a5ffb640 1 -- 172.21.15.26:0/1083855875 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff7a0036050 con 0x7ff7a81082c0 2024-09-11T23:08:48.574 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:45.870+0000 7ff7aecd0640 1 -- 172.21.15.26:0/1083855875 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7a819e170 con 0x7ff7a81082c0 2024-09-11T23:08:48.575 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:45.870+0000 7ff7a5ffb640 1 -- 172.21.15.26:0/1083855875 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff7a0042070 con 0x7ff7a81082c0 2024-09-11T23:08:48.575 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:45.870+0000 7ff7a5ffb640 1 -- 172.21.15.26:0/1083855875 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff7a003d820 con 0x7ff7a81082c0 2024-09-11T23:08:48.575 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:45.870+0000 7ff7aecd0640 1 -- 172.21.15.26:0/1083855875 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7a81a0e60 con 0x7ff7a81082c0 2024-09-11T23:08:48.575 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:45.870+0000 7ff7a5ffb640 1 -- 172.21.15.26:0/1083855875 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7ff7a003db20 con 0x7ff7a81082c0 2024-09-11T23:08:48.575 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:45.870+0000 7ff7a5ffb640 1 -- 172.21.15.26:0/1083855875 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7ff7a0007e00 con 0x7ff7a81082c0 2024-09-11T23:08:48.575 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:45.871+0000 7ff7aecd0640 1 -- 172.21.15.26:0/1083855875 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7a8108720 con 0x7ff7a81082c0 2024-09-11T23:08:48.575 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:45.873+0000 7ff7a5ffb640 1 -- 172.21.15.26:0/1083855875 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7ff7a00072f0 con 0x7ff7a81082c0 2024-09-11T23:08:48.575 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:45.933+0000 7ff7aecd0640 1 -- 172.21.15.26:0/1083855875 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7ff7a81a1030 con 0x7ff7a81082c0 2024-09-11T23:08:48.575 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:45.935+0000 7ff7a5ffb640 1 -- 172.21.15.26:0/1083855875 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7ff7a0047460 con 0x7ff7a81082c0 2024-09-11T23:08:48.575 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:45.935+0000 7ff7aecd0640 1 -- 172.21.15.26:0/1083855875 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff7a81082c0 msgr2=0x7ff7a819d9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:08:48.575 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:45.935+0000 7ff7aecd0640 1 --2- 172.21.15.26:0/1083855875 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff7a81082c0 0x7ff7a819d9a0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7ff7a0031ca0 tx=0x7ff7a0031cd0 comp rx=0 tx=0).stop 2024-09-11T23:08:48.575 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:45.935+0000 7ff7aecd0640 1 -- 172.21.15.26:0/1083855875 shutdown_connections 2024-09-11T23:08:48.576 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:45.935+0000 7ff7aecd0640 1 --2- 172.21.15.26:0/1083855875 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff7a81082c0 0x7ff7a819d9a0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:08:48.576 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:45.935+0000 7ff7aecd0640 1 -- 172.21.15.26:0/1083855875 >> 172.21.15.26:0/1083855875 conn(0x7ff7a8101e60 msgr2=0x7ff7a8102a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:08:48.576 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:45.935+0000 7ff7aecd0640 1 -- 172.21.15.26:0/1083855875 shutdown_connections 2024-09-11T23:08:48.576 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:45.936+0000 7ff7aecd0640 1 -- 172.21.15.26:0/1083855875 wait complete. 2024-09-11T23:08:48.576 INFO:teuthology.orchestra.run.smithi026.stdout:Wrote config to /etc/ceph/ceph.conf 2024-09-11T23:08:48.832 INFO:teuthology.orchestra.run.smithi026.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-09-11T23:08:48.832 INFO:teuthology.orchestra.run.smithi026.stdout:Creating mgr... 2024-09-11T23:08:48.832 INFO:teuthology.orchestra.run.smithi026.stdout:Verifying port 0.0.0.0:9283 ... 2024-09-11T23:08:48.833 INFO:teuthology.orchestra.run.smithi026.stdout:Verifying port 0.0.0.0:8765 ... 2024-09-11T23:08:49.714 INFO:teuthology.orchestra.run.smithi026.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-af7e6fc8-7092-11ef-bcea-c7b262605968@mgr.x 2024-09-11T23:08:49.715 INFO:teuthology.orchestra.run.smithi026.stdout:systemctl: stderr Failed to reset failed state of unit ceph-af7e6fc8-7092-11ef-bcea-c7b262605968@mgr.x.service: Unit ceph-af7e6fc8-7092-11ef-bcea-c7b262605968@mgr.x.service not loaded. 2024-09-11T23:08:50.081 INFO:teuthology.orchestra.run.smithi026.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-af7e6fc8-7092-11ef-bcea-c7b262605968.target.wants/ceph-af7e6fc8-7092-11ef-bcea-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-af7e6fc8-7092-11ef-bcea-c7b262605968@.service. 2024-09-11T23:08:50.501 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:08:50 smithi026 podman[31383]: 2024-09-11 23:08:50.22172218 +0000 UTC m=+0.016176916 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-11T23:08:51.501 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:08:51 smithi026 podman[31383]: 2024-09-11 23:08:51.238631756 +0000 UTC m=+1.033086491 container create 865f5a94615ba4a6d0c97dcd45d8e7009aebc46cdd202cab1917878c343caac6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T23:08:53.501 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:08:53 smithi026 podman[31383]: 2024-09-11 23:08:53.17638515 +0000 UTC m=+2.970839885 container init 865f5a94615ba4a6d0c97dcd45d8e7009aebc46cdd202cab1917878c343caac6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, GIT_CLEAN=True) 2024-09-11T23:08:53.501 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:08:53 smithi026 podman[31383]: 2024-09-11 23:08:53.180300452 +0000 UTC m=+2.974755188 container start 865f5a94615ba4a6d0c97dcd45d8e7009aebc46cdd202cab1917878c343caac6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3) 2024-09-11T23:08:53.501 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:08:53 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: 2024-09-11T23:08:53.318+0000 7fa2c63dc040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-11T23:08:53.501 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:08:53 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: 2024-09-11T23:08:53.377+0000 7fa2c63dc040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-11T23:08:53.906 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:08:53 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: 2024-09-11T23:08:53.534+0000 7fa2c63dc040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-11T23:08:53.906 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:08:53 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: 2024-09-11T23:08:53.604+0000 7fa2c63dc040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-11T23:08:53.906 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:08:53 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: 2024-09-11T23:08:53.759+0000 7fa2c63dc040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-11T23:08:53.906 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:08:53 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: 2024-09-11T23:08:53.906+0000 7fa2c63dc040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-11T23:08:54.123 INFO:teuthology.orchestra.run.smithi026.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-11T23:08:54.124 INFO:teuthology.orchestra.run.smithi026.stdout:systemctl: stdout disabled 2024-09-11T23:08:54.131 INFO:teuthology.orchestra.run.smithi026.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-11T23:08:54.132 INFO:teuthology.orchestra.run.smithi026.stdout:systemctl: stdout inactive 2024-09-11T23:08:54.132 INFO:teuthology.orchestra.run.smithi026.stdout:firewalld.service is not enabled 2024-09-11T23:08:54.132 INFO:teuthology.orchestra.run.smithi026.stdout:Not possible to enable service . firewalld.service is not available 2024-09-11T23:08:54.142 INFO:teuthology.orchestra.run.smithi026.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-11T23:08:54.142 INFO:teuthology.orchestra.run.smithi026.stdout:systemctl: stdout disabled 2024-09-11T23:08:54.150 INFO:teuthology.orchestra.run.smithi026.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-11T23:08:54.150 INFO:teuthology.orchestra.run.smithi026.stdout:systemctl: stdout inactive 2024-09-11T23:08:54.150 INFO:teuthology.orchestra.run.smithi026.stdout:firewalld.service is not enabled 2024-09-11T23:08:54.150 INFO:teuthology.orchestra.run.smithi026.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-09-11T23:08:54.150 INFO:teuthology.orchestra.run.smithi026.stdout:Waiting for mgr to start... 2024-09-11T23:08:54.150 INFO:teuthology.orchestra.run.smithi026.stdout:Waiting for mgr... 2024-09-11T23:08:54.251 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:08:54 smithi026 bash[31383]: 865f5a94615ba4a6d0c97dcd45d8e7009aebc46cdd202cab1917878c343caac6 2024-09-11T23:08:54.251 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:08:54 smithi026 systemd[1]: Started Ceph mgr.x for af7e6fc8-7092-11ef-bcea-c7b262605968. 2024-09-11T23:08:54.512 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:08:54 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: 2024-09-11T23:08:54.254+0000 7fa2c63dc040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-11T23:08:54.512 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:08:54 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: 2024-09-11T23:08:54.318+0000 7fa2c63dc040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-11T23:08:55.001 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:08:54 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: 2024-09-11T23:08:54.511+0000 7fa2c63dc040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-11T23:08:55.751 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:08:55 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: 2024-09-11T23:08:55.449+0000 7fa2c63dc040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-11T23:08:55.751 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:08:55 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: 2024-09-11T23:08:55.521+0000 7fa2c63dc040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-11T23:08:56.405 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:08:56 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: 2024-09-11T23:08:56.284+0000 7fa2c63dc040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-11T23:08:56.405 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:08:56 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: 2024-09-11T23:08:56.405+0000 7fa2c63dc040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-11T23:08:56.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:56 smithi026 ceph-mon[31172]: from='client.? 172.21.15.26:0/1086066170' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-11T23:08:56.751 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:08:56 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: 2024-09-11T23:08:56.559+0000 7fa2c63dc040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-11T23:08:56.751 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:08:56 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: 2024-09-11T23:08:56.627+0000 7fa2c63dc040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-11T23:08:57.095 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:08:56 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: 2024-09-11T23:08:56.791+0000 7fa2c63dc040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-11T23:08:57.095 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:08:56 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: 2024-09-11T23:08:56.863+0000 7fa2c63dc040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-11T23:08:57.095 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:08:56 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: 2024-09-11T23:08:56.927+0000 7fa2c63dc040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-11T23:08:57.095 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:08:57 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: 2024-09-11T23:08:57.094+0000 7fa2c63dc040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-11T23:08:57.249 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout 2024-09-11T23:08:57.252 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout { 2024-09-11T23:08:57.252 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "fsid": "af7e6fc8-7092-11ef-bcea-c7b262605968", 2024-09-11T23:08:57.252 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "health": { 2024-09-11T23:08:57.252 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-11T23:08:57.252 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-11T23:08:57.252 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-11T23:08:57.253 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout }, 2024-09-11T23:08:57.253 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-11T23:08:57.253 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-11T23:08:57.253 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout 0 2024-09-11T23:08:57.253 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout ], 2024-09-11T23:08:57.253 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-11T23:08:57.253 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "a" 2024-09-11T23:08:57.253 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout ], 2024-09-11T23:08:57.253 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "quorum_age": 11, 2024-09-11T23:08:57.253 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-11T23:08:57.253 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T23:08:57.254 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-11T23:08:57.254 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-11T23:08:57.254 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout }, 2024-09-11T23:08:57.254 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-11T23:08:57.254 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T23:08:57.254 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-11T23:08:57.254 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-11T23:08:57.254 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-11T23:08:57.262 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-11T23:08:57.262 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-11T23:08:57.262 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-11T23:08:57.262 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout }, 2024-09-11T23:08:57.262 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-11T23:08:57.262 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-11T23:08:57.262 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-11T23:08:57.262 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-11T23:08:57.262 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-11T23:08:57.262 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-11T23:08:57.263 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-11T23:08:57.263 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-11T23:08:57.263 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-11T23:08:57.263 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout }, 2024-09-11T23:08:57.263 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-11T23:08:57.263 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T23:08:57.263 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "btime": "2024-09-11T23:08:40:486266+0000", 2024-09-11T23:08:57.263 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-11T23:08:57.263 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-11T23:08:57.263 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout }, 2024-09-11T23:08:57.263 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-11T23:08:57.263 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-11T23:08:57.263 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-11T23:08:57.264 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-11T23:08:57.264 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-11T23:08:57.264 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-11T23:08:57.264 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "restful" 2024-09-11T23:08:57.264 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout ], 2024-09-11T23:08:57.264 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-11T23:08:57.264 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout }, 2024-09-11T23:08:57.264 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-11T23:08:57.264 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T23:08:57.264 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "modified": "2024-09-11T23:08:40.486946+0000", 2024-09-11T23:08:57.264 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-11T23:08:57.264 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout }, 2024-09-11T23:08:57.264 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-11T23:08:57.265 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout } 2024-09-11T23:08:57.265 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:56.532+0000 7fde17e1d640 1 Processor -- start 2024-09-11T23:08:57.265 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:56.532+0000 7fde17e1d640 1 -- start start 2024-09-11T23:08:57.265 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:56.532+0000 7fde17e1d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde1007de20 0x7fde1007c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:08:57.265 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:56.532+0000 7fde17e1d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fde1007c850 con 0x7fde1007de20 2024-09-11T23:08:57.265 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:56.532+0000 7fde15bb9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde1007de20 0x7fde1007c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:08:57.265 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:56.532+0000 7fde15bb9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde1007de20 0x7fde1007c280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55594/0 (socket says 172.21.15.26:55594) 2024-09-11T23:08:57.265 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:56.532+0000 7fde15bb9640 1 -- 172.21.15.26:0/3609040943 learned_addr learned my addr 172.21.15.26:0/3609040943 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:08:57.265 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:56.533+0000 7fde15bb9640 1 -- 172.21.15.26:0/3609040943 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde1007c9d0 con 0x7fde1007de20 2024-09-11T23:08:57.265 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:56.533+0000 7fde15bb9640 1 --2- 172.21.15.26:0/3609040943 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde1007de20 0x7fde1007c280 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fde0c009820 tx=0x7fde0c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff08af79aa51ff01 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:08:57.265 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:56.534+0000 7fde14bb7640 1 -- 172.21.15.26:0/3609040943 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fde0c037070 con 0x7fde1007de20 2024-09-11T23:08:57.265 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:56.534+0000 7fde14bb7640 1 -- 172.21.15.26:0/3609040943 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fde0c032900 con 0x7fde1007de20 2024-09-11T23:08:57.266 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:56.534+0000 7fde14bb7640 1 -- 172.21.15.26:0/3609040943 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fde0c030850 con 0x7fde1007de20 2024-09-11T23:08:57.266 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:56.534+0000 7fde17e1d640 1 -- 172.21.15.26:0/3609040943 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde1007de20 msgr2=0x7fde1007c280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:08:57.266 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:56.534+0000 7fde17e1d640 1 --2- 172.21.15.26:0/3609040943 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde1007de20 0x7fde1007c280 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fde0c009820 tx=0x7fde0c02a270 comp rx=0 tx=0).stop 2024-09-11T23:08:57.266 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:56.534+0000 7fde17e1d640 1 -- 172.21.15.26:0/3609040943 shutdown_connections 2024-09-11T23:08:57.266 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:56.534+0000 7fde17e1d640 1 --2- 172.21.15.26:0/3609040943 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde1007de20 0x7fde1007c280 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:08:57.266 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:56.534+0000 7fde17e1d640 1 -- 172.21.15.26:0/3609040943 >> 172.21.15.26:0/3609040943 conn(0x7fde10102fc0 msgr2=0x7fde101053e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:08:57.266 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:56.535+0000 7fde17e1d640 1 -- 172.21.15.26:0/3609040943 shutdown_connections 2024-09-11T23:08:57.266 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:56.535+0000 7fde17e1d640 1 -- 172.21.15.26:0/3609040943 wait complete. 2024-09-11T23:08:57.266 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:56.535+0000 7fde17e1d640 1 Processor -- start 2024-09-11T23:08:57.266 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:56.535+0000 7fde17e1d640 1 -- start start 2024-09-11T23:08:57.266 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:56.536+0000 7fde17e1d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde1007de20 0x7fde1019d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:08:57.266 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:56.536+0000 7fde17e1d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fde1010dde0 con 0x7fde1007de20 2024-09-11T23:08:57.266 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:56.536+0000 7fde15bb9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde1007de20 0x7fde1019d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:08:57.267 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:56.536+0000 7fde15bb9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde1007de20 0x7fde1019d9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55606/0 (socket says 172.21.15.26:55606) 2024-09-11T23:08:57.267 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:56.536+0000 7fde15bb9640 1 -- 172.21.15.26:0/1086066170 learned_addr learned my addr 172.21.15.26:0/1086066170 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:08:57.267 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:56.536+0000 7fde15bb9640 1 -- 172.21.15.26:0/1086066170 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde1019dee0 con 0x7fde1007de20 2024-09-11T23:08:57.267 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:56.536+0000 7fde15bb9640 1 --2- 172.21.15.26:0/1086066170 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde1007de20 0x7fde1019d9a0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fde0c0097f0 tx=0x7fde0c030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:08:57.267 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:56.536+0000 7fde06ffd640 1 -- 172.21.15.26:0/1086066170 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fde0c037040 con 0x7fde1007de20 2024-09-11T23:08:57.267 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:56.536+0000 7fde17e1d640 1 -- 172.21.15.26:0/1086066170 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde1019e170 con 0x7fde1007de20 2024-09-11T23:08:57.267 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:56.536+0000 7fde06ffd640 1 -- 172.21.15.26:0/1086066170 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fde0c040070 con 0x7fde1007de20 2024-09-11T23:08:57.267 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:56.536+0000 7fde06ffd640 1 -- 172.21.15.26:0/1086066170 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fde0c03a390 con 0x7fde1007de20 2024-09-11T23:08:57.267 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:56.537+0000 7fde17e1d640 1 -- 172.21.15.26:0/1086066170 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde101a0e60 con 0x7fde1007de20 2024-09-11T23:08:57.267 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:56.537+0000 7fde06ffd640 1 -- 172.21.15.26:0/1086066170 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fde0c03a630 con 0x7fde1007de20 2024-09-11T23:08:57.268 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:56.537+0000 7fde06ffd640 1 -- 172.21.15.26:0/1086066170 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fde0c032a90 con 0x7fde1007de20 2024-09-11T23:08:57.268 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:56.538+0000 7fde17e1d640 1 -- 172.21.15.26:0/1086066170 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde1007c300 con 0x7fde1007de20 2024-09-11T23:08:57.268 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:56.540+0000 7fde06ffd640 1 -- 172.21.15.26:0/1086066170 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fde0c045460 con 0x7fde1007de20 2024-09-11T23:08:57.268 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:56.602+0000 7fde17e1d640 1 -- 172.21.15.26:0/1086066170 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fde101a10d0 con 0x7fde1007de20 2024-09-11T23:08:57.268 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:56.602+0000 7fde06ffd640 1 -- 172.21.15.26:0/1086066170 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1292 (secure 0 0 0) 0x7fde0c03b430 con 0x7fde1007de20 2024-09-11T23:08:57.268 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:56.603+0000 7fde17e1d640 1 -- 172.21.15.26:0/1086066170 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde1007de20 msgr2=0x7fde1019d9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:08:57.268 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:56.603+0000 7fde17e1d640 1 --2- 172.21.15.26:0/1086066170 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde1007de20 0x7fde1019d9a0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fde0c0097f0 tx=0x7fde0c030d80 comp rx=0 tx=0).stop 2024-09-11T23:08:57.268 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:56.603+0000 7fde17e1d640 1 -- 172.21.15.26:0/1086066170 shutdown_connections 2024-09-11T23:08:57.268 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:56.603+0000 7fde17e1d640 1 --2- 172.21.15.26:0/1086066170 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde1007de20 0x7fde1019d9a0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:08:57.269 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:56.603+0000 7fde17e1d640 1 -- 172.21.15.26:0/1086066170 >> 172.21.15.26:0/1086066170 conn(0x7fde10102fc0 msgr2=0x7fde10103bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:08:57.269 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:56.603+0000 7fde17e1d640 1 -- 172.21.15.26:0/1086066170 shutdown_connections 2024-09-11T23:08:57.269 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:56.603+0000 7fde17e1d640 1 -- 172.21.15.26:0/1086066170 wait complete. 2024-09-11T23:08:57.269 INFO:teuthology.orchestra.run.smithi026.stdout:mgr not available, waiting (1/15)... 2024-09-11T23:08:57.383 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:08:57 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: 2024-09-11T23:08:57.163+0000 7fa2c63dc040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-11T23:08:57.383 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:08:57 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: 2024-09-11T23:08:57.232+0000 7fa2c63dc040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-11T23:08:57.384 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:08:57 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: 2024-09-11T23:08:57.321+0000 7fa2c63dc040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-11T23:08:57.384 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:08:57 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: 2024-09-11T23:08:57.383+0000 7fa2c63dc040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-11T23:08:57.751 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:08:57 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: 2024-09-11T23:08:57.445+0000 7fa2c63dc040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-11T23:08:58.751 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:08:58 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: 2024-09-11T23:08:58.271+0000 7fa2c63dc040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-11T23:08:58.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:58 smithi026 ceph-mon[31172]: Activating manager daemon x 2024-09-11T23:08:58.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:58 smithi026 ceph-mon[31172]: mgrmap e2: x(active, starting, since 0.00319307s) 2024-09-11T23:08:58.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:58 smithi026 ceph-mon[31172]: from='mgr.14100 172.21.15.26:0/2470627938' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-11T23:08:58.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:58 smithi026 ceph-mon[31172]: from='mgr.14100 172.21.15.26:0/2470627938' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-11T23:08:58.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:58 smithi026 ceph-mon[31172]: from='mgr.14100 172.21.15.26:0/2470627938' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-11T23:08:58.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:58 smithi026 ceph-mon[31172]: from='mgr.14100 172.21.15.26:0/2470627938' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T23:08:58.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:58 smithi026 ceph-mon[31172]: from='mgr.14100 172.21.15.26:0/2470627938' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-11T23:08:58.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:58 smithi026 ceph-mon[31172]: Manager daemon x is now available 2024-09-11T23:08:58.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:58 smithi026 ceph-mon[31172]: from='mgr.14100 172.21.15.26:0/2470627938' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-11T23:08:58.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:58 smithi026 ceph-mon[31172]: from='mgr.14100 172.21.15.26:0/2470627938' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-11T23:08:58.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:58 smithi026 ceph-mon[31172]: from='mgr.14100 172.21.15.26:0/2470627938' entity='mgr.x' 2024-09-11T23:08:58.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:58 smithi026 ceph-mon[31172]: from='mgr.14100 172.21.15.26:0/2470627938' entity='mgr.x' 2024-09-11T23:08:58.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:08:58 smithi026 ceph-mon[31172]: from='mgr.14100 172.21.15.26:0/2470627938' entity='mgr.x' 2024-09-11T23:09:00.473 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout 2024-09-11T23:09:00.474 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout { 2024-09-11T23:09:00.474 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "fsid": "af7e6fc8-7092-11ef-bcea-c7b262605968", 2024-09-11T23:09:00.474 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "health": { 2024-09-11T23:09:00.474 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-11T23:09:00.474 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-11T23:09:00.474 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-11T23:09:00.474 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout }, 2024-09-11T23:09:00.474 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-11T23:09:00.474 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-11T23:09:00.474 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout 0 2024-09-11T23:09:00.474 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout ], 2024-09-11T23:09:00.474 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-11T23:09:00.475 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "a" 2024-09-11T23:09:00.475 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout ], 2024-09-11T23:09:00.475 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "quorum_age": 14, 2024-09-11T23:09:00.475 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-11T23:09:00.475 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T23:09:00.475 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-11T23:09:00.475 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-11T23:09:00.475 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout }, 2024-09-11T23:09:00.475 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-11T23:09:00.475 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T23:09:00.475 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-11T23:09:00.475 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-11T23:09:00.475 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-11T23:09:00.475 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-11T23:09:00.476 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-11T23:09:00.476 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-11T23:09:00.476 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout }, 2024-09-11T23:09:00.476 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-11T23:09:00.476 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-11T23:09:00.476 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-11T23:09:00.476 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-11T23:09:00.476 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-11T23:09:00.476 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-11T23:09:00.476 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-11T23:09:00.476 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-11T23:09:00.476 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-11T23:09:00.476 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout }, 2024-09-11T23:09:00.476 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-11T23:09:00.477 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T23:09:00.477 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "btime": "2024-09-11T23:08:40:486266+0000", 2024-09-11T23:09:00.477 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-11T23:09:00.477 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-11T23:09:00.477 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout }, 2024-09-11T23:09:00.477 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-11T23:09:00.477 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-11T23:09:00.477 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-11T23:09:00.477 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-11T23:09:00.477 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-11T23:09:00.477 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-11T23:09:00.477 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "restful" 2024-09-11T23:09:00.477 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout ], 2024-09-11T23:09:00.478 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-11T23:09:00.478 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout }, 2024-09-11T23:09:00.478 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-11T23:09:00.478 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T23:09:00.478 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "modified": "2024-09-11T23:08:40.486946+0000", 2024-09-11T23:09:00.478 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-11T23:09:00.478 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout }, 2024-09-11T23:09:00.478 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-11T23:09:00.478 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout } 2024-09-11T23:09:00.478 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:59.644+0000 7fe899f14640 1 Processor -- start 2024-09-11T23:09:00.478 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:59.644+0000 7fe899f14640 1 -- start start 2024-09-11T23:09:00.478 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:59.644+0000 7fe899f14640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe89407de20 0x7fe89407c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:09:00.478 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:59.644+0000 7fe899f14640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fe89407c850 con 0x7fe89407de20 2024-09-11T23:09:00.479 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:59.645+0000 7fe8937fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe89407de20 0x7fe89407c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:09:00.479 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:59.645+0000 7fe8937fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe89407de20 0x7fe89407c280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55676/0 (socket says 172.21.15.26:55676) 2024-09-11T23:09:00.479 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:59.645+0000 7fe8937fe640 1 -- 172.21.15.26:0/2497495198 learned_addr learned my addr 172.21.15.26:0/2497495198 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:09:00.479 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:59.645+0000 7fe8937fe640 1 -- 172.21.15.26:0/2497495198 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe89407c9d0 con 0x7fe89407de20 2024-09-11T23:09:00.479 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:59.646+0000 7fe8937fe640 1 --2- 172.21.15.26:0/2497495198 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe89407de20 0x7fe89407c280 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fe884009820 tx=0x7fe88402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee317cf82d46167 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:09:00.479 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:59.646+0000 7fe8927fc640 1 -- 172.21.15.26:0/2497495198 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe884037070 con 0x7fe89407de20 2024-09-11T23:09:00.479 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:59.646+0000 7fe8927fc640 1 -- 172.21.15.26:0/2497495198 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe884032900 con 0x7fe89407de20 2024-09-11T23:09:00.479 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:59.646+0000 7fe8927fc640 1 -- 172.21.15.26:0/2497495198 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe884030850 con 0x7fe89407de20 2024-09-11T23:09:00.479 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:59.647+0000 7fe899f14640 1 -- 172.21.15.26:0/2497495198 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe89407de20 msgr2=0x7fe89407c280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:09:00.479 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:59.647+0000 7fe899f14640 1 --2- 172.21.15.26:0/2497495198 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe89407de20 0x7fe89407c280 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fe884009820 tx=0x7fe88402a270 comp rx=0 tx=0).stop 2024-09-11T23:09:00.479 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:59.647+0000 7fe899f14640 1 -- 172.21.15.26:0/2497495198 shutdown_connections 2024-09-11T23:09:00.479 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:59.647+0000 7fe899f14640 1 --2- 172.21.15.26:0/2497495198 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe89407de20 0x7fe89407c280 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:09:00.479 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:59.647+0000 7fe899f14640 1 -- 172.21.15.26:0/2497495198 >> 172.21.15.26:0/2497495198 conn(0x7fe894102f60 msgr2=0x7fe8941053a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:09:00.480 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:59.647+0000 7fe899f14640 1 -- 172.21.15.26:0/2497495198 shutdown_connections 2024-09-11T23:09:00.480 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:59.647+0000 7fe899f14640 1 -- 172.21.15.26:0/2497495198 wait complete. 2024-09-11T23:09:00.480 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:59.647+0000 7fe899f14640 1 Processor -- start 2024-09-11T23:09:00.480 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:59.648+0000 7fe899f14640 1 -- start start 2024-09-11T23:09:00.480 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:59.648+0000 7fe899f14640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe89407de20 0x7fe89419d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:09:00.480 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:59.648+0000 7fe899f14640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fe89410dde0 con 0x7fe89407de20 2024-09-11T23:09:00.480 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:59.648+0000 7fe8937fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe89407de20 0x7fe89419d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:09:00.480 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:59.648+0000 7fe8937fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe89407de20 0x7fe89419d9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55688/0 (socket says 172.21.15.26:55688) 2024-09-11T23:09:00.480 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:59.648+0000 7fe8937fe640 1 -- 172.21.15.26:0/545735000 learned_addr learned my addr 172.21.15.26:0/545735000 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:09:00.480 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:59.648+0000 7fe8937fe640 1 -- 172.21.15.26:0/545735000 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe89419df10 con 0x7fe89407de20 2024-09-11T23:09:00.480 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:59.648+0000 7fe8937fe640 1 --2- 172.21.15.26:0/545735000 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe89407de20 0x7fe89419d9d0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fe8840097f0 tx=0x7fe884030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:09:00.480 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:59.649+0000 7fe890ff9640 1 -- 172.21.15.26:0/545735000 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe884037040 con 0x7fe89407de20 2024-09-11T23:09:00.480 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:59.649+0000 7fe890ff9640 1 -- 172.21.15.26:0/545735000 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe884040070 con 0x7fe89407de20 2024-09-11T23:09:00.481 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:59.649+0000 7fe899f14640 1 -- 172.21.15.26:0/545735000 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe89419e1a0 con 0x7fe89407de20 2024-09-11T23:09:00.481 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:59.649+0000 7fe890ff9640 1 -- 172.21.15.26:0/545735000 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe88403a390 con 0x7fe89407de20 2024-09-11T23:09:00.481 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:59.649+0000 7fe899f14640 1 -- 172.21.15.26:0/545735000 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8941a0e90 con 0x7fe89407de20 2024-09-11T23:09:00.481 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:59.650+0000 7fe890ff9640 1 -- 172.21.15.26:0/545735000 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 3) ==== 46170+0+0 (secure 0 0 0) 0x7fe884032900 con 0x7fe89407de20 2024-09-11T23:09:00.481 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:59.650+0000 7fe890ff9640 1 --2- 172.21.15.26:0/545735000 >> [v2:172.21.15.26:6800/2673575521,v1:172.21.15.26:6801/2673575521] conn(0x7fe87c038e60 0x7fe87c03b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:09:00.481 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:59.650+0000 7fe899f14640 1 -- 172.21.15.26:0/545735000 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe89407c300 con 0x7fe89407de20 2024-09-11T23:09:00.481 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:59.650+0000 7fe890ff9640 1 -- 172.21.15.26:0/545735000 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fe88406be40 con 0x7fe89407de20 2024-09-11T23:09:00.481 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:59.650+0000 7fe892ffd640 1 --2- 172.21.15.26:0/545735000 >> [v2:172.21.15.26:6800/2673575521,v1:172.21.15.26:6801/2673575521] conn(0x7fe87c038e60 0x7fe87c03b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:09:00.481 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:59.650+0000 7fe892ffd640 1 --2- 172.21.15.26:0/545735000 >> [v2:172.21.15.26:6800/2673575521,v1:172.21.15.26:6801/2673575521] conn(0x7fe87c038e60 0x7fe87c03b320 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fe8880098c0 tx=0x7fe888006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:09:00.481 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:59.655+0000 7fe890ff9640 1 -- 172.21.15.26:0/545735000 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe884032bf0 con 0x7fe89407de20 2024-09-11T23:09:00.481 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:59.901+0000 7fe899f14640 1 -- 172.21.15.26:0/545735000 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fe8941a1060 con 0x7fe89407de20 2024-09-11T23:09:00.481 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:59.901+0000 7fe890ff9640 1 -- 172.21.15.26:0/545735000 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1291 (secure 0 0 0) 0x7fe884044ca0 con 0x7fe89407de20 2024-09-11T23:09:00.481 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:59.905+0000 7fe899f14640 1 -- 172.21.15.26:0/545735000 >> [v2:172.21.15.26:6800/2673575521,v1:172.21.15.26:6801/2673575521] conn(0x7fe87c038e60 msgr2=0x7fe87c03b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:09:00.482 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:59.905+0000 7fe899f14640 1 --2- 172.21.15.26:0/545735000 >> [v2:172.21.15.26:6800/2673575521,v1:172.21.15.26:6801/2673575521] conn(0x7fe87c038e60 0x7fe87c03b320 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fe8880098c0 tx=0x7fe888006db0 comp rx=0 tx=0).stop 2024-09-11T23:09:00.482 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:59.905+0000 7fe899f14640 1 -- 172.21.15.26:0/545735000 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe89407de20 msgr2=0x7fe89419d9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:09:00.482 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:59.905+0000 7fe899f14640 1 --2- 172.21.15.26:0/545735000 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe89407de20 0x7fe89419d9d0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fe8840097f0 tx=0x7fe884030d80 comp rx=0 tx=0).stop 2024-09-11T23:09:00.482 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:59.905+0000 7fe899f14640 1 -- 172.21.15.26:0/545735000 shutdown_connections 2024-09-11T23:09:00.482 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:59.905+0000 7fe899f14640 1 --2- 172.21.15.26:0/545735000 >> [v2:172.21.15.26:6800/2673575521,v1:172.21.15.26:6801/2673575521] conn(0x7fe87c038e60 0x7fe87c03b320 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:09:00.482 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:59.905+0000 7fe899f14640 1 --2- 172.21.15.26:0/545735000 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe89407de20 0x7fe89419d9d0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:09:00.482 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:59.905+0000 7fe899f14640 1 -- 172.21.15.26:0/545735000 >> 172.21.15.26:0/545735000 conn(0x7fe894102f60 msgr2=0x7fe894103b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:09:00.482 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:59.905+0000 7fe899f14640 1 -- 172.21.15.26:0/545735000 shutdown_connections 2024-09-11T23:09:00.482 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:08:59.905+0000 7fe899f14640 1 -- 172.21.15.26:0/545735000 wait complete. 2024-09-11T23:09:00.482 INFO:teuthology.orchestra.run.smithi026.stdout:mgr is available 2024-09-11T23:09:00.586 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:09:00 smithi026 ceph-mon[31172]: mgrmap e3: x(active, since 1.00764s) 2024-09-11T23:09:00.586 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:09:00 smithi026 ceph-mon[31172]: from='client.? 172.21.15.26:0/545735000' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-11T23:09:01.708 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:09:01 smithi026 ceph-mon[31172]: mgrmap e4: x(active, since 2s) 2024-09-11T23:09:01.708 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:09:01 smithi026 ceph-mon[31172]: from='client.? 172.21.15.26:0/3299190008' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-11T23:09:04.016 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout 2024-09-11T23:09:04.202 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout [global] 2024-09-11T23:09:04.202 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout fsid = af7e6fc8-7092-11ef-bcea-c7b262605968 2024-09-11T23:09:04.203 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.26:3300,v1:172.21.15.26:6789] 2024-09-11T23:09:04.203 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-11T23:09:04.203 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-11T23:09:04.203 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-11T23:09:04.203 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-11T23:09:04.203 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout 2024-09-11T23:09:04.203 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-11T23:09:04.203 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-11T23:09:04.203 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout 2024-09-11T23:09:04.203 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout [osd] 2024-09-11T23:09:04.203 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-11T23:09:04.203 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-11T23:09:04.203 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-11T23:09:04.204 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-11T23:09:04.204 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-11T23:09:04.204 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:00.870+0000 7f98c90ff640 1 Processor -- start 2024-09-11T23:09:04.204 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:00.870+0000 7f98c90ff640 1 -- start start 2024-09-11T23:09:04.204 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:00.870+0000 7f98c90ff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f98c41077d0 0x7f98c4107bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:09:04.204 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:00.870+0000 7f98c90ff640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f98c41081a0 con 0x7f98c41077d0 2024-09-11T23:09:04.204 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:00.870+0000 7f98c2d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f98c41077d0 0x7f98c4107bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:09:04.204 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:00.870+0000 7f98c2d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f98c41077d0 0x7f98c4107bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59052/0 (socket says 172.21.15.26:59052) 2024-09-11T23:09:04.204 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:00.870+0000 7f98c2d9d640 1 -- 172.21.15.26:0/2288687627 learned_addr learned my addr 172.21.15.26:0/2288687627 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:09:04.204 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:00.871+0000 7f98c2d9d640 1 -- 172.21.15.26:0/2288687627 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98c41089d0 con 0x7f98c41077d0 2024-09-11T23:09:04.204 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:00.871+0000 7f98c2d9d640 1 --2- 172.21.15.26:0/2288687627 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f98c41077d0 0x7f98c4107bd0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f98b4009820 tx=0x7f98b402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76d9a7e2bbd572c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:09:04.204 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:00.872+0000 7f98c1d9b640 1 -- 172.21.15.26:0/2288687627 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f98b4037070 con 0x7f98c41077d0 2024-09-11T23:09:04.205 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:00.872+0000 7f98c1d9b640 1 -- 172.21.15.26:0/2288687627 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f98b4032900 con 0x7f98c41077d0 2024-09-11T23:09:04.205 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:00.872+0000 7f98c1d9b640 1 -- 172.21.15.26:0/2288687627 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f98b4030850 con 0x7f98c41077d0 2024-09-11T23:09:04.205 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:00.872+0000 7f98c90ff640 1 -- 172.21.15.26:0/2288687627 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f98c41077d0 msgr2=0x7f98c4107bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:09:04.205 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:00.872+0000 7f98c90ff640 1 --2- 172.21.15.26:0/2288687627 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f98c41077d0 0x7f98c4107bd0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f98b4009820 tx=0x7f98b402a270 comp rx=0 tx=0).stop 2024-09-11T23:09:04.205 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:00.872+0000 7f98c90ff640 1 -- 172.21.15.26:0/2288687627 shutdown_connections 2024-09-11T23:09:04.205 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:00.872+0000 7f98c90ff640 1 --2- 172.21.15.26:0/2288687627 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f98c41077d0 0x7f98c4107bd0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:09:04.205 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:00.872+0000 7f98c90ff640 1 -- 172.21.15.26:0/2288687627 >> 172.21.15.26:0/2288687627 conn(0x7f98c4102f60 msgr2=0x7f98c41053a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:09:04.205 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:00.872+0000 7f98c90ff640 1 -- 172.21.15.26:0/2288687627 shutdown_connections 2024-09-11T23:09:04.205 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:00.872+0000 7f98c90ff640 1 -- 172.21.15.26:0/2288687627 wait complete. 2024-09-11T23:09:04.208 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:00.873+0000 7f98c90ff640 1 Processor -- start 2024-09-11T23:09:04.208 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:00.873+0000 7f98c90ff640 1 -- start start 2024-09-11T23:09:04.208 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:00.873+0000 7f98c90ff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f98c41077d0 0x7f98c407daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:09:04.208 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:00.873+0000 7f98c90ff640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f98c410bd80 con 0x7f98c41077d0 2024-09-11T23:09:04.208 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:00.874+0000 7f98c2d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f98c41077d0 0x7f98c407daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:09:04.208 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:00.874+0000 7f98c2d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f98c41077d0 0x7f98c407daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59062/0 (socket says 172.21.15.26:59062) 2024-09-11T23:09:04.209 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:00.874+0000 7f98c2d9d640 1 -- 172.21.15.26:0/3299190008 learned_addr learned my addr 172.21.15.26:0/3299190008 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:09:04.209 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:00.874+0000 7f98c2d9d640 1 -- 172.21.15.26:0/3299190008 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98c407e030 con 0x7f98c41077d0 2024-09-11T23:09:04.209 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:00.874+0000 7f98c2d9d640 1 --2- 172.21.15.26:0/3299190008 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f98c41077d0 0x7f98c407daf0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f98b4009940 tx=0x7f98b4032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:09:04.209 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:00.874+0000 7f98abfff640 1 -- 172.21.15.26:0/3299190008 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f98b4042070 con 0x7f98c41077d0 2024-09-11T23:09:04.209 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:00.874+0000 7f98c90ff640 1 -- 172.21.15.26:0/3299190008 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98c407c220 con 0x7f98c41077d0 2024-09-11T23:09:04.209 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:00.874+0000 7f98abfff640 1 -- 172.21.15.26:0/3299190008 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f98b40308c0 con 0x7f98c41077d0 2024-09-11T23:09:04.209 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:00.874+0000 7f98abfff640 1 -- 172.21.15.26:0/3299190008 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f98b4037040 con 0x7f98c41077d0 2024-09-11T23:09:04.209 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:00.874+0000 7f98c90ff640 1 -- 172.21.15.26:0/3299190008 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98c407c760 con 0x7f98c41077d0 2024-09-11T23:09:04.209 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:00.875+0000 7f98abfff640 1 -- 172.21.15.26:0/3299190008 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7f98b4045070 con 0x7f98c41077d0 2024-09-11T23:09:04.209 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:00.876+0000 7f98abfff640 1 --2- 172.21.15.26:0/3299190008 >> [v2:172.21.15.26:6800/2673575521,v1:172.21.15.26:6801/2673575521] conn(0x7f98ac038f20 0x7f98ac03b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:09:04.209 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:00.876+0000 7f98abfff640 1 -- 172.21.15.26:0/3299190008 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f98b406bfe0 con 0x7f98c41077d0 2024-09-11T23:09:04.209 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:00.876+0000 7f98c90ff640 1 -- 172.21.15.26:0/3299190008 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98c4107c50 con 0x7f98c41077d0 2024-09-11T23:09:04.209 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:00.876+0000 7f98c259c640 1 --2- 172.21.15.26:0/3299190008 >> [v2:172.21.15.26:6800/2673575521,v1:172.21.15.26:6801/2673575521] conn(0x7f98ac038f20 0x7f98ac03b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:09:04.210 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:00.876+0000 7f98c259c640 1 --2- 172.21.15.26:0/3299190008 >> [v2:172.21.15.26:6800/2673575521,v1:172.21.15.26:6801/2673575521] conn(0x7f98ac038f20 0x7f98ac03b3e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f98b80098c0 tx=0x7f98b8006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:09:04.210 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:00.881+0000 7f98abfff640 1 -- 172.21.15.26:0/3299190008 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f98b403cd90 con 0x7f98c41077d0 2024-09-11T23:09:04.210 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:01.056+0000 7f98c90ff640 1 -- 172.21.15.26:0/3299190008 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f98c407cf10 con 0x7f98c41077d0 2024-09-11T23:09:04.212 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:01.057+0000 7f98abfff640 1 -- 172.21.15.26:0/3299190008 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+429 (secure 0 0 0) 0x7f98b406ba00 con 0x7f98c41077d0 2024-09-11T23:09:04.212 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:01.059+0000 7f98c90ff640 1 -- 172.21.15.26:0/3299190008 >> [v2:172.21.15.26:6800/2673575521,v1:172.21.15.26:6801/2673575521] conn(0x7f98ac038f20 msgr2=0x7f98ac03b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:09:04.212 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:01.059+0000 7f98c90ff640 1 --2- 172.21.15.26:0/3299190008 >> [v2:172.21.15.26:6800/2673575521,v1:172.21.15.26:6801/2673575521] conn(0x7f98ac038f20 0x7f98ac03b3e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f98b80098c0 tx=0x7f98b8006db0 comp rx=0 tx=0).stop 2024-09-11T23:09:04.212 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:01.059+0000 7f98c90ff640 1 -- 172.21.15.26:0/3299190008 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f98c41077d0 msgr2=0x7f98c407daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:09:04.212 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:01.059+0000 7f98c90ff640 1 --2- 172.21.15.26:0/3299190008 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f98c41077d0 0x7f98c407daf0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f98b4009940 tx=0x7f98b4032ce0 comp rx=0 tx=0).stop 2024-09-11T23:09:04.212 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:01.059+0000 7f98c90ff640 1 -- 172.21.15.26:0/3299190008 shutdown_connections 2024-09-11T23:09:04.212 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:01.059+0000 7f98c90ff640 1 --2- 172.21.15.26:0/3299190008 >> [v2:172.21.15.26:6800/2673575521,v1:172.21.15.26:6801/2673575521] conn(0x7f98ac038f20 0x7f98ac03b3e0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:09:04.212 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:01.059+0000 7f98c90ff640 1 --2- 172.21.15.26:0/3299190008 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f98c41077d0 0x7f98c407daf0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:09:04.212 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:01.059+0000 7f98c90ff640 1 -- 172.21.15.26:0/3299190008 >> 172.21.15.26:0/3299190008 conn(0x7f98c4102f60 msgr2=0x7f98c4103b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:09:04.212 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:01.059+0000 7f98c90ff640 1 -- 172.21.15.26:0/3299190008 shutdown_connections 2024-09-11T23:09:04.212 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:01.059+0000 7f98c90ff640 1 -- 172.21.15.26:0/3299190008 wait complete. 2024-09-11T23:09:04.212 INFO:teuthology.orchestra.run.smithi026.stdout:Enabling cephadm module... 2024-09-11T23:09:04.891 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:09:04 smithi026 ceph-mon[31172]: from='client.? 172.21.15.26:0/161582989' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-09-11T23:09:04.891 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:09:04 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: ignoring --setuser ceph since I am not root 2024-09-11T23:09:04.891 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:09:04 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: ignoring --setgroup ceph since I am not root 2024-09-11T23:09:04.892 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:09:04 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: 2024-09-11T23:09:04.831+0000 7fb671a4a040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-11T23:09:04.892 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:09:04 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: 2024-09-11T23:09:04.890+0000 7fb671a4a040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-11T23:09:05.251 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:09:05 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: 2024-09-11T23:09:05.047+0000 7fb671a4a040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-11T23:09:05.251 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:09:05 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: 2024-09-11T23:09:05.116+0000 7fb671a4a040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-11T23:09:05.271 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:04.401+0000 7f018a844640 1 Processor -- start 2024-09-11T23:09:05.271 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:04.401+0000 7f018a844640 1 -- start start 2024-09-11T23:09:05.271 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:04.401+0000 7f018a844640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f01841077d0 0x7f0184107bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:09:05.271 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:04.401+0000 7f018a844640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f01841081a0 con 0x7f01841077d0 2024-09-11T23:09:05.271 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:04.401+0000 7f0183fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f01841077d0 0x7f0184107bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:09:05.272 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:04.401+0000 7f0183fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f01841077d0 0x7f0184107bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59068/0 (socket says 172.21.15.26:59068) 2024-09-11T23:09:05.272 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:04.401+0000 7f0183fff640 1 -- 172.21.15.26:0/669634096 learned_addr learned my addr 172.21.15.26:0/669634096 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:09:05.272 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:04.402+0000 7f0183fff640 1 -- 172.21.15.26:0/669634096 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01841089d0 con 0x7f01841077d0 2024-09-11T23:09:05.272 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:04.402+0000 7f0183fff640 1 --2- 172.21.15.26:0/669634096 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f01841077d0 0x7f0184107bd0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f0174009820 tx=0x7f017402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70c598bab6475154 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:09:05.272 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:04.403+0000 7f0182ffd640 1 -- 172.21.15.26:0/669634096 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0174037070 con 0x7f01841077d0 2024-09-11T23:09:05.272 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:04.403+0000 7f0182ffd640 1 -- 172.21.15.26:0/669634096 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0174032900 con 0x7f01841077d0 2024-09-11T23:09:05.272 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:04.403+0000 7f0182ffd640 1 -- 172.21.15.26:0/669634096 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0174030850 con 0x7f01841077d0 2024-09-11T23:09:05.272 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:04.403+0000 7f018a844640 1 -- 172.21.15.26:0/669634096 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f01841077d0 msgr2=0x7f0184107bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:09:05.272 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:04.403+0000 7f018a844640 1 --2- 172.21.15.26:0/669634096 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f01841077d0 0x7f0184107bd0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f0174009820 tx=0x7f017402a270 comp rx=0 tx=0).stop 2024-09-11T23:09:05.272 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:04.403+0000 7f018a844640 1 -- 172.21.15.26:0/669634096 shutdown_connections 2024-09-11T23:09:05.272 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:04.403+0000 7f018a844640 1 --2- 172.21.15.26:0/669634096 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f01841077d0 0x7f0184107bd0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:09:05.272 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:04.403+0000 7f018a844640 1 -- 172.21.15.26:0/669634096 >> 172.21.15.26:0/669634096 conn(0x7f0184102f60 msgr2=0x7f01841053a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:09:05.273 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:04.404+0000 7f018a844640 1 -- 172.21.15.26:0/669634096 shutdown_connections 2024-09-11T23:09:05.273 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:04.404+0000 7f018a844640 1 -- 172.21.15.26:0/669634096 wait complete. 2024-09-11T23:09:05.273 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:04.404+0000 7f018a844640 1 Processor -- start 2024-09-11T23:09:05.273 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:04.404+0000 7f018a844640 1 -- start start 2024-09-11T23:09:05.273 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:04.405+0000 7f018a844640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f01841077d0 0x7f018407daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:09:05.273 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:04.405+0000 7f018a844640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f018410bd80 con 0x7f01841077d0 2024-09-11T23:09:05.273 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:04.405+0000 7f0183fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f01841077d0 0x7f018407daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:09:05.273 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:04.405+0000 7f0183fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f01841077d0 0x7f018407daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59080/0 (socket says 172.21.15.26:59080) 2024-09-11T23:09:05.273 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:04.405+0000 7f0183fff640 1 -- 172.21.15.26:0/161582989 learned_addr learned my addr 172.21.15.26:0/161582989 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:09:05.273 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:04.405+0000 7f0183fff640 1 -- 172.21.15.26:0/161582989 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f018407e030 con 0x7f01841077d0 2024-09-11T23:09:05.273 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:04.405+0000 7f0183fff640 1 --2- 172.21.15.26:0/161582989 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f01841077d0 0x7f018407daf0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f01740097f0 tx=0x7f0174030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:09:05.273 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:04.405+0000 7f01817fa640 1 -- 172.21.15.26:0/161582989 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0174037040 con 0x7f01841077d0 2024-09-11T23:09:05.273 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:04.405+0000 7f01817fa640 1 -- 172.21.15.26:0/161582989 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0174040070 con 0x7f01841077d0 2024-09-11T23:09:05.274 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:04.405+0000 7f018a844640 1 -- 172.21.15.26:0/161582989 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f018407c220 con 0x7f01841077d0 2024-09-11T23:09:05.274 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:04.405+0000 7f01817fa640 1 -- 172.21.15.26:0/161582989 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f017403a390 con 0x7f01841077d0 2024-09-11T23:09:05.274 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:04.405+0000 7f018a844640 1 -- 172.21.15.26:0/161582989 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f018407c700 con 0x7f01841077d0 2024-09-11T23:09:05.274 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:04.406+0000 7f01817fa640 1 -- 172.21.15.26:0/161582989 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7f017403a630 con 0x7f01841077d0 2024-09-11T23:09:05.274 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:04.407+0000 7f018a844640 1 -- 172.21.15.26:0/161582989 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0184107c50 con 0x7f01841077d0 2024-09-11T23:09:05.274 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:04.407+0000 7f01817fa640 1 --2- 172.21.15.26:0/161582989 >> [v2:172.21.15.26:6800/2673575521,v1:172.21.15.26:6801/2673575521] conn(0x7f016c038f20 0x7f016c03b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:09:05.274 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:04.407+0000 7f01817fa640 1 -- 172.21.15.26:0/161582989 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f017406c350 con 0x7f01841077d0 2024-09-11T23:09:05.274 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:04.407+0000 7f01837fe640 1 --2- 172.21.15.26:0/161582989 >> [v2:172.21.15.26:6800/2673575521,v1:172.21.15.26:6801/2673575521] conn(0x7f016c038f20 0x7f016c03b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:09:05.274 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:04.407+0000 7f01837fe640 1 --2- 172.21.15.26:0/161582989 >> [v2:172.21.15.26:6800/2673575521,v1:172.21.15.26:6801/2673575521] conn(0x7f016c038f20 0x7f016c03b3e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f01780098c0 tx=0x7f0178006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:09:05.274 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:04.411+0000 7f01817fa640 1 -- 172.21.15.26:0/161582989 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f017406c0b0 con 0x7f01841077d0 2024-09-11T23:09:05.274 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:04.635+0000 7f018a844640 1 -- 172.21.15.26:0/161582989 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7f018407cfe0 con 0x7f01841077d0 2024-09-11T23:09:05.274 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:04.687+0000 7f01817fa640 1 -- 172.21.15.26:0/161582989 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7f0174043760 con 0x7f01841077d0 2024-09-11T23:09:05.274 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:04.688+0000 7f01817fa640 1 -- 172.21.15.26:0/161582989 <== mon.0 v2:172.21.15.26:3300/0 8 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7f017402fd40 con 0x7f01841077d0 2024-09-11T23:09:05.275 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:04.690+0000 7f018a844640 1 -- 172.21.15.26:0/161582989 >> [v2:172.21.15.26:6800/2673575521,v1:172.21.15.26:6801/2673575521] conn(0x7f016c038f20 msgr2=0x7f016c03b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:09:05.275 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:04.690+0000 7f018a844640 1 --2- 172.21.15.26:0/161582989 >> [v2:172.21.15.26:6800/2673575521,v1:172.21.15.26:6801/2673575521] conn(0x7f016c038f20 0x7f016c03b3e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f01780098c0 tx=0x7f0178006db0 comp rx=0 tx=0).stop 2024-09-11T23:09:05.275 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:04.690+0000 7f018a844640 1 -- 172.21.15.26:0/161582989 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f01841077d0 msgr2=0x7f018407daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:09:05.275 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:04.690+0000 7f018a844640 1 --2- 172.21.15.26:0/161582989 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f01841077d0 0x7f018407daf0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f01740097f0 tx=0x7f0174030d80 comp rx=0 tx=0).stop 2024-09-11T23:09:05.275 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:04.690+0000 7f018a844640 1 -- 172.21.15.26:0/161582989 shutdown_connections 2024-09-11T23:09:05.275 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:04.690+0000 7f018a844640 1 --2- 172.21.15.26:0/161582989 >> [v2:172.21.15.26:6800/2673575521,v1:172.21.15.26:6801/2673575521] conn(0x7f016c038f20 0x7f016c03b3e0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:09:05.275 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:04.690+0000 7f018a844640 1 --2- 172.21.15.26:0/161582989 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f01841077d0 0x7f018407daf0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:09:05.275 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:04.690+0000 7f018a844640 1 -- 172.21.15.26:0/161582989 >> 172.21.15.26:0/161582989 conn(0x7f0184102f60 msgr2=0x7f0184103b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:09:05.275 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:04.690+0000 7f018a844640 1 -- 172.21.15.26:0/161582989 shutdown_connections 2024-09-11T23:09:05.275 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:04.690+0000 7f018a844640 1 -- 172.21.15.26:0/161582989 wait complete. 2024-09-11T23:09:05.562 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:09:05 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: 2024-09-11T23:09:05.274+0000 7fb671a4a040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-11T23:09:05.562 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:09:05 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: 2024-09-11T23:09:05.432+0000 7fb671a4a040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-11T23:09:05.839 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:09:05 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: 2024-09-11T23:09:05.775+0000 7fb671a4a040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-11T23:09:05.840 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:09:05 smithi026 ceph-mon[31172]: from='client.? 172.21.15.26:0/161582989' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-09-11T23:09:05.840 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:09:05 smithi026 ceph-mon[31172]: mgrmap e5: x(active, since 6s) 2024-09-11T23:09:06.251 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:09:05 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: 2024-09-11T23:09:05.839+0000 7fb671a4a040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-11T23:09:06.251 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:09:06 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: 2024-09-11T23:09:06.043+0000 7fb671a4a040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-11T23:09:06.587 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout { 2024-09-11T23:09:06.587 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-09-11T23:09:06.587 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-11T23:09:06.587 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-11T23:09:06.587 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-11T23:09:06.587 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout } 2024-09-11T23:09:06.588 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:05.738+0000 7f2501fa4640 1 Processor -- start 2024-09-11T23:09:06.588 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:05.738+0000 7f2501fa4640 1 -- start start 2024-09-11T23:09:06.588 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:05.739+0000 7f2501fa4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24fc1066d0 0x7f24fc106ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:09:06.588 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:05.739+0000 7f2501fa4640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f24fc1070a0 con 0x7f24fc1066d0 2024-09-11T23:09:06.588 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:05.739+0000 7f24fb7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24fc1066d0 0x7f24fc106ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:09:06.588 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:05.739+0000 7f24fb7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24fc1066d0 0x7f24fc106ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59112/0 (socket says 172.21.15.26:59112) 2024-09-11T23:09:06.588 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:05.739+0000 7f24fb7fe640 1 -- 172.21.15.26:0/2283726375 learned_addr learned my addr 172.21.15.26:0/2283726375 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:09:06.588 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:05.739+0000 7f24fb7fe640 1 -- 172.21.15.26:0/2283726375 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24fc1078d0 con 0x7f24fc1066d0 2024-09-11T23:09:06.588 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:05.740+0000 7f24fb7fe640 1 --2- 172.21.15.26:0/2283726375 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24fc1066d0 0x7f24fc106ad0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f24e4009820 tx=0x7f24e402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c9970fc492d4412 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:09:06.588 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:05.740+0000 7f24fa7fc640 1 -- 172.21.15.26:0/2283726375 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f24e4037070 con 0x7f24fc1066d0 2024-09-11T23:09:06.589 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:05.740+0000 7f24fa7fc640 1 -- 172.21.15.26:0/2283726375 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f24e4032900 con 0x7f24fc1066d0 2024-09-11T23:09:06.589 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:05.740+0000 7f24fa7fc640 1 -- 172.21.15.26:0/2283726375 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f24e4030850 con 0x7f24fc1066d0 2024-09-11T23:09:06.589 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:05.741+0000 7f2501fa4640 1 -- 172.21.15.26:0/2283726375 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24fc1066d0 msgr2=0x7f24fc106ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:09:06.589 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:05.741+0000 7f2501fa4640 1 --2- 172.21.15.26:0/2283726375 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24fc1066d0 0x7f24fc106ad0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f24e4009820 tx=0x7f24e402a270 comp rx=0 tx=0).stop 2024-09-11T23:09:06.589 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:05.741+0000 7f2501fa4640 1 -- 172.21.15.26:0/2283726375 shutdown_connections 2024-09-11T23:09:06.589 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:05.741+0000 7f2501fa4640 1 --2- 172.21.15.26:0/2283726375 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24fc1066d0 0x7f24fc106ad0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:09:06.589 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:05.741+0000 7f2501fa4640 1 -- 172.21.15.26:0/2283726375 >> 172.21.15.26:0/2283726375 conn(0x7f24fc101e60 msgr2=0x7f24fc1042a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:09:06.589 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:05.741+0000 7f2501fa4640 1 -- 172.21.15.26:0/2283726375 shutdown_connections 2024-09-11T23:09:06.589 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:05.741+0000 7f2501fa4640 1 -- 172.21.15.26:0/2283726375 wait complete. 2024-09-11T23:09:06.589 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:05.741+0000 7f2501fa4640 1 Processor -- start 2024-09-11T23:09:06.590 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:05.742+0000 7f2501fa4640 1 -- start start 2024-09-11T23:09:06.590 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:05.742+0000 7f2501fa4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24fc1066d0 0x7f24fc191890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:09:06.590 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:05.742+0000 7f2501fa4640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f24fc10ac80 con 0x7f24fc1066d0 2024-09-11T23:09:06.590 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:05.742+0000 7f24fb7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24fc1066d0 0x7f24fc191890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:09:06.590 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:05.742+0000 7f24fb7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24fc1066d0 0x7f24fc191890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59116/0 (socket says 172.21.15.26:59116) 2024-09-11T23:09:06.590 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:05.742+0000 7f24fb7fe640 1 -- 172.21.15.26:0/1325972739 learned_addr learned my addr 172.21.15.26:0/1325972739 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:09:06.590 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:05.742+0000 7f24fb7fe640 1 -- 172.21.15.26:0/1325972739 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24fc191dd0 con 0x7f24fc1066d0 2024-09-11T23:09:06.590 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:05.742+0000 7f24fb7fe640 1 --2- 172.21.15.26:0/1325972739 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24fc1066d0 0x7f24fc191890 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f24e40097f0 tx=0x7f24e4030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:09:06.590 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:05.742+0000 7f24f8ff9640 1 -- 172.21.15.26:0/1325972739 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f24e4037040 con 0x7f24fc1066d0 2024-09-11T23:09:06.590 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:05.742+0000 7f24f8ff9640 1 -- 172.21.15.26:0/1325972739 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f24e4040070 con 0x7f24fc1066d0 2024-09-11T23:09:06.590 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:05.742+0000 7f2501fa4640 1 -- 172.21.15.26:0/1325972739 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24fc18ffc0 con 0x7f24fc1066d0 2024-09-11T23:09:06.591 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:05.743+0000 7f2501fa4640 1 -- 172.21.15.26:0/1325972739 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24fc1904a0 con 0x7f24fc1066d0 2024-09-11T23:09:06.591 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:05.743+0000 7f24f8ff9640 1 -- 172.21.15.26:0/1325972739 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f24e403a390 con 0x7f24fc1066d0 2024-09-11T23:09:06.591 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:05.744+0000 7f24f8ff9640 1 -- 172.21.15.26:0/1325972739 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7f24e403b430 con 0x7f24fc1066d0 2024-09-11T23:09:06.591 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:05.744+0000 7f2501fa4640 1 -- 172.21.15.26:0/1325972739 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24dc0052c0 con 0x7f24fc1066d0 2024-09-11T23:09:06.591 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:05.744+0000 7f24f8ff9640 1 --2- 172.21.15.26:0/1325972739 >> [v2:172.21.15.26:6800/2673575521,v1:172.21.15.26:6801/2673575521] conn(0x7f24e8038fc0 0x7f24e803b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:09:06.591 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:05.744+0000 7f24f8ff9640 1 -- 172.21.15.26:0/1325972739 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f24e406c0a0 con 0x7f24fc1066d0 2024-09-11T23:09:06.591 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:05.744+0000 7f24faffd640 1 -- 172.21.15.26:0/1325972739 >> [v2:172.21.15.26:6800/2673575521,v1:172.21.15.26:6801/2673575521] conn(0x7f24e8038fc0 msgr2=0x7f24e803b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.26:6800/2673575521 2024-09-11T23:09:06.591 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:05.744+0000 7f24faffd640 1 --2- 172.21.15.26:0/1325972739 >> [v2:172.21.15.26:6800/2673575521,v1:172.21.15.26:6801/2673575521] conn(0x7f24e8038fc0 0x7f24e803b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-11T23:09:06.591 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:05.749+0000 7f24f8ff9640 1 -- 172.21.15.26:0/1325972739 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f24e4032dd0 con 0x7f24fc1066d0 2024-09-11T23:09:06.591 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:05.945+0000 7f24faffd640 1 -- 172.21.15.26:0/1325972739 >> [v2:172.21.15.26:6800/2673575521,v1:172.21.15.26:6801/2673575521] conn(0x7f24e8038fc0 msgr2=0x7f24e803b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.26:6800/2673575521 2024-09-11T23:09:06.591 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:05.945+0000 7f24faffd640 1 --2- 172.21.15.26:0/1325972739 >> [v2:172.21.15.26:6800/2673575521,v1:172.21.15.26:6801/2673575521] conn(0x7f24e8038fc0 0x7f24e803b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-11T23:09:06.592 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:05.977+0000 7f2501fa4640 1 -- 172.21.15.26:0/1325972739 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f24dc005dc0 con 0x7f24fc1066d0 2024-09-11T23:09:06.592 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:05.977+0000 7f24f8ff9640 1 -- 172.21.15.26:0/1325972739 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7f24e406be00 con 0x7f24fc1066d0 2024-09-11T23:09:06.592 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:05.979+0000 7f2501fa4640 1 -- 172.21.15.26:0/1325972739 >> [v2:172.21.15.26:6800/2673575521,v1:172.21.15.26:6801/2673575521] conn(0x7f24e8038fc0 msgr2=0x7f24e803b480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-11T23:09:06.592 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:05.979+0000 7f2501fa4640 1 --2- 172.21.15.26:0/1325972739 >> [v2:172.21.15.26:6800/2673575521,v1:172.21.15.26:6801/2673575521] conn(0x7f24e8038fc0 0x7f24e803b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:09:06.592 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:05.979+0000 7f2501fa4640 1 -- 172.21.15.26:0/1325972739 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24fc1066d0 msgr2=0x7f24fc191890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:09:06.592 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:05.979+0000 7f2501fa4640 1 --2- 172.21.15.26:0/1325972739 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24fc1066d0 0x7f24fc191890 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f24e40097f0 tx=0x7f24e4030d80 comp rx=0 tx=0).stop 2024-09-11T23:09:06.592 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:05.979+0000 7f2501fa4640 1 -- 172.21.15.26:0/1325972739 shutdown_connections 2024-09-11T23:09:06.592 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:05.979+0000 7f2501fa4640 1 --2- 172.21.15.26:0/1325972739 >> [v2:172.21.15.26:6800/2673575521,v1:172.21.15.26:6801/2673575521] conn(0x7f24e8038fc0 0x7f24e803b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:09:06.592 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:05.979+0000 7f2501fa4640 1 --2- 172.21.15.26:0/1325972739 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24fc1066d0 0x7f24fc191890 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:09:06.592 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:05.979+0000 7f2501fa4640 1 -- 172.21.15.26:0/1325972739 >> 172.21.15.26:0/1325972739 conn(0x7f24fc101e60 msgr2=0x7f24fc102a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:09:06.592 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:05.980+0000 7f2501fa4640 1 -- 172.21.15.26:0/1325972739 shutdown_connections 2024-09-11T23:09:06.592 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:05.980+0000 7f2501fa4640 1 -- 172.21.15.26:0/1325972739 wait complete. 2024-09-11T23:09:06.592 INFO:teuthology.orchestra.run.smithi026.stdout:Waiting for the mgr to restart... 2024-09-11T23:09:06.592 INFO:teuthology.orchestra.run.smithi026.stdout:Waiting for mgr epoch 5... 2024-09-11T23:09:06.829 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:09:06 smithi026 ceph-mon[31172]: from='client.? 172.21.15.26:0/1325972739' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-11T23:09:07.193 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:09:06 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: 2024-09-11T23:09:06.991+0000 7fb671a4a040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-11T23:09:07.193 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:09:07 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: 2024-09-11T23:09:07.062+0000 7fb671a4a040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-11T23:09:07.932 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:09:07 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: 2024-09-11T23:09:07.811+0000 7fb671a4a040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-11T23:09:07.932 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:09:07 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: 2024-09-11T23:09:07.932+0000 7fb671a4a040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-11T23:09:08.251 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:09:08 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: 2024-09-11T23:09:08.081+0000 7fb671a4a040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-11T23:09:08.251 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:09:08 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: 2024-09-11T23:09:08.145+0000 7fb671a4a040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-11T23:09:08.602 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:09:08 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: 2024-09-11T23:09:08.302+0000 7fb671a4a040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-11T23:09:08.602 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:09:08 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: 2024-09-11T23:09:08.373+0000 7fb671a4a040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-11T23:09:08.602 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:09:08 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: 2024-09-11T23:09:08.436+0000 7fb671a4a040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-11T23:09:08.603 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:09:08 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: 2024-09-11T23:09:08.602+0000 7fb671a4a040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-11T23:09:08.889 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:09:08 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: 2024-09-11T23:09:08.670+0000 7fb671a4a040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-11T23:09:08.889 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:09:08 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: 2024-09-11T23:09:08.739+0000 7fb671a4a040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-11T23:09:08.889 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:09:08 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: 2024-09-11T23:09:08.826+0000 7fb671a4a040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-11T23:09:08.889 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:09:08 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: 2024-09-11T23:09:08.888+0000 7fb671a4a040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-11T23:09:09.251 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:09:08 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: 2024-09-11T23:09:08.950+0000 7fb671a4a040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-11T23:09:10.251 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:09:09 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: 2024-09-11T23:09:09.779+0000 7fb671a4a040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-11T23:09:10.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:09:09 smithi026 ceph-mon[31172]: Active manager daemon x restarted 2024-09-11T23:09:10.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:09:09 smithi026 ceph-mon[31172]: Activating manager daemon x 2024-09-11T23:09:10.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:09:09 smithi026 ceph-mon[31172]: osdmap e2: 0 total, 0 up, 0 in 2024-09-11T23:09:10.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:09:09 smithi026 ceph-mon[31172]: mgrmap e6: x(active, starting, since 0.00345399s) 2024-09-11T23:09:10.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:09:09 smithi026 ceph-mon[31172]: from='mgr.14116 172.21.15.26:0/3833484538' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T23:09:10.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:09:09 smithi026 ceph-mon[31172]: from='mgr.14116 172.21.15.26:0/3833484538' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-11T23:09:10.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:09:09 smithi026 ceph-mon[31172]: from='mgr.14116 172.21.15.26:0/3833484538' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-11T23:09:10.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:09:09 smithi026 ceph-mon[31172]: from='mgr.14116 172.21.15.26:0/3833484538' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-11T23:09:10.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:09:09 smithi026 ceph-mon[31172]: from='mgr.14116 172.21.15.26:0/3833484538' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-11T23:09:10.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:09:09 smithi026 ceph-mon[31172]: Manager daemon x is now available 2024-09-11T23:09:10.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:09:09 smithi026 ceph-mon[31172]: from='mgr.14116 172.21.15.26:0/3833484538' entity='mgr.x' 2024-09-11T23:09:10.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:09:09 smithi026 ceph-mon[31172]: from='mgr.14116 172.21.15.26:0/3833484538' entity='mgr.x' 2024-09-11T23:09:10.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:09:09 smithi026 ceph-mon[31172]: from='mgr.14116 172.21.15.26:0/3833484538' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:09:10.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:09:09 smithi026 ceph-mon[31172]: from='mgr.14116 172.21.15.26:0/3833484538' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:09:10.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:09:09 smithi026 ceph-mon[31172]: from='mgr.14116 172.21.15.26:0/3833484538' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-11T23:09:10.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:09:09 smithi026 ceph-mon[31172]: from='mgr.14116 172.21.15.26:0/3833484538' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-11T23:09:11.142 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:09:10 smithi026 ceph-mon[31172]: Found migration_current of "None". Setting to last migration. 2024-09-11T23:09:11.142 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:09:10 smithi026 ceph-mon[31172]: mgrmap e7: x(active, since 1.0056s) 2024-09-11T23:09:11.445 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout { 2024-09-11T23:09:11.446 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-09-11T23:09:11.446 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-11T23:09:11.446 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout } 2024-09-11T23:09:11.446 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:06.986+0000 7fc4abeb0640 1 Processor -- start 2024-09-11T23:09:11.446 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:06.987+0000 7fc4abeb0640 1 -- start start 2024-09-11T23:09:11.446 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:06.987+0000 7fc4abeb0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc4a41066f0 0x7fc4a4106af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:09:11.446 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:06.987+0000 7fc4abeb0640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc4a41070c0 con 0x7fc4a41066f0 2024-09-11T23:09:11.446 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:06.987+0000 7fc4a9c4c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc4a41066f0 0x7fc4a4106af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:09:11.446 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:06.987+0000 7fc4a9c4c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc4a41066f0 0x7fc4a4106af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59132/0 (socket says 172.21.15.26:59132) 2024-09-11T23:09:11.446 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:06.987+0000 7fc4a9c4c640 1 -- 172.21.15.26:0/3622369884 learned_addr learned my addr 172.21.15.26:0/3622369884 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:09:11.446 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:06.987+0000 7fc4a9c4c640 1 -- 172.21.15.26:0/3622369884 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4a41078f0 con 0x7fc4a41066f0 2024-09-11T23:09:11.447 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:06.988+0000 7fc4a9c4c640 1 --2- 172.21.15.26:0/3622369884 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc4a41066f0 0x7fc4a4106af0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fc4a0009820 tx=0x7fc4a002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47368c3aff8809b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:09:11.447 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:06.988+0000 7fc4a8c4a640 1 -- 172.21.15.26:0/3622369884 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc4a0037070 con 0x7fc4a41066f0 2024-09-11T23:09:11.447 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:06.988+0000 7fc4a8c4a640 1 -- 172.21.15.26:0/3622369884 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc4a0032900 con 0x7fc4a41066f0 2024-09-11T23:09:11.447 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:06.988+0000 7fc4a8c4a640 1 -- 172.21.15.26:0/3622369884 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc4a0030850 con 0x7fc4a41066f0 2024-09-11T23:09:11.447 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:06.989+0000 7fc4abeb0640 1 -- 172.21.15.26:0/3622369884 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc4a41066f0 msgr2=0x7fc4a4106af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:09:11.447 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:06.989+0000 7fc4abeb0640 1 --2- 172.21.15.26:0/3622369884 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc4a41066f0 0x7fc4a4106af0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fc4a0009820 tx=0x7fc4a002a270 comp rx=0 tx=0).stop 2024-09-11T23:09:11.447 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:06.989+0000 7fc4abeb0640 1 -- 172.21.15.26:0/3622369884 shutdown_connections 2024-09-11T23:09:11.447 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:06.989+0000 7fc4abeb0640 1 --2- 172.21.15.26:0/3622369884 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc4a41066f0 0x7fc4a4106af0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:09:11.447 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:06.989+0000 7fc4abeb0640 1 -- 172.21.15.26:0/3622369884 >> 172.21.15.26:0/3622369884 conn(0x7fc4a4101e80 msgr2=0x7fc4a41042c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:09:11.447 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:06.989+0000 7fc4abeb0640 1 -- 172.21.15.26:0/3622369884 shutdown_connections 2024-09-11T23:09:11.447 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:06.989+0000 7fc4abeb0640 1 -- 172.21.15.26:0/3622369884 wait complete. 2024-09-11T23:09:11.448 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:06.990+0000 7fc4abeb0640 1 Processor -- start 2024-09-11T23:09:11.448 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:06.990+0000 7fc4abeb0640 1 -- start start 2024-09-11T23:09:11.448 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:06.990+0000 7fc4abeb0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc4a41066f0 0x7fc4a419d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:09:11.448 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:06.990+0000 7fc4abeb0640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc4a410aca0 con 0x7fc4a41066f0 2024-09-11T23:09:11.448 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:06.990+0000 7fc4a9c4c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc4a41066f0 0x7fc4a419d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:09:11.448 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:06.990+0000 7fc4a9c4c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc4a41066f0 0x7fc4a419d9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59144/0 (socket says 172.21.15.26:59144) 2024-09-11T23:09:11.448 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:06.990+0000 7fc4a9c4c640 1 -- 172.21.15.26:0/827679218 learned_addr learned my addr 172.21.15.26:0/827679218 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:09:11.448 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:06.990+0000 7fc4a9c4c640 1 -- 172.21.15.26:0/827679218 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4a419df10 con 0x7fc4a41066f0 2024-09-11T23:09:11.448 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:06.991+0000 7fc4a9c4c640 1 --2- 172.21.15.26:0/827679218 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc4a41066f0 0x7fc4a419d9d0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fc4a0009940 tx=0x7fc4a0032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:09:11.448 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:06.991+0000 7fc49affd640 1 -- 172.21.15.26:0/827679218 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc4a0042070 con 0x7fc4a41066f0 2024-09-11T23:09:11.449 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:06.991+0000 7fc49affd640 1 -- 172.21.15.26:0/827679218 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc4a00308c0 con 0x7fc4a41066f0 2024-09-11T23:09:11.449 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:06.991+0000 7fc4abeb0640 1 -- 172.21.15.26:0/827679218 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4a419e1a0 con 0x7fc4a41066f0 2024-09-11T23:09:11.449 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:06.991+0000 7fc49affd640 1 -- 172.21.15.26:0/827679218 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc4a0037040 con 0x7fc4a41066f0 2024-09-11T23:09:11.449 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:06.991+0000 7fc4abeb0640 1 -- 172.21.15.26:0/827679218 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4a41a0e90 con 0x7fc4a41066f0 2024-09-11T23:09:11.449 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:06.992+0000 7fc49affd640 1 -- 172.21.15.26:0/827679218 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7fc4a0045070 con 0x7fc4a41066f0 2024-09-11T23:09:11.449 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:06.992+0000 7fc49affd640 1 --2- 172.21.15.26:0/827679218 >> [v2:172.21.15.26:6800/2673575521,v1:172.21.15.26:6801/2673575521] conn(0x7fc490038f70 0x7fc49003b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:09:11.449 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:06.992+0000 7fc49affd640 1 -- 172.21.15.26:0/827679218 --> [v2:172.21.15.26:6800/2673575521,v1:172.21.15.26:6801/2673575521] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fc49003bb40 con 0x7fc490038f70 2024-09-11T23:09:11.450 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:06.992+0000 7fc49affd640 1 -- 172.21.15.26:0/827679218 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fc4a006bff0 con 0x7fc4a41066f0 2024-09-11T23:09:11.450 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:06.992+0000 7fc4a944b640 1 -- 172.21.15.26:0/827679218 >> [v2:172.21.15.26:6800/2673575521,v1:172.21.15.26:6801/2673575521] conn(0x7fc490038f70 msgr2=0x7fc49003b430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.26:6800/2673575521 2024-09-11T23:09:11.450 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:06.992+0000 7fc4a944b640 1 --2- 172.21.15.26:0/827679218 >> [v2:172.21.15.26:6800/2673575521,v1:172.21.15.26:6801/2673575521] conn(0x7fc490038f70 0x7fc49003b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-11T23:09:11.450 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:07.193+0000 7fc4a944b640 1 -- 172.21.15.26:0/827679218 >> [v2:172.21.15.26:6800/2673575521,v1:172.21.15.26:6801/2673575521] conn(0x7fc490038f70 msgr2=0x7fc49003b430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.26:6800/2673575521 2024-09-11T23:09:11.450 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:07.193+0000 7fc4a944b640 1 --2- 172.21.15.26:0/827679218 >> [v2:172.21.15.26:6800/2673575521,v1:172.21.15.26:6801/2673575521] conn(0x7fc490038f70 0x7fc49003b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-11T23:09:11.450 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:07.594+0000 7fc4a944b640 1 -- 172.21.15.26:0/827679218 >> [v2:172.21.15.26:6800/2673575521,v1:172.21.15.26:6801/2673575521] conn(0x7fc490038f70 msgr2=0x7fc49003b430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.26:6800/2673575521 2024-09-11T23:09:11.450 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:07.594+0000 7fc4a944b640 1 --2- 172.21.15.26:0/827679218 >> [v2:172.21.15.26:6800/2673575521,v1:172.21.15.26:6801/2673575521] conn(0x7fc490038f70 0x7fc49003b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-11T23:09:11.450 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:08.395+0000 7fc4a944b640 1 -- 172.21.15.26:0/827679218 >> [v2:172.21.15.26:6800/2673575521,v1:172.21.15.26:6801/2673575521] conn(0x7fc490038f70 msgr2=0x7fc49003b430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.26:6800/2673575521 2024-09-11T23:09:11.450 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:08.395+0000 7fc4a944b640 1 --2- 172.21.15.26:0/827679218 >> [v2:172.21.15.26:6800/2673575521,v1:172.21.15.26:6801/2673575521] conn(0x7fc490038f70 0x7fc49003b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-11T23:09:11.450 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:09.783+0000 7fc49affd640 1 -- 172.21.15.26:0/827679218 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mgrmap(e 6) ==== 46054+0+0 (secure 0 0 0) 0x7fc4a00327b0 con 0x7fc4a41066f0 2024-09-11T23:09:11.450 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:09.783+0000 7fc49affd640 1 -- 172.21.15.26:0/827679218 >> [v2:172.21.15.26:6800/2673575521,v1:172.21.15.26:6801/2673575521] conn(0x7fc490038f70 msgr2=0x7fc49003b430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-11T23:09:11.451 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:09.783+0000 7fc49affd640 1 --2- 172.21.15.26:0/827679218 >> [v2:172.21.15.26:6800/2673575521,v1:172.21.15.26:6801/2673575521] conn(0x7fc490038f70 0x7fc49003b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:09:11.451 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:10.784+0000 7fc49affd640 1 -- 172.21.15.26:0/827679218 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mgrmap(e 7) ==== 46181+0+0 (secure 0 0 0) 0x7fc4a003c820 con 0x7fc4a41066f0 2024-09-11T23:09:11.451 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:10.784+0000 7fc49affd640 1 --2- 172.21.15.26:0/827679218 >> [v2:172.21.15.26:6800/677148151,v1:172.21.15.26:6801/677148151] conn(0x7fc49003c8a0 0x7fc49003ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:09:11.451 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:10.784+0000 7fc49affd640 1 -- 172.21.15.26:0/827679218 --> [v2:172.21.15.26:6800/677148151,v1:172.21.15.26:6801/677148151] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fc4a002bba0 con 0x7fc49003c8a0 2024-09-11T23:09:11.451 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:10.784+0000 7fc4a944b640 1 --2- 172.21.15.26:0/827679218 >> [v2:172.21.15.26:6800/677148151,v1:172.21.15.26:6801/677148151] conn(0x7fc49003c8a0 0x7fc49003ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:09:11.451 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:10.785+0000 7fc4a944b640 1 --2- 172.21.15.26:0/827679218 >> [v2:172.21.15.26:6800/677148151,v1:172.21.15.26:6801/677148151] conn(0x7fc49003c8a0 0x7fc49003ec90 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fc494003980 tx=0x7fc494006e10 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:09:11.451 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:10.786+0000 7fc49affd640 1 -- 172.21.15.26:0/827679218 <== mgr.14116 v2:172.21.15.26:6800/677148151 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7fc4a002bba0 con 0x7fc49003c8a0 2024-09-11T23:09:11.451 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:10.793+0000 7fc4abeb0640 1 -- 172.21.15.26:0/827679218 --> [v2:172.21.15.26:6800/677148151,v1:172.21.15.26:6801/677148151] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7fc4a4106b70 con 0x7fc49003c8a0 2024-09-11T23:09:11.451 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:10.793+0000 7fc49affd640 1 -- 172.21.15.26:0/827679218 <== mgr.14116 v2:172.21.15.26:6800/677148151 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7fc4a4106b70 con 0x7fc49003c8a0 2024-09-11T23:09:11.451 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:10.793+0000 7fc4abeb0640 1 -- 172.21.15.26:0/827679218 >> [v2:172.21.15.26:6800/677148151,v1:172.21.15.26:6801/677148151] conn(0x7fc49003c8a0 msgr2=0x7fc49003ec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:09:11.451 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:10.793+0000 7fc4abeb0640 1 --2- 172.21.15.26:0/827679218 >> [v2:172.21.15.26:6800/677148151,v1:172.21.15.26:6801/677148151] conn(0x7fc49003c8a0 0x7fc49003ec90 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fc494003980 tx=0x7fc494006e10 comp rx=0 tx=0).stop 2024-09-11T23:09:11.452 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:10.793+0000 7fc4abeb0640 1 -- 172.21.15.26:0/827679218 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc4a41066f0 msgr2=0x7fc4a419d9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:09:11.452 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:10.793+0000 7fc4abeb0640 1 --2- 172.21.15.26:0/827679218 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc4a41066f0 0x7fc4a419d9d0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fc4a0009940 tx=0x7fc4a0032ce0 comp rx=0 tx=0).stop 2024-09-11T23:09:11.452 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:10.793+0000 7fc4abeb0640 1 -- 172.21.15.26:0/827679218 shutdown_connections 2024-09-11T23:09:11.452 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:10.793+0000 7fc4abeb0640 1 --2- 172.21.15.26:0/827679218 >> [v2:172.21.15.26:6800/677148151,v1:172.21.15.26:6801/677148151] conn(0x7fc49003c8a0 0x7fc49003ec90 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:09:11.452 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:10.793+0000 7fc4abeb0640 1 --2- 172.21.15.26:0/827679218 >> [v2:172.21.15.26:6800/2673575521,v1:172.21.15.26:6801/2673575521] conn(0x7fc490038f70 0x7fc49003b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:09:11.452 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:10.793+0000 7fc4abeb0640 1 --2- 172.21.15.26:0/827679218 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc4a41066f0 0x7fc4a419d9d0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:09:11.452 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:10.793+0000 7fc4abeb0640 1 -- 172.21.15.26:0/827679218 >> 172.21.15.26:0/827679218 conn(0x7fc4a4101e80 msgr2=0x7fc4a4102e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:09:11.452 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:10.793+0000 7fc4abeb0640 1 -- 172.21.15.26:0/827679218 shutdown_connections 2024-09-11T23:09:11.452 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:10.793+0000 7fc4abeb0640 1 -- 172.21.15.26:0/827679218 wait complete. 2024-09-11T23:09:11.452 INFO:teuthology.orchestra.run.smithi026.stdout:mgr epoch 5 is available 2024-09-11T23:09:11.452 INFO:teuthology.orchestra.run.smithi026.stdout:Setting orchestrator backend to cephadm... 2024-09-11T23:09:12.373 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:09:12 smithi026 ceph-mon[31172]: from='client.14120 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-11T23:09:12.374 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:09:12 smithi026 ceph-mon[31172]: from='client.14120 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-11T23:09:12.374 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:09:12 smithi026 ceph-mon[31172]: from='mgr.14116 172.21.15.26:0/3833484538' entity='mgr.x' 2024-09-11T23:09:12.374 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:09:12 smithi026 ceph-mon[31172]: from='mgr.14116 172.21.15.26:0/3833484538' entity='mgr.x' 2024-09-11T23:09:12.374 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:09:12 smithi026 ceph-mon[31172]: from='mgr.14116 172.21.15.26:0/3833484538' entity='mgr.x' 2024-09-11T23:09:12.374 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:09:12 smithi026 ceph-mon[31172]: from='mgr.14116 172.21.15.26:0/3833484538' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:09:12.699 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:11.835+0000 7f07ec225640 1 Processor -- start 2024-09-11T23:09:12.699 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:11.835+0000 7f07ec225640 1 -- start start 2024-09-11T23:09:12.699 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:11.836+0000 7f07ec225640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07e41066d0 0x7f07e4106ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:09:12.699 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:11.836+0000 7f07ec225640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f07e41070a0 con 0x7f07e41066d0 2024-09-11T23:09:12.699 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:11.836+0000 7f07e9fc1640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07e41066d0 0x7f07e4106ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:09:12.699 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:11.836+0000 7f07e9fc1640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07e41066d0 0x7f07e4106ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45422/0 (socket says 172.21.15.26:45422) 2024-09-11T23:09:12.699 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:11.836+0000 7f07e9fc1640 1 -- 172.21.15.26:0/1600069762 learned_addr learned my addr 172.21.15.26:0/1600069762 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:09:12.699 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:11.836+0000 7f07e9fc1640 1 -- 172.21.15.26:0/1600069762 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07e41078d0 con 0x7f07e41066d0 2024-09-11T23:09:12.699 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:11.837+0000 7f07e9fc1640 1 --2- 172.21.15.26:0/1600069762 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07e41066d0 0x7f07e4106ad0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f07e0009820 tx=0x7f07e002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecf47034941711de server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:09:12.699 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:11.837+0000 7f07e8fbf640 1 -- 172.21.15.26:0/1600069762 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f07e0037070 con 0x7f07e41066d0 2024-09-11T23:09:12.700 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:11.837+0000 7f07e8fbf640 1 -- 172.21.15.26:0/1600069762 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f07e0032900 con 0x7f07e41066d0 2024-09-11T23:09:12.700 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:11.837+0000 7f07e8fbf640 1 -- 172.21.15.26:0/1600069762 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f07e0030850 con 0x7f07e41066d0 2024-09-11T23:09:12.700 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:11.838+0000 7f07ec225640 1 -- 172.21.15.26:0/1600069762 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07e41066d0 msgr2=0x7f07e4106ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:09:12.700 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:11.838+0000 7f07ec225640 1 --2- 172.21.15.26:0/1600069762 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07e41066d0 0x7f07e4106ad0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f07e0009820 tx=0x7f07e002a270 comp rx=0 tx=0).stop 2024-09-11T23:09:12.700 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:11.838+0000 7f07ec225640 1 -- 172.21.15.26:0/1600069762 shutdown_connections 2024-09-11T23:09:12.700 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:11.838+0000 7f07ec225640 1 --2- 172.21.15.26:0/1600069762 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07e41066d0 0x7f07e4106ad0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:09:12.700 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:11.838+0000 7f07ec225640 1 -- 172.21.15.26:0/1600069762 >> 172.21.15.26:0/1600069762 conn(0x7f07e4101e60 msgr2=0x7f07e41042a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:09:12.700 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:11.838+0000 7f07ec225640 1 -- 172.21.15.26:0/1600069762 shutdown_connections 2024-09-11T23:09:12.700 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:11.838+0000 7f07ec225640 1 -- 172.21.15.26:0/1600069762 wait complete. 2024-09-11T23:09:12.700 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:11.838+0000 7f07ec225640 1 Processor -- start 2024-09-11T23:09:12.700 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:11.838+0000 7f07ec225640 1 -- start start 2024-09-11T23:09:12.700 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:11.839+0000 7f07ec225640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07e41066d0 0x7f07e4191890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:09:12.700 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:11.839+0000 7f07ec225640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f07e410ac80 con 0x7f07e41066d0 2024-09-11T23:09:12.701 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:11.839+0000 7f07e9fc1640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07e41066d0 0x7f07e4191890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:09:12.701 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:11.839+0000 7f07e9fc1640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07e41066d0 0x7f07e4191890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45430/0 (socket says 172.21.15.26:45430) 2024-09-11T23:09:12.701 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:11.839+0000 7f07e9fc1640 1 -- 172.21.15.26:0/2684552636 learned_addr learned my addr 172.21.15.26:0/2684552636 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:09:12.701 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:11.839+0000 7f07e9fc1640 1 -- 172.21.15.26:0/2684552636 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07e4191dd0 con 0x7f07e41066d0 2024-09-11T23:09:12.701 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:11.839+0000 7f07e9fc1640 1 --2- 172.21.15.26:0/2684552636 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07e41066d0 0x7f07e4191890 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f07e0009940 tx=0x7f07e0032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:09:12.701 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:11.839+0000 7f07daffd640 1 -- 172.21.15.26:0/2684552636 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f07e0042070 con 0x7f07e41066d0 2024-09-11T23:09:12.701 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:11.839+0000 7f07ec225640 1 -- 172.21.15.26:0/2684552636 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07e418ffc0 con 0x7f07e41066d0 2024-09-11T23:09:12.701 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:11.839+0000 7f07ec225640 1 -- 172.21.15.26:0/2684552636 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07e4190500 con 0x7f07e41066d0 2024-09-11T23:09:12.701 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:11.839+0000 7f07daffd640 1 -- 172.21.15.26:0/2684552636 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f07e00308c0 con 0x7f07e41066d0 2024-09-11T23:09:12.701 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:11.840+0000 7f07daffd640 1 -- 172.21.15.26:0/2684552636 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f07e0037040 con 0x7f07e41066d0 2024-09-11T23:09:12.701 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:11.840+0000 7f07daffd640 1 -- 172.21.15.26:0/2684552636 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 7) ==== 46181+0+0 (secure 0 0 0) 0x7f07e0045070 con 0x7f07e41066d0 2024-09-11T23:09:12.701 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:11.840+0000 7f07daffd640 1 --2- 172.21.15.26:0/2684552636 >> [v2:172.21.15.26:6800/677148151,v1:172.21.15.26:6801/677148151] conn(0x7f07d0038eb0 0x7f07d003b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:09:12.701 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:11.841+0000 7f07daffd640 1 -- 172.21.15.26:0/2684552636 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f07e006be30 con 0x7f07e41066d0 2024-09-11T23:09:12.701 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:11.841+0000 7f07e97c0640 1 --2- 172.21.15.26:0/2684552636 >> [v2:172.21.15.26:6800/677148151,v1:172.21.15.26:6801/677148151] conn(0x7f07d0038eb0 0x7f07d003b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:09:12.702 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:11.841+0000 7f07ec225640 1 -- 172.21.15.26:0/2684552636 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07c8005290 con 0x7f07e41066d0 2024-09-11T23:09:12.702 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:11.841+0000 7f07e97c0640 1 --2- 172.21.15.26:0/2684552636 >> [v2:172.21.15.26:6800/677148151,v1:172.21.15.26:6801/677148151] conn(0x7f07d0038eb0 0x7f07d003b370 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f07d40098c0 tx=0x7f07d4006db0 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:09:12.702 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:11.846+0000 7f07daffd640 1 -- 172.21.15.26:0/2684552636 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f07e004b170 con 0x7f07e41066d0 2024-09-11T23:09:12.702 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:12.055+0000 7f07ec225640 1 -- 172.21.15.26:0/2684552636 --> [v2:172.21.15.26:6800/677148151,v1:172.21.15.26:6801/677148151] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7f07c8000bc0 con 0x7f07d0038eb0 2024-09-11T23:09:12.702 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:12.060+0000 7f07daffd640 1 -- 172.21.15.26:0/2684552636 <== mgr.14116 v2:172.21.15.26:6800/677148151 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f07c8000bc0 con 0x7f07d0038eb0 2024-09-11T23:09:12.702 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:12.063+0000 7f07ec225640 1 -- 172.21.15.26:0/2684552636 >> [v2:172.21.15.26:6800/677148151,v1:172.21.15.26:6801/677148151] conn(0x7f07d0038eb0 msgr2=0x7f07d003b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:09:12.702 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:12.063+0000 7f07ec225640 1 --2- 172.21.15.26:0/2684552636 >> [v2:172.21.15.26:6800/677148151,v1:172.21.15.26:6801/677148151] conn(0x7f07d0038eb0 0x7f07d003b370 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f07d40098c0 tx=0x7f07d4006db0 comp rx=0 tx=0).stop 2024-09-11T23:09:12.702 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:12.063+0000 7f07ec225640 1 -- 172.21.15.26:0/2684552636 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07e41066d0 msgr2=0x7f07e4191890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:09:12.702 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:12.063+0000 7f07ec225640 1 --2- 172.21.15.26:0/2684552636 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07e41066d0 0x7f07e4191890 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f07e0009940 tx=0x7f07e0032ce0 comp rx=0 tx=0).stop 2024-09-11T23:09:12.702 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:12.063+0000 7f07ec225640 1 -- 172.21.15.26:0/2684552636 shutdown_connections 2024-09-11T23:09:12.702 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:12.063+0000 7f07ec225640 1 --2- 172.21.15.26:0/2684552636 >> [v2:172.21.15.26:6800/677148151,v1:172.21.15.26:6801/677148151] conn(0x7f07d0038eb0 0x7f07d003b370 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:09:12.702 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:12.063+0000 7f07ec225640 1 --2- 172.21.15.26:0/2684552636 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07e41066d0 0x7f07e4191890 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:09:12.703 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:12.063+0000 7f07ec225640 1 -- 172.21.15.26:0/2684552636 >> 172.21.15.26:0/2684552636 conn(0x7f07e4101e60 msgr2=0x7f07e4102e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:09:12.703 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:12.064+0000 7f07ec225640 1 -- 172.21.15.26:0/2684552636 shutdown_connections 2024-09-11T23:09:12.703 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:12.064+0000 7f07ec225640 1 -- 172.21.15.26:0/2684552636 wait complete. 2024-09-11T23:09:13.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:09:13 smithi026 ceph-mon[31172]: from='client.14128 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:09:13.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:09:13 smithi026 ceph-mon[31172]: mgrmap e8: x(active, since 2s) 2024-09-11T23:09:13.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:09:13 smithi026 ceph-mon[31172]: from='mgr.14116 172.21.15.26:0/3833484538' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:09:13.251 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:09:13 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: 2024-09-11T23:09:13.006+0000 7fb5de56f640 -1 log_channel(cephadm) log [ERR] : [11/Sep/2024:23:09:13] ENGINE Error in HTTPServer.serve 2024-09-11T23:09:13.251 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:09:13 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: Traceback (most recent call last): 2024-09-11T23:09:13.251 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:09:13 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-11T23:09:13.252 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:09:13 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: self._connections.run(self.expiration_interval) 2024-09-11T23:09:13.252 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:09:13 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-11T23:09:13.252 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:09:13 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: self._run(expiration_interval) 2024-09-11T23:09:13.252 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:09:13 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-11T23:09:13.252 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:09:13 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: new_conn = self._from_server_socket(self.server.socket) 2024-09-11T23:09:13.252 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:09:13 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-11T23:09:13.252 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:09:13 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-11T23:09:13.252 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:09:13 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-11T23:09:13.252 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:09:13 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: s = self.context.wrap_socket( 2024-09-11T23:09:13.252 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:09:13 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-11T23:09:13.252 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:09:13 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: return self.sslsocket_class._create( 2024-09-11T23:09:13.252 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:09:13 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-11T23:09:13.252 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:09:13 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: self.do_handshake() 2024-09-11T23:09:13.253 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:09:13 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-11T23:09:13.253 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:09:13 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: self._sslobj.do_handshake() 2024-09-11T23:09:13.253 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:09:13 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-11T23:09:13.253 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:09:13 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: 2024-09-11T23:09:13.839 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout value unchanged 2024-09-11T23:09:13.839 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:13.107+0000 7fbcb08a8640 1 Processor -- start 2024-09-11T23:09:13.839 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:13.107+0000 7fbcb08a8640 1 -- start start 2024-09-11T23:09:13.839 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:13.108+0000 7fbcb08a8640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbcac1066f0 0x7fbcac106af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:09:13.840 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:13.108+0000 7fbcb08a8640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fbcac1070c0 con 0x7fbcac1066f0 2024-09-11T23:09:13.840 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:13.108+0000 7fbcaa59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbcac1066f0 0x7fbcac106af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:09:13.840 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:13.108+0000 7fbcaa59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbcac1066f0 0x7fbcac106af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45438/0 (socket says 172.21.15.26:45438) 2024-09-11T23:09:13.840 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:13.108+0000 7fbcaa59c640 1 -- 172.21.15.26:0/2822577910 learned_addr learned my addr 172.21.15.26:0/2822577910 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:09:13.840 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:13.108+0000 7fbcaa59c640 1 -- 172.21.15.26:0/2822577910 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcac1078f0 con 0x7fbcac1066f0 2024-09-11T23:09:13.840 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:13.109+0000 7fbcaa59c640 1 --2- 172.21.15.26:0/2822577910 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbcac1066f0 0x7fbcac106af0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fbc9c009e80 tx=0x7fbc9c02a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=faf60a2272526908 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:09:13.840 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:13.109+0000 7fbca959a640 1 -- 172.21.15.26:0/2822577910 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbc9c036070 con 0x7fbcac1066f0 2024-09-11T23:09:13.840 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:13.109+0000 7fbca959a640 1 -- 172.21.15.26:0/2822577910 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fbc9c0318c0 con 0x7fbcac1066f0 2024-09-11T23:09:13.840 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:13.109+0000 7fbca959a640 1 -- 172.21.15.26:0/2822577910 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbc9c007ab0 con 0x7fbcac1066f0 2024-09-11T23:09:13.840 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:13.109+0000 7fbcb08a8640 1 -- 172.21.15.26:0/2822577910 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbcac1066f0 msgr2=0x7fbcac106af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:09:13.840 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:13.109+0000 7fbcb08a8640 1 --2- 172.21.15.26:0/2822577910 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbcac1066f0 0x7fbcac106af0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fbc9c009e80 tx=0x7fbc9c02a7a0 comp rx=0 tx=0).stop 2024-09-11T23:09:13.840 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:13.109+0000 7fbcb08a8640 1 -- 172.21.15.26:0/2822577910 shutdown_connections 2024-09-11T23:09:13.840 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:13.109+0000 7fbcb08a8640 1 --2- 172.21.15.26:0/2822577910 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbcac1066f0 0x7fbcac106af0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:09:13.841 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:13.109+0000 7fbcb08a8640 1 -- 172.21.15.26:0/2822577910 >> 172.21.15.26:0/2822577910 conn(0x7fbcac101e80 msgr2=0x7fbcac1042c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:09:13.841 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:13.109+0000 7fbcb08a8640 1 -- 172.21.15.26:0/2822577910 shutdown_connections 2024-09-11T23:09:13.841 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:13.110+0000 7fbcb08a8640 1 -- 172.21.15.26:0/2822577910 wait complete. 2024-09-11T23:09:13.841 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:13.110+0000 7fbcb08a8640 1 Processor -- start 2024-09-11T23:09:13.841 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:13.110+0000 7fbcb08a8640 1 -- start start 2024-09-11T23:09:13.841 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:13.110+0000 7fbcb08a8640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbcac1066f0 0x7fbcac07b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:09:13.841 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:13.110+0000 7fbcb08a8640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fbcac10aca0 con 0x7fbcac1066f0 2024-09-11T23:09:13.841 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:13.111+0000 7fbcaa59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbcac1066f0 0x7fbcac07b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:09:13.841 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:13.111+0000 7fbcaa59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbcac1066f0 0x7fbcac07b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45444/0 (socket says 172.21.15.26:45444) 2024-09-11T23:09:13.841 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:13.111+0000 7fbcaa59c640 1 -- 172.21.15.26:0/1277587102 learned_addr learned my addr 172.21.15.26:0/1277587102 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:09:13.841 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:13.111+0000 7fbcaa59c640 1 -- 172.21.15.26:0/1277587102 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcac19c360 con 0x7fbcac1066f0 2024-09-11T23:09:13.841 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:13.111+0000 7fbcaa59c640 1 --2- 172.21.15.26:0/1277587102 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbcac1066f0 0x7fbcac07b9d0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fbc9c02ad90 tx=0x7fbc9c031ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:09:13.841 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:13.111+0000 7fbca37fe640 1 -- 172.21.15.26:0/1277587102 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbc9c036050 con 0x7fbcac1066f0 2024-09-11T23:09:13.842 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:13.111+0000 7fbca37fe640 1 -- 172.21.15.26:0/1277587102 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fbc9c042070 con 0x7fbcac1066f0 2024-09-11T23:09:13.842 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:13.111+0000 7fbcb08a8640 1 -- 172.21.15.26:0/1277587102 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcac1a0970 con 0x7fbcac1066f0 2024-09-11T23:09:13.842 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:13.111+0000 7fbca37fe640 1 -- 172.21.15.26:0/1277587102 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbc9c03d7e0 con 0x7fbcac1066f0 2024-09-11T23:09:13.842 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:13.111+0000 7fbcb08a8640 1 -- 172.21.15.26:0/1277587102 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcac1a0d10 con 0x7fbcac1066f0 2024-09-11T23:09:13.842 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:13.112+0000 7fbca37fe640 1 -- 172.21.15.26:0/1277587102 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fbc9c03dae0 con 0x7fbcac1066f0 2024-09-11T23:09:13.842 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:13.112+0000 7fbca37fe640 1 --2- 172.21.15.26:0/1277587102 >> [v2:172.21.15.26:6800/677148151,v1:172.21.15.26:6801/677148151] conn(0x7fbc98038f70 0x7fbc9803b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:09:13.842 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:13.112+0000 7fbcb08a8640 1 -- 172.21.15.26:0/1277587102 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbcac106b70 con 0x7fbcac1066f0 2024-09-11T23:09:13.842 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:13.112+0000 7fbca37fe640 1 -- 172.21.15.26:0/1277587102 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fbc9c007e00 con 0x7fbcac1066f0 2024-09-11T23:09:13.842 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:13.112+0000 7fbca9d9b640 1 --2- 172.21.15.26:0/1277587102 >> [v2:172.21.15.26:6800/677148151,v1:172.21.15.26:6801/677148151] conn(0x7fbc98038f70 0x7fbc9803b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:09:13.842 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:13.113+0000 7fbca9d9b640 1 --2- 172.21.15.26:0/1277587102 >> [v2:172.21.15.26:6800/677148151,v1:172.21.15.26:6801/677148151] conn(0x7fbc98038f70 0x7fbc9803b430 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fbc940098c0 tx=0x7fbc94006db0 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:09:13.842 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:13.117+0000 7fbca37fe640 1 -- 172.21.15.26:0/1277587102 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fbc9c03de20 con 0x7fbcac1066f0 2024-09-11T23:09:13.843 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:13.300+0000 7fbcb08a8640 1 -- 172.21.15.26:0/1277587102 --> [v2:172.21.15.26:6800/677148151,v1:172.21.15.26:6801/677148151] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7fbcac103150 con 0x7fbc98038f70 2024-09-11T23:09:13.843 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:13.300+0000 7fbca37fe640 1 -- 172.21.15.26:0/1277587102 <== mgr.14116 v2:172.21.15.26:6800/677148151 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7fbcac103150 con 0x7fbc98038f70 2024-09-11T23:09:13.843 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:13.302+0000 7fbcb08a8640 1 -- 172.21.15.26:0/1277587102 >> [v2:172.21.15.26:6800/677148151,v1:172.21.15.26:6801/677148151] conn(0x7fbc98038f70 msgr2=0x7fbc9803b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:09:13.843 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:13.302+0000 7fbcb08a8640 1 --2- 172.21.15.26:0/1277587102 >> [v2:172.21.15.26:6800/677148151,v1:172.21.15.26:6801/677148151] conn(0x7fbc98038f70 0x7fbc9803b430 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fbc940098c0 tx=0x7fbc94006db0 comp rx=0 tx=0).stop 2024-09-11T23:09:13.843 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:13.302+0000 7fbcb08a8640 1 -- 172.21.15.26:0/1277587102 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbcac1066f0 msgr2=0x7fbcac07b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:09:13.843 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:13.302+0000 7fbcb08a8640 1 --2- 172.21.15.26:0/1277587102 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbcac1066f0 0x7fbcac07b9d0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fbc9c02ad90 tx=0x7fbc9c031ca0 comp rx=0 tx=0).stop 2024-09-11T23:09:13.843 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:13.302+0000 7fbcb08a8640 1 -- 172.21.15.26:0/1277587102 shutdown_connections 2024-09-11T23:09:13.843 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:13.302+0000 7fbcb08a8640 1 --2- 172.21.15.26:0/1277587102 >> [v2:172.21.15.26:6800/677148151,v1:172.21.15.26:6801/677148151] conn(0x7fbc98038f70 0x7fbc9803b430 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:09:13.843 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:13.302+0000 7fbcb08a8640 1 --2- 172.21.15.26:0/1277587102 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbcac1066f0 0x7fbcac07b9d0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:09:13.843 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:13.302+0000 7fbcb08a8640 1 -- 172.21.15.26:0/1277587102 >> 172.21.15.26:0/1277587102 conn(0x7fbcac101e80 msgr2=0x7fbcac102a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:09:13.843 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:13.303+0000 7fbcb08a8640 1 -- 172.21.15.26:0/1277587102 shutdown_connections 2024-09-11T23:09:13.843 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:13.303+0000 7fbcb08a8640 1 -- 172.21.15.26:0/1277587102 wait complete. 2024-09-11T23:09:13.843 INFO:teuthology.orchestra.run.smithi026.stdout:Generating ssh key... 2024-09-11T23:09:14.242 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:09:14 smithi026 ceph-mon[31172]: [11/Sep/2024:23:09:12] ENGINE Bus STARTING 2024-09-11T23:09:14.242 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:09:14 smithi026 ceph-mon[31172]: [11/Sep/2024:23:09:13] ENGINE Serving on https://172.21.15.26:7150 2024-09-11T23:09:14.242 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:09:14 smithi026 ceph-mon[31172]: [11/Sep/2024:23:09:13] ENGINE Error in HTTPServer.serve 2024-09-11T23:09:14.242 INFO:journalctl@ceph.mon.a.smithi026.stdout: Traceback (most recent call last): 2024-09-11T23:09:14.242 INFO:journalctl@ceph.mon.a.smithi026.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-11T23:09:14.242 INFO:journalctl@ceph.mon.a.smithi026.stdout: self._connections.run(self.expiration_interval) 2024-09-11T23:09:14.242 INFO:journalctl@ceph.mon.a.smithi026.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-11T23:09:14.243 INFO:journalctl@ceph.mon.a.smithi026.stdout: self._run(expiration_interval) 2024-09-11T23:09:14.243 INFO:journalctl@ceph.mon.a.smithi026.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-11T23:09:14.243 INFO:journalctl@ceph.mon.a.smithi026.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-11T23:09:14.243 INFO:journalctl@ceph.mon.a.smithi026.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-11T23:09:14.243 INFO:journalctl@ceph.mon.a.smithi026.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-11T23:09:14.243 INFO:journalctl@ceph.mon.a.smithi026.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-11T23:09:14.243 INFO:journalctl@ceph.mon.a.smithi026.stdout: s = self.context.wrap_socket( 2024-09-11T23:09:14.243 INFO:journalctl@ceph.mon.a.smithi026.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-11T23:09:14.243 INFO:journalctl@ceph.mon.a.smithi026.stdout: return self.sslsocket_class._create( 2024-09-11T23:09:14.243 INFO:journalctl@ceph.mon.a.smithi026.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-11T23:09:14.243 INFO:journalctl@ceph.mon.a.smithi026.stdout: self.do_handshake() 2024-09-11T23:09:14.243 INFO:journalctl@ceph.mon.a.smithi026.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-11T23:09:14.243 INFO:journalctl@ceph.mon.a.smithi026.stdout: self._sslobj.do_handshake() 2024-09-11T23:09:14.243 INFO:journalctl@ceph.mon.a.smithi026.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-11T23:09:14.244 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:09:14 smithi026 ceph-mon[31172]: [11/Sep/2024:23:09:13] ENGINE Serving on http://172.21.15.26:8765 2024-09-11T23:09:14.244 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:09:14 smithi026 ceph-mon[31172]: [11/Sep/2024:23:09:13] ENGINE Bus STARTED 2024-09-11T23:09:14.244 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:09:14 smithi026 ceph-mon[31172]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:09:14.879 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:09:14 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: Generating public/private rsa key pair. 2024-09-11T23:09:14.879 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:09:14 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: Your identification has been saved in /tmp/tmpf2kr83e0/key 2024-09-11T23:09:14.879 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:09:14 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: Your public key has been saved in /tmp/tmpf2kr83e0/key.pub 2024-09-11T23:09:14.879 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:09:14 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: The key fingerprint is: 2024-09-11T23:09:14.879 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:09:14 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: SHA256:JN31tHgDeno5NnIM0vLB59kLUpRAtKKhzQGWoVH7sdE ceph-af7e6fc8-7092-11ef-bcea-c7b262605968 2024-09-11T23:09:14.879 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:09:14 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: The key's randomart image is: 2024-09-11T23:09:14.879 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:09:14 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: +---[RSA 3072]----+ 2024-09-11T23:09:14.879 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:09:14 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: | ..=o o+.+.. | 2024-09-11T23:09:14.879 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:09:14 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: | +.o o + =.= . | 2024-09-11T23:09:14.880 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:09:14 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: | . . * E O = = | 2024-09-11T23:09:14.880 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:09:14 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: | = X = X = . | 2024-09-11T23:09:14.880 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:09:14 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: | . * S = % . | 2024-09-11T23:09:14.880 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:09:14 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: | * + . | 2024-09-11T23:09:14.880 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:09:14 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: | . | 2024-09-11T23:09:14.880 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:09:14 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: | | 2024-09-11T23:09:14.880 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:09:14 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: | | 2024-09-11T23:09:14.880 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:09:14 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: +----[SHA256]-----+ 2024-09-11T23:09:15.163 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:14.245+0000 7f59cbb95640 1 Processor -- start 2024-09-11T23:09:15.163 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:14.246+0000 7f59cbb95640 1 -- start start 2024-09-11T23:09:15.163 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:14.246+0000 7f59cbb95640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f59c41077d0 0x7f59c4107bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:09:15.163 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:14.246+0000 7f59cbb95640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f59c41081a0 con 0x7f59c41077d0 2024-09-11T23:09:15.164 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:14.246+0000 7f59c9931640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f59c41077d0 0x7f59c4107bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:09:15.164 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:14.246+0000 7f59c9931640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f59c41077d0 0x7f59c4107bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45460/0 (socket says 172.21.15.26:45460) 2024-09-11T23:09:15.164 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:14.246+0000 7f59c9931640 1 -- 172.21.15.26:0/1798019057 learned_addr learned my addr 172.21.15.26:0/1798019057 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:09:15.164 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:14.246+0000 7f59c9931640 1 -- 172.21.15.26:0/1798019057 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59c41089d0 con 0x7f59c41077d0 2024-09-11T23:09:15.164 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:14.247+0000 7f59c9931640 1 --2- 172.21.15.26:0/1798019057 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f59c41077d0 0x7f59c4107bd0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f59c0009820 tx=0x7f59c002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd765f29d6edd644 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:09:15.164 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:14.247+0000 7f59c892f640 1 -- 172.21.15.26:0/1798019057 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f59c0037070 con 0x7f59c41077d0 2024-09-11T23:09:15.164 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:14.247+0000 7f59c892f640 1 -- 172.21.15.26:0/1798019057 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f59c0032900 con 0x7f59c41077d0 2024-09-11T23:09:15.164 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:14.247+0000 7f59c892f640 1 -- 172.21.15.26:0/1798019057 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f59c0030850 con 0x7f59c41077d0 2024-09-11T23:09:15.164 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:14.248+0000 7f59cbb95640 1 -- 172.21.15.26:0/1798019057 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f59c41077d0 msgr2=0x7f59c4107bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:09:15.164 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:14.248+0000 7f59cbb95640 1 --2- 172.21.15.26:0/1798019057 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f59c41077d0 0x7f59c4107bd0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f59c0009820 tx=0x7f59c002a270 comp rx=0 tx=0).stop 2024-09-11T23:09:15.164 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:14.248+0000 7f59cbb95640 1 -- 172.21.15.26:0/1798019057 shutdown_connections 2024-09-11T23:09:15.164 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:14.248+0000 7f59cbb95640 1 --2- 172.21.15.26:0/1798019057 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f59c41077d0 0x7f59c4107bd0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:09:15.165 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:14.248+0000 7f59cbb95640 1 -- 172.21.15.26:0/1798019057 >> 172.21.15.26:0/1798019057 conn(0x7f59c4102f60 msgr2=0x7f59c41053a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:09:15.165 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:14.248+0000 7f59cbb95640 1 -- 172.21.15.26:0/1798019057 shutdown_connections 2024-09-11T23:09:15.165 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:14.248+0000 7f59cbb95640 1 -- 172.21.15.26:0/1798019057 wait complete. 2024-09-11T23:09:15.165 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:14.249+0000 7f59cbb95640 1 Processor -- start 2024-09-11T23:09:15.165 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:14.249+0000 7f59cbb95640 1 -- start start 2024-09-11T23:09:15.165 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:14.249+0000 7f59cbb95640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f59c41077d0 0x7f59c419d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:09:15.165 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:14.249+0000 7f59cbb95640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f59c410bd80 con 0x7f59c41077d0 2024-09-11T23:09:15.165 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:14.249+0000 7f59c9931640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f59c41077d0 0x7f59c419d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:09:15.165 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:14.249+0000 7f59c9931640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f59c41077d0 0x7f59c419d9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45466/0 (socket says 172.21.15.26:45466) 2024-09-11T23:09:15.165 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:14.249+0000 7f59c9931640 1 -- 172.21.15.26:0/2906795861 learned_addr learned my addr 172.21.15.26:0/2906795861 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:09:15.165 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:14.249+0000 7f59c9931640 1 -- 172.21.15.26:0/2906795861 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59c419df20 con 0x7f59c41077d0 2024-09-11T23:09:15.165 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:14.249+0000 7f59c9931640 1 --2- 172.21.15.26:0/2906795861 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f59c41077d0 0x7f59c419d9e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f59c00097f0 tx=0x7f59c0030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:09:15.165 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:14.250+0000 7f59baffd640 1 -- 172.21.15.26:0/2906795861 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f59c0037040 con 0x7f59c41077d0 2024-09-11T23:09:15.165 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:14.250+0000 7f59baffd640 1 -- 172.21.15.26:0/2906795861 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f59c0040070 con 0x7f59c41077d0 2024-09-11T23:09:15.166 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:14.250+0000 7f59cbb95640 1 -- 172.21.15.26:0/2906795861 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59c419e1b0 con 0x7f59c41077d0 2024-09-11T23:09:15.166 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:14.250+0000 7f59baffd640 1 -- 172.21.15.26:0/2906795861 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f59c003a390 con 0x7f59c41077d0 2024-09-11T23:09:15.166 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:14.250+0000 7f59cbb95640 1 -- 172.21.15.26:0/2906795861 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59c41a0ea0 con 0x7f59c41077d0 2024-09-11T23:09:15.166 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:14.251+0000 7f59baffd640 1 -- 172.21.15.26:0/2906795861 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f59c0032900 con 0x7f59c41077d0 2024-09-11T23:09:15.166 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:14.251+0000 7f59cbb95640 1 -- 172.21.15.26:0/2906795861 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59c4107c50 con 0x7f59c41077d0 2024-09-11T23:09:15.166 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:14.251+0000 7f59baffd640 1 --2- 172.21.15.26:0/2906795861 >> [v2:172.21.15.26:6800/677148151,v1:172.21.15.26:6801/677148151] conn(0x7f59b0038fc0 0x7f59b003b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:09:15.166 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:14.251+0000 7f59baffd640 1 -- 172.21.15.26:0/2906795861 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f59c006c440 con 0x7f59c41077d0 2024-09-11T23:09:15.166 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:14.251+0000 7f59c9130640 1 --2- 172.21.15.26:0/2906795861 >> [v2:172.21.15.26:6800/677148151,v1:172.21.15.26:6801/677148151] conn(0x7f59b0038fc0 0x7f59b003b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:09:15.166 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:14.251+0000 7f59c9130640 1 --2- 172.21.15.26:0/2906795861 >> [v2:172.21.15.26:6800/677148151,v1:172.21.15.26:6801/677148151] conn(0x7f59b0038fc0 0x7f59b003b480 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f59b40098c0 tx=0x7f59b4006db0 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:09:15.166 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:14.256+0000 7f59baffd640 1 -- 172.21.15.26:0/2906795861 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f59c003a5e0 con 0x7f59c41077d0 2024-09-11T23:09:15.167 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:14.434+0000 7f59cbb95640 1 -- 172.21.15.26:0/2906795861 --> [v2:172.21.15.26:6800/677148151,v1:172.21.15.26:6801/677148151] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7f59c41041d0 con 0x7f59b0038fc0 2024-09-11T23:09:15.167 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:14.561+0000 7f59baffd640 1 -- 172.21.15.26:0/2906795861 <== mgr.14116 v2:172.21.15.26:6800/677148151 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f59c41041d0 con 0x7f59b0038fc0 2024-09-11T23:09:15.167 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:14.563+0000 7f59cbb95640 1 -- 172.21.15.26:0/2906795861 >> [v2:172.21.15.26:6800/677148151,v1:172.21.15.26:6801/677148151] conn(0x7f59b0038fc0 msgr2=0x7f59b003b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:09:15.167 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:14.563+0000 7f59cbb95640 1 --2- 172.21.15.26:0/2906795861 >> [v2:172.21.15.26:6800/677148151,v1:172.21.15.26:6801/677148151] conn(0x7f59b0038fc0 0x7f59b003b480 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f59b40098c0 tx=0x7f59b4006db0 comp rx=0 tx=0).stop 2024-09-11T23:09:15.167 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:14.563+0000 7f59cbb95640 1 -- 172.21.15.26:0/2906795861 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f59c41077d0 msgr2=0x7f59c419d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:09:15.167 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:14.563+0000 7f59cbb95640 1 --2- 172.21.15.26:0/2906795861 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f59c41077d0 0x7f59c419d9e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f59c00097f0 tx=0x7f59c0030d80 comp rx=0 tx=0).stop 2024-09-11T23:09:15.167 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:14.563+0000 7f59cbb95640 1 -- 172.21.15.26:0/2906795861 shutdown_connections 2024-09-11T23:09:15.167 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:14.563+0000 7f59cbb95640 1 --2- 172.21.15.26:0/2906795861 >> [v2:172.21.15.26:6800/677148151,v1:172.21.15.26:6801/677148151] conn(0x7f59b0038fc0 0x7f59b003b480 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:09:15.167 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:14.563+0000 7f59cbb95640 1 --2- 172.21.15.26:0/2906795861 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f59c41077d0 0x7f59c419d9e0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:09:15.167 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:14.563+0000 7f59cbb95640 1 -- 172.21.15.26:0/2906795861 >> 172.21.15.26:0/2906795861 conn(0x7f59c4102f60 msgr2=0x7f59c4103ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:09:15.167 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:14.564+0000 7f59cbb95640 1 -- 172.21.15.26:0/2906795861 shutdown_connections 2024-09-11T23:09:15.167 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:14.564+0000 7f59cbb95640 1 -- 172.21.15.26:0/2906795861 wait complete. 2024-09-11T23:09:15.749 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:09:15 smithi026 ceph-mon[31172]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:09:15.750 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:09:15 smithi026 ceph-mon[31172]: Generating ssh key... 2024-09-11T23:09:15.750 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:09:15 smithi026 ceph-mon[31172]: from='mgr.14116 172.21.15.26:0/3833484538' entity='mgr.x' 2024-09-11T23:09:15.750 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:09:15 smithi026 ceph-mon[31172]: from='mgr.14116 172.21.15.26:0/3833484538' entity='mgr.x' 2024-09-11T23:09:16.319 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCFk7Bjp26C4V+TSSoQjcdWbS8+pBY9JCBMm0AmEJqupQfY2rcrR/++Sooc2IShw9gu63Cvz7OCDriE0Z569kd+Jf7HXIT3WOH8nS/+F6NXMOvAxMFgS+MR1ycuFNtrD4zUqUs0cxQulzXCFt+Z86vnyIytrFg92hc5DQPWOAEPtbc07U6NBxnoi6s0nf7E5jeHBVjqXATOcDkcBIN0NyseFHA0usuGowhXne5DKFgFcZ4cxNrO0fQt/RK9TUW8DkBc3ETxksEVnkUG65+mV18C3BTSiybGArsnXWoS6dPBmlUwv4LqunI8FXqgLKj1pJC/ACsLZOo4pFg/w9l6qhYsJPCGeWJmrzlLtrJyXXtHFRzc5ouM8gm8sJkmW29Eri6ERlaX2SKyGMsuZNfcMDhyv0h+/OSUToFlVv/fzEfLVzPB3xI9n1Tws2TCDBYne7UcZW95cbrLFsXqiL1bAZsS8OLWpovxcndTk/DQlO9FUzxNpahMpcodBHNg/6iusB8= ceph-af7e6fc8-7092-11ef-bcea-c7b262605968 2024-09-11T23:09:16.319 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:15.560+0000 7f45668fc640 1 Processor -- start 2024-09-11T23:09:16.319 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:15.560+0000 7f45668fc640 1 -- start start 2024-09-11T23:09:16.319 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:15.561+0000 7f45668fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f45601066d0 0x7f4560106ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:09:16.319 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:15.561+0000 7f45668fc640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f45601070a0 con 0x7f45601066d0 2024-09-11T23:09:16.319 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:15.561+0000 7f455ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f45601066d0 0x7f4560106ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:09:16.319 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:15.561+0000 7f455ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f45601066d0 0x7f4560106ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45472/0 (socket says 172.21.15.26:45472) 2024-09-11T23:09:16.319 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:15.561+0000 7f455ffff640 1 -- 172.21.15.26:0/4075023154 learned_addr learned my addr 172.21.15.26:0/4075023154 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:09:16.319 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:15.561+0000 7f455ffff640 1 -- 172.21.15.26:0/4075023154 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45601078d0 con 0x7f45601066d0 2024-09-11T23:09:16.319 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:15.562+0000 7f455ffff640 1 --2- 172.21.15.26:0/4075023154 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f45601066d0 0x7f4560106ad0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f4550009820 tx=0x7f455002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=318f1d5d0f65ca04 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:09:16.320 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:15.562+0000 7f455effd640 1 -- 172.21.15.26:0/4075023154 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4550037070 con 0x7f45601066d0 2024-09-11T23:09:16.320 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:15.562+0000 7f455effd640 1 -- 172.21.15.26:0/4075023154 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4550032900 con 0x7f45601066d0 2024-09-11T23:09:16.320 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:15.562+0000 7f455effd640 1 -- 172.21.15.26:0/4075023154 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4550030850 con 0x7f45601066d0 2024-09-11T23:09:16.320 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:15.563+0000 7f45668fc640 1 -- 172.21.15.26:0/4075023154 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f45601066d0 msgr2=0x7f4560106ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:09:16.320 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:15.563+0000 7f45668fc640 1 --2- 172.21.15.26:0/4075023154 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f45601066d0 0x7f4560106ad0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f4550009820 tx=0x7f455002a270 comp rx=0 tx=0).stop 2024-09-11T23:09:16.320 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:15.563+0000 7f45668fc640 1 -- 172.21.15.26:0/4075023154 shutdown_connections 2024-09-11T23:09:16.320 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:15.563+0000 7f45668fc640 1 --2- 172.21.15.26:0/4075023154 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f45601066d0 0x7f4560106ad0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:09:16.320 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:15.563+0000 7f45668fc640 1 -- 172.21.15.26:0/4075023154 >> 172.21.15.26:0/4075023154 conn(0x7f4560101e60 msgr2=0x7f45601042a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:09:16.321 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:15.563+0000 7f45668fc640 1 -- 172.21.15.26:0/4075023154 shutdown_connections 2024-09-11T23:09:16.321 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:15.563+0000 7f45668fc640 1 -- 172.21.15.26:0/4075023154 wait complete. 2024-09-11T23:09:16.321 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:15.563+0000 7f45668fc640 1 Processor -- start 2024-09-11T23:09:16.321 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:15.563+0000 7f45668fc640 1 -- start start 2024-09-11T23:09:16.321 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:15.564+0000 7f45668fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f45601066d0 0x7f456019b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:09:16.321 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:15.564+0000 7f45668fc640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f456010ac80 con 0x7f45601066d0 2024-09-11T23:09:16.321 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:15.564+0000 7f455ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f45601066d0 0x7f456019b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:09:16.321 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:15.564+0000 7f455ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f45601066d0 0x7f456019b7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45480/0 (socket says 172.21.15.26:45480) 2024-09-11T23:09:16.321 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:15.564+0000 7f455ffff640 1 -- 172.21.15.26:0/4120445519 learned_addr learned my addr 172.21.15.26:0/4120445519 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:09:16.321 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:15.564+0000 7f455ffff640 1 -- 172.21.15.26:0/4120445519 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f456019bd00 con 0x7f45601066d0 2024-09-11T23:09:16.321 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:15.564+0000 7f455ffff640 1 --2- 172.21.15.26:0/4120445519 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f45601066d0 0x7f456019b7c0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f45500097f0 tx=0x7f4550030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:09:16.321 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:15.564+0000 7f455d7fa640 1 -- 172.21.15.26:0/4120445519 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4550037040 con 0x7f45601066d0 2024-09-11T23:09:16.321 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:15.565+0000 7f455d7fa640 1 -- 172.21.15.26:0/4120445519 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4550040070 con 0x7f45601066d0 2024-09-11T23:09:16.322 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:15.565+0000 7f45668fc640 1 -- 172.21.15.26:0/4120445519 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f456019bf90 con 0x7f45601066d0 2024-09-11T23:09:16.322 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:15.565+0000 7f455d7fa640 1 -- 172.21.15.26:0/4120445519 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f455003a390 con 0x7f45601066d0 2024-09-11T23:09:16.322 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:15.565+0000 7f45668fc640 1 -- 172.21.15.26:0/4120445519 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f456019ec80 con 0x7f45601066d0 2024-09-11T23:09:16.322 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:15.565+0000 7f455d7fa640 1 -- 172.21.15.26:0/4120445519 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f455003a5e0 con 0x7f45601066d0 2024-09-11T23:09:16.322 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:15.566+0000 7f455d7fa640 1 --2- 172.21.15.26:0/4120445519 >> [v2:172.21.15.26:6800/677148151,v1:172.21.15.26:6801/677148151] conn(0x7f4548038fc0 0x7f454803b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:09:16.322 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:15.566+0000 7f455d7fa640 1 -- 172.21.15.26:0/4120445519 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f455006c450 con 0x7f45601066d0 2024-09-11T23:09:16.322 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:15.566+0000 7f455f7fe640 1 --2- 172.21.15.26:0/4120445519 >> [v2:172.21.15.26:6800/677148151,v1:172.21.15.26:6801/677148151] conn(0x7f4548038fc0 0x7f454803b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:09:16.322 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:15.566+0000 7f45668fc640 1 -- 172.21.15.26:0/4120445519 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4560106b50 con 0x7f45601066d0 2024-09-11T23:09:16.322 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:15.566+0000 7f455f7fe640 1 --2- 172.21.15.26:0/4120445519 >> [v2:172.21.15.26:6800/677148151,v1:172.21.15.26:6801/677148151] conn(0x7f4548038fc0 0x7f454803b480 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f45540098c0 tx=0x7f4554006db0 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:09:16.322 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:15.571+0000 7f455d7fa640 1 -- 172.21.15.26:0/4120445519 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4550043760 con 0x7f45601066d0 2024-09-11T23:09:16.322 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:15.749+0000 7f45668fc640 1 -- 172.21.15.26:0/4120445519 --> [v2:172.21.15.26:6800/677148151,v1:172.21.15.26:6801/677148151] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7f456002d9f0 con 0x7f4548038fc0 2024-09-11T23:09:16.322 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:15.750+0000 7f455d7fa640 1 -- 172.21.15.26:0/4120445519 <== mgr.14116 v2:172.21.15.26:6800/677148151 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7f456002d9f0 con 0x7f4548038fc0 2024-09-11T23:09:16.323 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:15.752+0000 7f45668fc640 1 -- 172.21.15.26:0/4120445519 >> [v2:172.21.15.26:6800/677148151,v1:172.21.15.26:6801/677148151] conn(0x7f4548038fc0 msgr2=0x7f454803b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:09:16.323 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:15.752+0000 7f45668fc640 1 --2- 172.21.15.26:0/4120445519 >> [v2:172.21.15.26:6800/677148151,v1:172.21.15.26:6801/677148151] conn(0x7f4548038fc0 0x7f454803b480 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f45540098c0 tx=0x7f4554006db0 comp rx=0 tx=0).stop 2024-09-11T23:09:16.323 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:15.752+0000 7f45668fc640 1 -- 172.21.15.26:0/4120445519 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f45601066d0 msgr2=0x7f456019b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:09:16.323 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:15.752+0000 7f45668fc640 1 --2- 172.21.15.26:0/4120445519 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f45601066d0 0x7f456019b7c0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f45500097f0 tx=0x7f4550030d80 comp rx=0 tx=0).stop 2024-09-11T23:09:16.323 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:15.753+0000 7f45668fc640 1 -- 172.21.15.26:0/4120445519 shutdown_connections 2024-09-11T23:09:16.323 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:15.753+0000 7f45668fc640 1 --2- 172.21.15.26:0/4120445519 >> [v2:172.21.15.26:6800/677148151,v1:172.21.15.26:6801/677148151] conn(0x7f4548038fc0 0x7f454803b480 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:09:16.323 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:15.753+0000 7f45668fc640 1 --2- 172.21.15.26:0/4120445519 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f45601066d0 0x7f456019b7c0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:09:16.323 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:15.753+0000 7f45668fc640 1 -- 172.21.15.26:0/4120445519 >> 172.21.15.26:0/4120445519 conn(0x7f4560101e60 msgr2=0x7f4560102e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:09:16.323 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:15.753+0000 7f45668fc640 1 -- 172.21.15.26:0/4120445519 shutdown_connections 2024-09-11T23:09:16.323 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:15.753+0000 7f45668fc640 1 -- 172.21.15.26:0/4120445519 wait complete. 2024-09-11T23:09:16.323 INFO:teuthology.orchestra.run.smithi026.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-09-11T23:09:16.323 INFO:teuthology.orchestra.run.smithi026.stdout:Adding key to root@localhost authorized_keys... 2024-09-11T23:09:16.325 INFO:teuthology.orchestra.run.smithi026.stdout:Adding host smithi026... 2024-09-11T23:09:16.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:09:16 smithi026 ceph-mon[31172]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:09:17.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:09:17 smithi026 ceph-mon[31172]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi026", "addr": "172.21.15.26", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:09:18.851 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:09:18 smithi026 ceph-mon[31172]: Deploying cephadm binary to smithi026 2024-09-11T23:09:21.410 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout Added host 'smithi026' with addr '172.21.15.26' 2024-09-11T23:09:21.410 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:16.743+0000 7fdc98a0e640 1 Processor -- start 2024-09-11T23:09:21.411 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:16.743+0000 7fdc98a0e640 1 -- start start 2024-09-11T23:09:21.411 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:16.743+0000 7fdc98a0e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdc941077d0 0x7fdc94107bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:09:21.411 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:16.743+0000 7fdc98a0e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fdc941081a0 con 0x7fdc941077d0 2024-09-11T23:09:21.411 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:16.743+0000 7fdc9259c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdc941077d0 0x7fdc94107bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:09:21.411 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:16.743+0000 7fdc9259c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdc941077d0 0x7fdc94107bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45496/0 (socket says 172.21.15.26:45496) 2024-09-11T23:09:21.411 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:16.743+0000 7fdc9259c640 1 -- 172.21.15.26:0/57137881 learned_addr learned my addr 172.21.15.26:0/57137881 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:09:21.411 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:16.744+0000 7fdc9259c640 1 -- 172.21.15.26:0/57137881 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc941089d0 con 0x7fdc941077d0 2024-09-11T23:09:21.411 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:16.744+0000 7fdc9259c640 1 --2- 172.21.15.26:0/57137881 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdc941077d0 0x7fdc94107bd0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fdc84009820 tx=0x7fdc8402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da77037c8d332811 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:09:21.411 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:16.745+0000 7fdc9159a640 1 -- 172.21.15.26:0/57137881 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdc84037070 con 0x7fdc941077d0 2024-09-11T23:09:21.411 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:16.745+0000 7fdc9159a640 1 -- 172.21.15.26:0/57137881 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fdc84032900 con 0x7fdc941077d0 2024-09-11T23:09:21.411 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:16.745+0000 7fdc9159a640 1 -- 172.21.15.26:0/57137881 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdc84030850 con 0x7fdc941077d0 2024-09-11T23:09:21.411 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:16.745+0000 7fdc98a0e640 1 -- 172.21.15.26:0/57137881 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdc941077d0 msgr2=0x7fdc94107bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:09:21.412 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:16.745+0000 7fdc98a0e640 1 --2- 172.21.15.26:0/57137881 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdc941077d0 0x7fdc94107bd0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fdc84009820 tx=0x7fdc8402a270 comp rx=0 tx=0).stop 2024-09-11T23:09:21.412 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:16.745+0000 7fdc98a0e640 1 -- 172.21.15.26:0/57137881 shutdown_connections 2024-09-11T23:09:21.412 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:16.745+0000 7fdc98a0e640 1 --2- 172.21.15.26:0/57137881 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdc941077d0 0x7fdc94107bd0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:09:21.412 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:16.745+0000 7fdc98a0e640 1 -- 172.21.15.26:0/57137881 >> 172.21.15.26:0/57137881 conn(0x7fdc94102f60 msgr2=0x7fdc941053a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:09:21.412 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:16.745+0000 7fdc98a0e640 1 -- 172.21.15.26:0/57137881 shutdown_connections 2024-09-11T23:09:21.412 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:16.745+0000 7fdc98a0e640 1 -- 172.21.15.26:0/57137881 wait complete. 2024-09-11T23:09:21.412 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:16.746+0000 7fdc98a0e640 1 Processor -- start 2024-09-11T23:09:21.412 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:16.746+0000 7fdc98a0e640 1 -- start start 2024-09-11T23:09:21.412 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:16.746+0000 7fdc98a0e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdc941077d0 0x7fdc9419b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:09:21.412 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:16.746+0000 7fdc98a0e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fdc9410bd80 con 0x7fdc941077d0 2024-09-11T23:09:21.412 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:16.746+0000 7fdc9259c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdc941077d0 0x7fdc9419b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:09:21.412 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:16.746+0000 7fdc9259c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdc941077d0 0x7fdc9419b760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45512/0 (socket says 172.21.15.26:45512) 2024-09-11T23:09:21.412 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:16.746+0000 7fdc9259c640 1 -- 172.21.15.26:0/466111623 learned_addr learned my addr 172.21.15.26:0/466111623 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:09:21.413 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:16.747+0000 7fdc9259c640 1 -- 172.21.15.26:0/466111623 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc9419bca0 con 0x7fdc941077d0 2024-09-11T23:09:21.413 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:16.747+0000 7fdc9259c640 1 --2- 172.21.15.26:0/466111623 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdc941077d0 0x7fdc9419b760 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fdc84009940 tx=0x7fdc84032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:09:21.413 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:16.747+0000 7fdc837fe640 1 -- 172.21.15.26:0/466111623 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdc84042070 con 0x7fdc941077d0 2024-09-11T23:09:21.413 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:16.747+0000 7fdc837fe640 1 -- 172.21.15.26:0/466111623 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fdc840308c0 con 0x7fdc941077d0 2024-09-11T23:09:21.413 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:16.747+0000 7fdc98a0e640 1 -- 172.21.15.26:0/466111623 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc9419bf30 con 0x7fdc941077d0 2024-09-11T23:09:21.413 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:16.747+0000 7fdc837fe640 1 -- 172.21.15.26:0/466111623 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdc84037040 con 0x7fdc941077d0 2024-09-11T23:09:21.413 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:16.747+0000 7fdc98a0e640 1 -- 172.21.15.26:0/466111623 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc9419ec20 con 0x7fdc941077d0 2024-09-11T23:09:21.413 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:16.748+0000 7fdc837fe640 1 -- 172.21.15.26:0/466111623 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fdc84045070 con 0x7fdc941077d0 2024-09-11T23:09:21.413 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:16.748+0000 7fdc98a0e640 1 -- 172.21.15.26:0/466111623 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc94107c50 con 0x7fdc941077d0 2024-09-11T23:09:21.413 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:16.748+0000 7fdc837fe640 1 --2- 172.21.15.26:0/466111623 >> [v2:172.21.15.26:6800/677148151,v1:172.21.15.26:6801/677148151] conn(0x7fdc7c038fc0 0x7fdc7c03b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:09:21.413 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:16.748+0000 7fdc837fe640 1 -- 172.21.15.26:0/466111623 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fdc8406c020 con 0x7fdc941077d0 2024-09-11T23:09:21.413 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:16.749+0000 7fdc91d9b640 1 --2- 172.21.15.26:0/466111623 >> [v2:172.21.15.26:6800/677148151,v1:172.21.15.26:6801/677148151] conn(0x7fdc7c038fc0 0x7fdc7c03b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:09:21.414 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:16.749+0000 7fdc91d9b640 1 --2- 172.21.15.26:0/466111623 >> [v2:172.21.15.26:6800/677148151,v1:172.21.15.26:6801/677148151] conn(0x7fdc7c038fc0 0x7fdc7c03b480 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fdc880098c0 tx=0x7fdc88006db0 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:09:21.414 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:16.753+0000 7fdc837fe640 1 -- 172.21.15.26:0/466111623 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fdc84045520 con 0x7fdc941077d0 2024-09-11T23:09:21.414 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:16.934+0000 7fdc98a0e640 1 -- 172.21.15.26:0/466111623 --> [v2:172.21.15.26:6800/677148151,v1:172.21.15.26:6801/677148151] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi026", "addr": "172.21.15.26", "target": ["mon-mgr", ""]}) -- 0x7fdc94104230 con 0x7fdc7c038fc0 2024-09-11T23:09:21.414 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:20.779+0000 7fdc837fe640 1 -- 172.21.15.26:0/466111623 <== mgr.14116 v2:172.21.15.26:6800/677148151 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7fdc94104230 con 0x7fdc7c038fc0 2024-09-11T23:09:21.414 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:20.781+0000 7fdc98a0e640 1 -- 172.21.15.26:0/466111623 >> [v2:172.21.15.26:6800/677148151,v1:172.21.15.26:6801/677148151] conn(0x7fdc7c038fc0 msgr2=0x7fdc7c03b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:09:21.414 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:20.781+0000 7fdc98a0e640 1 --2- 172.21.15.26:0/466111623 >> [v2:172.21.15.26:6800/677148151,v1:172.21.15.26:6801/677148151] conn(0x7fdc7c038fc0 0x7fdc7c03b480 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fdc880098c0 tx=0x7fdc88006db0 comp rx=0 tx=0).stop 2024-09-11T23:09:21.414 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:20.781+0000 7fdc98a0e640 1 -- 172.21.15.26:0/466111623 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdc941077d0 msgr2=0x7fdc9419b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:09:21.414 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:20.781+0000 7fdc98a0e640 1 --2- 172.21.15.26:0/466111623 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdc941077d0 0x7fdc9419b760 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fdc84009940 tx=0x7fdc84032ce0 comp rx=0 tx=0).stop 2024-09-11T23:09:21.414 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:20.781+0000 7fdc98a0e640 1 -- 172.21.15.26:0/466111623 shutdown_connections 2024-09-11T23:09:21.414 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:20.781+0000 7fdc98a0e640 1 --2- 172.21.15.26:0/466111623 >> [v2:172.21.15.26:6800/677148151,v1:172.21.15.26:6801/677148151] conn(0x7fdc7c038fc0 0x7fdc7c03b480 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:09:21.414 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:20.781+0000 7fdc98a0e640 1 --2- 172.21.15.26:0/466111623 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdc941077d0 0x7fdc9419b760 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:09:21.414 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:20.781+0000 7fdc98a0e640 1 -- 172.21.15.26:0/466111623 >> 172.21.15.26:0/466111623 conn(0x7fdc94102f60 msgr2=0x7fdc94103b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:09:21.414 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:20.781+0000 7fdc98a0e640 1 -- 172.21.15.26:0/466111623 shutdown_connections 2024-09-11T23:09:21.415 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:20.781+0000 7fdc98a0e640 1 -- 172.21.15.26:0/466111623 wait complete. 2024-09-11T23:09:21.415 INFO:teuthology.orchestra.run.smithi026.stdout:Deploying unmanaged mon service... 2024-09-11T23:09:22.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:09:21 smithi026 ceph-mon[31172]: from='mgr.14116 172.21.15.26:0/3833484538' entity='mgr.x' 2024-09-11T23:09:22.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:09:21 smithi026 ceph-mon[31172]: Added host smithi026 2024-09-11T23:09:22.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:09:21 smithi026 ceph-mon[31172]: from='mgr.14116 172.21.15.26:0/3833484538' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:09:22.727 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-09-11T23:09:22.727 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:21.788+0000 7f8860111640 1 Processor -- start 2024-09-11T23:09:22.727 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:21.788+0000 7f8860111640 1 -- start start 2024-09-11T23:09:22.727 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:21.788+0000 7f8860111640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8858107cc0 0x7f88581080c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:09:22.727 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:21.788+0000 7f8860111640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f8858108690 con 0x7f8858107cc0 2024-09-11T23:09:22.728 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:21.789+0000 7f885dead640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8858107cc0 0x7f88581080c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:09:22.728 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:21.789+0000 7f885dead640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8858107cc0 0x7f88581080c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47536/0 (socket says 172.21.15.26:47536) 2024-09-11T23:09:22.728 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:21.789+0000 7f885dead640 1 -- 172.21.15.26:0/3503504321 learned_addr learned my addr 172.21.15.26:0/3503504321 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:09:22.728 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:21.789+0000 7f885dead640 1 -- 172.21.15.26:0/3503504321 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8858106920 con 0x7f8858107cc0 2024-09-11T23:09:22.728 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:21.790+0000 7f885dead640 1 --2- 172.21.15.26:0/3503504321 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8858107cc0 0x7f88581080c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f8854009820 tx=0x7f885402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bbdfa6c1c12021a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:09:22.728 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:21.790+0000 7f885ceab640 1 -- 172.21.15.26:0/3503504321 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8854037070 con 0x7f8858107cc0 2024-09-11T23:09:22.728 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:21.790+0000 7f885ceab640 1 -- 172.21.15.26:0/3503504321 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8854032900 con 0x7f8858107cc0 2024-09-11T23:09:22.728 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:21.790+0000 7f885ceab640 1 -- 172.21.15.26:0/3503504321 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8854030850 con 0x7f8858107cc0 2024-09-11T23:09:22.728 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:21.790+0000 7f8860111640 1 -- 172.21.15.26:0/3503504321 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8858107cc0 msgr2=0x7f88581080c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:09:22.728 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:21.790+0000 7f8860111640 1 --2- 172.21.15.26:0/3503504321 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8858107cc0 0x7f88581080c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f8854009820 tx=0x7f885402a270 comp rx=0 tx=0).stop 2024-09-11T23:09:22.729 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:21.790+0000 7f8860111640 1 -- 172.21.15.26:0/3503504321 shutdown_connections 2024-09-11T23:09:22.729 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:21.790+0000 7f8860111640 1 --2- 172.21.15.26:0/3503504321 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8858107cc0 0x7f88581080c0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:09:22.729 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:21.790+0000 7f8860111640 1 -- 172.21.15.26:0/3503504321 >> 172.21.15.26:0/3503504321 conn(0x7f8858101600 msgr2=0x7f8858103a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:09:22.729 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:21.790+0000 7f8860111640 1 -- 172.21.15.26:0/3503504321 shutdown_connections 2024-09-11T23:09:22.729 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:21.790+0000 7f8860111640 1 -- 172.21.15.26:0/3503504321 wait complete. 2024-09-11T23:09:22.729 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:21.791+0000 7f8860111640 1 Processor -- start 2024-09-11T23:09:22.729 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:21.791+0000 7f8860111640 1 -- start start 2024-09-11T23:09:22.729 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:21.791+0000 7f8860111640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8858107cc0 0x7f885819b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:09:22.729 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:21.791+0000 7f8860111640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f885810ace0 con 0x7f8858107cc0 2024-09-11T23:09:22.730 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:21.791+0000 7f885dead640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8858107cc0 0x7f885819b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:09:22.730 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:21.791+0000 7f885dead640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8858107cc0 0x7f885819b7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47546/0 (socket says 172.21.15.26:47546) 2024-09-11T23:09:22.730 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:21.791+0000 7f885dead640 1 -- 172.21.15.26:0/2033272492 learned_addr learned my addr 172.21.15.26:0/2033272492 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:09:22.730 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:21.792+0000 7f885dead640 1 -- 172.21.15.26:0/2033272492 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f885819bd30 con 0x7f8858107cc0 2024-09-11T23:09:22.730 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:21.792+0000 7f885dead640 1 --2- 172.21.15.26:0/2033272492 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8858107cc0 0x7f885819b7f0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f88540097f0 tx=0x7f8854030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:09:22.730 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:21.792+0000 7f884effd640 1 -- 172.21.15.26:0/2033272492 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8854037040 con 0x7f8858107cc0 2024-09-11T23:09:22.730 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:21.792+0000 7f884effd640 1 -- 172.21.15.26:0/2033272492 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8854040070 con 0x7f8858107cc0 2024-09-11T23:09:22.730 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:21.792+0000 7f8860111640 1 -- 172.21.15.26:0/2033272492 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f885819bfc0 con 0x7f8858107cc0 2024-09-11T23:09:22.730 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:21.792+0000 7f884effd640 1 -- 172.21.15.26:0/2033272492 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f885403a390 con 0x7f8858107cc0 2024-09-11T23:09:22.730 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:21.792+0000 7f8860111640 1 -- 172.21.15.26:0/2033272492 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f885819ecb0 con 0x7f8858107cc0 2024-09-11T23:09:22.730 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:21.793+0000 7f884effd640 1 -- 172.21.15.26:0/2033272492 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f8854032900 con 0x7f8858107cc0 2024-09-11T23:09:22.730 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:21.793+0000 7f8860111640 1 -- 172.21.15.26:0/2033272492 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8858108140 con 0x7f8858107cc0 2024-09-11T23:09:22.730 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:21.793+0000 7f884effd640 1 --2- 172.21.15.26:0/2033272492 >> [v2:172.21.15.26:6800/677148151,v1:172.21.15.26:6801/677148151] conn(0x7f8844038fc0 0x7f884403b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:09:22.730 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:21.793+0000 7f884effd640 1 -- 172.21.15.26:0/2033272492 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f885406c440 con 0x7f8858107cc0 2024-09-11T23:09:22.731 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:21.793+0000 7f885d6ac640 1 --2- 172.21.15.26:0/2033272492 >> [v2:172.21.15.26:6800/677148151,v1:172.21.15.26:6801/677148151] conn(0x7f8844038fc0 0x7f884403b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:09:22.731 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:21.794+0000 7f885d6ac640 1 --2- 172.21.15.26:0/2033272492 >> [v2:172.21.15.26:6800/677148151,v1:172.21.15.26:6801/677148151] conn(0x7f8844038fc0 0x7f884403b480 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f88480098c0 tx=0x7f8848006db0 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:09:22.731 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:21.798+0000 7f884effd640 1 -- 172.21.15.26:0/2033272492 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f885403a5e0 con 0x7f8858107cc0 2024-09-11T23:09:22.731 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:22.019+0000 7f8860111640 1 -- 172.21.15.26:0/2033272492 --> [v2:172.21.15.26:6800/677148151,v1:172.21.15.26:6801/677148151] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f885802db70 con 0x7f8844038fc0 2024-09-11T23:09:22.731 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:22.024+0000 7f884effd640 1 -- 172.21.15.26:0/2033272492 <== mgr.14116 v2:172.21.15.26:6800/677148151 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f885802db70 con 0x7f8844038fc0 2024-09-11T23:09:22.731 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:22.026+0000 7f8860111640 1 -- 172.21.15.26:0/2033272492 >> [v2:172.21.15.26:6800/677148151,v1:172.21.15.26:6801/677148151] conn(0x7f8844038fc0 msgr2=0x7f884403b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:09:22.731 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:22.026+0000 7f8860111640 1 --2- 172.21.15.26:0/2033272492 >> [v2:172.21.15.26:6800/677148151,v1:172.21.15.26:6801/677148151] conn(0x7f8844038fc0 0x7f884403b480 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f88480098c0 tx=0x7f8848006db0 comp rx=0 tx=0).stop 2024-09-11T23:09:22.731 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:22.026+0000 7f8860111640 1 -- 172.21.15.26:0/2033272492 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8858107cc0 msgr2=0x7f885819b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:09:22.731 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:22.026+0000 7f8860111640 1 --2- 172.21.15.26:0/2033272492 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8858107cc0 0x7f885819b7f0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f88540097f0 tx=0x7f8854030d80 comp rx=0 tx=0).stop 2024-09-11T23:09:22.731 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:22.026+0000 7f8860111640 1 -- 172.21.15.26:0/2033272492 shutdown_connections 2024-09-11T23:09:22.731 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:22.026+0000 7f8860111640 1 --2- 172.21.15.26:0/2033272492 >> [v2:172.21.15.26:6800/677148151,v1:172.21.15.26:6801/677148151] conn(0x7f8844038fc0 0x7f884403b480 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:09:22.731 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:22.026+0000 7f8860111640 1 --2- 172.21.15.26:0/2033272492 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8858107cc0 0x7f885819b7f0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:09:22.731 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:22.026+0000 7f8860111640 1 -- 172.21.15.26:0/2033272492 >> 172.21.15.26:0/2033272492 conn(0x7f8858101600 msgr2=0x7f8858102580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:09:22.732 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:22.026+0000 7f8860111640 1 -- 172.21.15.26:0/2033272492 shutdown_connections 2024-09-11T23:09:22.732 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:22.026+0000 7f8860111640 1 -- 172.21.15.26:0/2033272492 wait complete. 2024-09-11T23:09:22.732 INFO:teuthology.orchestra.run.smithi026.stdout:Deploying unmanaged mgr service... 2024-09-11T23:09:23.242 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:09:23 smithi026 ceph-mon[31172]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:09:23.242 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:09:23 smithi026 ceph-mon[31172]: Saving service mon spec with placement count:5 2024-09-11T23:09:23.242 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:09:23 smithi026 ceph-mon[31172]: from='mgr.14116 172.21.15.26:0/3833484538' entity='mgr.x' 2024-09-11T23:09:24.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:09:24 smithi026 ceph-mon[31172]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:09:24.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:09:24 smithi026 ceph-mon[31172]: Saving service mgr spec with placement count:2 2024-09-11T23:09:24.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:09:24 smithi026 ceph-mon[31172]: from='mgr.14116 172.21.15.26:0/3833484538' entity='mgr.x' 2024-09-11T23:09:24.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:09:24 smithi026 ceph-mon[31172]: from='mgr.14116 172.21.15.26:0/3833484538' entity='mgr.x' 2024-09-11T23:09:24.772 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-09-11T23:09:24.773 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:23.328+0000 7fccd72ac640 1 Processor -- start 2024-09-11T23:09:24.773 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:23.328+0000 7fccd72ac640 1 -- start start 2024-09-11T23:09:24.773 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:23.329+0000 7fccd72ac640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fccd01066d0 0x7fccd0106ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:09:24.773 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:23.329+0000 7fccd72ac640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fccd01070a0 con 0x7fccd01066d0 2024-09-11T23:09:24.773 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:23.329+0000 7fccd5048640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fccd01066d0 0x7fccd0106ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:09:24.773 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:23.329+0000 7fccd5048640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fccd01066d0 0x7fccd0106ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47560/0 (socket says 172.21.15.26:47560) 2024-09-11T23:09:24.773 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:23.329+0000 7fccd5048640 1 -- 172.21.15.26:0/1061607664 learned_addr learned my addr 172.21.15.26:0/1061607664 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:09:24.773 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:23.329+0000 7fccd5048640 1 -- 172.21.15.26:0/1061607664 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccd01078d0 con 0x7fccd01066d0 2024-09-11T23:09:24.773 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:23.330+0000 7fccd5048640 1 --2- 172.21.15.26:0/1061607664 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fccd01066d0 0x7fccd0106ad0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fcccc009820 tx=0x7fcccc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c20e49a767e8942 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:09:24.773 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:23.330+0000 7fccc7fff640 1 -- 172.21.15.26:0/1061607664 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcccc037070 con 0x7fccd01066d0 2024-09-11T23:09:24.773 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:23.330+0000 7fccc7fff640 1 -- 172.21.15.26:0/1061607664 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fcccc032900 con 0x7fccd01066d0 2024-09-11T23:09:24.774 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:23.330+0000 7fccc7fff640 1 -- 172.21.15.26:0/1061607664 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcccc030850 con 0x7fccd01066d0 2024-09-11T23:09:24.774 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:23.331+0000 7fccd72ac640 1 -- 172.21.15.26:0/1061607664 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fccd01066d0 msgr2=0x7fccd0106ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:09:24.774 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:23.331+0000 7fccd72ac640 1 --2- 172.21.15.26:0/1061607664 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fccd01066d0 0x7fccd0106ad0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fcccc009820 tx=0x7fcccc02a270 comp rx=0 tx=0).stop 2024-09-11T23:09:24.774 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:23.331+0000 7fccd72ac640 1 -- 172.21.15.26:0/1061607664 shutdown_connections 2024-09-11T23:09:24.774 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:23.331+0000 7fccd72ac640 1 --2- 172.21.15.26:0/1061607664 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fccd01066d0 0x7fccd0106ad0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:09:24.774 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:23.331+0000 7fccd72ac640 1 -- 172.21.15.26:0/1061607664 >> 172.21.15.26:0/1061607664 conn(0x7fccd0101e60 msgr2=0x7fccd01042a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:09:24.774 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:23.331+0000 7fccd72ac640 1 -- 172.21.15.26:0/1061607664 shutdown_connections 2024-09-11T23:09:24.774 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:23.331+0000 7fccd72ac640 1 -- 172.21.15.26:0/1061607664 wait complete. 2024-09-11T23:09:24.774 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:23.331+0000 7fccd72ac640 1 Processor -- start 2024-09-11T23:09:24.774 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:23.331+0000 7fccd72ac640 1 -- start start 2024-09-11T23:09:24.774 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:23.332+0000 7fccd72ac640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fccd01066d0 0x7fccd019b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:09:24.774 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:23.332+0000 7fccd72ac640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fccd010ac80 con 0x7fccd01066d0 2024-09-11T23:09:24.774 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:23.332+0000 7fccd5048640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fccd01066d0 0x7fccd019b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:09:24.775 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:23.332+0000 7fccd5048640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fccd01066d0 0x7fccd019b7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47566/0 (socket says 172.21.15.26:47566) 2024-09-11T23:09:24.775 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:23.332+0000 7fccd5048640 1 -- 172.21.15.26:0/1098775329 learned_addr learned my addr 172.21.15.26:0/1098775329 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:09:24.775 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:23.332+0000 7fccd5048640 1 -- 172.21.15.26:0/1098775329 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccd019bd30 con 0x7fccd01066d0 2024-09-11T23:09:24.775 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:23.332+0000 7fccd5048640 1 --2- 172.21.15.26:0/1098775329 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fccd01066d0 0x7fccd019b7f0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fcccc0097f0 tx=0x7fcccc030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:09:24.775 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:23.332+0000 7fccc67fc640 1 -- 172.21.15.26:0/1098775329 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcccc037040 con 0x7fccd01066d0 2024-09-11T23:09:24.775 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:23.332+0000 7fccd72ac640 1 -- 172.21.15.26:0/1098775329 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccd019bfc0 con 0x7fccd01066d0 2024-09-11T23:09:24.775 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:23.333+0000 7fccc67fc640 1 -- 172.21.15.26:0/1098775329 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fcccc040070 con 0x7fccd01066d0 2024-09-11T23:09:24.775 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:23.333+0000 7fccc67fc640 1 -- 172.21.15.26:0/1098775329 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcccc03a390 con 0x7fccd01066d0 2024-09-11T23:09:24.775 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:23.333+0000 7fccd72ac640 1 -- 172.21.15.26:0/1098775329 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fccd019ecb0 con 0x7fccd01066d0 2024-09-11T23:09:24.775 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:23.333+0000 7fccc67fc640 1 -- 172.21.15.26:0/1098775329 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fcccc03a630 con 0x7fccd01066d0 2024-09-11T23:09:24.775 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:23.334+0000 7fccd72ac640 1 -- 172.21.15.26:0/1098775329 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fccd0106b50 con 0x7fccd01066d0 2024-09-11T23:09:24.775 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:23.334+0000 7fccc67fc640 1 --2- 172.21.15.26:0/1098775329 >> [v2:172.21.15.26:6800/677148151,v1:172.21.15.26:6801/677148151] conn(0x7fccbc038f70 0x7fccbc03b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:09:24.775 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:23.334+0000 7fccc67fc640 1 -- 172.21.15.26:0/1098775329 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fcccc06c5e0 con 0x7fccd01066d0 2024-09-11T23:09:24.776 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:23.334+0000 7fccd4847640 1 --2- 172.21.15.26:0/1098775329 >> [v2:172.21.15.26:6800/677148151,v1:172.21.15.26:6801/677148151] conn(0x7fccbc038f70 0x7fccbc03b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:09:24.776 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:23.334+0000 7fccd4847640 1 --2- 172.21.15.26:0/1098775329 >> [v2:172.21.15.26:6800/677148151,v1:172.21.15.26:6801/677148151] conn(0x7fccbc038f70 0x7fccbc03b430 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fccc00098c0 tx=0x7fccc0006db0 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:09:24.776 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:23.339+0000 7fccc67fc640 1 -- 172.21.15.26:0/1098775329 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fcccc0439b0 con 0x7fccd01066d0 2024-09-11T23:09:24.776 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:23.531+0000 7fccd72ac640 1 -- 172.21.15.26:0/1098775329 --> [v2:172.21.15.26:6800/677148151,v1:172.21.15.26:6801/677148151] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7fccd002da60 con 0x7fccbc038f70 2024-09-11T23:09:24.776 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:23.535+0000 7fccc67fc640 1 -- 172.21.15.26:0/1098775329 <== mgr.14116 v2:172.21.15.26:6800/677148151 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fccd002da60 con 0x7fccbc038f70 2024-09-11T23:09:24.776 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:23.538+0000 7fccd72ac640 1 -- 172.21.15.26:0/1098775329 >> [v2:172.21.15.26:6800/677148151,v1:172.21.15.26:6801/677148151] conn(0x7fccbc038f70 msgr2=0x7fccbc03b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:09:24.776 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:23.538+0000 7fccd72ac640 1 --2- 172.21.15.26:0/1098775329 >> [v2:172.21.15.26:6800/677148151,v1:172.21.15.26:6801/677148151] conn(0x7fccbc038f70 0x7fccbc03b430 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fccc00098c0 tx=0x7fccc0006db0 comp rx=0 tx=0).stop 2024-09-11T23:09:24.776 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:23.538+0000 7fccd72ac640 1 -- 172.21.15.26:0/1098775329 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fccd01066d0 msgr2=0x7fccd019b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:09:24.776 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:23.538+0000 7fccd72ac640 1 --2- 172.21.15.26:0/1098775329 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fccd01066d0 0x7fccd019b7f0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fcccc0097f0 tx=0x7fcccc030d80 comp rx=0 tx=0).stop 2024-09-11T23:09:24.776 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:23.538+0000 7fccd72ac640 1 -- 172.21.15.26:0/1098775329 shutdown_connections 2024-09-11T23:09:24.776 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:23.538+0000 7fccd72ac640 1 --2- 172.21.15.26:0/1098775329 >> [v2:172.21.15.26:6800/677148151,v1:172.21.15.26:6801/677148151] conn(0x7fccbc038f70 0x7fccbc03b430 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:09:24.776 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:23.538+0000 7fccd72ac640 1 --2- 172.21.15.26:0/1098775329 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fccd01066d0 0x7fccd019b7f0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:09:24.777 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:23.538+0000 7fccd72ac640 1 -- 172.21.15.26:0/1098775329 >> 172.21.15.26:0/1098775329 conn(0x7fccd0101e60 msgr2=0x7fccd0102e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:09:24.777 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:23.538+0000 7fccd72ac640 1 -- 172.21.15.26:0/1098775329 shutdown_connections 2024-09-11T23:09:24.777 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:23.538+0000 7fccd72ac640 1 -- 172.21.15.26:0/1098775329 wait complete. 2024-09-11T23:09:26.139 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:25.181+0000 7fd566644640 1 Processor -- start 2024-09-11T23:09:26.139 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:25.182+0000 7fd566644640 1 -- start start 2024-09-11T23:09:26.139 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:25.182+0000 7fd566644640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd56007de00 0x7fd56007c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:09:26.139 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:25.182+0000 7fd566644640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd56007c830 con 0x7fd56007de00 2024-09-11T23:09:26.139 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:25.182+0000 7fd55ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd56007de00 0x7fd56007c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:09:26.139 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:25.182+0000 7fd55ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd56007de00 0x7fd56007c260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47574/0 (socket says 172.21.15.26:47574) 2024-09-11T23:09:26.139 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:25.182+0000 7fd55ffff640 1 -- 172.21.15.26:0/3150051735 learned_addr learned my addr 172.21.15.26:0/3150051735 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:09:26.139 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:25.182+0000 7fd55ffff640 1 -- 172.21.15.26:0/3150051735 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd56007c9b0 con 0x7fd56007de00 2024-09-11T23:09:26.139 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:25.183+0000 7fd55ffff640 1 --2- 172.21.15.26:0/3150051735 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd56007de00 0x7fd56007c260 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fd550009820 tx=0x7fd55002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdafb87e4036dfd4 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:09:26.139 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:25.183+0000 7fd55effd640 1 -- 172.21.15.26:0/3150051735 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd550037070 con 0x7fd56007de00 2024-09-11T23:09:26.139 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:25.183+0000 7fd55effd640 1 -- 172.21.15.26:0/3150051735 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd550032920 con 0x7fd56007de00 2024-09-11T23:09:26.140 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:25.183+0000 7fd55effd640 1 -- 172.21.15.26:0/3150051735 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd550030810 con 0x7fd56007de00 2024-09-11T23:09:26.140 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:25.184+0000 7fd566644640 1 -- 172.21.15.26:0/3150051735 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd56007de00 msgr2=0x7fd56007c260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:09:26.140 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:25.184+0000 7fd566644640 1 --2- 172.21.15.26:0/3150051735 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd56007de00 0x7fd56007c260 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fd550009820 tx=0x7fd55002a270 comp rx=0 tx=0).stop 2024-09-11T23:09:26.140 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:25.184+0000 7fd566644640 1 -- 172.21.15.26:0/3150051735 shutdown_connections 2024-09-11T23:09:26.140 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:25.184+0000 7fd566644640 1 --2- 172.21.15.26:0/3150051735 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd56007de00 0x7fd56007c260 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:09:26.140 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:25.184+0000 7fd566644640 1 -- 172.21.15.26:0/3150051735 >> 172.21.15.26:0/3150051735 conn(0x7fd560102f60 msgr2=0x7fd5601053a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:09:26.140 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:25.184+0000 7fd566644640 1 -- 172.21.15.26:0/3150051735 shutdown_connections 2024-09-11T23:09:26.140 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:25.184+0000 7fd566644640 1 -- 172.21.15.26:0/3150051735 wait complete. 2024-09-11T23:09:26.140 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:25.185+0000 7fd566644640 1 Processor -- start 2024-09-11T23:09:26.140 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:25.185+0000 7fd566644640 1 -- start start 2024-09-11T23:09:26.140 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:25.185+0000 7fd566644640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd56007de00 0x7fd56019d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:09:26.140 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:25.185+0000 7fd566644640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd56010ddf0 con 0x7fd56007de00 2024-09-11T23:09:26.141 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:25.185+0000 7fd55ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd56007de00 0x7fd56019d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:09:26.141 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:25.185+0000 7fd55ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd56007de00 0x7fd56019d980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47586/0 (socket says 172.21.15.26:47586) 2024-09-11T23:09:26.141 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:25.185+0000 7fd55ffff640 1 -- 172.21.15.26:0/2337395424 learned_addr learned my addr 172.21.15.26:0/2337395424 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:09:26.141 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:25.185+0000 7fd55ffff640 1 -- 172.21.15.26:0/2337395424 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd56019dec0 con 0x7fd56007de00 2024-09-11T23:09:26.141 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:25.186+0000 7fd55ffff640 1 --2- 172.21.15.26:0/2337395424 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd56007de00 0x7fd56019d980 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fd550030380 tx=0x7fd550032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:09:26.141 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:25.186+0000 7fd55d7fa640 1 -- 172.21.15.26:0/2337395424 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd550037040 con 0x7fd56007de00 2024-09-11T23:09:26.141 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:25.186+0000 7fd55d7fa640 1 -- 172.21.15.26:0/2337395424 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd550040070 con 0x7fd56007de00 2024-09-11T23:09:26.141 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:25.186+0000 7fd566644640 1 -- 172.21.15.26:0/2337395424 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd56019e150 con 0x7fd56007de00 2024-09-11T23:09:26.141 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:25.186+0000 7fd55d7fa640 1 -- 172.21.15.26:0/2337395424 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd55003a7b0 con 0x7fd56007de00 2024-09-11T23:09:26.141 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:25.186+0000 7fd566644640 1 -- 172.21.15.26:0/2337395424 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5601a0e40 con 0x7fd56007de00 2024-09-11T23:09:26.141 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:25.187+0000 7fd55d7fa640 1 -- 172.21.15.26:0/2337395424 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fd55003aa30 con 0x7fd56007de00 2024-09-11T23:09:26.141 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:25.187+0000 7fd55d7fa640 1 --2- 172.21.15.26:0/2337395424 >> [v2:172.21.15.26:6800/677148151,v1:172.21.15.26:6801/677148151] conn(0x7fd548038f70 0x7fd54803b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:09:26.141 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:25.187+0000 7fd566644640 1 -- 172.21.15.26:0/2337395424 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd56007c2e0 con 0x7fd56007de00 2024-09-11T23:09:26.142 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:25.187+0000 7fd55d7fa640 1 -- 172.21.15.26:0/2337395424 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fd55006c640 con 0x7fd56007de00 2024-09-11T23:09:26.142 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:25.187+0000 7fd55f7fe640 1 --2- 172.21.15.26:0/2337395424 >> [v2:172.21.15.26:6800/677148151,v1:172.21.15.26:6801/677148151] conn(0x7fd548038f70 0x7fd54803b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:09:26.142 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:25.188+0000 7fd55f7fe640 1 --2- 172.21.15.26:0/2337395424 >> [v2:172.21.15.26:6800/677148151,v1:172.21.15.26:6801/677148151] conn(0x7fd548038f70 0x7fd54803b430 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fd5540098c0 tx=0x7fd554006db0 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:09:26.142 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:25.192+0000 7fd55d7fa640 1 -- 172.21.15.26:0/2337395424 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd550044c70 con 0x7fd56007de00 2024-09-11T23:09:26.142 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:25.373+0000 7fd566644640 1 -- 172.21.15.26:0/2337395424 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7fd5601a1400 con 0x7fd56007de00 2024-09-11T23:09:26.142 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:25.375+0000 7fd55d7fa640 1 -- 172.21.15.26:0/2337395424 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7fd550041270 con 0x7fd56007de00 2024-09-11T23:09:26.142 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:25.378+0000 7fd566644640 1 -- 172.21.15.26:0/2337395424 >> [v2:172.21.15.26:6800/677148151,v1:172.21.15.26:6801/677148151] conn(0x7fd548038f70 msgr2=0x7fd54803b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:09:26.142 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:25.378+0000 7fd566644640 1 --2- 172.21.15.26:0/2337395424 >> [v2:172.21.15.26:6800/677148151,v1:172.21.15.26:6801/677148151] conn(0x7fd548038f70 0x7fd54803b430 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fd5540098c0 tx=0x7fd554006db0 comp rx=0 tx=0).stop 2024-09-11T23:09:26.142 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:25.378+0000 7fd566644640 1 -- 172.21.15.26:0/2337395424 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd56007de00 msgr2=0x7fd56019d980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:09:26.142 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:25.378+0000 7fd566644640 1 --2- 172.21.15.26:0/2337395424 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd56007de00 0x7fd56019d980 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fd550030380 tx=0x7fd550032670 comp rx=0 tx=0).stop 2024-09-11T23:09:26.142 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:25.378+0000 7fd566644640 1 -- 172.21.15.26:0/2337395424 shutdown_connections 2024-09-11T23:09:26.142 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:25.378+0000 7fd566644640 1 --2- 172.21.15.26:0/2337395424 >> [v2:172.21.15.26:6800/677148151,v1:172.21.15.26:6801/677148151] conn(0x7fd548038f70 0x7fd54803b430 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:09:26.142 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:25.378+0000 7fd566644640 1 --2- 172.21.15.26:0/2337395424 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd56007de00 0x7fd56019d980 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:09:26.143 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:25.378+0000 7fd566644640 1 -- 172.21.15.26:0/2337395424 >> 172.21.15.26:0/2337395424 conn(0x7fd560102f60 msgr2=0x7fd560103b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:09:26.143 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:25.378+0000 7fd566644640 1 -- 172.21.15.26:0/2337395424 shutdown_connections 2024-09-11T23:09:26.143 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:25.378+0000 7fd566644640 1 -- 172.21.15.26:0/2337395424 wait complete. 2024-09-11T23:09:26.387 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:09:26 smithi026 ceph-mon[31172]: from='mgr.14116 172.21.15.26:0/3833484538' entity='mgr.x' 2024-09-11T23:09:26.387 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:09:26 smithi026 ceph-mon[31172]: from='client.? 172.21.15.26:0/2337395424' entity='client.admin' 2024-09-11T23:09:27.807 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:09:27 smithi026 ceph-mon[31172]: from='client.? 172.21.15.26:0/2907464281' entity='client.admin' 2024-09-11T23:09:27.810 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:26.577+0000 7fcdeb358640 1 Processor -- start 2024-09-11T23:09:27.815 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:26.577+0000 7fcdeb358640 1 -- start start 2024-09-11T23:09:27.816 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:26.577+0000 7fcdeb358640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcde4106090 0x7fcde4106490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:09:27.816 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:26.577+0000 7fcdeb358640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fcde407bdd0 con 0x7fcde4106090 2024-09-11T23:09:27.816 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:26.577+0000 7fcde90f4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcde4106090 0x7fcde4106490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:09:27.816 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:26.578+0000 7fcde90f4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcde4106090 0x7fcde4106490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47594/0 (socket says 172.21.15.26:47594) 2024-09-11T23:09:27.816 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:26.578+0000 7fcde90f4640 1 -- 172.21.15.26:0/847806200 learned_addr learned my addr 172.21.15.26:0/847806200 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:09:27.816 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:26.578+0000 7fcde90f4640 1 -- 172.21.15.26:0/847806200 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcde41069d0 con 0x7fcde4106090 2024-09-11T23:09:27.816 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:26.578+0000 7fcde90f4640 1 --2- 172.21.15.26:0/847806200 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcde4106090 0x7fcde4106490 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fcde0009820 tx=0x7fcde002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3beaba7e6f70a11 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:09:27.816 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:26.579+0000 7fcddbfff640 1 -- 172.21.15.26:0/847806200 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcde0037070 con 0x7fcde4106090 2024-09-11T23:09:27.816 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:26.579+0000 7fcddbfff640 1 -- 172.21.15.26:0/847806200 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcde0032920 con 0x7fcde4106090 2024-09-11T23:09:27.816 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:26.579+0000 7fcddbfff640 1 -- 172.21.15.26:0/847806200 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcde0030810 con 0x7fcde4106090 2024-09-11T23:09:27.816 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:26.580+0000 7fcdeb358640 1 -- 172.21.15.26:0/847806200 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcde4106090 msgr2=0x7fcde4106490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:09:27.817 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:26.580+0000 7fcdeb358640 1 --2- 172.21.15.26:0/847806200 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcde4106090 0x7fcde4106490 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fcde0009820 tx=0x7fcde002a270 comp rx=0 tx=0).stop 2024-09-11T23:09:27.817 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:26.580+0000 7fcdeb358640 1 -- 172.21.15.26:0/847806200 shutdown_connections 2024-09-11T23:09:27.817 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:26.580+0000 7fcdeb358640 1 --2- 172.21.15.26:0/847806200 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcde4106090 0x7fcde4106490 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:09:27.817 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:26.580+0000 7fcdeb358640 1 -- 172.21.15.26:0/847806200 >> 172.21.15.26:0/847806200 conn(0x7fcde4101e60 msgr2=0x7fcde41042a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:09:27.817 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:26.580+0000 7fcdeb358640 1 -- 172.21.15.26:0/847806200 shutdown_connections 2024-09-11T23:09:27.817 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:26.580+0000 7fcdeb358640 1 -- 172.21.15.26:0/847806200 wait complete. 2024-09-11T23:09:27.817 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:26.581+0000 7fcdeb358640 1 Processor -- start 2024-09-11T23:09:27.817 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:26.581+0000 7fcdeb358640 1 -- start start 2024-09-11T23:09:27.817 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:26.581+0000 7fcdeb358640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcde4106090 0x7fcde419d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:09:27.817 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:26.581+0000 7fcdeb358640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fcde4107aa0 con 0x7fcde4106090 2024-09-11T23:09:27.817 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:26.581+0000 7fcde90f4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcde4106090 0x7fcde419d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:09:27.817 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:26.581+0000 7fcde90f4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcde4106090 0x7fcde419d980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47608/0 (socket says 172.21.15.26:47608) 2024-09-11T23:09:27.818 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:26.581+0000 7fcde90f4640 1 -- 172.21.15.26:0/2907464281 learned_addr learned my addr 172.21.15.26:0/2907464281 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:09:27.818 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:26.581+0000 7fcde90f4640 1 -- 172.21.15.26:0/2907464281 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcde419dec0 con 0x7fcde4106090 2024-09-11T23:09:27.818 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:26.582+0000 7fcde90f4640 1 --2- 172.21.15.26:0/2907464281 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcde4106090 0x7fcde419d980 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fcde00097f0 tx=0x7fcde00327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:09:27.818 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:26.582+0000 7fcdda7fc640 1 -- 172.21.15.26:0/2907464281 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcde0037040 con 0x7fcde4106090 2024-09-11T23:09:27.818 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:26.582+0000 7fcdda7fc640 1 -- 172.21.15.26:0/2907464281 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcde0030cc0 con 0x7fcde4106090 2024-09-11T23:09:27.818 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:26.582+0000 7fcdeb358640 1 -- 172.21.15.26:0/2907464281 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcde419e150 con 0x7fcde4106090 2024-09-11T23:09:27.818 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:26.582+0000 7fcdda7fc640 1 -- 172.21.15.26:0/2907464281 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcde003a6b0 con 0x7fcde4106090 2024-09-11T23:09:27.818 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:26.582+0000 7fcdeb358640 1 -- 172.21.15.26:0/2907464281 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcde41a0e40 con 0x7fcde4106090 2024-09-11T23:09:27.818 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:26.583+0000 7fcdda7fc640 1 -- 172.21.15.26:0/2907464281 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fcde003a850 con 0x7fcde4106090 2024-09-11T23:09:27.818 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:26.583+0000 7fcdeb358640 1 -- 172.21.15.26:0/2907464281 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcde4106490 con 0x7fcde4106090 2024-09-11T23:09:27.818 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:26.583+0000 7fcdda7fc640 1 --2- 172.21.15.26:0/2907464281 >> [v2:172.21.15.26:6800/677148151,v1:172.21.15.26:6801/677148151] conn(0x7fcdd0038fc0 0x7fcdd003b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:09:27.818 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:26.583+0000 7fcdda7fc640 1 -- 172.21.15.26:0/2907464281 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fcde006c460 con 0x7fcde4106090 2024-09-11T23:09:27.818 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:26.583+0000 7fcde88f3640 1 --2- 172.21.15.26:0/2907464281 >> [v2:172.21.15.26:6800/677148151,v1:172.21.15.26:6801/677148151] conn(0x7fcdd0038fc0 0x7fcdd003b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:09:27.819 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:26.584+0000 7fcde88f3640 1 --2- 172.21.15.26:0/2907464281 >> [v2:172.21.15.26:6800/677148151,v1:172.21.15.26:6801/677148151] conn(0x7fcdd0038fc0 0x7fcdd003b480 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fcdd40098c0 tx=0x7fcdd4006db0 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:09:27.819 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:26.588+0000 7fcdda7fc640 1 -- 172.21.15.26:0/2907464281 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fcde002fd60 con 0x7fcde4106090 2024-09-11T23:09:27.819 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:26.773+0000 7fcdeb358640 1 -- 172.21.15.26:0/2907464281 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7fcde402da10 con 0x7fcde4106090 2024-09-11T23:09:27.819 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:26.775+0000 7fcdda7fc640 1 -- 172.21.15.26:0/2907464281 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7fcde0044920 con 0x7fcde4106090 2024-09-11T23:09:27.819 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:26.777+0000 7fcdeb358640 1 -- 172.21.15.26:0/2907464281 >> [v2:172.21.15.26:6800/677148151,v1:172.21.15.26:6801/677148151] conn(0x7fcdd0038fc0 msgr2=0x7fcdd003b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:09:27.819 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:26.777+0000 7fcdeb358640 1 --2- 172.21.15.26:0/2907464281 >> [v2:172.21.15.26:6800/677148151,v1:172.21.15.26:6801/677148151] conn(0x7fcdd0038fc0 0x7fcdd003b480 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fcdd40098c0 tx=0x7fcdd4006db0 comp rx=0 tx=0).stop 2024-09-11T23:09:27.819 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:26.777+0000 7fcdeb358640 1 -- 172.21.15.26:0/2907464281 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcde4106090 msgr2=0x7fcde419d980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:09:27.819 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:26.777+0000 7fcdeb358640 1 --2- 172.21.15.26:0/2907464281 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcde4106090 0x7fcde419d980 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fcde00097f0 tx=0x7fcde00327b0 comp rx=0 tx=0).stop 2024-09-11T23:09:27.819 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:26.778+0000 7fcdeb358640 1 -- 172.21.15.26:0/2907464281 shutdown_connections 2024-09-11T23:09:27.819 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:26.778+0000 7fcdeb358640 1 --2- 172.21.15.26:0/2907464281 >> [v2:172.21.15.26:6800/677148151,v1:172.21.15.26:6801/677148151] conn(0x7fcdd0038fc0 0x7fcdd003b480 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:09:27.819 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:26.778+0000 7fcdeb358640 1 --2- 172.21.15.26:0/2907464281 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcde4106090 0x7fcde419d980 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:09:27.819 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:26.778+0000 7fcdeb358640 1 -- 172.21.15.26:0/2907464281 >> 172.21.15.26:0/2907464281 conn(0x7fcde4101e60 msgr2=0x7fcde4194640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:09:27.819 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:26.778+0000 7fcdeb358640 1 -- 172.21.15.26:0/2907464281 shutdown_connections 2024-09-11T23:09:27.820 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:26.778+0000 7fcdeb358640 1 -- 172.21.15.26:0/2907464281 wait complete. 2024-09-11T23:09:27.820 INFO:teuthology.orchestra.run.smithi026.stdout:Enabling the dashboard module... 2024-09-11T23:09:29.207 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:09:28 smithi026 ceph-mon[31172]: from='mgr.14116 172.21.15.26:0/3833484538' entity='mgr.x' 2024-09-11T23:09:29.207 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:09:28 smithi026 ceph-mon[31172]: from='client.? 172.21.15.26:0/1033405536' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-09-11T23:09:29.207 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:09:28 smithi026 ceph-mon[31172]: from='mgr.14116 172.21.15.26:0/3833484538' entity='mgr.x' 2024-09-11T23:09:29.208 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:09:28 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: ignoring --setuser ceph since I am not root 2024-09-11T23:09:29.208 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:09:28 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: ignoring --setgroup ceph since I am not root 2024-09-11T23:09:29.208 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:09:29 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: 2024-09-11T23:09:29.026+0000 7faab84fc040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-11T23:09:29.208 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:09:29 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: 2024-09-11T23:09:29.085+0000 7faab84fc040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-11T23:09:29.466 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:09:29 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: 2024-09-11T23:09:29.242+0000 7faab84fc040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-11T23:09:29.466 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:09:29 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: 2024-09-11T23:09:29.312+0000 7faab84fc040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-11T23:09:29.466 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:09:29 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: 2024-09-11T23:09:29.465+0000 7faab84fc040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-11T23:09:29.506 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:28.258+0000 7fcf643af640 1 Processor -- start 2024-09-11T23:09:29.507 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:28.258+0000 7fcf643af640 1 -- start start 2024-09-11T23:09:29.507 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:28.258+0000 7fcf643af640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcf5c1066e0 0x7fcf5c106ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:09:29.507 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:28.258+0000 7fcf643af640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fcf5c1070b0 con 0x7fcf5c1066e0 2024-09-11T23:09:29.507 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:28.259+0000 7fcf6214b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcf5c1066e0 0x7fcf5c106ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:09:29.507 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:28.259+0000 7fcf6214b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcf5c1066e0 0x7fcf5c106ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47622/0 (socket says 172.21.15.26:47622) 2024-09-11T23:09:29.507 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:28.259+0000 7fcf6214b640 1 -- 172.21.15.26:0/1407184367 learned_addr learned my addr 172.21.15.26:0/1407184367 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:09:29.507 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:28.259+0000 7fcf6214b640 1 -- 172.21.15.26:0/1407184367 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf5c1078e0 con 0x7fcf5c1066e0 2024-09-11T23:09:29.507 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:28.260+0000 7fcf6214b640 1 --2- 172.21.15.26:0/1407184367 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcf5c1066e0 0x7fcf5c106ae0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fcf58009820 tx=0x7fcf5802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8bed4030daaa327 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:09:29.507 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:28.260+0000 7fcf61149640 1 -- 172.21.15.26:0/1407184367 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcf58037070 con 0x7fcf5c1066e0 2024-09-11T23:09:29.507 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:28.260+0000 7fcf61149640 1 -- 172.21.15.26:0/1407184367 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf58032920 con 0x7fcf5c1066e0 2024-09-11T23:09:29.507 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:28.260+0000 7fcf61149640 1 -- 172.21.15.26:0/1407184367 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcf58030810 con 0x7fcf5c1066e0 2024-09-11T23:09:29.507 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:28.260+0000 7fcf643af640 1 -- 172.21.15.26:0/1407184367 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcf5c1066e0 msgr2=0x7fcf5c106ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:09:29.508 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:28.260+0000 7fcf643af640 1 --2- 172.21.15.26:0/1407184367 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcf5c1066e0 0x7fcf5c106ae0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fcf58009820 tx=0x7fcf5802a270 comp rx=0 tx=0).stop 2024-09-11T23:09:29.508 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:28.260+0000 7fcf643af640 1 -- 172.21.15.26:0/1407184367 shutdown_connections 2024-09-11T23:09:29.508 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:28.260+0000 7fcf643af640 1 --2- 172.21.15.26:0/1407184367 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcf5c1066e0 0x7fcf5c106ae0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:09:29.508 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:28.260+0000 7fcf643af640 1 -- 172.21.15.26:0/1407184367 >> 172.21.15.26:0/1407184367 conn(0x7fcf5c101ed0 msgr2=0x7fcf5c1042f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:09:29.508 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:28.261+0000 7fcf643af640 1 -- 172.21.15.26:0/1407184367 shutdown_connections 2024-09-11T23:09:29.508 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:28.261+0000 7fcf643af640 1 -- 172.21.15.26:0/1407184367 wait complete. 2024-09-11T23:09:29.508 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:28.261+0000 7fcf643af640 1 Processor -- start 2024-09-11T23:09:29.508 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:28.261+0000 7fcf643af640 1 -- start start 2024-09-11T23:09:29.508 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:28.261+0000 7fcf643af640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcf5c1066e0 0x7fcf5c19b800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:09:29.508 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:28.261+0000 7fcf643af640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fcf5c10ac90 con 0x7fcf5c1066e0 2024-09-11T23:09:29.508 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:28.261+0000 7fcf6214b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcf5c1066e0 0x7fcf5c19b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:09:29.508 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:28.261+0000 7fcf6214b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcf5c1066e0 0x7fcf5c19b800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47636/0 (socket says 172.21.15.26:47636) 2024-09-11T23:09:29.509 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:28.261+0000 7fcf6214b640 1 -- 172.21.15.26:0/1033405536 learned_addr learned my addr 172.21.15.26:0/1033405536 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:09:29.509 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:28.262+0000 7fcf6214b640 1 -- 172.21.15.26:0/1033405536 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf5c19bd40 con 0x7fcf5c1066e0 2024-09-11T23:09:29.509 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:28.262+0000 7fcf6214b640 1 --2- 172.21.15.26:0/1033405536 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcf5c1066e0 0x7fcf5c19b800 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fcf58030560 tx=0x7fcf58032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:09:29.509 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:28.262+0000 7fcf537fe640 1 -- 172.21.15.26:0/1033405536 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcf58037040 con 0x7fcf5c1066e0 2024-09-11T23:09:29.509 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:28.262+0000 7fcf643af640 1 -- 172.21.15.26:0/1033405536 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf5c19bfd0 con 0x7fcf5c1066e0 2024-09-11T23:09:29.509 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:28.262+0000 7fcf537fe640 1 -- 172.21.15.26:0/1033405536 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf58040070 con 0x7fcf5c1066e0 2024-09-11T23:09:29.509 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:28.262+0000 7fcf537fe640 1 -- 172.21.15.26:0/1033405536 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcf5803a740 con 0x7fcf5c1066e0 2024-09-11T23:09:29.509 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:28.262+0000 7fcf643af640 1 -- 172.21.15.26:0/1033405536 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf5c19ecc0 con 0x7fcf5c1066e0 2024-09-11T23:09:29.509 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:28.263+0000 7fcf537fe640 1 -- 172.21.15.26:0/1033405536 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fcf5803a9b0 con 0x7fcf5c1066e0 2024-09-11T23:09:29.509 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:28.263+0000 7fcf643af640 1 -- 172.21.15.26:0/1033405536 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf5c106b60 con 0x7fcf5c1066e0 2024-09-11T23:09:29.509 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:28.263+0000 7fcf537fe640 1 --2- 172.21.15.26:0/1033405536 >> [v2:172.21.15.26:6800/677148151,v1:172.21.15.26:6801/677148151] conn(0x7fcf48038f70 0x7fcf4803b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:09:29.509 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:28.263+0000 7fcf537fe640 1 -- 172.21.15.26:0/1033405536 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fcf5806c700 con 0x7fcf5c1066e0 2024-09-11T23:09:29.510 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:28.263+0000 7fcf6194a640 1 --2- 172.21.15.26:0/1033405536 >> [v2:172.21.15.26:6800/677148151,v1:172.21.15.26:6801/677148151] conn(0x7fcf48038f70 0x7fcf4803b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:09:29.510 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:28.264+0000 7fcf6194a640 1 --2- 172.21.15.26:0/1033405536 >> [v2:172.21.15.26:6800/677148151,v1:172.21.15.26:6801/677148151] conn(0x7fcf48038f70 0x7fcf4803b430 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fcf4c0098c0 tx=0x7fcf4c006db0 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:09:29.510 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:28.268+0000 7fcf537fe640 1 -- 172.21.15.26:0/1033405536 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fcf58032920 con 0x7fcf5c1066e0 2024-09-11T23:09:29.510 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:28.495+0000 7fcf643af640 1 -- 172.21.15.26:0/1033405536 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7fcf5c02d9b0 con 0x7fcf5c1066e0 2024-09-11T23:09:29.510 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:28.868+0000 7fcf537fe640 1 -- 172.21.15.26:0/1033405536 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7fcf5803b760 con 0x7fcf5c1066e0 2024-09-11T23:09:29.510 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:28.870+0000 7fcf537fe640 1 -- 172.21.15.26:0/1033405536 <== mon.0 v2:172.21.15.26:3300/0 8 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7fcf5803aca0 con 0x7fcf5c1066e0 2024-09-11T23:09:29.510 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:28.871+0000 7fcf643af640 1 -- 172.21.15.26:0/1033405536 >> [v2:172.21.15.26:6800/677148151,v1:172.21.15.26:6801/677148151] conn(0x7fcf48038f70 msgr2=0x7fcf4803b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:09:29.510 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:28.871+0000 7fcf643af640 1 --2- 172.21.15.26:0/1033405536 >> [v2:172.21.15.26:6800/677148151,v1:172.21.15.26:6801/677148151] conn(0x7fcf48038f70 0x7fcf4803b430 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fcf4c0098c0 tx=0x7fcf4c006db0 comp rx=0 tx=0).stop 2024-09-11T23:09:29.510 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:28.871+0000 7fcf643af640 1 -- 172.21.15.26:0/1033405536 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcf5c1066e0 msgr2=0x7fcf5c19b800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:09:29.510 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:28.871+0000 7fcf643af640 1 --2- 172.21.15.26:0/1033405536 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcf5c1066e0 0x7fcf5c19b800 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fcf58030560 tx=0x7fcf58032670 comp rx=0 tx=0).stop 2024-09-11T23:09:29.510 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:28.871+0000 7fcf643af640 1 -- 172.21.15.26:0/1033405536 shutdown_connections 2024-09-11T23:09:29.510 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:28.871+0000 7fcf643af640 1 --2- 172.21.15.26:0/1033405536 >> [v2:172.21.15.26:6800/677148151,v1:172.21.15.26:6801/677148151] conn(0x7fcf48038f70 0x7fcf4803b430 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:09:29.510 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:28.871+0000 7fcf643af640 1 --2- 172.21.15.26:0/1033405536 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcf5c1066e0 0x7fcf5c19b800 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:09:29.511 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:28.871+0000 7fcf643af640 1 -- 172.21.15.26:0/1033405536 >> 172.21.15.26:0/1033405536 conn(0x7fcf5c101ed0 msgr2=0x7fcf5c102e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:09:29.511 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:28.871+0000 7fcf643af640 1 -- 172.21.15.26:0/1033405536 shutdown_connections 2024-09-11T23:09:29.511 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:28.871+0000 7fcf643af640 1 -- 172.21.15.26:0/1033405536 wait complete. 2024-09-11T23:09:29.751 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:09:29 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: 2024-09-11T23:09:29.625+0000 7faab84fc040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-11T23:09:30.031 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:09:29 smithi026 ceph-mon[31172]: from='client.? 172.21.15.26:0/1033405536' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-09-11T23:09:30.031 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:09:29 smithi026 ceph-mon[31172]: mgrmap e9: x(active, since 19s) 2024-09-11T23:09:30.031 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:09:29 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: 2024-09-11T23:09:29.964+0000 7faab84fc040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-11T23:09:30.031 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:09:30 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: 2024-09-11T23:09:30.030+0000 7faab84fc040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-11T23:09:30.501 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:09:30 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: 2024-09-11T23:09:30.233+0000 7faab84fc040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-11T23:09:30.757 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout { 2024-09-11T23:09:30.757 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-09-11T23:09:30.757 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-11T23:09:30.758 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-11T23:09:30.758 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-11T23:09:30.758 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout } 2024-09-11T23:09:30.758 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:29.949+0000 7f3076e20640 1 Processor -- start 2024-09-11T23:09:30.758 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:29.949+0000 7f3076e20640 1 -- start start 2024-09-11T23:09:30.758 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:29.949+0000 7f3076e20640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3070108240 0x7f307010a630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:09:30.758 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:29.949+0000 7f3076e20640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f307007be30 con 0x7f3070108240 2024-09-11T23:09:30.758 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:29.949+0000 7f3074bbc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3070108240 0x7f307010a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:09:30.758 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:29.949+0000 7f3074bbc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3070108240 0x7f307010a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47676/0 (socket says 172.21.15.26:47676) 2024-09-11T23:09:30.758 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:29.949+0000 7f3074bbc640 1 -- 172.21.15.26:0/3281562209 learned_addr learned my addr 172.21.15.26:0/3281562209 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:09:30.758 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:29.950+0000 7f3074bbc640 1 -- 172.21.15.26:0/3281562209 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f307010ab70 con 0x7f3070108240 2024-09-11T23:09:30.759 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:29.950+0000 7f3074bbc640 1 --2- 172.21.15.26:0/3281562209 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3070108240 0x7f307010a630 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f3058009820 tx=0x7f305802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e885471169e84034 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:09:30.759 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:29.951+0000 7f306f7fe640 1 -- 172.21.15.26:0/3281562209 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3058037070 con 0x7f3070108240 2024-09-11T23:09:30.759 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:29.951+0000 7f306f7fe640 1 -- 172.21.15.26:0/3281562209 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3058032920 con 0x7f3070108240 2024-09-11T23:09:30.759 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:29.951+0000 7f306f7fe640 1 -- 172.21.15.26:0/3281562209 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3058030810 con 0x7f3070108240 2024-09-11T23:09:30.759 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:29.951+0000 7f3076e20640 1 -- 172.21.15.26:0/3281562209 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3070108240 msgr2=0x7f307010a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:09:30.759 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:29.951+0000 7f3076e20640 1 --2- 172.21.15.26:0/3281562209 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3070108240 0x7f307010a630 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f3058009820 tx=0x7f305802a270 comp rx=0 tx=0).stop 2024-09-11T23:09:30.759 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:29.951+0000 7f3076e20640 1 -- 172.21.15.26:0/3281562209 shutdown_connections 2024-09-11T23:09:30.759 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:29.951+0000 7f3076e20640 1 --2- 172.21.15.26:0/3281562209 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3070108240 0x7f307010a630 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:09:30.759 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:29.951+0000 7f3076e20640 1 -- 172.21.15.26:0/3281562209 >> 172.21.15.26:0/3281562209 conn(0x7f3070101e60 msgr2=0x7f30701042a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:09:30.759 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:29.952+0000 7f3076e20640 1 -- 172.21.15.26:0/3281562209 shutdown_connections 2024-09-11T23:09:30.760 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:29.952+0000 7f3076e20640 1 -- 172.21.15.26:0/3281562209 wait complete. 2024-09-11T23:09:30.760 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:29.952+0000 7f3076e20640 1 Processor -- start 2024-09-11T23:09:30.760 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:29.953+0000 7f3076e20640 1 -- start start 2024-09-11T23:09:30.760 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:29.953+0000 7f3076e20640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3070108240 0x7f3070107ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:09:30.760 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:29.953+0000 7f3076e20640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f307010bbe0 con 0x7f3070108240 2024-09-11T23:09:30.760 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:29.953+0000 7f3074bbc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3070108240 0x7f3070107ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:09:30.760 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:29.953+0000 7f3074bbc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3070108240 0x7f3070107ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47688/0 (socket says 172.21.15.26:47688) 2024-09-11T23:09:30.760 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:29.953+0000 7f3074bbc640 1 -- 172.21.15.26:0/4225547747 learned_addr learned my addr 172.21.15.26:0/4225547747 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:09:30.760 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:29.954+0000 7f3074bbc640 1 -- 172.21.15.26:0/4225547747 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3070108000 con 0x7f3070108240 2024-09-11T23:09:30.760 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:29.954+0000 7f3074bbc640 1 --2- 172.21.15.26:0/4225547747 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3070108240 0x7f3070107ac0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f3058030380 tx=0x7f3058032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:09:30.760 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:29.954+0000 7f306dffb640 1 -- 172.21.15.26:0/4225547747 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3058037040 con 0x7f3070108240 2024-09-11T23:09:30.760 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:29.954+0000 7f306dffb640 1 -- 172.21.15.26:0/4225547747 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3058040070 con 0x7f3070108240 2024-09-11T23:09:30.760 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:29.954+0000 7f3076e20640 1 -- 172.21.15.26:0/4225547747 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30701061f0 con 0x7f3070108240 2024-09-11T23:09:30.761 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:29.954+0000 7f306dffb640 1 -- 172.21.15.26:0/4225547747 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f305803a7b0 con 0x7f3070108240 2024-09-11T23:09:30.761 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:29.954+0000 7f3076e20640 1 -- 172.21.15.26:0/4225547747 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3070106730 con 0x7f3070108240 2024-09-11T23:09:30.761 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:29.955+0000 7f306dffb640 1 -- 172.21.15.26:0/4225547747 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7f305803a9e0 con 0x7f3070108240 2024-09-11T23:09:30.761 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:29.956+0000 7f3076e20640 1 -- 172.21.15.26:0/4225547747 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30701086a0 con 0x7f3070108240 2024-09-11T23:09:30.761 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:29.956+0000 7f306dffb640 1 --2- 172.21.15.26:0/4225547747 >> [v2:172.21.15.26:6800/677148151,v1:172.21.15.26:6801/677148151] conn(0x7f305c039020 0x7f305c03b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:09:30.761 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:29.956+0000 7f306dffb640 1 -- 172.21.15.26:0/4225547747 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f305806c650 con 0x7f3070108240 2024-09-11T23:09:30.761 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:29.956+0000 7f306ffff640 1 -- 172.21.15.26:0/4225547747 >> [v2:172.21.15.26:6800/677148151,v1:172.21.15.26:6801/677148151] conn(0x7f305c039020 msgr2=0x7f305c03b4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.26:6800/677148151 2024-09-11T23:09:30.761 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:29.956+0000 7f306ffff640 1 --2- 172.21.15.26:0/4225547747 >> [v2:172.21.15.26:6800/677148151,v1:172.21.15.26:6801/677148151] conn(0x7f305c039020 0x7f305c03b4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-11T23:09:30.761 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:29.961+0000 7f306dffb640 1 -- 172.21.15.26:0/4225547747 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3058044c70 con 0x7f3070108240 2024-09-11T23:09:30.761 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:30.157+0000 7f306ffff640 1 -- 172.21.15.26:0/4225547747 >> [v2:172.21.15.26:6800/677148151,v1:172.21.15.26:6801/677148151] conn(0x7f305c039020 msgr2=0x7f305c03b4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.26:6800/677148151 2024-09-11T23:09:30.761 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:30.157+0000 7f306ffff640 1 --2- 172.21.15.26:0/4225547747 >> [v2:172.21.15.26:6800/677148151,v1:172.21.15.26:6801/677148151] conn(0x7f305c039020 0x7f305c03b4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-11T23:09:30.761 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:30.198+0000 7f3076e20640 1 -- 172.21.15.26:0/4225547747 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f30701069b0 con 0x7f3070108240 2024-09-11T23:09:30.761 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:30.198+0000 7f306dffb640 1 -- 172.21.15.26:0/4225547747 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7f3058041270 con 0x7f3070108240 2024-09-11T23:09:30.762 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:30.201+0000 7f3076e20640 1 -- 172.21.15.26:0/4225547747 >> [v2:172.21.15.26:6800/677148151,v1:172.21.15.26:6801/677148151] conn(0x7f305c039020 msgr2=0x7f305c03b4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-11T23:09:30.762 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:30.201+0000 7f3076e20640 1 --2- 172.21.15.26:0/4225547747 >> [v2:172.21.15.26:6800/677148151,v1:172.21.15.26:6801/677148151] conn(0x7f305c039020 0x7f305c03b4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:09:30.762 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:30.201+0000 7f3076e20640 1 -- 172.21.15.26:0/4225547747 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3070108240 msgr2=0x7f3070107ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:09:30.762 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:30.201+0000 7f3076e20640 1 --2- 172.21.15.26:0/4225547747 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3070108240 0x7f3070107ac0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f3058030380 tx=0x7f3058032670 comp rx=0 tx=0).stop 2024-09-11T23:09:30.762 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:30.201+0000 7f3076e20640 1 -- 172.21.15.26:0/4225547747 shutdown_connections 2024-09-11T23:09:30.762 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:30.201+0000 7f3076e20640 1 --2- 172.21.15.26:0/4225547747 >> [v2:172.21.15.26:6800/677148151,v1:172.21.15.26:6801/677148151] conn(0x7f305c039020 0x7f305c03b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:09:30.762 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:30.201+0000 7f3076e20640 1 --2- 172.21.15.26:0/4225547747 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3070108240 0x7f3070107ac0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:09:30.762 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:30.201+0000 7f3076e20640 1 -- 172.21.15.26:0/4225547747 >> 172.21.15.26:0/4225547747 conn(0x7f3070101e60 msgr2=0x7f3070102ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:09:30.762 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:30.201+0000 7f3076e20640 1 -- 172.21.15.26:0/4225547747 shutdown_connections 2024-09-11T23:09:30.762 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:30.201+0000 7f3076e20640 1 -- 172.21.15.26:0/4225547747 wait complete. 2024-09-11T23:09:30.762 INFO:teuthology.orchestra.run.smithi026.stdout:Waiting for the mgr to restart... 2024-09-11T23:09:30.762 INFO:teuthology.orchestra.run.smithi026.stdout:Waiting for mgr epoch 9... 2024-09-11T23:09:30.992 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:09:30 smithi026 ceph-mon[31172]: from='client.? 172.21.15.26:0/4225547747' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-11T23:09:31.243 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:09:31 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: 2024-09-11T23:09:31.170+0000 7faab84fc040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-11T23:09:31.501 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:09:31 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: 2024-09-11T23:09:31.244+0000 7faab84fc040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-11T23:09:32.119 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:09:31 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: 2024-09-11T23:09:31.997+0000 7faab84fc040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-11T23:09:32.120 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:09:32 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: 2024-09-11T23:09:32.118+0000 7faab84fc040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-11T23:09:32.489 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:09:32 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: 2024-09-11T23:09:32.268+0000 7faab84fc040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-11T23:09:32.489 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:09:32 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: 2024-09-11T23:09:32.332+0000 7faab84fc040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-11T23:09:32.489 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:09:32 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: 2024-09-11T23:09:32.487+0000 7faab84fc040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-11T23:09:32.751 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:09:32 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: 2024-09-11T23:09:32.559+0000 7faab84fc040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-11T23:09:32.751 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:09:32 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: 2024-09-11T23:09:32.621+0000 7faab84fc040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-11T23:09:33.074 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:09:32 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: 2024-09-11T23:09:32.787+0000 7faab84fc040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-11T23:09:33.074 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:09:32 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: 2024-09-11T23:09:32.855+0000 7faab84fc040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-11T23:09:33.074 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:09:32 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: 2024-09-11T23:09:32.924+0000 7faab84fc040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-11T23:09:33.074 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:09:33 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: 2024-09-11T23:09:33.011+0000 7faab84fc040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-11T23:09:33.074 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:09:33 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: 2024-09-11T23:09:33.073+0000 7faab84fc040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-11T23:09:33.392 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:09:33 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: 2024-09-11T23:09:33.135+0000 7faab84fc040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-11T23:09:34.237 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:09:33 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: 2024-09-11T23:09:33.980+0000 7faab84fc040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-11T23:09:34.237 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:09:34 smithi026 ceph-mon[31172]: Active manager daemon x restarted 2024-09-11T23:09:34.238 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:09:34 smithi026 ceph-mon[31172]: Activating manager daemon x 2024-09-11T23:09:34.238 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:09:34 smithi026 ceph-mon[31172]: osdmap e3: 0 total, 0 up, 0 in 2024-09-11T23:09:34.238 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:09:34 smithi026 ceph-mon[31172]: mgrmap e10: x(active, starting, since 0.00357472s) 2024-09-11T23:09:34.238 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:09:34 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T23:09:34.238 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:09:34 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-11T23:09:34.238 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:09:34 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-11T23:09:34.238 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:09:34 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-11T23:09:34.238 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:09:34 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-11T23:09:34.238 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:09:34 smithi026 ceph-mon[31172]: Manager daemon x is now available 2024-09-11T23:09:34.238 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:09:34 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:09:34.238 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:09:34 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-11T23:09:34.238 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:09:34 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-11T23:09:35.946 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:09:35 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: 2024-09-11T23:09:35.874+0000 7faa1dfc5640 -1 log_channel(cephadm) log [ERR] : [11/Sep/2024:23:09:35] ENGINE Error in HTTPServer.serve 2024-09-11T23:09:35.946 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:09:35 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: Traceback (most recent call last): 2024-09-11T23:09:35.947 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:09:35 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-11T23:09:35.947 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:09:35 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: self._connections.run(self.expiration_interval) 2024-09-11T23:09:35.947 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:09:35 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-11T23:09:35.947 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:09:35 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: self._run(expiration_interval) 2024-09-11T23:09:35.947 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:09:35 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-11T23:09:35.947 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:09:35 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: new_conn = self._from_server_socket(self.server.socket) 2024-09-11T23:09:35.947 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:09:35 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-11T23:09:35.947 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:09:35 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-11T23:09:35.947 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:09:35 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-11T23:09:35.947 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:09:35 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: s = self.context.wrap_socket( 2024-09-11T23:09:35.947 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:09:35 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-11T23:09:35.947 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:09:35 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: return self.sslsocket_class._create( 2024-09-11T23:09:35.948 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:09:35 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-11T23:09:35.948 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:09:35 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: self.do_handshake() 2024-09-11T23:09:35.948 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:09:35 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-11T23:09:35.948 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:09:35 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: self._sslobj.do_handshake() 2024-09-11T23:09:35.948 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:09:35 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-11T23:09:35.948 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:09:35 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: 2024-09-11T23:09:35.948 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout { 2024-09-11T23:09:35.956 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-09-11T23:09:35.956 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-11T23:09:35.956 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout } 2024-09-11T23:09:35.956 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:31.231+0000 7fee190e6640 1 Processor -- start 2024-09-11T23:09:35.956 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:31.231+0000 7fee190e6640 1 -- start start 2024-09-11T23:09:35.956 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:31.231+0000 7fee190e6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee14108260 0x7fee1410a650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:09:35.956 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:31.231+0000 7fee190e6640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fee1407be30 con 0x7fee14108260 2024-09-11T23:09:35.956 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:31.232+0000 7fee12d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee14108260 0x7fee1410a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:09:35.956 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:31.232+0000 7fee12d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee14108260 0x7fee1410a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55378/0 (socket says 172.21.15.26:55378) 2024-09-11T23:09:35.956 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:31.232+0000 7fee12d9d640 1 -- 172.21.15.26:0/2559001798 learned_addr learned my addr 172.21.15.26:0/2559001798 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:09:35.957 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:31.232+0000 7fee12d9d640 1 -- 172.21.15.26:0/2559001798 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee1410ab90 con 0x7fee14108260 2024-09-11T23:09:35.957 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:31.233+0000 7fee12d9d640 1 --2- 172.21.15.26:0/2559001798 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee14108260 0x7fee1410a650 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fee04009820 tx=0x7fee0402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80ede1e87f5f9680 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:09:35.957 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:31.233+0000 7fee1259c640 1 -- 172.21.15.26:0/2559001798 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fee04037070 con 0x7fee14108260 2024-09-11T23:09:35.957 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:31.233+0000 7fee1259c640 1 -- 172.21.15.26:0/2559001798 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee04032920 con 0x7fee14108260 2024-09-11T23:09:35.957 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:31.233+0000 7fee1259c640 1 -- 172.21.15.26:0/2559001798 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fee04030810 con 0x7fee14108260 2024-09-11T23:09:35.957 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:31.233+0000 7fee190e6640 1 -- 172.21.15.26:0/2559001798 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee14108260 msgr2=0x7fee1410a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:09:35.957 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:31.233+0000 7fee190e6640 1 --2- 172.21.15.26:0/2559001798 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee14108260 0x7fee1410a650 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fee04009820 tx=0x7fee0402a270 comp rx=0 tx=0).stop 2024-09-11T23:09:35.957 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:31.234+0000 7fee190e6640 1 -- 172.21.15.26:0/2559001798 shutdown_connections 2024-09-11T23:09:35.957 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:31.234+0000 7fee190e6640 1 --2- 172.21.15.26:0/2559001798 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee14108260 0x7fee1410a650 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:09:35.957 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:31.234+0000 7fee190e6640 1 -- 172.21.15.26:0/2559001798 >> 172.21.15.26:0/2559001798 conn(0x7fee14101e80 msgr2=0x7fee141042c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:09:35.958 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:31.234+0000 7fee190e6640 1 -- 172.21.15.26:0/2559001798 shutdown_connections 2024-09-11T23:09:35.958 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:31.234+0000 7fee190e6640 1 -- 172.21.15.26:0/2559001798 wait complete. 2024-09-11T23:09:35.958 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:31.234+0000 7fee190e6640 1 Processor -- start 2024-09-11T23:09:35.958 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:31.234+0000 7fee190e6640 1 -- start start 2024-09-11T23:09:35.958 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:31.235+0000 7fee190e6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee14108260 0x7fee1419b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:09:35.958 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:31.235+0000 7fee190e6640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fee1410bc00 con 0x7fee14108260 2024-09-11T23:09:35.958 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:31.235+0000 7fee12d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee14108260 0x7fee1419b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:09:35.958 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:31.235+0000 7fee12d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee14108260 0x7fee1419b7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55380/0 (socket says 172.21.15.26:55380) 2024-09-11T23:09:35.958 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:31.235+0000 7fee12d9d640 1 -- 172.21.15.26:0/1688399074 learned_addr learned my addr 172.21.15.26:0/1688399074 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:09:35.958 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:31.235+0000 7fee12d9d640 1 -- 172.21.15.26:0/1688399074 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee1419bd00 con 0x7fee14108260 2024-09-11T23:09:35.958 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:31.235+0000 7fee12d9d640 1 --2- 172.21.15.26:0/1688399074 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee14108260 0x7fee1419b7c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fee04030380 tx=0x7fee04032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:09:35.958 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:31.235+0000 7fee10d99640 1 -- 172.21.15.26:0/1688399074 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fee04037040 con 0x7fee14108260 2024-09-11T23:09:35.958 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:31.235+0000 7fee10d99640 1 -- 172.21.15.26:0/1688399074 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee04040070 con 0x7fee14108260 2024-09-11T23:09:35.959 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:31.235+0000 7fee190e6640 1 -- 172.21.15.26:0/1688399074 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee1419bf90 con 0x7fee14108260 2024-09-11T23:09:35.959 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:31.235+0000 7fee10d99640 1 -- 172.21.15.26:0/1688399074 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fee0403a7b0 con 0x7fee14108260 2024-09-11T23:09:35.959 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:31.235+0000 7fee190e6640 1 -- 172.21.15.26:0/1688399074 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee1419ec80 con 0x7fee14108260 2024-09-11T23:09:35.959 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:31.236+0000 7fee10d99640 1 -- 172.21.15.26:0/1688399074 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7fee0403a9e0 con 0x7fee14108260 2024-09-11T23:09:35.959 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:31.236+0000 7fee10d99640 1 --2- 172.21.15.26:0/1688399074 >> [v2:172.21.15.26:6800/677148151,v1:172.21.15.26:6801/677148151] conn(0x7fee00039020 0x7fee0003b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:09:35.959 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:31.236+0000 7fee10d99640 1 -- 172.21.15.26:0/1688399074 --> [v2:172.21.15.26:6800/677148151,v1:172.21.15.26:6801/677148151] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fee0003bbf0 con 0x7fee00039020 2024-09-11T23:09:35.959 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:31.237+0000 7fee10d99640 1 -- 172.21.15.26:0/1688399074 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fee0406d660 con 0x7fee14108260 2024-09-11T23:09:35.959 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:31.237+0000 7fee0bfff640 1 -- 172.21.15.26:0/1688399074 >> [v2:172.21.15.26:6800/677148151,v1:172.21.15.26:6801/677148151] conn(0x7fee00039020 msgr2=0x7fee0003b4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.26:6800/677148151 2024-09-11T23:09:35.959 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:31.237+0000 7fee0bfff640 1 --2- 172.21.15.26:0/1688399074 >> [v2:172.21.15.26:6800/677148151,v1:172.21.15.26:6801/677148151] conn(0x7fee00039020 0x7fee0003b4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-11T23:09:35.959 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:31.438+0000 7fee0bfff640 1 -- 172.21.15.26:0/1688399074 >> [v2:172.21.15.26:6800/677148151,v1:172.21.15.26:6801/677148151] conn(0x7fee00039020 msgr2=0x7fee0003b4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.26:6800/677148151 2024-09-11T23:09:35.959 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:31.438+0000 7fee0bfff640 1 --2- 172.21.15.26:0/1688399074 >> [v2:172.21.15.26:6800/677148151,v1:172.21.15.26:6801/677148151] conn(0x7fee00039020 0x7fee0003b4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-11T23:09:35.959 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:31.839+0000 7fee0bfff640 1 -- 172.21.15.26:0/1688399074 >> [v2:172.21.15.26:6800/677148151,v1:172.21.15.26:6801/677148151] conn(0x7fee00039020 msgr2=0x7fee0003b4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.26:6800/677148151 2024-09-11T23:09:35.959 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:31.839+0000 7fee0bfff640 1 --2- 172.21.15.26:0/1688399074 >> [v2:172.21.15.26:6800/677148151,v1:172.21.15.26:6801/677148151] conn(0x7fee00039020 0x7fee0003b4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-11T23:09:35.960 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:32.640+0000 7fee0bfff640 1 -- 172.21.15.26:0/1688399074 >> [v2:172.21.15.26:6800/677148151,v1:172.21.15.26:6801/677148151] conn(0x7fee00039020 msgr2=0x7fee0003b4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.26:6800/677148151 2024-09-11T23:09:35.960 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:32.640+0000 7fee0bfff640 1 --2- 172.21.15.26:0/1688399074 >> [v2:172.21.15.26:6800/677148151,v1:172.21.15.26:6801/677148151] conn(0x7fee00039020 0x7fee0003b4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-11T23:09:35.960 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:33.984+0000 7fee10d99640 1 -- 172.21.15.26:0/1688399074 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mgrmap(e 10) ==== 46067+0+0 (secure 0 0 0) 0x7fee0406c110 con 0x7fee14108260 2024-09-11T23:09:35.960 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:33.984+0000 7fee10d99640 1 -- 172.21.15.26:0/1688399074 >> [v2:172.21.15.26:6800/677148151,v1:172.21.15.26:6801/677148151] conn(0x7fee00039020 msgr2=0x7fee0003b4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-11T23:09:35.960 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:33.984+0000 7fee10d99640 1 --2- 172.21.15.26:0/1688399074 >> [v2:172.21.15.26:6800/677148151,v1:172.21.15.26:6801/677148151] conn(0x7fee00039020 0x7fee0003b4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:09:35.960 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:34.985+0000 7fee10d99640 1 -- 172.21.15.26:0/1688399074 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mgrmap(e 11) ==== 46194+0+0 (secure 0 0 0) 0x7fee0403b8f0 con 0x7fee14108260 2024-09-11T23:09:35.960 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:34.985+0000 7fee10d99640 1 --2- 172.21.15.26:0/1688399074 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fee0003cae0 0x7fee0003eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:09:35.960 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:34.985+0000 7fee10d99640 1 -- 172.21.15.26:0/1688399074 --> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fee0403bdd0 con 0x7fee0003cae0 2024-09-11T23:09:35.960 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:34.986+0000 7fee0bfff640 1 --2- 172.21.15.26:0/1688399074 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fee0003cae0 0x7fee0003eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:09:35.960 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:34.986+0000 7fee0bfff640 1 --2- 172.21.15.26:0/1688399074 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fee0003cae0 0x7fee0003eed0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fedfc003980 tx=0x7fedfc0092b0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:09:35.960 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:34.987+0000 7fee10d99640 1 -- 172.21.15.26:0/1688399074 <== mgr.14148 v2:172.21.15.26:6800/110833859 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7fee0403bdd0 con 0x7fee0003cae0 2024-09-11T23:09:35.960 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:34.994+0000 7fee190e6640 1 -- 172.21.15.26:0/1688399074 --> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7fee141086c0 con 0x7fee0003cae0 2024-09-11T23:09:35.960 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:34.994+0000 7fee10d99640 1 -- 172.21.15.26:0/1688399074 <== mgr.14148 v2:172.21.15.26:6800/110833859 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7fee141086c0 con 0x7fee0003cae0 2024-09-11T23:09:35.961 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:34.995+0000 7fee190e6640 1 -- 172.21.15.26:0/1688399074 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fee0003cae0 msgr2=0x7fee0003eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:09:35.961 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:34.995+0000 7fee190e6640 1 --2- 172.21.15.26:0/1688399074 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fee0003cae0 0x7fee0003eed0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fedfc003980 tx=0x7fedfc0092b0 comp rx=0 tx=0).stop 2024-09-11T23:09:35.961 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:34.995+0000 7fee190e6640 1 -- 172.21.15.26:0/1688399074 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee14108260 msgr2=0x7fee1419b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:09:35.961 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:34.995+0000 7fee190e6640 1 --2- 172.21.15.26:0/1688399074 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee14108260 0x7fee1419b7c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fee04030380 tx=0x7fee04032670 comp rx=0 tx=0).stop 2024-09-11T23:09:35.961 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:34.995+0000 7fee190e6640 1 -- 172.21.15.26:0/1688399074 shutdown_connections 2024-09-11T23:09:35.961 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:34.995+0000 7fee190e6640 1 --2- 172.21.15.26:0/1688399074 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fee0003cae0 0x7fee0003eed0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:09:35.961 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:34.995+0000 7fee190e6640 1 --2- 172.21.15.26:0/1688399074 >> [v2:172.21.15.26:6800/677148151,v1:172.21.15.26:6801/677148151] conn(0x7fee00039020 0x7fee0003b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:09:35.961 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:34.995+0000 7fee190e6640 1 --2- 172.21.15.26:0/1688399074 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee14108260 0x7fee1419b7c0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:09:35.961 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:34.995+0000 7fee190e6640 1 -- 172.21.15.26:0/1688399074 >> 172.21.15.26:0/1688399074 conn(0x7fee14101e80 msgr2=0x7fee14102ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:09:35.961 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:34.995+0000 7fee190e6640 1 -- 172.21.15.26:0/1688399074 shutdown_connections 2024-09-11T23:09:35.961 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:34.995+0000 7fee190e6640 1 -- 172.21.15.26:0/1688399074 wait complete. 2024-09-11T23:09:35.961 INFO:teuthology.orchestra.run.smithi026.stdout:mgr epoch 9 is available 2024-09-11T23:09:35.962 INFO:teuthology.orchestra.run.smithi026.stdout:Generating a dashboard self-signed certificate... 2024-09-11T23:09:36.214 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:09:35 smithi026 ceph-mon[31172]: mgrmap e11: x(active, since 1.006s) 2024-09-11T23:09:36.214 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:09:35 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:09:36.214 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:09:35 smithi026 ceph-mon[31172]: [11/Sep/2024:23:09:35] ENGINE Bus STARTING 2024-09-11T23:09:36.215 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:09:35 smithi026 ceph-mon[31172]: [11/Sep/2024:23:09:35] ENGINE Serving on http://172.21.15.26:8765 2024-09-11T23:09:36.215 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:09:35 smithi026 ceph-mon[31172]: [11/Sep/2024:23:09:35] ENGINE Serving on https://172.21.15.26:7150 2024-09-11T23:09:36.215 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:09:35 smithi026 ceph-mon[31172]: [11/Sep/2024:23:09:35] ENGINE Bus STARTED 2024-09-11T23:09:36.215 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:09:35 smithi026 ceph-mon[31172]: [11/Sep/2024:23:09:35] ENGINE Error in HTTPServer.serve 2024-09-11T23:09:36.215 INFO:journalctl@ceph.mon.a.smithi026.stdout: Traceback (most recent call last): 2024-09-11T23:09:36.215 INFO:journalctl@ceph.mon.a.smithi026.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-11T23:09:36.215 INFO:journalctl@ceph.mon.a.smithi026.stdout: self._connections.run(self.expiration_interval) 2024-09-11T23:09:36.215 INFO:journalctl@ceph.mon.a.smithi026.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-11T23:09:36.215 INFO:journalctl@ceph.mon.a.smithi026.stdout: self._run(expiration_interval) 2024-09-11T23:09:36.215 INFO:journalctl@ceph.mon.a.smithi026.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-11T23:09:36.215 INFO:journalctl@ceph.mon.a.smithi026.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-11T23:09:36.215 INFO:journalctl@ceph.mon.a.smithi026.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-11T23:09:36.215 INFO:journalctl@ceph.mon.a.smithi026.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-11T23:09:36.215 INFO:journalctl@ceph.mon.a.smithi026.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-11T23:09:36.216 INFO:journalctl@ceph.mon.a.smithi026.stdout: s = self.context.wrap_socket( 2024-09-11T23:09:36.216 INFO:journalctl@ceph.mon.a.smithi026.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-11T23:09:36.216 INFO:journalctl@ceph.mon.a.smithi026.stdout: return self.sslsocket_class._create( 2024-09-11T23:09:36.216 INFO:journalctl@ceph.mon.a.smithi026.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-11T23:09:36.216 INFO:journalctl@ceph.mon.a.smithi026.stdout: self.do_handshake() 2024-09-11T23:09:36.216 INFO:journalctl@ceph.mon.a.smithi026.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-11T23:09:36.216 INFO:journalctl@ceph.mon.a.smithi026.stdout: self._sslobj.do_handshake() 2024-09-11T23:09:36.216 INFO:journalctl@ceph.mon.a.smithi026.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-11T23:09:37.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:09:36 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:09:37.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:09:36 smithi026 ceph-mon[31172]: mgrmap e12: x(active, since 2s) 2024-09-11T23:09:37.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:09:36 smithi026 ceph-mon[31172]: from='client.14160 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:09:37.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:09:36 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:09:37.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:09:36 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:09:37.770 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-09-11T23:09:37.770 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:36.396+0000 7f528dc1b640 1 Processor -- start 2024-09-11T23:09:37.770 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:36.396+0000 7f528dc1b640 1 -- start start 2024-09-11T23:09:37.771 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:36.396+0000 7f528dc1b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52881066e0 0x7f5288106ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:09:37.771 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:36.396+0000 7f528dc1b640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f52881070b0 con 0x7f52881066e0 2024-09-11T23:09:37.771 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:36.396+0000 7f52877fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52881066e0 0x7f5288106ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:09:37.771 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:36.396+0000 7f52877fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52881066e0 0x7f5288106ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55474/0 (socket says 172.21.15.26:55474) 2024-09-11T23:09:37.771 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:36.396+0000 7f52877fe640 1 -- 172.21.15.26:0/31585556 learned_addr learned my addr 172.21.15.26:0/31585556 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:09:37.771 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:36.396+0000 7f52877fe640 1 -- 172.21.15.26:0/31585556 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52881078e0 con 0x7f52881066e0 2024-09-11T23:09:37.772 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:36.397+0000 7f52877fe640 1 --2- 172.21.15.26:0/31585556 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52881066e0 0x7f5288106ae0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f5278009820 tx=0x7f527802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a347a7f7ddace8b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:09:37.772 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:36.397+0000 7f52867fc640 1 -- 172.21.15.26:0/31585556 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5278037070 con 0x7f52881066e0 2024-09-11T23:09:37.772 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:36.397+0000 7f52867fc640 1 -- 172.21.15.26:0/31585556 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5278032920 con 0x7f52881066e0 2024-09-11T23:09:37.772 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:36.398+0000 7f52867fc640 1 -- 172.21.15.26:0/31585556 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5278030810 con 0x7f52881066e0 2024-09-11T23:09:37.772 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:36.398+0000 7f528dc1b640 1 -- 172.21.15.26:0/31585556 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52881066e0 msgr2=0x7f5288106ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:09:37.773 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:36.398+0000 7f528dc1b640 1 --2- 172.21.15.26:0/31585556 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52881066e0 0x7f5288106ae0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f5278009820 tx=0x7f527802a270 comp rx=0 tx=0).stop 2024-09-11T23:09:37.773 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:36.398+0000 7f528dc1b640 1 -- 172.21.15.26:0/31585556 shutdown_connections 2024-09-11T23:09:37.773 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:36.398+0000 7f528dc1b640 1 --2- 172.21.15.26:0/31585556 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52881066e0 0x7f5288106ae0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:09:37.773 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:36.398+0000 7f528dc1b640 1 -- 172.21.15.26:0/31585556 >> 172.21.15.26:0/31585556 conn(0x7f5288101ed0 msgr2=0x7f52881042f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:09:37.773 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:36.398+0000 7f528dc1b640 1 -- 172.21.15.26:0/31585556 shutdown_connections 2024-09-11T23:09:37.773 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:36.398+0000 7f528dc1b640 1 -- 172.21.15.26:0/31585556 wait complete. 2024-09-11T23:09:37.774 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:36.399+0000 7f528dc1b640 1 Processor -- start 2024-09-11T23:09:37.774 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:36.399+0000 7f528dc1b640 1 -- start start 2024-09-11T23:09:37.774 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:36.399+0000 7f528dc1b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52881066e0 0x7f52881918a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:09:37.774 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:36.399+0000 7f528dc1b640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f528810ac90 con 0x7f52881066e0 2024-09-11T23:09:37.774 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:36.399+0000 7f52877fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52881066e0 0x7f52881918a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:09:37.775 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:36.399+0000 7f52877fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52881066e0 0x7f52881918a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55490/0 (socket says 172.21.15.26:55490) 2024-09-11T23:09:37.775 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:36.399+0000 7f52877fe640 1 -- 172.21.15.26:0/3140660354 learned_addr learned my addr 172.21.15.26:0/3140660354 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:09:37.775 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:36.399+0000 7f52877fe640 1 -- 172.21.15.26:0/3140660354 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5288191de0 con 0x7f52881066e0 2024-09-11T23:09:37.775 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:36.400+0000 7f52877fe640 1 --2- 172.21.15.26:0/3140660354 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52881066e0 0x7f52881918a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f52780097f0 tx=0x7f52780327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:09:37.775 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:36.400+0000 7f5284ff9640 1 -- 172.21.15.26:0/3140660354 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5278037040 con 0x7f52881066e0 2024-09-11T23:09:37.775 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:36.400+0000 7f5284ff9640 1 -- 172.21.15.26:0/3140660354 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5278030cc0 con 0x7f52881066e0 2024-09-11T23:09:37.776 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:36.400+0000 7f528dc1b640 1 -- 172.21.15.26:0/3140660354 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f528818ffd0 con 0x7f52881066e0 2024-09-11T23:09:37.776 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:36.400+0000 7f5284ff9640 1 -- 172.21.15.26:0/3140660354 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f527803a640 con 0x7f52881066e0 2024-09-11T23:09:37.776 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:36.400+0000 7f528dc1b640 1 -- 172.21.15.26:0/3140660354 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5288190510 con 0x7f52881066e0 2024-09-11T23:09:37.776 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:36.401+0000 7f528dc1b640 1 -- 172.21.15.26:0/3140660354 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5288106b60 con 0x7f52881066e0 2024-09-11T23:09:37.776 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:36.401+0000 7f5284ff9640 1 -- 172.21.15.26:0/3140660354 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7f527803a940 con 0x7f52881066e0 2024-09-11T23:09:37.777 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:36.401+0000 7f5284ff9640 1 --2- 172.21.15.26:0/3140660354 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f5270038fc0 0x7f527003b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:09:37.777 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:36.401+0000 7f5284ff9640 1 -- 172.21.15.26:0/3140660354 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f527806c780 con 0x7f52881066e0 2024-09-11T23:09:37.777 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:36.401+0000 7f5286ffd640 1 --2- 172.21.15.26:0/3140660354 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f5270038fc0 0x7f527003b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:09:37.777 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:36.402+0000 7f5286ffd640 1 --2- 172.21.15.26:0/3140660354 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f5270038fc0 0x7f527003b480 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f527c00ad30 tx=0x7f527c0093f0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:09:37.777 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:36.406+0000 7f5284ff9640 1 -- 172.21.15.26:0/3140660354 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f527803ad80 con 0x7f52881066e0 2024-09-11T23:09:37.778 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:36.606+0000 7f528dc1b640 1 -- 172.21.15.26:0/3140660354 --> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7f5288195650 con 0x7f5270038fc0 2024-09-11T23:09:37.778 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:36.694+0000 7f5284ff9640 1 -- 172.21.15.26:0/3140660354 <== mgr.14148 v2:172.21.15.26:6800/110833859 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7f5288195650 con 0x7f5270038fc0 2024-09-11T23:09:37.778 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:36.697+0000 7f528dc1b640 1 -- 172.21.15.26:0/3140660354 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f5270038fc0 msgr2=0x7f527003b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:09:37.778 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:36.697+0000 7f528dc1b640 1 --2- 172.21.15.26:0/3140660354 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f5270038fc0 0x7f527003b480 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f527c00ad30 tx=0x7f527c0093f0 comp rx=0 tx=0).stop 2024-09-11T23:09:37.778 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:36.697+0000 7f528dc1b640 1 -- 172.21.15.26:0/3140660354 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52881066e0 msgr2=0x7f52881918a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:09:37.778 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:36.697+0000 7f528dc1b640 1 --2- 172.21.15.26:0/3140660354 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52881066e0 0x7f52881918a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f52780097f0 tx=0x7f52780327b0 comp rx=0 tx=0).stop 2024-09-11T23:09:37.779 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:36.698+0000 7f528dc1b640 1 -- 172.21.15.26:0/3140660354 shutdown_connections 2024-09-11T23:09:37.779 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:36.698+0000 7f528dc1b640 1 --2- 172.21.15.26:0/3140660354 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f5270038fc0 0x7f527003b480 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:09:37.779 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:36.698+0000 7f528dc1b640 1 --2- 172.21.15.26:0/3140660354 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52881066e0 0x7f52881918a0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:09:37.779 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:36.698+0000 7f528dc1b640 1 -- 172.21.15.26:0/3140660354 >> 172.21.15.26:0/3140660354 conn(0x7f5288101ed0 msgr2=0x7f5288102e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:09:37.779 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:36.698+0000 7f528dc1b640 1 -- 172.21.15.26:0/3140660354 shutdown_connections 2024-09-11T23:09:37.779 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:36.698+0000 7f528dc1b640 1 -- 172.21.15.26:0/3140660354 wait complete. 2024-09-11T23:09:37.780 INFO:teuthology.orchestra.run.smithi026.stdout:Creating initial admin user... 2024-09-11T23:09:39.223 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$JN/a6/IlGF/JlyAbOQc21ugdpLF7i7GVri.fMa39zjJsrmciGq1je", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1726096178, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-09-11T23:09:39.223 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:38.202+0000 7f35d8f34640 1 Processor -- start 2024-09-11T23:09:39.223 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:38.202+0000 7f35d8f34640 1 -- start start 2024-09-11T23:09:39.223 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:38.202+0000 7f35d8f34640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f35d407de00 0x7f35d407c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:09:39.223 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:38.202+0000 7f35d8f34640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f35d407c830 con 0x7f35d407de00 2024-09-11T23:09:39.223 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:38.203+0000 7f35d259c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f35d407de00 0x7f35d407c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:09:39.223 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:38.203+0000 7f35d259c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f35d407de00 0x7f35d407c260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55500/0 (socket says 172.21.15.26:55500) 2024-09-11T23:09:39.223 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:38.203+0000 7f35d259c640 1 -- 172.21.15.26:0/216092852 learned_addr learned my addr 172.21.15.26:0/216092852 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:09:39.223 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:38.203+0000 7f35d259c640 1 -- 172.21.15.26:0/216092852 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35d407c9b0 con 0x7f35d407de00 2024-09-11T23:09:39.224 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:38.204+0000 7f35d259c640 1 --2- 172.21.15.26:0/216092852 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f35d407de00 0x7f35d407c260 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f35c4009820 tx=0x7f35c402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bbf596dd0143672 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:09:39.224 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:38.204+0000 7f35d159a640 1 -- 172.21.15.26:0/216092852 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f35c4037070 con 0x7f35d407de00 2024-09-11T23:09:39.224 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:38.204+0000 7f35d159a640 1 -- 172.21.15.26:0/216092852 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35c4032920 con 0x7f35d407de00 2024-09-11T23:09:39.224 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:38.204+0000 7f35d159a640 1 -- 172.21.15.26:0/216092852 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f35c4030810 con 0x7f35d407de00 2024-09-11T23:09:39.224 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:38.204+0000 7f35d8f34640 1 -- 172.21.15.26:0/216092852 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f35d407de00 msgr2=0x7f35d407c260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:09:39.224 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:38.204+0000 7f35d8f34640 1 --2- 172.21.15.26:0/216092852 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f35d407de00 0x7f35d407c260 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f35c4009820 tx=0x7f35c402a270 comp rx=0 tx=0).stop 2024-09-11T23:09:39.224 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:38.205+0000 7f35d8f34640 1 -- 172.21.15.26:0/216092852 shutdown_connections 2024-09-11T23:09:39.224 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:38.205+0000 7f35d8f34640 1 --2- 172.21.15.26:0/216092852 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f35d407de00 0x7f35d407c260 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:09:39.224 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:38.205+0000 7f35d8f34640 1 -- 172.21.15.26:0/216092852 >> 172.21.15.26:0/216092852 conn(0x7f35d4102f60 msgr2=0x7f35d41053a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:09:39.224 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:38.205+0000 7f35d8f34640 1 -- 172.21.15.26:0/216092852 shutdown_connections 2024-09-11T23:09:39.224 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:38.205+0000 7f35d8f34640 1 -- 172.21.15.26:0/216092852 wait complete. 2024-09-11T23:09:39.224 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:38.205+0000 7f35d8f34640 1 Processor -- start 2024-09-11T23:09:39.225 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:38.206+0000 7f35d8f34640 1 -- start start 2024-09-11T23:09:39.225 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:38.206+0000 7f35d8f34640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f35d407de00 0x7f35d419d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:09:39.225 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:38.206+0000 7f35d8f34640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f35d410ddf0 con 0x7f35d407de00 2024-09-11T23:09:39.225 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:38.206+0000 7f35d259c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f35d407de00 0x7f35d419d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:09:39.225 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:38.206+0000 7f35d259c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f35d407de00 0x7f35d419d980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55516/0 (socket says 172.21.15.26:55516) 2024-09-11T23:09:39.225 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:38.206+0000 7f35d259c640 1 -- 172.21.15.26:0/556317958 learned_addr learned my addr 172.21.15.26:0/556317958 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:09:39.225 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:38.206+0000 7f35d259c640 1 -- 172.21.15.26:0/556317958 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35d419dec0 con 0x7f35d407de00 2024-09-11T23:09:39.225 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:38.206+0000 7f35d259c640 1 --2- 172.21.15.26:0/556317958 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f35d407de00 0x7f35d419d980 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f35c4030380 tx=0x7f35c4032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:09:39.225 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:38.206+0000 7f35c37fe640 1 -- 172.21.15.26:0/556317958 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f35c4037040 con 0x7f35d407de00 2024-09-11T23:09:39.225 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:38.207+0000 7f35c37fe640 1 -- 172.21.15.26:0/556317958 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35c4040070 con 0x7f35d407de00 2024-09-11T23:09:39.225 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:38.207+0000 7f35d8f34640 1 -- 172.21.15.26:0/556317958 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35d419e150 con 0x7f35d407de00 2024-09-11T23:09:39.225 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:38.207+0000 7f35c37fe640 1 -- 172.21.15.26:0/556317958 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f35c403a7b0 con 0x7f35d407de00 2024-09-11T23:09:39.225 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:38.207+0000 7f35d8f34640 1 -- 172.21.15.26:0/556317958 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35d41a0e40 con 0x7f35d407de00 2024-09-11T23:09:39.225 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:38.208+0000 7f35c37fe640 1 -- 172.21.15.26:0/556317958 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7f35c403aa30 con 0x7f35d407de00 2024-09-11T23:09:39.226 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:38.208+0000 7f35d8f34640 1 -- 172.21.15.26:0/556317958 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35d407c2e0 con 0x7f35d407de00 2024-09-11T23:09:39.226 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:38.208+0000 7f35c37fe640 1 --2- 172.21.15.26:0/556317958 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f35bc038fc0 0x7f35bc03b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:09:39.226 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:38.208+0000 7f35c37fe640 1 -- 172.21.15.26:0/556317958 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f35c406c740 con 0x7f35d407de00 2024-09-11T23:09:39.226 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:38.208+0000 7f35d1d9b640 1 --2- 172.21.15.26:0/556317958 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f35bc038fc0 0x7f35bc03b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:09:39.226 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:38.208+0000 7f35d1d9b640 1 --2- 172.21.15.26:0/556317958 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f35bc038fc0 0x7f35bc03b480 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f35c80098c0 tx=0x7f35c8006db0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:09:39.226 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:38.213+0000 7f35c37fe640 1 -- 172.21.15.26:0/556317958 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f35c4041270 con 0x7f35d407de00 2024-09-11T23:09:39.226 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:38.404+0000 7f35d8f34640 1 -- 172.21.15.26:0/556317958 --> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7f35d41a1010 con 0x7f35bc038fc0 2024-09-11T23:09:39.226 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:38.653+0000 7f35c37fe640 1 -- 172.21.15.26:0/556317958 <== mgr.14148 v2:172.21.15.26:6800/110833859 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7f35d41a1010 con 0x7f35bc038fc0 2024-09-11T23:09:39.226 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:38.655+0000 7f35d8f34640 1 -- 172.21.15.26:0/556317958 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f35bc038fc0 msgr2=0x7f35bc03b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:09:39.226 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:38.655+0000 7f35d8f34640 1 --2- 172.21.15.26:0/556317958 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f35bc038fc0 0x7f35bc03b480 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f35c80098c0 tx=0x7f35c8006db0 comp rx=0 tx=0).stop 2024-09-11T23:09:39.226 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:38.655+0000 7f35d8f34640 1 -- 172.21.15.26:0/556317958 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f35d407de00 msgr2=0x7f35d419d980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:09:39.226 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:38.655+0000 7f35d8f34640 1 --2- 172.21.15.26:0/556317958 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f35d407de00 0x7f35d419d980 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f35c4030380 tx=0x7f35c4032670 comp rx=0 tx=0).stop 2024-09-11T23:09:39.227 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:38.655+0000 7f35d8f34640 1 -- 172.21.15.26:0/556317958 shutdown_connections 2024-09-11T23:09:39.227 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:38.655+0000 7f35d8f34640 1 --2- 172.21.15.26:0/556317958 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f35bc038fc0 0x7f35bc03b480 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:09:39.227 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:38.655+0000 7f35d8f34640 1 --2- 172.21.15.26:0/556317958 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f35d407de00 0x7f35d419d980 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:09:39.227 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:38.655+0000 7f35d8f34640 1 -- 172.21.15.26:0/556317958 >> 172.21.15.26:0/556317958 conn(0x7f35d4102f60 msgr2=0x7f35d4103b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:09:39.227 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:38.655+0000 7f35d8f34640 1 -- 172.21.15.26:0/556317958 shutdown_connections 2024-09-11T23:09:39.227 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:38.655+0000 7f35d8f34640 1 -- 172.21.15.26:0/556317958 wait complete. 2024-09-11T23:09:39.227 INFO:teuthology.orchestra.run.smithi026.stdout:Fetching dashboard port number... 2024-09-11T23:09:40.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:09:39 smithi026 ceph-mon[31172]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:09:40.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:09:39 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:09:40.514 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout 8443 2024-09-11T23:09:40.514 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:39.657+0000 7f97655d2640 1 Processor -- start 2024-09-11T23:09:40.515 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:39.657+0000 7f97655d2640 1 -- start start 2024-09-11T23:09:40.515 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:39.658+0000 7f97655d2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9760108320 0x7f976010a710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:09:40.515 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:39.658+0000 7f97655d2640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f976007bd50 con 0x7f9760108320 2024-09-11T23:09:40.515 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:39.658+0000 7f975effd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9760108320 0x7f976010a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:09:40.515 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:39.658+0000 7f975effd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9760108320 0x7f976010a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55528/0 (socket says 172.21.15.26:55528) 2024-09-11T23:09:40.515 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:39.658+0000 7f975effd640 1 -- 172.21.15.26:0/1383998198 learned_addr learned my addr 172.21.15.26:0/1383998198 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:09:40.515 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:39.658+0000 7f975effd640 1 -- 172.21.15.26:0/1383998198 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f976010ac50 con 0x7f9760108320 2024-09-11T23:09:40.515 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:39.659+0000 7f975effd640 1 --2- 172.21.15.26:0/1383998198 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9760108320 0x7f976010a710 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f9750009820 tx=0x7f975002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61fda3d01e2676e2 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:09:40.515 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:39.659+0000 7f975e7fc640 1 -- 172.21.15.26:0/1383998198 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9750037070 con 0x7f9760108320 2024-09-11T23:09:40.515 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:39.659+0000 7f975e7fc640 1 -- 172.21.15.26:0/1383998198 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9750032920 con 0x7f9760108320 2024-09-11T23:09:40.515 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:39.659+0000 7f975e7fc640 1 -- 172.21.15.26:0/1383998198 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9750030810 con 0x7f9760108320 2024-09-11T23:09:40.516 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:39.660+0000 7f97655d2640 1 -- 172.21.15.26:0/1383998198 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9760108320 msgr2=0x7f976010a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:09:40.516 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:39.660+0000 7f97655d2640 1 --2- 172.21.15.26:0/1383998198 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9760108320 0x7f976010a710 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f9750009820 tx=0x7f975002a270 comp rx=0 tx=0).stop 2024-09-11T23:09:40.516 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:39.660+0000 7f97655d2640 1 -- 172.21.15.26:0/1383998198 shutdown_connections 2024-09-11T23:09:40.516 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:39.660+0000 7f97655d2640 1 --2- 172.21.15.26:0/1383998198 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9760108320 0x7f976010a710 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:09:40.516 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:39.660+0000 7f97655d2640 1 -- 172.21.15.26:0/1383998198 >> 172.21.15.26:0/1383998198 conn(0x7f9760101e60 msgr2=0x7f97601042a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:09:40.516 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:39.660+0000 7f97655d2640 1 -- 172.21.15.26:0/1383998198 shutdown_connections 2024-09-11T23:09:40.516 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:39.660+0000 7f97655d2640 1 -- 172.21.15.26:0/1383998198 wait complete. 2024-09-11T23:09:40.516 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:39.660+0000 7f97655d2640 1 Processor -- start 2024-09-11T23:09:40.516 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:39.660+0000 7f97655d2640 1 -- start start 2024-09-11T23:09:40.516 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:39.660+0000 7f97655d2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9760108320 0x7f976019b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:09:40.517 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:39.661+0000 7f97655d2640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f976010bba0 con 0x7f9760108320 2024-09-11T23:09:40.517 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:39.661+0000 7f975effd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9760108320 0x7f976019b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:09:40.517 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:39.661+0000 7f975effd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9760108320 0x7f976019b790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55530/0 (socket says 172.21.15.26:55530) 2024-09-11T23:09:40.517 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:39.661+0000 7f975effd640 1 -- 172.21.15.26:0/108060147 learned_addr learned my addr 172.21.15.26:0/108060147 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:09:40.517 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:39.661+0000 7f975effd640 1 -- 172.21.15.26:0/108060147 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f976019bcd0 con 0x7f9760108320 2024-09-11T23:09:40.517 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:39.661+0000 7f975effd640 1 --2- 172.21.15.26:0/108060147 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9760108320 0x7f976019b790 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f97500097f0 tx=0x7f97500327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:09:40.517 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:39.661+0000 7f975cff9640 1 -- 172.21.15.26:0/108060147 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9750037040 con 0x7f9760108320 2024-09-11T23:09:40.518 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:39.661+0000 7f975cff9640 1 -- 172.21.15.26:0/108060147 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9750030cc0 con 0x7f9760108320 2024-09-11T23:09:40.518 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:39.661+0000 7f97655d2640 1 -- 172.21.15.26:0/108060147 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f976019bf60 con 0x7f9760108320 2024-09-11T23:09:40.518 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:39.661+0000 7f975cff9640 1 -- 172.21.15.26:0/108060147 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f975003a6b0 con 0x7f9760108320 2024-09-11T23:09:40.518 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:39.661+0000 7f97655d2640 1 -- 172.21.15.26:0/108060147 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f976019ec50 con 0x7f9760108320 2024-09-11T23:09:40.518 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:39.662+0000 7f975cff9640 1 -- 172.21.15.26:0/108060147 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7f975003a850 con 0x7f9760108320 2024-09-11T23:09:40.518 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:39.662+0000 7f975cff9640 1 --2- 172.21.15.26:0/108060147 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f974c039020 0x7f974c03b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:09:40.518 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:39.662+0000 7f975cff9640 1 -- 172.21.15.26:0/108060147 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f975006c5d0 con 0x7f9760108320 2024-09-11T23:09:40.518 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:39.662+0000 7f9757fff640 1 --2- 172.21.15.26:0/108060147 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f974c039020 0x7f974c03b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:09:40.518 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:39.662+0000 7f97655d2640 1 -- 172.21.15.26:0/108060147 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9760108780 con 0x7f9760108320 2024-09-11T23:09:40.519 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:39.663+0000 7f9757fff640 1 --2- 172.21.15.26:0/108060147 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f974c039020 0x7f974c03b4e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f97480098c0 tx=0x7f9748006db0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:09:40.519 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:39.667+0000 7f975cff9640 1 -- 172.21.15.26:0/108060147 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9750044920 con 0x7f9760108320 2024-09-11T23:09:40.519 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:39.841+0000 7f97655d2640 1 -- 172.21.15.26:0/108060147 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7f976019f070 con 0x7f9760108320 2024-09-11T23:09:40.519 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:39.842+0000 7f975cff9640 1 -- 172.21.15.26:0/108060147 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7f9750044400 con 0x7f9760108320 2024-09-11T23:09:40.519 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:39.844+0000 7f97655d2640 1 -- 172.21.15.26:0/108060147 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f974c039020 msgr2=0x7f974c03b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:09:40.519 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:39.844+0000 7f97655d2640 1 --2- 172.21.15.26:0/108060147 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f974c039020 0x7f974c03b4e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f97480098c0 tx=0x7f9748006db0 comp rx=0 tx=0).stop 2024-09-11T23:09:40.519 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:39.844+0000 7f97655d2640 1 -- 172.21.15.26:0/108060147 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9760108320 msgr2=0x7f976019b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:09:40.519 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:39.844+0000 7f97655d2640 1 --2- 172.21.15.26:0/108060147 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9760108320 0x7f976019b790 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f97500097f0 tx=0x7f97500327b0 comp rx=0 tx=0).stop 2024-09-11T23:09:40.519 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:39.844+0000 7f97655d2640 1 -- 172.21.15.26:0/108060147 shutdown_connections 2024-09-11T23:09:40.520 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:39.844+0000 7f97655d2640 1 --2- 172.21.15.26:0/108060147 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f974c039020 0x7f974c03b4e0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:09:40.520 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:39.844+0000 7f97655d2640 1 --2- 172.21.15.26:0/108060147 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9760108320 0x7f976019b790 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:09:40.520 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:39.844+0000 7f97655d2640 1 -- 172.21.15.26:0/108060147 >> 172.21.15.26:0/108060147 conn(0x7f9760101e60 msgr2=0x7f9760102a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:09:40.520 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:39.844+0000 7f97655d2640 1 -- 172.21.15.26:0/108060147 shutdown_connections 2024-09-11T23:09:40.520 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:39.844+0000 7f97655d2640 1 -- 172.21.15.26:0/108060147 wait complete. 2024-09-11T23:09:40.525 INFO:teuthology.orchestra.run.smithi026.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-11T23:09:40.525 INFO:teuthology.orchestra.run.smithi026.stdout:systemctl: stdout disabled 2024-09-11T23:09:40.534 INFO:teuthology.orchestra.run.smithi026.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-11T23:09:40.534 INFO:teuthology.orchestra.run.smithi026.stdout:systemctl: stdout inactive 2024-09-11T23:09:40.534 INFO:teuthology.orchestra.run.smithi026.stdout:firewalld.service is not enabled 2024-09-11T23:09:40.534 INFO:teuthology.orchestra.run.smithi026.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-09-11T23:09:40.537 INFO:teuthology.orchestra.run.smithi026.stdout:Ceph Dashboard is now available at: 2024-09-11T23:09:40.537 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-11T23:09:40.537 INFO:teuthology.orchestra.run.smithi026.stdout: URL: https://smithi026.front.sepia.ceph.com:8443/ 2024-09-11T23:09:40.537 INFO:teuthology.orchestra.run.smithi026.stdout: User: admin 2024-09-11T23:09:40.537 INFO:teuthology.orchestra.run.smithi026.stdout: Password: mmecvmqgtn 2024-09-11T23:09:40.537 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-11T23:09:40.537 INFO:teuthology.orchestra.run.smithi026.stdout:Saving cluster configuration to /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/config directory 2024-09-11T23:09:40.758 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:09:40 smithi026 ceph-mon[31172]: from='client.? 172.21.15.26:0/108060147' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-09-11T23:09:41.804 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:09:41 smithi026 ceph-mon[31172]: mgrmap e13: x(active, since 6s) 2024-09-11T23:09:41.804 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:09:41 smithi026 ceph-mon[31172]: from='client.? 172.21.15.26:0/2762741351' entity='client.admin' 2024-09-11T23:09:41.811 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:40.940+0000 7f0bc58f9640 1 Processor -- start 2024-09-11T23:09:41.812 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:40.941+0000 7f0bc58f9640 1 -- start start 2024-09-11T23:09:41.812 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:40.941+0000 7f0bc58f9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0bc007de00 0x7f0bc007c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:09:41.812 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:40.941+0000 7f0bc58f9640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f0bc007c830 con 0x7f0bc007de00 2024-09-11T23:09:41.812 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:40.941+0000 7f0bbeffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0bc007de00 0x7f0bc007c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:09:41.812 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:40.941+0000 7f0bbeffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0bc007de00 0x7f0bc007c260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41726/0 (socket says 172.21.15.26:41726) 2024-09-11T23:09:41.812 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:40.941+0000 7f0bbeffd640 1 -- 172.21.15.26:0/1317087598 learned_addr learned my addr 172.21.15.26:0/1317087598 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:09:41.812 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:40.941+0000 7f0bbeffd640 1 -- 172.21.15.26:0/1317087598 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bc007c9b0 con 0x7f0bc007de00 2024-09-11T23:09:41.812 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:40.942+0000 7f0bbeffd640 1 --2- 172.21.15.26:0/1317087598 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0bc007de00 0x7f0bc007c260 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f0bb0009820 tx=0x7f0bb002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78129adcbaa19e4b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:09:41.812 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:40.942+0000 7f0bbe7fc640 1 -- 172.21.15.26:0/1317087598 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0bb0037070 con 0x7f0bc007de00 2024-09-11T23:09:41.812 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:40.942+0000 7f0bbe7fc640 1 -- 172.21.15.26:0/1317087598 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0bb0032920 con 0x7f0bc007de00 2024-09-11T23:09:41.812 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:40.942+0000 7f0bbe7fc640 1 -- 172.21.15.26:0/1317087598 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0bb0030810 con 0x7f0bc007de00 2024-09-11T23:09:41.812 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:40.943+0000 7f0bc58f9640 1 -- 172.21.15.26:0/1317087598 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0bc007de00 msgr2=0x7f0bc007c260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:09:41.813 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:40.943+0000 7f0bc58f9640 1 --2- 172.21.15.26:0/1317087598 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0bc007de00 0x7f0bc007c260 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f0bb0009820 tx=0x7f0bb002a270 comp rx=0 tx=0).stop 2024-09-11T23:09:41.813 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:40.943+0000 7f0bc58f9640 1 -- 172.21.15.26:0/1317087598 shutdown_connections 2024-09-11T23:09:41.813 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:40.943+0000 7f0bc58f9640 1 --2- 172.21.15.26:0/1317087598 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0bc007de00 0x7f0bc007c260 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:09:41.813 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:40.943+0000 7f0bc58f9640 1 -- 172.21.15.26:0/1317087598 >> 172.21.15.26:0/1317087598 conn(0x7f0bc0102f60 msgr2=0x7f0bc01053a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:09:41.813 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:40.943+0000 7f0bc58f9640 1 -- 172.21.15.26:0/1317087598 shutdown_connections 2024-09-11T23:09:41.813 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:40.943+0000 7f0bc58f9640 1 -- 172.21.15.26:0/1317087598 wait complete. 2024-09-11T23:09:41.813 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:40.944+0000 7f0bc58f9640 1 Processor -- start 2024-09-11T23:09:41.813 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:40.944+0000 7f0bc58f9640 1 -- start start 2024-09-11T23:09:41.813 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:40.944+0000 7f0bc58f9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0bc007de00 0x7f0bc019d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:09:41.813 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:40.944+0000 7f0bc58f9640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f0bc010ddf0 con 0x7f0bc007de00 2024-09-11T23:09:41.813 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:40.944+0000 7f0bbeffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0bc007de00 0x7f0bc019d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:09:41.813 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:40.944+0000 7f0bbeffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0bc007de00 0x7f0bc019d980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41732/0 (socket says 172.21.15.26:41732) 2024-09-11T23:09:41.813 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:40.944+0000 7f0bbeffd640 1 -- 172.21.15.26:0/2762741351 learned_addr learned my addr 172.21.15.26:0/2762741351 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:09:41.813 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:40.944+0000 7f0bbeffd640 1 -- 172.21.15.26:0/2762741351 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bc019dec0 con 0x7f0bc007de00 2024-09-11T23:09:41.814 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:40.945+0000 7f0bbeffd640 1 --2- 172.21.15.26:0/2762741351 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0bc007de00 0x7f0bc019d980 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f0bb0030380 tx=0x7f0bb0032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:09:41.814 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:40.945+0000 7f0bc48f7640 1 -- 172.21.15.26:0/2762741351 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0bb0037040 con 0x7f0bc007de00 2024-09-11T23:09:41.814 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:40.945+0000 7f0bc48f7640 1 -- 172.21.15.26:0/2762741351 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0bb0040070 con 0x7f0bc007de00 2024-09-11T23:09:41.814 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:40.945+0000 7f0bc48f7640 1 -- 172.21.15.26:0/2762741351 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0bb003a7b0 con 0x7f0bc007de00 2024-09-11T23:09:41.814 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:40.945+0000 7f0bc58f9640 1 -- 172.21.15.26:0/2762741351 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bc019e150 con 0x7f0bc007de00 2024-09-11T23:09:41.814 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:40.945+0000 7f0bc58f9640 1 -- 172.21.15.26:0/2762741351 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bc01a0e40 con 0x7f0bc007de00 2024-09-11T23:09:41.814 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:40.946+0000 7f0bc48f7640 1 -- 172.21.15.26:0/2762741351 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f0bb003b420 con 0x7f0bc007de00 2024-09-11T23:09:41.814 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:40.946+0000 7f0bc58f9640 1 -- 172.21.15.26:0/2762741351 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0bc007c2e0 con 0x7f0bc007de00 2024-09-11T23:09:41.814 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:40.946+0000 7f0bc48f7640 1 --2- 172.21.15.26:0/2762741351 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f0bac0390c0 0x7f0bac03b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:09:41.814 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:40.946+0000 7f0bc48f7640 1 -- 172.21.15.26:0/2762741351 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f0bb006c850 con 0x7f0bc007de00 2024-09-11T23:09:41.814 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:40.946+0000 7f0bb7fff640 1 --2- 172.21.15.26:0/2762741351 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f0bac0390c0 0x7f0bac03b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:09:41.814 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:40.947+0000 7f0bb7fff640 1 --2- 172.21.15.26:0/2762741351 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f0bac0390c0 0x7f0bac03b580 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f0ba80098c0 tx=0x7f0ba8006db0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:09:41.814 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:40.951+0000 7f0bc48f7640 1 -- 172.21.15.26:0/2762741351 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0bb0042b50 con 0x7f0bc007de00 2024-09-11T23:09:41.815 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:41.217+0000 7f0bc58f9640 1 -- 172.21.15.26:0/2762741351 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7f0bc01a1010 con 0x7f0bc007de00 2024-09-11T23:09:41.815 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:41.219+0000 7f0bc48f7640 1 -- 172.21.15.26:0/2762741351 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) ==== 153+0+0 (secure 0 0 0) 0x7f0bb0043360 con 0x7f0bc007de00 2024-09-11T23:09:41.815 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-09-11T23:09:41.815 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:41.222+0000 7f0bc58f9640 1 -- 172.21.15.26:0/2762741351 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f0bac0390c0 msgr2=0x7f0bac03b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:09:41.815 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:41.222+0000 7f0bc58f9640 1 --2- 172.21.15.26:0/2762741351 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f0bac0390c0 0x7f0bac03b580 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f0ba80098c0 tx=0x7f0ba8006db0 comp rx=0 tx=0).stop 2024-09-11T23:09:41.815 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:41.222+0000 7f0bc58f9640 1 -- 172.21.15.26:0/2762741351 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0bc007de00 msgr2=0x7f0bc019d980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:09:41.815 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:41.222+0000 7f0bc58f9640 1 --2- 172.21.15.26:0/2762741351 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0bc007de00 0x7f0bc019d980 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f0bb0030380 tx=0x7f0bb0032670 comp rx=0 tx=0).stop 2024-09-11T23:09:41.815 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:41.222+0000 7f0bc58f9640 1 -- 172.21.15.26:0/2762741351 shutdown_connections 2024-09-11T23:09:41.815 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:41.222+0000 7f0bc58f9640 1 --2- 172.21.15.26:0/2762741351 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f0bac0390c0 0x7f0bac03b580 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:09:41.815 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:41.222+0000 7f0bc58f9640 1 --2- 172.21.15.26:0/2762741351 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0bc007de00 0x7f0bc019d980 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:09:41.815 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:41.222+0000 7f0bc58f9640 1 -- 172.21.15.26:0/2762741351 >> 172.21.15.26:0/2762741351 conn(0x7f0bc0102f60 msgr2=0x7f0bc0103b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:09:41.815 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:41.222+0000 7f0bc58f9640 1 -- 172.21.15.26:0/2762741351 shutdown_connections 2024-09-11T23:09:41.815 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-09-11T23:09:41.222+0000 7f0bc58f9640 1 -- 172.21.15.26:0/2762741351 wait complete. 2024-09-11T23:09:41.816 INFO:teuthology.orchestra.run.smithi026.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-09-11T23:09:41.816 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-11T23:09:41.816 INFO:teuthology.orchestra.run.smithi026.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-09-11T23:09:41.816 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-11T23:09:41.816 INFO:teuthology.orchestra.run.smithi026.stdout:Or, if you are only running a single cluster on this host: 2024-09-11T23:09:41.816 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-11T23:09:41.816 INFO:teuthology.orchestra.run.smithi026.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-09-11T23:09:41.816 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-11T23:09:41.816 INFO:teuthology.orchestra.run.smithi026.stdout:Please consider enabling telemetry to help improve Ceph: 2024-09-11T23:09:41.816 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-11T23:09:41.816 INFO:teuthology.orchestra.run.smithi026.stdout: ceph telemetry on 2024-09-11T23:09:41.816 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-11T23:09:41.817 INFO:teuthology.orchestra.run.smithi026.stdout:For more information see: 2024-09-11T23:09:41.817 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-11T23:09:41.817 INFO:teuthology.orchestra.run.smithi026.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-09-11T23:09:41.817 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-11T23:09:41.817 INFO:teuthology.orchestra.run.smithi026.stdout:Bootstrap complete. 2024-09-11T23:09:41.855 INFO:tasks.cephadm:Fetching config... 2024-09-11T23:09:41.855 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-09-11T23:09:41.856 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-09-11T23:09:41.873 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-09-11T23:09:41.873 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-09-11T23:09:41.873 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-09-11T23:09:41.929 INFO:tasks.cephadm:Fetching mon keyring... 2024-09-11T23:09:41.929 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-09-11T23:09:41.929 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd if=/var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/keyring of=/dev/stdout 2024-09-11T23:09:41.996 INFO:tasks.cephadm:Fetching pub ssh key... 2024-09-11T23:09:41.997 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-09-11T23:09:41.997 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-09-11T23:09:42.053 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-09-11T23:09:42.053 DEBUG:teuthology.orchestra.run.smithi026:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCFk7Bjp26C4V+TSSoQjcdWbS8+pBY9JCBMm0AmEJqupQfY2rcrR/++Sooc2IShw9gu63Cvz7OCDriE0Z569kd+Jf7HXIT3WOH8nS/+F6NXMOvAxMFgS+MR1ycuFNtrD4zUqUs0cxQulzXCFt+Z86vnyIytrFg92hc5DQPWOAEPtbc07U6NBxnoi6s0nf7E5jeHBVjqXATOcDkcBIN0NyseFHA0usuGowhXne5DKFgFcZ4cxNrO0fQt/RK9TUW8DkBc3ETxksEVnkUG65+mV18C3BTSiybGArsnXWoS6dPBmlUwv4LqunI8FXqgLKj1pJC/ACsLZOo4pFg/w9l6qhYsJPCGeWJmrzlLtrJyXXtHFRzc5ouM8gm8sJkmW29Eri6ERlaX2SKyGMsuZNfcMDhyv0h+/OSUToFlVv/fzEfLVzPB3xI9n1Tws2TCDBYne7UcZW95cbrLFsXqiL1bAZsS8OLWpovxcndTk/DQlO9FUzxNpahMpcodBHNg/6iusB8= ceph-af7e6fc8-7092-11ef-bcea-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-11T23:09:42.136 INFO:teuthology.orchestra.run.smithi026.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCFk7Bjp26C4V+TSSoQjcdWbS8+pBY9JCBMm0AmEJqupQfY2rcrR/++Sooc2IShw9gu63Cvz7OCDriE0Z569kd+Jf7HXIT3WOH8nS/+F6NXMOvAxMFgS+MR1ycuFNtrD4zUqUs0cxQulzXCFt+Z86vnyIytrFg92hc5DQPWOAEPtbc07U6NBxnoi6s0nf7E5jeHBVjqXATOcDkcBIN0NyseFHA0usuGowhXne5DKFgFcZ4cxNrO0fQt/RK9TUW8DkBc3ETxksEVnkUG65+mV18C3BTSiybGArsnXWoS6dPBmlUwv4LqunI8FXqgLKj1pJC/ACsLZOo4pFg/w9l6qhYsJPCGeWJmrzlLtrJyXXtHFRzc5ouM8gm8sJkmW29Eri6ERlaX2SKyGMsuZNfcMDhyv0h+/OSUToFlVv/fzEfLVzPB3xI9n1Tws2TCDBYne7UcZW95cbrLFsXqiL1bAZsS8OLWpovxcndTk/DQlO9FUzxNpahMpcodBHNg/6iusB8= ceph-af7e6fc8-7092-11ef-bcea-c7b262605968 2024-09-11T23:09:42.148 DEBUG:teuthology.orchestra.run.smithi076:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCFk7Bjp26C4V+TSSoQjcdWbS8+pBY9JCBMm0AmEJqupQfY2rcrR/++Sooc2IShw9gu63Cvz7OCDriE0Z569kd+Jf7HXIT3WOH8nS/+F6NXMOvAxMFgS+MR1ycuFNtrD4zUqUs0cxQulzXCFt+Z86vnyIytrFg92hc5DQPWOAEPtbc07U6NBxnoi6s0nf7E5jeHBVjqXATOcDkcBIN0NyseFHA0usuGowhXne5DKFgFcZ4cxNrO0fQt/RK9TUW8DkBc3ETxksEVnkUG65+mV18C3BTSiybGArsnXWoS6dPBmlUwv4LqunI8FXqgLKj1pJC/ACsLZOo4pFg/w9l6qhYsJPCGeWJmrzlLtrJyXXtHFRzc5ouM8gm8sJkmW29Eri6ERlaX2SKyGMsuZNfcMDhyv0h+/OSUToFlVv/fzEfLVzPB3xI9n1Tws2TCDBYne7UcZW95cbrLFsXqiL1bAZsS8OLWpovxcndTk/DQlO9FUzxNpahMpcodBHNg/6iusB8= ceph-af7e6fc8-7092-11ef-bcea-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-11T23:09:42.190 INFO:teuthology.orchestra.run.smithi076.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCFk7Bjp26C4V+TSSoQjcdWbS8+pBY9JCBMm0AmEJqupQfY2rcrR/++Sooc2IShw9gu63Cvz7OCDriE0Z569kd+Jf7HXIT3WOH8nS/+F6NXMOvAxMFgS+MR1ycuFNtrD4zUqUs0cxQulzXCFt+Z86vnyIytrFg92hc5DQPWOAEPtbc07U6NBxnoi6s0nf7E5jeHBVjqXATOcDkcBIN0NyseFHA0usuGowhXne5DKFgFcZ4cxNrO0fQt/RK9TUW8DkBc3ETxksEVnkUG65+mV18C3BTSiybGArsnXWoS6dPBmlUwv4LqunI8FXqgLKj1pJC/ACsLZOo4pFg/w9l6qhYsJPCGeWJmrzlLtrJyXXtHFRzc5ouM8gm8sJkmW29Eri6ERlaX2SKyGMsuZNfcMDhyv0h+/OSUToFlVv/fzEfLVzPB3xI9n1Tws2TCDBYne7UcZW95cbrLFsXqiL1bAZsS8OLWpovxcndTk/DQlO9FUzxNpahMpcodBHNg/6iusB8= ceph-af7e6fc8-7092-11ef-bcea-c7b262605968 2024-09-11T23:09:42.204 DEBUG:teuthology.orchestra.run.smithi136:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCFk7Bjp26C4V+TSSoQjcdWbS8+pBY9JCBMm0AmEJqupQfY2rcrR/++Sooc2IShw9gu63Cvz7OCDriE0Z569kd+Jf7HXIT3WOH8nS/+F6NXMOvAxMFgS+MR1ycuFNtrD4zUqUs0cxQulzXCFt+Z86vnyIytrFg92hc5DQPWOAEPtbc07U6NBxnoi6s0nf7E5jeHBVjqXATOcDkcBIN0NyseFHA0usuGowhXne5DKFgFcZ4cxNrO0fQt/RK9TUW8DkBc3ETxksEVnkUG65+mV18C3BTSiybGArsnXWoS6dPBmlUwv4LqunI8FXqgLKj1pJC/ACsLZOo4pFg/w9l6qhYsJPCGeWJmrzlLtrJyXXtHFRzc5ouM8gm8sJkmW29Eri6ERlaX2SKyGMsuZNfcMDhyv0h+/OSUToFlVv/fzEfLVzPB3xI9n1Tws2TCDBYne7UcZW95cbrLFsXqiL1bAZsS8OLWpovxcndTk/DQlO9FUzxNpahMpcodBHNg/6iusB8= ceph-af7e6fc8-7092-11ef-bcea-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-11T23:09:42.246 INFO:teuthology.orchestra.run.smithi136.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCFk7Bjp26C4V+TSSoQjcdWbS8+pBY9JCBMm0AmEJqupQfY2rcrR/++Sooc2IShw9gu63Cvz7OCDriE0Z569kd+Jf7HXIT3WOH8nS/+F6NXMOvAxMFgS+MR1ycuFNtrD4zUqUs0cxQulzXCFt+Z86vnyIytrFg92hc5DQPWOAEPtbc07U6NBxnoi6s0nf7E5jeHBVjqXATOcDkcBIN0NyseFHA0usuGowhXne5DKFgFcZ4cxNrO0fQt/RK9TUW8DkBc3ETxksEVnkUG65+mV18C3BTSiybGArsnXWoS6dPBmlUwv4LqunI8FXqgLKj1pJC/ACsLZOo4pFg/w9l6qhYsJPCGeWJmrzlLtrJyXXtHFRzc5ouM8gm8sJkmW29Eri6ERlaX2SKyGMsuZNfcMDhyv0h+/OSUToFlVv/fzEfLVzPB3xI9n1Tws2TCDBYne7UcZW95cbrLFsXqiL1bAZsS8OLWpovxcndTk/DQlO9FUzxNpahMpcodBHNg/6iusB8= ceph-af7e6fc8-7092-11ef-bcea-c7b262605968 2024-09-11T23:09:42.259 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-09-11T23:09:42.516 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:09:43.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:42.987+0000 7fe3ec4e8640 1 -- 172.21.15.26:0/673667113 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe3e4101600 msgr2=0x7fe3e4101a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:09:43.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:42.987+0000 7fe3ec4e8640 1 --2- 172.21.15.26:0/673667113 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe3e4101600 0x7fe3e4101a00 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe3e0009880 tx=0x7fe3e002a5d0 comp rx=0 tx=0).stop 2024-09-11T23:09:43.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:42.987+0000 7fe3ec4e8640 1 -- 172.21.15.26:0/673667113 shutdown_connections 2024-09-11T23:09:43.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:42.987+0000 7fe3ec4e8640 1 --2- 172.21.15.26:0/673667113 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe3e4101600 0x7fe3e4101a00 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:09:43.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:42.987+0000 7fe3ec4e8640 1 -- 172.21.15.26:0/673667113 >> 172.21.15.26:0/673667113 conn(0x7fe3e40fcdb0 msgr2=0x7fe3e40ff1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:09:43.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:42.987+0000 7fe3ec4e8640 1 -- 172.21.15.26:0/673667113 shutdown_connections 2024-09-11T23:09:43.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:42.987+0000 7fe3ec4e8640 1 -- 172.21.15.26:0/673667113 wait complete. 2024-09-11T23:09:43.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:42.987+0000 7fe3ec4e8640 1 Processor -- start 2024-09-11T23:09:43.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:42.988+0000 7fe3ec4e8640 1 -- start start 2024-09-11T23:09:43.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:42.988+0000 7fe3ec4e8640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe3e4101600 0x7fe3e4196db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:09:43.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:42.988+0000 7fe3ec4e8640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fe3e4107710 con 0x7fe3e4101600 2024-09-11T23:09:43.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:42.988+0000 7fe3ea284640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe3e4101600 0x7fe3e4196db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:09:43.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:42.988+0000 7fe3ea284640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe3e4101600 0x7fe3e4196db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41750/0 (socket says 172.21.15.26:41750) 2024-09-11T23:09:43.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:42.988+0000 7fe3ea284640 1 -- 172.21.15.26:0/461617794 learned_addr learned my addr 172.21.15.26:0/461617794 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:09:43.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:42.988+0000 7fe3ea284640 1 -- 172.21.15.26:0/461617794 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3e41972f0 con 0x7fe3e4101600 2024-09-11T23:09:43.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:42.988+0000 7fe3ea284640 1 --2- 172.21.15.26:0/461617794 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe3e4101600 0x7fe3e4196db0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe3e0004f60 tx=0x7fe3e00048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:09:43.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:42.988+0000 7fe3db7fe640 1 -- 172.21.15.26:0/461617794 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe3e003f070 con 0x7fe3e4101600 2024-09-11T23:09:43.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:42.988+0000 7fe3ec4e8640 1 -- 172.21.15.26:0/461617794 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3e4197580 con 0x7fe3e4101600 2024-09-11T23:09:43.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:42.988+0000 7fe3db7fe640 1 -- 172.21.15.26:0/461617794 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe3e0004cc0 con 0x7fe3e4101600 2024-09-11T23:09:43.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:42.988+0000 7fe3db7fe640 1 -- 172.21.15.26:0/461617794 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe3e0035030 con 0x7fe3e4101600 2024-09-11T23:09:43.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:42.988+0000 7fe3ec4e8640 1 -- 172.21.15.26:0/461617794 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3e419a230 con 0x7fe3e4101600 2024-09-11T23:09:43.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:42.989+0000 7fe3db7fe640 1 -- 172.21.15.26:0/461617794 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fe3e0042070 con 0x7fe3e4101600 2024-09-11T23:09:43.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:42.989+0000 7fe3ec4e8640 1 -- 172.21.15.26:0/461617794 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3e4107e90 con 0x7fe3e4101600 2024-09-11T23:09:43.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:42.990+0000 7fe3db7fe640 1 --2- 172.21.15.26:0/461617794 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fe3d0039060 0x7fe3d003b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:09:43.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:42.990+0000 7fe3db7fe640 1 -- 172.21.15.26:0/461617794 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fe3e006cf80 con 0x7fe3e4101600 2024-09-11T23:09:43.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:42.990+0000 7fe3e9a83640 1 --2- 172.21.15.26:0/461617794 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fe3d0039060 0x7fe3d003b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:09:43.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:42.990+0000 7fe3e9a83640 1 --2- 172.21.15.26:0/461617794 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fe3d0039060 0x7fe3d003b520 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fe3d40098c0 tx=0x7fe3d4006db0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:09:43.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:42.994+0000 7fe3db7fe640 1 -- 172.21.15.26:0/461617794 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe3e00433b0 con 0x7fe3e4101600 2024-09-11T23:09:43.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:43.171+0000 7fe3ec4e8640 1 -- 172.21.15.26:0/461617794 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7fe3e402d8a0 con 0x7fe3e4101600 2024-09-11T23:09:43.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:43.173+0000 7fe3db7fe640 1 -- 172.21.15.26:0/461617794 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7fe3e002f300 con 0x7fe3e4101600 2024-09-11T23:09:43.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:43.175+0000 7fe3ec4e8640 1 -- 172.21.15.26:0/461617794 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fe3d0039060 msgr2=0x7fe3d003b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:09:43.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:43.175+0000 7fe3ec4e8640 1 --2- 172.21.15.26:0/461617794 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fe3d0039060 0x7fe3d003b520 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fe3d40098c0 tx=0x7fe3d4006db0 comp rx=0 tx=0).stop 2024-09-11T23:09:43.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:43.175+0000 7fe3ec4e8640 1 -- 172.21.15.26:0/461617794 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe3e4101600 msgr2=0x7fe3e4196db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:09:43.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:43.175+0000 7fe3ec4e8640 1 --2- 172.21.15.26:0/461617794 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe3e4101600 0x7fe3e4196db0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe3e0004f60 tx=0x7fe3e00048f0 comp rx=0 tx=0).stop 2024-09-11T23:09:43.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:43.175+0000 7fe3ec4e8640 1 -- 172.21.15.26:0/461617794 shutdown_connections 2024-09-11T23:09:43.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:43.175+0000 7fe3ec4e8640 1 --2- 172.21.15.26:0/461617794 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fe3d0039060 0x7fe3d003b520 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:09:43.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:43.175+0000 7fe3ec4e8640 1 --2- 172.21.15.26:0/461617794 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe3e4101600 0x7fe3e4196db0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:09:43.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:43.175+0000 7fe3ec4e8640 1 -- 172.21.15.26:0/461617794 >> 172.21.15.26:0/461617794 conn(0x7fe3e40fcdb0 msgr2=0x7fe3e40fd770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:09:43.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:43.175+0000 7fe3ec4e8640 1 -- 172.21.15.26:0/461617794 shutdown_connections 2024-09-11T23:09:43.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:43.175+0000 7fe3ec4e8640 1 -- 172.21.15.26:0/461617794 wait complete. 2024-09-11T23:09:44.442 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:09:44 smithi026 ceph-mon[31172]: from='client.? 172.21.15.26:0/461617794' entity='client.admin' 2024-09-11T23:09:44.443 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-09-11T23:09:44.443 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-09-11T23:09:44.780 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:09:45.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:45.220+0000 7f1f7dd8b640 1 -- 172.21.15.26:0/2304437808 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f7810d600 msgr2=0x7f1f7810da00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:09:45.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:45.220+0000 7f1f7dd8b640 1 --2- 172.21.15.26:0/2304437808 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f7810d600 0x7f1f7810da00 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f1f68009880 tx=0x7f1f6802a5d0 comp rx=0 tx=0).stop 2024-09-11T23:09:45.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:45.220+0000 7f1f7dd8b640 1 -- 172.21.15.26:0/2304437808 shutdown_connections 2024-09-11T23:09:45.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:45.220+0000 7f1f7dd8b640 1 --2- 172.21.15.26:0/2304437808 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f7810d600 0x7f1f7810da00 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:09:45.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:45.220+0000 7f1f7dd8b640 1 -- 172.21.15.26:0/2304437808 >> 172.21.15.26:0/2304437808 conn(0x7f1f7806b7c0 msgr2=0x7f1f7806bbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:09:45.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:45.220+0000 7f1f7dd8b640 1 -- 172.21.15.26:0/2304437808 shutdown_connections 2024-09-11T23:09:45.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:45.220+0000 7f1f7dd8b640 1 -- 172.21.15.26:0/2304437808 wait complete. 2024-09-11T23:09:45.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:45.221+0000 7f1f7dd8b640 1 Processor -- start 2024-09-11T23:09:45.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:45.221+0000 7f1f7dd8b640 1 -- start start 2024-09-11T23:09:45.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:45.221+0000 7f1f7dd8b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f7810d600 0x7f1f781a1870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:09:45.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:45.221+0000 7f1f7dd8b640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f1f78112130 con 0x7f1f7810d600 2024-09-11T23:09:45.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:45.221+0000 7f1f777fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f7810d600 0x7f1f781a1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:09:45.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:45.221+0000 7f1f777fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f7810d600 0x7f1f781a1870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41770/0 (socket says 172.21.15.26:41770) 2024-09-11T23:09:45.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:45.221+0000 7f1f777fe640 1 -- 172.21.15.26:0/3943215700 learned_addr learned my addr 172.21.15.26:0/3943215700 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:09:45.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:45.221+0000 7f1f777fe640 1 -- 172.21.15.26:0/3943215700 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f781a1db0 con 0x7f1f7810d600 2024-09-11T23:09:45.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:45.222+0000 7f1f777fe640 1 --2- 172.21.15.26:0/3943215700 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f7810d600 0x7f1f781a1870 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1f68004f60 tx=0x7f1f680048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:09:45.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:45.222+0000 7f1f74ff9640 1 -- 172.21.15.26:0/3943215700 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1f6803f070 con 0x7f1f7810d600 2024-09-11T23:09:45.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:45.222+0000 7f1f74ff9640 1 -- 172.21.15.26:0/3943215700 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1f68004d40 con 0x7f1f7810d600 2024-09-11T23:09:45.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:45.222+0000 7f1f7dd8b640 1 -- 172.21.15.26:0/3943215700 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f781a2040 con 0x7f1f7810d600 2024-09-11T23:09:45.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:45.222+0000 7f1f74ff9640 1 -- 172.21.15.26:0/3943215700 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1f68035030 con 0x7f1f7810d600 2024-09-11T23:09:45.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:45.222+0000 7f1f7dd8b640 1 -- 172.21.15.26:0/3943215700 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f781a4d30 con 0x7f1f7810d600 2024-09-11T23:09:45.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:45.223+0000 7f1f74ff9640 1 -- 172.21.15.26:0/3943215700 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f1f68042070 con 0x7f1f7810d600 2024-09-11T23:09:45.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:45.223+0000 7f1f7dd8b640 1 -- 172.21.15.26:0/3943215700 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f781128b0 con 0x7f1f7810d600 2024-09-11T23:09:45.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:45.223+0000 7f1f74ff9640 1 --2- 172.21.15.26:0/3943215700 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f1f60039060 0x7f1f6003b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:09:45.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:45.223+0000 7f1f74ff9640 1 -- 172.21.15.26:0/3943215700 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f1f6806cd50 con 0x7f1f7810d600 2024-09-11T23:09:45.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:45.223+0000 7f1f76ffd640 1 --2- 172.21.15.26:0/3943215700 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f1f60039060 0x7f1f6003b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:09:45.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:45.224+0000 7f1f76ffd640 1 --2- 172.21.15.26:0/3943215700 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f1f60039060 0x7f1f6003b520 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f1f6c0098c0 tx=0x7f1f6c006db0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:09:45.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:45.228+0000 7f1f74ff9640 1 -- 172.21.15.26:0/3943215700 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1f68070030 con 0x7f1f7810d600 2024-09-11T23:09:45.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:45.410+0000 7f1f7dd8b640 1 -- 172.21.15.26:0/3943215700 --> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7f1f78062630 con 0x7f1f60039060 2024-09-11T23:09:45.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:45.413+0000 7f1f74ff9640 1 -- 172.21.15.26:0/3943215700 <== mgr.14148 v2:172.21.15.26:6800/110833859 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f1f78062630 con 0x7f1f60039060 2024-09-11T23:09:45.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:45.415+0000 7f1f7dd8b640 1 -- 172.21.15.26:0/3943215700 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f1f60039060 msgr2=0x7f1f6003b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:09:45.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:45.415+0000 7f1f7dd8b640 1 --2- 172.21.15.26:0/3943215700 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f1f60039060 0x7f1f6003b520 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f1f6c0098c0 tx=0x7f1f6c006db0 comp rx=0 tx=0).stop 2024-09-11T23:09:45.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:45.415+0000 7f1f7dd8b640 1 -- 172.21.15.26:0/3943215700 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f7810d600 msgr2=0x7f1f781a1870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:09:45.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:45.415+0000 7f1f7dd8b640 1 --2- 172.21.15.26:0/3943215700 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f7810d600 0x7f1f781a1870 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1f68004f60 tx=0x7f1f680048f0 comp rx=0 tx=0).stop 2024-09-11T23:09:45.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:45.415+0000 7f1f7dd8b640 1 -- 172.21.15.26:0/3943215700 shutdown_connections 2024-09-11T23:09:45.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:45.415+0000 7f1f7dd8b640 1 --2- 172.21.15.26:0/3943215700 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f1f60039060 0x7f1f6003b520 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:09:45.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:45.415+0000 7f1f7dd8b640 1 --2- 172.21.15.26:0/3943215700 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f7810d600 0x7f1f781a1870 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:09:45.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:45.415+0000 7f1f7dd8b640 1 -- 172.21.15.26:0/3943215700 >> 172.21.15.26:0/3943215700 conn(0x7f1f7806b7c0 msgr2=0x7f1f7810a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:09:45.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:45.415+0000 7f1f7dd8b640 1 -- 172.21.15.26:0/3943215700 shutdown_connections 2024-09-11T23:09:45.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:45.415+0000 7f1f7dd8b640 1 -- 172.21.15.26:0/3943215700 wait complete. 2024-09-11T23:09:46.071 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi076 2024-09-11T23:09:46.071 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-09-11T23:09:46.071 DEBUG:teuthology.orchestra.run.smithi076:> dd of=/etc/ceph/ceph.conf 2024-09-11T23:09:46.090 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-09-11T23:09:46.090 DEBUG:teuthology.orchestra.run.smithi076:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-11T23:09:46.148 INFO:tasks.cephadm:Adding host smithi076 to orchestrator... 2024-09-11T23:09:46.149 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph orch host add smithi076 2024-09-11T23:09:46.740 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:09:46.750 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:09:46 smithi026 ceph-mon[31172]: from='client.14170 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:09:46.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:09:46 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:09:47.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:47.288+0000 7f3f155da640 1 -- 172.21.15.26:0/1069215885 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3f10100540 msgr2=0x7f3f10100940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:09:47.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:47.288+0000 7f3f155da640 1 --2- 172.21.15.26:0/1069215885 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3f10100540 0x7f3f10100940 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f3f00009880 tx=0x7f3f0002a5d0 comp rx=0 tx=0).stop 2024-09-11T23:09:47.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:47.288+0000 7f3f155da640 1 -- 172.21.15.26:0/1069215885 shutdown_connections 2024-09-11T23:09:47.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:47.288+0000 7f3f155da640 1 --2- 172.21.15.26:0/1069215885 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3f10100540 0x7f3f10100940 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:09:47.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:47.288+0000 7f3f155da640 1 -- 172.21.15.26:0/1069215885 >> 172.21.15.26:0/1069215885 conn(0x7f3f100fbcd0 msgr2=0x7f3f100fe110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:09:47.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:47.288+0000 7f3f155da640 1 -- 172.21.15.26:0/1069215885 shutdown_connections 2024-09-11T23:09:47.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:47.288+0000 7f3f155da640 1 -- 172.21.15.26:0/1069215885 wait complete. 2024-09-11T23:09:47.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:47.289+0000 7f3f155da640 1 Processor -- start 2024-09-11T23:09:47.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:47.289+0000 7f3f155da640 1 -- start start 2024-09-11T23:09:47.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:47.289+0000 7f3f155da640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3f10100540 0x7f3f1006a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:09:47.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:47.289+0000 7f3f155da640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f3f101065f0 con 0x7f3f10100540 2024-09-11T23:09:47.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:47.289+0000 7f3f0effd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3f10100540 0x7f3f1006a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:09:47.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:47.289+0000 7f3f0effd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3f10100540 0x7f3f1006a130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41778/0 (socket says 172.21.15.26:41778) 2024-09-11T23:09:47.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:47.289+0000 7f3f0effd640 1 -- 172.21.15.26:0/1509676899 learned_addr learned my addr 172.21.15.26:0/1509676899 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:09:47.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:47.289+0000 7f3f0effd640 1 -- 172.21.15.26:0/1509676899 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f10199ab0 con 0x7f3f10100540 2024-09-11T23:09:47.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:47.289+0000 7f3f0effd640 1 --2- 172.21.15.26:0/1509676899 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3f10100540 0x7f3f1006a130 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3f00004f60 tx=0x7f3f000048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:09:47.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:47.290+0000 7f3f0cff9640 1 -- 172.21.15.26:0/1509676899 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3f0003f070 con 0x7f3f10100540 2024-09-11T23:09:47.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:47.290+0000 7f3f155da640 1 -- 172.21.15.26:0/1509676899 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f10199c80 con 0x7f3f10100540 2024-09-11T23:09:47.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:47.290+0000 7f3f0cff9640 1 -- 172.21.15.26:0/1509676899 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3f00004d40 con 0x7f3f10100540 2024-09-11T23:09:47.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:47.290+0000 7f3f0cff9640 1 -- 172.21.15.26:0/1509676899 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3f00035030 con 0x7f3f10100540 2024-09-11T23:09:47.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:47.290+0000 7f3f155da640 1 -- 172.21.15.26:0/1509676899 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f1019a020 con 0x7f3f10100540 2024-09-11T23:09:47.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:47.291+0000 7f3f0cff9640 1 -- 172.21.15.26:0/1509676899 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f3f00042070 con 0x7f3f10100540 2024-09-11T23:09:47.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:47.291+0000 7f3f155da640 1 -- 172.21.15.26:0/1509676899 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f10106d70 con 0x7f3f10100540 2024-09-11T23:09:47.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:47.291+0000 7f3f0cff9640 1 --2- 172.21.15.26:0/1509676899 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f3efc039060 0x7f3efc03b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:09:47.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:47.291+0000 7f3f0cff9640 1 -- 172.21.15.26:0/1509676899 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f3f0006cce0 con 0x7f3f10100540 2024-09-11T23:09:47.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:47.291+0000 7f3f07fff640 1 --2- 172.21.15.26:0/1509676899 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f3efc039060 0x7f3efc03b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:09:47.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:47.292+0000 7f3f07fff640 1 --2- 172.21.15.26:0/1509676899 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f3efc039060 0x7f3efc03b520 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f3ef80098c0 tx=0x7f3ef8006db0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:09:47.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:47.296+0000 7f3f0cff9640 1 -- 172.21.15.26:0/1509676899 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3f00039350 con 0x7f3f10100540 2024-09-11T23:09:47.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:47.484+0000 7f3f155da640 1 -- 172.21.15.26:0/1509676899 --> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi076", "target": ["mon-mgr", ""]}) -- 0x7f3f1002d930 con 0x7f3efc039060 2024-09-11T23:09:47.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:09:47 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:09:47.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:09:47 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:09:47.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:09:47 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi026", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:09:47.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:09:47 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:09:47.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:09:47 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:09:47.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:09:47 smithi026 ceph-mon[31172]: Updating smithi026:/etc/ceph/ceph.conf 2024-09-11T23:09:47.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:09:47 smithi026 ceph-mon[31172]: Updating smithi026:/var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T23:09:47.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:09:47 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:09:47.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:09:47 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:09:47.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:09:47 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:09:47.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:09:47 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:09:47.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:09:47 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:09:47.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:09:47 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:09:47.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:09:47 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:09:49.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:09:48 smithi026 ceph-mon[31172]: Updating smithi026:/etc/ceph/ceph.client.admin.keyring 2024-09-11T23:09:49.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:09:48 smithi026 ceph-mon[31172]: Updating smithi026:/var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T23:09:49.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:09:48 smithi026 ceph-mon[31172]: from='client.14172 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi076", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:09:50.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:09:49 smithi026 ceph-mon[31172]: Deploying cephadm binary to smithi076 2024-09-11T23:09:51.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:51.314+0000 7f3f0cff9640 1 -- 172.21.15.26:0/1509676899 <== mgr.14148 v2:172.21.15.26:6800/110833859 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f3f1002d930 con 0x7f3efc039060 2024-09-11T23:09:51.316 INFO:teuthology.orchestra.run.smithi026.stdout:Added host 'smithi076' with addr '172.21.15.76' 2024-09-11T23:09:51.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:51.317+0000 7f3f155da640 1 -- 172.21.15.26:0/1509676899 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f3efc039060 msgr2=0x7f3efc03b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:09:51.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:51.317+0000 7f3f155da640 1 --2- 172.21.15.26:0/1509676899 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f3efc039060 0x7f3efc03b520 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f3ef80098c0 tx=0x7f3ef8006db0 comp rx=0 tx=0).stop 2024-09-11T23:09:51.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:51.317+0000 7f3f155da640 1 -- 172.21.15.26:0/1509676899 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3f10100540 msgr2=0x7f3f1006a130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:09:51.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:51.317+0000 7f3f155da640 1 --2- 172.21.15.26:0/1509676899 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3f10100540 0x7f3f1006a130 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3f00004f60 tx=0x7f3f000048f0 comp rx=0 tx=0).stop 2024-09-11T23:09:51.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:51.317+0000 7f3f155da640 1 -- 172.21.15.26:0/1509676899 shutdown_connections 2024-09-11T23:09:51.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:51.317+0000 7f3f155da640 1 --2- 172.21.15.26:0/1509676899 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f3efc039060 0x7f3efc03b520 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:09:51.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:51.317+0000 7f3f155da640 1 --2- 172.21.15.26:0/1509676899 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3f10100540 0x7f3f1006a130 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:09:51.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:51.317+0000 7f3f155da640 1 -- 172.21.15.26:0/1509676899 >> 172.21.15.26:0/1509676899 conn(0x7f3f100fbcd0 msgr2=0x7f3f100fc6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:09:51.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:51.317+0000 7f3f155da640 1 -- 172.21.15.26:0/1509676899 shutdown_connections 2024-09-11T23:09:51.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:51.317+0000 7f3f155da640 1 -- 172.21.15.26:0/1509676899 wait complete. 2024-09-11T23:09:51.927 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph orch host ls --format=json 2024-09-11T23:09:52.176 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:09:52.618 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:09:52 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:09:52.618 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:09:52 smithi026 ceph-mon[31172]: Added host smithi076 2024-09-11T23:09:52.618 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:09:52 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:09:52.618 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:09:52 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:09:52.619 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:09:52 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:09:52.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:52.617+0000 7fc079a50640 1 -- 172.21.15.26:0/536269680 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc074106540 msgr2=0x7fc074106920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:09:52.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:52.617+0000 7fc079a50640 1 --2- 172.21.15.26:0/536269680 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc074106540 0x7fc074106920 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fc064009810 tx=0x7fc06402a540 comp rx=0 tx=0).stop 2024-09-11T23:09:52.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:52.617+0000 7fc079a50640 1 -- 172.21.15.26:0/536269680 shutdown_connections 2024-09-11T23:09:52.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:52.617+0000 7fc079a50640 1 --2- 172.21.15.26:0/536269680 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc074106540 0x7fc074106920 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:09:52.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:52.617+0000 7fc079a50640 1 -- 172.21.15.26:0/536269680 >> 172.21.15.26:0/536269680 conn(0x7fc074077310 msgr2=0x7fc0740fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:09:52.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:52.617+0000 7fc079a50640 1 -- 172.21.15.26:0/536269680 shutdown_connections 2024-09-11T23:09:52.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:52.617+0000 7fc079a50640 1 -- 172.21.15.26:0/536269680 wait complete. 2024-09-11T23:09:52.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:52.618+0000 7fc079a50640 1 Processor -- start 2024-09-11T23:09:52.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:52.618+0000 7fc079a50640 1 -- start start 2024-09-11T23:09:52.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:52.618+0000 7fc079a50640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc074106540 0x7fc074197f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:09:52.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:52.618+0000 7fc079a50640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc07410ab70 con 0x7fc074106540 2024-09-11T23:09:52.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:52.618+0000 7fc072ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc074106540 0x7fc074197f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:09:52.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:52.618+0000 7fc072ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc074106540 0x7fc074197f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57216/0 (socket says 172.21.15.26:57216) 2024-09-11T23:09:52.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:52.618+0000 7fc072ffd640 1 -- 172.21.15.26:0/553346053 learned_addr learned my addr 172.21.15.26:0/553346053 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:09:52.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:52.618+0000 7fc072ffd640 1 -- 172.21.15.26:0/553346053 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0741984b0 con 0x7fc074106540 2024-09-11T23:09:52.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:52.619+0000 7fc072ffd640 1 --2- 172.21.15.26:0/553346053 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc074106540 0x7fc074197f70 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fc0640045e0 tx=0x7fc064004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:09:52.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:52.619+0000 7fc078a4e640 1 -- 172.21.15.26:0/553346053 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc064030e70 con 0x7fc074106540 2024-09-11T23:09:52.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:52.619+0000 7fc078a4e640 1 -- 172.21.15.26:0/553346053 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc0640318e0 con 0x7fc074106540 2024-09-11T23:09:52.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:52.619+0000 7fc079a50640 1 -- 172.21.15.26:0/553346053 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc074198740 con 0x7fc074106540 2024-09-11T23:09:52.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:52.619+0000 7fc078a4e640 1 -- 172.21.15.26:0/553346053 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc0640049c0 con 0x7fc074106540 2024-09-11T23:09:52.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:52.619+0000 7fc079a50640 1 -- 172.21.15.26:0/553346053 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc074198ba0 con 0x7fc074106540 2024-09-11T23:09:52.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:52.620+0000 7fc078a4e640 1 -- 172.21.15.26:0/553346053 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fc064039040 con 0x7fc074106540 2024-09-11T23:09:52.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:52.620+0000 7fc078a4e640 1 --2- 172.21.15.26:0/553346053 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fc0600390c0 0x7fc06003b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:09:52.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:52.620+0000 7fc078a4e640 1 -- 172.21.15.26:0/553346053 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fc064031400 con 0x7fc074106540 2024-09-11T23:09:52.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:52.620+0000 7fc0727fc640 1 --2- 172.21.15.26:0/553346053 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fc0600390c0 0x7fc06003b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:09:52.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:52.620+0000 7fc079a50640 1 -- 172.21.15.26:0/553346053 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc074101bb0 con 0x7fc074106540 2024-09-11T23:09:52.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:52.620+0000 7fc0727fc640 1 --2- 172.21.15.26:0/553346053 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fc0600390c0 0x7fc06003b580 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fc05c00ad30 tx=0x7fc05c0093f0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:09:52.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:52.625+0000 7fc078a4e640 1 -- 172.21.15.26:0/553346053 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc064040030 con 0x7fc074106540 2024-09-11T23:09:52.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:52.809+0000 7fc079a50640 1 -- 172.21.15.26:0/553346053 --> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc0741069a0 con 0x7fc0600390c0 2024-09-11T23:09:52.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:52.810+0000 7fc078a4e640 1 -- 172.21.15.26:0/553346053 <== mgr.14148 v2:172.21.15.26:6800/110833859 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+159 (secure 0 0 0) 0x7fc0741069a0 con 0x7fc0600390c0 2024-09-11T23:09:52.813 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-11T23:09:52.813 INFO:teuthology.orchestra.run.smithi026.stdout:[{"addr": "172.21.15.26", "hostname": "smithi026", "labels": [], "status": ""}, {"addr": "172.21.15.76", "hostname": "smithi076", "labels": [], "status": ""}] 2024-09-11T23:09:52.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:52.813+0000 7fc079a50640 1 -- 172.21.15.26:0/553346053 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fc0600390c0 msgr2=0x7fc06003b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:09:52.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:52.813+0000 7fc079a50640 1 --2- 172.21.15.26:0/553346053 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fc0600390c0 0x7fc06003b580 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fc05c00ad30 tx=0x7fc05c0093f0 comp rx=0 tx=0).stop 2024-09-11T23:09:52.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:52.813+0000 7fc079a50640 1 -- 172.21.15.26:0/553346053 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc074106540 msgr2=0x7fc074197f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:09:52.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:52.813+0000 7fc079a50640 1 --2- 172.21.15.26:0/553346053 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc074106540 0x7fc074197f70 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fc0640045e0 tx=0x7fc064004610 comp rx=0 tx=0).stop 2024-09-11T23:09:52.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:52.813+0000 7fc079a50640 1 -- 172.21.15.26:0/553346053 shutdown_connections 2024-09-11T23:09:52.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:52.813+0000 7fc079a50640 1 --2- 172.21.15.26:0/553346053 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fc0600390c0 0x7fc06003b580 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:09:52.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:52.813+0000 7fc079a50640 1 --2- 172.21.15.26:0/553346053 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc074106540 0x7fc074197f70 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:09:52.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:52.813+0000 7fc079a50640 1 -- 172.21.15.26:0/553346053 >> 172.21.15.26:0/553346053 conn(0x7fc074077310 msgr2=0x7fc0740fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:09:52.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:52.813+0000 7fc079a50640 1 -- 172.21.15.26:0/553346053 shutdown_connections 2024-09-11T23:09:52.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:52.813+0000 7fc079a50640 1 -- 172.21.15.26:0/553346053 wait complete. 2024-09-11T23:09:53.457 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi136 2024-09-11T23:09:53.457 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-09-11T23:09:53.457 DEBUG:teuthology.orchestra.run.smithi136:> dd of=/etc/ceph/ceph.conf 2024-09-11T23:09:53.474 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-09-11T23:09:53.474 DEBUG:teuthology.orchestra.run.smithi136:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-11T23:09:53.530 INFO:tasks.cephadm:Adding host smithi136 to orchestrator... 2024-09-11T23:09:53.530 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph orch host add smithi136 2024-09-11T23:09:53.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:09:53 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:09:53.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:09:53 smithi026 ceph-mon[31172]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T23:09:53.777 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:09:54.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:54.215+0000 7feafd3bc640 1 -- 172.21.15.26:0/4090290307 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feaf8106560 msgr2=0x7feaf8106940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:09:54.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:54.215+0000 7feafd3bc640 1 --2- 172.21.15.26:0/4090290307 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feaf8106560 0x7feaf8106940 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7feae8009880 tx=0x7feae802a5d0 comp rx=0 tx=0).stop 2024-09-11T23:09:54.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:54.215+0000 7feafd3bc640 1 -- 172.21.15.26:0/4090290307 shutdown_connections 2024-09-11T23:09:54.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:54.215+0000 7feafd3bc640 1 --2- 172.21.15.26:0/4090290307 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feaf8106560 0x7feaf8106940 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:09:54.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:54.215+0000 7feafd3bc640 1 -- 172.21.15.26:0/4090290307 >> 172.21.15.26:0/4090290307 conn(0x7feaf80fc1b0 msgr2=0x7feaf80fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:09:54.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:54.216+0000 7feafd3bc640 1 -- 172.21.15.26:0/4090290307 shutdown_connections 2024-09-11T23:09:54.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:54.216+0000 7feafd3bc640 1 -- 172.21.15.26:0/4090290307 wait complete. 2024-09-11T23:09:54.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:54.216+0000 7feafd3bc640 1 Processor -- start 2024-09-11T23:09:54.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:54.216+0000 7feafd3bc640 1 -- start start 2024-09-11T23:09:54.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:54.216+0000 7feafd3bc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feaf8106560 0x7feaf80fc660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:09:54.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:54.216+0000 7feafd3bc640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7feaf810ab90 con 0x7feaf8106560 2024-09-11T23:09:54.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:54.217+0000 7feaf6ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feaf8106560 0x7feaf80fc660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:09:54.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:54.217+0000 7feaf6ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feaf8106560 0x7feaf80fc660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57234/0 (socket says 172.21.15.26:57234) 2024-09-11T23:09:54.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:54.217+0000 7feaf6ffd640 1 -- 172.21.15.26:0/3169389118 learned_addr learned my addr 172.21.15.26:0/3169389118 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:09:54.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:54.217+0000 7feaf6ffd640 1 -- 172.21.15.26:0/3169389118 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feaf80fe550 con 0x7feaf8106560 2024-09-11T23:09:54.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:54.217+0000 7feaf6ffd640 1 --2- 172.21.15.26:0/3169389118 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feaf8106560 0x7feaf80fc660 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7feae8009230 tx=0x7feae8003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:09:54.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:54.217+0000 7feaf4ff9640 1 -- 172.21.15.26:0/3169389118 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feae803f070 con 0x7feaf8106560 2024-09-11T23:09:54.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:54.217+0000 7feafd3bc640 1 -- 172.21.15.26:0/3169389118 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feaf80fcba0 con 0x7feaf8106560 2024-09-11T23:09:54.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:54.217+0000 7feaf4ff9640 1 -- 172.21.15.26:0/3169389118 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feae803bc80 con 0x7feaf8106560 2024-09-11T23:09:54.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:54.217+0000 7feaf4ff9640 1 -- 172.21.15.26:0/3169389118 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feae8035030 con 0x7feaf8106560 2024-09-11T23:09:54.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:54.217+0000 7feafd3bc640 1 -- 172.21.15.26:0/3169389118 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feaf80fd060 con 0x7feaf8106560 2024-09-11T23:09:54.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:54.218+0000 7feaf4ff9640 1 -- 172.21.15.26:0/3169389118 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7feae8042070 con 0x7feaf8106560 2024-09-11T23:09:54.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:54.218+0000 7feafd3bc640 1 -- 172.21.15.26:0/3169389118 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feaf8101bd0 con 0x7feaf8106560 2024-09-11T23:09:54.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:54.218+0000 7feaf4ff9640 1 --2- 172.21.15.26:0/3169389118 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7feae4039060 0x7feae403b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:09:54.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:54.218+0000 7feaf4ff9640 1 -- 172.21.15.26:0/3169389118 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7feae806cce0 con 0x7feaf8106560 2024-09-11T23:09:54.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:54.218+0000 7feaeffff640 1 --2- 172.21.15.26:0/3169389118 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7feae4039060 0x7feae403b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:09:54.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:54.219+0000 7feaeffff640 1 --2- 172.21.15.26:0/3169389118 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7feae4039060 0x7feae403b520 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7feae00098c0 tx=0x7feae0006db0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:09:54.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:54.223+0000 7feaf4ff9640 1 -- 172.21.15.26:0/3169389118 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7feae80372d0 con 0x7feaf8106560 2024-09-11T23:09:54.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:54.401+0000 7feafd3bc640 1 -- 172.21.15.26:0/3169389118 --> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi136", "target": ["mon-mgr", ""]}) -- 0x7feaf80fdb90 con 0x7feae4039060 2024-09-11T23:09:55.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:09:54 smithi026 ceph-mon[31172]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:09:55.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:09:54 smithi026 ceph-mon[31172]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi136", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:09:56.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:09:56 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:09:56.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:09:56 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:09:56.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:09:56 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:09:56.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:09:56 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:09:56.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:09:56 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi076", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:09:56.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:09:56 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:09:56.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:09:56 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:09:56.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:09:56 smithi026 ceph-mon[31172]: Updating smithi076:/etc/ceph/ceph.conf 2024-09-11T23:09:56.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:09:56 smithi026 ceph-mon[31172]: Deploying cephadm binary to smithi136 2024-09-11T23:09:56.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:09:56 smithi026 ceph-mon[31172]: Updating smithi076:/var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T23:09:56.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:09:56 smithi026 ceph-mon[31172]: Updating smithi076:/etc/ceph/ceph.client.admin.keyring 2024-09-11T23:09:56.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:09:56 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:09:56.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:09:56 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:09:56.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:09:56 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:09:57.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:09:57 smithi026 ceph-mon[31172]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:09:57.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:09:57 smithi026 ceph-mon[31172]: Updating smithi076:/var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T23:09:58.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:58.224+0000 7feaf4ff9640 1 -- 172.21.15.26:0/3169389118 <== mgr.14148 v2:172.21.15.26:6800/110833859 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7feaf80fdb90 con 0x7feae4039060 2024-09-11T23:09:58.225 INFO:teuthology.orchestra.run.smithi026.stdout:Added host 'smithi136' with addr '172.21.15.136' 2024-09-11T23:09:58.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:58.227+0000 7feafd3bc640 1 -- 172.21.15.26:0/3169389118 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7feae4039060 msgr2=0x7feae403b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:09:58.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:58.227+0000 7feafd3bc640 1 --2- 172.21.15.26:0/3169389118 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7feae4039060 0x7feae403b520 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7feae00098c0 tx=0x7feae0006db0 comp rx=0 tx=0).stop 2024-09-11T23:09:58.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:58.227+0000 7feafd3bc640 1 -- 172.21.15.26:0/3169389118 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feaf8106560 msgr2=0x7feaf80fc660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:09:58.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:58.227+0000 7feafd3bc640 1 --2- 172.21.15.26:0/3169389118 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feaf8106560 0x7feaf80fc660 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7feae8009230 tx=0x7feae8003d40 comp rx=0 tx=0).stop 2024-09-11T23:09:58.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:58.227+0000 7feafd3bc640 1 -- 172.21.15.26:0/3169389118 shutdown_connections 2024-09-11T23:09:58.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:58.227+0000 7feafd3bc640 1 --2- 172.21.15.26:0/3169389118 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7feae4039060 0x7feae403b520 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:09:58.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:58.227+0000 7feafd3bc640 1 --2- 172.21.15.26:0/3169389118 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feaf8106560 0x7feaf80fc660 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:09:58.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:58.227+0000 7feafd3bc640 1 -- 172.21.15.26:0/3169389118 >> 172.21.15.26:0/3169389118 conn(0x7feaf80fc1b0 msgr2=0x7feaf8069ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:09:58.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:58.227+0000 7feafd3bc640 1 -- 172.21.15.26:0/3169389118 shutdown_connections 2024-09-11T23:09:58.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:58.227+0000 7feafd3bc640 1 -- 172.21.15.26:0/3169389118 wait complete. 2024-09-11T23:09:58.932 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph orch host ls --format=json 2024-09-11T23:09:59.183 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:09:59.501 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:09:59 smithi026 ceph-mon[31172]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:09:59.501 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:09:59 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:09:59.501 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:09:59 smithi026 ceph-mon[31172]: Added host smithi136 2024-09-11T23:09:59.501 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:09:59 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:09:59.501 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:09:59 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:09:59.501 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:09:59 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:09:59.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:59.626+0000 7f9e54a71640 1 -- 172.21.15.26:0/2149836322 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9e50107660 msgr2=0x7f9e50107a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:09:59.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:59.626+0000 7f9e54a71640 1 --2- 172.21.15.26:0/2149836322 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9e50107660 0x7f9e50107a40 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f9e40009880 tx=0x7f9e4002a5d0 comp rx=0 tx=0).stop 2024-09-11T23:09:59.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:59.626+0000 7f9e54a71640 1 -- 172.21.15.26:0/2149836322 shutdown_connections 2024-09-11T23:09:59.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:59.626+0000 7f9e54a71640 1 --2- 172.21.15.26:0/2149836322 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9e50107660 0x7f9e50107a40 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:09:59.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:59.626+0000 7f9e54a71640 1 -- 172.21.15.26:0/2149836322 >> 172.21.15.26:0/2149836322 conn(0x7f9e500fd290 msgr2=0x7f9e500ff6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:09:59.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:59.626+0000 7f9e54a71640 1 -- 172.21.15.26:0/2149836322 shutdown_connections 2024-09-11T23:09:59.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:59.626+0000 7f9e54a71640 1 -- 172.21.15.26:0/2149836322 wait complete. 2024-09-11T23:09:59.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:59.627+0000 7f9e54a71640 1 Processor -- start 2024-09-11T23:09:59.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:59.627+0000 7f9e54a71640 1 -- start start 2024-09-11T23:09:59.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:59.627+0000 7f9e54a71640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9e50107660 0x7f9e501970f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:09:59.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:59.627+0000 7f9e54a71640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f9e5010bc90 con 0x7f9e50107660 2024-09-11T23:09:59.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:59.627+0000 7f9e4e59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9e50107660 0x7f9e501970f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:09:59.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:59.627+0000 7f9e4e59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9e50107660 0x7f9e501970f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57246/0 (socket says 172.21.15.26:57246) 2024-09-11T23:09:59.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:59.627+0000 7f9e4e59c640 1 -- 172.21.15.26:0/3935492075 learned_addr learned my addr 172.21.15.26:0/3935492075 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:09:59.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:59.627+0000 7f9e4e59c640 1 -- 172.21.15.26:0/3935492075 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e50197630 con 0x7f9e50107660 2024-09-11T23:09:59.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:59.628+0000 7f9e4e59c640 1 --2- 172.21.15.26:0/3935492075 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9e50107660 0x7f9e501970f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9e40009230 tx=0x7f9e40003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:09:59.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:59.628+0000 7f9e477fe640 1 -- 172.21.15.26:0/3935492075 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9e4003f070 con 0x7f9e50107660 2024-09-11T23:09:59.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:59.628+0000 7f9e477fe640 1 -- 172.21.15.26:0/3935492075 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e4003bc80 con 0x7f9e50107660 2024-09-11T23:09:59.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:59.628+0000 7f9e54a71640 1 -- 172.21.15.26:0/3935492075 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e501978c0 con 0x7f9e50107660 2024-09-11T23:09:59.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:59.628+0000 7f9e477fe640 1 -- 172.21.15.26:0/3935492075 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9e40035030 con 0x7f9e50107660 2024-09-11T23:09:59.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:59.628+0000 7f9e54a71640 1 -- 172.21.15.26:0/3935492075 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e5019b9c0 con 0x7f9e50107660 2024-09-11T23:09:59.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:59.629+0000 7f9e477fe640 1 -- 172.21.15.26:0/3935492075 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f9e40042070 con 0x7f9e50107660 2024-09-11T23:09:59.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:59.629+0000 7f9e477fe640 1 --2- 172.21.15.26:0/3935492075 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f9e3c0390c0 0x7f9e3c03b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:09:59.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:59.629+0000 7f9e54a71640 1 -- 172.21.15.26:0/3935492075 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e50102cd0 con 0x7f9e50107660 2024-09-11T23:09:59.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:59.629+0000 7f9e477fe640 1 -- 172.21.15.26:0/3935492075 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f9e4006cc90 con 0x7f9e50107660 2024-09-11T23:09:59.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:59.629+0000 7f9e4dd9b640 1 --2- 172.21.15.26:0/3935492075 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f9e3c0390c0 0x7f9e3c03b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:09:59.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:59.629+0000 7f9e4dd9b640 1 --2- 172.21.15.26:0/3935492075 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f9e3c0390c0 0x7f9e3c03b580 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f9e380098c0 tx=0x7f9e38006db0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:09:59.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:59.634+0000 7f9e477fe640 1 -- 172.21.15.26:0/3935492075 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9e400314c0 con 0x7f9e50107660 2024-09-11T23:09:59.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:59.814+0000 7f9e54a71640 1 -- 172.21.15.26:0/3935492075 --> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9e50107b20 con 0x7f9e3c0390c0 2024-09-11T23:09:59.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:59.815+0000 7f9e477fe640 1 -- 172.21.15.26:0/3935492075 <== mgr.14148 v2:172.21.15.26:6800/110833859 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+239 (secure 0 0 0) 0x7f9e50107b20 con 0x7f9e3c0390c0 2024-09-11T23:09:59.815 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-11T23:09:59.816 INFO:teuthology.orchestra.run.smithi026.stdout:[{"addr": "172.21.15.26", "hostname": "smithi026", "labels": [], "status": ""}, {"addr": "172.21.15.76", "hostname": "smithi076", "labels": [], "status": ""}, {"addr": "172.21.15.136", "hostname": "smithi136", "labels": [], "status": ""}] 2024-09-11T23:09:59.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:59.817+0000 7f9e54a71640 1 -- 172.21.15.26:0/3935492075 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f9e3c0390c0 msgr2=0x7f9e3c03b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:09:59.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:59.817+0000 7f9e54a71640 1 --2- 172.21.15.26:0/3935492075 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f9e3c0390c0 0x7f9e3c03b580 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f9e380098c0 tx=0x7f9e38006db0 comp rx=0 tx=0).stop 2024-09-11T23:09:59.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:59.817+0000 7f9e54a71640 1 -- 172.21.15.26:0/3935492075 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9e50107660 msgr2=0x7f9e501970f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:09:59.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:59.817+0000 7f9e54a71640 1 --2- 172.21.15.26:0/3935492075 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9e50107660 0x7f9e501970f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9e40009230 tx=0x7f9e40003d40 comp rx=0 tx=0).stop 2024-09-11T23:09:59.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:59.817+0000 7f9e54a71640 1 -- 172.21.15.26:0/3935492075 shutdown_connections 2024-09-11T23:09:59.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:59.817+0000 7f9e54a71640 1 --2- 172.21.15.26:0/3935492075 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f9e3c0390c0 0x7f9e3c03b580 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:09:59.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:59.817+0000 7f9e54a71640 1 --2- 172.21.15.26:0/3935492075 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9e50107660 0x7f9e501970f0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:09:59.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:59.817+0000 7f9e54a71640 1 -- 172.21.15.26:0/3935492075 >> 172.21.15.26:0/3935492075 conn(0x7f9e500fd290 msgr2=0x7f9e500febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:09:59.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:59.818+0000 7f9e54a71640 1 -- 172.21.15.26:0/3935492075 shutdown_connections 2024-09-11T23:09:59.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:09:59.818+0000 7f9e54a71640 1 -- 172.21.15.26:0/3935492075 wait complete. 2024-09-11T23:10:00.507 INFO:tasks.cephadm:Setting crush tunables to default 2024-09-11T23:10:00.507 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph osd crush tunables default 2024-09-11T23:10:00.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:00 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:10:00.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:00 smithi026 ceph-mon[31172]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T23:10:00.759 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:10:01.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:10:01.196+0000 7fc344012640 1 -- 172.21.15.26:0/4009247905 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc33c1027d0 msgr2=0x7fc33c102bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:10:01.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:10:01.196+0000 7fc344012640 1 --2- 172.21.15.26:0/4009247905 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc33c1027d0 0x7fc33c102bb0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fc338009880 tx=0x7fc33802a670 comp rx=0 tx=0).stop 2024-09-11T23:10:01.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:10:01.196+0000 7fc344012640 1 -- 172.21.15.26:0/4009247905 shutdown_connections 2024-09-11T23:10:01.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:10:01.196+0000 7fc344012640 1 --2- 172.21.15.26:0/4009247905 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc33c1027d0 0x7fc33c102bb0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:10:01.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:10:01.196+0000 7fc344012640 1 -- 172.21.15.26:0/4009247905 >> 172.21.15.26:0/4009247905 conn(0x7fc33c0fc190 msgr2=0x7fc33c0fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:10:01.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:10:01.196+0000 7fc344012640 1 -- 172.21.15.26:0/4009247905 shutdown_connections 2024-09-11T23:10:01.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:10:01.196+0000 7fc344012640 1 -- 172.21.15.26:0/4009247905 wait complete. 2024-09-11T23:10:01.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:10:01.197+0000 7fc344012640 1 Processor -- start 2024-09-11T23:10:01.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:10:01.197+0000 7fc344012640 1 -- start start 2024-09-11T23:10:01.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:10:01.197+0000 7fc344012640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc33c1027d0 0x7fc33c197170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:10:01.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:10:01.197+0000 7fc344012640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc33c10bcd0 con 0x7fc33c1027d0 2024-09-11T23:10:01.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:10:01.197+0000 7fc341dae640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc33c1027d0 0x7fc33c197170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:10:01.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:10:01.197+0000 7fc341dae640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc33c1027d0 0x7fc33c197170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37982/0 (socket says 172.21.15.26:37982) 2024-09-11T23:10:01.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:10:01.197+0000 7fc341dae640 1 -- 172.21.15.26:0/1823375458 learned_addr learned my addr 172.21.15.26:0/1823375458 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:10:01.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:10:01.197+0000 7fc341dae640 1 -- 172.21.15.26:0/1823375458 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc33c1976b0 con 0x7fc33c1027d0 2024-09-11T23:10:01.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:10:01.197+0000 7fc341dae640 1 --2- 172.21.15.26:0/1823375458 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc33c1027d0 0x7fc33c197170 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc338009230 tx=0x7fc338003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:10:01.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:10:01.197+0000 7fc332ffd640 1 -- 172.21.15.26:0/1823375458 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc33803f070 con 0x7fc33c1027d0 2024-09-11T23:10:01.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:10:01.198+0000 7fc332ffd640 1 -- 172.21.15.26:0/1823375458 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc33803bc80 con 0x7fc33c1027d0 2024-09-11T23:10:01.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:10:01.198+0000 7fc344012640 1 -- 172.21.15.26:0/1823375458 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc33c197940 con 0x7fc33c1027d0 2024-09-11T23:10:01.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:10:01.198+0000 7fc332ffd640 1 -- 172.21.15.26:0/1823375458 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc338035030 con 0x7fc33c1027d0 2024-09-11T23:10:01.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:10:01.198+0000 7fc344012640 1 -- 172.21.15.26:0/1823375458 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc33c197da0 con 0x7fc33c1027d0 2024-09-11T23:10:01.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:10:01.198+0000 7fc332ffd640 1 -- 172.21.15.26:0/1823375458 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fc338042070 con 0x7fc33c1027d0 2024-09-11T23:10:01.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:10:01.199+0000 7fc332ffd640 1 --2- 172.21.15.26:0/1823375458 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fc3280390c0 0x7fc32803b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:10:01.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:10:01.199+0000 7fc332ffd640 1 -- 172.21.15.26:0/1823375458 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fc33806ccf0 con 0x7fc33c1027d0 2024-09-11T23:10:01.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:10:01.199+0000 7fc344012640 1 -- 172.21.15.26:0/1823375458 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc33c06a750 con 0x7fc33c1027d0 2024-09-11T23:10:01.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:10:01.199+0000 7fc3415ad640 1 --2- 172.21.15.26:0/1823375458 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fc3280390c0 0x7fc32803b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:10:01.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:10:01.199+0000 7fc3415ad640 1 --2- 172.21.15.26:0/1823375458 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fc3280390c0 0x7fc32803b580 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fc32c0098c0 tx=0x7fc32c006db0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:10:01.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:10:01.204+0000 7fc332ffd640 1 -- 172.21.15.26:0/1823375458 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc3380314c0 con 0x7fc33c1027d0 2024-09-11T23:10:01.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:10:01.381+0000 7fc344012640 1 -- 172.21.15.26:0/1823375458 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7fc33c1984d0 con 0x7fc33c1027d0 2024-09-11T23:10:01.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:10:01.565+0000 7fc332ffd640 1 -- 172.21.15.26:0/1823375458 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7fc338031e20 con 0x7fc33c1027d0 2024-09-11T23:10:01.566 INFO:teuthology.orchestra.run.smithi026.stderr:adjusted tunables profile to default 2024-09-11T23:10:01.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:10:01.568+0000 7fc344012640 1 -- 172.21.15.26:0/1823375458 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fc3280390c0 msgr2=0x7fc32803b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:10:01.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:10:01.568+0000 7fc344012640 1 --2- 172.21.15.26:0/1823375458 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fc3280390c0 0x7fc32803b580 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fc32c0098c0 tx=0x7fc32c006db0 comp rx=0 tx=0).stop 2024-09-11T23:10:01.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:10:01.568+0000 7fc344012640 1 -- 172.21.15.26:0/1823375458 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc33c1027d0 msgr2=0x7fc33c197170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:10:01.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:10:01.568+0000 7fc344012640 1 --2- 172.21.15.26:0/1823375458 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc33c1027d0 0x7fc33c197170 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc338009230 tx=0x7fc338003d40 comp rx=0 tx=0).stop 2024-09-11T23:10:01.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:10:01.568+0000 7fc344012640 1 -- 172.21.15.26:0/1823375458 shutdown_connections 2024-09-11T23:10:01.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:10:01.568+0000 7fc344012640 1 --2- 172.21.15.26:0/1823375458 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fc3280390c0 0x7fc32803b580 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:10:01.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:10:01.568+0000 7fc344012640 1 --2- 172.21.15.26:0/1823375458 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc33c1027d0 0x7fc33c197170 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:10:01.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:10:01.568+0000 7fc344012640 1 -- 172.21.15.26:0/1823375458 >> 172.21.15.26:0/1823375458 conn(0x7fc33c0fc190 msgr2=0x7fc33c103580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:10:01.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:10:01.568+0000 7fc344012640 1 -- 172.21.15.26:0/1823375458 shutdown_connections 2024-09-11T23:10:01.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:10:01.568+0000 7fc344012640 1 -- 172.21.15.26:0/1823375458 wait complete. 2024-09-11T23:10:01.896 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:01 smithi026 ceph-mon[31172]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:10:01.896 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:01 smithi026 ceph-mon[31172]: from='client.? 172.21.15.26:0/1823375458' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-11T23:10:02.182 INFO:tasks.cephadm:Adding mon.a on smithi026 2024-09-11T23:10:02.182 INFO:tasks.cephadm:Adding mon.b on smithi076 2024-09-11T23:10:02.182 INFO:tasks.cephadm:Adding mon.c on smithi136 2024-09-11T23:10:02.182 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph orch apply mon '3;smithi026:172.21.15.26=a;smithi076:172.21.15.76=b;smithi136:172.21.15.136=c' 2024-09-11T23:10:02.445 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-11T23:10:02.491 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-11T23:10:03.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:02 smithi026 ceph-mon[31172]: from='client.? 172.21.15.26:0/1823375458' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-11T23:10:03.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:02 smithi026 ceph-mon[31172]: osdmap e4: 0 total, 0 up, 0 in 2024-09-11T23:10:03.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:02 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:10:03.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:02 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:10:03.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:02 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:10:03.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:02 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:10:03.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:02 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi136", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:10:03.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:02 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:10:03.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:02 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:10:03.037 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:03.016+0000 7fd2e41e9640 1 -- 172.21.15.136:0/4168844829 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2dc1015e0 msgr2=0x7fd2dc1019e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:10:03.037 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:03.016+0000 7fd2e41e9640 1 --2- 172.21.15.136:0/4168844829 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2dc1015e0 0x7fd2dc1019e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd2d8009880 tx=0x7fd2d802a5d0 comp rx=0 tx=0).stop 2024-09-11T23:10:03.037 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:03.016+0000 7fd2e41e9640 1 -- 172.21.15.136:0/4168844829 shutdown_connections 2024-09-11T23:10:03.037 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:03.016+0000 7fd2e41e9640 1 --2- 172.21.15.136:0/4168844829 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2dc1015e0 0x7fd2dc1019e0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:10:03.038 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:03.016+0000 7fd2e41e9640 1 -- 172.21.15.136:0/4168844829 >> 172.21.15.136:0/4168844829 conn(0x7fd2dc0fcd70 msgr2=0x7fd2dc0ff1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:10:03.038 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:03.016+0000 7fd2e41e9640 1 -- 172.21.15.136:0/4168844829 shutdown_connections 2024-09-11T23:10:03.038 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:03.016+0000 7fd2e41e9640 1 -- 172.21.15.136:0/4168844829 wait complete. 2024-09-11T23:10:03.038 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:03.017+0000 7fd2e41e9640 1 Processor -- start 2024-09-11T23:10:03.038 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:03.017+0000 7fd2e41e9640 1 -- start start 2024-09-11T23:10:03.038 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:03.017+0000 7fd2e41e9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2dc1015e0 0x7fd2dc196d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:10:03.038 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:03.017+0000 7fd2e41e9640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd2dc1076f0 con 0x7fd2dc1015e0 2024-09-11T23:10:03.038 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:03.018+0000 7fd2e1f85640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2dc1015e0 0x7fd2dc196d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:10:03.038 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:03.018+0000 7fd2e1f85640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2dc1015e0 0x7fd2dc196d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.136:53140/0 (socket says 172.21.15.136:53140) 2024-09-11T23:10:03.038 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:03.018+0000 7fd2e1f85640 1 -- 172.21.15.136:0/279987498 learned_addr learned my addr 172.21.15.136:0/279987498 (peer_addr_for_me v2:172.21.15.136:0/0) 2024-09-11T23:10:03.038 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:03.018+0000 7fd2e1f85640 1 -- 172.21.15.136:0/279987498 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2dc1972a0 con 0x7fd2dc1015e0 2024-09-11T23:10:03.038 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:03.018+0000 7fd2e1f85640 1 --2- 172.21.15.136:0/279987498 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2dc1015e0 0x7fd2dc196d60 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd2d8004f60 tx=0x7fd2d80048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:10:03.038 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:03.018+0000 7fd2d2ffd640 1 -- 172.21.15.136:0/279987498 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd2d803f070 con 0x7fd2dc1015e0 2024-09-11T23:10:03.039 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:03.018+0000 7fd2e41e9640 1 -- 172.21.15.136:0/279987498 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2dc197530 con 0x7fd2dc1015e0 2024-09-11T23:10:03.039 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:03.018+0000 7fd2d2ffd640 1 -- 172.21.15.136:0/279987498 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd2d8004d40 con 0x7fd2dc1015e0 2024-09-11T23:10:03.039 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:03.018+0000 7fd2d2ffd640 1 -- 172.21.15.136:0/279987498 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd2d8035030 con 0x7fd2dc1015e0 2024-09-11T23:10:03.039 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:03.018+0000 7fd2e41e9640 1 -- 172.21.15.136:0/279987498 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2dc19a220 con 0x7fd2dc1015e0 2024-09-11T23:10:03.039 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:03.019+0000 7fd2d2ffd640 1 -- 172.21.15.136:0/279987498 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fd2d8042070 con 0x7fd2dc1015e0 2024-09-11T23:10:03.039 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:03.019+0000 7fd2e41e9640 1 -- 172.21.15.136:0/279987498 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2dc107e70 con 0x7fd2dc1015e0 2024-09-11T23:10:03.039 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:03.019+0000 7fd2d2ffd640 1 --2- 172.21.15.136:0/279987498 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fd2c80390c0 0x7fd2c803b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:10:03.039 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:03.020+0000 7fd2d2ffd640 1 -- 172.21.15.136:0/279987498 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fd2d806cdb0 con 0x7fd2dc1015e0 2024-09-11T23:10:03.039 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:03.020+0000 7fd2e1784640 1 --2- 172.21.15.136:0/279987498 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fd2c80390c0 0x7fd2c803b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:10:03.040 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:03.020+0000 7fd2e1784640 1 --2- 172.21.15.136:0/279987498 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fd2c80390c0 0x7fd2c803b580 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fd2cc0098c0 tx=0x7fd2cc006db0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:10:03.040 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:03.024+0000 7fd2d2ffd640 1 -- 172.21.15.136:0/279987498 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd2d8042360 con 0x7fd2dc1015e0 2024-09-11T23:10:03.209 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:03.209+0000 7fd2e41e9640 1 -- 172.21.15.136:0/279987498 --> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi026:172.21.15.26=a;smithi076:172.21.15.76=b;smithi136:172.21.15.136=c", "target": ["mon-mgr", ""]}) -- 0x7fd2dc02d8f0 con 0x7fd2c80390c0 2024-09-11T23:10:03.215 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:03.215+0000 7fd2d2ffd640 1 -- 172.21.15.136:0/279987498 <== mgr.14148 v2:172.21.15.26:6800/110833859 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fd2dc02d8f0 con 0x7fd2c80390c0 2024-09-11T23:10:03.215 INFO:teuthology.orchestra.run.smithi136.stdout:Scheduled mon update... 2024-09-11T23:10:03.218 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:03.217+0000 7fd2e41e9640 1 -- 172.21.15.136:0/279987498 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fd2c80390c0 msgr2=0x7fd2c803b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:10:03.218 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:03.217+0000 7fd2e41e9640 1 --2- 172.21.15.136:0/279987498 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fd2c80390c0 0x7fd2c803b580 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fd2cc0098c0 tx=0x7fd2cc006db0 comp rx=0 tx=0).stop 2024-09-11T23:10:03.218 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:03.217+0000 7fd2e41e9640 1 -- 172.21.15.136:0/279987498 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2dc1015e0 msgr2=0x7fd2dc196d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:10:03.218 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:03.217+0000 7fd2e41e9640 1 --2- 172.21.15.136:0/279987498 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2dc1015e0 0x7fd2dc196d60 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd2d8004f60 tx=0x7fd2d80048f0 comp rx=0 tx=0).stop 2024-09-11T23:10:03.218 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:03.217+0000 7fd2e41e9640 1 -- 172.21.15.136:0/279987498 shutdown_connections 2024-09-11T23:10:03.218 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:03.217+0000 7fd2e41e9640 1 --2- 172.21.15.136:0/279987498 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fd2c80390c0 0x7fd2c803b580 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:10:03.218 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:03.217+0000 7fd2e41e9640 1 --2- 172.21.15.136:0/279987498 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2dc1015e0 0x7fd2dc196d60 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:10:03.218 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:03.217+0000 7fd2e41e9640 1 -- 172.21.15.136:0/279987498 >> 172.21.15.136:0/279987498 conn(0x7fd2dc0fcd70 msgr2=0x7fd2dc0fd7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:10:03.218 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:03.217+0000 7fd2e41e9640 1 -- 172.21.15.136:0/279987498 shutdown_connections 2024-09-11T23:10:03.218 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:03.217+0000 7fd2e41e9640 1 -- 172.21.15.136:0/279987498 wait complete. 2024-09-11T23:10:03.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:03 smithi026 ceph-mon[31172]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:10:03.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:03 smithi026 ceph-mon[31172]: Updating smithi136:/etc/ceph/ceph.conf 2024-09-11T23:10:03.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:03 smithi026 ceph-mon[31172]: Updating smithi136:/var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T23:10:03.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:03 smithi026 ceph-mon[31172]: Updating smithi136:/etc/ceph/ceph.client.admin.keyring 2024-09-11T23:10:03.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:03 smithi026 ceph-mon[31172]: Updating smithi136:/var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T23:10:03.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:03 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:10:03.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:03 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:10:03.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:03 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:10:03.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:03 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:10:03.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:03 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:10:03.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:03 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:10:03.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:03 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:10:03.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:03 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:10:03.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:03 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T23:10:03.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:03 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:10:03.794 DEBUG:teuthology.orchestra.run.smithi076:mon.b> sudo journalctl -f -n 0 -u ceph-af7e6fc8-7092-11ef-bcea-c7b262605968@mon.b.service 2024-09-11T23:10:03.797 DEBUG:teuthology.orchestra.run.smithi136:mon.c> sudo journalctl -f -n 0 -u ceph-af7e6fc8-7092-11ef-bcea-c7b262605968@mon.c.service 2024-09-11T23:10:03.800 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-11T23:10:03.800 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-11T23:10:04.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:04 smithi026 ceph-mon[31172]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi026:172.21.15.26=a;smithi076:172.21.15.76=b;smithi136:172.21.15.136=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:10:04.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:04 smithi026 ceph-mon[31172]: Saving service mon spec with placement smithi026:172.21.15.26=a;smithi076:172.21.15.76=b;smithi136:172.21.15.136=c;count:3 2024-09-11T23:10:04.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:04 smithi026 ceph-mon[31172]: Deploying daemon mon.c on smithi136 2024-09-11T23:10:04.946 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T23:10:06.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:05 smithi026 ceph-mon[31172]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:10:06.214 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:06.214+0000 7f0b99aea640 1 -- 172.21.15.136:0/3288599645 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0b94102750 msgr2=0x7f0b94102b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:10:06.269 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:06.214+0000 7f0b99aea640 1 --2- 172.21.15.136:0/3288599645 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0b94102750 0x7f0b94102b30 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f0b84009880 tx=0x7f0b8402a670 comp rx=0 tx=0).stop 2024-09-11T23:10:06.270 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:06.214+0000 7f0b99aea640 1 -- 172.21.15.136:0/3288599645 shutdown_connections 2024-09-11T23:10:06.270 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:06.214+0000 7f0b99aea640 1 --2- 172.21.15.136:0/3288599645 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0b94102750 0x7f0b94102b30 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:10:06.270 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:06.214+0000 7f0b99aea640 1 -- 172.21.15.136:0/3288599645 >> 172.21.15.136:0/3288599645 conn(0x7f0b940fc1b0 msgr2=0x7f0b940fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:10:06.270 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:06.214+0000 7f0b99aea640 1 -- 172.21.15.136:0/3288599645 shutdown_connections 2024-09-11T23:10:06.270 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:06.214+0000 7f0b99aea640 1 -- 172.21.15.136:0/3288599645 wait complete. 2024-09-11T23:10:06.273 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:06.215+0000 7f0b99aea640 1 Processor -- start 2024-09-11T23:10:06.273 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:06.215+0000 7f0b99aea640 1 -- start start 2024-09-11T23:10:06.273 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:06.215+0000 7f0b99aea640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0b94102750 0x7f0b941971c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:10:06.273 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:06.215+0000 7f0b99aea640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f0b9410bcc0 con 0x7f0b94102750 2024-09-11T23:10:06.273 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:06.216+0000 7f0b937fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0b94102750 0x7f0b941971c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:10:06.273 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:06.216+0000 7f0b937fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0b94102750 0x7f0b941971c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.136:53160/0 (socket says 172.21.15.136:53160) 2024-09-11T23:10:06.273 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:06.216+0000 7f0b937fe640 1 -- 172.21.15.136:0/1759833330 learned_addr learned my addr 172.21.15.136:0/1759833330 (peer_addr_for_me v2:172.21.15.136:0/0) 2024-09-11T23:10:06.273 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:06.216+0000 7f0b937fe640 1 -- 172.21.15.136:0/1759833330 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b94197700 con 0x7f0b94102750 2024-09-11T23:10:06.274 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:06.216+0000 7f0b937fe640 1 --2- 172.21.15.136:0/1759833330 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0b94102750 0x7f0b941971c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f0b84009230 tx=0x7f0b84003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:10:06.274 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:06.216+0000 7f0b90ff9640 1 -- 172.21.15.136:0/1759833330 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0b8403f070 con 0x7f0b94102750 2024-09-11T23:10:06.274 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:06.216+0000 7f0b99aea640 1 -- 172.21.15.136:0/1759833330 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b94197990 con 0x7f0b94102750 2024-09-11T23:10:06.274 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:06.216+0000 7f0b90ff9640 1 -- 172.21.15.136:0/1759833330 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b8403bc80 con 0x7f0b94102750 2024-09-11T23:10:06.274 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:06.217+0000 7f0b90ff9640 1 -- 172.21.15.136:0/1759833330 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0b84049060 con 0x7f0b94102750 2024-09-11T23:10:06.274 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:06.217+0000 7f0b99aea640 1 -- 172.21.15.136:0/1759833330 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b94197df0 con 0x7f0b94102750 2024-09-11T23:10:06.274 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:06.218+0000 7f0b90ff9640 1 -- 172.21.15.136:0/1759833330 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f0b84042070 con 0x7f0b94102750 2024-09-11T23:10:06.274 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:06.218+0000 7f0b99aea640 1 -- 172.21.15.136:0/1759833330 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b9406a7b0 con 0x7f0b94102750 2024-09-11T23:10:06.275 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:06.218+0000 7f0b90ff9640 1 --2- 172.21.15.136:0/1759833330 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f0b7c039060 0x7f0b7c03b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:10:06.275 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:06.218+0000 7f0b90ff9640 1 -- 172.21.15.136:0/1759833330 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f0b8406d110 con 0x7f0b94102750 2024-09-11T23:10:06.275 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:06.218+0000 7f0b92ffd640 1 --2- 172.21.15.136:0/1759833330 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f0b7c039060 0x7f0b7c03b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:10:06.275 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:06.218+0000 7f0b92ffd640 1 --2- 172.21.15.136:0/1759833330 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f0b7c039060 0x7f0b7c03b520 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f0b880098c0 tx=0x7f0b88006db0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:10:06.275 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:06.223+0000 7f0b90ff9640 1 -- 172.21.15.136:0/1759833330 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0b84031d60 con 0x7f0b94102750 2024-09-11T23:10:06.489 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:06.487+0000 7f0b99aea640 1 -- 172.21.15.136:0/1759833330 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f0b94198580 con 0x7f0b94102750 2024-09-11T23:10:06.489 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:06.488+0000 7f0b90ff9640 1 -- 172.21.15.136:0/1759833330 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+745 (secure 0 0 0) 0x7f0b840314c0 con 0x7f0b94102750 2024-09-11T23:10:06.489 INFO:teuthology.orchestra.run.smithi136.stderr:dumped monmap epoch 1 2024-09-11T23:10:06.490 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-09-11T23:10:06.490 INFO:teuthology.orchestra.run.smithi136.stdout:{"epoch":1,"fsid":"af7e6fc8-7092-11ef-bcea-c7b262605968","modified":"2024-09-11T23:08:37.446176Z","created":"2024-09-11T23:08:37.446176Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:3300","nonce":0},{"type":"v1","addr":"172.21.15.26:6789","nonce":0}]},"addr":"172.21.15.26:6789/0","public_addr":"172.21.15.26:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-11T23:10:06.491 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:06.491+0000 7f0b99aea640 1 -- 172.21.15.136:0/1759833330 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f0b7c039060 msgr2=0x7f0b7c03b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:10:06.491 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:06.491+0000 7f0b99aea640 1 --2- 172.21.15.136:0/1759833330 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f0b7c039060 0x7f0b7c03b520 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f0b880098c0 tx=0x7f0b88006db0 comp rx=0 tx=0).stop 2024-09-11T23:10:06.491 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:06.491+0000 7f0b99aea640 1 -- 172.21.15.136:0/1759833330 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0b94102750 msgr2=0x7f0b941971c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:10:06.491 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:06.491+0000 7f0b99aea640 1 --2- 172.21.15.136:0/1759833330 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0b94102750 0x7f0b941971c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f0b84009230 tx=0x7f0b84003d40 comp rx=0 tx=0).stop 2024-09-11T23:10:06.492 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:06.491+0000 7f0b99aea640 1 -- 172.21.15.136:0/1759833330 shutdown_connections 2024-09-11T23:10:06.492 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:06.491+0000 7f0b99aea640 1 --2- 172.21.15.136:0/1759833330 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f0b7c039060 0x7f0b7c03b520 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:10:06.492 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:06.491+0000 7f0b99aea640 1 --2- 172.21.15.136:0/1759833330 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0b94102750 0x7f0b941971c0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:10:06.492 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:06.491+0000 7f0b99aea640 1 -- 172.21.15.136:0/1759833330 >> 172.21.15.136:0/1759833330 conn(0x7f0b940fc1b0 msgr2=0x7f0b94103500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:10:06.492 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:06.491+0000 7f0b99aea640 1 -- 172.21.15.136:0/1759833330 shutdown_connections 2024-09-11T23:10:06.492 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:06.491+0000 7f0b99aea640 1 -- 172.21.15.136:0/1759833330 wait complete. 2024-09-11T23:10:07.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:06 smithi026 ceph-mon[31172]: from='client.? 172.21.15.136:0/1759833330' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-11T23:10:07.767 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:07 smithi136 ceph-mon[35121]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-09-11T23:10:08.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:07 smithi026 ceph-mon[31172]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:10:08.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:07 smithi136 ceph-mon[35121]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-09-11T23:10:08.240 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:07 smithi136 ceph-mon[35121]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-09-11T23:10:08.240 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:07 smithi136 ceph-mon[35121]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-09-11T23:10:08.240 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:07 smithi136 ceph-mon[35121]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-09-11T23:10:08.240 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:07 smithi136 ceph-mon[35121]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-09-11T23:10:08.240 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:07 smithi136 ceph-mon[35121]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-11T23:10:08.240 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:07 smithi136 ceph-mon[35121]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-11T23:10:08.240 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:07 smithi136 ceph-mon[35121]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-11T23:10:08.240 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:07 smithi136 ceph-mon[35121]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-11T23:10:08.240 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:07 smithi136 ceph-mon[35121]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi026:172.21.15.26=a;smithi076:172.21.15.76=b;smithi136:172.21.15.136=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:10:08.240 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:07 smithi136 ceph-mon[35121]: Saving service mon spec with placement smithi026:172.21.15.26=a;smithi076:172.21.15.76=b;smithi136:172.21.15.136=c;count:3 2024-09-11T23:10:08.240 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:07 smithi136 ceph-mon[35121]: Deploying daemon mon.c on smithi136 2024-09-11T23:10:08.240 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:07 smithi136 ceph-mon[35121]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:10:08.241 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:07 smithi136 ceph-mon[35121]: from='client.? 172.21.15.136:0/1759833330' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-11T23:10:08.241 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:07 smithi136 ceph-mon[35121]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:10:08.241 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:07 smithi136 ceph-mon[35121]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-09-11T23:10:08.241 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:07 smithi136 bash[35101]: a454789c9141578fc927a589f0159c277bd6b2fcf3e31354e4199cf1f2d31883 2024-09-11T23:10:08.241 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:07 smithi136 systemd[1]: Started Ceph mon.c for af7e6fc8-7092-11ef-bcea-c7b262605968. 2024-09-11T23:10:08.315 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-11T23:10:08.315 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-11T23:10:08.562 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T23:10:13.240 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:12 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T23:10:13.240 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:12 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T23:10:13.240 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:12 smithi136 ceph-mon[35121]: mon.a calling monitor election 2024-09-11T23:10:13.240 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:12 smithi136 ceph-mon[35121]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:10:13.240 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:12 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T23:10:13.240 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:12 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T23:10:13.240 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:12 smithi136 ceph-mon[35121]: mon.c calling monitor election 2024-09-11T23:10:13.241 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:12 smithi136 ceph-mon[35121]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:10:13.241 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:12 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T23:10:13.241 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:12 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T23:10:13.241 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:12 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T23:10:13.241 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:12 smithi136 ceph-mon[35121]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-11T23:10:13.241 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:12 smithi136 ceph-mon[35121]: monmap epoch 2 2024-09-11T23:10:13.241 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:12 smithi136 ceph-mon[35121]: fsid af7e6fc8-7092-11ef-bcea-c7b262605968 2024-09-11T23:10:13.241 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:12 smithi136 ceph-mon[35121]: last_changed 2024-09-11T23:10:07.776440+0000 2024-09-11T23:10:13.241 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:12 smithi136 ceph-mon[35121]: created 2024-09-11T23:08:37.446176+0000 2024-09-11T23:10:13.241 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:12 smithi136 ceph-mon[35121]: min_mon_release 19 (squid) 2024-09-11T23:10:13.241 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:12 smithi136 ceph-mon[35121]: election_strategy: 1 2024-09-11T23:10:13.242 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:12 smithi136 ceph-mon[35121]: 0: [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] mon.a 2024-09-11T23:10:13.242 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:12 smithi136 ceph-mon[35121]: 1: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.c 2024-09-11T23:10:13.242 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:12 smithi136 ceph-mon[35121]: fsmap 2024-09-11T23:10:13.242 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:12 smithi136 ceph-mon[35121]: osdmap e4: 0 total, 0 up, 0 in 2024-09-11T23:10:13.242 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:12 smithi136 ceph-mon[35121]: mgrmap e13: x(active, since 38s) 2024-09-11T23:10:13.242 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:12 smithi136 ceph-mon[35121]: overall HEALTH_OK 2024-09-11T23:10:13.242 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:12 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:10:13.242 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:12 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:10:13.242 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:12 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:10:13.242 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:12 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T23:10:13.243 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:12 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:10:13.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:12 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T23:10:13.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:12 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T23:10:13.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:12 smithi026 ceph-mon[31172]: mon.a calling monitor election 2024-09-11T23:10:13.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:12 smithi026 ceph-mon[31172]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:10:13.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:12 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T23:10:13.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:12 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T23:10:13.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:12 smithi026 ceph-mon[31172]: mon.c calling monitor election 2024-09-11T23:10:13.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:12 smithi026 ceph-mon[31172]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:10:13.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:12 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T23:10:13.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:12 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T23:10:13.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:12 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T23:10:13.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:12 smithi026 ceph-mon[31172]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-11T23:10:13.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:12 smithi026 ceph-mon[31172]: monmap epoch 2 2024-09-11T23:10:13.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:12 smithi026 ceph-mon[31172]: fsid af7e6fc8-7092-11ef-bcea-c7b262605968 2024-09-11T23:10:13.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:12 smithi026 ceph-mon[31172]: last_changed 2024-09-11T23:10:07.776440+0000 2024-09-11T23:10:13.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:12 smithi026 ceph-mon[31172]: created 2024-09-11T23:08:37.446176+0000 2024-09-11T23:10:13.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:12 smithi026 ceph-mon[31172]: min_mon_release 19 (squid) 2024-09-11T23:10:13.253 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:12 smithi026 ceph-mon[31172]: election_strategy: 1 2024-09-11T23:10:13.253 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:12 smithi026 ceph-mon[31172]: 0: [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] mon.a 2024-09-11T23:10:13.253 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:12 smithi026 ceph-mon[31172]: 1: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.c 2024-09-11T23:10:13.253 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:12 smithi026 ceph-mon[31172]: fsmap 2024-09-11T23:10:13.253 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:12 smithi026 ceph-mon[31172]: osdmap e4: 0 total, 0 up, 0 in 2024-09-11T23:10:13.253 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:12 smithi026 ceph-mon[31172]: mgrmap e13: x(active, since 38s) 2024-09-11T23:10:13.253 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:12 smithi026 ceph-mon[31172]: overall HEALTH_OK 2024-09-11T23:10:13.253 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:12 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:10:13.253 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:12 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:10:13.253 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:12 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:10:13.253 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:12 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T23:10:13.253 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:12 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:10:13.362 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:13.361+0000 7f15e4c16640 1 -- 172.21.15.136:0/1906377519 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15e0107660 msgr2=0x7f15cc005580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:10:13.362 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:13.361+0000 7f15e4c16640 1 --2- 172.21.15.136:0/1906377519 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15e0107660 0x7f15cc005580 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f15d0012c60 tx=0x7f15d002c620 comp rx=0 tx=0).stop 2024-09-11T23:10:13.362 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:13.361+0000 7f15e4c16640 1 -- 172.21.15.136:0/1906377519 shutdown_connections 2024-09-11T23:10:13.362 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:13.361+0000 7f15e4c16640 1 --2- 172.21.15.136:0/1906377519 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15e0107660 0x7f15cc005580 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:10:13.363 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:13.361+0000 7f15e4c16640 1 -- 172.21.15.136:0/1906377519 >> 172.21.15.136:0/1906377519 conn(0x7f15e00fd320 msgr2=0x7f15e00ff740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:10:13.363 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:13.361+0000 7f15e4c16640 1 -- 172.21.15.136:0/1906377519 shutdown_connections 2024-09-11T23:10:13.363 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:13.361+0000 7f15e4c16640 1 -- 172.21.15.136:0/1906377519 wait complete. 2024-09-11T23:10:13.365 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:13.362+0000 7f15e4c16640 1 Processor -- start 2024-09-11T23:10:13.365 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:13.363+0000 7f15e4c16640 1 -- start start 2024-09-11T23:10:13.365 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:13.363+0000 7f15e4c16640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15e0107660 0x7f15e00744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:10:13.365 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:13.363+0000 7f15e4c16640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f15e0074a30 0x7f15e0074e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:10:13.365 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:13.363+0000 7f15e4c16640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f15e0109780 con 0x7f15e0107660 2024-09-11T23:10:13.365 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:13.363+0000 7f15e4c16640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f15e0109600 con 0x7f15e0074a30 2024-09-11T23:10:13.366 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:13.363+0000 7f15d5d9b640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f15e0074a30 0x7f15e0074e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:10:13.366 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:13.363+0000 7f15d5d9b640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f15e0074a30 0x7f15e0074e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.136:50436/0 (socket says 172.21.15.136:50436) 2024-09-11T23:10:13.366 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:13.363+0000 7f15d5d9b640 1 -- 172.21.15.136:0/290518562 learned_addr learned my addr 172.21.15.136:0/290518562 (peer_addr_for_me v2:172.21.15.136:0/0) 2024-09-11T23:10:13.366 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:13.363+0000 7f15d5d9b640 1 -- 172.21.15.136:0/290518562 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f15e0074a30 msgr2=0x7f15e0074e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-09-11T23:10:13.366 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:13.363+0000 7f15d5d9b640 1 -- 172.21.15.136:0/290518562 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f15e0074a30 msgr2=0x7f15e0074e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-11T23:10:13.366 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:13.363+0000 7f15d5d9b640 1 --2- 172.21.15.136:0/290518562 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f15e0074a30 0x7f15e0074e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-11T23:10:13.366 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:13.363+0000 7f15d5d9b640 1 --2- 172.21.15.136:0/290518562 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f15e0074a30 0x7f15e0074e90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-11T23:10:13.366 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:13.363+0000 7f15de59c640 1 --2- 172.21.15.136:0/290518562 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15e0107660 0x7f15e00744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:10:13.366 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:13.364+0000 7f15de59c640 1 -- 172.21.15.136:0/290518562 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f15e0074a30 msgr2=0x7f15e0074e90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-11T23:10:13.366 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:13.364+0000 7f15de59c640 1 --2- 172.21.15.136:0/290518562 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f15e0074a30 0x7f15e0074e90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:10:13.366 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:13.364+0000 7f15de59c640 1 -- 172.21.15.136:0/290518562 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15e00763e0 con 0x7f15e0107660 2024-09-11T23:10:13.367 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:13.364+0000 7f15de59c640 1 --2- 172.21.15.136:0/290518562 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15e0107660 0x7f15e00744f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f15d0033cb0 tx=0x7f15d0033ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:10:13.367 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:13.364+0000 7f15d7fff640 1 -- 172.21.15.136:0/290518562 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f15d0004890 con 0x7f15e0107660 2024-09-11T23:10:13.367 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:13.364+0000 7f15d7fff640 1 -- 172.21.15.136:0/290518562 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f15d003d460 con 0x7f15e0107660 2024-09-11T23:10:13.367 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:13.364+0000 7f15e4c16640 1 -- 172.21.15.136:0/290518562 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15e01a3480 con 0x7f15e0107660 2024-09-11T23:10:13.367 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:13.364+0000 7f15d7fff640 1 -- 172.21.15.136:0/290518562 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f15d003c5c0 con 0x7f15e0107660 2024-09-11T23:10:13.367 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:13.365+0000 7f15e4c16640 1 -- 172.21.15.136:0/290518562 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15e01a3a10 con 0x7f15e0107660 2024-09-11T23:10:13.368 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:13.366+0000 7f15d7fff640 1 -- 172.21.15.136:0/290518562 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f15d0044070 con 0x7f15e0107660 2024-09-11T23:10:13.368 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:13.366+0000 7f15d7fff640 1 --2- 172.21.15.136:0/290518562 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f15c0038e50 0x7f15c003b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:10:13.368 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:13.366+0000 7f15e4c16640 1 -- 172.21.15.136:0/290518562 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15e0102da0 con 0x7f15e0107660 2024-09-11T23:10:13.369 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:13.366+0000 7f15d7fff640 1 -- 172.21.15.136:0/290518562 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f15d0042070 con 0x7f15e0107660 2024-09-11T23:10:13.369 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:13.366+0000 7f15d5d9b640 1 --2- 172.21.15.136:0/290518562 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f15c0038e50 0x7f15c003b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:10:13.369 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:13.367+0000 7f15d5d9b640 1 --2- 172.21.15.136:0/290518562 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f15c0038e50 0x7f15c003b310 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f15c8009100 tx=0x7f15c8008040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:10:13.372 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:13.372+0000 7f15d7fff640 1 -- 172.21.15.136:0/290518562 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f15d004d550 con 0x7f15e0107660 2024-09-11T23:10:13.603 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:13.603+0000 7f15e4c16640 1 -- 172.21.15.136:0/290518562 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f15e010a8b0 con 0x7f15e0107660 2024-09-11T23:10:13.603 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:13.603+0000 7f15d7fff640 1 -- 172.21.15.136:0/290518562 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7f15d004d2f0 con 0x7f15e0107660 2024-09-11T23:10:13.604 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-09-11T23:10:13.604 INFO:teuthology.orchestra.run.smithi136.stdout:{"epoch":2,"fsid":"af7e6fc8-7092-11ef-bcea-c7b262605968","modified":"2024-09-11T23:10:07.776440Z","created":"2024-09-11T23:08:37.446176Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:3300","nonce":0},{"type":"v1","addr":"172.21.15.26:6789","nonce":0}]},"addr":"172.21.15.26:6789/0","public_addr":"172.21.15.26:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:3300","nonce":0},{"type":"v1","addr":"172.21.15.136:6789","nonce":0}]},"addr":"172.21.15.136:6789/0","public_addr":"172.21.15.136:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-11T23:10:13.604 INFO:teuthology.orchestra.run.smithi136.stderr:dumped monmap epoch 2 2024-09-11T23:10:13.606 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:13.606+0000 7f15e4c16640 1 -- 172.21.15.136:0/290518562 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f15c0038e50 msgr2=0x7f15c003b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:10:13.606 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:13.606+0000 7f15e4c16640 1 --2- 172.21.15.136:0/290518562 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f15c0038e50 0x7f15c003b310 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f15c8009100 tx=0x7f15c8008040 comp rx=0 tx=0).stop 2024-09-11T23:10:13.606 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:13.606+0000 7f15e4c16640 1 -- 172.21.15.136:0/290518562 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15e0107660 msgr2=0x7f15e00744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:10:13.606 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:13.606+0000 7f15e4c16640 1 --2- 172.21.15.136:0/290518562 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15e0107660 0x7f15e00744f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f15d0033cb0 tx=0x7f15d0033ce0 comp rx=0 tx=0).stop 2024-09-11T23:10:13.607 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:13.606+0000 7f15e4c16640 1 -- 172.21.15.136:0/290518562 shutdown_connections 2024-09-11T23:10:13.607 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:13.606+0000 7f15e4c16640 1 --2- 172.21.15.136:0/290518562 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f15c0038e50 0x7f15c003b310 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:10:13.607 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:13.606+0000 7f15e4c16640 1 --2- 172.21.15.136:0/290518562 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f15e0074a30 0x7f15e0074e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:10:13.607 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:13.606+0000 7f15e4c16640 1 --2- 172.21.15.136:0/290518562 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15e0107660 0x7f15e00744f0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:10:13.607 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:13.606+0000 7f15e4c16640 1 -- 172.21.15.136:0/290518562 >> 172.21.15.136:0/290518562 conn(0x7f15e00fd320 msgr2=0x7f15e00ff2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:10:13.607 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:13.606+0000 7f15e4c16640 1 -- 172.21.15.136:0/290518562 shutdown_connections 2024-09-11T23:10:13.607 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:13.606+0000 7f15e4c16640 1 -- 172.21.15.136:0/290518562 wait complete. 2024-09-11T23:10:13.927 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:13 smithi136 ceph-mon[35121]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:10:13.927 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:13 smithi136 ceph-mon[35121]: Deploying daemon mon.b on smithi076 2024-09-11T23:10:13.927 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:13 smithi136 ceph-mon[35121]: from='client.? 172.21.15.136:0/290518562' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-11T23:10:13.927 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:13 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T23:10:14.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:13 smithi026 ceph-mon[31172]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:10:14.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:13 smithi026 ceph-mon[31172]: Deploying daemon mon.b on smithi076 2024-09-11T23:10:14.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:13 smithi026 ceph-mon[31172]: from='client.? 172.21.15.136:0/290518562' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-11T23:10:14.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:13 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T23:10:15.257 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-11T23:10:15.257 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-11T23:10:15.509 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T23:10:15.930 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:15 smithi136 ceph-mon[35121]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:10:15.953 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:15.953+0000 7f28583ab640 1 -- 172.21.15.136:0/2392795399 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2850106680 msgr2=0x7f2850106a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:10:15.953 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:15.953+0000 7f28583ab640 1 --2- 172.21.15.136:0/2392795399 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2850106680 0x7f2850106a60 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f2840009880 tx=0x7f284002a5d0 comp rx=0 tx=0).stop 2024-09-11T23:10:15.953 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:15.953+0000 7f28583ab640 1 -- 172.21.15.136:0/2392795399 shutdown_connections 2024-09-11T23:10:15.953 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:15.953+0000 7f28583ab640 1 --2- 172.21.15.136:0/2392795399 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2850106680 0x7f2850106a60 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:10:15.954 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:15.953+0000 7f28583ab640 1 -- 172.21.15.136:0/2392795399 >> 172.21.15.136:0/2392795399 conn(0x7f28500fc2d0 msgr2=0x7f28500fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:10:15.954 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:15.953+0000 7f28583ab640 1 -- 172.21.15.136:0/2392795399 shutdown_connections 2024-09-11T23:10:15.954 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:15.953+0000 7f28583ab640 1 -- 172.21.15.136:0/2392795399 wait complete. 2024-09-11T23:10:15.956 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:15.954+0000 7f28583ab640 1 Processor -- start 2024-09-11T23:10:15.956 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:15.954+0000 7f28583ab640 1 -- start start 2024-09-11T23:10:15.956 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:15.954+0000 7f28583ab640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2850106680 0x7f2850196310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:10:15.956 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:15.954+0000 7f28583ab640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2850196850 0x7f285019abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:10:15.956 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:15.954+0000 7f28583ab640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f285010acb0 con 0x7f2850196850 2024-09-11T23:10:15.957 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:15.954+0000 7f28583ab640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f285010ab30 con 0x7f2850106680 2024-09-11T23:10:15.957 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:15.954+0000 7f2856147640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2850106680 0x7f2850196310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:10:15.957 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:15.954+0000 7f2856147640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2850106680 0x7f2850196310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.136:50440/0 (socket says 172.21.15.136:50440) 2024-09-11T23:10:15.957 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:15.954+0000 7f2856147640 1 -- 172.21.15.136:0/1759814458 learned_addr learned my addr 172.21.15.136:0/1759814458 (peer_addr_for_me v2:172.21.15.136:0/0) 2024-09-11T23:10:15.957 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:15.954+0000 7f2855946640 1 --2- 172.21.15.136:0/1759814458 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2850196850 0x7f285019abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:10:15.957 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:15.954+0000 7f2856147640 1 -- 172.21.15.136:0/1759814458 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2850106680 msgr2=0x7f2850196310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-09-11T23:10:15.957 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:15.954+0000 7f2856147640 1 -- 172.21.15.136:0/1759814458 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2850106680 msgr2=0x7f2850196310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-11T23:10:15.957 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:15.954+0000 7f2856147640 1 --2- 172.21.15.136:0/1759814458 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2850106680 0x7f2850196310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-11T23:10:15.957 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:15.954+0000 7f2856147640 1 --2- 172.21.15.136:0/1759814458 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2850106680 0x7f2850196310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-11T23:10:15.957 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:15.955+0000 7f2855946640 1 -- 172.21.15.136:0/1759814458 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2850106680 msgr2=0x7f2850196310 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-11T23:10:15.958 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:15.955+0000 7f2855946640 1 --2- 172.21.15.136:0/1759814458 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2850106680 0x7f2850196310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:10:15.958 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:15.955+0000 7f2855946640 1 -- 172.21.15.136:0/1759814458 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2850196e70 con 0x7f2850196850 2024-09-11T23:10:15.958 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:15.955+0000 7f2855946640 1 --2- 172.21.15.136:0/1759814458 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2850196850 0x7f285019abe0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f284400b9b0 tx=0x7f284400be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:10:15.958 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:15.955+0000 7f284f7fe640 1 -- 172.21.15.136:0/1759814458 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f284400c920 con 0x7f2850196850 2024-09-11T23:10:15.958 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:15.955+0000 7f28583ab640 1 -- 172.21.15.136:0/1759814458 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f285019b120 con 0x7f2850196850 2024-09-11T23:10:15.958 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:15.955+0000 7f284f7fe640 1 -- 172.21.15.136:0/1759814458 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2844014440 con 0x7f2850196850 2024-09-11T23:10:15.958 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:15.955+0000 7f284f7fe640 1 -- 172.21.15.136:0/1759814458 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f2844012500 con 0x7f2850196850 2024-09-11T23:10:15.958 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:15.955+0000 7f28583ab640 1 -- 172.21.15.136:0/1759814458 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f285019b660 con 0x7f2850196850 2024-09-11T23:10:15.958 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:15.956+0000 7f28583ab640 1 -- 172.21.15.136:0/1759814458 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2850101cf0 con 0x7f2850196850 2024-09-11T23:10:15.958 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:15.956+0000 7f284f7fe640 1 -- 172.21.15.136:0/1759814458 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f284401c070 con 0x7f2850196850 2024-09-11T23:10:15.959 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:15.956+0000 7f284f7fe640 1 --2- 172.21.15.136:0/1759814458 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f2838038e00 0x7f283803b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:10:15.960 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:15.956+0000 7f284f7fe640 1 -- 172.21.15.136:0/1759814458 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f284404b960 con 0x7f2850196850 2024-09-11T23:10:15.960 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:15.957+0000 7f2856147640 1 --2- 172.21.15.136:0/1759814458 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f2838038e00 0x7f283803b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:10:15.960 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:15.957+0000 7f2856147640 1 --2- 172.21.15.136:0/1759814458 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f2838038e00 0x7f283803b2c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f28400096c0 tx=0x7f2840031400 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:10:15.962 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:15.962+0000 7f284f7fe640 1 -- 172.21.15.136:0/1759814458 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f284401b2c0 con 0x7f2850196850 2024-09-11T23:10:16.190 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:16.190+0000 7f28583ab640 1 -- 172.21.15.136:0/1759814458 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f285019b970 con 0x7f2850196850 2024-09-11T23:10:16.190 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:16.190+0000 7f284f7fe640 1 -- 172.21.15.136:0/1759814458 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7f2844010ca0 con 0x7f2850196850 2024-09-11T23:10:16.191 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-09-11T23:10:16.191 INFO:teuthology.orchestra.run.smithi136.stdout:{"epoch":2,"fsid":"af7e6fc8-7092-11ef-bcea-c7b262605968","modified":"2024-09-11T23:10:07.776440Z","created":"2024-09-11T23:08:37.446176Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:3300","nonce":0},{"type":"v1","addr":"172.21.15.26:6789","nonce":0}]},"addr":"172.21.15.26:6789/0","public_addr":"172.21.15.26:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:3300","nonce":0},{"type":"v1","addr":"172.21.15.136:6789","nonce":0}]},"addr":"172.21.15.136:6789/0","public_addr":"172.21.15.136:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-11T23:10:16.191 INFO:teuthology.orchestra.run.smithi136.stderr:dumped monmap epoch 2 2024-09-11T23:10:16.193 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:16.193+0000 7f28583ab640 1 -- 172.21.15.136:0/1759814458 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f2838038e00 msgr2=0x7f283803b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:10:16.193 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:16.193+0000 7f28583ab640 1 --2- 172.21.15.136:0/1759814458 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f2838038e00 0x7f283803b2c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f28400096c0 tx=0x7f2840031400 comp rx=0 tx=0).stop 2024-09-11T23:10:16.193 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:16.193+0000 7f28583ab640 1 -- 172.21.15.136:0/1759814458 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2850196850 msgr2=0x7f285019abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:10:16.193 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:16.193+0000 7f28583ab640 1 --2- 172.21.15.136:0/1759814458 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2850196850 0x7f285019abe0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f284400b9b0 tx=0x7f284400be70 comp rx=0 tx=0).stop 2024-09-11T23:10:16.193 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:16.193+0000 7f28583ab640 1 -- 172.21.15.136:0/1759814458 shutdown_connections 2024-09-11T23:10:16.193 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:16.193+0000 7f28583ab640 1 --2- 172.21.15.136:0/1759814458 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f2838038e00 0x7f283803b2c0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:10:16.193 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:16.193+0000 7f28583ab640 1 --2- 172.21.15.136:0/1759814458 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2850196850 0x7f285019abe0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:10:16.193 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:16.193+0000 7f28583ab640 1 --2- 172.21.15.136:0/1759814458 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2850106680 0x7f2850196310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:10:16.193 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:16.193+0000 7f28583ab640 1 -- 172.21.15.136:0/1759814458 >> 172.21.15.136:0/1759814458 conn(0x7f28500fc2d0 msgr2=0x7f28500fc6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:10:16.193 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:16.193+0000 7f28583ab640 1 -- 172.21.15.136:0/1759814458 shutdown_connections 2024-09-11T23:10:16.194 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:16.193+0000 7f28583ab640 1 -- 172.21.15.136:0/1759814458 wait complete. 2024-09-11T23:10:16.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:15 smithi026 ceph-mon[31172]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:10:16.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:16 smithi076 systemd[1]: Started Ceph mon.b for af7e6fc8-7092-11ef-bcea-c7b262605968. 2024-09-11T23:10:17.762 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-11T23:10:17.763 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-11T23:10:18.015 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T23:10:21.661 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:21.660+0000 7f2064791640 1 -- 172.21.15.136:0/898636774 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f205c102820 msgr2=0x7f2048005580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:10:21.661 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:21.660+0000 7f2064791640 1 --2- 172.21.15.136:0/898636774 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f205c102820 0x7f2048005580 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f2058009d90 tx=0x7f205802bef0 comp rx=0 tx=0).stop 2024-09-11T23:10:21.661 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:21.660+0000 7f2064791640 1 -- 172.21.15.136:0/898636774 shutdown_connections 2024-09-11T23:10:21.661 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:21.660+0000 7f2064791640 1 --2- 172.21.15.136:0/898636774 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f205c102820 0x7f2048005580 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:10:21.661 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:21.660+0000 7f2064791640 1 -- 172.21.15.136:0/898636774 >> 172.21.15.136:0/898636774 conn(0x7f205c0fc220 msgr2=0x7f205c0fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:10:21.662 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:21.661+0000 7f2064791640 1 -- 172.21.15.136:0/898636774 shutdown_connections 2024-09-11T23:10:21.662 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:21.661+0000 7f2064791640 1 -- 172.21.15.136:0/898636774 wait complete. 2024-09-11T23:10:21.664 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:21.661+0000 7f2064791640 1 Processor -- start 2024-09-11T23:10:21.664 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:21.661+0000 7f2064791640 1 -- start start 2024-09-11T23:10:21.664 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:21.662+0000 7f2064791640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f205c102820 0x7f205c1972a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:10:21.664 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:21.662+0000 7f2064791640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f205c1977e0 0x7f205c19bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:10:21.664 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:21.662+0000 7f2064791640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f205c19c0b0 0x7f205c19c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:10:21.664 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:21.662+0000 7f2064791640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f205c109730 con 0x7f205c19c0b0 2024-09-11T23:10:21.664 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:21.662+0000 7f2064791640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f205c1095b0 con 0x7f205c102820 2024-09-11T23:10:21.664 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:21.662+0000 7f2064791640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f205c1098b0 con 0x7f205c1977e0 2024-09-11T23:10:21.665 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:21.662+0000 7f2061d2c640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f205c1977e0 0x7f205c19bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:10:21.665 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:21.662+0000 7f2062d2e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f205c19c0b0 0x7f205c19c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:10:21.665 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:21.662+0000 7f2062d2e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f205c19c0b0 0x7f205c19c560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.136:50270/0 (socket says 172.21.15.136:50270) 2024-09-11T23:10:21.665 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:21.662+0000 7f2062d2e640 1 -- 172.21.15.136:0/1237166940 learned_addr learned my addr 172.21.15.136:0/1237166940 (peer_addr_for_me v2:172.21.15.136:0/0) 2024-09-11T23:10:21.665 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:21.662+0000 7f206252d640 1 --2- 172.21.15.136:0/1237166940 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f205c102820 0x7f205c1972a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:10:21.665 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:21.662+0000 7f2061d2c640 1 -- 172.21.15.136:0/1237166940 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f205c1977e0 msgr2=0x7f205c19bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-09-11T23:10:21.665 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:21.662+0000 7f2061d2c640 1 -- 172.21.15.136:0/1237166940 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f205c1977e0 msgr2=0x7f205c19bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-11T23:10:21.665 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:21.662+0000 7f2061d2c640 1 --2- 172.21.15.136:0/1237166940 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f205c1977e0 0x7f205c19bb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-11T23:10:21.665 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:21.662+0000 7f206252d640 1 -- 172.21.15.136:0/1237166940 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f205c102820 msgr2=0x7f205c1972a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-09-11T23:10:21.665 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:21.662+0000 7f206252d640 1 -- 172.21.15.136:0/1237166940 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f205c102820 msgr2=0x7f205c1972a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-11T23:10:21.665 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:21.662+0000 7f206252d640 1 --2- 172.21.15.136:0/1237166940 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f205c102820 0x7f205c1972a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-11T23:10:21.666 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:21.662+0000 7f2061d2c640 1 --2- 172.21.15.136:0/1237166940 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f205c1977e0 0x7f205c19bb70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-11T23:10:21.666 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:21.662+0000 7f206252d640 1 --2- 172.21.15.136:0/1237166940 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f205c102820 0x7f205c1972a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-11T23:10:21.666 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:21.662+0000 7f2062d2e640 1 -- 172.21.15.136:0/1237166940 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f205c1977e0 msgr2=0x7f205c19bb70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-11T23:10:21.666 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:21.662+0000 7f2062d2e640 1 --2- 172.21.15.136:0/1237166940 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f205c1977e0 0x7f205c19bb70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:10:21.666 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:21.662+0000 7f2062d2e640 1 -- 172.21.15.136:0/1237166940 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f205c102820 msgr2=0x7f205c1972a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-11T23:10:21.666 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:21.662+0000 7f2062d2e640 1 --2- 172.21.15.136:0/1237166940 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f205c102820 0x7f205c1972a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:10:21.666 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:21.662+0000 7f2062d2e640 1 -- 172.21.15.136:0/1237166940 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f205c1a3e20 con 0x7f205c19c0b0 2024-09-11T23:10:21.666 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:21.663+0000 7f2062d2e640 1 --2- 172.21.15.136:0/1237166940 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f205c19c0b0 0x7f205c19c560 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f205400b450 tx=0x7f205400b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:10:21.666 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:21.663+0000 7f20537fe640 1 -- 172.21.15.136:0/1237166940 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2054015020 con 0x7f205c19c0b0 2024-09-11T23:10:21.666 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:21.663+0000 7f2064791640 1 -- 172.21.15.136:0/1237166940 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f205c1a3ff0 con 0x7f205c19c0b0 2024-09-11T23:10:21.666 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:21.663+0000 7f20537fe640 1 -- 172.21.15.136:0/1237166940 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2054011d70 con 0x7f205c19c0b0 2024-09-11T23:10:21.666 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:21.664+0000 7f20537fe640 1 -- 172.21.15.136:0/1237166940 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f205400f340 con 0x7f205c19c0b0 2024-09-11T23:10:21.666 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:21.664+0000 7f2064791640 1 -- 172.21.15.136:0/1237166940 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f205c1a4500 con 0x7f205c19c0b0 2024-09-11T23:10:21.667 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:21.664+0000 7f20537fe640 1 -- 172.21.15.136:0/1237166940 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f2054018020 con 0x7f205c19c0b0 2024-09-11T23:10:21.667 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:21.665+0000 7f20537fe640 1 --2- 172.21.15.136:0/1237166940 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f20400394f0 0x7f204003b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:10:21.668 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:21.665+0000 7f20537fe640 1 -- 172.21.15.136:0/1237166940 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f2054011ad0 con 0x7f205c19c0b0 2024-09-11T23:10:21.668 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:21.665+0000 7f2064791640 1 -- 172.21.15.136:0/1237166940 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f205c10a840 con 0x7f205c19c0b0 2024-09-11T23:10:21.668 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:21.665+0000 7f206252d640 1 --2- 172.21.15.136:0/1237166940 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f20400394f0 0x7f204003b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:10:21.668 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:21.665+0000 7f206252d640 1 --2- 172.21.15.136:0/1237166940 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f20400394f0 0x7f204003b9b0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f2058009bd0 tx=0x7f2058009b40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:10:21.670 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:21.670+0000 7f20537fe640 1 -- 172.21.15.136:0/1237166940 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f205401f020 con 0x7f205c19c0b0 2024-09-11T23:10:21.913 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:21.913+0000 7f2064791640 1 -- 172.21.15.136:0/1237166940 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f205c106420 con 0x7f205c19c0b0 2024-09-11T23:10:21.913 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:21 smithi136 ceph-mon[35121]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:10:21.913 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:21 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T23:10:21.914 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:21 smithi136 ceph-mon[35121]: mon.c calling monitor election 2024-09-11T23:10:21.914 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:21 smithi136 ceph-mon[35121]: mon.a calling monitor election 2024-09-11T23:10:21.914 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:21 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T23:10:21.914 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:21 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T23:10:21.914 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:21 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T23:10:21.914 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:21 smithi136 ceph-mon[35121]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:10:21.914 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:21 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T23:10:21.914 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:21 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T23:10:21.914 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:21 smithi136 ceph-mon[35121]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:10:21.914 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:21 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T23:10:21.914 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:21 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T23:10:21.914 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:21 smithi136 ceph-mon[35121]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-11T23:10:21.914 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:21 smithi136 ceph-mon[35121]: monmap epoch 3 2024-09-11T23:10:21.915 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:21 smithi136 ceph-mon[35121]: fsid af7e6fc8-7092-11ef-bcea-c7b262605968 2024-09-11T23:10:21.915 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:21 smithi136 ceph-mon[35121]: last_changed 2024-09-11T23:10:16.639102+0000 2024-09-11T23:10:21.915 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:21 smithi136 ceph-mon[35121]: created 2024-09-11T23:08:37.446176+0000 2024-09-11T23:10:21.915 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:21 smithi136 ceph-mon[35121]: min_mon_release 19 (squid) 2024-09-11T23:10:21.915 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:21 smithi136 ceph-mon[35121]: election_strategy: 1 2024-09-11T23:10:21.915 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:21 smithi136 ceph-mon[35121]: 0: [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] mon.a 2024-09-11T23:10:21.915 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:21 smithi136 ceph-mon[35121]: 1: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.c 2024-09-11T23:10:21.915 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:21 smithi136 ceph-mon[35121]: 2: [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] mon.b 2024-09-11T23:10:21.915 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:21 smithi136 ceph-mon[35121]: fsmap 2024-09-11T23:10:21.915 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:21 smithi136 ceph-mon[35121]: osdmap e4: 0 total, 0 up, 0 in 2024-09-11T23:10:21.915 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:21 smithi136 ceph-mon[35121]: mgrmap e13: x(active, since 47s) 2024-09-11T23:10:21.915 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:21 smithi136 ceph-mon[35121]: overall HEALTH_OK 2024-09-11T23:10:21.915 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:21 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:10:21.915 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:21 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:10:21.916 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:21 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:10:21.916 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:21 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:10:21.916 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:21.913+0000 7f20537fe640 1 -- 172.21.15.136:0/1237166940 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1279 (secure 0 0 0) 0x7f205400d040 con 0x7f205c19c0b0 2024-09-11T23:10:21.916 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-09-11T23:10:21.916 INFO:teuthology.orchestra.run.smithi136.stdout:{"epoch":3,"fsid":"af7e6fc8-7092-11ef-bcea-c7b262605968","modified":"2024-09-11T23:10:16.639102Z","created":"2024-09-11T23:08:37.446176Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:3300","nonce":0},{"type":"v1","addr":"172.21.15.26:6789","nonce":0}]},"addr":"172.21.15.26:6789/0","public_addr":"172.21.15.26:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:3300","nonce":0},{"type":"v1","addr":"172.21.15.136:6789","nonce":0}]},"addr":"172.21.15.136:6789/0","public_addr":"172.21.15.136:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:3300","nonce":0},{"type":"v1","addr":"172.21.15.76:6789","nonce":0}]},"addr":"172.21.15.76:6789/0","public_addr":"172.21.15.76:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-11T23:10:21.917 INFO:teuthology.orchestra.run.smithi136.stderr:dumped monmap epoch 3 2024-09-11T23:10:21.917 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:21.916+0000 7f2064791640 1 -- 172.21.15.136:0/1237166940 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f20400394f0 msgr2=0x7f204003b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:10:21.917 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:21.916+0000 7f2064791640 1 --2- 172.21.15.136:0/1237166940 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f20400394f0 0x7f204003b9b0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f2058009bd0 tx=0x7f2058009b40 comp rx=0 tx=0).stop 2024-09-11T23:10:21.917 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:21.916+0000 7f2064791640 1 -- 172.21.15.136:0/1237166940 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f205c19c0b0 msgr2=0x7f205c19c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:10:21.917 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:21.916+0000 7f2064791640 1 --2- 172.21.15.136:0/1237166940 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f205c19c0b0 0x7f205c19c560 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f205400b450 tx=0x7f205400b910 comp rx=0 tx=0).stop 2024-09-11T23:10:21.918 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:21.916+0000 7f2064791640 1 -- 172.21.15.136:0/1237166940 shutdown_connections 2024-09-11T23:10:21.918 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:21.916+0000 7f2064791640 1 --2- 172.21.15.136:0/1237166940 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f20400394f0 0x7f204003b9b0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:10:21.918 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:21.916+0000 7f2064791640 1 --2- 172.21.15.136:0/1237166940 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f205c19c0b0 0x7f205c19c560 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:10:21.918 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:21.916+0000 7f2064791640 1 --2- 172.21.15.136:0/1237166940 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f205c1977e0 0x7f205c19bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:10:21.918 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:21.916+0000 7f2064791640 1 --2- 172.21.15.136:0/1237166940 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f205c102820 0x7f205c1972a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:10:21.918 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:21.916+0000 7f2064791640 1 -- 172.21.15.136:0/1237166940 >> 172.21.15.136:0/1237166940 conn(0x7f205c0fc220 msgr2=0x7f205c0fe0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:10:21.918 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:21.916+0000 7f2064791640 1 -- 172.21.15.136:0/1237166940 shutdown_connections 2024-09-11T23:10:21.918 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:21.916+0000 7f2064791640 1 -- 172.21.15.136:0/1237166940 wait complete. 2024-09-11T23:10:22.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:21 smithi026 ceph-mon[31172]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:10:22.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:21 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T23:10:22.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:21 smithi026 ceph-mon[31172]: mon.c calling monitor election 2024-09-11T23:10:22.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:21 smithi026 ceph-mon[31172]: mon.a calling monitor election 2024-09-11T23:10:22.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:21 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T23:10:22.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:21 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T23:10:22.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:21 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T23:10:22.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:21 smithi026 ceph-mon[31172]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:10:22.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:21 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T23:10:22.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:21 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T23:10:22.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:21 smithi026 ceph-mon[31172]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:10:22.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:21 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T23:10:22.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:21 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T23:10:22.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:21 smithi026 ceph-mon[31172]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-11T23:10:22.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:21 smithi026 ceph-mon[31172]: monmap epoch 3 2024-09-11T23:10:22.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:21 smithi026 ceph-mon[31172]: fsid af7e6fc8-7092-11ef-bcea-c7b262605968 2024-09-11T23:10:22.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:21 smithi026 ceph-mon[31172]: last_changed 2024-09-11T23:10:16.639102+0000 2024-09-11T23:10:22.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:21 smithi026 ceph-mon[31172]: created 2024-09-11T23:08:37.446176+0000 2024-09-11T23:10:22.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:21 smithi026 ceph-mon[31172]: min_mon_release 19 (squid) 2024-09-11T23:10:22.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:21 smithi026 ceph-mon[31172]: election_strategy: 1 2024-09-11T23:10:22.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:21 smithi026 ceph-mon[31172]: 0: [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] mon.a 2024-09-11T23:10:22.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:21 smithi026 ceph-mon[31172]: 1: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.c 2024-09-11T23:10:22.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:21 smithi026 ceph-mon[31172]: 2: [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] mon.b 2024-09-11T23:10:22.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:21 smithi026 ceph-mon[31172]: fsmap 2024-09-11T23:10:22.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:21 smithi026 ceph-mon[31172]: osdmap e4: 0 total, 0 up, 0 in 2024-09-11T23:10:22.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:21 smithi026 ceph-mon[31172]: mgrmap e13: x(active, since 47s) 2024-09-11T23:10:22.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:21 smithi026 ceph-mon[31172]: overall HEALTH_OK 2024-09-11T23:10:22.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:21 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:10:22.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:21 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:10:22.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:21 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:10:22.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:21 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:10:22.432 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-09-11T23:10:22.432 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph config generate-minimal-conf 2024-09-11T23:10:22.700 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:10:22.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:22 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:10:22.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:22 smithi026 ceph-mon[31172]: from='client.? 172.21.15.136:0/1237166940' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-11T23:10:22.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:22 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T23:10:22.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:22 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:10:22.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:22 smithi136 ceph-mon[35121]: from='client.? 172.21.15.136:0/1237166940' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-11T23:10:22.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:22 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T23:10:23.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:10:23.117+0000 7fd735ba5640 1 -- 172.21.15.26:0/2825036468 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd730102730 msgr2=0x7fd730102b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:10:23.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:10:23.117+0000 7fd735ba5640 1 --2- 172.21.15.26:0/2825036468 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd730102730 0x7fd730102b10 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fd720009810 tx=0x7fd72002a540 comp rx=0 tx=0).stop 2024-09-11T23:10:23.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:10:23.117+0000 7fd735ba5640 1 -- 172.21.15.26:0/2825036468 shutdown_connections 2024-09-11T23:10:23.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:10:23.117+0000 7fd735ba5640 1 --2- 172.21.15.26:0/2825036468 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd730102730 0x7fd730102b10 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:10:23.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:10:23.117+0000 7fd735ba5640 1 -- 172.21.15.26:0/2825036468 >> 172.21.15.26:0/2825036468 conn(0x7fd730077310 msgr2=0x7fd7300fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:10:23.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:10:23.117+0000 7fd735ba5640 1 -- 172.21.15.26:0/2825036468 shutdown_connections 2024-09-11T23:10:23.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:10:23.117+0000 7fd735ba5640 1 -- 172.21.15.26:0/2825036468 wait complete. 2024-09-11T23:10:23.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:10:23.118+0000 7fd735ba5640 1 Processor -- start 2024-09-11T23:10:23.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:10:23.118+0000 7fd735ba5640 1 -- start start 2024-09-11T23:10:23.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:10:23.118+0000 7fd735ba5640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd730102730 0x7fd730197350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:10:23.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:10:23.118+0000 7fd735ba5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd730197890 0x7fd73019bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:10:23.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:10:23.118+0000 7fd735ba5640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd73019c160 0x7fd73019c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:10:23.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:10:23.118+0000 7fd735ba5640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd73010bc80 con 0x7fd730197890 2024-09-11T23:10:23.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:10:23.118+0000 7fd735ba5640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fd73010bb00 con 0x7fd73019c160 2024-09-11T23:10:23.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:10:23.118+0000 7fd735ba5640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd73010be00 con 0x7fd730102730 2024-09-11T23:10:23.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:10:23.118+0000 7fd72effd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd730197890 0x7fd73019bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:10:23.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:10:23.118+0000 7fd72effd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd730197890 0x7fd73019bc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53974/0 (socket says 172.21.15.26:53974) 2024-09-11T23:10:23.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:10:23.118+0000 7fd72effd640 1 -- 172.21.15.26:0/2275914890 learned_addr learned my addr 172.21.15.26:0/2275914890 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:10:23.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:10:23.119+0000 7fd72f7fe640 1 --2- 172.21.15.26:0/2275914890 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd730102730 0x7fd730197350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:10:23.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:10:23.119+0000 7fd72ffff640 1 --2- 172.21.15.26:0/2275914890 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd73019c160 0x7fd73019c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:10:23.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:10:23.119+0000 7fd72effd640 1 -- 172.21.15.26:0/2275914890 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd730102730 msgr2=0x7fd730197350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:10:23.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:10:23.119+0000 7fd72effd640 1 --2- 172.21.15.26:0/2275914890 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd730102730 0x7fd730197350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:10:23.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:10:23.119+0000 7fd72effd640 1 -- 172.21.15.26:0/2275914890 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd73019c160 msgr2=0x7fd73019c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:10:23.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:10:23.119+0000 7fd72effd640 1 --2- 172.21.15.26:0/2275914890 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd73019c160 0x7fd73019c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:10:23.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:10:23.119+0000 7fd72effd640 1 -- 172.21.15.26:0/2275914890 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7301a3ed0 con 0x7fd730197890 2024-09-11T23:10:23.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:10:23.119+0000 7fd72effd640 1 --2- 172.21.15.26:0/2275914890 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd730197890 0x7fd73019bc20 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fd71800b520 tx=0x7fd71800b9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:10:23.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:10:23.119+0000 7fd72cff9640 1 -- 172.21.15.26:0/2275914890 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd718004230 con 0x7fd730197890 2024-09-11T23:10:23.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:10:23.119+0000 7fd735ba5640 1 -- 172.21.15.26:0/2275914890 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7301a4100 con 0x7fd730197890 2024-09-11T23:10:23.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:10:23.119+0000 7fd72cff9640 1 -- 172.21.15.26:0/2275914890 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd7180043d0 con 0x7fd730197890 2024-09-11T23:10:23.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:10:23.119+0000 7fd72cff9640 1 -- 172.21.15.26:0/2275914890 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd718003d60 con 0x7fd730197890 2024-09-11T23:10:23.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:10:23.119+0000 7fd735ba5640 1 -- 172.21.15.26:0/2275914890 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7301a4610 con 0x7fd730197890 2024-09-11T23:10:23.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:10:23.120+0000 7fd72cff9640 1 -- 172.21.15.26:0/2275914890 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fd71800e070 con 0x7fd730197890 2024-09-11T23:10:23.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:10:23.120+0000 7fd735ba5640 1 -- 172.21.15.26:0/2275914890 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd730069f90 con 0x7fd730197890 2024-09-11T23:10:23.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:10:23.120+0000 7fd72cff9640 1 --2- 172.21.15.26:0/2275914890 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fd71c0394f0 0x7fd71c03b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:10:23.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:10:23.120+0000 7fd72cff9640 1 -- 172.21.15.26:0/2275914890 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fd71804bbc0 con 0x7fd730197890 2024-09-11T23:10:23.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:10:23.121+0000 7fd72f7fe640 1 --2- 172.21.15.26:0/2275914890 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fd71c0394f0 0x7fd71c03b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:10:23.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:10:23.121+0000 7fd72f7fe640 1 --2- 172.21.15.26:0/2275914890 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fd71c0394f0 0x7fd71c03b9b0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd72002aab0 tx=0x7fd72002aa40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:10:23.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:10:23.125+0000 7fd72cff9640 1 -- 172.21.15.26:0/2275914890 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd718014020 con 0x7fd730197890 2024-09-11T23:10:23.307 INFO:teuthology.orchestra.run.smithi026.stdout:# minimal ceph.conf for af7e6fc8-7092-11ef-bcea-c7b262605968 2024-09-11T23:10:23.307 INFO:teuthology.orchestra.run.smithi026.stdout:[global] 2024-09-11T23:10:23.307 INFO:teuthology.orchestra.run.smithi026.stdout: fsid = af7e6fc8-7092-11ef-bcea-c7b262605968 2024-09-11T23:10:23.307 INFO:teuthology.orchestra.run.smithi026.stdout: mon_host = [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] 2024-09-11T23:10:23.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:10:23.306+0000 7fd735ba5640 1 -- 172.21.15.26:0/2275914890 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7fd7301a4970 con 0x7fd730197890 2024-09-11T23:10:23.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:10:23.306+0000 7fd72cff9640 1 -- 172.21.15.26:0/2275914890 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+273 (secure 0 0 0) 0x7fd718010d70 con 0x7fd730197890 2024-09-11T23:10:23.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:10:23.308+0000 7fd735ba5640 1 -- 172.21.15.26:0/2275914890 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fd71c0394f0 msgr2=0x7fd71c03b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:10:23.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:10:23.309+0000 7fd735ba5640 1 --2- 172.21.15.26:0/2275914890 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fd71c0394f0 0x7fd71c03b9b0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd72002aab0 tx=0x7fd72002aa40 comp rx=0 tx=0).stop 2024-09-11T23:10:23.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:10:23.309+0000 7fd735ba5640 1 -- 172.21.15.26:0/2275914890 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd730197890 msgr2=0x7fd73019bc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:10:23.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:10:23.309+0000 7fd735ba5640 1 --2- 172.21.15.26:0/2275914890 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd730197890 0x7fd73019bc20 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fd71800b520 tx=0x7fd71800b9e0 comp rx=0 tx=0).stop 2024-09-11T23:10:23.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:10:23.309+0000 7fd735ba5640 1 -- 172.21.15.26:0/2275914890 shutdown_connections 2024-09-11T23:10:23.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:10:23.309+0000 7fd735ba5640 1 --2- 172.21.15.26:0/2275914890 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fd71c0394f0 0x7fd71c03b9b0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:10:23.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:10:23.309+0000 7fd735ba5640 1 --2- 172.21.15.26:0/2275914890 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd73019c160 0x7fd73019c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:10:23.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:10:23.309+0000 7fd735ba5640 1 --2- 172.21.15.26:0/2275914890 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd730197890 0x7fd73019bc20 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:10:23.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:10:23.309+0000 7fd735ba5640 1 --2- 172.21.15.26:0/2275914890 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd730102730 0x7fd730197350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:10:23.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:10:23.309+0000 7fd735ba5640 1 -- 172.21.15.26:0/2275914890 >> 172.21.15.26:0/2275914890 conn(0x7fd730077310 msgr2=0x7fd7301035f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:10:23.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:10:23.309+0000 7fd735ba5640 1 -- 172.21.15.26:0/2275914890 shutdown_connections 2024-09-11T23:10:23.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:10:23.309+0000 7fd735ba5640 1 -- 172.21.15.26:0/2275914890 wait complete. 2024-09-11T23:10:24.042 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:23 smithi076 ceph-mon[35296]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:10:24.042 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:23 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T23:10:24.042 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:23 smithi076 ceph-mon[35296]: mon.c calling monitor election 2024-09-11T23:10:24.042 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:23 smithi076 ceph-mon[35296]: mon.a calling monitor election 2024-09-11T23:10:24.042 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:23 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T23:10:24.042 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:23 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T23:10:24.043 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:23 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T23:10:24.043 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:23 smithi076 ceph-mon[35296]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:10:24.043 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:23 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T23:10:24.043 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:23 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T23:10:24.043 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:23 smithi076 ceph-mon[35296]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:10:24.043 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:23 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T23:10:24.043 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:23 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T23:10:24.043 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:23 smithi076 ceph-mon[35296]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-11T23:10:24.043 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:23 smithi076 ceph-mon[35296]: monmap epoch 3 2024-09-11T23:10:24.043 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:23 smithi076 ceph-mon[35296]: fsid af7e6fc8-7092-11ef-bcea-c7b262605968 2024-09-11T23:10:24.043 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:23 smithi076 ceph-mon[35296]: last_changed 2024-09-11T23:10:16.639102+0000 2024-09-11T23:10:24.043 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:23 smithi076 ceph-mon[35296]: created 2024-09-11T23:08:37.446176+0000 2024-09-11T23:10:24.043 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:23 smithi076 ceph-mon[35296]: min_mon_release 19 (squid) 2024-09-11T23:10:24.044 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:23 smithi076 ceph-mon[35296]: election_strategy: 1 2024-09-11T23:10:24.044 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:23 smithi076 ceph-mon[35296]: 0: [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] mon.a 2024-09-11T23:10:24.044 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:23 smithi076 ceph-mon[35296]: 1: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.c 2024-09-11T23:10:24.044 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:23 smithi076 ceph-mon[35296]: 2: [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] mon.b 2024-09-11T23:10:24.044 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:23 smithi076 ceph-mon[35296]: fsmap 2024-09-11T23:10:24.044 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:23 smithi076 ceph-mon[35296]: osdmap e4: 0 total, 0 up, 0 in 2024-09-11T23:10:24.044 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:23 smithi076 ceph-mon[35296]: mgrmap e13: x(active, since 47s) 2024-09-11T23:10:24.044 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:23 smithi076 ceph-mon[35296]: overall HEALTH_OK 2024-09-11T23:10:24.044 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:23 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:10:24.044 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:23 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:10:24.044 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:23 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:10:24.044 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:23 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:10:24.044 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:23 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:10:24.044 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:23 smithi076 ceph-mon[35296]: from='client.? 172.21.15.136:0/1237166940' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-11T23:10:24.045 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:23 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T23:10:24.072 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-09-11T23:10:24.072 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-09-11T23:10:24.072 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd of=/etc/ceph/ceph.conf 2024-09-11T23:10:24.100 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-09-11T23:10:24.100 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-11T23:10:24.166 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-09-11T23:10:24.167 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd of=/etc/ceph/ceph.conf 2024-09-11T23:10:24.194 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-09-11T23:10:24.194 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-11T23:10:24.260 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-09-11T23:10:24.260 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd of=/etc/ceph/ceph.conf 2024-09-11T23:10:24.287 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-09-11T23:10:24.287 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-11T23:10:24.304 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:24 smithi076 ceph-mon[35296]: mon.b calling monitor election 2024-09-11T23:10:24.304 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:24 smithi076 ceph-mon[35296]: Updating smithi026:/etc/ceph/ceph.conf 2024-09-11T23:10:24.304 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:24 smithi076 ceph-mon[35296]: Updating smithi076:/etc/ceph/ceph.conf 2024-09-11T23:10:24.304 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:24 smithi076 ceph-mon[35296]: Updating smithi136:/etc/ceph/ceph.conf 2024-09-11T23:10:24.304 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:24 smithi076 ceph-mon[35296]: Updating smithi076:/var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T23:10:24.304 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:24 smithi076 ceph-mon[35296]: Updating smithi136:/var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T23:10:24.304 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:24 smithi076 ceph-mon[35296]: Updating smithi026:/var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T23:10:24.304 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:24 smithi076 ceph-mon[35296]: mon.b calling monitor election 2024-09-11T23:10:24.305 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:24 smithi076 ceph-mon[35296]: mon.c calling monitor election 2024-09-11T23:10:24.305 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:24 smithi076 ceph-mon[35296]: mon.a calling monitor election 2024-09-11T23:10:24.305 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:24 smithi076 ceph-mon[35296]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-11T23:10:24.305 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:24 smithi076 ceph-mon[35296]: monmap epoch 3 2024-09-11T23:10:24.305 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:24 smithi076 ceph-mon[35296]: fsid af7e6fc8-7092-11ef-bcea-c7b262605968 2024-09-11T23:10:24.305 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:24 smithi076 ceph-mon[35296]: last_changed 2024-09-11T23:10:16.639102+0000 2024-09-11T23:10:24.305 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:24 smithi076 ceph-mon[35296]: created 2024-09-11T23:08:37.446176+0000 2024-09-11T23:10:24.305 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:24 smithi076 ceph-mon[35296]: min_mon_release 19 (squid) 2024-09-11T23:10:24.305 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:24 smithi076 ceph-mon[35296]: election_strategy: 1 2024-09-11T23:10:24.305 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:24 smithi076 ceph-mon[35296]: 0: [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] mon.a 2024-09-11T23:10:24.305 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:24 smithi076 ceph-mon[35296]: 1: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.c 2024-09-11T23:10:24.305 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:24 smithi076 ceph-mon[35296]: 2: [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] mon.b 2024-09-11T23:10:24.305 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:24 smithi076 ceph-mon[35296]: fsmap 2024-09-11T23:10:24.305 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:24 smithi076 ceph-mon[35296]: osdmap e4: 0 total, 0 up, 0 in 2024-09-11T23:10:24.306 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:24 smithi076 ceph-mon[35296]: mgrmap e13: x(active, since 49s) 2024-09-11T23:10:24.306 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:24 smithi076 ceph-mon[35296]: overall HEALTH_OK 2024-09-11T23:10:24.306 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:24 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:10:24.306 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:24 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:10:24.306 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:24 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:10:24.306 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:24 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:10:24.306 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:24 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:10:24.306 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:24 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:10:24.306 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:24 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:10:24.306 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:24 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:10:24.306 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:24 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:10:24.306 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:24 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:10:24.306 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:24 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:10:24.306 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:24 smithi076 ceph-mon[35296]: Reconfiguring mon.a (unknown last config time)... 2024-09-11T23:10:24.307 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:24 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T23:10:24.307 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:24 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-11T23:10:24.307 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:24 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:10:24.307 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:24 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:10:24.320 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:24 smithi026 ceph-mon[31172]: mon.b calling monitor election 2024-09-11T23:10:24.320 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:24 smithi026 ceph-mon[31172]: Updating smithi026:/etc/ceph/ceph.conf 2024-09-11T23:10:24.320 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:24 smithi026 ceph-mon[31172]: Updating smithi076:/etc/ceph/ceph.conf 2024-09-11T23:10:24.320 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:24 smithi026 ceph-mon[31172]: Updating smithi136:/etc/ceph/ceph.conf 2024-09-11T23:10:24.321 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:24 smithi026 ceph-mon[31172]: Updating smithi076:/var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T23:10:24.321 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:24 smithi026 ceph-mon[31172]: Updating smithi136:/var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T23:10:24.321 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:24 smithi026 ceph-mon[31172]: Updating smithi026:/var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T23:10:24.321 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:24 smithi026 ceph-mon[31172]: mon.b calling monitor election 2024-09-11T23:10:24.321 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:24 smithi026 ceph-mon[31172]: mon.c calling monitor election 2024-09-11T23:10:24.321 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:24 smithi026 ceph-mon[31172]: mon.a calling monitor election 2024-09-11T23:10:24.321 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:24 smithi026 ceph-mon[31172]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-11T23:10:24.321 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:24 smithi026 ceph-mon[31172]: monmap epoch 3 2024-09-11T23:10:24.321 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:24 smithi026 ceph-mon[31172]: fsid af7e6fc8-7092-11ef-bcea-c7b262605968 2024-09-11T23:10:24.321 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:24 smithi026 ceph-mon[31172]: last_changed 2024-09-11T23:10:16.639102+0000 2024-09-11T23:10:24.321 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:24 smithi026 ceph-mon[31172]: created 2024-09-11T23:08:37.446176+0000 2024-09-11T23:10:24.321 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:24 smithi026 ceph-mon[31172]: min_mon_release 19 (squid) 2024-09-11T23:10:24.321 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:24 smithi026 ceph-mon[31172]: election_strategy: 1 2024-09-11T23:10:24.321 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:24 smithi026 ceph-mon[31172]: 0: [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] mon.a 2024-09-11T23:10:24.322 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:24 smithi026 ceph-mon[31172]: 1: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.c 2024-09-11T23:10:24.322 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:24 smithi026 ceph-mon[31172]: 2: [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] mon.b 2024-09-11T23:10:24.322 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:24 smithi026 ceph-mon[31172]: fsmap 2024-09-11T23:10:24.322 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:24 smithi026 ceph-mon[31172]: osdmap e4: 0 total, 0 up, 0 in 2024-09-11T23:10:24.322 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:24 smithi026 ceph-mon[31172]: mgrmap e13: x(active, since 49s) 2024-09-11T23:10:24.322 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:24 smithi026 ceph-mon[31172]: overall HEALTH_OK 2024-09-11T23:10:24.322 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:24 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:10:24.322 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:24 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:10:24.322 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:24 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:10:24.322 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:24 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:10:24.322 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:24 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:10:24.322 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:24 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:10:24.323 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:24 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:10:24.323 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:24 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:10:24.323 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:24 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:10:24.323 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:24 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:10:24.323 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:24 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:10:24.324 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:24 smithi026 ceph-mon[31172]: Reconfiguring mon.a (unknown last config time)... 2024-09-11T23:10:24.324 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:24 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T23:10:24.324 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:24 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-11T23:10:24.324 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:24 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:10:24.324 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:24 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:10:24.352 INFO:tasks.cephadm:Adding mgr.x on smithi026 2024-09-11T23:10:24.352 INFO:tasks.cephadm:Adding mgr.y on smithi076 2024-09-11T23:10:24.352 INFO:tasks.cephadm:Adding mgr.z on smithi136 2024-09-11T23:10:24.352 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph orch apply mgr '3;smithi026=x;smithi076=y;smithi136=z' 2024-09-11T23:10:24.629 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T23:10:24.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:24 smithi136 ceph-mon[35121]: mon.b calling monitor election 2024-09-11T23:10:24.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:24 smithi136 ceph-mon[35121]: Updating smithi026:/etc/ceph/ceph.conf 2024-09-11T23:10:24.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:24 smithi136 ceph-mon[35121]: Updating smithi076:/etc/ceph/ceph.conf 2024-09-11T23:10:24.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:24 smithi136 ceph-mon[35121]: Updating smithi136:/etc/ceph/ceph.conf 2024-09-11T23:10:24.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:24 smithi136 ceph-mon[35121]: Updating smithi076:/var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T23:10:24.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:24 smithi136 ceph-mon[35121]: Updating smithi136:/var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T23:10:24.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:24 smithi136 ceph-mon[35121]: Updating smithi026:/var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T23:10:24.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:24 smithi136 ceph-mon[35121]: mon.b calling monitor election 2024-09-11T23:10:24.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:24 smithi136 ceph-mon[35121]: mon.c calling monitor election 2024-09-11T23:10:24.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:24 smithi136 ceph-mon[35121]: mon.a calling monitor election 2024-09-11T23:10:24.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:24 smithi136 ceph-mon[35121]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-11T23:10:24.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:24 smithi136 ceph-mon[35121]: monmap epoch 3 2024-09-11T23:10:24.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:24 smithi136 ceph-mon[35121]: fsid af7e6fc8-7092-11ef-bcea-c7b262605968 2024-09-11T23:10:24.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:24 smithi136 ceph-mon[35121]: last_changed 2024-09-11T23:10:16.639102+0000 2024-09-11T23:10:24.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:24 smithi136 ceph-mon[35121]: created 2024-09-11T23:08:37.446176+0000 2024-09-11T23:10:24.741 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:24 smithi136 ceph-mon[35121]: min_mon_release 19 (squid) 2024-09-11T23:10:24.741 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:24 smithi136 ceph-mon[35121]: election_strategy: 1 2024-09-11T23:10:24.741 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:24 smithi136 ceph-mon[35121]: 0: [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] mon.a 2024-09-11T23:10:24.741 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:24 smithi136 ceph-mon[35121]: 1: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.c 2024-09-11T23:10:24.741 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:24 smithi136 ceph-mon[35121]: 2: [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] mon.b 2024-09-11T23:10:24.741 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:24 smithi136 ceph-mon[35121]: fsmap 2024-09-11T23:10:24.741 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:24 smithi136 ceph-mon[35121]: osdmap e4: 0 total, 0 up, 0 in 2024-09-11T23:10:24.741 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:24 smithi136 ceph-mon[35121]: mgrmap e13: x(active, since 49s) 2024-09-11T23:10:24.741 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:24 smithi136 ceph-mon[35121]: overall HEALTH_OK 2024-09-11T23:10:24.741 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:24 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:10:24.741 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:24 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:10:24.741 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:24 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:10:24.741 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:24 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:10:24.741 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:24 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:10:24.742 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:24 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:10:24.742 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:24 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:10:24.742 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:24 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:10:24.742 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:24 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:10:24.742 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:24 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:10:24.742 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:24 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:10:24.742 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:24 smithi136 ceph-mon[35121]: Reconfiguring mon.a (unknown last config time)... 2024-09-11T23:10:24.742 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:24 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T23:10:24.742 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:24 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-11T23:10:24.742 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:24 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:10:24.742 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:24 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:10:25.028 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:25.002+0000 7f0035d14640 1 -- 172.21.15.136:0/1867929019 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f00301016e0 msgr2=0x7f0030101ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:10:25.028 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:25.002+0000 7f0035d14640 1 --2- 172.21.15.136:0/1867929019 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f00301016e0 0x7f0030101ae0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f0020009880 tx=0x7f002002a5d0 comp rx=0 tx=0).stop 2024-09-11T23:10:25.028 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:25.002+0000 7f0035d14640 1 -- 172.21.15.136:0/1867929019 shutdown_connections 2024-09-11T23:10:25.028 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:25.002+0000 7f0035d14640 1 --2- 172.21.15.136:0/1867929019 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f00301016e0 0x7f0030101ae0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:10:25.029 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:25.002+0000 7f0035d14640 1 -- 172.21.15.136:0/1867929019 >> 172.21.15.136:0/1867929019 conn(0x7f00300fce90 msgr2=0x7f00300ff2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:10:25.029 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:25.002+0000 7f0035d14640 1 -- 172.21.15.136:0/1867929019 shutdown_connections 2024-09-11T23:10:25.029 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:25.002+0000 7f0035d14640 1 -- 172.21.15.136:0/1867929019 wait complete. 2024-09-11T23:10:25.029 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:25.003+0000 7f0035d14640 1 Processor -- start 2024-09-11T23:10:25.029 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:25.003+0000 7f0035d14640 1 -- start start 2024-09-11T23:10:25.029 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:25.003+0000 7f0035d14640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f00301016e0 0x7f0030075f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:10:25.029 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:25.003+0000 7f0035d14640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f00300745e0 0x7f0030074a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:10:25.029 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:25.003+0000 7f0035d14640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0030074fa0 0x7f00301a1020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:10:25.030 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:25.003+0000 7f0035d14640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f00301077f0 con 0x7f00301016e0 2024-09-11T23:10:25.030 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:25.003+0000 7f0035d14640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f0030107670 con 0x7f0030074fa0 2024-09-11T23:10:25.030 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:25.003+0000 7f0035d14640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f0030107970 con 0x7f00300745e0 2024-09-11T23:10:25.030 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:25.003+0000 7f002effd640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f00300745e0 0x7f0030074a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:10:25.030 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:25.003+0000 7f002f7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f00301016e0 0x7f0030075f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:10:25.030 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:25.003+0000 7f002effd640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f00300745e0 0x7f0030074a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.136:47574/0 (socket says 172.21.15.136:47574) 2024-09-11T23:10:25.031 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:25.003+0000 7f002effd640 1 -- 172.21.15.136:0/1466503964 learned_addr learned my addr 172.21.15.136:0/1466503964 (peer_addr_for_me v2:172.21.15.136:0/0) 2024-09-11T23:10:25.031 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:25.004+0000 7f002ffff640 1 --2- 172.21.15.136:0/1466503964 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0030074fa0 0x7f00301a1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:10:25.031 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:25.004+0000 7f002effd640 1 -- 172.21.15.136:0/1466503964 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0030074fa0 msgr2=0x7f00301a1020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:10:25.031 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:25.004+0000 7f002effd640 1 --2- 172.21.15.136:0/1466503964 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0030074fa0 0x7f00301a1020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:10:25.031 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:25.004+0000 7f002effd640 1 -- 172.21.15.136:0/1466503964 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f00301016e0 msgr2=0x7f0030075f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:10:25.031 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:25.004+0000 7f002effd640 1 --2- 172.21.15.136:0/1466503964 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f00301016e0 0x7f0030075f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:10:25.031 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:25.004+0000 7f002effd640 1 -- 172.21.15.136:0/1466503964 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00301a15b0 con 0x7f00300745e0 2024-09-11T23:10:25.031 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:25.004+0000 7f002f7fe640 1 --2- 172.21.15.136:0/1466503964 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f00301016e0 0x7f0030075f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:10:25.031 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:25.004+0000 7f002effd640 1 --2- 172.21.15.136:0/1466503964 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f00300745e0 0x7f0030074a60 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f002400b520 tx=0x7f002400b9e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:10:25.031 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:25.004+0000 7f002cff9640 1 -- 172.21.15.136:0/1466503964 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0024004230 con 0x7f00300745e0 2024-09-11T23:10:25.031 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:25.004+0000 7f0035d14640 1 -- 172.21.15.136:0/1466503964 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00301a18a0 con 0x7f00300745e0 2024-09-11T23:10:25.031 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:25.004+0000 7f002cff9640 1 -- 172.21.15.136:0/1466503964 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00240043d0 con 0x7f00300745e0 2024-09-11T23:10:25.031 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:25.004+0000 7f002cff9640 1 -- 172.21.15.136:0/1466503964 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0024003d60 con 0x7f00300745e0 2024-09-11T23:10:25.032 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:25.004+0000 7f0035d14640 1 -- 172.21.15.136:0/1466503964 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00301a1e60 con 0x7f00300745e0 2024-09-11T23:10:25.032 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:25.005+0000 7f002cff9640 1 -- 172.21.15.136:0/1466503964 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f002400e070 con 0x7f00300745e0 2024-09-11T23:10:25.032 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:25.005+0000 7f002cff9640 1 --2- 172.21.15.136:0/1466503964 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f00180394f0 0x7f001803b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:10:25.032 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:25.005+0000 7f0035d14640 1 -- 172.21.15.136:0/1466503964 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0030101b60 con 0x7f00300745e0 2024-09-11T23:10:25.032 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:25.005+0000 7f002cff9640 1 -- 172.21.15.136:0/1466503964 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f002404bc60 con 0x7f00300745e0 2024-09-11T23:10:25.032 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:25.006+0000 7f002f7fe640 1 --2- 172.21.15.136:0/1466503964 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f00180394f0 0x7f001803b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:10:25.032 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:25.006+0000 7f002f7fe640 1 --2- 172.21.15.136:0/1466503964 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f00180394f0 0x7f001803b9b0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f0020005ab0 tx=0x7f00200315d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:10:25.032 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:25.010+0000 7f002cff9640 1 -- 172.21.15.136:0/1466503964 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0024014020 con 0x7f00300745e0 2024-09-11T23:10:25.186 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:25.186+0000 7f0035d14640 1 -- 172.21.15.136:0/1466503964 --> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi026=x;smithi076=y;smithi136=z", "target": ["mon-mgr", ""]}) -- 0x7f0030062630 con 0x7f00180394f0 2024-09-11T23:10:25.192 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:25.192+0000 7f002cff9640 1 -- 172.21.15.136:0/1466503964 <== mgr.14148 v2:172.21.15.26:6800/110833859 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f0030062630 con 0x7f00180394f0 2024-09-11T23:10:25.192 INFO:teuthology.orchestra.run.smithi136.stdout:Scheduled mgr update... 2024-09-11T23:10:25.195 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:25.195+0000 7f0035d14640 1 -- 172.21.15.136:0/1466503964 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f00180394f0 msgr2=0x7f001803b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:10:25.195 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:25.195+0000 7f0035d14640 1 --2- 172.21.15.136:0/1466503964 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f00180394f0 0x7f001803b9b0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f0020005ab0 tx=0x7f00200315d0 comp rx=0 tx=0).stop 2024-09-11T23:10:25.195 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:25.195+0000 7f0035d14640 1 -- 172.21.15.136:0/1466503964 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f00300745e0 msgr2=0x7f0030074a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:10:25.195 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:25.195+0000 7f0035d14640 1 --2- 172.21.15.136:0/1466503964 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f00300745e0 0x7f0030074a60 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f002400b520 tx=0x7f002400b9e0 comp rx=0 tx=0).stop 2024-09-11T23:10:25.195 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:25.195+0000 7f0035d14640 1 -- 172.21.15.136:0/1466503964 shutdown_connections 2024-09-11T23:10:25.195 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:25.195+0000 7f0035d14640 1 --2- 172.21.15.136:0/1466503964 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f00180394f0 0x7f001803b9b0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:10:25.195 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:25.195+0000 7f0035d14640 1 --2- 172.21.15.136:0/1466503964 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0030074fa0 0x7f00301a1020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:10:25.195 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:25.195+0000 7f0035d14640 1 --2- 172.21.15.136:0/1466503964 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f00300745e0 0x7f0030074a60 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:10:25.195 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:25.195+0000 7f0035d14640 1 --2- 172.21.15.136:0/1466503964 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f00301016e0 0x7f0030075f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:10:25.195 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:25.195+0000 7f0035d14640 1 -- 172.21.15.136:0/1466503964 >> 172.21.15.136:0/1466503964 conn(0x7f00300fce90 msgr2=0x7f00300fd940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:10:25.195 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:25.195+0000 7f0035d14640 1 -- 172.21.15.136:0/1466503964 shutdown_connections 2024-09-11T23:10:25.196 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:10:25.195+0000 7f0035d14640 1 -- 172.21.15.136:0/1466503964 wait complete. 2024-09-11T23:10:25.501 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:25 smithi026 ceph-mon[31172]: Reconfiguring daemon mon.a on smithi026 2024-09-11T23:10:25.501 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:25 smithi026 ceph-mon[31172]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:10:25.501 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:25 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T23:10:25.501 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:25 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:10:25.553 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:25 smithi076 ceph-mon[35296]: Reconfiguring daemon mon.a on smithi026 2024-09-11T23:10:25.553 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:25 smithi076 ceph-mon[35296]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:10:25.553 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:25 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T23:10:25.553 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:25 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:10:25.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:25 smithi136 ceph-mon[35121]: Reconfiguring daemon mon.a on smithi026 2024-09-11T23:10:25.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:25 smithi136 ceph-mon[35121]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:10:25.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:25 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T23:10:25.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:25 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:10:25.774 DEBUG:teuthology.orchestra.run.smithi076:mgr.y> sudo journalctl -f -n 0 -u ceph-af7e6fc8-7092-11ef-bcea-c7b262605968@mgr.y.service 2024-09-11T23:10:25.777 DEBUG:teuthology.orchestra.run.smithi136:mgr.z> sudo journalctl -f -n 0 -u ceph-af7e6fc8-7092-11ef-bcea-c7b262605968@mgr.z.service 2024-09-11T23:10:25.780 INFO:tasks.cephadm:Deploying OSDs... 2024-09-11T23:10:25.780 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-09-11T23:10:25.780 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/scratch_devs of=/dev/stdout 2024-09-11T23:10:25.798 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-11T23:10:25.799 DEBUG:teuthology.orchestra.run.smithi026:> stat /dev/vg_nvme/lv_1 2024-09-11T23:10:25.856 INFO:teuthology.orchestra.run.smithi026.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-11T23:10:25.856 INFO:teuthology.orchestra.run.smithi026.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T23:10:25.857 INFO:teuthology.orchestra.run.smithi026.stdout:Device: 5h/5d Inode: 668 Links: 1 2024-09-11T23:10:25.857 INFO:teuthology.orchestra.run.smithi026.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T23:10:25.857 INFO:teuthology.orchestra.run.smithi026.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T23:10:25.857 INFO:teuthology.orchestra.run.smithi026.stdout:Access: 2024-09-11 23:09:45.015774528 +0000 2024-09-11T23:10:25.857 INFO:teuthology.orchestra.run.smithi026.stdout:Modify: 2024-09-11 23:05:33.973812409 +0000 2024-09-11T23:10:25.857 INFO:teuthology.orchestra.run.smithi026.stdout:Change: 2024-09-11 23:05:33.973812409 +0000 2024-09-11T23:10:25.857 INFO:teuthology.orchestra.run.smithi026.stdout: Birth: 2024-09-11 23:05:33.973812409 +0000 2024-09-11T23:10:25.861 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-11T23:10:25.925 INFO:teuthology.orchestra.run.smithi026.stderr:1+0 records in 2024-09-11T23:10:25.925 INFO:teuthology.orchestra.run.smithi026.stderr:1+0 records out 2024-09-11T23:10:25.925 INFO:teuthology.orchestra.run.smithi026.stderr:512 bytes copied, 0.000115138 s, 4.4 MB/s 2024-09-11T23:10:25.926 DEBUG:teuthology.orchestra.run.smithi026:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-11T23:10:25.984 DEBUG:teuthology.orchestra.run.smithi026:> stat /dev/vg_nvme/lv_2 2024-09-11T23:10:26.041 INFO:teuthology.orchestra.run.smithi026.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-11T23:10:26.041 INFO:teuthology.orchestra.run.smithi026.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T23:10:26.041 INFO:teuthology.orchestra.run.smithi026.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-09-11T23:10:26.041 INFO:teuthology.orchestra.run.smithi026.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T23:10:26.042 INFO:teuthology.orchestra.run.smithi026.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T23:10:26.042 INFO:teuthology.orchestra.run.smithi026.stdout:Access: 2024-09-11 23:09:45.051773964 +0000 2024-09-11T23:10:26.042 INFO:teuthology.orchestra.run.smithi026.stdout:Modify: 2024-09-11 23:05:34.302807278 +0000 2024-09-11T23:10:26.042 INFO:teuthology.orchestra.run.smithi026.stdout:Change: 2024-09-11 23:05:34.302807278 +0000 2024-09-11T23:10:26.042 INFO:teuthology.orchestra.run.smithi026.stdout: Birth: 2024-09-11 23:05:34.302807278 +0000 2024-09-11T23:10:26.042 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-11T23:10:26.106 INFO:teuthology.orchestra.run.smithi026.stderr:1+0 records in 2024-09-11T23:10:26.106 INFO:teuthology.orchestra.run.smithi026.stderr:1+0 records out 2024-09-11T23:10:26.106 INFO:teuthology.orchestra.run.smithi026.stderr:512 bytes copied, 0.000110386 s, 4.6 MB/s 2024-09-11T23:10:26.107 DEBUG:teuthology.orchestra.run.smithi026:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-11T23:10:26.165 DEBUG:teuthology.orchestra.run.smithi026:> stat /dev/vg_nvme/lv_3 2024-09-11T23:10:26.221 INFO:teuthology.orchestra.run.smithi026.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-11T23:10:26.221 INFO:teuthology.orchestra.run.smithi026.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T23:10:26.222 INFO:teuthology.orchestra.run.smithi026.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-09-11T23:10:26.222 INFO:teuthology.orchestra.run.smithi026.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T23:10:26.222 INFO:teuthology.orchestra.run.smithi026.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T23:10:26.222 INFO:teuthology.orchestra.run.smithi026.stdout:Access: 2024-09-11 23:09:45.087773400 +0000 2024-09-11T23:10:26.222 INFO:teuthology.orchestra.run.smithi026.stdout:Modify: 2024-09-11 23:05:34.645801929 +0000 2024-09-11T23:10:26.222 INFO:teuthology.orchestra.run.smithi026.stdout:Change: 2024-09-11 23:05:34.645801929 +0000 2024-09-11T23:10:26.222 INFO:teuthology.orchestra.run.smithi026.stdout: Birth: 2024-09-11 23:05:34.645801929 +0000 2024-09-11T23:10:26.222 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-11T23:10:26.287 INFO:teuthology.orchestra.run.smithi026.stderr:1+0 records in 2024-09-11T23:10:26.287 INFO:teuthology.orchestra.run.smithi026.stderr:1+0 records out 2024-09-11T23:10:26.287 INFO:teuthology.orchestra.run.smithi026.stderr:512 bytes copied, 8.977e-05 s, 5.7 MB/s 2024-09-11T23:10:26.288 DEBUG:teuthology.orchestra.run.smithi026:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-11T23:10:26.344 DEBUG:teuthology.orchestra.run.smithi026:> stat /dev/vg_nvme/lv_4 2024-09-11T23:10:26.401 INFO:teuthology.orchestra.run.smithi026.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-11T23:10:26.401 INFO:teuthology.orchestra.run.smithi026.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T23:10:26.401 INFO:teuthology.orchestra.run.smithi026.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-09-11T23:10:26.401 INFO:teuthology.orchestra.run.smithi026.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T23:10:26.402 INFO:teuthology.orchestra.run.smithi026.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T23:10:26.402 INFO:teuthology.orchestra.run.smithi026.stdout:Access: 2024-09-11 23:09:45.121772868 +0000 2024-09-11T23:10:26.402 INFO:teuthology.orchestra.run.smithi026.stdout:Modify: 2024-09-11 23:05:34.968796892 +0000 2024-09-11T23:10:26.402 INFO:teuthology.orchestra.run.smithi026.stdout:Change: 2024-09-11 23:05:34.968796892 +0000 2024-09-11T23:10:26.402 INFO:teuthology.orchestra.run.smithi026.stdout: Birth: 2024-09-11 23:05:34.968796892 +0000 2024-09-11T23:10:26.402 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-11T23:10:26.466 INFO:teuthology.orchestra.run.smithi026.stderr:1+0 records in 2024-09-11T23:10:26.466 INFO:teuthology.orchestra.run.smithi026.stderr:1+0 records out 2024-09-11T23:10:26.466 INFO:teuthology.orchestra.run.smithi026.stderr:512 bytes copied, 0.000106147 s, 4.8 MB/s 2024-09-11T23:10:26.467 DEBUG:teuthology.orchestra.run.smithi026:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-11T23:10:26.525 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-09-11T23:10:26.525 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/scratch_devs of=/dev/stdout 2024-09-11T23:10:26.543 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-11T23:10:26.544 DEBUG:teuthology.orchestra.run.smithi076:> stat /dev/vg_nvme/lv_1 2024-09-11T23:10:26.553 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:26 smithi076 ceph-mon[35296]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi026=x;smithi076=y;smithi136=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:10:26.553 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:26 smithi076 ceph-mon[35296]: Saving service mgr spec with placement smithi026=x;smithi076=y;smithi136=z;count:3 2024-09-11T23:10:26.553 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:26 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:10:26.553 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:26 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:10:26.553 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:26 smithi076 ceph-mon[35296]: Reconfiguring mgr.x (unknown last config time)... 2024-09-11T23:10:26.553 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:26 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T23:10:26.553 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:26 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-11T23:10:26.554 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:26 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:10:26.554 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:26 smithi076 ceph-mon[35296]: Reconfiguring daemon mgr.x on smithi026 2024-09-11T23:10:26.570 INFO:teuthology.orchestra.run.smithi076.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-11T23:10:26.570 INFO:teuthology.orchestra.run.smithi076.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T23:10:26.570 INFO:teuthology.orchestra.run.smithi076.stdout:Device: 5h/5d Inode: 664 Links: 1 2024-09-11T23:10:26.570 INFO:teuthology.orchestra.run.smithi076.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T23:10:26.570 INFO:teuthology.orchestra.run.smithi076.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T23:10:26.570 INFO:teuthology.orchestra.run.smithi076.stdout:Access: 2024-09-11 23:10:16.153014570 +0000 2024-09-11T23:10:26.571 INFO:teuthology.orchestra.run.smithi076.stdout:Modify: 2024-09-11 23:05:44.230450046 +0000 2024-09-11T23:10:26.571 INFO:teuthology.orchestra.run.smithi076.stdout:Change: 2024-09-11 23:05:44.230450046 +0000 2024-09-11T23:10:26.571 INFO:teuthology.orchestra.run.smithi076.stdout: Birth: 2024-09-11 23:05:44.230450046 +0000 2024-09-11T23:10:26.571 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-11T23:10:26.611 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:26 smithi026 ceph-mon[31172]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi026=x;smithi076=y;smithi136=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:10:26.612 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:26 smithi026 ceph-mon[31172]: Saving service mgr spec with placement smithi026=x;smithi076=y;smithi136=z;count:3 2024-09-11T23:10:26.612 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:26 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:10:26.612 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:26 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:10:26.612 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:26 smithi026 ceph-mon[31172]: Reconfiguring mgr.x (unknown last config time)... 2024-09-11T23:10:26.612 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:26 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T23:10:26.612 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:26 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-11T23:10:26.612 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:26 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:10:26.612 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:26 smithi026 ceph-mon[31172]: Reconfiguring daemon mgr.x on smithi026 2024-09-11T23:10:26.636 INFO:teuthology.orchestra.run.smithi076.stderr:1+0 records in 2024-09-11T23:10:26.637 INFO:teuthology.orchestra.run.smithi076.stderr:1+0 records out 2024-09-11T23:10:26.637 INFO:teuthology.orchestra.run.smithi076.stderr:512 bytes copied, 0.000316678 s, 1.6 MB/s 2024-09-11T23:10:26.638 DEBUG:teuthology.orchestra.run.smithi076:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-11T23:10:26.696 DEBUG:teuthology.orchestra.run.smithi076:> stat /dev/vg_nvme/lv_2 2024-09-11T23:10:26.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:26 smithi136 ceph-mon[35121]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi026=x;smithi076=y;smithi136=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:10:26.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:26 smithi136 ceph-mon[35121]: Saving service mgr spec with placement smithi026=x;smithi076=y;smithi136=z;count:3 2024-09-11T23:10:26.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:26 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:10:26.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:26 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:10:26.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:26 smithi136 ceph-mon[35121]: Reconfiguring mgr.x (unknown last config time)... 2024-09-11T23:10:26.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:26 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T23:10:26.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:26 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-11T23:10:26.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:26 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:10:26.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:26 smithi136 ceph-mon[35121]: Reconfiguring daemon mgr.x on smithi026 2024-09-11T23:10:26.752 INFO:teuthology.orchestra.run.smithi076.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-11T23:10:26.752 INFO:teuthology.orchestra.run.smithi076.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T23:10:26.752 INFO:teuthology.orchestra.run.smithi076.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-09-11T23:10:26.752 INFO:teuthology.orchestra.run.smithi076.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T23:10:26.752 INFO:teuthology.orchestra.run.smithi076.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T23:10:26.752 INFO:teuthology.orchestra.run.smithi076.stdout:Access: 2024-09-11 23:10:16.153014570 +0000 2024-09-11T23:10:26.752 INFO:teuthology.orchestra.run.smithi076.stdout:Modify: 2024-09-11 23:05:44.570439851 +0000 2024-09-11T23:10:26.752 INFO:teuthology.orchestra.run.smithi076.stdout:Change: 2024-09-11 23:05:44.570439851 +0000 2024-09-11T23:10:26.752 INFO:teuthology.orchestra.run.smithi076.stdout: Birth: 2024-09-11 23:05:44.570439851 +0000 2024-09-11T23:10:26.753 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-11T23:10:26.816 INFO:teuthology.orchestra.run.smithi076.stderr:1+0 records in 2024-09-11T23:10:26.816 INFO:teuthology.orchestra.run.smithi076.stderr:1+0 records out 2024-09-11T23:10:26.816 INFO:teuthology.orchestra.run.smithi076.stderr:512 bytes copied, 0.000180311 s, 2.8 MB/s 2024-09-11T23:10:26.817 DEBUG:teuthology.orchestra.run.smithi076:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-11T23:10:26.874 DEBUG:teuthology.orchestra.run.smithi076:> stat /dev/vg_nvme/lv_3 2024-09-11T23:10:26.930 INFO:teuthology.orchestra.run.smithi076.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-11T23:10:26.930 INFO:teuthology.orchestra.run.smithi076.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T23:10:26.930 INFO:teuthology.orchestra.run.smithi076.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-09-11T23:10:26.930 INFO:teuthology.orchestra.run.smithi076.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T23:10:26.930 INFO:teuthology.orchestra.run.smithi076.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T23:10:26.930 INFO:teuthology.orchestra.run.smithi076.stdout:Access: 2024-09-11 23:10:16.153014570 +0000 2024-09-11T23:10:26.930 INFO:teuthology.orchestra.run.smithi076.stdout:Modify: 2024-09-11 23:05:44.909429686 +0000 2024-09-11T23:10:26.930 INFO:teuthology.orchestra.run.smithi076.stdout:Change: 2024-09-11 23:05:44.909429686 +0000 2024-09-11T23:10:26.930 INFO:teuthology.orchestra.run.smithi076.stdout: Birth: 2024-09-11 23:05:44.909429686 +0000 2024-09-11T23:10:26.931 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-11T23:10:26.995 INFO:teuthology.orchestra.run.smithi076.stderr:1+0 records in 2024-09-11T23:10:26.995 INFO:teuthology.orchestra.run.smithi076.stderr:1+0 records out 2024-09-11T23:10:26.996 INFO:teuthology.orchestra.run.smithi076.stderr:512 bytes copied, 0.000214302 s, 2.4 MB/s 2024-09-11T23:10:26.997 DEBUG:teuthology.orchestra.run.smithi076:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-11T23:10:27.054 DEBUG:teuthology.orchestra.run.smithi076:> stat /dev/vg_nvme/lv_4 2024-09-11T23:10:27.109 INFO:teuthology.orchestra.run.smithi076.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-11T23:10:27.110 INFO:teuthology.orchestra.run.smithi076.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T23:10:27.110 INFO:teuthology.orchestra.run.smithi076.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-09-11T23:10:27.110 INFO:teuthology.orchestra.run.smithi076.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T23:10:27.110 INFO:teuthology.orchestra.run.smithi076.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T23:10:27.110 INFO:teuthology.orchestra.run.smithi076.stdout:Access: 2024-09-11 23:10:16.154014541 +0000 2024-09-11T23:10:27.110 INFO:teuthology.orchestra.run.smithi076.stdout:Modify: 2024-09-11 23:05:45.239419791 +0000 2024-09-11T23:10:27.110 INFO:teuthology.orchestra.run.smithi076.stdout:Change: 2024-09-11 23:05:45.239419791 +0000 2024-09-11T23:10:27.110 INFO:teuthology.orchestra.run.smithi076.stdout: Birth: 2024-09-11 23:05:45.239419791 +0000 2024-09-11T23:10:27.110 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-11T23:10:27.175 INFO:teuthology.orchestra.run.smithi076.stderr:1+0 records in 2024-09-11T23:10:27.175 INFO:teuthology.orchestra.run.smithi076.stderr:1+0 records out 2024-09-11T23:10:27.175 INFO:teuthology.orchestra.run.smithi076.stderr:512 bytes copied, 0.000242035 s, 2.1 MB/s 2024-09-11T23:10:27.177 DEBUG:teuthology.orchestra.run.smithi076:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-11T23:10:27.234 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-09-11T23:10:27.234 DEBUG:teuthology.orchestra.run.smithi136:> dd if=/scratch_devs of=/dev/stdout 2024-09-11T23:10:27.254 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-11T23:10:27.254 DEBUG:teuthology.orchestra.run.smithi136:> stat /dev/vg_nvme/lv_1 2024-09-11T23:10:27.313 INFO:teuthology.orchestra.run.smithi136.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-11T23:10:27.313 INFO:teuthology.orchestra.run.smithi136.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T23:10:27.313 INFO:teuthology.orchestra.run.smithi136.stdout:Device: 5h/5d Inode: 664 Links: 1 2024-09-11T23:10:27.313 INFO:teuthology.orchestra.run.smithi136.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T23:10:27.313 INFO:teuthology.orchestra.run.smithi136.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T23:10:27.313 INFO:teuthology.orchestra.run.smithi136.stdout:Access: 2024-09-11 23:10:07.039137232 +0000 2024-09-11T23:10:27.313 INFO:teuthology.orchestra.run.smithi136.stdout:Modify: 2024-09-11 23:05:54.379595329 +0000 2024-09-11T23:10:27.313 INFO:teuthology.orchestra.run.smithi136.stdout:Change: 2024-09-11 23:05:54.379595329 +0000 2024-09-11T23:10:27.313 INFO:teuthology.orchestra.run.smithi136.stdout: Birth: 2024-09-11 23:05:54.379595329 +0000 2024-09-11T23:10:27.314 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-11T23:10:27.380 INFO:teuthology.orchestra.run.smithi136.stderr:1+0 records in 2024-09-11T23:10:27.381 INFO:teuthology.orchestra.run.smithi136.stderr:1+0 records out 2024-09-11T23:10:27.381 INFO:teuthology.orchestra.run.smithi136.stderr:512 bytes copied, 0.000242024 s, 2.1 MB/s 2024-09-11T23:10:27.382 DEBUG:teuthology.orchestra.run.smithi136:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-11T23:10:27.440 DEBUG:teuthology.orchestra.run.smithi136:> stat /dev/vg_nvme/lv_2 2024-09-11T23:10:27.499 INFO:teuthology.orchestra.run.smithi136.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-11T23:10:27.499 INFO:teuthology.orchestra.run.smithi136.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T23:10:27.499 INFO:teuthology.orchestra.run.smithi136.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-09-11T23:10:27.499 INFO:teuthology.orchestra.run.smithi136.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T23:10:27.499 INFO:teuthology.orchestra.run.smithi136.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T23:10:27.499 INFO:teuthology.orchestra.run.smithi136.stdout:Access: 2024-09-11 23:10:07.039137232 +0000 2024-09-11T23:10:27.499 INFO:teuthology.orchestra.run.smithi136.stdout:Modify: 2024-09-11 23:05:54.732594220 +0000 2024-09-11T23:10:27.499 INFO:teuthology.orchestra.run.smithi136.stdout:Change: 2024-09-11 23:05:54.732594220 +0000 2024-09-11T23:10:27.499 INFO:teuthology.orchestra.run.smithi136.stdout: Birth: 2024-09-11 23:05:54.732594220 +0000 2024-09-11T23:10:27.500 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-11T23:10:27.564 INFO:teuthology.orchestra.run.smithi136.stderr:1+0 records in 2024-09-11T23:10:27.564 INFO:teuthology.orchestra.run.smithi136.stderr:1+0 records out 2024-09-11T23:10:27.565 INFO:teuthology.orchestra.run.smithi136.stderr:512 bytes copied, 0.000238467 s, 2.1 MB/s 2024-09-11T23:10:27.566 DEBUG:teuthology.orchestra.run.smithi136:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-11T23:10:27.624 DEBUG:teuthology.orchestra.run.smithi136:> stat /dev/vg_nvme/lv_3 2024-09-11T23:10:27.681 INFO:teuthology.orchestra.run.smithi136.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-11T23:10:27.681 INFO:teuthology.orchestra.run.smithi136.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T23:10:27.681 INFO:teuthology.orchestra.run.smithi136.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-09-11T23:10:27.681 INFO:teuthology.orchestra.run.smithi136.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T23:10:27.681 INFO:teuthology.orchestra.run.smithi136.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T23:10:27.681 INFO:teuthology.orchestra.run.smithi136.stdout:Access: 2024-09-11 23:10:07.040137229 +0000 2024-09-11T23:10:27.681 INFO:teuthology.orchestra.run.smithi136.stdout:Modify: 2024-09-11 23:05:55.065593173 +0000 2024-09-11T23:10:27.681 INFO:teuthology.orchestra.run.smithi136.stdout:Change: 2024-09-11 23:05:55.065593173 +0000 2024-09-11T23:10:27.681 INFO:teuthology.orchestra.run.smithi136.stdout: Birth: 2024-09-11 23:05:55.065593173 +0000 2024-09-11T23:10:27.682 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-11T23:10:27.746 INFO:teuthology.orchestra.run.smithi136.stderr:1+0 records in 2024-09-11T23:10:27.746 INFO:teuthology.orchestra.run.smithi136.stderr:1+0 records out 2024-09-11T23:10:27.746 INFO:teuthology.orchestra.run.smithi136.stderr:512 bytes copied, 0.000236495 s, 2.2 MB/s 2024-09-11T23:10:27.748 DEBUG:teuthology.orchestra.run.smithi136:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-11T23:10:27.805 DEBUG:teuthology.orchestra.run.smithi136:> stat /dev/vg_nvme/lv_4 2024-09-11T23:10:27.861 INFO:teuthology.orchestra.run.smithi136.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-11T23:10:27.861 INFO:teuthology.orchestra.run.smithi136.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T23:10:27.861 INFO:teuthology.orchestra.run.smithi136.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-09-11T23:10:27.861 INFO:teuthology.orchestra.run.smithi136.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T23:10:27.861 INFO:teuthology.orchestra.run.smithi136.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T23:10:27.862 INFO:teuthology.orchestra.run.smithi136.stdout:Access: 2024-09-11 23:10:07.040137229 +0000 2024-09-11T23:10:27.862 INFO:teuthology.orchestra.run.smithi136.stdout:Modify: 2024-09-11 23:05:55.409592091 +0000 2024-09-11T23:10:27.862 INFO:teuthology.orchestra.run.smithi136.stdout:Change: 2024-09-11 23:05:55.409592091 +0000 2024-09-11T23:10:27.862 INFO:teuthology.orchestra.run.smithi136.stdout: Birth: 2024-09-11 23:05:55.409592091 +0000 2024-09-11T23:10:27.862 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-11T23:10:27.926 INFO:teuthology.orchestra.run.smithi136.stderr:1+0 records in 2024-09-11T23:10:27.926 INFO:teuthology.orchestra.run.smithi136.stderr:1+0 records out 2024-09-11T23:10:27.926 INFO:teuthology.orchestra.run.smithi136.stderr:512 bytes copied, 0.000290521 s, 1.8 MB/s 2024-09-11T23:10:27.927 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:27 smithi136 ceph-mon[35121]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:10:27.927 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:27 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:10:27.927 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:27 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:10:27.927 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:27 smithi136 ceph-mon[35121]: Reconfiguring mon.b (monmap changed)... 2024-09-11T23:10:27.927 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:27 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T23:10:27.927 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:27 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-11T23:10:27.927 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:27 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:10:27.927 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:27 smithi136 ceph-mon[35121]: Reconfiguring daemon mon.b on smithi076 2024-09-11T23:10:27.928 DEBUG:teuthology.orchestra.run.smithi136:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-11T23:10:27.986 INFO:tasks.cephadm:Deploying osd.0 on smithi026 with /dev/vg_nvme/lv_4... 2024-09-11T23:10:27.986 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-11T23:10:28.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:27 smithi026 ceph-mon[31172]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:10:28.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:27 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:10:28.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:27 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:10:28.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:27 smithi026 ceph-mon[31172]: Reconfiguring mon.b (monmap changed)... 2024-09-11T23:10:28.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:27 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T23:10:28.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:27 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-11T23:10:28.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:27 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:10:28.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:27 smithi026 ceph-mon[31172]: Reconfiguring daemon mon.b on smithi076 2024-09-11T23:10:28.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:27 smithi076 ceph-mon[35296]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:10:28.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:27 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:10:28.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:27 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:10:28.054 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:27 smithi076 ceph-mon[35296]: Reconfiguring mon.b (monmap changed)... 2024-09-11T23:10:28.054 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:27 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T23:10:28.054 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:27 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-11T23:10:28.054 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:27 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:10:28.054 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:27 smithi076 ceph-mon[35296]: Reconfiguring daemon mon.b on smithi076 2024-09-11T23:10:28.245 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:10:29.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:28 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:10:29.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:28 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:10:29.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:28 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T23:10:29.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:28 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-11T23:10:29.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:28 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:10:29.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:28 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:10:29.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:28 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:10:29.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:28 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T23:10:29.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:28 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-11T23:10:29.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:28 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:10:29.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:28 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:10:29.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:28 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:10:29.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:28 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T23:10:29.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:28 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-11T23:10:29.304 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:28 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:10:30.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:29 smithi136 ceph-mon[35121]: Reconfiguring mon.c (monmap changed)... 2024-09-11T23:10:30.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:29 smithi136 ceph-mon[35121]: Reconfiguring daemon mon.c on smithi136 2024-09-11T23:10:30.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:29 smithi136 ceph-mon[35121]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:10:30.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:29 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:10:30.240 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:29 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:10:30.240 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:29 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:10:30.240 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:29 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:10:30.240 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:29 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:10:30.240 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:29 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:10:30.240 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:29 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T23:10:30.240 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:29 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-11T23:10:30.240 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:29 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-11T23:10:30.240 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:29 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:10:30.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:29 smithi026 ceph-mon[31172]: Reconfiguring mon.c (monmap changed)... 2024-09-11T23:10:30.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:29 smithi026 ceph-mon[31172]: Reconfiguring daemon mon.c on smithi136 2024-09-11T23:10:30.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:29 smithi026 ceph-mon[31172]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:10:30.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:29 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:10:30.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:29 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:10:30.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:29 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:10:30.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:29 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:10:30.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:29 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:10:30.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:29 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:10:30.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:29 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T23:10:30.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:29 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-11T23:10:30.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:29 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-11T23:10:30.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:29 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:10:30.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:29 smithi076 ceph-mon[35296]: Reconfiguring mon.c (monmap changed)... 2024-09-11T23:10:30.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:29 smithi076 ceph-mon[35296]: Reconfiguring daemon mon.c on smithi136 2024-09-11T23:10:30.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:29 smithi076 ceph-mon[35296]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:10:30.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:29 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:10:30.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:29 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:10:30.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:29 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:10:30.304 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:29 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:10:30.304 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:29 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:10:30.304 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:29 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:10:30.304 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:29 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T23:10:30.304 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:29 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-11T23:10:30.304 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:29 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-11T23:10:30.304 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:29 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:10:30.670 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-11T23:10:30.698 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi026:vg_nvme/lv_4 2024-09-11T23:10:30.945 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Sep 11 23:10:30 smithi136 systemd[1]: Starting Ceph mgr.z for af7e6fc8-7092-11ef-bcea-c7b262605968... 2024-09-11T23:10:30.995 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:10:31.211 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:30 smithi136 ceph-mon[35121]: Deploying daemon mgr.z on smithi136 2024-09-11T23:10:31.211 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:30 smithi136 ceph-mon[35121]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:10:31.211 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Sep 11 23:10:31 smithi136 podman[36910]: 2024-09-11 23:10:30.986831074 +0000 UTC m=+0.016313946 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T23:10:31.211 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Sep 11 23:10:31 smithi136 podman[36910]: 2024-09-11 23:10:31.095021219 +0000 UTC m=+0.124504137 container create b42acef488cab28427ccf47697eaefaee6af5d6bb282cec075a039cad38038c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-z, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T23:10:31.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:30 smithi026 ceph-mon[31172]: Deploying daemon mgr.z on smithi136 2024-09-11T23:10:31.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:30 smithi026 ceph-mon[31172]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:10:31.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:30 smithi076 ceph-mon[35296]: Deploying daemon mgr.z on smithi136 2024-09-11T23:10:31.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:30 smithi076 ceph-mon[35296]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:10:31.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:10:31.476+0000 7f84e2381640 1 -- 172.21.15.26:0/414336486 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f84dc101c00 msgr2=0x7f84dc102000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:10:31.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:10:31.476+0000 7f84e2381640 1 --2- 172.21.15.26:0/414336486 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f84dc101c00 0x7f84dc102000 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f84cc009900 tx=0x7f84cc02a410 comp rx=0 tx=0).stop 2024-09-11T23:10:31.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:10:31.476+0000 7f84e2381640 1 -- 172.21.15.26:0/414336486 shutdown_connections 2024-09-11T23:10:31.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:10:31.476+0000 7f84e2381640 1 --2- 172.21.15.26:0/414336486 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f84dc102e00 0x7f84dc103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:10:31.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:10:31.476+0000 7f84e2381640 1 --2- 172.21.15.26:0/414336486 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f84dc1037c0 0x7f84dc10a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:10:31.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:10:31.476+0000 7f84e2381640 1 --2- 172.21.15.26:0/414336486 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f84dc101c00 0x7f84dc102000 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:10:31.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:10:31.476+0000 7f84e2381640 1 -- 172.21.15.26:0/414336486 >> 172.21.15.26:0/414336486 conn(0x7f84dc0fd390 msgr2=0x7f84dc0ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:10:31.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:10:31.476+0000 7f84e2381640 1 -- 172.21.15.26:0/414336486 shutdown_connections 2024-09-11T23:10:31.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:10:31.476+0000 7f84e2381640 1 -- 172.21.15.26:0/414336486 wait complete. 2024-09-11T23:10:31.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:10:31.477+0000 7f84e2381640 1 Processor -- start 2024-09-11T23:10:31.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:10:31.477+0000 7f84e2381640 1 -- start start 2024-09-11T23:10:31.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:10:31.477+0000 7f84e2381640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f84dc101c00 0x7f84dc197ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:10:31.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:10:31.477+0000 7f84e2381640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f84dc102e00 0x7f84dc198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:10:31.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:10:31.477+0000 7f84e2381640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f84dc1037c0 0x7f84dc19f490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:10:31.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:10:31.477+0000 7f84e2381640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f84dc10d890 con 0x7f84dc102e00 2024-09-11T23:10:31.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:10:31.477+0000 7f84e2381640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f84dc10d710 con 0x7f84dc1037c0 2024-09-11T23:10:31.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:10:31.477+0000 7f84e2381640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f84dc10da10 con 0x7f84dc101c00 2024-09-11T23:10:31.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:10:31.477+0000 7f84db7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f84dc102e00 0x7f84dc198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:10:31.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:10:31.477+0000 7f84dbfff640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f84dc101c00 0x7f84dc197ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:10:31.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:10:31.477+0000 7f84dbfff640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f84dc101c00 0x7f84dc197ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:51778/0 (socket says 172.21.15.26:51778) 2024-09-11T23:10:31.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:10:31.477+0000 7f84dbfff640 1 -- 172.21.15.26:0/2626718813 learned_addr learned my addr 172.21.15.26:0/2626718813 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:10:31.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:10:31.478+0000 7f84e091e640 1 --2- 172.21.15.26:0/2626718813 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f84dc1037c0 0x7f84dc19f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:10:31.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:10:31.478+0000 7f84dbfff640 1 -- 172.21.15.26:0/2626718813 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f84dc1037c0 msgr2=0x7f84dc19f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:10:31.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:10:31.478+0000 7f84dbfff640 1 --2- 172.21.15.26:0/2626718813 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f84dc1037c0 0x7f84dc19f490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:10:31.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:10:31.478+0000 7f84dbfff640 1 -- 172.21.15.26:0/2626718813 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f84dc102e00 msgr2=0x7f84dc198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:10:31.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:10:31.478+0000 7f84dbfff640 1 --2- 172.21.15.26:0/2626718813 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f84dc102e00 0x7f84dc198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:10:31.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:10:31.478+0000 7f84dbfff640 1 -- 172.21.15.26:0/2626718813 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84dc19fb00 con 0x7f84dc101c00 2024-09-11T23:10:31.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:10:31.478+0000 7f84db7fe640 1 --2- 172.21.15.26:0/2626718813 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f84dc102e00 0x7f84dc198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:10:31.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:10:31.478+0000 7f84e091e640 1 --2- 172.21.15.26:0/2626718813 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f84dc1037c0 0x7f84dc19f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:10:31.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:10:31.478+0000 7f84dbfff640 1 --2- 172.21.15.26:0/2626718813 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f84dc101c00 0x7f84dc197ed0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f84cc031610 tx=0x7f84cc004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:10:31.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:10:31.478+0000 7f84d97fa640 1 -- 172.21.15.26:0/2626718813 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84cc03f030 con 0x7f84dc101c00 2024-09-11T23:10:31.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:10:31.478+0000 7f84d97fa640 1 -- 172.21.15.26:0/2626718813 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84cc003ad0 con 0x7f84dc101c00 2024-09-11T23:10:31.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:10:31.478+0000 7f84e2381640 1 -- 172.21.15.26:0/2626718813 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84dc19fd90 con 0x7f84dc101c00 2024-09-11T23:10:31.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:10:31.478+0000 7f84d97fa640 1 -- 172.21.15.26:0/2626718813 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84cc035070 con 0x7f84dc101c00 2024-09-11T23:10:31.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:10:31.478+0000 7f84e2381640 1 -- 172.21.15.26:0/2626718813 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84dc1a01c0 con 0x7f84dc101c00 2024-09-11T23:10:31.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:10:31.479+0000 7f84d97fa640 1 -- 172.21.15.26:0/2626718813 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f84cc042030 con 0x7f84dc101c00 2024-09-11T23:10:31.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:10:31.480+0000 7f84e2381640 1 -- 172.21.15.26:0/2626718813 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84dc108fa0 con 0x7f84dc101c00 2024-09-11T23:10:31.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:10:31.480+0000 7f84d97fa640 1 --2- 172.21.15.26:0/2626718813 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f84c40392c0 0x7f84c403b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:10:31.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:10:31.480+0000 7f84d97fa640 1 -- 172.21.15.26:0/2626718813 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f84cc06cb40 con 0x7f84dc101c00 2024-09-11T23:10:31.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:10:31.480+0000 7f84db7fe640 1 --2- 172.21.15.26:0/2626718813 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f84c40392c0 0x7f84c403b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:10:31.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:10:31.480+0000 7f84db7fe640 1 --2- 172.21.15.26:0/2626718813 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f84c40392c0 0x7f84c403b780 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f84dc1993f0 tx=0x7f84d0005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:10:31.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:10:31.485+0000 7f84d97fa640 1 -- 172.21.15.26:0/2626718813 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f84cc03ac70 con 0x7f84dc101c00 2024-09-11T23:10:31.641 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Sep 11 23:10:31 smithi136 podman[36910]: 2024-09-11 23:10:31.307855892 +0000 UTC m=+0.337338764 container init b42acef488cab28427ccf47697eaefaee6af5d6bb282cec075a039cad38038c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-z, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-09-11T23:10:31.641 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Sep 11 23:10:31 smithi136 podman[36910]: 2024-09-11 23:10:31.310729076 +0000 UTC m=+0.340211955 container start b42acef488cab28427ccf47697eaefaee6af5d6bb282cec075a039cad38038c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-z, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=) 2024-09-11T23:10:31.642 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Sep 11 23:10:31 smithi136 ceph-mgr[36929]: -- 172.21.15.136:0/831332593 <== mon.1 v2:172.21.15.136:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562dd2bb90e0 con 0x562dd2b96800 2024-09-11T23:10:31.642 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Sep 11 23:10:31 smithi136 bash[36910]: b42acef488cab28427ccf47697eaefaee6af5d6bb282cec075a039cad38038c8 2024-09-11T23:10:31.642 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Sep 11 23:10:31 smithi136 systemd[1]: Started Ceph mgr.z for af7e6fc8-7092-11ef-bcea-c7b262605968. 2024-09-11T23:10:31.642 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Sep 11 23:10:31 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-z[36925]: 2024-09-11T23:10:31.433+0000 7fabed7fe040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-11T23:10:31.642 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Sep 11 23:10:31 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-z[36925]: 2024-09-11T23:10:31.492+0000 7fabed7fe040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-11T23:10:31.642 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Sep 11 23:10:31 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-z[36925]: 2024-09-11T23:10:31.641+0000 7fabed7fe040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-11T23:10:31.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:10:31.670+0000 7f84e2381640 1 -- 172.21.15.26:0/2626718813 --> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi026:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f84dc062630 con 0x7f84c40392c0 2024-09-11T23:10:31.989 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Sep 11 23:10:31 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-z[36925]: 2024-09-11T23:10:31.707+0000 7fabed7fe040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-11T23:10:31.989 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Sep 11 23:10:31 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-z[36925]: 2024-09-11T23:10:31.854+0000 7fabed7fe040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-11T23:10:32.311 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Sep 11 23:10:31 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-z[36925]: 2024-09-11T23:10:31.995+0000 7fabed7fe040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-11T23:10:32.311 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Sep 11 23:10:32 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-z[36925]: 2024-09-11T23:10:32.310+0000 7fabed7fe040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-11T23:10:32.553 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:32 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:10:32.553 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:32 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:10:32.553 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:32 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:10:32.553 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:32 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T23:10:32.553 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:32 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-11T23:10:32.553 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:32 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-11T23:10:32.554 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:32 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:10:32.554 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:32 smithi076 ceph-mon[35296]: Deploying daemon mgr.y on smithi076 2024-09-11T23:10:32.554 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:32 smithi076 ceph-mon[35296]: from='client.24115 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi026:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:10:32.554 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:32 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T23:10:32.554 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:32 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T23:10:32.554 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:32 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:10:32.739 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Sep 11 23:10:32 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-z[36925]: 2024-09-11T23:10:32.371+0000 7fabed7fe040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-11T23:10:32.739 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Sep 11 23:10:32 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-z[36925]: 2024-09-11T23:10:32.556+0000 7fabed7fe040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-11T23:10:32.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:32 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:10:32.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:32 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:10:32.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:32 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:10:32.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:32 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T23:10:32.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:32 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-11T23:10:32.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:32 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-11T23:10:32.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:32 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:10:32.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:32 smithi136 ceph-mon[35121]: Deploying daemon mgr.y on smithi076 2024-09-11T23:10:32.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:32 smithi136 ceph-mon[35121]: from='client.24115 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi026:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:10:32.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:32 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T23:10:32.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:32 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T23:10:32.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:32 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:10:32.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:32 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:10:32.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:32 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:10:32.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:32 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:10:32.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:32 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T23:10:32.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:32 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-11T23:10:32.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:32 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-11T23:10:32.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:32 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:10:32.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:32 smithi026 ceph-mon[31172]: Deploying daemon mgr.y on smithi076 2024-09-11T23:10:32.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:32 smithi026 ceph-mon[31172]: from='client.24115 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi026:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:10:32.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:32 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T23:10:32.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:32 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T23:10:32.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:32 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:10:33.303 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Sep 11 23:10:33 smithi076 systemd[1]: Starting Ceph mgr.y for af7e6fc8-7092-11ef-bcea-c7b262605968... 2024-09-11T23:10:33.561 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:33 smithi026 ceph-mon[31172]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:10:33.718 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:33 smithi076 ceph-mon[35296]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:10:33.719 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Sep 11 23:10:33 smithi076 podman[36643]: 2024-09-11 23:10:33.416904748 +0000 UTC m=+0.111736063 container create 39fb8e0d330c81438cc8612ce0eb8d7c480ef27e87c9e38ba0338b0fc9d398ed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-y, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2) 2024-09-11T23:10:33.719 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Sep 11 23:10:33 smithi076 podman[36643]: 2024-09-11 23:10:33.320852886 +0000 UTC m=+0.015684211 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T23:10:33.719 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Sep 11 23:10:33 smithi076 podman[36643]: 2024-09-11 23:10:33.61925272 +0000 UTC m=+0.314084036 container init 39fb8e0d330c81438cc8612ce0eb8d7c480ef27e87c9e38ba0338b0fc9d398ed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-y, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2) 2024-09-11T23:10:33.719 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Sep 11 23:10:33 smithi076 podman[36643]: 2024-09-11 23:10:33.622074485 +0000 UTC m=+0.316905803 container start 39fb8e0d330c81438cc8612ce0eb8d7c480ef27e87c9e38ba0338b0fc9d398ed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-y, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.schema-version=1.0) 2024-09-11T23:10:33.719 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Sep 11 23:10:33 smithi076 ceph-mgr[36662]: -- 172.21.15.76:0/2806173107 <== mon.2 v2:172.21.15.76:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x565129c310e0 con 0x565129c0e800 2024-09-11T23:10:33.719 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Sep 11 23:10:33 smithi076 bash[36643]: 39fb8e0d330c81438cc8612ce0eb8d7c480ef27e87c9e38ba0338b0fc9d398ed 2024-09-11T23:10:33.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:33 smithi136 ceph-mon[35121]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:10:33.740 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Sep 11 23:10:33 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-z[36925]: 2024-09-11T23:10:33.438+0000 7fabed7fe040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-11T23:10:33.740 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Sep 11 23:10:33 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-z[36925]: 2024-09-11T23:10:33.506+0000 7fabed7fe040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-11T23:10:34.026 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Sep 11 23:10:33 smithi076 systemd[1]: Started Ceph mgr.y for af7e6fc8-7092-11ef-bcea-c7b262605968. 2024-09-11T23:10:34.026 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Sep 11 23:10:33 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-y[36658]: 2024-09-11T23:10:33.741+0000 7fe748bf4040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-11T23:10:34.026 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Sep 11 23:10:33 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-y[36658]: 2024-09-11T23:10:33.802+0000 7fe748bf4040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-11T23:10:34.026 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Sep 11 23:10:33 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-y[36658]: 2024-09-11T23:10:33.955+0000 7fe748bf4040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-11T23:10:34.026 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Sep 11 23:10:34 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-y[36658]: 2024-09-11T23:10:34.025+0000 7fe748bf4040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-11T23:10:34.303 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Sep 11 23:10:34 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-y[36658]: 2024-09-11T23:10:34.183+0000 7fe748bf4040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-11T23:10:34.514 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Sep 11 23:10:34 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-z[36925]: 2024-09-11T23:10:34.247+0000 7fabed7fe040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-11T23:10:34.514 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Sep 11 23:10:34 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-z[36925]: 2024-09-11T23:10:34.368+0000 7fabed7fe040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-11T23:10:34.514 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Sep 11 23:10:34 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-z[36925]: 2024-09-11T23:10:34.513+0000 7fabed7fe040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-11T23:10:34.637 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Sep 11 23:10:34 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-y[36658]: 2024-09-11T23:10:34.341+0000 7fe748bf4040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-11T23:10:34.787 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Sep 11 23:10:34 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-z[36925]: 2024-09-11T23:10:34.589+0000 7fabed7fe040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-11T23:10:34.788 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Sep 11 23:10:34 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-z[36925]: 2024-09-11T23:10:34.740+0000 7fabed7fe040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-11T23:10:34.915 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Sep 11 23:10:34 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-y[36658]: 2024-09-11T23:10:34.666+0000 7fe748bf4040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-11T23:10:34.915 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Sep 11 23:10:34 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-y[36658]: 2024-09-11T23:10:34.726+0000 7fe748bf4040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-11T23:10:34.915 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:34 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:10:34.915 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:34 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:10:34.915 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:34 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:10:34.915 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:34 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:10:34.915 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:34 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:10:34.915 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:34 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:10:34.915 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:34 smithi076 ceph-mon[35296]: from='client.? 172.21.15.26:0/2772353343' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a4d62fcf-c31e-41b1-932c-3d87ff640531"}]: dispatch 2024-09-11T23:10:34.916 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:34 smithi076 ceph-mon[35296]: from='client.? 172.21.15.26:0/2772353343' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a4d62fcf-c31e-41b1-932c-3d87ff640531"}]': finished 2024-09-11T23:10:34.916 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:34 smithi076 ceph-mon[35296]: osdmap e5: 1 total, 0 up, 1 in 2024-09-11T23:10:34.916 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:34 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T23:10:34.916 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:34 smithi076 ceph-mon[35296]: from='client.? 172.21.15.26:0/4258991865' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T23:10:35.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:34 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:10:35.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:34 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:10:35.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:34 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:10:35.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:34 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:10:35.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:34 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:10:35.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:34 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:10:35.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:34 smithi026 ceph-mon[31172]: from='client.? 172.21.15.26:0/2772353343' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a4d62fcf-c31e-41b1-932c-3d87ff640531"}]: dispatch 2024-09-11T23:10:35.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:34 smithi026 ceph-mon[31172]: from='client.? 172.21.15.26:0/2772353343' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a4d62fcf-c31e-41b1-932c-3d87ff640531"}]': finished 2024-09-11T23:10:35.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:34 smithi026 ceph-mon[31172]: osdmap e5: 1 total, 0 up, 1 in 2024-09-11T23:10:35.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:34 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T23:10:35.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:34 smithi026 ceph-mon[31172]: from='client.? 172.21.15.26:0/4258991865' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T23:10:35.042 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Sep 11 23:10:34 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-z[36925]: 2024-09-11T23:10:34.811+0000 7fabed7fe040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-11T23:10:35.043 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Sep 11 23:10:34 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-z[36925]: 2024-09-11T23:10:34.871+0000 7fabed7fe040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-11T23:10:35.043 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Sep 11 23:10:35 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-z[36925]: 2024-09-11T23:10:35.042+0000 7fabed7fe040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-11T23:10:35.043 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:34 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:10:35.043 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:34 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:10:35.043 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:34 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:10:35.043 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:34 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:10:35.043 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:34 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:10:35.043 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:34 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:10:35.043 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:34 smithi136 ceph-mon[35121]: from='client.? 172.21.15.26:0/2772353343' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a4d62fcf-c31e-41b1-932c-3d87ff640531"}]: dispatch 2024-09-11T23:10:35.043 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:34 smithi136 ceph-mon[35121]: from='client.? 172.21.15.26:0/2772353343' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a4d62fcf-c31e-41b1-932c-3d87ff640531"}]': finished 2024-09-11T23:10:35.044 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:34 smithi136 ceph-mon[35121]: osdmap e5: 1 total, 0 up, 1 in 2024-09-11T23:10:35.044 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:34 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T23:10:35.044 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:34 smithi136 ceph-mon[35121]: from='client.? 172.21.15.26:0/4258991865' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T23:10:35.303 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Sep 11 23:10:34 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-y[36658]: 2024-09-11T23:10:34.913+0000 7fe748bf4040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-11T23:10:35.323 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Sep 11 23:10:35 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-z[36925]: 2024-09-11T23:10:35.106+0000 7fabed7fe040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-11T23:10:35.323 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Sep 11 23:10:35 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-z[36925]: 2024-09-11T23:10:35.176+0000 7fabed7fe040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-11T23:10:35.323 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Sep 11 23:10:35 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-z[36925]: 2024-09-11T23:10:35.261+0000 7fabed7fe040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-11T23:10:35.324 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Sep 11 23:10:35 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-z[36925]: 2024-09-11T23:10:35.321+0000 7fabed7fe040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-11T23:10:35.739 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Sep 11 23:10:35 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-z[36925]: 2024-09-11T23:10:35.379+0000 7fabed7fe040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-11T23:10:36.165 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:35 smithi136 ceph-mon[35121]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:10:36.165 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:35 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:10:36.166 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:35 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:10:36.166 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:35 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:10:36.166 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:35 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:10:36.166 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:35 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:10:36.166 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:35 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:10:36.166 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:35 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:10:36.166 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:35 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:10:36.166 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:35 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:10:36.166 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:35 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:10:36.166 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:35 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:10:36.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:35 smithi026 ceph-mon[31172]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:10:36.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:35 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:10:36.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:35 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:10:36.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:35 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:10:36.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:35 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:10:36.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:35 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:10:36.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:35 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:10:36.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:35 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:10:36.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:35 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:10:36.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:35 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:10:36.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:35 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:10:36.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:35 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:10:36.303 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Sep 11 23:10:35 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-y[36658]: 2024-09-11T23:10:35.820+0000 7fe748bf4040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-11T23:10:36.303 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Sep 11 23:10:35 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-y[36658]: 2024-09-11T23:10:35.886+0000 7fe748bf4040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-11T23:10:36.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:35 smithi076 ceph-mon[35296]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:10:36.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:35 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:10:36.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:35 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:10:36.304 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:35 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:10:36.304 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:35 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:10:36.304 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:35 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:10:36.304 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:35 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:10:36.304 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:35 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:10:36.304 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:35 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:10:36.304 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:35 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:10:36.304 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:35 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:10:36.304 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:35 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:10:36.489 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Sep 11 23:10:36 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-z[36925]: 2024-09-11T23:10:36.165+0000 7fabed7fe040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-11T23:10:36.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:10:36.811+0000 7f84d97fa640 1 -- 172.21.15.26:0/2626718813 <== mon.1 v2:172.21.15.136:3300/0 7 ==== mgrmap(e 14) ==== 92060+0+0 (secure 0 0 0) 0x7f84cc03b460 con 0x7f84dc101c00 2024-09-11T23:10:36.857 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:36 smithi076 ceph-mon[35296]: Standby manager daemon z started 2024-09-11T23:10:36.857 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:36 smithi076 ceph-mon[35296]: from='mgr.? 172.21.15.136:0/2687286940' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-11T23:10:36.857 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:36 smithi076 ceph-mon[35296]: from='mgr.? 172.21.15.136:0/2687286940' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-11T23:10:36.857 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:36 smithi076 ceph-mon[35296]: from='mgr.? 172.21.15.136:0/2687286940' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-11T23:10:36.858 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:36 smithi076 ceph-mon[35296]: from='mgr.? 172.21.15.136:0/2687286940' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-11T23:10:36.858 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Sep 11 23:10:36 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-y[36658]: 2024-09-11T23:10:36.598+0000 7fe748bf4040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-11T23:10:36.858 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Sep 11 23:10:36 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-y[36658]: 2024-09-11T23:10:36.714+0000 7fe748bf4040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-11T23:10:36.858 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Sep 11 23:10:36 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-y[36658]: 2024-09-11T23:10:36.856+0000 7fe748bf4040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-11T23:10:37.131 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Sep 11 23:10:36 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-y[36658]: 2024-09-11T23:10:36.917+0000 7fe748bf4040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-11T23:10:37.131 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Sep 11 23:10:37 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-y[36658]: 2024-09-11T23:10:37.063+0000 7fe748bf4040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-11T23:10:37.131 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Sep 11 23:10:37 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-y[36658]: 2024-09-11T23:10:37.131+0000 7fe748bf4040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-11T23:10:37.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:36 smithi136 ceph-mon[35121]: Standby manager daemon z started 2024-09-11T23:10:37.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:36 smithi136 ceph-mon[35121]: from='mgr.? 172.21.15.136:0/2687286940' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-11T23:10:37.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:36 smithi136 ceph-mon[35121]: from='mgr.? 172.21.15.136:0/2687286940' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-11T23:10:37.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:36 smithi136 ceph-mon[35121]: from='mgr.? 172.21.15.136:0/2687286940' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-11T23:10:37.240 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:36 smithi136 ceph-mon[35121]: from='mgr.? 172.21.15.136:0/2687286940' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-11T23:10:37.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:36 smithi026 ceph-mon[31172]: Standby manager daemon z started 2024-09-11T23:10:37.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:36 smithi026 ceph-mon[31172]: from='mgr.? 172.21.15.136:0/2687286940' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-11T23:10:37.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:36 smithi026 ceph-mon[31172]: from='mgr.? 172.21.15.136:0/2687286940' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-11T23:10:37.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:36 smithi026 ceph-mon[31172]: from='mgr.? 172.21.15.136:0/2687286940' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-11T23:10:37.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:36 smithi026 ceph-mon[31172]: from='mgr.? 172.21.15.136:0/2687286940' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-11T23:10:37.414 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Sep 11 23:10:37 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-y[36658]: 2024-09-11T23:10:37.191+0000 7fe748bf4040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-11T23:10:37.415 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Sep 11 23:10:37 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-y[36658]: 2024-09-11T23:10:37.349+0000 7fe748bf4040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-11T23:10:37.415 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Sep 11 23:10:37 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-y[36658]: 2024-09-11T23:10:37.414+0000 7fe748bf4040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-11T23:10:37.681 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Sep 11 23:10:37 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-y[36658]: 2024-09-11T23:10:37.480+0000 7fe748bf4040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-11T23:10:37.681 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Sep 11 23:10:37 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-y[36658]: 2024-09-11T23:10:37.562+0000 7fe748bf4040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-11T23:10:37.681 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Sep 11 23:10:37 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-y[36658]: 2024-09-11T23:10:37.620+0000 7fe748bf4040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-11T23:10:37.681 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Sep 11 23:10:37 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-y[36658]: 2024-09-11T23:10:37.680+0000 7fe748bf4040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-11T23:10:38.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:37 smithi076 ceph-mon[35296]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:10:38.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:37 smithi076 ceph-mon[35296]: mgrmap e14: x(active, since 62s), standbys: z 2024-09-11T23:10:38.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:37 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-11T23:10:38.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:37 smithi136 ceph-mon[35121]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:10:38.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:37 smithi136 ceph-mon[35121]: mgrmap e14: x(active, since 62s), standbys: z 2024-09-11T23:10:38.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:37 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-11T23:10:38.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:37 smithi026 ceph-mon[31172]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:10:38.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:37 smithi026 ceph-mon[31172]: mgrmap e14: x(active, since 62s), standbys: z 2024-09-11T23:10:38.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:37 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-11T23:10:38.803 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Sep 11 23:10:38 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-y[36658]: 2024-09-11T23:10:38.465+0000 7fe748bf4040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-11T23:10:38.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:10:38.819+0000 7f84d97fa640 1 -- 172.21.15.26:0/2626718813 <== mon.1 v2:172.21.15.136:3300/0 8 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7f84cc0433c0 con 0x7f84dc101c00 2024-09-11T23:10:39.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:38 smithi136 ceph-mon[35121]: Standby manager daemon y started 2024-09-11T23:10:39.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:38 smithi136 ceph-mon[35121]: from='mgr.? 172.21.15.76:0/3446560674' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-11T23:10:39.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:38 smithi136 ceph-mon[35121]: from='mgr.? 172.21.15.76:0/3446560674' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-11T23:10:39.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:38 smithi136 ceph-mon[35121]: from='mgr.? 172.21.15.76:0/3446560674' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-11T23:10:39.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:38 smithi136 ceph-mon[35121]: from='mgr.? 172.21.15.76:0/3446560674' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-11T23:10:39.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:38 smithi026 ceph-mon[31172]: Standby manager daemon y started 2024-09-11T23:10:39.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:38 smithi026 ceph-mon[31172]: from='mgr.? 172.21.15.76:0/3446560674' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-11T23:10:39.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:38 smithi026 ceph-mon[31172]: from='mgr.? 172.21.15.76:0/3446560674' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-11T23:10:39.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:38 smithi026 ceph-mon[31172]: from='mgr.? 172.21.15.76:0/3446560674' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-11T23:10:39.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:38 smithi026 ceph-mon[31172]: from='mgr.? 172.21.15.76:0/3446560674' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-11T23:10:39.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:38 smithi076 ceph-mon[35296]: Standby manager daemon y started 2024-09-11T23:10:39.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:38 smithi076 ceph-mon[35296]: from='mgr.? 172.21.15.76:0/3446560674' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-11T23:10:39.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:38 smithi076 ceph-mon[35296]: from='mgr.? 172.21.15.76:0/3446560674' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-11T23:10:39.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:38 smithi076 ceph-mon[35296]: from='mgr.? 172.21.15.76:0/3446560674' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-11T23:10:39.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:38 smithi076 ceph-mon[35296]: from='mgr.? 172.21.15.76:0/3446560674' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-11T23:10:40.126 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:39 smithi026 ceph-mon[31172]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:10:40.126 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:39 smithi026 ceph-mon[31172]: mgrmap e15: x(active, since 64s), standbys: z, y 2024-09-11T23:10:40.126 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:39 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-11T23:10:40.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:39 smithi136 ceph-mon[35121]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:10:40.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:39 smithi136 ceph-mon[35121]: mgrmap e15: x(active, since 64s), standbys: z, y 2024-09-11T23:10:40.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:39 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-11T23:10:40.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:39 smithi076 ceph-mon[35296]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:10:40.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:39 smithi076 ceph-mon[35296]: mgrmap e15: x(active, since 64s), standbys: z, y 2024-09-11T23:10:40.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:39 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-11T23:10:42.077 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:41 smithi026 ceph-mon[31172]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:10:42.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:41 smithi136 ceph-mon[35121]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:10:42.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:41 smithi076 ceph-mon[35296]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:10:43.141 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:42 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-11T23:10:43.238 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:42 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:10:43.239 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:42 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-11T23:10:43.239 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:42 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:10:43.239 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:42 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-11T23:10:43.239 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:42 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:10:44.220 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:43 smithi026 ceph-mon[31172]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:10:44.221 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:43 smithi026 ceph-mon[31172]: Deploying daemon osd.0 on smithi026 2024-09-11T23:10:44.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:43 smithi136 ceph-mon[35121]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:10:44.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:43 smithi136 ceph-mon[35121]: Deploying daemon osd.0 on smithi026 2024-09-11T23:10:44.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:43 smithi076 ceph-mon[35296]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:10:44.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:43 smithi076 ceph-mon[35296]: Deploying daemon osd.0 on smithi026 2024-09-11T23:10:45.917 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:45 smithi026 ceph-mon[31172]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:10:46.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:45 smithi136 ceph-mon[35121]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:10:46.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:45 smithi076 ceph-mon[35296]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:10:48.123 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:47 smithi026 ceph-mon[31172]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:10:48.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:47 smithi136 ceph-mon[35121]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:10:48.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:47 smithi076 ceph-mon[35296]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:10:50.158 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:49 smithi026 ceph-mon[31172]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:10:50.158 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:49 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:10:50.158 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:49 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:10:50.158 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:49 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:10:50.241 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:49 smithi136 ceph-mon[35121]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:10:50.241 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:49 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:10:50.241 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:49 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:10:50.241 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:49 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:10:50.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:49 smithi076 ceph-mon[35296]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:10:50.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:49 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:10:50.304 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:49 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:10:50.304 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:49 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:10:52.118 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:51 smithi026 ceph-mon[31172]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:10:52.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:51 smithi136 ceph-mon[35121]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:10:52.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:51 smithi076 ceph-mon[35296]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:10:53.304 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:53 smithi076 ceph-mon[35296]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:10:53.304 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:53 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:10:53.304 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:53 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:10:53.304 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:53 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:10:53.304 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:53 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:10:53.305 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:53 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:10:53.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:53 smithi136 ceph-mon[35121]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:10:53.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:53 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:10:53.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:53 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:10:53.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:53 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:10:53.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:53 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:10:53.490 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:53 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:10:53.501 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:53 smithi026 ceph-mon[31172]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:10:53.501 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:53 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:10:53.501 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:53 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:10:53.501 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:53 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:10:53.501 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:53 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:10:53.501 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:53 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:10:53.925 INFO:teuthology.orchestra.run.smithi026.stdout:Created osd(s) 0 on host 'smithi026' 2024-09-11T23:10:53.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:10:53.922+0000 7f84d97fa640 1 -- 172.21.15.26:0/2626718813 <== mgr.14148 v2:172.21.15.26:6800/110833859 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f84dc062630 con 0x7f84c40392c0 2024-09-11T23:10:53.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:10:53.924+0000 7f84e2381640 1 -- 172.21.15.26:0/2626718813 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f84c40392c0 msgr2=0x7f84c403b780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:10:53.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:10:53.924+0000 7f84e2381640 1 --2- 172.21.15.26:0/2626718813 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f84c40392c0 0x7f84c403b780 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f84dc1993f0 tx=0x7f84d0005f90 comp rx=0 tx=0).stop 2024-09-11T23:10:53.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:10:53.924+0000 7f84e2381640 1 -- 172.21.15.26:0/2626718813 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f84dc101c00 msgr2=0x7f84dc197ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:10:53.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:10:53.924+0000 7f84e2381640 1 --2- 172.21.15.26:0/2626718813 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f84dc101c00 0x7f84dc197ed0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f84cc031610 tx=0x7f84cc004270 comp rx=0 tx=0).stop 2024-09-11T23:10:53.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:10:53.924+0000 7f84e2381640 1 -- 172.21.15.26:0/2626718813 shutdown_connections 2024-09-11T23:10:53.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:10:53.924+0000 7f84e2381640 1 --2- 172.21.15.26:0/2626718813 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f84c40392c0 0x7f84c403b780 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:10:53.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:10:53.924+0000 7f84e2381640 1 --2- 172.21.15.26:0/2626718813 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f84dc1037c0 0x7f84dc19f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:10:53.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:10:53.924+0000 7f84e2381640 1 --2- 172.21.15.26:0/2626718813 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f84dc102e00 0x7f84dc198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:10:53.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:10:53.924+0000 7f84e2381640 1 --2- 172.21.15.26:0/2626718813 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f84dc101c00 0x7f84dc197ed0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:10:53.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:10:53.924+0000 7f84e2381640 1 -- 172.21.15.26:0/2626718813 >> 172.21.15.26:0/2626718813 conn(0x7f84dc0fd390 msgr2=0x7f84dc106170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:10:53.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:10:53.924+0000 7f84e2381640 1 -- 172.21.15.26:0/2626718813 shutdown_connections 2024-09-11T23:10:53.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:10:53.924+0000 7f84e2381640 1 -- 172.21.15.26:0/2626718813 wait complete. 2024-09-11T23:10:54.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:54 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:10:54.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:54 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:10:54.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:54 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:10:54.249 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:54 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:10:54.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:54 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:10:54.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:54 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:10:54.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:54 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:10:54.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:54 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:10:54.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:54 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:10:54.601 DEBUG:teuthology.orchestra.run.smithi026:osd.0> sudo journalctl -f -n 0 -u ceph-af7e6fc8-7092-11ef-bcea-c7b262605968@osd.0.service 2024-09-11T23:10:54.604 INFO:tasks.cephadm:Deploying osd.1 on smithi076 with /dev/vg_nvme/lv_4... 2024-09-11T23:10:54.604 DEBUG:teuthology.orchestra.run.smithi076:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-11T23:10:54.845 INFO:teuthology.orchestra.run.smithi076.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T23:10:55.748 INFO:journalctl@ceph.osd.0.smithi026.stdout:Sep 11 23:10:55 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-0[44903]: 2024-09-11T23:10:55.632+0000 7f201da44640 -1 osd.0 0 log_to_monitors true 2024-09-11T23:10:56.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:55 smithi026 ceph-mon[31172]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:10:56.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:55 smithi026 ceph-mon[31172]: from='osd.0 [v2:172.21.15.26:6802/2014857905,v1:172.21.15.26:6803/2014857905]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-11T23:10:56.206 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:55 smithi076 ceph-mon[35296]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:10:56.206 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:55 smithi076 ceph-mon[35296]: from='osd.0 [v2:172.21.15.26:6802/2014857905,v1:172.21.15.26:6803/2014857905]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-11T23:10:56.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:55 smithi136 ceph-mon[35121]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:10:56.240 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:55 smithi136 ceph-mon[35121]: from='osd.0 [v2:172.21.15.26:6802/2014857905,v1:172.21.15.26:6803/2014857905]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-11T23:10:57.162 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:56 smithi026 ceph-mon[31172]: from='osd.0 [v2:172.21.15.26:6802/2014857905,v1:172.21.15.26:6803/2014857905]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-11T23:10:57.163 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:56 smithi026 ceph-mon[31172]: osdmap e6: 1 total, 0 up, 1 in 2024-09-11T23:10:57.163 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:56 smithi026 ceph-mon[31172]: from='osd.0 [v2:172.21.15.26:6802/2014857905,v1:172.21.15.26:6803/2014857905]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2024-09-11T23:10:57.163 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:56 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T23:10:57.163 INFO:journalctl@ceph.osd.0.smithi026.stdout:Sep 11 23:10:56 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-0[44903]: 2024-09-11T23:10:56.936+0000 7f20191a4640 -1 osd.0 0 waiting for initial osdmap 2024-09-11T23:10:57.163 INFO:journalctl@ceph.osd.0.smithi026.stdout:Sep 11 23:10:56 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-0[44903]: 2024-09-11T23:10:56.939+0000 7f20157de640 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T23:10:57.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:56 smithi136 ceph-mon[35121]: from='osd.0 [v2:172.21.15.26:6802/2014857905,v1:172.21.15.26:6803/2014857905]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-11T23:10:57.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:56 smithi136 ceph-mon[35121]: osdmap e6: 1 total, 0 up, 1 in 2024-09-11T23:10:57.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:56 smithi136 ceph-mon[35121]: from='osd.0 [v2:172.21.15.26:6802/2014857905,v1:172.21.15.26:6803/2014857905]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2024-09-11T23:10:57.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:56 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T23:10:57.292 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:56 smithi076 ceph-mon[35296]: from='osd.0 [v2:172.21.15.26:6802/2014857905,v1:172.21.15.26:6803/2014857905]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-11T23:10:57.292 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:56 smithi076 ceph-mon[35296]: osdmap e6: 1 total, 0 up, 1 in 2024-09-11T23:10:57.292 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:56 smithi076 ceph-mon[35296]: from='osd.0 [v2:172.21.15.26:6802/2014857905,v1:172.21.15.26:6803/2014857905]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2024-09-11T23:10:57.292 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:56 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T23:10:57.294 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-11T23:10:57.322 DEBUG:teuthology.orchestra.run.smithi076:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi076:vg_nvme/lv_4 2024-09-11T23:10:57.578 INFO:teuthology.orchestra.run.smithi076.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T23:10:58.127 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:10:58.120+0000 7f97b0bdf640 1 -- 172.21.15.76:0/407272668 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f97ac102690 msgr2=0x7f97ac108f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:10:58.127 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:10:58.120+0000 7f97b0bdf640 1 --2- 172.21.15.76:0/407272668 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f97ac102690 0x7f97ac108f20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f97a4009ca0 tx=0x7f97a402a8c0 comp rx=0 tx=0).stop 2024-09-11T23:10:58.128 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:10:58.120+0000 7f97b0bdf640 1 -- 172.21.15.76:0/407272668 shutdown_connections 2024-09-11T23:10:58.128 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:10:58.120+0000 7f97b0bdf640 1 --2- 172.21.15.76:0/407272668 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f97ac102690 0x7f97ac108f20 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:10:58.128 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:10:58.120+0000 7f97b0bdf640 1 --2- 172.21.15.76:0/407272668 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f97ac101cd0 0x7f97ac102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:10:58.128 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:10:58.120+0000 7f97b0bdf640 1 --2- 172.21.15.76:0/407272668 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f97ac100ad0 0x7f97ac100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:10:58.128 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:10:58.120+0000 7f97b0bdf640 1 -- 172.21.15.76:0/407272668 >> 172.21.15.76:0/407272668 conn(0x7f97ac0fc280 msgr2=0x7f97ac0fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:10:58.128 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:10:58.120+0000 7f97b0bdf640 1 -- 172.21.15.76:0/407272668 shutdown_connections 2024-09-11T23:10:58.128 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:10:58.120+0000 7f97b0bdf640 1 -- 172.21.15.76:0/407272668 wait complete. 2024-09-11T23:10:58.128 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:10:58.121+0000 7f97b0bdf640 1 Processor -- start 2024-09-11T23:10:58.128 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:10:58.121+0000 7f97b0bdf640 1 -- start start 2024-09-11T23:10:58.128 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:10:58.121+0000 7f97b0bdf640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f97ac100ad0 0x7f97ac06aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:10:58.128 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:10:58.121+0000 7f97b0bdf640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f97ac101cd0 0x7f97ac19cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:10:58.128 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:10:58.121+0000 7f97b0bdf640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f97ac102690 0x7f97ac19f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:10:58.129 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:10:58.121+0000 7f97b0bdf640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f97ac10c7a0 con 0x7f97ac100ad0 2024-09-11T23:10:58.129 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:10:58.121+0000 7f97b0bdf640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f97ac10c620 con 0x7f97ac102690 2024-09-11T23:10:58.129 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:10:58.121+0000 7f97b0bdf640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f97ac10c920 con 0x7f97ac101cd0 2024-09-11T23:10:58.129 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:10:58.121+0000 7f97aad9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f97ac102690 0x7f97ac19f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:10:58.129 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:10:58.121+0000 7f97aa59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f97ac100ad0 0x7f97ac06aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:10:58.129 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:10:58.121+0000 7f97aad9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f97ac102690 0x7f97ac19f1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51146/0 (socket says 172.21.15.76:51146) 2024-09-11T23:10:58.129 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:10:58.121+0000 7f97aad9d640 1 -- 172.21.15.76:0/4229077296 learned_addr learned my addr 172.21.15.76:0/4229077296 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-11T23:10:58.129 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:10:58.121+0000 7f97a9d9b640 1 --2- 172.21.15.76:0/4229077296 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f97ac101cd0 0x7f97ac19cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:10:58.129 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:10:58.122+0000 7f97aad9d640 1 -- 172.21.15.76:0/4229077296 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f97ac101cd0 msgr2=0x7f97ac19cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:10:58.129 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:10:58.122+0000 7f97aad9d640 1 --2- 172.21.15.76:0/4229077296 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f97ac101cd0 0x7f97ac19cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:10:58.129 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:10:58.122+0000 7f97aad9d640 1 -- 172.21.15.76:0/4229077296 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f97ac100ad0 msgr2=0x7f97ac06aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:10:58.129 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:10:58.122+0000 7f97aad9d640 1 --2- 172.21.15.76:0/4229077296 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f97ac100ad0 0x7f97ac06aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:10:58.129 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:10:58.122+0000 7f97aad9d640 1 -- 172.21.15.76:0/4229077296 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97ac19fb00 con 0x7f97ac102690 2024-09-11T23:10:58.130 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:10:58.122+0000 7f97aa59c640 1 --2- 172.21.15.76:0/4229077296 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f97ac100ad0 0x7f97ac06aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:10:58.130 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:10:58.122+0000 7f97aad9d640 1 --2- 172.21.15.76:0/4229077296 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f97ac102690 0x7f97ac19f1f0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f97a4009430 tx=0x7f97a40319f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:10:58.130 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:10:58.122+0000 7f979b7fe640 1 -- 172.21.15.76:0/4229077296 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97a4003b70 con 0x7f97ac102690 2024-09-11T23:10:58.130 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:10:58.122+0000 7f979b7fe640 1 -- 172.21.15.76:0/4229077296 <== mon.2 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f97a4003d10 con 0x7f97ac102690 2024-09-11T23:10:58.130 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:10:58.122+0000 7f97b0bdf640 1 -- 172.21.15.76:0/4229077296 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97ac19fd30 con 0x7f97ac102690 2024-09-11T23:10:58.130 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:10:58.122+0000 7f979b7fe640 1 -- 172.21.15.76:0/4229077296 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97a402fc50 con 0x7f97ac102690 2024-09-11T23:10:58.130 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:10:58.122+0000 7f97b0bdf640 1 -- 172.21.15.76:0/4229077296 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97ac1a0270 con 0x7f97ac102690 2024-09-11T23:10:58.130 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:10:58.123+0000 7f97b0bdf640 1 -- 172.21.15.76:0/4229077296 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97ac100ed0 con 0x7f97ac102690 2024-09-11T23:10:58.130 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:10:58.124+0000 7f979b7fe640 1 -- 172.21.15.76:0/4229077296 <== mon.2 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7f97a4004040 con 0x7f97ac102690 2024-09-11T23:10:58.130 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:10:58.125+0000 7f979b7fe640 1 --2- 172.21.15.76:0/4229077296 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f97940a2af0 0x7f97940a4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:10:58.130 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:10:58.125+0000 7f979b7fe640 1 -- 172.21.15.76:0/4229077296 <== mon.2 v2:172.21.15.76:3300/0 5 ==== osd_map(8..8 src has 1..8) ==== 1767+0+0 (secure 0 0 0) 0x7f97a40eea20 con 0x7f97ac102690 2024-09-11T23:10:58.130 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:10:58.125+0000 7f97aa59c640 1 --2- 172.21.15.76:0/4229077296 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f97940a2af0 0x7f97940a4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:10:58.130 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:10:58.125+0000 7f97aa59c640 1 --2- 172.21.15.76:0/4229077296 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f97940a2af0 0x7f97940a4fb0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f97ac101b30 tx=0x7f979c005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:10:58.131 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:10:58.129+0000 7f979b7fe640 1 -- 172.21.15.76:0/4229077296 <== mon.2 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f97a40bd3f0 con 0x7f97ac102690 2024-09-11T23:10:58.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:57 smithi136 ceph-mon[35121]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:10:58.240 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:57 smithi136 ceph-mon[35121]: from='osd.0 [v2:172.21.15.26:6802/2014857905,v1:172.21.15.26:6803/2014857905]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]': finished 2024-09-11T23:10:58.240 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:57 smithi136 ceph-mon[35121]: osdmap e7: 1 total, 0 up, 1 in 2024-09-11T23:10:58.240 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:57 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T23:10:58.240 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:57 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T23:10:58.240 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:57 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:10:58.240 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:57 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:10:58.240 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:57 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:10:58.240 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:57 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:10:58.240 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:57 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:10:58.240 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:57 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:10:58.240 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:57 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:10:58.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:57 smithi026 ceph-mon[31172]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:10:58.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:57 smithi026 ceph-mon[31172]: from='osd.0 [v2:172.21.15.26:6802/2014857905,v1:172.21.15.26:6803/2014857905]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]': finished 2024-09-11T23:10:58.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:57 smithi026 ceph-mon[31172]: osdmap e7: 1 total, 0 up, 1 in 2024-09-11T23:10:58.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:57 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T23:10:58.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:57 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T23:10:58.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:57 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:10:58.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:57 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:10:58.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:57 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:10:58.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:57 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:10:58.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:57 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:10:58.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:57 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:10:58.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:57 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:10:58.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:57 smithi076 ceph-mon[35296]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:10:58.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:57 smithi076 ceph-mon[35296]: from='osd.0 [v2:172.21.15.26:6802/2014857905,v1:172.21.15.26:6803/2014857905]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]': finished 2024-09-11T23:10:58.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:57 smithi076 ceph-mon[35296]: osdmap e7: 1 total, 0 up, 1 in 2024-09-11T23:10:58.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:57 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T23:10:58.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:57 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T23:10:58.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:57 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:10:58.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:57 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:10:58.304 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:57 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:10:58.304 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:57 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:10:58.304 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:57 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:10:58.304 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:57 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:10:58.304 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:57 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:10:58.311 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:10:58.311+0000 7f97b0bdf640 1 -- 172.21.15.76:0/4229077296 --> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi076:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f97ac062630 con 0x7f97940a2af0 2024-09-11T23:10:59.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:58 smithi136 ceph-mon[35121]: purged_snaps scrub starts 2024-09-11T23:10:59.240 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:58 smithi136 ceph-mon[35121]: purged_snaps scrub ok 2024-09-11T23:10:59.240 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:58 smithi136 ceph-mon[35121]: Detected new or changed devices on smithi026 2024-09-11T23:10:59.240 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:58 smithi136 ceph-mon[35121]: Adjusting osd_memory_target on smithi026 to 17092M 2024-09-11T23:10:59.240 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:58 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T23:10:59.240 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:58 smithi136 ceph-mon[35121]: osd.0 [v2:172.21.15.26:6802/2014857905,v1:172.21.15.26:6803/2014857905] boot 2024-09-11T23:10:59.240 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:58 smithi136 ceph-mon[35121]: osdmap e8: 1 total, 1 up, 1 in 2024-09-11T23:10:59.240 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:58 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T23:10:59.240 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:58 smithi136 ceph-mon[35121]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:10:59.240 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:58 smithi136 ceph-mon[35121]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi076:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:10:59.240 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:58 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T23:10:59.240 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:58 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T23:10:59.240 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:58 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:10:59.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:58 smithi026 ceph-mon[31172]: purged_snaps scrub starts 2024-09-11T23:10:59.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:58 smithi026 ceph-mon[31172]: purged_snaps scrub ok 2024-09-11T23:10:59.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:58 smithi026 ceph-mon[31172]: Detected new or changed devices on smithi026 2024-09-11T23:10:59.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:58 smithi026 ceph-mon[31172]: Adjusting osd_memory_target on smithi026 to 17092M 2024-09-11T23:10:59.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:58 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T23:10:59.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:58 smithi026 ceph-mon[31172]: osd.0 [v2:172.21.15.26:6802/2014857905,v1:172.21.15.26:6803/2014857905] boot 2024-09-11T23:10:59.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:58 smithi026 ceph-mon[31172]: osdmap e8: 1 total, 1 up, 1 in 2024-09-11T23:10:59.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:58 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T23:10:59.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:58 smithi026 ceph-mon[31172]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:10:59.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:58 smithi026 ceph-mon[31172]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi076:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:10:59.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:58 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T23:10:59.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:58 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T23:10:59.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:58 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:10:59.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:58 smithi076 ceph-mon[35296]: purged_snaps scrub starts 2024-09-11T23:10:59.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:58 smithi076 ceph-mon[35296]: purged_snaps scrub ok 2024-09-11T23:10:59.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:58 smithi076 ceph-mon[35296]: Detected new or changed devices on smithi026 2024-09-11T23:10:59.304 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:58 smithi076 ceph-mon[35296]: Adjusting osd_memory_target on smithi026 to 17092M 2024-09-11T23:10:59.304 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:58 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T23:10:59.304 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:58 smithi076 ceph-mon[35296]: osd.0 [v2:172.21.15.26:6802/2014857905,v1:172.21.15.26:6803/2014857905] boot 2024-09-11T23:10:59.304 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:58 smithi076 ceph-mon[35296]: osdmap e8: 1 total, 1 up, 1 in 2024-09-11T23:10:59.304 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:58 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T23:10:59.304 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:58 smithi076 ceph-mon[35296]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:10:59.304 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:58 smithi076 ceph-mon[35296]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi076:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:10:59.304 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:58 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T23:10:59.304 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:58 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T23:10:59.304 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:58 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:11:00.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:10:59 smithi136 ceph-mon[35121]: osdmap e9: 1 total, 1 up, 1 in 2024-09-11T23:11:00.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:10:59 smithi026 ceph-mon[31172]: osdmap e9: 1 total, 1 up, 1 in 2024-09-11T23:11:00.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:10:59 smithi076 ceph-mon[35296]: osdmap e9: 1 total, 1 up, 1 in 2024-09-11T23:11:01.172 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:00 smithi076 ceph-mon[35296]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T23:11:01.172 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:00 smithi076 ceph-mon[35296]: from='client.? 172.21.15.76:0/905523878' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e0d69112-3c0a-4fc5-8783-e41b4081e0c6"}]: dispatch 2024-09-11T23:11:01.172 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:00 smithi076 ceph-mon[35296]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e0d69112-3c0a-4fc5-8783-e41b4081e0c6"}]: dispatch 2024-09-11T23:11:01.172 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:00 smithi076 ceph-mon[35296]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e0d69112-3c0a-4fc5-8783-e41b4081e0c6"}]': finished 2024-09-11T23:11:01.173 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:00 smithi076 ceph-mon[35296]: osdmap e10: 2 total, 1 up, 2 in 2024-09-11T23:11:01.173 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:00 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T23:11:01.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:00 smithi136 ceph-mon[35121]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T23:11:01.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:00 smithi136 ceph-mon[35121]: from='client.? 172.21.15.76:0/905523878' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e0d69112-3c0a-4fc5-8783-e41b4081e0c6"}]: dispatch 2024-09-11T23:11:01.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:00 smithi136 ceph-mon[35121]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e0d69112-3c0a-4fc5-8783-e41b4081e0c6"}]: dispatch 2024-09-11T23:11:01.240 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:00 smithi136 ceph-mon[35121]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e0d69112-3c0a-4fc5-8783-e41b4081e0c6"}]': finished 2024-09-11T23:11:01.240 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:00 smithi136 ceph-mon[35121]: osdmap e10: 2 total, 1 up, 2 in 2024-09-11T23:11:01.240 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:00 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T23:11:01.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:00 smithi026 ceph-mon[31172]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T23:11:01.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:00 smithi026 ceph-mon[31172]: from='client.? 172.21.15.76:0/905523878' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e0d69112-3c0a-4fc5-8783-e41b4081e0c6"}]: dispatch 2024-09-11T23:11:01.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:00 smithi026 ceph-mon[31172]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e0d69112-3c0a-4fc5-8783-e41b4081e0c6"}]: dispatch 2024-09-11T23:11:01.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:00 smithi026 ceph-mon[31172]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e0d69112-3c0a-4fc5-8783-e41b4081e0c6"}]': finished 2024-09-11T23:11:01.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:00 smithi026 ceph-mon[31172]: osdmap e10: 2 total, 1 up, 2 in 2024-09-11T23:11:01.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:00 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T23:11:02.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:01 smithi136 ceph-mon[35121]: from='client.? 172.21.15.76:0/57987412' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T23:11:02.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:01 smithi026 ceph-mon[31172]: from='client.? 172.21.15.76:0/57987412' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T23:11:02.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:01 smithi076 ceph-mon[35296]: from='client.? 172.21.15.76:0/57987412' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T23:11:03.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:02 smithi136 ceph-mon[35121]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T23:11:03.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:02 smithi026 ceph-mon[31172]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T23:11:03.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:02 smithi076 ceph-mon[35296]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T23:11:05.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:05 smithi136 ceph-mon[35121]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T23:11:05.501 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:05 smithi026 ceph-mon[31172]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T23:11:05.520 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:05 smithi076 ceph-mon[35296]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T23:11:07.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:07 smithi076 ceph-mon[35296]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T23:11:07.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:07 smithi136 ceph-mon[35121]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T23:11:07.501 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:07 smithi026 ceph-mon[31172]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T23:11:09.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:09 smithi076 ceph-mon[35296]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T23:11:09.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:09 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-11T23:11:09.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:09 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:11:09.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:09 smithi076 ceph-mon[35296]: Deploying daemon osd.1 on smithi076 2024-09-11T23:11:09.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:09 smithi136 ceph-mon[35121]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T23:11:09.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:09 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-11T23:11:09.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:09 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:11:09.490 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:09 smithi136 ceph-mon[35121]: Deploying daemon osd.1 on smithi076 2024-09-11T23:11:09.501 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:09 smithi026 ceph-mon[31172]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T23:11:09.501 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:09 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-11T23:11:09.501 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:09 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:11:09.501 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:09 smithi026 ceph-mon[31172]: Deploying daemon osd.1 on smithi076 2024-09-11T23:11:11.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:11 smithi076 ceph-mon[35296]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T23:11:11.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:11 smithi136 ceph-mon[35121]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T23:11:11.501 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:11 smithi026 ceph-mon[31172]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T23:11:13.454 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:13 smithi076 ceph-mon[35296]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T23:11:13.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:13 smithi136 ceph-mon[35121]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T23:11:13.501 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:13 smithi026 ceph-mon[31172]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T23:11:15.379 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:15 smithi076 ceph-mon[35296]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T23:11:15.379 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:15 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:11:15.379 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:15 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:11:15.379 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:15 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:11:15.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:15 smithi136 ceph-mon[35121]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T23:11:15.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:15 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:11:15.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:15 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:11:15.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:15 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:11:15.501 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:15 smithi026 ceph-mon[31172]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T23:11:15.501 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:15 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:11:15.501 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:15 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:11:15.501 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:15 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:11:17.250 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:17 smithi076 ceph-mon[35296]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T23:11:17.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:17 smithi136 ceph-mon[35121]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T23:11:17.501 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:17 smithi026 ceph-mon[31172]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T23:11:18.328 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:11:18.327+0000 7f979b7fe640 1 -- 172.21.15.76:0/4229077296 <== mgr.14148 v2:172.21.15.26:6800/110833859 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f97ac062630 con 0x7f97940a2af0 2024-09-11T23:11:18.328 INFO:teuthology.orchestra.run.smithi076.stdout:Created osd(s) 1 on host 'smithi076' 2024-09-11T23:11:18.331 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:11:18.329+0000 7f97b0bdf640 1 -- 172.21.15.76:0/4229077296 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f97940a2af0 msgr2=0x7f97940a4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:11:18.331 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:11:18.329+0000 7f97b0bdf640 1 --2- 172.21.15.76:0/4229077296 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f97940a2af0 0x7f97940a4fb0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f97ac101b30 tx=0x7f979c005f90 comp rx=0 tx=0).stop 2024-09-11T23:11:18.331 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:11:18.329+0000 7f97b0bdf640 1 -- 172.21.15.76:0/4229077296 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f97ac102690 msgr2=0x7f97ac19f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:11:18.331 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:11:18.329+0000 7f97b0bdf640 1 --2- 172.21.15.76:0/4229077296 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f97ac102690 0x7f97ac19f1f0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f97a4009430 tx=0x7f97a40319f0 comp rx=0 tx=0).stop 2024-09-11T23:11:18.331 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:11:18.330+0000 7f97b0bdf640 1 -- 172.21.15.76:0/4229077296 shutdown_connections 2024-09-11T23:11:18.331 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:11:18.330+0000 7f97b0bdf640 1 --2- 172.21.15.76:0/4229077296 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f97940a2af0 0x7f97940a4fb0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:11:18.331 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:11:18.330+0000 7f97b0bdf640 1 --2- 172.21.15.76:0/4229077296 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f97ac102690 0x7f97ac19f1f0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:11:18.331 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:11:18.330+0000 7f97b0bdf640 1 --2- 172.21.15.76:0/4229077296 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f97ac101cd0 0x7f97ac19cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:11:18.332 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:11:18.330+0000 7f97b0bdf640 1 --2- 172.21.15.76:0/4229077296 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f97ac100ad0 0x7f97ac06aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:11:18.332 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:11:18.330+0000 7f97b0bdf640 1 -- 172.21.15.76:0/4229077296 >> 172.21.15.76:0/4229077296 conn(0x7f97ac0fc280 msgr2=0x7f97ac104fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:11:18.332 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:11:18.330+0000 7f97b0bdf640 1 -- 172.21.15.76:0/4229077296 shutdown_connections 2024-09-11T23:11:18.332 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:11:18.330+0000 7f97b0bdf640 1 -- 172.21.15.76:0/4229077296 wait complete. 2024-09-11T23:11:18.651 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:18 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:11:18.651 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:18 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:11:18.651 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:18 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:11:18.651 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:18 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:11:18.651 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:18 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:11:18.651 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:18 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:11:18.651 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:18 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:11:18.651 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:18 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:11:18.981 DEBUG:teuthology.orchestra.run.smithi076:osd.1> sudo journalctl -f -n 0 -u ceph-af7e6fc8-7092-11ef-bcea-c7b262605968@osd.1.service 2024-09-11T23:11:18.984 INFO:tasks.cephadm:Deploying osd.2 on smithi136 with /dev/vg_nvme/lv_4... 2024-09-11T23:11:18.984 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-11T23:11:18.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:18 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:11:18.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:18 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:11:18.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:18 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:11:18.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:18 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:11:18.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:18 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:11:18.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:18 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:11:18.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:18 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:11:18.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:18 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:11:19.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:18 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:11:19.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:18 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:11:19.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:18 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:11:19.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:18 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:11:19.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:18 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:11:19.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:18 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:11:19.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:18 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:11:19.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:18 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:11:19.235 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T23:11:19.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:19 smithi136 ceph-mon[35121]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T23:11:19.815 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:19 smithi076 ceph-mon[35296]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T23:11:20.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:19 smithi026 ceph-mon[31172]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T23:11:21.413 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-09-11T23:11:21.439 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi136:vg_nvme/lv_4 2024-09-11T23:11:21.449 INFO:journalctl@ceph.osd.1.smithi076.stdout:Sep 11 23:11:21 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-1[39779]: 2024-09-11T23:11:21.263+0000 7f1086ce8640 -1 osd.1 0 log_to_monitors true 2024-09-11T23:11:21.718 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:21 smithi136 ceph-mon[35121]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T23:11:21.718 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:21 smithi136 ceph-mon[35121]: from='osd.1 [v2:172.21.15.76:6800/333383087,v1:172.21.15.76:6801/333383087]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-11T23:11:21.718 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:21 smithi136 ceph-mon[35121]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-11T23:11:21.728 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T23:11:21.800 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:21 smithi076 ceph-mon[35296]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T23:11:21.801 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:21 smithi076 ceph-mon[35296]: from='osd.1 [v2:172.21.15.76:6800/333383087,v1:172.21.15.76:6801/333383087]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-11T23:11:21.801 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:21 smithi076 ceph-mon[35296]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-11T23:11:22.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:21 smithi026 ceph-mon[31172]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T23:11:22.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:21 smithi026 ceph-mon[31172]: from='osd.1 [v2:172.21.15.76:6800/333383087,v1:172.21.15.76:6801/333383087]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-11T23:11:22.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:21 smithi026 ceph-mon[31172]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-11T23:11:22.194 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:11:22.190+0000 7f52a5c13640 1 -- 172.21.15.136:0/4181683677 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f52a0101cd0 msgr2=0x7f52a0102150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:11:22.194 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:11:22.190+0000 7f52a5c13640 1 --2- 172.21.15.136:0/4181683677 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f52a0101cd0 0x7f52a0102150 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f5290009900 tx=0x7f529002a410 comp rx=0 tx=0).stop 2024-09-11T23:11:22.194 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:11:22.190+0000 7f52a5c13640 1 -- 172.21.15.136:0/4181683677 shutdown_connections 2024-09-11T23:11:22.194 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:11:22.190+0000 7f52a5c13640 1 --2- 172.21.15.136:0/4181683677 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52a0102690 0x7f52a0108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:11:22.194 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:11:22.190+0000 7f52a5c13640 1 --2- 172.21.15.136:0/4181683677 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f52a0101cd0 0x7f52a0102150 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:11:22.194 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:11:22.190+0000 7f52a5c13640 1 --2- 172.21.15.136:0/4181683677 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f52a0100ad0 0x7f52a0100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:11:22.194 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:11:22.190+0000 7f52a5c13640 1 -- 172.21.15.136:0/4181683677 >> 172.21.15.136:0/4181683677 conn(0x7f52a00fc280 msgr2=0x7f52a00fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:11:22.194 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:11:22.190+0000 7f52a5c13640 1 -- 172.21.15.136:0/4181683677 shutdown_connections 2024-09-11T23:11:22.194 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:11:22.190+0000 7f52a5c13640 1 -- 172.21.15.136:0/4181683677 wait complete. 2024-09-11T23:11:22.194 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:11:22.191+0000 7f52a5c13640 1 Processor -- start 2024-09-11T23:11:22.194 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:11:22.191+0000 7f52a5c13640 1 -- start start 2024-09-11T23:11:22.195 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:11:22.191+0000 7f52a5c13640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52a0100ad0 0x7f52a0198050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:11:22.195 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:11:22.191+0000 7f52a5c13640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f52a0101cd0 0x7f52a0198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:11:22.195 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:11:22.191+0000 7f52a5c13640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f52a0102690 0x7f52a019f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:11:22.195 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:11:22.191+0000 7f52a5c13640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f52a010c850 con 0x7f52a0100ad0 2024-09-11T23:11:22.195 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:11:22.191+0000 7f52a5c13640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f52a010c6d0 con 0x7f52a0101cd0 2024-09-11T23:11:22.195 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:11:22.191+0000 7f52a5c13640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f52a010c9d0 con 0x7f52a0102690 2024-09-11T23:11:22.195 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:11:22.191+0000 7f529ffff640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f52a0102690 0x7f52a019f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:11:22.195 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:11:22.191+0000 7f529ffff640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f52a0102690 0x7f52a019f610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.136:54452/0 (socket says 172.21.15.136:54452) 2024-09-11T23:11:22.195 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:11:22.191+0000 7f529ffff640 1 -- 172.21.15.136:0/2217177395 learned_addr learned my addr 172.21.15.136:0/2217177395 (peer_addr_for_me v2:172.21.15.136:0/0) 2024-09-11T23:11:22.195 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:11:22.191+0000 7f529effd640 1 --2- 172.21.15.136:0/2217177395 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f52a0101cd0 0x7f52a0198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:11:22.195 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:11:22.191+0000 7f529f7fe640 1 --2- 172.21.15.136:0/2217177395 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52a0100ad0 0x7f52a0198050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:11:22.195 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:11:22.192+0000 7f529ffff640 1 -- 172.21.15.136:0/2217177395 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f52a0101cd0 msgr2=0x7f52a0198590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:11:22.196 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:11:22.192+0000 7f529ffff640 1 --2- 172.21.15.136:0/2217177395 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f52a0101cd0 0x7f52a0198590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:11:22.196 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:11:22.192+0000 7f529ffff640 1 -- 172.21.15.136:0/2217177395 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52a0100ad0 msgr2=0x7f52a0198050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:11:22.196 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:11:22.192+0000 7f529ffff640 1 --2- 172.21.15.136:0/2217177395 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52a0100ad0 0x7f52a0198050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:11:22.196 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:11:22.192+0000 7f529ffff640 1 -- 172.21.15.136:0/2217177395 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52a019fc80 con 0x7f52a0102690 2024-09-11T23:11:22.196 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:11:22.192+0000 7f529ffff640 1 --2- 172.21.15.136:0/2217177395 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f52a0102690 0x7f52a019f610 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f529800b970 tx=0x7f529800bd70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:11:22.196 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:11:22.192+0000 7f529cff9640 1 -- 172.21.15.136:0/2217177395 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5298019070 con 0x7f52a0102690 2024-09-11T23:11:22.196 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:11:22.192+0000 7f529cff9640 1 -- 172.21.15.136:0/2217177395 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5298014440 con 0x7f52a0102690 2024-09-11T23:11:22.196 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:11:22.192+0000 7f52a5c13640 1 -- 172.21.15.136:0/2217177395 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52a019ff70 con 0x7f52a0102690 2024-09-11T23:11:22.196 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:11:22.192+0000 7f529cff9640 1 -- 172.21.15.136:0/2217177395 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52980075d0 con 0x7f52a0102690 2024-09-11T23:11:22.196 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:11:22.192+0000 7f52a5c13640 1 -- 172.21.15.136:0/2217177395 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52a01a0450 con 0x7f52a0102690 2024-09-11T23:11:22.198 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:11:22.193+0000 7f52a5c13640 1 -- 172.21.15.136:0/2217177395 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52a01a06e0 con 0x7f52a0102690 2024-09-11T23:11:22.198 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:11:22.194+0000 7f529cff9640 1 -- 172.21.15.136:0/2217177395 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7f529801c070 con 0x7f52a0102690 2024-09-11T23:11:22.198 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:11:22.195+0000 7f529cff9640 1 --2- 172.21.15.136:0/2217177395 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f52880a2b20 0x7f52880a4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:11:22.198 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:11:22.195+0000 7f529cff9640 1 -- 172.21.15.136:0/2217177395 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(11..11 src has 1..11) ==== 1903+0+0 (secure 0 0 0) 0x7f52980ccff0 con 0x7f52a0102690 2024-09-11T23:11:22.198 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:11:22.195+0000 7f529f7fe640 1 --2- 172.21.15.136:0/2217177395 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f52880a2b20 0x7f52880a4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:11:22.199 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:11:22.196+0000 7f529f7fe640 1 --2- 172.21.15.136:0/2217177395 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f52880a2b20 0x7f52880a4fe0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f52a0101b30 tx=0x7f5294009110 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:11:22.203 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:11:22.203+0000 7f529cff9640 1 -- 172.21.15.136:0/2217177395 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f529809b2f0 con 0x7f52a0102690 2024-09-11T23:11:22.387 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:11:22.386+0000 7f52a5c13640 1 -- 172.21.15.136:0/2217177395 --> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi136:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f52a01a08b0 con 0x7f52880a2b20 2024-09-11T23:11:22.718 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:22 smithi136 ceph-mon[35121]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-11T23:11:22.718 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:22 smithi136 ceph-mon[35121]: osdmap e11: 2 total, 1 up, 2 in 2024-09-11T23:11:22.718 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:22 smithi136 ceph-mon[35121]: from='osd.1 [v2:172.21.15.76:6800/333383087,v1:172.21.15.76:6801/333383087]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]: dispatch 2024-09-11T23:11:22.718 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:22 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T23:11:22.718 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:22 smithi136 ceph-mon[35121]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]: dispatch 2024-09-11T23:11:22.719 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:22 smithi136 ceph-mon[35121]: Detected new or changed devices on smithi076 2024-09-11T23:11:22.719 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:22 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:11:22.719 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:22 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:11:22.719 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:22 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:11:22.719 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:22 smithi136 ceph-mon[35121]: Adjusting osd_memory_target on smithi076 to 17088M 2024-09-11T23:11:22.719 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:22 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:11:22.719 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:22 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:11:22.719 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:22 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:11:22.719 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:22 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:11:22.719 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:22 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T23:11:22.719 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:22 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T23:11:22.719 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:22 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:11:22.720 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:22 smithi136 ceph-mon[35121]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]': finished 2024-09-11T23:11:23.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:22 smithi026 ceph-mon[31172]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-11T23:11:23.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:22 smithi026 ceph-mon[31172]: osdmap e11: 2 total, 1 up, 2 in 2024-09-11T23:11:23.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:22 smithi026 ceph-mon[31172]: from='osd.1 [v2:172.21.15.76:6800/333383087,v1:172.21.15.76:6801/333383087]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]: dispatch 2024-09-11T23:11:23.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:22 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T23:11:23.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:22 smithi026 ceph-mon[31172]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]: dispatch 2024-09-11T23:11:23.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:22 smithi026 ceph-mon[31172]: Detected new or changed devices on smithi076 2024-09-11T23:11:23.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:22 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:11:23.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:22 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:11:23.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:22 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:11:23.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:22 smithi026 ceph-mon[31172]: Adjusting osd_memory_target on smithi076 to 17088M 2024-09-11T23:11:23.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:22 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:11:23.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:22 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:11:23.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:22 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:11:23.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:22 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:11:23.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:22 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T23:11:23.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:22 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T23:11:23.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:22 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:11:23.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:22 smithi026 ceph-mon[31172]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]': finished 2024-09-11T23:11:23.053 INFO:journalctl@ceph.osd.1.smithi076.stdout:Sep 11 23:11:22 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-1[39779]: 2024-09-11T23:11:22.580+0000 7f1082448640 -1 osd.1 0 waiting for initial osdmap 2024-09-11T23:11:23.053 INFO:journalctl@ceph.osd.1.smithi076.stdout:Sep 11 23:11:22 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-1[39779]: 2024-09-11T23:11:22.584+0000 7f107ea82640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T23:11:23.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:22 smithi076 ceph-mon[35296]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-11T23:11:23.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:22 smithi076 ceph-mon[35296]: osdmap e11: 2 total, 1 up, 2 in 2024-09-11T23:11:23.054 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:22 smithi076 ceph-mon[35296]: from='osd.1 [v2:172.21.15.76:6800/333383087,v1:172.21.15.76:6801/333383087]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]: dispatch 2024-09-11T23:11:23.054 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:22 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T23:11:23.054 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:22 smithi076 ceph-mon[35296]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]: dispatch 2024-09-11T23:11:23.054 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:22 smithi076 ceph-mon[35296]: Detected new or changed devices on smithi076 2024-09-11T23:11:23.054 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:22 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:11:23.054 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:22 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:11:23.054 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:22 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:11:23.054 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:22 smithi076 ceph-mon[35296]: Adjusting osd_memory_target on smithi076 to 17088M 2024-09-11T23:11:23.054 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:22 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:11:23.054 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:22 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:11:23.054 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:22 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:11:23.054 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:22 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:11:23.054 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:22 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T23:11:23.055 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:22 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T23:11:23.055 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:22 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:11:23.055 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:22 smithi076 ceph-mon[35296]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]': finished 2024-09-11T23:11:23.619 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:23 smithi136 ceph-mon[35121]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T23:11:23.619 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:23 smithi136 ceph-mon[35121]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi136:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:11:23.619 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:23 smithi136 ceph-mon[35121]: osdmap e12: 2 total, 1 up, 2 in 2024-09-11T23:11:23.619 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:23 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T23:11:23.620 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:23 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T23:11:24.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:23 smithi026 ceph-mon[31172]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T23:11:24.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:23 smithi026 ceph-mon[31172]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi136:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:11:24.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:23 smithi026 ceph-mon[31172]: osdmap e12: 2 total, 1 up, 2 in 2024-09-11T23:11:24.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:23 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T23:11:24.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:23 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T23:11:24.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:23 smithi076 ceph-mon[35296]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T23:11:24.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:23 smithi076 ceph-mon[35296]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi136:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:11:24.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:23 smithi076 ceph-mon[35296]: osdmap e12: 2 total, 1 up, 2 in 2024-09-11T23:11:24.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:23 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T23:11:24.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:23 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T23:11:24.888 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:24 smithi136 ceph-mon[35121]: purged_snaps scrub starts 2024-09-11T23:11:24.889 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:24 smithi136 ceph-mon[35121]: purged_snaps scrub ok 2024-09-11T23:11:24.889 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:24 smithi136 ceph-mon[35121]: osd.1 [v2:172.21.15.76:6800/333383087,v1:172.21.15.76:6801/333383087] boot 2024-09-11T23:11:24.889 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:24 smithi136 ceph-mon[35121]: osdmap e13: 2 total, 2 up, 2 in 2024-09-11T23:11:24.889 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:24 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T23:11:25.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:24 smithi026 ceph-mon[31172]: purged_snaps scrub starts 2024-09-11T23:11:25.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:24 smithi026 ceph-mon[31172]: purged_snaps scrub ok 2024-09-11T23:11:25.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:24 smithi026 ceph-mon[31172]: osd.1 [v2:172.21.15.76:6800/333383087,v1:172.21.15.76:6801/333383087] boot 2024-09-11T23:11:25.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:24 smithi026 ceph-mon[31172]: osdmap e13: 2 total, 2 up, 2 in 2024-09-11T23:11:25.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:24 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T23:11:25.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:24 smithi076 ceph-mon[35296]: purged_snaps scrub starts 2024-09-11T23:11:25.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:24 smithi076 ceph-mon[35296]: purged_snaps scrub ok 2024-09-11T23:11:25.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:24 smithi076 ceph-mon[35296]: osd.1 [v2:172.21.15.76:6800/333383087,v1:172.21.15.76:6801/333383087] boot 2024-09-11T23:11:25.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:24 smithi076 ceph-mon[35296]: osdmap e13: 2 total, 2 up, 2 in 2024-09-11T23:11:25.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:24 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T23:11:25.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:25 smithi136 ceph-mon[35121]: pgmap v58: 0 pgs: ; 0 B data, 27 MiB used, 89 GiB / 89 GiB avail 2024-09-11T23:11:25.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:25 smithi136 ceph-mon[35121]: from='client.? 172.21.15.136:0/3783981725' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d578429e-bf23-4a1e-890f-7172955c0eb1"}]: dispatch 2024-09-11T23:11:25.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:25 smithi136 ceph-mon[35121]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d578429e-bf23-4a1e-890f-7172955c0eb1"}]: dispatch 2024-09-11T23:11:25.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:25 smithi136 ceph-mon[35121]: osdmap e14: 2 total, 2 up, 2 in 2024-09-11T23:11:25.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:25 smithi136 ceph-mon[35121]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d578429e-bf23-4a1e-890f-7172955c0eb1"}]': finished 2024-09-11T23:11:25.990 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:25 smithi136 ceph-mon[35121]: osdmap e15: 3 total, 2 up, 3 in 2024-09-11T23:11:25.990 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:25 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T23:11:25.990 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:25 smithi136 ceph-mon[35121]: from='client.? 172.21.15.136:0/1498394402' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T23:11:26.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:25 smithi026 ceph-mon[31172]: pgmap v58: 0 pgs: ; 0 B data, 27 MiB used, 89 GiB / 89 GiB avail 2024-09-11T23:11:26.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:25 smithi026 ceph-mon[31172]: from='client.? 172.21.15.136:0/3783981725' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d578429e-bf23-4a1e-890f-7172955c0eb1"}]: dispatch 2024-09-11T23:11:26.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:25 smithi026 ceph-mon[31172]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d578429e-bf23-4a1e-890f-7172955c0eb1"}]: dispatch 2024-09-11T23:11:26.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:25 smithi026 ceph-mon[31172]: osdmap e14: 2 total, 2 up, 2 in 2024-09-11T23:11:26.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:25 smithi026 ceph-mon[31172]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d578429e-bf23-4a1e-890f-7172955c0eb1"}]': finished 2024-09-11T23:11:26.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:25 smithi026 ceph-mon[31172]: osdmap e15: 3 total, 2 up, 3 in 2024-09-11T23:11:26.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:25 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T23:11:26.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:25 smithi026 ceph-mon[31172]: from='client.? 172.21.15.136:0/1498394402' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T23:11:26.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:25 smithi076 ceph-mon[35296]: pgmap v58: 0 pgs: ; 0 B data, 27 MiB used, 89 GiB / 89 GiB avail 2024-09-11T23:11:26.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:25 smithi076 ceph-mon[35296]: from='client.? 172.21.15.136:0/3783981725' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d578429e-bf23-4a1e-890f-7172955c0eb1"}]: dispatch 2024-09-11T23:11:26.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:25 smithi076 ceph-mon[35296]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d578429e-bf23-4a1e-890f-7172955c0eb1"}]: dispatch 2024-09-11T23:11:26.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:25 smithi076 ceph-mon[35296]: osdmap e14: 2 total, 2 up, 2 in 2024-09-11T23:11:26.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:25 smithi076 ceph-mon[35296]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d578429e-bf23-4a1e-890f-7172955c0eb1"}]': finished 2024-09-11T23:11:26.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:25 smithi076 ceph-mon[35296]: osdmap e15: 3 total, 2 up, 3 in 2024-09-11T23:11:26.054 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:25 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T23:11:26.054 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:25 smithi076 ceph-mon[35296]: from='client.? 172.21.15.136:0/1498394402' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T23:11:27.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:27 smithi136 ceph-mon[35121]: pgmap v61: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-11T23:11:28.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:27 smithi026 ceph-mon[31172]: pgmap v61: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-11T23:11:28.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:27 smithi076 ceph-mon[35296]: pgmap v61: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-11T23:11:29.961 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:29 smithi136 ceph-mon[35121]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-11T23:11:30.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:29 smithi026 ceph-mon[31172]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-11T23:11:30.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:29 smithi076 ceph-mon[35296]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-11T23:11:31.828 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:31 smithi136 ceph-mon[35121]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-11T23:11:32.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:31 smithi026 ceph-mon[31172]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-11T23:11:32.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:31 smithi076 ceph-mon[35296]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-11T23:11:33.686 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:33 smithi136 ceph-mon[35121]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-11T23:11:33.686 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:33 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-11T23:11:33.686 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:33 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:11:34.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:33 smithi026 ceph-mon[31172]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-11T23:11:34.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:33 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-11T23:11:34.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:33 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:11:34.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:33 smithi076 ceph-mon[35296]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-11T23:11:34.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:33 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-11T23:11:34.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:33 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:11:34.858 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:34 smithi136 ceph-mon[35121]: Deploying daemon osd.2 on smithi136 2024-09-11T23:11:35.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:34 smithi026 ceph-mon[31172]: Deploying daemon osd.2 on smithi136 2024-09-11T23:11:35.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:34 smithi076 ceph-mon[35296]: Deploying daemon osd.2 on smithi136 2024-09-11T23:11:35.919 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:35 smithi136 ceph-mon[35121]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-11T23:11:36.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:35 smithi026 ceph-mon[31172]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-11T23:11:36.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:35 smithi076 ceph-mon[35296]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-11T23:11:37.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:37 smithi136 ceph-mon[35121]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-11T23:11:38.000 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:37 smithi026 ceph-mon[31172]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-11T23:11:38.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:37 smithi076 ceph-mon[35296]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-11T23:11:39.861 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:39 smithi136 ceph-mon[35121]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-11T23:11:39.861 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:39 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:11:39.861 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:39 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:11:39.861 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:39 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:11:40.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:39 smithi026 ceph-mon[31172]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-11T23:11:40.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:39 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:11:40.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:39 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:11:40.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:39 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:11:40.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:39 smithi076 ceph-mon[35296]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-11T23:11:40.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:39 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:11:40.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:39 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:11:40.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:39 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:11:41.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:41 smithi136 ceph-mon[35121]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-11T23:11:42.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:41 smithi026 ceph-mon[31172]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-11T23:11:42.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:41 smithi076 ceph-mon[35296]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-11T23:11:42.807 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:42 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:11:42.808 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:42 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:11:42.808 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:42 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:11:42.808 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:42 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:11:42.808 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:42 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:11:43.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:42 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:11:43.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:42 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:11:43.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:42 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:11:43.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:42 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:11:43.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:42 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:11:43.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:42 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:11:43.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:42 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:11:43.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:42 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:11:43.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:42 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:11:43.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:42 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:11:43.081 INFO:teuthology.orchestra.run.smithi136.stdout:Created osd(s) 2 on host 'smithi136' 2024-09-11T23:11:43.081 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:11:43.080+0000 7f529cff9640 1 -- 172.21.15.136:0/2217177395 <== mgr.14148 v2:172.21.15.26:6800/110833859 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f52a01a08b0 con 0x7f52880a2b20 2024-09-11T23:11:43.083 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:11:43.082+0000 7f52a5c13640 1 -- 172.21.15.136:0/2217177395 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f52880a2b20 msgr2=0x7f52880a4fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:11:43.084 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:11:43.082+0000 7f52a5c13640 1 --2- 172.21.15.136:0/2217177395 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f52880a2b20 0x7f52880a4fe0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f52a0101b30 tx=0x7f5294009110 comp rx=0 tx=0).stop 2024-09-11T23:11:43.084 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:11:43.082+0000 7f52a5c13640 1 -- 172.21.15.136:0/2217177395 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f52a0102690 msgr2=0x7f52a019f610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:11:43.084 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:11:43.082+0000 7f52a5c13640 1 --2- 172.21.15.136:0/2217177395 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f52a0102690 0x7f52a019f610 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f529800b970 tx=0x7f529800bd70 comp rx=0 tx=0).stop 2024-09-11T23:11:43.084 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:11:43.082+0000 7f52a5c13640 1 -- 172.21.15.136:0/2217177395 shutdown_connections 2024-09-11T23:11:43.084 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:11:43.082+0000 7f52a5c13640 1 --2- 172.21.15.136:0/2217177395 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f52880a2b20 0x7f52880a4fe0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:11:43.084 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:11:43.082+0000 7f52a5c13640 1 --2- 172.21.15.136:0/2217177395 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f52a0101cd0 0x7f52a0198590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:11:43.084 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:11:43.082+0000 7f52a5c13640 1 --2- 172.21.15.136:0/2217177395 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f52a0102690 0x7f52a019f610 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:11:43.084 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:11:43.082+0000 7f52a5c13640 1 --2- 172.21.15.136:0/2217177395 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52a0100ad0 0x7f52a0198050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:11:43.084 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:11:43.082+0000 7f52a5c13640 1 -- 172.21.15.136:0/2217177395 >> 172.21.15.136:0/2217177395 conn(0x7f52a00fc280 msgr2=0x7f52a0069f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:11:43.084 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:11:43.083+0000 7f52a5c13640 1 -- 172.21.15.136:0/2217177395 shutdown_connections 2024-09-11T23:11:43.084 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:11:43.083+0000 7f52a5c13640 1 -- 172.21.15.136:0/2217177395 wait complete. 2024-09-11T23:11:43.713 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:43 smithi136 ceph-mon[35121]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-11T23:11:43.713 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:43 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:11:43.713 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:43 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:11:43.713 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:43 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:11:43.741 DEBUG:teuthology.orchestra.run.smithi136:osd.2> sudo journalctl -f -n 0 -u ceph-af7e6fc8-7092-11ef-bcea-c7b262605968@osd.2.service 2024-09-11T23:11:43.744 INFO:tasks.cephadm:Deploying osd.3 on smithi026 with /dev/vg_nvme/lv_3... 2024-09-11T23:11:43.744 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-11T23:11:43.954 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:43 smithi076 ceph-mon[35296]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-11T23:11:43.954 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:43 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:11:43.954 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:43 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:11:43.954 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:43 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:11:44.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:43 smithi026 ceph-mon[31172]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-11T23:11:44.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:43 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:11:44.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:43 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:11:44.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:43 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:11:44.013 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:11:45.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:45 smithi136 ceph-mon[35121]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-11T23:11:45.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:45 smithi136 ceph-mon[35121]: from='osd.2 [v2:172.21.15.136:6800/1271538911,v1:172.21.15.136:6801/1271538911]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-11T23:11:45.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:45 smithi136 ceph-mon[35121]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-11T23:11:45.740 INFO:journalctl@ceph.osd.2.smithi136.stdout:Sep 11 23:11:45 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-2[40102]: 2024-09-11T23:11:45.481+0000 7f25946e5640 -1 osd.2 0 log_to_monitors true 2024-09-11T23:11:45.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:45 smithi026 ceph-mon[31172]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-11T23:11:45.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:45 smithi026 ceph-mon[31172]: from='osd.2 [v2:172.21.15.136:6800/1271538911,v1:172.21.15.136:6801/1271538911]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-11T23:11:45.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:45 smithi026 ceph-mon[31172]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-11T23:11:46.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:45 smithi076 ceph-mon[35296]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-11T23:11:46.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:45 smithi076 ceph-mon[35296]: from='osd.2 [v2:172.21.15.136:6800/1271538911,v1:172.21.15.136:6801/1271538911]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-11T23:11:46.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:45 smithi076 ceph-mon[35296]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-11T23:11:46.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:46 smithi136 ceph-mon[35121]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-11T23:11:46.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:46 smithi136 ceph-mon[35121]: osdmap e16: 3 total, 2 up, 3 in 2024-09-11T23:11:46.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:46 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T23:11:46.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:46 smithi136 ceph-mon[35121]: from='osd.2 [v2:172.21.15.136:6800/1271538911,v1:172.21.15.136:6801/1271538911]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-09-11T23:11:46.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:46 smithi136 ceph-mon[35121]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-09-11T23:11:46.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:46 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:11:46.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:46 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:11:46.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:46 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:11:46.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:46 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:11:46.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:46 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:11:46.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:46 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:11:46.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:46 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:11:46.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:46 smithi026 ceph-mon[31172]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-11T23:11:46.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:46 smithi026 ceph-mon[31172]: osdmap e16: 3 total, 2 up, 3 in 2024-09-11T23:11:46.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:46 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T23:11:46.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:46 smithi026 ceph-mon[31172]: from='osd.2 [v2:172.21.15.136:6800/1271538911,v1:172.21.15.136:6801/1271538911]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-09-11T23:11:46.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:46 smithi026 ceph-mon[31172]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-09-11T23:11:46.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:46 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:11:46.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:46 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:11:46.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:46 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:11:46.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:46 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:11:46.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:46 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:11:46.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:46 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:11:46.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:46 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:11:46.795 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-11T23:11:46.823 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi026:vg_nvme/lv_3 2024-09-11T23:11:47.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:46 smithi076 ceph-mon[35296]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-11T23:11:47.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:46 smithi076 ceph-mon[35296]: osdmap e16: 3 total, 2 up, 3 in 2024-09-11T23:11:47.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:46 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T23:11:47.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:46 smithi076 ceph-mon[35296]: from='osd.2 [v2:172.21.15.136:6800/1271538911,v1:172.21.15.136:6801/1271538911]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-09-11T23:11:47.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:46 smithi076 ceph-mon[35296]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-09-11T23:11:47.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:46 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:11:47.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:46 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:11:47.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:46 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:11:47.054 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:46 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:11:47.054 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:46 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:11:47.054 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:46 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:11:47.054 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:46 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:11:47.080 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:11:47.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:11:47.521+0000 7f49dc8a4640 1 -- 172.21.15.26:0/1929304048 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f49d8102690 msgr2=0x7f49d8108f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:11:47.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:11:47.521+0000 7f49dc8a4640 1 --2- 172.21.15.26:0/1929304048 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f49d8102690 0x7f49d8108f20 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f49d0009ca0 tx=0x7f49d002a8c0 comp rx=0 tx=0).stop 2024-09-11T23:11:47.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:11:47.522+0000 7f49dc8a4640 1 -- 172.21.15.26:0/1929304048 shutdown_connections 2024-09-11T23:11:47.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:11:47.522+0000 7f49dc8a4640 1 --2- 172.21.15.26:0/1929304048 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f49d8101cd0 0x7f49d8102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:11:47.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:11:47.522+0000 7f49dc8a4640 1 --2- 172.21.15.26:0/1929304048 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f49d8102690 0x7f49d8108f20 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:11:47.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:11:47.522+0000 7f49dc8a4640 1 --2- 172.21.15.26:0/1929304048 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f49d8100ad0 0x7f49d8100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:11:47.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:11:47.522+0000 7f49dc8a4640 1 -- 172.21.15.26:0/1929304048 >> 172.21.15.26:0/1929304048 conn(0x7f49d80fc280 msgr2=0x7f49d80fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:11:47.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:11:47.522+0000 7f49dc8a4640 1 -- 172.21.15.26:0/1929304048 shutdown_connections 2024-09-11T23:11:47.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:11:47.522+0000 7f49dc8a4640 1 -- 172.21.15.26:0/1929304048 wait complete. 2024-09-11T23:11:47.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:11:47.522+0000 7f49dc8a4640 1 Processor -- start 2024-09-11T23:11:47.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:11:47.522+0000 7f49dc8a4640 1 -- start start 2024-09-11T23:11:47.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:11:47.523+0000 7f49dc8a4640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f49d8100ad0 0x7f49d819a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:11:47.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:11:47.523+0000 7f49dc8a4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f49d8101cd0 0x7f49d819a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:11:47.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:11:47.523+0000 7f49dc8a4640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f49d8102690 0x7f49d81a1650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:11:47.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:11:47.523+0000 7f49dc8a4640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f49d810c730 con 0x7f49d8101cd0 2024-09-11T23:11:47.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:11:47.523+0000 7f49dc8a4640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f49d810c5b0 con 0x7f49d8102690 2024-09-11T23:11:47.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:11:47.523+0000 7f49dc8a4640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f49d810c8b0 con 0x7f49d8100ad0 2024-09-11T23:11:47.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:11:47.523+0000 7f49cffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f49d8101cd0 0x7f49d819a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:11:47.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:11:47.523+0000 7f49cffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f49d8101cd0 0x7f49d819a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56850/0 (socket says 172.21.15.26:56850) 2024-09-11T23:11:47.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:11:47.523+0000 7f49cffff640 1 -- 172.21.15.26:0/2673979280 learned_addr learned my addr 172.21.15.26:0/2673979280 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:11:47.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:11:47.523+0000 7f49d659c640 1 --2- 172.21.15.26:0/2673979280 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f49d8100ad0 0x7f49d819a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:11:47.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:11:47.523+0000 7f49d6d9d640 1 --2- 172.21.15.26:0/2673979280 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f49d8102690 0x7f49d81a1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:11:47.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:11:47.523+0000 7f49cffff640 1 -- 172.21.15.26:0/2673979280 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f49d8100ad0 msgr2=0x7f49d819a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:11:47.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:11:47.523+0000 7f49cffff640 1 --2- 172.21.15.26:0/2673979280 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f49d8100ad0 0x7f49d819a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:11:47.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:11:47.523+0000 7f49cffff640 1 -- 172.21.15.26:0/2673979280 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f49d8102690 msgr2=0x7f49d81a1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:11:47.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:11:47.523+0000 7f49cffff640 1 --2- 172.21.15.26:0/2673979280 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f49d8102690 0x7f49d81a1650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:11:47.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:11:47.523+0000 7f49cffff640 1 -- 172.21.15.26:0/2673979280 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49d81a1cc0 con 0x7f49d8101cd0 2024-09-11T23:11:47.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:11:47.523+0000 7f49d659c640 1 --2- 172.21.15.26:0/2673979280 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f49d8100ad0 0x7f49d819a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:11:47.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:11:47.523+0000 7f49cffff640 1 --2- 172.21.15.26:0/2673979280 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f49d8101cd0 0x7f49d819a5d0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f49c000d950 tx=0x7f49c000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:11:47.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:11:47.523+0000 7f49cf7fe640 1 -- 172.21.15.26:0/2673979280 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49c0017070 con 0x7f49d8101cd0 2024-09-11T23:11:47.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:11:47.524+0000 7f49cf7fe640 1 -- 172.21.15.26:0/2673979280 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49c0009c30 con 0x7f49d8101cd0 2024-09-11T23:11:47.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:11:47.524+0000 7f49dc8a4640 1 -- 172.21.15.26:0/2673979280 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49d81a1fb0 con 0x7f49d8101cd0 2024-09-11T23:11:47.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:11:47.524+0000 7f49cf7fe640 1 -- 172.21.15.26:0/2673979280 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49c000f6b0 con 0x7f49d8101cd0 2024-09-11T23:11:47.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:11:47.524+0000 7f49dc8a4640 1 -- 172.21.15.26:0/2673979280 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49d81a24b0 con 0x7f49d8101cd0 2024-09-11T23:11:47.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:11:47.525+0000 7f49dc8a4640 1 -- 172.21.15.26:0/2673979280 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49d81a27a0 con 0x7f49d8101cd0 2024-09-11T23:11:47.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:11:47.527+0000 7f49cf7fe640 1 -- 172.21.15.26:0/2673979280 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7f49c00109d0 con 0x7f49d8101cd0 2024-09-11T23:11:47.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:11:47.528+0000 7f49cf7fe640 1 --2- 172.21.15.26:0/2673979280 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f49c40a2ba0 0x7f49c40a5060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:11:47.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:11:47.528+0000 7f49cf7fe640 1 -- 172.21.15.26:0/2673979280 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 2571+0+0 (secure 0 0 0) 0x7f49c00cd280 con 0x7f49d8101cd0 2024-09-11T23:11:47.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:11:47.528+0000 7f49d659c640 1 --2- 172.21.15.26:0/2673979280 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f49c40a2ba0 0x7f49c40a5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:11:47.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:11:47.528+0000 7f49d659c640 1 --2- 172.21.15.26:0/2673979280 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f49c40a2ba0 0x7f49c40a5060 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f49c8005fd0 tx=0x7f49c800b410 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:11:47.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:11:47.530+0000 7f49cf7fe640 1 -- 172.21.15.26:0/2673979280 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f49c009b360 con 0x7f49d8101cd0 2024-09-11T23:11:47.653 INFO:journalctl@ceph.osd.2.smithi136.stdout:Sep 11 23:11:47 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-2[40102]: 2024-09-11T23:11:47.279+0000 7f258fe45640 -1 osd.2 0 waiting for initial osdmap 2024-09-11T23:11:47.654 INFO:journalctl@ceph.osd.2.smithi136.stdout:Sep 11 23:11:47 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-2[40102]: 2024-09-11T23:11:47.288+0000 7f2588477640 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T23:11:47.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:11:47.717+0000 7f49dc8a4640 1 -- 172.21.15.26:0/2673979280 --> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi026:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f49d80fe4b0 con 0x7f49c40a2ba0 2024-09-11T23:11:47.810 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:47 smithi026 ceph-mon[31172]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-11T23:11:47.810 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:47 smithi026 ceph-mon[31172]: Detected new or changed devices on smithi136 2024-09-11T23:11:47.810 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:47 smithi026 ceph-mon[31172]: Adjusting osd_memory_target on smithi136 to 17088M 2024-09-11T23:11:47.810 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:47 smithi026 ceph-mon[31172]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]': finished 2024-09-11T23:11:47.810 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:47 smithi026 ceph-mon[31172]: osdmap e17: 3 total, 2 up, 3 in 2024-09-11T23:11:47.810 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:47 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T23:11:47.810 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:47 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T23:11:47.810 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:47 smithi026 ceph-mon[31172]: from='osd.2 ' entity='osd.2' 2024-09-11T23:11:47.811 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:47 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T23:11:47.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:47 smithi136 ceph-mon[35121]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-11T23:11:47.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:47 smithi136 ceph-mon[35121]: Detected new or changed devices on smithi136 2024-09-11T23:11:47.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:47 smithi136 ceph-mon[35121]: Adjusting osd_memory_target on smithi136 to 17088M 2024-09-11T23:11:47.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:47 smithi136 ceph-mon[35121]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]': finished 2024-09-11T23:11:47.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:47 smithi136 ceph-mon[35121]: osdmap e17: 3 total, 2 up, 3 in 2024-09-11T23:11:47.990 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:47 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T23:11:47.990 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:47 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T23:11:47.990 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:47 smithi136 ceph-mon[35121]: from='osd.2 ' entity='osd.2' 2024-09-11T23:11:47.990 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:47 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T23:11:48.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:47 smithi076 ceph-mon[35296]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-11T23:11:48.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:47 smithi076 ceph-mon[35296]: Detected new or changed devices on smithi136 2024-09-11T23:11:48.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:47 smithi076 ceph-mon[35296]: Adjusting osd_memory_target on smithi136 to 17088M 2024-09-11T23:11:48.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:47 smithi076 ceph-mon[35296]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]': finished 2024-09-11T23:11:48.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:47 smithi076 ceph-mon[35296]: osdmap e17: 3 total, 2 up, 3 in 2024-09-11T23:11:48.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:47 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T23:11:48.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:47 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T23:11:48.054 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:47 smithi076 ceph-mon[35296]: from='osd.2 ' entity='osd.2' 2024-09-11T23:11:48.054 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:47 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T23:11:48.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:48 smithi026 ceph-mon[31172]: purged_snaps scrub starts 2024-09-11T23:11:48.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:48 smithi026 ceph-mon[31172]: purged_snaps scrub ok 2024-09-11T23:11:48.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:48 smithi026 ceph-mon[31172]: from='client.14283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi026:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:11:48.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:48 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T23:11:48.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:48 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T23:11:48.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:48 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:11:48.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:48 smithi026 ceph-mon[31172]: osd.2 [v2:172.21.15.136:6800/1271538911,v1:172.21.15.136:6801/1271538911] boot 2024-09-11T23:11:48.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:48 smithi026 ceph-mon[31172]: osdmap e18: 3 total, 3 up, 3 in 2024-09-11T23:11:48.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:48 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T23:11:48.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:48 smithi136 ceph-mon[35121]: purged_snaps scrub starts 2024-09-11T23:11:48.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:48 smithi136 ceph-mon[35121]: purged_snaps scrub ok 2024-09-11T23:11:48.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:48 smithi136 ceph-mon[35121]: from='client.14283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi026:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:11:48.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:48 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T23:11:48.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:48 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T23:11:48.990 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:48 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:11:48.990 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:48 smithi136 ceph-mon[35121]: osd.2 [v2:172.21.15.136:6800/1271538911,v1:172.21.15.136:6801/1271538911] boot 2024-09-11T23:11:48.990 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:48 smithi136 ceph-mon[35121]: osdmap e18: 3 total, 3 up, 3 in 2024-09-11T23:11:48.990 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:48 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T23:11:49.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:48 smithi076 ceph-mon[35296]: purged_snaps scrub starts 2024-09-11T23:11:49.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:48 smithi076 ceph-mon[35296]: purged_snaps scrub ok 2024-09-11T23:11:49.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:48 smithi076 ceph-mon[35296]: from='client.14283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi026:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:11:49.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:48 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T23:11:49.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:48 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T23:11:49.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:48 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:11:49.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:48 smithi076 ceph-mon[35296]: osd.2 [v2:172.21.15.136:6800/1271538911,v1:172.21.15.136:6801/1271538911] boot 2024-09-11T23:11:49.054 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:48 smithi076 ceph-mon[35296]: osdmap e18: 3 total, 3 up, 3 in 2024-09-11T23:11:49.054 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:48 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T23:11:49.894 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:49 smithi026 ceph-mon[31172]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-11T23:11:49.904 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:49 smithi026 ceph-mon[31172]: osdmap e19: 3 total, 3 up, 3 in 2024-09-11T23:11:49.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:49 smithi136 ceph-mon[35121]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-11T23:11:49.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:49 smithi136 ceph-mon[35121]: osdmap e19: 3 total, 3 up, 3 in 2024-09-11T23:11:50.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:49 smithi076 ceph-mon[35296]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-11T23:11:50.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:49 smithi076 ceph-mon[35296]: osdmap e19: 3 total, 3 up, 3 in 2024-09-11T23:11:50.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:50 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-11T23:11:50.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:50 smithi136 ceph-mon[35121]: from='client.? 172.21.15.26:0/3823468088' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3e34623f-0ea5-4b10-b8a9-f8756c35d512"}]: dispatch 2024-09-11T23:11:50.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:50 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-11T23:11:50.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:50 smithi136 ceph-mon[35121]: from='client.? 172.21.15.26:0/3823468088' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3e34623f-0ea5-4b10-b8a9-f8756c35d512"}]': finished 2024-09-11T23:11:50.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:50 smithi136 ceph-mon[35121]: osdmap e20: 4 total, 3 up, 4 in 2024-09-11T23:11:50.990 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:50 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T23:11:50.990 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:50 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-11T23:11:51.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:50 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-11T23:11:51.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:50 smithi026 ceph-mon[31172]: from='client.? 172.21.15.26:0/3823468088' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3e34623f-0ea5-4b10-b8a9-f8756c35d512"}]: dispatch 2024-09-11T23:11:51.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:50 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-11T23:11:51.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:50 smithi026 ceph-mon[31172]: from='client.? 172.21.15.26:0/3823468088' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3e34623f-0ea5-4b10-b8a9-f8756c35d512"}]': finished 2024-09-11T23:11:51.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:50 smithi026 ceph-mon[31172]: osdmap e20: 4 total, 3 up, 4 in 2024-09-11T23:11:51.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:50 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T23:11:51.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:50 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-11T23:11:51.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:50 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-11T23:11:51.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:50 smithi076 ceph-mon[35296]: from='client.? 172.21.15.26:0/3823468088' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3e34623f-0ea5-4b10-b8a9-f8756c35d512"}]: dispatch 2024-09-11T23:11:51.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:50 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-11T23:11:51.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:50 smithi076 ceph-mon[35296]: from='client.? 172.21.15.26:0/3823468088' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3e34623f-0ea5-4b10-b8a9-f8756c35d512"}]': finished 2024-09-11T23:11:51.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:50 smithi076 ceph-mon[35296]: osdmap e20: 4 total, 3 up, 4 in 2024-09-11T23:11:51.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:50 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T23:11:51.054 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:50 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-11T23:11:51.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:51 smithi136 ceph-mon[35121]: pgmap v77: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-11T23:11:51.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:51 smithi136 ceph-mon[35121]: from='client.? 172.21.15.26:0/3052274500' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T23:11:51.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:51 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-11T23:11:51.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:51 smithi136 ceph-mon[35121]: osdmap e21: 4 total, 3 up, 4 in 2024-09-11T23:11:51.990 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:51 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T23:11:52.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:51 smithi026 ceph-mon[31172]: pgmap v77: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-11T23:11:52.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:51 smithi026 ceph-mon[31172]: from='client.? 172.21.15.26:0/3052274500' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T23:11:52.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:51 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-11T23:11:52.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:51 smithi026 ceph-mon[31172]: osdmap e21: 4 total, 3 up, 4 in 2024-09-11T23:11:52.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:51 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T23:11:52.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:51 smithi076 ceph-mon[35296]: pgmap v77: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-11T23:11:52.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:51 smithi076 ceph-mon[35296]: from='client.? 172.21.15.26:0/3052274500' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T23:11:52.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:51 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-11T23:11:52.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:51 smithi076 ceph-mon[35296]: osdmap e21: 4 total, 3 up, 4 in 2024-09-11T23:11:52.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:51 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T23:11:52.489 INFO:journalctl@ceph.osd.0.smithi026.stdout:Sep 11 23:11:52 smithi026 sudo[49863]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-11T23:11:52.489 INFO:journalctl@ceph.osd.0.smithi026.stdout:Sep 11 23:11:52 smithi026 sudo[49863]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T23:11:52.489 INFO:journalctl@ceph.osd.0.smithi026.stdout:Sep 11 23:11:52 smithi026 sudo[49863]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T23:11:52.489 INFO:journalctl@ceph.osd.0.smithi026.stdout:Sep 11 23:11:52 smithi026 sudo[49863]: pam_unix(sudo:session): session closed for user root 2024-09-11T23:11:52.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:11:52.676+0000 7f49cf7fe640 1 -- 172.21.15.26:0/2673979280 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f49c00979d0 con 0x7f49d8101cd0 2024-09-11T23:11:52.751 INFO:journalctl@ceph.osd.0.smithi026.stdout:Sep 11 23:11:52 smithi026 sudo[50235]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-11T23:11:52.751 INFO:journalctl@ceph.osd.0.smithi026.stdout:Sep 11 23:11:52 smithi026 sudo[50235]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T23:11:52.751 INFO:journalctl@ceph.osd.0.smithi026.stdout:Sep 11 23:11:52 smithi026 sudo[50235]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T23:11:52.751 INFO:journalctl@ceph.osd.0.smithi026.stdout:Sep 11 23:11:52 smithi026 sudo[50235]: pam_unix(sudo:session): session closed for user root 2024-09-11T23:11:53.021 INFO:journalctl@ceph.osd.1.smithi076.stdout:Sep 11 23:11:52 smithi076 sudo[43489]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-11T23:11:53.021 INFO:journalctl@ceph.osd.1.smithi076.stdout:Sep 11 23:11:52 smithi076 sudo[43489]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T23:11:53.021 INFO:journalctl@ceph.osd.1.smithi076.stdout:Sep 11 23:11:52 smithi076 sudo[43489]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T23:11:53.021 INFO:journalctl@ceph.osd.1.smithi076.stdout:Sep 11 23:11:52 smithi076 sudo[43489]: pam_unix(sudo:session): session closed for user root 2024-09-11T23:11:53.303 INFO:journalctl@ceph.osd.1.smithi076.stdout:Sep 11 23:11:53 smithi076 sudo[43492]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-11T23:11:53.303 INFO:journalctl@ceph.osd.1.smithi076.stdout:Sep 11 23:11:53 smithi076 sudo[43492]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T23:11:53.303 INFO:journalctl@ceph.osd.1.smithi076.stdout:Sep 11 23:11:53 smithi076 sudo[43492]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T23:11:53.303 INFO:journalctl@ceph.osd.1.smithi076.stdout:Sep 11 23:11:53 smithi076 sudo[43492]: pam_unix(sudo:session): session closed for user root 2024-09-11T23:11:53.545 INFO:journalctl@ceph.osd.2.smithi136.stdout:Sep 11 23:11:53 smithi136 sudo[43901]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-11T23:11:53.545 INFO:journalctl@ceph.osd.2.smithi136.stdout:Sep 11 23:11:53 smithi136 sudo[43901]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T23:11:53.546 INFO:journalctl@ceph.osd.2.smithi136.stdout:Sep 11 23:11:53 smithi136 sudo[43901]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T23:11:53.546 INFO:journalctl@ceph.osd.2.smithi136.stdout:Sep 11 23:11:53 smithi136 sudo[43901]: pam_unix(sudo:session): session closed for user root 2024-09-11T23:11:53.546 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:53 smithi136 ceph-mon[35121]: pgmap v80: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-11T23:11:53.546 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:53 smithi136 ceph-mon[35121]: osdmap e22: 4 total, 3 up, 4 in 2024-09-11T23:11:53.546 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:53 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T23:11:53.546 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:53 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:11:53.546 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:53 smithi136 ceph-mon[35121]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-11T23:11:53.546 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:53 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:11:53.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:53 smithi026 ceph-mon[31172]: pgmap v80: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-11T23:11:53.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:53 smithi026 ceph-mon[31172]: osdmap e22: 4 total, 3 up, 4 in 2024-09-11T23:11:53.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:53 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T23:11:53.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:53 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:11:53.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:53 smithi026 ceph-mon[31172]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-11T23:11:53.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:53 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:11:53.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:53 smithi076 ceph-mon[35296]: pgmap v80: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-11T23:11:53.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:53 smithi076 ceph-mon[35296]: osdmap e22: 4 total, 3 up, 4 in 2024-09-11T23:11:53.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:53 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T23:11:53.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:53 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:11:53.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:53 smithi076 ceph-mon[35296]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-11T23:11:53.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:53 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:11:53.989 INFO:journalctl@ceph.osd.2.smithi136.stdout:Sep 11 23:11:53 smithi136 sudo[43904]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-11T23:11:53.989 INFO:journalctl@ceph.osd.2.smithi136.stdout:Sep 11 23:11:53 smithi136 sudo[43904]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T23:11:53.989 INFO:journalctl@ceph.osd.2.smithi136.stdout:Sep 11 23:11:53 smithi136 sudo[43904]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T23:11:53.989 INFO:journalctl@ceph.osd.2.smithi136.stdout:Sep 11 23:11:53 smithi136 sudo[43904]: pam_unix(sudo:session): session closed for user root 2024-09-11T23:11:54.467 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:54 smithi026 sudo[50619]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-11T23:11:54.467 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:54 smithi026 sudo[50619]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T23:11:54.467 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:54 smithi026 sudo[50619]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T23:11:54.467 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:54 smithi026 sudo[50619]: pam_unix(sudo:session): session closed for user root 2024-09-11T23:11:54.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:54 smithi026 sudo[50626]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-11T23:11:54.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:54 smithi026 sudo[50626]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T23:11:54.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:54 smithi026 sudo[50626]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T23:11:54.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:54 smithi026 sudo[50626]: pam_unix(sudo:session): session closed for user root 2024-09-11T23:11:54.971 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:54 smithi136 sudo[43907]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-11T23:11:54.971 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:54 smithi136 sudo[43907]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T23:11:54.971 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:54 smithi136 sudo[43907]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T23:11:54.971 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:54 smithi136 sudo[43907]: pam_unix(sudo:session): session closed for user root 2024-09-11T23:11:55.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:54 smithi136 sudo[43911]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-11T23:11:55.240 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:54 smithi136 sudo[43911]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T23:11:55.240 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:54 smithi136 sudo[43911]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T23:11:55.240 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:54 smithi136 sudo[43911]: pam_unix(sudo:session): session closed for user root 2024-09-11T23:11:55.482 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:55 smithi076 sudo[43495]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-11T23:11:55.482 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:55 smithi076 sudo[43495]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T23:11:55.482 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:55 smithi076 sudo[43495]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T23:11:55.482 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:55 smithi076 sudo[43495]: pam_unix(sudo:session): session closed for user root 2024-09-11T23:11:55.671 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:55 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:11:55.672 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:55 smithi026 ceph-mon[31172]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T23:11:55.672 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:55 smithi026 ceph-mon[31172]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T23:11:55.672 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:55 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T23:11:55.672 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:55 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T23:11:55.672 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:55 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T23:11:55.672 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:55 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T23:11:55.672 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:55 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T23:11:55.672 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:55 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T23:11:55.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:55 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:11:55.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:55 smithi136 ceph-mon[35121]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T23:11:55.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:55 smithi136 ceph-mon[35121]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T23:11:55.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:55 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T23:11:55.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:55 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T23:11:55.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:55 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T23:11:55.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:55 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T23:11:55.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:55 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T23:11:55.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:55 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T23:11:55.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:55 smithi076 sudo[43498]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-11T23:11:55.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:55 smithi076 sudo[43498]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T23:11:55.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:55 smithi076 sudo[43498]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T23:11:55.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:55 smithi076 sudo[43498]: pam_unix(sudo:session): session closed for user root 2024-09-11T23:11:55.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:55 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:11:55.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:55 smithi076 ceph-mon[35296]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T23:11:55.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:55 smithi076 ceph-mon[35296]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T23:11:55.804 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:55 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T23:11:55.804 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:55 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T23:11:55.804 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:55 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T23:11:55.804 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:55 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T23:11:55.804 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:55 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T23:11:55.804 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:55 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T23:11:56.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:56 smithi026 ceph-mon[31172]: pgmap v82: 1 pgs: 1 unknown; 0 B data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-11T23:11:56.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:56 smithi026 ceph-mon[31172]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T23:11:56.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:56 smithi026 ceph-mon[31172]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T23:11:56.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:56 smithi026 ceph-mon[31172]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T23:11:56.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:56 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T23:11:56.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:56 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T23:11:56.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:56 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T23:11:56.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:56 smithi026 ceph-mon[31172]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T23:11:56.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:56 smithi076 ceph-mon[35296]: pgmap v82: 1 pgs: 1 unknown; 0 B data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-11T23:11:56.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:56 smithi076 ceph-mon[35296]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T23:11:56.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:56 smithi076 ceph-mon[35296]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T23:11:56.804 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:56 smithi076 ceph-mon[35296]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T23:11:56.804 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:56 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T23:11:56.804 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:56 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T23:11:56.804 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:56 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T23:11:56.804 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:56 smithi076 ceph-mon[35296]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T23:11:56.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:56 smithi136 ceph-mon[35121]: pgmap v82: 1 pgs: 1 unknown; 0 B data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-11T23:11:56.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:56 smithi136 ceph-mon[35121]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T23:11:56.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:56 smithi136 ceph-mon[35121]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T23:11:56.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:56 smithi136 ceph-mon[35121]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T23:11:56.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:56 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T23:11:56.990 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:56 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T23:11:56.990 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:56 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T23:11:56.990 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:56 smithi136 ceph-mon[35121]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T23:11:57.700 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:57 smithi026 ceph-mon[31172]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-09-11T23:11:57.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:57 smithi076 ceph-mon[35296]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-09-11T23:11:57.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:57 smithi136 ceph-mon[35121]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-09-11T23:11:59.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:59 smithi026 ceph-mon[31172]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-09-11T23:11:59.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:59 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-11T23:11:59.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:11:59 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:11:59.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:59 smithi076 ceph-mon[35296]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-09-11T23:11:59.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:59 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-11T23:11:59.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:11:59 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:11:59.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:59 smithi136 ceph-mon[35121]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-09-11T23:11:59.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:59 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-11T23:11:59.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:11:59 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:12:00.741 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:12:00 smithi026 ceph-mon[31172]: Deploying daemon osd.3 on smithi026 2024-09-11T23:12:00.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:12:00 smithi076 ceph-mon[35296]: Deploying daemon osd.3 on smithi026 2024-09-11T23:12:00.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:12:00 smithi136 ceph-mon[35121]: Deploying daemon osd.3 on smithi026 2024-09-11T23:12:01.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:12:01 smithi026 ceph-mon[31172]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 2.5 KiB/s rd, 65 KiB/s wr, 6 op/s 2024-09-11T23:12:01.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:12:01 smithi076 ceph-mon[35296]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 2.5 KiB/s rd, 65 KiB/s wr, 6 op/s 2024-09-11T23:12:01.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:12:01 smithi136 ceph-mon[35121]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 2.5 KiB/s rd, 65 KiB/s wr, 6 op/s 2024-09-11T23:12:03.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:12:03 smithi076 ceph-mon[35296]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 44 KiB/s wr, 4 op/s 2024-09-11T23:12:03.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:12:03 smithi136 ceph-mon[35121]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 44 KiB/s wr, 4 op/s 2024-09-11T23:12:04.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:12:03 smithi026 ceph-mon[31172]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 44 KiB/s wr, 4 op/s 2024-09-11T23:12:05.740 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:12:05 smithi026 ceph-mon[31172]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 33 KiB/s wr, 3 op/s 2024-09-11T23:12:05.740 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:12:05 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:12:05.741 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:12:05 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:12:05.741 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:12:05 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:12:05.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:12:05 smithi076 ceph-mon[35296]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 33 KiB/s wr, 3 op/s 2024-09-11T23:12:05.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:12:05 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:12:05.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:12:05 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:12:05.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:12:05 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:12:05.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:12:05 smithi136 ceph-mon[35121]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 33 KiB/s wr, 3 op/s 2024-09-11T23:12:05.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:12:05 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:12:05.990 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:12:05 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:12:05.990 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:12:05 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:12:07.634 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:12:07 smithi026 ceph-mon[31172]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2024-09-11T23:12:07.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:12:07 smithi076 ceph-mon[35296]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2024-09-11T23:12:07.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:12:07 smithi136 ceph-mon[35121]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2024-09-11T23:12:08.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:12:08 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:12:08.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:12:08 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:12:08.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:12:08 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:12:08.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:12:08 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:12:08.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:12:08 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:12:08.997 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:12:08 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:12:08.997 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:12:08 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:12:08.997 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:12:08 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:12:08.997 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:12:08 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:12:08.998 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:12:08 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:12:09.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:12:08 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:12:09.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:12:08 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:12:09.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:12:08 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:12:09.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:12:08 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:12:09.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:12:08 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:12:09.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:12:09.296+0000 7f49cf7fe640 1 -- 172.21.15.26:0/2673979280 <== mgr.14148 v2:172.21.15.26:6800/110833859 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f49d80fe4b0 con 0x7f49c40a2ba0 2024-09-11T23:12:09.297 INFO:teuthology.orchestra.run.smithi026.stdout:Created osd(s) 3 on host 'smithi026' 2024-09-11T23:12:09.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:12:09.298+0000 7f49dc8a4640 1 -- 172.21.15.26:0/2673979280 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f49c40a2ba0 msgr2=0x7f49c40a5060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:12:09.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:12:09.298+0000 7f49dc8a4640 1 --2- 172.21.15.26:0/2673979280 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f49c40a2ba0 0x7f49c40a5060 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f49c8005fd0 tx=0x7f49c800b410 comp rx=0 tx=0).stop 2024-09-11T23:12:09.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:12:09.298+0000 7f49dc8a4640 1 -- 172.21.15.26:0/2673979280 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f49d8101cd0 msgr2=0x7f49d819a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:12:09.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:12:09.298+0000 7f49dc8a4640 1 --2- 172.21.15.26:0/2673979280 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f49d8101cd0 0x7f49d819a5d0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f49c000d950 tx=0x7f49c000de10 comp rx=0 tx=0).stop 2024-09-11T23:12:09.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:12:09.298+0000 7f49dc8a4640 1 -- 172.21.15.26:0/2673979280 shutdown_connections 2024-09-11T23:12:09.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:12:09.298+0000 7f49dc8a4640 1 --2- 172.21.15.26:0/2673979280 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f49c40a2ba0 0x7f49c40a5060 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:12:09.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:12:09.298+0000 7f49dc8a4640 1 --2- 172.21.15.26:0/2673979280 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f49d8102690 0x7f49d81a1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:12:09.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:12:09.298+0000 7f49dc8a4640 1 --2- 172.21.15.26:0/2673979280 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f49d8101cd0 0x7f49d819a5d0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:12:09.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:12:09.298+0000 7f49dc8a4640 1 --2- 172.21.15.26:0/2673979280 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f49d8100ad0 0x7f49d819a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:12:09.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:12:09.298+0000 7f49dc8a4640 1 -- 172.21.15.26:0/2673979280 >> 172.21.15.26:0/2673979280 conn(0x7f49d80fc280 msgr2=0x7f49d80fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:12:09.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:12:09.299+0000 7f49dc8a4640 1 -- 172.21.15.26:0/2673979280 shutdown_connections 2024-09-11T23:12:09.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:12:09.299+0000 7f49dc8a4640 1 -- 172.21.15.26:0/2673979280 wait complete. 2024-09-11T23:12:09.921 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:12:09 smithi026 ceph-mon[31172]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-09-11T23:12:09.921 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:12:09 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:12:09.921 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:12:09 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:12:09.921 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:12:09 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:12:09.945 DEBUG:teuthology.orchestra.run.smithi026:osd.3> sudo journalctl -f -n 0 -u ceph-af7e6fc8-7092-11ef-bcea-c7b262605968@osd.3.service 2024-09-11T23:12:09.948 INFO:tasks.cephadm:Deploying osd.4 on smithi076 with /dev/vg_nvme/lv_3... 2024-09-11T23:12:09.948 DEBUG:teuthology.orchestra.run.smithi076:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-11T23:12:09.976 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:12:09 smithi076 ceph-mon[35296]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-09-11T23:12:09.977 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:12:09 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:12:09.977 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:12:09 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:12:09.977 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:12:09 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:12:09.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:12:09 smithi136 ceph-mon[35121]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-09-11T23:12:09.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:12:09 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:12:09.990 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:12:09 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:12:09.990 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:12:09 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:12:10.202 INFO:teuthology.orchestra.run.smithi076.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T23:12:11.586 INFO:journalctl@ceph.osd.3.smithi026.stdout:Sep 11 23:12:11 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-3[51852]: 2024-09-11T23:12:11.334+0000 7f0fea59e640 -1 osd.3 0 log_to_monitors true 2024-09-11T23:12:11.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:12:11 smithi136 ceph-mon[35121]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-09-11T23:12:11.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:12:11 smithi136 ceph-mon[35121]: from='osd.3 [v2:172.21.15.26:6810/1740304151,v1:172.21.15.26:6811/1740304151]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-11T23:12:12.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:12:11 smithi026 ceph-mon[31172]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-09-11T23:12:12.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:12:11 smithi026 ceph-mon[31172]: from='osd.3 [v2:172.21.15.26:6810/1740304151,v1:172.21.15.26:6811/1740304151]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-11T23:12:12.052 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:12:11 smithi076 ceph-mon[35296]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-09-11T23:12:12.052 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:12:11 smithi076 ceph-mon[35296]: from='osd.3 [v2:172.21.15.26:6810/1740304151,v1:172.21.15.26:6811/1740304151]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-11T23:12:12.792 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-11T23:12:12.818 DEBUG:teuthology.orchestra.run.smithi076:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi076:vg_nvme/lv_3 2024-09-11T23:12:12.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:12:12 smithi136 ceph-mon[35121]: from='osd.3 [v2:172.21.15.26:6810/1740304151,v1:172.21.15.26:6811/1740304151]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-11T23:12:12.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:12:12 smithi136 ceph-mon[35121]: osdmap e23: 4 total, 3 up, 4 in 2024-09-11T23:12:12.990 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:12:12 smithi136 ceph-mon[35121]: from='osd.3 [v2:172.21.15.26:6810/1740304151,v1:172.21.15.26:6811/1740304151]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2024-09-11T23:12:12.990 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:12:12 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T23:12:12.990 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:12:12 smithi136 ceph-mon[35121]: from='osd.3 [v2:172.21.15.26:6810/1740304151,v1:172.21.15.26:6811/1740304151]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]': finished 2024-09-11T23:12:12.990 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:12:12 smithi136 ceph-mon[35121]: osdmap e24: 4 total, 3 up, 4 in 2024-09-11T23:12:13.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:12:12 smithi026 ceph-mon[31172]: from='osd.3 [v2:172.21.15.26:6810/1740304151,v1:172.21.15.26:6811/1740304151]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-11T23:12:13.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:12:12 smithi026 ceph-mon[31172]: osdmap e23: 4 total, 3 up, 4 in 2024-09-11T23:12:13.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:12:12 smithi026 ceph-mon[31172]: from='osd.3 [v2:172.21.15.26:6810/1740304151,v1:172.21.15.26:6811/1740304151]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2024-09-11T23:12:13.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:12:12 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T23:12:13.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:12:12 smithi026 ceph-mon[31172]: from='osd.3 [v2:172.21.15.26:6810/1740304151,v1:172.21.15.26:6811/1740304151]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]': finished 2024-09-11T23:12:13.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:12:12 smithi026 ceph-mon[31172]: osdmap e24: 4 total, 3 up, 4 in 2024-09-11T23:12:13.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:12:12 smithi076 ceph-mon[35296]: from='osd.3 [v2:172.21.15.26:6810/1740304151,v1:172.21.15.26:6811/1740304151]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-11T23:12:13.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:12:12 smithi076 ceph-mon[35296]: osdmap e23: 4 total, 3 up, 4 in 2024-09-11T23:12:13.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:12:12 smithi076 ceph-mon[35296]: from='osd.3 [v2:172.21.15.26:6810/1740304151,v1:172.21.15.26:6811/1740304151]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2024-09-11T23:12:13.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:12:12 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T23:12:13.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:12:12 smithi076 ceph-mon[35296]: from='osd.3 [v2:172.21.15.26:6810/1740304151,v1:172.21.15.26:6811/1740304151]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]': finished 2024-09-11T23:12:13.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:12:12 smithi076 ceph-mon[35296]: osdmap e24: 4 total, 3 up, 4 in 2024-09-11T23:12:13.105 INFO:teuthology.orchestra.run.smithi076.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T23:12:13.528 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:12:13.525+0000 7f85f505a640 1 -- 172.21.15.76:0/2211002946 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f85f0100b10 msgr2=0x7f85f0100f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:12:13.528 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:12:13.525+0000 7f85f505a640 1 --2- 172.21.15.76:0/2211002946 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f85f0100b10 0x7f85f0100f10 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f85d8009900 tx=0x7f85d802a410 comp rx=0 tx=0).stop 2024-09-11T23:12:13.528 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:12:13.525+0000 7f85f505a640 1 -- 172.21.15.76:0/2211002946 shutdown_connections 2024-09-11T23:12:13.528 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:12:13.525+0000 7f85f505a640 1 --2- 172.21.15.76:0/2211002946 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f85f01026d0 0x7f85f0108f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:12:13.528 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:12:13.525+0000 7f85f505a640 1 --2- 172.21.15.76:0/2211002946 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f85f0101d10 0x7f85f0102190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:12:13.528 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:12:13.525+0000 7f85f505a640 1 --2- 172.21.15.76:0/2211002946 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f85f0100b10 0x7f85f0100f10 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:12:13.528 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:12:13.525+0000 7f85f505a640 1 -- 172.21.15.76:0/2211002946 >> 172.21.15.76:0/2211002946 conn(0x7f85f00fc2a0 msgr2=0x7f85f00fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:12:13.529 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:12:13.525+0000 7f85f505a640 1 -- 172.21.15.76:0/2211002946 shutdown_connections 2024-09-11T23:12:13.529 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:12:13.525+0000 7f85f505a640 1 -- 172.21.15.76:0/2211002946 wait complete. 2024-09-11T23:12:13.529 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:12:13.526+0000 7f85f505a640 1 Processor -- start 2024-09-11T23:12:13.529 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:12:13.526+0000 7f85f505a640 1 -- start start 2024-09-11T23:12:13.529 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:12:13.526+0000 7f85f505a640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f85f0100b10 0x7f85f0197f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:12:13.529 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:12:13.526+0000 7f85f505a640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f85f0101d10 0x7f85f01984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:12:13.529 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:12:13.526+0000 7f85f505a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f85f01026d0 0x7f85f019f530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:12:13.529 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:12:13.526+0000 7f85f505a640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f85f010c7a0 con 0x7f85f01026d0 2024-09-11T23:12:13.529 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:12:13.526+0000 7f85f505a640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f85f010c620 con 0x7f85f0101d10 2024-09-11T23:12:13.529 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:12:13.526+0000 7f85f505a640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f85f010c920 con 0x7f85f0100b10 2024-09-11T23:12:13.529 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:12:13.526+0000 7f85ee59c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f85f0101d10 0x7f85f01984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:12:13.529 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:12:13.526+0000 7f85ee59c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f85f0101d10 0x7f85f01984b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:60468/0 (socket says 172.21.15.76:60468) 2024-09-11T23:12:13.530 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:12:13.526+0000 7f85ee59c640 1 -- 172.21.15.76:0/1783365846 learned_addr learned my addr 172.21.15.76:0/1783365846 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-11T23:12:13.530 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:12:13.526+0000 7f85eed9d640 1 --2- 172.21.15.76:0/1783365846 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f85f0100b10 0x7f85f0197f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:12:13.530 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:12:13.526+0000 7f85ef59e640 1 --2- 172.21.15.76:0/1783365846 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f85f01026d0 0x7f85f019f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:12:13.530 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:12:13.527+0000 7f85ee59c640 1 -- 172.21.15.76:0/1783365846 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f85f0100b10 msgr2=0x7f85f0197f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:12:13.530 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:12:13.527+0000 7f85ee59c640 1 --2- 172.21.15.76:0/1783365846 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f85f0100b10 0x7f85f0197f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:12:13.530 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:12:13.527+0000 7f85ee59c640 1 -- 172.21.15.76:0/1783365846 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f85f01026d0 msgr2=0x7f85f019f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:12:13.530 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:12:13.527+0000 7f85ee59c640 1 --2- 172.21.15.76:0/1783365846 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f85f01026d0 0x7f85f019f530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:12:13.530 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:12:13.527+0000 7f85ee59c640 1 -- 172.21.15.76:0/1783365846 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85f019fba0 con 0x7f85f0101d10 2024-09-11T23:12:13.530 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:12:13.527+0000 7f85ee59c640 1 --2- 172.21.15.76:0/1783365846 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f85f0101d10 0x7f85f01984b0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f85e000e9f0 tx=0x7f85e000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:12:13.530 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:12:13.527+0000 7f85e7fff640 1 -- 172.21.15.76:0/1783365846 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85e000cd80 con 0x7f85f0101d10 2024-09-11T23:12:13.530 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:12:13.527+0000 7f85f505a640 1 -- 172.21.15.76:0/1783365846 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85f0102f90 con 0x7f85f0101d10 2024-09-11T23:12:13.530 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:12:13.527+0000 7f85e7fff640 1 -- 172.21.15.76:0/1783365846 <== mon.2 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85e0010910 con 0x7f85f0101d10 2024-09-11T23:12:13.530 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:12:13.527+0000 7f85e7fff640 1 -- 172.21.15.76:0/1783365846 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85e0018990 con 0x7f85f0101d10 2024-09-11T23:12:13.530 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:12:13.527+0000 7f85f505a640 1 -- 172.21.15.76:0/1783365846 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85f0103520 con 0x7f85f0101d10 2024-09-11T23:12:13.531 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:12:13.528+0000 7f85f505a640 1 -- 172.21.15.76:0/1783365846 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85f0103840 con 0x7f85f0101d10 2024-09-11T23:12:13.531 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:12:13.529+0000 7f85e7fff640 1 -- 172.21.15.76:0/1783365846 <== mon.2 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f85e0010ac0 con 0x7f85f0101d10 2024-09-11T23:12:13.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:12:13.530+0000 7f85e7fff640 1 --2- 172.21.15.76:0/1783365846 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f85dc0a2c30 0x7f85dc0a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:12:13.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:12:13.530+0000 7f85e7fff640 1 -- 172.21.15.76:0/1783365846 <== mon.2 v2:172.21.15.76:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 3389+0+0 (secure 0 0 0) 0x7f85e00cdf50 con 0x7f85f0101d10 2024-09-11T23:12:13.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:12:13.530+0000 7f85eed9d640 1 --2- 172.21.15.76:0/1783365846 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f85dc0a2c30 0x7f85dc0a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:12:13.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:12:13.531+0000 7f85eed9d640 1 --2- 172.21.15.76:0/1783365846 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f85dc0a2c30 0x7f85dc0a50f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f85d8005ab0 tx=0x7f85d8000bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:12:13.534 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:12:13.534+0000 7f85e7fff640 1 -- 172.21.15.76:0/1783365846 <== mon.2 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f85e009bba0 con 0x7f85f0101d10 2024-09-11T23:12:13.702 INFO:journalctl@ceph.osd.3.smithi026.stdout:Sep 11 23:12:13 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-3[51852]: 2024-09-11T23:12:13.417+0000 7f0fe6d35640 -1 osd.3 0 waiting for initial osdmap 2024-09-11T23:12:13.703 INFO:journalctl@ceph.osd.3.smithi026.stdout:Sep 11 23:12:13 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-3[51852]: 2024-09-11T23:12:13.423+0000 7f0fde330640 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T23:12:13.715 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:12:13.714+0000 7f85f505a640 1 -- 172.21.15.76:0/1783365846 --> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi076:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f85f006af10 con 0x7f85dc0a2c30 2024-09-11T23:12:13.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:12:13 smithi136 ceph-mon[35121]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-09-11T23:12:13.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:12:13 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T23:12:13.990 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:12:13 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T23:12:13.990 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:12:13 smithi136 ceph-mon[35121]: Detected new or changed devices on smithi026 2024-09-11T23:12:13.990 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:12:13 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:12:13.990 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:12:13 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:12:13.990 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:12:13 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:12:13.990 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:12:13 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:12:13.990 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:12:13 smithi136 ceph-mon[35121]: Adjusting osd_memory_target on smithi026 to 8546M 2024-09-11T23:12:13.990 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:12:13 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:12:13.990 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:12:13 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:12:13.990 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:12:13 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:12:13.990 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:12:13 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:12:13.990 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:12:13 smithi136 ceph-mon[35121]: from='osd.3 [v2:172.21.15.26:6810/1740304151,v1:172.21.15.26:6811/1740304151]' entity='osd.3' 2024-09-11T23:12:13.990 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:12:13 smithi136 ceph-mon[35121]: osd.3 [v2:172.21.15.26:6810/1740304151,v1:172.21.15.26:6811/1740304151] boot 2024-09-11T23:12:13.990 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:12:13 smithi136 ceph-mon[35121]: osdmap e25: 4 total, 4 up, 4 in 2024-09-11T23:12:13.991 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:12:13 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T23:12:14.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:12:13 smithi026 ceph-mon[31172]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-09-11T23:12:14.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:12:13 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T23:12:14.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:12:13 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T23:12:14.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:12:13 smithi026 ceph-mon[31172]: Detected new or changed devices on smithi026 2024-09-11T23:12:14.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:12:13 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:12:14.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:12:13 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:12:14.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:12:13 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:12:14.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:12:13 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:12:14.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:12:13 smithi026 ceph-mon[31172]: Adjusting osd_memory_target on smithi026 to 8546M 2024-09-11T23:12:14.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:12:13 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:12:14.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:12:13 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:12:14.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:12:13 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:12:14.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:12:13 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:12:14.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:12:13 smithi026 ceph-mon[31172]: from='osd.3 [v2:172.21.15.26:6810/1740304151,v1:172.21.15.26:6811/1740304151]' entity='osd.3' 2024-09-11T23:12:14.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:12:13 smithi026 ceph-mon[31172]: osd.3 [v2:172.21.15.26:6810/1740304151,v1:172.21.15.26:6811/1740304151] boot 2024-09-11T23:12:14.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:12:13 smithi026 ceph-mon[31172]: osdmap e25: 4 total, 4 up, 4 in 2024-09-11T23:12:14.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:12:13 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T23:12:14.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:12:13 smithi076 ceph-mon[35296]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-09-11T23:12:14.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:12:13 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T23:12:14.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:12:13 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T23:12:14.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:12:13 smithi076 ceph-mon[35296]: Detected new or changed devices on smithi026 2024-09-11T23:12:14.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:12:13 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:12:14.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:12:13 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:12:14.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:12:13 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:12:14.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:12:13 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:12:14.054 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:12:13 smithi076 ceph-mon[35296]: Adjusting osd_memory_target on smithi026 to 8546M 2024-09-11T23:12:14.054 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:12:13 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:12:14.054 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:12:13 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:12:14.054 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:12:13 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:12:14.054 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:12:13 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:12:14.054 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:12:13 smithi076 ceph-mon[35296]: from='osd.3 [v2:172.21.15.26:6810/1740304151,v1:172.21.15.26:6811/1740304151]' entity='osd.3' 2024-09-11T23:12:14.054 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:12:13 smithi076 ceph-mon[35296]: osd.3 [v2:172.21.15.26:6810/1740304151,v1:172.21.15.26:6811/1740304151] boot 2024-09-11T23:12:14.054 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:12:13 smithi076 ceph-mon[35296]: osdmap e25: 4 total, 4 up, 4 in 2024-09-11T23:12:14.054 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:12:13 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T23:12:14.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:12:14 smithi136 ceph-mon[35121]: purged_snaps scrub starts 2024-09-11T23:12:14.990 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:12:14 smithi136 ceph-mon[35121]: purged_snaps scrub ok 2024-09-11T23:12:14.990 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:12:14 smithi136 ceph-mon[35121]: from='client.24215 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi076:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:12:14.990 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:12:14 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T23:12:14.990 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:12:14 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T23:12:14.990 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:12:14 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:12:14.990 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:12:14 smithi136 ceph-mon[35121]: osdmap e26: 4 total, 4 up, 4 in 2024-09-11T23:12:15.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:12:14 smithi026 ceph-mon[31172]: purged_snaps scrub starts 2024-09-11T23:12:15.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:12:14 smithi026 ceph-mon[31172]: purged_snaps scrub ok 2024-09-11T23:12:15.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:12:14 smithi026 ceph-mon[31172]: from='client.24215 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi076:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:12:15.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:12:14 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T23:12:15.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:12:14 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T23:12:15.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:12:14 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:12:15.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:12:14 smithi026 ceph-mon[31172]: osdmap e26: 4 total, 4 up, 4 in 2024-09-11T23:12:15.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:12:14 smithi076 ceph-mon[35296]: purged_snaps scrub starts 2024-09-11T23:12:15.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:12:14 smithi076 ceph-mon[35296]: purged_snaps scrub ok 2024-09-11T23:12:15.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:12:14 smithi076 ceph-mon[35296]: from='client.24215 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi076:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:12:15.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:12:14 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T23:12:15.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:12:14 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T23:12:15.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:12:14 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:12:15.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:12:14 smithi076 ceph-mon[35296]: osdmap e26: 4 total, 4 up, 4 in 2024-09-11T23:12:15.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:12:15 smithi136 ceph-mon[35121]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-11T23:12:15.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:12:15 smithi136 ceph-mon[35121]: osdmap e27: 4 total, 4 up, 4 in 2024-09-11T23:12:16.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:12:15 smithi026 ceph-mon[31172]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-11T23:12:16.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:12:15 smithi026 ceph-mon[31172]: osdmap e27: 4 total, 4 up, 4 in 2024-09-11T23:12:16.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:12:15 smithi076 ceph-mon[35296]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-11T23:12:16.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:12:15 smithi076 ceph-mon[35296]: osdmap e27: 4 total, 4 up, 4 in 2024-09-11T23:12:16.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:12:16 smithi136 ceph-mon[35121]: from='client.? 172.21.15.76:0/749648917' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "77607929-f9ed-49a9-abb5-48cd88eec4ef"}]: dispatch 2024-09-11T23:12:16.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:12:16 smithi136 ceph-mon[35121]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "77607929-f9ed-49a9-abb5-48cd88eec4ef"}]: dispatch 2024-09-11T23:12:16.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:12:16 smithi136 ceph-mon[35121]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "77607929-f9ed-49a9-abb5-48cd88eec4ef"}]': finished 2024-09-11T23:12:16.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:12:16 smithi136 ceph-mon[35121]: osdmap e28: 5 total, 4 up, 5 in 2024-09-11T23:12:16.990 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:12:16 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T23:12:16.990 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:12:16 smithi136 ceph-mon[35121]: from='client.? 172.21.15.76:0/3758642102' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T23:12:17.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:12:16 smithi026 ceph-mon[31172]: from='client.? 172.21.15.76:0/749648917' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "77607929-f9ed-49a9-abb5-48cd88eec4ef"}]: dispatch 2024-09-11T23:12:17.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:12:16 smithi026 ceph-mon[31172]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "77607929-f9ed-49a9-abb5-48cd88eec4ef"}]: dispatch 2024-09-11T23:12:17.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:12:16 smithi026 ceph-mon[31172]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "77607929-f9ed-49a9-abb5-48cd88eec4ef"}]': finished 2024-09-11T23:12:17.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:12:16 smithi026 ceph-mon[31172]: osdmap e28: 5 total, 4 up, 5 in 2024-09-11T23:12:17.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:12:16 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T23:12:17.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:12:16 smithi026 ceph-mon[31172]: from='client.? 172.21.15.76:0/3758642102' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T23:12:17.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:12:16 smithi076 ceph-mon[35296]: from='client.? 172.21.15.76:0/749648917' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "77607929-f9ed-49a9-abb5-48cd88eec4ef"}]: dispatch 2024-09-11T23:12:17.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:12:16 smithi076 ceph-mon[35296]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "77607929-f9ed-49a9-abb5-48cd88eec4ef"}]: dispatch 2024-09-11T23:12:17.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:12:16 smithi076 ceph-mon[35296]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "77607929-f9ed-49a9-abb5-48cd88eec4ef"}]': finished 2024-09-11T23:12:17.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:12:16 smithi076 ceph-mon[35296]: osdmap e28: 5 total, 4 up, 5 in 2024-09-11T23:12:17.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:12:16 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T23:12:17.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:12:16 smithi076 ceph-mon[35296]: from='client.? 172.21.15.76:0/3758642102' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T23:12:17.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:12:17 smithi136 ceph-mon[35121]: pgmap v98: 1 pgs: 1 remapped+peering; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-09-11T23:12:17.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:12:17 smithi136 ceph-mon[35121]: osdmap e29: 5 total, 4 up, 5 in 2024-09-11T23:12:17.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:12:17 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T23:12:18.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:12:17 smithi026 ceph-mon[31172]: pgmap v98: 1 pgs: 1 remapped+peering; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-09-11T23:12:18.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:12:17 smithi026 ceph-mon[31172]: osdmap e29: 5 total, 4 up, 5 in 2024-09-11T23:12:18.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:12:17 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T23:12:18.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:12:17 smithi076 ceph-mon[35296]: pgmap v98: 1 pgs: 1 remapped+peering; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-09-11T23:12:18.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:12:17 smithi076 ceph-mon[35296]: osdmap e29: 5 total, 4 up, 5 in 2024-09-11T23:12:18.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:12:17 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T23:12:19.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:12:19 smithi136 ceph-mon[35121]: pgmap v101: 1 pgs: 1 remapped+peering; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-09-11T23:12:19.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:12:19 smithi136 ceph-mon[35121]: osdmap e30: 5 total, 4 up, 5 in 2024-09-11T23:12:19.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:12:19 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T23:12:19.501 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:12:19 smithi026 ceph-mon[31172]: pgmap v101: 1 pgs: 1 remapped+peering; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-09-11T23:12:19.501 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:12:19 smithi026 ceph-mon[31172]: osdmap e30: 5 total, 4 up, 5 in 2024-09-11T23:12:19.501 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:12:19 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T23:12:19.553 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:12:19 smithi076 ceph-mon[35296]: pgmap v101: 1 pgs: 1 remapped+peering; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-09-11T23:12:19.553 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:12:19 smithi076 ceph-mon[35296]: osdmap e30: 5 total, 4 up, 5 in 2024-09-11T23:12:19.553 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:12:19 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T23:12:21.484 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:12:21 smithi076 ceph-mon[35296]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 133 KiB/s, 0 objects/s recovering 2024-09-11T23:12:21.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:12:21 smithi136 ceph-mon[35121]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 133 KiB/s, 0 objects/s recovering 2024-09-11T23:12:21.501 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:12:21 smithi026 ceph-mon[31172]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 133 KiB/s, 0 objects/s recovering 2024-09-11T23:12:23.134 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:12:23 smithi076 ceph-mon[35296]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-11T23:12:23.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:12:23 smithi136 ceph-mon[35121]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-11T23:12:23.501 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:12:23 smithi026 ceph-mon[31172]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-11T23:12:25.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:12:25 smithi136 ceph-mon[35121]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-11T23:12:25.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:12:25 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-11T23:12:25.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:12:25 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:12:25.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:12:25 smithi136 ceph-mon[35121]: Deploying daemon osd.4 on smithi076 2024-09-11T23:12:25.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:12:25 smithi026 ceph-mon[31172]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-11T23:12:25.501 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:12:25 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-11T23:12:25.501 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:12:25 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:12:25.501 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:12:25 smithi026 ceph-mon[31172]: Deploying daemon osd.4 on smithi076 2024-09-11T23:12:25.553 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:12:25 smithi076 ceph-mon[35296]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-11T23:12:25.553 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:12:25 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-11T23:12:25.553 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:12:25 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:12:25.553 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:12:25 smithi076 ceph-mon[35296]: Deploying daemon osd.4 on smithi076 2024-09-11T23:12:27.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:12:27 smithi076 ceph-mon[35296]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-09-11T23:12:27.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:12:27 smithi136 ceph-mon[35121]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-09-11T23:12:27.501 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:12:27 smithi026 ceph-mon[31172]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-09-11T23:12:29.282 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:12:29 smithi076 ceph-mon[35296]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-11T23:12:29.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:12:29 smithi136 ceph-mon[35121]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-11T23:12:29.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:12:29 smithi026 ceph-mon[31172]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-11T23:12:31.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:12:31 smithi076 ceph-mon[35296]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-11T23:12:31.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:12:31 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:12:31.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:12:31 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:12:31.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:12:31 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:12:31.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:12:31 smithi136 ceph-mon[35121]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-11T23:12:31.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:12:31 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:12:31.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:12:31 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:12:31.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:12:31 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:12:31.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:12:31 smithi026 ceph-mon[31172]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-11T23:12:31.501 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:12:31 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:12:31.501 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:12:31 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:12:31.501 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:12:31 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:12:33.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:12:33 smithi076 ceph-mon[35296]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-09-11T23:12:33.458 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:12:33 smithi026 ceph-mon[31172]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-09-11T23:12:33.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:12:33 smithi136 ceph-mon[35121]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-09-11T23:12:34.163 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:12:34.162+0000 7f85e7fff640 1 -- 172.21.15.76:0/1783365846 <== mgr.14148 v2:172.21.15.26:6800/110833859 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f85f006af10 con 0x7f85dc0a2c30 2024-09-11T23:12:34.163 INFO:teuthology.orchestra.run.smithi076.stdout:Created osd(s) 4 on host 'smithi076' 2024-09-11T23:12:34.165 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:12:34.164+0000 7f85f505a640 1 -- 172.21.15.76:0/1783365846 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f85dc0a2c30 msgr2=0x7f85dc0a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:12:34.165 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:12:34.164+0000 7f85f505a640 1 --2- 172.21.15.76:0/1783365846 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f85dc0a2c30 0x7f85dc0a50f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f85d8005ab0 tx=0x7f85d8000bc0 comp rx=0 tx=0).stop 2024-09-11T23:12:34.165 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:12:34.164+0000 7f85f505a640 1 -- 172.21.15.76:0/1783365846 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f85f0101d10 msgr2=0x7f85f01984b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:12:34.165 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:12:34.164+0000 7f85f505a640 1 --2- 172.21.15.76:0/1783365846 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f85f0101d10 0x7f85f01984b0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f85e000e9f0 tx=0x7f85e000eeb0 comp rx=0 tx=0).stop 2024-09-11T23:12:34.165 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:12:34.164+0000 7f85f505a640 1 -- 172.21.15.76:0/1783365846 shutdown_connections 2024-09-11T23:12:34.165 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:12:34.164+0000 7f85f505a640 1 --2- 172.21.15.76:0/1783365846 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f85dc0a2c30 0x7f85dc0a50f0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:12:34.165 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:12:34.164+0000 7f85f505a640 1 --2- 172.21.15.76:0/1783365846 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f85f0101d10 0x7f85f01984b0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:12:34.165 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:12:34.164+0000 7f85f505a640 1 --2- 172.21.15.76:0/1783365846 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f85f01026d0 0x7f85f019f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:12:34.165 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:12:34.164+0000 7f85f505a640 1 --2- 172.21.15.76:0/1783365846 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f85f0100b10 0x7f85f0197f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:12:34.165 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:12:34.164+0000 7f85f505a640 1 -- 172.21.15.76:0/1783365846 >> 172.21.15.76:0/1783365846 conn(0x7f85f00fc2a0 msgr2=0x7f85f006a800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:12:34.165 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:12:34.164+0000 7f85f505a640 1 -- 172.21.15.76:0/1783365846 shutdown_connections 2024-09-11T23:12:34.165 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:12:34.164+0000 7f85f505a640 1 -- 172.21.15.76:0/1783365846 wait complete. 2024-09-11T23:12:34.731 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:12:34 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:12:34.731 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:12:34 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:12:34.731 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:12:34 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:12:34.731 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:12:34 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:12:34.731 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:12:34 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:12:34.732 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:12:34 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:12:34.732 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:12:34 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:12:34.732 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:12:34 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:12:34.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:12:34 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:12:34.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:12:34 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:12:34.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:12:34 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:12:34.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:12:34 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:12:34.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:12:34 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:12:34.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:12:34 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:12:34.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:12:34 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:12:34.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:12:34 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:12:34.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:12:34 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:12:34.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:12:34 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:12:34.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:12:34 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:12:34.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:12:34 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:12:34.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:12:34 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:12:34.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:12:34 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:12:34.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:12:34 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:12:34.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:12:34 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:12:34.843 DEBUG:teuthology.orchestra.run.smithi076:osd.4> sudo journalctl -f -n 0 -u ceph-af7e6fc8-7092-11ef-bcea-c7b262605968@osd.4.service 2024-09-11T23:12:34.845 INFO:tasks.cephadm:Deploying osd.5 on smithi136 with /dev/vg_nvme/lv_3... 2024-09-11T23:12:34.845 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-11T23:12:35.097 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T23:12:35.481 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:12:35 smithi136 ceph-mon[35121]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-09-11T23:12:35.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:12:35 smithi026 ceph-mon[31172]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-09-11T23:12:35.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:12:35 smithi076 ceph-mon[35296]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-09-11T23:12:37.146 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-09-11T23:12:37.171 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi136:vg_nvme/lv_3 2024-09-11T23:12:37.233 INFO:journalctl@ceph.osd.4.smithi076.stdout:Sep 11 23:12:36 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-4[46247]: 2024-09-11T23:12:36.873+0000 7fa6fb9d3640 -1 osd.4 0 log_to_monitors true 2024-09-11T23:12:37.433 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T23:12:37.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:12:37 smithi136 ceph-mon[35121]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-09-11T23:12:37.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:12:37 smithi136 ceph-mon[35121]: from='osd.4 [v2:172.21.15.76:6808/2702262182,v1:172.21.15.76:6809/2702262182]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-11T23:12:37.490 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:12:37 smithi136 ceph-mon[35121]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-11T23:12:37.553 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:12:37 smithi076 ceph-mon[35296]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-09-11T23:12:37.553 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:12:37 smithi076 ceph-mon[35296]: from='osd.4 [v2:172.21.15.76:6808/2702262182,v1:172.21.15.76:6809/2702262182]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-11T23:12:37.553 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:12:37 smithi076 ceph-mon[35296]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-11T23:12:37.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:12:37 smithi026 ceph-mon[31172]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-09-11T23:12:37.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:12:37 smithi026 ceph-mon[31172]: from='osd.4 [v2:172.21.15.76:6808/2702262182,v1:172.21.15.76:6809/2702262182]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-11T23:12:37.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:12:37 smithi026 ceph-mon[31172]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-11T23:12:37.900 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:12:37.897+0000 7fb436e42640 1 -- 172.21.15.136:0/2103717736 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb430101c00 msgr2=0x7fb430102000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:12:37.900 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:12:37.897+0000 7fb436e42640 1 --2- 172.21.15.136:0/2103717736 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb430101c00 0x7fb430102000 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fb424009900 tx=0x7fb42402a410 comp rx=0 tx=0).stop 2024-09-11T23:12:37.900 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:12:37.897+0000 7fb436e42640 1 -- 172.21.15.136:0/2103717736 shutdown_connections 2024-09-11T23:12:37.900 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:12:37.897+0000 7fb436e42640 1 --2- 172.21.15.136:0/2103717736 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb430102e00 0x7fb430103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:12:37.900 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:12:37.897+0000 7fb436e42640 1 --2- 172.21.15.136:0/2103717736 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb4301037c0 0x7fb43010a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:12:37.900 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:12:37.897+0000 7fb436e42640 1 --2- 172.21.15.136:0/2103717736 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb430101c00 0x7fb430102000 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:12:37.901 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:12:37.897+0000 7fb436e42640 1 -- 172.21.15.136:0/2103717736 >> 172.21.15.136:0/2103717736 conn(0x7fb4300fd3b0 msgr2=0x7fb4300ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:12:37.901 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:12:37.897+0000 7fb436e42640 1 -- 172.21.15.136:0/2103717736 shutdown_connections 2024-09-11T23:12:37.901 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:12:37.897+0000 7fb436e42640 1 -- 172.21.15.136:0/2103717736 wait complete. 2024-09-11T23:12:37.901 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:12:37.898+0000 7fb436e42640 1 Processor -- start 2024-09-11T23:12:37.901 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:12:37.898+0000 7fb436e42640 1 -- start start 2024-09-11T23:12:37.901 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:12:37.898+0000 7fb436e42640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb430101c00 0x7fb430197e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:12:37.901 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:12:37.898+0000 7fb436e42640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb430102e00 0x7fb4301983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:12:37.901 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:12:37.898+0000 7fb436e42640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb4301037c0 0x7fb43019f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:12:37.901 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:12:37.898+0000 7fb436e42640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb43010d880 con 0x7fb4301037c0 2024-09-11T23:12:37.901 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:12:37.898+0000 7fb436e42640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fb43010d700 con 0x7fb430101c00 2024-09-11T23:12:37.901 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:12:37.898+0000 7fb436e42640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb43010da00 con 0x7fb430102e00 2024-09-11T23:12:37.901 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:12:37.898+0000 7fb42ffff640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb430102e00 0x7fb4301983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:12:37.902 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:12:37.898+0000 7fb42ffff640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb430102e00 0x7fb4301983c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.136:58876/0 (socket says 172.21.15.136:58876) 2024-09-11T23:12:37.902 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:12:37.898+0000 7fb42ffff640 1 -- 172.21.15.136:0/443402133 learned_addr learned my addr 172.21.15.136:0/443402133 (peer_addr_for_me v2:172.21.15.136:0/0) 2024-09-11T23:12:37.902 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:12:37.898+0000 7fb434bde640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb430101c00 0x7fb430197e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:12:37.902 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:12:37.899+0000 7fb4353df640 1 --2- 172.21.15.136:0/443402133 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb4301037c0 0x7fb43019f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:12:37.902 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:12:37.899+0000 7fb42ffff640 1 -- 172.21.15.136:0/443402133 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb430101c00 msgr2=0x7fb430197e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:12:37.902 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:12:37.899+0000 7fb42ffff640 1 --2- 172.21.15.136:0/443402133 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb430101c00 0x7fb430197e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:12:37.902 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:12:37.899+0000 7fb42ffff640 1 -- 172.21.15.136:0/443402133 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb4301037c0 msgr2=0x7fb43019f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:12:37.902 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:12:37.899+0000 7fb42ffff640 1 --2- 172.21.15.136:0/443402133 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb4301037c0 0x7fb43019f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:12:37.902 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:12:37.899+0000 7fb42ffff640 1 -- 172.21.15.136:0/443402133 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb43019fab0 con 0x7fb430102e00 2024-09-11T23:12:37.902 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:12:37.899+0000 7fb42ffff640 1 --2- 172.21.15.136:0/443402133 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb430102e00 0x7fb4301983c0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fb41c00d770 tx=0x7fb41c00dc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:12:37.902 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:12:37.899+0000 7fb42dffb640 1 -- 172.21.15.136:0/443402133 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb41c017070 con 0x7fb430102e00 2024-09-11T23:12:37.902 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:12:37.899+0000 7fb436e42640 1 -- 172.21.15.136:0/443402133 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb43019fda0 con 0x7fb430102e00 2024-09-11T23:12:37.902 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:12:37.899+0000 7fb42dffb640 1 -- 172.21.15.136:0/443402133 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb41c005500 con 0x7fb430102e00 2024-09-11T23:12:37.903 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:12:37.899+0000 7fb42dffb640 1 -- 172.21.15.136:0/443402133 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb41c004e00 con 0x7fb430102e00 2024-09-11T23:12:37.903 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:12:37.899+0000 7fb436e42640 1 -- 172.21.15.136:0/443402133 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4301a0200 con 0x7fb430102e00 2024-09-11T23:12:37.903 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:12:37.901+0000 7fb436e42640 1 -- 172.21.15.136:0/443402133 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb430108fb0 con 0x7fb430102e00 2024-09-11T23:12:37.904 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:12:37.901+0000 7fb42dffb640 1 -- 172.21.15.136:0/443402133 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fb41c004fa0 con 0x7fb430102e00 2024-09-11T23:12:37.904 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:12:37.902+0000 7fb42dffb640 1 --2- 172.21.15.136:0/443402133 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fb4200a2bb0 0x7fb4200a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:12:37.904 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:12:37.902+0000 7fb42dffb640 1 -- 172.21.15.136:0/443402133 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 3805+0+0 (secure 0 0 0) 0x7fb41c0cd890 con 0x7fb430102e00 2024-09-11T23:12:37.904 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:12:37.902+0000 7fb434bde640 1 --2- 172.21.15.136:0/443402133 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fb4200a2bb0 0x7fb4200a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:12:37.904 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:12:37.903+0000 7fb434bde640 1 --2- 172.21.15.136:0/443402133 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fb4200a2bb0 0x7fb4200a5070 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb424005ab0 tx=0x7fb424000bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:12:37.907 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:12:37.906+0000 7fb42dffb640 1 -- 172.21.15.136:0/443402133 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb41c09b990 con 0x7fb430102e00 2024-09-11T23:12:38.089 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:12:38.088+0000 7fb436e42640 1 -- 172.21.15.136:0/443402133 --> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi136:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fb430062630 con 0x7fb4200a2bb0 2024-09-11T23:12:38.698 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:12:38 smithi136 ceph-mon[35121]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-11T23:12:38.699 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:12:38 smithi136 ceph-mon[35121]: osdmap e31: 5 total, 4 up, 5 in 2024-09-11T23:12:38.699 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:12:38 smithi136 ceph-mon[35121]: from='osd.4 [v2:172.21.15.76:6808/2702262182,v1:172.21.15.76:6809/2702262182]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]: dispatch 2024-09-11T23:12:38.699 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:12:38 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T23:12:38.699 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:12:38 smithi136 ceph-mon[35121]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]: dispatch 2024-09-11T23:12:38.699 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:12:38 smithi136 ceph-mon[35121]: Detected new or changed devices on smithi076 2024-09-11T23:12:38.699 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:12:38 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:12:38.699 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:12:38 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:12:38.699 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:12:38 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:12:38.699 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:12:38 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:12:38.699 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:12:38 smithi136 ceph-mon[35121]: Adjusting osd_memory_target on smithi076 to 8544M 2024-09-11T23:12:38.699 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:12:38 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:12:38.700 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:12:38 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:12:38.700 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:12:38 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:12:38.700 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:12:38 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:12:38.700 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:12:38 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T23:12:38.700 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:12:38 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T23:12:38.700 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:12:38 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:12:38.700 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:12:38 smithi136 ceph-mon[35121]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]': finished 2024-09-11T23:12:38.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:12:38 smithi026 ceph-mon[31172]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-11T23:12:38.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:12:38 smithi026 ceph-mon[31172]: osdmap e31: 5 total, 4 up, 5 in 2024-09-11T23:12:38.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:12:38 smithi026 ceph-mon[31172]: from='osd.4 [v2:172.21.15.76:6808/2702262182,v1:172.21.15.76:6809/2702262182]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]: dispatch 2024-09-11T23:12:38.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:12:38 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T23:12:38.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:12:38 smithi026 ceph-mon[31172]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]: dispatch 2024-09-11T23:12:38.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:12:38 smithi026 ceph-mon[31172]: Detected new or changed devices on smithi076 2024-09-11T23:12:38.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:12:38 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:12:38.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:12:38 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:12:38.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:12:38 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:12:38.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:12:38 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:12:38.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:12:38 smithi026 ceph-mon[31172]: Adjusting osd_memory_target on smithi076 to 8544M 2024-09-11T23:12:38.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:12:38 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:12:38.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:12:38 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:12:38.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:12:38 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:12:38.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:12:38 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:12:38.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:12:38 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T23:12:38.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:12:38 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T23:12:38.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:12:38 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:12:38.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:12:38 smithi026 ceph-mon[31172]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]': finished 2024-09-11T23:12:38.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:12:38 smithi076 ceph-mon[35296]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-11T23:12:38.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:12:38 smithi076 ceph-mon[35296]: osdmap e31: 5 total, 4 up, 5 in 2024-09-11T23:12:38.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:12:38 smithi076 ceph-mon[35296]: from='osd.4 [v2:172.21.15.76:6808/2702262182,v1:172.21.15.76:6809/2702262182]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]: dispatch 2024-09-11T23:12:38.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:12:38 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T23:12:38.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:12:38 smithi076 ceph-mon[35296]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]: dispatch 2024-09-11T23:12:38.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:12:38 smithi076 ceph-mon[35296]: Detected new or changed devices on smithi076 2024-09-11T23:12:38.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:12:38 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:12:38.804 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:12:38 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:12:38.804 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:12:38 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:12:38.804 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:12:38 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:12:38.804 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:12:38 smithi076 ceph-mon[35296]: Adjusting osd_memory_target on smithi076 to 8544M 2024-09-11T23:12:38.804 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:12:38 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:12:38.804 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:12:38 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:12:38.804 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:12:38 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:12:38.804 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:12:38 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:12:38.804 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:12:38 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T23:12:38.804 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:12:38 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T23:12:38.804 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:12:38 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:12:38.804 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:12:38 smithi076 ceph-mon[35296]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]': finished 2024-09-11T23:12:39.589 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:12:39 smithi136 ceph-mon[35121]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-09-11T23:12:39.589 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:12:39 smithi136 ceph-mon[35121]: from='client.24226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi136:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:12:39.589 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:12:39 smithi136 ceph-mon[35121]: osdmap e32: 5 total, 4 up, 5 in 2024-09-11T23:12:39.589 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:12:39 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T23:12:39.589 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:12:39 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T23:12:39.589 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:12:39 smithi136 ceph-mon[35121]: from='osd.4 ' entity='osd.4' 2024-09-11T23:12:39.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:12:39 smithi026 ceph-mon[31172]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-09-11T23:12:39.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:12:39 smithi026 ceph-mon[31172]: from='client.24226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi136:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:12:39.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:12:39 smithi026 ceph-mon[31172]: osdmap e32: 5 total, 4 up, 5 in 2024-09-11T23:12:39.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:12:39 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T23:12:39.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:12:39 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T23:12:39.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:12:39 smithi026 ceph-mon[31172]: from='osd.4 ' entity='osd.4' 2024-09-11T23:12:39.803 INFO:journalctl@ceph.osd.4.smithi076.stdout:Sep 11 23:12:39 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-4[46247]: 2024-09-11T23:12:39.355+0000 7fa6f7133640 -1 osd.4 0 waiting for initial osdmap 2024-09-11T23:12:39.803 INFO:journalctl@ceph.osd.4.smithi076.stdout:Sep 11 23:12:39 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-4[46247]: 2024-09-11T23:12:39.363+0000 7fa6ef765640 -1 osd.4 32 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T23:12:39.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:12:39 smithi076 ceph-mon[35296]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-09-11T23:12:39.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:12:39 smithi076 ceph-mon[35296]: from='client.24226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi136:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:12:39.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:12:39 smithi076 ceph-mon[35296]: osdmap e32: 5 total, 4 up, 5 in 2024-09-11T23:12:39.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:12:39 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T23:12:39.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:12:39 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T23:12:39.804 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:12:39 smithi076 ceph-mon[35296]: from='osd.4 ' entity='osd.4' 2024-09-11T23:12:40.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:12:40 smithi136 ceph-mon[35121]: purged_snaps scrub starts 2024-09-11T23:12:40.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:12:40 smithi136 ceph-mon[35121]: purged_snaps scrub ok 2024-09-11T23:12:40.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:12:40 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T23:12:40.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:12:40 smithi136 ceph-mon[35121]: from='client.? 172.21.15.136:0/1751618806' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "29e20e37-09a0-4924-8aaa-17939763a7c2"}]: dispatch 2024-09-11T23:12:40.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:12:40 smithi136 ceph-mon[35121]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "29e20e37-09a0-4924-8aaa-17939763a7c2"}]: dispatch 2024-09-11T23:12:40.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:12:40 smithi136 ceph-mon[35121]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "29e20e37-09a0-4924-8aaa-17939763a7c2"}]': finished 2024-09-11T23:12:40.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:12:40 smithi136 ceph-mon[35121]: osd.4 [v2:172.21.15.76:6808/2702262182,v1:172.21.15.76:6809/2702262182] boot 2024-09-11T23:12:40.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:12:40 smithi136 ceph-mon[35121]: osdmap e33: 6 total, 5 up, 6 in 2024-09-11T23:12:40.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:12:40 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T23:12:40.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:12:40 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T23:12:40.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:12:40 smithi026 ceph-mon[31172]: purged_snaps scrub starts 2024-09-11T23:12:40.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:12:40 smithi026 ceph-mon[31172]: purged_snaps scrub ok 2024-09-11T23:12:40.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:12:40 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T23:12:40.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:12:40 smithi026 ceph-mon[31172]: from='client.? 172.21.15.136:0/1751618806' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "29e20e37-09a0-4924-8aaa-17939763a7c2"}]: dispatch 2024-09-11T23:12:40.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:12:40 smithi026 ceph-mon[31172]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "29e20e37-09a0-4924-8aaa-17939763a7c2"}]: dispatch 2024-09-11T23:12:40.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:12:40 smithi026 ceph-mon[31172]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "29e20e37-09a0-4924-8aaa-17939763a7c2"}]': finished 2024-09-11T23:12:40.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:12:40 smithi026 ceph-mon[31172]: osd.4 [v2:172.21.15.76:6808/2702262182,v1:172.21.15.76:6809/2702262182] boot 2024-09-11T23:12:40.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:12:40 smithi026 ceph-mon[31172]: osdmap e33: 6 total, 5 up, 6 in 2024-09-11T23:12:40.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:12:40 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T23:12:40.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:12:40 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T23:12:40.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:12:40 smithi076 ceph-mon[35296]: purged_snaps scrub starts 2024-09-11T23:12:40.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:12:40 smithi076 ceph-mon[35296]: purged_snaps scrub ok 2024-09-11T23:12:40.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:12:40 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T23:12:40.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:12:40 smithi076 ceph-mon[35296]: from='client.? 172.21.15.136:0/1751618806' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "29e20e37-09a0-4924-8aaa-17939763a7c2"}]: dispatch 2024-09-11T23:12:40.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:12:40 smithi076 ceph-mon[35296]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "29e20e37-09a0-4924-8aaa-17939763a7c2"}]: dispatch 2024-09-11T23:12:40.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:12:40 smithi076 ceph-mon[35296]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "29e20e37-09a0-4924-8aaa-17939763a7c2"}]': finished 2024-09-11T23:12:40.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:12:40 smithi076 ceph-mon[35296]: osd.4 [v2:172.21.15.76:6808/2702262182,v1:172.21.15.76:6809/2702262182] boot 2024-09-11T23:12:40.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:12:40 smithi076 ceph-mon[35296]: osdmap e33: 6 total, 5 up, 6 in 2024-09-11T23:12:40.804 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:12:40 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T23:12:40.804 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:12:40 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T23:12:41.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:12:41 smithi076 ceph-mon[35296]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-09-11T23:12:41.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:12:41 smithi076 ceph-mon[35296]: from='client.? 172.21.15.136:0/3141280006' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T23:12:41.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:12:41 smithi136 ceph-mon[35121]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-09-11T23:12:41.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:12:41 smithi136 ceph-mon[35121]: from='client.? 172.21.15.136:0/3141280006' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T23:12:42.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:12:41 smithi026 ceph-mon[31172]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-09-11T23:12:42.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:12:41 smithi026 ceph-mon[31172]: from='client.? 172.21.15.136:0/3141280006' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T23:12:42.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:12:42 smithi076 ceph-mon[35296]: osdmap e34: 6 total, 5 up, 6 in 2024-09-11T23:12:42.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:12:42 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T23:12:42.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:12:42 smithi136 ceph-mon[35121]: osdmap e34: 6 total, 5 up, 6 in 2024-09-11T23:12:42.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:12:42 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T23:12:43.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:12:42 smithi026 ceph-mon[31172]: osdmap e34: 6 total, 5 up, 6 in 2024-09-11T23:12:43.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:12:42 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T23:12:43.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:12:43 smithi076 ceph-mon[35296]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-11T23:12:43.890 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:12:43 smithi136 ceph-mon[35121]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-11T23:12:44.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:12:43 smithi026 ceph-mon[31172]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-11T23:12:45.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:12:45 smithi136 ceph-mon[35121]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-11T23:12:45.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:12:45 smithi076 ceph-mon[35296]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-11T23:12:46.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:12:45 smithi026 ceph-mon[31172]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-11T23:12:47.779 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:12:47 smithi136 ceph-mon[35121]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-11T23:12:47.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:12:47 smithi076 ceph-mon[35296]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-11T23:12:48.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:12:47 smithi026 ceph-mon[31172]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-11T23:12:49.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:12:49 smithi136 ceph-mon[35121]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-11T23:12:49.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:12:49 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-11T23:12:49.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:12:49 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:12:49.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:12:49 smithi136 ceph-mon[35121]: Deploying daemon osd.5 on smithi136 2024-09-11T23:12:49.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:12:49 smithi076 ceph-mon[35296]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-11T23:12:49.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:12:49 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-11T23:12:49.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:12:49 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:12:49.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:12:49 smithi076 ceph-mon[35296]: Deploying daemon osd.5 on smithi136 2024-09-11T23:12:50.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:12:49 smithi026 ceph-mon[31172]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-11T23:12:50.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:12:49 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-11T23:12:50.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:12:49 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:12:50.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:12:49 smithi026 ceph-mon[31172]: Deploying daemon osd.5 on smithi136 2024-09-11T23:12:51.690 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:12:51 smithi136 ceph-mon[35121]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-11T23:12:51.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:12:51 smithi076 ceph-mon[35296]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-11T23:12:52.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:12:51 smithi026 ceph-mon[31172]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-11T23:12:53.646 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:12:53 smithi136 ceph-mon[35121]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-11T23:12:53.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:12:53 smithi076 ceph-mon[35296]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-11T23:12:54.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:12:53 smithi026 ceph-mon[31172]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-11T23:12:54.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:12:54 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:12:54.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:12:54 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:12:54.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:12:54 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:12:54.849 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:12:54 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:12:54.849 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:12:54 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:12:54.849 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:12:54 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:12:55.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:12:54 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:12:55.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:12:54 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:12:55.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:12:54 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:12:55.789 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:12:55 smithi136 ceph-mon[35121]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-11T23:12:55.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:12:55 smithi076 ceph-mon[35296]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-11T23:12:56.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:12:55 smithi026 ceph-mon[31172]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-11T23:12:57.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:12:57 smithi076 ceph-mon[35296]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-11T23:12:57.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:12:57 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:12:57.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:12:57 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:12:57.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:12:57 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:12:57.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:12:57 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:12:57.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:12:57 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:12:57.949 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:12:57 smithi136 ceph-mon[35121]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-11T23:12:57.949 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:12:57 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:12:57.949 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:12:57 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:12:57.949 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:12:57 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:12:57.949 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:12:57 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:12:57.949 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:12:57 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:12:58.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:12:57 smithi026 ceph-mon[31172]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-11T23:12:58.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:12:57 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:12:58.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:12:57 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:12:58.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:12:57 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:12:58.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:12:57 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:12:58.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:12:57 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:12:58.251 INFO:teuthology.orchestra.run.smithi136.stdout:Created osd(s) 5 on host 'smithi136' 2024-09-11T23:12:58.251 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:12:58.250+0000 7fb42dffb640 1 -- 172.21.15.136:0/443402133 <== mgr.14148 v2:172.21.15.26:6800/110833859 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fb430062630 con 0x7fb4200a2bb0 2024-09-11T23:12:58.253 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:12:58.253+0000 7fb436e42640 1 -- 172.21.15.136:0/443402133 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fb4200a2bb0 msgr2=0x7fb4200a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:12:58.253 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:12:58.253+0000 7fb436e42640 1 --2- 172.21.15.136:0/443402133 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fb4200a2bb0 0x7fb4200a5070 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb424005ab0 tx=0x7fb424000bc0 comp rx=0 tx=0).stop 2024-09-11T23:12:58.253 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:12:58.253+0000 7fb436e42640 1 -- 172.21.15.136:0/443402133 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb430102e00 msgr2=0x7fb4301983c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:12:58.253 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:12:58.253+0000 7fb436e42640 1 --2- 172.21.15.136:0/443402133 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb430102e00 0x7fb4301983c0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fb41c00d770 tx=0x7fb41c00dc30 comp rx=0 tx=0).stop 2024-09-11T23:12:58.253 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:12:58.253+0000 7fb436e42640 1 -- 172.21.15.136:0/443402133 shutdown_connections 2024-09-11T23:12:58.253 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:12:58.253+0000 7fb436e42640 1 --2- 172.21.15.136:0/443402133 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fb4200a2bb0 0x7fb4200a5070 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:12:58.253 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:12:58.253+0000 7fb436e42640 1 --2- 172.21.15.136:0/443402133 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb4301037c0 0x7fb43019f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:12:58.254 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:12:58.253+0000 7fb436e42640 1 --2- 172.21.15.136:0/443402133 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb430102e00 0x7fb4301983c0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:12:58.254 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:12:58.253+0000 7fb436e42640 1 --2- 172.21.15.136:0/443402133 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb430101c00 0x7fb430197e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:12:58.254 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:12:58.253+0000 7fb436e42640 1 -- 172.21.15.136:0/443402133 >> 172.21.15.136:0/443402133 conn(0x7fb4300fd3b0 msgr2=0x7fb430106180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:12:58.254 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:12:58.253+0000 7fb436e42640 1 -- 172.21.15.136:0/443402133 shutdown_connections 2024-09-11T23:12:58.254 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:12:58.253+0000 7fb436e42640 1 -- 172.21.15.136:0/443402133 wait complete. 2024-09-11T23:12:58.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:12:58 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:12:58.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:12:58 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:12:58.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:12:58 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:12:58.837 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:12:58 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:12:58.837 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:12:58 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:12:58.837 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:12:58 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:12:58.864 DEBUG:teuthology.orchestra.run.smithi136:osd.5> sudo journalctl -f -n 0 -u ceph-af7e6fc8-7092-11ef-bcea-c7b262605968@osd.5.service 2024-09-11T23:12:58.866 INFO:tasks.cephadm:Deploying osd.6 on smithi026 with /dev/vg_nvme/lv_2... 2024-09-11T23:12:58.866 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-11T23:12:58.896 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:12:58 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:12:58.896 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:12:58 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:12:58.896 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:12:58 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:12:59.137 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:12:59.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:12:59 smithi076 ceph-mon[35296]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-11T23:12:59.960 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:12:59 smithi136 ceph-mon[35121]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-11T23:13:00.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:12:59 smithi026 ceph-mon[31172]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-11T23:13:00.990 INFO:journalctl@ceph.osd.5.smithi136.stdout:Sep 11 23:13:00 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-5[46773]: 2024-09-11T23:13:00.730+0000 7f5a278da640 -1 osd.5 0 log_to_monitors true 2024-09-11T23:13:01.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:01 smithi026 ceph-mon[31172]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-11T23:13:01.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:01 smithi026 ceph-mon[31172]: from='osd.5 [v2:172.21.15.136:6808/1749229277,v1:172.21.15.136:6809/1749229277]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-11T23:13:01.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:01 smithi026 ceph-mon[31172]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-11T23:13:01.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:01 smithi076 ceph-mon[35296]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-11T23:13:01.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:01 smithi076 ceph-mon[35296]: from='osd.5 [v2:172.21.15.136:6808/1749229277,v1:172.21.15.136:6809/1749229277]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-11T23:13:01.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:01 smithi076 ceph-mon[35296]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-11T23:13:01.889 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-11T23:13:01.916 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi026:vg_nvme/lv_2 2024-09-11T23:13:01.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:01 smithi136 ceph-mon[35121]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-11T23:13:01.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:01 smithi136 ceph-mon[35121]: from='osd.5 [v2:172.21.15.136:6808/1749229277,v1:172.21.15.136:6809/1749229277]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-11T23:13:01.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:01 smithi136 ceph-mon[35121]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-11T23:13:02.215 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:13:02.600 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:02 smithi026 ceph-mon[31172]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-11T23:13:02.600 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:02 smithi026 ceph-mon[31172]: osdmap e35: 6 total, 5 up, 6 in 2024-09-11T23:13:02.600 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:02 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T23:13:02.600 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:02 smithi026 ceph-mon[31172]: from='osd.5 [v2:172.21.15.136:6808/1749229277,v1:172.21.15.136:6809/1749229277]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-09-11T23:13:02.600 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:02 smithi026 ceph-mon[31172]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-09-11T23:13:02.600 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:02 smithi026 ceph-mon[31172]: Detected new or changed devices on smithi136 2024-09-11T23:13:02.601 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:02 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:13:02.601 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:02 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:13:02.601 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:02 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:13:02.601 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:02 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:13:02.601 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:02 smithi026 ceph-mon[31172]: Adjusting osd_memory_target on smithi136 to 8544M 2024-09-11T23:13:02.601 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:02 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:13:02.601 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:02 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:13:02.601 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:02 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:13:02.601 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:02 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:13:02.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:13:02.773+0000 7fae664d5640 1 -- 172.21.15.26:0/1869752880 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae60102660 msgr2=0x7fae60104a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:13:02.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:13:02.773+0000 7fae664d5640 1 --2- 172.21.15.26:0/1869752880 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae60102660 0x7fae60104a50 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fae50009900 tx=0x7fae5002a3f0 comp rx=0 tx=0).stop 2024-09-11T23:13:02.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:13:02.773+0000 7fae664d5640 1 -- 172.21.15.26:0/1869752880 shutdown_connections 2024-09-11T23:13:02.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:13:02.773+0000 7fae664d5640 1 --2- 172.21.15.26:0/1869752880 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae60104f90 0x7fae60107380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:13:02.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:13:02.773+0000 7fae664d5640 1 --2- 172.21.15.26:0/1869752880 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fae601078c0 0x7fae60109d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:13:02.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:13:02.773+0000 7fae664d5640 1 --2- 172.21.15.26:0/1869752880 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae60102660 0x7fae60104a50 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:13:02.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:13:02.773+0000 7fae664d5640 1 -- 172.21.15.26:0/1869752880 >> 172.21.15.26:0/1869752880 conn(0x7fae600fc280 msgr2=0x7fae600fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:13:02.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:13:02.773+0000 7fae664d5640 1 -- 172.21.15.26:0/1869752880 shutdown_connections 2024-09-11T23:13:02.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:13:02.773+0000 7fae664d5640 1 -- 172.21.15.26:0/1869752880 wait complete. 2024-09-11T23:13:02.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:13:02.774+0000 7fae664d5640 1 Processor -- start 2024-09-11T23:13:02.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:13:02.774+0000 7fae664d5640 1 -- start start 2024-09-11T23:13:02.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:13:02.774+0000 7fae664d5640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae60102660 0x7fae60197ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:13:02.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:13:02.775+0000 7fae664d5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae60104f90 0x7fae60198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:13:02.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:13:02.775+0000 7fae664d5640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fae601078c0 0x7fae6019f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:13:02.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:13:02.775+0000 7fae664d5640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fae6010d890 con 0x7fae60104f90 2024-09-11T23:13:02.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:13:02.775+0000 7fae664d5640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fae6010d710 con 0x7fae60102660 2024-09-11T23:13:02.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:13:02.775+0000 7fae664d5640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fae6010da10 con 0x7fae601078c0 2024-09-11T23:13:02.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:13:02.775+0000 7fae5f7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae60104f90 0x7fae60198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:13:02.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:13:02.775+0000 7fae5f7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae60104f90 0x7fae60198400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34146/0 (socket says 172.21.15.26:34146) 2024-09-11T23:13:02.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:13:02.775+0000 7fae5f7fe640 1 -- 172.21.15.26:0/4187294383 learned_addr learned my addr 172.21.15.26:0/4187294383 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:13:02.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:13:02.775+0000 7fae64a72640 1 --2- 172.21.15.26:0/4187294383 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fae601078c0 0x7fae6019f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:13:02.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:13:02.775+0000 7fae5ffff640 1 --2- 172.21.15.26:0/4187294383 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae60102660 0x7fae60197ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:13:02.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:13:02.775+0000 7fae5f7fe640 1 -- 172.21.15.26:0/4187294383 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fae601078c0 msgr2=0x7fae6019f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:13:02.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:13:02.775+0000 7fae5f7fe640 1 --2- 172.21.15.26:0/4187294383 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fae601078c0 0x7fae6019f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:13:02.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:13:02.775+0000 7fae5f7fe640 1 -- 172.21.15.26:0/4187294383 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae60102660 msgr2=0x7fae60197ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:13:02.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:13:02.775+0000 7fae5f7fe640 1 --2- 172.21.15.26:0/4187294383 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae60102660 0x7fae60197ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:13:02.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:13:02.775+0000 7fae5f7fe640 1 -- 172.21.15.26:0/4187294383 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae6019faf0 con 0x7fae60104f90 2024-09-11T23:13:02.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:13:02.775+0000 7fae5ffff640 1 --2- 172.21.15.26:0/4187294383 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae60102660 0x7fae60197ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:13:02.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:13:02.775+0000 7fae5f7fe640 1 --2- 172.21.15.26:0/4187294383 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae60104f90 0x7fae60198400 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fae5400c970 tx=0x7fae5400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:13:02.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:13:02.775+0000 7fae5d7fa640 1 -- 172.21.15.26:0/4187294383 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae54014070 con 0x7fae60104f90 2024-09-11T23:13:02.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:13:02.775+0000 7fae664d5640 1 -- 172.21.15.26:0/4187294383 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae6019fde0 con 0x7fae60104f90 2024-09-11T23:13:02.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:13:02.775+0000 7fae5d7fa640 1 -- 172.21.15.26:0/4187294383 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae54004440 con 0x7fae60104f90 2024-09-11T23:13:02.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:13:02.775+0000 7fae5d7fa640 1 -- 172.21.15.26:0/4187294383 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae5400f450 con 0x7fae60104f90 2024-09-11T23:13:02.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:13:02.775+0000 7fae664d5640 1 -- 172.21.15.26:0/4187294383 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae601a02e0 con 0x7fae60104f90 2024-09-11T23:13:02.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:13:02.776+0000 7fae664d5640 1 -- 172.21.15.26:0/4187294383 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae600fdd70 con 0x7fae60104f90 2024-09-11T23:13:02.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:13:02.778+0000 7fae5d7fa640 1 -- 172.21.15.26:0/4187294383 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fae54017070 con 0x7fae60104f90 2024-09-11T23:13:02.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:13:02.778+0000 7fae5d7fa640 1 --2- 172.21.15.26:0/4187294383 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fae480a2bb0 0x7fae480a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:13:02.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:13:02.778+0000 7fae5ffff640 1 --2- 172.21.15.26:0/4187294383 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fae480a2bb0 0x7fae480a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:13:02.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:13:02.779+0000 7fae5d7fa640 1 -- 172.21.15.26:0/4187294383 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4253+0+0 (secure 0 0 0) 0x7fae540cdc40 con 0x7fae60104f90 2024-09-11T23:13:02.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:13:02.779+0000 7fae5ffff640 1 --2- 172.21.15.26:0/4187294383 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fae480a2bb0 0x7fae480a5070 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fae50000c00 tx=0x7fae50031400 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:13:02.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:13:02.786+0000 7fae5d7fa640 1 -- 172.21.15.26:0/4187294383 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fae5401e030 con 0x7fae60104f90 2024-09-11T23:13:02.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:13:02.969+0000 7fae664d5640 1 -- 172.21.15.26:0/4187294383 --> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi026:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fae60062630 con 0x7fae480a2bb0 2024-09-11T23:13:02.990 INFO:journalctl@ceph.osd.5.smithi136.stdout:Sep 11 23:13:02 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-5[46773]: 2024-09-11T23:13:02.562+0000 7f5a2303a640 -1 osd.5 0 waiting for initial osdmap 2024-09-11T23:13:02.990 INFO:journalctl@ceph.osd.5.smithi136.stdout:Sep 11 23:13:02 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-5[46773]: 2024-09-11T23:13:02.569+0000 7f5a1f674640 -1 osd.5 36 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T23:13:02.990 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:02 smithi136 ceph-mon[35121]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-11T23:13:02.990 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:02 smithi136 ceph-mon[35121]: osdmap e35: 6 total, 5 up, 6 in 2024-09-11T23:13:02.990 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:02 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T23:13:02.991 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:02 smithi136 ceph-mon[35121]: from='osd.5 [v2:172.21.15.136:6808/1749229277,v1:172.21.15.136:6809/1749229277]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-09-11T23:13:02.991 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:02 smithi136 ceph-mon[35121]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-09-11T23:13:02.991 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:02 smithi136 ceph-mon[35121]: Detected new or changed devices on smithi136 2024-09-11T23:13:02.991 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:02 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:13:02.991 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:02 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:13:02.991 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:02 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:13:02.991 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:02 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:13:02.991 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:02 smithi136 ceph-mon[35121]: Adjusting osd_memory_target on smithi136 to 8544M 2024-09-11T23:13:02.991 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:02 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:13:02.991 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:02 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:13:02.991 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:02 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:13:02.991 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:02 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:13:03.054 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:02 smithi076 ceph-mon[35296]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-11T23:13:03.054 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:02 smithi076 ceph-mon[35296]: osdmap e35: 6 total, 5 up, 6 in 2024-09-11T23:13:03.054 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:02 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T23:13:03.054 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:02 smithi076 ceph-mon[35296]: from='osd.5 [v2:172.21.15.136:6808/1749229277,v1:172.21.15.136:6809/1749229277]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-09-11T23:13:03.054 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:02 smithi076 ceph-mon[35296]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-09-11T23:13:03.054 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:02 smithi076 ceph-mon[35296]: Detected new or changed devices on smithi136 2024-09-11T23:13:03.054 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:02 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:13:03.054 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:02 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:13:03.054 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:02 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:13:03.054 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:02 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:13:03.055 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:02 smithi076 ceph-mon[35296]: Adjusting osd_memory_target on smithi136 to 8544M 2024-09-11T23:13:03.055 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:02 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:13:03.055 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:02 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:13:03.055 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:02 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:13:03.055 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:02 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:13:03.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:03 smithi026 ceph-mon[31172]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-11T23:13:03.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:03 smithi026 ceph-mon[31172]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]': finished 2024-09-11T23:13:03.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:03 smithi026 ceph-mon[31172]: osdmap e36: 6 total, 5 up, 6 in 2024-09-11T23:13:03.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:03 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T23:13:03.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:03 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T23:13:03.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:03 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T23:13:03.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:03 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T23:13:03.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:03 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:13:03.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:03 smithi136 ceph-mon[35121]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-11T23:13:03.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:03 smithi136 ceph-mon[35121]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]': finished 2024-09-11T23:13:03.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:03 smithi136 ceph-mon[35121]: osdmap e36: 6 total, 5 up, 6 in 2024-09-11T23:13:03.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:03 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T23:13:03.990 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:03 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T23:13:03.990 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:03 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T23:13:03.990 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:03 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T23:13:03.990 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:03 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:13:04.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:03 smithi076 ceph-mon[35296]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-11T23:13:04.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:03 smithi076 ceph-mon[35296]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]': finished 2024-09-11T23:13:04.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:03 smithi076 ceph-mon[35296]: osdmap e36: 6 total, 5 up, 6 in 2024-09-11T23:13:04.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:03 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T23:13:04.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:03 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T23:13:04.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:03 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T23:13:04.054 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:03 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T23:13:04.054 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:03 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:13:04.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:04 smithi136 ceph-mon[35121]: purged_snaps scrub starts 2024-09-11T23:13:04.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:04 smithi136 ceph-mon[35121]: purged_snaps scrub ok 2024-09-11T23:13:04.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:04 smithi136 ceph-mon[35121]: from='client.14352 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi026:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:13:04.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:04 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T23:13:04.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:04 smithi136 ceph-mon[35121]: osd.5 [v2:172.21.15.136:6808/1749229277,v1:172.21.15.136:6809/1749229277] boot 2024-09-11T23:13:04.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:04 smithi136 ceph-mon[35121]: osdmap e37: 6 total, 6 up, 6 in 2024-09-11T23:13:04.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:04 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T23:13:04.895 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:04 smithi026 ceph-mon[31172]: purged_snaps scrub starts 2024-09-11T23:13:04.895 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:04 smithi026 ceph-mon[31172]: purged_snaps scrub ok 2024-09-11T23:13:04.896 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:04 smithi026 ceph-mon[31172]: from='client.14352 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi026:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:13:04.896 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:04 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T23:13:04.896 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:04 smithi026 ceph-mon[31172]: osd.5 [v2:172.21.15.136:6808/1749229277,v1:172.21.15.136:6809/1749229277] boot 2024-09-11T23:13:04.896 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:04 smithi026 ceph-mon[31172]: osdmap e37: 6 total, 6 up, 6 in 2024-09-11T23:13:04.896 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:04 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T23:13:05.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:04 smithi076 ceph-mon[35296]: purged_snaps scrub starts 2024-09-11T23:13:05.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:04 smithi076 ceph-mon[35296]: purged_snaps scrub ok 2024-09-11T23:13:05.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:04 smithi076 ceph-mon[35296]: from='client.14352 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi026:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:13:05.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:04 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T23:13:05.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:04 smithi076 ceph-mon[35296]: osd.5 [v2:172.21.15.136:6808/1749229277,v1:172.21.15.136:6809/1749229277] boot 2024-09-11T23:13:05.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:04 smithi076 ceph-mon[35296]: osdmap e37: 6 total, 6 up, 6 in 2024-09-11T23:13:05.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:04 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T23:13:05.890 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:05 smithi026 ceph-mon[31172]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-11T23:13:05.891 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:05 smithi026 ceph-mon[31172]: osdmap e38: 6 total, 6 up, 6 in 2024-09-11T23:13:05.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:05 smithi136 ceph-mon[35121]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-11T23:13:05.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:05 smithi136 ceph-mon[35121]: osdmap e38: 6 total, 6 up, 6 in 2024-09-11T23:13:06.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:05 smithi076 ceph-mon[35296]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-11T23:13:06.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:05 smithi076 ceph-mon[35296]: osdmap e38: 6 total, 6 up, 6 in 2024-09-11T23:13:06.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:06 smithi136 ceph-mon[35121]: osdmap e39: 6 total, 6 up, 6 in 2024-09-11T23:13:06.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:06 smithi136 ceph-mon[35121]: from='client.? 172.21.15.26:0/407456850' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e0ddd1c0-1c46-41e6-b2f4-396bdcd2bf8a"}]: dispatch 2024-09-11T23:13:06.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:06 smithi136 ceph-mon[35121]: from='client.? 172.21.15.26:0/407456850' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e0ddd1c0-1c46-41e6-b2f4-396bdcd2bf8a"}]': finished 2024-09-11T23:13:06.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:06 smithi136 ceph-mon[35121]: osdmap e40: 7 total, 6 up, 7 in 2024-09-11T23:13:06.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:06 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T23:13:06.990 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:06 smithi136 ceph-mon[35121]: from='client.? 172.21.15.26:0/2275288337' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T23:13:07.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:06 smithi026 ceph-mon[31172]: osdmap e39: 6 total, 6 up, 6 in 2024-09-11T23:13:07.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:06 smithi026 ceph-mon[31172]: from='client.? 172.21.15.26:0/407456850' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e0ddd1c0-1c46-41e6-b2f4-396bdcd2bf8a"}]: dispatch 2024-09-11T23:13:07.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:06 smithi026 ceph-mon[31172]: from='client.? 172.21.15.26:0/407456850' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e0ddd1c0-1c46-41e6-b2f4-396bdcd2bf8a"}]': finished 2024-09-11T23:13:07.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:06 smithi026 ceph-mon[31172]: osdmap e40: 7 total, 6 up, 7 in 2024-09-11T23:13:07.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:06 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T23:13:07.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:06 smithi026 ceph-mon[31172]: from='client.? 172.21.15.26:0/2275288337' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T23:13:07.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:06 smithi076 ceph-mon[35296]: osdmap e39: 6 total, 6 up, 6 in 2024-09-11T23:13:07.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:06 smithi076 ceph-mon[35296]: from='client.? 172.21.15.26:0/407456850' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e0ddd1c0-1c46-41e6-b2f4-396bdcd2bf8a"}]: dispatch 2024-09-11T23:13:07.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:06 smithi076 ceph-mon[35296]: from='client.? 172.21.15.26:0/407456850' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e0ddd1c0-1c46-41e6-b2f4-396bdcd2bf8a"}]': finished 2024-09-11T23:13:07.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:06 smithi076 ceph-mon[35296]: osdmap e40: 7 total, 6 up, 7 in 2024-09-11T23:13:07.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:06 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T23:13:07.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:06 smithi076 ceph-mon[35296]: from='client.? 172.21.15.26:0/2275288337' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T23:13:07.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:07 smithi136 ceph-mon[35121]: pgmap v136: 1 pgs: 1 remapped+peering; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-09-11T23:13:07.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:07 smithi136 ceph-mon[35121]: osdmap e41: 7 total, 6 up, 7 in 2024-09-11T23:13:07.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:07 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T23:13:08.000 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:07 smithi026 ceph-mon[31172]: pgmap v136: 1 pgs: 1 remapped+peering; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-09-11T23:13:08.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:07 smithi026 ceph-mon[31172]: osdmap e41: 7 total, 6 up, 7 in 2024-09-11T23:13:08.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:07 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T23:13:08.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:07 smithi076 ceph-mon[35296]: pgmap v136: 1 pgs: 1 remapped+peering; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-09-11T23:13:08.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:07 smithi076 ceph-mon[35296]: osdmap e41: 7 total, 6 up, 7 in 2024-09-11T23:13:08.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:07 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T23:13:08.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:08 smithi136 ceph-mon[35121]: osdmap e42: 7 total, 6 up, 7 in 2024-09-11T23:13:08.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:08 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T23:13:09.000 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:08 smithi026 ceph-mon[31172]: osdmap e42: 7 total, 6 up, 7 in 2024-09-11T23:13:09.000 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:08 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T23:13:09.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:08 smithi076 ceph-mon[35296]: osdmap e42: 7 total, 6 up, 7 in 2024-09-11T23:13:09.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:08 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T23:13:09.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:09 smithi136 ceph-mon[35121]: pgmap v139: 1 pgs: 1 remapped+peering; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-09-11T23:13:10.000 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:09 smithi026 ceph-mon[31172]: pgmap v139: 1 pgs: 1 remapped+peering; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-09-11T23:13:10.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:09 smithi076 ceph-mon[35296]: pgmap v139: 1 pgs: 1 remapped+peering; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-09-11T23:13:11.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:11 smithi026 ceph-mon[31172]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 129 KiB/s, 0 objects/s recovering 2024-09-11T23:13:11.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:11 smithi136 ceph-mon[35121]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 129 KiB/s, 0 objects/s recovering 2024-09-11T23:13:12.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:11 smithi076 ceph-mon[35296]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 129 KiB/s, 0 objects/s recovering 2024-09-11T23:13:13.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:13 smithi026 ceph-mon[31172]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 90 KiB/s, 0 objects/s recovering 2024-09-11T23:13:13.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:13 smithi136 ceph-mon[35121]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 90 KiB/s, 0 objects/s recovering 2024-09-11T23:13:14.055 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:13 smithi076 ceph-mon[35296]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 90 KiB/s, 0 objects/s recovering 2024-09-11T23:13:14.801 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:14 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-11T23:13:14.801 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:14 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:13:14.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:14 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-11T23:13:14.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:14 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:13:15.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:14 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-11T23:13:15.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:14 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:13:15.874 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:15 smithi026 ceph-mon[31172]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-11T23:13:15.874 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:15 smithi026 ceph-mon[31172]: Deploying daemon osd.6 on smithi026 2024-09-11T23:13:15.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:15 smithi136 ceph-mon[35121]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-11T23:13:15.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:15 smithi136 ceph-mon[35121]: Deploying daemon osd.6 on smithi026 2024-09-11T23:13:16.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:15 smithi076 ceph-mon[35296]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-11T23:13:16.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:15 smithi076 ceph-mon[35296]: Deploying daemon osd.6 on smithi026 2024-09-11T23:13:17.849 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:17 smithi026 ceph-mon[31172]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-09-11T23:13:17.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:17 smithi136 ceph-mon[35121]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-09-11T23:13:18.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:17 smithi076 ceph-mon[35296]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-09-11T23:13:19.917 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:19 smithi026 ceph-mon[31172]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-09-11T23:13:19.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:19 smithi136 ceph-mon[35121]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-09-11T23:13:20.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:19 smithi076 ceph-mon[35296]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-09-11T23:13:20.874 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:20 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:13:20.874 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:20 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:13:20.874 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:20 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:13:20.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:20 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:13:20.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:20 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:13:20.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:20 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:13:21.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:20 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:13:21.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:20 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:13:21.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:20 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:13:21.914 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:21 smithi026 ceph-mon[31172]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-11T23:13:21.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:21 smithi136 ceph-mon[35121]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-11T23:13:22.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:21 smithi076 ceph-mon[35296]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-11T23:13:23.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:13:23.530+0000 7fae5d7fa640 1 -- 172.21.15.26:0/4187294383 <== mgr.14148 v2:172.21.15.26:6800/110833859 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fae60062630 con 0x7fae480a2bb0 2024-09-11T23:13:23.530 INFO:teuthology.orchestra.run.smithi026.stdout:Created osd(s) 6 on host 'smithi026' 2024-09-11T23:13:23.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:13:23.532+0000 7fae664d5640 1 -- 172.21.15.26:0/4187294383 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fae480a2bb0 msgr2=0x7fae480a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:13:23.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:13:23.532+0000 7fae664d5640 1 --2- 172.21.15.26:0/4187294383 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fae480a2bb0 0x7fae480a5070 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fae50000c00 tx=0x7fae50031400 comp rx=0 tx=0).stop 2024-09-11T23:13:23.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:13:23.532+0000 7fae664d5640 1 -- 172.21.15.26:0/4187294383 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae60104f90 msgr2=0x7fae60198400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:13:23.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:13:23.532+0000 7fae664d5640 1 --2- 172.21.15.26:0/4187294383 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae60104f90 0x7fae60198400 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fae5400c970 tx=0x7fae5400ce30 comp rx=0 tx=0).stop 2024-09-11T23:13:23.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:13:23.532+0000 7fae664d5640 1 -- 172.21.15.26:0/4187294383 shutdown_connections 2024-09-11T23:13:23.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:13:23.532+0000 7fae664d5640 1 --2- 172.21.15.26:0/4187294383 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fae480a2bb0 0x7fae480a5070 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:13:23.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:13:23.532+0000 7fae664d5640 1 --2- 172.21.15.26:0/4187294383 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fae601078c0 0x7fae6019f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:13:23.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:13:23.532+0000 7fae664d5640 1 --2- 172.21.15.26:0/4187294383 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae60104f90 0x7fae60198400 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:13:23.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:13:23.532+0000 7fae664d5640 1 --2- 172.21.15.26:0/4187294383 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae60102660 0x7fae60197ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:13:23.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:13:23.532+0000 7fae664d5640 1 -- 172.21.15.26:0/4187294383 >> 172.21.15.26:0/4187294383 conn(0x7fae600fc280 msgr2=0x7fae60102fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:13:23.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:13:23.532+0000 7fae664d5640 1 -- 172.21.15.26:0/4187294383 shutdown_connections 2024-09-11T23:13:23.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:13:23.532+0000 7fae664d5640 1 -- 172.21.15.26:0/4187294383 wait complete. 2024-09-11T23:13:24.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:23 smithi026 ceph-mon[31172]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-09-11T23:13:24.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:23 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:13:24.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:23 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:13:24.186 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:23 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:13:24.186 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:23 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:13:24.214 DEBUG:teuthology.orchestra.run.smithi026:osd.6> sudo journalctl -f -n 0 -u ceph-af7e6fc8-7092-11ef-bcea-c7b262605968@osd.6.service 2024-09-11T23:13:24.216 INFO:tasks.cephadm:Deploying osd.7 on smithi076 with /dev/vg_nvme/lv_2... 2024-09-11T23:13:24.217 DEBUG:teuthology.orchestra.run.smithi076:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-11T23:13:24.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:23 smithi136 ceph-mon[35121]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-09-11T23:13:24.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:23 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:13:24.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:23 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:13:24.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:23 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:13:24.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:23 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:13:24.244 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:23 smithi076 ceph-mon[35296]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-09-11T23:13:24.244 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:23 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:13:24.244 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:23 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:13:24.244 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:23 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:13:24.244 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:23 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:13:24.466 INFO:teuthology.orchestra.run.smithi076.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T23:13:26.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:25 smithi076 ceph-mon[35296]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-09-11T23:13:26.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:25 smithi136 ceph-mon[35121]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-09-11T23:13:26.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:25 smithi026 ceph-mon[31172]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-09-11T23:13:26.716 INFO:journalctl@ceph.osd.6.smithi026.stdout:Sep 11 23:13:26 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-6[58789]: 2024-09-11T23:13:26.523+0000 7f832c0a3640 -1 osd.6 0 log_to_monitors true 2024-09-11T23:13:26.852 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-11T23:13:26.878 DEBUG:teuthology.orchestra.run.smithi076:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi076:vg_nvme/lv_2 2024-09-11T23:13:27.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:26 smithi026 ceph-mon[31172]: from='osd.6 [v2:172.21.15.26:6818/1928245624,v1:172.21.15.26:6819/1928245624]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-11T23:13:27.167 INFO:teuthology.orchestra.run.smithi076.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T23:13:27.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:26 smithi136 ceph-mon[35121]: from='osd.6 [v2:172.21.15.26:6818/1928245624,v1:172.21.15.26:6819/1928245624]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-11T23:13:27.288 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:26 smithi076 ceph-mon[35296]: from='osd.6 [v2:172.21.15.26:6818/1928245624,v1:172.21.15.26:6819/1928245624]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-11T23:13:27.711 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:13:27.695+0000 7f543ce11640 1 -- 172.21.15.76:0/1503495889 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5438102690 msgr2=0x7f5438104a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:13:27.711 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:13:27.695+0000 7f543ce11640 1 --2- 172.21.15.76:0/1503495889 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5438102690 0x7f5438104a80 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f5428009900 tx=0x7f542802a410 comp rx=0 tx=0).stop 2024-09-11T23:13:27.711 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:13:27.695+0000 7f543ce11640 1 -- 172.21.15.76:0/1503495889 shutdown_connections 2024-09-11T23:13:27.711 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:13:27.695+0000 7f543ce11640 1 --2- 172.21.15.76:0/1503495889 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f54381078f0 0x7f5438109d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:13:27.711 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:13:27.695+0000 7f543ce11640 1 --2- 172.21.15.76:0/1503495889 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5438104fc0 0x7f54381073b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:13:27.712 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:13:27.695+0000 7f543ce11640 1 --2- 172.21.15.76:0/1503495889 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5438102690 0x7f5438104a80 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:13:27.712 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:13:27.695+0000 7f543ce11640 1 -- 172.21.15.76:0/1503495889 >> 172.21.15.76:0/1503495889 conn(0x7f54380fc290 msgr2=0x7f54380fe6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:13:27.712 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:13:27.695+0000 7f543ce11640 1 -- 172.21.15.76:0/1503495889 shutdown_connections 2024-09-11T23:13:27.712 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:13:27.695+0000 7f543ce11640 1 -- 172.21.15.76:0/1503495889 wait complete. 2024-09-11T23:13:27.712 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:13:27.696+0000 7f543ce11640 1 Processor -- start 2024-09-11T23:13:27.712 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:13:27.696+0000 7f543ce11640 1 -- start start 2024-09-11T23:13:27.712 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:13:27.696+0000 7f543ce11640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5438102690 0x7f54381980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:13:27.712 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:13:27.696+0000 7f543ce11640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5438104fc0 0x7f5438198600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:13:27.712 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:13:27.696+0000 7f543ce11640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f54381078f0 0x7f543819f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:13:27.712 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:13:27.696+0000 7f543ce11640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f543810d980 con 0x7f54381078f0 2024-09-11T23:13:27.712 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:13:27.696+0000 7f543ce11640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f543810d800 con 0x7f5438104fc0 2024-09-11T23:13:27.712 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:13:27.696+0000 7f543ce11640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f543810db00 con 0x7f5438102690 2024-09-11T23:13:27.713 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:13:27.696+0000 7f5435d9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5438104fc0 0x7f5438198600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:13:27.713 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:13:27.696+0000 7f5435d9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5438104fc0 0x7f5438198600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:53100/0 (socket says 172.21.15.76:53100) 2024-09-11T23:13:27.713 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:13:27.696+0000 7f5435d9b640 1 -- 172.21.15.76:0/2299310085 learned_addr learned my addr 172.21.15.76:0/2299310085 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-11T23:13:27.713 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:13:27.696+0000 7f543659c640 1 --2- 172.21.15.76:0/2299310085 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5438102690 0x7f54381980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:13:27.713 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:13:27.696+0000 7f5436d9d640 1 --2- 172.21.15.76:0/2299310085 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f54381078f0 0x7f543819f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:13:27.713 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:13:27.696+0000 7f5435d9b640 1 -- 172.21.15.76:0/2299310085 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5438102690 msgr2=0x7f54381980c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:13:27.713 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:13:27.696+0000 7f5435d9b640 1 --2- 172.21.15.76:0/2299310085 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5438102690 0x7f54381980c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:13:27.713 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:13:27.696+0000 7f5435d9b640 1 -- 172.21.15.76:0/2299310085 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f54381078f0 msgr2=0x7f543819f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:13:27.713 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:13:27.696+0000 7f5435d9b640 1 --2- 172.21.15.76:0/2299310085 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f54381078f0 0x7f543819f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:13:27.713 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:13:27.696+0000 7f5435d9b640 1 -- 172.21.15.76:0/2299310085 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f543819fcf0 con 0x7f5438104fc0 2024-09-11T23:13:27.713 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:13:27.697+0000 7f5436d9d640 1 --2- 172.21.15.76:0/2299310085 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f54381078f0 0x7f543819f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:13:27.713 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:13:27.697+0000 7f5435d9b640 1 --2- 172.21.15.76:0/2299310085 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5438104fc0 0x7f5438198600 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f542c00b450 tx=0x7f542c00b910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:13:27.714 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:13:27.697+0000 7f54277fe640 1 -- 172.21.15.76:0/2299310085 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f542c015020 con 0x7f5438104fc0 2024-09-11T23:13:27.714 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:13:27.697+0000 7f54277fe640 1 -- 172.21.15.76:0/2299310085 <== mon.2 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f542c011d70 con 0x7f5438104fc0 2024-09-11T23:13:27.714 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:13:27.697+0000 7f543ce11640 1 -- 172.21.15.76:0/2299310085 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f543819ffe0 con 0x7f5438104fc0 2024-09-11T23:13:27.714 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:13:27.697+0000 7f54277fe640 1 -- 172.21.15.76:0/2299310085 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f542c00f340 con 0x7f5438104fc0 2024-09-11T23:13:27.714 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:13:27.697+0000 7f543ce11640 1 -- 172.21.15.76:0/2299310085 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54381a04c0 con 0x7f5438104fc0 2024-09-11T23:13:27.714 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:13:27.698+0000 7f543ce11640 1 -- 172.21.15.76:0/2299310085 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54380fdd80 con 0x7f5438104fc0 2024-09-11T23:13:27.714 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:13:27.699+0000 7f54277fe640 1 -- 172.21.15.76:0/2299310085 <== mon.2 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f542c018020 con 0x7f5438104fc0 2024-09-11T23:13:27.714 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:13:27.700+0000 7f54277fe640 1 --2- 172.21.15.76:0/2299310085 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f54200a2bb0 0x7f54200a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:13:27.714 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:13:27.700+0000 7f54277fe640 1 -- 172.21.15.76:0/2299310085 <== mon.2 v2:172.21.15.76:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 4669+0+0 (secure 0 0 0) 0x7f542c0ce1e0 con 0x7f5438104fc0 2024-09-11T23:13:27.714 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:13:27.700+0000 7f543659c640 1 --2- 172.21.15.76:0/2299310085 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f54200a2bb0 0x7f54200a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:13:27.714 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:13:27.700+0000 7f543659c640 1 --2- 172.21.15.76:0/2299310085 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f54200a2bb0 0x7f54200a5070 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f5428000c00 tx=0x7f5428031400 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:13:27.714 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:13:27.703+0000 7f54277fe640 1 -- 172.21.15.76:0/2299310085 <== mon.2 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f542c01f020 con 0x7f5438104fc0 2024-09-11T23:13:27.879 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:13:27.879+0000 7f543ce11640 1 -- 172.21.15.76:0/2299310085 --> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi076:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f5438062630 con 0x7f54200a2bb0 2024-09-11T23:13:28.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:27 smithi136 ceph-mon[35121]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-09-11T23:13:28.240 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:27 smithi136 ceph-mon[35121]: from='osd.6 [v2:172.21.15.26:6818/1928245624,v1:172.21.15.26:6819/1928245624]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-11T23:13:28.240 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:27 smithi136 ceph-mon[35121]: osdmap e43: 7 total, 6 up, 7 in 2024-09-11T23:13:28.240 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:27 smithi136 ceph-mon[35121]: from='osd.6 [v2:172.21.15.26:6818/1928245624,v1:172.21.15.26:6819/1928245624]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2024-09-11T23:13:28.240 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:27 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T23:13:28.240 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:27 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:13:28.240 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:27 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:13:28.240 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:27 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:13:28.240 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:27 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:13:28.240 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:27 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:13:28.240 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:27 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:13:28.240 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:27 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:13:28.240 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:27 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:13:28.241 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:27 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:13:28.241 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:27 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:13:28.241 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:27 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:13:28.241 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:27 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:13:28.241 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:27 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:13:28.241 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:27 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T23:13:28.241 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:27 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T23:13:28.241 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:27 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:13:28.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:27 smithi026 ceph-mon[31172]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-09-11T23:13:28.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:27 smithi026 ceph-mon[31172]: from='osd.6 [v2:172.21.15.26:6818/1928245624,v1:172.21.15.26:6819/1928245624]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-11T23:13:28.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:27 smithi026 ceph-mon[31172]: osdmap e43: 7 total, 6 up, 7 in 2024-09-11T23:13:28.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:27 smithi026 ceph-mon[31172]: from='osd.6 [v2:172.21.15.26:6818/1928245624,v1:172.21.15.26:6819/1928245624]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2024-09-11T23:13:28.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:27 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T23:13:28.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:27 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:13:28.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:27 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:13:28.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:27 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:13:28.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:27 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:13:28.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:27 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:13:28.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:27 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:13:28.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:27 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:13:28.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:27 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:13:28.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:27 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:13:28.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:27 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:13:28.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:27 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:13:28.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:27 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:13:28.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:27 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:13:28.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:27 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T23:13:28.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:27 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T23:13:28.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:27 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:13:28.287 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:27 smithi076 ceph-mon[35296]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-09-11T23:13:28.287 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:27 smithi076 ceph-mon[35296]: from='osd.6 [v2:172.21.15.26:6818/1928245624,v1:172.21.15.26:6819/1928245624]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-11T23:13:28.287 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:27 smithi076 ceph-mon[35296]: osdmap e43: 7 total, 6 up, 7 in 2024-09-11T23:13:28.288 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:27 smithi076 ceph-mon[35296]: from='osd.6 [v2:172.21.15.26:6818/1928245624,v1:172.21.15.26:6819/1928245624]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2024-09-11T23:13:28.288 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:27 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T23:13:28.288 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:27 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:13:28.288 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:27 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:13:28.288 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:27 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:13:28.288 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:27 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:13:28.288 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:27 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:13:28.288 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:27 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:13:28.288 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:27 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:13:28.288 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:27 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:13:28.288 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:27 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:13:28.288 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:27 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:13:28.288 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:27 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:13:28.289 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:27 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:13:28.289 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:27 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:13:28.289 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:27 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T23:13:28.289 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:27 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T23:13:28.289 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:27 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:13:28.950 INFO:journalctl@ceph.osd.6.smithi026.stdout:Sep 11 23:13:28 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-6[58789]: 2024-09-11T23:13:28.623+0000 7f8327803640 -1 osd.6 0 waiting for initial osdmap 2024-09-11T23:13:28.950 INFO:journalctl@ceph.osd.6.smithi026.stdout:Sep 11 23:13:28 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-6[58789]: 2024-09-11T23:13:28.633+0000 7f831fe35640 -1 osd.6 44 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T23:13:29.210 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:28 smithi076 ceph-mon[35296]: Detected new or changed devices on smithi026 2024-09-11T23:13:29.211 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:28 smithi076 ceph-mon[35296]: Adjusting osd_memory_target on smithi026 to 5697M 2024-09-11T23:13:29.211 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:28 smithi076 ceph-mon[35296]: from='client.24278 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi076:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:13:29.211 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:28 smithi076 ceph-mon[35296]: from='osd.6 [v2:172.21.15.26:6818/1928245624,v1:172.21.15.26:6819/1928245624]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]': finished 2024-09-11T23:13:29.211 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:28 smithi076 ceph-mon[35296]: osdmap e44: 7 total, 6 up, 7 in 2024-09-11T23:13:29.211 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:28 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T23:13:29.211 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:28 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T23:13:29.211 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:28 smithi076 ceph-mon[35296]: from='osd.6 [v2:172.21.15.26:6818/1928245624,v1:172.21.15.26:6819/1928245624]' entity='osd.6' 2024-09-11T23:13:29.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:28 smithi136 ceph-mon[35121]: Detected new or changed devices on smithi026 2024-09-11T23:13:29.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:28 smithi136 ceph-mon[35121]: Adjusting osd_memory_target on smithi026 to 5697M 2024-09-11T23:13:29.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:28 smithi136 ceph-mon[35121]: from='client.24278 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi076:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:13:29.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:28 smithi136 ceph-mon[35121]: from='osd.6 [v2:172.21.15.26:6818/1928245624,v1:172.21.15.26:6819/1928245624]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]': finished 2024-09-11T23:13:29.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:28 smithi136 ceph-mon[35121]: osdmap e44: 7 total, 6 up, 7 in 2024-09-11T23:13:29.240 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:28 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T23:13:29.240 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:28 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T23:13:29.240 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:28 smithi136 ceph-mon[35121]: from='osd.6 [v2:172.21.15.26:6818/1928245624,v1:172.21.15.26:6819/1928245624]' entity='osd.6' 2024-09-11T23:13:29.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:28 smithi026 ceph-mon[31172]: Detected new or changed devices on smithi026 2024-09-11T23:13:29.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:28 smithi026 ceph-mon[31172]: Adjusting osd_memory_target on smithi026 to 5697M 2024-09-11T23:13:29.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:28 smithi026 ceph-mon[31172]: from='client.24278 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi076:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:13:29.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:28 smithi026 ceph-mon[31172]: from='osd.6 [v2:172.21.15.26:6818/1928245624,v1:172.21.15.26:6819/1928245624]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]': finished 2024-09-11T23:13:29.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:28 smithi026 ceph-mon[31172]: osdmap e44: 7 total, 6 up, 7 in 2024-09-11T23:13:29.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:28 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T23:13:29.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:28 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T23:13:29.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:28 smithi026 ceph-mon[31172]: from='osd.6 [v2:172.21.15.26:6818/1928245624,v1:172.21.15.26:6819/1928245624]' entity='osd.6' 2024-09-11T23:13:30.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:29 smithi136 ceph-mon[35121]: purged_snaps scrub starts 2024-09-11T23:13:30.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:29 smithi136 ceph-mon[35121]: purged_snaps scrub ok 2024-09-11T23:13:30.240 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:29 smithi136 ceph-mon[35121]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-09-11T23:13:30.240 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:29 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T23:13:30.240 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:29 smithi136 ceph-mon[35121]: osd.6 [v2:172.21.15.26:6818/1928245624,v1:172.21.15.26:6819/1928245624] boot 2024-09-11T23:13:30.240 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:29 smithi136 ceph-mon[35121]: osdmap e45: 7 total, 7 up, 7 in 2024-09-11T23:13:30.240 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:29 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T23:13:30.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:29 smithi026 ceph-mon[31172]: purged_snaps scrub starts 2024-09-11T23:13:30.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:29 smithi026 ceph-mon[31172]: purged_snaps scrub ok 2024-09-11T23:13:30.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:29 smithi026 ceph-mon[31172]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-09-11T23:13:30.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:29 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T23:13:30.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:29 smithi026 ceph-mon[31172]: osd.6 [v2:172.21.15.26:6818/1928245624,v1:172.21.15.26:6819/1928245624] boot 2024-09-11T23:13:30.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:29 smithi026 ceph-mon[31172]: osdmap e45: 7 total, 7 up, 7 in 2024-09-11T23:13:30.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:29 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T23:13:30.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:29 smithi076 ceph-mon[35296]: purged_snaps scrub starts 2024-09-11T23:13:30.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:29 smithi076 ceph-mon[35296]: purged_snaps scrub ok 2024-09-11T23:13:30.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:29 smithi076 ceph-mon[35296]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-09-11T23:13:30.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:29 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T23:13:30.304 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:29 smithi076 ceph-mon[35296]: osd.6 [v2:172.21.15.26:6818/1928245624,v1:172.21.15.26:6819/1928245624] boot 2024-09-11T23:13:30.304 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:29 smithi076 ceph-mon[35296]: osdmap e45: 7 total, 7 up, 7 in 2024-09-11T23:13:30.304 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:29 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T23:13:31.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:30 smithi136 ceph-mon[35121]: from='client.? 172.21.15.76:0/2659694963' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fe031f5c-d915-48fb-91a6-c76f252467fb"}]: dispatch 2024-09-11T23:13:31.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:30 smithi136 ceph-mon[35121]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fe031f5c-d915-48fb-91a6-c76f252467fb"}]: dispatch 2024-09-11T23:13:31.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:30 smithi136 ceph-mon[35121]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fe031f5c-d915-48fb-91a6-c76f252467fb"}]': finished 2024-09-11T23:13:31.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:30 smithi136 ceph-mon[35121]: osdmap e46: 8 total, 7 up, 8 in 2024-09-11T23:13:31.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:30 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T23:13:31.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:30 smithi136 ceph-mon[35121]: from='client.? 172.21.15.76:0/1010821096' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T23:13:31.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:30 smithi026 ceph-mon[31172]: from='client.? 172.21.15.76:0/2659694963' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fe031f5c-d915-48fb-91a6-c76f252467fb"}]: dispatch 2024-09-11T23:13:31.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:30 smithi026 ceph-mon[31172]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fe031f5c-d915-48fb-91a6-c76f252467fb"}]: dispatch 2024-09-11T23:13:31.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:30 smithi026 ceph-mon[31172]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fe031f5c-d915-48fb-91a6-c76f252467fb"}]': finished 2024-09-11T23:13:31.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:30 smithi026 ceph-mon[31172]: osdmap e46: 8 total, 7 up, 8 in 2024-09-11T23:13:31.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:30 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T23:13:31.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:30 smithi026 ceph-mon[31172]: from='client.? 172.21.15.76:0/1010821096' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T23:13:31.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:30 smithi076 ceph-mon[35296]: from='client.? 172.21.15.76:0/2659694963' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fe031f5c-d915-48fb-91a6-c76f252467fb"}]: dispatch 2024-09-11T23:13:31.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:30 smithi076 ceph-mon[35296]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fe031f5c-d915-48fb-91a6-c76f252467fb"}]: dispatch 2024-09-11T23:13:31.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:30 smithi076 ceph-mon[35296]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fe031f5c-d915-48fb-91a6-c76f252467fb"}]': finished 2024-09-11T23:13:31.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:30 smithi076 ceph-mon[35296]: osdmap e46: 8 total, 7 up, 8 in 2024-09-11T23:13:31.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:30 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T23:13:31.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:30 smithi076 ceph-mon[35296]: from='client.? 172.21.15.76:0/1010821096' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T23:13:32.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:31 smithi136 ceph-mon[35121]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-09-11T23:13:32.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:31 smithi136 ceph-mon[35121]: osdmap e47: 8 total, 7 up, 8 in 2024-09-11T23:13:32.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:31 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T23:13:32.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:31 smithi026 ceph-mon[31172]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-09-11T23:13:32.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:31 smithi026 ceph-mon[31172]: osdmap e47: 8 total, 7 up, 8 in 2024-09-11T23:13:32.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:31 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T23:13:32.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:31 smithi076 ceph-mon[35296]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-09-11T23:13:32.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:31 smithi076 ceph-mon[35296]: osdmap e47: 8 total, 7 up, 8 in 2024-09-11T23:13:32.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:31 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T23:13:34.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:33 smithi136 ceph-mon[35121]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-09-11T23:13:34.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:33 smithi026 ceph-mon[31172]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-09-11T23:13:34.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:33 smithi076 ceph-mon[35296]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-09-11T23:13:36.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:35 smithi076 ceph-mon[35296]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-09-11T23:13:36.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:35 smithi136 ceph-mon[35121]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-09-11T23:13:36.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:35 smithi026 ceph-mon[31172]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-09-11T23:13:38.143 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:37 smithi076 ceph-mon[35296]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-09-11T23:13:38.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:37 smithi136 ceph-mon[35121]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-09-11T23:13:38.253 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:37 smithi026 ceph-mon[31172]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-09-11T23:13:39.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:38 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-11T23:13:39.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:38 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:13:39.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:38 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-11T23:13:39.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:38 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:13:39.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:38 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-11T23:13:39.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:38 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:13:40.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:39 smithi136 ceph-mon[35121]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-09-11T23:13:40.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:39 smithi136 ceph-mon[35121]: Deploying daemon osd.7 on smithi076 2024-09-11T23:13:40.247 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:39 smithi076 ceph-mon[35296]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-09-11T23:13:40.248 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:39 smithi076 ceph-mon[35296]: Deploying daemon osd.7 on smithi076 2024-09-11T23:13:40.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:39 smithi026 ceph-mon[31172]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-09-11T23:13:40.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:39 smithi026 ceph-mon[31172]: Deploying daemon osd.7 on smithi076 2024-09-11T23:13:41.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:40 smithi136 ceph-mon[35121]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-09-11T23:13:41.249 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:40 smithi076 ceph-mon[35296]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-09-11T23:13:41.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:40 smithi026 ceph-mon[31172]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-09-11T23:13:43.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:43 smithi136 ceph-mon[35121]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-09-11T23:13:43.501 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:43 smithi026 ceph-mon[31172]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-09-11T23:13:43.553 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:43 smithi076 ceph-mon[35296]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-09-11T23:13:45.366 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:45 smithi076 ceph-mon[35296]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-09-11T23:13:45.366 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:45 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:13:45.366 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:45 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:13:45.366 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:45 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:13:45.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:45 smithi136 ceph-mon[35121]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-09-11T23:13:45.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:45 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:13:45.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:45 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:13:45.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:45 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:13:45.501 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:45 smithi026 ceph-mon[31172]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-09-11T23:13:45.501 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:45 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:13:45.501 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:45 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:13:45.501 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:45 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:13:47.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:47 smithi076 ceph-mon[35296]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-09-11T23:13:47.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:47 smithi136 ceph-mon[35121]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-09-11T23:13:47.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:47 smithi026 ceph-mon[31172]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-09-11T23:13:48.718 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:13:48.717+0000 7f54277fe640 1 -- 172.21.15.76:0/2299310085 <== mgr.14148 v2:172.21.15.26:6800/110833859 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f5438062630 con 0x7f54200a2bb0 2024-09-11T23:13:48.718 INFO:teuthology.orchestra.run.smithi076.stdout:Created osd(s) 7 on host 'smithi076' 2024-09-11T23:13:48.720 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:13:48.719+0000 7f543ce11640 1 -- 172.21.15.76:0/2299310085 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f54200a2bb0 msgr2=0x7f54200a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:13:48.720 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:13:48.719+0000 7f543ce11640 1 --2- 172.21.15.76:0/2299310085 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f54200a2bb0 0x7f54200a5070 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f5428000c00 tx=0x7f5428031400 comp rx=0 tx=0).stop 2024-09-11T23:13:48.720 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:13:48.719+0000 7f543ce11640 1 -- 172.21.15.76:0/2299310085 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5438104fc0 msgr2=0x7f5438198600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:13:48.720 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:13:48.719+0000 7f543ce11640 1 --2- 172.21.15.76:0/2299310085 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5438104fc0 0x7f5438198600 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f542c00b450 tx=0x7f542c00b910 comp rx=0 tx=0).stop 2024-09-11T23:13:48.720 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:13:48.719+0000 7f543ce11640 1 -- 172.21.15.76:0/2299310085 shutdown_connections 2024-09-11T23:13:48.720 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:13:48.719+0000 7f543ce11640 1 --2- 172.21.15.76:0/2299310085 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f54200a2bb0 0x7f54200a5070 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:13:48.720 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:13:48.719+0000 7f543ce11640 1 --2- 172.21.15.76:0/2299310085 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5438104fc0 0x7f5438198600 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:13:48.720 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:13:48.719+0000 7f543ce11640 1 --2- 172.21.15.76:0/2299310085 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f54381078f0 0x7f543819f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:13:48.720 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:13:48.719+0000 7f543ce11640 1 --2- 172.21.15.76:0/2299310085 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5438102690 0x7f54381980c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:13:48.721 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:13:48.719+0000 7f543ce11640 1 -- 172.21.15.76:0/2299310085 >> 172.21.15.76:0/2299310085 conn(0x7f54380fc290 msgr2=0x7f5438103110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:13:48.721 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:13:48.719+0000 7f543ce11640 1 -- 172.21.15.76:0/2299310085 shutdown_connections 2024-09-11T23:13:48.721 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:13:48.720+0000 7f543ce11640 1 -- 172.21.15.76:0/2299310085 wait complete. 2024-09-11T23:13:49.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:48 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:13:49.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:48 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:13:49.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:48 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:13:49.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:48 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:13:49.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:48 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:13:49.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:48 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:13:49.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:48 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:13:49.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:48 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:13:49.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:48 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:13:49.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:48 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:13:49.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:48 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:13:49.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:48 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:13:49.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:48 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:13:49.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:48 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:13:49.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:48 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:13:49.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:48 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:13:49.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:48 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:13:49.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:48 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:13:49.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:48 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:13:49.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:48 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:13:49.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:48 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:13:49.240 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:48 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:13:49.240 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:48 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:13:49.240 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:48 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:13:49.418 DEBUG:teuthology.orchestra.run.smithi076:osd.7> sudo journalctl -f -n 0 -u ceph-af7e6fc8-7092-11ef-bcea-c7b262605968@osd.7.service 2024-09-11T23:13:49.421 INFO:tasks.cephadm:Deploying osd.8 on smithi136 with /dev/vg_nvme/lv_2... 2024-09-11T23:13:49.424 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-11T23:13:49.681 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T23:13:50.014 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:49 smithi136 ceph-mon[35121]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-09-11T23:13:50.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:49 smithi076 ceph-mon[35296]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-09-11T23:13:50.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:49 smithi026 ceph-mon[31172]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-09-11T23:13:51.303 INFO:journalctl@ceph.osd.7.smithi076.stdout:Sep 11 23:13:50 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-7[52804]: 2024-09-11T23:13:50.949+0000 7f2903699640 -1 osd.7 0 log_to_monitors true 2024-09-11T23:13:52.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:51 smithi076 ceph-mon[35296]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-09-11T23:13:52.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:51 smithi076 ceph-mon[35296]: from='osd.7 [v2:172.21.15.76:6816/1409789455,v1:172.21.15.76:6817/1409789455]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-11T23:13:52.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:51 smithi076 ceph-mon[35296]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-11T23:13:52.134 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:51 smithi136 ceph-mon[35121]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-09-11T23:13:52.134 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:51 smithi136 ceph-mon[35121]: from='osd.7 [v2:172.21.15.76:6816/1409789455,v1:172.21.15.76:6817/1409789455]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-11T23:13:52.134 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:51 smithi136 ceph-mon[35121]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-11T23:13:52.137 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-09-11T23:13:52.164 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi136:vg_nvme/lv_2 2024-09-11T23:13:52.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:51 smithi026 ceph-mon[31172]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-09-11T23:13:52.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:51 smithi026 ceph-mon[31172]: from='osd.7 [v2:172.21.15.76:6816/1409789455,v1:172.21.15.76:6817/1409789455]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-11T23:13:52.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:51 smithi026 ceph-mon[31172]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-11T23:13:52.416 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T23:13:52.897 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:13:52.894+0000 7f4bc1c0f640 1 -- 172.21.15.136:0/3248904852 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4bbc107920 msgr2=0x7f4bbc109d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:13:52.897 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:13:52.894+0000 7f4bc1c0f640 1 --2- 172.21.15.136:0/3248904852 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4bbc107920 0x7f4bbc109d10 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f4bb4009ca0 tx=0x7f4bb402a8c0 comp rx=0 tx=0).stop 2024-09-11T23:13:52.897 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:13:52.895+0000 7f4bc1c0f640 1 -- 172.21.15.136:0/3248904852 shutdown_connections 2024-09-11T23:13:52.897 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:13:52.895+0000 7f4bc1c0f640 1 --2- 172.21.15.136:0/3248904852 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4bbc104ff0 0x7f4bbc1073e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:13:52.898 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:13:52.895+0000 7f4bc1c0f640 1 --2- 172.21.15.136:0/3248904852 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4bbc107920 0x7f4bbc109d10 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:13:52.898 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:13:52.895+0000 7f4bc1c0f640 1 --2- 172.21.15.136:0/3248904852 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4bbc1026c0 0x7f4bbc104ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:13:52.898 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:13:52.895+0000 7f4bc1c0f640 1 -- 172.21.15.136:0/3248904852 >> 172.21.15.136:0/3248904852 conn(0x7f4bbc0fc280 msgr2=0x7f4bbc0fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:13:52.898 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:13:52.895+0000 7f4bc1c0f640 1 -- 172.21.15.136:0/3248904852 shutdown_connections 2024-09-11T23:13:52.898 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:13:52.895+0000 7f4bc1c0f640 1 -- 172.21.15.136:0/3248904852 wait complete. 2024-09-11T23:13:52.898 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:13:52.895+0000 7f4bc1c0f640 1 Processor -- start 2024-09-11T23:13:52.898 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:13:52.896+0000 7f4bc1c0f640 1 -- start start 2024-09-11T23:13:52.898 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:13:52.896+0000 7f4bc1c0f640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4bbc1026c0 0x7f4bbc197f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:13:52.898 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:13:52.896+0000 7f4bc1c0f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4bbc104ff0 0x7f4bbc198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:13:52.898 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:13:52.896+0000 7f4bc1c0f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4bbc107920 0x7f4bbc19f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:13:52.898 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:13:52.896+0000 7f4bc1c0f640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f4bbc10d8d0 con 0x7f4bbc107920 2024-09-11T23:13:52.898 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:13:52.896+0000 7f4bc1c0f640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f4bbc10d750 con 0x7f4bbc104ff0 2024-09-11T23:13:52.898 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:13:52.896+0000 7f4bc1c0f640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f4bbc10da50 con 0x7f4bbc1026c0 2024-09-11T23:13:52.899 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:13:52.896+0000 7f4bbb7fe640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4bbc1026c0 0x7f4bbc197f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:13:52.899 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:13:52.896+0000 7f4bbb7fe640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4bbc1026c0 0x7f4bbc197f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.136:48086/0 (socket says 172.21.15.136:48086) 2024-09-11T23:13:52.899 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:13:52.896+0000 7f4bbb7fe640 1 -- 172.21.15.136:0/878756489 learned_addr learned my addr 172.21.15.136:0/878756489 (peer_addr_for_me v2:172.21.15.136:0/0) 2024-09-11T23:13:52.899 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:13:52.896+0000 7f4bbbfff640 1 --2- 172.21.15.136:0/878756489 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4bbc107920 0x7f4bbc19f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:13:52.899 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:13:52.896+0000 7f4bbaffd640 1 --2- 172.21.15.136:0/878756489 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4bbc104ff0 0x7f4bbc198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:13:52.899 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:13:52.896+0000 7f4bbb7fe640 1 -- 172.21.15.136:0/878756489 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4bbc104ff0 msgr2=0x7f4bbc198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:13:52.899 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:13:52.896+0000 7f4bbb7fe640 1 --2- 172.21.15.136:0/878756489 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4bbc104ff0 0x7f4bbc198470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:13:52.899 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:13:52.896+0000 7f4bbb7fe640 1 -- 172.21.15.136:0/878756489 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4bbc107920 msgr2=0x7f4bbc19f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:13:52.899 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:13:52.896+0000 7f4bbb7fe640 1 --2- 172.21.15.136:0/878756489 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4bbc107920 0x7f4bbc19f4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:13:52.899 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:13:52.896+0000 7f4bbb7fe640 1 -- 172.21.15.136:0/878756489 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bbc19fb60 con 0x7f4bbc1026c0 2024-09-11T23:13:52.899 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:13:52.896+0000 7f4bbbfff640 1 --2- 172.21.15.136:0/878756489 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4bbc107920 0x7f4bbc19f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:13:52.899 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:13:52.896+0000 7f4bbb7fe640 1 --2- 172.21.15.136:0/878756489 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4bbc1026c0 0x7f4bbc197f30 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f4bac00b4a0 tx=0x7f4bac00b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:13:52.899 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:13:52.897+0000 7f4bb8ff9640 1 -- 172.21.15.136:0/878756489 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bac014070 con 0x7f4bbc1026c0 2024-09-11T23:13:52.899 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:13:52.897+0000 7f4bb8ff9640 1 -- 172.21.15.136:0/878756489 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4bac010af0 con 0x7f4bbc1026c0 2024-09-11T23:13:52.900 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:13:52.897+0000 7f4bc1c0f640 1 -- 172.21.15.136:0/878756489 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4bbc19fe50 con 0x7f4bbc1026c0 2024-09-11T23:13:52.900 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:13:52.897+0000 7f4bb8ff9640 1 -- 172.21.15.136:0/878756489 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bac017d10 con 0x7f4bbc1026c0 2024-09-11T23:13:52.900 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:13:52.897+0000 7f4bc1c0f640 1 -- 172.21.15.136:0/878756489 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4bbc1a0380 con 0x7f4bbc1026c0 2024-09-11T23:13:52.900 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:13:52.898+0000 7f4bc1c0f640 1 -- 172.21.15.136:0/878756489 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4bbc0fdd70 con 0x7f4bbc1026c0 2024-09-11T23:13:52.901 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:13:52.899+0000 7f4bb8ff9640 1 -- 172.21.15.136:0/878756489 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f4bac017070 con 0x7f4bbc1026c0 2024-09-11T23:13:52.901 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:13:52.900+0000 7f4bb8ff9640 1 --2- 172.21.15.136:0/878756489 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f4ba40a2c30 0x7f4ba40a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:13:52.901 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:13:52.900+0000 7f4bb8ff9640 1 -- 172.21.15.136:0/878756489 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5117+0+0 (secure 0 0 0) 0x7f4bac0cdbb0 con 0x7f4bbc1026c0 2024-09-11T23:13:52.901 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:13:52.900+0000 7f4bbaffd640 1 --2- 172.21.15.136:0/878756489 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f4ba40a2c30 0x7f4ba40a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:13:52.901 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:13:52.900+0000 7f4bbaffd640 1 --2- 172.21.15.136:0/878756489 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f4ba40a2c30 0x7f4ba40a50f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f4bb0005fd0 tx=0x7f4bb0005850 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:13:52.903 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:13:52.903+0000 7f4bb8ff9640 1 -- 172.21.15.136:0/878756489 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4bac09b6f0 con 0x7f4bbc1026c0 2024-09-11T23:13:53.053 INFO:journalctl@ceph.osd.7.smithi076.stdout:Sep 11 23:13:52 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-7[52804]: 2024-09-11T23:13:52.771+0000 7f28fedf9640 -1 osd.7 0 waiting for initial osdmap 2024-09-11T23:13:53.053 INFO:journalctl@ceph.osd.7.smithi076.stdout:Sep 11 23:13:52 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-7[52804]: 2024-09-11T23:13:52.783+0000 7f28fb433640 -1 osd.7 49 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T23:13:53.054 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:52 smithi076 ceph-mon[35296]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-11T23:13:53.054 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:52 smithi076 ceph-mon[35296]: from='osd.7 [v2:172.21.15.76:6816/1409789455,v1:172.21.15.76:6817/1409789455]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]: dispatch 2024-09-11T23:13:53.054 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:52 smithi076 ceph-mon[35296]: osdmap e48: 8 total, 7 up, 8 in 2024-09-11T23:13:53.054 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:52 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T23:13:53.054 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:52 smithi076 ceph-mon[35296]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]: dispatch 2024-09-11T23:13:53.054 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:52 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:13:53.054 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:52 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:13:53.054 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:52 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:13:53.054 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:52 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:13:53.054 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:52 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:13:53.054 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:52 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:13:53.055 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:52 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:13:53.055 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:52 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:13:53.055 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:52 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:13:53.085 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:13:53.085+0000 7f4bc1c0f640 1 -- 172.21.15.136:0/878756489 --> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi136:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f4bbc062630 con 0x7f4ba40a2c30 2024-09-11T23:13:53.086 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:52 smithi136 ceph-mon[35121]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-11T23:13:53.086 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:52 smithi136 ceph-mon[35121]: from='osd.7 [v2:172.21.15.76:6816/1409789455,v1:172.21.15.76:6817/1409789455]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]: dispatch 2024-09-11T23:13:53.086 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:52 smithi136 ceph-mon[35121]: osdmap e48: 8 total, 7 up, 8 in 2024-09-11T23:13:53.086 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:52 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T23:13:53.086 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:52 smithi136 ceph-mon[35121]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]: dispatch 2024-09-11T23:13:53.086 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:52 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:13:53.086 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:52 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:13:53.086 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:52 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:13:53.086 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:52 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:13:53.086 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:52 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:13:53.086 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:52 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:13:53.087 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:52 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:13:53.087 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:52 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:13:53.087 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:52 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:13:53.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:52 smithi026 ceph-mon[31172]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-11T23:13:53.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:52 smithi026 ceph-mon[31172]: from='osd.7 [v2:172.21.15.76:6816/1409789455,v1:172.21.15.76:6817/1409789455]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]: dispatch 2024-09-11T23:13:53.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:52 smithi026 ceph-mon[31172]: osdmap e48: 8 total, 7 up, 8 in 2024-09-11T23:13:53.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:52 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T23:13:53.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:52 smithi026 ceph-mon[31172]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]: dispatch 2024-09-11T23:13:53.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:52 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:13:53.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:52 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:13:53.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:52 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:13:53.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:52 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:13:53.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:52 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:13:53.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:52 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:13:53.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:52 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:13:53.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:52 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:13:53.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:52 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:13:53.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:53 smithi136 ceph-mon[35121]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-09-11T23:13:53.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:53 smithi136 ceph-mon[35121]: Detected new or changed devices on smithi076 2024-09-11T23:13:53.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:53 smithi136 ceph-mon[35121]: Adjusting osd_memory_target on smithi076 to 5696M 2024-09-11T23:13:53.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:53 smithi136 ceph-mon[35121]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]': finished 2024-09-11T23:13:53.990 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:53 smithi136 ceph-mon[35121]: osdmap e49: 8 total, 7 up, 8 in 2024-09-11T23:13:53.990 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:53 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T23:13:53.990 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:53 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T23:13:53.990 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:53 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T23:13:53.990 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:53 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T23:13:53.990 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:53 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:13:54.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:53 smithi076 ceph-mon[35296]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-09-11T23:13:54.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:53 smithi076 ceph-mon[35296]: Detected new or changed devices on smithi076 2024-09-11T23:13:54.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:53 smithi076 ceph-mon[35296]: Adjusting osd_memory_target on smithi076 to 5696M 2024-09-11T23:13:54.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:53 smithi076 ceph-mon[35296]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]': finished 2024-09-11T23:13:54.054 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:53 smithi076 ceph-mon[35296]: osdmap e49: 8 total, 7 up, 8 in 2024-09-11T23:13:54.054 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:53 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T23:13:54.054 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:53 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T23:13:54.054 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:53 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T23:13:54.054 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:53 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T23:13:54.054 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:53 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:13:54.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:53 smithi026 ceph-mon[31172]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-09-11T23:13:54.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:53 smithi026 ceph-mon[31172]: Detected new or changed devices on smithi076 2024-09-11T23:13:54.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:53 smithi026 ceph-mon[31172]: Adjusting osd_memory_target on smithi076 to 5696M 2024-09-11T23:13:54.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:53 smithi026 ceph-mon[31172]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]': finished 2024-09-11T23:13:54.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:53 smithi026 ceph-mon[31172]: osdmap e49: 8 total, 7 up, 8 in 2024-09-11T23:13:54.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:53 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T23:13:54.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:53 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T23:13:54.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:53 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T23:13:54.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:53 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T23:13:54.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:53 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:13:54.990 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:54 smithi136 ceph-mon[35121]: purged_snaps scrub starts 2024-09-11T23:13:54.990 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:54 smithi136 ceph-mon[35121]: purged_snaps scrub ok 2024-09-11T23:13:54.990 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:54 smithi136 ceph-mon[35121]: from='client.24274 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi136:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:13:54.990 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:54 smithi136 ceph-mon[35121]: osd.7 [v2:172.21.15.76:6816/1409789455,v1:172.21.15.76:6817/1409789455] boot 2024-09-11T23:13:54.990 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:54 smithi136 ceph-mon[35121]: osdmap e50: 8 total, 8 up, 8 in 2024-09-11T23:13:54.990 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:54 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T23:13:55.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:54 smithi076 ceph-mon[35296]: purged_snaps scrub starts 2024-09-11T23:13:55.623 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:54 smithi076 ceph-mon[35296]: purged_snaps scrub ok 2024-09-11T23:13:55.624 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:54 smithi076 ceph-mon[35296]: from='client.24274 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi136:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:13:55.624 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:54 smithi076 ceph-mon[35296]: osd.7 [v2:172.21.15.76:6816/1409789455,v1:172.21.15.76:6817/1409789455] boot 2024-09-11T23:13:55.624 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:54 smithi076 ceph-mon[35296]: osdmap e50: 8 total, 8 up, 8 in 2024-09-11T23:13:55.624 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:54 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T23:13:55.625 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:54 smithi026 ceph-mon[31172]: purged_snaps scrub starts 2024-09-11T23:13:55.625 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:54 smithi026 ceph-mon[31172]: purged_snaps scrub ok 2024-09-11T23:13:55.625 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:54 smithi026 ceph-mon[31172]: from='client.24274 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi136:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:13:55.625 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:54 smithi026 ceph-mon[31172]: osd.7 [v2:172.21.15.76:6816/1409789455,v1:172.21.15.76:6817/1409789455] boot 2024-09-11T23:13:55.625 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:54 smithi026 ceph-mon[31172]: osdmap e50: 8 total, 8 up, 8 in 2024-09-11T23:13:55.625 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:54 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T23:13:55.925 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:55 smithi136 ceph-mon[35121]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-11T23:13:55.925 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:55 smithi136 ceph-mon[35121]: osdmap e51: 8 total, 8 up, 8 in 2024-09-11T23:13:55.925 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:55 smithi136 ceph-mon[35121]: from='client.? 172.21.15.136:0/1878574835' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "18851bde-9e6c-4e05-b4bf-ac5c3073eee7"}]: dispatch 2024-09-11T23:13:55.925 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:55 smithi136 ceph-mon[35121]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "18851bde-9e6c-4e05-b4bf-ac5c3073eee7"}]: dispatch 2024-09-11T23:13:55.925 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:55 smithi136 ceph-mon[35121]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "18851bde-9e6c-4e05-b4bf-ac5c3073eee7"}]': finished 2024-09-11T23:13:55.926 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:55 smithi136 ceph-mon[35121]: osdmap e52: 9 total, 8 up, 9 in 2024-09-11T23:13:55.926 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:55 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-11T23:13:55.926 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:55 smithi136 ceph-mon[35121]: osdmap e53: 9 total, 8 up, 9 in 2024-09-11T23:13:55.926 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:55 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-11T23:13:56.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:55 smithi076 ceph-mon[35296]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-11T23:13:56.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:55 smithi076 ceph-mon[35296]: osdmap e51: 8 total, 8 up, 8 in 2024-09-11T23:13:56.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:55 smithi076 ceph-mon[35296]: from='client.? 172.21.15.136:0/1878574835' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "18851bde-9e6c-4e05-b4bf-ac5c3073eee7"}]: dispatch 2024-09-11T23:13:56.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:55 smithi076 ceph-mon[35296]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "18851bde-9e6c-4e05-b4bf-ac5c3073eee7"}]: dispatch 2024-09-11T23:13:56.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:55 smithi076 ceph-mon[35296]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "18851bde-9e6c-4e05-b4bf-ac5c3073eee7"}]': finished 2024-09-11T23:13:56.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:55 smithi076 ceph-mon[35296]: osdmap e52: 9 total, 8 up, 9 in 2024-09-11T23:13:56.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:55 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-11T23:13:56.054 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:55 smithi076 ceph-mon[35296]: osdmap e53: 9 total, 8 up, 9 in 2024-09-11T23:13:56.054 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:55 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-11T23:13:56.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:55 smithi026 ceph-mon[31172]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-11T23:13:56.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:55 smithi026 ceph-mon[31172]: osdmap e51: 8 total, 8 up, 8 in 2024-09-11T23:13:56.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:55 smithi026 ceph-mon[31172]: from='client.? 172.21.15.136:0/1878574835' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "18851bde-9e6c-4e05-b4bf-ac5c3073eee7"}]: dispatch 2024-09-11T23:13:56.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:55 smithi026 ceph-mon[31172]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "18851bde-9e6c-4e05-b4bf-ac5c3073eee7"}]: dispatch 2024-09-11T23:13:56.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:55 smithi026 ceph-mon[31172]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "18851bde-9e6c-4e05-b4bf-ac5c3073eee7"}]': finished 2024-09-11T23:13:56.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:55 smithi026 ceph-mon[31172]: osdmap e52: 9 total, 8 up, 9 in 2024-09-11T23:13:56.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:55 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-11T23:13:56.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:55 smithi026 ceph-mon[31172]: osdmap e53: 9 total, 8 up, 9 in 2024-09-11T23:13:56.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:55 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-11T23:13:57.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:56 smithi076 ceph-mon[35296]: from='client.? 172.21.15.136:0/1738882923' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T23:13:57.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:56 smithi076 ceph-mon[35296]: osdmap e54: 9 total, 8 up, 9 in 2024-09-11T23:13:57.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:56 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-11T23:13:57.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:56 smithi136 ceph-mon[35121]: from='client.? 172.21.15.136:0/1738882923' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T23:13:57.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:56 smithi136 ceph-mon[35121]: osdmap e54: 9 total, 8 up, 9 in 2024-09-11T23:13:57.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:56 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-11T23:13:57.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:56 smithi026 ceph-mon[31172]: from='client.? 172.21.15.136:0/1738882923' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T23:13:57.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:56 smithi026 ceph-mon[31172]: osdmap e54: 9 total, 8 up, 9 in 2024-09-11T23:13:57.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:56 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-11T23:13:58.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:57 smithi076 ceph-mon[35296]: pgmap v174: 1 pgs: 1 remapped; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-11T23:13:58.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:57 smithi136 ceph-mon[35121]: pgmap v174: 1 pgs: 1 remapped; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-11T23:13:58.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:57 smithi026 ceph-mon[31172]: pgmap v174: 1 pgs: 1 remapped; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-11T23:14:00.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:13:59 smithi076 ceph-mon[35296]: pgmap v176: 1 pgs: 1 remapped; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-11T23:14:00.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:13:59 smithi136 ceph-mon[35121]: pgmap v176: 1 pgs: 1 remapped; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-11T23:14:00.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:13:59 smithi026 ceph-mon[31172]: pgmap v176: 1 pgs: 1 remapped; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-11T23:14:02.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:14:01 smithi076 ceph-mon[35296]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-11T23:14:02.090 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:14:01 smithi136 ceph-mon[35121]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-11T23:14:02.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:14:01 smithi026 ceph-mon[31172]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-11T23:14:04.052 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:14:03 smithi026 ceph-mon[31172]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-11T23:14:04.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:14:03 smithi076 ceph-mon[35296]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-11T23:14:04.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:14:03 smithi136 ceph-mon[35121]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-11T23:14:04.984 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:14:04 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-11T23:14:04.985 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:14:04 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:14:04.985 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:14:04 smithi136 ceph-mon[35121]: Deploying daemon osd.8 on smithi136 2024-09-11T23:14:05.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:14:04 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-11T23:14:05.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:14:04 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:14:05.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:14:04 smithi076 ceph-mon[35296]: Deploying daemon osd.8 on smithi136 2024-09-11T23:14:05.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:14:04 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-11T23:14:05.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:14:04 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:14:05.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:14:04 smithi026 ceph-mon[31172]: Deploying daemon osd.8 on smithi136 2024-09-11T23:14:05.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:14:05 smithi136 ceph-mon[35121]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-11T23:14:06.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:14:05 smithi026 ceph-mon[31172]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-11T23:14:06.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:14:05 smithi076 ceph-mon[35296]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-11T23:14:08.240 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:14:07 smithi136 ceph-mon[35121]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-11T23:14:08.253 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:14:07 smithi026 ceph-mon[31172]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-11T23:14:08.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:14:07 smithi076 ceph-mon[35296]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-11T23:14:09.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:14:09 smithi136 ceph-mon[35121]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-11T23:14:09.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:14:09 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:14:09.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:14:09 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:14:09.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:14:09 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:14:10.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:14:09 smithi026 ceph-mon[31172]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-11T23:14:10.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:14:09 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:14:10.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:14:09 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:14:10.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:14:09 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:14:10.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:14:09 smithi076 ceph-mon[35296]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-11T23:14:10.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:14:09 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:14:10.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:14:09 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:14:10.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:14:09 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:14:12.134 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:14:11 smithi136 ceph-mon[35121]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-11T23:14:12.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:14:11 smithi026 ceph-mon[31172]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-11T23:14:12.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:14:11 smithi076 ceph-mon[35296]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-11T23:14:13.337 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:14:13.337+0000 7f4bb8ff9640 1 -- 172.21.15.136:0/878756489 <== mgr.14148 v2:172.21.15.26:6800/110833859 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f4bbc062630 con 0x7f4ba40a2c30 2024-09-11T23:14:13.337 INFO:teuthology.orchestra.run.smithi136.stdout:Created osd(s) 8 on host 'smithi136' 2024-09-11T23:14:13.339 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:14:13.339+0000 7f4bc1c0f640 1 -- 172.21.15.136:0/878756489 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f4ba40a2c30 msgr2=0x7f4ba40a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:14:13.339 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:14:13.339+0000 7f4bc1c0f640 1 --2- 172.21.15.136:0/878756489 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f4ba40a2c30 0x7f4ba40a50f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f4bb0005fd0 tx=0x7f4bb0005850 comp rx=0 tx=0).stop 2024-09-11T23:14:13.339 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:14:13.339+0000 7f4bc1c0f640 1 -- 172.21.15.136:0/878756489 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4bbc1026c0 msgr2=0x7f4bbc197f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:14:13.340 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:14:13.339+0000 7f4bc1c0f640 1 --2- 172.21.15.136:0/878756489 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4bbc1026c0 0x7f4bbc197f30 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f4bac00b4a0 tx=0x7f4bac00b960 comp rx=0 tx=0).stop 2024-09-11T23:14:13.340 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:14:13.339+0000 7f4bc1c0f640 1 -- 172.21.15.136:0/878756489 shutdown_connections 2024-09-11T23:14:13.340 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:14:13.339+0000 7f4bc1c0f640 1 --2- 172.21.15.136:0/878756489 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f4ba40a2c30 0x7f4ba40a50f0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:14:13.340 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:14:13.339+0000 7f4bc1c0f640 1 --2- 172.21.15.136:0/878756489 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4bbc104ff0 0x7f4bbc198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:14:13.340 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:14:13.339+0000 7f4bc1c0f640 1 --2- 172.21.15.136:0/878756489 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4bbc107920 0x7f4bbc19f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:14:13.340 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:14:13.339+0000 7f4bc1c0f640 1 --2- 172.21.15.136:0/878756489 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4bbc1026c0 0x7f4bbc197f30 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:14:13.340 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:14:13.339+0000 7f4bc1c0f640 1 -- 172.21.15.136:0/878756489 >> 172.21.15.136:0/878756489 conn(0x7f4bbc0fc280 msgr2=0x7f4bbc102f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:14:13.340 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:14:13.340+0000 7f4bc1c0f640 1 -- 172.21.15.136:0/878756489 shutdown_connections 2024-09-11T23:14:13.340 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:14:13.340+0000 7f4bc1c0f640 1 -- 172.21.15.136:0/878756489 wait complete. 2024-09-11T23:14:13.716 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:14:13 smithi136 ceph-mon[35121]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-11T23:14:13.717 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:14:13 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:14:13.717 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:14:13 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:14:13.717 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:14:13 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:14:13.717 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:14:13 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:14:13.717 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:14:13 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:14:13.717 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:14:13 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:14:13.717 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:14:13 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:14:13.717 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:14:13 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:14:13.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:14:13 smithi026 ceph-mon[31172]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-11T23:14:13.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:14:13 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:14:13.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:14:13 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:14:13.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:14:13 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:14:13.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:14:13 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:14:13.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:14:13 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:14:13.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:14:13 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:14:13.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:14:13 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:14:13.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:14:13 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:14:13.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:14:13 smithi076 ceph-mon[35296]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-11T23:14:13.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:14:13 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:14:13.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:14:13 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:14:13.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:14:13 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:14:13.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:14:13 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:14:13.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:14:13 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:14:13.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:14:13 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:14:13.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:14:13 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:14:13.804 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:14:13 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:14:14.028 DEBUG:teuthology.orchestra.run.smithi136:osd.8> sudo journalctl -f -n 0 -u ceph-af7e6fc8-7092-11ef-bcea-c7b262605968@osd.8.service 2024-09-11T23:14:14.030 INFO:tasks.cephadm:Deploying osd.9 on smithi026 with /dev/vg_nvme/lv_1... 2024-09-11T23:14:14.030 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-11T23:14:14.303 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:14:15.367 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:14:15 smithi136 ceph-mon[35121]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-11T23:14:15.628 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:14:15 smithi026 ceph-mon[31172]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-11T23:14:15.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:14:15 smithi076 ceph-mon[35296]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-11T23:14:16.345 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 11 23:14:16 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-8[53569]: 2024-09-11T23:14:16.065+0000 7f99eaac5640 -1 osd.8 0 log_to_monitors true 2024-09-11T23:14:16.422 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:14:16 smithi026 ceph-mon[31172]: from='osd.8 [v2:172.21.15.136:6816/543619079,v1:172.21.15.136:6817/543619079]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-11T23:14:16.422 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:14:16 smithi026 ceph-mon[31172]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-11T23:14:16.623 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:14:16 smithi136 ceph-mon[35121]: from='osd.8 [v2:172.21.15.136:6816/543619079,v1:172.21.15.136:6817/543619079]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-11T23:14:16.623 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:14:16 smithi136 ceph-mon[35121]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-11T23:14:16.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:14:16 smithi076 ceph-mon[35296]: from='osd.8 [v2:172.21.15.136:6816/543619079,v1:172.21.15.136:6817/543619079]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-11T23:14:16.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:14:16 smithi076 ceph-mon[35296]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-11T23:14:17.181 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-11T23:14:17.207 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi026:vg_nvme/lv_1 2024-09-11T23:14:17.471 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:14:17.477 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:14:17 smithi026 ceph-mon[31172]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-11T23:14:17.477 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:14:17 smithi026 ceph-mon[31172]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-11T23:14:17.477 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:14:17 smithi026 ceph-mon[31172]: osdmap e55: 9 total, 8 up, 9 in 2024-09-11T23:14:17.477 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:14:17 smithi026 ceph-mon[31172]: from='osd.8 [v2:172.21.15.136:6816/543619079,v1:172.21.15.136:6817/543619079]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-09-11T23:14:17.477 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:14:17 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-11T23:14:17.477 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:14:17 smithi026 ceph-mon[31172]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-09-11T23:14:17.477 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:14:17 smithi026 ceph-mon[31172]: Detected new or changed devices on smithi136 2024-09-11T23:14:17.477 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:14:17 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:14:17.477 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:14:17 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:14:17.477 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:14:17 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:14:17.477 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:14:17 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:14:17.478 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:14:17 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:14:17.478 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:14:17 smithi026 ceph-mon[31172]: Adjusting osd_memory_target on smithi136 to 5696M 2024-09-11T23:14:17.478 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:14:17 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:14:17.478 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:14:17 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:14:17.478 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:14:17 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:14:17.478 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:14:17 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:14:17.739 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 11 23:14:17 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-8[53569]: 2024-09-11T23:14:17.355+0000 7f99e6225640 -1 osd.8 0 waiting for initial osdmap 2024-09-11T23:14:17.739 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 11 23:14:17 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-8[53569]: 2024-09-11T23:14:17.366+0000 7f99e285f640 -1 osd.8 56 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T23:14:17.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:14:17 smithi136 ceph-mon[35121]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-11T23:14:17.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:14:17 smithi136 ceph-mon[35121]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-11T23:14:17.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:14:17 smithi136 ceph-mon[35121]: osdmap e55: 9 total, 8 up, 9 in 2024-09-11T23:14:17.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:14:17 smithi136 ceph-mon[35121]: from='osd.8 [v2:172.21.15.136:6816/543619079,v1:172.21.15.136:6817/543619079]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-09-11T23:14:17.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:14:17 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-11T23:14:17.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:14:17 smithi136 ceph-mon[35121]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-09-11T23:14:17.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:14:17 smithi136 ceph-mon[35121]: Detected new or changed devices on smithi136 2024-09-11T23:14:17.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:14:17 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:14:17.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:14:17 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:14:17.741 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:14:17 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:14:17.741 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:14:17 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:14:17.741 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:14:17 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:14:17.741 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:14:17 smithi136 ceph-mon[35121]: Adjusting osd_memory_target on smithi136 to 5696M 2024-09-11T23:14:17.741 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:14:17 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:14:17.741 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:14:17 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:14:17.741 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:14:17 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:14:17.741 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:14:17 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:14:17.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:14:17 smithi076 ceph-mon[35296]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-11T23:14:17.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:14:17 smithi076 ceph-mon[35296]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-11T23:14:17.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:14:17 smithi076 ceph-mon[35296]: osdmap e55: 9 total, 8 up, 9 in 2024-09-11T23:14:17.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:14:17 smithi076 ceph-mon[35296]: from='osd.8 [v2:172.21.15.136:6816/543619079,v1:172.21.15.136:6817/543619079]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-09-11T23:14:17.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:14:17 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-11T23:14:17.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:14:17 smithi076 ceph-mon[35296]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-09-11T23:14:17.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:14:17 smithi076 ceph-mon[35296]: Detected new or changed devices on smithi136 2024-09-11T23:14:17.804 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:14:17 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:14:17.804 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:14:17 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:14:17.804 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:14:17 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:14:17.804 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:14:17 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:14:17.804 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:14:17 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:14:17.804 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:14:17 smithi076 ceph-mon[35296]: Adjusting osd_memory_target on smithi136 to 5696M 2024-09-11T23:14:17.804 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:14:17 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:14:17.804 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:14:17 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:14:17.804 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:14:17 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:14:17.804 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:14:17 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:14:17.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:14:17.950+0000 7f3a1bbe3640 1 -- 172.21.15.26:0/2667855708 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a14102e00 msgr2=0x7f3a14103280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:14:17.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:14:17.950+0000 7f3a1bbe3640 1 --2- 172.21.15.26:0/2667855708 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a14102e00 0x7f3a14103280 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f3a10009900 tx=0x7f3a1002a410 comp rx=0 tx=0).stop 2024-09-11T23:14:17.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:14:17.950+0000 7f3a1bbe3640 1 -- 172.21.15.26:0/2667855708 shutdown_connections 2024-09-11T23:14:17.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:14:17.950+0000 7f3a1bbe3640 1 --2- 172.21.15.26:0/2667855708 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3a141037c0 0x7f3a1410a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:14:17.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:14:17.950+0000 7f3a1bbe3640 1 --2- 172.21.15.26:0/2667855708 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a14102e00 0x7f3a14103280 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:14:17.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:14:17.950+0000 7f3a1bbe3640 1 --2- 172.21.15.26:0/2667855708 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3a14101c00 0x7f3a14102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:14:17.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:14:17.950+0000 7f3a1bbe3640 1 -- 172.21.15.26:0/2667855708 >> 172.21.15.26:0/2667855708 conn(0x7f3a140fd390 msgr2=0x7f3a140ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:14:17.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:14:17.950+0000 7f3a1bbe3640 1 -- 172.21.15.26:0/2667855708 shutdown_connections 2024-09-11T23:14:17.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:14:17.950+0000 7f3a1bbe3640 1 -- 172.21.15.26:0/2667855708 wait complete. 2024-09-11T23:14:17.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:14:17.951+0000 7f3a1bbe3640 1 Processor -- start 2024-09-11T23:14:17.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:14:17.951+0000 7f3a1bbe3640 1 -- start start 2024-09-11T23:14:17.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:14:17.951+0000 7f3a1bbe3640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3a14101c00 0x7f3a14075f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:14:17.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:14:17.951+0000 7f3a1bbe3640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3a14102e00 0x7f3a140745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:14:17.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:14:17.951+0000 7f3a1bbe3640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a141037c0 0x7f3a14074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:14:17.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:14:17.951+0000 7f3a1bbe3640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f3a1410d860 con 0x7f3a141037c0 2024-09-11T23:14:17.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:14:17.951+0000 7f3a1bbe3640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f3a1410d6e0 con 0x7f3a14102e00 2024-09-11T23:14:17.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:14:17.951+0000 7f3a1bbe3640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f3a1410d9e0 con 0x7f3a14101c00 2024-09-11T23:14:17.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:14:17.951+0000 7f3a1a180640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a141037c0 0x7f3a14074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:14:17.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:14:17.951+0000 7f3a1a180640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a141037c0 0x7f3a14074b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51702/0 (socket says 172.21.15.26:51702) 2024-09-11T23:14:17.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:14:17.951+0000 7f3a1a180640 1 -- 172.21.15.26:0/1028899475 learned_addr learned my addr 172.21.15.26:0/1028899475 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:14:17.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:14:17.951+0000 7f3a1917e640 1 --2- 172.21.15.26:0/1028899475 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3a14102e00 0x7f3a140745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:14:17.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:14:17.952+0000 7f3a1997f640 1 --2- 172.21.15.26:0/1028899475 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3a14101c00 0x7f3a14075f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:14:17.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:14:17.952+0000 7f3a1a180640 1 -- 172.21.15.26:0/1028899475 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3a14101c00 msgr2=0x7f3a14075f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:14:17.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:14:17.952+0000 7f3a1a180640 1 --2- 172.21.15.26:0/1028899475 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3a14101c00 0x7f3a14075f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:14:17.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:14:17.952+0000 7f3a1a180640 1 -- 172.21.15.26:0/1028899475 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3a14102e00 msgr2=0x7f3a140745e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:14:17.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:14:17.952+0000 7f3a1a180640 1 --2- 172.21.15.26:0/1028899475 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3a14102e00 0x7f3a140745e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:14:17.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:14:17.952+0000 7f3a1a180640 1 -- 172.21.15.26:0/1028899475 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a140753e0 con 0x7f3a141037c0 2024-09-11T23:14:17.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:14:17.952+0000 7f3a1997f640 1 --2- 172.21.15.26:0/1028899475 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3a14101c00 0x7f3a14075f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:14:17.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:14:17.952+0000 7f3a1a180640 1 --2- 172.21.15.26:0/1028899475 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a141037c0 0x7f3a14074b20 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f3a0c00eaa0 tx=0x7f3a0c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:14:17.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:14:17.952+0000 7f3a0affd640 1 -- 172.21.15.26:0/1028899475 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a0c00cdb0 con 0x7f3a141037c0 2024-09-11T23:14:17.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:14:17.952+0000 7f3a0affd640 1 -- 172.21.15.26:0/1028899475 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a0c005020 con 0x7f3a141037c0 2024-09-11T23:14:17.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:14:17.952+0000 7f3a1bbe3640 1 -- 172.21.15.26:0/1028899475 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a141a1fd0 con 0x7f3a141037c0 2024-09-11T23:14:17.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:14:17.952+0000 7f3a0affd640 1 -- 172.21.15.26:0/1028899475 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a0c0116b0 con 0x7f3a141037c0 2024-09-11T23:14:17.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:14:17.952+0000 7f3a1bbe3640 1 -- 172.21.15.26:0/1028899475 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a141a2490 con 0x7f3a141037c0 2024-09-11T23:14:17.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:14:17.953+0000 7f3a1bbe3640 1 -- 172.21.15.26:0/1028899475 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a14108fa0 con 0x7f3a141037c0 2024-09-11T23:14:17.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:14:17.954+0000 7f3a0affd640 1 -- 172.21.15.26:0/1028899475 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f3a0c005560 con 0x7f3a141037c0 2024-09-11T23:14:17.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:14:17.955+0000 7f3a0affd640 1 --2- 172.21.15.26:0/1028899475 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f3a000a2b80 0x7f3a000a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:14:17.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:14:17.955+0000 7f3a1997f640 1 --2- 172.21.15.26:0/1028899475 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f3a000a2b80 0x7f3a000a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:14:17.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:14:17.955+0000 7f3a0affd640 1 -- 172.21.15.26:0/1028899475 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5549+0+0 (secure 0 0 0) 0x7f3a0c0ce7c0 con 0x7f3a141037c0 2024-09-11T23:14:17.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:14:17.955+0000 7f3a1997f640 1 --2- 172.21.15.26:0/1028899475 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f3a000a2b80 0x7f3a000a5040 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f3a14102c60 tx=0x7f3a04005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:14:17.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:14:17.959+0000 7f3a0affd640 1 -- 172.21.15.26:0/1028899475 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3a0c016030 con 0x7f3a141037c0 2024-09-11T23:14:18.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:14:18.145+0000 7f3a1bbe3640 1 -- 172.21.15.26:0/1028899475 --> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi026:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f3a14062630 con 0x7f3a000a2b80 2024-09-11T23:14:18.501 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:14:18 smithi026 ceph-mon[31172]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]': finished 2024-09-11T23:14:18.501 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:14:18 smithi026 ceph-mon[31172]: osdmap e56: 9 total, 8 up, 9 in 2024-09-11T23:14:18.501 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:14:18 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-11T23:14:18.501 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:14:18 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-11T23:14:18.501 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:14:18 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T23:14:18.501 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:14:18 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T23:14:18.501 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:14:18 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:14:18.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:14:18 smithi136 ceph-mon[35121]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]': finished 2024-09-11T23:14:18.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:14:18 smithi136 ceph-mon[35121]: osdmap e56: 9 total, 8 up, 9 in 2024-09-11T23:14:18.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:14:18 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-11T23:14:18.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:14:18 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-11T23:14:18.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:14:18 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T23:14:18.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:14:18 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T23:14:18.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:14:18 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:14:18.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:14:18 smithi076 ceph-mon[35296]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]': finished 2024-09-11T23:14:18.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:14:18 smithi076 ceph-mon[35296]: osdmap e56: 9 total, 8 up, 9 in 2024-09-11T23:14:18.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:14:18 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-11T23:14:18.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:14:18 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-11T23:14:18.804 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:14:18 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T23:14:18.804 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:14:18 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T23:14:18.804 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:14:18 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:14:19.718 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:14:19 smithi026 ceph-mon[31172]: purged_snaps scrub starts 2024-09-11T23:14:19.718 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:14:19 smithi026 ceph-mon[31172]: purged_snaps scrub ok 2024-09-11T23:14:19.718 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:14:19 smithi026 ceph-mon[31172]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-11T23:14:19.718 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:14:19 smithi026 ceph-mon[31172]: from='client.14391 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi026:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:14:19.718 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:14:19 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-11T23:14:19.719 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:14:19 smithi026 ceph-mon[31172]: osd.8 [v2:172.21.15.136:6816/543619079,v1:172.21.15.136:6817/543619079] boot 2024-09-11T23:14:19.719 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:14:19 smithi026 ceph-mon[31172]: osdmap e57: 9 total, 9 up, 9 in 2024-09-11T23:14:19.719 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:14:19 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-11T23:14:19.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:14:19 smithi136 ceph-mon[35121]: purged_snaps scrub starts 2024-09-11T23:14:19.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:14:19 smithi136 ceph-mon[35121]: purged_snaps scrub ok 2024-09-11T23:14:19.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:14:19 smithi136 ceph-mon[35121]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-11T23:14:19.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:14:19 smithi136 ceph-mon[35121]: from='client.14391 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi026:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:14:19.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:14:19 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-11T23:14:19.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:14:19 smithi136 ceph-mon[35121]: osd.8 [v2:172.21.15.136:6816/543619079,v1:172.21.15.136:6817/543619079] boot 2024-09-11T23:14:19.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:14:19 smithi136 ceph-mon[35121]: osdmap e57: 9 total, 9 up, 9 in 2024-09-11T23:14:19.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:14:19 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-11T23:14:19.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:14:19 smithi076 ceph-mon[35296]: purged_snaps scrub starts 2024-09-11T23:14:19.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:14:19 smithi076 ceph-mon[35296]: purged_snaps scrub ok 2024-09-11T23:14:19.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:14:19 smithi076 ceph-mon[35296]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-11T23:14:19.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:14:19 smithi076 ceph-mon[35296]: from='client.14391 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi026:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:14:19.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:14:19 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-11T23:14:19.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:14:19 smithi076 ceph-mon[35296]: osd.8 [v2:172.21.15.136:6816/543619079,v1:172.21.15.136:6817/543619079] boot 2024-09-11T23:14:19.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:14:19 smithi076 ceph-mon[35296]: osdmap e57: 9 total, 9 up, 9 in 2024-09-11T23:14:19.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:14:19 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-11T23:14:20.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:14:20 smithi136 ceph-mon[35121]: osdmap e58: 9 total, 9 up, 9 in 2024-09-11T23:14:20.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:14:20 smithi026 ceph-mon[31172]: osdmap e58: 9 total, 9 up, 9 in 2024-09-11T23:14:20.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:14:20 smithi076 ceph-mon[35296]: osdmap e58: 9 total, 9 up, 9 in 2024-09-11T23:14:21.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:14:21 smithi136 ceph-mon[35121]: pgmap v191: 1 pgs: 1 remapped+peering; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-09-11T23:14:21.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:14:21 smithi136 ceph-mon[35121]: osdmap e59: 9 total, 9 up, 9 in 2024-09-11T23:14:21.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:14:21 smithi136 ceph-mon[35121]: from='client.? 172.21.15.26:0/1638712180' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "52a65eab-8446-47ad-bfe7-5133f7e3eb98"}]: dispatch 2024-09-11T23:14:21.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:14:21 smithi136 ceph-mon[35121]: from='client.? 172.21.15.26:0/1638712180' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "52a65eab-8446-47ad-bfe7-5133f7e3eb98"}]': finished 2024-09-11T23:14:21.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:14:21 smithi136 ceph-mon[35121]: osdmap e60: 10 total, 9 up, 10 in 2024-09-11T23:14:21.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:14:21 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T23:14:21.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:14:21 smithi136 ceph-mon[35121]: from='client.? 172.21.15.26:0/3156500597' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T23:14:21.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:14:21 smithi026 ceph-mon[31172]: pgmap v191: 1 pgs: 1 remapped+peering; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-09-11T23:14:21.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:14:21 smithi026 ceph-mon[31172]: osdmap e59: 9 total, 9 up, 9 in 2024-09-11T23:14:21.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:14:21 smithi026 ceph-mon[31172]: from='client.? 172.21.15.26:0/1638712180' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "52a65eab-8446-47ad-bfe7-5133f7e3eb98"}]: dispatch 2024-09-11T23:14:21.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:14:21 smithi026 ceph-mon[31172]: from='client.? 172.21.15.26:0/1638712180' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "52a65eab-8446-47ad-bfe7-5133f7e3eb98"}]': finished 2024-09-11T23:14:21.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:14:21 smithi026 ceph-mon[31172]: osdmap e60: 10 total, 9 up, 10 in 2024-09-11T23:14:21.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:14:21 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T23:14:21.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:14:21 smithi026 ceph-mon[31172]: from='client.? 172.21.15.26:0/3156500597' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T23:14:21.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:14:21 smithi076 ceph-mon[35296]: pgmap v191: 1 pgs: 1 remapped+peering; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-09-11T23:14:21.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:14:21 smithi076 ceph-mon[35296]: osdmap e59: 9 total, 9 up, 9 in 2024-09-11T23:14:21.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:14:21 smithi076 ceph-mon[35296]: from='client.? 172.21.15.26:0/1638712180' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "52a65eab-8446-47ad-bfe7-5133f7e3eb98"}]: dispatch 2024-09-11T23:14:21.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:14:21 smithi076 ceph-mon[35296]: from='client.? 172.21.15.26:0/1638712180' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "52a65eab-8446-47ad-bfe7-5133f7e3eb98"}]': finished 2024-09-11T23:14:21.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:14:21 smithi076 ceph-mon[35296]: osdmap e60: 10 total, 9 up, 10 in 2024-09-11T23:14:21.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:14:21 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T23:14:21.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:14:21 smithi076 ceph-mon[35296]: from='client.? 172.21.15.26:0/3156500597' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T23:14:22.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:14:22 smithi136 ceph-mon[35121]: osdmap e61: 10 total, 9 up, 10 in 2024-09-11T23:14:22.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:14:22 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T23:14:23.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:14:22 smithi026 ceph-mon[31172]: osdmap e61: 10 total, 9 up, 10 in 2024-09-11T23:14:23.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:14:22 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T23:14:23.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:14:22 smithi076 ceph-mon[35296]: osdmap e61: 10 total, 9 up, 10 in 2024-09-11T23:14:23.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:14:22 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T23:14:23.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:14:23 smithi136 ceph-mon[35121]: pgmap v195: 1 pgs: 1 remapped+peering; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-09-11T23:14:23.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:14:23 smithi136 ceph-mon[35121]: osdmap e62: 10 total, 9 up, 10 in 2024-09-11T23:14:23.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:14:23 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T23:14:24.000 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:14:23 smithi026 ceph-mon[31172]: pgmap v195: 1 pgs: 1 remapped+peering; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-09-11T23:14:24.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:14:23 smithi026 ceph-mon[31172]: osdmap e62: 10 total, 9 up, 10 in 2024-09-11T23:14:24.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:14:23 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T23:14:24.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:14:23 smithi076 ceph-mon[35296]: pgmap v195: 1 pgs: 1 remapped+peering; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-09-11T23:14:24.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:14:23 smithi076 ceph-mon[35296]: osdmap e62: 10 total, 9 up, 10 in 2024-09-11T23:14:24.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:14:23 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T23:14:25.881 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:14:25 smithi026 ceph-mon[31172]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-11T23:14:25.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:14:25 smithi136 ceph-mon[35121]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-11T23:14:26.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:14:25 smithi076 ceph-mon[35296]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-11T23:14:27.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:14:27 smithi136 ceph-mon[35121]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 101 KiB/s, 0 objects/s recovering 2024-09-11T23:14:28.000 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:14:27 smithi026 ceph-mon[31172]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 101 KiB/s, 0 objects/s recovering 2024-09-11T23:14:28.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:14:27 smithi076 ceph-mon[35296]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 101 KiB/s, 0 objects/s recovering 2024-09-11T23:14:29.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:14:29 smithi136 ceph-mon[35121]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-09-11T23:14:29.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:14:29 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-11T23:14:29.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:14:29 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:14:30.000 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:14:29 smithi026 ceph-mon[31172]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-09-11T23:14:30.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:14:29 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-11T23:14:30.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:14:29 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:14:30.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:14:29 smithi076 ceph-mon[35296]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-09-11T23:14:30.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:14:29 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-11T23:14:30.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:14:29 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:14:30.903 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:14:30 smithi026 ceph-mon[31172]: Deploying daemon osd.9 on smithi026 2024-09-11T23:14:30.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:14:30 smithi136 ceph-mon[35121]: Deploying daemon osd.9 on smithi026 2024-09-11T23:14:31.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:14:30 smithi076 ceph-mon[35296]: Deploying daemon osd.9 on smithi026 2024-09-11T23:14:31.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:14:31 smithi136 ceph-mon[35121]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-09-11T23:14:32.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:14:31 smithi026 ceph-mon[31172]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-09-11T23:14:32.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:14:31 smithi076 ceph-mon[35296]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-09-11T23:14:33.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:14:33 smithi136 ceph-mon[35121]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-11T23:14:34.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:14:33 smithi026 ceph-mon[31172]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-11T23:14:34.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:14:33 smithi076 ceph-mon[35296]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-11T23:14:35.962 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:14:35 smithi026 ceph-mon[31172]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-09-11T23:14:35.962 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:14:35 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:14:35.962 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:14:35 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:14:35.962 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:14:35 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:14:35.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:14:35 smithi136 ceph-mon[35121]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-09-11T23:14:35.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:14:35 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:14:35.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:14:35 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:14:35.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:14:35 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:14:36.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:14:35 smithi076 ceph-mon[35296]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-09-11T23:14:36.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:14:35 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:14:36.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:14:35 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:14:36.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:14:35 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:14:37.942 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:14:37 smithi026 ceph-mon[31172]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-09-11T23:14:37.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:14:37 smithi136 ceph-mon[35121]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-09-11T23:14:38.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:14:37 smithi076 ceph-mon[35296]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-09-11T23:14:39.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:14:39.196+0000 7f3a0affd640 1 -- 172.21.15.26:0/1028899475 <== mgr.14148 v2:172.21.15.26:6800/110833859 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f3a14062630 con 0x7f3a000a2b80 2024-09-11T23:14:39.196 INFO:teuthology.orchestra.run.smithi026.stdout:Created osd(s) 9 on host 'smithi026' 2024-09-11T23:14:39.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:14:39.199+0000 7f3a1bbe3640 1 -- 172.21.15.26:0/1028899475 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f3a000a2b80 msgr2=0x7f3a000a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:14:39.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:14:39.199+0000 7f3a1bbe3640 1 --2- 172.21.15.26:0/1028899475 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f3a000a2b80 0x7f3a000a5040 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f3a14102c60 tx=0x7f3a04005f90 comp rx=0 tx=0).stop 2024-09-11T23:14:39.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:14:39.199+0000 7f3a1bbe3640 1 -- 172.21.15.26:0/1028899475 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a141037c0 msgr2=0x7f3a14074b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:14:39.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:14:39.199+0000 7f3a1bbe3640 1 --2- 172.21.15.26:0/1028899475 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a141037c0 0x7f3a14074b20 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f3a0c00eaa0 tx=0x7f3a0c00ef60 comp rx=0 tx=0).stop 2024-09-11T23:14:39.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:14:39.199+0000 7f3a1bbe3640 1 -- 172.21.15.26:0/1028899475 shutdown_connections 2024-09-11T23:14:39.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:14:39.199+0000 7f3a1bbe3640 1 --2- 172.21.15.26:0/1028899475 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f3a000a2b80 0x7f3a000a5040 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:14:39.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:14:39.199+0000 7f3a1bbe3640 1 --2- 172.21.15.26:0/1028899475 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3a14102e00 0x7f3a140745e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:14:39.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:14:39.199+0000 7f3a1bbe3640 1 --2- 172.21.15.26:0/1028899475 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a141037c0 0x7f3a14074b20 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:14:39.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:14:39.199+0000 7f3a1bbe3640 1 --2- 172.21.15.26:0/1028899475 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3a14101c00 0x7f3a14075f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:14:39.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:14:39.199+0000 7f3a1bbe3640 1 -- 172.21.15.26:0/1028899475 >> 172.21.15.26:0/1028899475 conn(0x7f3a140fd390 msgr2=0x7f3a14106170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:14:39.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:14:39.199+0000 7f3a1bbe3640 1 -- 172.21.15.26:0/1028899475 shutdown_connections 2024-09-11T23:14:39.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:14:39.199+0000 7f3a1bbe3640 1 -- 172.21.15.26:0/1028899475 wait complete. 2024-09-11T23:14:39.501 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:14:39 smithi026 ceph-mon[31172]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-09-11T23:14:39.501 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:14:39 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:14:39.501 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:14:39 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:14:39.501 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:14:39 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:14:39.501 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:14:39 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:14:39.501 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:14:39 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:14:39.501 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:14:39 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:14:39.501 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:14:39 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:14:39.501 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:14:39 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:14:39.553 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:14:39 smithi076 ceph-mon[35296]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-09-11T23:14:39.553 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:14:39 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:14:39.553 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:14:39 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:14:39.553 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:14:39 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:14:39.553 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:14:39 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:14:39.553 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:14:39 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:14:39.553 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:14:39 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:14:39.554 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:14:39 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:14:39.554 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:14:39 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:14:39.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:14:39 smithi136 ceph-mon[35121]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-09-11T23:14:39.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:14:39 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:14:39.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:14:39 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:14:39.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:14:39 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:14:39.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:14:39 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:14:39.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:14:39 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:14:39.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:14:39 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:14:39.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:14:39 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:14:39.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:14:39 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:14:39.936 DEBUG:teuthology.orchestra.run.smithi026:osd.9> sudo journalctl -f -n 0 -u ceph-af7e6fc8-7092-11ef-bcea-c7b262605968@osd.9.service 2024-09-11T23:14:39.939 INFO:tasks.cephadm:Deploying osd.10 on smithi076 with /dev/vg_nvme/lv_1... 2024-09-11T23:14:39.939 DEBUG:teuthology.orchestra.run.smithi076:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-11T23:14:40.191 INFO:teuthology.orchestra.run.smithi076.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T23:14:41.479 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:14:41 smithi026 ceph-mon[31172]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-09-11T23:14:41.536 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:14:41 smithi076 ceph-mon[35296]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-09-11T23:14:41.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:14:41 smithi136 ceph-mon[35121]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-09-11T23:14:42.234 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:14:41 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9[65389]: 2024-09-11T23:14:41.839+0000 7fc757e54640 -1 osd.9 0 log_to_monitors true 2024-09-11T23:14:42.429 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:14:42 smithi076 ceph-mon[35296]: from='osd.9 [v2:172.21.15.26:6826/2201297064,v1:172.21.15.26:6827/2201297064]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-11T23:14:42.501 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:14:42 smithi026 ceph-mon[31172]: from='osd.9 [v2:172.21.15.26:6826/2201297064,v1:172.21.15.26:6827/2201297064]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-11T23:14:42.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:14:42 smithi136 ceph-mon[35121]: from='osd.9 [v2:172.21.15.26:6826/2201297064,v1:172.21.15.26:6827/2201297064]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-11T23:14:42.751 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-11T23:14:42.778 DEBUG:teuthology.orchestra.run.smithi076:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi076:vg_nvme/lv_1 2024-09-11T23:14:43.035 INFO:teuthology.orchestra.run.smithi076.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T23:14:43.313 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:14:43 smithi076 ceph-mon[35296]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-09-11T23:14:43.313 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:14:43 smithi076 ceph-mon[35296]: from='osd.9 [v2:172.21.15.26:6826/2201297064,v1:172.21.15.26:6827/2201297064]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-11T23:14:43.313 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:14:43 smithi076 ceph-mon[35296]: osdmap e63: 10 total, 9 up, 10 in 2024-09-11T23:14:43.313 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:14:43 smithi076 ceph-mon[35296]: from='osd.9 [v2:172.21.15.26:6826/2201297064,v1:172.21.15.26:6827/2201297064]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2024-09-11T23:14:43.313 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:14:43 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T23:14:43.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:14:43 smithi076 ceph-mon[35296]: Detected new or changed devices on smithi026 2024-09-11T23:14:43.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:14:43 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:14:43.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:14:43 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:14:43.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:14:43 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:14:43.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:14:43 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:14:43.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:14:43 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:14:43.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:14:43 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:14:43.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:14:43 smithi076 ceph-mon[35296]: Adjusting osd_memory_target on smithi026 to 4273M 2024-09-11T23:14:43.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:14:43 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:14:43.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:14:43 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:14:43.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:14:43 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:14:43.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:14:43 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:14:43.536 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:14:43.503+0000 7fc778faa640 1 -- 172.21.15.76:0/316858049 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc7741026a0 msgr2=0x7fc774104a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:14:43.536 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:14:43.503+0000 7fc778faa640 1 --2- 172.21.15.76:0/316858049 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc7741026a0 0x7fc774104a90 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fc764009900 tx=0x7fc76402a410 comp rx=0 tx=0).stop 2024-09-11T23:14:43.536 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:14:43.503+0000 7fc778faa640 1 -- 172.21.15.76:0/316858049 shutdown_connections 2024-09-11T23:14:43.536 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:14:43.503+0000 7fc778faa640 1 --2- 172.21.15.76:0/316858049 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc774107900 0x7fc774109d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:14:43.536 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:14:43.503+0000 7fc778faa640 1 --2- 172.21.15.76:0/316858049 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc774104fd0 0x7fc7741073c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:14:43.536 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:14:43.503+0000 7fc778faa640 1 --2- 172.21.15.76:0/316858049 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc7741026a0 0x7fc774104a90 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:14:43.536 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:14:43.503+0000 7fc778faa640 1 -- 172.21.15.76:0/316858049 >> 172.21.15.76:0/316858049 conn(0x7fc7740fc2a0 msgr2=0x7fc7740fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:14:43.536 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:14:43.503+0000 7fc778faa640 1 -- 172.21.15.76:0/316858049 shutdown_connections 2024-09-11T23:14:43.536 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:14:43.503+0000 7fc778faa640 1 -- 172.21.15.76:0/316858049 wait complete. 2024-09-11T23:14:43.536 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:14:43.504+0000 7fc778faa640 1 Processor -- start 2024-09-11T23:14:43.537 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:14:43.504+0000 7fc778faa640 1 -- start start 2024-09-11T23:14:43.537 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:14:43.504+0000 7fc778faa640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc7741026a0 0x7fc774102000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:14:43.537 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:14:43.504+0000 7fc778faa640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc774104fd0 0x7fc774100650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:14:43.537 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:14:43.504+0000 7fc778faa640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc774107900 0x7fc774100b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:14:43.537 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:14:43.504+0000 7fc778faa640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc77410d940 con 0x7fc774107900 2024-09-11T23:14:43.538 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:14:43.504+0000 7fc778faa640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fc77410d7c0 con 0x7fc774104fd0 2024-09-11T23:14:43.538 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:14:43.504+0000 7fc778faa640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fc77410dac0 con 0x7fc7741026a0 2024-09-11T23:14:43.538 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:14:43.504+0000 7fc771d9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc774104fd0 0x7fc774100650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:14:43.538 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:14:43.504+0000 7fc771d9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc774104fd0 0x7fc774100650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:54584/0 (socket says 172.21.15.76:54584) 2024-09-11T23:14:43.538 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:14:43.504+0000 7fc771d9b640 1 -- 172.21.15.76:0/1337097652 learned_addr learned my addr 172.21.15.76:0/1337097652 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-11T23:14:43.538 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:14:43.504+0000 7fc77259c640 1 --2- 172.21.15.76:0/1337097652 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc7741026a0 0x7fc774102000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:14:43.539 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:14:43.504+0000 7fc772d9d640 1 --2- 172.21.15.76:0/1337097652 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc774107900 0x7fc774100b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:14:43.539 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:14:43.505+0000 7fc771d9b640 1 -- 172.21.15.76:0/1337097652 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc7741026a0 msgr2=0x7fc774102000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:14:43.539 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:14:43.505+0000 7fc771d9b640 1 --2- 172.21.15.76:0/1337097652 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc7741026a0 0x7fc774102000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:14:43.539 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:14:43.505+0000 7fc771d9b640 1 -- 172.21.15.76:0/1337097652 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc774107900 msgr2=0x7fc774100b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:14:43.539 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:14:43.505+0000 7fc771d9b640 1 --2- 172.21.15.76:0/1337097652 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc774107900 0x7fc774100b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:14:43.539 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:14:43.505+0000 7fc771d9b640 1 -- 172.21.15.76:0/1337097652 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc774101450 con 0x7fc774104fd0 2024-09-11T23:14:43.539 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:14:43.505+0000 7fc771d9b640 1 --2- 172.21.15.76:0/1337097652 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc774104fd0 0x7fc774100650 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fc76800d740 tx=0x7fc76800dc00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:14:43.540 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:14:43.505+0000 7fc7637fe640 1 -- 172.21.15.76:0/1337097652 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc768017070 con 0x7fc774104fd0 2024-09-11T23:14:43.540 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:14:43.505+0000 7fc778faa640 1 -- 172.21.15.76:0/1337097652 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7741a2180 con 0x7fc774104fd0 2024-09-11T23:14:43.540 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:14:43.505+0000 7fc7637fe640 1 -- 172.21.15.76:0/1337097652 <== mon.2 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc76800de50 con 0x7fc774104fd0 2024-09-11T23:14:43.541 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:14:43.505+0000 7fc7637fe640 1 -- 172.21.15.76:0/1337097652 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc768004e00 con 0x7fc774104fd0 2024-09-11T23:14:43.541 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:14:43.505+0000 7fc778faa640 1 -- 172.21.15.76:0/1337097652 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7741a2710 con 0x7fc774104fd0 2024-09-11T23:14:43.541 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:14:43.506+0000 7fc778faa640 1 -- 172.21.15.76:0/1337097652 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7740fdd90 con 0x7fc774104fd0 2024-09-11T23:14:43.541 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:14:43.507+0000 7fc7637fe640 1 -- 172.21.15.76:0/1337097652 <== mon.2 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fc768004fa0 con 0x7fc774104fd0 2024-09-11T23:14:43.541 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:14:43.508+0000 7fc7637fe640 1 --2- 172.21.15.76:0/1337097652 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fc75c0a2b80 0x7fc75c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:14:43.542 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:14:43.508+0000 7fc7637fe640 1 -- 172.21.15.76:0/1337097652 <== mon.2 v2:172.21.15.76:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5981+0+0 (secure 0 0 0) 0x7fc7680ce3b0 con 0x7fc774104fd0 2024-09-11T23:14:43.542 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:14:43.508+0000 7fc77259c640 1 --2- 172.21.15.76:0/1337097652 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fc75c0a2b80 0x7fc75c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:14:43.542 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:14:43.508+0000 7fc77259c640 1 --2- 172.21.15.76:0/1337097652 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fc75c0a2b80 0x7fc75c0a5040 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fc7640314e0 tx=0x7fc764000bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:14:43.542 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:14:43.511+0000 7fc7637fe640 1 -- 172.21.15.76:0/1337097652 <== mon.2 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc76809b5e0 con 0x7fc774104fd0 2024-09-11T23:14:43.640 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:14:43 smithi136 ceph-mon[35121]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-09-11T23:14:43.640 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:14:43 smithi136 ceph-mon[35121]: from='osd.9 [v2:172.21.15.26:6826/2201297064,v1:172.21.15.26:6827/2201297064]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-11T23:14:43.641 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:14:43 smithi136 ceph-mon[35121]: osdmap e63: 10 total, 9 up, 10 in 2024-09-11T23:14:43.641 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:14:43 smithi136 ceph-mon[35121]: from='osd.9 [v2:172.21.15.26:6826/2201297064,v1:172.21.15.26:6827/2201297064]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2024-09-11T23:14:43.641 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:14:43 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T23:14:43.641 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:14:43 smithi136 ceph-mon[35121]: Detected new or changed devices on smithi026 2024-09-11T23:14:43.641 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:14:43 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:14:43.641 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:14:43 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:14:43.641 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:14:43 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:14:43.641 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:14:43 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:14:43.642 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:14:43 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:14:43.642 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:14:43 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:14:43.642 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:14:43 smithi136 ceph-mon[35121]: Adjusting osd_memory_target on smithi026 to 4273M 2024-09-11T23:14:43.642 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:14:43 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:14:43.642 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:14:43 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:14:43.643 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:14:43 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:14:43.643 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:14:43 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:14:43.685 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:14:43.685+0000 7fc778faa640 1 -- 172.21.15.76:0/1337097652 --> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi076:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fc774062630 con 0x7fc75c0a2b80 2024-09-11T23:14:43.751 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:14:43 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9[65389]: 2024-09-11T23:14:43.283+0000 7fc7545eb640 -1 osd.9 0 waiting for initial osdmap 2024-09-11T23:14:43.751 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:14:43 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9[65389]: 2024-09-11T23:14:43.298+0000 7fc74fbee640 -1 osd.9 64 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T23:14:43.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:14:43 smithi026 ceph-mon[31172]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-09-11T23:14:43.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:14:43 smithi026 ceph-mon[31172]: from='osd.9 [v2:172.21.15.26:6826/2201297064,v1:172.21.15.26:6827/2201297064]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-11T23:14:43.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:14:43 smithi026 ceph-mon[31172]: osdmap e63: 10 total, 9 up, 10 in 2024-09-11T23:14:43.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:14:43 smithi026 ceph-mon[31172]: from='osd.9 [v2:172.21.15.26:6826/2201297064,v1:172.21.15.26:6827/2201297064]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2024-09-11T23:14:43.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:14:43 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T23:14:43.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:14:43 smithi026 ceph-mon[31172]: Detected new or changed devices on smithi026 2024-09-11T23:14:43.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:14:43 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:14:43.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:14:43 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:14:43.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:14:43 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:14:43.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:14:43 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:14:43.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:14:43 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:14:43.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:14:43 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:14:43.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:14:43 smithi026 ceph-mon[31172]: Adjusting osd_memory_target on smithi026 to 4273M 2024-09-11T23:14:43.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:14:43 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:14:43.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:14:43 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:14:43.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:14:43 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:14:43.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:14:43 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:14:44.414 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:14:44 smithi076 ceph-mon[35296]: from='osd.9 [v2:172.21.15.26:6826/2201297064,v1:172.21.15.26:6827/2201297064]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]': finished 2024-09-11T23:14:44.415 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:14:44 smithi076 ceph-mon[35296]: osdmap e64: 10 total, 9 up, 10 in 2024-09-11T23:14:44.415 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:14:44 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T23:14:44.415 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:14:44 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T23:14:44.415 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:14:44 smithi076 ceph-mon[35296]: from='client.24338 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi076:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:14:44.415 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:14:44 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T23:14:44.415 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:14:44 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T23:14:44.415 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:14:44 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:14:44.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:14:44 smithi136 ceph-mon[35121]: from='osd.9 [v2:172.21.15.26:6826/2201297064,v1:172.21.15.26:6827/2201297064]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]': finished 2024-09-11T23:14:44.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:14:44 smithi136 ceph-mon[35121]: osdmap e64: 10 total, 9 up, 10 in 2024-09-11T23:14:44.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:14:44 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T23:14:44.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:14:44 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T23:14:44.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:14:44 smithi136 ceph-mon[35121]: from='client.24338 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi076:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:14:44.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:14:44 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T23:14:44.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:14:44 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T23:14:44.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:14:44 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:14:44.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:14:44 smithi026 ceph-mon[31172]: from='osd.9 [v2:172.21.15.26:6826/2201297064,v1:172.21.15.26:6827/2201297064]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]': finished 2024-09-11T23:14:44.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:14:44 smithi026 ceph-mon[31172]: osdmap e64: 10 total, 9 up, 10 in 2024-09-11T23:14:44.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:14:44 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T23:14:44.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:14:44 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T23:14:44.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:14:44 smithi026 ceph-mon[31172]: from='client.24338 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi076:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:14:44.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:14:44 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T23:14:44.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:14:44 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T23:14:44.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:14:44 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:14:45.501 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:14:45 smithi026 ceph-mon[31172]: purged_snaps scrub starts 2024-09-11T23:14:45.501 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:14:45 smithi026 ceph-mon[31172]: purged_snaps scrub ok 2024-09-11T23:14:45.501 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:14:45 smithi026 ceph-mon[31172]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-09-11T23:14:45.501 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:14:45 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T23:14:45.501 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:14:45 smithi026 ceph-mon[31172]: osd.9 [v2:172.21.15.26:6826/2201297064,v1:172.21.15.26:6827/2201297064] boot 2024-09-11T23:14:45.501 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:14:45 smithi026 ceph-mon[31172]: osdmap e65: 10 total, 10 up, 10 in 2024-09-11T23:14:45.501 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:14:45 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T23:14:45.553 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:14:45 smithi076 ceph-mon[35296]: purged_snaps scrub starts 2024-09-11T23:14:45.553 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:14:45 smithi076 ceph-mon[35296]: purged_snaps scrub ok 2024-09-11T23:14:45.553 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:14:45 smithi076 ceph-mon[35296]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-09-11T23:14:45.554 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:14:45 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T23:14:45.554 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:14:45 smithi076 ceph-mon[35296]: osd.9 [v2:172.21.15.26:6826/2201297064,v1:172.21.15.26:6827/2201297064] boot 2024-09-11T23:14:45.554 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:14:45 smithi076 ceph-mon[35296]: osdmap e65: 10 total, 10 up, 10 in 2024-09-11T23:14:45.554 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:14:45 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T23:14:45.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:14:45 smithi136 ceph-mon[35121]: purged_snaps scrub starts 2024-09-11T23:14:45.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:14:45 smithi136 ceph-mon[35121]: purged_snaps scrub ok 2024-09-11T23:14:45.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:14:45 smithi136 ceph-mon[35121]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-09-11T23:14:45.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:14:45 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T23:14:45.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:14:45 smithi136 ceph-mon[35121]: osd.9 [v2:172.21.15.26:6826/2201297064,v1:172.21.15.26:6827/2201297064] boot 2024-09-11T23:14:45.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:14:45 smithi136 ceph-mon[35121]: osdmap e65: 10 total, 10 up, 10 in 2024-09-11T23:14:45.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:14:45 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T23:14:46.553 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:14:46 smithi076 ceph-mon[35296]: osdmap e66: 10 total, 10 up, 10 in 2024-09-11T23:14:46.553 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:14:46 smithi076 ceph-mon[35296]: from='client.? 172.21.15.76:0/503320439' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b758d4bb-da62-4c5f-a992-bb6a145d35d9"}]: dispatch 2024-09-11T23:14:46.553 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:14:46 smithi076 ceph-mon[35296]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b758d4bb-da62-4c5f-a992-bb6a145d35d9"}]: dispatch 2024-09-11T23:14:46.553 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:14:46 smithi076 ceph-mon[35296]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b758d4bb-da62-4c5f-a992-bb6a145d35d9"}]': finished 2024-09-11T23:14:46.553 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:14:46 smithi076 ceph-mon[35296]: osdmap e67: 11 total, 10 up, 11 in 2024-09-11T23:14:46.553 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:14:46 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-11T23:14:46.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:14:46 smithi136 ceph-mon[35121]: osdmap e66: 10 total, 10 up, 10 in 2024-09-11T23:14:46.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:14:46 smithi136 ceph-mon[35121]: from='client.? 172.21.15.76:0/503320439' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b758d4bb-da62-4c5f-a992-bb6a145d35d9"}]: dispatch 2024-09-11T23:14:46.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:14:46 smithi136 ceph-mon[35121]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b758d4bb-da62-4c5f-a992-bb6a145d35d9"}]: dispatch 2024-09-11T23:14:46.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:14:46 smithi136 ceph-mon[35121]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b758d4bb-da62-4c5f-a992-bb6a145d35d9"}]': finished 2024-09-11T23:14:46.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:14:46 smithi136 ceph-mon[35121]: osdmap e67: 11 total, 10 up, 11 in 2024-09-11T23:14:46.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:14:46 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-11T23:14:46.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:14:46 smithi026 ceph-mon[31172]: osdmap e66: 10 total, 10 up, 10 in 2024-09-11T23:14:46.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:14:46 smithi026 ceph-mon[31172]: from='client.? 172.21.15.76:0/503320439' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b758d4bb-da62-4c5f-a992-bb6a145d35d9"}]: dispatch 2024-09-11T23:14:46.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:14:46 smithi026 ceph-mon[31172]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b758d4bb-da62-4c5f-a992-bb6a145d35d9"}]: dispatch 2024-09-11T23:14:46.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:14:46 smithi026 ceph-mon[31172]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b758d4bb-da62-4c5f-a992-bb6a145d35d9"}]': finished 2024-09-11T23:14:46.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:14:46 smithi026 ceph-mon[31172]: osdmap e67: 11 total, 10 up, 11 in 2024-09-11T23:14:46.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:14:46 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-11T23:14:47.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:14:47 smithi136 ceph-mon[35121]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-09-11T23:14:47.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:14:47 smithi136 ceph-mon[35121]: from='client.? 172.21.15.76:0/2425866224' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T23:14:47.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:14:47 smithi026 ceph-mon[31172]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-09-11T23:14:47.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:14:47 smithi026 ceph-mon[31172]: from='client.? 172.21.15.76:0/2425866224' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T23:14:47.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:14:47 smithi076 ceph-mon[35296]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-09-11T23:14:47.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:14:47 smithi076 ceph-mon[35296]: from='client.? 172.21.15.76:0/2425866224' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T23:14:49.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:14:49 smithi136 ceph-mon[35121]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-09-11T23:14:49.750 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:14:49 smithi026 ceph-mon[31172]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-09-11T23:14:49.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:14:49 smithi076 ceph-mon[35296]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-09-11T23:14:51.425 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:14:51 smithi076 ceph-mon[35296]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-09-11T23:14:51.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:14:51 smithi136 ceph-mon[35121]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-09-11T23:14:51.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:14:51 smithi026 ceph-mon[31172]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-09-11T23:14:53.553 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:14:53 smithi076 ceph-mon[35296]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-09-11T23:14:53.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:14:53 smithi136 ceph-mon[35121]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-09-11T23:14:53.750 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:14:53 smithi026 ceph-mon[31172]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-09-11T23:14:55.587 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:14:55 smithi076 ceph-mon[35296]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-09-11T23:14:55.587 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:14:55 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-11T23:14:55.587 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:14:55 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:14:55.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:14:55 smithi136 ceph-mon[35121]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-09-11T23:14:55.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:14:55 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-11T23:14:55.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:14:55 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:14:55.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:14:55 smithi026 ceph-mon[31172]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-09-11T23:14:55.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:14:55 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-11T23:14:55.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:14:55 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:14:56.673 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:14:56 smithi076 ceph-mon[35296]: Deploying daemon osd.10 on smithi076 2024-09-11T23:14:56.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:14:56 smithi136 ceph-mon[35121]: Deploying daemon osd.10 on smithi076 2024-09-11T23:14:56.750 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:14:56 smithi026 ceph-mon[31172]: Deploying daemon osd.10 on smithi076 2024-09-11T23:14:57.688 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:14:57 smithi076 ceph-mon[35296]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-09-11T23:14:57.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:14:57 smithi136 ceph-mon[35121]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-09-11T23:14:57.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:14:57 smithi026 ceph-mon[31172]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-09-11T23:14:59.590 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:14:59 smithi076 ceph-mon[35296]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-09-11T23:14:59.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:14:59 smithi136 ceph-mon[35121]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-09-11T23:14:59.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:14:59 smithi026 ceph-mon[31172]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-09-11T23:15:01.584 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:01 smithi076 ceph-mon[35296]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-09-11T23:15:01.584 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:01 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:15:01.584 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:01 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:15:01.584 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:01 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:15:01.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:01 smithi136 ceph-mon[35121]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-09-11T23:15:01.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:01 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:15:01.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:01 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:15:01.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:01 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:15:01.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:01 smithi026 ceph-mon[31172]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-09-11T23:15:01.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:01 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:15:01.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:01 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:15:01.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:01 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:15:03.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:03 smithi136 ceph-mon[35121]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-09-11T23:15:03.750 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:03 smithi026 ceph-mon[31172]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-09-11T23:15:03.752 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:03 smithi076 ceph-mon[35296]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-09-11T23:15:04.380 INFO:teuthology.orchestra.run.smithi076.stdout:Created osd(s) 10 on host 'smithi076' 2024-09-11T23:15:04.380 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:15:04.379+0000 7fc7637fe640 1 -- 172.21.15.76:0/1337097652 <== mgr.14148 v2:172.21.15.26:6800/110833859 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fc774062630 con 0x7fc75c0a2b80 2024-09-11T23:15:04.381 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:15:04.381+0000 7fc778faa640 1 -- 172.21.15.76:0/1337097652 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fc75c0a2b80 msgr2=0x7fc75c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:15:04.382 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:15:04.381+0000 7fc778faa640 1 --2- 172.21.15.76:0/1337097652 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fc75c0a2b80 0x7fc75c0a5040 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fc7640314e0 tx=0x7fc764000bc0 comp rx=0 tx=0).stop 2024-09-11T23:15:04.382 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:15:04.381+0000 7fc778faa640 1 -- 172.21.15.76:0/1337097652 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc774104fd0 msgr2=0x7fc774100650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:15:04.382 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:15:04.381+0000 7fc778faa640 1 --2- 172.21.15.76:0/1337097652 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc774104fd0 0x7fc774100650 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fc76800d740 tx=0x7fc76800dc00 comp rx=0 tx=0).stop 2024-09-11T23:15:04.382 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:15:04.381+0000 7fc778faa640 1 -- 172.21.15.76:0/1337097652 shutdown_connections 2024-09-11T23:15:04.382 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:15:04.381+0000 7fc778faa640 1 --2- 172.21.15.76:0/1337097652 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fc75c0a2b80 0x7fc75c0a5040 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:15:04.382 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:15:04.381+0000 7fc778faa640 1 --2- 172.21.15.76:0/1337097652 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc774104fd0 0x7fc774100650 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:15:04.382 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:15:04.381+0000 7fc778faa640 1 --2- 172.21.15.76:0/1337097652 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc774107900 0x7fc774100b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:15:04.382 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:15:04.381+0000 7fc778faa640 1 --2- 172.21.15.76:0/1337097652 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc7741026a0 0x7fc774102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:15:04.382 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:15:04.381+0000 7fc778faa640 1 -- 172.21.15.76:0/1337097652 >> 172.21.15.76:0/1337097652 conn(0x7fc7740fc2a0 msgr2=0x7fc774102fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:15:04.382 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:15:04.381+0000 7fc778faa640 1 -- 172.21.15.76:0/1337097652 shutdown_connections 2024-09-11T23:15:04.382 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-11T23:15:04.381+0000 7fc778faa640 1 -- 172.21.15.76:0/1337097652 wait complete. 2024-09-11T23:15:04.976 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:04 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:15:04.977 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:04 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:15:04.977 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:04 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:15:04.977 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:04 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:15:04.977 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:04 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:15:04.977 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:04 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:15:04.977 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:04 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:15:04.977 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:04 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:15:04.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:04 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:15:04.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:04 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:15:04.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:04 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:15:04.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:04 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:15:04.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:04 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:15:04.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:04 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:15:04.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:04 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:15:04.990 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:04 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:15:05.000 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:04 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:15:05.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:04 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:15:05.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:04 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:15:05.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:04 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:15:05.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:04 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:15:05.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:04 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:15:05.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:04 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:15:05.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:04 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:15:05.004 DEBUG:teuthology.orchestra.run.smithi076:osd.10> sudo journalctl -f -n 0 -u ceph-af7e6fc8-7092-11ef-bcea-c7b262605968@osd.10.service 2024-09-11T23:15:05.006 INFO:tasks.cephadm:Deploying osd.11 on smithi136 with /dev/vg_nvme/lv_1... 2024-09-11T23:15:05.007 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-11T23:15:05.264 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T23:15:05.934 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:05 smithi076 ceph-mon[35296]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-09-11T23:15:05.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:05 smithi136 ceph-mon[35121]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-09-11T23:15:06.000 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:05 smithi026 ceph-mon[31172]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-09-11T23:15:07.553 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:15:07 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:15:07.209+0000 7fcb79006640 -1 osd.10 0 log_to_monitors true 2024-09-11T23:15:07.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:07 smithi136 ceph-mon[35121]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-09-11T23:15:07.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:07 smithi136 ceph-mon[35121]: from='osd.10 [v2:172.21.15.76:6824/3184128096,v1:172.21.15.76:6825/3184128096]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-11T23:15:07.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:07 smithi136 ceph-mon[35121]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-11T23:15:07.785 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-09-11T23:15:07.812 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi136:vg_nvme/lv_1 2024-09-11T23:15:07.934 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:07 smithi076 ceph-mon[35296]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-09-11T23:15:07.934 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:07 smithi076 ceph-mon[35296]: from='osd.10 [v2:172.21.15.76:6824/3184128096,v1:172.21.15.76:6825/3184128096]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-11T23:15:07.934 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:07 smithi076 ceph-mon[35296]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-11T23:15:08.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:07 smithi026 ceph-mon[31172]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-09-11T23:15:08.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:07 smithi026 ceph-mon[31172]: from='osd.10 [v2:172.21.15.76:6824/3184128096,v1:172.21.15.76:6825/3184128096]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-11T23:15:08.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:07 smithi026 ceph-mon[31172]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-11T23:15:08.065 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T23:15:08.515 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:15:08.513+0000 7f582c61d640 1 -- 172.21.15.136:0/1016756889 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5824076220 msgr2=0x7f5824074680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:15:08.515 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:15:08.513+0000 7f582c61d640 1 --2- 172.21.15.136:0/1016756889 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5824076220 0x7f5824074680 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f5814009900 tx=0x7f581402a420 comp rx=0 tx=0).stop 2024-09-11T23:15:08.515 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:15:08.513+0000 7f582c61d640 1 -- 172.21.15.136:0/1016756889 shutdown_connections 2024-09-11T23:15:08.516 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:15:08.513+0000 7f582c61d640 1 --2- 172.21.15.136:0/1016756889 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5824074c50 0x7f58240750b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:15:08.516 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:15:08.513+0000 7f582c61d640 1 --2- 172.21.15.136:0/1016756889 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f582410e1e0 0x7f58241105d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:15:08.516 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:15:08.513+0000 7f582c61d640 1 --2- 172.21.15.136:0/1016756889 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5824076220 0x7f5824074680 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:15:08.516 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:15:08.513+0000 7f582c61d640 1 -- 172.21.15.136:0/1016756889 >> 172.21.15.136:0/1016756889 conn(0x7f58240fd390 msgr2=0x7f58240ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:15:08.516 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:15:08.513+0000 7f582c61d640 1 -- 172.21.15.136:0/1016756889 shutdown_connections 2024-09-11T23:15:08.516 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:15:08.513+0000 7f582c61d640 1 -- 172.21.15.136:0/1016756889 wait complete. 2024-09-11T23:15:08.516 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:15:08.514+0000 7f582c61d640 1 Processor -- start 2024-09-11T23:15:08.516 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:15:08.514+0000 7f582c61d640 1 -- start start 2024-09-11T23:15:08.516 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:15:08.514+0000 7f582c61d640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5824074c50 0x7f582419e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:15:08.516 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:15:08.514+0000 7f582c61d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5824076220 0x7f582419eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:15:08.516 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:15:08.514+0000 7f582c61d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f582410e1e0 0x7f58241a5b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:15:08.516 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:15:08.514+0000 7f582c61d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f5824113f00 con 0x7f5824076220 2024-09-11T23:15:08.517 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:15:08.514+0000 7f582c61d640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f5824113d80 con 0x7f582410e1e0 2024-09-11T23:15:08.517 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:15:08.514+0000 7f582c61d640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f5824114080 con 0x7f5824074c50 2024-09-11T23:15:08.517 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:15:08.514+0000 7f582a3b9640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5824074c50 0x7f582419e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:15:08.517 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:15:08.514+0000 7f582a3b9640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5824074c50 0x7f582419e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.136:60054/0 (socket says 172.21.15.136:60054) 2024-09-11T23:15:08.517 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:15:08.514+0000 7f582a3b9640 1 -- 172.21.15.136:0/4039015228 learned_addr learned my addr 172.21.15.136:0/4039015228 (peer_addr_for_me v2:172.21.15.136:0/0) 2024-09-11T23:15:08.517 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:15:08.514+0000 7f582abba640 1 --2- 172.21.15.136:0/4039015228 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f582410e1e0 0x7f58241a5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:15:08.517 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:15:08.514+0000 7f5829bb8640 1 --2- 172.21.15.136:0/4039015228 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5824076220 0x7f582419eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:15:08.517 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:15:08.514+0000 7f582a3b9640 1 -- 172.21.15.136:0/4039015228 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f582410e1e0 msgr2=0x7f58241a5b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:15:08.517 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:15:08.514+0000 7f582a3b9640 1 --2- 172.21.15.136:0/4039015228 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f582410e1e0 0x7f58241a5b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:15:08.517 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:15:08.514+0000 7f582a3b9640 1 -- 172.21.15.136:0/4039015228 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5824076220 msgr2=0x7f582419eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:15:08.517 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:15:08.514+0000 7f582a3b9640 1 --2- 172.21.15.136:0/4039015228 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5824076220 0x7f582419eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:15:08.517 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:15:08.514+0000 7f582a3b9640 1 -- 172.21.15.136:0/4039015228 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58241a61b0 con 0x7f5824074c50 2024-09-11T23:15:08.517 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:15:08.514+0000 7f5829bb8640 1 --2- 172.21.15.136:0/4039015228 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5824076220 0x7f582419eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:15:08.518 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:15:08.515+0000 7f582a3b9640 1 --2- 172.21.15.136:0/4039015228 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5824074c50 0x7f582419e580 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f5814006010 tx=0x7f58140043b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:15:08.518 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:15:08.515+0000 7f581b7fe640 1 -- 172.21.15.136:0/4039015228 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f581403f030 con 0x7f5824074c50 2024-09-11T23:15:08.518 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:15:08.515+0000 7f582c61d640 1 -- 172.21.15.136:0/4039015228 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58241a6440 con 0x7f5824074c50 2024-09-11T23:15:08.518 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:15:08.515+0000 7f581b7fe640 1 -- 172.21.15.136:0/4039015228 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5814003ce0 con 0x7f5824074c50 2024-09-11T23:15:08.518 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:15:08.515+0000 7f581b7fe640 1 -- 172.21.15.136:0/4039015228 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5814035070 con 0x7f5824074c50 2024-09-11T23:15:08.518 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:15:08.515+0000 7f582c61d640 1 -- 172.21.15.136:0/4039015228 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58241a69f0 con 0x7f5824074c50 2024-09-11T23:15:08.519 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:15:08.516+0000 7f582c61d640 1 -- 172.21.15.136:0/4039015228 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5824103b80 con 0x7f5824074c50 2024-09-11T23:15:08.519 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:15:08.518+0000 7f581b7fe640 1 -- 172.21.15.136:0/4039015228 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f5814042030 con 0x7f5824074c50 2024-09-11T23:15:08.519 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:15:08.518+0000 7f581b7fe640 1 --2- 172.21.15.136:0/4039015228 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f58100a2bb0 0x7f58100a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:15:08.519 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:15:08.518+0000 7f581b7fe640 1 -- 172.21.15.136:0/4039015228 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 6398+0+0 (secure 0 0 0) 0x7f58140ef1a0 con 0x7f5824074c50 2024-09-11T23:15:08.519 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:15:08.519+0000 7f5829bb8640 1 --2- 172.21.15.136:0/4039015228 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f58100a2bb0 0x7f58100a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:15:08.519 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:15:08.519+0000 7f5829bb8640 1 --2- 172.21.15.136:0/4039015228 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f58100a2bb0 0x7f58100a5070 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f582419faa0 tx=0x7f5820005d30 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:15:08.522 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:15:08.522+0000 7f581b7fe640 1 -- 172.21.15.136:0/4039015228 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f58140bc880 con 0x7f5824074c50 2024-09-11T23:15:08.694 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:15:08.694+0000 7f582c61d640 1 -- 172.21.15.136:0/4039015228 --> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi136:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f5824062630 con 0x7f58100a2bb0 2024-09-11T23:15:08.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:08 smithi136 ceph-mon[35121]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-11T23:15:08.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:08 smithi136 ceph-mon[35121]: osdmap e68: 11 total, 10 up, 11 in 2024-09-11T23:15:08.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:08 smithi136 ceph-mon[35121]: from='osd.10 [v2:172.21.15.76:6824/3184128096,v1:172.21.15.76:6825/3184128096]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]: dispatch 2024-09-11T23:15:08.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:08 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-11T23:15:08.990 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:08 smithi136 ceph-mon[35121]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]: dispatch 2024-09-11T23:15:08.990 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:08 smithi136 ceph-mon[35121]: Detected new or changed devices on smithi076 2024-09-11T23:15:08.990 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:08 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:15:08.990 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:08 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:15:08.990 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:08 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:15:08.990 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:08 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:15:08.990 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:08 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:15:08.990 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:08 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:15:08.990 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:08 smithi136 ceph-mon[35121]: Adjusting osd_memory_target on smithi076 to 4272M 2024-09-11T23:15:08.990 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:08 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:15:08.990 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:08 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:15:08.990 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:08 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:15:08.990 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:08 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:15:08.991 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:08 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T23:15:09.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:08 smithi026 ceph-mon[31172]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-11T23:15:09.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:08 smithi026 ceph-mon[31172]: osdmap e68: 11 total, 10 up, 11 in 2024-09-11T23:15:09.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:08 smithi026 ceph-mon[31172]: from='osd.10 [v2:172.21.15.76:6824/3184128096,v1:172.21.15.76:6825/3184128096]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]: dispatch 2024-09-11T23:15:09.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:08 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-11T23:15:09.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:08 smithi026 ceph-mon[31172]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]: dispatch 2024-09-11T23:15:09.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:08 smithi026 ceph-mon[31172]: Detected new or changed devices on smithi076 2024-09-11T23:15:09.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:08 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:15:09.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:08 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:15:09.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:08 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:15:09.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:08 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:15:09.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:08 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:15:09.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:08 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:15:09.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:08 smithi026 ceph-mon[31172]: Adjusting osd_memory_target on smithi076 to 4272M 2024-09-11T23:15:09.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:08 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:15:09.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:08 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:15:09.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:08 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:15:09.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:08 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:15:09.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:08 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T23:15:09.053 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:15:08 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:15:08.710+0000 7fcb74766640 -1 osd.10 0 waiting for initial osdmap 2024-09-11T23:15:09.053 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:15:08 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:15:08.723+0000 7fcb70da0640 -1 osd.10 69 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T23:15:09.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:08 smithi076 ceph-mon[35296]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-11T23:15:09.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:08 smithi076 ceph-mon[35296]: osdmap e68: 11 total, 10 up, 11 in 2024-09-11T23:15:09.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:08 smithi076 ceph-mon[35296]: from='osd.10 [v2:172.21.15.76:6824/3184128096,v1:172.21.15.76:6825/3184128096]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]: dispatch 2024-09-11T23:15:09.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:08 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-11T23:15:09.054 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:08 smithi076 ceph-mon[35296]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]: dispatch 2024-09-11T23:15:09.054 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:08 smithi076 ceph-mon[35296]: Detected new or changed devices on smithi076 2024-09-11T23:15:09.054 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:08 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:15:09.054 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:08 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:15:09.054 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:08 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:15:09.054 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:08 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:15:09.054 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:08 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:15:09.054 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:08 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:15:09.054 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:08 smithi076 ceph-mon[35296]: Adjusting osd_memory_target on smithi076 to 4272M 2024-09-11T23:15:09.054 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:08 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:15:09.054 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:08 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:15:09.054 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:08 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:15:09.054 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:08 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:15:09.055 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:08 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T23:15:09.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:09 smithi136 ceph-mon[35121]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-09-11T23:15:09.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:09 smithi136 ceph-mon[35121]: from='client.24343 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi136:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:15:09.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:09 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T23:15:09.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:09 smithi136 ceph-mon[35121]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]': finished 2024-09-11T23:15:09.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:09 smithi136 ceph-mon[35121]: osdmap e69: 11 total, 10 up, 11 in 2024-09-11T23:15:09.990 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:09 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:15:09.990 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:09 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-11T23:15:09.990 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:09 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-11T23:15:10.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:09 smithi026 ceph-mon[31172]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-09-11T23:15:10.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:09 smithi026 ceph-mon[31172]: from='client.24343 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi136:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:15:10.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:09 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T23:15:10.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:09 smithi026 ceph-mon[31172]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]': finished 2024-09-11T23:15:10.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:09 smithi026 ceph-mon[31172]: osdmap e69: 11 total, 10 up, 11 in 2024-09-11T23:15:10.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:09 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:15:10.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:09 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-11T23:15:10.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:09 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-11T23:15:10.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:09 smithi076 ceph-mon[35296]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-09-11T23:15:10.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:09 smithi076 ceph-mon[35296]: from='client.24343 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi136:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:15:10.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:09 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T23:15:10.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:09 smithi076 ceph-mon[35296]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]': finished 2024-09-11T23:15:10.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:09 smithi076 ceph-mon[35296]: osdmap e69: 11 total, 10 up, 11 in 2024-09-11T23:15:10.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:09 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:15:10.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:09 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-11T23:15:10.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:09 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-11T23:15:10.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:10 smithi136 ceph-mon[35121]: purged_snaps scrub starts 2024-09-11T23:15:10.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:10 smithi136 ceph-mon[35121]: purged_snaps scrub ok 2024-09-11T23:15:10.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:10 smithi136 ceph-mon[35121]: osd.10 [v2:172.21.15.76:6824/3184128096,v1:172.21.15.76:6825/3184128096] boot 2024-09-11T23:15:10.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:10 smithi136 ceph-mon[35121]: osdmap e70: 11 total, 11 up, 11 in 2024-09-11T23:15:10.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:10 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-11T23:15:10.990 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:10 smithi136 ceph-mon[35121]: osdmap e71: 11 total, 11 up, 11 in 2024-09-11T23:15:11.000 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:10 smithi026 ceph-mon[31172]: purged_snaps scrub starts 2024-09-11T23:15:11.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:10 smithi026 ceph-mon[31172]: purged_snaps scrub ok 2024-09-11T23:15:11.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:10 smithi026 ceph-mon[31172]: osd.10 [v2:172.21.15.76:6824/3184128096,v1:172.21.15.76:6825/3184128096] boot 2024-09-11T23:15:11.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:10 smithi026 ceph-mon[31172]: osdmap e70: 11 total, 11 up, 11 in 2024-09-11T23:15:11.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:10 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-11T23:15:11.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:10 smithi026 ceph-mon[31172]: osdmap e71: 11 total, 11 up, 11 in 2024-09-11T23:15:11.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:10 smithi076 ceph-mon[35296]: purged_snaps scrub starts 2024-09-11T23:15:11.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:10 smithi076 ceph-mon[35296]: purged_snaps scrub ok 2024-09-11T23:15:11.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:10 smithi076 ceph-mon[35296]: osd.10 [v2:172.21.15.76:6824/3184128096,v1:172.21.15.76:6825/3184128096] boot 2024-09-11T23:15:11.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:10 smithi076 ceph-mon[35296]: osdmap e70: 11 total, 11 up, 11 in 2024-09-11T23:15:11.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:10 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-11T23:15:11.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:10 smithi076 ceph-mon[35296]: osdmap e71: 11 total, 11 up, 11 in 2024-09-11T23:15:11.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:11 smithi136 ceph-mon[35121]: pgmap v228: 1 pgs: 1 peering; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-09-11T23:15:11.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:11 smithi136 ceph-mon[35121]: from='client.? 172.21.15.136:0/1604306932' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8fddda46-e22b-4d0c-ac48-e038fb29dfec"}]: dispatch 2024-09-11T23:15:11.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:11 smithi136 ceph-mon[35121]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8fddda46-e22b-4d0c-ac48-e038fb29dfec"}]: dispatch 2024-09-11T23:15:11.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:11 smithi136 ceph-mon[35121]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8fddda46-e22b-4d0c-ac48-e038fb29dfec"}]': finished 2024-09-11T23:15:11.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:11 smithi136 ceph-mon[35121]: osdmap e72: 12 total, 11 up, 12 in 2024-09-11T23:15:11.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:11 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T23:15:11.990 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:11 smithi136 ceph-mon[35121]: from='client.? 172.21.15.136:0/2225148268' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T23:15:12.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:11 smithi026 ceph-mon[31172]: pgmap v228: 1 pgs: 1 peering; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-09-11T23:15:12.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:11 smithi026 ceph-mon[31172]: from='client.? 172.21.15.136:0/1604306932' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8fddda46-e22b-4d0c-ac48-e038fb29dfec"}]: dispatch 2024-09-11T23:15:12.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:11 smithi026 ceph-mon[31172]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8fddda46-e22b-4d0c-ac48-e038fb29dfec"}]: dispatch 2024-09-11T23:15:12.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:11 smithi026 ceph-mon[31172]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8fddda46-e22b-4d0c-ac48-e038fb29dfec"}]': finished 2024-09-11T23:15:12.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:11 smithi026 ceph-mon[31172]: osdmap e72: 12 total, 11 up, 12 in 2024-09-11T23:15:12.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:11 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T23:15:12.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:11 smithi026 ceph-mon[31172]: from='client.? 172.21.15.136:0/2225148268' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T23:15:12.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:11 smithi076 ceph-mon[35296]: pgmap v228: 1 pgs: 1 peering; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-09-11T23:15:12.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:11 smithi076 ceph-mon[35296]: from='client.? 172.21.15.136:0/1604306932' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8fddda46-e22b-4d0c-ac48-e038fb29dfec"}]: dispatch 2024-09-11T23:15:12.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:11 smithi076 ceph-mon[35296]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8fddda46-e22b-4d0c-ac48-e038fb29dfec"}]: dispatch 2024-09-11T23:15:12.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:11 smithi076 ceph-mon[35296]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8fddda46-e22b-4d0c-ac48-e038fb29dfec"}]': finished 2024-09-11T23:15:12.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:11 smithi076 ceph-mon[35296]: osdmap e72: 12 total, 11 up, 12 in 2024-09-11T23:15:12.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:11 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T23:15:12.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:11 smithi076 ceph-mon[35296]: from='client.? 172.21.15.136:0/2225148268' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T23:15:13.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:13 smithi136 ceph-mon[35121]: pgmap v231: 1 pgs: 1 peering; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-09-11T23:15:13.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:13 smithi136 ceph-mon[35121]: osdmap e73: 12 total, 11 up, 12 in 2024-09-11T23:15:13.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:13 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T23:15:13.501 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:13 smithi026 ceph-mon[31172]: pgmap v231: 1 pgs: 1 peering; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-09-11T23:15:13.501 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:13 smithi026 ceph-mon[31172]: osdmap e73: 12 total, 11 up, 12 in 2024-09-11T23:15:13.501 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:13 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T23:15:13.553 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:13 smithi076 ceph-mon[35296]: pgmap v231: 1 pgs: 1 peering; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-09-11T23:15:13.553 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:13 smithi076 ceph-mon[35296]: osdmap e73: 12 total, 11 up, 12 in 2024-09-11T23:15:13.553 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:13 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T23:15:14.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:14 smithi136 ceph-mon[35121]: osdmap e74: 12 total, 11 up, 12 in 2024-09-11T23:15:14.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:14 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T23:15:14.501 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:14 smithi026 ceph-mon[31172]: osdmap e74: 12 total, 11 up, 12 in 2024-09-11T23:15:14.501 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:14 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T23:15:14.553 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:14 smithi076 ceph-mon[35296]: osdmap e74: 12 total, 11 up, 12 in 2024-09-11T23:15:14.553 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:14 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T23:15:15.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:15 smithi136 ceph-mon[35121]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-11T23:15:15.501 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:15 smithi026 ceph-mon[31172]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-11T23:15:15.552 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:15 smithi076 ceph-mon[35296]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-11T23:15:17.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:17 smithi136 ceph-mon[35121]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 104 KiB/s, 0 objects/s recovering 2024-09-11T23:15:17.649 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:17 smithi076 ceph-mon[35296]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 104 KiB/s, 0 objects/s recovering 2024-09-11T23:15:17.649 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:17 smithi026 ceph-mon[31172]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 104 KiB/s, 0 objects/s recovering 2024-09-11T23:15:19.228 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:19 smithi136 ceph-mon[35121]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-09-11T23:15:19.501 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:19 smithi026 ceph-mon[31172]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-09-11T23:15:19.553 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:19 smithi076 ceph-mon[35296]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-09-11T23:15:20.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:20 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-11T23:15:20.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:20 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:15:20.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:20 smithi136 ceph-mon[35121]: Deploying daemon osd.11 on smithi136 2024-09-11T23:15:20.501 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:20 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-11T23:15:20.501 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:20 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:15:20.501 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:20 smithi026 ceph-mon[31172]: Deploying daemon osd.11 on smithi136 2024-09-11T23:15:20.553 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:20 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-11T23:15:20.553 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:20 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:15:20.553 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:20 smithi076 ceph-mon[35296]: Deploying daemon osd.11 on smithi136 2024-09-11T23:15:21.440 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:21 smithi136 ceph-mon[35121]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-11T23:15:21.501 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:21 smithi026 ceph-mon[31172]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-11T23:15:21.553 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:21 smithi076 ceph-mon[35296]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-11T23:15:23.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:23 smithi136 ceph-mon[35121]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-11T23:15:23.501 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:23 smithi026 ceph-mon[31172]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-11T23:15:23.553 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:23 smithi076 ceph-mon[35296]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-11T23:15:25.471 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:25 smithi136 ceph-mon[35121]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 53 KiB/s, 0 objects/s recovering 2024-09-11T23:15:25.501 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:25 smithi026 ceph-mon[31172]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 53 KiB/s, 0 objects/s recovering 2024-09-11T23:15:25.553 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:25 smithi076 ceph-mon[35296]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 53 KiB/s, 0 objects/s recovering 2024-09-11T23:15:26.421 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:26 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:15:26.421 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:26 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:15:26.421 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:26 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:15:26.501 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:26 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:15:26.501 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:26 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:15:26.501 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:26 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:15:26.553 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:26 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:15:26.553 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:26 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:15:26.553 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:26 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:15:27.878 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:27 smithi136 ceph-mon[35121]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-09-11T23:15:28.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:27 smithi076 ceph-mon[35296]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-09-11T23:15:28.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:27 smithi026 ceph-mon[31172]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-09-11T23:15:29.269 INFO:teuthology.orchestra.run.smithi136.stdout:Created osd(s) 11 on host 'smithi136' 2024-09-11T23:15:29.270 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:15:29.269+0000 7f581b7fe640 1 -- 172.21.15.136:0/4039015228 <== mgr.14148 v2:172.21.15.26:6800/110833859 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f5824062630 con 0x7f58100a2bb0 2024-09-11T23:15:29.272 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:15:29.272+0000 7f582c61d640 1 -- 172.21.15.136:0/4039015228 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f58100a2bb0 msgr2=0x7f58100a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:15:29.272 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:15:29.272+0000 7f582c61d640 1 --2- 172.21.15.136:0/4039015228 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f58100a2bb0 0x7f58100a5070 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f582419faa0 tx=0x7f5820005d30 comp rx=0 tx=0).stop 2024-09-11T23:15:29.272 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:15:29.272+0000 7f582c61d640 1 -- 172.21.15.136:0/4039015228 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5824074c50 msgr2=0x7f582419e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:15:29.272 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:15:29.272+0000 7f582c61d640 1 --2- 172.21.15.136:0/4039015228 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5824074c50 0x7f582419e580 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f5814006010 tx=0x7f58140043b0 comp rx=0 tx=0).stop 2024-09-11T23:15:29.272 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:15:29.272+0000 7f582c61d640 1 -- 172.21.15.136:0/4039015228 shutdown_connections 2024-09-11T23:15:29.272 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:15:29.272+0000 7f582c61d640 1 --2- 172.21.15.136:0/4039015228 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f58100a2bb0 0x7f58100a5070 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:15:29.272 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:15:29.272+0000 7f582c61d640 1 --2- 172.21.15.136:0/4039015228 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f582410e1e0 0x7f58241a5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:15:29.273 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:15:29.272+0000 7f582c61d640 1 --2- 172.21.15.136:0/4039015228 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5824076220 0x7f582419eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:15:29.273 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:15:29.272+0000 7f582c61d640 1 --2- 172.21.15.136:0/4039015228 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5824074c50 0x7f582419e580 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:15:29.273 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:15:29.272+0000 7f582c61d640 1 -- 172.21.15.136:0/4039015228 >> 172.21.15.136:0/4039015228 conn(0x7f58240fd390 msgr2=0x7f58240ff780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:15:29.273 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:15:29.272+0000 7f582c61d640 1 -- 172.21.15.136:0/4039015228 shutdown_connections 2024-09-11T23:15:29.273 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:15:29.272+0000 7f582c61d640 1 -- 172.21.15.136:0/4039015228 wait complete. 2024-09-11T23:15:29.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:29 smithi076 ceph-mon[35296]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-09-11T23:15:29.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:29 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:15:29.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:29 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:15:29.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:29 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:15:29.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:29 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:15:29.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:29 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:15:29.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:29 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:15:29.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:29 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:15:29.804 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:29 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:15:29.949 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:29 smithi136 ceph-mon[35121]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-09-11T23:15:29.949 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:29 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:15:29.949 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:29 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:15:29.949 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:29 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:15:29.949 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:29 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:15:29.949 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:29 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:15:29.949 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:29 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:15:29.949 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:29 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:15:29.950 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:29 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:15:29.975 DEBUG:teuthology.orchestra.run.smithi136:osd.11> sudo journalctl -f -n 0 -u ceph-af7e6fc8-7092-11ef-bcea-c7b262605968@osd.11.service 2024-09-11T23:15:29.978 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-09-11T23:15:29.978 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph osd stat -f json 2024-09-11T23:15:30.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:29 smithi026 ceph-mon[31172]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-09-11T23:15:30.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:29 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:15:30.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:29 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:15:30.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:29 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:15:30.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:29 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:15:30.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:29 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:15:30.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:29 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:15:30.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:29 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:15:30.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:29 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:15:30.251 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:15:30.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:30.719+0000 7fe442544640 1 -- 172.21.15.26:0/899783122 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe43c107980 msgr2=0x7fe43c109d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:15:30.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:30.719+0000 7fe442544640 1 --2- 172.21.15.26:0/899783122 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe43c107980 0x7fe43c109d70 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fe434009ca0 tx=0x7fe43402a8c0 comp rx=0 tx=0).stop 2024-09-11T23:15:30.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:30.719+0000 7fe442544640 1 -- 172.21.15.26:0/899783122 shutdown_connections 2024-09-11T23:15:30.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:30.719+0000 7fe442544640 1 --2- 172.21.15.26:0/899783122 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe43c105050 0x7fe43c107440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:15:30.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:30.719+0000 7fe442544640 1 --2- 172.21.15.26:0/899783122 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe43c107980 0x7fe43c109d70 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:15:30.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:30.719+0000 7fe442544640 1 --2- 172.21.15.26:0/899783122 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe43c102720 0x7fe43c104b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:15:30.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:30.719+0000 7fe442544640 1 -- 172.21.15.26:0/899783122 >> 172.21.15.26:0/899783122 conn(0x7fe43c0fc280 msgr2=0x7fe43c0fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:15:30.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:30.719+0000 7fe442544640 1 -- 172.21.15.26:0/899783122 shutdown_connections 2024-09-11T23:15:30.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:30.719+0000 7fe442544640 1 -- 172.21.15.26:0/899783122 wait complete. 2024-09-11T23:15:30.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:30.719+0000 7fe442544640 1 Processor -- start 2024-09-11T23:15:30.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:30.720+0000 7fe442544640 1 -- start start 2024-09-11T23:15:30.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:30.720+0000 7fe442544640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe43c102720 0x7fe43c197eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:15:30.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:30.720+0000 7fe442544640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe43c105050 0x7fe43c1983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:15:30.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:30.720+0000 7fe442544640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe43c107980 0x7fe43c19f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:15:30.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:30.720+0000 7fe442544640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fe43c10d8e0 con 0x7fe43c102720 2024-09-11T23:15:30.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:30.720+0000 7fe442544640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fe43c10d760 con 0x7fe43c107980 2024-09-11T23:15:30.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:30.720+0000 7fe442544640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe43c10da60 con 0x7fe43c105050 2024-09-11T23:15:30.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:30.720+0000 7fe43bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe43c102720 0x7fe43c197eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:15:30.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:30.720+0000 7fe43bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe43c102720 0x7fe43c197eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52568/0 (socket says 172.21.15.26:52568) 2024-09-11T23:15:30.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:30.720+0000 7fe43bfff640 1 -- 172.21.15.26:0/2632789998 learned_addr learned my addr 172.21.15.26:0/2632789998 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:15:30.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:30.720+0000 7fe43b7fe640 1 --2- 172.21.15.26:0/2632789998 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe43c105050 0x7fe43c1983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:15:30.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:30.720+0000 7fe440ae1640 1 --2- 172.21.15.26:0/2632789998 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe43c107980 0x7fe43c19f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:15:30.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:30.720+0000 7fe43bfff640 1 -- 172.21.15.26:0/2632789998 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe43c105050 msgr2=0x7fe43c1983f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:15:30.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:30.720+0000 7fe43bfff640 1 --2- 172.21.15.26:0/2632789998 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe43c105050 0x7fe43c1983f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:15:30.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:30.720+0000 7fe43bfff640 1 -- 172.21.15.26:0/2632789998 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe43c107980 msgr2=0x7fe43c19f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:15:30.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:30.720+0000 7fe43bfff640 1 --2- 172.21.15.26:0/2632789998 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe43c107980 0x7fe43c19f470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:15:30.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:30.720+0000 7fe43bfff640 1 -- 172.21.15.26:0/2632789998 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe43c19fae0 con 0x7fe43c102720 2024-09-11T23:15:30.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:30.720+0000 7fe43bfff640 1 --2- 172.21.15.26:0/2632789998 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe43c102720 0x7fe43c197eb0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fe42c00c920 tx=0x7fe42c00cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:15:30.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:30.721+0000 7fe4397fa640 1 -- 172.21.15.26:0/2632789998 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe42c007bb0 con 0x7fe43c102720 2024-09-11T23:15:30.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:30.721+0000 7fe442544640 1 -- 172.21.15.26:0/2632789998 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe43c19fdd0 con 0x7fe43c102720 2024-09-11T23:15:30.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:30.721+0000 7fe4397fa640 1 -- 172.21.15.26:0/2632789998 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe42c007d50 con 0x7fe43c102720 2024-09-11T23:15:30.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:30.721+0000 7fe4397fa640 1 -- 172.21.15.26:0/2632789998 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe42c018780 con 0x7fe43c102720 2024-09-11T23:15:30.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:30.721+0000 7fe442544640 1 -- 172.21.15.26:0/2632789998 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe43c1a02b0 con 0x7fe43c102720 2024-09-11T23:15:30.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:30.722+0000 7fe442544640 1 -- 172.21.15.26:0/2632789998 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe43c0fdd70 con 0x7fe43c102720 2024-09-11T23:15:30.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:30.723+0000 7fe4397fa640 1 -- 172.21.15.26:0/2632789998 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fe42c00f450 con 0x7fe43c102720 2024-09-11T23:15:30.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:30.724+0000 7fe4397fa640 1 --2- 172.21.15.26:0/2632789998 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fe4240a2bb0 0x7fe4240a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:15:30.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:30.724+0000 7fe43b7fe640 1 --2- 172.21.15.26:0/2632789998 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fe4240a2bb0 0x7fe4240a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:15:30.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:30.724+0000 7fe4397fa640 1 -- 172.21.15.26:0/2632789998 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 6809+0+0 (secure 0 0 0) 0x7fe42c0cede0 con 0x7fe43c102720 2024-09-11T23:15:30.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:30.724+0000 7fe43b7fe640 1 --2- 172.21.15.26:0/2632789998 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fe4240a2bb0 0x7fe4240a5070 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fe43c1993d0 tx=0x7fe430009110 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:15:30.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:30.727+0000 7fe4397fa640 1 -- 172.21.15.26:0/2632789998 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe42c014030 con 0x7fe43c102720 2024-09-11T23:15:30.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:30.904+0000 7fe442544640 1 -- 172.21.15.26:0/2632789998 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7fe43c02da30 con 0x7fe43c102720 2024-09-11T23:15:30.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:30.905+0000 7fe4397fa640 1 -- 172.21.15.26:0/2632789998 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v74) ==== 74+0+133 (secure 0 0 0) 0x7fe42c09c390 con 0x7fe43c102720 2024-09-11T23:15:30.905 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-11T23:15:30.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:30.907+0000 7fe442544640 1 -- 172.21.15.26:0/2632789998 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fe4240a2bb0 msgr2=0x7fe4240a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:15:30.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:30.907+0000 7fe442544640 1 --2- 172.21.15.26:0/2632789998 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fe4240a2bb0 0x7fe4240a5070 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fe43c1993d0 tx=0x7fe430009110 comp rx=0 tx=0).stop 2024-09-11T23:15:30.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:30.907+0000 7fe442544640 1 -- 172.21.15.26:0/2632789998 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe43c102720 msgr2=0x7fe43c197eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:15:30.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:30.907+0000 7fe442544640 1 --2- 172.21.15.26:0/2632789998 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe43c102720 0x7fe43c197eb0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fe42c00c920 tx=0x7fe42c00cd20 comp rx=0 tx=0).stop 2024-09-11T23:15:30.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:30.907+0000 7fe442544640 1 -- 172.21.15.26:0/2632789998 shutdown_connections 2024-09-11T23:15:30.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:30.907+0000 7fe442544640 1 --2- 172.21.15.26:0/2632789998 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fe4240a2bb0 0x7fe4240a5070 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:15:30.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:30.907+0000 7fe442544640 1 --2- 172.21.15.26:0/2632789998 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe43c107980 0x7fe43c19f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:15:30.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:30.907+0000 7fe442544640 1 --2- 172.21.15.26:0/2632789998 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe43c105050 0x7fe43c1983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:15:30.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:30.907+0000 7fe442544640 1 --2- 172.21.15.26:0/2632789998 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe43c102720 0x7fe43c197eb0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:15:30.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:30.907+0000 7fe442544640 1 -- 172.21.15.26:0/2632789998 >> 172.21.15.26:0/2632789998 conn(0x7fe43c0fc280 msgr2=0x7fe43c102fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:15:30.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:30.907+0000 7fe442544640 1 -- 172.21.15.26:0/2632789998 shutdown_connections 2024-09-11T23:15:30.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:30.907+0000 7fe442544640 1 -- 172.21.15.26:0/2632789998 wait complete. 2024-09-11T23:15:31.723 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:31 smithi026 ceph-mon[31172]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-09-11T23:15:31.723 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:31 smithi026 ceph-mon[31172]: from='client.? 172.21.15.26:0/2632789998' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-11T23:15:31.761 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":74,"num_osds":12,"num_up_osds":11,"osd_up_since":1726096509,"num_in_osds":12,"osd_in_since":1726096511,"num_remapped_pgs":0} 2024-09-11T23:15:31.802 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:31 smithi076 ceph-mon[35296]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-09-11T23:15:31.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:31 smithi076 ceph-mon[35296]: from='client.? 172.21.15.26:0/2632789998' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-11T23:15:31.849 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:31 smithi136 ceph-mon[35121]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-09-11T23:15:31.849 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:31 smithi136 ceph-mon[35121]: from='client.? 172.21.15.26:0/2632789998' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-11T23:15:32.216 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:15:31 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:15:31.848+0000 7f20e989a640 -1 osd.11 0 log_to_monitors true 2024-09-11T23:15:32.762 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph osd stat -f json 2024-09-11T23:15:32.795 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:32 smithi026 ceph-mon[31172]: from='osd.11 [v2:172.21.15.136:6824/1960689685,v1:172.21.15.136:6825/1960689685]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-11T23:15:32.796 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:32 smithi026 ceph-mon[31172]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-11T23:15:32.802 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:32 smithi076 ceph-mon[35296]: from='osd.11 [v2:172.21.15.136:6824/1960689685,v1:172.21.15.136:6825/1960689685]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-11T23:15:32.802 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:32 smithi076 ceph-mon[35296]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-11T23:15:32.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:32 smithi136 ceph-mon[35121]: from='osd.11 [v2:172.21.15.136:6824/1960689685,v1:172.21.15.136:6825/1960689685]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-11T23:15:32.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:32 smithi136 ceph-mon[35121]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-11T23:15:33.038 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:15:33.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:33.535+0000 7fb72bac6640 1 -- 172.21.15.26:0/2179894790 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb724102690 msgr2=0x7fb724108f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:15:33.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:33.535+0000 7fb72bac6640 1 --2- 172.21.15.26:0/2179894790 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb724102690 0x7fb724108f20 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fb71c009ca0 tx=0x7fb71c02a8c0 comp rx=0 tx=0).stop 2024-09-11T23:15:33.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:33.535+0000 7fb72bac6640 1 -- 172.21.15.26:0/2179894790 shutdown_connections 2024-09-11T23:15:33.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:33.535+0000 7fb72bac6640 1 --2- 172.21.15.26:0/2179894790 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb724102690 0x7fb724108f20 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:15:33.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:33.535+0000 7fb72bac6640 1 --2- 172.21.15.26:0/2179894790 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb724101cd0 0x7fb724102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:15:33.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:33.535+0000 7fb72bac6640 1 --2- 172.21.15.26:0/2179894790 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb724100ad0 0x7fb724100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:15:33.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:33.535+0000 7fb72bac6640 1 -- 172.21.15.26:0/2179894790 >> 172.21.15.26:0/2179894790 conn(0x7fb7240fc280 msgr2=0x7fb7240fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:15:33.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:33.535+0000 7fb72bac6640 1 -- 172.21.15.26:0/2179894790 shutdown_connections 2024-09-11T23:15:33.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:33.535+0000 7fb72bac6640 1 -- 172.21.15.26:0/2179894790 wait complete. 2024-09-11T23:15:33.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:33.536+0000 7fb72bac6640 1 Processor -- start 2024-09-11T23:15:33.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:33.536+0000 7fb72bac6640 1 -- start start 2024-09-11T23:15:33.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:33.536+0000 7fb72bac6640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb724100ad0 0x7fb72406aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:15:33.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:33.536+0000 7fb72bac6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb724101cd0 0x7fb72419cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:15:33.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:33.536+0000 7fb72bac6640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb724102690 0x7fb72419f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:15:33.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:33.536+0000 7fb72bac6640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb72410c750 con 0x7fb724101cd0 2024-09-11T23:15:33.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:33.536+0000 7fb72bac6640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fb72410c5d0 con 0x7fb724102690 2024-09-11T23:15:33.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:33.536+0000 7fb72bac6640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb72410c8d0 con 0x7fb724100ad0 2024-09-11T23:15:33.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:33.536+0000 7fb729061640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb724101cd0 0x7fb72419cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:15:33.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:33.536+0000 7fb729862640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb724100ad0 0x7fb72406aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:15:33.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:33.536+0000 7fb729061640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb724101cd0 0x7fb72419cbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52578/0 (socket says 172.21.15.26:52578) 2024-09-11T23:15:33.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:33.536+0000 7fb729061640 1 -- 172.21.15.26:0/3897993062 learned_addr learned my addr 172.21.15.26:0/3897993062 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:15:33.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:33.537+0000 7fb72a063640 1 --2- 172.21.15.26:0/3897993062 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb724102690 0x7fb72419f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:15:33.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:33.537+0000 7fb729061640 1 -- 172.21.15.26:0/3897993062 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb724100ad0 msgr2=0x7fb72406aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:15:33.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:33.537+0000 7fb729061640 1 --2- 172.21.15.26:0/3897993062 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb724100ad0 0x7fb72406aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:15:33.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:33.537+0000 7fb729061640 1 -- 172.21.15.26:0/3897993062 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb724102690 msgr2=0x7fb72419f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:15:33.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:33.537+0000 7fb729061640 1 --2- 172.21.15.26:0/3897993062 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb724102690 0x7fb72419f1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:15:33.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:33.537+0000 7fb729061640 1 -- 172.21.15.26:0/3897993062 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb72419fb00 con 0x7fb724101cd0 2024-09-11T23:15:33.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:33.537+0000 7fb729061640 1 --2- 172.21.15.26:0/3897993062 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb724101cd0 0x7fb72419cbd0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fb71400d770 tx=0x7fb71400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:15:33.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:33.537+0000 7fb71affd640 1 -- 172.21.15.26:0/3897993062 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb714017070 con 0x7fb724101cd0 2024-09-11T23:15:33.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:33.537+0000 7fb71affd640 1 -- 172.21.15.26:0/3897993062 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb714005500 con 0x7fb724101cd0 2024-09-11T23:15:33.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:33.537+0000 7fb72bac6640 1 -- 172.21.15.26:0/3897993062 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb72419fd90 con 0x7fb724101cd0 2024-09-11T23:15:33.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:33.537+0000 7fb71affd640 1 -- 172.21.15.26:0/3897993062 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb714004e00 con 0x7fb724101cd0 2024-09-11T23:15:33.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:33.537+0000 7fb72bac6640 1 -- 172.21.15.26:0/3897993062 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7241a0270 con 0x7fb724101cd0 2024-09-11T23:15:33.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:33.538+0000 7fb72bac6640 1 -- 172.21.15.26:0/3897993062 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb724107e50 con 0x7fb724101cd0 2024-09-11T23:15:33.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:33.540+0000 7fb71affd640 1 -- 172.21.15.26:0/3897993062 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fb714004fa0 con 0x7fb724101cd0 2024-09-11T23:15:33.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:33.541+0000 7fb71affd640 1 --2- 172.21.15.26:0/3897993062 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fb7100a2c00 0x7fb7100a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:15:33.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:33.541+0000 7fb729862640 1 --2- 172.21.15.26:0/3897993062 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fb7100a2c00 0x7fb7100a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:15:33.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:33.541+0000 7fb71affd640 1 -- 172.21.15.26:0/3897993062 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 6847+0+0 (secure 0 0 0) 0x7fb7140ce8d0 con 0x7fb724101cd0 2024-09-11T23:15:33.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:33.541+0000 7fb729862640 1 --2- 172.21.15.26:0/3897993062 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fb7100a2c00 0x7fb7100a50c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fb724101b30 tx=0x7fb720005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:15:33.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:33.544+0000 7fb71affd640 1 -- 172.21.15.26:0/3897993062 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb71409bdd0 con 0x7fb724101cd0 2024-09-11T23:15:33.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:33.723+0000 7fb72bac6640 1 -- 172.21.15.26:0/3897993062 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7fb72402da10 con 0x7fb724101cd0 2024-09-11T23:15:33.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:33.723+0000 7fb71affd640 1 -- 172.21.15.26:0/3897993062 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v76) ==== 74+0+133 (secure 0 0 0) 0x7fb71409b6d0 con 0x7fb724101cd0 2024-09-11T23:15:33.724 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-11T23:15:33.724 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:33 smithi026 ceph-mon[31172]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-09-11T23:15:33.725 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:33 smithi026 ceph-mon[31172]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-11T23:15:33.725 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:33 smithi026 ceph-mon[31172]: osdmap e75: 12 total, 11 up, 12 in 2024-09-11T23:15:33.725 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:33 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T23:15:33.725 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:33 smithi026 ceph-mon[31172]: from='osd.11 [v2:172.21.15.136:6824/1960689685,v1:172.21.15.136:6825/1960689685]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-09-11T23:15:33.725 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:33 smithi026 ceph-mon[31172]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-09-11T23:15:33.725 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:33 smithi026 ceph-mon[31172]: Detected new or changed devices on smithi136 2024-09-11T23:15:33.725 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:33 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:15:33.725 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:33 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:15:33.725 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:33 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:15:33.726 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:33 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:15:33.726 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:33 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:15:33.726 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:33 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:15:33.726 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:33 smithi026 ceph-mon[31172]: Adjusting osd_memory_target on smithi136 to 4272M 2024-09-11T23:15:33.726 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:33 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:15:33.726 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:33 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:15:33.726 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:33 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:15:33.726 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:33 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:15:33.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:33.726+0000 7fb72bac6640 1 -- 172.21.15.26:0/3897993062 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fb7100a2c00 msgr2=0x7fb7100a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:15:33.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:33.726+0000 7fb72bac6640 1 --2- 172.21.15.26:0/3897993062 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fb7100a2c00 0x7fb7100a50c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fb724101b30 tx=0x7fb720005f90 comp rx=0 tx=0).stop 2024-09-11T23:15:33.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:33.726+0000 7fb72bac6640 1 -- 172.21.15.26:0/3897993062 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb724101cd0 msgr2=0x7fb72419cbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:15:33.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:33.726+0000 7fb72bac6640 1 --2- 172.21.15.26:0/3897993062 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb724101cd0 0x7fb72419cbd0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fb71400d770 tx=0x7fb71400dc30 comp rx=0 tx=0).stop 2024-09-11T23:15:33.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:33.726+0000 7fb72bac6640 1 -- 172.21.15.26:0/3897993062 shutdown_connections 2024-09-11T23:15:33.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:33.726+0000 7fb72bac6640 1 --2- 172.21.15.26:0/3897993062 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fb7100a2c00 0x7fb7100a50c0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:15:33.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:33.726+0000 7fb72bac6640 1 --2- 172.21.15.26:0/3897993062 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb724102690 0x7fb72419f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:15:33.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:33.726+0000 7fb72bac6640 1 --2- 172.21.15.26:0/3897993062 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb724101cd0 0x7fb72419cbd0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:15:33.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:33.726+0000 7fb72bac6640 1 --2- 172.21.15.26:0/3897993062 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb724100ad0 0x7fb72406aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:15:33.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:33.726+0000 7fb72bac6640 1 -- 172.21.15.26:0/3897993062 >> 172.21.15.26:0/3897993062 conn(0x7fb7240fc280 msgr2=0x7fb724105020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:15:33.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:33.726+0000 7fb72bac6640 1 -- 172.21.15.26:0/3897993062 shutdown_connections 2024-09-11T23:15:33.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:33.726+0000 7fb72bac6640 1 -- 172.21.15.26:0/3897993062 wait complete. 2024-09-11T23:15:33.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:33 smithi076 ceph-mon[35296]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-09-11T23:15:33.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:33 smithi076 ceph-mon[35296]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-11T23:15:33.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:33 smithi076 ceph-mon[35296]: osdmap e75: 12 total, 11 up, 12 in 2024-09-11T23:15:33.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:33 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T23:15:33.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:33 smithi076 ceph-mon[35296]: from='osd.11 [v2:172.21.15.136:6824/1960689685,v1:172.21.15.136:6825/1960689685]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-09-11T23:15:33.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:33 smithi076 ceph-mon[35296]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-09-11T23:15:33.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:33 smithi076 ceph-mon[35296]: Detected new or changed devices on smithi136 2024-09-11T23:15:33.804 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:33 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:15:33.804 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:33 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:15:33.804 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:33 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:15:33.804 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:33 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:15:33.804 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:33 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:15:33.804 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:33 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:15:33.804 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:33 smithi076 ceph-mon[35296]: Adjusting osd_memory_target on smithi136 to 4272M 2024-09-11T23:15:33.804 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:33 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:15:33.804 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:33 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:15:33.804 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:33 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:15:33.804 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:33 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:15:33.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:33 smithi136 ceph-mon[35121]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-09-11T23:15:33.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:33 smithi136 ceph-mon[35121]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-11T23:15:33.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:33 smithi136 ceph-mon[35121]: osdmap e75: 12 total, 11 up, 12 in 2024-09-11T23:15:33.990 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:33 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T23:15:33.990 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:33 smithi136 ceph-mon[35121]: from='osd.11 [v2:172.21.15.136:6824/1960689685,v1:172.21.15.136:6825/1960689685]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-09-11T23:15:33.990 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:33 smithi136 ceph-mon[35121]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-09-11T23:15:33.990 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:33 smithi136 ceph-mon[35121]: Detected new or changed devices on smithi136 2024-09-11T23:15:33.990 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:33 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:15:33.990 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:33 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:15:33.990 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:33 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:15:33.990 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:33 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:15:33.990 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:33 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:15:33.991 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:33 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:15:33.991 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:33 smithi136 ceph-mon[35121]: Adjusting osd_memory_target on smithi136 to 4272M 2024-09-11T23:15:33.991 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:33 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:15:33.991 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:33 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:15:33.991 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:33 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:15:33.991 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:33 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:15:34.489 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:15:34 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:15:34.177+0000 7f20e4ffa640 -1 osd.11 0 waiting for initial osdmap 2024-09-11T23:15:34.489 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:15:34 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:15:34.192+0000 7f20dd62c640 -1 osd.11 76 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T23:15:34.505 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":76,"num_osds":12,"num_up_osds":11,"osd_up_since":1726096509,"num_in_osds":12,"osd_in_since":1726096511,"num_remapped_pgs":0} 2024-09-11T23:15:34.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:34 smithi026 ceph-mon[31172]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]': finished 2024-09-11T23:15:34.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:34 smithi026 ceph-mon[31172]: osdmap e76: 12 total, 11 up, 12 in 2024-09-11T23:15:34.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:34 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T23:15:34.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:34 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T23:15:34.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:34 smithi026 ceph-mon[31172]: from='client.? 172.21.15.26:0/3897993062' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-11T23:15:34.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:34 smithi026 ceph-mon[31172]: from='osd.11 ' entity='osd.11' 2024-09-11T23:15:34.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:34 smithi076 ceph-mon[35296]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]': finished 2024-09-11T23:15:34.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:34 smithi076 ceph-mon[35296]: osdmap e76: 12 total, 11 up, 12 in 2024-09-11T23:15:34.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:34 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T23:15:34.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:34 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T23:15:34.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:34 smithi076 ceph-mon[35296]: from='client.? 172.21.15.26:0/3897993062' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-11T23:15:34.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:34 smithi076 ceph-mon[35296]: from='osd.11 ' entity='osd.11' 2024-09-11T23:15:34.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:34 smithi136 ceph-mon[35121]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]': finished 2024-09-11T23:15:34.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:34 smithi136 ceph-mon[35121]: osdmap e76: 12 total, 11 up, 12 in 2024-09-11T23:15:34.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:34 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T23:15:34.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:34 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T23:15:34.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:34 smithi136 ceph-mon[35121]: from='client.? 172.21.15.26:0/3897993062' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-11T23:15:34.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:34 smithi136 ceph-mon[35121]: from='osd.11 ' entity='osd.11' 2024-09-11T23:15:35.506 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph osd stat -f json 2024-09-11T23:15:35.783 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:15:35.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:35 smithi076 ceph-mon[35296]: purged_snaps scrub starts 2024-09-11T23:15:35.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:35 smithi076 ceph-mon[35296]: purged_snaps scrub ok 2024-09-11T23:15:35.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:35 smithi076 ceph-mon[35296]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-09-11T23:15:35.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:35 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T23:15:35.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:35 smithi076 ceph-mon[35296]: osd.11 [v2:172.21.15.136:6824/1960689685,v1:172.21.15.136:6825/1960689685] boot 2024-09-11T23:15:35.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:35 smithi076 ceph-mon[35296]: osdmap e77: 12 total, 12 up, 12 in 2024-09-11T23:15:35.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:35 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T23:15:35.905 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:35 smithi026 ceph-mon[31172]: purged_snaps scrub starts 2024-09-11T23:15:35.905 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:35 smithi026 ceph-mon[31172]: purged_snaps scrub ok 2024-09-11T23:15:35.905 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:35 smithi026 ceph-mon[31172]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-09-11T23:15:35.905 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:35 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T23:15:35.905 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:35 smithi026 ceph-mon[31172]: osd.11 [v2:172.21.15.136:6824/1960689685,v1:172.21.15.136:6825/1960689685] boot 2024-09-11T23:15:35.905 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:35 smithi026 ceph-mon[31172]: osdmap e77: 12 total, 12 up, 12 in 2024-09-11T23:15:35.905 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:35 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T23:15:35.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:35 smithi136 ceph-mon[35121]: purged_snaps scrub starts 2024-09-11T23:15:35.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:35 smithi136 ceph-mon[35121]: purged_snaps scrub ok 2024-09-11T23:15:35.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:35 smithi136 ceph-mon[35121]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-09-11T23:15:35.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:35 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T23:15:35.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:35 smithi136 ceph-mon[35121]: osd.11 [v2:172.21.15.136:6824/1960689685,v1:172.21.15.136:6825/1960689685] boot 2024-09-11T23:15:35.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:35 smithi136 ceph-mon[35121]: osdmap e77: 12 total, 12 up, 12 in 2024-09-11T23:15:35.990 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:35 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T23:15:36.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:36.244+0000 7f4b8dd8e640 1 -- 172.21.15.26:0/368446191 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4b88101cd0 msgr2=0x7f4b88102150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:15:36.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:36.244+0000 7f4b8dd8e640 1 --2- 172.21.15.26:0/368446191 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4b88101cd0 0x7f4b88102150 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f4b7c009900 tx=0x7f4b7c02a410 comp rx=0 tx=0).stop 2024-09-11T23:15:36.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:36.244+0000 7f4b8dd8e640 1 -- 172.21.15.26:0/368446191 shutdown_connections 2024-09-11T23:15:36.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:36.244+0000 7f4b8dd8e640 1 --2- 172.21.15.26:0/368446191 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4b88102690 0x7f4b88108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:15:36.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:36.244+0000 7f4b8dd8e640 1 --2- 172.21.15.26:0/368446191 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4b88101cd0 0x7f4b88102150 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:15:36.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:36.244+0000 7f4b8dd8e640 1 --2- 172.21.15.26:0/368446191 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4b88100ad0 0x7f4b88100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:15:36.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:36.244+0000 7f4b8dd8e640 1 -- 172.21.15.26:0/368446191 >> 172.21.15.26:0/368446191 conn(0x7f4b880fc280 msgr2=0x7f4b880fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:15:36.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:36.244+0000 7f4b8dd8e640 1 -- 172.21.15.26:0/368446191 shutdown_connections 2024-09-11T23:15:36.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:36.244+0000 7f4b8dd8e640 1 -- 172.21.15.26:0/368446191 wait complete. 2024-09-11T23:15:36.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:36.245+0000 7f4b8dd8e640 1 Processor -- start 2024-09-11T23:15:36.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:36.245+0000 7f4b8dd8e640 1 -- start start 2024-09-11T23:15:36.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:36.245+0000 7f4b8dd8e640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4b88100ad0 0x7f4b88197e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:15:36.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:36.245+0000 7f4b8dd8e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4b88101cd0 0x7f4b881983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:15:36.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:36.245+0000 7f4b8dd8e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4b88102690 0x7f4b8819f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:15:36.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:36.245+0000 7f4b8dd8e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f4b8810d880 con 0x7f4b88101cd0 2024-09-11T23:15:36.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:36.245+0000 7f4b8dd8e640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f4b8810d700 con 0x7f4b88102690 2024-09-11T23:15:36.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:36.245+0000 7f4b8dd8e640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f4b8810da00 con 0x7f4b88100ad0 2024-09-11T23:15:36.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:36.245+0000 7f4b86ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4b88101cd0 0x7f4b881983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:15:36.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:36.245+0000 7f4b86ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4b88101cd0 0x7f4b881983d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52588/0 (socket says 172.21.15.26:52588) 2024-09-11T23:15:36.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:36.245+0000 7f4b86ffd640 1 -- 172.21.15.26:0/2231704876 learned_addr learned my addr 172.21.15.26:0/2231704876 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:15:36.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:36.245+0000 7f4b877fe640 1 --2- 172.21.15.26:0/2231704876 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4b88100ad0 0x7f4b88197e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:15:36.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:36.245+0000 7f4b87fff640 1 --2- 172.21.15.26:0/2231704876 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4b88102690 0x7f4b8819f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:15:36.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:36.246+0000 7f4b86ffd640 1 -- 172.21.15.26:0/2231704876 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4b88100ad0 msgr2=0x7f4b88197e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:15:36.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:36.246+0000 7f4b86ffd640 1 --2- 172.21.15.26:0/2231704876 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4b88100ad0 0x7f4b88197e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:15:36.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:36.246+0000 7f4b86ffd640 1 -- 172.21.15.26:0/2231704876 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4b88102690 msgr2=0x7f4b8819f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:15:36.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:36.246+0000 7f4b86ffd640 1 --2- 172.21.15.26:0/2231704876 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4b88102690 0x7f4b8819f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:15:36.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:36.246+0000 7f4b86ffd640 1 -- 172.21.15.26:0/2231704876 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b8819fac0 con 0x7f4b88101cd0 2024-09-11T23:15:36.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:36.246+0000 7f4b86ffd640 1 --2- 172.21.15.26:0/2231704876 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4b88101cd0 0x7f4b881983d0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f4b7c009230 tx=0x7f4b7c004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:15:36.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:36.246+0000 7f4b84ff9640 1 -- 172.21.15.26:0/2231704876 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b7c03f030 con 0x7f4b88101cd0 2024-09-11T23:15:36.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:36.246+0000 7f4b84ff9640 1 -- 172.21.15.26:0/2231704876 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b7c003ce0 con 0x7f4b88101cd0 2024-09-11T23:15:36.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:36.246+0000 7f4b8dd8e640 1 -- 172.21.15.26:0/2231704876 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b8819fd50 con 0x7f4b88101cd0 2024-09-11T23:15:36.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:36.246+0000 7f4b84ff9640 1 -- 172.21.15.26:0/2231704876 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b7c035070 con 0x7f4b88101cd0 2024-09-11T23:15:36.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:36.246+0000 7f4b8dd8e640 1 -- 172.21.15.26:0/2231704876 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b881a01d0 con 0x7f4b88101cd0 2024-09-11T23:15:36.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:36.247+0000 7f4b8dd8e640 1 -- 172.21.15.26:0/2231704876 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b88107ed0 con 0x7f4b88101cd0 2024-09-11T23:15:36.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:36.248+0000 7f4b84ff9640 1 -- 172.21.15.26:0/2231704876 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f4b7c042030 con 0x7f4b88101cd0 2024-09-11T23:15:36.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:36.249+0000 7f4b84ff9640 1 --2- 172.21.15.26:0/2231704876 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f4b700a2bb0 0x7f4b700a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:15:36.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:36.249+0000 7f4b877fe640 1 --2- 172.21.15.26:0/2231704876 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f4b700a2bb0 0x7f4b700a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:15:36.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:36.250+0000 7f4b84ff9640 1 -- 172.21.15.26:0/2231704876 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7f4b7c0ef420 con 0x7f4b88101cd0 2024-09-11T23:15:36.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:36.250+0000 7f4b877fe640 1 --2- 172.21.15.26:0/2231704876 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f4b700a2bb0 0x7f4b700a5070 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f4b78005c40 tx=0x7f4b78005bd0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:15:36.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:36.253+0000 7f4b84ff9640 1 -- 172.21.15.26:0/2231704876 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4b7c0bc780 con 0x7f4b88101cd0 2024-09-11T23:15:36.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:36.455+0000 7f4b8dd8e640 1 -- 172.21.15.26:0/2231704876 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f4b88062df0 con 0x7f4b88101cd0 2024-09-11T23:15:36.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:36.455+0000 7f4b84ff9640 1 -- 172.21.15.26:0/2231704876 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v77) ==== 74+0+133 (secure 0 0 0) 0x7f4b7c0bc080 con 0x7f4b88101cd0 2024-09-11T23:15:36.456 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-11T23:15:36.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:36.457+0000 7f4b8dd8e640 1 -- 172.21.15.26:0/2231704876 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f4b700a2bb0 msgr2=0x7f4b700a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:15:36.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:36.457+0000 7f4b8dd8e640 1 --2- 172.21.15.26:0/2231704876 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f4b700a2bb0 0x7f4b700a5070 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f4b78005c40 tx=0x7f4b78005bd0 comp rx=0 tx=0).stop 2024-09-11T23:15:36.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:36.457+0000 7f4b8dd8e640 1 -- 172.21.15.26:0/2231704876 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4b88101cd0 msgr2=0x7f4b881983d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:15:36.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:36.457+0000 7f4b8dd8e640 1 --2- 172.21.15.26:0/2231704876 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4b88101cd0 0x7f4b881983d0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f4b7c009230 tx=0x7f4b7c004430 comp rx=0 tx=0).stop 2024-09-11T23:15:36.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:36.457+0000 7f4b8dd8e640 1 -- 172.21.15.26:0/2231704876 shutdown_connections 2024-09-11T23:15:36.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:36.457+0000 7f4b8dd8e640 1 --2- 172.21.15.26:0/2231704876 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f4b700a2bb0 0x7f4b700a5070 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:15:36.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:36.457+0000 7f4b8dd8e640 1 --2- 172.21.15.26:0/2231704876 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4b88102690 0x7f4b8819f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:15:36.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:36.457+0000 7f4b8dd8e640 1 --2- 172.21.15.26:0/2231704876 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4b88101cd0 0x7f4b881983d0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:15:36.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:36.457+0000 7f4b8dd8e640 1 --2- 172.21.15.26:0/2231704876 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4b88100ad0 0x7f4b88197e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:15:36.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:36.457+0000 7f4b8dd8e640 1 -- 172.21.15.26:0/2231704876 >> 172.21.15.26:0/2231704876 conn(0x7f4b880fc280 msgr2=0x7f4b881050a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:15:36.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:36.457+0000 7f4b8dd8e640 1 -- 172.21.15.26:0/2231704876 shutdown_connections 2024-09-11T23:15:36.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:36.457+0000 7f4b8dd8e640 1 -- 172.21.15.26:0/2231704876 wait complete. 2024-09-11T23:15:36.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:36 smithi026 ceph-mon[31172]: from='client.? 172.21.15.26:0/2231704876' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-11T23:15:36.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:36 smithi136 ceph-mon[35121]: from='client.? 172.21.15.26:0/2231704876' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-11T23:15:37.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:36 smithi076 ceph-mon[35296]: from='client.? 172.21.15.26:0/2231704876' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-11T23:15:37.266 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":77,"num_osds":12,"num_up_osds":12,"osd_up_since":1726096535,"num_in_osds":12,"osd_in_since":1726096511,"num_remapped_pgs":0} 2024-09-11T23:15:37.267 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-11T23:15:37.523 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:15:37.900 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:37 smithi026 ceph-mon[31172]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:15:37.901 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:37 smithi026 ceph-mon[31172]: osdmap e78: 12 total, 12 up, 12 in 2024-09-11T23:15:37.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:37.983+0000 7fccc4a98640 1 -- 172.21.15.26:0/1254997226 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fccc0102ef0 msgr2=0x7fccc010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:15:37.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:37.983+0000 7fccc4a98640 1 --2- 172.21.15.26:0/1254997226 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fccc0102ef0 0x7fccc010da40 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fccb8009ca0 tx=0x7fccb802a950 comp rx=0 tx=0).stop 2024-09-11T23:15:37.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:37.983+0000 7fccc4a98640 1 -- 172.21.15.26:0/1254997226 shutdown_connections 2024-09-11T23:15:37.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:37.983+0000 7fccc4a98640 1 --2- 172.21.15.26:0/1254997226 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fccc0102ef0 0x7fccc010da40 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:15:37.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:37.983+0000 7fccc4a98640 1 --2- 172.21.15.26:0/1254997226 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fccc0102570 0x7fccc01029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:15:37.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:37.983+0000 7fccc4a98640 1 --2- 172.21.15.26:0/1254997226 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fccc0100e20 0x7fccc0101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:15:37.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:37.983+0000 7fccc4a98640 1 -- 172.21.15.26:0/1254997226 >> 172.21.15.26:0/1254997226 conn(0x7fccc00fc7d0 msgr2=0x7fccc00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:15:37.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:37.983+0000 7fccc4a98640 1 -- 172.21.15.26:0/1254997226 shutdown_connections 2024-09-11T23:15:37.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:37.983+0000 7fccc4a98640 1 -- 172.21.15.26:0/1254997226 wait complete. 2024-09-11T23:15:37.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:37.984+0000 7fccc4a98640 1 Processor -- start 2024-09-11T23:15:37.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:37.984+0000 7fccc4a98640 1 -- start start 2024-09-11T23:15:37.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:37.984+0000 7fccc4a98640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fccc0100e20 0x7fccc019a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:15:37.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:37.984+0000 7fccc4a98640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fccc0102570 0x7fccc019ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:15:37.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:37.984+0000 7fccc4a98640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fccc0102ef0 0x7fccc019efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:15:37.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:37.984+0000 7fccc4a98640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fccc0110fa0 con 0x7fccc0102ef0 2024-09-11T23:15:37.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:37.984+0000 7fccc4a98640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fccc0110e20 con 0x7fccc0102570 2024-09-11T23:15:37.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:37.984+0000 7fccc4a98640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fccc0111120 con 0x7fccc0100e20 2024-09-11T23:15:37.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:37.984+0000 7fccbed9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fccc0102ef0 0x7fccc019efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:15:37.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:37.984+0000 7fccbed9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fccc0102ef0 0x7fccc019efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52614/0 (socket says 172.21.15.26:52614) 2024-09-11T23:15:37.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:37.984+0000 7fccbed9d640 1 -- 172.21.15.26:0/1036662833 learned_addr learned my addr 172.21.15.26:0/1036662833 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:15:37.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:37.984+0000 7fccbdd9b640 1 --2- 172.21.15.26:0/1036662833 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fccc0102570 0x7fccc019ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:15:37.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:37.984+0000 7fccbe59c640 1 --2- 172.21.15.26:0/1036662833 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fccc0100e20 0x7fccc019a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:15:37.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:37.984+0000 7fccbed9d640 1 -- 172.21.15.26:0/1036662833 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fccc0100e20 msgr2=0x7fccc019a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:15:37.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:37.984+0000 7fccbed9d640 1 --2- 172.21.15.26:0/1036662833 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fccc0100e20 0x7fccc019a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:15:37.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:37.984+0000 7fccbed9d640 1 -- 172.21.15.26:0/1036662833 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fccc0102570 msgr2=0x7fccc019ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:15:37.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:37.984+0000 7fccbed9d640 1 --2- 172.21.15.26:0/1036662833 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fccc0102570 0x7fccc019ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:15:37.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:37.984+0000 7fccbed9d640 1 -- 172.21.15.26:0/1036662833 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccc019f680 con 0x7fccc0102ef0 2024-09-11T23:15:37.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:37.985+0000 7fccbed9d640 1 --2- 172.21.15.26:0/1036662833 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fccc0102ef0 0x7fccc019efa0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fccb802ae70 tx=0x7fccb802f5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:15:37.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:37.985+0000 7fccb77fe640 1 -- 172.21.15.26:0/1036662833 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccb8034040 con 0x7fccc0102ef0 2024-09-11T23:15:37.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:37.985+0000 7fccb77fe640 1 -- 172.21.15.26:0/1036662833 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fccb8030920 con 0x7fccc0102ef0 2024-09-11T23:15:37.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:37.985+0000 7fccc4a98640 1 -- 172.21.15.26:0/1036662833 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccc019f910 con 0x7fccc0102ef0 2024-09-11T23:15:37.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:37.985+0000 7fccb77fe640 1 -- 172.21.15.26:0/1036662833 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccb80047f0 con 0x7fccc0102ef0 2024-09-11T23:15:37.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:37.985+0000 7fccc4a98640 1 -- 172.21.15.26:0/1036662833 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fccc01a7250 con 0x7fccc0102ef0 2024-09-11T23:15:37.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:37 smithi136 ceph-mon[35121]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:15:37.990 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:37 smithi136 ceph-mon[35121]: osdmap e78: 12 total, 12 up, 12 in 2024-09-11T23:15:37.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:37.986+0000 7fccc4a98640 1 -- 172.21.15.26:0/1036662833 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fccc0101200 con 0x7fccc0102ef0 2024-09-11T23:15:37.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:37.987+0000 7fccb77fe640 1 -- 172.21.15.26:0/1036662833 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fccb8037020 con 0x7fccc0102ef0 2024-09-11T23:15:37.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:37.988+0000 7fccb77fe640 1 --2- 172.21.15.26:0/1036662833 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fccac0a2c00 0x7fccac0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:15:37.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:37.988+0000 7fccbe59c640 1 --2- 172.21.15.26:0/1036662833 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fccac0a2c00 0x7fccac0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:15:37.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:37.988+0000 7fccb77fe640 1 -- 172.21.15.26:0/1036662833 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7127+0+0 (secure 0 0 0) 0x7fccb80ef9c0 con 0x7fccc0102ef0 2024-09-11T23:15:37.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:37.988+0000 7fccbe59c640 1 --2- 172.21.15.26:0/1036662833 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fccac0a2c00 0x7fccac0a50c0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fccb0009990 tx=0x7fccb0009110 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:15:37.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:37.992+0000 7fccb77fe640 1 -- 172.21.15.26:0/1036662833 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fccb80f1e20 con 0x7fccc0102ef0 2024-09-11T23:15:38.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:37 smithi076 ceph-mon[35296]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:15:38.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:37 smithi076 ceph-mon[35296]: osdmap e78: 12 total, 12 up, 12 in 2024-09-11T23:15:38.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:38.170+0000 7fccc4a98640 1 -- 172.21.15.26:0/1036662833 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fccc01a7480 con 0x7fccc0102ef0 2024-09-11T23:15:38.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:38.171+0000 7fccb77fe640 1 -- 172.21.15.26:0/1036662833 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) ==== 74+0+18441 (secure 0 0 0) 0x7fccb80bcdc0 con 0x7fccc0102ef0 2024-09-11T23:15:38.171 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-11T23:15:38.171 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":78,"fsid":"af7e6fc8-7092-11ef-bcea-c7b262605968","created":"2024-09-11T23:08:40.486438+0000","modified":"2024-09-11T23:15:36.548470+0000","last_up_change":"2024-09-11T23:15:35.179446+0000","last_in_change":"2024-09-11T23:15:11.064627+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T23:11:50.494160+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a4d62fcf-c31e-41b1-932c-3d87ff640531","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":2014857905},{"type":"v1","addr":"172.21.15.26:6803","nonce":2014857905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":2014857905},{"type":"v1","addr":"172.21.15.26:6805","nonce":2014857905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":2014857905},{"type":"v1","addr":"172.21.15.26:6809","nonce":2014857905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":2014857905},{"type":"v1","addr":"172.21.15.26:6807","nonce":2014857905}]},"public_addr":"172.21.15.26:6803/2014857905","cluster_addr":"172.21.15.26:6805/2014857905","heartbeat_back_addr":"172.21.15.26:6809/2014857905","heartbeat_front_addr":"172.21.15.26:6807/2014857905","state":["exists","up"]},{"osd":1,"uuid":"e0d69112-3c0a-4fc5-8783-e41b4081e0c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":333383087},{"type":"v1","addr":"172.21.15.76:6801","nonce":333383087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":333383087},{"type":"v1","addr":"172.21.15.76:6803","nonce":333383087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":333383087},{"type":"v1","addr":"172.21.15.76:6807","nonce":333383087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":333383087},{"type":"v1","addr":"172.21.15.76:6805","nonce":333383087}]},"public_addr":"172.21.15.76:6801/333383087","cluster_addr":"172.21.15.76:6803/333383087","heartbeat_back_addr":"172.21.15.76:6807/333383087","heartbeat_front_addr":"172.21.15.76:6805/333383087","state":["exists","up"]},{"osd":2,"uuid":"d578429e-bf23-4a1e-890f-7172955c0eb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":1271538911},{"type":"v1","addr":"172.21.15.136:6801","nonce":1271538911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":1271538911},{"type":"v1","addr":"172.21.15.136:6803","nonce":1271538911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":1271538911},{"type":"v1","addr":"172.21.15.136:6807","nonce":1271538911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":1271538911},{"type":"v1","addr":"172.21.15.136:6805","nonce":1271538911}]},"public_addr":"172.21.15.136:6801/1271538911","cluster_addr":"172.21.15.136:6803/1271538911","heartbeat_back_addr":"172.21.15.136:6807/1271538911","heartbeat_front_addr":"172.21.15.136:6805/1271538911","state":["exists","up"]},{"osd":3,"uuid":"3e34623f-0ea5-4b10-b8a9-f8756c35d512","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":1740304151},{"type":"v1","addr":"172.21.15.26:6811","nonce":1740304151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":1740304151},{"type":"v1","addr":"172.21.15.26:6813","nonce":1740304151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1740304151},{"type":"v1","addr":"172.21.15.26:6817","nonce":1740304151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":1740304151},{"type":"v1","addr":"172.21.15.26:6815","nonce":1740304151}]},"public_addr":"172.21.15.26:6811/1740304151","cluster_addr":"172.21.15.26:6813/1740304151","heartbeat_back_addr":"172.21.15.26:6817/1740304151","heartbeat_front_addr":"172.21.15.26:6815/1740304151","state":["exists","up"]},{"osd":4,"uuid":"77607929-f9ed-49a9-abb5-48cd88eec4ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":2702262182},{"type":"v1","addr":"172.21.15.76:6809","nonce":2702262182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":2702262182},{"type":"v1","addr":"172.21.15.76:6811","nonce":2702262182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":2702262182},{"type":"v1","addr":"172.21.15.76:6815","nonce":2702262182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":2702262182},{"type":"v1","addr":"172.21.15.76:6813","nonce":2702262182}]},"public_addr":"172.21.15.76:6809/2702262182","cluster_addr":"172.21.15.76:6811/2702262182","heartbeat_back_addr":"172.21.15.76:6815/2702262182","heartbeat_front_addr":"172.21.15.76:6813/2702262182","state":["exists","up"]},{"osd":5,"uuid":"29e20e37-09a0-4924-8aaa-17939763a7c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":1749229277},{"type":"v1","addr":"172.21.15.136:6809","nonce":1749229277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":1749229277},{"type":"v1","addr":"172.21.15.136:6811","nonce":1749229277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":1749229277},{"type":"v1","addr":"172.21.15.136:6815","nonce":1749229277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":1749229277},{"type":"v1","addr":"172.21.15.136:6813","nonce":1749229277}]},"public_addr":"172.21.15.136:6809/1749229277","cluster_addr":"172.21.15.136:6811/1749229277","heartbeat_back_addr":"172.21.15.136:6815/1749229277","heartbeat_front_addr":"172.21.15.136:6813/1749229277","state":["exists","up"]},{"osd":6,"uuid":"e0ddd1c0-1c46-41e6-b2f4-396bdcd2bf8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1928245624},{"type":"v1","addr":"172.21.15.26:6819","nonce":1928245624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1928245624},{"type":"v1","addr":"172.21.15.26:6821","nonce":1928245624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":1928245624},{"type":"v1","addr":"172.21.15.26:6825","nonce":1928245624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1928245624},{"type":"v1","addr":"172.21.15.26:6823","nonce":1928245624}]},"public_addr":"172.21.15.26:6819/1928245624","cluster_addr":"172.21.15.26:6821/1928245624","heartbeat_back_addr":"172.21.15.26:6825/1928245624","heartbeat_front_addr":"172.21.15.26:6823/1928245624","state":["exists","up"]},{"osd":7,"uuid":"fe031f5c-d915-48fb-91a6-c76f252467fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":1409789455},{"type":"v1","addr":"172.21.15.76:6817","nonce":1409789455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":1409789455},{"type":"v1","addr":"172.21.15.76:6819","nonce":1409789455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":1409789455},{"type":"v1","addr":"172.21.15.76:6823","nonce":1409789455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":1409789455},{"type":"v1","addr":"172.21.15.76:6821","nonce":1409789455}]},"public_addr":"172.21.15.76:6817/1409789455","cluster_addr":"172.21.15.76:6819/1409789455","heartbeat_back_addr":"172.21.15.76:6823/1409789455","heartbeat_front_addr":"172.21.15.76:6821/1409789455","state":["exists","up"]},{"osd":8,"uuid":"18851bde-9e6c-4e05-b4bf-ac5c3073eee7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":543619079},{"type":"v1","addr":"172.21.15.136:6817","nonce":543619079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":543619079},{"type":"v1","addr":"172.21.15.136:6819","nonce":543619079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":543619079},{"type":"v1","addr":"172.21.15.136:6823","nonce":543619079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":543619079},{"type":"v1","addr":"172.21.15.136:6821","nonce":543619079}]},"public_addr":"172.21.15.136:6817/543619079","cluster_addr":"172.21.15.136:6819/543619079","heartbeat_back_addr":"172.21.15.136:6823/543619079","heartbeat_front_addr":"172.21.15.136:6821/543619079","state":["exists","up"]},{"osd":9,"uuid":"52a65eab-8446-47ad-bfe7-5133f7e3eb98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":2201297064},{"type":"v1","addr":"172.21.15.26:6827","nonce":2201297064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":2201297064},{"type":"v1","addr":"172.21.15.26:6829","nonce":2201297064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":2201297064},{"type":"v1","addr":"172.21.15.26:6833","nonce":2201297064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":2201297064},{"type":"v1","addr":"172.21.15.26:6831","nonce":2201297064}]},"public_addr":"172.21.15.26:6827/2201297064","cluster_addr":"172.21.15.26:6829/2201297064","heartbeat_back_addr":"172.21.15.26:6833/2201297064","heartbeat_front_addr":"172.21.15.26:6831/2201297064","state":["exists","up"]},{"osd":10,"uuid":"b758d4bb-da62-4c5f-a992-bb6a145d35d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":3184128096},{"type":"v1","addr":"172.21.15.76:6825","nonce":3184128096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":3184128096},{"type":"v1","addr":"172.21.15.76:6827","nonce":3184128096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":3184128096},{"type":"v1","addr":"172.21.15.76:6831","nonce":3184128096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":3184128096},{"type":"v1","addr":"172.21.15.76:6829","nonce":3184128096}]},"public_addr":"172.21.15.76:6825/3184128096","cluster_addr":"172.21.15.76:6827/3184128096","heartbeat_back_addr":"172.21.15.76:6831/3184128096","heartbeat_front_addr":"172.21.15.76:6829/3184128096","state":["exists","up"]},{"osd":11,"uuid":"8fddda46-e22b-4d0c-ac48-e038fb29dfec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":1960689685},{"type":"v1","addr":"172.21.15.136:6825","nonce":1960689685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":1960689685},{"type":"v1","addr":"172.21.15.136:6827","nonce":1960689685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":1960689685},{"type":"v1","addr":"172.21.15.136:6831","nonce":1960689685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":1960689685},{"type":"v1","addr":"172.21.15.136:6829","nonce":1960689685}]},"public_addr":"172.21.15.136:6825/1960689685","cluster_addr":"172.21.15.136:6827/1960689685","heartbeat_back_addr":"172.21.15.136:6831/1960689685","heartbeat_front_addr":"172.21.15.136:6829/1960689685","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:10:56.656177+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:11:22.234031+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:11:46.464102+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:12:12.353967+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:12:37.836200+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:13:01.728839+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:13:27.527230+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:13:51.981393+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:14:17.035464+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:14:42.805029+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:15:08.208999+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:15:32.800844+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.26:6801/677148151":"2024-09-12T23:09:33.983310+0000","172.21.15.26:6800/677148151":"2024-09-12T23:09:33.983310+0000","172.21.15.26:0/3597996003":"2024-09-12T23:09:33.983310+0000","172.21.15.26:6801/2673575521":"2024-09-12T23:09:09.781665+0000","172.21.15.26:0/883212708":"2024-09-12T23:09:33.983310+0000","172.21.15.26:0/2107281413":"2024-09-12T23:09:33.983310+0000","172.21.15.26:6800/2673575521":"2024-09-12T23:09:09.781665+0000","172.21.15.26:0/2817092653":"2024-09-12T23:09:09.781665+0000","172.21.15.26:0/17142658":"2024-09-12T23:09:09.781665+0000","172.21.15.26:0/3861103130":"2024-09-12T23:09:09.781665+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T23:15:38.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:38.173+0000 7fccc4a98640 1 -- 172.21.15.26:0/1036662833 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fccac0a2c00 msgr2=0x7fccac0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:15:38.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:38.173+0000 7fccc4a98640 1 --2- 172.21.15.26:0/1036662833 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fccac0a2c00 0x7fccac0a50c0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fccb0009990 tx=0x7fccb0009110 comp rx=0 tx=0).stop 2024-09-11T23:15:38.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:38.173+0000 7fccc4a98640 1 -- 172.21.15.26:0/1036662833 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fccc0102ef0 msgr2=0x7fccc019efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:15:38.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:38.173+0000 7fccc4a98640 1 --2- 172.21.15.26:0/1036662833 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fccc0102ef0 0x7fccc019efa0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fccb802ae70 tx=0x7fccb802f5d0 comp rx=0 tx=0).stop 2024-09-11T23:15:38.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:38.173+0000 7fccc4a98640 1 -- 172.21.15.26:0/1036662833 shutdown_connections 2024-09-11T23:15:38.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:38.173+0000 7fccc4a98640 1 --2- 172.21.15.26:0/1036662833 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fccac0a2c00 0x7fccac0a50c0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:15:38.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:38.173+0000 7fccc4a98640 1 --2- 172.21.15.26:0/1036662833 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fccc0102570 0x7fccc019ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:15:38.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:38.173+0000 7fccc4a98640 1 --2- 172.21.15.26:0/1036662833 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fccc0102ef0 0x7fccc019efa0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:15:38.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:38.173+0000 7fccc4a98640 1 --2- 172.21.15.26:0/1036662833 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fccc0100e20 0x7fccc019a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:15:38.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:38.173+0000 7fccc4a98640 1 -- 172.21.15.26:0/1036662833 >> 172.21.15.26:0/1036662833 conn(0x7fccc00fc7d0 msgr2=0x7fccc006a470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:15:38.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:38.173+0000 7fccc4a98640 1 -- 172.21.15.26:0/1036662833 shutdown_connections 2024-09-11T23:15:38.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:38.173+0000 7fccc4a98640 1 -- 172.21.15.26:0/1036662833 wait complete. 2024-09-11T23:15:38.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:38 smithi026 ceph-mon[31172]: from='client.? 172.21.15.26:0/1036662833' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T23:15:38.812 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-11T23:11:50.494160+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-11T23:15:38.813 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-09-11T23:15:38.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:38 smithi136 ceph-mon[35121]: from='client.? 172.21.15.26:0/1036662833' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T23:15:39.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:38 smithi076 ceph-mon[35296]: from='client.? 172.21.15.26:0/1036662833' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T23:15:39.069 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:15:39.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:39.542+0000 7f1035cdd640 1 -- 172.21.15.26:0/2933932059 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1030106a80 msgr2=0x7f1030106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:15:39.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:39.542+0000 7f1035cdd640 1 --2- 172.21.15.26:0/2933932059 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1030106a80 0x7f1030106e60 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f1020009930 tx=0x7f102002a420 comp rx=0 tx=0).stop 2024-09-11T23:15:39.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:39.542+0000 7f1035cdd640 1 -- 172.21.15.26:0/2933932059 shutdown_connections 2024-09-11T23:15:39.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:39.542+0000 7f1035cdd640 1 --2- 172.21.15.26:0/2933932059 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1030100a80 0x7f1030100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:15:39.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:39.542+0000 7f1035cdd640 1 --2- 172.21.15.26:0/2933932059 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1030101420 0x7f103010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:15:39.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:39.542+0000 7f1035cdd640 1 --2- 172.21.15.26:0/2933932059 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1030106a80 0x7f1030106e60 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:15:39.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:39.542+0000 7f1035cdd640 1 -- 172.21.15.26:0/2933932059 >> 172.21.15.26:0/2933932059 conn(0x7f10300fc7a0 msgr2=0x7f10300febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:15:39.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:39.542+0000 7f1035cdd640 1 -- 172.21.15.26:0/2933932059 shutdown_connections 2024-09-11T23:15:39.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:39.542+0000 7f1035cdd640 1 -- 172.21.15.26:0/2933932059 wait complete. 2024-09-11T23:15:39.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:39.543+0000 7f1035cdd640 1 Processor -- start 2024-09-11T23:15:39.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:39.543+0000 7f1035cdd640 1 -- start start 2024-09-11T23:15:39.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:39.543+0000 7f1035cdd640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1030100a80 0x7f1030199260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:15:39.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:39.543+0000 7f1035cdd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1030101420 0x7f10301997a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:15:39.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:39.543+0000 7f1035cdd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1030106a80 0x7f10301a42e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:15:39.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:39.543+0000 7f1035cdd640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f1030110ed0 con 0x7f1030101420 2024-09-11T23:15:39.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:39.543+0000 7f1035cdd640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f1030110d50 con 0x7f1030106a80 2024-09-11T23:15:39.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:39.543+0000 7f1035cdd640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f1030111050 con 0x7f1030100a80 2024-09-11T23:15:39.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:39.543+0000 7f1027fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1030101420 0x7f10301997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:15:39.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:39.543+0000 7f1027fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1030101420 0x7f10301997a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52628/0 (socket says 172.21.15.26:52628) 2024-09-11T23:15:39.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:39.544+0000 7f1027fff640 1 -- 172.21.15.26:0/3017112316 learned_addr learned my addr 172.21.15.26:0/3017112316 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:15:39.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:39.544+0000 7f102f7fe640 1 --2- 172.21.15.26:0/3017112316 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1030100a80 0x7f1030199260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:15:39.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:39.544+0000 7f102ffff640 1 --2- 172.21.15.26:0/3017112316 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1030106a80 0x7f10301a42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:15:39.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:39.544+0000 7f1027fff640 1 -- 172.21.15.26:0/3017112316 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1030100a80 msgr2=0x7f1030199260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:15:39.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:39.544+0000 7f1027fff640 1 --2- 172.21.15.26:0/3017112316 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1030100a80 0x7f1030199260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:15:39.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:39.544+0000 7f1027fff640 1 -- 172.21.15.26:0/3017112316 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1030106a80 msgr2=0x7f10301a42e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:15:39.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:39.544+0000 7f1027fff640 1 --2- 172.21.15.26:0/3017112316 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1030106a80 0x7f10301a42e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:15:39.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:39.544+0000 7f1027fff640 1 -- 172.21.15.26:0/3017112316 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10301a4820 con 0x7f1030101420 2024-09-11T23:15:39.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:39.544+0000 7f102f7fe640 1 --2- 172.21.15.26:0/3017112316 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1030100a80 0x7f1030199260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:15:39.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:39.544+0000 7f102ffff640 1 --2- 172.21.15.26:0/3017112316 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1030106a80 0x7f10301a42e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:15:39.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:39.544+0000 7f1027fff640 1 --2- 172.21.15.26:0/3017112316 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1030101420 0x7f10301997a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f101800d950 tx=0x7f101800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:15:39.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:39.544+0000 7f102d7fa640 1 -- 172.21.15.26:0/3017112316 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1018017070 con 0x7f1030101420 2024-09-11T23:15:39.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:39.544+0000 7f102d7fa640 1 -- 172.21.15.26:0/3017112316 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1018009c30 con 0x7f1030101420 2024-09-11T23:15:39.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:39.544+0000 7f1035cdd640 1 -- 172.21.15.26:0/3017112316 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10301a49f0 con 0x7f1030101420 2024-09-11T23:15:39.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:39.544+0000 7f102d7fa640 1 -- 172.21.15.26:0/3017112316 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f101800f6b0 con 0x7f1030101420 2024-09-11T23:15:39.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:39.544+0000 7f1035cdd640 1 -- 172.21.15.26:0/3017112316 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10301a4f00 con 0x7f1030101420 2024-09-11T23:15:39.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:39.545+0000 7f1035cdd640 1 -- 172.21.15.26:0/3017112316 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1030100ee0 con 0x7f1030101420 2024-09-11T23:15:39.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:39.546+0000 7f102d7fa640 1 -- 172.21.15.26:0/3017112316 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f10180109d0 con 0x7f1030101420 2024-09-11T23:15:39.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:39.547+0000 7f102d7fa640 1 --2- 172.21.15.26:0/3017112316 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f101c0a2b80 0x7f101c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:15:39.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:39.547+0000 7f102f7fe640 1 --2- 172.21.15.26:0/3017112316 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f101c0a2b80 0x7f101c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:15:39.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:39.547+0000 7f102d7fa640 1 -- 172.21.15.26:0/3017112316 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7127+0+0 (secure 0 0 0) 0x7f10180ce8e0 con 0x7f1030101420 2024-09-11T23:15:39.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:39.548+0000 7f102f7fe640 1 --2- 172.21.15.26:0/3017112316 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f101c0a2b80 0x7f101c0a5040 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f1020006010 tx=0x7f1020005a20 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:15:39.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:39.551+0000 7f102d7fa640 1 -- 172.21.15.26:0/3017112316 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1018009de0 con 0x7f1030101420 2024-09-11T23:15:39.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:39.729+0000 7f1035cdd640 1 -- 172.21.15.26:0/3017112316 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f10301a5130 con 0x7f1030101420 2024-09-11T23:15:39.730 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:39 smithi026 ceph-mon[31172]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:15:39.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:39.730+0000 7f102d7fa640 1 -- 172.21.15.26:0/3017112316 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v78) ==== 93+0+10 (secure 0 0 0) 0x7f101809bce0 con 0x7f1030101420 2024-09-11T23:15:39.731 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-09-11T23:15:39.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:39.732+0000 7f1035cdd640 1 -- 172.21.15.26:0/3017112316 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f101c0a2b80 msgr2=0x7f101c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:15:39.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:39.732+0000 7f1035cdd640 1 --2- 172.21.15.26:0/3017112316 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f101c0a2b80 0x7f101c0a5040 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f1020006010 tx=0x7f1020005a20 comp rx=0 tx=0).stop 2024-09-11T23:15:39.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:39.732+0000 7f1035cdd640 1 -- 172.21.15.26:0/3017112316 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1030101420 msgr2=0x7f10301997a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:15:39.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:39.732+0000 7f1035cdd640 1 --2- 172.21.15.26:0/3017112316 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1030101420 0x7f10301997a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f101800d950 tx=0x7f101800de10 comp rx=0 tx=0).stop 2024-09-11T23:15:39.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:39.732+0000 7f1035cdd640 1 -- 172.21.15.26:0/3017112316 shutdown_connections 2024-09-11T23:15:39.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:39.732+0000 7f1035cdd640 1 --2- 172.21.15.26:0/3017112316 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f101c0a2b80 0x7f101c0a5040 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:15:39.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:39.732+0000 7f1035cdd640 1 --2- 172.21.15.26:0/3017112316 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1030106a80 0x7f10301a42e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:15:39.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:39.732+0000 7f1035cdd640 1 --2- 172.21.15.26:0/3017112316 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1030101420 0x7f10301997a0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:15:39.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:39.732+0000 7f1035cdd640 1 --2- 172.21.15.26:0/3017112316 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1030100a80 0x7f1030199260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:15:39.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:39.732+0000 7f1035cdd640 1 -- 172.21.15.26:0/3017112316 >> 172.21.15.26:0/3017112316 conn(0x7f10300fc7a0 msgr2=0x7f10300fdf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:15:39.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:39.733+0000 7f1035cdd640 1 -- 172.21.15.26:0/3017112316 shutdown_connections 2024-09-11T23:15:39.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:39.733+0000 7f1035cdd640 1 -- 172.21.15.26:0/3017112316 wait complete. 2024-09-11T23:15:39.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:39 smithi136 ceph-mon[35121]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:15:40.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:39 smithi076 ceph-mon[35296]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:15:40.487 INFO:tasks.cephadm:Adding mds.a on smithi026 2024-09-11T23:15:40.487 INFO:tasks.cephadm:Adding mds.d on smithi026 2024-09-11T23:15:40.487 INFO:tasks.cephadm:Adding mds.g on smithi026 2024-09-11T23:15:40.487 INFO:tasks.cephadm:Adding mds.j on smithi026 2024-09-11T23:15:40.487 INFO:tasks.cephadm:Adding mds.b on smithi076 2024-09-11T23:15:40.487 INFO:tasks.cephadm:Adding mds.e on smithi076 2024-09-11T23:15:40.487 INFO:tasks.cephadm:Adding mds.h on smithi076 2024-09-11T23:15:40.487 INFO:tasks.cephadm:Adding mds.k on smithi076 2024-09-11T23:15:40.488 INFO:tasks.cephadm:Adding mds.c on smithi136 2024-09-11T23:15:40.488 INFO:tasks.cephadm:Adding mds.f on smithi136 2024-09-11T23:15:40.488 INFO:tasks.cephadm:Adding mds.i on smithi136 2024-09-11T23:15:40.488 INFO:tasks.cephadm:Adding mds.l on smithi136 2024-09-11T23:15:40.488 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph orch apply mds all '12;smithi026=a;smithi026=d;smithi026=g;smithi026=j;smithi076=b;smithi076=e;smithi076=h;smithi076=k;smithi136=c;smithi136=f;smithi136=i;smithi136=l' 2024-09-11T23:15:40.750 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T23:15:40.750 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:40 smithi026 ceph-mon[31172]: from='client.? 172.21.15.26:0/3017112316' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-11T23:15:40.871 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:40 smithi136 ceph-mon[35121]: from='client.? 172.21.15.26:0/3017112316' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-11T23:15:41.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:40 smithi076 ceph-mon[35296]: from='client.? 172.21.15.26:0/3017112316' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-11T23:15:41.223 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:15:41.190+0000 7f576237e640 1 -- 172.21.15.136:0/3579642810 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f575c105790 msgr2=0x7f575c105c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:15:41.223 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:15:41.190+0000 7f576237e640 1 --2- 172.21.15.136:0/3579642810 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f575c105790 0x7f575c105c40 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f5754009ca0 tx=0x7f575402a8c0 comp rx=0 tx=0).stop 2024-09-11T23:15:41.224 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:15:41.190+0000 7f576237e640 1 -- 172.21.15.136:0/3579642810 shutdown_connections 2024-09-11T23:15:41.224 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:15:41.190+0000 7f576237e640 1 --2- 172.21.15.136:0/3579642810 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f575c105790 0x7f575c105c40 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:15:41.224 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:15:41.190+0000 7f576237e640 1 --2- 172.21.15.136:0/3579642810 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f575c102de0 0x7f575c105250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:15:41.224 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:15:41.190+0000 7f576237e640 1 --2- 172.21.15.136:0/3579642810 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f575c100490 0x7f575c1028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:15:41.224 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:15:41.190+0000 7f576237e640 1 -- 172.21.15.136:0/3579642810 >> 172.21.15.136:0/3579642810 conn(0x7f575c0fc2a0 msgr2=0x7f575c0fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:15:41.224 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:15:41.190+0000 7f576237e640 1 -- 172.21.15.136:0/3579642810 shutdown_connections 2024-09-11T23:15:41.224 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:15:41.190+0000 7f576237e640 1 -- 172.21.15.136:0/3579642810 wait complete. 2024-09-11T23:15:41.224 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:15:41.191+0000 7f576237e640 1 Processor -- start 2024-09-11T23:15:41.224 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:15:41.191+0000 7f576237e640 1 -- start start 2024-09-11T23:15:41.224 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:15:41.191+0000 7f576237e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f575c100490 0x7f575c19e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:15:41.224 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:15:41.191+0000 7f576237e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f575c102de0 0x7f575c19e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:15:41.224 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:15:41.191+0000 7f576237e640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f575c105790 0x7f575c1a5960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:15:41.224 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:15:41.192+0000 7f576237e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f575c113ea0 con 0x7f575c102de0 2024-09-11T23:15:41.225 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:15:41.192+0000 7f576091b640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f575c105790 0x7f575c1a5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:15:41.225 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:15:41.192+0000 7f576091b640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f575c105790 0x7f575c1a5960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.136:57606/0 (socket says 172.21.15.136:57606) 2024-09-11T23:15:41.225 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:15:41.192+0000 7f576091b640 1 -- 172.21.15.136:0/1130366985 learned_addr learned my addr 172.21.15.136:0/1130366985 (peer_addr_for_me v2:172.21.15.136:0/0) 2024-09-11T23:15:41.225 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:15:41.192+0000 7f575bfff640 1 --2- 172.21.15.136:0/1130366985 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f575c100490 0x7f575c19e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:15:41.225 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:15:41.192+0000 7f576237e640 1 -- 172.21.15.136:0/1130366985 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f575c113d20 con 0x7f575c100490 2024-09-11T23:15:41.225 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:15:41.192+0000 7f576237e640 1 -- 172.21.15.136:0/1130366985 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f575c114020 con 0x7f575c105790 2024-09-11T23:15:41.225 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:15:41.192+0000 7f575b7fe640 1 --2- 172.21.15.136:0/1130366985 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f575c102de0 0x7f575c19e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:15:41.225 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:15:41.192+0000 7f575bfff640 1 -- 172.21.15.136:0/1130366985 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f575c105790 msgr2=0x7f575c1a5960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:15:41.225 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:15:41.192+0000 7f575bfff640 1 --2- 172.21.15.136:0/1130366985 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f575c105790 0x7f575c1a5960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:15:41.225 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:15:41.192+0000 7f575bfff640 1 -- 172.21.15.136:0/1130366985 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f575c102de0 msgr2=0x7f575c19e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:15:41.225 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:15:41.192+0000 7f575bfff640 1 --2- 172.21.15.136:0/1130366985 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f575c102de0 0x7f575c19e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:15:41.225 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:15:41.192+0000 7f575bfff640 1 -- 172.21.15.136:0/1130366985 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f575c1a5fd0 con 0x7f575c100490 2024-09-11T23:15:41.225 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:15:41.192+0000 7f576091b640 1 --2- 172.21.15.136:0/1130366985 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f575c105790 0x7f575c1a5960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:15:41.226 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:15:41.192+0000 7f575bfff640 1 --2- 172.21.15.136:0/1130366985 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f575c100490 0x7f575c19e3a0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f574c00d6d0 tx=0x7f574c00dad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:15:41.226 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:15:41.193+0000 7f57597fa640 1 -- 172.21.15.136:0/1130366985 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f574c017070 con 0x7f575c100490 2024-09-11T23:15:41.226 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:15:41.193+0000 7f57597fa640 1 -- 172.21.15.136:0/1130366985 <== mon.2 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f574c0047a0 con 0x7f575c100490 2024-09-11T23:15:41.226 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:15:41.193+0000 7f576237e640 1 -- 172.21.15.136:0/1130366985 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f575c1a62c0 con 0x7f575c100490 2024-09-11T23:15:41.226 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:15:41.193+0000 7f57597fa640 1 -- 172.21.15.136:0/1130366985 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f574c009c80 con 0x7f575c100490 2024-09-11T23:15:41.226 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:15:41.193+0000 7f576237e640 1 -- 172.21.15.136:0/1130366985 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f575c1a6800 con 0x7f575c100490 2024-09-11T23:15:41.226 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:15:41.194+0000 7f576237e640 1 -- 172.21.15.136:0/1130366985 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f575c0fdd90 con 0x7f575c100490 2024-09-11T23:15:41.226 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:15:41.195+0000 7f57597fa640 1 -- 172.21.15.136:0/1130366985 <== mon.2 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f574c004950 con 0x7f575c100490 2024-09-11T23:15:41.226 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:15:41.195+0000 7f57597fa640 1 --2- 172.21.15.136:0/1130366985 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f57440a2bb0 0x7f57440a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:15:41.226 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:15:41.196+0000 7f57597fa640 1 -- 172.21.15.136:0/1130366985 <== mon.2 v2:172.21.15.76:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7127+0+0 (secure 0 0 0) 0x7f574c0ce430 con 0x7f575c100490 2024-09-11T23:15:41.226 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:15:41.196+0000 7f575b7fe640 1 --2- 172.21.15.136:0/1130366985 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f57440a2bb0 0x7f57440a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:15:41.226 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:15:41.196+0000 7f575b7fe640 1 --2- 172.21.15.136:0/1130366985 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f57440a2bb0 0x7f57440a5070 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f575c19f8c0 tx=0x7f575000a2d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:15:41.226 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:15:41.200+0000 7f57597fa640 1 -- 172.21.15.136:0/1130366985 <== mon.2 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f574c009e20 con 0x7f575c100490 2024-09-11T23:15:41.381 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:15:41.381+0000 7f576237e640 1 -- 172.21.15.136:0/1130366985 --> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi026=a;smithi026=d;smithi026=g;smithi026=j;smithi076=b;smithi076=e;smithi076=h;smithi076=k;smithi136=c;smithi136=f;smithi136=i;smithi136=l", "target": ["mon-mgr", ""]}) -- 0x7f575c0628f0 con 0x7f57440a2bb0 2024-09-11T23:15:41.393 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:15:41.389+0000 7f57597fa640 1 -- 172.21.15.136:0/1130366985 <== mgr.14148 v2:172.21.15.26:6800/110833859 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f575c0628f0 con 0x7f57440a2bb0 2024-09-11T23:15:41.393 INFO:teuthology.orchestra.run.smithi136.stdout:Scheduled mds.all update... 2024-09-11T23:15:41.394 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:15:41.392+0000 7f576237e640 1 -- 172.21.15.136:0/1130366985 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f57440a2bb0 msgr2=0x7f57440a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:15:41.394 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:15:41.392+0000 7f576237e640 1 --2- 172.21.15.136:0/1130366985 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f57440a2bb0 0x7f57440a5070 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f575c19f8c0 tx=0x7f575000a2d0 comp rx=0 tx=0).stop 2024-09-11T23:15:41.394 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:15:41.392+0000 7f576237e640 1 -- 172.21.15.136:0/1130366985 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f575c100490 msgr2=0x7f575c19e3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:15:41.394 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:15:41.392+0000 7f576237e640 1 --2- 172.21.15.136:0/1130366985 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f575c100490 0x7f575c19e3a0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f574c00d6d0 tx=0x7f574c00dad0 comp rx=0 tx=0).stop 2024-09-11T23:15:41.394 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:15:41.392+0000 7f576237e640 1 -- 172.21.15.136:0/1130366985 shutdown_connections 2024-09-11T23:15:41.394 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:15:41.392+0000 7f576237e640 1 --2- 172.21.15.136:0/1130366985 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f57440a2bb0 0x7f57440a5070 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:15:41.394 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:15:41.392+0000 7f576237e640 1 --2- 172.21.15.136:0/1130366985 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f575c105790 0x7f575c1a5960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:15:41.394 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:15:41.392+0000 7f576237e640 1 --2- 172.21.15.136:0/1130366985 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f575c102de0 0x7f575c19e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:15:41.394 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:15:41.392+0000 7f576237e640 1 --2- 172.21.15.136:0/1130366985 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f575c100490 0x7f575c19e3a0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:15:41.394 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:15:41.392+0000 7f576237e640 1 -- 172.21.15.136:0/1130366985 >> 172.21.15.136:0/1130366985 conn(0x7f575c0fc2a0 msgr2=0x7f575c100cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:15:41.394 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:15:41.392+0000 7f576237e640 1 -- 172.21.15.136:0/1130366985 shutdown_connections 2024-09-11T23:15:41.395 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-11T23:15:41.392+0000 7f576237e640 1 -- 172.21.15.136:0/1130366985 wait complete. 2024-09-11T23:15:41.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:41 smithi136 ceph-mon[35121]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:15:41.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:41 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:15:41.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:41 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:15:42.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:41 smithi026 ceph-mon[31172]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:15:42.004 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:41 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:15:42.004 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:41 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:15:42.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:41 smithi076 ceph-mon[35296]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:15:42.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:41 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:15:42.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:41 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:15:42.138 DEBUG:teuthology.orchestra.run.smithi026:mds.a> sudo journalctl -f -n 0 -u ceph-af7e6fc8-7092-11ef-bcea-c7b262605968@mds.a.service 2024-09-11T23:15:42.140 DEBUG:teuthology.orchestra.run.smithi026:mds.d> sudo journalctl -f -n 0 -u ceph-af7e6fc8-7092-11ef-bcea-c7b262605968@mds.d.service 2024-09-11T23:15:42.184 DEBUG:teuthology.orchestra.run.smithi026:mds.g> sudo journalctl -f -n 0 -u ceph-af7e6fc8-7092-11ef-bcea-c7b262605968@mds.g.service 2024-09-11T23:15:42.228 DEBUG:teuthology.orchestra.run.smithi026:mds.j> sudo journalctl -f -n 0 -u ceph-af7e6fc8-7092-11ef-bcea-c7b262605968@mds.j.service 2024-09-11T23:15:42.270 DEBUG:teuthology.orchestra.run.smithi076:mds.b> sudo journalctl -f -n 0 -u ceph-af7e6fc8-7092-11ef-bcea-c7b262605968@mds.b.service 2024-09-11T23:15:42.273 DEBUG:teuthology.orchestra.run.smithi076:mds.e> sudo journalctl -f -n 0 -u ceph-af7e6fc8-7092-11ef-bcea-c7b262605968@mds.e.service 2024-09-11T23:15:42.316 DEBUG:teuthology.orchestra.run.smithi076:mds.h> sudo journalctl -f -n 0 -u ceph-af7e6fc8-7092-11ef-bcea-c7b262605968@mds.h.service 2024-09-11T23:15:42.359 DEBUG:teuthology.orchestra.run.smithi076:mds.k> sudo journalctl -f -n 0 -u ceph-af7e6fc8-7092-11ef-bcea-c7b262605968@mds.k.service 2024-09-11T23:15:42.402 DEBUG:teuthology.orchestra.run.smithi136:mds.c> sudo journalctl -f -n 0 -u ceph-af7e6fc8-7092-11ef-bcea-c7b262605968@mds.c.service 2024-09-11T23:15:42.405 DEBUG:teuthology.orchestra.run.smithi136:mds.f> sudo journalctl -f -n 0 -u ceph-af7e6fc8-7092-11ef-bcea-c7b262605968@mds.f.service 2024-09-11T23:15:42.448 DEBUG:teuthology.orchestra.run.smithi136:mds.i> sudo journalctl -f -n 0 -u ceph-af7e6fc8-7092-11ef-bcea-c7b262605968@mds.i.service 2024-09-11T23:15:42.491 DEBUG:teuthology.orchestra.run.smithi136:mds.l> sudo journalctl -f -n 0 -u ceph-af7e6fc8-7092-11ef-bcea-c7b262605968@mds.l.service 2024-09-11T23:15:42.534 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-09-11T23:15:42.534 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:15:42.699 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:42 smithi136 ceph-mon[35121]: from='client.24395 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi026=a;smithi026=d;smithi026=g;smithi026=j;smithi076=b;smithi076=e;smithi076=h;smithi076=k;smithi136=c;smithi136=f;smithi136=i;smithi136=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:15:42.699 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:42 smithi136 ceph-mon[35121]: Saving service mds.all spec with placement smithi026=a;smithi026=d;smithi026=g;smithi026=j;smithi076=b;smithi076=e;smithi076=h;smithi076=k;smithi136=c;smithi136=f;smithi136=i;smithi136=l;count:12 2024-09-11T23:15:42.699 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:42 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:15:42.699 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:42 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:15:42.699 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:42 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:15:42.699 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:42 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T23:15:42.700 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:42 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T23:15:42.700 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:42 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:15:42.700 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:42 smithi136 ceph-mon[35121]: Deploying daemon mds.l on smithi136 2024-09-11T23:15:42.787 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:15:42.909 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:42 smithi026 ceph-mon[31172]: from='client.24395 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi026=a;smithi026=d;smithi026=g;smithi026=j;smithi076=b;smithi076=e;smithi076=h;smithi076=k;smithi136=c;smithi136=f;smithi136=i;smithi136=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:15:42.909 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:42 smithi026 ceph-mon[31172]: Saving service mds.all spec with placement smithi026=a;smithi026=d;smithi026=g;smithi026=j;smithi076=b;smithi076=e;smithi076=h;smithi076=k;smithi136=c;smithi136=f;smithi136=i;smithi136=l;count:12 2024-09-11T23:15:42.909 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:42 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:15:42.909 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:42 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:15:42.909 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:42 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:15:42.909 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:42 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T23:15:42.910 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:42 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T23:15:42.910 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:42 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:15:42.910 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:42 smithi026 ceph-mon[31172]: Deploying daemon mds.l on smithi136 2024-09-11T23:15:43.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:42 smithi076 ceph-mon[35296]: from='client.24395 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi026=a;smithi026=d;smithi026=g;smithi026=j;smithi076=b;smithi076=e;smithi076=h;smithi076=k;smithi136=c;smithi136=f;smithi136=i;smithi136=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:15:43.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:42 smithi076 ceph-mon[35296]: Saving service mds.all spec with placement smithi026=a;smithi026=d;smithi026=g;smithi026=j;smithi076=b;smithi076=e;smithi076=h;smithi076=k;smithi136=c;smithi136=f;smithi136=i;smithi136=l;count:12 2024-09-11T23:15:43.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:42 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:15:43.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:42 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:15:43.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:42 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:15:43.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:42 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T23:15:43.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:42 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T23:15:43.054 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:42 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:15:43.054 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:42 smithi076 ceph-mon[35296]: Deploying daemon mds.l on smithi136 2024-09-11T23:15:43.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:43.230+0000 7f835d50d640 1 -- 172.21.15.26:0/3915624061 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f83581000d0 msgr2=0x7f8358100530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:15:43.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:43.230+0000 7f835d50d640 1 --2- 172.21.15.26:0/3915624061 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f83581000d0 0x7f8358100530 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f834c009900 tx=0x7f834c02a410 comp rx=0 tx=0).stop 2024-09-11T23:15:43.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:43.230+0000 7f835d50d640 1 -- 172.21.15.26:0/3915624061 shutdown_connections 2024-09-11T23:15:43.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:43.230+0000 7f835d50d640 1 --2- 172.21.15.26:0/3915624061 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8358100a70 0x7f835810ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:15:43.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:43.230+0000 7f835d50d640 1 --2- 172.21.15.26:0/3915624061 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f83581000d0 0x7f8358100530 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:15:43.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:43.230+0000 7f835d50d640 1 --2- 172.21.15.26:0/3915624061 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8358106ba0 0x7f8358106f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:15:43.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:43.230+0000 7f835d50d640 1 -- 172.21.15.26:0/3915624061 >> 172.21.15.26:0/3915624061 conn(0x7f8358077210 msgr2=0x7f83580fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:15:43.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:43.230+0000 7f835d50d640 1 -- 172.21.15.26:0/3915624061 shutdown_connections 2024-09-11T23:15:43.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:43.230+0000 7f835d50d640 1 -- 172.21.15.26:0/3915624061 wait complete. 2024-09-11T23:15:43.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:43.231+0000 7f835d50d640 1 Processor -- start 2024-09-11T23:15:43.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:43.231+0000 7f835d50d640 1 -- start start 2024-09-11T23:15:43.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:43.231+0000 7f835d50d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f83581000d0 0x7f8358198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:15:43.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:43.231+0000 7f835d50d640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8358100a70 0x7f8358198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:15:43.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:43.231+0000 7f835d50d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8358106ba0 0x7f835819cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:15:43.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:43.232+0000 7f835d50d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f8358112090 con 0x7f8358106ba0 2024-09-11T23:15:43.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:43.232+0000 7f835d50d640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f8358111f10 con 0x7f83581000d0 2024-09-11T23:15:43.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:43.232+0000 7f835d50d640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f8358112210 con 0x7f8358100a70 2024-09-11T23:15:43.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:43.232+0000 7f83577fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8358106ba0 0x7f835819cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:15:43.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:43.232+0000 7f83577fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8358106ba0 0x7f835819cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38762/0 (socket says 172.21.15.26:38762) 2024-09-11T23:15:43.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:43.232+0000 7f83567fc640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8358100a70 0x7f8358198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:15:43.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:43.232+0000 7f83577fe640 1 -- 172.21.15.26:0/347219893 learned_addr learned my addr 172.21.15.26:0/347219893 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:15:43.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:43.232+0000 7f8356ffd640 1 --2- 172.21.15.26:0/347219893 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f83581000d0 0x7f8358198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:15:43.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:43.232+0000 7f83567fc640 1 -- 172.21.15.26:0/347219893 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f83581000d0 msgr2=0x7f8358198510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:15:43.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:43.232+0000 7f83567fc640 1 --2- 172.21.15.26:0/347219893 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f83581000d0 0x7f8358198510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:15:43.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:43.232+0000 7f83567fc640 1 -- 172.21.15.26:0/347219893 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8358106ba0 msgr2=0x7f835819cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:15:43.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:43.232+0000 7f83567fc640 1 --2- 172.21.15.26:0/347219893 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8358106ba0 0x7f835819cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:15:43.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:43.232+0000 7f83567fc640 1 -- 172.21.15.26:0/347219893 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f835819d4c0 con 0x7f8358100a70 2024-09-11T23:15:43.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:43.232+0000 7f83577fe640 1 --2- 172.21.15.26:0/347219893 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8358106ba0 0x7f835819cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:15:43.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:43.232+0000 7f83567fc640 1 --2- 172.21.15.26:0/347219893 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8358100a70 0x7f8358198a50 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f834c005940 tx=0x7f834c003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:15:43.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:43.233+0000 7f833ffff640 1 -- 172.21.15.26:0/347219893 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f834c03f070 con 0x7f8358100a70 2024-09-11T23:15:43.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:43.233+0000 7f833ffff640 1 -- 172.21.15.26:0/347219893 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f834c031c30 con 0x7f8358100a70 2024-09-11T23:15:43.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:43.233+0000 7f835d50d640 1 -- 172.21.15.26:0/347219893 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f835819d750 con 0x7f8358100a70 2024-09-11T23:15:43.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:43.233+0000 7f833ffff640 1 -- 172.21.15.26:0/347219893 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f834c035030 con 0x7f8358100a70 2024-09-11T23:15:43.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:43.233+0000 7f835d50d640 1 -- 172.21.15.26:0/347219893 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f835819dbe0 con 0x7f8358100a70 2024-09-11T23:15:43.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:43.234+0000 7f835d50d640 1 -- 172.21.15.26:0/347219893 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8358199870 con 0x7f8358100a70 2024-09-11T23:15:43.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:43.235+0000 7f833ffff640 1 -- 172.21.15.26:0/347219893 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f834c042070 con 0x7f8358100a70 2024-09-11T23:15:43.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:43.236+0000 7f833ffff640 1 --2- 172.21.15.26:0/347219893 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f83400a2b80 0x7f83400a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:15:43.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:43.236+0000 7f8356ffd640 1 --2- 172.21.15.26:0/347219893 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f83400a2b80 0x7f83400a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:15:43.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:43.236+0000 7f833ffff640 1 -- 172.21.15.26:0/347219893 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7127+0+0 (secure 0 0 0) 0x7f834c0ef7e0 con 0x7f8358100a70 2024-09-11T23:15:43.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:43.236+0000 7f8356ffd640 1 --2- 172.21.15.26:0/347219893 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f83400a2b80 0x7f83400a5040 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f8348009bd0 tx=0x7f8348005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:15:43.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:43.239+0000 7f833ffff640 1 -- 172.21.15.26:0/347219893 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f834c031de0 con 0x7f8358100a70 2024-09-11T23:15:43.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:43.451+0000 7f835d50d640 1 -- 172.21.15.26:0/347219893 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f835810a090 con 0x7f8358100a70 2024-09-11T23:15:43.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:43.452+0000 7f833ffff640 1 -- 172.21.15.26:0/347219893 <== mon.1 v2:172.21.15.136:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+611 (secure 0 0 0) 0x7f834c0bc4f0 con 0x7f8358100a70 2024-09-11T23:15:43.452 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-11T23:15:43.452 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":1,"btime":"2024-09-11T23:08:40:486266+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-09-11T23:15:43.452 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 1 2024-09-11T23:15:43.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:43.454+0000 7f835d50d640 1 -- 172.21.15.26:0/347219893 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f83400a2b80 msgr2=0x7f83400a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:15:43.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:43.454+0000 7f835d50d640 1 --2- 172.21.15.26:0/347219893 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f83400a2b80 0x7f83400a5040 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f8348009bd0 tx=0x7f8348005f90 comp rx=0 tx=0).stop 2024-09-11T23:15:43.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:43.454+0000 7f835d50d640 1 -- 172.21.15.26:0/347219893 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8358100a70 msgr2=0x7f8358198a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:15:43.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:43.454+0000 7f835d50d640 1 --2- 172.21.15.26:0/347219893 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8358100a70 0x7f8358198a50 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f834c005940 tx=0x7f834c003c10 comp rx=0 tx=0).stop 2024-09-11T23:15:43.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:43.454+0000 7f835d50d640 1 -- 172.21.15.26:0/347219893 shutdown_connections 2024-09-11T23:15:43.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:43.454+0000 7f835d50d640 1 --2- 172.21.15.26:0/347219893 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f83400a2b80 0x7f83400a5040 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:15:43.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:43.454+0000 7f835d50d640 1 --2- 172.21.15.26:0/347219893 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8358106ba0 0x7f835819cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:15:43.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:43.454+0000 7f835d50d640 1 --2- 172.21.15.26:0/347219893 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8358100a70 0x7f8358198a50 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:15:43.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:43.454+0000 7f835d50d640 1 --2- 172.21.15.26:0/347219893 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f83581000d0 0x7f8358198510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:15:43.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:43.454+0000 7f835d50d640 1 -- 172.21.15.26:0/347219893 >> 172.21.15.26:0/347219893 conn(0x7f8358077210 msgr2=0x7f8358104c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:15:43.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:43.454+0000 7f835d50d640 1 -- 172.21.15.26:0/347219893 shutdown_connections 2024-09-11T23:15:43.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:43.454+0000 7f835d50d640 1 -- 172.21.15.26:0/347219893 wait complete. 2024-09-11T23:15:43.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:43 smithi136 ceph-mon[35121]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:15:43.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:43 smithi136 ceph-mon[35121]: from='client.? 172.21.15.26:0/347219893' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:15:44.237 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:43 smithi026 ceph-mon[31172]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:15:44.237 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:43 smithi026 ceph-mon[31172]: from='client.? 172.21.15.26:0/347219893' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:15:44.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:43 smithi076 ceph-mon[35296]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:15:44.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:43 smithi076 ceph-mon[35296]: from='client.? 172.21.15.26:0/347219893' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:15:44.739 INFO:journalctl@ceph.mds.l.smithi136.stdout:Sep 11 23:15:44 smithi136 systemd[1]: Started Ceph mds.l for af7e6fc8-7092-11ef-bcea-c7b262605968. 2024-09-11T23:15:45.553 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:45 smithi076 ceph-mon[35296]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:15:45.553 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:45 smithi076 ceph-mon[35296]: mds.? [v2:172.21.15.136:6832/2019841455,v1:172.21.15.136:6833/2019841455] up:boot 2024-09-11T23:15:45.553 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:45 smithi076 ceph-mon[35296]: fsmap 1 up:standby 2024-09-11T23:15:45.553 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:45 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-11T23:15:45.553 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:45 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:15:45.553 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:45 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:15:45.553 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:45 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:15:45.553 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:45 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T23:15:45.554 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:45 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T23:15:45.554 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:45 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:15:45.554 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:45 smithi076 ceph-mon[35296]: Deploying daemon mds.k on smithi076 2024-09-11T23:15:45.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:45 smithi136 ceph-mon[35121]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:15:45.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:45 smithi136 ceph-mon[35121]: mds.? [v2:172.21.15.136:6832/2019841455,v1:172.21.15.136:6833/2019841455] up:boot 2024-09-11T23:15:45.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:45 smithi136 ceph-mon[35121]: fsmap 1 up:standby 2024-09-11T23:15:45.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:45 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-11T23:15:45.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:45 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:15:45.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:45 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:15:45.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:45 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:15:45.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:45 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T23:15:45.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:45 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T23:15:45.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:45 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:15:45.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:45 smithi136 ceph-mon[35121]: Deploying daemon mds.k on smithi076 2024-09-11T23:15:45.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:45 smithi026 ceph-mon[31172]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:15:45.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:45 smithi026 ceph-mon[31172]: mds.? [v2:172.21.15.136:6832/2019841455,v1:172.21.15.136:6833/2019841455] up:boot 2024-09-11T23:15:45.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:45 smithi026 ceph-mon[31172]: fsmap 1 up:standby 2024-09-11T23:15:45.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:45 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-11T23:15:45.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:45 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:15:45.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:45 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:15:45.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:45 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:15:45.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:45 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T23:15:45.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:45 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T23:15:45.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:45 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:15:45.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:45 smithi026 ceph-mon[31172]: Deploying daemon mds.k on smithi076 2024-09-11T23:15:46.268 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:15:46.538 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:15:46.553 INFO:journalctl@ceph.mds.k.smithi076.stdout:Sep 11 23:15:46 smithi076 systemd[1]: Starting Ceph mds.k for af7e6fc8-7092-11ef-bcea-c7b262605968... 2024-09-11T23:15:46.841 INFO:journalctl@ceph.mds.k.smithi076.stdout:Sep 11 23:15:46 smithi076 podman[64389]: 2024-09-11 23:15:46.490198702 +0000 UTC m=+0.015762195 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T23:15:46.841 INFO:journalctl@ceph.mds.k.smithi076.stdout:Sep 11 23:15:46 smithi076 podman[64389]: 2024-09-11 23:15:46.603783308 +0000 UTC m=+0.129346804 container create 0a8b3488447032b560cf12b53ead05570c08493a5f0654814100e8fb935e0e33 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mds-k, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, CEPH_POINT_RELEASE=) 2024-09-11T23:15:46.841 INFO:journalctl@ceph.mds.k.smithi076.stdout:Sep 11 23:15:46 smithi076 podman[64389]: 2024-09-11 23:15:46.81519868 +0000 UTC m=+0.340762174 container init 0a8b3488447032b560cf12b53ead05570c08493a5f0654814100e8fb935e0e33 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mds-k, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0) 2024-09-11T23:15:46.841 INFO:journalctl@ceph.mds.k.smithi076.stdout:Sep 11 23:15:46 smithi076 podman[64389]: 2024-09-11 23:15:46.818046818 +0000 UTC m=+0.343610318 container start 0a8b3488447032b560cf12b53ead05570c08493a5f0654814100e8fb935e0e33 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mds-k, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, ceph=True, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2) 2024-09-11T23:15:47.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:47.011+0000 7f547a18d640 1 -- 172.21.15.26:0/1534521391 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5474102ef0 msgr2=0x7f547410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:15:47.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:47.011+0000 7f547a18d640 1 --2- 172.21.15.26:0/1534521391 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5474102ef0 0x7f547410da40 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f546c009ca0 tx=0x7f546c02a950 comp rx=0 tx=0).stop 2024-09-11T23:15:47.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:47.011+0000 7f547a18d640 1 -- 172.21.15.26:0/1534521391 shutdown_connections 2024-09-11T23:15:47.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:47.011+0000 7f547a18d640 1 --2- 172.21.15.26:0/1534521391 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5474102570 0x7f54741029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:15:47.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:47.011+0000 7f547a18d640 1 --2- 172.21.15.26:0/1534521391 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5474102ef0 0x7f547410da40 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:15:47.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:47.011+0000 7f547a18d640 1 --2- 172.21.15.26:0/1534521391 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5474100e20 0x7f5474101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:15:47.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:47.011+0000 7f547a18d640 1 -- 172.21.15.26:0/1534521391 >> 172.21.15.26:0/1534521391 conn(0x7f54740fc7d0 msgr2=0x7f54740febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:15:47.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:47.011+0000 7f547a18d640 1 -- 172.21.15.26:0/1534521391 shutdown_connections 2024-09-11T23:15:47.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:47.011+0000 7f547a18d640 1 -- 172.21.15.26:0/1534521391 wait complete. 2024-09-11T23:15:47.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:47.012+0000 7f547a18d640 1 Processor -- start 2024-09-11T23:15:47.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:47.012+0000 7f547a18d640 1 -- start start 2024-09-11T23:15:47.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:47.012+0000 7f547a18d640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5474100e20 0x7f5474199300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:15:47.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:47.012+0000 7f547a18d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5474102570 0x7f5474199840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:15:47.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:47.012+0000 7f547a18d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5474102ef0 0x7f54741a4380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:15:47.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:47.012+0000 7f547a18d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f5474110fd0 con 0x7f5474102ef0 2024-09-11T23:15:47.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:47.012+0000 7f547a18d640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f5474110e50 con 0x7f5474102570 2024-09-11T23:15:47.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:47.012+0000 7f547a18d640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f5474111150 con 0x7f5474100e20 2024-09-11T23:15:47.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:47.013+0000 7f5472ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5474102570 0x7f5474199840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:15:47.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:47.013+0000 7f5473fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5474102ef0 0x7f54741a4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:15:47.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:47.013+0000 7f54737fe640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5474100e20 0x7f5474199300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:15:47.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:47.013+0000 7f5472ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5474102570 0x7f5474199840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:55318/0 (socket says 172.21.15.26:55318) 2024-09-11T23:15:47.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:47.013+0000 7f5472ffd640 1 -- 172.21.15.26:0/3281556092 learned_addr learned my addr 172.21.15.26:0/3281556092 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:15:47.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:47.013+0000 7f5472ffd640 1 -- 172.21.15.26:0/3281556092 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5474100e20 msgr2=0x7f5474199300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:15:47.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:47.013+0000 7f5472ffd640 1 --2- 172.21.15.26:0/3281556092 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5474100e20 0x7f5474199300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:15:47.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:47.013+0000 7f5472ffd640 1 -- 172.21.15.26:0/3281556092 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5474102ef0 msgr2=0x7f54741a4380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:15:47.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:47.013+0000 7f5472ffd640 1 --2- 172.21.15.26:0/3281556092 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5474102ef0 0x7f54741a4380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:15:47.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:47.013+0000 7f5472ffd640 1 -- 172.21.15.26:0/3281556092 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54741a48c0 con 0x7f5474102570 2024-09-11T23:15:47.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:47.013+0000 7f5472ffd640 1 --2- 172.21.15.26:0/3281556092 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5474102570 0x7f5474199840 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f546800ea60 tx=0x7f546800ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:15:47.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:47.013+0000 7f5470ff9640 1 -- 172.21.15.26:0/3281556092 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f546800cde0 con 0x7f5474102570 2024-09-11T23:15:47.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:47.013+0000 7f5470ff9640 1 -- 172.21.15.26:0/3281556092 <== mon.2 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54680035f0 con 0x7f5474102570 2024-09-11T23:15:47.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:47.013+0000 7f5470ff9640 1 -- 172.21.15.26:0/3281556092 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5468010670 con 0x7f5474102570 2024-09-11T23:15:47.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:47.013+0000 7f547a18d640 1 -- 172.21.15.26:0/3281556092 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54741970f0 con 0x7f5474102570 2024-09-11T23:15:47.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:47.014+0000 7f547a18d640 1 -- 172.21.15.26:0/3281556092 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5474197680 con 0x7f5474102570 2024-09-11T23:15:47.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:47.015+0000 7f547a18d640 1 -- 172.21.15.26:0/3281556092 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5474101200 con 0x7f5474102570 2024-09-11T23:15:47.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:47.016+0000 7f5470ff9640 1 -- 172.21.15.26:0/3281556092 <== mon.2 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f5468010810 con 0x7f5474102570 2024-09-11T23:15:47.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:47.017+0000 7f5470ff9640 1 --2- 172.21.15.26:0/3281556092 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f545c0a2c00 0x7f545c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:15:47.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:47.017+0000 7f54737fe640 1 --2- 172.21.15.26:0/3281556092 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f545c0a2c00 0x7f545c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:15:47.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:47.017+0000 7f5470ff9640 1 -- 172.21.15.26:0/3281556092 <== mon.2 v2:172.21.15.76:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7127+0+0 (secure 0 0 0) 0x7f54680cee20 con 0x7f5474102570 2024-09-11T23:15:47.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:47.017+0000 7f54737fe640 1 --2- 172.21.15.26:0/3281556092 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f545c0a2c00 0x7f545c0a50c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f5464009bd0 tx=0x7f5464005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:15:47.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:47.021+0000 7f5470ff9640 1 -- 172.21.15.26:0/3281556092 <== mon.2 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f546809c180 con 0x7f5474102570 2024-09-11T23:15:47.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:47.262+0000 7f547a18d640 1 -- 172.21.15.26:0/3281556092 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5474197940 con 0x7f5474102570 2024-09-11T23:15:47.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:47.263+0000 7f5470ff9640 1 -- 172.21.15.26:0/3281556092 <== mon.2 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1438 (secure 0 0 0) 0x7f546809ba80 con 0x7f5474102570 2024-09-11T23:15:47.263 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-11T23:15:47.263 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":2,"btime":"2024-09-11T23:15:44:314618+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6833/2019841455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":2019841455},{"type":"v1","addr":"172.21.15.136:6833","nonce":2019841455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-11T23:15:47.263 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 2 2024-09-11T23:15:47.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:47.265+0000 7f547a18d640 1 -- 172.21.15.26:0/3281556092 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f545c0a2c00 msgr2=0x7f545c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:15:47.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:47.265+0000 7f547a18d640 1 --2- 172.21.15.26:0/3281556092 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f545c0a2c00 0x7f545c0a50c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f5464009bd0 tx=0x7f5464005f90 comp rx=0 tx=0).stop 2024-09-11T23:15:47.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:47.265+0000 7f547a18d640 1 -- 172.21.15.26:0/3281556092 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5474102570 msgr2=0x7f5474199840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:15:47.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:47.265+0000 7f547a18d640 1 --2- 172.21.15.26:0/3281556092 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5474102570 0x7f5474199840 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f546800ea60 tx=0x7f546800ef20 comp rx=0 tx=0).stop 2024-09-11T23:15:47.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:47.265+0000 7f547a18d640 1 -- 172.21.15.26:0/3281556092 shutdown_connections 2024-09-11T23:15:47.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:47.265+0000 7f547a18d640 1 --2- 172.21.15.26:0/3281556092 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f545c0a2c00 0x7f545c0a50c0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:15:47.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:47.265+0000 7f547a18d640 1 --2- 172.21.15.26:0/3281556092 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5474102570 0x7f5474199840 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:15:47.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:47.265+0000 7f547a18d640 1 --2- 172.21.15.26:0/3281556092 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5474102ef0 0x7f54741a4380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:15:47.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:47.265+0000 7f547a18d640 1 --2- 172.21.15.26:0/3281556092 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5474100e20 0x7f5474199300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:15:47.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:47.265+0000 7f547a18d640 1 -- 172.21.15.26:0/3281556092 >> 172.21.15.26:0/3281556092 conn(0x7f54740fc7d0 msgr2=0x7f5474106870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:15:47.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:47.265+0000 7f547a18d640 1 -- 172.21.15.26:0/3281556092 shutdown_connections 2024-09-11T23:15:47.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:47.265+0000 7f547a18d640 1 -- 172.21.15.26:0/3281556092 wait complete. 2024-09-11T23:15:47.303 INFO:journalctl@ceph.mds.k.smithi076.stdout:Sep 11 23:15:46 smithi076 ceph-mds[64409]: -- 172.21.15.76:0/4197412933 <== mon.2 v2:172.21.15.76:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557ed7e3d860 con 0x557ed7e1b400 2024-09-11T23:15:47.303 INFO:journalctl@ceph.mds.k.smithi076.stdout:Sep 11 23:15:46 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mds-k[64405]: starting mds.k at 2024-09-11T23:15:47.303 INFO:journalctl@ceph.mds.k.smithi076.stdout:Sep 11 23:15:46 smithi076 bash[64389]: 0a8b3488447032b560cf12b53ead05570c08493a5f0654814100e8fb935e0e33 2024-09-11T23:15:47.303 INFO:journalctl@ceph.mds.k.smithi076.stdout:Sep 11 23:15:46 smithi076 systemd[1]: Started Ceph mds.k for af7e6fc8-7092-11ef-bcea-c7b262605968. 2024-09-11T23:15:47.674 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:47 smithi026 ceph-mon[31172]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:15:47.674 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:47 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:15:47.674 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:47 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:15:47.674 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:47 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:15:47.675 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:47 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T23:15:47.675 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:47 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T23:15:47.675 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:47 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:15:47.675 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:47 smithi026 ceph-mon[31172]: from='client.? 172.21.15.26:0/3281556092' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:15:47.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:47 smithi136 ceph-mon[35121]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:15:47.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:47 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:15:47.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:47 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:15:47.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:47 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:15:47.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:47 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T23:15:47.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:47 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T23:15:47.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:47 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:15:47.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:47 smithi136 ceph-mon[35121]: from='client.? 172.21.15.26:0/3281556092' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:15:47.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:47 smithi076 ceph-mon[35296]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:15:47.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:47 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:15:47.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:47 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:15:47.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:47 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:15:47.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:47 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T23:15:47.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:47 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T23:15:47.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:47 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:15:47.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:47 smithi076 ceph-mon[35296]: from='client.? 172.21.15.26:0/3281556092' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:15:48.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:48 smithi136 ceph-mon[35121]: Deploying daemon mds.j on smithi026 2024-09-11T23:15:48.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:48 smithi136 ceph-mon[35121]: mds.? [v2:172.21.15.76:6832/3438147050,v1:172.21.15.76:6833/3438147050] up:boot 2024-09-11T23:15:48.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:48 smithi136 ceph-mon[35121]: fsmap 2 up:standby 2024-09-11T23:15:48.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:48 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-11T23:15:48.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:48 smithi026 ceph-mon[31172]: Deploying daemon mds.j on smithi026 2024-09-11T23:15:48.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:48 smithi026 ceph-mon[31172]: mds.? [v2:172.21.15.76:6832/3438147050,v1:172.21.15.76:6833/3438147050] up:boot 2024-09-11T23:15:48.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:48 smithi026 ceph-mon[31172]: fsmap 2 up:standby 2024-09-11T23:15:48.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:48 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-11T23:15:48.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:48 smithi076 ceph-mon[35296]: Deploying daemon mds.j on smithi026 2024-09-11T23:15:48.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:48 smithi076 ceph-mon[35296]: mds.? [v2:172.21.15.76:6832/3438147050,v1:172.21.15.76:6833/3438147050] up:boot 2024-09-11T23:15:48.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:48 smithi076 ceph-mon[35296]: fsmap 2 up:standby 2024-09-11T23:15:48.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:48 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-11T23:15:49.501 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:49 smithi026 ceph-mon[31172]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:15:49.552 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:49 smithi076 ceph-mon[35296]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:15:49.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:49 smithi136 ceph-mon[35121]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:15:49.939 INFO:journalctl@ceph.mds.j.smithi026.stdout:Sep 11 23:15:49 smithi026 systemd[1]: Starting Ceph mds.j for af7e6fc8-7092-11ef-bcea-c7b262605968... 2024-09-11T23:15:50.037 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:15:50.199 INFO:journalctl@ceph.mds.j.smithi026.stdout:Sep 11 23:15:49 smithi026 podman[70901]: 2024-09-11 23:15:49.841749615 +0000 UTC m=+0.016525025 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T23:15:50.199 INFO:journalctl@ceph.mds.j.smithi026.stdout:Sep 11 23:15:49 smithi026 podman[70901]: 2024-09-11 23:15:49.963680067 +0000 UTC m=+0.138455509 container create f4bc32015a30ec9583fe84c7986e61c4daae17b7096e52e1613908266c88eb07 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mds-j, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-09-11T23:15:50.199 INFO:journalctl@ceph.mds.j.smithi026.stdout:Sep 11 23:15:50 smithi026 podman[70901]: 2024-09-11 23:15:50.199586466 +0000 UTC m=+0.374361864 container init f4bc32015a30ec9583fe84c7986e61c4daae17b7096e52e1613908266c88eb07 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mds-j, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, ceph=True, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-09-11T23:15:50.408 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:15:50.501 INFO:journalctl@ceph.mds.j.smithi026.stdout:Sep 11 23:15:50 smithi026 podman[70901]: 2024-09-11 23:15:50.202674346 +0000 UTC m=+0.377449744 container start f4bc32015a30ec9583fe84c7986e61c4daae17b7096e52e1613908266c88eb07 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mds-j, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, ceph=True, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T23:15:50.501 INFO:journalctl@ceph.mds.j.smithi026.stdout:Sep 11 23:15:50 smithi026 ceph-mds[70938]: -- 172.21.15.26:0/3391224865 <== mon.0 v2:172.21.15.26:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558922cebc20 con 0x558922cc9000 2024-09-11T23:15:50.501 INFO:journalctl@ceph.mds.j.smithi026.stdout:Sep 11 23:15:50 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mds-j[70934]: starting mds.j at 2024-09-11T23:15:50.501 INFO:journalctl@ceph.mds.j.smithi026.stdout:Sep 11 23:15:50 smithi026 bash[70901]: f4bc32015a30ec9583fe84c7986e61c4daae17b7096e52e1613908266c88eb07 2024-09-11T23:15:50.501 INFO:journalctl@ceph.mds.j.smithi026.stdout:Sep 11 23:15:50 smithi026 systemd[1]: Started Ceph mds.j for af7e6fc8-7092-11ef-bcea-c7b262605968. 2024-09-11T23:15:50.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:50.894+0000 7f228cd44640 1 -- 172.21.15.26:0/2930658597 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2288109120 msgr2=0x7f2288110ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:15:50.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:50.894+0000 7f228cd44640 1 --2- 172.21.15.26:0/2930658597 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2288109120 0x7f2288110ca0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f2280009ca0 tx=0x7f228002a8c0 comp rx=0 tx=0).stop 2024-09-11T23:15:50.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:50.895+0000 7f228cd44640 1 -- 172.21.15.26:0/2930658597 shutdown_connections 2024-09-11T23:15:50.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:50.895+0000 7f228cd44640 1 --2- 172.21.15.26:0/2930658597 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f22881049a0 0x7f22881089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:15:50.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:50.895+0000 7f228cd44640 1 --2- 172.21.15.26:0/2930658597 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2288109120 0x7f2288110ca0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:15:50.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:50.895+0000 7f228cd44640 1 --2- 172.21.15.26:0/2930658597 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2288103ff0 0x7f22881043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:15:50.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:50.895+0000 7f228cd44640 1 -- 172.21.15.26:0/2930658597 >> 172.21.15.26:0/2930658597 conn(0x7f22880ffab0 msgr2=0x7f2288101ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:15:50.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:50.895+0000 7f228cd44640 1 -- 172.21.15.26:0/2930658597 shutdown_connections 2024-09-11T23:15:50.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:50.895+0000 7f228cd44640 1 -- 172.21.15.26:0/2930658597 wait complete. 2024-09-11T23:15:50.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:50.895+0000 7f228cd44640 1 Processor -- start 2024-09-11T23:15:50.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:50.895+0000 7f228cd44640 1 -- start start 2024-09-11T23:15:50.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:50.896+0000 7f228cd44640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2288103ff0 0x7f228819a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:15:50.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:50.896+0000 7f228cd44640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f22881049a0 0x7f228819abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:15:50.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:50.896+0000 7f228cd44640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2288109120 0x7f228819ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:15:50.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:50.896+0000 7f228cd44640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f2288114130 con 0x7f2288109120 2024-09-11T23:15:50.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:50.896+0000 7f228cd44640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f2288113fb0 con 0x7f22881049a0 2024-09-11T23:15:50.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:50.896+0000 7f228cd44640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f22881142b0 con 0x7f2288103ff0 2024-09-11T23:15:50.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:50.896+0000 7f2286d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2288109120 0x7f228819ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:15:50.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:50.896+0000 7f2286d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2288109120 0x7f228819ef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35576/0 (socket says 172.21.15.26:35576) 2024-09-11T23:15:50.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:50.896+0000 7f2286d9d640 1 -- 172.21.15.26:0/301918559 learned_addr learned my addr 172.21.15.26:0/301918559 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:15:50.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:50.896+0000 7f2285d9b640 1 --2- 172.21.15.26:0/301918559 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f22881049a0 0x7f228819abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:15:50.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:50.896+0000 7f228659c640 1 --2- 172.21.15.26:0/301918559 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2288103ff0 0x7f228819a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:15:50.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:50.896+0000 7f2285d9b640 1 -- 172.21.15.26:0/301918559 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2288103ff0 msgr2=0x7f228819a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:15:50.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:50.896+0000 7f2285d9b640 1 --2- 172.21.15.26:0/301918559 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2288103ff0 0x7f228819a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:15:50.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:50.896+0000 7f2285d9b640 1 -- 172.21.15.26:0/301918559 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2288109120 msgr2=0x7f228819ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:15:50.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:50.896+0000 7f2285d9b640 1 --2- 172.21.15.26:0/301918559 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2288109120 0x7f228819ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:15:50.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:50.896+0000 7f2285d9b640 1 -- 172.21.15.26:0/301918559 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f228819f640 con 0x7f22881049a0 2024-09-11T23:15:50.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:50.896+0000 7f2286d9d640 1 --2- 172.21.15.26:0/301918559 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2288109120 0x7f228819ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:15:50.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:50.896+0000 7f2285d9b640 1 --2- 172.21.15.26:0/301918559 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f22881049a0 0x7f228819abd0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f227c00d6d0 tx=0x7f227c00db90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:15:50.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:50.897+0000 7f22777fe640 1 -- 172.21.15.26:0/301918559 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f227c017070 con 0x7f22881049a0 2024-09-11T23:15:50.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:50.897+0000 7f228cd44640 1 -- 172.21.15.26:0/301918559 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f228819f930 con 0x7f22881049a0 2024-09-11T23:15:50.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:50.897+0000 7f22777fe640 1 -- 172.21.15.26:0/301918559 <== mon.2 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f227c009820 con 0x7f22881049a0 2024-09-11T23:15:50.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:50.897+0000 7f22777fe640 1 -- 172.21.15.26:0/301918559 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f227c004cd0 con 0x7f22881049a0 2024-09-11T23:15:50.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:50.897+0000 7f228cd44640 1 -- 172.21.15.26:0/301918559 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22881a7210 con 0x7f22881049a0 2024-09-11T23:15:50.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:50.898+0000 7f228cd44640 1 -- 172.21.15.26:0/301918559 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22881043d0 con 0x7f22881049a0 2024-09-11T23:15:50.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:50.899+0000 7f22777fe640 1 -- 172.21.15.26:0/301918559 <== mon.2 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f227c0099d0 con 0x7f22881049a0 2024-09-11T23:15:50.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:50.899+0000 7f22777fe640 1 --2- 172.21.15.26:0/301918559 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f22700a2b80 0x7f22700a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:15:50.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:50.900+0000 7f228659c640 1 --2- 172.21.15.26:0/301918559 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f22700a2b80 0x7f22700a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:15:50.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:50.900+0000 7f22777fe640 1 -- 172.21.15.26:0/301918559 <== mon.2 v2:172.21.15.76:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7127+0+0 (secure 0 0 0) 0x7f227c0ce610 con 0x7f22881049a0 2024-09-11T23:15:50.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:50.900+0000 7f228659c640 1 --2- 172.21.15.26:0/301918559 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f22700a2b80 0x7f22700a5040 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f2278005d00 tx=0x7f227800b410 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:15:50.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:50.902+0000 7f22777fe640 1 -- 172.21.15.26:0/301918559 <== mon.2 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f227c09b3a0 con 0x7f22881049a0 2024-09-11T23:15:51.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:51.116+0000 7f228cd44640 1 -- 172.21.15.26:0/301918559 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f228819b9f0 con 0x7f22881049a0 2024-09-11T23:15:51.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:51.117+0000 7f22777fe640 1 -- 172.21.15.26:0/301918559 <== mon.2 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3085 (secure 0 0 0) 0x7f227c09f6a0 con 0x7f22881049a0 2024-09-11T23:15:51.117 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-11T23:15:51.117 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":4,"btime":"2024-09-11T23:15:50:331206+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14499,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/367659515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":367659515},{"type":"v1","addr":"172.21.15.26:6835","nonce":367659515}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6833/2019841455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":2019841455},{"type":"v1","addr":"172.21.15.136:6833","nonce":2019841455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6833/3438147050","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3438147050},{"type":"v1","addr":"172.21.15.76:6833","nonce":3438147050}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-09-11T23:15:51.118 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 4 2024-09-11T23:15:51.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:51.119+0000 7f228cd44640 1 -- 172.21.15.26:0/301918559 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f22700a2b80 msgr2=0x7f22700a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:15:51.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:51.119+0000 7f228cd44640 1 --2- 172.21.15.26:0/301918559 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f22700a2b80 0x7f22700a5040 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f2278005d00 tx=0x7f227800b410 comp rx=0 tx=0).stop 2024-09-11T23:15:51.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:51.119+0000 7f228cd44640 1 -- 172.21.15.26:0/301918559 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f22881049a0 msgr2=0x7f228819abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:15:51.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:51.119+0000 7f228cd44640 1 --2- 172.21.15.26:0/301918559 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f22881049a0 0x7f228819abd0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f227c00d6d0 tx=0x7f227c00db90 comp rx=0 tx=0).stop 2024-09-11T23:15:51.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:51.119+0000 7f228cd44640 1 -- 172.21.15.26:0/301918559 shutdown_connections 2024-09-11T23:15:51.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:51.119+0000 7f228cd44640 1 --2- 172.21.15.26:0/301918559 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f22700a2b80 0x7f22700a5040 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:15:51.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:51.119+0000 7f228cd44640 1 --2- 172.21.15.26:0/301918559 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f22881049a0 0x7f228819abd0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:15:51.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:51.119+0000 7f228cd44640 1 --2- 172.21.15.26:0/301918559 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2288109120 0x7f228819ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:15:51.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:51.119+0000 7f228cd44640 1 --2- 172.21.15.26:0/301918559 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2288103ff0 0x7f228819a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:15:51.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:51.119+0000 7f228cd44640 1 -- 172.21.15.26:0/301918559 >> 172.21.15.26:0/301918559 conn(0x7f22880ffab0 msgr2=0x7f2288100dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:15:51.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:51.119+0000 7f228cd44640 1 -- 172.21.15.26:0/301918559 shutdown_connections 2024-09-11T23:15:51.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:51.119+0000 7f228cd44640 1 -- 172.21.15.26:0/301918559 wait complete. 2024-09-11T23:15:51.649 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:51 smithi026 ceph-mon[31172]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:15:51.649 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:51 smithi026 ceph-mon[31172]: mds.? [v2:172.21.15.26:6834/367659515,v1:172.21.15.26:6835/367659515] up:boot 2024-09-11T23:15:51.649 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:51 smithi026 ceph-mon[31172]: fsmap 3 up:standby 2024-09-11T23:15:51.649 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:51 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-11T23:15:51.649 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:51 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:15:51.649 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:51 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:15:51.649 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:51 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:15:51.650 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:51 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T23:15:51.650 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:51 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T23:15:51.650 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:51 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:15:51.650 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:51 smithi026 ceph-mon[31172]: Deploying daemon mds.d on smithi026 2024-09-11T23:15:51.650 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:51 smithi026 ceph-mon[31172]: from='client.? 172.21.15.26:0/301918559' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:15:51.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:51 smithi136 ceph-mon[35121]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:15:51.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:51 smithi136 ceph-mon[35121]: mds.? [v2:172.21.15.26:6834/367659515,v1:172.21.15.26:6835/367659515] up:boot 2024-09-11T23:15:51.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:51 smithi136 ceph-mon[35121]: fsmap 3 up:standby 2024-09-11T23:15:51.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:51 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-11T23:15:51.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:51 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:15:51.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:51 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:15:51.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:51 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:15:51.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:51 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T23:15:51.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:51 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T23:15:51.741 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:51 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:15:51.741 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:51 smithi136 ceph-mon[35121]: Deploying daemon mds.d on smithi026 2024-09-11T23:15:51.741 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:51 smithi136 ceph-mon[35121]: from='client.? 172.21.15.26:0/301918559' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:15:51.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:51 smithi076 ceph-mon[35296]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:15:51.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:51 smithi076 ceph-mon[35296]: mds.? [v2:172.21.15.26:6834/367659515,v1:172.21.15.26:6835/367659515] up:boot 2024-09-11T23:15:51.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:51 smithi076 ceph-mon[35296]: fsmap 3 up:standby 2024-09-11T23:15:51.804 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:51 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-11T23:15:51.804 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:51 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:15:51.804 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:51 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:15:51.804 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:51 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:15:51.804 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:51 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T23:15:51.805 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:51 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T23:15:51.805 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:51 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:15:51.805 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:51 smithi076 ceph-mon[35296]: Deploying daemon mds.d on smithi026 2024-09-11T23:15:51.805 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:51 smithi076 ceph-mon[35296]: from='client.? 172.21.15.26:0/301918559' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:15:53.238 INFO:journalctl@ceph.mds.d.smithi026.stdout:Sep 11 23:15:53 smithi026 systemd[1]: Starting Ceph mds.d for af7e6fc8-7092-11ef-bcea-c7b262605968... 2024-09-11T23:15:53.501 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:53 smithi026 ceph-mon[31172]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:15:53.501 INFO:journalctl@ceph.mds.d.smithi026.stdout:Sep 11 23:15:53 smithi026 podman[71318]: 2024-09-11 23:15:53.140883971 +0000 UTC m=+0.016306511 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T23:15:53.501 INFO:journalctl@ceph.mds.d.smithi026.stdout:Sep 11 23:15:53 smithi026 podman[71318]: 2024-09-11 23:15:53.283792732 +0000 UTC m=+0.159215261 container create 25e20ec815ee50f0cf3aa6bd444631953c5d3e6aa8f21635da503a6ccc770f96 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mds-d, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2024-09-11T23:15:53.751 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:53 smithi136 ceph-mon[35121]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:15:53.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:53 smithi076 ceph-mon[35296]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:15:54.001 INFO:journalctl@ceph.mds.d.smithi026.stdout:Sep 11 23:15:53 smithi026 podman[71318]: 2024-09-11 23:15:53.519738458 +0000 UTC m=+0.395161004 container init 25e20ec815ee50f0cf3aa6bd444631953c5d3e6aa8f21635da503a6ccc770f96 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mds-d, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, ceph=True, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-09-11T23:15:54.001 INFO:journalctl@ceph.mds.d.smithi026.stdout:Sep 11 23:15:53 smithi026 podman[71318]: 2024-09-11 23:15:53.522634345 +0000 UTC m=+0.398056882 container start 25e20ec815ee50f0cf3aa6bd444631953c5d3e6aa8f21635da503a6ccc770f96 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mds-d, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-09-11T23:15:54.001 INFO:journalctl@ceph.mds.d.smithi026.stdout:Sep 11 23:15:53 smithi026 ceph-mds[71337]: -- 172.21.15.26:0/1003235109 <== mon.0 v2:172.21.15.26:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55fc59f99860 con 0x55fc59f76800 2024-09-11T23:15:54.001 INFO:journalctl@ceph.mds.d.smithi026.stdout:Sep 11 23:15:53 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mds-d[71333]: starting mds.d at 2024-09-11T23:15:54.001 INFO:journalctl@ceph.mds.d.smithi026.stdout:Sep 11 23:15:53 smithi026 bash[71318]: 25e20ec815ee50f0cf3aa6bd444631953c5d3e6aa8f21635da503a6ccc770f96 2024-09-11T23:15:54.001 INFO:journalctl@ceph.mds.d.smithi026.stdout:Sep 11 23:15:53 smithi026 systemd[1]: Started Ceph mds.d for af7e6fc8-7092-11ef-bcea-c7b262605968. 2024-09-11T23:15:54.945 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:15:54.975 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:54 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:15:54.975 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:54 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:15:54.975 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:54 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:15:54.976 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:54 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T23:15:54.976 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:54 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T23:15:54.976 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:54 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:15:54.976 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:54 smithi026 ceph-mon[31172]: Deploying daemon mds.e on smithi076 2024-09-11T23:15:54.976 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:54 smithi026 ceph-mon[31172]: mds.? [v2:172.21.15.26:6836/2041065299,v1:172.21.15.26:6837/2041065299] up:boot 2024-09-11T23:15:54.976 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:54 smithi026 ceph-mon[31172]: fsmap 4 up:standby 2024-09-11T23:15:54.976 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:54 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-11T23:15:54.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:54 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:15:54.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:54 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:15:54.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:54 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:15:54.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:54 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T23:15:54.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:54 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T23:15:54.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:54 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:15:54.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:54 smithi136 ceph-mon[35121]: Deploying daemon mds.e on smithi076 2024-09-11T23:15:54.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:54 smithi136 ceph-mon[35121]: mds.? [v2:172.21.15.26:6836/2041065299,v1:172.21.15.26:6837/2041065299] up:boot 2024-09-11T23:15:54.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:54 smithi136 ceph-mon[35121]: fsmap 4 up:standby 2024-09-11T23:15:54.990 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:54 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-11T23:15:55.048 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:54 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:15:55.049 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:54 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:15:55.049 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:54 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:15:55.049 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:54 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T23:15:55.049 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:54 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T23:15:55.049 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:54 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:15:55.049 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:54 smithi076 ceph-mon[35296]: Deploying daemon mds.e on smithi076 2024-09-11T23:15:55.049 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:54 smithi076 ceph-mon[35296]: mds.? [v2:172.21.15.26:6836/2041065299,v1:172.21.15.26:6837/2041065299] up:boot 2024-09-11T23:15:55.049 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:54 smithi076 ceph-mon[35296]: fsmap 4 up:standby 2024-09-11T23:15:55.049 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:54 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-11T23:15:55.220 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:15:55.685 INFO:journalctl@ceph.mds.e.smithi076.stdout:Sep 11 23:15:55 smithi076 systemd[1]: Starting Ceph mds.e for af7e6fc8-7092-11ef-bcea-c7b262605968... 2024-09-11T23:15:55.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:55.671+0000 7fefaf557640 1 -- 172.21.15.26:0/1151665000 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fefa810e480 msgr2=0x7fefa8110870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:15:55.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:55.671+0000 7fefaf557640 1 --2- 172.21.15.26:0/1151665000 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fefa810e480 0x7fefa8110870 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fefa0009ca0 tx=0x7fefa002a8c0 comp rx=0 tx=0).stop 2024-09-11T23:15:55.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:55.672+0000 7fefaf557640 1 -- 172.21.15.26:0/1151665000 shutdown_connections 2024-09-11T23:15:55.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:55.672+0000 7fefaf557640 1 --2- 172.21.15.26:0/1151665000 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fefa810e480 0x7fefa8110870 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:15:55.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:55.672+0000 7fefaf557640 1 --2- 172.21.15.26:0/1151665000 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fefa8101280 0x7fefa810df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:15:55.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:55.672+0000 7fefaf557640 1 --2- 172.21.15.26:0/1151665000 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fefa81008d0 0x7fefa8100cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:15:55.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:55.672+0000 7fefaf557640 1 -- 172.21.15.26:0/1151665000 >> 172.21.15.26:0/1151665000 conn(0x7fefa80fc7a0 msgr2=0x7fefa80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:15:55.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:55.672+0000 7fefaf557640 1 -- 172.21.15.26:0/1151665000 shutdown_connections 2024-09-11T23:15:55.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:55.672+0000 7fefaf557640 1 -- 172.21.15.26:0/1151665000 wait complete. 2024-09-11T23:15:55.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:55.672+0000 7fefaf557640 1 Processor -- start 2024-09-11T23:15:55.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:55.672+0000 7fefaf557640 1 -- start start 2024-09-11T23:15:55.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:55.672+0000 7fefaf557640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fefa81008d0 0x7fefa819a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:15:55.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:55.672+0000 7fefaf557640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fefa8101280 0x7fefa819ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:15:55.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:55.672+0000 7fefaf557640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fefa810e480 0x7fefa819efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:15:55.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:55.672+0000 7fefaf557640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fefa81141c0 con 0x7fefa8101280 2024-09-11T23:15:55.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:55.672+0000 7fefaf557640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fefa8114040 con 0x7fefa81008d0 2024-09-11T23:15:55.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:55.672+0000 7fefaf557640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fefa8114340 con 0x7fefa810e480 2024-09-11T23:15:55.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:55.673+0000 7fefacaf2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fefa8101280 0x7fefa819ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:15:55.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:55.673+0000 7fefacaf2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fefa8101280 0x7fefa819ac20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35620/0 (socket says 172.21.15.26:35620) 2024-09-11T23:15:55.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:55.673+0000 7fefacaf2640 1 -- 172.21.15.26:0/3977378118 learned_addr learned my addr 172.21.15.26:0/3977378118 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:15:55.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:55.673+0000 7fefad2f3640 1 --2- 172.21.15.26:0/3977378118 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fefa81008d0 0x7fefa819a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:15:55.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:55.673+0000 7fefadaf4640 1 --2- 172.21.15.26:0/3977378118 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fefa810e480 0x7fefa819efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:15:55.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:55.673+0000 7fefacaf2640 1 -- 172.21.15.26:0/3977378118 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fefa810e480 msgr2=0x7fefa819efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:15:55.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:55.673+0000 7fefacaf2640 1 --2- 172.21.15.26:0/3977378118 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fefa810e480 0x7fefa819efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:15:55.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:55.673+0000 7fefacaf2640 1 -- 172.21.15.26:0/3977378118 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fefa81008d0 msgr2=0x7fefa819a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:15:55.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:55.673+0000 7fefacaf2640 1 --2- 172.21.15.26:0/3977378118 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fefa81008d0 0x7fefa819a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:15:55.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:55.673+0000 7fefacaf2640 1 -- 172.21.15.26:0/3977378118 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefa819f690 con 0x7fefa8101280 2024-09-11T23:15:55.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:55.673+0000 7fefacaf2640 1 --2- 172.21.15.26:0/3977378118 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fefa8101280 0x7fefa819ac20 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fef9800b4a0 tx=0x7fef9800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:15:55.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:55.673+0000 7fef9e7fc640 1 -- 172.21.15.26:0/3977378118 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef98014070 con 0x7fefa8101280 2024-09-11T23:15:55.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:55.673+0000 7fef9e7fc640 1 -- 172.21.15.26:0/3977378118 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef98010af0 con 0x7fefa8101280 2024-09-11T23:15:55.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:55.673+0000 7fefaf557640 1 -- 172.21.15.26:0/3977378118 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefa819f980 con 0x7fefa8101280 2024-09-11T23:15:55.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:55.673+0000 7fef9e7fc640 1 -- 172.21.15.26:0/3977378118 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef98017d10 con 0x7fefa8101280 2024-09-11T23:15:55.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:55.673+0000 7fefaf557640 1 -- 172.21.15.26:0/3977378118 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefa81a7260 con 0x7fefa8101280 2024-09-11T23:15:55.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:55.674+0000 7fefaf557640 1 -- 172.21.15.26:0/3977378118 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fefa8100cb0 con 0x7fefa8101280 2024-09-11T23:15:55.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:55.675+0000 7fef9e7fc640 1 -- 172.21.15.26:0/3977378118 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fef98017070 con 0x7fefa8101280 2024-09-11T23:15:55.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:55.676+0000 7fef9e7fc640 1 --2- 172.21.15.26:0/3977378118 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fef940a2b80 0x7fef940a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:15:55.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:55.676+0000 7fefad2f3640 1 --2- 172.21.15.26:0/3977378118 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fef940a2b80 0x7fef940a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:15:55.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:55.676+0000 7fef9e7fc640 1 -- 172.21.15.26:0/3977378118 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7127+0+0 (secure 0 0 0) 0x7fef980cdf90 con 0x7fefa8101280 2024-09-11T23:15:55.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:55.677+0000 7fefad2f3640 1 --2- 172.21.15.26:0/3977378118 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fef940a2b80 0x7fef940a5040 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fefa400bbe0 tx=0x7fefa400b390 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:15:55.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:55.680+0000 7fef9e7fc640 1 -- 172.21.15.26:0/3977378118 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fef9809b370 con 0x7fefa8101280 2024-09-11T23:15:55.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:55.894+0000 7fefaf557640 1 -- 172.21.15.26:0/3977378118 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fefa8109080 con 0x7fefa8101280 2024-09-11T23:15:55.894 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:55 smithi026 ceph-mon[31172]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:15:55.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:55.894+0000 7fef9e7fc640 1 -- 172.21.15.26:0/3977378118 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3910 (secure 0 0 0) 0x7fef9809ac70 con 0x7fefa8101280 2024-09-11T23:15:55.895 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-11T23:15:55.895 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":5,"btime":"2024-09-11T23:15:54:340127+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14499,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/367659515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":367659515},{"type":"v1","addr":"172.21.15.26:6835","nonce":367659515}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14511,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/2041065299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":2041065299},{"type":"v1","addr":"172.21.15.26:6837","nonce":2041065299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6833/2019841455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":2019841455},{"type":"v1","addr":"172.21.15.136:6833","nonce":2019841455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6833/3438147050","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3438147050},{"type":"v1","addr":"172.21.15.76:6833","nonce":3438147050}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-09-11T23:15:55.896 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 5 2024-09-11T23:15:55.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:55.897+0000 7fefaf557640 1 -- 172.21.15.26:0/3977378118 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fef940a2b80 msgr2=0x7fef940a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:15:55.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:55.897+0000 7fefaf557640 1 --2- 172.21.15.26:0/3977378118 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fef940a2b80 0x7fef940a5040 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fefa400bbe0 tx=0x7fefa400b390 comp rx=0 tx=0).stop 2024-09-11T23:15:55.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:55.897+0000 7fefaf557640 1 -- 172.21.15.26:0/3977378118 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fefa8101280 msgr2=0x7fefa819ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:15:55.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:55.897+0000 7fefaf557640 1 --2- 172.21.15.26:0/3977378118 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fefa8101280 0x7fefa819ac20 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fef9800b4a0 tx=0x7fef9800b960 comp rx=0 tx=0).stop 2024-09-11T23:15:55.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:55.897+0000 7fefaf557640 1 -- 172.21.15.26:0/3977378118 shutdown_connections 2024-09-11T23:15:55.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:55.897+0000 7fefaf557640 1 --2- 172.21.15.26:0/3977378118 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fef940a2b80 0x7fef940a5040 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:15:55.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:55.897+0000 7fefaf557640 1 --2- 172.21.15.26:0/3977378118 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fefa810e480 0x7fefa819efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:15:55.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:55.897+0000 7fefaf557640 1 --2- 172.21.15.26:0/3977378118 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fefa8101280 0x7fefa819ac20 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:15:55.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:55.897+0000 7fefaf557640 1 --2- 172.21.15.26:0/3977378118 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fefa81008d0 0x7fefa819a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:15:55.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:55.897+0000 7fefaf557640 1 -- 172.21.15.26:0/3977378118 >> 172.21.15.26:0/3977378118 conn(0x7fefa80fc7a0 msgr2=0x7fefa80feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:15:55.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:55.897+0000 7fefaf557640 1 -- 172.21.15.26:0/3977378118 shutdown_connections 2024-09-11T23:15:55.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:55.897+0000 7fefaf557640 1 -- 172.21.15.26:0/3977378118 wait complete. 2024-09-11T23:15:55.970 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:55 smithi076 ceph-mon[35296]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:15:55.970 INFO:journalctl@ceph.mds.e.smithi076.stdout:Sep 11 23:15:55 smithi076 podman[64702]: 2024-09-11 23:15:55.633748293 +0000 UTC m=+0.016079634 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T23:15:55.970 INFO:journalctl@ceph.mds.e.smithi076.stdout:Sep 11 23:15:55 smithi076 podman[64702]: 2024-09-11 23:15:55.751607606 +0000 UTC m=+0.133938928 container create 6f0b7c1d8960b84a39a890e90d7721272855c7ff693dec739914568fa7c8bb1b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mds-e, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-09-11T23:15:55.971 INFO:journalctl@ceph.mds.e.smithi076.stdout:Sep 11 23:15:55 smithi076 podman[64702]: 2024-09-11 23:15:55.970822077 +0000 UTC m=+0.353153403 container init 6f0b7c1d8960b84a39a890e90d7721272855c7ff693dec739914568fa7c8bb1b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mds-e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0) 2024-09-11T23:15:55.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:55 smithi136 ceph-mon[35121]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:15:56.303 INFO:journalctl@ceph.mds.e.smithi076.stdout:Sep 11 23:15:55 smithi076 podman[64702]: 2024-09-11 23:15:55.97366814 +0000 UTC m=+0.355999472 container start 6f0b7c1d8960b84a39a890e90d7721272855c7ff693dec739914568fa7c8bb1b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mds-e, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T23:15:56.303 INFO:journalctl@ceph.mds.e.smithi076.stdout:Sep 11 23:15:55 smithi076 ceph-mds[64721]: -- 172.21.15.76:0/1603942503 <== mon.2 v2:172.21.15.76:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e73dd35860 con 0x55e73dd13400 2024-09-11T23:15:56.303 INFO:journalctl@ceph.mds.e.smithi076.stdout:Sep 11 23:15:55 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mds-e[64717]: starting mds.e at 2024-09-11T23:15:56.303 INFO:journalctl@ceph.mds.e.smithi076.stdout:Sep 11 23:15:56 smithi076 bash[64702]: 6f0b7c1d8960b84a39a890e90d7721272855c7ff693dec739914568fa7c8bb1b 2024-09-11T23:15:56.303 INFO:journalctl@ceph.mds.e.smithi076.stdout:Sep 11 23:15:56 smithi076 systemd[1]: Started Ceph mds.e for af7e6fc8-7092-11ef-bcea-c7b262605968. 2024-09-11T23:15:56.814 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:56 smithi136 ceph-mon[35121]: from='client.? 172.21.15.26:0/3977378118' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:15:56.814 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:56 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:15:56.814 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:56 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:15:56.814 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:56 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:15:56.814 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:56 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T23:15:56.814 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:56 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T23:15:56.815 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:56 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:15:57.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:56 smithi026 ceph-mon[31172]: from='client.? 172.21.15.26:0/3977378118' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:15:57.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:56 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:15:57.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:56 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:15:57.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:56 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:15:57.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:56 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T23:15:57.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:56 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T23:15:57.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:56 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:15:57.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:56 smithi076 ceph-mon[35296]: from='client.? 172.21.15.26:0/3977378118' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:15:57.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:56 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:15:57.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:56 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:15:57.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:56 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:15:57.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:56 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T23:15:57.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:56 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T23:15:57.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:56 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:15:57.848 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:57 smithi136 ceph-mon[35121]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:15:57.848 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:57 smithi136 ceph-mon[35121]: Deploying daemon mds.f on smithi136 2024-09-11T23:15:57.849 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:57 smithi136 ceph-mon[35121]: mds.? [v2:172.21.15.76:6834/393308499,v1:172.21.15.76:6835/393308499] up:boot 2024-09-11T23:15:57.849 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:57 smithi136 ceph-mon[35121]: fsmap 5 up:standby 2024-09-11T23:15:57.849 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:57 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-11T23:15:58.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:57 smithi026 ceph-mon[31172]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:15:58.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:57 smithi026 ceph-mon[31172]: Deploying daemon mds.f on smithi136 2024-09-11T23:15:58.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:57 smithi026 ceph-mon[31172]: mds.? [v2:172.21.15.76:6834/393308499,v1:172.21.15.76:6835/393308499] up:boot 2024-09-11T23:15:58.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:57 smithi026 ceph-mon[31172]: fsmap 5 up:standby 2024-09-11T23:15:58.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:57 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-11T23:15:58.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:57 smithi076 ceph-mon[35296]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:15:58.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:57 smithi076 ceph-mon[35296]: Deploying daemon mds.f on smithi136 2024-09-11T23:15:58.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:57 smithi076 ceph-mon[35296]: mds.? [v2:172.21.15.76:6834/393308499,v1:172.21.15.76:6835/393308499] up:boot 2024-09-11T23:15:58.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:57 smithi076 ceph-mon[35296]: fsmap 5 up:standby 2024-09-11T23:15:58.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:57 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-11T23:15:58.239 INFO:journalctl@ceph.mds.f.smithi136.stdout:Sep 11 23:15:58 smithi136 systemd[1]: Starting Ceph mds.f for af7e6fc8-7092-11ef-bcea-c7b262605968... 2024-09-11T23:15:58.504 INFO:journalctl@ceph.mds.f.smithi136.stdout:Sep 11 23:15:58 smithi136 podman[64724]: 2024-09-11 23:15:58.142351735 +0000 UTC m=+0.015763874 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T23:15:58.504 INFO:journalctl@ceph.mds.f.smithi136.stdout:Sep 11 23:15:58 smithi136 podman[64724]: 2024-09-11 23:15:58.250234615 +0000 UTC m=+0.123646793 container create 64c00ef5e0989f7f79ee650d059f8fb5502a07b80d397cac1f969db4e5ea9301 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mds-f, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux ) 2024-09-11T23:15:58.504 INFO:journalctl@ceph.mds.f.smithi136.stdout:Sep 11 23:15:58 smithi136 podman[64724]: 2024-09-11 23:15:58.477886117 +0000 UTC m=+0.351298249 container init 64c00ef5e0989f7f79ee650d059f8fb5502a07b80d397cac1f969db4e5ea9301 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mds-f, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T23:15:58.504 INFO:journalctl@ceph.mds.f.smithi136.stdout:Sep 11 23:15:58 smithi136 podman[64724]: 2024-09-11 23:15:58.480569463 +0000 UTC m=+0.353981603 container start 64c00ef5e0989f7f79ee650d059f8fb5502a07b80d397cac1f969db4e5ea9301 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mds-f, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T23:15:58.505 INFO:journalctl@ceph.mds.f.smithi136.stdout:Sep 11 23:15:58 smithi136 ceph-mds[64743]: -- 172.21.15.136:0/97702134 <== mon.1 v2:172.21.15.136:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55be03fd9860 con 0x55be03fb7400 2024-09-11T23:15:58.684 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:15:58.953 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:15:58.989 INFO:journalctl@ceph.mds.f.smithi136.stdout:Sep 11 23:15:58 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mds-f[64739]: starting mds.f at 2024-09-11T23:15:58.989 INFO:journalctl@ceph.mds.f.smithi136.stdout:Sep 11 23:15:58 smithi136 bash[64724]: 64c00ef5e0989f7f79ee650d059f8fb5502a07b80d397cac1f969db4e5ea9301 2024-09-11T23:15:58.989 INFO:journalctl@ceph.mds.f.smithi136.stdout:Sep 11 23:15:58 smithi136 systemd[1]: Started Ceph mds.f for af7e6fc8-7092-11ef-bcea-c7b262605968. 2024-09-11T23:15:59.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:59.427+0000 7f637b7e5640 1 -- 172.21.15.26:0/47974748 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6374074590 msgr2=0x7f6374074970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:15:59.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:59.427+0000 7f637b7e5640 1 --2- 172.21.15.26:0/47974748 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6374074590 0x7f6374074970 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f6370009900 tx=0x7f637002a480 comp rx=0 tx=0).stop 2024-09-11T23:15:59.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:59.427+0000 7f637b7e5640 1 -- 172.21.15.26:0/47974748 shutdown_connections 2024-09-11T23:15:59.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:59.427+0000 7f637b7e5640 1 --2- 172.21.15.26:0/47974748 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6374112b90 0x7f6374114f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:15:59.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:59.427+0000 7f637b7e5640 1 --2- 172.21.15.26:0/47974748 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6374074eb0 0x7f6374112650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:15:59.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:59.427+0000 7f637b7e5640 1 --2- 172.21.15.26:0/47974748 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6374074590 0x7f6374074970 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:15:59.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:59.427+0000 7f637b7e5640 1 -- 172.21.15.26:0/47974748 >> 172.21.15.26:0/47974748 conn(0x7f63740fd8c0 msgr2=0x7f63740ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:15:59.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:59.427+0000 7f637b7e5640 1 -- 172.21.15.26:0/47974748 shutdown_connections 2024-09-11T23:15:59.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:59.427+0000 7f637b7e5640 1 -- 172.21.15.26:0/47974748 wait complete. 2024-09-11T23:15:59.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:59.427+0000 7f637b7e5640 1 Processor -- start 2024-09-11T23:15:59.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:59.427+0000 7f637b7e5640 1 -- start start 2024-09-11T23:15:59.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:59.428+0000 7f637b7e5640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6374074590 0x7f63741a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:15:59.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:59.428+0000 7f637b7e5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6374074eb0 0x7f63741a4be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:15:59.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:59.428+0000 7f637b7e5640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6374112b90 0x7f637419e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:15:59.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:59.428+0000 7f637b7e5640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f6374118550 con 0x7f6374074eb0 2024-09-11T23:15:59.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:59.428+0000 7f637b7e5640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f63741183d0 con 0x7f6374074590 2024-09-11T23:15:59.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:59.428+0000 7f637b7e5640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f63741186d0 con 0x7f6374112b90 2024-09-11T23:15:59.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:59.428+0000 7f6378d80640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6374074eb0 0x7f63741a4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:15:59.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:59.428+0000 7f6378d80640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6374074eb0 0x7f63741a4be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35640/0 (socket says 172.21.15.26:35640) 2024-09-11T23:15:59.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:59.428+0000 7f6378d80640 1 -- 172.21.15.26:0/2355566953 learned_addr learned my addr 172.21.15.26:0/2355566953 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:15:59.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:59.428+0000 7f6379581640 1 --2- 172.21.15.26:0/2355566953 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6374074590 0x7f63741a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:15:59.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:59.428+0000 7f6379d82640 1 --2- 172.21.15.26:0/2355566953 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6374112b90 0x7f637419e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:15:59.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:59.428+0000 7f6378d80640 1 -- 172.21.15.26:0/2355566953 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6374112b90 msgr2=0x7f637419e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:15:59.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:59.428+0000 7f6378d80640 1 --2- 172.21.15.26:0/2355566953 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6374112b90 0x7f637419e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:15:59.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:59.428+0000 7f6378d80640 1 -- 172.21.15.26:0/2355566953 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6374074590 msgr2=0x7f63741a46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:15:59.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:59.428+0000 7f6378d80640 1 --2- 172.21.15.26:0/2355566953 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6374074590 0x7f63741a46a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:15:59.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:59.428+0000 7f6378d80640 1 -- 172.21.15.26:0/2355566953 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f637419ef90 con 0x7f6374074eb0 2024-09-11T23:15:59.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:59.428+0000 7f6379581640 1 --2- 172.21.15.26:0/2355566953 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6374074590 0x7f63741a46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:15:59.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:59.428+0000 7f6378d80640 1 --2- 172.21.15.26:0/2355566953 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6374074eb0 0x7f63741a4be0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f636400c920 tx=0x7f636400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:15:59.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:59.428+0000 7f636a7fc640 1 -- 172.21.15.26:0/2355566953 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6364007c70 con 0x7f6374074eb0 2024-09-11T23:15:59.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:59.428+0000 7f637b7e5640 1 -- 172.21.15.26:0/2355566953 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f637419f280 con 0x7f6374074eb0 2024-09-11T23:15:59.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:59.428+0000 7f636a7fc640 1 -- 172.21.15.26:0/2355566953 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6364007e10 con 0x7f6374074eb0 2024-09-11T23:15:59.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:59.428+0000 7f636a7fc640 1 -- 172.21.15.26:0/2355566953 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6364018890 con 0x7f6374074eb0 2024-09-11T23:15:59.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:59.429+0000 7f637b7e5640 1 -- 172.21.15.26:0/2355566953 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63741ab570 con 0x7f6374074eb0 2024-09-11T23:15:59.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:59.429+0000 7f637b7e5640 1 -- 172.21.15.26:0/2355566953 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6374074970 con 0x7f6374074eb0 2024-09-11T23:15:59.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:59.431+0000 7f636a7fc640 1 -- 172.21.15.26:0/2355566953 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f6364018a70 con 0x7f6374074eb0 2024-09-11T23:15:59.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:59.432+0000 7f636a7fc640 1 --2- 172.21.15.26:0/2355566953 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f63600a2b80 0x7f63600a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:15:59.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:59.432+0000 7f6379581640 1 --2- 172.21.15.26:0/2355566953 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f63600a2b80 0x7f63600a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:15:59.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:59.432+0000 7f636a7fc640 1 -- 172.21.15.26:0/2355566953 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7127+0+0 (secure 0 0 0) 0x7f63640cf0a0 con 0x7f6374074eb0 2024-09-11T23:15:59.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:59.433+0000 7f6379581640 1 --2- 172.21.15.26:0/2355566953 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f63600a2b80 0x7f63600a5040 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f6370000c00 tx=0x7f6370005a20 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:15:59.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:59.435+0000 7f636a7fc640 1 -- 172.21.15.26:0/2355566953 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6364014030 con 0x7f6374074eb0 2024-09-11T23:15:59.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:59.649+0000 7f637b7e5640 1 -- 172.21.15.26:0/2355566953 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f637419fd90 con 0x7f6374074eb0 2024-09-11T23:15:59.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:59.649+0000 7f636a7fc640 1 -- 172.21.15.26:0/2355566953 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5560 (secure 0 0 0) 0x7f637419fd90 con 0x7f6374074eb0 2024-09-11T23:15:59.650 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-11T23:15:59.650 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":7,"btime":"2024-09-11T23:15:58:692360+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14499,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/367659515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":367659515},{"type":"v1","addr":"172.21.15.26:6835","nonce":367659515}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14511,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/2041065299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":2041065299},{"type":"v1","addr":"172.21.15.26:6837","nonce":2041065299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6833/2019841455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":2019841455},{"type":"v1","addr":"172.21.15.136:6833","nonce":2019841455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6833/3438147050","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3438147050},{"type":"v1","addr":"172.21.15.76:6833","nonce":3438147050}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24439,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/3617665975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3617665975},{"type":"v1","addr":"172.21.15.136:6835","nonce":3617665975}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24443,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/393308499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":393308499},{"type":"v1","addr":"172.21.15.76:6835","nonce":393308499}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-09-11T23:15:59.650 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 7 2024-09-11T23:15:59.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:59.652+0000 7f637b7e5640 1 -- 172.21.15.26:0/2355566953 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f63600a2b80 msgr2=0x7f63600a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:15:59.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:59.652+0000 7f637b7e5640 1 --2- 172.21.15.26:0/2355566953 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f63600a2b80 0x7f63600a5040 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f6370000c00 tx=0x7f6370005a20 comp rx=0 tx=0).stop 2024-09-11T23:15:59.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:59.652+0000 7f637b7e5640 1 -- 172.21.15.26:0/2355566953 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6374074eb0 msgr2=0x7f63741a4be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:15:59.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:59.652+0000 7f637b7e5640 1 --2- 172.21.15.26:0/2355566953 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6374074eb0 0x7f63741a4be0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f636400c920 tx=0x7f636400cde0 comp rx=0 tx=0).stop 2024-09-11T23:15:59.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:59.652+0000 7f637b7e5640 1 -- 172.21.15.26:0/2355566953 shutdown_connections 2024-09-11T23:15:59.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:59.652+0000 7f637b7e5640 1 --2- 172.21.15.26:0/2355566953 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f63600a2b80 0x7f63600a5040 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:15:59.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:59.652+0000 7f637b7e5640 1 --2- 172.21.15.26:0/2355566953 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6374112b90 0x7f637419e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:15:59.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:59.652+0000 7f637b7e5640 1 --2- 172.21.15.26:0/2355566953 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6374074eb0 0x7f63741a4be0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:15:59.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:59.652+0000 7f637b7e5640 1 --2- 172.21.15.26:0/2355566953 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6374074590 0x7f63741a46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:15:59.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:59.652+0000 7f637b7e5640 1 -- 172.21.15.26:0/2355566953 >> 172.21.15.26:0/2355566953 conn(0x7f63740fd8c0 msgr2=0x7f63740fefb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:15:59.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:59.652+0000 7f637b7e5640 1 -- 172.21.15.26:0/2355566953 shutdown_connections 2024-09-11T23:15:59.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:15:59.652+0000 7f637b7e5640 1 -- 172.21.15.26:0/2355566953 wait complete. 2024-09-11T23:15:59.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:59 smithi136 ceph-mon[35121]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:15:59.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:59 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:15:59.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:59 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:15:59.990 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:59 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:15:59.990 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:59 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T23:15:59.990 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:59 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T23:15:59.990 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:59 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:15:59.990 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:59 smithi136 ceph-mon[35121]: Deploying daemon mds.c on smithi136 2024-09-11T23:15:59.990 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:59 smithi136 ceph-mon[35121]: mds.? [v2:172.21.15.136:6834/3617665975,v1:172.21.15.136:6835/3617665975] up:boot 2024-09-11T23:15:59.990 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:59 smithi136 ceph-mon[35121]: fsmap 6 up:standby 2024-09-11T23:15:59.990 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:15:59 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-11T23:16:00.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:59 smithi026 ceph-mon[31172]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:16:00.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:59 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:16:00.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:59 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:16:00.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:59 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:16:00.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:59 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T23:16:00.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:59 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T23:16:00.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:59 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:16:00.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:59 smithi026 ceph-mon[31172]: Deploying daemon mds.c on smithi136 2024-09-11T23:16:00.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:59 smithi026 ceph-mon[31172]: mds.? [v2:172.21.15.136:6834/3617665975,v1:172.21.15.136:6835/3617665975] up:boot 2024-09-11T23:16:00.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:59 smithi026 ceph-mon[31172]: fsmap 6 up:standby 2024-09-11T23:16:00.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:15:59 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-11T23:16:00.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:59 smithi076 ceph-mon[35296]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:16:00.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:59 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:16:00.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:59 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:16:00.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:59 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:16:00.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:59 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T23:16:00.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:59 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T23:16:00.054 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:59 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:16:00.054 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:59 smithi076 ceph-mon[35296]: Deploying daemon mds.c on smithi136 2024-09-11T23:16:00.054 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:59 smithi076 ceph-mon[35296]: mds.? [v2:172.21.15.136:6834/3617665975,v1:172.21.15.136:6835/3617665975] up:boot 2024-09-11T23:16:00.054 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:59 smithi076 ceph-mon[35296]: fsmap 6 up:standby 2024-09-11T23:16:00.054 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:15:59 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-11T23:16:00.642 INFO:journalctl@ceph.mds.c.smithi136.stdout:Sep 11 23:16:00 smithi136 systemd[1]: Starting Ceph mds.c for af7e6fc8-7092-11ef-bcea-c7b262605968... 2024-09-11T23:16:00.955 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:00 smithi136 ceph-mon[35121]: from='client.? 172.21.15.26:0/2355566953' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:16:00.956 INFO:journalctl@ceph.mds.c.smithi136.stdout:Sep 11 23:16:00 smithi136 podman[65026]: 2024-09-11 23:16:00.630135715 +0000 UTC m=+0.016091211 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T23:16:00.956 INFO:journalctl@ceph.mds.c.smithi136.stdout:Sep 11 23:16:00 smithi136 podman[65026]: 2024-09-11 23:16:00.75412493 +0000 UTC m=+0.140080426 container create a8bc986f7080e19e4c4f2a05c015fdb63f204e4d1aafe91156eb8d50f14786ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mds-c, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0) 2024-09-11T23:16:01.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:00 smithi026 ceph-mon[31172]: from='client.? 172.21.15.26:0/2355566953' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:16:01.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:00 smithi076 ceph-mon[35296]: from='client.? 172.21.15.26:0/2355566953' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:16:01.239 INFO:journalctl@ceph.mds.c.smithi136.stdout:Sep 11 23:16:00 smithi136 podman[65026]: 2024-09-11 23:16:00.956511687 +0000 UTC m=+0.342467173 container init a8bc986f7080e19e4c4f2a05c015fdb63f204e4d1aafe91156eb8d50f14786ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mds-c, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-09-11T23:16:01.239 INFO:journalctl@ceph.mds.c.smithi136.stdout:Sep 11 23:16:00 smithi136 podman[65026]: 2024-09-11 23:16:00.959358015 +0000 UTC m=+0.345313510 container start a8bc986f7080e19e4c4f2a05c015fdb63f204e4d1aafe91156eb8d50f14786ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mds-c, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-09-11T23:16:01.239 INFO:journalctl@ceph.mds.c.smithi136.stdout:Sep 11 23:16:00 smithi136 ceph-mds[65045]: -- 172.21.15.136:0/367127784 <== mon.1 v2:172.21.15.136:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55677ffbd860 con 0x55677ff9b000 2024-09-11T23:16:01.239 INFO:journalctl@ceph.mds.c.smithi136.stdout:Sep 11 23:16:00 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mds-c[65041]: starting mds.c at 2024-09-11T23:16:01.239 INFO:journalctl@ceph.mds.c.smithi136.stdout:Sep 11 23:16:01 smithi136 bash[65026]: a8bc986f7080e19e4c4f2a05c015fdb63f204e4d1aafe91156eb8d50f14786ef 2024-09-11T23:16:01.240 INFO:journalctl@ceph.mds.c.smithi136.stdout:Sep 11 23:16:01 smithi136 systemd[1]: Started Ceph mds.c for af7e6fc8-7092-11ef-bcea-c7b262605968. 2024-09-11T23:16:01.867 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:01 smithi076 ceph-mon[35296]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:16:01.867 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:01 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:16:01.867 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:01 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:16:01.867 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:01 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:16:01.867 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:01 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T23:16:01.867 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:01 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T23:16:01.867 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:01 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:16:01.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:01 smithi136 ceph-mon[35121]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:16:01.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:01 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:16:01.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:01 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:16:01.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:01 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:16:01.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:01 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T23:16:01.990 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:01 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T23:16:01.990 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:01 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:16:02.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:01 smithi026 ceph-mon[31172]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:16:02.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:01 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:16:02.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:01 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:16:02.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:01 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:16:02.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:01 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T23:16:02.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:01 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T23:16:02.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:01 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:16:02.345 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:16:02.615 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:16:02.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:02 smithi136 ceph-mon[35121]: Deploying daemon mds.h on smithi076 2024-09-11T23:16:02.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:02 smithi136 ceph-mon[35121]: mds.? [v2:172.21.15.136:6836/1633082387,v1:172.21.15.136:6837/1633082387] up:boot 2024-09-11T23:16:02.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:02 smithi136 ceph-mon[35121]: fsmap 7 up:standby 2024-09-11T23:16:02.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:02 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-11T23:16:02.998 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:02 smithi076 ceph-mon[35296]: Deploying daemon mds.h on smithi076 2024-09-11T23:16:02.998 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:02 smithi076 ceph-mon[35296]: mds.? [v2:172.21.15.136:6836/1633082387,v1:172.21.15.136:6837/1633082387] up:boot 2024-09-11T23:16:02.998 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:02 smithi076 ceph-mon[35296]: fsmap 7 up:standby 2024-09-11T23:16:02.998 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:02 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-11T23:16:03.000 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:02 smithi026 ceph-mon[31172]: Deploying daemon mds.h on smithi076 2024-09-11T23:16:03.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:02 smithi026 ceph-mon[31172]: mds.? [v2:172.21.15.136:6836/1633082387,v1:172.21.15.136:6837/1633082387] up:boot 2024-09-11T23:16:03.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:02 smithi026 ceph-mon[31172]: fsmap 7 up:standby 2024-09-11T23:16:03.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:02 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-11T23:16:03.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:03.087+0000 7fad9ef29640 1 -- 172.21.15.26:0/700484300 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fad98102570 msgr2=0x7fad981029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:16:03.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:03.087+0000 7fad9ef29640 1 --2- 172.21.15.26:0/700484300 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fad98102570 0x7fad981029b0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fad84009900 tx=0x7fad8402a480 comp rx=0 tx=0).stop 2024-09-11T23:16:03.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:03.087+0000 7fad9ef29640 1 -- 172.21.15.26:0/700484300 shutdown_connections 2024-09-11T23:16:03.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:03.087+0000 7fad9ef29640 1 --2- 172.21.15.26:0/700484300 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fad98102ef0 0x7fad9810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:03.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:03.087+0000 7fad9ef29640 1 --2- 172.21.15.26:0/700484300 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fad98102570 0x7fad981029b0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:03.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:03.087+0000 7fad9ef29640 1 --2- 172.21.15.26:0/700484300 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fad98100e20 0x7fad98101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:03.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:03.087+0000 7fad9ef29640 1 -- 172.21.15.26:0/700484300 >> 172.21.15.26:0/700484300 conn(0x7fad980fc7d0 msgr2=0x7fad980febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:16:03.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:03.087+0000 7fad9ef29640 1 -- 172.21.15.26:0/700484300 shutdown_connections 2024-09-11T23:16:03.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:03.088+0000 7fad9ef29640 1 -- 172.21.15.26:0/700484300 wait complete. 2024-09-11T23:16:03.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:03.088+0000 7fad9ef29640 1 Processor -- start 2024-09-11T23:16:03.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:03.088+0000 7fad9ef29640 1 -- start start 2024-09-11T23:16:03.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:03.088+0000 7fad9ef29640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fad98100e20 0x7fad981993c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:16:03.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:03.088+0000 7fad9ef29640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fad98102570 0x7fad98199900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:16:03.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:03.088+0000 7fad9ef29640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fad98102ef0 0x7fad981a4440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:16:03.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:03.088+0000 7fad9ef29640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fad98110f50 con 0x7fad98100e20 2024-09-11T23:16:03.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:03.088+0000 7fad9ef29640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fad98110dd0 con 0x7fad98102570 2024-09-11T23:16:03.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:03.088+0000 7fad9ef29640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fad981110d0 con 0x7fad98102ef0 2024-09-11T23:16:03.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:03.089+0000 7fad9ccc5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fad98100e20 0x7fad981993c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:16:03.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:03.089+0000 7fad9ccc5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fad98100e20 0x7fad981993c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49454/0 (socket says 172.21.15.26:49454) 2024-09-11T23:16:03.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:03.089+0000 7fad9ccc5640 1 -- 172.21.15.26:0/1418263453 learned_addr learned my addr 172.21.15.26:0/1418263453 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:16:03.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:03.089+0000 7fad97fff640 1 --2- 172.21.15.26:0/1418263453 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fad98102570 0x7fad98199900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:16:03.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:03.089+0000 7fad9d4c6640 1 --2- 172.21.15.26:0/1418263453 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fad98102ef0 0x7fad981a4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:16:03.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:03.089+0000 7fad9ccc5640 1 -- 172.21.15.26:0/1418263453 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fad98102ef0 msgr2=0x7fad981a4440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:16:03.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:03.089+0000 7fad9ccc5640 1 --2- 172.21.15.26:0/1418263453 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fad98102ef0 0x7fad981a4440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:03.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:03.089+0000 7fad9ccc5640 1 -- 172.21.15.26:0/1418263453 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fad98102570 msgr2=0x7fad98199900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:16:03.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:03.089+0000 7fad9ccc5640 1 --2- 172.21.15.26:0/1418263453 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fad98102570 0x7fad98199900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:03.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:03.089+0000 7fad9ccc5640 1 -- 172.21.15.26:0/1418263453 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad981a4980 con 0x7fad98100e20 2024-09-11T23:16:03.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:03.089+0000 7fad9ccc5640 1 --2- 172.21.15.26:0/1418263453 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fad98100e20 0x7fad981993c0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fad8c00e9f0 tx=0x7fad8c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:16:03.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:03.089+0000 7fad95ffb640 1 -- 172.21.15.26:0/1418263453 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad8c00cd80 con 0x7fad98100e20 2024-09-11T23:16:03.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:03.089+0000 7fad95ffb640 1 -- 172.21.15.26:0/1418263453 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad8c010910 con 0x7fad98100e20 2024-09-11T23:16:03.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:03.089+0000 7fad9ef29640 1 -- 172.21.15.26:0/1418263453 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad981a4b50 con 0x7fad98100e20 2024-09-11T23:16:03.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:03.089+0000 7fad95ffb640 1 -- 172.21.15.26:0/1418263453 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad8c018990 con 0x7fad98100e20 2024-09-11T23:16:03.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:03.090+0000 7fad9ef29640 1 -- 172.21.15.26:0/1418263453 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad981a5010 con 0x7fad98100e20 2024-09-11T23:16:03.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:03.091+0000 7fad9ef29640 1 -- 172.21.15.26:0/1418263453 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad98101200 con 0x7fad98100e20 2024-09-11T23:16:03.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:03.092+0000 7fad95ffb640 1 -- 172.21.15.26:0/1418263453 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fad8c010ac0 con 0x7fad98100e20 2024-09-11T23:16:03.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:03.092+0000 7fad95ffb640 1 --2- 172.21.15.26:0/1418263453 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fad880a2b80 0x7fad880a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:16:03.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:03.093+0000 7fad97fff640 1 --2- 172.21.15.26:0/1418263453 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fad880a2b80 0x7fad880a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:16:03.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:03.093+0000 7fad95ffb640 1 -- 172.21.15.26:0/1418263453 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7127+0+0 (secure 0 0 0) 0x7fad8c0cee90 con 0x7fad98100e20 2024-09-11T23:16:03.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:03.093+0000 7fad97fff640 1 --2- 172.21.15.26:0/1418263453 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fad880a2b80 0x7fad880a5040 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fad84009230 tx=0x7fad840058d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:16:03.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:03.097+0000 7fad95ffb640 1 -- 172.21.15.26:0/1418263453 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fad8c09bc40 con 0x7fad98100e20 2024-09-11T23:16:03.304 INFO:journalctl@ceph.mds.h.smithi076.stdout:Sep 11 23:16:03 smithi076 systemd[1]: Starting Ceph mds.h for af7e6fc8-7092-11ef-bcea-c7b262605968... 2024-09-11T23:16:03.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:03.311+0000 7fad9ef29640 1 -- 172.21.15.26:0/1418263453 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fad9806a560 con 0x7fad98100e20 2024-09-11T23:16:03.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:03.312+0000 7fad95ffb640 1 -- 172.21.15.26:0/1418263453 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6388 (secure 0 0 0) 0x7fad8c09ff40 con 0x7fad98100e20 2024-09-11T23:16:03.313 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-11T23:16:03.313 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":8,"btime":"2024-09-11T23:16:01:642599+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14499,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/367659515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":367659515},{"type":"v1","addr":"172.21.15.26:6835","nonce":367659515}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14511,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/2041065299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":2041065299},{"type":"v1","addr":"172.21.15.26:6837","nonce":2041065299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6833/2019841455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":2019841455},{"type":"v1","addr":"172.21.15.136:6833","nonce":2019841455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6833/3438147050","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3438147050},{"type":"v1","addr":"172.21.15.76:6833","nonce":3438147050}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24439,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/3617665975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3617665975},{"type":"v1","addr":"172.21.15.136:6835","nonce":3617665975}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24443,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/393308499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":393308499},{"type":"v1","addr":"172.21.15.76:6835","nonce":393308499}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24448,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/1633082387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":1633082387},{"type":"v1","addr":"172.21.15.136:6837","nonce":1633082387}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-09-11T23:16:03.313 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 8 2024-09-11T23:16:03.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:03.314+0000 7fad9ef29640 1 -- 172.21.15.26:0/1418263453 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fad880a2b80 msgr2=0x7fad880a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:16:03.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:03.314+0000 7fad9ef29640 1 --2- 172.21.15.26:0/1418263453 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fad880a2b80 0x7fad880a5040 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fad84009230 tx=0x7fad840058d0 comp rx=0 tx=0).stop 2024-09-11T23:16:03.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:03.314+0000 7fad9ef29640 1 -- 172.21.15.26:0/1418263453 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fad98100e20 msgr2=0x7fad981993c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:16:03.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:03.314+0000 7fad9ef29640 1 --2- 172.21.15.26:0/1418263453 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fad98100e20 0x7fad981993c0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fad8c00e9f0 tx=0x7fad8c00eeb0 comp rx=0 tx=0).stop 2024-09-11T23:16:03.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:03.314+0000 7fad9ef29640 1 -- 172.21.15.26:0/1418263453 shutdown_connections 2024-09-11T23:16:03.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:03.314+0000 7fad9ef29640 1 --2- 172.21.15.26:0/1418263453 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fad880a2b80 0x7fad880a5040 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:03.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:03.314+0000 7fad9ef29640 1 --2- 172.21.15.26:0/1418263453 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fad98102570 0x7fad98199900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:03.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:03.314+0000 7fad9ef29640 1 --2- 172.21.15.26:0/1418263453 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fad98102ef0 0x7fad981a4440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:03.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:03.314+0000 7fad9ef29640 1 --2- 172.21.15.26:0/1418263453 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fad98100e20 0x7fad981993c0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:03.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:03.314+0000 7fad9ef29640 1 -- 172.21.15.26:0/1418263453 >> 172.21.15.26:0/1418263453 conn(0x7fad980fc7d0 msgr2=0x7fad980febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:16:03.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:03.314+0000 7fad9ef29640 1 -- 172.21.15.26:0/1418263453 shutdown_connections 2024-09-11T23:16:03.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:03.314+0000 7fad9ef29640 1 -- 172.21.15.26:0/1418263453 wait complete. 2024-09-11T23:16:03.656 INFO:journalctl@ceph.mds.h.smithi076.stdout:Sep 11 23:16:03 smithi076 podman[64999]: 2024-09-11 23:16:03.302168817 +0000 UTC m=+0.015926504 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T23:16:03.656 INFO:journalctl@ceph.mds.h.smithi076.stdout:Sep 11 23:16:03 smithi076 podman[64999]: 2024-09-11 23:16:03.433733216 +0000 UTC m=+0.147490893 container create 359eac0589c22758f5dfccb41c2351b3766f952837b5989f09def23b13a7dff8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mds-h, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-09-11T23:16:03.917 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:03 smithi026 ceph-mon[31172]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:16:03.917 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:03 smithi026 ceph-mon[31172]: from='client.? 172.21.15.26:0/1418263453' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:16:03.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:03 smithi136 ceph-mon[35121]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:16:03.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:03 smithi136 ceph-mon[35121]: from='client.? 172.21.15.26:0/1418263453' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:16:04.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:03 smithi076 ceph-mon[35296]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:16:04.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:03 smithi076 ceph-mon[35296]: from='client.? 172.21.15.26:0/1418263453' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:16:04.053 INFO:journalctl@ceph.mds.h.smithi076.stdout:Sep 11 23:16:03 smithi076 podman[64999]: 2024-09-11 23:16:03.661486888 +0000 UTC m=+0.375244563 container init 359eac0589c22758f5dfccb41c2351b3766f952837b5989f09def23b13a7dff8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mds-h, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True) 2024-09-11T23:16:04.053 INFO:journalctl@ceph.mds.h.smithi076.stdout:Sep 11 23:16:03 smithi076 podman[64999]: 2024-09-11 23:16:03.664390702 +0000 UTC m=+0.378148377 container start 359eac0589c22758f5dfccb41c2351b3766f952837b5989f09def23b13a7dff8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mds-h, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True) 2024-09-11T23:16:04.053 INFO:journalctl@ceph.mds.h.smithi076.stdout:Sep 11 23:16:03 smithi076 ceph-mds[65019]: -- 172.21.15.76:0/3353681487 <== mon.2 v2:172.21.15.76:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559a2d7a3860 con 0x559a2d781400 2024-09-11T23:16:04.053 INFO:journalctl@ceph.mds.h.smithi076.stdout:Sep 11 23:16:03 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mds-h[65015]: starting mds.h at 2024-09-11T23:16:04.054 INFO:journalctl@ceph.mds.h.smithi076.stdout:Sep 11 23:16:03 smithi076 bash[64999]: 359eac0589c22758f5dfccb41c2351b3766f952837b5989f09def23b13a7dff8 2024-09-11T23:16:04.054 INFO:journalctl@ceph.mds.h.smithi076.stdout:Sep 11 23:16:03 smithi076 systemd[1]: Started Ceph mds.h for af7e6fc8-7092-11ef-bcea-c7b262605968. 2024-09-11T23:16:05.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:04 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:16:05.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:04 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:16:05.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:04 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:16:05.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:04 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T23:16:05.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:04 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T23:16:05.240 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:04 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:16:05.240 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:04 smithi136 ceph-mon[35121]: Deploying daemon mds.i on smithi136 2024-09-11T23:16:05.240 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:04 smithi136 ceph-mon[35121]: mds.? [v2:172.21.15.76:6836/2078978748,v1:172.21.15.76:6837/2078978748] up:boot 2024-09-11T23:16:05.240 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:04 smithi136 ceph-mon[35121]: fsmap 8 up:standby 2024-09-11T23:16:05.240 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:04 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-11T23:16:05.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:04 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:16:05.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:04 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:16:05.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:04 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:16:05.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:04 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T23:16:05.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:04 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T23:16:05.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:04 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:16:05.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:04 smithi026 ceph-mon[31172]: Deploying daemon mds.i on smithi136 2024-09-11T23:16:05.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:04 smithi026 ceph-mon[31172]: mds.? [v2:172.21.15.76:6836/2078978748,v1:172.21.15.76:6837/2078978748] up:boot 2024-09-11T23:16:05.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:04 smithi026 ceph-mon[31172]: fsmap 8 up:standby 2024-09-11T23:16:05.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:04 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-11T23:16:05.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:04 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:16:05.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:04 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:16:05.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:04 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:16:05.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:04 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T23:16:05.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:04 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T23:16:05.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:04 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:16:05.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:04 smithi076 ceph-mon[35296]: Deploying daemon mds.i on smithi136 2024-09-11T23:16:05.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:04 smithi076 ceph-mon[35296]: mds.? [v2:172.21.15.76:6836/2078978748,v1:172.21.15.76:6837/2078978748] up:boot 2024-09-11T23:16:05.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:04 smithi076 ceph-mon[35296]: fsmap 8 up:standby 2024-09-11T23:16:05.304 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:04 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-11T23:16:05.825 INFO:journalctl@ceph.mds.i.smithi136.stdout:Sep 11 23:16:05 smithi136 systemd[1]: Starting Ceph mds.i for af7e6fc8-7092-11ef-bcea-c7b262605968... 2024-09-11T23:16:05.955 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:16:06.084 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:05 smithi136 ceph-mon[35121]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:16:06.084 INFO:journalctl@ceph.mds.i.smithi136.stdout:Sep 11 23:16:05 smithi136 podman[65327]: 2024-09-11 23:16:05.832486575 +0000 UTC m=+0.017312888 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T23:16:06.084 INFO:journalctl@ceph.mds.i.smithi136.stdout:Sep 11 23:16:05 smithi136 podman[65327]: 2024-09-11 23:16:05.974862568 +0000 UTC m=+0.159688913 container create 796ec4a7873a26da8a0bf790952430c64f6e5978969c10d8c0c7fb07e0972029 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mds-i, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T23:16:06.228 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:16:06.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:05 smithi026 ceph-mon[31172]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:16:06.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:05 smithi076 ceph-mon[35296]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:16:06.489 INFO:journalctl@ceph.mds.i.smithi136.stdout:Sep 11 23:16:06 smithi136 podman[65327]: 2024-09-11 23:16:06.194074938 +0000 UTC m=+0.378901238 container init 796ec4a7873a26da8a0bf790952430c64f6e5978969c10d8c0c7fb07e0972029 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mds-i, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T23:16:06.489 INFO:journalctl@ceph.mds.i.smithi136.stdout:Sep 11 23:16:06 smithi136 podman[65327]: 2024-09-11 23:16:06.196836793 +0000 UTC m=+0.381663090 container start 796ec4a7873a26da8a0bf790952430c64f6e5978969c10d8c0c7fb07e0972029 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mds-i, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-09-11T23:16:06.489 INFO:journalctl@ceph.mds.i.smithi136.stdout:Sep 11 23:16:06 smithi136 ceph-mds[65345]: -- 172.21.15.136:0/4253259444 <== mon.1 v2:172.21.15.136:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555c8f181860 con 0x555c8f15e800 2024-09-11T23:16:06.489 INFO:journalctl@ceph.mds.i.smithi136.stdout:Sep 11 23:16:06 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mds-i[65341]: starting mds.i at 2024-09-11T23:16:06.489 INFO:journalctl@ceph.mds.i.smithi136.stdout:Sep 11 23:16:06 smithi136 bash[65327]: 796ec4a7873a26da8a0bf790952430c64f6e5978969c10d8c0c7fb07e0972029 2024-09-11T23:16:06.489 INFO:journalctl@ceph.mds.i.smithi136.stdout:Sep 11 23:16:06 smithi136 systemd[1]: Started Ceph mds.i for af7e6fc8-7092-11ef-bcea-c7b262605968. 2024-09-11T23:16:06.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:06.727+0000 7fed33120640 1 -- 172.21.15.26:0/2859323392 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed2c101420 msgr2=0x7fed2c10d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:16:06.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:06.727+0000 7fed33120640 1 --2- 172.21.15.26:0/2859323392 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed2c101420 0x7fed2c10d950 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fed24009cd0 tx=0x7fed2402a8e0 comp rx=0 tx=0).stop 2024-09-11T23:16:06.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:06.727+0000 7fed33120640 1 -- 172.21.15.26:0/2859323392 shutdown_connections 2024-09-11T23:16:06.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:06.727+0000 7fed33120640 1 --2- 172.21.15.26:0/2859323392 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed2c101420 0x7fed2c10d950 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:06.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:06.727+0000 7fed33120640 1 --2- 172.21.15.26:0/2859323392 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fed2c100a80 0x7fed2c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:06.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:06.727+0000 7fed33120640 1 --2- 172.21.15.26:0/2859323392 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fed2c106a80 0x7fed2c106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:06.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:06.727+0000 7fed33120640 1 -- 172.21.15.26:0/2859323392 >> 172.21.15.26:0/2859323392 conn(0x7fed2c0fc7a0 msgr2=0x7fed2c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:16:06.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:06.727+0000 7fed33120640 1 -- 172.21.15.26:0/2859323392 shutdown_connections 2024-09-11T23:16:06.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:06.727+0000 7fed33120640 1 -- 172.21.15.26:0/2859323392 wait complete. 2024-09-11T23:16:06.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:06.728+0000 7fed33120640 1 Processor -- start 2024-09-11T23:16:06.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:06.728+0000 7fed33120640 1 -- start start 2024-09-11T23:16:06.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:06.728+0000 7fed33120640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fed2c100a80 0x7fed2c1992b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:16:06.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:06.728+0000 7fed33120640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fed2c101420 0x7fed2c1997f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:16:06.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:06.728+0000 7fed33120640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed2c106a80 0x7fed2c1a4330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:16:06.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:06.728+0000 7fed33120640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fed2c110e80 con 0x7fed2c106a80 2024-09-11T23:16:06.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:06.728+0000 7fed33120640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fed2c110d00 con 0x7fed2c101420 2024-09-11T23:16:06.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:06.728+0000 7fed33120640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fed2c111000 con 0x7fed2c100a80 2024-09-11T23:16:06.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:06.728+0000 7fed316bd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed2c106a80 0x7fed2c1a4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:16:06.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:06.728+0000 7fed316bd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed2c106a80 0x7fed2c1a4330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49470/0 (socket says 172.21.15.26:49470) 2024-09-11T23:16:06.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:06.728+0000 7fed316bd640 1 -- 172.21.15.26:0/4220002239 learned_addr learned my addr 172.21.15.26:0/4220002239 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:16:06.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:06.728+0000 7fed2bfff640 1 --2- 172.21.15.26:0/4220002239 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fed2c101420 0x7fed2c1997f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:16:06.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:06.729+0000 7fed30ebc640 1 --2- 172.21.15.26:0/4220002239 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fed2c100a80 0x7fed2c1992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:16:06.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:06.729+0000 7fed316bd640 1 -- 172.21.15.26:0/4220002239 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fed2c100a80 msgr2=0x7fed2c1992b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:16:06.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:06.729+0000 7fed316bd640 1 --2- 172.21.15.26:0/4220002239 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fed2c100a80 0x7fed2c1992b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:06.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:06.729+0000 7fed316bd640 1 -- 172.21.15.26:0/4220002239 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fed2c101420 msgr2=0x7fed2c1997f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:16:06.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:06.729+0000 7fed316bd640 1 --2- 172.21.15.26:0/4220002239 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fed2c101420 0x7fed2c1997f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:06.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:06.729+0000 7fed316bd640 1 -- 172.21.15.26:0/4220002239 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed2c1a4870 con 0x7fed2c106a80 2024-09-11T23:16:06.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:06.729+0000 7fed316bd640 1 --2- 172.21.15.26:0/4220002239 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed2c106a80 0x7fed2c1a4330 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fed24005cc0 tx=0x7fed24003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:16:06.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:06.729+0000 7fed29ffb640 1 -- 172.21.15.26:0/4220002239 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed24004040 con 0x7fed2c106a80 2024-09-11T23:16:06.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:06.729+0000 7fed29ffb640 1 -- 172.21.15.26:0/4220002239 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed240046c0 con 0x7fed2c106a80 2024-09-11T23:16:06.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:06.729+0000 7fed33120640 1 -- 172.21.15.26:0/4220002239 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed2c1a4a40 con 0x7fed2c106a80 2024-09-11T23:16:06.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:06.729+0000 7fed29ffb640 1 -- 172.21.15.26:0/4220002239 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed24046480 con 0x7fed2c106a80 2024-09-11T23:16:06.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:06.729+0000 7fed33120640 1 -- 172.21.15.26:0/4220002239 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed2c1a4f20 con 0x7fed2c106a80 2024-09-11T23:16:06.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:06.730+0000 7fed33120640 1 -- 172.21.15.26:0/4220002239 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed2c100ee0 con 0x7fed2c106a80 2024-09-11T23:16:06.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:06.732+0000 7fed29ffb640 1 -- 172.21.15.26:0/4220002239 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fed240305a0 con 0x7fed2c106a80 2024-09-11T23:16:06.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:06.732+0000 7fed29ffb640 1 --2- 172.21.15.26:0/4220002239 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fed1c0a2c00 0x7fed1c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:16:06.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:06.732+0000 7fed29ffb640 1 -- 172.21.15.26:0/4220002239 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7127+0+0 (secure 0 0 0) 0x7fed2402e030 con 0x7fed2c106a80 2024-09-11T23:16:06.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:06.732+0000 7fed30ebc640 1 --2- 172.21.15.26:0/4220002239 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fed1c0a2c00 0x7fed1c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:16:06.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:06.733+0000 7fed30ebc640 1 --2- 172.21.15.26:0/4220002239 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fed1c0a2c00 0x7fed1c0a50c0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fed20005d90 tx=0x7fed20005d20 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:16:06.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:06.735+0000 7fed29ffb640 1 -- 172.21.15.26:0/4220002239 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fed2404b060 con 0x7fed2c106a80 2024-09-11T23:16:06.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:06.956+0000 7fed33120640 1 -- 172.21.15.26:0/4220002239 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fed2c06ad80 con 0x7fed2c106a80 2024-09-11T23:16:06.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:06.957+0000 7fed29ffb640 1 -- 172.21.15.26:0/4220002239 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8043 (secure 0 0 0) 0x7fed2c06ad80 con 0x7fed2c106a80 2024-09-11T23:16:06.958 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-11T23:16:06.958 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":10,"btime":"2024-09-11T23:16:06:824999+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14499,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/367659515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":367659515},{"type":"v1","addr":"172.21.15.26:6835","nonce":367659515}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14511,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/2041065299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":2041065299},{"type":"v1","addr":"172.21.15.26:6837","nonce":2041065299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6833/2019841455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":2019841455},{"type":"v1","addr":"172.21.15.136:6833","nonce":2019841455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6833/3438147050","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3438147050},{"type":"v1","addr":"172.21.15.76:6833","nonce":3438147050}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24439,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/3617665975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3617665975},{"type":"v1","addr":"172.21.15.136:6835","nonce":3617665975}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24443,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/393308499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":393308499},{"type":"v1","addr":"172.21.15.76:6835","nonce":393308499}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24448,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/1633082387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":1633082387},{"type":"v1","addr":"172.21.15.136:6837","nonce":1633082387}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24458,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6837/2078978748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":2078978748},{"type":"v1","addr":"172.21.15.76:6837","nonce":2078978748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24463,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/3211398630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":3211398630},{"type":"v1","addr":"172.21.15.136:6839","nonce":3211398630}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-09-11T23:16:06.959 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 10 2024-09-11T23:16:06.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:06.960+0000 7fed33120640 1 -- 172.21.15.26:0/4220002239 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fed1c0a2c00 msgr2=0x7fed1c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:16:06.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:06.960+0000 7fed33120640 1 --2- 172.21.15.26:0/4220002239 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fed1c0a2c00 0x7fed1c0a50c0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fed20005d90 tx=0x7fed20005d20 comp rx=0 tx=0).stop 2024-09-11T23:16:06.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:06.960+0000 7fed33120640 1 -- 172.21.15.26:0/4220002239 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed2c106a80 msgr2=0x7fed2c1a4330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:16:06.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:06.960+0000 7fed33120640 1 --2- 172.21.15.26:0/4220002239 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed2c106a80 0x7fed2c1a4330 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fed24005cc0 tx=0x7fed24003970 comp rx=0 tx=0).stop 2024-09-11T23:16:06.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:06.960+0000 7fed33120640 1 -- 172.21.15.26:0/4220002239 shutdown_connections 2024-09-11T23:16:06.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:06.960+0000 7fed33120640 1 --2- 172.21.15.26:0/4220002239 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fed1c0a2c00 0x7fed1c0a50c0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:06.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:06.960+0000 7fed33120640 1 --2- 172.21.15.26:0/4220002239 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fed2c101420 0x7fed2c1997f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:06.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:06.960+0000 7fed33120640 1 --2- 172.21.15.26:0/4220002239 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed2c106a80 0x7fed2c1a4330 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:06.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:06.960+0000 7fed33120640 1 --2- 172.21.15.26:0/4220002239 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fed2c100a80 0x7fed2c1992b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:06.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:06.960+0000 7fed33120640 1 -- 172.21.15.26:0/4220002239 >> 172.21.15.26:0/4220002239 conn(0x7fed2c0fc7a0 msgr2=0x7fed2c0fce70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:16:06.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:06.960+0000 7fed33120640 1 -- 172.21.15.26:0/4220002239 shutdown_connections 2024-09-11T23:16:06.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:06.961+0000 7fed33120640 1 -- 172.21.15.26:0/4220002239 wait complete. 2024-09-11T23:16:07.596 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:07 smithi026 ceph-mon[31172]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:16:07.597 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:07 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:16:07.597 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:07 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:16:07.597 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:07 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:16:07.597 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:07 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T23:16:07.597 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:07 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T23:16:07.597 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:07 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:16:07.597 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:07 smithi026 ceph-mon[31172]: Deploying daemon mds.g on smithi026 2024-09-11T23:16:07.597 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:07 smithi026 ceph-mon[31172]: mds.? [v2:172.21.15.136:6838/3211398630,v1:172.21.15.136:6839/3211398630] up:boot 2024-09-11T23:16:07.597 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:07 smithi026 ceph-mon[31172]: fsmap 9 up:standby 2024-09-11T23:16:07.597 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:07 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-11T23:16:07.597 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:07 smithi026 ceph-mon[31172]: from='client.? 172.21.15.26:0/4220002239' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:16:07.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:07 smithi136 ceph-mon[35121]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:16:07.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:07 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:16:07.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:07 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:16:07.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:07 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:16:07.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:07 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T23:16:07.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:07 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T23:16:07.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:07 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:16:07.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:07 smithi136 ceph-mon[35121]: Deploying daemon mds.g on smithi026 2024-09-11T23:16:07.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:07 smithi136 ceph-mon[35121]: mds.? [v2:172.21.15.136:6838/3211398630,v1:172.21.15.136:6839/3211398630] up:boot 2024-09-11T23:16:07.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:07 smithi136 ceph-mon[35121]: fsmap 9 up:standby 2024-09-11T23:16:07.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:07 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-11T23:16:07.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:07 smithi136 ceph-mon[35121]: from='client.? 172.21.15.26:0/4220002239' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:16:07.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:07 smithi076 ceph-mon[35296]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:16:07.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:07 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:16:07.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:07 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:16:07.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:07 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:16:07.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:07 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T23:16:07.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:07 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T23:16:07.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:07 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:16:07.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:07 smithi076 ceph-mon[35296]: Deploying daemon mds.g on smithi026 2024-09-11T23:16:07.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:07 smithi076 ceph-mon[35296]: mds.? [v2:172.21.15.136:6838/3211398630,v1:172.21.15.136:6839/3211398630] up:boot 2024-09-11T23:16:07.804 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:07 smithi076 ceph-mon[35296]: fsmap 9 up:standby 2024-09-11T23:16:07.804 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:07 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-11T23:16:07.804 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:07 smithi076 ceph-mon[35296]: from='client.? 172.21.15.26:0/4220002239' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:16:09.338 INFO:journalctl@ceph.mds.g.smithi026.stdout:Sep 11 23:16:09 smithi026 systemd[1]: Starting Ceph mds.g for af7e6fc8-7092-11ef-bcea-c7b262605968... 2024-09-11T23:16:09.599 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:16:09.626 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:09 smithi026 ceph-mon[31172]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:16:09.626 INFO:journalctl@ceph.mds.g.smithi026.stdout:Sep 11 23:16:09 smithi026 podman[72082]: 2024-09-11 23:16:09.392925164 +0000 UTC m=+0.016224646 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T23:16:09.626 INFO:journalctl@ceph.mds.g.smithi026.stdout:Sep 11 23:16:09 smithi026 podman[72082]: 2024-09-11 23:16:09.508380902 +0000 UTC m=+0.131680430 container create 79a2897e16eff7cb1d8488bff9be0e3baaa01d46a3f1b87378f87e808c14c063 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mds-g, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T23:16:09.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:09 smithi136 ceph-mon[35121]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:16:09.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:09 smithi076 ceph-mon[35296]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:16:09.949 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:16:10.001 INFO:journalctl@ceph.mds.g.smithi026.stdout:Sep 11 23:16:09 smithi026 podman[72082]: 2024-09-11 23:16:09.767176564 +0000 UTC m=+0.390476056 container init 79a2897e16eff7cb1d8488bff9be0e3baaa01d46a3f1b87378f87e808c14c063 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mds-g, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T23:16:10.001 INFO:journalctl@ceph.mds.g.smithi026.stdout:Sep 11 23:16:09 smithi026 podman[72082]: 2024-09-11 23:16:09.770393466 +0000 UTC m=+0.393692948 container start 79a2897e16eff7cb1d8488bff9be0e3baaa01d46a3f1b87378f87e808c14c063 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mds-g, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_CLEAN=True, io.buildah.version=1.37.2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS) 2024-09-11T23:16:10.001 INFO:journalctl@ceph.mds.g.smithi026.stdout:Sep 11 23:16:09 smithi026 ceph-mds[72119]: -- 172.21.15.26:0/1323873831 <== mon.0 v2:172.21.15.26:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558ffa5f7860 con 0x558ffa5d4800 2024-09-11T23:16:10.001 INFO:journalctl@ceph.mds.g.smithi026.stdout:Sep 11 23:16:09 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mds-g[72114]: starting mds.g at 2024-09-11T23:16:10.001 INFO:journalctl@ceph.mds.g.smithi026.stdout:Sep 11 23:16:09 smithi026 bash[72082]: 79a2897e16eff7cb1d8488bff9be0e3baaa01d46a3f1b87378f87e808c14c063 2024-09-11T23:16:10.001 INFO:journalctl@ceph.mds.g.smithi026.stdout:Sep 11 23:16:09 smithi026 systemd[1]: Started Ceph mds.g for af7e6fc8-7092-11ef-bcea-c7b262605968. 2024-09-11T23:16:10.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:10.439+0000 7f0019e5c640 1 -- 172.21.15.26:0/1262587612 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0014106f20 msgr2=0x7f001410eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:16:10.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:10.439+0000 7f0019e5c640 1 --2- 172.21.15.26:0/1262587612 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0014106f20 0x7f001410eaa0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f000c009ca0 tx=0x7f000c02a8c0 comp rx=0 tx=0).stop 2024-09-11T23:16:10.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:10.439+0000 7f0019e5c640 1 -- 172.21.15.26:0/1262587612 shutdown_connections 2024-09-11T23:16:10.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:10.439+0000 7f0019e5c640 1 --2- 172.21.15.26:0/1262587612 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f00141027a0 0x7f00141067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:10.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:10.439+0000 7f0019e5c640 1 --2- 172.21.15.26:0/1262587612 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0014106f20 0x7f001410eaa0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:10.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:10.439+0000 7f0019e5c640 1 --2- 172.21.15.26:0/1262587612 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0014101df0 0x7f00141021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:10.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:10.439+0000 7f0019e5c640 1 -- 172.21.15.26:0/1262587612 >> 172.21.15.26:0/1262587612 conn(0x7f00140fd8b0 msgr2=0x7f00140ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:16:10.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:10.439+0000 7f0019e5c640 1 -- 172.21.15.26:0/1262587612 shutdown_connections 2024-09-11T23:16:10.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:10.439+0000 7f0019e5c640 1 -- 172.21.15.26:0/1262587612 wait complete. 2024-09-11T23:16:10.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:10.439+0000 7f0019e5c640 1 Processor -- start 2024-09-11T23:16:10.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:10.439+0000 7f0019e5c640 1 -- start start 2024-09-11T23:16:10.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:10.440+0000 7f0019e5c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0014101df0 0x7f00140744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:16:10.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:10.440+0000 7f0019e5c640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f00141027a0 0x7f0014074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:16:10.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:10.440+0000 7f0019e5c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0014106f20 0x7f0014074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:16:10.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:10.440+0000 7f0019e5c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f0014112070 con 0x7f0014106f20 2024-09-11T23:16:10.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:10.440+0000 7f0019e5c640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f0014111ef0 con 0x7f0014101df0 2024-09-11T23:16:10.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:10.440+0000 7f0019e5c640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f00141121f0 con 0x7f00141027a0 2024-09-11T23:16:10.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:10.440+0000 7f0013fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0014106f20 0x7f0014074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:16:10.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:10.440+0000 7f0013fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0014106f20 0x7f0014074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49510/0 (socket says 172.21.15.26:49510) 2024-09-11T23:16:10.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:10.440+0000 7f0013fff640 1 -- 172.21.15.26:0/162092820 learned_addr learned my addr 172.21.15.26:0/162092820 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:16:10.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:10.440+0000 7f0012ffd640 1 --2- 172.21.15.26:0/162092820 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f00141027a0 0x7f0014074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:16:10.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:10.440+0000 7f00137fe640 1 --2- 172.21.15.26:0/162092820 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0014101df0 0x7f00140744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:16:10.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:10.440+0000 7f0013fff640 1 -- 172.21.15.26:0/162092820 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f00141027a0 msgr2=0x7f0014074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:16:10.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:10.440+0000 7f0013fff640 1 --2- 172.21.15.26:0/162092820 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f00141027a0 0x7f0014074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:10.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:10.440+0000 7f0013fff640 1 -- 172.21.15.26:0/162092820 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0014101df0 msgr2=0x7f00140744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:16:10.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:10.440+0000 7f0013fff640 1 --2- 172.21.15.26:0/162092820 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0014101df0 0x7f00140744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:10.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:10.440+0000 7f0013fff640 1 -- 172.21.15.26:0/162092820 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00141a6a50 con 0x7f0014106f20 2024-09-11T23:16:10.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:10.440+0000 7f00137fe640 1 --2- 172.21.15.26:0/162092820 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0014101df0 0x7f00140744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:16:10.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:10.440+0000 7f0012ffd640 1 --2- 172.21.15.26:0/162092820 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f00141027a0 0x7f0014074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:16:10.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:10.440+0000 7f0013fff640 1 --2- 172.21.15.26:0/162092820 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0014106f20 0x7f0014074f70 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f000c003f40 tx=0x7f000c003f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:16:10.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:10.441+0000 7f0010ff9640 1 -- 172.21.15.26:0/162092820 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f000c0318c0 con 0x7f0014106f20 2024-09-11T23:16:10.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:10.441+0000 7f0010ff9640 1 -- 172.21.15.26:0/162092820 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f000c004520 con 0x7f0014106f20 2024-09-11T23:16:10.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:10.441+0000 7f0019e5c640 1 -- 172.21.15.26:0/162092820 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00141a6ce0 con 0x7f0014106f20 2024-09-11T23:16:10.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:10.441+0000 7f0010ff9640 1 -- 172.21.15.26:0/162092820 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f000c02fe10 con 0x7f0014106f20 2024-09-11T23:16:10.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:10.441+0000 7f0019e5c640 1 -- 172.21.15.26:0/162092820 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00141a7220 con 0x7f0014106f20 2024-09-11T23:16:10.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:10.442+0000 7f0019e5c640 1 -- 172.21.15.26:0/162092820 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0014102250 con 0x7f0014106f20 2024-09-11T23:16:10.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:10.443+0000 7f0010ff9640 1 -- 172.21.15.26:0/162092820 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f000c0046d0 con 0x7f0014106f20 2024-09-11T23:16:10.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:10.444+0000 7f0010ff9640 1 --2- 172.21.15.26:0/162092820 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7efffc0a2b80 0x7efffc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:16:10.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:10.444+0000 7f00137fe640 1 --2- 172.21.15.26:0/162092820 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7efffc0a2b80 0x7efffc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:16:10.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:10.444+0000 7f0010ff9640 1 -- 172.21.15.26:0/162092820 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7127+0+0 (secure 0 0 0) 0x7f000c0f0470 con 0x7f0014106f20 2024-09-11T23:16:10.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:10.444+0000 7f00137fe640 1 --2- 172.21.15.26:0/162092820 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7efffc0a2b80 0x7efffc0a5040 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f0004006fd0 tx=0x7f0004008040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:16:10.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:10.447+0000 7f0010ff9640 1 -- 172.21.15.26:0/162092820 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f000c0bd870 con 0x7f0014106f20 2024-09-11T23:16:10.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:10.662+0000 7f0019e5c640 1 -- 172.21.15.26:0/162092820 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f001410a100 con 0x7f0014106f20 2024-09-11T23:16:10.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:10.663+0000 7f0010ff9640 1 -- 172.21.15.26:0/162092820 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8869 (secure 0 0 0) 0x7f000c0bd170 con 0x7f0014106f20 2024-09-11T23:16:10.664 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-11T23:16:10.665 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":11,"btime":"2024-09-11T23:16:10:338063+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14499,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/367659515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":367659515},{"type":"v1","addr":"172.21.15.26:6835","nonce":367659515}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14511,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/2041065299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":2041065299},{"type":"v1","addr":"172.21.15.26:6837","nonce":2041065299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14550,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6839/2233068305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":2233068305},{"type":"v1","addr":"172.21.15.26:6839","nonce":2233068305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6833/2019841455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":2019841455},{"type":"v1","addr":"172.21.15.136:6833","nonce":2019841455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6833/3438147050","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3438147050},{"type":"v1","addr":"172.21.15.76:6833","nonce":3438147050}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24439,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/3617665975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3617665975},{"type":"v1","addr":"172.21.15.136:6835","nonce":3617665975}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24443,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/393308499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":393308499},{"type":"v1","addr":"172.21.15.76:6835","nonce":393308499}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24448,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/1633082387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":1633082387},{"type":"v1","addr":"172.21.15.136:6837","nonce":1633082387}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24458,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6837/2078978748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":2078978748},{"type":"v1","addr":"172.21.15.76:6837","nonce":2078978748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24463,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/3211398630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":3211398630},{"type":"v1","addr":"172.21.15.136:6839","nonce":3211398630}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-09-11T23:16:10.665 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 11 2024-09-11T23:16:10.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:10.665+0000 7f0019e5c640 1 -- 172.21.15.26:0/162092820 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7efffc0a2b80 msgr2=0x7efffc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:16:10.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:10.665+0000 7f0019e5c640 1 --2- 172.21.15.26:0/162092820 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7efffc0a2b80 0x7efffc0a5040 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f0004006fd0 tx=0x7f0004008040 comp rx=0 tx=0).stop 2024-09-11T23:16:10.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:10.665+0000 7f0019e5c640 1 -- 172.21.15.26:0/162092820 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0014106f20 msgr2=0x7f0014074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:16:10.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:10.665+0000 7f0019e5c640 1 --2- 172.21.15.26:0/162092820 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0014106f20 0x7f0014074f70 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f000c003f40 tx=0x7f000c003f70 comp rx=0 tx=0).stop 2024-09-11T23:16:10.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:10.665+0000 7f0019e5c640 1 -- 172.21.15.26:0/162092820 shutdown_connections 2024-09-11T23:16:10.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:10.666+0000 7f0019e5c640 1 --2- 172.21.15.26:0/162092820 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7efffc0a2b80 0x7efffc0a5040 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:10.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:10.666+0000 7f0019e5c640 1 --2- 172.21.15.26:0/162092820 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0014106f20 0x7f0014074f70 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:10.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:10.666+0000 7f0019e5c640 1 --2- 172.21.15.26:0/162092820 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f00141027a0 0x7f0014074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:10.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:10.666+0000 7f0019e5c640 1 --2- 172.21.15.26:0/162092820 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0014101df0 0x7f00140744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:10.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:10.666+0000 7f0019e5c640 1 -- 172.21.15.26:0/162092820 >> 172.21.15.26:0/162092820 conn(0x7f00140fd8b0 msgr2=0x7f00140fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:16:10.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:10.666+0000 7f0019e5c640 1 -- 172.21.15.26:0/162092820 shutdown_connections 2024-09-11T23:16:10.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:10.666+0000 7f0019e5c640 1 -- 172.21.15.26:0/162092820 wait complete. 2024-09-11T23:16:11.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:10 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:16:11.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:10 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:16:11.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:10 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:16:11.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:10 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T23:16:11.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:10 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T23:16:11.240 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:10 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:16:11.240 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:10 smithi136 ceph-mon[35121]: Deploying daemon mds.a on smithi026 2024-09-11T23:16:11.240 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:10 smithi136 ceph-mon[35121]: mds.? [v2:172.21.15.26:6838/2233068305,v1:172.21.15.26:6839/2233068305] up:boot 2024-09-11T23:16:11.240 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:10 smithi136 ceph-mon[35121]: fsmap 10 up:standby 2024-09-11T23:16:11.240 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:10 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-11T23:16:11.240 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:10 smithi136 ceph-mon[35121]: from='client.? 172.21.15.26:0/162092820' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:16:11.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:10 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:16:11.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:10 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:16:11.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:10 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:16:11.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:10 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T23:16:11.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:10 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T23:16:11.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:10 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:16:11.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:10 smithi026 ceph-mon[31172]: Deploying daemon mds.a on smithi026 2024-09-11T23:16:11.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:10 smithi026 ceph-mon[31172]: mds.? [v2:172.21.15.26:6838/2233068305,v1:172.21.15.26:6839/2233068305] up:boot 2024-09-11T23:16:11.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:10 smithi026 ceph-mon[31172]: fsmap 10 up:standby 2024-09-11T23:16:11.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:10 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-11T23:16:11.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:10 smithi026 ceph-mon[31172]: from='client.? 172.21.15.26:0/162092820' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:16:11.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:10 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:16:11.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:10 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:16:11.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:10 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:16:11.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:10 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T23:16:11.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:10 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T23:16:11.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:10 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:16:11.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:10 smithi076 ceph-mon[35296]: Deploying daemon mds.a on smithi026 2024-09-11T23:16:11.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:10 smithi076 ceph-mon[35296]: mds.? [v2:172.21.15.26:6838/2233068305,v1:172.21.15.26:6839/2233068305] up:boot 2024-09-11T23:16:11.304 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:10 smithi076 ceph-mon[35296]: fsmap 10 up:standby 2024-09-11T23:16:11.304 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:10 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-11T23:16:11.304 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:10 smithi076 ceph-mon[35296]: from='client.? 172.21.15.26:0/162092820' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:16:12.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:11 smithi136 ceph-mon[35121]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:16:12.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:11 smithi026 ceph-mon[31172]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:16:12.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:11 smithi076 ceph-mon[35296]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:16:13.001 INFO:journalctl@ceph.mds.a.smithi026.stdout:Sep 11 23:16:12 smithi026 systemd[1]: Starting Ceph mds.a for af7e6fc8-7092-11ef-bcea-c7b262605968... 2024-09-11T23:16:13.382 INFO:journalctl@ceph.mds.a.smithi026.stdout:Sep 11 23:16:13 smithi026 podman[72501]: 2024-09-11 23:16:13.033844328 +0000 UTC m=+0.016888984 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T23:16:13.382 INFO:journalctl@ceph.mds.a.smithi026.stdout:Sep 11 23:16:13 smithi026 podman[72501]: 2024-09-11 23:16:13.14529763 +0000 UTC m=+0.128342275 container create d6d92c598f2937c16f121752b993d7e9ce493b868f6426b714bfd8a4d3dc0750 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mds-a, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2) 2024-09-11T23:16:13.382 INFO:journalctl@ceph.mds.a.smithi026.stdout:Sep 11 23:16:13 smithi026 podman[72501]: 2024-09-11 23:16:13.355882662 +0000 UTC m=+0.338927317 container init d6d92c598f2937c16f121752b993d7e9ce493b868f6426b714bfd8a4d3dc0750 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mds-a, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=) 2024-09-11T23:16:13.383 INFO:journalctl@ceph.mds.a.smithi026.stdout:Sep 11 23:16:13 smithi026 podman[72501]: 2024-09-11 23:16:13.358780339 +0000 UTC m=+0.341824993 container start d6d92c598f2937c16f121752b993d7e9ce493b868f6426b714bfd8a4d3dc0750 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mds-a, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2) 2024-09-11T23:16:13.453 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:16:13.720 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:16:13.751 INFO:journalctl@ceph.mds.a.smithi026.stdout:Sep 11 23:16:13 smithi026 ceph-mds[72520]: -- 172.21.15.26:0/2328467695 <== mon.0 v2:172.21.15.26:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5605c8fcd860 con 0x5605c8fab400 2024-09-11T23:16:13.751 INFO:journalctl@ceph.mds.a.smithi026.stdout:Sep 11 23:16:13 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mds-a[72516]: starting mds.a at 2024-09-11T23:16:13.751 INFO:journalctl@ceph.mds.a.smithi026.stdout:Sep 11 23:16:13 smithi026 bash[72501]: d6d92c598f2937c16f121752b993d7e9ce493b868f6426b714bfd8a4d3dc0750 2024-09-11T23:16:13.751 INFO:journalctl@ceph.mds.a.smithi026.stdout:Sep 11 23:16:13 smithi026 systemd[1]: Started Ceph mds.a for af7e6fc8-7092-11ef-bcea-c7b262605968. 2024-09-11T23:16:14.101 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:13 smithi026 ceph-mon[31172]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:16:14.101 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:13 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:16:14.101 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:13 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:16:14.101 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:13 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:16:14.102 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:13 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T23:16:14.102 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:13 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T23:16:14.102 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:13 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:16:14.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:14.184+0000 7ff008acc640 1 -- 172.21.15.26:0/1171359982 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff004102ef0 msgr2=0x7ff00410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:16:14.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:14.184+0000 7ff008acc640 1 --2- 172.21.15.26:0/1171359982 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff004102ef0 0x7ff00410da40 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7feffc009ca0 tx=0x7feffc02a950 comp rx=0 tx=0).stop 2024-09-11T23:16:14.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:14.184+0000 7ff008acc640 1 -- 172.21.15.26:0/1171359982 shutdown_connections 2024-09-11T23:16:14.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:14.184+0000 7ff008acc640 1 --2- 172.21.15.26:0/1171359982 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff004102570 0x7ff0041029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:14.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:14.184+0000 7ff008acc640 1 --2- 172.21.15.26:0/1171359982 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff004102ef0 0x7ff00410da40 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:14.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:14.184+0000 7ff008acc640 1 --2- 172.21.15.26:0/1171359982 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff004100e20 0x7ff004101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:14.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:14.184+0000 7ff008acc640 1 -- 172.21.15.26:0/1171359982 >> 172.21.15.26:0/1171359982 conn(0x7ff0040fc7d0 msgr2=0x7ff0040febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:16:14.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:14.184+0000 7ff008acc640 1 -- 172.21.15.26:0/1171359982 shutdown_connections 2024-09-11T23:16:14.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:14.184+0000 7ff008acc640 1 -- 172.21.15.26:0/1171359982 wait complete. 2024-09-11T23:16:14.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:14.185+0000 7ff008acc640 1 Processor -- start 2024-09-11T23:16:14.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:14.185+0000 7ff008acc640 1 -- start start 2024-09-11T23:16:14.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:14.185+0000 7ff008acc640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff004100e20 0x7ff00419a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:16:14.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:14.185+0000 7ff008acc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff004102570 0x7ff00419ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:16:14.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:14.185+0000 7ff008acc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff004102ef0 0x7ff00419efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:16:14.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:14.185+0000 7ff008acc640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff004110fa0 con 0x7ff004102570 2024-09-11T23:16:14.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:14.185+0000 7ff008acc640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7ff004110e20 con 0x7ff004102ef0 2024-09-11T23:16:14.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:14.185+0000 7ff008acc640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ff004111120 con 0x7ff004100e20 2024-09-11T23:16:14.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:14.185+0000 7ff001d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff004102570 0x7ff00419ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:16:14.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:14.185+0000 7ff001d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff004102570 0x7ff00419ac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37862/0 (socket says 172.21.15.26:37862) 2024-09-11T23:16:14.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:14.185+0000 7ff001d9b640 1 -- 172.21.15.26:0/727271872 learned_addr learned my addr 172.21.15.26:0/727271872 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:16:14.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:14.185+0000 7ff002d9d640 1 --2- 172.21.15.26:0/727271872 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff004102ef0 0x7ff00419efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:16:14.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:14.185+0000 7ff00259c640 1 --2- 172.21.15.26:0/727271872 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff004100e20 0x7ff00419a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:16:14.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:14.185+0000 7ff001d9b640 1 -- 172.21.15.26:0/727271872 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff004100e20 msgr2=0x7ff00419a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:16:14.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:14.185+0000 7ff001d9b640 1 --2- 172.21.15.26:0/727271872 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff004100e20 0x7ff00419a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:14.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:14.185+0000 7ff001d9b640 1 -- 172.21.15.26:0/727271872 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff004102ef0 msgr2=0x7ff00419efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:16:14.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:14.185+0000 7ff001d9b640 1 --2- 172.21.15.26:0/727271872 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff004102ef0 0x7ff00419efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:14.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:14.185+0000 7ff001d9b640 1 -- 172.21.15.26:0/727271872 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff00419f6b0 con 0x7ff004102570 2024-09-11T23:16:14.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:14.185+0000 7ff00259c640 1 --2- 172.21.15.26:0/727271872 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff004100e20 0x7ff00419a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:16:14.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:14.185+0000 7ff001d9b640 1 --2- 172.21.15.26:0/727271872 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff004102570 0x7ff00419ac40 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fefec00d950 tx=0x7fefec00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:16:14.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:14.186+0000 7feffb7fe640 1 -- 172.21.15.26:0/727271872 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefec017070 con 0x7ff004102570 2024-09-11T23:16:14.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:14.186+0000 7ff008acc640 1 -- 172.21.15.26:0/727271872 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff00419f9a0 con 0x7ff004102570 2024-09-11T23:16:14.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:14.186+0000 7feffb7fe640 1 -- 172.21.15.26:0/727271872 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fefec009c30 con 0x7ff004102570 2024-09-11T23:16:14.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:14.186+0000 7feffb7fe640 1 -- 172.21.15.26:0/727271872 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefec00f6b0 con 0x7ff004102570 2024-09-11T23:16:14.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:14.186+0000 7ff008acc640 1 -- 172.21.15.26:0/727271872 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0041a7280 con 0x7ff004102570 2024-09-11T23:16:14.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:14.187+0000 7ff008acc640 1 -- 172.21.15.26:0/727271872 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff004101200 con 0x7ff004102570 2024-09-11T23:16:14.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:14.188+0000 7feffb7fe640 1 -- 172.21.15.26:0/727271872 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fefec0109d0 con 0x7ff004102570 2024-09-11T23:16:14.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:14.189+0000 7feffb7fe640 1 --2- 172.21.15.26:0/727271872 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7feff00a2b80 0x7feff00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:16:14.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:14.189+0000 7ff00259c640 1 --2- 172.21.15.26:0/727271872 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7feff00a2b80 0x7feff00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:16:14.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:14.189+0000 7feffb7fe640 1 -- 172.21.15.26:0/727271872 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7127+0+0 (secure 0 0 0) 0x7fefec0ce7d0 con 0x7ff004102570 2024-09-11T23:16:14.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:14.190+0000 7ff00259c640 1 --2- 172.21.15.26:0/727271872 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7feff00a2b80 0x7feff00a5040 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7feff4006fd0 tx=0x7feff4008040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:16:14.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:14.192+0000 7feffb7fe640 1 -- 172.21.15.26:0/727271872 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fefec009de0 con 0x7ff004102570 2024-09-11T23:16:14.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:13 smithi136 ceph-mon[35121]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:16:14.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:13 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:16:14.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:13 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:16:14.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:13 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:16:14.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:13 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T23:16:14.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:13 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T23:16:14.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:13 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:16:14.282 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:13 smithi076 ceph-mon[35296]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:16:14.283 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:13 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:16:14.283 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:13 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:16:14.283 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:13 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:16:14.283 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:13 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T23:16:14.283 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:13 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T23:16:14.283 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:13 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:16:14.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:14.405+0000 7ff008acc640 1 -- 172.21.15.26:0/727271872 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff00419ba60 con 0x7ff004102570 2024-09-11T23:16:14.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:14.405+0000 7feffb7fe640 1 -- 172.21.15.26:0/727271872 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9695 (secure 0 0 0) 0x7ff00419ba60 con 0x7ff004102570 2024-09-11T23:16:14.407 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-11T23:16:14.407 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":12,"btime":"2024-09-11T23:16:13:949023+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14499,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/367659515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":367659515},{"type":"v1","addr":"172.21.15.26:6835","nonce":367659515}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14511,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/2041065299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":2041065299},{"type":"v1","addr":"172.21.15.26:6837","nonce":2041065299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14550,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6839/2233068305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":2233068305},{"type":"v1","addr":"172.21.15.26:6839","nonce":2233068305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14562,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6841/2984982336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":2984982336},{"type":"v1","addr":"172.21.15.26:6841","nonce":2984982336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6833/2019841455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":2019841455},{"type":"v1","addr":"172.21.15.136:6833","nonce":2019841455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6833/3438147050","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3438147050},{"type":"v1","addr":"172.21.15.76:6833","nonce":3438147050}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24439,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/3617665975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3617665975},{"type":"v1","addr":"172.21.15.136:6835","nonce":3617665975}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24443,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/393308499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":393308499},{"type":"v1","addr":"172.21.15.76:6835","nonce":393308499}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24448,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/1633082387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":1633082387},{"type":"v1","addr":"172.21.15.136:6837","nonce":1633082387}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24458,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6837/2078978748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":2078978748},{"type":"v1","addr":"172.21.15.76:6837","nonce":2078978748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24463,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/3211398630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":3211398630},{"type":"v1","addr":"172.21.15.136:6839","nonce":3211398630}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-09-11T23:16:14.407 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 12 2024-09-11T23:16:14.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:14.407+0000 7ff008acc640 1 -- 172.21.15.26:0/727271872 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7feff00a2b80 msgr2=0x7feff00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:16:14.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:14.407+0000 7ff008acc640 1 --2- 172.21.15.26:0/727271872 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7feff00a2b80 0x7feff00a5040 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7feff4006fd0 tx=0x7feff4008040 comp rx=0 tx=0).stop 2024-09-11T23:16:14.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:14.407+0000 7ff008acc640 1 -- 172.21.15.26:0/727271872 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff004102570 msgr2=0x7ff00419ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:16:14.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:14.407+0000 7ff008acc640 1 --2- 172.21.15.26:0/727271872 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff004102570 0x7ff00419ac40 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fefec00d950 tx=0x7fefec00de10 comp rx=0 tx=0).stop 2024-09-11T23:16:14.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:14.407+0000 7ff008acc640 1 -- 172.21.15.26:0/727271872 shutdown_connections 2024-09-11T23:16:14.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:14.407+0000 7ff008acc640 1 --2- 172.21.15.26:0/727271872 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7feff00a2b80 0x7feff00a5040 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:14.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:14.407+0000 7ff008acc640 1 --2- 172.21.15.26:0/727271872 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff004102ef0 0x7ff00419efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:14.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:14.407+0000 7ff008acc640 1 --2- 172.21.15.26:0/727271872 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff004102570 0x7ff00419ac40 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:14.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:14.407+0000 7ff008acc640 1 --2- 172.21.15.26:0/727271872 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff004100e20 0x7ff00419a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:14.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:14.407+0000 7ff008acc640 1 -- 172.21.15.26:0/727271872 >> 172.21.15.26:0/727271872 conn(0x7ff0040fc7d0 msgr2=0x7ff00406a4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:16:14.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:14.408+0000 7ff008acc640 1 -- 172.21.15.26:0/727271872 shutdown_connections 2024-09-11T23:16:14.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:14.408+0000 7ff008acc640 1 -- 172.21.15.26:0/727271872 wait complete. 2024-09-11T23:16:15.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:14 smithi026 ceph-mon[31172]: Deploying daemon mds.b on smithi076 2024-09-11T23:16:15.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:14 smithi026 ceph-mon[31172]: mds.? [v2:172.21.15.26:6840/2984982336,v1:172.21.15.26:6841/2984982336] up:boot 2024-09-11T23:16:15.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:14 smithi026 ceph-mon[31172]: fsmap 11 up:standby 2024-09-11T23:16:15.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:14 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-11T23:16:15.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:14 smithi026 ceph-mon[31172]: from='client.? 172.21.15.26:0/727271872' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:16:15.212 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:14 smithi076 ceph-mon[35296]: Deploying daemon mds.b on smithi076 2024-09-11T23:16:15.212 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:14 smithi076 ceph-mon[35296]: mds.? [v2:172.21.15.26:6840/2984982336,v1:172.21.15.26:6841/2984982336] up:boot 2024-09-11T23:16:15.212 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:14 smithi076 ceph-mon[35296]: fsmap 11 up:standby 2024-09-11T23:16:15.213 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:14 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-11T23:16:15.213 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:14 smithi076 ceph-mon[35296]: from='client.? 172.21.15.26:0/727271872' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:16:15.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:14 smithi136 ceph-mon[35121]: Deploying daemon mds.b on smithi076 2024-09-11T23:16:15.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:14 smithi136 ceph-mon[35121]: mds.? [v2:172.21.15.26:6840/2984982336,v1:172.21.15.26:6841/2984982336] up:boot 2024-09-11T23:16:15.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:14 smithi136 ceph-mon[35121]: fsmap 11 up:standby 2024-09-11T23:16:15.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:14 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-11T23:16:15.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:14 smithi136 ceph-mon[35121]: from='client.? 172.21.15.26:0/727271872' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:16:16.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:15 smithi136 ceph-mon[35121]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:16:16.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:15 smithi026 ceph-mon[31172]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:16:16.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:15 smithi076 ceph-mon[35296]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:16:16.303 INFO:journalctl@ceph.mds.b.smithi076.stdout:Sep 11 23:16:16 smithi076 systemd[1]: Started Ceph mds.b for af7e6fc8-7092-11ef-bcea-c7b262605968. 2024-09-11T23:16:17.070 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:16:17.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:17 smithi026 ceph-mon[31172]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:16:17.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:17 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:16:17.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:17 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:16:17.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:17 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:16:17.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:17 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:16:17.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:17 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:16:17.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:17 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:16:17.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:17 smithi026 ceph-mon[31172]: mds.? [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] up:boot 2024-09-11T23:16:17.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:17 smithi026 ceph-mon[31172]: fsmap 12 up:standby 2024-09-11T23:16:17.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:17 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-11T23:16:17.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:17 smithi136 ceph-mon[35121]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:16:17.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:17 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:16:17.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:17 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:16:17.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:17 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:16:17.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:17 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:16:17.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:17 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:16:17.490 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:17 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:16:17.490 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:17 smithi136 ceph-mon[35121]: mds.? [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] up:boot 2024-09-11T23:16:17.490 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:17 smithi136 ceph-mon[35121]: fsmap 12 up:standby 2024-09-11T23:16:17.490 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:17 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-11T23:16:17.512 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:16:17.553 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:17 smithi076 ceph-mon[35296]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:16:17.553 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:17 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:16:17.553 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:17 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:16:17.553 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:17 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:16:17.553 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:17 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:16:17.553 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:17 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:16:17.553 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:17 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:16:17.553 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:17 smithi076 ceph-mon[35296]: mds.? [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] up:boot 2024-09-11T23:16:17.553 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:17 smithi076 ceph-mon[35296]: fsmap 12 up:standby 2024-09-11T23:16:17.553 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:17 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-11T23:16:17.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:17.970+0000 7fd2635ad640 1 -- 172.21.15.26:0/230564379 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd25c1055b0 msgr2=0x7fd25c10d130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:16:17.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:17.970+0000 7fd2635ad640 1 --2- 172.21.15.26:0/230564379 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd25c1055b0 0x7fd25c10d130 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fd254009ca0 tx=0x7fd25402a8c0 comp rx=0 tx=0).stop 2024-09-11T23:16:17.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:17.971+0000 7fd2635ad640 1 -- 172.21.15.26:0/230564379 shutdown_connections 2024-09-11T23:16:17.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:17.971+0000 7fd2635ad640 1 --2- 172.21.15.26:0/230564379 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd25c100e30 0x7fd25c104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:17.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:17.971+0000 7fd2635ad640 1 --2- 172.21.15.26:0/230564379 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd25c1055b0 0x7fd25c10d130 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:17.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:17.971+0000 7fd2635ad640 1 --2- 172.21.15.26:0/230564379 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd25c100480 0x7fd25c100860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:17.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:17.971+0000 7fd2635ad640 1 -- 172.21.15.26:0/230564379 >> 172.21.15.26:0/230564379 conn(0x7fd25c077210 msgr2=0x7fd25c0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:16:17.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:17.971+0000 7fd2635ad640 1 -- 172.21.15.26:0/230564379 shutdown_connections 2024-09-11T23:16:17.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:17.971+0000 7fd2635ad640 1 -- 172.21.15.26:0/230564379 wait complete. 2024-09-11T23:16:17.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:17.971+0000 7fd2635ad640 1 Processor -- start 2024-09-11T23:16:17.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:17.971+0000 7fd2635ad640 1 -- start start 2024-09-11T23:16:17.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:17.972+0000 7fd2635ad640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd25c100480 0x7fd25c10e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:16:17.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:17.972+0000 7fd2635ad640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd25c100e30 0x7fd25c10ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:16:17.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:17.972+0000 7fd2635ad640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd25c1055b0 0x7fd25c10f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:16:17.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:17.972+0000 7fd2635ad640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd25c111f90 con 0x7fd25c100480 2024-09-11T23:16:17.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:17.972+0000 7fd2635ad640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fd25c111e10 con 0x7fd25c100e30 2024-09-11T23:16:17.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:17.972+0000 7fd2635ad640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd25c112110 con 0x7fd25c1055b0 2024-09-11T23:16:17.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:17.972+0000 7fd261349640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd25c100480 0x7fd25c10e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:16:17.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:17.972+0000 7fd261349640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd25c100480 0x7fd25c10e850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37888/0 (socket says 172.21.15.26:37888) 2024-09-11T23:16:17.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:17.972+0000 7fd261349640 1 -- 172.21.15.26:0/2941584685 learned_addr learned my addr 172.21.15.26:0/2941584685 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:16:17.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:17.972+0000 7fd260b48640 1 --2- 172.21.15.26:0/2941584685 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd25c100e30 0x7fd25c10ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:16:17.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:17.972+0000 7fd261b4a640 1 --2- 172.21.15.26:0/2941584685 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd25c1055b0 0x7fd25c10f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:16:17.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:17.972+0000 7fd261349640 1 -- 172.21.15.26:0/2941584685 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd25c1055b0 msgr2=0x7fd25c10f2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:16:17.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:17.972+0000 7fd261349640 1 --2- 172.21.15.26:0/2941584685 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd25c1055b0 0x7fd25c10f2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:17.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:17.972+0000 7fd261349640 1 -- 172.21.15.26:0/2941584685 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd25c100e30 msgr2=0x7fd25c10ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:16:17.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:17.972+0000 7fd261349640 1 --2- 172.21.15.26:0/2941584685 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd25c100e30 0x7fd25c10ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:17.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:17.972+0000 7fd261349640 1 -- 172.21.15.26:0/2941584685 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd25c1a6810 con 0x7fd25c100480 2024-09-11T23:16:17.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:17.972+0000 7fd261349640 1 --2- 172.21.15.26:0/2941584685 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd25c100480 0x7fd25c10e850 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fd25800d950 tx=0x7fd25800dd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:16:17.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:17.972+0000 7fd2527fc640 1 -- 172.21.15.26:0/2941584685 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd258017030 con 0x7fd25c100480 2024-09-11T23:16:17.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:17.972+0000 7fd2527fc640 1 -- 172.21.15.26:0/2941584685 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd258004020 con 0x7fd25c100480 2024-09-11T23:16:17.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:17.973+0000 7fd2527fc640 1 -- 172.21.15.26:0/2941584685 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd258005650 con 0x7fd25c100480 2024-09-11T23:16:17.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:17.973+0000 7fd2635ad640 1 -- 172.21.15.26:0/2941584685 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd25c1a6b00 con 0x7fd25c100480 2024-09-11T23:16:17.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:17.973+0000 7fd2635ad640 1 -- 172.21.15.26:0/2941584685 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd25c1a6ff0 con 0x7fd25c100480 2024-09-11T23:16:17.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:17.974+0000 7fd2635ad640 1 -- 172.21.15.26:0/2941584685 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd25c100860 con 0x7fd25c100480 2024-09-11T23:16:17.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:17.976+0000 7fd2527fc640 1 -- 172.21.15.26:0/2941584685 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fd2580041d0 con 0x7fd25c100480 2024-09-11T23:16:17.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:17.976+0000 7fd2527fc640 1 --2- 172.21.15.26:0/2941584685 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fd2480a2c00 0x7fd2480a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:16:17.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:17.977+0000 7fd2527fc640 1 -- 172.21.15.26:0/2941584685 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7127+0+0 (secure 0 0 0) 0x7fd2580ceb50 con 0x7fd25c100480 2024-09-11T23:16:17.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:17.977+0000 7fd260b48640 1 --2- 172.21.15.26:0/2941584685 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fd2480a2c00 0x7fd2480a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:16:17.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:17.977+0000 7fd260b48640 1 --2- 172.21.15.26:0/2941584685 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fd2480a2c00 0x7fd2480a50c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fd25c1102e0 tx=0x7fd24c00a2d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:16:17.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:17.980+0000 7fd2527fc640 1 -- 172.21.15.26:0/2941584685 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd25809bfd0 con 0x7fd25c100480 2024-09-11T23:16:18.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:18.202+0000 7fd2635ad640 1 -- 172.21.15.26:0/2941584685 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd25c10fd80 con 0x7fd25c100480 2024-09-11T23:16:18.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:18.205+0000 7fd2527fc640 1 -- 172.21.15.26:0/2941584685 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10521 (secure 0 0 0) 0x7fd25809b8d0 con 0x7fd25c100480 2024-09-11T23:16:18.207 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-11T23:16:18.207 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":13,"btime":"2024-09-11T23:16:16:961953+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14499,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/367659515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":367659515},{"type":"v1","addr":"172.21.15.26:6835","nonce":367659515}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14511,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/2041065299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":2041065299},{"type":"v1","addr":"172.21.15.26:6837","nonce":2041065299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14550,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6839/2233068305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":2233068305},{"type":"v1","addr":"172.21.15.26:6839","nonce":2233068305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14562,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6841/2984982336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":2984982336},{"type":"v1","addr":"172.21.15.26:6841","nonce":2984982336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6833/2019841455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":2019841455},{"type":"v1","addr":"172.21.15.136:6833","nonce":2019841455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6833/3438147050","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3438147050},{"type":"v1","addr":"172.21.15.76:6833","nonce":3438147050}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24439,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/3617665975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3617665975},{"type":"v1","addr":"172.21.15.136:6835","nonce":3617665975}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24443,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/393308499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":393308499},{"type":"v1","addr":"172.21.15.76:6835","nonce":393308499}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24448,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/1633082387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":1633082387},{"type":"v1","addr":"172.21.15.136:6837","nonce":1633082387}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24458,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6837/2078978748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":2078978748},{"type":"v1","addr":"172.21.15.76:6837","nonce":2078978748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24463,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/3211398630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":3211398630},{"type":"v1","addr":"172.21.15.136:6839","nonce":3211398630}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24485,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6839/2077637090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6838","nonce":2077637090},{"type":"v1","addr":"172.21.15.76:6839","nonce":2077637090}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-09-11T23:16:18.207 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 13 2024-09-11T23:16:18.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:18.208+0000 7fd2635ad640 1 -- 172.21.15.26:0/2941584685 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fd2480a2c00 msgr2=0x7fd2480a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:16:18.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:18.208+0000 7fd2635ad640 1 --2- 172.21.15.26:0/2941584685 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fd2480a2c00 0x7fd2480a50c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fd25c1102e0 tx=0x7fd24c00a2d0 comp rx=0 tx=0).stop 2024-09-11T23:16:18.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:18.208+0000 7fd2635ad640 1 -- 172.21.15.26:0/2941584685 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd25c100480 msgr2=0x7fd25c10e850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:16:18.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:18.208+0000 7fd2635ad640 1 --2- 172.21.15.26:0/2941584685 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd25c100480 0x7fd25c10e850 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fd25800d950 tx=0x7fd25800dd50 comp rx=0 tx=0).stop 2024-09-11T23:16:18.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:18.208+0000 7fd2635ad640 1 -- 172.21.15.26:0/2941584685 shutdown_connections 2024-09-11T23:16:18.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:18.208+0000 7fd2635ad640 1 --2- 172.21.15.26:0/2941584685 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fd2480a2c00 0x7fd2480a50c0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:18.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:18.208+0000 7fd2635ad640 1 --2- 172.21.15.26:0/2941584685 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd25c100e30 0x7fd25c10ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:18.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:18.208+0000 7fd2635ad640 1 --2- 172.21.15.26:0/2941584685 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd25c1055b0 0x7fd25c10f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:18.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:18.208+0000 7fd2635ad640 1 --2- 172.21.15.26:0/2941584685 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd25c100480 0x7fd25c10e850 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:18.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:18.208+0000 7fd2635ad640 1 -- 172.21.15.26:0/2941584685 >> 172.21.15.26:0/2941584685 conn(0x7fd25c077210 msgr2=0x7fd25c103c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:16:18.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:18.208+0000 7fd2635ad640 1 -- 172.21.15.26:0/2941584685 shutdown_connections 2024-09-11T23:16:18.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:18.208+0000 7fd2635ad640 1 -- 172.21.15.26:0/2941584685 wait complete. 2024-09-11T23:16:18.869 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:18 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:16:18.869 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:18 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:16:18.869 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:18 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:16:18.869 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:18 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:16:18.870 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:18 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:16:18.870 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:18 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:16:18.870 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:18 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:16:18.870 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:18 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:16:18.870 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:18 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:16:18.870 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:18 smithi026 ceph-mon[31172]: from='client.? 172.21.15.26:0/2941584685' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:16:18.870 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:18 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:16:18.898 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph --format=json-pretty osd lspools 2024-09-11T23:16:19.021 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:18 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:16:19.021 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:18 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:16:19.021 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:18 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:16:19.022 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:18 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:16:19.022 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:18 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:16:19.022 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:18 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:16:19.022 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:18 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:16:19.022 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:18 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:16:19.022 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:18 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:16:19.022 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:18 smithi136 ceph-mon[35121]: from='client.? 172.21.15.26:0/2941584685' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:16:19.022 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:18 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:16:19.027 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:18 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:16:19.027 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:18 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:16:19.027 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:18 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:16:19.027 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:18 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:16:19.027 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:18 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:16:19.028 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:18 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:16:19.028 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:18 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:16:19.028 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:18 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:16:19.028 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:18 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:16:19.028 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:18 smithi076 ceph-mon[35296]: from='client.? 172.21.15.26:0/2941584685' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:16:19.028 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:18 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:16:19.818 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:16:20.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:19 smithi076 ceph-mon[35296]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:16:20.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:19 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:16:20.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:19 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:16:20.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:19 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:16:20.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:19 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:16:20.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:19 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:16:20.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:19 smithi026 ceph-mon[31172]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:16:20.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:19 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:16:20.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:19 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:16:20.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:19 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:16:20.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:19 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:16:20.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:19 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:16:20.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:19 smithi136 ceph-mon[35121]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:16:20.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:19 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:16:20.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:19 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:16:20.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:19 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:16:20.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:19 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:16:20.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:19 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:16:20.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:20.280+0000 7fe418545640 1 -- 172.21.15.26:0/3714347381 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe410100a80 msgr2=0x7fe410100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:16:20.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:20.280+0000 7fe418545640 1 --2- 172.21.15.26:0/3714347381 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe410100a80 0x7fe410100ee0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fe400009930 tx=0x7fe40002a420 comp rx=0 tx=0).stop 2024-09-11T23:16:20.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:20.280+0000 7fe418545640 1 -- 172.21.15.26:0/3714347381 shutdown_connections 2024-09-11T23:16:20.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:20.280+0000 7fe418545640 1 --2- 172.21.15.26:0/3714347381 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe410101420 0x7fe41010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:20.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:20.280+0000 7fe418545640 1 --2- 172.21.15.26:0/3714347381 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe410100a80 0x7fe410100ee0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:20.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:20.280+0000 7fe418545640 1 --2- 172.21.15.26:0/3714347381 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe410106a80 0x7fe410106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:20.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:20.280+0000 7fe418545640 1 -- 172.21.15.26:0/3714347381 >> 172.21.15.26:0/3714347381 conn(0x7fe4100fc7a0 msgr2=0x7fe4100febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:16:20.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:20.280+0000 7fe418545640 1 -- 172.21.15.26:0/3714347381 shutdown_connections 2024-09-11T23:16:20.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:20.280+0000 7fe418545640 1 -- 172.21.15.26:0/3714347381 wait complete. 2024-09-11T23:16:20.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:20.281+0000 7fe418545640 1 Processor -- start 2024-09-11T23:16:20.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:20.281+0000 7fe418545640 1 -- start start 2024-09-11T23:16:20.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:20.281+0000 7fe418545640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe410100a80 0x7fe4101973b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:16:20.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:20.281+0000 7fe418545640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe410101420 0x7fe4101978f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:16:20.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:20.281+0000 7fe418545640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe410106a80 0x7fe41019bc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:16:20.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:20.281+0000 7fe418545640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fe410110ed0 con 0x7fe410101420 2024-09-11T23:16:20.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:20.281+0000 7fe418545640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fe410110d50 con 0x7fe410106a80 2024-09-11T23:16:20.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:20.281+0000 7fe418545640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe410111050 con 0x7fe410100a80 2024-09-11T23:16:20.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:20.281+0000 7fe415ae0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe410101420 0x7fe4101978f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:16:20.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:20.281+0000 7fe415ae0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe410101420 0x7fe4101978f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37900/0 (socket says 172.21.15.26:37900) 2024-09-11T23:16:20.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:20.281+0000 7fe415ae0640 1 -- 172.21.15.26:0/4236958195 learned_addr learned my addr 172.21.15.26:0/4236958195 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:16:20.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:20.282+0000 7fe4162e1640 1 --2- 172.21.15.26:0/4236958195 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe410100a80 0x7fe4101973b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:16:20.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:20.282+0000 7fe416ae2640 1 --2- 172.21.15.26:0/4236958195 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe410106a80 0x7fe41019bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:16:20.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:20.282+0000 7fe415ae0640 1 -- 172.21.15.26:0/4236958195 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe410100a80 msgr2=0x7fe4101973b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:16:20.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:20.282+0000 7fe415ae0640 1 --2- 172.21.15.26:0/4236958195 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe410100a80 0x7fe4101973b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:20.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:20.282+0000 7fe415ae0640 1 -- 172.21.15.26:0/4236958195 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe410106a80 msgr2=0x7fe41019bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:16:20.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:20.282+0000 7fe415ae0640 1 --2- 172.21.15.26:0/4236958195 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe410106a80 0x7fe41019bc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:20.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:20.282+0000 7fe415ae0640 1 -- 172.21.15.26:0/4236958195 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe41019c360 con 0x7fe410101420 2024-09-11T23:16:20.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:20.282+0000 7fe415ae0640 1 --2- 172.21.15.26:0/4236958195 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe410101420 0x7fe4101978f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fe40002a940 tx=0x7fe400003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:16:20.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:20.282+0000 7fe4077fe640 1 -- 172.21.15.26:0/4236958195 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe400035070 con 0x7fe410101420 2024-09-11T23:16:20.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:20.282+0000 7fe4077fe640 1 -- 172.21.15.26:0/4236958195 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe400031a90 con 0x7fe410101420 2024-09-11T23:16:20.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:20.282+0000 7fe418545640 1 -- 172.21.15.26:0/4236958195 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe41019c5f0 con 0x7fe410101420 2024-09-11T23:16:20.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:20.282+0000 7fe4077fe640 1 -- 172.21.15.26:0/4236958195 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe400038bf0 con 0x7fe410101420 2024-09-11T23:16:20.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:20.282+0000 7fe418545640 1 -- 172.21.15.26:0/4236958195 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe41006a790 con 0x7fe410101420 2024-09-11T23:16:20.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:20.283+0000 7fe418545640 1 -- 172.21.15.26:0/4236958195 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4101021c0 con 0x7fe410101420 2024-09-11T23:16:20.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:20.284+0000 7fe4077fe640 1 -- 172.21.15.26:0/4236958195 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fe400031c40 con 0x7fe410101420 2024-09-11T23:16:20.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:20.285+0000 7fe4077fe640 1 --2- 172.21.15.26:0/4236958195 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fe3fc0a2b80 0x7fe3fc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:16:20.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:20.285+0000 7fe4162e1640 1 --2- 172.21.15.26:0/4236958195 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fe3fc0a2b80 0x7fe3fc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:16:20.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:20.285+0000 7fe4077fe640 1 -- 172.21.15.26:0/4236958195 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7127+0+0 (secure 0 0 0) 0x7fe4000f0cf0 con 0x7fe410101420 2024-09-11T23:16:20.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:20.286+0000 7fe4162e1640 1 --2- 172.21.15.26:0/4236958195 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fe3fc0a2b80 0x7fe3fc0a5040 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fe40c00bb60 tx=0x7fe40c00b390 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:16:20.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:20.289+0000 7fe4077fe640 1 -- 172.21.15.26:0/4236958195 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe4000f5070 con 0x7fe410101420 2024-09-11T23:16:20.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:20.486+0000 7fe418545640 1 -- 172.21.15.26:0/4236958195 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fe41006b1b0 con 0x7fe410101420 2024-09-11T23:16:20.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:20.487+0000 7fe4077fe640 1 -- 172.21.15.26:0/4236958195 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v78) ==== 84+0+65 (secure 0 0 0) 0x7fe4000be0f0 con 0x7fe410101420 2024-09-11T23:16:20.488 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-11T23:16:20.488 INFO:teuthology.orchestra.run.smithi026.stdout:[ 2024-09-11T23:16:20.488 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-11T23:16:20.489 INFO:teuthology.orchestra.run.smithi026.stdout: "poolnum": 1, 2024-09-11T23:16:20.489 INFO:teuthology.orchestra.run.smithi026.stdout: "poolname": ".mgr" 2024-09-11T23:16:20.489 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:16:20.489 INFO:teuthology.orchestra.run.smithi026.stdout:] 2024-09-11T23:16:20.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:20.489+0000 7fe418545640 1 -- 172.21.15.26:0/4236958195 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fe3fc0a2b80 msgr2=0x7fe3fc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:16:20.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:20.489+0000 7fe418545640 1 --2- 172.21.15.26:0/4236958195 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fe3fc0a2b80 0x7fe3fc0a5040 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fe40c00bb60 tx=0x7fe40c00b390 comp rx=0 tx=0).stop 2024-09-11T23:16:20.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:20.489+0000 7fe418545640 1 -- 172.21.15.26:0/4236958195 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe410101420 msgr2=0x7fe4101978f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:16:20.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:20.489+0000 7fe418545640 1 --2- 172.21.15.26:0/4236958195 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe410101420 0x7fe4101978f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fe40002a940 tx=0x7fe400003b90 comp rx=0 tx=0).stop 2024-09-11T23:16:20.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:20.489+0000 7fe418545640 1 -- 172.21.15.26:0/4236958195 shutdown_connections 2024-09-11T23:16:20.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:20.489+0000 7fe418545640 1 --2- 172.21.15.26:0/4236958195 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fe3fc0a2b80 0x7fe3fc0a5040 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:20.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:20.489+0000 7fe418545640 1 --2- 172.21.15.26:0/4236958195 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe410106a80 0x7fe41019bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:20.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:20.489+0000 7fe418545640 1 --2- 172.21.15.26:0/4236958195 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe410101420 0x7fe4101978f0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:20.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:20.489+0000 7fe418545640 1 --2- 172.21.15.26:0/4236958195 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe410100a80 0x7fe4101973b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:20.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:20.489+0000 7fe418545640 1 -- 172.21.15.26:0/4236958195 >> 172.21.15.26:0/4236958195 conn(0x7fe4100fc7a0 msgr2=0x7fe4100fcfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:16:20.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:20.489+0000 7fe418545640 1 -- 172.21.15.26:0/4236958195 shutdown_connections 2024-09-11T23:16:20.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:20.489+0000 7fe418545640 1 -- 172.21.15.26:0/4236958195 wait complete. 2024-09-11T23:16:21.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:20 smithi076 ceph-mon[35296]: from='client.? 172.21.15.26:0/4236958195' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-11T23:16:21.104 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:20 smithi026 ceph-mon[31172]: from='client.? 172.21.15.26:0/4236958195' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-11T23:16:21.135 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-11T23:16:21.135 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-11T23:16:21.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:20 smithi136 ceph-mon[35121]: from='client.? 172.21.15.26:0/4236958195' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-11T23:16:21.405 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:16:21.798 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:21 smithi026 ceph-mon[31172]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:16:21.799 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:21 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:16:21.799 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:21 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:16:21.799 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:21 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:16:21.799 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:21 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:16:21.799 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:21 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:16:21.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:21.884+0000 7f03a3d8d640 1 -- 172.21.15.26:0/1317083348 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f039c100a80 msgr2=0x7f039c100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:16:21.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:21.884+0000 7f03a3d8d640 1 --2- 172.21.15.26:0/1317083348 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f039c100a80 0x7f039c100ee0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f0390009930 tx=0x7f039002a420 comp rx=0 tx=0).stop 2024-09-11T23:16:21.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:21.884+0000 7f03a3d8d640 1 -- 172.21.15.26:0/1317083348 shutdown_connections 2024-09-11T23:16:21.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:21.884+0000 7f03a3d8d640 1 --2- 172.21.15.26:0/1317083348 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f039c101420 0x7f039c10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:21.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:21.884+0000 7f03a3d8d640 1 --2- 172.21.15.26:0/1317083348 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f039c100a80 0x7f039c100ee0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:21.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:21.884+0000 7f03a3d8d640 1 --2- 172.21.15.26:0/1317083348 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f039c106a80 0x7f039c106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:21.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:21.884+0000 7f03a3d8d640 1 -- 172.21.15.26:0/1317083348 >> 172.21.15.26:0/1317083348 conn(0x7f039c0fc7a0 msgr2=0x7f039c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:16:21.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:21.884+0000 7f03a3d8d640 1 -- 172.21.15.26:0/1317083348 shutdown_connections 2024-09-11T23:16:21.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:21.884+0000 7f03a3d8d640 1 -- 172.21.15.26:0/1317083348 wait complete. 2024-09-11T23:16:21.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:21.885+0000 7f03a3d8d640 1 Processor -- start 2024-09-11T23:16:21.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:21.885+0000 7f03a3d8d640 1 -- start start 2024-09-11T23:16:21.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:21.885+0000 7f03a3d8d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f039c100a80 0x7f039c199350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:16:21.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:21.885+0000 7f03a3d8d640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f039c101420 0x7f039c199890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:16:21.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:21.885+0000 7f03a3d8d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f039c106a80 0x7f039c1a43d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:16:21.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:21.885+0000 7f03a3d8d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f039c110ea0 con 0x7f039c100a80 2024-09-11T23:16:21.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:21.885+0000 7f03a3d8d640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f039c110d20 con 0x7f039c106a80 2024-09-11T23:16:21.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:21.885+0000 7f03a3d8d640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f039c111020 con 0x7f039c101420 2024-09-11T23:16:21.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:21.885+0000 7f03a1b29640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f039c100a80 0x7f039c199350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:16:21.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:21.886+0000 7f03a1b29640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f039c100a80 0x7f039c199350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37706/0 (socket says 172.21.15.26:37706) 2024-09-11T23:16:21.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:21.886+0000 7f03a1b29640 1 -- 172.21.15.26:0/2125195218 learned_addr learned my addr 172.21.15.26:0/2125195218 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:16:21.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:21.886+0000 7f03a1328640 1 --2- 172.21.15.26:0/2125195218 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f039c101420 0x7f039c199890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:16:21.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:21.886+0000 7f03a1b29640 1 -- 172.21.15.26:0/2125195218 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f039c101420 msgr2=0x7f039c199890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:16:21.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:21.886+0000 7f03a1b29640 1 --2- 172.21.15.26:0/2125195218 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f039c101420 0x7f039c199890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:21.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:21.886+0000 7f03a1b29640 1 -- 172.21.15.26:0/2125195218 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f039c106a80 msgr2=0x7f039c1a43d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:16:21.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:21.886+0000 7f03a1b29640 1 --2- 172.21.15.26:0/2125195218 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f039c106a80 0x7f039c1a43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:21.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:21.886+0000 7f03a1b29640 1 -- 172.21.15.26:0/2125195218 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f039c1a4910 con 0x7f039c100a80 2024-09-11T23:16:21.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:21.886+0000 7f03a1328640 1 --2- 172.21.15.26:0/2125195218 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f039c101420 0x7f039c199890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:16:21.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:21.886+0000 7f03a1b29640 1 --2- 172.21.15.26:0/2125195218 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f039c100a80 0x7f039c199350 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f039800b4a0 tx=0x7f039800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:16:21.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:21.886+0000 7f038effd640 1 -- 172.21.15.26:0/2125195218 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0398014070 con 0x7f039c100a80 2024-09-11T23:16:21.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:21.886+0000 7f038effd640 1 -- 172.21.15.26:0/2125195218 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0398010af0 con 0x7f039c100a80 2024-09-11T23:16:21.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:21.886+0000 7f03a3d8d640 1 -- 172.21.15.26:0/2125195218 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f039c1a4ae0 con 0x7f039c100a80 2024-09-11T23:16:21.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:21.886+0000 7f038effd640 1 -- 172.21.15.26:0/2125195218 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0398017d10 con 0x7f039c100a80 2024-09-11T23:16:21.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:21.886+0000 7f03a3d8d640 1 -- 172.21.15.26:0/2125195218 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f039c1a5020 con 0x7f039c100a80 2024-09-11T23:16:21.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:21.887+0000 7f03a3d8d640 1 -- 172.21.15.26:0/2125195218 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f039c100ee0 con 0x7f039c100a80 2024-09-11T23:16:21.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:21.888+0000 7f038effd640 1 -- 172.21.15.26:0/2125195218 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f0398017070 con 0x7f039c100a80 2024-09-11T23:16:21.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:21.889+0000 7f038effd640 1 --2- 172.21.15.26:0/2125195218 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f03880a2c00 0x7f03880a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:16:21.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:21.889+0000 7f03a1328640 1 --2- 172.21.15.26:0/2125195218 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f03880a2c00 0x7f03880a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:16:21.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:21.889+0000 7f038effd640 1 -- 172.21.15.26:0/2125195218 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7127+0+0 (secure 0 0 0) 0x7f03980ce3c0 con 0x7f039c100a80 2024-09-11T23:16:21.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:21.890+0000 7f03a1328640 1 --2- 172.21.15.26:0/2125195218 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f03880a2c00 0x7f03880a50c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f0390000c00 tx=0x7f0390032d50 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:16:21.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:21.893+0000 7f038effd640 1 -- 172.21.15.26:0/2125195218 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f039809b720 con 0x7f039c100a80 2024-09-11T23:16:22.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:21 smithi076 ceph-mon[35296]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:16:22.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:21 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:16:22.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:21 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:16:22.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:21 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:16:22.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:21 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:16:22.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:21 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:16:22.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:22.070+0000 7f03a3d8d640 1 -- 172.21.15.26:0/2125195218 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f039c06ad80 con 0x7f039c100a80 2024-09-11T23:16:22.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:21 smithi136 ceph-mon[35121]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:16:22.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:21 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:16:22.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:21 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:16:22.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:21 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:16:22.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:21 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:16:22.240 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:21 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:16:22.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:22.776+0000 7f038effd640 1 -- 172.21.15.26:0/2125195218 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v79) ==== 138+0+0 (secure 0 0 0) 0x7f039809b020 con 0x7f039c100a80 2024-09-11T23:16:22.777 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_metadata' created 2024-09-11T23:16:22.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:22.779+0000 7f03a3d8d640 1 -- 172.21.15.26:0/2125195218 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f03880a2c00 msgr2=0x7f03880a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:16:22.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:22.779+0000 7f03a3d8d640 1 --2- 172.21.15.26:0/2125195218 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f03880a2c00 0x7f03880a50c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f0390000c00 tx=0x7f0390032d50 comp rx=0 tx=0).stop 2024-09-11T23:16:22.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:22.779+0000 7f03a3d8d640 1 -- 172.21.15.26:0/2125195218 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f039c100a80 msgr2=0x7f039c199350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:16:22.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:22.779+0000 7f03a3d8d640 1 --2- 172.21.15.26:0/2125195218 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f039c100a80 0x7f039c199350 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f039800b4a0 tx=0x7f039800b960 comp rx=0 tx=0).stop 2024-09-11T23:16:22.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:22.779+0000 7f03a3d8d640 1 -- 172.21.15.26:0/2125195218 shutdown_connections 2024-09-11T23:16:22.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:22.779+0000 7f03a3d8d640 1 --2- 172.21.15.26:0/2125195218 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f03880a2c00 0x7f03880a50c0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:22.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:22.779+0000 7f03a3d8d640 1 --2- 172.21.15.26:0/2125195218 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f039c106a80 0x7f039c1a43d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:22.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:22.779+0000 7f03a3d8d640 1 --2- 172.21.15.26:0/2125195218 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f039c101420 0x7f039c199890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:22.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:22.779+0000 7f03a3d8d640 1 --2- 172.21.15.26:0/2125195218 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f039c100a80 0x7f039c199350 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:22.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:22.779+0000 7f03a3d8d640 1 -- 172.21.15.26:0/2125195218 >> 172.21.15.26:0/2125195218 conn(0x7f039c0fc7a0 msgr2=0x7f039c0fcf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:16:22.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:22.779+0000 7f03a3d8d640 1 -- 172.21.15.26:0/2125195218 shutdown_connections 2024-09-11T23:16:22.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:22.779+0000 7f03a3d8d640 1 -- 172.21.15.26:0/2125195218 wait complete. 2024-09-11T23:16:23.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:22 smithi076 ceph-mon[35296]: from='client.? 172.21.15.26:0/2125195218' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-11T23:16:23.111 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:22 smithi026 ceph-mon[31172]: from='client.? 172.21.15.26:0/2125195218' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-11T23:16:23.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:22 smithi136 ceph-mon[35121]: from='client.? 172.21.15.26:0/2125195218' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-11T23:16:23.463 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-11T23:16:23.727 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:16:24.046 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:23 smithi026 ceph-mon[31172]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:16:24.047 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:23 smithi026 ceph-mon[31172]: from='client.? 172.21.15.26:0/2125195218' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-11T23:16:24.047 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:23 smithi026 ceph-mon[31172]: osdmap e79: 12 total, 12 up, 12 in 2024-09-11T23:16:24.052 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:23 smithi076 ceph-mon[35296]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:16:24.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:23 smithi076 ceph-mon[35296]: from='client.? 172.21.15.26:0/2125195218' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-11T23:16:24.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:23 smithi076 ceph-mon[35296]: osdmap e79: 12 total, 12 up, 12 in 2024-09-11T23:16:24.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:23 smithi136 ceph-mon[35121]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:16:24.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:23 smithi136 ceph-mon[35121]: from='client.? 172.21.15.26:0/2125195218' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-11T23:16:24.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:23 smithi136 ceph-mon[35121]: osdmap e79: 12 total, 12 up, 12 in 2024-09-11T23:16:24.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:24.261+0000 7fb08e280640 1 -- 172.21.15.26:0/3979846027 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb088101de0 msgr2=0x7fb0881021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:16:24.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:24.261+0000 7fb08e280640 1 --2- 172.21.15.26:0/3979846027 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb088101de0 0x7fb0881021c0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fb078009900 tx=0x7fb07802a410 comp rx=0 tx=0).stop 2024-09-11T23:16:24.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:24.261+0000 7fb08e280640 1 -- 172.21.15.26:0/3979846027 shutdown_connections 2024-09-11T23:16:24.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:24.261+0000 7fb08e280640 1 --2- 172.21.15.26:0/3979846027 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb088106f10 0x7fb08810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:24.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:24.261+0000 7fb08e280640 1 --2- 172.21.15.26:0/3979846027 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb088102790 0x7fb0881067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:24.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:24.261+0000 7fb08e280640 1 --2- 172.21.15.26:0/3979846027 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb088101de0 0x7fb0881021c0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:24.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:24.261+0000 7fb08e280640 1 -- 172.21.15.26:0/3979846027 >> 172.21.15.26:0/3979846027 conn(0x7fb0880fd8c0 msgr2=0x7fb0880ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:16:24.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:24.261+0000 7fb08e280640 1 -- 172.21.15.26:0/3979846027 shutdown_connections 2024-09-11T23:16:24.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:24.261+0000 7fb08e280640 1 -- 172.21.15.26:0/3979846027 wait complete. 2024-09-11T23:16:24.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:24.261+0000 7fb08e280640 1 Processor -- start 2024-09-11T23:16:24.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:24.262+0000 7fb08e280640 1 -- start start 2024-09-11T23:16:24.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:24.262+0000 7fb08e280640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb088101de0 0x7fb0880744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:16:24.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:24.262+0000 7fb08e280640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb088102790 0x7fb088074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:16:24.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:24.262+0000 7fb08e280640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb088106f10 0x7fb088074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:16:24.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:24.262+0000 7fb08e280640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb088111f70 con 0x7fb088101de0 2024-09-11T23:16:24.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:24.262+0000 7fb08e280640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fb088111df0 con 0x7fb088102790 2024-09-11T23:16:24.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:24.262+0000 7fb08e280640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb0881120f0 con 0x7fb088106f10 2024-09-11T23:16:24.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:24.262+0000 7fb087fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb088101de0 0x7fb0880744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:16:24.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:24.262+0000 7fb087fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb088101de0 0x7fb0880744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37728/0 (socket says 172.21.15.26:37728) 2024-09-11T23:16:24.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:24.262+0000 7fb087fff640 1 -- 172.21.15.26:0/394875174 learned_addr learned my addr 172.21.15.26:0/394875174 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:16:24.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:24.262+0000 7fb07ffff640 1 --2- 172.21.15.26:0/394875174 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb088102790 0x7fb088074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:16:24.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:24.262+0000 7fb08c81d640 1 --2- 172.21.15.26:0/394875174 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb088106f10 0x7fb088074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:16:24.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:24.262+0000 7fb087fff640 1 -- 172.21.15.26:0/394875174 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb088106f10 msgr2=0x7fb088074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:16:24.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:24.262+0000 7fb087fff640 1 --2- 172.21.15.26:0/394875174 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb088106f10 0x7fb088074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:24.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:24.262+0000 7fb087fff640 1 -- 172.21.15.26:0/394875174 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb088102790 msgr2=0x7fb088074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:16:24.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:24.262+0000 7fb087fff640 1 --2- 172.21.15.26:0/394875174 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb088102790 0x7fb088074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:24.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:24.262+0000 7fb087fff640 1 -- 172.21.15.26:0/394875174 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0881a6970 con 0x7fb088101de0 2024-09-11T23:16:24.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:24.262+0000 7fb087fff640 1 --2- 172.21.15.26:0/394875174 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb088101de0 0x7fb0880744f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fb078006010 tx=0x7fb0780042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:16:24.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:24.263+0000 7fb085ffb640 1 -- 172.21.15.26:0/394875174 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb07803f030 con 0x7fb088101de0 2024-09-11T23:16:24.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:24.263+0000 7fb085ffb640 1 -- 172.21.15.26:0/394875174 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb078003ce0 con 0x7fb088101de0 2024-09-11T23:16:24.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:24.263+0000 7fb08e280640 1 -- 172.21.15.26:0/394875174 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0881a6c00 con 0x7fb088101de0 2024-09-11T23:16:24.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:24.263+0000 7fb085ffb640 1 -- 172.21.15.26:0/394875174 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb078035070 con 0x7fb088101de0 2024-09-11T23:16:24.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:24.263+0000 7fb08e280640 1 -- 172.21.15.26:0/394875174 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0881a70e0 con 0x7fb088101de0 2024-09-11T23:16:24.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:24.264+0000 7fb08e280640 1 -- 172.21.15.26:0/394875174 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0881021c0 con 0x7fb088101de0 2024-09-11T23:16:24.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:24.265+0000 7fb085ffb640 1 -- 172.21.15.26:0/394875174 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fb078042030 con 0x7fb088101de0 2024-09-11T23:16:24.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:24.265+0000 7fb085ffb640 1 --2- 172.21.15.26:0/394875174 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fb0740a2b80 0x7fb0740a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:16:24.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:24.266+0000 7fb085ffb640 1 -- 172.21.15.26:0/394875174 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7497+0+0 (secure 0 0 0) 0x7fb0780ef4d0 con 0x7fb088101de0 2024-09-11T23:16:24.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:24.266+0000 7fb07ffff640 1 --2- 172.21.15.26:0/394875174 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fb0740a2b80 0x7fb0740a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:16:24.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:24.266+0000 7fb07ffff640 1 --2- 172.21.15.26:0/394875174 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fb0740a2b80 0x7fb0740a5040 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fb088075f80 tx=0x7fb070005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:16:24.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:24.269+0000 7fb085ffb640 1 -- 172.21.15.26:0/394875174 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb0780bc820 con 0x7fb088101de0 2024-09-11T23:16:24.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:24.445+0000 7fb08e280640 1 -- 172.21.15.26:0/394875174 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fb08810a0f0 con 0x7fb088101de0 2024-09-11T23:16:24.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:24.878+0000 7fb085ffb640 1 -- 172.21.15.26:0/394875174 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v81) ==== 170+0+0 (secure 0 0 0) 0x7fb0780dc120 con 0x7fb088101de0 2024-09-11T23:16:24.878 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_data' created 2024-09-11T23:16:24.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:24.880+0000 7fb08e280640 1 -- 172.21.15.26:0/394875174 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fb0740a2b80 msgr2=0x7fb0740a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:16:24.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:24.880+0000 7fb08e280640 1 --2- 172.21.15.26:0/394875174 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fb0740a2b80 0x7fb0740a5040 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fb088075f80 tx=0x7fb070005f90 comp rx=0 tx=0).stop 2024-09-11T23:16:24.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:24.880+0000 7fb08e280640 1 -- 172.21.15.26:0/394875174 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb088101de0 msgr2=0x7fb0880744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:16:24.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:24.880+0000 7fb08e280640 1 --2- 172.21.15.26:0/394875174 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb088101de0 0x7fb0880744f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fb078006010 tx=0x7fb0780042f0 comp rx=0 tx=0).stop 2024-09-11T23:16:24.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:24.881+0000 7fb08e280640 1 -- 172.21.15.26:0/394875174 shutdown_connections 2024-09-11T23:16:24.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:24.881+0000 7fb08e280640 1 --2- 172.21.15.26:0/394875174 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fb0740a2b80 0x7fb0740a5040 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:24.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:24.881+0000 7fb08e280640 1 --2- 172.21.15.26:0/394875174 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb088102790 0x7fb088074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:24.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:24.881+0000 7fb08e280640 1 --2- 172.21.15.26:0/394875174 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb088106f10 0x7fb088074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:24.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:24.881+0000 7fb08e280640 1 --2- 172.21.15.26:0/394875174 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb088101de0 0x7fb0880744f0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:24.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:24.881+0000 7fb08e280640 1 -- 172.21.15.26:0/394875174 >> 172.21.15.26:0/394875174 conn(0x7fb0880fd8c0 msgr2=0x7fb0880fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:16:24.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:24.881+0000 7fb08e280640 1 -- 172.21.15.26:0/394875174 shutdown_connections 2024-09-11T23:16:24.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:24.881+0000 7fb08e280640 1 -- 172.21.15.26:0/394875174 wait complete. 2024-09-11T23:16:25.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:24 smithi136 ceph-mon[35121]: osdmap e80: 12 total, 12 up, 12 in 2024-09-11T23:16:25.242 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:24 smithi026 ceph-mon[31172]: osdmap e80: 12 total, 12 up, 12 in 2024-09-11T23:16:25.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:24 smithi076 ceph-mon[35296]: osdmap e80: 12 total, 12 up, 12 in 2024-09-11T23:16:25.617 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-11T23:16:25.874 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:16:25.899 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:25 smithi026 ceph-mon[31172]: pgmap v275: 33 pgs: 1 active+clean, 3 creating+activating, 3 creating+peering, 26 unknown; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:16:25.899 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:25 smithi026 ceph-mon[31172]: from='client.? 172.21.15.26:0/394875174' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-11T23:16:25.899 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:25 smithi026 ceph-mon[31172]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T23:16:25.899 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:25 smithi026 ceph-mon[31172]: from='client.? 172.21.15.26:0/394875174' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-11T23:16:25.899 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:25 smithi026 ceph-mon[31172]: osdmap e81: 12 total, 12 up, 12 in 2024-09-11T23:16:25.899 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:25 smithi026 ceph-mon[31172]: osdmap e82: 12 total, 12 up, 12 in 2024-09-11T23:16:26.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:25 smithi136 ceph-mon[35121]: pgmap v275: 33 pgs: 1 active+clean, 3 creating+activating, 3 creating+peering, 26 unknown; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:16:26.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:25 smithi136 ceph-mon[35121]: from='client.? 172.21.15.26:0/394875174' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-11T23:16:26.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:25 smithi136 ceph-mon[35121]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T23:16:26.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:25 smithi136 ceph-mon[35121]: from='client.? 172.21.15.26:0/394875174' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-11T23:16:26.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:25 smithi136 ceph-mon[35121]: osdmap e81: 12 total, 12 up, 12 in 2024-09-11T23:16:26.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:25 smithi136 ceph-mon[35121]: osdmap e82: 12 total, 12 up, 12 in 2024-09-11T23:16:26.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:25 smithi076 ceph-mon[35296]: pgmap v275: 33 pgs: 1 active+clean, 3 creating+activating, 3 creating+peering, 26 unknown; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:16:26.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:25 smithi076 ceph-mon[35296]: from='client.? 172.21.15.26:0/394875174' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-11T23:16:26.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:25 smithi076 ceph-mon[35296]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T23:16:26.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:25 smithi076 ceph-mon[35296]: from='client.? 172.21.15.26:0/394875174' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-11T23:16:26.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:25 smithi076 ceph-mon[35296]: osdmap e81: 12 total, 12 up, 12 in 2024-09-11T23:16:26.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:25 smithi076 ceph-mon[35296]: osdmap e82: 12 total, 12 up, 12 in 2024-09-11T23:16:26.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:26.322+0000 7f19eb952640 1 -- 172.21.15.26:0/2920939068 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f19e410c170 msgr2=0x7f19e410e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:16:26.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:26.322+0000 7f19eb952640 1 --2- 172.21.15.26:0/2920939068 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f19e410c170 0x7f19e410e610 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f19dc009ca0 tx=0x7f19dc02a950 comp rx=0 tx=0).stop 2024-09-11T23:16:26.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:26.322+0000 7f19eb952640 1 -- 172.21.15.26:0/2920939068 shutdown_connections 2024-09-11T23:16:26.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:26.322+0000 7f19eb952640 1 --2- 172.21.15.26:0/2920939068 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f19e410c170 0x7f19e410e610 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:26.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:26.322+0000 7f19eb952640 1 --2- 172.21.15.26:0/2920939068 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f19e4103660 0x7f19e410bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:26.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:26.322+0000 7f19eb952640 1 --2- 172.21.15.26:0/2920939068 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f19e4102d40 0x7f19e4103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:26.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:26.322+0000 7f19eb952640 1 -- 172.21.15.26:0/2920939068 >> 172.21.15.26:0/2920939068 conn(0x7f19e40fc7a0 msgr2=0x7f19e40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:16:26.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:26.322+0000 7f19eb952640 1 -- 172.21.15.26:0/2920939068 shutdown_connections 2024-09-11T23:16:26.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:26.322+0000 7f19eb952640 1 -- 172.21.15.26:0/2920939068 wait complete. 2024-09-11T23:16:26.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:26.323+0000 7f19eb952640 1 Processor -- start 2024-09-11T23:16:26.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:26.323+0000 7f19eb952640 1 -- start start 2024-09-11T23:16:26.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:26.323+0000 7f19eb952640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f19e4102d40 0x7f19e41009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:16:26.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:26.323+0000 7f19eb952640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f19e4103660 0x7f19e4100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:16:26.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:26.323+0000 7f19eb952640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f19e410c170 0x7f19e4101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:16:26.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:26.323+0000 7f19eb952640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f19e4111f70 con 0x7f19e4102d40 2024-09-11T23:16:26.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:26.323+0000 7f19eb952640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f19e4111df0 con 0x7f19e410c170 2024-09-11T23:16:26.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:26.323+0000 7f19eb952640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f19e41120f0 con 0x7f19e4103660 2024-09-11T23:16:26.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:26.323+0000 7f19e96ee640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f19e4102d40 0x7f19e41009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:16:26.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:26.323+0000 7f19e96ee640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f19e4102d40 0x7f19e41009a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37738/0 (socket says 172.21.15.26:37738) 2024-09-11T23:16:26.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:26.323+0000 7f19e96ee640 1 -- 172.21.15.26:0/4114852986 learned_addr learned my addr 172.21.15.26:0/4114852986 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:16:26.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:26.323+0000 7f19e8eed640 1 --2- 172.21.15.26:0/4114852986 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f19e4103660 0x7f19e4100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:16:26.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:26.324+0000 7f19e9eef640 1 --2- 172.21.15.26:0/4114852986 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f19e410c170 0x7f19e4101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:16:26.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:26.324+0000 7f19e96ee640 1 -- 172.21.15.26:0/4114852986 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f19e4103660 msgr2=0x7f19e4100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:16:26.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:26.324+0000 7f19e96ee640 1 --2- 172.21.15.26:0/4114852986 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f19e4103660 0x7f19e4100ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:26.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:26.324+0000 7f19e96ee640 1 -- 172.21.15.26:0/4114852986 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f19e410c170 msgr2=0x7f19e4101420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:16:26.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:26.324+0000 7f19e96ee640 1 --2- 172.21.15.26:0/4114852986 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f19e410c170 0x7f19e4101420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:26.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:26.324+0000 7f19e96ee640 1 -- 172.21.15.26:0/4114852986 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19e41a67f0 con 0x7f19e4102d40 2024-09-11T23:16:26.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:26.324+0000 7f19e8eed640 1 --2- 172.21.15.26:0/4114852986 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f19e4103660 0x7f19e4100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:16:26.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:26.324+0000 7f19e9eef640 1 --2- 172.21.15.26:0/4114852986 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f19e410c170 0x7f19e4101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:16:26.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:26.324+0000 7f19e96ee640 1 --2- 172.21.15.26:0/4114852986 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f19e4102d40 0x7f19e41009a0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f19e000c890 tx=0x7f19e000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:16:26.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:26.324+0000 7f19da7fc640 1 -- 172.21.15.26:0/4114852986 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19e0004220 con 0x7f19e4102d40 2024-09-11T23:16:26.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:26.324+0000 7f19da7fc640 1 -- 172.21.15.26:0/4114852986 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f19e00043c0 con 0x7f19e4102d40 2024-09-11T23:16:26.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:26.324+0000 7f19eb952640 1 -- 172.21.15.26:0/4114852986 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19e41a6ae0 con 0x7f19e4102d40 2024-09-11T23:16:26.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:26.324+0000 7f19da7fc640 1 -- 172.21.15.26:0/4114852986 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19e00049e0 con 0x7f19e4102d40 2024-09-11T23:16:26.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:26.324+0000 7f19eb952640 1 -- 172.21.15.26:0/4114852986 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19e41a7070 con 0x7f19e4102d40 2024-09-11T23:16:26.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:26.325+0000 7f19eb952640 1 -- 172.21.15.26:0/4114852986 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19e4103120 con 0x7f19e4102d40 2024-09-11T23:16:26.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:26.327+0000 7f19da7fc640 1 -- 172.21.15.26:0/4114852986 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f19e00079b0 con 0x7f19e4102d40 2024-09-11T23:16:26.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:26.327+0000 7f19da7fc640 1 --2- 172.21.15.26:0/4114852986 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f19d00a2c00 0x7f19d00a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:16:26.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:26.327+0000 7f19e8eed640 1 --2- 172.21.15.26:0/4114852986 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f19d00a2c00 0x7f19d00a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:16:26.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:26.327+0000 7f19da7fc640 1 -- 172.21.15.26:0/4114852986 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7879+0+0 (secure 0 0 0) 0x7f19e00cef00 con 0x7f19e4102d40 2024-09-11T23:16:26.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:26.328+0000 7f19e8eed640 1 --2- 172.21.15.26:0/4114852986 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f19d00a2c00 0x7f19d00a50c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f19e4102430 tx=0x7f19d400a5e0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:16:26.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:26.331+0000 7f19da7fc640 1 -- 172.21.15.26:0/4114852986 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f19e0014030 con 0x7f19e4102d40 2024-09-11T23:16:26.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:26.542+0000 7f19eb952640 1 -- 172.21.15.26:0/4114852986 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f19e4101ed0 con 0x7f19e4102d40 2024-09-11T23:16:26.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:26.903+0000 7f19da7fc640 1 -- 172.21.15.26:0/4114852986 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7f19e009c090 con 0x7f19e4102d40 2024-09-11T23:16:26.904 INFO:teuthology.orchestra.run.smithi026.stderr:new fs with metadata pool 2 and data pool 3 2024-09-11T23:16:26.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:26.907+0000 7f19eb952640 1 -- 172.21.15.26:0/4114852986 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f19d00a2c00 msgr2=0x7f19d00a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:16:26.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:26.907+0000 7f19eb952640 1 --2- 172.21.15.26:0/4114852986 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f19d00a2c00 0x7f19d00a50c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f19e4102430 tx=0x7f19d400a5e0 comp rx=0 tx=0).stop 2024-09-11T23:16:26.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:26.907+0000 7f19eb952640 1 -- 172.21.15.26:0/4114852986 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f19e4102d40 msgr2=0x7f19e41009a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:16:26.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:26.907+0000 7f19eb952640 1 --2- 172.21.15.26:0/4114852986 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f19e4102d40 0x7f19e41009a0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f19e000c890 tx=0x7f19e000cd50 comp rx=0 tx=0).stop 2024-09-11T23:16:26.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:26.907+0000 7f19eb952640 1 -- 172.21.15.26:0/4114852986 shutdown_connections 2024-09-11T23:16:26.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:26.907+0000 7f19eb952640 1 --2- 172.21.15.26:0/4114852986 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f19d00a2c00 0x7f19d00a50c0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:26.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:26.907+0000 7f19eb952640 1 --2- 172.21.15.26:0/4114852986 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f19e410c170 0x7f19e4101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:26.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:26.907+0000 7f19eb952640 1 --2- 172.21.15.26:0/4114852986 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f19e4103660 0x7f19e4100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:26.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:26.907+0000 7f19eb952640 1 --2- 172.21.15.26:0/4114852986 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f19e4102d40 0x7f19e41009a0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:26.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:26.907+0000 7f19eb952640 1 -- 172.21.15.26:0/4114852986 >> 172.21.15.26:0/4114852986 conn(0x7f19e40fc7a0 msgr2=0x7f19e40fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:16:26.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:26.907+0000 7f19eb952640 1 -- 172.21.15.26:0/4114852986 shutdown_connections 2024-09-11T23:16:26.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:26.907+0000 7f19eb952640 1 -- 172.21.15.26:0/4114852986 wait complete. 2024-09-11T23:16:27.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:26 smithi136 ceph-mon[35121]: from='client.? 172.21.15.26:0/4114852986' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-11T23:16:27.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:26 smithi136 ceph-mon[35121]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-11T23:16:27.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:26 smithi026 ceph-mon[31172]: from='client.? 172.21.15.26:0/4114852986' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-11T23:16:27.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:26 smithi026 ceph-mon[31172]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-11T23:16:27.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:26 smithi076 ceph-mon[35296]: from='client.? 172.21.15.26:0/4114852986' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-11T23:16:27.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:26 smithi076 ceph-mon[35296]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-11T23:16:27.561 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-11T23:16:27.822 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:16:28.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:27 smithi136 ceph-mon[35121]: pgmap v278: 97 pgs: 27 active+clean, 3 creating+activating, 24 creating+peering, 43 unknown; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:16:28.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:27 smithi136 ceph-mon[35121]: osdmap e83: 12 total, 12 up, 12 in 2024-09-11T23:16:28.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:27 smithi136 ceph-mon[35121]: from='client.? 172.21.15.26:0/4114852986' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-11T23:16:28.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:27 smithi136 ceph-mon[35121]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-11T23:16:28.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:27 smithi136 ceph-mon[35121]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-11T23:16:28.245 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:27 smithi026 ceph-mon[31172]: pgmap v278: 97 pgs: 27 active+clean, 3 creating+activating, 24 creating+peering, 43 unknown; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:16:28.245 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:27 smithi026 ceph-mon[31172]: osdmap e83: 12 total, 12 up, 12 in 2024-09-11T23:16:28.245 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:27 smithi026 ceph-mon[31172]: from='client.? 172.21.15.26:0/4114852986' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-11T23:16:28.245 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:27 smithi026 ceph-mon[31172]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-11T23:16:28.245 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:27 smithi026 ceph-mon[31172]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-11T23:16:28.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:27 smithi076 ceph-mon[35296]: pgmap v278: 97 pgs: 27 active+clean, 3 creating+activating, 24 creating+peering, 43 unknown; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:16:28.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:27 smithi076 ceph-mon[35296]: osdmap e83: 12 total, 12 up, 12 in 2024-09-11T23:16:28.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:27 smithi076 ceph-mon[35296]: from='client.? 172.21.15.26:0/4114852986' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-11T23:16:28.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:27 smithi076 ceph-mon[35296]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-11T23:16:28.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:27 smithi076 ceph-mon[35296]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-11T23:16:28.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:28.327+0000 7fcdd0eca640 1 -- 172.21.15.26:0/864564962 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcdcc106f10 msgr2=0x7fcdcc10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:16:28.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:28.327+0000 7fcdd0eca640 1 --2- 172.21.15.26:0/864564962 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcdcc106f10 0x7fcdcc10ea90 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fcdc4009ca0 tx=0x7fcdc402a8c0 comp rx=0 tx=0).stop 2024-09-11T23:16:28.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:28.327+0000 7fcdd0eca640 1 -- 172.21.15.26:0/864564962 shutdown_connections 2024-09-11T23:16:28.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:28.327+0000 7fcdd0eca640 1 --2- 172.21.15.26:0/864564962 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcdcc106f10 0x7fcdcc10ea90 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:28.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:28.327+0000 7fcdd0eca640 1 --2- 172.21.15.26:0/864564962 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcdcc102790 0x7fcdcc1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:28.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:28.327+0000 7fcdd0eca640 1 --2- 172.21.15.26:0/864564962 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcdcc101de0 0x7fcdcc1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:28.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:28.327+0000 7fcdd0eca640 1 -- 172.21.15.26:0/864564962 >> 172.21.15.26:0/864564962 conn(0x7fcdcc0fd8c0 msgr2=0x7fcdcc0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:16:28.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:28.327+0000 7fcdd0eca640 1 -- 172.21.15.26:0/864564962 shutdown_connections 2024-09-11T23:16:28.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:28.327+0000 7fcdd0eca640 1 -- 172.21.15.26:0/864564962 wait complete. 2024-09-11T23:16:28.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:28.328+0000 7fcdd0eca640 1 Processor -- start 2024-09-11T23:16:28.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:28.328+0000 7fcdd0eca640 1 -- start start 2024-09-11T23:16:28.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:28.328+0000 7fcdd0eca640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcdcc101de0 0x7fcdcc1985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:16:28.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:28.328+0000 7fcdd0eca640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcdcc102790 0x7fcdcc198b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:16:28.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:28.328+0000 7fcdd0eca640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcdcc106f10 0x7fcdcc19cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:16:28.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:28.328+0000 7fcdd0eca640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fcdcc112010 con 0x7fcdcc106f10 2024-09-11T23:16:28.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:28.328+0000 7fcdd0eca640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fcdcc111e90 con 0x7fcdcc101de0 2024-09-11T23:16:28.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:28.328+0000 7fcdd0eca640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fcdcc112190 con 0x7fcdcc102790 2024-09-11T23:16:28.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:28.328+0000 7fcdcad9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcdcc106f10 0x7fcdcc19cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:16:28.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:28.328+0000 7fcdcad9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcdcc106f10 0x7fcdcc19cec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37754/0 (socket says 172.21.15.26:37754) 2024-09-11T23:16:28.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:28.328+0000 7fcdcad9d640 1 -- 172.21.15.26:0/197723433 learned_addr learned my addr 172.21.15.26:0/197723433 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:16:28.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:28.328+0000 7fcdca59c640 1 --2- 172.21.15.26:0/197723433 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcdcc101de0 0x7fcdcc1985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:16:28.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:28.328+0000 7fcdcad9d640 1 -- 172.21.15.26:0/197723433 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcdcc102790 msgr2=0x7fcdcc198b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:16:28.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:28.328+0000 7fcdc9d9b640 1 --2- 172.21.15.26:0/197723433 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcdcc102790 0x7fcdcc198b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:16:28.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:28.328+0000 7fcdcad9d640 1 --2- 172.21.15.26:0/197723433 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcdcc102790 0x7fcdcc198b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:28.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:28.328+0000 7fcdcad9d640 1 -- 172.21.15.26:0/197723433 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcdcc101de0 msgr2=0x7fcdcc1985f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:16:28.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:28.328+0000 7fcdcad9d640 1 --2- 172.21.15.26:0/197723433 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcdcc101de0 0x7fcdcc1985f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:28.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:28.328+0000 7fcdcad9d640 1 -- 172.21.15.26:0/197723433 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdcc19d5a0 con 0x7fcdcc106f10 2024-09-11T23:16:28.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:28.328+0000 7fcdca59c640 1 --2- 172.21.15.26:0/197723433 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcdcc101de0 0x7fcdcc1985f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:16:28.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:28.329+0000 7fcdcad9d640 1 --2- 172.21.15.26:0/197723433 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcdcc106f10 0x7fcdcc19cec0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fcdc4031a60 tx=0x7fcdc4031a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:16:28.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:28.329+0000 7fcdbb7fe640 1 -- 172.21.15.26:0/197723433 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdc4003c30 con 0x7fcdcc106f10 2024-09-11T23:16:28.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:28.329+0000 7fcdbb7fe640 1 -- 172.21.15.26:0/197723433 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcdc4003dd0 con 0x7fcdcc106f10 2024-09-11T23:16:28.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:28.329+0000 7fcdd0eca640 1 -- 172.21.15.26:0/197723433 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcdcc19d830 con 0x7fcdcc106f10 2024-09-11T23:16:28.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:28.329+0000 7fcdbb7fe640 1 -- 172.21.15.26:0/197723433 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdc40047e0 con 0x7fcdcc106f10 2024-09-11T23:16:28.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:28.329+0000 7fcdd0eca640 1 -- 172.21.15.26:0/197723433 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcdcc1a5170 con 0x7fcdcc106f10 2024-09-11T23:16:28.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:28.330+0000 7fcdd0eca640 1 -- 172.21.15.26:0/197723433 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcdcc102240 con 0x7fcdcc106f10 2024-09-11T23:16:28.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:28.332+0000 7fcdbb7fe640 1 -- 172.21.15.26:0/197723433 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fcdc402f790 con 0x7fcdcc106f10 2024-09-11T23:16:28.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:28.333+0000 7fcdbb7fe640 1 --2- 172.21.15.26:0/197723433 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fcdb40a2b80 0x7fcdb40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:16:28.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:28.333+0000 7fcdca59c640 1 --2- 172.21.15.26:0/197723433 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fcdb40a2b80 0x7fcdb40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:16:28.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:28.333+0000 7fcdbb7fe640 1 -- 172.21.15.26:0/197723433 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7979+0+0 (secure 0 0 0) 0x7fcdc40f05c0 con 0x7fcdcc106f10 2024-09-11T23:16:28.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:28.333+0000 7fcdca59c640 1 --2- 172.21.15.26:0/197723433 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fcdb40a2b80 0x7fcdb40a5040 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fcdbc009930 tx=0x7fcdbc008040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:16:28.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:28.336+0000 7fcdbb7fe640 1 -- 172.21.15.26:0/197723433 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fcdc40bd660 con 0x7fcdcc106f10 2024-09-11T23:16:28.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:28.511+0000 7fcdd0eca640 1 -- 172.21.15.26:0/197723433 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcdcc199950 con 0x7fcdcc106f10 2024-09-11T23:16:28.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:28.513+0000 7fcdbb7fe640 1 -- 172.21.15.26:0/197723433 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+22252 (secure 0 0 0) 0x7fcdcc199950 con 0x7fcdcc106f10 2024-09-11T23:16:28.513 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-11T23:16:28.513 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":83,"fsid":"af7e6fc8-7092-11ef-bcea-c7b262605968","created":"2024-09-11T23:08:40.486438+0000","modified":"2024-09-11T23:16:26.544002+0000","last_up_change":"2024-09-11T23:15:35.179446+0000","last_in_change":"2024-09-11T23:15:11.064627+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T23:11:50.494160+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-11T23:16:22.503928+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-11T23:16:24.876554+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a4d62fcf-c31e-41b1-932c-3d87ff640531","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":2014857905},{"type":"v1","addr":"172.21.15.26:6803","nonce":2014857905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":2014857905},{"type":"v1","addr":"172.21.15.26:6805","nonce":2014857905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":2014857905},{"type":"v1","addr":"172.21.15.26:6809","nonce":2014857905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":2014857905},{"type":"v1","addr":"172.21.15.26:6807","nonce":2014857905}]},"public_addr":"172.21.15.26:6803/2014857905","cluster_addr":"172.21.15.26:6805/2014857905","heartbeat_back_addr":"172.21.15.26:6809/2014857905","heartbeat_front_addr":"172.21.15.26:6807/2014857905","state":["exists","up"]},{"osd":1,"uuid":"e0d69112-3c0a-4fc5-8783-e41b4081e0c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":333383087},{"type":"v1","addr":"172.21.15.76:6801","nonce":333383087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":333383087},{"type":"v1","addr":"172.21.15.76:6803","nonce":333383087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":333383087},{"type":"v1","addr":"172.21.15.76:6807","nonce":333383087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":333383087},{"type":"v1","addr":"172.21.15.76:6805","nonce":333383087}]},"public_addr":"172.21.15.76:6801/333383087","cluster_addr":"172.21.15.76:6803/333383087","heartbeat_back_addr":"172.21.15.76:6807/333383087","heartbeat_front_addr":"172.21.15.76:6805/333383087","state":["exists","up"]},{"osd":2,"uuid":"d578429e-bf23-4a1e-890f-7172955c0eb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":1271538911},{"type":"v1","addr":"172.21.15.136:6801","nonce":1271538911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":1271538911},{"type":"v1","addr":"172.21.15.136:6803","nonce":1271538911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":1271538911},{"type":"v1","addr":"172.21.15.136:6807","nonce":1271538911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":1271538911},{"type":"v1","addr":"172.21.15.136:6805","nonce":1271538911}]},"public_addr":"172.21.15.136:6801/1271538911","cluster_addr":"172.21.15.136:6803/1271538911","heartbeat_back_addr":"172.21.15.136:6807/1271538911","heartbeat_front_addr":"172.21.15.136:6805/1271538911","state":["exists","up"]},{"osd":3,"uuid":"3e34623f-0ea5-4b10-b8a9-f8756c35d512","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":1740304151},{"type":"v1","addr":"172.21.15.26:6811","nonce":1740304151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":1740304151},{"type":"v1","addr":"172.21.15.26:6813","nonce":1740304151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1740304151},{"type":"v1","addr":"172.21.15.26:6817","nonce":1740304151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":1740304151},{"type":"v1","addr":"172.21.15.26:6815","nonce":1740304151}]},"public_addr":"172.21.15.26:6811/1740304151","cluster_addr":"172.21.15.26:6813/1740304151","heartbeat_back_addr":"172.21.15.26:6817/1740304151","heartbeat_front_addr":"172.21.15.26:6815/1740304151","state":["exists","up"]},{"osd":4,"uuid":"77607929-f9ed-49a9-abb5-48cd88eec4ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":2702262182},{"type":"v1","addr":"172.21.15.76:6809","nonce":2702262182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":2702262182},{"type":"v1","addr":"172.21.15.76:6811","nonce":2702262182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":2702262182},{"type":"v1","addr":"172.21.15.76:6815","nonce":2702262182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":2702262182},{"type":"v1","addr":"172.21.15.76:6813","nonce":2702262182}]},"public_addr":"172.21.15.76:6809/2702262182","cluster_addr":"172.21.15.76:6811/2702262182","heartbeat_back_addr":"172.21.15.76:6815/2702262182","heartbeat_front_addr":"172.21.15.76:6813/2702262182","state":["exists","up"]},{"osd":5,"uuid":"29e20e37-09a0-4924-8aaa-17939763a7c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":1749229277},{"type":"v1","addr":"172.21.15.136:6809","nonce":1749229277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":1749229277},{"type":"v1","addr":"172.21.15.136:6811","nonce":1749229277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":1749229277},{"type":"v1","addr":"172.21.15.136:6815","nonce":1749229277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":1749229277},{"type":"v1","addr":"172.21.15.136:6813","nonce":1749229277}]},"public_addr":"172.21.15.136:6809/1749229277","cluster_addr":"172.21.15.136:6811/1749229277","heartbeat_back_addr":"172.21.15.136:6815/1749229277","heartbeat_front_addr":"172.21.15.136:6813/1749229277","state":["exists","up"]},{"osd":6,"uuid":"e0ddd1c0-1c46-41e6-b2f4-396bdcd2bf8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1928245624},{"type":"v1","addr":"172.21.15.26:6819","nonce":1928245624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1928245624},{"type":"v1","addr":"172.21.15.26:6821","nonce":1928245624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":1928245624},{"type":"v1","addr":"172.21.15.26:6825","nonce":1928245624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1928245624},{"type":"v1","addr":"172.21.15.26:6823","nonce":1928245624}]},"public_addr":"172.21.15.26:6819/1928245624","cluster_addr":"172.21.15.26:6821/1928245624","heartbeat_back_addr":"172.21.15.26:6825/1928245624","heartbeat_front_addr":"172.21.15.26:6823/1928245624","state":["exists","up"]},{"osd":7,"uuid":"fe031f5c-d915-48fb-91a6-c76f252467fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":1409789455},{"type":"v1","addr":"172.21.15.76:6817","nonce":1409789455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":1409789455},{"type":"v1","addr":"172.21.15.76:6819","nonce":1409789455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":1409789455},{"type":"v1","addr":"172.21.15.76:6823","nonce":1409789455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":1409789455},{"type":"v1","addr":"172.21.15.76:6821","nonce":1409789455}]},"public_addr":"172.21.15.76:6817/1409789455","cluster_addr":"172.21.15.76:6819/1409789455","heartbeat_back_addr":"172.21.15.76:6823/1409789455","heartbeat_front_addr":"172.21.15.76:6821/1409789455","state":["exists","up"]},{"osd":8,"uuid":"18851bde-9e6c-4e05-b4bf-ac5c3073eee7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":543619079},{"type":"v1","addr":"172.21.15.136:6817","nonce":543619079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":543619079},{"type":"v1","addr":"172.21.15.136:6819","nonce":543619079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":543619079},{"type":"v1","addr":"172.21.15.136:6823","nonce":543619079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":543619079},{"type":"v1","addr":"172.21.15.136:6821","nonce":543619079}]},"public_addr":"172.21.15.136:6817/543619079","cluster_addr":"172.21.15.136:6819/543619079","heartbeat_back_addr":"172.21.15.136:6823/543619079","heartbeat_front_addr":"172.21.15.136:6821/543619079","state":["exists","up"]},{"osd":9,"uuid":"52a65eab-8446-47ad-bfe7-5133f7e3eb98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":2201297064},{"type":"v1","addr":"172.21.15.26:6827","nonce":2201297064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":2201297064},{"type":"v1","addr":"172.21.15.26:6829","nonce":2201297064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":2201297064},{"type":"v1","addr":"172.21.15.26:6833","nonce":2201297064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":2201297064},{"type":"v1","addr":"172.21.15.26:6831","nonce":2201297064}]},"public_addr":"172.21.15.26:6827/2201297064","cluster_addr":"172.21.15.26:6829/2201297064","heartbeat_back_addr":"172.21.15.26:6833/2201297064","heartbeat_front_addr":"172.21.15.26:6831/2201297064","state":["exists","up"]},{"osd":10,"uuid":"b758d4bb-da62-4c5f-a992-bb6a145d35d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":3184128096},{"type":"v1","addr":"172.21.15.76:6825","nonce":3184128096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":3184128096},{"type":"v1","addr":"172.21.15.76:6827","nonce":3184128096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":3184128096},{"type":"v1","addr":"172.21.15.76:6831","nonce":3184128096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":3184128096},{"type":"v1","addr":"172.21.15.76:6829","nonce":3184128096}]},"public_addr":"172.21.15.76:6825/3184128096","cluster_addr":"172.21.15.76:6827/3184128096","heartbeat_back_addr":"172.21.15.76:6831/3184128096","heartbeat_front_addr":"172.21.15.76:6829/3184128096","state":["exists","up"]},{"osd":11,"uuid":"8fddda46-e22b-4d0c-ac48-e038fb29dfec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":1960689685},{"type":"v1","addr":"172.21.15.136:6825","nonce":1960689685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":1960689685},{"type":"v1","addr":"172.21.15.136:6827","nonce":1960689685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":1960689685},{"type":"v1","addr":"172.21.15.136:6831","nonce":1960689685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":1960689685},{"type":"v1","addr":"172.21.15.136:6829","nonce":1960689685}]},"public_addr":"172.21.15.136:6825/1960689685","cluster_addr":"172.21.15.136:6827/1960689685","heartbeat_back_addr":"172.21.15.136:6831/1960689685","heartbeat_front_addr":"172.21.15.136:6829/1960689685","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:10:56.656177+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:11:22.234031+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:11:46.464102+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:12:12.353967+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:12:37.836200+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:13:01.728839+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:13:27.527230+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:13:51.981393+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:14:17.035464+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:14:42.805029+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:15:08.208999+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:15:32.800844+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.26:6801/677148151":"2024-09-12T23:09:33.983310+0000","172.21.15.26:6800/677148151":"2024-09-12T23:09:33.983310+0000","172.21.15.26:0/3597996003":"2024-09-12T23:09:33.983310+0000","172.21.15.26:6801/2673575521":"2024-09-12T23:09:09.781665+0000","172.21.15.26:0/883212708":"2024-09-12T23:09:33.983310+0000","172.21.15.26:0/2107281413":"2024-09-12T23:09:33.983310+0000","172.21.15.26:6800/2673575521":"2024-09-12T23:09:09.781665+0000","172.21.15.26:0/2817092653":"2024-09-12T23:09:09.781665+0000","172.21.15.26:0/17142658":"2024-09-12T23:09:09.781665+0000","172.21.15.26:0/3861103130":"2024-09-12T23:09:09.781665+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T23:16:28.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:28.515+0000 7fcdd0eca640 1 -- 172.21.15.26:0/197723433 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fcdb40a2b80 msgr2=0x7fcdb40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:16:28.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:28.515+0000 7fcdd0eca640 1 --2- 172.21.15.26:0/197723433 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fcdb40a2b80 0x7fcdb40a5040 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fcdbc009930 tx=0x7fcdbc008040 comp rx=0 tx=0).stop 2024-09-11T23:16:28.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:28.515+0000 7fcdd0eca640 1 -- 172.21.15.26:0/197723433 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcdcc106f10 msgr2=0x7fcdcc19cec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:16:28.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:28.515+0000 7fcdd0eca640 1 --2- 172.21.15.26:0/197723433 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcdcc106f10 0x7fcdcc19cec0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fcdc4031a60 tx=0x7fcdc4031a90 comp rx=0 tx=0).stop 2024-09-11T23:16:28.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:28.515+0000 7fcdd0eca640 1 -- 172.21.15.26:0/197723433 shutdown_connections 2024-09-11T23:16:28.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:28.515+0000 7fcdd0eca640 1 --2- 172.21.15.26:0/197723433 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fcdb40a2b80 0x7fcdb40a5040 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:28.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:28.515+0000 7fcdd0eca640 1 --2- 172.21.15.26:0/197723433 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcdcc106f10 0x7fcdcc19cec0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:28.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:28.515+0000 7fcdd0eca640 1 --2- 172.21.15.26:0/197723433 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcdcc102790 0x7fcdcc198b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:28.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:28.515+0000 7fcdd0eca640 1 --2- 172.21.15.26:0/197723433 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcdcc101de0 0x7fcdcc1985f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:28.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:28.515+0000 7fcdd0eca640 1 -- 172.21.15.26:0/197723433 >> 172.21.15.26:0/197723433 conn(0x7fcdcc0fd8c0 msgr2=0x7fcdcc0fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:16:28.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:28.515+0000 7fcdd0eca640 1 -- 172.21.15.26:0/197723433 shutdown_connections 2024-09-11T23:16:28.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:28.515+0000 7fcdd0eca640 1 -- 172.21.15.26:0/197723433 wait complete. 2024-09-11T23:16:29.174 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:28 smithi026 ceph-mon[31172]: mds.? [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] up:active 2024-09-11T23:16:29.174 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:28 smithi026 ceph-mon[31172]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-11T23:16:29.175 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:28 smithi026 ceph-mon[31172]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-11T23:16:29.175 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:28 smithi026 ceph-mon[31172]: from='client.? 172.21.15.26:0/197723433' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T23:16:29.204 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-11T23:16:29.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:28 smithi136 ceph-mon[35121]: mds.? [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] up:active 2024-09-11T23:16:29.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:28 smithi136 ceph-mon[35121]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-11T23:16:29.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:28 smithi136 ceph-mon[35121]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-11T23:16:29.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:28 smithi136 ceph-mon[35121]: from='client.? 172.21.15.26:0/197723433' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T23:16:29.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:28 smithi076 ceph-mon[35296]: mds.? [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] up:active 2024-09-11T23:16:29.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:28 smithi076 ceph-mon[35296]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-11T23:16:29.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:28 smithi076 ceph-mon[35296]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-11T23:16:29.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:28 smithi076 ceph-mon[35296]: from='client.? 172.21.15.26:0/197723433' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T23:16:29.462 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:16:29.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:29.927+0000 7fb158bfd640 1 -- 172.21.15.26:0/521911571 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb154102d30 msgr2=0x7fb154103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:16:29.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:29.927+0000 7fb158bfd640 1 --2- 172.21.15.26:0/521911571 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb154102d30 0x7fb154103110 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fb144009cd0 tx=0x7fb14402a910 comp rx=0 tx=0).stop 2024-09-11T23:16:29.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:29.927+0000 7fb158bfd640 1 -- 172.21.15.26:0/521911571 shutdown_connections 2024-09-11T23:16:29.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:29.927+0000 7fb158bfd640 1 --2- 172.21.15.26:0/521911571 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb1541082a0 0x7fb15410ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:29.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:29.927+0000 7fb158bfd640 1 --2- 172.21.15.26:0/521911571 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb1541036e0 0x7fb154107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:29.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:29.927+0000 7fb158bfd640 1 --2- 172.21.15.26:0/521911571 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb154102d30 0x7fb154103110 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:29.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:29.927+0000 7fb158bfd640 1 -- 172.21.15.26:0/521911571 >> 172.21.15.26:0/521911571 conn(0x7fb1540fc7d0 msgr2=0x7fb1540febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:16:29.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:29.927+0000 7fb158bfd640 1 -- 172.21.15.26:0/521911571 shutdown_connections 2024-09-11T23:16:29.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:29.927+0000 7fb158bfd640 1 -- 172.21.15.26:0/521911571 wait complete. 2024-09-11T23:16:29.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:29.928+0000 7fb158bfd640 1 Processor -- start 2024-09-11T23:16:29.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:29.928+0000 7fb158bfd640 1 -- start start 2024-09-11T23:16:29.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:29.928+0000 7fb158bfd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb154102d30 0x7fb1541985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:16:29.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:29.928+0000 7fb158bfd640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb1541036e0 0x7fb154198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:16:29.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:29.928+0000 7fb158bfd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb1541082a0 0x7fb15419ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:16:29.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:29.928+0000 7fb158bfd640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb1541120e0 con 0x7fb154102d30 2024-09-11T23:16:29.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:29.928+0000 7fb158bfd640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fb154111f60 con 0x7fb1541082a0 2024-09-11T23:16:29.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:29.928+0000 7fb158bfd640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb154112260 con 0x7fb1541036e0 2024-09-11T23:16:29.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:29.928+0000 7fb15259c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb154102d30 0x7fb1541985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:16:29.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:29.928+0000 7fb15259c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb154102d30 0x7fb1541985b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37776/0 (socket says 172.21.15.26:37776) 2024-09-11T23:16:29.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:29.928+0000 7fb15259c640 1 -- 172.21.15.26:0/15388100 learned_addr learned my addr 172.21.15.26:0/15388100 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:16:29.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:29.928+0000 7fb152d9d640 1 --2- 172.21.15.26:0/15388100 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb1541082a0 0x7fb15419ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:16:29.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:29.928+0000 7fb151d9b640 1 --2- 172.21.15.26:0/15388100 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb1541036e0 0x7fb154198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:16:29.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:29.928+0000 7fb15259c640 1 -- 172.21.15.26:0/15388100 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb1541036e0 msgr2=0x7fb154198af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:16:29.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:29.928+0000 7fb15259c640 1 --2- 172.21.15.26:0/15388100 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb1541036e0 0x7fb154198af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:29.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:29.928+0000 7fb15259c640 1 -- 172.21.15.26:0/15388100 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb1541082a0 msgr2=0x7fb15419ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:16:29.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:29.928+0000 7fb15259c640 1 --2- 172.21.15.26:0/15388100 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb1541082a0 0x7fb15419ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:29.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:29.928+0000 7fb15259c640 1 -- 172.21.15.26:0/15388100 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb15419d560 con 0x7fb154102d30 2024-09-11T23:16:29.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:29.928+0000 7fb152d9d640 1 --2- 172.21.15.26:0/15388100 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb1541082a0 0x7fb15419ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:16:29.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:29.929+0000 7fb15259c640 1 --2- 172.21.15.26:0/15388100 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb154102d30 0x7fb1541985b0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fb1440098d0 tx=0x7fb144031460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:16:29.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:29.929+0000 7fb14b7fe640 1 -- 172.21.15.26:0/15388100 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb144031850 con 0x7fb154102d30 2024-09-11T23:16:29.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:29.929+0000 7fb158bfd640 1 -- 172.21.15.26:0/15388100 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb15419d7f0 con 0x7fb154102d30 2024-09-11T23:16:29.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:29.929+0000 7fb14b7fe640 1 -- 172.21.15.26:0/15388100 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb144004520 con 0x7fb154102d30 2024-09-11T23:16:29.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:29.929+0000 7fb14b7fe640 1 -- 172.21.15.26:0/15388100 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb14402fe20 con 0x7fb154102d30 2024-09-11T23:16:29.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:29.929+0000 7fb158bfd640 1 -- 172.21.15.26:0/15388100 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1541a5130 con 0x7fb154102d30 2024-09-11T23:16:29.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:29.930+0000 7fb158bfd640 1 -- 172.21.15.26:0/15388100 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb154103110 con 0x7fb154102d30 2024-09-11T23:16:29.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:29.931+0000 7fb14b7fe640 1 -- 172.21.15.26:0/15388100 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fb144004040 con 0x7fb154102d30 2024-09-11T23:16:29.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:29.932+0000 7fb14b7fe640 1 --2- 172.21.15.26:0/15388100 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fb1400a2b80 0x7fb1400a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:16:29.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:29.932+0000 7fb151d9b640 1 --2- 172.21.15.26:0/15388100 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fb1400a2b80 0x7fb1400a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:16:29.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:29.932+0000 7fb14b7fe640 1 -- 172.21.15.26:0/15388100 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7979+0+0 (secure 0 0 0) 0x7fb1440f0760 con 0x7fb154102d30 2024-09-11T23:16:29.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:29.932+0000 7fb151d9b640 1 --2- 172.21.15.26:0/15388100 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fb1400a2b80 0x7fb1400a5040 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fb1540fedf0 tx=0x7fb13c008040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:16:29.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:29.935+0000 7fb14b7fe640 1 -- 172.21.15.26:0/15388100 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb14404d3a0 con 0x7fb154102d30 2024-09-11T23:16:30.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:30.111+0000 7fb158bfd640 1 -- 172.21.15.26:0/15388100 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb15410a2b0 con 0x7fb154102d30 2024-09-11T23:16:30.111 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:29 smithi026 ceph-mon[31172]: pgmap v280: 97 pgs: 27 active+clean, 3 creating+activating, 24 creating+peering, 43 unknown; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:16:30.112 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:29 smithi026 ceph-mon[31172]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-11T23:16:30.112 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:29 smithi026 ceph-mon[31172]: Cluster is now healthy 2024-09-11T23:16:30.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:30.112+0000 7fb14b7fe640 1 -- 172.21.15.26:0/15388100 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+22252 (secure 0 0 0) 0x7fb1440bd810 con 0x7fb154102d30 2024-09-11T23:16:30.113 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-11T23:16:30.113 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":83,"fsid":"af7e6fc8-7092-11ef-bcea-c7b262605968","created":"2024-09-11T23:08:40.486438+0000","modified":"2024-09-11T23:16:26.544002+0000","last_up_change":"2024-09-11T23:15:35.179446+0000","last_in_change":"2024-09-11T23:15:11.064627+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T23:11:50.494160+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-11T23:16:22.503928+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-11T23:16:24.876554+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a4d62fcf-c31e-41b1-932c-3d87ff640531","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":2014857905},{"type":"v1","addr":"172.21.15.26:6803","nonce":2014857905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":2014857905},{"type":"v1","addr":"172.21.15.26:6805","nonce":2014857905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":2014857905},{"type":"v1","addr":"172.21.15.26:6809","nonce":2014857905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":2014857905},{"type":"v1","addr":"172.21.15.26:6807","nonce":2014857905}]},"public_addr":"172.21.15.26:6803/2014857905","cluster_addr":"172.21.15.26:6805/2014857905","heartbeat_back_addr":"172.21.15.26:6809/2014857905","heartbeat_front_addr":"172.21.15.26:6807/2014857905","state":["exists","up"]},{"osd":1,"uuid":"e0d69112-3c0a-4fc5-8783-e41b4081e0c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":333383087},{"type":"v1","addr":"172.21.15.76:6801","nonce":333383087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":333383087},{"type":"v1","addr":"172.21.15.76:6803","nonce":333383087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":333383087},{"type":"v1","addr":"172.21.15.76:6807","nonce":333383087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":333383087},{"type":"v1","addr":"172.21.15.76:6805","nonce":333383087}]},"public_addr":"172.21.15.76:6801/333383087","cluster_addr":"172.21.15.76:6803/333383087","heartbeat_back_addr":"172.21.15.76:6807/333383087","heartbeat_front_addr":"172.21.15.76:6805/333383087","state":["exists","up"]},{"osd":2,"uuid":"d578429e-bf23-4a1e-890f-7172955c0eb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":1271538911},{"type":"v1","addr":"172.21.15.136:6801","nonce":1271538911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":1271538911},{"type":"v1","addr":"172.21.15.136:6803","nonce":1271538911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":1271538911},{"type":"v1","addr":"172.21.15.136:6807","nonce":1271538911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":1271538911},{"type":"v1","addr":"172.21.15.136:6805","nonce":1271538911}]},"public_addr":"172.21.15.136:6801/1271538911","cluster_addr":"172.21.15.136:6803/1271538911","heartbeat_back_addr":"172.21.15.136:6807/1271538911","heartbeat_front_addr":"172.21.15.136:6805/1271538911","state":["exists","up"]},{"osd":3,"uuid":"3e34623f-0ea5-4b10-b8a9-f8756c35d512","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":1740304151},{"type":"v1","addr":"172.21.15.26:6811","nonce":1740304151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":1740304151},{"type":"v1","addr":"172.21.15.26:6813","nonce":1740304151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1740304151},{"type":"v1","addr":"172.21.15.26:6817","nonce":1740304151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":1740304151},{"type":"v1","addr":"172.21.15.26:6815","nonce":1740304151}]},"public_addr":"172.21.15.26:6811/1740304151","cluster_addr":"172.21.15.26:6813/1740304151","heartbeat_back_addr":"172.21.15.26:6817/1740304151","heartbeat_front_addr":"172.21.15.26:6815/1740304151","state":["exists","up"]},{"osd":4,"uuid":"77607929-f9ed-49a9-abb5-48cd88eec4ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":2702262182},{"type":"v1","addr":"172.21.15.76:6809","nonce":2702262182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":2702262182},{"type":"v1","addr":"172.21.15.76:6811","nonce":2702262182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":2702262182},{"type":"v1","addr":"172.21.15.76:6815","nonce":2702262182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":2702262182},{"type":"v1","addr":"172.21.15.76:6813","nonce":2702262182}]},"public_addr":"172.21.15.76:6809/2702262182","cluster_addr":"172.21.15.76:6811/2702262182","heartbeat_back_addr":"172.21.15.76:6815/2702262182","heartbeat_front_addr":"172.21.15.76:6813/2702262182","state":["exists","up"]},{"osd":5,"uuid":"29e20e37-09a0-4924-8aaa-17939763a7c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":1749229277},{"type":"v1","addr":"172.21.15.136:6809","nonce":1749229277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":1749229277},{"type":"v1","addr":"172.21.15.136:6811","nonce":1749229277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":1749229277},{"type":"v1","addr":"172.21.15.136:6815","nonce":1749229277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":1749229277},{"type":"v1","addr":"172.21.15.136:6813","nonce":1749229277}]},"public_addr":"172.21.15.136:6809/1749229277","cluster_addr":"172.21.15.136:6811/1749229277","heartbeat_back_addr":"172.21.15.136:6815/1749229277","heartbeat_front_addr":"172.21.15.136:6813/1749229277","state":["exists","up"]},{"osd":6,"uuid":"e0ddd1c0-1c46-41e6-b2f4-396bdcd2bf8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1928245624},{"type":"v1","addr":"172.21.15.26:6819","nonce":1928245624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1928245624},{"type":"v1","addr":"172.21.15.26:6821","nonce":1928245624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":1928245624},{"type":"v1","addr":"172.21.15.26:6825","nonce":1928245624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1928245624},{"type":"v1","addr":"172.21.15.26:6823","nonce":1928245624}]},"public_addr":"172.21.15.26:6819/1928245624","cluster_addr":"172.21.15.26:6821/1928245624","heartbeat_back_addr":"172.21.15.26:6825/1928245624","heartbeat_front_addr":"172.21.15.26:6823/1928245624","state":["exists","up"]},{"osd":7,"uuid":"fe031f5c-d915-48fb-91a6-c76f252467fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":1409789455},{"type":"v1","addr":"172.21.15.76:6817","nonce":1409789455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":1409789455},{"type":"v1","addr":"172.21.15.76:6819","nonce":1409789455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":1409789455},{"type":"v1","addr":"172.21.15.76:6823","nonce":1409789455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":1409789455},{"type":"v1","addr":"172.21.15.76:6821","nonce":1409789455}]},"public_addr":"172.21.15.76:6817/1409789455","cluster_addr":"172.21.15.76:6819/1409789455","heartbeat_back_addr":"172.21.15.76:6823/1409789455","heartbeat_front_addr":"172.21.15.76:6821/1409789455","state":["exists","up"]},{"osd":8,"uuid":"18851bde-9e6c-4e05-b4bf-ac5c3073eee7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":543619079},{"type":"v1","addr":"172.21.15.136:6817","nonce":543619079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":543619079},{"type":"v1","addr":"172.21.15.136:6819","nonce":543619079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":543619079},{"type":"v1","addr":"172.21.15.136:6823","nonce":543619079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":543619079},{"type":"v1","addr":"172.21.15.136:6821","nonce":543619079}]},"public_addr":"172.21.15.136:6817/543619079","cluster_addr":"172.21.15.136:6819/543619079","heartbeat_back_addr":"172.21.15.136:6823/543619079","heartbeat_front_addr":"172.21.15.136:6821/543619079","state":["exists","up"]},{"osd":9,"uuid":"52a65eab-8446-47ad-bfe7-5133f7e3eb98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":2201297064},{"type":"v1","addr":"172.21.15.26:6827","nonce":2201297064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":2201297064},{"type":"v1","addr":"172.21.15.26:6829","nonce":2201297064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":2201297064},{"type":"v1","addr":"172.21.15.26:6833","nonce":2201297064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":2201297064},{"type":"v1","addr":"172.21.15.26:6831","nonce":2201297064}]},"public_addr":"172.21.15.26:6827/2201297064","cluster_addr":"172.21.15.26:6829/2201297064","heartbeat_back_addr":"172.21.15.26:6833/2201297064","heartbeat_front_addr":"172.21.15.26:6831/2201297064","state":["exists","up"]},{"osd":10,"uuid":"b758d4bb-da62-4c5f-a992-bb6a145d35d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":3184128096},{"type":"v1","addr":"172.21.15.76:6825","nonce":3184128096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":3184128096},{"type":"v1","addr":"172.21.15.76:6827","nonce":3184128096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":3184128096},{"type":"v1","addr":"172.21.15.76:6831","nonce":3184128096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":3184128096},{"type":"v1","addr":"172.21.15.76:6829","nonce":3184128096}]},"public_addr":"172.21.15.76:6825/3184128096","cluster_addr":"172.21.15.76:6827/3184128096","heartbeat_back_addr":"172.21.15.76:6831/3184128096","heartbeat_front_addr":"172.21.15.76:6829/3184128096","state":["exists","up"]},{"osd":11,"uuid":"8fddda46-e22b-4d0c-ac48-e038fb29dfec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":1960689685},{"type":"v1","addr":"172.21.15.136:6825","nonce":1960689685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":1960689685},{"type":"v1","addr":"172.21.15.136:6827","nonce":1960689685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":1960689685},{"type":"v1","addr":"172.21.15.136:6831","nonce":1960689685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":1960689685},{"type":"v1","addr":"172.21.15.136:6829","nonce":1960689685}]},"public_addr":"172.21.15.136:6825/1960689685","cluster_addr":"172.21.15.136:6827/1960689685","heartbeat_back_addr":"172.21.15.136:6831/1960689685","heartbeat_front_addr":"172.21.15.136:6829/1960689685","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:10:56.656177+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:11:22.234031+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:11:46.464102+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:12:12.353967+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:12:37.836200+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:13:01.728839+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:13:27.527230+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:13:51.981393+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:14:17.035464+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:14:42.805029+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:15:08.208999+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:15:32.800844+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.26:6801/677148151":"2024-09-12T23:09:33.983310+0000","172.21.15.26:6800/677148151":"2024-09-12T23:09:33.983310+0000","172.21.15.26:0/3597996003":"2024-09-12T23:09:33.983310+0000","172.21.15.26:6801/2673575521":"2024-09-12T23:09:09.781665+0000","172.21.15.26:0/883212708":"2024-09-12T23:09:33.983310+0000","172.21.15.26:0/2107281413":"2024-09-12T23:09:33.983310+0000","172.21.15.26:6800/2673575521":"2024-09-12T23:09:09.781665+0000","172.21.15.26:0/2817092653":"2024-09-12T23:09:09.781665+0000","172.21.15.26:0/17142658":"2024-09-12T23:09:09.781665+0000","172.21.15.26:0/3861103130":"2024-09-12T23:09:09.781665+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T23:16:30.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:30.115+0000 7fb158bfd640 1 -- 172.21.15.26:0/15388100 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fb1400a2b80 msgr2=0x7fb1400a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:16:30.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:30.115+0000 7fb158bfd640 1 --2- 172.21.15.26:0/15388100 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fb1400a2b80 0x7fb1400a5040 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fb1540fedf0 tx=0x7fb13c008040 comp rx=0 tx=0).stop 2024-09-11T23:16:30.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:30.115+0000 7fb158bfd640 1 -- 172.21.15.26:0/15388100 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb154102d30 msgr2=0x7fb1541985b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:16:30.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:30.115+0000 7fb158bfd640 1 --2- 172.21.15.26:0/15388100 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb154102d30 0x7fb1541985b0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fb1440098d0 tx=0x7fb144031460 comp rx=0 tx=0).stop 2024-09-11T23:16:30.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:30.115+0000 7fb158bfd640 1 -- 172.21.15.26:0/15388100 shutdown_connections 2024-09-11T23:16:30.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:30.115+0000 7fb158bfd640 1 --2- 172.21.15.26:0/15388100 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fb1400a2b80 0x7fb1400a5040 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:30.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:30.115+0000 7fb158bfd640 1 --2- 172.21.15.26:0/15388100 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb1541082a0 0x7fb15419ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:30.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:30.115+0000 7fb158bfd640 1 --2- 172.21.15.26:0/15388100 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb1541036e0 0x7fb154198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:30.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:30.115+0000 7fb158bfd640 1 --2- 172.21.15.26:0/15388100 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb154102d30 0x7fb1541985b0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:30.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:30.115+0000 7fb158bfd640 1 -- 172.21.15.26:0/15388100 >> 172.21.15.26:0/15388100 conn(0x7fb1540fc7d0 msgr2=0x7fb154105d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:16:30.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:30.115+0000 7fb158bfd640 1 -- 172.21.15.26:0/15388100 shutdown_connections 2024-09-11T23:16:30.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:30.115+0000 7fb158bfd640 1 -- 172.21.15.26:0/15388100 wait complete. 2024-09-11T23:16:30.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:29 smithi136 ceph-mon[35121]: pgmap v280: 97 pgs: 27 active+clean, 3 creating+activating, 24 creating+peering, 43 unknown; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:16:30.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:29 smithi136 ceph-mon[35121]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-11T23:16:30.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:29 smithi136 ceph-mon[35121]: Cluster is now healthy 2024-09-11T23:16:30.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:29 smithi076 ceph-mon[35296]: pgmap v280: 97 pgs: 27 active+clean, 3 creating+activating, 24 creating+peering, 43 unknown; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:16:30.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:29 smithi076 ceph-mon[35296]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-11T23:16:30.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:29 smithi076 ceph-mon[35296]: Cluster is now healthy 2024-09-11T23:16:30.838 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph fs set cephfs standby_count_wanted 0 2024-09-11T23:16:31.103 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:16:31.224 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:30 smithi026 ceph-mon[31172]: from='client.? 172.21.15.26:0/15388100' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T23:16:31.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:30 smithi136 ceph-mon[35121]: from='client.? 172.21.15.26:0/15388100' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T23:16:31.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:30 smithi076 ceph-mon[35296]: from='client.? 172.21.15.26:0/15388100' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T23:16:31.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:31.543+0000 7fe4c953b640 1 -- 172.21.15.26:0/2559191354 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe4c4101de0 msgr2=0x7fe4c41021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:16:31.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:31.543+0000 7fe4c953b640 1 --2- 172.21.15.26:0/2559191354 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe4c4101de0 0x7fe4c41021c0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fe4b4009900 tx=0x7fe4b402a410 comp rx=0 tx=0).stop 2024-09-11T23:16:31.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:31.544+0000 7fe4c953b640 1 -- 172.21.15.26:0/2559191354 shutdown_connections 2024-09-11T23:16:31.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:31.544+0000 7fe4c953b640 1 --2- 172.21.15.26:0/2559191354 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe4c4102790 0x7fe4c41067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:31.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:31.544+0000 7fe4c953b640 1 --2- 172.21.15.26:0/2559191354 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe4c4106f10 0x7fe4c410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:31.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:31.544+0000 7fe4c953b640 1 --2- 172.21.15.26:0/2559191354 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe4c4101de0 0x7fe4c41021c0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:31.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:31.544+0000 7fe4c953b640 1 -- 172.21.15.26:0/2559191354 >> 172.21.15.26:0/2559191354 conn(0x7fe4c40fd8c0 msgr2=0x7fe4c40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:16:31.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:31.544+0000 7fe4c953b640 1 -- 172.21.15.26:0/2559191354 shutdown_connections 2024-09-11T23:16:31.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:31.544+0000 7fe4c953b640 1 -- 172.21.15.26:0/2559191354 wait complete. 2024-09-11T23:16:31.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:31.544+0000 7fe4c953b640 1 Processor -- start 2024-09-11T23:16:31.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:31.544+0000 7fe4c953b640 1 -- start start 2024-09-11T23:16:31.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:31.545+0000 7fe4c953b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe4c4101de0 0x7fe4c41984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:16:31.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:31.545+0000 7fe4c953b640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe4c4102790 0x7fe4c4198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:16:31.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:31.545+0000 7fe4c953b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe4c4106f10 0x7fe4c419cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:16:31.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:31.545+0000 7fe4c953b640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fe4c4111f70 con 0x7fe4c4101de0 2024-09-11T23:16:31.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:31.545+0000 7fe4c953b640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fe4c4111df0 con 0x7fe4c4106f10 2024-09-11T23:16:31.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:31.545+0000 7fe4c953b640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe4c41120f0 con 0x7fe4c4102790 2024-09-11T23:16:31.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:31.545+0000 7fe4c2ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe4c4101de0 0x7fe4c41984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:16:31.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:31.545+0000 7fe4c2ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe4c4101de0 0x7fe4c41984d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60976/0 (socket says 172.21.15.26:60976) 2024-09-11T23:16:31.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:31.545+0000 7fe4c2ffd640 1 -- 172.21.15.26:0/1139561065 learned_addr learned my addr 172.21.15.26:0/1139561065 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:16:31.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:31.545+0000 7fe4c27fc640 1 --2- 172.21.15.26:0/1139561065 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe4c4102790 0x7fe4c4198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:16:31.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:31.545+0000 7fe4c37fe640 1 --2- 172.21.15.26:0/1139561065 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe4c4106f10 0x7fe4c419cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:16:31.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:31.545+0000 7fe4c2ffd640 1 -- 172.21.15.26:0/1139561065 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe4c4102790 msgr2=0x7fe4c4198a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:16:31.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:31.545+0000 7fe4c2ffd640 1 --2- 172.21.15.26:0/1139561065 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe4c4102790 0x7fe4c4198a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:31.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:31.545+0000 7fe4c2ffd640 1 -- 172.21.15.26:0/1139561065 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe4c4106f10 msgr2=0x7fe4c419cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:16:31.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:31.545+0000 7fe4c2ffd640 1 --2- 172.21.15.26:0/1139561065 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe4c4106f10 0x7fe4c419cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:31.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:31.545+0000 7fe4c2ffd640 1 -- 172.21.15.26:0/1139561065 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4c419d480 con 0x7fe4c4101de0 2024-09-11T23:16:31.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:31.545+0000 7fe4c37fe640 1 --2- 172.21.15.26:0/1139561065 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe4c4106f10 0x7fe4c419cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:16:31.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:31.545+0000 7fe4c2ffd640 1 --2- 172.21.15.26:0/1139561065 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe4c4101de0 0x7fe4c41984d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fe4b4006010 tx=0x7fe4b40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:16:31.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:31.545+0000 7fe4abfff640 1 -- 172.21.15.26:0/1139561065 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4b403f030 con 0x7fe4c4101de0 2024-09-11T23:16:31.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:31.546+0000 7fe4c953b640 1 -- 172.21.15.26:0/1139561065 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4c419d710 con 0x7fe4c4101de0 2024-09-11T23:16:31.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:31.546+0000 7fe4abfff640 1 -- 172.21.15.26:0/1139561065 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4b4003ce0 con 0x7fe4c4101de0 2024-09-11T23:16:31.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:31.546+0000 7fe4abfff640 1 -- 172.21.15.26:0/1139561065 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4b4035070 con 0x7fe4c4101de0 2024-09-11T23:16:31.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:31.546+0000 7fe4c953b640 1 -- 172.21.15.26:0/1139561065 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4c41a5050 con 0x7fe4c4101de0 2024-09-11T23:16:31.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:31.547+0000 7fe4c953b640 1 -- 172.21.15.26:0/1139561065 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4c41032c0 con 0x7fe4c4101de0 2024-09-11T23:16:31.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:31.548+0000 7fe4abfff640 1 -- 172.21.15.26:0/1139561065 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fe4b4042030 con 0x7fe4c4101de0 2024-09-11T23:16:31.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:31.548+0000 7fe4abfff640 1 --2- 172.21.15.26:0/1139561065 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fe4ac0a2b80 0x7fe4ac0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:16:31.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:31.548+0000 7fe4abfff640 1 -- 172.21.15.26:0/1139561065 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7979+0+0 (secure 0 0 0) 0x7fe4b40ef480 con 0x7fe4c4101de0 2024-09-11T23:16:31.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:31.549+0000 7fe4c27fc640 1 --2- 172.21.15.26:0/1139561065 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fe4ac0a2b80 0x7fe4ac0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:16:31.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:31.549+0000 7fe4c27fc640 1 --2- 172.21.15.26:0/1139561065 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fe4ac0a2b80 0x7fe4ac0a5040 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fe4c40ff840 tx=0x7fe4b800b410 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:16:31.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:31.553+0000 7fe4abfff640 1 -- 172.21.15.26:0/1139561065 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe4b40bc5f0 con 0x7fe4c4101de0 2024-09-11T23:16:31.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:31.764+0000 7fe4c953b640 1 -- 172.21.15.26:0/1139561065 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fe4c410a0f0 con 0x7fe4c4101de0 2024-09-11T23:16:31.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:31.913+0000 7fe4abfff640 1 -- 172.21.15.26:0/1139561065 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) ==== 118+0+0 (secure 0 0 0) 0x7fe4b40bbef0 con 0x7fe4c4101de0 2024-09-11T23:16:31.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:31.915+0000 7fe4c953b640 1 -- 172.21.15.26:0/1139561065 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fe4ac0a2b80 msgr2=0x7fe4ac0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:16:31.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:31.915+0000 7fe4c953b640 1 --2- 172.21.15.26:0/1139561065 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fe4ac0a2b80 0x7fe4ac0a5040 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fe4c40ff840 tx=0x7fe4b800b410 comp rx=0 tx=0).stop 2024-09-11T23:16:31.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:31.915+0000 7fe4c953b640 1 -- 172.21.15.26:0/1139561065 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe4c4101de0 msgr2=0x7fe4c41984d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:16:31.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:31.915+0000 7fe4c953b640 1 --2- 172.21.15.26:0/1139561065 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe4c4101de0 0x7fe4c41984d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fe4b4006010 tx=0x7fe4b40042f0 comp rx=0 tx=0).stop 2024-09-11T23:16:31.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:31.915+0000 7fe4c953b640 1 -- 172.21.15.26:0/1139561065 shutdown_connections 2024-09-11T23:16:31.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:31.915+0000 7fe4c953b640 1 --2- 172.21.15.26:0/1139561065 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fe4ac0a2b80 0x7fe4ac0a5040 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:31.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:31.915+0000 7fe4c953b640 1 --2- 172.21.15.26:0/1139561065 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe4c4106f10 0x7fe4c419cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:31.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:31.915+0000 7fe4c953b640 1 --2- 172.21.15.26:0/1139561065 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe4c4102790 0x7fe4c4198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:31.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:31.915+0000 7fe4c953b640 1 --2- 172.21.15.26:0/1139561065 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe4c4101de0 0x7fe4c41984d0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:31.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:31.915+0000 7fe4c953b640 1 -- 172.21.15.26:0/1139561065 >> 172.21.15.26:0/1139561065 conn(0x7fe4c40fd8c0 msgr2=0x7fe4c40fecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:16:31.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:31.915+0000 7fe4c953b640 1 -- 172.21.15.26:0/1139561065 shutdown_connections 2024-09-11T23:16:31.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:31.915+0000 7fe4c953b640 1 -- 172.21.15.26:0/1139561065 wait complete. 2024-09-11T23:16:32.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:31 smithi136 ceph-mon[35121]: pgmap v281: 97 pgs: 82 active+clean, 15 creating+peering; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-09-11T23:16:32.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:31 smithi136 ceph-mon[35121]: from='client.? 172.21.15.26:0/1139561065' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-11T23:16:32.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:31 smithi026 ceph-mon[31172]: pgmap v281: 97 pgs: 82 active+clean, 15 creating+peering; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-09-11T23:16:32.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:31 smithi026 ceph-mon[31172]: from='client.? 172.21.15.26:0/1139561065' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-11T23:16:32.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:31 smithi076 ceph-mon[35296]: pgmap v281: 97 pgs: 82 active+clean, 15 creating+peering; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-09-11T23:16:32.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:31 smithi076 ceph-mon[35296]: from='client.? 172.21.15.26:0/1139561065' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-11T23:16:32.303 INFO:journalctl@ceph.mds.b.smithi076.stdout:Sep 11 23:16:31 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mds-b[65334]: 2024-09-11T23:16:31.973+0000 7fef227ea640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-11T23:16:32.730 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-09-11T23:16:32.731 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph fs set cephfs max_mds 5 2024-09-11T23:16:32.993 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:16:33.000 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:32 smithi026 ceph-mon[31172]: from='client.? 172.21.15.26:0/1139561065' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-11T23:16:33.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:32 smithi026 ceph-mon[31172]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-11T23:16:33.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:32 smithi136 ceph-mon[35121]: from='client.? 172.21.15.26:0/1139561065' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-11T23:16:33.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:32 smithi136 ceph-mon[35121]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-11T23:16:33.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:32 smithi076 ceph-mon[35296]: from='client.? 172.21.15.26:0/1139561065' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-11T23:16:33.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:32 smithi076 ceph-mon[35296]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-11T23:16:33.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:33.426+0000 7f63101fb640 1 -- 172.21.15.26:0/3052620501 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6308101420 msgr2=0x7f630810d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:16:33.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:33.426+0000 7f63101fb640 1 --2- 172.21.15.26:0/3052620501 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6308101420 0x7f630810d950 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f6300009cd0 tx=0x7f630002a8e0 comp rx=0 tx=0).stop 2024-09-11T23:16:33.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:33.426+0000 7f63101fb640 1 -- 172.21.15.26:0/3052620501 shutdown_connections 2024-09-11T23:16:33.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:33.426+0000 7f63101fb640 1 --2- 172.21.15.26:0/3052620501 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6308101420 0x7f630810d950 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:33.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:33.426+0000 7f63101fb640 1 --2- 172.21.15.26:0/3052620501 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6308100a80 0x7f6308100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:33.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:33.426+0000 7f63101fb640 1 --2- 172.21.15.26:0/3052620501 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6308106a80 0x7f6308106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:33.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:33.426+0000 7f63101fb640 1 -- 172.21.15.26:0/3052620501 >> 172.21.15.26:0/3052620501 conn(0x7f63080fc7a0 msgr2=0x7f63080febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:16:33.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:33.426+0000 7f63101fb640 1 -- 172.21.15.26:0/3052620501 shutdown_connections 2024-09-11T23:16:33.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:33.426+0000 7f63101fb640 1 -- 172.21.15.26:0/3052620501 wait complete. 2024-09-11T23:16:33.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:33.427+0000 7f63101fb640 1 Processor -- start 2024-09-11T23:16:33.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:33.427+0000 7f63101fb640 1 -- start start 2024-09-11T23:16:33.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:33.428+0000 7f63101fb640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6308100a80 0x7f630819a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:16:33.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:33.428+0000 7f63101fb640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6308101420 0x7f630819abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:16:33.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:33.428+0000 7f63101fb640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6308106a80 0x7f630819ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:16:33.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:33.428+0000 7f63101fb640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f6308110e80 con 0x7f6308106a80 2024-09-11T23:16:33.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:33.428+0000 7f63101fb640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f6308110d00 con 0x7f6308100a80 2024-09-11T23:16:33.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:33.428+0000 7f63101fb640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f6308111000 con 0x7f6308101420 2024-09-11T23:16:33.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:33.428+0000 7f630e798640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6308106a80 0x7f630819ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:16:33.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:33.428+0000 7f630e798640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6308106a80 0x7f630819ef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60990/0 (socket says 172.21.15.26:60990) 2024-09-11T23:16:33.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:33.428+0000 7f630e798640 1 -- 172.21.15.26:0/4145615794 learned_addr learned my addr 172.21.15.26:0/4145615794 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:16:33.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:33.428+0000 7f630d796640 1 --2- 172.21.15.26:0/4145615794 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6308101420 0x7f630819abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:16:33.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:33.428+0000 7f630df97640 1 --2- 172.21.15.26:0/4145615794 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6308100a80 0x7f630819a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:16:33.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:33.428+0000 7f630e798640 1 -- 172.21.15.26:0/4145615794 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6308101420 msgr2=0x7f630819abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:16:33.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:33.428+0000 7f630e798640 1 --2- 172.21.15.26:0/4145615794 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6308101420 0x7f630819abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:33.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:33.428+0000 7f630e798640 1 -- 172.21.15.26:0/4145615794 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6308100a80 msgr2=0x7f630819a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:16:33.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:33.428+0000 7f630e798640 1 --2- 172.21.15.26:0/4145615794 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6308100a80 0x7f630819a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:33.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:33.428+0000 7f630e798640 1 -- 172.21.15.26:0/4145615794 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f630819f650 con 0x7f6308106a80 2024-09-11T23:16:33.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:33.428+0000 7f630df97640 1 --2- 172.21.15.26:0/4145615794 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6308100a80 0x7f630819a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:16:33.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:33.428+0000 7f630e798640 1 --2- 172.21.15.26:0/4145615794 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6308106a80 0x7f630819ef70 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f6300005cc0 tx=0x7f6300003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:16:33.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:33.428+0000 7f62feffd640 1 -- 172.21.15.26:0/4145615794 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6300004040 con 0x7f6308106a80 2024-09-11T23:16:33.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:33.429+0000 7f63101fb640 1 -- 172.21.15.26:0/4145615794 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f630819f8e0 con 0x7f6308106a80 2024-09-11T23:16:33.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:33.429+0000 7f62feffd640 1 -- 172.21.15.26:0/4145615794 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63000046c0 con 0x7f6308106a80 2024-09-11T23:16:33.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:33.429+0000 7f62feffd640 1 -- 172.21.15.26:0/4145615794 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6300046480 con 0x7f6308106a80 2024-09-11T23:16:33.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:33.429+0000 7f63101fb640 1 -- 172.21.15.26:0/4145615794 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63081a7220 con 0x7f6308106a80 2024-09-11T23:16:33.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:33.430+0000 7f63101fb640 1 -- 172.21.15.26:0/4145615794 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6308100ee0 con 0x7f6308106a80 2024-09-11T23:16:33.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:33.431+0000 7f62feffd640 1 -- 172.21.15.26:0/4145615794 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f63000305a0 con 0x7f6308106a80 2024-09-11T23:16:33.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:33.432+0000 7f62feffd640 1 --2- 172.21.15.26:0/4145615794 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f62f40a2b80 0x7f62f40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:16:33.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:33.432+0000 7f630df97640 1 --2- 172.21.15.26:0/4145615794 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f62f40a2b80 0x7f62f40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:16:33.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:33.432+0000 7f62feffd640 1 -- 172.21.15.26:0/4145615794 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7979+0+0 (secure 0 0 0) 0x7f630002e030 con 0x7f6308106a80 2024-09-11T23:16:33.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:33.432+0000 7f630df97640 1 --2- 172.21.15.26:0/4145615794 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f62f40a2b80 0x7f62f40a5040 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f6304005fd0 tx=0x7f63040094d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:16:33.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:33.435+0000 7f62feffd640 1 -- 172.21.15.26:0/4145615794 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f630004b060 con 0x7f6308106a80 2024-09-11T23:16:33.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:33.651+0000 7f63101fb640 1 -- 172.21.15.26:0/4145615794 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) -- 0x7f63081a73f0 con 0x7f6308106a80 2024-09-11T23:16:33.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:33.924+0000 7f62feffd640 1 -- 172.21.15.26:0/4145615794 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v18) ==== 105+0+0 (secure 0 0 0) 0x7f63000bda00 con 0x7f6308106a80 2024-09-11T23:16:33.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:33.927+0000 7f63101fb640 1 -- 172.21.15.26:0/4145615794 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f62f40a2b80 msgr2=0x7f62f40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:16:33.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:33.927+0000 7f63101fb640 1 --2- 172.21.15.26:0/4145615794 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f62f40a2b80 0x7f62f40a5040 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f6304005fd0 tx=0x7f63040094d0 comp rx=0 tx=0).stop 2024-09-11T23:16:33.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:33.927+0000 7f63101fb640 1 -- 172.21.15.26:0/4145615794 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6308106a80 msgr2=0x7f630819ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:16:33.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:33.927+0000 7f63101fb640 1 --2- 172.21.15.26:0/4145615794 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6308106a80 0x7f630819ef70 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f6300005cc0 tx=0x7f6300003970 comp rx=0 tx=0).stop 2024-09-11T23:16:33.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:33.927+0000 7f63101fb640 1 -- 172.21.15.26:0/4145615794 shutdown_connections 2024-09-11T23:16:33.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:33.927+0000 7f63101fb640 1 --2- 172.21.15.26:0/4145615794 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f62f40a2b80 0x7f62f40a5040 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:33.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:33.927+0000 7f63101fb640 1 --2- 172.21.15.26:0/4145615794 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6308106a80 0x7f630819ef70 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:33.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:33.927+0000 7f63101fb640 1 --2- 172.21.15.26:0/4145615794 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6308101420 0x7f630819abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:33.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:33.927+0000 7f63101fb640 1 --2- 172.21.15.26:0/4145615794 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6308100a80 0x7f630819a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:33.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:33.927+0000 7f63101fb640 1 -- 172.21.15.26:0/4145615794 >> 172.21.15.26:0/4145615794 conn(0x7f63080fc7a0 msgr2=0x7f630810c720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:16:33.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:33.927+0000 7f63101fb640 1 -- 172.21.15.26:0/4145615794 shutdown_connections 2024-09-11T23:16:33.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:33.927+0000 7f63101fb640 1 -- 172.21.15.26:0/4145615794 wait complete. 2024-09-11T23:16:33.969 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:33 smithi026 ceph-mon[31172]: pgmap v282: 97 pgs: 97 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s wr, 6 op/s 2024-09-11T23:16:33.969 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:33 smithi026 ceph-mon[31172]: from='client.? 172.21.15.26:0/4145615794' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-09-11T23:16:34.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:33 smithi136 ceph-mon[35121]: pgmap v282: 97 pgs: 97 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s wr, 6 op/s 2024-09-11T23:16:34.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:33 smithi136 ceph-mon[35121]: from='client.? 172.21.15.26:0/4145615794' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-09-11T23:16:34.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:33 smithi076 ceph-mon[35296]: pgmap v282: 97 pgs: 97 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s wr, 6 op/s 2024-09-11T23:16:34.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:33 smithi076 ceph-mon[35296]: from='client.? 172.21.15.26:0/4145615794' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-09-11T23:16:34.599 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph fs set cephfs allow_standby_replay true 2024-09-11T23:16:34.871 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:16:35.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:34 smithi136 ceph-mon[35121]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-11T23:16:35.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:34 smithi136 ceph-mon[35121]: from='client.? 172.21.15.26:0/4145615794' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-09-11T23:16:35.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:34 smithi136 ceph-mon[35121]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-11T23:16:35.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:34 smithi136 ceph-mon[35121]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-11T23:16:35.240 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:34 smithi136 ceph-mon[35121]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-09-11T23:16:35.240 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:34 smithi136 ceph-mon[35121]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-09-11T23:16:35.240 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:34 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]: dispatch 2024-09-11T23:16:35.240 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:34 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-09-11T23:16:35.240 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:34 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]: dispatch 2024-09-11T23:16:35.240 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:34 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4]}]: dispatch 2024-09-11T23:16:35.240 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:34 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8]}]: dispatch 2024-09-11T23:16:35.240 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:34 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.18", "id": [3, 9]}]: dispatch 2024-09-11T23:16:35.240 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:34 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]: dispatch 2024-09-11T23:16:35.240 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:34 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 6]}]: dispatch 2024-09-11T23:16:35.240 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:34 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2024-09-11T23:16:35.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:34 smithi026 ceph-mon[31172]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-11T23:16:35.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:34 smithi026 ceph-mon[31172]: from='client.? 172.21.15.26:0/4145615794' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-09-11T23:16:35.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:34 smithi026 ceph-mon[31172]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-11T23:16:35.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:34 smithi026 ceph-mon[31172]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-11T23:16:35.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:34 smithi026 ceph-mon[31172]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-09-11T23:16:35.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:34 smithi026 ceph-mon[31172]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-09-11T23:16:35.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:34 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]: dispatch 2024-09-11T23:16:35.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:34 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-09-11T23:16:35.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:34 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]: dispatch 2024-09-11T23:16:35.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:34 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4]}]: dispatch 2024-09-11T23:16:35.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:34 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8]}]: dispatch 2024-09-11T23:16:35.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:34 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.18", "id": [3, 9]}]: dispatch 2024-09-11T23:16:35.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:34 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]: dispatch 2024-09-11T23:16:35.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:34 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 6]}]: dispatch 2024-09-11T23:16:35.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:34 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2024-09-11T23:16:35.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:34 smithi076 ceph-mon[35296]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-11T23:16:35.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:34 smithi076 ceph-mon[35296]: from='client.? 172.21.15.26:0/4145615794' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-09-11T23:16:35.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:34 smithi076 ceph-mon[35296]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-11T23:16:35.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:34 smithi076 ceph-mon[35296]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-11T23:16:35.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:34 smithi076 ceph-mon[35296]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-09-11T23:16:35.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:34 smithi076 ceph-mon[35296]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-09-11T23:16:35.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:34 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]: dispatch 2024-09-11T23:16:35.304 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:34 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-09-11T23:16:35.304 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:34 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]: dispatch 2024-09-11T23:16:35.304 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:34 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4]}]: dispatch 2024-09-11T23:16:35.304 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:34 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8]}]: dispatch 2024-09-11T23:16:35.304 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:34 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.18", "id": [3, 9]}]: dispatch 2024-09-11T23:16:35.304 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:34 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]: dispatch 2024-09-11T23:16:35.304 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:34 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 6]}]: dispatch 2024-09-11T23:16:35.304 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:34 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2024-09-11T23:16:35.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:35.371+0000 7fe28727a640 1 -- 172.21.15.26:0/311989167 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe280106a80 msgr2=0x7fe280106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:16:35.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:35.371+0000 7fe28727a640 1 --2- 172.21.15.26:0/311989167 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe280106a80 0x7fe280106e60 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fe27c009930 tx=0x7fe27c02a420 comp rx=0 tx=0).stop 2024-09-11T23:16:35.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:35.371+0000 7fe28727a640 1 -- 172.21.15.26:0/311989167 shutdown_connections 2024-09-11T23:16:35.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:35.371+0000 7fe28727a640 1 --2- 172.21.15.26:0/311989167 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe280100a80 0x7fe280100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:35.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:35.371+0000 7fe28727a640 1 --2- 172.21.15.26:0/311989167 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe280101420 0x7fe28010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:35.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:35.371+0000 7fe28727a640 1 --2- 172.21.15.26:0/311989167 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe280106a80 0x7fe280106e60 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:35.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:35.371+0000 7fe28727a640 1 -- 172.21.15.26:0/311989167 >> 172.21.15.26:0/311989167 conn(0x7fe2800fc7a0 msgr2=0x7fe2800febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:16:35.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:35.371+0000 7fe28727a640 1 -- 172.21.15.26:0/311989167 shutdown_connections 2024-09-11T23:16:35.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:35.371+0000 7fe28727a640 1 -- 172.21.15.26:0/311989167 wait complete. 2024-09-11T23:16:35.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:35.371+0000 7fe28727a640 1 Processor -- start 2024-09-11T23:16:35.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:35.371+0000 7fe28727a640 1 -- start start 2024-09-11T23:16:35.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:35.372+0000 7fe28727a640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe280100a80 0x7fe280198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:16:35.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:35.372+0000 7fe28727a640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe280101420 0x7fe280198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:16:35.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:35.372+0000 7fe28727a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe280106a80 0x7fe28019cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:16:35.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:35.372+0000 7fe28727a640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fe280110e40 con 0x7fe280106a80 2024-09-11T23:16:35.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:35.372+0000 7fe28727a640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fe280110cc0 con 0x7fe280101420 2024-09-11T23:16:35.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:35.372+0000 7fe28727a640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe280110fc0 con 0x7fe280100a80 2024-09-11T23:16:35.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:35.372+0000 7fe285817640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe280106a80 0x7fe28019cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:16:35.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:35.372+0000 7fe285817640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe280106a80 0x7fe28019cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:32774/0 (socket says 172.21.15.26:32774) 2024-09-11T23:16:35.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:35.372+0000 7fe285817640 1 -- 172.21.15.26:0/3165220220 learned_addr learned my addr 172.21.15.26:0/3165220220 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:16:35.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:35.372+0000 7fe284815640 1 --2- 172.21.15.26:0/3165220220 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe280101420 0x7fe280198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:16:35.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:35.372+0000 7fe285016640 1 --2- 172.21.15.26:0/3165220220 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe280100a80 0x7fe280198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:16:35.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:35.372+0000 7fe285817640 1 -- 172.21.15.26:0/3165220220 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe280100a80 msgr2=0x7fe280198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:16:35.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:35.372+0000 7fe285817640 1 --2- 172.21.15.26:0/3165220220 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe280100a80 0x7fe280198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:35.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:35.372+0000 7fe285817640 1 -- 172.21.15.26:0/3165220220 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe280101420 msgr2=0x7fe280198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:16:35.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:35.372+0000 7fe285817640 1 --2- 172.21.15.26:0/3165220220 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe280101420 0x7fe280198950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:35.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:35.372+0000 7fe285817640 1 -- 172.21.15.26:0/3165220220 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe28019d3c0 con 0x7fe280106a80 2024-09-11T23:16:35.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:35.372+0000 7fe285016640 1 --2- 172.21.15.26:0/3165220220 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe280100a80 0x7fe280198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:16:35.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:35.372+0000 7fe285817640 1 --2- 172.21.15.26:0/3165220220 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe280106a80 0x7fe28019cce0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fe27800cad0 tx=0x7fe27800ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:16:35.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:35.372+0000 7fe2767fc640 1 -- 172.21.15.26:0/3165220220 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe27800ad60 con 0x7fe280106a80 2024-09-11T23:16:35.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:35.372+0000 7fe2767fc640 1 -- 172.21.15.26:0/3165220220 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe278005020 con 0x7fe280106a80 2024-09-11T23:16:35.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:35.373+0000 7fe28727a640 1 -- 172.21.15.26:0/3165220220 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe28019d6b0 con 0x7fe280106a80 2024-09-11T23:16:35.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:35.373+0000 7fe2767fc640 1 -- 172.21.15.26:0/3165220220 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe27800f6b0 con 0x7fe280106a80 2024-09-11T23:16:35.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:35.373+0000 7fe28727a640 1 -- 172.21.15.26:0/3165220220 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2801a4f90 con 0x7fe280106a80 2024-09-11T23:16:35.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:35.374+0000 7fe28727a640 1 -- 172.21.15.26:0/3165220220 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe280100ee0 con 0x7fe280106a80 2024-09-11T23:16:35.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:35.375+0000 7fe2767fc640 1 -- 172.21.15.26:0/3165220220 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fe278005560 con 0x7fe280106a80 2024-09-11T23:16:35.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:35.375+0000 7fe2767fc640 1 --2- 172.21.15.26:0/3165220220 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fe26c0a2b80 0x7fe26c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:16:35.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:35.375+0000 7fe2767fc640 1 -- 172.21.15.26:0/3165220220 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7fe2780cf030 con 0x7fe280106a80 2024-09-11T23:16:35.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:35.375+0000 7fe285016640 1 --2- 172.21.15.26:0/3165220220 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fe26c0a2b80 0x7fe26c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:16:35.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:35.376+0000 7fe285016640 1 --2- 172.21.15.26:0/3165220220 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fe26c0a2b80 0x7fe26c0a5040 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fe27c000c00 tx=0x7fe27c005a20 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:16:35.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:35.380+0000 7fe2767fc640 1 -- 172.21.15.26:0/3165220220 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe278014030 con 0x7fe280106a80 2024-09-11T23:16:35.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:35.592+0000 7fe28727a640 1 -- 172.21.15.26:0/3165220220 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7fe28002d7c0 con 0x7fe280106a80 2024-09-11T23:16:35.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:35.949+0000 7fe2767fc640 1 -- 172.21.15.26:0/3165220220 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v22) ==== 121+0+0 (secure 0 0 0) 0x7fe278084e20 con 0x7fe280106a80 2024-09-11T23:16:35.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:35.951+0000 7fe28727a640 1 -- 172.21.15.26:0/3165220220 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fe26c0a2b80 msgr2=0x7fe26c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:16:35.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:35.951+0000 7fe28727a640 1 --2- 172.21.15.26:0/3165220220 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fe26c0a2b80 0x7fe26c0a5040 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fe27c000c00 tx=0x7fe27c005a20 comp rx=0 tx=0).stop 2024-09-11T23:16:35.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:35.951+0000 7fe28727a640 1 -- 172.21.15.26:0/3165220220 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe280106a80 msgr2=0x7fe28019cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:16:35.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:35.951+0000 7fe28727a640 1 --2- 172.21.15.26:0/3165220220 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe280106a80 0x7fe28019cce0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fe27800cad0 tx=0x7fe27800ced0 comp rx=0 tx=0).stop 2024-09-11T23:16:35.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:35.951+0000 7fe28727a640 1 -- 172.21.15.26:0/3165220220 shutdown_connections 2024-09-11T23:16:35.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:35.951+0000 7fe28727a640 1 --2- 172.21.15.26:0/3165220220 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fe26c0a2b80 0x7fe26c0a5040 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:35.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:35.951+0000 7fe28727a640 1 --2- 172.21.15.26:0/3165220220 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe280101420 0x7fe280198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:35.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:35.951+0000 7fe28727a640 1 --2- 172.21.15.26:0/3165220220 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe280106a80 0x7fe28019cce0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:35.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:35.951+0000 7fe28727a640 1 --2- 172.21.15.26:0/3165220220 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe280100a80 0x7fe280198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:35.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:35.951+0000 7fe28727a640 1 -- 172.21.15.26:0/3165220220 >> 172.21.15.26:0/3165220220 conn(0x7fe2800fc7a0 msgr2=0x7fe28006aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:16:35.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:35.952+0000 7fe28727a640 1 -- 172.21.15.26:0/3165220220 shutdown_connections 2024-09-11T23:16:35.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:35.952+0000 7fe28727a640 1 -- 172.21.15.26:0/3165220220 wait complete. 2024-09-11T23:16:36.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:35 smithi136 ceph-mon[35121]: pgmap v283: 97 pgs: 97 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-09-11T23:16:36.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:35 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]': finished 2024-09-11T23:16:36.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:35 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-09-11T23:16:36.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:35 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]': finished 2024-09-11T23:16:36.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:35 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4]}]': finished 2024-09-11T23:16:36.240 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:35 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8]}]': finished 2024-09-11T23:16:36.240 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:35 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.18", "id": [3, 9]}]': finished 2024-09-11T23:16:36.240 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:35 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2024-09-11T23:16:36.240 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:35 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 6]}]': finished 2024-09-11T23:16:36.240 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:35 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2024-09-11T23:16:36.240 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:35 smithi136 ceph-mon[35121]: osdmap e84: 12 total, 12 up, 12 in 2024-09-11T23:16:36.240 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:35 smithi136 ceph-mon[35121]: mds.? [v2:172.21.15.136:6838/3211398630,v1:172.21.15.136:6839/3211398630] up:active 2024-09-11T23:16:36.240 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:35 smithi136 ceph-mon[35121]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-11T23:16:36.240 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:35 smithi136 ceph-mon[35121]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-09-11T23:16:36.240 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:35 smithi136 ceph-mon[35121]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-09-11T23:16:36.240 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:35 smithi136 ceph-mon[35121]: from='client.? 172.21.15.26:0/3165220220' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-11T23:16:36.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:35 smithi026 ceph-mon[31172]: pgmap v283: 97 pgs: 97 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-09-11T23:16:36.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:35 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]': finished 2024-09-11T23:16:36.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:35 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-09-11T23:16:36.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:35 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]': finished 2024-09-11T23:16:36.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:35 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4]}]': finished 2024-09-11T23:16:36.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:35 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8]}]': finished 2024-09-11T23:16:36.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:35 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.18", "id": [3, 9]}]': finished 2024-09-11T23:16:36.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:35 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2024-09-11T23:16:36.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:35 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 6]}]': finished 2024-09-11T23:16:36.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:35 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2024-09-11T23:16:36.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:35 smithi026 ceph-mon[31172]: osdmap e84: 12 total, 12 up, 12 in 2024-09-11T23:16:36.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:35 smithi026 ceph-mon[31172]: mds.? [v2:172.21.15.136:6838/3211398630,v1:172.21.15.136:6839/3211398630] up:active 2024-09-11T23:16:36.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:35 smithi026 ceph-mon[31172]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-11T23:16:36.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:35 smithi026 ceph-mon[31172]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-09-11T23:16:36.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:35 smithi026 ceph-mon[31172]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-09-11T23:16:36.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:35 smithi026 ceph-mon[31172]: from='client.? 172.21.15.26:0/3165220220' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-11T23:16:36.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:35 smithi076 ceph-mon[35296]: pgmap v283: 97 pgs: 97 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-09-11T23:16:36.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:35 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]': finished 2024-09-11T23:16:36.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:35 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-09-11T23:16:36.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:35 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]': finished 2024-09-11T23:16:36.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:35 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4]}]': finished 2024-09-11T23:16:36.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:35 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8]}]': finished 2024-09-11T23:16:36.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:35 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.18", "id": [3, 9]}]': finished 2024-09-11T23:16:36.304 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:35 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2024-09-11T23:16:36.304 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:35 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 6]}]': finished 2024-09-11T23:16:36.304 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:35 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2024-09-11T23:16:36.304 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:35 smithi076 ceph-mon[35296]: osdmap e84: 12 total, 12 up, 12 in 2024-09-11T23:16:36.304 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:35 smithi076 ceph-mon[35296]: mds.? [v2:172.21.15.136:6838/3211398630,v1:172.21.15.136:6839/3211398630] up:active 2024-09-11T23:16:36.304 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:35 smithi076 ceph-mon[35296]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-11T23:16:36.304 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:35 smithi076 ceph-mon[35296]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-09-11T23:16:36.304 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:35 smithi076 ceph-mon[35296]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-09-11T23:16:36.304 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:35 smithi076 ceph-mon[35296]: from='client.? 172.21.15.26:0/3165220220' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-11T23:16:36.669 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph fs set cephfs session_timeout 300 2024-09-11T23:16:36.931 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:16:37.210 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:36 smithi026 ceph-mon[31172]: osdmap e85: 12 total, 12 up, 12 in 2024-09-11T23:16:37.210 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:36 smithi026 ceph-mon[31172]: from='client.? 172.21.15.26:0/3165220220' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-11T23:16:37.210 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:36 smithi026 ceph-mon[31172]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-09-11T23:16:37.210 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:36 smithi026 ceph-mon[31172]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-09-11T23:16:37.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:36 smithi136 ceph-mon[35121]: osdmap e85: 12 total, 12 up, 12 in 2024-09-11T23:16:37.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:36 smithi136 ceph-mon[35121]: from='client.? 172.21.15.26:0/3165220220' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-11T23:16:37.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:36 smithi136 ceph-mon[35121]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-09-11T23:16:37.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:36 smithi136 ceph-mon[35121]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-09-11T23:16:37.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:36 smithi076 ceph-mon[35296]: osdmap e85: 12 total, 12 up, 12 in 2024-09-11T23:16:37.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:36 smithi076 ceph-mon[35296]: from='client.? 172.21.15.26:0/3165220220' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-11T23:16:37.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:36 smithi076 ceph-mon[35296]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-09-11T23:16:37.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:36 smithi076 ceph-mon[35296]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-09-11T23:16:37.303 INFO:journalctl@ceph.mds.b.smithi076.stdout:Sep 11 23:16:36 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mds-b[65334]: 2024-09-11T23:16:36.973+0000 7fef227ea640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-09-11T23:16:37.303 INFO:journalctl@ceph.mds.b.smithi076.stdout:Sep 11 23:16:36 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mds-b[65334]: 2024-09-11T23:16:36.973+0000 7fef227ea640 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-09-11T23:16:37.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:37.383+0000 7fd37e5ee640 1 -- 172.21.15.26:0/1672332845 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd378100e20 msgr2=0x7fd378101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:16:37.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:37.383+0000 7fd37e5ee640 1 --2- 172.21.15.26:0/1672332845 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd378100e20 0x7fd378101200 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fd368009900 tx=0x7fd36802a480 comp rx=0 tx=0).stop 2024-09-11T23:16:37.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:37.383+0000 7fd37e5ee640 1 -- 172.21.15.26:0/1672332845 shutdown_connections 2024-09-11T23:16:37.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:37.383+0000 7fd37e5ee640 1 --2- 172.21.15.26:0/1672332845 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd378102570 0x7fd3781029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:37.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:37.383+0000 7fd37e5ee640 1 --2- 172.21.15.26:0/1672332845 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd378102ef0 0x7fd37810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:37.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:37.383+0000 7fd37e5ee640 1 --2- 172.21.15.26:0/1672332845 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd378100e20 0x7fd378101200 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:37.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:37.383+0000 7fd37e5ee640 1 -- 172.21.15.26:0/1672332845 >> 172.21.15.26:0/1672332845 conn(0x7fd3780fc7d0 msgr2=0x7fd3780febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:16:37.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:37.383+0000 7fd37e5ee640 1 -- 172.21.15.26:0/1672332845 shutdown_connections 2024-09-11T23:16:37.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:37.383+0000 7fd37e5ee640 1 -- 172.21.15.26:0/1672332845 wait complete. 2024-09-11T23:16:37.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:37.384+0000 7fd37e5ee640 1 Processor -- start 2024-09-11T23:16:37.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:37.384+0000 7fd37e5ee640 1 -- start start 2024-09-11T23:16:37.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:37.384+0000 7fd37e5ee640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd378100e20 0x7fd378198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:16:37.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:37.384+0000 7fd37e5ee640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd378102570 0x7fd378198bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:16:37.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:37.384+0000 7fd37e5ee640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd378102ef0 0x7fd37819cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:16:37.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:37.384+0000 7fd37e5ee640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd378111020 con 0x7fd378102ef0 2024-09-11T23:16:37.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:37.384+0000 7fd37e5ee640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fd378110ea0 con 0x7fd378100e20 2024-09-11T23:16:37.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:37.384+0000 7fd37e5ee640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd3781111a0 con 0x7fd378102570 2024-09-11T23:16:37.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:37.384+0000 7fd37cb8b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd378102ef0 0x7fd37819cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:16:37.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:37.384+0000 7fd37cb8b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd378102ef0 0x7fd37819cf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:32798/0 (socket says 172.21.15.26:32798) 2024-09-11T23:16:37.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:37.384+0000 7fd37cb8b640 1 -- 172.21.15.26:0/3403948535 learned_addr learned my addr 172.21.15.26:0/3403948535 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:16:37.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:37.384+0000 7fd377fff640 1 --2- 172.21.15.26:0/3403948535 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd378100e20 0x7fd378198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:16:37.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:37.384+0000 7fd36ffff640 1 --2- 172.21.15.26:0/3403948535 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd378102570 0x7fd378198bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:16:37.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:37.384+0000 7fd37cb8b640 1 -- 172.21.15.26:0/3403948535 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd378102570 msgr2=0x7fd378198bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:16:37.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:37.384+0000 7fd37cb8b640 1 --2- 172.21.15.26:0/3403948535 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd378102570 0x7fd378198bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:37.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:37.384+0000 7fd37cb8b640 1 -- 172.21.15.26:0/3403948535 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd378100e20 msgr2=0x7fd378198670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:16:37.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:37.384+0000 7fd37cb8b640 1 --2- 172.21.15.26:0/3403948535 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd378100e20 0x7fd378198670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:37.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:37.384+0000 7fd37cb8b640 1 -- 172.21.15.26:0/3403948535 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd37819d620 con 0x7fd378102ef0 2024-09-11T23:16:37.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:37.385+0000 7fd37cb8b640 1 --2- 172.21.15.26:0/3403948535 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd378102ef0 0x7fd37819cf40 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fd37000b970 tx=0x7fd37000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:16:37.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:37.385+0000 7fd375ffb640 1 -- 172.21.15.26:0/3403948535 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd370019070 con 0x7fd378102ef0 2024-09-11T23:16:37.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:37.385+0000 7fd37e5ee640 1 -- 172.21.15.26:0/3403948535 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd37819d910 con 0x7fd378102ef0 2024-09-11T23:16:37.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:37.385+0000 7fd375ffb640 1 -- 172.21.15.26:0/3403948535 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd37000cb40 con 0x7fd378102ef0 2024-09-11T23:16:37.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:37.385+0000 7fd375ffb640 1 -- 172.21.15.26:0/3403948535 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd370007650 con 0x7fd378102ef0 2024-09-11T23:16:37.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:37.385+0000 7fd37e5ee640 1 -- 172.21.15.26:0/3403948535 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3780fcbb0 con 0x7fd378102ef0 2024-09-11T23:16:37.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:37.386+0000 7fd37e5ee640 1 -- 172.21.15.26:0/3403948535 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd378101280 con 0x7fd378102ef0 2024-09-11T23:16:37.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:37.387+0000 7fd375ffb640 1 -- 172.21.15.26:0/3403948535 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fd37001c070 con 0x7fd378102ef0 2024-09-11T23:16:37.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:37.388+0000 7fd375ffb640 1 --2- 172.21.15.26:0/3403948535 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fd3640a2b80 0x7fd3640a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:16:37.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:37.388+0000 7fd377fff640 1 --2- 172.21.15.26:0/3403948535 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fd3640a2b80 0x7fd3640a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:16:37.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:37.388+0000 7fd375ffb640 1 -- 172.21.15.26:0/3403948535 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fd3700ce5a0 con 0x7fd378102ef0 2024-09-11T23:16:37.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:37.388+0000 7fd377fff640 1 --2- 172.21.15.26:0/3403948535 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fd3640a2b80 0x7fd3640a5040 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fd368000c00 tx=0x7fd3680058d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:16:37.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:37.392+0000 7fd375ffb640 1 -- 172.21.15.26:0/3403948535 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd37009aeb0 con 0x7fd378102ef0 2024-09-11T23:16:37.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:37.603+0000 7fd37e5ee640 1 -- 172.21.15.26:0/3403948535 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7fd37810a0a0 con 0x7fd378102ef0 2024-09-11T23:16:37.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:37.962+0000 7fd375ffb640 1 -- 172.21.15.26:0/3403948535 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v25) ==== 115+0+0 (secure 0 0 0) 0x7fd37009f1b0 con 0x7fd378102ef0 2024-09-11T23:16:37.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:37.965+0000 7fd37e5ee640 1 -- 172.21.15.26:0/3403948535 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fd3640a2b80 msgr2=0x7fd3640a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:16:37.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:37.965+0000 7fd37e5ee640 1 --2- 172.21.15.26:0/3403948535 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fd3640a2b80 0x7fd3640a5040 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fd368000c00 tx=0x7fd3680058d0 comp rx=0 tx=0).stop 2024-09-11T23:16:37.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:37.965+0000 7fd37e5ee640 1 -- 172.21.15.26:0/3403948535 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd378102ef0 msgr2=0x7fd37819cf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:16:37.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:37.965+0000 7fd37e5ee640 1 --2- 172.21.15.26:0/3403948535 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd378102ef0 0x7fd37819cf40 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fd37000b970 tx=0x7fd37000be30 comp rx=0 tx=0).stop 2024-09-11T23:16:37.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:37.965+0000 7fd37e5ee640 1 -- 172.21.15.26:0/3403948535 shutdown_connections 2024-09-11T23:16:37.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:37.965+0000 7fd37e5ee640 1 --2- 172.21.15.26:0/3403948535 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fd3640a2b80 0x7fd3640a5040 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:37.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:37.965+0000 7fd37e5ee640 1 --2- 172.21.15.26:0/3403948535 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd378102ef0 0x7fd37819cf40 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:37.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:37.965+0000 7fd37e5ee640 1 --2- 172.21.15.26:0/3403948535 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd378102570 0x7fd378198bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:37.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:37.965+0000 7fd37e5ee640 1 --2- 172.21.15.26:0/3403948535 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd378100e20 0x7fd378198670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:37.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:37.965+0000 7fd37e5ee640 1 -- 172.21.15.26:0/3403948535 >> 172.21.15.26:0/3403948535 conn(0x7fd3780fc7d0 msgr2=0x7fd37806a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:16:37.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:37.965+0000 7fd37e5ee640 1 -- 172.21.15.26:0/3403948535 shutdown_connections 2024-09-11T23:16:37.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:37.965+0000 7fd37e5ee640 1 -- 172.21.15.26:0/3403948535 wait complete. 2024-09-11T23:16:38.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:37 smithi136 ceph-mon[35121]: pgmap v286: 97 pgs: 5 peering, 92 active+clean; 581 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s wr, 12 op/s 2024-09-11T23:16:38.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:37 smithi136 ceph-mon[35121]: mds.? [v2:172.21.15.76:6836/2078978748,v1:172.21.15.76:6837/2078978748] up:active 2024-09-11T23:16:38.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:37 smithi136 ceph-mon[35121]: daemon mds.c assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-09-11T23:16:38.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:37 smithi136 ceph-mon[35121]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-09-11T23:16:38.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:37 smithi136 ceph-mon[35121]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:creating} 8 up:standby 2024-09-11T23:16:38.240 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:37 smithi136 ceph-mon[35121]: daemon mds.c is now active in filesystem cephfs as rank 3 2024-09-11T23:16:38.240 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:37 smithi136 ceph-mon[35121]: from='client.? 172.21.15.26:0/3403948535' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-11T23:16:38.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:37 smithi026 ceph-mon[31172]: pgmap v286: 97 pgs: 5 peering, 92 active+clean; 581 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s wr, 12 op/s 2024-09-11T23:16:38.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:37 smithi026 ceph-mon[31172]: mds.? [v2:172.21.15.76:6836/2078978748,v1:172.21.15.76:6837/2078978748] up:active 2024-09-11T23:16:38.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:37 smithi026 ceph-mon[31172]: daemon mds.c assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-09-11T23:16:38.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:37 smithi026 ceph-mon[31172]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-09-11T23:16:38.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:37 smithi026 ceph-mon[31172]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:creating} 8 up:standby 2024-09-11T23:16:38.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:37 smithi026 ceph-mon[31172]: daemon mds.c is now active in filesystem cephfs as rank 3 2024-09-11T23:16:38.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:37 smithi026 ceph-mon[31172]: from='client.? 172.21.15.26:0/3403948535' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-11T23:16:38.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:37 smithi076 ceph-mon[35296]: pgmap v286: 97 pgs: 5 peering, 92 active+clean; 581 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s wr, 12 op/s 2024-09-11T23:16:38.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:37 smithi076 ceph-mon[35296]: mds.? [v2:172.21.15.76:6836/2078978748,v1:172.21.15.76:6837/2078978748] up:active 2024-09-11T23:16:38.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:37 smithi076 ceph-mon[35296]: daemon mds.c assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-09-11T23:16:38.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:37 smithi076 ceph-mon[35296]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-09-11T23:16:38.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:37 smithi076 ceph-mon[35296]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:creating} 8 up:standby 2024-09-11T23:16:38.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:37 smithi076 ceph-mon[35296]: daemon mds.c is now active in filesystem cephfs as rank 3 2024-09-11T23:16:38.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:37 smithi076 ceph-mon[35296]: from='client.? 172.21.15.26:0/3403948535' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-11T23:16:38.729 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated'} subvols for filesystem 'cephfs' 2024-09-11T23:16:38.729 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph fs subvolumegroup create cephfs qa 2024-09-11T23:16:39.002 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:16:39.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:38 smithi136 ceph-mon[35121]: from='client.? 172.21.15.26:0/3403948535' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-11T23:16:39.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:38 smithi136 ceph-mon[35121]: mds.? [v2:172.21.15.136:6836/1633082387,v1:172.21.15.136:6837/1633082387] up:active 2024-09-11T23:16:39.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:38 smithi136 ceph-mon[35121]: daemon mds.e assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-09-11T23:16:39.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:38 smithi136 ceph-mon[35121]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-11T23:16:39.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:38 smithi136 ceph-mon[35121]: Cluster is now healthy 2024-09-11T23:16:39.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:38 smithi136 ceph-mon[35121]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:active} 8 up:standby 2024-09-11T23:16:39.240 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:38 smithi136 ceph-mon[35121]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-09-11T23:16:39.240 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:38 smithi136 ceph-mon[35121]: daemon mds.e is now active in filesystem cephfs as rank 4 2024-09-11T23:16:39.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:38 smithi026 ceph-mon[31172]: from='client.? 172.21.15.26:0/3403948535' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-11T23:16:39.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:38 smithi026 ceph-mon[31172]: mds.? [v2:172.21.15.136:6836/1633082387,v1:172.21.15.136:6837/1633082387] up:active 2024-09-11T23:16:39.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:38 smithi026 ceph-mon[31172]: daemon mds.e assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-09-11T23:16:39.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:38 smithi026 ceph-mon[31172]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-11T23:16:39.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:38 smithi026 ceph-mon[31172]: Cluster is now healthy 2024-09-11T23:16:39.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:38 smithi026 ceph-mon[31172]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:active} 8 up:standby 2024-09-11T23:16:39.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:38 smithi026 ceph-mon[31172]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-09-11T23:16:39.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:38 smithi026 ceph-mon[31172]: daemon mds.e is now active in filesystem cephfs as rank 4 2024-09-11T23:16:39.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:38 smithi076 ceph-mon[35296]: from='client.? 172.21.15.26:0/3403948535' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-11T23:16:39.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:38 smithi076 ceph-mon[35296]: mds.? [v2:172.21.15.136:6836/1633082387,v1:172.21.15.136:6837/1633082387] up:active 2024-09-11T23:16:39.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:38 smithi076 ceph-mon[35296]: daemon mds.e assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-09-11T23:16:39.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:38 smithi076 ceph-mon[35296]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-11T23:16:39.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:38 smithi076 ceph-mon[35296]: Cluster is now healthy 2024-09-11T23:16:39.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:38 smithi076 ceph-mon[35296]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:active} 8 up:standby 2024-09-11T23:16:39.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:38 smithi076 ceph-mon[35296]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-09-11T23:16:39.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:38 smithi076 ceph-mon[35296]: daemon mds.e is now active in filesystem cephfs as rank 4 2024-09-11T23:16:39.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:39.466+0000 7f3506a0f640 1 -- 172.21.15.26:0/3983842763 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3500102790 msgr2=0x7f35001067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:16:39.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:39.466+0000 7f3506a0f640 1 --2- 172.21.15.26:0/3983842763 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3500102790 0x7f35001067e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f34e8009900 tx=0x7f34e802a410 comp rx=0 tx=0).stop 2024-09-11T23:16:39.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:39.466+0000 7f3506a0f640 1 -- 172.21.15.26:0/3983842763 shutdown_connections 2024-09-11T23:16:39.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:39.466+0000 7f3506a0f640 1 --2- 172.21.15.26:0/3983842763 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3500106f10 0x7f350010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:39.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:39.466+0000 7f3506a0f640 1 --2- 172.21.15.26:0/3983842763 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3500102790 0x7f35001067e0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:39.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:39.466+0000 7f3506a0f640 1 --2- 172.21.15.26:0/3983842763 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3500101de0 0x7f35001021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:39.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:39.466+0000 7f3506a0f640 1 -- 172.21.15.26:0/3983842763 >> 172.21.15.26:0/3983842763 conn(0x7f35000fd8c0 msgr2=0x7f35000ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:16:39.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:39.466+0000 7f3506a0f640 1 -- 172.21.15.26:0/3983842763 shutdown_connections 2024-09-11T23:16:39.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:39.466+0000 7f3506a0f640 1 -- 172.21.15.26:0/3983842763 wait complete. 2024-09-11T23:16:39.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:39.467+0000 7f3506a0f640 1 Processor -- start 2024-09-11T23:16:39.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:39.467+0000 7f3506a0f640 1 -- start start 2024-09-11T23:16:39.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:39.467+0000 7f3506a0f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3500101de0 0x7f35000744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:16:39.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:39.467+0000 7f3506a0f640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3500102790 0x7f3500074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:16:39.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:39.467+0000 7f3506a0f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3500106f10 0x7f3500074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:16:39.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:39.467+0000 7f3506a0f640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f3500111f70 con 0x7f3500101de0 2024-09-11T23:16:39.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:39.467+0000 7f3506a0f640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f3500111df0 con 0x7f3500106f10 2024-09-11T23:16:39.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:39.467+0000 7f3506a0f640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f35001120f0 con 0x7f3500102790 2024-09-11T23:16:39.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:39.467+0000 7f34fffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3500101de0 0x7f35000744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:16:39.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:39.468+0000 7f34fffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3500101de0 0x7f35000744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:32828/0 (socket says 172.21.15.26:32828) 2024-09-11T23:16:39.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:39.468+0000 7f34fffff640 1 -- 172.21.15.26:0/1137460863 learned_addr learned my addr 172.21.15.26:0/1137460863 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:16:39.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:39.468+0000 7f3504fac640 1 --2- 172.21.15.26:0/1137460863 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3500106f10 0x7f3500074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:16:39.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:39.468+0000 7f34f7fff640 1 --2- 172.21.15.26:0/1137460863 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3500102790 0x7f3500074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:16:39.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:39.468+0000 7f34fffff640 1 -- 172.21.15.26:0/1137460863 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3500102790 msgr2=0x7f3500074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:16:39.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:39.468+0000 7f34fffff640 1 --2- 172.21.15.26:0/1137460863 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3500102790 0x7f3500074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:39.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:39.468+0000 7f34fffff640 1 -- 172.21.15.26:0/1137460863 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3500106f10 msgr2=0x7f3500074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:16:39.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:39.468+0000 7f34fffff640 1 --2- 172.21.15.26:0/1137460863 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3500106f10 0x7f3500074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:39.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:39.468+0000 7f34fffff640 1 -- 172.21.15.26:0/1137460863 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35001aae40 con 0x7f3500101de0 2024-09-11T23:16:39.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:39.468+0000 7f34fffff640 1 --2- 172.21.15.26:0/1137460863 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3500101de0 0x7f35000744f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f34f000c890 tx=0x7f34f000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:16:39.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:39.468+0000 7f34fdffb640 1 -- 172.21.15.26:0/1137460863 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34f0004220 con 0x7f3500101de0 2024-09-11T23:16:39.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:39.468+0000 7f3506a0f640 1 -- 172.21.15.26:0/1137460863 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35001ab070 con 0x7f3500101de0 2024-09-11T23:16:39.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:39.468+0000 7f34fdffb640 1 -- 172.21.15.26:0/1137460863 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f34f00043c0 con 0x7f3500101de0 2024-09-11T23:16:39.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:39.468+0000 7f34fdffb640 1 -- 172.21.15.26:0/1137460863 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34f0004a00 con 0x7f3500101de0 2024-09-11T23:16:39.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:39.468+0000 7f3506a0f640 1 -- 172.21.15.26:0/1137460863 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35001ab5d0 con 0x7f3500101de0 2024-09-11T23:16:39.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:39.469+0000 7f3506a0f640 1 -- 172.21.15.26:0/1137460863 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35001021c0 con 0x7f3500101de0 2024-09-11T23:16:39.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:39.470+0000 7f34fdffb640 1 -- 172.21.15.26:0/1137460863 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f34f00079b0 con 0x7f3500101de0 2024-09-11T23:16:39.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:39.471+0000 7f34fdffb640 1 --2- 172.21.15.26:0/1137460863 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f34ec0a2b80 0x7f34ec0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:16:39.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:39.471+0000 7f34fdffb640 1 -- 172.21.15.26:0/1137460863 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f34f00ced40 con 0x7f3500101de0 2024-09-11T23:16:39.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:39.471+0000 7f34f7fff640 1 --2- 172.21.15.26:0/1137460863 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f34ec0a2b80 0x7f34ec0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:16:39.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:39.473+0000 7f34f7fff640 1 --2- 172.21.15.26:0/1137460863 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f34ec0a2b80 0x7f34ec0a5040 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f3500108520 tx=0x7f34e8005a20 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:16:39.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:39.476+0000 7f34fdffb640 1 -- 172.21.15.26:0/1137460863 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f34f0014030 con 0x7f3500101de0 2024-09-11T23:16:39.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:39.674+0000 7f3506a0f640 1 -- 172.21.15.26:0/1137460863 --> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f3500062630 con 0x7f34ec0a2b80 2024-09-11T23:16:39.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:39.701+0000 7f34fdffb640 1 -- 172.21.15.26:0/1137460863 <== mgr.14148 v2:172.21.15.26:6800/110833859 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f3500062630 con 0x7f34ec0a2b80 2024-09-11T23:16:39.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:39.703+0000 7f3506a0f640 1 -- 172.21.15.26:0/1137460863 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f34ec0a2b80 msgr2=0x7f34ec0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:16:39.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:39.703+0000 7f3506a0f640 1 --2- 172.21.15.26:0/1137460863 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f34ec0a2b80 0x7f34ec0a5040 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f3500108520 tx=0x7f34e8005a20 comp rx=0 tx=0).stop 2024-09-11T23:16:39.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:39.703+0000 7f3506a0f640 1 -- 172.21.15.26:0/1137460863 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3500101de0 msgr2=0x7f35000744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:16:39.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:39.703+0000 7f3506a0f640 1 --2- 172.21.15.26:0/1137460863 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3500101de0 0x7f35000744f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f34f000c890 tx=0x7f34f000cd50 comp rx=0 tx=0).stop 2024-09-11T23:16:39.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:39.703+0000 7f3506a0f640 1 -- 172.21.15.26:0/1137460863 shutdown_connections 2024-09-11T23:16:39.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:39.703+0000 7f3506a0f640 1 --2- 172.21.15.26:0/1137460863 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f34ec0a2b80 0x7f34ec0a5040 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:39.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:39.703+0000 7f3506a0f640 1 --2- 172.21.15.26:0/1137460863 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3500106f10 0x7f3500074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:39.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:39.703+0000 7f3506a0f640 1 --2- 172.21.15.26:0/1137460863 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3500102790 0x7f3500074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:39.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:39.703+0000 7f3506a0f640 1 --2- 172.21.15.26:0/1137460863 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3500101de0 0x7f35000744f0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:39.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:39.703+0000 7f3506a0f640 1 -- 172.21.15.26:0/1137460863 >> 172.21.15.26:0/1137460863 conn(0x7f35000fd8c0 msgr2=0x7f35000fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:16:39.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:39.703+0000 7f3506a0f640 1 -- 172.21.15.26:0/1137460863 shutdown_connections 2024-09-11T23:16:39.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:39.703+0000 7f3506a0f640 1 -- 172.21.15.26:0/1137460863 wait complete. 2024-09-11T23:16:39.975 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:16:39 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: 2024-09-11T23:16:39.679+0000 7faa24091640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T23:16:39.975 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:16:39 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: 2024-09-11T23:16:39.679+0000 7faa24091640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T23:16:39.975 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:16:39 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: 2024-09-11T23:16:39.679+0000 7faa24091640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T23:16:39.975 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:16:39 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: 2024-09-11T23:16:39.679+0000 7faa24091640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T23:16:39.975 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:16:39 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: 2024-09-11T23:16:39.679+0000 7faa24091640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T23:16:40.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:39 smithi136 ceph-mon[35121]: pgmap v287: 97 pgs: 5 peering, 92 active+clean; 581 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s wr, 8 op/s 2024-09-11T23:16:40.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:39 smithi136 ceph-mon[35121]: mds.? [v2:172.21.15.76:6834/393308499,v1:172.21.15.76:6835/393308499] up:active 2024-09-11T23:16:40.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:39 smithi136 ceph-mon[35121]: fsmap cephfs:5 7 up:standby 5 up:active 2024-09-11T23:16:40.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:39 smithi136 ceph-mon[35121]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:16:40.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:39 smithi026 ceph-mon[31172]: pgmap v287: 97 pgs: 5 peering, 92 active+clean; 581 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s wr, 8 op/s 2024-09-11T23:16:40.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:39 smithi026 ceph-mon[31172]: mds.? [v2:172.21.15.76:6834/393308499,v1:172.21.15.76:6835/393308499] up:active 2024-09-11T23:16:40.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:39 smithi026 ceph-mon[31172]: fsmap cephfs:5 7 up:standby 5 up:active 2024-09-11T23:16:40.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:39 smithi026 ceph-mon[31172]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:16:40.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:39 smithi076 ceph-mon[35296]: pgmap v287: 97 pgs: 5 peering, 92 active+clean; 581 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s wr, 8 op/s 2024-09-11T23:16:40.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:39 smithi076 ceph-mon[35296]: mds.? [v2:172.21.15.76:6834/393308499,v1:172.21.15.76:6835/393308499] up:active 2024-09-11T23:16:40.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:39 smithi076 ceph-mon[35296]: fsmap cephfs:5 7 up:standby 5 up:active 2024-09-11T23:16:40.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:39 smithi076 ceph-mon[35296]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:16:40.438 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-09-11T23:16:40.697 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:16:41.080 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:40 smithi026 ceph-mon[31172]: from='client.14646 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:16:41.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:41.166+0000 7fc6c2076640 1 -- 172.21.15.26:0/2442294554 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc6bc10c1b0 msgr2=0x7fc6bc10e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:16:41.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:41.166+0000 7fc6c2076640 1 --2- 172.21.15.26:0/2442294554 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc6bc10c1b0 0x7fc6bc10e620 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fc6b4009ca0 tx=0x7fc6b402a950 comp rx=0 tx=0).stop 2024-09-11T23:16:41.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:41.166+0000 7fc6c2076640 1 -- 172.21.15.26:0/2442294554 shutdown_connections 2024-09-11T23:16:41.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:41.167+0000 7fc6c2076640 1 --2- 172.21.15.26:0/2442294554 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc6bc1036f0 0x7fc6bc10bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:41.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:41.167+0000 7fc6c2076640 1 --2- 172.21.15.26:0/2442294554 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc6bc10c1b0 0x7fc6bc10e620 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:41.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:41.167+0000 7fc6c2076640 1 --2- 172.21.15.26:0/2442294554 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc6bc102dd0 0x7fc6bc1031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:41.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:41.167+0000 7fc6c2076640 1 -- 172.21.15.26:0/2442294554 >> 172.21.15.26:0/2442294554 conn(0x7fc6bc0fc7d0 msgr2=0x7fc6bc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:16:41.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:41.167+0000 7fc6c2076640 1 -- 172.21.15.26:0/2442294554 shutdown_connections 2024-09-11T23:16:41.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:41.167+0000 7fc6c2076640 1 -- 172.21.15.26:0/2442294554 wait complete. 2024-09-11T23:16:41.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:41.167+0000 7fc6c2076640 1 Processor -- start 2024-09-11T23:16:41.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:41.167+0000 7fc6c2076640 1 -- start start 2024-09-11T23:16:41.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:41.168+0000 7fc6c2076640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc6bc102dd0 0x7fc6bc1983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:16:41.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:41.168+0000 7fc6c2076640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc6bc1036f0 0x7fc6bc1988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:16:41.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:41.168+0000 7fc6c2076640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc6bc10c1b0 0x7fc6bc19cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:16:41.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:41.168+0000 7fc6c2076640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc6bc111f80 con 0x7fc6bc102dd0 2024-09-11T23:16:41.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:41.168+0000 7fc6c2076640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fc6bc111e00 con 0x7fc6bc1036f0 2024-09-11T23:16:41.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:41.168+0000 7fc6c2076640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fc6bc112100 con 0x7fc6bc10c1b0 2024-09-11T23:16:41.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:41.168+0000 7fc6bb7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc6bc102dd0 0x7fc6bc1983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:16:41.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:41.168+0000 7fc6bb7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc6bc102dd0 0x7fc6bc1983a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39702/0 (socket says 172.21.15.26:39702) 2024-09-11T23:16:41.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:41.168+0000 7fc6bb7fe640 1 -- 172.21.15.26:0/1844976936 learned_addr learned my addr 172.21.15.26:0/1844976936 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:16:41.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:41.168+0000 7fc6baffd640 1 --2- 172.21.15.26:0/1844976936 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc6bc1036f0 0x7fc6bc1988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:16:41.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:41.168+0000 7fc6bb7fe640 1 -- 172.21.15.26:0/1844976936 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc6bc10c1b0 msgr2=0x7fc6bc19cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:16:41.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:41.168+0000 7fc6bb7fe640 1 --2- 172.21.15.26:0/1844976936 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc6bc10c1b0 0x7fc6bc19cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:41.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:41.168+0000 7fc6bb7fe640 1 -- 172.21.15.26:0/1844976936 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc6bc1036f0 msgr2=0x7fc6bc1988e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:16:41.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:41.168+0000 7fc6bb7fe640 1 --2- 172.21.15.26:0/1844976936 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc6bc1036f0 0x7fc6bc1988e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:41.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:41.168+0000 7fc6bb7fe640 1 -- 172.21.15.26:0/1844976936 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6bc19d350 con 0x7fc6bc102dd0 2024-09-11T23:16:41.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:41.168+0000 7fc6baffd640 1 --2- 172.21.15.26:0/1844976936 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc6bc1036f0 0x7fc6bc1988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:16:41.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:41.168+0000 7fc6bb7fe640 1 --2- 172.21.15.26:0/1844976936 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc6bc102dd0 0x7fc6bc1983a0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fc6ac00e9f0 tx=0x7fc6ac00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:16:41.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:41.168+0000 7fc6b8ff9640 1 -- 172.21.15.26:0/1844976936 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6ac00cd80 con 0x7fc6bc102dd0 2024-09-11T23:16:41.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:41.168+0000 7fc6c2076640 1 -- 172.21.15.26:0/1844976936 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6bc19d640 con 0x7fc6bc102dd0 2024-09-11T23:16:41.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:41.169+0000 7fc6b8ff9640 1 -- 172.21.15.26:0/1844976936 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc6ac005500 con 0x7fc6bc102dd0 2024-09-11T23:16:41.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:41.169+0000 7fc6b8ff9640 1 -- 172.21.15.26:0/1844976936 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6ac010690 con 0x7fc6bc102dd0 2024-09-11T23:16:41.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:41.169+0000 7fc6c2076640 1 -- 172.21.15.26:0/1844976936 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6bc1a4f20 con 0x7fc6bc102dd0 2024-09-11T23:16:41.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:41.170+0000 7fc6c2076640 1 -- 172.21.15.26:0/1844976936 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6bc1031b0 con 0x7fc6bc102dd0 2024-09-11T23:16:41.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:41.171+0000 7fc6b8ff9640 1 -- 172.21.15.26:0/1844976936 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fc6ac0046c0 con 0x7fc6bc102dd0 2024-09-11T23:16:41.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:41.171+0000 7fc6b8ff9640 1 --2- 172.21.15.26:0/1844976936 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fc6a40a2c10 0x7fc6a40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:16:41.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:41.171+0000 7fc6baffd640 1 --2- 172.21.15.26:0/1844976936 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fc6a40a2c10 0x7fc6a40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:16:41.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:41.171+0000 7fc6b8ff9640 1 -- 172.21.15.26:0/1844976936 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fc6ac0cf710 con 0x7fc6bc102dd0 2024-09-11T23:16:41.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:41.172+0000 7fc6baffd640 1 --2- 172.21.15.26:0/1844976936 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fc6a40a2c10 0x7fc6a40a50d0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fc6bc10d460 tx=0x7fc6b0005d20 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:16:41.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:41.176+0000 7fc6b8ff9640 1 -- 172.21.15.26:0/1844976936 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc6ac09c510 con 0x7fc6bc102dd0 2024-09-11T23:16:41.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:40 smithi076 ceph-mon[35296]: from='client.14646 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:16:41.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:41.378+0000 7fc6c2076640 1 -- 172.21.15.26:0/1844976936 --> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fc6bc062630 con 0x7fc6a40a2c10 2024-09-11T23:16:41.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:41.404+0000 7fc6b8ff9640 1 -- 172.21.15.26:0/1844976936 <== mgr.14148 v2:172.21.15.26:6800/110833859 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fc6bc062630 con 0x7fc6a40a2c10 2024-09-11T23:16:41.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:41.406+0000 7fc6c2076640 1 -- 172.21.15.26:0/1844976936 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fc6a40a2c10 msgr2=0x7fc6a40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:16:41.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:41.406+0000 7fc6c2076640 1 --2- 172.21.15.26:0/1844976936 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fc6a40a2c10 0x7fc6a40a50d0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fc6bc10d460 tx=0x7fc6b0005d20 comp rx=0 tx=0).stop 2024-09-11T23:16:41.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:41.406+0000 7fc6c2076640 1 -- 172.21.15.26:0/1844976936 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc6bc102dd0 msgr2=0x7fc6bc1983a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:16:41.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:41.406+0000 7fc6c2076640 1 --2- 172.21.15.26:0/1844976936 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc6bc102dd0 0x7fc6bc1983a0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fc6ac00e9f0 tx=0x7fc6ac00edf0 comp rx=0 tx=0).stop 2024-09-11T23:16:41.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:41.406+0000 7fc6c2076640 1 -- 172.21.15.26:0/1844976936 shutdown_connections 2024-09-11T23:16:41.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:41.406+0000 7fc6c2076640 1 --2- 172.21.15.26:0/1844976936 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fc6a40a2c10 0x7fc6a40a50d0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:41.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:41.406+0000 7fc6c2076640 1 --2- 172.21.15.26:0/1844976936 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc6bc1036f0 0x7fc6bc1988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:41.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:41.406+0000 7fc6c2076640 1 --2- 172.21.15.26:0/1844976936 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc6bc10c1b0 0x7fc6bc19cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:41.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:41.406+0000 7fc6c2076640 1 --2- 172.21.15.26:0/1844976936 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc6bc102dd0 0x7fc6bc1983a0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:41.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:41.406+0000 7fc6c2076640 1 -- 172.21.15.26:0/1844976936 >> 172.21.15.26:0/1844976936 conn(0x7fc6bc0fc7d0 msgr2=0x7fc6bc0fd430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:16:41.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:41.407+0000 7fc6c2076640 1 -- 172.21.15.26:0/1844976936 shutdown_connections 2024-09-11T23:16:41.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:41.407+0000 7fc6c2076640 1 -- 172.21.15.26:0/1844976936 wait complete. 2024-09-11T23:16:41.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:40 smithi136 ceph-mon[35121]: from='client.14646 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:16:42.109 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:41 smithi026 ceph-mon[31172]: pgmap v288: 97 pgs: 5 peering, 92 active+clean; 584 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 5.4 KiB/s wr, 17 op/s; 0 B/s, 0 objects/s recovering 2024-09-11T23:16:42.109 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:41 smithi026 ceph-mon[31172]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-11T23:16:42.140 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-09-11T23:16:42.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:41 smithi076 ceph-mon[35296]: pgmap v288: 97 pgs: 5 peering, 92 active+clean; 584 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 5.4 KiB/s wr, 17 op/s; 0 B/s, 0 objects/s recovering 2024-09-11T23:16:42.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:41 smithi076 ceph-mon[35296]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-11T23:16:42.303 INFO:journalctl@ceph.mds.b.smithi076.stdout:Sep 11 23:16:41 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mds-b[65334]: 2024-09-11T23:16:41.973+0000 7fef227ea640 -1 mds.pinger is_rank_lagging: rank=3 was never sent ping request. 2024-09-11T23:16:42.303 INFO:journalctl@ceph.mds.b.smithi076.stdout:Sep 11 23:16:41 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mds-b[65334]: 2024-09-11T23:16:41.973+0000 7fef227ea640 -1 mds.pinger is_rank_lagging: rank=4 was never sent ping request. 2024-09-11T23:16:42.398 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:16:42.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:41 smithi136 ceph-mon[35121]: pgmap v288: 97 pgs: 5 peering, 92 active+clean; 584 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 5.4 KiB/s wr, 17 op/s; 0 B/s, 0 objects/s recovering 2024-09-11T23:16:42.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:41 smithi136 ceph-mon[35121]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-11T23:16:42.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:42.865+0000 7f1fd8a20640 1 -- 172.21.15.26:0/376877355 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1fd4101de0 msgr2=0x7f1fd41021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:16:42.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:42.865+0000 7f1fd8a20640 1 --2- 172.21.15.26:0/376877355 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1fd4101de0 0x7f1fd41021c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f1fc4009900 tx=0x7f1fc402a410 comp rx=0 tx=0).stop 2024-09-11T23:16:42.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:42.865+0000 7f1fd8a20640 1 -- 172.21.15.26:0/376877355 shutdown_connections 2024-09-11T23:16:42.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:42.865+0000 7f1fd8a20640 1 --2- 172.21.15.26:0/376877355 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1fd4102790 0x7f1fd41067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:42.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:42.865+0000 7f1fd8a20640 1 --2- 172.21.15.26:0/376877355 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1fd4106f10 0x7f1fd410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:42.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:42.865+0000 7f1fd8a20640 1 --2- 172.21.15.26:0/376877355 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1fd4101de0 0x7f1fd41021c0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:42.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:42.865+0000 7f1fd8a20640 1 -- 172.21.15.26:0/376877355 >> 172.21.15.26:0/376877355 conn(0x7f1fd40fd8c0 msgr2=0x7f1fd40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:16:42.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:42.865+0000 7f1fd8a20640 1 -- 172.21.15.26:0/376877355 shutdown_connections 2024-09-11T23:16:42.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:42.865+0000 7f1fd8a20640 1 -- 172.21.15.26:0/376877355 wait complete. 2024-09-11T23:16:42.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:42.866+0000 7f1fd8a20640 1 Processor -- start 2024-09-11T23:16:42.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:42.866+0000 7f1fd8a20640 1 -- start start 2024-09-11T23:16:42.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:42.866+0000 7f1fd8a20640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1fd4101de0 0x7f1fd4198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:16:42.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:42.866+0000 7f1fd8a20640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1fd4102790 0x7f1fd41988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:16:42.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:42.866+0000 7f1fd8a20640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1fd4106f10 0x7f1fd419cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:16:42.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:42.866+0000 7f1fd8a20640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f1fd4111f70 con 0x7f1fd4102790 2024-09-11T23:16:42.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:42.866+0000 7f1fd8a20640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f1fd4111df0 con 0x7f1fd4101de0 2024-09-11T23:16:42.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:42.866+0000 7f1fd8a20640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f1fd41120f0 con 0x7f1fd4106f10 2024-09-11T23:16:42.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:42.866+0000 7f1fd1d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1fd4102790 0x7f1fd41988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:16:42.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:42.867+0000 7f1fd1d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1fd4102790 0x7f1fd41988b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39714/0 (socket says 172.21.15.26:39714) 2024-09-11T23:16:42.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:42.867+0000 7f1fd1d9b640 1 -- 172.21.15.26:0/4163719558 learned_addr learned my addr 172.21.15.26:0/4163719558 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:16:42.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:42.867+0000 7f1fd2d9d640 1 --2- 172.21.15.26:0/4163719558 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1fd4106f10 0x7f1fd419cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:16:42.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:42.867+0000 7f1fd259c640 1 --2- 172.21.15.26:0/4163719558 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1fd4101de0 0x7f1fd4198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:16:42.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:42.867+0000 7f1fd1d9b640 1 -- 172.21.15.26:0/4163719558 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1fd4106f10 msgr2=0x7f1fd419cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:16:42.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:42.867+0000 7f1fd1d9b640 1 --2- 172.21.15.26:0/4163719558 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1fd4106f10 0x7f1fd419cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:42.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:42.867+0000 7f1fd1d9b640 1 -- 172.21.15.26:0/4163719558 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1fd4101de0 msgr2=0x7f1fd4198370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:16:42.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:42.867+0000 7f1fd1d9b640 1 --2- 172.21.15.26:0/4163719558 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1fd4101de0 0x7f1fd4198370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:42.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:42.867+0000 7f1fd1d9b640 1 -- 172.21.15.26:0/4163719558 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fd419d320 con 0x7f1fd4102790 2024-09-11T23:16:42.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:42.867+0000 7f1fd1d9b640 1 --2- 172.21.15.26:0/4163719558 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1fd4102790 0x7f1fd41988b0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f1fc800c920 tx=0x7f1fc800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:16:42.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:42.867+0000 7f1fc37fe640 1 -- 172.21.15.26:0/4163719558 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fc8007c70 con 0x7f1fd4102790 2024-09-11T23:16:42.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:42.867+0000 7f1fc37fe640 1 -- 172.21.15.26:0/4163719558 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1fc8007e10 con 0x7f1fd4102790 2024-09-11T23:16:42.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:42.867+0000 7f1fd8a20640 1 -- 172.21.15.26:0/4163719558 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fd419d610 con 0x7f1fd4102790 2024-09-11T23:16:42.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:42.868+0000 7f1fc37fe640 1 -- 172.21.15.26:0/4163719558 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fc8018890 con 0x7f1fd4102790 2024-09-11T23:16:42.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:42.868+0000 7f1fd8a20640 1 -- 172.21.15.26:0/4163719558 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fd41a4ef0 con 0x7f1fd4102790 2024-09-11T23:16:42.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:42.868+0000 7f1fd8a20640 1 -- 172.21.15.26:0/4163719558 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1fd41021c0 con 0x7f1fd4102790 2024-09-11T23:16:42.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:42.869+0000 7f1fc37fe640 1 -- 172.21.15.26:0/4163719558 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f1fc8018b30 con 0x7f1fd4102790 2024-09-11T23:16:42.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:42.870+0000 7f1fc37fe640 1 --2- 172.21.15.26:0/4163719558 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f1fbc0a2c10 0x7f1fbc0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:16:42.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:42.870+0000 7f1fd259c640 1 --2- 172.21.15.26:0/4163719558 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f1fbc0a2c10 0x7f1fbc0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:16:42.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:42.870+0000 7f1fc37fe640 1 -- 172.21.15.26:0/4163719558 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f1fc80cf3d0 con 0x7f1fd4102790 2024-09-11T23:16:42.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:42.871+0000 7f1fd259c640 1 --2- 172.21.15.26:0/4163719558 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f1fbc0a2c10 0x7f1fbc0a50d0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f1fc4005ab0 tx=0x7f1fc4005a20 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:16:42.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:42.874+0000 7f1fc37fe640 1 -- 172.21.15.26:0/4163719558 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1fc8014030 con 0x7f1fd4102790 2024-09-11T23:16:43.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:43.077+0000 7f1fd8a20640 1 -- 172.21.15.26:0/4163719558 --> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f1fd4062630 con 0x7f1fbc0a2c10 2024-09-11T23:16:43.078 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:42 smithi026 ceph-mon[31172]: from='client.14658 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:16:43.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:43.082+0000 7f1fc37fe640 1 -- 172.21.15.26:0/4163719558 <== mgr.14148 v2:172.21.15.26:6800/110833859 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f1fd4062630 con 0x7f1fbc0a2c10 2024-09-11T23:16:43.082 INFO:teuthology.orchestra.run.smithi026.stdout:/volumes/qa/sv_0/63702ea3-e15d-461f-bc4b-eaea5c94d9d2 2024-09-11T23:16:43.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:43.084+0000 7f1fd8a20640 1 -- 172.21.15.26:0/4163719558 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f1fbc0a2c10 msgr2=0x7f1fbc0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:16:43.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:43.084+0000 7f1fd8a20640 1 --2- 172.21.15.26:0/4163719558 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f1fbc0a2c10 0x7f1fbc0a50d0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f1fc4005ab0 tx=0x7f1fc4005a20 comp rx=0 tx=0).stop 2024-09-11T23:16:43.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:43.084+0000 7f1fd8a20640 1 -- 172.21.15.26:0/4163719558 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1fd4102790 msgr2=0x7f1fd41988b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:16:43.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:43.084+0000 7f1fd8a20640 1 --2- 172.21.15.26:0/4163719558 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1fd4102790 0x7f1fd41988b0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f1fc800c920 tx=0x7f1fc800cde0 comp rx=0 tx=0).stop 2024-09-11T23:16:43.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:43.084+0000 7f1fd8a20640 1 -- 172.21.15.26:0/4163719558 shutdown_connections 2024-09-11T23:16:43.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:43.084+0000 7f1fd8a20640 1 --2- 172.21.15.26:0/4163719558 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f1fbc0a2c10 0x7f1fbc0a50d0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:43.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:43.084+0000 7f1fd8a20640 1 --2- 172.21.15.26:0/4163719558 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1fd4106f10 0x7f1fd419cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:43.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:43.084+0000 7f1fd8a20640 1 --2- 172.21.15.26:0/4163719558 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1fd4102790 0x7f1fd41988b0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:43.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:43.084+0000 7f1fd8a20640 1 --2- 172.21.15.26:0/4163719558 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1fd4101de0 0x7f1fd4198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:43.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:43.084+0000 7f1fd8a20640 1 -- 172.21.15.26:0/4163719558 >> 172.21.15.26:0/4163719558 conn(0x7f1fd40fd8c0 msgr2=0x7f1fd40fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:16:43.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:43.084+0000 7f1fd8a20640 1 -- 172.21.15.26:0/4163719558 shutdown_connections 2024-09-11T23:16:43.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:43.084+0000 7f1fd8a20640 1 -- 172.21.15.26:0/4163719558 wait complete. 2024-09-11T23:16:43.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:42 smithi076 ceph-mon[35296]: from='client.14658 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:16:43.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:42 smithi136 ceph-mon[35121]: from='client.14658 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:16:43.809 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-09-11T23:16:44.074 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:16:44.195 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:43 smithi026 ceph-mon[31172]: pgmap v289: 97 pgs: 97 active+clean; 591 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 7.1 KiB/s wr, 25 op/s; 0 B/s, 0 objects/s recovering 2024-09-11T23:16:44.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:43 smithi076 ceph-mon[35296]: pgmap v289: 97 pgs: 97 active+clean; 591 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 7.1 KiB/s wr, 25 op/s; 0 B/s, 0 objects/s recovering 2024-09-11T23:16:44.390 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:43 smithi136 ceph-mon[35121]: pgmap v289: 97 pgs: 97 active+clean; 591 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 7.1 KiB/s wr, 25 op/s; 0 B/s, 0 objects/s recovering 2024-09-11T23:16:44.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:44.539+0000 7f6d6403a640 1 -- 172.21.15.26:0/3694960642 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6d5c101280 msgr2=0x7f6d5c10df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:16:44.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:44.539+0000 7f6d6403a640 1 --2- 172.21.15.26:0/3694960642 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6d5c101280 0x7f6d5c10df40 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f6d4c009900 tx=0x7f6d4c02a410 comp rx=0 tx=0).stop 2024-09-11T23:16:44.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:44.539+0000 7f6d6403a640 1 -- 172.21.15.26:0/3694960642 shutdown_connections 2024-09-11T23:16:44.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:44.539+0000 7f6d6403a640 1 --2- 172.21.15.26:0/3694960642 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6d5c10e480 0x7f6d5c110870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:44.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:44.539+0000 7f6d6403a640 1 --2- 172.21.15.26:0/3694960642 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6d5c101280 0x7f6d5c10df40 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:44.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:44.539+0000 7f6d6403a640 1 --2- 172.21.15.26:0/3694960642 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6d5c1008d0 0x7f6d5c100cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:44.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:44.539+0000 7f6d6403a640 1 -- 172.21.15.26:0/3694960642 >> 172.21.15.26:0/3694960642 conn(0x7f6d5c0fc7a0 msgr2=0x7f6d5c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:16:44.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:44.539+0000 7f6d6403a640 1 -- 172.21.15.26:0/3694960642 shutdown_connections 2024-09-11T23:16:44.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:44.539+0000 7f6d6403a640 1 -- 172.21.15.26:0/3694960642 wait complete. 2024-09-11T23:16:44.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:44.540+0000 7f6d6403a640 1 Processor -- start 2024-09-11T23:16:44.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:44.540+0000 7f6d6403a640 1 -- start start 2024-09-11T23:16:44.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:44.540+0000 7f6d6403a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6d5c1008d0 0x7f6d5c19a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:16:44.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:44.540+0000 7f6d6403a640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6d5c101280 0x7f6d5c19ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:16:44.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:44.540+0000 7f6d6403a640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6d5c10e480 0x7f6d5c19ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:16:44.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:44.540+0000 7f6d6403a640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f6d5c114190 con 0x7f6d5c1008d0 2024-09-11T23:16:44.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:44.540+0000 7f6d6403a640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f6d5c114010 con 0x7f6d5c10e480 2024-09-11T23:16:44.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:44.540+0000 7f6d6403a640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f6d5c114310 con 0x7f6d5c101280 2024-09-11T23:16:44.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:44.540+0000 7f6d61dd6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6d5c1008d0 0x7f6d5c19a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:16:44.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:44.540+0000 7f6d61dd6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6d5c1008d0 0x7f6d5c19a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39740/0 (socket says 172.21.15.26:39740) 2024-09-11T23:16:44.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:44.540+0000 7f6d61dd6640 1 -- 172.21.15.26:0/3134785517 learned_addr learned my addr 172.21.15.26:0/3134785517 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:16:44.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:44.540+0000 7f6d625d7640 1 --2- 172.21.15.26:0/3134785517 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6d5c10e480 0x7f6d5c19ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:16:44.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:44.540+0000 7f6d615d5640 1 --2- 172.21.15.26:0/3134785517 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6d5c101280 0x7f6d5c19ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:16:44.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:44.541+0000 7f6d61dd6640 1 -- 172.21.15.26:0/3134785517 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6d5c101280 msgr2=0x7f6d5c19ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:16:44.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:44.541+0000 7f6d61dd6640 1 --2- 172.21.15.26:0/3134785517 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6d5c101280 0x7f6d5c19ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:44.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:44.541+0000 7f6d61dd6640 1 -- 172.21.15.26:0/3134785517 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6d5c10e480 msgr2=0x7f6d5c19ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:16:44.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:44.541+0000 7f6d61dd6640 1 --2- 172.21.15.26:0/3134785517 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6d5c10e480 0x7f6d5c19ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:44.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:44.541+0000 7f6d61dd6640 1 -- 172.21.15.26:0/3134785517 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d5c19f5e0 con 0x7f6d5c1008d0 2024-09-11T23:16:44.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:44.541+0000 7f6d61dd6640 1 --2- 172.21.15.26:0/3134785517 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6d5c1008d0 0x7f6d5c19a630 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f6d5800e9f0 tx=0x7f6d5800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:16:44.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:44.541+0000 7f6d52ffd640 1 -- 172.21.15.26:0/3134785517 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d5800cd80 con 0x7f6d5c1008d0 2024-09-11T23:16:44.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:44.541+0000 7f6d52ffd640 1 -- 172.21.15.26:0/3134785517 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d58010910 con 0x7f6d5c1008d0 2024-09-11T23:16:44.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:44.541+0000 7f6d6403a640 1 -- 172.21.15.26:0/3134785517 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d5c19f8d0 con 0x7f6d5c1008d0 2024-09-11T23:16:44.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:44.541+0000 7f6d52ffd640 1 -- 172.21.15.26:0/3134785517 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d58018990 con 0x7f6d5c1008d0 2024-09-11T23:16:44.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:44.541+0000 7f6d6403a640 1 -- 172.21.15.26:0/3134785517 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d5c1a71b0 con 0x7f6d5c1008d0 2024-09-11T23:16:44.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:44.542+0000 7f6d6403a640 1 -- 172.21.15.26:0/3134785517 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d5c100cb0 con 0x7f6d5c1008d0 2024-09-11T23:16:44.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:44.543+0000 7f6d52ffd640 1 -- 172.21.15.26:0/3134785517 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f6d58010ac0 con 0x7f6d5c1008d0 2024-09-11T23:16:44.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:44.544+0000 7f6d52ffd640 1 --2- 172.21.15.26:0/3134785517 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f6d480a2c90 0x7f6d480a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:16:44.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:44.544+0000 7f6d615d5640 1 --2- 172.21.15.26:0/3134785517 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f6d480a2c90 0x7f6d480a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:16:44.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:44.544+0000 7f6d52ffd640 1 -- 172.21.15.26:0/3134785517 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f6d580cf4e0 con 0x7f6d5c1008d0 2024-09-11T23:16:44.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:44.544+0000 7f6d615d5640 1 --2- 172.21.15.26:0/3134785517 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f6d480a2c90 0x7f6d480a5150 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f6d5c10ecb0 tx=0x7f6d4c005a20 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:16:44.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:44.548+0000 7f6d52ffd640 1 -- 172.21.15.26:0/3134785517 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6d5809c2e0 con 0x7f6d5c1008d0 2024-09-11T23:16:44.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:44.750+0000 7f6d6403a640 1 -- 172.21.15.26:0/3134785517 --> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f6d5c062630 con 0x7f6d480a2c90 2024-09-11T23:16:44.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:44.774+0000 7f6d52ffd640 1 -- 172.21.15.26:0/3134785517 <== mgr.14148 v2:172.21.15.26:6800/110833859 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f6d5c062630 con 0x7f6d480a2c90 2024-09-11T23:16:44.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:44.777+0000 7f6d6403a640 1 -- 172.21.15.26:0/3134785517 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f6d480a2c90 msgr2=0x7f6d480a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:16:44.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:44.777+0000 7f6d6403a640 1 --2- 172.21.15.26:0/3134785517 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f6d480a2c90 0x7f6d480a5150 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f6d5c10ecb0 tx=0x7f6d4c005a20 comp rx=0 tx=0).stop 2024-09-11T23:16:44.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:44.777+0000 7f6d6403a640 1 -- 172.21.15.26:0/3134785517 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6d5c1008d0 msgr2=0x7f6d5c19a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:16:44.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:44.777+0000 7f6d6403a640 1 --2- 172.21.15.26:0/3134785517 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6d5c1008d0 0x7f6d5c19a630 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f6d5800e9f0 tx=0x7f6d5800eeb0 comp rx=0 tx=0).stop 2024-09-11T23:16:44.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:44.777+0000 7f6d6403a640 1 -- 172.21.15.26:0/3134785517 shutdown_connections 2024-09-11T23:16:44.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:44.777+0000 7f6d6403a640 1 --2- 172.21.15.26:0/3134785517 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f6d480a2c90 0x7f6d480a5150 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:44.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:44.777+0000 7f6d6403a640 1 --2- 172.21.15.26:0/3134785517 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6d5c10e480 0x7f6d5c19ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:44.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:44.777+0000 7f6d6403a640 1 --2- 172.21.15.26:0/3134785517 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6d5c101280 0x7f6d5c19ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:44.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:44.777+0000 7f6d6403a640 1 --2- 172.21.15.26:0/3134785517 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6d5c1008d0 0x7f6d5c19a630 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:44.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:44.777+0000 7f6d6403a640 1 -- 172.21.15.26:0/3134785517 >> 172.21.15.26:0/3134785517 conn(0x7f6d5c0fc7a0 msgr2=0x7f6d5c0feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:16:44.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:44.777+0000 7f6d6403a640 1 -- 172.21.15.26:0/3134785517 shutdown_connections 2024-09-11T23:16:44.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:44.777+0000 7f6d6403a640 1 -- 172.21.15.26:0/3134785517 wait complete. 2024-09-11T23:16:45.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:45 smithi076 ceph-mon[35296]: from='client.14664 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:16:45.407 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:45 smithi136 ceph-mon[35121]: from='client.14664 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:16:45.490 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:45 smithi026 ceph-mon[31172]: from='client.14664 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:16:45.520 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-09-11T23:16:45.779 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:16:46.177 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:46 smithi026 ceph-mon[31172]: pgmap v290: 97 pgs: 97 active+clean; 591 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 5.8 KiB/s wr, 22 op/s; 0 B/s, 0 objects/s recovering 2024-09-11T23:16:46.177 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:46 smithi026 ceph-mon[31172]: from='client.14670 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:16:46.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:46.262+0000 7f57090b5640 1 -- 172.21.15.26:0/541781674 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f570410c170 msgr2=0x7f570410e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:16:46.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:46.262+0000 7f57090b5640 1 --2- 172.21.15.26:0/541781674 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f570410c170 0x7f570410e610 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f56fc009ca0 tx=0x7f56fc02a950 comp rx=0 tx=0).stop 2024-09-11T23:16:46.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:46.262+0000 7f57090b5640 1 -- 172.21.15.26:0/541781674 shutdown_connections 2024-09-11T23:16:46.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:46.262+0000 7f57090b5640 1 --2- 172.21.15.26:0/541781674 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5704103660 0x7f570410bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:46.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:46.262+0000 7f57090b5640 1 --2- 172.21.15.26:0/541781674 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f570410c170 0x7f570410e610 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:46.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:46.262+0000 7f57090b5640 1 --2- 172.21.15.26:0/541781674 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5704102d40 0x7f5704103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:46.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:46.262+0000 7f57090b5640 1 -- 172.21.15.26:0/541781674 >> 172.21.15.26:0/541781674 conn(0x7f57040fc7a0 msgr2=0x7f57040febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:16:46.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:46.262+0000 7f57090b5640 1 -- 172.21.15.26:0/541781674 shutdown_connections 2024-09-11T23:16:46.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:46.262+0000 7f57090b5640 1 -- 172.21.15.26:0/541781674 wait complete. 2024-09-11T23:16:46.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:46.263+0000 7f57090b5640 1 Processor -- start 2024-09-11T23:16:46.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:46.263+0000 7f57090b5640 1 -- start start 2024-09-11T23:16:46.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:46.264+0000 7f57090b5640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5704102d40 0x7f5704198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:16:46.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:46.264+0000 7f57090b5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5704103660 0x7f57041988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:16:46.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:46.264+0000 7f57090b5640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f570410c170 0x7f570419cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:16:46.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:46.264+0000 7f57090b5640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f5704111f70 con 0x7f5704103660 2024-09-11T23:16:46.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:46.264+0000 7f57090b5640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f5704111df0 con 0x7f570410c170 2024-09-11T23:16:46.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:46.264+0000 7f57090b5640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f57041120f0 con 0x7f5704102d40 2024-09-11T23:16:46.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:46.264+0000 7f570259c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5704103660 0x7f57041988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:16:46.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:46.264+0000 7f5702d9d640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5704102d40 0x7f5704198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:16:46.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:46.264+0000 7f570259c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5704103660 0x7f57041988b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39764/0 (socket says 172.21.15.26:39764) 2024-09-11T23:16:46.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:46.264+0000 7f570259c640 1 -- 172.21.15.26:0/4171734668 learned_addr learned my addr 172.21.15.26:0/4171734668 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:16:46.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:46.264+0000 7f570359e640 1 --2- 172.21.15.26:0/4171734668 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f570410c170 0x7f570419cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:16:46.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:46.264+0000 7f570259c640 1 -- 172.21.15.26:0/4171734668 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5704102d40 msgr2=0x7f5704198370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:16:46.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:46.264+0000 7f570259c640 1 --2- 172.21.15.26:0/4171734668 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5704102d40 0x7f5704198370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:46.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:46.264+0000 7f570259c640 1 -- 172.21.15.26:0/4171734668 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f570410c170 msgr2=0x7f570419cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:16:46.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:46.264+0000 7f570259c640 1 --2- 172.21.15.26:0/4171734668 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f570410c170 0x7f570419cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:46.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:46.264+0000 7f570259c640 1 -- 172.21.15.26:0/4171734668 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f570419d320 con 0x7f5704103660 2024-09-11T23:16:46.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:46.264+0000 7f5702d9d640 1 --2- 172.21.15.26:0/4171734668 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5704102d40 0x7f5704198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:16:46.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:46.264+0000 7f570259c640 1 --2- 172.21.15.26:0/4171734668 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5704103660 0x7f57041988b0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f56f800c970 tx=0x7f56f800cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:16:46.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:46.264+0000 7f56ebfff640 1 -- 172.21.15.26:0/4171734668 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56f8007bb0 con 0x7f5704103660 2024-09-11T23:16:46.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:46.264+0000 7f56ebfff640 1 -- 172.21.15.26:0/4171734668 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f56f8007d50 con 0x7f5704103660 2024-09-11T23:16:46.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:46.264+0000 7f57090b5640 1 -- 172.21.15.26:0/4171734668 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f570419d610 con 0x7f5704103660 2024-09-11T23:16:46.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:46.264+0000 7f56ebfff640 1 -- 172.21.15.26:0/4171734668 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56f8018730 con 0x7f5704103660 2024-09-11T23:16:46.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:46.265+0000 7f57090b5640 1 -- 172.21.15.26:0/4171734668 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57041a4ef0 con 0x7f5704103660 2024-09-11T23:16:46.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:46.266+0000 7f57090b5640 1 -- 172.21.15.26:0/4171734668 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5704103120 con 0x7f5704103660 2024-09-11T23:16:46.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:46.267+0000 7f56ebfff640 1 -- 172.21.15.26:0/4171734668 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f56f800f450 con 0x7f5704103660 2024-09-11T23:16:46.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:46.267+0000 7f56ebfff640 1 --2- 172.21.15.26:0/4171734668 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f56ec0a2c10 0x7f56ec0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:16:46.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:46.268+0000 7f5702d9d640 1 --2- 172.21.15.26:0/4171734668 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f56ec0a2c10 0x7f56ec0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:16:46.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:46.268+0000 7f56ebfff640 1 -- 172.21.15.26:0/4171734668 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f56f80cf130 con 0x7f5704103660 2024-09-11T23:16:46.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:46.268+0000 7f5702d9d640 1 --2- 172.21.15.26:0/4171734668 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f56ec0a2c10 0x7f56ec0a50d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f56f400afb0 tx=0x7f56f400a2d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:16:46.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:46.272+0000 7f56ebfff640 1 -- 172.21.15.26:0/4171734668 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f56f8014030 con 0x7f5704103660 2024-09-11T23:16:46.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:46 smithi076 ceph-mon[35296]: pgmap v290: 97 pgs: 97 active+clean; 591 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 5.8 KiB/s wr, 22 op/s; 0 B/s, 0 objects/s recovering 2024-09-11T23:16:46.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:46 smithi076 ceph-mon[35296]: from='client.14670 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:16:46.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:46.478+0000 7f57090b5640 1 -- 172.21.15.26:0/4171734668 --> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7f57040feac0 con 0x7f56ec0a2c10 2024-09-11T23:16:46.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:46.483+0000 7f56ebfff640 1 -- 172.21.15.26:0/4171734668 <== mgr.14148 v2:172.21.15.26:6800/110833859 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f57040feac0 con 0x7f56ec0a2c10 2024-09-11T23:16:46.483 INFO:teuthology.orchestra.run.smithi026.stdout:/volumes/qa/sv_1/9fc8f023-d646-4fec-98c7-1f3d972f60a9 2024-09-11T23:16:46.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:46.485+0000 7f57090b5640 1 -- 172.21.15.26:0/4171734668 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f56ec0a2c10 msgr2=0x7f56ec0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:16:46.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:46.485+0000 7f57090b5640 1 --2- 172.21.15.26:0/4171734668 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f56ec0a2c10 0x7f56ec0a50d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f56f400afb0 tx=0x7f56f400a2d0 comp rx=0 tx=0).stop 2024-09-11T23:16:46.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:46.485+0000 7f57090b5640 1 -- 172.21.15.26:0/4171734668 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5704103660 msgr2=0x7f57041988b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:16:46.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:46.485+0000 7f57090b5640 1 --2- 172.21.15.26:0/4171734668 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5704103660 0x7f57041988b0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f56f800c970 tx=0x7f56f800cd70 comp rx=0 tx=0).stop 2024-09-11T23:16:46.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:46.485+0000 7f57090b5640 1 -- 172.21.15.26:0/4171734668 shutdown_connections 2024-09-11T23:16:46.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:46.485+0000 7f57090b5640 1 --2- 172.21.15.26:0/4171734668 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f56ec0a2c10 0x7f56ec0a50d0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:46.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:46.485+0000 7f57090b5640 1 --2- 172.21.15.26:0/4171734668 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f570410c170 0x7f570419cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:46.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:46.485+0000 7f57090b5640 1 --2- 172.21.15.26:0/4171734668 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5704103660 0x7f57041988b0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:46.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:46.485+0000 7f57090b5640 1 --2- 172.21.15.26:0/4171734668 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5704102d40 0x7f5704198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:46.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:46.485+0000 7f57090b5640 1 -- 172.21.15.26:0/4171734668 >> 172.21.15.26:0/4171734668 conn(0x7f57040fc7a0 msgr2=0x7f57040fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:16:46.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:46.485+0000 7f57090b5640 1 -- 172.21.15.26:0/4171734668 shutdown_connections 2024-09-11T23:16:46.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:46.485+0000 7f57090b5640 1 -- 172.21.15.26:0/4171734668 wait complete. 2024-09-11T23:16:46.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:46 smithi136 ceph-mon[35121]: pgmap v290: 97 pgs: 97 active+clean; 591 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 5.8 KiB/s wr, 22 op/s; 0 B/s, 0 objects/s recovering 2024-09-11T23:16:46.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:46 smithi136 ceph-mon[35121]: from='client.14670 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:16:47.192 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:47 smithi026 ceph-mon[31172]: pgmap v291: 97 pgs: 97 active+clean; 623 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.9 KiB/s wr, 22 op/s; 0 B/s, 0 objects/s recovering 2024-09-11T23:16:47.192 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:47 smithi026 ceph-mon[31172]: from='client.14676 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:16:47.195 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:16:47.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:47 smithi076 ceph-mon[35296]: pgmap v291: 97 pgs: 97 active+clean; 623 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.9 KiB/s wr, 22 op/s; 0 B/s, 0 objects/s recovering 2024-09-11T23:16:47.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:47 smithi076 ceph-mon[35296]: from='client.14676 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:16:47.457 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:16:47.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:47 smithi136 ceph-mon[35121]: pgmap v291: 97 pgs: 97 active+clean; 623 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.9 KiB/s wr, 22 op/s; 0 B/s, 0 objects/s recovering 2024-09-11T23:16:47.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:47 smithi136 ceph-mon[35121]: from='client.14676 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:16:47.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:47.926+0000 7f4df94c7640 1 -- 172.21.15.26:0/3379873756 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4df4100d00 msgr2=0x7f4df4106840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:16:47.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:47.926+0000 7f4df94c7640 1 --2- 172.21.15.26:0/3379873756 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4df4100d00 0x7f4df4106840 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f4de8009930 tx=0x7f4de802a420 comp rx=0 tx=0).stop 2024-09-11T23:16:47.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:47.926+0000 7f4df94c7640 1 -- 172.21.15.26:0/3379873756 shutdown_connections 2024-09-11T23:16:47.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:47.926+0000 7f4df94c7640 1 --2- 172.21.15.26:0/3379873756 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4df4106f50 0x7f4df410eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:47.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:47.926+0000 7f4df94c7640 1 --2- 172.21.15.26:0/3379873756 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4df4100d00 0x7f4df4106840 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:47.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:47.926+0000 7f4df94c7640 1 --2- 172.21.15.26:0/3379873756 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4df4100350 0x7f4df4100730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:47.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:47.926+0000 7f4df94c7640 1 -- 172.21.15.26:0/3379873756 >> 172.21.15.26:0/3379873756 conn(0x7f4df4077210 msgr2=0x7f4df40fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:16:47.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:47.926+0000 7f4df94c7640 1 -- 172.21.15.26:0/3379873756 shutdown_connections 2024-09-11T23:16:47.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:47.926+0000 7f4df94c7640 1 -- 172.21.15.26:0/3379873756 wait complete. 2024-09-11T23:16:47.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:47.927+0000 7f4df94c7640 1 Processor -- start 2024-09-11T23:16:47.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:47.927+0000 7f4df94c7640 1 -- start start 2024-09-11T23:16:47.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:47.927+0000 7f4df94c7640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4df4100350 0x7f4df41012d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:16:47.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:47.927+0000 7f4df94c7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4df4100d00 0x7f4df4101810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:16:47.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:47.927+0000 7f4df94c7640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4df4106f50 0x7f4df4101d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:16:47.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:47.927+0000 7f4df94c7640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f4df4111ff0 con 0x7f4df4100d00 2024-09-11T23:16:47.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:47.927+0000 7f4df94c7640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f4df4111e70 con 0x7f4df4106f50 2024-09-11T23:16:47.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:47.927+0000 7f4df94c7640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f4df4112170 con 0x7f4df4100350 2024-09-11T23:16:47.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:47.927+0000 7f4df27fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4df4100d00 0x7f4df4101810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:16:47.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:47.927+0000 7f4df27fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4df4100d00 0x7f4df4101810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39784/0 (socket says 172.21.15.26:39784) 2024-09-11T23:16:47.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:47.927+0000 7f4df27fc640 1 -- 172.21.15.26:0/2344364655 learned_addr learned my addr 172.21.15.26:0/2344364655 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:16:47.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:47.927+0000 7f4df2ffd640 1 --2- 172.21.15.26:0/2344364655 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4df4100350 0x7f4df41012d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:16:47.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:47.927+0000 7f4df27fc640 1 -- 172.21.15.26:0/2344364655 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4df4100350 msgr2=0x7f4df41012d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:16:47.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:47.927+0000 7f4df37fe640 1 --2- 172.21.15.26:0/2344364655 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4df4106f50 0x7f4df4101d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:16:47.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:47.927+0000 7f4df27fc640 1 --2- 172.21.15.26:0/2344364655 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4df4100350 0x7f4df41012d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:47.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:47.927+0000 7f4df27fc640 1 -- 172.21.15.26:0/2344364655 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4df4106f50 msgr2=0x7f4df4101d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:16:47.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:47.927+0000 7f4df27fc640 1 --2- 172.21.15.26:0/2344364655 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4df4106f50 0x7f4df4101d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:47.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:47.927+0000 7f4df27fc640 1 -- 172.21.15.26:0/2344364655 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4df41a69d0 con 0x7f4df4100d00 2024-09-11T23:16:47.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:47.928+0000 7f4df27fc640 1 --2- 172.21.15.26:0/2344364655 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4df4100d00 0x7f4df4101810 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f4de802a940 tx=0x7f4de8003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:16:47.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:47.928+0000 7f4ddbfff640 1 -- 172.21.15.26:0/2344364655 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4de803f070 con 0x7f4df4100d00 2024-09-11T23:16:47.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:47.928+0000 7f4ddbfff640 1 -- 172.21.15.26:0/2344364655 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4de8031c90 con 0x7f4df4100d00 2024-09-11T23:16:47.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:47.928+0000 7f4df94c7640 1 -- 172.21.15.26:0/2344364655 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4df41a6c60 con 0x7f4df4100d00 2024-09-11T23:16:47.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:47.928+0000 7f4ddbfff640 1 -- 172.21.15.26:0/2344364655 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4de8035030 con 0x7f4df4100d00 2024-09-11T23:16:47.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:47.928+0000 7f4df94c7640 1 -- 172.21.15.26:0/2344364655 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4df41a7140 con 0x7f4df4100d00 2024-09-11T23:16:47.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:47.929+0000 7f4df94c7640 1 -- 172.21.15.26:0/2344364655 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4df4100730 con 0x7f4df4100d00 2024-09-11T23:16:47.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:47.931+0000 7f4ddbfff640 1 -- 172.21.15.26:0/2344364655 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f4de8042070 con 0x7f4df4100d00 2024-09-11T23:16:47.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:47.932+0000 7f4ddbfff640 1 --2- 172.21.15.26:0/2344364655 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f4ddc0a2c10 0x7f4ddc0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:16:47.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:47.932+0000 7f4df2ffd640 1 --2- 172.21.15.26:0/2344364655 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f4ddc0a2c10 0x7f4ddc0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:16:47.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:47.932+0000 7f4ddbfff640 1 -- 172.21.15.26:0/2344364655 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f4de80efc40 con 0x7f4df4100d00 2024-09-11T23:16:47.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:47.932+0000 7f4df2ffd640 1 --2- 172.21.15.26:0/2344364655 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f4ddc0a2c10 0x7f4ddc0a50d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f4de4005fd0 tx=0x7f4de4009680 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:16:47.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:47.935+0000 7f4ddbfff640 1 -- 172.21.15.26:0/2344364655 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4de80bca00 con 0x7f4df4100d00 2024-09-11T23:16:48.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:48.149+0000 7f4df94c7640 1 -- 172.21.15.26:0/2344364655 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4df4102800 con 0x7f4df4100d00 2024-09-11T23:16:48.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:48.150+0000 7f4ddbfff640 1 -- 172.21.15.26:0/2344364655 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11971 (secure 0 0 0) 0x7f4de80bc300 con 0x7f4df4100d00 2024-09-11T23:16:48.150 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-11T23:16:48.151 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":28,"btime":"2024-09-11T23:16:38:976987+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14499,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/367659515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":367659515},{"type":"v1","addr":"172.21.15.26:6835","nonce":367659515}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14511,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/2041065299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":2041065299},{"type":"v1","addr":"172.21.15.26:6837","nonce":2041065299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:16:26.544203+0000","modified":"2024-09-11T23:16:38.976970+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24485,"mds_1":24463,"mds_2":24458,"mds_3":24448,"mds_4":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14550":{"gid":14550,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6839/2233068305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":2233068305},{"type":"v1","addr":"172.21.15.26:6839","nonce":2233068305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14562":{"gid":14562,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6841/2984982336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":2984982336},{"type":"v1","addr":"172.21.15.26:6841","nonce":2984982336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6833/2019841455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":2019841455},{"type":"v1","addr":"172.21.15.136:6833","nonce":2019841455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.76:6833/3438147050","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3438147050},{"type":"v1","addr":"172.21.15.76:6833","nonce":3438147050}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6835/3617665975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3617665975},{"type":"v1","addr":"172.21.15.136:6835","nonce":3617665975}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":12,"addr":"172.21.15.76:6835/393308499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":393308499},{"type":"v1","addr":"172.21.15.76:6835","nonce":393308499}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":10,"addr":"172.21.15.136:6837/1633082387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":1633082387},{"type":"v1","addr":"172.21.15.136:6837","nonce":1633082387}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.76:6837/2078978748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":2078978748},{"type":"v1","addr":"172.21.15.76:6837","nonce":2078978748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.136:6839/3211398630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":3211398630},{"type":"v1","addr":"172.21.15.136:6839","nonce":3211398630}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.76:6839/2077637090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6838","nonce":2077637090},{"type":"v1","addr":"172.21.15.76:6839","nonce":2077637090}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24485,"qdb_cluster":[24485,24463,24458,24448,24443]},"id":1}]} 2024-09-11T23:16:48.151 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 28 2024-09-11T23:16:48.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:48.152+0000 7f4df94c7640 1 -- 172.21.15.26:0/2344364655 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f4ddc0a2c10 msgr2=0x7f4ddc0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:16:48.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:48.152+0000 7f4df94c7640 1 --2- 172.21.15.26:0/2344364655 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f4ddc0a2c10 0x7f4ddc0a50d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f4de4005fd0 tx=0x7f4de4009680 comp rx=0 tx=0).stop 2024-09-11T23:16:48.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:48.152+0000 7f4df94c7640 1 -- 172.21.15.26:0/2344364655 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4df4100d00 msgr2=0x7f4df4101810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:16:48.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:48.152+0000 7f4df94c7640 1 --2- 172.21.15.26:0/2344364655 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4df4100d00 0x7f4df4101810 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f4de802a940 tx=0x7f4de8003b90 comp rx=0 tx=0).stop 2024-09-11T23:16:48.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:48.152+0000 7f4df94c7640 1 -- 172.21.15.26:0/2344364655 shutdown_connections 2024-09-11T23:16:48.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:48.152+0000 7f4df94c7640 1 --2- 172.21.15.26:0/2344364655 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f4ddc0a2c10 0x7f4ddc0a50d0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:48.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:48.152+0000 7f4df94c7640 1 --2- 172.21.15.26:0/2344364655 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4df4106f50 0x7f4df4101d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:48.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:48.152+0000 7f4df94c7640 1 --2- 172.21.15.26:0/2344364655 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4df4100d00 0x7f4df4101810 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:48.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:48.152+0000 7f4df94c7640 1 --2- 172.21.15.26:0/2344364655 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4df4100350 0x7f4df41012d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:48.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:48.152+0000 7f4df94c7640 1 -- 172.21.15.26:0/2344364655 >> 172.21.15.26:0/2344364655 conn(0x7f4df4077210 msgr2=0x7f4df40fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:16:48.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:48.152+0000 7f4df94c7640 1 -- 172.21.15.26:0/2344364655 shutdown_connections 2024-09-11T23:16:48.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:48.153+0000 7f4df94c7640 1 -- 172.21.15.26:0/2344364655 wait complete. 2024-09-11T23:16:48.940 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-11T23:16:49.204 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:16:49.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:49 smithi026 ceph-mon[31172]: pgmap v292: 97 pgs: 97 active+clean; 623 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 6.7 KiB/s wr, 18 op/s; 0 B/s, 0 objects/s recovering 2024-09-11T23:16:49.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:49 smithi026 ceph-mon[31172]: from='client.? 172.21.15.26:0/2344364655' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:16:49.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:49 smithi136 ceph-mon[35121]: pgmap v292: 97 pgs: 97 active+clean; 623 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 6.7 KiB/s wr, 18 op/s; 0 B/s, 0 objects/s recovering 2024-09-11T23:16:49.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:49 smithi136 ceph-mon[35121]: from='client.? 172.21.15.26:0/2344364655' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:16:49.553 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:49 smithi076 ceph-mon[35296]: pgmap v292: 97 pgs: 97 active+clean; 623 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 6.7 KiB/s wr, 18 op/s; 0 B/s, 0 objects/s recovering 2024-09-11T23:16:49.553 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:49 smithi076 ceph-mon[35296]: from='client.? 172.21.15.26:0/2344364655' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:16:49.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:49.715+0000 7f47d63e0640 1 -- 172.21.15.26:0/3443377800 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47d010c170 msgr2=0x7f47d010e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:16:49.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:49.715+0000 7f47d63e0640 1 --2- 172.21.15.26:0/3443377800 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47d010c170 0x7f47d010e610 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f47c8009ca0 tx=0x7f47c802a970 comp rx=0 tx=0).stop 2024-09-11T23:16:49.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:49.715+0000 7f47d63e0640 1 -- 172.21.15.26:0/3443377800 shutdown_connections 2024-09-11T23:16:49.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:49.715+0000 7f47d63e0640 1 --2- 172.21.15.26:0/3443377800 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f47d0103660 0x7f47d010bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:49.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:49.715+0000 7f47d63e0640 1 --2- 172.21.15.26:0/3443377800 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47d010c170 0x7f47d010e610 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:49.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:49.715+0000 7f47d63e0640 1 --2- 172.21.15.26:0/3443377800 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f47d0102d40 0x7f47d0103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:49.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:49.715+0000 7f47d63e0640 1 -- 172.21.15.26:0/3443377800 >> 172.21.15.26:0/3443377800 conn(0x7f47d00fc7a0 msgr2=0x7f47d00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:16:49.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:49.716+0000 7f47d63e0640 1 -- 172.21.15.26:0/3443377800 shutdown_connections 2024-09-11T23:16:49.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:49.716+0000 7f47d63e0640 1 -- 172.21.15.26:0/3443377800 wait complete. 2024-09-11T23:16:49.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:49.716+0000 7f47d63e0640 1 Processor -- start 2024-09-11T23:16:49.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:49.716+0000 7f47d63e0640 1 -- start start 2024-09-11T23:16:49.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:49.716+0000 7f47d63e0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47d0102d40 0x7f47d01984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:16:49.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:49.716+0000 7f47d63e0640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f47d0103660 0x7f47d0198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:16:49.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:49.717+0000 7f47d63e0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f47d010c170 0x7f47d019cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:16:49.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:49.717+0000 7f47d63e0640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f47d0111fd0 con 0x7f47d0102d40 2024-09-11T23:16:49.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:49.717+0000 7f47d63e0640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f47d0111e50 con 0x7f47d010c170 2024-09-11T23:16:49.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:49.717+0000 7f47d63e0640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f47d0112150 con 0x7f47d0103660 2024-09-11T23:16:49.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:49.717+0000 7f47cffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47d0102d40 0x7f47d01984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:16:49.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:49.717+0000 7f47cffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47d0102d40 0x7f47d01984d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39802/0 (socket says 172.21.15.26:39802) 2024-09-11T23:16:49.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:49.717+0000 7f47cffff640 1 -- 172.21.15.26:0/3595198696 learned_addr learned my addr 172.21.15.26:0/3595198696 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:16:49.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:49.717+0000 7f47d497d640 1 --2- 172.21.15.26:0/3595198696 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f47d010c170 0x7f47d019cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:16:49.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:49.717+0000 7f47cffff640 1 -- 172.21.15.26:0/3595198696 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f47d0103660 msgr2=0x7f47d0198a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:16:49.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:49.717+0000 7f47cffff640 1 --2- 172.21.15.26:0/3595198696 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f47d0103660 0x7f47d0198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:49.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:49.717+0000 7f47cffff640 1 -- 172.21.15.26:0/3595198696 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f47d010c170 msgr2=0x7f47d019cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:16:49.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:49.717+0000 7f47cffff640 1 --2- 172.21.15.26:0/3595198696 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f47d010c170 0x7f47d019cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:49.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:49.717+0000 7f47cffff640 1 -- 172.21.15.26:0/3595198696 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47d019d480 con 0x7f47d0102d40 2024-09-11T23:16:49.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:49.717+0000 7f47d497d640 1 --2- 172.21.15.26:0/3595198696 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f47d010c170 0x7f47d019cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:16:49.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:49.717+0000 7f47cffff640 1 --2- 172.21.15.26:0/3595198696 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47d0102d40 0x7f47d01984d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f47c000b450 tx=0x7f47c000b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:16:49.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:49.717+0000 7f47cd7fa640 1 -- 172.21.15.26:0/3595198696 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47c0015020 con 0x7f47d0102d40 2024-09-11T23:16:49.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:49.717+0000 7f47cd7fa640 1 -- 172.21.15.26:0/3595198696 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47c0011d70 con 0x7f47d0102d40 2024-09-11T23:16:49.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:49.717+0000 7f47d63e0640 1 -- 172.21.15.26:0/3595198696 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47d019d770 con 0x7f47d0102d40 2024-09-11T23:16:49.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:49.717+0000 7f47cd7fa640 1 -- 172.21.15.26:0/3595198696 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47c000f340 con 0x7f47d0102d40 2024-09-11T23:16:49.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:49.717+0000 7f47d63e0640 1 -- 172.21.15.26:0/3595198696 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47d01a5050 con 0x7f47d0102d40 2024-09-11T23:16:49.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:49.718+0000 7f47d63e0640 1 -- 172.21.15.26:0/3595198696 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47d0103120 con 0x7f47d0102d40 2024-09-11T23:16:49.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:49.720+0000 7f47cd7fa640 1 -- 172.21.15.26:0/3595198696 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f47c0018020 con 0x7f47d0102d40 2024-09-11T23:16:49.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:49.721+0000 7f47cd7fa640 1 --2- 172.21.15.26:0/3595198696 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f47b80a2c10 0x7f47b80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:16:49.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:49.721+0000 7f47cf7fe640 1 --2- 172.21.15.26:0/3595198696 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f47b80a2c10 0x7f47b80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:16:49.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:49.721+0000 7f47cd7fa640 1 -- 172.21.15.26:0/3595198696 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f47c00cf120 con 0x7f47d0102d40 2024-09-11T23:16:49.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:49.721+0000 7f47cf7fe640 1 --2- 172.21.15.26:0/3595198696 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f47b80a2c10 0x7f47b80a50d0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f47d010d4b0 tx=0x7f47c4005c90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:16:49.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:49.724+0000 7f47cd7fa640 1 -- 172.21.15.26:0/3595198696 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f47c001f020 con 0x7f47d0102d40 2024-09-11T23:16:49.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:49.902+0000 7f47d63e0640 1 -- 172.21.15.26:0/3595198696 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f47d0107240 con 0x7f47d0102d40 2024-09-11T23:16:49.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:49.905+0000 7f47cd7fa640 1 -- 172.21.15.26:0/3595198696 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+22733 (secure 0 0 0) 0x7f47c000d040 con 0x7f47d0102d40 2024-09-11T23:16:49.905 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-11T23:16:49.906 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":85,"fsid":"af7e6fc8-7092-11ef-bcea-c7b262605968","created":"2024-09-11T23:08:40.486438+0000","modified":"2024-09-11T23:16:35.935151+0000","last_up_change":"2024-09-11T23:15:35.179446+0000","last_in_change":"2024-09-11T23:15:11.064627+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T23:11:50.494160+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-11T23:16:22.503928+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-11T23:16:24.876554+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a4d62fcf-c31e-41b1-932c-3d87ff640531","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":2014857905},{"type":"v1","addr":"172.21.15.26:6803","nonce":2014857905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":2014857905},{"type":"v1","addr":"172.21.15.26:6805","nonce":2014857905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":2014857905},{"type":"v1","addr":"172.21.15.26:6809","nonce":2014857905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":2014857905},{"type":"v1","addr":"172.21.15.26:6807","nonce":2014857905}]},"public_addr":"172.21.15.26:6803/2014857905","cluster_addr":"172.21.15.26:6805/2014857905","heartbeat_back_addr":"172.21.15.26:6809/2014857905","heartbeat_front_addr":"172.21.15.26:6807/2014857905","state":["exists","up"]},{"osd":1,"uuid":"e0d69112-3c0a-4fc5-8783-e41b4081e0c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":333383087},{"type":"v1","addr":"172.21.15.76:6801","nonce":333383087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":333383087},{"type":"v1","addr":"172.21.15.76:6803","nonce":333383087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":333383087},{"type":"v1","addr":"172.21.15.76:6807","nonce":333383087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":333383087},{"type":"v1","addr":"172.21.15.76:6805","nonce":333383087}]},"public_addr":"172.21.15.76:6801/333383087","cluster_addr":"172.21.15.76:6803/333383087","heartbeat_back_addr":"172.21.15.76:6807/333383087","heartbeat_front_addr":"172.21.15.76:6805/333383087","state":["exists","up"]},{"osd":2,"uuid":"d578429e-bf23-4a1e-890f-7172955c0eb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":1271538911},{"type":"v1","addr":"172.21.15.136:6801","nonce":1271538911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":1271538911},{"type":"v1","addr":"172.21.15.136:6803","nonce":1271538911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":1271538911},{"type":"v1","addr":"172.21.15.136:6807","nonce":1271538911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":1271538911},{"type":"v1","addr":"172.21.15.136:6805","nonce":1271538911}]},"public_addr":"172.21.15.136:6801/1271538911","cluster_addr":"172.21.15.136:6803/1271538911","heartbeat_back_addr":"172.21.15.136:6807/1271538911","heartbeat_front_addr":"172.21.15.136:6805/1271538911","state":["exists","up"]},{"osd":3,"uuid":"3e34623f-0ea5-4b10-b8a9-f8756c35d512","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":1740304151},{"type":"v1","addr":"172.21.15.26:6811","nonce":1740304151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":1740304151},{"type":"v1","addr":"172.21.15.26:6813","nonce":1740304151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1740304151},{"type":"v1","addr":"172.21.15.26:6817","nonce":1740304151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":1740304151},{"type":"v1","addr":"172.21.15.26:6815","nonce":1740304151}]},"public_addr":"172.21.15.26:6811/1740304151","cluster_addr":"172.21.15.26:6813/1740304151","heartbeat_back_addr":"172.21.15.26:6817/1740304151","heartbeat_front_addr":"172.21.15.26:6815/1740304151","state":["exists","up"]},{"osd":4,"uuid":"77607929-f9ed-49a9-abb5-48cd88eec4ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":2702262182},{"type":"v1","addr":"172.21.15.76:6809","nonce":2702262182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":2702262182},{"type":"v1","addr":"172.21.15.76:6811","nonce":2702262182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":2702262182},{"type":"v1","addr":"172.21.15.76:6815","nonce":2702262182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":2702262182},{"type":"v1","addr":"172.21.15.76:6813","nonce":2702262182}]},"public_addr":"172.21.15.76:6809/2702262182","cluster_addr":"172.21.15.76:6811/2702262182","heartbeat_back_addr":"172.21.15.76:6815/2702262182","heartbeat_front_addr":"172.21.15.76:6813/2702262182","state":["exists","up"]},{"osd":5,"uuid":"29e20e37-09a0-4924-8aaa-17939763a7c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":1749229277},{"type":"v1","addr":"172.21.15.136:6809","nonce":1749229277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":1749229277},{"type":"v1","addr":"172.21.15.136:6811","nonce":1749229277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":1749229277},{"type":"v1","addr":"172.21.15.136:6815","nonce":1749229277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":1749229277},{"type":"v1","addr":"172.21.15.136:6813","nonce":1749229277}]},"public_addr":"172.21.15.136:6809/1749229277","cluster_addr":"172.21.15.136:6811/1749229277","heartbeat_back_addr":"172.21.15.136:6815/1749229277","heartbeat_front_addr":"172.21.15.136:6813/1749229277","state":["exists","up"]},{"osd":6,"uuid":"e0ddd1c0-1c46-41e6-b2f4-396bdcd2bf8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1928245624},{"type":"v1","addr":"172.21.15.26:6819","nonce":1928245624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1928245624},{"type":"v1","addr":"172.21.15.26:6821","nonce":1928245624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":1928245624},{"type":"v1","addr":"172.21.15.26:6825","nonce":1928245624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1928245624},{"type":"v1","addr":"172.21.15.26:6823","nonce":1928245624}]},"public_addr":"172.21.15.26:6819/1928245624","cluster_addr":"172.21.15.26:6821/1928245624","heartbeat_back_addr":"172.21.15.26:6825/1928245624","heartbeat_front_addr":"172.21.15.26:6823/1928245624","state":["exists","up"]},{"osd":7,"uuid":"fe031f5c-d915-48fb-91a6-c76f252467fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":1409789455},{"type":"v1","addr":"172.21.15.76:6817","nonce":1409789455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":1409789455},{"type":"v1","addr":"172.21.15.76:6819","nonce":1409789455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":1409789455},{"type":"v1","addr":"172.21.15.76:6823","nonce":1409789455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":1409789455},{"type":"v1","addr":"172.21.15.76:6821","nonce":1409789455}]},"public_addr":"172.21.15.76:6817/1409789455","cluster_addr":"172.21.15.76:6819/1409789455","heartbeat_back_addr":"172.21.15.76:6823/1409789455","heartbeat_front_addr":"172.21.15.76:6821/1409789455","state":["exists","up"]},{"osd":8,"uuid":"18851bde-9e6c-4e05-b4bf-ac5c3073eee7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":543619079},{"type":"v1","addr":"172.21.15.136:6817","nonce":543619079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":543619079},{"type":"v1","addr":"172.21.15.136:6819","nonce":543619079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":543619079},{"type":"v1","addr":"172.21.15.136:6823","nonce":543619079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":543619079},{"type":"v1","addr":"172.21.15.136:6821","nonce":543619079}]},"public_addr":"172.21.15.136:6817/543619079","cluster_addr":"172.21.15.136:6819/543619079","heartbeat_back_addr":"172.21.15.136:6823/543619079","heartbeat_front_addr":"172.21.15.136:6821/543619079","state":["exists","up"]},{"osd":9,"uuid":"52a65eab-8446-47ad-bfe7-5133f7e3eb98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":2201297064},{"type":"v1","addr":"172.21.15.26:6827","nonce":2201297064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":2201297064},{"type":"v1","addr":"172.21.15.26:6829","nonce":2201297064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":2201297064},{"type":"v1","addr":"172.21.15.26:6833","nonce":2201297064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":2201297064},{"type":"v1","addr":"172.21.15.26:6831","nonce":2201297064}]},"public_addr":"172.21.15.26:6827/2201297064","cluster_addr":"172.21.15.26:6829/2201297064","heartbeat_back_addr":"172.21.15.26:6833/2201297064","heartbeat_front_addr":"172.21.15.26:6831/2201297064","state":["exists","up"]},{"osd":10,"uuid":"b758d4bb-da62-4c5f-a992-bb6a145d35d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":3184128096},{"type":"v1","addr":"172.21.15.76:6825","nonce":3184128096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":3184128096},{"type":"v1","addr":"172.21.15.76:6827","nonce":3184128096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":3184128096},{"type":"v1","addr":"172.21.15.76:6831","nonce":3184128096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":3184128096},{"type":"v1","addr":"172.21.15.76:6829","nonce":3184128096}]},"public_addr":"172.21.15.76:6825/3184128096","cluster_addr":"172.21.15.76:6827/3184128096","heartbeat_back_addr":"172.21.15.76:6831/3184128096","heartbeat_front_addr":"172.21.15.76:6829/3184128096","state":["exists","up"]},{"osd":11,"uuid":"8fddda46-e22b-4d0c-ac48-e038fb29dfec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":1960689685},{"type":"v1","addr":"172.21.15.136:6825","nonce":1960689685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":1960689685},{"type":"v1","addr":"172.21.15.136:6827","nonce":1960689685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":1960689685},{"type":"v1","addr":"172.21.15.136:6831","nonce":1960689685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":1960689685},{"type":"v1","addr":"172.21.15.136:6829","nonce":1960689685}]},"public_addr":"172.21.15.136:6825/1960689685","cluster_addr":"172.21.15.136:6827/1960689685","heartbeat_back_addr":"172.21.15.136:6831/1960689685","heartbeat_front_addr":"172.21.15.136:6829/1960689685","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:10:56.656177+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:11:22.234031+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:11:46.464102+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:12:12.353967+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:12:37.836200+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:13:01.728839+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:13:27.527230+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:13:51.981393+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:14:17.035464+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:14:42.805029+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:15:08.208999+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:15:32.800844+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.18","mappings":[{"from":3,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":6}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.26:6801/677148151":"2024-09-12T23:09:33.983310+0000","172.21.15.26:6800/677148151":"2024-09-12T23:09:33.983310+0000","172.21.15.26:0/3597996003":"2024-09-12T23:09:33.983310+0000","172.21.15.26:6801/2673575521":"2024-09-12T23:09:09.781665+0000","172.21.15.26:0/883212708":"2024-09-12T23:09:33.983310+0000","172.21.15.26:0/2107281413":"2024-09-12T23:09:33.983310+0000","172.21.15.26:6800/2673575521":"2024-09-12T23:09:09.781665+0000","172.21.15.26:0/2817092653":"2024-09-12T23:09:09.781665+0000","172.21.15.26:0/17142658":"2024-09-12T23:09:09.781665+0000","172.21.15.26:0/3861103130":"2024-09-12T23:09:09.781665+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T23:16:49.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:49.907+0000 7f47d63e0640 1 -- 172.21.15.26:0/3595198696 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f47b80a2c10 msgr2=0x7f47b80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:16:49.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:49.907+0000 7f47d63e0640 1 --2- 172.21.15.26:0/3595198696 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f47b80a2c10 0x7f47b80a50d0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f47d010d4b0 tx=0x7f47c4005c90 comp rx=0 tx=0).stop 2024-09-11T23:16:49.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:49.907+0000 7f47d63e0640 1 -- 172.21.15.26:0/3595198696 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47d0102d40 msgr2=0x7f47d01984d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:16:49.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:49.907+0000 7f47d63e0640 1 --2- 172.21.15.26:0/3595198696 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47d0102d40 0x7f47d01984d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f47c000b450 tx=0x7f47c000b910 comp rx=0 tx=0).stop 2024-09-11T23:16:49.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:49.908+0000 7f47d63e0640 1 -- 172.21.15.26:0/3595198696 shutdown_connections 2024-09-11T23:16:49.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:49.908+0000 7f47d63e0640 1 --2- 172.21.15.26:0/3595198696 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f47b80a2c10 0x7f47b80a50d0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:49.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:49.908+0000 7f47d63e0640 1 --2- 172.21.15.26:0/3595198696 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f47d010c170 0x7f47d019cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:49.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:49.908+0000 7f47d63e0640 1 --2- 172.21.15.26:0/3595198696 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f47d0103660 0x7f47d0198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:49.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:49.908+0000 7f47d63e0640 1 --2- 172.21.15.26:0/3595198696 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47d0102d40 0x7f47d01984d0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:49.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:49.908+0000 7f47d63e0640 1 -- 172.21.15.26:0/3595198696 >> 172.21.15.26:0/3595198696 conn(0x7f47d00fc7a0 msgr2=0x7f47d00fd400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:16:49.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:49.908+0000 7f47d63e0640 1 -- 172.21.15.26:0/3595198696 shutdown_connections 2024-09-11T23:16:49.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:49.908+0000 7f47d63e0640 1 -- 172.21.15.26:0/3595198696 wait complete. 2024-09-11T23:16:50.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:50 smithi026 ceph-mon[31172]: from='client.? 172.21.15.26:0/3595198696' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T23:16:50.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:50 smithi136 ceph-mon[35121]: from='client.? 172.21.15.26:0/3595198696' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T23:16:50.553 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:50 smithi076 ceph-mon[35296]: from='client.? 172.21.15.26:0/3595198696' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T23:16:50.623 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-11T23:16:50.623 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-11T23:16:50.887 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:16:51.260 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:51 smithi026 ceph-mon[31172]: pgmap v293: 97 pgs: 97 active+clean; 624 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 6.7 KiB/s wr, 20 op/s; 0 B/s, 0 objects/s recovering 2024-09-11T23:16:51.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:51.338+0000 7f0668461640 1 -- 172.21.15.26:0/2373999 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0660101de0 msgr2=0x7f06601021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:16:51.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:51.338+0000 7f0668461640 1 --2- 172.21.15.26:0/2373999 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0660101de0 0x7f06601021c0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f065c009900 tx=0x7f065c02a410 comp rx=0 tx=0).stop 2024-09-11T23:16:51.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:51.338+0000 7f0668461640 1 -- 172.21.15.26:0/2373999 shutdown_connections 2024-09-11T23:16:51.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:51.338+0000 7f0668461640 1 --2- 172.21.15.26:0/2373999 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0660102790 0x7f06601067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:51.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:51.338+0000 7f0668461640 1 --2- 172.21.15.26:0/2373999 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0660106f10 0x7f066010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:51.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:51.338+0000 7f0668461640 1 --2- 172.21.15.26:0/2373999 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0660101de0 0x7f06601021c0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:51.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:51.338+0000 7f0668461640 1 -- 172.21.15.26:0/2373999 >> 172.21.15.26:0/2373999 conn(0x7f06600fd8c0 msgr2=0x7f06600ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:16:51.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:51.338+0000 7f0668461640 1 -- 172.21.15.26:0/2373999 shutdown_connections 2024-09-11T23:16:51.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:51.338+0000 7f0668461640 1 -- 172.21.15.26:0/2373999 wait complete. 2024-09-11T23:16:51.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:51.339+0000 7f0668461640 1 Processor -- start 2024-09-11T23:16:51.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:51.339+0000 7f0668461640 1 -- start start 2024-09-11T23:16:51.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:51.339+0000 7f0668461640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0660101de0 0x7f0660198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:16:51.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:51.339+0000 7f0668461640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0660102790 0x7f06601988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:16:51.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:51.339+0000 7f0668461640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0660106f10 0x7f066019cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:16:51.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:51.339+0000 7f0668461640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f0660111f70 con 0x7f0660101de0 2024-09-11T23:16:51.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:51.339+0000 7f0668461640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f0660111df0 con 0x7f0660106f10 2024-09-11T23:16:51.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:51.339+0000 7f0668461640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f06601120f0 con 0x7f0660102790 2024-09-11T23:16:51.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:51.339+0000 7f06661fd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0660101de0 0x7f0660198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:16:51.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:51.339+0000 7f06661fd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0660101de0 0x7f0660198370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42084/0 (socket says 172.21.15.26:42084) 2024-09-11T23:16:51.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:51.339+0000 7f06661fd640 1 -- 172.21.15.26:0/1076445674 learned_addr learned my addr 172.21.15.26:0/1076445674 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:16:51.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:51.339+0000 7f06659fc640 1 --2- 172.21.15.26:0/1076445674 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0660102790 0x7f06601988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:16:51.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:51.340+0000 7f06669fe640 1 --2- 172.21.15.26:0/1076445674 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0660106f10 0x7f066019cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:16:51.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:51.340+0000 7f06661fd640 1 -- 172.21.15.26:0/1076445674 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0660102790 msgr2=0x7f06601988b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:16:51.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:51.340+0000 7f06661fd640 1 --2- 172.21.15.26:0/1076445674 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0660102790 0x7f06601988b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:51.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:51.340+0000 7f06661fd640 1 -- 172.21.15.26:0/1076445674 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0660106f10 msgr2=0x7f066019cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:16:51.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:51.340+0000 7f06661fd640 1 --2- 172.21.15.26:0/1076445674 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0660106f10 0x7f066019cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:51.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:51.340+0000 7f06661fd640 1 -- 172.21.15.26:0/1076445674 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f066019d320 con 0x7f0660101de0 2024-09-11T23:16:51.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:51.340+0000 7f06659fc640 1 --2- 172.21.15.26:0/1076445674 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0660102790 0x7f06601988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:16:51.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:51.340+0000 7f06669fe640 1 --2- 172.21.15.26:0/1076445674 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0660106f10 0x7f066019cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:16:51.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:51.340+0000 7f06661fd640 1 --2- 172.21.15.26:0/1076445674 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0660101de0 0x7f0660198370 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f065c006010 tx=0x7f065c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:16:51.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:51.340+0000 7f06537fe640 1 -- 172.21.15.26:0/1076445674 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f065c03f030 con 0x7f0660101de0 2024-09-11T23:16:51.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:51.340+0000 7f0668461640 1 -- 172.21.15.26:0/1076445674 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f066019d5b0 con 0x7f0660101de0 2024-09-11T23:16:51.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:51.340+0000 7f06537fe640 1 -- 172.21.15.26:0/1076445674 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f065c003ce0 con 0x7f0660101de0 2024-09-11T23:16:51.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:51.340+0000 7f06537fe640 1 -- 172.21.15.26:0/1076445674 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f065c035070 con 0x7f0660101de0 2024-09-11T23:16:51.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:51.340+0000 7f0668461640 1 -- 172.21.15.26:0/1076445674 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06601a4ef0 con 0x7f0660101de0 2024-09-11T23:16:51.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:51.341+0000 7f0668461640 1 -- 172.21.15.26:0/1076445674 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06601021c0 con 0x7f0660101de0 2024-09-11T23:16:51.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:51.342+0000 7f06537fe640 1 -- 172.21.15.26:0/1076445674 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f065c042030 con 0x7f0660101de0 2024-09-11T23:16:51.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:51.343+0000 7f06537fe640 1 --2- 172.21.15.26:0/1076445674 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f064c0a2c10 0x7f064c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:16:51.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:51.343+0000 7f06659fc640 1 --2- 172.21.15.26:0/1076445674 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f064c0a2c10 0x7f064c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:16:51.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:51.343+0000 7f06537fe640 1 -- 172.21.15.26:0/1076445674 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f065c0ef790 con 0x7f0660101de0 2024-09-11T23:16:51.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:51.343+0000 7f06659fc640 1 --2- 172.21.15.26:0/1076445674 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f064c0a2c10 0x7f064c0a50d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f06601088e0 tx=0x7f065400a5c0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:16:51.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:51.347+0000 7f06537fe640 1 -- 172.21.15.26:0/1076445674 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f065c0bc650 con 0x7f0660101de0 2024-09-11T23:16:51.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:51 smithi136 ceph-mon[35121]: pgmap v293: 97 pgs: 97 active+clean; 624 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 6.7 KiB/s wr, 20 op/s; 0 B/s, 0 objects/s recovering 2024-09-11T23:16:51.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:51.520+0000 7f0668461640 1 -- 172.21.15.26:0/1076445674 --> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0660062630 con 0x7f064c0a2c10 2024-09-11T23:16:51.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:51.527+0000 7f06537fe640 1 -- 172.21.15.26:0/1076445674 <== mgr.14148 v2:172.21.15.26:6800/110833859 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258750 (secure 0 0 0) 0x7f0660062630 con 0x7f064c0a2c10 2024-09-11T23:16:51.533 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-11T23:16:51.536 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-09-11T23:16:51.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:51.529+0000 7f0668461640 1 -- 172.21.15.26:0/1076445674 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f064c0a2c10 msgr2=0x7f064c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:16:51.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:51.529+0000 7f0668461640 1 --2- 172.21.15.26:0/1076445674 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f064c0a2c10 0x7f064c0a50d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f06601088e0 tx=0x7f065400a5c0 comp rx=0 tx=0).stop 2024-09-11T23:16:51.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:51.530+0000 7f0668461640 1 -- 172.21.15.26:0/1076445674 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0660101de0 msgr2=0x7f0660198370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:16:51.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:51.530+0000 7f0668461640 1 --2- 172.21.15.26:0/1076445674 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0660101de0 0x7f0660198370 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f065c006010 tx=0x7f065c0042f0 comp rx=0 tx=0).stop 2024-09-11T23:16:51.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:51.530+0000 7f0668461640 1 -- 172.21.15.26:0/1076445674 shutdown_connections 2024-09-11T23:16:51.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:51.530+0000 7f0668461640 1 --2- 172.21.15.26:0/1076445674 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f064c0a2c10 0x7f064c0a50d0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:51.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:51.530+0000 7f0668461640 1 --2- 172.21.15.26:0/1076445674 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0660106f10 0x7f066019cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:51.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:51.530+0000 7f0668461640 1 --2- 172.21.15.26:0/1076445674 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0660102790 0x7f06601988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:51.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:51.530+0000 7f0668461640 1 --2- 172.21.15.26:0/1076445674 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0660101de0 0x7f0660198370 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:51.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:51.530+0000 7f0668461640 1 -- 172.21.15.26:0/1076445674 >> 172.21.15.26:0/1076445674 conn(0x7f06600fd8c0 msgr2=0x7f06600fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:16:51.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:51.530+0000 7f0668461640 1 -- 172.21.15.26:0/1076445674 shutdown_connections 2024-09-11T23:16:51.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:51.530+0000 7f0668461640 1 -- 172.21.15.26:0/1076445674 wait complete. 2024-09-11T23:16:51.553 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:51 smithi076 ceph-mon[35296]: pgmap v293: 97 pgs: 97 active+clean; 624 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 6.7 KiB/s wr, 20 op/s; 0 B/s, 0 objects/s recovering 2024-09-11T23:16:52.253 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:52 smithi026 ceph-mon[31172]: from='client.14694 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T23:16:52.284 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":293,"stamp":"2024-09-11T23:16:50.084992+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":638482,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":256,"num_read_kb":231,"num_write":407,"num_write_kb":3048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":334,"ondisk_log_size":334,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":379060,"kb_used_data":13168,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124562764,"statfs":{"total":1151940427776,"available":1151552270336,"internally_reserved":0,"allocated":13484032,"data_stored":7300256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":374453561},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":43586,"num_objects":52,"num_object_clones":0,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":-1,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":120,"num_read_kb":112,"num_write":126,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":35,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002740"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.941549+0000","last_change":"2024-09-11T23:16:25.899106+0000","last_active":"2024-09-11T23:16:35.941549+0000","last_peered":"2024-09-11T23:16:35.941549+0000","last_clean":"2024-09-11T23:16:35.941549+0000","last_became_active":"2024-09-11T23:16:25.898654+0000","last_became_peered":"2024-09-11T23:16:25.898654+0000","last_unstale":"2024-09-11T23:16:35.941549+0000","last_undegraded":"2024-09-11T23:16:35.941549+0000","last_fullsized":"2024-09-11T23:16:35.941549+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:21:41.490905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:36.226970+0000","last_change":"2024-09-11T23:16:25.895629+0000","last_active":"2024-09-11T23:16:36.226970+0000","last_peered":"2024-09-11T23:16:36.226970+0000","last_clean":"2024-09-11T23:16:36.226970+0000","last_became_active":"2024-09-11T23:16:25.895308+0000","last_became_peered":"2024-09-11T23:16:25.895308+0000","last_unstale":"2024-09-11T23:16:36.226970+0000","last_undegraded":"2024-09-11T23:16:36.226970+0000","last_fullsized":"2024-09-11T23:16:36.226970+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:39:42.549970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:36.226468+0000","last_change":"2024-09-11T23:16:25.907113+0000","last_active":"2024-09-11T23:16:36.226468+0000","last_peered":"2024-09-11T23:16:36.226468+0000","last_clean":"2024-09-11T23:16:36.226468+0000","last_became_active":"2024-09-11T23:16:25.906803+0000","last_became_peered":"2024-09-11T23:16:25.906803+0000","last_unstale":"2024-09-11T23:16:36.226468+0000","last_undegraded":"2024-09-11T23:16:36.226468+0000","last_fullsized":"2024-09-11T23:16:36.226468+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:36:49.303932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.946167+0000","last_change":"2024-09-11T23:16:35.946167+0000","last_active":"2024-09-11T23:16:35.946167+0000","last_peered":"2024-09-11T23:16:35.946167+0000","last_clean":"2024-09-11T23:16:35.946167+0000","last_became_active":"2024-09-11T23:16:35.945786+0000","last_became_peered":"2024-09-11T23:16:35.945786+0000","last_unstale":"2024-09-11T23:16:35.946167+0000","last_undegraded":"2024-09-11T23:16:35.946167+0000","last_fullsized":"2024-09-11T23:16:35.946167+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:13:03.064910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.944477+0000","last_change":"2024-09-11T23:16:25.897909+0000","last_active":"2024-09-11T23:16:35.944477+0000","last_peered":"2024-09-11T23:16:35.944477+0000","last_clean":"2024-09-11T23:16:35.944477+0000","last_became_active":"2024-09-11T23:16:25.897569+0000","last_became_peered":"2024-09-11T23:16:25.897569+0000","last_unstale":"2024-09-11T23:16:35.944477+0000","last_undegraded":"2024-09-11T23:16:35.944477+0000","last_fullsized":"2024-09-11T23:16:35.944477+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:25:44.727254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.942367+0000","last_change":"2024-09-11T23:16:25.897594+0000","last_active":"2024-09-11T23:16:35.942367+0000","last_peered":"2024-09-11T23:16:35.942367+0000","last_clean":"2024-09-11T23:16:35.942367+0000","last_became_active":"2024-09-11T23:16:25.897223+0000","last_became_peered":"2024-09-11T23:16:25.897223+0000","last_unstale":"2024-09-11T23:16:35.942367+0000","last_undegraded":"2024-09-11T23:16:35.942367+0000","last_fullsized":"2024-09-11T23:16:35.942367+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:10:50.763455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.943511+0000","last_change":"2024-09-11T23:16:35.943511+0000","last_active":"2024-09-11T23:16:35.943511+0000","last_peered":"2024-09-11T23:16:35.943511+0000","last_clean":"2024-09-11T23:16:35.943511+0000","last_became_active":"2024-09-11T23:16:35.943075+0000","last_became_peered":"2024-09-11T23:16:35.943075+0000","last_unstale":"2024-09-11T23:16:35.943511+0000","last_undegraded":"2024-09-11T23:16:35.943511+0000","last_fullsized":"2024-09-11T23:16:35.943511+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:53:43.892421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:36.226370+0000","last_change":"2024-09-11T23:16:25.903380+0000","last_active":"2024-09-11T23:16:36.226370+0000","last_peered":"2024-09-11T23:16:36.226370+0000","last_clean":"2024-09-11T23:16:36.226370+0000","last_became_active":"2024-09-11T23:16:25.903093+0000","last_became_peered":"2024-09-11T23:16:25.903093+0000","last_unstale":"2024-09-11T23:16:36.226370+0000","last_undegraded":"2024-09-11T23:16:36.226370+0000","last_fullsized":"2024-09-11T23:16:36.226370+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:58:50.944623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.939171+0000","last_change":"2024-09-11T23:16:25.908166+0000","last_active":"2024-09-11T23:16:35.939171+0000","last_peered":"2024-09-11T23:16:35.939171+0000","last_clean":"2024-09-11T23:16:35.939171+0000","last_became_active":"2024-09-11T23:16:25.907749+0000","last_became_peered":"2024-09-11T23:16:25.907749+0000","last_unstale":"2024-09-11T23:16:35.939171+0000","last_undegraded":"2024-09-11T23:16:35.939171+0000","last_fullsized":"2024-09-11T23:16:35.939171+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:11:16.705931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.944259+0000","last_change":"2024-09-11T23:16:35.944259+0000","last_active":"2024-09-11T23:16:35.944259+0000","last_peered":"2024-09-11T23:16:35.944259+0000","last_clean":"2024-09-11T23:16:35.944259+0000","last_became_active":"2024-09-11T23:16:35.943767+0000","last_became_peered":"2024-09-11T23:16:35.943767+0000","last_unstale":"2024-09-11T23:16:35.944259+0000","last_undegraded":"2024-09-11T23:16:35.944259+0000","last_fullsized":"2024-09-11T23:16:35.944259+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:20:13.127854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.941922+0000","last_change":"2024-09-11T23:16:25.903256+0000","last_active":"2024-09-11T23:16:35.941922+0000","last_peered":"2024-09-11T23:16:35.941922+0000","last_clean":"2024-09-11T23:16:35.941922+0000","last_became_active":"2024-09-11T23:16:25.902954+0000","last_became_peered":"2024-09-11T23:16:25.902954+0000","last_unstale":"2024-09-11T23:16:35.941922+0000","last_undegraded":"2024-09-11T23:16:35.941922+0000","last_fullsized":"2024-09-11T23:16:35.941922+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:43:48.651400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:36.226960+0000","last_change":"2024-09-11T23:16:25.897656+0000","last_active":"2024-09-11T23:16:36.226960+0000","last_peered":"2024-09-11T23:16:36.226960+0000","last_clean":"2024-09-11T23:16:36.226960+0000","last_became_active":"2024-09-11T23:16:25.897103+0000","last_became_peered":"2024-09-11T23:16:25.897103+0000","last_unstale":"2024-09-11T23:16:36.226960+0000","last_undegraded":"2024-09-11T23:16:36.226960+0000","last_fullsized":"2024-09-11T23:16:36.226960+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:14:34.341383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.938158+0000","last_change":"2024-09-11T23:16:25.902800+0000","last_active":"2024-09-11T23:16:35.938158+0000","last_peered":"2024-09-11T23:16:35.938158+0000","last_clean":"2024-09-11T23:16:35.938158+0000","last_became_active":"2024-09-11T23:16:25.902414+0000","last_became_peered":"2024-09-11T23:16:25.902414+0000","last_unstale":"2024-09-11T23:16:35.938158+0000","last_undegraded":"2024-09-11T23:16:35.938158+0000","last_fullsized":"2024-09-11T23:16:35.938158+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:16:32.427749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:36.226337+0000","last_change":"2024-09-11T23:16:25.894410+0000","last_active":"2024-09-11T23:16:36.226337+0000","last_peered":"2024-09-11T23:16:36.226337+0000","last_clean":"2024-09-11T23:16:36.226337+0000","last_became_active":"2024-09-11T23:16:25.894067+0000","last_became_peered":"2024-09-11T23:16:25.894067+0000","last_unstale":"2024-09-11T23:16:36.226337+0000","last_undegraded":"2024-09-11T23:16:36.226337+0000","last_fullsized":"2024-09-11T23:16:36.226337+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:03:11.913063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:36.226260+0000","last_change":"2024-09-11T23:16:25.903440+0000","last_active":"2024-09-11T23:16:36.226260+0000","last_peered":"2024-09-11T23:16:36.226260+0000","last_clean":"2024-09-11T23:16:36.226260+0000","last_became_active":"2024-09-11T23:16:25.903122+0000","last_became_peered":"2024-09-11T23:16:25.903122+0000","last_unstale":"2024-09-11T23:16:36.226260+0000","last_undegraded":"2024-09-11T23:16:36.226260+0000","last_fullsized":"2024-09-11T23:16:36.226260+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:03:41.065314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.938680+0000","last_change":"2024-09-11T23:16:25.905336+0000","last_active":"2024-09-11T23:16:35.938680+0000","last_peered":"2024-09-11T23:16:35.938680+0000","last_clean":"2024-09-11T23:16:35.938680+0000","last_became_active":"2024-09-11T23:16:25.904699+0000","last_became_peered":"2024-09-11T23:16:25.904699+0000","last_unstale":"2024-09-11T23:16:35.938680+0000","last_undegraded":"2024-09-11T23:16:35.938680+0000","last_fullsized":"2024-09-11T23:16:35.938680+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:16:13.448585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.941545+0000","last_change":"2024-09-11T23:16:25.897977+0000","last_active":"2024-09-11T23:16:35.941545+0000","last_peered":"2024-09-11T23:16:35.941545+0000","last_clean":"2024-09-11T23:16:35.941545+0000","last_became_active":"2024-09-11T23:16:25.897672+0000","last_became_peered":"2024-09-11T23:16:25.897672+0000","last_unstale":"2024-09-11T23:16:35.941545+0000","last_undegraded":"2024-09-11T23:16:35.941545+0000","last_fullsized":"2024-09-11T23:16:35.941545+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:42:56.575066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.939453+0000","last_change":"2024-09-11T23:16:25.897063+0000","last_active":"2024-09-11T23:16:35.939453+0000","last_peered":"2024-09-11T23:16:35.939453+0000","last_clean":"2024-09-11T23:16:35.939453+0000","last_became_active":"2024-09-11T23:16:25.896770+0000","last_became_peered":"2024-09-11T23:16:25.896770+0000","last_unstale":"2024-09-11T23:16:35.939453+0000","last_undegraded":"2024-09-11T23:16:35.939453+0000","last_fullsized":"2024-09-11T23:16:35.939453+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:55:44.735171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.939570+0000","last_change":"2024-09-11T23:16:25.903423+0000","last_active":"2024-09-11T23:16:35.939570+0000","last_peered":"2024-09-11T23:16:35.939570+0000","last_clean":"2024-09-11T23:16:35.939570+0000","last_became_active":"2024-09-11T23:16:25.902906+0000","last_became_peered":"2024-09-11T23:16:25.902906+0000","last_unstale":"2024-09-11T23:16:35.939570+0000","last_undegraded":"2024-09-11T23:16:35.939570+0000","last_fullsized":"2024-09-11T23:16:35.939570+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:40:49.326045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.942800+0000","last_change":"2024-09-11T23:16:25.896564+0000","last_active":"2024-09-11T23:16:35.942800+0000","last_peered":"2024-09-11T23:16:35.942800+0000","last_clean":"2024-09-11T23:16:35.942800+0000","last_became_active":"2024-09-11T23:16:25.896301+0000","last_became_peered":"2024-09-11T23:16:25.896301+0000","last_unstale":"2024-09-11T23:16:35.942800+0000","last_undegraded":"2024-09-11T23:16:35.942800+0000","last_fullsized":"2024-09-11T23:16:35.942800+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:30:30.074853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.938246+0000","last_change":"2024-09-11T23:16:25.903491+0000","last_active":"2024-09-11T23:16:35.938246+0000","last_peered":"2024-09-11T23:16:35.938246+0000","last_clean":"2024-09-11T23:16:35.938246+0000","last_became_active":"2024-09-11T23:16:25.901340+0000","last_became_peered":"2024-09-11T23:16:25.901340+0000","last_unstale":"2024-09-11T23:16:35.938246+0000","last_undegraded":"2024-09-11T23:16:35.938246+0000","last_fullsized":"2024-09-11T23:16:35.938246+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:41:16.058034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.938800+0000","last_change":"2024-09-11T23:16:25.901084+0000","last_active":"2024-09-11T23:16:35.938800+0000","last_peered":"2024-09-11T23:16:35.938800+0000","last_clean":"2024-09-11T23:16:35.938800+0000","last_became_active":"2024-09-11T23:16:25.900726+0000","last_became_peered":"2024-09-11T23:16:25.900726+0000","last_unstale":"2024-09-11T23:16:35.938800+0000","last_undegraded":"2024-09-11T23:16:35.938800+0000","last_fullsized":"2024-09-11T23:16:35.938800+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:57:40.955645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:36.226559+0000","last_change":"2024-09-11T23:16:25.900461+0000","last_active":"2024-09-11T23:16:36.226559+0000","last_peered":"2024-09-11T23:16:36.226559+0000","last_clean":"2024-09-11T23:16:36.226559+0000","last_became_active":"2024-09-11T23:16:25.900174+0000","last_became_peered":"2024-09-11T23:16:25.900174+0000","last_unstale":"2024-09-11T23:16:36.226559+0000","last_undegraded":"2024-09-11T23:16:36.226559+0000","last_fullsized":"2024-09-11T23:16:36.226559+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:51:50.953198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.940957+0000","last_change":"2024-09-11T23:16:25.907569+0000","last_active":"2024-09-11T23:16:35.940957+0000","last_peered":"2024-09-11T23:16:35.940957+0000","last_clean":"2024-09-11T23:16:35.940957+0000","last_became_active":"2024-09-11T23:16:25.907091+0000","last_became_peered":"2024-09-11T23:16:25.907091+0000","last_unstale":"2024-09-11T23:16:35.940957+0000","last_undegraded":"2024-09-11T23:16:35.940957+0000","last_fullsized":"2024-09-11T23:16:35.940957+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:53:45.961167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:36.226935+0000","last_change":"2024-09-11T23:16:25.903246+0000","last_active":"2024-09-11T23:16:36.226935+0000","last_peered":"2024-09-11T23:16:36.226935+0000","last_clean":"2024-09-11T23:16:36.226935+0000","last_became_active":"2024-09-11T23:16:25.902907+0000","last_became_peered":"2024-09-11T23:16:25.902907+0000","last_unstale":"2024-09-11T23:16:36.226935+0000","last_undegraded":"2024-09-11T23:16:36.226935+0000","last_fullsized":"2024-09-11T23:16:36.226935+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:20:38.868384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.939763+0000","last_change":"2024-09-11T23:16:25.906298+0000","last_active":"2024-09-11T23:16:35.939763+0000","last_peered":"2024-09-11T23:16:35.939763+0000","last_clean":"2024-09-11T23:16:35.939763+0000","last_became_active":"2024-09-11T23:16:25.905350+0000","last_became_peered":"2024-09-11T23:16:25.905350+0000","last_unstale":"2024-09-11T23:16:35.939763+0000","last_undegraded":"2024-09-11T23:16:35.939763+0000","last_fullsized":"2024-09-11T23:16:35.939763+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:04:22.122514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.941073+0000","last_change":"2024-09-11T23:16:25.902114+0000","last_active":"2024-09-11T23:16:35.941073+0000","last_peered":"2024-09-11T23:16:35.941073+0000","last_clean":"2024-09-11T23:16:35.941073+0000","last_became_active":"2024-09-11T23:16:25.901742+0000","last_became_peered":"2024-09-11T23:16:25.901742+0000","last_unstale":"2024-09-11T23:16:35.941073+0000","last_undegraded":"2024-09-11T23:16:35.941073+0000","last_fullsized":"2024-09-11T23:16:35.941073+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:02:10.640801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.938325+0000","last_change":"2024-09-11T23:16:25.898117+0000","last_active":"2024-09-11T23:16:35.938325+0000","last_peered":"2024-09-11T23:16:35.938325+0000","last_clean":"2024-09-11T23:16:35.938325+0000","last_became_active":"2024-09-11T23:16:25.897818+0000","last_became_peered":"2024-09-11T23:16:25.897818+0000","last_unstale":"2024-09-11T23:16:35.938325+0000","last_undegraded":"2024-09-11T23:16:35.938325+0000","last_fullsized":"2024-09-11T23:16:35.938325+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:27:37.175471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:36.226573+0000","last_change":"2024-09-11T23:16:25.905600+0000","last_active":"2024-09-11T23:16:36.226573+0000","last_peered":"2024-09-11T23:16:36.226573+0000","last_clean":"2024-09-11T23:16:36.226573+0000","last_became_active":"2024-09-11T23:16:25.903924+0000","last_became_peered":"2024-09-11T23:16:25.903924+0000","last_unstale":"2024-09-11T23:16:36.226573+0000","last_undegraded":"2024-09-11T23:16:36.226573+0000","last_fullsized":"2024-09-11T23:16:36.226573+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:20:29.929079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.939290+0000","last_change":"2024-09-11T23:16:25.904763+0000","last_active":"2024-09-11T23:16:35.939290+0000","last_peered":"2024-09-11T23:16:35.939290+0000","last_clean":"2024-09-11T23:16:35.939290+0000","last_became_active":"2024-09-11T23:16:25.904406+0000","last_became_peered":"2024-09-11T23:16:25.904406+0000","last_unstale":"2024-09-11T23:16:35.939290+0000","last_undegraded":"2024-09-11T23:16:35.939290+0000","last_fullsized":"2024-09-11T23:16:35.939290+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:27:38.918017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.939346+0000","last_change":"2024-09-11T23:16:25.906573+0000","last_active":"2024-09-11T23:16:35.939346+0000","last_peered":"2024-09-11T23:16:35.939346+0000","last_clean":"2024-09-11T23:16:35.939346+0000","last_became_active":"2024-09-11T23:16:25.904771+0000","last_became_peered":"2024-09-11T23:16:25.904771+0000","last_unstale":"2024-09-11T23:16:35.939346+0000","last_undegraded":"2024-09-11T23:16:35.939346+0000","last_fullsized":"2024-09-11T23:16:35.939346+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:04:17.090648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.942191+0000","last_change":"2024-09-11T23:16:25.902123+0000","last_active":"2024-09-11T23:16:35.942191+0000","last_peered":"2024-09-11T23:16:35.942191+0000","last_clean":"2024-09-11T23:16:35.942191+0000","last_became_active":"2024-09-11T23:16:25.901756+0000","last_became_peered":"2024-09-11T23:16:25.901756+0000","last_unstale":"2024-09-11T23:16:35.942191+0000","last_undegraded":"2024-09-11T23:16:35.942191+0000","last_fullsized":"2024-09-11T23:16:35.942191+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:17:37.074976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.944098+0000","last_change":"2024-09-11T23:16:25.904217+0000","last_active":"2024-09-11T23:16:35.944098+0000","last_peered":"2024-09-11T23:16:35.944098+0000","last_clean":"2024-09-11T23:16:35.944098+0000","last_became_active":"2024-09-11T23:16:25.903522+0000","last_became_peered":"2024-09-11T23:16:25.903522+0000","last_unstale":"2024-09-11T23:16:35.944098+0000","last_undegraded":"2024-09-11T23:16:35.944098+0000","last_fullsized":"2024-09-11T23:16:35.944098+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:29:46.634475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"83'5","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:48.000881+0000","last_change":"2024-09-11T23:16:23.793696+0000","last_active":"2024-09-11T23:16:48.000881+0000","last_peered":"2024-09-11T23:16:48.000881+0000","last_clean":"2024-09-11T23:16:48.000881+0000","last_became_active":"2024-09-11T23:16:23.793062+0000","last_became_peered":"2024-09-11T23:16:23.793062+0000","last_unstale":"2024-09-11T23:16:48.000881+0000","last_undegraded":"2024-09-11T23:16:48.000881+0000","last_fullsized":"2024-09-11T23:16:48.000881+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:00:25.655817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":13,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.941878+0000","last_change":"2024-09-11T23:16:25.906673+0000","last_active":"2024-09-11T23:16:35.941878+0000","last_peered":"2024-09-11T23:16:35.941878+0000","last_clean":"2024-09-11T23:16:35.941878+0000","last_became_active":"2024-09-11T23:16:25.906239+0000","last_became_peered":"2024-09-11T23:16:25.906239+0000","last_unstale":"2024-09-11T23:16:35.941878+0000","last_undegraded":"2024-09-11T23:16:35.941878+0000","last_fullsized":"2024-09-11T23:16:35.941878+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:15:54.907269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"85'7","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:36.973576+0000","last_change":"2024-09-11T23:16:23.793302+0000","last_active":"2024-09-11T23:16:36.973576+0000","last_peered":"2024-09-11T23:16:36.973576+0000","last_clean":"2024-09-11T23:16:36.973576+0000","last_became_active":"2024-09-11T23:16:23.792874+0000","last_became_peered":"2024-09-11T23:16:23.792874+0000","last_unstale":"2024-09-11T23:16:36.973576+0000","last_undegraded":"2024-09-11T23:16:36.973576+0000","last_fullsized":"2024-09-11T23:16:36.973576+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:52:59.558223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"85'3","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.937984+0000","last_change":"2024-09-11T23:16:23.792999+0000","last_active":"2024-09-11T23:16:35.937984+0000","last_peered":"2024-09-11T23:16:35.937984+0000","last_clean":"2024-09-11T23:16:35.937984+0000","last_became_active":"2024-09-11T23:16:23.792630+0000","last_became_peered":"2024-09-11T23:16:23.792630+0000","last_unstale":"2024-09-11T23:16:35.937984+0000","last_undegraded":"2024-09-11T23:16:35.937984+0000","last_fullsized":"2024-09-11T23:16:35.937984+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:09:28.001060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.943722+0000","last_change":"2024-09-11T23:16:35.943722+0000","last_active":"2024-09-11T23:16:35.943722+0000","last_peered":"2024-09-11T23:16:35.943722+0000","last_clean":"2024-09-11T23:16:35.943722+0000","last_became_active":"2024-09-11T23:16:35.943355+0000","last_became_peered":"2024-09-11T23:16:35.943355+0000","last_unstale":"2024-09-11T23:16:35.943722+0000","last_undegraded":"2024-09-11T23:16:35.943722+0000","last_fullsized":"2024-09-11T23:16:35.943722+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:26:22.769824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,6],"acting":[9,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"85'3","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:37.978674+0000","last_change":"2024-09-11T23:16:23.797945+0000","last_active":"2024-09-11T23:16:37.978674+0000","last_peered":"2024-09-11T23:16:37.978674+0000","last_clean":"2024-09-11T23:16:37.978674+0000","last_became_active":"2024-09-11T23:16:23.797572+0000","last_became_peered":"2024-09-11T23:16:23.797572+0000","last_unstale":"2024-09-11T23:16:37.978674+0000","last_undegraded":"2024-09-11T23:16:37.978674+0000","last_fullsized":"2024-09-11T23:16:37.978674+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:04:59.017668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.939247+0000","last_change":"2024-09-11T23:16:25.891587+0000","last_active":"2024-09-11T23:16:35.939247+0000","last_peered":"2024-09-11T23:16:35.939247+0000","last_clean":"2024-09-11T23:16:35.939247+0000","last_became_active":"2024-09-11T23:16:25.891190+0000","last_became_peered":"2024-09-11T23:16:25.891190+0000","last_unstale":"2024-09-11T23:16:35.939247+0000","last_undegraded":"2024-09-11T23:16:35.939247+0000","last_fullsized":"2024-09-11T23:16:35.939247+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:57:05.177843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"85'4","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:44.996222+0000","last_change":"2024-09-11T23:16:23.790129+0000","last_active":"2024-09-11T23:16:44.996222+0000","last_peered":"2024-09-11T23:16:44.996222+0000","last_clean":"2024-09-11T23:16:44.996222+0000","last_became_active":"2024-09-11T23:16:23.789720+0000","last_became_peered":"2024-09-11T23:16:23.789720+0000","last_unstale":"2024-09-11T23:16:44.996222+0000","last_undegraded":"2024-09-11T23:16:44.996222+0000","last_fullsized":"2024-09-11T23:16:44.996222+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:33:58.388513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.939104+0000","last_change":"2024-09-11T23:16:25.901014+0000","last_active":"2024-09-11T23:16:35.939104+0000","last_peered":"2024-09-11T23:16:35.939104+0000","last_clean":"2024-09-11T23:16:35.939104+0000","last_became_active":"2024-09-11T23:16:25.900280+0000","last_became_peered":"2024-09-11T23:16:25.900280+0000","last_unstale":"2024-09-11T23:16:35.939104+0000","last_undegraded":"2024-09-11T23:16:35.939104+0000","last_fullsized":"2024-09-11T23:16:35.939104+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:14:03.662204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"85'4","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:37.978517+0000","last_change":"2024-09-11T23:16:23.797944+0000","last_active":"2024-09-11T23:16:37.978517+0000","last_peered":"2024-09-11T23:16:37.978517+0000","last_clean":"2024-09-11T23:16:37.978517+0000","last_became_active":"2024-09-11T23:16:23.797520+0000","last_became_peered":"2024-09-11T23:16:23.797520+0000","last_unstale":"2024-09-11T23:16:37.978517+0000","last_undegraded":"2024-09-11T23:16:37.978517+0000","last_fullsized":"2024-09-11T23:16:37.978517+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:29:06.957518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:36.226839+0000","last_change":"2024-09-11T23:16:25.905379+0000","last_active":"2024-09-11T23:16:36.226839+0000","last_peered":"2024-09-11T23:16:36.226839+0000","last_clean":"2024-09-11T23:16:36.226839+0000","last_became_active":"2024-09-11T23:16:25.905085+0000","last_became_peered":"2024-09-11T23:16:25.905085+0000","last_unstale":"2024-09-11T23:16:36.226839+0000","last_undegraded":"2024-09-11T23:16:36.226839+0000","last_fullsized":"2024-09-11T23:16:36.226839+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:22:35.129736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"85'5","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:36.972662+0000","last_change":"2024-09-11T23:16:24.075556+0000","last_active":"2024-09-11T23:16:36.972662+0000","last_peered":"2024-09-11T23:16:36.972662+0000","last_clean":"2024-09-11T23:16:36.972662+0000","last_became_active":"2024-09-11T23:16:24.074783+0000","last_became_peered":"2024-09-11T23:16:24.074783+0000","last_unstale":"2024-09-11T23:16:36.972662+0000","last_undegraded":"2024-09-11T23:16:36.972662+0000","last_fullsized":"2024-09-11T23:16:36.972662+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:55:09.962342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.938922+0000","last_change":"2024-09-11T23:16:25.907720+0000","last_active":"2024-09-11T23:16:35.938922+0000","last_peered":"2024-09-11T23:16:35.938922+0000","last_clean":"2024-09-11T23:16:35.938922+0000","last_became_active":"2024-09-11T23:16:25.907335+0000","last_became_peered":"2024-09-11T23:16:25.907335+0000","last_unstale":"2024-09-11T23:16:35.938922+0000","last_undegraded":"2024-09-11T23:16:35.938922+0000","last_fullsized":"2024-09-11T23:16:35.938922+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:46:14.519384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"85'22","reported_seq":63,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:44.998015+0000","last_change":"2024-09-11T23:16:23.794321+0000","last_active":"2024-09-11T23:16:44.998015+0000","last_peered":"2024-09-11T23:16:44.998015+0000","last_clean":"2024-09-11T23:16:44.998015+0000","last_became_active":"2024-09-11T23:16:23.793955+0000","last_became_peered":"2024-09-11T23:16:23.793955+0000","last_unstale":"2024-09-11T23:16:44.998015+0000","last_undegraded":"2024-09-11T23:16:44.998015+0000","last_fullsized":"2024-09-11T23:16:44.998015+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:47:46.442656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41065,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":19,"num_read_kb":49,"num_write":24,"num_write_kb":45,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.945323+0000","last_change":"2024-09-11T23:16:25.896598+0000","last_active":"2024-09-11T23:16:35.945323+0000","last_peered":"2024-09-11T23:16:35.945323+0000","last_clean":"2024-09-11T23:16:35.945323+0000","last_became_active":"2024-09-11T23:16:25.896293+0000","last_became_peered":"2024-09-11T23:16:25.896293+0000","last_unstale":"2024-09-11T23:16:35.945323+0000","last_undegraded":"2024-09-11T23:16:35.945323+0000","last_fullsized":"2024-09-11T23:16:35.945323+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:04:11.489644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"85'9","reported_seq":33,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:38.985116+0000","last_change":"2024-09-11T23:16:23.791486+0000","last_active":"2024-09-11T23:16:38.985116+0000","last_peered":"2024-09-11T23:16:38.985116+0000","last_clean":"2024-09-11T23:16:38.985116+0000","last_became_active":"2024-09-11T23:16:23.790786+0000","last_became_peered":"2024-09-11T23:16:23.790786+0000","last_unstale":"2024-09-11T23:16:38.985116+0000","last_undegraded":"2024-09-11T23:16:38.985116+0000","last_fullsized":"2024-09-11T23:16:38.985116+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:03:52.742560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.937975+0000","last_change":"2024-09-11T23:16:25.897382+0000","last_active":"2024-09-11T23:16:35.937975+0000","last_peered":"2024-09-11T23:16:35.937975+0000","last_clean":"2024-09-11T23:16:35.937975+0000","last_became_active":"2024-09-11T23:16:25.897073+0000","last_became_peered":"2024-09-11T23:16:25.897073+0000","last_unstale":"2024-09-11T23:16:35.937975+0000","last_undegraded":"2024-09-11T23:16:35.937975+0000","last_fullsized":"2024-09-11T23:16:35.937975+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:15:59.780100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"85'5","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:37.977834+0000","last_change":"2024-09-11T23:16:23.793886+0000","last_active":"2024-09-11T23:16:37.977834+0000","last_peered":"2024-09-11T23:16:37.977834+0000","last_clean":"2024-09-11T23:16:37.977834+0000","last_became_active":"2024-09-11T23:16:23.793066+0000","last_became_peered":"2024-09-11T23:16:23.793066+0000","last_unstale":"2024-09-11T23:16:37.977834+0000","last_undegraded":"2024-09-11T23:16:37.977834+0000","last_fullsized":"2024-09-11T23:16:37.977834+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:07:03.882546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.942540+0000","last_change":"2024-09-11T23:16:25.904575+0000","last_active":"2024-09-11T23:16:35.942540+0000","last_peered":"2024-09-11T23:16:35.942540+0000","last_clean":"2024-09-11T23:16:35.942540+0000","last_became_active":"2024-09-11T23:16:25.904224+0000","last_became_peered":"2024-09-11T23:16:25.904224+0000","last_unstale":"2024-09-11T23:16:35.942540+0000","last_undegraded":"2024-09-11T23:16:35.942540+0000","last_fullsized":"2024-09-11T23:16:35.942540+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:42:13.036931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"85'5","reported_seq":36,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:43.995445+0000","last_change":"2024-09-11T23:16:23.796073+0000","last_active":"2024-09-11T23:16:43.995445+0000","last_peered":"2024-09-11T23:16:43.995445+0000","last_clean":"2024-09-11T23:16:43.995445+0000","last_became_active":"2024-09-11T23:16:23.794850+0000","last_became_peered":"2024-09-11T23:16:23.794850+0000","last_unstale":"2024-09-11T23:16:43.995445+0000","last_undegraded":"2024-09-11T23:16:43.995445+0000","last_fullsized":"2024-09-11T23:16:43.995445+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:57:22.361662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":10,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.940823+0000","last_change":"2024-09-11T23:16:25.908700+0000","last_active":"2024-09-11T23:16:35.940823+0000","last_peered":"2024-09-11T23:16:35.940823+0000","last_clean":"2024-09-11T23:16:35.940823+0000","last_became_active":"2024-09-11T23:16:25.908373+0000","last_became_peered":"2024-09-11T23:16:25.908373+0000","last_unstale":"2024-09-11T23:16:35.940823+0000","last_undegraded":"2024-09-11T23:16:35.940823+0000","last_fullsized":"2024-09-11T23:16:35.940823+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:20:18.536079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"85'17","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:37.976750+0000","last_change":"2024-09-11T23:16:24.075927+0000","last_active":"2024-09-11T23:16:37.976750+0000","last_peered":"2024-09-11T23:16:37.976750+0000","last_clean":"2024-09-11T23:16:37.976750+0000","last_became_active":"2024-09-11T23:16:24.075527+0000","last_became_peered":"2024-09-11T23:16:24.075527+0000","last_unstale":"2024-09-11T23:16:37.976750+0000","last_undegraded":"2024-09-11T23:16:37.976750+0000","last_fullsized":"2024-09-11T23:16:37.976750+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:27:54.622059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.947475+0000","last_change":"2024-09-11T23:16:35.947475+0000","last_active":"2024-09-11T23:16:35.947475+0000","last_peered":"2024-09-11T23:16:35.947475+0000","last_clean":"2024-09-11T23:16:35.947475+0000","last_became_active":"2024-09-11T23:16:35.947028+0000","last_became_peered":"2024-09-11T23:16:35.947028+0000","last_unstale":"2024-09-11T23:16:35.947475+0000","last_undegraded":"2024-09-11T23:16:35.947475+0000","last_fullsized":"2024-09-11T23:16:35.947475+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:18:13.479554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'2","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:37.950468+0000","last_change":"2024-09-11T23:16:23.793298+0000","last_active":"2024-09-11T23:16:37.950468+0000","last_peered":"2024-09-11T23:16:37.950468+0000","last_clean":"2024-09-11T23:16:37.950468+0000","last_became_active":"2024-09-11T23:16:23.792947+0000","last_became_peered":"2024-09-11T23:16:23.792947+0000","last_unstale":"2024-09-11T23:16:37.950468+0000","last_undegraded":"2024-09-11T23:16:37.950468+0000","last_fullsized":"2024-09-11T23:16:37.950468+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:32:21.294560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.941169+0000","last_change":"2024-09-11T23:16:25.902348+0000","last_active":"2024-09-11T23:16:35.941169+0000","last_peered":"2024-09-11T23:16:35.941169+0000","last_clean":"2024-09-11T23:16:35.941169+0000","last_became_active":"2024-09-11T23:16:25.901686+0000","last_became_peered":"2024-09-11T23:16:25.901686+0000","last_unstale":"2024-09-11T23:16:35.941169+0000","last_undegraded":"2024-09-11T23:16:35.941169+0000","last_fullsized":"2024-09-11T23:16:35.941169+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:00:15.543003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"85'5","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:37.975894+0000","last_change":"2024-09-11T23:16:23.790984+0000","last_active":"2024-09-11T23:16:37.975894+0000","last_peered":"2024-09-11T23:16:37.975894+0000","last_clean":"2024-09-11T23:16:37.975894+0000","last_became_active":"2024-09-11T23:16:23.790569+0000","last_became_peered":"2024-09-11T23:16:23.790569+0000","last_unstale":"2024-09-11T23:16:37.975894+0000","last_undegraded":"2024-09-11T23:16:37.975894+0000","last_fullsized":"2024-09-11T23:16:37.975894+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:57:39.675793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.941423+0000","last_change":"2024-09-11T23:16:25.899249+0000","last_active":"2024-09-11T23:16:35.941423+0000","last_peered":"2024-09-11T23:16:35.941423+0000","last_clean":"2024-09-11T23:16:35.941423+0000","last_became_active":"2024-09-11T23:16:25.898914+0000","last_became_peered":"2024-09-11T23:16:25.898914+0000","last_unstale":"2024-09-11T23:16:35.941423+0000","last_undegraded":"2024-09-11T23:16:35.941423+0000","last_fullsized":"2024-09-11T23:16:35.941423+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:17:07.545770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'3","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:37.977945+0000","last_change":"2024-09-11T23:16:23.787387+0000","last_active":"2024-09-11T23:16:37.977945+0000","last_peered":"2024-09-11T23:16:37.977945+0000","last_clean":"2024-09-11T23:16:37.977945+0000","last_became_active":"2024-09-11T23:16:23.787014+0000","last_became_peered":"2024-09-11T23:16:23.787014+0000","last_unstale":"2024-09-11T23:16:37.977945+0000","last_undegraded":"2024-09-11T23:16:37.977945+0000","last_fullsized":"2024-09-11T23:16:37.977945+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:45:52.004253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"85'1","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.941159+0000","last_change":"2024-09-11T23:16:25.908304+0000","last_active":"2024-09-11T23:16:35.941159+0000","last_peered":"2024-09-11T23:16:35.941159+0000","last_clean":"2024-09-11T23:16:35.941159+0000","last_became_active":"2024-09-11T23:16:25.907900+0000","last_became_peered":"2024-09-11T23:16:25.907900+0000","last_unstale":"2024-09-11T23:16:35.941159+0000","last_undegraded":"2024-09-11T23:16:35.941159+0000","last_fullsized":"2024-09-11T23:16:35.941159+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:43:26.247682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"85'6","reported_seq":41,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:46.999370+0000","last_change":"2024-09-11T23:16:24.074610+0000","last_active":"2024-09-11T23:16:46.999370+0000","last_peered":"2024-09-11T23:16:46.999370+0000","last_clean":"2024-09-11T23:16:46.999370+0000","last_became_active":"2024-09-11T23:16:24.074210+0000","last_became_peered":"2024-09-11T23:16:24.074210+0000","last_unstale":"2024-09-11T23:16:46.999370+0000","last_undegraded":"2024-09-11T23:16:46.999370+0000","last_fullsized":"2024-09-11T23:16:46.999370+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:09:37.167050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1563,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":13,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.941591+0000","last_change":"2024-09-11T23:16:25.903120+0000","last_active":"2024-09-11T23:16:35.941591+0000","last_peered":"2024-09-11T23:16:35.941591+0000","last_clean":"2024-09-11T23:16:35.941591+0000","last_became_active":"2024-09-11T23:16:25.902770+0000","last_became_peered":"2024-09-11T23:16:25.902770+0000","last_unstale":"2024-09-11T23:16:35.941591+0000","last_undegraded":"2024-09-11T23:16:35.941591+0000","last_fullsized":"2024-09-11T23:16:35.941591+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:03:30.041088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":346,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.941279+0000","last_change":"2024-09-11T23:15:13.115055+0000","last_active":"2024-09-11T23:16:35.941279+0000","last_peered":"2024-09-11T23:16:35.941279+0000","last_clean":"2024-09-11T23:16:35.941279+0000","last_became_active":"2024-09-11T23:15:13.114480+0000","last_became_peered":"2024-09-11T23:15:13.114480+0000","last_unstale":"2024-09-11T23:16:35.941279+0000","last_undegraded":"2024-09-11T23:16:35.941279+0000","last_fullsized":"2024-09-11T23:16:35.941279+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:11:50.495799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:11:50.495799+0000","last_clean_scrub_stamp":"2024-09-11T23:11:50.495799+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:15:53.681230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"85'6","reported_seq":31,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:38.985260+0000","last_change":"2024-09-11T23:16:23.793996+0000","last_active":"2024-09-11T23:16:38.985260+0000","last_peered":"2024-09-11T23:16:38.985260+0000","last_clean":"2024-09-11T23:16:38.985260+0000","last_became_active":"2024-09-11T23:16:23.793681+0000","last_became_peered":"2024-09-11T23:16:23.793681+0000","last_unstale":"2024-09-11T23:16:38.985260+0000","last_undegraded":"2024-09-11T23:16:38.985260+0000","last_fullsized":"2024-09-11T23:16:38.985260+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:52:50.463623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:36.227084+0000","last_change":"2024-09-11T23:16:25.905133+0000","last_active":"2024-09-11T23:16:36.227084+0000","last_peered":"2024-09-11T23:16:36.227084+0000","last_clean":"2024-09-11T23:16:36.227084+0000","last_became_active":"2024-09-11T23:16:25.904819+0000","last_became_peered":"2024-09-11T23:16:25.904819+0000","last_unstale":"2024-09-11T23:16:36.227084+0000","last_undegraded":"2024-09-11T23:16:36.227084+0000","last_fullsized":"2024-09-11T23:16:36.227084+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:51:09.664343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"85'4","reported_seq":32,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:36.231286+0000","last_change":"2024-09-11T23:16:36.228806+0000","last_active":"2024-09-11T23:16:36.231286+0000","last_peered":"2024-09-11T23:16:36.231286+0000","last_clean":"2024-09-11T23:16:36.231286+0000","last_became_active":"2024-09-11T23:16:36.228311+0000","last_became_peered":"2024-09-11T23:16:36.228311+0000","last_unstale":"2024-09-11T23:16:36.231286+0000","last_undegraded":"2024-09-11T23:16:36.231286+0000","last_fullsized":"2024-09-11T23:16:36.231286+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:42:32.856691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4],"acting":[7,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.937900+0000","last_change":"2024-09-11T23:16:25.898935+0000","last_active":"2024-09-11T23:16:35.937900+0000","last_peered":"2024-09-11T23:16:35.937900+0000","last_clean":"2024-09-11T23:16:35.937900+0000","last_became_active":"2024-09-11T23:16:25.898618+0000","last_became_peered":"2024-09-11T23:16:25.898618+0000","last_unstale":"2024-09-11T23:16:35.937900+0000","last_undegraded":"2024-09-11T23:16:35.937900+0000","last_fullsized":"2024-09-11T23:16:35.937900+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:13:29.104911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"85'5","reported_seq":39,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:47.999580+0000","last_change":"2024-09-11T23:16:24.073613+0000","last_active":"2024-09-11T23:16:47.999580+0000","last_peered":"2024-09-11T23:16:47.999580+0000","last_clean":"2024-09-11T23:16:47.999580+0000","last_became_active":"2024-09-11T23:16:24.073262+0000","last_became_peered":"2024-09-11T23:16:24.073262+0000","last_unstale":"2024-09-11T23:16:47.999580+0000","last_undegraded":"2024-09-11T23:16:47.999580+0000","last_fullsized":"2024-09-11T23:16:47.999580+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:31:58.653657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":12,"num_read_kb":12,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.941067+0000","last_change":"2024-09-11T23:16:25.903725+0000","last_active":"2024-09-11T23:16:35.941067+0000","last_peered":"2024-09-11T23:16:35.941067+0000","last_clean":"2024-09-11T23:16:35.941067+0000","last_became_active":"2024-09-11T23:16:25.903375+0000","last_became_peered":"2024-09-11T23:16:25.903375+0000","last_unstale":"2024-09-11T23:16:35.941067+0000","last_undegraded":"2024-09-11T23:16:35.941067+0000","last_fullsized":"2024-09-11T23:16:35.941067+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:55:02.779575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"85'5","reported_seq":38,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:46.998806+0000","last_change":"2024-09-11T23:16:23.793932+0000","last_active":"2024-09-11T23:16:46.998806+0000","last_peered":"2024-09-11T23:16:46.998806+0000","last_clean":"2024-09-11T23:16:46.998806+0000","last_became_active":"2024-09-11T23:16:23.793111+0000","last_became_peered":"2024-09-11T23:16:23.793111+0000","last_unstale":"2024-09-11T23:16:46.998806+0000","last_undegraded":"2024-09-11T23:16:46.998806+0000","last_fullsized":"2024-09-11T23:16:46.998806+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:14:09.087222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.942165+0000","last_change":"2024-09-11T23:16:25.905020+0000","last_active":"2024-09-11T23:16:35.942165+0000","last_peered":"2024-09-11T23:16:35.942165+0000","last_clean":"2024-09-11T23:16:35.942165+0000","last_became_active":"2024-09-11T23:16:25.904675+0000","last_became_peered":"2024-09-11T23:16:25.904675+0000","last_unstale":"2024-09-11T23:16:35.942165+0000","last_undegraded":"2024-09-11T23:16:35.942165+0000","last_fullsized":"2024-09-11T23:16:35.942165+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:13:57.388773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"85'1","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.939622+0000","last_change":"2024-09-11T23:16:24.073630+0000","last_active":"2024-09-11T23:16:35.939622+0000","last_peered":"2024-09-11T23:16:35.939622+0000","last_clean":"2024-09-11T23:16:35.939622+0000","last_became_active":"2024-09-11T23:16:24.073301+0000","last_became_peered":"2024-09-11T23:16:24.073301+0000","last_unstale":"2024-09-11T23:16:35.939622+0000","last_undegraded":"2024-09-11T23:16:35.939622+0000","last_fullsized":"2024-09-11T23:16:35.939622+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:41:59.374581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"85'1","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.942393+0000","last_change":"2024-09-11T23:16:25.894124+0000","last_active":"2024-09-11T23:16:35.942393+0000","last_peered":"2024-09-11T23:16:35.942393+0000","last_clean":"2024-09-11T23:16:35.942393+0000","last_became_active":"2024-09-11T23:16:25.893751+0000","last_became_peered":"2024-09-11T23:16:25.893751+0000","last_unstale":"2024-09-11T23:16:35.942393+0000","last_undegraded":"2024-09-11T23:16:35.942393+0000","last_fullsized":"2024-09-11T23:16:35.942393+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:00:28.265206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"85'4","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:37.976960+0000","last_change":"2024-09-11T23:16:24.073700+0000","last_active":"2024-09-11T23:16:37.976960+0000","last_peered":"2024-09-11T23:16:37.976960+0000","last_clean":"2024-09-11T23:16:37.976960+0000","last_became_active":"2024-09-11T23:16:24.073283+0000","last_became_peered":"2024-09-11T23:16:24.073283+0000","last_unstale":"2024-09-11T23:16:37.976960+0000","last_undegraded":"2024-09-11T23:16:37.976960+0000","last_fullsized":"2024-09-11T23:16:37.976960+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:41:43.536834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.938154+0000","last_change":"2024-09-11T23:16:25.900532+0000","last_active":"2024-09-11T23:16:35.938154+0000","last_peered":"2024-09-11T23:16:35.938154+0000","last_clean":"2024-09-11T23:16:35.938154+0000","last_became_active":"2024-09-11T23:16:25.899466+0000","last_became_peered":"2024-09-11T23:16:25.899466+0000","last_unstale":"2024-09-11T23:16:35.938154+0000","last_undegraded":"2024-09-11T23:16:35.938154+0000","last_fullsized":"2024-09-11T23:16:35.938154+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:10:01.152323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"85'10","reported_seq":32,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:41.912906+0000","last_change":"2024-09-11T23:16:23.788789+0000","last_active":"2024-09-11T23:16:41.912906+0000","last_peered":"2024-09-11T23:16:41.912906+0000","last_clean":"2024-09-11T23:16:41.912906+0000","last_became_active":"2024-09-11T23:16:23.788422+0000","last_became_peered":"2024-09-11T23:16:23.788422+0000","last_unstale":"2024-09-11T23:16:41.912906+0000","last_undegraded":"2024-09-11T23:16:41.912906+0000","last_fullsized":"2024-09-11T23:16:41.912906+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:09:14.310205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:36.227116+0000","last_change":"2024-09-11T23:16:25.895511+0000","last_active":"2024-09-11T23:16:36.227116+0000","last_peered":"2024-09-11T23:16:36.227116+0000","last_clean":"2024-09-11T23:16:36.227116+0000","last_became_active":"2024-09-11T23:16:25.895143+0000","last_became_peered":"2024-09-11T23:16:25.895143+0000","last_unstale":"2024-09-11T23:16:36.227116+0000","last_undegraded":"2024-09-11T23:16:36.227116+0000","last_fullsized":"2024-09-11T23:16:36.227116+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:31:16.498371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"85'7","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:37.977151+0000","last_change":"2024-09-11T23:16:24.019064+0000","last_active":"2024-09-11T23:16:37.977151+0000","last_peered":"2024-09-11T23:16:37.977151+0000","last_clean":"2024-09-11T23:16:37.977151+0000","last_became_active":"2024-09-11T23:16:24.018658+0000","last_became_peered":"2024-09-11T23:16:24.018658+0000","last_unstale":"2024-09-11T23:16:37.977151+0000","last_undegraded":"2024-09-11T23:16:37.977151+0000","last_fullsized":"2024-09-11T23:16:37.977151+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:17:41.009507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:36.227194+0000","last_change":"2024-09-11T23:16:25.903553+0000","last_active":"2024-09-11T23:16:36.227194+0000","last_peered":"2024-09-11T23:16:36.227194+0000","last_clean":"2024-09-11T23:16:36.227194+0000","last_became_active":"2024-09-11T23:16:25.903221+0000","last_became_peered":"2024-09-11T23:16:25.903221+0000","last_unstale":"2024-09-11T23:16:36.227194+0000","last_undegraded":"2024-09-11T23:16:36.227194+0000","last_fullsized":"2024-09-11T23:16:36.227194+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:16:51.872517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"83'2","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.938847+0000","last_change":"2024-09-11T23:16:24.073784+0000","last_active":"2024-09-11T23:16:35.938847+0000","last_peered":"2024-09-11T23:16:35.938847+0000","last_clean":"2024-09-11T23:16:35.938847+0000","last_became_active":"2024-09-11T23:16:24.073406+0000","last_became_peered":"2024-09-11T23:16:24.073406+0000","last_unstale":"2024-09-11T23:16:35.938847+0000","last_undegraded":"2024-09-11T23:16:35.938847+0000","last_fullsized":"2024-09-11T23:16:35.938847+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:31:24.617025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.945715+0000","last_change":"2024-09-11T23:16:25.894878+0000","last_active":"2024-09-11T23:16:35.945715+0000","last_peered":"2024-09-11T23:16:35.945715+0000","last_clean":"2024-09-11T23:16:35.945715+0000","last_became_active":"2024-09-11T23:16:25.894543+0000","last_became_peered":"2024-09-11T23:16:25.894543+0000","last_unstale":"2024-09-11T23:16:35.945715+0000","last_undegraded":"2024-09-11T23:16:35.945715+0000","last_fullsized":"2024-09-11T23:16:35.945715+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:34:13.435754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"85'7","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:37.976060+0000","last_change":"2024-09-11T23:16:23.786502+0000","last_active":"2024-09-11T23:16:37.976060+0000","last_peered":"2024-09-11T23:16:37.976060+0000","last_clean":"2024-09-11T23:16:37.976060+0000","last_became_active":"2024-09-11T23:16:23.786182+0000","last_became_peered":"2024-09-11T23:16:23.786182+0000","last_unstale":"2024-09-11T23:16:37.976060+0000","last_undegraded":"2024-09-11T23:16:37.976060+0000","last_fullsized":"2024-09-11T23:16:37.976060+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:59:07.978244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.942309+0000","last_change":"2024-09-11T23:16:25.899625+0000","last_active":"2024-09-11T23:16:35.942309+0000","last_peered":"2024-09-11T23:16:35.942309+0000","last_clean":"2024-09-11T23:16:35.942309+0000","last_became_active":"2024-09-11T23:16:25.899265+0000","last_became_peered":"2024-09-11T23:16:25.899265+0000","last_unstale":"2024-09-11T23:16:35.942309+0000","last_undegraded":"2024-09-11T23:16:35.942309+0000","last_fullsized":"2024-09-11T23:16:35.942309+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:36:06.907622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"85'3","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:36.971236+0000","last_change":"2024-09-11T23:16:36.228991+0000","last_active":"2024-09-11T23:16:36.971236+0000","last_peered":"2024-09-11T23:16:36.971236+0000","last_clean":"2024-09-11T23:16:36.971236+0000","last_became_active":"2024-09-11T23:16:36.227701+0000","last_became_peered":"2024-09-11T23:16:36.227701+0000","last_unstale":"2024-09-11T23:16:36.971236+0000","last_undegraded":"2024-09-11T23:16:36.971236+0000","last_fullsized":"2024-09-11T23:16:36.971236+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:37:57.478572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:36.226661+0000","last_change":"2024-09-11T23:16:25.894803+0000","last_active":"2024-09-11T23:16:36.226661+0000","last_peered":"2024-09-11T23:16:36.226661+0000","last_clean":"2024-09-11T23:16:36.226661+0000","last_became_active":"2024-09-11T23:16:25.894429+0000","last_became_peered":"2024-09-11T23:16:25.894429+0000","last_unstale":"2024-09-11T23:16:36.226661+0000","last_undegraded":"2024-09-11T23:16:36.226661+0000","last_fullsized":"2024-09-11T23:16:36.226661+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:39:15.868910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"83'4","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.945644+0000","last_change":"2024-09-11T23:16:23.794271+0000","last_active":"2024-09-11T23:16:35.945644+0000","last_peered":"2024-09-11T23:16:35.945644+0000","last_clean":"2024-09-11T23:16:35.945644+0000","last_became_active":"2024-09-11T23:16:23.793919+0000","last_became_peered":"2024-09-11T23:16:23.793919+0000","last_unstale":"2024-09-11T23:16:35.945644+0000","last_undegraded":"2024-09-11T23:16:35.945644+0000","last_fullsized":"2024-09-11T23:16:35.945644+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:30:28.907312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.941192+0000","last_change":"2024-09-11T23:16:25.900969+0000","last_active":"2024-09-11T23:16:35.941192+0000","last_peered":"2024-09-11T23:16:35.941192+0000","last_clean":"2024-09-11T23:16:35.941192+0000","last_became_active":"2024-09-11T23:16:25.900425+0000","last_became_peered":"2024-09-11T23:16:25.900425+0000","last_unstale":"2024-09-11T23:16:35.941192+0000","last_undegraded":"2024-09-11T23:16:35.941192+0000","last_fullsized":"2024-09-11T23:16:35.941192+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:03:25.185773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.941358+0000","last_change":"2024-09-11T23:16:25.911605+0000","last_active":"2024-09-11T23:16:35.941358+0000","last_peered":"2024-09-11T23:16:35.941358+0000","last_clean":"2024-09-11T23:16:35.941358+0000","last_became_active":"2024-09-11T23:16:25.911180+0000","last_became_peered":"2024-09-11T23:16:25.911180+0000","last_unstale":"2024-09-11T23:16:35.941358+0000","last_undegraded":"2024-09-11T23:16:35.941358+0000","last_fullsized":"2024-09-11T23:16:35.941358+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:55:38.062606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"85'5","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:37.979007+0000","last_change":"2024-09-11T23:16:35.949528+0000","last_active":"2024-09-11T23:16:37.979007+0000","last_peered":"2024-09-11T23:16:37.979007+0000","last_clean":"2024-09-11T23:16:37.979007+0000","last_became_active":"2024-09-11T23:16:35.945679+0000","last_became_peered":"2024-09-11T23:16:35.945679+0000","last_unstale":"2024-09-11T23:16:37.979007+0000","last_undegraded":"2024-09-11T23:16:37.979007+0000","last_fullsized":"2024-09-11T23:16:37.979007+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:14:24.706044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.938792+0000","last_change":"2024-09-11T23:16:25.900526+0000","last_active":"2024-09-11T23:16:35.938792+0000","last_peered":"2024-09-11T23:16:35.938792+0000","last_clean":"2024-09-11T23:16:35.938792+0000","last_became_active":"2024-09-11T23:16:25.900148+0000","last_became_peered":"2024-09-11T23:16:25.900148+0000","last_unstale":"2024-09-11T23:16:35.938792+0000","last_undegraded":"2024-09-11T23:16:35.938792+0000","last_fullsized":"2024-09-11T23:16:35.938792+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:26:10.582473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"85'12","reported_seq":45,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:38.986047+0000","last_change":"2024-09-11T23:16:36.233273+0000","last_active":"2024-09-11T23:16:38.986047+0000","last_peered":"2024-09-11T23:16:38.986047+0000","last_clean":"2024-09-11T23:16:38.986047+0000","last_became_active":"2024-09-11T23:16:36.228358+0000","last_became_peered":"2024-09-11T23:16:36.228358+0000","last_unstale":"2024-09-11T23:16:38.986047+0000","last_undegraded":"2024-09-11T23:16:38.986047+0000","last_fullsized":"2024-09-11T23:16:38.986047+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:06:31.225795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:36.226734+0000","last_change":"2024-09-11T23:16:25.907874+0000","last_active":"2024-09-11T23:16:36.226734+0000","last_peered":"2024-09-11T23:16:36.226734+0000","last_clean":"2024-09-11T23:16:36.226734+0000","last_became_active":"2024-09-11T23:16:25.907491+0000","last_became_peered":"2024-09-11T23:16:25.907491+0000","last_unstale":"2024-09-11T23:16:36.226734+0000","last_undegraded":"2024-09-11T23:16:36.226734+0000","last_fullsized":"2024-09-11T23:16:36.226734+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:09:20.984249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"84'2","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:36.227306+0000","last_change":"2024-09-11T23:16:24.074355+0000","last_active":"2024-09-11T23:16:36.227306+0000","last_peered":"2024-09-11T23:16:36.227306+0000","last_clean":"2024-09-11T23:16:36.227306+0000","last_became_active":"2024-09-11T23:16:24.074047+0000","last_became_peered":"2024-09-11T23:16:24.074047+0000","last_unstale":"2024-09-11T23:16:36.227306+0000","last_undegraded":"2024-09-11T23:16:36.227306+0000","last_fullsized":"2024-09-11T23:16:36.227306+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:31:58.804420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:36.226393+0000","last_change":"2024-09-11T23:16:25.907701+0000","last_active":"2024-09-11T23:16:36.226393+0000","last_peered":"2024-09-11T23:16:36.226393+0000","last_clean":"2024-09-11T23:16:36.226393+0000","last_became_active":"2024-09-11T23:16:25.907379+0000","last_became_peered":"2024-09-11T23:16:25.907379+0000","last_unstale":"2024-09-11T23:16:36.226393+0000","last_undegraded":"2024-09-11T23:16:36.226393+0000","last_fullsized":"2024-09-11T23:16:36.226393+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:12:03.639055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"85'13","reported_seq":43,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:43.994432+0000","last_change":"2024-09-11T23:16:23.794293+0000","last_active":"2024-09-11T23:16:43.994432+0000","last_peered":"2024-09-11T23:16:43.994432+0000","last_clean":"2024-09-11T23:16:43.994432+0000","last_became_active":"2024-09-11T23:16:23.793974+0000","last_became_peered":"2024-09-11T23:16:23.793974+0000","last_unstale":"2024-09-11T23:16:43.994432+0000","last_undegraded":"2024-09-11T23:16:43.994432+0000","last_fullsized":"2024-09-11T23:16:43.994432+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:24:39.170188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":47880,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":120,"num_read_kb":112,"num_write":224,"num_write_kb":108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":520192,"data_stored":155114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":195,"ondisk_log_size":195,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":77,"seq":330712481809,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32108,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713044,"statfs":{"total":95995035648,"available":95962157056,"internally_reserved":0,"allocated":946176,"data_stored":449945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":70,"seq":300647710741,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28608,"kb_used_data":1584,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716544,"statfs":{"total":95995035648,"available":95965741056,"internally_reserved":0,"allocated":1622016,"data_stored":1081064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":65,"seq":279172874267,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":966656,"data_stored":448900,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":57,"seq":244813135905,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":958464,"data_stored":448696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":50,"seq":214748364837,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28548,"kb_used_data":1524,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716604,"statfs":{"total":95995035648,"available":95965802496,"internally_reserved":0,"allocated":1560576,"data_stored":1041405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528361,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36272,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708880,"statfs":{"total":95995035648,"available":95957893120,"internally_reserved":0,"allocated":950272,"data_stored":448617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790000,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28560,"kb_used_data":1536,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716592,"statfs":{"total":95995035648,"available":95965790208,"internally_reserved":0,"allocated":1572864,"data_stored":1041083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920820,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36272,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708880,"statfs":{"total":95995035648,"available":95957893120,"internally_reserved":0,"allocated":950272,"data_stored":448478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182458,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40448,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704704,"statfs":{"total":95995035648,"available":95953616896,"internally_reserved":0,"allocated":966656,"data_stored":448442,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411391,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36340,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708812,"statfs":{"total":95995035648,"available":95957823488,"internally_reserved":0,"allocated":1019904,"data_stored":496945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574916,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":999424,"data_stored":496332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738441,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":970752,"data_stored":450349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":48554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":49167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":42801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T23:16:52.291 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-11T23:16:52.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:52 smithi136 ceph-mon[35121]: from='client.14694 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T23:16:52.550 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:16:52.552 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:52 smithi076 ceph-mon[35296]: from='client.14694 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T23:16:53.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:53.022+0000 7fe642b0f640 1 -- 172.21.15.26:0/2698807779 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe63c10c170 msgr2=0x7fe63c10e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:16:53.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:53.022+0000 7fe642b0f640 1 --2- 172.21.15.26:0/2698807779 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe63c10c170 0x7fe63c10e610 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fe634009ca0 tx=0x7fe63402a950 comp rx=0 tx=0).stop 2024-09-11T23:16:53.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:53.022+0000 7fe642b0f640 1 -- 172.21.15.26:0/2698807779 shutdown_connections 2024-09-11T23:16:53.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:53.022+0000 7fe642b0f640 1 --2- 172.21.15.26:0/2698807779 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe63c103660 0x7fe63c10bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:53.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:53.022+0000 7fe642b0f640 1 --2- 172.21.15.26:0/2698807779 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe63c10c170 0x7fe63c10e610 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:53.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:53.022+0000 7fe642b0f640 1 --2- 172.21.15.26:0/2698807779 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe63c102d40 0x7fe63c103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:53.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:53.022+0000 7fe642b0f640 1 -- 172.21.15.26:0/2698807779 >> 172.21.15.26:0/2698807779 conn(0x7fe63c0fc7a0 msgr2=0x7fe63c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:16:53.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:53.022+0000 7fe642b0f640 1 -- 172.21.15.26:0/2698807779 shutdown_connections 2024-09-11T23:16:53.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:53.022+0000 7fe642b0f640 1 -- 172.21.15.26:0/2698807779 wait complete. 2024-09-11T23:16:53.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:53.023+0000 7fe642b0f640 1 Processor -- start 2024-09-11T23:16:53.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:53.023+0000 7fe642b0f640 1 -- start start 2024-09-11T23:16:53.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:53.023+0000 7fe642b0f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe63c102d40 0x7fe63c1009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:16:53.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:53.023+0000 7fe642b0f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe63c103660 0x7fe63c100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:16:53.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:53.023+0000 7fe642b0f640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe63c10c170 0x7fe63c101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:16:53.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:53.023+0000 7fe642b0f640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fe63c111f70 con 0x7fe63c102d40 2024-09-11T23:16:53.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:53.023+0000 7fe642b0f640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fe63c111df0 con 0x7fe63c103660 2024-09-11T23:16:53.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:53.023+0000 7fe642b0f640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe63c1120f0 con 0x7fe63c10c170 2024-09-11T23:16:53.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:53.023+0000 7fe6408ab640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe63c102d40 0x7fe63c1009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:16:53.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:53.023+0000 7fe6408ab640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe63c102d40 0x7fe63c1009a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42110/0 (socket says 172.21.15.26:42110) 2024-09-11T23:16:53.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:53.023+0000 7fe6408ab640 1 -- 172.21.15.26:0/2323788815 learned_addr learned my addr 172.21.15.26:0/2323788815 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:16:53.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:53.023+0000 7fe6410ac640 1 --2- 172.21.15.26:0/2323788815 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe63c10c170 0x7fe63c101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:16:53.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:53.024+0000 7fe63bfff640 1 --2- 172.21.15.26:0/2323788815 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe63c103660 0x7fe63c100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:16:53.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:53.024+0000 7fe6408ab640 1 -- 172.21.15.26:0/2323788815 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe63c10c170 msgr2=0x7fe63c101420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:16:53.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:53.024+0000 7fe6408ab640 1 --2- 172.21.15.26:0/2323788815 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe63c10c170 0x7fe63c101420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:53.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:53.024+0000 7fe6408ab640 1 -- 172.21.15.26:0/2323788815 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe63c103660 msgr2=0x7fe63c100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:16:53.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:53.024+0000 7fe6408ab640 1 --2- 172.21.15.26:0/2323788815 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe63c103660 0x7fe63c100ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:53.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:53.024+0000 7fe6408ab640 1 -- 172.21.15.26:0/2323788815 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe63c1a68d0 con 0x7fe63c102d40 2024-09-11T23:16:53.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:53.024+0000 7fe6410ac640 1 --2- 172.21.15.26:0/2323788815 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe63c10c170 0x7fe63c101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:16:53.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:53.024+0000 7fe6408ab640 1 --2- 172.21.15.26:0/2323788815 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe63c102d40 0x7fe63c1009a0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fe63000d770 tx=0x7fe63000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:16:53.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:53.024+0000 7fe639ffb640 1 -- 172.21.15.26:0/2323788815 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe630017070 con 0x7fe63c102d40 2024-09-11T23:16:53.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:53.024+0000 7fe639ffb640 1 -- 172.21.15.26:0/2323788815 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe630005500 con 0x7fe63c102d40 2024-09-11T23:16:53.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:53.024+0000 7fe642b0f640 1 -- 172.21.15.26:0/2323788815 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe63c1a6bc0 con 0x7fe63c102d40 2024-09-11T23:16:53.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:53.024+0000 7fe639ffb640 1 -- 172.21.15.26:0/2323788815 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe630004e00 con 0x7fe63c102d40 2024-09-11T23:16:53.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:53.024+0000 7fe642b0f640 1 -- 172.21.15.26:0/2323788815 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe63c1a7100 con 0x7fe63c102d40 2024-09-11T23:16:53.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:53.025+0000 7fe642b0f640 1 -- 172.21.15.26:0/2323788815 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe63c103120 con 0x7fe63c102d40 2024-09-11T23:16:53.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:53.026+0000 7fe639ffb640 1 -- 172.21.15.26:0/2323788815 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fe630004fa0 con 0x7fe63c102d40 2024-09-11T23:16:53.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:53.027+0000 7fe639ffb640 1 --2- 172.21.15.26:0/2323788815 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fe62c0a2c90 0x7fe62c0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:16:53.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:53.027+0000 7fe63bfff640 1 --2- 172.21.15.26:0/2323788815 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fe62c0a2c90 0x7fe62c0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:16:53.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:53.027+0000 7fe639ffb640 1 -- 172.21.15.26:0/2323788815 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fe6300cebe0 con 0x7fe63c102d40 2024-09-11T23:16:53.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:53.028+0000 7fe63bfff640 1 --2- 172.21.15.26:0/2323788815 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fe62c0a2c90 0x7fe62c0a5150 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fe63c102430 tx=0x7fe62800b2e0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:16:53.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:53.031+0000 7fe639ffb640 1 -- 172.21.15.26:0/2323788815 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe63009b9e0 con 0x7fe63c102d40 2024-09-11T23:16:53.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:53.206+0000 7fe642b0f640 1 -- 172.21.15.26:0/2323788815 --> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe63c0feac0 con 0x7fe62c0a2c90 2024-09-11T23:16:53.207 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:53 smithi026 ceph-mon[31172]: pgmap v294: 97 pgs: 97 active+clean; 629 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.1 KiB/s wr, 16 op/s; 0 B/s, 0 objects/s recovering 2024-09-11T23:16:53.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:53.213+0000 7fe639ffb640 1 -- 172.21.15.26:0/2323788815 <== mgr.14148 v2:172.21.15.26:6800/110833859 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258751 (secure 0 0 0) 0x7fe63c0feac0 con 0x7fe62c0a2c90 2024-09-11T23:16:53.216 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-11T23:16:53.216 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-09-11T23:16:53.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:53.217+0000 7fe642b0f640 1 -- 172.21.15.26:0/2323788815 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fe62c0a2c90 msgr2=0x7fe62c0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:16:53.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:53.217+0000 7fe642b0f640 1 --2- 172.21.15.26:0/2323788815 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fe62c0a2c90 0x7fe62c0a5150 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fe63c102430 tx=0x7fe62800b2e0 comp rx=0 tx=0).stop 2024-09-11T23:16:53.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:53.217+0000 7fe642b0f640 1 -- 172.21.15.26:0/2323788815 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe63c102d40 msgr2=0x7fe63c1009a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:16:53.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:53.217+0000 7fe642b0f640 1 --2- 172.21.15.26:0/2323788815 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe63c102d40 0x7fe63c1009a0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fe63000d770 tx=0x7fe63000dc30 comp rx=0 tx=0).stop 2024-09-11T23:16:53.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:53.217+0000 7fe642b0f640 1 -- 172.21.15.26:0/2323788815 shutdown_connections 2024-09-11T23:16:53.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:53.217+0000 7fe642b0f640 1 --2- 172.21.15.26:0/2323788815 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fe62c0a2c90 0x7fe62c0a5150 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:53.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:53.217+0000 7fe642b0f640 1 --2- 172.21.15.26:0/2323788815 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe63c103660 0x7fe63c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:53.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:53.217+0000 7fe642b0f640 1 --2- 172.21.15.26:0/2323788815 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe63c10c170 0x7fe63c101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:53.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:53.217+0000 7fe642b0f640 1 --2- 172.21.15.26:0/2323788815 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe63c102d40 0x7fe63c1009a0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:53.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:53.217+0000 7fe642b0f640 1 -- 172.21.15.26:0/2323788815 >> 172.21.15.26:0/2323788815 conn(0x7fe63c0fc7a0 msgr2=0x7fe63c0fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:16:53.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:53.217+0000 7fe642b0f640 1 -- 172.21.15.26:0/2323788815 shutdown_connections 2024-09-11T23:16:53.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:53.217+0000 7fe642b0f640 1 -- 172.21.15.26:0/2323788815 wait complete. 2024-09-11T23:16:53.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:53 smithi136 ceph-mon[35121]: pgmap v294: 97 pgs: 97 active+clean; 629 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.1 KiB/s wr, 16 op/s; 0 B/s, 0 objects/s recovering 2024-09-11T23:16:53.553 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:53 smithi076 ceph-mon[35296]: pgmap v294: 97 pgs: 97 active+clean; 629 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.1 KiB/s wr, 16 op/s; 0 B/s, 0 objects/s recovering 2024-09-11T23:16:53.934 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":294,"stamp":"2024-09-11T23:16:52.085541+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":644219,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":287,"num_read_kb":257,"num_write":414,"num_write_kb":3056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":338,"ondisk_log_size":338,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":379068,"kb_used_data":13176,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124562756,"statfs":{"total":1151940427776,"available":1151552262144,"internally_reserved":0,"allocated":13492224,"data_stored":7305993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":374453561},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":46635,"num_objects":20,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":-1,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":134,"num_read_kb":122,"num_write":66,"num_write_kb":61,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002869"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.941549+0000","last_change":"2024-09-11T23:16:25.899106+0000","last_active":"2024-09-11T23:16:35.941549+0000","last_peered":"2024-09-11T23:16:35.941549+0000","last_clean":"2024-09-11T23:16:35.941549+0000","last_became_active":"2024-09-11T23:16:25.898654+0000","last_became_peered":"2024-09-11T23:16:25.898654+0000","last_unstale":"2024-09-11T23:16:35.941549+0000","last_undegraded":"2024-09-11T23:16:35.941549+0000","last_fullsized":"2024-09-11T23:16:35.941549+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:21:41.490905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:36.226970+0000","last_change":"2024-09-11T23:16:25.895629+0000","last_active":"2024-09-11T23:16:36.226970+0000","last_peered":"2024-09-11T23:16:36.226970+0000","last_clean":"2024-09-11T23:16:36.226970+0000","last_became_active":"2024-09-11T23:16:25.895308+0000","last_became_peered":"2024-09-11T23:16:25.895308+0000","last_unstale":"2024-09-11T23:16:36.226970+0000","last_undegraded":"2024-09-11T23:16:36.226970+0000","last_fullsized":"2024-09-11T23:16:36.226970+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:39:42.549970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:36.226468+0000","last_change":"2024-09-11T23:16:25.907113+0000","last_active":"2024-09-11T23:16:36.226468+0000","last_peered":"2024-09-11T23:16:36.226468+0000","last_clean":"2024-09-11T23:16:36.226468+0000","last_became_active":"2024-09-11T23:16:25.906803+0000","last_became_peered":"2024-09-11T23:16:25.906803+0000","last_unstale":"2024-09-11T23:16:36.226468+0000","last_undegraded":"2024-09-11T23:16:36.226468+0000","last_fullsized":"2024-09-11T23:16:36.226468+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:36:49.303932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.946167+0000","last_change":"2024-09-11T23:16:35.946167+0000","last_active":"2024-09-11T23:16:35.946167+0000","last_peered":"2024-09-11T23:16:35.946167+0000","last_clean":"2024-09-11T23:16:35.946167+0000","last_became_active":"2024-09-11T23:16:35.945786+0000","last_became_peered":"2024-09-11T23:16:35.945786+0000","last_unstale":"2024-09-11T23:16:35.946167+0000","last_undegraded":"2024-09-11T23:16:35.946167+0000","last_fullsized":"2024-09-11T23:16:35.946167+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:13:03.064910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.944477+0000","last_change":"2024-09-11T23:16:25.897909+0000","last_active":"2024-09-11T23:16:35.944477+0000","last_peered":"2024-09-11T23:16:35.944477+0000","last_clean":"2024-09-11T23:16:35.944477+0000","last_became_active":"2024-09-11T23:16:25.897569+0000","last_became_peered":"2024-09-11T23:16:25.897569+0000","last_unstale":"2024-09-11T23:16:35.944477+0000","last_undegraded":"2024-09-11T23:16:35.944477+0000","last_fullsized":"2024-09-11T23:16:35.944477+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:25:44.727254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.942367+0000","last_change":"2024-09-11T23:16:25.897594+0000","last_active":"2024-09-11T23:16:35.942367+0000","last_peered":"2024-09-11T23:16:35.942367+0000","last_clean":"2024-09-11T23:16:35.942367+0000","last_became_active":"2024-09-11T23:16:25.897223+0000","last_became_peered":"2024-09-11T23:16:25.897223+0000","last_unstale":"2024-09-11T23:16:35.942367+0000","last_undegraded":"2024-09-11T23:16:35.942367+0000","last_fullsized":"2024-09-11T23:16:35.942367+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:10:50.763455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.943511+0000","last_change":"2024-09-11T23:16:35.943511+0000","last_active":"2024-09-11T23:16:35.943511+0000","last_peered":"2024-09-11T23:16:35.943511+0000","last_clean":"2024-09-11T23:16:35.943511+0000","last_became_active":"2024-09-11T23:16:35.943075+0000","last_became_peered":"2024-09-11T23:16:35.943075+0000","last_unstale":"2024-09-11T23:16:35.943511+0000","last_undegraded":"2024-09-11T23:16:35.943511+0000","last_fullsized":"2024-09-11T23:16:35.943511+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:53:43.892421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:36.226370+0000","last_change":"2024-09-11T23:16:25.903380+0000","last_active":"2024-09-11T23:16:36.226370+0000","last_peered":"2024-09-11T23:16:36.226370+0000","last_clean":"2024-09-11T23:16:36.226370+0000","last_became_active":"2024-09-11T23:16:25.903093+0000","last_became_peered":"2024-09-11T23:16:25.903093+0000","last_unstale":"2024-09-11T23:16:36.226370+0000","last_undegraded":"2024-09-11T23:16:36.226370+0000","last_fullsized":"2024-09-11T23:16:36.226370+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:58:50.944623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.939171+0000","last_change":"2024-09-11T23:16:25.908166+0000","last_active":"2024-09-11T23:16:35.939171+0000","last_peered":"2024-09-11T23:16:35.939171+0000","last_clean":"2024-09-11T23:16:35.939171+0000","last_became_active":"2024-09-11T23:16:25.907749+0000","last_became_peered":"2024-09-11T23:16:25.907749+0000","last_unstale":"2024-09-11T23:16:35.939171+0000","last_undegraded":"2024-09-11T23:16:35.939171+0000","last_fullsized":"2024-09-11T23:16:35.939171+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:11:16.705931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.944259+0000","last_change":"2024-09-11T23:16:35.944259+0000","last_active":"2024-09-11T23:16:35.944259+0000","last_peered":"2024-09-11T23:16:35.944259+0000","last_clean":"2024-09-11T23:16:35.944259+0000","last_became_active":"2024-09-11T23:16:35.943767+0000","last_became_peered":"2024-09-11T23:16:35.943767+0000","last_unstale":"2024-09-11T23:16:35.944259+0000","last_undegraded":"2024-09-11T23:16:35.944259+0000","last_fullsized":"2024-09-11T23:16:35.944259+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:20:13.127854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.941922+0000","last_change":"2024-09-11T23:16:25.903256+0000","last_active":"2024-09-11T23:16:35.941922+0000","last_peered":"2024-09-11T23:16:35.941922+0000","last_clean":"2024-09-11T23:16:35.941922+0000","last_became_active":"2024-09-11T23:16:25.902954+0000","last_became_peered":"2024-09-11T23:16:25.902954+0000","last_unstale":"2024-09-11T23:16:35.941922+0000","last_undegraded":"2024-09-11T23:16:35.941922+0000","last_fullsized":"2024-09-11T23:16:35.941922+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:43:48.651400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:36.226960+0000","last_change":"2024-09-11T23:16:25.897656+0000","last_active":"2024-09-11T23:16:36.226960+0000","last_peered":"2024-09-11T23:16:36.226960+0000","last_clean":"2024-09-11T23:16:36.226960+0000","last_became_active":"2024-09-11T23:16:25.897103+0000","last_became_peered":"2024-09-11T23:16:25.897103+0000","last_unstale":"2024-09-11T23:16:36.226960+0000","last_undegraded":"2024-09-11T23:16:36.226960+0000","last_fullsized":"2024-09-11T23:16:36.226960+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:14:34.341383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.938158+0000","last_change":"2024-09-11T23:16:25.902800+0000","last_active":"2024-09-11T23:16:35.938158+0000","last_peered":"2024-09-11T23:16:35.938158+0000","last_clean":"2024-09-11T23:16:35.938158+0000","last_became_active":"2024-09-11T23:16:25.902414+0000","last_became_peered":"2024-09-11T23:16:25.902414+0000","last_unstale":"2024-09-11T23:16:35.938158+0000","last_undegraded":"2024-09-11T23:16:35.938158+0000","last_fullsized":"2024-09-11T23:16:35.938158+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:16:32.427749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:36.226337+0000","last_change":"2024-09-11T23:16:25.894410+0000","last_active":"2024-09-11T23:16:36.226337+0000","last_peered":"2024-09-11T23:16:36.226337+0000","last_clean":"2024-09-11T23:16:36.226337+0000","last_became_active":"2024-09-11T23:16:25.894067+0000","last_became_peered":"2024-09-11T23:16:25.894067+0000","last_unstale":"2024-09-11T23:16:36.226337+0000","last_undegraded":"2024-09-11T23:16:36.226337+0000","last_fullsized":"2024-09-11T23:16:36.226337+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:03:11.913063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:36.226260+0000","last_change":"2024-09-11T23:16:25.903440+0000","last_active":"2024-09-11T23:16:36.226260+0000","last_peered":"2024-09-11T23:16:36.226260+0000","last_clean":"2024-09-11T23:16:36.226260+0000","last_became_active":"2024-09-11T23:16:25.903122+0000","last_became_peered":"2024-09-11T23:16:25.903122+0000","last_unstale":"2024-09-11T23:16:36.226260+0000","last_undegraded":"2024-09-11T23:16:36.226260+0000","last_fullsized":"2024-09-11T23:16:36.226260+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:03:41.065314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.938680+0000","last_change":"2024-09-11T23:16:25.905336+0000","last_active":"2024-09-11T23:16:35.938680+0000","last_peered":"2024-09-11T23:16:35.938680+0000","last_clean":"2024-09-11T23:16:35.938680+0000","last_became_active":"2024-09-11T23:16:25.904699+0000","last_became_peered":"2024-09-11T23:16:25.904699+0000","last_unstale":"2024-09-11T23:16:35.938680+0000","last_undegraded":"2024-09-11T23:16:35.938680+0000","last_fullsized":"2024-09-11T23:16:35.938680+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:16:13.448585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.941545+0000","last_change":"2024-09-11T23:16:25.897977+0000","last_active":"2024-09-11T23:16:35.941545+0000","last_peered":"2024-09-11T23:16:35.941545+0000","last_clean":"2024-09-11T23:16:35.941545+0000","last_became_active":"2024-09-11T23:16:25.897672+0000","last_became_peered":"2024-09-11T23:16:25.897672+0000","last_unstale":"2024-09-11T23:16:35.941545+0000","last_undegraded":"2024-09-11T23:16:35.941545+0000","last_fullsized":"2024-09-11T23:16:35.941545+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:42:56.575066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.939453+0000","last_change":"2024-09-11T23:16:25.897063+0000","last_active":"2024-09-11T23:16:35.939453+0000","last_peered":"2024-09-11T23:16:35.939453+0000","last_clean":"2024-09-11T23:16:35.939453+0000","last_became_active":"2024-09-11T23:16:25.896770+0000","last_became_peered":"2024-09-11T23:16:25.896770+0000","last_unstale":"2024-09-11T23:16:35.939453+0000","last_undegraded":"2024-09-11T23:16:35.939453+0000","last_fullsized":"2024-09-11T23:16:35.939453+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:55:44.735171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.939570+0000","last_change":"2024-09-11T23:16:25.903423+0000","last_active":"2024-09-11T23:16:35.939570+0000","last_peered":"2024-09-11T23:16:35.939570+0000","last_clean":"2024-09-11T23:16:35.939570+0000","last_became_active":"2024-09-11T23:16:25.902906+0000","last_became_peered":"2024-09-11T23:16:25.902906+0000","last_unstale":"2024-09-11T23:16:35.939570+0000","last_undegraded":"2024-09-11T23:16:35.939570+0000","last_fullsized":"2024-09-11T23:16:35.939570+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:40:49.326045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.942800+0000","last_change":"2024-09-11T23:16:25.896564+0000","last_active":"2024-09-11T23:16:35.942800+0000","last_peered":"2024-09-11T23:16:35.942800+0000","last_clean":"2024-09-11T23:16:35.942800+0000","last_became_active":"2024-09-11T23:16:25.896301+0000","last_became_peered":"2024-09-11T23:16:25.896301+0000","last_unstale":"2024-09-11T23:16:35.942800+0000","last_undegraded":"2024-09-11T23:16:35.942800+0000","last_fullsized":"2024-09-11T23:16:35.942800+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:30:30.074853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.938246+0000","last_change":"2024-09-11T23:16:25.903491+0000","last_active":"2024-09-11T23:16:35.938246+0000","last_peered":"2024-09-11T23:16:35.938246+0000","last_clean":"2024-09-11T23:16:35.938246+0000","last_became_active":"2024-09-11T23:16:25.901340+0000","last_became_peered":"2024-09-11T23:16:25.901340+0000","last_unstale":"2024-09-11T23:16:35.938246+0000","last_undegraded":"2024-09-11T23:16:35.938246+0000","last_fullsized":"2024-09-11T23:16:35.938246+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:41:16.058034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.938800+0000","last_change":"2024-09-11T23:16:25.901084+0000","last_active":"2024-09-11T23:16:35.938800+0000","last_peered":"2024-09-11T23:16:35.938800+0000","last_clean":"2024-09-11T23:16:35.938800+0000","last_became_active":"2024-09-11T23:16:25.900726+0000","last_became_peered":"2024-09-11T23:16:25.900726+0000","last_unstale":"2024-09-11T23:16:35.938800+0000","last_undegraded":"2024-09-11T23:16:35.938800+0000","last_fullsized":"2024-09-11T23:16:35.938800+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:57:40.955645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:36.226559+0000","last_change":"2024-09-11T23:16:25.900461+0000","last_active":"2024-09-11T23:16:36.226559+0000","last_peered":"2024-09-11T23:16:36.226559+0000","last_clean":"2024-09-11T23:16:36.226559+0000","last_became_active":"2024-09-11T23:16:25.900174+0000","last_became_peered":"2024-09-11T23:16:25.900174+0000","last_unstale":"2024-09-11T23:16:36.226559+0000","last_undegraded":"2024-09-11T23:16:36.226559+0000","last_fullsized":"2024-09-11T23:16:36.226559+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:51:50.953198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.940957+0000","last_change":"2024-09-11T23:16:25.907569+0000","last_active":"2024-09-11T23:16:35.940957+0000","last_peered":"2024-09-11T23:16:35.940957+0000","last_clean":"2024-09-11T23:16:35.940957+0000","last_became_active":"2024-09-11T23:16:25.907091+0000","last_became_peered":"2024-09-11T23:16:25.907091+0000","last_unstale":"2024-09-11T23:16:35.940957+0000","last_undegraded":"2024-09-11T23:16:35.940957+0000","last_fullsized":"2024-09-11T23:16:35.940957+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:53:45.961167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:36.226935+0000","last_change":"2024-09-11T23:16:25.903246+0000","last_active":"2024-09-11T23:16:36.226935+0000","last_peered":"2024-09-11T23:16:36.226935+0000","last_clean":"2024-09-11T23:16:36.226935+0000","last_became_active":"2024-09-11T23:16:25.902907+0000","last_became_peered":"2024-09-11T23:16:25.902907+0000","last_unstale":"2024-09-11T23:16:36.226935+0000","last_undegraded":"2024-09-11T23:16:36.226935+0000","last_fullsized":"2024-09-11T23:16:36.226935+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:20:38.868384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.939763+0000","last_change":"2024-09-11T23:16:25.906298+0000","last_active":"2024-09-11T23:16:35.939763+0000","last_peered":"2024-09-11T23:16:35.939763+0000","last_clean":"2024-09-11T23:16:35.939763+0000","last_became_active":"2024-09-11T23:16:25.905350+0000","last_became_peered":"2024-09-11T23:16:25.905350+0000","last_unstale":"2024-09-11T23:16:35.939763+0000","last_undegraded":"2024-09-11T23:16:35.939763+0000","last_fullsized":"2024-09-11T23:16:35.939763+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:04:22.122514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.941073+0000","last_change":"2024-09-11T23:16:25.902114+0000","last_active":"2024-09-11T23:16:35.941073+0000","last_peered":"2024-09-11T23:16:35.941073+0000","last_clean":"2024-09-11T23:16:35.941073+0000","last_became_active":"2024-09-11T23:16:25.901742+0000","last_became_peered":"2024-09-11T23:16:25.901742+0000","last_unstale":"2024-09-11T23:16:35.941073+0000","last_undegraded":"2024-09-11T23:16:35.941073+0000","last_fullsized":"2024-09-11T23:16:35.941073+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:02:10.640801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.938325+0000","last_change":"2024-09-11T23:16:25.898117+0000","last_active":"2024-09-11T23:16:35.938325+0000","last_peered":"2024-09-11T23:16:35.938325+0000","last_clean":"2024-09-11T23:16:35.938325+0000","last_became_active":"2024-09-11T23:16:25.897818+0000","last_became_peered":"2024-09-11T23:16:25.897818+0000","last_unstale":"2024-09-11T23:16:35.938325+0000","last_undegraded":"2024-09-11T23:16:35.938325+0000","last_fullsized":"2024-09-11T23:16:35.938325+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:27:37.175471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:36.226573+0000","last_change":"2024-09-11T23:16:25.905600+0000","last_active":"2024-09-11T23:16:36.226573+0000","last_peered":"2024-09-11T23:16:36.226573+0000","last_clean":"2024-09-11T23:16:36.226573+0000","last_became_active":"2024-09-11T23:16:25.903924+0000","last_became_peered":"2024-09-11T23:16:25.903924+0000","last_unstale":"2024-09-11T23:16:36.226573+0000","last_undegraded":"2024-09-11T23:16:36.226573+0000","last_fullsized":"2024-09-11T23:16:36.226573+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:20:29.929079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.939290+0000","last_change":"2024-09-11T23:16:25.904763+0000","last_active":"2024-09-11T23:16:35.939290+0000","last_peered":"2024-09-11T23:16:35.939290+0000","last_clean":"2024-09-11T23:16:35.939290+0000","last_became_active":"2024-09-11T23:16:25.904406+0000","last_became_peered":"2024-09-11T23:16:25.904406+0000","last_unstale":"2024-09-11T23:16:35.939290+0000","last_undegraded":"2024-09-11T23:16:35.939290+0000","last_fullsized":"2024-09-11T23:16:35.939290+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:27:38.918017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.939346+0000","last_change":"2024-09-11T23:16:25.906573+0000","last_active":"2024-09-11T23:16:35.939346+0000","last_peered":"2024-09-11T23:16:35.939346+0000","last_clean":"2024-09-11T23:16:35.939346+0000","last_became_active":"2024-09-11T23:16:25.904771+0000","last_became_peered":"2024-09-11T23:16:25.904771+0000","last_unstale":"2024-09-11T23:16:35.939346+0000","last_undegraded":"2024-09-11T23:16:35.939346+0000","last_fullsized":"2024-09-11T23:16:35.939346+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:04:17.090648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.942191+0000","last_change":"2024-09-11T23:16:25.902123+0000","last_active":"2024-09-11T23:16:35.942191+0000","last_peered":"2024-09-11T23:16:35.942191+0000","last_clean":"2024-09-11T23:16:35.942191+0000","last_became_active":"2024-09-11T23:16:25.901756+0000","last_became_peered":"2024-09-11T23:16:25.901756+0000","last_unstale":"2024-09-11T23:16:35.942191+0000","last_undegraded":"2024-09-11T23:16:35.942191+0000","last_fullsized":"2024-09-11T23:16:35.942191+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:17:37.074976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.944098+0000","last_change":"2024-09-11T23:16:25.904217+0000","last_active":"2024-09-11T23:16:35.944098+0000","last_peered":"2024-09-11T23:16:35.944098+0000","last_clean":"2024-09-11T23:16:35.944098+0000","last_became_active":"2024-09-11T23:16:25.903522+0000","last_became_peered":"2024-09-11T23:16:25.903522+0000","last_unstale":"2024-09-11T23:16:35.944098+0000","last_undegraded":"2024-09-11T23:16:35.944098+0000","last_fullsized":"2024-09-11T23:16:35.944098+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:29:46.634475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"83'5","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:48.000881+0000","last_change":"2024-09-11T23:16:23.793696+0000","last_active":"2024-09-11T23:16:48.000881+0000","last_peered":"2024-09-11T23:16:48.000881+0000","last_clean":"2024-09-11T23:16:48.000881+0000","last_became_active":"2024-09-11T23:16:23.793062+0000","last_became_peered":"2024-09-11T23:16:23.793062+0000","last_unstale":"2024-09-11T23:16:48.000881+0000","last_undegraded":"2024-09-11T23:16:48.000881+0000","last_fullsized":"2024-09-11T23:16:48.000881+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:00:25.655817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":13,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.941878+0000","last_change":"2024-09-11T23:16:25.906673+0000","last_active":"2024-09-11T23:16:35.941878+0000","last_peered":"2024-09-11T23:16:35.941878+0000","last_clean":"2024-09-11T23:16:35.941878+0000","last_became_active":"2024-09-11T23:16:25.906239+0000","last_became_peered":"2024-09-11T23:16:25.906239+0000","last_unstale":"2024-09-11T23:16:35.941878+0000","last_undegraded":"2024-09-11T23:16:35.941878+0000","last_fullsized":"2024-09-11T23:16:35.941878+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:15:54.907269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"85'7","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:36.973576+0000","last_change":"2024-09-11T23:16:23.793302+0000","last_active":"2024-09-11T23:16:36.973576+0000","last_peered":"2024-09-11T23:16:36.973576+0000","last_clean":"2024-09-11T23:16:36.973576+0000","last_became_active":"2024-09-11T23:16:23.792874+0000","last_became_peered":"2024-09-11T23:16:23.792874+0000","last_unstale":"2024-09-11T23:16:36.973576+0000","last_undegraded":"2024-09-11T23:16:36.973576+0000","last_fullsized":"2024-09-11T23:16:36.973576+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:52:59.558223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"85'3","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.937984+0000","last_change":"2024-09-11T23:16:23.792999+0000","last_active":"2024-09-11T23:16:35.937984+0000","last_peered":"2024-09-11T23:16:35.937984+0000","last_clean":"2024-09-11T23:16:35.937984+0000","last_became_active":"2024-09-11T23:16:23.792630+0000","last_became_peered":"2024-09-11T23:16:23.792630+0000","last_unstale":"2024-09-11T23:16:35.937984+0000","last_undegraded":"2024-09-11T23:16:35.937984+0000","last_fullsized":"2024-09-11T23:16:35.937984+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:09:28.001060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.943722+0000","last_change":"2024-09-11T23:16:35.943722+0000","last_active":"2024-09-11T23:16:35.943722+0000","last_peered":"2024-09-11T23:16:35.943722+0000","last_clean":"2024-09-11T23:16:35.943722+0000","last_became_active":"2024-09-11T23:16:35.943355+0000","last_became_peered":"2024-09-11T23:16:35.943355+0000","last_unstale":"2024-09-11T23:16:35.943722+0000","last_undegraded":"2024-09-11T23:16:35.943722+0000","last_fullsized":"2024-09-11T23:16:35.943722+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:26:22.769824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,6],"acting":[9,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"85'3","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:37.978674+0000","last_change":"2024-09-11T23:16:23.797945+0000","last_active":"2024-09-11T23:16:37.978674+0000","last_peered":"2024-09-11T23:16:37.978674+0000","last_clean":"2024-09-11T23:16:37.978674+0000","last_became_active":"2024-09-11T23:16:23.797572+0000","last_became_peered":"2024-09-11T23:16:23.797572+0000","last_unstale":"2024-09-11T23:16:37.978674+0000","last_undegraded":"2024-09-11T23:16:37.978674+0000","last_fullsized":"2024-09-11T23:16:37.978674+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:04:59.017668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.939247+0000","last_change":"2024-09-11T23:16:25.891587+0000","last_active":"2024-09-11T23:16:35.939247+0000","last_peered":"2024-09-11T23:16:35.939247+0000","last_clean":"2024-09-11T23:16:35.939247+0000","last_became_active":"2024-09-11T23:16:25.891190+0000","last_became_peered":"2024-09-11T23:16:25.891190+0000","last_unstale":"2024-09-11T23:16:35.939247+0000","last_undegraded":"2024-09-11T23:16:35.939247+0000","last_fullsized":"2024-09-11T23:16:35.939247+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:57:05.177843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"85'4","reported_seq":50,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:50.002620+0000","last_change":"2024-09-11T23:16:23.790129+0000","last_active":"2024-09-11T23:16:50.002620+0000","last_peered":"2024-09-11T23:16:50.002620+0000","last_clean":"2024-09-11T23:16:50.002620+0000","last_became_active":"2024-09-11T23:16:23.789720+0000","last_became_peered":"2024-09-11T23:16:23.789720+0000","last_unstale":"2024-09-11T23:16:50.002620+0000","last_undegraded":"2024-09-11T23:16:50.002620+0000","last_fullsized":"2024-09-11T23:16:50.002620+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:33:58.388513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.939104+0000","last_change":"2024-09-11T23:16:25.901014+0000","last_active":"2024-09-11T23:16:35.939104+0000","last_peered":"2024-09-11T23:16:35.939104+0000","last_clean":"2024-09-11T23:16:35.939104+0000","last_became_active":"2024-09-11T23:16:25.900280+0000","last_became_peered":"2024-09-11T23:16:25.900280+0000","last_unstale":"2024-09-11T23:16:35.939104+0000","last_undegraded":"2024-09-11T23:16:35.939104+0000","last_fullsized":"2024-09-11T23:16:35.939104+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:14:03.662204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"85'4","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:37.978517+0000","last_change":"2024-09-11T23:16:23.797944+0000","last_active":"2024-09-11T23:16:37.978517+0000","last_peered":"2024-09-11T23:16:37.978517+0000","last_clean":"2024-09-11T23:16:37.978517+0000","last_became_active":"2024-09-11T23:16:23.797520+0000","last_became_peered":"2024-09-11T23:16:23.797520+0000","last_unstale":"2024-09-11T23:16:37.978517+0000","last_undegraded":"2024-09-11T23:16:37.978517+0000","last_fullsized":"2024-09-11T23:16:37.978517+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:29:06.957518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:36.226839+0000","last_change":"2024-09-11T23:16:25.905379+0000","last_active":"2024-09-11T23:16:36.226839+0000","last_peered":"2024-09-11T23:16:36.226839+0000","last_clean":"2024-09-11T23:16:36.226839+0000","last_became_active":"2024-09-11T23:16:25.905085+0000","last_became_peered":"2024-09-11T23:16:25.905085+0000","last_unstale":"2024-09-11T23:16:36.226839+0000","last_undegraded":"2024-09-11T23:16:36.226839+0000","last_fullsized":"2024-09-11T23:16:36.226839+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:22:35.129736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"85'5","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:36.972662+0000","last_change":"2024-09-11T23:16:24.075556+0000","last_active":"2024-09-11T23:16:36.972662+0000","last_peered":"2024-09-11T23:16:36.972662+0000","last_clean":"2024-09-11T23:16:36.972662+0000","last_became_active":"2024-09-11T23:16:24.074783+0000","last_became_peered":"2024-09-11T23:16:24.074783+0000","last_unstale":"2024-09-11T23:16:36.972662+0000","last_undegraded":"2024-09-11T23:16:36.972662+0000","last_fullsized":"2024-09-11T23:16:36.972662+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:55:09.962342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.938922+0000","last_change":"2024-09-11T23:16:25.907720+0000","last_active":"2024-09-11T23:16:35.938922+0000","last_peered":"2024-09-11T23:16:35.938922+0000","last_clean":"2024-09-11T23:16:35.938922+0000","last_became_active":"2024-09-11T23:16:25.907335+0000","last_became_peered":"2024-09-11T23:16:25.907335+0000","last_unstale":"2024-09-11T23:16:35.938922+0000","last_undegraded":"2024-09-11T23:16:35.938922+0000","last_fullsized":"2024-09-11T23:16:35.938922+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:46:14.519384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"85'24","reported_seq":76,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:50.005190+0000","last_change":"2024-09-11T23:16:23.794321+0000","last_active":"2024-09-11T23:16:50.005190+0000","last_peered":"2024-09-11T23:16:50.005190+0000","last_clean":"2024-09-11T23:16:50.005190+0000","last_became_active":"2024-09-11T23:16:23.793955+0000","last_became_peered":"2024-09-11T23:16:23.793955+0000","last_unstale":"2024-09-11T23:16:50.005190+0000","last_undegraded":"2024-09-11T23:16:50.005190+0000","last_fullsized":"2024-09-11T23:16:50.005190+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:47:46.442656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46802,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":30,"num_read_kb":60,"num_write":26,"num_write_kb":52,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.945323+0000","last_change":"2024-09-11T23:16:25.896598+0000","last_active":"2024-09-11T23:16:35.945323+0000","last_peered":"2024-09-11T23:16:35.945323+0000","last_clean":"2024-09-11T23:16:35.945323+0000","last_became_active":"2024-09-11T23:16:25.896293+0000","last_became_peered":"2024-09-11T23:16:25.896293+0000","last_unstale":"2024-09-11T23:16:35.945323+0000","last_undegraded":"2024-09-11T23:16:35.945323+0000","last_fullsized":"2024-09-11T23:16:35.945323+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:04:11.489644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"85'9","reported_seq":33,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:38.985116+0000","last_change":"2024-09-11T23:16:23.791486+0000","last_active":"2024-09-11T23:16:38.985116+0000","last_peered":"2024-09-11T23:16:38.985116+0000","last_clean":"2024-09-11T23:16:38.985116+0000","last_became_active":"2024-09-11T23:16:23.790786+0000","last_became_peered":"2024-09-11T23:16:23.790786+0000","last_unstale":"2024-09-11T23:16:38.985116+0000","last_undegraded":"2024-09-11T23:16:38.985116+0000","last_fullsized":"2024-09-11T23:16:38.985116+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:03:52.742560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.937975+0000","last_change":"2024-09-11T23:16:25.897382+0000","last_active":"2024-09-11T23:16:35.937975+0000","last_peered":"2024-09-11T23:16:35.937975+0000","last_clean":"2024-09-11T23:16:35.937975+0000","last_became_active":"2024-09-11T23:16:25.897073+0000","last_became_peered":"2024-09-11T23:16:25.897073+0000","last_unstale":"2024-09-11T23:16:35.937975+0000","last_undegraded":"2024-09-11T23:16:35.937975+0000","last_fullsized":"2024-09-11T23:16:35.937975+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:15:59.780100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"85'5","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:37.977834+0000","last_change":"2024-09-11T23:16:23.793886+0000","last_active":"2024-09-11T23:16:37.977834+0000","last_peered":"2024-09-11T23:16:37.977834+0000","last_clean":"2024-09-11T23:16:37.977834+0000","last_became_active":"2024-09-11T23:16:23.793066+0000","last_became_peered":"2024-09-11T23:16:23.793066+0000","last_unstale":"2024-09-11T23:16:37.977834+0000","last_undegraded":"2024-09-11T23:16:37.977834+0000","last_fullsized":"2024-09-11T23:16:37.977834+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:07:03.882546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.942540+0000","last_change":"2024-09-11T23:16:25.904575+0000","last_active":"2024-09-11T23:16:35.942540+0000","last_peered":"2024-09-11T23:16:35.942540+0000","last_clean":"2024-09-11T23:16:35.942540+0000","last_became_active":"2024-09-11T23:16:25.904224+0000","last_became_peered":"2024-09-11T23:16:25.904224+0000","last_unstale":"2024-09-11T23:16:35.942540+0000","last_undegraded":"2024-09-11T23:16:35.942540+0000","last_fullsized":"2024-09-11T23:16:35.942540+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:42:13.036931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"85'5","reported_seq":41,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:49.001440+0000","last_change":"2024-09-11T23:16:23.796073+0000","last_active":"2024-09-11T23:16:49.001440+0000","last_peered":"2024-09-11T23:16:49.001440+0000","last_clean":"2024-09-11T23:16:49.001440+0000","last_became_active":"2024-09-11T23:16:23.794850+0000","last_became_peered":"2024-09-11T23:16:23.794850+0000","last_unstale":"2024-09-11T23:16:49.001440+0000","last_undegraded":"2024-09-11T23:16:49.001440+0000","last_fullsized":"2024-09-11T23:16:49.001440+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:57:22.361662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":15,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.940823+0000","last_change":"2024-09-11T23:16:25.908700+0000","last_active":"2024-09-11T23:16:35.940823+0000","last_peered":"2024-09-11T23:16:35.940823+0000","last_clean":"2024-09-11T23:16:35.940823+0000","last_became_active":"2024-09-11T23:16:25.908373+0000","last_became_peered":"2024-09-11T23:16:25.908373+0000","last_unstale":"2024-09-11T23:16:35.940823+0000","last_undegraded":"2024-09-11T23:16:35.940823+0000","last_fullsized":"2024-09-11T23:16:35.940823+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:20:18.536079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"85'17","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:37.976750+0000","last_change":"2024-09-11T23:16:24.075927+0000","last_active":"2024-09-11T23:16:37.976750+0000","last_peered":"2024-09-11T23:16:37.976750+0000","last_clean":"2024-09-11T23:16:37.976750+0000","last_became_active":"2024-09-11T23:16:24.075527+0000","last_became_peered":"2024-09-11T23:16:24.075527+0000","last_unstale":"2024-09-11T23:16:37.976750+0000","last_undegraded":"2024-09-11T23:16:37.976750+0000","last_fullsized":"2024-09-11T23:16:37.976750+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:27:54.622059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.947475+0000","last_change":"2024-09-11T23:16:35.947475+0000","last_active":"2024-09-11T23:16:35.947475+0000","last_peered":"2024-09-11T23:16:35.947475+0000","last_clean":"2024-09-11T23:16:35.947475+0000","last_became_active":"2024-09-11T23:16:35.947028+0000","last_became_peered":"2024-09-11T23:16:35.947028+0000","last_unstale":"2024-09-11T23:16:35.947475+0000","last_undegraded":"2024-09-11T23:16:35.947475+0000","last_fullsized":"2024-09-11T23:16:35.947475+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:18:13.479554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'2","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:37.950468+0000","last_change":"2024-09-11T23:16:23.793298+0000","last_active":"2024-09-11T23:16:37.950468+0000","last_peered":"2024-09-11T23:16:37.950468+0000","last_clean":"2024-09-11T23:16:37.950468+0000","last_became_active":"2024-09-11T23:16:23.792947+0000","last_became_peered":"2024-09-11T23:16:23.792947+0000","last_unstale":"2024-09-11T23:16:37.950468+0000","last_undegraded":"2024-09-11T23:16:37.950468+0000","last_fullsized":"2024-09-11T23:16:37.950468+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:32:21.294560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.941169+0000","last_change":"2024-09-11T23:16:25.902348+0000","last_active":"2024-09-11T23:16:35.941169+0000","last_peered":"2024-09-11T23:16:35.941169+0000","last_clean":"2024-09-11T23:16:35.941169+0000","last_became_active":"2024-09-11T23:16:25.901686+0000","last_became_peered":"2024-09-11T23:16:25.901686+0000","last_unstale":"2024-09-11T23:16:35.941169+0000","last_undegraded":"2024-09-11T23:16:35.941169+0000","last_fullsized":"2024-09-11T23:16:35.941169+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:00:15.543003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"85'5","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:37.975894+0000","last_change":"2024-09-11T23:16:23.790984+0000","last_active":"2024-09-11T23:16:37.975894+0000","last_peered":"2024-09-11T23:16:37.975894+0000","last_clean":"2024-09-11T23:16:37.975894+0000","last_became_active":"2024-09-11T23:16:23.790569+0000","last_became_peered":"2024-09-11T23:16:23.790569+0000","last_unstale":"2024-09-11T23:16:37.975894+0000","last_undegraded":"2024-09-11T23:16:37.975894+0000","last_fullsized":"2024-09-11T23:16:37.975894+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:57:39.675793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.941423+0000","last_change":"2024-09-11T23:16:25.899249+0000","last_active":"2024-09-11T23:16:35.941423+0000","last_peered":"2024-09-11T23:16:35.941423+0000","last_clean":"2024-09-11T23:16:35.941423+0000","last_became_active":"2024-09-11T23:16:25.898914+0000","last_became_peered":"2024-09-11T23:16:25.898914+0000","last_unstale":"2024-09-11T23:16:35.941423+0000","last_undegraded":"2024-09-11T23:16:35.941423+0000","last_fullsized":"2024-09-11T23:16:35.941423+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:17:07.545770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'3","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:37.977945+0000","last_change":"2024-09-11T23:16:23.787387+0000","last_active":"2024-09-11T23:16:37.977945+0000","last_peered":"2024-09-11T23:16:37.977945+0000","last_clean":"2024-09-11T23:16:37.977945+0000","last_became_active":"2024-09-11T23:16:23.787014+0000","last_became_peered":"2024-09-11T23:16:23.787014+0000","last_unstale":"2024-09-11T23:16:37.977945+0000","last_undegraded":"2024-09-11T23:16:37.977945+0000","last_fullsized":"2024-09-11T23:16:37.977945+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:45:52.004253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"85'1","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.941159+0000","last_change":"2024-09-11T23:16:25.908304+0000","last_active":"2024-09-11T23:16:35.941159+0000","last_peered":"2024-09-11T23:16:35.941159+0000","last_clean":"2024-09-11T23:16:35.941159+0000","last_became_active":"2024-09-11T23:16:25.907900+0000","last_became_peered":"2024-09-11T23:16:25.907900+0000","last_unstale":"2024-09-11T23:16:35.941159+0000","last_undegraded":"2024-09-11T23:16:35.941159+0000","last_fullsized":"2024-09-11T23:16:35.941159+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:43:26.247682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"85'6","reported_seq":41,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:46.999370+0000","last_change":"2024-09-11T23:16:24.074610+0000","last_active":"2024-09-11T23:16:46.999370+0000","last_peered":"2024-09-11T23:16:46.999370+0000","last_clean":"2024-09-11T23:16:46.999370+0000","last_became_active":"2024-09-11T23:16:24.074210+0000","last_became_peered":"2024-09-11T23:16:24.074210+0000","last_unstale":"2024-09-11T23:16:46.999370+0000","last_undegraded":"2024-09-11T23:16:46.999370+0000","last_fullsized":"2024-09-11T23:16:46.999370+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:09:37.167050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1563,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":13,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.941591+0000","last_change":"2024-09-11T23:16:25.903120+0000","last_active":"2024-09-11T23:16:35.941591+0000","last_peered":"2024-09-11T23:16:35.941591+0000","last_clean":"2024-09-11T23:16:35.941591+0000","last_became_active":"2024-09-11T23:16:25.902770+0000","last_became_peered":"2024-09-11T23:16:25.902770+0000","last_unstale":"2024-09-11T23:16:35.941591+0000","last_undegraded":"2024-09-11T23:16:35.941591+0000","last_fullsized":"2024-09-11T23:16:35.941591+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:03:30.041088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":346,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.941279+0000","last_change":"2024-09-11T23:15:13.115055+0000","last_active":"2024-09-11T23:16:35.941279+0000","last_peered":"2024-09-11T23:16:35.941279+0000","last_clean":"2024-09-11T23:16:35.941279+0000","last_became_active":"2024-09-11T23:15:13.114480+0000","last_became_peered":"2024-09-11T23:15:13.114480+0000","last_unstale":"2024-09-11T23:16:35.941279+0000","last_undegraded":"2024-09-11T23:16:35.941279+0000","last_fullsized":"2024-09-11T23:16:35.941279+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:11:50.495799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:11:50.495799+0000","last_clean_scrub_stamp":"2024-09-11T23:11:50.495799+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:15:53.681230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"85'6","reported_seq":31,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:38.985260+0000","last_change":"2024-09-11T23:16:23.793996+0000","last_active":"2024-09-11T23:16:38.985260+0000","last_peered":"2024-09-11T23:16:38.985260+0000","last_clean":"2024-09-11T23:16:38.985260+0000","last_became_active":"2024-09-11T23:16:23.793681+0000","last_became_peered":"2024-09-11T23:16:23.793681+0000","last_unstale":"2024-09-11T23:16:38.985260+0000","last_undegraded":"2024-09-11T23:16:38.985260+0000","last_fullsized":"2024-09-11T23:16:38.985260+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:52:50.463623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:36.227084+0000","last_change":"2024-09-11T23:16:25.905133+0000","last_active":"2024-09-11T23:16:36.227084+0000","last_peered":"2024-09-11T23:16:36.227084+0000","last_clean":"2024-09-11T23:16:36.227084+0000","last_became_active":"2024-09-11T23:16:25.904819+0000","last_became_peered":"2024-09-11T23:16:25.904819+0000","last_unstale":"2024-09-11T23:16:36.227084+0000","last_undegraded":"2024-09-11T23:16:36.227084+0000","last_fullsized":"2024-09-11T23:16:36.227084+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:51:09.664343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"85'4","reported_seq":32,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:36.231286+0000","last_change":"2024-09-11T23:16:36.228806+0000","last_active":"2024-09-11T23:16:36.231286+0000","last_peered":"2024-09-11T23:16:36.231286+0000","last_clean":"2024-09-11T23:16:36.231286+0000","last_became_active":"2024-09-11T23:16:36.228311+0000","last_became_peered":"2024-09-11T23:16:36.228311+0000","last_unstale":"2024-09-11T23:16:36.231286+0000","last_undegraded":"2024-09-11T23:16:36.231286+0000","last_fullsized":"2024-09-11T23:16:36.231286+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:42:32.856691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4],"acting":[7,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.937900+0000","last_change":"2024-09-11T23:16:25.898935+0000","last_active":"2024-09-11T23:16:35.937900+0000","last_peered":"2024-09-11T23:16:35.937900+0000","last_clean":"2024-09-11T23:16:35.937900+0000","last_became_active":"2024-09-11T23:16:25.898618+0000","last_became_peered":"2024-09-11T23:16:25.898618+0000","last_unstale":"2024-09-11T23:16:35.937900+0000","last_undegraded":"2024-09-11T23:16:35.937900+0000","last_fullsized":"2024-09-11T23:16:35.937900+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:13:29.104911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"85'5","reported_seq":39,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:47.999580+0000","last_change":"2024-09-11T23:16:24.073613+0000","last_active":"2024-09-11T23:16:47.999580+0000","last_peered":"2024-09-11T23:16:47.999580+0000","last_clean":"2024-09-11T23:16:47.999580+0000","last_became_active":"2024-09-11T23:16:24.073262+0000","last_became_peered":"2024-09-11T23:16:24.073262+0000","last_unstale":"2024-09-11T23:16:47.999580+0000","last_undegraded":"2024-09-11T23:16:47.999580+0000","last_fullsized":"2024-09-11T23:16:47.999580+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:31:58.653657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":12,"num_read_kb":12,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.941067+0000","last_change":"2024-09-11T23:16:25.903725+0000","last_active":"2024-09-11T23:16:35.941067+0000","last_peered":"2024-09-11T23:16:35.941067+0000","last_clean":"2024-09-11T23:16:35.941067+0000","last_became_active":"2024-09-11T23:16:25.903375+0000","last_became_peered":"2024-09-11T23:16:25.903375+0000","last_unstale":"2024-09-11T23:16:35.941067+0000","last_undegraded":"2024-09-11T23:16:35.941067+0000","last_fullsized":"2024-09-11T23:16:35.941067+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:55:02.779575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"85'5","reported_seq":38,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:46.998806+0000","last_change":"2024-09-11T23:16:23.793932+0000","last_active":"2024-09-11T23:16:46.998806+0000","last_peered":"2024-09-11T23:16:46.998806+0000","last_clean":"2024-09-11T23:16:46.998806+0000","last_became_active":"2024-09-11T23:16:23.793111+0000","last_became_peered":"2024-09-11T23:16:23.793111+0000","last_unstale":"2024-09-11T23:16:46.998806+0000","last_undegraded":"2024-09-11T23:16:46.998806+0000","last_fullsized":"2024-09-11T23:16:46.998806+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:14:09.087222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.942165+0000","last_change":"2024-09-11T23:16:25.905020+0000","last_active":"2024-09-11T23:16:35.942165+0000","last_peered":"2024-09-11T23:16:35.942165+0000","last_clean":"2024-09-11T23:16:35.942165+0000","last_became_active":"2024-09-11T23:16:25.904675+0000","last_became_peered":"2024-09-11T23:16:25.904675+0000","last_unstale":"2024-09-11T23:16:35.942165+0000","last_undegraded":"2024-09-11T23:16:35.942165+0000","last_fullsized":"2024-09-11T23:16:35.942165+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:13:57.388773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"85'1","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.939622+0000","last_change":"2024-09-11T23:16:24.073630+0000","last_active":"2024-09-11T23:16:35.939622+0000","last_peered":"2024-09-11T23:16:35.939622+0000","last_clean":"2024-09-11T23:16:35.939622+0000","last_became_active":"2024-09-11T23:16:24.073301+0000","last_became_peered":"2024-09-11T23:16:24.073301+0000","last_unstale":"2024-09-11T23:16:35.939622+0000","last_undegraded":"2024-09-11T23:16:35.939622+0000","last_fullsized":"2024-09-11T23:16:35.939622+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:41:59.374581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"85'1","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.942393+0000","last_change":"2024-09-11T23:16:25.894124+0000","last_active":"2024-09-11T23:16:35.942393+0000","last_peered":"2024-09-11T23:16:35.942393+0000","last_clean":"2024-09-11T23:16:35.942393+0000","last_became_active":"2024-09-11T23:16:25.893751+0000","last_became_peered":"2024-09-11T23:16:25.893751+0000","last_unstale":"2024-09-11T23:16:35.942393+0000","last_undegraded":"2024-09-11T23:16:35.942393+0000","last_fullsized":"2024-09-11T23:16:35.942393+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:00:28.265206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"85'4","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:37.976960+0000","last_change":"2024-09-11T23:16:24.073700+0000","last_active":"2024-09-11T23:16:37.976960+0000","last_peered":"2024-09-11T23:16:37.976960+0000","last_clean":"2024-09-11T23:16:37.976960+0000","last_became_active":"2024-09-11T23:16:24.073283+0000","last_became_peered":"2024-09-11T23:16:24.073283+0000","last_unstale":"2024-09-11T23:16:37.976960+0000","last_undegraded":"2024-09-11T23:16:37.976960+0000","last_fullsized":"2024-09-11T23:16:37.976960+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:41:43.536834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.938154+0000","last_change":"2024-09-11T23:16:25.900532+0000","last_active":"2024-09-11T23:16:35.938154+0000","last_peered":"2024-09-11T23:16:35.938154+0000","last_clean":"2024-09-11T23:16:35.938154+0000","last_became_active":"2024-09-11T23:16:25.899466+0000","last_became_peered":"2024-09-11T23:16:25.899466+0000","last_unstale":"2024-09-11T23:16:35.938154+0000","last_undegraded":"2024-09-11T23:16:35.938154+0000","last_fullsized":"2024-09-11T23:16:35.938154+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:10:01.152323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"85'12","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:46.913831+0000","last_change":"2024-09-11T23:16:23.788789+0000","last_active":"2024-09-11T23:16:46.913831+0000","last_peered":"2024-09-11T23:16:46.913831+0000","last_clean":"2024-09-11T23:16:46.913831+0000","last_became_active":"2024-09-11T23:16:23.788422+0000","last_became_peered":"2024-09-11T23:16:23.788422+0000","last_unstale":"2024-09-11T23:16:46.913831+0000","last_undegraded":"2024-09-11T23:16:46.913831+0000","last_fullsized":"2024-09-11T23:16:46.913831+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:09:14.310205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:36.227116+0000","last_change":"2024-09-11T23:16:25.895511+0000","last_active":"2024-09-11T23:16:36.227116+0000","last_peered":"2024-09-11T23:16:36.227116+0000","last_clean":"2024-09-11T23:16:36.227116+0000","last_became_active":"2024-09-11T23:16:25.895143+0000","last_became_peered":"2024-09-11T23:16:25.895143+0000","last_unstale":"2024-09-11T23:16:36.227116+0000","last_undegraded":"2024-09-11T23:16:36.227116+0000","last_fullsized":"2024-09-11T23:16:36.227116+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:31:16.498371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"85'7","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:37.977151+0000","last_change":"2024-09-11T23:16:24.019064+0000","last_active":"2024-09-11T23:16:37.977151+0000","last_peered":"2024-09-11T23:16:37.977151+0000","last_clean":"2024-09-11T23:16:37.977151+0000","last_became_active":"2024-09-11T23:16:24.018658+0000","last_became_peered":"2024-09-11T23:16:24.018658+0000","last_unstale":"2024-09-11T23:16:37.977151+0000","last_undegraded":"2024-09-11T23:16:37.977151+0000","last_fullsized":"2024-09-11T23:16:37.977151+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:17:41.009507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:36.227194+0000","last_change":"2024-09-11T23:16:25.903553+0000","last_active":"2024-09-11T23:16:36.227194+0000","last_peered":"2024-09-11T23:16:36.227194+0000","last_clean":"2024-09-11T23:16:36.227194+0000","last_became_active":"2024-09-11T23:16:25.903221+0000","last_became_peered":"2024-09-11T23:16:25.903221+0000","last_unstale":"2024-09-11T23:16:36.227194+0000","last_undegraded":"2024-09-11T23:16:36.227194+0000","last_fullsized":"2024-09-11T23:16:36.227194+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:16:51.872517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"83'2","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.938847+0000","last_change":"2024-09-11T23:16:24.073784+0000","last_active":"2024-09-11T23:16:35.938847+0000","last_peered":"2024-09-11T23:16:35.938847+0000","last_clean":"2024-09-11T23:16:35.938847+0000","last_became_active":"2024-09-11T23:16:24.073406+0000","last_became_peered":"2024-09-11T23:16:24.073406+0000","last_unstale":"2024-09-11T23:16:35.938847+0000","last_undegraded":"2024-09-11T23:16:35.938847+0000","last_fullsized":"2024-09-11T23:16:35.938847+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:31:24.617025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.945715+0000","last_change":"2024-09-11T23:16:25.894878+0000","last_active":"2024-09-11T23:16:35.945715+0000","last_peered":"2024-09-11T23:16:35.945715+0000","last_clean":"2024-09-11T23:16:35.945715+0000","last_became_active":"2024-09-11T23:16:25.894543+0000","last_became_peered":"2024-09-11T23:16:25.894543+0000","last_unstale":"2024-09-11T23:16:35.945715+0000","last_undegraded":"2024-09-11T23:16:35.945715+0000","last_fullsized":"2024-09-11T23:16:35.945715+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:34:13.435754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"85'7","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:37.976060+0000","last_change":"2024-09-11T23:16:23.786502+0000","last_active":"2024-09-11T23:16:37.976060+0000","last_peered":"2024-09-11T23:16:37.976060+0000","last_clean":"2024-09-11T23:16:37.976060+0000","last_became_active":"2024-09-11T23:16:23.786182+0000","last_became_peered":"2024-09-11T23:16:23.786182+0000","last_unstale":"2024-09-11T23:16:37.976060+0000","last_undegraded":"2024-09-11T23:16:37.976060+0000","last_fullsized":"2024-09-11T23:16:37.976060+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:59:07.978244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.942309+0000","last_change":"2024-09-11T23:16:25.899625+0000","last_active":"2024-09-11T23:16:35.942309+0000","last_peered":"2024-09-11T23:16:35.942309+0000","last_clean":"2024-09-11T23:16:35.942309+0000","last_became_active":"2024-09-11T23:16:25.899265+0000","last_became_peered":"2024-09-11T23:16:25.899265+0000","last_unstale":"2024-09-11T23:16:35.942309+0000","last_undegraded":"2024-09-11T23:16:35.942309+0000","last_fullsized":"2024-09-11T23:16:35.942309+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:36:06.907622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"85'3","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:36.971236+0000","last_change":"2024-09-11T23:16:36.228991+0000","last_active":"2024-09-11T23:16:36.971236+0000","last_peered":"2024-09-11T23:16:36.971236+0000","last_clean":"2024-09-11T23:16:36.971236+0000","last_became_active":"2024-09-11T23:16:36.227701+0000","last_became_peered":"2024-09-11T23:16:36.227701+0000","last_unstale":"2024-09-11T23:16:36.971236+0000","last_undegraded":"2024-09-11T23:16:36.971236+0000","last_fullsized":"2024-09-11T23:16:36.971236+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:37:57.478572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:36.226661+0000","last_change":"2024-09-11T23:16:25.894803+0000","last_active":"2024-09-11T23:16:36.226661+0000","last_peered":"2024-09-11T23:16:36.226661+0000","last_clean":"2024-09-11T23:16:36.226661+0000","last_became_active":"2024-09-11T23:16:25.894429+0000","last_became_peered":"2024-09-11T23:16:25.894429+0000","last_unstale":"2024-09-11T23:16:36.226661+0000","last_undegraded":"2024-09-11T23:16:36.226661+0000","last_fullsized":"2024-09-11T23:16:36.226661+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:39:15.868910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"83'4","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.945644+0000","last_change":"2024-09-11T23:16:23.794271+0000","last_active":"2024-09-11T23:16:35.945644+0000","last_peered":"2024-09-11T23:16:35.945644+0000","last_clean":"2024-09-11T23:16:35.945644+0000","last_became_active":"2024-09-11T23:16:23.793919+0000","last_became_peered":"2024-09-11T23:16:23.793919+0000","last_unstale":"2024-09-11T23:16:35.945644+0000","last_undegraded":"2024-09-11T23:16:35.945644+0000","last_fullsized":"2024-09-11T23:16:35.945644+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:30:28.907312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.941192+0000","last_change":"2024-09-11T23:16:25.900969+0000","last_active":"2024-09-11T23:16:35.941192+0000","last_peered":"2024-09-11T23:16:35.941192+0000","last_clean":"2024-09-11T23:16:35.941192+0000","last_became_active":"2024-09-11T23:16:25.900425+0000","last_became_peered":"2024-09-11T23:16:25.900425+0000","last_unstale":"2024-09-11T23:16:35.941192+0000","last_undegraded":"2024-09-11T23:16:35.941192+0000","last_fullsized":"2024-09-11T23:16:35.941192+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:03:25.185773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.941358+0000","last_change":"2024-09-11T23:16:25.911605+0000","last_active":"2024-09-11T23:16:35.941358+0000","last_peered":"2024-09-11T23:16:35.941358+0000","last_clean":"2024-09-11T23:16:35.941358+0000","last_became_active":"2024-09-11T23:16:25.911180+0000","last_became_peered":"2024-09-11T23:16:25.911180+0000","last_unstale":"2024-09-11T23:16:35.941358+0000","last_undegraded":"2024-09-11T23:16:35.941358+0000","last_fullsized":"2024-09-11T23:16:35.941358+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:55:38.062606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"85'5","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:37.979007+0000","last_change":"2024-09-11T23:16:35.949528+0000","last_active":"2024-09-11T23:16:37.979007+0000","last_peered":"2024-09-11T23:16:37.979007+0000","last_clean":"2024-09-11T23:16:37.979007+0000","last_became_active":"2024-09-11T23:16:35.945679+0000","last_became_peered":"2024-09-11T23:16:35.945679+0000","last_unstale":"2024-09-11T23:16:37.979007+0000","last_undegraded":"2024-09-11T23:16:37.979007+0000","last_fullsized":"2024-09-11T23:16:37.979007+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:14:24.706044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.938792+0000","last_change":"2024-09-11T23:16:25.900526+0000","last_active":"2024-09-11T23:16:35.938792+0000","last_peered":"2024-09-11T23:16:35.938792+0000","last_clean":"2024-09-11T23:16:35.938792+0000","last_became_active":"2024-09-11T23:16:25.900148+0000","last_became_peered":"2024-09-11T23:16:25.900148+0000","last_unstale":"2024-09-11T23:16:35.938792+0000","last_undegraded":"2024-09-11T23:16:35.938792+0000","last_fullsized":"2024-09-11T23:16:35.938792+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:26:10.582473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"85'12","reported_seq":45,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:38.986047+0000","last_change":"2024-09-11T23:16:36.233273+0000","last_active":"2024-09-11T23:16:38.986047+0000","last_peered":"2024-09-11T23:16:38.986047+0000","last_clean":"2024-09-11T23:16:38.986047+0000","last_became_active":"2024-09-11T23:16:36.228358+0000","last_became_peered":"2024-09-11T23:16:36.228358+0000","last_unstale":"2024-09-11T23:16:38.986047+0000","last_undegraded":"2024-09-11T23:16:38.986047+0000","last_fullsized":"2024-09-11T23:16:38.986047+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:06:31.225795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:36.226734+0000","last_change":"2024-09-11T23:16:25.907874+0000","last_active":"2024-09-11T23:16:36.226734+0000","last_peered":"2024-09-11T23:16:36.226734+0000","last_clean":"2024-09-11T23:16:36.226734+0000","last_became_active":"2024-09-11T23:16:25.907491+0000","last_became_peered":"2024-09-11T23:16:25.907491+0000","last_unstale":"2024-09-11T23:16:36.226734+0000","last_undegraded":"2024-09-11T23:16:36.226734+0000","last_fullsized":"2024-09-11T23:16:36.226734+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:09:20.984249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"84'2","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:36.227306+0000","last_change":"2024-09-11T23:16:24.074355+0000","last_active":"2024-09-11T23:16:36.227306+0000","last_peered":"2024-09-11T23:16:36.227306+0000","last_clean":"2024-09-11T23:16:36.227306+0000","last_became_active":"2024-09-11T23:16:24.074047+0000","last_became_peered":"2024-09-11T23:16:24.074047+0000","last_unstale":"2024-09-11T23:16:36.227306+0000","last_undegraded":"2024-09-11T23:16:36.227306+0000","last_fullsized":"2024-09-11T23:16:36.227306+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:31:58.804420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:36.226393+0000","last_change":"2024-09-11T23:16:25.907701+0000","last_active":"2024-09-11T23:16:36.226393+0000","last_peered":"2024-09-11T23:16:36.226393+0000","last_clean":"2024-09-11T23:16:36.226393+0000","last_became_active":"2024-09-11T23:16:25.907379+0000","last_became_peered":"2024-09-11T23:16:25.907379+0000","last_unstale":"2024-09-11T23:16:36.226393+0000","last_undegraded":"2024-09-11T23:16:36.226393+0000","last_fullsized":"2024-09-11T23:16:36.226393+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:12:03.639055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"85'13","reported_seq":48,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:49.000778+0000","last_change":"2024-09-11T23:16:23.794293+0000","last_active":"2024-09-11T23:16:49.000778+0000","last_peered":"2024-09-11T23:16:49.000778+0000","last_clean":"2024-09-11T23:16:49.000778+0000","last_became_active":"2024-09-11T23:16:23.793974+0000","last_became_peered":"2024-09-11T23:16:23.793974+0000","last_unstale":"2024-09-11T23:16:49.000778+0000","last_undegraded":"2024-09-11T23:16:49.000778+0000","last_fullsized":"2024-09-11T23:16:49.000778+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:24:39.170188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":13,"num_read_kb":13,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":53617,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":151,"num_read_kb":138,"num_write":231,"num_write_kb":116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":528384,"data_stored":160851,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":199,"ondisk_log_size":199,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":77,"seq":330712481810,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32108,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713044,"statfs":{"total":95995035648,"available":95962157056,"internally_reserved":0,"allocated":946176,"data_stored":449945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":70,"seq":300647710742,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28616,"kb_used_data":1592,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716536,"statfs":{"total":95995035648,"available":95965732864,"internally_reserved":0,"allocated":1630208,"data_stored":1086801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":65,"seq":279172874268,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":966656,"data_stored":448900,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":57,"seq":244813135905,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":958464,"data_stored":448696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":50,"seq":214748364837,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28548,"kb_used_data":1524,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716604,"statfs":{"total":95995035648,"available":95965802496,"internally_reserved":0,"allocated":1560576,"data_stored":1041405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528361,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36272,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708880,"statfs":{"total":95995035648,"available":95957893120,"internally_reserved":0,"allocated":950272,"data_stored":448617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790000,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28560,"kb_used_data":1536,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716592,"statfs":{"total":95995035648,"available":95965790208,"internally_reserved":0,"allocated":1572864,"data_stored":1041083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920821,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36272,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708880,"statfs":{"total":95995035648,"available":95957893120,"internally_reserved":0,"allocated":950272,"data_stored":448478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182458,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40448,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704704,"statfs":{"total":95995035648,"available":95953616896,"internally_reserved":0,"allocated":966656,"data_stored":448442,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411391,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36340,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708812,"statfs":{"total":95995035648,"available":95957823488,"internally_reserved":0,"allocated":1019904,"data_stored":496945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574916,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":999424,"data_stored":496332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738441,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":970752,"data_stored":450349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":48554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":49167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":48538,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T23:16:53.938 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-11T23:16:53.939 INFO:tasks.cephadm:Setting up client nodes... 2024-09-11T23:16:53.939 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-11T23:16:54.155 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:54 smithi026 ceph-mon[31172]: from='client.14700 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T23:16:54.202 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:16:54.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:54 smithi136 ceph-mon[35121]: from='client.14700 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T23:16:54.553 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:54 smithi076 ceph-mon[35296]: from='client.14700 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T23:16:54.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:54.689+0000 7f56a003d640 1 -- 172.21.15.26:0/3803533841 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5698100490 msgr2=0x7f5698100890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:16:54.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:54.689+0000 7f56a003d640 1 --2- 172.21.15.26:0/3803533841 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5698100490 0x7f5698100890 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f5694009900 tx=0x7f569402a410 comp rx=0 tx=0).stop 2024-09-11T23:16:54.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:54.689+0000 7f56a003d640 1 -- 172.21.15.26:0/3803533841 shutdown_connections 2024-09-11T23:16:54.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:54.689+0000 7f56a003d640 1 --2- 172.21.15.26:0/3803533841 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5698100dd0 0x7f5698101230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:54.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:54.689+0000 7f56a003d640 1 --2- 172.21.15.26:0/3803533841 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5698101770 0x7f5698101c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:54.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:54.689+0000 7f56a003d640 1 --2- 172.21.15.26:0/3803533841 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5698100490 0x7f5698100890 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:54.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:54.689+0000 7f56a003d640 1 -- 172.21.15.26:0/3803533841 >> 172.21.15.26:0/3803533841 conn(0x7f56980fc2a0 msgr2=0x7f56980fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:16:54.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:54.689+0000 7f56a003d640 1 -- 172.21.15.26:0/3803533841 shutdown_connections 2024-09-11T23:16:54.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:54.689+0000 7f56a003d640 1 -- 172.21.15.26:0/3803533841 wait complete. 2024-09-11T23:16:54.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:54.690+0000 7f56a003d640 1 Processor -- start 2024-09-11T23:16:54.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:54.690+0000 7f56a003d640 1 -- start start 2024-09-11T23:16:54.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:54.690+0000 7f56a003d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5698100490 0x7f569819a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:16:54.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:54.690+0000 7f56a003d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5698100dd0 0x7f569819a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:16:54.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:54.690+0000 7f56a003d640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5698101770 0x7f56981a1760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:16:54.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:54.690+0000 7f56a003d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f569810fb00 con 0x7f5698100490 2024-09-11T23:16:54.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:54.690+0000 7f56a003d640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f569810f980 con 0x7f5698100dd0 2024-09-11T23:16:54.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:54.690+0000 7f56a003d640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f569810fc80 con 0x7f5698101770 2024-09-11T23:16:54.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:54.690+0000 7f569ddd9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5698100490 0x7f569819a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:16:54.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:54.691+0000 7f569ddd9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5698100490 0x7f569819a1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42124/0 (socket says 172.21.15.26:42124) 2024-09-11T23:16:54.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:54.691+0000 7f569ddd9640 1 -- 172.21.15.26:0/2456498460 learned_addr learned my addr 172.21.15.26:0/2456498460 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:16:54.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:54.691+0000 7f569e5da640 1 --2- 172.21.15.26:0/2456498460 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5698101770 0x7f56981a1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:16:54.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:54.691+0000 7f569d5d8640 1 --2- 172.21.15.26:0/2456498460 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5698100dd0 0x7f569819a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:16:54.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:54.691+0000 7f569ddd9640 1 -- 172.21.15.26:0/2456498460 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5698101770 msgr2=0x7f56981a1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:16:54.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:54.691+0000 7f569ddd9640 1 --2- 172.21.15.26:0/2456498460 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5698101770 0x7f56981a1760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:54.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:54.691+0000 7f569ddd9640 1 -- 172.21.15.26:0/2456498460 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5698100dd0 msgr2=0x7f569819a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:16:54.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:54.691+0000 7f569ddd9640 1 --2- 172.21.15.26:0/2456498460 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5698100dd0 0x7f569819a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:54.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:54.691+0000 7f569ddd9640 1 -- 172.21.15.26:0/2456498460 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56981a1dd0 con 0x7f5698100490 2024-09-11T23:16:54.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:54.691+0000 7f569e5da640 1 --2- 172.21.15.26:0/2456498460 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5698101770 0x7f56981a1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:16:54.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:54.691+0000 7f569d5d8640 1 --2- 172.21.15.26:0/2456498460 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5698100dd0 0x7f569819a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:16:54.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:54.691+0000 7f569ddd9640 1 --2- 172.21.15.26:0/2456498460 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5698100490 0x7f569819a1a0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f5694009230 tx=0x7f5694004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:16:54.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:54.691+0000 7f568effd640 1 -- 172.21.15.26:0/2456498460 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f569403f030 con 0x7f5698100490 2024-09-11T23:16:54.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:54.691+0000 7f568effd640 1 -- 172.21.15.26:0/2456498460 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5694003ce0 con 0x7f5698100490 2024-09-11T23:16:54.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:54.691+0000 7f56a003d640 1 -- 172.21.15.26:0/2456498460 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56981a2060 con 0x7f5698100490 2024-09-11T23:16:54.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:54.691+0000 7f568effd640 1 -- 172.21.15.26:0/2456498460 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5694035070 con 0x7f5698100490 2024-09-11T23:16:54.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:54.691+0000 7f56a003d640 1 -- 172.21.15.26:0/2456498460 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56981a2610 con 0x7f5698100490 2024-09-11T23:16:54.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:54.692+0000 7f56a003d640 1 -- 172.21.15.26:0/2456498460 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f569810b1e0 con 0x7f5698100490 2024-09-11T23:16:54.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:54.693+0000 7f568effd640 1 -- 172.21.15.26:0/2456498460 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f5694042030 con 0x7f5698100490 2024-09-11T23:16:54.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:54.694+0000 7f568effd640 1 --2- 172.21.15.26:0/2456498460 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f56840a2cc0 0x7f56840a5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:16:54.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:54.694+0000 7f568effd640 1 -- 172.21.15.26:0/2456498460 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f56940ef950 con 0x7f5698100490 2024-09-11T23:16:54.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:54.694+0000 7f569d5d8640 1 --2- 172.21.15.26:0/2456498460 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f56840a2cc0 0x7f56840a5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:16:54.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:54.695+0000 7f569d5d8640 1 --2- 172.21.15.26:0/2456498460 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f56840a2cc0 0x7f56840a5180 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f569819b6c0 tx=0x7f5688008040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:16:54.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:54.698+0000 7f568effd640 1 -- 172.21.15.26:0/2456498460 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f56940bc710 con 0x7f5698100490 2024-09-11T23:16:54.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:54.939+0000 7f56a003d640 1 -- 172.21.15.26:0/2456498460 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f569802d7c0 con 0x7f5698100490 2024-09-11T23:16:54.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:54.941+0000 7f568effd640 1 -- 172.21.15.26:0/2456498460 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7f56940bc010 con 0x7f5698100490 2024-09-11T23:16:54.941 INFO:teuthology.orchestra.run.smithi026.stdout:[client.0] 2024-09-11T23:16:54.941 INFO:teuthology.orchestra.run.smithi026.stdout: key = AQDmJOJm0vYROBAACbAobe+EAf2TTdBw/CCr+Q== 2024-09-11T23:16:54.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:54.943+0000 7f56a003d640 1 -- 172.21.15.26:0/2456498460 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f56840a2cc0 msgr2=0x7f56840a5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:16:54.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:54.943+0000 7f56a003d640 1 --2- 172.21.15.26:0/2456498460 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f56840a2cc0 0x7f56840a5180 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f569819b6c0 tx=0x7f5688008040 comp rx=0 tx=0).stop 2024-09-11T23:16:54.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:54.943+0000 7f56a003d640 1 -- 172.21.15.26:0/2456498460 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5698100490 msgr2=0x7f569819a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:16:54.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:54.943+0000 7f56a003d640 1 --2- 172.21.15.26:0/2456498460 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5698100490 0x7f569819a1a0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f5694009230 tx=0x7f5694004430 comp rx=0 tx=0).stop 2024-09-11T23:16:54.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:54.943+0000 7f56a003d640 1 -- 172.21.15.26:0/2456498460 shutdown_connections 2024-09-11T23:16:54.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:54.943+0000 7f56a003d640 1 --2- 172.21.15.26:0/2456498460 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f56840a2cc0 0x7f56840a5180 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:54.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:54.943+0000 7f56a003d640 1 --2- 172.21.15.26:0/2456498460 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5698100dd0 0x7f569819a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:54.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:54.943+0000 7f56a003d640 1 --2- 172.21.15.26:0/2456498460 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5698101770 0x7f56981a1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:54.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:54.943+0000 7f56a003d640 1 --2- 172.21.15.26:0/2456498460 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5698100490 0x7f569819a1a0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:54.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:54.943+0000 7f56a003d640 1 -- 172.21.15.26:0/2456498460 >> 172.21.15.26:0/2456498460 conn(0x7f56980fc2a0 msgr2=0x7f56980fdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:16:54.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:54.944+0000 7f56a003d640 1 -- 172.21.15.26:0/2456498460 shutdown_connections 2024-09-11T23:16:54.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:54.944+0000 7f56a003d640 1 -- 172.21.15.26:0/2456498460 wait complete. 2024-09-11T23:16:55.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:55 smithi026 ceph-mon[31172]: pgmap v295: 97 pgs: 97 active+clean; 629 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 3.9 KiB/s wr, 11 op/s 2024-09-11T23:16:55.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:55 smithi026 ceph-mon[31172]: from='client.? 172.21.15.26:0/2456498460' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-11T23:16:55.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:55 smithi026 ceph-mon[31172]: from='client.? 172.21.15.26:0/2456498460' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-11T23:16:55.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:55 smithi076 ceph-mon[35296]: pgmap v295: 97 pgs: 97 active+clean; 629 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 3.9 KiB/s wr, 11 op/s 2024-09-11T23:16:55.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:55 smithi076 ceph-mon[35296]: from='client.? 172.21.15.26:0/2456498460' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-11T23:16:55.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:55 smithi076 ceph-mon[35296]: from='client.? 172.21.15.26:0/2456498460' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-11T23:16:55.490 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:55 smithi136 ceph-mon[35121]: pgmap v295: 97 pgs: 97 active+clean; 629 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 3.9 KiB/s wr, 11 op/s 2024-09-11T23:16:55.490 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:55 smithi136 ceph-mon[35121]: from='client.? 172.21.15.26:0/2456498460' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-11T23:16:55.490 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:55 smithi136 ceph-mon[35121]: from='client.? 172.21.15.26:0/2456498460' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-11T23:16:55.705 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-09-11T23:16:55.705 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-09-11T23:16:55.705 DEBUG:teuthology.orchestra.run.smithi026:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-11T23:16:55.746 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-11T23:16:55.747 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-09-11T23:16:55.747 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph mgr dump --format=json 2024-09-11T23:16:56.047 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:16:56.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:56.521+0000 7f3ea0871640 1 -- 172.21.15.26:0/461595240 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3e9c101de0 msgr2=0x7f3e9c1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:16:56.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:56.521+0000 7f3ea0871640 1 --2- 172.21.15.26:0/461595240 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3e9c101de0 0x7f3e9c1021c0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f3e8c009900 tx=0x7f3e8c02a410 comp rx=0 tx=0).stop 2024-09-11T23:16:56.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:56.521+0000 7f3ea0871640 1 -- 172.21.15.26:0/461595240 shutdown_connections 2024-09-11T23:16:56.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:56.521+0000 7f3ea0871640 1 --2- 172.21.15.26:0/461595240 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3e9c106f10 0x7f3e9c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:56.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:56.521+0000 7f3ea0871640 1 --2- 172.21.15.26:0/461595240 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3e9c102790 0x7f3e9c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:56.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:56.521+0000 7f3ea0871640 1 --2- 172.21.15.26:0/461595240 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3e9c101de0 0x7f3e9c1021c0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:56.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:56.521+0000 7f3ea0871640 1 -- 172.21.15.26:0/461595240 >> 172.21.15.26:0/461595240 conn(0x7f3e9c0fd8c0 msgr2=0x7f3e9c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:16:56.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:56.521+0000 7f3ea0871640 1 -- 172.21.15.26:0/461595240 shutdown_connections 2024-09-11T23:16:56.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:56.521+0000 7f3ea0871640 1 -- 172.21.15.26:0/461595240 wait complete. 2024-09-11T23:16:56.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:56.522+0000 7f3ea0871640 1 Processor -- start 2024-09-11T23:16:56.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:56.522+0000 7f3ea0871640 1 -- start start 2024-09-11T23:16:56.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:56.522+0000 7f3ea0871640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3e9c101de0 0x7f3e9c1985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:16:56.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:56.522+0000 7f3ea0871640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3e9c102790 0x7f3e9c198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:16:56.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:56.522+0000 7f3ea0871640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3e9c106f10 0x7f3e9c19ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:16:56.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:56.522+0000 7f3ea0871640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f3e9c112010 con 0x7f3e9c101de0 2024-09-11T23:16:56.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:56.522+0000 7f3ea0871640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f3e9c111e90 con 0x7f3e9c102790 2024-09-11T23:16:56.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:56.522+0000 7f3ea0871640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f3e9c112190 con 0x7f3e9c106f10 2024-09-11T23:16:56.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:56.522+0000 7f3e9a59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3e9c101de0 0x7f3e9c1985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:16:56.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:56.522+0000 7f3e9a59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3e9c101de0 0x7f3e9c1985b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42140/0 (socket says 172.21.15.26:42140) 2024-09-11T23:16:56.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:56.522+0000 7f3e9a59c640 1 -- 172.21.15.26:0/619094564 learned_addr learned my addr 172.21.15.26:0/619094564 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:16:56.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:56.522+0000 7f3e93fff640 1 --2- 172.21.15.26:0/619094564 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3e9c102790 0x7f3e9c198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:16:56.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:56.522+0000 7f3e9ad9d640 1 --2- 172.21.15.26:0/619094564 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3e9c106f10 0x7f3e9c19ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:16:56.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:56.523+0000 7f3e9a59c640 1 -- 172.21.15.26:0/619094564 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3e9c106f10 msgr2=0x7f3e9c19ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:16:56.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:56.523+0000 7f3e9a59c640 1 --2- 172.21.15.26:0/619094564 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3e9c106f10 0x7f3e9c19ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:56.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:56.523+0000 7f3e9a59c640 1 -- 172.21.15.26:0/619094564 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3e9c102790 msgr2=0x7f3e9c198af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:16:56.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:56.523+0000 7f3e9a59c640 1 --2- 172.21.15.26:0/619094564 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3e9c102790 0x7f3e9c198af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:56.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:56.523+0000 7f3e9a59c640 1 -- 172.21.15.26:0/619094564 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e9c19d560 con 0x7f3e9c101de0 2024-09-11T23:16:56.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:56.523+0000 7f3e9a59c640 1 --2- 172.21.15.26:0/619094564 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3e9c101de0 0x7f3e9c1985b0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f3e8c005ab0 tx=0x7f3e8c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:16:56.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:56.523+0000 7f3e937fe640 1 -- 172.21.15.26:0/619094564 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e8c03f030 con 0x7f3e9c101de0 2024-09-11T23:16:56.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:56.523+0000 7f3ea0871640 1 -- 172.21.15.26:0/619094564 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e9c19d7f0 con 0x7f3e9c101de0 2024-09-11T23:16:56.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:56.523+0000 7f3e937fe640 1 -- 172.21.15.26:0/619094564 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e8c003ce0 con 0x7f3e9c101de0 2024-09-11T23:16:56.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:56.523+0000 7f3e937fe640 1 -- 172.21.15.26:0/619094564 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e8c035070 con 0x7f3e9c101de0 2024-09-11T23:16:56.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:56.523+0000 7f3ea0871640 1 -- 172.21.15.26:0/619094564 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e9c1a5130 con 0x7f3e9c101de0 2024-09-11T23:16:56.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:56.524+0000 7f3ea0871640 1 -- 172.21.15.26:0/619094564 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e9c102240 con 0x7f3e9c101de0 2024-09-11T23:16:56.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:56.525+0000 7f3e937fe640 1 -- 172.21.15.26:0/619094564 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f3e8c042030 con 0x7f3e9c101de0 2024-09-11T23:16:56.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:56.526+0000 7f3e937fe640 1 --2- 172.21.15.26:0/619094564 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f3e880a2c10 0x7f3e880a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:16:56.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:56.526+0000 7f3e93fff640 1 --2- 172.21.15.26:0/619094564 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f3e880a2c10 0x7f3e880a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:16:56.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:56.526+0000 7f3e937fe640 1 -- 172.21.15.26:0/619094564 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f3e8c0ef830 con 0x7f3e9c101de0 2024-09-11T23:16:56.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:56.526+0000 7f3e93fff640 1 --2- 172.21.15.26:0/619094564 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f3e880a2c10 0x7f3e880a50d0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f3e9c0ffde0 tx=0x7f3e84008040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:16:56.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:56.530+0000 7f3e937fe640 1 -- 172.21.15.26:0/619094564 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3e8c0bc5f0 con 0x7f3e9c101de0 2024-09-11T23:16:56.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:56.761+0000 7f3ea0871640 1 -- 172.21.15.26:0/619094564 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f3e9c10a0f0 con 0x7f3e9c101de0 2024-09-11T23:16:56.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:56.765+0000 7f3e937fe640 1 -- 172.21.15.26:0/619094564 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+263198 (secure 0 0 0) 0x7f3e8c0bbef0 con 0x7f3e9c101de0 2024-09-11T23:16:56.769 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-11T23:16:56.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:56.769+0000 7f3ea0871640 1 -- 172.21.15.26:0/619094564 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f3e880a2c10 msgr2=0x7f3e880a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:16:56.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:56.769+0000 7f3ea0871640 1 --2- 172.21.15.26:0/619094564 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f3e880a2c10 0x7f3e880a50d0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f3e9c0ffde0 tx=0x7f3e84008040 comp rx=0 tx=0).stop 2024-09-11T23:16:56.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:56.769+0000 7f3ea0871640 1 -- 172.21.15.26:0/619094564 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3e9c101de0 msgr2=0x7f3e9c1985b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:16:56.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:56.769+0000 7f3ea0871640 1 --2- 172.21.15.26:0/619094564 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3e9c101de0 0x7f3e9c1985b0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f3e8c005ab0 tx=0x7f3e8c0042f0 comp rx=0 tx=0).stop 2024-09-11T23:16:56.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:56.769+0000 7f3ea0871640 1 -- 172.21.15.26:0/619094564 shutdown_connections 2024-09-11T23:16:56.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:56.769+0000 7f3ea0871640 1 --2- 172.21.15.26:0/619094564 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f3e880a2c10 0x7f3e880a50d0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:56.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:56.769+0000 7f3ea0871640 1 --2- 172.21.15.26:0/619094564 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3e9c102790 0x7f3e9c198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:56.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:56.769+0000 7f3ea0871640 1 --2- 172.21.15.26:0/619094564 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3e9c106f10 0x7f3e9c19ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:56.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:56.769+0000 7f3ea0871640 1 --2- 172.21.15.26:0/619094564 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3e9c101de0 0x7f3e9c1985b0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:56.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:56.769+0000 7f3ea0871640 1 -- 172.21.15.26:0/619094564 >> 172.21.15.26:0/619094564 conn(0x7f3e9c0fd8c0 msgr2=0x7f3e9c0fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:16:56.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:56.769+0000 7f3ea0871640 1 -- 172.21.15.26:0/619094564 shutdown_connections 2024-09-11T23:16:56.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:56.769+0000 7f3ea0871640 1 -- 172.21.15.26:0/619094564 wait complete. 2024-09-11T23:16:57.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:57 smithi136 ceph-mon[35121]: pgmap v296: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.7 KiB/s wr, 14 op/s 2024-09-11T23:16:57.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:57 smithi136 ceph-mon[35121]: from='client.? 172.21.15.26:0/619094564' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-11T23:16:57.501 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:57 smithi026 ceph-mon[31172]: pgmap v296: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.7 KiB/s wr, 14 op/s 2024-09-11T23:16:57.501 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:57 smithi026 ceph-mon[31172]: from='client.? 172.21.15.26:0/619094564' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-11T23:16:57.532 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":17,"flags":0,"active_gid":14148,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":110833859},{"type":"v1","addr":"172.21.15.26:6801","nonce":110833859}]},"active_addr":"172.21.15.26:6801/110833859","active_change":"2024-09-11T23:09:33.983459+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24113,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.26:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.26:0","nonce":2626831531}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.26:0","nonce":1584355185}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.26:0","nonce":2559066894}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.26:0","nonce":1256290054}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.26:0","nonce":2556253152}]}]} 2024-09-11T23:16:57.538 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-09-11T23:16:57.538 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-09-11T23:16:57.539 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-11T23:16:57.552 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:57 smithi076 ceph-mon[35296]: pgmap v296: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.7 KiB/s wr, 14 op/s 2024-09-11T23:16:57.553 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:57 smithi076 ceph-mon[35296]: from='client.? 172.21.15.26:0/619094564' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-11T23:16:57.797 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:16:58.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:58.254+0000 7fb26c9e0640 1 -- 172.21.15.26:0/3882241592 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb268100a60 msgr2=0x7fb268100ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:16:58.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:58.254+0000 7fb26c9e0640 1 --2- 172.21.15.26:0/3882241592 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb268100a60 0x7fb268100ec0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fb25c009930 tx=0x7fb25c02a420 comp rx=0 tx=0).stop 2024-09-11T23:16:58.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:58.254+0000 7fb26c9e0640 1 -- 172.21.15.26:0/3882241592 shutdown_connections 2024-09-11T23:16:58.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:58.254+0000 7fb26c9e0640 1 --2- 172.21.15.26:0/3882241592 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb268101400 0x7fb26810d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:58.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:58.254+0000 7fb26c9e0640 1 --2- 172.21.15.26:0/3882241592 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb268100a60 0x7fb268100ec0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:58.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:58.254+0000 7fb26c9e0640 1 --2- 172.21.15.26:0/3882241592 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb268106a60 0x7fb268106e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:58.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:58.254+0000 7fb26c9e0640 1 -- 172.21.15.26:0/3882241592 >> 172.21.15.26:0/3882241592 conn(0x7fb2680fc780 msgr2=0x7fb2680feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:16:58.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:58.254+0000 7fb26c9e0640 1 -- 172.21.15.26:0/3882241592 shutdown_connections 2024-09-11T23:16:58.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:58.254+0000 7fb26c9e0640 1 -- 172.21.15.26:0/3882241592 wait complete. 2024-09-11T23:16:58.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:58.255+0000 7fb26c9e0640 1 Processor -- start 2024-09-11T23:16:58.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:58.255+0000 7fb26c9e0640 1 -- start start 2024-09-11T23:16:58.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:58.255+0000 7fb26c9e0640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb268100a60 0x7fb2680fcd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:16:58.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:58.255+0000 7fb26c9e0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb268101400 0x7fb2680fd260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:16:58.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:58.255+0000 7fb26c9e0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb268106a60 0x7fb2680fd7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:16:58.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:58.255+0000 7fb26c9e0640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb268110e80 con 0x7fb268106a60 2024-09-11T23:16:58.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:58.255+0000 7fb26c9e0640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fb268110d00 con 0x7fb268101400 2024-09-11T23:16:58.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:58.255+0000 7fb26c9e0640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb268111000 con 0x7fb268100a60 2024-09-11T23:16:58.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:58.255+0000 7fb266d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb268106a60 0x7fb2680fd7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:16:58.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:58.255+0000 7fb266d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb268106a60 0x7fb2680fd7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42152/0 (socket says 172.21.15.26:42152) 2024-09-11T23:16:58.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:58.255+0000 7fb266d9d640 1 -- 172.21.15.26:0/4176755280 learned_addr learned my addr 172.21.15.26:0/4176755280 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:16:58.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:58.255+0000 7fb265d9b640 1 --2- 172.21.15.26:0/4176755280 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb268101400 0x7fb2680fd260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:16:58.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:58.255+0000 7fb26659c640 1 --2- 172.21.15.26:0/4176755280 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb268100a60 0x7fb2680fcd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:16:58.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:58.256+0000 7fb266d9d640 1 -- 172.21.15.26:0/4176755280 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb268100a60 msgr2=0x7fb2680fcd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:16:58.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:58.256+0000 7fb266d9d640 1 --2- 172.21.15.26:0/4176755280 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb268100a60 0x7fb2680fcd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:58.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:58.256+0000 7fb266d9d640 1 -- 172.21.15.26:0/4176755280 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb268101400 msgr2=0x7fb2680fd260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:16:58.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:58.256+0000 7fb266d9d640 1 --2- 172.21.15.26:0/4176755280 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb268101400 0x7fb2680fd260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:58.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:58.256+0000 7fb266d9d640 1 -- 172.21.15.26:0/4176755280 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2681a69a0 con 0x7fb268106a60 2024-09-11T23:16:58.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:58.256+0000 7fb266d9d640 1 --2- 172.21.15.26:0/4176755280 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb268106a60 0x7fb2680fd7a0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fb26000b970 tx=0x7fb26000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:16:58.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:58.256+0000 7fb2577fe640 1 -- 172.21.15.26:0/4176755280 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb260019070 con 0x7fb268106a60 2024-09-11T23:16:58.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:58.256+0000 7fb2577fe640 1 -- 172.21.15.26:0/4176755280 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb260014440 con 0x7fb268106a60 2024-09-11T23:16:58.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:58.256+0000 7fb26c9e0640 1 -- 172.21.15.26:0/4176755280 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2681a6c90 con 0x7fb268106a60 2024-09-11T23:16:58.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:58.256+0000 7fb2577fe640 1 -- 172.21.15.26:0/4176755280 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2600075d0 con 0x7fb268106a60 2024-09-11T23:16:58.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:58.256+0000 7fb26c9e0640 1 -- 172.21.15.26:0/4176755280 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2681a71a0 con 0x7fb268106a60 2024-09-11T23:16:58.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:58.257+0000 7fb26c9e0640 1 -- 172.21.15.26:0/4176755280 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb268100ec0 con 0x7fb268106a60 2024-09-11T23:16:58.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:58.258+0000 7fb2577fe640 1 -- 172.21.15.26:0/4176755280 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fb26001c070 con 0x7fb268106a60 2024-09-11T23:16:58.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:58.259+0000 7fb2577fe640 1 --2- 172.21.15.26:0/4176755280 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fb2500a2c90 0x7fb2500a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:16:58.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:58.259+0000 7fb26659c640 1 --2- 172.21.15.26:0/4176755280 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fb2500a2c90 0x7fb2500a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:16:58.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:58.259+0000 7fb2577fe640 1 -- 172.21.15.26:0/4176755280 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fb2600cea80 con 0x7fb268106a60 2024-09-11T23:16:58.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:58.259+0000 7fb26659c640 1 --2- 172.21.15.26:0/4176755280 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fb2500a2c90 0x7fb2500a5150 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fb25800abe0 tx=0x7fb25800a2d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:16:58.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:58.262+0000 7fb2577fe640 1 -- 172.21.15.26:0/4176755280 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb26009b340 con 0x7fb268106a60 2024-09-11T23:16:58.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:58.439+0000 7fb26c9e0640 1 -- 172.21.15.26:0/4176755280 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb26810a000 con 0x7fb268106a60 2024-09-11T23:16:58.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:58.441+0000 7fb2577fe640 1 -- 172.21.15.26:0/4176755280 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+22733 (secure 0 0 0) 0x7fb26009f640 con 0x7fb268106a60 2024-09-11T23:16:58.441 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-11T23:16:58.441 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":85,"fsid":"af7e6fc8-7092-11ef-bcea-c7b262605968","created":"2024-09-11T23:08:40.486438+0000","modified":"2024-09-11T23:16:35.935151+0000","last_up_change":"2024-09-11T23:15:35.179446+0000","last_in_change":"2024-09-11T23:15:11.064627+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T23:11:50.494160+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-11T23:16:22.503928+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-11T23:16:24.876554+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a4d62fcf-c31e-41b1-932c-3d87ff640531","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":2014857905},{"type":"v1","addr":"172.21.15.26:6803","nonce":2014857905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":2014857905},{"type":"v1","addr":"172.21.15.26:6805","nonce":2014857905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":2014857905},{"type":"v1","addr":"172.21.15.26:6809","nonce":2014857905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":2014857905},{"type":"v1","addr":"172.21.15.26:6807","nonce":2014857905}]},"public_addr":"172.21.15.26:6803/2014857905","cluster_addr":"172.21.15.26:6805/2014857905","heartbeat_back_addr":"172.21.15.26:6809/2014857905","heartbeat_front_addr":"172.21.15.26:6807/2014857905","state":["exists","up"]},{"osd":1,"uuid":"e0d69112-3c0a-4fc5-8783-e41b4081e0c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":333383087},{"type":"v1","addr":"172.21.15.76:6801","nonce":333383087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":333383087},{"type":"v1","addr":"172.21.15.76:6803","nonce":333383087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":333383087},{"type":"v1","addr":"172.21.15.76:6807","nonce":333383087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":333383087},{"type":"v1","addr":"172.21.15.76:6805","nonce":333383087}]},"public_addr":"172.21.15.76:6801/333383087","cluster_addr":"172.21.15.76:6803/333383087","heartbeat_back_addr":"172.21.15.76:6807/333383087","heartbeat_front_addr":"172.21.15.76:6805/333383087","state":["exists","up"]},{"osd":2,"uuid":"d578429e-bf23-4a1e-890f-7172955c0eb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":1271538911},{"type":"v1","addr":"172.21.15.136:6801","nonce":1271538911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":1271538911},{"type":"v1","addr":"172.21.15.136:6803","nonce":1271538911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":1271538911},{"type":"v1","addr":"172.21.15.136:6807","nonce":1271538911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":1271538911},{"type":"v1","addr":"172.21.15.136:6805","nonce":1271538911}]},"public_addr":"172.21.15.136:6801/1271538911","cluster_addr":"172.21.15.136:6803/1271538911","heartbeat_back_addr":"172.21.15.136:6807/1271538911","heartbeat_front_addr":"172.21.15.136:6805/1271538911","state":["exists","up"]},{"osd":3,"uuid":"3e34623f-0ea5-4b10-b8a9-f8756c35d512","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":1740304151},{"type":"v1","addr":"172.21.15.26:6811","nonce":1740304151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":1740304151},{"type":"v1","addr":"172.21.15.26:6813","nonce":1740304151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1740304151},{"type":"v1","addr":"172.21.15.26:6817","nonce":1740304151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":1740304151},{"type":"v1","addr":"172.21.15.26:6815","nonce":1740304151}]},"public_addr":"172.21.15.26:6811/1740304151","cluster_addr":"172.21.15.26:6813/1740304151","heartbeat_back_addr":"172.21.15.26:6817/1740304151","heartbeat_front_addr":"172.21.15.26:6815/1740304151","state":["exists","up"]},{"osd":4,"uuid":"77607929-f9ed-49a9-abb5-48cd88eec4ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":2702262182},{"type":"v1","addr":"172.21.15.76:6809","nonce":2702262182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":2702262182},{"type":"v1","addr":"172.21.15.76:6811","nonce":2702262182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":2702262182},{"type":"v1","addr":"172.21.15.76:6815","nonce":2702262182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":2702262182},{"type":"v1","addr":"172.21.15.76:6813","nonce":2702262182}]},"public_addr":"172.21.15.76:6809/2702262182","cluster_addr":"172.21.15.76:6811/2702262182","heartbeat_back_addr":"172.21.15.76:6815/2702262182","heartbeat_front_addr":"172.21.15.76:6813/2702262182","state":["exists","up"]},{"osd":5,"uuid":"29e20e37-09a0-4924-8aaa-17939763a7c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":1749229277},{"type":"v1","addr":"172.21.15.136:6809","nonce":1749229277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":1749229277},{"type":"v1","addr":"172.21.15.136:6811","nonce":1749229277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":1749229277},{"type":"v1","addr":"172.21.15.136:6815","nonce":1749229277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":1749229277},{"type":"v1","addr":"172.21.15.136:6813","nonce":1749229277}]},"public_addr":"172.21.15.136:6809/1749229277","cluster_addr":"172.21.15.136:6811/1749229277","heartbeat_back_addr":"172.21.15.136:6815/1749229277","heartbeat_front_addr":"172.21.15.136:6813/1749229277","state":["exists","up"]},{"osd":6,"uuid":"e0ddd1c0-1c46-41e6-b2f4-396bdcd2bf8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1928245624},{"type":"v1","addr":"172.21.15.26:6819","nonce":1928245624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1928245624},{"type":"v1","addr":"172.21.15.26:6821","nonce":1928245624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":1928245624},{"type":"v1","addr":"172.21.15.26:6825","nonce":1928245624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1928245624},{"type":"v1","addr":"172.21.15.26:6823","nonce":1928245624}]},"public_addr":"172.21.15.26:6819/1928245624","cluster_addr":"172.21.15.26:6821/1928245624","heartbeat_back_addr":"172.21.15.26:6825/1928245624","heartbeat_front_addr":"172.21.15.26:6823/1928245624","state":["exists","up"]},{"osd":7,"uuid":"fe031f5c-d915-48fb-91a6-c76f252467fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":1409789455},{"type":"v1","addr":"172.21.15.76:6817","nonce":1409789455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":1409789455},{"type":"v1","addr":"172.21.15.76:6819","nonce":1409789455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":1409789455},{"type":"v1","addr":"172.21.15.76:6823","nonce":1409789455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":1409789455},{"type":"v1","addr":"172.21.15.76:6821","nonce":1409789455}]},"public_addr":"172.21.15.76:6817/1409789455","cluster_addr":"172.21.15.76:6819/1409789455","heartbeat_back_addr":"172.21.15.76:6823/1409789455","heartbeat_front_addr":"172.21.15.76:6821/1409789455","state":["exists","up"]},{"osd":8,"uuid":"18851bde-9e6c-4e05-b4bf-ac5c3073eee7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":543619079},{"type":"v1","addr":"172.21.15.136:6817","nonce":543619079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":543619079},{"type":"v1","addr":"172.21.15.136:6819","nonce":543619079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":543619079},{"type":"v1","addr":"172.21.15.136:6823","nonce":543619079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":543619079},{"type":"v1","addr":"172.21.15.136:6821","nonce":543619079}]},"public_addr":"172.21.15.136:6817/543619079","cluster_addr":"172.21.15.136:6819/543619079","heartbeat_back_addr":"172.21.15.136:6823/543619079","heartbeat_front_addr":"172.21.15.136:6821/543619079","state":["exists","up"]},{"osd":9,"uuid":"52a65eab-8446-47ad-bfe7-5133f7e3eb98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":2201297064},{"type":"v1","addr":"172.21.15.26:6827","nonce":2201297064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":2201297064},{"type":"v1","addr":"172.21.15.26:6829","nonce":2201297064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":2201297064},{"type":"v1","addr":"172.21.15.26:6833","nonce":2201297064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":2201297064},{"type":"v1","addr":"172.21.15.26:6831","nonce":2201297064}]},"public_addr":"172.21.15.26:6827/2201297064","cluster_addr":"172.21.15.26:6829/2201297064","heartbeat_back_addr":"172.21.15.26:6833/2201297064","heartbeat_front_addr":"172.21.15.26:6831/2201297064","state":["exists","up"]},{"osd":10,"uuid":"b758d4bb-da62-4c5f-a992-bb6a145d35d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":3184128096},{"type":"v1","addr":"172.21.15.76:6825","nonce":3184128096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":3184128096},{"type":"v1","addr":"172.21.15.76:6827","nonce":3184128096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":3184128096},{"type":"v1","addr":"172.21.15.76:6831","nonce":3184128096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":3184128096},{"type":"v1","addr":"172.21.15.76:6829","nonce":3184128096}]},"public_addr":"172.21.15.76:6825/3184128096","cluster_addr":"172.21.15.76:6827/3184128096","heartbeat_back_addr":"172.21.15.76:6831/3184128096","heartbeat_front_addr":"172.21.15.76:6829/3184128096","state":["exists","up"]},{"osd":11,"uuid":"8fddda46-e22b-4d0c-ac48-e038fb29dfec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":1960689685},{"type":"v1","addr":"172.21.15.136:6825","nonce":1960689685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":1960689685},{"type":"v1","addr":"172.21.15.136:6827","nonce":1960689685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":1960689685},{"type":"v1","addr":"172.21.15.136:6831","nonce":1960689685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":1960689685},{"type":"v1","addr":"172.21.15.136:6829","nonce":1960689685}]},"public_addr":"172.21.15.136:6825/1960689685","cluster_addr":"172.21.15.136:6827/1960689685","heartbeat_back_addr":"172.21.15.136:6831/1960689685","heartbeat_front_addr":"172.21.15.136:6829/1960689685","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:10:56.656177+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:11:22.234031+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:11:46.464102+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:12:12.353967+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:12:37.836200+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:13:01.728839+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:13:27.527230+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:13:51.981393+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:14:17.035464+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:14:42.805029+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:15:08.208999+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:15:32.800844+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.18","mappings":[{"from":3,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":6}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.26:6801/677148151":"2024-09-12T23:09:33.983310+0000","172.21.15.26:6800/677148151":"2024-09-12T23:09:33.983310+0000","172.21.15.26:0/3597996003":"2024-09-12T23:09:33.983310+0000","172.21.15.26:6801/2673575521":"2024-09-12T23:09:09.781665+0000","172.21.15.26:0/883212708":"2024-09-12T23:09:33.983310+0000","172.21.15.26:0/2107281413":"2024-09-12T23:09:33.983310+0000","172.21.15.26:6800/2673575521":"2024-09-12T23:09:09.781665+0000","172.21.15.26:0/2817092653":"2024-09-12T23:09:09.781665+0000","172.21.15.26:0/17142658":"2024-09-12T23:09:09.781665+0000","172.21.15.26:0/3861103130":"2024-09-12T23:09:09.781665+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T23:16:58.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:58.443+0000 7fb26c9e0640 1 -- 172.21.15.26:0/4176755280 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fb2500a2c90 msgr2=0x7fb2500a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:16:58.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:58.443+0000 7fb26c9e0640 1 --2- 172.21.15.26:0/4176755280 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fb2500a2c90 0x7fb2500a5150 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fb25800abe0 tx=0x7fb25800a2d0 comp rx=0 tx=0).stop 2024-09-11T23:16:58.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:58.443+0000 7fb26c9e0640 1 -- 172.21.15.26:0/4176755280 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb268106a60 msgr2=0x7fb2680fd7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:16:58.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:58.443+0000 7fb26c9e0640 1 --2- 172.21.15.26:0/4176755280 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb268106a60 0x7fb2680fd7a0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fb26000b970 tx=0x7fb26000bd70 comp rx=0 tx=0).stop 2024-09-11T23:16:58.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:58.443+0000 7fb26c9e0640 1 -- 172.21.15.26:0/4176755280 shutdown_connections 2024-09-11T23:16:58.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:58.443+0000 7fb26c9e0640 1 --2- 172.21.15.26:0/4176755280 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fb2500a2c90 0x7fb2500a5150 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:58.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:58.443+0000 7fb26c9e0640 1 --2- 172.21.15.26:0/4176755280 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb268101400 0x7fb2680fd260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:58.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:58.443+0000 7fb26c9e0640 1 --2- 172.21.15.26:0/4176755280 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb268106a60 0x7fb2680fd7a0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:58.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:58.443+0000 7fb26c9e0640 1 --2- 172.21.15.26:0/4176755280 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb268100a60 0x7fb2680fcd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:58.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:58.443+0000 7fb26c9e0640 1 -- 172.21.15.26:0/4176755280 >> 172.21.15.26:0/4176755280 conn(0x7fb2680fc780 msgr2=0x7fb26806aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:16:58.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:58.443+0000 7fb26c9e0640 1 -- 172.21.15.26:0/4176755280 shutdown_connections 2024-09-11T23:16:58.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:58.443+0000 7fb26c9e0640 1 -- 172.21.15.26:0/4176755280 wait complete. 2024-09-11T23:16:59.202 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-09-11T23:16:59.202 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-11T23:16:59.463 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:16:59.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:59 smithi136 ceph-mon[35121]: pgmap v297: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.6 KiB/s wr, 9 op/s 2024-09-11T23:16:59.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:16:59 smithi136 ceph-mon[35121]: from='client.? 172.21.15.26:0/4176755280' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T23:16:59.501 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:59 smithi026 ceph-mon[31172]: pgmap v297: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.6 KiB/s wr, 9 op/s 2024-09-11T23:16:59.501 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:16:59 smithi026 ceph-mon[31172]: from='client.? 172.21.15.26:0/4176755280' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T23:16:59.553 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:59 smithi076 ceph-mon[35296]: pgmap v297: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.6 KiB/s wr, 9 op/s 2024-09-11T23:16:59.553 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:16:59 smithi076 ceph-mon[35296]: from='client.? 172.21.15.26:0/4176755280' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T23:16:59.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:59.971+0000 7f604b121640 1 -- 172.21.15.26:0/3926974468 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6044102d90 msgr2=0x7f6044103170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:16:59.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:59.971+0000 7f604b121640 1 --2- 172.21.15.26:0/3926974468 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6044102d90 0x7f6044103170 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f602c009930 tx=0x7f602c02a420 comp rx=0 tx=0).stop 2024-09-11T23:16:59.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:59.971+0000 7f604b121640 1 -- 172.21.15.26:0/3926974468 shutdown_connections 2024-09-11T23:16:59.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:59.971+0000 7f604b121640 1 --2- 172.21.15.26:0/3926974468 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6044103740 0x7f6044107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:59.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:59.971+0000 7f604b121640 1 --2- 172.21.15.26:0/3926974468 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6044108290 0x7f604410ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:59.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:59.971+0000 7f604b121640 1 --2- 172.21.15.26:0/3926974468 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6044102d90 0x7f6044103170 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:59.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:59.971+0000 7f604b121640 1 -- 172.21.15.26:0/3926974468 >> 172.21.15.26:0/3926974468 conn(0x7f60440fc7d0 msgr2=0x7f60440febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:16:59.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:59.971+0000 7f604b121640 1 -- 172.21.15.26:0/3926974468 shutdown_connections 2024-09-11T23:16:59.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:59.971+0000 7f604b121640 1 -- 172.21.15.26:0/3926974468 wait complete. 2024-09-11T23:16:59.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:59.972+0000 7f604b121640 1 Processor -- start 2024-09-11T23:16:59.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:59.972+0000 7f604b121640 1 -- start start 2024-09-11T23:16:59.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:59.972+0000 7f604b121640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6044102d90 0x7f60441009f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:16:59.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:59.972+0000 7f604b121640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6044103740 0x7f6044100f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:16:59.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:59.972+0000 7f604b121640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6044108290 0x7f6044101470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:16:59.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:59.972+0000 7f604b121640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f6044112090 con 0x7f6044102d90 2024-09-11T23:16:59.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:59.972+0000 7f604b121640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f6044111f10 con 0x7f6044103740 2024-09-11T23:16:59.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:59.972+0000 7f604b121640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f6044112210 con 0x7f6044108290 2024-09-11T23:16:59.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:59.972+0000 7f6048ebd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6044102d90 0x7f60441009f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:16:59.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:59.972+0000 7f6048ebd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6044102d90 0x7f60441009f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42180/0 (socket says 172.21.15.26:42180) 2024-09-11T23:16:59.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:59.972+0000 7f6048ebd640 1 -- 172.21.15.26:0/1885476112 learned_addr learned my addr 172.21.15.26:0/1885476112 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:16:59.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:59.973+0000 7f60496be640 1 --2- 172.21.15.26:0/1885476112 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6044108290 0x7f6044101470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:16:59.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:59.973+0000 7f6048ebd640 1 -- 172.21.15.26:0/1885476112 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6044108290 msgr2=0x7f6044101470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:16:59.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:59.973+0000 7f6048ebd640 1 --2- 172.21.15.26:0/1885476112 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6044108290 0x7f6044101470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:59.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:59.973+0000 7f6048ebd640 1 -- 172.21.15.26:0/1885476112 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6044103740 msgr2=0x7f6044100f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:16:59.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:59.973+0000 7f6043fff640 1 --2- 172.21.15.26:0/1885476112 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6044103740 0x7f6044100f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:16:59.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:59.973+0000 7f6048ebd640 1 --2- 172.21.15.26:0/1885476112 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6044103740 0x7f6044100f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:59.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:59.973+0000 7f6048ebd640 1 -- 172.21.15.26:0/1885476112 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60441a6ab0 con 0x7f6044102d90 2024-09-11T23:16:59.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:59.973+0000 7f6048ebd640 1 --2- 172.21.15.26:0/1885476112 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6044102d90 0x7f60441009f0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f602c009230 tx=0x7f602c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:16:59.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:59.973+0000 7f6041ffb640 1 -- 172.21.15.26:0/1885476112 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f602c03f030 con 0x7f6044102d90 2024-09-11T23:16:59.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:59.973+0000 7f604b121640 1 -- 172.21.15.26:0/1885476112 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60441a6d40 con 0x7f6044102d90 2024-09-11T23:16:59.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:59.973+0000 7f6041ffb640 1 -- 172.21.15.26:0/1885476112 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f602c003ad0 con 0x7f6044102d90 2024-09-11T23:16:59.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:59.973+0000 7f6041ffb640 1 -- 172.21.15.26:0/1885476112 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f602c035070 con 0x7f6044102d90 2024-09-11T23:16:59.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:59.973+0000 7f604b121640 1 -- 172.21.15.26:0/1885476112 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60441a71c0 con 0x7f6044102d90 2024-09-11T23:16:59.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:59.974+0000 7f604b121640 1 -- 172.21.15.26:0/1885476112 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f604406a7f0 con 0x7f6044102d90 2024-09-11T23:16:59.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:59.975+0000 7f6041ffb640 1 -- 172.21.15.26:0/1885476112 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f602c042030 con 0x7f6044102d90 2024-09-11T23:16:59.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:59.976+0000 7f6041ffb640 1 --2- 172.21.15.26:0/1885476112 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f60300a2c10 0x7f60300a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:16:59.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:59.976+0000 7f6041ffb640 1 -- 172.21.15.26:0/1885476112 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f602c0ef900 con 0x7f6044102d90 2024-09-11T23:16:59.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:59.976+0000 7f6043fff640 1 --2- 172.21.15.26:0/1885476112 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f60300a2c10 0x7f60300a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:16:59.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:59.977+0000 7f6043fff640 1 --2- 172.21.15.26:0/1885476112 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f60300a2c10 0x7f60300a50d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f6034005c40 tx=0x7f6034005bd0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:16:59.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:16:59.980+0000 7f6041ffb640 1 -- 172.21.15.26:0/1885476112 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f602c0bc6c0 con 0x7f6044102d90 2024-09-11T23:17:00.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:00.158+0000 7f604b121640 1 -- 172.21.15.26:0/1885476112 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6044103170 con 0x7f6044102d90 2024-09-11T23:17:00.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:00.160+0000 7f6041ffb640 1 -- 172.21.15.26:0/1885476112 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+22733 (secure 0 0 0) 0x7f602c0bbfc0 con 0x7f6044102d90 2024-09-11T23:17:00.161 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-11T23:17:00.161 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":85,"fsid":"af7e6fc8-7092-11ef-bcea-c7b262605968","created":"2024-09-11T23:08:40.486438+0000","modified":"2024-09-11T23:16:35.935151+0000","last_up_change":"2024-09-11T23:15:35.179446+0000","last_in_change":"2024-09-11T23:15:11.064627+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T23:11:50.494160+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-11T23:16:22.503928+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-11T23:16:24.876554+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a4d62fcf-c31e-41b1-932c-3d87ff640531","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":2014857905},{"type":"v1","addr":"172.21.15.26:6803","nonce":2014857905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":2014857905},{"type":"v1","addr":"172.21.15.26:6805","nonce":2014857905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":2014857905},{"type":"v1","addr":"172.21.15.26:6809","nonce":2014857905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":2014857905},{"type":"v1","addr":"172.21.15.26:6807","nonce":2014857905}]},"public_addr":"172.21.15.26:6803/2014857905","cluster_addr":"172.21.15.26:6805/2014857905","heartbeat_back_addr":"172.21.15.26:6809/2014857905","heartbeat_front_addr":"172.21.15.26:6807/2014857905","state":["exists","up"]},{"osd":1,"uuid":"e0d69112-3c0a-4fc5-8783-e41b4081e0c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":333383087},{"type":"v1","addr":"172.21.15.76:6801","nonce":333383087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":333383087},{"type":"v1","addr":"172.21.15.76:6803","nonce":333383087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":333383087},{"type":"v1","addr":"172.21.15.76:6807","nonce":333383087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":333383087},{"type":"v1","addr":"172.21.15.76:6805","nonce":333383087}]},"public_addr":"172.21.15.76:6801/333383087","cluster_addr":"172.21.15.76:6803/333383087","heartbeat_back_addr":"172.21.15.76:6807/333383087","heartbeat_front_addr":"172.21.15.76:6805/333383087","state":["exists","up"]},{"osd":2,"uuid":"d578429e-bf23-4a1e-890f-7172955c0eb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":1271538911},{"type":"v1","addr":"172.21.15.136:6801","nonce":1271538911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":1271538911},{"type":"v1","addr":"172.21.15.136:6803","nonce":1271538911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":1271538911},{"type":"v1","addr":"172.21.15.136:6807","nonce":1271538911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":1271538911},{"type":"v1","addr":"172.21.15.136:6805","nonce":1271538911}]},"public_addr":"172.21.15.136:6801/1271538911","cluster_addr":"172.21.15.136:6803/1271538911","heartbeat_back_addr":"172.21.15.136:6807/1271538911","heartbeat_front_addr":"172.21.15.136:6805/1271538911","state":["exists","up"]},{"osd":3,"uuid":"3e34623f-0ea5-4b10-b8a9-f8756c35d512","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":1740304151},{"type":"v1","addr":"172.21.15.26:6811","nonce":1740304151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":1740304151},{"type":"v1","addr":"172.21.15.26:6813","nonce":1740304151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1740304151},{"type":"v1","addr":"172.21.15.26:6817","nonce":1740304151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":1740304151},{"type":"v1","addr":"172.21.15.26:6815","nonce":1740304151}]},"public_addr":"172.21.15.26:6811/1740304151","cluster_addr":"172.21.15.26:6813/1740304151","heartbeat_back_addr":"172.21.15.26:6817/1740304151","heartbeat_front_addr":"172.21.15.26:6815/1740304151","state":["exists","up"]},{"osd":4,"uuid":"77607929-f9ed-49a9-abb5-48cd88eec4ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":2702262182},{"type":"v1","addr":"172.21.15.76:6809","nonce":2702262182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":2702262182},{"type":"v1","addr":"172.21.15.76:6811","nonce":2702262182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":2702262182},{"type":"v1","addr":"172.21.15.76:6815","nonce":2702262182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":2702262182},{"type":"v1","addr":"172.21.15.76:6813","nonce":2702262182}]},"public_addr":"172.21.15.76:6809/2702262182","cluster_addr":"172.21.15.76:6811/2702262182","heartbeat_back_addr":"172.21.15.76:6815/2702262182","heartbeat_front_addr":"172.21.15.76:6813/2702262182","state":["exists","up"]},{"osd":5,"uuid":"29e20e37-09a0-4924-8aaa-17939763a7c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":1749229277},{"type":"v1","addr":"172.21.15.136:6809","nonce":1749229277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":1749229277},{"type":"v1","addr":"172.21.15.136:6811","nonce":1749229277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":1749229277},{"type":"v1","addr":"172.21.15.136:6815","nonce":1749229277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":1749229277},{"type":"v1","addr":"172.21.15.136:6813","nonce":1749229277}]},"public_addr":"172.21.15.136:6809/1749229277","cluster_addr":"172.21.15.136:6811/1749229277","heartbeat_back_addr":"172.21.15.136:6815/1749229277","heartbeat_front_addr":"172.21.15.136:6813/1749229277","state":["exists","up"]},{"osd":6,"uuid":"e0ddd1c0-1c46-41e6-b2f4-396bdcd2bf8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1928245624},{"type":"v1","addr":"172.21.15.26:6819","nonce":1928245624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1928245624},{"type":"v1","addr":"172.21.15.26:6821","nonce":1928245624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":1928245624},{"type":"v1","addr":"172.21.15.26:6825","nonce":1928245624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1928245624},{"type":"v1","addr":"172.21.15.26:6823","nonce":1928245624}]},"public_addr":"172.21.15.26:6819/1928245624","cluster_addr":"172.21.15.26:6821/1928245624","heartbeat_back_addr":"172.21.15.26:6825/1928245624","heartbeat_front_addr":"172.21.15.26:6823/1928245624","state":["exists","up"]},{"osd":7,"uuid":"fe031f5c-d915-48fb-91a6-c76f252467fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":1409789455},{"type":"v1","addr":"172.21.15.76:6817","nonce":1409789455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":1409789455},{"type":"v1","addr":"172.21.15.76:6819","nonce":1409789455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":1409789455},{"type":"v1","addr":"172.21.15.76:6823","nonce":1409789455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":1409789455},{"type":"v1","addr":"172.21.15.76:6821","nonce":1409789455}]},"public_addr":"172.21.15.76:6817/1409789455","cluster_addr":"172.21.15.76:6819/1409789455","heartbeat_back_addr":"172.21.15.76:6823/1409789455","heartbeat_front_addr":"172.21.15.76:6821/1409789455","state":["exists","up"]},{"osd":8,"uuid":"18851bde-9e6c-4e05-b4bf-ac5c3073eee7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":543619079},{"type":"v1","addr":"172.21.15.136:6817","nonce":543619079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":543619079},{"type":"v1","addr":"172.21.15.136:6819","nonce":543619079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":543619079},{"type":"v1","addr":"172.21.15.136:6823","nonce":543619079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":543619079},{"type":"v1","addr":"172.21.15.136:6821","nonce":543619079}]},"public_addr":"172.21.15.136:6817/543619079","cluster_addr":"172.21.15.136:6819/543619079","heartbeat_back_addr":"172.21.15.136:6823/543619079","heartbeat_front_addr":"172.21.15.136:6821/543619079","state":["exists","up"]},{"osd":9,"uuid":"52a65eab-8446-47ad-bfe7-5133f7e3eb98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":2201297064},{"type":"v1","addr":"172.21.15.26:6827","nonce":2201297064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":2201297064},{"type":"v1","addr":"172.21.15.26:6829","nonce":2201297064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":2201297064},{"type":"v1","addr":"172.21.15.26:6833","nonce":2201297064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":2201297064},{"type":"v1","addr":"172.21.15.26:6831","nonce":2201297064}]},"public_addr":"172.21.15.26:6827/2201297064","cluster_addr":"172.21.15.26:6829/2201297064","heartbeat_back_addr":"172.21.15.26:6833/2201297064","heartbeat_front_addr":"172.21.15.26:6831/2201297064","state":["exists","up"]},{"osd":10,"uuid":"b758d4bb-da62-4c5f-a992-bb6a145d35d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":3184128096},{"type":"v1","addr":"172.21.15.76:6825","nonce":3184128096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":3184128096},{"type":"v1","addr":"172.21.15.76:6827","nonce":3184128096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":3184128096},{"type":"v1","addr":"172.21.15.76:6831","nonce":3184128096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":3184128096},{"type":"v1","addr":"172.21.15.76:6829","nonce":3184128096}]},"public_addr":"172.21.15.76:6825/3184128096","cluster_addr":"172.21.15.76:6827/3184128096","heartbeat_back_addr":"172.21.15.76:6831/3184128096","heartbeat_front_addr":"172.21.15.76:6829/3184128096","state":["exists","up"]},{"osd":11,"uuid":"8fddda46-e22b-4d0c-ac48-e038fb29dfec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":1960689685},{"type":"v1","addr":"172.21.15.136:6825","nonce":1960689685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":1960689685},{"type":"v1","addr":"172.21.15.136:6827","nonce":1960689685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":1960689685},{"type":"v1","addr":"172.21.15.136:6831","nonce":1960689685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":1960689685},{"type":"v1","addr":"172.21.15.136:6829","nonce":1960689685}]},"public_addr":"172.21.15.136:6825/1960689685","cluster_addr":"172.21.15.136:6827/1960689685","heartbeat_back_addr":"172.21.15.136:6831/1960689685","heartbeat_front_addr":"172.21.15.136:6829/1960689685","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:10:56.656177+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:11:22.234031+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:11:46.464102+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:12:12.353967+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:12:37.836200+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:13:01.728839+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:13:27.527230+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:13:51.981393+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:14:17.035464+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:14:42.805029+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:15:08.208999+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:15:32.800844+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.18","mappings":[{"from":3,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":6}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.26:6801/677148151":"2024-09-12T23:09:33.983310+0000","172.21.15.26:6800/677148151":"2024-09-12T23:09:33.983310+0000","172.21.15.26:0/3597996003":"2024-09-12T23:09:33.983310+0000","172.21.15.26:6801/2673575521":"2024-09-12T23:09:09.781665+0000","172.21.15.26:0/883212708":"2024-09-12T23:09:33.983310+0000","172.21.15.26:0/2107281413":"2024-09-12T23:09:33.983310+0000","172.21.15.26:6800/2673575521":"2024-09-12T23:09:09.781665+0000","172.21.15.26:0/2817092653":"2024-09-12T23:09:09.781665+0000","172.21.15.26:0/17142658":"2024-09-12T23:09:09.781665+0000","172.21.15.26:0/3861103130":"2024-09-12T23:09:09.781665+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T23:17:00.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:00.163+0000 7f604b121640 1 -- 172.21.15.26:0/1885476112 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f60300a2c10 msgr2=0x7f60300a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:00.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:00.163+0000 7f604b121640 1 --2- 172.21.15.26:0/1885476112 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f60300a2c10 0x7f60300a50d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f6034005c40 tx=0x7f6034005bd0 comp rx=0 tx=0).stop 2024-09-11T23:17:00.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:00.163+0000 7f604b121640 1 -- 172.21.15.26:0/1885476112 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6044102d90 msgr2=0x7f60441009f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:00.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:00.163+0000 7f604b121640 1 --2- 172.21.15.26:0/1885476112 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6044102d90 0x7f60441009f0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f602c009230 tx=0x7f602c004270 comp rx=0 tx=0).stop 2024-09-11T23:17:00.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:00.163+0000 7f604b121640 1 -- 172.21.15.26:0/1885476112 shutdown_connections 2024-09-11T23:17:00.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:00.163+0000 7f604b121640 1 --2- 172.21.15.26:0/1885476112 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f60300a2c10 0x7f60300a50d0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:00.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:00.163+0000 7f604b121640 1 --2- 172.21.15.26:0/1885476112 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6044103740 0x7f6044100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:00.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:00.163+0000 7f604b121640 1 --2- 172.21.15.26:0/1885476112 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6044108290 0x7f6044101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:00.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:00.163+0000 7f604b121640 1 --2- 172.21.15.26:0/1885476112 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6044102d90 0x7f60441009f0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:00.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:00.163+0000 7f604b121640 1 -- 172.21.15.26:0/1885476112 >> 172.21.15.26:0/1885476112 conn(0x7f60440fc7d0 msgr2=0x7f6044106370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:17:00.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:00.163+0000 7f604b121640 1 -- 172.21.15.26:0/1885476112 shutdown_connections 2024-09-11T23:17:00.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:00.163+0000 7f604b121640 1 -- 172.21.15.26:0/1885476112 wait complete. 2024-09-11T23:17:00.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:17:00 smithi136 ceph-mon[35121]: from='client.? 172.21.15.26:0/1885476112' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T23:17:00.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:17:00 smithi026 ceph-mon[31172]: from='client.? 172.21.15.26:0/1885476112' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T23:17:00.552 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:17:00 smithi076 ceph-mon[35296]: from='client.? 172.21.15.26:0/1885476112' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T23:17:01.046 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-09-11T23:17:01.046 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-09-11T23:17:01.046 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-09-11T23:17:01.047 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-09-11T23:17:01.047 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-09-11T23:17:01.048 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-09-11T23:17:01.048 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-09-11T23:17:01.048 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-09-11T23:17:01.049 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-09-11T23:17:01.049 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-09-11T23:17:01.050 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-09-11T23:17:01.050 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-09-11T23:17:01.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:17:01 smithi136 ceph-mon[35121]: pgmap v298: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2024-09-11T23:17:01.501 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:17:01 smithi026 ceph-mon[31172]: pgmap v298: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2024-09-11T23:17:01.553 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:17:01 smithi076 ceph-mon[35296]: pgmap v298: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2024-09-11T23:17:01.690 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:17:02.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:02.214+0000 7f9e50d4b640 1 -- 172.21.15.26:0/3866022957 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9e4c100b20 msgr2=0x7f9e4c100f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:02.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:02.214+0000 7f9e50d4b640 1 --2- 172.21.15.26:0/3866022957 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9e4c100b20 0x7f9e4c100f00 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f9e3c009ca0 tx=0x7f9e3c02a8c0 comp rx=0 tx=0).stop 2024-09-11T23:17:02.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:02.214+0000 7f9e50d4b640 1 -- 172.21.15.26:0/3866022957 shutdown_connections 2024-09-11T23:17:02.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:02.214+0000 7f9e50d4b640 1 --2- 172.21.15.26:0/3866022957 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9e4c10e500 0x7f9e4c110910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:02.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:02.214+0000 7f9e50d4b640 1 --2- 172.21.15.26:0/3866022957 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9e4c101440 0x7f9e4c10df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:02.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:02.214+0000 7f9e50d4b640 1 --2- 172.21.15.26:0/3866022957 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9e4c100b20 0x7f9e4c100f00 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:02.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:02.214+0000 7f9e50d4b640 1 -- 172.21.15.26:0/3866022957 >> 172.21.15.26:0/3866022957 conn(0x7f9e4c0fc7d0 msgr2=0x7f9e4c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:17:02.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:02.214+0000 7f9e50d4b640 1 -- 172.21.15.26:0/3866022957 shutdown_connections 2024-09-11T23:17:02.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:02.214+0000 7f9e50d4b640 1 -- 172.21.15.26:0/3866022957 wait complete. 2024-09-11T23:17:02.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:02.215+0000 7f9e50d4b640 1 Processor -- start 2024-09-11T23:17:02.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:02.215+0000 7f9e50d4b640 1 -- start start 2024-09-11T23:17:02.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:02.215+0000 7f9e50d4b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9e4c100b20 0x7f9e4c1a0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:02.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:02.215+0000 7f9e50d4b640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9e4c101440 0x7f9e4c1a0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:02.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:02.215+0000 7f9e50d4b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9e4c10e500 0x7f9e4c19a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:02.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:02.215+0000 7f9e50d4b640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f9e4c114370 con 0x7f9e4c100b20 2024-09-11T23:17:02.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:02.215+0000 7f9e50d4b640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f9e4c1141f0 con 0x7f9e4c10e500 2024-09-11T23:17:02.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:02.215+0000 7f9e50d4b640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f9e4c1144f0 con 0x7f9e4c101440 2024-09-11T23:17:02.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:02.215+0000 7f9e4a59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9e4c100b20 0x7f9e4c1a0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:02.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:02.215+0000 7f9e4a59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9e4c100b20 0x7f9e4c1a0670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52276/0 (socket says 172.21.15.26:52276) 2024-09-11T23:17:02.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:02.215+0000 7f9e4a59c640 1 -- 172.21.15.26:0/3754108161 learned_addr learned my addr 172.21.15.26:0/3754108161 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:17:02.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:02.215+0000 7f9e4ad9d640 1 --2- 172.21.15.26:0/3754108161 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9e4c10e500 0x7f9e4c19a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:02.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:02.215+0000 7f9e4a59c640 1 -- 172.21.15.26:0/3754108161 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9e4c101440 msgr2=0x7f9e4c1a0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:02.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:02.215+0000 7f9e49d9b640 1 --2- 172.21.15.26:0/3754108161 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9e4c101440 0x7f9e4c1a0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:02.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:02.215+0000 7f9e4a59c640 1 --2- 172.21.15.26:0/3754108161 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9e4c101440 0x7f9e4c1a0bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:02.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:02.215+0000 7f9e4a59c640 1 -- 172.21.15.26:0/3754108161 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9e4c10e500 msgr2=0x7f9e4c19a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:02.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:02.215+0000 7f9e4a59c640 1 --2- 172.21.15.26:0/3754108161 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9e4c10e500 0x7f9e4c19a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:02.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:02.215+0000 7f9e4a59c640 1 -- 172.21.15.26:0/3754108161 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e4c19afb0 con 0x7f9e4c100b20 2024-09-11T23:17:02.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:02.215+0000 7f9e4ad9d640 1 --2- 172.21.15.26:0/3754108161 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9e4c10e500 0x7f9e4c19a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:17:02.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:02.215+0000 7f9e4a59c640 1 --2- 172.21.15.26:0/3754108161 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9e4c100b20 0x7f9e4c1a0670 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f9e3c0097b0 tx=0x7f9e3c031b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:17:02.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:02.216+0000 7f9e437fe640 1 -- 172.21.15.26:0/3754108161 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e3c02fba0 con 0x7f9e4c100b20 2024-09-11T23:17:02.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:02.216+0000 7f9e50d4b640 1 -- 172.21.15.26:0/3754108161 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e4c19b240 con 0x7f9e4c100b20 2024-09-11T23:17:02.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:02.216+0000 7f9e437fe640 1 -- 172.21.15.26:0/3754108161 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e3c02fd40 con 0x7f9e4c100b20 2024-09-11T23:17:02.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:02.216+0000 7f9e437fe640 1 -- 172.21.15.26:0/3754108161 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e3c03b930 con 0x7f9e4c100b20 2024-09-11T23:17:02.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:02.216+0000 7f9e50d4b640 1 -- 172.21.15.26:0/3754108161 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e4c1a7420 con 0x7f9e4c100b20 2024-09-11T23:17:02.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:02.217+0000 7f9e50d4b640 1 -- 172.21.15.26:0/3754108161 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9e4c062940 con 0x7f9e4c100b20 2024-09-11T23:17:02.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:02.218+0000 7f9e437fe640 1 -- 172.21.15.26:0/3754108161 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f9e3c030cc0 con 0x7f9e4c100b20 2024-09-11T23:17:02.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:02.218+0000 7f9e437fe640 1 --2- 172.21.15.26:0/3754108161 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f9e380a2c10 0x7f9e380a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:02.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:02.219+0000 7f9e49d9b640 1 --2- 172.21.15.26:0/3754108161 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f9e380a2c10 0x7f9e380a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:02.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:02.219+0000 7f9e437fe640 1 -- 172.21.15.26:0/3754108161 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f9e3c0f0a30 con 0x7f9e4c100b20 2024-09-11T23:17:02.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:02.219+0000 7f9e437fe640 1 --2- 172.21.15.26:0/3754108161 >> [v2:172.21.15.26:6818/1928245624,v1:172.21.15.26:6819/1928245624] conn(0x7f9e380ac840 0x7f9e380aece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:02.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:02.219+0000 7f9e437fe640 1 -- 172.21.15.26:0/3754108161 --> [v2:172.21.15.26:6818/1928245624,v1:172.21.15.26:6819/1928245624] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9e380af3b0 con 0x7f9e380ac840 2024-09-11T23:17:02.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:02.219+0000 7f9e437fe640 1 -- 172.21.15.26:0/3754108161 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7f9e3c0f0e60 con 0x7f9e4c100b20 2024-09-11T23:17:02.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:02.219+0000 7f9e4ad9d640 1 --2- 172.21.15.26:0/3754108161 >> [v2:172.21.15.26:6818/1928245624,v1:172.21.15.26:6819/1928245624] conn(0x7f9e380ac840 0x7f9e380aece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:02.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:02.219+0000 7f9e49d9b640 1 --2- 172.21.15.26:0/3754108161 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f9e380a2c10 0x7f9e380a50d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f9e4c19bf90 tx=0x7f9e34008040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:17:02.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:02.219+0000 7f9e4ad9d640 1 --2- 172.21.15.26:0/3754108161 >> [v2:172.21.15.26:6818/1928245624,v1:172.21.15.26:6819/1928245624] conn(0x7f9e380ac840 0x7f9e380aece0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:17:02.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:02.220+0000 7f9e437fe640 1 -- 172.21.15.26:0/3754108161 <== osd.6 v2:172.21.15.26:6818/1928245624 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9e380af3b0 con 0x7f9e380ac840 2024-09-11T23:17:02.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:02.234+0000 7f9e50d4b640 1 -- 172.21.15.26:0/3754108161 --> [v2:172.21.15.26:6818/1928245624,v1:172.21.15.26:6819/1928245624] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9e4c062dc0 con 0x7f9e380ac840 2024-09-11T23:17:02.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:02.234+0000 7f9e437fe640 1 -- 172.21.15.26:0/3754108161 <== osd.6 v2:172.21.15.26:6818/1928245624 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f9e4c062dc0 con 0x7f9e380ac840 2024-09-11T23:17:02.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:02.234+0000 7f9e50d4b640 1 -- 172.21.15.26:0/3754108161 >> [v2:172.21.15.26:6818/1928245624,v1:172.21.15.26:6819/1928245624] conn(0x7f9e380ac840 msgr2=0x7f9e380aece0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:02.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:02.234+0000 7f9e50d4b640 1 --2- 172.21.15.26:0/3754108161 >> [v2:172.21.15.26:6818/1928245624,v1:172.21.15.26:6819/1928245624] conn(0x7f9e380ac840 0x7f9e380aece0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:02.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:02.234+0000 7f9e50d4b640 1 -- 172.21.15.26:0/3754108161 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f9e380a2c10 msgr2=0x7f9e380a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:02.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:02.234+0000 7f9e50d4b640 1 --2- 172.21.15.26:0/3754108161 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f9e380a2c10 0x7f9e380a50d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f9e4c19bf90 tx=0x7f9e34008040 comp rx=0 tx=0).stop 2024-09-11T23:17:02.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:02.234+0000 7f9e50d4b640 1 -- 172.21.15.26:0/3754108161 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9e4c100b20 msgr2=0x7f9e4c1a0670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:02.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:02.234+0000 7f9e50d4b640 1 --2- 172.21.15.26:0/3754108161 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9e4c100b20 0x7f9e4c1a0670 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f9e3c0097b0 tx=0x7f9e3c031b10 comp rx=0 tx=0).stop 2024-09-11T23:17:02.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:02.234+0000 7f9e4ad9d640 1 -- 172.21.15.26:0/3754108161 reap_dead start 2024-09-11T23:17:02.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:02.234+0000 7f9e50d4b640 1 -- 172.21.15.26:0/3754108161 shutdown_connections 2024-09-11T23:17:02.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:02.234+0000 7f9e50d4b640 1 -- 172.21.15.26:0/3754108161 >> 172.21.15.26:0/3754108161 conn(0x7f9e4c0fc7d0 msgr2=0x7f9e4c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:17:02.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:02.235+0000 7f9e50d4b640 1 -- 172.21.15.26:0/3754108161 shutdown_connections 2024-09-11T23:17:02.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:02.235+0000 7f9e50d4b640 1 -- 172.21.15.26:0/3754108161 wait complete. 2024-09-11T23:17:03.011 INFO:teuthology.orchestra.run.smithi026.stdout:193273528364 2024-09-11T23:17:03.011 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-09-11T23:17:03.088 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:17:03.088 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:17:03.089 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:17:03.094 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:17:03.094 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:17:03.102 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:17:03.107 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:17:03.117 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:17:03.118 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:17:03.201 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:17:03.206 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:17:03.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:17:03 smithi026 ceph-mon[31172]: pgmap v299: 97 pgs: 97 active+clean; 641 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2024-09-11T23:17:03.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:17:03 smithi136 ceph-mon[35121]: pgmap v299: 97 pgs: 97 active+clean; 641 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2024-09-11T23:17:03.553 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:17:03 smithi076 ceph-mon[35296]: pgmap v299: 97 pgs: 97 active+clean; 641 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2024-09-11T23:17:04.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:04.521+0000 7f36c31b5640 1 -- 172.21.15.26:0/2162795143 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f36bc100e20 msgr2=0x7f36bc101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:04.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:04.522+0000 7f36c31b5640 1 --2- 172.21.15.26:0/2162795143 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f36bc100e20 0x7f36bc101200 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f36b0009900 tx=0x7f36b002a480 comp rx=0 tx=0).stop 2024-09-11T23:17:04.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:04.522+0000 7f36c31b5640 1 -- 172.21.15.26:0/2162795143 shutdown_connections 2024-09-11T23:17:04.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:04.522+0000 7f36c31b5640 1 --2- 172.21.15.26:0/2162795143 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f36bc102570 0x7f36bc1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:04.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:04.522+0000 7f36c31b5640 1 --2- 172.21.15.26:0/2162795143 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f36bc102ef0 0x7f36bc10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:04.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:04.522+0000 7f36c31b5640 1 --2- 172.21.15.26:0/2162795143 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f36bc100e20 0x7f36bc101200 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:04.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:04.522+0000 7f36c31b5640 1 -- 172.21.15.26:0/2162795143 >> 172.21.15.26:0/2162795143 conn(0x7f36bc0fc7d0 msgr2=0x7f36bc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:17:04.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:04.522+0000 7f36c31b5640 1 -- 172.21.15.26:0/2162795143 shutdown_connections 2024-09-11T23:17:04.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:04.522+0000 7f36c31b5640 1 -- 172.21.15.26:0/2162795143 wait complete. 2024-09-11T23:17:04.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:04.522+0000 7f36c31b5640 1 Processor -- start 2024-09-11T23:17:04.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:04.523+0000 7f36c31b5640 1 -- start start 2024-09-11T23:17:04.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:04.523+0000 7f36c31b5640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f36bc100e20 0x7f36bc197440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:04.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:04.523+0000 7f36c31b5640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f36bc102570 0x7f36bc197980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:04.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:04.523+0000 7f36c31b5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f36bc102ef0 0x7f36bc19bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:04.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:04.523+0000 7f36c31b5640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f36bc110fa0 con 0x7f36bc102ef0 2024-09-11T23:17:04.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:04.523+0000 7f36c31b5640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f36bc110e20 con 0x7f36bc102570 2024-09-11T23:17:04.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:04.523+0000 7f36c31b5640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f36bc111120 con 0x7f36bc100e20 2024-09-11T23:17:04.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:04.523+0000 7f36c1752640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f36bc102ef0 0x7f36bc19bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:04.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:04.523+0000 7f36c1752640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f36bc102ef0 0x7f36bc19bd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52296/0 (socket says 172.21.15.26:52296) 2024-09-11T23:17:04.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:04.523+0000 7f36c1752640 1 -- 172.21.15.26:0/2818174455 learned_addr learned my addr 172.21.15.26:0/2818174455 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:17:04.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:04.523+0000 7f36bbfff640 1 --2- 172.21.15.26:0/2818174455 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f36bc102570 0x7f36bc197980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:04.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:04.523+0000 7f36c1752640 1 -- 172.21.15.26:0/2818174455 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f36bc100e20 msgr2=0x7f36bc197440 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-11T23:17:04.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:04.523+0000 7f36c1752640 1 --2- 172.21.15.26:0/2818174455 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f36bc100e20 0x7f36bc197440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:04.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:04.523+0000 7f36c1752640 1 -- 172.21.15.26:0/2818174455 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f36bc102570 msgr2=0x7f36bc197980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:04.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:04.523+0000 7f36c1752640 1 --2- 172.21.15.26:0/2818174455 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f36bc102570 0x7f36bc197980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:04.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:04.523+0000 7f36c1752640 1 -- 172.21.15.26:0/2818174455 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36bc19c3f0 con 0x7f36bc102ef0 2024-09-11T23:17:04.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:04.523+0000 7f36bbfff640 1 --2- 172.21.15.26:0/2818174455 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f36bc102570 0x7f36bc197980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:17:04.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:04.523+0000 7f36c1752640 1 --2- 172.21.15.26:0/2818174455 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f36bc102ef0 0x7f36bc19bd10 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f36b400b970 tx=0x7f36b400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:17:04.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:04.524+0000 7f36b9ffb640 1 -- 172.21.15.26:0/2818174455 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36b4019070 con 0x7f36bc102ef0 2024-09-11T23:17:04.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:04.524+0000 7f36b9ffb640 1 -- 172.21.15.26:0/2818174455 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36b400cb40 con 0x7f36bc102ef0 2024-09-11T23:17:04.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:04.524+0000 7f36c31b5640 1 -- 172.21.15.26:0/2818174455 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36bc19c6e0 con 0x7f36bc102ef0 2024-09-11T23:17:04.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:04.524+0000 7f36b9ffb640 1 -- 172.21.15.26:0/2818174455 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36b4007650 con 0x7f36bc102ef0 2024-09-11T23:17:04.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:04.524+0000 7f36c31b5640 1 -- 172.21.15.26:0/2818174455 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36bc1a3fc0 con 0x7f36bc102ef0 2024-09-11T23:17:04.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:04.525+0000 7f36c31b5640 1 -- 172.21.15.26:0/2818174455 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f36bc062980 con 0x7f36bc102ef0 2024-09-11T23:17:04.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:04.526+0000 7f36b9ffb640 1 -- 172.21.15.26:0/2818174455 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f36b401c070 con 0x7f36bc102ef0 2024-09-11T23:17:04.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:04.527+0000 7f36b9ffb640 1 --2- 172.21.15.26:0/2818174455 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f36ac0a2c10 0x7f36ac0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:04.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:04.527+0000 7f36c0f51640 1 --2- 172.21.15.26:0/2818174455 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f36ac0a2c10 0x7f36ac0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:04.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:04.527+0000 7f36b9ffb640 1 -- 172.21.15.26:0/2818174455 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f36b40cef30 con 0x7f36bc102ef0 2024-09-11T23:17:04.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:04.527+0000 7f36b9ffb640 1 --2- 172.21.15.26:0/2818174455 >> [v2:172.21.15.76:6808/2702262182,v1:172.21.15.76:6809/2702262182] conn(0x7f36ac0ac840 0x7f36ac0aece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:04.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:04.527+0000 7f36b9ffb640 1 -- 172.21.15.26:0/2818174455 --> [v2:172.21.15.76:6808/2702262182,v1:172.21.15.76:6809/2702262182] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f36b40047d0 con 0x7f36ac0ac840 2024-09-11T23:17:04.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:04.527+0000 7f36b9ffb640 1 -- 172.21.15.26:0/2818174455 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7f36b40cf360 con 0x7f36bc102ef0 2024-09-11T23:17:04.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:04.527+0000 7f36c0f51640 1 --2- 172.21.15.26:0/2818174455 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f36ac0a2c10 0x7f36ac0a50d0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f36b00314e0 tx=0x7f36b0000bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:17:04.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:04.527+0000 7f36bbfff640 1 --2- 172.21.15.26:0/2818174455 >> [v2:172.21.15.76:6808/2702262182,v1:172.21.15.76:6809/2702262182] conn(0x7f36ac0ac840 0x7f36ac0aece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:04.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:04.528+0000 7f36bbfff640 1 --2- 172.21.15.26:0/2818174455 >> [v2:172.21.15.76:6808/2702262182,v1:172.21.15.76:6809/2702262182] conn(0x7f36ac0ac840 0x7f36ac0aece0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:17:04.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:04.529+0000 7f36b9ffb640 1 -- 172.21.15.26:0/2818174455 <== osd.4 v2:172.21.15.76:6808/2702262182 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f36b40047d0 con 0x7f36ac0ac840 2024-09-11T23:17:04.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:04.556+0000 7f36c31b5640 1 -- 172.21.15.26:0/2818174455 --> [v2:172.21.15.76:6808/2702262182,v1:172.21.15.76:6809/2702262182] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f36bc062e00 con 0x7f36ac0ac840 2024-09-11T23:17:04.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:04.557+0000 7f36b9ffb640 1 -- 172.21.15.26:0/2818174455 <== osd.4 v2:172.21.15.76:6808/2702262182 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f36bc062e00 con 0x7f36ac0ac840 2024-09-11T23:17:04.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:04.557+0000 7f36c31b5640 1 -- 172.21.15.26:0/2818174455 >> [v2:172.21.15.76:6808/2702262182,v1:172.21.15.76:6809/2702262182] conn(0x7f36ac0ac840 msgr2=0x7f36ac0aece0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:04.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:04.557+0000 7f36c31b5640 1 --2- 172.21.15.26:0/2818174455 >> [v2:172.21.15.76:6808/2702262182,v1:172.21.15.76:6809/2702262182] conn(0x7f36ac0ac840 0x7f36ac0aece0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:04.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:04.558+0000 7f36c31b5640 1 -- 172.21.15.26:0/2818174455 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f36ac0a2c10 msgr2=0x7f36ac0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:04.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:04.558+0000 7f36c31b5640 1 --2- 172.21.15.26:0/2818174455 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f36ac0a2c10 0x7f36ac0a50d0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f36b00314e0 tx=0x7f36b0000bc0 comp rx=0 tx=0).stop 2024-09-11T23:17:04.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:04.558+0000 7f36c31b5640 1 -- 172.21.15.26:0/2818174455 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f36bc102ef0 msgr2=0x7f36bc19bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:04.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:04.558+0000 7f36c31b5640 1 --2- 172.21.15.26:0/2818174455 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f36bc102ef0 0x7f36bc19bd10 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f36b400b970 tx=0x7f36b400be30 comp rx=0 tx=0).stop 2024-09-11T23:17:04.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:04.558+0000 7f36c1752640 1 -- 172.21.15.26:0/2818174455 reap_dead start 2024-09-11T23:17:04.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:04.558+0000 7f36c31b5640 1 -- 172.21.15.26:0/2818174455 shutdown_connections 2024-09-11T23:17:04.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:04.558+0000 7f36c31b5640 1 -- 172.21.15.26:0/2818174455 >> 172.21.15.26:0/2818174455 conn(0x7f36bc0fc7d0 msgr2=0x7f36bc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:17:04.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:04.558+0000 7f36c31b5640 1 -- 172.21.15.26:0/2818174455 shutdown_connections 2024-09-11T23:17:04.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:04.558+0000 7f36c31b5640 1 -- 172.21.15.26:0/2818174455 wait complete. 2024-09-11T23:17:05.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:05.176+0000 7fb1ef844640 1 -- 172.21.15.26:0/2089059790 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb1e8103740 msgr2=0x7fb1e8107b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:05.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:05.176+0000 7fb1ef844640 1 --2- 172.21.15.26:0/2089059790 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb1e8103740 0x7fb1e8107b80 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fb1d8009930 tx=0x7fb1d802a420 comp rx=0 tx=0).stop 2024-09-11T23:17:05.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:05.176+0000 7fb1ef844640 1 -- 172.21.15.26:0/2089059790 shutdown_connections 2024-09-11T23:17:05.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:05.176+0000 7fb1ef844640 1 --2- 172.21.15.26:0/2089059790 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb1e8108290 0x7fb1e810ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:05.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:05.176+0000 7fb1ef844640 1 --2- 172.21.15.26:0/2089059790 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb1e8103740 0x7fb1e8107b80 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:05.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:05.176+0000 7fb1ef844640 1 --2- 172.21.15.26:0/2089059790 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb1e8102d90 0x7fb1e8103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:05.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:05.177+0000 7fb1ef844640 1 -- 172.21.15.26:0/2089059790 >> 172.21.15.26:0/2089059790 conn(0x7fb1e80fc7d0 msgr2=0x7fb1e80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:17:05.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:05.177+0000 7fb1ef844640 1 -- 172.21.15.26:0/2089059790 shutdown_connections 2024-09-11T23:17:05.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:05.177+0000 7fb1ef844640 1 -- 172.21.15.26:0/2089059790 wait complete. 2024-09-11T23:17:05.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:05.177+0000 7fb1ef844640 1 Processor -- start 2024-09-11T23:17:05.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:05.178+0000 7fb1ef844640 1 -- start start 2024-09-11T23:17:05.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:05.178+0000 7fb1ef844640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb1e8102d90 0x7fb1e819e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:05.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:05.178+0000 7fb1ef844640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb1e8103740 0x7fb1e819e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:05.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:05.178+0000 7fb1ef844640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb1e8108290 0x7fb1e81985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:05.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:05.178+0000 7fb1ef844640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb1e8112130 con 0x7fb1e8102d90 2024-09-11T23:17:05.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:05.178+0000 7fb1ef844640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fb1e8111fb0 con 0x7fb1e8108290 2024-09-11T23:17:05.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:05.178+0000 7fb1ef844640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb1e81122b0 con 0x7fb1e8103740 2024-09-11T23:17:05.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:05.178+0000 7fb1ed5e0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb1e8102d90 0x7fb1e819e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:05.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:05.178+0000 7fb1ed5e0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb1e8102d90 0x7fb1e819e3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52314/0 (socket says 172.21.15.26:52314) 2024-09-11T23:17:05.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:05.178+0000 7fb1ed5e0640 1 -- 172.21.15.26:0/946659081 learned_addr learned my addr 172.21.15.26:0/946659081 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:17:05.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:05.178+0000 7fb1edde1640 1 --2- 172.21.15.26:0/946659081 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb1e8108290 0x7fb1e81985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:05.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:05.178+0000 7fb1ecddf640 1 --2- 172.21.15.26:0/946659081 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb1e8103740 0x7fb1e819e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:05.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:05.178+0000 7fb1ed5e0640 1 -- 172.21.15.26:0/946659081 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb1e8103740 msgr2=0x7fb1e819e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:05.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:05.178+0000 7fb1ed5e0640 1 --2- 172.21.15.26:0/946659081 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb1e8103740 0x7fb1e819e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:05.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:05.178+0000 7fb1ed5e0640 1 -- 172.21.15.26:0/946659081 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb1e8108290 msgr2=0x7fb1e81985b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:05.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:05.178+0000 7fb1ed5e0640 1 --2- 172.21.15.26:0/946659081 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb1e8108290 0x7fb1e81985b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:05.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:05.178+0000 7fb1ed5e0640 1 -- 172.21.15.26:0/946659081 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1e8198ce0 con 0x7fb1e8102d90 2024-09-11T23:17:05.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:05.179+0000 7fb1ed5e0640 1 --2- 172.21.15.26:0/946659081 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb1e8102d90 0x7fb1e819e3f0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fb1e400b4a0 tx=0x7fb1e400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:17:05.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:05.179+0000 7fb1de7fc640 1 -- 172.21.15.26:0/946659081 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1e4014070 con 0x7fb1e8102d90 2024-09-11T23:17:05.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:05.179+0000 7fb1ef844640 1 -- 172.21.15.26:0/946659081 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1e8198fd0 con 0x7fb1e8102d90 2024-09-11T23:17:05.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:05.179+0000 7fb1de7fc640 1 -- 172.21.15.26:0/946659081 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb1e4010af0 con 0x7fb1e8102d90 2024-09-11T23:17:05.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:05.179+0000 7fb1de7fc640 1 -- 172.21.15.26:0/946659081 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1e4017d10 con 0x7fb1e8102d90 2024-09-11T23:17:05.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:05.179+0000 7fb1ef844640 1 -- 172.21.15.26:0/946659081 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1e81a52c0 con 0x7fb1e8102d90 2024-09-11T23:17:05.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:05.180+0000 7fb1ef844640 1 -- 172.21.15.26:0/946659081 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb1e8062900 con 0x7fb1e8102d90 2024-09-11T23:17:05.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:05.181+0000 7fb1de7fc640 1 -- 172.21.15.26:0/946659081 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fb1e4017070 con 0x7fb1e8102d90 2024-09-11T23:17:05.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:05.182+0000 7fb1de7fc640 1 --2- 172.21.15.26:0/946659081 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fb1d40a2c10 0x7fb1d40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:05.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:05.182+0000 7fb1ecddf640 1 --2- 172.21.15.26:0/946659081 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fb1d40a2c10 0x7fb1d40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:05.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:05.182+0000 7fb1de7fc640 1 -- 172.21.15.26:0/946659081 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fb1e40ce580 con 0x7fb1e8102d90 2024-09-11T23:17:05.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:05.182+0000 7fb1de7fc640 1 --2- 172.21.15.26:0/946659081 >> [v2:172.21.15.76:6824/3184128096,v1:172.21.15.76:6825/3184128096] conn(0x7fb1d40ac840 0x7fb1d40aece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:05.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:05.182+0000 7fb1de7fc640 1 -- 172.21.15.26:0/946659081 --> [v2:172.21.15.76:6824/3184128096,v1:172.21.15.76:6825/3184128096] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb1e401ce50 con 0x7fb1d40ac840 2024-09-11T23:17:05.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:05.182+0000 7fb1de7fc640 1 -- 172.21.15.26:0/946659081 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7fb1e40d5050 con 0x7fb1e8102d90 2024-09-11T23:17:05.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:05.182+0000 7fb1ecddf640 1 --2- 172.21.15.26:0/946659081 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fb1d40a2c10 0x7fb1d40a50d0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fb1d8006010 tx=0x7fb1d8031690 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:17:05.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:05.183+0000 7fb1edde1640 1 --2- 172.21.15.26:0/946659081 >> [v2:172.21.15.76:6824/3184128096,v1:172.21.15.76:6825/3184128096] conn(0x7fb1d40ac840 0x7fb1d40aece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:05.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:05.183+0000 7fb1edde1640 1 --2- 172.21.15.26:0/946659081 >> [v2:172.21.15.76:6824/3184128096,v1:172.21.15.76:6825/3184128096] conn(0x7fb1d40ac840 0x7fb1d40aece0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:17:05.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:05.184+0000 7fb1de7fc640 1 -- 172.21.15.26:0/946659081 <== osd.10 v2:172.21.15.76:6824/3184128096 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb1e401ce50 con 0x7fb1d40ac840 2024-09-11T23:17:05.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:05.198+0000 7fb1ef844640 1 -- 172.21.15.26:0/946659081 --> [v2:172.21.15.76:6824/3184128096,v1:172.21.15.76:6825/3184128096] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb1e8062d80 con 0x7fb1d40ac840 2024-09-11T23:17:05.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:05.199+0000 7fb1de7fc640 1 -- 172.21.15.26:0/946659081 <== osd.10 v2:172.21.15.76:6824/3184128096 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fb1e8062d80 con 0x7fb1d40ac840 2024-09-11T23:17:05.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:05.199+0000 7fb1ef844640 1 -- 172.21.15.26:0/946659081 >> [v2:172.21.15.76:6824/3184128096,v1:172.21.15.76:6825/3184128096] conn(0x7fb1d40ac840 msgr2=0x7fb1d40aece0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:05.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:05.199+0000 7fb1ef844640 1 --2- 172.21.15.26:0/946659081 >> [v2:172.21.15.76:6824/3184128096,v1:172.21.15.76:6825/3184128096] conn(0x7fb1d40ac840 0x7fb1d40aece0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:05.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:05.199+0000 7fb1ef844640 1 -- 172.21.15.26:0/946659081 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fb1d40a2c10 msgr2=0x7fb1d40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:05.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:05.199+0000 7fb1ef844640 1 --2- 172.21.15.26:0/946659081 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fb1d40a2c10 0x7fb1d40a50d0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fb1d8006010 tx=0x7fb1d8031690 comp rx=0 tx=0).stop 2024-09-11T23:17:05.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:05.199+0000 7fb1ef844640 1 -- 172.21.15.26:0/946659081 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb1e8102d90 msgr2=0x7fb1e819e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:05.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:05.199+0000 7fb1ef844640 1 --2- 172.21.15.26:0/946659081 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb1e8102d90 0x7fb1e819e3f0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fb1e400b4a0 tx=0x7fb1e400b960 comp rx=0 tx=0).stop 2024-09-11T23:17:05.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:05.199+0000 7fb1edde1640 1 -- 172.21.15.26:0/946659081 reap_dead start 2024-09-11T23:17:05.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:05.199+0000 7fb1ef844640 1 -- 172.21.15.26:0/946659081 shutdown_connections 2024-09-11T23:17:05.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:05.199+0000 7fb1ef844640 1 -- 172.21.15.26:0/946659081 >> 172.21.15.26:0/946659081 conn(0x7fb1e80fc7d0 msgr2=0x7fb1e8105e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:17:05.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:05.200+0000 7fb1ef844640 1 -- 172.21.15.26:0/946659081 shutdown_connections 2024-09-11T23:17:05.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:05.200+0000 7fb1ef844640 1 -- 172.21.15.26:0/946659081 wait complete. 2024-09-11T23:17:05.447 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:17:05 smithi026 ceph-mon[31172]: pgmap v300: 97 pgs: 97 active+clean; 641 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-09-11T23:17:05.490 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:17:05 smithi136 ceph-mon[35121]: pgmap v300: 97 pgs: 97 active+clean; 641 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-09-11T23:17:05.667 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:17:05 smithi076 ceph-mon[35296]: pgmap v300: 97 pgs: 97 active+clean; 641 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-09-11T23:17:05.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:05.989+0000 7f38c29bc640 1 -- 172.21.15.26:0/445394525 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38bc102790 msgr2=0x7f38bc1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:05.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:05.989+0000 7f38c29bc640 1 --2- 172.21.15.26:0/445394525 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38bc102790 0x7f38bc1067e0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f38b0009900 tx=0x7f38b002a3f0 comp rx=0 tx=0).stop 2024-09-11T23:17:05.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:05.989+0000 7f38c29bc640 1 -- 172.21.15.26:0/445394525 shutdown_connections 2024-09-11T23:17:05.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:05.989+0000 7f38c29bc640 1 --2- 172.21.15.26:0/445394525 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f38bc106f10 0x7f38bc10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:05.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:05.989+0000 7f38c29bc640 1 --2- 172.21.15.26:0/445394525 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38bc102790 0x7f38bc1067e0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:05.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:05.989+0000 7f38c29bc640 1 --2- 172.21.15.26:0/445394525 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f38bc101de0 0x7f38bc1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:05.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:05.989+0000 7f38c29bc640 1 -- 172.21.15.26:0/445394525 >> 172.21.15.26:0/445394525 conn(0x7f38bc0fd8c0 msgr2=0x7f38bc0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:17:05.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:05.989+0000 7f38c29bc640 1 -- 172.21.15.26:0/445394525 shutdown_connections 2024-09-11T23:17:05.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:05.990+0000 7f38c29bc640 1 -- 172.21.15.26:0/445394525 wait complete. 2024-09-11T23:17:05.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:05.990+0000 7f38c29bc640 1 Processor -- start 2024-09-11T23:17:05.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:05.990+0000 7f38c29bc640 1 -- start start 2024-09-11T23:17:05.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:05.991+0000 7f38c29bc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f38bc101de0 0x7f38bc1983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:05.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:05.991+0000 7f38c29bc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38bc102790 0x7f38bc1988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:05.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:05.991+0000 7f38c29bc640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f38bc106f10 0x7f38bc19cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:05.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:05.991+0000 7f38c29bc640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f38bc111f80 con 0x7f38bc102790 2024-09-11T23:17:05.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:05.991+0000 7f38c29bc640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f38bc111e00 con 0x7f38bc101de0 2024-09-11T23:17:05.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:05.991+0000 7f38c29bc640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f38bc112100 con 0x7f38bc106f10 2024-09-11T23:17:05.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:05.991+0000 7f38bb7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38bc102790 0x7f38bc1988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:05.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:05.991+0000 7f38bbfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f38bc101de0 0x7f38bc1983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:05.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:05.991+0000 7f38bbfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f38bc101de0 0x7f38bc1983a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:60710/0 (socket says 172.21.15.26:60710) 2024-09-11T23:17:05.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:05.991+0000 7f38bb7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38bc102790 0x7f38bc1988e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52318/0 (socket says 172.21.15.26:52318) 2024-09-11T23:17:05.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:05.991+0000 7f38bb7fe640 1 -- 172.21.15.26:0/1239572378 learned_addr learned my addr 172.21.15.26:0/1239572378 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:17:05.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:05.991+0000 7f38c0f59640 1 --2- 172.21.15.26:0/1239572378 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f38bc106f10 0x7f38bc19cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:05.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:05.991+0000 7f38bb7fe640 1 -- 172.21.15.26:0/1239572378 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f38bc106f10 msgr2=0x7f38bc19cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:05.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:05.991+0000 7f38bb7fe640 1 --2- 172.21.15.26:0/1239572378 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f38bc106f10 0x7f38bc19cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:05.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:05.991+0000 7f38bb7fe640 1 -- 172.21.15.26:0/1239572378 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f38bc101de0 msgr2=0x7f38bc1983a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:05.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:05.991+0000 7f38bb7fe640 1 --2- 172.21.15.26:0/1239572378 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f38bc101de0 0x7f38bc1983a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:05.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:05.991+0000 7f38bb7fe640 1 -- 172.21.15.26:0/1239572378 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38bc19d350 con 0x7f38bc102790 2024-09-11T23:17:05.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:05.992+0000 7f38bb7fe640 1 --2- 172.21.15.26:0/1239572378 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38bc102790 0x7f38bc1988e0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f38b0006010 tx=0x7f38b0003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:17:05.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:05.992+0000 7f38b97fa640 1 -- 172.21.15.26:0/1239572378 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38b003f070 con 0x7f38bc102790 2024-09-11T23:17:05.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:05.992+0000 7f38c29bc640 1 -- 172.21.15.26:0/1239572378 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38bc19d5e0 con 0x7f38bc102790 2024-09-11T23:17:05.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:05.992+0000 7f38c29bc640 1 -- 172.21.15.26:0/1239572378 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38bc1a4f20 con 0x7f38bc102790 2024-09-11T23:17:05.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:05.992+0000 7f38b97fa640 1 -- 172.21.15.26:0/1239572378 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f38b0031c90 con 0x7f38bc102790 2024-09-11T23:17:05.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:05.992+0000 7f38b97fa640 1 -- 172.21.15.26:0/1239572378 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38b0035030 con 0x7f38bc102790 2024-09-11T23:17:05.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:05.993+0000 7f38c29bc640 1 -- 172.21.15.26:0/1239572378 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f38bc062900 con 0x7f38bc102790 2024-09-11T23:17:05.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:05.994+0000 7f38b97fa640 1 -- 172.21.15.26:0/1239572378 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f38b0042070 con 0x7f38bc102790 2024-09-11T23:17:05.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:05.995+0000 7f38b97fa640 1 --2- 172.21.15.26:0/1239572378 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f38a40a70a0 0x7f38a40a9560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:05.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:05.995+0000 7f38bbfff640 1 --2- 172.21.15.26:0/1239572378 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f38a40a70a0 0x7f38a40a9560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:05.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:05.995+0000 7f38b97fa640 1 -- 172.21.15.26:0/1239572378 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f38b00eff80 con 0x7f38bc102790 2024-09-11T23:17:05.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:05.995+0000 7f38b97fa640 1 --2- 172.21.15.26:0/1239572378 >> [v2:172.21.15.136:6800/1271538911,v1:172.21.15.136:6801/1271538911] conn(0x7f38a40b0cd0 0x7f38a40b3170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:05.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:05.995+0000 7f38b97fa640 1 -- 172.21.15.26:0/1239572378 --> [v2:172.21.15.136:6800/1271538911,v1:172.21.15.136:6801/1271538911] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f38b0038e90 con 0x7f38a40b0cd0 2024-09-11T23:17:05.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:05.995+0000 7f38b97fa640 1 -- 172.21.15.26:0/1239572378 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7f38b00f03b0 con 0x7f38bc102790 2024-09-11T23:17:05.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:05.996+0000 7f38bbfff640 1 --2- 172.21.15.26:0/1239572378 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f38a40a70a0 0x7f38a40a9560 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f38ac00abe0 tx=0x7f38ac00a2d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:17:05.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:05.996+0000 7f38c0f59640 1 --2- 172.21.15.26:0/1239572378 >> [v2:172.21.15.136:6800/1271538911,v1:172.21.15.136:6801/1271538911] conn(0x7f38a40b0cd0 0x7f38a40b3170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:05.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:05.996+0000 7f38c0f59640 1 --2- 172.21.15.26:0/1239572378 >> [v2:172.21.15.136:6800/1271538911,v1:172.21.15.136:6801/1271538911] conn(0x7f38a40b0cd0 0x7f38a40b3170 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:17:05.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:05.997+0000 7f38b97fa640 1 -- 172.21.15.26:0/1239572378 <== osd.2 v2:172.21.15.136:6800/1271538911 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f38b0038e90 con 0x7f38a40b0cd0 2024-09-11T23:17:06.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:06.011+0000 7f38c29bc640 1 -- 172.21.15.26:0/1239572378 --> [v2:172.21.15.136:6800/1271538911,v1:172.21.15.136:6801/1271538911] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f38bc062d00 con 0x7f38a40b0cd0 2024-09-11T23:17:06.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:06.011+0000 7f38b97fa640 1 -- 172.21.15.26:0/1239572378 <== osd.2 v2:172.21.15.136:6800/1271538911 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f38bc062d00 con 0x7f38a40b0cd0 2024-09-11T23:17:06.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:06.012+0000 7f38c29bc640 1 -- 172.21.15.26:0/1239572378 >> [v2:172.21.15.136:6800/1271538911,v1:172.21.15.136:6801/1271538911] conn(0x7f38a40b0cd0 msgr2=0x7f38a40b3170 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:06.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:06.012+0000 7f38c29bc640 1 --2- 172.21.15.26:0/1239572378 >> [v2:172.21.15.136:6800/1271538911,v1:172.21.15.136:6801/1271538911] conn(0x7f38a40b0cd0 0x7f38a40b3170 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:06.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:06.012+0000 7f38c29bc640 1 -- 172.21.15.26:0/1239572378 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f38a40a70a0 msgr2=0x7f38a40a9560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:06.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:06.012+0000 7f38c29bc640 1 --2- 172.21.15.26:0/1239572378 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f38a40a70a0 0x7f38a40a9560 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f38ac00abe0 tx=0x7f38ac00a2d0 comp rx=0 tx=0).stop 2024-09-11T23:17:06.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:06.012+0000 7f38c29bc640 1 -- 172.21.15.26:0/1239572378 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38bc102790 msgr2=0x7f38bc1988e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:06.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:06.012+0000 7f38c29bc640 1 --2- 172.21.15.26:0/1239572378 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38bc102790 0x7f38bc1988e0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f38b0006010 tx=0x7f38b0003ba0 comp rx=0 tx=0).stop 2024-09-11T23:17:06.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:06.012+0000 7f38c0f59640 1 -- 172.21.15.26:0/1239572378 reap_dead start 2024-09-11T23:17:06.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:06.012+0000 7f38c29bc640 1 -- 172.21.15.26:0/1239572378 shutdown_connections 2024-09-11T23:17:06.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:06.012+0000 7f38c29bc640 1 -- 172.21.15.26:0/1239572378 >> 172.21.15.26:0/1239572378 conn(0x7f38bc0fd8c0 msgr2=0x7f38bc0feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:17:06.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:06.012+0000 7f38c29bc640 1 -- 172.21.15.26:0/1239572378 shutdown_connections 2024-09-11T23:17:06.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:06.012+0000 7f38c29bc640 1 -- 172.21.15.26:0/1239572378 wait complete. 2024-09-11T23:17:07.375 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:17:07 smithi026 ceph-mon[31172]: pgmap v301: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 1.4 KiB/s wr, 12 op/s 2024-09-11T23:17:07.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:17:07 smithi136 ceph-mon[35121]: pgmap v301: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 1.4 KiB/s wr, 12 op/s 2024-09-11T23:17:07.496 INFO:teuthology.orchestra.run.smithi026.stdout:300647710745 2024-09-11T23:17:07.496 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-09-11T23:17:07.553 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:17:07 smithi076 ceph-mon[35296]: pgmap v301: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 1.4 KiB/s wr, 12 op/s 2024-09-11T23:17:07.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:07.672+0000 7fba0a03e640 1 -- 172.21.15.26:0/4139769341 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fba04100e20 msgr2=0x7fba04101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:07.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:07.672+0000 7fba0a03e640 1 --2- 172.21.15.26:0/4139769341 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fba04100e20 0x7fba04101200 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fb9f4009900 tx=0x7fb9f402a480 comp rx=0 tx=0).stop 2024-09-11T23:17:07.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:07.672+0000 7fba0a03e640 1 -- 172.21.15.26:0/4139769341 shutdown_connections 2024-09-11T23:17:07.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:07.672+0000 7fba0a03e640 1 --2- 172.21.15.26:0/4139769341 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fba04102570 0x7fba041029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:07.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:07.672+0000 7fba0a03e640 1 --2- 172.21.15.26:0/4139769341 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fba04102ef0 0x7fba0410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:07.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:07.672+0000 7fba0a03e640 1 --2- 172.21.15.26:0/4139769341 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fba04100e20 0x7fba04101200 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:07.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:07.672+0000 7fba0a03e640 1 -- 172.21.15.26:0/4139769341 >> 172.21.15.26:0/4139769341 conn(0x7fba040fc7d0 msgr2=0x7fba040febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:17:07.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:07.672+0000 7fba0a03e640 1 -- 172.21.15.26:0/4139769341 shutdown_connections 2024-09-11T23:17:07.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:07.672+0000 7fba0a03e640 1 -- 172.21.15.26:0/4139769341 wait complete. 2024-09-11T23:17:07.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:07.673+0000 7fba0a03e640 1 Processor -- start 2024-09-11T23:17:07.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:07.673+0000 7fba0a03e640 1 -- start start 2024-09-11T23:17:07.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:07.673+0000 7fba0a03e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fba04100e20 0x7fba041a46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:07.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:07.673+0000 7fba0a03e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fba04102570 0x7fba041a4c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:07.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:07.673+0000 7fba0a03e640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fba04102ef0 0x7fba0419ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:07.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:07.673+0000 7fba0a03e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fba04110f50 con 0x7fba04102570 2024-09-11T23:17:07.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:07.673+0000 7fba0a03e640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fba04110dd0 con 0x7fba04100e20 2024-09-11T23:17:07.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:07.673+0000 7fba0a03e640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fba041110d0 con 0x7fba04102ef0 2024-09-11T23:17:07.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:07.673+0000 7fba02ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fba04102570 0x7fba041a4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:07.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:07.673+0000 7fba02ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fba04102570 0x7fba041a4c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52342/0 (socket says 172.21.15.26:52342) 2024-09-11T23:17:07.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:07.673+0000 7fba02ffd640 1 -- 172.21.15.26:0/827523258 learned_addr learned my addr 172.21.15.26:0/827523258 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:17:07.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:07.674+0000 7fba037fe640 1 --2- 172.21.15.26:0/827523258 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fba04100e20 0x7fba041a46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:07.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:07.674+0000 7fba03fff640 1 --2- 172.21.15.26:0/827523258 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fba04102ef0 0x7fba0419ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:07.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:07.674+0000 7fba02ffd640 1 -- 172.21.15.26:0/827523258 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fba04102ef0 msgr2=0x7fba0419ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:07.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:07.674+0000 7fba02ffd640 1 --2- 172.21.15.26:0/827523258 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fba04102ef0 0x7fba0419ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:07.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:07.674+0000 7fba02ffd640 1 -- 172.21.15.26:0/827523258 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fba04100e20 msgr2=0x7fba041a46e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:07.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:07.674+0000 7fba02ffd640 1 --2- 172.21.15.26:0/827523258 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fba04100e20 0x7fba041a46e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:07.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:07.674+0000 7fba02ffd640 1 -- 172.21.15.26:0/827523258 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba0419f340 con 0x7fba04102570 2024-09-11T23:17:07.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:07.674+0000 7fba03fff640 1 --2- 172.21.15.26:0/827523258 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fba04102ef0 0x7fba0419ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:17:07.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:07.674+0000 7fba02ffd640 1 --2- 172.21.15.26:0/827523258 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fba04102570 0x7fba041a4c20 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fb9f800d950 tx=0x7fb9f800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:17:07.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:07.674+0000 7fba00ff9640 1 -- 172.21.15.26:0/827523258 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9f8017030 con 0x7fba04102570 2024-09-11T23:17:07.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:07.674+0000 7fba0a03e640 1 -- 172.21.15.26:0/827523258 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba0419f630 con 0x7fba04102570 2024-09-11T23:17:07.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:07.674+0000 7fba00ff9640 1 -- 172.21.15.26:0/827523258 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9f8004020 con 0x7fba04102570 2024-09-11T23:17:07.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:07.674+0000 7fba00ff9640 1 -- 172.21.15.26:0/827523258 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9f80047a0 con 0x7fba04102570 2024-09-11T23:17:07.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:07.674+0000 7fba0a03e640 1 -- 172.21.15.26:0/827523258 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba041ab5b0 con 0x7fba04102570 2024-09-11T23:17:07.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:07.675+0000 7fba0a03e640 1 -- 172.21.15.26:0/827523258 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fba040fded0 con 0x7fba04102570 2024-09-11T23:17:07.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:07.677+0000 7fba00ff9640 1 -- 172.21.15.26:0/827523258 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fb9f80041d0 con 0x7fba04102570 2024-09-11T23:17:07.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:07.677+0000 7fba00ff9640 1 --2- 172.21.15.26:0/827523258 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fb9ec0a2c10 0x7fb9ec0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:07.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:07.678+0000 7fba00ff9640 1 -- 172.21.15.26:0/827523258 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fb9f80cf3f0 con 0x7fba04102570 2024-09-11T23:17:07.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:07.678+0000 7fba037fe640 1 --2- 172.21.15.26:0/827523258 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fb9ec0a2c10 0x7fb9ec0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:07.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:07.678+0000 7fba00ff9640 1 --2- 172.21.15.26:0/827523258 >> [v2:172.21.15.136:6816/543619079,v1:172.21.15.136:6817/543619079] conn(0x7fb9ec0ac840 0x7fb9ec0aece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:07.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:07.678+0000 7fba00ff9640 1 -- 172.21.15.26:0/827523258 --> [v2:172.21.15.136:6816/543619079,v1:172.21.15.136:6817/543619079] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb9ec0af3b0 con 0x7fb9ec0ac840 2024-09-11T23:17:07.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:07.678+0000 7fba00ff9640 1 -- 172.21.15.26:0/827523258 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7fb9f80cf7a0 con 0x7fba04102570 2024-09-11T23:17:07.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:07.678+0000 7fba03fff640 1 --2- 172.21.15.26:0/827523258 >> [v2:172.21.15.136:6816/543619079,v1:172.21.15.136:6817/543619079] conn(0x7fb9ec0ac840 0x7fb9ec0aece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:07.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:07.678+0000 7fba037fe640 1 --2- 172.21.15.26:0/827523258 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fb9ec0a2c10 0x7fb9ec0a50d0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fb9f4000c00 tx=0x7fb9f4005a20 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:17:07.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:07.679+0000 7fba03fff640 1 --2- 172.21.15.26:0/827523258 >> [v2:172.21.15.136:6816/543619079,v1:172.21.15.136:6817/543619079] conn(0x7fb9ec0ac840 0x7fb9ec0aece0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:17:07.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:07.679+0000 7fba00ff9640 1 -- 172.21.15.26:0/827523258 <== osd.8 v2:172.21.15.136:6816/543619079 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb9ec0af3b0 con 0x7fb9ec0ac840 2024-09-11T23:17:07.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:07.695+0000 7fba0a03e640 1 -- 172.21.15.26:0/827523258 --> [v2:172.21.15.136:6816/543619079,v1:172.21.15.136:6817/543619079] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fba041021b0 con 0x7fb9ec0ac840 2024-09-11T23:17:07.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:07.696+0000 7fba00ff9640 1 -- 172.21.15.26:0/827523258 <== osd.8 v2:172.21.15.136:6816/543619079 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fba041021b0 con 0x7fb9ec0ac840 2024-09-11T23:17:07.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:07.697+0000 7fba0a03e640 1 -- 172.21.15.26:0/827523258 >> [v2:172.21.15.136:6816/543619079,v1:172.21.15.136:6817/543619079] conn(0x7fb9ec0ac840 msgr2=0x7fb9ec0aece0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:07.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:07.697+0000 7fba0a03e640 1 --2- 172.21.15.26:0/827523258 >> [v2:172.21.15.136:6816/543619079,v1:172.21.15.136:6817/543619079] conn(0x7fb9ec0ac840 0x7fb9ec0aece0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:07.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:07.697+0000 7fba0a03e640 1 -- 172.21.15.26:0/827523258 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fb9ec0a2c10 msgr2=0x7fb9ec0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:07.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:07.697+0000 7fba0a03e640 1 --2- 172.21.15.26:0/827523258 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fb9ec0a2c10 0x7fb9ec0a50d0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fb9f4000c00 tx=0x7fb9f4005a20 comp rx=0 tx=0).stop 2024-09-11T23:17:07.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:07.697+0000 7fba0a03e640 1 -- 172.21.15.26:0/827523258 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fba04102570 msgr2=0x7fba041a4c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:07.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:07.697+0000 7fba0a03e640 1 --2- 172.21.15.26:0/827523258 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fba04102570 0x7fba041a4c20 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fb9f800d950 tx=0x7fb9f800de10 comp rx=0 tx=0).stop 2024-09-11T23:17:07.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:07.697+0000 7fba03fff640 1 -- 172.21.15.26:0/827523258 reap_dead start 2024-09-11T23:17:07.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:07.697+0000 7fba0a03e640 1 -- 172.21.15.26:0/827523258 shutdown_connections 2024-09-11T23:17:07.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:07.697+0000 7fba0a03e640 1 -- 172.21.15.26:0/827523258 >> 172.21.15.26:0/827523258 conn(0x7fba040fc7d0 msgr2=0x7fba0406a790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:17:07.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:07.697+0000 7fba0a03e640 1 -- 172.21.15.26:0/827523258 shutdown_connections 2024-09-11T23:17:07.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:07.697+0000 7fba0a03e640 1 -- 172.21.15.26:0/827523258 wait complete. 2024-09-11T23:17:08.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:08.368+0000 7fcfa4794640 1 -- 172.21.15.26:0/1568656879 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcf9c105590 msgr2=0x7fcf9c10d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:08.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:08.368+0000 7fcfa4794640 1 --2- 172.21.15.26:0/1568656879 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcf9c105590 0x7fcf9c10d110 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fcf94009ca0 tx=0x7fcf9402a8c0 comp rx=0 tx=0).stop 2024-09-11T23:17:08.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:08.368+0000 7fcfa4794640 1 -- 172.21.15.26:0/1568656879 shutdown_connections 2024-09-11T23:17:08.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:08.368+0000 7fcfa4794640 1 --2- 172.21.15.26:0/1568656879 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcf9c105590 0x7fcf9c10d110 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:08.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:08.368+0000 7fcfa4794640 1 --2- 172.21.15.26:0/1568656879 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcf9c100e10 0x7fcf9c104e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:08.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:08.368+0000 7fcfa4794640 1 --2- 172.21.15.26:0/1568656879 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcf9c100460 0x7fcf9c100840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:08.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:08.368+0000 7fcfa4794640 1 -- 172.21.15.26:0/1568656879 >> 172.21.15.26:0/1568656879 conn(0x7fcf9c0fbee0 msgr2=0x7fcf9c0fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:17:08.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:08.368+0000 7fcfa4794640 1 -- 172.21.15.26:0/1568656879 shutdown_connections 2024-09-11T23:17:08.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:08.368+0000 7fcfa4794640 1 -- 172.21.15.26:0/1568656879 wait complete. 2024-09-11T23:17:08.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:08.369+0000 7fcfa4794640 1 Processor -- start 2024-09-11T23:17:08.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:08.369+0000 7fcfa4794640 1 -- start start 2024-09-11T23:17:08.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:08.369+0000 7fcfa4794640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcf9c100460 0x7fcf9c1992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:08.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:08.369+0000 7fcfa4794640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcf9c100e10 0x7fcf9c199830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:08.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:08.369+0000 7fcfa4794640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcf9c105590 0x7fcf9c1a4370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:08.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:08.369+0000 7fcfa4794640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fcf9c110610 con 0x7fcf9c105590 2024-09-11T23:17:08.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:08.369+0000 7fcfa4794640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fcf9c110490 con 0x7fcf9c100e10 2024-09-11T23:17:08.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:08.369+0000 7fcfa4794640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fcf9c110790 con 0x7fcf9c100460 2024-09-11T23:17:08.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:08.369+0000 7fcfa2d31640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcf9c105590 0x7fcf9c1a4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:08.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:08.369+0000 7fcfa2d31640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcf9c105590 0x7fcf9c1a4370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52366/0 (socket says 172.21.15.26:52366) 2024-09-11T23:17:08.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:08.369+0000 7fcfa2d31640 1 -- 172.21.15.26:0/4158856176 learned_addr learned my addr 172.21.15.26:0/4158856176 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:17:08.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:08.369+0000 7fcfa1d2f640 1 --2- 172.21.15.26:0/4158856176 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcf9c100e10 0x7fcf9c199830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:08.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:08.369+0000 7fcfa2d31640 1 -- 172.21.15.26:0/4158856176 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcf9c100460 msgr2=0x7fcf9c1992f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:08.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:08.370+0000 7fcfa2d31640 1 --2- 172.21.15.26:0/4158856176 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcf9c100460 0x7fcf9c1992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:08.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:08.370+0000 7fcfa2d31640 1 -- 172.21.15.26:0/4158856176 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcf9c100e10 msgr2=0x7fcf9c199830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:08.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:08.370+0000 7fcfa2d31640 1 --2- 172.21.15.26:0/4158856176 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcf9c100e10 0x7fcf9c199830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:08.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:08.370+0000 7fcfa2d31640 1 -- 172.21.15.26:0/4158856176 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf9c1a48b0 con 0x7fcf9c105590 2024-09-11T23:17:08.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:08.370+0000 7fcfa1d2f640 1 --2- 172.21.15.26:0/4158856176 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcf9c100e10 0x7fcf9c199830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:17:08.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:08.370+0000 7fcfa2d31640 1 --2- 172.21.15.26:0/4158856176 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcf9c105590 0x7fcf9c1a4370 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fcf94031a60 tx=0x7fcf94031a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:17:08.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:08.370+0000 7fcf937fe640 1 -- 172.21.15.26:0/4158856176 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf94003c30 con 0x7fcf9c105590 2024-09-11T23:17:08.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:08.370+0000 7fcf937fe640 1 -- 172.21.15.26:0/4158856176 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf94003dd0 con 0x7fcf9c105590 2024-09-11T23:17:08.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:08.370+0000 7fcfa4794640 1 -- 172.21.15.26:0/4158856176 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf9c1a4a80 con 0x7fcf9c105590 2024-09-11T23:17:08.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:08.370+0000 7fcf937fe640 1 -- 172.21.15.26:0/4158856176 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf940047e0 con 0x7fcf9c105590 2024-09-11T23:17:08.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:08.370+0000 7fcfa4794640 1 -- 172.21.15.26:0/4158856176 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf9c1a4f30 con 0x7fcf9c105590 2024-09-11T23:17:08.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:08.371+0000 7fcfa4794640 1 -- 172.21.15.26:0/4158856176 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fcf9c104110 con 0x7fcf9c105590 2024-09-11T23:17:08.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:08.372+0000 7fcf937fe640 1 -- 172.21.15.26:0/4158856176 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fcf9402f790 con 0x7fcf9c105590 2024-09-11T23:17:08.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:08.373+0000 7fcf937fe640 1 --2- 172.21.15.26:0/4158856176 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fcf880a2c10 0x7fcf880a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:08.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:08.373+0000 7fcfa2530640 1 --2- 172.21.15.26:0/4158856176 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fcf880a2c10 0x7fcf880a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:08.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:08.373+0000 7fcf937fe640 1 -- 172.21.15.26:0/4158856176 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fcf940f0810 con 0x7fcf9c105590 2024-09-11T23:17:08.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:08.373+0000 7fcf937fe640 1 --2- 172.21.15.26:0/4158856176 >> [v2:172.21.15.26:6810/1740304151,v1:172.21.15.26:6811/1740304151] conn(0x7fcf880ac840 0x7fcf880aece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:08.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:08.373+0000 7fcf937fe640 1 -- 172.21.15.26:0/4158856176 --> [v2:172.21.15.26:6810/1740304151,v1:172.21.15.26:6811/1740304151] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fcf880af3b0 con 0x7fcf880ac840 2024-09-11T23:17:08.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:08.373+0000 7fcf937fe640 1 -- 172.21.15.26:0/4158856176 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7fcf940f0bc0 con 0x7fcf9c105590 2024-09-11T23:17:08.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:08.373+0000 7fcfa1d2f640 1 --2- 172.21.15.26:0/4158856176 >> [v2:172.21.15.26:6810/1740304151,v1:172.21.15.26:6811/1740304151] conn(0x7fcf880ac840 0x7fcf880aece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:08.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:08.373+0000 7fcfa2530640 1 --2- 172.21.15.26:0/4158856176 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fcf880a2c10 0x7fcf880a50d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fcf9800a990 tx=0x7fcf98005c70 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:17:08.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:08.374+0000 7fcfa1d2f640 1 --2- 172.21.15.26:0/4158856176 >> [v2:172.21.15.26:6810/1740304151,v1:172.21.15.26:6811/1740304151] conn(0x7fcf880ac840 0x7fcf880aece0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:17:08.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:08.374+0000 7fcf937fe640 1 -- 172.21.15.26:0/4158856176 <== osd.3 v2:172.21.15.26:6810/1740304151 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fcf880af3b0 con 0x7fcf880ac840 2024-09-11T23:17:08.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:08.388+0000 7fcfa4794640 1 -- 172.21.15.26:0/4158856176 --> [v2:172.21.15.26:6810/1740304151,v1:172.21.15.26:6811/1740304151] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fcf9c101940 con 0x7fcf880ac840 2024-09-11T23:17:08.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:08.388+0000 7fcf937fe640 1 -- 172.21.15.26:0/4158856176 <== osd.3 v2:172.21.15.26:6810/1740304151 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fcf9c101940 con 0x7fcf880ac840 2024-09-11T23:17:08.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:08.389+0000 7fcfa4794640 1 -- 172.21.15.26:0/4158856176 >> [v2:172.21.15.26:6810/1740304151,v1:172.21.15.26:6811/1740304151] conn(0x7fcf880ac840 msgr2=0x7fcf880aece0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:08.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:08.389+0000 7fcfa4794640 1 --2- 172.21.15.26:0/4158856176 >> [v2:172.21.15.26:6810/1740304151,v1:172.21.15.26:6811/1740304151] conn(0x7fcf880ac840 0x7fcf880aece0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:08.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:08.389+0000 7fcfa4794640 1 -- 172.21.15.26:0/4158856176 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fcf880a2c10 msgr2=0x7fcf880a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:08.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:08.389+0000 7fcfa4794640 1 --2- 172.21.15.26:0/4158856176 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fcf880a2c10 0x7fcf880a50d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fcf9800a990 tx=0x7fcf98005c70 comp rx=0 tx=0).stop 2024-09-11T23:17:08.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:08.389+0000 7fcfa4794640 1 -- 172.21.15.26:0/4158856176 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcf9c105590 msgr2=0x7fcf9c1a4370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:08.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:08.389+0000 7fcfa4794640 1 --2- 172.21.15.26:0/4158856176 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcf9c105590 0x7fcf9c1a4370 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fcf94031a60 tx=0x7fcf94031a90 comp rx=0 tx=0).stop 2024-09-11T23:17:08.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:08.389+0000 7fcfa2d31640 1 -- 172.21.15.26:0/4158856176 reap_dead start 2024-09-11T23:17:08.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:08.389+0000 7fcfa4794640 1 -- 172.21.15.26:0/4158856176 shutdown_connections 2024-09-11T23:17:08.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:08.389+0000 7fcfa4794640 1 -- 172.21.15.26:0/4158856176 >> 172.21.15.26:0/4158856176 conn(0x7fcf9c0fbee0 msgr2=0x7fcf9c0fcbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:17:08.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:08.389+0000 7fcfa4794640 1 -- 172.21.15.26:0/4158856176 shutdown_connections 2024-09-11T23:17:08.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:08.389+0000 7fcfa4794640 1 -- 172.21.15.26:0/4158856176 wait complete. 2024-09-11T23:17:09.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:06.181+0000 7fd5a64bc640 1 -- 172.21.15.26:0/2052051023 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5a0103660 msgr2=0x7fd5a010bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:09.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:06.181+0000 7fd5a64bc640 1 --2- 172.21.15.26:0/2052051023 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5a0103660 0x7fd5a010bb50 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fd594009900 tx=0x7fd59402a480 comp rx=0 tx=0).stop 2024-09-11T23:17:09.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:06.181+0000 7fd5a64bc640 1 -- 172.21.15.26:0/2052051023 shutdown_connections 2024-09-11T23:17:09.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:06.181+0000 7fd5a64bc640 1 --2- 172.21.15.26:0/2052051023 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd5a010c170 0x7fd5a010e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:09.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:06.181+0000 7fd5a64bc640 1 --2- 172.21.15.26:0/2052051023 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5a0103660 0x7fd5a010bb50 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:09.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:06.181+0000 7fd5a64bc640 1 --2- 172.21.15.26:0/2052051023 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd5a0102d40 0x7fd5a0103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:09.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:06.181+0000 7fd5a64bc640 1 -- 172.21.15.26:0/2052051023 >> 172.21.15.26:0/2052051023 conn(0x7fd5a00fc7a0 msgr2=0x7fd5a00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:17:09.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:06.181+0000 7fd5a64bc640 1 -- 172.21.15.26:0/2052051023 shutdown_connections 2024-09-11T23:17:09.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:06.181+0000 7fd5a64bc640 1 -- 172.21.15.26:0/2052051023 wait complete. 2024-09-11T23:17:09.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:06.182+0000 7fd5a64bc640 1 Processor -- start 2024-09-11T23:17:09.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:06.182+0000 7fd5a64bc640 1 -- start start 2024-09-11T23:17:09.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:06.183+0000 7fd5a64bc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5a0102d40 0x7fd5a01984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:09.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:06.183+0000 7fd5a64bc640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd5a0103660 0x7fd5a01989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:09.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:06.183+0000 7fd5a64bc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd5a010c170 0x7fd5a019cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:09.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:06.183+0000 7fd5a64bc640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd5a0111fd0 con 0x7fd5a0102d40 2024-09-11T23:17:09.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:06.183+0000 7fd5a64bc640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fd5a0111e50 con 0x7fd5a010c170 2024-09-11T23:17:09.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:06.183+0000 7fd5a64bc640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd5a0112150 con 0x7fd5a0103660 2024-09-11T23:17:09.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:06.183+0000 7fd59ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5a0102d40 0x7fd5a01984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:09.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:06.183+0000 7fd59ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5a0102d40 0x7fd5a01984b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52332/0 (socket says 172.21.15.26:52332) 2024-09-11T23:17:09.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:06.183+0000 7fd59ffff640 1 -- 172.21.15.26:0/4044467399 learned_addr learned my addr 172.21.15.26:0/4044467399 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:17:09.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:06.183+0000 7fd5a4a59640 1 --2- 172.21.15.26:0/4044467399 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd5a010c170 0x7fd5a019cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:09.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:06.183+0000 7fd59f7fe640 1 --2- 172.21.15.26:0/4044467399 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd5a0103660 0x7fd5a01989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:09.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:06.183+0000 7fd59ffff640 1 -- 172.21.15.26:0/4044467399 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd5a0103660 msgr2=0x7fd5a01989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:09.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:06.183+0000 7fd59ffff640 1 --2- 172.21.15.26:0/4044467399 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd5a0103660 0x7fd5a01989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:09.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:06.183+0000 7fd59ffff640 1 -- 172.21.15.26:0/4044467399 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd5a010c170 msgr2=0x7fd5a019cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:09.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:06.183+0000 7fd59ffff640 1 --2- 172.21.15.26:0/4044467399 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd5a010c170 0x7fd5a019cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:09.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:06.183+0000 7fd59ffff640 1 -- 172.21.15.26:0/4044467399 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5a019d460 con 0x7fd5a0102d40 2024-09-11T23:17:09.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:06.183+0000 7fd59f7fe640 1 --2- 172.21.15.26:0/4044467399 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd5a0103660 0x7fd5a01989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:17:09.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:06.183+0000 7fd5a4a59640 1 --2- 172.21.15.26:0/4044467399 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd5a010c170 0x7fd5a019cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:17:09.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:06.183+0000 7fd59ffff640 1 --2- 172.21.15.26:0/4044467399 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5a0102d40 0x7fd5a01984b0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fd59000d770 tx=0x7fd59000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:17:09.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:06.183+0000 7fd59d7fa640 1 -- 172.21.15.26:0/4044467399 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd590017070 con 0x7fd5a0102d40 2024-09-11T23:17:09.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:06.183+0000 7fd5a64bc640 1 -- 172.21.15.26:0/4044467399 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5a019d750 con 0x7fd5a0102d40 2024-09-11T23:17:09.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:06.183+0000 7fd59d7fa640 1 -- 172.21.15.26:0/4044467399 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd590005500 con 0x7fd5a0102d40 2024-09-11T23:17:09.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:06.184+0000 7fd59d7fa640 1 -- 172.21.15.26:0/4044467399 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd590004e00 con 0x7fd5a0102d40 2024-09-11T23:17:09.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:06.184+0000 7fd5a64bc640 1 -- 172.21.15.26:0/4044467399 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5a01a5030 con 0x7fd5a0102d40 2024-09-11T23:17:09.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:06.184+0000 7fd5a64bc640 1 -- 172.21.15.26:0/4044467399 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd5a0062980 con 0x7fd5a0102d40 2024-09-11T23:17:09.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:06.186+0000 7fd59d7fa640 1 -- 172.21.15.26:0/4044467399 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fd590004fa0 con 0x7fd5a0102d40 2024-09-11T23:17:09.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:06.186+0000 7fd59d7fa640 1 --2- 172.21.15.26:0/4044467399 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fd5880a2c10 0x7fd5880a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:09.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:06.186+0000 7fd59f7fe640 1 --2- 172.21.15.26:0/4044467399 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fd5880a2c10 0x7fd5880a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:09.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:06.186+0000 7fd59d7fa640 1 -- 172.21.15.26:0/4044467399 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fd5900cf0d0 con 0x7fd5a0102d40 2024-09-11T23:17:09.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:06.187+0000 7fd59d7fa640 1 --2- 172.21.15.26:0/4044467399 >> [v2:172.21.15.136:6824/1960689685,v1:172.21.15.136:6825/1960689685] conn(0x7fd5880ac840 0x7fd5880aece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:09.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:06.187+0000 7fd59d7fa640 1 -- 172.21.15.26:0/4044467399 --> [v2:172.21.15.136:6824/1960689685,v1:172.21.15.136:6825/1960689685] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd5880af3b0 con 0x7fd5880ac840 2024-09-11T23:17:09.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:06.187+0000 7fd59d7fa640 1 -- 172.21.15.26:0/4044467399 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7fd5900cf500 con 0x7fd5a0102d40 2024-09-11T23:17:09.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:06.187+0000 7fd59f7fe640 1 --2- 172.21.15.26:0/4044467399 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fd5880a2c10 0x7fd5880a50d0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fd594000c00 tx=0x7fd594032c70 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:17:09.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:06.187+0000 7fd5a4a59640 1 --2- 172.21.15.26:0/4044467399 >> [v2:172.21.15.136:6824/1960689685,v1:172.21.15.136:6825/1960689685] conn(0x7fd5880ac840 0x7fd5880aece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:09.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:06.187+0000 7fd5a4a59640 1 --2- 172.21.15.26:0/4044467399 >> [v2:172.21.15.136:6824/1960689685,v1:172.21.15.136:6825/1960689685] conn(0x7fd5880ac840 0x7fd5880aece0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:17:09.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:06.188+0000 7fd59d7fa640 1 -- 172.21.15.26:0/4044467399 <== osd.11 v2:172.21.15.136:6824/1960689685 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd5880af3b0 con 0x7fd5880ac840 2024-09-11T23:17:09.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:06.202+0000 7fd5a64bc640 1 -- 172.21.15.26:0/4044467399 --> [v2:172.21.15.136:6824/1960689685,v1:172.21.15.136:6825/1960689685] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd5a0062e00 con 0x7fd5880ac840 2024-09-11T23:17:09.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:06.203+0000 7fd59d7fa640 1 -- 172.21.15.26:0/4044467399 <== osd.11 v2:172.21.15.136:6824/1960689685 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fd5a0062e00 con 0x7fd5880ac840 2024-09-11T23:17:09.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:06.203+0000 7fd5a64bc640 1 -- 172.21.15.26:0/4044467399 >> [v2:172.21.15.136:6824/1960689685,v1:172.21.15.136:6825/1960689685] conn(0x7fd5880ac840 msgr2=0x7fd5880aece0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:09.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:06.203+0000 7fd5a64bc640 1 --2- 172.21.15.26:0/4044467399 >> [v2:172.21.15.136:6824/1960689685,v1:172.21.15.136:6825/1960689685] conn(0x7fd5880ac840 0x7fd5880aece0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:09.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:06.203+0000 7fd5a64bc640 1 -- 172.21.15.26:0/4044467399 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fd5880a2c10 msgr2=0x7fd5880a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:09.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:06.203+0000 7fd5a64bc640 1 --2- 172.21.15.26:0/4044467399 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fd5880a2c10 0x7fd5880a50d0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fd594000c00 tx=0x7fd594032c70 comp rx=0 tx=0).stop 2024-09-11T23:17:09.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:06.203+0000 7fd5a64bc640 1 -- 172.21.15.26:0/4044467399 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5a0102d40 msgr2=0x7fd5a01984b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:09.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:06.203+0000 7fd5a64bc640 1 --2- 172.21.15.26:0/4044467399 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5a0102d40 0x7fd5a01984b0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fd59000d770 tx=0x7fd59000dc30 comp rx=0 tx=0).stop 2024-09-11T23:17:09.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:06.203+0000 7fd5a4a59640 1 -- 172.21.15.26:0/4044467399 reap_dead start 2024-09-11T23:17:09.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:06.203+0000 7fd5a64bc640 1 -- 172.21.15.26:0/4044467399 shutdown_connections 2024-09-11T23:17:09.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:06.203+0000 7fd5a64bc640 1 -- 172.21.15.26:0/4044467399 >> 172.21.15.26:0/4044467399 conn(0x7fd5a00fc7a0 msgr2=0x7fd5a00fd400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:17:09.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:06.203+0000 7fd5a64bc640 1 -- 172.21.15.26:0/4044467399 shutdown_connections 2024-09-11T23:17:09.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:06.203+0000 7fd5a64bc640 1 -- 172.21.15.26:0/4044467399 wait complete. 2024-09-11T23:17:09.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:17:09 smithi136 ceph-mon[35121]: pgmap v302: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 597 B/s wr, 8 op/s 2024-09-11T23:17:09.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:17:09 smithi026 ceph-mon[31172]: pgmap v302: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 597 B/s wr, 8 op/s 2024-09-11T23:17:09.553 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:17:09 smithi076 ceph-mon[35296]: pgmap v302: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 597 B/s wr, 8 op/s 2024-09-11T23:17:10.160 INFO:teuthology.orchestra.run.smithi026.stdout:330712481814 2024-09-11T23:17:10.161 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-09-11T23:17:10.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.312+0000 7f307c659640 1 -- 172.21.15.26:0/876657685 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3074106f10 msgr2=0x7f307410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:10.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.312+0000 7f307c659640 1 --2- 172.21.15.26:0/876657685 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3074106f10 0x7f307410ea90 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f306c009ca0 tx=0x7f306c02a8c0 comp rx=0 tx=0).stop 2024-09-11T23:17:10.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.312+0000 7f307c659640 1 -- 172.21.15.26:0/876657685 shutdown_connections 2024-09-11T23:17:10.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.312+0000 7f307c659640 1 --2- 172.21.15.26:0/876657685 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3074102790 0x7f30741067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:10.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.312+0000 7f307c659640 1 --2- 172.21.15.26:0/876657685 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3074106f10 0x7f307410ea90 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:10.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.312+0000 7f307c659640 1 --2- 172.21.15.26:0/876657685 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3074101de0 0x7f30741021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:10.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.312+0000 7f307c659640 1 -- 172.21.15.26:0/876657685 >> 172.21.15.26:0/876657685 conn(0x7f30740fd880 msgr2=0x7f30740ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:17:10.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.313+0000 7f307c659640 1 -- 172.21.15.26:0/876657685 shutdown_connections 2024-09-11T23:17:10.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.313+0000 7f307c659640 1 -- 172.21.15.26:0/876657685 wait complete. 2024-09-11T23:17:10.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.313+0000 7f307c659640 1 Processor -- start 2024-09-11T23:17:10.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.313+0000 7f307c659640 1 -- start start 2024-09-11T23:17:10.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.313+0000 7f307c659640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3074101de0 0x7f30740744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:10.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.313+0000 7f307c659640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3074102790 0x7f3074074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:10.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.313+0000 7f307c659640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3074106f10 0x7f3074074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:10.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.313+0000 7f307c659640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f3074111f70 con 0x7f3074102790 2024-09-11T23:17:10.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.313+0000 7f307c659640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f3074111df0 con 0x7f3074106f10 2024-09-11T23:17:10.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.313+0000 7f307c659640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f30741120f0 con 0x7f3074101de0 2024-09-11T23:17:10.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.313+0000 7f3079bf4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3074102790 0x7f3074074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:10.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.313+0000 7f3079bf4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3074102790 0x7f3074074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52386/0 (socket says 172.21.15.26:52386) 2024-09-11T23:17:10.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.314+0000 7f3079bf4640 1 -- 172.21.15.26:0/145220532 learned_addr learned my addr 172.21.15.26:0/145220532 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:17:10.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.314+0000 7f307a3f5640 1 --2- 172.21.15.26:0/145220532 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3074101de0 0x7f30740744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:10.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.314+0000 7f307abf6640 1 --2- 172.21.15.26:0/145220532 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3074106f10 0x7f3074074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:10.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.314+0000 7f3079bf4640 1 -- 172.21.15.26:0/145220532 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3074101de0 msgr2=0x7f30740744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:10.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.314+0000 7f3079bf4640 1 --2- 172.21.15.26:0/145220532 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3074101de0 0x7f30740744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:10.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.314+0000 7f3079bf4640 1 -- 172.21.15.26:0/145220532 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3074106f10 msgr2=0x7f3074074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:10.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.314+0000 7f3079bf4640 1 --2- 172.21.15.26:0/145220532 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3074106f10 0x7f3074074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:10.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.314+0000 7f3079bf4640 1 -- 172.21.15.26:0/145220532 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30741a6870 con 0x7f3074102790 2024-09-11T23:17:10.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.314+0000 7f307abf6640 1 --2- 172.21.15.26:0/145220532 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3074106f10 0x7f3074074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:17:10.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.314+0000 7f307a3f5640 1 --2- 172.21.15.26:0/145220532 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3074101de0 0x7f30740744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:17:10.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.314+0000 7f3079bf4640 1 --2- 172.21.15.26:0/145220532 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3074102790 0x7f3074074a30 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f306400e9f0 tx=0x7f306400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:17:10.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.314+0000 7f306b7fe640 1 -- 172.21.15.26:0/145220532 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f306400cd80 con 0x7f3074102790 2024-09-11T23:17:10.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.314+0000 7f306b7fe640 1 -- 172.21.15.26:0/145220532 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3064010910 con 0x7f3074102790 2024-09-11T23:17:10.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.314+0000 7f307c659640 1 -- 172.21.15.26:0/145220532 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30741a6b60 con 0x7f3074102790 2024-09-11T23:17:10.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.314+0000 7f306b7fe640 1 -- 172.21.15.26:0/145220532 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3064018990 con 0x7f3074102790 2024-09-11T23:17:10.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.314+0000 7f307c659640 1 -- 172.21.15.26:0/145220532 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30741a70f0 con 0x7f3074102790 2024-09-11T23:17:10.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.315+0000 7f307c659640 1 -- 172.21.15.26:0/145220532 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f30740628c0 con 0x7f3074102790 2024-09-11T23:17:10.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.317+0000 7f306b7fe640 1 -- 172.21.15.26:0/145220532 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f3064010ac0 con 0x7f3074102790 2024-09-11T23:17:10.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.318+0000 7f306b7fe640 1 --2- 172.21.15.26:0/145220532 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f30600a2c90 0x7f30600a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:10.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.318+0000 7f307a3f5640 1 --2- 172.21.15.26:0/145220532 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f30600a2c90 0x7f30600a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:10.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.318+0000 7f306b7fe640 1 -- 172.21.15.26:0/145220532 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f30640cfba0 con 0x7f3074102790 2024-09-11T23:17:10.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.318+0000 7f306b7fe640 1 --2- 172.21.15.26:0/145220532 >> [v2:172.21.15.26:6826/2201297064,v1:172.21.15.26:6827/2201297064] conn(0x7f30600ac8c0 0x7f30600aed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:10.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.318+0000 7f306b7fe640 1 -- 172.21.15.26:0/145220532 --> [v2:172.21.15.26:6826/2201297064,v1:172.21.15.26:6827/2201297064] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f30600af430 con 0x7f30600ac8c0 2024-09-11T23:17:10.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.318+0000 7f306b7fe640 1 -- 172.21.15.26:0/145220532 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7f30640cffd0 con 0x7f3074102790 2024-09-11T23:17:10.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.318+0000 7f307abf6640 1 --2- 172.21.15.26:0/145220532 >> [v2:172.21.15.26:6826/2201297064,v1:172.21.15.26:6827/2201297064] conn(0x7f30600ac8c0 0x7f30600aed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:10.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.319+0000 7f307a3f5640 1 --2- 172.21.15.26:0/145220532 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f30600a2c90 0x7f30600a5150 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f30700058c0 tx=0x7f3070005850 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:17:10.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.319+0000 7f307abf6640 1 --2- 172.21.15.26:0/145220532 >> [v2:172.21.15.26:6826/2201297064,v1:172.21.15.26:6827/2201297064] conn(0x7f30600ac8c0 0x7f30600aed60 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:17:10.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.319+0000 7f306b7fe640 1 -- 172.21.15.26:0/145220532 <== osd.9 v2:172.21.15.26:6826/2201297064 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f30600af430 con 0x7f30600ac8c0 2024-09-11T23:17:10.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.333+0000 7f307c659640 1 -- 172.21.15.26:0/145220532 --> [v2:172.21.15.26:6826/2201297064,v1:172.21.15.26:6827/2201297064] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f3074062cc0 con 0x7f30600ac8c0 2024-09-11T23:17:10.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.334+0000 7f306b7fe640 1 -- 172.21.15.26:0/145220532 <== osd.9 v2:172.21.15.26:6826/2201297064 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f3074062cc0 con 0x7f30600ac8c0 2024-09-11T23:17:10.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.334+0000 7f307c659640 1 -- 172.21.15.26:0/145220532 >> [v2:172.21.15.26:6826/2201297064,v1:172.21.15.26:6827/2201297064] conn(0x7f30600ac8c0 msgr2=0x7f30600aed60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:10.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.334+0000 7f307c659640 1 --2- 172.21.15.26:0/145220532 >> [v2:172.21.15.26:6826/2201297064,v1:172.21.15.26:6827/2201297064] conn(0x7f30600ac8c0 0x7f30600aed60 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:10.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.335+0000 7f307c659640 1 -- 172.21.15.26:0/145220532 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f30600a2c90 msgr2=0x7f30600a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:10.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.335+0000 7f307c659640 1 --2- 172.21.15.26:0/145220532 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f30600a2c90 0x7f30600a5150 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f30700058c0 tx=0x7f3070005850 comp rx=0 tx=0).stop 2024-09-11T23:17:10.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.335+0000 7f307c659640 1 -- 172.21.15.26:0/145220532 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3074102790 msgr2=0x7f3074074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:10.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.335+0000 7f307c659640 1 --2- 172.21.15.26:0/145220532 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3074102790 0x7f3074074a30 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f306400e9f0 tx=0x7f306400eeb0 comp rx=0 tx=0).stop 2024-09-11T23:17:10.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.335+0000 7f307abf6640 1 -- 172.21.15.26:0/145220532 reap_dead start 2024-09-11T23:17:10.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.335+0000 7f307c659640 1 -- 172.21.15.26:0/145220532 shutdown_connections 2024-09-11T23:17:10.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.335+0000 7f307c659640 1 -- 172.21.15.26:0/145220532 >> 172.21.15.26:0/145220532 conn(0x7f30740fd880 msgr2=0x7f30740feb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:17:10.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.336+0000 7f307c659640 1 -- 172.21.15.26:0/145220532 shutdown_connections 2024-09-11T23:17:10.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.337+0000 7f307c659640 1 -- 172.21.15.26:0/145220532 wait complete. 2024-09-11T23:17:10.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.893+0000 7fb5f7d09640 1 -- 172.21.15.26:0/3655337551 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb5f0100a80 msgr2=0x7fb5f0100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:10.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.893+0000 7fb5f7d09640 1 --2- 172.21.15.26:0/3655337551 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb5f0100a80 0x7fb5f0100ee0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fb5e0009930 tx=0x7fb5e002a420 comp rx=0 tx=0).stop 2024-09-11T23:17:10.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.893+0000 7fb5f7d09640 1 -- 172.21.15.26:0/3655337551 shutdown_connections 2024-09-11T23:17:10.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.893+0000 7fb5f7d09640 1 --2- 172.21.15.26:0/3655337551 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb5f0101420 0x7fb5f010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:10.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.893+0000 7fb5f7d09640 1 --2- 172.21.15.26:0/3655337551 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb5f0100a80 0x7fb5f0100ee0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:10.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.893+0000 7fb5f7d09640 1 --2- 172.21.15.26:0/3655337551 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb5f0106a80 0x7fb5f0106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:10.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.893+0000 7fb5f7d09640 1 -- 172.21.15.26:0/3655337551 >> 172.21.15.26:0/3655337551 conn(0x7fb5f00fc7a0 msgr2=0x7fb5f00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:17:10.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.893+0000 7fb5f7d09640 1 -- 172.21.15.26:0/3655337551 shutdown_connections 2024-09-11T23:17:10.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.894+0000 7fb5f7d09640 1 -- 172.21.15.26:0/3655337551 wait complete. 2024-09-11T23:17:10.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.894+0000 7fb5f7d09640 1 Processor -- start 2024-09-11T23:17:10.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.894+0000 7fb5f7d09640 1 -- start start 2024-09-11T23:17:10.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.894+0000 7fb5f7d09640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb5f0100a80 0x7fb5f0199320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:10.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.894+0000 7fb5f7d09640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb5f0101420 0x7fb5f0199860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:10.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.894+0000 7fb5f7d09640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb5f0106a80 0x7fb5f01a43a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:10.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.894+0000 7fb5f7d09640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb5f0110ed0 con 0x7fb5f0101420 2024-09-11T23:17:10.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.894+0000 7fb5f7d09640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fb5f0110d50 con 0x7fb5f0106a80 2024-09-11T23:17:10.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.894+0000 7fb5f7d09640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb5f0111050 con 0x7fb5f0100a80 2024-09-11T23:17:10.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.894+0000 7fb5f52a4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb5f0101420 0x7fb5f0199860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:10.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.894+0000 7fb5f52a4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb5f0101420 0x7fb5f0199860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52734/0 (socket says 172.21.15.26:52734) 2024-09-11T23:17:10.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.894+0000 7fb5f52a4640 1 -- 172.21.15.26:0/3528960217 learned_addr learned my addr 172.21.15.26:0/3528960217 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:17:10.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.895+0000 7fb5f5aa5640 1 --2- 172.21.15.26:0/3528960217 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb5f0100a80 0x7fb5f0199320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:10.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.895+0000 7fb5f52a4640 1 -- 172.21.15.26:0/3528960217 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb5f0100a80 msgr2=0x7fb5f0199320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:10.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.895+0000 7fb5f62a6640 1 --2- 172.21.15.26:0/3528960217 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb5f0106a80 0x7fb5f01a43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:10.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.895+0000 7fb5f52a4640 1 --2- 172.21.15.26:0/3528960217 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb5f0100a80 0x7fb5f0199320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:10.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.895+0000 7fb5f52a4640 1 -- 172.21.15.26:0/3528960217 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb5f0106a80 msgr2=0x7fb5f01a43a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:10.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.895+0000 7fb5f52a4640 1 --2- 172.21.15.26:0/3528960217 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb5f0106a80 0x7fb5f01a43a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:10.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.895+0000 7fb5f52a4640 1 -- 172.21.15.26:0/3528960217 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5f01a48e0 con 0x7fb5f0101420 2024-09-11T23:17:10.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.895+0000 7fb5f5aa5640 1 --2- 172.21.15.26:0/3528960217 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb5f0100a80 0x7fb5f0199320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:17:10.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.895+0000 7fb5f62a6640 1 --2- 172.21.15.26:0/3528960217 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb5f0106a80 0x7fb5f01a43a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:17:10.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.895+0000 7fb5f52a4640 1 --2- 172.21.15.26:0/3528960217 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb5f0101420 0x7fb5f0199860 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fb5e002a940 tx=0x7fb5e0003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:17:10.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.895+0000 7fb5e6ffd640 1 -- 172.21.15.26:0/3528960217 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5e003f070 con 0x7fb5f0101420 2024-09-11T23:17:10.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.895+0000 7fb5e6ffd640 1 -- 172.21.15.26:0/3528960217 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb5e0031c90 con 0x7fb5f0101420 2024-09-11T23:17:10.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.895+0000 7fb5f7d09640 1 -- 172.21.15.26:0/3528960217 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5f01a4ab0 con 0x7fb5f0101420 2024-09-11T23:17:10.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.895+0000 7fb5e6ffd640 1 -- 172.21.15.26:0/3528960217 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5e0035030 con 0x7fb5f0101420 2024-09-11T23:17:10.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.895+0000 7fb5f7d09640 1 -- 172.21.15.26:0/3528960217 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5f01a4f60 con 0x7fb5f0101420 2024-09-11T23:17:10.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.896+0000 7fb5f7d09640 1 -- 172.21.15.26:0/3528960217 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb5f0062900 con 0x7fb5f0101420 2024-09-11T23:17:10.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.897+0000 7fb5e6ffd640 1 -- 172.21.15.26:0/3528960217 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fb5e0042070 con 0x7fb5f0101420 2024-09-11T23:17:10.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.898+0000 7fb5e6ffd640 1 --2- 172.21.15.26:0/3528960217 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fb5dc0a2c90 0x7fb5dc0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:10.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.898+0000 7fb5e6ffd640 1 -- 172.21.15.26:0/3528960217 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fb5e00f00c0 con 0x7fb5f0101420 2024-09-11T23:17:10.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.898+0000 7fb5f5aa5640 1 --2- 172.21.15.26:0/3528960217 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fb5dc0a2c90 0x7fb5dc0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:10.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.898+0000 7fb5e6ffd640 1 --2- 172.21.15.26:0/3528960217 >> [v2:172.21.15.76:6816/1409789455,v1:172.21.15.76:6817/1409789455] conn(0x7fb5dc0ac8c0 0x7fb5dc0aed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:10.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.898+0000 7fb5e6ffd640 1 -- 172.21.15.26:0/3528960217 --> [v2:172.21.15.76:6816/1409789455,v1:172.21.15.76:6817/1409789455] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb5e000b0b0 con 0x7fb5dc0ac8c0 2024-09-11T23:17:10.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.898+0000 7fb5e6ffd640 1 -- 172.21.15.26:0/3528960217 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7fb5e00f04f0 con 0x7fb5f0101420 2024-09-11T23:17:10.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.899+0000 7fb5f5aa5640 1 --2- 172.21.15.26:0/3528960217 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fb5dc0a2c90 0x7fb5dc0a5150 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fb5ec009ba0 tx=0x7fb5ec005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:17:10.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.899+0000 7fb5f62a6640 1 --2- 172.21.15.26:0/3528960217 >> [v2:172.21.15.76:6816/1409789455,v1:172.21.15.76:6817/1409789455] conn(0x7fb5dc0ac8c0 0x7fb5dc0aed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:10.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.899+0000 7fb5f62a6640 1 --2- 172.21.15.26:0/3528960217 >> [v2:172.21.15.76:6816/1409789455,v1:172.21.15.76:6817/1409789455] conn(0x7fb5dc0ac8c0 0x7fb5dc0aed60 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:17:10.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.900+0000 7fb5e6ffd640 1 -- 172.21.15.26:0/3528960217 <== osd.7 v2:172.21.15.76:6816/1409789455 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb5e000b0b0 con 0x7fb5dc0ac8c0 2024-09-11T23:17:10.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.914+0000 7fb5f7d09640 1 -- 172.21.15.26:0/3528960217 --> [v2:172.21.15.76:6816/1409789455,v1:172.21.15.76:6817/1409789455] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb5f0062d80 con 0x7fb5dc0ac8c0 2024-09-11T23:17:10.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.914+0000 7fb5e6ffd640 1 -- 172.21.15.26:0/3528960217 <== osd.7 v2:172.21.15.76:6816/1409789455 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fb5f0062d80 con 0x7fb5dc0ac8c0 2024-09-11T23:17:10.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.915+0000 7fb5f7d09640 1 -- 172.21.15.26:0/3528960217 >> [v2:172.21.15.76:6816/1409789455,v1:172.21.15.76:6817/1409789455] conn(0x7fb5dc0ac8c0 msgr2=0x7fb5dc0aed60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:10.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.915+0000 7fb5f7d09640 1 --2- 172.21.15.26:0/3528960217 >> [v2:172.21.15.76:6816/1409789455,v1:172.21.15.76:6817/1409789455] conn(0x7fb5dc0ac8c0 0x7fb5dc0aed60 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:10.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.915+0000 7fb5f7d09640 1 -- 172.21.15.26:0/3528960217 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fb5dc0a2c90 msgr2=0x7fb5dc0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:10.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.915+0000 7fb5f7d09640 1 --2- 172.21.15.26:0/3528960217 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fb5dc0a2c90 0x7fb5dc0a5150 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fb5ec009ba0 tx=0x7fb5ec005f90 comp rx=0 tx=0).stop 2024-09-11T23:17:10.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.915+0000 7fb5f7d09640 1 -- 172.21.15.26:0/3528960217 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb5f0101420 msgr2=0x7fb5f0199860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:10.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.915+0000 7fb5f7d09640 1 --2- 172.21.15.26:0/3528960217 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb5f0101420 0x7fb5f0199860 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fb5e002a940 tx=0x7fb5e0003b90 comp rx=0 tx=0).stop 2024-09-11T23:17:10.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.915+0000 7fb5f62a6640 1 -- 172.21.15.26:0/3528960217 reap_dead start 2024-09-11T23:17:10.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.915+0000 7fb5f7d09640 1 -- 172.21.15.26:0/3528960217 shutdown_connections 2024-09-11T23:17:10.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.915+0000 7fb5f7d09640 1 -- 172.21.15.26:0/3528960217 >> 172.21.15.26:0/3528960217 conn(0x7fb5f00fc7a0 msgr2=0x7fb5f00fcfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:17:10.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.915+0000 7fb5f7d09640 1 -- 172.21.15.26:0/3528960217 shutdown_connections 2024-09-11T23:17:10.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.915+0000 7fb5f7d09640 1 -- 172.21.15.26:0/3528960217 wait complete. 2024-09-11T23:17:11.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:11.362+0000 7f1a5099f640 1 -- 172.21.15.26:0/1469093307 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1a4c102380 msgr2=0x7f1a4c10e9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:11.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:11.362+0000 7f1a5099f640 1 --2- 172.21.15.26:0/1469093307 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1a4c102380 0x7f1a4c10e9c0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f1a44009cd0 tx=0x7f1a4402a8e0 comp rx=0 tx=0).stop 2024-09-11T23:17:11.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:11.362+0000 7f1a5099f640 1 -- 172.21.15.26:0/1469093307 shutdown_connections 2024-09-11T23:17:11.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:11.362+0000 7f1a5099f640 1 --2- 172.21.15.26:0/1469093307 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1a4c1019e0 0x7f1a4c101e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:11.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:11.362+0000 7f1a5099f640 1 --2- 172.21.15.26:0/1469093307 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1a4c102380 0x7f1a4c10e9c0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:11.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:11.362+0000 7f1a5099f640 1 --2- 172.21.15.26:0/1469093307 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1a4c107af0 0x7f1a4c107ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:11.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:11.362+0000 7f1a5099f640 1 -- 172.21.15.26:0/1469093307 >> 172.21.15.26:0/1469093307 conn(0x7f1a4c0fd700 msgr2=0x7f1a4c0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:17:11.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:11.362+0000 7f1a5099f640 1 -- 172.21.15.26:0/1469093307 shutdown_connections 2024-09-11T23:17:11.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:11.362+0000 7f1a5099f640 1 -- 172.21.15.26:0/1469093307 wait complete. 2024-09-11T23:17:11.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:11.362+0000 7f1a5099f640 1 Processor -- start 2024-09-11T23:17:11.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:11.363+0000 7f1a5099f640 1 -- start start 2024-09-11T23:17:11.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:11.363+0000 7f1a5099f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1a4c1019e0 0x7f1a4c1982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:11.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:11.363+0000 7f1a5099f640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1a4c102380 0x7f1a4c198830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:11.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:11.363+0000 7f1a5099f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1a4c107af0 0x7f1a4c19cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:11.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:11.363+0000 7f1a5099f640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f1a4c111ef0 con 0x7f1a4c107af0 2024-09-11T23:17:11.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:11.363+0000 7f1a5099f640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f1a4c111d70 con 0x7f1a4c1019e0 2024-09-11T23:17:11.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:11.363+0000 7f1a5099f640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f1a4c112070 con 0x7f1a4c102380 2024-09-11T23:17:11.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:11.363+0000 7f1a4bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1a4c107af0 0x7f1a4c19cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:11.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:11.363+0000 7f1a4bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1a4c107af0 0x7f1a4c19cbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52754/0 (socket says 172.21.15.26:52754) 2024-09-11T23:17:11.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:11.363+0000 7f1a4bfff640 1 -- 172.21.15.26:0/2129933022 learned_addr learned my addr 172.21.15.26:0/2129933022 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:17:11.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:11.363+0000 7f1a4b7fe640 1 --2- 172.21.15.26:0/2129933022 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1a4c1019e0 0x7f1a4c1982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:11.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:11.363+0000 7f1a4affd640 1 --2- 172.21.15.26:0/2129933022 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1a4c102380 0x7f1a4c198830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:11.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:11.363+0000 7f1a4bfff640 1 -- 172.21.15.26:0/2129933022 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1a4c102380 msgr2=0x7f1a4c198830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:11.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:11.363+0000 7f1a4bfff640 1 --2- 172.21.15.26:0/2129933022 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1a4c102380 0x7f1a4c198830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:11.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:11.363+0000 7f1a4bfff640 1 -- 172.21.15.26:0/2129933022 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1a4c1019e0 msgr2=0x7f1a4c1982f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:11.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:11.363+0000 7f1a4bfff640 1 --2- 172.21.15.26:0/2129933022 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1a4c1019e0 0x7f1a4c1982f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:11.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:11.363+0000 7f1a4bfff640 1 -- 172.21.15.26:0/2129933022 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a4c19d2a0 con 0x7f1a4c107af0 2024-09-11T23:17:11.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:11.364+0000 7f1a4bfff640 1 --2- 172.21.15.26:0/2129933022 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1a4c107af0 0x7f1a4c19cbc0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f1a44006010 tx=0x7f1a44003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:17:11.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:11.364+0000 7f1a48ff9640 1 -- 172.21.15.26:0/2129933022 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a44004450 con 0x7f1a4c107af0 2024-09-11T23:17:11.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:11.364+0000 7f1a48ff9640 1 -- 172.21.15.26:0/2129933022 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a44003e10 con 0x7f1a4c107af0 2024-09-11T23:17:11.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:11.364+0000 7f1a5099f640 1 -- 172.21.15.26:0/2129933022 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a4c19d530 con 0x7f1a4c107af0 2024-09-11T23:17:11.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:11.364+0000 7f1a48ff9640 1 -- 172.21.15.26:0/2129933022 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a44046480 con 0x7f1a4c107af0 2024-09-11T23:17:11.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:11.364+0000 7f1a5099f640 1 -- 172.21.15.26:0/2129933022 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a4c1a4e70 con 0x7f1a4c107af0 2024-09-11T23:17:11.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:11.365+0000 7f1a5099f640 1 -- 172.21.15.26:0/2129933022 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1a4c101e40 con 0x7f1a4c107af0 2024-09-11T23:17:11.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:11.366+0000 7f1a48ff9640 1 -- 172.21.15.26:0/2129933022 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f1a440045f0 con 0x7f1a4c107af0 2024-09-11T23:17:11.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:11.367+0000 7f1a48ff9640 1 --2- 172.21.15.26:0/2129933022 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f1a340a2c10 0x7f1a340a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:11.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:11.367+0000 7f1a4b7fe640 1 --2- 172.21.15.26:0/2129933022 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f1a340a2c10 0x7f1a340a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:11.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:11.367+0000 7f1a48ff9640 1 -- 172.21.15.26:0/2129933022 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f1a4402e030 con 0x7f1a4c107af0 2024-09-11T23:17:11.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:11.367+0000 7f1a48ff9640 1 --2- 172.21.15.26:0/2129933022 >> [v2:172.21.15.76:6800/333383087,v1:172.21.15.76:6801/333383087] conn(0x7f1a340ac840 0x7f1a340aece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:11.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:11.367+0000 7f1a48ff9640 1 -- 172.21.15.26:0/2129933022 --> [v2:172.21.15.76:6800/333383087,v1:172.21.15.76:6801/333383087] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1a440020b0 con 0x7f1a340ac840 2024-09-11T23:17:11.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:11.367+0000 7f1a48ff9640 1 -- 172.21.15.26:0/2129933022 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7f1a440f0f40 con 0x7f1a4c107af0 2024-09-11T23:17:11.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:11.367+0000 7f1a4b7fe640 1 --2- 172.21.15.26:0/2129933022 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f1a340a2c10 0x7f1a340a50d0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f1a3c00ab60 tx=0x7f1a3c00a2d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:17:11.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:11.367+0000 7f1a4affd640 1 --2- 172.21.15.26:0/2129933022 >> [v2:172.21.15.76:6800/333383087,v1:172.21.15.76:6801/333383087] conn(0x7f1a340ac840 0x7f1a340aece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:11.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:11.368+0000 7f1a4affd640 1 --2- 172.21.15.26:0/2129933022 >> [v2:172.21.15.76:6800/333383087,v1:172.21.15.76:6801/333383087] conn(0x7f1a340ac840 0x7f1a340aece0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:17:11.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:11.369+0000 7f1a48ff9640 1 -- 172.21.15.26:0/2129933022 <== osd.1 v2:172.21.15.76:6800/333383087 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1a440020b0 con 0x7f1a340ac840 2024-09-11T23:17:11.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:11.382+0000 7f1a5099f640 1 -- 172.21.15.26:0/2129933022 --> [v2:172.21.15.76:6800/333383087,v1:172.21.15.76:6801/333383087] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f1a4c0ff730 con 0x7f1a340ac840 2024-09-11T23:17:11.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:11.383+0000 7f1a48ff9640 1 -- 172.21.15.26:0/2129933022 <== osd.1 v2:172.21.15.76:6800/333383087 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f1a4c0ff730 con 0x7f1a340ac840 2024-09-11T23:17:11.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:11.383+0000 7f1a5099f640 1 -- 172.21.15.26:0/2129933022 >> [v2:172.21.15.76:6800/333383087,v1:172.21.15.76:6801/333383087] conn(0x7f1a340ac840 msgr2=0x7f1a340aece0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:11.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:11.383+0000 7f1a5099f640 1 --2- 172.21.15.26:0/2129933022 >> [v2:172.21.15.76:6800/333383087,v1:172.21.15.76:6801/333383087] conn(0x7f1a340ac840 0x7f1a340aece0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:11.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:11.383+0000 7f1a5099f640 1 -- 172.21.15.26:0/2129933022 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f1a340a2c10 msgr2=0x7f1a340a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:11.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:11.383+0000 7f1a5099f640 1 --2- 172.21.15.26:0/2129933022 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f1a340a2c10 0x7f1a340a50d0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f1a3c00ab60 tx=0x7f1a3c00a2d0 comp rx=0 tx=0).stop 2024-09-11T23:17:11.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:11.383+0000 7f1a5099f640 1 -- 172.21.15.26:0/2129933022 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1a4c107af0 msgr2=0x7f1a4c19cbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:11.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:11.383+0000 7f1a5099f640 1 --2- 172.21.15.26:0/2129933022 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1a4c107af0 0x7f1a4c19cbc0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f1a44006010 tx=0x7f1a44003c20 comp rx=0 tx=0).stop 2024-09-11T23:17:11.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:11.383+0000 7f1a4bfff640 1 -- 172.21.15.26:0/2129933022 reap_dead start 2024-09-11T23:17:11.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:11.384+0000 7f1a5099f640 1 -- 172.21.15.26:0/2129933022 shutdown_connections 2024-09-11T23:17:11.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:11.384+0000 7f1a5099f640 1 -- 172.21.15.26:0/2129933022 >> 172.21.15.26:0/2129933022 conn(0x7f1a4c0fd700 msgr2=0x7f1a4c0fed50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:17:11.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:11.384+0000 7f1a5099f640 1 -- 172.21.15.26:0/2129933022 shutdown_connections 2024-09-11T23:17:11.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:11.384+0000 7f1a5099f640 1 -- 172.21.15.26:0/2129933022 wait complete. 2024-09-11T23:17:11.473 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:17:11 smithi026 ceph-mon[31172]: pgmap v303: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 597 B/s wr, 10 op/s 2024-09-11T23:17:11.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:17:11 smithi136 ceph-mon[35121]: pgmap v303: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 597 B/s wr, 10 op/s 2024-09-11T23:17:11.553 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:17:11 smithi076 ceph-mon[35296]: pgmap v303: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 597 B/s wr, 10 op/s 2024-09-11T23:17:11.958 INFO:teuthology.orchestra.run.smithi026.stdout:77309411395 2024-09-11T23:17:11.958 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-09-11T23:17:12.111 INFO:teuthology.orchestra.run.smithi026.stdout:107374182462 2024-09-11T23:17:12.111 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-09-11T23:17:12.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.538+0000 7f36481e9640 1 -- 172.21.15.26:0/3815895345 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3640101de0 msgr2=0x7f36401021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:12.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.538+0000 7f36481e9640 1 --2- 172.21.15.26:0/3815895345 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3640101de0 0x7f36401021c0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f363c009900 tx=0x7f363c02a410 comp rx=0 tx=0).stop 2024-09-11T23:17:12.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.538+0000 7f36481e9640 1 -- 172.21.15.26:0/3815895345 shutdown_connections 2024-09-11T23:17:12.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.538+0000 7f36481e9640 1 --2- 172.21.15.26:0/3815895345 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3640106f10 0x7f364010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:12.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.538+0000 7f36481e9640 1 --2- 172.21.15.26:0/3815895345 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3640102790 0x7f36401067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:12.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.538+0000 7f36481e9640 1 --2- 172.21.15.26:0/3815895345 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3640101de0 0x7f36401021c0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:12.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.538+0000 7f36481e9640 1 -- 172.21.15.26:0/3815895345 >> 172.21.15.26:0/3815895345 conn(0x7f36400fd8c0 msgr2=0x7f36400ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:17:12.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.538+0000 7f36481e9640 1 -- 172.21.15.26:0/3815895345 shutdown_connections 2024-09-11T23:17:12.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.538+0000 7f36481e9640 1 -- 172.21.15.26:0/3815895345 wait complete. 2024-09-11T23:17:12.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.539+0000 7f36481e9640 1 Processor -- start 2024-09-11T23:17:12.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.539+0000 7f36481e9640 1 -- start start 2024-09-11T23:17:12.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.539+0000 7f36481e9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3640101de0 0x7f3640198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:12.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.539+0000 7f36481e9640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3640102790 0x7f3640198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:12.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.539+0000 7f36481e9640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3640106f10 0x7f364019cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:12.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.539+0000 7f36481e9640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f3640111f70 con 0x7f3640101de0 2024-09-11T23:17:12.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.539+0000 7f36481e9640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f3640111df0 con 0x7f3640102790 2024-09-11T23:17:12.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.539+0000 7f36481e9640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f36401120f0 con 0x7f3640106f10 2024-09-11T23:17:12.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.539+0000 7f3645f85640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3640101de0 0x7f3640198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:12.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.539+0000 7f3645f85640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3640101de0 0x7f3640198450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52404/0 (socket says 172.21.15.26:52404) 2024-09-11T23:17:12.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.539+0000 7f3645f85640 1 -- 172.21.15.26:0/616698560 learned_addr learned my addr 172.21.15.26:0/616698560 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:17:12.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.539+0000 7f3645784640 1 --2- 172.21.15.26:0/616698560 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3640102790 0x7f3640198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:12.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.539+0000 7f3645f85640 1 -- 172.21.15.26:0/616698560 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3640106f10 msgr2=0x7f364019cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:12.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.539+0000 7f3645f85640 1 --2- 172.21.15.26:0/616698560 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3640106f10 0x7f364019cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:12.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.539+0000 7f3645f85640 1 -- 172.21.15.26:0/616698560 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3640102790 msgr2=0x7f3640198990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:12.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.539+0000 7f3645f85640 1 --2- 172.21.15.26:0/616698560 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3640102790 0x7f3640198990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:12.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.539+0000 7f3645f85640 1 -- 172.21.15.26:0/616698560 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f364019d400 con 0x7f3640101de0 2024-09-11T23:17:12.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.540+0000 7f3645f85640 1 --2- 172.21.15.26:0/616698560 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3640101de0 0x7f3640198450 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f363c006010 tx=0x7f363c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:17:12.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.540+0000 7f3636ffd640 1 -- 172.21.15.26:0/616698560 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f363c03f030 con 0x7f3640101de0 2024-09-11T23:17:12.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.540+0000 7f3636ffd640 1 -- 172.21.15.26:0/616698560 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f363c003ce0 con 0x7f3640101de0 2024-09-11T23:17:12.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.540+0000 7f36481e9640 1 -- 172.21.15.26:0/616698560 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f364019d690 con 0x7f3640101de0 2024-09-11T23:17:12.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.540+0000 7f3636ffd640 1 -- 172.21.15.26:0/616698560 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f363c035070 con 0x7f3640101de0 2024-09-11T23:17:12.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.540+0000 7f36481e9640 1 -- 172.21.15.26:0/616698560 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36401a4fd0 con 0x7f3640101de0 2024-09-11T23:17:12.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.541+0000 7f36481e9640 1 -- 172.21.15.26:0/616698560 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3640062900 con 0x7f3640101de0 2024-09-11T23:17:12.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.542+0000 7f3636ffd640 1 -- 172.21.15.26:0/616698560 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f363c042030 con 0x7f3640101de0 2024-09-11T23:17:12.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.543+0000 7f3636ffd640 1 --2- 172.21.15.26:0/616698560 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f362c0a2c90 0x7f362c0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:12.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.543+0000 7f3645784640 1 --2- 172.21.15.26:0/616698560 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f362c0a2c90 0x7f362c0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:12.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.543+0000 7f3636ffd640 1 -- 172.21.15.26:0/616698560 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f363c0efe20 con 0x7f3640101de0 2024-09-11T23:17:12.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.543+0000 7f3636ffd640 1 --2- 172.21.15.26:0/616698560 >> [v2:172.21.15.136:6808/1749229277,v1:172.21.15.136:6809/1749229277] conn(0x7f362c0ac8c0 0x7f362c0aed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:12.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.543+0000 7f3636ffd640 1 -- 172.21.15.26:0/616698560 --> [v2:172.21.15.136:6808/1749229277,v1:172.21.15.136:6809/1749229277] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f363c001d30 con 0x7f362c0ac8c0 2024-09-11T23:17:12.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.543+0000 7f3636ffd640 1 -- 172.21.15.26:0/616698560 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7f363c0f0250 con 0x7f3640101de0 2024-09-11T23:17:12.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.543+0000 7f3645784640 1 --2- 172.21.15.26:0/616698560 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f362c0a2c90 0x7f362c0a5150 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f3630005fd0 tx=0x7f3630005850 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:17:12.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.543+0000 7f3646786640 1 --2- 172.21.15.26:0/616698560 >> [v2:172.21.15.136:6808/1749229277,v1:172.21.15.136:6809/1749229277] conn(0x7f362c0ac8c0 0x7f362c0aed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:12.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.544+0000 7f3646786640 1 --2- 172.21.15.26:0/616698560 >> [v2:172.21.15.136:6808/1749229277,v1:172.21.15.136:6809/1749229277] conn(0x7f362c0ac8c0 0x7f362c0aed60 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:17:12.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.545+0000 7f3636ffd640 1 -- 172.21.15.26:0/616698560 <== osd.5 v2:172.21.15.136:6808/1749229277 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f363c001d30 con 0x7f362c0ac8c0 2024-09-11T23:17:12.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.558+0000 7f36481e9640 1 -- 172.21.15.26:0/616698560 --> [v2:172.21.15.136:6808/1749229277,v1:172.21.15.136:6809/1749229277] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f3640062d00 con 0x7f362c0ac8c0 2024-09-11T23:17:12.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.559+0000 7f3636ffd640 1 -- 172.21.15.26:0/616698560 <== osd.5 v2:172.21.15.136:6808/1749229277 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f3640062d00 con 0x7f362c0ac8c0 2024-09-11T23:17:12.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.559+0000 7f36481e9640 1 -- 172.21.15.26:0/616698560 >> [v2:172.21.15.136:6808/1749229277,v1:172.21.15.136:6809/1749229277] conn(0x7f362c0ac8c0 msgr2=0x7f362c0aed60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:12.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.559+0000 7f36481e9640 1 --2- 172.21.15.26:0/616698560 >> [v2:172.21.15.136:6808/1749229277,v1:172.21.15.136:6809/1749229277] conn(0x7f362c0ac8c0 0x7f362c0aed60 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:12.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.559+0000 7f36481e9640 1 -- 172.21.15.26:0/616698560 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f362c0a2c90 msgr2=0x7f362c0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:12.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.559+0000 7f36481e9640 1 --2- 172.21.15.26:0/616698560 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f362c0a2c90 0x7f362c0a5150 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f3630005fd0 tx=0x7f3630005850 comp rx=0 tx=0).stop 2024-09-11T23:17:12.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.559+0000 7f36481e9640 1 -- 172.21.15.26:0/616698560 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3640101de0 msgr2=0x7f3640198450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:12.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.560+0000 7f36481e9640 1 --2- 172.21.15.26:0/616698560 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3640101de0 0x7f3640198450 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f363c006010 tx=0x7f363c0042f0 comp rx=0 tx=0).stop 2024-09-11T23:17:12.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.560+0000 7f3646786640 1 -- 172.21.15.26:0/616698560 reap_dead start 2024-09-11T23:17:12.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.560+0000 7f36481e9640 1 -- 172.21.15.26:0/616698560 shutdown_connections 2024-09-11T23:17:12.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.560+0000 7f36481e9640 1 -- 172.21.15.26:0/616698560 >> 172.21.15.26:0/616698560 conn(0x7f36400fd8c0 msgr2=0x7f36400febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:17:12.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.560+0000 7f36481e9640 1 -- 172.21.15.26:0/616698560 shutdown_connections 2024-09-11T23:17:12.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:10.560+0000 7f36481e9640 1 -- 172.21.15.26:0/616698560 wait complete. 2024-09-11T23:17:12.874 INFO:teuthology.orchestra.run.smithi026.stdout:158913790005 2024-09-11T23:17:12.874 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-09-11T23:17:13.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:13.030+0000 7f5abb865640 1 -- 172.21.15.26:0/4107614903 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5ab4102570 msgr2=0x7f5ab41029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:13.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:13.030+0000 7f5abb865640 1 --2- 172.21.15.26:0/4107614903 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5ab4102570 0x7f5ab41029b0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f5aa8009900 tx=0x7f5aa802a480 comp rx=0 tx=0).stop 2024-09-11T23:17:13.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:13.030+0000 7f5abb865640 1 -- 172.21.15.26:0/4107614903 shutdown_connections 2024-09-11T23:17:13.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:13.030+0000 7f5abb865640 1 --2- 172.21.15.26:0/4107614903 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5ab4102ef0 0x7f5ab410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:13.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:13.030+0000 7f5abb865640 1 --2- 172.21.15.26:0/4107614903 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5ab4102570 0x7f5ab41029b0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:13.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:13.030+0000 7f5abb865640 1 --2- 172.21.15.26:0/4107614903 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5ab4100e20 0x7f5ab4101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:13.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:13.030+0000 7f5abb865640 1 -- 172.21.15.26:0/4107614903 >> 172.21.15.26:0/4107614903 conn(0x7f5ab40fc7d0 msgr2=0x7f5ab40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:17:13.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:13.030+0000 7f5abb865640 1 -- 172.21.15.26:0/4107614903 shutdown_connections 2024-09-11T23:17:13.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:13.030+0000 7f5abb865640 1 -- 172.21.15.26:0/4107614903 wait complete. 2024-09-11T23:17:13.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:13.031+0000 7f5abb865640 1 Processor -- start 2024-09-11T23:17:13.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:13.031+0000 7f5abb865640 1 -- start start 2024-09-11T23:17:13.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:13.031+0000 7f5abb865640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5ab4100e20 0x7f5ab40fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:13.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:13.031+0000 7f5abb865640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5ab4102570 0x7f5ab40fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:13.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:13.031+0000 7f5abb865640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5ab4102ef0 0x7f5ab40fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:13.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:13.031+0000 7f5abb865640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f5ab4110f80 con 0x7f5ab4102570 2024-09-11T23:17:13.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:13.031+0000 7f5abb865640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f5ab4110e00 con 0x7f5ab4102ef0 2024-09-11T23:17:13.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:13.031+0000 7f5abb865640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f5ab4111100 con 0x7f5ab4100e20 2024-09-11T23:17:13.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:13.031+0000 7f5ab8e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5ab4102570 0x7f5ab40fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:13.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:13.031+0000 7f5ab8e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5ab4102570 0x7f5ab40fd1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52764/0 (socket says 172.21.15.26:52764) 2024-09-11T23:17:13.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:13.031+0000 7f5ab8e00640 1 -- 172.21.15.26:0/1503431067 learned_addr learned my addr 172.21.15.26:0/1503431067 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:17:13.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:13.031+0000 7f5ab9601640 1 --2- 172.21.15.26:0/1503431067 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5ab4100e20 0x7f5ab40fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:13.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:13.031+0000 7f5ab9e02640 1 --2- 172.21.15.26:0/1503431067 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5ab4102ef0 0x7f5ab40fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:13.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:13.031+0000 7f5ab8e00640 1 -- 172.21.15.26:0/1503431067 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5ab4100e20 msgr2=0x7f5ab40fcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:13.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:13.031+0000 7f5ab8e00640 1 --2- 172.21.15.26:0/1503431067 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5ab4100e20 0x7f5ab40fcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:13.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:13.031+0000 7f5ab8e00640 1 -- 172.21.15.26:0/1503431067 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5ab4102ef0 msgr2=0x7f5ab40fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:13.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:13.031+0000 7f5ab8e00640 1 --2- 172.21.15.26:0/1503431067 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5ab4102ef0 0x7f5ab40fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:13.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:13.032+0000 7f5ab8e00640 1 -- 172.21.15.26:0/1503431067 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ab41a6a70 con 0x7f5ab4102570 2024-09-11T23:17:13.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:13.032+0000 7f5ab8e00640 1 --2- 172.21.15.26:0/1503431067 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5ab4102570 0x7f5ab40fd1c0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f5aa8006010 tx=0x7f5aa8003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:17:13.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:13.032+0000 7f5ab27fc640 1 -- 172.21.15.26:0/1503431067 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aa803f070 con 0x7f5ab4102570 2024-09-11T23:17:13.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:13.032+0000 7f5abb865640 1 -- 172.21.15.26:0/1503431067 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ab41a6d00 con 0x7f5ab4102570 2024-09-11T23:17:13.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:13.032+0000 7f5ab27fc640 1 -- 172.21.15.26:0/1503431067 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5aa8031c30 con 0x7f5ab4102570 2024-09-11T23:17:13.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:13.032+0000 7f5ab27fc640 1 -- 172.21.15.26:0/1503431067 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aa8035030 con 0x7f5ab4102570 2024-09-11T23:17:13.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:13.032+0000 7f5abb865640 1 -- 172.21.15.26:0/1503431067 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ab41a7180 con 0x7f5ab4102570 2024-09-11T23:17:13.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:13.033+0000 7f5abb865640 1 -- 172.21.15.26:0/1503431067 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5ab40628c0 con 0x7f5ab4102570 2024-09-11T23:17:13.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:13.034+0000 7f5ab27fc640 1 -- 172.21.15.26:0/1503431067 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f5aa8042070 con 0x7f5ab4102570 2024-09-11T23:17:13.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:13.035+0000 7f5ab27fc640 1 --2- 172.21.15.26:0/1503431067 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f5a9c0a2c10 0x7f5a9c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:13.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:13.035+0000 7f5ab27fc640 1 -- 172.21.15.26:0/1503431067 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f5aa80efd80 con 0x7f5ab4102570 2024-09-11T23:17:13.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:13.035+0000 7f5ab9601640 1 --2- 172.21.15.26:0/1503431067 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f5a9c0a2c10 0x7f5a9c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:13.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:13.035+0000 7f5ab27fc640 1 --2- 172.21.15.26:0/1503431067 >> [v2:172.21.15.26:6802/2014857905,v1:172.21.15.26:6803/2014857905] conn(0x7f5a9c0ac840 0x7f5a9c0aece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:13.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:13.035+0000 7f5ab27fc640 1 -- 172.21.15.26:0/1503431067 --> [v2:172.21.15.26:6802/2014857905,v1:172.21.15.26:6803/2014857905] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5aa8038e90 con 0x7f5a9c0ac840 2024-09-11T23:17:13.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:13.035+0000 7f5ab27fc640 1 -- 172.21.15.26:0/1503431067 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7f5aa80f01b0 con 0x7f5ab4102570 2024-09-11T23:17:13.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:13.035+0000 7f5ab9e02640 1 --2- 172.21.15.26:0/1503431067 >> [v2:172.21.15.26:6802/2014857905,v1:172.21.15.26:6803/2014857905] conn(0x7f5a9c0ac840 0x7f5a9c0aece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:13.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:13.035+0000 7f5ab9601640 1 --2- 172.21.15.26:0/1503431067 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f5a9c0a2c10 0x7f5a9c0a50d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f5aa4005fd0 tx=0x7f5aa4009680 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:17:13.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:13.036+0000 7f5ab9e02640 1 --2- 172.21.15.26:0/1503431067 >> [v2:172.21.15.26:6802/2014857905,v1:172.21.15.26:6803/2014857905] conn(0x7f5a9c0ac840 0x7f5a9c0aece0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:17:13.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:13.036+0000 7f5ab27fc640 1 -- 172.21.15.26:0/1503431067 <== osd.0 v2:172.21.15.26:6802/2014857905 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5aa8038e90 con 0x7f5a9c0ac840 2024-09-11T23:17:13.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:13.050+0000 7f5abb865640 1 -- 172.21.15.26:0/1503431067 --> [v2:172.21.15.26:6802/2014857905,v1:172.21.15.26:6803/2014857905] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5ab4062cf0 con 0x7f5a9c0ac840 2024-09-11T23:17:13.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:13.050+0000 7f5ab27fc640 1 -- 172.21.15.26:0/1503431067 <== osd.0 v2:172.21.15.26:6802/2014857905 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f5ab4062cf0 con 0x7f5a9c0ac840 2024-09-11T23:17:13.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:13.051+0000 7f5abb865640 1 -- 172.21.15.26:0/1503431067 >> [v2:172.21.15.26:6802/2014857905,v1:172.21.15.26:6803/2014857905] conn(0x7f5a9c0ac840 msgr2=0x7f5a9c0aece0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:13.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:13.051+0000 7f5abb865640 1 --2- 172.21.15.26:0/1503431067 >> [v2:172.21.15.26:6802/2014857905,v1:172.21.15.26:6803/2014857905] conn(0x7f5a9c0ac840 0x7f5a9c0aece0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:13.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:13.051+0000 7f5abb865640 1 -- 172.21.15.26:0/1503431067 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f5a9c0a2c10 msgr2=0x7f5a9c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:13.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:13.051+0000 7f5abb865640 1 --2- 172.21.15.26:0/1503431067 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f5a9c0a2c10 0x7f5a9c0a50d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f5aa4005fd0 tx=0x7f5aa4009680 comp rx=0 tx=0).stop 2024-09-11T23:17:13.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:13.051+0000 7f5abb865640 1 -- 172.21.15.26:0/1503431067 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5ab4102570 msgr2=0x7f5ab40fd1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:13.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:13.051+0000 7f5abb865640 1 --2- 172.21.15.26:0/1503431067 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5ab4102570 0x7f5ab40fd1c0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f5aa8006010 tx=0x7f5aa8003c10 comp rx=0 tx=0).stop 2024-09-11T23:17:13.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:13.051+0000 7f5ab9e02640 1 -- 172.21.15.26:0/1503431067 reap_dead start 2024-09-11T23:17:13.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:13.051+0000 7f5abb865640 1 -- 172.21.15.26:0/1503431067 shutdown_connections 2024-09-11T23:17:13.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:13.051+0000 7f5abb865640 1 -- 172.21.15.26:0/1503431067 >> 172.21.15.26:0/1503431067 conn(0x7f5ab40fc7d0 msgr2=0x7f5ab406a270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:17:13.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:13.051+0000 7f5abb865640 1 -- 172.21.15.26:0/1503431067 shutdown_connections 2024-09-11T23:17:13.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:13.051+0000 7f5abb865640 1 -- 172.21.15.26:0/1503431067 wait complete. 2024-09-11T23:17:13.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:17:13 smithi136 ceph-mon[35121]: pgmap v304: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 597 B/s wr, 11 op/s 2024-09-11T23:17:13.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:17:13 smithi026 ceph-mon[31172]: pgmap v304: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 597 B/s wr, 11 op/s 2024-09-11T23:17:13.553 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:17:13 smithi076 ceph-mon[35296]: pgmap v304: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 597 B/s wr, 11 op/s 2024-09-11T23:17:14.183 INFO:teuthology.orchestra.run.smithi026.stdout:55834574921 2024-09-11T23:17:14.184 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-09-11T23:17:14.609 INFO:teuthology.orchestra.run.smithi026.stdout:244813135909 2024-09-11T23:17:14.610 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-09-11T23:17:15.048 INFO:teuthology.orchestra.run.smithi026.stdout:214748364842 2024-09-11T23:17:15.048 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-09-11T23:17:15.446 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:17:15 smithi026 ceph-mon[31172]: pgmap v305: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-11T23:17:15.476 INFO:teuthology.orchestra.run.smithi026.stdout:141733920824 2024-09-11T23:17:15.476 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-09-11T23:17:15.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:17:15 smithi136 ceph-mon[35121]: pgmap v305: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-11T23:17:15.553 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:17:15 smithi076 ceph-mon[35296]: pgmap v305: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-11T23:17:15.901 INFO:teuthology.orchestra.run.smithi026.stdout:34359738446 2024-09-11T23:17:15.902 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-09-11T23:17:16.392 INFO:teuthology.orchestra.run.smithi026.stdout:279172874273 2024-09-11T23:17:16.393 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-09-11T23:17:16.416 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:17:16.497 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:17:16.501 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:17:16.502 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:17:16.503 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:17:16.505 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:17:16.510 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:17:16.510 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:17:16.514 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:17:17.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:17.148+0000 7ff1a7591640 1 -- 172.21.15.26:0/386865009 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1a0102570 msgr2=0x7ff1a01029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:17.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:17.148+0000 7ff1a7591640 1 --2- 172.21.15.26:0/386865009 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1a0102570 0x7ff1a01029b0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7ff190009900 tx=0x7ff19002a480 comp rx=0 tx=0).stop 2024-09-11T23:17:17.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:17.148+0000 7ff1a7591640 1 -- 172.21.15.26:0/386865009 shutdown_connections 2024-09-11T23:17:17.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:17.148+0000 7ff1a7591640 1 --2- 172.21.15.26:0/386865009 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff1a0102ef0 0x7ff1a010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:17.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:17.148+0000 7ff1a7591640 1 --2- 172.21.15.26:0/386865009 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1a0102570 0x7ff1a01029b0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:17.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:17.148+0000 7ff1a7591640 1 --2- 172.21.15.26:0/386865009 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff1a0100e20 0x7ff1a0101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:17.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:17.148+0000 7ff1a7591640 1 -- 172.21.15.26:0/386865009 >> 172.21.15.26:0/386865009 conn(0x7ff1a00fc7d0 msgr2=0x7ff1a00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:17:17.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:17.148+0000 7ff1a7591640 1 -- 172.21.15.26:0/386865009 shutdown_connections 2024-09-11T23:17:17.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:17.148+0000 7ff1a7591640 1 -- 172.21.15.26:0/386865009 wait complete. 2024-09-11T23:17:17.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:17.149+0000 7ff1a7591640 1 Processor -- start 2024-09-11T23:17:17.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:17.149+0000 7ff1a7591640 1 -- start start 2024-09-11T23:17:17.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:17.149+0000 7ff1a7591640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff1a0100e20 0x7ff1a0198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:17.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:17.149+0000 7ff1a7591640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1a0102570 0x7ff1a0198ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:17.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:17.149+0000 7ff1a7591640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff1a0102ef0 0x7ff1a019ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:17.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:17.149+0000 7ff1a7591640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff1a0110f80 con 0x7ff1a0102570 2024-09-11T23:17:17.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:17.149+0000 7ff1a7591640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7ff1a0110e00 con 0x7ff1a0100e20 2024-09-11T23:17:17.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:17.149+0000 7ff1a7591640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ff1a0111100 con 0x7ff1a0102ef0 2024-09-11T23:17:17.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:17.149+0000 7ff1a4b2c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1a0102570 0x7ff1a0198ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:17.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:17.149+0000 7ff1a4b2c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1a0102570 0x7ff1a0198ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52774/0 (socket says 172.21.15.26:52774) 2024-09-11T23:17:17.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:17.149+0000 7ff1a4b2c640 1 -- 172.21.15.26:0/4184501267 learned_addr learned my addr 172.21.15.26:0/4184501267 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:17:17.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:17.149+0000 7ff1a532d640 1 --2- 172.21.15.26:0/4184501267 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff1a0100e20 0x7ff1a0198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:17.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:17.149+0000 7ff1a5b2e640 1 --2- 172.21.15.26:0/4184501267 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff1a0102ef0 0x7ff1a019ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:17.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:17.150+0000 7ff1a4b2c640 1 -- 172.21.15.26:0/4184501267 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff1a0102ef0 msgr2=0x7ff1a019ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:17.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:17.150+0000 7ff1a4b2c640 1 --2- 172.21.15.26:0/4184501267 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff1a0102ef0 0x7ff1a019ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:17.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:17.150+0000 7ff1a4b2c640 1 -- 172.21.15.26:0/4184501267 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff1a0100e20 msgr2=0x7ff1a0198590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:17.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:17.150+0000 7ff1a4b2c640 1 --2- 172.21.15.26:0/4184501267 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff1a0100e20 0x7ff1a0198590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:17.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:17.150+0000 7ff1a4b2c640 1 -- 172.21.15.26:0/4184501267 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1a019d540 con 0x7ff1a0102570 2024-09-11T23:17:17.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:17.150+0000 7ff1a5b2e640 1 --2- 172.21.15.26:0/4184501267 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff1a0102ef0 0x7ff1a019ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:17:17.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:17.150+0000 7ff1a4b2c640 1 --2- 172.21.15.26:0/4184501267 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1a0102570 0x7ff1a0198ad0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7ff190006010 tx=0x7ff190003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:17:17.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:17.150+0000 7ff1967fc640 1 -- 172.21.15.26:0/4184501267 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff19003f070 con 0x7ff1a0102570 2024-09-11T23:17:17.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:17.150+0000 7ff1967fc640 1 -- 172.21.15.26:0/4184501267 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff190031c30 con 0x7ff1a0102570 2024-09-11T23:17:17.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:17.150+0000 7ff1a7591640 1 -- 172.21.15.26:0/4184501267 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1a019d7d0 con 0x7ff1a0102570 2024-09-11T23:17:17.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:17.150+0000 7ff1967fc640 1 -- 172.21.15.26:0/4184501267 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff190035030 con 0x7ff1a0102570 2024-09-11T23:17:17.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:17.150+0000 7ff1a7591640 1 -- 172.21.15.26:0/4184501267 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1a01a5110 con 0x7ff1a0102570 2024-09-11T23:17:17.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:17.151+0000 7ff1a7591640 1 -- 172.21.15.26:0/4184501267 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1a0101200 con 0x7ff1a0102570 2024-09-11T23:17:17.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:17.152+0000 7ff1967fc640 1 -- 172.21.15.26:0/4184501267 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7ff190042070 con 0x7ff1a0102570 2024-09-11T23:17:17.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:17.153+0000 7ff1967fc640 1 --2- 172.21.15.26:0/4184501267 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7ff18c0a2c90 0x7ff18c0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:17.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:17.153+0000 7ff1a532d640 1 --2- 172.21.15.26:0/4184501267 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7ff18c0a2c90 0x7ff18c0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:17.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:17.153+0000 7ff1967fc640 1 -- 172.21.15.26:0/4184501267 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7ff1900ef6c0 con 0x7ff1a0102570 2024-09-11T23:17:17.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:17.153+0000 7ff1a532d640 1 --2- 172.21.15.26:0/4184501267 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7ff18c0a2c90 0x7ff18c0a5150 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ff19c005fd0 tx=0x7ff19c00b390 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:17:17.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:17.157+0000 7ff1967fc640 1 -- 172.21.15.26:0/4184501267 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff190031de0 con 0x7ff1a0102570 2024-09-11T23:17:17.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:17.337+0000 7ff1a7591640 1 -- 172.21.15.26:0/4184501267 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7ff1a010a0a0 con 0x7ff1a0102570 2024-09-11T23:17:17.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:17.337+0000 7ff1967fc640 1 -- 172.21.15.26:0/4184501267 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7ff1900bc480 con 0x7ff1a0102570 2024-09-11T23:17:17.338 INFO:teuthology.orchestra.run.smithi026.stdout:193273528367 2024-09-11T23:17:17.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:17.340+0000 7ff1a7591640 1 -- 172.21.15.26:0/4184501267 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7ff18c0a2c90 msgr2=0x7ff18c0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:17.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:17.340+0000 7ff1a7591640 1 --2- 172.21.15.26:0/4184501267 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7ff18c0a2c90 0x7ff18c0a5150 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ff19c005fd0 tx=0x7ff19c00b390 comp rx=0 tx=0).stop 2024-09-11T23:17:17.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:17.340+0000 7ff1a7591640 1 -- 172.21.15.26:0/4184501267 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1a0102570 msgr2=0x7ff1a0198ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:17.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:17.340+0000 7ff1a7591640 1 --2- 172.21.15.26:0/4184501267 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1a0102570 0x7ff1a0198ad0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7ff190006010 tx=0x7ff190003c10 comp rx=0 tx=0).stop 2024-09-11T23:17:17.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:17.340+0000 7ff1a7591640 1 -- 172.21.15.26:0/4184501267 shutdown_connections 2024-09-11T23:17:17.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:17.340+0000 7ff1a7591640 1 --2- 172.21.15.26:0/4184501267 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7ff18c0a2c90 0x7ff18c0a5150 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:17.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:17.340+0000 7ff1a7591640 1 --2- 172.21.15.26:0/4184501267 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff1a0102ef0 0x7ff1a019ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:17.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:17.340+0000 7ff1a7591640 1 --2- 172.21.15.26:0/4184501267 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1a0102570 0x7ff1a0198ad0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:17.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:17.340+0000 7ff1a7591640 1 --2- 172.21.15.26:0/4184501267 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff1a0100e20 0x7ff1a0198590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:17.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:17.340+0000 7ff1a7591640 1 -- 172.21.15.26:0/4184501267 >> 172.21.15.26:0/4184501267 conn(0x7ff1a00fc7d0 msgr2=0x7ff1a006a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:17:17.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:17.340+0000 7ff1a7591640 1 -- 172.21.15.26:0/4184501267 shutdown_connections 2024-09-11T23:17:17.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:17.340+0000 7ff1a7591640 1 -- 172.21.15.26:0/4184501267 wait complete. 2024-09-11T23:17:17.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:17.350+0000 7f1ecece2640 1 -- 172.21.15.26:0/1016995949 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ec8101de0 msgr2=0x7f1ec81021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:17.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:17.350+0000 7f1ecece2640 1 --2- 172.21.15.26:0/1016995949 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ec8101de0 0x7f1ec81021c0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc009900 tx=0x7f1ebc02a410 comp rx=0 tx=0).stop 2024-09-11T23:17:17.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:17.350+0000 7f1ecece2640 1 -- 172.21.15.26:0/1016995949 shutdown_connections 2024-09-11T23:17:17.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:17.350+0000 7f1ecece2640 1 --2- 172.21.15.26:0/1016995949 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1ec8102790 0x7f1ec81067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:17.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:17.350+0000 7f1ecece2640 1 --2- 172.21.15.26:0/1016995949 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1ec8106f10 0x7f1ec810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:17.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:17.350+0000 7f1ecece2640 1 --2- 172.21.15.26:0/1016995949 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ec8101de0 0x7f1ec81021c0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:17.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:17.350+0000 7f1ecece2640 1 -- 172.21.15.26:0/1016995949 >> 172.21.15.26:0/1016995949 conn(0x7f1ec80fd8c0 msgr2=0x7f1ec80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:17:17.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:17.350+0000 7f1ecece2640 1 -- 172.21.15.26:0/1016995949 shutdown_connections 2024-09-11T23:17:17.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:17.350+0000 7f1ecece2640 1 -- 172.21.15.26:0/1016995949 wait complete. 2024-09-11T23:17:17.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:17.350+0000 7f1ecece2640 1 Processor -- start 2024-09-11T23:17:17.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:17.351+0000 7f1ecece2640 1 -- start start 2024-09-11T23:17:17.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:17.351+0000 7f1ecece2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ec8101de0 0x7f1ec819a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:17.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:17.351+0000 7f1ecece2640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1ec8102790 0x7f1ec819ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:17.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:17.351+0000 7f1ecece2640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1ec8106f10 0x7f1ec819eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:17.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:17.351+0000 7f1ecece2640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f1ec8111f70 con 0x7f1ec8101de0 2024-09-11T23:17:17.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:17.351+0000 7f1ecece2640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f1ec8111df0 con 0x7f1ec8106f10 2024-09-11T23:17:17.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:17.351+0000 7f1ecece2640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f1ec81120f0 con 0x7f1ec8102790 2024-09-11T23:17:17.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:17.351+0000 7f1ecca7e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ec8101de0 0x7f1ec819a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:17.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:17.351+0000 7f1ecca7e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ec8101de0 0x7f1ec819a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52784/0 (socket says 172.21.15.26:52784) 2024-09-11T23:17:17.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:17.351+0000 7f1ecca7e640 1 -- 172.21.15.26:0/465120876 learned_addr learned my addr 172.21.15.26:0/465120876 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:17:17.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:17.351+0000 7f1ec7fff640 1 --2- 172.21.15.26:0/465120876 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1ec8102790 0x7f1ec819ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:17.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:17.351+0000 7f1ecd27f640 1 --2- 172.21.15.26:0/465120876 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1ec8106f10 0x7f1ec819eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:17.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:17.351+0000 7f1ecca7e640 1 -- 172.21.15.26:0/465120876 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1ec8102790 msgr2=0x7f1ec819ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:17.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:17.351+0000 7f1ecca7e640 1 --2- 172.21.15.26:0/465120876 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1ec8102790 0x7f1ec819ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:17.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:17.351+0000 7f1ecca7e640 1 -- 172.21.15.26:0/465120876 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1ec8106f10 msgr2=0x7f1ec819eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:17.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:17.351+0000 7f1ecca7e640 1 --2- 172.21.15.26:0/465120876 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1ec8106f10 0x7f1ec819eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:17.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:17.351+0000 7f1ecca7e640 1 -- 172.21.15.26:0/465120876 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ec819f590 con 0x7f1ec8101de0 2024-09-11T23:17:17.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:17.351+0000 7f1ec7fff640 1 --2- 172.21.15.26:0/465120876 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1ec8102790 0x7f1ec819ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:17:17.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:17.351+0000 7f1ecd27f640 1 --2- 172.21.15.26:0/465120876 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1ec8106f10 0x7f1ec819eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:17:17.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:17.352+0000 7f1ecca7e640 1 --2- 172.21.15.26:0/465120876 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ec8101de0 0x7f1ec819a5e0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc005ab0 tx=0x7f1ebc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:17:17.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:17.352+0000 7f1ec5ffb640 1 -- 172.21.15.26:0/465120876 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ebc03f030 con 0x7f1ec8101de0 2024-09-11T23:17:17.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:17.352+0000 7f1ec5ffb640 1 -- 172.21.15.26:0/465120876 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ebc003ce0 con 0x7f1ec8101de0 2024-09-11T23:17:17.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:17.352+0000 7f1ecece2640 1 -- 172.21.15.26:0/465120876 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ec819f820 con 0x7f1ec8101de0 2024-09-11T23:17:17.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:17.352+0000 7f1ec5ffb640 1 -- 172.21.15.26:0/465120876 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ebc035070 con 0x7f1ec8101de0 2024-09-11T23:17:17.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:17.352+0000 7f1ecece2640 1 -- 172.21.15.26:0/465120876 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ec81a7160 con 0x7f1ec8101de0 2024-09-11T23:17:17.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:17.353+0000 7f1ecece2640 1 -- 172.21.15.26:0/465120876 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ec81021c0 con 0x7f1ec8101de0 2024-09-11T23:17:17.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:17.355+0000 7f1ec5ffb640 1 -- 172.21.15.26:0/465120876 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f1ebc042030 con 0x7f1ec8101de0 2024-09-11T23:17:17.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:17.355+0000 7f1ec5ffb640 1 --2- 172.21.15.26:0/465120876 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f1eb80a2c10 0x7f1eb80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:17.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:17.355+0000 7f1ec5ffb640 1 -- 172.21.15.26:0/465120876 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f1ebc0efe20 con 0x7f1ec8101de0 2024-09-11T23:17:17.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:17.355+0000 7f1ec7fff640 1 --2- 172.21.15.26:0/465120876 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f1eb80a2c10 0x7f1eb80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:17.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:17.356+0000 7f1ec7fff640 1 --2- 172.21.15.26:0/465120876 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f1eb80a2c10 0x7f1eb80a50d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f1ec80ff6f0 tx=0x7f1eb4005ec0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:17:17.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:17.362+0000 7f1ec5ffb640 1 -- 172.21.15.26:0/465120876 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1ebc0bcc20 con 0x7f1ec8101de0 2024-09-11T23:17:17.478 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:17:17 smithi026 ceph-mon[31172]: pgmap v306: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 255 B/s wr, 11 op/s 2024-09-11T23:17:17.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:17:17 smithi136 ceph-mon[35121]: pgmap v306: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 255 B/s wr, 11 op/s 2024-09-11T23:17:17.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:17.544+0000 7f1ecece2640 1 -- 172.21.15.26:0/465120876 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f1ec81a7360 con 0x7f1ec8101de0 2024-09-11T23:17:17.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:17.544+0000 7f1ec5ffb640 1 -- 172.21.15.26:0/465120876 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f1ebc0bc520 con 0x7f1ec8101de0 2024-09-11T23:17:17.546 INFO:teuthology.orchestra.run.smithi026.stdout:330712481816 2024-09-11T23:17:17.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:17.546+0000 7f1ecece2640 1 -- 172.21.15.26:0/465120876 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f1eb80a2c10 msgr2=0x7f1eb80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:17.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:17.546+0000 7f1ecece2640 1 --2- 172.21.15.26:0/465120876 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f1eb80a2c10 0x7f1eb80a50d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f1ec80ff6f0 tx=0x7f1eb4005ec0 comp rx=0 tx=0).stop 2024-09-11T23:17:17.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:17.546+0000 7f1ecece2640 1 -- 172.21.15.26:0/465120876 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ec8101de0 msgr2=0x7f1ec819a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:17.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:17.546+0000 7f1ecece2640 1 --2- 172.21.15.26:0/465120876 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ec8101de0 0x7f1ec819a5e0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc005ab0 tx=0x7f1ebc0042f0 comp rx=0 tx=0).stop 2024-09-11T23:17:17.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:17.546+0000 7f1ecece2640 1 -- 172.21.15.26:0/465120876 shutdown_connections 2024-09-11T23:17:17.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:17.546+0000 7f1ecece2640 1 --2- 172.21.15.26:0/465120876 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f1eb80a2c10 0x7f1eb80a50d0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:17.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:17.546+0000 7f1ecece2640 1 --2- 172.21.15.26:0/465120876 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1ec8106f10 0x7f1ec819eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:17.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:17.546+0000 7f1ecece2640 1 --2- 172.21.15.26:0/465120876 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1ec8102790 0x7f1ec819ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:17.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:17.546+0000 7f1ecece2640 1 --2- 172.21.15.26:0/465120876 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ec8101de0 0x7f1ec819a5e0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:17.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:17.547+0000 7f1ecece2640 1 -- 172.21.15.26:0/465120876 >> 172.21.15.26:0/465120876 conn(0x7f1ec80fd8c0 msgr2=0x7f1ec80feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:17:17.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:17.547+0000 7f1ecece2640 1 -- 172.21.15.26:0/465120876 shutdown_connections 2024-09-11T23:17:17.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:17.547+0000 7f1ecece2640 1 -- 172.21.15.26:0/465120876 wait complete. 2024-09-11T23:17:17.553 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:17:17 smithi076 ceph-mon[35296]: pgmap v306: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 255 B/s wr, 11 op/s 2024-09-11T23:17:18.268 INFO:tasks.cephadm.ceph_manager.ceph:need seq 330712481814 got 330712481816 for osd.11 2024-09-11T23:17:18.269 DEBUG:teuthology.parallel:result is None 2024-09-11T23:17:18.374 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:17:18 smithi026 ceph-mon[31172]: from='client.? 172.21.15.26:0/4184501267' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-11T23:17:18.374 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:17:18 smithi026 ceph-mon[31172]: from='client.? 172.21.15.26:0/465120876' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-11T23:17:18.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:17:18 smithi136 ceph-mon[35121]: from='client.? 172.21.15.26:0/4184501267' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-11T23:17:18.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:17:18 smithi136 ceph-mon[35121]: from='client.? 172.21.15.26:0/465120876' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-11T23:17:18.553 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:17:18 smithi076 ceph-mon[35296]: from='client.? 172.21.15.26:0/4184501267' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-11T23:17:18.553 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:17:18 smithi076 ceph-mon[35296]: from='client.? 172.21.15.26:0/465120876' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-11T23:17:18.684 INFO:tasks.cephadm.ceph_manager.ceph:need seq 193273528364 got 193273528367 for osd.6 2024-09-11T23:17:18.684 DEBUG:teuthology.parallel:result is None 2024-09-11T23:17:19.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:17:19 smithi136 ceph-mon[35121]: pgmap v307: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T23:17:19.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:17:19 smithi026 ceph-mon[31172]: pgmap v307: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T23:17:19.553 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:17:19 smithi076 ceph-mon[35296]: pgmap v307: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T23:17:19.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:19.583+0000 7fe780db1640 1 -- 172.21.15.26:0/3825355356 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe77c102df0 msgr2=0x7fe77c1031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:19.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:19.584+0000 7fe780db1640 1 --2- 172.21.15.26:0/3825355356 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe77c102df0 0x7fe77c1031d0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fe76c009ca0 tx=0x7fe76c02a8e0 comp rx=0 tx=0).stop 2024-09-11T23:17:19.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:19.584+0000 7fe780db1640 1 -- 172.21.15.26:0/3825355356 shutdown_connections 2024-09-11T23:17:19.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:19.584+0000 7fe780db1640 1 --2- 172.21.15.26:0/3825355356 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe77c10c2f0 0x7fe77c10e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:19.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:19.584+0000 7fe780db1640 1 --2- 172.21.15.26:0/3825355356 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe77c1037a0 0x7fe77c10bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:19.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:19.584+0000 7fe780db1640 1 --2- 172.21.15.26:0/3825355356 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe77c102df0 0x7fe77c1031d0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:19.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:19.584+0000 7fe780db1640 1 -- 172.21.15.26:0/3825355356 >> 172.21.15.26:0/3825355356 conn(0x7fe77c0fc7d0 msgr2=0x7fe77c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:17:19.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:19.584+0000 7fe780db1640 1 -- 172.21.15.26:0/3825355356 shutdown_connections 2024-09-11T23:17:19.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:19.584+0000 7fe780db1640 1 -- 172.21.15.26:0/3825355356 wait complete. 2024-09-11T23:17:19.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:19.584+0000 7fe780db1640 1 Processor -- start 2024-09-11T23:17:19.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:19.585+0000 7fe780db1640 1 -- start start 2024-09-11T23:17:19.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:19.585+0000 7fe780db1640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe77c102df0 0x7fe77c1986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:19.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:19.585+0000 7fe780db1640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe77c1037a0 0x7fe77c198c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:19.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:19.585+0000 7fe780db1640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe77c10c2f0 0x7fe77c19cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:19.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:19.585+0000 7fe780db1640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fe77c1121d0 con 0x7fe77c1037a0 2024-09-11T23:17:19.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:19.585+0000 7fe780db1640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fe77c112050 con 0x7fe77c10c2f0 2024-09-11T23:17:19.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:19.585+0000 7fe780db1640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe77c112350 con 0x7fe77c102df0 2024-09-11T23:17:19.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:19.585+0000 7fe779d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe77c1037a0 0x7fe77c198c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:19.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:19.585+0000 7fe779d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe77c1037a0 0x7fe77c198c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52800/0 (socket says 172.21.15.26:52800) 2024-09-11T23:17:19.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:19.585+0000 7fe779d9b640 1 -- 172.21.15.26:0/2606212294 learned_addr learned my addr 172.21.15.26:0/2606212294 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:17:19.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:19.585+0000 7fe77a59c640 1 --2- 172.21.15.26:0/2606212294 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe77c102df0 0x7fe77c1986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:19.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:19.585+0000 7fe77ad9d640 1 --2- 172.21.15.26:0/2606212294 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe77c10c2f0 0x7fe77c19cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:19.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:19.585+0000 7fe779d9b640 1 -- 172.21.15.26:0/2606212294 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe77c102df0 msgr2=0x7fe77c1986c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:19.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:19.585+0000 7fe779d9b640 1 --2- 172.21.15.26:0/2606212294 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe77c102df0 0x7fe77c1986c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:19.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:19.585+0000 7fe779d9b640 1 -- 172.21.15.26:0/2606212294 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe77c10c2f0 msgr2=0x7fe77c19cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:19.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:19.585+0000 7fe779d9b640 1 --2- 172.21.15.26:0/2606212294 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe77c10c2f0 0x7fe77c19cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:19.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:19.585+0000 7fe779d9b640 1 -- 172.21.15.26:0/2606212294 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe77c19d670 con 0x7fe77c1037a0 2024-09-11T23:17:19.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:19.585+0000 7fe77ad9d640 1 --2- 172.21.15.26:0/2606212294 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe77c10c2f0 0x7fe77c19cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:17:19.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:19.586+0000 7fe779d9b640 1 --2- 172.21.15.26:0/2606212294 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe77c1037a0 0x7fe77c198c00 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fe764006fd0 tx=0x7fe76400b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:17:19.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:19.586+0000 7fe7737fe640 1 -- 172.21.15.26:0/2606212294 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe764011830 con 0x7fe77c1037a0 2024-09-11T23:17:19.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:19.586+0000 7fe7737fe640 1 -- 172.21.15.26:0/2606212294 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe764016070 con 0x7fe77c1037a0 2024-09-11T23:17:19.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:19.586+0000 7fe780db1640 1 -- 172.21.15.26:0/2606212294 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe77c19d960 con 0x7fe77c1037a0 2024-09-11T23:17:19.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:19.586+0000 7fe7737fe640 1 -- 172.21.15.26:0/2606212294 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe76400f2e0 con 0x7fe77c1037a0 2024-09-11T23:17:19.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:19.586+0000 7fe780db1640 1 -- 172.21.15.26:0/2606212294 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe77c1a5240 con 0x7fe77c1037a0 2024-09-11T23:17:19.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:19.587+0000 7fe780db1640 1 -- 172.21.15.26:0/2606212294 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe77c06a790 con 0x7fe77c1037a0 2024-09-11T23:17:19.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:19.588+0000 7fe7737fe640 1 -- 172.21.15.26:0/2606212294 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fe76401b460 con 0x7fe77c1037a0 2024-09-11T23:17:19.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:19.588+0000 7fe7737fe640 1 --2- 172.21.15.26:0/2606212294 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fe7680a2c10 0x7fe7680a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:19.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:19.589+0000 7fe77a59c640 1 --2- 172.21.15.26:0/2606212294 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fe7680a2c10 0x7fe7680a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:19.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:19.589+0000 7fe7737fe640 1 -- 172.21.15.26:0/2606212294 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fe7640cee50 con 0x7fe77c1037a0 2024-09-11T23:17:19.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:19.589+0000 7fe77a59c640 1 --2- 172.21.15.26:0/2606212294 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fe7680a2c10 0x7fe7680a50d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fe76c0098d0 tx=0x7fe76c009900 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:17:19.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:19.592+0000 7fe7737fe640 1 -- 172.21.15.26:0/2606212294 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe76409bd20 con 0x7fe77c1037a0 2024-09-11T23:17:19.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:19.770+0000 7fe780db1640 1 -- 172.21.15.26:0/2606212294 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7fe77c1031d0 con 0x7fe77c1037a0 2024-09-11T23:17:19.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:19.770+0000 7fe7737fe640 1 -- 172.21.15.26:0/2606212294 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fe77c1031d0 con 0x7fe77c1037a0 2024-09-11T23:17:19.772 INFO:teuthology.orchestra.run.smithi026.stdout:214748364843 2024-09-11T23:17:19.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:19.773+0000 7fe780db1640 1 -- 172.21.15.26:0/2606212294 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fe7680a2c10 msgr2=0x7fe7680a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:19.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:19.773+0000 7fe780db1640 1 --2- 172.21.15.26:0/2606212294 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fe7680a2c10 0x7fe7680a50d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fe76c0098d0 tx=0x7fe76c009900 comp rx=0 tx=0).stop 2024-09-11T23:17:19.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:19.773+0000 7fe780db1640 1 -- 172.21.15.26:0/2606212294 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe77c1037a0 msgr2=0x7fe77c198c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:19.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:19.773+0000 7fe780db1640 1 --2- 172.21.15.26:0/2606212294 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe77c1037a0 0x7fe77c198c00 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fe764006fd0 tx=0x7fe76400b8e0 comp rx=0 tx=0).stop 2024-09-11T23:17:19.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:19.773+0000 7fe780db1640 1 -- 172.21.15.26:0/2606212294 shutdown_connections 2024-09-11T23:17:19.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:19.773+0000 7fe780db1640 1 --2- 172.21.15.26:0/2606212294 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fe7680a2c10 0x7fe7680a50d0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:19.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:19.773+0000 7fe780db1640 1 --2- 172.21.15.26:0/2606212294 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe77c10c2f0 0x7fe77c19cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:19.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:19.773+0000 7fe780db1640 1 --2- 172.21.15.26:0/2606212294 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe77c1037a0 0x7fe77c198c00 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:19.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:19.773+0000 7fe780db1640 1 --2- 172.21.15.26:0/2606212294 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe77c102df0 0x7fe77c1986c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:19.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:19.773+0000 7fe780db1640 1 -- 172.21.15.26:0/2606212294 >> 172.21.15.26:0/2606212294 conn(0x7fe77c0fc7d0 msgr2=0x7fe77c0fd4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:17:19.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:19.773+0000 7fe780db1640 1 -- 172.21.15.26:0/2606212294 shutdown_connections 2024-09-11T23:17:19.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:19.773+0000 7fe780db1640 1 -- 172.21.15.26:0/2606212294 wait complete. 2024-09-11T23:17:19.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:19.785+0000 7f3ce9139640 1 -- 172.21.15.26:0/2213111691 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3ce4100100 msgr2=0x7f3ce4100520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:19.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:19.785+0000 7f3ce9139640 1 --2- 172.21.15.26:0/2213111691 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3ce4100100 0x7f3ce4100520 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f3cd8009900 tx=0x7f3cd802a410 comp rx=0 tx=0).stop 2024-09-11T23:17:19.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:19.785+0000 7f3ce9139640 1 -- 172.21.15.26:0/2213111691 shutdown_connections 2024-09-11T23:17:19.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:19.785+0000 7f3ce9139640 1 --2- 172.21.15.26:0/2213111691 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3ce4100c80 0x7f3ce4110890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:19.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:19.785+0000 7f3ce9139640 1 --2- 172.21.15.26:0/2213111691 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3ce4100100 0x7f3ce4100520 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:19.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:19.785+0000 7f3ce9139640 1 --2- 172.21.15.26:0/2213111691 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3ce4102a50 0x7f3ce4102e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:19.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:19.785+0000 7f3ce9139640 1 -- 172.21.15.26:0/2213111691 >> 172.21.15.26:0/2213111691 conn(0x7f3ce4077230 msgr2=0x7f3ce40fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:17:19.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:19.785+0000 7f3ce9139640 1 -- 172.21.15.26:0/2213111691 shutdown_connections 2024-09-11T23:17:19.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:19.785+0000 7f3ce9139640 1 -- 172.21.15.26:0/2213111691 wait complete. 2024-09-11T23:17:19.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:19.786+0000 7f3ce9139640 1 Processor -- start 2024-09-11T23:17:19.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:19.786+0000 7f3ce9139640 1 -- start start 2024-09-11T23:17:19.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:19.786+0000 7f3ce9139640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3ce4100100 0x7f3ce419a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:19.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:19.786+0000 7f3ce9139640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3ce4100c80 0x7f3ce419aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:19.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:19.786+0000 7f3ce9139640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3ce4102a50 0x7f3ce419ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:19.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:19.786+0000 7f3ce9139640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f3ce4114120 con 0x7f3ce4100c80 2024-09-11T23:17:19.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:19.786+0000 7f3ce9139640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f3ce4113fa0 con 0x7f3ce4102a50 2024-09-11T23:17:19.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:19.786+0000 7f3ce9139640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f3ce41142a0 con 0x7f3ce4100100 2024-09-11T23:17:19.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:19.786+0000 7f3ce259c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3ce4100c80 0x7f3ce419aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:19.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:19.786+0000 7f3ce259c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3ce4100c80 0x7f3ce419aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52818/0 (socket says 172.21.15.26:52818) 2024-09-11T23:17:19.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:19.786+0000 7f3ce259c640 1 -- 172.21.15.26:0/3813064660 learned_addr learned my addr 172.21.15.26:0/3813064660 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:17:19.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:19.786+0000 7f3ce359e640 1 --2- 172.21.15.26:0/3813064660 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3ce4102a50 0x7f3ce419ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:19.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:19.787+0000 7f3ce2d9d640 1 --2- 172.21.15.26:0/3813064660 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3ce4100100 0x7f3ce419a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:19.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:19.787+0000 7f3ce359e640 1 -- 172.21.15.26:0/3813064660 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3ce4100100 msgr2=0x7f3ce419a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:19.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:19.787+0000 7f3ce359e640 1 --2- 172.21.15.26:0/3813064660 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3ce4100100 0x7f3ce419a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:19.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:19.787+0000 7f3ce359e640 1 -- 172.21.15.26:0/3813064660 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3ce4100c80 msgr2=0x7f3ce419aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:19.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:19.787+0000 7f3ce359e640 1 --2- 172.21.15.26:0/3813064660 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3ce4100c80 0x7f3ce419aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:19.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:19.787+0000 7f3ce359e640 1 -- 172.21.15.26:0/3813064660 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ce419f610 con 0x7f3ce4102a50 2024-09-11T23:17:19.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:19.787+0000 7f3ce359e640 1 --2- 172.21.15.26:0/3813064660 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3ce4102a50 0x7f3ce419ef30 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f3cdc009fd0 tx=0x7f3cdc00ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:17:19.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:19.787+0000 7f3ccbfff640 1 -- 172.21.15.26:0/3813064660 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cdc00ce20 con 0x7f3ce4102a50 2024-09-11T23:17:19.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:19.787+0000 7f3ce9139640 1 -- 172.21.15.26:0/3813064660 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ce419f900 con 0x7f3ce4102a50 2024-09-11T23:17:19.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:19.787+0000 7f3ccbfff640 1 -- 172.21.15.26:0/3813064660 <== mon.2 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3cdc005020 con 0x7f3ce4102a50 2024-09-11T23:17:19.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:19.787+0000 7f3ccbfff640 1 -- 172.21.15.26:0/3813064660 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cdc010720 con 0x7f3ce4102a50 2024-09-11T23:17:19.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:19.787+0000 7f3ce9139640 1 -- 172.21.15.26:0/3813064660 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ce41a71e0 con 0x7f3ce4102a50 2024-09-11T23:17:19.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:19.789+0000 7f3ce9139640 1 -- 172.21.15.26:0/3813064660 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ce4100520 con 0x7f3ce4102a50 2024-09-11T23:17:19.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:19.790+0000 7f3ccbfff640 1 -- 172.21.15.26:0/3813064660 <== mon.2 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f3cdc005540 con 0x7f3ce4102a50 2024-09-11T23:17:19.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:19.791+0000 7f3ccbfff640 1 --2- 172.21.15.26:0/3813064660 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f3ccc0a2c10 0x7f3ccc0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:19.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:19.791+0000 7f3ccbfff640 1 -- 172.21.15.26:0/3813064660 <== mon.2 v2:172.21.15.76:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f3cdc0cf840 con 0x7f3ce4102a50 2024-09-11T23:17:19.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:19.791+0000 7f3ce2d9d640 1 --2- 172.21.15.26:0/3813064660 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f3ccc0a2c10 0x7f3ccc0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:19.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:19.791+0000 7f3ce2d9d640 1 --2- 172.21.15.26:0/3813064660 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f3ccc0a2c10 0x7f3ccc0a50d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f3cd4005fd0 tx=0x7f3cd400b390 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:17:19.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:19.795+0000 7f3ccbfff640 1 -- 172.21.15.26:0/3813064660 <== mon.2 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3cdc09c640 con 0x7f3ce4102a50 2024-09-11T23:17:19.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:19.978+0000 7f3ce9139640 1 -- 172.21.15.26:0/3813064660 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f3ce410bfc0 con 0x7f3ce4102a50 2024-09-11T23:17:19.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:19.978+0000 7f3ccbfff640 1 -- 172.21.15.26:0/3813064660 <== mon.2 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f3cdc09bf40 con 0x7f3ce4102a50 2024-09-11T23:17:19.980 INFO:teuthology.orchestra.run.smithi026.stdout:107374182464 2024-09-11T23:17:19.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:19.980+0000 7f3ce9139640 1 -- 172.21.15.26:0/3813064660 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f3ccc0a2c10 msgr2=0x7f3ccc0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:19.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:19.980+0000 7f3ce9139640 1 --2- 172.21.15.26:0/3813064660 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f3ccc0a2c10 0x7f3ccc0a50d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f3cd4005fd0 tx=0x7f3cd400b390 comp rx=0 tx=0).stop 2024-09-11T23:17:19.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:19.980+0000 7f3ce9139640 1 -- 172.21.15.26:0/3813064660 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3ce4102a50 msgr2=0x7f3ce419ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:19.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:19.980+0000 7f3ce9139640 1 --2- 172.21.15.26:0/3813064660 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3ce4102a50 0x7f3ce419ef30 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f3cdc009fd0 tx=0x7f3cdc00ef10 comp rx=0 tx=0).stop 2024-09-11T23:17:19.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:19.980+0000 7f3ce9139640 1 -- 172.21.15.26:0/3813064660 shutdown_connections 2024-09-11T23:17:19.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:19.980+0000 7f3ce9139640 1 --2- 172.21.15.26:0/3813064660 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f3ccc0a2c10 0x7f3ccc0a50d0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:19.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:19.980+0000 7f3ce9139640 1 --2- 172.21.15.26:0/3813064660 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3ce4102a50 0x7f3ce419ef30 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:19.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:19.980+0000 7f3ce9139640 1 --2- 172.21.15.26:0/3813064660 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3ce4100c80 0x7f3ce419aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:19.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:19.980+0000 7f3ce9139640 1 --2- 172.21.15.26:0/3813064660 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3ce4100100 0x7f3ce419a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:19.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:19.980+0000 7f3ce9139640 1 -- 172.21.15.26:0/3813064660 >> 172.21.15.26:0/3813064660 conn(0x7f3ce4077230 msgr2=0x7f3ce40fd9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:17:19.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:19.981+0000 7f3ce9139640 1 -- 172.21.15.26:0/3813064660 shutdown_connections 2024-09-11T23:17:19.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:19.981+0000 7f3ce9139640 1 -- 172.21.15.26:0/3813064660 wait complete. 2024-09-11T23:17:20.342 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:17:20 smithi026 ceph-mon[31172]: from='client.? 172.21.15.26:0/2606212294' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-11T23:17:20.342 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:17:20 smithi026 ceph-mon[31172]: from='client.? 172.21.15.26:0/3813064660' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-11T23:17:20.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:17:20 smithi136 ceph-mon[35121]: from='client.? 172.21.15.26:0/2606212294' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-11T23:17:20.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:17:20 smithi136 ceph-mon[35121]: from='client.? 172.21.15.26:0/3813064660' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-11T23:17:20.552 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:17:20 smithi076 ceph-mon[35296]: from='client.? 172.21.15.26:0/2606212294' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-11T23:17:20.553 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:17:20 smithi076 ceph-mon[35296]: from='client.? 172.21.15.26:0/3813064660' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-11T23:17:20.699 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182462 got 107374182464 for osd.3 2024-09-11T23:17:20.699 DEBUG:teuthology.parallel:result is None 2024-09-11T23:17:21.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:17:21 smithi136 ceph-mon[35121]: pgmap v308: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-11T23:17:21.493 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:17:21 smithi026 ceph-mon[31172]: pgmap v308: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-11T23:17:21.552 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:17:21 smithi076 ceph-mon[35296]: pgmap v308: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-11T23:17:21.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:21.576+0000 7f8240d11640 1 -- 172.21.15.26:0/3658620065 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f823c100e20 msgr2=0x7f823c101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:21.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:21.576+0000 7f8240d11640 1 --2- 172.21.15.26:0/3658620065 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f823c100e20 0x7f823c101200 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f822c009900 tx=0x7f822c02a480 comp rx=0 tx=0).stop 2024-09-11T23:17:21.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:21.576+0000 7f8240d11640 1 -- 172.21.15.26:0/3658620065 shutdown_connections 2024-09-11T23:17:21.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:21.576+0000 7f8240d11640 1 --2- 172.21.15.26:0/3658620065 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f823c102ef0 0x7f823c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:21.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:21.576+0000 7f8240d11640 1 --2- 172.21.15.26:0/3658620065 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f823c102570 0x7f823c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:21.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:21.576+0000 7f8240d11640 1 --2- 172.21.15.26:0/3658620065 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f823c100e20 0x7f823c101200 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:21.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:21.576+0000 7f8240d11640 1 -- 172.21.15.26:0/3658620065 >> 172.21.15.26:0/3658620065 conn(0x7f823c0fc7d0 msgr2=0x7f823c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:17:21.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:21.576+0000 7f8240d11640 1 -- 172.21.15.26:0/3658620065 shutdown_connections 2024-09-11T23:17:21.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:21.576+0000 7f8240d11640 1 -- 172.21.15.26:0/3658620065 wait complete. 2024-09-11T23:17:21.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:21.577+0000 7f8240d11640 1 Processor -- start 2024-09-11T23:17:21.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:21.577+0000 7f8240d11640 1 -- start start 2024-09-11T23:17:21.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:21.577+0000 7f8240d11640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f823c100e20 0x7f823c198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:21.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:21.577+0000 7f8240d11640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f823c102570 0x7f823c198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:21.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:21.577+0000 7f8240d11640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f823c102ef0 0x7f823c19cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:21.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:21.577+0000 7f8240d11640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f823c110f80 con 0x7f823c102570 2024-09-11T23:17:21.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:21.577+0000 7f8240d11640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f823c110e00 con 0x7f823c102ef0 2024-09-11T23:17:21.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:21.577+0000 7f8240d11640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f823c111100 con 0x7f823c100e20 2024-09-11T23:17:21.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:21.577+0000 7f8233fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f823c102570 0x7f823c198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:21.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:21.577+0000 7f8233fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f823c102570 0x7f823c198a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34916/0 (socket says 172.21.15.26:34916) 2024-09-11T23:17:21.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:21.577+0000 7f8233fff640 1 -- 172.21.15.26:0/1437651357 learned_addr learned my addr 172.21.15.26:0/1437651357 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:17:21.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:21.577+0000 7f823a59c640 1 --2- 172.21.15.26:0/1437651357 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f823c100e20 0x7f823c198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:21.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:21.577+0000 7f823ad9d640 1 --2- 172.21.15.26:0/1437651357 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f823c102ef0 0x7f823c19cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:21.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:21.577+0000 7f8233fff640 1 -- 172.21.15.26:0/1437651357 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f823c100e20 msgr2=0x7f823c198500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:21.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:21.577+0000 7f8233fff640 1 --2- 172.21.15.26:0/1437651357 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f823c100e20 0x7f823c198500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:21.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:21.577+0000 7f8233fff640 1 -- 172.21.15.26:0/1437651357 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f823c102ef0 msgr2=0x7f823c19cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:21.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:21.577+0000 7f8233fff640 1 --2- 172.21.15.26:0/1437651357 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f823c102ef0 0x7f823c19cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:21.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:21.577+0000 7f8233fff640 1 -- 172.21.15.26:0/1437651357 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f823c19d4b0 con 0x7f823c102570 2024-09-11T23:17:21.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:21.578+0000 7f8233fff640 1 --2- 172.21.15.26:0/1437651357 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f823c102570 0x7f823c198a40 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f822400d950 tx=0x7f822400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:17:21.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:21.578+0000 7f82337fe640 1 -- 172.21.15.26:0/1437651357 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8224017030 con 0x7f823c102570 2024-09-11T23:17:21.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:21.578+0000 7f8240d11640 1 -- 172.21.15.26:0/1437651357 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f823c19d7a0 con 0x7f823c102570 2024-09-11T23:17:21.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:21.578+0000 7f82337fe640 1 -- 172.21.15.26:0/1437651357 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8224004020 con 0x7f823c102570 2024-09-11T23:17:21.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:21.578+0000 7f82337fe640 1 -- 172.21.15.26:0/1437651357 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82240047a0 con 0x7f823c102570 2024-09-11T23:17:21.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:21.578+0000 7f8240d11640 1 -- 172.21.15.26:0/1437651357 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f823c1a5080 con 0x7f823c102570 2024-09-11T23:17:21.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:21.579+0000 7f8240d11640 1 -- 172.21.15.26:0/1437651357 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f823c101200 con 0x7f823c102570 2024-09-11T23:17:21.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:21.580+0000 7f82337fe640 1 -- 172.21.15.26:0/1437651357 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f82240041d0 con 0x7f823c102570 2024-09-11T23:17:21.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:21.581+0000 7f82337fe640 1 --2- 172.21.15.26:0/1437651357 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f82280a2c10 0x7f82280a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:21.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:21.581+0000 7f82337fe640 1 -- 172.21.15.26:0/1437651357 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f82240cebd0 con 0x7f823c102570 2024-09-11T23:17:21.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:21.581+0000 7f823a59c640 1 --2- 172.21.15.26:0/1437651357 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f82280a2c10 0x7f82280a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:21.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:21.582+0000 7f823a59c640 1 --2- 172.21.15.26:0/1437651357 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f82280a2c10 0x7f82280a50d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f822c009230 tx=0x7f822c0058d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:17:21.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:21.585+0000 7f82337fe640 1 -- 172.21.15.26:0/1437651357 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f822409b990 con 0x7f823c102570 2024-09-11T23:17:21.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:21.768+0000 7f8240d11640 1 -- 172.21.15.26:0/1437651357 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f823c103610 con 0x7f823c102570 2024-09-11T23:17:21.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:21.768+0000 7f82337fe640 1 -- 172.21.15.26:0/1437651357 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f822409b290 con 0x7f823c102570 2024-09-11T23:17:21.769 INFO:teuthology.orchestra.run.smithi026.stdout:141733920827 2024-09-11T23:17:21.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:21.770+0000 7f8240d11640 1 -- 172.21.15.26:0/1437651357 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f82280a2c10 msgr2=0x7f82280a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:21.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:21.770+0000 7f8240d11640 1 --2- 172.21.15.26:0/1437651357 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f82280a2c10 0x7f82280a50d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f822c009230 tx=0x7f822c0058d0 comp rx=0 tx=0).stop 2024-09-11T23:17:21.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:21.770+0000 7f8240d11640 1 -- 172.21.15.26:0/1437651357 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f823c102570 msgr2=0x7f823c198a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:21.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:21.770+0000 7f8240d11640 1 --2- 172.21.15.26:0/1437651357 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f823c102570 0x7f823c198a40 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f822400d950 tx=0x7f822400de10 comp rx=0 tx=0).stop 2024-09-11T23:17:21.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:21.770+0000 7f8240d11640 1 -- 172.21.15.26:0/1437651357 shutdown_connections 2024-09-11T23:17:21.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:21.770+0000 7f8240d11640 1 --2- 172.21.15.26:0/1437651357 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f82280a2c10 0x7f82280a50d0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:21.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:21.770+0000 7f8240d11640 1 --2- 172.21.15.26:0/1437651357 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f823c102ef0 0x7f823c19cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:21.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:21.770+0000 7f8240d11640 1 --2- 172.21.15.26:0/1437651357 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f823c102570 0x7f823c198a40 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:21.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:21.770+0000 7f8240d11640 1 --2- 172.21.15.26:0/1437651357 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f823c100e20 0x7f823c198500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:21.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:21.770+0000 7f8240d11640 1 -- 172.21.15.26:0/1437651357 >> 172.21.15.26:0/1437651357 conn(0x7f823c0fc7d0 msgr2=0x7f823c06a1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:17:21.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:21.770+0000 7f8240d11640 1 -- 172.21.15.26:0/1437651357 shutdown_connections 2024-09-11T23:17:21.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:21.770+0000 7f8240d11640 1 -- 172.21.15.26:0/1437651357 wait complete. 2024-09-11T23:17:21.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:21.776+0000 7f3f86758640 1 -- 172.21.15.26:0/1825426830 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3f80102570 msgr2=0x7f3f801029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:21.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:21.776+0000 7f3f86758640 1 --2- 172.21.15.26:0/1825426830 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3f80102570 0x7f3f801029b0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f3f74009900 tx=0x7f3f7402a480 comp rx=0 tx=0).stop 2024-09-11T23:17:21.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:21.776+0000 7f3f86758640 1 -- 172.21.15.26:0/1825426830 shutdown_connections 2024-09-11T23:17:21.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:21.776+0000 7f3f86758640 1 --2- 172.21.15.26:0/1825426830 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3f80102ef0 0x7f3f8010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:21.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:21.776+0000 7f3f86758640 1 --2- 172.21.15.26:0/1825426830 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3f80102570 0x7f3f801029b0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:21.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:21.776+0000 7f3f86758640 1 --2- 172.21.15.26:0/1825426830 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3f80100e20 0x7f3f80101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:21.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:21.776+0000 7f3f86758640 1 -- 172.21.15.26:0/1825426830 >> 172.21.15.26:0/1825426830 conn(0x7f3f800fc7d0 msgr2=0x7f3f800febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:17:21.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:21.776+0000 7f3f86758640 1 -- 172.21.15.26:0/1825426830 shutdown_connections 2024-09-11T23:17:21.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:21.776+0000 7f3f86758640 1 -- 172.21.15.26:0/1825426830 wait complete. 2024-09-11T23:17:21.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:21.777+0000 7f3f86758640 1 Processor -- start 2024-09-11T23:17:21.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:21.777+0000 7f3f86758640 1 -- start start 2024-09-11T23:17:21.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:21.778+0000 7f3f86758640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3f80100e20 0x7f3f800fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:21.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:21.778+0000 7f3f86758640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3f80102570 0x7f3f800fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:21.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:21.778+0000 7f3f86758640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3f80102ef0 0x7f3f800fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:21.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:21.778+0000 7f3f86758640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f3f80110f50 con 0x7f3f80100e20 2024-09-11T23:17:21.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:21.778+0000 7f3f86758640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f3f80110dd0 con 0x7f3f80102570 2024-09-11T23:17:21.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:21.778+0000 7f3f86758640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f3f801110d0 con 0x7f3f80102ef0 2024-09-11T23:17:21.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:21.778+0000 7f3f7ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3f80100e20 0x7f3f800fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:21.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:21.778+0000 7f3f7ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3f80100e20 0x7f3f800fcc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34936/0 (socket says 172.21.15.26:34936) 2024-09-11T23:17:21.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:21.778+0000 7f3f7ffff640 1 -- 172.21.15.26:0/1008361522 learned_addr learned my addr 172.21.15.26:0/1008361522 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:17:21.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:21.778+0000 7f3f84cf5640 1 --2- 172.21.15.26:0/1008361522 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3f80102ef0 0x7f3f800fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:21.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:21.778+0000 7f3f7f7fe640 1 --2- 172.21.15.26:0/1008361522 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3f80102570 0x7f3f800fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:21.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:21.778+0000 7f3f7ffff640 1 -- 172.21.15.26:0/1008361522 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3f80102ef0 msgr2=0x7f3f800fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:21.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:21.778+0000 7f3f7ffff640 1 --2- 172.21.15.26:0/1008361522 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3f80102ef0 0x7f3f800fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:21.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:21.778+0000 7f3f7ffff640 1 -- 172.21.15.26:0/1008361522 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3f80102570 msgr2=0x7f3f800fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:21.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:21.778+0000 7f3f7ffff640 1 --2- 172.21.15.26:0/1008361522 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3f80102570 0x7f3f800fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:21.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:21.778+0000 7f3f7ffff640 1 -- 172.21.15.26:0/1008361522 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f801a69b0 con 0x7f3f80100e20 2024-09-11T23:17:21.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:21.778+0000 7f3f7ffff640 1 --2- 172.21.15.26:0/1008361522 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3f80100e20 0x7f3f800fcc80 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f3f7000d950 tx=0x7f3f7000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:17:21.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:21.778+0000 7f3f7d7fa640 1 -- 172.21.15.26:0/1008361522 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f70017030 con 0x7f3f80100e20 2024-09-11T23:17:21.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:21.778+0000 7f3f7d7fa640 1 -- 172.21.15.26:0/1008361522 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3f70004020 con 0x7f3f80100e20 2024-09-11T23:17:21.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:21.778+0000 7f3f7d7fa640 1 -- 172.21.15.26:0/1008361522 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f700047a0 con 0x7f3f80100e20 2024-09-11T23:17:21.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:21.778+0000 7f3f86758640 1 -- 172.21.15.26:0/1008361522 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f801a6ca0 con 0x7f3f80100e20 2024-09-11T23:17:21.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:21.779+0000 7f3f86758640 1 -- 172.21.15.26:0/1008361522 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f801a71b0 con 0x7f3f80100e20 2024-09-11T23:17:21.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:21.780+0000 7f3f86758640 1 -- 172.21.15.26:0/1008361522 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f80101200 con 0x7f3f80100e20 2024-09-11T23:17:21.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:21.782+0000 7f3f7d7fa640 1 -- 172.21.15.26:0/1008361522 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f3f700041d0 con 0x7f3f80100e20 2024-09-11T23:17:21.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:21.782+0000 7f3f7d7fa640 1 --2- 172.21.15.26:0/1008361522 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f3f680a2c10 0x7f3f680a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:21.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:21.783+0000 7f3f7d7fa640 1 -- 172.21.15.26:0/1008361522 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f3f700cf1a0 con 0x7f3f80100e20 2024-09-11T23:17:21.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:21.783+0000 7f3f7f7fe640 1 --2- 172.21.15.26:0/1008361522 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f3f680a2c10 0x7f3f680a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:21.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:21.783+0000 7f3f7f7fe640 1 --2- 172.21.15.26:0/1008361522 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f3f680a2c10 0x7f3f680a50d0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f3f74009230 tx=0x7f3f740058d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:17:21.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:21.787+0000 7f3f7d7fa640 1 -- 172.21.15.26:0/1008361522 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3f7009bfa0 con 0x7f3f80100e20 2024-09-11T23:17:21.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:21.974+0000 7f3f86758640 1 -- 172.21.15.26:0/1008361522 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7f3f800fe1b0 con 0x7f3f80100e20 2024-09-11T23:17:21.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:21.974+0000 7f3f7d7fa640 1 -- 172.21.15.26:0/1008361522 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f3f800fe1b0 con 0x7f3f80100e20 2024-09-11T23:17:21.975 INFO:teuthology.orchestra.run.smithi026.stdout:300647710748 2024-09-11T23:17:21.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:21.977+0000 7f3f86758640 1 -- 172.21.15.26:0/1008361522 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f3f680a2c10 msgr2=0x7f3f680a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:21.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:21.977+0000 7f3f86758640 1 --2- 172.21.15.26:0/1008361522 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f3f680a2c10 0x7f3f680a50d0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f3f74009230 tx=0x7f3f740058d0 comp rx=0 tx=0).stop 2024-09-11T23:17:21.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:21.977+0000 7f3f86758640 1 -- 172.21.15.26:0/1008361522 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3f80100e20 msgr2=0x7f3f800fcc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:21.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:21.977+0000 7f3f86758640 1 --2- 172.21.15.26:0/1008361522 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3f80100e20 0x7f3f800fcc80 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f3f7000d950 tx=0x7f3f7000de10 comp rx=0 tx=0).stop 2024-09-11T23:17:21.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:21.977+0000 7f3f86758640 1 -- 172.21.15.26:0/1008361522 shutdown_connections 2024-09-11T23:17:21.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:21.977+0000 7f3f86758640 1 --2- 172.21.15.26:0/1008361522 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f3f680a2c10 0x7f3f680a50d0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:21.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:21.977+0000 7f3f86758640 1 --2- 172.21.15.26:0/1008361522 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3f80102570 0x7f3f800fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:21.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:21.977+0000 7f3f86758640 1 --2- 172.21.15.26:0/1008361522 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3f80102ef0 0x7f3f800fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:21.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:21.977+0000 7f3f86758640 1 --2- 172.21.15.26:0/1008361522 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3f80100e20 0x7f3f800fcc80 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:21.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:21.977+0000 7f3f86758640 1 -- 172.21.15.26:0/1008361522 >> 172.21.15.26:0/1008361522 conn(0x7f3f800fc7d0 msgr2=0x7f3f8006a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:17:21.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:21.977+0000 7f3f86758640 1 -- 172.21.15.26:0/1008361522 shutdown_connections 2024-09-11T23:17:21.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:21.977+0000 7f3f86758640 1 -- 172.21.15.26:0/1008361522 wait complete. 2024-09-11T23:17:22.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:22.127+0000 7f8a583d9640 1 -- 172.21.15.26:0/3598303224 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8a5010c180 msgr2=0x7f8a5010e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:22.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:22.127+0000 7f8a583d9640 1 --2- 172.21.15.26:0/3598303224 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8a5010c180 0x7f8a5010e600 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f8a48009ca0 tx=0x7f8a4802a950 comp rx=0 tx=0).stop 2024-09-11T23:17:22.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:22.127+0000 7f8a583d9640 1 -- 172.21.15.26:0/3598303224 shutdown_connections 2024-09-11T23:17:22.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:22.127+0000 7f8a583d9640 1 --2- 172.21.15.26:0/3598303224 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8a5010c180 0x7f8a5010e600 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:22.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:22.127+0000 7f8a583d9640 1 --2- 172.21.15.26:0/3598303224 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8a501036c0 0x7f8a5010bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:22.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:22.127+0000 7f8a583d9640 1 --2- 172.21.15.26:0/3598303224 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8a50102da0 0x7f8a50103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:22.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:22.127+0000 7f8a583d9640 1 -- 172.21.15.26:0/3598303224 >> 172.21.15.26:0/3598303224 conn(0x7f8a500fc7a0 msgr2=0x7f8a500febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:17:22.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:22.127+0000 7f8a583d9640 1 -- 172.21.15.26:0/3598303224 shutdown_connections 2024-09-11T23:17:22.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:22.127+0000 7f8a583d9640 1 -- 172.21.15.26:0/3598303224 wait complete. 2024-09-11T23:17:22.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:22.128+0000 7f8a583d9640 1 Processor -- start 2024-09-11T23:17:22.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:22.128+0000 7f8a583d9640 1 -- start start 2024-09-11T23:17:22.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:22.128+0000 7f8a583d9640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8a50102da0 0x7f8a50198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:22.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:22.128+0000 7f8a583d9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8a501036c0 0x7f8a50198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:22.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:22.128+0000 7f8a583d9640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8a5010c180 0x7f8a5019cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:22.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:22.128+0000 7f8a583d9640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f8a50111f90 con 0x7f8a501036c0 2024-09-11T23:17:22.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:22.128+0000 7f8a583d9640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f8a50111e10 con 0x7f8a50102da0 2024-09-11T23:17:22.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:22.128+0000 7f8a583d9640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f8a50112110 con 0x7f8a5010c180 2024-09-11T23:17:22.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:22.128+0000 7f8a55974640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8a501036c0 0x7f8a50198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:22.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:22.128+0000 7f8a55974640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8a501036c0 0x7f8a50198870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34964/0 (socket says 172.21.15.26:34964) 2024-09-11T23:17:22.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:22.128+0000 7f8a55974640 1 -- 172.21.15.26:0/1631323133 learned_addr learned my addr 172.21.15.26:0/1631323133 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:17:22.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:22.129+0000 7f8a56976640 1 --2- 172.21.15.26:0/1631323133 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8a5010c180 0x7f8a5019cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:22.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:22.129+0000 7f8a56175640 1 --2- 172.21.15.26:0/1631323133 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8a50102da0 0x7f8a50198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:22.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:22.129+0000 7f8a55974640 1 -- 172.21.15.26:0/1631323133 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8a5010c180 msgr2=0x7f8a5019cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:22.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:22.129+0000 7f8a55974640 1 --2- 172.21.15.26:0/1631323133 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8a5010c180 0x7f8a5019cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:22.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:22.129+0000 7f8a55974640 1 -- 172.21.15.26:0/1631323133 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8a50102da0 msgr2=0x7f8a50198330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:22.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:22.129+0000 7f8a55974640 1 --2- 172.21.15.26:0/1631323133 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8a50102da0 0x7f8a50198330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:22.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:22.129+0000 7f8a55974640 1 -- 172.21.15.26:0/1631323133 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a5019d2e0 con 0x7f8a501036c0 2024-09-11T23:17:22.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:22.129+0000 7f8a56175640 1 --2- 172.21.15.26:0/1631323133 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8a50102da0 0x7f8a50198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:17:22.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:22.129+0000 7f8a55974640 1 --2- 172.21.15.26:0/1631323133 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8a501036c0 0x7f8a50198870 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f8a4000e9f0 tx=0x7f8a4000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:17:22.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:22.129+0000 7f8a477fe640 1 -- 172.21.15.26:0/1631323133 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a4000cd80 con 0x7f8a501036c0 2024-09-11T23:17:22.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:22.129+0000 7f8a477fe640 1 -- 172.21.15.26:0/1631323133 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a40005500 con 0x7f8a501036c0 2024-09-11T23:17:22.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:22.129+0000 7f8a583d9640 1 -- 172.21.15.26:0/1631323133 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a5019d5d0 con 0x7f8a501036c0 2024-09-11T23:17:22.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:22.129+0000 7f8a477fe640 1 -- 172.21.15.26:0/1631323133 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a40010690 con 0x7f8a501036c0 2024-09-11T23:17:22.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:22.129+0000 7f8a583d9640 1 -- 172.21.15.26:0/1631323133 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a501a4eb0 con 0x7f8a501036c0 2024-09-11T23:17:22.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:22.130+0000 7f8a583d9640 1 -- 172.21.15.26:0/1631323133 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a50103180 con 0x7f8a501036c0 2024-09-11T23:17:22.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:22.131+0000 7f8a477fe640 1 -- 172.21.15.26:0/1631323133 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f8a400046c0 con 0x7f8a501036c0 2024-09-11T23:17:22.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:22.132+0000 7f8a477fe640 1 --2- 172.21.15.26:0/1631323133 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f8a3c0a2c10 0x7f8a3c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:22.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:22.132+0000 7f8a56175640 1 --2- 172.21.15.26:0/1631323133 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f8a3c0a2c10 0x7f8a3c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:22.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:22.133+0000 7f8a477fe640 1 -- 172.21.15.26:0/1631323133 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f8a400cf840 con 0x7f8a501036c0 2024-09-11T23:17:22.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:22.133+0000 7f8a56175640 1 --2- 172.21.15.26:0/1631323133 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f8a3c0a2c10 0x7f8a3c0a50d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f8a4c00abe0 tx=0x7f8a4c00a5c0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:17:22.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:22.136+0000 7f8a477fe640 1 -- 172.21.15.26:0/1631323133 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8a400541b0 con 0x7f8a501036c0 2024-09-11T23:17:22.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:22.318+0000 7f8a583d9640 1 -- 172.21.15.26:0/1631323133 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f8a501a5150 con 0x7f8a501036c0 2024-09-11T23:17:22.319 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:17:22 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:17:22.319 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:17:22 smithi026 ceph-mon[31172]: from='client.? 172.21.15.26:0/1437651357' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-11T23:17:22.319 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:17:22 smithi026 ceph-mon[31172]: from='client.? 172.21.15.26:0/1008361522' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-11T23:17:22.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:22.318+0000 7f8a477fe640 1 -- 172.21.15.26:0/1631323133 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f8a4009c600 con 0x7f8a501036c0 2024-09-11T23:17:22.320 INFO:teuthology.orchestra.run.smithi026.stdout:158913790007 2024-09-11T23:17:22.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:22.320+0000 7f8a583d9640 1 -- 172.21.15.26:0/1631323133 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f8a3c0a2c10 msgr2=0x7f8a3c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:22.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:22.320+0000 7f8a583d9640 1 --2- 172.21.15.26:0/1631323133 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f8a3c0a2c10 0x7f8a3c0a50d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f8a4c00abe0 tx=0x7f8a4c00a5c0 comp rx=0 tx=0).stop 2024-09-11T23:17:22.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:22.320+0000 7f8a583d9640 1 -- 172.21.15.26:0/1631323133 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8a501036c0 msgr2=0x7f8a50198870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:22.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:22.320+0000 7f8a583d9640 1 --2- 172.21.15.26:0/1631323133 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8a501036c0 0x7f8a50198870 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f8a4000e9f0 tx=0x7f8a4000edf0 comp rx=0 tx=0).stop 2024-09-11T23:17:22.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:22.320+0000 7f8a583d9640 1 -- 172.21.15.26:0/1631323133 shutdown_connections 2024-09-11T23:17:22.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:22.320+0000 7f8a583d9640 1 --2- 172.21.15.26:0/1631323133 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f8a3c0a2c10 0x7f8a3c0a50d0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:22.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:22.320+0000 7f8a583d9640 1 --2- 172.21.15.26:0/1631323133 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8a5010c180 0x7f8a5019cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:22.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:22.320+0000 7f8a583d9640 1 --2- 172.21.15.26:0/1631323133 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8a501036c0 0x7f8a50198870 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:22.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:22.320+0000 7f8a583d9640 1 --2- 172.21.15.26:0/1631323133 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8a50102da0 0x7f8a50198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:22.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:22.320+0000 7f8a583d9640 1 -- 172.21.15.26:0/1631323133 >> 172.21.15.26:0/1631323133 conn(0x7f8a500fc7a0 msgr2=0x7f8a500fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:17:22.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:22.320+0000 7f8a583d9640 1 -- 172.21.15.26:0/1631323133 shutdown_connections 2024-09-11T23:17:22.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:22.320+0000 7f8a583d9640 1 -- 172.21.15.26:0/1631323133 wait complete. 2024-09-11T23:17:22.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:22.342+0000 7fb1a2029640 1 -- 172.21.15.26:0/2104728625 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb19c109110 msgr2=0x7fb19c110c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:22.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:22.342+0000 7fb1a2029640 1 --2- 172.21.15.26:0/2104728625 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb19c109110 0x7fb19c110c90 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fb194009ca0 tx=0x7fb19402a8c0 comp rx=0 tx=0).stop 2024-09-11T23:17:22.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:22.342+0000 7fb1a2029640 1 -- 172.21.15.26:0/2104728625 shutdown_connections 2024-09-11T23:17:22.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:22.342+0000 7fb1a2029640 1 --2- 172.21.15.26:0/2104728625 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb19c104990 0x7fb19c1089e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:22.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:22.342+0000 7fb1a2029640 1 --2- 172.21.15.26:0/2104728625 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb19c109110 0x7fb19c110c90 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:22.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:22.342+0000 7fb1a2029640 1 --2- 172.21.15.26:0/2104728625 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb19c103fe0 0x7fb19c1043c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:22.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:22.342+0000 7fb1a2029640 1 -- 172.21.15.26:0/2104728625 >> 172.21.15.26:0/2104728625 conn(0x7fb19c0ffac0 msgr2=0x7fb19c101ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:17:22.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:22.342+0000 7fb1a2029640 1 -- 172.21.15.26:0/2104728625 shutdown_connections 2024-09-11T23:17:22.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:22.342+0000 7fb1a2029640 1 -- 172.21.15.26:0/2104728625 wait complete. 2024-09-11T23:17:22.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:22.342+0000 7fb1a2029640 1 Processor -- start 2024-09-11T23:17:22.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:22.343+0000 7fb1a2029640 1 -- start start 2024-09-11T23:17:22.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:22.343+0000 7fb1a2029640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb19c103fe0 0x7fb19c19a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:22.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:22.343+0000 7fb1a2029640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb19c104990 0x7fb19c19ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:22.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:22.343+0000 7fb1a2029640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb19c109110 0x7fb19c19eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:22.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:22.343+0000 7fb1a2029640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb19c114120 con 0x7fb19c104990 2024-09-11T23:17:22.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:22.343+0000 7fb1a2029640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fb19c113fa0 con 0x7fb19c103fe0 2024-09-11T23:17:22.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:22.343+0000 7fb1a2029640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb19c1142a0 con 0x7fb19c109110 2024-09-11T23:17:22.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:22.343+0000 7fb19affd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb19c104990 0x7fb19c19ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:22.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:22.343+0000 7fb19affd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb19c104990 0x7fb19c19ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34974/0 (socket says 172.21.15.26:34974) 2024-09-11T23:17:22.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:22.343+0000 7fb19affd640 1 -- 172.21.15.26:0/3252526603 learned_addr learned my addr 172.21.15.26:0/3252526603 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:17:22.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:22.343+0000 7fb19b7fe640 1 --2- 172.21.15.26:0/3252526603 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb19c103fe0 0x7fb19c19a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:22.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:22.343+0000 7fb19bfff640 1 --2- 172.21.15.26:0/3252526603 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb19c109110 0x7fb19c19eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:22.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:22.343+0000 7fb19affd640 1 -- 172.21.15.26:0/3252526603 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb19c109110 msgr2=0x7fb19c19eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:22.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:22.343+0000 7fb19affd640 1 --2- 172.21.15.26:0/3252526603 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb19c109110 0x7fb19c19eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:22.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:22.343+0000 7fb19affd640 1 -- 172.21.15.26:0/3252526603 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb19c103fe0 msgr2=0x7fb19c19a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:22.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:22.343+0000 7fb19affd640 1 --2- 172.21.15.26:0/3252526603 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb19c103fe0 0x7fb19c19a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:22.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:22.343+0000 7fb19affd640 1 -- 172.21.15.26:0/3252526603 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb19c19f590 con 0x7fb19c104990 2024-09-11T23:17:22.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:22.344+0000 7fb19affd640 1 --2- 172.21.15.26:0/3252526603 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb19c104990 0x7fb19c19ab20 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fb19000e9f0 tx=0x7fb19000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:17:22.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:22.344+0000 7fb198ff9640 1 -- 172.21.15.26:0/3252526603 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb19000cd80 con 0x7fb19c104990 2024-09-11T23:17:22.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:22.344+0000 7fb1a2029640 1 -- 172.21.15.26:0/3252526603 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb19c19f880 con 0x7fb19c104990 2024-09-11T23:17:22.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:22.344+0000 7fb198ff9640 1 -- 172.21.15.26:0/3252526603 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb190010910 con 0x7fb19c104990 2024-09-11T23:17:22.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:22.344+0000 7fb198ff9640 1 -- 172.21.15.26:0/3252526603 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1900188f0 con 0x7fb19c104990 2024-09-11T23:17:22.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:22.344+0000 7fb1a2029640 1 -- 172.21.15.26:0/3252526603 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb19c1a7160 con 0x7fb19c104990 2024-09-11T23:17:22.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:22.345+0000 7fb1a2029640 1 -- 172.21.15.26:0/3252526603 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb19c1043c0 con 0x7fb19c104990 2024-09-11T23:17:22.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:22.347+0000 7fb198ff9640 1 -- 172.21.15.26:0/3252526603 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fb190010ac0 con 0x7fb19c104990 2024-09-11T23:17:22.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:22.347+0000 7fb198ff9640 1 --2- 172.21.15.26:0/3252526603 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fb1840a2c10 0x7fb1840a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:22.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:22.348+0000 7fb198ff9640 1 -- 172.21.15.26:0/3252526603 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fb1900cf9d0 con 0x7fb19c104990 2024-09-11T23:17:22.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:22.348+0000 7fb19b7fe640 1 --2- 172.21.15.26:0/3252526603 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fb1840a2c10 0x7fb1840a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:22.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:22.348+0000 7fb19b7fe640 1 --2- 172.21.15.26:0/3252526603 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fb1840a2c10 0x7fb1840a50d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fb18c00bbe0 tx=0x7fb18c00b390 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:17:22.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:22.351+0000 7fb198ff9640 1 -- 172.21.15.26:0/3252526603 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb19009c790 con 0x7fb19c104990 2024-09-11T23:17:22.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:17:22 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:17:22.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:17:22 smithi136 ceph-mon[35121]: from='client.? 172.21.15.26:0/1437651357' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-11T23:17:22.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:17:22 smithi136 ceph-mon[35121]: from='client.? 172.21.15.26:0/1008361522' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-11T23:17:22.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:22.534+0000 7fb1a2029640 1 -- 172.21.15.26:0/3252526603 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fb19c10bdc0 con 0x7fb19c104990 2024-09-11T23:17:22.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:22.535+0000 7fb198ff9640 1 -- 172.21.15.26:0/3252526603 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fb19009c090 con 0x7fb19c104990 2024-09-11T23:17:22.536 INFO:teuthology.orchestra.run.smithi026.stdout:55834574923 2024-09-11T23:17:22.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:22.537+0000 7fb1a2029640 1 -- 172.21.15.26:0/3252526603 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fb1840a2c10 msgr2=0x7fb1840a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:22.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:22.537+0000 7fb1a2029640 1 --2- 172.21.15.26:0/3252526603 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fb1840a2c10 0x7fb1840a50d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fb18c00bbe0 tx=0x7fb18c00b390 comp rx=0 tx=0).stop 2024-09-11T23:17:22.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:22.537+0000 7fb1a2029640 1 -- 172.21.15.26:0/3252526603 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb19c104990 msgr2=0x7fb19c19ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:22.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:22.537+0000 7fb1a2029640 1 --2- 172.21.15.26:0/3252526603 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb19c104990 0x7fb19c19ab20 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fb19000e9f0 tx=0x7fb19000eeb0 comp rx=0 tx=0).stop 2024-09-11T23:17:22.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:22.537+0000 7fb1a2029640 1 -- 172.21.15.26:0/3252526603 shutdown_connections 2024-09-11T23:17:22.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:22.537+0000 7fb1a2029640 1 --2- 172.21.15.26:0/3252526603 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fb1840a2c10 0x7fb1840a50d0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:22.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:22.537+0000 7fb1a2029640 1 --2- 172.21.15.26:0/3252526603 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb19c109110 0x7fb19c19eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:22.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:22.537+0000 7fb1a2029640 1 --2- 172.21.15.26:0/3252526603 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb19c104990 0x7fb19c19ab20 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:22.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:22.537+0000 7fb1a2029640 1 --2- 172.21.15.26:0/3252526603 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb19c103fe0 0x7fb19c19a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:22.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:22.537+0000 7fb1a2029640 1 -- 172.21.15.26:0/3252526603 >> 172.21.15.26:0/3252526603 conn(0x7fb19c0ffac0 msgr2=0x7fb19c100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:17:22.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:22.537+0000 7fb1a2029640 1 -- 172.21.15.26:0/3252526603 shutdown_connections 2024-09-11T23:17:22.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:22.537+0000 7fb1a2029640 1 -- 172.21.15.26:0/3252526603 wait complete. 2024-09-11T23:17:22.552 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:17:22 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:17:22.552 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:17:22 smithi076 ceph-mon[35296]: from='client.? 172.21.15.26:0/1437651357' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-11T23:17:22.553 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:17:22 smithi076 ceph-mon[35296]: from='client.? 172.21.15.26:0/1008361522' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-11T23:17:23.173 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574921 got 55834574923 for osd.1 2024-09-11T23:17:23.173 DEBUG:teuthology.parallel:result is None 2024-09-11T23:17:23.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:17:23 smithi136 ceph-mon[35121]: pgmap v309: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-11T23:17:23.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:17:23 smithi136 ceph-mon[35121]: from='client.? 172.21.15.26:0/1631323133' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-11T23:17:23.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:17:23 smithi136 ceph-mon[35121]: from='client.? 172.21.15.26:0/3252526603' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-11T23:17:23.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:17:23 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:17:23.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:17:23 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:17:23.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:17:23 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:17:23.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:17:23 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:17:23.501 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:17:23 smithi026 ceph-mon[31172]: pgmap v309: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-11T23:17:23.501 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:17:23 smithi026 ceph-mon[31172]: from='client.? 172.21.15.26:0/1631323133' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-11T23:17:23.501 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:17:23 smithi026 ceph-mon[31172]: from='client.? 172.21.15.26:0/3252526603' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-11T23:17:23.501 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:17:23 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:17:23.501 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:17:23 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:17:23.501 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:17:23 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:17:23.501 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:17:23 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:17:23.553 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:17:23 smithi076 ceph-mon[35296]: pgmap v309: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-11T23:17:23.553 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:17:23 smithi076 ceph-mon[35296]: from='client.? 172.21.15.26:0/1631323133' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-11T23:17:23.553 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:17:23 smithi076 ceph-mon[35296]: from='client.? 172.21.15.26:0/3252526603' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-11T23:17:23.553 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:17:23 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:17:23.553 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:17:23 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:17:23.553 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:17:23 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:17:23.553 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:17:23 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:17:23.571 INFO:tasks.cephadm.ceph_manager.ceph:need seq 214748364842 got 214748364843 for osd.7 2024-09-11T23:17:23.572 DEBUG:teuthology.parallel:result is None 2024-09-11T23:17:24.423 INFO:tasks.cephadm.ceph_manager.ceph:need seq 141733920824 got 141733920827 for osd.4 2024-09-11T23:17:24.423 DEBUG:teuthology.parallel:result is None 2024-09-11T23:17:25.173 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:17:25.190 INFO:tasks.cephadm.ceph_manager.ceph:need seq 158913790005 got 158913790007 for osd.5 2024-09-11T23:17:25.190 DEBUG:teuthology.parallel:result is None 2024-09-11T23:17:25.390 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:17:25 smithi026 ceph-mon[31172]: pgmap v310: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T23:17:25.553 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:17:25 smithi076 ceph-mon[35296]: pgmap v310: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T23:17:25.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:25.709+0000 7f2730e54640 1 -- 172.21.15.26:0/1452303616 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f272c106f10 msgr2=0x7f272c10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:25.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:25.709+0000 7f2730e54640 1 --2- 172.21.15.26:0/1452303616 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f272c106f10 0x7f272c10ea90 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f2724009ca0 tx=0x7f272402a8c0 comp rx=0 tx=0).stop 2024-09-11T23:17:25.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:25.709+0000 7f2730e54640 1 -- 172.21.15.26:0/1452303616 shutdown_connections 2024-09-11T23:17:25.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:25.709+0000 7f2730e54640 1 --2- 172.21.15.26:0/1452303616 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f272c102790 0x7f272c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:25.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:25.709+0000 7f2730e54640 1 --2- 172.21.15.26:0/1452303616 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f272c106f10 0x7f272c10ea90 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:25.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:25.709+0000 7f2730e54640 1 --2- 172.21.15.26:0/1452303616 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f272c101de0 0x7f272c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:25.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:25.709+0000 7f2730e54640 1 -- 172.21.15.26:0/1452303616 >> 172.21.15.26:0/1452303616 conn(0x7f272c0fd8c0 msgr2=0x7f272c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:17:25.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:25.709+0000 7f2730e54640 1 -- 172.21.15.26:0/1452303616 shutdown_connections 2024-09-11T23:17:25.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:25.709+0000 7f2730e54640 1 -- 172.21.15.26:0/1452303616 wait complete. 2024-09-11T23:17:25.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:25.710+0000 7f2730e54640 1 Processor -- start 2024-09-11T23:17:25.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:25.710+0000 7f2730e54640 1 -- start start 2024-09-11T23:17:25.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:25.710+0000 7f2730e54640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f272c101de0 0x7f272c198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:25.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:25.710+0000 7f2730e54640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f272c102790 0x7f272c198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:25.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:25.710+0000 7f2730e54640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f272c106f10 0x7f272c19cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:25.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:25.710+0000 7f2730e54640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f272c111f70 con 0x7f272c102790 2024-09-11T23:17:25.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:25.710+0000 7f2730e54640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f272c111df0 con 0x7f272c106f10 2024-09-11T23:17:25.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:25.710+0000 7f2730e54640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f272c1120f0 con 0x7f272c101de0 2024-09-11T23:17:25.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:25.711+0000 7f2729d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f272c102790 0x7f272c198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:25.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:25.711+0000 7f2729d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f272c102790 0x7f272c198870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34992/0 (socket says 172.21.15.26:34992) 2024-09-11T23:17:25.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:25.711+0000 7f2729d9b640 1 -- 172.21.15.26:0/2965746375 learned_addr learned my addr 172.21.15.26:0/2965746375 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:17:25.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:25.711+0000 7f272a59c640 1 --2- 172.21.15.26:0/2965746375 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f272c101de0 0x7f272c198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:25.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:25.711+0000 7f272ad9d640 1 --2- 172.21.15.26:0/2965746375 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f272c106f10 0x7f272c19cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:25.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:25.711+0000 7f2729d9b640 1 -- 172.21.15.26:0/2965746375 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f272c101de0 msgr2=0x7f272c198330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:25.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:25.711+0000 7f2729d9b640 1 --2- 172.21.15.26:0/2965746375 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f272c101de0 0x7f272c198330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:25.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:25.711+0000 7f2729d9b640 1 -- 172.21.15.26:0/2965746375 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f272c106f10 msgr2=0x7f272c19cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:25.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:25.711+0000 7f2729d9b640 1 --2- 172.21.15.26:0/2965746375 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f272c106f10 0x7f272c19cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:25.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:25.711+0000 7f2729d9b640 1 -- 172.21.15.26:0/2965746375 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f272c19d2e0 con 0x7f272c102790 2024-09-11T23:17:25.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:25.711+0000 7f272ad9d640 1 --2- 172.21.15.26:0/2965746375 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f272c106f10 0x7f272c19cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:17:25.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:25.711+0000 7f2729d9b640 1 --2- 172.21.15.26:0/2965746375 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f272c102790 0x7f272c198870 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f272000c920 tx=0x7f272000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:17:25.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:25.711+0000 7f271b7fe640 1 -- 172.21.15.26:0/2965746375 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2720007c70 con 0x7f272c102790 2024-09-11T23:17:25.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:25.711+0000 7f271b7fe640 1 -- 172.21.15.26:0/2965746375 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2720007e10 con 0x7f272c102790 2024-09-11T23:17:25.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:25.711+0000 7f2730e54640 1 -- 172.21.15.26:0/2965746375 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f272c19d5d0 con 0x7f272c102790 2024-09-11T23:17:25.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:25.711+0000 7f271b7fe640 1 -- 172.21.15.26:0/2965746375 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2720018890 con 0x7f272c102790 2024-09-11T23:17:25.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:25.711+0000 7f2730e54640 1 -- 172.21.15.26:0/2965746375 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f272c1a4eb0 con 0x7f272c102790 2024-09-11T23:17:25.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:25.712+0000 7f2730e54640 1 -- 172.21.15.26:0/2965746375 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f272c1021c0 con 0x7f272c102790 2024-09-11T23:17:25.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:25.714+0000 7f271b7fe640 1 -- 172.21.15.26:0/2965746375 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f2720018a70 con 0x7f272c102790 2024-09-11T23:17:25.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:25.715+0000 7f271b7fe640 1 --2- 172.21.15.26:0/2965746375 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f27140a2c90 0x7f27140a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:25.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:25.715+0000 7f272a59c640 1 --2- 172.21.15.26:0/2965746375 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f27140a2c90 0x7f27140a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:25.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:25.715+0000 7f271b7fe640 1 -- 172.21.15.26:0/2965746375 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f27200cf310 con 0x7f272c102790 2024-09-11T23:17:25.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:25.715+0000 7f272a59c640 1 --2- 172.21.15.26:0/2965746375 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f27140a2c90 0x7f27140a5150 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f271c009bd0 tx=0x7f271c005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:17:25.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:25.719+0000 7f271b7fe640 1 -- 172.21.15.26:0/2965746375 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2720014030 con 0x7f272c102790 2024-09-11T23:17:25.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:17:25 smithi136 ceph-mon[35121]: pgmap v310: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T23:17:25.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:25.896+0000 7f2730e54640 1 -- 172.21.15.26:0/2965746375 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f272c1a51e0 con 0x7f272c102790 2024-09-11T23:17:25.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:25.896+0000 7f271b7fe640 1 -- 172.21.15.26:0/2965746375 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f272009c0d0 con 0x7f272c102790 2024-09-11T23:17:25.897 INFO:teuthology.orchestra.run.smithi026.stdout:77309411399 2024-09-11T23:17:25.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:25.898+0000 7f2730e54640 1 -- 172.21.15.26:0/2965746375 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f27140a2c90 msgr2=0x7f27140a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:25.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:25.898+0000 7f2730e54640 1 --2- 172.21.15.26:0/2965746375 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f27140a2c90 0x7f27140a5150 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f271c009bd0 tx=0x7f271c005f90 comp rx=0 tx=0).stop 2024-09-11T23:17:25.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:25.898+0000 7f2730e54640 1 -- 172.21.15.26:0/2965746375 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f272c102790 msgr2=0x7f272c198870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:25.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:25.898+0000 7f2730e54640 1 --2- 172.21.15.26:0/2965746375 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f272c102790 0x7f272c198870 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f272000c920 tx=0x7f272000cde0 comp rx=0 tx=0).stop 2024-09-11T23:17:25.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:25.898+0000 7f2730e54640 1 -- 172.21.15.26:0/2965746375 shutdown_connections 2024-09-11T23:17:25.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:25.898+0000 7f2730e54640 1 --2- 172.21.15.26:0/2965746375 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f27140a2c90 0x7f27140a5150 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:25.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:25.898+0000 7f2730e54640 1 --2- 172.21.15.26:0/2965746375 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f272c106f10 0x7f272c19cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:25.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:25.898+0000 7f2730e54640 1 --2- 172.21.15.26:0/2965746375 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f272c102790 0x7f272c198870 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:25.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:25.898+0000 7f2730e54640 1 --2- 172.21.15.26:0/2965746375 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f272c101de0 0x7f272c198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:25.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:25.898+0000 7f2730e54640 1 -- 172.21.15.26:0/2965746375 >> 172.21.15.26:0/2965746375 conn(0x7f272c0fd8c0 msgr2=0x7f272c0fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:17:25.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:25.898+0000 7f2730e54640 1 -- 172.21.15.26:0/2965746375 shutdown_connections 2024-09-11T23:17:25.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:25.898+0000 7f2730e54640 1 -- 172.21.15.26:0/2965746375 wait complete. 2024-09-11T23:17:26.435 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:17:26.467 INFO:tasks.cephadm.ceph_manager.ceph:need seq 300647710745 got 300647710748 for osd.10 2024-09-11T23:17:26.467 DEBUG:teuthology.parallel:result is None 2024-09-11T23:17:26.501 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:17:26 smithi026 ceph-mon[31172]: from='client.? 172.21.15.26:0/2965746375' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-11T23:17:26.553 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:17:26 smithi076 ceph-mon[35296]: from='client.? 172.21.15.26:0/2965746375' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-11T23:17:26.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:17:26 smithi136 ceph-mon[35121]: from='client.? 172.21.15.26:0/2965746375' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-11T23:17:26.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:26.862+0000 7fa84a0e7640 1 -- 172.21.15.26:0/246201183 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa844102790 msgr2=0x7fa8441067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:26.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:26.862+0000 7fa84a0e7640 1 --2- 172.21.15.26:0/246201183 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa844102790 0x7fa8441067e0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fa82c009900 tx=0x7fa82c02a410 comp rx=0 tx=0).stop 2024-09-11T23:17:26.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:26.862+0000 7fa84a0e7640 1 -- 172.21.15.26:0/246201183 shutdown_connections 2024-09-11T23:17:26.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:26.862+0000 7fa84a0e7640 1 --2- 172.21.15.26:0/246201183 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa844106f10 0x7fa84410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:26.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:26.862+0000 7fa84a0e7640 1 --2- 172.21.15.26:0/246201183 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa844102790 0x7fa8441067e0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:26.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:26.862+0000 7fa84a0e7640 1 --2- 172.21.15.26:0/246201183 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa844101de0 0x7fa8441021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:26.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:26.862+0000 7fa84a0e7640 1 -- 172.21.15.26:0/246201183 >> 172.21.15.26:0/246201183 conn(0x7fa8440fd8c0 msgr2=0x7fa8440ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:17:26.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:26.862+0000 7fa84a0e7640 1 -- 172.21.15.26:0/246201183 shutdown_connections 2024-09-11T23:17:26.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:26.862+0000 7fa84a0e7640 1 -- 172.21.15.26:0/246201183 wait complete. 2024-09-11T23:17:26.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:26.863+0000 7fa84a0e7640 1 Processor -- start 2024-09-11T23:17:26.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:26.863+0000 7fa84a0e7640 1 -- start start 2024-09-11T23:17:26.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:26.864+0000 7fa84a0e7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa844101de0 0x7fa844198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:26.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:26.864+0000 7fa84a0e7640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa844102790 0x7fa8441989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:26.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:26.864+0000 7fa84a0e7640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa844106f10 0x7fa84419cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:26.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:26.864+0000 7fa84a0e7640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fa844111f90 con 0x7fa844101de0 2024-09-11T23:17:26.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:26.864+0000 7fa84a0e7640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fa844111e10 con 0x7fa844102790 2024-09-11T23:17:26.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:26.864+0000 7fa84a0e7640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fa844112110 con 0x7fa844106f10 2024-09-11T23:17:26.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:26.864+0000 7fa8437fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa844101de0 0x7fa844198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:26.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:26.864+0000 7fa8437fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa844101de0 0x7fa844198490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35008/0 (socket says 172.21.15.26:35008) 2024-09-11T23:17:26.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:26.864+0000 7fa8437fe640 1 -- 172.21.15.26:0/3590337978 learned_addr learned my addr 172.21.15.26:0/3590337978 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:17:26.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:26.864+0000 7fa842ffd640 1 --2- 172.21.15.26:0/3590337978 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa844102790 0x7fa8441989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:26.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:26.864+0000 7fa843fff640 1 --2- 172.21.15.26:0/3590337978 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa844106f10 0x7fa84419cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:26.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:26.864+0000 7fa8437fe640 1 -- 172.21.15.26:0/3590337978 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa844106f10 msgr2=0x7fa84419cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:26.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:26.864+0000 7fa8437fe640 1 --2- 172.21.15.26:0/3590337978 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa844106f10 0x7fa84419cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:26.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:26.864+0000 7fa8437fe640 1 -- 172.21.15.26:0/3590337978 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa844102790 msgr2=0x7fa8441989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:26.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:26.864+0000 7fa8437fe640 1 --2- 172.21.15.26:0/3590337978 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa844102790 0x7fa8441989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:26.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:26.864+0000 7fa8437fe640 1 -- 172.21.15.26:0/3590337978 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa84419d440 con 0x7fa844101de0 2024-09-11T23:17:26.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:26.864+0000 7fa843fff640 1 --2- 172.21.15.26:0/3590337978 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa844106f10 0x7fa84419cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:17:26.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:26.864+0000 7fa8437fe640 1 --2- 172.21.15.26:0/3590337978 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa844101de0 0x7fa844198490 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fa83400c890 tx=0x7fa83400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:17:26.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:26.864+0000 7fa840ff9640 1 -- 172.21.15.26:0/3590337978 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa834004220 con 0x7fa844101de0 2024-09-11T23:17:26.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:26.865+0000 7fa84a0e7640 1 -- 172.21.15.26:0/3590337978 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa84419d730 con 0x7fa844101de0 2024-09-11T23:17:26.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:26.865+0000 7fa840ff9640 1 -- 172.21.15.26:0/3590337978 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa8340043c0 con 0x7fa844101de0 2024-09-11T23:17:26.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:26.865+0000 7fa840ff9640 1 -- 172.21.15.26:0/3590337978 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8340049e0 con 0x7fa844101de0 2024-09-11T23:17:26.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:26.865+0000 7fa84a0e7640 1 -- 172.21.15.26:0/3590337978 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8441a5010 con 0x7fa844101de0 2024-09-11T23:17:26.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:26.866+0000 7fa84a0e7640 1 -- 172.21.15.26:0/3590337978 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8441021c0 con 0x7fa844101de0 2024-09-11T23:17:26.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:26.867+0000 7fa840ff9640 1 -- 172.21.15.26:0/3590337978 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fa8340079b0 con 0x7fa844101de0 2024-09-11T23:17:26.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:26.868+0000 7fa840ff9640 1 --2- 172.21.15.26:0/3590337978 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fa8300a2c10 0x7fa8300a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:26.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:26.868+0000 7fa842ffd640 1 --2- 172.21.15.26:0/3590337978 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fa8300a2c10 0x7fa8300a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:26.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:26.868+0000 7fa840ff9640 1 -- 172.21.15.26:0/3590337978 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fa8340cf2e0 con 0x7fa844101de0 2024-09-11T23:17:26.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:26.868+0000 7fa842ffd640 1 --2- 172.21.15.26:0/3590337978 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fa8300a2c10 0x7fa8300a50d0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fa8441088e0 tx=0x7fa82c031490 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:17:26.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:26.872+0000 7fa840ff9640 1 -- 172.21.15.26:0/3590337978 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa834014030 con 0x7fa844101de0 2024-09-11T23:17:26.903 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:17:27.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:27.058+0000 7fa84a0e7640 1 -- 172.21.15.26:0/3590337978 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7fa8441997f0 con 0x7fa844101de0 2024-09-11T23:17:27.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:27.058+0000 7fa840ff9640 1 -- 172.21.15.26:0/3590337978 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fa83409c0e0 con 0x7fa844101de0 2024-09-11T23:17:27.060 INFO:teuthology.orchestra.run.smithi026.stdout:244813135913 2024-09-11T23:17:27.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:27.060+0000 7fa84a0e7640 1 -- 172.21.15.26:0/3590337978 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fa8300a2c10 msgr2=0x7fa8300a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:27.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:27.060+0000 7fa84a0e7640 1 --2- 172.21.15.26:0/3590337978 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fa8300a2c10 0x7fa8300a50d0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fa8441088e0 tx=0x7fa82c031490 comp rx=0 tx=0).stop 2024-09-11T23:17:27.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:27.060+0000 7fa84a0e7640 1 -- 172.21.15.26:0/3590337978 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa844101de0 msgr2=0x7fa844198490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:27.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:27.060+0000 7fa84a0e7640 1 --2- 172.21.15.26:0/3590337978 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa844101de0 0x7fa844198490 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fa83400c890 tx=0x7fa83400cd50 comp rx=0 tx=0).stop 2024-09-11T23:17:27.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:27.060+0000 7fa84a0e7640 1 -- 172.21.15.26:0/3590337978 shutdown_connections 2024-09-11T23:17:27.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:27.061+0000 7fa84a0e7640 1 --2- 172.21.15.26:0/3590337978 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fa8300a2c10 0x7fa8300a50d0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:27.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:27.061+0000 7fa84a0e7640 1 --2- 172.21.15.26:0/3590337978 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa844102790 0x7fa8441989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:27.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:27.061+0000 7fa84a0e7640 1 --2- 172.21.15.26:0/3590337978 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa844106f10 0x7fa84419cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:27.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:27.061+0000 7fa84a0e7640 1 --2- 172.21.15.26:0/3590337978 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa844101de0 0x7fa844198490 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:27.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:27.061+0000 7fa84a0e7640 1 -- 172.21.15.26:0/3590337978 >> 172.21.15.26:0/3590337978 conn(0x7fa8440fd8c0 msgr2=0x7fa8440fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:17:27.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:27.061+0000 7fa84a0e7640 1 -- 172.21.15.26:0/3590337978 shutdown_connections 2024-09-11T23:17:27.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:27.061+0000 7fa84a0e7640 1 -- 172.21.15.26:0/3590337978 wait complete. 2024-09-11T23:17:27.426 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:17:27 smithi026 ceph-mon[31172]: pgmap v311: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-11T23:17:27.426 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:17:27 smithi026 ceph-mon[31172]: from='client.? 172.21.15.26:0/3590337978' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-11T23:17:27.454 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411395 got 77309411399 for osd.2 2024-09-11T23:17:27.455 DEBUG:teuthology.parallel:result is None 2024-09-11T23:17:27.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:27.103+0000 7fc13b68d640 1 -- 172.21.15.26:0/3076400517 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc134100a80 msgr2=0x7fc134100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:27.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:27.103+0000 7fc13b68d640 1 --2- 172.21.15.26:0/3076400517 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc134100a80 0x7fc134100ee0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fc128009930 tx=0x7fc12802a420 comp rx=0 tx=0).stop 2024-09-11T23:17:27.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:27.103+0000 7fc13b68d640 1 -- 172.21.15.26:0/3076400517 shutdown_connections 2024-09-11T23:17:27.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:27.103+0000 7fc13b68d640 1 --2- 172.21.15.26:0/3076400517 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc134101420 0x7fc13410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:27.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:27.103+0000 7fc13b68d640 1 --2- 172.21.15.26:0/3076400517 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc134100a80 0x7fc134100ee0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:27.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:27.103+0000 7fc13b68d640 1 --2- 172.21.15.26:0/3076400517 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc134106a80 0x7fc134106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:27.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:27.103+0000 7fc13b68d640 1 -- 172.21.15.26:0/3076400517 >> 172.21.15.26:0/3076400517 conn(0x7fc1340fc7a0 msgr2=0x7fc1340febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:17:27.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:27.103+0000 7fc13b68d640 1 -- 172.21.15.26:0/3076400517 shutdown_connections 2024-09-11T23:17:27.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:27.103+0000 7fc13b68d640 1 -- 172.21.15.26:0/3076400517 wait complete. 2024-09-11T23:17:27.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:27.104+0000 7fc13b68d640 1 Processor -- start 2024-09-11T23:17:27.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:27.104+0000 7fc13b68d640 1 -- start start 2024-09-11T23:17:27.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:27.104+0000 7fc13b68d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc134100a80 0x7fc1340fcdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:27.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:27.104+0000 7fc13b68d640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc134101420 0x7fc1340fd310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:27.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:27.104+0000 7fc13b68d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc134106a80 0x7fc1340fd850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:27.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:27.104+0000 7fc13b68d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc134110e80 con 0x7fc134100a80 2024-09-11T23:17:27.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:27.104+0000 7fc13b68d640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fc134110d00 con 0x7fc134106a80 2024-09-11T23:17:27.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:27.104+0000 7fc13b68d640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fc134111000 con 0x7fc134101420 2024-09-11T23:17:27.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:27.104+0000 7fc139429640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc134100a80 0x7fc1340fcdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:27.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:27.104+0000 7fc139429640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc134100a80 0x7fc1340fcdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35024/0 (socket says 172.21.15.26:35024) 2024-09-11T23:17:27.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:27.104+0000 7fc139429640 1 -- 172.21.15.26:0/580416053 learned_addr learned my addr 172.21.15.26:0/580416053 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:17:27.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:27.104+0000 7fc139c2a640 1 --2- 172.21.15.26:0/580416053 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc134106a80 0x7fc1340fd850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:27.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:27.104+0000 7fc138c28640 1 --2- 172.21.15.26:0/580416053 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc134101420 0x7fc1340fd310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:27.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:27.104+0000 7fc139429640 1 -- 172.21.15.26:0/580416053 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc134101420 msgr2=0x7fc1340fd310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:27.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:27.104+0000 7fc139429640 1 --2- 172.21.15.26:0/580416053 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc134101420 0x7fc1340fd310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:27.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:27.104+0000 7fc139429640 1 -- 172.21.15.26:0/580416053 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc134106a80 msgr2=0x7fc1340fd850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:27.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:27.104+0000 7fc139429640 1 --2- 172.21.15.26:0/580416053 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc134106a80 0x7fc1340fd850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:27.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:27.104+0000 7fc139429640 1 -- 172.21.15.26:0/580416053 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1341a6940 con 0x7fc134100a80 2024-09-11T23:17:27.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:27.104+0000 7fc138c28640 1 --2- 172.21.15.26:0/580416053 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc134101420 0x7fc1340fd310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:17:27.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:27.105+0000 7fc139429640 1 --2- 172.21.15.26:0/580416053 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc134100a80 0x7fc1340fcdd0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fc12400cce0 tx=0x7fc12400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:17:27.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:27.105+0000 7fc1327fc640 1 -- 172.21.15.26:0/580416053 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc124004b20 con 0x7fc134100a80 2024-09-11T23:17:27.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:27.105+0000 7fc1327fc640 1 -- 172.21.15.26:0/580416053 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc124004cc0 con 0x7fc134100a80 2024-09-11T23:17:27.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:27.105+0000 7fc13b68d640 1 -- 172.21.15.26:0/580416053 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1341a6c30 con 0x7fc134100a80 2024-09-11T23:17:27.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:27.105+0000 7fc1327fc640 1 -- 172.21.15.26:0/580416053 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc12400f7d0 con 0x7fc134100a80 2024-09-11T23:17:27.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:27.105+0000 7fc13b68d640 1 -- 172.21.15.26:0/580416053 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1341a71c0 con 0x7fc134100a80 2024-09-11T23:17:27.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:27.106+0000 7fc13b68d640 1 -- 172.21.15.26:0/580416053 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc134100ee0 con 0x7fc134100a80 2024-09-11T23:17:27.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:27.108+0000 7fc1327fc640 1 -- 172.21.15.26:0/580416053 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fc12400f970 con 0x7fc134100a80 2024-09-11T23:17:27.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:27.109+0000 7fc1327fc640 1 --2- 172.21.15.26:0/580416053 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fc11c0a2c90 0x7fc11c0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:27.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:27.109+0000 7fc1327fc640 1 -- 172.21.15.26:0/580416053 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fc1240cf640 con 0x7fc134100a80 2024-09-11T23:17:27.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:27.109+0000 7fc138c28640 1 --2- 172.21.15.26:0/580416053 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fc11c0a2c90 0x7fc11c0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:27.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:27.109+0000 7fc138c28640 1 --2- 172.21.15.26:0/580416053 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fc11c0a2c90 0x7fc11c0a5150 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fc128000c00 tx=0x7fc128032c70 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:17:27.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:27.112+0000 7fc1327fc640 1 -- 172.21.15.26:0/580416053 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc124014030 con 0x7fc134100a80 2024-09-11T23:17:27.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:27.290+0000 7fc13b68d640 1 -- 172.21.15.26:0/580416053 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7fc1340fe300 con 0x7fc134100a80 2024-09-11T23:17:27.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:27.290+0000 7fc1327fc640 1 -- 172.21.15.26:0/580416053 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fc12409c480 con 0x7fc134100a80 2024-09-11T23:17:27.547 INFO:teuthology.orchestra.run.smithi026.stdout:279172874276 2024-09-11T23:17:27.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:27.293+0000 7fc13b68d640 1 -- 172.21.15.26:0/580416053 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fc11c0a2c90 msgr2=0x7fc11c0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:27.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:27.293+0000 7fc13b68d640 1 --2- 172.21.15.26:0/580416053 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fc11c0a2c90 0x7fc11c0a5150 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fc128000c00 tx=0x7fc128032c70 comp rx=0 tx=0).stop 2024-09-11T23:17:27.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:27.293+0000 7fc13b68d640 1 -- 172.21.15.26:0/580416053 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc134100a80 msgr2=0x7fc1340fcdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:27.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:27.293+0000 7fc13b68d640 1 --2- 172.21.15.26:0/580416053 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc134100a80 0x7fc1340fcdd0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fc12400cce0 tx=0x7fc12400a620 comp rx=0 tx=0).stop 2024-09-11T23:17:27.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:27.293+0000 7fc13b68d640 1 -- 172.21.15.26:0/580416053 shutdown_connections 2024-09-11T23:17:27.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:27.293+0000 7fc13b68d640 1 --2- 172.21.15.26:0/580416053 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fc11c0a2c90 0x7fc11c0a5150 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:27.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:27.293+0000 7fc13b68d640 1 --2- 172.21.15.26:0/580416053 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc134106a80 0x7fc1340fd850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:27.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:27.293+0000 7fc13b68d640 1 --2- 172.21.15.26:0/580416053 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc134101420 0x7fc1340fd310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:27.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:27.293+0000 7fc13b68d640 1 --2- 172.21.15.26:0/580416053 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc134100a80 0x7fc1340fcdd0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:27.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:27.293+0000 7fc13b68d640 1 -- 172.21.15.26:0/580416053 >> 172.21.15.26:0/580416053 conn(0x7fc1340fc7a0 msgr2=0x7fc13406aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:17:27.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:27.293+0000 7fc13b68d640 1 -- 172.21.15.26:0/580416053 shutdown_connections 2024-09-11T23:17:27.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:27.293+0000 7fc13b68d640 1 -- 172.21.15.26:0/580416053 wait complete. 2024-09-11T23:17:27.553 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:17:27 smithi076 ceph-mon[35296]: pgmap v311: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-11T23:17:27.553 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:17:27 smithi076 ceph-mon[35296]: from='client.? 172.21.15.26:0/3590337978' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-11T23:17:27.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:17:27 smithi136 ceph-mon[35121]: pgmap v311: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-11T23:17:27.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:17:27 smithi136 ceph-mon[35121]: from='client.? 172.21.15.26:0/3590337978' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-11T23:17:28.150 INFO:tasks.cephadm.ceph_manager.ceph:need seq 279172874273 got 279172874276 for osd.9 2024-09-11T23:17:28.150 DEBUG:teuthology.parallel:result is None 2024-09-11T23:17:28.400 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:17:28 smithi026 ceph-mon[31172]: from='client.? 172.21.15.26:0/580416053' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-11T23:17:28.553 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:17:28 smithi076 ceph-mon[35296]: from='client.? 172.21.15.26:0/580416053' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-11T23:17:28.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:17:28 smithi136 ceph-mon[35121]: from='client.? 172.21.15.26:0/580416053' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-11T23:17:29.057 INFO:tasks.cephadm.ceph_manager.ceph:need seq 244813135909 got 244813135913 for osd.8 2024-09-11T23:17:29.057 DEBUG:teuthology.parallel:result is None 2024-09-11T23:17:29.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:29.248+0000 7f309cfa6640 1 -- 172.21.15.26:0/966040767 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3098100a90 msgr2=0x7f3098100ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:29.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:29.248+0000 7f309cfa6640 1 --2- 172.21.15.26:0/966040767 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3098100a90 0x7f3098100ef0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f308c009930 tx=0x7f308c02a420 comp rx=0 tx=0).stop 2024-09-11T23:17:29.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:29.248+0000 7f309cfa6640 1 -- 172.21.15.26:0/966040767 shutdown_connections 2024-09-11T23:17:29.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:29.248+0000 7f309cfa6640 1 --2- 172.21.15.26:0/966040767 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3098101430 0x7f309810d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:29.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:29.248+0000 7f309cfa6640 1 --2- 172.21.15.26:0/966040767 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3098100a90 0x7f3098100ef0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:29.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:29.248+0000 7f309cfa6640 1 --2- 172.21.15.26:0/966040767 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3098106a90 0x7f3098106e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:29.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:29.248+0000 7f309cfa6640 1 -- 172.21.15.26:0/966040767 >> 172.21.15.26:0/966040767 conn(0x7f30980fc7d0 msgr2=0x7f30980febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:17:29.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:29.248+0000 7f309cfa6640 1 -- 172.21.15.26:0/966040767 shutdown_connections 2024-09-11T23:17:29.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:29.248+0000 7f309cfa6640 1 -- 172.21.15.26:0/966040767 wait complete. 2024-09-11T23:17:29.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:29.249+0000 7f309cfa6640 1 Processor -- start 2024-09-11T23:17:29.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:29.249+0000 7f309cfa6640 1 -- start start 2024-09-11T23:17:29.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:29.249+0000 7f309cfa6640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3098100a90 0x7f309819a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:29.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:29.249+0000 7f309cfa6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3098101430 0x7f309819abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:29.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:29.249+0000 7f309cfa6640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3098106a90 0x7f309819ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:29.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:29.249+0000 7f309cfa6640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f3098111fb0 con 0x7f3098101430 2024-09-11T23:17:29.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:29.249+0000 7f309cfa6640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f3098111e30 con 0x7f3098100a90 2024-09-11T23:17:29.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:29.249+0000 7f309cfa6640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f3098112130 con 0x7f3098106a90 2024-09-11T23:17:29.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:29.249+0000 7f3095d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3098101430 0x7f309819abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:29.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:29.249+0000 7f3095d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3098101430 0x7f309819abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35042/0 (socket says 172.21.15.26:35042) 2024-09-11T23:17:29.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:29.249+0000 7f3095d9b640 1 -- 172.21.15.26:0/1580024108 learned_addr learned my addr 172.21.15.26:0/1580024108 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:17:29.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:29.249+0000 7f3096d9d640 1 --2- 172.21.15.26:0/1580024108 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3098106a90 0x7f309819ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:29.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:29.249+0000 7f309659c640 1 --2- 172.21.15.26:0/1580024108 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3098100a90 0x7f309819a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:29.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:29.250+0000 7f3095d9b640 1 -- 172.21.15.26:0/1580024108 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3098106a90 msgr2=0x7f309819ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:29.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:29.250+0000 7f3095d9b640 1 --2- 172.21.15.26:0/1580024108 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3098106a90 0x7f309819ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:29.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:29.250+0000 7f3095d9b640 1 -- 172.21.15.26:0/1580024108 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3098100a90 msgr2=0x7f309819a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:29.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:29.250+0000 7f3095d9b640 1 --2- 172.21.15.26:0/1580024108 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3098100a90 0x7f309819a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:29.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:29.250+0000 7f3095d9b640 1 -- 172.21.15.26:0/1580024108 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f309819f640 con 0x7f3098101430 2024-09-11T23:17:29.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:29.250+0000 7f3095d9b640 1 --2- 172.21.15.26:0/1580024108 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3098101430 0x7f309819abd0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f308c009230 tx=0x7f308c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:17:29.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:29.250+0000 7f30877fe640 1 -- 172.21.15.26:0/1580024108 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f308c03f030 con 0x7f3098101430 2024-09-11T23:17:29.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:29.250+0000 7f30877fe640 1 -- 172.21.15.26:0/1580024108 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f308c003ad0 con 0x7f3098101430 2024-09-11T23:17:29.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:29.250+0000 7f309cfa6640 1 -- 172.21.15.26:0/1580024108 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f309819f8d0 con 0x7f3098101430 2024-09-11T23:17:29.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:29.250+0000 7f30877fe640 1 -- 172.21.15.26:0/1580024108 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f308c035070 con 0x7f3098101430 2024-09-11T23:17:29.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:29.250+0000 7f309cfa6640 1 -- 172.21.15.26:0/1580024108 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30981a7210 con 0x7f3098101430 2024-09-11T23:17:29.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:29.251+0000 7f309cfa6640 1 -- 172.21.15.26:0/1580024108 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3098100ef0 con 0x7f3098101430 2024-09-11T23:17:29.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:29.253+0000 7f30877fe640 1 -- 172.21.15.26:0/1580024108 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f308c042030 con 0x7f3098101430 2024-09-11T23:17:29.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:29.253+0000 7f30877fe640 1 --2- 172.21.15.26:0/1580024108 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f30800a2c10 0x7f30800a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:29.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:29.253+0000 7f309659c640 1 --2- 172.21.15.26:0/1580024108 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f30800a2c10 0x7f30800a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:29.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:29.253+0000 7f30877fe640 1 -- 172.21.15.26:0/1580024108 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f308c0efac0 con 0x7f3098101430 2024-09-11T23:17:29.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:29.254+0000 7f309659c640 1 --2- 172.21.15.26:0/1580024108 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f30800a2c10 0x7f30800a50d0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f308800bb60 tx=0x7f308800b390 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:17:29.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:29.257+0000 7f30877fe640 1 -- 172.21.15.26:0/1580024108 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f308c0bc270 con 0x7f3098101430 2024-09-11T23:17:29.472 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:17:29 smithi026 ceph-mon[31172]: pgmap v312: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T23:17:29.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:29.471+0000 7f309cfa6640 1 -- 172.21.15.26:0/1580024108 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f30981a7470 con 0x7f3098101430 2024-09-11T23:17:29.472 INFO:teuthology.orchestra.run.smithi026.stdout:34359738449 2024-09-11T23:17:29.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:29.472+0000 7f30877fe640 1 -- 172.21.15.26:0/1580024108 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f308c0c0570 con 0x7f3098101430 2024-09-11T23:17:29.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:29.474+0000 7f309cfa6640 1 -- 172.21.15.26:0/1580024108 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f30800a2c10 msgr2=0x7f30800a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:29.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:29.474+0000 7f309cfa6640 1 --2- 172.21.15.26:0/1580024108 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f30800a2c10 0x7f30800a50d0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f308800bb60 tx=0x7f308800b390 comp rx=0 tx=0).stop 2024-09-11T23:17:29.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:29.474+0000 7f309cfa6640 1 -- 172.21.15.26:0/1580024108 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3098101430 msgr2=0x7f309819abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:29.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:29.474+0000 7f309cfa6640 1 --2- 172.21.15.26:0/1580024108 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3098101430 0x7f309819abd0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f308c009230 tx=0x7f308c004270 comp rx=0 tx=0).stop 2024-09-11T23:17:29.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:29.474+0000 7f309cfa6640 1 -- 172.21.15.26:0/1580024108 shutdown_connections 2024-09-11T23:17:29.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:29.474+0000 7f309cfa6640 1 --2- 172.21.15.26:0/1580024108 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f30800a2c10 0x7f30800a50d0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:29.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:29.474+0000 7f309cfa6640 1 --2- 172.21.15.26:0/1580024108 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3098106a90 0x7f309819ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:29.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:29.474+0000 7f309cfa6640 1 --2- 172.21.15.26:0/1580024108 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3098101430 0x7f309819abd0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:29.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:29.474+0000 7f309cfa6640 1 --2- 172.21.15.26:0/1580024108 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3098100a90 0x7f309819a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:29.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:29.474+0000 7f309cfa6640 1 -- 172.21.15.26:0/1580024108 >> 172.21.15.26:0/1580024108 conn(0x7f30980fc7d0 msgr2=0x7f309810c0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:17:29.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:29.474+0000 7f309cfa6640 1 -- 172.21.15.26:0/1580024108 shutdown_connections 2024-09-11T23:17:29.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:29.474+0000 7f309cfa6640 1 -- 172.21.15.26:0/1580024108 wait complete. 2024-09-11T23:17:29.553 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:17:29 smithi076 ceph-mon[35296]: pgmap v312: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T23:17:29.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:17:29 smithi136 ceph-mon[35121]: pgmap v312: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T23:17:30.446 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:17:30 smithi026 ceph-mon[31172]: from='client.? 172.21.15.26:0/1580024108' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-11T23:17:30.553 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:17:30 smithi076 ceph-mon[35296]: from='client.? 172.21.15.26:0/1580024108' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-11T23:17:30.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:17:30 smithi136 ceph-mon[35121]: from='client.? 172.21.15.26:0/1580024108' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-11T23:17:30.740 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738446 got 34359738449 for osd.0 2024-09-11T23:17:30.741 DEBUG:teuthology.parallel:result is None 2024-09-11T23:17:30.741 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-11T23:17:30.741 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-11T23:17:31.023 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:17:31.421 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:17:31 smithi026 ceph-mon[31172]: pgmap v313: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-11T23:17:31.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:31.505+0000 7f640411e640 1 -- 172.21.15.26:0/510970844 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63fc100e20 msgr2=0x7f63fc101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:31.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:31.505+0000 7f640411e640 1 --2- 172.21.15.26:0/510970844 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63fc100e20 0x7f63fc101200 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f63f8009900 tx=0x7f63f802a480 comp rx=0 tx=0).stop 2024-09-11T23:17:31.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:31.505+0000 7f640411e640 1 -- 172.21.15.26:0/510970844 shutdown_connections 2024-09-11T23:17:31.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:31.505+0000 7f640411e640 1 --2- 172.21.15.26:0/510970844 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f63fc102570 0x7f63fc1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:31.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:31.505+0000 7f640411e640 1 --2- 172.21.15.26:0/510970844 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f63fc102ef0 0x7f63fc10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:31.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:31.505+0000 7f640411e640 1 --2- 172.21.15.26:0/510970844 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63fc100e20 0x7f63fc101200 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:31.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:31.505+0000 7f640411e640 1 -- 172.21.15.26:0/510970844 >> 172.21.15.26:0/510970844 conn(0x7f63fc0fc7d0 msgr2=0x7f63fc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:17:31.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:31.505+0000 7f640411e640 1 -- 172.21.15.26:0/510970844 shutdown_connections 2024-09-11T23:17:31.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:31.505+0000 7f640411e640 1 -- 172.21.15.26:0/510970844 wait complete. 2024-09-11T23:17:31.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:31.506+0000 7f640411e640 1 Processor -- start 2024-09-11T23:17:31.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:31.506+0000 7f640411e640 1 -- start start 2024-09-11T23:17:31.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:31.506+0000 7f640411e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63fc100e20 0x7f63fc10fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:31.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:31.506+0000 7f640411e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f63fc102570 0x7f63fc10ff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:31.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:31.506+0000 7f640411e640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f63fc102ef0 0x7f63fc110480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:31.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:31.506+0000 7f640411e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f63fc1120a0 con 0x7f63fc100e20 2024-09-11T23:17:31.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:31.506+0000 7f640411e640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f63fc111f20 con 0x7f63fc102570 2024-09-11T23:17:31.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:31.506+0000 7f640411e640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f63fc112220 con 0x7f63fc102ef0 2024-09-11T23:17:31.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:31.506+0000 7f6401eba640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63fc100e20 0x7f63fc10fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:31.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:31.506+0000 7f6401eba640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63fc100e20 0x7f63fc10fa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47292/0 (socket says 172.21.15.26:47292) 2024-09-11T23:17:31.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:31.506+0000 7f6401eba640 1 -- 172.21.15.26:0/3081341503 learned_addr learned my addr 172.21.15.26:0/3081341503 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:17:31.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:31.506+0000 7f64016b9640 1 --2- 172.21.15.26:0/3081341503 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f63fc102570 0x7f63fc10ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:31.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:31.507+0000 7f6401eba640 1 -- 172.21.15.26:0/3081341503 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f63fc102ef0 msgr2=0x7f63fc110480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:31.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:31.507+0000 7f6401eba640 1 --2- 172.21.15.26:0/3081341503 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f63fc102ef0 0x7f63fc110480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:31.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:31.507+0000 7f6401eba640 1 -- 172.21.15.26:0/3081341503 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f63fc102570 msgr2=0x7f63fc10ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:31.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:31.507+0000 7f6401eba640 1 --2- 172.21.15.26:0/3081341503 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f63fc102570 0x7f63fc10ff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:31.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:31.507+0000 7f6401eba640 1 -- 172.21.15.26:0/3081341503 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63fc1a6a00 con 0x7f63fc100e20 2024-09-11T23:17:31.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:31.507+0000 7f6401eba640 1 --2- 172.21.15.26:0/3081341503 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63fc100e20 0x7f63fc10fa00 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f63f8006010 tx=0x7f63f8003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:17:31.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:31.507+0000 7f63f2ffd640 1 -- 172.21.15.26:0/3081341503 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63f803f070 con 0x7f63fc100e20 2024-09-11T23:17:31.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:31.507+0000 7f640411e640 1 -- 172.21.15.26:0/3081341503 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63fc1a6c90 con 0x7f63fc100e20 2024-09-11T23:17:31.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:31.507+0000 7f63f2ffd640 1 -- 172.21.15.26:0/3081341503 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63f8031c30 con 0x7f63fc100e20 2024-09-11T23:17:31.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:31.507+0000 7f63f2ffd640 1 -- 172.21.15.26:0/3081341503 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63f8035030 con 0x7f63fc100e20 2024-09-11T23:17:31.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:31.507+0000 7f640411e640 1 -- 172.21.15.26:0/3081341503 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63fc1a71c0 con 0x7f63fc100e20 2024-09-11T23:17:31.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:31.508+0000 7f640411e640 1 -- 172.21.15.26:0/3081341503 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63fc101200 con 0x7f63fc100e20 2024-09-11T23:17:31.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:31.509+0000 7f63f2ffd640 1 -- 172.21.15.26:0/3081341503 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f63f8042070 con 0x7f63fc100e20 2024-09-11T23:17:31.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:31.510+0000 7f63f2ffd640 1 --2- 172.21.15.26:0/3081341503 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f63e80a2c10 0x7f63e80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:31.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:31.510+0000 7f64016b9640 1 --2- 172.21.15.26:0/3081341503 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f63e80a2c10 0x7f63e80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:31.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:31.510+0000 7f63f2ffd640 1 -- 172.21.15.26:0/3081341503 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f63f80efe20 con 0x7f63fc100e20 2024-09-11T23:17:31.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:31.511+0000 7f64016b9640 1 --2- 172.21.15.26:0/3081341503 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f63e80a2c10 0x7f63e80a50d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f63fc111490 tx=0x7f63ec00b390 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:17:31.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:31.513+0000 7f63f2ffd640 1 -- 172.21.15.26:0/3081341503 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f63f8031de0 con 0x7f63fc100e20 2024-09-11T23:17:31.552 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:17:31 smithi076 ceph-mon[35296]: pgmap v313: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-11T23:17:31.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:31.688+0000 7f640411e640 1 -- 172.21.15.26:0/3081341503 --> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f63fc062630 con 0x7f63e80a2c10 2024-09-11T23:17:31.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:31.700+0000 7f63f2ffd640 1 -- 172.21.15.26:0/3081341503 <== mgr.14148 v2:172.21.15.26:6800/110833859 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258746 (secure 0 0 0) 0x7f63fc062630 con 0x7f63e80a2c10 2024-09-11T23:17:31.703 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-11T23:17:31.704 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-09-11T23:17:31.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:31.703+0000 7f640411e640 1 -- 172.21.15.26:0/3081341503 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f63e80a2c10 msgr2=0x7f63e80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:31.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:31.703+0000 7f640411e640 1 --2- 172.21.15.26:0/3081341503 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f63e80a2c10 0x7f63e80a50d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f63fc111490 tx=0x7f63ec00b390 comp rx=0 tx=0).stop 2024-09-11T23:17:31.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:31.703+0000 7f640411e640 1 -- 172.21.15.26:0/3081341503 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63fc100e20 msgr2=0x7f63fc10fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:31.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:31.703+0000 7f640411e640 1 --2- 172.21.15.26:0/3081341503 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63fc100e20 0x7f63fc10fa00 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f63f8006010 tx=0x7f63f8003c10 comp rx=0 tx=0).stop 2024-09-11T23:17:31.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:31.703+0000 7f640411e640 1 -- 172.21.15.26:0/3081341503 shutdown_connections 2024-09-11T23:17:31.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:31.703+0000 7f640411e640 1 --2- 172.21.15.26:0/3081341503 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f63e80a2c10 0x7f63e80a50d0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:31.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:31.703+0000 7f640411e640 1 --2- 172.21.15.26:0/3081341503 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f63fc102570 0x7f63fc10ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:31.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:31.703+0000 7f640411e640 1 --2- 172.21.15.26:0/3081341503 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f63fc102ef0 0x7f63fc110480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:31.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:31.703+0000 7f640411e640 1 --2- 172.21.15.26:0/3081341503 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63fc100e20 0x7f63fc10fa00 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:31.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:31.703+0000 7f640411e640 1 -- 172.21.15.26:0/3081341503 >> 172.21.15.26:0/3081341503 conn(0x7f63fc0fc7d0 msgr2=0x7f63fc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:17:31.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:31.703+0000 7f640411e640 1 -- 172.21.15.26:0/3081341503 shutdown_connections 2024-09-11T23:17:31.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:31.703+0000 7f640411e640 1 -- 172.21.15.26:0/3081341503 wait complete. 2024-09-11T23:17:31.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:17:31 smithi136 ceph-mon[35121]: pgmap v313: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-11T23:17:32.498 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":313,"stamp":"2024-09-11T23:17:30.095710+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657181,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":660,"num_read_kb":451,"num_write":424,"num_write_kb":3073,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":346,"ondisk_log_size":346,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":379104,"kb_used_data":13212,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124562720,"statfs":{"total":1151940427776,"available":1151552225280,"internally_reserved":0,"allocated":13529088,"data_stored":7344879,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":374453561},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":120,"num_read_kb":55,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003068"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.941549+0000","last_change":"2024-09-11T23:16:25.899106+0000","last_active":"2024-09-11T23:16:35.941549+0000","last_peered":"2024-09-11T23:16:35.941549+0000","last_clean":"2024-09-11T23:16:35.941549+0000","last_became_active":"2024-09-11T23:16:25.898654+0000","last_became_peered":"2024-09-11T23:16:25.898654+0000","last_unstale":"2024-09-11T23:16:35.941549+0000","last_undegraded":"2024-09-11T23:16:35.941549+0000","last_fullsized":"2024-09-11T23:16:35.941549+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:21:41.490905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:36.226970+0000","last_change":"2024-09-11T23:16:25.895629+0000","last_active":"2024-09-11T23:16:36.226970+0000","last_peered":"2024-09-11T23:16:36.226970+0000","last_clean":"2024-09-11T23:16:36.226970+0000","last_became_active":"2024-09-11T23:16:25.895308+0000","last_became_peered":"2024-09-11T23:16:25.895308+0000","last_unstale":"2024-09-11T23:16:36.226970+0000","last_undegraded":"2024-09-11T23:16:36.226970+0000","last_fullsized":"2024-09-11T23:16:36.226970+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:39:42.549970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:36.226468+0000","last_change":"2024-09-11T23:16:25.907113+0000","last_active":"2024-09-11T23:16:36.226468+0000","last_peered":"2024-09-11T23:16:36.226468+0000","last_clean":"2024-09-11T23:16:36.226468+0000","last_became_active":"2024-09-11T23:16:25.906803+0000","last_became_peered":"2024-09-11T23:16:25.906803+0000","last_unstale":"2024-09-11T23:16:36.226468+0000","last_undegraded":"2024-09-11T23:16:36.226468+0000","last_fullsized":"2024-09-11T23:16:36.226468+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:36:49.303932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.946167+0000","last_change":"2024-09-11T23:16:35.946167+0000","last_active":"2024-09-11T23:16:35.946167+0000","last_peered":"2024-09-11T23:16:35.946167+0000","last_clean":"2024-09-11T23:16:35.946167+0000","last_became_active":"2024-09-11T23:16:35.945786+0000","last_became_peered":"2024-09-11T23:16:35.945786+0000","last_unstale":"2024-09-11T23:16:35.946167+0000","last_undegraded":"2024-09-11T23:16:35.946167+0000","last_fullsized":"2024-09-11T23:16:35.946167+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:13:03.064910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.944477+0000","last_change":"2024-09-11T23:16:25.897909+0000","last_active":"2024-09-11T23:16:35.944477+0000","last_peered":"2024-09-11T23:16:35.944477+0000","last_clean":"2024-09-11T23:16:35.944477+0000","last_became_active":"2024-09-11T23:16:25.897569+0000","last_became_peered":"2024-09-11T23:16:25.897569+0000","last_unstale":"2024-09-11T23:16:35.944477+0000","last_undegraded":"2024-09-11T23:16:35.944477+0000","last_fullsized":"2024-09-11T23:16:35.944477+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:25:44.727254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.942367+0000","last_change":"2024-09-11T23:16:25.897594+0000","last_active":"2024-09-11T23:16:35.942367+0000","last_peered":"2024-09-11T23:16:35.942367+0000","last_clean":"2024-09-11T23:16:35.942367+0000","last_became_active":"2024-09-11T23:16:25.897223+0000","last_became_peered":"2024-09-11T23:16:25.897223+0000","last_unstale":"2024-09-11T23:16:35.942367+0000","last_undegraded":"2024-09-11T23:16:35.942367+0000","last_fullsized":"2024-09-11T23:16:35.942367+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:10:50.763455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.943511+0000","last_change":"2024-09-11T23:16:35.943511+0000","last_active":"2024-09-11T23:16:35.943511+0000","last_peered":"2024-09-11T23:16:35.943511+0000","last_clean":"2024-09-11T23:16:35.943511+0000","last_became_active":"2024-09-11T23:16:35.943075+0000","last_became_peered":"2024-09-11T23:16:35.943075+0000","last_unstale":"2024-09-11T23:16:35.943511+0000","last_undegraded":"2024-09-11T23:16:35.943511+0000","last_fullsized":"2024-09-11T23:16:35.943511+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:53:43.892421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:36.226370+0000","last_change":"2024-09-11T23:16:25.903380+0000","last_active":"2024-09-11T23:16:36.226370+0000","last_peered":"2024-09-11T23:16:36.226370+0000","last_clean":"2024-09-11T23:16:36.226370+0000","last_became_active":"2024-09-11T23:16:25.903093+0000","last_became_peered":"2024-09-11T23:16:25.903093+0000","last_unstale":"2024-09-11T23:16:36.226370+0000","last_undegraded":"2024-09-11T23:16:36.226370+0000","last_fullsized":"2024-09-11T23:16:36.226370+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:58:50.944623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.939171+0000","last_change":"2024-09-11T23:16:25.908166+0000","last_active":"2024-09-11T23:16:35.939171+0000","last_peered":"2024-09-11T23:16:35.939171+0000","last_clean":"2024-09-11T23:16:35.939171+0000","last_became_active":"2024-09-11T23:16:25.907749+0000","last_became_peered":"2024-09-11T23:16:25.907749+0000","last_unstale":"2024-09-11T23:16:35.939171+0000","last_undegraded":"2024-09-11T23:16:35.939171+0000","last_fullsized":"2024-09-11T23:16:35.939171+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:11:16.705931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.944259+0000","last_change":"2024-09-11T23:16:35.944259+0000","last_active":"2024-09-11T23:16:35.944259+0000","last_peered":"2024-09-11T23:16:35.944259+0000","last_clean":"2024-09-11T23:16:35.944259+0000","last_became_active":"2024-09-11T23:16:35.943767+0000","last_became_peered":"2024-09-11T23:16:35.943767+0000","last_unstale":"2024-09-11T23:16:35.944259+0000","last_undegraded":"2024-09-11T23:16:35.944259+0000","last_fullsized":"2024-09-11T23:16:35.944259+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:20:13.127854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.941922+0000","last_change":"2024-09-11T23:16:25.903256+0000","last_active":"2024-09-11T23:16:35.941922+0000","last_peered":"2024-09-11T23:16:35.941922+0000","last_clean":"2024-09-11T23:16:35.941922+0000","last_became_active":"2024-09-11T23:16:25.902954+0000","last_became_peered":"2024-09-11T23:16:25.902954+0000","last_unstale":"2024-09-11T23:16:35.941922+0000","last_undegraded":"2024-09-11T23:16:35.941922+0000","last_fullsized":"2024-09-11T23:16:35.941922+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:43:48.651400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:36.226960+0000","last_change":"2024-09-11T23:16:25.897656+0000","last_active":"2024-09-11T23:16:36.226960+0000","last_peered":"2024-09-11T23:16:36.226960+0000","last_clean":"2024-09-11T23:16:36.226960+0000","last_became_active":"2024-09-11T23:16:25.897103+0000","last_became_peered":"2024-09-11T23:16:25.897103+0000","last_unstale":"2024-09-11T23:16:36.226960+0000","last_undegraded":"2024-09-11T23:16:36.226960+0000","last_fullsized":"2024-09-11T23:16:36.226960+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:14:34.341383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.938158+0000","last_change":"2024-09-11T23:16:25.902800+0000","last_active":"2024-09-11T23:16:35.938158+0000","last_peered":"2024-09-11T23:16:35.938158+0000","last_clean":"2024-09-11T23:16:35.938158+0000","last_became_active":"2024-09-11T23:16:25.902414+0000","last_became_peered":"2024-09-11T23:16:25.902414+0000","last_unstale":"2024-09-11T23:16:35.938158+0000","last_undegraded":"2024-09-11T23:16:35.938158+0000","last_fullsized":"2024-09-11T23:16:35.938158+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:16:32.427749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:36.226337+0000","last_change":"2024-09-11T23:16:25.894410+0000","last_active":"2024-09-11T23:16:36.226337+0000","last_peered":"2024-09-11T23:16:36.226337+0000","last_clean":"2024-09-11T23:16:36.226337+0000","last_became_active":"2024-09-11T23:16:25.894067+0000","last_became_peered":"2024-09-11T23:16:25.894067+0000","last_unstale":"2024-09-11T23:16:36.226337+0000","last_undegraded":"2024-09-11T23:16:36.226337+0000","last_fullsized":"2024-09-11T23:16:36.226337+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:03:11.913063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:36.226260+0000","last_change":"2024-09-11T23:16:25.903440+0000","last_active":"2024-09-11T23:16:36.226260+0000","last_peered":"2024-09-11T23:16:36.226260+0000","last_clean":"2024-09-11T23:16:36.226260+0000","last_became_active":"2024-09-11T23:16:25.903122+0000","last_became_peered":"2024-09-11T23:16:25.903122+0000","last_unstale":"2024-09-11T23:16:36.226260+0000","last_undegraded":"2024-09-11T23:16:36.226260+0000","last_fullsized":"2024-09-11T23:16:36.226260+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:03:41.065314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.938680+0000","last_change":"2024-09-11T23:16:25.905336+0000","last_active":"2024-09-11T23:16:35.938680+0000","last_peered":"2024-09-11T23:16:35.938680+0000","last_clean":"2024-09-11T23:16:35.938680+0000","last_became_active":"2024-09-11T23:16:25.904699+0000","last_became_peered":"2024-09-11T23:16:25.904699+0000","last_unstale":"2024-09-11T23:16:35.938680+0000","last_undegraded":"2024-09-11T23:16:35.938680+0000","last_fullsized":"2024-09-11T23:16:35.938680+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:16:13.448585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.941545+0000","last_change":"2024-09-11T23:16:25.897977+0000","last_active":"2024-09-11T23:16:35.941545+0000","last_peered":"2024-09-11T23:16:35.941545+0000","last_clean":"2024-09-11T23:16:35.941545+0000","last_became_active":"2024-09-11T23:16:25.897672+0000","last_became_peered":"2024-09-11T23:16:25.897672+0000","last_unstale":"2024-09-11T23:16:35.941545+0000","last_undegraded":"2024-09-11T23:16:35.941545+0000","last_fullsized":"2024-09-11T23:16:35.941545+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:42:56.575066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.939453+0000","last_change":"2024-09-11T23:16:25.897063+0000","last_active":"2024-09-11T23:16:35.939453+0000","last_peered":"2024-09-11T23:16:35.939453+0000","last_clean":"2024-09-11T23:16:35.939453+0000","last_became_active":"2024-09-11T23:16:25.896770+0000","last_became_peered":"2024-09-11T23:16:25.896770+0000","last_unstale":"2024-09-11T23:16:35.939453+0000","last_undegraded":"2024-09-11T23:16:35.939453+0000","last_fullsized":"2024-09-11T23:16:35.939453+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:55:44.735171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.939570+0000","last_change":"2024-09-11T23:16:25.903423+0000","last_active":"2024-09-11T23:16:35.939570+0000","last_peered":"2024-09-11T23:16:35.939570+0000","last_clean":"2024-09-11T23:16:35.939570+0000","last_became_active":"2024-09-11T23:16:25.902906+0000","last_became_peered":"2024-09-11T23:16:25.902906+0000","last_unstale":"2024-09-11T23:16:35.939570+0000","last_undegraded":"2024-09-11T23:16:35.939570+0000","last_fullsized":"2024-09-11T23:16:35.939570+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:40:49.326045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.942800+0000","last_change":"2024-09-11T23:16:25.896564+0000","last_active":"2024-09-11T23:16:35.942800+0000","last_peered":"2024-09-11T23:16:35.942800+0000","last_clean":"2024-09-11T23:16:35.942800+0000","last_became_active":"2024-09-11T23:16:25.896301+0000","last_became_peered":"2024-09-11T23:16:25.896301+0000","last_unstale":"2024-09-11T23:16:35.942800+0000","last_undegraded":"2024-09-11T23:16:35.942800+0000","last_fullsized":"2024-09-11T23:16:35.942800+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:30:30.074853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.938246+0000","last_change":"2024-09-11T23:16:25.903491+0000","last_active":"2024-09-11T23:16:35.938246+0000","last_peered":"2024-09-11T23:16:35.938246+0000","last_clean":"2024-09-11T23:16:35.938246+0000","last_became_active":"2024-09-11T23:16:25.901340+0000","last_became_peered":"2024-09-11T23:16:25.901340+0000","last_unstale":"2024-09-11T23:16:35.938246+0000","last_undegraded":"2024-09-11T23:16:35.938246+0000","last_fullsized":"2024-09-11T23:16:35.938246+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:41:16.058034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.938800+0000","last_change":"2024-09-11T23:16:25.901084+0000","last_active":"2024-09-11T23:16:35.938800+0000","last_peered":"2024-09-11T23:16:35.938800+0000","last_clean":"2024-09-11T23:16:35.938800+0000","last_became_active":"2024-09-11T23:16:25.900726+0000","last_became_peered":"2024-09-11T23:16:25.900726+0000","last_unstale":"2024-09-11T23:16:35.938800+0000","last_undegraded":"2024-09-11T23:16:35.938800+0000","last_fullsized":"2024-09-11T23:16:35.938800+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:57:40.955645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:36.226559+0000","last_change":"2024-09-11T23:16:25.900461+0000","last_active":"2024-09-11T23:16:36.226559+0000","last_peered":"2024-09-11T23:16:36.226559+0000","last_clean":"2024-09-11T23:16:36.226559+0000","last_became_active":"2024-09-11T23:16:25.900174+0000","last_became_peered":"2024-09-11T23:16:25.900174+0000","last_unstale":"2024-09-11T23:16:36.226559+0000","last_undegraded":"2024-09-11T23:16:36.226559+0000","last_fullsized":"2024-09-11T23:16:36.226559+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:51:50.953198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.940957+0000","last_change":"2024-09-11T23:16:25.907569+0000","last_active":"2024-09-11T23:16:35.940957+0000","last_peered":"2024-09-11T23:16:35.940957+0000","last_clean":"2024-09-11T23:16:35.940957+0000","last_became_active":"2024-09-11T23:16:25.907091+0000","last_became_peered":"2024-09-11T23:16:25.907091+0000","last_unstale":"2024-09-11T23:16:35.940957+0000","last_undegraded":"2024-09-11T23:16:35.940957+0000","last_fullsized":"2024-09-11T23:16:35.940957+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:53:45.961167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:36.226935+0000","last_change":"2024-09-11T23:16:25.903246+0000","last_active":"2024-09-11T23:16:36.226935+0000","last_peered":"2024-09-11T23:16:36.226935+0000","last_clean":"2024-09-11T23:16:36.226935+0000","last_became_active":"2024-09-11T23:16:25.902907+0000","last_became_peered":"2024-09-11T23:16:25.902907+0000","last_unstale":"2024-09-11T23:16:36.226935+0000","last_undegraded":"2024-09-11T23:16:36.226935+0000","last_fullsized":"2024-09-11T23:16:36.226935+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:20:38.868384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.939763+0000","last_change":"2024-09-11T23:16:25.906298+0000","last_active":"2024-09-11T23:16:35.939763+0000","last_peered":"2024-09-11T23:16:35.939763+0000","last_clean":"2024-09-11T23:16:35.939763+0000","last_became_active":"2024-09-11T23:16:25.905350+0000","last_became_peered":"2024-09-11T23:16:25.905350+0000","last_unstale":"2024-09-11T23:16:35.939763+0000","last_undegraded":"2024-09-11T23:16:35.939763+0000","last_fullsized":"2024-09-11T23:16:35.939763+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:04:22.122514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.941073+0000","last_change":"2024-09-11T23:16:25.902114+0000","last_active":"2024-09-11T23:16:35.941073+0000","last_peered":"2024-09-11T23:16:35.941073+0000","last_clean":"2024-09-11T23:16:35.941073+0000","last_became_active":"2024-09-11T23:16:25.901742+0000","last_became_peered":"2024-09-11T23:16:25.901742+0000","last_unstale":"2024-09-11T23:16:35.941073+0000","last_undegraded":"2024-09-11T23:16:35.941073+0000","last_fullsized":"2024-09-11T23:16:35.941073+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:02:10.640801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.938325+0000","last_change":"2024-09-11T23:16:25.898117+0000","last_active":"2024-09-11T23:16:35.938325+0000","last_peered":"2024-09-11T23:16:35.938325+0000","last_clean":"2024-09-11T23:16:35.938325+0000","last_became_active":"2024-09-11T23:16:25.897818+0000","last_became_peered":"2024-09-11T23:16:25.897818+0000","last_unstale":"2024-09-11T23:16:35.938325+0000","last_undegraded":"2024-09-11T23:16:35.938325+0000","last_fullsized":"2024-09-11T23:16:35.938325+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:27:37.175471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:36.226573+0000","last_change":"2024-09-11T23:16:25.905600+0000","last_active":"2024-09-11T23:16:36.226573+0000","last_peered":"2024-09-11T23:16:36.226573+0000","last_clean":"2024-09-11T23:16:36.226573+0000","last_became_active":"2024-09-11T23:16:25.903924+0000","last_became_peered":"2024-09-11T23:16:25.903924+0000","last_unstale":"2024-09-11T23:16:36.226573+0000","last_undegraded":"2024-09-11T23:16:36.226573+0000","last_fullsized":"2024-09-11T23:16:36.226573+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:20:29.929079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.939290+0000","last_change":"2024-09-11T23:16:25.904763+0000","last_active":"2024-09-11T23:16:35.939290+0000","last_peered":"2024-09-11T23:16:35.939290+0000","last_clean":"2024-09-11T23:16:35.939290+0000","last_became_active":"2024-09-11T23:16:25.904406+0000","last_became_peered":"2024-09-11T23:16:25.904406+0000","last_unstale":"2024-09-11T23:16:35.939290+0000","last_undegraded":"2024-09-11T23:16:35.939290+0000","last_fullsized":"2024-09-11T23:16:35.939290+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:27:38.918017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.939346+0000","last_change":"2024-09-11T23:16:25.906573+0000","last_active":"2024-09-11T23:16:35.939346+0000","last_peered":"2024-09-11T23:16:35.939346+0000","last_clean":"2024-09-11T23:16:35.939346+0000","last_became_active":"2024-09-11T23:16:25.904771+0000","last_became_peered":"2024-09-11T23:16:25.904771+0000","last_unstale":"2024-09-11T23:16:35.939346+0000","last_undegraded":"2024-09-11T23:16:35.939346+0000","last_fullsized":"2024-09-11T23:16:35.939346+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:04:17.090648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.942191+0000","last_change":"2024-09-11T23:16:25.902123+0000","last_active":"2024-09-11T23:16:35.942191+0000","last_peered":"2024-09-11T23:16:35.942191+0000","last_clean":"2024-09-11T23:16:35.942191+0000","last_became_active":"2024-09-11T23:16:25.901756+0000","last_became_peered":"2024-09-11T23:16:25.901756+0000","last_unstale":"2024-09-11T23:16:35.942191+0000","last_undegraded":"2024-09-11T23:16:35.942191+0000","last_fullsized":"2024-09-11T23:16:35.942191+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:17:37.074976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.944098+0000","last_change":"2024-09-11T23:16:25.904217+0000","last_active":"2024-09-11T23:16:35.944098+0000","last_peered":"2024-09-11T23:16:35.944098+0000","last_clean":"2024-09-11T23:16:35.944098+0000","last_became_active":"2024-09-11T23:16:25.903522+0000","last_became_peered":"2024-09-11T23:16:25.903522+0000","last_unstale":"2024-09-11T23:16:35.944098+0000","last_undegraded":"2024-09-11T23:16:35.944098+0000","last_fullsized":"2024-09-11T23:16:35.944098+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:29:46.634475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"83'5","reported_seq":80,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:17:28.052380+0000","last_change":"2024-09-11T23:16:23.793696+0000","last_active":"2024-09-11T23:17:28.052380+0000","last_peered":"2024-09-11T23:17:28.052380+0000","last_clean":"2024-09-11T23:17:28.052380+0000","last_became_active":"2024-09-11T23:16:23.793062+0000","last_became_peered":"2024-09-11T23:16:23.793062+0000","last_unstale":"2024-09-11T23:17:28.052380+0000","last_undegraded":"2024-09-11T23:17:28.052380+0000","last_fullsized":"2024-09-11T23:17:28.052380+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:00:25.655817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":53,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.941878+0000","last_change":"2024-09-11T23:16:25.906673+0000","last_active":"2024-09-11T23:16:35.941878+0000","last_peered":"2024-09-11T23:16:35.941878+0000","last_clean":"2024-09-11T23:16:35.941878+0000","last_became_active":"2024-09-11T23:16:25.906239+0000","last_became_peered":"2024-09-11T23:16:25.906239+0000","last_unstale":"2024-09-11T23:16:35.941878+0000","last_undegraded":"2024-09-11T23:16:35.941878+0000","last_fullsized":"2024-09-11T23:16:35.941878+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:15:54.907269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"85'7","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:36.973576+0000","last_change":"2024-09-11T23:16:23.793302+0000","last_active":"2024-09-11T23:16:36.973576+0000","last_peered":"2024-09-11T23:16:36.973576+0000","last_clean":"2024-09-11T23:16:36.973576+0000","last_became_active":"2024-09-11T23:16:23.792874+0000","last_became_peered":"2024-09-11T23:16:23.792874+0000","last_unstale":"2024-09-11T23:16:36.973576+0000","last_undegraded":"2024-09-11T23:16:36.973576+0000","last_fullsized":"2024-09-11T23:16:36.973576+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:52:59.558223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"85'3","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.937984+0000","last_change":"2024-09-11T23:16:23.792999+0000","last_active":"2024-09-11T23:16:35.937984+0000","last_peered":"2024-09-11T23:16:35.937984+0000","last_clean":"2024-09-11T23:16:35.937984+0000","last_became_active":"2024-09-11T23:16:23.792630+0000","last_became_peered":"2024-09-11T23:16:23.792630+0000","last_unstale":"2024-09-11T23:16:35.937984+0000","last_undegraded":"2024-09-11T23:16:35.937984+0000","last_fullsized":"2024-09-11T23:16:35.937984+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:09:28.001060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.943722+0000","last_change":"2024-09-11T23:16:35.943722+0000","last_active":"2024-09-11T23:16:35.943722+0000","last_peered":"2024-09-11T23:16:35.943722+0000","last_clean":"2024-09-11T23:16:35.943722+0000","last_became_active":"2024-09-11T23:16:35.943355+0000","last_became_peered":"2024-09-11T23:16:35.943355+0000","last_unstale":"2024-09-11T23:16:35.943722+0000","last_undegraded":"2024-09-11T23:16:35.943722+0000","last_fullsized":"2024-09-11T23:16:35.943722+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:26:22.769824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,6],"acting":[9,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"85'3","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:37.978674+0000","last_change":"2024-09-11T23:16:23.797945+0000","last_active":"2024-09-11T23:16:37.978674+0000","last_peered":"2024-09-11T23:16:37.978674+0000","last_clean":"2024-09-11T23:16:37.978674+0000","last_became_active":"2024-09-11T23:16:23.797572+0000","last_became_peered":"2024-09-11T23:16:23.797572+0000","last_unstale":"2024-09-11T23:16:37.978674+0000","last_undegraded":"2024-09-11T23:16:37.978674+0000","last_fullsized":"2024-09-11T23:16:37.978674+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:04:59.017668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.939247+0000","last_change":"2024-09-11T23:16:25.891587+0000","last_active":"2024-09-11T23:16:35.939247+0000","last_peered":"2024-09-11T23:16:35.939247+0000","last_clean":"2024-09-11T23:16:35.939247+0000","last_became_active":"2024-09-11T23:16:25.891190+0000","last_became_peered":"2024-09-11T23:16:25.891190+0000","last_unstale":"2024-09-11T23:16:35.939247+0000","last_undegraded":"2024-09-11T23:16:35.939247+0000","last_fullsized":"2024-09-11T23:16:35.939247+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:57:05.177843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"85'4","reported_seq":120,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:17:25.047896+0000","last_change":"2024-09-11T23:16:23.790129+0000","last_active":"2024-09-11T23:17:25.047896+0000","last_peered":"2024-09-11T23:17:25.047896+0000","last_clean":"2024-09-11T23:17:25.047896+0000","last_became_active":"2024-09-11T23:16:23.789720+0000","last_became_peered":"2024-09-11T23:16:23.789720+0000","last_unstale":"2024-09-11T23:17:25.047896+0000","last_undegraded":"2024-09-11T23:17:25.047896+0000","last_fullsized":"2024-09-11T23:17:25.047896+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:33:58.388513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":94,"num_read_kb":94,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.939104+0000","last_change":"2024-09-11T23:16:25.901014+0000","last_active":"2024-09-11T23:16:35.939104+0000","last_peered":"2024-09-11T23:16:35.939104+0000","last_clean":"2024-09-11T23:16:35.939104+0000","last_became_active":"2024-09-11T23:16:25.900280+0000","last_became_peered":"2024-09-11T23:16:25.900280+0000","last_unstale":"2024-09-11T23:16:35.939104+0000","last_undegraded":"2024-09-11T23:16:35.939104+0000","last_fullsized":"2024-09-11T23:16:35.939104+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:14:03.662204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"85'4","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:37.978517+0000","last_change":"2024-09-11T23:16:23.797944+0000","last_active":"2024-09-11T23:16:37.978517+0000","last_peered":"2024-09-11T23:16:37.978517+0000","last_clean":"2024-09-11T23:16:37.978517+0000","last_became_active":"2024-09-11T23:16:23.797520+0000","last_became_peered":"2024-09-11T23:16:23.797520+0000","last_unstale":"2024-09-11T23:16:37.978517+0000","last_undegraded":"2024-09-11T23:16:37.978517+0000","last_fullsized":"2024-09-11T23:16:37.978517+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:29:06.957518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:36.226839+0000","last_change":"2024-09-11T23:16:25.905379+0000","last_active":"2024-09-11T23:16:36.226839+0000","last_peered":"2024-09-11T23:16:36.226839+0000","last_clean":"2024-09-11T23:16:36.226839+0000","last_became_active":"2024-09-11T23:16:25.905085+0000","last_became_peered":"2024-09-11T23:16:25.905085+0000","last_unstale":"2024-09-11T23:16:36.226839+0000","last_undegraded":"2024-09-11T23:16:36.226839+0000","last_fullsized":"2024-09-11T23:16:36.226839+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:22:35.129736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"85'5","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:36.972662+0000","last_change":"2024-09-11T23:16:24.075556+0000","last_active":"2024-09-11T23:16:36.972662+0000","last_peered":"2024-09-11T23:16:36.972662+0000","last_clean":"2024-09-11T23:16:36.972662+0000","last_became_active":"2024-09-11T23:16:24.074783+0000","last_became_peered":"2024-09-11T23:16:24.074783+0000","last_unstale":"2024-09-11T23:16:36.972662+0000","last_undegraded":"2024-09-11T23:16:36.972662+0000","last_fullsized":"2024-09-11T23:16:36.972662+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:55:09.962342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.938922+0000","last_change":"2024-09-11T23:16:25.907720+0000","last_active":"2024-09-11T23:16:35.938922+0000","last_peered":"2024-09-11T23:16:35.938922+0000","last_clean":"2024-09-11T23:16:35.938922+0000","last_became_active":"2024-09-11T23:16:25.907335+0000","last_became_peered":"2024-09-11T23:16:25.907335+0000","last_unstale":"2024-09-11T23:16:35.938922+0000","last_undegraded":"2024-09-11T23:16:35.938922+0000","last_fullsized":"2024-09-11T23:16:35.938922+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:46:14.519384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"85'30","reported_seq":155,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:17:25.059375+0000","last_change":"2024-09-11T23:16:23.794321+0000","last_active":"2024-09-11T23:17:25.059375+0000","last_peered":"2024-09-11T23:17:25.059375+0000","last_clean":"2024-09-11T23:17:25.059375+0000","last_became_active":"2024-09-11T23:16:23.793955+0000","last_became_peered":"2024-09-11T23:16:23.793955+0000","last_unstale":"2024-09-11T23:17:25.059375+0000","last_undegraded":"2024-09-11T23:17:25.059375+0000","last_fullsized":"2024-09-11T23:17:25.059375+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:47:46.442656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59764,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":103,"num_read_kb":109,"num_write":32,"num_write_kb":69,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.945323+0000","last_change":"2024-09-11T23:16:25.896598+0000","last_active":"2024-09-11T23:16:35.945323+0000","last_peered":"2024-09-11T23:16:35.945323+0000","last_clean":"2024-09-11T23:16:35.945323+0000","last_became_active":"2024-09-11T23:16:25.896293+0000","last_became_peered":"2024-09-11T23:16:25.896293+0000","last_unstale":"2024-09-11T23:16:35.945323+0000","last_undegraded":"2024-09-11T23:16:35.945323+0000","last_fullsized":"2024-09-11T23:16:35.945323+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:04:11.489644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"85'9","reported_seq":33,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:38.985116+0000","last_change":"2024-09-11T23:16:23.791486+0000","last_active":"2024-09-11T23:16:38.985116+0000","last_peered":"2024-09-11T23:16:38.985116+0000","last_clean":"2024-09-11T23:16:38.985116+0000","last_became_active":"2024-09-11T23:16:23.790786+0000","last_became_peered":"2024-09-11T23:16:23.790786+0000","last_unstale":"2024-09-11T23:16:38.985116+0000","last_undegraded":"2024-09-11T23:16:38.985116+0000","last_fullsized":"2024-09-11T23:16:38.985116+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:03:52.742560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.937975+0000","last_change":"2024-09-11T23:16:25.897382+0000","last_active":"2024-09-11T23:16:35.937975+0000","last_peered":"2024-09-11T23:16:35.937975+0000","last_clean":"2024-09-11T23:16:35.937975+0000","last_became_active":"2024-09-11T23:16:25.897073+0000","last_became_peered":"2024-09-11T23:16:25.897073+0000","last_unstale":"2024-09-11T23:16:35.937975+0000","last_undegraded":"2024-09-11T23:16:35.937975+0000","last_fullsized":"2024-09-11T23:16:35.937975+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:15:59.780100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"85'5","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:37.977834+0000","last_change":"2024-09-11T23:16:23.793886+0000","last_active":"2024-09-11T23:16:37.977834+0000","last_peered":"2024-09-11T23:16:37.977834+0000","last_clean":"2024-09-11T23:16:37.977834+0000","last_became_active":"2024-09-11T23:16:23.793066+0000","last_became_peered":"2024-09-11T23:16:23.793066+0000","last_unstale":"2024-09-11T23:16:37.977834+0000","last_undegraded":"2024-09-11T23:16:37.977834+0000","last_fullsized":"2024-09-11T23:16:37.977834+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:07:03.882546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.942540+0000","last_change":"2024-09-11T23:16:25.904575+0000","last_active":"2024-09-11T23:16:35.942540+0000","last_peered":"2024-09-11T23:16:35.942540+0000","last_clean":"2024-09-11T23:16:35.942540+0000","last_became_active":"2024-09-11T23:16:25.904224+0000","last_became_peered":"2024-09-11T23:16:25.904224+0000","last_unstale":"2024-09-11T23:16:35.942540+0000","last_undegraded":"2024-09-11T23:16:35.942540+0000","last_fullsized":"2024-09-11T23:16:35.942540+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:42:13.036931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"85'5","reported_seq":76,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:17:24.047095+0000","last_change":"2024-09-11T23:16:23.796073+0000","last_active":"2024-09-11T23:17:24.047095+0000","last_peered":"2024-09-11T23:17:24.047095+0000","last_clean":"2024-09-11T23:17:24.047095+0000","last_became_active":"2024-09-11T23:16:23.794850+0000","last_became_peered":"2024-09-11T23:16:23.794850+0000","last_unstale":"2024-09-11T23:17:24.047095+0000","last_undegraded":"2024-09-11T23:17:24.047095+0000","last_fullsized":"2024-09-11T23:17:24.047095+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:57:22.361662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":50,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.940823+0000","last_change":"2024-09-11T23:16:25.908700+0000","last_active":"2024-09-11T23:16:35.940823+0000","last_peered":"2024-09-11T23:16:35.940823+0000","last_clean":"2024-09-11T23:16:35.940823+0000","last_became_active":"2024-09-11T23:16:25.908373+0000","last_became_peered":"2024-09-11T23:16:25.908373+0000","last_unstale":"2024-09-11T23:16:35.940823+0000","last_undegraded":"2024-09-11T23:16:35.940823+0000","last_fullsized":"2024-09-11T23:16:35.940823+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:20:18.536079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"85'17","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:37.976750+0000","last_change":"2024-09-11T23:16:24.075927+0000","last_active":"2024-09-11T23:16:37.976750+0000","last_peered":"2024-09-11T23:16:37.976750+0000","last_clean":"2024-09-11T23:16:37.976750+0000","last_became_active":"2024-09-11T23:16:24.075527+0000","last_became_peered":"2024-09-11T23:16:24.075527+0000","last_unstale":"2024-09-11T23:16:37.976750+0000","last_undegraded":"2024-09-11T23:16:37.976750+0000","last_fullsized":"2024-09-11T23:16:37.976750+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:27:54.622059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.947475+0000","last_change":"2024-09-11T23:16:35.947475+0000","last_active":"2024-09-11T23:16:35.947475+0000","last_peered":"2024-09-11T23:16:35.947475+0000","last_clean":"2024-09-11T23:16:35.947475+0000","last_became_active":"2024-09-11T23:16:35.947028+0000","last_became_peered":"2024-09-11T23:16:35.947028+0000","last_unstale":"2024-09-11T23:16:35.947475+0000","last_undegraded":"2024-09-11T23:16:35.947475+0000","last_fullsized":"2024-09-11T23:16:35.947475+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:18:13.479554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'2","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:37.950468+0000","last_change":"2024-09-11T23:16:23.793298+0000","last_active":"2024-09-11T23:16:37.950468+0000","last_peered":"2024-09-11T23:16:37.950468+0000","last_clean":"2024-09-11T23:16:37.950468+0000","last_became_active":"2024-09-11T23:16:23.792947+0000","last_became_peered":"2024-09-11T23:16:23.792947+0000","last_unstale":"2024-09-11T23:16:37.950468+0000","last_undegraded":"2024-09-11T23:16:37.950468+0000","last_fullsized":"2024-09-11T23:16:37.950468+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:32:21.294560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.941169+0000","last_change":"2024-09-11T23:16:25.902348+0000","last_active":"2024-09-11T23:16:35.941169+0000","last_peered":"2024-09-11T23:16:35.941169+0000","last_clean":"2024-09-11T23:16:35.941169+0000","last_became_active":"2024-09-11T23:16:25.901686+0000","last_became_peered":"2024-09-11T23:16:25.901686+0000","last_unstale":"2024-09-11T23:16:35.941169+0000","last_undegraded":"2024-09-11T23:16:35.941169+0000","last_fullsized":"2024-09-11T23:16:35.941169+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:00:15.543003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"85'5","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:37.975894+0000","last_change":"2024-09-11T23:16:23.790984+0000","last_active":"2024-09-11T23:16:37.975894+0000","last_peered":"2024-09-11T23:16:37.975894+0000","last_clean":"2024-09-11T23:16:37.975894+0000","last_became_active":"2024-09-11T23:16:23.790569+0000","last_became_peered":"2024-09-11T23:16:23.790569+0000","last_unstale":"2024-09-11T23:16:37.975894+0000","last_undegraded":"2024-09-11T23:16:37.975894+0000","last_fullsized":"2024-09-11T23:16:37.975894+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:57:39.675793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.941423+0000","last_change":"2024-09-11T23:16:25.899249+0000","last_active":"2024-09-11T23:16:35.941423+0000","last_peered":"2024-09-11T23:16:35.941423+0000","last_clean":"2024-09-11T23:16:35.941423+0000","last_became_active":"2024-09-11T23:16:25.898914+0000","last_became_peered":"2024-09-11T23:16:25.898914+0000","last_unstale":"2024-09-11T23:16:35.941423+0000","last_undegraded":"2024-09-11T23:16:35.941423+0000","last_fullsized":"2024-09-11T23:16:35.941423+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:17:07.545770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'3","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:37.977945+0000","last_change":"2024-09-11T23:16:23.787387+0000","last_active":"2024-09-11T23:16:37.977945+0000","last_peered":"2024-09-11T23:16:37.977945+0000","last_clean":"2024-09-11T23:16:37.977945+0000","last_became_active":"2024-09-11T23:16:23.787014+0000","last_became_peered":"2024-09-11T23:16:23.787014+0000","last_unstale":"2024-09-11T23:16:37.977945+0000","last_undegraded":"2024-09-11T23:16:37.977945+0000","last_fullsized":"2024-09-11T23:16:37.977945+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:45:52.004253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"85'1","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.941159+0000","last_change":"2024-09-11T23:16:25.908304+0000","last_active":"2024-09-11T23:16:35.941159+0000","last_peered":"2024-09-11T23:16:35.941159+0000","last_clean":"2024-09-11T23:16:35.941159+0000","last_became_active":"2024-09-11T23:16:25.907900+0000","last_became_peered":"2024-09-11T23:16:25.907900+0000","last_unstale":"2024-09-11T23:16:35.941159+0000","last_undegraded":"2024-09-11T23:16:35.941159+0000","last_fullsized":"2024-09-11T23:16:35.941159+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:43:26.247682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"85'6","reported_seq":81,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:17:27.051370+0000","last_change":"2024-09-11T23:16:24.074610+0000","last_active":"2024-09-11T23:17:27.051370+0000","last_peered":"2024-09-11T23:17:27.051370+0000","last_clean":"2024-09-11T23:17:27.051370+0000","last_became_active":"2024-09-11T23:16:24.074210+0000","last_became_peered":"2024-09-11T23:16:24.074210+0000","last_unstale":"2024-09-11T23:17:27.051370+0000","last_undegraded":"2024-09-11T23:17:27.051370+0000","last_fullsized":"2024-09-11T23:17:27.051370+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:09:37.167050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1563,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":53,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.941591+0000","last_change":"2024-09-11T23:16:25.903120+0000","last_active":"2024-09-11T23:16:35.941591+0000","last_peered":"2024-09-11T23:16:35.941591+0000","last_clean":"2024-09-11T23:16:35.941591+0000","last_became_active":"2024-09-11T23:16:25.902770+0000","last_became_peered":"2024-09-11T23:16:25.902770+0000","last_unstale":"2024-09-11T23:16:35.941591+0000","last_undegraded":"2024-09-11T23:16:35.941591+0000","last_fullsized":"2024-09-11T23:16:35.941591+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:03:30.041088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":346,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.941279+0000","last_change":"2024-09-11T23:15:13.115055+0000","last_active":"2024-09-11T23:16:35.941279+0000","last_peered":"2024-09-11T23:16:35.941279+0000","last_clean":"2024-09-11T23:16:35.941279+0000","last_became_active":"2024-09-11T23:15:13.114480+0000","last_became_peered":"2024-09-11T23:15:13.114480+0000","last_unstale":"2024-09-11T23:16:35.941279+0000","last_undegraded":"2024-09-11T23:16:35.941279+0000","last_fullsized":"2024-09-11T23:16:35.941279+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:11:50.495799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:11:50.495799+0000","last_clean_scrub_stamp":"2024-09-11T23:11:50.495799+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:15:53.681230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"85'6","reported_seq":31,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:38.985260+0000","last_change":"2024-09-11T23:16:23.793996+0000","last_active":"2024-09-11T23:16:38.985260+0000","last_peered":"2024-09-11T23:16:38.985260+0000","last_clean":"2024-09-11T23:16:38.985260+0000","last_became_active":"2024-09-11T23:16:23.793681+0000","last_became_peered":"2024-09-11T23:16:23.793681+0000","last_unstale":"2024-09-11T23:16:38.985260+0000","last_undegraded":"2024-09-11T23:16:38.985260+0000","last_fullsized":"2024-09-11T23:16:38.985260+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:52:50.463623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:36.227084+0000","last_change":"2024-09-11T23:16:25.905133+0000","last_active":"2024-09-11T23:16:36.227084+0000","last_peered":"2024-09-11T23:16:36.227084+0000","last_clean":"2024-09-11T23:16:36.227084+0000","last_became_active":"2024-09-11T23:16:25.904819+0000","last_became_peered":"2024-09-11T23:16:25.904819+0000","last_unstale":"2024-09-11T23:16:36.227084+0000","last_undegraded":"2024-09-11T23:16:36.227084+0000","last_fullsized":"2024-09-11T23:16:36.227084+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:51:09.664343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"85'4","reported_seq":32,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:36.231286+0000","last_change":"2024-09-11T23:16:36.228806+0000","last_active":"2024-09-11T23:16:36.231286+0000","last_peered":"2024-09-11T23:16:36.231286+0000","last_clean":"2024-09-11T23:16:36.231286+0000","last_became_active":"2024-09-11T23:16:36.228311+0000","last_became_peered":"2024-09-11T23:16:36.228311+0000","last_unstale":"2024-09-11T23:16:36.231286+0000","last_undegraded":"2024-09-11T23:16:36.231286+0000","last_fullsized":"2024-09-11T23:16:36.231286+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:42:32.856691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4],"acting":[7,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.937900+0000","last_change":"2024-09-11T23:16:25.898935+0000","last_active":"2024-09-11T23:16:35.937900+0000","last_peered":"2024-09-11T23:16:35.937900+0000","last_clean":"2024-09-11T23:16:35.937900+0000","last_became_active":"2024-09-11T23:16:25.898618+0000","last_became_peered":"2024-09-11T23:16:25.898618+0000","last_unstale":"2024-09-11T23:16:35.937900+0000","last_undegraded":"2024-09-11T23:16:35.937900+0000","last_fullsized":"2024-09-11T23:16:35.937900+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:13:29.104911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"85'5","reported_seq":79,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:17:28.051750+0000","last_change":"2024-09-11T23:16:24.073613+0000","last_active":"2024-09-11T23:17:28.051750+0000","last_peered":"2024-09-11T23:17:28.051750+0000","last_clean":"2024-09-11T23:17:28.051750+0000","last_became_active":"2024-09-11T23:16:24.073262+0000","last_became_peered":"2024-09-11T23:16:24.073262+0000","last_unstale":"2024-09-11T23:17:28.051750+0000","last_undegraded":"2024-09-11T23:17:28.051750+0000","last_fullsized":"2024-09-11T23:17:28.051750+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:31:58.653657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":52,"num_read_kb":52,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.941067+0000","last_change":"2024-09-11T23:16:25.903725+0000","last_active":"2024-09-11T23:16:35.941067+0000","last_peered":"2024-09-11T23:16:35.941067+0000","last_clean":"2024-09-11T23:16:35.941067+0000","last_became_active":"2024-09-11T23:16:25.903375+0000","last_became_peered":"2024-09-11T23:16:25.903375+0000","last_unstale":"2024-09-11T23:16:35.941067+0000","last_undegraded":"2024-09-11T23:16:35.941067+0000","last_fullsized":"2024-09-11T23:16:35.941067+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:55:02.779575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"85'5","reported_seq":78,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:17:27.050537+0000","last_change":"2024-09-11T23:16:23.793932+0000","last_active":"2024-09-11T23:17:27.050537+0000","last_peered":"2024-09-11T23:17:27.050537+0000","last_clean":"2024-09-11T23:17:27.050537+0000","last_became_active":"2024-09-11T23:16:23.793111+0000","last_became_peered":"2024-09-11T23:16:23.793111+0000","last_unstale":"2024-09-11T23:17:27.050537+0000","last_undegraded":"2024-09-11T23:17:27.050537+0000","last_fullsized":"2024-09-11T23:17:27.050537+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:14:09.087222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":51,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.942165+0000","last_change":"2024-09-11T23:16:25.905020+0000","last_active":"2024-09-11T23:16:35.942165+0000","last_peered":"2024-09-11T23:16:35.942165+0000","last_clean":"2024-09-11T23:16:35.942165+0000","last_became_active":"2024-09-11T23:16:25.904675+0000","last_became_peered":"2024-09-11T23:16:25.904675+0000","last_unstale":"2024-09-11T23:16:35.942165+0000","last_undegraded":"2024-09-11T23:16:35.942165+0000","last_fullsized":"2024-09-11T23:16:35.942165+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:13:57.388773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"85'1","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.939622+0000","last_change":"2024-09-11T23:16:24.073630+0000","last_active":"2024-09-11T23:16:35.939622+0000","last_peered":"2024-09-11T23:16:35.939622+0000","last_clean":"2024-09-11T23:16:35.939622+0000","last_became_active":"2024-09-11T23:16:24.073301+0000","last_became_peered":"2024-09-11T23:16:24.073301+0000","last_unstale":"2024-09-11T23:16:35.939622+0000","last_undegraded":"2024-09-11T23:16:35.939622+0000","last_fullsized":"2024-09-11T23:16:35.939622+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:41:59.374581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"85'1","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.942393+0000","last_change":"2024-09-11T23:16:25.894124+0000","last_active":"2024-09-11T23:16:35.942393+0000","last_peered":"2024-09-11T23:16:35.942393+0000","last_clean":"2024-09-11T23:16:35.942393+0000","last_became_active":"2024-09-11T23:16:25.893751+0000","last_became_peered":"2024-09-11T23:16:25.893751+0000","last_unstale":"2024-09-11T23:16:35.942393+0000","last_undegraded":"2024-09-11T23:16:35.942393+0000","last_fullsized":"2024-09-11T23:16:35.942393+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:00:28.265206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"85'4","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:37.976960+0000","last_change":"2024-09-11T23:16:24.073700+0000","last_active":"2024-09-11T23:16:37.976960+0000","last_peered":"2024-09-11T23:16:37.976960+0000","last_clean":"2024-09-11T23:16:37.976960+0000","last_became_active":"2024-09-11T23:16:24.073283+0000","last_became_peered":"2024-09-11T23:16:24.073283+0000","last_unstale":"2024-09-11T23:16:37.976960+0000","last_undegraded":"2024-09-11T23:16:37.976960+0000","last_fullsized":"2024-09-11T23:16:37.976960+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:41:43.536834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.938154+0000","last_change":"2024-09-11T23:16:25.900532+0000","last_active":"2024-09-11T23:16:35.938154+0000","last_peered":"2024-09-11T23:16:35.938154+0000","last_clean":"2024-09-11T23:16:35.938154+0000","last_became_active":"2024-09-11T23:16:25.899466+0000","last_became_peered":"2024-09-11T23:16:25.899466+0000","last_unstale":"2024-09-11T23:16:35.938154+0000","last_undegraded":"2024-09-11T23:16:35.938154+0000","last_fullsized":"2024-09-11T23:16:35.938154+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:10:01.152323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"85'14","reported_seq":36,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:50.914551+0000","last_change":"2024-09-11T23:16:23.788789+0000","last_active":"2024-09-11T23:16:50.914551+0000","last_peered":"2024-09-11T23:16:50.914551+0000","last_clean":"2024-09-11T23:16:50.914551+0000","last_became_active":"2024-09-11T23:16:23.788422+0000","last_became_peered":"2024-09-11T23:16:23.788422+0000","last_unstale":"2024-09-11T23:16:50.914551+0000","last_undegraded":"2024-09-11T23:16:50.914551+0000","last_fullsized":"2024-09-11T23:16:50.914551+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:09:14.310205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:36.227116+0000","last_change":"2024-09-11T23:16:25.895511+0000","last_active":"2024-09-11T23:16:36.227116+0000","last_peered":"2024-09-11T23:16:36.227116+0000","last_clean":"2024-09-11T23:16:36.227116+0000","last_became_active":"2024-09-11T23:16:25.895143+0000","last_became_peered":"2024-09-11T23:16:25.895143+0000","last_unstale":"2024-09-11T23:16:36.227116+0000","last_undegraded":"2024-09-11T23:16:36.227116+0000","last_fullsized":"2024-09-11T23:16:36.227116+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:31:16.498371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"85'7","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:37.977151+0000","last_change":"2024-09-11T23:16:24.019064+0000","last_active":"2024-09-11T23:16:37.977151+0000","last_peered":"2024-09-11T23:16:37.977151+0000","last_clean":"2024-09-11T23:16:37.977151+0000","last_became_active":"2024-09-11T23:16:24.018658+0000","last_became_peered":"2024-09-11T23:16:24.018658+0000","last_unstale":"2024-09-11T23:16:37.977151+0000","last_undegraded":"2024-09-11T23:16:37.977151+0000","last_fullsized":"2024-09-11T23:16:37.977151+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:17:41.009507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:36.227194+0000","last_change":"2024-09-11T23:16:25.903553+0000","last_active":"2024-09-11T23:16:36.227194+0000","last_peered":"2024-09-11T23:16:36.227194+0000","last_clean":"2024-09-11T23:16:36.227194+0000","last_became_active":"2024-09-11T23:16:25.903221+0000","last_became_peered":"2024-09-11T23:16:25.903221+0000","last_unstale":"2024-09-11T23:16:36.227194+0000","last_undegraded":"2024-09-11T23:16:36.227194+0000","last_fullsized":"2024-09-11T23:16:36.227194+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:16:51.872517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"83'2","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.938847+0000","last_change":"2024-09-11T23:16:24.073784+0000","last_active":"2024-09-11T23:16:35.938847+0000","last_peered":"2024-09-11T23:16:35.938847+0000","last_clean":"2024-09-11T23:16:35.938847+0000","last_became_active":"2024-09-11T23:16:24.073406+0000","last_became_peered":"2024-09-11T23:16:24.073406+0000","last_unstale":"2024-09-11T23:16:35.938847+0000","last_undegraded":"2024-09-11T23:16:35.938847+0000","last_fullsized":"2024-09-11T23:16:35.938847+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:31:24.617025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.945715+0000","last_change":"2024-09-11T23:16:25.894878+0000","last_active":"2024-09-11T23:16:35.945715+0000","last_peered":"2024-09-11T23:16:35.945715+0000","last_clean":"2024-09-11T23:16:35.945715+0000","last_became_active":"2024-09-11T23:16:25.894543+0000","last_became_peered":"2024-09-11T23:16:25.894543+0000","last_unstale":"2024-09-11T23:16:35.945715+0000","last_undegraded":"2024-09-11T23:16:35.945715+0000","last_fullsized":"2024-09-11T23:16:35.945715+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:34:13.435754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"85'7","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:37.976060+0000","last_change":"2024-09-11T23:16:23.786502+0000","last_active":"2024-09-11T23:16:37.976060+0000","last_peered":"2024-09-11T23:16:37.976060+0000","last_clean":"2024-09-11T23:16:37.976060+0000","last_became_active":"2024-09-11T23:16:23.786182+0000","last_became_peered":"2024-09-11T23:16:23.786182+0000","last_unstale":"2024-09-11T23:16:37.976060+0000","last_undegraded":"2024-09-11T23:16:37.976060+0000","last_fullsized":"2024-09-11T23:16:37.976060+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:59:07.978244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.942309+0000","last_change":"2024-09-11T23:16:25.899625+0000","last_active":"2024-09-11T23:16:35.942309+0000","last_peered":"2024-09-11T23:16:35.942309+0000","last_clean":"2024-09-11T23:16:35.942309+0000","last_became_active":"2024-09-11T23:16:25.899265+0000","last_became_peered":"2024-09-11T23:16:25.899265+0000","last_unstale":"2024-09-11T23:16:35.942309+0000","last_undegraded":"2024-09-11T23:16:35.942309+0000","last_fullsized":"2024-09-11T23:16:35.942309+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:36:06.907622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"85'3","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:36.971236+0000","last_change":"2024-09-11T23:16:36.228991+0000","last_active":"2024-09-11T23:16:36.971236+0000","last_peered":"2024-09-11T23:16:36.971236+0000","last_clean":"2024-09-11T23:16:36.971236+0000","last_became_active":"2024-09-11T23:16:36.227701+0000","last_became_peered":"2024-09-11T23:16:36.227701+0000","last_unstale":"2024-09-11T23:16:36.971236+0000","last_undegraded":"2024-09-11T23:16:36.971236+0000","last_fullsized":"2024-09-11T23:16:36.971236+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:37:57.478572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:36.226661+0000","last_change":"2024-09-11T23:16:25.894803+0000","last_active":"2024-09-11T23:16:36.226661+0000","last_peered":"2024-09-11T23:16:36.226661+0000","last_clean":"2024-09-11T23:16:36.226661+0000","last_became_active":"2024-09-11T23:16:25.894429+0000","last_became_peered":"2024-09-11T23:16:25.894429+0000","last_unstale":"2024-09-11T23:16:36.226661+0000","last_undegraded":"2024-09-11T23:16:36.226661+0000","last_fullsized":"2024-09-11T23:16:36.226661+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:39:15.868910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"83'4","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.945644+0000","last_change":"2024-09-11T23:16:23.794271+0000","last_active":"2024-09-11T23:16:35.945644+0000","last_peered":"2024-09-11T23:16:35.945644+0000","last_clean":"2024-09-11T23:16:35.945644+0000","last_became_active":"2024-09-11T23:16:23.793919+0000","last_became_peered":"2024-09-11T23:16:23.793919+0000","last_unstale":"2024-09-11T23:16:35.945644+0000","last_undegraded":"2024-09-11T23:16:35.945644+0000","last_fullsized":"2024-09-11T23:16:35.945644+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:30:28.907312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.941192+0000","last_change":"2024-09-11T23:16:25.900969+0000","last_active":"2024-09-11T23:16:35.941192+0000","last_peered":"2024-09-11T23:16:35.941192+0000","last_clean":"2024-09-11T23:16:35.941192+0000","last_became_active":"2024-09-11T23:16:25.900425+0000","last_became_peered":"2024-09-11T23:16:25.900425+0000","last_unstale":"2024-09-11T23:16:35.941192+0000","last_undegraded":"2024-09-11T23:16:35.941192+0000","last_fullsized":"2024-09-11T23:16:35.941192+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:03:25.185773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.941358+0000","last_change":"2024-09-11T23:16:25.911605+0000","last_active":"2024-09-11T23:16:35.941358+0000","last_peered":"2024-09-11T23:16:35.941358+0000","last_clean":"2024-09-11T23:16:35.941358+0000","last_became_active":"2024-09-11T23:16:25.911180+0000","last_became_peered":"2024-09-11T23:16:25.911180+0000","last_unstale":"2024-09-11T23:16:35.941358+0000","last_undegraded":"2024-09-11T23:16:35.941358+0000","last_fullsized":"2024-09-11T23:16:35.941358+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:55:38.062606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"85'5","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:37.979007+0000","last_change":"2024-09-11T23:16:35.949528+0000","last_active":"2024-09-11T23:16:37.979007+0000","last_peered":"2024-09-11T23:16:37.979007+0000","last_clean":"2024-09-11T23:16:37.979007+0000","last_became_active":"2024-09-11T23:16:35.945679+0000","last_became_peered":"2024-09-11T23:16:35.945679+0000","last_unstale":"2024-09-11T23:16:37.979007+0000","last_undegraded":"2024-09-11T23:16:37.979007+0000","last_fullsized":"2024-09-11T23:16:37.979007+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:14:24.706044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.938792+0000","last_change":"2024-09-11T23:16:25.900526+0000","last_active":"2024-09-11T23:16:35.938792+0000","last_peered":"2024-09-11T23:16:35.938792+0000","last_clean":"2024-09-11T23:16:35.938792+0000","last_became_active":"2024-09-11T23:16:25.900148+0000","last_became_peered":"2024-09-11T23:16:25.900148+0000","last_unstale":"2024-09-11T23:16:35.938792+0000","last_undegraded":"2024-09-11T23:16:35.938792+0000","last_fullsized":"2024-09-11T23:16:35.938792+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:26:10.582473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"85'12","reported_seq":45,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:38.986047+0000","last_change":"2024-09-11T23:16:36.233273+0000","last_active":"2024-09-11T23:16:38.986047+0000","last_peered":"2024-09-11T23:16:38.986047+0000","last_clean":"2024-09-11T23:16:38.986047+0000","last_became_active":"2024-09-11T23:16:36.228358+0000","last_became_peered":"2024-09-11T23:16:36.228358+0000","last_unstale":"2024-09-11T23:16:38.986047+0000","last_undegraded":"2024-09-11T23:16:38.986047+0000","last_fullsized":"2024-09-11T23:16:38.986047+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:06:31.225795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:36.226734+0000","last_change":"2024-09-11T23:16:25.907874+0000","last_active":"2024-09-11T23:16:36.226734+0000","last_peered":"2024-09-11T23:16:36.226734+0000","last_clean":"2024-09-11T23:16:36.226734+0000","last_became_active":"2024-09-11T23:16:25.907491+0000","last_became_peered":"2024-09-11T23:16:25.907491+0000","last_unstale":"2024-09-11T23:16:36.226734+0000","last_undegraded":"2024-09-11T23:16:36.226734+0000","last_fullsized":"2024-09-11T23:16:36.226734+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:09:20.984249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"84'2","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:36.227306+0000","last_change":"2024-09-11T23:16:24.074355+0000","last_active":"2024-09-11T23:16:36.227306+0000","last_peered":"2024-09-11T23:16:36.227306+0000","last_clean":"2024-09-11T23:16:36.227306+0000","last_became_active":"2024-09-11T23:16:24.074047+0000","last_became_peered":"2024-09-11T23:16:24.074047+0000","last_unstale":"2024-09-11T23:16:36.227306+0000","last_undegraded":"2024-09-11T23:16:36.227306+0000","last_fullsized":"2024-09-11T23:16:36.227306+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:31:58.804420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:36.226393+0000","last_change":"2024-09-11T23:16:25.907701+0000","last_active":"2024-09-11T23:16:36.226393+0000","last_peered":"2024-09-11T23:16:36.226393+0000","last_clean":"2024-09-11T23:16:36.226393+0000","last_became_active":"2024-09-11T23:16:25.907379+0000","last_became_peered":"2024-09-11T23:16:25.907379+0000","last_unstale":"2024-09-11T23:16:36.226393+0000","last_undegraded":"2024-09-11T23:16:36.226393+0000","last_fullsized":"2024-09-11T23:16:36.226393+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:12:03.639055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"85'13","reported_seq":83,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:17:24.046157+0000","last_change":"2024-09-11T23:16:23.794293+0000","last_active":"2024-09-11T23:17:24.046157+0000","last_peered":"2024-09-11T23:17:24.046157+0000","last_clean":"2024-09-11T23:17:24.046157+0000","last_became_active":"2024-09-11T23:16:23.793974+0000","last_became_peered":"2024-09-11T23:16:23.793974+0000","last_unstale":"2024-09-11T23:17:24.046157+0000","last_undegraded":"2024-09-11T23:17:24.046157+0000","last_fullsized":"2024-09-11T23:17:24.046157+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:24:39.170188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":48,"num_read_kb":48,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66579,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":524,"num_read_kb":332,"num_write":241,"num_write_kb":133,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":199737,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":207,"ondisk_log_size":207,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":77,"seq":330712481818,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32108,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713044,"statfs":{"total":95995035648,"available":95962157056,"internally_reserved":0,"allocated":946176,"data_stored":449945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":70,"seq":300647710750,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28628,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716524,"statfs":{"total":95995035648,"available":95965720576,"internally_reserved":0,"allocated":1642496,"data_stored":1099763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":65,"seq":279172874276,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":966656,"data_stored":448900,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":57,"seq":244813135914,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":958464,"data_stored":448696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":50,"seq":214748364846,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28548,"kb_used_data":1524,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716604,"statfs":{"total":95995035648,"available":95965802496,"internally_reserved":0,"allocated":1560576,"data_stored":1041405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528370,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36272,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708880,"statfs":{"total":95995035648,"available":95957893120,"internally_reserved":0,"allocated":950272,"data_stored":448617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790009,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28560,"kb_used_data":1536,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716592,"statfs":{"total":95995035648,"available":95965790208,"internally_reserved":0,"allocated":1572864,"data_stored":1041083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920829,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36272,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708880,"statfs":{"total":95995035648,"available":95957893120,"internally_reserved":0,"allocated":950272,"data_stored":448478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182467,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40448,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704704,"statfs":{"total":95995035648,"available":95953616896,"internally_reserved":0,"allocated":966656,"data_stored":448442,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411400,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36352,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708800,"statfs":{"total":95995035648,"available":95957811200,"internally_reserved":0,"allocated":1032192,"data_stored":509907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574925,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1011712,"data_stored":509294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738450,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":970752,"data_stored":450349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":61516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":62129,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":61500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T23:17:32.502 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-11T23:17:32.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:17:32 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:17:32.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:17:32 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:17:32.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:17:32 smithi136 ceph-mon[35121]: from='client.14874 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T23:17:32.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:17:32 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:17:32.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:17:32 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:17:32.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:17:32 smithi136 ceph-mon[35121]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:17:32.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:17:32 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:17:32.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:17:32 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:17:32.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:17:32 smithi026 ceph-mon[31172]: from='client.14874 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T23:17:32.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:17:32 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:17:32.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:17:32 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:17:32.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:17:32 smithi026 ceph-mon[31172]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:17:32.759 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:17:32.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:17:32 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:17:32.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:17:32 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:17:32.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:17:32 smithi076 ceph-mon[35296]: from='client.14874 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T23:17:32.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:17:32 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:17:32.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:17:32 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:17:32.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:17:32 smithi076 ceph-mon[35296]: from='mgr.14148 172.21.15.26:0/1316924508' entity='mgr.x' 2024-09-11T23:17:33.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:33.195+0000 7fdf2ab5a640 1 -- 172.21.15.26:0/808311394 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf24101420 msgr2=0x7fdf2410d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:33.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:33.195+0000 7fdf2ab5a640 1 --2- 172.21.15.26:0/808311394 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf24101420 0x7fdf2410d950 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fdf1c009cd0 tx=0x7fdf1c02a8e0 comp rx=0 tx=0).stop 2024-09-11T23:17:33.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:33.195+0000 7fdf2ab5a640 1 -- 172.21.15.26:0/808311394 shutdown_connections 2024-09-11T23:17:33.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:33.195+0000 7fdf2ab5a640 1 --2- 172.21.15.26:0/808311394 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdf24100a80 0x7fdf24100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:33.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:33.195+0000 7fdf2ab5a640 1 --2- 172.21.15.26:0/808311394 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf24101420 0x7fdf2410d950 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:33.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:33.195+0000 7fdf2ab5a640 1 --2- 172.21.15.26:0/808311394 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdf24106a80 0x7fdf24106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:33.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:33.195+0000 7fdf2ab5a640 1 -- 172.21.15.26:0/808311394 >> 172.21.15.26:0/808311394 conn(0x7fdf240fc7a0 msgr2=0x7fdf240febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:17:33.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:33.195+0000 7fdf2ab5a640 1 -- 172.21.15.26:0/808311394 shutdown_connections 2024-09-11T23:17:33.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:33.195+0000 7fdf2ab5a640 1 -- 172.21.15.26:0/808311394 wait complete. 2024-09-11T23:17:33.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:33.196+0000 7fdf2ab5a640 1 Processor -- start 2024-09-11T23:17:33.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:33.196+0000 7fdf2ab5a640 1 -- start start 2024-09-11T23:17:33.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:33.196+0000 7fdf2ab5a640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdf24100a80 0x7fdf241a4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:33.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:33.196+0000 7fdf2ab5a640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdf24101420 0x7fdf241a4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:33.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:33.196+0000 7fdf2ab5a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf24106a80 0x7fdf2419eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:33.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:33.196+0000 7fdf2ab5a640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fdf24110ea0 con 0x7fdf24106a80 2024-09-11T23:17:33.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:33.196+0000 7fdf2ab5a640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fdf24110d20 con 0x7fdf24101420 2024-09-11T23:17:33.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:33.196+0000 7fdf2ab5a640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fdf24111020 con 0x7fdf24100a80 2024-09-11T23:17:33.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:33.196+0000 7fdf290f7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf24106a80 0x7fdf2419eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:33.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:33.196+0000 7fdf290f7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf24106a80 0x7fdf2419eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47306/0 (socket says 172.21.15.26:47306) 2024-09-11T23:17:33.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:33.196+0000 7fdf23fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdf24101420 0x7fdf241a4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:33.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:33.196+0000 7fdf290f7640 1 -- 172.21.15.26:0/2477242723 learned_addr learned my addr 172.21.15.26:0/2477242723 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:17:33.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:33.196+0000 7fdf288f6640 1 --2- 172.21.15.26:0/2477242723 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdf24100a80 0x7fdf241a4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:33.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:33.197+0000 7fdf290f7640 1 -- 172.21.15.26:0/2477242723 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdf24100a80 msgr2=0x7fdf241a4770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:33.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:33.197+0000 7fdf290f7640 1 --2- 172.21.15.26:0/2477242723 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdf24100a80 0x7fdf241a4770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:33.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:33.197+0000 7fdf290f7640 1 -- 172.21.15.26:0/2477242723 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdf24101420 msgr2=0x7fdf241a4cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:33.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:33.197+0000 7fdf290f7640 1 --2- 172.21.15.26:0/2477242723 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdf24101420 0x7fdf241a4cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:33.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:33.197+0000 7fdf290f7640 1 -- 172.21.15.26:0/2477242723 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf2419f3f0 con 0x7fdf24106a80 2024-09-11T23:17:33.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:33.197+0000 7fdf290f7640 1 --2- 172.21.15.26:0/2477242723 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf24106a80 0x7fdf2419eb80 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fdf1c005cc0 tx=0x7fdf1c003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:17:33.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:33.197+0000 7fdf21ffb640 1 -- 172.21.15.26:0/2477242723 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf1c004040 con 0x7fdf24106a80 2024-09-11T23:17:33.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:33.197+0000 7fdf21ffb640 1 -- 172.21.15.26:0/2477242723 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf1c0046c0 con 0x7fdf24106a80 2024-09-11T23:17:33.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:33.197+0000 7fdf2ab5a640 1 -- 172.21.15.26:0/2477242723 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf2419f680 con 0x7fdf24106a80 2024-09-11T23:17:33.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:33.197+0000 7fdf21ffb640 1 -- 172.21.15.26:0/2477242723 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf1c030e70 con 0x7fdf24106a80 2024-09-11T23:17:33.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:33.197+0000 7fdf2ab5a640 1 -- 172.21.15.26:0/2477242723 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf241ab6f0 con 0x7fdf24106a80 2024-09-11T23:17:33.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:33.199+0000 7fdf2ab5a640 1 -- 172.21.15.26:0/2477242723 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf241021c0 con 0x7fdf24106a80 2024-09-11T23:17:33.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:33.201+0000 7fdf21ffb640 1 -- 172.21.15.26:0/2477242723 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fdf1c0305a0 con 0x7fdf24106a80 2024-09-11T23:17:33.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:33.201+0000 7fdf21ffb640 1 --2- 172.21.15.26:0/2477242723 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fdf0c0a2c10 0x7fdf0c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:33.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:33.202+0000 7fdf288f6640 1 --2- 172.21.15.26:0/2477242723 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fdf0c0a2c10 0x7fdf0c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:33.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:33.202+0000 7fdf21ffb640 1 -- 172.21.15.26:0/2477242723 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fdf1c02e030 con 0x7fdf24106a80 2024-09-11T23:17:33.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:33.202+0000 7fdf288f6640 1 --2- 172.21.15.26:0/2477242723 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fdf0c0a2c10 0x7fdf0c0a50d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fdf14009990 tx=0x7fdf14009110 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:17:33.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:33.205+0000 7fdf21ffb640 1 -- 172.21.15.26:0/2477242723 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fdf1c0bd650 con 0x7fdf24106a80 2024-09-11T23:17:33.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:33.376+0000 7fdf2ab5a640 1 -- 172.21.15.26:0/2477242723 --> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fdf240fd470 con 0x7fdf0c0a2c10 2024-09-11T23:17:33.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:33.383+0000 7fdf21ffb640 1 -- 172.21.15.26:0/2477242723 <== mgr.14148 v2:172.21.15.26:6800/110833859 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258748 (secure 0 0 0) 0x7fdf240fd470 con 0x7fdf0c0a2c10 2024-09-11T23:17:33.385 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-11T23:17:33.386 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-09-11T23:17:33.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:33.385+0000 7fdf2ab5a640 1 -- 172.21.15.26:0/2477242723 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fdf0c0a2c10 msgr2=0x7fdf0c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:33.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:33.385+0000 7fdf2ab5a640 1 --2- 172.21.15.26:0/2477242723 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fdf0c0a2c10 0x7fdf0c0a50d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fdf14009990 tx=0x7fdf14009110 comp rx=0 tx=0).stop 2024-09-11T23:17:33.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:33.385+0000 7fdf2ab5a640 1 -- 172.21.15.26:0/2477242723 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf24106a80 msgr2=0x7fdf2419eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:33.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:33.385+0000 7fdf2ab5a640 1 --2- 172.21.15.26:0/2477242723 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf24106a80 0x7fdf2419eb80 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fdf1c005cc0 tx=0x7fdf1c003970 comp rx=0 tx=0).stop 2024-09-11T23:17:33.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:33.386+0000 7fdf2ab5a640 1 -- 172.21.15.26:0/2477242723 shutdown_connections 2024-09-11T23:17:33.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:33.386+0000 7fdf2ab5a640 1 --2- 172.21.15.26:0/2477242723 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fdf0c0a2c10 0x7fdf0c0a50d0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:33.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:33.386+0000 7fdf2ab5a640 1 --2- 172.21.15.26:0/2477242723 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdf24101420 0x7fdf241a4cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:33.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:33.386+0000 7fdf2ab5a640 1 --2- 172.21.15.26:0/2477242723 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf24106a80 0x7fdf2419eb80 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:33.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:33.386+0000 7fdf2ab5a640 1 --2- 172.21.15.26:0/2477242723 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdf24100a80 0x7fdf241a4770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:33.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:33.386+0000 7fdf2ab5a640 1 -- 172.21.15.26:0/2477242723 >> 172.21.15.26:0/2477242723 conn(0x7fdf240fc7a0 msgr2=0x7fdf240fcd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:17:33.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:33.386+0000 7fdf2ab5a640 1 -- 172.21.15.26:0/2477242723 shutdown_connections 2024-09-11T23:17:33.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:33.386+0000 7fdf2ab5a640 1 -- 172.21.15.26:0/2477242723 wait complete. 2024-09-11T23:17:33.889 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:17:33 smithi026 ceph-mon[31172]: pgmap v314: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-11T23:17:33.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:17:33 smithi136 ceph-mon[35121]: pgmap v314: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-11T23:17:34.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:17:33 smithi076 ceph-mon[35296]: pgmap v314: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-11T23:17:34.165 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":314,"stamp":"2024-09-11T23:17:32.096205+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657181,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":690,"num_read_kb":471,"num_write":424,"num_write_kb":3073,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":346,"ondisk_log_size":346,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":379104,"kb_used_data":13212,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124562720,"statfs":{"total":1151940427776,"available":1151552225280,"internally_reserved":0,"allocated":13529088,"data_stored":7344879,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":374453561},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":130,"num_read_kb":70,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003056"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.941549+0000","last_change":"2024-09-11T23:16:25.899106+0000","last_active":"2024-09-11T23:16:35.941549+0000","last_peered":"2024-09-11T23:16:35.941549+0000","last_clean":"2024-09-11T23:16:35.941549+0000","last_became_active":"2024-09-11T23:16:25.898654+0000","last_became_peered":"2024-09-11T23:16:25.898654+0000","last_unstale":"2024-09-11T23:16:35.941549+0000","last_undegraded":"2024-09-11T23:16:35.941549+0000","last_fullsized":"2024-09-11T23:16:35.941549+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:21:41.490905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:36.226970+0000","last_change":"2024-09-11T23:16:25.895629+0000","last_active":"2024-09-11T23:16:36.226970+0000","last_peered":"2024-09-11T23:16:36.226970+0000","last_clean":"2024-09-11T23:16:36.226970+0000","last_became_active":"2024-09-11T23:16:25.895308+0000","last_became_peered":"2024-09-11T23:16:25.895308+0000","last_unstale":"2024-09-11T23:16:36.226970+0000","last_undegraded":"2024-09-11T23:16:36.226970+0000","last_fullsized":"2024-09-11T23:16:36.226970+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:39:42.549970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:36.226468+0000","last_change":"2024-09-11T23:16:25.907113+0000","last_active":"2024-09-11T23:16:36.226468+0000","last_peered":"2024-09-11T23:16:36.226468+0000","last_clean":"2024-09-11T23:16:36.226468+0000","last_became_active":"2024-09-11T23:16:25.906803+0000","last_became_peered":"2024-09-11T23:16:25.906803+0000","last_unstale":"2024-09-11T23:16:36.226468+0000","last_undegraded":"2024-09-11T23:16:36.226468+0000","last_fullsized":"2024-09-11T23:16:36.226468+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:36:49.303932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.946167+0000","last_change":"2024-09-11T23:16:35.946167+0000","last_active":"2024-09-11T23:16:35.946167+0000","last_peered":"2024-09-11T23:16:35.946167+0000","last_clean":"2024-09-11T23:16:35.946167+0000","last_became_active":"2024-09-11T23:16:35.945786+0000","last_became_peered":"2024-09-11T23:16:35.945786+0000","last_unstale":"2024-09-11T23:16:35.946167+0000","last_undegraded":"2024-09-11T23:16:35.946167+0000","last_fullsized":"2024-09-11T23:16:35.946167+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:13:03.064910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.944477+0000","last_change":"2024-09-11T23:16:25.897909+0000","last_active":"2024-09-11T23:16:35.944477+0000","last_peered":"2024-09-11T23:16:35.944477+0000","last_clean":"2024-09-11T23:16:35.944477+0000","last_became_active":"2024-09-11T23:16:25.897569+0000","last_became_peered":"2024-09-11T23:16:25.897569+0000","last_unstale":"2024-09-11T23:16:35.944477+0000","last_undegraded":"2024-09-11T23:16:35.944477+0000","last_fullsized":"2024-09-11T23:16:35.944477+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:25:44.727254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.942367+0000","last_change":"2024-09-11T23:16:25.897594+0000","last_active":"2024-09-11T23:16:35.942367+0000","last_peered":"2024-09-11T23:16:35.942367+0000","last_clean":"2024-09-11T23:16:35.942367+0000","last_became_active":"2024-09-11T23:16:25.897223+0000","last_became_peered":"2024-09-11T23:16:25.897223+0000","last_unstale":"2024-09-11T23:16:35.942367+0000","last_undegraded":"2024-09-11T23:16:35.942367+0000","last_fullsized":"2024-09-11T23:16:35.942367+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:10:50.763455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.943511+0000","last_change":"2024-09-11T23:16:35.943511+0000","last_active":"2024-09-11T23:16:35.943511+0000","last_peered":"2024-09-11T23:16:35.943511+0000","last_clean":"2024-09-11T23:16:35.943511+0000","last_became_active":"2024-09-11T23:16:35.943075+0000","last_became_peered":"2024-09-11T23:16:35.943075+0000","last_unstale":"2024-09-11T23:16:35.943511+0000","last_undegraded":"2024-09-11T23:16:35.943511+0000","last_fullsized":"2024-09-11T23:16:35.943511+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:53:43.892421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:36.226370+0000","last_change":"2024-09-11T23:16:25.903380+0000","last_active":"2024-09-11T23:16:36.226370+0000","last_peered":"2024-09-11T23:16:36.226370+0000","last_clean":"2024-09-11T23:16:36.226370+0000","last_became_active":"2024-09-11T23:16:25.903093+0000","last_became_peered":"2024-09-11T23:16:25.903093+0000","last_unstale":"2024-09-11T23:16:36.226370+0000","last_undegraded":"2024-09-11T23:16:36.226370+0000","last_fullsized":"2024-09-11T23:16:36.226370+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:58:50.944623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.939171+0000","last_change":"2024-09-11T23:16:25.908166+0000","last_active":"2024-09-11T23:16:35.939171+0000","last_peered":"2024-09-11T23:16:35.939171+0000","last_clean":"2024-09-11T23:16:35.939171+0000","last_became_active":"2024-09-11T23:16:25.907749+0000","last_became_peered":"2024-09-11T23:16:25.907749+0000","last_unstale":"2024-09-11T23:16:35.939171+0000","last_undegraded":"2024-09-11T23:16:35.939171+0000","last_fullsized":"2024-09-11T23:16:35.939171+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:11:16.705931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.944259+0000","last_change":"2024-09-11T23:16:35.944259+0000","last_active":"2024-09-11T23:16:35.944259+0000","last_peered":"2024-09-11T23:16:35.944259+0000","last_clean":"2024-09-11T23:16:35.944259+0000","last_became_active":"2024-09-11T23:16:35.943767+0000","last_became_peered":"2024-09-11T23:16:35.943767+0000","last_unstale":"2024-09-11T23:16:35.944259+0000","last_undegraded":"2024-09-11T23:16:35.944259+0000","last_fullsized":"2024-09-11T23:16:35.944259+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:20:13.127854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.941922+0000","last_change":"2024-09-11T23:16:25.903256+0000","last_active":"2024-09-11T23:16:35.941922+0000","last_peered":"2024-09-11T23:16:35.941922+0000","last_clean":"2024-09-11T23:16:35.941922+0000","last_became_active":"2024-09-11T23:16:25.902954+0000","last_became_peered":"2024-09-11T23:16:25.902954+0000","last_unstale":"2024-09-11T23:16:35.941922+0000","last_undegraded":"2024-09-11T23:16:35.941922+0000","last_fullsized":"2024-09-11T23:16:35.941922+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:43:48.651400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:36.226960+0000","last_change":"2024-09-11T23:16:25.897656+0000","last_active":"2024-09-11T23:16:36.226960+0000","last_peered":"2024-09-11T23:16:36.226960+0000","last_clean":"2024-09-11T23:16:36.226960+0000","last_became_active":"2024-09-11T23:16:25.897103+0000","last_became_peered":"2024-09-11T23:16:25.897103+0000","last_unstale":"2024-09-11T23:16:36.226960+0000","last_undegraded":"2024-09-11T23:16:36.226960+0000","last_fullsized":"2024-09-11T23:16:36.226960+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:14:34.341383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.938158+0000","last_change":"2024-09-11T23:16:25.902800+0000","last_active":"2024-09-11T23:16:35.938158+0000","last_peered":"2024-09-11T23:16:35.938158+0000","last_clean":"2024-09-11T23:16:35.938158+0000","last_became_active":"2024-09-11T23:16:25.902414+0000","last_became_peered":"2024-09-11T23:16:25.902414+0000","last_unstale":"2024-09-11T23:16:35.938158+0000","last_undegraded":"2024-09-11T23:16:35.938158+0000","last_fullsized":"2024-09-11T23:16:35.938158+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:16:32.427749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:36.226337+0000","last_change":"2024-09-11T23:16:25.894410+0000","last_active":"2024-09-11T23:16:36.226337+0000","last_peered":"2024-09-11T23:16:36.226337+0000","last_clean":"2024-09-11T23:16:36.226337+0000","last_became_active":"2024-09-11T23:16:25.894067+0000","last_became_peered":"2024-09-11T23:16:25.894067+0000","last_unstale":"2024-09-11T23:16:36.226337+0000","last_undegraded":"2024-09-11T23:16:36.226337+0000","last_fullsized":"2024-09-11T23:16:36.226337+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:03:11.913063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:36.226260+0000","last_change":"2024-09-11T23:16:25.903440+0000","last_active":"2024-09-11T23:16:36.226260+0000","last_peered":"2024-09-11T23:16:36.226260+0000","last_clean":"2024-09-11T23:16:36.226260+0000","last_became_active":"2024-09-11T23:16:25.903122+0000","last_became_peered":"2024-09-11T23:16:25.903122+0000","last_unstale":"2024-09-11T23:16:36.226260+0000","last_undegraded":"2024-09-11T23:16:36.226260+0000","last_fullsized":"2024-09-11T23:16:36.226260+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:03:41.065314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.938680+0000","last_change":"2024-09-11T23:16:25.905336+0000","last_active":"2024-09-11T23:16:35.938680+0000","last_peered":"2024-09-11T23:16:35.938680+0000","last_clean":"2024-09-11T23:16:35.938680+0000","last_became_active":"2024-09-11T23:16:25.904699+0000","last_became_peered":"2024-09-11T23:16:25.904699+0000","last_unstale":"2024-09-11T23:16:35.938680+0000","last_undegraded":"2024-09-11T23:16:35.938680+0000","last_fullsized":"2024-09-11T23:16:35.938680+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:16:13.448585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.941545+0000","last_change":"2024-09-11T23:16:25.897977+0000","last_active":"2024-09-11T23:16:35.941545+0000","last_peered":"2024-09-11T23:16:35.941545+0000","last_clean":"2024-09-11T23:16:35.941545+0000","last_became_active":"2024-09-11T23:16:25.897672+0000","last_became_peered":"2024-09-11T23:16:25.897672+0000","last_unstale":"2024-09-11T23:16:35.941545+0000","last_undegraded":"2024-09-11T23:16:35.941545+0000","last_fullsized":"2024-09-11T23:16:35.941545+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:42:56.575066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.939453+0000","last_change":"2024-09-11T23:16:25.897063+0000","last_active":"2024-09-11T23:16:35.939453+0000","last_peered":"2024-09-11T23:16:35.939453+0000","last_clean":"2024-09-11T23:16:35.939453+0000","last_became_active":"2024-09-11T23:16:25.896770+0000","last_became_peered":"2024-09-11T23:16:25.896770+0000","last_unstale":"2024-09-11T23:16:35.939453+0000","last_undegraded":"2024-09-11T23:16:35.939453+0000","last_fullsized":"2024-09-11T23:16:35.939453+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:55:44.735171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.939570+0000","last_change":"2024-09-11T23:16:25.903423+0000","last_active":"2024-09-11T23:16:35.939570+0000","last_peered":"2024-09-11T23:16:35.939570+0000","last_clean":"2024-09-11T23:16:35.939570+0000","last_became_active":"2024-09-11T23:16:25.902906+0000","last_became_peered":"2024-09-11T23:16:25.902906+0000","last_unstale":"2024-09-11T23:16:35.939570+0000","last_undegraded":"2024-09-11T23:16:35.939570+0000","last_fullsized":"2024-09-11T23:16:35.939570+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:40:49.326045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.942800+0000","last_change":"2024-09-11T23:16:25.896564+0000","last_active":"2024-09-11T23:16:35.942800+0000","last_peered":"2024-09-11T23:16:35.942800+0000","last_clean":"2024-09-11T23:16:35.942800+0000","last_became_active":"2024-09-11T23:16:25.896301+0000","last_became_peered":"2024-09-11T23:16:25.896301+0000","last_unstale":"2024-09-11T23:16:35.942800+0000","last_undegraded":"2024-09-11T23:16:35.942800+0000","last_fullsized":"2024-09-11T23:16:35.942800+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:30:30.074853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.938246+0000","last_change":"2024-09-11T23:16:25.903491+0000","last_active":"2024-09-11T23:16:35.938246+0000","last_peered":"2024-09-11T23:16:35.938246+0000","last_clean":"2024-09-11T23:16:35.938246+0000","last_became_active":"2024-09-11T23:16:25.901340+0000","last_became_peered":"2024-09-11T23:16:25.901340+0000","last_unstale":"2024-09-11T23:16:35.938246+0000","last_undegraded":"2024-09-11T23:16:35.938246+0000","last_fullsized":"2024-09-11T23:16:35.938246+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:41:16.058034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.938800+0000","last_change":"2024-09-11T23:16:25.901084+0000","last_active":"2024-09-11T23:16:35.938800+0000","last_peered":"2024-09-11T23:16:35.938800+0000","last_clean":"2024-09-11T23:16:35.938800+0000","last_became_active":"2024-09-11T23:16:25.900726+0000","last_became_peered":"2024-09-11T23:16:25.900726+0000","last_unstale":"2024-09-11T23:16:35.938800+0000","last_undegraded":"2024-09-11T23:16:35.938800+0000","last_fullsized":"2024-09-11T23:16:35.938800+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:57:40.955645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:36.226559+0000","last_change":"2024-09-11T23:16:25.900461+0000","last_active":"2024-09-11T23:16:36.226559+0000","last_peered":"2024-09-11T23:16:36.226559+0000","last_clean":"2024-09-11T23:16:36.226559+0000","last_became_active":"2024-09-11T23:16:25.900174+0000","last_became_peered":"2024-09-11T23:16:25.900174+0000","last_unstale":"2024-09-11T23:16:36.226559+0000","last_undegraded":"2024-09-11T23:16:36.226559+0000","last_fullsized":"2024-09-11T23:16:36.226559+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:51:50.953198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.940957+0000","last_change":"2024-09-11T23:16:25.907569+0000","last_active":"2024-09-11T23:16:35.940957+0000","last_peered":"2024-09-11T23:16:35.940957+0000","last_clean":"2024-09-11T23:16:35.940957+0000","last_became_active":"2024-09-11T23:16:25.907091+0000","last_became_peered":"2024-09-11T23:16:25.907091+0000","last_unstale":"2024-09-11T23:16:35.940957+0000","last_undegraded":"2024-09-11T23:16:35.940957+0000","last_fullsized":"2024-09-11T23:16:35.940957+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:53:45.961167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:36.226935+0000","last_change":"2024-09-11T23:16:25.903246+0000","last_active":"2024-09-11T23:16:36.226935+0000","last_peered":"2024-09-11T23:16:36.226935+0000","last_clean":"2024-09-11T23:16:36.226935+0000","last_became_active":"2024-09-11T23:16:25.902907+0000","last_became_peered":"2024-09-11T23:16:25.902907+0000","last_unstale":"2024-09-11T23:16:36.226935+0000","last_undegraded":"2024-09-11T23:16:36.226935+0000","last_fullsized":"2024-09-11T23:16:36.226935+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:20:38.868384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.939763+0000","last_change":"2024-09-11T23:16:25.906298+0000","last_active":"2024-09-11T23:16:35.939763+0000","last_peered":"2024-09-11T23:16:35.939763+0000","last_clean":"2024-09-11T23:16:35.939763+0000","last_became_active":"2024-09-11T23:16:25.905350+0000","last_became_peered":"2024-09-11T23:16:25.905350+0000","last_unstale":"2024-09-11T23:16:35.939763+0000","last_undegraded":"2024-09-11T23:16:35.939763+0000","last_fullsized":"2024-09-11T23:16:35.939763+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:04:22.122514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.941073+0000","last_change":"2024-09-11T23:16:25.902114+0000","last_active":"2024-09-11T23:16:35.941073+0000","last_peered":"2024-09-11T23:16:35.941073+0000","last_clean":"2024-09-11T23:16:35.941073+0000","last_became_active":"2024-09-11T23:16:25.901742+0000","last_became_peered":"2024-09-11T23:16:25.901742+0000","last_unstale":"2024-09-11T23:16:35.941073+0000","last_undegraded":"2024-09-11T23:16:35.941073+0000","last_fullsized":"2024-09-11T23:16:35.941073+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:02:10.640801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.938325+0000","last_change":"2024-09-11T23:16:25.898117+0000","last_active":"2024-09-11T23:16:35.938325+0000","last_peered":"2024-09-11T23:16:35.938325+0000","last_clean":"2024-09-11T23:16:35.938325+0000","last_became_active":"2024-09-11T23:16:25.897818+0000","last_became_peered":"2024-09-11T23:16:25.897818+0000","last_unstale":"2024-09-11T23:16:35.938325+0000","last_undegraded":"2024-09-11T23:16:35.938325+0000","last_fullsized":"2024-09-11T23:16:35.938325+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:27:37.175471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:36.226573+0000","last_change":"2024-09-11T23:16:25.905600+0000","last_active":"2024-09-11T23:16:36.226573+0000","last_peered":"2024-09-11T23:16:36.226573+0000","last_clean":"2024-09-11T23:16:36.226573+0000","last_became_active":"2024-09-11T23:16:25.903924+0000","last_became_peered":"2024-09-11T23:16:25.903924+0000","last_unstale":"2024-09-11T23:16:36.226573+0000","last_undegraded":"2024-09-11T23:16:36.226573+0000","last_fullsized":"2024-09-11T23:16:36.226573+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:20:29.929079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.939290+0000","last_change":"2024-09-11T23:16:25.904763+0000","last_active":"2024-09-11T23:16:35.939290+0000","last_peered":"2024-09-11T23:16:35.939290+0000","last_clean":"2024-09-11T23:16:35.939290+0000","last_became_active":"2024-09-11T23:16:25.904406+0000","last_became_peered":"2024-09-11T23:16:25.904406+0000","last_unstale":"2024-09-11T23:16:35.939290+0000","last_undegraded":"2024-09-11T23:16:35.939290+0000","last_fullsized":"2024-09-11T23:16:35.939290+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:27:38.918017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.939346+0000","last_change":"2024-09-11T23:16:25.906573+0000","last_active":"2024-09-11T23:16:35.939346+0000","last_peered":"2024-09-11T23:16:35.939346+0000","last_clean":"2024-09-11T23:16:35.939346+0000","last_became_active":"2024-09-11T23:16:25.904771+0000","last_became_peered":"2024-09-11T23:16:25.904771+0000","last_unstale":"2024-09-11T23:16:35.939346+0000","last_undegraded":"2024-09-11T23:16:35.939346+0000","last_fullsized":"2024-09-11T23:16:35.939346+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:04:17.090648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.942191+0000","last_change":"2024-09-11T23:16:25.902123+0000","last_active":"2024-09-11T23:16:35.942191+0000","last_peered":"2024-09-11T23:16:35.942191+0000","last_clean":"2024-09-11T23:16:35.942191+0000","last_became_active":"2024-09-11T23:16:25.901756+0000","last_became_peered":"2024-09-11T23:16:25.901756+0000","last_unstale":"2024-09-11T23:16:35.942191+0000","last_undegraded":"2024-09-11T23:16:35.942191+0000","last_fullsized":"2024-09-11T23:16:35.942191+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:17:37.074976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.944098+0000","last_change":"2024-09-11T23:16:25.904217+0000","last_active":"2024-09-11T23:16:35.944098+0000","last_peered":"2024-09-11T23:16:35.944098+0000","last_clean":"2024-09-11T23:16:35.944098+0000","last_became_active":"2024-09-11T23:16:25.903522+0000","last_became_peered":"2024-09-11T23:16:25.903522+0000","last_unstale":"2024-09-11T23:16:35.944098+0000","last_undegraded":"2024-09-11T23:16:35.944098+0000","last_fullsized":"2024-09-11T23:16:35.944098+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:29:46.634475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"83'5","reported_seq":80,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:17:28.052380+0000","last_change":"2024-09-11T23:16:23.793696+0000","last_active":"2024-09-11T23:17:28.052380+0000","last_peered":"2024-09-11T23:17:28.052380+0000","last_clean":"2024-09-11T23:17:28.052380+0000","last_became_active":"2024-09-11T23:16:23.793062+0000","last_became_peered":"2024-09-11T23:16:23.793062+0000","last_unstale":"2024-09-11T23:17:28.052380+0000","last_undegraded":"2024-09-11T23:17:28.052380+0000","last_fullsized":"2024-09-11T23:17:28.052380+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:00:25.655817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":53,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.941878+0000","last_change":"2024-09-11T23:16:25.906673+0000","last_active":"2024-09-11T23:16:35.941878+0000","last_peered":"2024-09-11T23:16:35.941878+0000","last_clean":"2024-09-11T23:16:35.941878+0000","last_became_active":"2024-09-11T23:16:25.906239+0000","last_became_peered":"2024-09-11T23:16:25.906239+0000","last_unstale":"2024-09-11T23:16:35.941878+0000","last_undegraded":"2024-09-11T23:16:35.941878+0000","last_fullsized":"2024-09-11T23:16:35.941878+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:15:54.907269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"85'7","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:36.973576+0000","last_change":"2024-09-11T23:16:23.793302+0000","last_active":"2024-09-11T23:16:36.973576+0000","last_peered":"2024-09-11T23:16:36.973576+0000","last_clean":"2024-09-11T23:16:36.973576+0000","last_became_active":"2024-09-11T23:16:23.792874+0000","last_became_peered":"2024-09-11T23:16:23.792874+0000","last_unstale":"2024-09-11T23:16:36.973576+0000","last_undegraded":"2024-09-11T23:16:36.973576+0000","last_fullsized":"2024-09-11T23:16:36.973576+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:52:59.558223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"85'3","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.937984+0000","last_change":"2024-09-11T23:16:23.792999+0000","last_active":"2024-09-11T23:16:35.937984+0000","last_peered":"2024-09-11T23:16:35.937984+0000","last_clean":"2024-09-11T23:16:35.937984+0000","last_became_active":"2024-09-11T23:16:23.792630+0000","last_became_peered":"2024-09-11T23:16:23.792630+0000","last_unstale":"2024-09-11T23:16:35.937984+0000","last_undegraded":"2024-09-11T23:16:35.937984+0000","last_fullsized":"2024-09-11T23:16:35.937984+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:09:28.001060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.943722+0000","last_change":"2024-09-11T23:16:35.943722+0000","last_active":"2024-09-11T23:16:35.943722+0000","last_peered":"2024-09-11T23:16:35.943722+0000","last_clean":"2024-09-11T23:16:35.943722+0000","last_became_active":"2024-09-11T23:16:35.943355+0000","last_became_peered":"2024-09-11T23:16:35.943355+0000","last_unstale":"2024-09-11T23:16:35.943722+0000","last_undegraded":"2024-09-11T23:16:35.943722+0000","last_fullsized":"2024-09-11T23:16:35.943722+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:26:22.769824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,6],"acting":[9,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"85'3","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:37.978674+0000","last_change":"2024-09-11T23:16:23.797945+0000","last_active":"2024-09-11T23:16:37.978674+0000","last_peered":"2024-09-11T23:16:37.978674+0000","last_clean":"2024-09-11T23:16:37.978674+0000","last_became_active":"2024-09-11T23:16:23.797572+0000","last_became_peered":"2024-09-11T23:16:23.797572+0000","last_unstale":"2024-09-11T23:16:37.978674+0000","last_undegraded":"2024-09-11T23:16:37.978674+0000","last_fullsized":"2024-09-11T23:16:37.978674+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:04:59.017668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.939247+0000","last_change":"2024-09-11T23:16:25.891587+0000","last_active":"2024-09-11T23:16:35.939247+0000","last_peered":"2024-09-11T23:16:35.939247+0000","last_clean":"2024-09-11T23:16:35.939247+0000","last_became_active":"2024-09-11T23:16:25.891190+0000","last_became_peered":"2024-09-11T23:16:25.891190+0000","last_unstale":"2024-09-11T23:16:35.939247+0000","last_undegraded":"2024-09-11T23:16:35.939247+0000","last_fullsized":"2024-09-11T23:16:35.939247+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:57:05.177843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"85'4","reported_seq":130,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:17:30.054432+0000","last_change":"2024-09-11T23:16:23.790129+0000","last_active":"2024-09-11T23:17:30.054432+0000","last_peered":"2024-09-11T23:17:30.054432+0000","last_clean":"2024-09-11T23:17:30.054432+0000","last_became_active":"2024-09-11T23:16:23.789720+0000","last_became_peered":"2024-09-11T23:16:23.789720+0000","last_unstale":"2024-09-11T23:17:30.054432+0000","last_undegraded":"2024-09-11T23:17:30.054432+0000","last_fullsized":"2024-09-11T23:17:30.054432+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:33:58.388513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":104,"num_read_kb":104,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.939104+0000","last_change":"2024-09-11T23:16:25.901014+0000","last_active":"2024-09-11T23:16:35.939104+0000","last_peered":"2024-09-11T23:16:35.939104+0000","last_clean":"2024-09-11T23:16:35.939104+0000","last_became_active":"2024-09-11T23:16:25.900280+0000","last_became_peered":"2024-09-11T23:16:25.900280+0000","last_unstale":"2024-09-11T23:16:35.939104+0000","last_undegraded":"2024-09-11T23:16:35.939104+0000","last_fullsized":"2024-09-11T23:16:35.939104+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:14:03.662204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"85'4","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:37.978517+0000","last_change":"2024-09-11T23:16:23.797944+0000","last_active":"2024-09-11T23:16:37.978517+0000","last_peered":"2024-09-11T23:16:37.978517+0000","last_clean":"2024-09-11T23:16:37.978517+0000","last_became_active":"2024-09-11T23:16:23.797520+0000","last_became_peered":"2024-09-11T23:16:23.797520+0000","last_unstale":"2024-09-11T23:16:37.978517+0000","last_undegraded":"2024-09-11T23:16:37.978517+0000","last_fullsized":"2024-09-11T23:16:37.978517+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:29:06.957518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:36.226839+0000","last_change":"2024-09-11T23:16:25.905379+0000","last_active":"2024-09-11T23:16:36.226839+0000","last_peered":"2024-09-11T23:16:36.226839+0000","last_clean":"2024-09-11T23:16:36.226839+0000","last_became_active":"2024-09-11T23:16:25.905085+0000","last_became_peered":"2024-09-11T23:16:25.905085+0000","last_unstale":"2024-09-11T23:16:36.226839+0000","last_undegraded":"2024-09-11T23:16:36.226839+0000","last_fullsized":"2024-09-11T23:16:36.226839+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:22:35.129736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"85'5","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:36.972662+0000","last_change":"2024-09-11T23:16:24.075556+0000","last_active":"2024-09-11T23:16:36.972662+0000","last_peered":"2024-09-11T23:16:36.972662+0000","last_clean":"2024-09-11T23:16:36.972662+0000","last_became_active":"2024-09-11T23:16:24.074783+0000","last_became_peered":"2024-09-11T23:16:24.074783+0000","last_unstale":"2024-09-11T23:16:36.972662+0000","last_undegraded":"2024-09-11T23:16:36.972662+0000","last_fullsized":"2024-09-11T23:16:36.972662+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:55:09.962342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.938922+0000","last_change":"2024-09-11T23:16:25.907720+0000","last_active":"2024-09-11T23:16:35.938922+0000","last_peered":"2024-09-11T23:16:35.938922+0000","last_clean":"2024-09-11T23:16:35.938922+0000","last_became_active":"2024-09-11T23:16:25.907335+0000","last_became_peered":"2024-09-11T23:16:25.907335+0000","last_unstale":"2024-09-11T23:16:35.938922+0000","last_undegraded":"2024-09-11T23:16:35.938922+0000","last_fullsized":"2024-09-11T23:16:35.938922+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:46:14.519384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"85'30","reported_seq":165,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:17:30.067312+0000","last_change":"2024-09-11T23:16:23.794321+0000","last_active":"2024-09-11T23:17:30.067312+0000","last_peered":"2024-09-11T23:17:30.067312+0000","last_clean":"2024-09-11T23:17:30.067312+0000","last_became_active":"2024-09-11T23:16:23.793955+0000","last_became_peered":"2024-09-11T23:16:23.793955+0000","last_unstale":"2024-09-11T23:17:30.067312+0000","last_undegraded":"2024-09-11T23:17:30.067312+0000","last_fullsized":"2024-09-11T23:17:30.067312+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:47:46.442656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59764,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":113,"num_read_kb":114,"num_write":32,"num_write_kb":69,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.945323+0000","last_change":"2024-09-11T23:16:25.896598+0000","last_active":"2024-09-11T23:16:35.945323+0000","last_peered":"2024-09-11T23:16:35.945323+0000","last_clean":"2024-09-11T23:16:35.945323+0000","last_became_active":"2024-09-11T23:16:25.896293+0000","last_became_peered":"2024-09-11T23:16:25.896293+0000","last_unstale":"2024-09-11T23:16:35.945323+0000","last_undegraded":"2024-09-11T23:16:35.945323+0000","last_fullsized":"2024-09-11T23:16:35.945323+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:04:11.489644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"85'9","reported_seq":33,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:38.985116+0000","last_change":"2024-09-11T23:16:23.791486+0000","last_active":"2024-09-11T23:16:38.985116+0000","last_peered":"2024-09-11T23:16:38.985116+0000","last_clean":"2024-09-11T23:16:38.985116+0000","last_became_active":"2024-09-11T23:16:23.790786+0000","last_became_peered":"2024-09-11T23:16:23.790786+0000","last_unstale":"2024-09-11T23:16:38.985116+0000","last_undegraded":"2024-09-11T23:16:38.985116+0000","last_fullsized":"2024-09-11T23:16:38.985116+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:03:52.742560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.937975+0000","last_change":"2024-09-11T23:16:25.897382+0000","last_active":"2024-09-11T23:16:35.937975+0000","last_peered":"2024-09-11T23:16:35.937975+0000","last_clean":"2024-09-11T23:16:35.937975+0000","last_became_active":"2024-09-11T23:16:25.897073+0000","last_became_peered":"2024-09-11T23:16:25.897073+0000","last_unstale":"2024-09-11T23:16:35.937975+0000","last_undegraded":"2024-09-11T23:16:35.937975+0000","last_fullsized":"2024-09-11T23:16:35.937975+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:15:59.780100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"85'5","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:37.977834+0000","last_change":"2024-09-11T23:16:23.793886+0000","last_active":"2024-09-11T23:16:37.977834+0000","last_peered":"2024-09-11T23:16:37.977834+0000","last_clean":"2024-09-11T23:16:37.977834+0000","last_became_active":"2024-09-11T23:16:23.793066+0000","last_became_peered":"2024-09-11T23:16:23.793066+0000","last_unstale":"2024-09-11T23:16:37.977834+0000","last_undegraded":"2024-09-11T23:16:37.977834+0000","last_fullsized":"2024-09-11T23:16:37.977834+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:07:03.882546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.942540+0000","last_change":"2024-09-11T23:16:25.904575+0000","last_active":"2024-09-11T23:16:35.942540+0000","last_peered":"2024-09-11T23:16:35.942540+0000","last_clean":"2024-09-11T23:16:35.942540+0000","last_became_active":"2024-09-11T23:16:25.904224+0000","last_became_peered":"2024-09-11T23:16:25.904224+0000","last_unstale":"2024-09-11T23:16:35.942540+0000","last_undegraded":"2024-09-11T23:16:35.942540+0000","last_fullsized":"2024-09-11T23:16:35.942540+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:42:13.036931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"85'5","reported_seq":81,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:17:29.053747+0000","last_change":"2024-09-11T23:16:23.796073+0000","last_active":"2024-09-11T23:17:29.053747+0000","last_peered":"2024-09-11T23:17:29.053747+0000","last_clean":"2024-09-11T23:17:29.053747+0000","last_became_active":"2024-09-11T23:16:23.794850+0000","last_became_peered":"2024-09-11T23:16:23.794850+0000","last_unstale":"2024-09-11T23:17:29.053747+0000","last_undegraded":"2024-09-11T23:17:29.053747+0000","last_fullsized":"2024-09-11T23:17:29.053747+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:57:22.361662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":55,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.940823+0000","last_change":"2024-09-11T23:16:25.908700+0000","last_active":"2024-09-11T23:16:35.940823+0000","last_peered":"2024-09-11T23:16:35.940823+0000","last_clean":"2024-09-11T23:16:35.940823+0000","last_became_active":"2024-09-11T23:16:25.908373+0000","last_became_peered":"2024-09-11T23:16:25.908373+0000","last_unstale":"2024-09-11T23:16:35.940823+0000","last_undegraded":"2024-09-11T23:16:35.940823+0000","last_fullsized":"2024-09-11T23:16:35.940823+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:20:18.536079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"85'17","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:37.976750+0000","last_change":"2024-09-11T23:16:24.075927+0000","last_active":"2024-09-11T23:16:37.976750+0000","last_peered":"2024-09-11T23:16:37.976750+0000","last_clean":"2024-09-11T23:16:37.976750+0000","last_became_active":"2024-09-11T23:16:24.075527+0000","last_became_peered":"2024-09-11T23:16:24.075527+0000","last_unstale":"2024-09-11T23:16:37.976750+0000","last_undegraded":"2024-09-11T23:16:37.976750+0000","last_fullsized":"2024-09-11T23:16:37.976750+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:27:54.622059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.947475+0000","last_change":"2024-09-11T23:16:35.947475+0000","last_active":"2024-09-11T23:16:35.947475+0000","last_peered":"2024-09-11T23:16:35.947475+0000","last_clean":"2024-09-11T23:16:35.947475+0000","last_became_active":"2024-09-11T23:16:35.947028+0000","last_became_peered":"2024-09-11T23:16:35.947028+0000","last_unstale":"2024-09-11T23:16:35.947475+0000","last_undegraded":"2024-09-11T23:16:35.947475+0000","last_fullsized":"2024-09-11T23:16:35.947475+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:18:13.479554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'2","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:37.950468+0000","last_change":"2024-09-11T23:16:23.793298+0000","last_active":"2024-09-11T23:16:37.950468+0000","last_peered":"2024-09-11T23:16:37.950468+0000","last_clean":"2024-09-11T23:16:37.950468+0000","last_became_active":"2024-09-11T23:16:23.792947+0000","last_became_peered":"2024-09-11T23:16:23.792947+0000","last_unstale":"2024-09-11T23:16:37.950468+0000","last_undegraded":"2024-09-11T23:16:37.950468+0000","last_fullsized":"2024-09-11T23:16:37.950468+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:32:21.294560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.941169+0000","last_change":"2024-09-11T23:16:25.902348+0000","last_active":"2024-09-11T23:16:35.941169+0000","last_peered":"2024-09-11T23:16:35.941169+0000","last_clean":"2024-09-11T23:16:35.941169+0000","last_became_active":"2024-09-11T23:16:25.901686+0000","last_became_peered":"2024-09-11T23:16:25.901686+0000","last_unstale":"2024-09-11T23:16:35.941169+0000","last_undegraded":"2024-09-11T23:16:35.941169+0000","last_fullsized":"2024-09-11T23:16:35.941169+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:00:15.543003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"85'5","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:37.975894+0000","last_change":"2024-09-11T23:16:23.790984+0000","last_active":"2024-09-11T23:16:37.975894+0000","last_peered":"2024-09-11T23:16:37.975894+0000","last_clean":"2024-09-11T23:16:37.975894+0000","last_became_active":"2024-09-11T23:16:23.790569+0000","last_became_peered":"2024-09-11T23:16:23.790569+0000","last_unstale":"2024-09-11T23:16:37.975894+0000","last_undegraded":"2024-09-11T23:16:37.975894+0000","last_fullsized":"2024-09-11T23:16:37.975894+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:57:39.675793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.941423+0000","last_change":"2024-09-11T23:16:25.899249+0000","last_active":"2024-09-11T23:16:35.941423+0000","last_peered":"2024-09-11T23:16:35.941423+0000","last_clean":"2024-09-11T23:16:35.941423+0000","last_became_active":"2024-09-11T23:16:25.898914+0000","last_became_peered":"2024-09-11T23:16:25.898914+0000","last_unstale":"2024-09-11T23:16:35.941423+0000","last_undegraded":"2024-09-11T23:16:35.941423+0000","last_fullsized":"2024-09-11T23:16:35.941423+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:17:07.545770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'3","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:37.977945+0000","last_change":"2024-09-11T23:16:23.787387+0000","last_active":"2024-09-11T23:16:37.977945+0000","last_peered":"2024-09-11T23:16:37.977945+0000","last_clean":"2024-09-11T23:16:37.977945+0000","last_became_active":"2024-09-11T23:16:23.787014+0000","last_became_peered":"2024-09-11T23:16:23.787014+0000","last_unstale":"2024-09-11T23:16:37.977945+0000","last_undegraded":"2024-09-11T23:16:37.977945+0000","last_fullsized":"2024-09-11T23:16:37.977945+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:45:52.004253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"85'1","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.941159+0000","last_change":"2024-09-11T23:16:25.908304+0000","last_active":"2024-09-11T23:16:35.941159+0000","last_peered":"2024-09-11T23:16:35.941159+0000","last_clean":"2024-09-11T23:16:35.941159+0000","last_became_active":"2024-09-11T23:16:25.907900+0000","last_became_peered":"2024-09-11T23:16:25.907900+0000","last_unstale":"2024-09-11T23:16:35.941159+0000","last_undegraded":"2024-09-11T23:16:35.941159+0000","last_fullsized":"2024-09-11T23:16:35.941159+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:43:26.247682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"85'6","reported_seq":81,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:17:27.051370+0000","last_change":"2024-09-11T23:16:24.074610+0000","last_active":"2024-09-11T23:17:27.051370+0000","last_peered":"2024-09-11T23:17:27.051370+0000","last_clean":"2024-09-11T23:17:27.051370+0000","last_became_active":"2024-09-11T23:16:24.074210+0000","last_became_peered":"2024-09-11T23:16:24.074210+0000","last_unstale":"2024-09-11T23:17:27.051370+0000","last_undegraded":"2024-09-11T23:17:27.051370+0000","last_fullsized":"2024-09-11T23:17:27.051370+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:09:37.167050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1563,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":53,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.941591+0000","last_change":"2024-09-11T23:16:25.903120+0000","last_active":"2024-09-11T23:16:35.941591+0000","last_peered":"2024-09-11T23:16:35.941591+0000","last_clean":"2024-09-11T23:16:35.941591+0000","last_became_active":"2024-09-11T23:16:25.902770+0000","last_became_peered":"2024-09-11T23:16:25.902770+0000","last_unstale":"2024-09-11T23:16:35.941591+0000","last_undegraded":"2024-09-11T23:16:35.941591+0000","last_fullsized":"2024-09-11T23:16:35.941591+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:03:30.041088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":346,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.941279+0000","last_change":"2024-09-11T23:15:13.115055+0000","last_active":"2024-09-11T23:16:35.941279+0000","last_peered":"2024-09-11T23:16:35.941279+0000","last_clean":"2024-09-11T23:16:35.941279+0000","last_became_active":"2024-09-11T23:15:13.114480+0000","last_became_peered":"2024-09-11T23:15:13.114480+0000","last_unstale":"2024-09-11T23:16:35.941279+0000","last_undegraded":"2024-09-11T23:16:35.941279+0000","last_fullsized":"2024-09-11T23:16:35.941279+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:11:50.495799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:11:50.495799+0000","last_clean_scrub_stamp":"2024-09-11T23:11:50.495799+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:15:53.681230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"85'6","reported_seq":31,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:38.985260+0000","last_change":"2024-09-11T23:16:23.793996+0000","last_active":"2024-09-11T23:16:38.985260+0000","last_peered":"2024-09-11T23:16:38.985260+0000","last_clean":"2024-09-11T23:16:38.985260+0000","last_became_active":"2024-09-11T23:16:23.793681+0000","last_became_peered":"2024-09-11T23:16:23.793681+0000","last_unstale":"2024-09-11T23:16:38.985260+0000","last_undegraded":"2024-09-11T23:16:38.985260+0000","last_fullsized":"2024-09-11T23:16:38.985260+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:52:50.463623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:36.227084+0000","last_change":"2024-09-11T23:16:25.905133+0000","last_active":"2024-09-11T23:16:36.227084+0000","last_peered":"2024-09-11T23:16:36.227084+0000","last_clean":"2024-09-11T23:16:36.227084+0000","last_became_active":"2024-09-11T23:16:25.904819+0000","last_became_peered":"2024-09-11T23:16:25.904819+0000","last_unstale":"2024-09-11T23:16:36.227084+0000","last_undegraded":"2024-09-11T23:16:36.227084+0000","last_fullsized":"2024-09-11T23:16:36.227084+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:51:09.664343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"85'4","reported_seq":32,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:36.231286+0000","last_change":"2024-09-11T23:16:36.228806+0000","last_active":"2024-09-11T23:16:36.231286+0000","last_peered":"2024-09-11T23:16:36.231286+0000","last_clean":"2024-09-11T23:16:36.231286+0000","last_became_active":"2024-09-11T23:16:36.228311+0000","last_became_peered":"2024-09-11T23:16:36.228311+0000","last_unstale":"2024-09-11T23:16:36.231286+0000","last_undegraded":"2024-09-11T23:16:36.231286+0000","last_fullsized":"2024-09-11T23:16:36.231286+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:42:32.856691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4],"acting":[7,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.937900+0000","last_change":"2024-09-11T23:16:25.898935+0000","last_active":"2024-09-11T23:16:35.937900+0000","last_peered":"2024-09-11T23:16:35.937900+0000","last_clean":"2024-09-11T23:16:35.937900+0000","last_became_active":"2024-09-11T23:16:25.898618+0000","last_became_peered":"2024-09-11T23:16:25.898618+0000","last_unstale":"2024-09-11T23:16:35.937900+0000","last_undegraded":"2024-09-11T23:16:35.937900+0000","last_fullsized":"2024-09-11T23:16:35.937900+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:13:29.104911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"85'5","reported_seq":79,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:17:28.051750+0000","last_change":"2024-09-11T23:16:24.073613+0000","last_active":"2024-09-11T23:17:28.051750+0000","last_peered":"2024-09-11T23:17:28.051750+0000","last_clean":"2024-09-11T23:17:28.051750+0000","last_became_active":"2024-09-11T23:16:24.073262+0000","last_became_peered":"2024-09-11T23:16:24.073262+0000","last_unstale":"2024-09-11T23:17:28.051750+0000","last_undegraded":"2024-09-11T23:17:28.051750+0000","last_fullsized":"2024-09-11T23:17:28.051750+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:31:58.653657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":52,"num_read_kb":52,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.941067+0000","last_change":"2024-09-11T23:16:25.903725+0000","last_active":"2024-09-11T23:16:35.941067+0000","last_peered":"2024-09-11T23:16:35.941067+0000","last_clean":"2024-09-11T23:16:35.941067+0000","last_became_active":"2024-09-11T23:16:25.903375+0000","last_became_peered":"2024-09-11T23:16:25.903375+0000","last_unstale":"2024-09-11T23:16:35.941067+0000","last_undegraded":"2024-09-11T23:16:35.941067+0000","last_fullsized":"2024-09-11T23:16:35.941067+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:55:02.779575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"85'5","reported_seq":78,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:17:27.050537+0000","last_change":"2024-09-11T23:16:23.793932+0000","last_active":"2024-09-11T23:17:27.050537+0000","last_peered":"2024-09-11T23:17:27.050537+0000","last_clean":"2024-09-11T23:17:27.050537+0000","last_became_active":"2024-09-11T23:16:23.793111+0000","last_became_peered":"2024-09-11T23:16:23.793111+0000","last_unstale":"2024-09-11T23:17:27.050537+0000","last_undegraded":"2024-09-11T23:17:27.050537+0000","last_fullsized":"2024-09-11T23:17:27.050537+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:14:09.087222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":51,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.942165+0000","last_change":"2024-09-11T23:16:25.905020+0000","last_active":"2024-09-11T23:16:35.942165+0000","last_peered":"2024-09-11T23:16:35.942165+0000","last_clean":"2024-09-11T23:16:35.942165+0000","last_became_active":"2024-09-11T23:16:25.904675+0000","last_became_peered":"2024-09-11T23:16:25.904675+0000","last_unstale":"2024-09-11T23:16:35.942165+0000","last_undegraded":"2024-09-11T23:16:35.942165+0000","last_fullsized":"2024-09-11T23:16:35.942165+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:13:57.388773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"85'1","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.939622+0000","last_change":"2024-09-11T23:16:24.073630+0000","last_active":"2024-09-11T23:16:35.939622+0000","last_peered":"2024-09-11T23:16:35.939622+0000","last_clean":"2024-09-11T23:16:35.939622+0000","last_became_active":"2024-09-11T23:16:24.073301+0000","last_became_peered":"2024-09-11T23:16:24.073301+0000","last_unstale":"2024-09-11T23:16:35.939622+0000","last_undegraded":"2024-09-11T23:16:35.939622+0000","last_fullsized":"2024-09-11T23:16:35.939622+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:41:59.374581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"85'1","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.942393+0000","last_change":"2024-09-11T23:16:25.894124+0000","last_active":"2024-09-11T23:16:35.942393+0000","last_peered":"2024-09-11T23:16:35.942393+0000","last_clean":"2024-09-11T23:16:35.942393+0000","last_became_active":"2024-09-11T23:16:25.893751+0000","last_became_peered":"2024-09-11T23:16:25.893751+0000","last_unstale":"2024-09-11T23:16:35.942393+0000","last_undegraded":"2024-09-11T23:16:35.942393+0000","last_fullsized":"2024-09-11T23:16:35.942393+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:00:28.265206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"85'4","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:37.976960+0000","last_change":"2024-09-11T23:16:24.073700+0000","last_active":"2024-09-11T23:16:37.976960+0000","last_peered":"2024-09-11T23:16:37.976960+0000","last_clean":"2024-09-11T23:16:37.976960+0000","last_became_active":"2024-09-11T23:16:24.073283+0000","last_became_peered":"2024-09-11T23:16:24.073283+0000","last_unstale":"2024-09-11T23:16:37.976960+0000","last_undegraded":"2024-09-11T23:16:37.976960+0000","last_fullsized":"2024-09-11T23:16:37.976960+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:41:43.536834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.938154+0000","last_change":"2024-09-11T23:16:25.900532+0000","last_active":"2024-09-11T23:16:35.938154+0000","last_peered":"2024-09-11T23:16:35.938154+0000","last_clean":"2024-09-11T23:16:35.938154+0000","last_became_active":"2024-09-11T23:16:25.899466+0000","last_became_peered":"2024-09-11T23:16:25.899466+0000","last_unstale":"2024-09-11T23:16:35.938154+0000","last_undegraded":"2024-09-11T23:16:35.938154+0000","last_fullsized":"2024-09-11T23:16:35.938154+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:10:01.152323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"85'14","reported_seq":36,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:50.914551+0000","last_change":"2024-09-11T23:16:23.788789+0000","last_active":"2024-09-11T23:16:50.914551+0000","last_peered":"2024-09-11T23:16:50.914551+0000","last_clean":"2024-09-11T23:16:50.914551+0000","last_became_active":"2024-09-11T23:16:23.788422+0000","last_became_peered":"2024-09-11T23:16:23.788422+0000","last_unstale":"2024-09-11T23:16:50.914551+0000","last_undegraded":"2024-09-11T23:16:50.914551+0000","last_fullsized":"2024-09-11T23:16:50.914551+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:09:14.310205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:36.227116+0000","last_change":"2024-09-11T23:16:25.895511+0000","last_active":"2024-09-11T23:16:36.227116+0000","last_peered":"2024-09-11T23:16:36.227116+0000","last_clean":"2024-09-11T23:16:36.227116+0000","last_became_active":"2024-09-11T23:16:25.895143+0000","last_became_peered":"2024-09-11T23:16:25.895143+0000","last_unstale":"2024-09-11T23:16:36.227116+0000","last_undegraded":"2024-09-11T23:16:36.227116+0000","last_fullsized":"2024-09-11T23:16:36.227116+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:31:16.498371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"85'7","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:37.977151+0000","last_change":"2024-09-11T23:16:24.019064+0000","last_active":"2024-09-11T23:16:37.977151+0000","last_peered":"2024-09-11T23:16:37.977151+0000","last_clean":"2024-09-11T23:16:37.977151+0000","last_became_active":"2024-09-11T23:16:24.018658+0000","last_became_peered":"2024-09-11T23:16:24.018658+0000","last_unstale":"2024-09-11T23:16:37.977151+0000","last_undegraded":"2024-09-11T23:16:37.977151+0000","last_fullsized":"2024-09-11T23:16:37.977151+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:17:41.009507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:36.227194+0000","last_change":"2024-09-11T23:16:25.903553+0000","last_active":"2024-09-11T23:16:36.227194+0000","last_peered":"2024-09-11T23:16:36.227194+0000","last_clean":"2024-09-11T23:16:36.227194+0000","last_became_active":"2024-09-11T23:16:25.903221+0000","last_became_peered":"2024-09-11T23:16:25.903221+0000","last_unstale":"2024-09-11T23:16:36.227194+0000","last_undegraded":"2024-09-11T23:16:36.227194+0000","last_fullsized":"2024-09-11T23:16:36.227194+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:16:51.872517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"83'2","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.938847+0000","last_change":"2024-09-11T23:16:24.073784+0000","last_active":"2024-09-11T23:16:35.938847+0000","last_peered":"2024-09-11T23:16:35.938847+0000","last_clean":"2024-09-11T23:16:35.938847+0000","last_became_active":"2024-09-11T23:16:24.073406+0000","last_became_peered":"2024-09-11T23:16:24.073406+0000","last_unstale":"2024-09-11T23:16:35.938847+0000","last_undegraded":"2024-09-11T23:16:35.938847+0000","last_fullsized":"2024-09-11T23:16:35.938847+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:31:24.617025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.945715+0000","last_change":"2024-09-11T23:16:25.894878+0000","last_active":"2024-09-11T23:16:35.945715+0000","last_peered":"2024-09-11T23:16:35.945715+0000","last_clean":"2024-09-11T23:16:35.945715+0000","last_became_active":"2024-09-11T23:16:25.894543+0000","last_became_peered":"2024-09-11T23:16:25.894543+0000","last_unstale":"2024-09-11T23:16:35.945715+0000","last_undegraded":"2024-09-11T23:16:35.945715+0000","last_fullsized":"2024-09-11T23:16:35.945715+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:34:13.435754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"85'7","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:37.976060+0000","last_change":"2024-09-11T23:16:23.786502+0000","last_active":"2024-09-11T23:16:37.976060+0000","last_peered":"2024-09-11T23:16:37.976060+0000","last_clean":"2024-09-11T23:16:37.976060+0000","last_became_active":"2024-09-11T23:16:23.786182+0000","last_became_peered":"2024-09-11T23:16:23.786182+0000","last_unstale":"2024-09-11T23:16:37.976060+0000","last_undegraded":"2024-09-11T23:16:37.976060+0000","last_fullsized":"2024-09-11T23:16:37.976060+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:59:07.978244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.942309+0000","last_change":"2024-09-11T23:16:25.899625+0000","last_active":"2024-09-11T23:16:35.942309+0000","last_peered":"2024-09-11T23:16:35.942309+0000","last_clean":"2024-09-11T23:16:35.942309+0000","last_became_active":"2024-09-11T23:16:25.899265+0000","last_became_peered":"2024-09-11T23:16:25.899265+0000","last_unstale":"2024-09-11T23:16:35.942309+0000","last_undegraded":"2024-09-11T23:16:35.942309+0000","last_fullsized":"2024-09-11T23:16:35.942309+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:36:06.907622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"85'3","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:36.971236+0000","last_change":"2024-09-11T23:16:36.228991+0000","last_active":"2024-09-11T23:16:36.971236+0000","last_peered":"2024-09-11T23:16:36.971236+0000","last_clean":"2024-09-11T23:16:36.971236+0000","last_became_active":"2024-09-11T23:16:36.227701+0000","last_became_peered":"2024-09-11T23:16:36.227701+0000","last_unstale":"2024-09-11T23:16:36.971236+0000","last_undegraded":"2024-09-11T23:16:36.971236+0000","last_fullsized":"2024-09-11T23:16:36.971236+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:37:57.478572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:36.226661+0000","last_change":"2024-09-11T23:16:25.894803+0000","last_active":"2024-09-11T23:16:36.226661+0000","last_peered":"2024-09-11T23:16:36.226661+0000","last_clean":"2024-09-11T23:16:36.226661+0000","last_became_active":"2024-09-11T23:16:25.894429+0000","last_became_peered":"2024-09-11T23:16:25.894429+0000","last_unstale":"2024-09-11T23:16:36.226661+0000","last_undegraded":"2024-09-11T23:16:36.226661+0000","last_fullsized":"2024-09-11T23:16:36.226661+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:39:15.868910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"83'4","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.945644+0000","last_change":"2024-09-11T23:16:23.794271+0000","last_active":"2024-09-11T23:16:35.945644+0000","last_peered":"2024-09-11T23:16:35.945644+0000","last_clean":"2024-09-11T23:16:35.945644+0000","last_became_active":"2024-09-11T23:16:23.793919+0000","last_became_peered":"2024-09-11T23:16:23.793919+0000","last_unstale":"2024-09-11T23:16:35.945644+0000","last_undegraded":"2024-09-11T23:16:35.945644+0000","last_fullsized":"2024-09-11T23:16:35.945644+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:30:28.907312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.941192+0000","last_change":"2024-09-11T23:16:25.900969+0000","last_active":"2024-09-11T23:16:35.941192+0000","last_peered":"2024-09-11T23:16:35.941192+0000","last_clean":"2024-09-11T23:16:35.941192+0000","last_became_active":"2024-09-11T23:16:25.900425+0000","last_became_peered":"2024-09-11T23:16:25.900425+0000","last_unstale":"2024-09-11T23:16:35.941192+0000","last_undegraded":"2024-09-11T23:16:35.941192+0000","last_fullsized":"2024-09-11T23:16:35.941192+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:03:25.185773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.941358+0000","last_change":"2024-09-11T23:16:25.911605+0000","last_active":"2024-09-11T23:16:35.941358+0000","last_peered":"2024-09-11T23:16:35.941358+0000","last_clean":"2024-09-11T23:16:35.941358+0000","last_became_active":"2024-09-11T23:16:25.911180+0000","last_became_peered":"2024-09-11T23:16:25.911180+0000","last_unstale":"2024-09-11T23:16:35.941358+0000","last_undegraded":"2024-09-11T23:16:35.941358+0000","last_fullsized":"2024-09-11T23:16:35.941358+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:55:38.062606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"85'5","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:37.979007+0000","last_change":"2024-09-11T23:16:35.949528+0000","last_active":"2024-09-11T23:16:37.979007+0000","last_peered":"2024-09-11T23:16:37.979007+0000","last_clean":"2024-09-11T23:16:37.979007+0000","last_became_active":"2024-09-11T23:16:35.945679+0000","last_became_peered":"2024-09-11T23:16:35.945679+0000","last_unstale":"2024-09-11T23:16:37.979007+0000","last_undegraded":"2024-09-11T23:16:37.979007+0000","last_fullsized":"2024-09-11T23:16:37.979007+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:14:24.706044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:35.938792+0000","last_change":"2024-09-11T23:16:25.900526+0000","last_active":"2024-09-11T23:16:35.938792+0000","last_peered":"2024-09-11T23:16:35.938792+0000","last_clean":"2024-09-11T23:16:35.938792+0000","last_became_active":"2024-09-11T23:16:25.900148+0000","last_became_peered":"2024-09-11T23:16:25.900148+0000","last_unstale":"2024-09-11T23:16:35.938792+0000","last_undegraded":"2024-09-11T23:16:35.938792+0000","last_fullsized":"2024-09-11T23:16:35.938792+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:26:10.582473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"85'12","reported_seq":45,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:38.986047+0000","last_change":"2024-09-11T23:16:36.233273+0000","last_active":"2024-09-11T23:16:38.986047+0000","last_peered":"2024-09-11T23:16:38.986047+0000","last_clean":"2024-09-11T23:16:38.986047+0000","last_became_active":"2024-09-11T23:16:36.228358+0000","last_became_peered":"2024-09-11T23:16:36.228358+0000","last_unstale":"2024-09-11T23:16:38.986047+0000","last_undegraded":"2024-09-11T23:16:38.986047+0000","last_fullsized":"2024-09-11T23:16:38.986047+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:06:31.225795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:36.226734+0000","last_change":"2024-09-11T23:16:25.907874+0000","last_active":"2024-09-11T23:16:36.226734+0000","last_peered":"2024-09-11T23:16:36.226734+0000","last_clean":"2024-09-11T23:16:36.226734+0000","last_became_active":"2024-09-11T23:16:25.907491+0000","last_became_peered":"2024-09-11T23:16:25.907491+0000","last_unstale":"2024-09-11T23:16:36.226734+0000","last_undegraded":"2024-09-11T23:16:36.226734+0000","last_fullsized":"2024-09-11T23:16:36.226734+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:09:20.984249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"84'2","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:36.227306+0000","last_change":"2024-09-11T23:16:24.074355+0000","last_active":"2024-09-11T23:16:36.227306+0000","last_peered":"2024-09-11T23:16:36.227306+0000","last_clean":"2024-09-11T23:16:36.227306+0000","last_became_active":"2024-09-11T23:16:24.074047+0000","last_became_peered":"2024-09-11T23:16:24.074047+0000","last_unstale":"2024-09-11T23:16:36.227306+0000","last_undegraded":"2024-09-11T23:16:36.227306+0000","last_fullsized":"2024-09-11T23:16:36.227306+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:31:58.804420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:16:36.226393+0000","last_change":"2024-09-11T23:16:25.907701+0000","last_active":"2024-09-11T23:16:36.226393+0000","last_peered":"2024-09-11T23:16:36.226393+0000","last_clean":"2024-09-11T23:16:36.226393+0000","last_became_active":"2024-09-11T23:16:25.907379+0000","last_became_peered":"2024-09-11T23:16:25.907379+0000","last_unstale":"2024-09-11T23:16:36.226393+0000","last_undegraded":"2024-09-11T23:16:36.226393+0000","last_fullsized":"2024-09-11T23:16:36.226393+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:12:03.639055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"85'13","reported_seq":88,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:17:29.052467+0000","last_change":"2024-09-11T23:16:23.794293+0000","last_active":"2024-09-11T23:17:29.052467+0000","last_peered":"2024-09-11T23:17:29.052467+0000","last_clean":"2024-09-11T23:17:29.052467+0000","last_became_active":"2024-09-11T23:16:23.793974+0000","last_became_peered":"2024-09-11T23:16:23.793974+0000","last_unstale":"2024-09-11T23:17:29.052467+0000","last_undegraded":"2024-09-11T23:17:29.052467+0000","last_fullsized":"2024-09-11T23:17:29.052467+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:24:39.170188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":53,"num_read_kb":53,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66579,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":554,"num_read_kb":352,"num_write":241,"num_write_kb":133,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":199737,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":207,"ondisk_log_size":207,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":77,"seq":330712481819,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32108,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713044,"statfs":{"total":95995035648,"available":95962157056,"internally_reserved":0,"allocated":946176,"data_stored":449945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":70,"seq":300647710751,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28628,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716524,"statfs":{"total":95995035648,"available":95965720576,"internally_reserved":0,"allocated":1642496,"data_stored":1099763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":65,"seq":279172874277,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":966656,"data_stored":448900,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":57,"seq":244813135914,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":958464,"data_stored":448696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":50,"seq":214748364846,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28548,"kb_used_data":1524,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716604,"statfs":{"total":95995035648,"available":95965802496,"internally_reserved":0,"allocated":1560576,"data_stored":1041405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528370,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36272,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708880,"statfs":{"total":95995035648,"available":95957893120,"internally_reserved":0,"allocated":950272,"data_stored":448617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790009,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28560,"kb_used_data":1536,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716592,"statfs":{"total":95995035648,"available":95965790208,"internally_reserved":0,"allocated":1572864,"data_stored":1041083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920830,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36272,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708880,"statfs":{"total":95995035648,"available":95957893120,"internally_reserved":0,"allocated":950272,"data_stored":448478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182467,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40448,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704704,"statfs":{"total":95995035648,"available":95953616896,"internally_reserved":0,"allocated":966656,"data_stored":448442,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411400,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36352,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708800,"statfs":{"total":95995035648,"available":95957811200,"internally_reserved":0,"allocated":1032192,"data_stored":509907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574925,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1011712,"data_stored":509294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738450,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":970752,"data_stored":450349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":61516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":62129,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":61500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T23:17:34.169 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-11T23:17:34.169 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-11T23:17:34.169 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-09-11T23:17:34.169 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph health --format=json 2024-09-11T23:17:34.433 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:17:34.864 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:17:34 smithi026 ceph-mon[31172]: from='client.14880 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T23:17:34.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:34.946+0000 7f2630019640 1 -- 172.21.15.26:0/384738155 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2628106f10 msgr2=0x7f262810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:34.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:34.946+0000 7f2630019640 1 --2- 172.21.15.26:0/384738155 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2628106f10 0x7f262810ea90 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f2620009ca0 tx=0x7f262002a8c0 comp rx=0 tx=0).stop 2024-09-11T23:17:34.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:34.946+0000 7f2630019640 1 -- 172.21.15.26:0/384738155 shutdown_connections 2024-09-11T23:17:34.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:34.946+0000 7f2630019640 1 --2- 172.21.15.26:0/384738155 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2628106f10 0x7f262810ea90 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:34.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:34.946+0000 7f2630019640 1 --2- 172.21.15.26:0/384738155 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2628102790 0x7f26281067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:34.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:34.946+0000 7f2630019640 1 --2- 172.21.15.26:0/384738155 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2628101de0 0x7f26281021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:34.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:34.946+0000 7f2630019640 1 -- 172.21.15.26:0/384738155 >> 172.21.15.26:0/384738155 conn(0x7f26280fd8c0 msgr2=0x7f26280ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:17:34.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:34.946+0000 7f2630019640 1 -- 172.21.15.26:0/384738155 shutdown_connections 2024-09-11T23:17:34.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:34.946+0000 7f2630019640 1 -- 172.21.15.26:0/384738155 wait complete. 2024-09-11T23:17:34.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:34.947+0000 7f2630019640 1 Processor -- start 2024-09-11T23:17:34.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:34.947+0000 7f2630019640 1 -- start start 2024-09-11T23:17:34.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:34.947+0000 7f2630019640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2628101de0 0x7f26280744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:34.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:34.947+0000 7f2630019640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2628102790 0x7f2628074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:34.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:34.947+0000 7f2630019640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2628106f10 0x7f2628074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:34.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:34.947+0000 7f2630019640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f2628111f90 con 0x7f2628102790 2024-09-11T23:17:34.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:34.947+0000 7f2630019640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f2628111e10 con 0x7f2628101de0 2024-09-11T23:17:34.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:34.947+0000 7f2630019640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f2628112110 con 0x7f2628106f10 2024-09-11T23:17:34.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:34.947+0000 7f262d5b4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2628102790 0x7f2628074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:34.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:34.947+0000 7f262d5b4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2628102790 0x7f2628074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47336/0 (socket says 172.21.15.26:47336) 2024-09-11T23:17:34.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:34.947+0000 7f262d5b4640 1 -- 172.21.15.26:0/125170372 learned_addr learned my addr 172.21.15.26:0/125170372 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:17:34.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:34.947+0000 7f262ddb5640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2628101de0 0x7f26280744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:34.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:34.947+0000 7f262e5b6640 1 --2- 172.21.15.26:0/125170372 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2628106f10 0x7f2628074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:34.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:34.947+0000 7f262d5b4640 1 -- 172.21.15.26:0/125170372 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2628106f10 msgr2=0x7f2628074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:34.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:34.947+0000 7f262d5b4640 1 --2- 172.21.15.26:0/125170372 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2628106f10 0x7f2628074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:34.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:34.947+0000 7f262d5b4640 1 -- 172.21.15.26:0/125170372 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2628101de0 msgr2=0x7f26280744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:34.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:34.947+0000 7f262d5b4640 1 --2- 172.21.15.26:0/125170372 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2628101de0 0x7f26280744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:34.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:34.947+0000 7f262d5b4640 1 -- 172.21.15.26:0/125170372 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26281a6930 con 0x7f2628102790 2024-09-11T23:17:34.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:34.948+0000 7f262d5b4640 1 --2- 172.21.15.26:0/125170372 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2628102790 0x7f2628074a30 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f261800b4a0 tx=0x7f261800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:17:34.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:34.948+0000 7f261effd640 1 -- 172.21.15.26:0/125170372 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2618014070 con 0x7f2628102790 2024-09-11T23:17:34.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:34.948+0000 7f2630019640 1 -- 172.21.15.26:0/125170372 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26281a6c20 con 0x7f2628102790 2024-09-11T23:17:34.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:34.948+0000 7f261effd640 1 -- 172.21.15.26:0/125170372 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2618010af0 con 0x7f2628102790 2024-09-11T23:17:34.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:34.948+0000 7f261effd640 1 -- 172.21.15.26:0/125170372 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2618017d10 con 0x7f2628102790 2024-09-11T23:17:34.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:34.948+0000 7f2630019640 1 -- 172.21.15.26:0/125170372 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26281a7110 con 0x7f2628102790 2024-09-11T23:17:34.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:34.949+0000 7f2630019640 1 -- 172.21.15.26:0/125170372 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26281021c0 con 0x7f2628102790 2024-09-11T23:17:34.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:34.950+0000 7f261effd640 1 -- 172.21.15.26:0/125170372 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f2618017070 con 0x7f2628102790 2024-09-11T23:17:34.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:34.951+0000 7f261effd640 1 --2- 172.21.15.26:0/125170372 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f26140a2c10 0x7f26140a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:34.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:34.951+0000 7f262ddb5640 1 --2- 172.21.15.26:0/125170372 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f26140a2c10 0x7f26140a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:34.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:34.951+0000 7f261effd640 1 -- 172.21.15.26:0/125170372 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f26180ce490 con 0x7f2628102790 2024-09-11T23:17:34.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:34.951+0000 7f262ddb5640 1 --2- 172.21.15.26:0/125170372 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f26140a2c10 0x7f26140a50d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f2624005d00 tx=0x7f262400b410 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:17:34.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:34.955+0000 7f261effd640 1 -- 172.21.15.26:0/125170372 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f261809b3a0 con 0x7f2628102790 2024-09-11T23:17:35.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:17:34 smithi076 ceph-mon[35296]: from='client.14880 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T23:17:35.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:35.170+0000 7f2630019640 1 -- 172.21.15.26:0/125170372 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f262810a0f0 con 0x7f2628102790 2024-09-11T23:17:35.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:35.171+0000 7f261effd640 1 -- 172.21.15.26:0/125170372 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f261809aca0 con 0x7f2628102790 2024-09-11T23:17:35.171 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-11T23:17:35.171 INFO:teuthology.orchestra.run.smithi026.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-11T23:17:35.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:35.173+0000 7f2630019640 1 -- 172.21.15.26:0/125170372 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f26140a2c10 msgr2=0x7f26140a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:35.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:35.173+0000 7f2630019640 1 --2- 172.21.15.26:0/125170372 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f26140a2c10 0x7f26140a50d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f2624005d00 tx=0x7f262400b410 comp rx=0 tx=0).stop 2024-09-11T23:17:35.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:35.173+0000 7f2630019640 1 -- 172.21.15.26:0/125170372 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2628102790 msgr2=0x7f2628074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:35.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:35.173+0000 7f2630019640 1 --2- 172.21.15.26:0/125170372 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2628102790 0x7f2628074a30 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f261800b4a0 tx=0x7f261800b960 comp rx=0 tx=0).stop 2024-09-11T23:17:35.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:35.173+0000 7f2630019640 1 -- 172.21.15.26:0/125170372 shutdown_connections 2024-09-11T23:17:35.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:35.173+0000 7f2630019640 1 --2- 172.21.15.26:0/125170372 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f26140a2c10 0x7f26140a50d0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:35.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:35.173+0000 7f2630019640 1 --2- 172.21.15.26:0/125170372 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2628106f10 0x7f2628074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:35.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:35.173+0000 7f2630019640 1 --2- 172.21.15.26:0/125170372 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2628102790 0x7f2628074a30 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:35.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:35.173+0000 7f2630019640 1 --2- 172.21.15.26:0/125170372 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2628101de0 0x7f26280744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:35.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:35.173+0000 7f2630019640 1 -- 172.21.15.26:0/125170372 >> 172.21.15.26:0/125170372 conn(0x7f26280fd8c0 msgr2=0x7f26280fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:17:35.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:35.173+0000 7f2630019640 1 -- 172.21.15.26:0/125170372 shutdown_connections 2024-09-11T23:17:35.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:35.173+0000 7f2630019640 1 -- 172.21.15.26:0/125170372 wait complete. 2024-09-11T23:17:35.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:17:34 smithi136 ceph-mon[35121]: from='client.14880 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T23:17:35.819 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:17:35 smithi026 ceph-mon[31172]: pgmap v315: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-11T23:17:35.819 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:17:35 smithi026 ceph-mon[31172]: from='client.? 172.21.15.26:0/125170372' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-11T23:17:35.850 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-09-11T23:17:35.851 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:17:36.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:17:35 smithi076 ceph-mon[35296]: pgmap v315: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-11T23:17:36.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:17:35 smithi076 ceph-mon[35296]: from='client.? 172.21.15.26:0/125170372' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-11T23:17:36.114 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:17:36.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:17:35 smithi136 ceph-mon[35121]: pgmap v315: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-11T23:17:36.418 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:17:35 smithi136 ceph-mon[35121]: from='client.? 172.21.15.26:0/125170372' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-11T23:17:36.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:36.579+0000 7fe13b075640 1 -- 172.21.15.26:0/3861280521 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe134102d40 msgr2=0x7fe134103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:36.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:36.579+0000 7fe13b075640 1 --2- 172.21.15.26:0/3861280521 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe134102d40 0x7fe134103120 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fe124009900 tx=0x7fe12402a480 comp rx=0 tx=0).stop 2024-09-11T23:17:36.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:36.579+0000 7fe13b075640 1 -- 172.21.15.26:0/3861280521 shutdown_connections 2024-09-11T23:17:36.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:36.579+0000 7fe13b075640 1 --2- 172.21.15.26:0/3861280521 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe134103660 0x7fe13410bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:36.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:36.579+0000 7fe13b075640 1 --2- 172.21.15.26:0/3861280521 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe13410c170 0x7fe13410e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:36.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:36.579+0000 7fe13b075640 1 --2- 172.21.15.26:0/3861280521 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe134102d40 0x7fe134103120 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:36.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:36.579+0000 7fe13b075640 1 -- 172.21.15.26:0/3861280521 >> 172.21.15.26:0/3861280521 conn(0x7fe1340fc7a0 msgr2=0x7fe1340febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:17:36.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:36.579+0000 7fe13b075640 1 -- 172.21.15.26:0/3861280521 shutdown_connections 2024-09-11T23:17:36.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:36.579+0000 7fe13b075640 1 -- 172.21.15.26:0/3861280521 wait complete. 2024-09-11T23:17:36.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:36.580+0000 7fe13b075640 1 Processor -- start 2024-09-11T23:17:36.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:36.580+0000 7fe13b075640 1 -- start start 2024-09-11T23:17:36.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:36.580+0000 7fe13b075640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe134102d40 0x7fe134198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:36.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:36.580+0000 7fe13b075640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe134103660 0x7fe1341989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:36.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:36.580+0000 7fe13b075640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe13410c170 0x7fe13419cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:36.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:36.580+0000 7fe13b075640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fe134111fb0 con 0x7fe134102d40 2024-09-11T23:17:36.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:36.580+0000 7fe13b075640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fe134111e30 con 0x7fe134103660 2024-09-11T23:17:36.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:36.580+0000 7fe13b075640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe134112130 con 0x7fe13410c170 2024-09-11T23:17:36.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:36.580+0000 7fe138e11640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe134102d40 0x7fe134198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:36.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:36.580+0000 7fe138e11640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe134102d40 0x7fe134198490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47348/0 (socket says 172.21.15.26:47348) 2024-09-11T23:17:36.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:36.580+0000 7fe138e11640 1 -- 172.21.15.26:0/1812373592 learned_addr learned my addr 172.21.15.26:0/1812373592 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:17:36.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:36.580+0000 7fe133fff640 1 --2- 172.21.15.26:0/1812373592 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe134103660 0x7fe1341989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:36.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:36.581+0000 7fe138e11640 1 -- 172.21.15.26:0/1812373592 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe13410c170 msgr2=0x7fe13419cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:36.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:36.581+0000 7fe138e11640 1 --2- 172.21.15.26:0/1812373592 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe13410c170 0x7fe13419cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:36.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:36.581+0000 7fe138e11640 1 -- 172.21.15.26:0/1812373592 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe134103660 msgr2=0x7fe1341989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:36.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:36.581+0000 7fe138e11640 1 --2- 172.21.15.26:0/1812373592 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe134103660 0x7fe1341989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:36.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:36.581+0000 7fe138e11640 1 -- 172.21.15.26:0/1812373592 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe13419d440 con 0x7fe134102d40 2024-09-11T23:17:36.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:36.581+0000 7fe133fff640 1 --2- 172.21.15.26:0/1812373592 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe134103660 0x7fe1341989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:17:36.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:36.581+0000 7fe138e11640 1 --2- 172.21.15.26:0/1812373592 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe134102d40 0x7fe134198490 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fe124006010 tx=0x7fe1240042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:17:36.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:36.581+0000 7fe131ffb640 1 -- 172.21.15.26:0/1812373592 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe12403f030 con 0x7fe134102d40 2024-09-11T23:17:36.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:36.581+0000 7fe13b075640 1 -- 172.21.15.26:0/1812373592 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe13419d6d0 con 0x7fe134102d40 2024-09-11T23:17:36.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:36.581+0000 7fe131ffb640 1 -- 172.21.15.26:0/1812373592 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe124003ce0 con 0x7fe134102d40 2024-09-11T23:17:36.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:36.581+0000 7fe131ffb640 1 -- 172.21.15.26:0/1812373592 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe124035070 con 0x7fe134102d40 2024-09-11T23:17:36.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:36.581+0000 7fe13b075640 1 -- 172.21.15.26:0/1812373592 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1341a5010 con 0x7fe134102d40 2024-09-11T23:17:36.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:36.583+0000 7fe13b075640 1 -- 172.21.15.26:0/1812373592 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe13406b070 con 0x7fe134102d40 2024-09-11T23:17:36.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:36.584+0000 7fe131ffb640 1 -- 172.21.15.26:0/1812373592 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fe124042030 con 0x7fe134102d40 2024-09-11T23:17:36.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:36.585+0000 7fe131ffb640 1 --2- 172.21.15.26:0/1812373592 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fe11c0a2c10 0x7fe11c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:36.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:36.585+0000 7fe133fff640 1 --2- 172.21.15.26:0/1812373592 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fe11c0a2c10 0x7fe11c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:36.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:36.585+0000 7fe131ffb640 1 -- 172.21.15.26:0/1812373592 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fe1240efba0 con 0x7fe134102d40 2024-09-11T23:17:36.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:36.586+0000 7fe133fff640 1 --2- 172.21.15.26:0/1812373592 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fe11c0a2c10 0x7fe11c0a50d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fe1280058c0 tx=0x7fe128005850 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:17:36.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:36.589+0000 7fe131ffb640 1 -- 172.21.15.26:0/1812373592 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe1240bca60 con 0x7fe134102d40 2024-09-11T23:17:36.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:36.800+0000 7fe13b075640 1 -- 172.21.15.26:0/1812373592 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe134103120 con 0x7fe134102d40 2024-09-11T23:17:36.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:36.800+0000 7fe131ffb640 1 -- 172.21.15.26:0/1812373592 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11971 (secure 0 0 0) 0x7fe134103120 con 0x7fe134102d40 2024-09-11T23:17:36.801 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-11T23:17:36.801 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":28,"btime":"2024-09-11T23:16:38:976987+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14499,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/367659515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":367659515},{"type":"v1","addr":"172.21.15.26:6835","nonce":367659515}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14511,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/2041065299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":2041065299},{"type":"v1","addr":"172.21.15.26:6837","nonce":2041065299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:16:26.544203+0000","modified":"2024-09-11T23:16:38.976970+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24485,"mds_1":24463,"mds_2":24458,"mds_3":24448,"mds_4":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14550":{"gid":14550,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6839/2233068305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":2233068305},{"type":"v1","addr":"172.21.15.26:6839","nonce":2233068305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14562":{"gid":14562,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6841/2984982336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":2984982336},{"type":"v1","addr":"172.21.15.26:6841","nonce":2984982336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6833/2019841455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":2019841455},{"type":"v1","addr":"172.21.15.136:6833","nonce":2019841455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.76:6833/3438147050","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3438147050},{"type":"v1","addr":"172.21.15.76:6833","nonce":3438147050}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6835/3617665975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3617665975},{"type":"v1","addr":"172.21.15.136:6835","nonce":3617665975}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":12,"addr":"172.21.15.76:6835/393308499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":393308499},{"type":"v1","addr":"172.21.15.76:6835","nonce":393308499}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":10,"addr":"172.21.15.136:6837/1633082387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":1633082387},{"type":"v1","addr":"172.21.15.136:6837","nonce":1633082387}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.76:6837/2078978748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":2078978748},{"type":"v1","addr":"172.21.15.76:6837","nonce":2078978748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.136:6839/3211398630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":3211398630},{"type":"v1","addr":"172.21.15.136:6839","nonce":3211398630}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.76:6839/2077637090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6838","nonce":2077637090},{"type":"v1","addr":"172.21.15.76:6839","nonce":2077637090}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24485,"qdb_cluster":[24485,24463,24458,24448,24443]},"id":1}]} 2024-09-11T23:17:36.801 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 28 2024-09-11T23:17:36.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:36.802+0000 7fe13b075640 1 -- 172.21.15.26:0/1812373592 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fe11c0a2c10 msgr2=0x7fe11c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:36.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:36.802+0000 7fe13b075640 1 --2- 172.21.15.26:0/1812373592 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fe11c0a2c10 0x7fe11c0a50d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fe1280058c0 tx=0x7fe128005850 comp rx=0 tx=0).stop 2024-09-11T23:17:36.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:36.802+0000 7fe13b075640 1 -- 172.21.15.26:0/1812373592 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe134102d40 msgr2=0x7fe134198490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:36.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:36.802+0000 7fe13b075640 1 --2- 172.21.15.26:0/1812373592 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe134102d40 0x7fe134198490 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fe124006010 tx=0x7fe1240042f0 comp rx=0 tx=0).stop 2024-09-11T23:17:36.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:36.803+0000 7fe13b075640 1 -- 172.21.15.26:0/1812373592 shutdown_connections 2024-09-11T23:17:36.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:36.803+0000 7fe13b075640 1 --2- 172.21.15.26:0/1812373592 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fe11c0a2c10 0x7fe11c0a50d0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:36.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:36.803+0000 7fe13b075640 1 --2- 172.21.15.26:0/1812373592 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe134103660 0x7fe1341989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:36.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:36.803+0000 7fe13b075640 1 --2- 172.21.15.26:0/1812373592 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe13410c170 0x7fe13419cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:36.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:36.803+0000 7fe13b075640 1 --2- 172.21.15.26:0/1812373592 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe134102d40 0x7fe134198490 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:36.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:36.803+0000 7fe13b075640 1 -- 172.21.15.26:0/1812373592 >> 172.21.15.26:0/1812373592 conn(0x7fe1340fc7a0 msgr2=0x7fe1340fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:17:36.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:36.803+0000 7fe13b075640 1 -- 172.21.15.26:0/1812373592 shutdown_connections 2024-09-11T23:17:36.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:36.803+0000 7fe13b075640 1 -- 172.21.15.26:0/1812373592 wait complete. 2024-09-11T23:17:37.473 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:17:37.732 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:17:38.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:17:37 smithi076 ceph-mon[35296]: pgmap v316: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-11T23:17:38.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:17:37 smithi076 ceph-mon[35296]: from='client.? 172.21.15.26:0/1812373592' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:17:38.112 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:17:37 smithi026 ceph-mon[31172]: pgmap v316: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-11T23:17:38.112 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:17:37 smithi026 ceph-mon[31172]: from='client.? 172.21.15.26:0/1812373592' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:17:38.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:38.193+0000 7f4519991640 1 -- 172.21.15.26:0/3768451797 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4514101420 msgr2=0x7f451410d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:38.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:38.193+0000 7f4519991640 1 --2- 172.21.15.26:0/3768451797 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4514101420 0x7f451410d950 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f450c009cd0 tx=0x7f450c02a8e0 comp rx=0 tx=0).stop 2024-09-11T23:17:38.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:38.194+0000 7f4519991640 1 -- 172.21.15.26:0/3768451797 shutdown_connections 2024-09-11T23:17:38.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:38.194+0000 7f4519991640 1 --2- 172.21.15.26:0/3768451797 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4514100a80 0x7f4514100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:38.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:38.194+0000 7f4519991640 1 --2- 172.21.15.26:0/3768451797 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4514101420 0x7f451410d950 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:38.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:38.194+0000 7f4519991640 1 --2- 172.21.15.26:0/3768451797 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4514106a80 0x7f4514106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:38.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:38.194+0000 7f4519991640 1 -- 172.21.15.26:0/3768451797 >> 172.21.15.26:0/3768451797 conn(0x7f45140fc7a0 msgr2=0x7f45140febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:17:38.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:38.194+0000 7f4519991640 1 -- 172.21.15.26:0/3768451797 shutdown_connections 2024-09-11T23:17:38.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:38.194+0000 7f4519991640 1 -- 172.21.15.26:0/3768451797 wait complete. 2024-09-11T23:17:38.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:38.194+0000 7f4519991640 1 Processor -- start 2024-09-11T23:17:38.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:38.194+0000 7f4519991640 1 -- start start 2024-09-11T23:17:38.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:38.194+0000 7f4519991640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4514100a80 0x7f4514102650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:38.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:38.194+0000 7f4519991640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4514101420 0x7f4514102b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:38.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:38.194+0000 7f4519991640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4514106a80 0x7f45141030d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:38.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:38.194+0000 7f4519991640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f4514110e80 con 0x7f4514101420 2024-09-11T23:17:38.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:38.194+0000 7f4519991640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f4514110d00 con 0x7f4514100a80 2024-09-11T23:17:38.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:38.194+0000 7f4519991640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f4514111000 con 0x7f4514106a80 2024-09-11T23:17:38.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:38.195+0000 7f450bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4514101420 0x7f4514102b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:38.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:38.195+0000 7f450bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4514101420 0x7f4514102b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47370/0 (socket says 172.21.15.26:47370) 2024-09-11T23:17:38.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:38.195+0000 7f450bfff640 1 -- 172.21.15.26:0/2060295164 learned_addr learned my addr 172.21.15.26:0/2060295164 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:17:38.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:38.195+0000 7f45137fe640 1 --2- 172.21.15.26:0/2060295164 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4514106a80 0x7f45141030d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:38.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:38.195+0000 7f4512ffd640 1 --2- 172.21.15.26:0/2060295164 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4514100a80 0x7f4514102650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:38.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:38.195+0000 7f450bfff640 1 -- 172.21.15.26:0/2060295164 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4514106a80 msgr2=0x7f45141030d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:38.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:38.195+0000 7f450bfff640 1 --2- 172.21.15.26:0/2060295164 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4514106a80 0x7f45141030d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:38.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:38.195+0000 7f450bfff640 1 -- 172.21.15.26:0/2060295164 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4514100a80 msgr2=0x7f4514102650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:38.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:38.195+0000 7f450bfff640 1 --2- 172.21.15.26:0/2060295164 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4514100a80 0x7f4514102650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:38.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:38.195+0000 7f450bfff640 1 -- 172.21.15.26:0/2060295164 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45141a6890 con 0x7f4514101420 2024-09-11T23:17:38.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:38.195+0000 7f4512ffd640 1 --2- 172.21.15.26:0/2060295164 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4514100a80 0x7f4514102650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:17:38.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:38.195+0000 7f450bfff640 1 --2- 172.21.15.26:0/2060295164 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4514101420 0x7f4514102b90 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f44fc00c890 tx=0x7f44fc00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:17:38.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:38.195+0000 7f451898f640 1 -- 172.21.15.26:0/2060295164 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44fc004220 con 0x7f4514101420 2024-09-11T23:17:38.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:38.195+0000 7f4519991640 1 -- 172.21.15.26:0/2060295164 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45141a6b80 con 0x7f4514101420 2024-09-11T23:17:38.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:38.195+0000 7f451898f640 1 -- 172.21.15.26:0/2060295164 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f44fc0043c0 con 0x7f4514101420 2024-09-11T23:17:38.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:38.195+0000 7f451898f640 1 -- 172.21.15.26:0/2060295164 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44fc0049e0 con 0x7f4514101420 2024-09-11T23:17:38.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:38.195+0000 7f4519991640 1 -- 172.21.15.26:0/2060295164 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45141a70c0 con 0x7f4514101420 2024-09-11T23:17:38.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:38.196+0000 7f4519991640 1 -- 172.21.15.26:0/2060295164 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4514100ee0 con 0x7f4514101420 2024-09-11T23:17:38.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:38.198+0000 7f451898f640 1 -- 172.21.15.26:0/2060295164 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f44fc0079b0 con 0x7f4514101420 2024-09-11T23:17:38.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:38.198+0000 7f451898f640 1 --2- 172.21.15.26:0/2060295164 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f45000a2c10 0x7f45000a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:38.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:38.198+0000 7f451898f640 1 -- 172.21.15.26:0/2060295164 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f44fc0cf2e0 con 0x7f4514101420 2024-09-11T23:17:38.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:38.198+0000 7f4512ffd640 1 --2- 172.21.15.26:0/2060295164 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f45000a2c10 0x7f45000a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:38.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:38.199+0000 7f4512ffd640 1 --2- 172.21.15.26:0/2060295164 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f45000a2c10 0x7f45000a50d0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f45040058c0 tx=0x7f4504005cb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:17:38.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:38.201+0000 7f451898f640 1 -- 172.21.15.26:0/2060295164 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f44fc014030 con 0x7f4514101420 2024-09-11T23:17:38.240 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:17:37 smithi136 ceph-mon[35121]: pgmap v316: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-11T23:17:38.240 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:17:37 smithi136 ceph-mon[35121]: from='client.? 172.21.15.26:0/1812373592' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:17:38.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:38.412+0000 7f4519991640 1 -- 172.21.15.26:0/2060295164 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4514103b80 con 0x7f4514101420 2024-09-11T23:17:38.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:38.412+0000 7f451898f640 1 -- 172.21.15.26:0/2060295164 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11971 (secure 0 0 0) 0x7f4514103b80 con 0x7f4514101420 2024-09-11T23:17:38.413 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-11T23:17:38.413 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":28,"btime":"2024-09-11T23:16:38:976987+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14499,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/367659515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":367659515},{"type":"v1","addr":"172.21.15.26:6835","nonce":367659515}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14511,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/2041065299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":2041065299},{"type":"v1","addr":"172.21.15.26:6837","nonce":2041065299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:16:26.544203+0000","modified":"2024-09-11T23:16:38.976970+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24485,"mds_1":24463,"mds_2":24458,"mds_3":24448,"mds_4":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14550":{"gid":14550,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6839/2233068305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":2233068305},{"type":"v1","addr":"172.21.15.26:6839","nonce":2233068305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14562":{"gid":14562,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6841/2984982336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":2984982336},{"type":"v1","addr":"172.21.15.26:6841","nonce":2984982336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6833/2019841455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":2019841455},{"type":"v1","addr":"172.21.15.136:6833","nonce":2019841455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.76:6833/3438147050","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3438147050},{"type":"v1","addr":"172.21.15.76:6833","nonce":3438147050}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6835/3617665975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3617665975},{"type":"v1","addr":"172.21.15.136:6835","nonce":3617665975}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":12,"addr":"172.21.15.76:6835/393308499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":393308499},{"type":"v1","addr":"172.21.15.76:6835","nonce":393308499}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":10,"addr":"172.21.15.136:6837/1633082387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":1633082387},{"type":"v1","addr":"172.21.15.136:6837","nonce":1633082387}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.76:6837/2078978748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":2078978748},{"type":"v1","addr":"172.21.15.76:6837","nonce":2078978748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.136:6839/3211398630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":3211398630},{"type":"v1","addr":"172.21.15.136:6839","nonce":3211398630}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.76:6839/2077637090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6838","nonce":2077637090},{"type":"v1","addr":"172.21.15.76:6839","nonce":2077637090}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24485,"qdb_cluster":[24485,24463,24458,24448,24443]},"id":1}]} 2024-09-11T23:17:38.413 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 28 2024-09-11T23:17:38.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:38.414+0000 7f4519991640 1 -- 172.21.15.26:0/2060295164 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f45000a2c10 msgr2=0x7f45000a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:38.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:38.414+0000 7f4519991640 1 --2- 172.21.15.26:0/2060295164 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f45000a2c10 0x7f45000a50d0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f45040058c0 tx=0x7f4504005cb0 comp rx=0 tx=0).stop 2024-09-11T23:17:38.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:38.414+0000 7f4519991640 1 -- 172.21.15.26:0/2060295164 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4514101420 msgr2=0x7f4514102b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:38.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:38.414+0000 7f4519991640 1 --2- 172.21.15.26:0/2060295164 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4514101420 0x7f4514102b90 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f44fc00c890 tx=0x7f44fc00cd50 comp rx=0 tx=0).stop 2024-09-11T23:17:38.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:38.414+0000 7f4519991640 1 -- 172.21.15.26:0/2060295164 shutdown_connections 2024-09-11T23:17:38.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:38.414+0000 7f4519991640 1 --2- 172.21.15.26:0/2060295164 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f45000a2c10 0x7f45000a50d0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:38.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:38.414+0000 7f4519991640 1 --2- 172.21.15.26:0/2060295164 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4514106a80 0x7f45141030d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:38.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:38.414+0000 7f4519991640 1 --2- 172.21.15.26:0/2060295164 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4514101420 0x7f4514102b90 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:38.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:38.414+0000 7f4519991640 1 --2- 172.21.15.26:0/2060295164 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4514100a80 0x7f4514102650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:38.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:38.414+0000 7f4519991640 1 -- 172.21.15.26:0/2060295164 >> 172.21.15.26:0/2060295164 conn(0x7f45140fc7a0 msgr2=0x7f451406a830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:17:38.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:38.414+0000 7f4519991640 1 -- 172.21.15.26:0/2060295164 shutdown_connections 2024-09-11T23:17:38.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:38.415+0000 7f4519991640 1 -- 172.21.15.26:0/2060295164 wait complete. 2024-09-11T23:17:39.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:17:38 smithi076 ceph-mon[35296]: from='client.? 172.21.15.26:0/2060295164' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:17:39.063 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:17:38 smithi026 ceph-mon[31172]: from='client.? 172.21.15.26:0/2060295164' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:17:39.065 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-11T23:17:39.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:17:38 smithi136 ceph-mon[35121]: from='client.? 172.21.15.26:0/2060295164' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:17:39.322 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:17:39.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:39.792+0000 7efc999a2640 1 -- 172.21.15.26:0/860362129 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc940ffb60 msgr2=0x7efc940fffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:39.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:39.792+0000 7efc999a2640 1 --2- 172.21.15.26:0/860362129 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc940ffb60 0x7efc940fffc0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7efc7c009900 tx=0x7efc7c02a410 comp rx=0 tx=0).stop 2024-09-11T23:17:39.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:39.792+0000 7efc999a2640 1 -- 172.21.15.26:0/860362129 shutdown_connections 2024-09-11T23:17:39.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:39.792+0000 7efc999a2640 1 --2- 172.21.15.26:0/860362129 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efc94100500 0x7efc9410ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:39.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:39.792+0000 7efc999a2640 1 --2- 172.21.15.26:0/860362129 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc940ffb60 0x7efc940fffc0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:39.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:39.792+0000 7efc999a2640 1 --2- 172.21.15.26:0/860362129 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efc94106b80 0x7efc94106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:39.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:39.792+0000 7efc999a2640 1 -- 172.21.15.26:0/860362129 >> 172.21.15.26:0/860362129 conn(0x7efc940fba90 msgr2=0x7efc940fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:17:39.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:39.792+0000 7efc999a2640 1 -- 172.21.15.26:0/860362129 shutdown_connections 2024-09-11T23:17:39.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:39.792+0000 7efc999a2640 1 -- 172.21.15.26:0/860362129 wait complete. 2024-09-11T23:17:39.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:39.793+0000 7efc999a2640 1 Processor -- start 2024-09-11T23:17:39.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:39.793+0000 7efc999a2640 1 -- start start 2024-09-11T23:17:39.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:39.793+0000 7efc999a2640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efc940ffb60 0x7efc94198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:39.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:39.793+0000 7efc999a2640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efc94100500 0x7efc94198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:39.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:39.793+0000 7efc999a2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc94106b80 0x7efc9419cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:39.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:39.793+0000 7efc999a2640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7efc94111f60 con 0x7efc94106b80 2024-09-11T23:17:39.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:39.793+0000 7efc999a2640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7efc94111de0 con 0x7efc940ffb60 2024-09-11T23:17:39.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:39.793+0000 7efc999a2640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7efc941120e0 con 0x7efc94100500 2024-09-11T23:17:39.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:39.793+0000 7efc937fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc94106b80 0x7efc9419cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:39.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:39.793+0000 7efc937fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc94106b80 0x7efc9419cd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47388/0 (socket says 172.21.15.26:47388) 2024-09-11T23:17:39.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:39.793+0000 7efc937fe640 1 -- 172.21.15.26:0/4052305603 learned_addr learned my addr 172.21.15.26:0/4052305603 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:17:39.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:39.793+0000 7efc92ffd640 1 --2- 172.21.15.26:0/4052305603 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efc940ffb60 0x7efc94198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:39.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:39.793+0000 7efc8bfff640 1 --2- 172.21.15.26:0/4052305603 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efc94100500 0x7efc94198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:39.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:39.794+0000 7efc937fe640 1 -- 172.21.15.26:0/4052305603 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efc94100500 msgr2=0x7efc94198980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:39.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:39.794+0000 7efc937fe640 1 --2- 172.21.15.26:0/4052305603 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efc94100500 0x7efc94198980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:39.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:39.794+0000 7efc937fe640 1 -- 172.21.15.26:0/4052305603 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efc940ffb60 msgr2=0x7efc94198440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:39.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:39.794+0000 7efc937fe640 1 --2- 172.21.15.26:0/4052305603 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efc940ffb60 0x7efc94198440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:39.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:39.794+0000 7efc937fe640 1 -- 172.21.15.26:0/4052305603 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc9419d3f0 con 0x7efc94106b80 2024-09-11T23:17:39.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:39.794+0000 7efc937fe640 1 --2- 172.21.15.26:0/4052305603 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc94106b80 0x7efc9419cd10 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7efc8c00e9d0 tx=0x7efc8c00ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:17:39.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:39.794+0000 7efc989a0640 1 -- 172.21.15.26:0/4052305603 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc8c00cd80 con 0x7efc94106b80 2024-09-11T23:17:39.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:39.794+0000 7efc999a2640 1 -- 172.21.15.26:0/4052305603 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc9419d6e0 con 0x7efc94106b80 2024-09-11T23:17:39.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:39.794+0000 7efc989a0640 1 -- 172.21.15.26:0/4052305603 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc8c005020 con 0x7efc94106b80 2024-09-11T23:17:39.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:39.794+0000 7efc989a0640 1 -- 172.21.15.26:0/4052305603 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc8c0116d0 con 0x7efc94106b80 2024-09-11T23:17:39.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:39.794+0000 7efc999a2640 1 -- 172.21.15.26:0/4052305603 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc941a4fc0 con 0x7efc94106b80 2024-09-11T23:17:39.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:39.795+0000 7efc999a2640 1 -- 172.21.15.26:0/4052305603 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc94101080 con 0x7efc94106b80 2024-09-11T23:17:39.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:39.796+0000 7efc989a0640 1 -- 172.21.15.26:0/4052305603 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7efc8c0051d0 con 0x7efc94106b80 2024-09-11T23:17:39.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:39.797+0000 7efc989a0640 1 --2- 172.21.15.26:0/4052305603 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7efc800a2c10 0x7efc800a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:39.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:39.797+0000 7efc92ffd640 1 --2- 172.21.15.26:0/4052305603 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7efc800a2c10 0x7efc800a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:39.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:39.797+0000 7efc989a0640 1 -- 172.21.15.26:0/4052305603 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7efc8c0cf6f0 con 0x7efc94106b80 2024-09-11T23:17:39.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:39.798+0000 7efc92ffd640 1 --2- 172.21.15.26:0/4052305603 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7efc800a2c10 0x7efc800a50d0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7efc84005d00 tx=0x7efc8400b410 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:17:39.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:39.801+0000 7efc989a0640 1 -- 172.21.15.26:0/4052305603 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7efc8c016030 con 0x7efc94106b80 2024-09-11T23:17:39.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:39.979+0000 7efc999a2640 1 -- 172.21.15.26:0/4052305603 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7efc94109b50 con 0x7efc94106b80 2024-09-11T23:17:39.979 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:17:39 smithi026 ceph-mon[31172]: pgmap v317: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T23:17:39.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:39.981+0000 7efc989a0640 1 -- 172.21.15.26:0/4052305603 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+22733 (secure 0 0 0) 0x7efc8c09be60 con 0x7efc94106b80 2024-09-11T23:17:39.982 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-11T23:17:39.982 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":85,"fsid":"af7e6fc8-7092-11ef-bcea-c7b262605968","created":"2024-09-11T23:08:40.486438+0000","modified":"2024-09-11T23:16:35.935151+0000","last_up_change":"2024-09-11T23:15:35.179446+0000","last_in_change":"2024-09-11T23:15:11.064627+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T23:11:50.494160+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-11T23:16:22.503928+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-11T23:16:24.876554+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a4d62fcf-c31e-41b1-932c-3d87ff640531","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":2014857905},{"type":"v1","addr":"172.21.15.26:6803","nonce":2014857905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":2014857905},{"type":"v1","addr":"172.21.15.26:6805","nonce":2014857905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":2014857905},{"type":"v1","addr":"172.21.15.26:6809","nonce":2014857905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":2014857905},{"type":"v1","addr":"172.21.15.26:6807","nonce":2014857905}]},"public_addr":"172.21.15.26:6803/2014857905","cluster_addr":"172.21.15.26:6805/2014857905","heartbeat_back_addr":"172.21.15.26:6809/2014857905","heartbeat_front_addr":"172.21.15.26:6807/2014857905","state":["exists","up"]},{"osd":1,"uuid":"e0d69112-3c0a-4fc5-8783-e41b4081e0c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":333383087},{"type":"v1","addr":"172.21.15.76:6801","nonce":333383087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":333383087},{"type":"v1","addr":"172.21.15.76:6803","nonce":333383087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":333383087},{"type":"v1","addr":"172.21.15.76:6807","nonce":333383087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":333383087},{"type":"v1","addr":"172.21.15.76:6805","nonce":333383087}]},"public_addr":"172.21.15.76:6801/333383087","cluster_addr":"172.21.15.76:6803/333383087","heartbeat_back_addr":"172.21.15.76:6807/333383087","heartbeat_front_addr":"172.21.15.76:6805/333383087","state":["exists","up"]},{"osd":2,"uuid":"d578429e-bf23-4a1e-890f-7172955c0eb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":1271538911},{"type":"v1","addr":"172.21.15.136:6801","nonce":1271538911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":1271538911},{"type":"v1","addr":"172.21.15.136:6803","nonce":1271538911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":1271538911},{"type":"v1","addr":"172.21.15.136:6807","nonce":1271538911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":1271538911},{"type":"v1","addr":"172.21.15.136:6805","nonce":1271538911}]},"public_addr":"172.21.15.136:6801/1271538911","cluster_addr":"172.21.15.136:6803/1271538911","heartbeat_back_addr":"172.21.15.136:6807/1271538911","heartbeat_front_addr":"172.21.15.136:6805/1271538911","state":["exists","up"]},{"osd":3,"uuid":"3e34623f-0ea5-4b10-b8a9-f8756c35d512","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":1740304151},{"type":"v1","addr":"172.21.15.26:6811","nonce":1740304151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":1740304151},{"type":"v1","addr":"172.21.15.26:6813","nonce":1740304151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1740304151},{"type":"v1","addr":"172.21.15.26:6817","nonce":1740304151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":1740304151},{"type":"v1","addr":"172.21.15.26:6815","nonce":1740304151}]},"public_addr":"172.21.15.26:6811/1740304151","cluster_addr":"172.21.15.26:6813/1740304151","heartbeat_back_addr":"172.21.15.26:6817/1740304151","heartbeat_front_addr":"172.21.15.26:6815/1740304151","state":["exists","up"]},{"osd":4,"uuid":"77607929-f9ed-49a9-abb5-48cd88eec4ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":2702262182},{"type":"v1","addr":"172.21.15.76:6809","nonce":2702262182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":2702262182},{"type":"v1","addr":"172.21.15.76:6811","nonce":2702262182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":2702262182},{"type":"v1","addr":"172.21.15.76:6815","nonce":2702262182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":2702262182},{"type":"v1","addr":"172.21.15.76:6813","nonce":2702262182}]},"public_addr":"172.21.15.76:6809/2702262182","cluster_addr":"172.21.15.76:6811/2702262182","heartbeat_back_addr":"172.21.15.76:6815/2702262182","heartbeat_front_addr":"172.21.15.76:6813/2702262182","state":["exists","up"]},{"osd":5,"uuid":"29e20e37-09a0-4924-8aaa-17939763a7c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":1749229277},{"type":"v1","addr":"172.21.15.136:6809","nonce":1749229277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":1749229277},{"type":"v1","addr":"172.21.15.136:6811","nonce":1749229277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":1749229277},{"type":"v1","addr":"172.21.15.136:6815","nonce":1749229277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":1749229277},{"type":"v1","addr":"172.21.15.136:6813","nonce":1749229277}]},"public_addr":"172.21.15.136:6809/1749229277","cluster_addr":"172.21.15.136:6811/1749229277","heartbeat_back_addr":"172.21.15.136:6815/1749229277","heartbeat_front_addr":"172.21.15.136:6813/1749229277","state":["exists","up"]},{"osd":6,"uuid":"e0ddd1c0-1c46-41e6-b2f4-396bdcd2bf8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1928245624},{"type":"v1","addr":"172.21.15.26:6819","nonce":1928245624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1928245624},{"type":"v1","addr":"172.21.15.26:6821","nonce":1928245624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":1928245624},{"type":"v1","addr":"172.21.15.26:6825","nonce":1928245624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1928245624},{"type":"v1","addr":"172.21.15.26:6823","nonce":1928245624}]},"public_addr":"172.21.15.26:6819/1928245624","cluster_addr":"172.21.15.26:6821/1928245624","heartbeat_back_addr":"172.21.15.26:6825/1928245624","heartbeat_front_addr":"172.21.15.26:6823/1928245624","state":["exists","up"]},{"osd":7,"uuid":"fe031f5c-d915-48fb-91a6-c76f252467fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":1409789455},{"type":"v1","addr":"172.21.15.76:6817","nonce":1409789455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":1409789455},{"type":"v1","addr":"172.21.15.76:6819","nonce":1409789455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":1409789455},{"type":"v1","addr":"172.21.15.76:6823","nonce":1409789455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":1409789455},{"type":"v1","addr":"172.21.15.76:6821","nonce":1409789455}]},"public_addr":"172.21.15.76:6817/1409789455","cluster_addr":"172.21.15.76:6819/1409789455","heartbeat_back_addr":"172.21.15.76:6823/1409789455","heartbeat_front_addr":"172.21.15.76:6821/1409789455","state":["exists","up"]},{"osd":8,"uuid":"18851bde-9e6c-4e05-b4bf-ac5c3073eee7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":543619079},{"type":"v1","addr":"172.21.15.136:6817","nonce":543619079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":543619079},{"type":"v1","addr":"172.21.15.136:6819","nonce":543619079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":543619079},{"type":"v1","addr":"172.21.15.136:6823","nonce":543619079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":543619079},{"type":"v1","addr":"172.21.15.136:6821","nonce":543619079}]},"public_addr":"172.21.15.136:6817/543619079","cluster_addr":"172.21.15.136:6819/543619079","heartbeat_back_addr":"172.21.15.136:6823/543619079","heartbeat_front_addr":"172.21.15.136:6821/543619079","state":["exists","up"]},{"osd":9,"uuid":"52a65eab-8446-47ad-bfe7-5133f7e3eb98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":2201297064},{"type":"v1","addr":"172.21.15.26:6827","nonce":2201297064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":2201297064},{"type":"v1","addr":"172.21.15.26:6829","nonce":2201297064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":2201297064},{"type":"v1","addr":"172.21.15.26:6833","nonce":2201297064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":2201297064},{"type":"v1","addr":"172.21.15.26:6831","nonce":2201297064}]},"public_addr":"172.21.15.26:6827/2201297064","cluster_addr":"172.21.15.26:6829/2201297064","heartbeat_back_addr":"172.21.15.26:6833/2201297064","heartbeat_front_addr":"172.21.15.26:6831/2201297064","state":["exists","up"]},{"osd":10,"uuid":"b758d4bb-da62-4c5f-a992-bb6a145d35d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":3184128096},{"type":"v1","addr":"172.21.15.76:6825","nonce":3184128096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":3184128096},{"type":"v1","addr":"172.21.15.76:6827","nonce":3184128096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":3184128096},{"type":"v1","addr":"172.21.15.76:6831","nonce":3184128096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":3184128096},{"type":"v1","addr":"172.21.15.76:6829","nonce":3184128096}]},"public_addr":"172.21.15.76:6825/3184128096","cluster_addr":"172.21.15.76:6827/3184128096","heartbeat_back_addr":"172.21.15.76:6831/3184128096","heartbeat_front_addr":"172.21.15.76:6829/3184128096","state":["exists","up"]},{"osd":11,"uuid":"8fddda46-e22b-4d0c-ac48-e038fb29dfec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":1960689685},{"type":"v1","addr":"172.21.15.136:6825","nonce":1960689685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":1960689685},{"type":"v1","addr":"172.21.15.136:6827","nonce":1960689685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":1960689685},{"type":"v1","addr":"172.21.15.136:6831","nonce":1960689685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":1960689685},{"type":"v1","addr":"172.21.15.136:6829","nonce":1960689685}]},"public_addr":"172.21.15.136:6825/1960689685","cluster_addr":"172.21.15.136:6827/1960689685","heartbeat_back_addr":"172.21.15.136:6831/1960689685","heartbeat_front_addr":"172.21.15.136:6829/1960689685","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:10:56.656177+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:11:22.234031+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:11:46.464102+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:12:12.353967+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:12:37.836200+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:13:01.728839+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:13:27.527230+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:13:51.981393+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:14:17.035464+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:14:42.805029+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:15:08.208999+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:15:32.800844+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.18","mappings":[{"from":3,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":6}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.26:6801/677148151":"2024-09-12T23:09:33.983310+0000","172.21.15.26:6800/677148151":"2024-09-12T23:09:33.983310+0000","172.21.15.26:0/3597996003":"2024-09-12T23:09:33.983310+0000","172.21.15.26:6801/2673575521":"2024-09-12T23:09:09.781665+0000","172.21.15.26:0/883212708":"2024-09-12T23:09:33.983310+0000","172.21.15.26:0/2107281413":"2024-09-12T23:09:33.983310+0000","172.21.15.26:6800/2673575521":"2024-09-12T23:09:09.781665+0000","172.21.15.26:0/2817092653":"2024-09-12T23:09:09.781665+0000","172.21.15.26:0/17142658":"2024-09-12T23:09:09.781665+0000","172.21.15.26:0/3861103130":"2024-09-12T23:09:09.781665+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T23:17:39.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:39.983+0000 7efc999a2640 1 -- 172.21.15.26:0/4052305603 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7efc800a2c10 msgr2=0x7efc800a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:39.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:39.983+0000 7efc999a2640 1 --2- 172.21.15.26:0/4052305603 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7efc800a2c10 0x7efc800a50d0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7efc84005d00 tx=0x7efc8400b410 comp rx=0 tx=0).stop 2024-09-11T23:17:39.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:39.984+0000 7efc999a2640 1 -- 172.21.15.26:0/4052305603 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc94106b80 msgr2=0x7efc9419cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:39.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:39.984+0000 7efc999a2640 1 --2- 172.21.15.26:0/4052305603 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc94106b80 0x7efc9419cd10 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7efc8c00e9d0 tx=0x7efc8c00ee90 comp rx=0 tx=0).stop 2024-09-11T23:17:39.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:39.984+0000 7efc999a2640 1 -- 172.21.15.26:0/4052305603 shutdown_connections 2024-09-11T23:17:39.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:39.984+0000 7efc999a2640 1 --2- 172.21.15.26:0/4052305603 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7efc800a2c10 0x7efc800a50d0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:39.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:39.984+0000 7efc999a2640 1 --2- 172.21.15.26:0/4052305603 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc94106b80 0x7efc9419cd10 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:39.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:39.984+0000 7efc999a2640 1 --2- 172.21.15.26:0/4052305603 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efc94100500 0x7efc94198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:39.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:39.984+0000 7efc999a2640 1 --2- 172.21.15.26:0/4052305603 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efc940ffb60 0x7efc94198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:39.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:39.984+0000 7efc999a2640 1 -- 172.21.15.26:0/4052305603 >> 172.21.15.26:0/4052305603 conn(0x7efc940fba90 msgr2=0x7efc940fc3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:17:39.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:39.984+0000 7efc999a2640 1 -- 172.21.15.26:0/4052305603 shutdown_connections 2024-09-11T23:17:39.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:39.984+0000 7efc999a2640 1 -- 172.21.15.26:0/4052305603 wait complete. 2024-09-11T23:17:40.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:17:39 smithi076 ceph-mon[35296]: pgmap v317: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T23:17:40.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:17:39 smithi136 ceph-mon[35121]: pgmap v317: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T23:17:40.695 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:17:40.960 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:17:41.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:17:40 smithi026 ceph-mon[31172]: from='client.? 172.21.15.26:0/4052305603' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T23:17:41.052 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:17:40 smithi076 ceph-mon[35296]: from='client.? 172.21.15.26:0/4052305603' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T23:17:41.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:17:40 smithi136 ceph-mon[35121]: from='client.? 172.21.15.26:0/4052305603' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T23:17:41.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:41.402+0000 7f17d1766640 1 -- 172.21.15.26:0/599536610 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f17cc101de0 msgr2=0x7f17cc1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:41.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:41.402+0000 7f17d1766640 1 --2- 172.21.15.26:0/599536610 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f17cc101de0 0x7f17cc1021c0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f17bc009900 tx=0x7f17bc02a410 comp rx=0 tx=0).stop 2024-09-11T23:17:41.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:41.402+0000 7f17d1766640 1 -- 172.21.15.26:0/599536610 shutdown_connections 2024-09-11T23:17:41.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:41.402+0000 7f17d1766640 1 --2- 172.21.15.26:0/599536610 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f17cc102790 0x7f17cc1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:41.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:41.402+0000 7f17d1766640 1 --2- 172.21.15.26:0/599536610 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f17cc106f10 0x7f17cc10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:41.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:41.402+0000 7f17d1766640 1 --2- 172.21.15.26:0/599536610 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f17cc101de0 0x7f17cc1021c0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:41.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:41.402+0000 7f17d1766640 1 -- 172.21.15.26:0/599536610 >> 172.21.15.26:0/599536610 conn(0x7f17cc0fd8c0 msgr2=0x7f17cc0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:17:41.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:41.402+0000 7f17d1766640 1 -- 172.21.15.26:0/599536610 shutdown_connections 2024-09-11T23:17:41.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:41.402+0000 7f17d1766640 1 -- 172.21.15.26:0/599536610 wait complete. 2024-09-11T23:17:41.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:41.403+0000 7f17d1766640 1 Processor -- start 2024-09-11T23:17:41.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:41.403+0000 7f17d1766640 1 -- start start 2024-09-11T23:17:41.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:41.403+0000 7f17d1766640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f17cc101de0 0x7f17cc10deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:41.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:41.403+0000 7f17d1766640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f17cc102790 0x7f17cc10e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:41.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:41.403+0000 7f17d1766640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f17cc106f10 0x7f17cc1a34e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:41.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:41.403+0000 7f17d1766640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f17cc112010 con 0x7f17cc106f10 2024-09-11T23:17:41.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:41.403+0000 7f17d1766640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f17cc111e90 con 0x7f17cc101de0 2024-09-11T23:17:41.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:41.403+0000 7f17d1766640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f17cc112190 con 0x7f17cc102790 2024-09-11T23:17:41.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:41.403+0000 7f17cb7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f17cc106f10 0x7f17cc1a34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:41.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:41.404+0000 7f17cb7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f17cc106f10 0x7f17cc1a34e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59656/0 (socket says 172.21.15.26:59656) 2024-09-11T23:17:41.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:41.404+0000 7f17cb7fe640 1 -- 172.21.15.26:0/1216163118 learned_addr learned my addr 172.21.15.26:0/1216163118 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:17:41.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:41.404+0000 7f17ca7fc640 1 --2- 172.21.15.26:0/1216163118 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f17cc102790 0x7f17cc10e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:41.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:41.404+0000 7f17caffd640 1 --2- 172.21.15.26:0/1216163118 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f17cc101de0 0x7f17cc10deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:41.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:41.404+0000 7f17cb7fe640 1 -- 172.21.15.26:0/1216163118 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f17cc102790 msgr2=0x7f17cc10e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:41.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:41.404+0000 7f17cb7fe640 1 --2- 172.21.15.26:0/1216163118 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f17cc102790 0x7f17cc10e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:41.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:41.404+0000 7f17cb7fe640 1 -- 172.21.15.26:0/1216163118 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f17cc101de0 msgr2=0x7f17cc10deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:41.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:41.404+0000 7f17cb7fe640 1 --2- 172.21.15.26:0/1216163118 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f17cc101de0 0x7f17cc10deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:41.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:41.404+0000 7f17cb7fe640 1 -- 172.21.15.26:0/1216163118 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17cc1a3c10 con 0x7f17cc106f10 2024-09-11T23:17:41.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:41.404+0000 7f17caffd640 1 --2- 172.21.15.26:0/1216163118 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f17cc101de0 0x7f17cc10deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:17:41.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:41.404+0000 7f17cb7fe640 1 --2- 172.21.15.26:0/1216163118 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f17cc106f10 0x7f17cc1a34e0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f17c400b970 tx=0x7f17c400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:17:41.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:41.404+0000 7f17b3fff640 1 -- 172.21.15.26:0/1216163118 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17c4019070 con 0x7f17cc106f10 2024-09-11T23:17:41.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:41.404+0000 7f17d1766640 1 -- 172.21.15.26:0/1216163118 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17cc1a3f00 con 0x7f17cc106f10 2024-09-11T23:17:41.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:41.404+0000 7f17b3fff640 1 -- 172.21.15.26:0/1216163118 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17c4014440 con 0x7f17cc106f10 2024-09-11T23:17:41.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:41.404+0000 7f17b3fff640 1 -- 172.21.15.26:0/1216163118 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17c40075d0 con 0x7f17cc106f10 2024-09-11T23:17:41.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:41.405+0000 7f17d1766640 1 -- 172.21.15.26:0/1216163118 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17cc1ab790 con 0x7f17cc106f10 2024-09-11T23:17:41.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:41.405+0000 7f17d1766640 1 -- 172.21.15.26:0/1216163118 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17cc102240 con 0x7f17cc106f10 2024-09-11T23:17:41.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:41.407+0000 7f17b3fff640 1 -- 172.21.15.26:0/1216163118 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f17c401c070 con 0x7f17cc106f10 2024-09-11T23:17:41.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:41.407+0000 7f17b3fff640 1 --2- 172.21.15.26:0/1216163118 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f17b40a2c10 0x7f17b40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:41.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:41.408+0000 7f17b3fff640 1 -- 172.21.15.26:0/1216163118 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f17c40ceb70 con 0x7f17cc106f10 2024-09-11T23:17:41.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:41.408+0000 7f17caffd640 1 --2- 172.21.15.26:0/1216163118 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f17b40a2c10 0x7f17b40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:41.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:41.408+0000 7f17caffd640 1 --2- 172.21.15.26:0/1216163118 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f17b40a2c10 0x7f17b40a50d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f17bc000c00 tx=0x7f17bc032d50 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:17:41.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:41.411+0000 7f17b3fff640 1 -- 172.21.15.26:0/1216163118 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f17c409ba80 con 0x7f17cc106f10 2024-09-11T23:17:41.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:41.620+0000 7f17d1766640 1 -- 172.21.15.26:0/1216163118 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f17cc02d7c0 con 0x7f17cc106f10 2024-09-11T23:17:41.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:41.621+0000 7f17b3fff640 1 -- 172.21.15.26:0/1216163118 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11971 (secure 0 0 0) 0x7f17c409b380 con 0x7f17cc106f10 2024-09-11T23:17:41.622 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-11T23:17:41.622 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":28,"btime":"2024-09-11T23:16:38:976987+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14499,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/367659515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":367659515},{"type":"v1","addr":"172.21.15.26:6835","nonce":367659515}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14511,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/2041065299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":2041065299},{"type":"v1","addr":"172.21.15.26:6837","nonce":2041065299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:16:26.544203+0000","modified":"2024-09-11T23:16:38.976970+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24485,"mds_1":24463,"mds_2":24458,"mds_3":24448,"mds_4":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14550":{"gid":14550,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6839/2233068305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":2233068305},{"type":"v1","addr":"172.21.15.26:6839","nonce":2233068305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14562":{"gid":14562,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6841/2984982336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":2984982336},{"type":"v1","addr":"172.21.15.26:6841","nonce":2984982336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6833/2019841455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":2019841455},{"type":"v1","addr":"172.21.15.136:6833","nonce":2019841455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.76:6833/3438147050","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3438147050},{"type":"v1","addr":"172.21.15.76:6833","nonce":3438147050}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6835/3617665975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3617665975},{"type":"v1","addr":"172.21.15.136:6835","nonce":3617665975}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":12,"addr":"172.21.15.76:6835/393308499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":393308499},{"type":"v1","addr":"172.21.15.76:6835","nonce":393308499}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":10,"addr":"172.21.15.136:6837/1633082387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":1633082387},{"type":"v1","addr":"172.21.15.136:6837","nonce":1633082387}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.76:6837/2078978748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":2078978748},{"type":"v1","addr":"172.21.15.76:6837","nonce":2078978748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.136:6839/3211398630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":3211398630},{"type":"v1","addr":"172.21.15.136:6839","nonce":3211398630}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.76:6839/2077637090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6838","nonce":2077637090},{"type":"v1","addr":"172.21.15.76:6839","nonce":2077637090}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24485,"qdb_cluster":[24485,24463,24458,24448,24443]},"id":1}]} 2024-09-11T23:17:41.622 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 28 2024-09-11T23:17:41.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:41.624+0000 7f17d1766640 1 -- 172.21.15.26:0/1216163118 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f17b40a2c10 msgr2=0x7f17b40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:41.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:41.624+0000 7f17d1766640 1 --2- 172.21.15.26:0/1216163118 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f17b40a2c10 0x7f17b40a50d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f17bc000c00 tx=0x7f17bc032d50 comp rx=0 tx=0).stop 2024-09-11T23:17:41.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:41.624+0000 7f17d1766640 1 -- 172.21.15.26:0/1216163118 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f17cc106f10 msgr2=0x7f17cc1a34e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:41.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:41.624+0000 7f17d1766640 1 --2- 172.21.15.26:0/1216163118 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f17cc106f10 0x7f17cc1a34e0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f17c400b970 tx=0x7f17c400bd70 comp rx=0 tx=0).stop 2024-09-11T23:17:41.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:41.624+0000 7f17d1766640 1 -- 172.21.15.26:0/1216163118 shutdown_connections 2024-09-11T23:17:41.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:41.624+0000 7f17d1766640 1 --2- 172.21.15.26:0/1216163118 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f17b40a2c10 0x7f17b40a50d0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:41.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:41.624+0000 7f17d1766640 1 --2- 172.21.15.26:0/1216163118 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f17cc106f10 0x7f17cc1a34e0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:41.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:41.624+0000 7f17d1766640 1 --2- 172.21.15.26:0/1216163118 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f17cc102790 0x7f17cc10e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:41.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:41.624+0000 7f17d1766640 1 --2- 172.21.15.26:0/1216163118 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f17cc101de0 0x7f17cc10deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:41.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:41.624+0000 7f17d1766640 1 -- 172.21.15.26:0/1216163118 >> 172.21.15.26:0/1216163118 conn(0x7f17cc0fd8c0 msgr2=0x7f17cc0fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:17:41.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:41.624+0000 7f17d1766640 1 -- 172.21.15.26:0/1216163118 shutdown_connections 2024-09-11T23:17:41.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:41.624+0000 7f17d1766640 1 -- 172.21.15.26:0/1216163118 wait complete. 2024-09-11T23:17:42.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:17:41 smithi076 ceph-mon[35296]: pgmap v318: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-11T23:17:42.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:17:41 smithi076 ceph-mon[35296]: from='client.? 172.21.15.26:0/1216163118' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:17:42.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:17:41 smithi136 ceph-mon[35121]: pgmap v318: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-11T23:17:42.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:17:41 smithi136 ceph-mon[35121]: from='client.? 172.21.15.26:0/1216163118' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:17:42.249 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:17:41 smithi026 ceph-mon[31172]: pgmap v318: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-11T23:17:42.249 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:17:41 smithi026 ceph-mon[31172]: from='client.? 172.21.15.26:0/1216163118' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:17:42.283 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 28, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-11T23:16:26.544203+0000', 'modified': '2024-09-11T23:16:38.976970+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24485, 'mds_1': 24463, 'mds_2': 24458, 'mds_3': 24448, 'mds_4': 24443}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14550': {'gid': 14550, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.26:6839/2233068305', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6838', 'nonce': 2233068305}, {'type': 'v1', 'addr': '172.21.15.26:6839', 'nonce': 2233068305}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14562': {'gid': 14562, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.26:6841/2984982336', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6840', 'nonce': 2984982336}, {'type': 'v1', 'addr': '172.21.15.26:6841', 'nonce': 2984982336}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24403': {'gid': 24403, 'name': 'l', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.136:6833/2019841455', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6832', 'nonce': 2019841455}, {'type': 'v1', 'addr': '172.21.15.136:6833', 'nonce': 2019841455}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24407': {'gid': 24407, 'name': 'k', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.76:6833/3438147050', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6832', 'nonce': 3438147050}, {'type': 'v1', 'addr': '172.21.15.76:6833', 'nonce': 3438147050}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24439': {'gid': 24439, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.136:6835/3617665975', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6834', 'nonce': 3617665975}, {'type': 'v1', 'addr': '172.21.15.136:6835', 'nonce': 3617665975}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24443': {'gid': 24443, 'name': 'e', 'rank': 4, 'incarnation': 26, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.76:6835/393308499', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6834', 'nonce': 393308499}, {'type': 'v1', 'addr': '172.21.15.76:6835', 'nonce': 393308499}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24448': {'gid': 24448, 'name': 'c', 'rank': 3, 'incarnation': 24, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.136:6837/1633082387', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6836', 'nonce': 1633082387}, {'type': 'v1', 'addr': '172.21.15.136:6837', 'nonce': 1633082387}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24458': {'gid': 24458, 'name': 'h', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.76:6837/2078978748', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6836', 'nonce': 2078978748}, {'type': 'v1', 'addr': '172.21.15.76:6837', 'nonce': 2078978748}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24463': {'gid': 24463, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.136:6839/3211398630', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6838', 'nonce': 3211398630}, {'type': 'v1', 'addr': '172.21.15.136:6839', 'nonce': 3211398630}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24485': {'gid': 24485, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.76:6839/2077637090', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6838', 'nonce': 2077637090}, {'type': 'v1', 'addr': '172.21.15.76:6839', 'nonce': 2077637090}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24485, 'qdb_cluster': [24485, 24463, 24458, 24448, 24443]} 2024-09-11T23:17:42.283 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-09-11T23:17:42.284 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph tell mds.e status 2024-09-11T23:17:42.545 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:17:43.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:42.994+0000 7f48cd765640 1 -- 172.21.15.26:0/760320375 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f48c8100e20 msgr2=0x7f48c8101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:43.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:42.994+0000 7f48cd765640 1 --2- 172.21.15.26:0/760320375 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f48c8100e20 0x7f48c8101200 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f48b8009ca0 tx=0x7f48b802a950 comp rx=0 tx=0).stop 2024-09-11T23:17:43.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:42.994+0000 7f48cd765640 1 -- 172.21.15.26:0/760320375 shutdown_connections 2024-09-11T23:17:43.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:42.994+0000 7f48cd765640 1 --2- 172.21.15.26:0/760320375 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f48c8102570 0x7f48c81029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:43.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:42.994+0000 7f48cd765640 1 --2- 172.21.15.26:0/760320375 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f48c8102ef0 0x7f48c810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:43.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:42.994+0000 7f48cd765640 1 --2- 172.21.15.26:0/760320375 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f48c8100e20 0x7f48c8101200 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:43.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:42.994+0000 7f48cd765640 1 -- 172.21.15.26:0/760320375 >> 172.21.15.26:0/760320375 conn(0x7f48c80fc7d0 msgr2=0x7f48c80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:17:43.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:42.994+0000 7f48cd765640 1 -- 172.21.15.26:0/760320375 shutdown_connections 2024-09-11T23:17:43.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:42.994+0000 7f48cd765640 1 -- 172.21.15.26:0/760320375 wait complete. 2024-09-11T23:17:43.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:42.995+0000 7f48cd765640 1 Processor -- start 2024-09-11T23:17:43.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:42.995+0000 7f48cd765640 1 -- start start 2024-09-11T23:17:43.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:42.995+0000 7f48cd765640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f48c8100e20 0x7f48c8197100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:43.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:42.995+0000 7f48cd765640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f48c8102570 0x7f48c8197640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:43.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:42.995+0000 7f48cd765640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f48c8102ef0 0x7f48c8197b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:43.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:42.995+0000 7f48cd765640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f48c8110f80 con 0x7f48c8102570 2024-09-11T23:17:43.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:42.995+0000 7f48cd765640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f48c8110e00 con 0x7f48c8102ef0 2024-09-11T23:17:43.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:42.995+0000 7f48cd765640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f48c8111100 con 0x7f48c8100e20 2024-09-11T23:17:43.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:42.995+0000 7f48c67fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f48c8102570 0x7f48c8197640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:43.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:42.995+0000 7f48c67fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f48c8102570 0x7f48c8197640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59662/0 (socket says 172.21.15.26:59662) 2024-09-11T23:17:43.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:42.995+0000 7f48c67fc640 1 -- 172.21.15.26:0/788706896 learned_addr learned my addr 172.21.15.26:0/788706896 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:17:43.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:42.995+0000 7f48c77fe640 1 --2- 172.21.15.26:0/788706896 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f48c8102ef0 0x7f48c8197b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:43.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:42.995+0000 7f48c67fc640 1 -- 172.21.15.26:0/788706896 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f48c8100e20 msgr2=0x7f48c8197100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:43.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:42.995+0000 7f48c67fc640 1 --2- 172.21.15.26:0/788706896 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f48c8100e20 0x7f48c8197100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:43.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:42.995+0000 7f48c67fc640 1 -- 172.21.15.26:0/788706896 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f48c8102ef0 msgr2=0x7f48c8197b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:43.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:42.995+0000 7f48c67fc640 1 --2- 172.21.15.26:0/788706896 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f48c8102ef0 0x7f48c8197b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:43.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:42.995+0000 7f48c67fc640 1 -- 172.21.15.26:0/788706896 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48c81a6970 con 0x7f48c8102570 2024-09-11T23:17:43.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:42.995+0000 7f48c77fe640 1 --2- 172.21.15.26:0/788706896 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f48c8102ef0 0x7f48c8197b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:17:43.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:42.996+0000 7f48c67fc640 1 --2- 172.21.15.26:0/788706896 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f48c8102570 0x7f48c8197640 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f48bc00d950 tx=0x7f48bc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:17:43.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:42.996+0000 7f48affff640 1 -- 172.21.15.26:0/788706896 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48bc017030 con 0x7f48c8102570 2024-09-11T23:17:43.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:42.996+0000 7f48cd765640 1 -- 172.21.15.26:0/788706896 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48c81a6c60 con 0x7f48c8102570 2024-09-11T23:17:43.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:42.996+0000 7f48affff640 1 -- 172.21.15.26:0/788706896 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f48bc004020 con 0x7f48c8102570 2024-09-11T23:17:43.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:42.996+0000 7f48affff640 1 -- 172.21.15.26:0/788706896 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48bc0047a0 con 0x7f48c8102570 2024-09-11T23:17:43.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:42.996+0000 7f48cd765640 1 -- 172.21.15.26:0/788706896 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48c81a71a0 con 0x7f48c8102570 2024-09-11T23:17:43.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:42.999+0000 7f48affff640 1 -- 172.21.15.26:0/788706896 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f48bc0041d0 con 0x7f48c8102570 2024-09-11T23:17:43.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:42.999+0000 7f48cfb3e740 1 Processor -- start 2024-09-11T23:17:43.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:42.999+0000 7f48cfb3e740 1 -- start start 2024-09-11T23:17:43.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:42.999+0000 7f48cfb3e740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cc601ea9d0 0x55cc601eadb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:43.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:42.999+0000 7f48cfb3e740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cc601eb2f0 0x55cc601f5670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:43.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:42.999+0000 7f48cfb3e740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cc601f5bb0 0x55cc601f7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:43.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:42.999+0000 7f48cfb3e740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55cc60151090 con 0x55cc601eb2f0 2024-09-11T23:17:43.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:42.999+0000 7f48cfb3e740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55cc600f8a70 con 0x55cc601ea9d0 2024-09-11T23:17:43.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:42.999+0000 7f48cfb3e740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55cc6014d050 con 0x55cc601f5bb0 2024-09-11T23:17:43.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:42.999+0000 7f48c6ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cc601eb2f0 0x55cc601f5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:43.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:42.999+0000 7f48c6ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cc601eb2f0 0x55cc601f5670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59672/0 (socket says 172.21.15.26:59672) 2024-09-11T23:17:43.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:42.999+0000 7f48c6ffd640 1 -- 172.21.15.26:0/1077521001 learned_addr learned my addr 172.21.15.26:0/1077521001 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:17:43.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.000+0000 7f48c67fc640 1 --2- 172.21.15.26:0/1077521001 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cc601f5bb0 0x55cc601f7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:43.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.000+0000 7f48c77fe640 1 --2- 172.21.15.26:0/1077521001 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cc601ea9d0 0x55cc601eadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:43.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.000+0000 7f48c6ffd640 1 -- 172.21.15.26:0/1077521001 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cc601f5bb0 msgr2=0x55cc601f7fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:17:43.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.000+0000 7f48c6ffd640 1 --2- 172.21.15.26:0/1077521001 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cc601f5bb0 0x55cc601f7fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:43.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.000+0000 7f48affff640 1 --2- 172.21.15.26:0/788706896 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f48b00a2c10 0x7f48b00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:43.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.000+0000 7f48c6ffd640 1 -- 172.21.15.26:0/1077521001 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cc601ea9d0 msgr2=0x55cc601eadb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:17:43.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.000+0000 7f48c6ffd640 1 --2- 172.21.15.26:0/1077521001 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cc601ea9d0 0x55cc601eadb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:43.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.000+0000 7f48c6ffd640 1 -- 172.21.15.26:0/1077521001 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cc60105130 con 0x55cc601eb2f0 2024-09-11T23:17:43.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.000+0000 7f48c77fe640 1 --2- 172.21.15.26:0/1077521001 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cc601ea9d0 0x55cc601eadb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:17:43.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.000+0000 7f48c67fc640 1 --2- 172.21.15.26:0/1077521001 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cc601f5bb0 0x55cc601f7fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:17:43.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.000+0000 7f48c77fe640 1 --2- 172.21.15.26:0/788706896 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f48b00a2c10 0x7f48b00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:43.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.000+0000 7f48affff640 1 -- 172.21.15.26:0/788706896 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f48bc0cecf0 con 0x7f48c8102570 2024-09-11T23:17:43.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.000+0000 7f48c6ffd640 1 --2- 172.21.15.26:0/1077521001 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cc601eb2f0 0x55cc601f5670 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f48b8005e30 tx=0x7f48b8009800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c13d8e10db80d8f9 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:17:43.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.000+0000 7f48ad7fa640 1 -- 172.21.15.26:0/1077521001 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48b8034040 con 0x55cc601eb2f0 2024-09-11T23:17:43.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.000+0000 7f48ad7fa640 1 -- 172.21.15.26:0/1077521001 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f48b802fb30 con 0x55cc601eb2f0 2024-09-11T23:17:43.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.000+0000 7f48ad7fa640 1 -- 172.21.15.26:0/1077521001 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48b8004440 con 0x55cc601eb2f0 2024-09-11T23:17:43.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.001+0000 7f48cfb3e740 1 -- 172.21.15.26:0/1077521001 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cc601eb2f0 msgr2=0x55cc601f5670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:43.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.001+0000 7f48cfb3e740 1 --2- 172.21.15.26:0/1077521001 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cc601eb2f0 0x55cc601f5670 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f48b8005e30 tx=0x7f48b8009800 comp rx=0 tx=0).stop 2024-09-11T23:17:43.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.001+0000 7f48c77fe640 1 --2- 172.21.15.26:0/788706896 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f48b00a2c10 0x7f48b00a50d0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f48c0005fd0 tx=0x7f48c00046c0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:17:43.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.001+0000 7f48cfb3e740 1 -- 172.21.15.26:0/1077521001 shutdown_connections 2024-09-11T23:17:43.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.001+0000 7f48cfb3e740 1 --2- 172.21.15.26:0/1077521001 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cc601f5bb0 0x55cc601f7fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:43.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.001+0000 7f48cfb3e740 1 --2- 172.21.15.26:0/1077521001 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cc601eb2f0 0x55cc601f5670 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:43.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.001+0000 7f48cfb3e740 1 --2- 172.21.15.26:0/1077521001 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cc601ea9d0 0x55cc601eadb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:43.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.001+0000 7f48cfb3e740 1 -- 172.21.15.26:0/1077521001 >> 172.21.15.26:0/1077521001 conn(0x55cc601e6f60 msgr2=0x55cc601e7340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:17:43.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.001+0000 7f48cfb3e740 1 -- 172.21.15.26:0/1077521001 shutdown_connections 2024-09-11T23:17:43.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.001+0000 7f48cfb3e740 1 -- 172.21.15.26:0/1077521001 wait complete. 2024-09-11T23:17:43.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.001+0000 7f48cfb3e740 1 -- start start 2024-09-11T23:17:43.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.001+0000 7f48cfb3e740 1 Processor -- start 2024-09-11T23:17:43.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.002+0000 7f48cfb3e740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cc601ea9d0 0x55cc601f53a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:43.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.002+0000 7f48cfb3e740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cc601eb2f0 0x55cc601f26f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:43.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.002+0000 7f48cfb3e740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cc601f5bb0 0x55cc601f2c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:43.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.002+0000 7f48cfb3e740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55cc601f8780 con 0x55cc601ea9d0 2024-09-11T23:17:43.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.002+0000 7f48cfb3e740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55cc6010bf20 con 0x55cc601f5bb0 2024-09-11T23:17:43.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.002+0000 7f48cfb3e740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55cc601f8900 con 0x55cc601eb2f0 2024-09-11T23:17:43.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.002+0000 7f48cfb3e740 10 client.0 _ll_register_callbacks cb 0x55cc600d5030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:17:43.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.002+0000 7f48c6ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cc601ea9d0 0x55cc601f53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:43.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.002+0000 7f48c6ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cc601ea9d0 0x55cc601f53a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59684/0 (socket says 172.21.15.26:59684) 2024-09-11T23:17:43.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.002+0000 7f48c6ffd640 1 -- 172.21.15.26:0/3481734742 learned_addr learned my addr 172.21.15.26:0/3481734742 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:17:43.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.002+0000 7f48c77fe640 1 --2- 172.21.15.26:0/3481734742 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cc601f5bb0 0x55cc601f2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:43.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.002+0000 7f48c6ffd640 1 -- 172.21.15.26:0/3481734742 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cc601eb2f0 msgr2=0x55cc601f26f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:17:43.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.002+0000 7f48c6ffd640 1 --2- 172.21.15.26:0/3481734742 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cc601eb2f0 0x55cc601f26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:43.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.002+0000 7f48c6ffd640 1 -- 172.21.15.26:0/3481734742 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cc601f5bb0 msgr2=0x55cc601f2c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:17:43.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.002+0000 7f48c6ffd640 1 --2- 172.21.15.26:0/3481734742 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cc601f5bb0 0x55cc601f2c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:43.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.002+0000 7f48c6ffd640 1 -- 172.21.15.26:0/3481734742 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cc5fec22b0 con 0x55cc601ea9d0 2024-09-11T23:17:43.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.002+0000 7f48c6ffd640 1 --2- 172.21.15.26:0/3481734742 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cc601ea9d0 0x55cc601f53a0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f48b8009a30 tx=0x7f48b802b0d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbe6200692ecab8c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:17:43.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.002+0000 7f48aaff5640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:17:43.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.003+0000 7f48aaff5640 1 -- 172.21.15.26:0/3481734742 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48b8050020 con 0x55cc601ea9d0 2024-09-11T23:17:43.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.003+0000 7f48aaff5640 1 -- 172.21.15.26:0/3481734742 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f48b802f5a0 con 0x55cc601ea9d0 2024-09-11T23:17:43.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.003+0000 7f48cfb3e740 1 -- 172.21.15.26:0/3481734742 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f48b802fb30 con 0x55cc601ea9d0 2024-09-11T23:17:43.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.003+0000 7f48aaff5640 1 -- 172.21.15.26:0/3481734742 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f48b8048040 con 0x55cc601ea9d0 2024-09-11T23:17:43.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.003+0000 7f48cfb3e740 10 client.14922 fetch_fsmap learned FSMap version 28 2024-09-11T23:17:43.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.003+0000 7f48cfb3e740 1 -- 172.21.15.26:0/3481734742 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55cc60204020 con 0x55cc601ea9d0 2024-09-11T23:17:43.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.003+0000 7f48aaff5640 1 -- 172.21.15.26:0/3481734742 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f48b802fb30 con 0x55cc601ea9d0 2024-09-11T23:17:43.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.003+0000 7f48aaff5640 1 -- 172.21.15.26:0/3481734742 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f48b804b030 con 0x55cc601ea9d0 2024-09-11T23:17:43.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.003+0000 7f48cfb3e740 10 client.14922 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:17:43.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.003+0000 7f48cfb3e740 10 client.14922 resolve_mds: resolved name 'e' to daemon mds.e 2024-09-11T23:17:43.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.003+0000 7f48cfb3e740 20 client.14922 populate_metadata read hostname 'smithi026' 2024-09-11T23:17:43.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.003+0000 7f48cfb3e740 1 --2- 172.21.15.26:0/3481734742 >> [v2:172.21.15.76:6834/393308499,v1:172.21.15.76:6835/393308499] conn(0x55cc60206160 0x55cc602075d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:43.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.003+0000 7f48cfb3e740 4 client.14922 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:17:43.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.003+0000 7f48cfb3e740 1 -- 172.21.15.26:0/3481734742 --> [v2:172.21.15.76:6834/393308499,v1:172.21.15.76:6835/393308499] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cc6015e600 con 0x55cc60206160 2024-09-11T23:17:43.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.004+0000 7f48c67fc640 1 --2- 172.21.15.26:0/3481734742 >> [v2:172.21.15.76:6834/393308499,v1:172.21.15.76:6835/393308499] conn(0x55cc60206160 0x55cc602075d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:43.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.004+0000 7f48c67fc640 1 --2- 172.21.15.26:0/3481734742 >> [v2:172.21.15.76:6834/393308499,v1:172.21.15.76:6835/393308499] conn(0x55cc60206160 0x55cc602075d0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=6541481eacc8246c server_cookie=f363fd9b85b1ad3e in_seq=0 out_seq=0 2024-09-11T23:17:43.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.004+0000 7f48aaff5640 10 client.14922 ms_handle_connect on v2:172.21.15.76:6834/393308499 2024-09-11T23:17:43.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.005+0000 7f48aaff5640 1 -- 172.21.15.26:0/3481734742 <== mds.4 v2:172.21.15.76:6834/393308499 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55cc6015e600 con 0x55cc60206160 2024-09-11T23:17:43.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.005+0000 7f48aaff5640 10 client.14922 handle_command_reply: tid=0 2024-09-11T23:17:43.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.005+0000 7f48cfb3e740 1 client.14922 shutdown 2024-09-11T23:17:43.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.005+0000 7f48cfb3e740 1 -- 172.21.15.26:0/3481734742 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cc601ea9d0 msgr2=0x55cc601f53a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:43.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.005+0000 7f48cfb3e740 1 --2- 172.21.15.26:0/3481734742 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cc601ea9d0 0x55cc601f53a0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f48b8009a30 tx=0x7f48b802b0d0 comp rx=0 tx=0).stop 2024-09-11T23:17:43.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.005+0000 7f48cfb3e740 1 -- 172.21.15.26:0/3481734742 shutdown_connections 2024-09-11T23:17:43.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.005+0000 7f48cfb3e740 1 --2- 172.21.15.26:0/3481734742 >> [v2:172.21.15.76:6834/393308499,v1:172.21.15.76:6835/393308499] conn(0x55cc60206160 0x55cc602075d0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:43.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.005+0000 7f48cfb3e740 1 --2- 172.21.15.26:0/3481734742 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cc601f5bb0 0x55cc601f2c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:43.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.005+0000 7f48aaff5640 0 client.14922 ms_handle_reset on v2:172.21.15.76:6834/393308499 2024-09-11T23:17:43.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.005+0000 7f48cfb3e740 1 --2- 172.21.15.26:0/3481734742 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cc601eb2f0 0x55cc601f26f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:43.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.005+0000 7f48cfb3e740 1 --2- 172.21.15.26:0/3481734742 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cc601ea9d0 0x55cc601f53a0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:43.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.005+0000 7f48cfb3e740 1 -- 172.21.15.26:0/3481734742 >> 172.21.15.26:0/3481734742 conn(0x55cc601e6f60 msgr2=0x55cc601ecc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:17:43.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.005+0000 7f48cfb3e740 1 -- 172.21.15.26:0/3481734742 shutdown_connections 2024-09-11T23:17:43.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.005+0000 7f48cfb3e740 1 -- 172.21.15.26:0/3481734742 wait complete. 2024-09-11T23:17:43.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.005+0000 7f48cfb3e740 20 client.14922 trim_cache size 0 max 16384 2024-09-11T23:17:43.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.025+0000 7f48cfb3e740 1 Processor -- start 2024-09-11T23:17:43.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.025+0000 7f48cfb3e740 1 -- start start 2024-09-11T23:17:43.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.025+0000 7f48cfb3e740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cc601ea9d0 0x55cc602075d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:43.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.026+0000 7f48cfb3e740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cc601eb2f0 0x55cc601f0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:43.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.026+0000 7f48cfb3e740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cc601f5bb0 0x55cc601f1500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:43.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.026+0000 7f48cfb3e740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55cc60205910 con 0x55cc601eb2f0 2024-09-11T23:17:43.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.026+0000 7f48cfb3e740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55cc60205fe0 con 0x55cc601f5bb0 2024-09-11T23:17:43.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.026+0000 7f48cfb3e740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55cc602066b0 con 0x55cc601ea9d0 2024-09-11T23:17:43.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.026+0000 7f48c67fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cc601eb2f0 0x55cc601f0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:43.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.026+0000 7f48c67fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cc601eb2f0 0x55cc601f0fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59686/0 (socket says 172.21.15.26:59686) 2024-09-11T23:17:43.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.026+0000 7f48c67fc640 1 -- 172.21.15.26:0/413328450 learned_addr learned my addr 172.21.15.26:0/413328450 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:17:43.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.026+0000 7f48c6ffd640 1 --2- 172.21.15.26:0/413328450 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cc601ea9d0 0x55cc602075d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:43.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.026+0000 7f48c67fc640 1 -- 172.21.15.26:0/413328450 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cc601ea9d0 msgr2=0x55cc602075d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:17:43.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.026+0000 7f48c77fe640 1 --2- 172.21.15.26:0/413328450 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cc601f5bb0 0x55cc601f1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:43.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.026+0000 7f48c67fc640 1 --2- 172.21.15.26:0/413328450 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cc601ea9d0 0x55cc602075d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:43.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.026+0000 7f48c67fc640 1 -- 172.21.15.26:0/413328450 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cc601f5bb0 msgr2=0x55cc601f1500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:17:43.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.026+0000 7f48c67fc640 1 --2- 172.21.15.26:0/413328450 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cc601f5bb0 0x55cc601f1500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:43.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.026+0000 7f48c67fc640 1 -- 172.21.15.26:0/413328450 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cc601eb120 con 0x55cc601eb2f0 2024-09-11T23:17:43.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.026+0000 7f48c67fc640 1 --2- 172.21.15.26:0/413328450 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cc601eb2f0 0x55cc601f0fc0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f48bc0970d0 tx=0x7f48bc0cf020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=955e27ab5b869ed5 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:17:43.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.026+0000 7f48acff9640 1 -- 172.21.15.26:0/413328450 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48bc09baf0 con 0x55cc601eb2f0 2024-09-11T23:17:43.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.026+0000 7f48acff9640 1 -- 172.21.15.26:0/413328450 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f48bc09b3f0 con 0x55cc601eb2f0 2024-09-11T23:17:43.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.026+0000 7f48acff9640 1 -- 172.21.15.26:0/413328450 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48bc0a3bd0 con 0x55cc601eb2f0 2024-09-11T23:17:43.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.027+0000 7f48cfb3e740 1 -- 172.21.15.26:0/413328450 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cc601eb2f0 msgr2=0x55cc601f0fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:43.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.027+0000 7f48cfb3e740 1 --2- 172.21.15.26:0/413328450 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cc601eb2f0 0x55cc601f0fc0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f48bc0970d0 tx=0x7f48bc0cf020 comp rx=0 tx=0).stop 2024-09-11T23:17:43.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.027+0000 7f48cfb3e740 1 -- 172.21.15.26:0/413328450 shutdown_connections 2024-09-11T23:17:43.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.027+0000 7f48cfb3e740 1 --2- 172.21.15.26:0/413328450 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cc601f5bb0 0x55cc601f1500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:43.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.027+0000 7f48cfb3e740 1 --2- 172.21.15.26:0/413328450 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cc601eb2f0 0x55cc601f0fc0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:43.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.027+0000 7f48cfb3e740 1 --2- 172.21.15.26:0/413328450 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cc601ea9d0 0x55cc602075d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:43.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.027+0000 7f48cfb3e740 1 -- 172.21.15.26:0/413328450 >> 172.21.15.26:0/413328450 conn(0x55cc601e6f60 msgr2=0x55cc601e7340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:17:43.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.027+0000 7f48cfb3e740 1 -- 172.21.15.26:0/413328450 shutdown_connections 2024-09-11T23:17:43.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.027+0000 7f48cfb3e740 1 -- 172.21.15.26:0/413328450 wait complete. 2024-09-11T23:17:43.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.028+0000 7f48cfb3e740 1 -- start start 2024-09-11T23:17:43.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.028+0000 7f48cfb3e740 1 Processor -- start 2024-09-11T23:17:43.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.028+0000 7f48cfb3e740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cc601ea9d0 0x55cc6020c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:43.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.028+0000 7f48cfb3e740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cc601eb2f0 0x55cc6020cae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:43.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.028+0000 7f48cfb3e740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cc601f5bb0 0x55cc60208be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:43.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.028+0000 7f48cfb3e740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55cc60207450 con 0x55cc601eb2f0 2024-09-11T23:17:43.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.028+0000 7f48cfb3e740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55cc60206d80 con 0x55cc601f5bb0 2024-09-11T23:17:43.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.028+0000 7f48cfb3e740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55cc60207b20 con 0x55cc601ea9d0 2024-09-11T23:17:43.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.028+0000 7f48cfb3e740 10 client.0 _ll_register_callbacks cb 0x7f48b8005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:17:43.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.028+0000 7f48c67fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cc601eb2f0 0x55cc6020cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:43.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.028+0000 7f48c67fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cc601eb2f0 0x55cc6020cae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59696/0 (socket says 172.21.15.26:59696) 2024-09-11T23:17:43.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.028+0000 7f48c67fc640 1 -- 172.21.15.26:0/1791378859 learned_addr learned my addr 172.21.15.26:0/1791378859 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:17:43.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.028+0000 7f48c6ffd640 1 --2- 172.21.15.26:0/1791378859 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cc601ea9d0 0x55cc6020c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:43.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.028+0000 7f48c77fe640 1 --2- 172.21.15.26:0/1791378859 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cc601f5bb0 0x55cc60208be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:43.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.028+0000 7f48c67fc640 1 -- 172.21.15.26:0/1791378859 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cc601ea9d0 msgr2=0x55cc6020c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:17:43.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.028+0000 7f48c67fc640 1 --2- 172.21.15.26:0/1791378859 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cc601ea9d0 0x55cc6020c5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:43.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.028+0000 7f48c67fc640 1 -- 172.21.15.26:0/1791378859 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cc601f5bb0 msgr2=0x55cc60208be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:17:43.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.028+0000 7f48c67fc640 1 --2- 172.21.15.26:0/1791378859 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cc601f5bb0 0x55cc60208be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:43.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.028+0000 7f48c67fc640 1 -- 172.21.15.26:0/1791378859 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cc60209220 con 0x55cc601eb2f0 2024-09-11T23:17:43.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.028+0000 7f48c77fe640 1 --2- 172.21.15.26:0/1791378859 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cc601f5bb0 0x55cc60208be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:17:43.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.028+0000 7f48c6ffd640 1 --2- 172.21.15.26:0/1791378859 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cc601ea9d0 0x55cc6020c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:17:43.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.029+0000 7f48c67fc640 1 --2- 172.21.15.26:0/1791378859 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cc601eb2f0 0x55cc6020cae0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f48bc0cc720 tx=0x7f48bc09f7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12289f1d892a6b4e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:17:43.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.029+0000 7f48aaff5640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:17:43.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.029+0000 7f48aaff5640 1 -- 172.21.15.26:0/1791378859 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48bc0a4060 con 0x55cc601eb2f0 2024-09-11T23:17:43.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.029+0000 7f48cfb3e740 1 -- 172.21.15.26:0/1791378859 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f48bc09b3f0 con 0x55cc601eb2f0 2024-09-11T23:17:43.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.029+0000 7f48aaff5640 1 -- 172.21.15.26:0/1791378859 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f48bc0cc8f0 con 0x55cc601eb2f0 2024-09-11T23:17:43.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.029+0000 7f48aaff5640 1 -- 172.21.15.26:0/1791378859 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f48bc0b4180 con 0x55cc601eb2f0 2024-09-11T23:17:43.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.029+0000 7f48cfb3e740 10 client.14928 fetch_fsmap learned FSMap version 28 2024-09-11T23:17:43.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.029+0000 7f48cfb3e740 1 -- 172.21.15.26:0/1791378859 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55cc6020dd60 con 0x55cc601eb2f0 2024-09-11T23:17:43.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.029+0000 7f48aaff5640 1 -- 172.21.15.26:0/1791378859 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f48bc09b3f0 con 0x55cc601eb2f0 2024-09-11T23:17:43.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.029+0000 7f48aaff5640 1 -- 172.21.15.26:0/1791378859 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f48bc066260 con 0x55cc601eb2f0 2024-09-11T23:17:43.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.029+0000 7f48cfb3e740 10 client.14928 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:17:43.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.029+0000 7f48cfb3e740 10 client.14928 resolve_mds: resolved name 'e' to daemon mds.e 2024-09-11T23:17:43.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.029+0000 7f48cfb3e740 20 client.14928 populate_metadata read hostname 'smithi026' 2024-09-11T23:17:43.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.029+0000 7f48cfb3e740 1 --2- 172.21.15.26:0/1791378859 >> [v2:172.21.15.76:6834/393308499,v1:172.21.15.76:6835/393308499] conn(0x55cc60274780 0x55cc60273490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:43.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.030+0000 7f48cfb3e740 4 client.14928 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-11T23:17:43.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.030+0000 7f48cfb3e740 1 -- 172.21.15.26:0/1791378859 --> [v2:172.21.15.76:6834/393308499,v1:172.21.15.76:6835/393308499] -- command(tid 0: {"prefix": "status"}) -- 0x55cc60158b10 con 0x55cc60274780 2024-09-11T23:17:43.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.030+0000 7f48c6ffd640 1 --2- 172.21.15.26:0/1791378859 >> [v2:172.21.15.76:6834/393308499,v1:172.21.15.76:6835/393308499] conn(0x55cc60274780 0x55cc60273490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:43.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.030+0000 7f48c6ffd640 1 --2- 172.21.15.26:0/1791378859 >> [v2:172.21.15.76:6834/393308499,v1:172.21.15.76:6835/393308499] conn(0x55cc60274780 0x55cc60273490 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=665dce189795cdd4 server_cookie=a4918ab28c0e2518 in_seq=0 out_seq=0 2024-09-11T23:17:43.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.030+0000 7f48aaff5640 10 client.14928 ms_handle_connect on v2:172.21.15.76:6834/393308499 2024-09-11T23:17:43.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.031+0000 7f48aaff5640 1 -- 172.21.15.26:0/1791378859 <== mds.4 v2:172.21.15.76:6834/393308499 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55cc60158b10 con 0x55cc60274780 2024-09-11T23:17:43.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.031+0000 7f48aaff5640 10 client.14928 handle_command_reply: tid=0 2024-09-11T23:17:43.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.031+0000 7f48cfb3e740 1 client.14928 shutdown 2024-09-11T23:17:43.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.031+0000 7f48cfb3e740 1 -- 172.21.15.26:0/1791378859 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cc601eb2f0 msgr2=0x55cc6020cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:43.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.031+0000 7f48cfb3e740 1 --2- 172.21.15.26:0/1791378859 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cc601eb2f0 0x55cc6020cae0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f48bc0cc720 tx=0x7f48bc09f7a0 comp rx=0 tx=0).stop 2024-09-11T23:17:43.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.031+0000 7f48cfb3e740 1 -- 172.21.15.26:0/1791378859 shutdown_connections 2024-09-11T23:17:43.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.031+0000 7f48cfb3e740 1 --2- 172.21.15.26:0/1791378859 >> [v2:172.21.15.76:6834/393308499,v1:172.21.15.76:6835/393308499] conn(0x55cc60274780 0x55cc60273490 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:43.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.031+0000 7f48cfb3e740 1 --2- 172.21.15.26:0/1791378859 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cc601f5bb0 0x55cc60208be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:43.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.031+0000 7f48cfb3e740 1 --2- 172.21.15.26:0/1791378859 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cc601eb2f0 0x55cc6020cae0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:43.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.031+0000 7f48cfb3e740 1 --2- 172.21.15.26:0/1791378859 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cc601ea9d0 0x55cc6020c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:43.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.031+0000 7f48cfb3e740 1 -- 172.21.15.26:0/1791378859 >> 172.21.15.26:0/1791378859 conn(0x55cc601e6f60 msgr2=0x55cc601f0b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:17:43.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.031+0000 7f48aaff5640 0 client.14928 ms_handle_reset on v2:172.21.15.76:6834/393308499 2024-09-11T23:17:43.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.031+0000 7f48cfb3e740 1 -- 172.21.15.26:0/1791378859 shutdown_connections 2024-09-11T23:17:43.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.031+0000 7f48cfb3e740 1 -- 172.21.15.26:0/1791378859 wait complete. 2024-09-11T23:17:43.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.031+0000 7f48cfb3e740 20 client.14928 trim_cache size 0 max 16384 2024-09-11T23:17:43.045 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-11T23:17:43.046 INFO:teuthology.orchestra.run.smithi026.stdout: "cluster_fsid": "af7e6fc8-7092-11ef-bcea-c7b262605968", 2024-09-11T23:17:43.046 INFO:teuthology.orchestra.run.smithi026.stdout: "whoami": 4, 2024-09-11T23:17:43.046 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 24443, 2024-09-11T23:17:43.046 INFO:teuthology.orchestra.run.smithi026.stdout: "want_state": "up:active", 2024-09-11T23:17:43.047 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "up:active", 2024-09-11T23:17:43.047 INFO:teuthology.orchestra.run.smithi026.stdout: "fs_name": "cephfs", 2024-09-11T23:17:43.047 INFO:teuthology.orchestra.run.smithi026.stdout: "rank_uptime": 65.063126054999998, 2024-09-11T23:17:43.047 INFO:teuthology.orchestra.run.smithi026.stdout: "mdsmap_epoch": 28, 2024-09-11T23:17:43.047 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch": 85, 2024-09-11T23:17:43.047 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch_barrier": 85, 2024-09-11T23:17:43.047 INFO:teuthology.orchestra.run.smithi026.stdout: "uptime": 107.03229223300001 2024-09-11T23:17:43.047 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-11T23:17:43.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.032+0000 7f48cd765640 1 -- 172.21.15.26:0/788706896 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f48b00a2c10 msgr2=0x7f48b00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:43.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.032+0000 7f48cd765640 1 --2- 172.21.15.26:0/788706896 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f48b00a2c10 0x7f48b00a50d0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f48c0005fd0 tx=0x7f48c00046c0 comp rx=0 tx=0).stop 2024-09-11T23:17:43.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.032+0000 7f48cd765640 1 -- 172.21.15.26:0/788706896 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f48c8102570 msgr2=0x7f48c8197640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:43.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.032+0000 7f48cd765640 1 --2- 172.21.15.26:0/788706896 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f48c8102570 0x7f48c8197640 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f48bc00d950 tx=0x7f48bc00de10 comp rx=0 tx=0).stop 2024-09-11T23:17:43.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.032+0000 7f48cd765640 1 -- 172.21.15.26:0/788706896 shutdown_connections 2024-09-11T23:17:43.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.032+0000 7f48cd765640 1 --2- 172.21.15.26:0/788706896 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f48b00a2c10 0x7f48b00a50d0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:43.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.032+0000 7f48cd765640 1 --2- 172.21.15.26:0/788706896 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f48c8102ef0 0x7f48c8197b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:43.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.032+0000 7f48cd765640 1 --2- 172.21.15.26:0/788706896 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f48c8102570 0x7f48c8197640 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:43.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.032+0000 7f48cd765640 1 --2- 172.21.15.26:0/788706896 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f48c8100e20 0x7f48c8197100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:43.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.032+0000 7f48cd765640 1 -- 172.21.15.26:0/788706896 >> 172.21.15.26:0/788706896 conn(0x7f48c80fc7d0 msgr2=0x7f48c80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:17:43.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.032+0000 7f48cd765640 1 -- 172.21.15.26:0/788706896 shutdown_connections 2024-09-11T23:17:43.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:43.032+0000 7f48cd765640 1 -- 172.21.15.26:0/788706896 wait complete. 2024-09-11T23:17:43.716 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph tell mds.c status 2024-09-11T23:17:43.982 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:17:44.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:17:43 smithi026 ceph-mon[31172]: pgmap v319: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-11T23:17:44.052 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:17:43 smithi076 ceph-mon[35296]: pgmap v319: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-11T23:17:44.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:17:43 smithi136 ceph-mon[35121]: pgmap v319: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-11T23:17:44.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.451+0000 7f115e9aa640 1 -- 172.21.15.26:0/954970969 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1158102ef0 msgr2=0x7f115810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:44.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.451+0000 7f115e9aa640 1 --2- 172.21.15.26:0/954970969 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1158102ef0 0x7f115810da40 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f1150009ca0 tx=0x7f115002a950 comp rx=0 tx=0).stop 2024-09-11T23:17:44.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.451+0000 7f115e9aa640 1 -- 172.21.15.26:0/954970969 shutdown_connections 2024-09-11T23:17:44.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.451+0000 7f115e9aa640 1 --2- 172.21.15.26:0/954970969 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1158102ef0 0x7f115810da40 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:44.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.451+0000 7f115e9aa640 1 --2- 172.21.15.26:0/954970969 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1158102570 0x7f11581029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:44.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.451+0000 7f115e9aa640 1 --2- 172.21.15.26:0/954970969 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1158100e20 0x7f1158101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:44.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.451+0000 7f115e9aa640 1 -- 172.21.15.26:0/954970969 >> 172.21.15.26:0/954970969 conn(0x7f11580fc7d0 msgr2=0x7f11580febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:17:44.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.451+0000 7f115e9aa640 1 -- 172.21.15.26:0/954970969 shutdown_connections 2024-09-11T23:17:44.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.451+0000 7f115e9aa640 1 -- 172.21.15.26:0/954970969 wait complete. 2024-09-11T23:17:44.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.451+0000 7f115e9aa640 1 Processor -- start 2024-09-11T23:17:44.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.452+0000 7f115e9aa640 1 -- start start 2024-09-11T23:17:44.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.452+0000 7f115e9aa640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1158100e20 0x7f11581984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:44.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.452+0000 7f115e9aa640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1158102570 0x7f11581989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:44.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.452+0000 7f115e9aa640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1158102ef0 0x7f115819cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:44.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.452+0000 7f115e9aa640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f1158110fd0 con 0x7f1158100e20 2024-09-11T23:17:44.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.452+0000 7f115e9aa640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f1158110e50 con 0x7f1158102ef0 2024-09-11T23:17:44.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.452+0000 7f115e9aa640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f1158111150 con 0x7f1158102570 2024-09-11T23:17:44.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.452+0000 7f1157fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1158100e20 0x7f11581984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:44.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.452+0000 7f1157fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1158100e20 0x7f11581984b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59714/0 (socket says 172.21.15.26:59714) 2024-09-11T23:17:44.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.452+0000 7f1157fff640 1 -- 172.21.15.26:0/3116712001 learned_addr learned my addr 172.21.15.26:0/3116712001 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:17:44.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.452+0000 7f114ffff640 1 --2- 172.21.15.26:0/3116712001 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1158102570 0x7f11581989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:44.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.452+0000 7f115cf47640 1 --2- 172.21.15.26:0/3116712001 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1158102ef0 0x7f115819cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:44.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.452+0000 7f1157fff640 1 -- 172.21.15.26:0/3116712001 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1158102570 msgr2=0x7f11581989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:44.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.452+0000 7f1157fff640 1 --2- 172.21.15.26:0/3116712001 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1158102570 0x7f11581989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:44.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.452+0000 7f1157fff640 1 -- 172.21.15.26:0/3116712001 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1158102ef0 msgr2=0x7f115819cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:44.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.452+0000 7f1157fff640 1 --2- 172.21.15.26:0/3116712001 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1158102ef0 0x7f115819cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:44.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.452+0000 7f1157fff640 1 -- 172.21.15.26:0/3116712001 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f115819d460 con 0x7f1158100e20 2024-09-11T23:17:44.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.452+0000 7f1157fff640 1 --2- 172.21.15.26:0/3116712001 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1158100e20 0x7f11581984b0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f114800d950 tx=0x7f114800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:17:44.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.453+0000 7f1155ffb640 1 -- 172.21.15.26:0/3116712001 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1148017070 con 0x7f1158100e20 2024-09-11T23:17:44.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.453+0000 7f1155ffb640 1 -- 172.21.15.26:0/3116712001 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1148009c30 con 0x7f1158100e20 2024-09-11T23:17:44.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.453+0000 7f115e9aa640 1 -- 172.21.15.26:0/3116712001 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f115819d750 con 0x7f1158100e20 2024-09-11T23:17:44.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.453+0000 7f1155ffb640 1 -- 172.21.15.26:0/3116712001 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f114800f6b0 con 0x7f1158100e20 2024-09-11T23:17:44.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.453+0000 7f115e9aa640 1 -- 172.21.15.26:0/3116712001 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11581a5030 con 0x7f1158100e20 2024-09-11T23:17:44.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.455+0000 7f1155ffb640 1 -- 172.21.15.26:0/3116712001 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f11480109d0 con 0x7f1158100e20 2024-09-11T23:17:44.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.455+0000 7f1160d83740 1 Processor -- start 2024-09-11T23:17:44.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.455+0000 7f1160d83740 1 -- start start 2024-09-11T23:17:44.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.455+0000 7f1160d83740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561b029ba9d0 0x561b029badb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:44.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.455+0000 7f1160d83740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561b029bb2f0 0x561b029c5670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:44.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.455+0000 7f1160d83740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561b029c5bb0 0x561b029c7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:44.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.455+0000 7f1160d83740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x561b02921090 con 0x561b029bb2f0 2024-09-11T23:17:44.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.456+0000 7f1160d83740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x561b028c8a70 con 0x561b029ba9d0 2024-09-11T23:17:44.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.456+0000 7f1160d83740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x561b0291d050 con 0x561b029c5bb0 2024-09-11T23:17:44.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.456+0000 7f1157fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561b029bb2f0 0x561b029c5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:44.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.456+0000 7f1157fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561b029bb2f0 0x561b029c5670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59722/0 (socket says 172.21.15.26:59722) 2024-09-11T23:17:44.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.456+0000 7f1157fff640 1 -- 172.21.15.26:0/3330360743 learned_addr learned my addr 172.21.15.26:0/3330360743 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:17:44.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.456+0000 7f115cf47640 1 --2- 172.21.15.26:0/3330360743 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561b029ba9d0 0x561b029badb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:44.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.456+0000 7f114ffff640 1 --2- 172.21.15.26:0/3330360743 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561b029c5bb0 0x561b029c7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:44.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.456+0000 7f1157fff640 1 -- 172.21.15.26:0/3330360743 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561b029c5bb0 msgr2=0x561b029c7fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:17:44.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.456+0000 7f1157fff640 1 --2- 172.21.15.26:0/3330360743 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561b029c5bb0 0x561b029c7fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:44.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.456+0000 7f1157fff640 1 -- 172.21.15.26:0/3330360743 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561b029ba9d0 msgr2=0x561b029badb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:17:44.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.456+0000 7f1157fff640 1 --2- 172.21.15.26:0/3330360743 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561b029ba9d0 0x561b029badb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:44.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.456+0000 7f1157fff640 1 -- 172.21.15.26:0/3330360743 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561b028d5130 con 0x561b029bb2f0 2024-09-11T23:17:44.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.456+0000 7f1155ffb640 1 --2- 172.21.15.26:0/3116712001 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f11440a2c90 0x7f11440a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:44.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.456+0000 7f115cf47640 1 --2- 172.21.15.26:0/3116712001 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f11440a2c90 0x7f11440a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:44.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.456+0000 7f1157fff640 1 --2- 172.21.15.26:0/3330360743 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561b029bb2f0 0x561b029c5670 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f11480d2ef0 tx=0x7f11480d5840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb6b5b083ca0c975 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:17:44.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.456+0000 7f1155ffb640 1 -- 172.21.15.26:0/3116712001 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f11480cee00 con 0x7f1158100e20 2024-09-11T23:17:44.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.456+0000 7f114e7fc640 1 -- 172.21.15.26:0/3330360743 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1148097760 con 0x561b029bb2f0 2024-09-11T23:17:44.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.456+0000 7f114e7fc640 1 -- 172.21.15.26:0/3330360743 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f114809bbc0 con 0x561b029bb2f0 2024-09-11T23:17:44.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.457+0000 7f114e7fc640 1 -- 172.21.15.26:0/3330360743 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f114809ae20 con 0x561b029bb2f0 2024-09-11T23:17:44.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.457+0000 7f1160d83740 1 -- 172.21.15.26:0/3330360743 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561b029bb2f0 msgr2=0x561b029c5670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:44.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.457+0000 7f115cf47640 1 --2- 172.21.15.26:0/3116712001 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f11440a2c90 0x7f11440a5150 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f1150005fa0 tx=0x7f1150004a40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:17:44.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.457+0000 7f1160d83740 1 --2- 172.21.15.26:0/3330360743 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561b029bb2f0 0x561b029c5670 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f11480d2ef0 tx=0x7f11480d5840 comp rx=0 tx=0).stop 2024-09-11T23:17:44.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.457+0000 7f1160d83740 1 -- 172.21.15.26:0/3330360743 shutdown_connections 2024-09-11T23:17:44.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.457+0000 7f1160d83740 1 --2- 172.21.15.26:0/3330360743 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561b029c5bb0 0x561b029c7fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:44.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.457+0000 7f1160d83740 1 --2- 172.21.15.26:0/3330360743 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561b029bb2f0 0x561b029c5670 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:44.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.457+0000 7f1160d83740 1 --2- 172.21.15.26:0/3330360743 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561b029ba9d0 0x561b029badb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:44.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.457+0000 7f1160d83740 1 -- 172.21.15.26:0/3330360743 >> 172.21.15.26:0/3330360743 conn(0x561b029b6f60 msgr2=0x561b029b7340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:17:44.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.457+0000 7f1160d83740 1 -- 172.21.15.26:0/3330360743 shutdown_connections 2024-09-11T23:17:44.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.457+0000 7f1160d83740 1 -- 172.21.15.26:0/3330360743 wait complete. 2024-09-11T23:17:44.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.458+0000 7f1160d83740 1 -- start start 2024-09-11T23:17:44.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.458+0000 7f1160d83740 1 Processor -- start 2024-09-11T23:17:44.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.458+0000 7f1160d83740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561b029ba9d0 0x561b029c5440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:44.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.458+0000 7f1160d83740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561b029bb2f0 0x561b029c2790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:44.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.458+0000 7f1160d83740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561b029c5bb0 0x561b029c2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:44.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.458+0000 7f1160d83740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x561b029c8780 con 0x561b029bb2f0 2024-09-11T23:17:44.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.458+0000 7f1160d83740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x561b028dbf20 con 0x561b029ba9d0 2024-09-11T23:17:44.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.458+0000 7f1160d83740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x561b029c8900 con 0x561b029c5bb0 2024-09-11T23:17:44.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.458+0000 7f1160d83740 10 client.0 _ll_register_callbacks cb 0x561b028a5030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:17:44.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.458+0000 7f114ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561b029bb2f0 0x561b029c2790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:44.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.458+0000 7f114ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561b029bb2f0 0x561b029c2790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59730/0 (socket says 172.21.15.26:59730) 2024-09-11T23:17:44.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.458+0000 7f114ffff640 1 -- 172.21.15.26:0/4141522523 learned_addr learned my addr 172.21.15.26:0/4141522523 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:17:44.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.458+0000 7f115cf47640 1 --2- 172.21.15.26:0/4141522523 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561b029c5bb0 0x561b029c2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:44.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.459+0000 7f1157fff640 1 --2- 172.21.15.26:0/4141522523 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561b029ba9d0 0x561b029c5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:44.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.459+0000 7f114ffff640 1 -- 172.21.15.26:0/4141522523 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561b029c5bb0 msgr2=0x561b029c2cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:17:44.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.459+0000 7f114ffff640 1 --2- 172.21.15.26:0/4141522523 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561b029c5bb0 0x561b029c2cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:44.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.459+0000 7f114ffff640 1 -- 172.21.15.26:0/4141522523 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561b029ba9d0 msgr2=0x561b029c5440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:17:44.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.459+0000 7f114ffff640 1 --2- 172.21.15.26:0/4141522523 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561b029ba9d0 0x561b029c5440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:44.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.459+0000 7f114ffff640 1 -- 172.21.15.26:0/4141522523 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561b026922b0 con 0x561b029bb2f0 2024-09-11T23:17:44.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.459+0000 7f1157fff640 1 --2- 172.21.15.26:0/4141522523 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561b029ba9d0 0x561b029c5440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:17:44.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.459+0000 7f115cf47640 1 --2- 172.21.15.26:0/4141522523 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561b029c5bb0 0x561b029c2cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:17:44.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.459+0000 7f114ffff640 1 --2- 172.21.15.26:0/4141522523 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561b029bb2f0 0x561b029c2790 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f114000b920 tx=0x7f114000bde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc1f555ba9bfafa server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:17:44.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.459+0000 7f11377fe640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:17:44.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.459+0000 7f11377fe640 1 -- 172.21.15.26:0/4141522523 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1140003ea0 con 0x561b029bb2f0 2024-09-11T23:17:44.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.459+0000 7f11377fe640 1 -- 172.21.15.26:0/4141522523 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f114001b640 con 0x561b029bb2f0 2024-09-11T23:17:44.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.459+0000 7f1160d83740 1 -- 172.21.15.26:0/4141522523 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f114809bbc0 con 0x561b029bb2f0 2024-09-11T23:17:44.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.459+0000 7f11377fe640 1 -- 172.21.15.26:0/4141522523 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f114001b840 con 0x561b029bb2f0 2024-09-11T23:17:44.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.459+0000 7f1160d83740 10 client.14940 fetch_fsmap learned FSMap version 28 2024-09-11T23:17:44.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.459+0000 7f1160d83740 1 -- 172.21.15.26:0/4141522523 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x561b029d4140 con 0x561b029bb2f0 2024-09-11T23:17:44.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.459+0000 7f11377fe640 1 -- 172.21.15.26:0/4141522523 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f114809bbc0 con 0x561b029bb2f0 2024-09-11T23:17:44.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.459+0000 7f11377fe640 1 -- 172.21.15.26:0/4141522523 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f1140019020 con 0x561b029bb2f0 2024-09-11T23:17:44.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.460+0000 7f1160d83740 10 client.14940 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:17:44.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.460+0000 7f1160d83740 10 client.14940 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-11T23:17:44.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.460+0000 7f1160d83740 20 client.14940 populate_metadata read hostname 'smithi026' 2024-09-11T23:17:44.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.460+0000 7f1160d83740 1 --2- 172.21.15.26:0/4141522523 >> [v2:172.21.15.136:6836/1633082387,v1:172.21.15.136:6837/1633082387] conn(0x561b029d67b0 0x561b029d7c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:44.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.460+0000 7f1160d83740 4 client.14940 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:17:44.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.460+0000 7f1160d83740 1 -- 172.21.15.26:0/4141522523 --> [v2:172.21.15.136:6836/1633082387,v1:172.21.15.136:6837/1633082387] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561b0292e600 con 0x561b029d67b0 2024-09-11T23:17:44.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.460+0000 7f1157fff640 1 --2- 172.21.15.26:0/4141522523 >> [v2:172.21.15.136:6836/1633082387,v1:172.21.15.136:6837/1633082387] conn(0x561b029d67b0 0x561b029d7c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:44.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.460+0000 7f11377fe640 10 client.14940 ms_handle_connect on v2:172.21.15.136:6836/1633082387 2024-09-11T23:17:44.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.460+0000 7f1157fff640 1 --2- 172.21.15.26:0/4141522523 >> [v2:172.21.15.136:6836/1633082387,v1:172.21.15.136:6837/1633082387] conn(0x561b029d67b0 0x561b029d7c20 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=9f2c5478320c8326 server_cookie=beabd63c0c4fb3c3 in_seq=0 out_seq=0 2024-09-11T23:17:44.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.461+0000 7f11377fe640 1 -- 172.21.15.26:0/4141522523 <== mds.3 v2:172.21.15.136:6836/1633082387 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561b0292e600 con 0x561b029d67b0 2024-09-11T23:17:44.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.461+0000 7f11377fe640 10 client.14940 handle_command_reply: tid=0 2024-09-11T23:17:44.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.461+0000 7f1160d83740 1 client.14940 shutdown 2024-09-11T23:17:44.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.461+0000 7f1160d83740 1 -- 172.21.15.26:0/4141522523 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561b029bb2f0 msgr2=0x561b029c2790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:44.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.461+0000 7f1160d83740 1 --2- 172.21.15.26:0/4141522523 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561b029bb2f0 0x561b029c2790 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f114000b920 tx=0x7f114000bde0 comp rx=0 tx=0).stop 2024-09-11T23:17:44.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.461+0000 7f1160d83740 1 -- 172.21.15.26:0/4141522523 shutdown_connections 2024-09-11T23:17:44.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.461+0000 7f1160d83740 1 --2- 172.21.15.26:0/4141522523 >> [v2:172.21.15.136:6836/1633082387,v1:172.21.15.136:6837/1633082387] conn(0x561b029d67b0 0x561b029d7c20 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:44.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.461+0000 7f1160d83740 1 --2- 172.21.15.26:0/4141522523 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561b029c5bb0 0x561b029c2cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:44.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.461+0000 7f1160d83740 1 --2- 172.21.15.26:0/4141522523 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561b029bb2f0 0x561b029c2790 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:44.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.461+0000 7f11377fe640 0 client.14940 ms_handle_reset on v2:172.21.15.136:6836/1633082387 2024-09-11T23:17:44.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.461+0000 7f1160d83740 1 --2- 172.21.15.26:0/4141522523 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561b029ba9d0 0x561b029c5440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:44.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.461+0000 7f1160d83740 1 -- 172.21.15.26:0/4141522523 >> 172.21.15.26:0/4141522523 conn(0x561b029b6f60 msgr2=0x561b029bcc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:17:44.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.461+0000 7f1160d83740 1 -- 172.21.15.26:0/4141522523 shutdown_connections 2024-09-11T23:17:44.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.462+0000 7f1160d83740 1 -- 172.21.15.26:0/4141522523 wait complete. 2024-09-11T23:17:44.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.462+0000 7f1160d83740 20 client.14940 trim_cache size 0 max 16384 2024-09-11T23:17:44.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.479+0000 7f1160d83740 1 Processor -- start 2024-09-11T23:17:44.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.479+0000 7f1160d83740 1 -- start start 2024-09-11T23:17:44.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.479+0000 7f1160d83740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561b029ba9d0 0x561b029d7c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:44.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.479+0000 7f1160d83740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561b029bb2f0 0x561b029bfa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:44.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.479+0000 7f1160d83740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561b029c5bb0 0x561b029bff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:44.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.480+0000 7f1160d83740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x561b029d5f60 con 0x561b029bb2f0 2024-09-11T23:17:44.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.480+0000 7f1160d83740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x561b029d6630 con 0x561b029c5bb0 2024-09-11T23:17:44.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.480+0000 7f1160d83740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x561b029d6d00 con 0x561b029ba9d0 2024-09-11T23:17:44.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.480+0000 7f114ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561b029bb2f0 0x561b029bfa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:44.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.480+0000 7f114ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561b029bb2f0 0x561b029bfa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59744/0 (socket says 172.21.15.26:59744) 2024-09-11T23:17:44.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.480+0000 7f114ffff640 1 -- 172.21.15.26:0/855874136 learned_addr learned my addr 172.21.15.26:0/855874136 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:17:44.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.480+0000 7f115cf47640 1 --2- 172.21.15.26:0/855874136 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561b029c5bb0 0x561b029bff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:44.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.480+0000 7f1157fff640 1 --2- 172.21.15.26:0/855874136 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561b029ba9d0 0x561b029d7c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:44.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.480+0000 7f114ffff640 1 -- 172.21.15.26:0/855874136 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561b029ba9d0 msgr2=0x561b029d7c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:17:44.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.480+0000 7f114ffff640 1 --2- 172.21.15.26:0/855874136 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561b029ba9d0 0x561b029d7c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:44.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.480+0000 7f114ffff640 1 -- 172.21.15.26:0/855874136 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561b029c5bb0 msgr2=0x561b029bff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:17:44.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.480+0000 7f114ffff640 1 --2- 172.21.15.26:0/855874136 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561b029c5bb0 0x561b029bff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:44.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.480+0000 7f114ffff640 1 -- 172.21.15.26:0/855874136 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561b029c0ce0 con 0x561b029bb2f0 2024-09-11T23:17:44.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.480+0000 7f1157fff640 1 --2- 172.21.15.26:0/855874136 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561b029ba9d0 0x561b029d7c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:17:44.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.480+0000 7f114ffff640 1 --2- 172.21.15.26:0/855874136 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561b029bb2f0 0x561b029bfa10 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f1140011e80 tx=0x7f1140005970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b56603c9a376654 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:17:44.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.480+0000 7f114dffb640 1 -- 172.21.15.26:0/855874136 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1140000e60 con 0x561b029bb2f0 2024-09-11T23:17:44.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.480+0000 7f114dffb640 1 -- 172.21.15.26:0/855874136 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f114002a070 con 0x561b029bb2f0 2024-09-11T23:17:44.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.480+0000 7f114dffb640 1 -- 172.21.15.26:0/855874136 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1140005400 con 0x561b029bb2f0 2024-09-11T23:17:44.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.481+0000 7f1160d83740 1 -- 172.21.15.26:0/855874136 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561b029bb2f0 msgr2=0x561b029bfa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:44.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.481+0000 7f1160d83740 1 --2- 172.21.15.26:0/855874136 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561b029bb2f0 0x561b029bfa10 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f1140011e80 tx=0x7f1140005970 comp rx=0 tx=0).stop 2024-09-11T23:17:44.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.481+0000 7f1160d83740 1 -- 172.21.15.26:0/855874136 shutdown_connections 2024-09-11T23:17:44.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.481+0000 7f1160d83740 1 --2- 172.21.15.26:0/855874136 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561b029c5bb0 0x561b029bff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:44.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.481+0000 7f1160d83740 1 --2- 172.21.15.26:0/855874136 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561b029bb2f0 0x561b029bfa10 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:44.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.481+0000 7f1160d83740 1 --2- 172.21.15.26:0/855874136 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561b029ba9d0 0x561b029d7c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:44.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.481+0000 7f1160d83740 1 -- 172.21.15.26:0/855874136 >> 172.21.15.26:0/855874136 conn(0x561b029b6f60 msgr2=0x561b029c1060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:17:44.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.481+0000 7f1160d83740 1 -- 172.21.15.26:0/855874136 shutdown_connections 2024-09-11T23:17:44.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.481+0000 7f1160d83740 1 -- 172.21.15.26:0/855874136 wait complete. 2024-09-11T23:17:44.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.481+0000 7f1160d83740 1 -- start start 2024-09-11T23:17:44.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.482+0000 7f1160d83740 1 Processor -- start 2024-09-11T23:17:44.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.482+0000 7f1160d83740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561b029ba9d0 0x561b029bede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:44.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.482+0000 7f1160d83740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561b029bb2f0 0x561b029bf320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:44.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.482+0000 7f1160d83740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561b029c5bb0 0x561b029dee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:44.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.482+0000 7f1160d83740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x561b029d7aa0 con 0x561b029bb2f0 2024-09-11T23:17:44.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.482+0000 7f1160d83740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x561b029d73d0 con 0x561b029c5bb0 2024-09-11T23:17:44.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.482+0000 7f1160d83740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x561b029d8170 con 0x561b029ba9d0 2024-09-11T23:17:44.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.482+0000 7f1160d83740 10 client.0 _ll_register_callbacks cb 0x7f1140004160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:17:44.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.482+0000 7f114ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561b029bb2f0 0x561b029bf320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:44.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.482+0000 7f114ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561b029bb2f0 0x561b029bf320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59750/0 (socket says 172.21.15.26:59750) 2024-09-11T23:17:44.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.482+0000 7f114ffff640 1 -- 172.21.15.26:0/3248389157 learned_addr learned my addr 172.21.15.26:0/3248389157 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:17:44.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.482+0000 7f115cf47640 1 --2- 172.21.15.26:0/3248389157 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561b029c5bb0 0x561b029dee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:44.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.482+0000 7f114ffff640 1 -- 172.21.15.26:0/3248389157 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561b029ba9d0 msgr2=0x561b029bede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:17:44.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.482+0000 7f114ffff640 1 --2- 172.21.15.26:0/3248389157 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561b029ba9d0 0x561b029bede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:44.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.482+0000 7f114ffff640 1 -- 172.21.15.26:0/3248389157 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561b029c5bb0 msgr2=0x561b029dee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:17:44.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.482+0000 7f114ffff640 1 --2- 172.21.15.26:0/3248389157 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561b029c5bb0 0x561b029dee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:44.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.482+0000 7f114ffff640 1 -- 172.21.15.26:0/3248389157 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561b029bfbc0 con 0x561b029bb2f0 2024-09-11T23:17:44.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.482+0000 7f115cf47640 1 --2- 172.21.15.26:0/3248389157 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561b029c5bb0 0x561b029dee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:17:44.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.482+0000 7f114ffff640 1 --2- 172.21.15.26:0/3248389157 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561b029bb2f0 0x561b029bf320 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f1140011c90 tx=0x7f1140005410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eadcead8700c6f21 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:17:44.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.482+0000 7f11377fe640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:17:44.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.483+0000 7f11377fe640 1 -- 172.21.15.26:0/3248389157 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f114001bdf0 con 0x561b029bb2f0 2024-09-11T23:17:44.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.483+0000 7f11377fe640 1 -- 172.21.15.26:0/3248389157 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f1140005600 con 0x561b029bb2f0 2024-09-11T23:17:44.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.483+0000 7f1160d83740 1 -- 172.21.15.26:0/3248389157 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f114002a070 con 0x561b029bb2f0 2024-09-11T23:17:44.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.483+0000 7f11377fe640 1 -- 172.21.15.26:0/3248389157 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f114003f050 con 0x561b029bb2f0 2024-09-11T23:17:44.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.483+0000 7f1160d83740 10 client.14946 fetch_fsmap learned FSMap version 28 2024-09-11T23:17:44.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.483+0000 7f1160d83740 1 -- 172.21.15.26:0/3248389157 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x561b029e03f0 con 0x561b029bb2f0 2024-09-11T23:17:44.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.483+0000 7f11377fe640 1 -- 172.21.15.26:0/3248389157 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f114002a070 con 0x561b029bb2f0 2024-09-11T23:17:44.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.483+0000 7f11377fe640 1 -- 172.21.15.26:0/3248389157 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f1140036070 con 0x561b029bb2f0 2024-09-11T23:17:44.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.483+0000 7f1160d83740 10 client.14946 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:17:44.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.483+0000 7f1160d83740 10 client.14946 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-11T23:17:44.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.483+0000 7f1160d83740 20 client.14946 populate_metadata read hostname 'smithi026' 2024-09-11T23:17:44.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.483+0000 7f1160d83740 1 --2- 172.21.15.26:0/3248389157 >> [v2:172.21.15.136:6836/1633082387,v1:172.21.15.136:6837/1633082387] conn(0x561b02a44d80 0x561b02a43a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:44.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.483+0000 7f1160d83740 4 client.14946 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-11T23:17:44.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.483+0000 7f1160d83740 1 -- 172.21.15.26:0/3248389157 --> [v2:172.21.15.136:6836/1633082387,v1:172.21.15.136:6837/1633082387] -- command(tid 0: {"prefix": "status"}) -- 0x561b02928b10 con 0x561b02a44d80 2024-09-11T23:17:44.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.483+0000 7f1157fff640 1 --2- 172.21.15.26:0/3248389157 >> [v2:172.21.15.136:6836/1633082387,v1:172.21.15.136:6837/1633082387] conn(0x561b02a44d80 0x561b02a43a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:44.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.484+0000 7f1157fff640 1 --2- 172.21.15.26:0/3248389157 >> [v2:172.21.15.136:6836/1633082387,v1:172.21.15.136:6837/1633082387] conn(0x561b02a44d80 0x561b02a43a90 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=524094c952cf79b7 server_cookie=fbc9ee7c9d6cbb8b in_seq=0 out_seq=0 2024-09-11T23:17:44.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.484+0000 7f11377fe640 10 client.14946 ms_handle_connect on v2:172.21.15.136:6836/1633082387 2024-09-11T23:17:44.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.484+0000 7f11377fe640 1 -- 172.21.15.26:0/3248389157 <== mds.3 v2:172.21.15.136:6836/1633082387 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x561b02928b10 con 0x561b02a44d80 2024-09-11T23:17:44.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.484+0000 7f11377fe640 10 client.14946 handle_command_reply: tid=0 2024-09-11T23:17:44.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.484+0000 7f1160d83740 1 client.14946 shutdown 2024-09-11T23:17:44.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.484+0000 7f1160d83740 1 -- 172.21.15.26:0/3248389157 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561b029bb2f0 msgr2=0x561b029bf320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:44.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.484+0000 7f1160d83740 1 --2- 172.21.15.26:0/3248389157 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561b029bb2f0 0x561b029bf320 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f1140011c90 tx=0x7f1140005410 comp rx=0 tx=0).stop 2024-09-11T23:17:44.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.484+0000 7f1160d83740 1 -- 172.21.15.26:0/3248389157 shutdown_connections 2024-09-11T23:17:44.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.484+0000 7f1160d83740 1 --2- 172.21.15.26:0/3248389157 >> [v2:172.21.15.136:6836/1633082387,v1:172.21.15.136:6837/1633082387] conn(0x561b02a44d80 0x561b02a43a90 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:44.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.484+0000 7f1160d83740 1 --2- 172.21.15.26:0/3248389157 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561b029c5bb0 0x561b029dee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:44.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.484+0000 7f1160d83740 1 --2- 172.21.15.26:0/3248389157 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561b029bb2f0 0x561b029bf320 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:44.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.484+0000 7f1160d83740 1 --2- 172.21.15.26:0/3248389157 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561b029ba9d0 0x561b029bede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:44.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.484+0000 7f1160d83740 1 -- 172.21.15.26:0/3248389157 >> 172.21.15.26:0/3248389157 conn(0x561b029b6f60 msgr2=0x561b029bc470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:17:44.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.484+0000 7f11377fe640 0 client.14946 ms_handle_reset on v2:172.21.15.136:6836/1633082387 2024-09-11T23:17:44.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.484+0000 7f1160d83740 1 -- 172.21.15.26:0/3248389157 shutdown_connections 2024-09-11T23:17:44.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.484+0000 7f1160d83740 1 -- 172.21.15.26:0/3248389157 wait complete. 2024-09-11T23:17:44.511 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-11T23:17:44.512 INFO:teuthology.orchestra.run.smithi026.stdout: "cluster_fsid": "af7e6fc8-7092-11ef-bcea-c7b262605968", 2024-09-11T23:17:44.512 INFO:teuthology.orchestra.run.smithi026.stdout: "whoami": 3, 2024-09-11T23:17:44.512 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 24448, 2024-09-11T23:17:44.512 INFO:teuthology.orchestra.run.smithi026.stdout: "want_state": "up:active", 2024-09-11T23:17:44.512 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "up:active", 2024-09-11T23:17:44.512 INFO:teuthology.orchestra.run.smithi026.stdout: "fs_name": "cephfs", 2024-09-11T23:17:44.512 INFO:teuthology.orchestra.run.smithi026.stdout: "rank_uptime": 67.521670076000007, 2024-09-11T23:17:44.512 INFO:teuthology.orchestra.run.smithi026.stdout: "mdsmap_epoch": 28, 2024-09-11T23:17:44.512 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch": 85, 2024-09-11T23:17:44.512 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch_barrier": 85, 2024-09-11T23:17:44.512 INFO:teuthology.orchestra.run.smithi026.stdout: "uptime": 103.49966275600001 2024-09-11T23:17:44.512 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-11T23:17:44.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.484+0000 7f1160d83740 20 client.14946 trim_cache size 0 max 16384 2024-09-11T23:17:44.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.485+0000 7f115e9aa640 1 -- 172.21.15.26:0/3116712001 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f11440a2c90 msgr2=0x7f11440a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:44.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.485+0000 7f115e9aa640 1 --2- 172.21.15.26:0/3116712001 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f11440a2c90 0x7f11440a5150 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f1150005fa0 tx=0x7f1150004a40 comp rx=0 tx=0).stop 2024-09-11T23:17:44.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.485+0000 7f115e9aa640 1 -- 172.21.15.26:0/3116712001 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1158100e20 msgr2=0x7f11581984b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:44.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.485+0000 7f115e9aa640 1 --2- 172.21.15.26:0/3116712001 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1158100e20 0x7f11581984b0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f114800d950 tx=0x7f114800de10 comp rx=0 tx=0).stop 2024-09-11T23:17:44.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.485+0000 7f115e9aa640 1 -- 172.21.15.26:0/3116712001 shutdown_connections 2024-09-11T23:17:44.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.485+0000 7f115e9aa640 1 --2- 172.21.15.26:0/3116712001 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f11440a2c90 0x7f11440a5150 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:44.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.485+0000 7f115e9aa640 1 --2- 172.21.15.26:0/3116712001 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1158102ef0 0x7f115819cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:44.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.485+0000 7f115e9aa640 1 --2- 172.21.15.26:0/3116712001 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1158102570 0x7f11581989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:44.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.485+0000 7f115e9aa640 1 --2- 172.21.15.26:0/3116712001 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1158100e20 0x7f11581984b0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:44.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.485+0000 7f115e9aa640 1 -- 172.21.15.26:0/3116712001 >> 172.21.15.26:0/3116712001 conn(0x7f11580fc7d0 msgr2=0x7f115806a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:17:44.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.485+0000 7f115e9aa640 1 -- 172.21.15.26:0/3116712001 shutdown_connections 2024-09-11T23:17:44.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:44.485+0000 7f115e9aa640 1 -- 172.21.15.26:0/3116712001 wait complete. 2024-09-11T23:17:45.238 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph tell mds.h status 2024-09-11T23:17:45.498 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:17:45.901 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:17:45 smithi026 ceph-mon[31172]: pgmap v320: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-11T23:17:46.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:45.986+0000 7fa57cbc1640 1 -- 172.21.15.26:0/3553506224 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa578101de0 msgr2=0x7fa5781021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:46.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:45.986+0000 7fa57cbc1640 1 --2- 172.21.15.26:0/3553506224 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa578101de0 0x7fa5781021c0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fa568009900 tx=0x7fa56802a410 comp rx=0 tx=0).stop 2024-09-11T23:17:46.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:45.986+0000 7fa57cbc1640 1 -- 172.21.15.26:0/3553506224 shutdown_connections 2024-09-11T23:17:46.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:45.986+0000 7fa57cbc1640 1 --2- 172.21.15.26:0/3553506224 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa578102790 0x7fa5781067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:46.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:45.986+0000 7fa57cbc1640 1 --2- 172.21.15.26:0/3553506224 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa578106f10 0x7fa57810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:46.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:45.986+0000 7fa57cbc1640 1 --2- 172.21.15.26:0/3553506224 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa578101de0 0x7fa5781021c0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:46.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:45.986+0000 7fa57cbc1640 1 -- 172.21.15.26:0/3553506224 >> 172.21.15.26:0/3553506224 conn(0x7fa5780fd8c0 msgr2=0x7fa5780ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:17:46.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:45.986+0000 7fa57cbc1640 1 -- 172.21.15.26:0/3553506224 shutdown_connections 2024-09-11T23:17:46.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:45.986+0000 7fa57cbc1640 1 -- 172.21.15.26:0/3553506224 wait complete. 2024-09-11T23:17:46.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:45.987+0000 7fa57cbc1640 1 Processor -- start 2024-09-11T23:17:46.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:45.987+0000 7fa57cbc1640 1 -- start start 2024-09-11T23:17:46.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:45.987+0000 7fa57cbc1640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa578101de0 0x7fa5781984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:46.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:45.987+0000 7fa57cbc1640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa578102790 0x7fa5781989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:46.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:45.987+0000 7fa57cbc1640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa578106f10 0x7fa57819cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:46.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:45.987+0000 7fa57cbc1640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fa578111f90 con 0x7fa578106f10 2024-09-11T23:17:46.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:45.987+0000 7fa57cbc1640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fa578111e10 con 0x7fa578101de0 2024-09-11T23:17:46.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:45.987+0000 7fa57cbc1640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fa578112110 con 0x7fa578102790 2024-09-11T23:17:46.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:45.987+0000 7fa576d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa578106f10 0x7fa57819cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:46.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:45.987+0000 7fa57659c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa578101de0 0x7fa5781984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:46.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:45.987+0000 7fa57659c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa578101de0 0x7fa5781984b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:55578/0 (socket says 172.21.15.26:55578) 2024-09-11T23:17:46.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:45.987+0000 7fa57659c640 1 -- 172.21.15.26:0/2711699198 learned_addr learned my addr 172.21.15.26:0/2711699198 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:17:46.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:45.987+0000 7fa575d9b640 1 --2- 172.21.15.26:0/2711699198 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa578102790 0x7fa5781989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:46.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:45.987+0000 7fa576d9d640 1 -- 172.21.15.26:0/2711699198 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa578102790 msgr2=0x7fa5781989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:46.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:45.987+0000 7fa576d9d640 1 --2- 172.21.15.26:0/2711699198 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa578102790 0x7fa5781989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:46.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:45.987+0000 7fa576d9d640 1 -- 172.21.15.26:0/2711699198 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa578101de0 msgr2=0x7fa5781984b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:46.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:45.987+0000 7fa576d9d640 1 --2- 172.21.15.26:0/2711699198 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa578101de0 0x7fa5781984b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:46.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:45.987+0000 7fa576d9d640 1 -- 172.21.15.26:0/2711699198 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa57819d460 con 0x7fa578106f10 2024-09-11T23:17:46.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:45.988+0000 7fa576d9d640 1 --2- 172.21.15.26:0/2711699198 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa578106f10 0x7fa57819cd80 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fa57000be20 tx=0x7fa57000bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:17:46.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:45.988+0000 7fa5677fe640 1 -- 172.21.15.26:0/2711699198 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa570019070 con 0x7fa578106f10 2024-09-11T23:17:46.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:45.988+0000 7fa57cbc1640 1 -- 172.21.15.26:0/2711699198 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa57819d750 con 0x7fa578106f10 2024-09-11T23:17:46.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:45.988+0000 7fa5677fe640 1 -- 172.21.15.26:0/2711699198 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa570014920 con 0x7fa578106f10 2024-09-11T23:17:46.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:45.988+0000 7fa5677fe640 1 -- 172.21.15.26:0/2711699198 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa570012920 con 0x7fa578106f10 2024-09-11T23:17:46.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:45.988+0000 7fa57cbc1640 1 -- 172.21.15.26:0/2711699198 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5781a5030 con 0x7fa578106f10 2024-09-11T23:17:46.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:45.990+0000 7fa5677fe640 1 -- 172.21.15.26:0/2711699198 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fa57001c070 con 0x7fa578106f10 2024-09-11T23:17:46.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:45.990+0000 7fa57ef9a740 1 Processor -- start 2024-09-11T23:17:46.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:45.991+0000 7fa57ef9a740 1 -- start start 2024-09-11T23:17:46.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:45.991+0000 7fa57ef9a740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fcd2e9a9d0 0x55fcd2e9adb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:46.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:45.991+0000 7fa57ef9a740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fcd2e9b2f0 0x55fcd2ea5670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:46.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:45.991+0000 7fa57ef9a740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fcd2ea5bb0 0x55fcd2ea7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:46.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:45.991+0000 7fa57ef9a740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55fcd2e01090 con 0x55fcd2e9b2f0 2024-09-11T23:17:46.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:45.991+0000 7fa57ef9a740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55fcd2da8a70 con 0x55fcd2ea5bb0 2024-09-11T23:17:46.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:45.991+0000 7fa57ef9a740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55fcd2dfd050 con 0x55fcd2e9a9d0 2024-09-11T23:17:46.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:45.991+0000 7fa5677fe640 1 --2- 172.21.15.26:0/2711699198 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fa5600a2c10 0x7fa5600a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:46.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:45.991+0000 7fa57659c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fcd2e9b2f0 0x55fcd2ea5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:46.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:45.991+0000 7fa57659c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fcd2e9b2f0 0x55fcd2ea5670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59772/0 (socket says 172.21.15.26:59772) 2024-09-11T23:17:46.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:45.991+0000 7fa57659c640 1 -- 172.21.15.26:0/438438008 learned_addr learned my addr 172.21.15.26:0/438438008 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:17:46.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:45.991+0000 7fa576d9d640 1 --2- 172.21.15.26:0/2711699198 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fa5600a2c10 0x7fa5600a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:46.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:45.991+0000 7fa575d9b640 1 --2- 172.21.15.26:0/438438008 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fcd2ea5bb0 0x55fcd2ea7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:46.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:45.991+0000 7fa575d9b640 1 --2- 172.21.15.26:0/438438008 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fcd2e9a9d0 0x55fcd2e9adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:46.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:45.991+0000 7fa5677fe640 1 -- 172.21.15.26:0/2711699198 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fa5700ce960 con 0x7fa578106f10 2024-09-11T23:17:46.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:45.992+0000 7fa57659c640 1 -- 172.21.15.26:0/438438008 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fcd2e9a9d0 msgr2=0x55fcd2e9adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:17:46.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:45.992+0000 7fa57659c640 1 --2- 172.21.15.26:0/438438008 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fcd2e9a9d0 0x55fcd2e9adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:46.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:45.992+0000 7fa57659c640 1 -- 172.21.15.26:0/438438008 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fcd2ea5bb0 msgr2=0x55fcd2ea7fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:17:46.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:45.992+0000 7fa57659c640 1 --2- 172.21.15.26:0/438438008 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fcd2ea5bb0 0x55fcd2ea7fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:46.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:45.992+0000 7fa57659c640 1 -- 172.21.15.26:0/438438008 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fcd2db5130 con 0x55fcd2e9b2f0 2024-09-11T23:17:46.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:45.992+0000 7fa575d9b640 1 --2- 172.21.15.26:0/438438008 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fcd2ea5bb0 0x55fcd2ea7fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:17:46.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:45.992+0000 7fa576d9d640 1 --2- 172.21.15.26:0/2711699198 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fa5600a2c10 0x7fa5600a50d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa570065e30 tx=0x7fa57009e040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:17:46.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:45.992+0000 7fa57659c640 1 --2- 172.21.15.26:0/438438008 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fcd2e9b2f0 0x55fcd2ea5670 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fa568009a20 tx=0x7fa568032d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aafc156c6bd77371 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:17:46.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:45.992+0000 7fa564ff9640 1 -- 172.21.15.26:0/438438008 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa568040070 con 0x55fcd2e9b2f0 2024-09-11T23:17:46.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:45.992+0000 7fa564ff9640 1 -- 172.21.15.26:0/438438008 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa568031870 con 0x55fcd2e9b2f0 2024-09-11T23:17:46.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:45.992+0000 7fa564ff9640 1 -- 172.21.15.26:0/438438008 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa568035040 con 0x55fcd2e9b2f0 2024-09-11T23:17:46.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:45.993+0000 7fa57ef9a740 1 -- 172.21.15.26:0/438438008 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fcd2e9b2f0 msgr2=0x55fcd2ea5670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:46.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:45.993+0000 7fa57ef9a740 1 --2- 172.21.15.26:0/438438008 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fcd2e9b2f0 0x55fcd2ea5670 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fa568009a20 tx=0x7fa568032d60 comp rx=0 tx=0).stop 2024-09-11T23:17:46.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:45.993+0000 7fa57ef9a740 1 -- 172.21.15.26:0/438438008 shutdown_connections 2024-09-11T23:17:46.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:45.993+0000 7fa57ef9a740 1 --2- 172.21.15.26:0/438438008 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fcd2ea5bb0 0x55fcd2ea7fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:46.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:45.993+0000 7fa57ef9a740 1 --2- 172.21.15.26:0/438438008 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fcd2e9b2f0 0x55fcd2ea5670 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:46.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:45.993+0000 7fa57ef9a740 1 --2- 172.21.15.26:0/438438008 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fcd2e9a9d0 0x55fcd2e9adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:46.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:45.993+0000 7fa57ef9a740 1 -- 172.21.15.26:0/438438008 >> 172.21.15.26:0/438438008 conn(0x55fcd2e96f60 msgr2=0x55fcd2e97340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:17:46.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:45.993+0000 7fa57ef9a740 1 -- 172.21.15.26:0/438438008 shutdown_connections 2024-09-11T23:17:46.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:45.993+0000 7fa57ef9a740 1 -- 172.21.15.26:0/438438008 wait complete. 2024-09-11T23:17:46.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:45.993+0000 7fa57ef9a740 1 -- start start 2024-09-11T23:17:46.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:45.993+0000 7fa57ef9a740 1 Processor -- start 2024-09-11T23:17:46.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:45.994+0000 7fa57ef9a740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fcd2e9a9d0 0x55fcd2ea5490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:46.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:45.994+0000 7fa57ef9a740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fcd2e9b2f0 0x55fcd2ea27e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:46.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:45.994+0000 7fa57ef9a740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fcd2ea5bb0 0x55fcd2ea2d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:46.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:45.994+0000 7fa57ef9a740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55fcd2ea8780 con 0x55fcd2e9b2f0 2024-09-11T23:17:46.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:45.994+0000 7fa57ef9a740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55fcd2dbbf20 con 0x55fcd2e9a9d0 2024-09-11T23:17:46.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:45.994+0000 7fa57ef9a740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55fcd2ea8900 con 0x55fcd2ea5bb0 2024-09-11T23:17:46.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:45.994+0000 7fa57ef9a740 10 client.0 _ll_register_callbacks cb 0x55fcd2d85030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:17:46.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:45.994+0000 7fa57659c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fcd2e9b2f0 0x55fcd2ea27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:46.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:45.994+0000 7fa57659c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fcd2e9b2f0 0x55fcd2ea27e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59780/0 (socket says 172.21.15.26:59780) 2024-09-11T23:17:46.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:45.994+0000 7fa57659c640 1 -- 172.21.15.26:0/3739489157 learned_addr learned my addr 172.21.15.26:0/3739489157 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:17:46.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:45.994+0000 7fa575d9b640 1 --2- 172.21.15.26:0/3739489157 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fcd2e9a9d0 0x55fcd2ea5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:46.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:45.994+0000 7fa575d9b640 1 --2- 172.21.15.26:0/3739489157 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fcd2ea5bb0 0x55fcd2ea2d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:46.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:45.994+0000 7fa57659c640 1 -- 172.21.15.26:0/3739489157 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fcd2ea5bb0 msgr2=0x55fcd2ea2d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:17:46.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:45.994+0000 7fa57659c640 1 --2- 172.21.15.26:0/3739489157 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fcd2ea5bb0 0x55fcd2ea2d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:46.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:45.994+0000 7fa57659c640 1 -- 172.21.15.26:0/3739489157 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fcd2e9a9d0 msgr2=0x55fcd2ea5490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:17:46.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:45.994+0000 7fa57659c640 1 --2- 172.21.15.26:0/3739489157 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fcd2e9a9d0 0x55fcd2ea5490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:46.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:45.994+0000 7fa57659c640 1 -- 172.21.15.26:0/3739489157 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fcd2b722b0 con 0x55fcd2e9b2f0 2024-09-11T23:17:46.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:45.994+0000 7fa575d9b640 1 --2- 172.21.15.26:0/3739489157 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fcd2ea5bb0 0x55fcd2ea2d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:17:46.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:45.994+0000 7fa57659c640 1 --2- 172.21.15.26:0/3739489157 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fcd2e9b2f0 0x55fcd2ea27e0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fa568000c00 tx=0x7fa568004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b345d9a11091109e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:17:46.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:45.994+0000 7fa55dffb640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:17:46.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:45.994+0000 7fa55dffb640 1 -- 172.21.15.26:0/3739489157 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa56804b020 con 0x55fcd2e9b2f0 2024-09-11T23:17:46.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:45.994+0000 7fa57ef9a740 1 -- 172.21.15.26:0/3739489157 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa568031870 con 0x55fcd2e9b2f0 2024-09-11T23:17:46.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:45.994+0000 7fa55dffb640 1 -- 172.21.15.26:0/3739489157 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fa568004740 con 0x55fcd2e9b2f0 2024-09-11T23:17:46.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:45.995+0000 7fa55dffb640 1 -- 172.21.15.26:0/3739489157 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa568035070 con 0x55fcd2e9b2f0 2024-09-11T23:17:46.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:45.995+0000 7fa57ef9a740 10 client.14958 fetch_fsmap learned FSMap version 28 2024-09-11T23:17:46.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:45.995+0000 7fa57ef9a740 1 -- 172.21.15.26:0/3739489157 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55fcd2eb40f0 con 0x55fcd2e9b2f0 2024-09-11T23:17:46.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:45.995+0000 7fa55dffb640 1 -- 172.21.15.26:0/3739489157 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa568031870 con 0x55fcd2e9b2f0 2024-09-11T23:17:46.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:45.995+0000 7fa55dffb640 1 -- 172.21.15.26:0/3739489157 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fa568003950 con 0x55fcd2e9b2f0 2024-09-11T23:17:46.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:45.995+0000 7fa57ef9a740 10 client.14958 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:17:46.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:45.995+0000 7fa57ef9a740 10 client.14958 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-11T23:17:46.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:45.995+0000 7fa57ef9a740 20 client.14958 populate_metadata read hostname 'smithi026' 2024-09-11T23:17:46.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:45.995+0000 7fa57ef9a740 1 --2- 172.21.15.26:0/3739489157 >> [v2:172.21.15.76:6836/2078978748,v1:172.21.15.76:6837/2078978748] conn(0x55fcd2eb6290 0x55fcd2eb7700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:46.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:45.995+0000 7fa57ef9a740 4 client.14958 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:17:46.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:45.995+0000 7fa57ef9a740 1 -- 172.21.15.26:0/3739489157 --> [v2:172.21.15.76:6836/2078978748,v1:172.21.15.76:6837/2078978748] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fcd2e0e600 con 0x55fcd2eb6290 2024-09-11T23:17:46.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:45.995+0000 7fa575d9b640 1 --2- 172.21.15.26:0/3739489157 >> [v2:172.21.15.76:6836/2078978748,v1:172.21.15.76:6837/2078978748] conn(0x55fcd2eb6290 0x55fcd2eb7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:46.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:45.996+0000 7fa575d9b640 1 --2- 172.21.15.26:0/3739489157 >> [v2:172.21.15.76:6836/2078978748,v1:172.21.15.76:6837/2078978748] conn(0x55fcd2eb6290 0x55fcd2eb7700 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d5a5841df476f64f server_cookie=e00a3cef0027a208 in_seq=0 out_seq=0 2024-09-11T23:17:46.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:45.996+0000 7fa55dffb640 10 client.14958 ms_handle_connect on v2:172.21.15.76:6836/2078978748 2024-09-11T23:17:46.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:45.996+0000 7fa55dffb640 1 -- 172.21.15.26:0/3739489157 <== mds.2 v2:172.21.15.76:6836/2078978748 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fcd2e0e600 con 0x55fcd2eb6290 2024-09-11T23:17:46.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:45.996+0000 7fa55dffb640 10 client.14958 handle_command_reply: tid=0 2024-09-11T23:17:46.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:45.996+0000 7fa57ef9a740 1 client.14958 shutdown 2024-09-11T23:17:46.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:45.996+0000 7fa57ef9a740 1 -- 172.21.15.26:0/3739489157 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fcd2e9b2f0 msgr2=0x55fcd2ea27e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:46.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:45.996+0000 7fa57ef9a740 1 --2- 172.21.15.26:0/3739489157 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fcd2e9b2f0 0x55fcd2ea27e0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fa568000c00 tx=0x7fa568004270 comp rx=0 tx=0).stop 2024-09-11T23:17:46.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:45.996+0000 7fa57ef9a740 1 -- 172.21.15.26:0/3739489157 shutdown_connections 2024-09-11T23:17:46.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:45.996+0000 7fa57ef9a740 1 --2- 172.21.15.26:0/3739489157 >> [v2:172.21.15.76:6836/2078978748,v1:172.21.15.76:6837/2078978748] conn(0x55fcd2eb6290 0x55fcd2eb7700 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:46.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:45.996+0000 7fa57ef9a740 1 --2- 172.21.15.26:0/3739489157 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fcd2ea5bb0 0x55fcd2ea2d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:46.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:45.996+0000 7fa57ef9a740 1 --2- 172.21.15.26:0/3739489157 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fcd2e9b2f0 0x55fcd2ea27e0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:46.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:45.996+0000 7fa55dffb640 0 client.14958 ms_handle_reset on v2:172.21.15.76:6836/2078978748 2024-09-11T23:17:46.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:45.996+0000 7fa57ef9a740 1 --2- 172.21.15.26:0/3739489157 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fcd2e9a9d0 0x55fcd2ea5490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:46.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:45.996+0000 7fa57ef9a740 1 -- 172.21.15.26:0/3739489157 >> 172.21.15.26:0/3739489157 conn(0x55fcd2e96f60 msgr2=0x55fcd2e9cc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:17:46.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:45.997+0000 7fa57ef9a740 1 -- 172.21.15.26:0/3739489157 shutdown_connections 2024-09-11T23:17:46.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:45.997+0000 7fa57ef9a740 1 -- 172.21.15.26:0/3739489157 wait complete. 2024-09-11T23:17:46.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:45.997+0000 7fa57ef9a740 20 client.14958 trim_cache size 0 max 16384 2024-09-11T23:17:46.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:46.015+0000 7fa57ef9a740 1 Processor -- start 2024-09-11T23:17:46.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:46.015+0000 7fa57ef9a740 1 -- start start 2024-09-11T23:17:46.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:46.015+0000 7fa57ef9a740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fcd2e9a9d0 0x55fcd2e9fa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:46.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:46.015+0000 7fa57ef9a740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fcd2e9b2f0 0x55fcd2e9ff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:46.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:46.015+0000 7fa57ef9a740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fcd2ea5bb0 0x55fcd2e955c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:46.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:46.015+0000 7fa57ef9a740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55fcd2eb5a40 con 0x55fcd2e9a9d0 2024-09-11T23:17:46.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:46.015+0000 7fa57ef9a740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55fcd2eb6110 con 0x55fcd2e9b2f0 2024-09-11T23:17:46.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:46.015+0000 7fa57ef9a740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55fcd2eb67e0 con 0x55fcd2ea5bb0 2024-09-11T23:17:46.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:46.015+0000 7fa575d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fcd2e9a9d0 0x55fcd2e9fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:46.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:46.015+0000 7fa575d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fcd2e9a9d0 0x55fcd2e9fa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59794/0 (socket says 172.21.15.26:59794) 2024-09-11T23:17:46.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:46.015+0000 7fa575d9b640 1 -- 172.21.15.26:0/2786138637 learned_addr learned my addr 172.21.15.26:0/2786138637 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:17:46.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:46.015+0000 7fa57659c640 1 --2- 172.21.15.26:0/2786138637 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fcd2e9b2f0 0x55fcd2e9ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:46.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:46.015+0000 7fa575d9b640 1 --2- 172.21.15.26:0/2786138637 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fcd2ea5bb0 0x55fcd2e955c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:46.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:46.016+0000 7fa575d9b640 1 -- 172.21.15.26:0/2786138637 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fcd2ea5bb0 msgr2=0x55fcd2e955c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:17:46.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:46.016+0000 7fa575d9b640 1 --2- 172.21.15.26:0/2786138637 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fcd2ea5bb0 0x55fcd2e955c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:46.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:46.016+0000 7fa575d9b640 1 -- 172.21.15.26:0/2786138637 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fcd2e9b2f0 msgr2=0x55fcd2e9ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:17:46.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:46.016+0000 7fa575d9b640 1 --2- 172.21.15.26:0/2786138637 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fcd2e9b2f0 0x55fcd2e9ff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:46.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:46.016+0000 7fa575d9b640 1 -- 172.21.15.26:0/2786138637 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fcd2e97aa0 con 0x55fcd2e9a9d0 2024-09-11T23:17:46.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:46.016+0000 7fa57659c640 1 --2- 172.21.15.26:0/2786138637 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fcd2e9b2f0 0x55fcd2e9ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:17:46.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:46.016+0000 7fa575d9b640 1 --2- 172.21.15.26:0/2786138637 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fcd2e9a9d0 0x55fcd2e9fa40 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fa5780fdce0 tx=0x7fa56c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=776eaf2a29a3248a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:17:46.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:46.016+0000 7fa55ffff640 1 -- 172.21.15.26:0/2786138637 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa56c025040 con 0x55fcd2e9a9d0 2024-09-11T23:17:46.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:46.016+0000 7fa55ffff640 1 -- 172.21.15.26:0/2786138637 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa56c010880 con 0x55fcd2e9a9d0 2024-09-11T23:17:46.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:46.016+0000 7fa55ffff640 1 -- 172.21.15.26:0/2786138637 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa56c005310 con 0x55fcd2e9a9d0 2024-09-11T23:17:46.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:46.017+0000 7fa57ef9a740 1 -- 172.21.15.26:0/2786138637 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fcd2e9a9d0 msgr2=0x55fcd2e9fa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:46.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:46.017+0000 7fa57ef9a740 1 --2- 172.21.15.26:0/2786138637 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fcd2e9a9d0 0x55fcd2e9fa40 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fa5780fdce0 tx=0x7fa56c004730 comp rx=0 tx=0).stop 2024-09-11T23:17:46.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:46.017+0000 7fa57ef9a740 1 -- 172.21.15.26:0/2786138637 shutdown_connections 2024-09-11T23:17:46.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:46.017+0000 7fa57ef9a740 1 --2- 172.21.15.26:0/2786138637 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fcd2e9b2f0 0x55fcd2e9ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:46.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:46.017+0000 7fa57ef9a740 1 --2- 172.21.15.26:0/2786138637 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fcd2ea5bb0 0x55fcd2e955c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:46.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:46.017+0000 7fa57ef9a740 1 --2- 172.21.15.26:0/2786138637 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fcd2e9a9d0 0x55fcd2e9fa40 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:46.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:46.017+0000 7fa57ef9a740 1 -- 172.21.15.26:0/2786138637 >> 172.21.15.26:0/2786138637 conn(0x55fcd2e96f60 msgr2=0x55fcd2ea4750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:17:46.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:46.017+0000 7fa57ef9a740 1 -- 172.21.15.26:0/2786138637 shutdown_connections 2024-09-11T23:17:46.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:46.017+0000 7fa57ef9a740 1 -- 172.21.15.26:0/2786138637 wait complete. 2024-09-11T23:17:46.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:46.017+0000 7fa57ef9a740 1 -- start start 2024-09-11T23:17:46.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:46.017+0000 7fa57ef9a740 1 Processor -- start 2024-09-11T23:17:46.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:46.017+0000 7fa57ef9a740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fcd2e9a9d0 0x55fcd2ebb8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:46.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:46.017+0000 7fa57ef9a740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fcd2e9b2f0 0x55fcd2ebbe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:46.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:46.017+0000 7fa57ef9a740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fcd2ea5bb0 0x55fcd2ebc350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:46.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:46.017+0000 7fa57ef9a740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55fcd2eb7580 con 0x55fcd2ea5bb0 2024-09-11T23:17:46.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:46.017+0000 7fa57ef9a740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55fcd2eb6eb0 con 0x55fcd2e9a9d0 2024-09-11T23:17:46.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:46.017+0000 7fa57ef9a740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55fcd2eb7c50 con 0x55fcd2e9b2f0 2024-09-11T23:17:46.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:46.017+0000 7fa57ef9a740 10 client.0 _ll_register_callbacks cb 0x7fa56c00b860 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:17:46.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:46.017+0000 7fa575d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fcd2ea5bb0 0x55fcd2ebc350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:46.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:46.018+0000 7fa575d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fcd2ea5bb0 0x55fcd2ebc350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59808/0 (socket says 172.21.15.26:59808) 2024-09-11T23:17:46.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:46.018+0000 7fa575d9b640 1 -- 172.21.15.26:0/2627660479 learned_addr learned my addr 172.21.15.26:0/2627660479 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:17:46.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:46.018+0000 7fa57659c640 1 --2- 172.21.15.26:0/2627660479 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fcd2e9b2f0 0x55fcd2ebbe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:46.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:46.018+0000 7fa575d9b640 1 --2- 172.21.15.26:0/2627660479 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fcd2e9a9d0 0x55fcd2ebb8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:46.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:46.018+0000 7fa575d9b640 1 -- 172.21.15.26:0/2627660479 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fcd2e9b2f0 msgr2=0x55fcd2ebbe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:17:46.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:46.018+0000 7fa575d9b640 1 --2- 172.21.15.26:0/2627660479 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fcd2e9b2f0 0x55fcd2ebbe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:46.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:46.018+0000 7fa575d9b640 1 -- 172.21.15.26:0/2627660479 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fcd2e9a9d0 msgr2=0x55fcd2ebb8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:17:46.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:46.018+0000 7fa575d9b640 1 --2- 172.21.15.26:0/2627660479 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fcd2e9a9d0 0x55fcd2ebb8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:46.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:46.018+0000 7fa575d9b640 1 -- 172.21.15.26:0/2627660479 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fcd2e9a800 con 0x55fcd2ea5bb0 2024-09-11T23:17:46.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:46.018+0000 7fa57659c640 1 --2- 172.21.15.26:0/2627660479 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fcd2e9b2f0 0x55fcd2ebbe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:17:46.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:46.018+0000 7fa575d9b640 1 --2- 172.21.15.26:0/2627660479 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fcd2ea5bb0 0x55fcd2ebc350 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fa56c00b830 tx=0x7fa56c00b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af961b3491523a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:17:46.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:46.018+0000 7fa55dffb640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:17:46.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:46.018+0000 7fa55dffb640 1 -- 172.21.15.26:0/2627660479 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa56c02f070 con 0x55fcd2ea5bb0 2024-09-11T23:17:46.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:46.018+0000 7fa55dffb640 1 -- 172.21.15.26:0/2627660479 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fa56c010a30 con 0x55fcd2ea5bb0 2024-09-11T23:17:46.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:46.018+0000 7fa57ef9a740 1 -- 172.21.15.26:0/2627660479 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa56c010880 con 0x55fcd2ea5bb0 2024-09-11T23:17:46.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:46.018+0000 7fa55dffb640 1 -- 172.21.15.26:0/2627660479 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa56c035040 con 0x55fcd2ea5bb0 2024-09-11T23:17:46.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:46.018+0000 7fa57ef9a740 10 client.14964 fetch_fsmap learned FSMap version 28 2024-09-11T23:17:46.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:46.018+0000 7fa57ef9a740 1 -- 172.21.15.26:0/2627660479 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55fcd2f20c50 con 0x55fcd2ea5bb0 2024-09-11T23:17:46.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:46.019+0000 7fa55dffb640 1 -- 172.21.15.26:0/2627660479 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa56c010880 con 0x55fcd2ea5bb0 2024-09-11T23:17:46.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:46.019+0000 7fa55dffb640 1 -- 172.21.15.26:0/2627660479 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fa56c003bf0 con 0x55fcd2ea5bb0 2024-09-11T23:17:46.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:46.019+0000 7fa57ef9a740 10 client.14964 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:17:46.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:46.019+0000 7fa57ef9a740 10 client.14964 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-11T23:17:46.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:46.019+0000 7fa57ef9a740 20 client.14964 populate_metadata read hostname 'smithi026' 2024-09-11T23:17:46.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:46.019+0000 7fa57ef9a740 1 --2- 172.21.15.26:0/2627660479 >> [v2:172.21.15.76:6836/2078978748,v1:172.21.15.76:6837/2078978748] conn(0x55fcd2f25320 0x55fcd2f24030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:46.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:46.019+0000 7fa57ef9a740 4 client.14964 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-11T23:17:46.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:46.019+0000 7fa57ef9a740 1 -- 172.21.15.26:0/2627660479 --> [v2:172.21.15.76:6836/2078978748,v1:172.21.15.76:6837/2078978748] -- command(tid 0: {"prefix": "status"}) -- 0x55fcd2e08b10 con 0x55fcd2f25320 2024-09-11T23:17:46.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:46.019+0000 7fa57659c640 1 --2- 172.21.15.26:0/2627660479 >> [v2:172.21.15.76:6836/2078978748,v1:172.21.15.76:6837/2078978748] conn(0x55fcd2f25320 0x55fcd2f24030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:46.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:46.019+0000 7fa57659c640 1 --2- 172.21.15.26:0/2627660479 >> [v2:172.21.15.76:6836/2078978748,v1:172.21.15.76:6837/2078978748] conn(0x55fcd2f25320 0x55fcd2f24030 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=fda3c82fa0cc442f server_cookie=75b29c226b888a72 in_seq=0 out_seq=0 2024-09-11T23:17:46.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:46.019+0000 7fa55dffb640 10 client.14964 ms_handle_connect on v2:172.21.15.76:6836/2078978748 2024-09-11T23:17:46.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:46.020+0000 7fa55dffb640 1 -- 172.21.15.26:0/2627660479 <== mds.2 v2:172.21.15.76:6836/2078978748 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55fcd2e08b10 con 0x55fcd2f25320 2024-09-11T23:17:46.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:46.020+0000 7fa55dffb640 10 client.14964 handle_command_reply: tid=0 2024-09-11T23:17:46.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:46.020+0000 7fa57ef9a740 1 client.14964 shutdown 2024-09-11T23:17:46.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:46.020+0000 7fa57ef9a740 1 -- 172.21.15.26:0/2627660479 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fcd2ea5bb0 msgr2=0x55fcd2ebc350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:46.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:46.020+0000 7fa57ef9a740 1 --2- 172.21.15.26:0/2627660479 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fcd2ea5bb0 0x55fcd2ebc350 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fa56c00b830 tx=0x7fa56c00b990 comp rx=0 tx=0).stop 2024-09-11T23:17:46.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:46.020+0000 7fa57ef9a740 1 -- 172.21.15.26:0/2627660479 shutdown_connections 2024-09-11T23:17:46.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:46.020+0000 7fa57ef9a740 1 --2- 172.21.15.26:0/2627660479 >> [v2:172.21.15.76:6836/2078978748,v1:172.21.15.76:6837/2078978748] conn(0x55fcd2f25320 0x55fcd2f24030 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:46.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:46.020+0000 7fa57ef9a740 1 --2- 172.21.15.26:0/2627660479 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fcd2ea5bb0 0x55fcd2ebc350 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:46.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:46.020+0000 7fa57ef9a740 1 --2- 172.21.15.26:0/2627660479 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fcd2e9b2f0 0x55fcd2ebbe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:46.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:46.020+0000 7fa57ef9a740 1 --2- 172.21.15.26:0/2627660479 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fcd2e9a9d0 0x55fcd2ebb8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:46.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:46.020+0000 7fa57ef9a740 1 -- 172.21.15.26:0/2627660479 >> 172.21.15.26:0/2627660479 conn(0x55fcd2e96f60 msgr2=0x55fcd2ea20a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:17:46.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:46.020+0000 7fa55dffb640 0 client.14964 ms_handle_reset on v2:172.21.15.76:6836/2078978748 2024-09-11T23:17:46.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:46.020+0000 7fa57ef9a740 1 -- 172.21.15.26:0/2627660479 shutdown_connections 2024-09-11T23:17:46.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:46.020+0000 7fa57ef9a740 1 -- 172.21.15.26:0/2627660479 wait complete. 2024-09-11T23:17:46.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:46.020+0000 7fa57ef9a740 20 client.14964 trim_cache size 0 max 16384 2024-09-11T23:17:46.028 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-11T23:17:46.028 INFO:teuthology.orchestra.run.smithi026.stdout: "cluster_fsid": "af7e6fc8-7092-11ef-bcea-c7b262605968", 2024-09-11T23:17:46.028 INFO:teuthology.orchestra.run.smithi026.stdout: "whoami": 2, 2024-09-11T23:17:46.028 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 24458, 2024-09-11T23:17:46.028 INFO:teuthology.orchestra.run.smithi026.stdout: "want_state": "up:active", 2024-09-11T23:17:46.028 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "up:active", 2024-09-11T23:17:46.028 INFO:teuthology.orchestra.run.smithi026.stdout: "fs_name": "cephfs", 2024-09-11T23:17:46.028 INFO:teuthology.orchestra.run.smithi026.stdout: "rank_uptime": 71.074982320000004, 2024-09-11T23:17:46.028 INFO:teuthology.orchestra.run.smithi026.stdout: "mdsmap_epoch": 28, 2024-09-11T23:17:46.028 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch": 85, 2024-09-11T23:17:46.029 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch_barrier": 85, 2024-09-11T23:17:46.029 INFO:teuthology.orchestra.run.smithi026.stdout: "uptime": 102.33029340500001 2024-09-11T23:17:46.029 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-11T23:17:46.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:46.020+0000 7fa57cbc1640 1 -- 172.21.15.26:0/2711699198 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fa5600a2c10 msgr2=0x7fa5600a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:46.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:46.020+0000 7fa57cbc1640 1 --2- 172.21.15.26:0/2711699198 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fa5600a2c10 0x7fa5600a50d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa570065e30 tx=0x7fa57009e040 comp rx=0 tx=0).stop 2024-09-11T23:17:46.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:46.020+0000 7fa57cbc1640 1 -- 172.21.15.26:0/2711699198 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa578106f10 msgr2=0x7fa57819cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:46.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:46.020+0000 7fa57cbc1640 1 --2- 172.21.15.26:0/2711699198 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa578106f10 0x7fa57819cd80 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fa57000be20 tx=0x7fa57000bf10 comp rx=0 tx=0).stop 2024-09-11T23:17:46.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:46.020+0000 7fa57cbc1640 1 -- 172.21.15.26:0/2711699198 shutdown_connections 2024-09-11T23:17:46.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:46.020+0000 7fa57cbc1640 1 --2- 172.21.15.26:0/2711699198 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fa5600a2c10 0x7fa5600a50d0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:46.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:46.020+0000 7fa57cbc1640 1 --2- 172.21.15.26:0/2711699198 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa578106f10 0x7fa57819cd80 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:46.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:46.020+0000 7fa57cbc1640 1 --2- 172.21.15.26:0/2711699198 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa578102790 0x7fa5781989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:46.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:46.020+0000 7fa57cbc1640 1 --2- 172.21.15.26:0/2711699198 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa578101de0 0x7fa5781984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:46.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:46.020+0000 7fa57cbc1640 1 -- 172.21.15.26:0/2711699198 >> 172.21.15.26:0/2711699198 conn(0x7fa5780fd8c0 msgr2=0x7fa5780fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:17:46.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:46.021+0000 7fa57cbc1640 1 -- 172.21.15.26:0/2711699198 shutdown_connections 2024-09-11T23:17:46.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:46.021+0000 7fa57cbc1640 1 -- 172.21.15.26:0/2711699198 wait complete. 2024-09-11T23:17:46.053 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:17:45 smithi136 ceph-mon[35121]: pgmap v320: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-11T23:17:46.135 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:17:45 smithi076 ceph-mon[35296]: pgmap v320: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-11T23:17:46.711 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph tell mds.i status 2024-09-11T23:17:46.971 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:17:47.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.408+0000 7ff09d132640 1 -- 172.21.15.26:0/3828435518 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff098100a80 msgr2=0x7ff098100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:47.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.408+0000 7ff09d132640 1 --2- 172.21.15.26:0/3828435518 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff098100a80 0x7ff098100ee0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7ff08c009930 tx=0x7ff08c02a420 comp rx=0 tx=0).stop 2024-09-11T23:17:47.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.408+0000 7ff09d132640 1 -- 172.21.15.26:0/3828435518 shutdown_connections 2024-09-11T23:17:47.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.408+0000 7ff09d132640 1 --2- 172.21.15.26:0/3828435518 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff098101420 0x7ff09810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:47.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.408+0000 7ff09d132640 1 --2- 172.21.15.26:0/3828435518 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff098100a80 0x7ff098100ee0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:47.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.408+0000 7ff09d132640 1 --2- 172.21.15.26:0/3828435518 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff098106a80 0x7ff098106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:47.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.408+0000 7ff09d132640 1 -- 172.21.15.26:0/3828435518 >> 172.21.15.26:0/3828435518 conn(0x7ff0980fc7a0 msgr2=0x7ff0980febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:17:47.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.408+0000 7ff09d132640 1 -- 172.21.15.26:0/3828435518 shutdown_connections 2024-09-11T23:17:47.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.408+0000 7ff09d132640 1 -- 172.21.15.26:0/3828435518 wait complete. 2024-09-11T23:17:47.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.409+0000 7ff09d132640 1 Processor -- start 2024-09-11T23:17:47.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.409+0000 7ff09d132640 1 -- start start 2024-09-11T23:17:47.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.409+0000 7ff09d132640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff098100a80 0x7ff0980fce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:47.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.409+0000 7ff09d132640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff098101420 0x7ff0980fd3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:47.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.409+0000 7ff09d132640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff098106a80 0x7ff0980fd910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:47.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.409+0000 7ff09d132640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff098110e80 con 0x7ff098101420 2024-09-11T23:17:47.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.409+0000 7ff09d132640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7ff098110d00 con 0x7ff098100a80 2024-09-11T23:17:47.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.409+0000 7ff09d132640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ff098111000 con 0x7ff098106a80 2024-09-11T23:17:47.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.409+0000 7ff09659c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff098101420 0x7ff0980fd3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:47.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.409+0000 7ff09659c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff098101420 0x7ff0980fd3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59834/0 (socket says 172.21.15.26:59834) 2024-09-11T23:17:47.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.409+0000 7ff09659c640 1 -- 172.21.15.26:0/1700939496 learned_addr learned my addr 172.21.15.26:0/1700939496 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:17:47.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.409+0000 7ff09759e640 1 --2- 172.21.15.26:0/1700939496 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff098106a80 0x7ff0980fd910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:47.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.409+0000 7ff096d9d640 1 --2- 172.21.15.26:0/1700939496 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff098100a80 0x7ff0980fce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:47.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.409+0000 7ff09659c640 1 -- 172.21.15.26:0/1700939496 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff098106a80 msgr2=0x7ff0980fd910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:47.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.409+0000 7ff09659c640 1 --2- 172.21.15.26:0/1700939496 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff098106a80 0x7ff0980fd910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:47.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.409+0000 7ff09659c640 1 -- 172.21.15.26:0/1700939496 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff098100a80 msgr2=0x7ff0980fce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:47.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.409+0000 7ff09659c640 1 --2- 172.21.15.26:0/1700939496 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff098100a80 0x7ff0980fce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:47.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.410+0000 7ff09659c640 1 -- 172.21.15.26:0/1700939496 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0981a6930 con 0x7ff098101420 2024-09-11T23:17:47.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.410+0000 7ff09659c640 1 --2- 172.21.15.26:0/1700939496 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff098101420 0x7ff0980fd3d0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7ff08c009230 tx=0x7ff08c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:17:47.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.410+0000 7ff07ffff640 1 -- 172.21.15.26:0/1700939496 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff08c03f030 con 0x7ff098101420 2024-09-11T23:17:47.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.410+0000 7ff09d132640 1 -- 172.21.15.26:0/1700939496 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0981a6bc0 con 0x7ff098101420 2024-09-11T23:17:47.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.410+0000 7ff07ffff640 1 -- 172.21.15.26:0/1700939496 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff08c003ad0 con 0x7ff098101420 2024-09-11T23:17:47.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.410+0000 7ff07ffff640 1 -- 172.21.15.26:0/1700939496 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff08c035070 con 0x7ff098101420 2024-09-11T23:17:47.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.410+0000 7ff09d132640 1 -- 172.21.15.26:0/1700939496 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0981a70f0 con 0x7ff098101420 2024-09-11T23:17:47.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.412+0000 7ff07ffff640 1 -- 172.21.15.26:0/1700939496 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7ff08c042030 con 0x7ff098101420 2024-09-11T23:17:47.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.413+0000 7ff09f50b740 1 Processor -- start 2024-09-11T23:17:47.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.413+0000 7ff09f50b740 1 -- start start 2024-09-11T23:17:47.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.413+0000 7ff07ffff640 1 --2- 172.21.15.26:0/1700939496 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7ff0800a2c10 0x7ff0800a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:47.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.413+0000 7ff09f50b740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564e254e79d0 0x564e254e7db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:47.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.413+0000 7ff09f50b740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564e254e82f0 0x564e254f2670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:47.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.413+0000 7ff09f50b740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564e254f2bb0 0x564e254f4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:47.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.413+0000 7ff09f50b740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x564e2544e090 con 0x564e254f2bb0 2024-09-11T23:17:47.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.413+0000 7ff09f50b740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x564e253f5a70 con 0x564e254e82f0 2024-09-11T23:17:47.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.413+0000 7ff09f50b740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x564e2544a050 con 0x564e254e79d0 2024-09-11T23:17:47.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.413+0000 7ff07ffff640 1 -- 172.21.15.26:0/1700939496 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7ff08c0ef660 con 0x7ff098101420 2024-09-11T23:17:47.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.413+0000 7ff096d9d640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564e254e79d0 0x564e254e7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:47.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.413+0000 7ff09659c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564e254e82f0 0x564e254f2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:47.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.413+0000 7ff096d9d640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564e254e79d0 0x564e254e7db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:50160/0 (socket says 172.21.15.26:50160) 2024-09-11T23:17:47.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.413+0000 7ff096d9d640 1 -- 172.21.15.26:0/1424696892 learned_addr learned my addr 172.21.15.26:0/1424696892 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:17:47.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.413+0000 7ff096d9d640 1 -- 172.21.15.26:0/1424696892 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564e254e82f0 msgr2=0x564e254f2670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:17:47.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.413+0000 7ff096d9d640 1 --2- 172.21.15.26:0/1424696892 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564e254e82f0 0x564e254f2670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:47.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.413+0000 7ff096d9d640 1 -- 172.21.15.26:0/1424696892 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564e254f2bb0 msgr2=0x564e254f4fa0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T23:17:47.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.413+0000 7ff096d9d640 1 --2- 172.21.15.26:0/1424696892 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564e254f2bb0 0x564e254f4fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:47.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.413+0000 7ff096d9d640 1 -- 172.21.15.26:0/1424696892 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e25402130 con 0x564e254e79d0 2024-09-11T23:17:47.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.413+0000 7ff09759e640 1 --2- 172.21.15.26:0/1700939496 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7ff0800a2c10 0x7ff0800a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:47.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.414+0000 7ff096d9d640 1 --2- 172.21.15.26:0/1424696892 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564e254e79d0 0x564e254e7db0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff08800bc90 tx=0x7ff0880107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ea881ec8f31d83cf server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:17:47.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.414+0000 7ff07d7fa640 1 -- 172.21.15.26:0/1424696892 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff08801c480 con 0x564e254e79d0 2024-09-11T23:17:47.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.414+0000 7ff07d7fa640 1 -- 172.21.15.26:0/1424696892 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff08801cb00 con 0x564e254e79d0 2024-09-11T23:17:47.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.414+0000 7ff07d7fa640 1 -- 172.21.15.26:0/1424696892 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff088011bb0 con 0x564e254e79d0 2024-09-11T23:17:47.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.414+0000 7ff09f50b740 1 -- 172.21.15.26:0/1424696892 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564e254e79d0 msgr2=0x564e254e7db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:47.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.415+0000 7ff09f50b740 1 --2- 172.21.15.26:0/1424696892 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564e254e79d0 0x564e254e7db0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff08800bc90 tx=0x7ff0880107a0 comp rx=0 tx=0).stop 2024-09-11T23:17:47.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.414+0000 7ff09759e640 1 --2- 172.21.15.26:0/1700939496 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7ff0800a2c10 0x7ff0800a50d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x564e254e52e0 tx=0x7ff090009600 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:17:47.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.415+0000 7ff09f50b740 1 -- 172.21.15.26:0/1424696892 shutdown_connections 2024-09-11T23:17:47.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.415+0000 7ff09f50b740 1 --2- 172.21.15.26:0/1424696892 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564e254e82f0 0x564e254f2670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:47.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.415+0000 7ff09f50b740 1 --2- 172.21.15.26:0/1424696892 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564e254f2bb0 0x564e254f4fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:47.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.415+0000 7ff09f50b740 1 --2- 172.21.15.26:0/1424696892 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564e254e79d0 0x564e254e7db0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:47.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.415+0000 7ff09f50b740 1 -- 172.21.15.26:0/1424696892 >> 172.21.15.26:0/1424696892 conn(0x564e254e3f60 msgr2=0x564e254e4340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:17:47.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.415+0000 7ff09f50b740 1 -- 172.21.15.26:0/1424696892 shutdown_connections 2024-09-11T23:17:47.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.415+0000 7ff09f50b740 1 -- 172.21.15.26:0/1424696892 wait complete. 2024-09-11T23:17:47.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.415+0000 7ff09f50b740 1 -- start start 2024-09-11T23:17:47.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.415+0000 7ff09f50b740 1 Processor -- start 2024-09-11T23:17:47.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.415+0000 7ff09f50b740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564e254e79d0 0x564e254f2440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:47.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.415+0000 7ff09f50b740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564e254e82f0 0x564e254ef790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:47.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.415+0000 7ff09f50b740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564e254f2bb0 0x564e254efcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:47.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.415+0000 7ff09f50b740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x564e254f5780 con 0x564e254e82f0 2024-09-11T23:17:47.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.415+0000 7ff09f50b740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x564e25408f20 con 0x564e254e79d0 2024-09-11T23:17:47.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.415+0000 7ff09f50b740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x564e254f5900 con 0x564e254f2bb0 2024-09-11T23:17:47.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.415+0000 7ff09f50b740 10 client.0 _ll_register_callbacks cb 0x564e253d2030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:17:47.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.415+0000 7ff09659c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564e254e82f0 0x564e254ef790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:47.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.415+0000 7ff096d9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564e254e79d0 0x564e254f2440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:47.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.415+0000 7ff09659c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564e254e82f0 0x564e254ef790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59848/0 (socket says 172.21.15.26:59848) 2024-09-11T23:17:47.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.415+0000 7ff09659c640 1 -- 172.21.15.26:0/3433207485 learned_addr learned my addr 172.21.15.26:0/3433207485 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:17:47.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.415+0000 7ff09759e640 1 --2- 172.21.15.26:0/3433207485 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564e254f2bb0 0x564e254efcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:47.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.416+0000 7ff096d9d640 1 -- 172.21.15.26:0/3433207485 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564e254f2bb0 msgr2=0x564e254efcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:17:47.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.416+0000 7ff096d9d640 1 --2- 172.21.15.26:0/3433207485 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564e254f2bb0 0x564e254efcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:47.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.416+0000 7ff096d9d640 1 -- 172.21.15.26:0/3433207485 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564e254e82f0 msgr2=0x564e254ef790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:17:47.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.416+0000 7ff096d9d640 1 --2- 172.21.15.26:0/3433207485 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564e254e82f0 0x564e254ef790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:47.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.416+0000 7ff096d9d640 1 -- 172.21.15.26:0/3433207485 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e251bf2b0 con 0x564e254e79d0 2024-09-11T23:17:47.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.416+0000 7ff09659c640 1 --2- 172.21.15.26:0/3433207485 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564e254e82f0 0x564e254ef790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:17:47.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.416+0000 7ff096d9d640 1 --2- 172.21.15.26:0/3433207485 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564e254e79d0 0x564e254f2440 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff088010ca0 tx=0x7ff08801c730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a6c5404c24ee3239 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:17:47.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.416+0000 7ff07aff5640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-11T23:17:47.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.416+0000 7ff07aff5640 1 -- 172.21.15.26:0/3433207485 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff08801c920 con 0x564e254e79d0 2024-09-11T23:17:47.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.416+0000 7ff09f50b740 1 -- 172.21.15.26:0/3433207485 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff08801cb00 con 0x564e254e79d0 2024-09-11T23:17:47.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.416+0000 7ff07aff5640 1 -- 172.21.15.26:0/3433207485 <== mon.2 v2:172.21.15.76:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7ff088011b40 con 0x564e254e79d0 2024-09-11T23:17:47.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.416+0000 7ff07aff5640 1 -- 172.21.15.26:0/3433207485 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff088011d40 con 0x564e254e79d0 2024-09-11T23:17:47.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.416+0000 7ff09f50b740 10 client.24728 fetch_fsmap learned FSMap version 28 2024-09-11T23:17:47.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.416+0000 7ff09f50b740 1 -- 172.21.15.26:0/3433207485 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564e255010f0 con 0x564e254e79d0 2024-09-11T23:17:47.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.417+0000 7ff07aff5640 1 -- 172.21.15.26:0/3433207485 <== mon.2 v2:172.21.15.76:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff08801cb00 con 0x564e254e79d0 2024-09-11T23:17:47.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.417+0000 7ff07aff5640 1 -- 172.21.15.26:0/3433207485 <== mon.2 v2:172.21.15.76:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7ff0880046c0 con 0x564e254e79d0 2024-09-11T23:17:47.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.417+0000 7ff09f50b740 10 client.24728 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:17:47.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.417+0000 7ff09f50b740 10 client.24728 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-11T23:17:47.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.417+0000 7ff09f50b740 20 client.24728 populate_metadata read hostname 'smithi026' 2024-09-11T23:17:47.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.417+0000 7ff09f50b740 1 --2- 172.21.15.26:0/3433207485 >> [v2:172.21.15.136:6838/3211398630,v1:172.21.15.136:6839/3211398630] conn(0x564e255034b0 0x564e25504920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:47.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.417+0000 7ff09f50b740 4 client.24728 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:17:47.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.417+0000 7ff09f50b740 1 -- 172.21.15.26:0/3433207485 --> [v2:172.21.15.136:6838/3211398630,v1:172.21.15.136:6839/3211398630] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564e2545b600 con 0x564e255034b0 2024-09-11T23:17:47.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.417+0000 7ff09659c640 1 --2- 172.21.15.26:0/3433207485 >> [v2:172.21.15.136:6838/3211398630,v1:172.21.15.136:6839/3211398630] conn(0x564e255034b0 0x564e25504920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:47.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.417+0000 7ff09659c640 1 --2- 172.21.15.26:0/3433207485 >> [v2:172.21.15.136:6838/3211398630,v1:172.21.15.136:6839/3211398630] conn(0x564e255034b0 0x564e25504920 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fdca572b671c4b89 server_cookie=c544c63d3755df78 in_seq=0 out_seq=0 2024-09-11T23:17:47.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.417+0000 7ff07aff5640 10 client.24728 ms_handle_connect on v2:172.21.15.136:6838/3211398630 2024-09-11T23:17:47.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.418+0000 7ff07aff5640 1 -- 172.21.15.26:0/3433207485 <== mds.1 v2:172.21.15.136:6838/3211398630 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564e2545b600 con 0x564e255034b0 2024-09-11T23:17:47.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.418+0000 7ff07aff5640 10 client.24728 handle_command_reply: tid=0 2024-09-11T23:17:47.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.418+0000 7ff09f50b740 1 client.24728 shutdown 2024-09-11T23:17:47.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.418+0000 7ff09f50b740 1 -- 172.21.15.26:0/3433207485 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564e254e79d0 msgr2=0x564e254f2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:47.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.418+0000 7ff09f50b740 1 --2- 172.21.15.26:0/3433207485 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564e254e79d0 0x564e254f2440 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff088010ca0 tx=0x7ff08801c730 comp rx=0 tx=0).stop 2024-09-11T23:17:47.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.418+0000 7ff09f50b740 1 -- 172.21.15.26:0/3433207485 shutdown_connections 2024-09-11T23:17:47.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.418+0000 7ff09f50b740 1 --2- 172.21.15.26:0/3433207485 >> [v2:172.21.15.136:6838/3211398630,v1:172.21.15.136:6839/3211398630] conn(0x564e255034b0 0x564e25504920 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:47.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.418+0000 7ff09f50b740 1 --2- 172.21.15.26:0/3433207485 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564e254f2bb0 0x564e254efcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:47.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.418+0000 7ff09f50b740 1 --2- 172.21.15.26:0/3433207485 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564e254e82f0 0x564e254ef790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:47.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.418+0000 7ff07aff5640 0 client.24728 ms_handle_reset on v2:172.21.15.136:6838/3211398630 2024-09-11T23:17:47.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.418+0000 7ff09f50b740 1 --2- 172.21.15.26:0/3433207485 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564e254e79d0 0x564e254f2440 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:47.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.418+0000 7ff09f50b740 1 -- 172.21.15.26:0/3433207485 >> 172.21.15.26:0/3433207485 conn(0x564e254e3f60 msgr2=0x564e254e9c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:17:47.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.418+0000 7ff09f50b740 1 -- 172.21.15.26:0/3433207485 shutdown_connections 2024-09-11T23:17:47.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.418+0000 7ff09f50b740 1 -- 172.21.15.26:0/3433207485 wait complete. 2024-09-11T23:17:47.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.419+0000 7ff09f50b740 20 client.24728 trim_cache size 0 max 16384 2024-09-11T23:17:47.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.439+0000 7ff09f50b740 1 Processor -- start 2024-09-11T23:17:47.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.439+0000 7ff09f50b740 1 -- start start 2024-09-11T23:17:47.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.439+0000 7ff09f50b740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564e254e79d0 0x564e25504920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:47.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.439+0000 7ff09f50b740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564e254e82f0 0x564e254eca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:47.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.439+0000 7ff09f50b740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564e254f2bb0 0x564e254ecf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:47.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.439+0000 7ff09f50b740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x564e25502c60 con 0x564e254e82f0 2024-09-11T23:17:47.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.439+0000 7ff09f50b740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x564e25503330 con 0x564e254f2bb0 2024-09-11T23:17:47.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.439+0000 7ff09f50b740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x564e25503a00 con 0x564e254e79d0 2024-09-11T23:17:47.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.439+0000 7ff09659c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564e254e82f0 0x564e254eca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:47.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.439+0000 7ff09659c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564e254e82f0 0x564e254eca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59852/0 (socket says 172.21.15.26:59852) 2024-09-11T23:17:47.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.439+0000 7ff09659c640 1 -- 172.21.15.26:0/1831856272 learned_addr learned my addr 172.21.15.26:0/1831856272 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:17:47.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.439+0000 7ff096d9d640 1 --2- 172.21.15.26:0/1831856272 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564e254e79d0 0x564e25504920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:47.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.440+0000 7ff09759e640 1 --2- 172.21.15.26:0/1831856272 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564e254f2bb0 0x564e254ecf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:47.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.440+0000 7ff09659c640 1 -- 172.21.15.26:0/1831856272 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564e254e79d0 msgr2=0x564e25504920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:17:47.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.440+0000 7ff09659c640 1 --2- 172.21.15.26:0/1831856272 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564e254e79d0 0x564e25504920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:47.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.440+0000 7ff09659c640 1 -- 172.21.15.26:0/1831856272 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564e254f2bb0 msgr2=0x564e254ecf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:17:47.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.440+0000 7ff09659c640 1 --2- 172.21.15.26:0/1831856272 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564e254f2bb0 0x564e254ecf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:47.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.440+0000 7ff09659c640 1 -- 172.21.15.26:0/1831856272 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e254f02d0 con 0x564e254e82f0 2024-09-11T23:17:47.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.440+0000 7ff09659c640 1 --2- 172.21.15.26:0/1831856272 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564e254e82f0 0x564e254eca10 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7ff08c009a50 tx=0x7ff08c0ef040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=330bb29ebdc9b69f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:17:47.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.440+0000 7ff07cff9640 1 -- 172.21.15.26:0/1831856272 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff08c049040 con 0x564e254e82f0 2024-09-11T23:17:47.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.440+0000 7ff07cff9640 1 -- 172.21.15.26:0/1831856272 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff08c0ba0d0 con 0x564e254e82f0 2024-09-11T23:17:47.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.440+0000 7ff07cff9640 1 -- 172.21.15.26:0/1831856272 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff08c0b8440 con 0x564e254e82f0 2024-09-11T23:17:47.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.441+0000 7ff09f50b740 1 -- 172.21.15.26:0/1831856272 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564e254e82f0 msgr2=0x564e254eca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:47.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.441+0000 7ff09f50b740 1 --2- 172.21.15.26:0/1831856272 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564e254e82f0 0x564e254eca10 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7ff08c009a50 tx=0x7ff08c0ef040 comp rx=0 tx=0).stop 2024-09-11T23:17:47.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.441+0000 7ff09f50b740 1 -- 172.21.15.26:0/1831856272 shutdown_connections 2024-09-11T23:17:47.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.441+0000 7ff09f50b740 1 --2- 172.21.15.26:0/1831856272 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564e254f2bb0 0x564e254ecf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:47.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.441+0000 7ff09f50b740 1 --2- 172.21.15.26:0/1831856272 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564e254e82f0 0x564e254eca10 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:47.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.441+0000 7ff09f50b740 1 --2- 172.21.15.26:0/1831856272 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564e254e79d0 0x564e25504920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:47.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.441+0000 7ff09f50b740 1 -- 172.21.15.26:0/1831856272 >> 172.21.15.26:0/1831856272 conn(0x564e254e3f60 msgr2=0x564e254e3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:17:47.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.441+0000 7ff09f50b740 1 -- 172.21.15.26:0/1831856272 shutdown_connections 2024-09-11T23:17:47.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.441+0000 7ff09f50b740 1 -- 172.21.15.26:0/1831856272 wait complete. 2024-09-11T23:17:47.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.441+0000 7ff09f50b740 1 -- start start 2024-09-11T23:17:47.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.441+0000 7ff09f50b740 1 Processor -- start 2024-09-11T23:17:47.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.441+0000 7ff09f50b740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564e254e79d0 0x564e25508280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:47.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.441+0000 7ff09f50b740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564e254e82f0 0x564e255087c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:47.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.441+0000 7ff09f50b740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564e254f2bb0 0x564e25565bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:47.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.441+0000 7ff09f50b740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x564e255047a0 con 0x564e254e79d0 2024-09-11T23:17:47.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.441+0000 7ff09f50b740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x564e255040d0 con 0x564e254e82f0 2024-09-11T23:17:47.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.441+0000 7ff09f50b740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x564e25504e70 con 0x564e254f2bb0 2024-09-11T23:17:47.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.442+0000 7ff09f50b740 10 client.0 _ll_register_callbacks cb 0x7ff08c0095e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:17:47.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.442+0000 7ff096d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564e254e79d0 0x564e25508280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:47.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.442+0000 7ff096d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564e254e79d0 0x564e25508280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59858/0 (socket says 172.21.15.26:59858) 2024-09-11T23:17:47.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.442+0000 7ff096d9d640 1 -- 172.21.15.26:0/232662207 learned_addr learned my addr 172.21.15.26:0/232662207 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:17:47.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.442+0000 7ff09759e640 1 --2- 172.21.15.26:0/232662207 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564e254f2bb0 0x564e25565bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:47.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.442+0000 7ff09659c640 1 --2- 172.21.15.26:0/232662207 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564e254e82f0 0x564e255087c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:47.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.442+0000 7ff096d9d640 1 -- 172.21.15.26:0/232662207 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564e254f2bb0 msgr2=0x564e25565bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:17:47.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.442+0000 7ff096d9d640 1 --2- 172.21.15.26:0/232662207 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564e254f2bb0 0x564e25565bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:47.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.442+0000 7ff096d9d640 1 -- 172.21.15.26:0/232662207 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564e254e82f0 msgr2=0x564e255087c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:17:47.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.442+0000 7ff096d9d640 1 --2- 172.21.15.26:0/232662207 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564e254e82f0 0x564e255087c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:47.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.442+0000 7ff096d9d640 1 -- 172.21.15.26:0/232662207 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e25566290 con 0x564e254e79d0 2024-09-11T23:17:47.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.442+0000 7ff096d9d640 1 --2- 172.21.15.26:0/232662207 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564e254e79d0 0x564e25508280 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7ff088004d90 tx=0x7ff088004dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=694272d9edfc753b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:17:47.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.442+0000 7ff07aff5640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:17:47.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.442+0000 7ff07aff5640 1 -- 172.21.15.26:0/232662207 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff088011b40 con 0x564e254e79d0 2024-09-11T23:17:47.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.442+0000 7ff07aff5640 1 -- 172.21.15.26:0/232662207 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7ff08800bd80 con 0x564e254e79d0 2024-09-11T23:17:47.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.442+0000 7ff09f50b740 1 -- 172.21.15.26:0/232662207 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff08c0ba0d0 con 0x564e254e79d0 2024-09-11T23:17:47.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.442+0000 7ff07aff5640 1 -- 172.21.15.26:0/232662207 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff08801c7a0 con 0x564e254e79d0 2024-09-11T23:17:47.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.442+0000 7ff09f50b740 10 client.14979 fetch_fsmap learned FSMap version 28 2024-09-11T23:17:47.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.443+0000 7ff09f50b740 1 -- 172.21.15.26:0/232662207 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564e2556d620 con 0x564e254e79d0 2024-09-11T23:17:47.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.443+0000 7ff07aff5640 1 -- 172.21.15.26:0/232662207 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff08c0ba0d0 con 0x564e254e79d0 2024-09-11T23:17:47.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.443+0000 7ff07aff5640 1 -- 172.21.15.26:0/232662207 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7ff08803b020 con 0x564e254e79d0 2024-09-11T23:17:47.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.443+0000 7ff09f50b740 10 client.14979 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:17:47.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.443+0000 7ff09f50b740 10 client.14979 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-11T23:17:47.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.443+0000 7ff09f50b740 20 client.14979 populate_metadata read hostname 'smithi026' 2024-09-11T23:17:47.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.443+0000 7ff09f50b740 1 --2- 172.21.15.26:0/232662207 >> [v2:172.21.15.136:6838/3211398630,v1:172.21.15.136:6839/3211398630] conn(0x564e255721f0 0x564e25570f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:47.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.443+0000 7ff09f50b740 4 client.14979 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-11T23:17:47.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.443+0000 7ff09f50b740 1 -- 172.21.15.26:0/232662207 --> [v2:172.21.15.136:6838/3211398630,v1:172.21.15.136:6839/3211398630] -- command(tid 0: {"prefix": "status"}) -- 0x564e25455b10 con 0x564e255721f0 2024-09-11T23:17:47.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.443+0000 7ff09659c640 1 --2- 172.21.15.26:0/232662207 >> [v2:172.21.15.136:6838/3211398630,v1:172.21.15.136:6839/3211398630] conn(0x564e255721f0 0x564e25570f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:47.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.444+0000 7ff09659c640 1 --2- 172.21.15.26:0/232662207 >> [v2:172.21.15.136:6838/3211398630,v1:172.21.15.136:6839/3211398630] conn(0x564e255721f0 0x564e25570f00 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=522d17dc54401621 server_cookie=7a6103f9051556c0 in_seq=0 out_seq=0 2024-09-11T23:17:47.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.444+0000 7ff07aff5640 10 client.14979 ms_handle_connect on v2:172.21.15.136:6838/3211398630 2024-09-11T23:17:47.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.444+0000 7ff07aff5640 1 -- 172.21.15.26:0/232662207 <== mds.1 v2:172.21.15.136:6838/3211398630 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x564e25455b10 con 0x564e255721f0 2024-09-11T23:17:47.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.444+0000 7ff07aff5640 10 client.14979 handle_command_reply: tid=0 2024-09-11T23:17:47.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.444+0000 7ff09f50b740 1 client.14979 shutdown 2024-09-11T23:17:47.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.444+0000 7ff09f50b740 1 -- 172.21.15.26:0/232662207 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564e254e79d0 msgr2=0x564e25508280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:47.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.444+0000 7ff09f50b740 1 --2- 172.21.15.26:0/232662207 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564e254e79d0 0x564e25508280 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7ff088004d90 tx=0x7ff088004dc0 comp rx=0 tx=0).stop 2024-09-11T23:17:47.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.444+0000 7ff09f50b740 1 -- 172.21.15.26:0/232662207 shutdown_connections 2024-09-11T23:17:47.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.444+0000 7ff09f50b740 1 --2- 172.21.15.26:0/232662207 >> [v2:172.21.15.136:6838/3211398630,v1:172.21.15.136:6839/3211398630] conn(0x564e255721f0 0x564e25570f00 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:47.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.444+0000 7ff09f50b740 1 --2- 172.21.15.26:0/232662207 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564e254e82f0 0x564e255087c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:47.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.444+0000 7ff09f50b740 1 --2- 172.21.15.26:0/232662207 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564e254f2bb0 0x564e25565bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:47.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.444+0000 7ff09f50b740 1 --2- 172.21.15.26:0/232662207 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564e254e79d0 0x564e25508280 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:47.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.444+0000 7ff09f50b740 1 -- 172.21.15.26:0/232662207 >> 172.21.15.26:0/232662207 conn(0x564e254e3f60 msgr2=0x564e254f0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:17:47.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.444+0000 7ff07aff5640 0 client.14979 ms_handle_reset on v2:172.21.15.136:6838/3211398630 2024-09-11T23:17:47.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.444+0000 7ff09f50b740 1 -- 172.21.15.26:0/232662207 shutdown_connections 2024-09-11T23:17:47.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.444+0000 7ff09f50b740 1 -- 172.21.15.26:0/232662207 wait complete. 2024-09-11T23:17:47.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.444+0000 7ff09f50b740 20 client.14979 trim_cache size 0 max 16384 2024-09-11T23:17:47.452 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-11T23:17:47.452 INFO:teuthology.orchestra.run.smithi026.stdout: "cluster_fsid": "af7e6fc8-7092-11ef-bcea-c7b262605968", 2024-09-11T23:17:47.452 INFO:teuthology.orchestra.run.smithi026.stdout: "whoami": 1, 2024-09-11T23:17:47.452 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 24463, 2024-09-11T23:17:47.452 INFO:teuthology.orchestra.run.smithi026.stdout: "want_state": "up:active", 2024-09-11T23:17:47.452 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "up:active", 2024-09-11T23:17:47.452 INFO:teuthology.orchestra.run.smithi026.stdout: "fs_name": "cephfs", 2024-09-11T23:17:47.452 INFO:teuthology.orchestra.run.smithi026.stdout: "rank_uptime": 73.513620791999998, 2024-09-11T23:17:47.452 INFO:teuthology.orchestra.run.smithi026.stdout: "mdsmap_epoch": 28, 2024-09-11T23:17:47.453 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch": 85, 2024-09-11T23:17:47.453 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch_barrier": 85, 2024-09-11T23:17:47.453 INFO:teuthology.orchestra.run.smithi026.stdout: "uptime": 101.22234285499999 2024-09-11T23:17:47.453 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-11T23:17:47.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.445+0000 7ff09d132640 1 -- 172.21.15.26:0/1700939496 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7ff0800a2c10 msgr2=0x7ff0800a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:47.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.445+0000 7ff09d132640 1 --2- 172.21.15.26:0/1700939496 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7ff0800a2c10 0x7ff0800a50d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x564e254e52e0 tx=0x7ff090009600 comp rx=0 tx=0).stop 2024-09-11T23:17:47.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.445+0000 7ff09d132640 1 -- 172.21.15.26:0/1700939496 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff098101420 msgr2=0x7ff0980fd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:47.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.445+0000 7ff09d132640 1 --2- 172.21.15.26:0/1700939496 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff098101420 0x7ff0980fd3d0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7ff08c009230 tx=0x7ff08c004270 comp rx=0 tx=0).stop 2024-09-11T23:17:47.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.445+0000 7ff09d132640 1 -- 172.21.15.26:0/1700939496 shutdown_connections 2024-09-11T23:17:47.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.445+0000 7ff09d132640 1 --2- 172.21.15.26:0/1700939496 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7ff0800a2c10 0x7ff0800a50d0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:47.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.445+0000 7ff09d132640 1 --2- 172.21.15.26:0/1700939496 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff098106a80 0x7ff0980fd910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:47.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.445+0000 7ff09d132640 1 --2- 172.21.15.26:0/1700939496 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff098101420 0x7ff0980fd3d0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:47.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.445+0000 7ff09d132640 1 --2- 172.21.15.26:0/1700939496 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff098100a80 0x7ff0980fce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:47.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.445+0000 7ff09d132640 1 -- 172.21.15.26:0/1700939496 >> 172.21.15.26:0/1700939496 conn(0x7ff0980fc7a0 msgr2=0x7ff09806aad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:17:47.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.445+0000 7ff09d132640 1 -- 172.21.15.26:0/1700939496 shutdown_connections 2024-09-11T23:17:47.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:47.445+0000 7ff09d132640 1 -- 172.21.15.26:0/1700939496 wait complete. 2024-09-11T23:17:48.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:17:47 smithi076 ceph-mon[35296]: pgmap v321: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-11T23:17:48.163 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:17:47 smithi026 ceph-mon[31172]: pgmap v321: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-11T23:17:48.195 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph tell mds.b status 2024-09-11T23:17:48.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:17:47 smithi136 ceph-mon[35121]: pgmap v321: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-11T23:17:48.456 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:17:48.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.941+0000 7feba47f1640 1 -- 172.21.15.26:0/93596891 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb9c102570 msgr2=0x7feb9c1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:48.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.941+0000 7feba47f1640 1 --2- 172.21.15.26:0/93596891 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb9c102570 0x7feb9c1029b0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7feb90009900 tx=0x7feb9002a480 comp rx=0 tx=0).stop 2024-09-11T23:17:48.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.941+0000 7feba47f1640 1 -- 172.21.15.26:0/93596891 shutdown_connections 2024-09-11T23:17:48.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.941+0000 7feba47f1640 1 --2- 172.21.15.26:0/93596891 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7feb9c102ef0 0x7feb9c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:48.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.941+0000 7feba47f1640 1 --2- 172.21.15.26:0/93596891 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb9c102570 0x7feb9c1029b0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:48.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.941+0000 7feba47f1640 1 --2- 172.21.15.26:0/93596891 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7feb9c100e20 0x7feb9c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:48.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.941+0000 7feba47f1640 1 -- 172.21.15.26:0/93596891 >> 172.21.15.26:0/93596891 conn(0x7feb9c0fc7d0 msgr2=0x7feb9c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:17:48.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.941+0000 7feba47f1640 1 -- 172.21.15.26:0/93596891 shutdown_connections 2024-09-11T23:17:48.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.941+0000 7feba47f1640 1 -- 172.21.15.26:0/93596891 wait complete. 2024-09-11T23:17:48.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.942+0000 7feba47f1640 1 Processor -- start 2024-09-11T23:17:48.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.942+0000 7feba47f1640 1 -- start start 2024-09-11T23:17:48.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.942+0000 7feba47f1640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb9c100e20 0x7feb9c1993c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:48.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.942+0000 7feba47f1640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7feb9c102570 0x7feb9c199900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:48.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.942+0000 7feba47f1640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7feb9c102ef0 0x7feb9c1a4440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:48.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.942+0000 7feba47f1640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7feb9c110f50 con 0x7feb9c100e20 2024-09-11T23:17:48.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.942+0000 7feba47f1640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7feb9c110dd0 con 0x7feb9c102ef0 2024-09-11T23:17:48.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.942+0000 7feba47f1640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7feb9c1110d0 con 0x7feb9c102570 2024-09-11T23:17:48.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.942+0000 7feba258d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb9c100e20 0x7feb9c1993c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:48.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.942+0000 7feba258d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb9c100e20 0x7feb9c1993c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59872/0 (socket says 172.21.15.26:59872) 2024-09-11T23:17:48.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.942+0000 7feba258d640 1 -- 172.21.15.26:0/2582309996 learned_addr learned my addr 172.21.15.26:0/2582309996 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:17:48.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.942+0000 7feba1d8c640 1 --2- 172.21.15.26:0/2582309996 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7feb9c102570 0x7feb9c199900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:48.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.942+0000 7feba2d8e640 1 --2- 172.21.15.26:0/2582309996 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7feb9c102ef0 0x7feb9c1a4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:48.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.942+0000 7feba258d640 1 -- 172.21.15.26:0/2582309996 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7feb9c102570 msgr2=0x7feb9c199900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:48.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.942+0000 7feba258d640 1 --2- 172.21.15.26:0/2582309996 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7feb9c102570 0x7feb9c199900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:48.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.942+0000 7feba258d640 1 -- 172.21.15.26:0/2582309996 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7feb9c102ef0 msgr2=0x7feb9c1a4440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:48.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.942+0000 7feba258d640 1 --2- 172.21.15.26:0/2582309996 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7feb9c102ef0 0x7feb9c1a4440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:48.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.942+0000 7feba258d640 1 -- 172.21.15.26:0/2582309996 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb9c1a4980 con 0x7feb9c100e20 2024-09-11T23:17:48.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.942+0000 7feba258d640 1 --2- 172.21.15.26:0/2582309996 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb9c100e20 0x7feb9c1993c0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7feb9800cce0 tx=0x7feb9800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:17:48.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.943+0000 7feb8f7fe640 1 -- 172.21.15.26:0/2582309996 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb98004b20 con 0x7feb9c100e20 2024-09-11T23:17:48.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.943+0000 7feb8f7fe640 1 -- 172.21.15.26:0/2582309996 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb98004cc0 con 0x7feb9c100e20 2024-09-11T23:17:48.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.943+0000 7feb8f7fe640 1 -- 172.21.15.26:0/2582309996 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb9800f7d0 con 0x7feb9c100e20 2024-09-11T23:17:48.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.943+0000 7feba47f1640 1 -- 172.21.15.26:0/2582309996 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb9c1a4b50 con 0x7feb9c100e20 2024-09-11T23:17:48.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.943+0000 7feba47f1640 1 -- 172.21.15.26:0/2582309996 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb9c1a5060 con 0x7feb9c100e20 2024-09-11T23:17:48.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.945+0000 7feb8f7fe640 1 -- 172.21.15.26:0/2582309996 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7feb9800f970 con 0x7feb9c100e20 2024-09-11T23:17:48.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.945+0000 7feba6bca740 1 Processor -- start 2024-09-11T23:17:48.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.946+0000 7feba6bca740 1 -- start start 2024-09-11T23:17:48.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.946+0000 7feba6bca740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5566df5ac8b0 0x5566df5acc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:48.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.946+0000 7feba6bca740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5566df5ad1d0 0x5566df5bba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:48.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.946+0000 7feba6bca740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5566df5bbfc0 0x5566df5be3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:48.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.946+0000 7feba6bca740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5566df513090 con 0x5566df5bbfc0 2024-09-11T23:17:48.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.946+0000 7feba6bca740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5566df4baa70 con 0x5566df5ad1d0 2024-09-11T23:17:48.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.946+0000 7feba6bca740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5566df50f050 con 0x5566df5ac8b0 2024-09-11T23:17:48.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.946+0000 7feba1d8c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5566df5bbfc0 0x5566df5be3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:48.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.946+0000 7feba1d8c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5566df5bbfc0 0x5566df5be3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59882/0 (socket says 172.21.15.26:59882) 2024-09-11T23:17:48.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.946+0000 7feb8f7fe640 1 --2- 172.21.15.26:0/2582309996 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7feb880a2c90 0x7feb880a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:48.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.946+0000 7feba1d8c640 1 -- 172.21.15.26:0/1665026073 learned_addr learned my addr 172.21.15.26:0/1665026073 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:17:48.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.946+0000 7feba258d640 1 --2- 172.21.15.26:0/1665026073 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5566df5ad1d0 0x5566df5bba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:48.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.946+0000 7feba2d8e640 1 --2- 172.21.15.26:0/1665026073 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5566df5ac8b0 0x5566df5acc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:48.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.946+0000 7feba2d8e640 1 --2- 172.21.15.26:0/2582309996 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7feb880a2c90 0x7feb880a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:48.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.946+0000 7feba1d8c640 1 -- 172.21.15.26:0/1665026073 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5566df5ac8b0 msgr2=0x5566df5acc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:17:48.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.946+0000 7feba1d8c640 1 --2- 172.21.15.26:0/1665026073 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5566df5ac8b0 0x5566df5acc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:48.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.946+0000 7feb8f7fe640 1 -- 172.21.15.26:0/2582309996 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7feb980cf1a0 con 0x7feb9c100e20 2024-09-11T23:17:48.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.946+0000 7feba1d8c640 1 -- 172.21.15.26:0/1665026073 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5566df5ad1d0 msgr2=0x5566df5bba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:17:48.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.946+0000 7feba1d8c640 1 --2- 172.21.15.26:0/1665026073 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5566df5ad1d0 0x5566df5bba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:48.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.946+0000 7feba1d8c640 1 -- 172.21.15.26:0/1665026073 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566df4c7130 con 0x5566df5bbfc0 2024-09-11T23:17:48.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.947+0000 7feba1d8c640 1 --2- 172.21.15.26:0/1665026073 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5566df5bbfc0 0x5566df5be3b0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7feb90009900 tx=0x7feb90004200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2664653ff8d5af2e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:17:48.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.947+0000 7feba2d8e640 1 --2- 172.21.15.26:0/2582309996 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7feb880a2c90 0x7feb880a5150 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7feb94005fd0 tx=0x7feb94005f60 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:17:48.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.947+0000 7feb8cff9640 1 -- 172.21.15.26:0/1665026073 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb90035030 con 0x5566df5bbfc0 2024-09-11T23:17:48.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.947+0000 7feb8cff9640 1 -- 172.21.15.26:0/1665026073 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb90040070 con 0x5566df5bbfc0 2024-09-11T23:17:48.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.947+0000 7feb8cff9640 1 -- 172.21.15.26:0/1665026073 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb90031c50 con 0x5566df5bbfc0 2024-09-11T23:17:48.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.948+0000 7feba6bca740 1 -- 172.21.15.26:0/1665026073 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5566df5bbfc0 msgr2=0x5566df5be3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:48.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.948+0000 7feba6bca740 1 --2- 172.21.15.26:0/1665026073 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5566df5bbfc0 0x5566df5be3b0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7feb90009900 tx=0x7feb90004200 comp rx=0 tx=0).stop 2024-09-11T23:17:48.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.948+0000 7feba6bca740 1 -- 172.21.15.26:0/1665026073 shutdown_connections 2024-09-11T23:17:48.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.948+0000 7feba6bca740 1 --2- 172.21.15.26:0/1665026073 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5566df5ad1d0 0x5566df5bba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:48.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.948+0000 7feba6bca740 1 --2- 172.21.15.26:0/1665026073 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5566df5bbfc0 0x5566df5be3b0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:48.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.948+0000 7feba6bca740 1 --2- 172.21.15.26:0/1665026073 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5566df5ac8b0 0x5566df5acc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:48.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.948+0000 7feba6bca740 1 -- 172.21.15.26:0/1665026073 >> 172.21.15.26:0/1665026073 conn(0x5566df5a8f60 msgr2=0x5566df5a9340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:17:48.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.948+0000 7feba6bca740 1 -- 172.21.15.26:0/1665026073 shutdown_connections 2024-09-11T23:17:48.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.948+0000 7feba6bca740 1 -- 172.21.15.26:0/1665026073 wait complete. 2024-09-11T23:17:48.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.948+0000 7feba6bca740 1 -- start start 2024-09-11T23:17:48.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.948+0000 7feba6bca740 1 Processor -- start 2024-09-11T23:17:48.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.948+0000 7feba6bca740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5566df5ac8b0 0x5566df5bb7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:48.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.948+0000 7feba6bca740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5566df5ad1d0 0x5566df5b8b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:48.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.948+0000 7feba6bca740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5566df5bbfc0 0x5566df5b9060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:48.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.948+0000 7feba6bca740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5566df5beb90 con 0x5566df5ad1d0 2024-09-11T23:17:48.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.948+0000 7feba6bca740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5566df4cdf20 con 0x5566df5bbfc0 2024-09-11T23:17:48.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.948+0000 7feba6bca740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5566df5bed10 con 0x5566df5ac8b0 2024-09-11T23:17:48.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.949+0000 7feba6bca740 10 client.0 _ll_register_callbacks cb 0x5566df497030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:17:48.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.949+0000 7feba1d8c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5566df5ad1d0 0x5566df5b8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:48.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.949+0000 7feba1d8c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5566df5ad1d0 0x5566df5b8b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59890/0 (socket says 172.21.15.26:59890) 2024-09-11T23:17:48.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.949+0000 7feba1d8c640 1 -- 172.21.15.26:0/3857401815 learned_addr learned my addr 172.21.15.26:0/3857401815 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:17:48.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.949+0000 7feba258d640 1 --2- 172.21.15.26:0/3857401815 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5566df5ac8b0 0x5566df5bb7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:48.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.949+0000 7feba2d8e640 1 --2- 172.21.15.26:0/3857401815 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5566df5bbfc0 0x5566df5b9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:48.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.949+0000 7feba1d8c640 1 -- 172.21.15.26:0/3857401815 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5566df5ac8b0 msgr2=0x5566df5bb7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:17:48.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.949+0000 7feba1d8c640 1 --2- 172.21.15.26:0/3857401815 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5566df5ac8b0 0x5566df5bb7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:48.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.949+0000 7feba1d8c640 1 -- 172.21.15.26:0/3857401815 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5566df5bbfc0 msgr2=0x5566df5b9060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:17:48.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.949+0000 7feba1d8c640 1 --2- 172.21.15.26:0/3857401815 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5566df5bbfc0 0x5566df5b9060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:48.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.949+0000 7feba1d8c640 1 -- 172.21.15.26:0/3857401815 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5566df2842b0 con 0x5566df5ad1d0 2024-09-11T23:17:48.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.949+0000 7feba2d8e640 1 --2- 172.21.15.26:0/3857401815 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5566df5bbfc0 0x5566df5b9060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:17:48.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.949+0000 7feba1d8c640 1 --2- 172.21.15.26:0/3857401815 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5566df5ad1d0 0x5566df5b8b20 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7feb9002a9a0 tx=0x7feb90005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6260e61f05c60f1e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:17:48.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.949+0000 7feb85ffb640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:17:48.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.949+0000 7feb85ffb640 1 -- 172.21.15.26:0/3857401815 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb90035070 con 0x5566df5ad1d0 2024-09-11T23:17:48.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.949+0000 7feb85ffb640 1 -- 172.21.15.26:0/3857401815 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7feb90003a00 con 0x5566df5ad1d0 2024-09-11T23:17:48.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.949+0000 7feba6bca740 1 -- 172.21.15.26:0/3857401815 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb90040070 con 0x5566df5ad1d0 2024-09-11T23:17:48.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.949+0000 7feb85ffb640 1 -- 172.21.15.26:0/3857401815 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7feb90009520 con 0x5566df5ad1d0 2024-09-11T23:17:48.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.949+0000 7feba6bca740 10 client.14991 fetch_fsmap learned FSMap version 28 2024-09-11T23:17:48.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.949+0000 7feba6bca740 1 -- 172.21.15.26:0/3857401815 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5566df5ca490 con 0x5566df5ad1d0 2024-09-11T23:17:48.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.950+0000 7feb85ffb640 1 -- 172.21.15.26:0/3857401815 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb90040070 con 0x5566df5ad1d0 2024-09-11T23:17:48.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.950+0000 7feb85ffb640 1 -- 172.21.15.26:0/3857401815 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7feb90045950 con 0x5566df5ad1d0 2024-09-11T23:17:48.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.950+0000 7feba6bca740 10 client.14991 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:17:48.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.950+0000 7feba6bca740 10 client.14991 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-11T23:17:48.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.950+0000 7feba6bca740 20 client.14991 populate_metadata read hostname 'smithi026' 2024-09-11T23:17:48.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.950+0000 7feba6bca740 1 --2- 172.21.15.26:0/3857401815 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5566df5cc5c0 0x5566df5cda30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:48.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.950+0000 7feba6bca740 4 client.14991 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:17:48.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.950+0000 7feba6bca740 1 -- 172.21.15.26:0/3857401815 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5566df520600 con 0x5566df5cc5c0 2024-09-11T23:17:48.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.950+0000 7feba258d640 1 --2- 172.21.15.26:0/3857401815 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5566df5cc5c0 0x5566df5cda30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:48.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.951+0000 7feba258d640 1 --2- 172.21.15.26:0/3857401815 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5566df5cc5c0 0x5566df5cda30 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d98b5ee133da4152 server_cookie=bfea927e82c827c3 in_seq=0 out_seq=0 2024-09-11T23:17:48.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.951+0000 7feb85ffb640 10 client.14991 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:17:48.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.951+0000 7feb85ffb640 1 -- 172.21.15.26:0/3857401815 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5566df520600 con 0x5566df5cc5c0 2024-09-11T23:17:48.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.951+0000 7feb85ffb640 10 client.14991 handle_command_reply: tid=0 2024-09-11T23:17:48.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.951+0000 7feba6bca740 1 client.14991 shutdown 2024-09-11T23:17:48.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.951+0000 7feba6bca740 1 -- 172.21.15.26:0/3857401815 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5566df5ad1d0 msgr2=0x5566df5b8b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:48.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.951+0000 7feba6bca740 1 --2- 172.21.15.26:0/3857401815 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5566df5ad1d0 0x5566df5b8b20 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7feb9002a9a0 tx=0x7feb90005c00 comp rx=0 tx=0).stop 2024-09-11T23:17:48.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.951+0000 7feba6bca740 1 -- 172.21.15.26:0/3857401815 shutdown_connections 2024-09-11T23:17:48.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.951+0000 7feba6bca740 1 --2- 172.21.15.26:0/3857401815 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5566df5bbfc0 0x5566df5b9060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:48.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.951+0000 7feba6bca740 1 --2- 172.21.15.26:0/3857401815 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5566df5cc5c0 0x5566df5cda30 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:48.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.951+0000 7feb85ffb640 0 client.14991 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:17:48.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.952+0000 7feba6bca740 1 --2- 172.21.15.26:0/3857401815 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5566df5ad1d0 0x5566df5b8b20 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:48.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.952+0000 7feba6bca740 1 --2- 172.21.15.26:0/3857401815 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5566df5ac8b0 0x5566df5bb7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:48.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.952+0000 7feba6bca740 1 -- 172.21.15.26:0/3857401815 >> 172.21.15.26:0/3857401815 conn(0x5566df5a8f60 msgr2=0x5566df5aecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:17:48.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.952+0000 7feba6bca740 1 -- 172.21.15.26:0/3857401815 shutdown_connections 2024-09-11T23:17:48.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.952+0000 7feba6bca740 1 -- 172.21.15.26:0/3857401815 wait complete. 2024-09-11T23:17:48.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.952+0000 7feba6bca740 20 client.14991 trim_cache size 0 max 16384 2024-09-11T23:17:48.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.970+0000 7feba6bca740 1 Processor -- start 2024-09-11T23:17:48.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.970+0000 7feba6bca740 1 -- start start 2024-09-11T23:17:48.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.970+0000 7feba6bca740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5566df5ac8b0 0x5566df5cbef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:48.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.970+0000 7feba6bca740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5566df5ad1d0 0x5566df5ccc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:48.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.970+0000 7feba6bca740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5566df5bbfc0 0x5566df5b4240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:48.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.970+0000 7feba6bca740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5566df5cbd70 con 0x5566df5ac8b0 2024-09-11T23:17:48.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.970+0000 7feba6bca740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5566df5cc440 con 0x5566df5ad1d0 2024-09-11T23:17:48.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.970+0000 7feba6bca740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5566df5ccb10 con 0x5566df5bbfc0 2024-09-11T23:17:48.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.970+0000 7feba258d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5566df5ac8b0 0x5566df5cbef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:48.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.970+0000 7feba258d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5566df5ac8b0 0x5566df5cbef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59902/0 (socket says 172.21.15.26:59902) 2024-09-11T23:17:48.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.970+0000 7feba258d640 1 -- 172.21.15.26:0/1797157761 learned_addr learned my addr 172.21.15.26:0/1797157761 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:17:48.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.971+0000 7feba1d8c640 1 --2- 172.21.15.26:0/1797157761 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5566df5ad1d0 0x5566df5ccc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:48.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.971+0000 7feba2d8e640 1 --2- 172.21.15.26:0/1797157761 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5566df5bbfc0 0x5566df5b4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:48.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.971+0000 7feba258d640 1 -- 172.21.15.26:0/1797157761 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5566df5bbfc0 msgr2=0x5566df5b4240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:17:48.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.971+0000 7feba258d640 1 --2- 172.21.15.26:0/1797157761 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5566df5bbfc0 0x5566df5b4240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:48.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.971+0000 7feba258d640 1 -- 172.21.15.26:0/1797157761 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5566df5ad1d0 msgr2=0x5566df5ccc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:17:48.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.971+0000 7feba258d640 1 --2- 172.21.15.26:0/1797157761 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5566df5ad1d0 0x5566df5ccc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:48.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.971+0000 7feba258d640 1 -- 172.21.15.26:0/1797157761 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566df5a8a00 con 0x5566df5ac8b0 2024-09-11T23:17:48.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.971+0000 7feba1d8c640 1 --2- 172.21.15.26:0/1797157761 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5566df5ad1d0 0x5566df5ccc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:17:48.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.971+0000 7feba258d640 1 --2- 172.21.15.26:0/1797157761 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5566df5ac8b0 0x5566df5cbef0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7feb98000f80 tx=0x7feb98086040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e57dab4324fa87e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:17:48.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.971+0000 7feb87fff640 1 -- 172.21.15.26:0/1797157761 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb9801d040 con 0x5566df5ac8b0 2024-09-11T23:17:48.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.971+0000 7feb87fff640 1 -- 172.21.15.26:0/1797157761 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb98066170 con 0x5566df5ac8b0 2024-09-11T23:17:48.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.971+0000 7feb87fff640 1 -- 172.21.15.26:0/1797157761 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb980cced0 con 0x5566df5ac8b0 2024-09-11T23:17:48.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.972+0000 7feba6bca740 1 -- 172.21.15.26:0/1797157761 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5566df5ac8b0 msgr2=0x5566df5cbef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:48.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.972+0000 7feba6bca740 1 --2- 172.21.15.26:0/1797157761 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5566df5ac8b0 0x5566df5cbef0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7feb98000f80 tx=0x7feb98086040 comp rx=0 tx=0).stop 2024-09-11T23:17:48.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.972+0000 7feba6bca740 1 -- 172.21.15.26:0/1797157761 shutdown_connections 2024-09-11T23:17:48.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.972+0000 7feba6bca740 1 --2- 172.21.15.26:0/1797157761 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5566df5ad1d0 0x5566df5ccc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:48.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.972+0000 7feba6bca740 1 --2- 172.21.15.26:0/1797157761 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5566df5bbfc0 0x5566df5b4240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:48.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.972+0000 7feba6bca740 1 --2- 172.21.15.26:0/1797157761 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5566df5ac8b0 0x5566df5cbef0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:48.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.972+0000 7feba6bca740 1 -- 172.21.15.26:0/1797157761 >> 172.21.15.26:0/1797157761 conn(0x5566df5a8f60 msgr2=0x5566df5a9340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:17:48.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.972+0000 7feba6bca740 1 -- 172.21.15.26:0/1797157761 shutdown_connections 2024-09-11T23:17:48.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.972+0000 7feba6bca740 1 -- 172.21.15.26:0/1797157761 wait complete. 2024-09-11T23:17:48.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.972+0000 7feba6bca740 1 -- start start 2024-09-11T23:17:48.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.972+0000 7feba6bca740 1 Processor -- start 2024-09-11T23:17:48.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.973+0000 7feba6bca740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5566df5ac8b0 0x5566df5d5f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:48.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.973+0000 7feba6bca740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5566df5ad1d0 0x5566df5d6460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:48.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.973+0000 7feba6bca740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5566df5bbfc0 0x5566df5cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:48.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.973+0000 7feba6bca740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5566df5cd8b0 con 0x5566df5bbfc0 2024-09-11T23:17:48.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.973+0000 7feba6bca740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5566df5cd1e0 con 0x5566df5ac8b0 2024-09-11T23:17:48.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.973+0000 7feba6bca740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5566df5cdf80 con 0x5566df5ad1d0 2024-09-11T23:17:48.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.973+0000 7feba2d8e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5566df5bbfc0 0x5566df5cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:48.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.973+0000 7feba2d8e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5566df5bbfc0 0x5566df5cf280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59918/0 (socket says 172.21.15.26:59918) 2024-09-11T23:17:48.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.973+0000 7feba2d8e640 1 -- 172.21.15.26:0/2652704132 learned_addr learned my addr 172.21.15.26:0/2652704132 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:17:48.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.973+0000 7feba6bca740 10 client.0 _ll_register_callbacks cb 0x7feb98097510 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:17:48.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.973+0000 7feba1d8c640 1 --2- 172.21.15.26:0/2652704132 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5566df5ad1d0 0x5566df5d6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:48.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.973+0000 7feba258d640 1 --2- 172.21.15.26:0/2652704132 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5566df5ac8b0 0x5566df5d5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:48.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.973+0000 7feba2d8e640 1 -- 172.21.15.26:0/2652704132 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5566df5ad1d0 msgr2=0x5566df5d6460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:17:48.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.973+0000 7feba2d8e640 1 --2- 172.21.15.26:0/2652704132 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5566df5ad1d0 0x5566df5d6460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:48.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.973+0000 7feba2d8e640 1 -- 172.21.15.26:0/2652704132 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5566df5ac8b0 msgr2=0x5566df5d5f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:17:48.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.973+0000 7feba2d8e640 1 --2- 172.21.15.26:0/2652704132 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5566df5ac8b0 0x5566df5d5f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:48.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.973+0000 7feba2d8e640 1 -- 172.21.15.26:0/2652704132 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5566df5cfab0 con 0x5566df5bbfc0 2024-09-11T23:17:48.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.973+0000 7feba258d640 1 --2- 172.21.15.26:0/2652704132 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5566df5ac8b0 0x5566df5d5f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:17:48.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.973+0000 7feba2d8e640 1 --2- 172.21.15.26:0/2652704132 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5566df5bbfc0 0x5566df5cf280 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7feb9400b010 tx=0x7feb940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68b09e1423f3a169 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:17:48.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.973+0000 7feb85ffb640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:17:48.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.973+0000 7feb85ffb640 1 -- 172.21.15.26:0/2652704132 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb940114a0 con 0x5566df5bbfc0 2024-09-11T23:17:48.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.973+0000 7feb85ffb640 1 -- 172.21.15.26:0/2652704132 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7feb94005020 con 0x5566df5bbfc0 2024-09-11T23:17:48.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.973+0000 7feba6bca740 1 -- 172.21.15.26:0/2652704132 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5566df51ab10 con 0x5566df5bbfc0 2024-09-11T23:17:48.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.974+0000 7feb85ffb640 1 -- 172.21.15.26:0/2652704132 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7feb9400a840 con 0x5566df5bbfc0 2024-09-11T23:17:48.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.974+0000 7feba6bca740 10 client.14997 fetch_fsmap learned FSMap version 28 2024-09-11T23:17:48.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.974+0000 7feba6bca740 1 -- 172.21.15.26:0/2652704132 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5566df5d4520 con 0x5566df5bbfc0 2024-09-11T23:17:48.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.974+0000 7feb85ffb640 1 -- 172.21.15.26:0/2652704132 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5566df51ab10 con 0x5566df5bbfc0 2024-09-11T23:17:48.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.974+0000 7feb85ffb640 1 -- 172.21.15.26:0/2652704132 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7feb940283f0 con 0x5566df5bbfc0 2024-09-11T23:17:48.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.974+0000 7feba6bca740 10 client.14997 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:17:48.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.974+0000 7feba6bca740 10 client.14997 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-11T23:17:48.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.974+0000 7feba6bca740 20 client.14997 populate_metadata read hostname 'smithi026' 2024-09-11T23:17:48.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.974+0000 7feba6bca740 1 --2- 172.21.15.26:0/2652704132 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5566df5cc5c0 0x5566df63a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:48.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.974+0000 7feba6bca740 4 client.14997 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-11T23:17:48.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.974+0000 7feba6bca740 1 -- 172.21.15.26:0/2652704132 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "status"}) -- 0x5566df4970e0 con 0x5566df5cc5c0 2024-09-11T23:17:48.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.974+0000 7feba258d640 1 --2- 172.21.15.26:0/2652704132 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5566df5cc5c0 0x5566df63a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:48.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.975+0000 7feb85ffb640 10 client.14997 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:17:48.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.975+0000 7feba258d640 1 --2- 172.21.15.26:0/2652704132 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5566df5cc5c0 0x5566df63a160 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=44d5f67ae2fb3c1d server_cookie=74ee8770dc43b9da in_seq=0 out_seq=0 2024-09-11T23:17:48.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.975+0000 7feb85ffb640 1 -- 172.21.15.26:0/2652704132 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5566df4970e0 con 0x5566df5cc5c0 2024-09-11T23:17:48.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.975+0000 7feb85ffb640 10 client.14997 handle_command_reply: tid=0 2024-09-11T23:17:48.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.975+0000 7feba6bca740 1 client.14997 shutdown 2024-09-11T23:17:48.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.975+0000 7feba6bca740 1 -- 172.21.15.26:0/2652704132 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5566df5bbfc0 msgr2=0x5566df5cf280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:48.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.975+0000 7feba6bca740 1 --2- 172.21.15.26:0/2652704132 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5566df5bbfc0 0x5566df5cf280 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7feb9400b010 tx=0x7feb940046c0 comp rx=0 tx=0).stop 2024-09-11T23:17:48.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.975+0000 7feba6bca740 1 -- 172.21.15.26:0/2652704132 shutdown_connections 2024-09-11T23:17:48.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.975+0000 7feba6bca740 1 --2- 172.21.15.26:0/2652704132 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5566df5cc5c0 0x5566df63a160 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:48.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.975+0000 7feba6bca740 1 --2- 172.21.15.26:0/2652704132 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5566df5bbfc0 0x5566df5cf280 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:48.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.975+0000 7feba6bca740 1 --2- 172.21.15.26:0/2652704132 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5566df5ad1d0 0x5566df5d6460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:48.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.975+0000 7feba6bca740 1 --2- 172.21.15.26:0/2652704132 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5566df5ac8b0 0x5566df5d5f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:48.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.975+0000 7feba6bca740 1 -- 172.21.15.26:0/2652704132 >> 172.21.15.26:0/2652704132 conn(0x5566df5a8f60 msgr2=0x5566df5a3ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:17:48.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.975+0000 7feb85ffb640 0 client.14997 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:17:48.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.975+0000 7feba6bca740 1 -- 172.21.15.26:0/2652704132 shutdown_connections 2024-09-11T23:17:48.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.975+0000 7feba6bca740 1 -- 172.21.15.26:0/2652704132 wait complete. 2024-09-11T23:17:48.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.975+0000 7feba6bca740 20 client.14997 trim_cache size 0 max 16384 2024-09-11T23:17:48.987 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-11T23:17:48.987 INFO:teuthology.orchestra.run.smithi026.stdout: "cluster_fsid": "af7e6fc8-7092-11ef-bcea-c7b262605968", 2024-09-11T23:17:48.988 INFO:teuthology.orchestra.run.smithi026.stdout: "whoami": 0, 2024-09-11T23:17:48.988 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 24485, 2024-09-11T23:17:48.988 INFO:teuthology.orchestra.run.smithi026.stdout: "want_state": "up:active", 2024-09-11T23:17:48.988 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "up:active", 2024-09-11T23:17:48.988 INFO:teuthology.orchestra.run.smithi026.stdout: "fs_name": "cephfs", 2024-09-11T23:17:48.988 INFO:teuthology.orchestra.run.smithi026.stdout: "rank_uptime": 82.070096504000006, 2024-09-11T23:17:48.988 INFO:teuthology.orchestra.run.smithi026.stdout: "mdsmap_epoch": 28, 2024-09-11T23:17:48.988 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch": 85, 2024-09-11T23:17:48.988 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch_barrier": 85, 2024-09-11T23:17:48.988 INFO:teuthology.orchestra.run.smithi026.stdout: "uptime": 92.886501609999996 2024-09-11T23:17:48.988 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-11T23:17:48.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.976+0000 7feba47f1640 1 -- 172.21.15.26:0/2582309996 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7feb880a2c90 msgr2=0x7feb880a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:48.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.976+0000 7feba47f1640 1 --2- 172.21.15.26:0/2582309996 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7feb880a2c90 0x7feb880a5150 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7feb94005fd0 tx=0x7feb94005f60 comp rx=0 tx=0).stop 2024-09-11T23:17:48.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.976+0000 7feba47f1640 1 -- 172.21.15.26:0/2582309996 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb9c100e20 msgr2=0x7feb9c1993c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:48.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.976+0000 7feba47f1640 1 --2- 172.21.15.26:0/2582309996 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb9c100e20 0x7feb9c1993c0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7feb9800cce0 tx=0x7feb9800a620 comp rx=0 tx=0).stop 2024-09-11T23:17:48.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.976+0000 7feba47f1640 1 -- 172.21.15.26:0/2582309996 shutdown_connections 2024-09-11T23:17:48.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.976+0000 7feba47f1640 1 --2- 172.21.15.26:0/2582309996 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7feb880a2c90 0x7feb880a5150 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:48.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.976+0000 7feba47f1640 1 --2- 172.21.15.26:0/2582309996 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7feb9c102ef0 0x7feb9c1a4440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:48.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.976+0000 7feba47f1640 1 --2- 172.21.15.26:0/2582309996 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7feb9c102570 0x7feb9c199900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:48.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.976+0000 7feba47f1640 1 --2- 172.21.15.26:0/2582309996 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb9c100e20 0x7feb9c1993c0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:48.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.976+0000 7feba47f1640 1 -- 172.21.15.26:0/2582309996 >> 172.21.15.26:0/2582309996 conn(0x7feb9c0fc7d0 msgr2=0x7feb9c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:17:48.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.976+0000 7feba47f1640 1 -- 172.21.15.26:0/2582309996 shutdown_connections 2024-09-11T23:17:48.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:48.976+0000 7feba47f1640 1 -- 172.21.15.26:0/2582309996 wait complete. 2024-09-11T23:17:49.768 INFO:tasks.cephadm:Setup complete, yielding 2024-09-11T23:17:49.768 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-11T23:17:49.780 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi026.front.sepia.ceph.com 2024-09-11T23:17:49.780 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- bash -c 'ceph orch status' 2024-09-11T23:17:50.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:17:49 smithi026 ceph-mon[31172]: pgmap v322: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T23:17:50.038 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:17:50.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:17:49 smithi136 ceph-mon[35121]: pgmap v322: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T23:17:50.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:17:49 smithi076 ceph-mon[35296]: pgmap v322: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T23:17:50.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:50.549+0000 7f7f4fdb4640 1 -- 172.21.15.26:0/2614951021 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7f48101c00 msgr2=0x7f7f48102000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:50.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:50.549+0000 7f7f4fdb4640 1 --2- 172.21.15.26:0/2614951021 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7f48101c00 0x7f7f48102000 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f7f44009900 tx=0x7f7f4402a3f0 comp rx=0 tx=0).stop 2024-09-11T23:17:50.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:50.549+0000 7f7f4fdb4640 1 -- 172.21.15.26:0/2614951021 shutdown_connections 2024-09-11T23:17:50.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:50.549+0000 7f7f4fdb4640 1 --2- 172.21.15.26:0/2614951021 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7f481037c0 0x7f7f4810a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:50.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:50.549+0000 7f7f4fdb4640 1 --2- 172.21.15.26:0/2614951021 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7f48102e00 0x7f7f48103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:50.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:50.549+0000 7f7f4fdb4640 1 --2- 172.21.15.26:0/2614951021 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7f48101c00 0x7f7f48102000 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:50.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:50.549+0000 7f7f4fdb4640 1 -- 172.21.15.26:0/2614951021 >> 172.21.15.26:0/2614951021 conn(0x7f7f480fd390 msgr2=0x7f7f480ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:17:50.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:50.549+0000 7f7f4fdb4640 1 -- 172.21.15.26:0/2614951021 shutdown_connections 2024-09-11T23:17:50.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:50.549+0000 7f7f4fdb4640 1 -- 172.21.15.26:0/2614951021 wait complete. 2024-09-11T23:17:50.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:50.549+0000 7f7f4fdb4640 1 Processor -- start 2024-09-11T23:17:50.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:50.549+0000 7f7f4fdb4640 1 -- start start 2024-09-11T23:17:50.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:50.550+0000 7f7f4fdb4640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7f48101c00 0x7f7f48075f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:50.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:50.550+0000 7f7f4fdb4640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7f48102e00 0x7f7f480745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:50.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:50.550+0000 7f7f4fdb4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7f481037c0 0x7f7f48074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:50.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:50.550+0000 7f7f4fdb4640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f7f4810d8b0 con 0x7f7f481037c0 2024-09-11T23:17:50.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:50.550+0000 7f7f4fdb4640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f7f4810d730 con 0x7f7f48101c00 2024-09-11T23:17:50.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:50.550+0000 7f7f4fdb4640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f7f4810da30 con 0x7f7f48102e00 2024-09-11T23:17:50.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:50.550+0000 7f7f4e351640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7f481037c0 0x7f7f48074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:50.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:50.550+0000 7f7f4e351640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7f481037c0 0x7f7f48074b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59480/0 (socket says 172.21.15.26:59480) 2024-09-11T23:17:50.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:50.550+0000 7f7f4e351640 1 -- 172.21.15.26:0/982900783 learned_addr learned my addr 172.21.15.26:0/982900783 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:17:50.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:50.550+0000 7f7f4d34f640 1 --2- 172.21.15.26:0/982900783 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7f48102e00 0x7f7f480745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:50.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:50.550+0000 7f7f4db50640 1 --2- 172.21.15.26:0/982900783 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7f48101c00 0x7f7f48075f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:50.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:50.550+0000 7f7f4e351640 1 -- 172.21.15.26:0/982900783 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7f48102e00 msgr2=0x7f7f480745e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:50.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:50.550+0000 7f7f4e351640 1 --2- 172.21.15.26:0/982900783 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7f48102e00 0x7f7f480745e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:50.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:50.550+0000 7f7f4e351640 1 -- 172.21.15.26:0/982900783 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7f48101c00 msgr2=0x7f7f48075f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:50.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:50.550+0000 7f7f4e351640 1 --2- 172.21.15.26:0/982900783 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7f48101c00 0x7f7f48075f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:50.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:50.550+0000 7f7f4e351640 1 -- 172.21.15.26:0/982900783 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f480753e0 con 0x7f7f481037c0 2024-09-11T23:17:50.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:50.550+0000 7f7f4d34f640 1 --2- 172.21.15.26:0/982900783 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7f48102e00 0x7f7f480745e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:17:50.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:50.550+0000 7f7f4e351640 1 --2- 172.21.15.26:0/982900783 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7f481037c0 0x7f7f48074b20 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f7f4000cce0 tx=0x7f7f4000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:17:50.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:50.550+0000 7f7f3effd640 1 -- 172.21.15.26:0/982900783 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f4000f450 con 0x7f7f481037c0 2024-09-11T23:17:50.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:50.550+0000 7f7f3effd640 1 -- 172.21.15.26:0/982900783 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f4000fad0 con 0x7f7f481037c0 2024-09-11T23:17:50.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:50.550+0000 7f7f3effd640 1 -- 172.21.15.26:0/982900783 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f40018c10 con 0x7f7f481037c0 2024-09-11T23:17:50.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:50.551+0000 7f7f4fdb4640 1 -- 172.21.15.26:0/982900783 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f481a1f80 con 0x7f7f481037c0 2024-09-11T23:17:50.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:50.551+0000 7f7f4fdb4640 1 -- 172.21.15.26:0/982900783 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f481a23e0 con 0x7f7f481037c0 2024-09-11T23:17:50.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:50.551+0000 7f7f4fdb4640 1 -- 172.21.15.26:0/982900783 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f48108ed0 con 0x7f7f481037c0 2024-09-11T23:17:50.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:50.553+0000 7f7f3effd640 1 -- 172.21.15.26:0/982900783 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f7f4000f5f0 con 0x7f7f481037c0 2024-09-11T23:17:50.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:50.554+0000 7f7f3effd640 1 --2- 172.21.15.26:0/982900783 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f7f340a2c90 0x7f7f340a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:50.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:50.554+0000 7f7f4db50640 1 --2- 172.21.15.26:0/982900783 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f7f340a2c90 0x7f7f340a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:50.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:50.554+0000 7f7f3effd640 1 -- 172.21.15.26:0/982900783 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f7f400cf8c0 con 0x7f7f481037c0 2024-09-11T23:17:50.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:50.554+0000 7f7f4db50640 1 --2- 172.21.15.26:0/982900783 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f7f340a2c90 0x7f7f340a5150 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f7f4402a9e0 tx=0x7f7f44031400 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:17:50.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:50.558+0000 7f7f3effd640 1 -- 172.21.15.26:0/982900783 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7f40014030 con 0x7f7f481037c0 2024-09-11T23:17:50.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:50.752+0000 7f7f4fdb4640 1 -- 172.21.15.26:0/982900783 --> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f7f48062630 con 0x7f7f340a2c90 2024-09-11T23:17:50.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:50.753+0000 7f7f3effd640 1 -- 172.21.15.26:0/982900783 <== mgr.14148 v2:172.21.15.26:6800/110833859 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f7f48062630 con 0x7f7f340a2c90 2024-09-11T23:17:50.753 INFO:teuthology.orchestra.run.smithi026.stdout:Backend: cephadm 2024-09-11T23:17:50.753 INFO:teuthology.orchestra.run.smithi026.stdout:Available: Yes 2024-09-11T23:17:50.753 INFO:teuthology.orchestra.run.smithi026.stdout:Paused: No 2024-09-11T23:17:50.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:50.755+0000 7f7f4fdb4640 1 -- 172.21.15.26:0/982900783 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f7f340a2c90 msgr2=0x7f7f340a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:50.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:50.755+0000 7f7f4fdb4640 1 --2- 172.21.15.26:0/982900783 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f7f340a2c90 0x7f7f340a5150 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f7f4402a9e0 tx=0x7f7f44031400 comp rx=0 tx=0).stop 2024-09-11T23:17:50.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:50.755+0000 7f7f4fdb4640 1 -- 172.21.15.26:0/982900783 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7f481037c0 msgr2=0x7f7f48074b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:50.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:50.755+0000 7f7f4fdb4640 1 --2- 172.21.15.26:0/982900783 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7f481037c0 0x7f7f48074b20 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f7f4000cce0 tx=0x7f7f4000a620 comp rx=0 tx=0).stop 2024-09-11T23:17:50.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:50.755+0000 7f7f4fdb4640 1 -- 172.21.15.26:0/982900783 shutdown_connections 2024-09-11T23:17:50.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:50.755+0000 7f7f4fdb4640 1 --2- 172.21.15.26:0/982900783 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f7f340a2c90 0x7f7f340a5150 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:50.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:50.755+0000 7f7f4fdb4640 1 --2- 172.21.15.26:0/982900783 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7f481037c0 0x7f7f48074b20 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:50.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:50.755+0000 7f7f4fdb4640 1 --2- 172.21.15.26:0/982900783 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7f48102e00 0x7f7f480745e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:50.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:50.755+0000 7f7f4fdb4640 1 --2- 172.21.15.26:0/982900783 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7f48101c00 0x7f7f48075f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:50.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:50.755+0000 7f7f4fdb4640 1 -- 172.21.15.26:0/982900783 >> 172.21.15.26:0/982900783 conn(0x7f7f480fd390 msgr2=0x7f7f481060a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:17:50.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:50.755+0000 7f7f4fdb4640 1 -- 172.21.15.26:0/982900783 shutdown_connections 2024-09-11T23:17:50.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:50.755+0000 7f7f4fdb4640 1 -- 172.21.15.26:0/982900783 wait complete. 2024-09-11T23:17:51.473 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- bash -c 'ceph orch ps' 2024-09-11T23:17:51.739 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:17:52.134 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:17:51 smithi026 ceph-mon[31172]: pgmap v323: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-11T23:17:52.134 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:17:51 smithi026 ceph-mon[31172]: from='client.15003 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:17:52.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:52.216+0000 7fbc4507e640 1 -- 172.21.15.26:0/3339765261 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbc40107900 msgr2=0x7fbc40109d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:52.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:52.216+0000 7fbc4507e640 1 --2- 172.21.15.26:0/3339765261 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbc40107900 0x7fbc40109d50 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fbc38009ca0 tx=0x7fbc3802a8c0 comp rx=0 tx=0).stop 2024-09-11T23:17:52.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:52.216+0000 7fbc4507e640 1 -- 172.21.15.26:0/3339765261 shutdown_connections 2024-09-11T23:17:52.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:52.216+0000 7fbc4507e640 1 --2- 172.21.15.26:0/3339765261 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbc40104fd0 0x7fbc401073c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:52.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:52.216+0000 7fbc4507e640 1 --2- 172.21.15.26:0/3339765261 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbc40107900 0x7fbc40109d50 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:52.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:52.216+0000 7fbc4507e640 1 --2- 172.21.15.26:0/3339765261 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbc401026a0 0x7fbc40104a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:52.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:52.216+0000 7fbc4507e640 1 -- 172.21.15.26:0/3339765261 >> 172.21.15.26:0/3339765261 conn(0x7fbc400fc2a0 msgr2=0x7fbc400fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:17:52.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:52.216+0000 7fbc4507e640 1 -- 172.21.15.26:0/3339765261 shutdown_connections 2024-09-11T23:17:52.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:52.216+0000 7fbc4507e640 1 -- 172.21.15.26:0/3339765261 wait complete. 2024-09-11T23:17:52.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:52.217+0000 7fbc4507e640 1 Processor -- start 2024-09-11T23:17:52.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:52.217+0000 7fbc4507e640 1 -- start start 2024-09-11T23:17:52.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:52.217+0000 7fbc4507e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbc401026a0 0x7fbc40197e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:52.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:52.217+0000 7fbc4507e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbc40104fd0 0x7fbc40198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:52.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:52.217+0000 7fbc4507e640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbc40107900 0x7fbc4019f400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:52.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:52.217+0000 7fbc4507e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fbc4010d8a0 con 0x7fbc40104fd0 2024-09-11T23:17:52.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:52.217+0000 7fbc4507e640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fbc4010d720 con 0x7fbc401026a0 2024-09-11T23:17:52.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:52.217+0000 7fbc4507e640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fbc4010da20 con 0x7fbc40107900 2024-09-11T23:17:52.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:52.217+0000 7fbc3e59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbc40104fd0 0x7fbc40198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:52.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:52.217+0000 7fbc3e59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbc40104fd0 0x7fbc40198380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59500/0 (socket says 172.21.15.26:59500) 2024-09-11T23:17:52.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:52.217+0000 7fbc3e59c640 1 -- 172.21.15.26:0/3253888124 learned_addr learned my addr 172.21.15.26:0/3253888124 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:17:52.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:52.217+0000 7fbc3ed9d640 1 --2- 172.21.15.26:0/3253888124 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbc401026a0 0x7fbc40197e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:52.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:52.217+0000 7fbc3f59e640 1 --2- 172.21.15.26:0/3253888124 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbc40107900 0x7fbc4019f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:52.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:52.218+0000 7fbc3e59c640 1 -- 172.21.15.26:0/3253888124 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbc40107900 msgr2=0x7fbc4019f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:52.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:52.218+0000 7fbc3e59c640 1 --2- 172.21.15.26:0/3253888124 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbc40107900 0x7fbc4019f400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:52.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:52.218+0000 7fbc3e59c640 1 -- 172.21.15.26:0/3253888124 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbc401026a0 msgr2=0x7fbc40197e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:52.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:52.218+0000 7fbc3e59c640 1 --2- 172.21.15.26:0/3253888124 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbc401026a0 0x7fbc40197e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:52.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:52.218+0000 7fbc3e59c640 1 -- 172.21.15.26:0/3253888124 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc4019fa70 con 0x7fbc40104fd0 2024-09-11T23:17:52.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:52.218+0000 7fbc3f59e640 1 --2- 172.21.15.26:0/3253888124 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbc40107900 0x7fbc4019f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:17:52.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:52.218+0000 7fbc3e59c640 1 --2- 172.21.15.26:0/3253888124 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbc40104fd0 0x7fbc40198380 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fbc3400c970 tx=0x7fbc3400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:17:52.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:52.218+0000 7fbc27fff640 1 -- 172.21.15.26:0/3253888124 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc34014070 con 0x7fbc40104fd0 2024-09-11T23:17:52.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:52.218+0000 7fbc4507e640 1 -- 172.21.15.26:0/3253888124 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc4019fd60 con 0x7fbc40104fd0 2024-09-11T23:17:52.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:52.218+0000 7fbc27fff640 1 -- 172.21.15.26:0/3253888124 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc34004440 con 0x7fbc40104fd0 2024-09-11T23:17:52.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:52.218+0000 7fbc27fff640 1 -- 172.21.15.26:0/3253888124 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc3400f450 con 0x7fbc40104fd0 2024-09-11T23:17:52.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:52.218+0000 7fbc4507e640 1 -- 172.21.15.26:0/3253888124 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc401a0210 con 0x7fbc40104fd0 2024-09-11T23:17:52.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:52.219+0000 7fbc4507e640 1 -- 172.21.15.26:0/3253888124 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc400fdd90 con 0x7fbc40104fd0 2024-09-11T23:17:52.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:52.220+0000 7fbc27fff640 1 -- 172.21.15.26:0/3253888124 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fbc34017070 con 0x7fbc40104fd0 2024-09-11T23:17:52.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:52.221+0000 7fbc27fff640 1 --2- 172.21.15.26:0/3253888124 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fbc280a2c40 0x7fbc280a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:52.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:52.221+0000 7fbc3ed9d640 1 --2- 172.21.15.26:0/3253888124 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fbc280a2c40 0x7fbc280a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:52.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:52.221+0000 7fbc27fff640 1 -- 172.21.15.26:0/3253888124 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fbc340cf400 con 0x7fbc40104fd0 2024-09-11T23:17:52.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:52.222+0000 7fbc3ed9d640 1 --2- 172.21.15.26:0/3253888124 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fbc280a2c40 0x7fbc280a5100 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fbc30005d00 tx=0x7fbc3000b410 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:17:52.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:52.225+0000 7fbc27fff640 1 -- 172.21.15.26:0/3253888124 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fbc3401e030 con 0x7fbc40104fd0 2024-09-11T23:17:52.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:17:51 smithi136 ceph-mon[35121]: pgmap v323: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-11T23:17:52.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:17:51 smithi136 ceph-mon[35121]: from='client.15003 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:17:52.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:17:51 smithi076 ceph-mon[35296]: pgmap v323: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-11T23:17:52.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:17:51 smithi076 ceph-mon[35296]: from='client.15003 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:17:52.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:52.409+0000 7fbc4507e640 1 -- 172.21.15.26:0/3253888124 --> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7fbc40062630 con 0x7fbc280a2c40 2024-09-11T23:17:52.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:52.421+0000 7fbc27fff640 1 -- 172.21.15.26:0/3253888124 <== mgr.14148 v2:172.21.15.26:6800/110833859 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4154 (secure 0 0 0) 0x7fbc40062630 con 0x7fbc280a2c40 2024-09-11T23:17:52.422 INFO:teuthology.orchestra.run.smithi026.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-11T23:17:52.422 INFO:teuthology.orchestra.run.smithi026.stdout:mds.a smithi026 running (99s) 21s ago 98s 19.2M - 19.1.1-298-gb53b6d3e 54467abc55f8 d6d92c598f29 2024-09-11T23:17:52.422 INFO:teuthology.orchestra.run.smithi026.stdout:mds.b smithi076 running (96s) 29s ago 96s 26.6M - 19.1.1-298-gb53b6d3e 54467abc55f8 74534ea440b6 2024-09-11T23:17:52.422 INFO:teuthology.orchestra.run.smithi026.stdout:mds.c smithi136 running (111s) 29s ago 111s 20.2M - 19.1.1-298-gb53b6d3e 54467abc55f8 a8bc986f7080 2024-09-11T23:17:52.422 INFO:teuthology.orchestra.run.smithi026.stdout:mds.d smithi026 running (119s) 21s ago 118s 14.4M - 19.1.1-298-gb53b6d3e 54467abc55f8 25e20ec815ee 2024-09-11T23:17:52.422 INFO:teuthology.orchestra.run.smithi026.stdout:mds.e smithi076 running (116s) 29s ago 116s 17.9M - 19.1.1-298-gb53b6d3e 54467abc55f8 6f0b7c1d8960 2024-09-11T23:17:52.422 INFO:teuthology.orchestra.run.smithi026.stdout:mds.f smithi136 running (114s) 29s ago 113s 20.0M - 19.1.1-298-gb53b6d3e 54467abc55f8 64c00ef5e098 2024-09-11T23:17:52.422 INFO:teuthology.orchestra.run.smithi026.stdout:mds.g smithi026 running (103s) 21s ago 102s 19.4M - 19.1.1-298-gb53b6d3e 54467abc55f8 79a2897e16ef 2024-09-11T23:17:52.422 INFO:teuthology.orchestra.run.smithi026.stdout:mds.h smithi076 running (109s) 29s ago 108s 18.1M - 19.1.1-298-gb53b6d3e 54467abc55f8 359eac0589c2 2024-09-11T23:17:52.422 INFO:teuthology.orchestra.run.smithi026.stdout:mds.i smithi136 running (106s) 29s ago 106s 18.4M - 19.1.1-298-gb53b6d3e 54467abc55f8 796ec4a7873a 2024-09-11T23:17:52.422 INFO:teuthology.orchestra.run.smithi026.stdout:mds.j smithi026 running (2m) 21s ago 2m 13.7M - 19.1.1-298-gb53b6d3e 54467abc55f8 f4bc32015a30 2024-09-11T23:17:52.422 INFO:teuthology.orchestra.run.smithi026.stdout:mds.k smithi076 running (2m) 29s ago 2m 16.8M - 19.1.1-298-gb53b6d3e 54467abc55f8 0a8b34884470 2024-09-11T23:17:52.423 INFO:teuthology.orchestra.run.smithi026.stdout:mds.l smithi136 running (2m) 29s ago 2m 16.6M - 19.1.1-298-gb53b6d3e 54467abc55f8 959a71ce7433 2024-09-11T23:17:52.423 INFO:teuthology.orchestra.run.smithi026.stdout:mgr.x smithi026 *:9283,8765 running (9m) 21s ago 8m 509M - 19.1.1-298-gb53b6d3e 54467abc55f8 865f5a94615b 2024-09-11T23:17:52.423 INFO:teuthology.orchestra.run.smithi026.stdout:mgr.y smithi076 *:8443,8765 running (7m) 29s ago 7m 441M - 19.1.1-298-gb53b6d3e 54467abc55f8 39fb8e0d330c 2024-09-11T23:17:52.423 INFO:teuthology.orchestra.run.smithi026.stdout:mgr.z smithi136 *:8443,8765 running (7m) 29s ago 7m 442M - 19.1.1-298-gb53b6d3e 54467abc55f8 b42acef488ca 2024-09-11T23:17:52.423 INFO:teuthology.orchestra.run.smithi026.stdout:mon.a smithi026 running (9m) 21s ago 9m 51.6M 2048M 19.1.1-298-gb53b6d3e 54467abc55f8 fb7a9fd245e4 2024-09-11T23:17:52.423 INFO:teuthology.orchestra.run.smithi026.stdout:mon.b smithi076 running (7m) 29s ago 7m 44.3M 2048M 19.1.1-298-gb53b6d3e 54467abc55f8 6320275d55dc 2024-09-11T23:17:52.423 INFO:teuthology.orchestra.run.smithi026.stdout:mon.c smithi136 running (7m) 29s ago 7m 42.7M 2048M 19.1.1-298-gb53b6d3e 54467abc55f8 a454789c9141 2024-09-11T23:17:52.423 INFO:teuthology.orchestra.run.smithi026.stdout:osd.0 smithi026 running (7m) 21s ago 7m 49.2M 4273M 19.1.1-298-gb53b6d3e 54467abc55f8 4a24d5ec0f7e 2024-09-11T23:17:52.423 INFO:teuthology.orchestra.run.smithi026.stdout:osd.1 smithi076 running (6m) 29s ago 6m 48.8M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 4a73508cf27b 2024-09-11T23:17:52.423 INFO:teuthology.orchestra.run.smithi026.stdout:osd.2 smithi136 running (6m) 29s ago 6m 57.0M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 4e53b54442b8 2024-09-11T23:17:52.423 INFO:teuthology.orchestra.run.smithi026.stdout:osd.3 smithi026 running (5m) 21s ago 5m 53.8M 4273M 19.1.1-298-gb53b6d3e 54467abc55f8 0ac22a9bc215 2024-09-11T23:17:52.423 INFO:teuthology.orchestra.run.smithi026.stdout:osd.4 smithi076 running (5m) 29s ago 5m 52.7M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 c29747542f6b 2024-09-11T23:17:52.423 INFO:teuthology.orchestra.run.smithi026.stdout:osd.5 smithi136 running (4m) 29s ago 4m 47.2M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 da022b14d4c2 2024-09-11T23:17:52.423 INFO:teuthology.orchestra.run.smithi026.stdout:osd.6 smithi026 running (4m) 21s ago 4m 50.2M 4273M 19.1.1-298-gb53b6d3e 54467abc55f8 b9b1c5a26366 2024-09-11T23:17:52.424 INFO:teuthology.orchestra.run.smithi026.stdout:osd.7 smithi076 running (4m) 29s ago 4m 46.2M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 c69a833bbf62 2024-09-11T23:17:52.424 INFO:teuthology.orchestra.run.smithi026.stdout:osd.8 smithi136 running (3m) 29s ago 3m 48.6M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 6dbd11d47ec3 2024-09-11T23:17:52.424 INFO:teuthology.orchestra.run.smithi026.stdout:osd.9 smithi026 running (3m) 21s ago 3m 47.0M 4273M 19.1.1-298-gb53b6d3e 54467abc55f8 1749c992d30d 2024-09-11T23:17:52.424 INFO:teuthology.orchestra.run.smithi026.stdout:osd.10 smithi076 running (2m) 29s ago 2m 46.9M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 1f4f299bfbb2 2024-09-11T23:17:52.424 INFO:teuthology.orchestra.run.smithi026.stdout:osd.11 smithi136 running (2m) 29s ago 2m 50.2M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 fd25674f12e3 2024-09-11T23:17:52.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:52.424+0000 7fbc4507e640 1 -- 172.21.15.26:0/3253888124 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fbc280a2c40 msgr2=0x7fbc280a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:52.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:52.424+0000 7fbc4507e640 1 --2- 172.21.15.26:0/3253888124 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fbc280a2c40 0x7fbc280a5100 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fbc30005d00 tx=0x7fbc3000b410 comp rx=0 tx=0).stop 2024-09-11T23:17:52.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:52.424+0000 7fbc4507e640 1 -- 172.21.15.26:0/3253888124 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbc40104fd0 msgr2=0x7fbc40198380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:52.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:52.424+0000 7fbc4507e640 1 --2- 172.21.15.26:0/3253888124 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbc40104fd0 0x7fbc40198380 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fbc3400c970 tx=0x7fbc3400ce30 comp rx=0 tx=0).stop 2024-09-11T23:17:52.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:52.424+0000 7fbc4507e640 1 -- 172.21.15.26:0/3253888124 shutdown_connections 2024-09-11T23:17:52.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:52.424+0000 7fbc4507e640 1 --2- 172.21.15.26:0/3253888124 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fbc280a2c40 0x7fbc280a5100 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:52.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:52.424+0000 7fbc4507e640 1 --2- 172.21.15.26:0/3253888124 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbc40107900 0x7fbc4019f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:52.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:52.424+0000 7fbc4507e640 1 --2- 172.21.15.26:0/3253888124 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbc40104fd0 0x7fbc40198380 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:52.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:52.424+0000 7fbc4507e640 1 --2- 172.21.15.26:0/3253888124 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbc401026a0 0x7fbc40197e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:52.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:52.424+0000 7fbc4507e640 1 -- 172.21.15.26:0/3253888124 >> 172.21.15.26:0/3253888124 conn(0x7fbc400fc2a0 msgr2=0x7fbc40102f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:17:52.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:52.425+0000 7fbc4507e640 1 -- 172.21.15.26:0/3253888124 shutdown_connections 2024-09-11T23:17:52.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:52.425+0000 7fbc4507e640 1 -- 172.21.15.26:0/3253888124 wait complete. 2024-09-11T23:17:53.059 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- bash -c 'ceph orch ls' 2024-09-11T23:17:53.316 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:17:53.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:53.756+0000 7f10c0ad7640 1 -- 172.21.15.26:0/3623384409 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f10bc101cd0 msgr2=0x7f10bc102150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:53.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:53.756+0000 7f10c0ad7640 1 --2- 172.21.15.26:0/3623384409 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f10bc101cd0 0x7f10bc102150 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f10b0009900 tx=0x7f10b002a410 comp rx=0 tx=0).stop 2024-09-11T23:17:53.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:53.757+0000 7f10c0ad7640 1 -- 172.21.15.26:0/3623384409 shutdown_connections 2024-09-11T23:17:53.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:53.757+0000 7f10c0ad7640 1 --2- 172.21.15.26:0/3623384409 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f10bc102690 0x7f10bc108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:53.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:53.757+0000 7f10c0ad7640 1 --2- 172.21.15.26:0/3623384409 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f10bc101cd0 0x7f10bc102150 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:53.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:53.757+0000 7f10c0ad7640 1 --2- 172.21.15.26:0/3623384409 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f10bc100ad0 0x7f10bc100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:53.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:53.757+0000 7f10c0ad7640 1 -- 172.21.15.26:0/3623384409 >> 172.21.15.26:0/3623384409 conn(0x7f10bc0fc280 msgr2=0x7f10bc0fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:17:53.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:53.757+0000 7f10c0ad7640 1 -- 172.21.15.26:0/3623384409 shutdown_connections 2024-09-11T23:17:53.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:53.757+0000 7f10c0ad7640 1 -- 172.21.15.26:0/3623384409 wait complete. 2024-09-11T23:17:53.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:53.757+0000 7f10c0ad7640 1 Processor -- start 2024-09-11T23:17:53.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:53.757+0000 7f10c0ad7640 1 -- start start 2024-09-11T23:17:53.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:53.758+0000 7f10c0ad7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f10bc100ad0 0x7f10bc06aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:53.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:53.758+0000 7f10c0ad7640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f10bc101cd0 0x7f10bc19cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:53.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:53.758+0000 7f10c0ad7640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f10bc102690 0x7f10bc19f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:53.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:53.758+0000 7f10c0ad7640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f10bc10c7d0 con 0x7f10bc100ad0 2024-09-11T23:17:53.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:53.758+0000 7f10c0ad7640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f10bc10c650 con 0x7f10bc102690 2024-09-11T23:17:53.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:53.758+0000 7f10c0ad7640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f10bc10c950 con 0x7f10bc101cd0 2024-09-11T23:17:53.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:53.758+0000 7f10ba59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f10bc100ad0 0x7f10bc06aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:53.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:53.758+0000 7f10ba59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f10bc100ad0 0x7f10bc06aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59506/0 (socket says 172.21.15.26:59506) 2024-09-11T23:17:53.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:53.758+0000 7f10ba59c640 1 -- 172.21.15.26:0/1142652626 learned_addr learned my addr 172.21.15.26:0/1142652626 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:17:53.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:53.758+0000 7f10b9d9b640 1 --2- 172.21.15.26:0/1142652626 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f10bc101cd0 0x7f10bc19cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:53.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:53.758+0000 7f10ba59c640 1 -- 172.21.15.26:0/1142652626 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f10bc101cd0 msgr2=0x7f10bc19cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:53.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:53.758+0000 7f10ba59c640 1 --2- 172.21.15.26:0/1142652626 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f10bc101cd0 0x7f10bc19cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:53.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:53.758+0000 7f10bad9d640 1 --2- 172.21.15.26:0/1142652626 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f10bc102690 0x7f10bc19f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:53.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:53.758+0000 7f10ba59c640 1 -- 172.21.15.26:0/1142652626 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f10bc102690 msgr2=0x7f10bc19f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:53.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:53.758+0000 7f10ba59c640 1 --2- 172.21.15.26:0/1142652626 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f10bc102690 0x7f10bc19f320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:53.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:53.758+0000 7f10ba59c640 1 -- 172.21.15.26:0/1142652626 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10bc19fc30 con 0x7f10bc100ad0 2024-09-11T23:17:53.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:53.758+0000 7f10ba59c640 1 --2- 172.21.15.26:0/1142652626 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f10bc100ad0 0x7f10bc06aad0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f10ac00b450 tx=0x7f10ac00b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:17:53.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:53.758+0000 7f10ab7fe640 1 -- 172.21.15.26:0/1142652626 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10ac015020 con 0x7f10bc100ad0 2024-09-11T23:17:53.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:53.758+0000 7f10ab7fe640 1 -- 172.21.15.26:0/1142652626 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f10ac011d70 con 0x7f10bc100ad0 2024-09-11T23:17:53.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:53.758+0000 7f10c0ad7640 1 -- 172.21.15.26:0/1142652626 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10bc19fec0 con 0x7f10bc100ad0 2024-09-11T23:17:53.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:53.759+0000 7f10ab7fe640 1 -- 172.21.15.26:0/1142652626 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10ac00f340 con 0x7f10bc100ad0 2024-09-11T23:17:53.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:53.759+0000 7f10c0ad7640 1 -- 172.21.15.26:0/1142652626 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10bc1a03f0 con 0x7f10bc100ad0 2024-09-11T23:17:53.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:53.759+0000 7f10c0ad7640 1 -- 172.21.15.26:0/1142652626 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10bc107ec0 con 0x7f10bc100ad0 2024-09-11T23:17:53.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:53.761+0000 7f10ab7fe640 1 -- 172.21.15.26:0/1142652626 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f10ac018020 con 0x7f10bc100ad0 2024-09-11T23:17:53.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:53.761+0000 7f10ab7fe640 1 --2- 172.21.15.26:0/1142652626 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f10a40a2c10 0x7f10a40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:53.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:53.761+0000 7f10b9d9b640 1 --2- 172.21.15.26:0/1142652626 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f10a40a2c10 0x7f10a40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:53.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:53.761+0000 7f10ab7fe640 1 -- 172.21.15.26:0/1142652626 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f10ac0cf0e0 con 0x7f10bc100ad0 2024-09-11T23:17:53.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:53.762+0000 7f10b9d9b640 1 --2- 172.21.15.26:0/1142652626 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f10a40a2c10 0x7f10a40a50d0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f10b0006010 tx=0x7f10b0000bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:17:53.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:53.765+0000 7f10ab7fe640 1 -- 172.21.15.26:0/1142652626 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f10ac01f020 con 0x7f10bc100ad0 2024-09-11T23:17:53.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:53.955+0000 7f10c0ad7640 1 -- 172.21.15.26:0/1142652626 --> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f10bc062630 con 0x7f10a40a2c10 2024-09-11T23:17:53.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:53.961+0000 7f10ab7fe640 1 -- 172.21.15.26:0/1142652626 <== mgr.14148 v2:172.21.15.26:6800/110833859 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f10bc062630 con 0x7f10a40a2c10 2024-09-11T23:17:53.961 INFO:teuthology.orchestra.run.smithi026.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-09-11T23:17:53.961 INFO:teuthology.orchestra.run.smithi026.stdout:mds.all 12/12 31s ago 2m smithi026=a;smithi026=d;smithi026=g;smithi026=j;smithi076=b;smithi076=e;smithi076=h;smithi076=k;smithi136=c;smithi136=f;smithi136=i;smithi136=l;count:12 2024-09-11T23:17:53.961 INFO:teuthology.orchestra.run.smithi026.stdout:mgr 3/3 31s ago 7m smithi026=x;smithi076=y;smithi136=z;count:3 2024-09-11T23:17:53.961 INFO:teuthology.orchestra.run.smithi026.stdout:mon 3/3 31s ago 7m smithi026:172.21.15.26=a;smithi076:172.21.15.76=b;smithi136:172.21.15.136=c;count:3 2024-09-11T23:17:53.962 INFO:teuthology.orchestra.run.smithi026.stdout:osd 12 31s ago - 2024-09-11T23:17:53.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:53.963+0000 7f10c0ad7640 1 -- 172.21.15.26:0/1142652626 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f10a40a2c10 msgr2=0x7f10a40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:53.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:53.963+0000 7f10c0ad7640 1 --2- 172.21.15.26:0/1142652626 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f10a40a2c10 0x7f10a40a50d0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f10b0006010 tx=0x7f10b0000bc0 comp rx=0 tx=0).stop 2024-09-11T23:17:53.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:53.963+0000 7f10c0ad7640 1 -- 172.21.15.26:0/1142652626 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f10bc100ad0 msgr2=0x7f10bc06aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:53.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:53.963+0000 7f10c0ad7640 1 --2- 172.21.15.26:0/1142652626 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f10bc100ad0 0x7f10bc06aad0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f10ac00b450 tx=0x7f10ac00b910 comp rx=0 tx=0).stop 2024-09-11T23:17:53.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:53.963+0000 7f10c0ad7640 1 -- 172.21.15.26:0/1142652626 shutdown_connections 2024-09-11T23:17:53.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:53.963+0000 7f10c0ad7640 1 --2- 172.21.15.26:0/1142652626 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f10a40a2c10 0x7f10a40a50d0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:53.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:53.963+0000 7f10c0ad7640 1 --2- 172.21.15.26:0/1142652626 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f10bc102690 0x7f10bc19f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:53.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:53.963+0000 7f10c0ad7640 1 --2- 172.21.15.26:0/1142652626 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f10bc101cd0 0x7f10bc19cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:53.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:53.963+0000 7f10c0ad7640 1 --2- 172.21.15.26:0/1142652626 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f10bc100ad0 0x7f10bc06aad0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:53.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:53.963+0000 7f10c0ad7640 1 -- 172.21.15.26:0/1142652626 >> 172.21.15.26:0/1142652626 conn(0x7f10bc0fc280 msgr2=0x7f10bc105090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:17:53.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:53.963+0000 7f10c0ad7640 1 -- 172.21.15.26:0/1142652626 shutdown_connections 2024-09-11T23:17:53.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:53.963+0000 7f10c0ad7640 1 -- 172.21.15.26:0/1142652626 wait complete. 2024-09-11T23:17:54.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:17:53 smithi136 ceph-mon[35121]: pgmap v324: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-11T23:17:54.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:17:53 smithi136 ceph-mon[35121]: from='client.15009 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:17:54.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:17:53 smithi026 ceph-mon[31172]: pgmap v324: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-11T23:17:54.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:17:53 smithi026 ceph-mon[31172]: from='client.15009 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:17:54.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:17:53 smithi076 ceph-mon[35296]: pgmap v324: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-11T23:17:54.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:17:53 smithi076 ceph-mon[35296]: from='client.15009 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:17:54.616 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- bash -c 'ceph orch host ls' 2024-09-11T23:17:54.874 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:17:55.130 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:17:54 smithi026 ceph-mon[31172]: from='client.15015 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:17:55.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:17:54 smithi136 ceph-mon[35121]: from='client.15015 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:17:55.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:17:54 smithi076 ceph-mon[35296]: from='client.15015 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:17:55.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:55.358+0000 7fee69324640 1 -- 172.21.15.26:0/3811537058 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee64100b10 msgr2=0x7fee64100f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:55.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:55.358+0000 7fee69324640 1 --2- 172.21.15.26:0/3811537058 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee64100b10 0x7fee64100f10 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fee54009900 tx=0x7fee5402a410 comp rx=0 tx=0).stop 2024-09-11T23:17:55.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:55.358+0000 7fee69324640 1 -- 172.21.15.26:0/3811537058 shutdown_connections 2024-09-11T23:17:55.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:55.358+0000 7fee69324640 1 --2- 172.21.15.26:0/3811537058 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fee641026d0 0x7fee64108f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:55.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:55.358+0000 7fee69324640 1 --2- 172.21.15.26:0/3811537058 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fee64101d10 0x7fee64102190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:55.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:55.358+0000 7fee69324640 1 --2- 172.21.15.26:0/3811537058 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee64100b10 0x7fee64100f10 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:55.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:55.358+0000 7fee69324640 1 -- 172.21.15.26:0/3811537058 >> 172.21.15.26:0/3811537058 conn(0x7fee640fc2a0 msgr2=0x7fee640fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:17:55.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:55.358+0000 7fee69324640 1 -- 172.21.15.26:0/3811537058 shutdown_connections 2024-09-11T23:17:55.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:55.358+0000 7fee69324640 1 -- 172.21.15.26:0/3811537058 wait complete. 2024-09-11T23:17:55.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:55.358+0000 7fee69324640 1 Processor -- start 2024-09-11T23:17:55.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:55.359+0000 7fee69324640 1 -- start start 2024-09-11T23:17:55.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:55.359+0000 7fee69324640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fee64100b10 0x7fee64197fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:55.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:55.359+0000 7fee69324640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee64101d10 0x7fee641984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:55.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:55.359+0000 7fee69324640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fee641026d0 0x7fee6419f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:55.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:55.359+0000 7fee69324640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fee6410d920 con 0x7fee64101d10 2024-09-11T23:17:55.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:55.359+0000 7fee69324640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fee6410d7a0 con 0x7fee64100b10 2024-09-11T23:17:55.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:55.359+0000 7fee69324640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fee6410daa0 con 0x7fee641026d0 2024-09-11T23:17:55.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:55.359+0000 7fee627fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee64101d10 0x7fee641984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:55.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:55.359+0000 7fee627fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee64101d10 0x7fee641984f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59522/0 (socket says 172.21.15.26:59522) 2024-09-11T23:17:55.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:55.359+0000 7fee627fc640 1 -- 172.21.15.26:0/3128326415 learned_addr learned my addr 172.21.15.26:0/3128326415 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:17:55.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:55.359+0000 7fee62ffd640 1 --2- 172.21.15.26:0/3128326415 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fee64100b10 0x7fee64197fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:55.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:55.359+0000 7fee637fe640 1 --2- 172.21.15.26:0/3128326415 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fee641026d0 0x7fee6419f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:55.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:55.359+0000 7fee627fc640 1 -- 172.21.15.26:0/3128326415 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fee641026d0 msgr2=0x7fee6419f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:55.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:55.359+0000 7fee627fc640 1 --2- 172.21.15.26:0/3128326415 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fee641026d0 0x7fee6419f570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:55.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:55.359+0000 7fee627fc640 1 -- 172.21.15.26:0/3128326415 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fee64100b10 msgr2=0x7fee64197fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:55.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:55.359+0000 7fee627fc640 1 --2- 172.21.15.26:0/3128326415 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fee64100b10 0x7fee64197fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:55.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:55.359+0000 7fee627fc640 1 -- 172.21.15.26:0/3128326415 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee6419fbe0 con 0x7fee64101d10 2024-09-11T23:17:55.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:55.359+0000 7fee637fe640 1 --2- 172.21.15.26:0/3128326415 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fee641026d0 0x7fee6419f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:17:55.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:55.359+0000 7fee627fc640 1 --2- 172.21.15.26:0/3128326415 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee64101d10 0x7fee641984f0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fee5800c970 tx=0x7fee5800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:17:55.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:55.360+0000 7fee4bfff640 1 -- 172.21.15.26:0/3128326415 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee58007bb0 con 0x7fee64101d10 2024-09-11T23:17:55.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:55.360+0000 7fee4bfff640 1 -- 172.21.15.26:0/3128326415 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee58007d50 con 0x7fee64101d10 2024-09-11T23:17:55.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:55.360+0000 7fee69324640 1 -- 172.21.15.26:0/3128326415 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee6419fed0 con 0x7fee64101d10 2024-09-11T23:17:55.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:55.360+0000 7fee4bfff640 1 -- 172.21.15.26:0/3128326415 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee5800f450 con 0x7fee64101d10 2024-09-11T23:17:55.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:55.360+0000 7fee69324640 1 -- 172.21.15.26:0/3128326415 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee641a03b0 con 0x7fee64101d10 2024-09-11T23:17:55.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:55.361+0000 7fee69324640 1 -- 172.21.15.26:0/3128326415 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee64107f60 con 0x7fee64101d10 2024-09-11T23:17:55.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:55.362+0000 7fee4bfff640 1 -- 172.21.15.26:0/3128326415 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fee58004fe0 con 0x7fee64101d10 2024-09-11T23:17:55.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:55.362+0000 7fee4bfff640 1 --2- 172.21.15.26:0/3128326415 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fee4c0a2c40 0x7fee4c0a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:55.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:55.363+0000 7fee62ffd640 1 --2- 172.21.15.26:0/3128326415 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fee4c0a2c40 0x7fee4c0a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:55.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:55.363+0000 7fee4bfff640 1 -- 172.21.15.26:0/3128326415 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fee580cef60 con 0x7fee64101d10 2024-09-11T23:17:55.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:55.363+0000 7fee62ffd640 1 --2- 172.21.15.26:0/3128326415 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fee4c0a2c40 0x7fee4c0a5100 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fee54005ab0 tx=0x7fee54000bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:17:55.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:55.367+0000 7fee4bfff640 1 -- 172.21.15.26:0/3128326415 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fee58014030 con 0x7fee64101d10 2024-09-11T23:17:55.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:55.546+0000 7fee69324640 1 -- 172.21.15.26:0/3128326415 --> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7fee64062630 con 0x7fee4c0a2c40 2024-09-11T23:17:55.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:55.547+0000 7fee4bfff640 1 -- 172.21.15.26:0/3128326415 <== mgr.14148 v2:172.21.15.26:6800/110833859 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7fee64062630 con 0x7fee4c0a2c40 2024-09-11T23:17:55.547 INFO:teuthology.orchestra.run.smithi026.stdout:HOST ADDR LABELS STATUS 2024-09-11T23:17:55.547 INFO:teuthology.orchestra.run.smithi026.stdout:smithi026 172.21.15.26 2024-09-11T23:17:55.547 INFO:teuthology.orchestra.run.smithi026.stdout:smithi076 172.21.15.76 2024-09-11T23:17:55.548 INFO:teuthology.orchestra.run.smithi026.stdout:smithi136 172.21.15.136 2024-09-11T23:17:55.548 INFO:teuthology.orchestra.run.smithi026.stdout:3 hosts in cluster 2024-09-11T23:17:55.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:55.549+0000 7fee69324640 1 -- 172.21.15.26:0/3128326415 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fee4c0a2c40 msgr2=0x7fee4c0a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:55.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:55.549+0000 7fee69324640 1 --2- 172.21.15.26:0/3128326415 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fee4c0a2c40 0x7fee4c0a5100 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fee54005ab0 tx=0x7fee54000bc0 comp rx=0 tx=0).stop 2024-09-11T23:17:55.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:55.549+0000 7fee69324640 1 -- 172.21.15.26:0/3128326415 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee64101d10 msgr2=0x7fee641984f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:55.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:55.549+0000 7fee69324640 1 --2- 172.21.15.26:0/3128326415 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee64101d10 0x7fee641984f0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fee5800c970 tx=0x7fee5800ce30 comp rx=0 tx=0).stop 2024-09-11T23:17:55.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:55.549+0000 7fee69324640 1 -- 172.21.15.26:0/3128326415 shutdown_connections 2024-09-11T23:17:55.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:55.549+0000 7fee69324640 1 --2- 172.21.15.26:0/3128326415 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fee4c0a2c40 0x7fee4c0a5100 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:55.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:55.549+0000 7fee69324640 1 --2- 172.21.15.26:0/3128326415 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fee641026d0 0x7fee6419f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:55.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:55.549+0000 7fee69324640 1 --2- 172.21.15.26:0/3128326415 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee64101d10 0x7fee641984f0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:55.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:55.549+0000 7fee69324640 1 --2- 172.21.15.26:0/3128326415 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fee64100b10 0x7fee64197fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:55.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:55.549+0000 7fee69324640 1 -- 172.21.15.26:0/3128326415 >> 172.21.15.26:0/3128326415 conn(0x7fee640fc2a0 msgr2=0x7fee64105130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:17:55.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:55.549+0000 7fee69324640 1 -- 172.21.15.26:0/3128326415 shutdown_connections 2024-09-11T23:17:55.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:55.549+0000 7fee69324640 1 -- 172.21.15.26:0/3128326415 wait complete. 2024-09-11T23:17:56.203 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:17:55 smithi026 ceph-mon[31172]: pgmap v325: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-11T23:17:56.231 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- bash -c 'ceph orch device ls' 2024-09-11T23:17:56.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:17:55 smithi136 ceph-mon[35121]: pgmap v325: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-11T23:17:56.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:17:55 smithi076 ceph-mon[35296]: pgmap v325: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-11T23:17:56.487 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:17:56.914 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:17:56 smithi026 ceph-mon[31172]: from='client.15021 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:17:56.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:56.914+0000 7f55ab6f8640 1 -- 172.21.15.26:0/2323501138 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f55a4101d10 msgr2=0x7f55a4102190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:56.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:56.914+0000 7f55ab6f8640 1 --2- 172.21.15.26:0/2323501138 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f55a4101d10 0x7f55a4102190 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f5598009900 tx=0x7f559802a410 comp rx=0 tx=0).stop 2024-09-11T23:17:56.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:56.914+0000 7f55ab6f8640 1 -- 172.21.15.26:0/2323501138 shutdown_connections 2024-09-11T23:17:56.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:56.914+0000 7f55ab6f8640 1 --2- 172.21.15.26:0/2323501138 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f55a41026d0 0x7f55a4108f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:56.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:56.914+0000 7f55ab6f8640 1 --2- 172.21.15.26:0/2323501138 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f55a4101d10 0x7f55a4102190 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:56.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:56.914+0000 7f55ab6f8640 1 --2- 172.21.15.26:0/2323501138 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f55a4100b10 0x7f55a4100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:56.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:56.914+0000 7f55ab6f8640 1 -- 172.21.15.26:0/2323501138 >> 172.21.15.26:0/2323501138 conn(0x7f55a40fc2a0 msgr2=0x7f55a40fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:17:56.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:56.914+0000 7f55ab6f8640 1 -- 172.21.15.26:0/2323501138 shutdown_connections 2024-09-11T23:17:56.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:56.914+0000 7f55ab6f8640 1 -- 172.21.15.26:0/2323501138 wait complete. 2024-09-11T23:17:56.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:56.915+0000 7f55ab6f8640 1 Processor -- start 2024-09-11T23:17:56.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:56.915+0000 7f55ab6f8640 1 -- start start 2024-09-11T23:17:56.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:56.915+0000 7f55ab6f8640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f55a4100b10 0x7f55a406b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:56.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:56.915+0000 7f55ab6f8640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f55a4101d10 0x7f55a419cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:56.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:56.915+0000 7f55ab6f8640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f55a41026d0 0x7f55a419f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:56.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:56.915+0000 7f55ab6f8640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f55a410c810 con 0x7f55a4100b10 2024-09-11T23:17:56.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:56.915+0000 7f55ab6f8640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f55a410c690 con 0x7f55a41026d0 2024-09-11T23:17:56.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:56.915+0000 7f55ab6f8640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f55a410c990 con 0x7f55a4101d10 2024-09-11T23:17:56.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:56.915+0000 7f55a9494640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f55a4100b10 0x7f55a406b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:56.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:56.915+0000 7f55a9494640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f55a4100b10 0x7f55a406b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59546/0 (socket says 172.21.15.26:59546) 2024-09-11T23:17:56.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:56.915+0000 7f55a9494640 1 -- 172.21.15.26:0/1572388385 learned_addr learned my addr 172.21.15.26:0/1572388385 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:17:56.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:56.915+0000 7f55a9c95640 1 --2- 172.21.15.26:0/1572388385 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f55a41026d0 0x7f55a419f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:56.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:56.915+0000 7f55a9494640 1 -- 172.21.15.26:0/1572388385 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f55a4101d10 msgr2=0x7f55a419cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:56.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:56.915+0000 7f55a9494640 1 --2- 172.21.15.26:0/1572388385 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f55a4101d10 0x7f55a419cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:56.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:56.915+0000 7f55a9494640 1 -- 172.21.15.26:0/1572388385 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f55a41026d0 msgr2=0x7f55a419f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:56.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:56.915+0000 7f55a9494640 1 --2- 172.21.15.26:0/1572388385 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f55a41026d0 0x7f55a419f350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:56.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:56.915+0000 7f55a9494640 1 -- 172.21.15.26:0/1572388385 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55a419fc60 con 0x7f55a4100b10 2024-09-11T23:17:56.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:56.915+0000 7f55a9c95640 1 --2- 172.21.15.26:0/1572388385 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f55a41026d0 0x7f55a419f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:17:56.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:56.916+0000 7f55a9494640 1 --2- 172.21.15.26:0/1572388385 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f55a4100b10 0x7f55a406b2f0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f55a000b450 tx=0x7f55a000b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:17:56.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:56.916+0000 7f55967fc640 1 -- 172.21.15.26:0/1572388385 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55a0015020 con 0x7f55a4100b10 2024-09-11T23:17:56.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:56.916+0000 7f55967fc640 1 -- 172.21.15.26:0/1572388385 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55a0011d70 con 0x7f55a4100b10 2024-09-11T23:17:56.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:56.916+0000 7f55967fc640 1 -- 172.21.15.26:0/1572388385 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55a000f340 con 0x7f55a4100b10 2024-09-11T23:17:56.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:56.916+0000 7f55ab6f8640 1 -- 172.21.15.26:0/1572388385 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55a419fef0 con 0x7f55a4100b10 2024-09-11T23:17:56.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:56.916+0000 7f55ab6f8640 1 -- 172.21.15.26:0/1572388385 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55a41a03a0 con 0x7f55a4100b10 2024-09-11T23:17:56.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:56.917+0000 7f55ab6f8640 1 -- 172.21.15.26:0/1572388385 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55a4107f80 con 0x7f55a4100b10 2024-09-11T23:17:56.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:56.918+0000 7f55967fc640 1 -- 172.21.15.26:0/1572388385 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f55a0018020 con 0x7f55a4100b10 2024-09-11T23:17:56.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:56.919+0000 7f55967fc640 1 --2- 172.21.15.26:0/1572388385 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f55900a2c90 0x7f55900a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:56.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:56.919+0000 7f55a8c93640 1 --2- 172.21.15.26:0/1572388385 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f55900a2c90 0x7f55900a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:56.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:56.919+0000 7f55967fc640 1 -- 172.21.15.26:0/1572388385 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f55a00cec00 con 0x7f55a4100b10 2024-09-11T23:17:56.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:56.919+0000 7f55a8c93640 1 --2- 172.21.15.26:0/1572388385 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f55900a2c90 0x7f55900a5150 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f55980096c0 tx=0x7f5598031400 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:17:56.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:56.923+0000 7f55967fc640 1 -- 172.21.15.26:0/1572388385 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f55a001f020 con 0x7f55a4100b10 2024-09-11T23:17:57.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:17:56 smithi076 ceph-mon[35296]: from='client.15021 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:17:57.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:57.108+0000 7f55ab6f8640 1 -- 172.21.15.26:0/1572388385 --> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7f55a4062630 con 0x7f55900a2c90 2024-09-11T23:17:57.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:57.109+0000 7f55967fc640 1 -- 172.21.15.26:0/1572388385 <== mgr.14148 v2:172.21.15.26:6800/110833859 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7f55a4062630 con 0x7f55900a2c90 2024-09-11T23:17:57.109 INFO:teuthology.orchestra.run.smithi026.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-09-11T23:17:57.109 INFO:teuthology.orchestra.run.smithi026.stdout:smithi026 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB74760020480DGN 447G No 3m ago Has a FileSystem, LVM detected 2024-09-11T23:17:57.109 INFO:teuthology.orchestra.run.smithi026.stdout:smithi076 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400U4400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-09-11T23:17:57.110 INFO:teuthology.orchestra.run.smithi026.stdout:smithi136 /dev/nvme0n1 ssd INTEL_SSDPEDMX020T4_CVPD514100122P0SGN 1863G No 2m ago Has a FileSystem, LVM detected 2024-09-11T23:17:57.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:57.111+0000 7f55ab6f8640 1 -- 172.21.15.26:0/1572388385 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f55900a2c90 msgr2=0x7f55900a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:57.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:57.111+0000 7f55ab6f8640 1 --2- 172.21.15.26:0/1572388385 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f55900a2c90 0x7f55900a5150 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f55980096c0 tx=0x7f5598031400 comp rx=0 tx=0).stop 2024-09-11T23:17:57.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:57.111+0000 7f55ab6f8640 1 -- 172.21.15.26:0/1572388385 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f55a4100b10 msgr2=0x7f55a406b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:57.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:57.111+0000 7f55ab6f8640 1 --2- 172.21.15.26:0/1572388385 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f55a4100b10 0x7f55a406b2f0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f55a000b450 tx=0x7f55a000b910 comp rx=0 tx=0).stop 2024-09-11T23:17:57.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:57.111+0000 7f55ab6f8640 1 -- 172.21.15.26:0/1572388385 shutdown_connections 2024-09-11T23:17:57.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:57.111+0000 7f55ab6f8640 1 --2- 172.21.15.26:0/1572388385 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f55900a2c90 0x7f55900a5150 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:57.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:57.111+0000 7f55ab6f8640 1 --2- 172.21.15.26:0/1572388385 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f55a41026d0 0x7f55a419f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:57.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:57.111+0000 7f55ab6f8640 1 --2- 172.21.15.26:0/1572388385 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f55a4101d10 0x7f55a419cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:57.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:57.111+0000 7f55ab6f8640 1 --2- 172.21.15.26:0/1572388385 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f55a4100b10 0x7f55a406b2f0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:57.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:57.111+0000 7f55ab6f8640 1 -- 172.21.15.26:0/1572388385 >> 172.21.15.26:0/1572388385 conn(0x7f55a40fc2a0 msgr2=0x7f55a4105150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:17:57.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:57.112+0000 7f55ab6f8640 1 -- 172.21.15.26:0/1572388385 shutdown_connections 2024-09-11T23:17:57.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:57.112+0000 7f55ab6f8640 1 -- 172.21.15.26:0/1572388385 wait complete. 2024-09-11T23:17:57.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:17:56 smithi136 ceph-mon[35121]: from='client.15021 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:17:57.810 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-11T23:17:57.819 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi026.front.sepia.ceph.com 2024-09-11T23:17:57.819 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- bash -c 'ceph fs dump' 2024-09-11T23:17:58.081 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:17:58.203 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:17:57 smithi026 ceph-mon[31172]: pgmap v326: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-11T23:17:58.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:17:57 smithi136 ceph-mon[35121]: pgmap v326: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-11T23:17:58.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:17:57 smithi076 ceph-mon[35296]: pgmap v326: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-11T23:17:58.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:58.511+0000 7fee5a3a2640 1 -- 172.21.15.26:0/285226564 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee54101a40 msgr2=0x7fee54101e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:58.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:58.511+0000 7fee5a3a2640 1 --2- 172.21.15.26:0/285226564 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee54101a40 0x7fee54101e40 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fee50009900 tx=0x7fee5002a410 comp rx=0 tx=0).stop 2024-09-11T23:17:58.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:58.511+0000 7fee5a3a2640 1 -- 172.21.15.26:0/285226564 shutdown_connections 2024-09-11T23:17:58.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:58.511+0000 7fee5a3a2640 1 --2- 172.21.15.26:0/285226564 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fee54102c40 0x7fee541030c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:58.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:58.511+0000 7fee5a3a2640 1 --2- 172.21.15.26:0/285226564 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fee54103600 0x7fee54109e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:58.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:58.511+0000 7fee5a3a2640 1 --2- 172.21.15.26:0/285226564 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee54101a40 0x7fee54101e40 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:58.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:58.511+0000 7fee5a3a2640 1 -- 172.21.15.26:0/285226564 >> 172.21.15.26:0/285226564 conn(0x7fee540fd1f0 msgr2=0x7fee540ff610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:17:58.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:58.511+0000 7fee5a3a2640 1 -- 172.21.15.26:0/285226564 shutdown_connections 2024-09-11T23:17:58.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:58.511+0000 7fee5a3a2640 1 -- 172.21.15.26:0/285226564 wait complete. 2024-09-11T23:17:58.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:58.512+0000 7fee5a3a2640 1 Processor -- start 2024-09-11T23:17:58.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:58.512+0000 7fee5a3a2640 1 -- start start 2024-09-11T23:17:58.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:58.512+0000 7fee5a3a2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee54101a40 0x7fee54197da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:58.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:58.512+0000 7fee5a3a2640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fee54102c40 0x7fee541982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:58.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:58.512+0000 7fee5a3a2640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fee54103600 0x7fee5419f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:58.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:58.512+0000 7fee5a3a2640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fee5410d720 con 0x7fee54101a40 2024-09-11T23:17:58.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:58.512+0000 7fee5a3a2640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fee5410d5a0 con 0x7fee54103600 2024-09-11T23:17:58.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:58.512+0000 7fee5a3a2640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fee5410d8a0 con 0x7fee54102c40 2024-09-11T23:17:58.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:58.512+0000 7fee593a0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee54101a40 0x7fee54197da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:58.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:58.513+0000 7fee593a0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee54101a40 0x7fee54197da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59560/0 (socket says 172.21.15.26:59560) 2024-09-11T23:17:58.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:58.513+0000 7fee593a0640 1 -- 172.21.15.26:0/4026551369 learned_addr learned my addr 172.21.15.26:0/4026551369 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:17:58.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:58.513+0000 7fee58b9f640 1 --2- 172.21.15.26:0/4026551369 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fee54102c40 0x7fee541982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:58.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:58.513+0000 7fee59ba1640 1 --2- 172.21.15.26:0/4026551369 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fee54103600 0x7fee5419f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:58.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:58.513+0000 7fee593a0640 1 -- 172.21.15.26:0/4026551369 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fee54102c40 msgr2=0x7fee541982e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:58.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:58.513+0000 7fee593a0640 1 --2- 172.21.15.26:0/4026551369 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fee54102c40 0x7fee541982e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:58.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:58.513+0000 7fee593a0640 1 -- 172.21.15.26:0/4026551369 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fee54103600 msgr2=0x7fee5419f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:58.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:58.513+0000 7fee593a0640 1 --2- 172.21.15.26:0/4026551369 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fee54103600 0x7fee5419f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:58.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:58.513+0000 7fee593a0640 1 -- 172.21.15.26:0/4026551369 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee5419f9d0 con 0x7fee54101a40 2024-09-11T23:17:58.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:58.513+0000 7fee59ba1640 1 --2- 172.21.15.26:0/4026551369 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fee54103600 0x7fee5419f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:17:58.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:58.513+0000 7fee593a0640 1 --2- 172.21.15.26:0/4026551369 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee54101a40 0x7fee54197da0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fee50031610 tx=0x7fee500043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:17:58.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:58.513+0000 7fee4a7fc640 1 -- 172.21.15.26:0/4026551369 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee5003f030 con 0x7fee54101a40 2024-09-11T23:17:58.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:58.513+0000 7fee4a7fc640 1 -- 172.21.15.26:0/4026551369 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee50003ce0 con 0x7fee54101a40 2024-09-11T23:17:58.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:58.513+0000 7fee5a3a2640 1 -- 172.21.15.26:0/4026551369 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee5419fc60 con 0x7fee54101a40 2024-09-11T23:17:58.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:58.513+0000 7fee4a7fc640 1 -- 172.21.15.26:0/4026551369 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee50035070 con 0x7fee54101a40 2024-09-11T23:17:58.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:58.513+0000 7fee5a3a2640 1 -- 172.21.15.26:0/4026551369 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee541a0140 con 0x7fee54101a40 2024-09-11T23:17:58.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:58.514+0000 7fee5a3a2640 1 -- 172.21.15.26:0/4026551369 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee54108e10 con 0x7fee54101a40 2024-09-11T23:17:58.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:58.516+0000 7fee4a7fc640 1 -- 172.21.15.26:0/4026551369 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fee50042030 con 0x7fee54101a40 2024-09-11T23:17:58.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:58.516+0000 7fee4a7fc640 1 --2- 172.21.15.26:0/4026551369 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fee400a2c40 0x7fee400a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:58.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:58.516+0000 7fee58b9f640 1 --2- 172.21.15.26:0/4026551369 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fee400a2c40 0x7fee400a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:58.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:58.516+0000 7fee4a7fc640 1 -- 172.21.15.26:0/4026551369 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fee500ef980 con 0x7fee54101a40 2024-09-11T23:17:58.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:58.517+0000 7fee58b9f640 1 --2- 172.21.15.26:0/4026551369 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fee400a2c40 0x7fee400a5100 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fee541992c0 tx=0x7fee44009110 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:17:58.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:58.519+0000 7fee4a7fc640 1 -- 172.21.15.26:0/4026551369 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fee500bc780 con 0x7fee54101a40 2024-09-11T23:17:58.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:58.732+0000 7fee5a3a2640 1 -- 172.21.15.26:0/4026551369 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7fee5402d7c0 con 0x7fee54101a40 2024-09-11T23:17:58.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:58.733+0000 7fee4a7fc640 1 -- 172.21.15.26:0/4026551369 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 28 v28) ==== 76+0+3127 (secure 0 0 0) 0x7fee500bc080 con 0x7fee54101a40 2024-09-11T23:17:58.733 INFO:teuthology.orchestra.run.smithi026.stdout:e28 2024-09-11T23:17:58.733 INFO:teuthology.orchestra.run.smithi026.stdout:btime 2024-09-11T23:16:38:976987+0000 2024-09-11T23:17:58.733 INFO:teuthology.orchestra.run.smithi026.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-11T23:17:58.733 INFO:teuthology.orchestra.run.smithi026.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-11T23:17:58.733 INFO:teuthology.orchestra.run.smithi026.stdout:legacy client fscid: 1 2024-09-11T23:17:58.733 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-11T23:17:58.734 INFO:teuthology.orchestra.run.smithi026.stdout:Filesystem 'cephfs' (1) 2024-09-11T23:17:58.734 INFO:teuthology.orchestra.run.smithi026.stdout:fs_name cephfs 2024-09-11T23:17:58.734 INFO:teuthology.orchestra.run.smithi026.stdout:epoch 28 2024-09-11T23:17:58.734 INFO:teuthology.orchestra.run.smithi026.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-09-11T23:17:58.734 INFO:teuthology.orchestra.run.smithi026.stdout:created 2024-09-11T23:16:26.544203+0000 2024-09-11T23:17:58.734 INFO:teuthology.orchestra.run.smithi026.stdout:modified 2024-09-11T23:16:38.976970+0000 2024-09-11T23:17:58.734 INFO:teuthology.orchestra.run.smithi026.stdout:tableserver 0 2024-09-11T23:17:58.734 INFO:teuthology.orchestra.run.smithi026.stdout:root 0 2024-09-11T23:17:58.734 INFO:teuthology.orchestra.run.smithi026.stdout:session_timeout 300 2024-09-11T23:17:58.734 INFO:teuthology.orchestra.run.smithi026.stdout:session_autoclose 300 2024-09-11T23:17:58.734 INFO:teuthology.orchestra.run.smithi026.stdout:max_file_size 1099511627776 2024-09-11T23:17:58.734 INFO:teuthology.orchestra.run.smithi026.stdout:max_xattr_size 65536 2024-09-11T23:17:58.734 INFO:teuthology.orchestra.run.smithi026.stdout:required_client_features {} 2024-09-11T23:17:58.735 INFO:teuthology.orchestra.run.smithi026.stdout:last_failure 0 2024-09-11T23:17:58.735 INFO:teuthology.orchestra.run.smithi026.stdout:last_failure_osd_epoch 0 2024-09-11T23:17:58.735 INFO:teuthology.orchestra.run.smithi026.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-11T23:17:58.735 INFO:teuthology.orchestra.run.smithi026.stdout:max_mds 5 2024-09-11T23:17:58.735 INFO:teuthology.orchestra.run.smithi026.stdout:in 0,1,2,3,4 2024-09-11T23:17:58.735 INFO:teuthology.orchestra.run.smithi026.stdout:up {0=24485,1=24463,2=24458,3=24448,4=24443} 2024-09-11T23:17:58.735 INFO:teuthology.orchestra.run.smithi026.stdout:failed 2024-09-11T23:17:58.735 INFO:teuthology.orchestra.run.smithi026.stdout:damaged 2024-09-11T23:17:58.735 INFO:teuthology.orchestra.run.smithi026.stdout:stopped 2024-09-11T23:17:58.735 INFO:teuthology.orchestra.run.smithi026.stdout:data_pools [3] 2024-09-11T23:17:58.735 INFO:teuthology.orchestra.run.smithi026.stdout:metadata_pool 2 2024-09-11T23:17:58.735 INFO:teuthology.orchestra.run.smithi026.stdout:inline_data disabled 2024-09-11T23:17:58.735 INFO:teuthology.orchestra.run.smithi026.stdout:balancer 2024-09-11T23:17:58.735 INFO:teuthology.orchestra.run.smithi026.stdout:bal_rank_mask -1 2024-09-11T23:17:58.736 INFO:teuthology.orchestra.run.smithi026.stdout:standby_count_wanted 0 2024-09-11T23:17:58.736 INFO:teuthology.orchestra.run.smithi026.stdout:qdb_cluster leader: 24485 members: 24485,24463,24458,24448,24443 2024-09-11T23:17:58.736 INFO:teuthology.orchestra.run.smithi026.stdout:[mds.b{0:24485} state up:active seq 4 addr [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] compat {c=[1],r=[1],i=[1fff]}] 2024-09-11T23:17:58.736 INFO:teuthology.orchestra.run.smithi026.stdout:[mds.f{0:24439} state up:standby-replay seq 1 addr [v2:172.21.15.136:6834/3617665975,v1:172.21.15.136:6835/3617665975] compat {c=[1],r=[1],i=[1fff]}] 2024-09-11T23:17:58.736 INFO:teuthology.orchestra.run.smithi026.stdout:[mds.i{1:24463} state up:active seq 8 addr [v2:172.21.15.136:6838/3211398630,v1:172.21.15.136:6839/3211398630] compat {c=[1],r=[1],i=[1fff]}] 2024-09-11T23:17:58.736 INFO:teuthology.orchestra.run.smithi026.stdout:[mds.k{1:24407} state up:standby-replay seq 1 addr [v2:172.21.15.76:6832/3438147050,v1:172.21.15.76:6833/3438147050] compat {c=[1],r=[1],i=[1fff]}] 2024-09-11T23:17:58.736 INFO:teuthology.orchestra.run.smithi026.stdout:[mds.h{2:24458} state up:active seq 10 addr [v2:172.21.15.76:6836/2078978748,v1:172.21.15.76:6837/2078978748] compat {c=[1],r=[1],i=[1fff]}] 2024-09-11T23:17:58.736 INFO:teuthology.orchestra.run.smithi026.stdout:[mds.l{2:24403} state up:standby-replay seq 1 addr [v2:172.21.15.136:6832/2019841455,v1:172.21.15.136:6833/2019841455] compat {c=[1],r=[1],i=[1fff]}] 2024-09-11T23:17:58.736 INFO:teuthology.orchestra.run.smithi026.stdout:[mds.c{3:24448} state up:active seq 10 addr [v2:172.21.15.136:6836/1633082387,v1:172.21.15.136:6837/1633082387] compat {c=[1],r=[1],i=[1fff]}] 2024-09-11T23:17:58.736 INFO:teuthology.orchestra.run.smithi026.stdout:[mds.a{3:14562} state up:standby-replay seq 1 addr [v2:172.21.15.26:6840/2984982336,v1:172.21.15.26:6841/2984982336] compat {c=[1],r=[1],i=[1fff]}] 2024-09-11T23:17:58.736 INFO:teuthology.orchestra.run.smithi026.stdout:[mds.e{4:24443} state up:active seq 12 addr [v2:172.21.15.76:6834/393308499,v1:172.21.15.76:6835/393308499] compat {c=[1],r=[1],i=[1fff]}] 2024-09-11T23:17:58.736 INFO:teuthology.orchestra.run.smithi026.stdout:[mds.g{4:14550} state up:standby-replay seq 1 addr [v2:172.21.15.26:6838/2233068305,v1:172.21.15.26:6839/2233068305] compat {c=[1],r=[1],i=[1fff]}] 2024-09-11T23:17:58.736 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-11T23:17:58.736 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-11T23:17:58.737 INFO:teuthology.orchestra.run.smithi026.stdout:Standby daemons: 2024-09-11T23:17:58.737 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-11T23:17:58.737 INFO:teuthology.orchestra.run.smithi026.stdout:[mds.j{-1:14499} state up:standby seq 1 addr [v2:172.21.15.26:6834/367659515,v1:172.21.15.26:6835/367659515] compat {c=[1],r=[1],i=[1fff]}] 2024-09-11T23:17:58.737 INFO:teuthology.orchestra.run.smithi026.stdout:[mds.d{-1:14511} state up:standby seq 1 addr [v2:172.21.15.26:6836/2041065299,v1:172.21.15.26:6837/2041065299] compat {c=[1],r=[1],i=[1fff]}] 2024-09-11T23:17:58.738 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 28 2024-09-11T23:17:58.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:58.735+0000 7fee5a3a2640 1 -- 172.21.15.26:0/4026551369 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fee400a2c40 msgr2=0x7fee400a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:58.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:58.735+0000 7fee5a3a2640 1 --2- 172.21.15.26:0/4026551369 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fee400a2c40 0x7fee400a5100 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fee541992c0 tx=0x7fee44009110 comp rx=0 tx=0).stop 2024-09-11T23:17:58.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:58.735+0000 7fee5a3a2640 1 -- 172.21.15.26:0/4026551369 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee54101a40 msgr2=0x7fee54197da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:58.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:58.735+0000 7fee5a3a2640 1 --2- 172.21.15.26:0/4026551369 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee54101a40 0x7fee54197da0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fee50031610 tx=0x7fee500043b0 comp rx=0 tx=0).stop 2024-09-11T23:17:58.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:58.735+0000 7fee5a3a2640 1 -- 172.21.15.26:0/4026551369 shutdown_connections 2024-09-11T23:17:58.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:58.735+0000 7fee5a3a2640 1 --2- 172.21.15.26:0/4026551369 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fee400a2c40 0x7fee400a5100 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:58.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:58.735+0000 7fee5a3a2640 1 --2- 172.21.15.26:0/4026551369 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fee54103600 0x7fee5419f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:58.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:58.735+0000 7fee5a3a2640 1 --2- 172.21.15.26:0/4026551369 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fee54102c40 0x7fee541982e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:58.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:58.735+0000 7fee5a3a2640 1 --2- 172.21.15.26:0/4026551369 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee54101a40 0x7fee54197da0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:58.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:58.735+0000 7fee5a3a2640 1 -- 172.21.15.26:0/4026551369 >> 172.21.15.26:0/4026551369 conn(0x7fee540fd1f0 msgr2=0x7fee54105fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:17:58.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:58.735+0000 7fee5a3a2640 1 -- 172.21.15.26:0/4026551369 shutdown_connections 2024-09-11T23:17:58.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:17:58.735+0000 7fee5a3a2640 1 -- 172.21.15.26:0/4026551369 wait complete. 2024-09-11T23:17:59.206 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:17:58 smithi026 ceph-mon[31172]: from='client.15027 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:17:59.206 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:17:58 smithi026 ceph-mon[31172]: from='client.? 172.21.15.26:0/4026551369' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-11T23:17:59.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:17:58 smithi136 ceph-mon[35121]: from='client.15027 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:17:59.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:17:58 smithi136 ceph-mon[35121]: from='client.? 172.21.15.26:0/4026551369' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-11T23:17:59.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:17:58 smithi076 ceph-mon[35296]: from='client.15027 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:17:59.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:17:58 smithi076 ceph-mon[35296]: from='client.? 172.21.15.26:0/4026551369' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-11T23:17:59.649 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- bash -c 'ceph osd dump' 2024-09-11T23:17:59.914 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:18:00.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:17:59 smithi026 ceph-mon[31172]: pgmap v327: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T23:18:00.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:17:59 smithi136 ceph-mon[35121]: pgmap v327: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T23:18:00.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:17:59 smithi076 ceph-mon[35296]: pgmap v327: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T23:18:00.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:00.361+0000 7f026ca79640 1 -- 172.21.15.26:0/1812885393 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0268100b10 msgr2=0x7f0268100f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:00.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:00.361+0000 7f026ca79640 1 --2- 172.21.15.26:0/1812885393 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0268100b10 0x7f0268100f10 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f0258009900 tx=0x7f025802a410 comp rx=0 tx=0).stop 2024-09-11T23:18:00.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:00.361+0000 7f026ca79640 1 -- 172.21.15.26:0/1812885393 shutdown_connections 2024-09-11T23:18:00.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:00.361+0000 7f026ca79640 1 --2- 172.21.15.26:0/1812885393 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f02681026d0 0x7f0268108f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:00.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:00.361+0000 7f026ca79640 1 --2- 172.21.15.26:0/1812885393 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0268101d10 0x7f0268102190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:00.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:00.361+0000 7f026ca79640 1 --2- 172.21.15.26:0/1812885393 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0268100b10 0x7f0268100f10 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:00.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:00.361+0000 7f026ca79640 1 -- 172.21.15.26:0/1812885393 >> 172.21.15.26:0/1812885393 conn(0x7f02680fc2a0 msgr2=0x7f02680fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:18:00.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:00.361+0000 7f026ca79640 1 -- 172.21.15.26:0/1812885393 shutdown_connections 2024-09-11T23:18:00.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:00.361+0000 7f026ca79640 1 -- 172.21.15.26:0/1812885393 wait complete. 2024-09-11T23:18:00.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:00.362+0000 7f026ca79640 1 Processor -- start 2024-09-11T23:18:00.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:00.362+0000 7f026ca79640 1 -- start start 2024-09-11T23:18:00.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:00.362+0000 7f026ca79640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0268100b10 0x7f026810b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:00.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:00.362+0000 7f026ca79640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0268101d10 0x7f0268109ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:00.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:00.362+0000 7f026ca79640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f02681026d0 0x7f026810a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:00.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:00.362+0000 7f026ca79640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f026810d980 con 0x7f0268100b10 2024-09-11T23:18:00.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:00.362+0000 7f026ca79640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f026810d800 con 0x7f0268101d10 2024-09-11T23:18:00.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:00.362+0000 7f026ca79640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f026810db00 con 0x7f02681026d0 2024-09-11T23:18:00.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:00.362+0000 7f026659c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0268100b10 0x7f026810b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:00.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:00.362+0000 7f026659c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0268100b10 0x7f026810b490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59584/0 (socket says 172.21.15.26:59584) 2024-09-11T23:18:00.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:00.362+0000 7f026659c640 1 -- 172.21.15.26:0/458742169 learned_addr learned my addr 172.21.15.26:0/458742169 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:18:00.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:00.362+0000 7f0265d9b640 1 --2- 172.21.15.26:0/458742169 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0268101d10 0x7f0268109ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:00.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:00.362+0000 7f0266d9d640 1 --2- 172.21.15.26:0/458742169 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f02681026d0 0x7f026810a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:00.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:00.362+0000 7f026659c640 1 -- 172.21.15.26:0/458742169 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f02681026d0 msgr2=0x7f026810a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:00.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:00.362+0000 7f026659c640 1 --2- 172.21.15.26:0/458742169 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f02681026d0 0x7f026810a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:00.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:00.363+0000 7f026659c640 1 -- 172.21.15.26:0/458742169 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0268101d10 msgr2=0x7f0268109ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:00.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:00.363+0000 7f026659c640 1 --2- 172.21.15.26:0/458742169 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0268101d10 0x7f0268109ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:00.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:00.363+0000 7f026659c640 1 -- 172.21.15.26:0/458742169 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f026810a8e0 con 0x7f0268100b10 2024-09-11T23:18:00.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:00.363+0000 7f0265d9b640 1 --2- 172.21.15.26:0/458742169 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0268101d10 0x7f0268109ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:18:00.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:00.363+0000 7f026659c640 1 --2- 172.21.15.26:0/458742169 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0268100b10 0x7f026810b490 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f0258031610 tx=0x7f02580043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:18:00.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:00.363+0000 7f02577fe640 1 -- 172.21.15.26:0/458742169 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f025803f030 con 0x7f0268100b10 2024-09-11T23:18:00.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:00.363+0000 7f026ca79640 1 -- 172.21.15.26:0/458742169 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02681a21c0 con 0x7f0268100b10 2024-09-11T23:18:00.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:00.363+0000 7f02577fe640 1 -- 172.21.15.26:0/458742169 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0258003ce0 con 0x7f0268100b10 2024-09-11T23:18:00.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:00.363+0000 7f02577fe640 1 -- 172.21.15.26:0/458742169 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0258035070 con 0x7f0268100b10 2024-09-11T23:18:00.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:00.363+0000 7f026ca79640 1 -- 172.21.15.26:0/458742169 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02681a26f0 con 0x7f0268100b10 2024-09-11T23:18:00.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:00.364+0000 7f026ca79640 1 -- 172.21.15.26:0/458742169 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0268108000 con 0x7f0268100b10 2024-09-11T23:18:00.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:00.366+0000 7f02577fe640 1 -- 172.21.15.26:0/458742169 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f0258042030 con 0x7f0268100b10 2024-09-11T23:18:00.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:00.367+0000 7f02577fe640 1 --2- 172.21.15.26:0/458742169 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f02500a2c10 0x7f02500a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:00.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:00.367+0000 7f0265d9b640 1 --2- 172.21.15.26:0/458742169 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f02500a2c10 0x7f02500a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:00.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:00.367+0000 7f02577fe640 1 -- 172.21.15.26:0/458742169 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f02580efad0 con 0x7f0268100b10 2024-09-11T23:18:00.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:00.367+0000 7f0265d9b640 1 --2- 172.21.15.26:0/458742169 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f02500a2c10 0x7f02500a50d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f026810b210 tx=0x7f025c008040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:18:00.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:00.370+0000 7f02577fe640 1 -- 172.21.15.26:0/458742169 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f02580bc8d0 con 0x7f0268100b10 2024-09-11T23:18:00.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:00.549+0000 7f026ca79640 1 -- 172.21.15.26:0/458742169 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f026802d8e0 con 0x7f0268100b10 2024-09-11T23:18:00.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:00.550+0000 7f02577fe640 1 -- 172.21.15.26:0/458742169 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v85) ==== 56+0+5295 (secure 0 0 0) 0x7f02580bc1d0 con 0x7f0268100b10 2024-09-11T23:18:00.550 INFO:teuthology.orchestra.run.smithi026.stdout:epoch 85 2024-09-11T23:18:00.550 INFO:teuthology.orchestra.run.smithi026.stdout:fsid af7e6fc8-7092-11ef-bcea-c7b262605968 2024-09-11T23:18:00.550 INFO:teuthology.orchestra.run.smithi026.stdout:created 2024-09-11T23:08:40.486438+0000 2024-09-11T23:18:00.550 INFO:teuthology.orchestra.run.smithi026.stdout:modified 2024-09-11T23:16:35.935151+0000 2024-09-11T23:18:00.551 INFO:teuthology.orchestra.run.smithi026.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-11T23:18:00.551 INFO:teuthology.orchestra.run.smithi026.stdout:crush_version 26 2024-09-11T23:18:00.551 INFO:teuthology.orchestra.run.smithi026.stdout:full_ratio 0.95 2024-09-11T23:18:00.551 INFO:teuthology.orchestra.run.smithi026.stdout:backfillfull_ratio 0.9 2024-09-11T23:18:00.551 INFO:teuthology.orchestra.run.smithi026.stdout:nearfull_ratio 0.85 2024-09-11T23:18:00.551 INFO:teuthology.orchestra.run.smithi026.stdout:require_min_compat_client luminous 2024-09-11T23:18:00.551 INFO:teuthology.orchestra.run.smithi026.stdout:min_compat_client luminous 2024-09-11T23:18:00.551 INFO:teuthology.orchestra.run.smithi026.stdout:require_osd_release squid 2024-09-11T23:18:00.551 INFO:teuthology.orchestra.run.smithi026.stdout:stretch_mode_enabled false 2024-09-11T23:18:00.551 INFO:teuthology.orchestra.run.smithi026.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 22 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-09-11T23:18:00.552 INFO:teuthology.orchestra.run.smithi026.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 83 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-09-11T23:18:00.552 INFO:teuthology.orchestra.run.smithi026.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 83 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-09-11T23:18:00.552 INFO:teuthology.orchestra.run.smithi026.stdout:max_osd 12 2024-09-11T23:18:00.552 INFO:teuthology.orchestra.run.smithi026.stdout:osd.0 up in weight 1 up_from 8 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.26:6802/2014857905,v1:172.21.15.26:6803/2014857905] [v2:172.21.15.26:6804/2014857905,v1:172.21.15.26:6805/2014857905] exists,up a4d62fcf-c31e-41b1-932c-3d87ff640531 2024-09-11T23:18:00.552 INFO:teuthology.orchestra.run.smithi026.stdout:osd.1 up in weight 1 up_from 13 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.76:6800/333383087,v1:172.21.15.76:6801/333383087] [v2:172.21.15.76:6802/333383087,v1:172.21.15.76:6803/333383087] exists,up e0d69112-3c0a-4fc5-8783-e41b4081e0c6 2024-09-11T23:18:00.552 INFO:teuthology.orchestra.run.smithi026.stdout:osd.2 up in weight 1 up_from 18 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.136:6800/1271538911,v1:172.21.15.136:6801/1271538911] [v2:172.21.15.136:6802/1271538911,v1:172.21.15.136:6803/1271538911] exists,up d578429e-bf23-4a1e-890f-7172955c0eb1 2024-09-11T23:18:00.552 INFO:teuthology.orchestra.run.smithi026.stdout:osd.3 up in weight 1 up_from 25 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.26:6810/1740304151,v1:172.21.15.26:6811/1740304151] [v2:172.21.15.26:6812/1740304151,v1:172.21.15.26:6813/1740304151] exists,up 3e34623f-0ea5-4b10-b8a9-f8756c35d512 2024-09-11T23:18:00.552 INFO:teuthology.orchestra.run.smithi026.stdout:osd.4 up in weight 1 up_from 33 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.76:6808/2702262182,v1:172.21.15.76:6809/2702262182] [v2:172.21.15.76:6810/2702262182,v1:172.21.15.76:6811/2702262182] exists,up 77607929-f9ed-49a9-abb5-48cd88eec4ef 2024-09-11T23:18:00.552 INFO:teuthology.orchestra.run.smithi026.stdout:osd.5 up in weight 1 up_from 37 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.136:6808/1749229277,v1:172.21.15.136:6809/1749229277] [v2:172.21.15.136:6810/1749229277,v1:172.21.15.136:6811/1749229277] exists,up 29e20e37-09a0-4924-8aaa-17939763a7c2 2024-09-11T23:18:00.552 INFO:teuthology.orchestra.run.smithi026.stdout:osd.6 up in weight 1 up_from 45 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.26:6818/1928245624,v1:172.21.15.26:6819/1928245624] [v2:172.21.15.26:6820/1928245624,v1:172.21.15.26:6821/1928245624] exists,up e0ddd1c0-1c46-41e6-b2f4-396bdcd2bf8a 2024-09-11T23:18:00.552 INFO:teuthology.orchestra.run.smithi026.stdout:osd.7 up in weight 1 up_from 50 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.76:6816/1409789455,v1:172.21.15.76:6817/1409789455] [v2:172.21.15.76:6818/1409789455,v1:172.21.15.76:6819/1409789455] exists,up fe031f5c-d915-48fb-91a6-c76f252467fb 2024-09-11T23:18:00.553 INFO:teuthology.orchestra.run.smithi026.stdout:osd.8 up in weight 1 up_from 57 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.136:6816/543619079,v1:172.21.15.136:6817/543619079] [v2:172.21.15.136:6818/543619079,v1:172.21.15.136:6819/543619079] exists,up 18851bde-9e6c-4e05-b4bf-ac5c3073eee7 2024-09-11T23:18:00.553 INFO:teuthology.orchestra.run.smithi026.stdout:osd.9 up in weight 1 up_from 65 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.26:6826/2201297064,v1:172.21.15.26:6827/2201297064] [v2:172.21.15.26:6828/2201297064,v1:172.21.15.26:6829/2201297064] exists,up 52a65eab-8446-47ad-bfe7-5133f7e3eb98 2024-09-11T23:18:00.553 INFO:teuthology.orchestra.run.smithi026.stdout:osd.10 up in weight 1 up_from 70 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.76:6824/3184128096,v1:172.21.15.76:6825/3184128096] [v2:172.21.15.76:6826/3184128096,v1:172.21.15.76:6827/3184128096] exists,up b758d4bb-da62-4c5f-a992-bb6a145d35d9 2024-09-11T23:18:00.553 INFO:teuthology.orchestra.run.smithi026.stdout:osd.11 up in weight 1 up_from 77 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.136:6824/1960689685,v1:172.21.15.136:6825/1960689685] [v2:172.21.15.136:6826/1960689685,v1:172.21.15.136:6827/1960689685] exists,up 8fddda46-e22b-4d0c-ac48-e038fb29dfec 2024-09-11T23:18:00.553 INFO:teuthology.orchestra.run.smithi026.stdout:pg_upmap_items 2.0 [8,4] 2024-09-11T23:18:00.553 INFO:teuthology.orchestra.run.smithi026.stdout:pg_upmap_items 2.a [2,11] 2024-09-11T23:18:00.553 INFO:teuthology.orchestra.run.smithi026.stdout:pg_upmap_items 2.1c [8,11] 2024-09-11T23:18:00.553 INFO:teuthology.orchestra.run.smithi026.stdout:pg_upmap_items 2.1d [2,4] 2024-09-11T23:18:00.553 INFO:teuthology.orchestra.run.smithi026.stdout:pg_upmap_items 3.11 [4,8] 2024-09-11T23:18:00.553 INFO:teuthology.orchestra.run.smithi026.stdout:pg_upmap_items 3.18 [3,9] 2024-09-11T23:18:00.553 INFO:teuthology.orchestra.run.smithi026.stdout:pg_upmap_items 3.36 [4,9] 2024-09-11T23:18:00.553 INFO:teuthology.orchestra.run.smithi026.stdout:pg_upmap_items 3.39 [5,6] 2024-09-11T23:18:00.553 INFO:teuthology.orchestra.run.smithi026.stdout:pg_upmap_items 3.3c [4,9] 2024-09-11T23:18:00.554 INFO:teuthology.orchestra.run.smithi026.stdout:blocklist 172.21.15.26:6801/677148151 expires 2024-09-12T23:09:33.983310+0000 2024-09-11T23:18:00.554 INFO:teuthology.orchestra.run.smithi026.stdout:blocklist 172.21.15.26:6800/677148151 expires 2024-09-12T23:09:33.983310+0000 2024-09-11T23:18:00.554 INFO:teuthology.orchestra.run.smithi026.stdout:blocklist 172.21.15.26:0/3597996003 expires 2024-09-12T23:09:33.983310+0000 2024-09-11T23:18:00.554 INFO:teuthology.orchestra.run.smithi026.stdout:blocklist 172.21.15.26:6801/2673575521 expires 2024-09-12T23:09:09.781665+0000 2024-09-11T23:18:00.554 INFO:teuthology.orchestra.run.smithi026.stdout:blocklist 172.21.15.26:0/883212708 expires 2024-09-12T23:09:33.983310+0000 2024-09-11T23:18:00.554 INFO:teuthology.orchestra.run.smithi026.stdout:blocklist 172.21.15.26:0/2107281413 expires 2024-09-12T23:09:33.983310+0000 2024-09-11T23:18:00.554 INFO:teuthology.orchestra.run.smithi026.stdout:blocklist 172.21.15.26:6800/2673575521 expires 2024-09-12T23:09:09.781665+0000 2024-09-11T23:18:00.554 INFO:teuthology.orchestra.run.smithi026.stdout:blocklist 172.21.15.26:0/2817092653 expires 2024-09-12T23:09:09.781665+0000 2024-09-11T23:18:00.554 INFO:teuthology.orchestra.run.smithi026.stdout:blocklist 172.21.15.26:0/17142658 expires 2024-09-12T23:09:09.781665+0000 2024-09-11T23:18:00.554 INFO:teuthology.orchestra.run.smithi026.stdout:blocklist 172.21.15.26:0/3861103130 expires 2024-09-12T23:09:09.781665+0000 2024-09-11T23:18:00.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:00.553+0000 7f026ca79640 1 -- 172.21.15.26:0/458742169 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f02500a2c10 msgr2=0x7f02500a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:00.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:00.553+0000 7f026ca79640 1 --2- 172.21.15.26:0/458742169 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f02500a2c10 0x7f02500a50d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f026810b210 tx=0x7f025c008040 comp rx=0 tx=0).stop 2024-09-11T23:18:00.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:00.553+0000 7f026ca79640 1 -- 172.21.15.26:0/458742169 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0268100b10 msgr2=0x7f026810b490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:00.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:00.553+0000 7f026ca79640 1 --2- 172.21.15.26:0/458742169 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0268100b10 0x7f026810b490 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f0258031610 tx=0x7f02580043b0 comp rx=0 tx=0).stop 2024-09-11T23:18:00.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:00.553+0000 7f026ca79640 1 -- 172.21.15.26:0/458742169 shutdown_connections 2024-09-11T23:18:00.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:00.553+0000 7f026ca79640 1 --2- 172.21.15.26:0/458742169 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f02500a2c10 0x7f02500a50d0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:00.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:00.553+0000 7f026ca79640 1 --2- 172.21.15.26:0/458742169 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0268101d10 0x7f0268109ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:00.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:00.553+0000 7f026ca79640 1 --2- 172.21.15.26:0/458742169 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f02681026d0 0x7f026810a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:00.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:00.553+0000 7f026ca79640 1 --2- 172.21.15.26:0/458742169 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0268100b10 0x7f026810b490 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:00.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:00.553+0000 7f026ca79640 1 -- 172.21.15.26:0/458742169 >> 172.21.15.26:0/458742169 conn(0x7f02680fc2a0 msgr2=0x7f02681051d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:18:00.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:00.553+0000 7f026ca79640 1 -- 172.21.15.26:0/458742169 shutdown_connections 2024-09-11T23:18:00.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:00.553+0000 7f026ca79640 1 -- 172.21.15.26:0/458742169 wait complete. 2024-09-11T23:18:01.186 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:00 smithi026 ceph-mon[31172]: from='client.? 172.21.15.26:0/458742169' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-11T23:18:01.218 INFO:teuthology.run_tasks:Running task fs.ready... 2024-09-11T23:18:01.231 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:18:01.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:00 smithi136 ceph-mon[35121]: from='client.? 172.21.15.26:0/458742169' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-11T23:18:01.304 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:00 smithi076 ceph-mon[35296]: from='client.? 172.21.15.26:0/458742169' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-11T23:18:01.490 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:18:01.864 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:01 smithi026 ceph-mon[31172]: pgmap v328: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-11T23:18:01.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:01.949+0000 7f016507b640 1 -- 172.21.15.26:0/1777338474 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0160101de0 msgr2=0x7f01601021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:01.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:01.949+0000 7f016507b640 1 --2- 172.21.15.26:0/1777338474 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0160101de0 0x7f01601021c0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f0150009900 tx=0x7f015002a410 comp rx=0 tx=0).stop 2024-09-11T23:18:01.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:01.949+0000 7f016507b640 1 -- 172.21.15.26:0/1777338474 shutdown_connections 2024-09-11T23:18:01.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:01.949+0000 7f016507b640 1 --2- 172.21.15.26:0/1777338474 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0160106f10 0x7f016010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:01.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:01.949+0000 7f016507b640 1 --2- 172.21.15.26:0/1777338474 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0160102790 0x7f01601067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:01.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:01.949+0000 7f016507b640 1 --2- 172.21.15.26:0/1777338474 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0160101de0 0x7f01601021c0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:01.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:01.949+0000 7f016507b640 1 -- 172.21.15.26:0/1777338474 >> 172.21.15.26:0/1777338474 conn(0x7f01600fd8c0 msgr2=0x7f01600ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:18:01.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:01.949+0000 7f016507b640 1 -- 172.21.15.26:0/1777338474 shutdown_connections 2024-09-11T23:18:01.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:01.949+0000 7f016507b640 1 -- 172.21.15.26:0/1777338474 wait complete. 2024-09-11T23:18:01.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:01.949+0000 7f016507b640 1 Processor -- start 2024-09-11T23:18:01.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:01.949+0000 7f016507b640 1 -- start start 2024-09-11T23:18:01.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:01.950+0000 7f016507b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0160101de0 0x7f01600744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:01.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:01.950+0000 7f016507b640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0160102790 0x7f0160074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:01.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:01.950+0000 7f016507b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0160106f10 0x7f0160074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:01.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:01.950+0000 7f016507b640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f0160111f30 con 0x7f0160101de0 2024-09-11T23:18:01.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:01.950+0000 7f016507b640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f0160111db0 con 0x7f0160106f10 2024-09-11T23:18:01.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:01.950+0000 7f016507b640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f01601120b0 con 0x7f0160102790 2024-09-11T23:18:01.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:01.950+0000 7f015ed9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0160101de0 0x7f01600744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:01.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:01.950+0000 7f015ed9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0160101de0 0x7f01600744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33172/0 (socket says 172.21.15.26:33172) 2024-09-11T23:18:01.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:01.950+0000 7f015ed9d640 1 -- 172.21.15.26:0/15379066 learned_addr learned my addr 172.21.15.26:0/15379066 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:18:01.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:01.950+0000 7f015f59e640 1 --2- 172.21.15.26:0/15379066 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0160106f10 0x7f0160074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:01.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:01.950+0000 7f0157fff640 1 --2- 172.21.15.26:0/15379066 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0160102790 0x7f0160074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:01.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:01.950+0000 7f015ed9d640 1 -- 172.21.15.26:0/15379066 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0160102790 msgr2=0x7f0160074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:01.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:01.950+0000 7f015ed9d640 1 --2- 172.21.15.26:0/15379066 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0160102790 0x7f0160074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:01.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:01.950+0000 7f015ed9d640 1 -- 172.21.15.26:0/15379066 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0160106f10 msgr2=0x7f0160074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:01.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:01.950+0000 7f015ed9d640 1 --2- 172.21.15.26:0/15379066 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0160106f10 0x7f0160074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:01.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:01.950+0000 7f015ed9d640 1 -- 172.21.15.26:0/15379066 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01601a6980 con 0x7f0160101de0 2024-09-11T23:18:01.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:01.950+0000 7f0157fff640 1 --2- 172.21.15.26:0/15379066 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0160102790 0x7f0160074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:18:01.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:01.950+0000 7f015ed9d640 1 --2- 172.21.15.26:0/15379066 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0160101de0 0x7f01600744f0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f0150006010 tx=0x7f01500042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:18:01.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:01.951+0000 7f015cd99640 1 -- 172.21.15.26:0/15379066 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f015003f030 con 0x7f0160101de0 2024-09-11T23:18:01.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:01.951+0000 7f015cd99640 1 -- 172.21.15.26:0/15379066 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0150003ce0 con 0x7f0160101de0 2024-09-11T23:18:01.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:01.951+0000 7f015cd99640 1 -- 172.21.15.26:0/15379066 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0150035070 con 0x7f0160101de0 2024-09-11T23:18:01.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:01.951+0000 7f016507b640 1 -- 172.21.15.26:0/15379066 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01601a6c10 con 0x7f0160101de0 2024-09-11T23:18:01.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:01.951+0000 7f016507b640 1 -- 172.21.15.26:0/15379066 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01601a7140 con 0x7f0160101de0 2024-09-11T23:18:01.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:01.952+0000 7f016507b640 1 -- 172.21.15.26:0/15379066 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01601021c0 con 0x7f0160101de0 2024-09-11T23:18:01.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:01.954+0000 7f015cd99640 1 -- 172.21.15.26:0/15379066 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f0150042030 con 0x7f0160101de0 2024-09-11T23:18:01.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:01.954+0000 7f015cd99640 1 --2- 172.21.15.26:0/15379066 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f014c0a2c90 0x7f014c0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:01.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:01.954+0000 7f0157fff640 1 --2- 172.21.15.26:0/15379066 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f014c0a2c90 0x7f014c0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:01.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:01.955+0000 7f015cd99640 1 -- 172.21.15.26:0/15379066 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f01500efa60 con 0x7f0160101de0 2024-09-11T23:18:01.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:01.955+0000 7f0157fff640 1 --2- 172.21.15.26:0/15379066 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f014c0a2c90 0x7f014c0a5150 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f01601087b0 tx=0x7f0148005e50 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:18:01.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:01.958+0000 7f015cd99640 1 -- 172.21.15.26:0/15379066 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f01500bc920 con 0x7f0160101de0 2024-09-11T23:18:02.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:02.172+0000 7f016507b640 1 -- 172.21.15.26:0/15379066 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f016010a0f0 con 0x7f0160101de0 2024-09-11T23:18:02.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:02.173+0000 7f015cd99640 1 -- 172.21.15.26:0/15379066 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11971 (secure 0 0 0) 0x7f01500bc220 con 0x7f0160101de0 2024-09-11T23:18:02.174 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-11T23:18:02.174 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":28,"btime":"2024-09-11T23:16:38:976987+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14499,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/367659515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":367659515},{"type":"v1","addr":"172.21.15.26:6835","nonce":367659515}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14511,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/2041065299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":2041065299},{"type":"v1","addr":"172.21.15.26:6837","nonce":2041065299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:16:26.544203+0000","modified":"2024-09-11T23:16:38.976970+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24485,"mds_1":24463,"mds_2":24458,"mds_3":24448,"mds_4":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14550":{"gid":14550,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6839/2233068305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":2233068305},{"type":"v1","addr":"172.21.15.26:6839","nonce":2233068305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14562":{"gid":14562,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6841/2984982336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":2984982336},{"type":"v1","addr":"172.21.15.26:6841","nonce":2984982336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6833/2019841455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":2019841455},{"type":"v1","addr":"172.21.15.136:6833","nonce":2019841455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.76:6833/3438147050","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3438147050},{"type":"v1","addr":"172.21.15.76:6833","nonce":3438147050}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6835/3617665975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3617665975},{"type":"v1","addr":"172.21.15.136:6835","nonce":3617665975}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":12,"addr":"172.21.15.76:6835/393308499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":393308499},{"type":"v1","addr":"172.21.15.76:6835","nonce":393308499}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":10,"addr":"172.21.15.136:6837/1633082387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":1633082387},{"type":"v1","addr":"172.21.15.136:6837","nonce":1633082387}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.76:6837/2078978748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":2078978748},{"type":"v1","addr":"172.21.15.76:6837","nonce":2078978748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.136:6839/3211398630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":3211398630},{"type":"v1","addr":"172.21.15.136:6839","nonce":3211398630}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.76:6839/2077637090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6838","nonce":2077637090},{"type":"v1","addr":"172.21.15.76:6839","nonce":2077637090}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24485,"qdb_cluster":[24485,24463,24458,24448,24443]},"id":1}]} 2024-09-11T23:18:02.174 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 28 2024-09-11T23:18:02.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:02.175+0000 7f016507b640 1 -- 172.21.15.26:0/15379066 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f014c0a2c90 msgr2=0x7f014c0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:02.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:02.175+0000 7f016507b640 1 --2- 172.21.15.26:0/15379066 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f014c0a2c90 0x7f014c0a5150 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f01601087b0 tx=0x7f0148005e50 comp rx=0 tx=0).stop 2024-09-11T23:18:02.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:02.175+0000 7f016507b640 1 -- 172.21.15.26:0/15379066 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0160101de0 msgr2=0x7f01600744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:02.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:02.175+0000 7f016507b640 1 --2- 172.21.15.26:0/15379066 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0160101de0 0x7f01600744f0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f0150006010 tx=0x7f01500042f0 comp rx=0 tx=0).stop 2024-09-11T23:18:02.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:02.175+0000 7f016507b640 1 -- 172.21.15.26:0/15379066 shutdown_connections 2024-09-11T23:18:02.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:02.175+0000 7f016507b640 1 --2- 172.21.15.26:0/15379066 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f014c0a2c90 0x7f014c0a5150 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:02.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:02.175+0000 7f016507b640 1 --2- 172.21.15.26:0/15379066 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0160106f10 0x7f0160074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:02.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:02.175+0000 7f016507b640 1 --2- 172.21.15.26:0/15379066 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0160102790 0x7f0160074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:02.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:02.175+0000 7f016507b640 1 --2- 172.21.15.26:0/15379066 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0160101de0 0x7f01600744f0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:02.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:02.175+0000 7f016507b640 1 -- 172.21.15.26:0/15379066 >> 172.21.15.26:0/15379066 conn(0x7f01600fd8c0 msgr2=0x7f01600fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:18:02.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:02.175+0000 7f016507b640 1 -- 172.21.15.26:0/15379066 shutdown_connections 2024-09-11T23:18:02.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:02.175+0000 7f016507b640 1 -- 172.21.15.26:0/15379066 wait complete. 2024-09-11T23:18:02.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:01 smithi136 ceph-mon[35121]: pgmap v328: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-11T23:18:02.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:01 smithi076 ceph-mon[35296]: pgmap v328: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-11T23:18:02.904 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:02 smithi026 ceph-mon[31172]: from='client.? 172.21.15.26:0/15379066' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:18:02.939 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:18:03.204 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:18:03.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:02 smithi136 ceph-mon[35121]: from='client.? 172.21.15.26:0/15379066' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:18:03.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:02 smithi076 ceph-mon[35296]: from='client.? 172.21.15.26:0/15379066' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:18:03.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:03.662+0000 7fd87cb1a640 1 -- 172.21.15.26:0/3580084582 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd878102790 msgr2=0x7fd8781067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:03.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:03.662+0000 7fd87cb1a640 1 --2- 172.21.15.26:0/3580084582 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd878102790 0x7fd8781067e0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fd860009900 tx=0x7fd86002a410 comp rx=0 tx=0).stop 2024-09-11T23:18:03.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:03.662+0000 7fd87cb1a640 1 -- 172.21.15.26:0/3580084582 shutdown_connections 2024-09-11T23:18:03.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:03.662+0000 7fd87cb1a640 1 --2- 172.21.15.26:0/3580084582 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd878106f10 0x7fd87810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:03.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:03.662+0000 7fd87cb1a640 1 --2- 172.21.15.26:0/3580084582 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd878102790 0x7fd8781067e0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:03.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:03.662+0000 7fd87cb1a640 1 --2- 172.21.15.26:0/3580084582 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd878101de0 0x7fd8781021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:03.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:03.662+0000 7fd87cb1a640 1 -- 172.21.15.26:0/3580084582 >> 172.21.15.26:0/3580084582 conn(0x7fd8780fd8c0 msgr2=0x7fd8780ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:18:03.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:03.662+0000 7fd87cb1a640 1 -- 172.21.15.26:0/3580084582 shutdown_connections 2024-09-11T23:18:03.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:03.662+0000 7fd87cb1a640 1 -- 172.21.15.26:0/3580084582 wait complete. 2024-09-11T23:18:03.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:03.663+0000 7fd87cb1a640 1 Processor -- start 2024-09-11T23:18:03.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:03.663+0000 7fd87cb1a640 1 -- start start 2024-09-11T23:18:03.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:03.663+0000 7fd87cb1a640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd878101de0 0x7fd878198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:03.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:03.663+0000 7fd87cb1a640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd878102790 0x7fd878198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:03.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:03.663+0000 7fd87cb1a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd878106f10 0x7fd87819cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:03.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:03.663+0000 7fd87cb1a640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd878111f70 con 0x7fd878106f10 2024-09-11T23:18:03.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:03.663+0000 7fd87cb1a640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fd878111df0 con 0x7fd878101de0 2024-09-11T23:18:03.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:03.663+0000 7fd87cb1a640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd8781120f0 con 0x7fd878102790 2024-09-11T23:18:03.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:03.664+0000 7fd876d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd878106f10 0x7fd87819cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:03.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:03.664+0000 7fd876d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd878106f10 0x7fd87819cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33198/0 (socket says 172.21.15.26:33198) 2024-09-11T23:18:03.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:03.664+0000 7fd876d9d640 1 -- 172.21.15.26:0/1671480159 learned_addr learned my addr 172.21.15.26:0/1671480159 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:18:03.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:03.664+0000 7fd86ffff640 1 --2- 172.21.15.26:0/1671480159 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd878102790 0x7fd878198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:03.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:03.664+0000 7fd87659c640 1 --2- 172.21.15.26:0/1671480159 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd878101de0 0x7fd878198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:03.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:03.664+0000 7fd876d9d640 1 -- 172.21.15.26:0/1671480159 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd878102790 msgr2=0x7fd878198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:03.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:03.664+0000 7fd876d9d640 1 --2- 172.21.15.26:0/1671480159 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd878102790 0x7fd878198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:03.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:03.664+0000 7fd876d9d640 1 -- 172.21.15.26:0/1671480159 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd878101de0 msgr2=0x7fd878198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:03.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:03.664+0000 7fd876d9d640 1 --2- 172.21.15.26:0/1671480159 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd878101de0 0x7fd878198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:03.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:03.664+0000 7fd876d9d640 1 -- 172.21.15.26:0/1671480159 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd87819d3e0 con 0x7fd878106f10 2024-09-11T23:18:03.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:03.664+0000 7fd876d9d640 1 --2- 172.21.15.26:0/1671480159 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd878106f10 0x7fd87819cd00 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fd87000eaa0 tx=0x7fd87000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:18:03.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:03.664+0000 7fd86f7fe640 1 -- 172.21.15.26:0/1671480159 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd87000cdb0 con 0x7fd878106f10 2024-09-11T23:18:03.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:03.664+0000 7fd86f7fe640 1 -- 172.21.15.26:0/1671480159 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd870005020 con 0x7fd878106f10 2024-09-11T23:18:03.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:03.664+0000 7fd87cb1a640 1 -- 172.21.15.26:0/1671480159 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd87819d6d0 con 0x7fd878106f10 2024-09-11T23:18:03.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:03.664+0000 7fd86f7fe640 1 -- 172.21.15.26:0/1671480159 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8700116b0 con 0x7fd878106f10 2024-09-11T23:18:03.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:03.664+0000 7fd87cb1a640 1 -- 172.21.15.26:0/1671480159 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8781a4fb0 con 0x7fd878106f10 2024-09-11T23:18:03.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:03.665+0000 7fd87cb1a640 1 -- 172.21.15.26:0/1671480159 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8781021c0 con 0x7fd878106f10 2024-09-11T23:18:03.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:03.667+0000 7fd86f7fe640 1 -- 172.21.15.26:0/1671480159 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fd870005560 con 0x7fd878106f10 2024-09-11T23:18:03.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:03.667+0000 7fd86f7fe640 1 --2- 172.21.15.26:0/1671480159 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fd8640a2c10 0x7fd8640a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:03.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:03.667+0000 7fd87659c640 1 --2- 172.21.15.26:0/1671480159 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fd8640a2c10 0x7fd8640a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:03.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:03.667+0000 7fd86f7fe640 1 -- 172.21.15.26:0/1671480159 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fd8700cf3e0 con 0x7fd878106f10 2024-09-11T23:18:03.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:03.668+0000 7fd87659c640 1 --2- 172.21.15.26:0/1671480159 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fd8640a2c10 0x7fd8640a50d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fd868005fd0 tx=0x7fd868005850 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:18:03.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:03.671+0000 7fd86f7fe640 1 -- 172.21.15.26:0/1671480159 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd870016030 con 0x7fd878106f10 2024-09-11T23:18:03.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:03.886+0000 7fd87cb1a640 1 -- 172.21.15.26:0/1671480159 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd87810a150 con 0x7fd878106f10 2024-09-11T23:18:03.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:03.887+0000 7fd86f7fe640 1 -- 172.21.15.26:0/1671480159 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11971 (secure 0 0 0) 0x7fd87009c1e0 con 0x7fd878106f10 2024-09-11T23:18:03.887 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-11T23:18:03.887 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":28,"btime":"2024-09-11T23:16:38:976987+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14499,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/367659515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":367659515},{"type":"v1","addr":"172.21.15.26:6835","nonce":367659515}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14511,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/2041065299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":2041065299},{"type":"v1","addr":"172.21.15.26:6837","nonce":2041065299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:16:26.544203+0000","modified":"2024-09-11T23:16:38.976970+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24485,"mds_1":24463,"mds_2":24458,"mds_3":24448,"mds_4":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14550":{"gid":14550,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6839/2233068305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":2233068305},{"type":"v1","addr":"172.21.15.26:6839","nonce":2233068305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14562":{"gid":14562,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6841/2984982336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":2984982336},{"type":"v1","addr":"172.21.15.26:6841","nonce":2984982336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6833/2019841455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":2019841455},{"type":"v1","addr":"172.21.15.136:6833","nonce":2019841455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.76:6833/3438147050","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3438147050},{"type":"v1","addr":"172.21.15.76:6833","nonce":3438147050}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6835/3617665975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3617665975},{"type":"v1","addr":"172.21.15.136:6835","nonce":3617665975}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":12,"addr":"172.21.15.76:6835/393308499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":393308499},{"type":"v1","addr":"172.21.15.76:6835","nonce":393308499}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":10,"addr":"172.21.15.136:6837/1633082387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":1633082387},{"type":"v1","addr":"172.21.15.136:6837","nonce":1633082387}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.76:6837/2078978748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":2078978748},{"type":"v1","addr":"172.21.15.76:6837","nonce":2078978748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.136:6839/3211398630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":3211398630},{"type":"v1","addr":"172.21.15.136:6839","nonce":3211398630}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.76:6839/2077637090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6838","nonce":2077637090},{"type":"v1","addr":"172.21.15.76:6839","nonce":2077637090}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24485,"qdb_cluster":[24485,24463,24458,24448,24443]},"id":1}]} 2024-09-11T23:18:03.888 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 28 2024-09-11T23:18:03.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:03.889+0000 7fd87cb1a640 1 -- 172.21.15.26:0/1671480159 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fd8640a2c10 msgr2=0x7fd8640a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:03.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:03.889+0000 7fd87cb1a640 1 --2- 172.21.15.26:0/1671480159 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fd8640a2c10 0x7fd8640a50d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fd868005fd0 tx=0x7fd868005850 comp rx=0 tx=0).stop 2024-09-11T23:18:03.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:03.889+0000 7fd87cb1a640 1 -- 172.21.15.26:0/1671480159 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd878106f10 msgr2=0x7fd87819cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:03.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:03.889+0000 7fd87cb1a640 1 --2- 172.21.15.26:0/1671480159 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd878106f10 0x7fd87819cd00 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fd87000eaa0 tx=0x7fd87000ef60 comp rx=0 tx=0).stop 2024-09-11T23:18:03.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:03.889+0000 7fd87cb1a640 1 -- 172.21.15.26:0/1671480159 shutdown_connections 2024-09-11T23:18:03.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:03.889+0000 7fd87cb1a640 1 --2- 172.21.15.26:0/1671480159 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fd8640a2c10 0x7fd8640a50d0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:03.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:03.889+0000 7fd87cb1a640 1 --2- 172.21.15.26:0/1671480159 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd878106f10 0x7fd87819cd00 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:03.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:03.889+0000 7fd87cb1a640 1 --2- 172.21.15.26:0/1671480159 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd878102790 0x7fd878198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:03.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:03.889+0000 7fd87cb1a640 1 --2- 172.21.15.26:0/1671480159 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd878101de0 0x7fd878198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:03.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:03.889+0000 7fd87cb1a640 1 -- 172.21.15.26:0/1671480159 >> 172.21.15.26:0/1671480159 conn(0x7fd8780fd8c0 msgr2=0x7fd8780fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:18:03.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:03.889+0000 7fd87cb1a640 1 -- 172.21.15.26:0/1671480159 shutdown_connections 2024-09-11T23:18:03.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:03.889+0000 7fd87cb1a640 1 -- 172.21.15.26:0/1671480159 wait complete. 2024-09-11T23:18:04.241 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:03 smithi136 ceph-mon[35121]: pgmap v329: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-11T23:18:04.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:03 smithi026 ceph-mon[31172]: pgmap v329: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-11T23:18:04.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:03 smithi076 ceph-mon[35296]: pgmap v329: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-11T23:18:04.731 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-11T23:18:05.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:04 smithi026 ceph-mon[31172]: from='client.? 172.21.15.26:0/1671480159' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:18:05.007 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:18:05.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:04 smithi136 ceph-mon[35121]: from='client.? 172.21.15.26:0/1671480159' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:18:05.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:04 smithi076 ceph-mon[35296]: from='client.? 172.21.15.26:0/1671480159' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:18:05.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:05.496+0000 7f68926dd640 1 -- 172.21.15.26:0/2960998990 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f688c102570 msgr2=0x7f688c1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:05.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:05.496+0000 7f68926dd640 1 --2- 172.21.15.26:0/2960998990 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f688c102570 0x7f688c1029b0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f6874009900 tx=0x7f687402a480 comp rx=0 tx=0).stop 2024-09-11T23:18:05.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:05.496+0000 7f68926dd640 1 -- 172.21.15.26:0/2960998990 shutdown_connections 2024-09-11T23:18:05.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:05.496+0000 7f68926dd640 1 --2- 172.21.15.26:0/2960998990 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f688c102ef0 0x7f688c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:05.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:05.496+0000 7f68926dd640 1 --2- 172.21.15.26:0/2960998990 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f688c102570 0x7f688c1029b0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:05.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:05.496+0000 7f68926dd640 1 --2- 172.21.15.26:0/2960998990 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f688c100e20 0x7f688c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:05.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:05.496+0000 7f68926dd640 1 -- 172.21.15.26:0/2960998990 >> 172.21.15.26:0/2960998990 conn(0x7f688c0fc7d0 msgr2=0x7f688c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:18:05.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:05.496+0000 7f68926dd640 1 -- 172.21.15.26:0/2960998990 shutdown_connections 2024-09-11T23:18:05.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:05.496+0000 7f68926dd640 1 -- 172.21.15.26:0/2960998990 wait complete. 2024-09-11T23:18:05.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:05.497+0000 7f68926dd640 1 Processor -- start 2024-09-11T23:18:05.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:05.497+0000 7f68926dd640 1 -- start start 2024-09-11T23:18:05.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:05.498+0000 7f68926dd640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f688c100e20 0x7f688c1992d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:05.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:05.498+0000 7f68926dd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f688c102570 0x7f688c199810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:05.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:05.498+0000 7f68926dd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f688c102ef0 0x7f688c1a4350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:05.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:05.498+0000 7f68926dd640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f688c110f80 con 0x7f688c102570 2024-09-11T23:18:05.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:05.498+0000 7f68926dd640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f688c110e00 con 0x7f688c102ef0 2024-09-11T23:18:05.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:05.498+0000 7f68926dd640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f688c111100 con 0x7f688c100e20 2024-09-11T23:18:05.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:05.498+0000 7f6883fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f688c102570 0x7f688c199810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:05.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:05.498+0000 7f688bfff640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f688c100e20 0x7f688c1992d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:05.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:05.498+0000 7f6883fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f688c102570 0x7f688c199810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33212/0 (socket says 172.21.15.26:33212) 2024-09-11T23:18:05.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:05.498+0000 7f6883fff640 1 -- 172.21.15.26:0/3057548469 learned_addr learned my addr 172.21.15.26:0/3057548469 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:18:05.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:05.498+0000 7f6890c7a640 1 --2- 172.21.15.26:0/3057548469 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f688c102ef0 0x7f688c1a4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:05.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:05.498+0000 7f6883fff640 1 -- 172.21.15.26:0/3057548469 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f688c100e20 msgr2=0x7f688c1992d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:05.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:05.498+0000 7f6883fff640 1 --2- 172.21.15.26:0/3057548469 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f688c100e20 0x7f688c1992d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:05.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:05.498+0000 7f6883fff640 1 -- 172.21.15.26:0/3057548469 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f688c102ef0 msgr2=0x7f688c1a4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:05.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:05.498+0000 7f6883fff640 1 --2- 172.21.15.26:0/3057548469 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f688c102ef0 0x7f688c1a4350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:05.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:05.498+0000 7f6883fff640 1 -- 172.21.15.26:0/3057548469 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f688c1a4890 con 0x7f688c102570 2024-09-11T23:18:05.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:05.498+0000 7f6883fff640 1 --2- 172.21.15.26:0/3057548469 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f688c102570 0x7f688c199810 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f6874006010 tx=0x7f6874003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:18:05.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:05.499+0000 7f6889ffb640 1 -- 172.21.15.26:0/3057548469 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f687403f070 con 0x7f688c102570 2024-09-11T23:18:05.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:05.499+0000 7f6889ffb640 1 -- 172.21.15.26:0/3057548469 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6874031c30 con 0x7f688c102570 2024-09-11T23:18:05.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:05.499+0000 7f68926dd640 1 -- 172.21.15.26:0/3057548469 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f688c1a4a60 con 0x7f688c102570 2024-09-11T23:18:05.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:05.499+0000 7f6889ffb640 1 -- 172.21.15.26:0/3057548469 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6874035030 con 0x7f688c102570 2024-09-11T23:18:05.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:05.499+0000 7f68926dd640 1 -- 172.21.15.26:0/3057548469 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f688c1a4f10 con 0x7f688c102570 2024-09-11T23:18:05.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:05.500+0000 7f68926dd640 1 -- 172.21.15.26:0/3057548469 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f688c101200 con 0x7f688c102570 2024-09-11T23:18:05.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:05.501+0000 7f6889ffb640 1 -- 172.21.15.26:0/3057548469 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f6874042070 con 0x7f688c102570 2024-09-11T23:18:05.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:05.502+0000 7f6889ffb640 1 --2- 172.21.15.26:0/3057548469 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f68780a2b80 0x7f68780a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:05.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:05.502+0000 7f688bfff640 1 --2- 172.21.15.26:0/3057548469 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f68780a2b80 0x7f68780a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:05.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:05.502+0000 7f6889ffb640 1 -- 172.21.15.26:0/3057548469 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f68740efc10 con 0x7f688c102570 2024-09-11T23:18:05.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:05.503+0000 7f688bfff640 1 --2- 172.21.15.26:0/3057548469 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f68780a2b80 0x7f68780a5040 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f687c00bbe0 tx=0x7f687c00b390 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:18:05.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:05.506+0000 7f6889ffb640 1 -- 172.21.15.26:0/3057548469 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6874031de0 con 0x7f688c102570 2024-09-11T23:18:05.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:05.683+0000 7f68926dd640 1 -- 172.21.15.26:0/3057548469 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f688c1a5140 con 0x7f688c102570 2024-09-11T23:18:05.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:05.684+0000 7f6889ffb640 1 -- 172.21.15.26:0/3057548469 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+22733 (secure 0 0 0) 0x7f68740bc520 con 0x7f688c102570 2024-09-11T23:18:05.685 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-11T23:18:05.685 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":85,"fsid":"af7e6fc8-7092-11ef-bcea-c7b262605968","created":"2024-09-11T23:08:40.486438+0000","modified":"2024-09-11T23:16:35.935151+0000","last_up_change":"2024-09-11T23:15:35.179446+0000","last_in_change":"2024-09-11T23:15:11.064627+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T23:11:50.494160+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-11T23:16:22.503928+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-11T23:16:24.876554+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a4d62fcf-c31e-41b1-932c-3d87ff640531","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":2014857905},{"type":"v1","addr":"172.21.15.26:6803","nonce":2014857905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":2014857905},{"type":"v1","addr":"172.21.15.26:6805","nonce":2014857905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":2014857905},{"type":"v1","addr":"172.21.15.26:6809","nonce":2014857905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":2014857905},{"type":"v1","addr":"172.21.15.26:6807","nonce":2014857905}]},"public_addr":"172.21.15.26:6803/2014857905","cluster_addr":"172.21.15.26:6805/2014857905","heartbeat_back_addr":"172.21.15.26:6809/2014857905","heartbeat_front_addr":"172.21.15.26:6807/2014857905","state":["exists","up"]},{"osd":1,"uuid":"e0d69112-3c0a-4fc5-8783-e41b4081e0c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":333383087},{"type":"v1","addr":"172.21.15.76:6801","nonce":333383087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":333383087},{"type":"v1","addr":"172.21.15.76:6803","nonce":333383087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":333383087},{"type":"v1","addr":"172.21.15.76:6807","nonce":333383087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":333383087},{"type":"v1","addr":"172.21.15.76:6805","nonce":333383087}]},"public_addr":"172.21.15.76:6801/333383087","cluster_addr":"172.21.15.76:6803/333383087","heartbeat_back_addr":"172.21.15.76:6807/333383087","heartbeat_front_addr":"172.21.15.76:6805/333383087","state":["exists","up"]},{"osd":2,"uuid":"d578429e-bf23-4a1e-890f-7172955c0eb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":1271538911},{"type":"v1","addr":"172.21.15.136:6801","nonce":1271538911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":1271538911},{"type":"v1","addr":"172.21.15.136:6803","nonce":1271538911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":1271538911},{"type":"v1","addr":"172.21.15.136:6807","nonce":1271538911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":1271538911},{"type":"v1","addr":"172.21.15.136:6805","nonce":1271538911}]},"public_addr":"172.21.15.136:6801/1271538911","cluster_addr":"172.21.15.136:6803/1271538911","heartbeat_back_addr":"172.21.15.136:6807/1271538911","heartbeat_front_addr":"172.21.15.136:6805/1271538911","state":["exists","up"]},{"osd":3,"uuid":"3e34623f-0ea5-4b10-b8a9-f8756c35d512","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":1740304151},{"type":"v1","addr":"172.21.15.26:6811","nonce":1740304151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":1740304151},{"type":"v1","addr":"172.21.15.26:6813","nonce":1740304151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1740304151},{"type":"v1","addr":"172.21.15.26:6817","nonce":1740304151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":1740304151},{"type":"v1","addr":"172.21.15.26:6815","nonce":1740304151}]},"public_addr":"172.21.15.26:6811/1740304151","cluster_addr":"172.21.15.26:6813/1740304151","heartbeat_back_addr":"172.21.15.26:6817/1740304151","heartbeat_front_addr":"172.21.15.26:6815/1740304151","state":["exists","up"]},{"osd":4,"uuid":"77607929-f9ed-49a9-abb5-48cd88eec4ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":2702262182},{"type":"v1","addr":"172.21.15.76:6809","nonce":2702262182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":2702262182},{"type":"v1","addr":"172.21.15.76:6811","nonce":2702262182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":2702262182},{"type":"v1","addr":"172.21.15.76:6815","nonce":2702262182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":2702262182},{"type":"v1","addr":"172.21.15.76:6813","nonce":2702262182}]},"public_addr":"172.21.15.76:6809/2702262182","cluster_addr":"172.21.15.76:6811/2702262182","heartbeat_back_addr":"172.21.15.76:6815/2702262182","heartbeat_front_addr":"172.21.15.76:6813/2702262182","state":["exists","up"]},{"osd":5,"uuid":"29e20e37-09a0-4924-8aaa-17939763a7c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":1749229277},{"type":"v1","addr":"172.21.15.136:6809","nonce":1749229277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":1749229277},{"type":"v1","addr":"172.21.15.136:6811","nonce":1749229277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":1749229277},{"type":"v1","addr":"172.21.15.136:6815","nonce":1749229277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":1749229277},{"type":"v1","addr":"172.21.15.136:6813","nonce":1749229277}]},"public_addr":"172.21.15.136:6809/1749229277","cluster_addr":"172.21.15.136:6811/1749229277","heartbeat_back_addr":"172.21.15.136:6815/1749229277","heartbeat_front_addr":"172.21.15.136:6813/1749229277","state":["exists","up"]},{"osd":6,"uuid":"e0ddd1c0-1c46-41e6-b2f4-396bdcd2bf8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1928245624},{"type":"v1","addr":"172.21.15.26:6819","nonce":1928245624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1928245624},{"type":"v1","addr":"172.21.15.26:6821","nonce":1928245624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":1928245624},{"type":"v1","addr":"172.21.15.26:6825","nonce":1928245624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1928245624},{"type":"v1","addr":"172.21.15.26:6823","nonce":1928245624}]},"public_addr":"172.21.15.26:6819/1928245624","cluster_addr":"172.21.15.26:6821/1928245624","heartbeat_back_addr":"172.21.15.26:6825/1928245624","heartbeat_front_addr":"172.21.15.26:6823/1928245624","state":["exists","up"]},{"osd":7,"uuid":"fe031f5c-d915-48fb-91a6-c76f252467fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":1409789455},{"type":"v1","addr":"172.21.15.76:6817","nonce":1409789455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":1409789455},{"type":"v1","addr":"172.21.15.76:6819","nonce":1409789455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":1409789455},{"type":"v1","addr":"172.21.15.76:6823","nonce":1409789455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":1409789455},{"type":"v1","addr":"172.21.15.76:6821","nonce":1409789455}]},"public_addr":"172.21.15.76:6817/1409789455","cluster_addr":"172.21.15.76:6819/1409789455","heartbeat_back_addr":"172.21.15.76:6823/1409789455","heartbeat_front_addr":"172.21.15.76:6821/1409789455","state":["exists","up"]},{"osd":8,"uuid":"18851bde-9e6c-4e05-b4bf-ac5c3073eee7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":543619079},{"type":"v1","addr":"172.21.15.136:6817","nonce":543619079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":543619079},{"type":"v1","addr":"172.21.15.136:6819","nonce":543619079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":543619079},{"type":"v1","addr":"172.21.15.136:6823","nonce":543619079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":543619079},{"type":"v1","addr":"172.21.15.136:6821","nonce":543619079}]},"public_addr":"172.21.15.136:6817/543619079","cluster_addr":"172.21.15.136:6819/543619079","heartbeat_back_addr":"172.21.15.136:6823/543619079","heartbeat_front_addr":"172.21.15.136:6821/543619079","state":["exists","up"]},{"osd":9,"uuid":"52a65eab-8446-47ad-bfe7-5133f7e3eb98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":2201297064},{"type":"v1","addr":"172.21.15.26:6827","nonce":2201297064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":2201297064},{"type":"v1","addr":"172.21.15.26:6829","nonce":2201297064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":2201297064},{"type":"v1","addr":"172.21.15.26:6833","nonce":2201297064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":2201297064},{"type":"v1","addr":"172.21.15.26:6831","nonce":2201297064}]},"public_addr":"172.21.15.26:6827/2201297064","cluster_addr":"172.21.15.26:6829/2201297064","heartbeat_back_addr":"172.21.15.26:6833/2201297064","heartbeat_front_addr":"172.21.15.26:6831/2201297064","state":["exists","up"]},{"osd":10,"uuid":"b758d4bb-da62-4c5f-a992-bb6a145d35d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":3184128096},{"type":"v1","addr":"172.21.15.76:6825","nonce":3184128096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":3184128096},{"type":"v1","addr":"172.21.15.76:6827","nonce":3184128096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":3184128096},{"type":"v1","addr":"172.21.15.76:6831","nonce":3184128096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":3184128096},{"type":"v1","addr":"172.21.15.76:6829","nonce":3184128096}]},"public_addr":"172.21.15.76:6825/3184128096","cluster_addr":"172.21.15.76:6827/3184128096","heartbeat_back_addr":"172.21.15.76:6831/3184128096","heartbeat_front_addr":"172.21.15.76:6829/3184128096","state":["exists","up"]},{"osd":11,"uuid":"8fddda46-e22b-4d0c-ac48-e038fb29dfec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":1960689685},{"type":"v1","addr":"172.21.15.136:6825","nonce":1960689685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":1960689685},{"type":"v1","addr":"172.21.15.136:6827","nonce":1960689685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":1960689685},{"type":"v1","addr":"172.21.15.136:6831","nonce":1960689685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":1960689685},{"type":"v1","addr":"172.21.15.136:6829","nonce":1960689685}]},"public_addr":"172.21.15.136:6825/1960689685","cluster_addr":"172.21.15.136:6827/1960689685","heartbeat_back_addr":"172.21.15.136:6831/1960689685","heartbeat_front_addr":"172.21.15.136:6829/1960689685","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:10:56.656177+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:11:22.234031+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:11:46.464102+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:12:12.353967+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:12:37.836200+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:13:01.728839+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:13:27.527230+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:13:51.981393+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:14:17.035464+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:14:42.805029+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:15:08.208999+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:15:32.800844+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.18","mappings":[{"from":3,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":6}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.26:6801/677148151":"2024-09-12T23:09:33.983310+0000","172.21.15.26:6800/677148151":"2024-09-12T23:09:33.983310+0000","172.21.15.26:0/3597996003":"2024-09-12T23:09:33.983310+0000","172.21.15.26:6801/2673575521":"2024-09-12T23:09:09.781665+0000","172.21.15.26:0/883212708":"2024-09-12T23:09:33.983310+0000","172.21.15.26:0/2107281413":"2024-09-12T23:09:33.983310+0000","172.21.15.26:6800/2673575521":"2024-09-12T23:09:09.781665+0000","172.21.15.26:0/2817092653":"2024-09-12T23:09:09.781665+0000","172.21.15.26:0/17142658":"2024-09-12T23:09:09.781665+0000","172.21.15.26:0/3861103130":"2024-09-12T23:09:09.781665+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T23:18:05.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:05.686+0000 7f68926dd640 1 -- 172.21.15.26:0/3057548469 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f68780a2b80 msgr2=0x7f68780a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:05.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:05.686+0000 7f68926dd640 1 --2- 172.21.15.26:0/3057548469 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f68780a2b80 0x7f68780a5040 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f687c00bbe0 tx=0x7f687c00b390 comp rx=0 tx=0).stop 2024-09-11T23:18:05.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:05.686+0000 7f68926dd640 1 -- 172.21.15.26:0/3057548469 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f688c102570 msgr2=0x7f688c199810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:05.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:05.686+0000 7f68926dd640 1 --2- 172.21.15.26:0/3057548469 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f688c102570 0x7f688c199810 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f6874006010 tx=0x7f6874003c10 comp rx=0 tx=0).stop 2024-09-11T23:18:05.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:05.687+0000 7f68926dd640 1 -- 172.21.15.26:0/3057548469 shutdown_connections 2024-09-11T23:18:05.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:05.687+0000 7f68926dd640 1 --2- 172.21.15.26:0/3057548469 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f68780a2b80 0x7f68780a5040 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:05.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:05.687+0000 7f68926dd640 1 --2- 172.21.15.26:0/3057548469 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f688c102ef0 0x7f688c1a4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:05.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:05.687+0000 7f68926dd640 1 --2- 172.21.15.26:0/3057548469 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f688c102570 0x7f688c199810 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:05.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:05.687+0000 7f68926dd640 1 --2- 172.21.15.26:0/3057548469 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f688c100e20 0x7f688c1992d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:05.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:05.687+0000 7f68926dd640 1 -- 172.21.15.26:0/3057548469 >> 172.21.15.26:0/3057548469 conn(0x7f688c0fc7d0 msgr2=0x7f688c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:18:05.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:05.687+0000 7f68926dd640 1 -- 172.21.15.26:0/3057548469 shutdown_connections 2024-09-11T23:18:05.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:05.687+0000 7f68926dd640 1 -- 172.21.15.26:0/3057548469 wait complete. 2024-09-11T23:18:06.167 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:05 smithi026 ceph-mon[31172]: pgmap v330: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-11T23:18:06.167 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:05 smithi026 ceph-mon[31172]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-09-11T23:18:06.167 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:05 smithi026 ceph-mon[31172]: from='client.? 172.21.15.26:0/3057548469' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T23:18:06.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:05 smithi136 ceph-mon[35121]: pgmap v330: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-11T23:18:06.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:05 smithi136 ceph-mon[35121]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-09-11T23:18:06.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:05 smithi136 ceph-mon[35121]: from='client.? 172.21.15.26:0/3057548469' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T23:18:06.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:05 smithi076 ceph-mon[35296]: pgmap v330: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-11T23:18:06.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:05 smithi076 ceph-mon[35296]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-09-11T23:18:06.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:05 smithi076 ceph-mon[35296]: from='client.? 172.21.15.26:0/3057548469' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T23:18:06.382 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 28, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-11T23:16:26.544203+0000', 'modified': '2024-09-11T23:16:38.976970+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24485, 'mds_1': 24463, 'mds_2': 24458, 'mds_3': 24448, 'mds_4': 24443}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14550': {'gid': 14550, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.26:6839/2233068305', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6838', 'nonce': 2233068305}, {'type': 'v1', 'addr': '172.21.15.26:6839', 'nonce': 2233068305}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14562': {'gid': 14562, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.26:6841/2984982336', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6840', 'nonce': 2984982336}, {'type': 'v1', 'addr': '172.21.15.26:6841', 'nonce': 2984982336}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24403': {'gid': 24403, 'name': 'l', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.136:6833/2019841455', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6832', 'nonce': 2019841455}, {'type': 'v1', 'addr': '172.21.15.136:6833', 'nonce': 2019841455}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24407': {'gid': 24407, 'name': 'k', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.76:6833/3438147050', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6832', 'nonce': 3438147050}, {'type': 'v1', 'addr': '172.21.15.76:6833', 'nonce': 3438147050}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24439': {'gid': 24439, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.136:6835/3617665975', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6834', 'nonce': 3617665975}, {'type': 'v1', 'addr': '172.21.15.136:6835', 'nonce': 3617665975}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24443': {'gid': 24443, 'name': 'e', 'rank': 4, 'incarnation': 26, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.76:6835/393308499', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6834', 'nonce': 393308499}, {'type': 'v1', 'addr': '172.21.15.76:6835', 'nonce': 393308499}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24448': {'gid': 24448, 'name': 'c', 'rank': 3, 'incarnation': 24, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.136:6837/1633082387', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6836', 'nonce': 1633082387}, {'type': 'v1', 'addr': '172.21.15.136:6837', 'nonce': 1633082387}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24458': {'gid': 24458, 'name': 'h', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.76:6837/2078978748', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6836', 'nonce': 2078978748}, {'type': 'v1', 'addr': '172.21.15.76:6837', 'nonce': 2078978748}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24463': {'gid': 24463, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.136:6839/3211398630', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6838', 'nonce': 3211398630}, {'type': 'v1', 'addr': '172.21.15.136:6839', 'nonce': 3211398630}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24485': {'gid': 24485, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.76:6839/2077637090', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6838', 'nonce': 2077637090}, {'type': 'v1', 'addr': '172.21.15.76:6839', 'nonce': 2077637090}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24485, 'qdb_cluster': [24485, 24463, 24458, 24448, 24443]} 2024-09-11T23:18:06.382 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-09-11T23:18:06.383 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph tell mds.e status 2024-09-11T23:18:06.642 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:18:07.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.168+0000 7f71fcea9640 1 -- 172.21.15.26:0/638449717 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f71f8100a80 msgr2=0x7f71f8100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:07.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.168+0000 7f71fcea9640 1 --2- 172.21.15.26:0/638449717 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f71f8100a80 0x7f71f8100ee0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f71ec009930 tx=0x7f71ec02a420 comp rx=0 tx=0).stop 2024-09-11T23:18:07.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.168+0000 7f71fcea9640 1 -- 172.21.15.26:0/638449717 shutdown_connections 2024-09-11T23:18:07.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.168+0000 7f71fcea9640 1 --2- 172.21.15.26:0/638449717 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f71f8101420 0x7f71f810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:07.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.168+0000 7f71fcea9640 1 --2- 172.21.15.26:0/638449717 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f71f8100a80 0x7f71f8100ee0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:07.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.168+0000 7f71fcea9640 1 --2- 172.21.15.26:0/638449717 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f71f8106a80 0x7f71f8106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:07.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.168+0000 7f71fcea9640 1 -- 172.21.15.26:0/638449717 >> 172.21.15.26:0/638449717 conn(0x7f71f80fc7a0 msgr2=0x7f71f80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:18:07.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.168+0000 7f71fcea9640 1 -- 172.21.15.26:0/638449717 shutdown_connections 2024-09-11T23:18:07.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.168+0000 7f71fcea9640 1 -- 172.21.15.26:0/638449717 wait complete. 2024-09-11T23:18:07.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.169+0000 7f71fcea9640 1 Processor -- start 2024-09-11T23:18:07.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.169+0000 7f71fcea9640 1 -- start start 2024-09-11T23:18:07.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.169+0000 7f71fcea9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f71f8100a80 0x7f71f80fcda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:07.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.169+0000 7f71fcea9640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f71f8101420 0x7f71f80fd2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:07.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.169+0000 7f71fcea9640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f71f8106a80 0x7f71f80fd820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:07.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.169+0000 7f71fcea9640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f71f8110e80 con 0x7f71f8100a80 2024-09-11T23:18:07.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.169+0000 7f71fcea9640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f71f8110d00 con 0x7f71f8106a80 2024-09-11T23:18:07.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.169+0000 7f71fcea9640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f71f8111000 con 0x7f71f8101420 2024-09-11T23:18:07.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.169+0000 7f71f659c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f71f8100a80 0x7f71f80fcda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:07.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.169+0000 7f71f659c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f71f8100a80 0x7f71f80fcda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33234/0 (socket says 172.21.15.26:33234) 2024-09-11T23:18:07.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.169+0000 7f71f659c640 1 -- 172.21.15.26:0/3308495499 learned_addr learned my addr 172.21.15.26:0/3308495499 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:18:07.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.169+0000 7f71f6d9d640 1 --2- 172.21.15.26:0/3308495499 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f71f8106a80 0x7f71f80fd820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:07.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.169+0000 7f71f659c640 1 -- 172.21.15.26:0/3308495499 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f71f8101420 msgr2=0x7f71f80fd2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:07.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.169+0000 7f71f659c640 1 --2- 172.21.15.26:0/3308495499 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f71f8101420 0x7f71f80fd2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:07.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.169+0000 7f71f659c640 1 -- 172.21.15.26:0/3308495499 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f71f8106a80 msgr2=0x7f71f80fd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:07.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.169+0000 7f71f659c640 1 --2- 172.21.15.26:0/3308495499 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f71f8106a80 0x7f71f80fd820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:07.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.169+0000 7f71f659c640 1 -- 172.21.15.26:0/3308495499 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71f81a68c0 con 0x7f71f8100a80 2024-09-11T23:18:07.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.170+0000 7f71f659c640 1 --2- 172.21.15.26:0/3308495499 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f71f8100a80 0x7f71f80fcda0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f71e800d950 tx=0x7f71e800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:18:07.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.170+0000 7f71e77fe640 1 -- 172.21.15.26:0/3308495499 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71e8017070 con 0x7f71f8100a80 2024-09-11T23:18:07.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.170+0000 7f71e77fe640 1 -- 172.21.15.26:0/3308495499 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71e8009c30 con 0x7f71f8100a80 2024-09-11T23:18:07.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.170+0000 7f71fcea9640 1 -- 172.21.15.26:0/3308495499 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71f81a6bb0 con 0x7f71f8100a80 2024-09-11T23:18:07.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.170+0000 7f71e77fe640 1 -- 172.21.15.26:0/3308495499 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71e800f6b0 con 0x7f71f8100a80 2024-09-11T23:18:07.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.170+0000 7f71fcea9640 1 -- 172.21.15.26:0/3308495499 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71f81a7140 con 0x7f71f8100a80 2024-09-11T23:18:07.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.173+0000 7f71e77fe640 1 -- 172.21.15.26:0/3308495499 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f71e80109d0 con 0x7f71f8100a80 2024-09-11T23:18:07.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.173+0000 7f71ff282740 1 Processor -- start 2024-09-11T23:18:07.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.173+0000 7f71ff282740 1 -- start start 2024-09-11T23:18:07.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.174+0000 7f71ff282740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e33a62b8f0 0x55e33a62bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:07.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.174+0000 7f71ff282740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e33a62c210 0x55e33a63ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:07.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.174+0000 7f71ff282740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e33a63f3c0 0x55e33a6417b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:07.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.174+0000 7f71ff282740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55e33a592090 con 0x55e33a62b8f0 2024-09-11T23:18:07.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.174+0000 7f71ff282740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55e33a539a70 con 0x55e33a63f3c0 2024-09-11T23:18:07.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.174+0000 7f71ff282740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55e33a58e050 con 0x55e33a62c210 2024-09-11T23:18:07.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.174+0000 7f71f6d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e33a62b8f0 0x55e33a62bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:07.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.174+0000 7f71f6d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e33a62b8f0 0x55e33a62bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33244/0 (socket says 172.21.15.26:33244) 2024-09-11T23:18:07.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.174+0000 7f71f6d9d640 1 -- 172.21.15.26:0/442757917 learned_addr learned my addr 172.21.15.26:0/442757917 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:18:07.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.174+0000 7f71e77fe640 1 --2- 172.21.15.26:0/3308495499 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f71e00a2c00 0x7f71e00a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:07.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.174+0000 7f71f659c640 1 --2- 172.21.15.26:0/442757917 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e33a62c210 0x55e33a63ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:07.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.174+0000 7f71f5d9b640 1 --2- 172.21.15.26:0/442757917 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e33a63f3c0 0x55e33a6417b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:07.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.174+0000 7f71f6d9d640 1 -- 172.21.15.26:0/442757917 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e33a62c210 msgr2=0x55e33a63ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:18:07.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.174+0000 7f71f6d9d640 1 --2- 172.21.15.26:0/442757917 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e33a62c210 0x55e33a63ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:07.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.174+0000 7f71f6d9d640 1 -- 172.21.15.26:0/442757917 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e33a63f3c0 msgr2=0x55e33a6417b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:18:07.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.174+0000 7f71f6d9d640 1 --2- 172.21.15.26:0/442757917 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e33a63f3c0 0x55e33a6417b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:07.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.174+0000 7f71e77fe640 1 -- 172.21.15.26:0/3308495499 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f71e80cec10 con 0x7f71f8100a80 2024-09-11T23:18:07.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.174+0000 7f71f6d9d640 1 -- 172.21.15.26:0/442757917 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e33a546130 con 0x55e33a62b8f0 2024-09-11T23:18:07.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.174+0000 7f71f6d9d640 1 --2- 172.21.15.26:0/3308495499 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f71e00a2c00 0x7f71e00a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:07.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.174+0000 7f71f6d9d640 1 --2- 172.21.15.26:0/442757917 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e33a62b8f0 0x55e33a62bcd0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f71f0007fd0 tx=0x7f71f000e7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c492205398d152b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:18:07.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.175+0000 7f71e4ff9640 1 -- 172.21.15.26:0/442757917 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71f0022070 con 0x55e33a62b8f0 2024-09-11T23:18:07.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.175+0000 7f71e4ff9640 1 -- 172.21.15.26:0/442757917 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71f0011c30 con 0x55e33a62b8f0 2024-09-11T23:18:07.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.175+0000 7f71e4ff9640 1 -- 172.21.15.26:0/442757917 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71f001a600 con 0x55e33a62b8f0 2024-09-11T23:18:07.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.175+0000 7f71f6d9d640 1 --2- 172.21.15.26:0/3308495499 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f71e00a2c00 0x7f71e00a50c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f71f001f000 tx=0x7f71f0020000 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:18:07.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.175+0000 7f71ff282740 1 -- 172.21.15.26:0/442757917 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e33a62b8f0 msgr2=0x55e33a62bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:07.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.175+0000 7f71ff282740 1 --2- 172.21.15.26:0/442757917 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e33a62b8f0 0x55e33a62bcd0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f71f0007fd0 tx=0x7f71f000e7e0 comp rx=0 tx=0).stop 2024-09-11T23:18:07.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.175+0000 7f71ff282740 1 -- 172.21.15.26:0/442757917 shutdown_connections 2024-09-11T23:18:07.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.175+0000 7f71ff282740 1 --2- 172.21.15.26:0/442757917 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e33a63f3c0 0x55e33a6417b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:07.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.175+0000 7f71ff282740 1 --2- 172.21.15.26:0/442757917 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e33a62c210 0x55e33a63ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:07.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.175+0000 7f71ff282740 1 --2- 172.21.15.26:0/442757917 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e33a62b8f0 0x55e33a62bcd0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:07.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.175+0000 7f71ff282740 1 -- 172.21.15.26:0/442757917 >> 172.21.15.26:0/442757917 conn(0x55e33a627f60 msgr2=0x55e33a628340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:18:07.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.176+0000 7f71ff282740 1 -- 172.21.15.26:0/442757917 shutdown_connections 2024-09-11T23:18:07.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.176+0000 7f71ff282740 1 -- 172.21.15.26:0/442757917 wait complete. 2024-09-11T23:18:07.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.176+0000 7f71ff282740 1 -- start start 2024-09-11T23:18:07.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.176+0000 7f71ff282740 1 Processor -- start 2024-09-11T23:18:07.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.176+0000 7f71ff282740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e33a62b8f0 0x55e33a63eca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:07.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.176+0000 7f71ff282740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e33a62c210 0x55e33a63bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:07.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.176+0000 7f71ff282740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e33a63f3c0 0x55e33a63c530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:07.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.176+0000 7f71ff282740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55e33a641f30 con 0x55e33a62c210 2024-09-11T23:18:07.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.176+0000 7f71ff282740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55e33a54cf20 con 0x55e33a63f3c0 2024-09-11T23:18:07.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.176+0000 7f71ff282740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55e33a6420b0 con 0x55e33a62b8f0 2024-09-11T23:18:07.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.176+0000 7f71ff282740 10 client.0 _ll_register_callbacks cb 0x55e33a516030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:18:07.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.176+0000 7f71f5d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e33a62c210 0x55e33a63bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:07.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.176+0000 7f71f5d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e33a62c210 0x55e33a63bff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33246/0 (socket says 172.21.15.26:33246) 2024-09-11T23:18:07.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.176+0000 7f71f5d9b640 1 -- 172.21.15.26:0/3757429114 learned_addr learned my addr 172.21.15.26:0/3757429114 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:18:07.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.176+0000 7f71f6d9d640 1 --2- 172.21.15.26:0/3757429114 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e33a63f3c0 0x55e33a63c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:07.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.177+0000 7f71f659c640 1 --2- 172.21.15.26:0/3757429114 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e33a62b8f0 0x55e33a63eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:07.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.177+0000 7f71f5d9b640 1 -- 172.21.15.26:0/3757429114 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e33a62b8f0 msgr2=0x55e33a63eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:18:07.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.177+0000 7f71f5d9b640 1 --2- 172.21.15.26:0/3757429114 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e33a62b8f0 0x55e33a63eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:07.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.177+0000 7f71f5d9b640 1 -- 172.21.15.26:0/3757429114 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e33a63f3c0 msgr2=0x55e33a63c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:18:07.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.177+0000 7f71f5d9b640 1 --2- 172.21.15.26:0/3757429114 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e33a63f3c0 0x55e33a63c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:07.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.177+0000 7f71f5d9b640 1 -- 172.21.15.26:0/3757429114 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e33a3032b0 con 0x55e33a62c210 2024-09-11T23:18:07.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.177+0000 7f71f5d9b640 1 --2- 172.21.15.26:0/3757429114 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e33a62c210 0x55e33a63bff0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f71ec009930 tx=0x7f71ec0317e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45a41c981c1359f8 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:18:07.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.177+0000 7f71ddffb640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:18:07.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.177+0000 7f71ddffb640 1 -- 172.21.15.26:0/3757429114 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71ec040070 con 0x55e33a62c210 2024-09-11T23:18:07.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.177+0000 7f71ddffb640 1 -- 172.21.15.26:0/3757429114 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f71ec004230 con 0x55e33a62c210 2024-09-11T23:18:07.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.177+0000 7f71ff282740 1 -- 172.21.15.26:0/3757429114 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f71f0011c30 con 0x55e33a62c210 2024-09-11T23:18:07.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.177+0000 7f71ddffb640 1 -- 172.21.15.26:0/3757429114 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f71ec04c050 con 0x55e33a62c210 2024-09-11T23:18:07.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.177+0000 7f71ff282740 10 client.15069 fetch_fsmap learned FSMap version 28 2024-09-11T23:18:07.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.177+0000 7f71ff282740 1 -- 172.21.15.26:0/3757429114 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e33a64d960 con 0x55e33a62c210 2024-09-11T23:18:07.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.177+0000 7f71ddffb640 1 -- 172.21.15.26:0/3757429114 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71f0011c30 con 0x55e33a62c210 2024-09-11T23:18:07.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.178+0000 7f71ddffb640 1 -- 172.21.15.26:0/3757429114 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f71ec03a040 con 0x55e33a62c210 2024-09-11T23:18:07.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.178+0000 7f71ff282740 10 client.15069 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:18:07.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.178+0000 7f71ff282740 10 client.15069 resolve_mds: resolved name 'e' to daemon mds.e 2024-09-11T23:18:07.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.178+0000 7f71ff282740 20 client.15069 populate_metadata read hostname 'smithi026' 2024-09-11T23:18:07.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.178+0000 7f71ff282740 1 --2- 172.21.15.26:0/3757429114 >> [v2:172.21.15.76:6834/393308499,v1:172.21.15.76:6835/393308499] conn(0x55e33a64ffb0 0x55e33a651420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:07.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.178+0000 7f71ff282740 4 client.15069 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:18:07.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.178+0000 7f71ff282740 1 -- 172.21.15.26:0/3757429114 --> [v2:172.21.15.76:6834/393308499,v1:172.21.15.76:6835/393308499] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e33a59f600 con 0x55e33a64ffb0 2024-09-11T23:18:07.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.178+0000 7f71f659c640 1 --2- 172.21.15.26:0/3757429114 >> [v2:172.21.15.76:6834/393308499,v1:172.21.15.76:6835/393308499] conn(0x55e33a64ffb0 0x55e33a651420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:07.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.178+0000 7f71ddffb640 10 client.15069 ms_handle_connect on v2:172.21.15.76:6834/393308499 2024-09-11T23:18:07.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.178+0000 7f71f659c640 1 --2- 172.21.15.26:0/3757429114 >> [v2:172.21.15.76:6834/393308499,v1:172.21.15.76:6835/393308499] conn(0x55e33a64ffb0 0x55e33a651420 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=2b66de3d2f762473 server_cookie=5d6001081e490c56 in_seq=0 out_seq=0 2024-09-11T23:18:07.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.179+0000 7f71ddffb640 1 -- 172.21.15.26:0/3757429114 <== mds.4 v2:172.21.15.76:6834/393308499 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e33a59f600 con 0x55e33a64ffb0 2024-09-11T23:18:07.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.179+0000 7f71ddffb640 10 client.15069 handle_command_reply: tid=0 2024-09-11T23:18:07.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.179+0000 7f71ff282740 1 client.15069 shutdown 2024-09-11T23:18:07.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.179+0000 7f71ff282740 1 -- 172.21.15.26:0/3757429114 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e33a62c210 msgr2=0x55e33a63bff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:07.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.179+0000 7f71ff282740 1 --2- 172.21.15.26:0/3757429114 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e33a62c210 0x55e33a63bff0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f71ec009930 tx=0x7f71ec0317e0 comp rx=0 tx=0).stop 2024-09-11T23:18:07.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.179+0000 7f71ff282740 1 -- 172.21.15.26:0/3757429114 shutdown_connections 2024-09-11T23:18:07.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.179+0000 7f71ff282740 1 --2- 172.21.15.26:0/3757429114 >> [v2:172.21.15.76:6834/393308499,v1:172.21.15.76:6835/393308499] conn(0x55e33a64ffb0 0x55e33a651420 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:07.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.179+0000 7f71ff282740 1 --2- 172.21.15.26:0/3757429114 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e33a63f3c0 0x55e33a63c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:07.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.179+0000 7f71ff282740 1 --2- 172.21.15.26:0/3757429114 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e33a62c210 0x55e33a63bff0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:07.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.179+0000 7f71ddffb640 0 client.15069 ms_handle_reset on v2:172.21.15.76:6834/393308499 2024-09-11T23:18:07.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.179+0000 7f71ff282740 1 --2- 172.21.15.26:0/3757429114 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e33a62b8f0 0x55e33a63eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:07.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.179+0000 7f71ff282740 1 -- 172.21.15.26:0/3757429114 >> 172.21.15.26:0/3757429114 conn(0x55e33a627f60 msgr2=0x55e33a62dbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:18:07.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.179+0000 7f71ff282740 1 -- 172.21.15.26:0/3757429114 shutdown_connections 2024-09-11T23:18:07.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.179+0000 7f71ff282740 1 -- 172.21.15.26:0/3757429114 wait complete. 2024-09-11T23:18:07.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.180+0000 7f71ff282740 20 client.15069 trim_cache size 0 max 16384 2024-09-11T23:18:07.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.197+0000 7f71ff282740 1 Processor -- start 2024-09-11T23:18:07.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.197+0000 7f71ff282740 1 -- start start 2024-09-11T23:18:07.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.198+0000 7f71ff282740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e33a62b8f0 0x55e33a651420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:07.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.198+0000 7f71ff282740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e33a62c210 0x55e33a637530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:07.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.198+0000 7f71ff282740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e33a63f3c0 0x55e33a639250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:07.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.198+0000 7f71ff282740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55e33a64f760 con 0x55e33a63f3c0 2024-09-11T23:18:07.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.198+0000 7f71ff282740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55e33a64fe30 con 0x55e33a62b8f0 2024-09-11T23:18:07.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.198+0000 7f71ff282740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55e33a650500 con 0x55e33a62c210 2024-09-11T23:18:07.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.198+0000 7f71f6d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e33a63f3c0 0x55e33a639250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:07.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.198+0000 7f71f6d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e33a63f3c0 0x55e33a639250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33252/0 (socket says 172.21.15.26:33252) 2024-09-11T23:18:07.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.198+0000 7f71f6d9d640 1 -- 172.21.15.26:0/2762838541 learned_addr learned my addr 172.21.15.26:0/2762838541 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:18:07.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.198+0000 7f71f5d9b640 1 --2- 172.21.15.26:0/2762838541 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e33a62c210 0x55e33a637530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:07.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.198+0000 7f71f659c640 1 --2- 172.21.15.26:0/2762838541 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e33a62b8f0 0x55e33a651420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:07.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.198+0000 7f71f6d9d640 1 -- 172.21.15.26:0/2762838541 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e33a62c210 msgr2=0x55e33a637530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:18:07.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.198+0000 7f71f6d9d640 1 --2- 172.21.15.26:0/2762838541 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e33a62c210 0x55e33a637530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:07.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.198+0000 7f71f6d9d640 1 -- 172.21.15.26:0/2762838541 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e33a62b8f0 msgr2=0x55e33a651420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:18:07.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.198+0000 7f71f6d9d640 1 --2- 172.21.15.26:0/2762838541 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e33a62b8f0 0x55e33a651420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:07.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.198+0000 7f71f6d9d640 1 -- 172.21.15.26:0/2762838541 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e33a62ce40 con 0x55e33a63f3c0 2024-09-11T23:18:07.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.198+0000 7f71f5d9b640 1 --2- 172.21.15.26:0/2762838541 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e33a62c210 0x55e33a637530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:18:07.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.198+0000 7f71f6d9d640 1 --2- 172.21.15.26:0/2762838541 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e33a63f3c0 0x55e33a639250 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f71f0007fd0 tx=0x7f71f0026000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80b7dc344ad6929b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:18:07.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.198+0000 7f71dffff640 1 -- 172.21.15.26:0/2762838541 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71f0022040 con 0x55e33a63f3c0 2024-09-11T23:18:07.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.198+0000 7f71dffff640 1 -- 172.21.15.26:0/2762838541 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71f0035070 con 0x55e33a63f3c0 2024-09-11T23:18:07.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.198+0000 7f71dffff640 1 -- 172.21.15.26:0/2762838541 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71f001aac0 con 0x55e33a63f3c0 2024-09-11T23:18:07.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.199+0000 7f71ff282740 1 -- 172.21.15.26:0/2762838541 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e33a63f3c0 msgr2=0x55e33a639250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:07.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.199+0000 7f71ff282740 1 --2- 172.21.15.26:0/2762838541 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e33a63f3c0 0x55e33a639250 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f71f0007fd0 tx=0x7f71f0026000 comp rx=0 tx=0).stop 2024-09-11T23:18:07.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.199+0000 7f71ff282740 1 -- 172.21.15.26:0/2762838541 shutdown_connections 2024-09-11T23:18:07.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.199+0000 7f71ff282740 1 --2- 172.21.15.26:0/2762838541 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e33a63f3c0 0x55e33a639250 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:07.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.199+0000 7f71ff282740 1 --2- 172.21.15.26:0/2762838541 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e33a62c210 0x55e33a637530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:07.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.199+0000 7f71ff282740 1 --2- 172.21.15.26:0/2762838541 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e33a62b8f0 0x55e33a651420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:07.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.199+0000 7f71ff282740 1 -- 172.21.15.26:0/2762838541 >> 172.21.15.26:0/2762838541 conn(0x55e33a627f60 msgr2=0x55e33a636e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:18:07.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.199+0000 7f71ff282740 1 -- 172.21.15.26:0/2762838541 shutdown_connections 2024-09-11T23:18:07.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.199+0000 7f71ff282740 1 -- 172.21.15.26:0/2762838541 wait complete. 2024-09-11T23:18:07.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.199+0000 7f71ff282740 1 -- start start 2024-09-11T23:18:07.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.200+0000 7f71ff282740 1 Processor -- start 2024-09-11T23:18:07.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.200+0000 7f71ff282740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e33a62b8f0 0x55e33a659ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:07.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.200+0000 7f71ff282740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e33a62c210 0x55e33a6531f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:07.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.200+0000 7f71ff282740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e33a63f3c0 0x55e33a653820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:07.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.200+0000 7f71ff282740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55e33a6512a0 con 0x55e33a63f3c0 2024-09-11T23:18:07.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.200+0000 7f71ff282740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55e33a650bd0 con 0x55e33a62b8f0 2024-09-11T23:18:07.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.200+0000 7f71ff282740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55e33a651970 con 0x55e33a62c210 2024-09-11T23:18:07.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.200+0000 7f71ff282740 10 client.0 _ll_register_callbacks cb 0x7f71ec009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:18:07.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.200+0000 7f71f6d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e33a63f3c0 0x55e33a653820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:07.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.200+0000 7f71f659c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e33a62b8f0 0x55e33a659ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:07.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.200+0000 7f71f659c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e33a62b8f0 0x55e33a659ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:37930/0 (socket says 172.21.15.26:37930) 2024-09-11T23:18:07.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.200+0000 7f71f659c640 1 -- 172.21.15.26:0/3459769431 learned_addr learned my addr 172.21.15.26:0/3459769431 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:18:07.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.200+0000 7f71f5d9b640 1 --2- 172.21.15.26:0/3459769431 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e33a62c210 0x55e33a6531f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:07.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.200+0000 7f71f6d9d640 1 -- 172.21.15.26:0/3459769431 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e33a62c210 msgr2=0x55e33a6531f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:18:07.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.200+0000 7f71f6d9d640 1 --2- 172.21.15.26:0/3459769431 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e33a62c210 0x55e33a6531f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:07.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.200+0000 7f71f6d9d640 1 -- 172.21.15.26:0/3459769431 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e33a62b8f0 msgr2=0x55e33a659ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:18:07.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.200+0000 7f71f6d9d640 1 --2- 172.21.15.26:0/3459769431 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e33a62b8f0 0x55e33a659ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:07.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.200+0000 7f71f6d9d640 1 -- 172.21.15.26:0/3459769431 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e33a6579e0 con 0x55e33a63f3c0 2024-09-11T23:18:07.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.200+0000 7f71f659c640 1 --2- 172.21.15.26:0/3459769431 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e33a62b8f0 0x55e33a659ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:18:07.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.201+0000 7f71f6d9d640 1 --2- 172.21.15.26:0/3459769431 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e33a63f3c0 0x55e33a653820 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f71f000ece0 tx=0x7f71f001cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5166dcc12ab3ece2 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:18:07.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.201+0000 7f71ddffb640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:18:07.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.201+0000 7f71ddffb640 1 -- 172.21.15.26:0/3459769431 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71f0022070 con 0x55e33a63f3c0 2024-09-11T23:18:07.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.201+0000 7f71ddffb640 1 -- 172.21.15.26:0/3459769431 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f71f001a7e0 con 0x55e33a63f3c0 2024-09-11T23:18:07.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.201+0000 7f71ff282740 1 -- 172.21.15.26:0/3459769431 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f71f0035070 con 0x55e33a63f3c0 2024-09-11T23:18:07.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.201+0000 7f71ddffb640 1 -- 172.21.15.26:0/3459769431 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f71f001a9e0 con 0x55e33a63f3c0 2024-09-11T23:18:07.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.201+0000 7f71ff282740 10 client.15075 fetch_fsmap learned FSMap version 28 2024-09-11T23:18:07.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.201+0000 7f71ff282740 1 -- 172.21.15.26:0/3459769431 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e33a6ba7d0 con 0x55e33a63f3c0 2024-09-11T23:18:07.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.201+0000 7f71ddffb640 1 -- 172.21.15.26:0/3459769431 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71f0035070 con 0x55e33a63f3c0 2024-09-11T23:18:07.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.201+0000 7f71ddffb640 1 -- 172.21.15.26:0/3459769431 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f71f0019040 con 0x55e33a63f3c0 2024-09-11T23:18:07.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.201+0000 7f71ff282740 10 client.15075 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:18:07.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.201+0000 7f71ff282740 10 client.15075 resolve_mds: resolved name 'e' to daemon mds.e 2024-09-11T23:18:07.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.201+0000 7f71ff282740 20 client.15075 populate_metadata read hostname 'smithi026' 2024-09-11T23:18:07.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.201+0000 7f71ff282740 1 --2- 172.21.15.26:0/3459769431 >> [v2:172.21.15.76:6834/393308499,v1:172.21.15.76:6835/393308499] conn(0x55e33a6bec10 0x55e33a6bd920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:07.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.201+0000 7f71ff282740 4 client.15075 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-11T23:18:07.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.201+0000 7f71ff282740 1 -- 172.21.15.26:0/3459769431 --> [v2:172.21.15.76:6834/393308499,v1:172.21.15.76:6835/393308499] -- command(tid 0: {"prefix": "status"}) -- 0x55e33a599b10 con 0x55e33a6bec10 2024-09-11T23:18:07.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.202+0000 7f71f659c640 1 --2- 172.21.15.26:0/3459769431 >> [v2:172.21.15.76:6834/393308499,v1:172.21.15.76:6835/393308499] conn(0x55e33a6bec10 0x55e33a6bd920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:07.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.202+0000 7f71f659c640 1 --2- 172.21.15.26:0/3459769431 >> [v2:172.21.15.76:6834/393308499,v1:172.21.15.76:6835/393308499] conn(0x55e33a6bec10 0x55e33a6bd920 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c8934008d4583a2f server_cookie=ba40cdfc00264b15 in_seq=0 out_seq=0 2024-09-11T23:18:07.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.202+0000 7f71ddffb640 10 client.15075 ms_handle_connect on v2:172.21.15.76:6834/393308499 2024-09-11T23:18:07.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.202+0000 7f71ddffb640 1 -- 172.21.15.26:0/3459769431 <== mds.4 v2:172.21.15.76:6834/393308499 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55e33a599b10 con 0x55e33a6bec10 2024-09-11T23:18:07.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.202+0000 7f71ddffb640 10 client.15075 handle_command_reply: tid=0 2024-09-11T23:18:07.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.202+0000 7f71ff282740 1 client.15075 shutdown 2024-09-11T23:18:07.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.202+0000 7f71ff282740 1 -- 172.21.15.26:0/3459769431 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e33a63f3c0 msgr2=0x55e33a653820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:07.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.202+0000 7f71ff282740 1 --2- 172.21.15.26:0/3459769431 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e33a63f3c0 0x55e33a653820 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f71f000ece0 tx=0x7f71f001cd80 comp rx=0 tx=0).stop 2024-09-11T23:18:07.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.202+0000 7f71ff282740 1 -- 172.21.15.26:0/3459769431 shutdown_connections 2024-09-11T23:18:07.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.202+0000 7f71ff282740 1 --2- 172.21.15.26:0/3459769431 >> [v2:172.21.15.76:6834/393308499,v1:172.21.15.76:6835/393308499] conn(0x55e33a6bec10 0x55e33a6bd920 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:07.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.202+0000 7f71ff282740 1 --2- 172.21.15.26:0/3459769431 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e33a63f3c0 0x55e33a653820 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:07.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.202+0000 7f71ff282740 1 --2- 172.21.15.26:0/3459769431 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e33a62c210 0x55e33a6531f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:07.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.202+0000 7f71ff282740 1 --2- 172.21.15.26:0/3459769431 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e33a62b8f0 0x55e33a659ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:07.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.202+0000 7f71ff282740 1 -- 172.21.15.26:0/3459769431 >> 172.21.15.26:0/3459769431 conn(0x55e33a627f60 msgr2=0x55e33a637640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:18:07.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.202+0000 7f71ddffb640 0 client.15075 ms_handle_reset on v2:172.21.15.76:6834/393308499 2024-09-11T23:18:07.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.203+0000 7f71ff282740 1 -- 172.21.15.26:0/3459769431 shutdown_connections 2024-09-11T23:18:07.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.203+0000 7f71ff282740 1 -- 172.21.15.26:0/3459769431 wait complete. 2024-09-11T23:18:07.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.203+0000 7f71ff282740 20 client.15075 trim_cache size 0 max 16384 2024-09-11T23:18:07.212 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-11T23:18:07.213 INFO:teuthology.orchestra.run.smithi026.stdout: "cluster_fsid": "af7e6fc8-7092-11ef-bcea-c7b262605968", 2024-09-11T23:18:07.213 INFO:teuthology.orchestra.run.smithi026.stdout: "whoami": 4, 2024-09-11T23:18:07.213 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 24443, 2024-09-11T23:18:07.213 INFO:teuthology.orchestra.run.smithi026.stdout: "want_state": "up:active", 2024-09-11T23:18:07.213 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "up:active", 2024-09-11T23:18:07.213 INFO:teuthology.orchestra.run.smithi026.stdout: "fs_name": "cephfs", 2024-09-11T23:18:07.213 INFO:teuthology.orchestra.run.smithi026.stdout: "rank_uptime": 89.234891844000003, 2024-09-11T23:18:07.213 INFO:teuthology.orchestra.run.smithi026.stdout: "mdsmap_epoch": 28, 2024-09-11T23:18:07.213 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch": 85, 2024-09-11T23:18:07.214 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch_barrier": 85, 2024-09-11T23:18:07.214 INFO:teuthology.orchestra.run.smithi026.stdout: "uptime": 131.204058266 2024-09-11T23:18:07.214 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-11T23:18:07.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.203+0000 7f71fcea9640 1 -- 172.21.15.26:0/3308495499 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f71e00a2c00 msgr2=0x7f71e00a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:07.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.203+0000 7f71fcea9640 1 --2- 172.21.15.26:0/3308495499 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f71e00a2c00 0x7f71e00a50c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f71f001f000 tx=0x7f71f0020000 comp rx=0 tx=0).stop 2024-09-11T23:18:07.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.203+0000 7f71fcea9640 1 -- 172.21.15.26:0/3308495499 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f71f8100a80 msgr2=0x7f71f80fcda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:07.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.203+0000 7f71fcea9640 1 --2- 172.21.15.26:0/3308495499 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f71f8100a80 0x7f71f80fcda0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f71e800d950 tx=0x7f71e800de10 comp rx=0 tx=0).stop 2024-09-11T23:18:07.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.203+0000 7f71fcea9640 1 -- 172.21.15.26:0/3308495499 shutdown_connections 2024-09-11T23:18:07.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.203+0000 7f71fcea9640 1 --2- 172.21.15.26:0/3308495499 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f71e00a2c00 0x7f71e00a50c0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:07.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.203+0000 7f71fcea9640 1 --2- 172.21.15.26:0/3308495499 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f71f8106a80 0x7f71f80fd820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:07.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.203+0000 7f71fcea9640 1 --2- 172.21.15.26:0/3308495499 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f71f8101420 0x7f71f80fd2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:07.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.203+0000 7f71fcea9640 1 --2- 172.21.15.26:0/3308495499 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f71f8100a80 0x7f71f80fcda0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:07.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.203+0000 7f71fcea9640 1 -- 172.21.15.26:0/3308495499 >> 172.21.15.26:0/3308495499 conn(0x7f71f80fc7a0 msgr2=0x7f71f806aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:18:07.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.203+0000 7f71fcea9640 1 -- 172.21.15.26:0/3308495499 shutdown_connections 2024-09-11T23:18:07.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:07.203+0000 7f71fcea9640 1 -- 172.21.15.26:0/3308495499 wait complete. 2024-09-11T23:18:07.971 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph tell mds.c status 2024-09-11T23:18:08.230 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:18:08.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:07 smithi136 ceph-mon[35121]: pgmap v331: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 341 B/s wr, 12 op/s 2024-09-11T23:18:08.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:07 smithi026 ceph-mon[31172]: pgmap v331: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 341 B/s wr, 12 op/s 2024-09-11T23:18:08.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:07 smithi076 ceph-mon[35296]: pgmap v331: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 341 B/s wr, 12 op/s 2024-09-11T23:18:08.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.694+0000 7fd5e22fb640 1 -- 172.21.15.26:0/3218929737 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5dc101de0 msgr2=0x7fd5dc1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:08.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.694+0000 7fd5e22fb640 1 --2- 172.21.15.26:0/3218929737 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5dc101de0 0x7fd5dc1021c0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fd5cc009900 tx=0x7fd5cc02a410 comp rx=0 tx=0).stop 2024-09-11T23:18:08.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.694+0000 7fd5e22fb640 1 -- 172.21.15.26:0/3218929737 shutdown_connections 2024-09-11T23:18:08.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.694+0000 7fd5e22fb640 1 --2- 172.21.15.26:0/3218929737 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd5dc102790 0x7fd5dc1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:08.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.694+0000 7fd5e22fb640 1 --2- 172.21.15.26:0/3218929737 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd5dc106f10 0x7fd5dc10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:08.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.694+0000 7fd5e22fb640 1 --2- 172.21.15.26:0/3218929737 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5dc101de0 0x7fd5dc1021c0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:08.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.694+0000 7fd5e22fb640 1 -- 172.21.15.26:0/3218929737 >> 172.21.15.26:0/3218929737 conn(0x7fd5dc0fd8c0 msgr2=0x7fd5dc0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:18:08.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.694+0000 7fd5e22fb640 1 -- 172.21.15.26:0/3218929737 shutdown_connections 2024-09-11T23:18:08.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.694+0000 7fd5e22fb640 1 -- 172.21.15.26:0/3218929737 wait complete. 2024-09-11T23:18:08.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.695+0000 7fd5e22fb640 1 Processor -- start 2024-09-11T23:18:08.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.695+0000 7fd5e22fb640 1 -- start start 2024-09-11T23:18:08.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.695+0000 7fd5e22fb640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd5dc101de0 0x7fd5dc0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:08.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.695+0000 7fd5e22fb640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5dc102790 0x7fd5dc074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:08.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.695+0000 7fd5e22fb640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd5dc106f10 0x7fd5dc074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:08.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.695+0000 7fd5e22fb640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd5dc112010 con 0x7fd5dc102790 2024-09-11T23:18:08.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.695+0000 7fd5e22fb640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fd5dc111e90 con 0x7fd5dc101de0 2024-09-11T23:18:08.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.695+0000 7fd5e22fb640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd5dc112190 con 0x7fd5dc106f10 2024-09-11T23:18:08.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.695+0000 7fd5db7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5dc102790 0x7fd5dc074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:08.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.695+0000 7fd5db7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5dc102790 0x7fd5dc074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33272/0 (socket says 172.21.15.26:33272) 2024-09-11T23:18:08.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.695+0000 7fd5db7fe640 1 -- 172.21.15.26:0/1679853206 learned_addr learned my addr 172.21.15.26:0/1679853206 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:18:08.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.696+0000 7fd5e0898640 1 --2- 172.21.15.26:0/1679853206 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd5dc106f10 0x7fd5dc074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:08.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.696+0000 7fd5db7fe640 1 -- 172.21.15.26:0/1679853206 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd5dc106f10 msgr2=0x7fd5dc074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:08.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.696+0000 7fd5dbfff640 1 --2- 172.21.15.26:0/1679853206 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd5dc101de0 0x7fd5dc0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:08.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.696+0000 7fd5db7fe640 1 --2- 172.21.15.26:0/1679853206 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd5dc106f10 0x7fd5dc074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:08.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.696+0000 7fd5db7fe640 1 -- 172.21.15.26:0/1679853206 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd5dc101de0 msgr2=0x7fd5dc0744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:08.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.696+0000 7fd5db7fe640 1 --2- 172.21.15.26:0/1679853206 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd5dc101de0 0x7fd5dc0744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:08.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.696+0000 7fd5db7fe640 1 -- 172.21.15.26:0/1679853206 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5dc1a6a70 con 0x7fd5dc102790 2024-09-11T23:18:08.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.696+0000 7fd5e0898640 1 --2- 172.21.15.26:0/1679853206 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd5dc106f10 0x7fd5dc074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:18:08.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.696+0000 7fd5dbfff640 1 --2- 172.21.15.26:0/1679853206 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd5dc101de0 0x7fd5dc0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:18:08.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.696+0000 7fd5db7fe640 1 --2- 172.21.15.26:0/1679853206 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5dc102790 0x7fd5dc074a30 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fd5d000b4a0 tx=0x7fd5d000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:18:08.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.696+0000 7fd5d97fa640 1 -- 172.21.15.26:0/1679853206 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5d0014070 con 0x7fd5dc102790 2024-09-11T23:18:08.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.696+0000 7fd5e22fb640 1 -- 172.21.15.26:0/1679853206 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5dc1a6d60 con 0x7fd5dc102790 2024-09-11T23:18:08.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.696+0000 7fd5d97fa640 1 -- 172.21.15.26:0/1679853206 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd5d0010af0 con 0x7fd5dc102790 2024-09-11T23:18:08.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.696+0000 7fd5d97fa640 1 -- 172.21.15.26:0/1679853206 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5d0017d10 con 0x7fd5dc102790 2024-09-11T23:18:08.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.696+0000 7fd5e22fb640 1 -- 172.21.15.26:0/1679853206 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5dc1a7260 con 0x7fd5dc102790 2024-09-11T23:18:08.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.698+0000 7fd5d97fa640 1 -- 172.21.15.26:0/1679853206 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fd5d0017070 con 0x7fd5dc102790 2024-09-11T23:18:08.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.699+0000 7fd5e46d4740 1 Processor -- start 2024-09-11T23:18:08.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.699+0000 7fd5e46d4740 1 -- start start 2024-09-11T23:18:08.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.699+0000 7fd5e46d4740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564ad741e9d0 0x564ad741edb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:08.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.699+0000 7fd5e46d4740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564ad741f2f0 0x564ad7429670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:08.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.699+0000 7fd5e46d4740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564ad7429bb0 0x564ad742bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:08.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.699+0000 7fd5e46d4740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x564ad7385090 con 0x564ad741f2f0 2024-09-11T23:18:08.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.699+0000 7fd5e46d4740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x564ad732ca70 con 0x564ad7429bb0 2024-09-11T23:18:08.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.699+0000 7fd5e46d4740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x564ad7381050 con 0x564ad741e9d0 2024-09-11T23:18:08.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.699+0000 7fd5dbfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564ad741f2f0 0x564ad7429670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:08.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.699+0000 7fd5dbfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564ad741f2f0 0x564ad7429670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33286/0 (socket says 172.21.15.26:33286) 2024-09-11T23:18:08.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.699+0000 7fd5dbfff640 1 -- 172.21.15.26:0/1399857794 learned_addr learned my addr 172.21.15.26:0/1399857794 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:18:08.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.699+0000 7fd5d97fa640 1 --2- 172.21.15.26:0/1679853206 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fd5c40a2b80 0x7fd5c40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:08.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.699+0000 7fd5db7fe640 1 --2- 172.21.15.26:0/1399857794 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564ad7429bb0 0x564ad742bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:08.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.699+0000 7fd5e0898640 1 --2- 172.21.15.26:0/1679853206 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fd5c40a2b80 0x7fd5c40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:08.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.700+0000 7fd5dbfff640 1 -- 172.21.15.26:0/1399857794 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564ad741e9d0 msgr2=0x564ad741edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:18:08.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.700+0000 7fd5dbfff640 1 --2- 172.21.15.26:0/1399857794 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564ad741e9d0 0x564ad741edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:08.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.700+0000 7fd5dbfff640 1 -- 172.21.15.26:0/1399857794 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564ad7429bb0 msgr2=0x564ad742bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:18:08.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.700+0000 7fd5dbfff640 1 --2- 172.21.15.26:0/1399857794 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564ad7429bb0 0x564ad742bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:08.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.700+0000 7fd5dbfff640 1 -- 172.21.15.26:0/1399857794 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ad7339130 con 0x564ad741f2f0 2024-09-11T23:18:08.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.700+0000 7fd5d97fa640 1 -- 172.21.15.26:0/1679853206 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fd5d00ce490 con 0x7fd5dc102790 2024-09-11T23:18:08.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.700+0000 7fd5dbfff640 1 --2- 172.21.15.26:0/1399857794 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564ad741f2f0 0x564ad7429670 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fd5cc000c00 tx=0x7fd5cc032d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9313a1f40f8f0ef0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:18:08.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.700+0000 7fd5e0898640 1 --2- 172.21.15.26:0/1679853206 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fd5c40a2b80 0x7fd5c40a5040 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fd5d4009f30 tx=0x7fd5d40094d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:18:08.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.700+0000 7fd5c27fc640 1 -- 172.21.15.26:0/1399857794 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5cc03f070 con 0x564ad741f2f0 2024-09-11T23:18:08.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.700+0000 7fd5c27fc640 1 -- 172.21.15.26:0/1399857794 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd5cc0317b0 con 0x564ad741f2f0 2024-09-11T23:18:08.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.700+0000 7fd5c27fc640 1 -- 172.21.15.26:0/1399857794 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5cc035070 con 0x564ad741f2f0 2024-09-11T23:18:08.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.701+0000 7fd5e46d4740 1 -- 172.21.15.26:0/1399857794 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564ad741f2f0 msgr2=0x564ad7429670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:08.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.701+0000 7fd5e46d4740 1 --2- 172.21.15.26:0/1399857794 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564ad741f2f0 0x564ad7429670 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fd5cc000c00 tx=0x7fd5cc032d70 comp rx=0 tx=0).stop 2024-09-11T23:18:08.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.701+0000 7fd5e46d4740 1 -- 172.21.15.26:0/1399857794 shutdown_connections 2024-09-11T23:18:08.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.701+0000 7fd5e46d4740 1 --2- 172.21.15.26:0/1399857794 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564ad7429bb0 0x564ad742bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:08.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.701+0000 7fd5e46d4740 1 --2- 172.21.15.26:0/1399857794 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564ad741f2f0 0x564ad7429670 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:08.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.701+0000 7fd5e46d4740 1 --2- 172.21.15.26:0/1399857794 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564ad741e9d0 0x564ad741edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:08.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.701+0000 7fd5e46d4740 1 -- 172.21.15.26:0/1399857794 >> 172.21.15.26:0/1399857794 conn(0x564ad741af60 msgr2=0x564ad741b340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:18:08.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.701+0000 7fd5e46d4740 1 -- 172.21.15.26:0/1399857794 shutdown_connections 2024-09-11T23:18:08.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.701+0000 7fd5e46d4740 1 -- 172.21.15.26:0/1399857794 wait complete. 2024-09-11T23:18:08.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.701+0000 7fd5e46d4740 1 -- start start 2024-09-11T23:18:08.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.701+0000 7fd5e46d4740 1 Processor -- start 2024-09-11T23:18:08.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.702+0000 7fd5e46d4740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564ad741e9d0 0x564ad74293b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:08.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.702+0000 7fd5e46d4740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564ad741f2f0 0x564ad7426700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:08.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.702+0000 7fd5e46d4740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564ad7429bb0 0x564ad7426c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:08.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.702+0000 7fd5e46d4740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x564ad742c780 con 0x564ad741f2f0 2024-09-11T23:18:08.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.702+0000 7fd5e46d4740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x564ad733ff20 con 0x564ad7429bb0 2024-09-11T23:18:08.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.702+0000 7fd5e46d4740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x564ad742c900 con 0x564ad741e9d0 2024-09-11T23:18:08.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.702+0000 7fd5e46d4740 10 client.0 _ll_register_callbacks cb 0x564ad7309030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:18:08.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.702+0000 7fd5db7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564ad741f2f0 0x564ad7426700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:08.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.702+0000 7fd5db7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564ad741f2f0 0x564ad7426700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33294/0 (socket says 172.21.15.26:33294) 2024-09-11T23:18:08.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.702+0000 7fd5db7fe640 1 -- 172.21.15.26:0/1994940409 learned_addr learned my addr 172.21.15.26:0/1994940409 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:18:08.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.702+0000 7fd5dbfff640 1 --2- 172.21.15.26:0/1994940409 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564ad741e9d0 0x564ad74293b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:08.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.702+0000 7fd5e0898640 1 --2- 172.21.15.26:0/1994940409 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564ad7429bb0 0x564ad7426c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:08.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.702+0000 7fd5db7fe640 1 -- 172.21.15.26:0/1994940409 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564ad741e9d0 msgr2=0x564ad74293b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:18:08.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.702+0000 7fd5db7fe640 1 --2- 172.21.15.26:0/1994940409 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564ad741e9d0 0x564ad74293b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:08.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.702+0000 7fd5db7fe640 1 -- 172.21.15.26:0/1994940409 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564ad7429bb0 msgr2=0x564ad7426c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:18:08.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.702+0000 7fd5db7fe640 1 --2- 172.21.15.26:0/1994940409 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564ad7429bb0 0x564ad7426c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:08.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.702+0000 7fd5db7fe640 1 -- 172.21.15.26:0/1994940409 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ad70f62b0 con 0x564ad741f2f0 2024-09-11T23:18:08.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.702+0000 7fd5db7fe640 1 --2- 172.21.15.26:0/1994940409 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564ad741f2f0 0x564ad7426700 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fd5d00cbf50 tx=0x7fd5d0069c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9855969c54d2068a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:18:08.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.702+0000 7fd5bb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:18:08.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.703+0000 7fd5bb7fe640 1 -- 172.21.15.26:0/1994940409 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5d001e070 con 0x564ad741f2f0 2024-09-11T23:18:08.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.703+0000 7fd5bb7fe640 1 -- 172.21.15.26:0/1994940409 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fd5d0017360 con 0x564ad741f2f0 2024-09-11T23:18:08.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.703+0000 7fd5e46d4740 1 -- 172.21.15.26:0/1994940409 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd5cc0317b0 con 0x564ad741f2f0 2024-09-11T23:18:08.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.703+0000 7fd5bb7fe640 1 -- 172.21.15.26:0/1994940409 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd5d009b3b0 con 0x564ad741f2f0 2024-09-11T23:18:08.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.703+0000 7fd5e46d4740 10 client.15087 fetch_fsmap learned FSMap version 28 2024-09-11T23:18:08.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.703+0000 7fd5e46d4740 1 -- 172.21.15.26:0/1994940409 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564ad7438040 con 0x564ad741f2f0 2024-09-11T23:18:08.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.703+0000 7fd5bb7fe640 1 -- 172.21.15.26:0/1994940409 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd5cc0317b0 con 0x564ad741f2f0 2024-09-11T23:18:08.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.703+0000 7fd5bb7fe640 1 -- 172.21.15.26:0/1994940409 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fd5d0097320 con 0x564ad741f2f0 2024-09-11T23:18:08.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.703+0000 7fd5e46d4740 10 client.15087 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:18:08.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.703+0000 7fd5e46d4740 10 client.15087 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-11T23:18:08.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.703+0000 7fd5e46d4740 20 client.15087 populate_metadata read hostname 'smithi026' 2024-09-11T23:18:08.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.703+0000 7fd5e46d4740 1 --2- 172.21.15.26:0/1994940409 >> [v2:172.21.15.136:6836/1633082387,v1:172.21.15.136:6837/1633082387] conn(0x564ad743a5b0 0x564ad743ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:08.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.703+0000 7fd5e46d4740 4 client.15087 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:18:08.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.703+0000 7fd5e46d4740 1 -- 172.21.15.26:0/1994940409 --> [v2:172.21.15.136:6836/1633082387,v1:172.21.15.136:6837/1633082387] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564ad7392600 con 0x564ad743a5b0 2024-09-11T23:18:08.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.704+0000 7fd5dbfff640 1 --2- 172.21.15.26:0/1994940409 >> [v2:172.21.15.136:6836/1633082387,v1:172.21.15.136:6837/1633082387] conn(0x564ad743a5b0 0x564ad743ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:08.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.704+0000 7fd5bb7fe640 10 client.15087 ms_handle_connect on v2:172.21.15.136:6836/1633082387 2024-09-11T23:18:08.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.704+0000 7fd5dbfff640 1 --2- 172.21.15.26:0/1994940409 >> [v2:172.21.15.136:6836/1633082387,v1:172.21.15.136:6837/1633082387] conn(0x564ad743a5b0 0x564ad743ba20 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=1d4478187f3fa041 server_cookie=65be317ebc4fbaac in_seq=0 out_seq=0 2024-09-11T23:18:08.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.705+0000 7fd5bb7fe640 1 -- 172.21.15.26:0/1994940409 <== mds.3 v2:172.21.15.136:6836/1633082387 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564ad7392600 con 0x564ad743a5b0 2024-09-11T23:18:08.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.705+0000 7fd5bb7fe640 10 client.15087 handle_command_reply: tid=0 2024-09-11T23:18:08.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.705+0000 7fd5e46d4740 1 client.15087 shutdown 2024-09-11T23:18:08.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.705+0000 7fd5e46d4740 1 -- 172.21.15.26:0/1994940409 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564ad741f2f0 msgr2=0x564ad7426700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:08.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.705+0000 7fd5e46d4740 1 --2- 172.21.15.26:0/1994940409 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564ad741f2f0 0x564ad7426700 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fd5d00cbf50 tx=0x7fd5d0069c30 comp rx=0 tx=0).stop 2024-09-11T23:18:08.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.705+0000 7fd5e46d4740 1 -- 172.21.15.26:0/1994940409 shutdown_connections 2024-09-11T23:18:08.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.705+0000 7fd5e46d4740 1 --2- 172.21.15.26:0/1994940409 >> [v2:172.21.15.136:6836/1633082387,v1:172.21.15.136:6837/1633082387] conn(0x564ad743a5b0 0x564ad743ba20 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:08.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.705+0000 7fd5e46d4740 1 --2- 172.21.15.26:0/1994940409 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564ad7429bb0 0x564ad7426c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:08.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.705+0000 7fd5e46d4740 1 --2- 172.21.15.26:0/1994940409 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564ad741f2f0 0x564ad7426700 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:08.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.705+0000 7fd5bb7fe640 0 client.15087 ms_handle_reset on v2:172.21.15.136:6836/1633082387 2024-09-11T23:18:08.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.705+0000 7fd5e46d4740 1 --2- 172.21.15.26:0/1994940409 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564ad741e9d0 0x564ad74293b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:08.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.705+0000 7fd5e46d4740 1 -- 172.21.15.26:0/1994940409 >> 172.21.15.26:0/1994940409 conn(0x564ad741af60 msgr2=0x564ad7420c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:18:08.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.705+0000 7fd5e46d4740 1 -- 172.21.15.26:0/1994940409 shutdown_connections 2024-09-11T23:18:08.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.705+0000 7fd5e46d4740 1 -- 172.21.15.26:0/1994940409 wait complete. 2024-09-11T23:18:08.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.705+0000 7fd5e46d4740 20 client.15087 trim_cache size 0 max 16384 2024-09-11T23:18:08.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.723+0000 7fd5e46d4740 1 Processor -- start 2024-09-11T23:18:08.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.723+0000 7fd5e46d4740 1 -- start start 2024-09-11T23:18:08.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.723+0000 7fd5e46d4740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564ad741e9d0 0x564ad743ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:08.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.723+0000 7fd5e46d4740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564ad741f2f0 0x564ad7425fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:08.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.723+0000 7fd5e46d4740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564ad7429bb0 0x564ad7424ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:08.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.723+0000 7fd5e46d4740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x564ad7439d60 con 0x564ad7429bb0 2024-09-11T23:18:08.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.723+0000 7fd5e46d4740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x564ad743a430 con 0x564ad741e9d0 2024-09-11T23:18:08.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.723+0000 7fd5e46d4740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x564ad743ab00 con 0x564ad741f2f0 2024-09-11T23:18:08.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.723+0000 7fd5e0898640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564ad7429bb0 0x564ad7424ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:08.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.723+0000 7fd5e0898640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564ad7429bb0 0x564ad7424ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33310/0 (socket says 172.21.15.26:33310) 2024-09-11T23:18:08.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.723+0000 7fd5e0898640 1 -- 172.21.15.26:0/505776204 learned_addr learned my addr 172.21.15.26:0/505776204 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:18:08.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.723+0000 7fd5dbfff640 1 --2- 172.21.15.26:0/505776204 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564ad741e9d0 0x564ad743ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:08.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.724+0000 7fd5e0898640 1 -- 172.21.15.26:0/505776204 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564ad741f2f0 msgr2=0x564ad7425fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:18:08.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.724+0000 7fd5db7fe640 1 --2- 172.21.15.26:0/505776204 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564ad741f2f0 0x564ad7425fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:08.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.724+0000 7fd5e0898640 1 --2- 172.21.15.26:0/505776204 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564ad741f2f0 0x564ad7425fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:08.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.724+0000 7fd5e0898640 1 -- 172.21.15.26:0/505776204 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564ad741e9d0 msgr2=0x564ad743ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:18:08.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.724+0000 7fd5e0898640 1 --2- 172.21.15.26:0/505776204 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564ad741e9d0 0x564ad743ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:08.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.724+0000 7fd5e0898640 1 -- 172.21.15.26:0/505776204 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ad741f120 con 0x564ad7429bb0 2024-09-11T23:18:08.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.724+0000 7fd5dbfff640 1 --2- 172.21.15.26:0/505776204 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564ad741e9d0 0x564ad743ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:18:08.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.724+0000 7fd5e0898640 1 --2- 172.21.15.26:0/505776204 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564ad7429bb0 0x564ad7424ff0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fd5d400e8e0 tx=0x7fd5d4010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f3c21ab0c9d423d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:18:08.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.724+0000 7fd5c1ffb640 1 -- 172.21.15.26:0/505776204 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5d400cd70 con 0x564ad7429bb0 2024-09-11T23:18:08.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.724+0000 7fd5c1ffb640 1 -- 172.21.15.26:0/505776204 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd5d4012070 con 0x564ad7429bb0 2024-09-11T23:18:08.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.724+0000 7fd5c1ffb640 1 -- 172.21.15.26:0/505776204 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5d401c780 con 0x564ad7429bb0 2024-09-11T23:18:08.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.725+0000 7fd5e46d4740 1 -- 172.21.15.26:0/505776204 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564ad7429bb0 msgr2=0x564ad7424ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:08.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.725+0000 7fd5e46d4740 1 --2- 172.21.15.26:0/505776204 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564ad7429bb0 0x564ad7424ff0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fd5d400e8e0 tx=0x7fd5d4010000 comp rx=0 tx=0).stop 2024-09-11T23:18:08.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.725+0000 7fd5e46d4740 1 -- 172.21.15.26:0/505776204 shutdown_connections 2024-09-11T23:18:08.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.725+0000 7fd5e46d4740 1 --2- 172.21.15.26:0/505776204 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564ad7429bb0 0x564ad7424ff0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:08.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.725+0000 7fd5e46d4740 1 --2- 172.21.15.26:0/505776204 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564ad741f2f0 0x564ad7425fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:08.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.725+0000 7fd5e46d4740 1 --2- 172.21.15.26:0/505776204 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564ad741e9d0 0x564ad743ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:08.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.725+0000 7fd5e46d4740 1 -- 172.21.15.26:0/505776204 >> 172.21.15.26:0/505776204 conn(0x564ad741af60 msgr2=0x564ad741b340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:18:08.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.725+0000 7fd5e46d4740 1 -- 172.21.15.26:0/505776204 shutdown_connections 2024-09-11T23:18:08.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.725+0000 7fd5e46d4740 1 -- 172.21.15.26:0/505776204 wait complete. 2024-09-11T23:18:08.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.725+0000 7fd5e46d4740 1 -- start start 2024-09-11T23:18:08.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.725+0000 7fd5e46d4740 1 Processor -- start 2024-09-11T23:18:08.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.726+0000 7fd5e46d4740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564ad741e9d0 0x564ad7444000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:08.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.726+0000 7fd5e46d4740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564ad741f2f0 0x564ad7444540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:08.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.726+0000 7fd5e46d4740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564ad7429bb0 0x564ad743f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:08.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.726+0000 7fd5e46d4740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x564ad743b8a0 con 0x564ad741e9d0 2024-09-11T23:18:08.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.726+0000 7fd5e46d4740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x564ad743b1d0 con 0x564ad7429bb0 2024-09-11T23:18:08.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.726+0000 7fd5e46d4740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x564ad743bf70 con 0x564ad741f2f0 2024-09-11T23:18:08.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.726+0000 7fd5dbfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564ad741e9d0 0x564ad7444000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:08.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.726+0000 7fd5e46d4740 10 client.0 _ll_register_callbacks cb 0x7fd5d000b3f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:18:08.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.726+0000 7fd5dbfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564ad741e9d0 0x564ad7444000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33324/0 (socket says 172.21.15.26:33324) 2024-09-11T23:18:08.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.726+0000 7fd5dbfff640 1 -- 172.21.15.26:0/4055220639 learned_addr learned my addr 172.21.15.26:0/4055220639 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:18:08.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.726+0000 7fd5db7fe640 1 --2- 172.21.15.26:0/4055220639 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564ad741f2f0 0x564ad7444540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:08.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.726+0000 7fd5e0898640 1 --2- 172.21.15.26:0/4055220639 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564ad7429bb0 0x564ad743f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:08.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.726+0000 7fd5dbfff640 1 -- 172.21.15.26:0/4055220639 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564ad741f2f0 msgr2=0x564ad7444540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:18:08.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.726+0000 7fd5dbfff640 1 --2- 172.21.15.26:0/4055220639 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564ad741f2f0 0x564ad7444540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:08.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.726+0000 7fd5dbfff640 1 -- 172.21.15.26:0/4055220639 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564ad7429bb0 msgr2=0x564ad743f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:18:08.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.726+0000 7fd5dbfff640 1 --2- 172.21.15.26:0/4055220639 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564ad7429bb0 0x564ad743f3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:08.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.726+0000 7fd5dbfff640 1 -- 172.21.15.26:0/4055220639 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ad743fc10 con 0x564ad741e9d0 2024-09-11T23:18:08.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.726+0000 7fd5e0898640 1 --2- 172.21.15.26:0/4055220639 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564ad7429bb0 0x564ad743f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:18:08.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.726+0000 7fd5dbfff640 1 --2- 172.21.15.26:0/4055220639 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564ad741e9d0 0x564ad7444000 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fd5cc0053e0 tx=0x7fd5cc005410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a636941ff9b9cfc server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:18:08.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.726+0000 7fd5bb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:18:08.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.726+0000 7fd5bb7fe640 1 -- 172.21.15.26:0/4055220639 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5cc03f030 con 0x564ad741e9d0 2024-09-11T23:18:08.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.726+0000 7fd5bb7fe640 1 -- 172.21.15.26:0/4055220639 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fd5cc004240 con 0x564ad741e9d0 2024-09-11T23:18:08.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.726+0000 7fd5e46d4740 1 -- 172.21.15.26:0/4055220639 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd5d4012070 con 0x564ad741e9d0 2024-09-11T23:18:08.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.727+0000 7fd5bb7fe640 1 -- 172.21.15.26:0/4055220639 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd5cc035040 con 0x564ad741e9d0 2024-09-11T23:18:08.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.727+0000 7fd5e46d4740 10 client.15093 fetch_fsmap learned FSMap version 28 2024-09-11T23:18:08.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.727+0000 7fd5e46d4740 1 -- 172.21.15.26:0/4055220639 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564ad74a48a0 con 0x564ad741e9d0 2024-09-11T23:18:08.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.727+0000 7fd5bb7fe640 1 -- 172.21.15.26:0/4055220639 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd5d4012070 con 0x564ad741e9d0 2024-09-11T23:18:08.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.727+0000 7fd5bb7fe640 1 -- 172.21.15.26:0/4055220639 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fd5cc003820 con 0x564ad741e9d0 2024-09-11T23:18:08.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.727+0000 7fd5e46d4740 10 client.15093 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:18:08.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.727+0000 7fd5e46d4740 10 client.15093 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-11T23:18:08.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.727+0000 7fd5e46d4740 20 client.15093 populate_metadata read hostname 'smithi026' 2024-09-11T23:18:08.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.727+0000 7fd5e46d4740 1 --2- 172.21.15.26:0/4055220639 >> [v2:172.21.15.136:6836/1633082387,v1:172.21.15.136:6837/1633082387] conn(0x564ad74a8fe0 0x564ad74a7cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:08.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.727+0000 7fd5e46d4740 4 client.15093 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-11T23:18:08.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.727+0000 7fd5e46d4740 1 -- 172.21.15.26:0/4055220639 --> [v2:172.21.15.136:6836/1633082387,v1:172.21.15.136:6837/1633082387] -- command(tid 0: {"prefix": "status"}) -- 0x564ad738cb10 con 0x564ad74a8fe0 2024-09-11T23:18:08.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.728+0000 7fd5db7fe640 1 --2- 172.21.15.26:0/4055220639 >> [v2:172.21.15.136:6836/1633082387,v1:172.21.15.136:6837/1633082387] conn(0x564ad74a8fe0 0x564ad74a7cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:08.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.728+0000 7fd5db7fe640 1 --2- 172.21.15.26:0/4055220639 >> [v2:172.21.15.136:6836/1633082387,v1:172.21.15.136:6837/1633082387] conn(0x564ad74a8fe0 0x564ad74a7cf0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ce3cce4fbb29b584 server_cookie=979246c42a816952 in_seq=0 out_seq=0 2024-09-11T23:18:08.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.728+0000 7fd5bb7fe640 10 client.15093 ms_handle_connect on v2:172.21.15.136:6836/1633082387 2024-09-11T23:18:08.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.728+0000 7fd5bb7fe640 1 -- 172.21.15.26:0/4055220639 <== mds.3 v2:172.21.15.136:6836/1633082387 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x564ad738cb10 con 0x564ad74a8fe0 2024-09-11T23:18:08.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.728+0000 7fd5bb7fe640 10 client.15093 handle_command_reply: tid=0 2024-09-11T23:18:08.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.728+0000 7fd5e46d4740 1 client.15093 shutdown 2024-09-11T23:18:08.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.728+0000 7fd5e46d4740 1 -- 172.21.15.26:0/4055220639 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564ad741e9d0 msgr2=0x564ad7444000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:08.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.728+0000 7fd5e46d4740 1 --2- 172.21.15.26:0/4055220639 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564ad741e9d0 0x564ad7444000 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fd5cc0053e0 tx=0x7fd5cc005410 comp rx=0 tx=0).stop 2024-09-11T23:18:08.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.728+0000 7fd5e46d4740 1 -- 172.21.15.26:0/4055220639 shutdown_connections 2024-09-11T23:18:08.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.728+0000 7fd5e46d4740 1 --2- 172.21.15.26:0/4055220639 >> [v2:172.21.15.136:6836/1633082387,v1:172.21.15.136:6837/1633082387] conn(0x564ad74a8fe0 0x564ad74a7cf0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:08.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.728+0000 7fd5e46d4740 1 --2- 172.21.15.26:0/4055220639 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564ad7429bb0 0x564ad743f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:08.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.728+0000 7fd5e46d4740 1 --2- 172.21.15.26:0/4055220639 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564ad741f2f0 0x564ad7444540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:08.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.728+0000 7fd5bb7fe640 0 client.15093 ms_handle_reset on v2:172.21.15.136:6836/1633082387 2024-09-11T23:18:08.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.728+0000 7fd5e46d4740 1 --2- 172.21.15.26:0/4055220639 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564ad741e9d0 0x564ad7444000 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:08.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.728+0000 7fd5e46d4740 1 -- 172.21.15.26:0/4055220639 >> 172.21.15.26:0/4055220639 conn(0x564ad741af60 msgr2=0x564ad7424f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:18:08.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.728+0000 7fd5e46d4740 1 -- 172.21.15.26:0/4055220639 shutdown_connections 2024-09-11T23:18:08.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.729+0000 7fd5e46d4740 1 -- 172.21.15.26:0/4055220639 wait complete. 2024-09-11T23:18:08.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.729+0000 7fd5e46d4740 20 client.15093 trim_cache size 0 max 16384 2024-09-11T23:18:08.740 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-11T23:18:08.741 INFO:teuthology.orchestra.run.smithi026.stdout: "cluster_fsid": "af7e6fc8-7092-11ef-bcea-c7b262605968", 2024-09-11T23:18:08.741 INFO:teuthology.orchestra.run.smithi026.stdout: "whoami": 3, 2024-09-11T23:18:08.741 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 24448, 2024-09-11T23:18:08.741 INFO:teuthology.orchestra.run.smithi026.stdout: "want_state": "up:active", 2024-09-11T23:18:08.741 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "up:active", 2024-09-11T23:18:08.741 INFO:teuthology.orchestra.run.smithi026.stdout: "fs_name": "cephfs", 2024-09-11T23:18:08.741 INFO:teuthology.orchestra.run.smithi026.stdout: "rank_uptime": 91.765808446999998, 2024-09-11T23:18:08.741 INFO:teuthology.orchestra.run.smithi026.stdout: "mdsmap_epoch": 28, 2024-09-11T23:18:08.741 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch": 85, 2024-09-11T23:18:08.741 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch_barrier": 85, 2024-09-11T23:18:08.741 INFO:teuthology.orchestra.run.smithi026.stdout: "uptime": 127.74380085599999 2024-09-11T23:18:08.742 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-11T23:18:08.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.729+0000 7fd5e22fb640 1 -- 172.21.15.26:0/1679853206 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fd5c40a2b80 msgr2=0x7fd5c40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:08.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.729+0000 7fd5e22fb640 1 --2- 172.21.15.26:0/1679853206 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fd5c40a2b80 0x7fd5c40a5040 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fd5d4009f30 tx=0x7fd5d40094d0 comp rx=0 tx=0).stop 2024-09-11T23:18:08.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.729+0000 7fd5e22fb640 1 -- 172.21.15.26:0/1679853206 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5dc102790 msgr2=0x7fd5dc074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:08.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.729+0000 7fd5e22fb640 1 --2- 172.21.15.26:0/1679853206 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5dc102790 0x7fd5dc074a30 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fd5d000b4a0 tx=0x7fd5d000b960 comp rx=0 tx=0).stop 2024-09-11T23:18:08.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.729+0000 7fd5e22fb640 1 -- 172.21.15.26:0/1679853206 shutdown_connections 2024-09-11T23:18:08.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.729+0000 7fd5e22fb640 1 --2- 172.21.15.26:0/1679853206 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fd5c40a2b80 0x7fd5c40a5040 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:08.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.729+0000 7fd5e22fb640 1 --2- 172.21.15.26:0/1679853206 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd5dc106f10 0x7fd5dc074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:08.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.729+0000 7fd5e22fb640 1 --2- 172.21.15.26:0/1679853206 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5dc102790 0x7fd5dc074a30 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:08.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.729+0000 7fd5e22fb640 1 --2- 172.21.15.26:0/1679853206 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd5dc101de0 0x7fd5dc0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:08.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.729+0000 7fd5e22fb640 1 -- 172.21.15.26:0/1679853206 >> 172.21.15.26:0/1679853206 conn(0x7fd5dc0fd8c0 msgr2=0x7fd5dc0fecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:18:08.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.729+0000 7fd5e22fb640 1 -- 172.21.15.26:0/1679853206 shutdown_connections 2024-09-11T23:18:08.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:08.729+0000 7fd5e22fb640 1 -- 172.21.15.26:0/1679853206 wait complete. 2024-09-11T23:18:09.530 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph tell mds.h status 2024-09-11T23:18:09.786 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:18:10.212 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:09 smithi026 ceph-mon[31172]: pgmap v332: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 341 B/s wr, 8 op/s 2024-09-11T23:18:10.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:09 smithi136 ceph-mon[35121]: pgmap v332: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 341 B/s wr, 8 op/s 2024-09-11T23:18:10.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.294+0000 7f00cf6f7640 1 -- 172.21.15.26:0/1689213508 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f00c8103780 msgr2=0x7f00c8107bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:10.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.294+0000 7f00cf6f7640 1 --2- 172.21.15.26:0/1689213508 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f00c8103780 0x7f00c8107bc0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f00b8009930 tx=0x7f00b802a420 comp rx=0 tx=0).stop 2024-09-11T23:18:10.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.294+0000 7f00cf6f7640 1 -- 172.21.15.26:0/1689213508 shutdown_connections 2024-09-11T23:18:10.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.294+0000 7f00cf6f7640 1 --2- 172.21.15.26:0/1689213508 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f00c8108300 0x7f00c810ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:10.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.294+0000 7f00cf6f7640 1 --2- 172.21.15.26:0/1689213508 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f00c8103780 0x7f00c8107bc0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:10.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.294+0000 7f00cf6f7640 1 --2- 172.21.15.26:0/1689213508 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f00c8102dd0 0x7f00c81031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:10.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.294+0000 7f00cf6f7640 1 -- 172.21.15.26:0/1689213508 >> 172.21.15.26:0/1689213508 conn(0x7f00c80fc7d0 msgr2=0x7f00c80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:18:10.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.294+0000 7f00cf6f7640 1 -- 172.21.15.26:0/1689213508 shutdown_connections 2024-09-11T23:18:10.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.294+0000 7f00cf6f7640 1 -- 172.21.15.26:0/1689213508 wait complete. 2024-09-11T23:18:10.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.295+0000 7f00cf6f7640 1 Processor -- start 2024-09-11T23:18:10.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.295+0000 7f00cf6f7640 1 -- start start 2024-09-11T23:18:10.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.295+0000 7f00cf6f7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f00c8102dd0 0x7f00c8198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:10.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.295+0000 7f00cf6f7640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f00c8103780 0x7f00c8198c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:10.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.295+0000 7f00cf6f7640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f00c8108300 0x7f00c819d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:10.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.295+0000 7f00cf6f7640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f00c8112150 con 0x7f00c8102dd0 2024-09-11T23:18:10.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.295+0000 7f00cf6f7640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f00c8111fd0 con 0x7f00c8108300 2024-09-11T23:18:10.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.295+0000 7f00cf6f7640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f00c81122d0 con 0x7f00c8103780 2024-09-11T23:18:10.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.295+0000 7f00cd493640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f00c8102dd0 0x7f00c8198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:10.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.295+0000 7f00cd493640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f00c8102dd0 0x7f00c8198750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33340/0 (socket says 172.21.15.26:33340) 2024-09-11T23:18:10.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.295+0000 7f00cd493640 1 -- 172.21.15.26:0/1362497864 learned_addr learned my addr 172.21.15.26:0/1362497864 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:18:10.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.295+0000 7f00ccc92640 1 --2- 172.21.15.26:0/1362497864 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f00c8103780 0x7f00c8198c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:10.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.295+0000 7f00cdc94640 1 --2- 172.21.15.26:0/1362497864 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f00c8108300 0x7f00c819d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:10.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.295+0000 7f00cd493640 1 -- 172.21.15.26:0/1362497864 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f00c8103780 msgr2=0x7f00c8198c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:10.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.295+0000 7f00cd493640 1 --2- 172.21.15.26:0/1362497864 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f00c8103780 0x7f00c8198c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:10.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.295+0000 7f00cd493640 1 -- 172.21.15.26:0/1362497864 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f00c8108300 msgr2=0x7f00c819d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:10.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.295+0000 7f00cd493640 1 --2- 172.21.15.26:0/1362497864 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f00c8108300 0x7f00c819d020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:10.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.295+0000 7f00cd493640 1 -- 172.21.15.26:0/1362497864 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00c819d700 con 0x7f00c8102dd0 2024-09-11T23:18:10.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.296+0000 7f00ccc92640 1 --2- 172.21.15.26:0/1362497864 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f00c8103780 0x7f00c8198c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:18:10.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.296+0000 7f00cd493640 1 --2- 172.21.15.26:0/1362497864 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f00c8102dd0 0x7f00c8198750 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f00c400b4a0 tx=0x7f00c400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:18:10.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.296+0000 7f00be7fc640 1 -- 172.21.15.26:0/1362497864 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00c4014070 con 0x7f00c8102dd0 2024-09-11T23:18:10.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.296+0000 7f00cf6f7640 1 -- 172.21.15.26:0/1362497864 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00c819d9f0 con 0x7f00c8102dd0 2024-09-11T23:18:10.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.296+0000 7f00be7fc640 1 -- 172.21.15.26:0/1362497864 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00c4010af0 con 0x7f00c8102dd0 2024-09-11T23:18:10.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.296+0000 7f00be7fc640 1 -- 172.21.15.26:0/1362497864 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00c4017d10 con 0x7f00c8102dd0 2024-09-11T23:18:10.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.296+0000 7f00cf6f7640 1 -- 172.21.15.26:0/1362497864 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00c81a52d0 con 0x7f00c8102dd0 2024-09-11T23:18:10.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.298+0000 7f00be7fc640 1 -- 172.21.15.26:0/1362497864 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f00c4017070 con 0x7f00c8102dd0 2024-09-11T23:18:10.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.298+0000 7f00d1ad0740 1 Processor -- start 2024-09-11T23:18:10.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.299+0000 7f00d1ad0740 1 -- start start 2024-09-11T23:18:10.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.299+0000 7f00d1ad0740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a6c626b8d0 0x55a6c626bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:10.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.299+0000 7f00d1ad0740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a6c626c1f0 0x55a6c627aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:10.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.299+0000 7f00d1ad0740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6c627afc0 0x55a6c627d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:10.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.299+0000 7f00d1ad0740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55a6c61d2090 con 0x55a6c627afc0 2024-09-11T23:18:10.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.299+0000 7f00d1ad0740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55a6c6179a70 con 0x55a6c626c1f0 2024-09-11T23:18:10.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.299+0000 7f00d1ad0740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55a6c61ce050 con 0x55a6c626b8d0 2024-09-11T23:18:10.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.299+0000 7f00ccc92640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6c627afc0 0x55a6c627d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:10.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.299+0000 7f00ccc92640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6c627afc0 0x55a6c627d3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33342/0 (socket says 172.21.15.26:33342) 2024-09-11T23:18:10.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.299+0000 7f00ccc92640 1 -- 172.21.15.26:0/3926065088 learned_addr learned my addr 172.21.15.26:0/3926065088 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:18:10.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.299+0000 7f00cd493640 1 --2- 172.21.15.26:0/3926065088 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a6c626c1f0 0x55a6c627aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:10.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.299+0000 7f00cdc94640 1 --2- 172.21.15.26:0/3926065088 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a6c626b8d0 0x55a6c626bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:10.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.299+0000 7f00be7fc640 1 --2- 172.21.15.26:0/1362497864 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f00b40a2b80 0x7f00b40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:10.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.299+0000 7f00ccc92640 1 -- 172.21.15.26:0/3926065088 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a6c626b8d0 msgr2=0x55a6c626bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:18:10.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.299+0000 7f00ccc92640 1 --2- 172.21.15.26:0/3926065088 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a6c626b8d0 0x55a6c626bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:10.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.299+0000 7f00ccc92640 1 -- 172.21.15.26:0/3926065088 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a6c626c1f0 msgr2=0x55a6c627aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:18:10.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.299+0000 7f00ccc92640 1 --2- 172.21.15.26:0/3926065088 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a6c626c1f0 0x55a6c627aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:10.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.299+0000 7f00ccc92640 1 -- 172.21.15.26:0/3926065088 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a6c6186130 con 0x55a6c627afc0 2024-09-11T23:18:10.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.300+0000 7f00cdc94640 1 --2- 172.21.15.26:0/1362497864 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f00b40a2b80 0x7f00b40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:10.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.300+0000 7f00be7fc640 1 -- 172.21.15.26:0/1362497864 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f00c40ce4c0 con 0x7f00c8102dd0 2024-09-11T23:18:10.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.300+0000 7f00ccc92640 1 --2- 172.21.15.26:0/3926065088 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6c627afc0 0x55a6c627d3b0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f00b8000c00 tx=0x7f00b8009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a62b1faa19d4d76 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:18:10.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.300+0000 7f00b37fe640 1 -- 172.21.15.26:0/3926065088 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00b803f070 con 0x55a6c627afc0 2024-09-11T23:18:10.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.300+0000 7f00b37fe640 1 -- 172.21.15.26:0/3926065088 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00b80317b0 con 0x55a6c627afc0 2024-09-11T23:18:10.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.300+0000 7f00b37fe640 1 -- 172.21.15.26:0/3926065088 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00b8035070 con 0x55a6c627afc0 2024-09-11T23:18:10.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.301+0000 7f00d1ad0740 1 -- 172.21.15.26:0/3926065088 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6c627afc0 msgr2=0x55a6c627d3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:10.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.301+0000 7f00cdc94640 1 --2- 172.21.15.26:0/1362497864 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f00b40a2b80 0x7f00b40a5040 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f00c0007e10 tx=0x7f00c0004b50 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:18:10.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.301+0000 7f00d1ad0740 1 --2- 172.21.15.26:0/3926065088 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6c627afc0 0x55a6c627d3b0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f00b8000c00 tx=0x7f00b8009680 comp rx=0 tx=0).stop 2024-09-11T23:18:10.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.301+0000 7f00d1ad0740 1 -- 172.21.15.26:0/3926065088 shutdown_connections 2024-09-11T23:18:10.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.301+0000 7f00d1ad0740 1 --2- 172.21.15.26:0/3926065088 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a6c626c1f0 0x55a6c627aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:10.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.301+0000 7f00d1ad0740 1 --2- 172.21.15.26:0/3926065088 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6c627afc0 0x55a6c627d3b0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:10.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.301+0000 7f00d1ad0740 1 --2- 172.21.15.26:0/3926065088 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a6c626b8d0 0x55a6c626bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:10.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.301+0000 7f00d1ad0740 1 -- 172.21.15.26:0/3926065088 >> 172.21.15.26:0/3926065088 conn(0x55a6c6267f60 msgr2=0x55a6c6268340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:18:10.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.301+0000 7f00d1ad0740 1 -- 172.21.15.26:0/3926065088 shutdown_connections 2024-09-11T23:18:10.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.301+0000 7f00d1ad0740 1 -- 172.21.15.26:0/3926065088 wait complete. 2024-09-11T23:18:10.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.301+0000 7f00d1ad0740 1 -- start start 2024-09-11T23:18:10.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.301+0000 7f00d1ad0740 1 Processor -- start 2024-09-11T23:18:10.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.302+0000 7f00d1ad0740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6c626b8d0 0x55a6c627a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:10.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.302+0000 7f00d1ad0740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a6c626c1f0 0x55a6c6277b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:10.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.302+0000 7f00d1ad0740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a6c627afc0 0x55a6c6278090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:10.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.302+0000 7f00d1ad0740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55a6c627db90 con 0x55a6c626b8d0 2024-09-11T23:18:10.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.302+0000 7f00d1ad0740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55a6c618cf20 con 0x55a6c626c1f0 2024-09-11T23:18:10.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.302+0000 7f00d1ad0740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55a6c627dd10 con 0x55a6c627afc0 2024-09-11T23:18:10.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.302+0000 7f00d1ad0740 10 client.0 _ll_register_callbacks cb 0x55a6c6156030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:18:10.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.302+0000 7f00cd493640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6c626b8d0 0x55a6c627a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:10.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.302+0000 7f00cd493640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6c626b8d0 0x55a6c627a800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33344/0 (socket says 172.21.15.26:33344) 2024-09-11T23:18:10.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.302+0000 7f00cd493640 1 -- 172.21.15.26:0/946864613 learned_addr learned my addr 172.21.15.26:0/946864613 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:18:10.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.302+0000 7f00ccc92640 1 --2- 172.21.15.26:0/946864613 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a6c626c1f0 0x55a6c6277b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:10.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.302+0000 7f00cdc94640 1 --2- 172.21.15.26:0/946864613 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a6c627afc0 0x55a6c6278090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:10.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.302+0000 7f00cd493640 1 -- 172.21.15.26:0/946864613 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a6c627afc0 msgr2=0x55a6c6278090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:18:10.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.302+0000 7f00cd493640 1 --2- 172.21.15.26:0/946864613 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a6c627afc0 0x55a6c6278090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:10.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.302+0000 7f00cd493640 1 -- 172.21.15.26:0/946864613 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a6c626c1f0 msgr2=0x55a6c6277b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:18:10.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.302+0000 7f00cd493640 1 --2- 172.21.15.26:0/946864613 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a6c626c1f0 0x55a6c6277b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:10.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.302+0000 7f00cd493640 1 -- 172.21.15.26:0/946864613 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a6c5f432b0 con 0x55a6c626b8d0 2024-09-11T23:18:10.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.302+0000 7f00ccc92640 1 --2- 172.21.15.26:0/946864613 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a6c626c1f0 0x55a6c6277b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:18:10.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.303+0000 7f00cd493640 1 --2- 172.21.15.26:0/946864613 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6c626b8d0 0x55a6c627a800 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f00c409acb0 tx=0x7f00c40d1a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20d8831049a7b79d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:18:10.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.303+0000 7f00b0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:18:10.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.303+0000 7f00b0ff9640 1 -- 172.21.15.26:0/946864613 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00c401e070 con 0x55a6c626b8d0 2024-09-11T23:18:10.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.303+0000 7f00b0ff9640 1 -- 172.21.15.26:0/946864613 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f00c4010ca0 con 0x55a6c626b8d0 2024-09-11T23:18:10.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.303+0000 7f00d1ad0740 1 -- 172.21.15.26:0/946864613 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f00b80317b0 con 0x55a6c626b8d0 2024-09-11T23:18:10.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.303+0000 7f00b0ff9640 1 -- 172.21.15.26:0/946864613 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f00c409b420 con 0x55a6c626b8d0 2024-09-11T23:18:10.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.303+0000 7f00d1ad0740 10 client.15105 fetch_fsmap learned FSMap version 28 2024-09-11T23:18:10.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.303+0000 7f00d1ad0740 1 -- 172.21.15.26:0/946864613 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a6c6289490 con 0x55a6c626b8d0 2024-09-11T23:18:10.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.303+0000 7f00b0ff9640 1 -- 172.21.15.26:0/946864613 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00b80317b0 con 0x55a6c626b8d0 2024-09-11T23:18:10.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.303+0000 7f00b0ff9640 1 -- 172.21.15.26:0/946864613 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f00c4097390 con 0x55a6c626b8d0 2024-09-11T23:18:10.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.303+0000 7f00d1ad0740 10 client.15105 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:18:10.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.303+0000 7f00d1ad0740 10 client.15105 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-11T23:18:10.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.303+0000 7f00d1ad0740 20 client.15105 populate_metadata read hostname 'smithi026' 2024-09-11T23:18:10.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.303+0000 7f00d1ad0740 1 --2- 172.21.15.26:0/946864613 >> [v2:172.21.15.76:6836/2078978748,v1:172.21.15.76:6837/2078978748] conn(0x55a6c628bad0 0x55a6c628cf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:10.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.303+0000 7f00d1ad0740 4 client.15105 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:18:10.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.303+0000 7f00d1ad0740 1 -- 172.21.15.26:0/946864613 --> [v2:172.21.15.76:6836/2078978748,v1:172.21.15.76:6837/2078978748] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a6c61df600 con 0x55a6c628bad0 2024-09-11T23:18:10.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.304+0000 7f00ccc92640 1 --2- 172.21.15.26:0/946864613 >> [v2:172.21.15.76:6836/2078978748,v1:172.21.15.76:6837/2078978748] conn(0x55a6c628bad0 0x55a6c628cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:10.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.304+0000 7f00b0ff9640 10 client.15105 ms_handle_connect on v2:172.21.15.76:6836/2078978748 2024-09-11T23:18:10.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.304+0000 7f00ccc92640 1 --2- 172.21.15.26:0/946864613 >> [v2:172.21.15.76:6836/2078978748,v1:172.21.15.76:6837/2078978748] conn(0x55a6c628bad0 0x55a6c628cf40 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9f4f18a7067d45e0 server_cookie=dcb110bd66c34224 in_seq=0 out_seq=0 2024-09-11T23:18:10.313 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:09 smithi076 ceph-mon[35296]: pgmap v332: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 341 B/s wr, 8 op/s 2024-09-11T23:18:10.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.305+0000 7f00b0ff9640 1 -- 172.21.15.26:0/946864613 <== mds.2 v2:172.21.15.76:6836/2078978748 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a6c61df600 con 0x55a6c628bad0 2024-09-11T23:18:10.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.305+0000 7f00b0ff9640 10 client.15105 handle_command_reply: tid=0 2024-09-11T23:18:10.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.305+0000 7f00d1ad0740 1 client.15105 shutdown 2024-09-11T23:18:10.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.305+0000 7f00d1ad0740 1 -- 172.21.15.26:0/946864613 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6c626b8d0 msgr2=0x55a6c627a800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:10.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.305+0000 7f00d1ad0740 1 --2- 172.21.15.26:0/946864613 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6c626b8d0 0x55a6c627a800 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f00c409acb0 tx=0x7f00c40d1a70 comp rx=0 tx=0).stop 2024-09-11T23:18:10.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.305+0000 7f00d1ad0740 1 -- 172.21.15.26:0/946864613 shutdown_connections 2024-09-11T23:18:10.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.305+0000 7f00d1ad0740 1 --2- 172.21.15.26:0/946864613 >> [v2:172.21.15.76:6836/2078978748,v1:172.21.15.76:6837/2078978748] conn(0x55a6c628bad0 0x55a6c628cf40 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:10.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.305+0000 7f00d1ad0740 1 --2- 172.21.15.26:0/946864613 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a6c626c1f0 0x55a6c6277b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:10.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.305+0000 7f00d1ad0740 1 --2- 172.21.15.26:0/946864613 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a6c627afc0 0x55a6c6278090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:10.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.305+0000 7f00d1ad0740 1 --2- 172.21.15.26:0/946864613 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6c626b8d0 0x55a6c627a800 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:10.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.305+0000 7f00d1ad0740 1 -- 172.21.15.26:0/946864613 >> 172.21.15.26:0/946864613 conn(0x55a6c6267f60 msgr2=0x55a6c626dcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:18:10.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.305+0000 7f00b0ff9640 0 client.15105 ms_handle_reset on v2:172.21.15.76:6836/2078978748 2024-09-11T23:18:10.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.305+0000 7f00d1ad0740 1 -- 172.21.15.26:0/946864613 shutdown_connections 2024-09-11T23:18:10.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.305+0000 7f00d1ad0740 1 -- 172.21.15.26:0/946864613 wait complete. 2024-09-11T23:18:10.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.305+0000 7f00d1ad0740 20 client.15105 trim_cache size 0 max 16384 2024-09-11T23:18:10.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.323+0000 7f00d1ad0740 1 Processor -- start 2024-09-11T23:18:10.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.323+0000 7f00d1ad0740 1 -- start start 2024-09-11T23:18:10.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.323+0000 7f00d1ad0740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a6c626c1f0 0x55a6c628cf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:10.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.323+0000 7f00d1ad0740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a6c627afc0 0x55a6c628c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:10.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.323+0000 7f00d1ad0740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6c628b400 0x55a6c6262ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:10.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.323+0000 7f00d1ad0740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55a6c628b280 con 0x55a6c628b400 2024-09-11T23:18:10.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.323+0000 7f00d1ad0740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55a6c628b950 con 0x55a6c627afc0 2024-09-11T23:18:10.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.323+0000 7f00d1ad0740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55a6c628c020 con 0x55a6c626c1f0 2024-09-11T23:18:10.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.323+0000 7f00cdc94640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6c628b400 0x55a6c6262ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:10.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.324+0000 7f00cdc94640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6c628b400 0x55a6c6262ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33360/0 (socket says 172.21.15.26:33360) 2024-09-11T23:18:10.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.324+0000 7f00cdc94640 1 -- 172.21.15.26:0/2631387157 learned_addr learned my addr 172.21.15.26:0/2631387157 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:18:10.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.324+0000 7f00cd493640 1 --2- 172.21.15.26:0/2631387157 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a6c626c1f0 0x55a6c628cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:10.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.324+0000 7f00cdc94640 1 -- 172.21.15.26:0/2631387157 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a6c626c1f0 msgr2=0x55a6c628cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:18:10.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.324+0000 7f00ccc92640 1 --2- 172.21.15.26:0/2631387157 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a6c627afc0 0x55a6c628c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:10.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.324+0000 7f00cdc94640 1 --2- 172.21.15.26:0/2631387157 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a6c626c1f0 0x55a6c628cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:10.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.324+0000 7f00cdc94640 1 -- 172.21.15.26:0/2631387157 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a6c627afc0 msgr2=0x55a6c628c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:18:10.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.324+0000 7f00cdc94640 1 --2- 172.21.15.26:0/2631387157 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a6c627afc0 0x55a6c628c1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:10.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.324+0000 7f00cdc94640 1 -- 172.21.15.26:0/2631387157 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a6c627ab70 con 0x55a6c628b400 2024-09-11T23:18:10.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.324+0000 7f00cd493640 1 --2- 172.21.15.26:0/2631387157 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a6c626c1f0 0x55a6c628cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:18:10.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.324+0000 7f00cdc94640 1 --2- 172.21.15.26:0/2631387157 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6c628b400 0x55a6c6262ce0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f00c0007a50 tx=0x7f00c0012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30336ff8412b3c13 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:18:10.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.324+0000 7f00b2ffd640 1 -- 172.21.15.26:0/2631387157 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00c0023070 con 0x55a6c628b400 2024-09-11T23:18:10.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.324+0000 7f00b2ffd640 1 -- 172.21.15.26:0/2631387157 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00c000c040 con 0x55a6c628b400 2024-09-11T23:18:10.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.324+0000 7f00b2ffd640 1 -- 172.21.15.26:0/2631387157 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00c001e7b0 con 0x55a6c628b400 2024-09-11T23:18:10.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.325+0000 7f00d1ad0740 1 -- 172.21.15.26:0/2631387157 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6c628b400 msgr2=0x55a6c6262ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:10.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.325+0000 7f00d1ad0740 1 --2- 172.21.15.26:0/2631387157 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6c628b400 0x55a6c6262ce0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f00c0007a50 tx=0x7f00c0012000 comp rx=0 tx=0).stop 2024-09-11T23:18:10.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.325+0000 7f00d1ad0740 1 -- 172.21.15.26:0/2631387157 shutdown_connections 2024-09-11T23:18:10.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.325+0000 7f00d1ad0740 1 --2- 172.21.15.26:0/2631387157 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a6c627afc0 0x55a6c628c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:10.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.325+0000 7f00d1ad0740 1 --2- 172.21.15.26:0/2631387157 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6c628b400 0x55a6c6262ce0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:10.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.325+0000 7f00d1ad0740 1 --2- 172.21.15.26:0/2631387157 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a6c626c1f0 0x55a6c628cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:10.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.325+0000 7f00d1ad0740 1 -- 172.21.15.26:0/2631387157 >> 172.21.15.26:0/2631387157 conn(0x55a6c6267f60 msgr2=0x55a6c6268340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:18:10.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.325+0000 7f00d1ad0740 1 -- 172.21.15.26:0/2631387157 shutdown_connections 2024-09-11T23:18:10.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.325+0000 7f00d1ad0740 1 -- 172.21.15.26:0/2631387157 wait complete. 2024-09-11T23:18:10.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.325+0000 7f00d1ad0740 1 -- start start 2024-09-11T23:18:10.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.325+0000 7f00d1ad0740 1 Processor -- start 2024-09-11T23:18:10.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.326+0000 7f00d1ad0740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a6c626c1f0 0x55a6c628e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:10.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.326+0000 7f00d1ad0740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a6c627afc0 0x55a6c628e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:10.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.326+0000 7f00d1ad0740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6c628b400 0x55a6c62f41d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:10.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.326+0000 7f00d1ad0740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55a6c628cdc0 con 0x55a6c628b400 2024-09-11T23:18:10.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.326+0000 7f00d1ad0740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55a6c628c6f0 con 0x55a6c627afc0 2024-09-11T23:18:10.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.326+0000 7f00d1ad0740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55a6c628d490 con 0x55a6c626c1f0 2024-09-11T23:18:10.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.326+0000 7f00d1ad0740 10 client.0 _ll_register_callbacks cb 0x7f00c4017eb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:18:10.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.326+0000 7f00cdc94640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6c628b400 0x55a6c62f41d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:10.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.326+0000 7f00cdc94640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6c628b400 0x55a6c62f41d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33366/0 (socket says 172.21.15.26:33366) 2024-09-11T23:18:10.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.326+0000 7f00cdc94640 1 -- 172.21.15.26:0/2161190982 learned_addr learned my addr 172.21.15.26:0/2161190982 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:18:10.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.326+0000 7f00ccc92640 1 --2- 172.21.15.26:0/2161190982 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a6c627afc0 0x55a6c628e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:10.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.326+0000 7f00cd493640 1 --2- 172.21.15.26:0/2161190982 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a6c626c1f0 0x55a6c628e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:10.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.326+0000 7f00cdc94640 1 -- 172.21.15.26:0/2161190982 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a6c626c1f0 msgr2=0x55a6c628e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:18:10.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.326+0000 7f00cdc94640 1 --2- 172.21.15.26:0/2161190982 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a6c626c1f0 0x55a6c628e2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:10.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.326+0000 7f00cdc94640 1 -- 172.21.15.26:0/2161190982 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a6c627afc0 msgr2=0x55a6c628e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:18:10.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.326+0000 7f00cdc94640 1 --2- 172.21.15.26:0/2161190982 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a6c627afc0 0x55a6c628e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:10.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.326+0000 7f00cdc94640 1 -- 172.21.15.26:0/2161190982 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a6c62f4710 con 0x55a6c628b400 2024-09-11T23:18:10.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.326+0000 7f00cdc94640 1 --2- 172.21.15.26:0/2161190982 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6c628b400 0x55a6c62f41d0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f00c0007810 tx=0x7f00c001fa40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69f0c733d58aa44c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:18:10.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.326+0000 7f00b0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:18:10.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.327+0000 7f00b0ff9640 1 -- 172.21.15.26:0/2161190982 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00c0023040 con 0x55a6c628b400 2024-09-11T23:18:10.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.327+0000 7f00b0ff9640 1 -- 172.21.15.26:0/2161190982 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f00c0028450 con 0x55a6c628b400 2024-09-11T23:18:10.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.327+0000 7f00d1ad0740 1 -- 172.21.15.26:0/2161190982 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f00c000c040 con 0x55a6c628b400 2024-09-11T23:18:10.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.327+0000 7f00b0ff9640 1 -- 172.21.15.26:0/2161190982 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f00c0028650 con 0x55a6c628b400 2024-09-11T23:18:10.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.327+0000 7f00d1ad0740 10 client.15111 fetch_fsmap learned FSMap version 28 2024-09-11T23:18:10.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.327+0000 7f00d1ad0740 1 -- 172.21.15.26:0/2161190982 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a6c62f57d0 con 0x55a6c628b400 2024-09-11T23:18:10.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.327+0000 7f00b0ff9640 1 -- 172.21.15.26:0/2161190982 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00c000c040 con 0x55a6c628b400 2024-09-11T23:18:10.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.327+0000 7f00b0ff9640 1 -- 172.21.15.26:0/2161190982 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f00c001e6e0 con 0x55a6c628b400 2024-09-11T23:18:10.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.327+0000 7f00d1ad0740 10 client.15111 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:18:10.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.327+0000 7f00d1ad0740 10 client.15111 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-11T23:18:10.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.327+0000 7f00d1ad0740 20 client.15111 populate_metadata read hostname 'smithi026' 2024-09-11T23:18:10.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.327+0000 7f00d1ad0740 1 --2- 172.21.15.26:0/2161190982 >> [v2:172.21.15.76:6836/2078978748,v1:172.21.15.76:6837/2078978748] conn(0x55a6c62fa150 0x55a6c62f8e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:10.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.327+0000 7f00d1ad0740 4 client.15111 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-11T23:18:10.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.327+0000 7f00d1ad0740 1 -- 172.21.15.26:0/2161190982 --> [v2:172.21.15.76:6836/2078978748,v1:172.21.15.76:6837/2078978748] -- command(tid 0: {"prefix": "status"}) -- 0x55a6c61d9b10 con 0x55a6c62fa150 2024-09-11T23:18:10.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.328+0000 7f00cd493640 1 --2- 172.21.15.26:0/2161190982 >> [v2:172.21.15.76:6836/2078978748,v1:172.21.15.76:6837/2078978748] conn(0x55a6c62fa150 0x55a6c62f8e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:10.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.328+0000 7f00cd493640 1 --2- 172.21.15.26:0/2161190982 >> [v2:172.21.15.76:6836/2078978748,v1:172.21.15.76:6837/2078978748] conn(0x55a6c62fa150 0x55a6c62f8e60 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b98f099655439488 server_cookie=db24de94b6920759 in_seq=0 out_seq=0 2024-09-11T23:18:10.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.328+0000 7f00b0ff9640 10 client.15111 ms_handle_connect on v2:172.21.15.76:6836/2078978748 2024-09-11T23:18:10.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.328+0000 7f00b0ff9640 1 -- 172.21.15.26:0/2161190982 <== mds.2 v2:172.21.15.76:6836/2078978748 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55a6c61d9b10 con 0x55a6c62fa150 2024-09-11T23:18:10.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.328+0000 7f00b0ff9640 10 client.15111 handle_command_reply: tid=0 2024-09-11T23:18:10.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.328+0000 7f00d1ad0740 1 client.15111 shutdown 2024-09-11T23:18:10.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.328+0000 7f00d1ad0740 1 -- 172.21.15.26:0/2161190982 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6c628b400 msgr2=0x55a6c62f41d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:10.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.328+0000 7f00d1ad0740 1 --2- 172.21.15.26:0/2161190982 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6c628b400 0x55a6c62f41d0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f00c0007810 tx=0x7f00c001fa40 comp rx=0 tx=0).stop 2024-09-11T23:18:10.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.328+0000 7f00d1ad0740 1 -- 172.21.15.26:0/2161190982 shutdown_connections 2024-09-11T23:18:10.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.328+0000 7f00d1ad0740 1 --2- 172.21.15.26:0/2161190982 >> [v2:172.21.15.76:6836/2078978748,v1:172.21.15.76:6837/2078978748] conn(0x55a6c62fa150 0x55a6c62f8e60 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:10.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.328+0000 7f00d1ad0740 1 --2- 172.21.15.26:0/2161190982 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a6c627afc0 0x55a6c628e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:10.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.328+0000 7f00d1ad0740 1 --2- 172.21.15.26:0/2161190982 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6c628b400 0x55a6c62f41d0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:10.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.328+0000 7f00d1ad0740 1 --2- 172.21.15.26:0/2161190982 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a6c626c1f0 0x55a6c628e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:10.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.328+0000 7f00d1ad0740 1 -- 172.21.15.26:0/2161190982 >> 172.21.15.26:0/2161190982 conn(0x55a6c6267f60 msgr2=0x55a6c6262ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:18:10.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.328+0000 7f00b0ff9640 0 client.15111 ms_handle_reset on v2:172.21.15.76:6836/2078978748 2024-09-11T23:18:10.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.329+0000 7f00d1ad0740 1 -- 172.21.15.26:0/2161190982 shutdown_connections 2024-09-11T23:18:10.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.329+0000 7f00d1ad0740 1 -- 172.21.15.26:0/2161190982 wait complete. 2024-09-11T23:18:10.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.329+0000 7f00d1ad0740 20 client.15111 trim_cache size 0 max 16384 2024-09-11T23:18:10.336 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-11T23:18:10.336 INFO:teuthology.orchestra.run.smithi026.stdout: "cluster_fsid": "af7e6fc8-7092-11ef-bcea-c7b262605968", 2024-09-11T23:18:10.336 INFO:teuthology.orchestra.run.smithi026.stdout: "whoami": 2, 2024-09-11T23:18:10.337 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 24458, 2024-09-11T23:18:10.337 INFO:teuthology.orchestra.run.smithi026.stdout: "want_state": "up:active", 2024-09-11T23:18:10.337 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "up:active", 2024-09-11T23:18:10.337 INFO:teuthology.orchestra.run.smithi026.stdout: "fs_name": "cephfs", 2024-09-11T23:18:10.337 INFO:teuthology.orchestra.run.smithi026.stdout: "rank_uptime": 95.384632808000006, 2024-09-11T23:18:10.337 INFO:teuthology.orchestra.run.smithi026.stdout: "mdsmap_epoch": 28, 2024-09-11T23:18:10.337 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch": 85, 2024-09-11T23:18:10.337 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch_barrier": 85, 2024-09-11T23:18:10.337 INFO:teuthology.orchestra.run.smithi026.stdout: "uptime": 126.63994595299999 2024-09-11T23:18:10.337 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-11T23:18:10.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.329+0000 7f00cf6f7640 1 -- 172.21.15.26:0/1362497864 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f00b40a2b80 msgr2=0x7f00b40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:10.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.329+0000 7f00cf6f7640 1 --2- 172.21.15.26:0/1362497864 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f00b40a2b80 0x7f00b40a5040 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f00c0007e10 tx=0x7f00c0004b50 comp rx=0 tx=0).stop 2024-09-11T23:18:10.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.329+0000 7f00cf6f7640 1 -- 172.21.15.26:0/1362497864 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f00c8102dd0 msgr2=0x7f00c8198750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:10.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.329+0000 7f00cf6f7640 1 --2- 172.21.15.26:0/1362497864 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f00c8102dd0 0x7f00c8198750 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f00c400b4a0 tx=0x7f00c400b960 comp rx=0 tx=0).stop 2024-09-11T23:18:10.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.329+0000 7f00cf6f7640 1 -- 172.21.15.26:0/1362497864 shutdown_connections 2024-09-11T23:18:10.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.329+0000 7f00cf6f7640 1 --2- 172.21.15.26:0/1362497864 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f00b40a2b80 0x7f00b40a5040 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:10.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.329+0000 7f00cf6f7640 1 --2- 172.21.15.26:0/1362497864 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f00c8108300 0x7f00c819d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:10.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.329+0000 7f00cf6f7640 1 --2- 172.21.15.26:0/1362497864 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f00c8103780 0x7f00c8198c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:10.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.329+0000 7f00cf6f7640 1 --2- 172.21.15.26:0/1362497864 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f00c8102dd0 0x7f00c8198750 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:10.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.329+0000 7f00cf6f7640 1 -- 172.21.15.26:0/1362497864 >> 172.21.15.26:0/1362497864 conn(0x7f00c80fc7d0 msgr2=0x7f00c8105f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:18:10.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.329+0000 7f00cf6f7640 1 -- 172.21.15.26:0/1362497864 shutdown_connections 2024-09-11T23:18:10.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:10.329+0000 7f00cf6f7640 1 -- 172.21.15.26:0/1362497864 wait complete. 2024-09-11T23:18:11.171 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph tell mds.i status 2024-09-11T23:18:11.434 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:18:11.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.905+0000 7fbf95457640 1 -- 172.21.15.26:0/754792976 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbf90106a90 msgr2=0x7fbf90106e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:11.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.905+0000 7fbf95457640 1 --2- 172.21.15.26:0/754792976 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbf90106a90 0x7fbf90106e70 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fbf80009cd0 tx=0x7fbf8002a8e0 comp rx=0 tx=0).stop 2024-09-11T23:18:11.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.905+0000 7fbf95457640 1 -- 172.21.15.26:0/754792976 shutdown_connections 2024-09-11T23:18:11.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.905+0000 7fbf95457640 1 --2- 172.21.15.26:0/754792976 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbf90100a90 0x7fbf90100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:11.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.905+0000 7fbf95457640 1 --2- 172.21.15.26:0/754792976 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbf90101430 0x7fbf9010d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:11.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.905+0000 7fbf95457640 1 --2- 172.21.15.26:0/754792976 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbf90106a90 0x7fbf90106e70 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:11.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.905+0000 7fbf95457640 1 -- 172.21.15.26:0/754792976 >> 172.21.15.26:0/754792976 conn(0x7fbf900fc7d0 msgr2=0x7fbf900febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:18:11.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.905+0000 7fbf95457640 1 -- 172.21.15.26:0/754792976 shutdown_connections 2024-09-11T23:18:11.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.905+0000 7fbf95457640 1 -- 172.21.15.26:0/754792976 wait complete. 2024-09-11T23:18:11.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.905+0000 7fbf95457640 1 Processor -- start 2024-09-11T23:18:11.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.905+0000 7fbf95457640 1 -- start start 2024-09-11T23:18:11.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.906+0000 7fbf95457640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbf90100a90 0x7fbf901a0440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:11.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.906+0000 7fbf95457640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbf90101430 0x7fbf901a0980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:11.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.906+0000 7fbf95457640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbf90106a90 0x7fbf9019a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:11.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.906+0000 7fbf95457640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fbf90110f30 con 0x7fbf90101430 2024-09-11T23:18:11.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.906+0000 7fbf95457640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fbf90110db0 con 0x7fbf90106a90 2024-09-11T23:18:11.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.906+0000 7fbf95457640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fbf901110b0 con 0x7fbf90100a90 2024-09-11T23:18:11.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.906+0000 7fbf8e7fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbf90101430 0x7fbf901a0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:11.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.906+0000 7fbf8e7fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbf90101430 0x7fbf901a0980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44582/0 (socket says 172.21.15.26:44582) 2024-09-11T23:18:11.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.906+0000 7fbf8e7fc640 1 -- 172.21.15.26:0/2821701976 learned_addr learned my addr 172.21.15.26:0/2821701976 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:18:11.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.906+0000 7fbf8effd640 1 --2- 172.21.15.26:0/2821701976 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbf90100a90 0x7fbf901a0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:11.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.906+0000 7fbf8f7fe640 1 --2- 172.21.15.26:0/2821701976 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbf90106a90 0x7fbf9019a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:11.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.906+0000 7fbf8e7fc640 1 -- 172.21.15.26:0/2821701976 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbf90100a90 msgr2=0x7fbf901a0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:11.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.906+0000 7fbf8e7fc640 1 --2- 172.21.15.26:0/2821701976 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbf90100a90 0x7fbf901a0440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:11.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.906+0000 7fbf8e7fc640 1 -- 172.21.15.26:0/2821701976 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbf90106a90 msgr2=0x7fbf9019a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:11.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.906+0000 7fbf8e7fc640 1 --2- 172.21.15.26:0/2821701976 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbf90106a90 0x7fbf9019a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:11.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.906+0000 7fbf8e7fc640 1 -- 172.21.15.26:0/2821701976 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf9019ad30 con 0x7fbf90101430 2024-09-11T23:18:11.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.906+0000 7fbf8f7fe640 1 --2- 172.21.15.26:0/2821701976 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbf90106a90 0x7fbf9019a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:18:11.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.906+0000 7fbf8e7fc640 1 --2- 172.21.15.26:0/2821701976 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbf90101430 0x7fbf901a0980 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fbf7800b4a0 tx=0x7fbf7800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:18:11.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.906+0000 7fbf87fff640 1 -- 172.21.15.26:0/2821701976 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf78014070 con 0x7fbf90101430 2024-09-11T23:18:11.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.906+0000 7fbf95457640 1 -- 172.21.15.26:0/2821701976 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf9019b020 con 0x7fbf90101430 2024-09-11T23:18:11.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.906+0000 7fbf87fff640 1 -- 172.21.15.26:0/2821701976 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf78010af0 con 0x7fbf90101430 2024-09-11T23:18:11.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.906+0000 7fbf87fff640 1 -- 172.21.15.26:0/2821701976 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf78017d10 con 0x7fbf90101430 2024-09-11T23:18:11.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.906+0000 7fbf95457640 1 -- 172.21.15.26:0/2821701976 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf901a7310 con 0x7fbf90101430 2024-09-11T23:18:11.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.909+0000 7fbf87fff640 1 -- 172.21.15.26:0/2821701976 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fbf78017070 con 0x7fbf90101430 2024-09-11T23:18:11.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.909+0000 7fbf97830740 1 Processor -- start 2024-09-11T23:18:11.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.909+0000 7fbf97830740 1 -- start start 2024-09-11T23:18:11.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.910+0000 7fbf97830740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eb8a7999d0 0x55eb8a799db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:11.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.910+0000 7fbf97830740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eb8a79a2f0 0x55eb8a7a4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:11.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.910+0000 7fbf97830740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55eb8a7a4bb0 0x55eb8a7a6fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:11.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.910+0000 7fbf97830740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55eb8a700090 con 0x55eb8a7999d0 2024-09-11T23:18:11.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.910+0000 7fbf97830740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55eb8a6a7a70 con 0x55eb8a7a4bb0 2024-09-11T23:18:11.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.910+0000 7fbf97830740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55eb8a6fc050 con 0x55eb8a79a2f0 2024-09-11T23:18:11.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.910+0000 7fbf8f7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eb8a7999d0 0x55eb8a799db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:11.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.910+0000 7fbf87fff640 1 --2- 172.21.15.26:0/2821701976 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fbf7c0a2b80 0x7fbf7c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:11.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.910+0000 7fbf8f7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eb8a7999d0 0x55eb8a799db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44596/0 (socket says 172.21.15.26:44596) 2024-09-11T23:18:11.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.910+0000 7fbf8f7fe640 1 -- 172.21.15.26:0/835820503 learned_addr learned my addr 172.21.15.26:0/835820503 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:18:11.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.910+0000 7fbf8effd640 1 --2- 172.21.15.26:0/835820503 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eb8a79a2f0 0x55eb8a7a4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:11.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.910+0000 7fbf8e7fc640 1 --2- 172.21.15.26:0/835820503 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55eb8a7a4bb0 0x55eb8a7a6fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:11.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.910+0000 7fbf8f7fe640 1 --2- 172.21.15.26:0/2821701976 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fbf7c0a2b80 0x7fbf7c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:11.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.910+0000 7fbf87fff640 1 -- 172.21.15.26:0/2821701976 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fbf780ce4c0 con 0x7fbf90101430 2024-09-11T23:18:11.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.910+0000 7fbf8f7fe640 1 -- 172.21.15.26:0/835820503 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eb8a79a2f0 msgr2=0x55eb8a7a4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:18:11.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.910+0000 7fbf8f7fe640 1 --2- 172.21.15.26:0/835820503 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eb8a79a2f0 0x55eb8a7a4670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:11.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.910+0000 7fbf8f7fe640 1 -- 172.21.15.26:0/835820503 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55eb8a7a4bb0 msgr2=0x55eb8a7a6fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:18:11.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.910+0000 7fbf8f7fe640 1 --2- 172.21.15.26:0/835820503 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55eb8a7a4bb0 0x55eb8a7a6fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:11.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.910+0000 7fbf8f7fe640 1 -- 172.21.15.26:0/835820503 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb8a6b4130 con 0x55eb8a7999d0 2024-09-11T23:18:11.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.910+0000 7fbf8e7fc640 1 --2- 172.21.15.26:0/835820503 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55eb8a7a4bb0 0x55eb8a7a6fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:18:11.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.910+0000 7fbf8f7fe640 1 --2- 172.21.15.26:0/835820503 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eb8a7999d0 0x55eb8a799db0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fbf880133b0 tx=0x7fbf88013870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fef04788a64ede server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:18:11.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.911+0000 7fbf857fa640 1 -- 172.21.15.26:0/835820503 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf8801a070 con 0x55eb8a7999d0 2024-09-11T23:18:11.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.911+0000 7fbf857fa640 1 -- 172.21.15.26:0/835820503 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf8800ab00 con 0x55eb8a7999d0 2024-09-11T23:18:11.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.911+0000 7fbf857fa640 1 -- 172.21.15.26:0/835820503 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf8800f040 con 0x55eb8a7999d0 2024-09-11T23:18:11.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.911+0000 7fbf8f7fe640 1 --2- 172.21.15.26:0/2821701976 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fbf7c0a2b80 0x7fbf7c0a5040 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fbf8801e000 tx=0x7fbf8801c000 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:18:11.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.911+0000 7fbf97830740 1 -- 172.21.15.26:0/835820503 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eb8a7999d0 msgr2=0x55eb8a799db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:11.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.911+0000 7fbf97830740 1 --2- 172.21.15.26:0/835820503 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eb8a7999d0 0x55eb8a799db0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fbf880133b0 tx=0x7fbf88013870 comp rx=0 tx=0).stop 2024-09-11T23:18:11.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.911+0000 7fbf97830740 1 -- 172.21.15.26:0/835820503 shutdown_connections 2024-09-11T23:18:11.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.911+0000 7fbf97830740 1 --2- 172.21.15.26:0/835820503 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55eb8a7a4bb0 0x55eb8a7a6fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:11.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.911+0000 7fbf97830740 1 --2- 172.21.15.26:0/835820503 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eb8a79a2f0 0x55eb8a7a4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:11.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.911+0000 7fbf97830740 1 --2- 172.21.15.26:0/835820503 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eb8a7999d0 0x55eb8a799db0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:11.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.911+0000 7fbf97830740 1 -- 172.21.15.26:0/835820503 >> 172.21.15.26:0/835820503 conn(0x55eb8a795f60 msgr2=0x55eb8a796340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:18:11.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.912+0000 7fbf97830740 1 -- 172.21.15.26:0/835820503 shutdown_connections 2024-09-11T23:18:11.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.912+0000 7fbf97830740 1 -- 172.21.15.26:0/835820503 wait complete. 2024-09-11T23:18:11.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.912+0000 7fbf97830740 1 -- start start 2024-09-11T23:18:11.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.912+0000 7fbf97830740 1 Processor -- start 2024-09-11T23:18:11.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.912+0000 7fbf97830740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55eb8a7999d0 0x55eb8a7a43f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:11.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.912+0000 7fbf97830740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eb8a79a2f0 0x55eb8a7a1740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:11.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.912+0000 7fbf97830740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eb8a7a4bb0 0x55eb8a7a1c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:11.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.912+0000 7fbf97830740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55eb8a7a7780 con 0x55eb8a7a4bb0 2024-09-11T23:18:11.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.912+0000 7fbf97830740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55eb8a6baf20 con 0x55eb8a7999d0 2024-09-11T23:18:11.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.912+0000 7fbf97830740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55eb8a7a7900 con 0x55eb8a79a2f0 2024-09-11T23:18:11.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.912+0000 7fbf97830740 10 client.0 _ll_register_callbacks cb 0x55eb8a684030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:18:11.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.912+0000 7fbf8f7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eb8a7a4bb0 0x55eb8a7a1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:11.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.912+0000 7fbf8f7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eb8a7a4bb0 0x55eb8a7a1c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44612/0 (socket says 172.21.15.26:44612) 2024-09-11T23:18:11.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.912+0000 7fbf8f7fe640 1 -- 172.21.15.26:0/2051532134 learned_addr learned my addr 172.21.15.26:0/2051532134 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:18:11.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.912+0000 7fbf8e7fc640 1 --2- 172.21.15.26:0/2051532134 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eb8a79a2f0 0x55eb8a7a1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:11.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.912+0000 7fbf8effd640 1 --2- 172.21.15.26:0/2051532134 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55eb8a7999d0 0x55eb8a7a43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:11.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.913+0000 7fbf8f7fe640 1 -- 172.21.15.26:0/2051532134 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eb8a79a2f0 msgr2=0x55eb8a7a1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:18:11.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.913+0000 7fbf8f7fe640 1 --2- 172.21.15.26:0/2051532134 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eb8a79a2f0 0x55eb8a7a1740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:11.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.913+0000 7fbf8f7fe640 1 -- 172.21.15.26:0/2051532134 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55eb8a7999d0 msgr2=0x55eb8a7a43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:18:11.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.913+0000 7fbf8f7fe640 1 --2- 172.21.15.26:0/2051532134 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55eb8a7999d0 0x55eb8a7a43f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:11.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.913+0000 7fbf8f7fe640 1 -- 172.21.15.26:0/2051532134 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb8a4712b0 con 0x55eb8a7a4bb0 2024-09-11T23:18:11.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.913+0000 7fbf8f7fe640 1 --2- 172.21.15.26:0/2051532134 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eb8a7a4bb0 0x55eb8a7a1c80 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fbf88015f90 tx=0x7fbf8800ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d6cf7bc8446dae3 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:18:11.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.913+0000 7fbf767fc640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:18:11.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.913+0000 7fbf767fc640 1 -- 172.21.15.26:0/2051532134 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf8802c020 con 0x55eb8a7a4bb0 2024-09-11T23:18:11.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.913+0000 7fbf767fc640 1 -- 172.21.15.26:0/2051532134 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fbf8801f830 con 0x55eb8a7a4bb0 2024-09-11T23:18:11.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.913+0000 7fbf97830740 1 -- 172.21.15.26:0/2051532134 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbf8800ab00 con 0x55eb8a7a4bb0 2024-09-11T23:18:11.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.913+0000 7fbf767fc640 1 -- 172.21.15.26:0/2051532134 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fbf8801a040 con 0x55eb8a7a4bb0 2024-09-11T23:18:11.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.913+0000 7fbf97830740 10 client.15123 fetch_fsmap learned FSMap version 28 2024-09-11T23:18:11.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.913+0000 7fbf97830740 1 -- 172.21.15.26:0/2051532134 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55eb8a7b2fb0 con 0x55eb8a7a4bb0 2024-09-11T23:18:11.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.913+0000 7fbf767fc640 1 -- 172.21.15.26:0/2051532134 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf8800ab00 con 0x55eb8a7a4bb0 2024-09-11T23:18:11.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.914+0000 7fbf767fc640 1 -- 172.21.15.26:0/2051532134 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fbf88015040 con 0x55eb8a7a4bb0 2024-09-11T23:18:11.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.914+0000 7fbf97830740 10 client.15123 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:18:11.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.914+0000 7fbf97830740 10 client.15123 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-11T23:18:11.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.914+0000 7fbf97830740 20 client.15123 populate_metadata read hostname 'smithi026' 2024-09-11T23:18:11.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.914+0000 7fbf97830740 1 --2- 172.21.15.26:0/2051532134 >> [v2:172.21.15.136:6838/3211398630,v1:172.21.15.136:6839/3211398630] conn(0x55eb8a7b5210 0x55eb8a7b6680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:11.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.914+0000 7fbf97830740 4 client.15123 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:18:11.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.914+0000 7fbf97830740 1 -- 172.21.15.26:0/2051532134 --> [v2:172.21.15.136:6838/3211398630,v1:172.21.15.136:6839/3211398630] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eb8a70d600 con 0x55eb8a7b5210 2024-09-11T23:18:11.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.914+0000 7fbf8effd640 1 --2- 172.21.15.26:0/2051532134 >> [v2:172.21.15.136:6838/3211398630,v1:172.21.15.136:6839/3211398630] conn(0x55eb8a7b5210 0x55eb8a7b6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:11.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.914+0000 7fbf767fc640 10 client.15123 ms_handle_connect on v2:172.21.15.136:6838/3211398630 2024-09-11T23:18:11.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.914+0000 7fbf8effd640 1 --2- 172.21.15.26:0/2051532134 >> [v2:172.21.15.136:6838/3211398630,v1:172.21.15.136:6839/3211398630] conn(0x55eb8a7b5210 0x55eb8a7b6680 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a230a3c9c5e2284d server_cookie=302a392107711b3e in_seq=0 out_seq=0 2024-09-11T23:18:11.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.915+0000 7fbf767fc640 1 -- 172.21.15.26:0/2051532134 <== mds.1 v2:172.21.15.136:6838/3211398630 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55eb8a70d600 con 0x55eb8a7b5210 2024-09-11T23:18:11.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.915+0000 7fbf767fc640 10 client.15123 handle_command_reply: tid=0 2024-09-11T23:18:11.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.915+0000 7fbf97830740 1 client.15123 shutdown 2024-09-11T23:18:11.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.915+0000 7fbf97830740 1 -- 172.21.15.26:0/2051532134 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eb8a7a4bb0 msgr2=0x55eb8a7a1c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:11.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.915+0000 7fbf97830740 1 --2- 172.21.15.26:0/2051532134 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eb8a7a4bb0 0x55eb8a7a1c80 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fbf88015f90 tx=0x7fbf8800ace0 comp rx=0 tx=0).stop 2024-09-11T23:18:11.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.915+0000 7fbf97830740 1 -- 172.21.15.26:0/2051532134 shutdown_connections 2024-09-11T23:18:11.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.915+0000 7fbf97830740 1 --2- 172.21.15.26:0/2051532134 >> [v2:172.21.15.136:6838/3211398630,v1:172.21.15.136:6839/3211398630] conn(0x55eb8a7b5210 0x55eb8a7b6680 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:11.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.915+0000 7fbf97830740 1 --2- 172.21.15.26:0/2051532134 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eb8a7a4bb0 0x55eb8a7a1c80 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:11.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.915+0000 7fbf97830740 1 --2- 172.21.15.26:0/2051532134 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eb8a79a2f0 0x55eb8a7a1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:11.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.915+0000 7fbf767fc640 0 client.15123 ms_handle_reset on v2:172.21.15.136:6838/3211398630 2024-09-11T23:18:11.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.915+0000 7fbf97830740 1 --2- 172.21.15.26:0/2051532134 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55eb8a7999d0 0x55eb8a7a43f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:11.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.915+0000 7fbf97830740 1 -- 172.21.15.26:0/2051532134 >> 172.21.15.26:0/2051532134 conn(0x55eb8a795f60 msgr2=0x55eb8a79bc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:18:11.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.916+0000 7fbf97830740 1 -- 172.21.15.26:0/2051532134 shutdown_connections 2024-09-11T23:18:11.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.916+0000 7fbf97830740 1 -- 172.21.15.26:0/2051532134 wait complete. 2024-09-11T23:18:11.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.916+0000 7fbf97830740 20 client.15123 trim_cache size 0 max 16384 2024-09-11T23:18:11.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.934+0000 7fbf97830740 1 Processor -- start 2024-09-11T23:18:11.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.934+0000 7fbf97830740 1 -- start start 2024-09-11T23:18:11.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.934+0000 7fbf97830740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55eb8a7999d0 0x55eb8a7b58e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:11.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.934+0000 7fbf97830740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eb8a79a2f0 0x55eb8a7a35a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:11.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.934+0000 7fbf97830740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eb8a7a4bb0 0x55eb8a79e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:11.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.934+0000 7fbf97830740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55eb8a7b49c0 con 0x55eb8a79a2f0 2024-09-11T23:18:11.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.934+0000 7fbf97830740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55eb8a7b5090 con 0x55eb8a7999d0 2024-09-11T23:18:11.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.934+0000 7fbf97830740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55eb8a7b5760 con 0x55eb8a7a4bb0 2024-09-11T23:18:11.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.934+0000 7fbf8e7fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eb8a79a2f0 0x55eb8a7a35a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:11.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.934+0000 7fbf8e7fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eb8a79a2f0 0x55eb8a7a35a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44628/0 (socket says 172.21.15.26:44628) 2024-09-11T23:18:11.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.934+0000 7fbf8e7fc640 1 -- 172.21.15.26:0/1750836610 learned_addr learned my addr 172.21.15.26:0/1750836610 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:18:11.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.934+0000 7fbf8effd640 1 --2- 172.21.15.26:0/1750836610 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55eb8a7999d0 0x55eb8a7b58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:11.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.935+0000 7fbf8f7fe640 1 --2- 172.21.15.26:0/1750836610 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eb8a7a4bb0 0x55eb8a79e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:11.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.935+0000 7fbf8e7fc640 1 -- 172.21.15.26:0/1750836610 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eb8a7a4bb0 msgr2=0x55eb8a79e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:18:11.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.935+0000 7fbf8e7fc640 1 --2- 172.21.15.26:0/1750836610 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eb8a7a4bb0 0x55eb8a79e9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:11.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.935+0000 7fbf8e7fc640 1 -- 172.21.15.26:0/1750836610 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55eb8a7999d0 msgr2=0x55eb8a7b58e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:18:11.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.935+0000 7fbf8e7fc640 1 --2- 172.21.15.26:0/1750836610 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55eb8a7999d0 0x55eb8a7b58e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:11.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.935+0000 7fbf8e7fc640 1 -- 172.21.15.26:0/1750836610 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb8a7a72e0 con 0x55eb8a79a2f0 2024-09-11T23:18:11.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.935+0000 7fbf8e7fc640 1 --2- 172.21.15.26:0/1750836610 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eb8a79a2f0 0x55eb8a7a35a0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fbf78000c00 tx=0x7fbf7801c310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e656576143afa371 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:18:11.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.935+0000 7fbf84ff9640 1 -- 172.21.15.26:0/1750836610 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf7801e040 con 0x55eb8a79a2f0 2024-09-11T23:18:11.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.935+0000 7fbf84ff9640 1 -- 172.21.15.26:0/1750836610 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf780a3990 con 0x55eb8a79a2f0 2024-09-11T23:18:11.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.935+0000 7fbf84ff9640 1 -- 172.21.15.26:0/1750836610 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf7801ecf0 con 0x55eb8a79a2f0 2024-09-11T23:18:11.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.936+0000 7fbf97830740 1 -- 172.21.15.26:0/1750836610 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eb8a79a2f0 msgr2=0x55eb8a7a35a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:11.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.936+0000 7fbf97830740 1 --2- 172.21.15.26:0/1750836610 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eb8a79a2f0 0x55eb8a7a35a0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fbf78000c00 tx=0x7fbf7801c310 comp rx=0 tx=0).stop 2024-09-11T23:18:11.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.936+0000 7fbf97830740 1 -- 172.21.15.26:0/1750836610 shutdown_connections 2024-09-11T23:18:11.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.936+0000 7fbf97830740 1 --2- 172.21.15.26:0/1750836610 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eb8a7a4bb0 0x55eb8a79e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:11.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.936+0000 7fbf97830740 1 --2- 172.21.15.26:0/1750836610 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eb8a79a2f0 0x55eb8a7a35a0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:11.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.936+0000 7fbf97830740 1 --2- 172.21.15.26:0/1750836610 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55eb8a7999d0 0x55eb8a7b58e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:11.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.936+0000 7fbf97830740 1 -- 172.21.15.26:0/1750836610 >> 172.21.15.26:0/1750836610 conn(0x55eb8a795f60 msgr2=0x55eb8a7a2e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:18:11.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.936+0000 7fbf97830740 1 -- 172.21.15.26:0/1750836610 shutdown_connections 2024-09-11T23:18:11.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.936+0000 7fbf97830740 1 -- 172.21.15.26:0/1750836610 wait complete. 2024-09-11T23:18:11.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.936+0000 7fbf97830740 1 -- start start 2024-09-11T23:18:11.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.936+0000 7fbf97830740 1 Processor -- start 2024-09-11T23:18:11.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.937+0000 7fbf97830740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55eb8a7999d0 0x55eb8a7bec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:11.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.937+0000 7fbf97830740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eb8a79a2f0 0x55eb8a7bf170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:11.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.937+0000 7fbf97830740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eb8a7a4bb0 0x55eb8a7ba010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:11.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.937+0000 7fbf97830740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55eb8a7b6500 con 0x55eb8a7a4bb0 2024-09-11T23:18:11.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.937+0000 7fbf97830740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55eb8a7b5e30 con 0x55eb8a7999d0 2024-09-11T23:18:11.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.937+0000 7fbf97830740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55eb8a7b6bd0 con 0x55eb8a79a2f0 2024-09-11T23:18:11.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.937+0000 7fbf97830740 10 client.0 _ll_register_callbacks cb 0x7fbf80009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:18:11.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.937+0000 7fbf8f7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eb8a7a4bb0 0x55eb8a7ba010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:11.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.937+0000 7fbf8e7fc640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eb8a79a2f0 0x55eb8a7bf170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:11.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.937+0000 7fbf8f7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eb8a7a4bb0 0x55eb8a7ba010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44644/0 (socket says 172.21.15.26:44644) 2024-09-11T23:18:11.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.937+0000 7fbf8f7fe640 1 -- 172.21.15.26:0/1164183444 learned_addr learned my addr 172.21.15.26:0/1164183444 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:18:11.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.937+0000 7fbf8effd640 1 --2- 172.21.15.26:0/1164183444 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55eb8a7999d0 0x55eb8a7bec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:11.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.937+0000 7fbf8f7fe640 1 -- 172.21.15.26:0/1164183444 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eb8a79a2f0 msgr2=0x55eb8a7bf170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:18:11.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.937+0000 7fbf8f7fe640 1 --2- 172.21.15.26:0/1164183444 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eb8a79a2f0 0x55eb8a7bf170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:11.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.937+0000 7fbf8f7fe640 1 -- 172.21.15.26:0/1164183444 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55eb8a7999d0 msgr2=0x55eb8a7bec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:18:11.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.937+0000 7fbf8f7fe640 1 --2- 172.21.15.26:0/1164183444 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55eb8a7999d0 0x55eb8a7bec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:11.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.937+0000 7fbf8f7fe640 1 -- 172.21.15.26:0/1164183444 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb8a7ba840 con 0x55eb8a7a4bb0 2024-09-11T23:18:11.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.937+0000 7fbf8effd640 1 --2- 172.21.15.26:0/1164183444 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55eb8a7999d0 0x55eb8a7bec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:18:11.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.937+0000 7fbf8f7fe640 1 --2- 172.21.15.26:0/1164183444 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eb8a7a4bb0 0x55eb8a7ba010 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fbf8800a2c0 tx=0x7fbf880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=160ac0e65fd20e64 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:18:11.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.937+0000 7fbf767fc640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:18:11.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.938+0000 7fbf767fc640 1 -- 172.21.15.26:0/1164183444 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf8802c040 con 0x55eb8a7a4bb0 2024-09-11T23:18:11.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.938+0000 7fbf767fc640 1 -- 172.21.15.26:0/1164183444 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fbf8801db70 con 0x55eb8a7a4bb0 2024-09-11T23:18:11.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.938+0000 7fbf97830740 1 -- 172.21.15.26:0/1164183444 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbf780a3990 con 0x55eb8a7a4bb0 2024-09-11T23:18:11.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.938+0000 7fbf767fc640 1 -- 172.21.15.26:0/1164183444 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fbf88013d70 con 0x55eb8a7a4bb0 2024-09-11T23:18:11.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.938+0000 7fbf97830740 10 client.15129 fetch_fsmap learned FSMap version 28 2024-09-11T23:18:11.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.938+0000 7fbf97830740 1 -- 172.21.15.26:0/1164183444 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55eb8a81f4c0 con 0x55eb8a7a4bb0 2024-09-11T23:18:11.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.938+0000 7fbf767fc640 1 -- 172.21.15.26:0/1164183444 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf780a3990 con 0x55eb8a7a4bb0 2024-09-11T23:18:11.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.938+0000 7fbf767fc640 1 -- 172.21.15.26:0/1164183444 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fbf8803a020 con 0x55eb8a7a4bb0 2024-09-11T23:18:11.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.938+0000 7fbf97830740 10 client.15129 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:18:11.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.938+0000 7fbf97830740 10 client.15129 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-11T23:18:11.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.938+0000 7fbf97830740 20 client.15129 populate_metadata read hostname 'smithi026' 2024-09-11T23:18:11.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.938+0000 7fbf97830740 1 --2- 172.21.15.26:0/1164183444 >> [v2:172.21.15.136:6838/3211398630,v1:172.21.15.136:6839/3211398630] conn(0x55eb8a8240d0 0x55eb8a822de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:11.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.938+0000 7fbf97830740 4 client.15129 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-11T23:18:11.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.938+0000 7fbf97830740 1 -- 172.21.15.26:0/1164183444 --> [v2:172.21.15.136:6838/3211398630,v1:172.21.15.136:6839/3211398630] -- command(tid 0: {"prefix": "status"}) -- 0x55eb8a707b10 con 0x55eb8a8240d0 2024-09-11T23:18:11.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.939+0000 7fbf8effd640 1 --2- 172.21.15.26:0/1164183444 >> [v2:172.21.15.136:6838/3211398630,v1:172.21.15.136:6839/3211398630] conn(0x55eb8a8240d0 0x55eb8a822de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:11.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.939+0000 7fbf8effd640 1 --2- 172.21.15.26:0/1164183444 >> [v2:172.21.15.136:6838/3211398630,v1:172.21.15.136:6839/3211398630] conn(0x55eb8a8240d0 0x55eb8a822de0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d5d14ada67d0de79 server_cookie=6de00e1069f83b38 in_seq=0 out_seq=0 2024-09-11T23:18:11.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.939+0000 7fbf767fc640 10 client.15129 ms_handle_connect on v2:172.21.15.136:6838/3211398630 2024-09-11T23:18:11.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.939+0000 7fbf767fc640 1 -- 172.21.15.26:0/1164183444 <== mds.1 v2:172.21.15.136:6838/3211398630 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55eb8a707b10 con 0x55eb8a8240d0 2024-09-11T23:18:11.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.939+0000 7fbf767fc640 10 client.15129 handle_command_reply: tid=0 2024-09-11T23:18:11.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.939+0000 7fbf97830740 1 client.15129 shutdown 2024-09-11T23:18:11.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.939+0000 7fbf97830740 1 -- 172.21.15.26:0/1164183444 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eb8a7a4bb0 msgr2=0x55eb8a7ba010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:11.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.939+0000 7fbf97830740 1 --2- 172.21.15.26:0/1164183444 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eb8a7a4bb0 0x55eb8a7ba010 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fbf8800a2c0 tx=0x7fbf880046c0 comp rx=0 tx=0).stop 2024-09-11T23:18:11.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.939+0000 7fbf97830740 1 -- 172.21.15.26:0/1164183444 shutdown_connections 2024-09-11T23:18:11.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.939+0000 7fbf97830740 1 --2- 172.21.15.26:0/1164183444 >> [v2:172.21.15.136:6838/3211398630,v1:172.21.15.136:6839/3211398630] conn(0x55eb8a8240d0 0x55eb8a822de0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:11.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.939+0000 7fbf97830740 1 --2- 172.21.15.26:0/1164183444 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eb8a7a4bb0 0x55eb8a7ba010 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:11.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.939+0000 7fbf97830740 1 --2- 172.21.15.26:0/1164183444 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eb8a79a2f0 0x55eb8a7bf170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:11.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.939+0000 7fbf97830740 1 --2- 172.21.15.26:0/1164183444 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55eb8a7999d0 0x55eb8a7bec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:11.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.939+0000 7fbf767fc640 0 client.15129 ms_handle_reset on v2:172.21.15.136:6838/3211398630 2024-09-11T23:18:11.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.939+0000 7fbf97830740 1 -- 172.21.15.26:0/1164183444 >> 172.21.15.26:0/1164183444 conn(0x55eb8a795f60 msgr2=0x55eb8a795000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:18:11.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.939+0000 7fbf97830740 1 -- 172.21.15.26:0/1164183444 shutdown_connections 2024-09-11T23:18:11.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.939+0000 7fbf97830740 1 -- 172.21.15.26:0/1164183444 wait complete. 2024-09-11T23:18:11.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.940+0000 7fbf97830740 20 client.15129 trim_cache size 0 max 16384 2024-09-11T23:18:11.947 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-11T23:18:11.947 INFO:teuthology.orchestra.run.smithi026.stdout: "cluster_fsid": "af7e6fc8-7092-11ef-bcea-c7b262605968", 2024-09-11T23:18:11.947 INFO:teuthology.orchestra.run.smithi026.stdout: "whoami": 1, 2024-09-11T23:18:11.948 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 24463, 2024-09-11T23:18:11.948 INFO:teuthology.orchestra.run.smithi026.stdout: "want_state": "up:active", 2024-09-11T23:18:11.948 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "up:active", 2024-09-11T23:18:11.948 INFO:teuthology.orchestra.run.smithi026.stdout: "fs_name": "cephfs", 2024-09-11T23:18:11.948 INFO:teuthology.orchestra.run.smithi026.stdout: "rank_uptime": 98.009810178999999, 2024-09-11T23:18:11.948 INFO:teuthology.orchestra.run.smithi026.stdout: "mdsmap_epoch": 28, 2024-09-11T23:18:11.948 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch": 85, 2024-09-11T23:18:11.948 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch_barrier": 85, 2024-09-11T23:18:11.948 INFO:teuthology.orchestra.run.smithi026.stdout: "uptime": 125.718532142 2024-09-11T23:18:11.949 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-11T23:18:11.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.940+0000 7fbf95457640 1 -- 172.21.15.26:0/2821701976 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fbf7c0a2b80 msgr2=0x7fbf7c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:11.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.940+0000 7fbf95457640 1 --2- 172.21.15.26:0/2821701976 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fbf7c0a2b80 0x7fbf7c0a5040 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fbf8801e000 tx=0x7fbf8801c000 comp rx=0 tx=0).stop 2024-09-11T23:18:11.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.940+0000 7fbf95457640 1 -- 172.21.15.26:0/2821701976 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbf90101430 msgr2=0x7fbf901a0980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:11.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.940+0000 7fbf95457640 1 --2- 172.21.15.26:0/2821701976 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbf90101430 0x7fbf901a0980 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fbf7800b4a0 tx=0x7fbf7800b960 comp rx=0 tx=0).stop 2024-09-11T23:18:11.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.940+0000 7fbf95457640 1 -- 172.21.15.26:0/2821701976 shutdown_connections 2024-09-11T23:18:11.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.940+0000 7fbf95457640 1 --2- 172.21.15.26:0/2821701976 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fbf7c0a2b80 0x7fbf7c0a5040 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:11.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.940+0000 7fbf95457640 1 --2- 172.21.15.26:0/2821701976 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbf90106a90 0x7fbf9019a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:11.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.940+0000 7fbf95457640 1 --2- 172.21.15.26:0/2821701976 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbf90101430 0x7fbf901a0980 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:11.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.940+0000 7fbf95457640 1 --2- 172.21.15.26:0/2821701976 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbf90100a90 0x7fbf901a0440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:11.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.940+0000 7fbf95457640 1 -- 172.21.15.26:0/2821701976 >> 172.21.15.26:0/2821701976 conn(0x7fbf900fc7d0 msgr2=0x7fbf9006aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:18:11.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.940+0000 7fbf95457640 1 -- 172.21.15.26:0/2821701976 shutdown_connections 2024-09-11T23:18:11.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:11.940+0000 7fbf95457640 1 -- 172.21.15.26:0/2821701976 wait complete. 2024-09-11T23:18:12.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:11 smithi136 ceph-mon[35121]: pgmap v333: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 10 op/s 2024-09-11T23:18:12.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:11 smithi026 ceph-mon[31172]: pgmap v333: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 10 op/s 2024-09-11T23:18:12.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:11 smithi076 ceph-mon[35296]: pgmap v333: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 10 op/s 2024-09-11T23:18:12.821 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph tell mds.b status 2024-09-11T23:18:13.087 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:18:13.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.570+0000 7f039ebe8640 1 -- 172.21.15.26:0/1685239730 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0398102790 msgr2=0x7f03981067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:13.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.570+0000 7f039ebe8640 1 --2- 172.21.15.26:0/1685239730 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0398102790 0x7f03981067e0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f0380009ca0 tx=0x7f038002a8c0 comp rx=0 tx=0).stop 2024-09-11T23:18:13.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.570+0000 7f039ebe8640 1 -- 172.21.15.26:0/1685239730 shutdown_connections 2024-09-11T23:18:13.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.570+0000 7f039ebe8640 1 --2- 172.21.15.26:0/1685239730 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0398106f10 0x7f039810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:13.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.570+0000 7f039ebe8640 1 --2- 172.21.15.26:0/1685239730 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0398102790 0x7f03981067e0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:13.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.570+0000 7f039ebe8640 1 --2- 172.21.15.26:0/1685239730 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0398101de0 0x7f03981021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:13.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.570+0000 7f039ebe8640 1 -- 172.21.15.26:0/1685239730 >> 172.21.15.26:0/1685239730 conn(0x7f03980fd8c0 msgr2=0x7f03980ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:18:13.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.570+0000 7f039ebe8640 1 -- 172.21.15.26:0/1685239730 shutdown_connections 2024-09-11T23:18:13.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.570+0000 7f039ebe8640 1 -- 172.21.15.26:0/1685239730 wait complete. 2024-09-11T23:18:13.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.571+0000 7f039ebe8640 1 Processor -- start 2024-09-11T23:18:13.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.571+0000 7f039ebe8640 1 -- start start 2024-09-11T23:18:13.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.571+0000 7f039ebe8640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0398101de0 0x7f03981985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:13.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.571+0000 7f039ebe8640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0398102790 0x7f0398198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:13.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.571+0000 7f039ebe8640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0398106f10 0x7f039819ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:13.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.571+0000 7f039ebe8640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f0398112010 con 0x7f0398101de0 2024-09-11T23:18:13.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.571+0000 7f039ebe8640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f0398111e90 con 0x7f0398102790 2024-09-11T23:18:13.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.571+0000 7f039ebe8640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f0398112190 con 0x7f0398106f10 2024-09-11T23:18:13.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.571+0000 7f039c984640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0398101de0 0x7f03981985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:13.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.571+0000 7f039c984640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0398101de0 0x7f03981985b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44658/0 (socket says 172.21.15.26:44658) 2024-09-11T23:18:13.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.571+0000 7f039c984640 1 -- 172.21.15.26:0/1875362152 learned_addr learned my addr 172.21.15.26:0/1875362152 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:18:13.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.571+0000 7f0397fff640 1 --2- 172.21.15.26:0/1875362152 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0398102790 0x7f0398198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:13.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.571+0000 7f039d185640 1 --2- 172.21.15.26:0/1875362152 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0398106f10 0x7f039819ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:13.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.572+0000 7f039c984640 1 -- 172.21.15.26:0/1875362152 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0398106f10 msgr2=0x7f039819ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:13.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.572+0000 7f039c984640 1 --2- 172.21.15.26:0/1875362152 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0398106f10 0x7f039819ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:13.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.572+0000 7f039c984640 1 -- 172.21.15.26:0/1875362152 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0398102790 msgr2=0x7f0398198af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:13.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.572+0000 7f039c984640 1 --2- 172.21.15.26:0/1875362152 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0398102790 0x7f0398198af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:13.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.572+0000 7f039c984640 1 -- 172.21.15.26:0/1875362152 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f039819d560 con 0x7f0398101de0 2024-09-11T23:18:13.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.572+0000 7f0397fff640 1 --2- 172.21.15.26:0/1875362152 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0398102790 0x7f0398198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:18:13.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.572+0000 7f039c984640 1 --2- 172.21.15.26:0/1875362152 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0398101de0 0x7f03981985b0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f038800b4a0 tx=0x7f038800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:18:13.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.572+0000 7f0395ffb640 1 -- 172.21.15.26:0/1875362152 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0388014070 con 0x7f0398101de0 2024-09-11T23:18:13.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.572+0000 7f0395ffb640 1 -- 172.21.15.26:0/1875362152 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0388010af0 con 0x7f0398101de0 2024-09-11T23:18:13.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.572+0000 7f0395ffb640 1 -- 172.21.15.26:0/1875362152 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0388017d10 con 0x7f0398101de0 2024-09-11T23:18:13.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.572+0000 7f039ebe8640 1 -- 172.21.15.26:0/1875362152 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f039819d850 con 0x7f0398101de0 2024-09-11T23:18:13.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.572+0000 7f039ebe8640 1 -- 172.21.15.26:0/1875362152 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03981a5130 con 0x7f0398101de0 2024-09-11T23:18:13.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.575+0000 7f0395ffb640 1 -- 172.21.15.26:0/1875362152 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f0388017070 con 0x7f0398101de0 2024-09-11T23:18:13.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.575+0000 7f03a0fc1740 1 Processor -- start 2024-09-11T23:18:13.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.575+0000 7f03a0fc1740 1 -- start start 2024-09-11T23:18:13.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.575+0000 7f03a0fc1740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5654ed2bc9d0 0x5654ed2bcdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:13.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.575+0000 7f03a0fc1740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5654ed2bd2f0 0x5654ed2c7670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:13.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.575+0000 7f03a0fc1740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5654ed2c7bb0 0x5654ed2c9fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:13.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.575+0000 7f03a0fc1740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5654ed223090 con 0x5654ed2bc9d0 2024-09-11T23:18:13.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.575+0000 7f03a0fc1740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5654ed1caa70 con 0x5654ed2bd2f0 2024-09-11T23:18:13.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.575+0000 7f03a0fc1740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5654ed21f050 con 0x5654ed2c7bb0 2024-09-11T23:18:13.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.575+0000 7f0395ffb640 1 --2- 172.21.15.26:0/1875362152 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f03840a2c00 0x7f03840a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:13.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.575+0000 7f039d185640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5654ed2bc9d0 0x5654ed2bcdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:13.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.575+0000 7f039d185640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5654ed2bc9d0 0x5654ed2bcdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44666/0 (socket says 172.21.15.26:44666) 2024-09-11T23:18:13.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.575+0000 7f039d185640 1 -- 172.21.15.26:0/2493733378 learned_addr learned my addr 172.21.15.26:0/2493733378 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:18:13.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.576+0000 7f0397fff640 1 --2- 172.21.15.26:0/2493733378 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5654ed2c7bb0 0x5654ed2c9fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:13.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.576+0000 7f0395ffb640 1 -- 172.21.15.26:0/1875362152 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f03880ce450 con 0x7f0398101de0 2024-09-11T23:18:13.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.576+0000 7f039d185640 1 -- 172.21.15.26:0/2493733378 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5654ed2c7bb0 msgr2=0x5654ed2c9fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:18:13.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.576+0000 7f039d185640 1 --2- 172.21.15.26:0/2493733378 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5654ed2c7bb0 0x5654ed2c9fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:13.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.576+0000 7f039d185640 1 -- 172.21.15.26:0/2493733378 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5654ed2bd2f0 msgr2=0x5654ed2c7670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:18:13.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.576+0000 7f039c984640 1 --2- 172.21.15.26:0/2493733378 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5654ed2bd2f0 0x5654ed2c7670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:13.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.576+0000 7f039d185640 1 --2- 172.21.15.26:0/2493733378 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5654ed2bd2f0 0x5654ed2c7670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:13.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.576+0000 7f039d185640 1 -- 172.21.15.26:0/2493733378 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654ed1d7130 con 0x5654ed2bc9d0 2024-09-11T23:18:13.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.576+0000 7f039d185640 1 --2- 172.21.15.26:0/1875362152 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f03840a2c00 0x7f03840a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:13.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.576+0000 7f039d185640 1 --2- 172.21.15.26:0/2493733378 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5654ed2bc9d0 0x5654ed2bcdb0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f039000ba50 tx=0x7f039000bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cc8c7c6d045562c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:18:13.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.576+0000 7f038effd640 1 -- 172.21.15.26:0/2493733378 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0390016d60 con 0x5654ed2bc9d0 2024-09-11T23:18:13.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.576+0000 7f038effd640 1 -- 172.21.15.26:0/2493733378 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0390010040 con 0x5654ed2bc9d0 2024-09-11T23:18:13.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.576+0000 7f038effd640 1 -- 172.21.15.26:0/2493733378 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0390015610 con 0x5654ed2bc9d0 2024-09-11T23:18:13.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.576+0000 7f039d185640 1 --2- 172.21.15.26:0/1875362152 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f03840a2c00 0x7f03840a50c0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f039001e000 tx=0x7f039001f000 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:18:13.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.577+0000 7f03a0fc1740 1 -- 172.21.15.26:0/2493733378 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5654ed2bc9d0 msgr2=0x5654ed2bcdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:13.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.577+0000 7f03a0fc1740 1 --2- 172.21.15.26:0/2493733378 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5654ed2bc9d0 0x5654ed2bcdb0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f039000ba50 tx=0x7f039000bf10 comp rx=0 tx=0).stop 2024-09-11T23:18:13.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.577+0000 7f03a0fc1740 1 -- 172.21.15.26:0/2493733378 shutdown_connections 2024-09-11T23:18:13.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.577+0000 7f03a0fc1740 1 --2- 172.21.15.26:0/2493733378 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5654ed2bd2f0 0x5654ed2c7670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:13.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.577+0000 7f03a0fc1740 1 --2- 172.21.15.26:0/2493733378 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5654ed2c7bb0 0x5654ed2c9fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:13.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.577+0000 7f03a0fc1740 1 --2- 172.21.15.26:0/2493733378 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5654ed2bc9d0 0x5654ed2bcdb0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:13.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.577+0000 7f03a0fc1740 1 -- 172.21.15.26:0/2493733378 >> 172.21.15.26:0/2493733378 conn(0x5654ed2b8f60 msgr2=0x5654ed2b9340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:18:13.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.577+0000 7f03a0fc1740 1 -- 172.21.15.26:0/2493733378 shutdown_connections 2024-09-11T23:18:13.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.577+0000 7f03a0fc1740 1 -- 172.21.15.26:0/2493733378 wait complete. 2024-09-11T23:18:13.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.578+0000 7f03a0fc1740 1 -- start start 2024-09-11T23:18:13.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.578+0000 7f03a0fc1740 1 Processor -- start 2024-09-11T23:18:13.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.578+0000 7f03a0fc1740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5654ed2bc9d0 0x5654ed2c73b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:13.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.578+0000 7f03a0fc1740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5654ed2bd2f0 0x5654ed2c4750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:13.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.578+0000 7f03a0fc1740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5654ed2c7bb0 0x5654ed2c4c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:13.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.578+0000 7f03a0fc1740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5654ed2ca780 con 0x5654ed2c7bb0 2024-09-11T23:18:13.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.578+0000 7f03a0fc1740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5654ed1ddf20 con 0x5654ed2bd2f0 2024-09-11T23:18:13.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.578+0000 7f03a0fc1740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5654ed2ca900 con 0x5654ed2bc9d0 2024-09-11T23:18:13.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.578+0000 7f03a0fc1740 10 client.0 _ll_register_callbacks cb 0x5654ed1a7030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:18:13.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.578+0000 7f039d185640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5654ed2c7bb0 0x5654ed2c4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:13.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.578+0000 7f039d185640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5654ed2c7bb0 0x5654ed2c4c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44668/0 (socket says 172.21.15.26:44668) 2024-09-11T23:18:13.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.578+0000 7f039d185640 1 -- 172.21.15.26:0/564705159 learned_addr learned my addr 172.21.15.26:0/564705159 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:18:13.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.578+0000 7f039c984640 1 --2- 172.21.15.26:0/564705159 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5654ed2bc9d0 0x5654ed2c73b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:13.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.578+0000 7f0397fff640 1 --2- 172.21.15.26:0/564705159 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5654ed2bd2f0 0x5654ed2c4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:13.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.579+0000 7f039d185640 1 -- 172.21.15.26:0/564705159 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5654ed2bc9d0 msgr2=0x5654ed2c73b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:18:13.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.579+0000 7f039d185640 1 --2- 172.21.15.26:0/564705159 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5654ed2bc9d0 0x5654ed2c73b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:13.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.579+0000 7f039d185640 1 -- 172.21.15.26:0/564705159 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5654ed2bd2f0 msgr2=0x5654ed2c4750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:18:13.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.579+0000 7f039d185640 1 --2- 172.21.15.26:0/564705159 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5654ed2bd2f0 0x5654ed2c4750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:13.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.579+0000 7f039d185640 1 -- 172.21.15.26:0/564705159 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654ecf942b0 con 0x5654ed2c7bb0 2024-09-11T23:18:13.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.579+0000 7f039c984640 1 --2- 172.21.15.26:0/564705159 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5654ed2bc9d0 0x5654ed2c73b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:18:13.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.579+0000 7f039d185640 1 --2- 172.21.15.26:0/564705159 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5654ed2c7bb0 0x5654ed2c4c90 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f0390004810 tx=0x7f039000f8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d67ea9d4d191be7 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:18:13.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.579+0000 7f0377fff640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:18:13.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.579+0000 7f0377fff640 1 -- 172.21.15.26:0/564705159 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0390030020 con 0x5654ed2c7bb0 2024-09-11T23:18:13.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.579+0000 7f0377fff640 1 -- 172.21.15.26:0/564705159 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f0390021440 con 0x5654ed2c7bb0 2024-09-11T23:18:13.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.579+0000 7f03a0fc1740 1 -- 172.21.15.26:0/564705159 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0390010040 con 0x5654ed2c7bb0 2024-09-11T23:18:13.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.579+0000 7f0377fff640 1 -- 172.21.15.26:0/564705159 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0390021e40 con 0x5654ed2c7bb0 2024-09-11T23:18:13.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.579+0000 7f03a0fc1740 10 client.15141 fetch_fsmap learned FSMap version 28 2024-09-11T23:18:13.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.579+0000 7f03a0fc1740 1 -- 172.21.15.26:0/564705159 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5654ed2d6040 con 0x5654ed2c7bb0 2024-09-11T23:18:13.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.580+0000 7f0377fff640 1 -- 172.21.15.26:0/564705159 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0390010040 con 0x5654ed2c7bb0 2024-09-11T23:18:13.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.580+0000 7f0377fff640 1 -- 172.21.15.26:0/564705159 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f039002c420 con 0x5654ed2c7bb0 2024-09-11T23:18:13.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.580+0000 7f03a0fc1740 10 client.15141 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:18:13.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.580+0000 7f03a0fc1740 10 client.15141 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-11T23:18:13.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.580+0000 7f03a0fc1740 20 client.15141 populate_metadata read hostname 'smithi026' 2024-09-11T23:18:13.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.580+0000 7f03a0fc1740 1 --2- 172.21.15.26:0/564705159 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5654ed2d8400 0x5654ed2d9870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:13.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.580+0000 7f03a0fc1740 4 client.15141 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:18:13.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.580+0000 7f03a0fc1740 1 -- 172.21.15.26:0/564705159 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5654ed230600 con 0x5654ed2d8400 2024-09-11T23:18:13.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.580+0000 7f039c984640 1 --2- 172.21.15.26:0/564705159 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5654ed2d8400 0x5654ed2d9870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:13.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.581+0000 7f039c984640 1 --2- 172.21.15.26:0/564705159 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5654ed2d8400 0x5654ed2d9870 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4586965a77d91393 server_cookie=4c156be0f466a6ae in_seq=0 out_seq=0 2024-09-11T23:18:13.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.581+0000 7f0377fff640 10 client.15141 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:18:13.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.581+0000 7f0377fff640 1 -- 172.21.15.26:0/564705159 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5654ed230600 con 0x5654ed2d8400 2024-09-11T23:18:13.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.581+0000 7f0377fff640 10 client.15141 handle_command_reply: tid=0 2024-09-11T23:18:13.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.581+0000 7f03a0fc1740 1 client.15141 shutdown 2024-09-11T23:18:13.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.582+0000 7f03a0fc1740 1 -- 172.21.15.26:0/564705159 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5654ed2c7bb0 msgr2=0x5654ed2c4c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:13.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.582+0000 7f03a0fc1740 1 --2- 172.21.15.26:0/564705159 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5654ed2c7bb0 0x5654ed2c4c90 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f0390004810 tx=0x7f039000f8c0 comp rx=0 tx=0).stop 2024-09-11T23:18:13.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.582+0000 7f03a0fc1740 1 -- 172.21.15.26:0/564705159 shutdown_connections 2024-09-11T23:18:13.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.582+0000 7f03a0fc1740 1 --2- 172.21.15.26:0/564705159 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5654ed2bd2f0 0x5654ed2c4750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:13.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.582+0000 7f03a0fc1740 1 --2- 172.21.15.26:0/564705159 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5654ed2d8400 0x5654ed2d9870 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:13.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.582+0000 7f0377fff640 0 client.15141 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:18:13.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.582+0000 7f03a0fc1740 1 --2- 172.21.15.26:0/564705159 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5654ed2c7bb0 0x5654ed2c4c90 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:13.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.582+0000 7f03a0fc1740 1 --2- 172.21.15.26:0/564705159 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5654ed2bc9d0 0x5654ed2c73b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:13.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.582+0000 7f03a0fc1740 1 -- 172.21.15.26:0/564705159 >> 172.21.15.26:0/564705159 conn(0x5654ed2b8f60 msgr2=0x5654ed2bec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:18:13.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.582+0000 7f03a0fc1740 1 -- 172.21.15.26:0/564705159 shutdown_connections 2024-09-11T23:18:13.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.582+0000 7f03a0fc1740 1 -- 172.21.15.26:0/564705159 wait complete. 2024-09-11T23:18:13.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.582+0000 7f03a0fc1740 20 client.15141 trim_cache size 0 max 16384 2024-09-11T23:18:13.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.600+0000 7f03a0fc1740 1 Processor -- start 2024-09-11T23:18:13.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.600+0000 7f03a0fc1740 1 -- start start 2024-09-11T23:18:13.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.600+0000 7f03a0fc1740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5654ed2bc9d0 0x5654ed2d9870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:13.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.600+0000 7f03a0fc1740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5654ed2bd2f0 0x5654ed2e1b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:13.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.600+0000 7f03a0fc1740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5654ed2c7bb0 0x5654ed2e2090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:13.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.600+0000 7f03a0fc1740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5654ed2d7bb0 con 0x5654ed2bd2f0 2024-09-11T23:18:13.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.600+0000 7f03a0fc1740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5654ed2d8280 con 0x5654ed2bc9d0 2024-09-11T23:18:13.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.600+0000 7f03a0fc1740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5654ed2d8950 con 0x5654ed2c7bb0 2024-09-11T23:18:13.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.600+0000 7f0397fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5654ed2bd2f0 0x5654ed2e1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:13.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.600+0000 7f0397fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5654ed2bd2f0 0x5654ed2e1b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44670/0 (socket says 172.21.15.26:44670) 2024-09-11T23:18:13.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.600+0000 7f0397fff640 1 -- 172.21.15.26:0/1789639560 learned_addr learned my addr 172.21.15.26:0/1789639560 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:18:13.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.600+0000 7f039c984640 1 --2- 172.21.15.26:0/1789639560 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5654ed2bc9d0 0x5654ed2d9870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:13.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.600+0000 7f039d185640 1 --2- 172.21.15.26:0/1789639560 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5654ed2c7bb0 0x5654ed2e2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:13.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.600+0000 7f0397fff640 1 -- 172.21.15.26:0/1789639560 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5654ed2c7bb0 msgr2=0x5654ed2e2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:18:13.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.600+0000 7f0397fff640 1 --2- 172.21.15.26:0/1789639560 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5654ed2c7bb0 0x5654ed2e2090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:13.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.600+0000 7f0397fff640 1 -- 172.21.15.26:0/1789639560 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5654ed2bc9d0 msgr2=0x5654ed2d9870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:18:13.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.600+0000 7f0397fff640 1 --2- 172.21.15.26:0/1789639560 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5654ed2bc9d0 0x5654ed2d9870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:13.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.600+0000 7f0397fff640 1 -- 172.21.15.26:0/1789639560 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654ed2b8a00 con 0x5654ed2bd2f0 2024-09-11T23:18:13.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.601+0000 7f0397fff640 1 --2- 172.21.15.26:0/1789639560 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5654ed2bd2f0 0x5654ed2e1b50 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f0380009ca0 tx=0x7f0380009360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20d35be6251fe774 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:18:13.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.601+0000 7f038e7fc640 1 -- 172.21.15.26:0/1789639560 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0380003b70 con 0x5654ed2bd2f0 2024-09-11T23:18:13.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.601+0000 7f038e7fc640 1 -- 172.21.15.26:0/1789639560 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0380036040 con 0x5654ed2bd2f0 2024-09-11T23:18:13.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.601+0000 7f038e7fc640 1 -- 172.21.15.26:0/1789639560 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0380004750 con 0x5654ed2bd2f0 2024-09-11T23:18:13.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.601+0000 7f03a0fc1740 1 -- 172.21.15.26:0/1789639560 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5654ed2bd2f0 msgr2=0x5654ed2e1b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:13.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.601+0000 7f03a0fc1740 1 --2- 172.21.15.26:0/1789639560 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5654ed2bd2f0 0x5654ed2e1b50 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f0380009ca0 tx=0x7f0380009360 comp rx=0 tx=0).stop 2024-09-11T23:18:13.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.601+0000 7f03a0fc1740 1 -- 172.21.15.26:0/1789639560 shutdown_connections 2024-09-11T23:18:13.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.601+0000 7f03a0fc1740 1 --2- 172.21.15.26:0/1789639560 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5654ed2c7bb0 0x5654ed2e2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:13.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.601+0000 7f03a0fc1740 1 --2- 172.21.15.26:0/1789639560 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5654ed2bd2f0 0x5654ed2e1b50 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:13.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.601+0000 7f03a0fc1740 1 --2- 172.21.15.26:0/1789639560 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5654ed2bc9d0 0x5654ed2d9870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:13.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.601+0000 7f03a0fc1740 1 -- 172.21.15.26:0/1789639560 >> 172.21.15.26:0/1789639560 conn(0x5654ed2b8f60 msgr2=0x5654ed2c3020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:18:13.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.601+0000 7f03a0fc1740 1 -- 172.21.15.26:0/1789639560 shutdown_connections 2024-09-11T23:18:13.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.601+0000 7f03a0fc1740 1 -- 172.21.15.26:0/1789639560 wait complete. 2024-09-11T23:18:13.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.602+0000 7f03a0fc1740 1 -- start start 2024-09-11T23:18:13.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.602+0000 7f03a0fc1740 1 Processor -- start 2024-09-11T23:18:13.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.602+0000 7f03a0fc1740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5654ed2bc9d0 0x5654ed2dcbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:13.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.602+0000 7f03a0fc1740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5654ed2bd2f0 0x5654ed2dd110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:13.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.602+0000 7f03a0fc1740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5654ed2c7bb0 0x5654ed340b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:13.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.602+0000 7f03a0fc1740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5654ed2d96f0 con 0x5654ed2c7bb0 2024-09-11T23:18:13.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.602+0000 7f03a0fc1740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5654ed2d9020 con 0x5654ed2bc9d0 2024-09-11T23:18:13.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.602+0000 7f03a0fc1740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5654ed2d9dc0 con 0x5654ed2bd2f0 2024-09-11T23:18:13.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.602+0000 7f03a0fc1740 10 client.0 _ll_register_callbacks cb 0x7f0380009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:18:13.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.602+0000 7f039d185640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5654ed2c7bb0 0x5654ed340b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:13.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.602+0000 7f039d185640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5654ed2c7bb0 0x5654ed340b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44674/0 (socket says 172.21.15.26:44674) 2024-09-11T23:18:13.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.602+0000 7f039d185640 1 -- 172.21.15.26:0/2222901035 learned_addr learned my addr 172.21.15.26:0/2222901035 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:18:13.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.602+0000 7f0397fff640 1 --2- 172.21.15.26:0/2222901035 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5654ed2bd2f0 0x5654ed2dd110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:13.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.603+0000 7f039c984640 1 --2- 172.21.15.26:0/2222901035 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5654ed2bc9d0 0x5654ed2dcbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:13.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.603+0000 7f039d185640 1 -- 172.21.15.26:0/2222901035 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5654ed2bd2f0 msgr2=0x5654ed2dd110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:18:13.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.603+0000 7f039d185640 1 --2- 172.21.15.26:0/2222901035 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5654ed2bd2f0 0x5654ed2dd110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:13.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.603+0000 7f039d185640 1 -- 172.21.15.26:0/2222901035 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5654ed2bc9d0 msgr2=0x5654ed2dcbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:18:13.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.603+0000 7f039d185640 1 --2- 172.21.15.26:0/2222901035 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5654ed2bc9d0 0x5654ed2dcbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:13.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.603+0000 7f039d185640 1 -- 172.21.15.26:0/2222901035 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654ed2dd9e0 con 0x5654ed2c7bb0 2024-09-11T23:18:13.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.603+0000 7f039d185640 1 --2- 172.21.15.26:0/2222901035 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5654ed2c7bb0 0x5654ed340b00 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f0390000ea0 tx=0x7f039000b310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6a26e9aa990b675 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:18:13.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.603+0000 7f0377fff640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:18:13.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.603+0000 7f0377fff640 1 -- 172.21.15.26:0/2222901035 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0390030040 con 0x5654ed2c7bb0 2024-09-11T23:18:13.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.603+0000 7f0377fff640 1 -- 172.21.15.26:0/2222901035 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f039002c420 con 0x5654ed2c7bb0 2024-09-11T23:18:13.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.603+0000 7f03a0fc1740 1 -- 172.21.15.26:0/2222901035 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0380036040 con 0x5654ed2c7bb0 2024-09-11T23:18:13.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.603+0000 7f0377fff640 1 -- 172.21.15.26:0/2222901035 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f039002ccc0 con 0x5654ed2c7bb0 2024-09-11T23:18:13.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.603+0000 7f03a0fc1740 10 client.15147 fetch_fsmap learned FSMap version 28 2024-09-11T23:18:13.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.603+0000 7f03a0fc1740 1 -- 172.21.15.26:0/2222901035 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5654ed342140 con 0x5654ed2c7bb0 2024-09-11T23:18:13.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.603+0000 7f0377fff640 1 -- 172.21.15.26:0/2222901035 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0380036040 con 0x5654ed2c7bb0 2024-09-11T23:18:13.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.603+0000 7f0377fff640 1 -- 172.21.15.26:0/2222901035 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f0390015b90 con 0x5654ed2c7bb0 2024-09-11T23:18:13.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.604+0000 7f03a0fc1740 10 client.15147 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:18:13.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.604+0000 7f03a0fc1740 10 client.15147 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-11T23:18:13.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.604+0000 7f03a0fc1740 20 client.15147 populate_metadata read hostname 'smithi026' 2024-09-11T23:18:13.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.604+0000 7f03a0fc1740 1 --2- 172.21.15.26:0/2222901035 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5654ed2d8400 0x5654ed3459b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:13.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.604+0000 7f03a0fc1740 4 client.15147 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-11T23:18:13.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.604+0000 7f03a0fc1740 1 -- 172.21.15.26:0/2222901035 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "status"}) -- 0x5654ed22ab10 con 0x5654ed2d8400 2024-09-11T23:18:13.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.604+0000 7f039c984640 1 --2- 172.21.15.26:0/2222901035 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5654ed2d8400 0x5654ed3459b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:13.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.604+0000 7f039c984640 1 --2- 172.21.15.26:0/2222901035 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5654ed2d8400 0x5654ed3459b0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f9c3c57c60a99dd2 server_cookie=9b6735aebc507c89 in_seq=0 out_seq=0 2024-09-11T23:18:13.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.604+0000 7f0377fff640 10 client.15147 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:18:13.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.604+0000 7f0377fff640 1 -- 172.21.15.26:0/2222901035 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5654ed22ab10 con 0x5654ed2d8400 2024-09-11T23:18:13.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.604+0000 7f0377fff640 10 client.15147 handle_command_reply: tid=0 2024-09-11T23:18:13.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.604+0000 7f03a0fc1740 1 client.15147 shutdown 2024-09-11T23:18:13.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.604+0000 7f03a0fc1740 1 -- 172.21.15.26:0/2222901035 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5654ed2c7bb0 msgr2=0x5654ed340b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:13.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.604+0000 7f03a0fc1740 1 --2- 172.21.15.26:0/2222901035 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5654ed2c7bb0 0x5654ed340b00 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f0390000ea0 tx=0x7f039000b310 comp rx=0 tx=0).stop 2024-09-11T23:18:13.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.605+0000 7f03a0fc1740 1 -- 172.21.15.26:0/2222901035 shutdown_connections 2024-09-11T23:18:13.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.605+0000 7f03a0fc1740 1 --2- 172.21.15.26:0/2222901035 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5654ed2d8400 0x5654ed3459b0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:13.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.605+0000 7f0377fff640 0 client.15147 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:18:13.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.605+0000 7f03a0fc1740 1 --2- 172.21.15.26:0/2222901035 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5654ed2c7bb0 0x5654ed340b00 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:13.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.605+0000 7f03a0fc1740 1 --2- 172.21.15.26:0/2222901035 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5654ed2bd2f0 0x5654ed2dd110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:13.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.605+0000 7f03a0fc1740 1 --2- 172.21.15.26:0/2222901035 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5654ed2bc9d0 0x5654ed2dcbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:13.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.605+0000 7f03a0fc1740 1 -- 172.21.15.26:0/2222901035 >> 172.21.15.26:0/2222901035 conn(0x5654ed2b8f60 msgr2=0x5654ed2b7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:18:13.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.605+0000 7f03a0fc1740 1 -- 172.21.15.26:0/2222901035 shutdown_connections 2024-09-11T23:18:13.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.605+0000 7f03a0fc1740 1 -- 172.21.15.26:0/2222901035 wait complete. 2024-09-11T23:18:13.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.605+0000 7f03a0fc1740 20 client.15147 trim_cache size 0 max 16384 2024-09-11T23:18:13.612 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-11T23:18:13.612 INFO:teuthology.orchestra.run.smithi026.stdout: "cluster_fsid": "af7e6fc8-7092-11ef-bcea-c7b262605968", 2024-09-11T23:18:13.612 INFO:teuthology.orchestra.run.smithi026.stdout: "whoami": 0, 2024-09-11T23:18:13.612 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 24485, 2024-09-11T23:18:13.612 INFO:teuthology.orchestra.run.smithi026.stdout: "want_state": "up:active", 2024-09-11T23:18:13.613 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "up:active", 2024-09-11T23:18:13.613 INFO:teuthology.orchestra.run.smithi026.stdout: "fs_name": "cephfs", 2024-09-11T23:18:13.613 INFO:teuthology.orchestra.run.smithi026.stdout: "rank_uptime": 106.700591961, 2024-09-11T23:18:13.613 INFO:teuthology.orchestra.run.smithi026.stdout: "mdsmap_epoch": 28, 2024-09-11T23:18:13.613 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch": 85, 2024-09-11T23:18:13.613 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch_barrier": 85, 2024-09-11T23:18:13.613 INFO:teuthology.orchestra.run.smithi026.stdout: "uptime": 117.51699718899999 2024-09-11T23:18:13.613 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-11T23:18:13.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.605+0000 7f039ebe8640 1 -- 172.21.15.26:0/1875362152 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f03840a2c00 msgr2=0x7f03840a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:13.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.605+0000 7f039ebe8640 1 --2- 172.21.15.26:0/1875362152 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f03840a2c00 0x7f03840a50c0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f039001e000 tx=0x7f039001f000 comp rx=0 tx=0).stop 2024-09-11T23:18:13.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.605+0000 7f039ebe8640 1 -- 172.21.15.26:0/1875362152 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0398101de0 msgr2=0x7f03981985b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:13.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.605+0000 7f039ebe8640 1 --2- 172.21.15.26:0/1875362152 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0398101de0 0x7f03981985b0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f038800b4a0 tx=0x7f038800b960 comp rx=0 tx=0).stop 2024-09-11T23:18:13.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.605+0000 7f039ebe8640 1 -- 172.21.15.26:0/1875362152 shutdown_connections 2024-09-11T23:18:13.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.605+0000 7f039ebe8640 1 --2- 172.21.15.26:0/1875362152 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f03840a2c00 0x7f03840a50c0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:13.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.606+0000 7f039ebe8640 1 --2- 172.21.15.26:0/1875362152 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0398102790 0x7f0398198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:13.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.606+0000 7f039ebe8640 1 --2- 172.21.15.26:0/1875362152 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0398106f10 0x7f039819ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:13.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.606+0000 7f039ebe8640 1 --2- 172.21.15.26:0/1875362152 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0398101de0 0x7f03981985b0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:13.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.606+0000 7f039ebe8640 1 -- 172.21.15.26:0/1875362152 >> 172.21.15.26:0/1875362152 conn(0x7f03980fd8c0 msgr2=0x7f03980fed30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:18:13.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.606+0000 7f039ebe8640 1 -- 172.21.15.26:0/1875362152 shutdown_connections 2024-09-11T23:18:13.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:13.606+0000 7f039ebe8640 1 -- 172.21.15.26:0/1875362152 wait complete. 2024-09-11T23:18:14.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:13 smithi136 ceph-mon[35121]: pgmap v334: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 11 op/s 2024-09-11T23:18:14.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:13 smithi026 ceph-mon[31172]: pgmap v334: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 11 op/s 2024-09-11T23:18:14.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:13 smithi076 ceph-mon[35296]: pgmap v334: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 11 op/s 2024-09-11T23:18:14.535 INFO:teuthology.run_tasks:Running task sequential... 2024-09-11T23:18:15.747 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-09-11T23:18:15.747 INFO:teuthology.task.sequential:In sequential, running task print... 2024-09-11T23:18:15.748 INFO:teuthology.task.print:Enabling mgr modules 2024-09-11T23:18:15.748 INFO:teuthology.task.sequential:In sequential, running task exec... 2024-09-11T23:18:15.749 INFO:teuthology.task.exec:Executing custom commands... 2024-09-11T23:18:15.749 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi026.front.sepia.ceph.com 2024-09-11T23:18:15.749 DEBUG:teuthology.orchestra.run.smithi026:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-09-11T23:18:15.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:15.862+0000 7f8ee3a53640 1 -- 172.21.15.26:0/3846258526 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8edc104f10 msgr2=0x7f8edc107300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:15.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:15.862+0000 7f8ee3a53640 1 --2- 172.21.15.26:0/3846258526 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8edc104f10 0x7f8edc107300 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f8ecc009900 tx=0x7f8ecc02a410 comp rx=0 tx=0).stop 2024-09-11T23:18:15.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:15.862+0000 7f8ee3a53640 1 -- 172.21.15.26:0/3846258526 shutdown_connections 2024-09-11T23:18:15.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:15.862+0000 7f8ee3a53640 1 --2- 172.21.15.26:0/3846258526 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8edc107840 0x7f8edc109cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:15.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:15.862+0000 7f8ee3a53640 1 --2- 172.21.15.26:0/3846258526 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8edc104f10 0x7f8edc107300 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:15.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:15.862+0000 7f8ee3a53640 1 --2- 172.21.15.26:0/3846258526 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8edc1025e0 0x7f8edc1049d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:15.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:15.862+0000 7f8ee3a53640 1 -- 172.21.15.26:0/3846258526 >> 172.21.15.26:0/3846258526 conn(0x7f8edc0fc1e0 msgr2=0x7f8edc0fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:18:15.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:15.862+0000 7f8ee3a53640 1 -- 172.21.15.26:0/3846258526 shutdown_connections 2024-09-11T23:18:15.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:15.863+0000 7f8ee3a53640 1 -- 172.21.15.26:0/3846258526 wait complete. 2024-09-11T23:18:15.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:15.863+0000 7f8ee3a53640 1 Processor -- start 2024-09-11T23:18:15.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:15.863+0000 7f8ee3a53640 1 -- start start 2024-09-11T23:18:15.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:15.863+0000 7f8ee3a53640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8edc1025e0 0x7f8edc197e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:15.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:15.863+0000 7f8ee3a53640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8edc104f10 0x7f8edc1983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:15.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:15.863+0000 7f8ee3a53640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8edc107840 0x7f8edc19f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:15.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:15.863+0000 7f8ee3a53640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f8edc10d7f0 con 0x7f8edc107840 2024-09-11T23:18:15.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:15.863+0000 7f8ee3a53640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f8edc10d670 con 0x7f8edc104f10 2024-09-11T23:18:15.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:15.863+0000 7f8ee3a53640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f8edc10d970 con 0x7f8edc1025e0 2024-09-11T23:18:15.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:15.863+0000 7f8ee1ff0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8edc107840 0x7f8edc19f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:15.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:15.864+0000 7f8ee1ff0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8edc107840 0x7f8edc19f420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44678/0 (socket says 172.21.15.26:44678) 2024-09-11T23:18:15.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:15.864+0000 7f8ee17ef640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8edc1025e0 0x7f8edc197e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:15.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:15.864+0000 7f8ee1ff0640 1 -- 172.21.15.26:0/3734003794 learned_addr learned my addr 172.21.15.26:0/3734003794 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:18:15.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:15.864+0000 7f8ee0fee640 1 --2- 172.21.15.26:0/3734003794 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8edc104f10 0x7f8edc1983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:15.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:15.864+0000 7f8ee17ef640 1 -- 172.21.15.26:0/3734003794 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8edc104f10 msgr2=0x7f8edc1983b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:15.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:15.864+0000 7f8ee17ef640 1 --2- 172.21.15.26:0/3734003794 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8edc104f10 0x7f8edc1983b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:15.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:15.864+0000 7f8ee17ef640 1 -- 172.21.15.26:0/3734003794 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8edc107840 msgr2=0x7f8edc19f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:15.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:15.864+0000 7f8ee17ef640 1 --2- 172.21.15.26:0/3734003794 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8edc107840 0x7f8edc19f420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:15.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:15.864+0000 7f8ee17ef640 1 -- 172.21.15.26:0/3734003794 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8edc19fa90 con 0x7f8edc1025e0 2024-09-11T23:18:15.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:15.864+0000 7f8ee0fee640 1 --2- 172.21.15.26:0/3734003794 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8edc104f10 0x7f8edc1983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:18:15.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:15.864+0000 7f8ee17ef640 1 --2- 172.21.15.26:0/3734003794 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8edc1025e0 0x7f8edc197e70 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f8ed8005fd0 tx=0x7f8ed800dde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:18:15.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:15.867+0000 7f8ed27fc640 1 -- 172.21.15.26:0/3734003794 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ed8009c30 con 0x7f8edc1025e0 2024-09-11T23:18:15.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:15.867+0000 7f8ed27fc640 1 -- 172.21.15.26:0/3734003794 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ed8009dd0 con 0x7f8edc1025e0 2024-09-11T23:18:15.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:15.867+0000 7f8ee3a53640 1 -- 172.21.15.26:0/3734003794 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8edc19fd80 con 0x7f8edc1025e0 2024-09-11T23:18:15.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:15.867+0000 7f8ed27fc640 1 -- 172.21.15.26:0/3734003794 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ed8010460 con 0x7f8edc1025e0 2024-09-11T23:18:15.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:15.867+0000 7f8ee3a53640 1 -- 172.21.15.26:0/3734003794 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8edc1a0240 con 0x7f8edc1025e0 2024-09-11T23:18:15.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:15.868+0000 7f8ee3a53640 1 -- 172.21.15.26:0/3734003794 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8edc0fdcd0 con 0x7f8edc1025e0 2024-09-11T23:18:15.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:15.869+0000 7f8ed27fc640 1 -- 172.21.15.26:0/3734003794 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f8ed8003f50 con 0x7f8edc1025e0 2024-09-11T23:18:15.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:15.870+0000 7f8ed27fc640 1 --2- 172.21.15.26:0/3734003794 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f8ec80a2cb0 0x7f8ec80a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:15.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:15.870+0000 7f8ee0fee640 1 --2- 172.21.15.26:0/3734003794 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f8ec80a2cb0 0x7f8ec80a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:15.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:15.870+0000 7f8ed27fc640 1 -- 172.21.15.26:0/3734003794 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f8ed80cebb0 con 0x7f8edc1025e0 2024-09-11T23:18:15.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:15.871+0000 7f8ee0fee640 1 --2- 172.21.15.26:0/3734003794 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f8ec80a2cb0 0x7f8ec80a5170 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f8ecc005ab0 tx=0x7f8ecc000bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:18:15.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:15.874+0000 7f8ed27fc640 1 -- 172.21.15.26:0/3734003794 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8ed8017020 con 0x7f8edc1025e0 2024-09-11T23:18:16.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:16.098+0000 7f8ee3a53640 1 -- 172.21.15.26:0/3734003794 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) -- 0x7f8edc02da30 con 0x7f8edc1025e0 2024-09-11T23:18:16.115 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:15 smithi026 ceph-mon[31172]: pgmap v335: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 9 op/s 2024-09-11T23:18:16.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:15 smithi136 ceph-mon[35121]: pgmap v335: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 9 op/s 2024-09-11T23:18:16.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:15 smithi076 ceph-mon[35296]: pgmap v335: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 9 op/s 2024-09-11T23:18:16.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:16.884+0000 7f8ed27fc640 1 -- 172.21.15.26:0/3734003794 <== mon.1 v2:172.21.15.136:3300/0 7 ==== mgrmap(e 19) ==== 137850+0+0 (secure 0 0 0) 0x7f8ed8097bb0 con 0x7f8edc1025e0 2024-09-11T23:18:16.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:16.884+0000 7f8ed27fc640 1 -- 172.21.15.26:0/3734003794 <== mon.1 v2:172.21.15.136:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v19) ==== 92+0+0 (secure 0 0 0) 0x7f8ed80a4350 con 0x7f8edc1025e0 2024-09-11T23:18:16.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:16.888+0000 7f8ee3a53640 1 -- 172.21.15.26:0/3734003794 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f8ec80a2cb0 msgr2=0x7f8ec80a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:16.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:16.888+0000 7f8ee3a53640 1 --2- 172.21.15.26:0/3734003794 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f8ec80a2cb0 0x7f8ec80a5170 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f8ecc005ab0 tx=0x7f8ecc000bc0 comp rx=0 tx=0).stop 2024-09-11T23:18:16.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:16.888+0000 7f8ee3a53640 1 -- 172.21.15.26:0/3734003794 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8edc1025e0 msgr2=0x7f8edc197e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:16.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:16.888+0000 7f8ee3a53640 1 --2- 172.21.15.26:0/3734003794 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8edc1025e0 0x7f8edc197e70 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f8ed8005fd0 tx=0x7f8ed800dde0 comp rx=0 tx=0).stop 2024-09-11T23:18:16.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:16.888+0000 7f8ee3a53640 1 -- 172.21.15.26:0/3734003794 shutdown_connections 2024-09-11T23:18:16.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:16.888+0000 7f8ee3a53640 1 --2- 172.21.15.26:0/3734003794 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f8ec80a2cb0 0x7f8ec80a5170 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:16.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:16.888+0000 7f8ee3a53640 1 --2- 172.21.15.26:0/3734003794 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8edc104f10 0x7f8edc1983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:16.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:16.888+0000 7f8ee3a53640 1 --2- 172.21.15.26:0/3734003794 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8edc107840 0x7f8edc19f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:16.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:16.888+0000 7f8ee3a53640 1 --2- 172.21.15.26:0/3734003794 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8edc1025e0 0x7f8edc197e70 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:16.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:16.888+0000 7f8ee3a53640 1 -- 172.21.15.26:0/3734003794 >> 172.21.15.26:0/3734003794 conn(0x7f8edc0fc1e0 msgr2=0x7f8edc103040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:18:16.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:16.888+0000 7f8ee3a53640 1 -- 172.21.15.26:0/3734003794 shutdown_connections 2024-09-11T23:18:16.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:16.888+0000 7f8ee3a53640 1 -- 172.21.15.26:0/3734003794 wait complete. 2024-09-11T23:18:16.906 INFO:teuthology.run_tasks:Running task kclient... 2024-09-11T23:18:16.932 INFO:tasks.kclient:Mounting kernel clients... 2024-09-11T23:18:16.933 INFO:tasks.kclient:config is {'client.0': {}} 2024-09-11T23:18:16.933 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T23:18:16.933 DEBUG:teuthology.orchestra.run.smithi026:> ip netns list 2024-09-11T23:18:16.952 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T23:18:16.953 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip link delete ceph-brx 2024-09-11T23:18:17.023 INFO:teuthology.orchestra.run.smithi026.stderr:Cannot find device "ceph-brx" 2024-09-11T23:18:17.025 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T23:18:17.025 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=secure', 'wsync'], 'syntax': 'v2', 'mount_subvol_num': 1} 2024-09-11T23:18:17.025 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-11T23:18:17.025 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-11T23:18:17.026 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph fs ls 2024-09-11T23:18:17.235 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:16 smithi076 ceph-mon[35296]: from='client.? 172.21.15.26:0/3734003794' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-09-11T23:18:17.235 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:16 smithi076 ceph-mon[35296]: from='client.? ' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-09-11T23:18:17.235 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Sep 11 23:18:16 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-y[36658]: ignoring --setuser ceph since I am not root 2024-09-11T23:18:17.235 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Sep 11 23:18:16 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-y[36658]: ignoring --setgroup ceph since I am not root 2024-09-11T23:18:17.236 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Sep 11 23:18:16 smithi076 ceph-mgr[36662]: -- 172.21.15.76:0/3887131459 <== mon.2 v2:172.21.15.76:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5576378630e0 con 0x557637841400 2024-09-11T23:18:17.236 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Sep 11 23:18:17 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-y[36658]: 2024-09-11T23:18:17.028+0000 7eff6b6f4040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-11T23:18:17.236 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Sep 11 23:18:17 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-y[36658]: 2024-09-11T23:18:17.086+0000 7eff6b6f4040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-11T23:18:17.236 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Sep 11 23:18:17 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-y[36658]: 2024-09-11T23:18:17.234+0000 7eff6b6f4040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-11T23:18:17.238 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:16 smithi136 ceph-mon[35121]: from='client.? 172.21.15.26:0/3734003794' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-09-11T23:18:17.238 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:16 smithi136 ceph-mon[35121]: from='client.? ' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-09-11T23:18:17.238 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Sep 11 23:18:16 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-z[36925]: ignoring --setuser ceph since I am not root 2024-09-11T23:18:17.239 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Sep 11 23:18:16 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-z[36925]: ignoring --setgroup ceph since I am not root 2024-09-11T23:18:17.239 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Sep 11 23:18:17 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-z[36925]: 2024-09-11T23:18:17.032+0000 7f5717a18040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-11T23:18:17.239 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Sep 11 23:18:17 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-z[36925]: 2024-09-11T23:18:17.089+0000 7f5717a18040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-11T23:18:17.239 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Sep 11 23:18:17 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-z[36925]: 2024-09-11T23:18:17.238+0000 7f5717a18040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-11T23:18:17.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:16 smithi026 ceph-mon[31172]: from='client.? 172.21.15.26:0/3734003794' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-09-11T23:18:17.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:16 smithi026 ceph-mon[31172]: from='client.? ' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-09-11T23:18:17.251 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:18:16 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: ignoring --setuser ceph since I am not root 2024-09-11T23:18:17.251 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:18:16 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: ignoring --setgroup ceph since I am not root 2024-09-11T23:18:17.251 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:18:17 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: 2024-09-11T23:18:17.045+0000 7f7d35c1b040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-11T23:18:17.251 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:18:17 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: 2024-09-11T23:18:17.107+0000 7f7d35c1b040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-11T23:18:17.331 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:18:17.489 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Sep 11 23:18:17 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-z[36925]: 2024-09-11T23:18:17.306+0000 7f5717a18040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-11T23:18:17.489 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Sep 11 23:18:17 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-z[36925]: 2024-09-11T23:18:17.453+0000 7f5717a18040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-11T23:18:17.553 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Sep 11 23:18:17 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-y[36658]: 2024-09-11T23:18:17.301+0000 7eff6b6f4040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-11T23:18:17.553 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Sep 11 23:18:17 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-y[36658]: 2024-09-11T23:18:17.447+0000 7eff6b6f4040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-11T23:18:17.583 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:18:17 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: 2024-09-11T23:18:17.268+0000 7f7d35c1b040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-11T23:18:17.583 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:18:17 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: 2024-09-11T23:18:17.349+0000 7f7d35c1b040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-11T23:18:17.583 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:18:17 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: 2024-09-11T23:18:17.507+0000 7f7d35c1b040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-11T23:18:17.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:17.780+0000 7f2fce2be640 1 -- 172.21.15.26:0/2164181545 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2fc81082a0 msgr2=0x7f2fc810ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:17.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:17.780+0000 7f2fce2be640 1 --2- 172.21.15.26:0/2164181545 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2fc81082a0 0x7f2fc810ec40 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f2fc0009cd0 tx=0x7f2fc002a8e0 comp rx=0 tx=0).stop 2024-09-11T23:18:17.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:17.780+0000 7f2fce2be640 1 -- 172.21.15.26:0/2164181545 shutdown_connections 2024-09-11T23:18:17.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:17.780+0000 7f2fce2be640 1 --2- 172.21.15.26:0/2164181545 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2fc81082a0 0x7f2fc810ec40 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:17.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:17.780+0000 7f2fce2be640 1 --2- 172.21.15.26:0/2164181545 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2fc81036e0 0x7f2fc8107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:17.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:17.780+0000 7f2fce2be640 1 --2- 172.21.15.26:0/2164181545 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2fc8102d30 0x7f2fc8103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:17.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:17.780+0000 7f2fce2be640 1 -- 172.21.15.26:0/2164181545 >> 172.21.15.26:0/2164181545 conn(0x7f2fc80fc7d0 msgr2=0x7f2fc80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:18:17.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:17.780+0000 7f2fce2be640 1 -- 172.21.15.26:0/2164181545 shutdown_connections 2024-09-11T23:18:17.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:17.780+0000 7f2fce2be640 1 -- 172.21.15.26:0/2164181545 wait complete. 2024-09-11T23:18:17.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:17.780+0000 7f2fce2be640 1 Processor -- start 2024-09-11T23:18:17.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:17.780+0000 7f2fce2be640 1 -- start start 2024-09-11T23:18:17.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:17.781+0000 7f2fce2be640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2fc8102d30 0x7f2fc8100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:17.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:17.781+0000 7f2fce2be640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2fc81036e0 0x7f2fc8100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:17.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:17.781+0000 7f2fce2be640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2fc81082a0 0x7f2fc8101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:17.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:17.781+0000 7f2fce2be640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f2fc81120a0 con 0x7f2fc8102d30 2024-09-11T23:18:17.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:17.781+0000 7f2fce2be640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f2fc8111f20 con 0x7f2fc81036e0 2024-09-11T23:18:17.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:17.781+0000 7f2fce2be640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f2fc8112220 con 0x7f2fc81082a0 2024-09-11T23:18:17.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:17.781+0000 7f2fc7fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2fc8102d30 0x7f2fc8100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:17.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:17.781+0000 7f2fc7fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2fc8102d30 0x7f2fc8100990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44718/0 (socket says 172.21.15.26:44718) 2024-09-11T23:18:17.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:17.781+0000 7f2fc7fff640 1 -- 172.21.15.26:0/90659427 learned_addr learned my addr 172.21.15.26:0/90659427 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:18:17.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:17.781+0000 7f2fcc85b640 1 --2- 172.21.15.26:0/90659427 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2fc81082a0 0x7f2fc8101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:17.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:17.781+0000 7f2fbffff640 1 --2- 172.21.15.26:0/90659427 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2fc81036e0 0x7f2fc8100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:17.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:17.781+0000 7f2fc7fff640 1 -- 172.21.15.26:0/90659427 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2fc81082a0 msgr2=0x7f2fc8101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:17.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:17.781+0000 7f2fc7fff640 1 --2- 172.21.15.26:0/90659427 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2fc81082a0 0x7f2fc8101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:17.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:17.781+0000 7f2fc7fff640 1 -- 172.21.15.26:0/90659427 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2fc81036e0 msgr2=0x7f2fc8100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:17.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:17.781+0000 7f2fc7fff640 1 --2- 172.21.15.26:0/90659427 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2fc81036e0 0x7f2fc8100ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:17.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:17.781+0000 7f2fc7fff640 1 -- 172.21.15.26:0/90659427 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fc81a6a20 con 0x7f2fc8102d30 2024-09-11T23:18:17.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:17.781+0000 7f2fc7fff640 1 --2- 172.21.15.26:0/90659427 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2fc8102d30 0x7f2fc8100990 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f2fb800ea60 tx=0x7f2fb800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:18:17.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:17.781+0000 7f2fc5ffb640 1 -- 172.21.15.26:0/90659427 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fb800cde0 con 0x7f2fc8102d30 2024-09-11T23:18:17.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:17.781+0000 7f2fce2be640 1 -- 172.21.15.26:0/90659427 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fc81a6d10 con 0x7f2fc8102d30 2024-09-11T23:18:17.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:17.781+0000 7f2fc5ffb640 1 -- 172.21.15.26:0/90659427 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2fb80035f0 con 0x7f2fc8102d30 2024-09-11T23:18:17.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:17.781+0000 7f2fc5ffb640 1 -- 172.21.15.26:0/90659427 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fb8010670 con 0x7f2fc8102d30 2024-09-11T23:18:17.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:17.781+0000 7f2fce2be640 1 -- 172.21.15.26:0/90659427 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fc81a7240 con 0x7f2fc8102d30 2024-09-11T23:18:17.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:17.783+0000 7f2fce2be640 1 -- 172.21.15.26:0/90659427 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2fc8103110 con 0x7f2fc8102d30 2024-09-11T23:18:17.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:17.784+0000 7f2fc5ffb640 1 -- 172.21.15.26:0/90659427 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 19) ==== 137850+0+0 (secure 0 0 0) 0x7f2fb80037a0 con 0x7f2fc8102d30 2024-09-11T23:18:17.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:17.785+0000 7f2fc5ffb640 1 --2- 172.21.15.26:0/90659427 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f2fb40a2bd0 0x7f2fb40a5090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:17.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:17.785+0000 7f2fbffff640 1 -- 172.21.15.26:0/90659427 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f2fb40a2bd0 msgr2=0x7f2fb40a5090 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.26:6800/110833859 2024-09-11T23:18:17.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:17.785+0000 7f2fc5ffb640 1 -- 172.21.15.26:0/90659427 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f2fb80cf480 con 0x7f2fc8102d30 2024-09-11T23:18:17.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:17.785+0000 7f2fbffff640 1 --2- 172.21.15.26:0/90659427 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f2fb40a2bd0 0x7f2fb40a5090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-11T23:18:17.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:17.788+0000 7f2fc5ffb640 1 -- 172.21.15.26:0/90659427 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2fb809c430 con 0x7f2fc8102d30 2024-09-11T23:18:17.873 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Sep 11 23:18:17 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-y[36658]: 2024-09-11T23:18:17.589+0000 7eff6b6f4040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-11T23:18:17.873 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Sep 11 23:18:17 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-z[36925]: 2024-09-11T23:18:17.595+0000 7f5717a18040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-11T23:18:17.874 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:18:17 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: 2024-09-11T23:18:17.660+0000 7f7d35c1b040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-11T23:18:17.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:17.986+0000 7f2fbffff640 1 -- 172.21.15.26:0/90659427 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f2fb40a2bd0 msgr2=0x7f2fb40a5090 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.26:6800/110833859 2024-09-11T23:18:17.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:17.986+0000 7f2fbffff640 1 --2- 172.21.15.26:0/90659427 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f2fb40a2bd0 0x7f2fb40a5090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-11T23:18:18.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:18.025+0000 7f2fce2be640 1 -- 172.21.15.26:0/90659427 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f2fc8101ec0 con 0x7f2fc8102d30 2024-09-11T23:18:18.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:18.025+0000 7f2fc5ffb640 1 -- 172.21.15.26:0/90659427 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v28) ==== 53+0+73 (secure 0 0 0) 0x7f2fc8101ec0 con 0x7f2fc8102d30 2024-09-11T23:18:18.026 INFO:teuthology.orchestra.run.smithi026.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-11T23:18:18.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:18.027+0000 7f2fce2be640 1 -- 172.21.15.26:0/90659427 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f2fb40a2bd0 msgr2=0x7f2fb40a5090 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-11T23:18:18.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:18.027+0000 7f2fce2be640 1 --2- 172.21.15.26:0/90659427 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f2fb40a2bd0 0x7f2fb40a5090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:18.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:18.027+0000 7f2fce2be640 1 -- 172.21.15.26:0/90659427 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2fc8102d30 msgr2=0x7f2fc8100990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:18.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:18.027+0000 7f2fce2be640 1 --2- 172.21.15.26:0/90659427 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2fc8102d30 0x7f2fc8100990 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f2fb800ea60 tx=0x7f2fb800ef20 comp rx=0 tx=0).stop 2024-09-11T23:18:18.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:18.027+0000 7f2fce2be640 1 -- 172.21.15.26:0/90659427 shutdown_connections 2024-09-11T23:18:18.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:18.027+0000 7f2fce2be640 1 --2- 172.21.15.26:0/90659427 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f2fb40a2bd0 0x7f2fb40a5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:18.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:18.027+0000 7f2fce2be640 1 --2- 172.21.15.26:0/90659427 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2fc81036e0 0x7f2fc8100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:18.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:18.027+0000 7f2fce2be640 1 --2- 172.21.15.26:0/90659427 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2fc81082a0 0x7f2fc8101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:18.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:18.027+0000 7f2fce2be640 1 --2- 172.21.15.26:0/90659427 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2fc8102d30 0x7f2fc8100990 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:18.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:18.027+0000 7f2fce2be640 1 -- 172.21.15.26:0/90659427 >> 172.21.15.26:0/90659427 conn(0x7f2fc80fc7d0 msgr2=0x7f2fc8105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:18:18.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:18.027+0000 7f2fce2be640 1 -- 172.21.15.26:0/90659427 shutdown_connections 2024-09-11T23:18:18.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:18.027+0000 7f2fce2be640 1 -- 172.21.15.26:0/90659427 wait complete. 2024-09-11T23:18:18.152 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:17 smithi076 ceph-mon[35296]: from='client.? ' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-09-11T23:18:18.152 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:17 smithi076 ceph-mon[35296]: mgrmap e19: x(active, since 8m), standbys: z, y 2024-09-11T23:18:18.152 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Sep 11 23:18:17 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-y[36658]: 2024-09-11T23:18:17.904+0000 7eff6b6f4040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-11T23:18:18.152 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Sep 11 23:18:17 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-y[36658]: 2024-09-11T23:18:17.966+0000 7eff6b6f4040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-11T23:18:18.152 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Sep 11 23:18:18 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-y[36658]: 2024-09-11T23:18:18.151+0000 7eff6b6f4040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-11T23:18:18.160 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:17 smithi136 ceph-mon[35121]: from='client.? ' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-09-11T23:18:18.160 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:17 smithi136 ceph-mon[35121]: mgrmap e19: x(active, since 8m), standbys: z, y 2024-09-11T23:18:18.160 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Sep 11 23:18:17 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-z[36925]: 2024-09-11T23:18:17.910+0000 7f5717a18040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-11T23:18:18.160 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Sep 11 23:18:17 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-z[36925]: 2024-09-11T23:18:17.973+0000 7f5717a18040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-11T23:18:18.251 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:18:18 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: 2024-09-11T23:18:18.017+0000 7f7d35c1b040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-11T23:18:18.251 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:18:18 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: 2024-09-11T23:18:18.086+0000 7f7d35c1b040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-11T23:18:18.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:17 smithi026 ceph-mon[31172]: from='client.? ' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-09-11T23:18:18.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:17 smithi026 ceph-mon[31172]: mgrmap e19: x(active, since 8m), standbys: z, y 2024-09-11T23:18:18.489 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Sep 11 23:18:18 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-z[36925]: 2024-09-11T23:18:18.160+0000 7f5717a18040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-11T23:18:18.750 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:18:18 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: 2024-09-11T23:18:18.283+0000 7f7d35c1b040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-11T23:18:18.798 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-11T23:18:18.798 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-11T23:18:18.798 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi026.front.sepia.ceph.com 2024-09-11T23:18:18.798 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-11T23:18:18.798 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-11T23:18:18.798 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-11T23:18:18.798 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-11T23:18:18.798 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-11T23:18:18.798 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-11T23:18:18.799 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T23:18:18.799 DEBUG:teuthology.orchestra.run.smithi026:> ip addr 2024-09-11T23:18:18.816 INFO:teuthology.orchestra.run.smithi026.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-11T23:18:18.816 INFO:teuthology.orchestra.run.smithi026.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-11T23:18:18.816 INFO:teuthology.orchestra.run.smithi026.stdout: inet 127.0.0.1/8 scope host lo 2024-09-11T23:18:18.816 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-09-11T23:18:18.816 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 ::1/128 scope host 2024-09-11T23:18:18.816 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-09-11T23:18:18.816 INFO:teuthology.orchestra.run.smithi026.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-11T23:18:18.816 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether 0c:c4:7a:6b:ee:6a brd ff:ff:ff:ff:ff:ff 2024-09-11T23:18:18.816 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp7s0f0 2024-09-11T23:18:18.816 INFO:teuthology.orchestra.run.smithi026.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-11T23:18:18.817 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether 0c:c4:7a:6b:ee:6b brd ff:ff:ff:ff:ff:ff 2024-09-11T23:18:18.817 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp7s0f1 2024-09-11T23:18:18.817 INFO:teuthology.orchestra.run.smithi026.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-11T23:18:18.817 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether ac:1f:6b:a5:ca:c0 brd ff:ff:ff:ff:ff:ff 2024-09-11T23:18:18.817 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp1s0f0 2024-09-11T23:18:18.817 INFO:teuthology.orchestra.run.smithi026.stdout: inet 172.21.15.26/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-11T23:18:18.817 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft 42220sec preferred_lft 42220sec 2024-09-11T23:18:18.817 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 fe80::ae1f:6bff:fea5:cac0/64 scope link 2024-09-11T23:18:18.817 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-09-11T23:18:18.817 INFO:teuthology.orchestra.run.smithi026.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-11T23:18:18.817 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether ac:1f:6b:a5:ca:c1 brd ff:ff:ff:ff:ff:ff 2024-09-11T23:18:18.817 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp1s0f1 2024-09-11T23:18:18.818 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-11T23:18:18.818 DEBUG:teuthology.orchestra.run.smithi026:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-11T23:18:18.818 DEBUG:teuthology.orchestra.run.smithi026:> set -e 2024-09-11T23:18:18.818 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip link add name ceph-brx type bridge 2024-09-11T23:18:18.818 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip addr flush dev ceph-brx 2024-09-11T23:18:18.818 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip link set ceph-brx up 2024-09-11T23:18:18.818 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-11T23:18:18.818 DEBUG:teuthology.orchestra.run.smithi026:> ') 2024-09-11T23:18:18.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:18 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-11T23:18:19.019 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:18 smithi026 ceph-mon[31172]: from='client.? 172.21.15.26:0/90659427' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-11T23:18:19.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T23:18:19.028 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T23:18:19.029 DEBUG:teuthology.orchestra.run.smithi026:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-11T23:18:19.057 INFO:teuthology.orchestra.run.smithi026.stdout:1 2024-09-11T23:18:19.059 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T23:18:19.059 DEBUG:teuthology.orchestra.run.smithi026:> route 2024-09-11T23:18:19.125 INFO:teuthology.orchestra.run.smithi026.stdout:Kernel IP routing table 2024-09-11T23:18:19.125 INFO:teuthology.orchestra.run.smithi026.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-11T23:18:19.125 INFO:teuthology.orchestra.run.smithi026.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-11T23:18:19.125 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-11T23:18:19.125 INFO:teuthology.orchestra.run.smithi026.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-11T23:18:19.126 DEBUG:teuthology.orchestra.run.smithi026:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-11T23:18:19.126 DEBUG:teuthology.orchestra.run.smithi026:> set -e 2024-09-11T23:18:19.126 DEBUG:teuthology.orchestra.run.smithi026:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-11T23:18:19.127 DEBUG:teuthology.orchestra.run.smithi026:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-11T23:18:19.127 DEBUG:teuthology.orchestra.run.smithi026:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-11T23:18:19.127 DEBUG:teuthology.orchestra.run.smithi026:> ') 2024-09-11T23:18:19.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:19 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-11T23:18:19.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:18 smithi136 ceph-mon[35121]: from='client.? 172.21.15.26:0/90659427' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-11T23:18:19.239 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Sep 11 23:18:19 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-z[36925]: 2024-09-11T23:18:19.051+0000 7f5717a18040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-11T23:18:19.239 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Sep 11 23:18:19 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-z[36925]: 2024-09-11T23:18:19.120+0000 7f5717a18040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-11T23:18:19.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:18 smithi076 ceph-mon[35296]: from='client.? 172.21.15.26:0/90659427' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-11T23:18:19.303 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Sep 11 23:18:19 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-y[36658]: 2024-09-11T23:18:19.030+0000 7eff6b6f4040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-11T23:18:19.303 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Sep 11 23:18:19 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-y[36658]: 2024-09-11T23:18:19.097+0000 7eff6b6f4040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-11T23:18:19.318 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:18:19 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: 2024-09-11T23:18:19.243+0000 7f7d35c1b040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-11T23:18:19.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T23:18:19.455 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T23:18:19.456 DEBUG:teuthology.orchestra.run.smithi026:> ip netns list 2024-09-11T23:18:19.472 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T23:18:19.472 DEBUG:teuthology.orchestra.run.smithi026:> ip netns list-id 2024-09-11T23:18:19.528 DEBUG:teuthology.orchestra.run.smithi026:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-11T23:18:19.528 DEBUG:teuthology.orchestra.run.smithi026:> set -e 2024-09-11T23:18:19.528 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-11T23:18:19.528 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-09-11T23:18:19.529 DEBUG:teuthology.orchestra.run.smithi026:> ') 2024-09-11T23:18:19.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:19 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-11T23:18:19.628 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:18:19 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: 2024-09-11T23:18:19.317+0000 7f7d35c1b040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-11T23:18:19.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T23:18:19.652 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-09-11T23:18:19.652 DEBUG:teuthology.orchestra.run.smithi026:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-11T23:18:19.652 DEBUG:teuthology.orchestra.run.smithi026:> set -e 2024-09-11T23:18:19.652 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-09-11T23:18:19.653 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-11T23:18:19.653 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-09-11T23:18:19.653 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-09-11T23:18:19.653 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-09-11T23:18:19.653 DEBUG:teuthology.orchestra.run.smithi026:> ') 2024-09-11T23:18:19.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:19 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-11T23:18:19.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T23:18:19.791 DEBUG:teuthology.orchestra.run.smithi026:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-11T23:18:19.791 DEBUG:teuthology.orchestra.run.smithi026:> set -e 2024-09-11T23:18:19.791 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip link set brx.0 up 2024-09-11T23:18:19.791 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip link set dev brx.0 master ceph-brx 2024-09-11T23:18:19.791 DEBUG:teuthology.orchestra.run.smithi026:> ') 2024-09-11T23:18:19.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:19 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-11T23:18:19.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T23:18:19.871 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-11T23:18:19.871 DEBUG:teuthology.orchestra.run.smithi026:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-11T23:18:19.896 INFO:teuthology.orchestra.run.smithi026.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-11T23:18:19.896 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-11T23:18:19.896 DEBUG:teuthology.orchestra.run.smithi026:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-11T23:18:19.953 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-09-11T23:18:19.953 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-09-11T23:18:19.953 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/volumes/qa/sv_1/9fc8f023-d646-4fec-98c7-1f3d972f60a9 2024-09-11T23:18:19.953 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T23:18:19.953 DEBUG:teuthology.orchestra.run.smithi026:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/volumes/qa/sv_1/9fc8f023-d646-4fec-98c7-1f3d972f60a9 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=secure,wsync,nofallback 2024-09-11T23:18:20.055 INFO:teuthology.orchestra.run.smithi026.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=secure,wsync,nofallback 2024-09-11T23:18:20.055 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=secure,wsync". 2024-09-11T23:18:20.065 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Sep 11 23:18:19 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-y[36658]: 2024-09-11T23:18:19.804+0000 7eff6b6f4040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-11T23:18:20.065 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Sep 11 23:18:19 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-y[36658]: 2024-09-11T23:18:19.919+0000 7eff6b6f4040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-11T23:18:20.105 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Sep 11 23:18:19 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-z[36925]: 2024-09-11T23:18:19.840+0000 7f5717a18040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-11T23:18:20.105 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Sep 11 23:18:19 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-z[36925]: 2024-09-11T23:18:19.958+0000 7f5717a18040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-11T23:18:20.196 INFO:teuthology.orchestra.run.smithi026.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=secure,wsync,nofallback 2024-09-11T23:18:20.196 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=secure,wsync". 2024-09-11T23:18:20.196 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: resolved to: "172.21.15.26:3300,172.21.15.76:3300,172.21.15.136:3300" 2024-09-11T23:18:20.196 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: trying mount with new device syntax: 0@af7e6fc8-7092-11ef-bcea-c7b262605968.cephfs=/volumes/qa/sv_1/9fc8f023-d646-4fec-98c7-1f3d972f60a9 2024-09-11T23:18:20.196 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=secure,wsync,name=0,key=0,mon_addr=172.21.15.26:3300/172.21.15.76:3300/172.21.15.136:3300" will pass to kernel 2024-09-11T23:18:20.197 INFO:teuthology.orchestra.run.smithi026.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-11T23:18:20.197 INFO:teuthology.orchestra.run.smithi026.stdout: You just mounted a file system that supports labels which does not 2024-09-11T23:18:20.197 INFO:teuthology.orchestra.run.smithi026.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-11T23:18:20.197 INFO:teuthology.orchestra.run.smithi026.stdout: applications will generate AVC messages and not be allowed access to 2024-09-11T23:18:20.197 INFO:teuthology.orchestra.run.smithi026.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-11T23:18:20.199 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-11T23:18:20.199 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T23:18:20.199 DEBUG:teuthology.orchestra.run.smithi026:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-11T23:18:20.227 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-11T23:18:20.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:20 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-11T23:18:20.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T23:18:20.326 INFO:teuthology.orchestra.run.smithi026.stdout:client15180 2024-09-11T23:18:20.326 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-09-11T23:18:20.326 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T23:18:20.343 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-11T23:18:20.346 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Sep 11 23:18:20 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-y[36658]: 2024-09-11T23:18:20.064+0000 7eff6b6f4040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-11T23:18:20.346 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Sep 11 23:18:20 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-y[36658]: 2024-09-11T23:18:20.126+0000 7eff6b6f4040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-11T23:18:20.346 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Sep 11 23:18:20 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-y[36658]: 2024-09-11T23:18:20.278+0000 7eff6b6f4040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-11T23:18:20.346 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Sep 11 23:18:20 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-y[36658]: 2024-09-11T23:18:20.345+0000 7eff6b6f4040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-11T23:18:20.387 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Sep 11 23:18:20 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-z[36925]: 2024-09-11T23:18:20.104+0000 7f5717a18040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-11T23:18:20.387 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Sep 11 23:18:20 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-z[36925]: 2024-09-11T23:18:20.166+0000 7f5717a18040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-11T23:18:20.387 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Sep 11 23:18:20 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-z[36925]: 2024-09-11T23:18:20.318+0000 7f5717a18040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-11T23:18:20.392 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:18:20 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: 2024-09-11T23:18:20.110+0000 7f7d35c1b040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-11T23:18:20.392 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:18:20 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: 2024-09-11T23:18:20.237+0000 7f7d35c1b040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-11T23:18:20.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:20 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-11T23:18:20.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T23:18:20.445 INFO:teuthology.orchestra.run.smithi026.stdout:client15180 2024-09-11T23:18:20.445 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-11T23:18:20.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:20 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-11T23:18:20.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T23:18:20.545 INFO:teuthology.orchestra.run.smithi026.stdout:client15180 2024-09-11T23:18:20.545 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968.client15180/status') 2024-09-11T23:18:20.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:20 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-11T23:18:20.606 INFO:teuthology.orchestra.run.smithi026.stdout:instance: client.15180 (3)192.168.0.1:0/3637924025 2024-09-11T23:18:20.606 INFO:teuthology.orchestra.run.smithi026.stdout:blocklisted: false 2024-09-11T23:18:20.607 INFO:teuthology.orchestra.run.smithi026.stderr:0+1 records in 2024-09-11T23:18:20.607 INFO:teuthology.orchestra.run.smithi026.stderr:0+1 records out 2024-09-11T23:18:20.607 INFO:teuthology.orchestra.run.smithi026.stderr:70 bytes copied, 2.8734e-05 s, 2.4 MB/s 2024-09-11T23:18:20.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T23:18:20.615 INFO:teuthology.run_tasks:Running task exec... 2024-09-11T23:18:20.628 INFO:teuthology.task.exec:Executing custom commands... 2024-09-11T23:18:20.629 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi026.front.sepia.ceph.com 2024-09-11T23:18:20.629 DEBUG:teuthology.orchestra.run.smithi026:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate true' 2024-09-11T23:18:20.630 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Sep 11 23:18:20 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-y[36658]: 2024-09-11T23:18:20.405+0000 7eff6b6f4040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-11T23:18:20.630 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Sep 11 23:18:20 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-y[36658]: 2024-09-11T23:18:20.563+0000 7eff6b6f4040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-11T23:18:20.653 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:18:20 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: 2024-09-11T23:18:20.391+0000 7f7d35c1b040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-11T23:18:20.653 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:18:20 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: 2024-09-11T23:18:20.458+0000 7f7d35c1b040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-11T23:18:20.653 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:18:20 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: 2024-09-11T23:18:20.623+0000 7f7d35c1b040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-11T23:18:20.672 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Sep 11 23:18:20 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-z[36925]: 2024-09-11T23:18:20.387+0000 7f5717a18040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-11T23:18:20.672 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Sep 11 23:18:20 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-z[36925]: 2024-09-11T23:18:20.448+0000 7f5717a18040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-11T23:18:20.672 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Sep 11 23:18:20 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-z[36925]: 2024-09-11T23:18:20.607+0000 7f5717a18040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-11T23:18:20.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:20.737+0000 7f0fcb80e640 1 -- 172.21.15.26:0/3616105732 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0fc4101c30 msgr2=0x7f0fc41020b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:20.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:20.737+0000 7f0fcb80e640 1 --2- 172.21.15.26:0/3616105732 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0fc4101c30 0x7f0fc41020b0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f0fb4009900 tx=0x7f0fb402a410 comp rx=0 tx=0).stop 2024-09-11T23:18:20.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:20.737+0000 7f0fcb80e640 1 -- 172.21.15.26:0/3616105732 shutdown_connections 2024-09-11T23:18:20.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:20.737+0000 7f0fcb80e640 1 --2- 172.21.15.26:0/3616105732 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0fc41025f0 0x7f0fc4108e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:20.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:20.737+0000 7f0fcb80e640 1 --2- 172.21.15.26:0/3616105732 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0fc4101c30 0x7f0fc41020b0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:20.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:20.737+0000 7f0fcb80e640 1 --2- 172.21.15.26:0/3616105732 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0fc4100a40 0x7f0fc4100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:20.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:20.737+0000 7f0fcb80e640 1 -- 172.21.15.26:0/3616105732 >> 172.21.15.26:0/3616105732 conn(0x7f0fc40fc1e0 msgr2=0x7f0fc40fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:18:20.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:20.737+0000 7f0fcb80e640 1 -- 172.21.15.26:0/3616105732 shutdown_connections 2024-09-11T23:18:20.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:20.737+0000 7f0fcb80e640 1 -- 172.21.15.26:0/3616105732 wait complete. 2024-09-11T23:18:20.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:20.737+0000 7f0fcb80e640 1 Processor -- start 2024-09-11T23:18:20.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:20.737+0000 7f0fcb80e640 1 -- start start 2024-09-11T23:18:20.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:20.738+0000 7f0fcb80e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0fc4100a40 0x7f0fc4196cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:20.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:20.738+0000 7f0fcb80e640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0fc4101c30 0x7f0fc41971f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:20.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:20.738+0000 7f0fcb80e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0fc41025f0 0x7f0fc419e250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:20.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:20.738+0000 7f0fcb80e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f0fc410c6c0 con 0x7f0fc4100a40 2024-09-11T23:18:20.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:20.738+0000 7f0fcb80e640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f0fc410c540 con 0x7f0fc41025f0 2024-09-11T23:18:20.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:20.738+0000 7f0fcb80e640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f0fc410c840 con 0x7f0fc4101c30 2024-09-11T23:18:20.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:20.738+0000 7f0fc95aa640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0fc4100a40 0x7f0fc4196cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:20.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:20.738+0000 7f0fc95aa640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0fc4100a40 0x7f0fc4196cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47098/0 (socket says 172.21.15.26:47098) 2024-09-11T23:18:20.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:20.738+0000 7f0fc95aa640 1 -- 172.21.15.26:0/2507209398 learned_addr learned my addr 172.21.15.26:0/2507209398 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:18:20.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:20.738+0000 7f0fc8da9640 1 --2- 172.21.15.26:0/2507209398 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0fc4101c30 0x7f0fc41971f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:20.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:20.738+0000 7f0fc9dab640 1 --2- 172.21.15.26:0/2507209398 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0fc41025f0 0x7f0fc419e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:20.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:20.738+0000 7f0fc95aa640 1 -- 172.21.15.26:0/2507209398 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0fc4101c30 msgr2=0x7f0fc41971f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:20.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:20.738+0000 7f0fc95aa640 1 --2- 172.21.15.26:0/2507209398 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0fc4101c30 0x7f0fc41971f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:20.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:20.738+0000 7f0fc95aa640 1 -- 172.21.15.26:0/2507209398 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0fc41025f0 msgr2=0x7f0fc419e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:20.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:20.738+0000 7f0fc95aa640 1 --2- 172.21.15.26:0/2507209398 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0fc41025f0 0x7f0fc419e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:20.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:20.738+0000 7f0fc95aa640 1 -- 172.21.15.26:0/2507209398 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fc419e8f0 con 0x7f0fc4100a40 2024-09-11T23:18:20.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:20.738+0000 7f0fc95aa640 1 --2- 172.21.15.26:0/2507209398 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0fc4100a40 0x7f0fc4196cb0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f0fc000ead0 tx=0x7f0fc000ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:18:20.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:20.738+0000 7f0fba7fc640 1 -- 172.21.15.26:0/2507209398 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fc000bbd0 con 0x7f0fc4100a40 2024-09-11T23:18:20.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:20.739+0000 7f0fcb80e640 1 -- 172.21.15.26:0/2507209398 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0fc4069900 con 0x7f0fc4100a40 2024-09-11T23:18:20.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:20.739+0000 7f0fba7fc640 1 -- 172.21.15.26:0/2507209398 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0fc000bd70 con 0x7f0fc4100a40 2024-09-11T23:18:20.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:20.739+0000 7f0fba7fc640 1 -- 172.21.15.26:0/2507209398 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fc0017410 con 0x7f0fc4100a40 2024-09-11T23:18:20.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:20.739+0000 7f0fcb80e640 1 -- 172.21.15.26:0/2507209398 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0fc4069e20 con 0x7f0fc4100a40 2024-09-11T23:18:20.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:20.740+0000 7f0fcb80e640 1 -- 172.21.15.26:0/2507209398 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0fc4100e40 con 0x7f0fc4100a40 2024-09-11T23:18:20.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:20.741+0000 7f0fba7fc640 1 -- 172.21.15.26:0/2507209398 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 19) ==== 137850+0+0 (secure 0 0 0) 0x7f0fc001e020 con 0x7f0fc4100a40 2024-09-11T23:18:20.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:20.741+0000 7f0fba7fc640 1 --2- 172.21.15.26:0/2507209398 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f0fb00a2c80 0x7f0fb00a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:20.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:20.742+0000 7f0fc8da9640 1 -- 172.21.15.26:0/2507209398 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f0fb00a2c80 msgr2=0x7f0fb00a5140 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.26:6800/110833859 2024-09-11T23:18:20.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:20.742+0000 7f0fba7fc640 1 -- 172.21.15.26:0/2507209398 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f0fc00d0730 con 0x7f0fc4100a40 2024-09-11T23:18:20.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:20.742+0000 7f0fc8da9640 1 --2- 172.21.15.26:0/2507209398 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f0fb00a2c80 0x7f0fb00a5140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-11T23:18:20.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:20.750+0000 7f0fba7fc640 1 -- 172.21.15.26:0/2507209398 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0fc009d690 con 0x7f0fc4100a40 2024-09-11T23:18:20.896 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Sep 11 23:18:20 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-y[36658]: 2024-09-11T23:18:20.628+0000 7eff6b6f4040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-11T23:18:20.897 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Sep 11 23:18:20 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-y[36658]: 2024-09-11T23:18:20.694+0000 7eff6b6f4040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-11T23:18:20.897 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Sep 11 23:18:20 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-y[36658]: 2024-09-11T23:18:20.777+0000 7eff6b6f4040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-11T23:18:20.897 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Sep 11 23:18:20 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-y[36658]: 2024-09-11T23:18:20.836+0000 7eff6b6f4040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-11T23:18:20.897 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Sep 11 23:18:20 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-y[36658]: 2024-09-11T23:18:20.896+0000 7eff6b6f4040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-11T23:18:20.941 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:18:20 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: 2024-09-11T23:18:20.699+0000 7f7d35c1b040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-11T23:18:20.941 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:18:20 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: 2024-09-11T23:18:20.769+0000 7f7d35c1b040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-11T23:18:20.941 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:18:20 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: 2024-09-11T23:18:20.940+0000 7f7d35c1b040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-11T23:18:20.942 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Sep 11 23:18:20 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-z[36925]: 2024-09-11T23:18:20.671+0000 7f5717a18040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-11T23:18:20.942 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Sep 11 23:18:20 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-z[36925]: 2024-09-11T23:18:20.738+0000 7f5717a18040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-11T23:18:20.942 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Sep 11 23:18:20 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-z[36925]: 2024-09-11T23:18:20.822+0000 7f5717a18040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-11T23:18:20.942 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Sep 11 23:18:20 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-z[36925]: 2024-09-11T23:18:20.882+0000 7f5717a18040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-11T23:18:20.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:20.943+0000 7f0fc8da9640 1 -- 172.21.15.26:0/2507209398 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f0fb00a2c80 msgr2=0x7f0fb00a5140 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.26:6800/110833859 2024-09-11T23:18:20.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:20.943+0000 7f0fc8da9640 1 --2- 172.21.15.26:0/2507209398 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f0fb00a2c80 0x7f0fb00a5140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-11T23:18:20.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:20.967+0000 7f0fcb80e640 1 -- 172.21.15.26:0/2507209398 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} v 0) -- 0x7f0fc419eb80 con 0x7f0fc4100a40 2024-09-11T23:18:21.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.018+0000 7f0fba7fc640 1 -- 172.21.15.26:0/2507209398 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]=0 v29) ==== 117+0+0 (secure 0 0 0) 0x7f0fc009cf90 con 0x7f0fc4100a40 2024-09-11T23:18:21.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.022+0000 7f0fcb80e640 1 -- 172.21.15.26:0/2507209398 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f0fb00a2c80 msgr2=0x7f0fb00a5140 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-11T23:18:21.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.022+0000 7f0fcb80e640 1 --2- 172.21.15.26:0/2507209398 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f0fb00a2c80 0x7f0fb00a5140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:21.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.022+0000 7f0fcb80e640 1 -- 172.21.15.26:0/2507209398 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0fc4100a40 msgr2=0x7f0fc4196cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:21.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.022+0000 7f0fcb80e640 1 --2- 172.21.15.26:0/2507209398 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0fc4100a40 0x7f0fc4196cb0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f0fc000ead0 tx=0x7f0fc000ef90 comp rx=0 tx=0).stop 2024-09-11T23:18:21.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.022+0000 7f0fcb80e640 1 -- 172.21.15.26:0/2507209398 shutdown_connections 2024-09-11T23:18:21.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.022+0000 7f0fcb80e640 1 --2- 172.21.15.26:0/2507209398 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f0fb00a2c80 0x7f0fb00a5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:21.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.022+0000 7f0fcb80e640 1 --2- 172.21.15.26:0/2507209398 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0fc41025f0 0x7f0fc419e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:21.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.022+0000 7f0fcb80e640 1 --2- 172.21.15.26:0/2507209398 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0fc4101c30 0x7f0fc41971f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:21.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.022+0000 7f0fcb80e640 1 --2- 172.21.15.26:0/2507209398 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0fc4100a40 0x7f0fc4196cb0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:21.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.022+0000 7f0fcb80e640 1 -- 172.21.15.26:0/2507209398 >> 172.21.15.26:0/2507209398 conn(0x7f0fc40fc1e0 msgr2=0x7f0fc4104f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:18:21.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.022+0000 7f0fcb80e640 1 -- 172.21.15.26:0/2507209398 shutdown_connections 2024-09-11T23:18:21.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.022+0000 7f0fcb80e640 1 -- 172.21.15.26:0/2507209398 wait complete. 2024-09-11T23:18:21.035 INFO:teuthology.run_tasks:Running task check-counter... 2024-09-11T23:18:21.051 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-09-11T23:18:21.066 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-11T23:18:21.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.180+0000 7fd834523640 1 -- 172.21.15.26:0/1487910849 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd82c100a10 msgr2=0x7fd82c100e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:21.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.180+0000 7fd834523640 1 --2- 172.21.15.26:0/1487910849 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd82c100a10 0x7fd82c100e10 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fd828009900 tx=0x7fd82802a410 comp rx=0 tx=0).stop 2024-09-11T23:18:21.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.181+0000 7fd834523640 1 -- 172.21.15.26:0/1487910849 shutdown_connections 2024-09-11T23:18:21.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.181+0000 7fd834523640 1 --2- 172.21.15.26:0/1487910849 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd82c1025c0 0x7fd82c108e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:21.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.181+0000 7fd834523640 1 --2- 172.21.15.26:0/1487910849 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd82c101c00 0x7fd82c102080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:21.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.181+0000 7fd834523640 1 --2- 172.21.15.26:0/1487910849 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd82c100a10 0x7fd82c100e10 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:21.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.181+0000 7fd834523640 1 -- 172.21.15.26:0/1487910849 >> 172.21.15.26:0/1487910849 conn(0x7fd82c0fc170 msgr2=0x7fd82c0fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:18:21.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.181+0000 7fd834523640 1 -- 172.21.15.26:0/1487910849 shutdown_connections 2024-09-11T23:18:21.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.181+0000 7fd834523640 1 -- 172.21.15.26:0/1487910849 wait complete. 2024-09-11T23:18:21.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.181+0000 7fd834523640 1 Processor -- start 2024-09-11T23:18:21.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.182+0000 7fd834523640 1 -- start start 2024-09-11T23:18:21.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.182+0000 7fd834523640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd82c100a10 0x7fd82c197e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:21.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.182+0000 7fd834523640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd82c101c00 0x7fd82c1983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:21.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.182+0000 7fd834523640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd82c1025c0 0x7fd82c19f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:21.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.182+0000 7fd834523640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd82c10d7e0 con 0x7fd82c101c00 2024-09-11T23:18:21.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.182+0000 7fd834523640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fd82c10d660 con 0x7fd82c100a10 2024-09-11T23:18:21.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.182+0000 7fd834523640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd82c10d960 con 0x7fd82c1025c0 2024-09-11T23:18:21.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.182+0000 7fd8322bf640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd82c100a10 0x7fd82c197e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:21.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.182+0000 7fd831abe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd82c101c00 0x7fd82c1983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:21.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.182+0000 7fd8322bf640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd82c100a10 0x7fd82c197e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:60018/0 (socket says 172.21.15.26:60018) 2024-09-11T23:18:21.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.182+0000 7fd8322bf640 1 -- 172.21.15.26:0/2029199807 learned_addr learned my addr 172.21.15.26:0/2029199807 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:18:21.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.182+0000 7fd832ac0640 1 --2- 172.21.15.26:0/2029199807 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd82c1025c0 0x7fd82c19f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:21.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.182+0000 7fd831abe640 1 -- 172.21.15.26:0/2029199807 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd82c1025c0 msgr2=0x7fd82c19f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:21.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.182+0000 7fd831abe640 1 --2- 172.21.15.26:0/2029199807 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd82c1025c0 0x7fd82c19f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:21.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.182+0000 7fd831abe640 1 -- 172.21.15.26:0/2029199807 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd82c100a10 msgr2=0x7fd82c197e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:21.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.182+0000 7fd831abe640 1 --2- 172.21.15.26:0/2029199807 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd82c100a10 0x7fd82c197e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:21.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.182+0000 7fd831abe640 1 -- 172.21.15.26:0/2029199807 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd82c19faa0 con 0x7fd82c101c00 2024-09-11T23:18:21.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.182+0000 7fd831abe640 1 --2- 172.21.15.26:0/2029199807 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd82c101c00 0x7fd82c1983c0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fd81c00e9f0 tx=0x7fd81c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:18:21.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.183+0000 7fd8237fe640 1 -- 172.21.15.26:0/2029199807 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd81c00ccc0 con 0x7fd82c101c00 2024-09-11T23:18:21.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.183+0000 7fd8237fe640 1 -- 172.21.15.26:0/2029199807 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd81c005020 con 0x7fd82c101c00 2024-09-11T23:18:21.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.183+0000 7fd834523640 1 -- 172.21.15.26:0/2029199807 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd82c109870 con 0x7fd82c101c00 2024-09-11T23:18:21.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.183+0000 7fd8237fe640 1 -- 172.21.15.26:0/2029199807 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd81c0049e0 con 0x7fd82c101c00 2024-09-11T23:18:21.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.183+0000 7fd834523640 1 -- 172.21.15.26:0/2029199807 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd82c109d70 con 0x7fd82c101c00 2024-09-11T23:18:21.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.184+0000 7fd834523640 1 -- 172.21.15.26:0/2029199807 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd82c107ea0 con 0x7fd82c101c00 2024-09-11T23:18:21.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.185+0000 7fd8237fe640 1 -- 172.21.15.26:0/2029199807 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 19) ==== 137850+0+0 (secure 0 0 0) 0x7fd81c005560 con 0x7fd82c101c00 2024-09-11T23:18:21.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.186+0000 7fd8237fe640 1 --2- 172.21.15.26:0/2029199807 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fd8180a2be0 0x7fd8180a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:21.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.186+0000 7fd8322bf640 1 -- 172.21.15.26:0/2029199807 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fd8180a2be0 msgr2=0x7fd8180a50a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.26:6800/110833859 2024-09-11T23:18:21.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.186+0000 7fd8237fe640 1 -- 172.21.15.26:0/2029199807 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fd81c0cf0c0 con 0x7fd82c101c00 2024-09-11T23:18:21.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.186+0000 7fd8322bf640 1 --2- 172.21.15.26:0/2029199807 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fd8180a2be0 0x7fd8180a50a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-11T23:18:21.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.189+0000 7fd8237fe640 1 -- 172.21.15.26:0/2029199807 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd81c0051d0 con 0x7fd82c101c00 2024-09-11T23:18:21.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:21 smithi136 ceph-mon[35121]: from='client.? 172.21.15.26:0/2507209398' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-09-11T23:18:21.239 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Sep 11 23:18:20 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-z[36925]: 2024-09-11T23:18:20.942+0000 7f5717a18040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-11T23:18:21.241 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:21 smithi026 ceph-mon[31172]: from='client.? 172.21.15.26:0/2507209398' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-09-11T23:18:21.242 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:18:21 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: 2024-09-11T23:18:21.010+0000 7f7d35c1b040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-11T23:18:21.242 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:18:21 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: 2024-09-11T23:18:21.083+0000 7f7d35c1b040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-11T23:18:21.242 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:18:21 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: 2024-09-11T23:18:21.174+0000 7f7d35c1b040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-11T23:18:21.242 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:18:21 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: 2024-09-11T23:18:21.240+0000 7f7d35c1b040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-11T23:18:21.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:21 smithi076 ceph-mon[35296]: from='client.? 172.21.15.26:0/2507209398' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-09-11T23:18:21.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.367+0000 7fd834523640 1 -- 172.21.15.26:0/2029199807 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd82c02da10 con 0x7fd82c101c00 2024-09-11T23:18:21.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.369+0000 7fd8237fe640 1 -- 172.21.15.26:0/2029199807 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+22733 (secure 0 0 0) 0x7fd81c09c0b0 con 0x7fd82c101c00 2024-09-11T23:18:21.370 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-11T23:18:21.370 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":85,"fsid":"af7e6fc8-7092-11ef-bcea-c7b262605968","created":"2024-09-11T23:08:40.486438+0000","modified":"2024-09-11T23:16:35.935151+0000","last_up_change":"2024-09-11T23:15:35.179446+0000","last_in_change":"2024-09-11T23:15:11.064627+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T23:11:50.494160+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-11T23:16:22.503928+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-11T23:16:24.876554+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a4d62fcf-c31e-41b1-932c-3d87ff640531","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":2014857905},{"type":"v1","addr":"172.21.15.26:6803","nonce":2014857905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":2014857905},{"type":"v1","addr":"172.21.15.26:6805","nonce":2014857905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":2014857905},{"type":"v1","addr":"172.21.15.26:6809","nonce":2014857905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":2014857905},{"type":"v1","addr":"172.21.15.26:6807","nonce":2014857905}]},"public_addr":"172.21.15.26:6803/2014857905","cluster_addr":"172.21.15.26:6805/2014857905","heartbeat_back_addr":"172.21.15.26:6809/2014857905","heartbeat_front_addr":"172.21.15.26:6807/2014857905","state":["exists","up"]},{"osd":1,"uuid":"e0d69112-3c0a-4fc5-8783-e41b4081e0c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":333383087},{"type":"v1","addr":"172.21.15.76:6801","nonce":333383087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":333383087},{"type":"v1","addr":"172.21.15.76:6803","nonce":333383087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":333383087},{"type":"v1","addr":"172.21.15.76:6807","nonce":333383087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":333383087},{"type":"v1","addr":"172.21.15.76:6805","nonce":333383087}]},"public_addr":"172.21.15.76:6801/333383087","cluster_addr":"172.21.15.76:6803/333383087","heartbeat_back_addr":"172.21.15.76:6807/333383087","heartbeat_front_addr":"172.21.15.76:6805/333383087","state":["exists","up"]},{"osd":2,"uuid":"d578429e-bf23-4a1e-890f-7172955c0eb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":1271538911},{"type":"v1","addr":"172.21.15.136:6801","nonce":1271538911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":1271538911},{"type":"v1","addr":"172.21.15.136:6803","nonce":1271538911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":1271538911},{"type":"v1","addr":"172.21.15.136:6807","nonce":1271538911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":1271538911},{"type":"v1","addr":"172.21.15.136:6805","nonce":1271538911}]},"public_addr":"172.21.15.136:6801/1271538911","cluster_addr":"172.21.15.136:6803/1271538911","heartbeat_back_addr":"172.21.15.136:6807/1271538911","heartbeat_front_addr":"172.21.15.136:6805/1271538911","state":["exists","up"]},{"osd":3,"uuid":"3e34623f-0ea5-4b10-b8a9-f8756c35d512","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":1740304151},{"type":"v1","addr":"172.21.15.26:6811","nonce":1740304151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":1740304151},{"type":"v1","addr":"172.21.15.26:6813","nonce":1740304151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1740304151},{"type":"v1","addr":"172.21.15.26:6817","nonce":1740304151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":1740304151},{"type":"v1","addr":"172.21.15.26:6815","nonce":1740304151}]},"public_addr":"172.21.15.26:6811/1740304151","cluster_addr":"172.21.15.26:6813/1740304151","heartbeat_back_addr":"172.21.15.26:6817/1740304151","heartbeat_front_addr":"172.21.15.26:6815/1740304151","state":["exists","up"]},{"osd":4,"uuid":"77607929-f9ed-49a9-abb5-48cd88eec4ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":2702262182},{"type":"v1","addr":"172.21.15.76:6809","nonce":2702262182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":2702262182},{"type":"v1","addr":"172.21.15.76:6811","nonce":2702262182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":2702262182},{"type":"v1","addr":"172.21.15.76:6815","nonce":2702262182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":2702262182},{"type":"v1","addr":"172.21.15.76:6813","nonce":2702262182}]},"public_addr":"172.21.15.76:6809/2702262182","cluster_addr":"172.21.15.76:6811/2702262182","heartbeat_back_addr":"172.21.15.76:6815/2702262182","heartbeat_front_addr":"172.21.15.76:6813/2702262182","state":["exists","up"]},{"osd":5,"uuid":"29e20e37-09a0-4924-8aaa-17939763a7c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":1749229277},{"type":"v1","addr":"172.21.15.136:6809","nonce":1749229277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":1749229277},{"type":"v1","addr":"172.21.15.136:6811","nonce":1749229277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":1749229277},{"type":"v1","addr":"172.21.15.136:6815","nonce":1749229277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":1749229277},{"type":"v1","addr":"172.21.15.136:6813","nonce":1749229277}]},"public_addr":"172.21.15.136:6809/1749229277","cluster_addr":"172.21.15.136:6811/1749229277","heartbeat_back_addr":"172.21.15.136:6815/1749229277","heartbeat_front_addr":"172.21.15.136:6813/1749229277","state":["exists","up"]},{"osd":6,"uuid":"e0ddd1c0-1c46-41e6-b2f4-396bdcd2bf8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1928245624},{"type":"v1","addr":"172.21.15.26:6819","nonce":1928245624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1928245624},{"type":"v1","addr":"172.21.15.26:6821","nonce":1928245624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":1928245624},{"type":"v1","addr":"172.21.15.26:6825","nonce":1928245624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1928245624},{"type":"v1","addr":"172.21.15.26:6823","nonce":1928245624}]},"public_addr":"172.21.15.26:6819/1928245624","cluster_addr":"172.21.15.26:6821/1928245624","heartbeat_back_addr":"172.21.15.26:6825/1928245624","heartbeat_front_addr":"172.21.15.26:6823/1928245624","state":["exists","up"]},{"osd":7,"uuid":"fe031f5c-d915-48fb-91a6-c76f252467fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":1409789455},{"type":"v1","addr":"172.21.15.76:6817","nonce":1409789455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":1409789455},{"type":"v1","addr":"172.21.15.76:6819","nonce":1409789455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":1409789455},{"type":"v1","addr":"172.21.15.76:6823","nonce":1409789455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":1409789455},{"type":"v1","addr":"172.21.15.76:6821","nonce":1409789455}]},"public_addr":"172.21.15.76:6817/1409789455","cluster_addr":"172.21.15.76:6819/1409789455","heartbeat_back_addr":"172.21.15.76:6823/1409789455","heartbeat_front_addr":"172.21.15.76:6821/1409789455","state":["exists","up"]},{"osd":8,"uuid":"18851bde-9e6c-4e05-b4bf-ac5c3073eee7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":543619079},{"type":"v1","addr":"172.21.15.136:6817","nonce":543619079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":543619079},{"type":"v1","addr":"172.21.15.136:6819","nonce":543619079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":543619079},{"type":"v1","addr":"172.21.15.136:6823","nonce":543619079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":543619079},{"type":"v1","addr":"172.21.15.136:6821","nonce":543619079}]},"public_addr":"172.21.15.136:6817/543619079","cluster_addr":"172.21.15.136:6819/543619079","heartbeat_back_addr":"172.21.15.136:6823/543619079","heartbeat_front_addr":"172.21.15.136:6821/543619079","state":["exists","up"]},{"osd":9,"uuid":"52a65eab-8446-47ad-bfe7-5133f7e3eb98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":2201297064},{"type":"v1","addr":"172.21.15.26:6827","nonce":2201297064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":2201297064},{"type":"v1","addr":"172.21.15.26:6829","nonce":2201297064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":2201297064},{"type":"v1","addr":"172.21.15.26:6833","nonce":2201297064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":2201297064},{"type":"v1","addr":"172.21.15.26:6831","nonce":2201297064}]},"public_addr":"172.21.15.26:6827/2201297064","cluster_addr":"172.21.15.26:6829/2201297064","heartbeat_back_addr":"172.21.15.26:6833/2201297064","heartbeat_front_addr":"172.21.15.26:6831/2201297064","state":["exists","up"]},{"osd":10,"uuid":"b758d4bb-da62-4c5f-a992-bb6a145d35d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":3184128096},{"type":"v1","addr":"172.21.15.76:6825","nonce":3184128096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":3184128096},{"type":"v1","addr":"172.21.15.76:6827","nonce":3184128096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":3184128096},{"type":"v1","addr":"172.21.15.76:6831","nonce":3184128096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":3184128096},{"type":"v1","addr":"172.21.15.76:6829","nonce":3184128096}]},"public_addr":"172.21.15.76:6825/3184128096","cluster_addr":"172.21.15.76:6827/3184128096","heartbeat_back_addr":"172.21.15.76:6831/3184128096","heartbeat_front_addr":"172.21.15.76:6829/3184128096","state":["exists","up"]},{"osd":11,"uuid":"8fddda46-e22b-4d0c-ac48-e038fb29dfec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":1960689685},{"type":"v1","addr":"172.21.15.136:6825","nonce":1960689685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":1960689685},{"type":"v1","addr":"172.21.15.136:6827","nonce":1960689685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":1960689685},{"type":"v1","addr":"172.21.15.136:6831","nonce":1960689685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":1960689685},{"type":"v1","addr":"172.21.15.136:6829","nonce":1960689685}]},"public_addr":"172.21.15.136:6825/1960689685","cluster_addr":"172.21.15.136:6827/1960689685","heartbeat_back_addr":"172.21.15.136:6831/1960689685","heartbeat_front_addr":"172.21.15.136:6829/1960689685","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:10:56.656177+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:11:22.234031+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:11:46.464102+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:12:12.353967+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:12:37.836200+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:13:01.728839+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:13:27.527230+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:13:51.981393+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:14:17.035464+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:14:42.805029+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:15:08.208999+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:15:32.800844+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.18","mappings":[{"from":3,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":6}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.26:6801/677148151":"2024-09-12T23:09:33.983310+0000","172.21.15.26:6800/677148151":"2024-09-12T23:09:33.983310+0000","172.21.15.26:0/3597996003":"2024-09-12T23:09:33.983310+0000","172.21.15.26:6801/2673575521":"2024-09-12T23:09:09.781665+0000","172.21.15.26:0/883212708":"2024-09-12T23:09:33.983310+0000","172.21.15.26:0/2107281413":"2024-09-12T23:09:33.983310+0000","172.21.15.26:6800/2673575521":"2024-09-12T23:09:09.781665+0000","172.21.15.26:0/2817092653":"2024-09-12T23:09:09.781665+0000","172.21.15.26:0/17142658":"2024-09-12T23:09:09.781665+0000","172.21.15.26:0/3861103130":"2024-09-12T23:09:09.781665+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T23:18:21.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.371+0000 7fd834523640 1 -- 172.21.15.26:0/2029199807 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fd8180a2be0 msgr2=0x7fd8180a50a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-11T23:18:21.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.371+0000 7fd834523640 1 --2- 172.21.15.26:0/2029199807 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fd8180a2be0 0x7fd8180a50a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:21.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.371+0000 7fd834523640 1 -- 172.21.15.26:0/2029199807 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd82c101c00 msgr2=0x7fd82c1983c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:21.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.371+0000 7fd834523640 1 --2- 172.21.15.26:0/2029199807 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd82c101c00 0x7fd82c1983c0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fd81c00e9f0 tx=0x7fd81c00eeb0 comp rx=0 tx=0).stop 2024-09-11T23:18:21.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.371+0000 7fd834523640 1 -- 172.21.15.26:0/2029199807 shutdown_connections 2024-09-11T23:18:21.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.371+0000 7fd834523640 1 --2- 172.21.15.26:0/2029199807 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fd8180a2be0 0x7fd8180a50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:21.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.371+0000 7fd834523640 1 --2- 172.21.15.26:0/2029199807 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd82c1025c0 0x7fd82c19f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:21.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.371+0000 7fd834523640 1 --2- 172.21.15.26:0/2029199807 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd82c101c00 0x7fd82c1983c0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:21.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.371+0000 7fd834523640 1 --2- 172.21.15.26:0/2029199807 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd82c100a10 0x7fd82c197e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:21.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.372+0000 7fd834523640 1 -- 172.21.15.26:0/2029199807 >> 172.21.15.26:0/2029199807 conn(0x7fd82c0fc170 msgr2=0x7fd82c105070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:18:21.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.372+0000 7fd834523640 1 -- 172.21.15.26:0/2029199807 shutdown_connections 2024-09-11T23:18:21.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.372+0000 7fd834523640 1 -- 172.21.15.26:0/2029199807 wait complete. 2024-09-11T23:18:21.384 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-11T23:11:50.494160+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-11T23:16:22.503928+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-11T23:16:24.876554+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-11T23:18:21.384 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-11T23:18:21.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.492+0000 7f8811235640 1 -- 172.21.15.26:0/2323827188 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f880c104eb0 msgr2=0x7f880c1072a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:21.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.492+0000 7f8811235640 1 --2- 172.21.15.26:0/2323827188 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f880c104eb0 0x7f880c1072a0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f8800009900 tx=0x7f880002a410 comp rx=0 tx=0).stop 2024-09-11T23:18:21.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.492+0000 7f8811235640 1 -- 172.21.15.26:0/2323827188 shutdown_connections 2024-09-11T23:18:21.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.492+0000 7f8811235640 1 --2- 172.21.15.26:0/2323827188 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f880c1077e0 0x7f880c109c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:21.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.492+0000 7f8811235640 1 --2- 172.21.15.26:0/2323827188 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f880c104eb0 0x7f880c1072a0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:21.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.492+0000 7f8811235640 1 --2- 172.21.15.26:0/2323827188 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f880c102580 0x7f880c104970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:21.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.492+0000 7f8811235640 1 -- 172.21.15.26:0/2323827188 >> 172.21.15.26:0/2323827188 conn(0x7f880c0fc140 msgr2=0x7f880c0fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:18:21.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.492+0000 7f8811235640 1 -- 172.21.15.26:0/2323827188 shutdown_connections 2024-09-11T23:18:21.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.492+0000 7f8811235640 1 -- 172.21.15.26:0/2323827188 wait complete. 2024-09-11T23:18:21.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.492+0000 7f8811235640 1 Processor -- start 2024-09-11T23:18:21.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.493+0000 7f8811235640 1 -- start start 2024-09-11T23:18:21.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.493+0000 7f8811235640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f880c102580 0x7f880c197f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:21.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.493+0000 7f8811235640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f880c104eb0 0x7f880c198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:21.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.493+0000 7f8811235640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f880c1077e0 0x7f880c19f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:21.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.493+0000 7f8811235640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f880c10d790 con 0x7f880c104eb0 2024-09-11T23:18:21.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.493+0000 7f8811235640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f880c10d610 con 0x7f880c102580 2024-09-11T23:18:21.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.493+0000 7f8811235640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f880c10d910 con 0x7f880c1077e0 2024-09-11T23:18:21.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.493+0000 7f880a59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f880c104eb0 0x7f880c198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:21.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.493+0000 7f880ad9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f880c102580 0x7f880c197f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:21.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.493+0000 7f880a59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f880c104eb0 0x7f880c198440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47146/0 (socket says 172.21.15.26:47146) 2024-09-11T23:18:21.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.493+0000 7f880a59c640 1 -- 172.21.15.26:0/1723698173 learned_addr learned my addr 172.21.15.26:0/1723698173 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:18:21.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.493+0000 7f880b59e640 1 --2- 172.21.15.26:0/1723698173 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f880c1077e0 0x7f880c19f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:21.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.493+0000 7f880a59c640 1 -- 172.21.15.26:0/1723698173 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f880c1077e0 msgr2=0x7f880c19f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:21.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.493+0000 7f880a59c640 1 --2- 172.21.15.26:0/1723698173 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f880c1077e0 0x7f880c19f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:21.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.493+0000 7f880a59c640 1 -- 172.21.15.26:0/1723698173 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f880c102580 msgr2=0x7f880c197f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:21.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.493+0000 7f880a59c640 1 --2- 172.21.15.26:0/1723698173 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f880c102580 0x7f880c197f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:21.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.493+0000 7f880a59c640 1 -- 172.21.15.26:0/1723698173 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f880c19fb20 con 0x7f880c104eb0 2024-09-11T23:18:21.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.493+0000 7f880ad9d640 1 --2- 172.21.15.26:0/1723698173 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f880c102580 0x7f880c197f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:18:21.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.493+0000 7f880b59e640 1 --2- 172.21.15.26:0/1723698173 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f880c1077e0 0x7f880c19f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:18:21.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.494+0000 7f880a59c640 1 --2- 172.21.15.26:0/1723698173 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f880c104eb0 0x7f880c198440 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f8800031610 tx=0x7f8800004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:18:21.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.494+0000 7f87f3fff640 1 -- 172.21.15.26:0/1723698173 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f880003f030 con 0x7f880c104eb0 2024-09-11T23:18:21.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.494+0000 7f8811235640 1 -- 172.21.15.26:0/1723698173 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f880c19fdb0 con 0x7f880c104eb0 2024-09-11T23:18:21.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.494+0000 7f87f3fff640 1 -- 172.21.15.26:0/1723698173 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8800003ad0 con 0x7f880c104eb0 2024-09-11T23:18:21.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.494+0000 7f87f3fff640 1 -- 172.21.15.26:0/1723698173 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8800035070 con 0x7f880c104eb0 2024-09-11T23:18:21.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.494+0000 7f8811235640 1 -- 172.21.15.26:0/1723698173 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f880c1a0210 con 0x7f880c104eb0 2024-09-11T23:18:21.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.495+0000 7f8811235640 1 -- 172.21.15.26:0/1723698173 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f880c0fdc30 con 0x7f880c104eb0 2024-09-11T23:18:21.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.496+0000 7f87f3fff640 1 -- 172.21.15.26:0/1723698173 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 19) ==== 137850+0+0 (secure 0 0 0) 0x7f8800042030 con 0x7f880c104eb0 2024-09-11T23:18:21.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.497+0000 7f87f3fff640 1 --2- 172.21.15.26:0/1723698173 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f87f40a2be0 0x7f87f40a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:21.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.497+0000 7f880ad9d640 1 -- 172.21.15.26:0/1723698173 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f87f40a2be0 msgr2=0x7f87f40a50a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.26:6800/110833859 2024-09-11T23:18:21.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.497+0000 7f87f3fff640 1 -- 172.21.15.26:0/1723698173 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f88000ef620 con 0x7f880c104eb0 2024-09-11T23:18:21.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.497+0000 7f880ad9d640 1 --2- 172.21.15.26:0/1723698173 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f87f40a2be0 0x7f87f40a50a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-11T23:18:21.500 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:18:21 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: 2024-09-11T23:18:21.305+0000 7f7d35c1b040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-11T23:18:21.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.501+0000 7f87f3fff640 1 -- 172.21.15.26:0/1723698173 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f88000bbf30 con 0x7f880c104eb0 2024-09-11T23:18:21.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.677+0000 7f8811235640 1 -- 172.21.15.26:0/1723698173 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f880c062ef0 con 0x7f880c104eb0 2024-09-11T23:18:21.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.677+0000 7f87f3fff640 1 -- 172.21.15.26:0/1723698173 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v85) ==== 93+0+10 (secure 0 0 0) 0x7f88000c0230 con 0x7f880c104eb0 2024-09-11T23:18:21.678 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-09-11T23:18:21.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.679+0000 7f8811235640 1 -- 172.21.15.26:0/1723698173 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f87f40a2be0 msgr2=0x7f87f40a50a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-11T23:18:21.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.679+0000 7f8811235640 1 --2- 172.21.15.26:0/1723698173 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f87f40a2be0 0x7f87f40a50a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:21.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.679+0000 7f8811235640 1 -- 172.21.15.26:0/1723698173 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f880c104eb0 msgr2=0x7f880c198440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:21.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.679+0000 7f8811235640 1 --2- 172.21.15.26:0/1723698173 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f880c104eb0 0x7f880c198440 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f8800031610 tx=0x7f8800004270 comp rx=0 tx=0).stop 2024-09-11T23:18:21.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.679+0000 7f8811235640 1 -- 172.21.15.26:0/1723698173 shutdown_connections 2024-09-11T23:18:21.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.679+0000 7f8811235640 1 --2- 172.21.15.26:0/1723698173 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f87f40a2be0 0x7f87f40a50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:21.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.679+0000 7f8811235640 1 --2- 172.21.15.26:0/1723698173 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f880c1077e0 0x7f880c19f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:21.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.679+0000 7f8811235640 1 --2- 172.21.15.26:0/1723698173 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f880c104eb0 0x7f880c198440 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:21.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.679+0000 7f8811235640 1 --2- 172.21.15.26:0/1723698173 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f880c102580 0x7f880c197f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:21.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.679+0000 7f8811235640 1 -- 172.21.15.26:0/1723698173 >> 172.21.15.26:0/1723698173 conn(0x7f880c0fc140 msgr2=0x7f880c102e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:18:21.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.679+0000 7f8811235640 1 -- 172.21.15.26:0/1723698173 shutdown_connections 2024-09-11T23:18:21.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.679+0000 7f8811235640 1 -- 172.21.15.26:0/1723698173 wait complete. 2024-09-11T23:18:21.691 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-11T23:18:21.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.798+0000 7fcc88be4640 1 -- 172.21.15.26:0/4043636165 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc84104eb0 msgr2=0x7fcc841072a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:21.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.798+0000 7fcc88be4640 1 --2- 172.21.15.26:0/4043636165 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc84104eb0 0x7fcc841072a0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fcc78009900 tx=0x7fcc7802a410 comp rx=0 tx=0).stop 2024-09-11T23:18:21.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.798+0000 7fcc88be4640 1 -- 172.21.15.26:0/4043636165 shutdown_connections 2024-09-11T23:18:21.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.798+0000 7fcc88be4640 1 --2- 172.21.15.26:0/4043636165 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcc841077e0 0x7fcc84109c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:21.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.798+0000 7fcc88be4640 1 --2- 172.21.15.26:0/4043636165 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc84104eb0 0x7fcc841072a0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:21.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.798+0000 7fcc88be4640 1 --2- 172.21.15.26:0/4043636165 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcc84102580 0x7fcc84104970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:21.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.798+0000 7fcc88be4640 1 -- 172.21.15.26:0/4043636165 >> 172.21.15.26:0/4043636165 conn(0x7fcc840fc140 msgr2=0x7fcc840fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:18:21.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.798+0000 7fcc88be4640 1 -- 172.21.15.26:0/4043636165 shutdown_connections 2024-09-11T23:18:21.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.798+0000 7fcc88be4640 1 -- 172.21.15.26:0/4043636165 wait complete. 2024-09-11T23:18:21.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.799+0000 7fcc88be4640 1 Processor -- start 2024-09-11T23:18:21.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.799+0000 7fcc88be4640 1 -- start start 2024-09-11T23:18:21.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.799+0000 7fcc88be4640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcc84102580 0x7fcc84197ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:21.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.799+0000 7fcc88be4640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcc84104eb0 0x7fcc84198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:21.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.799+0000 7fcc88be4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc841077e0 0x7fcc8419f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:21.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.799+0000 7fcc88be4640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fcc8410d7e0 con 0x7fcc841077e0 2024-09-11T23:18:21.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.799+0000 7fcc88be4640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fcc8410d660 con 0x7fcc84102580 2024-09-11T23:18:21.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.799+0000 7fcc88be4640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fcc8410d960 con 0x7fcc84104eb0 2024-09-11T23:18:21.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.800+0000 7fcc82d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc841077e0 0x7fcc8419f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:21.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.800+0000 7fcc82d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc841077e0 0x7fcc8419f4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47170/0 (socket says 172.21.15.26:47170) 2024-09-11T23:18:21.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.800+0000 7fcc82d9d640 1 -- 172.21.15.26:0/2731367814 learned_addr learned my addr 172.21.15.26:0/2731367814 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:18:21.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.800+0000 7fcc81d9b640 1 --2- 172.21.15.26:0/2731367814 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcc84104eb0 0x7fcc84198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:21.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.800+0000 7fcc8259c640 1 --2- 172.21.15.26:0/2731367814 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcc84102580 0x7fcc84197ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:21.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.800+0000 7fcc82d9d640 1 -- 172.21.15.26:0/2731367814 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcc84104eb0 msgr2=0x7fcc84198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:21.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.800+0000 7fcc82d9d640 1 --2- 172.21.15.26:0/2731367814 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcc84104eb0 0x7fcc84198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:21.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.800+0000 7fcc82d9d640 1 -- 172.21.15.26:0/2731367814 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcc84102580 msgr2=0x7fcc84197ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:21.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.800+0000 7fcc82d9d640 1 --2- 172.21.15.26:0/2731367814 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcc84102580 0x7fcc84197ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:21.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.800+0000 7fcc82d9d640 1 -- 172.21.15.26:0/2731367814 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc8419fb10 con 0x7fcc841077e0 2024-09-11T23:18:21.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.800+0000 7fcc81d9b640 1 --2- 172.21.15.26:0/2731367814 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcc84104eb0 0x7fcc84198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:18:21.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.800+0000 7fcc82d9d640 1 --2- 172.21.15.26:0/2731367814 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc841077e0 0x7fcc8419f4a0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fcc7c00eb40 tx=0x7fcc7c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:18:21.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.800+0000 7fcc737fe640 1 -- 172.21.15.26:0/2731367814 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc7c004b20 con 0x7fcc841077e0 2024-09-11T23:18:21.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.800+0000 7fcc737fe640 1 -- 172.21.15.26:0/2731367814 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc7c004cc0 con 0x7fcc841077e0 2024-09-11T23:18:21.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.800+0000 7fcc88be4640 1 -- 172.21.15.26:0/2731367814 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc8419fe00 con 0x7fcc841077e0 2024-09-11T23:18:21.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.800+0000 7fcc737fe640 1 -- 172.21.15.26:0/2731367814 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc7c010850 con 0x7fcc841077e0 2024-09-11T23:18:21.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.800+0000 7fcc88be4640 1 -- 172.21.15.26:0/2731367814 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc841a0270 con 0x7fcc841077e0 2024-09-11T23:18:21.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.802+0000 7fcc88be4640 1 -- 172.21.15.26:0/2731367814 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc840fdc30 con 0x7fcc841077e0 2024-09-11T23:18:21.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.802+0000 7fcc737fe640 1 -- 172.21.15.26:0/2731367814 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 19) ==== 137850+0+0 (secure 0 0 0) 0x7fcc7c005020 con 0x7fcc841077e0 2024-09-11T23:18:21.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.803+0000 7fcc737fe640 1 --2- 172.21.15.26:0/2731367814 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fcc6c0a2be0 0x7fcc6c0a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:21.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.803+0000 7fcc8259c640 1 -- 172.21.15.26:0/2731367814 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fcc6c0a2be0 msgr2=0x7fcc6c0a50a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.26:6800/110833859 2024-09-11T23:18:21.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.803+0000 7fcc737fe640 1 -- 172.21.15.26:0/2731367814 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fcc7c0cf900 con 0x7fcc841077e0 2024-09-11T23:18:21.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.803+0000 7fcc8259c640 1 --2- 172.21.15.26:0/2731367814 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fcc6c0a2be0 0x7fcc6c0a50a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-11T23:18:21.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.807+0000 7fcc737fe640 1 -- 172.21.15.26:0/2731367814 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fcc7c09c860 con 0x7fcc841077e0 2024-09-11T23:18:21.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.985+0000 7fcc88be4640 1 -- 172.21.15.26:0/2731367814 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fcc84062df0 con 0x7fcc841077e0 2024-09-11T23:18:21.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.985+0000 7fcc737fe640 1 -- 172.21.15.26:0/2731367814 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v85) ==== 104+0+11 (secure 0 0 0) 0x7fcc7c09c160 con 0x7fcc841077e0 2024-09-11T23:18:21.986 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2024-09-11T23:18:21.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.987+0000 7fcc88be4640 1 -- 172.21.15.26:0/2731367814 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fcc6c0a2be0 msgr2=0x7fcc6c0a50a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-11T23:18:21.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.987+0000 7fcc88be4640 1 --2- 172.21.15.26:0/2731367814 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fcc6c0a2be0 0x7fcc6c0a50a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:21.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.987+0000 7fcc88be4640 1 -- 172.21.15.26:0/2731367814 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc841077e0 msgr2=0x7fcc8419f4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:21.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.987+0000 7fcc88be4640 1 --2- 172.21.15.26:0/2731367814 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc841077e0 0x7fcc8419f4a0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fcc7c00eb40 tx=0x7fcc7c00c2d0 comp rx=0 tx=0).stop 2024-09-11T23:18:21.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.987+0000 7fcc88be4640 1 -- 172.21.15.26:0/2731367814 shutdown_connections 2024-09-11T23:18:21.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.987+0000 7fcc88be4640 1 --2- 172.21.15.26:0/2731367814 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7fcc6c0a2be0 0x7fcc6c0a50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:21.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.987+0000 7fcc88be4640 1 --2- 172.21.15.26:0/2731367814 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc841077e0 0x7fcc8419f4a0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:21.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.987+0000 7fcc88be4640 1 --2- 172.21.15.26:0/2731367814 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcc84104eb0 0x7fcc84198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:21.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.987+0000 7fcc88be4640 1 --2- 172.21.15.26:0/2731367814 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcc84102580 0x7fcc84197ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:21.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.988+0000 7fcc88be4640 1 -- 172.21.15.26:0/2731367814 >> 172.21.15.26:0/2731367814 conn(0x7fcc840fc140 msgr2=0x7fcc84102fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:18:21.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.988+0000 7fcc88be4640 1 -- 172.21.15.26:0/2731367814 shutdown_connections 2024-09-11T23:18:21.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:21.988+0000 7fcc88be4640 1 -- 172.21.15.26:0/2731367814 wait complete. 2024-09-11T23:18:22.000 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-11T23:18:22.026 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Sep 11 23:18:21 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-z[36925]: 2024-09-11T23:18:21.744+0000 7f5717a18040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-11T23:18:22.026 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Sep 11 23:18:21 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-y[36658]: 2024-09-11T23:18:21.689+0000 7eff6b6f4040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-11T23:18:22.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:22.111+0000 7f5af21fd640 1 -- 172.21.15.26:0/4108912961 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5aec102620 msgr2=0x7f5aec104a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:22.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:22.111+0000 7f5af21fd640 1 --2- 172.21.15.26:0/4108912961 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5aec102620 0x7f5aec104a10 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f5adc009900 tx=0x7f5adc02a410 comp rx=0 tx=0).stop 2024-09-11T23:18:22.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:22.111+0000 7f5af21fd640 1 -- 172.21.15.26:0/4108912961 shutdown_connections 2024-09-11T23:18:22.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:22.111+0000 7f5af21fd640 1 --2- 172.21.15.26:0/4108912961 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5aec107880 0x7f5aec109c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:22.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:22.111+0000 7f5af21fd640 1 --2- 172.21.15.26:0/4108912961 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5aec104f50 0x7f5aec107340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:22.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:22.111+0000 7f5af21fd640 1 --2- 172.21.15.26:0/4108912961 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5aec102620 0x7f5aec104a10 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:22.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:22.111+0000 7f5af21fd640 1 -- 172.21.15.26:0/4108912961 >> 172.21.15.26:0/4108912961 conn(0x7f5aec0fc140 msgr2=0x7f5aec0fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:18:22.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:22.111+0000 7f5af21fd640 1 -- 172.21.15.26:0/4108912961 shutdown_connections 2024-09-11T23:18:22.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:22.111+0000 7f5af21fd640 1 -- 172.21.15.26:0/4108912961 wait complete. 2024-09-11T23:18:22.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:22.111+0000 7f5af21fd640 1 Processor -- start 2024-09-11T23:18:22.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:22.112+0000 7f5af21fd640 1 -- start start 2024-09-11T23:18:22.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:22.112+0000 7f5af21fd640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5aec102620 0x7f5aec197f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:22.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:22.112+0000 7f5af21fd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5aec104f50 0x7f5aec198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:22.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:22.112+0000 7f5af21fd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5aec107880 0x7f5aec19f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:22.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:22.112+0000 7f5af21fd640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f5aec10d7c0 con 0x7f5aec107880 2024-09-11T23:18:22.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:22.112+0000 7f5af21fd640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f5aec10d640 con 0x7f5aec104f50 2024-09-11T23:18:22.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:22.112+0000 7f5af21fd640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f5aec10d940 con 0x7f5aec102620 2024-09-11T23:18:22.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:22.112+0000 7f5aebfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5aec107880 0x7f5aec19f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:22.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:22.112+0000 7f5aebfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5aec107880 0x7f5aec19f4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47186/0 (socket says 172.21.15.26:47186) 2024-09-11T23:18:22.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:22.112+0000 7f5aeaffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5aec104f50 0x7f5aec198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:22.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:22.112+0000 7f5aebfff640 1 -- 172.21.15.26:0/3126939036 learned_addr learned my addr 172.21.15.26:0/3126939036 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:18:22.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:22.112+0000 7f5aeb7fe640 1 --2- 172.21.15.26:0/3126939036 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5aec102620 0x7f5aec197f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:22.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:22.112+0000 7f5aeaffd640 1 -- 172.21.15.26:0/3126939036 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5aec102620 msgr2=0x7f5aec197f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:22.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:22.112+0000 7f5aeaffd640 1 --2- 172.21.15.26:0/3126939036 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5aec102620 0x7f5aec197f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:22.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:22.112+0000 7f5aeaffd640 1 -- 172.21.15.26:0/3126939036 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5aec107880 msgr2=0x7f5aec19f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:22.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:22.112+0000 7f5aeaffd640 1 --2- 172.21.15.26:0/3126939036 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5aec107880 0x7f5aec19f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:22.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:22.112+0000 7f5aeaffd640 1 -- 172.21.15.26:0/3126939036 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5aec19fb40 con 0x7f5aec104f50 2024-09-11T23:18:22.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:22.112+0000 7f5aebfff640 1 --2- 172.21.15.26:0/3126939036 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5aec107880 0x7f5aec19f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:18:22.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:22.112+0000 7f5aeaffd640 1 --2- 172.21.15.26:0/3126939036 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5aec104f50 0x7f5aec198460 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5ae000e9c0 tx=0x7f5ae000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:18:22.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:22.113+0000 7f5ae8ff9640 1 -- 172.21.15.26:0/3126939036 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ae000ccc0 con 0x7f5aec104f50 2024-09-11T23:18:22.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:22.113+0000 7f5ae8ff9640 1 -- 172.21.15.26:0/3126939036 <== mon.2 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ae0005020 con 0x7f5aec104f50 2024-09-11T23:18:22.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:22.113+0000 7f5af21fd640 1 -- 172.21.15.26:0/3126939036 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5aec19fe30 con 0x7f5aec104f50 2024-09-11T23:18:22.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:22.113+0000 7f5ae8ff9640 1 -- 172.21.15.26:0/3126939036 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ae0004a00 con 0x7f5aec104f50 2024-09-11T23:18:22.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:22.113+0000 7f5af21fd640 1 -- 172.21.15.26:0/3126939036 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5aec1a02a0 con 0x7f5aec104f50 2024-09-11T23:18:22.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:22.114+0000 7f5af21fd640 1 -- 172.21.15.26:0/3126939036 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5aec0fdc30 con 0x7f5aec104f50 2024-09-11T23:18:22.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:22.115+0000 7f5ae8ff9640 1 -- 172.21.15.26:0/3126939036 <== mon.2 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 20) ==== 137850+0+0 (secure 0 0 0) 0x7f5ae00051d0 con 0x7f5aec104f50 2024-09-11T23:18:22.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:22.115+0000 7f5ae8ff9640 1 --2- 172.21.15.26:0/3126939036 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f5ad40a2be0 0x7f5ad40a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:22.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:22.115+0000 7f5aeb7fe640 1 -- 172.21.15.26:0/3126939036 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f5ad40a2be0 msgr2=0x7f5ad40a50a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.26:6800/110833859 2024-09-11T23:18:22.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:22.115+0000 7f5ae8ff9640 1 -- 172.21.15.26:0/3126939036 <== mon.2 v2:172.21.15.76:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f5ae00cefd0 con 0x7f5aec104f50 2024-09-11T23:18:22.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:22.115+0000 7f5aeb7fe640 1 --2- 172.21.15.26:0/3126939036 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f5ad40a2be0 0x7f5ad40a50a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-11T23:18:22.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:22.120+0000 7f5ae8ff9640 1 -- 172.21.15.26:0/3126939036 <== mon.2 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5ae009bf30 con 0x7f5aec104f50 2024-09-11T23:18:22.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:22.191+0000 7f5ae8ff9640 1 -- 172.21.15.26:0/3126939036 <== mon.2 v2:172.21.15.76:3300/0 7 ==== mgrmap(e 21) ==== 137518+0+0 (secure 0 0 0) 0x7f5ae0097ea0 con 0x7f5aec104f50 2024-09-11T23:18:22.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:22.191+0000 7f5ae8ff9640 1 -- 172.21.15.26:0/3126939036 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f5ad40a2be0 msgr2=0x7f5ad40a50a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-11T23:18:22.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:22.191+0000 7f5ae8ff9640 1 --2- 172.21.15.26:0/3126939036 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f5ad40a2be0 0x7f5ad40a50a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:22.251 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:18:22 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: 2024-09-11T23:18:22.174+0000 7f7d35c1b040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-11T23:18:22.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:22 smithi026 ceph-mon[31172]: from='client.? 172.21.15.26:0/2507209398' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-09-11T23:18:22.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:22 smithi026 ceph-mon[31172]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:18:22.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:22 smithi026 ceph-mon[31172]: from='client.? 172.21.15.26:0/2029199807' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T23:18:22.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:22 smithi026 ceph-mon[31172]: from='client.? 172.21.15.26:0/1723698173' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-11T23:18:22.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:22 smithi026 ceph-mon[31172]: Standby manager daemon y restarted 2024-09-11T23:18:22.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:22 smithi026 ceph-mon[31172]: Standby manager daemon y started 2024-09-11T23:18:22.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:22 smithi026 ceph-mon[31172]: from='mgr.? 172.21.15.76:0/1726212331' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-11T23:18:22.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:22 smithi026 ceph-mon[31172]: from='mgr.? 172.21.15.76:0/1726212331' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-11T23:18:22.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:22 smithi026 ceph-mon[31172]: from='mgr.? 172.21.15.76:0/1726212331' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-11T23:18:22.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:22 smithi026 ceph-mon[31172]: from='mgr.? 172.21.15.76:0/1726212331' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-11T23:18:22.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:22 smithi026 ceph-mon[31172]: Standby manager daemon z restarted 2024-09-11T23:18:22.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:22 smithi026 ceph-mon[31172]: Standby manager daemon z started 2024-09-11T23:18:22.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:22 smithi026 ceph-mon[31172]: from='mgr.? 172.21.15.136:0/4131137347' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-11T23:18:22.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:22 smithi026 ceph-mon[31172]: from='mgr.? 172.21.15.136:0/4131137347' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-11T23:18:22.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:22 smithi026 ceph-mon[31172]: from='mgr.? 172.21.15.136:0/4131137347' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-11T23:18:22.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:22 smithi026 ceph-mon[31172]: from='mgr.? 172.21.15.136:0/4131137347' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-11T23:18:22.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:22 smithi026 ceph-mon[31172]: from='client.? 172.21.15.26:0/2731367814' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-11T23:18:22.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:22 smithi076 ceph-mon[35296]: from='client.? 172.21.15.26:0/2507209398' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-09-11T23:18:22.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:22 smithi076 ceph-mon[35296]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:18:22.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:22 smithi076 ceph-mon[35296]: from='client.? 172.21.15.26:0/2029199807' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T23:18:22.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:22 smithi076 ceph-mon[35296]: from='client.? 172.21.15.26:0/1723698173' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-11T23:18:22.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:22 smithi076 ceph-mon[35296]: Standby manager daemon y restarted 2024-09-11T23:18:22.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:22 smithi076 ceph-mon[35296]: Standby manager daemon y started 2024-09-11T23:18:22.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:22 smithi076 ceph-mon[35296]: from='mgr.? 172.21.15.76:0/1726212331' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-11T23:18:22.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:22 smithi076 ceph-mon[35296]: from='mgr.? 172.21.15.76:0/1726212331' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-11T23:18:22.304 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:22 smithi076 ceph-mon[35296]: from='mgr.? 172.21.15.76:0/1726212331' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-11T23:18:22.304 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:22 smithi076 ceph-mon[35296]: from='mgr.? 172.21.15.76:0/1726212331' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-11T23:18:22.304 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:22 smithi076 ceph-mon[35296]: Standby manager daemon z restarted 2024-09-11T23:18:22.304 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:22 smithi076 ceph-mon[35296]: Standby manager daemon z started 2024-09-11T23:18:22.304 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:22 smithi076 ceph-mon[35296]: from='mgr.? 172.21.15.136:0/4131137347' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-11T23:18:22.304 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:22 smithi076 ceph-mon[35296]: from='mgr.? 172.21.15.136:0/4131137347' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-11T23:18:22.304 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:22 smithi076 ceph-mon[35296]: from='mgr.? 172.21.15.136:0/4131137347' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-11T23:18:22.304 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:22 smithi076 ceph-mon[35296]: from='mgr.? 172.21.15.136:0/4131137347' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-11T23:18:22.304 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:22 smithi076 ceph-mon[35296]: from='client.? 172.21.15.26:0/2731367814' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-11T23:18:22.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:22.350+0000 7f5af21fd640 1 -- 172.21.15.26:0/3126939036 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f5aec02d7c0 con 0x7f5aec104f50 2024-09-11T23:18:22.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:22.351+0000 7f5ae8ff9640 1 -- 172.21.15.26:0/3126939036 <== mon.2 v2:172.21.15.76:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v86) ==== 100+0+11 (secure 0 0 0) 0x7f5ae00654a0 con 0x7f5aec104f50 2024-09-11T23:18:22.352 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2024-09-11T23:18:22.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:22.355+0000 7f5af21fd640 1 -- 172.21.15.26:0/3126939036 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5aec104f50 msgr2=0x7f5aec198460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:22.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:22.355+0000 7f5af21fd640 1 --2- 172.21.15.26:0/3126939036 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5aec104f50 0x7f5aec198460 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5ae000e9c0 tx=0x7f5ae000ee80 comp rx=0 tx=0).stop 2024-09-11T23:18:22.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:22.355+0000 7f5af21fd640 1 -- 172.21.15.26:0/3126939036 shutdown_connections 2024-09-11T23:18:22.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:22.355+0000 7f5af21fd640 1 --2- 172.21.15.26:0/3126939036 >> [v2:172.21.15.26:6800/110833859,v1:172.21.15.26:6801/110833859] conn(0x7f5ad40a2be0 0x7f5ad40a50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:22.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:22.355+0000 7f5af21fd640 1 --2- 172.21.15.26:0/3126939036 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5aec104f50 0x7f5aec198460 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:22.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:22.355+0000 7f5af21fd640 1 --2- 172.21.15.26:0/3126939036 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5aec107880 0x7f5aec19f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:22.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:22.355+0000 7f5af21fd640 1 --2- 172.21.15.26:0/3126939036 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5aec102620 0x7f5aec197f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:22.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:22.356+0000 7f5af21fd640 1 -- 172.21.15.26:0/3126939036 >> 172.21.15.26:0/3126939036 conn(0x7f5aec0fc140 msgr2=0x7f5aec102ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:18:22.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:22.356+0000 7f5af21fd640 1 -- 172.21.15.26:0/3126939036 shutdown_connections 2024-09-11T23:18:22.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:22.357+0000 7f5af21fd640 1 -- 172.21.15.26:0/3126939036 wait complete. 2024-09-11T23:18:22.373 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-09-11T23:18:22.373 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:18:22.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:22 smithi136 ceph-mon[35121]: from='client.? 172.21.15.26:0/2507209398' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-09-11T23:18:22.490 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:22 smithi136 ceph-mon[35121]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:18:22.490 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:22 smithi136 ceph-mon[35121]: from='client.? 172.21.15.26:0/2029199807' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T23:18:22.490 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:22 smithi136 ceph-mon[35121]: from='client.? 172.21.15.26:0/1723698173' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-11T23:18:22.490 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:22 smithi136 ceph-mon[35121]: Standby manager daemon y restarted 2024-09-11T23:18:22.490 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:22 smithi136 ceph-mon[35121]: Standby manager daemon y started 2024-09-11T23:18:22.490 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:22 smithi136 ceph-mon[35121]: from='mgr.? 172.21.15.76:0/1726212331' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-11T23:18:22.490 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:22 smithi136 ceph-mon[35121]: from='mgr.? 172.21.15.76:0/1726212331' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-11T23:18:22.490 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:22 smithi136 ceph-mon[35121]: from='mgr.? 172.21.15.76:0/1726212331' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-11T23:18:22.490 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:22 smithi136 ceph-mon[35121]: from='mgr.? 172.21.15.76:0/1726212331' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-11T23:18:22.490 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:22 smithi136 ceph-mon[35121]: Standby manager daemon z restarted 2024-09-11T23:18:22.490 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:22 smithi136 ceph-mon[35121]: Standby manager daemon z started 2024-09-11T23:18:22.490 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:22 smithi136 ceph-mon[35121]: from='mgr.? 172.21.15.136:0/4131137347' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-11T23:18:22.490 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:22 smithi136 ceph-mon[35121]: from='mgr.? 172.21.15.136:0/4131137347' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-11T23:18:22.490 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:22 smithi136 ceph-mon[35121]: from='mgr.? 172.21.15.136:0/4131137347' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-11T23:18:22.491 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:22 smithi136 ceph-mon[35121]: from='mgr.? 172.21.15.136:0/4131137347' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-11T23:18:22.491 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:22 smithi136 ceph-mon[35121]: from='client.? 172.21.15.26:0/2731367814' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-11T23:18:22.697 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:18:22.753 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:18:22 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: 2024-09-11T23:18:22.465+0000 7f7ca0473640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T23:18:22.753 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:18:22 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: 2024-09-11T23:18:22.465+0000 7f7ca0473640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T23:18:22.754 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:18:22 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: 2024-09-11T23:18:22.465+0000 7f7ca0473640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T23:18:22.754 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:18:22 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: 2024-09-11T23:18:22.465+0000 7f7ca0473640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T23:18:22.754 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:18:22 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: 2024-09-11T23:18:22.465+0000 7f7ca0473640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T23:18:22.754 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:18:22 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: 2024-09-11T23:18:22.466+0000 7f7c9cc6c640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T23:18:22.754 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:18:22 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: 2024-09-11T23:18:22.466+0000 7f7c9cc6c640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T23:18:22.754 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:18:22 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: 2024-09-11T23:18:22.466+0000 7f7c9cc6c640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T23:18:22.754 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:18:22 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: 2024-09-11T23:18:22.466+0000 7f7c9cc6c640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T23:18:22.754 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:18:22 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: 2024-09-11T23:18:22.466+0000 7f7c9cc6c640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T23:18:23.127 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:23 smithi026 ceph-mon[31172]: mgrmap e20: x(active, since 8m), standbys: y, z 2024-09-11T23:18:23.127 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:23 smithi026 ceph-mon[31172]: Active manager daemon x restarted 2024-09-11T23:18:23.127 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:23 smithi026 ceph-mon[31172]: Activating manager daemon x 2024-09-11T23:18:23.127 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:23 smithi026 ceph-mon[31172]: osdmap e86: 12 total, 12 up, 12 in 2024-09-11T23:18:23.127 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:23 smithi026 ceph-mon[31172]: mgrmap e21: x(active, starting, since 0.0124342s), standbys: y, z 2024-09-11T23:18:23.127 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:23 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T23:18:23.128 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:23 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T23:18:23.128 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:23 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T23:18:23.128 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:23 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-11T23:18:23.128 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:23 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-11T23:18:23.128 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:23 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-11T23:18:23.128 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:23 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-11T23:18:23.128 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:23 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-11T23:18:23.128 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:23 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-11T23:18:23.128 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:23 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-11T23:18:23.128 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:23 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-11T23:18:23.129 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:23 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-11T23:18:23.129 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:23 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-11T23:18:23.129 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:23 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-11T23:18:23.129 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:23 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-11T23:18:23.129 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:23 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-11T23:18:23.129 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:23 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-11T23:18:23.129 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:23 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-11T23:18:23.129 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:23 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T23:18:23.129 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:23 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T23:18:23.129 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:23 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T23:18:23.129 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:23 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T23:18:23.130 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:23 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T23:18:23.130 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:23 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T23:18:23.130 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:23 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T23:18:23.130 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:23 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T23:18:23.130 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:23 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-11T23:18:23.130 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:23 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T23:18:23.130 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:23 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-11T23:18:23.130 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:23 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T23:18:23.130 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:23 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-11T23:18:23.130 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:23 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-11T23:18:23.130 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:23 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-11T23:18:23.131 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:23 smithi026 ceph-mon[31172]: Manager daemon x is now available 2024-09-11T23:18:23.131 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:23 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:18:23.131 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:23 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-11T23:18:23.131 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:23 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-11T23:18:23.131 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:23 smithi026 ceph-mon[31172]: from='client.? 172.21.15.26:0/3126939036' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-11T23:18:23.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:23.217+0000 7fb336ceb640 1 -- 172.21.15.26:0/1969330732 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb330108300 msgr2=0x7fb33010ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:23.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:23.217+0000 7fb336ceb640 1 --2- 172.21.15.26:0/1969330732 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb330108300 0x7fb33010ec70 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fb328009cd0 tx=0x7fb32802a8d0 comp rx=0 tx=0).stop 2024-09-11T23:18:23.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:23.217+0000 7fb336ceb640 1 -- 172.21.15.26:0/1969330732 shutdown_connections 2024-09-11T23:18:23.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:23.217+0000 7fb336ceb640 1 --2- 172.21.15.26:0/1969330732 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb330103780 0x7fb330107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:23.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:23.217+0000 7fb336ceb640 1 --2- 172.21.15.26:0/1969330732 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb330108300 0x7fb33010ec70 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:23.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:23.217+0000 7fb336ceb640 1 --2- 172.21.15.26:0/1969330732 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb330102dd0 0x7fb3301031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:23.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:23.217+0000 7fb336ceb640 1 -- 172.21.15.26:0/1969330732 >> 172.21.15.26:0/1969330732 conn(0x7fb3300fc7d0 msgr2=0x7fb3300febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:18:23.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:23.217+0000 7fb336ceb640 1 -- 172.21.15.26:0/1969330732 shutdown_connections 2024-09-11T23:18:23.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:23.217+0000 7fb336ceb640 1 -- 172.21.15.26:0/1969330732 wait complete. 2024-09-11T23:18:23.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:23.218+0000 7fb336ceb640 1 Processor -- start 2024-09-11T23:18:23.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:23.218+0000 7fb336ceb640 1 -- start start 2024-09-11T23:18:23.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:23.218+0000 7fb336ceb640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb330102dd0 0x7fb330198580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:23.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:23.218+0000 7fb336ceb640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb330103780 0x7fb330198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:23.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:23.218+0000 7fb336ceb640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb330108300 0x7fb33019ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:23.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:23.218+0000 7fb336ceb640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb3301120d0 con 0x7fb330103780 2024-09-11T23:18:23.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:23.218+0000 7fb336ceb640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fb330111f50 con 0x7fb330108300 2024-09-11T23:18:23.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:23.218+0000 7fb336ceb640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb330112250 con 0x7fb330102dd0 2024-09-11T23:18:23.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:23.218+0000 7fb334a87640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb330102dd0 0x7fb330198580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:23.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:23.218+0000 7fb32ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb330103780 0x7fb330198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:23.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:23.218+0000 7fb334a87640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb330102dd0 0x7fb330198580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:58898/0 (socket says 172.21.15.26:58898) 2024-09-11T23:18:23.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:23.218+0000 7fb334a87640 1 -- 172.21.15.26:0/4138975810 learned_addr learned my addr 172.21.15.26:0/4138975810 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:18:23.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:23.219+0000 7fb335288640 1 --2- 172.21.15.26:0/4138975810 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb330108300 0x7fb33019ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:23.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:23.219+0000 7fb334a87640 1 -- 172.21.15.26:0/4138975810 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb330108300 msgr2=0x7fb33019ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:23.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:23.219+0000 7fb334a87640 1 --2- 172.21.15.26:0/4138975810 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb330108300 0x7fb33019ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:23.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:23.219+0000 7fb334a87640 1 -- 172.21.15.26:0/4138975810 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb330103780 msgr2=0x7fb330198ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:23.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:23.219+0000 7fb334a87640 1 --2- 172.21.15.26:0/4138975810 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb330103780 0x7fb330198ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:23.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:23.219+0000 7fb334a87640 1 -- 172.21.15.26:0/4138975810 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb33019d530 con 0x7fb330102dd0 2024-09-11T23:18:23.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:23.219+0000 7fb335288640 1 --2- 172.21.15.26:0/4138975810 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb330108300 0x7fb33019ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:18:23.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:23.219+0000 7fb32ffff640 1 --2- 172.21.15.26:0/4138975810 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb330103780 0x7fb330198ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:18:23.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:23.219+0000 7fb334a87640 1 --2- 172.21.15.26:0/4138975810 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb330102dd0 0x7fb330198580 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fb32400d950 tx=0x7fb32400de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:18:23.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:23.219+0000 7fb32dffb640 1 -- 172.21.15.26:0/4138975810 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb324017070 con 0x7fb330102dd0 2024-09-11T23:18:23.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:23.219+0000 7fb32dffb640 1 -- 172.21.15.26:0/4138975810 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb324009c30 con 0x7fb330102dd0 2024-09-11T23:18:23.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:23.219+0000 7fb336ceb640 1 -- 172.21.15.26:0/4138975810 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb33019d820 con 0x7fb330102dd0 2024-09-11T23:18:23.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:23.219+0000 7fb32dffb640 1 -- 172.21.15.26:0/4138975810 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb32400f6b0 con 0x7fb330102dd0 2024-09-11T23:18:23.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:23.219+0000 7fb336ceb640 1 -- 172.21.15.26:0/4138975810 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3301a5100 con 0x7fb330102dd0 2024-09-11T23:18:23.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:23.221+0000 7fb336ceb640 1 -- 172.21.15.26:0/4138975810 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3301031b0 con 0x7fb330102dd0 2024-09-11T23:18:23.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:23.222+0000 7fb32dffb640 1 -- 172.21.15.26:0/4138975810 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 22) ==== 137645+0+0 (secure 0 0 0) 0x7fb3240109d0 con 0x7fb330102dd0 2024-09-11T23:18:23.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:23.222+0000 7fb32dffb640 1 --2- 172.21.15.26:0/4138975810 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fb3200a2970 0x7fb3200a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:23.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:23.222+0000 7fb32ffff640 1 --2- 172.21.15.26:0/4138975810 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fb3200a2970 0x7fb3200a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:23.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:23.222+0000 7fb32dffb640 1 -- 172.21.15.26:0/4138975810 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fb3240ce9e0 con 0x7fb330102dd0 2024-09-11T23:18:23.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:23.223+0000 7fb32ffff640 1 --2- 172.21.15.26:0/4138975810 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fb3200a2970 0x7fb3200a4e30 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fb3300fee70 tx=0x7fb31c00b410 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:18:23.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:23.227+0000 7fb32dffb640 1 -- 172.21.15.26:0/4138975810 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb324009de0 con 0x7fb330102dd0 2024-09-11T23:18:23.304 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:23 smithi076 ceph-mon[35296]: mgrmap e20: x(active, since 8m), standbys: y, z 2024-09-11T23:18:23.304 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:23 smithi076 ceph-mon[35296]: Active manager daemon x restarted 2024-09-11T23:18:23.304 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:23 smithi076 ceph-mon[35296]: Activating manager daemon x 2024-09-11T23:18:23.304 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:23 smithi076 ceph-mon[35296]: osdmap e86: 12 total, 12 up, 12 in 2024-09-11T23:18:23.304 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:23 smithi076 ceph-mon[35296]: mgrmap e21: x(active, starting, since 0.0124342s), standbys: y, z 2024-09-11T23:18:23.304 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:23 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T23:18:23.304 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:23 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T23:18:23.304 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:23 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T23:18:23.304 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:23 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-11T23:18:23.305 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:23 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-11T23:18:23.305 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:23 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-11T23:18:23.305 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:23 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-11T23:18:23.305 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:23 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-11T23:18:23.305 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:23 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-11T23:18:23.305 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:23 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-11T23:18:23.305 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:23 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-11T23:18:23.305 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:23 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-11T23:18:23.305 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:23 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-11T23:18:23.305 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:23 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-11T23:18:23.305 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:23 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-11T23:18:23.305 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:23 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-11T23:18:23.305 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:23 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-11T23:18:23.305 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:23 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-11T23:18:23.306 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:23 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T23:18:23.306 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:23 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T23:18:23.306 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:23 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T23:18:23.306 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:23 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T23:18:23.306 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:23 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T23:18:23.306 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:23 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T23:18:23.306 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:23 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T23:18:23.306 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:23 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T23:18:23.306 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:23 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-11T23:18:23.306 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:23 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T23:18:23.306 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:23 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-11T23:18:23.306 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:23 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T23:18:23.306 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:23 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-11T23:18:23.306 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:23 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-11T23:18:23.307 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:23 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-11T23:18:23.307 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:23 smithi076 ceph-mon[35296]: Manager daemon x is now available 2024-09-11T23:18:23.307 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:23 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:18:23.307 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:23 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-11T23:18:23.307 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:23 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-11T23:18:23.307 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:23 smithi076 ceph-mon[35296]: from='client.? 172.21.15.26:0/3126939036' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-11T23:18:23.480 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:23 smithi136 ceph-mon[35121]: mgrmap e20: x(active, since 8m), standbys: y, z 2024-09-11T23:18:23.480 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:23 smithi136 ceph-mon[35121]: Active manager daemon x restarted 2024-09-11T23:18:23.481 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:23 smithi136 ceph-mon[35121]: Activating manager daemon x 2024-09-11T23:18:23.481 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:23 smithi136 ceph-mon[35121]: osdmap e86: 12 total, 12 up, 12 in 2024-09-11T23:18:23.481 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:23 smithi136 ceph-mon[35121]: mgrmap e21: x(active, starting, since 0.0124342s), standbys: y, z 2024-09-11T23:18:23.481 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:23 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T23:18:23.481 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:23 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T23:18:23.481 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:23 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T23:18:23.481 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:23 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-11T23:18:23.481 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:23 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-11T23:18:23.481 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:23 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-11T23:18:23.481 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:23 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-11T23:18:23.481 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:23 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-11T23:18:23.482 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:23 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-11T23:18:23.482 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:23 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-11T23:18:23.482 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:23 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-11T23:18:23.482 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:23 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-11T23:18:23.482 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:23 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-11T23:18:23.482 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:23 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-11T23:18:23.482 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:23 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-11T23:18:23.482 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:23 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-11T23:18:23.482 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:23 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-11T23:18:23.482 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:23 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-11T23:18:23.482 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:23 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T23:18:23.483 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:23 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T23:18:23.483 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:23 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T23:18:23.483 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:23 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T23:18:23.483 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:23 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T23:18:23.483 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:23 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T23:18:23.483 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:23 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T23:18:23.483 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:23 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T23:18:23.483 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:23 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-11T23:18:23.483 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:23 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T23:18:23.483 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:23 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-11T23:18:23.483 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:23 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T23:18:23.483 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:23 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-11T23:18:23.484 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:23 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-11T23:18:23.484 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:23 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-11T23:18:23.484 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:23 smithi136 ceph-mon[35121]: Manager daemon x is now available 2024-09-11T23:18:23.484 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:23 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:18:23.484 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:23 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-11T23:18:23.484 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:23 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-11T23:18:23.484 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:23 smithi136 ceph-mon[35121]: from='client.? 172.21.15.26:0/3126939036' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-11T23:18:23.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:23.483+0000 7fb336ceb640 1 -- 172.21.15.26:0/4138975810 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb33010a290 con 0x7fb330102dd0 2024-09-11T23:18:23.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:23.484+0000 7fb32dffb640 1 -- 172.21.15.26:0/4138975810 <== mon.1 v2:172.21.15.136:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11971 (secure 0 0 0) 0x7fb32409bb20 con 0x7fb330102dd0 2024-09-11T23:18:23.486 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-11T23:18:23.486 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":29,"btime":"2024-09-11T23:18:21:019324+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14499,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/367659515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":367659515},{"type":"v1","addr":"172.21.15.26:6835","nonce":367659515}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14511,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/2041065299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":2041065299},{"type":"v1","addr":"172.21.15.26:6837","nonce":2041065299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:16:26.544203+0000","modified":"2024-09-11T23:18:20.969151+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24485,"mds_1":24463,"mds_2":24458,"mds_3":24448,"mds_4":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14550":{"gid":14550,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6839/2233068305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":2233068305},{"type":"v1","addr":"172.21.15.26:6839","nonce":2233068305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14562":{"gid":14562,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6841/2984982336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":2984982336},{"type":"v1","addr":"172.21.15.26:6841","nonce":2984982336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6833/2019841455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":2019841455},{"type":"v1","addr":"172.21.15.136:6833","nonce":2019841455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.76:6833/3438147050","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3438147050},{"type":"v1","addr":"172.21.15.76:6833","nonce":3438147050}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6835/3617665975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3617665975},{"type":"v1","addr":"172.21.15.136:6835","nonce":3617665975}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":12,"addr":"172.21.15.76:6835/393308499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":393308499},{"type":"v1","addr":"172.21.15.76:6835","nonce":393308499}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":10,"addr":"172.21.15.136:6837/1633082387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":1633082387},{"type":"v1","addr":"172.21.15.136:6837","nonce":1633082387}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.76:6837/2078978748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":2078978748},{"type":"v1","addr":"172.21.15.76:6837","nonce":2078978748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.136:6839/3211398630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":3211398630},{"type":"v1","addr":"172.21.15.136:6839","nonce":3211398630}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.76:6839/2077637090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6838","nonce":2077637090},{"type":"v1","addr":"172.21.15.76:6839","nonce":2077637090}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24485,"qdb_cluster":[24443,24448,24458,24463,24485]},"id":1}]} 2024-09-11T23:18:23.486 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 29 2024-09-11T23:18:23.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:23.487+0000 7fb336ceb640 1 -- 172.21.15.26:0/4138975810 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fb3200a2970 msgr2=0x7fb3200a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:23.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:23.487+0000 7fb336ceb640 1 --2- 172.21.15.26:0/4138975810 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fb3200a2970 0x7fb3200a4e30 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fb3300fee70 tx=0x7fb31c00b410 comp rx=0 tx=0).stop 2024-09-11T23:18:23.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:23.487+0000 7fb336ceb640 1 -- 172.21.15.26:0/4138975810 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb330102dd0 msgr2=0x7fb330198580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:23.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:23.487+0000 7fb336ceb640 1 --2- 172.21.15.26:0/4138975810 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb330102dd0 0x7fb330198580 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fb32400d950 tx=0x7fb32400de10 comp rx=0 tx=0).stop 2024-09-11T23:18:23.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:23.487+0000 7fb336ceb640 1 -- 172.21.15.26:0/4138975810 shutdown_connections 2024-09-11T23:18:23.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:23.487+0000 7fb336ceb640 1 --2- 172.21.15.26:0/4138975810 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fb3200a2970 0x7fb3200a4e30 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:23.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:23.487+0000 7fb336ceb640 1 --2- 172.21.15.26:0/4138975810 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb330108300 0x7fb33019ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:23.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:23.487+0000 7fb336ceb640 1 --2- 172.21.15.26:0/4138975810 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb330103780 0x7fb330198ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:23.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:23.487+0000 7fb336ceb640 1 --2- 172.21.15.26:0/4138975810 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb330102dd0 0x7fb330198580 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:23.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:23.487+0000 7fb336ceb640 1 -- 172.21.15.26:0/4138975810 >> 172.21.15.26:0/4138975810 conn(0x7fb3300fc7d0 msgr2=0x7fb330105e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:18:23.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:23.487+0000 7fb336ceb640 1 -- 172.21.15.26:0/4138975810 shutdown_connections 2024-09-11T23:18:23.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:23.487+0000 7fb336ceb640 1 -- 172.21.15.26:0/4138975810 wait complete. 2024-09-11T23:18:24.425 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:24 smithi026 ceph-mon[31172]: mgrmap e22: x(active, since 1.02227s), standbys: y, z 2024-09-11T23:18:24.425 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:24 smithi026 ceph-mon[31172]: from='client.? 172.21.15.26:0/4138975810' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:18:24.426 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:24 smithi026 ceph-mon[31172]: [11/Sep/2024:23:18:23] ENGINE Bus STARTING 2024-09-11T23:18:24.426 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:24 smithi026 ceph-mon[31172]: [11/Sep/2024:23:18:23] ENGINE Serving on http://172.21.15.26:8765 2024-09-11T23:18:24.426 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:18:24 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: 2024-09-11T23:18:24.037+0000 7f7c83c3b640 -1 log_channel(cephadm) log [ERR] : [11/Sep/2024:23:18:24] ENGINE Error in HTTPServer.serve 2024-09-11T23:18:24.426 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:18:24 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: Traceback (most recent call last): 2024-09-11T23:18:24.426 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:18:24 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-11T23:18:24.426 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:18:24 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: self._connections.run(self.expiration_interval) 2024-09-11T23:18:24.426 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:18:24 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-11T23:18:24.426 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:18:24 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: self._run(expiration_interval) 2024-09-11T23:18:24.426 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:18:24 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-11T23:18:24.426 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:18:24 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: new_conn = self._from_server_socket(self.server.socket) 2024-09-11T23:18:24.426 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:18:24 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-11T23:18:24.427 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:18:24 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-11T23:18:24.427 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:18:24 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-11T23:18:24.427 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:18:24 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: s = self.context.wrap_socket( 2024-09-11T23:18:24.427 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:18:24 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-11T23:18:24.427 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:18:24 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: return self.sslsocket_class._create( 2024-09-11T23:18:24.427 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:18:24 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-11T23:18:24.427 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:18:24 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: self.do_handshake() 2024-09-11T23:18:24.427 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:18:24 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-11T23:18:24.427 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:18:24 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: self._sslobj.do_handshake() 2024-09-11T23:18:24.427 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:18:24 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-11T23:18:24.427 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:18:24 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: 2024-09-11T23:18:24.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:24 smithi136 ceph-mon[35121]: mgrmap e22: x(active, since 1.02227s), standbys: y, z 2024-09-11T23:18:24.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:24 smithi136 ceph-mon[35121]: from='client.? 172.21.15.26:0/4138975810' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:18:24.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:24 smithi136 ceph-mon[35121]: [11/Sep/2024:23:18:23] ENGINE Bus STARTING 2024-09-11T23:18:24.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:24 smithi136 ceph-mon[35121]: [11/Sep/2024:23:18:23] ENGINE Serving on http://172.21.15.26:8765 2024-09-11T23:18:24.553 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:24 smithi076 ceph-mon[35296]: mgrmap e22: x(active, since 1.02227s), standbys: y, z 2024-09-11T23:18:24.553 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:24 smithi076 ceph-mon[35296]: from='client.? 172.21.15.26:0/4138975810' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:18:24.553 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:24 smithi076 ceph-mon[35296]: [11/Sep/2024:23:18:23] ENGINE Bus STARTING 2024-09-11T23:18:24.553 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:24 smithi076 ceph-mon[35296]: [11/Sep/2024:23:18:23] ENGINE Serving on http://172.21.15.26:8765 2024-09-11T23:18:24.872 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-09-11T23:18:24.872 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-09-11T23:18:24.872 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T23:18:24.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:24.993+0000 7f13c239f640 1 -- 172.21.15.26:0/3738655956 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f13bc101c00 msgr2=0x7f13bc102080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:24.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:24.993+0000 7f13c239f640 1 --2- 172.21.15.26:0/3738655956 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f13bc101c00 0x7f13bc102080 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f13b0009900 tx=0x7f13b002a410 comp rx=0 tx=0).stop 2024-09-11T23:18:24.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:24.994+0000 7f13c239f640 1 -- 172.21.15.26:0/3738655956 shutdown_connections 2024-09-11T23:18:24.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:24.994+0000 7f13c239f640 1 --2- 172.21.15.26:0/3738655956 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f13bc1025c0 0x7f13bc108e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:24.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:24.994+0000 7f13c239f640 1 --2- 172.21.15.26:0/3738655956 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f13bc101c00 0x7f13bc102080 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:24.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:24.994+0000 7f13c239f640 1 --2- 172.21.15.26:0/3738655956 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f13bc100a10 0x7f13bc100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:24.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:24.994+0000 7f13c239f640 1 -- 172.21.15.26:0/3738655956 >> 172.21.15.26:0/3738655956 conn(0x7f13bc0fc170 msgr2=0x7f13bc0fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:18:24.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:24.994+0000 7f13c239f640 1 -- 172.21.15.26:0/3738655956 shutdown_connections 2024-09-11T23:18:24.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:24.994+0000 7f13c239f640 1 -- 172.21.15.26:0/3738655956 wait complete. 2024-09-11T23:18:24.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:24.994+0000 7f13c239f640 1 Processor -- start 2024-09-11T23:18:24.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:24.994+0000 7f13c239f640 1 -- start start 2024-09-11T23:18:24.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:24.995+0000 7f13c239f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f13bc100a10 0x7f13bc197e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:24.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:24.995+0000 7f13c239f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f13bc101c00 0x7f13bc1983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:24.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:24.995+0000 7f13c239f640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f13bc1025c0 0x7f13bc19f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:24.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:24.995+0000 7f13c239f640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f13bc10d7e0 con 0x7f13bc101c00 2024-09-11T23:18:24.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:24.995+0000 7f13c239f640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f13bc10d660 con 0x7f13bc100a10 2024-09-11T23:18:24.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:24.995+0000 7f13c239f640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f13bc10d960 con 0x7f13bc1025c0 2024-09-11T23:18:24.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:24.995+0000 7f13bb7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f13bc101c00 0x7f13bc1983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:24.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:24.995+0000 7f13bb7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f13bc101c00 0x7f13bc1983c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47356/0 (socket says 172.21.15.26:47356) 2024-09-11T23:18:24.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:24.995+0000 7f13bb7fe640 1 -- 172.21.15.26:0/354157413 learned_addr learned my addr 172.21.15.26:0/354157413 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:18:24.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:24.995+0000 7f13bbfff640 1 --2- 172.21.15.26:0/354157413 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f13bc100a10 0x7f13bc197e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:24.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:24.995+0000 7f13c093c640 1 --2- 172.21.15.26:0/354157413 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f13bc1025c0 0x7f13bc19f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:24.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:24.995+0000 7f13bb7fe640 1 -- 172.21.15.26:0/354157413 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f13bc1025c0 msgr2=0x7f13bc19f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:24.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:24.995+0000 7f13bb7fe640 1 --2- 172.21.15.26:0/354157413 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f13bc1025c0 0x7f13bc19f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:24.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:24.995+0000 7f13bb7fe640 1 -- 172.21.15.26:0/354157413 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f13bc100a10 msgr2=0x7f13bc197e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:24.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:24.995+0000 7f13bb7fe640 1 --2- 172.21.15.26:0/354157413 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f13bc100a10 0x7f13bc197e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:24.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:24.995+0000 7f13bb7fe640 1 -- 172.21.15.26:0/354157413 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13bc19faa0 con 0x7f13bc101c00 2024-09-11T23:18:24.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:24.995+0000 7f13bb7fe640 1 --2- 172.21.15.26:0/354157413 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f13bc101c00 0x7f13bc1983c0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f13b0009230 tx=0x7f13b0031b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:18:25.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:24.996+0000 7f13b97fa640 1 -- 172.21.15.26:0/354157413 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13b003f030 con 0x7f13bc101c00 2024-09-11T23:18:25.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:24.996+0000 7f13b97fa640 1 -- 172.21.15.26:0/354157413 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f13b0003ad0 con 0x7f13bc101c00 2024-09-11T23:18:25.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:24.996+0000 7f13c239f640 1 -- 172.21.15.26:0/354157413 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13bc19fd30 con 0x7f13bc101c00 2024-09-11T23:18:25.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:24.996+0000 7f13b97fa640 1 -- 172.21.15.26:0/354157413 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13b0035070 con 0x7f13bc101c00 2024-09-11T23:18:25.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:24.996+0000 7f13c239f640 1 -- 172.21.15.26:0/354157413 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13bc1a01d0 con 0x7f13bc101c00 2024-09-11T23:18:25.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:24.997+0000 7f13c239f640 1 -- 172.21.15.26:0/354157413 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13bc107e70 con 0x7f13bc101c00 2024-09-11T23:18:25.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:24.998+0000 7f13b97fa640 1 -- 172.21.15.26:0/354157413 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 23) ==== 137953+0+0 (secure 0 0 0) 0x7f13b0042030 con 0x7f13bc101c00 2024-09-11T23:18:25.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:24.999+0000 7f13b97fa640 1 --2- 172.21.15.26:0/354157413 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f13a40a2d00 0x7f13a40a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:25.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:24.999+0000 7f13b97fa640 1 -- 172.21.15.26:0/354157413 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f13b00ef950 con 0x7f13bc101c00 2024-09-11T23:18:25.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:24.999+0000 7f13bbfff640 1 --2- 172.21.15.26:0/354157413 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f13a40a2d00 0x7f13a40a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:25.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:24.999+0000 7f13bbfff640 1 --2- 172.21.15.26:0/354157413 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f13a40a2d00 0x7f13a40a51c0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f13bc101a60 tx=0x7f13ac005f90 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:18:25.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:25.003+0000 7f13b97fa640 1 -- 172.21.15.26:0/354157413 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f13b00bbf30 con 0x7f13bc101c00 2024-09-11T23:18:25.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:25.221+0000 7f13c239f640 1 -- 172.21.15.26:0/354157413 --> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f13bc062630 con 0x7f13a40a2d00 2024-09-11T23:18:25.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:25.228+0000 7f13b97fa640 1 -- 172.21.15.26:0/354157413 <== mgr.15171 v2:172.21.15.26:6800/1989555790 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+257778 (secure 0 0 0) 0x7f13bc062630 con 0x7f13a40a2d00 2024-09-11T23:18:25.234 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-11T23:18:25.236 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-09-11T23:18:25.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:25.232+0000 7f13c239f640 1 -- 172.21.15.26:0/354157413 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f13a40a2d00 msgr2=0x7f13a40a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:25.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:25.232+0000 7f13c239f640 1 --2- 172.21.15.26:0/354157413 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f13a40a2d00 0x7f13a40a51c0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f13bc101a60 tx=0x7f13ac005f90 comp rx=0 tx=0).stop 2024-09-11T23:18:25.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:25.232+0000 7f13c239f640 1 -- 172.21.15.26:0/354157413 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f13bc101c00 msgr2=0x7f13bc1983c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:25.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:25.232+0000 7f13c239f640 1 --2- 172.21.15.26:0/354157413 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f13bc101c00 0x7f13bc1983c0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f13b0009230 tx=0x7f13b0031b60 comp rx=0 tx=0).stop 2024-09-11T23:18:25.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:25.232+0000 7f13c239f640 1 -- 172.21.15.26:0/354157413 shutdown_connections 2024-09-11T23:18:25.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:25.232+0000 7f13c239f640 1 --2- 172.21.15.26:0/354157413 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f13a40a2d00 0x7f13a40a51c0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:25.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:25.232+0000 7f13c239f640 1 --2- 172.21.15.26:0/354157413 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f13bc1025c0 0x7f13bc19f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:25.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:25.232+0000 7f13c239f640 1 --2- 172.21.15.26:0/354157413 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f13bc101c00 0x7f13bc1983c0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:25.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:25.232+0000 7f13c239f640 1 --2- 172.21.15.26:0/354157413 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f13bc100a10 0x7f13bc197e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:25.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:25.232+0000 7f13c239f640 1 -- 172.21.15.26:0/354157413 >> 172.21.15.26:0/354157413 conn(0x7f13bc0fc170 msgr2=0x7f13bc105040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:18:25.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:25.232+0000 7f13c239f640 1 -- 172.21.15.26:0/354157413 shutdown_connections 2024-09-11T23:18:25.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:25.232+0000 7f13c239f640 1 -- 172.21.15.26:0/354157413 wait complete. 2024-09-11T23:18:25.245 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":4,"stamp":"2024-09-11T23:18:24.197005+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1189948,"num_objects":104,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1325,"num_read_kb":828,"num_write":555,"num_write_kb":4402,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":937,"ondisk_log_size":937,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":380772,"kb_used_data":14560,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124561052,"statfs":{"total":1151940427776,"available":1151550517248,"internally_reserved":0,"allocated":14909440,"data_stored":8639524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":374453561},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.991701"},"pg_stats":[{"pgid":"3.3f","version":"85'8","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.329067+0000","last_change":"2024-09-11T23:16:25.899106+0000","last_active":"2024-09-11T23:18:22.329067+0000","last_peered":"2024-09-11T23:18:22.329067+0000","last_clean":"2024-09-11T23:18:22.329067+0000","last_became_active":"2024-09-11T23:16:25.898654+0000","last_became_peered":"2024-09-11T23:16:25.898654+0000","last_unstale":"2024-09-11T23:18:22.329067+0000","last_undegraded":"2024-09-11T23:18:22.329067+0000","last_fullsized":"2024-09-11T23:18:22.329067+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:21:41.490905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"85'4","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.544200+0000","last_change":"2024-09-11T23:16:25.895629+0000","last_active":"2024-09-11T23:18:22.544200+0000","last_peered":"2024-09-11T23:18:22.544200+0000","last_clean":"2024-09-11T23:18:22.544200+0000","last_became_active":"2024-09-11T23:16:25.895308+0000","last_became_peered":"2024-09-11T23:16:25.895308+0000","last_unstale":"2024-09-11T23:18:22.544200+0000","last_undegraded":"2024-09-11T23:18:22.544200+0000","last_fullsized":"2024-09-11T23:18:22.544200+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:39:42.549970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"85'7","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.544117+0000","last_change":"2024-09-11T23:16:25.907113+0000","last_active":"2024-09-11T23:18:22.544117+0000","last_peered":"2024-09-11T23:18:22.544117+0000","last_clean":"2024-09-11T23:18:22.544117+0000","last_became_active":"2024-09-11T23:16:25.906803+0000","last_became_peered":"2024-09-11T23:16:25.906803+0000","last_unstale":"2024-09-11T23:18:22.544117+0000","last_undegraded":"2024-09-11T23:18:22.544117+0000","last_fullsized":"2024-09-11T23:18:22.544117+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:36:49.303932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"85'8","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.292802+0000","last_change":"2024-09-11T23:16:35.946167+0000","last_active":"2024-09-11T23:18:22.292802+0000","last_peered":"2024-09-11T23:18:22.292802+0000","last_clean":"2024-09-11T23:18:22.292802+0000","last_became_active":"2024-09-11T23:16:35.945786+0000","last_became_peered":"2024-09-11T23:16:35.945786+0000","last_unstale":"2024-09-11T23:18:22.292802+0000","last_undegraded":"2024-09-11T23:18:22.292802+0000","last_fullsized":"2024-09-11T23:18:22.292802+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:13:03.064910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"85'4","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.295860+0000","last_change":"2024-09-11T23:16:25.897909+0000","last_active":"2024-09-11T23:18:22.295860+0000","last_peered":"2024-09-11T23:18:22.295860+0000","last_clean":"2024-09-11T23:18:22.295860+0000","last_became_active":"2024-09-11T23:16:25.897569+0000","last_became_peered":"2024-09-11T23:16:25.897569+0000","last_unstale":"2024-09-11T23:18:22.295860+0000","last_undegraded":"2024-09-11T23:18:22.295860+0000","last_fullsized":"2024-09-11T23:18:22.295860+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:25:44.727254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"85'6","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.302845+0000","last_change":"2024-09-11T23:16:25.897594+0000","last_active":"2024-09-11T23:18:22.302845+0000","last_peered":"2024-09-11T23:18:22.302845+0000","last_clean":"2024-09-11T23:18:22.302845+0000","last_became_active":"2024-09-11T23:16:25.897223+0000","last_became_peered":"2024-09-11T23:16:25.897223+0000","last_unstale":"2024-09-11T23:18:22.302845+0000","last_undegraded":"2024-09-11T23:18:22.302845+0000","last_fullsized":"2024-09-11T23:18:22.302845+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:10:50.763455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"85'5","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328293+0000","last_change":"2024-09-11T23:16:35.943511+0000","last_active":"2024-09-11T23:18:22.328293+0000","last_peered":"2024-09-11T23:18:22.328293+0000","last_clean":"2024-09-11T23:18:22.328293+0000","last_became_active":"2024-09-11T23:16:35.943075+0000","last_became_peered":"2024-09-11T23:16:35.943075+0000","last_unstale":"2024-09-11T23:18:22.328293+0000","last_undegraded":"2024-09-11T23:18:22.328293+0000","last_fullsized":"2024-09-11T23:18:22.328293+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:53:43.892421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"85'9","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.182677+0000","last_change":"2024-09-11T23:16:25.903380+0000","last_active":"2024-09-11T23:18:22.182677+0000","last_peered":"2024-09-11T23:18:22.182677+0000","last_clean":"2024-09-11T23:18:22.182677+0000","last_became_active":"2024-09-11T23:16:25.903093+0000","last_became_peered":"2024-09-11T23:16:25.903093+0000","last_unstale":"2024-09-11T23:18:22.182677+0000","last_undegraded":"2024-09-11T23:18:22.182677+0000","last_fullsized":"2024-09-11T23:18:22.182677+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:58:50.944623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"85'11","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.329153+0000","last_change":"2024-09-11T23:16:25.908166+0000","last_active":"2024-09-11T23:18:22.329153+0000","last_peered":"2024-09-11T23:18:22.329153+0000","last_clean":"2024-09-11T23:18:22.329153+0000","last_became_active":"2024-09-11T23:16:25.907749+0000","last_became_peered":"2024-09-11T23:16:25.907749+0000","last_unstale":"2024-09-11T23:18:22.329153+0000","last_undegraded":"2024-09-11T23:18:22.329153+0000","last_fullsized":"2024-09-11T23:18:22.329153+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:11:16.705931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"85'16","reported_seq":41,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.303286+0000","last_change":"2024-09-11T23:16:35.944259+0000","last_active":"2024-09-11T23:18:22.303286+0000","last_peered":"2024-09-11T23:18:22.303286+0000","last_clean":"2024-09-11T23:18:22.303286+0000","last_became_active":"2024-09-11T23:16:35.943767+0000","last_became_peered":"2024-09-11T23:16:35.943767+0000","last_unstale":"2024-09-11T23:18:22.303286+0000","last_undegraded":"2024-09-11T23:18:22.303286+0000","last_fullsized":"2024-09-11T23:18:22.303286+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:20:13.127854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"85'6","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328788+0000","last_change":"2024-09-11T23:16:25.903256+0000","last_active":"2024-09-11T23:18:22.328788+0000","last_peered":"2024-09-11T23:18:22.328788+0000","last_clean":"2024-09-11T23:18:22.328788+0000","last_became_active":"2024-09-11T23:16:25.902954+0000","last_became_peered":"2024-09-11T23:16:25.902954+0000","last_unstale":"2024-09-11T23:18:22.328788+0000","last_undegraded":"2024-09-11T23:18:22.328788+0000","last_fullsized":"2024-09-11T23:18:22.328788+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:43:48.651400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"85'7","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.187001+0000","last_change":"2024-09-11T23:16:25.897656+0000","last_active":"2024-09-11T23:18:22.187001+0000","last_peered":"2024-09-11T23:18:22.187001+0000","last_clean":"2024-09-11T23:18:22.187001+0000","last_became_active":"2024-09-11T23:16:25.897103+0000","last_became_peered":"2024-09-11T23:16:25.897103+0000","last_unstale":"2024-09-11T23:18:22.187001+0000","last_undegraded":"2024-09-11T23:18:22.187001+0000","last_fullsized":"2024-09-11T23:18:22.187001+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:14:34.341383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"85'10","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.329625+0000","last_change":"2024-09-11T23:16:25.902800+0000","last_active":"2024-09-11T23:18:22.329625+0000","last_peered":"2024-09-11T23:18:22.329625+0000","last_clean":"2024-09-11T23:18:22.329625+0000","last_became_active":"2024-09-11T23:16:25.902414+0000","last_became_peered":"2024-09-11T23:16:25.902414+0000","last_unstale":"2024-09-11T23:18:22.329625+0000","last_undegraded":"2024-09-11T23:18:22.329625+0000","last_fullsized":"2024-09-11T23:18:22.329625+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:16:32.427749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"85'7","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.543876+0000","last_change":"2024-09-11T23:16:25.894410+0000","last_active":"2024-09-11T23:18:22.543876+0000","last_peered":"2024-09-11T23:18:22.543876+0000","last_clean":"2024-09-11T23:18:22.543876+0000","last_became_active":"2024-09-11T23:16:25.894067+0000","last_became_peered":"2024-09-11T23:16:25.894067+0000","last_unstale":"2024-09-11T23:18:22.543876+0000","last_undegraded":"2024-09-11T23:18:22.543876+0000","last_fullsized":"2024-09-11T23:18:22.543876+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:03:11.913063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"85'7","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.544140+0000","last_change":"2024-09-11T23:16:25.903440+0000","last_active":"2024-09-11T23:18:22.544140+0000","last_peered":"2024-09-11T23:18:22.544140+0000","last_clean":"2024-09-11T23:18:22.544140+0000","last_became_active":"2024-09-11T23:16:25.903122+0000","last_became_peered":"2024-09-11T23:16:25.903122+0000","last_unstale":"2024-09-11T23:18:22.544140+0000","last_undegraded":"2024-09-11T23:18:22.544140+0000","last_fullsized":"2024-09-11T23:18:22.544140+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:03:41.065314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"85'9","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328034+0000","last_change":"2024-09-11T23:16:25.905336+0000","last_active":"2024-09-11T23:18:22.328034+0000","last_peered":"2024-09-11T23:18:22.328034+0000","last_clean":"2024-09-11T23:18:22.328034+0000","last_became_active":"2024-09-11T23:16:25.904699+0000","last_became_peered":"2024-09-11T23:16:25.904699+0000","last_unstale":"2024-09-11T23:18:22.328034+0000","last_undegraded":"2024-09-11T23:18:22.328034+0000","last_fullsized":"2024-09-11T23:18:22.328034+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:16:13.448585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"85'8","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.329144+0000","last_change":"2024-09-11T23:16:25.897977+0000","last_active":"2024-09-11T23:18:22.329144+0000","last_peered":"2024-09-11T23:18:22.329144+0000","last_clean":"2024-09-11T23:18:22.329144+0000","last_became_active":"2024-09-11T23:16:25.897672+0000","last_became_peered":"2024-09-11T23:16:25.897672+0000","last_unstale":"2024-09-11T23:18:22.329144+0000","last_undegraded":"2024-09-11T23:18:22.329144+0000","last_fullsized":"2024-09-11T23:18:22.329144+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:42:56.575066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"85'10","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.293509+0000","last_change":"2024-09-11T23:16:25.897063+0000","last_active":"2024-09-11T23:18:22.293509+0000","last_peered":"2024-09-11T23:18:22.293509+0000","last_clean":"2024-09-11T23:18:22.293509+0000","last_became_active":"2024-09-11T23:16:25.896770+0000","last_became_peered":"2024-09-11T23:16:25.896770+0000","last_unstale":"2024-09-11T23:18:22.293509+0000","last_undegraded":"2024-09-11T23:18:22.293509+0000","last_fullsized":"2024-09-11T23:18:22.293509+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:55:44.735171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"85'7","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328943+0000","last_change":"2024-09-11T23:16:25.903423+0000","last_active":"2024-09-11T23:18:22.328943+0000","last_peered":"2024-09-11T23:18:22.328943+0000","last_clean":"2024-09-11T23:18:22.328943+0000","last_became_active":"2024-09-11T23:16:25.902906+0000","last_became_peered":"2024-09-11T23:16:25.902906+0000","last_unstale":"2024-09-11T23:18:22.328943+0000","last_undegraded":"2024-09-11T23:18:22.328943+0000","last_fullsized":"2024-09-11T23:18:22.328943+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:40:49.326045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"85'8","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328557+0000","last_change":"2024-09-11T23:16:25.896564+0000","last_active":"2024-09-11T23:18:22.328557+0000","last_peered":"2024-09-11T23:18:22.328557+0000","last_clean":"2024-09-11T23:18:22.328557+0000","last_became_active":"2024-09-11T23:16:25.896301+0000","last_became_peered":"2024-09-11T23:16:25.896301+0000","last_unstale":"2024-09-11T23:18:22.328557+0000","last_undegraded":"2024-09-11T23:18:22.328557+0000","last_fullsized":"2024-09-11T23:18:22.328557+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:30:30.074853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"85'11","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.329804+0000","last_change":"2024-09-11T23:16:25.903491+0000","last_active":"2024-09-11T23:18:22.329804+0000","last_peered":"2024-09-11T23:18:22.329804+0000","last_clean":"2024-09-11T23:18:22.329804+0000","last_became_active":"2024-09-11T23:16:25.901340+0000","last_became_peered":"2024-09-11T23:16:25.901340+0000","last_unstale":"2024-09-11T23:18:22.329804+0000","last_undegraded":"2024-09-11T23:18:22.329804+0000","last_fullsized":"2024-09-11T23:18:22.329804+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:41:16.058034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"85'10","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328233+0000","last_change":"2024-09-11T23:16:25.901084+0000","last_active":"2024-09-11T23:18:22.328233+0000","last_peered":"2024-09-11T23:18:22.328233+0000","last_clean":"2024-09-11T23:18:22.328233+0000","last_became_active":"2024-09-11T23:16:25.900726+0000","last_became_peered":"2024-09-11T23:16:25.900726+0000","last_unstale":"2024-09-11T23:18:22.328233+0000","last_undegraded":"2024-09-11T23:18:22.328233+0000","last_fullsized":"2024-09-11T23:18:22.328233+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:57:40.955645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"85'8","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.182688+0000","last_change":"2024-09-11T23:16:25.900461+0000","last_active":"2024-09-11T23:18:22.182688+0000","last_peered":"2024-09-11T23:18:22.182688+0000","last_clean":"2024-09-11T23:18:22.182688+0000","last_became_active":"2024-09-11T23:16:25.900174+0000","last_became_peered":"2024-09-11T23:16:25.900174+0000","last_unstale":"2024-09-11T23:18:22.182688+0000","last_undegraded":"2024-09-11T23:18:22.182688+0000","last_fullsized":"2024-09-11T23:18:22.182688+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:51:50.953198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"85'6","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.184452+0000","last_change":"2024-09-11T23:16:25.907569+0000","last_active":"2024-09-11T23:18:22.184452+0000","last_peered":"2024-09-11T23:18:22.184452+0000","last_clean":"2024-09-11T23:18:22.184452+0000","last_became_active":"2024-09-11T23:16:25.907091+0000","last_became_peered":"2024-09-11T23:16:25.907091+0000","last_unstale":"2024-09-11T23:18:22.184452+0000","last_undegraded":"2024-09-11T23:18:22.184452+0000","last_fullsized":"2024-09-11T23:18:22.184452+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:53:45.961167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"85'2","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.183204+0000","last_change":"2024-09-11T23:16:25.903246+0000","last_active":"2024-09-11T23:18:22.183204+0000","last_peered":"2024-09-11T23:18:22.183204+0000","last_clean":"2024-09-11T23:18:22.183204+0000","last_became_active":"2024-09-11T23:16:25.902907+0000","last_became_peered":"2024-09-11T23:16:25.902907+0000","last_unstale":"2024-09-11T23:18:22.183204+0000","last_undegraded":"2024-09-11T23:18:22.183204+0000","last_fullsized":"2024-09-11T23:18:22.183204+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:20:38.868384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"85'15","reported_seq":34,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328979+0000","last_change":"2024-09-11T23:16:25.906298+0000","last_active":"2024-09-11T23:18:22.328979+0000","last_peered":"2024-09-11T23:18:22.328979+0000","last_clean":"2024-09-11T23:18:22.328979+0000","last_became_active":"2024-09-11T23:16:25.905350+0000","last_became_peered":"2024-09-11T23:16:25.905350+0000","last_unstale":"2024-09-11T23:18:22.328979+0000","last_undegraded":"2024-09-11T23:18:22.328979+0000","last_fullsized":"2024-09-11T23:18:22.328979+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:04:22.122514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"85'6","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328869+0000","last_change":"2024-09-11T23:16:25.902114+0000","last_active":"2024-09-11T23:18:22.328869+0000","last_peered":"2024-09-11T23:18:22.328869+0000","last_clean":"2024-09-11T23:18:22.328869+0000","last_became_active":"2024-09-11T23:16:25.901742+0000","last_became_peered":"2024-09-11T23:16:25.901742+0000","last_unstale":"2024-09-11T23:18:22.328869+0000","last_undegraded":"2024-09-11T23:18:22.328869+0000","last_fullsized":"2024-09-11T23:18:22.328869+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:02:10.640801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"85'4","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.329796+0000","last_change":"2024-09-11T23:16:25.898117+0000","last_active":"2024-09-11T23:18:22.329796+0000","last_peered":"2024-09-11T23:18:22.329796+0000","last_clean":"2024-09-11T23:18:22.329796+0000","last_became_active":"2024-09-11T23:16:25.897818+0000","last_became_peered":"2024-09-11T23:16:25.897818+0000","last_unstale":"2024-09-11T23:18:22.329796+0000","last_undegraded":"2024-09-11T23:18:22.329796+0000","last_fullsized":"2024-09-11T23:18:22.329796+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:27:37.175471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"85'7","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.544057+0000","last_change":"2024-09-11T23:16:25.905600+0000","last_active":"2024-09-11T23:18:22.544057+0000","last_peered":"2024-09-11T23:18:22.544057+0000","last_clean":"2024-09-11T23:18:22.544057+0000","last_became_active":"2024-09-11T23:16:25.903924+0000","last_became_peered":"2024-09-11T23:16:25.903924+0000","last_unstale":"2024-09-11T23:18:22.544057+0000","last_undegraded":"2024-09-11T23:18:22.544057+0000","last_fullsized":"2024-09-11T23:18:22.544057+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:20:29.929079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"85'8","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328449+0000","last_change":"2024-09-11T23:16:25.904763+0000","last_active":"2024-09-11T23:18:22.328449+0000","last_peered":"2024-09-11T23:18:22.328449+0000","last_clean":"2024-09-11T23:18:22.328449+0000","last_became_active":"2024-09-11T23:16:25.904406+0000","last_became_peered":"2024-09-11T23:16:25.904406+0000","last_unstale":"2024-09-11T23:18:22.328449+0000","last_undegraded":"2024-09-11T23:18:22.328449+0000","last_fullsized":"2024-09-11T23:18:22.328449+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:27:38.918017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"85'12","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328598+0000","last_change":"2024-09-11T23:16:25.906573+0000","last_active":"2024-09-11T23:18:22.328598+0000","last_peered":"2024-09-11T23:18:22.328598+0000","last_clean":"2024-09-11T23:18:22.328598+0000","last_became_active":"2024-09-11T23:16:25.904771+0000","last_became_peered":"2024-09-11T23:16:25.904771+0000","last_unstale":"2024-09-11T23:18:22.328598+0000","last_undegraded":"2024-09-11T23:18:22.328598+0000","last_fullsized":"2024-09-11T23:18:22.328598+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:04:17.090648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"85'9","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328348+0000","last_change":"2024-09-11T23:16:25.902123+0000","last_active":"2024-09-11T23:18:22.328348+0000","last_peered":"2024-09-11T23:18:22.328348+0000","last_clean":"2024-09-11T23:18:22.328348+0000","last_became_active":"2024-09-11T23:16:25.901756+0000","last_became_peered":"2024-09-11T23:16:25.901756+0000","last_unstale":"2024-09-11T23:18:22.328348+0000","last_undegraded":"2024-09-11T23:18:22.328348+0000","last_fullsized":"2024-09-11T23:18:22.328348+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:17:37.074976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"85'7","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.295802+0000","last_change":"2024-09-11T23:16:25.904217+0000","last_active":"2024-09-11T23:18:22.295802+0000","last_peered":"2024-09-11T23:18:22.295802+0000","last_clean":"2024-09-11T23:18:22.295802+0000","last_became_active":"2024-09-11T23:16:25.903522+0000","last_became_peered":"2024-09-11T23:16:25.903522+0000","last_unstale":"2024-09-11T23:18:22.295802+0000","last_undegraded":"2024-09-11T23:18:22.295802+0000","last_fullsized":"2024-09-11T23:18:22.295802+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:29:46.634475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"86'7","reported_seq":140,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:23.124462+0000","last_change":"2024-09-11T23:16:23.793696+0000","last_active":"2024-09-11T23:18:23.124462+0000","last_peered":"2024-09-11T23:18:23.124462+0000","last_clean":"2024-09-11T23:18:23.124462+0000","last_became_active":"2024-09-11T23:16:23.793062+0000","last_became_peered":"2024-09-11T23:16:23.793062+0000","last_unstale":"2024-09-11T23:18:23.124462+0000","last_undegraded":"2024-09-11T23:18:23.124462+0000","last_fullsized":"2024-09-11T23:18:23.124462+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:00:25.655817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1323,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":109,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"85'8","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.184453+0000","last_change":"2024-09-11T23:16:25.906673+0000","last_active":"2024-09-11T23:18:22.184453+0000","last_peered":"2024-09-11T23:18:22.184453+0000","last_clean":"2024-09-11T23:18:22.184453+0000","last_became_active":"2024-09-11T23:16:25.906239+0000","last_became_peered":"2024-09-11T23:16:25.906239+0000","last_unstale":"2024-09-11T23:18:22.184453+0000","last_undegraded":"2024-09-11T23:18:22.184453+0000","last_fullsized":"2024-09-11T23:18:22.184453+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:15:54.907269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"85'7","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.302905+0000","last_change":"2024-09-11T23:16:23.793302+0000","last_active":"2024-09-11T23:18:22.302905+0000","last_peered":"2024-09-11T23:18:22.302905+0000","last_clean":"2024-09-11T23:18:22.302905+0000","last_became_active":"2024-09-11T23:16:23.792874+0000","last_became_peered":"2024-09-11T23:16:23.792874+0000","last_unstale":"2024-09-11T23:18:22.302905+0000","last_undegraded":"2024-09-11T23:18:22.302905+0000","last_fullsized":"2024-09-11T23:18:22.302905+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:52:59.558223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"85'3","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328845+0000","last_change":"2024-09-11T23:16:23.792999+0000","last_active":"2024-09-11T23:18:22.328845+0000","last_peered":"2024-09-11T23:18:22.328845+0000","last_clean":"2024-09-11T23:18:22.328845+0000","last_became_active":"2024-09-11T23:16:23.792630+0000","last_became_peered":"2024-09-11T23:16:23.792630+0000","last_unstale":"2024-09-11T23:18:22.328845+0000","last_undegraded":"2024-09-11T23:18:22.328845+0000","last_fullsized":"2024-09-11T23:18:22.328845+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:09:28.001060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"85'8","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.292734+0000","last_change":"2024-09-11T23:16:35.943722+0000","last_active":"2024-09-11T23:18:22.292734+0000","last_peered":"2024-09-11T23:18:22.292734+0000","last_clean":"2024-09-11T23:18:22.292734+0000","last_became_active":"2024-09-11T23:16:35.943355+0000","last_became_peered":"2024-09-11T23:16:35.943355+0000","last_unstale":"2024-09-11T23:18:22.292734+0000","last_undegraded":"2024-09-11T23:18:22.292734+0000","last_fullsized":"2024-09-11T23:18:22.292734+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:26:22.769824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,6],"acting":[9,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"85'3","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.184646+0000","last_change":"2024-09-11T23:16:23.797945+0000","last_active":"2024-09-11T23:18:22.184646+0000","last_peered":"2024-09-11T23:18:22.184646+0000","last_clean":"2024-09-11T23:18:22.184646+0000","last_became_active":"2024-09-11T23:16:23.797572+0000","last_became_peered":"2024-09-11T23:16:23.797572+0000","last_unstale":"2024-09-11T23:18:22.184646+0000","last_undegraded":"2024-09-11T23:18:22.184646+0000","last_fullsized":"2024-09-11T23:18:22.184646+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:04:59.017668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"85'11","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.292877+0000","last_change":"2024-09-11T23:16:25.891587+0000","last_active":"2024-09-11T23:18:22.292877+0000","last_peered":"2024-09-11T23:18:22.292877+0000","last_clean":"2024-09-11T23:18:22.292877+0000","last_became_active":"2024-09-11T23:16:25.891190+0000","last_became_peered":"2024-09-11T23:16:25.891190+0000","last_unstale":"2024-09-11T23:18:22.292877+0000","last_undegraded":"2024-09-11T23:18:22.292877+0000","last_fullsized":"2024-09-11T23:18:22.292877+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:57:05.177843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"85'6","reported_seq":240,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:23.123749+0000","last_change":"2024-09-11T23:16:23.790129+0000","last_active":"2024-09-11T23:18:23.123749+0000","last_peered":"2024-09-11T23:18:23.123749+0000","last_clean":"2024-09-11T23:18:23.123749+0000","last_became_active":"2024-09-11T23:16:23.789720+0000","last_became_peered":"2024-09-11T23:16:23.789720+0000","last_unstale":"2024-09-11T23:18:23.123749+0000","last_undegraded":"2024-09-11T23:18:23.123749+0000","last_fullsized":"2024-09-11T23:18:23.123749+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:33:58.388513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":210,"num_read_kb":210,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"85'10","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.329142+0000","last_change":"2024-09-11T23:16:25.901014+0000","last_active":"2024-09-11T23:18:22.329142+0000","last_peered":"2024-09-11T23:18:22.329142+0000","last_clean":"2024-09-11T23:18:22.329142+0000","last_became_active":"2024-09-11T23:16:25.900280+0000","last_became_peered":"2024-09-11T23:16:25.900280+0000","last_unstale":"2024-09-11T23:18:22.329142+0000","last_undegraded":"2024-09-11T23:18:22.329142+0000","last_fullsized":"2024-09-11T23:18:22.329142+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:14:03.662204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"85'4","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328532+0000","last_change":"2024-09-11T23:16:23.797944+0000","last_active":"2024-09-11T23:18:22.328532+0000","last_peered":"2024-09-11T23:18:22.328532+0000","last_clean":"2024-09-11T23:18:22.328532+0000","last_became_active":"2024-09-11T23:16:23.797520+0000","last_became_peered":"2024-09-11T23:16:23.797520+0000","last_unstale":"2024-09-11T23:18:22.328532+0000","last_undegraded":"2024-09-11T23:18:22.328532+0000","last_fullsized":"2024-09-11T23:18:22.328532+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:29:06.957518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"85'9","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.544713+0000","last_change":"2024-09-11T23:16:25.905379+0000","last_active":"2024-09-11T23:18:22.544713+0000","last_peered":"2024-09-11T23:18:22.544713+0000","last_clean":"2024-09-11T23:18:22.544713+0000","last_became_active":"2024-09-11T23:16:25.905085+0000","last_became_peered":"2024-09-11T23:16:25.905085+0000","last_unstale":"2024-09-11T23:18:22.544713+0000","last_undegraded":"2024-09-11T23:18:22.544713+0000","last_fullsized":"2024-09-11T23:18:22.544713+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:22:35.129736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"85'5","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.184874+0000","last_change":"2024-09-11T23:16:24.075556+0000","last_active":"2024-09-11T23:18:22.184874+0000","last_peered":"2024-09-11T23:18:22.184874+0000","last_clean":"2024-09-11T23:18:22.184874+0000","last_became_active":"2024-09-11T23:16:24.074783+0000","last_became_peered":"2024-09-11T23:16:24.074783+0000","last_unstale":"2024-09-11T23:18:22.184874+0000","last_undegraded":"2024-09-11T23:18:22.184874+0000","last_fullsized":"2024-09-11T23:18:22.184874+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:55:09.962342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"85'11","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328166+0000","last_change":"2024-09-11T23:16:25.907720+0000","last_active":"2024-09-11T23:18:22.328166+0000","last_peered":"2024-09-11T23:18:22.328166+0000","last_clean":"2024-09-11T23:18:22.328166+0000","last_became_active":"2024-09-11T23:16:25.907335+0000","last_became_peered":"2024-09-11T23:16:25.907335+0000","last_unstale":"2024-09-11T23:18:22.328166+0000","last_undegraded":"2024-09-11T23:18:22.328166+0000","last_fullsized":"2024-09-11T23:18:22.328166+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:46:14.519384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"86'38","reported_seq":284,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:23.145509+0000","last_change":"2024-09-11T23:16:23.794321+0000","last_active":"2024-09-11T23:18:23.145509+0000","last_peered":"2024-09-11T23:18:23.145509+0000","last_clean":"2024-09-11T23:18:23.145509+0000","last_became_active":"2024-09-11T23:16:23.793955+0000","last_became_peered":"2024-09-11T23:16:23.793955+0000","last_unstale":"2024-09-11T23:18:23.145509+0000","last_undegraded":"2024-09-11T23:18:23.145509+0000","last_fullsized":"2024-09-11T23:18:23.145509+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:47:46.442656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":66480,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":222,"num_read_kb":175,"num_write":40,"num_write_kb":82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"85'6","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.296053+0000","last_change":"2024-09-11T23:16:25.896598+0000","last_active":"2024-09-11T23:18:22.296053+0000","last_peered":"2024-09-11T23:18:22.296053+0000","last_clean":"2024-09-11T23:18:22.296053+0000","last_became_active":"2024-09-11T23:16:25.896293+0000","last_became_peered":"2024-09-11T23:16:25.896293+0000","last_unstale":"2024-09-11T23:18:22.296053+0000","last_undegraded":"2024-09-11T23:18:22.296053+0000","last_fullsized":"2024-09-11T23:18:22.296053+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:04:11.489644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"85'9","reported_seq":35,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328677+0000","last_change":"2024-09-11T23:16:23.791486+0000","last_active":"2024-09-11T23:18:22.328677+0000","last_peered":"2024-09-11T23:18:22.328677+0000","last_clean":"2024-09-11T23:18:22.328677+0000","last_became_active":"2024-09-11T23:16:23.790786+0000","last_became_peered":"2024-09-11T23:16:23.790786+0000","last_unstale":"2024-09-11T23:18:22.328677+0000","last_undegraded":"2024-09-11T23:18:22.328677+0000","last_fullsized":"2024-09-11T23:18:22.328677+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:03:52.742560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"85'9","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.329171+0000","last_change":"2024-09-11T23:16:25.897382+0000","last_active":"2024-09-11T23:18:22.329171+0000","last_peered":"2024-09-11T23:18:22.329171+0000","last_clean":"2024-09-11T23:18:22.329171+0000","last_became_active":"2024-09-11T23:16:25.897073+0000","last_became_peered":"2024-09-11T23:16:25.897073+0000","last_unstale":"2024-09-11T23:18:22.329171+0000","last_undegraded":"2024-09-11T23:18:22.329171+0000","last_fullsized":"2024-09-11T23:18:22.329171+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:15:59.780100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"85'5","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.184727+0000","last_change":"2024-09-11T23:16:23.793886+0000","last_active":"2024-09-11T23:18:22.184727+0000","last_peered":"2024-09-11T23:18:22.184727+0000","last_clean":"2024-09-11T23:18:22.184727+0000","last_became_active":"2024-09-11T23:16:23.793066+0000","last_became_peered":"2024-09-11T23:16:23.793066+0000","last_unstale":"2024-09-11T23:18:22.184727+0000","last_undegraded":"2024-09-11T23:18:22.184727+0000","last_fullsized":"2024-09-11T23:18:22.184727+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:07:03.882546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"85'15","reported_seq":34,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328552+0000","last_change":"2024-09-11T23:16:25.904575+0000","last_active":"2024-09-11T23:18:22.328552+0000","last_peered":"2024-09-11T23:18:22.328552+0000","last_clean":"2024-09-11T23:18:22.328552+0000","last_became_active":"2024-09-11T23:16:25.904224+0000","last_became_peered":"2024-09-11T23:16:25.904224+0000","last_unstale":"2024-09-11T23:18:22.328552+0000","last_undegraded":"2024-09-11T23:18:22.328552+0000","last_fullsized":"2024-09-11T23:18:22.328552+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:42:13.036931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"85'5","reported_seq":136,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.187045+0000","last_change":"2024-09-11T23:16:23.796073+0000","last_active":"2024-09-11T23:18:22.187045+0000","last_peered":"2024-09-11T23:18:22.187045+0000","last_clean":"2024-09-11T23:18:22.187045+0000","last_became_active":"2024-09-11T23:16:23.794850+0000","last_became_peered":"2024-09-11T23:16:23.794850+0000","last_unstale":"2024-09-11T23:18:22.187045+0000","last_undegraded":"2024-09-11T23:18:22.187045+0000","last_fullsized":"2024-09-11T23:18:22.187045+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:57:22.361662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":108,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"85'6","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328221+0000","last_change":"2024-09-11T23:16:25.908700+0000","last_active":"2024-09-11T23:18:22.328221+0000","last_peered":"2024-09-11T23:18:22.328221+0000","last_clean":"2024-09-11T23:18:22.328221+0000","last_became_active":"2024-09-11T23:16:25.908373+0000","last_became_peered":"2024-09-11T23:16:25.908373+0000","last_unstale":"2024-09-11T23:18:22.328221+0000","last_undegraded":"2024-09-11T23:18:22.328221+0000","last_fullsized":"2024-09-11T23:18:22.328221+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:20:18.536079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"85'17","reported_seq":42,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.295792+0000","last_change":"2024-09-11T23:16:24.075927+0000","last_active":"2024-09-11T23:18:22.295792+0000","last_peered":"2024-09-11T23:18:22.295792+0000","last_clean":"2024-09-11T23:18:22.295792+0000","last_became_active":"2024-09-11T23:16:24.075527+0000","last_became_peered":"2024-09-11T23:16:24.075527+0000","last_unstale":"2024-09-11T23:18:22.295792+0000","last_undegraded":"2024-09-11T23:18:22.295792+0000","last_fullsized":"2024-09-11T23:18:22.295792+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:27:54.622059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"85'6","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.184380+0000","last_change":"2024-09-11T23:16:35.947475+0000","last_active":"2024-09-11T23:18:22.184380+0000","last_peered":"2024-09-11T23:18:22.184380+0000","last_clean":"2024-09-11T23:18:22.184380+0000","last_became_active":"2024-09-11T23:16:35.947028+0000","last_became_peered":"2024-09-11T23:16:35.947028+0000","last_unstale":"2024-09-11T23:18:22.184380+0000","last_undegraded":"2024-09-11T23:18:22.184380+0000","last_fullsized":"2024-09-11T23:18:22.184380+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:18:13.479554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'2","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.296198+0000","last_change":"2024-09-11T23:16:23.793298+0000","last_active":"2024-09-11T23:18:22.296198+0000","last_peered":"2024-09-11T23:18:22.296198+0000","last_clean":"2024-09-11T23:18:22.296198+0000","last_became_active":"2024-09-11T23:16:23.792947+0000","last_became_peered":"2024-09-11T23:16:23.792947+0000","last_unstale":"2024-09-11T23:18:22.296198+0000","last_undegraded":"2024-09-11T23:18:22.296198+0000","last_fullsized":"2024-09-11T23:18:22.296198+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:32:21.294560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"85'5","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328871+0000","last_change":"2024-09-11T23:16:25.902348+0000","last_active":"2024-09-11T23:18:22.328871+0000","last_peered":"2024-09-11T23:18:22.328871+0000","last_clean":"2024-09-11T23:18:22.328871+0000","last_became_active":"2024-09-11T23:16:25.901686+0000","last_became_peered":"2024-09-11T23:16:25.901686+0000","last_unstale":"2024-09-11T23:18:22.328871+0000","last_undegraded":"2024-09-11T23:18:22.328871+0000","last_fullsized":"2024-09-11T23:18:22.328871+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:00:15.543003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"85'5","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.330216+0000","last_change":"2024-09-11T23:16:23.790984+0000","last_active":"2024-09-11T23:18:22.330216+0000","last_peered":"2024-09-11T23:18:22.330216+0000","last_clean":"2024-09-11T23:18:22.330216+0000","last_became_active":"2024-09-11T23:16:23.790569+0000","last_became_peered":"2024-09-11T23:16:23.790569+0000","last_unstale":"2024-09-11T23:18:22.330216+0000","last_undegraded":"2024-09-11T23:18:22.330216+0000","last_fullsized":"2024-09-11T23:18:22.330216+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:57:39.675793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"85'9","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328796+0000","last_change":"2024-09-11T23:16:25.899249+0000","last_active":"2024-09-11T23:18:22.328796+0000","last_peered":"2024-09-11T23:18:22.328796+0000","last_clean":"2024-09-11T23:18:22.328796+0000","last_became_active":"2024-09-11T23:16:25.898914+0000","last_became_peered":"2024-09-11T23:16:25.898914+0000","last_unstale":"2024-09-11T23:18:22.328796+0000","last_undegraded":"2024-09-11T23:18:22.328796+0000","last_fullsized":"2024-09-11T23:18:22.328796+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:17:07.545770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"86'41","reported_seq":73,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.429528+0000","last_change":"2024-09-11T23:16:23.787387+0000","last_active":"2024-09-11T23:18:22.429528+0000","last_peered":"2024-09-11T23:18:22.429528+0000","last_clean":"2024-09-11T23:18:22.429528+0000","last_became_active":"2024-09-11T23:16:23.787014+0000","last_became_peered":"2024-09-11T23:16:23.787014+0000","last_unstale":"2024-09-11T23:18:22.429528+0000","last_undegraded":"2024-09-11T23:18:22.429528+0000","last_fullsized":"2024-09-11T23:18:22.429528+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:45:52.004253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"85'7","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328671+0000","last_change":"2024-09-11T23:16:25.908304+0000","last_active":"2024-09-11T23:18:22.328671+0000","last_peered":"2024-09-11T23:18:22.328671+0000","last_clean":"2024-09-11T23:18:22.328671+0000","last_became_active":"2024-09-11T23:16:25.907900+0000","last_became_peered":"2024-09-11T23:16:25.907900+0000","last_unstale":"2024-09-11T23:18:22.328671+0000","last_undegraded":"2024-09-11T23:18:22.328671+0000","last_fullsized":"2024-09-11T23:18:22.328671+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:43:26.247682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"85'7","reported_seq":140,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328822+0000","last_change":"2024-09-11T23:16:24.074610+0000","last_active":"2024-09-11T23:18:22.328822+0000","last_peered":"2024-09-11T23:18:22.328822+0000","last_clean":"2024-09-11T23:18:22.328822+0000","last_became_active":"2024-09-11T23:16:24.074210+0000","last_became_peered":"2024-09-11T23:16:24.074210+0000","last_unstale":"2024-09-11T23:18:22.328822+0000","last_undegraded":"2024-09-11T23:18:22.328822+0000","last_fullsized":"2024-09-11T23:18:22.328822+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:09:37.167050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1704,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":109,"num_read_kb":4,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"85'7","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328842+0000","last_change":"2024-09-11T23:16:25.903120+0000","last_active":"2024-09-11T23:18:22.328842+0000","last_peered":"2024-09-11T23:18:22.328842+0000","last_clean":"2024-09-11T23:18:22.328842+0000","last_became_active":"2024-09-11T23:16:25.902770+0000","last_became_peered":"2024-09-11T23:16:25.902770+0000","last_unstale":"2024-09-11T23:18:22.328842+0000","last_undegraded":"2024-09-11T23:18:22.328842+0000","last_fullsized":"2024-09-11T23:18:22.328842+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:03:30.041088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":349,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328487+0000","last_change":"2024-09-11T23:15:13.115055+0000","last_active":"2024-09-11T23:18:22.328487+0000","last_peered":"2024-09-11T23:18:22.328487+0000","last_clean":"2024-09-11T23:18:22.328487+0000","last_became_active":"2024-09-11T23:15:13.114480+0000","last_became_peered":"2024-09-11T23:15:13.114480+0000","last_unstale":"2024-09-11T23:18:22.328487+0000","last_undegraded":"2024-09-11T23:18:22.328487+0000","last_fullsized":"2024-09-11T23:18:22.328487+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:11:50.495799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:11:50.495799+0000","last_clean_scrub_stamp":"2024-09-11T23:11:50.495799+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:15:53.681230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"85'6","reported_seq":33,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.187094+0000","last_change":"2024-09-11T23:16:23.793996+0000","last_active":"2024-09-11T23:18:22.187094+0000","last_peered":"2024-09-11T23:18:22.187094+0000","last_clean":"2024-09-11T23:18:22.187094+0000","last_became_active":"2024-09-11T23:16:23.793681+0000","last_became_peered":"2024-09-11T23:16:23.793681+0000","last_unstale":"2024-09-11T23:18:22.187094+0000","last_undegraded":"2024-09-11T23:18:22.187094+0000","last_fullsized":"2024-09-11T23:18:22.187094+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:52:50.463623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"85'8","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.182924+0000","last_change":"2024-09-11T23:16:25.905133+0000","last_active":"2024-09-11T23:18:22.182924+0000","last_peered":"2024-09-11T23:18:22.182924+0000","last_clean":"2024-09-11T23:18:22.182924+0000","last_became_active":"2024-09-11T23:16:25.904819+0000","last_became_peered":"2024-09-11T23:16:25.904819+0000","last_unstale":"2024-09-11T23:18:22.182924+0000","last_undegraded":"2024-09-11T23:18:22.182924+0000","last_fullsized":"2024-09-11T23:18:22.182924+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:51:09.664343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"85'4","reported_seq":34,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328367+0000","last_change":"2024-09-11T23:16:36.228806+0000","last_active":"2024-09-11T23:18:22.328367+0000","last_peered":"2024-09-11T23:18:22.328367+0000","last_clean":"2024-09-11T23:18:22.328367+0000","last_became_active":"2024-09-11T23:16:36.228311+0000","last_became_peered":"2024-09-11T23:16:36.228311+0000","last_unstale":"2024-09-11T23:18:22.328367+0000","last_undegraded":"2024-09-11T23:18:22.328367+0000","last_fullsized":"2024-09-11T23:18:22.328367+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:42:32.856691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4],"acting":[7,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"85'5","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328703+0000","last_change":"2024-09-11T23:16:25.898935+0000","last_active":"2024-09-11T23:18:22.328703+0000","last_peered":"2024-09-11T23:18:22.328703+0000","last_clean":"2024-09-11T23:18:22.328703+0000","last_became_active":"2024-09-11T23:16:25.898618+0000","last_became_peered":"2024-09-11T23:16:25.898618+0000","last_unstale":"2024-09-11T23:18:22.328703+0000","last_undegraded":"2024-09-11T23:18:22.328703+0000","last_fullsized":"2024-09-11T23:18:22.328703+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:13:29.104911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"85'5","reported_seq":135,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.329013+0000","last_change":"2024-09-11T23:16:24.073613+0000","last_active":"2024-09-11T23:18:22.329013+0000","last_peered":"2024-09-11T23:18:22.329013+0000","last_clean":"2024-09-11T23:18:22.329013+0000","last_became_active":"2024-09-11T23:16:24.073262+0000","last_became_peered":"2024-09-11T23:16:24.073262+0000","last_unstale":"2024-09-11T23:18:22.329013+0000","last_undegraded":"2024-09-11T23:18:22.329013+0000","last_fullsized":"2024-09-11T23:18:22.329013+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:31:58.653657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":106,"num_read_kb":106,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"85'8","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.184479+0000","last_change":"2024-09-11T23:16:25.903725+0000","last_active":"2024-09-11T23:18:22.184479+0000","last_peered":"2024-09-11T23:18:22.184479+0000","last_clean":"2024-09-11T23:18:22.184479+0000","last_became_active":"2024-09-11T23:16:25.903375+0000","last_became_peered":"2024-09-11T23:16:25.903375+0000","last_unstale":"2024-09-11T23:18:22.184479+0000","last_undegraded":"2024-09-11T23:18:22.184479+0000","last_fullsized":"2024-09-11T23:18:22.184479+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:55:02.779575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"86'6","reported_seq":137,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:23.123405+0000","last_change":"2024-09-11T23:16:23.793932+0000","last_active":"2024-09-11T23:18:23.123405+0000","last_peered":"2024-09-11T23:18:23.123405+0000","last_clean":"2024-09-11T23:18:23.123405+0000","last_became_active":"2024-09-11T23:16:23.793111+0000","last_became_peered":"2024-09-11T23:16:23.793111+0000","last_unstale":"2024-09-11T23:18:23.123405+0000","last_undegraded":"2024-09-11T23:18:23.123405+0000","last_fullsized":"2024-09-11T23:18:23.123405+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:14:09.087222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":107,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"85'7","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.302982+0000","last_change":"2024-09-11T23:16:25.905020+0000","last_active":"2024-09-11T23:18:22.302982+0000","last_peered":"2024-09-11T23:18:22.302982+0000","last_clean":"2024-09-11T23:18:22.302982+0000","last_became_active":"2024-09-11T23:16:25.904675+0000","last_became_peered":"2024-09-11T23:16:25.904675+0000","last_unstale":"2024-09-11T23:18:22.302982+0000","last_undegraded":"2024-09-11T23:18:22.302982+0000","last_fullsized":"2024-09-11T23:18:22.302982+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:13:57.388773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"85'1","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.293261+0000","last_change":"2024-09-11T23:16:24.073630+0000","last_active":"2024-09-11T23:18:22.293261+0000","last_peered":"2024-09-11T23:18:22.293261+0000","last_clean":"2024-09-11T23:18:22.293261+0000","last_became_active":"2024-09-11T23:16:24.073301+0000","last_became_peered":"2024-09-11T23:16:24.073301+0000","last_unstale":"2024-09-11T23:18:22.293261+0000","last_undegraded":"2024-09-11T23:18:22.293261+0000","last_fullsized":"2024-09-11T23:18:22.293261+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:41:59.374581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"85'5","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.303051+0000","last_change":"2024-09-11T23:16:25.894124+0000","last_active":"2024-09-11T23:18:22.303051+0000","last_peered":"2024-09-11T23:18:22.303051+0000","last_clean":"2024-09-11T23:18:22.303051+0000","last_became_active":"2024-09-11T23:16:25.893751+0000","last_became_peered":"2024-09-11T23:16:25.893751+0000","last_unstale":"2024-09-11T23:18:22.303051+0000","last_undegraded":"2024-09-11T23:18:22.303051+0000","last_fullsized":"2024-09-11T23:18:22.303051+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:00:28.265206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"85'4","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.296214+0000","last_change":"2024-09-11T23:16:24.073700+0000","last_active":"2024-09-11T23:18:22.296214+0000","last_peered":"2024-09-11T23:18:22.296214+0000","last_clean":"2024-09-11T23:18:22.296214+0000","last_became_active":"2024-09-11T23:16:24.073283+0000","last_became_peered":"2024-09-11T23:16:24.073283+0000","last_unstale":"2024-09-11T23:18:22.296214+0000","last_undegraded":"2024-09-11T23:18:22.296214+0000","last_fullsized":"2024-09-11T23:18:22.296214+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:41:43.536834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"85'9","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.329920+0000","last_change":"2024-09-11T23:16:25.900532+0000","last_active":"2024-09-11T23:18:22.329920+0000","last_peered":"2024-09-11T23:18:22.329920+0000","last_clean":"2024-09-11T23:18:22.329920+0000","last_became_active":"2024-09-11T23:16:25.899466+0000","last_became_peered":"2024-09-11T23:16:25.899466+0000","last_unstale":"2024-09-11T23:18:22.329920+0000","last_undegraded":"2024-09-11T23:18:22.329920+0000","last_fullsized":"2024-09-11T23:18:22.329920+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:10:01.152323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"86'50","reported_seq":97,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.432609+0000","last_change":"2024-09-11T23:16:23.788789+0000","last_active":"2024-09-11T23:18:22.432609+0000","last_peered":"2024-09-11T23:18:22.432609+0000","last_clean":"2024-09-11T23:18:22.432609+0000","last_became_active":"2024-09-11T23:16:23.788422+0000","last_became_peered":"2024-09-11T23:16:23.788422+0000","last_unstale":"2024-09-11T23:18:22.432609+0000","last_undegraded":"2024-09-11T23:18:22.432609+0000","last_fullsized":"2024-09-11T23:18:22.432609+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":0,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:09:14.310205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":89,"num_write_kb":837,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"85'5","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.183270+0000","last_change":"2024-09-11T23:16:25.895511+0000","last_active":"2024-09-11T23:18:22.183270+0000","last_peered":"2024-09-11T23:18:22.183270+0000","last_clean":"2024-09-11T23:18:22.183270+0000","last_became_active":"2024-09-11T23:16:25.895143+0000","last_became_peered":"2024-09-11T23:16:25.895143+0000","last_unstale":"2024-09-11T23:18:22.183270+0000","last_undegraded":"2024-09-11T23:18:22.183270+0000","last_fullsized":"2024-09-11T23:18:22.183270+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:31:16.498371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"85'7","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.187209+0000","last_change":"2024-09-11T23:16:24.019064+0000","last_active":"2024-09-11T23:18:22.187209+0000","last_peered":"2024-09-11T23:18:22.187209+0000","last_clean":"2024-09-11T23:18:22.187209+0000","last_became_active":"2024-09-11T23:16:24.018658+0000","last_became_peered":"2024-09-11T23:16:24.018658+0000","last_unstale":"2024-09-11T23:18:22.187209+0000","last_undegraded":"2024-09-11T23:18:22.187209+0000","last_fullsized":"2024-09-11T23:18:22.187209+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:17:41.009507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"85'7","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.544339+0000","last_change":"2024-09-11T23:16:25.903553+0000","last_active":"2024-09-11T23:18:22.544339+0000","last_peered":"2024-09-11T23:18:22.544339+0000","last_clean":"2024-09-11T23:18:22.544339+0000","last_became_active":"2024-09-11T23:16:25.903221+0000","last_became_peered":"2024-09-11T23:16:25.903221+0000","last_unstale":"2024-09-11T23:18:22.544339+0000","last_undegraded":"2024-09-11T23:18:22.544339+0000","last_fullsized":"2024-09-11T23:18:22.544339+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:16:51.872517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"83'2","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328140+0000","last_change":"2024-09-11T23:16:24.073784+0000","last_active":"2024-09-11T23:18:22.328140+0000","last_peered":"2024-09-11T23:18:22.328140+0000","last_clean":"2024-09-11T23:18:22.328140+0000","last_became_active":"2024-09-11T23:16:24.073406+0000","last_became_peered":"2024-09-11T23:16:24.073406+0000","last_unstale":"2024-09-11T23:18:22.328140+0000","last_undegraded":"2024-09-11T23:18:22.328140+0000","last_fullsized":"2024-09-11T23:18:22.328140+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:31:24.617025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"85'2","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.295796+0000","last_change":"2024-09-11T23:16:25.894878+0000","last_active":"2024-09-11T23:18:22.295796+0000","last_peered":"2024-09-11T23:18:22.295796+0000","last_clean":"2024-09-11T23:18:22.295796+0000","last_became_active":"2024-09-11T23:16:25.894543+0000","last_became_peered":"2024-09-11T23:16:25.894543+0000","last_unstale":"2024-09-11T23:18:22.295796+0000","last_undegraded":"2024-09-11T23:18:22.295796+0000","last_fullsized":"2024-09-11T23:18:22.295796+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:34:13.435754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"85'7","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.182832+0000","last_change":"2024-09-11T23:16:23.786502+0000","last_active":"2024-09-11T23:18:22.182832+0000","last_peered":"2024-09-11T23:18:22.182832+0000","last_clean":"2024-09-11T23:18:22.182832+0000","last_became_active":"2024-09-11T23:16:23.786182+0000","last_became_peered":"2024-09-11T23:16:23.786182+0000","last_unstale":"2024-09-11T23:18:22.182832+0000","last_undegraded":"2024-09-11T23:18:22.182832+0000","last_fullsized":"2024-09-11T23:18:22.182832+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:59:07.978244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"85'9","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328520+0000","last_change":"2024-09-11T23:16:25.899625+0000","last_active":"2024-09-11T23:18:22.328520+0000","last_peered":"2024-09-11T23:18:22.328520+0000","last_clean":"2024-09-11T23:18:22.328520+0000","last_became_active":"2024-09-11T23:16:25.899265+0000","last_became_peered":"2024-09-11T23:16:25.899265+0000","last_unstale":"2024-09-11T23:18:22.328520+0000","last_undegraded":"2024-09-11T23:18:22.328520+0000","last_fullsized":"2024-09-11T23:18:22.328520+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:36:06.907622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"86'4","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328472+0000","last_change":"2024-09-11T23:16:36.228991+0000","last_active":"2024-09-11T23:18:22.328472+0000","last_peered":"2024-09-11T23:18:22.328472+0000","last_clean":"2024-09-11T23:18:22.328472+0000","last_became_active":"2024-09-11T23:16:36.227701+0000","last_became_peered":"2024-09-11T23:16:36.227701+0000","last_unstale":"2024-09-11T23:18:22.328472+0000","last_undegraded":"2024-09-11T23:18:22.328472+0000","last_fullsized":"2024-09-11T23:18:22.328472+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:37:57.478572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"85'8","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.182913+0000","last_change":"2024-09-11T23:16:25.894803+0000","last_active":"2024-09-11T23:18:22.182913+0000","last_peered":"2024-09-11T23:18:22.182913+0000","last_clean":"2024-09-11T23:18:22.182913+0000","last_became_active":"2024-09-11T23:16:25.894429+0000","last_became_peered":"2024-09-11T23:16:25.894429+0000","last_unstale":"2024-09-11T23:18:22.182913+0000","last_undegraded":"2024-09-11T23:18:22.182913+0000","last_fullsized":"2024-09-11T23:18:22.182913+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:39:15.868910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"83'4","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.295871+0000","last_change":"2024-09-11T23:16:23.794271+0000","last_active":"2024-09-11T23:18:22.295871+0000","last_peered":"2024-09-11T23:18:22.295871+0000","last_clean":"2024-09-11T23:18:22.295871+0000","last_became_active":"2024-09-11T23:16:23.793919+0000","last_became_peered":"2024-09-11T23:16:23.793919+0000","last_unstale":"2024-09-11T23:18:22.295871+0000","last_undegraded":"2024-09-11T23:18:22.295871+0000","last_fullsized":"2024-09-11T23:18:22.295871+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:30:28.907312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"85'5","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328395+0000","last_change":"2024-09-11T23:16:25.900969+0000","last_active":"2024-09-11T23:18:22.328395+0000","last_peered":"2024-09-11T23:18:22.328395+0000","last_clean":"2024-09-11T23:18:22.328395+0000","last_became_active":"2024-09-11T23:16:25.900425+0000","last_became_peered":"2024-09-11T23:16:25.900425+0000","last_unstale":"2024-09-11T23:18:22.328395+0000","last_undegraded":"2024-09-11T23:18:22.328395+0000","last_fullsized":"2024-09-11T23:18:22.328395+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:03:25.185773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"85'15","reported_seq":34,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.184711+0000","last_change":"2024-09-11T23:16:25.911605+0000","last_active":"2024-09-11T23:18:22.184711+0000","last_peered":"2024-09-11T23:18:22.184711+0000","last_clean":"2024-09-11T23:18:22.184711+0000","last_became_active":"2024-09-11T23:16:25.911180+0000","last_became_peered":"2024-09-11T23:16:25.911180+0000","last_unstale":"2024-09-11T23:18:22.184711+0000","last_undegraded":"2024-09-11T23:18:22.184711+0000","last_fullsized":"2024-09-11T23:18:22.184711+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:55:38.062606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"85'5","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328425+0000","last_change":"2024-09-11T23:16:35.949528+0000","last_active":"2024-09-11T23:18:22.328425+0000","last_peered":"2024-09-11T23:18:22.328425+0000","last_clean":"2024-09-11T23:18:22.328425+0000","last_became_active":"2024-09-11T23:16:35.945679+0000","last_became_peered":"2024-09-11T23:16:35.945679+0000","last_unstale":"2024-09-11T23:18:22.328425+0000","last_undegraded":"2024-09-11T23:18:22.328425+0000","last_fullsized":"2024-09-11T23:18:22.328425+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:14:24.706044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"85'9","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328158+0000","last_change":"2024-09-11T23:16:25.900526+0000","last_active":"2024-09-11T23:18:22.328158+0000","last_peered":"2024-09-11T23:18:22.328158+0000","last_clean":"2024-09-11T23:18:22.328158+0000","last_became_active":"2024-09-11T23:16:25.900148+0000","last_became_peered":"2024-09-11T23:16:25.900148+0000","last_unstale":"2024-09-11T23:18:22.328158+0000","last_undegraded":"2024-09-11T23:18:22.328158+0000","last_fullsized":"2024-09-11T23:18:22.328158+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:26:10.582473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"86'13","reported_seq":48,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.296144+0000","last_change":"2024-09-11T23:16:36.233273+0000","last_active":"2024-09-11T23:18:22.296144+0000","last_peered":"2024-09-11T23:18:22.296144+0000","last_clean":"2024-09-11T23:18:22.296144+0000","last_became_active":"2024-09-11T23:16:36.228358+0000","last_became_peered":"2024-09-11T23:16:36.228358+0000","last_unstale":"2024-09-11T23:18:22.296144+0000","last_undegraded":"2024-09-11T23:18:22.296144+0000","last_fullsized":"2024-09-11T23:18:22.296144+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:06:31.225795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"85'6","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.544571+0000","last_change":"2024-09-11T23:16:25.907874+0000","last_active":"2024-09-11T23:18:22.544571+0000","last_peered":"2024-09-11T23:18:22.544571+0000","last_clean":"2024-09-11T23:18:22.544571+0000","last_became_active":"2024-09-11T23:16:25.907491+0000","last_became_peered":"2024-09-11T23:16:25.907491+0000","last_unstale":"2024-09-11T23:18:22.544571+0000","last_undegraded":"2024-09-11T23:18:22.544571+0000","last_fullsized":"2024-09-11T23:18:22.544571+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:09:20.984249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"84'2","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.183287+0000","last_change":"2024-09-11T23:16:24.074355+0000","last_active":"2024-09-11T23:18:22.183287+0000","last_peered":"2024-09-11T23:18:22.183287+0000","last_clean":"2024-09-11T23:18:22.183287+0000","last_became_active":"2024-09-11T23:16:24.074047+0000","last_became_peered":"2024-09-11T23:16:24.074047+0000","last_unstale":"2024-09-11T23:18:22.183287+0000","last_undegraded":"2024-09-11T23:18:22.183287+0000","last_fullsized":"2024-09-11T23:18:22.183287+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:31:58.804420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"85'5","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.182923+0000","last_change":"2024-09-11T23:16:25.907701+0000","last_active":"2024-09-11T23:18:22.182923+0000","last_peered":"2024-09-11T23:18:22.182923+0000","last_clean":"2024-09-11T23:18:22.182923+0000","last_became_active":"2024-09-11T23:16:25.907379+0000","last_became_peered":"2024-09-11T23:16:25.907379+0000","last_unstale":"2024-09-11T23:18:22.182923+0000","last_undegraded":"2024-09-11T23:18:22.182923+0000","last_fullsized":"2024-09-11T23:18:22.182923+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:12:03.639055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"86'14","reported_seq":144,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.497378+0000","last_change":"2024-09-11T23:16:23.794293+0000","last_active":"2024-09-11T23:18:22.497378+0000","last_peered":"2024-09-11T23:18:22.497378+0000","last_clean":"2024-09-11T23:18:22.497378+0000","last_became_active":"2024-09-11T23:16:23.793974+0000","last_became_peered":"2024-09-11T23:16:23.793974+0000","last_unstale":"2024-09-11T23:18:22.497378+0000","last_undegraded":"2024-09-11T23:18:22.497378+0000","last_fullsized":"2024-09-11T23:18:22.497378+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:24:39.170188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":106,"num_read_kb":106,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":599346,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1189,"num_read_kb":709,"num_write":372,"num_write_kb":1462,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2174976,"data_stored":1798038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":298,"ondisk_log_size":298,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":11,"up_from":77,"seq":330712481829,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32128,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713024,"statfs":{"total":95995035648,"available":95962136576,"internally_reserved":0,"allocated":962560,"data_stored":459042,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":70,"seq":300647710761,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28844,"kb_used_data":1760,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716308,"statfs":{"total":95995035648,"available":95965499392,"internally_reserved":0,"allocated":1802240,"data_stored":1247035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":65,"seq":279172874287,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28372,"kb_used_data":1344,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716780,"statfs":{"total":95995035648,"available":95965982720,"internally_reserved":0,"allocated":1376256,"data_stored":851072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":57,"seq":244813135924,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":958464,"data_stored":448696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":50,"seq":214748364856,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28568,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716584,"statfs":{"total":95995035648,"available":95965782016,"internally_reserved":0,"allocated":1576960,"data_stored":1051455,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528380,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36292,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708860,"statfs":{"total":95995035648,"available":95957872640,"internally_reserved":0,"allocated":966656,"data_stored":457966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790019,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28768,"kb_used_data":1684,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716384,"statfs":{"total":95995035648,"available":95965577216,"internally_reserved":0,"allocated":1724416,"data_stored":1182340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920840,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36292,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708860,"statfs":{"total":95995035648,"available":95957872640,"internally_reserved":0,"allocated":966656,"data_stored":457434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182477,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":41160,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93703992,"statfs":{"total":95995035648,"available":95952887808,"internally_reserved":0,"allocated":1511424,"data_stored":982214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411410,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36380,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708772,"statfs":{"total":95995035648,"available":95957782528,"internally_reserved":0,"allocated":1056768,"data_stored":526421,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574935,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1036288,"data_stored":525359,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738460,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":970752,"data_stored":450490,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":68625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":69687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":557056,"data_stored":525246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":180224,"data_stored":135121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":421888,"data_stored":394134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":394338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":258048,"data_stored":199816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2308,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T23:18:25.252 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T23:18:25.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:25.398+0000 7fe67f3bd640 1 -- 172.21.15.26:0/1850877967 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe678101c00 msgr2=0x7fe678102080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:25.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:25.398+0000 7fe67f3bd640 1 --2- 172.21.15.26:0/1850877967 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe678101c00 0x7fe678102080 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fe668009900 tx=0x7fe66802a410 comp rx=0 tx=0).stop 2024-09-11T23:18:25.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:25.398+0000 7fe67f3bd640 1 -- 172.21.15.26:0/1850877967 shutdown_connections 2024-09-11T23:18:25.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:25.398+0000 7fe67f3bd640 1 --2- 172.21.15.26:0/1850877967 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe6781025c0 0x7fe678108e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:25.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:25.398+0000 7fe67f3bd640 1 --2- 172.21.15.26:0/1850877967 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe678101c00 0x7fe678102080 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:25.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:25.398+0000 7fe67f3bd640 1 --2- 172.21.15.26:0/1850877967 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe678100a10 0x7fe678100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:25.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:25.398+0000 7fe67f3bd640 1 -- 172.21.15.26:0/1850877967 >> 172.21.15.26:0/1850877967 conn(0x7fe6780fc170 msgr2=0x7fe6780fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:18:25.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:25.398+0000 7fe67f3bd640 1 -- 172.21.15.26:0/1850877967 shutdown_connections 2024-09-11T23:18:25.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:25.398+0000 7fe67f3bd640 1 -- 172.21.15.26:0/1850877967 wait complete. 2024-09-11T23:18:25.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:25.399+0000 7fe67f3bd640 1 Processor -- start 2024-09-11T23:18:25.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:25.399+0000 7fe67f3bd640 1 -- start start 2024-09-11T23:18:25.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:25.399+0000 7fe67f3bd640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe678100a10 0x7fe678197e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:25.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:25.399+0000 7fe67f3bd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe678101c00 0x7fe678198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:25.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:25.399+0000 7fe67f3bd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe6781025c0 0x7fe67819f400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:25.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:25.399+0000 7fe67f3bd640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fe67810c670 con 0x7fe6781025c0 2024-09-11T23:18:25.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:25.399+0000 7fe67f3bd640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fe67810c4f0 con 0x7fe678101c00 2024-09-11T23:18:25.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:25.399+0000 7fe67f3bd640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe67810c7f0 con 0x7fe678100a10 2024-09-11T23:18:25.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:25.399+0000 7fe67d95a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe6781025c0 0x7fe67819f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:25.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:25.399+0000 7fe67d95a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe6781025c0 0x7fe67819f400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47380/0 (socket says 172.21.15.26:47380) 2024-09-11T23:18:25.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:25.399+0000 7fe67d95a640 1 -- 172.21.15.26:0/2860248745 learned_addr learned my addr 172.21.15.26:0/2860248745 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:18:25.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:25.400+0000 7fe67d159640 1 --2- 172.21.15.26:0/2860248745 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe678100a10 0x7fe678197e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:25.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:25.400+0000 7fe67c958640 1 --2- 172.21.15.26:0/2860248745 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe678101c00 0x7fe678198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:25.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:25.400+0000 7fe67d95a640 1 -- 172.21.15.26:0/2860248745 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe678100a10 msgr2=0x7fe678197e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:25.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:25.400+0000 7fe67d95a640 1 --2- 172.21.15.26:0/2860248745 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe678100a10 0x7fe678197e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:25.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:25.400+0000 7fe67d95a640 1 -- 172.21.15.26:0/2860248745 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe678101c00 msgr2=0x7fe678198390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:25.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:25.400+0000 7fe67d95a640 1 --2- 172.21.15.26:0/2860248745 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe678101c00 0x7fe678198390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:25.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:25.400+0000 7fe67d95a640 1 -- 172.21.15.26:0/2860248745 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe67819fa70 con 0x7fe6781025c0 2024-09-11T23:18:25.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:25.400+0000 7fe67d95a640 1 --2- 172.21.15.26:0/2860248745 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe6781025c0 0x7fe67819f400 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fe67000eaa0 tx=0x7fe67000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:18:25.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:25.400+0000 7fe66e7fc640 1 -- 172.21.15.26:0/2860248745 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe67000cdb0 con 0x7fe6781025c0 2024-09-11T23:18:25.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:25.400+0000 7fe66e7fc640 1 -- 172.21.15.26:0/2860248745 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe670005020 con 0x7fe6781025c0 2024-09-11T23:18:25.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:25.400+0000 7fe67f3bd640 1 -- 172.21.15.26:0/2860248745 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe67819fd60 con 0x7fe6781025c0 2024-09-11T23:18:25.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:25.400+0000 7fe66e7fc640 1 -- 172.21.15.26:0/2860248745 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6700116b0 con 0x7fe6781025c0 2024-09-11T23:18:25.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:25.400+0000 7fe67f3bd640 1 -- 172.21.15.26:0/2860248745 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6781a0220 con 0x7fe6781025c0 2024-09-11T23:18:25.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:25.401+0000 7fe67f3bd640 1 -- 172.21.15.26:0/2860248745 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6781a04b0 con 0x7fe6781025c0 2024-09-11T23:18:25.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:25.402+0000 7fe66e7fc640 1 -- 172.21.15.26:0/2860248745 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 23) ==== 137953+0+0 (secure 0 0 0) 0x7fe670005560 con 0x7fe6781025c0 2024-09-11T23:18:25.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:25.403+0000 7fe66e7fc640 1 --2- 172.21.15.26:0/2860248745 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fe6640a2d80 0x7fe6640a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:25.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:25.403+0000 7fe67d159640 1 --2- 172.21.15.26:0/2860248745 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fe6640a2d80 0x7fe6640a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:25.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:25.404+0000 7fe66e7fc640 1 -- 172.21.15.26:0/2860248745 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fe6700cf700 con 0x7fe6781025c0 2024-09-11T23:18:25.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:25.404+0000 7fe67d159640 1 --2- 172.21.15.26:0/2860248745 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fe6640a2d80 0x7fe6640a5240 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fe674005db0 tx=0x7fe674005d40 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:18:25.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:25.407+0000 7fe66e7fc640 1 -- 172.21.15.26:0/2860248745 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe670016030 con 0x7fe6781025c0 2024-09-11T23:18:25.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:25 smithi136 ceph-mon[35121]: [11/Sep/2024:23:18:24] ENGINE Serving on https://172.21.15.26:7150 2024-09-11T23:18:25.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:25 smithi136 ceph-mon[35121]: [11/Sep/2024:23:18:24] ENGINE Bus STARTED 2024-09-11T23:18:25.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:25 smithi136 ceph-mon[35121]: [11/Sep/2024:23:18:24] ENGINE Error in HTTPServer.serve 2024-09-11T23:18:25.489 INFO:journalctl@ceph.mon.c.smithi136.stdout: Traceback (most recent call last): 2024-09-11T23:18:25.490 INFO:journalctl@ceph.mon.c.smithi136.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-11T23:18:25.490 INFO:journalctl@ceph.mon.c.smithi136.stdout: self._connections.run(self.expiration_interval) 2024-09-11T23:18:25.490 INFO:journalctl@ceph.mon.c.smithi136.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-11T23:18:25.490 INFO:journalctl@ceph.mon.c.smithi136.stdout: self._run(expiration_interval) 2024-09-11T23:18:25.490 INFO:journalctl@ceph.mon.c.smithi136.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-11T23:18:25.490 INFO:journalctl@ceph.mon.c.smithi136.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-11T23:18:25.490 INFO:journalctl@ceph.mon.c.smithi136.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-11T23:18:25.490 INFO:journalctl@ceph.mon.c.smithi136.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-11T23:18:25.490 INFO:journalctl@ceph.mon.c.smithi136.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-11T23:18:25.490 INFO:journalctl@ceph.mon.c.smithi136.stdout: s = self.context.wrap_socket( 2024-09-11T23:18:25.490 INFO:journalctl@ceph.mon.c.smithi136.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-11T23:18:25.490 INFO:journalctl@ceph.mon.c.smithi136.stdout: return self.sslsocket_class._create( 2024-09-11T23:18:25.490 INFO:journalctl@ceph.mon.c.smithi136.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-11T23:18:25.490 INFO:journalctl@ceph.mon.c.smithi136.stdout: self.do_handshake() 2024-09-11T23:18:25.491 INFO:journalctl@ceph.mon.c.smithi136.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-11T23:18:25.491 INFO:journalctl@ceph.mon.c.smithi136.stdout: self._sslobj.do_handshake() 2024-09-11T23:18:25.491 INFO:journalctl@ceph.mon.c.smithi136.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-11T23:18:25.491 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:25 smithi136 ceph-mon[35121]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:18:25.491 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:25 smithi136 ceph-mon[35121]: mgrmap e23: x(active, since 2s), standbys: y, z 2024-09-11T23:18:25.491 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:25 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:18:25.491 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:25 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:18:25.491 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:25 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:18:25.491 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:25 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:18:25.491 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:25 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:18:25.491 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:25 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:18:25.491 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:25 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:18:25.491 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:25 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:18:25.491 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:25 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:18:25.492 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:25 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:18:25.492 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:25 smithi136 ceph-mon[35121]: Adjusting osd_memory_target on smithi076 to 176.0M 2024-09-11T23:18:25.492 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:25 smithi136 ceph-mon[35121]: Unable to set osd_memory_target on smithi076 to 184591052: error parsing value: Value '184591052' is below minimum 939524096 2024-09-11T23:18:25.492 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:25 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:18:25.492 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:25 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:18:25.492 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:25 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:18:25.492 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:25 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:18:25.492 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:25 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:18:25.492 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:25 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:18:25.492 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:25 smithi136 ceph-mon[35121]: Adjusting osd_memory_target on smithi136 to 176.0M 2024-09-11T23:18:25.492 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:25 smithi136 ceph-mon[35121]: Unable to set osd_memory_target on smithi136 to 184574566: error parsing value: Value '184574566' is below minimum 939524096 2024-09-11T23:18:25.501 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:25 smithi026 ceph-mon[31172]: [11/Sep/2024:23:18:24] ENGINE Serving on https://172.21.15.26:7150 2024-09-11T23:18:25.501 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:25 smithi026 ceph-mon[31172]: [11/Sep/2024:23:18:24] ENGINE Bus STARTED 2024-09-11T23:18:25.501 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:25 smithi026 ceph-mon[31172]: [11/Sep/2024:23:18:24] ENGINE Error in HTTPServer.serve 2024-09-11T23:18:25.501 INFO:journalctl@ceph.mon.a.smithi026.stdout: Traceback (most recent call last): 2024-09-11T23:18:25.501 INFO:journalctl@ceph.mon.a.smithi026.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-11T23:18:25.501 INFO:journalctl@ceph.mon.a.smithi026.stdout: self._connections.run(self.expiration_interval) 2024-09-11T23:18:25.502 INFO:journalctl@ceph.mon.a.smithi026.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-11T23:18:25.502 INFO:journalctl@ceph.mon.a.smithi026.stdout: self._run(expiration_interval) 2024-09-11T23:18:25.502 INFO:journalctl@ceph.mon.a.smithi026.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-11T23:18:25.502 INFO:journalctl@ceph.mon.a.smithi026.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-11T23:18:25.502 INFO:journalctl@ceph.mon.a.smithi026.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-11T23:18:25.502 INFO:journalctl@ceph.mon.a.smithi026.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-11T23:18:25.502 INFO:journalctl@ceph.mon.a.smithi026.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-11T23:18:25.502 INFO:journalctl@ceph.mon.a.smithi026.stdout: s = self.context.wrap_socket( 2024-09-11T23:18:25.502 INFO:journalctl@ceph.mon.a.smithi026.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-11T23:18:25.502 INFO:journalctl@ceph.mon.a.smithi026.stdout: return self.sslsocket_class._create( 2024-09-11T23:18:25.502 INFO:journalctl@ceph.mon.a.smithi026.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-11T23:18:25.502 INFO:journalctl@ceph.mon.a.smithi026.stdout: self.do_handshake() 2024-09-11T23:18:25.502 INFO:journalctl@ceph.mon.a.smithi026.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-11T23:18:25.503 INFO:journalctl@ceph.mon.a.smithi026.stdout: self._sslobj.do_handshake() 2024-09-11T23:18:25.503 INFO:journalctl@ceph.mon.a.smithi026.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-11T23:18:25.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:25 smithi026 ceph-mon[31172]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:18:25.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:25 smithi026 ceph-mon[31172]: mgrmap e23: x(active, since 2s), standbys: y, z 2024-09-11T23:18:25.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:25 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:18:25.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:25 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:18:25.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:25 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:18:25.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:25 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:18:25.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:25 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:18:25.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:25 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:18:25.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:25 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:18:25.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:25 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:18:25.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:25 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:18:25.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:25 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:18:25.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:25 smithi026 ceph-mon[31172]: Adjusting osd_memory_target on smithi076 to 176.0M 2024-09-11T23:18:25.504 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:25 smithi026 ceph-mon[31172]: Unable to set osd_memory_target on smithi076 to 184591052: error parsing value: Value '184591052' is below minimum 939524096 2024-09-11T23:18:25.504 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:25 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:18:25.504 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:25 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:18:25.504 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:25 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:18:25.504 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:25 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:18:25.504 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:25 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:18:25.504 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:25 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:18:25.504 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:25 smithi026 ceph-mon[31172]: Adjusting osd_memory_target on smithi136 to 176.0M 2024-09-11T23:18:25.504 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:25 smithi026 ceph-mon[31172]: Unable to set osd_memory_target on smithi136 to 184574566: error parsing value: Value '184574566' is below minimum 939524096 2024-09-11T23:18:25.553 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:25 smithi076 ceph-mon[35296]: [11/Sep/2024:23:18:24] ENGINE Serving on https://172.21.15.26:7150 2024-09-11T23:18:25.553 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:25 smithi076 ceph-mon[35296]: [11/Sep/2024:23:18:24] ENGINE Bus STARTED 2024-09-11T23:18:25.553 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:25 smithi076 ceph-mon[35296]: [11/Sep/2024:23:18:24] ENGINE Error in HTTPServer.serve 2024-09-11T23:18:25.554 INFO:journalctl@ceph.mon.b.smithi076.stdout: Traceback (most recent call last): 2024-09-11T23:18:25.554 INFO:journalctl@ceph.mon.b.smithi076.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-11T23:18:25.554 INFO:journalctl@ceph.mon.b.smithi076.stdout: self._connections.run(self.expiration_interval) 2024-09-11T23:18:25.554 INFO:journalctl@ceph.mon.b.smithi076.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-11T23:18:25.554 INFO:journalctl@ceph.mon.b.smithi076.stdout: self._run(expiration_interval) 2024-09-11T23:18:25.554 INFO:journalctl@ceph.mon.b.smithi076.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-11T23:18:25.554 INFO:journalctl@ceph.mon.b.smithi076.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-11T23:18:25.554 INFO:journalctl@ceph.mon.b.smithi076.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-11T23:18:25.554 INFO:journalctl@ceph.mon.b.smithi076.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-11T23:18:25.554 INFO:journalctl@ceph.mon.b.smithi076.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-11T23:18:25.554 INFO:journalctl@ceph.mon.b.smithi076.stdout: s = self.context.wrap_socket( 2024-09-11T23:18:25.554 INFO:journalctl@ceph.mon.b.smithi076.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-11T23:18:25.554 INFO:journalctl@ceph.mon.b.smithi076.stdout: return self.sslsocket_class._create( 2024-09-11T23:18:25.554 INFO:journalctl@ceph.mon.b.smithi076.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-11T23:18:25.555 INFO:journalctl@ceph.mon.b.smithi076.stdout: self.do_handshake() 2024-09-11T23:18:25.555 INFO:journalctl@ceph.mon.b.smithi076.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-11T23:18:25.555 INFO:journalctl@ceph.mon.b.smithi076.stdout: self._sslobj.do_handshake() 2024-09-11T23:18:25.555 INFO:journalctl@ceph.mon.b.smithi076.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-11T23:18:25.555 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:25 smithi076 ceph-mon[35296]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:18:25.555 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:25 smithi076 ceph-mon[35296]: mgrmap e23: x(active, since 2s), standbys: y, z 2024-09-11T23:18:25.555 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:25 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:18:25.555 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:25 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:18:25.555 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:25 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:18:25.555 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:25 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:18:25.555 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:25 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:18:25.555 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:25 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:18:25.555 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:25 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:18:25.555 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:25 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:18:25.556 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:25 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:18:25.556 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:25 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:18:25.556 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:25 smithi076 ceph-mon[35296]: Adjusting osd_memory_target on smithi076 to 176.0M 2024-09-11T23:18:25.556 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:25 smithi076 ceph-mon[35296]: Unable to set osd_memory_target on smithi076 to 184591052: error parsing value: Value '184591052' is below minimum 939524096 2024-09-11T23:18:25.556 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:25 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:18:25.556 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:25 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:18:25.556 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:25 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:18:25.556 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:25 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:18:25.556 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:25 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:18:25.556 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:25 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:18:25.556 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:25 smithi076 ceph-mon[35296]: Adjusting osd_memory_target on smithi136 to 176.0M 2024-09-11T23:18:25.556 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:25 smithi076 ceph-mon[35296]: Unable to set osd_memory_target on smithi136 to 184574566: error parsing value: Value '184574566' is below minimum 939524096 2024-09-11T23:18:25.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:25.583+0000 7fe67f3bd640 1 -- 172.21.15.26:0/2860248745 --> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe6781a0680 con 0x7fe6640a2d80 2024-09-11T23:18:25.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:25.589+0000 7fe66e7fc640 1 -- 172.21.15.26:0/2860248745 <== mgr.15171 v2:172.21.15.26:6800/1989555790 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+257778 (secure 0 0 0) 0x7fe6781a0680 con 0x7fe6640a2d80 2024-09-11T23:18:25.593 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-11T23:18:25.595 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-09-11T23:18:25.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:25.592+0000 7fe67f3bd640 1 -- 172.21.15.26:0/2860248745 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fe6640a2d80 msgr2=0x7fe6640a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:25.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:25.592+0000 7fe67f3bd640 1 --2- 172.21.15.26:0/2860248745 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fe6640a2d80 0x7fe6640a5240 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fe674005db0 tx=0x7fe674005d40 comp rx=0 tx=0).stop 2024-09-11T23:18:25.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:25.592+0000 7fe67f3bd640 1 -- 172.21.15.26:0/2860248745 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe6781025c0 msgr2=0x7fe67819f400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:25.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:25.592+0000 7fe67f3bd640 1 --2- 172.21.15.26:0/2860248745 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe6781025c0 0x7fe67819f400 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fe67000eaa0 tx=0x7fe67000ef60 comp rx=0 tx=0).stop 2024-09-11T23:18:25.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:25.592+0000 7fe67f3bd640 1 -- 172.21.15.26:0/2860248745 shutdown_connections 2024-09-11T23:18:25.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:25.592+0000 7fe67f3bd640 1 --2- 172.21.15.26:0/2860248745 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fe6640a2d80 0x7fe6640a5240 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:25.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:25.592+0000 7fe67f3bd640 1 --2- 172.21.15.26:0/2860248745 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe678101c00 0x7fe678198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:25.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:25.592+0000 7fe67f3bd640 1 --2- 172.21.15.26:0/2860248745 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe6781025c0 0x7fe67819f400 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:25.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:25.592+0000 7fe67f3bd640 1 --2- 172.21.15.26:0/2860248745 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe678100a10 0x7fe678197e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:25.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:25.592+0000 7fe67f3bd640 1 -- 172.21.15.26:0/2860248745 >> 172.21.15.26:0/2860248745 conn(0x7fe6780fc170 msgr2=0x7fe6780693f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:18:25.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:25.593+0000 7fe67f3bd640 1 -- 172.21.15.26:0/2860248745 shutdown_connections 2024-09-11T23:18:25.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:25.593+0000 7fe67f3bd640 1 -- 172.21.15.26:0/2860248745 wait complete. 2024-09-11T23:18:25.605 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":4,"stamp":"2024-09-11T23:18:24.197005+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1189948,"num_objects":104,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1325,"num_read_kb":828,"num_write":555,"num_write_kb":4402,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":937,"ondisk_log_size":937,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":380772,"kb_used_data":14560,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124561052,"statfs":{"total":1151940427776,"available":1151550517248,"internally_reserved":0,"allocated":14909440,"data_stored":8639524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":374453561},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.991701"},"pg_stats":[{"pgid":"3.3f","version":"85'8","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.329067+0000","last_change":"2024-09-11T23:16:25.899106+0000","last_active":"2024-09-11T23:18:22.329067+0000","last_peered":"2024-09-11T23:18:22.329067+0000","last_clean":"2024-09-11T23:18:22.329067+0000","last_became_active":"2024-09-11T23:16:25.898654+0000","last_became_peered":"2024-09-11T23:16:25.898654+0000","last_unstale":"2024-09-11T23:18:22.329067+0000","last_undegraded":"2024-09-11T23:18:22.329067+0000","last_fullsized":"2024-09-11T23:18:22.329067+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:21:41.490905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"85'4","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.544200+0000","last_change":"2024-09-11T23:16:25.895629+0000","last_active":"2024-09-11T23:18:22.544200+0000","last_peered":"2024-09-11T23:18:22.544200+0000","last_clean":"2024-09-11T23:18:22.544200+0000","last_became_active":"2024-09-11T23:16:25.895308+0000","last_became_peered":"2024-09-11T23:16:25.895308+0000","last_unstale":"2024-09-11T23:18:22.544200+0000","last_undegraded":"2024-09-11T23:18:22.544200+0000","last_fullsized":"2024-09-11T23:18:22.544200+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:39:42.549970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"85'7","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.544117+0000","last_change":"2024-09-11T23:16:25.907113+0000","last_active":"2024-09-11T23:18:22.544117+0000","last_peered":"2024-09-11T23:18:22.544117+0000","last_clean":"2024-09-11T23:18:22.544117+0000","last_became_active":"2024-09-11T23:16:25.906803+0000","last_became_peered":"2024-09-11T23:16:25.906803+0000","last_unstale":"2024-09-11T23:18:22.544117+0000","last_undegraded":"2024-09-11T23:18:22.544117+0000","last_fullsized":"2024-09-11T23:18:22.544117+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:36:49.303932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"85'8","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.292802+0000","last_change":"2024-09-11T23:16:35.946167+0000","last_active":"2024-09-11T23:18:22.292802+0000","last_peered":"2024-09-11T23:18:22.292802+0000","last_clean":"2024-09-11T23:18:22.292802+0000","last_became_active":"2024-09-11T23:16:35.945786+0000","last_became_peered":"2024-09-11T23:16:35.945786+0000","last_unstale":"2024-09-11T23:18:22.292802+0000","last_undegraded":"2024-09-11T23:18:22.292802+0000","last_fullsized":"2024-09-11T23:18:22.292802+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:13:03.064910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"85'4","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.295860+0000","last_change":"2024-09-11T23:16:25.897909+0000","last_active":"2024-09-11T23:18:22.295860+0000","last_peered":"2024-09-11T23:18:22.295860+0000","last_clean":"2024-09-11T23:18:22.295860+0000","last_became_active":"2024-09-11T23:16:25.897569+0000","last_became_peered":"2024-09-11T23:16:25.897569+0000","last_unstale":"2024-09-11T23:18:22.295860+0000","last_undegraded":"2024-09-11T23:18:22.295860+0000","last_fullsized":"2024-09-11T23:18:22.295860+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:25:44.727254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"85'6","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.302845+0000","last_change":"2024-09-11T23:16:25.897594+0000","last_active":"2024-09-11T23:18:22.302845+0000","last_peered":"2024-09-11T23:18:22.302845+0000","last_clean":"2024-09-11T23:18:22.302845+0000","last_became_active":"2024-09-11T23:16:25.897223+0000","last_became_peered":"2024-09-11T23:16:25.897223+0000","last_unstale":"2024-09-11T23:18:22.302845+0000","last_undegraded":"2024-09-11T23:18:22.302845+0000","last_fullsized":"2024-09-11T23:18:22.302845+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:10:50.763455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"85'5","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328293+0000","last_change":"2024-09-11T23:16:35.943511+0000","last_active":"2024-09-11T23:18:22.328293+0000","last_peered":"2024-09-11T23:18:22.328293+0000","last_clean":"2024-09-11T23:18:22.328293+0000","last_became_active":"2024-09-11T23:16:35.943075+0000","last_became_peered":"2024-09-11T23:16:35.943075+0000","last_unstale":"2024-09-11T23:18:22.328293+0000","last_undegraded":"2024-09-11T23:18:22.328293+0000","last_fullsized":"2024-09-11T23:18:22.328293+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:53:43.892421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"85'9","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.182677+0000","last_change":"2024-09-11T23:16:25.903380+0000","last_active":"2024-09-11T23:18:22.182677+0000","last_peered":"2024-09-11T23:18:22.182677+0000","last_clean":"2024-09-11T23:18:22.182677+0000","last_became_active":"2024-09-11T23:16:25.903093+0000","last_became_peered":"2024-09-11T23:16:25.903093+0000","last_unstale":"2024-09-11T23:18:22.182677+0000","last_undegraded":"2024-09-11T23:18:22.182677+0000","last_fullsized":"2024-09-11T23:18:22.182677+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:58:50.944623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"85'11","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.329153+0000","last_change":"2024-09-11T23:16:25.908166+0000","last_active":"2024-09-11T23:18:22.329153+0000","last_peered":"2024-09-11T23:18:22.329153+0000","last_clean":"2024-09-11T23:18:22.329153+0000","last_became_active":"2024-09-11T23:16:25.907749+0000","last_became_peered":"2024-09-11T23:16:25.907749+0000","last_unstale":"2024-09-11T23:18:22.329153+0000","last_undegraded":"2024-09-11T23:18:22.329153+0000","last_fullsized":"2024-09-11T23:18:22.329153+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:11:16.705931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"85'16","reported_seq":41,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.303286+0000","last_change":"2024-09-11T23:16:35.944259+0000","last_active":"2024-09-11T23:18:22.303286+0000","last_peered":"2024-09-11T23:18:22.303286+0000","last_clean":"2024-09-11T23:18:22.303286+0000","last_became_active":"2024-09-11T23:16:35.943767+0000","last_became_peered":"2024-09-11T23:16:35.943767+0000","last_unstale":"2024-09-11T23:18:22.303286+0000","last_undegraded":"2024-09-11T23:18:22.303286+0000","last_fullsized":"2024-09-11T23:18:22.303286+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:20:13.127854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"85'6","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328788+0000","last_change":"2024-09-11T23:16:25.903256+0000","last_active":"2024-09-11T23:18:22.328788+0000","last_peered":"2024-09-11T23:18:22.328788+0000","last_clean":"2024-09-11T23:18:22.328788+0000","last_became_active":"2024-09-11T23:16:25.902954+0000","last_became_peered":"2024-09-11T23:16:25.902954+0000","last_unstale":"2024-09-11T23:18:22.328788+0000","last_undegraded":"2024-09-11T23:18:22.328788+0000","last_fullsized":"2024-09-11T23:18:22.328788+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:43:48.651400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"85'7","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.187001+0000","last_change":"2024-09-11T23:16:25.897656+0000","last_active":"2024-09-11T23:18:22.187001+0000","last_peered":"2024-09-11T23:18:22.187001+0000","last_clean":"2024-09-11T23:18:22.187001+0000","last_became_active":"2024-09-11T23:16:25.897103+0000","last_became_peered":"2024-09-11T23:16:25.897103+0000","last_unstale":"2024-09-11T23:18:22.187001+0000","last_undegraded":"2024-09-11T23:18:22.187001+0000","last_fullsized":"2024-09-11T23:18:22.187001+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:14:34.341383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"85'10","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.329625+0000","last_change":"2024-09-11T23:16:25.902800+0000","last_active":"2024-09-11T23:18:22.329625+0000","last_peered":"2024-09-11T23:18:22.329625+0000","last_clean":"2024-09-11T23:18:22.329625+0000","last_became_active":"2024-09-11T23:16:25.902414+0000","last_became_peered":"2024-09-11T23:16:25.902414+0000","last_unstale":"2024-09-11T23:18:22.329625+0000","last_undegraded":"2024-09-11T23:18:22.329625+0000","last_fullsized":"2024-09-11T23:18:22.329625+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:16:32.427749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"85'7","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.543876+0000","last_change":"2024-09-11T23:16:25.894410+0000","last_active":"2024-09-11T23:18:22.543876+0000","last_peered":"2024-09-11T23:18:22.543876+0000","last_clean":"2024-09-11T23:18:22.543876+0000","last_became_active":"2024-09-11T23:16:25.894067+0000","last_became_peered":"2024-09-11T23:16:25.894067+0000","last_unstale":"2024-09-11T23:18:22.543876+0000","last_undegraded":"2024-09-11T23:18:22.543876+0000","last_fullsized":"2024-09-11T23:18:22.543876+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:03:11.913063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"85'7","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.544140+0000","last_change":"2024-09-11T23:16:25.903440+0000","last_active":"2024-09-11T23:18:22.544140+0000","last_peered":"2024-09-11T23:18:22.544140+0000","last_clean":"2024-09-11T23:18:22.544140+0000","last_became_active":"2024-09-11T23:16:25.903122+0000","last_became_peered":"2024-09-11T23:16:25.903122+0000","last_unstale":"2024-09-11T23:18:22.544140+0000","last_undegraded":"2024-09-11T23:18:22.544140+0000","last_fullsized":"2024-09-11T23:18:22.544140+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:03:41.065314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"85'9","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328034+0000","last_change":"2024-09-11T23:16:25.905336+0000","last_active":"2024-09-11T23:18:22.328034+0000","last_peered":"2024-09-11T23:18:22.328034+0000","last_clean":"2024-09-11T23:18:22.328034+0000","last_became_active":"2024-09-11T23:16:25.904699+0000","last_became_peered":"2024-09-11T23:16:25.904699+0000","last_unstale":"2024-09-11T23:18:22.328034+0000","last_undegraded":"2024-09-11T23:18:22.328034+0000","last_fullsized":"2024-09-11T23:18:22.328034+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:16:13.448585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"85'8","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.329144+0000","last_change":"2024-09-11T23:16:25.897977+0000","last_active":"2024-09-11T23:18:22.329144+0000","last_peered":"2024-09-11T23:18:22.329144+0000","last_clean":"2024-09-11T23:18:22.329144+0000","last_became_active":"2024-09-11T23:16:25.897672+0000","last_became_peered":"2024-09-11T23:16:25.897672+0000","last_unstale":"2024-09-11T23:18:22.329144+0000","last_undegraded":"2024-09-11T23:18:22.329144+0000","last_fullsized":"2024-09-11T23:18:22.329144+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:42:56.575066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"85'10","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.293509+0000","last_change":"2024-09-11T23:16:25.897063+0000","last_active":"2024-09-11T23:18:22.293509+0000","last_peered":"2024-09-11T23:18:22.293509+0000","last_clean":"2024-09-11T23:18:22.293509+0000","last_became_active":"2024-09-11T23:16:25.896770+0000","last_became_peered":"2024-09-11T23:16:25.896770+0000","last_unstale":"2024-09-11T23:18:22.293509+0000","last_undegraded":"2024-09-11T23:18:22.293509+0000","last_fullsized":"2024-09-11T23:18:22.293509+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:55:44.735171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"85'7","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328943+0000","last_change":"2024-09-11T23:16:25.903423+0000","last_active":"2024-09-11T23:18:22.328943+0000","last_peered":"2024-09-11T23:18:22.328943+0000","last_clean":"2024-09-11T23:18:22.328943+0000","last_became_active":"2024-09-11T23:16:25.902906+0000","last_became_peered":"2024-09-11T23:16:25.902906+0000","last_unstale":"2024-09-11T23:18:22.328943+0000","last_undegraded":"2024-09-11T23:18:22.328943+0000","last_fullsized":"2024-09-11T23:18:22.328943+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:40:49.326045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"85'8","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328557+0000","last_change":"2024-09-11T23:16:25.896564+0000","last_active":"2024-09-11T23:18:22.328557+0000","last_peered":"2024-09-11T23:18:22.328557+0000","last_clean":"2024-09-11T23:18:22.328557+0000","last_became_active":"2024-09-11T23:16:25.896301+0000","last_became_peered":"2024-09-11T23:16:25.896301+0000","last_unstale":"2024-09-11T23:18:22.328557+0000","last_undegraded":"2024-09-11T23:18:22.328557+0000","last_fullsized":"2024-09-11T23:18:22.328557+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:30:30.074853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"85'11","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.329804+0000","last_change":"2024-09-11T23:16:25.903491+0000","last_active":"2024-09-11T23:18:22.329804+0000","last_peered":"2024-09-11T23:18:22.329804+0000","last_clean":"2024-09-11T23:18:22.329804+0000","last_became_active":"2024-09-11T23:16:25.901340+0000","last_became_peered":"2024-09-11T23:16:25.901340+0000","last_unstale":"2024-09-11T23:18:22.329804+0000","last_undegraded":"2024-09-11T23:18:22.329804+0000","last_fullsized":"2024-09-11T23:18:22.329804+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:41:16.058034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"85'10","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328233+0000","last_change":"2024-09-11T23:16:25.901084+0000","last_active":"2024-09-11T23:18:22.328233+0000","last_peered":"2024-09-11T23:18:22.328233+0000","last_clean":"2024-09-11T23:18:22.328233+0000","last_became_active":"2024-09-11T23:16:25.900726+0000","last_became_peered":"2024-09-11T23:16:25.900726+0000","last_unstale":"2024-09-11T23:18:22.328233+0000","last_undegraded":"2024-09-11T23:18:22.328233+0000","last_fullsized":"2024-09-11T23:18:22.328233+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:57:40.955645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"85'8","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.182688+0000","last_change":"2024-09-11T23:16:25.900461+0000","last_active":"2024-09-11T23:18:22.182688+0000","last_peered":"2024-09-11T23:18:22.182688+0000","last_clean":"2024-09-11T23:18:22.182688+0000","last_became_active":"2024-09-11T23:16:25.900174+0000","last_became_peered":"2024-09-11T23:16:25.900174+0000","last_unstale":"2024-09-11T23:18:22.182688+0000","last_undegraded":"2024-09-11T23:18:22.182688+0000","last_fullsized":"2024-09-11T23:18:22.182688+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:51:50.953198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"85'6","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.184452+0000","last_change":"2024-09-11T23:16:25.907569+0000","last_active":"2024-09-11T23:18:22.184452+0000","last_peered":"2024-09-11T23:18:22.184452+0000","last_clean":"2024-09-11T23:18:22.184452+0000","last_became_active":"2024-09-11T23:16:25.907091+0000","last_became_peered":"2024-09-11T23:16:25.907091+0000","last_unstale":"2024-09-11T23:18:22.184452+0000","last_undegraded":"2024-09-11T23:18:22.184452+0000","last_fullsized":"2024-09-11T23:18:22.184452+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:53:45.961167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"85'2","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.183204+0000","last_change":"2024-09-11T23:16:25.903246+0000","last_active":"2024-09-11T23:18:22.183204+0000","last_peered":"2024-09-11T23:18:22.183204+0000","last_clean":"2024-09-11T23:18:22.183204+0000","last_became_active":"2024-09-11T23:16:25.902907+0000","last_became_peered":"2024-09-11T23:16:25.902907+0000","last_unstale":"2024-09-11T23:18:22.183204+0000","last_undegraded":"2024-09-11T23:18:22.183204+0000","last_fullsized":"2024-09-11T23:18:22.183204+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:20:38.868384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"85'15","reported_seq":34,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328979+0000","last_change":"2024-09-11T23:16:25.906298+0000","last_active":"2024-09-11T23:18:22.328979+0000","last_peered":"2024-09-11T23:18:22.328979+0000","last_clean":"2024-09-11T23:18:22.328979+0000","last_became_active":"2024-09-11T23:16:25.905350+0000","last_became_peered":"2024-09-11T23:16:25.905350+0000","last_unstale":"2024-09-11T23:18:22.328979+0000","last_undegraded":"2024-09-11T23:18:22.328979+0000","last_fullsized":"2024-09-11T23:18:22.328979+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:04:22.122514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"85'6","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328869+0000","last_change":"2024-09-11T23:16:25.902114+0000","last_active":"2024-09-11T23:18:22.328869+0000","last_peered":"2024-09-11T23:18:22.328869+0000","last_clean":"2024-09-11T23:18:22.328869+0000","last_became_active":"2024-09-11T23:16:25.901742+0000","last_became_peered":"2024-09-11T23:16:25.901742+0000","last_unstale":"2024-09-11T23:18:22.328869+0000","last_undegraded":"2024-09-11T23:18:22.328869+0000","last_fullsized":"2024-09-11T23:18:22.328869+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:02:10.640801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"85'4","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.329796+0000","last_change":"2024-09-11T23:16:25.898117+0000","last_active":"2024-09-11T23:18:22.329796+0000","last_peered":"2024-09-11T23:18:22.329796+0000","last_clean":"2024-09-11T23:18:22.329796+0000","last_became_active":"2024-09-11T23:16:25.897818+0000","last_became_peered":"2024-09-11T23:16:25.897818+0000","last_unstale":"2024-09-11T23:18:22.329796+0000","last_undegraded":"2024-09-11T23:18:22.329796+0000","last_fullsized":"2024-09-11T23:18:22.329796+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:27:37.175471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"85'7","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.544057+0000","last_change":"2024-09-11T23:16:25.905600+0000","last_active":"2024-09-11T23:18:22.544057+0000","last_peered":"2024-09-11T23:18:22.544057+0000","last_clean":"2024-09-11T23:18:22.544057+0000","last_became_active":"2024-09-11T23:16:25.903924+0000","last_became_peered":"2024-09-11T23:16:25.903924+0000","last_unstale":"2024-09-11T23:18:22.544057+0000","last_undegraded":"2024-09-11T23:18:22.544057+0000","last_fullsized":"2024-09-11T23:18:22.544057+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:20:29.929079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"85'8","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328449+0000","last_change":"2024-09-11T23:16:25.904763+0000","last_active":"2024-09-11T23:18:22.328449+0000","last_peered":"2024-09-11T23:18:22.328449+0000","last_clean":"2024-09-11T23:18:22.328449+0000","last_became_active":"2024-09-11T23:16:25.904406+0000","last_became_peered":"2024-09-11T23:16:25.904406+0000","last_unstale":"2024-09-11T23:18:22.328449+0000","last_undegraded":"2024-09-11T23:18:22.328449+0000","last_fullsized":"2024-09-11T23:18:22.328449+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:27:38.918017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"85'12","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328598+0000","last_change":"2024-09-11T23:16:25.906573+0000","last_active":"2024-09-11T23:18:22.328598+0000","last_peered":"2024-09-11T23:18:22.328598+0000","last_clean":"2024-09-11T23:18:22.328598+0000","last_became_active":"2024-09-11T23:16:25.904771+0000","last_became_peered":"2024-09-11T23:16:25.904771+0000","last_unstale":"2024-09-11T23:18:22.328598+0000","last_undegraded":"2024-09-11T23:18:22.328598+0000","last_fullsized":"2024-09-11T23:18:22.328598+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:04:17.090648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"85'9","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328348+0000","last_change":"2024-09-11T23:16:25.902123+0000","last_active":"2024-09-11T23:18:22.328348+0000","last_peered":"2024-09-11T23:18:22.328348+0000","last_clean":"2024-09-11T23:18:22.328348+0000","last_became_active":"2024-09-11T23:16:25.901756+0000","last_became_peered":"2024-09-11T23:16:25.901756+0000","last_unstale":"2024-09-11T23:18:22.328348+0000","last_undegraded":"2024-09-11T23:18:22.328348+0000","last_fullsized":"2024-09-11T23:18:22.328348+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:17:37.074976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"85'7","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.295802+0000","last_change":"2024-09-11T23:16:25.904217+0000","last_active":"2024-09-11T23:18:22.295802+0000","last_peered":"2024-09-11T23:18:22.295802+0000","last_clean":"2024-09-11T23:18:22.295802+0000","last_became_active":"2024-09-11T23:16:25.903522+0000","last_became_peered":"2024-09-11T23:16:25.903522+0000","last_unstale":"2024-09-11T23:18:22.295802+0000","last_undegraded":"2024-09-11T23:18:22.295802+0000","last_fullsized":"2024-09-11T23:18:22.295802+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:29:46.634475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"86'7","reported_seq":140,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:23.124462+0000","last_change":"2024-09-11T23:16:23.793696+0000","last_active":"2024-09-11T23:18:23.124462+0000","last_peered":"2024-09-11T23:18:23.124462+0000","last_clean":"2024-09-11T23:18:23.124462+0000","last_became_active":"2024-09-11T23:16:23.793062+0000","last_became_peered":"2024-09-11T23:16:23.793062+0000","last_unstale":"2024-09-11T23:18:23.124462+0000","last_undegraded":"2024-09-11T23:18:23.124462+0000","last_fullsized":"2024-09-11T23:18:23.124462+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:00:25.655817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1323,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":109,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"85'8","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.184453+0000","last_change":"2024-09-11T23:16:25.906673+0000","last_active":"2024-09-11T23:18:22.184453+0000","last_peered":"2024-09-11T23:18:22.184453+0000","last_clean":"2024-09-11T23:18:22.184453+0000","last_became_active":"2024-09-11T23:16:25.906239+0000","last_became_peered":"2024-09-11T23:16:25.906239+0000","last_unstale":"2024-09-11T23:18:22.184453+0000","last_undegraded":"2024-09-11T23:18:22.184453+0000","last_fullsized":"2024-09-11T23:18:22.184453+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:15:54.907269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"85'7","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.302905+0000","last_change":"2024-09-11T23:16:23.793302+0000","last_active":"2024-09-11T23:18:22.302905+0000","last_peered":"2024-09-11T23:18:22.302905+0000","last_clean":"2024-09-11T23:18:22.302905+0000","last_became_active":"2024-09-11T23:16:23.792874+0000","last_became_peered":"2024-09-11T23:16:23.792874+0000","last_unstale":"2024-09-11T23:18:22.302905+0000","last_undegraded":"2024-09-11T23:18:22.302905+0000","last_fullsized":"2024-09-11T23:18:22.302905+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:52:59.558223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"85'3","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328845+0000","last_change":"2024-09-11T23:16:23.792999+0000","last_active":"2024-09-11T23:18:22.328845+0000","last_peered":"2024-09-11T23:18:22.328845+0000","last_clean":"2024-09-11T23:18:22.328845+0000","last_became_active":"2024-09-11T23:16:23.792630+0000","last_became_peered":"2024-09-11T23:16:23.792630+0000","last_unstale":"2024-09-11T23:18:22.328845+0000","last_undegraded":"2024-09-11T23:18:22.328845+0000","last_fullsized":"2024-09-11T23:18:22.328845+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:09:28.001060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"85'8","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.292734+0000","last_change":"2024-09-11T23:16:35.943722+0000","last_active":"2024-09-11T23:18:22.292734+0000","last_peered":"2024-09-11T23:18:22.292734+0000","last_clean":"2024-09-11T23:18:22.292734+0000","last_became_active":"2024-09-11T23:16:35.943355+0000","last_became_peered":"2024-09-11T23:16:35.943355+0000","last_unstale":"2024-09-11T23:18:22.292734+0000","last_undegraded":"2024-09-11T23:18:22.292734+0000","last_fullsized":"2024-09-11T23:18:22.292734+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:26:22.769824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,6],"acting":[9,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"85'3","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.184646+0000","last_change":"2024-09-11T23:16:23.797945+0000","last_active":"2024-09-11T23:18:22.184646+0000","last_peered":"2024-09-11T23:18:22.184646+0000","last_clean":"2024-09-11T23:18:22.184646+0000","last_became_active":"2024-09-11T23:16:23.797572+0000","last_became_peered":"2024-09-11T23:16:23.797572+0000","last_unstale":"2024-09-11T23:18:22.184646+0000","last_undegraded":"2024-09-11T23:18:22.184646+0000","last_fullsized":"2024-09-11T23:18:22.184646+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:04:59.017668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"85'11","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.292877+0000","last_change":"2024-09-11T23:16:25.891587+0000","last_active":"2024-09-11T23:18:22.292877+0000","last_peered":"2024-09-11T23:18:22.292877+0000","last_clean":"2024-09-11T23:18:22.292877+0000","last_became_active":"2024-09-11T23:16:25.891190+0000","last_became_peered":"2024-09-11T23:16:25.891190+0000","last_unstale":"2024-09-11T23:18:22.292877+0000","last_undegraded":"2024-09-11T23:18:22.292877+0000","last_fullsized":"2024-09-11T23:18:22.292877+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:57:05.177843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"85'6","reported_seq":240,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:23.123749+0000","last_change":"2024-09-11T23:16:23.790129+0000","last_active":"2024-09-11T23:18:23.123749+0000","last_peered":"2024-09-11T23:18:23.123749+0000","last_clean":"2024-09-11T23:18:23.123749+0000","last_became_active":"2024-09-11T23:16:23.789720+0000","last_became_peered":"2024-09-11T23:16:23.789720+0000","last_unstale":"2024-09-11T23:18:23.123749+0000","last_undegraded":"2024-09-11T23:18:23.123749+0000","last_fullsized":"2024-09-11T23:18:23.123749+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:33:58.388513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":210,"num_read_kb":210,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"85'10","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.329142+0000","last_change":"2024-09-11T23:16:25.901014+0000","last_active":"2024-09-11T23:18:22.329142+0000","last_peered":"2024-09-11T23:18:22.329142+0000","last_clean":"2024-09-11T23:18:22.329142+0000","last_became_active":"2024-09-11T23:16:25.900280+0000","last_became_peered":"2024-09-11T23:16:25.900280+0000","last_unstale":"2024-09-11T23:18:22.329142+0000","last_undegraded":"2024-09-11T23:18:22.329142+0000","last_fullsized":"2024-09-11T23:18:22.329142+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:14:03.662204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"85'4","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328532+0000","last_change":"2024-09-11T23:16:23.797944+0000","last_active":"2024-09-11T23:18:22.328532+0000","last_peered":"2024-09-11T23:18:22.328532+0000","last_clean":"2024-09-11T23:18:22.328532+0000","last_became_active":"2024-09-11T23:16:23.797520+0000","last_became_peered":"2024-09-11T23:16:23.797520+0000","last_unstale":"2024-09-11T23:18:22.328532+0000","last_undegraded":"2024-09-11T23:18:22.328532+0000","last_fullsized":"2024-09-11T23:18:22.328532+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:29:06.957518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"85'9","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.544713+0000","last_change":"2024-09-11T23:16:25.905379+0000","last_active":"2024-09-11T23:18:22.544713+0000","last_peered":"2024-09-11T23:18:22.544713+0000","last_clean":"2024-09-11T23:18:22.544713+0000","last_became_active":"2024-09-11T23:16:25.905085+0000","last_became_peered":"2024-09-11T23:16:25.905085+0000","last_unstale":"2024-09-11T23:18:22.544713+0000","last_undegraded":"2024-09-11T23:18:22.544713+0000","last_fullsized":"2024-09-11T23:18:22.544713+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:22:35.129736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"85'5","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.184874+0000","last_change":"2024-09-11T23:16:24.075556+0000","last_active":"2024-09-11T23:18:22.184874+0000","last_peered":"2024-09-11T23:18:22.184874+0000","last_clean":"2024-09-11T23:18:22.184874+0000","last_became_active":"2024-09-11T23:16:24.074783+0000","last_became_peered":"2024-09-11T23:16:24.074783+0000","last_unstale":"2024-09-11T23:18:22.184874+0000","last_undegraded":"2024-09-11T23:18:22.184874+0000","last_fullsized":"2024-09-11T23:18:22.184874+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:55:09.962342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"85'11","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328166+0000","last_change":"2024-09-11T23:16:25.907720+0000","last_active":"2024-09-11T23:18:22.328166+0000","last_peered":"2024-09-11T23:18:22.328166+0000","last_clean":"2024-09-11T23:18:22.328166+0000","last_became_active":"2024-09-11T23:16:25.907335+0000","last_became_peered":"2024-09-11T23:16:25.907335+0000","last_unstale":"2024-09-11T23:18:22.328166+0000","last_undegraded":"2024-09-11T23:18:22.328166+0000","last_fullsized":"2024-09-11T23:18:22.328166+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:46:14.519384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"86'38","reported_seq":284,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:23.145509+0000","last_change":"2024-09-11T23:16:23.794321+0000","last_active":"2024-09-11T23:18:23.145509+0000","last_peered":"2024-09-11T23:18:23.145509+0000","last_clean":"2024-09-11T23:18:23.145509+0000","last_became_active":"2024-09-11T23:16:23.793955+0000","last_became_peered":"2024-09-11T23:16:23.793955+0000","last_unstale":"2024-09-11T23:18:23.145509+0000","last_undegraded":"2024-09-11T23:18:23.145509+0000","last_fullsized":"2024-09-11T23:18:23.145509+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:47:46.442656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":66480,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":222,"num_read_kb":175,"num_write":40,"num_write_kb":82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"85'6","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.296053+0000","last_change":"2024-09-11T23:16:25.896598+0000","last_active":"2024-09-11T23:18:22.296053+0000","last_peered":"2024-09-11T23:18:22.296053+0000","last_clean":"2024-09-11T23:18:22.296053+0000","last_became_active":"2024-09-11T23:16:25.896293+0000","last_became_peered":"2024-09-11T23:16:25.896293+0000","last_unstale":"2024-09-11T23:18:22.296053+0000","last_undegraded":"2024-09-11T23:18:22.296053+0000","last_fullsized":"2024-09-11T23:18:22.296053+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:04:11.489644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"85'9","reported_seq":35,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328677+0000","last_change":"2024-09-11T23:16:23.791486+0000","last_active":"2024-09-11T23:18:22.328677+0000","last_peered":"2024-09-11T23:18:22.328677+0000","last_clean":"2024-09-11T23:18:22.328677+0000","last_became_active":"2024-09-11T23:16:23.790786+0000","last_became_peered":"2024-09-11T23:16:23.790786+0000","last_unstale":"2024-09-11T23:18:22.328677+0000","last_undegraded":"2024-09-11T23:18:22.328677+0000","last_fullsized":"2024-09-11T23:18:22.328677+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:03:52.742560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"85'9","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.329171+0000","last_change":"2024-09-11T23:16:25.897382+0000","last_active":"2024-09-11T23:18:22.329171+0000","last_peered":"2024-09-11T23:18:22.329171+0000","last_clean":"2024-09-11T23:18:22.329171+0000","last_became_active":"2024-09-11T23:16:25.897073+0000","last_became_peered":"2024-09-11T23:16:25.897073+0000","last_unstale":"2024-09-11T23:18:22.329171+0000","last_undegraded":"2024-09-11T23:18:22.329171+0000","last_fullsized":"2024-09-11T23:18:22.329171+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:15:59.780100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"85'5","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.184727+0000","last_change":"2024-09-11T23:16:23.793886+0000","last_active":"2024-09-11T23:18:22.184727+0000","last_peered":"2024-09-11T23:18:22.184727+0000","last_clean":"2024-09-11T23:18:22.184727+0000","last_became_active":"2024-09-11T23:16:23.793066+0000","last_became_peered":"2024-09-11T23:16:23.793066+0000","last_unstale":"2024-09-11T23:18:22.184727+0000","last_undegraded":"2024-09-11T23:18:22.184727+0000","last_fullsized":"2024-09-11T23:18:22.184727+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:07:03.882546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"85'15","reported_seq":34,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328552+0000","last_change":"2024-09-11T23:16:25.904575+0000","last_active":"2024-09-11T23:18:22.328552+0000","last_peered":"2024-09-11T23:18:22.328552+0000","last_clean":"2024-09-11T23:18:22.328552+0000","last_became_active":"2024-09-11T23:16:25.904224+0000","last_became_peered":"2024-09-11T23:16:25.904224+0000","last_unstale":"2024-09-11T23:18:22.328552+0000","last_undegraded":"2024-09-11T23:18:22.328552+0000","last_fullsized":"2024-09-11T23:18:22.328552+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:42:13.036931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"85'5","reported_seq":136,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.187045+0000","last_change":"2024-09-11T23:16:23.796073+0000","last_active":"2024-09-11T23:18:22.187045+0000","last_peered":"2024-09-11T23:18:22.187045+0000","last_clean":"2024-09-11T23:18:22.187045+0000","last_became_active":"2024-09-11T23:16:23.794850+0000","last_became_peered":"2024-09-11T23:16:23.794850+0000","last_unstale":"2024-09-11T23:18:22.187045+0000","last_undegraded":"2024-09-11T23:18:22.187045+0000","last_fullsized":"2024-09-11T23:18:22.187045+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:57:22.361662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":108,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"85'6","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328221+0000","last_change":"2024-09-11T23:16:25.908700+0000","last_active":"2024-09-11T23:18:22.328221+0000","last_peered":"2024-09-11T23:18:22.328221+0000","last_clean":"2024-09-11T23:18:22.328221+0000","last_became_active":"2024-09-11T23:16:25.908373+0000","last_became_peered":"2024-09-11T23:16:25.908373+0000","last_unstale":"2024-09-11T23:18:22.328221+0000","last_undegraded":"2024-09-11T23:18:22.328221+0000","last_fullsized":"2024-09-11T23:18:22.328221+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:20:18.536079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"85'17","reported_seq":42,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.295792+0000","last_change":"2024-09-11T23:16:24.075927+0000","last_active":"2024-09-11T23:18:22.295792+0000","last_peered":"2024-09-11T23:18:22.295792+0000","last_clean":"2024-09-11T23:18:22.295792+0000","last_became_active":"2024-09-11T23:16:24.075527+0000","last_became_peered":"2024-09-11T23:16:24.075527+0000","last_unstale":"2024-09-11T23:18:22.295792+0000","last_undegraded":"2024-09-11T23:18:22.295792+0000","last_fullsized":"2024-09-11T23:18:22.295792+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:27:54.622059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"85'6","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.184380+0000","last_change":"2024-09-11T23:16:35.947475+0000","last_active":"2024-09-11T23:18:22.184380+0000","last_peered":"2024-09-11T23:18:22.184380+0000","last_clean":"2024-09-11T23:18:22.184380+0000","last_became_active":"2024-09-11T23:16:35.947028+0000","last_became_peered":"2024-09-11T23:16:35.947028+0000","last_unstale":"2024-09-11T23:18:22.184380+0000","last_undegraded":"2024-09-11T23:18:22.184380+0000","last_fullsized":"2024-09-11T23:18:22.184380+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:18:13.479554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'2","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.296198+0000","last_change":"2024-09-11T23:16:23.793298+0000","last_active":"2024-09-11T23:18:22.296198+0000","last_peered":"2024-09-11T23:18:22.296198+0000","last_clean":"2024-09-11T23:18:22.296198+0000","last_became_active":"2024-09-11T23:16:23.792947+0000","last_became_peered":"2024-09-11T23:16:23.792947+0000","last_unstale":"2024-09-11T23:18:22.296198+0000","last_undegraded":"2024-09-11T23:18:22.296198+0000","last_fullsized":"2024-09-11T23:18:22.296198+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:32:21.294560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"85'5","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328871+0000","last_change":"2024-09-11T23:16:25.902348+0000","last_active":"2024-09-11T23:18:22.328871+0000","last_peered":"2024-09-11T23:18:22.328871+0000","last_clean":"2024-09-11T23:18:22.328871+0000","last_became_active":"2024-09-11T23:16:25.901686+0000","last_became_peered":"2024-09-11T23:16:25.901686+0000","last_unstale":"2024-09-11T23:18:22.328871+0000","last_undegraded":"2024-09-11T23:18:22.328871+0000","last_fullsized":"2024-09-11T23:18:22.328871+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:00:15.543003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"85'5","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.330216+0000","last_change":"2024-09-11T23:16:23.790984+0000","last_active":"2024-09-11T23:18:22.330216+0000","last_peered":"2024-09-11T23:18:22.330216+0000","last_clean":"2024-09-11T23:18:22.330216+0000","last_became_active":"2024-09-11T23:16:23.790569+0000","last_became_peered":"2024-09-11T23:16:23.790569+0000","last_unstale":"2024-09-11T23:18:22.330216+0000","last_undegraded":"2024-09-11T23:18:22.330216+0000","last_fullsized":"2024-09-11T23:18:22.330216+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:57:39.675793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"85'9","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328796+0000","last_change":"2024-09-11T23:16:25.899249+0000","last_active":"2024-09-11T23:18:22.328796+0000","last_peered":"2024-09-11T23:18:22.328796+0000","last_clean":"2024-09-11T23:18:22.328796+0000","last_became_active":"2024-09-11T23:16:25.898914+0000","last_became_peered":"2024-09-11T23:16:25.898914+0000","last_unstale":"2024-09-11T23:18:22.328796+0000","last_undegraded":"2024-09-11T23:18:22.328796+0000","last_fullsized":"2024-09-11T23:18:22.328796+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:17:07.545770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"86'41","reported_seq":73,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.429528+0000","last_change":"2024-09-11T23:16:23.787387+0000","last_active":"2024-09-11T23:18:22.429528+0000","last_peered":"2024-09-11T23:18:22.429528+0000","last_clean":"2024-09-11T23:18:22.429528+0000","last_became_active":"2024-09-11T23:16:23.787014+0000","last_became_peered":"2024-09-11T23:16:23.787014+0000","last_unstale":"2024-09-11T23:18:22.429528+0000","last_undegraded":"2024-09-11T23:18:22.429528+0000","last_fullsized":"2024-09-11T23:18:22.429528+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:45:52.004253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"85'7","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328671+0000","last_change":"2024-09-11T23:16:25.908304+0000","last_active":"2024-09-11T23:18:22.328671+0000","last_peered":"2024-09-11T23:18:22.328671+0000","last_clean":"2024-09-11T23:18:22.328671+0000","last_became_active":"2024-09-11T23:16:25.907900+0000","last_became_peered":"2024-09-11T23:16:25.907900+0000","last_unstale":"2024-09-11T23:18:22.328671+0000","last_undegraded":"2024-09-11T23:18:22.328671+0000","last_fullsized":"2024-09-11T23:18:22.328671+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:43:26.247682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"85'7","reported_seq":140,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328822+0000","last_change":"2024-09-11T23:16:24.074610+0000","last_active":"2024-09-11T23:18:22.328822+0000","last_peered":"2024-09-11T23:18:22.328822+0000","last_clean":"2024-09-11T23:18:22.328822+0000","last_became_active":"2024-09-11T23:16:24.074210+0000","last_became_peered":"2024-09-11T23:16:24.074210+0000","last_unstale":"2024-09-11T23:18:22.328822+0000","last_undegraded":"2024-09-11T23:18:22.328822+0000","last_fullsized":"2024-09-11T23:18:22.328822+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:09:37.167050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1704,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":109,"num_read_kb":4,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"85'7","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328842+0000","last_change":"2024-09-11T23:16:25.903120+0000","last_active":"2024-09-11T23:18:22.328842+0000","last_peered":"2024-09-11T23:18:22.328842+0000","last_clean":"2024-09-11T23:18:22.328842+0000","last_became_active":"2024-09-11T23:16:25.902770+0000","last_became_peered":"2024-09-11T23:16:25.902770+0000","last_unstale":"2024-09-11T23:18:22.328842+0000","last_undegraded":"2024-09-11T23:18:22.328842+0000","last_fullsized":"2024-09-11T23:18:22.328842+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:03:30.041088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":349,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328487+0000","last_change":"2024-09-11T23:15:13.115055+0000","last_active":"2024-09-11T23:18:22.328487+0000","last_peered":"2024-09-11T23:18:22.328487+0000","last_clean":"2024-09-11T23:18:22.328487+0000","last_became_active":"2024-09-11T23:15:13.114480+0000","last_became_peered":"2024-09-11T23:15:13.114480+0000","last_unstale":"2024-09-11T23:18:22.328487+0000","last_undegraded":"2024-09-11T23:18:22.328487+0000","last_fullsized":"2024-09-11T23:18:22.328487+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:11:50.495799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:11:50.495799+0000","last_clean_scrub_stamp":"2024-09-11T23:11:50.495799+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:15:53.681230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"85'6","reported_seq":33,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.187094+0000","last_change":"2024-09-11T23:16:23.793996+0000","last_active":"2024-09-11T23:18:22.187094+0000","last_peered":"2024-09-11T23:18:22.187094+0000","last_clean":"2024-09-11T23:18:22.187094+0000","last_became_active":"2024-09-11T23:16:23.793681+0000","last_became_peered":"2024-09-11T23:16:23.793681+0000","last_unstale":"2024-09-11T23:18:22.187094+0000","last_undegraded":"2024-09-11T23:18:22.187094+0000","last_fullsized":"2024-09-11T23:18:22.187094+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:52:50.463623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"85'8","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.182924+0000","last_change":"2024-09-11T23:16:25.905133+0000","last_active":"2024-09-11T23:18:22.182924+0000","last_peered":"2024-09-11T23:18:22.182924+0000","last_clean":"2024-09-11T23:18:22.182924+0000","last_became_active":"2024-09-11T23:16:25.904819+0000","last_became_peered":"2024-09-11T23:16:25.904819+0000","last_unstale":"2024-09-11T23:18:22.182924+0000","last_undegraded":"2024-09-11T23:18:22.182924+0000","last_fullsized":"2024-09-11T23:18:22.182924+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:51:09.664343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"85'4","reported_seq":34,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328367+0000","last_change":"2024-09-11T23:16:36.228806+0000","last_active":"2024-09-11T23:18:22.328367+0000","last_peered":"2024-09-11T23:18:22.328367+0000","last_clean":"2024-09-11T23:18:22.328367+0000","last_became_active":"2024-09-11T23:16:36.228311+0000","last_became_peered":"2024-09-11T23:16:36.228311+0000","last_unstale":"2024-09-11T23:18:22.328367+0000","last_undegraded":"2024-09-11T23:18:22.328367+0000","last_fullsized":"2024-09-11T23:18:22.328367+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:42:32.856691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4],"acting":[7,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"85'5","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328703+0000","last_change":"2024-09-11T23:16:25.898935+0000","last_active":"2024-09-11T23:18:22.328703+0000","last_peered":"2024-09-11T23:18:22.328703+0000","last_clean":"2024-09-11T23:18:22.328703+0000","last_became_active":"2024-09-11T23:16:25.898618+0000","last_became_peered":"2024-09-11T23:16:25.898618+0000","last_unstale":"2024-09-11T23:18:22.328703+0000","last_undegraded":"2024-09-11T23:18:22.328703+0000","last_fullsized":"2024-09-11T23:18:22.328703+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:13:29.104911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"85'5","reported_seq":135,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.329013+0000","last_change":"2024-09-11T23:16:24.073613+0000","last_active":"2024-09-11T23:18:22.329013+0000","last_peered":"2024-09-11T23:18:22.329013+0000","last_clean":"2024-09-11T23:18:22.329013+0000","last_became_active":"2024-09-11T23:16:24.073262+0000","last_became_peered":"2024-09-11T23:16:24.073262+0000","last_unstale":"2024-09-11T23:18:22.329013+0000","last_undegraded":"2024-09-11T23:18:22.329013+0000","last_fullsized":"2024-09-11T23:18:22.329013+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:31:58.653657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":106,"num_read_kb":106,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"85'8","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.184479+0000","last_change":"2024-09-11T23:16:25.903725+0000","last_active":"2024-09-11T23:18:22.184479+0000","last_peered":"2024-09-11T23:18:22.184479+0000","last_clean":"2024-09-11T23:18:22.184479+0000","last_became_active":"2024-09-11T23:16:25.903375+0000","last_became_peered":"2024-09-11T23:16:25.903375+0000","last_unstale":"2024-09-11T23:18:22.184479+0000","last_undegraded":"2024-09-11T23:18:22.184479+0000","last_fullsized":"2024-09-11T23:18:22.184479+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:55:02.779575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"86'6","reported_seq":137,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:23.123405+0000","last_change":"2024-09-11T23:16:23.793932+0000","last_active":"2024-09-11T23:18:23.123405+0000","last_peered":"2024-09-11T23:18:23.123405+0000","last_clean":"2024-09-11T23:18:23.123405+0000","last_became_active":"2024-09-11T23:16:23.793111+0000","last_became_peered":"2024-09-11T23:16:23.793111+0000","last_unstale":"2024-09-11T23:18:23.123405+0000","last_undegraded":"2024-09-11T23:18:23.123405+0000","last_fullsized":"2024-09-11T23:18:23.123405+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:14:09.087222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":107,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"85'7","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.302982+0000","last_change":"2024-09-11T23:16:25.905020+0000","last_active":"2024-09-11T23:18:22.302982+0000","last_peered":"2024-09-11T23:18:22.302982+0000","last_clean":"2024-09-11T23:18:22.302982+0000","last_became_active":"2024-09-11T23:16:25.904675+0000","last_became_peered":"2024-09-11T23:16:25.904675+0000","last_unstale":"2024-09-11T23:18:22.302982+0000","last_undegraded":"2024-09-11T23:18:22.302982+0000","last_fullsized":"2024-09-11T23:18:22.302982+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:13:57.388773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"85'1","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.293261+0000","last_change":"2024-09-11T23:16:24.073630+0000","last_active":"2024-09-11T23:18:22.293261+0000","last_peered":"2024-09-11T23:18:22.293261+0000","last_clean":"2024-09-11T23:18:22.293261+0000","last_became_active":"2024-09-11T23:16:24.073301+0000","last_became_peered":"2024-09-11T23:16:24.073301+0000","last_unstale":"2024-09-11T23:18:22.293261+0000","last_undegraded":"2024-09-11T23:18:22.293261+0000","last_fullsized":"2024-09-11T23:18:22.293261+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:41:59.374581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"85'5","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.303051+0000","last_change":"2024-09-11T23:16:25.894124+0000","last_active":"2024-09-11T23:18:22.303051+0000","last_peered":"2024-09-11T23:18:22.303051+0000","last_clean":"2024-09-11T23:18:22.303051+0000","last_became_active":"2024-09-11T23:16:25.893751+0000","last_became_peered":"2024-09-11T23:16:25.893751+0000","last_unstale":"2024-09-11T23:18:22.303051+0000","last_undegraded":"2024-09-11T23:18:22.303051+0000","last_fullsized":"2024-09-11T23:18:22.303051+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:00:28.265206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"85'4","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.296214+0000","last_change":"2024-09-11T23:16:24.073700+0000","last_active":"2024-09-11T23:18:22.296214+0000","last_peered":"2024-09-11T23:18:22.296214+0000","last_clean":"2024-09-11T23:18:22.296214+0000","last_became_active":"2024-09-11T23:16:24.073283+0000","last_became_peered":"2024-09-11T23:16:24.073283+0000","last_unstale":"2024-09-11T23:18:22.296214+0000","last_undegraded":"2024-09-11T23:18:22.296214+0000","last_fullsized":"2024-09-11T23:18:22.296214+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:41:43.536834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"85'9","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.329920+0000","last_change":"2024-09-11T23:16:25.900532+0000","last_active":"2024-09-11T23:18:22.329920+0000","last_peered":"2024-09-11T23:18:22.329920+0000","last_clean":"2024-09-11T23:18:22.329920+0000","last_became_active":"2024-09-11T23:16:25.899466+0000","last_became_peered":"2024-09-11T23:16:25.899466+0000","last_unstale":"2024-09-11T23:18:22.329920+0000","last_undegraded":"2024-09-11T23:18:22.329920+0000","last_fullsized":"2024-09-11T23:18:22.329920+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:10:01.152323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"86'50","reported_seq":97,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.432609+0000","last_change":"2024-09-11T23:16:23.788789+0000","last_active":"2024-09-11T23:18:22.432609+0000","last_peered":"2024-09-11T23:18:22.432609+0000","last_clean":"2024-09-11T23:18:22.432609+0000","last_became_active":"2024-09-11T23:16:23.788422+0000","last_became_peered":"2024-09-11T23:16:23.788422+0000","last_unstale":"2024-09-11T23:18:22.432609+0000","last_undegraded":"2024-09-11T23:18:22.432609+0000","last_fullsized":"2024-09-11T23:18:22.432609+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":0,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:09:14.310205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":89,"num_write_kb":837,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"85'5","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.183270+0000","last_change":"2024-09-11T23:16:25.895511+0000","last_active":"2024-09-11T23:18:22.183270+0000","last_peered":"2024-09-11T23:18:22.183270+0000","last_clean":"2024-09-11T23:18:22.183270+0000","last_became_active":"2024-09-11T23:16:25.895143+0000","last_became_peered":"2024-09-11T23:16:25.895143+0000","last_unstale":"2024-09-11T23:18:22.183270+0000","last_undegraded":"2024-09-11T23:18:22.183270+0000","last_fullsized":"2024-09-11T23:18:22.183270+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:31:16.498371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"85'7","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.187209+0000","last_change":"2024-09-11T23:16:24.019064+0000","last_active":"2024-09-11T23:18:22.187209+0000","last_peered":"2024-09-11T23:18:22.187209+0000","last_clean":"2024-09-11T23:18:22.187209+0000","last_became_active":"2024-09-11T23:16:24.018658+0000","last_became_peered":"2024-09-11T23:16:24.018658+0000","last_unstale":"2024-09-11T23:18:22.187209+0000","last_undegraded":"2024-09-11T23:18:22.187209+0000","last_fullsized":"2024-09-11T23:18:22.187209+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:17:41.009507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"85'7","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.544339+0000","last_change":"2024-09-11T23:16:25.903553+0000","last_active":"2024-09-11T23:18:22.544339+0000","last_peered":"2024-09-11T23:18:22.544339+0000","last_clean":"2024-09-11T23:18:22.544339+0000","last_became_active":"2024-09-11T23:16:25.903221+0000","last_became_peered":"2024-09-11T23:16:25.903221+0000","last_unstale":"2024-09-11T23:18:22.544339+0000","last_undegraded":"2024-09-11T23:18:22.544339+0000","last_fullsized":"2024-09-11T23:18:22.544339+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:16:51.872517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"83'2","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328140+0000","last_change":"2024-09-11T23:16:24.073784+0000","last_active":"2024-09-11T23:18:22.328140+0000","last_peered":"2024-09-11T23:18:22.328140+0000","last_clean":"2024-09-11T23:18:22.328140+0000","last_became_active":"2024-09-11T23:16:24.073406+0000","last_became_peered":"2024-09-11T23:16:24.073406+0000","last_unstale":"2024-09-11T23:18:22.328140+0000","last_undegraded":"2024-09-11T23:18:22.328140+0000","last_fullsized":"2024-09-11T23:18:22.328140+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:31:24.617025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"85'2","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.295796+0000","last_change":"2024-09-11T23:16:25.894878+0000","last_active":"2024-09-11T23:18:22.295796+0000","last_peered":"2024-09-11T23:18:22.295796+0000","last_clean":"2024-09-11T23:18:22.295796+0000","last_became_active":"2024-09-11T23:16:25.894543+0000","last_became_peered":"2024-09-11T23:16:25.894543+0000","last_unstale":"2024-09-11T23:18:22.295796+0000","last_undegraded":"2024-09-11T23:18:22.295796+0000","last_fullsized":"2024-09-11T23:18:22.295796+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:34:13.435754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"85'7","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.182832+0000","last_change":"2024-09-11T23:16:23.786502+0000","last_active":"2024-09-11T23:18:22.182832+0000","last_peered":"2024-09-11T23:18:22.182832+0000","last_clean":"2024-09-11T23:18:22.182832+0000","last_became_active":"2024-09-11T23:16:23.786182+0000","last_became_peered":"2024-09-11T23:16:23.786182+0000","last_unstale":"2024-09-11T23:18:22.182832+0000","last_undegraded":"2024-09-11T23:18:22.182832+0000","last_fullsized":"2024-09-11T23:18:22.182832+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:59:07.978244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"85'9","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328520+0000","last_change":"2024-09-11T23:16:25.899625+0000","last_active":"2024-09-11T23:18:22.328520+0000","last_peered":"2024-09-11T23:18:22.328520+0000","last_clean":"2024-09-11T23:18:22.328520+0000","last_became_active":"2024-09-11T23:16:25.899265+0000","last_became_peered":"2024-09-11T23:16:25.899265+0000","last_unstale":"2024-09-11T23:18:22.328520+0000","last_undegraded":"2024-09-11T23:18:22.328520+0000","last_fullsized":"2024-09-11T23:18:22.328520+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:36:06.907622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"86'4","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328472+0000","last_change":"2024-09-11T23:16:36.228991+0000","last_active":"2024-09-11T23:18:22.328472+0000","last_peered":"2024-09-11T23:18:22.328472+0000","last_clean":"2024-09-11T23:18:22.328472+0000","last_became_active":"2024-09-11T23:16:36.227701+0000","last_became_peered":"2024-09-11T23:16:36.227701+0000","last_unstale":"2024-09-11T23:18:22.328472+0000","last_undegraded":"2024-09-11T23:18:22.328472+0000","last_fullsized":"2024-09-11T23:18:22.328472+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:37:57.478572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"85'8","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.182913+0000","last_change":"2024-09-11T23:16:25.894803+0000","last_active":"2024-09-11T23:18:22.182913+0000","last_peered":"2024-09-11T23:18:22.182913+0000","last_clean":"2024-09-11T23:18:22.182913+0000","last_became_active":"2024-09-11T23:16:25.894429+0000","last_became_peered":"2024-09-11T23:16:25.894429+0000","last_unstale":"2024-09-11T23:18:22.182913+0000","last_undegraded":"2024-09-11T23:18:22.182913+0000","last_fullsized":"2024-09-11T23:18:22.182913+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:39:15.868910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"83'4","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.295871+0000","last_change":"2024-09-11T23:16:23.794271+0000","last_active":"2024-09-11T23:18:22.295871+0000","last_peered":"2024-09-11T23:18:22.295871+0000","last_clean":"2024-09-11T23:18:22.295871+0000","last_became_active":"2024-09-11T23:16:23.793919+0000","last_became_peered":"2024-09-11T23:16:23.793919+0000","last_unstale":"2024-09-11T23:18:22.295871+0000","last_undegraded":"2024-09-11T23:18:22.295871+0000","last_fullsized":"2024-09-11T23:18:22.295871+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:30:28.907312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"85'5","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328395+0000","last_change":"2024-09-11T23:16:25.900969+0000","last_active":"2024-09-11T23:18:22.328395+0000","last_peered":"2024-09-11T23:18:22.328395+0000","last_clean":"2024-09-11T23:18:22.328395+0000","last_became_active":"2024-09-11T23:16:25.900425+0000","last_became_peered":"2024-09-11T23:16:25.900425+0000","last_unstale":"2024-09-11T23:18:22.328395+0000","last_undegraded":"2024-09-11T23:18:22.328395+0000","last_fullsized":"2024-09-11T23:18:22.328395+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:03:25.185773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"85'15","reported_seq":34,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.184711+0000","last_change":"2024-09-11T23:16:25.911605+0000","last_active":"2024-09-11T23:18:22.184711+0000","last_peered":"2024-09-11T23:18:22.184711+0000","last_clean":"2024-09-11T23:18:22.184711+0000","last_became_active":"2024-09-11T23:16:25.911180+0000","last_became_peered":"2024-09-11T23:16:25.911180+0000","last_unstale":"2024-09-11T23:18:22.184711+0000","last_undegraded":"2024-09-11T23:18:22.184711+0000","last_fullsized":"2024-09-11T23:18:22.184711+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:55:38.062606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"85'5","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328425+0000","last_change":"2024-09-11T23:16:35.949528+0000","last_active":"2024-09-11T23:18:22.328425+0000","last_peered":"2024-09-11T23:18:22.328425+0000","last_clean":"2024-09-11T23:18:22.328425+0000","last_became_active":"2024-09-11T23:16:35.945679+0000","last_became_peered":"2024-09-11T23:16:35.945679+0000","last_unstale":"2024-09-11T23:18:22.328425+0000","last_undegraded":"2024-09-11T23:18:22.328425+0000","last_fullsized":"2024-09-11T23:18:22.328425+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:14:24.706044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"85'9","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328158+0000","last_change":"2024-09-11T23:16:25.900526+0000","last_active":"2024-09-11T23:18:22.328158+0000","last_peered":"2024-09-11T23:18:22.328158+0000","last_clean":"2024-09-11T23:18:22.328158+0000","last_became_active":"2024-09-11T23:16:25.900148+0000","last_became_peered":"2024-09-11T23:16:25.900148+0000","last_unstale":"2024-09-11T23:18:22.328158+0000","last_undegraded":"2024-09-11T23:18:22.328158+0000","last_fullsized":"2024-09-11T23:18:22.328158+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:26:10.582473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"86'13","reported_seq":48,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.296144+0000","last_change":"2024-09-11T23:16:36.233273+0000","last_active":"2024-09-11T23:18:22.296144+0000","last_peered":"2024-09-11T23:18:22.296144+0000","last_clean":"2024-09-11T23:18:22.296144+0000","last_became_active":"2024-09-11T23:16:36.228358+0000","last_became_peered":"2024-09-11T23:16:36.228358+0000","last_unstale":"2024-09-11T23:18:22.296144+0000","last_undegraded":"2024-09-11T23:18:22.296144+0000","last_fullsized":"2024-09-11T23:18:22.296144+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:06:31.225795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"85'6","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.544571+0000","last_change":"2024-09-11T23:16:25.907874+0000","last_active":"2024-09-11T23:18:22.544571+0000","last_peered":"2024-09-11T23:18:22.544571+0000","last_clean":"2024-09-11T23:18:22.544571+0000","last_became_active":"2024-09-11T23:16:25.907491+0000","last_became_peered":"2024-09-11T23:16:25.907491+0000","last_unstale":"2024-09-11T23:18:22.544571+0000","last_undegraded":"2024-09-11T23:18:22.544571+0000","last_fullsized":"2024-09-11T23:18:22.544571+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:09:20.984249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"84'2","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.183287+0000","last_change":"2024-09-11T23:16:24.074355+0000","last_active":"2024-09-11T23:18:22.183287+0000","last_peered":"2024-09-11T23:18:22.183287+0000","last_clean":"2024-09-11T23:18:22.183287+0000","last_became_active":"2024-09-11T23:16:24.074047+0000","last_became_peered":"2024-09-11T23:16:24.074047+0000","last_unstale":"2024-09-11T23:18:22.183287+0000","last_undegraded":"2024-09-11T23:18:22.183287+0000","last_fullsized":"2024-09-11T23:18:22.183287+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:31:58.804420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"85'5","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.182923+0000","last_change":"2024-09-11T23:16:25.907701+0000","last_active":"2024-09-11T23:18:22.182923+0000","last_peered":"2024-09-11T23:18:22.182923+0000","last_clean":"2024-09-11T23:18:22.182923+0000","last_became_active":"2024-09-11T23:16:25.907379+0000","last_became_peered":"2024-09-11T23:16:25.907379+0000","last_unstale":"2024-09-11T23:18:22.182923+0000","last_undegraded":"2024-09-11T23:18:22.182923+0000","last_fullsized":"2024-09-11T23:18:22.182923+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:12:03.639055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"86'14","reported_seq":144,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.497378+0000","last_change":"2024-09-11T23:16:23.794293+0000","last_active":"2024-09-11T23:18:22.497378+0000","last_peered":"2024-09-11T23:18:22.497378+0000","last_clean":"2024-09-11T23:18:22.497378+0000","last_became_active":"2024-09-11T23:16:23.793974+0000","last_became_peered":"2024-09-11T23:16:23.793974+0000","last_unstale":"2024-09-11T23:18:22.497378+0000","last_undegraded":"2024-09-11T23:18:22.497378+0000","last_fullsized":"2024-09-11T23:18:22.497378+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:24:39.170188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":106,"num_read_kb":106,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":599346,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1189,"num_read_kb":709,"num_write":372,"num_write_kb":1462,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2174976,"data_stored":1798038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":298,"ondisk_log_size":298,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":11,"up_from":77,"seq":330712481829,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32128,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713024,"statfs":{"total":95995035648,"available":95962136576,"internally_reserved":0,"allocated":962560,"data_stored":459042,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":70,"seq":300647710761,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28844,"kb_used_data":1760,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716308,"statfs":{"total":95995035648,"available":95965499392,"internally_reserved":0,"allocated":1802240,"data_stored":1247035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":65,"seq":279172874287,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28372,"kb_used_data":1344,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716780,"statfs":{"total":95995035648,"available":95965982720,"internally_reserved":0,"allocated":1376256,"data_stored":851072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":57,"seq":244813135924,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":958464,"data_stored":448696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":50,"seq":214748364856,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28568,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716584,"statfs":{"total":95995035648,"available":95965782016,"internally_reserved":0,"allocated":1576960,"data_stored":1051455,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528380,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36292,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708860,"statfs":{"total":95995035648,"available":95957872640,"internally_reserved":0,"allocated":966656,"data_stored":457966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790019,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28768,"kb_used_data":1684,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716384,"statfs":{"total":95995035648,"available":95965577216,"internally_reserved":0,"allocated":1724416,"data_stored":1182340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920840,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36292,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708860,"statfs":{"total":95995035648,"available":95957872640,"internally_reserved":0,"allocated":966656,"data_stored":457434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182477,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":41160,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93703992,"statfs":{"total":95995035648,"available":95952887808,"internally_reserved":0,"allocated":1511424,"data_stored":982214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411410,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36380,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708772,"statfs":{"total":95995035648,"available":95957782528,"internally_reserved":0,"allocated":1056768,"data_stored":526421,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574935,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1036288,"data_stored":525359,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738460,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":970752,"data_stored":450490,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":68625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":69687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":557056,"data_stored":525246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":180224,"data_stored":135121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":421888,"data_stored":394134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":394338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":258048,"data_stored":199816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2308,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T23:18:25.611 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-09-11T23:18:25.611 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T23:18:25.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:25.754+0000 7f7b70b5f640 1 -- 172.21.15.26:0/239015168 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b6c104f60 msgr2=0x7f7b6c107350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:25.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:25.754+0000 7f7b70b5f640 1 --2- 172.21.15.26:0/239015168 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b6c104f60 0x7f7b6c107350 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f7b54009900 tx=0x7f7b5402a410 comp rx=0 tx=0).stop 2024-09-11T23:18:25.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:25.754+0000 7f7b70b5f640 1 -- 172.21.15.26:0/239015168 shutdown_connections 2024-09-11T23:18:25.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:25.754+0000 7f7b70b5f640 1 --2- 172.21.15.26:0/239015168 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7b6c107890 0x7f7b6c109c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:25.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:25.754+0000 7f7b70b5f640 1 --2- 172.21.15.26:0/239015168 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b6c104f60 0x7f7b6c107350 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:25.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:25.754+0000 7f7b70b5f640 1 --2- 172.21.15.26:0/239015168 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7b6c102630 0x7f7b6c104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:25.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:25.754+0000 7f7b70b5f640 1 -- 172.21.15.26:0/239015168 >> 172.21.15.26:0/239015168 conn(0x7f7b6c0fc170 msgr2=0x7f7b6c0fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:18:25.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:25.754+0000 7f7b70b5f640 1 -- 172.21.15.26:0/239015168 shutdown_connections 2024-09-11T23:18:25.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:25.754+0000 7f7b70b5f640 1 -- 172.21.15.26:0/239015168 wait complete. 2024-09-11T23:18:25.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:25.755+0000 7f7b70b5f640 1 Processor -- start 2024-09-11T23:18:25.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:25.755+0000 7f7b70b5f640 1 -- start start 2024-09-11T23:18:25.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:25.755+0000 7f7b70b5f640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7b6c102630 0x7f7b6c197f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:25.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:25.755+0000 7f7b70b5f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b6c104f60 0x7f7b6c198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:25.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:25.755+0000 7f7b70b5f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7b6c107890 0x7f7b6c19f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:25.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:25.755+0000 7f7b70b5f640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f7b6c10d7f0 con 0x7f7b6c104f60 2024-09-11T23:18:25.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:25.755+0000 7f7b70b5f640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f7b6c10d670 con 0x7f7b6c107890 2024-09-11T23:18:25.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:25.755+0000 7f7b70b5f640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f7b6c10d970 con 0x7f7b6c102630 2024-09-11T23:18:25.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:25.755+0000 7f7b63fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b6c104f60 0x7f7b6c198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:25.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:25.755+0000 7f7b63fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b6c104f60 0x7f7b6c198450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47402/0 (socket says 172.21.15.26:47402) 2024-09-11T23:18:25.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:25.755+0000 7f7b63fff640 1 -- 172.21.15.26:0/3108837959 learned_addr learned my addr 172.21.15.26:0/3108837959 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:18:25.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:25.755+0000 7f7b6a59c640 1 --2- 172.21.15.26:0/3108837959 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7b6c102630 0x7f7b6c197f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:25.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:25.755+0000 7f7b6ad9d640 1 --2- 172.21.15.26:0/3108837959 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7b6c107890 0x7f7b6c19f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:25.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:25.755+0000 7f7b63fff640 1 -- 172.21.15.26:0/3108837959 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7b6c102630 msgr2=0x7f7b6c197f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:25.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:25.755+0000 7f7b63fff640 1 --2- 172.21.15.26:0/3108837959 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7b6c102630 0x7f7b6c197f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:25.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:25.755+0000 7f7b63fff640 1 -- 172.21.15.26:0/3108837959 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7b6c107890 msgr2=0x7f7b6c19f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:25.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:25.755+0000 7f7b63fff640 1 --2- 172.21.15.26:0/3108837959 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7b6c107890 0x7f7b6c19f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:25.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:25.755+0000 7f7b63fff640 1 -- 172.21.15.26:0/3108837959 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b6c19fb30 con 0x7f7b6c104f60 2024-09-11T23:18:25.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:25.756+0000 7f7b63fff640 1 --2- 172.21.15.26:0/3108837959 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b6c104f60 0x7f7b6c198450 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f7b54031440 tx=0x7f7b54032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:18:25.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:25.756+0000 7f7b637fe640 1 -- 172.21.15.26:0/3108837959 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b5403f030 con 0x7f7b6c104f60 2024-09-11T23:18:25.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:25.756+0000 7f7b70b5f640 1 -- 172.21.15.26:0/3108837959 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b6c19fdc0 con 0x7f7b6c104f60 2024-09-11T23:18:25.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:25.756+0000 7f7b637fe640 1 -- 172.21.15.26:0/3108837959 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b54003ce0 con 0x7f7b6c104f60 2024-09-11T23:18:25.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:25.756+0000 7f7b637fe640 1 -- 172.21.15.26:0/3108837959 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b54035070 con 0x7f7b6c104f60 2024-09-11T23:18:25.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:25.756+0000 7f7b70b5f640 1 -- 172.21.15.26:0/3108837959 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b6c1a0270 con 0x7f7b6c104f60 2024-09-11T23:18:25.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:25.757+0000 7f7b70b5f640 1 -- 172.21.15.26:0/3108837959 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b6c0fdc60 con 0x7f7b6c104f60 2024-09-11T23:18:25.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:25.759+0000 7f7b637fe640 1 -- 172.21.15.26:0/3108837959 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 23) ==== 137953+0+0 (secure 0 0 0) 0x7f7b54042030 con 0x7f7b6c104f60 2024-09-11T23:18:25.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:25.760+0000 7f7b637fe640 1 --2- 172.21.15.26:0/3108837959 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f7b580a2d00 0x7f7b580a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:25.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:25.760+0000 7f7b637fe640 1 -- 172.21.15.26:0/3108837959 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f7b540ef5f0 con 0x7f7b6c104f60 2024-09-11T23:18:25.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:25.760+0000 7f7b6a59c640 1 --2- 172.21.15.26:0/3108837959 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f7b580a2d00 0x7f7b580a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:25.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:25.760+0000 7f7b6a59c640 1 --2- 172.21.15.26:0/3108837959 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f7b580a2d00 0x7f7b580a51c0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f7b5c009bd0 tx=0x7f7b5c005f90 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:18:25.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:25.765+0000 7f7b637fe640 1 -- 172.21.15.26:0/3108837959 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7b540bc220 con 0x7f7b6c104f60 2024-09-11T23:18:25.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:25.947+0000 7f7b70b5f640 1 -- 172.21.15.26:0/3108837959 --> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7b6c062630 con 0x7f7b580a2d00 2024-09-11T23:18:25.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:25.953+0000 7f7b637fe640 1 -- 172.21.15.26:0/3108837959 <== mgr.15171 v2:172.21.15.26:6800/1989555790 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+257778 (secure 0 0 0) 0x7f7b6c062630 con 0x7f7b580a2d00 2024-09-11T23:18:25.957 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-11T23:18:25.958 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-09-11T23:18:25.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:25.956+0000 7f7b70b5f640 1 -- 172.21.15.26:0/3108837959 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f7b580a2d00 msgr2=0x7f7b580a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:25.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:25.956+0000 7f7b70b5f640 1 --2- 172.21.15.26:0/3108837959 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f7b580a2d00 0x7f7b580a51c0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f7b5c009bd0 tx=0x7f7b5c005f90 comp rx=0 tx=0).stop 2024-09-11T23:18:25.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:25.956+0000 7f7b70b5f640 1 -- 172.21.15.26:0/3108837959 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b6c104f60 msgr2=0x7f7b6c198450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:25.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:25.956+0000 7f7b70b5f640 1 --2- 172.21.15.26:0/3108837959 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b6c104f60 0x7f7b6c198450 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f7b54031440 tx=0x7f7b54032ce0 comp rx=0 tx=0).stop 2024-09-11T23:18:25.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:25.957+0000 7f7b70b5f640 1 -- 172.21.15.26:0/3108837959 shutdown_connections 2024-09-11T23:18:25.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:25.957+0000 7f7b70b5f640 1 --2- 172.21.15.26:0/3108837959 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f7b580a2d00 0x7f7b580a51c0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:25.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:25.957+0000 7f7b70b5f640 1 --2- 172.21.15.26:0/3108837959 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7b6c107890 0x7f7b6c19f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:25.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:25.957+0000 7f7b70b5f640 1 --2- 172.21.15.26:0/3108837959 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b6c104f60 0x7f7b6c198450 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:25.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:25.957+0000 7f7b70b5f640 1 --2- 172.21.15.26:0/3108837959 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7b6c102630 0x7f7b6c197f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:25.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:25.957+0000 7f7b70b5f640 1 -- 172.21.15.26:0/3108837959 >> 172.21.15.26:0/3108837959 conn(0x7f7b6c0fc170 msgr2=0x7f7b6c102fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:18:25.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:25.957+0000 7f7b70b5f640 1 -- 172.21.15.26:0/3108837959 shutdown_connections 2024-09-11T23:18:25.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:25.957+0000 7f7b70b5f640 1 -- 172.21.15.26:0/3108837959 wait complete. 2024-09-11T23:18:25.969 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":4,"stamp":"2024-09-11T23:18:24.197005+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1189948,"num_objects":104,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1325,"num_read_kb":828,"num_write":555,"num_write_kb":4402,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":937,"ondisk_log_size":937,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":380772,"kb_used_data":14560,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124561052,"statfs":{"total":1151940427776,"available":1151550517248,"internally_reserved":0,"allocated":14909440,"data_stored":8639524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":374453561},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.991701"},"pg_stats":[{"pgid":"3.3f","version":"85'8","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.329067+0000","last_change":"2024-09-11T23:16:25.899106+0000","last_active":"2024-09-11T23:18:22.329067+0000","last_peered":"2024-09-11T23:18:22.329067+0000","last_clean":"2024-09-11T23:18:22.329067+0000","last_became_active":"2024-09-11T23:16:25.898654+0000","last_became_peered":"2024-09-11T23:16:25.898654+0000","last_unstale":"2024-09-11T23:18:22.329067+0000","last_undegraded":"2024-09-11T23:18:22.329067+0000","last_fullsized":"2024-09-11T23:18:22.329067+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:21:41.490905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"85'4","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.544200+0000","last_change":"2024-09-11T23:16:25.895629+0000","last_active":"2024-09-11T23:18:22.544200+0000","last_peered":"2024-09-11T23:18:22.544200+0000","last_clean":"2024-09-11T23:18:22.544200+0000","last_became_active":"2024-09-11T23:16:25.895308+0000","last_became_peered":"2024-09-11T23:16:25.895308+0000","last_unstale":"2024-09-11T23:18:22.544200+0000","last_undegraded":"2024-09-11T23:18:22.544200+0000","last_fullsized":"2024-09-11T23:18:22.544200+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:39:42.549970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"85'7","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.544117+0000","last_change":"2024-09-11T23:16:25.907113+0000","last_active":"2024-09-11T23:18:22.544117+0000","last_peered":"2024-09-11T23:18:22.544117+0000","last_clean":"2024-09-11T23:18:22.544117+0000","last_became_active":"2024-09-11T23:16:25.906803+0000","last_became_peered":"2024-09-11T23:16:25.906803+0000","last_unstale":"2024-09-11T23:18:22.544117+0000","last_undegraded":"2024-09-11T23:18:22.544117+0000","last_fullsized":"2024-09-11T23:18:22.544117+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:36:49.303932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"85'8","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.292802+0000","last_change":"2024-09-11T23:16:35.946167+0000","last_active":"2024-09-11T23:18:22.292802+0000","last_peered":"2024-09-11T23:18:22.292802+0000","last_clean":"2024-09-11T23:18:22.292802+0000","last_became_active":"2024-09-11T23:16:35.945786+0000","last_became_peered":"2024-09-11T23:16:35.945786+0000","last_unstale":"2024-09-11T23:18:22.292802+0000","last_undegraded":"2024-09-11T23:18:22.292802+0000","last_fullsized":"2024-09-11T23:18:22.292802+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:13:03.064910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"85'4","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.295860+0000","last_change":"2024-09-11T23:16:25.897909+0000","last_active":"2024-09-11T23:18:22.295860+0000","last_peered":"2024-09-11T23:18:22.295860+0000","last_clean":"2024-09-11T23:18:22.295860+0000","last_became_active":"2024-09-11T23:16:25.897569+0000","last_became_peered":"2024-09-11T23:16:25.897569+0000","last_unstale":"2024-09-11T23:18:22.295860+0000","last_undegraded":"2024-09-11T23:18:22.295860+0000","last_fullsized":"2024-09-11T23:18:22.295860+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:25:44.727254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"85'6","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.302845+0000","last_change":"2024-09-11T23:16:25.897594+0000","last_active":"2024-09-11T23:18:22.302845+0000","last_peered":"2024-09-11T23:18:22.302845+0000","last_clean":"2024-09-11T23:18:22.302845+0000","last_became_active":"2024-09-11T23:16:25.897223+0000","last_became_peered":"2024-09-11T23:16:25.897223+0000","last_unstale":"2024-09-11T23:18:22.302845+0000","last_undegraded":"2024-09-11T23:18:22.302845+0000","last_fullsized":"2024-09-11T23:18:22.302845+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:10:50.763455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"85'5","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328293+0000","last_change":"2024-09-11T23:16:35.943511+0000","last_active":"2024-09-11T23:18:22.328293+0000","last_peered":"2024-09-11T23:18:22.328293+0000","last_clean":"2024-09-11T23:18:22.328293+0000","last_became_active":"2024-09-11T23:16:35.943075+0000","last_became_peered":"2024-09-11T23:16:35.943075+0000","last_unstale":"2024-09-11T23:18:22.328293+0000","last_undegraded":"2024-09-11T23:18:22.328293+0000","last_fullsized":"2024-09-11T23:18:22.328293+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:53:43.892421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"85'9","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.182677+0000","last_change":"2024-09-11T23:16:25.903380+0000","last_active":"2024-09-11T23:18:22.182677+0000","last_peered":"2024-09-11T23:18:22.182677+0000","last_clean":"2024-09-11T23:18:22.182677+0000","last_became_active":"2024-09-11T23:16:25.903093+0000","last_became_peered":"2024-09-11T23:16:25.903093+0000","last_unstale":"2024-09-11T23:18:22.182677+0000","last_undegraded":"2024-09-11T23:18:22.182677+0000","last_fullsized":"2024-09-11T23:18:22.182677+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:58:50.944623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"85'11","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.329153+0000","last_change":"2024-09-11T23:16:25.908166+0000","last_active":"2024-09-11T23:18:22.329153+0000","last_peered":"2024-09-11T23:18:22.329153+0000","last_clean":"2024-09-11T23:18:22.329153+0000","last_became_active":"2024-09-11T23:16:25.907749+0000","last_became_peered":"2024-09-11T23:16:25.907749+0000","last_unstale":"2024-09-11T23:18:22.329153+0000","last_undegraded":"2024-09-11T23:18:22.329153+0000","last_fullsized":"2024-09-11T23:18:22.329153+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:11:16.705931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"85'16","reported_seq":41,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.303286+0000","last_change":"2024-09-11T23:16:35.944259+0000","last_active":"2024-09-11T23:18:22.303286+0000","last_peered":"2024-09-11T23:18:22.303286+0000","last_clean":"2024-09-11T23:18:22.303286+0000","last_became_active":"2024-09-11T23:16:35.943767+0000","last_became_peered":"2024-09-11T23:16:35.943767+0000","last_unstale":"2024-09-11T23:18:22.303286+0000","last_undegraded":"2024-09-11T23:18:22.303286+0000","last_fullsized":"2024-09-11T23:18:22.303286+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:20:13.127854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"85'6","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328788+0000","last_change":"2024-09-11T23:16:25.903256+0000","last_active":"2024-09-11T23:18:22.328788+0000","last_peered":"2024-09-11T23:18:22.328788+0000","last_clean":"2024-09-11T23:18:22.328788+0000","last_became_active":"2024-09-11T23:16:25.902954+0000","last_became_peered":"2024-09-11T23:16:25.902954+0000","last_unstale":"2024-09-11T23:18:22.328788+0000","last_undegraded":"2024-09-11T23:18:22.328788+0000","last_fullsized":"2024-09-11T23:18:22.328788+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:43:48.651400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"85'7","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.187001+0000","last_change":"2024-09-11T23:16:25.897656+0000","last_active":"2024-09-11T23:18:22.187001+0000","last_peered":"2024-09-11T23:18:22.187001+0000","last_clean":"2024-09-11T23:18:22.187001+0000","last_became_active":"2024-09-11T23:16:25.897103+0000","last_became_peered":"2024-09-11T23:16:25.897103+0000","last_unstale":"2024-09-11T23:18:22.187001+0000","last_undegraded":"2024-09-11T23:18:22.187001+0000","last_fullsized":"2024-09-11T23:18:22.187001+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:14:34.341383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"85'10","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.329625+0000","last_change":"2024-09-11T23:16:25.902800+0000","last_active":"2024-09-11T23:18:22.329625+0000","last_peered":"2024-09-11T23:18:22.329625+0000","last_clean":"2024-09-11T23:18:22.329625+0000","last_became_active":"2024-09-11T23:16:25.902414+0000","last_became_peered":"2024-09-11T23:16:25.902414+0000","last_unstale":"2024-09-11T23:18:22.329625+0000","last_undegraded":"2024-09-11T23:18:22.329625+0000","last_fullsized":"2024-09-11T23:18:22.329625+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:16:32.427749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"85'7","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.543876+0000","last_change":"2024-09-11T23:16:25.894410+0000","last_active":"2024-09-11T23:18:22.543876+0000","last_peered":"2024-09-11T23:18:22.543876+0000","last_clean":"2024-09-11T23:18:22.543876+0000","last_became_active":"2024-09-11T23:16:25.894067+0000","last_became_peered":"2024-09-11T23:16:25.894067+0000","last_unstale":"2024-09-11T23:18:22.543876+0000","last_undegraded":"2024-09-11T23:18:22.543876+0000","last_fullsized":"2024-09-11T23:18:22.543876+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:03:11.913063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"85'7","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.544140+0000","last_change":"2024-09-11T23:16:25.903440+0000","last_active":"2024-09-11T23:18:22.544140+0000","last_peered":"2024-09-11T23:18:22.544140+0000","last_clean":"2024-09-11T23:18:22.544140+0000","last_became_active":"2024-09-11T23:16:25.903122+0000","last_became_peered":"2024-09-11T23:16:25.903122+0000","last_unstale":"2024-09-11T23:18:22.544140+0000","last_undegraded":"2024-09-11T23:18:22.544140+0000","last_fullsized":"2024-09-11T23:18:22.544140+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:03:41.065314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"85'9","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328034+0000","last_change":"2024-09-11T23:16:25.905336+0000","last_active":"2024-09-11T23:18:22.328034+0000","last_peered":"2024-09-11T23:18:22.328034+0000","last_clean":"2024-09-11T23:18:22.328034+0000","last_became_active":"2024-09-11T23:16:25.904699+0000","last_became_peered":"2024-09-11T23:16:25.904699+0000","last_unstale":"2024-09-11T23:18:22.328034+0000","last_undegraded":"2024-09-11T23:18:22.328034+0000","last_fullsized":"2024-09-11T23:18:22.328034+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:16:13.448585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"85'8","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.329144+0000","last_change":"2024-09-11T23:16:25.897977+0000","last_active":"2024-09-11T23:18:22.329144+0000","last_peered":"2024-09-11T23:18:22.329144+0000","last_clean":"2024-09-11T23:18:22.329144+0000","last_became_active":"2024-09-11T23:16:25.897672+0000","last_became_peered":"2024-09-11T23:16:25.897672+0000","last_unstale":"2024-09-11T23:18:22.329144+0000","last_undegraded":"2024-09-11T23:18:22.329144+0000","last_fullsized":"2024-09-11T23:18:22.329144+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:42:56.575066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"85'10","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.293509+0000","last_change":"2024-09-11T23:16:25.897063+0000","last_active":"2024-09-11T23:18:22.293509+0000","last_peered":"2024-09-11T23:18:22.293509+0000","last_clean":"2024-09-11T23:18:22.293509+0000","last_became_active":"2024-09-11T23:16:25.896770+0000","last_became_peered":"2024-09-11T23:16:25.896770+0000","last_unstale":"2024-09-11T23:18:22.293509+0000","last_undegraded":"2024-09-11T23:18:22.293509+0000","last_fullsized":"2024-09-11T23:18:22.293509+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:55:44.735171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"85'7","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328943+0000","last_change":"2024-09-11T23:16:25.903423+0000","last_active":"2024-09-11T23:18:22.328943+0000","last_peered":"2024-09-11T23:18:22.328943+0000","last_clean":"2024-09-11T23:18:22.328943+0000","last_became_active":"2024-09-11T23:16:25.902906+0000","last_became_peered":"2024-09-11T23:16:25.902906+0000","last_unstale":"2024-09-11T23:18:22.328943+0000","last_undegraded":"2024-09-11T23:18:22.328943+0000","last_fullsized":"2024-09-11T23:18:22.328943+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:40:49.326045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"85'8","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328557+0000","last_change":"2024-09-11T23:16:25.896564+0000","last_active":"2024-09-11T23:18:22.328557+0000","last_peered":"2024-09-11T23:18:22.328557+0000","last_clean":"2024-09-11T23:18:22.328557+0000","last_became_active":"2024-09-11T23:16:25.896301+0000","last_became_peered":"2024-09-11T23:16:25.896301+0000","last_unstale":"2024-09-11T23:18:22.328557+0000","last_undegraded":"2024-09-11T23:18:22.328557+0000","last_fullsized":"2024-09-11T23:18:22.328557+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:30:30.074853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"85'11","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.329804+0000","last_change":"2024-09-11T23:16:25.903491+0000","last_active":"2024-09-11T23:18:22.329804+0000","last_peered":"2024-09-11T23:18:22.329804+0000","last_clean":"2024-09-11T23:18:22.329804+0000","last_became_active":"2024-09-11T23:16:25.901340+0000","last_became_peered":"2024-09-11T23:16:25.901340+0000","last_unstale":"2024-09-11T23:18:22.329804+0000","last_undegraded":"2024-09-11T23:18:22.329804+0000","last_fullsized":"2024-09-11T23:18:22.329804+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:41:16.058034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"85'10","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328233+0000","last_change":"2024-09-11T23:16:25.901084+0000","last_active":"2024-09-11T23:18:22.328233+0000","last_peered":"2024-09-11T23:18:22.328233+0000","last_clean":"2024-09-11T23:18:22.328233+0000","last_became_active":"2024-09-11T23:16:25.900726+0000","last_became_peered":"2024-09-11T23:16:25.900726+0000","last_unstale":"2024-09-11T23:18:22.328233+0000","last_undegraded":"2024-09-11T23:18:22.328233+0000","last_fullsized":"2024-09-11T23:18:22.328233+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:57:40.955645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"85'8","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.182688+0000","last_change":"2024-09-11T23:16:25.900461+0000","last_active":"2024-09-11T23:18:22.182688+0000","last_peered":"2024-09-11T23:18:22.182688+0000","last_clean":"2024-09-11T23:18:22.182688+0000","last_became_active":"2024-09-11T23:16:25.900174+0000","last_became_peered":"2024-09-11T23:16:25.900174+0000","last_unstale":"2024-09-11T23:18:22.182688+0000","last_undegraded":"2024-09-11T23:18:22.182688+0000","last_fullsized":"2024-09-11T23:18:22.182688+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:51:50.953198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"85'6","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.184452+0000","last_change":"2024-09-11T23:16:25.907569+0000","last_active":"2024-09-11T23:18:22.184452+0000","last_peered":"2024-09-11T23:18:22.184452+0000","last_clean":"2024-09-11T23:18:22.184452+0000","last_became_active":"2024-09-11T23:16:25.907091+0000","last_became_peered":"2024-09-11T23:16:25.907091+0000","last_unstale":"2024-09-11T23:18:22.184452+0000","last_undegraded":"2024-09-11T23:18:22.184452+0000","last_fullsized":"2024-09-11T23:18:22.184452+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:53:45.961167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"85'2","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.183204+0000","last_change":"2024-09-11T23:16:25.903246+0000","last_active":"2024-09-11T23:18:22.183204+0000","last_peered":"2024-09-11T23:18:22.183204+0000","last_clean":"2024-09-11T23:18:22.183204+0000","last_became_active":"2024-09-11T23:16:25.902907+0000","last_became_peered":"2024-09-11T23:16:25.902907+0000","last_unstale":"2024-09-11T23:18:22.183204+0000","last_undegraded":"2024-09-11T23:18:22.183204+0000","last_fullsized":"2024-09-11T23:18:22.183204+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:20:38.868384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"85'15","reported_seq":34,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328979+0000","last_change":"2024-09-11T23:16:25.906298+0000","last_active":"2024-09-11T23:18:22.328979+0000","last_peered":"2024-09-11T23:18:22.328979+0000","last_clean":"2024-09-11T23:18:22.328979+0000","last_became_active":"2024-09-11T23:16:25.905350+0000","last_became_peered":"2024-09-11T23:16:25.905350+0000","last_unstale":"2024-09-11T23:18:22.328979+0000","last_undegraded":"2024-09-11T23:18:22.328979+0000","last_fullsized":"2024-09-11T23:18:22.328979+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:04:22.122514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"85'6","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328869+0000","last_change":"2024-09-11T23:16:25.902114+0000","last_active":"2024-09-11T23:18:22.328869+0000","last_peered":"2024-09-11T23:18:22.328869+0000","last_clean":"2024-09-11T23:18:22.328869+0000","last_became_active":"2024-09-11T23:16:25.901742+0000","last_became_peered":"2024-09-11T23:16:25.901742+0000","last_unstale":"2024-09-11T23:18:22.328869+0000","last_undegraded":"2024-09-11T23:18:22.328869+0000","last_fullsized":"2024-09-11T23:18:22.328869+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:02:10.640801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"85'4","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.329796+0000","last_change":"2024-09-11T23:16:25.898117+0000","last_active":"2024-09-11T23:18:22.329796+0000","last_peered":"2024-09-11T23:18:22.329796+0000","last_clean":"2024-09-11T23:18:22.329796+0000","last_became_active":"2024-09-11T23:16:25.897818+0000","last_became_peered":"2024-09-11T23:16:25.897818+0000","last_unstale":"2024-09-11T23:18:22.329796+0000","last_undegraded":"2024-09-11T23:18:22.329796+0000","last_fullsized":"2024-09-11T23:18:22.329796+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:27:37.175471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"85'7","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.544057+0000","last_change":"2024-09-11T23:16:25.905600+0000","last_active":"2024-09-11T23:18:22.544057+0000","last_peered":"2024-09-11T23:18:22.544057+0000","last_clean":"2024-09-11T23:18:22.544057+0000","last_became_active":"2024-09-11T23:16:25.903924+0000","last_became_peered":"2024-09-11T23:16:25.903924+0000","last_unstale":"2024-09-11T23:18:22.544057+0000","last_undegraded":"2024-09-11T23:18:22.544057+0000","last_fullsized":"2024-09-11T23:18:22.544057+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:20:29.929079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"85'8","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328449+0000","last_change":"2024-09-11T23:16:25.904763+0000","last_active":"2024-09-11T23:18:22.328449+0000","last_peered":"2024-09-11T23:18:22.328449+0000","last_clean":"2024-09-11T23:18:22.328449+0000","last_became_active":"2024-09-11T23:16:25.904406+0000","last_became_peered":"2024-09-11T23:16:25.904406+0000","last_unstale":"2024-09-11T23:18:22.328449+0000","last_undegraded":"2024-09-11T23:18:22.328449+0000","last_fullsized":"2024-09-11T23:18:22.328449+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:27:38.918017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"85'12","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328598+0000","last_change":"2024-09-11T23:16:25.906573+0000","last_active":"2024-09-11T23:18:22.328598+0000","last_peered":"2024-09-11T23:18:22.328598+0000","last_clean":"2024-09-11T23:18:22.328598+0000","last_became_active":"2024-09-11T23:16:25.904771+0000","last_became_peered":"2024-09-11T23:16:25.904771+0000","last_unstale":"2024-09-11T23:18:22.328598+0000","last_undegraded":"2024-09-11T23:18:22.328598+0000","last_fullsized":"2024-09-11T23:18:22.328598+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:04:17.090648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"85'9","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328348+0000","last_change":"2024-09-11T23:16:25.902123+0000","last_active":"2024-09-11T23:18:22.328348+0000","last_peered":"2024-09-11T23:18:22.328348+0000","last_clean":"2024-09-11T23:18:22.328348+0000","last_became_active":"2024-09-11T23:16:25.901756+0000","last_became_peered":"2024-09-11T23:16:25.901756+0000","last_unstale":"2024-09-11T23:18:22.328348+0000","last_undegraded":"2024-09-11T23:18:22.328348+0000","last_fullsized":"2024-09-11T23:18:22.328348+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:17:37.074976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"85'7","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.295802+0000","last_change":"2024-09-11T23:16:25.904217+0000","last_active":"2024-09-11T23:18:22.295802+0000","last_peered":"2024-09-11T23:18:22.295802+0000","last_clean":"2024-09-11T23:18:22.295802+0000","last_became_active":"2024-09-11T23:16:25.903522+0000","last_became_peered":"2024-09-11T23:16:25.903522+0000","last_unstale":"2024-09-11T23:18:22.295802+0000","last_undegraded":"2024-09-11T23:18:22.295802+0000","last_fullsized":"2024-09-11T23:18:22.295802+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:29:46.634475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"86'7","reported_seq":140,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:23.124462+0000","last_change":"2024-09-11T23:16:23.793696+0000","last_active":"2024-09-11T23:18:23.124462+0000","last_peered":"2024-09-11T23:18:23.124462+0000","last_clean":"2024-09-11T23:18:23.124462+0000","last_became_active":"2024-09-11T23:16:23.793062+0000","last_became_peered":"2024-09-11T23:16:23.793062+0000","last_unstale":"2024-09-11T23:18:23.124462+0000","last_undegraded":"2024-09-11T23:18:23.124462+0000","last_fullsized":"2024-09-11T23:18:23.124462+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:00:25.655817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1323,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":109,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"85'8","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.184453+0000","last_change":"2024-09-11T23:16:25.906673+0000","last_active":"2024-09-11T23:18:22.184453+0000","last_peered":"2024-09-11T23:18:22.184453+0000","last_clean":"2024-09-11T23:18:22.184453+0000","last_became_active":"2024-09-11T23:16:25.906239+0000","last_became_peered":"2024-09-11T23:16:25.906239+0000","last_unstale":"2024-09-11T23:18:22.184453+0000","last_undegraded":"2024-09-11T23:18:22.184453+0000","last_fullsized":"2024-09-11T23:18:22.184453+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:15:54.907269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"85'7","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.302905+0000","last_change":"2024-09-11T23:16:23.793302+0000","last_active":"2024-09-11T23:18:22.302905+0000","last_peered":"2024-09-11T23:18:22.302905+0000","last_clean":"2024-09-11T23:18:22.302905+0000","last_became_active":"2024-09-11T23:16:23.792874+0000","last_became_peered":"2024-09-11T23:16:23.792874+0000","last_unstale":"2024-09-11T23:18:22.302905+0000","last_undegraded":"2024-09-11T23:18:22.302905+0000","last_fullsized":"2024-09-11T23:18:22.302905+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:52:59.558223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"85'3","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328845+0000","last_change":"2024-09-11T23:16:23.792999+0000","last_active":"2024-09-11T23:18:22.328845+0000","last_peered":"2024-09-11T23:18:22.328845+0000","last_clean":"2024-09-11T23:18:22.328845+0000","last_became_active":"2024-09-11T23:16:23.792630+0000","last_became_peered":"2024-09-11T23:16:23.792630+0000","last_unstale":"2024-09-11T23:18:22.328845+0000","last_undegraded":"2024-09-11T23:18:22.328845+0000","last_fullsized":"2024-09-11T23:18:22.328845+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:09:28.001060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"85'8","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.292734+0000","last_change":"2024-09-11T23:16:35.943722+0000","last_active":"2024-09-11T23:18:22.292734+0000","last_peered":"2024-09-11T23:18:22.292734+0000","last_clean":"2024-09-11T23:18:22.292734+0000","last_became_active":"2024-09-11T23:16:35.943355+0000","last_became_peered":"2024-09-11T23:16:35.943355+0000","last_unstale":"2024-09-11T23:18:22.292734+0000","last_undegraded":"2024-09-11T23:18:22.292734+0000","last_fullsized":"2024-09-11T23:18:22.292734+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:26:22.769824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,6],"acting":[9,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"85'3","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.184646+0000","last_change":"2024-09-11T23:16:23.797945+0000","last_active":"2024-09-11T23:18:22.184646+0000","last_peered":"2024-09-11T23:18:22.184646+0000","last_clean":"2024-09-11T23:18:22.184646+0000","last_became_active":"2024-09-11T23:16:23.797572+0000","last_became_peered":"2024-09-11T23:16:23.797572+0000","last_unstale":"2024-09-11T23:18:22.184646+0000","last_undegraded":"2024-09-11T23:18:22.184646+0000","last_fullsized":"2024-09-11T23:18:22.184646+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:04:59.017668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"85'11","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.292877+0000","last_change":"2024-09-11T23:16:25.891587+0000","last_active":"2024-09-11T23:18:22.292877+0000","last_peered":"2024-09-11T23:18:22.292877+0000","last_clean":"2024-09-11T23:18:22.292877+0000","last_became_active":"2024-09-11T23:16:25.891190+0000","last_became_peered":"2024-09-11T23:16:25.891190+0000","last_unstale":"2024-09-11T23:18:22.292877+0000","last_undegraded":"2024-09-11T23:18:22.292877+0000","last_fullsized":"2024-09-11T23:18:22.292877+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:57:05.177843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"85'6","reported_seq":240,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:23.123749+0000","last_change":"2024-09-11T23:16:23.790129+0000","last_active":"2024-09-11T23:18:23.123749+0000","last_peered":"2024-09-11T23:18:23.123749+0000","last_clean":"2024-09-11T23:18:23.123749+0000","last_became_active":"2024-09-11T23:16:23.789720+0000","last_became_peered":"2024-09-11T23:16:23.789720+0000","last_unstale":"2024-09-11T23:18:23.123749+0000","last_undegraded":"2024-09-11T23:18:23.123749+0000","last_fullsized":"2024-09-11T23:18:23.123749+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:33:58.388513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":210,"num_read_kb":210,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"85'10","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.329142+0000","last_change":"2024-09-11T23:16:25.901014+0000","last_active":"2024-09-11T23:18:22.329142+0000","last_peered":"2024-09-11T23:18:22.329142+0000","last_clean":"2024-09-11T23:18:22.329142+0000","last_became_active":"2024-09-11T23:16:25.900280+0000","last_became_peered":"2024-09-11T23:16:25.900280+0000","last_unstale":"2024-09-11T23:18:22.329142+0000","last_undegraded":"2024-09-11T23:18:22.329142+0000","last_fullsized":"2024-09-11T23:18:22.329142+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:14:03.662204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"85'4","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328532+0000","last_change":"2024-09-11T23:16:23.797944+0000","last_active":"2024-09-11T23:18:22.328532+0000","last_peered":"2024-09-11T23:18:22.328532+0000","last_clean":"2024-09-11T23:18:22.328532+0000","last_became_active":"2024-09-11T23:16:23.797520+0000","last_became_peered":"2024-09-11T23:16:23.797520+0000","last_unstale":"2024-09-11T23:18:22.328532+0000","last_undegraded":"2024-09-11T23:18:22.328532+0000","last_fullsized":"2024-09-11T23:18:22.328532+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:29:06.957518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"85'9","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.544713+0000","last_change":"2024-09-11T23:16:25.905379+0000","last_active":"2024-09-11T23:18:22.544713+0000","last_peered":"2024-09-11T23:18:22.544713+0000","last_clean":"2024-09-11T23:18:22.544713+0000","last_became_active":"2024-09-11T23:16:25.905085+0000","last_became_peered":"2024-09-11T23:16:25.905085+0000","last_unstale":"2024-09-11T23:18:22.544713+0000","last_undegraded":"2024-09-11T23:18:22.544713+0000","last_fullsized":"2024-09-11T23:18:22.544713+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:22:35.129736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"85'5","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.184874+0000","last_change":"2024-09-11T23:16:24.075556+0000","last_active":"2024-09-11T23:18:22.184874+0000","last_peered":"2024-09-11T23:18:22.184874+0000","last_clean":"2024-09-11T23:18:22.184874+0000","last_became_active":"2024-09-11T23:16:24.074783+0000","last_became_peered":"2024-09-11T23:16:24.074783+0000","last_unstale":"2024-09-11T23:18:22.184874+0000","last_undegraded":"2024-09-11T23:18:22.184874+0000","last_fullsized":"2024-09-11T23:18:22.184874+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:55:09.962342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"85'11","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328166+0000","last_change":"2024-09-11T23:16:25.907720+0000","last_active":"2024-09-11T23:18:22.328166+0000","last_peered":"2024-09-11T23:18:22.328166+0000","last_clean":"2024-09-11T23:18:22.328166+0000","last_became_active":"2024-09-11T23:16:25.907335+0000","last_became_peered":"2024-09-11T23:16:25.907335+0000","last_unstale":"2024-09-11T23:18:22.328166+0000","last_undegraded":"2024-09-11T23:18:22.328166+0000","last_fullsized":"2024-09-11T23:18:22.328166+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:46:14.519384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"86'38","reported_seq":284,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:23.145509+0000","last_change":"2024-09-11T23:16:23.794321+0000","last_active":"2024-09-11T23:18:23.145509+0000","last_peered":"2024-09-11T23:18:23.145509+0000","last_clean":"2024-09-11T23:18:23.145509+0000","last_became_active":"2024-09-11T23:16:23.793955+0000","last_became_peered":"2024-09-11T23:16:23.793955+0000","last_unstale":"2024-09-11T23:18:23.145509+0000","last_undegraded":"2024-09-11T23:18:23.145509+0000","last_fullsized":"2024-09-11T23:18:23.145509+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:47:46.442656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":66480,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":222,"num_read_kb":175,"num_write":40,"num_write_kb":82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"85'6","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.296053+0000","last_change":"2024-09-11T23:16:25.896598+0000","last_active":"2024-09-11T23:18:22.296053+0000","last_peered":"2024-09-11T23:18:22.296053+0000","last_clean":"2024-09-11T23:18:22.296053+0000","last_became_active":"2024-09-11T23:16:25.896293+0000","last_became_peered":"2024-09-11T23:16:25.896293+0000","last_unstale":"2024-09-11T23:18:22.296053+0000","last_undegraded":"2024-09-11T23:18:22.296053+0000","last_fullsized":"2024-09-11T23:18:22.296053+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:04:11.489644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"85'9","reported_seq":35,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328677+0000","last_change":"2024-09-11T23:16:23.791486+0000","last_active":"2024-09-11T23:18:22.328677+0000","last_peered":"2024-09-11T23:18:22.328677+0000","last_clean":"2024-09-11T23:18:22.328677+0000","last_became_active":"2024-09-11T23:16:23.790786+0000","last_became_peered":"2024-09-11T23:16:23.790786+0000","last_unstale":"2024-09-11T23:18:22.328677+0000","last_undegraded":"2024-09-11T23:18:22.328677+0000","last_fullsized":"2024-09-11T23:18:22.328677+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:03:52.742560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"85'9","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.329171+0000","last_change":"2024-09-11T23:16:25.897382+0000","last_active":"2024-09-11T23:18:22.329171+0000","last_peered":"2024-09-11T23:18:22.329171+0000","last_clean":"2024-09-11T23:18:22.329171+0000","last_became_active":"2024-09-11T23:16:25.897073+0000","last_became_peered":"2024-09-11T23:16:25.897073+0000","last_unstale":"2024-09-11T23:18:22.329171+0000","last_undegraded":"2024-09-11T23:18:22.329171+0000","last_fullsized":"2024-09-11T23:18:22.329171+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:15:59.780100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"85'5","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.184727+0000","last_change":"2024-09-11T23:16:23.793886+0000","last_active":"2024-09-11T23:18:22.184727+0000","last_peered":"2024-09-11T23:18:22.184727+0000","last_clean":"2024-09-11T23:18:22.184727+0000","last_became_active":"2024-09-11T23:16:23.793066+0000","last_became_peered":"2024-09-11T23:16:23.793066+0000","last_unstale":"2024-09-11T23:18:22.184727+0000","last_undegraded":"2024-09-11T23:18:22.184727+0000","last_fullsized":"2024-09-11T23:18:22.184727+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:07:03.882546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"85'15","reported_seq":34,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328552+0000","last_change":"2024-09-11T23:16:25.904575+0000","last_active":"2024-09-11T23:18:22.328552+0000","last_peered":"2024-09-11T23:18:22.328552+0000","last_clean":"2024-09-11T23:18:22.328552+0000","last_became_active":"2024-09-11T23:16:25.904224+0000","last_became_peered":"2024-09-11T23:16:25.904224+0000","last_unstale":"2024-09-11T23:18:22.328552+0000","last_undegraded":"2024-09-11T23:18:22.328552+0000","last_fullsized":"2024-09-11T23:18:22.328552+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:42:13.036931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"85'5","reported_seq":136,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.187045+0000","last_change":"2024-09-11T23:16:23.796073+0000","last_active":"2024-09-11T23:18:22.187045+0000","last_peered":"2024-09-11T23:18:22.187045+0000","last_clean":"2024-09-11T23:18:22.187045+0000","last_became_active":"2024-09-11T23:16:23.794850+0000","last_became_peered":"2024-09-11T23:16:23.794850+0000","last_unstale":"2024-09-11T23:18:22.187045+0000","last_undegraded":"2024-09-11T23:18:22.187045+0000","last_fullsized":"2024-09-11T23:18:22.187045+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:57:22.361662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":108,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"85'6","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328221+0000","last_change":"2024-09-11T23:16:25.908700+0000","last_active":"2024-09-11T23:18:22.328221+0000","last_peered":"2024-09-11T23:18:22.328221+0000","last_clean":"2024-09-11T23:18:22.328221+0000","last_became_active":"2024-09-11T23:16:25.908373+0000","last_became_peered":"2024-09-11T23:16:25.908373+0000","last_unstale":"2024-09-11T23:18:22.328221+0000","last_undegraded":"2024-09-11T23:18:22.328221+0000","last_fullsized":"2024-09-11T23:18:22.328221+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:20:18.536079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"85'17","reported_seq":42,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.295792+0000","last_change":"2024-09-11T23:16:24.075927+0000","last_active":"2024-09-11T23:18:22.295792+0000","last_peered":"2024-09-11T23:18:22.295792+0000","last_clean":"2024-09-11T23:18:22.295792+0000","last_became_active":"2024-09-11T23:16:24.075527+0000","last_became_peered":"2024-09-11T23:16:24.075527+0000","last_unstale":"2024-09-11T23:18:22.295792+0000","last_undegraded":"2024-09-11T23:18:22.295792+0000","last_fullsized":"2024-09-11T23:18:22.295792+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:27:54.622059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"85'6","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.184380+0000","last_change":"2024-09-11T23:16:35.947475+0000","last_active":"2024-09-11T23:18:22.184380+0000","last_peered":"2024-09-11T23:18:22.184380+0000","last_clean":"2024-09-11T23:18:22.184380+0000","last_became_active":"2024-09-11T23:16:35.947028+0000","last_became_peered":"2024-09-11T23:16:35.947028+0000","last_unstale":"2024-09-11T23:18:22.184380+0000","last_undegraded":"2024-09-11T23:18:22.184380+0000","last_fullsized":"2024-09-11T23:18:22.184380+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:18:13.479554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'2","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.296198+0000","last_change":"2024-09-11T23:16:23.793298+0000","last_active":"2024-09-11T23:18:22.296198+0000","last_peered":"2024-09-11T23:18:22.296198+0000","last_clean":"2024-09-11T23:18:22.296198+0000","last_became_active":"2024-09-11T23:16:23.792947+0000","last_became_peered":"2024-09-11T23:16:23.792947+0000","last_unstale":"2024-09-11T23:18:22.296198+0000","last_undegraded":"2024-09-11T23:18:22.296198+0000","last_fullsized":"2024-09-11T23:18:22.296198+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:32:21.294560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"85'5","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328871+0000","last_change":"2024-09-11T23:16:25.902348+0000","last_active":"2024-09-11T23:18:22.328871+0000","last_peered":"2024-09-11T23:18:22.328871+0000","last_clean":"2024-09-11T23:18:22.328871+0000","last_became_active":"2024-09-11T23:16:25.901686+0000","last_became_peered":"2024-09-11T23:16:25.901686+0000","last_unstale":"2024-09-11T23:18:22.328871+0000","last_undegraded":"2024-09-11T23:18:22.328871+0000","last_fullsized":"2024-09-11T23:18:22.328871+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:00:15.543003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"85'5","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.330216+0000","last_change":"2024-09-11T23:16:23.790984+0000","last_active":"2024-09-11T23:18:22.330216+0000","last_peered":"2024-09-11T23:18:22.330216+0000","last_clean":"2024-09-11T23:18:22.330216+0000","last_became_active":"2024-09-11T23:16:23.790569+0000","last_became_peered":"2024-09-11T23:16:23.790569+0000","last_unstale":"2024-09-11T23:18:22.330216+0000","last_undegraded":"2024-09-11T23:18:22.330216+0000","last_fullsized":"2024-09-11T23:18:22.330216+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:57:39.675793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"85'9","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328796+0000","last_change":"2024-09-11T23:16:25.899249+0000","last_active":"2024-09-11T23:18:22.328796+0000","last_peered":"2024-09-11T23:18:22.328796+0000","last_clean":"2024-09-11T23:18:22.328796+0000","last_became_active":"2024-09-11T23:16:25.898914+0000","last_became_peered":"2024-09-11T23:16:25.898914+0000","last_unstale":"2024-09-11T23:18:22.328796+0000","last_undegraded":"2024-09-11T23:18:22.328796+0000","last_fullsized":"2024-09-11T23:18:22.328796+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:17:07.545770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"86'41","reported_seq":73,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.429528+0000","last_change":"2024-09-11T23:16:23.787387+0000","last_active":"2024-09-11T23:18:22.429528+0000","last_peered":"2024-09-11T23:18:22.429528+0000","last_clean":"2024-09-11T23:18:22.429528+0000","last_became_active":"2024-09-11T23:16:23.787014+0000","last_became_peered":"2024-09-11T23:16:23.787014+0000","last_unstale":"2024-09-11T23:18:22.429528+0000","last_undegraded":"2024-09-11T23:18:22.429528+0000","last_fullsized":"2024-09-11T23:18:22.429528+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:45:52.004253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"85'7","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328671+0000","last_change":"2024-09-11T23:16:25.908304+0000","last_active":"2024-09-11T23:18:22.328671+0000","last_peered":"2024-09-11T23:18:22.328671+0000","last_clean":"2024-09-11T23:18:22.328671+0000","last_became_active":"2024-09-11T23:16:25.907900+0000","last_became_peered":"2024-09-11T23:16:25.907900+0000","last_unstale":"2024-09-11T23:18:22.328671+0000","last_undegraded":"2024-09-11T23:18:22.328671+0000","last_fullsized":"2024-09-11T23:18:22.328671+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:43:26.247682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"85'7","reported_seq":140,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328822+0000","last_change":"2024-09-11T23:16:24.074610+0000","last_active":"2024-09-11T23:18:22.328822+0000","last_peered":"2024-09-11T23:18:22.328822+0000","last_clean":"2024-09-11T23:18:22.328822+0000","last_became_active":"2024-09-11T23:16:24.074210+0000","last_became_peered":"2024-09-11T23:16:24.074210+0000","last_unstale":"2024-09-11T23:18:22.328822+0000","last_undegraded":"2024-09-11T23:18:22.328822+0000","last_fullsized":"2024-09-11T23:18:22.328822+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:09:37.167050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1704,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":109,"num_read_kb":4,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"85'7","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328842+0000","last_change":"2024-09-11T23:16:25.903120+0000","last_active":"2024-09-11T23:18:22.328842+0000","last_peered":"2024-09-11T23:18:22.328842+0000","last_clean":"2024-09-11T23:18:22.328842+0000","last_became_active":"2024-09-11T23:16:25.902770+0000","last_became_peered":"2024-09-11T23:16:25.902770+0000","last_unstale":"2024-09-11T23:18:22.328842+0000","last_undegraded":"2024-09-11T23:18:22.328842+0000","last_fullsized":"2024-09-11T23:18:22.328842+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:03:30.041088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":349,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328487+0000","last_change":"2024-09-11T23:15:13.115055+0000","last_active":"2024-09-11T23:18:22.328487+0000","last_peered":"2024-09-11T23:18:22.328487+0000","last_clean":"2024-09-11T23:18:22.328487+0000","last_became_active":"2024-09-11T23:15:13.114480+0000","last_became_peered":"2024-09-11T23:15:13.114480+0000","last_unstale":"2024-09-11T23:18:22.328487+0000","last_undegraded":"2024-09-11T23:18:22.328487+0000","last_fullsized":"2024-09-11T23:18:22.328487+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:11:50.495799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:11:50.495799+0000","last_clean_scrub_stamp":"2024-09-11T23:11:50.495799+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:15:53.681230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"85'6","reported_seq":33,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.187094+0000","last_change":"2024-09-11T23:16:23.793996+0000","last_active":"2024-09-11T23:18:22.187094+0000","last_peered":"2024-09-11T23:18:22.187094+0000","last_clean":"2024-09-11T23:18:22.187094+0000","last_became_active":"2024-09-11T23:16:23.793681+0000","last_became_peered":"2024-09-11T23:16:23.793681+0000","last_unstale":"2024-09-11T23:18:22.187094+0000","last_undegraded":"2024-09-11T23:18:22.187094+0000","last_fullsized":"2024-09-11T23:18:22.187094+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:52:50.463623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"85'8","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.182924+0000","last_change":"2024-09-11T23:16:25.905133+0000","last_active":"2024-09-11T23:18:22.182924+0000","last_peered":"2024-09-11T23:18:22.182924+0000","last_clean":"2024-09-11T23:18:22.182924+0000","last_became_active":"2024-09-11T23:16:25.904819+0000","last_became_peered":"2024-09-11T23:16:25.904819+0000","last_unstale":"2024-09-11T23:18:22.182924+0000","last_undegraded":"2024-09-11T23:18:22.182924+0000","last_fullsized":"2024-09-11T23:18:22.182924+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:51:09.664343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"85'4","reported_seq":34,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328367+0000","last_change":"2024-09-11T23:16:36.228806+0000","last_active":"2024-09-11T23:18:22.328367+0000","last_peered":"2024-09-11T23:18:22.328367+0000","last_clean":"2024-09-11T23:18:22.328367+0000","last_became_active":"2024-09-11T23:16:36.228311+0000","last_became_peered":"2024-09-11T23:16:36.228311+0000","last_unstale":"2024-09-11T23:18:22.328367+0000","last_undegraded":"2024-09-11T23:18:22.328367+0000","last_fullsized":"2024-09-11T23:18:22.328367+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:42:32.856691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4],"acting":[7,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"85'5","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328703+0000","last_change":"2024-09-11T23:16:25.898935+0000","last_active":"2024-09-11T23:18:22.328703+0000","last_peered":"2024-09-11T23:18:22.328703+0000","last_clean":"2024-09-11T23:18:22.328703+0000","last_became_active":"2024-09-11T23:16:25.898618+0000","last_became_peered":"2024-09-11T23:16:25.898618+0000","last_unstale":"2024-09-11T23:18:22.328703+0000","last_undegraded":"2024-09-11T23:18:22.328703+0000","last_fullsized":"2024-09-11T23:18:22.328703+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:13:29.104911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"85'5","reported_seq":135,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.329013+0000","last_change":"2024-09-11T23:16:24.073613+0000","last_active":"2024-09-11T23:18:22.329013+0000","last_peered":"2024-09-11T23:18:22.329013+0000","last_clean":"2024-09-11T23:18:22.329013+0000","last_became_active":"2024-09-11T23:16:24.073262+0000","last_became_peered":"2024-09-11T23:16:24.073262+0000","last_unstale":"2024-09-11T23:18:22.329013+0000","last_undegraded":"2024-09-11T23:18:22.329013+0000","last_fullsized":"2024-09-11T23:18:22.329013+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:31:58.653657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":106,"num_read_kb":106,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"85'8","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.184479+0000","last_change":"2024-09-11T23:16:25.903725+0000","last_active":"2024-09-11T23:18:22.184479+0000","last_peered":"2024-09-11T23:18:22.184479+0000","last_clean":"2024-09-11T23:18:22.184479+0000","last_became_active":"2024-09-11T23:16:25.903375+0000","last_became_peered":"2024-09-11T23:16:25.903375+0000","last_unstale":"2024-09-11T23:18:22.184479+0000","last_undegraded":"2024-09-11T23:18:22.184479+0000","last_fullsized":"2024-09-11T23:18:22.184479+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:55:02.779575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"86'6","reported_seq":137,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:23.123405+0000","last_change":"2024-09-11T23:16:23.793932+0000","last_active":"2024-09-11T23:18:23.123405+0000","last_peered":"2024-09-11T23:18:23.123405+0000","last_clean":"2024-09-11T23:18:23.123405+0000","last_became_active":"2024-09-11T23:16:23.793111+0000","last_became_peered":"2024-09-11T23:16:23.793111+0000","last_unstale":"2024-09-11T23:18:23.123405+0000","last_undegraded":"2024-09-11T23:18:23.123405+0000","last_fullsized":"2024-09-11T23:18:23.123405+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:14:09.087222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":107,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"85'7","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.302982+0000","last_change":"2024-09-11T23:16:25.905020+0000","last_active":"2024-09-11T23:18:22.302982+0000","last_peered":"2024-09-11T23:18:22.302982+0000","last_clean":"2024-09-11T23:18:22.302982+0000","last_became_active":"2024-09-11T23:16:25.904675+0000","last_became_peered":"2024-09-11T23:16:25.904675+0000","last_unstale":"2024-09-11T23:18:22.302982+0000","last_undegraded":"2024-09-11T23:18:22.302982+0000","last_fullsized":"2024-09-11T23:18:22.302982+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:13:57.388773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"85'1","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.293261+0000","last_change":"2024-09-11T23:16:24.073630+0000","last_active":"2024-09-11T23:18:22.293261+0000","last_peered":"2024-09-11T23:18:22.293261+0000","last_clean":"2024-09-11T23:18:22.293261+0000","last_became_active":"2024-09-11T23:16:24.073301+0000","last_became_peered":"2024-09-11T23:16:24.073301+0000","last_unstale":"2024-09-11T23:18:22.293261+0000","last_undegraded":"2024-09-11T23:18:22.293261+0000","last_fullsized":"2024-09-11T23:18:22.293261+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:41:59.374581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"85'5","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.303051+0000","last_change":"2024-09-11T23:16:25.894124+0000","last_active":"2024-09-11T23:18:22.303051+0000","last_peered":"2024-09-11T23:18:22.303051+0000","last_clean":"2024-09-11T23:18:22.303051+0000","last_became_active":"2024-09-11T23:16:25.893751+0000","last_became_peered":"2024-09-11T23:16:25.893751+0000","last_unstale":"2024-09-11T23:18:22.303051+0000","last_undegraded":"2024-09-11T23:18:22.303051+0000","last_fullsized":"2024-09-11T23:18:22.303051+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:00:28.265206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"85'4","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.296214+0000","last_change":"2024-09-11T23:16:24.073700+0000","last_active":"2024-09-11T23:18:22.296214+0000","last_peered":"2024-09-11T23:18:22.296214+0000","last_clean":"2024-09-11T23:18:22.296214+0000","last_became_active":"2024-09-11T23:16:24.073283+0000","last_became_peered":"2024-09-11T23:16:24.073283+0000","last_unstale":"2024-09-11T23:18:22.296214+0000","last_undegraded":"2024-09-11T23:18:22.296214+0000","last_fullsized":"2024-09-11T23:18:22.296214+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:41:43.536834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"85'9","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.329920+0000","last_change":"2024-09-11T23:16:25.900532+0000","last_active":"2024-09-11T23:18:22.329920+0000","last_peered":"2024-09-11T23:18:22.329920+0000","last_clean":"2024-09-11T23:18:22.329920+0000","last_became_active":"2024-09-11T23:16:25.899466+0000","last_became_peered":"2024-09-11T23:16:25.899466+0000","last_unstale":"2024-09-11T23:18:22.329920+0000","last_undegraded":"2024-09-11T23:18:22.329920+0000","last_fullsized":"2024-09-11T23:18:22.329920+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:10:01.152323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"86'50","reported_seq":97,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.432609+0000","last_change":"2024-09-11T23:16:23.788789+0000","last_active":"2024-09-11T23:18:22.432609+0000","last_peered":"2024-09-11T23:18:22.432609+0000","last_clean":"2024-09-11T23:18:22.432609+0000","last_became_active":"2024-09-11T23:16:23.788422+0000","last_became_peered":"2024-09-11T23:16:23.788422+0000","last_unstale":"2024-09-11T23:18:22.432609+0000","last_undegraded":"2024-09-11T23:18:22.432609+0000","last_fullsized":"2024-09-11T23:18:22.432609+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":0,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:09:14.310205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":89,"num_write_kb":837,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"85'5","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.183270+0000","last_change":"2024-09-11T23:16:25.895511+0000","last_active":"2024-09-11T23:18:22.183270+0000","last_peered":"2024-09-11T23:18:22.183270+0000","last_clean":"2024-09-11T23:18:22.183270+0000","last_became_active":"2024-09-11T23:16:25.895143+0000","last_became_peered":"2024-09-11T23:16:25.895143+0000","last_unstale":"2024-09-11T23:18:22.183270+0000","last_undegraded":"2024-09-11T23:18:22.183270+0000","last_fullsized":"2024-09-11T23:18:22.183270+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:31:16.498371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"85'7","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.187209+0000","last_change":"2024-09-11T23:16:24.019064+0000","last_active":"2024-09-11T23:18:22.187209+0000","last_peered":"2024-09-11T23:18:22.187209+0000","last_clean":"2024-09-11T23:18:22.187209+0000","last_became_active":"2024-09-11T23:16:24.018658+0000","last_became_peered":"2024-09-11T23:16:24.018658+0000","last_unstale":"2024-09-11T23:18:22.187209+0000","last_undegraded":"2024-09-11T23:18:22.187209+0000","last_fullsized":"2024-09-11T23:18:22.187209+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:17:41.009507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"85'7","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.544339+0000","last_change":"2024-09-11T23:16:25.903553+0000","last_active":"2024-09-11T23:18:22.544339+0000","last_peered":"2024-09-11T23:18:22.544339+0000","last_clean":"2024-09-11T23:18:22.544339+0000","last_became_active":"2024-09-11T23:16:25.903221+0000","last_became_peered":"2024-09-11T23:16:25.903221+0000","last_unstale":"2024-09-11T23:18:22.544339+0000","last_undegraded":"2024-09-11T23:18:22.544339+0000","last_fullsized":"2024-09-11T23:18:22.544339+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:16:51.872517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"83'2","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328140+0000","last_change":"2024-09-11T23:16:24.073784+0000","last_active":"2024-09-11T23:18:22.328140+0000","last_peered":"2024-09-11T23:18:22.328140+0000","last_clean":"2024-09-11T23:18:22.328140+0000","last_became_active":"2024-09-11T23:16:24.073406+0000","last_became_peered":"2024-09-11T23:16:24.073406+0000","last_unstale":"2024-09-11T23:18:22.328140+0000","last_undegraded":"2024-09-11T23:18:22.328140+0000","last_fullsized":"2024-09-11T23:18:22.328140+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:31:24.617025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"85'2","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.295796+0000","last_change":"2024-09-11T23:16:25.894878+0000","last_active":"2024-09-11T23:18:22.295796+0000","last_peered":"2024-09-11T23:18:22.295796+0000","last_clean":"2024-09-11T23:18:22.295796+0000","last_became_active":"2024-09-11T23:16:25.894543+0000","last_became_peered":"2024-09-11T23:16:25.894543+0000","last_unstale":"2024-09-11T23:18:22.295796+0000","last_undegraded":"2024-09-11T23:18:22.295796+0000","last_fullsized":"2024-09-11T23:18:22.295796+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:34:13.435754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"85'7","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.182832+0000","last_change":"2024-09-11T23:16:23.786502+0000","last_active":"2024-09-11T23:18:22.182832+0000","last_peered":"2024-09-11T23:18:22.182832+0000","last_clean":"2024-09-11T23:18:22.182832+0000","last_became_active":"2024-09-11T23:16:23.786182+0000","last_became_peered":"2024-09-11T23:16:23.786182+0000","last_unstale":"2024-09-11T23:18:22.182832+0000","last_undegraded":"2024-09-11T23:18:22.182832+0000","last_fullsized":"2024-09-11T23:18:22.182832+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:59:07.978244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"85'9","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328520+0000","last_change":"2024-09-11T23:16:25.899625+0000","last_active":"2024-09-11T23:18:22.328520+0000","last_peered":"2024-09-11T23:18:22.328520+0000","last_clean":"2024-09-11T23:18:22.328520+0000","last_became_active":"2024-09-11T23:16:25.899265+0000","last_became_peered":"2024-09-11T23:16:25.899265+0000","last_unstale":"2024-09-11T23:18:22.328520+0000","last_undegraded":"2024-09-11T23:18:22.328520+0000","last_fullsized":"2024-09-11T23:18:22.328520+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:36:06.907622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"86'4","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328472+0000","last_change":"2024-09-11T23:16:36.228991+0000","last_active":"2024-09-11T23:18:22.328472+0000","last_peered":"2024-09-11T23:18:22.328472+0000","last_clean":"2024-09-11T23:18:22.328472+0000","last_became_active":"2024-09-11T23:16:36.227701+0000","last_became_peered":"2024-09-11T23:16:36.227701+0000","last_unstale":"2024-09-11T23:18:22.328472+0000","last_undegraded":"2024-09-11T23:18:22.328472+0000","last_fullsized":"2024-09-11T23:18:22.328472+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:37:57.478572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"85'8","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.182913+0000","last_change":"2024-09-11T23:16:25.894803+0000","last_active":"2024-09-11T23:18:22.182913+0000","last_peered":"2024-09-11T23:18:22.182913+0000","last_clean":"2024-09-11T23:18:22.182913+0000","last_became_active":"2024-09-11T23:16:25.894429+0000","last_became_peered":"2024-09-11T23:16:25.894429+0000","last_unstale":"2024-09-11T23:18:22.182913+0000","last_undegraded":"2024-09-11T23:18:22.182913+0000","last_fullsized":"2024-09-11T23:18:22.182913+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:39:15.868910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"83'4","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.295871+0000","last_change":"2024-09-11T23:16:23.794271+0000","last_active":"2024-09-11T23:18:22.295871+0000","last_peered":"2024-09-11T23:18:22.295871+0000","last_clean":"2024-09-11T23:18:22.295871+0000","last_became_active":"2024-09-11T23:16:23.793919+0000","last_became_peered":"2024-09-11T23:16:23.793919+0000","last_unstale":"2024-09-11T23:18:22.295871+0000","last_undegraded":"2024-09-11T23:18:22.295871+0000","last_fullsized":"2024-09-11T23:18:22.295871+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:30:28.907312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"85'5","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328395+0000","last_change":"2024-09-11T23:16:25.900969+0000","last_active":"2024-09-11T23:18:22.328395+0000","last_peered":"2024-09-11T23:18:22.328395+0000","last_clean":"2024-09-11T23:18:22.328395+0000","last_became_active":"2024-09-11T23:16:25.900425+0000","last_became_peered":"2024-09-11T23:16:25.900425+0000","last_unstale":"2024-09-11T23:18:22.328395+0000","last_undegraded":"2024-09-11T23:18:22.328395+0000","last_fullsized":"2024-09-11T23:18:22.328395+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:03:25.185773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"85'15","reported_seq":34,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.184711+0000","last_change":"2024-09-11T23:16:25.911605+0000","last_active":"2024-09-11T23:18:22.184711+0000","last_peered":"2024-09-11T23:18:22.184711+0000","last_clean":"2024-09-11T23:18:22.184711+0000","last_became_active":"2024-09-11T23:16:25.911180+0000","last_became_peered":"2024-09-11T23:16:25.911180+0000","last_unstale":"2024-09-11T23:18:22.184711+0000","last_undegraded":"2024-09-11T23:18:22.184711+0000","last_fullsized":"2024-09-11T23:18:22.184711+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:55:38.062606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"85'5","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328425+0000","last_change":"2024-09-11T23:16:35.949528+0000","last_active":"2024-09-11T23:18:22.328425+0000","last_peered":"2024-09-11T23:18:22.328425+0000","last_clean":"2024-09-11T23:18:22.328425+0000","last_became_active":"2024-09-11T23:16:35.945679+0000","last_became_peered":"2024-09-11T23:16:35.945679+0000","last_unstale":"2024-09-11T23:18:22.328425+0000","last_undegraded":"2024-09-11T23:18:22.328425+0000","last_fullsized":"2024-09-11T23:18:22.328425+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:14:24.706044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"85'9","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328158+0000","last_change":"2024-09-11T23:16:25.900526+0000","last_active":"2024-09-11T23:18:22.328158+0000","last_peered":"2024-09-11T23:18:22.328158+0000","last_clean":"2024-09-11T23:18:22.328158+0000","last_became_active":"2024-09-11T23:16:25.900148+0000","last_became_peered":"2024-09-11T23:16:25.900148+0000","last_unstale":"2024-09-11T23:18:22.328158+0000","last_undegraded":"2024-09-11T23:18:22.328158+0000","last_fullsized":"2024-09-11T23:18:22.328158+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:26:10.582473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"86'13","reported_seq":48,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.296144+0000","last_change":"2024-09-11T23:16:36.233273+0000","last_active":"2024-09-11T23:18:22.296144+0000","last_peered":"2024-09-11T23:18:22.296144+0000","last_clean":"2024-09-11T23:18:22.296144+0000","last_became_active":"2024-09-11T23:16:36.228358+0000","last_became_peered":"2024-09-11T23:16:36.228358+0000","last_unstale":"2024-09-11T23:18:22.296144+0000","last_undegraded":"2024-09-11T23:18:22.296144+0000","last_fullsized":"2024-09-11T23:18:22.296144+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:06:31.225795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"85'6","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.544571+0000","last_change":"2024-09-11T23:16:25.907874+0000","last_active":"2024-09-11T23:18:22.544571+0000","last_peered":"2024-09-11T23:18:22.544571+0000","last_clean":"2024-09-11T23:18:22.544571+0000","last_became_active":"2024-09-11T23:16:25.907491+0000","last_became_peered":"2024-09-11T23:16:25.907491+0000","last_unstale":"2024-09-11T23:18:22.544571+0000","last_undegraded":"2024-09-11T23:18:22.544571+0000","last_fullsized":"2024-09-11T23:18:22.544571+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:09:20.984249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"84'2","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.183287+0000","last_change":"2024-09-11T23:16:24.074355+0000","last_active":"2024-09-11T23:18:22.183287+0000","last_peered":"2024-09-11T23:18:22.183287+0000","last_clean":"2024-09-11T23:18:22.183287+0000","last_became_active":"2024-09-11T23:16:24.074047+0000","last_became_peered":"2024-09-11T23:16:24.074047+0000","last_unstale":"2024-09-11T23:18:22.183287+0000","last_undegraded":"2024-09-11T23:18:22.183287+0000","last_fullsized":"2024-09-11T23:18:22.183287+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:31:58.804420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"85'5","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.182923+0000","last_change":"2024-09-11T23:16:25.907701+0000","last_active":"2024-09-11T23:18:22.182923+0000","last_peered":"2024-09-11T23:18:22.182923+0000","last_clean":"2024-09-11T23:18:22.182923+0000","last_became_active":"2024-09-11T23:16:25.907379+0000","last_became_peered":"2024-09-11T23:16:25.907379+0000","last_unstale":"2024-09-11T23:18:22.182923+0000","last_undegraded":"2024-09-11T23:18:22.182923+0000","last_fullsized":"2024-09-11T23:18:22.182923+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:12:03.639055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"86'14","reported_seq":144,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.497378+0000","last_change":"2024-09-11T23:16:23.794293+0000","last_active":"2024-09-11T23:18:22.497378+0000","last_peered":"2024-09-11T23:18:22.497378+0000","last_clean":"2024-09-11T23:18:22.497378+0000","last_became_active":"2024-09-11T23:16:23.793974+0000","last_became_peered":"2024-09-11T23:16:23.793974+0000","last_unstale":"2024-09-11T23:18:22.497378+0000","last_undegraded":"2024-09-11T23:18:22.497378+0000","last_fullsized":"2024-09-11T23:18:22.497378+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:24:39.170188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":106,"num_read_kb":106,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":599346,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1189,"num_read_kb":709,"num_write":372,"num_write_kb":1462,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2174976,"data_stored":1798038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":298,"ondisk_log_size":298,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":11,"up_from":77,"seq":330712481829,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32128,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713024,"statfs":{"total":95995035648,"available":95962136576,"internally_reserved":0,"allocated":962560,"data_stored":459042,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":70,"seq":300647710761,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28844,"kb_used_data":1760,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716308,"statfs":{"total":95995035648,"available":95965499392,"internally_reserved":0,"allocated":1802240,"data_stored":1247035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":65,"seq":279172874287,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28372,"kb_used_data":1344,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716780,"statfs":{"total":95995035648,"available":95965982720,"internally_reserved":0,"allocated":1376256,"data_stored":851072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":57,"seq":244813135924,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":958464,"data_stored":448696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":50,"seq":214748364856,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28568,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716584,"statfs":{"total":95995035648,"available":95965782016,"internally_reserved":0,"allocated":1576960,"data_stored":1051455,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528380,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36292,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708860,"statfs":{"total":95995035648,"available":95957872640,"internally_reserved":0,"allocated":966656,"data_stored":457966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790019,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28768,"kb_used_data":1684,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716384,"statfs":{"total":95995035648,"available":95965577216,"internally_reserved":0,"allocated":1724416,"data_stored":1182340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920840,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36292,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708860,"statfs":{"total":95995035648,"available":95957872640,"internally_reserved":0,"allocated":966656,"data_stored":457434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182477,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":41160,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93703992,"statfs":{"total":95995035648,"available":95952887808,"internally_reserved":0,"allocated":1511424,"data_stored":982214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411410,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36380,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708772,"statfs":{"total":95995035648,"available":95957782528,"internally_reserved":0,"allocated":1056768,"data_stored":526421,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574935,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1036288,"data_stored":525359,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738460,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":970752,"data_stored":450490,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":68625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":69687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":557056,"data_stored":525246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":180224,"data_stored":135121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":421888,"data_stored":394134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":394338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":258048,"data_stored":199816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2308,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T23:18:25.974 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:18:26.321 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:18:26.603 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:26 smithi026 ceph-mon[31172]: from='client.15261 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T23:18:26.603 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:26 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:18:26.603 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:26 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:18:26.603 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:26 smithi026 ceph-mon[31172]: from='client.15267 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T23:18:26.603 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:26 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:18:26.604 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:26 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:18:26.604 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:26 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:18:26.604 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:26 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:18:26.604 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:26 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:18:26.604 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:26 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:18:26.604 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:26 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:18:26.604 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:26 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:18:26.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:26 smithi136 ceph-mon[35121]: from='client.15261 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T23:18:26.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:26 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:18:26.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:26 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:18:26.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:26 smithi136 ceph-mon[35121]: from='client.15267 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T23:18:26.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:26 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:18:26.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:26 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:18:26.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:26 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:18:26.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:26 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:18:26.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:26 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:18:26.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:26 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:18:26.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:26 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:18:26.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:26 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:18:26.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:26 smithi076 ceph-mon[35296]: from='client.15261 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T23:18:26.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:26 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:18:26.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:26 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:18:26.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:26 smithi076 ceph-mon[35296]: from='client.15267 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T23:18:26.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:26 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:18:26.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:26 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:18:26.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:26 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:18:26.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:26 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:18:26.804 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:26 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:18:26.804 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:26 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:18:26.804 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:26 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:18:26.804 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:26 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:18:26.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:26.857+0000 7fe2350c3640 1 -- 172.21.15.26:0/1794022639 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe230103ff0 msgr2=0x7fe2301043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:26.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:26.857+0000 7fe2350c3640 1 --2- 172.21.15.26:0/1794022639 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe230103ff0 0x7fe2301043d0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fe220009900 tx=0x7fe22002a410 comp rx=0 tx=0).stop 2024-09-11T23:18:26.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:26.857+0000 7fe2350c3640 1 -- 172.21.15.26:0/1794022639 shutdown_connections 2024-09-11T23:18:26.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:26.857+0000 7fe2350c3640 1 --2- 172.21.15.26:0/1794022639 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe2301049a0 0x7fe2301089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:26.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:26.857+0000 7fe2350c3640 1 --2- 172.21.15.26:0/1794022639 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe230109120 0x7fe230110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:26.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:26.857+0000 7fe2350c3640 1 --2- 172.21.15.26:0/1794022639 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe230103ff0 0x7fe2301043d0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:26.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:26.857+0000 7fe2350c3640 1 -- 172.21.15.26:0/1794022639 >> 172.21.15.26:0/1794022639 conn(0x7fe2300ffa90 msgr2=0x7fe230101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:18:26.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:26.857+0000 7fe2350c3640 1 -- 172.21.15.26:0/1794022639 shutdown_connections 2024-09-11T23:18:26.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:26.857+0000 7fe2350c3640 1 -- 172.21.15.26:0/1794022639 wait complete. 2024-09-11T23:18:26.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:26.858+0000 7fe2350c3640 1 Processor -- start 2024-09-11T23:18:26.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:26.858+0000 7fe2350c3640 1 -- start start 2024-09-11T23:18:26.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:26.858+0000 7fe2350c3640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe230103ff0 0x7fe23019a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:26.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:26.858+0000 7fe2350c3640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe2301049a0 0x7fe23019ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:26.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:26.858+0000 7fe2350c3640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe230109120 0x7fe23019eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:26.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:26.858+0000 7fe2350c3640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fe230114150 con 0x7fe230109120 2024-09-11T23:18:26.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:26.858+0000 7fe2350c3640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fe230113fd0 con 0x7fe230103ff0 2024-09-11T23:18:26.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:26.858+0000 7fe2350c3640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe2301142d0 con 0x7fe2301049a0 2024-09-11T23:18:26.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:26.858+0000 7fe22ed9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe230103ff0 0x7fe23019a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:26.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:26.858+0000 7fe22f59e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe230109120 0x7fe23019eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:26.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:26.858+0000 7fe22ed9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe230103ff0 0x7fe23019a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:60262/0 (socket says 172.21.15.26:60262) 2024-09-11T23:18:26.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:26.858+0000 7fe22ed9d640 1 -- 172.21.15.26:0/3507156155 learned_addr learned my addr 172.21.15.26:0/3507156155 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:18:26.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:26.858+0000 7fe22e59c640 1 --2- 172.21.15.26:0/3507156155 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe2301049a0 0x7fe23019ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:26.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:26.859+0000 7fe22f59e640 1 -- 172.21.15.26:0/3507156155 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe2301049a0 msgr2=0x7fe23019ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:26.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:26.859+0000 7fe22f59e640 1 --2- 172.21.15.26:0/3507156155 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe2301049a0 0x7fe23019ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:26.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:26.859+0000 7fe22f59e640 1 -- 172.21.15.26:0/3507156155 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe230103ff0 msgr2=0x7fe23019a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:26.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:26.859+0000 7fe22f59e640 1 --2- 172.21.15.26:0/3507156155 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe230103ff0 0x7fe23019a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:26.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:26.859+0000 7fe22f59e640 1 -- 172.21.15.26:0/3507156155 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe23019f5c0 con 0x7fe230109120 2024-09-11T23:18:26.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:26.859+0000 7fe22ed9d640 1 --2- 172.21.15.26:0/3507156155 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe230103ff0 0x7fe23019a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:18:26.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:26.859+0000 7fe22e59c640 1 --2- 172.21.15.26:0/3507156155 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe2301049a0 0x7fe23019ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:18:26.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:26.859+0000 7fe22f59e640 1 --2- 172.21.15.26:0/3507156155 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe230109120 0x7fe23019eee0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fe228009fd0 tx=0x7fe22800ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:18:26.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:26.859+0000 7fe217fff640 1 -- 172.21.15.26:0/3507156155 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe22800ce20 con 0x7fe230109120 2024-09-11T23:18:26.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:26.859+0000 7fe217fff640 1 -- 172.21.15.26:0/3507156155 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe228005020 con 0x7fe230109120 2024-09-11T23:18:26.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:26.859+0000 7fe217fff640 1 -- 172.21.15.26:0/3507156155 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe228010720 con 0x7fe230109120 2024-09-11T23:18:26.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:26.859+0000 7fe2350c3640 1 -- 172.21.15.26:0/3507156155 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe23019f8b0 con 0x7fe230109120 2024-09-11T23:18:26.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:26.859+0000 7fe2350c3640 1 -- 172.21.15.26:0/3507156155 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2301a7190 con 0x7fe230109120 2024-09-11T23:18:26.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:26.861+0000 7fe2350c3640 1 -- 172.21.15.26:0/3507156155 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2301043d0 con 0x7fe230109120 2024-09-11T23:18:26.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:26.862+0000 7fe217fff640 1 -- 172.21.15.26:0/3507156155 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 23) ==== 137953+0+0 (secure 0 0 0) 0x7fe228005540 con 0x7fe230109120 2024-09-11T23:18:26.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:26.863+0000 7fe217fff640 1 --2- 172.21.15.26:0/3507156155 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fe2180a2cf0 0x7fe2180a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:26.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:26.863+0000 7fe22ed9d640 1 --2- 172.21.15.26:0/3507156155 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fe2180a2cf0 0x7fe2180a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:26.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:26.863+0000 7fe217fff640 1 -- 172.21.15.26:0/3507156155 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fe2280cf9d0 con 0x7fe230109120 2024-09-11T23:18:26.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:26.863+0000 7fe22ed9d640 1 --2- 172.21.15.26:0/3507156155 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fe2180a2cf0 0x7fe2180a51b0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fe2200096c0 tx=0x7fe220005a20 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:18:26.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:26.867+0000 7fe217fff640 1 -- 172.21.15.26:0/3507156155 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe22809c600 con 0x7fe230109120 2024-09-11T23:18:27.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:27.008+0000 7fe217fff640 1 -- 172.21.15.26:0/3507156155 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mgrmap(e 24) ==== 138009+0+0 (secure 0 0 0) 0x7fe2280cd3b0 con 0x7fe230109120 2024-09-11T23:18:27.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:27.105+0000 7fe2350c3640 1 -- 172.21.15.26:0/3507156155 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe23019b970 con 0x7fe230109120 2024-09-11T23:18:27.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:27.105+0000 7fe217fff640 1 -- 172.21.15.26:0/3507156155 <== mon.0 v2:172.21.15.26:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11971 (secure 0 0 0) 0x7fe2280a2aa0 con 0x7fe230109120 2024-09-11T23:18:27.107 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-11T23:18:27.107 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":29,"btime":"2024-09-11T23:18:21:019324+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14499,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/367659515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":367659515},{"type":"v1","addr":"172.21.15.26:6835","nonce":367659515}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14511,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/2041065299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":2041065299},{"type":"v1","addr":"172.21.15.26:6837","nonce":2041065299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:16:26.544203+0000","modified":"2024-09-11T23:18:20.969151+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24485,"mds_1":24463,"mds_2":24458,"mds_3":24448,"mds_4":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14550":{"gid":14550,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6839/2233068305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":2233068305},{"type":"v1","addr":"172.21.15.26:6839","nonce":2233068305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14562":{"gid":14562,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6841/2984982336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":2984982336},{"type":"v1","addr":"172.21.15.26:6841","nonce":2984982336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6833/2019841455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":2019841455},{"type":"v1","addr":"172.21.15.136:6833","nonce":2019841455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.76:6833/3438147050","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3438147050},{"type":"v1","addr":"172.21.15.76:6833","nonce":3438147050}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6835/3617665975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3617665975},{"type":"v1","addr":"172.21.15.136:6835","nonce":3617665975}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":12,"addr":"172.21.15.76:6835/393308499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":393308499},{"type":"v1","addr":"172.21.15.76:6835","nonce":393308499}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":10,"addr":"172.21.15.136:6837/1633082387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":1633082387},{"type":"v1","addr":"172.21.15.136:6837","nonce":1633082387}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.76:6837/2078978748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":2078978748},{"type":"v1","addr":"172.21.15.76:6837","nonce":2078978748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.136:6839/3211398630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":3211398630},{"type":"v1","addr":"172.21.15.136:6839","nonce":3211398630}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.76:6839/2077637090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6838","nonce":2077637090},{"type":"v1","addr":"172.21.15.76:6839","nonce":2077637090}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24485,"qdb_cluster":[24443,24448,24458,24463,24485]},"id":1}]} 2024-09-11T23:18:27.108 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 29 2024-09-11T23:18:27.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:27.108+0000 7fe2350c3640 1 -- 172.21.15.26:0/3507156155 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fe2180a2cf0 msgr2=0x7fe2180a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:27.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:27.108+0000 7fe2350c3640 1 --2- 172.21.15.26:0/3507156155 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fe2180a2cf0 0x7fe2180a51b0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fe2200096c0 tx=0x7fe220005a20 comp rx=0 tx=0).stop 2024-09-11T23:18:27.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:27.108+0000 7fe2350c3640 1 -- 172.21.15.26:0/3507156155 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe230109120 msgr2=0x7fe23019eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:27.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:27.108+0000 7fe2350c3640 1 --2- 172.21.15.26:0/3507156155 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe230109120 0x7fe23019eee0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fe228009fd0 tx=0x7fe22800ef10 comp rx=0 tx=0).stop 2024-09-11T23:18:27.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:27.108+0000 7fe2350c3640 1 -- 172.21.15.26:0/3507156155 shutdown_connections 2024-09-11T23:18:27.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:27.108+0000 7fe2350c3640 1 --2- 172.21.15.26:0/3507156155 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fe2180a2cf0 0x7fe2180a51b0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:27.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:27.108+0000 7fe2350c3640 1 --2- 172.21.15.26:0/3507156155 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe230109120 0x7fe23019eee0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:27.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:27.108+0000 7fe2350c3640 1 --2- 172.21.15.26:0/3507156155 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe2301049a0 0x7fe23019ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:27.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:27.108+0000 7fe2350c3640 1 --2- 172.21.15.26:0/3507156155 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe230103ff0 0x7fe23019a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:27.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:27.108+0000 7fe2350c3640 1 -- 172.21.15.26:0/3507156155 >> 172.21.15.26:0/3507156155 conn(0x7fe2300ffa90 msgr2=0x7fe230100df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:18:27.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:27.108+0000 7fe2350c3640 1 -- 172.21.15.26:0/3507156155 shutdown_connections 2024-09-11T23:18:27.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:27.108+0000 7fe2350c3640 1 -- 172.21.15.26:0/3507156155 wait complete. 2024-09-11T23:18:27.474 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:27 smithi026 ceph-mon[31172]: from='client.15273 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T23:18:27.474 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:27 smithi026 ceph-mon[31172]: Adjusting osd_memory_target on smithi026 to 177.0M 2024-09-11T23:18:27.474 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:27 smithi026 ceph-mon[31172]: Unable to set osd_memory_target on smithi026 to 185689190: error parsing value: Value '185689190' is below minimum 939524096 2024-09-11T23:18:27.474 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:27 smithi026 ceph-mon[31172]: Updating smithi026:/etc/ceph/ceph.conf 2024-09-11T23:18:27.474 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:27 smithi026 ceph-mon[31172]: Updating smithi076:/etc/ceph/ceph.conf 2024-09-11T23:18:27.474 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:27 smithi026 ceph-mon[31172]: Updating smithi136:/etc/ceph/ceph.conf 2024-09-11T23:18:27.474 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:27 smithi026 ceph-mon[31172]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:18:27.475 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:27 smithi026 ceph-mon[31172]: Updating smithi076:/var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T23:18:27.475 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:27 smithi026 ceph-mon[31172]: Updating smithi026:/var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T23:18:27.475 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:27 smithi026 ceph-mon[31172]: Updating smithi136:/var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T23:18:27.475 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:27 smithi026 ceph-mon[31172]: Updating smithi076:/etc/ceph/ceph.client.admin.keyring 2024-09-11T23:18:27.475 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:27 smithi026 ceph-mon[31172]: Updating smithi136:/etc/ceph/ceph.client.admin.keyring 2024-09-11T23:18:27.475 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:27 smithi026 ceph-mon[31172]: Updating smithi026:/etc/ceph/ceph.client.admin.keyring 2024-09-11T23:18:27.475 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:27 smithi026 ceph-mon[31172]: Updating smithi076:/var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T23:18:27.475 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:27 smithi026 ceph-mon[31172]: Updating smithi136:/var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T23:18:27.475 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:27 smithi026 ceph-mon[31172]: Updating smithi026:/var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T23:18:27.475 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:27 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:18:27.475 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:27 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:18:27.475 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:27 smithi026 ceph-mon[31172]: mgrmap e24: x(active, since 4s), standbys: y, z 2024-09-11T23:18:27.475 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:27 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:18:27.475 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:27 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:18:27.476 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:27 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:18:27.476 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:27 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:18:27.476 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:27 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:18:27.476 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:27 smithi026 ceph-mon[31172]: from='client.? 172.21.15.26:0/3507156155' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:18:27.476 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:27 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:18:27.476 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:27 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:18:27.476 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:27 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:18:27.476 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:27 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:18:27.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:27 smithi136 ceph-mon[35121]: from='client.15273 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T23:18:27.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:27 smithi136 ceph-mon[35121]: Adjusting osd_memory_target on smithi026 to 177.0M 2024-09-11T23:18:27.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:27 smithi136 ceph-mon[35121]: Unable to set osd_memory_target on smithi026 to 185689190: error parsing value: Value '185689190' is below minimum 939524096 2024-09-11T23:18:27.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:27 smithi136 ceph-mon[35121]: Updating smithi026:/etc/ceph/ceph.conf 2024-09-11T23:18:27.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:27 smithi136 ceph-mon[35121]: Updating smithi076:/etc/ceph/ceph.conf 2024-09-11T23:18:27.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:27 smithi136 ceph-mon[35121]: Updating smithi136:/etc/ceph/ceph.conf 2024-09-11T23:18:27.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:27 smithi136 ceph-mon[35121]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:18:27.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:27 smithi136 ceph-mon[35121]: Updating smithi076:/var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T23:18:27.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:27 smithi136 ceph-mon[35121]: Updating smithi026:/var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T23:18:27.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:27 smithi136 ceph-mon[35121]: Updating smithi136:/var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T23:18:27.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:27 smithi136 ceph-mon[35121]: Updating smithi076:/etc/ceph/ceph.client.admin.keyring 2024-09-11T23:18:27.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:27 smithi136 ceph-mon[35121]: Updating smithi136:/etc/ceph/ceph.client.admin.keyring 2024-09-11T23:18:27.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:27 smithi136 ceph-mon[35121]: Updating smithi026:/etc/ceph/ceph.client.admin.keyring 2024-09-11T23:18:27.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:27 smithi136 ceph-mon[35121]: Updating smithi076:/var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T23:18:27.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:27 smithi136 ceph-mon[35121]: Updating smithi136:/var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T23:18:27.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:27 smithi136 ceph-mon[35121]: Updating smithi026:/var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T23:18:27.741 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:27 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:18:27.741 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:27 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:18:27.741 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:27 smithi136 ceph-mon[35121]: mgrmap e24: x(active, since 4s), standbys: y, z 2024-09-11T23:18:27.741 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:27 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:18:27.741 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:27 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:18:27.741 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:27 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:18:27.741 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:27 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:18:27.741 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:27 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:18:27.741 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:27 smithi136 ceph-mon[35121]: from='client.? 172.21.15.26:0/3507156155' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:18:27.741 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:27 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:18:27.742 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:27 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:18:27.742 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:27 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:18:27.742 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:27 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:18:27.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:27 smithi076 ceph-mon[35296]: from='client.15273 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T23:18:27.804 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:27 smithi076 ceph-mon[35296]: Adjusting osd_memory_target on smithi026 to 177.0M 2024-09-11T23:18:27.804 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:27 smithi076 ceph-mon[35296]: Unable to set osd_memory_target on smithi026 to 185689190: error parsing value: Value '185689190' is below minimum 939524096 2024-09-11T23:18:27.804 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:27 smithi076 ceph-mon[35296]: Updating smithi026:/etc/ceph/ceph.conf 2024-09-11T23:18:27.804 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:27 smithi076 ceph-mon[35296]: Updating smithi076:/etc/ceph/ceph.conf 2024-09-11T23:18:27.804 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:27 smithi076 ceph-mon[35296]: Updating smithi136:/etc/ceph/ceph.conf 2024-09-11T23:18:27.804 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:27 smithi076 ceph-mon[35296]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:18:27.804 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:27 smithi076 ceph-mon[35296]: Updating smithi076:/var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T23:18:27.804 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:27 smithi076 ceph-mon[35296]: Updating smithi026:/var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T23:18:27.804 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:27 smithi076 ceph-mon[35296]: Updating smithi136:/var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T23:18:27.805 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:27 smithi076 ceph-mon[35296]: Updating smithi076:/etc/ceph/ceph.client.admin.keyring 2024-09-11T23:18:27.805 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:27 smithi076 ceph-mon[35296]: Updating smithi136:/etc/ceph/ceph.client.admin.keyring 2024-09-11T23:18:27.805 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:27 smithi076 ceph-mon[35296]: Updating smithi026:/etc/ceph/ceph.client.admin.keyring 2024-09-11T23:18:27.805 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:27 smithi076 ceph-mon[35296]: Updating smithi076:/var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T23:18:27.805 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:27 smithi076 ceph-mon[35296]: Updating smithi136:/var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T23:18:27.805 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:27 smithi076 ceph-mon[35296]: Updating smithi026:/var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T23:18:27.805 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:27 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:18:27.805 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:27 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:18:27.805 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:27 smithi076 ceph-mon[35296]: mgrmap e24: x(active, since 4s), standbys: y, z 2024-09-11T23:18:27.805 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:27 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:18:27.805 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:27 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:18:27.805 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:27 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:18:27.805 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:27 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:18:27.806 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:27 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:18:27.806 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:27 smithi076 ceph-mon[35296]: from='client.? 172.21.15.26:0/3507156155' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:18:27.806 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:27 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:18:27.806 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:27 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:18:27.806 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:27 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:18:27.806 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:27 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:18:27.850 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-11T23:18:28.108 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:18:28.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:28.564+0000 7f5cf2d4c640 1 -- 172.21.15.26:0/4279705685 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5cec106f40 msgr2=0x7f5cec10eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:28.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:28.564+0000 7f5cf2d4c640 1 --2- 172.21.15.26:0/4279705685 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5cec106f40 0x7f5cec10eac0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f5ce4009ca0 tx=0x7f5ce402a8c0 comp rx=0 tx=0).stop 2024-09-11T23:18:28.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:28.564+0000 7f5cf2d4c640 1 -- 172.21.15.26:0/4279705685 shutdown_connections 2024-09-11T23:18:28.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:28.564+0000 7f5cf2d4c640 1 --2- 172.21.15.26:0/4279705685 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5cec1027c0 0x7f5cec106810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:28.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:28.564+0000 7f5cf2d4c640 1 --2- 172.21.15.26:0/4279705685 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5cec106f40 0x7f5cec10eac0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:28.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:28.564+0000 7f5cf2d4c640 1 --2- 172.21.15.26:0/4279705685 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5cec101e10 0x7f5cec1021f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:28.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:28.564+0000 7f5cf2d4c640 1 -- 172.21.15.26:0/4279705685 >> 172.21.15.26:0/4279705685 conn(0x7f5cec0fd8d0 msgr2=0x7f5cec0ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:18:28.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:28.565+0000 7f5cf2d4c640 1 -- 172.21.15.26:0/4279705685 shutdown_connections 2024-09-11T23:18:28.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:28.565+0000 7f5cf2d4c640 1 -- 172.21.15.26:0/4279705685 wait complete. 2024-09-11T23:18:28.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:28.565+0000 7f5cf2d4c640 1 Processor -- start 2024-09-11T23:18:28.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:28.565+0000 7f5cf2d4c640 1 -- start start 2024-09-11T23:18:28.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:28.565+0000 7f5cf2d4c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5cec101e10 0x7f5cec198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:28.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:28.565+0000 7f5cf2d4c640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5cec1027c0 0x7f5cec198860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:28.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:28.565+0000 7f5cf2d4c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5cec106f40 0x7f5cec19cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:28.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:28.565+0000 7f5cf2d4c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f5cec111fa0 con 0x7f5cec101e10 2024-09-11T23:18:28.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:28.565+0000 7f5cf2d4c640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f5cec111e20 con 0x7f5cec106f40 2024-09-11T23:18:28.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:28.565+0000 7f5cf2d4c640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f5cec112120 con 0x7f5cec1027c0 2024-09-11T23:18:28.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:28.565+0000 7f5cf0ae8640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5cec101e10 0x7f5cec198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:28.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:28.565+0000 7f5cf0ae8640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5cec101e10 0x7f5cec198320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47452/0 (socket says 172.21.15.26:47452) 2024-09-11T23:18:28.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:28.566+0000 7f5cf0ae8640 1 -- 172.21.15.26:0/2440193382 learned_addr learned my addr 172.21.15.26:0/2440193382 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:18:28.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:28.566+0000 7f5cebfff640 1 --2- 172.21.15.26:0/2440193382 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5cec1027c0 0x7f5cec198860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:28.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:28.566+0000 7f5cf0ae8640 1 -- 172.21.15.26:0/2440193382 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5cec1027c0 msgr2=0x7f5cec198860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:28.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:28.566+0000 7f5cf0ae8640 1 --2- 172.21.15.26:0/2440193382 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5cec1027c0 0x7f5cec198860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:28.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:28.566+0000 7f5cf12e9640 1 --2- 172.21.15.26:0/2440193382 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5cec106f40 0x7f5cec19cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:28.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:28.566+0000 7f5cf0ae8640 1 -- 172.21.15.26:0/2440193382 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5cec106f40 msgr2=0x7f5cec19cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:28.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:28.566+0000 7f5cf0ae8640 1 --2- 172.21.15.26:0/2440193382 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5cec106f40 0x7f5cec19cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:28.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:28.566+0000 7f5cf0ae8640 1 -- 172.21.15.26:0/2440193382 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cec19d2d0 con 0x7f5cec101e10 2024-09-11T23:18:28.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:28.566+0000 7f5cf0ae8640 1 --2- 172.21.15.26:0/2440193382 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5cec101e10 0x7f5cec198320 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f5ce000c920 tx=0x7f5ce000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:18:28.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:28.566+0000 7f5ce9ffb640 1 -- 172.21.15.26:0/2440193382 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ce0007c70 con 0x7f5cec101e10 2024-09-11T23:18:28.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:28.566+0000 7f5ce9ffb640 1 -- 172.21.15.26:0/2440193382 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ce0007e10 con 0x7f5cec101e10 2024-09-11T23:18:28.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:28.566+0000 7f5cf2d4c640 1 -- 172.21.15.26:0/2440193382 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cec19d5c0 con 0x7f5cec101e10 2024-09-11T23:18:28.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:28.566+0000 7f5ce9ffb640 1 -- 172.21.15.26:0/2440193382 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ce0018890 con 0x7f5cec101e10 2024-09-11T23:18:28.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:28.566+0000 7f5cf2d4c640 1 -- 172.21.15.26:0/2440193382 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cec1a4ea0 con 0x7f5cec101e10 2024-09-11T23:18:28.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:28.567+0000 7f5cf2d4c640 1 -- 172.21.15.26:0/2440193382 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5cec1021f0 con 0x7f5cec101e10 2024-09-11T23:18:28.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:28.568+0000 7f5ce9ffb640 1 -- 172.21.15.26:0/2440193382 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 24) ==== 138009+0+0 (secure 0 0 0) 0x7f5ce0018a70 con 0x7f5cec101e10 2024-09-11T23:18:28.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:28.569+0000 7f5ce9ffb640 1 --2- 172.21.15.26:0/2440193382 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f5cdc0a2d80 0x7f5cdc0a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:28.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:28.569+0000 7f5ce9ffb640 1 -- 172.21.15.26:0/2440193382 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f5ce00cf7e0 con 0x7f5cec101e10 2024-09-11T23:18:28.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:28.569+0000 7f5cebfff640 1 --2- 172.21.15.26:0/2440193382 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f5cdc0a2d80 0x7f5cdc0a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:28.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:28.570+0000 7f5cebfff640 1 --2- 172.21.15.26:0/2440193382 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f5cdc0a2d80 0x7f5cdc0a5240 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f5cd8005fd0 tx=0x7f5cd8005850 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:18:28.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:28.573+0000 7f5ce9ffb640 1 -- 172.21.15.26:0/2440193382 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5ce0014030 con 0x7f5cec101e10 2024-09-11T23:18:28.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:28.751+0000 7f5cf2d4c640 1 -- 172.21.15.26:0/2440193382 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5cec199680 con 0x7f5cec101e10 2024-09-11T23:18:28.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:28.753+0000 7f5ce9ffb640 1 -- 172.21.15.26:0/2440193382 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+23109 (secure 0 0 0) 0x7f5ce000de20 con 0x7f5cec101e10 2024-09-11T23:18:28.754 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-11T23:18:28.754 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":86,"fsid":"af7e6fc8-7092-11ef-bcea-c7b262605968","created":"2024-09-11T23:08:40.486438+0000","modified":"2024-09-11T23:18:22.177962+0000","last_up_change":"2024-09-11T23:15:35.179446+0000","last_in_change":"2024-09-11T23:15:11.064627+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T23:11:50.494160+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-11T23:16:22.503928+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-11T23:16:24.876554+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a4d62fcf-c31e-41b1-932c-3d87ff640531","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":2014857905},{"type":"v1","addr":"172.21.15.26:6803","nonce":2014857905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":2014857905},{"type":"v1","addr":"172.21.15.26:6805","nonce":2014857905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":2014857905},{"type":"v1","addr":"172.21.15.26:6809","nonce":2014857905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":2014857905},{"type":"v1","addr":"172.21.15.26:6807","nonce":2014857905}]},"public_addr":"172.21.15.26:6803/2014857905","cluster_addr":"172.21.15.26:6805/2014857905","heartbeat_back_addr":"172.21.15.26:6809/2014857905","heartbeat_front_addr":"172.21.15.26:6807/2014857905","state":["exists","up"]},{"osd":1,"uuid":"e0d69112-3c0a-4fc5-8783-e41b4081e0c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":333383087},{"type":"v1","addr":"172.21.15.76:6801","nonce":333383087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":333383087},{"type":"v1","addr":"172.21.15.76:6803","nonce":333383087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":333383087},{"type":"v1","addr":"172.21.15.76:6807","nonce":333383087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":333383087},{"type":"v1","addr":"172.21.15.76:6805","nonce":333383087}]},"public_addr":"172.21.15.76:6801/333383087","cluster_addr":"172.21.15.76:6803/333383087","heartbeat_back_addr":"172.21.15.76:6807/333383087","heartbeat_front_addr":"172.21.15.76:6805/333383087","state":["exists","up"]},{"osd":2,"uuid":"d578429e-bf23-4a1e-890f-7172955c0eb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":1271538911},{"type":"v1","addr":"172.21.15.136:6801","nonce":1271538911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":1271538911},{"type":"v1","addr":"172.21.15.136:6803","nonce":1271538911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":1271538911},{"type":"v1","addr":"172.21.15.136:6807","nonce":1271538911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":1271538911},{"type":"v1","addr":"172.21.15.136:6805","nonce":1271538911}]},"public_addr":"172.21.15.136:6801/1271538911","cluster_addr":"172.21.15.136:6803/1271538911","heartbeat_back_addr":"172.21.15.136:6807/1271538911","heartbeat_front_addr":"172.21.15.136:6805/1271538911","state":["exists","up"]},{"osd":3,"uuid":"3e34623f-0ea5-4b10-b8a9-f8756c35d512","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":1740304151},{"type":"v1","addr":"172.21.15.26:6811","nonce":1740304151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":1740304151},{"type":"v1","addr":"172.21.15.26:6813","nonce":1740304151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1740304151},{"type":"v1","addr":"172.21.15.26:6817","nonce":1740304151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":1740304151},{"type":"v1","addr":"172.21.15.26:6815","nonce":1740304151}]},"public_addr":"172.21.15.26:6811/1740304151","cluster_addr":"172.21.15.26:6813/1740304151","heartbeat_back_addr":"172.21.15.26:6817/1740304151","heartbeat_front_addr":"172.21.15.26:6815/1740304151","state":["exists","up"]},{"osd":4,"uuid":"77607929-f9ed-49a9-abb5-48cd88eec4ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":2702262182},{"type":"v1","addr":"172.21.15.76:6809","nonce":2702262182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":2702262182},{"type":"v1","addr":"172.21.15.76:6811","nonce":2702262182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":2702262182},{"type":"v1","addr":"172.21.15.76:6815","nonce":2702262182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":2702262182},{"type":"v1","addr":"172.21.15.76:6813","nonce":2702262182}]},"public_addr":"172.21.15.76:6809/2702262182","cluster_addr":"172.21.15.76:6811/2702262182","heartbeat_back_addr":"172.21.15.76:6815/2702262182","heartbeat_front_addr":"172.21.15.76:6813/2702262182","state":["exists","up"]},{"osd":5,"uuid":"29e20e37-09a0-4924-8aaa-17939763a7c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":1749229277},{"type":"v1","addr":"172.21.15.136:6809","nonce":1749229277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":1749229277},{"type":"v1","addr":"172.21.15.136:6811","nonce":1749229277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":1749229277},{"type":"v1","addr":"172.21.15.136:6815","nonce":1749229277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":1749229277},{"type":"v1","addr":"172.21.15.136:6813","nonce":1749229277}]},"public_addr":"172.21.15.136:6809/1749229277","cluster_addr":"172.21.15.136:6811/1749229277","heartbeat_back_addr":"172.21.15.136:6815/1749229277","heartbeat_front_addr":"172.21.15.136:6813/1749229277","state":["exists","up"]},{"osd":6,"uuid":"e0ddd1c0-1c46-41e6-b2f4-396bdcd2bf8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1928245624},{"type":"v1","addr":"172.21.15.26:6819","nonce":1928245624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1928245624},{"type":"v1","addr":"172.21.15.26:6821","nonce":1928245624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":1928245624},{"type":"v1","addr":"172.21.15.26:6825","nonce":1928245624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1928245624},{"type":"v1","addr":"172.21.15.26:6823","nonce":1928245624}]},"public_addr":"172.21.15.26:6819/1928245624","cluster_addr":"172.21.15.26:6821/1928245624","heartbeat_back_addr":"172.21.15.26:6825/1928245624","heartbeat_front_addr":"172.21.15.26:6823/1928245624","state":["exists","up"]},{"osd":7,"uuid":"fe031f5c-d915-48fb-91a6-c76f252467fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":1409789455},{"type":"v1","addr":"172.21.15.76:6817","nonce":1409789455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":1409789455},{"type":"v1","addr":"172.21.15.76:6819","nonce":1409789455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":1409789455},{"type":"v1","addr":"172.21.15.76:6823","nonce":1409789455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":1409789455},{"type":"v1","addr":"172.21.15.76:6821","nonce":1409789455}]},"public_addr":"172.21.15.76:6817/1409789455","cluster_addr":"172.21.15.76:6819/1409789455","heartbeat_back_addr":"172.21.15.76:6823/1409789455","heartbeat_front_addr":"172.21.15.76:6821/1409789455","state":["exists","up"]},{"osd":8,"uuid":"18851bde-9e6c-4e05-b4bf-ac5c3073eee7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":543619079},{"type":"v1","addr":"172.21.15.136:6817","nonce":543619079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":543619079},{"type":"v1","addr":"172.21.15.136:6819","nonce":543619079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":543619079},{"type":"v1","addr":"172.21.15.136:6823","nonce":543619079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":543619079},{"type":"v1","addr":"172.21.15.136:6821","nonce":543619079}]},"public_addr":"172.21.15.136:6817/543619079","cluster_addr":"172.21.15.136:6819/543619079","heartbeat_back_addr":"172.21.15.136:6823/543619079","heartbeat_front_addr":"172.21.15.136:6821/543619079","state":["exists","up"]},{"osd":9,"uuid":"52a65eab-8446-47ad-bfe7-5133f7e3eb98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":2201297064},{"type":"v1","addr":"172.21.15.26:6827","nonce":2201297064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":2201297064},{"type":"v1","addr":"172.21.15.26:6829","nonce":2201297064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":2201297064},{"type":"v1","addr":"172.21.15.26:6833","nonce":2201297064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":2201297064},{"type":"v1","addr":"172.21.15.26:6831","nonce":2201297064}]},"public_addr":"172.21.15.26:6827/2201297064","cluster_addr":"172.21.15.26:6829/2201297064","heartbeat_back_addr":"172.21.15.26:6833/2201297064","heartbeat_front_addr":"172.21.15.26:6831/2201297064","state":["exists","up"]},{"osd":10,"uuid":"b758d4bb-da62-4c5f-a992-bb6a145d35d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":3184128096},{"type":"v1","addr":"172.21.15.76:6825","nonce":3184128096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":3184128096},{"type":"v1","addr":"172.21.15.76:6827","nonce":3184128096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":3184128096},{"type":"v1","addr":"172.21.15.76:6831","nonce":3184128096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":3184128096},{"type":"v1","addr":"172.21.15.76:6829","nonce":3184128096}]},"public_addr":"172.21.15.76:6825/3184128096","cluster_addr":"172.21.15.76:6827/3184128096","heartbeat_back_addr":"172.21.15.76:6831/3184128096","heartbeat_front_addr":"172.21.15.76:6829/3184128096","state":["exists","up"]},{"osd":11,"uuid":"8fddda46-e22b-4d0c-ac48-e038fb29dfec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":1960689685},{"type":"v1","addr":"172.21.15.136:6825","nonce":1960689685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":1960689685},{"type":"v1","addr":"172.21.15.136:6827","nonce":1960689685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":1960689685},{"type":"v1","addr":"172.21.15.136:6831","nonce":1960689685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":1960689685},{"type":"v1","addr":"172.21.15.136:6829","nonce":1960689685}]},"public_addr":"172.21.15.136:6825/1960689685","cluster_addr":"172.21.15.136:6827/1960689685","heartbeat_back_addr":"172.21.15.136:6831/1960689685","heartbeat_front_addr":"172.21.15.136:6829/1960689685","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:10:56.656177+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:11:22.234031+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:11:46.464102+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:12:12.353967+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:12:37.836200+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:13:01.728839+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:13:27.527230+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:13:51.981393+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:14:17.035464+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:14:42.805029+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:15:08.208999+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:15:32.800844+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.18","mappings":[{"from":3,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":6}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.26:0/2626831531":"2024-09-12T23:18:22.177934+0000","172.21.15.26:0/3861103130":"2024-09-12T23:09:09.781665+0000","172.21.15.26:0/17142658":"2024-09-12T23:09:09.781665+0000","172.21.15.26:0/2817092653":"2024-09-12T23:09:09.781665+0000","172.21.15.26:6801/110833859":"2024-09-12T23:18:22.177934+0000","172.21.15.26:0/2107281413":"2024-09-12T23:09:33.983310+0000","172.21.15.26:0/883212708":"2024-09-12T23:09:33.983310+0000","172.21.15.26:6800/2673575521":"2024-09-12T23:09:09.781665+0000","172.21.15.26:0/2559066894":"2024-09-12T23:18:22.177934+0000","172.21.15.26:6801/2673575521":"2024-09-12T23:09:09.781665+0000","172.21.15.26:0/3597996003":"2024-09-12T23:09:33.983310+0000","172.21.15.26:6800/677148151":"2024-09-12T23:09:33.983310+0000","172.21.15.26:6801/677148151":"2024-09-12T23:09:33.983310+0000","172.21.15.26:0/1584355185":"2024-09-12T23:18:22.177934+0000","172.21.15.26:6800/110833859":"2024-09-12T23:18:22.177934+0000","172.21.15.26:0/1256290054":"2024-09-12T23:18:22.177934+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T23:18:28.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:28.755+0000 7f5cf2d4c640 1 -- 172.21.15.26:0/2440193382 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f5cdc0a2d80 msgr2=0x7f5cdc0a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:28.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:28.755+0000 7f5cf2d4c640 1 --2- 172.21.15.26:0/2440193382 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f5cdc0a2d80 0x7f5cdc0a5240 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f5cd8005fd0 tx=0x7f5cd8005850 comp rx=0 tx=0).stop 2024-09-11T23:18:28.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:28.755+0000 7f5cf2d4c640 1 -- 172.21.15.26:0/2440193382 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5cec101e10 msgr2=0x7f5cec198320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:28.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:28.755+0000 7f5cf2d4c640 1 --2- 172.21.15.26:0/2440193382 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5cec101e10 0x7f5cec198320 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f5ce000c920 tx=0x7f5ce000cde0 comp rx=0 tx=0).stop 2024-09-11T23:18:28.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:28.755+0000 7f5cf2d4c640 1 -- 172.21.15.26:0/2440193382 shutdown_connections 2024-09-11T23:18:28.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:28.755+0000 7f5cf2d4c640 1 --2- 172.21.15.26:0/2440193382 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f5cdc0a2d80 0x7f5cdc0a5240 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:28.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:28.755+0000 7f5cf2d4c640 1 --2- 172.21.15.26:0/2440193382 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5cec106f40 0x7f5cec19cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:28.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:28.755+0000 7f5cf2d4c640 1 --2- 172.21.15.26:0/2440193382 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5cec1027c0 0x7f5cec198860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:28.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:28.755+0000 7f5cf2d4c640 1 --2- 172.21.15.26:0/2440193382 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5cec101e10 0x7f5cec198320 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:28.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:28.755+0000 7f5cf2d4c640 1 -- 172.21.15.26:0/2440193382 >> 172.21.15.26:0/2440193382 conn(0x7f5cec0fd8d0 msgr2=0x7f5cec0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:18:28.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:28.755+0000 7f5cf2d4c640 1 -- 172.21.15.26:0/2440193382 shutdown_connections 2024-09-11T23:18:28.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:28.755+0000 7f5cf2d4c640 1 -- 172.21.15.26:0/2440193382 wait complete. 2024-09-11T23:18:29.421 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:29 smithi026 ceph-mon[31172]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 0 B/s wr, 26 op/s 2024-09-11T23:18:29.421 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:29 smithi026 ceph-mon[31172]: from='client.? 172.21.15.26:0/2440193382' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T23:18:29.452 DEBUG:tasks.fwd_scrub:Yielding 2024-09-11T23:18:29.452 INFO:teuthology.run_tasks:Running task exec... 2024-09-11T23:18:29.462 INFO:teuthology.task.exec:Executing custom commands... 2024-09-11T23:18:29.462 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi026.front.sepia.ceph.com 2024-09-11T23:18:29.462 DEBUG:teuthology.orchestra.run.smithi026:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/log_level debug' 2024-09-11T23:18:29.463 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-09-11T23:18:29.463 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-11T23:18:29.463 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-11T23:18:29.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:29.573+0000 7f373d178640 1 -- 172.21.15.26:0/2140984584 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3738100520 msgr2=0x7f373810e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:29.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:29.573+0000 7f373d178640 1 --2- 172.21.15.26:0/2140984584 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3738100520 0x7f373810e9a0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f3730009ca0 tx=0x7f373002a8c0 comp rx=0 tx=0).stop 2024-09-11T23:18:29.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:29.573+0000 7f373d178640 1 -- 172.21.15.26:0/2140984584 shutdown_connections 2024-09-11T23:18:29.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:29.573+0000 7f373d178640 1 --2- 172.21.15.26:0/2140984584 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f37380ffb80 0x7f37380fffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:29.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:29.573+0000 7f373d178640 1 --2- 172.21.15.26:0/2140984584 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3738100520 0x7f373810e9a0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:29.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:29.573+0000 7f373d178640 1 --2- 172.21.15.26:0/2140984584 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3738106b10 0x7f3738106ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:29.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:29.573+0000 7f373d178640 1 -- 172.21.15.26:0/2140984584 >> 172.21.15.26:0/2140984584 conn(0x7f37380fba30 msgr2=0x7f37380fde50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:18:29.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:29.573+0000 7f373d178640 1 -- 172.21.15.26:0/2140984584 shutdown_connections 2024-09-11T23:18:29.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:29.573+0000 7f373d178640 1 -- 172.21.15.26:0/2140984584 wait complete. 2024-09-11T23:18:29.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:29.573+0000 7f373d178640 1 Processor -- start 2024-09-11T23:18:29.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:29.574+0000 7f373d178640 1 -- start start 2024-09-11T23:18:29.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:29.574+0000 7f373d178640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f37380ffb80 0x7f37381015f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:29.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:29.574+0000 7f373d178640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3738100520 0x7f3738101b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:29.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:29.574+0000 7f373d178640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3738106b10 0x7f3738102070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:29.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:29.574+0000 7f373d178640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f3738111f70 con 0x7f3738100520 2024-09-11T23:18:29.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:29.574+0000 7f373d178640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f3738111df0 con 0x7f3738106b10 2024-09-11T23:18:29.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:29.574+0000 7f373d178640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f37381120f0 con 0x7f37380ffb80 2024-09-11T23:18:29.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:29.574+0000 7f373659c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3738100520 0x7f3738101b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:29.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:29.574+0000 7f373659c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3738100520 0x7f3738101b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47462/0 (socket says 172.21.15.26:47462) 2024-09-11T23:18:29.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:29.574+0000 7f373659c640 1 -- 172.21.15.26:0/1205641655 learned_addr learned my addr 172.21.15.26:0/1205641655 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:18:29.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:29.574+0000 7f373759e640 1 --2- 172.21.15.26:0/1205641655 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3738106b10 0x7f3738102070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:29.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:29.574+0000 7f3736d9d640 1 --2- 172.21.15.26:0/1205641655 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f37380ffb80 0x7f37381015f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:29.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:29.574+0000 7f373659c640 1 -- 172.21.15.26:0/1205641655 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f37380ffb80 msgr2=0x7f37381015f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:29.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:29.574+0000 7f373659c640 1 --2- 172.21.15.26:0/1205641655 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f37380ffb80 0x7f37381015f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:29.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:29.574+0000 7f373659c640 1 -- 172.21.15.26:0/1205641655 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3738106b10 msgr2=0x7f3738102070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:29.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:29.574+0000 7f373659c640 1 --2- 172.21.15.26:0/1205641655 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3738106b10 0x7f3738102070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:29.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:29.574+0000 7f373659c640 1 -- 172.21.15.26:0/1205641655 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37381a6740 con 0x7f3738100520 2024-09-11T23:18:29.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:29.575+0000 7f373659c640 1 --2- 172.21.15.26:0/1205641655 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3738100520 0x7f3738101b30 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f372c00e9f0 tx=0x7f372c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:18:29.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:29.575+0000 7f371ffff640 1 -- 172.21.15.26:0/1205641655 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f372c00cd80 con 0x7f3738100520 2024-09-11T23:18:29.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:29.575+0000 7f371ffff640 1 -- 172.21.15.26:0/1205641655 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f372c010910 con 0x7f3738100520 2024-09-11T23:18:29.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:29.575+0000 7f373d178640 1 -- 172.21.15.26:0/1205641655 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37381a6a30 con 0x7f3738100520 2024-09-11T23:18:29.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:29.575+0000 7f371ffff640 1 -- 172.21.15.26:0/1205641655 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f372c018990 con 0x7f3738100520 2024-09-11T23:18:29.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:29.575+0000 7f373d178640 1 -- 172.21.15.26:0/1205641655 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37381a6f80 con 0x7f3738100520 2024-09-11T23:18:29.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:29.576+0000 7f373d178640 1 -- 172.21.15.26:0/1205641655 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37380fffe0 con 0x7f3738100520 2024-09-11T23:18:29.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:29.577+0000 7f371ffff640 1 -- 172.21.15.26:0/1205641655 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 24) ==== 138009+0+0 (secure 0 0 0) 0x7f372c010ac0 con 0x7f3738100520 2024-09-11T23:18:29.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:29.578+0000 7f371ffff640 1 --2- 172.21.15.26:0/1205641655 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f37200a2e40 0x7f37200a5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:29.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:29.578+0000 7f3736d9d640 1 --2- 172.21.15.26:0/1205641655 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f37200a2e40 0x7f37200a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:29.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:29.578+0000 7f371ffff640 1 -- 172.21.15.26:0/1205641655 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f372c0cfa00 con 0x7f3738100520 2024-09-11T23:18:29.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:29.579+0000 7f3736d9d640 1 --2- 172.21.15.26:0/1205641655 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f37200a2e40 0x7f37200a5300 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f3728005fd0 tx=0x7f372800b510 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:18:29.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:29.582+0000 7f371ffff640 1 -- 172.21.15.26:0/1205641655 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f372c09c460 con 0x7f3738100520 2024-09-11T23:18:29.729 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:18:29.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:29 smithi136 ceph-mon[35121]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 0 B/s wr, 26 op/s 2024-09-11T23:18:29.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:29 smithi136 ceph-mon[35121]: from='client.? 172.21.15.26:0/2440193382' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T23:18:29.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:29.772+0000 7f373d178640 1 -- 172.21.15.26:0/1205641655 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/log_level}] v 0) -- 0x7f37381a7250 con 0x7f3738100520 2024-09-11T23:18:29.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:29.776+0000 7f371ffff640 1 -- 172.21.15.26:0/1205641655 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/log_level}]=0 v28)=0 v28) ==== 129+0+0 (secure 0 0 0) 0x7f372c09bd60 con 0x7f3738100520 2024-09-11T23:18:29.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:29.779+0000 7f373d178640 1 -- 172.21.15.26:0/1205641655 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f37200a2e40 msgr2=0x7f37200a5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:29.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:29.779+0000 7f373d178640 1 --2- 172.21.15.26:0/1205641655 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f37200a2e40 0x7f37200a5300 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f3728005fd0 tx=0x7f372800b510 comp rx=0 tx=0).stop 2024-09-11T23:18:29.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:29.779+0000 7f373d178640 1 -- 172.21.15.26:0/1205641655 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3738100520 msgr2=0x7f3738101b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:29.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:29.779+0000 7f373d178640 1 --2- 172.21.15.26:0/1205641655 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3738100520 0x7f3738101b30 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f372c00e9f0 tx=0x7f372c00eeb0 comp rx=0 tx=0).stop 2024-09-11T23:18:29.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:29.779+0000 7f373d178640 1 -- 172.21.15.26:0/1205641655 shutdown_connections 2024-09-11T23:18:29.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:29.779+0000 7f373d178640 1 --2- 172.21.15.26:0/1205641655 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f37200a2e40 0x7f37200a5300 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:29.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:29.779+0000 7f373d178640 1 --2- 172.21.15.26:0/1205641655 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3738106b10 0x7f3738102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:29.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:29.779+0000 7f373d178640 1 --2- 172.21.15.26:0/1205641655 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3738100520 0x7f3738101b30 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:29.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:29.779+0000 7f373d178640 1 --2- 172.21.15.26:0/1205641655 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f37380ffb80 0x7f37381015f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:29.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:29.779+0000 7f373d178640 1 -- 172.21.15.26:0/1205641655 >> 172.21.15.26:0/1205641655 conn(0x7f37380fba30 msgr2=0x7f37380fbf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:18:29.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:29.779+0000 7f373d178640 1 -- 172.21.15.26:0/1205641655 shutdown_connections 2024-09-11T23:18:29.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:29.779+0000 7f373d178640 1 -- 172.21.15.26:0/1205641655 wait complete. 2024-09-11T23:18:29.792 DEBUG:teuthology.orchestra.run.smithi026:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2024-09-11T23:18:29.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:29 smithi076 ceph-mon[35296]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 0 B/s wr, 26 op/s 2024-09-11T23:18:29.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:29 smithi076 ceph-mon[35296]: from='client.? 172.21.15.26:0/2440193382' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T23:18:29.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:29.899+0000 7f0d96ae4640 1 -- 172.21.15.26:0/955380162 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0d90102d20 msgr2=0x7f0d90103100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:29.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:29.899+0000 7f0d96ae4640 1 --2- 172.21.15.26:0/955380162 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0d90102d20 0x7f0d90103100 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f0d84009900 tx=0x7f0d8402a480 comp rx=0 tx=0).stop 2024-09-11T23:18:29.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:29.899+0000 7f0d96ae4640 1 -- 172.21.15.26:0/955380162 shutdown_connections 2024-09-11T23:18:29.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:29.899+0000 7f0d96ae4640 1 --2- 172.21.15.26:0/955380162 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0d9010c0f0 0x7f0d9010e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:29.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:29.899+0000 7f0d96ae4640 1 --2- 172.21.15.26:0/955380162 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0d90103640 0x7f0d9010bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:29.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:29.899+0000 7f0d96ae4640 1 --2- 172.21.15.26:0/955380162 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0d90102d20 0x7f0d90103100 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:29.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:29.899+0000 7f0d96ae4640 1 -- 172.21.15.26:0/955380162 >> 172.21.15.26:0/955380162 conn(0x7f0d900fc6b0 msgr2=0x7f0d900fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:18:29.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:29.899+0000 7f0d96ae4640 1 -- 172.21.15.26:0/955380162 shutdown_connections 2024-09-11T23:18:29.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:29.899+0000 7f0d96ae4640 1 -- 172.21.15.26:0/955380162 wait complete. 2024-09-11T23:18:29.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:29.900+0000 7f0d96ae4640 1 Processor -- start 2024-09-11T23:18:29.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:29.900+0000 7f0d96ae4640 1 -- start start 2024-09-11T23:18:29.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:29.900+0000 7f0d96ae4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0d90102d20 0x7f0d90198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:29.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:29.900+0000 7f0d96ae4640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0d90103640 0x7f0d90198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:29.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:29.900+0000 7f0d96ae4640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0d9010c0f0 0x7f0d9019cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:29.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:29.900+0000 7f0d96ae4640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f0d90111ef0 con 0x7f0d90102d20 2024-09-11T23:18:29.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:29.900+0000 7f0d96ae4640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f0d90111d70 con 0x7f0d90103640 2024-09-11T23:18:29.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:29.900+0000 7f0d96ae4640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f0d90112070 con 0x7f0d9010c0f0 2024-09-11T23:18:29.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:29.900+0000 7f0d94880640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0d90102d20 0x7f0d90198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:29.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:29.900+0000 7f0d94880640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0d90102d20 0x7f0d90198420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47490/0 (socket says 172.21.15.26:47490) 2024-09-11T23:18:29.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:29.900+0000 7f0d94880640 1 -- 172.21.15.26:0/2896367954 learned_addr learned my addr 172.21.15.26:0/2896367954 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:18:29.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:29.900+0000 7f0d8ffff640 1 --2- 172.21.15.26:0/2896367954 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0d90103640 0x7f0d90198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:29.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:29.900+0000 7f0d95081640 1 --2- 172.21.15.26:0/2896367954 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0d9010c0f0 0x7f0d9019cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:29.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:29.900+0000 7f0d94880640 1 -- 172.21.15.26:0/2896367954 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0d9010c0f0 msgr2=0x7f0d9019cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:29.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:29.900+0000 7f0d94880640 1 --2- 172.21.15.26:0/2896367954 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0d9010c0f0 0x7f0d9019cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:29.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:29.900+0000 7f0d94880640 1 -- 172.21.15.26:0/2896367954 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0d90103640 msgr2=0x7f0d90198960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:29.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:29.900+0000 7f0d94880640 1 --2- 172.21.15.26:0/2896367954 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0d90103640 0x7f0d90198960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:29.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:29.900+0000 7f0d94880640 1 -- 172.21.15.26:0/2896367954 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d9019d410 con 0x7f0d90102d20 2024-09-11T23:18:29.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:29.901+0000 7f0d94880640 1 --2- 172.21.15.26:0/2896367954 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0d90102d20 0x7f0d90198420 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f0d84006010 tx=0x7f0d840042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:18:29.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:29.901+0000 7f0d8dffb640 1 -- 172.21.15.26:0/2896367954 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d8403f030 con 0x7f0d90102d20 2024-09-11T23:18:29.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:29.901+0000 7f0d8dffb640 1 -- 172.21.15.26:0/2896367954 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d84003ce0 con 0x7f0d90102d20 2024-09-11T23:18:29.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:29.901+0000 7f0d96ae4640 1 -- 172.21.15.26:0/2896367954 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d9019d6a0 con 0x7f0d90102d20 2024-09-11T23:18:29.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:29.901+0000 7f0d8dffb640 1 -- 172.21.15.26:0/2896367954 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d84035070 con 0x7f0d90102d20 2024-09-11T23:18:29.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:29.901+0000 7f0d96ae4640 1 -- 172.21.15.26:0/2896367954 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d901a4f90 con 0x7f0d90102d20 2024-09-11T23:18:29.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:29.902+0000 7f0d96ae4640 1 -- 172.21.15.26:0/2896367954 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d90103100 con 0x7f0d90102d20 2024-09-11T23:18:29.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:29.903+0000 7f0d8dffb640 1 -- 172.21.15.26:0/2896367954 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 24) ==== 138009+0+0 (secure 0 0 0) 0x7f0d84042030 con 0x7f0d90102d20 2024-09-11T23:18:29.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:29.904+0000 7f0d8dffb640 1 --2- 172.21.15.26:0/2896367954 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f0d800a2ec0 0x7f0d800a5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:29.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:29.904+0000 7f0d8ffff640 1 --2- 172.21.15.26:0/2896367954 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f0d800a2ec0 0x7f0d800a5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:29.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:29.904+0000 7f0d8dffb640 1 -- 172.21.15.26:0/2896367954 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f0d840efa50 con 0x7f0d90102d20 2024-09-11T23:18:29.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:29.905+0000 7f0d8ffff640 1 --2- 172.21.15.26:0/2896367954 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f0d800a2ec0 0x7f0d800a5380 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f0d9010d4e0 tx=0x7f0d7c00b390 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:18:29.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:29.908+0000 7f0d8dffb640 1 -- 172.21.15.26:0/2896367954 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0d840bbf20 con 0x7f0d90102d20 2024-09-11T23:18:30.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.087+0000 7f0d96ae4640 1 -- 172.21.15.26:0/2896367954 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7f0d901071a0 con 0x7f0d90102d20 2024-09-11T23:18:30.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.091+0000 7f0d8dffb640 1 -- 172.21.15.26:0/2896367954 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v29)=0 v29) ==== 138+0+0 (secure 0 0 0) 0x7f0d840c0220 con 0x7f0d90102d20 2024-09-11T23:18:30.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.093+0000 7f0d96ae4640 1 -- 172.21.15.26:0/2896367954 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f0d800a2ec0 msgr2=0x7f0d800a5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:30.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.093+0000 7f0d96ae4640 1 --2- 172.21.15.26:0/2896367954 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f0d800a2ec0 0x7f0d800a5380 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f0d9010d4e0 tx=0x7f0d7c00b390 comp rx=0 tx=0).stop 2024-09-11T23:18:30.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.093+0000 7f0d96ae4640 1 -- 172.21.15.26:0/2896367954 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0d90102d20 msgr2=0x7f0d90198420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:30.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.093+0000 7f0d96ae4640 1 --2- 172.21.15.26:0/2896367954 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0d90102d20 0x7f0d90198420 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f0d84006010 tx=0x7f0d840042f0 comp rx=0 tx=0).stop 2024-09-11T23:18:30.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.093+0000 7f0d96ae4640 1 -- 172.21.15.26:0/2896367954 shutdown_connections 2024-09-11T23:18:30.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.093+0000 7f0d96ae4640 1 --2- 172.21.15.26:0/2896367954 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f0d800a2ec0 0x7f0d800a5380 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:30.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.093+0000 7f0d96ae4640 1 --2- 172.21.15.26:0/2896367954 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0d90103640 0x7f0d90198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:30.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.093+0000 7f0d96ae4640 1 --2- 172.21.15.26:0/2896367954 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0d9010c0f0 0x7f0d9019cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:30.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.093+0000 7f0d96ae4640 1 --2- 172.21.15.26:0/2896367954 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0d90102d20 0x7f0d90198420 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:30.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.093+0000 7f0d96ae4640 1 -- 172.21.15.26:0/2896367954 >> 172.21.15.26:0/2896367954 conn(0x7f0d900fc6b0 msgr2=0x7f0d900fd350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:18:30.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.094+0000 7f0d96ae4640 1 -- 172.21.15.26:0/2896367954 shutdown_connections 2024-09-11T23:18:30.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.094+0000 7f0d96ae4640 1 -- 172.21.15.26:0/2896367954 wait complete. 2024-09-11T23:18:30.107 DEBUG:teuthology.orchestra.run.smithi026:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2024-09-11T23:18:30.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.216+0000 7f9a8868d640 1 -- 172.21.15.26:0/4024404943 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9a80107b70 msgr2=0x7f9a80107f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:30.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.216+0000 7f9a8868d640 1 --2- 172.21.15.26:0/4024404943 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9a80107b70 0x7f9a80107f50 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f9a78009930 tx=0x7f9a7802a420 comp rx=0 tx=0).stop 2024-09-11T23:18:30.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.216+0000 7f9a8868d640 1 -- 172.21.15.26:0/4024404943 shutdown_connections 2024-09-11T23:18:30.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.216+0000 7f9a8868d640 1 --2- 172.21.15.26:0/4024404943 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9a80101ad0 0x7f9a80101f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:30.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.216+0000 7f9a8868d640 1 --2- 172.21.15.26:0/4024404943 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9a80102470 0x7f9a8010e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:30.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.216+0000 7f9a8868d640 1 --2- 172.21.15.26:0/4024404943 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9a80107b70 0x7f9a80107f50 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:30.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.216+0000 7f9a8868d640 1 -- 172.21.15.26:0/4024404943 >> 172.21.15.26:0/4024404943 conn(0x7f9a800fd7d0 msgr2=0x7f9a800ffbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:18:30.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.216+0000 7f9a8868d640 1 -- 172.21.15.26:0/4024404943 shutdown_connections 2024-09-11T23:18:30.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.216+0000 7f9a8868d640 1 -- 172.21.15.26:0/4024404943 wait complete. 2024-09-11T23:18:30.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.216+0000 7f9a8868d640 1 Processor -- start 2024-09-11T23:18:30.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.217+0000 7f9a8868d640 1 -- start start 2024-09-11T23:18:30.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.217+0000 7f9a8868d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9a80101ad0 0x7f9a800744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:30.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.217+0000 7f9a8868d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9a80102470 0x7f9a80074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:30.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.217+0000 7f9a8868d640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9a80107b70 0x7f9a80074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:30.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.217+0000 7f9a8868d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f9a80111f20 con 0x7f9a80102470 2024-09-11T23:18:30.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.217+0000 7f9a8868d640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f9a80111da0 con 0x7f9a80101ad0 2024-09-11T23:18:30.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.217+0000 7f9a8868d640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f9a801120a0 con 0x7f9a80107b70 2024-09-11T23:18:30.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.217+0000 7f9a85c28640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9a80102470 0x7f9a80074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:30.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.217+0000 7f9a85c28640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9a80102470 0x7f9a80074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47502/0 (socket says 172.21.15.26:47502) 2024-09-11T23:18:30.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.217+0000 7f9a85c28640 1 -- 172.21.15.26:0/91796576 learned_addr learned my addr 172.21.15.26:0/91796576 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:18:30.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.217+0000 7f9a86c2a640 1 --2- 172.21.15.26:0/91796576 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9a80107b70 0x7f9a80074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:30.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.217+0000 7f9a86429640 1 --2- 172.21.15.26:0/91796576 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9a80101ad0 0x7f9a800744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:30.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.217+0000 7f9a85c28640 1 -- 172.21.15.26:0/91796576 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9a80107b70 msgr2=0x7f9a80074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:30.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.217+0000 7f9a85c28640 1 --2- 172.21.15.26:0/91796576 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9a80107b70 0x7f9a80074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:30.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.217+0000 7f9a85c28640 1 -- 172.21.15.26:0/91796576 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9a80101ad0 msgr2=0x7f9a800744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:30.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.217+0000 7f9a85c28640 1 --2- 172.21.15.26:0/91796576 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9a80101ad0 0x7f9a800744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:30.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.217+0000 7f9a85c28640 1 -- 172.21.15.26:0/91796576 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a801a68b0 con 0x7f9a80102470 2024-09-11T23:18:30.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.217+0000 7f9a86429640 1 --2- 172.21.15.26:0/91796576 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9a80101ad0 0x7f9a800744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:18:30.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.217+0000 7f9a85c28640 1 --2- 172.21.15.26:0/91796576 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9a80102470 0x7f9a80074a30 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f9a7400e9f0 tx=0x7f9a7400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:18:30.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.218+0000 7f9a7f7fe640 1 -- 172.21.15.26:0/91796576 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a7400cd80 con 0x7f9a80102470 2024-09-11T23:18:30.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.218+0000 7f9a8868d640 1 -- 172.21.15.26:0/91796576 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a801a6ba0 con 0x7f9a80102470 2024-09-11T23:18:30.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.218+0000 7f9a7f7fe640 1 -- 172.21.15.26:0/91796576 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a74010910 con 0x7f9a80102470 2024-09-11T23:18:30.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.218+0000 7f9a7f7fe640 1 -- 172.21.15.26:0/91796576 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a740188f0 con 0x7f9a80102470 2024-09-11T23:18:30.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.218+0000 7f9a8868d640 1 -- 172.21.15.26:0/91796576 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a801a7140 con 0x7f9a80102470 2024-09-11T23:18:30.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.219+0000 7f9a8868d640 1 -- 172.21.15.26:0/91796576 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a80101f30 con 0x7f9a80102470 2024-09-11T23:18:30.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.220+0000 7f9a7f7fe640 1 -- 172.21.15.26:0/91796576 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 24) ==== 138009+0+0 (secure 0 0 0) 0x7f9a74010ac0 con 0x7f9a80102470 2024-09-11T23:18:30.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.220+0000 7f9a7f7fe640 1 --2- 172.21.15.26:0/91796576 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f9a680a2e40 0x7f9a680a5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:30.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.221+0000 7f9a7f7fe640 1 -- 172.21.15.26:0/91796576 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f9a740cfad0 con 0x7f9a80102470 2024-09-11T23:18:30.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.221+0000 7f9a86429640 1 --2- 172.21.15.26:0/91796576 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f9a680a2e40 0x7f9a680a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:30.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.221+0000 7f9a86429640 1 --2- 172.21.15.26:0/91796576 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f9a680a2e40 0x7f9a680a5300 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f9a78000c00 tx=0x7f9a78005a20 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:18:30.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.229+0000 7f9a7f7fe640 1 -- 172.21.15.26:0/91796576 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9a7409c530 con 0x7f9a80102470 2024-09-11T23:18:30.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.257+0000 7fd8f4278640 1 -- 172.21.15.26:0/1740214894 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd8ec102ef0 msgr2=0x7fd8ec10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:30.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.257+0000 7fd8f4278640 1 --2- 172.21.15.26:0/1740214894 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd8ec102ef0 0x7fd8ec10da40 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fd8e4009ca0 tx=0x7fd8e402a950 comp rx=0 tx=0).stop 2024-09-11T23:18:30.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.257+0000 7fd8f4278640 1 -- 172.21.15.26:0/1740214894 shutdown_connections 2024-09-11T23:18:30.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.257+0000 7fd8f4278640 1 --2- 172.21.15.26:0/1740214894 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd8ec102570 0x7fd8ec1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:30.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.257+0000 7fd8f4278640 1 --2- 172.21.15.26:0/1740214894 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd8ec102ef0 0x7fd8ec10da40 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:30.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.257+0000 7fd8f4278640 1 --2- 172.21.15.26:0/1740214894 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd8ec100e20 0x7fd8ec101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:30.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.257+0000 7fd8f4278640 1 -- 172.21.15.26:0/1740214894 >> 172.21.15.26:0/1740214894 conn(0x7fd8ec0fc7d0 msgr2=0x7fd8ec0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:18:30.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.258+0000 7fd8f4278640 1 -- 172.21.15.26:0/1740214894 shutdown_connections 2024-09-11T23:18:30.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.258+0000 7fd8f4278640 1 -- 172.21.15.26:0/1740214894 wait complete. 2024-09-11T23:18:30.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.258+0000 7fd8f4278640 1 Processor -- start 2024-09-11T23:18:30.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.258+0000 7fd8f4278640 1 -- start start 2024-09-11T23:18:30.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.258+0000 7fd8f4278640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd8ec100e20 0x7fd8ec19a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:30.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.258+0000 7fd8f4278640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd8ec102570 0x7fd8ec19ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:30.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.258+0000 7fd8f4278640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd8ec102ef0 0x7fd8ec19eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:30.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.258+0000 7fd8f4278640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd8ec110fd0 con 0x7fd8ec100e20 2024-09-11T23:18:30.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.258+0000 7fd8f4278640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fd8ec110e50 con 0x7fd8ec102570 2024-09-11T23:18:30.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.258+0000 7fd8f4278640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd8ec111150 con 0x7fd8ec102ef0 2024-09-11T23:18:30.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.258+0000 7fd8f2014640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd8ec100e20 0x7fd8ec19a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:30.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.259+0000 7fd8f2014640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd8ec100e20 0x7fd8ec19a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47526/0 (socket says 172.21.15.26:47526) 2024-09-11T23:18:30.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.259+0000 7fd8f2014640 1 -- 172.21.15.26:0/291700581 learned_addr learned my addr 172.21.15.26:0/291700581 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:18:30.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.259+0000 7fd8f2815640 1 --2- 172.21.15.26:0/291700581 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd8ec102ef0 0x7fd8ec19eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:30.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.259+0000 7fd8f1813640 1 --2- 172.21.15.26:0/291700581 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd8ec102570 0x7fd8ec19ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:30.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.259+0000 7fd8f2014640 1 -- 172.21.15.26:0/291700581 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd8ec102ef0 msgr2=0x7fd8ec19eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:30.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.259+0000 7fd8f2014640 1 --2- 172.21.15.26:0/291700581 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd8ec102ef0 0x7fd8ec19eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:30.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.259+0000 7fd8f2014640 1 -- 172.21.15.26:0/291700581 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd8ec102570 msgr2=0x7fd8ec19ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:30.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.259+0000 7fd8f2014640 1 --2- 172.21.15.26:0/291700581 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd8ec102570 0x7fd8ec19ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:30.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.259+0000 7fd8f2014640 1 -- 172.21.15.26:0/291700581 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8ec19f5a0 con 0x7fd8ec100e20 2024-09-11T23:18:30.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.259+0000 7fd8f1813640 1 --2- 172.21.15.26:0/291700581 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd8ec102570 0x7fd8ec19ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:18:30.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.259+0000 7fd8f2014640 1 --2- 172.21.15.26:0/291700581 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd8ec100e20 0x7fd8ec19a5f0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fd8e800d770 tx=0x7fd8e800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:18:30.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.259+0000 7fd8e37fe640 1 -- 172.21.15.26:0/291700581 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8e8017070 con 0x7fd8ec100e20 2024-09-11T23:18:30.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.259+0000 7fd8e37fe640 1 -- 172.21.15.26:0/291700581 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd8e8005500 con 0x7fd8ec100e20 2024-09-11T23:18:30.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.259+0000 7fd8f4278640 1 -- 172.21.15.26:0/291700581 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8ec19f890 con 0x7fd8ec100e20 2024-09-11T23:18:30.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.259+0000 7fd8e37fe640 1 -- 172.21.15.26:0/291700581 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8e8004e00 con 0x7fd8ec100e20 2024-09-11T23:18:30.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.259+0000 7fd8f4278640 1 -- 172.21.15.26:0/291700581 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8ec1a7170 con 0x7fd8ec100e20 2024-09-11T23:18:30.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.262+0000 7fd8e37fe640 1 -- 172.21.15.26:0/291700581 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 24) ==== 138009+0+0 (secure 0 0 0) 0x7fd8e8004fa0 con 0x7fd8ec100e20 2024-09-11T23:18:30.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.262+0000 7fd8f6651740 1 Processor -- start 2024-09-11T23:18:30.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.262+0000 7fd8f6651740 1 -- start start 2024-09-11T23:18:30.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.263+0000 7fd8f6651740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e255aa7b60 0x55e255aa7f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:30.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.263+0000 7fd8f6651740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e255aa8480 0x55e255ab2800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:30.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.263+0000 7fd8f6651740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e255ab2d40 0x55e255ab5130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:30.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.263+0000 7fd8f6651740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55e255a0d280 con 0x55e255aa8480 2024-09-11T23:18:30.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.263+0000 7fd8f6651740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55e2559b4b10 con 0x55e255ab2d40 2024-09-11T23:18:30.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.263+0000 7fd8f6651740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55e255a09240 con 0x55e255aa7b60 2024-09-11T23:18:30.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.263+0000 7fd8f2014640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e255aa8480 0x55e255ab2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:30.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.263+0000 7fd8e37fe640 1 --2- 172.21.15.26:0/291700581 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fd8d80a2e00 0x7fd8d80a52c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:30.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.263+0000 7fd8f2014640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e255aa8480 0x55e255ab2800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47530/0 (socket says 172.21.15.26:47530) 2024-09-11T23:18:30.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.263+0000 7fd8f2014640 1 -- 172.21.15.26:0/1622051601 learned_addr learned my addr 172.21.15.26:0/1622051601 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:18:30.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.263+0000 7fd8f2815640 1 --2- 172.21.15.26:0/291700581 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fd8d80a2e00 0x7fd8d80a52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:30.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.263+0000 7fd8f2815640 1 --2- 172.21.15.26:0/1622051601 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e255aa7b60 0x55e255aa7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:30.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.263+0000 7fd8e37fe640 1 -- 172.21.15.26:0/291700581 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fd8e80cf060 con 0x7fd8ec100e20 2024-09-11T23:18:30.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.263+0000 7fd8f2014640 1 -- 172.21.15.26:0/1622051601 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e255aa7b60 msgr2=0x55e255aa7f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:18:30.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.263+0000 7fd8f2014640 1 --2- 172.21.15.26:0/1622051601 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e255aa7b60 0x55e255aa7f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:30.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.263+0000 7fd8f1813640 1 --2- 172.21.15.26:0/1622051601 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e255ab2d40 0x55e255ab5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:30.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.263+0000 7fd8f2014640 1 -- 172.21.15.26:0/1622051601 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e255ab2d40 msgr2=0x55e255ab5130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:18:30.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.263+0000 7fd8f2014640 1 --2- 172.21.15.26:0/1622051601 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e255ab2d40 0x55e255ab5130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:30.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.263+0000 7fd8f2014640 1 -- 172.21.15.26:0/1622051601 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e2559c11f0 con 0x55e255aa8480 2024-09-11T23:18:30.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.263+0000 7fd8f1813640 1 --2- 172.21.15.26:0/1622051601 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e255ab2d40 0x55e255ab5130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:18:30.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.264+0000 7fd8f2014640 1 --2- 172.21.15.26:0/1622051601 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e255aa8480 0x55e255ab2800 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fd8e80a4000 tx=0x7fd8e8009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b79fe7a9e7d40dd server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:18:30.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.264+0000 7fd8f2815640 1 --2- 172.21.15.26:0/291700581 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fd8d80a2e00 0x7fd8d80a52c0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fd8e4009a60 tx=0x7fd8e4004a40 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:18:30.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.264+0000 7fd8e0dea640 1 -- 172.21.15.26:0/1622051601 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8e8020070 con 0x55e255aa8480 2024-09-11T23:18:30.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.264+0000 7fd8e0dea640 1 -- 172.21.15.26:0/1622051601 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd8e80046c0 con 0x55e255aa8480 2024-09-11T23:18:30.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.264+0000 7fd8e0dea640 1 -- 172.21.15.26:0/1622051601 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8e8009cc0 con 0x55e255aa8480 2024-09-11T23:18:30.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.265+0000 7fd8f6651740 1 -- 172.21.15.26:0/1622051601 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e255aa8480 msgr2=0x55e255ab2800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:30.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.265+0000 7fd8f6651740 1 --2- 172.21.15.26:0/1622051601 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e255aa8480 0x55e255ab2800 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fd8e80a4000 tx=0x7fd8e8009c30 comp rx=0 tx=0).stop 2024-09-11T23:18:30.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.265+0000 7fd8f6651740 1 -- 172.21.15.26:0/1622051601 shutdown_connections 2024-09-11T23:18:30.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.265+0000 7fd8f6651740 1 --2- 172.21.15.26:0/1622051601 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e255ab2d40 0x55e255ab5130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:30.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.265+0000 7fd8f6651740 1 --2- 172.21.15.26:0/1622051601 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e255aa8480 0x55e255ab2800 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:30.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.265+0000 7fd8f6651740 1 --2- 172.21.15.26:0/1622051601 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e255aa7b60 0x55e255aa7f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:30.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.265+0000 7fd8f6651740 1 -- 172.21.15.26:0/1622051601 >> 172.21.15.26:0/1622051601 conn(0x55e255aa40f0 msgr2=0x55e255aa44d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:18:30.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.265+0000 7fd8f6651740 1 -- 172.21.15.26:0/1622051601 shutdown_connections 2024-09-11T23:18:30.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.265+0000 7fd8f6651740 1 -- 172.21.15.26:0/1622051601 wait complete. 2024-09-11T23:18:30.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.265+0000 7fd8f6651740 1 -- start start 2024-09-11T23:18:30.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.265+0000 7fd8f6651740 1 Processor -- start 2024-09-11T23:18:30.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.265+0000 7fd8f6651740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e255aa7b60 0x55e255aafca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:30.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.265+0000 7fd8f6651740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e255aa8480 0x55e255ab01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:30.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.265+0000 7fd8f6651740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e255ab2d40 0x55e255ab0720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:30.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.266+0000 7fd8f6651740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55e255ab58b0 con 0x55e255ab2d40 2024-09-11T23:18:30.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.266+0000 7fd8f6651740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55e2559c7f10 con 0x55e255aa7b60 2024-09-11T23:18:30.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.266+0000 7fd8f6651740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55e255ab5a30 con 0x55e255aa8480 2024-09-11T23:18:30.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.266+0000 7fd8f6651740 10 client.0 _ll_register_callbacks cb 0x55e255990a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:18:30.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.266+0000 7fd8f2815640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e255ab2d40 0x55e255ab0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:30.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.266+0000 7fd8f2815640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e255ab2d40 0x55e255ab0720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47540/0 (socket says 172.21.15.26:47540) 2024-09-11T23:18:30.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.266+0000 7fd8f2815640 1 -- 172.21.15.26:0/2173468141 learned_addr learned my addr 172.21.15.26:0/2173468141 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:18:30.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.266+0000 7fd8f1813640 1 --2- 172.21.15.26:0/2173468141 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e255aa8480 0x55e255ab01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:30.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.266+0000 7fd8f2014640 1 --2- 172.21.15.26:0/2173468141 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e255aa7b60 0x55e255aafca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:30.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.266+0000 7fd8f2815640 1 -- 172.21.15.26:0/2173468141 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e255aa8480 msgr2=0x55e255ab01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:18:30.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.266+0000 7fd8f2815640 1 --2- 172.21.15.26:0/2173468141 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e255aa8480 0x55e255ab01e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:30.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.266+0000 7fd8f2815640 1 -- 172.21.15.26:0/2173468141 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e255aa7b60 msgr2=0x55e255aafca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:18:30.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.266+0000 7fd8f2815640 1 --2- 172.21.15.26:0/2173468141 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e255aa7b60 0x55e255aafca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:30.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.266+0000 7fd8f2815640 1 -- 172.21.15.26:0/2173468141 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e25577e6a0 con 0x55e255ab2d40 2024-09-11T23:18:30.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.266+0000 7fd8f2014640 1 --2- 172.21.15.26:0/2173468141 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e255aa7b60 0x55e255aafca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:18:30.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.267+0000 7fd8f2815640 1 --2- 172.21.15.26:0/2173468141 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e255ab2d40 0x55e255ab0720 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fd8e4005fa0 tx=0x7fd8e40095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=285d547d6d37517b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:18:30.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.267+0000 7fd8d5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:18:30.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.267+0000 7fd8d5ffb640 1 -- 172.21.15.26:0/2173468141 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8e404e030 con 0x55e255ab2d40 2024-09-11T23:18:30.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.267+0000 7fd8f6651740 1 -- 172.21.15.26:0/2173468141 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55e255a1a7f0 con 0x55e255ab2d40 2024-09-11T23:18:30.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.267+0000 7fd8d5ffb640 1 -- 172.21.15.26:0/2173468141 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fd8e4044070 con 0x55e255ab2d40 2024-09-11T23:18:30.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.267+0000 7fd8d5ffb640 1 -- 172.21.15.26:0/2173468141 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd8e4034040 con 0x55e255ab2d40 2024-09-11T23:18:30.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.267+0000 7fd8f6651740 10 client.15315 fetch_fsmap learned FSMap version 29 2024-09-11T23:18:30.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.267+0000 7fd8f6651740 1 -- 172.21.15.26:0/2173468141 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e255ac14f0 con 0x55e255ab2d40 2024-09-11T23:18:30.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.267+0000 7fd8d5ffb640 1 -- 172.21.15.26:0/2173468141 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55e255a1a7f0 con 0x55e255ab2d40 2024-09-11T23:18:30.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.267+0000 7fd8d5ffb640 1 -- 172.21.15.26:0/2173468141 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fd8e4047050 con 0x55e255ab2d40 2024-09-11T23:18:30.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.267+0000 7fd8f6651740 10 client.15315 fetch_fsmap finished waiting for FSMap version 29 2024-09-11T23:18:30.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.267+0000 7fd8f6651740 10 client.15315 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:18:30.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.267+0000 7fd8f6651740 20 client.15315 populate_metadata read hostname 'smithi026' 2024-09-11T23:18:30.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.268+0000 7fd8f6651740 1 --2- 172.21.15.26:0/2173468141 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55e255ac29c0 0x55e255ac4e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:30.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.268+0000 7fd8f6651740 4 client.15315 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:18:30.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.268+0000 7fd8f6651740 1 -- 172.21.15.26:0/2173468141 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e255a14d00 con 0x55e255ac29c0 2024-09-11T23:18:30.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.268+0000 7fd8f2014640 1 --2- 172.21.15.26:0/2173468141 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55e255ac29c0 0x55e255ac4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:30.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.268+0000 7fd8d5ffb640 10 client.15315 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:18:30.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.268+0000 7fd8f2014640 1 --2- 172.21.15.26:0/2173468141 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55e255ac29c0 0x55e255ac4e20 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=edf247e4c2d67bc2 server_cookie=9d1d9b661116cbd6 in_seq=0 out_seq=0 2024-09-11T23:18:30.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.269+0000 7fd8d5ffb640 1 -- 172.21.15.26:0/2173468141 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e255a14d00 con 0x55e255ac29c0 2024-09-11T23:18:30.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.269+0000 7fd8d5ffb640 10 client.15315 handle_command_reply: tid=0 2024-09-11T23:18:30.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.269+0000 7fd8f6651740 1 client.15315 shutdown 2024-09-11T23:18:30.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.269+0000 7fd8f6651740 1 -- 172.21.15.26:0/2173468141 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e255ab2d40 msgr2=0x55e255ab0720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:30.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.269+0000 7fd8f6651740 1 --2- 172.21.15.26:0/2173468141 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e255ab2d40 0x55e255ab0720 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fd8e4005fa0 tx=0x7fd8e40095c0 comp rx=0 tx=0).stop 2024-09-11T23:18:30.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.269+0000 7fd8f6651740 1 -- 172.21.15.26:0/2173468141 shutdown_connections 2024-09-11T23:18:30.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.269+0000 7fd8f6651740 1 --2- 172.21.15.26:0/2173468141 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55e255ac29c0 0x55e255ac4e20 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:30.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.269+0000 7fd8f6651740 1 --2- 172.21.15.26:0/2173468141 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e255ab2d40 0x55e255ab0720 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:30.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.269+0000 7fd8f6651740 1 --2- 172.21.15.26:0/2173468141 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e255aa8480 0x55e255ab01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:30.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.269+0000 7fd8f6651740 1 --2- 172.21.15.26:0/2173468141 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e255aa7b60 0x55e255aafca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:30.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.269+0000 7fd8f6651740 1 -- 172.21.15.26:0/2173468141 >> 172.21.15.26:0/2173468141 conn(0x55e255aa40f0 msgr2=0x55e255aa9dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:18:30.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.269+0000 7fd8d5ffb640 0 client.15315 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:18:30.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.270+0000 7fd8f6651740 1 -- 172.21.15.26:0/2173468141 shutdown_connections 2024-09-11T23:18:30.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.270+0000 7fd8f6651740 1 -- 172.21.15.26:0/2173468141 wait complete. 2024-09-11T23:18:30.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.270+0000 7fd8f6651740 20 client.15315 trim_cache size 0 max 16384 2024-09-11T23:18:30.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.288+0000 7fd8f6651740 1 Processor -- start 2024-09-11T23:18:30.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.288+0000 7fd8f6651740 1 -- start start 2024-09-11T23:18:30.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.288+0000 7fd8f6651740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e255aa7b60 0x55e255aa9cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:30.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.288+0000 7fd8f6651740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e255aa8480 0x55e255aca6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:30.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.288+0000 7fd8f6651740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e255ab2d40 0x55e255acac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:30.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.288+0000 7fd8f6651740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd8ec1af950 con 0x55e255aa7b60 2024-09-11T23:18:30.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.288+0000 7fd8f6651740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fd8ec1b1490 con 0x55e255aa8480 2024-09-11T23:18:30.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.288+0000 7fd8f6651740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd8ec1b0020 con 0x55e255ab2d40 2024-09-11T23:18:30.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.288+0000 7fd8f2014640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e255aa7b60 0x55e255aa9cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:30.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.288+0000 7fd8f2014640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e255aa7b60 0x55e255aa9cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47542/0 (socket says 172.21.15.26:47542) 2024-09-11T23:18:30.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.288+0000 7fd8f2014640 1 -- 172.21.15.26:0/3589964853 learned_addr learned my addr 172.21.15.26:0/3589964853 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:18:30.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.288+0000 7fd8f2815640 1 --2- 172.21.15.26:0/3589964853 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e255ab2d40 0x55e255acac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:30.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.288+0000 7fd8f1813640 1 --2- 172.21.15.26:0/3589964853 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e255aa8480 0x55e255aca6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:30.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.288+0000 7fd8f2014640 1 -- 172.21.15.26:0/3589964853 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e255ab2d40 msgr2=0x55e255acac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:18:30.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.288+0000 7fd8f2014640 1 --2- 172.21.15.26:0/3589964853 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e255ab2d40 0x55e255acac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:30.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.288+0000 7fd8f2014640 1 -- 172.21.15.26:0/3589964853 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e255aa8480 msgr2=0x55e255aca6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:18:30.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.288+0000 7fd8f2014640 1 --2- 172.21.15.26:0/3589964853 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e255aa8480 0x55e255aca6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:30.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.288+0000 7fd8f2014640 1 -- 172.21.15.26:0/3589964853 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e255a9f830 con 0x55e255aa7b60 2024-09-11T23:18:30.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.288+0000 7fd8f2815640 1 --2- 172.21.15.26:0/3589964853 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e255ab2d40 0x55e255acac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:18:30.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.289+0000 7fd8f2014640 1 --2- 172.21.15.26:0/3589964853 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e255aa7b60 0x55e255aa9cc0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fd8e8097100 tx=0x7fd8e801d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f21b3093e5bb9142 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:18:30.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.289+0000 7fd8d7fff640 1 -- 172.21.15.26:0/3589964853 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8e8020040 con 0x55e255aa7b60 2024-09-11T23:18:30.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.289+0000 7fd8d7fff640 1 -- 172.21.15.26:0/3589964853 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd8e809b420 con 0x55e255aa7b60 2024-09-11T23:18:30.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.289+0000 7fd8d7fff640 1 -- 172.21.15.26:0/3589964853 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8e80cc9b0 con 0x55e255aa7b60 2024-09-11T23:18:30.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.290+0000 7fd8f6651740 1 -- 172.21.15.26:0/3589964853 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e255aa7b60 msgr2=0x55e255aa9cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:30.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.290+0000 7fd8f6651740 1 --2- 172.21.15.26:0/3589964853 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e255aa7b60 0x55e255aa9cc0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fd8e8097100 tx=0x7fd8e801d040 comp rx=0 tx=0).stop 2024-09-11T23:18:30.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.290+0000 7fd8f6651740 1 -- 172.21.15.26:0/3589964853 shutdown_connections 2024-09-11T23:18:30.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.290+0000 7fd8f6651740 1 --2- 172.21.15.26:0/3589964853 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e255aa8480 0x55e255aca6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:30.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.290+0000 7fd8f6651740 1 --2- 172.21.15.26:0/3589964853 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e255ab2d40 0x55e255acac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:30.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.290+0000 7fd8f6651740 1 --2- 172.21.15.26:0/3589964853 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e255aa7b60 0x55e255aa9cc0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:30.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.290+0000 7fd8f6651740 1 -- 172.21.15.26:0/3589964853 >> 172.21.15.26:0/3589964853 conn(0x55e255aa40f0 msgr2=0x55e255aa95b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:18:30.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.290+0000 7fd8f6651740 1 -- 172.21.15.26:0/3589964853 shutdown_connections 2024-09-11T23:18:30.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.290+0000 7fd8f6651740 1 -- 172.21.15.26:0/3589964853 wait complete. 2024-09-11T23:18:30.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.290+0000 7fd8f6651740 1 -- start start 2024-09-11T23:18:30.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.290+0000 7fd8f6651740 1 Processor -- start 2024-09-11T23:18:30.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.291+0000 7fd8f6651740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e255aa7b60 0x55e255abb3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:30.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.291+0000 7fd8f6651740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e255aa8480 0x55e255ab6760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:30.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.291+0000 7fd8f6651740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e255ab2d40 0x55e255ab6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:30.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.291+0000 7fd8f6651740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd8ec1b0dc0 con 0x55e255aa7b60 2024-09-11T23:18:30.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.291+0000 7fd8f6651740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fd8ec1b06f0 con 0x55e255aa8480 2024-09-11T23:18:30.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.291+0000 7fd8f6651740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd8ec1ad930 con 0x55e255ab2d40 2024-09-11T23:18:30.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.291+0000 7fd8f6651740 10 client.0 _ll_register_callbacks cb 0x7fd8e4009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:18:30.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.291+0000 7fd8f2014640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e255aa7b60 0x55e255abb3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:30.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.291+0000 7fd8f2014640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e255aa7b60 0x55e255abb3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47552/0 (socket says 172.21.15.26:47552) 2024-09-11T23:18:30.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.291+0000 7fd8f2014640 1 -- 172.21.15.26:0/2428101440 learned_addr learned my addr 172.21.15.26:0/2428101440 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:18:30.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.291+0000 7fd8f2815640 1 --2- 172.21.15.26:0/2428101440 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e255ab2d40 0x55e255ab6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:30.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.291+0000 7fd8f1813640 1 --2- 172.21.15.26:0/2428101440 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e255aa8480 0x55e255ab6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:30.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.291+0000 7fd8f2014640 1 -- 172.21.15.26:0/2428101440 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e255ab2d40 msgr2=0x55e255ab6ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:18:30.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.291+0000 7fd8f2014640 1 --2- 172.21.15.26:0/2428101440 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e255ab2d40 0x55e255ab6ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:30.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.291+0000 7fd8f2014640 1 -- 172.21.15.26:0/2428101440 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e255aa8480 msgr2=0x55e255ab6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:18:30.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.291+0000 7fd8f2014640 1 --2- 172.21.15.26:0/2428101440 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e255aa8480 0x55e255ab6760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:30.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.291+0000 7fd8f2014640 1 -- 172.21.15.26:0/2428101440 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e255ab7480 con 0x55e255aa7b60 2024-09-11T23:18:30.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.291+0000 7fd8f2815640 1 --2- 172.21.15.26:0/2428101440 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e255ab2d40 0x55e255ab6ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:18:30.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.291+0000 7fd8f2014640 1 --2- 172.21.15.26:0/2428101440 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e255aa7b60 0x55e255abb3e0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fd8e809b120 tx=0x7fd8e80048a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de11864da8ce30d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:18:30.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.291+0000 7fd8d5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:18:30.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.291+0000 7fd8d5ffb640 1 -- 172.21.15.26:0/2428101440 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8e80b9060 con 0x55e255aa7b60 2024-09-11T23:18:30.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.291+0000 7fd8d5ffb640 1 -- 172.21.15.26:0/2428101440 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fd8e8009cc0 con 0x55e255aa7b60 2024-09-11T23:18:30.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.291+0000 7fd8f6651740 1 -- 172.21.15.26:0/2428101440 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd8e809b420 con 0x55e255aa7b60 2024-09-11T23:18:30.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.292+0000 7fd8d5ffb640 1 -- 172.21.15.26:0/2428101440 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd8e80b3070 con 0x55e255aa7b60 2024-09-11T23:18:30.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.292+0000 7fd8f6651740 10 client.15321 fetch_fsmap learned FSMap version 29 2024-09-11T23:18:30.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.292+0000 7fd8f6651740 1 -- 172.21.15.26:0/2428101440 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e255b2d710 con 0x55e255aa7b60 2024-09-11T23:18:30.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.292+0000 7fd8d5ffb640 1 -- 172.21.15.26:0/2428101440 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd8e809b420 con 0x55e255aa7b60 2024-09-11T23:18:30.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.292+0000 7fd8d5ffb640 1 -- 172.21.15.26:0/2428101440 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fd8e809a2e0 con 0x55e255aa7b60 2024-09-11T23:18:30.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.292+0000 7fd8f6651740 10 client.15321 fetch_fsmap finished waiting for FSMap version 29 2024-09-11T23:18:30.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.292+0000 7fd8f6651740 10 client.15321 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:18:30.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.292+0000 7fd8f6651740 20 client.15321 populate_metadata read hostname 'smithi026' 2024-09-11T23:18:30.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.292+0000 7fd8f6651740 1 --2- 172.21.15.26:0/2428101440 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55e255b2d990 0x55e255b2fd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:30.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.292+0000 7fd8f6651740 4 client.15321 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-11T23:18:30.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.292+0000 7fd8f6651740 1 -- 172.21.15.26:0/2428101440 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55e255991110 con 0x55e255b2d990 2024-09-11T23:18:30.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.293+0000 7fd8f1813640 1 --2- 172.21.15.26:0/2428101440 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55e255b2d990 0x55e255b2fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:30.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.293+0000 7fd8f1813640 1 --2- 172.21.15.26:0/2428101440 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55e255b2d990 0x55e255b2fd80 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd176434a10dec11 server_cookie=6e2072f6f037599b in_seq=0 out_seq=0 2024-09-11T23:18:30.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.293+0000 7fd8d5ffb640 10 client.15321 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:18:30.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.294+0000 7fd8d5ffb640 1 -- 172.21.15.26:0/2428101440 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55e255991110 con 0x55e255b2d990 2024-09-11T23:18:30.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.294+0000 7fd8d5ffb640 10 client.15321 handle_command_reply: tid=0 2024-09-11T23:18:30.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.294+0000 7fd8f6651740 1 client.15321 shutdown 2024-09-11T23:18:30.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.294+0000 7fd8f6651740 1 -- 172.21.15.26:0/2428101440 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e255aa7b60 msgr2=0x55e255abb3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:30.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.294+0000 7fd8f6651740 1 --2- 172.21.15.26:0/2428101440 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e255aa7b60 0x55e255abb3e0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fd8e809b120 tx=0x7fd8e80048a0 comp rx=0 tx=0).stop 2024-09-11T23:18:30.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.294+0000 7fd8f6651740 1 -- 172.21.15.26:0/2428101440 shutdown_connections 2024-09-11T23:18:30.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.294+0000 7fd8f6651740 1 --2- 172.21.15.26:0/2428101440 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e255aa8480 0x55e255ab6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:30.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.294+0000 7fd8f6651740 1 --2- 172.21.15.26:0/2428101440 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55e255b2d990 0x55e255b2fd80 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:30.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.294+0000 7fd8f6651740 1 --2- 172.21.15.26:0/2428101440 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e255ab2d40 0x55e255ab6ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:30.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.294+0000 7fd8d5ffb640 0 client.15321 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:18:30.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.294+0000 7fd8f6651740 1 --2- 172.21.15.26:0/2428101440 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e255aa7b60 0x55e255abb3e0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:30.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.294+0000 7fd8f6651740 1 -- 172.21.15.26:0/2428101440 >> 172.21.15.26:0/2428101440 conn(0x55e255aa40f0 msgr2=0x55e255ab1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:18:30.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.295+0000 7fd8f6651740 1 -- 172.21.15.26:0/2428101440 shutdown_connections 2024-09-11T23:18:30.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.295+0000 7fd8f6651740 1 -- 172.21.15.26:0/2428101440 wait complete. 2024-09-11T23:18:30.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.295+0000 7fd8f6651740 20 client.15321 trim_cache size 0 max 16384 2024-09-11T23:18:30.301 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-11T23:18:30.302 INFO:teuthology.orchestra.run.smithi026.stdout: "return_code": 0, 2024-09-11T23:18:30.303 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_tag": "b9e592f0-d6de-49e1-90fc-1b4b7dd06f2a", 2024-09-11T23:18:30.303 INFO:teuthology.orchestra.run.smithi026.stdout: "mode": "asynchronous" 2024-09-11T23:18:30.303 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-11T23:18:30.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.295+0000 7fd8f4278640 1 -- 172.21.15.26:0/291700581 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fd8d80a2e00 msgr2=0x7fd8d80a52c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:30.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.295+0000 7fd8f4278640 1 --2- 172.21.15.26:0/291700581 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fd8d80a2e00 0x7fd8d80a52c0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fd8e4009a60 tx=0x7fd8e4004a40 comp rx=0 tx=0).stop 2024-09-11T23:18:30.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.295+0000 7fd8f4278640 1 -- 172.21.15.26:0/291700581 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd8ec100e20 msgr2=0x7fd8ec19a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:30.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.295+0000 7fd8f4278640 1 --2- 172.21.15.26:0/291700581 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd8ec100e20 0x7fd8ec19a5f0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fd8e800d770 tx=0x7fd8e800dc30 comp rx=0 tx=0).stop 2024-09-11T23:18:30.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.295+0000 7fd8f4278640 1 -- 172.21.15.26:0/291700581 shutdown_connections 2024-09-11T23:18:30.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.295+0000 7fd8f4278640 1 --2- 172.21.15.26:0/291700581 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fd8d80a2e00 0x7fd8d80a52c0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:30.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.295+0000 7fd8f4278640 1 --2- 172.21.15.26:0/291700581 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd8ec102570 0x7fd8ec19ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:30.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.295+0000 7fd8f4278640 1 --2- 172.21.15.26:0/291700581 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd8ec102ef0 0x7fd8ec19eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:30.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.295+0000 7fd8f4278640 1 --2- 172.21.15.26:0/291700581 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd8ec100e20 0x7fd8ec19a5f0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:30.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.296+0000 7fd8f4278640 1 -- 172.21.15.26:0/291700581 >> 172.21.15.26:0/291700581 conn(0x7fd8ec0fc7d0 msgr2=0x7fd8ec06a790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:18:30.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.296+0000 7fd8f4278640 1 -- 172.21.15.26:0/291700581 shutdown_connections 2024-09-11T23:18:30.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.296+0000 7fd8f4278640 1 -- 172.21.15.26:0/291700581 wait complete. 2024-09-11T23:18:30.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.414+0000 7f9a8868d640 1 -- 172.21.15.26:0/91796576 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7f9a801a73d0 con 0x7f9a80102470 2024-09-11T23:18:30.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.418+0000 7f9a7f7fe640 1 -- 172.21.15.26:0/91796576 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v30)=0 v30) ==== 133+0+0 (secure 0 0 0) 0x7f9a7409be30 con 0x7f9a80102470 2024-09-11T23:18:30.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.421+0000 7f9a8868d640 1 -- 172.21.15.26:0/91796576 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f9a680a2e40 msgr2=0x7f9a680a5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:30.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.421+0000 7f9a8868d640 1 --2- 172.21.15.26:0/91796576 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f9a680a2e40 0x7f9a680a5300 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f9a78000c00 tx=0x7f9a78005a20 comp rx=0 tx=0).stop 2024-09-11T23:18:30.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.421+0000 7f9a8868d640 1 -- 172.21.15.26:0/91796576 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9a80102470 msgr2=0x7f9a80074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:30.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.421+0000 7f9a8868d640 1 --2- 172.21.15.26:0/91796576 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9a80102470 0x7f9a80074a30 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f9a7400e9f0 tx=0x7f9a7400eeb0 comp rx=0 tx=0).stop 2024-09-11T23:18:30.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.421+0000 7f9a8868d640 1 -- 172.21.15.26:0/91796576 shutdown_connections 2024-09-11T23:18:30.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.421+0000 7f9a8868d640 1 --2- 172.21.15.26:0/91796576 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f9a680a2e40 0x7f9a680a5300 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:30.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.421+0000 7f9a8868d640 1 --2- 172.21.15.26:0/91796576 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9a80107b70 0x7f9a80074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:30.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.421+0000 7f9a8868d640 1 --2- 172.21.15.26:0/91796576 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9a80102470 0x7f9a80074a30 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:30.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.421+0000 7f9a8868d640 1 --2- 172.21.15.26:0/91796576 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9a80101ad0 0x7f9a800744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:30.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.421+0000 7f9a8868d640 1 -- 172.21.15.26:0/91796576 >> 172.21.15.26:0/91796576 conn(0x7f9a800fd7d0 msgr2=0x7f9a800fef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:18:30.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.421+0000 7f9a8868d640 1 -- 172.21.15.26:0/91796576 shutdown_connections 2024-09-11T23:18:30.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.421+0000 7f9a8868d640 1 -- 172.21.15.26:0/91796576 wait complete. 2024-09-11T23:18:30.433 DEBUG:teuthology.orchestra.run.smithi026:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-09-11T23:18:30.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.539+0000 7fc9959c4640 1 -- 172.21.15.26:0/1888812719 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc990101390 msgr2=0x7fc99010d8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:30.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.539+0000 7fc9959c4640 1 --2- 172.21.15.26:0/1888812719 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc990101390 0x7fc99010d8c0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fc988009cd0 tx=0x7fc98802a8e0 comp rx=0 tx=0).stop 2024-09-11T23:18:30.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.539+0000 7fc9959c4640 1 -- 172.21.15.26:0/1888812719 shutdown_connections 2024-09-11T23:18:30.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.539+0000 7fc9959c4640 1 --2- 172.21.15.26:0/1888812719 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc990101390 0x7fc99010d8c0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:30.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.539+0000 7fc9959c4640 1 --2- 172.21.15.26:0/1888812719 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc9901009f0 0x7fc990100e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:30.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.539+0000 7fc9959c4640 1 --2- 172.21.15.26:0/1888812719 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc990106a00 0x7fc990106de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:30.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.539+0000 7fc9959c4640 1 -- 172.21.15.26:0/1888812719 >> 172.21.15.26:0/1888812719 conn(0x7fc9900fc6b0 msgr2=0x7fc9900fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:18:30.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.539+0000 7fc9959c4640 1 -- 172.21.15.26:0/1888812719 shutdown_connections 2024-09-11T23:18:30.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.539+0000 7fc9959c4640 1 -- 172.21.15.26:0/1888812719 wait complete. 2024-09-11T23:18:30.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.540+0000 7fc9959c4640 1 Processor -- start 2024-09-11T23:18:30.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.540+0000 7fc9959c4640 1 -- start start 2024-09-11T23:18:30.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.540+0000 7fc9959c4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc9901009f0 0x7fc9900fcc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:30.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.540+0000 7fc9959c4640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc990101390 0x7fc9900fd140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:30.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.540+0000 7fc9959c4640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc990106a00 0x7fc9900fd680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:30.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.540+0000 7fc9959c4640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc990110e40 con 0x7fc9901009f0 2024-09-11T23:18:30.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.540+0000 7fc9959c4640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fc990110cc0 con 0x7fc990101390 2024-09-11T23:18:30.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.540+0000 7fc9959c4640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fc990110fc0 con 0x7fc990106a00 2024-09-11T23:18:30.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.540+0000 7fc98effd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc9901009f0 0x7fc9900fcc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:30.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.540+0000 7fc98effd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc9901009f0 0x7fc9900fcc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47578/0 (socket says 172.21.15.26:47578) 2024-09-11T23:18:30.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.540+0000 7fc98effd640 1 -- 172.21.15.26:0/1229305170 learned_addr learned my addr 172.21.15.26:0/1229305170 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:18:30.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.540+0000 7fc98f7fe640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc990106a00 0x7fc9900fd680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:30.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.540+0000 7fc98e7fc640 1 --2- 172.21.15.26:0/1229305170 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc990101390 0x7fc9900fd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:30.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.540+0000 7fc98effd640 1 -- 172.21.15.26:0/1229305170 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc990106a00 msgr2=0x7fc9900fd680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:30.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.540+0000 7fc98effd640 1 --2- 172.21.15.26:0/1229305170 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc990106a00 0x7fc9900fd680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:30.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.540+0000 7fc98effd640 1 -- 172.21.15.26:0/1229305170 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc990101390 msgr2=0x7fc9900fd140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:30.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.540+0000 7fc98effd640 1 --2- 172.21.15.26:0/1229305170 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc990101390 0x7fc9900fd140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:30.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.540+0000 7fc98effd640 1 -- 172.21.15.26:0/1229305170 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9901a6860 con 0x7fc9901009f0 2024-09-11T23:18:30.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.540+0000 7fc98e7fc640 1 --2- 172.21.15.26:0/1229305170 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc990101390 0x7fc9900fd140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:18:30.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.541+0000 7fc98effd640 1 --2- 172.21.15.26:0/1229305170 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc9901009f0 0x7fc9900fcc00 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fc97800e9f0 tx=0x7fc97800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:18:30.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.541+0000 7fc9949c2640 1 -- 172.21.15.26:0/1229305170 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc97800cd80 con 0x7fc9901009f0 2024-09-11T23:18:30.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.541+0000 7fc9949c2640 1 -- 172.21.15.26:0/1229305170 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc978010910 con 0x7fc9901009f0 2024-09-11T23:18:30.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.541+0000 7fc9949c2640 1 -- 172.21.15.26:0/1229305170 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc978018990 con 0x7fc9901009f0 2024-09-11T23:18:30.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.541+0000 7fc9959c4640 1 -- 172.21.15.26:0/1229305170 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9901a6b50 con 0x7fc9901009f0 2024-09-11T23:18:30.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.541+0000 7fc9959c4640 1 -- 172.21.15.26:0/1229305170 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9901a70a0 con 0x7fc9901009f0 2024-09-11T23:18:30.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.542+0000 7fc9959c4640 1 -- 172.21.15.26:0/1229305170 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc990100e50 con 0x7fc9901009f0 2024-09-11T23:18:30.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.543+0000 7fc9949c2640 1 -- 172.21.15.26:0/1229305170 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 24) ==== 138009+0+0 (secure 0 0 0) 0x7fc978010ac0 con 0x7fc9901009f0 2024-09-11T23:18:30.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.544+0000 7fc9949c2640 1 --2- 172.21.15.26:0/1229305170 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fc97c0a2e40 0x7fc97c0a5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:30.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.544+0000 7fc9949c2640 1 -- 172.21.15.26:0/1229305170 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fc9780cfce0 con 0x7fc9901009f0 2024-09-11T23:18:30.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.544+0000 7fc98e7fc640 1 --2- 172.21.15.26:0/1229305170 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fc97c0a2e40 0x7fc97c0a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:30.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.544+0000 7fc98e7fc640 1 --2- 172.21.15.26:0/1229305170 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fc97c0a2e40 0x7fc97c0a5300 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fc9900fe690 tx=0x7fc980005f90 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:18:30.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.547+0000 7fc9949c2640 1 -- 172.21.15.26:0/1229305170 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc97809c7a0 con 0x7fc9901009f0 2024-09-11T23:18:30.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.747+0000 7fc9959c4640 1 -- 172.21.15.26:0/1229305170 --> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7fc990062630 con 0x7fc97c0a2e40 2024-09-11T23:18:30.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.816+0000 7fc9949c2640 1 -- 172.21.15.26:0/1229305170 <== mgr.15171 v2:172.21.15.26:6800/1989555790 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fc990062630 con 0x7fc97c0a2e40 2024-09-11T23:18:30.817 INFO:teuthology.orchestra.run.smithi026.stdout:Schedule set for path / 2024-09-11T23:18:30.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.818+0000 7fc9959c4640 1 -- 172.21.15.26:0/1229305170 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fc97c0a2e40 msgr2=0x7fc97c0a5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:30.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.818+0000 7fc9959c4640 1 --2- 172.21.15.26:0/1229305170 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fc97c0a2e40 0x7fc97c0a5300 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fc9900fe690 tx=0x7fc980005f90 comp rx=0 tx=0).stop 2024-09-11T23:18:30.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.818+0000 7fc9959c4640 1 -- 172.21.15.26:0/1229305170 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc9901009f0 msgr2=0x7fc9900fcc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:30.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.818+0000 7fc9959c4640 1 --2- 172.21.15.26:0/1229305170 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc9901009f0 0x7fc9900fcc00 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fc97800e9f0 tx=0x7fc97800eeb0 comp rx=0 tx=0).stop 2024-09-11T23:18:30.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.818+0000 7fc9959c4640 1 -- 172.21.15.26:0/1229305170 shutdown_connections 2024-09-11T23:18:30.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.818+0000 7fc9959c4640 1 --2- 172.21.15.26:0/1229305170 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fc97c0a2e40 0x7fc97c0a5300 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:30.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.818+0000 7fc9959c4640 1 --2- 172.21.15.26:0/1229305170 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc990101390 0x7fc9900fd140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:30.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.818+0000 7fc9959c4640 1 --2- 172.21.15.26:0/1229305170 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc990106a00 0x7fc9900fd680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:30.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.818+0000 7fc9959c4640 1 --2- 172.21.15.26:0/1229305170 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc9901009f0 0x7fc9900fcc00 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:30.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.818+0000 7fc9959c4640 1 -- 172.21.15.26:0/1229305170 >> 172.21.15.26:0/1229305170 conn(0x7fc9900fc6b0 msgr2=0x7fc99006a4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:18:30.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.818+0000 7fc9959c4640 1 -- 172.21.15.26:0/1229305170 shutdown_connections 2024-09-11T23:18:30.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.818+0000 7fc9959c4640 1 -- 172.21.15.26:0/1229305170 wait complete. 2024-09-11T23:18:30.830 DEBUG:teuthology.orchestra.run.smithi026:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-09-11T23:18:30.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.937+0000 7f35b4f4b640 1 -- 172.21.15.26:0/2993011154 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f35b010c110 msgr2=0x7f35b010e590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:30.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.937+0000 7f35b4f4b640 1 --2- 172.21.15.26:0/2993011154 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f35b010c110 0x7f35b010e590 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f35a8009ca0 tx=0x7f35a802a950 comp rx=0 tx=0).stop 2024-09-11T23:18:30.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.937+0000 7f35b4f4b640 1 -- 172.21.15.26:0/2993011154 shutdown_connections 2024-09-11T23:18:30.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.937+0000 7f35b4f4b640 1 --2- 172.21.15.26:0/2993011154 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f35b01035e0 0x7f35b010bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:30.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.937+0000 7f35b4f4b640 1 --2- 172.21.15.26:0/2993011154 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f35b010c110 0x7f35b010e590 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:30.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.937+0000 7f35b4f4b640 1 --2- 172.21.15.26:0/2993011154 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f35b0102cc0 0x7f35b01030a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:30.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.937+0000 7f35b4f4b640 1 -- 172.21.15.26:0/2993011154 >> 172.21.15.26:0/2993011154 conn(0x7f35b00fc6b0 msgr2=0x7f35b00fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:18:30.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.937+0000 7f35b4f4b640 1 -- 172.21.15.26:0/2993011154 shutdown_connections 2024-09-11T23:18:30.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.937+0000 7f35b4f4b640 1 -- 172.21.15.26:0/2993011154 wait complete. 2024-09-11T23:18:30.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.937+0000 7f35b4f4b640 1 Processor -- start 2024-09-11T23:18:30.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.937+0000 7f35b4f4b640 1 -- start start 2024-09-11T23:18:30.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.937+0000 7f35b4f4b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f35b0102cc0 0x7f35b0198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:30.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.937+0000 7f35b4f4b640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f35b01035e0 0x7f35b0198840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:30.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.937+0000 7f35b4f4b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f35b010c110 0x7f35b019cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:30.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.937+0000 7f35b4f4b640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f35b0111ef0 con 0x7f35b010c110 2024-09-11T23:18:30.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.937+0000 7f35b4f4b640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f35b0111d70 con 0x7f35b0102cc0 2024-09-11T23:18:30.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.937+0000 7f35b4f4b640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f35b0112070 con 0x7f35b01035e0 2024-09-11T23:18:30.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.937+0000 7f35aed9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f35b010c110 0x7f35b019cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:30.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.937+0000 7f35aed9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f35b010c110 0x7f35b019cbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60496/0 (socket says 172.21.15.26:60496) 2024-09-11T23:18:30.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.937+0000 7f35aed9d640 1 -- 172.21.15.26:0/2281188041 learned_addr learned my addr 172.21.15.26:0/2281188041 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:18:30.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.938+0000 7f35add9b640 1 --2- 172.21.15.26:0/2281188041 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f35b01035e0 0x7f35b0198840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:30.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.938+0000 7f35ae59c640 1 --2- 172.21.15.26:0/2281188041 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f35b0102cc0 0x7f35b0198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:30.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.938+0000 7f35aed9d640 1 -- 172.21.15.26:0/2281188041 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f35b01035e0 msgr2=0x7f35b0198840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:30.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.938+0000 7f35aed9d640 1 --2- 172.21.15.26:0/2281188041 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f35b01035e0 0x7f35b0198840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:30.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.938+0000 7f35aed9d640 1 -- 172.21.15.26:0/2281188041 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f35b0102cc0 msgr2=0x7f35b0198300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:30.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.938+0000 7f35aed9d640 1 --2- 172.21.15.26:0/2281188041 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f35b0102cc0 0x7f35b0198300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:30.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.938+0000 7f35aed9d640 1 -- 172.21.15.26:0/2281188041 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35b019d2f0 con 0x7f35b010c110 2024-09-11T23:18:30.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.938+0000 7f35aed9d640 1 --2- 172.21.15.26:0/2281188041 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f35b010c110 0x7f35b019cbc0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f35a802ae70 tx=0x7f35a802f5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:18:30.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.938+0000 7f359f7fe640 1 -- 172.21.15.26:0/2281188041 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35a8034040 con 0x7f35b010c110 2024-09-11T23:18:30.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.938+0000 7f359f7fe640 1 -- 172.21.15.26:0/2281188041 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35a8030920 con 0x7f35b010c110 2024-09-11T23:18:30.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.938+0000 7f359f7fe640 1 -- 172.21.15.26:0/2281188041 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35a8045480 con 0x7f35b010c110 2024-09-11T23:18:30.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.938+0000 7f35b4f4b640 1 -- 172.21.15.26:0/2281188041 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35b019d580 con 0x7f35b010c110 2024-09-11T23:18:30.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.938+0000 7f35b4f4b640 1 -- 172.21.15.26:0/2281188041 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35b01a4e70 con 0x7f35b010c110 2024-09-11T23:18:30.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.939+0000 7f35b4f4b640 1 -- 172.21.15.26:0/2281188041 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35b006abe0 con 0x7f35b010c110 2024-09-11T23:18:30.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.941+0000 7f359f7fe640 1 -- 172.21.15.26:0/2281188041 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 24) ==== 138009+0+0 (secure 0 0 0) 0x7f35a8037020 con 0x7f35b010c110 2024-09-11T23:18:30.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.941+0000 7f359f7fe640 1 --2- 172.21.15.26:0/2281188041 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f35980a2e40 0x7f35980a5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:30.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.941+0000 7f359f7fe640 1 -- 172.21.15.26:0/2281188041 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f35a80f04f0 con 0x7f35b010c110 2024-09-11T23:18:30.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.941+0000 7f35ae59c640 1 --2- 172.21.15.26:0/2281188041 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f35980a2e40 0x7f35980a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:30.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.942+0000 7f35ae59c640 1 --2- 172.21.15.26:0/2281188041 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f35980a2e40 0x7f35980a5300 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f35a0005fd0 tx=0x7f35a0005850 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:18:30.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:30.945+0000 7f359f7fe640 1 -- 172.21.15.26:0/2281188041 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f35a80bd0c0 con 0x7f35b010c110 2024-09-11T23:18:31.047 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:30 smithi026 ceph-mon[31172]: from='client.? 172.21.15.26:0/1205641655' entity='client.admin' 2024-09-11T23:18:31.047 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:30 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:18:31.047 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:30 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:18:31.047 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:30 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:18:31.047 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:30 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:18:31.047 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:30 smithi026 ceph-mon[31172]: from='client.? 172.21.15.26:0/2896367954' entity='client.admin' 2024-09-11T23:18:31.048 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:30 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:18:31.048 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:30 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:18:31.048 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:30 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:18:31.048 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:30 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:18:31.048 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:30 smithi026 ceph-mon[31172]: from='client.? 172.21.15.26:0/91796576' entity='client.admin' 2024-09-11T23:18:31.048 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:30 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:18:31.048 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:30 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:18:31.048 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:30 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:18:31.049 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:30 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:18:31.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:30 smithi076 ceph-mon[35296]: from='client.? 172.21.15.26:0/1205641655' entity='client.admin' 2024-09-11T23:18:31.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:30 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:18:31.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:30 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:18:31.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:30 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:18:31.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:30 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:18:31.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:30 smithi076 ceph-mon[35296]: from='client.? 172.21.15.26:0/2896367954' entity='client.admin' 2024-09-11T23:18:31.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:30 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:18:31.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:30 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:18:31.054 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:30 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:18:31.054 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:30 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:18:31.054 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:30 smithi076 ceph-mon[35296]: from='client.? 172.21.15.26:0/91796576' entity='client.admin' 2024-09-11T23:18:31.054 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:30 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:18:31.054 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:30 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:18:31.054 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:30 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:18:31.054 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:30 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:18:31.074 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T23:18:31.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.146+0000 7f35b4f4b640 1 -- 172.21.15.26:0/2281188041 --> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) -- 0x7f35b00fea70 con 0x7f35980a2e40 2024-09-11T23:18:31.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.213+0000 7f359f7fe640 1 -- 172.21.15.26:0/2281188041 <== mgr.15171 v2:172.21.15.26:6800/1989555790 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+26 (secure 0 0 0) 0x7f35b00fea70 con 0x7f35980a2e40 2024-09-11T23:18:31.214 INFO:teuthology.orchestra.run.smithi026.stdout:Retention added to path / 2024-09-11T23:18:31.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.215+0000 7f35b4f4b640 1 -- 172.21.15.26:0/2281188041 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f35980a2e40 msgr2=0x7f35980a5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:31.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.215+0000 7f35b4f4b640 1 --2- 172.21.15.26:0/2281188041 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f35980a2e40 0x7f35980a5300 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f35a0005fd0 tx=0x7f35a0005850 comp rx=0 tx=0).stop 2024-09-11T23:18:31.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.215+0000 7f35b4f4b640 1 -- 172.21.15.26:0/2281188041 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f35b010c110 msgr2=0x7f35b019cbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:31.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.215+0000 7f35b4f4b640 1 --2- 172.21.15.26:0/2281188041 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f35b010c110 0x7f35b019cbc0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f35a802ae70 tx=0x7f35a802f5b0 comp rx=0 tx=0).stop 2024-09-11T23:18:31.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.215+0000 7f35b4f4b640 1 -- 172.21.15.26:0/2281188041 shutdown_connections 2024-09-11T23:18:31.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.215+0000 7f35b4f4b640 1 --2- 172.21.15.26:0/2281188041 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f35980a2e40 0x7f35980a5300 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:31.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.215+0000 7f35b4f4b640 1 --2- 172.21.15.26:0/2281188041 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f35b010c110 0x7f35b019cbc0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:31.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.215+0000 7f35b4f4b640 1 --2- 172.21.15.26:0/2281188041 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f35b01035e0 0x7f35b0198840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:31.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.215+0000 7f35b4f4b640 1 --2- 172.21.15.26:0/2281188041 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f35b0102cc0 0x7f35b0198300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:31.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.215+0000 7f35b4f4b640 1 -- 172.21.15.26:0/2281188041 >> 172.21.15.26:0/2281188041 conn(0x7f35b00fc6b0 msgr2=0x7f35b00fd2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:18:31.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.216+0000 7f35b4f4b640 1 -- 172.21.15.26:0/2281188041 shutdown_connections 2024-09-11T23:18:31.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.216+0000 7f35b4f4b640 1 -- 172.21.15.26:0/2281188041 wait complete. 2024-09-11T23:18:31.227 DEBUG:teuthology.orchestra.run.smithi026:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-09-11T23:18:31.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:30 smithi136 ceph-mon[35121]: from='client.? 172.21.15.26:0/1205641655' entity='client.admin' 2024-09-11T23:18:31.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:30 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:18:31.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:30 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:18:31.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:30 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:18:31.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:30 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:18:31.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:30 smithi136 ceph-mon[35121]: from='client.? 172.21.15.26:0/2896367954' entity='client.admin' 2024-09-11T23:18:31.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:30 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:18:31.240 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:30 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:18:31.240 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:30 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:18:31.240 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:30 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:18:31.240 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:30 smithi136 ceph-mon[35121]: from='client.? 172.21.15.26:0/91796576' entity='client.admin' 2024-09-11T23:18:31.240 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:30 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:18:31.240 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:30 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:18:31.240 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:30 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:18:31.240 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:30 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:18:31.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.336+0000 7f2aa365c640 1 -- 172.21.15.26:0/1900608057 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a9c074eb0 msgr2=0x7f2a9c1125a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:31.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.336+0000 7f2aa365c640 1 --2- 172.21.15.26:0/1900608057 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a9c074eb0 0x7f2a9c1125a0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f2a8c009900 tx=0x7f2a8c02a480 comp rx=0 tx=0).stop 2024-09-11T23:18:31.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.336+0000 7f2aa365c640 1 -- 172.21.15.26:0/1900608057 shutdown_connections 2024-09-11T23:18:31.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.336+0000 7f2aa365c640 1 --2- 172.21.15.26:0/1900608057 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2a9c112ae0 0x7f2a9c114ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:31.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.336+0000 7f2aa365c640 1 --2- 172.21.15.26:0/1900608057 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a9c074eb0 0x7f2a9c1125a0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:31.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.336+0000 7f2aa365c640 1 --2- 172.21.15.26:0/1900608057 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2a9c074590 0x7f2a9c074970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:31.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.336+0000 7f2aa365c640 1 -- 172.21.15.26:0/1900608057 >> 172.21.15.26:0/1900608057 conn(0x7f2a9c0fd7d0 msgr2=0x7f2a9c0ffbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:18:31.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.337+0000 7f2aa365c640 1 -- 172.21.15.26:0/1900608057 shutdown_connections 2024-09-11T23:18:31.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.337+0000 7f2aa365c640 1 -- 172.21.15.26:0/1900608057 wait complete. 2024-09-11T23:18:31.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.337+0000 7f2aa365c640 1 Processor -- start 2024-09-11T23:18:31.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.337+0000 7f2aa365c640 1 -- start start 2024-09-11T23:18:31.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.337+0000 7f2aa365c640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2a9c074590 0x7f2a9c1a4680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:31.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.338+0000 7f2aa365c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2a9c074eb0 0x7f2a9c1a4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:31.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.338+0000 7f2aa365c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a9c112ae0 0x7f2a9c19e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:31.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.338+0000 7f2aa365c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f2a9c118490 con 0x7f2a9c112ae0 2024-09-11T23:18:31.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.338+0000 7f2aa365c640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f2a9c118310 con 0x7f2a9c074eb0 2024-09-11T23:18:31.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.338+0000 7f2aa365c640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f2a9c118610 con 0x7f2a9c074590 2024-09-11T23:18:31.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.338+0000 7f2aa1bf9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a9c112ae0 0x7f2a9c19e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:31.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.338+0000 7f2aa1bf9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a9c112ae0 0x7f2a9c19e740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60512/0 (socket says 172.21.15.26:60512) 2024-09-11T23:18:31.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.338+0000 7f2aa1bf9640 1 -- 172.21.15.26:0/1594139706 learned_addr learned my addr 172.21.15.26:0/1594139706 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:18:31.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.338+0000 7f2aa0bf7640 1 --2- 172.21.15.26:0/1594139706 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2a9c074eb0 0x7f2a9c1a4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:31.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.338+0000 7f2aa13f8640 1 --2- 172.21.15.26:0/1594139706 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2a9c074590 0x7f2a9c1a4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:31.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.338+0000 7f2aa1bf9640 1 -- 172.21.15.26:0/1594139706 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2a9c074590 msgr2=0x7f2a9c1a4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:31.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.338+0000 7f2aa1bf9640 1 --2- 172.21.15.26:0/1594139706 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2a9c074590 0x7f2a9c1a4680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:31.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.338+0000 7f2aa1bf9640 1 -- 172.21.15.26:0/1594139706 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2a9c074eb0 msgr2=0x7f2a9c1a4bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:31.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.338+0000 7f2aa1bf9640 1 --2- 172.21.15.26:0/1594139706 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2a9c074eb0 0x7f2a9c1a4bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:31.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.338+0000 7f2aa1bf9640 1 -- 172.21.15.26:0/1594139706 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a9c19f000 con 0x7f2a9c112ae0 2024-09-11T23:18:31.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.339+0000 7f2aa1bf9640 1 --2- 172.21.15.26:0/1594139706 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a9c112ae0 0x7f2a9c19e740 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f2a9400eaa0 tx=0x7f2a9400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:18:31.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.339+0000 7f2a927fc640 1 -- 172.21.15.26:0/1594139706 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a9400cdb0 con 0x7f2a9c112ae0 2024-09-11T23:18:31.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.339+0000 7f2a927fc640 1 -- 172.21.15.26:0/1594139706 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a94005020 con 0x7f2a9c112ae0 2024-09-11T23:18:31.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.339+0000 7f2aa365c640 1 -- 172.21.15.26:0/1594139706 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a9c19f2f0 con 0x7f2a9c112ae0 2024-09-11T23:18:31.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.339+0000 7f2a927fc640 1 -- 172.21.15.26:0/1594139706 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a940116b0 con 0x7f2a9c112ae0 2024-09-11T23:18:31.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.339+0000 7f2aa365c640 1 -- 172.21.15.26:0/1594139706 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a9c1ab600 con 0x7f2a9c112ae0 2024-09-11T23:18:31.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.340+0000 7f2aa365c640 1 -- 172.21.15.26:0/1594139706 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a9c19f630 con 0x7f2a9c112ae0 2024-09-11T23:18:31.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.341+0000 7f2a927fc640 1 -- 172.21.15.26:0/1594139706 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 24) ==== 138009+0+0 (secure 0 0 0) 0x7f2a94004b20 con 0x7f2a9c112ae0 2024-09-11T23:18:31.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.343+0000 7f2a927fc640 1 --2- 172.21.15.26:0/1594139706 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f2a880a2ec0 0x7f2a880a5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:31.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.343+0000 7f2a927fc640 1 -- 172.21.15.26:0/1594139706 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f2a940cf710 con 0x7f2a9c112ae0 2024-09-11T23:18:31.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.343+0000 7f2aa13f8640 1 --2- 172.21.15.26:0/1594139706 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f2a880a2ec0 0x7f2a880a5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:31.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.343+0000 7f2aa13f8640 1 --2- 172.21.15.26:0/1594139706 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f2a880a2ec0 0x7f2a880a5380 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f2a98005d00 tx=0x7f2a9800b410 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:18:31.345 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:18:31.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.347+0000 7f2a927fc640 1 -- 172.21.15.26:0/1594139706 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2a94016030 con 0x7f2a9c112ae0 2024-09-11T23:18:31.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.555+0000 7f2aa365c640 1 -- 172.21.15.26:0/1594139706 --> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f2a9c0ff470 con 0x7f2a880a2ec0 2024-09-11T23:18:31.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.562+0000 7f2a927fc640 1 -- 172.21.15.26:0/1594139706 <== mgr.15171 v2:172.21.15.26:6800/1989555790 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+298 (secure 0 0 0) 0x7f2a9c0ff470 con 0x7f2a880a2ec0 2024-09-11T23:18:31.563 INFO:teuthology.orchestra.run.smithi026.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-09-11T00:00:00", "created": "2024-09-11T23:18:30", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-09-11T23:18:31.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.564+0000 7f2aa365c640 1 -- 172.21.15.26:0/1594139706 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f2a880a2ec0 msgr2=0x7f2a880a5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:31.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.564+0000 7f2aa365c640 1 --2- 172.21.15.26:0/1594139706 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f2a880a2ec0 0x7f2a880a5380 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f2a98005d00 tx=0x7f2a9800b410 comp rx=0 tx=0).stop 2024-09-11T23:18:31.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.564+0000 7f2aa365c640 1 -- 172.21.15.26:0/1594139706 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a9c112ae0 msgr2=0x7f2a9c19e740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:31.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.564+0000 7f2aa365c640 1 --2- 172.21.15.26:0/1594139706 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a9c112ae0 0x7f2a9c19e740 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f2a9400eaa0 tx=0x7f2a9400ef60 comp rx=0 tx=0).stop 2024-09-11T23:18:31.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.564+0000 7f2aa365c640 1 -- 172.21.15.26:0/1594139706 shutdown_connections 2024-09-11T23:18:31.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.564+0000 7f2aa365c640 1 --2- 172.21.15.26:0/1594139706 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f2a880a2ec0 0x7f2a880a5380 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:31.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.564+0000 7f2aa365c640 1 --2- 172.21.15.26:0/1594139706 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2a9c074eb0 0x7f2a9c1a4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:31.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.564+0000 7f2aa365c640 1 --2- 172.21.15.26:0/1594139706 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a9c112ae0 0x7f2a9c19e740 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:31.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.564+0000 7f2aa365c640 1 --2- 172.21.15.26:0/1594139706 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2a9c074590 0x7f2a9c1a4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:31.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.564+0000 7f2aa365c640 1 -- 172.21.15.26:0/1594139706 >> 172.21.15.26:0/1594139706 conn(0x7f2a9c0fd7d0 msgr2=0x7f2a9c0fed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:18:31.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.565+0000 7f2aa365c640 1 -- 172.21.15.26:0/1594139706 shutdown_connections 2024-09-11T23:18:31.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.565+0000 7f2aa365c640 1 -- 172.21.15.26:0/1594139706 wait complete. 2024-09-11T23:18:31.577 DEBUG:teuthology.orchestra.run.smithi026:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-09-11T23:18:31.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.716+0000 7fa359c27640 1 -- 172.21.15.26:0/2879220494 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa354103640 msgr2=0x7fa35410bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:31.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.716+0000 7fa359c27640 1 --2- 172.21.15.26:0/2879220494 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa354103640 0x7fa35410bb20 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fa348009900 tx=0x7fa34802a480 comp rx=0 tx=0).stop 2024-09-11T23:18:31.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.716+0000 7fa359c27640 1 -- 172.21.15.26:0/2879220494 shutdown_connections 2024-09-11T23:18:31.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.716+0000 7fa359c27640 1 --2- 172.21.15.26:0/2879220494 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa35410c0f0 0x7fa35410e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:31.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.716+0000 7fa359c27640 1 --2- 172.21.15.26:0/2879220494 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa354103640 0x7fa35410bb20 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:31.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.716+0000 7fa359c27640 1 --2- 172.21.15.26:0/2879220494 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa354102d20 0x7fa354103100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:31.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.716+0000 7fa359c27640 1 -- 172.21.15.26:0/2879220494 >> 172.21.15.26:0/2879220494 conn(0x7fa354077990 msgr2=0x7fa3540fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:18:31.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.716+0000 7fa359c27640 1 -- 172.21.15.26:0/2879220494 shutdown_connections 2024-09-11T23:18:31.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.716+0000 7fa359c27640 1 -- 172.21.15.26:0/2879220494 wait complete. 2024-09-11T23:18:31.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.716+0000 7fa359c27640 1 Processor -- start 2024-09-11T23:18:31.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.716+0000 7fa359c27640 1 -- start start 2024-09-11T23:18:31.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.717+0000 7fa359c27640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa354102d20 0x7fa354198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:31.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.717+0000 7fa359c27640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa354103640 0x7fa3541988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:31.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.717+0000 7fa359c27640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa35410c0f0 0x7fa35419cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:31.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.717+0000 7fa359c27640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fa354111f10 con 0x7fa35410c0f0 2024-09-11T23:18:31.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.717+0000 7fa359c27640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fa354111d90 con 0x7fa354103640 2024-09-11T23:18:31.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.717+0000 7fa359c27640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fa354112090 con 0x7fa354102d20 2024-09-11T23:18:31.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.717+0000 7fa353fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa35410c0f0 0x7fa35419cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:31.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.717+0000 7fa3537fe640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa354102d20 0x7fa354198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:31.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.717+0000 7fa3537fe640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa354102d20 0x7fa354198380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:50450/0 (socket says 172.21.15.26:50450) 2024-09-11T23:18:31.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.717+0000 7fa3537fe640 1 -- 172.21.15.26:0/3497051571 learned_addr learned my addr 172.21.15.26:0/3497051571 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:18:31.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.717+0000 7fa352ffd640 1 --2- 172.21.15.26:0/3497051571 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa354103640 0x7fa3541988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:31.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.717+0000 7fa353fff640 1 -- 172.21.15.26:0/3497051571 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa354102d20 msgr2=0x7fa354198380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:31.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.717+0000 7fa353fff640 1 --2- 172.21.15.26:0/3497051571 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa354102d20 0x7fa354198380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:31.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.717+0000 7fa353fff640 1 -- 172.21.15.26:0/3497051571 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa354103640 msgr2=0x7fa3541988c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:31.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.717+0000 7fa353fff640 1 --2- 172.21.15.26:0/3497051571 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa354103640 0x7fa3541988c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:31.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.717+0000 7fa353fff640 1 -- 172.21.15.26:0/3497051571 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa35419d370 con 0x7fa35410c0f0 2024-09-11T23:18:31.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.717+0000 7fa3537fe640 1 --2- 172.21.15.26:0/3497051571 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa354102d20 0x7fa354198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:18:31.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.717+0000 7fa352ffd640 1 --2- 172.21.15.26:0/3497051571 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa354103640 0x7fa3541988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:18:31.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.717+0000 7fa353fff640 1 --2- 172.21.15.26:0/3497051571 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa35410c0f0 0x7fa35419cc40 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fa34c00e940 tx=0x7fa34c00ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:18:31.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.717+0000 7fa350ff9640 1 -- 172.21.15.26:0/3497051571 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa34c004d20 con 0x7fa35410c0f0 2024-09-11T23:18:31.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.718+0000 7fa350ff9640 1 -- 172.21.15.26:0/3497051571 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa34c005020 con 0x7fa35410c0f0 2024-09-11T23:18:31.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.718+0000 7fa359c27640 1 -- 172.21.15.26:0/3497051571 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa35419d660 con 0x7fa35410c0f0 2024-09-11T23:18:31.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.718+0000 7fa350ff9640 1 -- 172.21.15.26:0/3497051571 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa34c0116b0 con 0x7fa35410c0f0 2024-09-11T23:18:31.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.718+0000 7fa359c27640 1 -- 172.21.15.26:0/3497051571 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3541a4ef0 con 0x7fa35410c0f0 2024-09-11T23:18:31.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.719+0000 7fa359c27640 1 -- 172.21.15.26:0/3497051571 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa354103100 con 0x7fa35410c0f0 2024-09-11T23:18:31.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.720+0000 7fa350ff9640 1 -- 172.21.15.26:0/3497051571 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 24) ==== 138009+0+0 (secure 0 0 0) 0x7fa34c0054d0 con 0x7fa35410c0f0 2024-09-11T23:18:31.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.721+0000 7fa350ff9640 1 --2- 172.21.15.26:0/3497051571 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fa33c0a2e40 0x7fa33c0a5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:31.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.721+0000 7fa3537fe640 1 --2- 172.21.15.26:0/3497051571 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fa33c0a2e40 0x7fa33c0a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:31.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.721+0000 7fa350ff9640 1 -- 172.21.15.26:0/3497051571 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fa34c0cf9d0 con 0x7fa35410c0f0 2024-09-11T23:18:31.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.721+0000 7fa3537fe640 1 --2- 172.21.15.26:0/3497051571 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fa33c0a2e40 0x7fa33c0a5300 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fa344011550 tx=0x7fa344005f90 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:18:31.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.724+0000 7fa350ff9640 1 -- 172.21.15.26:0/3497051571 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa34c016030 con 0x7fa35410c0f0 2024-09-11T23:18:31.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.822+0000 7efc08b15640 1 -- 172.21.15.26:0/24968971 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc04100e20 msgr2=0x7efc04101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:31.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.822+0000 7efc08b15640 1 --2- 172.21.15.26:0/24968971 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc04100e20 0x7efc04101200 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7efbf4009900 tx=0x7efbf402a480 comp rx=0 tx=0).stop 2024-09-11T23:18:31.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.822+0000 7efc08b15640 1 -- 172.21.15.26:0/24968971 shutdown_connections 2024-09-11T23:18:31.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.822+0000 7efc08b15640 1 --2- 172.21.15.26:0/24968971 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efc04102570 0x7efc041029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:31.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.822+0000 7efc08b15640 1 --2- 172.21.15.26:0/24968971 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efc04102ef0 0x7efc0410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:31.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.822+0000 7efc08b15640 1 --2- 172.21.15.26:0/24968971 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc04100e20 0x7efc04101200 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:31.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.822+0000 7efc08b15640 1 -- 172.21.15.26:0/24968971 >> 172.21.15.26:0/24968971 conn(0x7efc040fc7d0 msgr2=0x7efc040febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:18:31.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.822+0000 7efc08b15640 1 -- 172.21.15.26:0/24968971 shutdown_connections 2024-09-11T23:18:31.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.822+0000 7efc08b15640 1 -- 172.21.15.26:0/24968971 wait complete. 2024-09-11T23:18:31.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.823+0000 7efc08b15640 1 Processor -- start 2024-09-11T23:18:31.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.823+0000 7efc08b15640 1 -- start start 2024-09-11T23:18:31.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.823+0000 7efc08b15640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efc04100e20 0x7efc041992d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:31.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.823+0000 7efc08b15640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efc04102570 0x7efc04199810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:31.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.823+0000 7efc08b15640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc04102ef0 0x7efc041a4350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:31.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.823+0000 7efc08b15640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7efc04110f80 con 0x7efc04102ef0 2024-09-11T23:18:31.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.823+0000 7efc08b15640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7efc04110e00 con 0x7efc04100e20 2024-09-11T23:18:31.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.823+0000 7efc08b15640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7efc04111100 con 0x7efc04102570 2024-09-11T23:18:31.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.823+0000 7efc02d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc04102ef0 0x7efc041a4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:31.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.823+0000 7efc02d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc04102ef0 0x7efc041a4350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60542/0 (socket says 172.21.15.26:60542) 2024-09-11T23:18:31.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.823+0000 7efc02d9d640 1 -- 172.21.15.26:0/3434485275 learned_addr learned my addr 172.21.15.26:0/3434485275 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:18:31.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.823+0000 7efc0259c640 1 --2- 172.21.15.26:0/3434485275 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efc04100e20 0x7efc041992d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:31.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.823+0000 7efc01d9b640 1 --2- 172.21.15.26:0/3434485275 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efc04102570 0x7efc04199810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:31.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.823+0000 7efc02d9d640 1 -- 172.21.15.26:0/3434485275 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efc04102570 msgr2=0x7efc04199810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:31.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.823+0000 7efc02d9d640 1 --2- 172.21.15.26:0/3434485275 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efc04102570 0x7efc04199810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:31.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.823+0000 7efc02d9d640 1 -- 172.21.15.26:0/3434485275 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efc04100e20 msgr2=0x7efc041992d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:31.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.823+0000 7efc02d9d640 1 --2- 172.21.15.26:0/3434485275 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efc04100e20 0x7efc041992d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:31.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.823+0000 7efc02d9d640 1 -- 172.21.15.26:0/3434485275 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc041a4890 con 0x7efc04102ef0 2024-09-11T23:18:31.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.823+0000 7efc01d9b640 1 --2- 172.21.15.26:0/3434485275 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efc04102570 0x7efc04199810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:18:31.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.823+0000 7efc0259c640 1 --2- 172.21.15.26:0/3434485275 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efc04100e20 0x7efc041992d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:18:31.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.824+0000 7efc02d9d640 1 --2- 172.21.15.26:0/3434485275 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc04102ef0 0x7efc041a4350 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7efbfc00cce0 tx=0x7efbfc00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:18:31.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.824+0000 7efbf37fe640 1 -- 172.21.15.26:0/3434485275 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbfc00f450 con 0x7efc04102ef0 2024-09-11T23:18:31.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.824+0000 7efbf37fe640 1 -- 172.21.15.26:0/3434485275 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efbfc00fad0 con 0x7efc04102ef0 2024-09-11T23:18:31.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.824+0000 7efc08b15640 1 -- 172.21.15.26:0/3434485275 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc041a4a60 con 0x7efc04102ef0 2024-09-11T23:18:31.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.824+0000 7efbf37fe640 1 -- 172.21.15.26:0/3434485275 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbfc018c10 con 0x7efc04102ef0 2024-09-11T23:18:31.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.824+0000 7efc08b15640 1 -- 172.21.15.26:0/3434485275 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc041a4f20 con 0x7efc04102ef0 2024-09-11T23:18:31.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.826+0000 7efbf37fe640 1 -- 172.21.15.26:0/3434485275 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 24) ==== 138009+0+0 (secure 0 0 0) 0x7efbfc00f5f0 con 0x7efc04102ef0 2024-09-11T23:18:31.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.826+0000 7efc0aeee740 1 Processor -- start 2024-09-11T23:18:31.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.827+0000 7efc0aeee740 1 -- start start 2024-09-11T23:18:31.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.827+0000 7efc0aeee740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f3758394a0 0x55f375839880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:31.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.827+0000 7efc0aeee740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f375839dc0 0x55f375844140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:31.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.827+0000 7efc0aeee740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f375844680 0x55f375846a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:31.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.827+0000 7efc0aeee740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55f37579f2e0 con 0x55f375839dc0 2024-09-11T23:18:31.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.827+0000 7efc0aeee740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55f375746ce0 con 0x55f375844680 2024-09-11T23:18:31.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.827+0000 7efc0aeee740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55f37579b2a0 con 0x55f3758394a0 2024-09-11T23:18:31.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.827+0000 7efbf37fe640 1 --2- 172.21.15.26:0/3434485275 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7efbec0a2d80 0x7efbec0a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:31.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.827+0000 7efc0259c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f375839dc0 0x55f375844140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:31.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.827+0000 7efc0259c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f375839dc0 0x55f375844140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60546/0 (socket says 172.21.15.26:60546) 2024-09-11T23:18:31.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.827+0000 7efc0259c640 1 -- 172.21.15.26:0/4079471353 learned_addr learned my addr 172.21.15.26:0/4079471353 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:18:31.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.827+0000 7efc02d9d640 1 --2- 172.21.15.26:0/3434485275 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7efbec0a2d80 0x7efbec0a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:31.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.827+0000 7efc01d9b640 1 --2- 172.21.15.26:0/4079471353 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f3758394a0 0x55f375839880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:31.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.827+0000 7efbf37fe640 1 -- 172.21.15.26:0/3434485275 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7efbfc0cfd30 con 0x7efc04102ef0 2024-09-11T23:18:31.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.827+0000 7efc01d9b640 1 --2- 172.21.15.26:0/4079471353 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f375844680 0x55f375846a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:31.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.827+0000 7efc0259c640 1 -- 172.21.15.26:0/4079471353 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f3758394a0 msgr2=0x55f375839880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:18:31.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.827+0000 7efc0259c640 1 --2- 172.21.15.26:0/4079471353 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f3758394a0 0x55f375839880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:31.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.827+0000 7efc0259c640 1 -- 172.21.15.26:0/4079471353 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f375844680 msgr2=0x55f375846a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:18:31.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.827+0000 7efc0259c640 1 --2- 172.21.15.26:0/4079471353 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f375844680 0x55f375846a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:31.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.827+0000 7efc0259c640 1 -- 172.21.15.26:0/4079471353 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f37572b440 con 0x55f375839dc0 2024-09-11T23:18:31.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.828+0000 7efc0259c640 1 --2- 172.21.15.26:0/4079471353 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f375839dc0 0x55f375844140 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7efbf40314e0 tx=0x7efbf4004200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbf21ee3f9cf0c01 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:18:31.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.828+0000 7efc02d9d640 1 --2- 172.21.15.26:0/3434485275 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7efbec0a2d80 0x7efbec0a5240 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7efbfc0cd430 tx=0x7efbfc00af30 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:18:31.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.828+0000 7efbf0ff9640 1 -- 172.21.15.26:0/4079471353 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbf4035030 con 0x55f375839dc0 2024-09-11T23:18:31.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.828+0000 7efbf0ff9640 1 -- 172.21.15.26:0/4079471353 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efbf4005520 con 0x55f375839dc0 2024-09-11T23:18:31.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.828+0000 7efbf0ff9640 1 -- 172.21.15.26:0/4079471353 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbf4003850 con 0x55f375839dc0 2024-09-11T23:18:31.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.829+0000 7efc0aeee740 1 -- 172.21.15.26:0/4079471353 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f375839dc0 msgr2=0x55f375844140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:31.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.829+0000 7efc0aeee740 1 --2- 172.21.15.26:0/4079471353 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f375839dc0 0x55f375844140 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7efbf40314e0 tx=0x7efbf4004200 comp rx=0 tx=0).stop 2024-09-11T23:18:31.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.829+0000 7efc0aeee740 1 -- 172.21.15.26:0/4079471353 shutdown_connections 2024-09-11T23:18:31.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.829+0000 7efc0aeee740 1 --2- 172.21.15.26:0/4079471353 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f375844680 0x55f375846a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:31.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.829+0000 7efc0aeee740 1 --2- 172.21.15.26:0/4079471353 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f375839dc0 0x55f375844140 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:31.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.829+0000 7efc0aeee740 1 --2- 172.21.15.26:0/4079471353 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f3758394a0 0x55f375839880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:31.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.829+0000 7efc0aeee740 1 -- 172.21.15.26:0/4079471353 >> 172.21.15.26:0/4079471353 conn(0x55f375835a30 msgr2=0x55f375835e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:18:31.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.829+0000 7efc0aeee740 1 -- 172.21.15.26:0/4079471353 shutdown_connections 2024-09-11T23:18:31.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.829+0000 7efc0aeee740 1 -- 172.21.15.26:0/4079471353 wait complete. 2024-09-11T23:18:31.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.829+0000 7efc0aeee740 1 -- start start 2024-09-11T23:18:31.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.829+0000 7efc0aeee740 1 Processor -- start 2024-09-11T23:18:31.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.829+0000 7efc0aeee740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f3758394a0 0x55f375843f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:31.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.829+0000 7efc0aeee740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f375839dc0 0x55f3758411b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:31.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.829+0000 7efc0aeee740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f375844680 0x55f3758416f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:31.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.829+0000 7efc0aeee740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55f375847250 con 0x55f3758394a0 2024-09-11T23:18:31.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.829+0000 7efc0aeee740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55f37575a050 con 0x55f375839dc0 2024-09-11T23:18:31.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.829+0000 7efc0aeee740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55f3758473d0 con 0x55f375844680 2024-09-11T23:18:31.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.830+0000 7efc0aeee740 10 client.0 _ll_register_callbacks cb 0x55f37579a160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:18:31.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.830+0000 7efc01d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f3758394a0 0x55f375843f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:31.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.830+0000 7efc01d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f3758394a0 0x55f375843f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60552/0 (socket says 172.21.15.26:60552) 2024-09-11T23:18:31.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.830+0000 7efc01d9b640 1 -- 172.21.15.26:0/99969317 learned_addr learned my addr 172.21.15.26:0/99969317 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:18:31.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.830+0000 7efc01d9b640 1 --2- 172.21.15.26:0/99969317 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f375844680 0x55f3758416f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:31.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.830+0000 7efc0259c640 1 --2- 172.21.15.26:0/99969317 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f375839dc0 0x55f3758411b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:31.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.830+0000 7efc01d9b640 1 -- 172.21.15.26:0/99969317 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f375844680 msgr2=0x55f3758416f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:18:31.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.830+0000 7efc01d9b640 1 --2- 172.21.15.26:0/99969317 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f375844680 0x55f3758416f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:31.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.830+0000 7efc01d9b640 1 -- 172.21.15.26:0/99969317 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f375839dc0 msgr2=0x55f3758411b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:18:31.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.830+0000 7efc01d9b640 1 --2- 172.21.15.26:0/99969317 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f375839dc0 0x55f3758411b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:31.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.830+0000 7efc01d9b640 1 -- 172.21.15.26:0/99969317 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3755103c0 con 0x55f3758394a0 2024-09-11T23:18:31.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.830+0000 7efc01d9b640 1 --2- 172.21.15.26:0/99969317 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f3758394a0 0x55f375843f00 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7efbf8013cd0 tx=0x7efbf8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=314482304c6d295b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:18:31.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.830+0000 7efbe9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:18:31.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.830+0000 7efbe9ffb640 1 -- 172.21.15.26:0/99969317 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbf8004220 con 0x55f3758394a0 2024-09-11T23:18:31.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.830+0000 7efbe9ffb640 1 -- 172.21.15.26:0/99969317 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7efbf80043c0 con 0x55f3758394a0 2024-09-11T23:18:31.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.830+0000 7efc0aeee740 1 -- 172.21.15.26:0/99969317 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efbf4005520 con 0x55f3758394a0 2024-09-11T23:18:31.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.830+0000 7efbe9ffb640 1 -- 172.21.15.26:0/99969317 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7efbf8005020 con 0x55f3758394a0 2024-09-11T23:18:31.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.831+0000 7efc0aeee740 10 client.15357 fetch_fsmap learned FSMap version 29 2024-09-11T23:18:31.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.831+0000 7efc0aeee740 1 -- 172.21.15.26:0/99969317 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f375852bf0 con 0x55f3758394a0 2024-09-11T23:18:31.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.831+0000 7efbe9ffb640 1 -- 172.21.15.26:0/99969317 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efbf4005520 con 0x55f3758394a0 2024-09-11T23:18:31.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.831+0000 7efbe9ffb640 1 -- 172.21.15.26:0/99969317 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7efbf800f040 con 0x55f3758394a0 2024-09-11T23:18:31.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.831+0000 7efc0aeee740 10 client.15357 fetch_fsmap finished waiting for FSMap version 29 2024-09-11T23:18:31.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.831+0000 7efc0aeee740 10 client.15357 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:18:31.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.831+0000 7efc0aeee740 20 client.15357 populate_metadata read hostname 'smithi026' 2024-09-11T23:18:31.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.831+0000 7efc0aeee740 1 --2- 172.21.15.26:0/99969317 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55f3758540b0 0x55f3758564e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:31.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.831+0000 7efc0aeee740 4 client.15357 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:18:31.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.831+0000 7efc0aeee740 1 -- 172.21.15.26:0/99969317 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f3757ac850 con 0x55f3758540b0 2024-09-11T23:18:31.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.831+0000 7efc0259c640 1 --2- 172.21.15.26:0/99969317 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55f3758540b0 0x55f3758564e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:31.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.832+0000 7efbe9ffb640 10 client.15357 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:18:31.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.832+0000 7efc0259c640 1 --2- 172.21.15.26:0/99969317 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55f3758540b0 0x55f3758564e0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fbc0f3d4028fcba7 server_cookie=3a51f4d24bf93b78 in_seq=0 out_seq=0 2024-09-11T23:18:31.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.832+0000 7efbe9ffb640 1 -- 172.21.15.26:0/99969317 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f3757ac850 con 0x55f3758540b0 2024-09-11T23:18:31.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.832+0000 7efbe9ffb640 10 client.15357 handle_command_reply: tid=0 2024-09-11T23:18:31.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.832+0000 7efc0aeee740 1 client.15357 shutdown 2024-09-11T23:18:31.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.833+0000 7efc0aeee740 1 -- 172.21.15.26:0/99969317 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f3758394a0 msgr2=0x55f375843f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:31.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.833+0000 7efc0aeee740 1 --2- 172.21.15.26:0/99969317 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f3758394a0 0x55f375843f00 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7efbf8013cd0 tx=0x7efbf8004730 comp rx=0 tx=0).stop 2024-09-11T23:18:31.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.833+0000 7efc0aeee740 1 -- 172.21.15.26:0/99969317 shutdown_connections 2024-09-11T23:18:31.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.833+0000 7efc0aeee740 1 --2- 172.21.15.26:0/99969317 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f375839dc0 0x55f3758411b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:31.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.833+0000 7efc0aeee740 1 --2- 172.21.15.26:0/99969317 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55f3758540b0 0x55f3758564e0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:31.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.833+0000 7efbe9ffb640 0 client.15357 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:18:31.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.833+0000 7efc0aeee740 1 --2- 172.21.15.26:0/99969317 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f375844680 0x55f3758416f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:31.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.833+0000 7efc0aeee740 1 --2- 172.21.15.26:0/99969317 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f3758394a0 0x55f375843f00 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:31.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.833+0000 7efc0aeee740 1 -- 172.21.15.26:0/99969317 >> 172.21.15.26:0/99969317 conn(0x55f375835a30 msgr2=0x55f37583b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:18:31.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.833+0000 7efc0aeee740 1 -- 172.21.15.26:0/99969317 shutdown_connections 2024-09-11T23:18:31.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.833+0000 7efc0aeee740 1 -- 172.21.15.26:0/99969317 wait complete. 2024-09-11T23:18:31.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.833+0000 7efc0aeee740 20 client.15357 trim_cache size 0 max 16384 2024-09-11T23:18:31.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.850+0000 7efc0aeee740 1 Processor -- start 2024-09-11T23:18:31.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.850+0000 7efc0aeee740 1 -- start start 2024-09-11T23:18:31.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.850+0000 7efc0aeee740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f3758394a0 0x55f37583fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:31.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.850+0000 7efc0aeee740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f375839dc0 0x55f37585bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:31.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.850+0000 7efc0aeee740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f375844680 0x55f37585c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:31.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.850+0000 7efc0aeee740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7efc041ad810 con 0x55f375844680 2024-09-11T23:18:31.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.850+0000 7efc0aeee740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7efc041af350 con 0x55f375839dc0 2024-09-11T23:18:31.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.850+0000 7efc0aeee740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7efc041adee0 con 0x55f3758394a0 2024-09-11T23:18:31.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.850+0000 7efc01d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f375844680 0x55f37585c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:31.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.850+0000 7efc01d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f375844680 0x55f37585c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60558/0 (socket says 172.21.15.26:60558) 2024-09-11T23:18:31.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.850+0000 7efc01d9b640 1 -- 172.21.15.26:0/2632379217 learned_addr learned my addr 172.21.15.26:0/2632379217 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:18:31.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.851+0000 7efc0259c640 1 --2- 172.21.15.26:0/2632379217 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f375839dc0 0x55f37585bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:31.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.851+0000 7efc01d9b640 1 --2- 172.21.15.26:0/2632379217 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f3758394a0 0x55f37583fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:31.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.851+0000 7efc01d9b640 1 -- 172.21.15.26:0/2632379217 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f3758394a0 msgr2=0x55f37583fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:18:31.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.851+0000 7efc01d9b640 1 --2- 172.21.15.26:0/2632379217 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f3758394a0 0x55f37583fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:31.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.851+0000 7efc01d9b640 1 -- 172.21.15.26:0/2632379217 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f375839dc0 msgr2=0x55f37585bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:18:31.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.851+0000 7efc01d9b640 1 --2- 172.21.15.26:0/2632379217 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f375839dc0 0x55f37585bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:31.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.851+0000 7efc01d9b640 1 -- 172.21.15.26:0/2632379217 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f375840160 con 0x55f375844680 2024-09-11T23:18:31.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.851+0000 7efc0259c640 1 --2- 172.21.15.26:0/2632379217 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f375839dc0 0x55f37585bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:18:31.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.851+0000 7efc01d9b640 1 --2- 172.21.15.26:0/2632379217 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f375844680 0x55f37585c2f0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7efbf8004700 tx=0x7efbf800b3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6567d8bb7425ad1 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:18:31.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.851+0000 7efbebfff640 1 -- 172.21.15.26:0/2632379217 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbf80053c0 con 0x55f375844680 2024-09-11T23:18:31.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.851+0000 7efbebfff640 1 -- 172.21.15.26:0/2632379217 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efbf8005560 con 0x55f375844680 2024-09-11T23:18:31.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.851+0000 7efbebfff640 1 -- 172.21.15.26:0/2632379217 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbf800fb90 con 0x55f375844680 2024-09-11T23:18:31.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.852+0000 7efc0aeee740 1 -- 172.21.15.26:0/2632379217 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f375844680 msgr2=0x55f37585c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:31.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.852+0000 7efc0aeee740 1 --2- 172.21.15.26:0/2632379217 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f375844680 0x55f37585c2f0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7efbf8004700 tx=0x7efbf800b3a0 comp rx=0 tx=0).stop 2024-09-11T23:18:31.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.852+0000 7efc0aeee740 1 -- 172.21.15.26:0/2632379217 shutdown_connections 2024-09-11T23:18:31.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.852+0000 7efc0aeee740 1 --2- 172.21.15.26:0/2632379217 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f375839dc0 0x55f37585bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:31.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.852+0000 7efc0aeee740 1 --2- 172.21.15.26:0/2632379217 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f375844680 0x55f37585c2f0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:31.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.852+0000 7efc0aeee740 1 --2- 172.21.15.26:0/2632379217 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f3758394a0 0x55f37583fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:31.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.852+0000 7efc0aeee740 1 -- 172.21.15.26:0/2632379217 >> 172.21.15.26:0/2632379217 conn(0x55f375835a30 msgr2=0x55f375834b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:18:31.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.852+0000 7efc0aeee740 1 -- 172.21.15.26:0/2632379217 shutdown_connections 2024-09-11T23:18:31.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.852+0000 7efc0aeee740 1 -- 172.21.15.26:0/2632379217 wait complete. 2024-09-11T23:18:31.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.852+0000 7efc0aeee740 1 -- start start 2024-09-11T23:18:31.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.852+0000 7efc0aeee740 1 Processor -- start 2024-09-11T23:18:31.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.853+0000 7efc0aeee740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f3758394a0 0x55f37584a470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:31.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.853+0000 7efc0aeee740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f375839dc0 0x55f37584a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:31.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.853+0000 7efc0aeee740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f3758540b0 0x55f37584aef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:31.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.853+0000 7efc0aeee740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7efc041aec80 con 0x55f3758540b0 2024-09-11T23:18:31.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.853+0000 7efc0aeee740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7efc041ae5b0 con 0x55f375839dc0 2024-09-11T23:18:31.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.853+0000 7efc0aeee740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7efc041ab7f0 con 0x55f3758394a0 2024-09-11T23:18:31.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.853+0000 7efc0aeee740 10 client.0 _ll_register_callbacks cb 0x7efbf800bd00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:18:31.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.853+0000 7efc01d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f3758540b0 0x55f37584aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:31.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.853+0000 7efc01d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f3758540b0 0x55f37584aef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60568/0 (socket says 172.21.15.26:60568) 2024-09-11T23:18:31.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.853+0000 7efc01d9b640 1 -- 172.21.15.26:0/191417561 learned_addr learned my addr 172.21.15.26:0/191417561 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:18:31.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.853+0000 7efc01d9b640 1 --2- 172.21.15.26:0/191417561 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f3758394a0 0x55f37584a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:31.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.853+0000 7efc0259c640 1 --2- 172.21.15.26:0/191417561 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f375839dc0 0x55f37584a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:31.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.853+0000 7efc01d9b640 1 -- 172.21.15.26:0/191417561 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f3758394a0 msgr2=0x55f37584a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:18:31.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.853+0000 7efc01d9b640 1 --2- 172.21.15.26:0/191417561 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f3758394a0 0x55f37584a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:31.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.853+0000 7efc01d9b640 1 -- 172.21.15.26:0/191417561 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f375839dc0 msgr2=0x55f37584a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:18:31.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.853+0000 7efc01d9b640 1 --2- 172.21.15.26:0/191417561 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f375839dc0 0x55f37584a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:31.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.853+0000 7efc01d9b640 1 -- 172.21.15.26:0/191417561 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3758bdef0 con 0x55f3758540b0 2024-09-11T23:18:31.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.854+0000 7efc01d9b640 1 --2- 172.21.15.26:0/191417561 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f3758540b0 0x55f37584aef0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7efbf8004700 tx=0x7efbf800f6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71eccfa49cb6f027 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:18:31.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.854+0000 7efbe9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:18:31.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.854+0000 7efbe9ffb640 1 -- 172.21.15.26:0/191417561 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbf800f910 con 0x55f3758540b0 2024-09-11T23:18:31.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.854+0000 7efbe9ffb640 1 -- 172.21.15.26:0/191417561 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7efbf800fab0 con 0x55f3758540b0 2024-09-11T23:18:31.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.854+0000 7efc0aeee740 1 -- 172.21.15.26:0/191417561 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efbf8005560 con 0x55f3758540b0 2024-09-11T23:18:31.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.854+0000 7efbe9ffb640 1 -- 172.21.15.26:0/191417561 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7efbf800fcb0 con 0x55f3758540b0 2024-09-11T23:18:31.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.854+0000 7efc0aeee740 10 client.15363 fetch_fsmap learned FSMap version 29 2024-09-11T23:18:31.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.854+0000 7efc0aeee740 1 -- 172.21.15.26:0/191417561 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f3758bf1c0 con 0x55f3758540b0 2024-09-11T23:18:31.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.854+0000 7efbe9ffb640 1 -- 172.21.15.26:0/191417561 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efbf8005560 con 0x55f3758540b0 2024-09-11T23:18:31.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.854+0000 7efbe9ffb640 1 -- 172.21.15.26:0/191417561 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7efbf802d660 con 0x55f3758540b0 2024-09-11T23:18:31.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.854+0000 7efc0aeee740 10 client.15363 fetch_fsmap finished waiting for FSMap version 29 2024-09-11T23:18:31.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.854+0000 7efc0aeee740 10 client.15363 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:18:31.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.854+0000 7efc0aeee740 20 client.15363 populate_metadata read hostname 'smithi026' 2024-09-11T23:18:31.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.854+0000 7efc0aeee740 1 --2- 172.21.15.26:0/191417561 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55f3758bf7c0 0x55f3758c1bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:31.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.854+0000 7efc0aeee740 4 client.15363 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T23:18:31.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.854+0000 7efc0aeee740 1 -- 172.21.15.26:0/191417561 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f3757a6d60 con 0x55f3758bf7c0 2024-09-11T23:18:31.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.855+0000 7efc0259c640 1 --2- 172.21.15.26:0/191417561 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55f3758bf7c0 0x55f3758c1bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:31.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.855+0000 7efc0259c640 1 --2- 172.21.15.26:0/191417561 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55f3758bf7c0 0x55f3758c1bb0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af7c35fa49898b1b server_cookie=bf3848cd938946e2 in_seq=0 out_seq=0 2024-09-11T23:18:31.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.855+0000 7efbe9ffb640 10 client.15363 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:18:31.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.856+0000 7efbe9ffb640 1 -- 172.21.15.26:0/191417561 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55f3757a6d60 con 0x55f3758bf7c0 2024-09-11T23:18:31.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.856+0000 7efbe9ffb640 10 client.15363 handle_command_reply: tid=0 2024-09-11T23:18:31.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.856+0000 7efc0aeee740 1 client.15363 shutdown 2024-09-11T23:18:31.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.856+0000 7efc0aeee740 1 -- 172.21.15.26:0/191417561 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f3758540b0 msgr2=0x55f37584aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:31.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.856+0000 7efc0aeee740 1 --2- 172.21.15.26:0/191417561 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f3758540b0 0x55f37584aef0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7efbf8004700 tx=0x7efbf800f6d0 comp rx=0 tx=0).stop 2024-09-11T23:18:31.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.856+0000 7efc0aeee740 1 -- 172.21.15.26:0/191417561 shutdown_connections 2024-09-11T23:18:31.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.856+0000 7efc0aeee740 1 --2- 172.21.15.26:0/191417561 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f375839dc0 0x55f37584a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:31.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.856+0000 7efc0aeee740 1 --2- 172.21.15.26:0/191417561 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55f3758bf7c0 0x55f3758c1bb0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:31.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.856+0000 7efc0aeee740 1 --2- 172.21.15.26:0/191417561 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f3758540b0 0x55f37584aef0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:31.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.856+0000 7efc0aeee740 1 --2- 172.21.15.26:0/191417561 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f3758394a0 0x55f37584a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:31.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.856+0000 7efc0aeee740 1 -- 172.21.15.26:0/191417561 >> 172.21.15.26:0/191417561 conn(0x55f375835a30 msgr2=0x55f375834b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:18:31.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.856+0000 7efbe9ffb640 0 client.15363 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:18:31.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.856+0000 7efc0aeee740 1 -- 172.21.15.26:0/191417561 shutdown_connections 2024-09-11T23:18:31.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.856+0000 7efc0aeee740 1 -- 172.21.15.26:0/191417561 wait complete. 2024-09-11T23:18:31.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.856+0000 7efc0aeee740 20 client.15363 trim_cache size 0 max 16384 2024-09-11T23:18:31.861 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-11T23:18:31.862 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-11T23:18:31.862 INFO:teuthology.orchestra.run.smithi026.stdout: "scrubs": { 2024-09-11T23:18:31.862 INFO:teuthology.orchestra.run.smithi026.stdout: "b9e592f0-d6de-49e1-90fc-1b4b7dd06f2a": { 2024-09-11T23:18:31.862 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/", 2024-09-11T23:18:31.862 INFO:teuthology.orchestra.run.smithi026.stdout: "tag": "b9e592f0-d6de-49e1-90fc-1b4b7dd06f2a", 2024-09-11T23:18:31.862 INFO:teuthology.orchestra.run.smithi026.stdout: "options": "recursive,force" 2024-09-11T23:18:31.862 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:18:31.862 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:18:31.862 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-11T23:18:31.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.856+0000 7efc08b15640 1 -- 172.21.15.26:0/3434485275 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7efbec0a2d80 msgr2=0x7efbec0a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:31.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.856+0000 7efc08b15640 1 --2- 172.21.15.26:0/3434485275 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7efbec0a2d80 0x7efbec0a5240 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7efbfc0cd430 tx=0x7efbfc00af30 comp rx=0 tx=0).stop 2024-09-11T23:18:31.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.856+0000 7efc08b15640 1 -- 172.21.15.26:0/3434485275 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc04102ef0 msgr2=0x7efc041a4350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:31.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.856+0000 7efc08b15640 1 --2- 172.21.15.26:0/3434485275 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc04102ef0 0x7efc041a4350 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7efbfc00cce0 tx=0x7efbfc00a620 comp rx=0 tx=0).stop 2024-09-11T23:18:31.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.857+0000 7efc08b15640 1 -- 172.21.15.26:0/3434485275 shutdown_connections 2024-09-11T23:18:31.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.857+0000 7efc08b15640 1 --2- 172.21.15.26:0/3434485275 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7efbec0a2d80 0x7efbec0a5240 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:31.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.857+0000 7efc08b15640 1 --2- 172.21.15.26:0/3434485275 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc04102ef0 0x7efc041a4350 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:31.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.857+0000 7efc08b15640 1 --2- 172.21.15.26:0/3434485275 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efc04102570 0x7efc04199810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:31.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.857+0000 7efc08b15640 1 --2- 172.21.15.26:0/3434485275 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efc04100e20 0x7efc041992d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:31.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.857+0000 7efc08b15640 1 -- 172.21.15.26:0/3434485275 >> 172.21.15.26:0/3434485275 conn(0x7efc040fc7d0 msgr2=0x7efc040fe060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:18:31.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.857+0000 7efc08b15640 1 -- 172.21.15.26:0/3434485275 shutdown_connections 2024-09-11T23:18:31.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.857+0000 7efc08b15640 1 -- 172.21.15.26:0/3434485275 wait complete. 2024-09-11T23:18:31.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.940+0000 7fa359c27640 1 -- 172.21.15.26:0/3497051571 --> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7fa354062630 con 0x7fa33c0a2e40 2024-09-11T23:18:31.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.947+0000 7fa350ff9640 1 -- 172.21.15.26:0/3497051571 <== mgr.15171 v2:172.21.15.26:6800/1989555790 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7fa354062630 con 0x7fa33c0a2e40 2024-09-11T23:18:31.947 INFO:teuthology.orchestra.run.smithi026.stdout:/ 1m 6m3h 2024-09-11T23:18:31.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.949+0000 7fa359c27640 1 -- 172.21.15.26:0/3497051571 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fa33c0a2e40 msgr2=0x7fa33c0a5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:31.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.949+0000 7fa359c27640 1 --2- 172.21.15.26:0/3497051571 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fa33c0a2e40 0x7fa33c0a5300 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fa344011550 tx=0x7fa344005f90 comp rx=0 tx=0).stop 2024-09-11T23:18:31.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.949+0000 7fa359c27640 1 -- 172.21.15.26:0/3497051571 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa35410c0f0 msgr2=0x7fa35419cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:31.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.949+0000 7fa359c27640 1 --2- 172.21.15.26:0/3497051571 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa35410c0f0 0x7fa35419cc40 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fa34c00e940 tx=0x7fa34c00ee00 comp rx=0 tx=0).stop 2024-09-11T23:18:31.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.949+0000 7fa359c27640 1 -- 172.21.15.26:0/3497051571 shutdown_connections 2024-09-11T23:18:31.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.949+0000 7fa359c27640 1 --2- 172.21.15.26:0/3497051571 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fa33c0a2e40 0x7fa33c0a5300 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:31.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.949+0000 7fa359c27640 1 --2- 172.21.15.26:0/3497051571 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa354103640 0x7fa3541988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:31.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.949+0000 7fa359c27640 1 --2- 172.21.15.26:0/3497051571 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa35410c0f0 0x7fa35419cc40 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:31.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.949+0000 7fa359c27640 1 --2- 172.21.15.26:0/3497051571 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa354102d20 0x7fa354198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:31.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.949+0000 7fa359c27640 1 -- 172.21.15.26:0/3497051571 >> 172.21.15.26:0/3497051571 conn(0x7fa354077990 msgr2=0x7fa3540fd0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:18:31.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.949+0000 7fa359c27640 1 -- 172.21.15.26:0/3497051571 shutdown_connections 2024-09-11T23:18:31.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:31.949+0000 7fa359c27640 1 -- 172.21.15.26:0/3497051571 wait complete. 2024-09-11T23:18:31.960 DEBUG:teuthology.orchestra.run.smithi026:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-09-11T23:18:31.984 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:31 smithi026 ceph-mon[31172]: pgmap v7: 97 pgs: 97 active+clean; 1.1 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 0 B/s wr, 24 op/s 2024-09-11T23:18:31.984 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:31 smithi026 ceph-mon[31172]: scrub queued for path: / 2024-09-11T23:18:31.984 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:31 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:18:31.984 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:31 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:18:31.984 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:31 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:18:31.984 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:31 smithi026 ceph-mon[31172]: from='client.15327 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:18:31.987 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-09-11T23:18:31.998 INFO:teuthology.run_tasks:Running task background_exec... 2024-09-11T23:18:32.033 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi026.front.sepia.ceph.com 2024-09-11T23:18:32.033 DEBUG:teuthology.orchestra.run.smithi026:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-09-11T23:18:32.036 INFO:teuthology.run_tasks:Running task quiescer... 2024-09-11T23:18:32.051 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-11T23:18:32.052 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-11T23:18:32.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:31 smithi076 ceph-mon[35296]: pgmap v7: 97 pgs: 97 active+clean; 1.1 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 0 B/s wr, 24 op/s 2024-09-11T23:18:32.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:31 smithi076 ceph-mon[35296]: scrub queued for path: / 2024-09-11T23:18:32.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:31 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:18:32.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:31 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:18:32.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:31 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:18:32.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:31 smithi076 ceph-mon[35296]: from='client.15327 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:18:32.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:31 smithi136 ceph-mon[35121]: pgmap v7: 97 pgs: 97 active+clean; 1.1 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 0 B/s wr, 24 op/s 2024-09-11T23:18:32.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:31 smithi136 ceph-mon[35121]: scrub queued for path: / 2024-09-11T23:18:32.240 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:31 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:18:32.240 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:31 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:18:32.240 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:31 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:18:32.240 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:31 smithi136 ceph-mon[35121]: from='client.15327 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:18:32.575 INFO:tasks.cephfs.filesystem:scrub status for tag:b9e592f0-d6de-49e1-90fc-1b4b7dd06f2a - {'path': '/', 'tag': 'b9e592f0-d6de-49e1-90fc-1b4b7dd06f2a', 'options': 'recursive,force'} 2024-09-11T23:18:32.644 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:18:32.877 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:32 smithi026 ceph-mon[31172]: from='client.15333 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:18:32.877 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:32 smithi026 ceph-mon[31172]: from='client.15339 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:18:32.877 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:32 smithi026 ceph-mon[31172]: from='client.15345 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:18:33.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:32 smithi076 ceph-mon[35296]: from='client.15333 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:18:33.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:32 smithi076 ceph-mon[35296]: from='client.15339 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:18:33.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:32 smithi076 ceph-mon[35296]: from='client.15345 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:18:33.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:33.082+0000 7feaf952e640 1 -- 172.21.15.26:0/974995363 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feaf4101df0 msgr2=0x7feaf41021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:33.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:33.082+0000 7feaf952e640 1 --2- 172.21.15.26:0/974995363 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feaf4101df0 0x7feaf41021d0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7feae4009900 tx=0x7feae402a410 comp rx=0 tx=0).stop 2024-09-11T23:18:33.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:33.082+0000 7feaf952e640 1 -- 172.21.15.26:0/974995363 shutdown_connections 2024-09-11T23:18:33.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:33.082+0000 7feaf952e640 1 --2- 172.21.15.26:0/974995363 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7feaf4106f20 0x7feaf410eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:33.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:33.082+0000 7feaf952e640 1 --2- 172.21.15.26:0/974995363 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7feaf41027a0 0x7feaf41067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:33.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:33.082+0000 7feaf952e640 1 --2- 172.21.15.26:0/974995363 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feaf4101df0 0x7feaf41021d0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:33.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:33.082+0000 7feaf952e640 1 -- 172.21.15.26:0/974995363 >> 172.21.15.26:0/974995363 conn(0x7feaf40fd8f0 msgr2=0x7feaf40ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:18:33.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:33.082+0000 7feaf952e640 1 -- 172.21.15.26:0/974995363 shutdown_connections 2024-09-11T23:18:33.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:33.082+0000 7feaf952e640 1 -- 172.21.15.26:0/974995363 wait complete. 2024-09-11T23:18:33.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:33.082+0000 7feaf952e640 1 Processor -- start 2024-09-11T23:18:33.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:33.083+0000 7feaf952e640 1 -- start start 2024-09-11T23:18:33.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:33.083+0000 7feaf952e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7feaf4101df0 0x7feaf41983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:33.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:33.083+0000 7feaf952e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feaf41027a0 0x7feaf4198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:33.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:33.083+0000 7feaf952e640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7feaf4106f20 0x7feaf419cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:33.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:33.083+0000 7feaf952e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7feaf4111f80 con 0x7feaf41027a0 2024-09-11T23:18:33.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:33.083+0000 7feaf952e640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7feaf4111e00 con 0x7feaf4101df0 2024-09-11T23:18:33.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:33.083+0000 7feaf952e640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7feaf4112100 con 0x7feaf4106f20 2024-09-11T23:18:33.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:33.083+0000 7feaf27fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feaf41027a0 0x7feaf4198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:33.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:33.083+0000 7feaf2ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7feaf4101df0 0x7feaf41983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:33.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:33.083+0000 7feaf27fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feaf41027a0 0x7feaf4198900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60586/0 (socket says 172.21.15.26:60586) 2024-09-11T23:18:33.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:33.083+0000 7feaf27fc640 1 -- 172.21.15.26:0/2429797036 learned_addr learned my addr 172.21.15.26:0/2429797036 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:18:33.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:33.083+0000 7feaf37fe640 1 --2- 172.21.15.26:0/2429797036 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7feaf4106f20 0x7feaf419cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:33.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:33.083+0000 7feaf27fc640 1 -- 172.21.15.26:0/2429797036 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7feaf4106f20 msgr2=0x7feaf419cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:33.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:33.083+0000 7feaf27fc640 1 --2- 172.21.15.26:0/2429797036 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7feaf4106f20 0x7feaf419cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:33.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:33.083+0000 7feaf27fc640 1 -- 172.21.15.26:0/2429797036 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7feaf4101df0 msgr2=0x7feaf41983c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:33.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:33.083+0000 7feaf27fc640 1 --2- 172.21.15.26:0/2429797036 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7feaf4101df0 0x7feaf41983c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:33.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:33.083+0000 7feaf27fc640 1 -- 172.21.15.26:0/2429797036 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feaf419d370 con 0x7feaf41027a0 2024-09-11T23:18:33.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:33.083+0000 7feaf37fe640 1 --2- 172.21.15.26:0/2429797036 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7feaf4106f20 0x7feaf419cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:18:33.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:33.084+0000 7feaf27fc640 1 --2- 172.21.15.26:0/2429797036 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feaf41027a0 0x7feaf4198900 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7feae800d770 tx=0x7feae800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:18:33.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:33.084+0000 7feadbfff640 1 -- 172.21.15.26:0/2429797036 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feae8017070 con 0x7feaf41027a0 2024-09-11T23:18:33.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:33.084+0000 7feadbfff640 1 -- 172.21.15.26:0/2429797036 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feae8005500 con 0x7feaf41027a0 2024-09-11T23:18:33.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:33.084+0000 7feaf952e640 1 -- 172.21.15.26:0/2429797036 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feaf419d660 con 0x7feaf41027a0 2024-09-11T23:18:33.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:33.084+0000 7feadbfff640 1 -- 172.21.15.26:0/2429797036 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feae8004e00 con 0x7feaf41027a0 2024-09-11T23:18:33.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:33.084+0000 7feaf952e640 1 -- 172.21.15.26:0/2429797036 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feaf41a4f40 con 0x7feaf41027a0 2024-09-11T23:18:33.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:33.085+0000 7feaf952e640 1 -- 172.21.15.26:0/2429797036 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feaf41021d0 con 0x7feaf41027a0 2024-09-11T23:18:33.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:33.086+0000 7feadbfff640 1 -- 172.21.15.26:0/2429797036 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 24) ==== 138009+0+0 (secure 0 0 0) 0x7feae8004fa0 con 0x7feaf41027a0 2024-09-11T23:18:33.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:33.087+0000 7feadbfff640 1 --2- 172.21.15.26:0/2429797036 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7feadc0a2e00 0x7feadc0a52c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:33.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:33.087+0000 7feaf2ffd640 1 --2- 172.21.15.26:0/2429797036 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7feadc0a2e00 0x7feadc0a52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:33.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:33.087+0000 7feadbfff640 1 -- 172.21.15.26:0/2429797036 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7feae80cefb0 con 0x7feaf41027a0 2024-09-11T23:18:33.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:33.088+0000 7feaf2ffd640 1 --2- 172.21.15.26:0/2429797036 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7feadc0a2e00 0x7feadc0a52c0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7feae4009230 tx=0x7feae40058d0 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:18:33.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:33.091+0000 7feadbfff640 1 -- 172.21.15.26:0/2429797036 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7feae809ba70 con 0x7feaf41027a0 2024-09-11T23:18:33.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:32 smithi136 ceph-mon[35121]: from='client.15333 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:18:33.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:32 smithi136 ceph-mon[35121]: from='client.15339 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:18:33.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:32 smithi136 ceph-mon[35121]: from='client.15345 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:18:33.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:33.267+0000 7feaf952e640 1 -- 172.21.15.26:0/2429797036 --> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7feaf410a160 con 0x7feadc0a2e00 2024-09-11T23:18:33.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:33.274+0000 7feadbfff640 1 -- 172.21.15.26:0/2429797036 <== mgr.15171 v2:172.21.15.26:6800/1989555790 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+257784 (secure 0 0 0) 0x7feaf410a160 con 0x7feadc0a2e00 2024-09-11T23:18:33.276 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-11T23:18:33.278 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-09-11T23:18:33.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:33.277+0000 7feaf952e640 1 -- 172.21.15.26:0/2429797036 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7feadc0a2e00 msgr2=0x7feadc0a52c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:33.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:33.277+0000 7feaf952e640 1 --2- 172.21.15.26:0/2429797036 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7feadc0a2e00 0x7feadc0a52c0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7feae4009230 tx=0x7feae40058d0 comp rx=0 tx=0).stop 2024-09-11T23:18:33.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:33.277+0000 7feaf952e640 1 -- 172.21.15.26:0/2429797036 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feaf41027a0 msgr2=0x7feaf4198900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:33.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:33.277+0000 7feaf952e640 1 --2- 172.21.15.26:0/2429797036 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feaf41027a0 0x7feaf4198900 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7feae800d770 tx=0x7feae800dc30 comp rx=0 tx=0).stop 2024-09-11T23:18:33.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:33.277+0000 7feaf952e640 1 -- 172.21.15.26:0/2429797036 shutdown_connections 2024-09-11T23:18:33.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:33.277+0000 7feaf952e640 1 --2- 172.21.15.26:0/2429797036 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7feadc0a2e00 0x7feadc0a52c0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:33.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:33.277+0000 7feaf952e640 1 --2- 172.21.15.26:0/2429797036 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7feaf4106f20 0x7feaf419cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:33.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:33.277+0000 7feaf952e640 1 --2- 172.21.15.26:0/2429797036 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feaf41027a0 0x7feaf4198900 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:33.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:33.277+0000 7feaf952e640 1 --2- 172.21.15.26:0/2429797036 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7feaf4101df0 0x7feaf41983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:33.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:33.277+0000 7feaf952e640 1 -- 172.21.15.26:0/2429797036 >> 172.21.15.26:0/2429797036 conn(0x7feaf40fd8f0 msgr2=0x7feaf40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:18:33.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:33.277+0000 7feaf952e640 1 -- 172.21.15.26:0/2429797036 shutdown_connections 2024-09-11T23:18:33.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:33.277+0000 7feaf952e640 1 -- 172.21.15.26:0/2429797036 wait complete. 2024-09-11T23:18:33.955 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:33 smithi026 ceph-mon[31172]: pgmap v8: 97 pgs: 97 active+clean; 1.1 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 0 B/s wr, 19 op/s 2024-09-11T23:18:33.983 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":8,"stamp":"2024-09-11T23:18:32.198792+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1189948,"num_objects":104,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1476,"num_read_kb":1064,"num_write":575,"num_write_kb":4402,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":948,"ondisk_log_size":948,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":381192,"kb_used_data":14976,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124560632,"statfs":{"total":1151940427776,"available":1151550087168,"internally_reserved":0,"allocated":15335424,"data_stored":9050652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":374453561},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":151,"num_read_kb":236,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.993488"},"pg_stats":[{"pgid":"3.3f","version":"85'8","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.329067+0000","last_change":"2024-09-11T23:16:25.899106+0000","last_active":"2024-09-11T23:18:22.329067+0000","last_peered":"2024-09-11T23:18:22.329067+0000","last_clean":"2024-09-11T23:18:22.329067+0000","last_became_active":"2024-09-11T23:16:25.898654+0000","last_became_peered":"2024-09-11T23:16:25.898654+0000","last_unstale":"2024-09-11T23:18:22.329067+0000","last_undegraded":"2024-09-11T23:18:22.329067+0000","last_fullsized":"2024-09-11T23:18:22.329067+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:21:41.490905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"85'4","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.544200+0000","last_change":"2024-09-11T23:16:25.895629+0000","last_active":"2024-09-11T23:18:22.544200+0000","last_peered":"2024-09-11T23:18:22.544200+0000","last_clean":"2024-09-11T23:18:22.544200+0000","last_became_active":"2024-09-11T23:16:25.895308+0000","last_became_peered":"2024-09-11T23:16:25.895308+0000","last_unstale":"2024-09-11T23:18:22.544200+0000","last_undegraded":"2024-09-11T23:18:22.544200+0000","last_fullsized":"2024-09-11T23:18:22.544200+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:39:42.549970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"85'7","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.544117+0000","last_change":"2024-09-11T23:16:25.907113+0000","last_active":"2024-09-11T23:18:22.544117+0000","last_peered":"2024-09-11T23:18:22.544117+0000","last_clean":"2024-09-11T23:18:22.544117+0000","last_became_active":"2024-09-11T23:16:25.906803+0000","last_became_peered":"2024-09-11T23:16:25.906803+0000","last_unstale":"2024-09-11T23:18:22.544117+0000","last_undegraded":"2024-09-11T23:18:22.544117+0000","last_fullsized":"2024-09-11T23:18:22.544117+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:36:49.303932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"85'8","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.292802+0000","last_change":"2024-09-11T23:16:35.946167+0000","last_active":"2024-09-11T23:18:22.292802+0000","last_peered":"2024-09-11T23:18:22.292802+0000","last_clean":"2024-09-11T23:18:22.292802+0000","last_became_active":"2024-09-11T23:16:35.945786+0000","last_became_peered":"2024-09-11T23:16:35.945786+0000","last_unstale":"2024-09-11T23:18:22.292802+0000","last_undegraded":"2024-09-11T23:18:22.292802+0000","last_fullsized":"2024-09-11T23:18:22.292802+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:13:03.064910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"85'4","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.295860+0000","last_change":"2024-09-11T23:16:25.897909+0000","last_active":"2024-09-11T23:18:22.295860+0000","last_peered":"2024-09-11T23:18:22.295860+0000","last_clean":"2024-09-11T23:18:22.295860+0000","last_became_active":"2024-09-11T23:16:25.897569+0000","last_became_peered":"2024-09-11T23:16:25.897569+0000","last_unstale":"2024-09-11T23:18:22.295860+0000","last_undegraded":"2024-09-11T23:18:22.295860+0000","last_fullsized":"2024-09-11T23:18:22.295860+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:25:44.727254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"85'6","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.302845+0000","last_change":"2024-09-11T23:16:25.897594+0000","last_active":"2024-09-11T23:18:22.302845+0000","last_peered":"2024-09-11T23:18:22.302845+0000","last_clean":"2024-09-11T23:18:22.302845+0000","last_became_active":"2024-09-11T23:16:25.897223+0000","last_became_peered":"2024-09-11T23:16:25.897223+0000","last_unstale":"2024-09-11T23:18:22.302845+0000","last_undegraded":"2024-09-11T23:18:22.302845+0000","last_fullsized":"2024-09-11T23:18:22.302845+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:10:50.763455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"85'5","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328293+0000","last_change":"2024-09-11T23:16:35.943511+0000","last_active":"2024-09-11T23:18:22.328293+0000","last_peered":"2024-09-11T23:18:22.328293+0000","last_clean":"2024-09-11T23:18:22.328293+0000","last_became_active":"2024-09-11T23:16:35.943075+0000","last_became_peered":"2024-09-11T23:16:35.943075+0000","last_unstale":"2024-09-11T23:18:22.328293+0000","last_undegraded":"2024-09-11T23:18:22.328293+0000","last_fullsized":"2024-09-11T23:18:22.328293+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:53:43.892421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"85'9","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.182677+0000","last_change":"2024-09-11T23:16:25.903380+0000","last_active":"2024-09-11T23:18:22.182677+0000","last_peered":"2024-09-11T23:18:22.182677+0000","last_clean":"2024-09-11T23:18:22.182677+0000","last_became_active":"2024-09-11T23:16:25.903093+0000","last_became_peered":"2024-09-11T23:16:25.903093+0000","last_unstale":"2024-09-11T23:18:22.182677+0000","last_undegraded":"2024-09-11T23:18:22.182677+0000","last_fullsized":"2024-09-11T23:18:22.182677+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:58:50.944623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"85'11","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.329153+0000","last_change":"2024-09-11T23:16:25.908166+0000","last_active":"2024-09-11T23:18:22.329153+0000","last_peered":"2024-09-11T23:18:22.329153+0000","last_clean":"2024-09-11T23:18:22.329153+0000","last_became_active":"2024-09-11T23:16:25.907749+0000","last_became_peered":"2024-09-11T23:16:25.907749+0000","last_unstale":"2024-09-11T23:18:22.329153+0000","last_undegraded":"2024-09-11T23:18:22.329153+0000","last_fullsized":"2024-09-11T23:18:22.329153+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:11:16.705931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"85'16","reported_seq":41,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.303286+0000","last_change":"2024-09-11T23:16:35.944259+0000","last_active":"2024-09-11T23:18:22.303286+0000","last_peered":"2024-09-11T23:18:22.303286+0000","last_clean":"2024-09-11T23:18:22.303286+0000","last_became_active":"2024-09-11T23:16:35.943767+0000","last_became_peered":"2024-09-11T23:16:35.943767+0000","last_unstale":"2024-09-11T23:18:22.303286+0000","last_undegraded":"2024-09-11T23:18:22.303286+0000","last_fullsized":"2024-09-11T23:18:22.303286+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:20:13.127854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"85'6","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328788+0000","last_change":"2024-09-11T23:16:25.903256+0000","last_active":"2024-09-11T23:18:22.328788+0000","last_peered":"2024-09-11T23:18:22.328788+0000","last_clean":"2024-09-11T23:18:22.328788+0000","last_became_active":"2024-09-11T23:16:25.902954+0000","last_became_peered":"2024-09-11T23:16:25.902954+0000","last_unstale":"2024-09-11T23:18:22.328788+0000","last_undegraded":"2024-09-11T23:18:22.328788+0000","last_fullsized":"2024-09-11T23:18:22.328788+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:43:48.651400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"85'7","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.187001+0000","last_change":"2024-09-11T23:16:25.897656+0000","last_active":"2024-09-11T23:18:22.187001+0000","last_peered":"2024-09-11T23:18:22.187001+0000","last_clean":"2024-09-11T23:18:22.187001+0000","last_became_active":"2024-09-11T23:16:25.897103+0000","last_became_peered":"2024-09-11T23:16:25.897103+0000","last_unstale":"2024-09-11T23:18:22.187001+0000","last_undegraded":"2024-09-11T23:18:22.187001+0000","last_fullsized":"2024-09-11T23:18:22.187001+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:14:34.341383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"85'10","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.329625+0000","last_change":"2024-09-11T23:16:25.902800+0000","last_active":"2024-09-11T23:18:22.329625+0000","last_peered":"2024-09-11T23:18:22.329625+0000","last_clean":"2024-09-11T23:18:22.329625+0000","last_became_active":"2024-09-11T23:16:25.902414+0000","last_became_peered":"2024-09-11T23:16:25.902414+0000","last_unstale":"2024-09-11T23:18:22.329625+0000","last_undegraded":"2024-09-11T23:18:22.329625+0000","last_fullsized":"2024-09-11T23:18:22.329625+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:16:32.427749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"85'7","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.543876+0000","last_change":"2024-09-11T23:16:25.894410+0000","last_active":"2024-09-11T23:18:22.543876+0000","last_peered":"2024-09-11T23:18:22.543876+0000","last_clean":"2024-09-11T23:18:22.543876+0000","last_became_active":"2024-09-11T23:16:25.894067+0000","last_became_peered":"2024-09-11T23:16:25.894067+0000","last_unstale":"2024-09-11T23:18:22.543876+0000","last_undegraded":"2024-09-11T23:18:22.543876+0000","last_fullsized":"2024-09-11T23:18:22.543876+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:03:11.913063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"85'7","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.544140+0000","last_change":"2024-09-11T23:16:25.903440+0000","last_active":"2024-09-11T23:18:22.544140+0000","last_peered":"2024-09-11T23:18:22.544140+0000","last_clean":"2024-09-11T23:18:22.544140+0000","last_became_active":"2024-09-11T23:16:25.903122+0000","last_became_peered":"2024-09-11T23:16:25.903122+0000","last_unstale":"2024-09-11T23:18:22.544140+0000","last_undegraded":"2024-09-11T23:18:22.544140+0000","last_fullsized":"2024-09-11T23:18:22.544140+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:03:41.065314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"85'9","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328034+0000","last_change":"2024-09-11T23:16:25.905336+0000","last_active":"2024-09-11T23:18:22.328034+0000","last_peered":"2024-09-11T23:18:22.328034+0000","last_clean":"2024-09-11T23:18:22.328034+0000","last_became_active":"2024-09-11T23:16:25.904699+0000","last_became_peered":"2024-09-11T23:16:25.904699+0000","last_unstale":"2024-09-11T23:18:22.328034+0000","last_undegraded":"2024-09-11T23:18:22.328034+0000","last_fullsized":"2024-09-11T23:18:22.328034+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:16:13.448585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"85'8","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.329144+0000","last_change":"2024-09-11T23:16:25.897977+0000","last_active":"2024-09-11T23:18:22.329144+0000","last_peered":"2024-09-11T23:18:22.329144+0000","last_clean":"2024-09-11T23:18:22.329144+0000","last_became_active":"2024-09-11T23:16:25.897672+0000","last_became_peered":"2024-09-11T23:16:25.897672+0000","last_unstale":"2024-09-11T23:18:22.329144+0000","last_undegraded":"2024-09-11T23:18:22.329144+0000","last_fullsized":"2024-09-11T23:18:22.329144+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:42:56.575066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"85'10","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.293509+0000","last_change":"2024-09-11T23:16:25.897063+0000","last_active":"2024-09-11T23:18:22.293509+0000","last_peered":"2024-09-11T23:18:22.293509+0000","last_clean":"2024-09-11T23:18:22.293509+0000","last_became_active":"2024-09-11T23:16:25.896770+0000","last_became_peered":"2024-09-11T23:16:25.896770+0000","last_unstale":"2024-09-11T23:18:22.293509+0000","last_undegraded":"2024-09-11T23:18:22.293509+0000","last_fullsized":"2024-09-11T23:18:22.293509+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:55:44.735171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"85'7","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328943+0000","last_change":"2024-09-11T23:16:25.903423+0000","last_active":"2024-09-11T23:18:22.328943+0000","last_peered":"2024-09-11T23:18:22.328943+0000","last_clean":"2024-09-11T23:18:22.328943+0000","last_became_active":"2024-09-11T23:16:25.902906+0000","last_became_peered":"2024-09-11T23:16:25.902906+0000","last_unstale":"2024-09-11T23:18:22.328943+0000","last_undegraded":"2024-09-11T23:18:22.328943+0000","last_fullsized":"2024-09-11T23:18:22.328943+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:40:49.326045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"85'8","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328557+0000","last_change":"2024-09-11T23:16:25.896564+0000","last_active":"2024-09-11T23:18:22.328557+0000","last_peered":"2024-09-11T23:18:22.328557+0000","last_clean":"2024-09-11T23:18:22.328557+0000","last_became_active":"2024-09-11T23:16:25.896301+0000","last_became_peered":"2024-09-11T23:16:25.896301+0000","last_unstale":"2024-09-11T23:18:22.328557+0000","last_undegraded":"2024-09-11T23:18:22.328557+0000","last_fullsized":"2024-09-11T23:18:22.328557+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:30:30.074853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"85'11","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.329804+0000","last_change":"2024-09-11T23:16:25.903491+0000","last_active":"2024-09-11T23:18:22.329804+0000","last_peered":"2024-09-11T23:18:22.329804+0000","last_clean":"2024-09-11T23:18:22.329804+0000","last_became_active":"2024-09-11T23:16:25.901340+0000","last_became_peered":"2024-09-11T23:16:25.901340+0000","last_unstale":"2024-09-11T23:18:22.329804+0000","last_undegraded":"2024-09-11T23:18:22.329804+0000","last_fullsized":"2024-09-11T23:18:22.329804+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:41:16.058034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"85'10","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328233+0000","last_change":"2024-09-11T23:16:25.901084+0000","last_active":"2024-09-11T23:18:22.328233+0000","last_peered":"2024-09-11T23:18:22.328233+0000","last_clean":"2024-09-11T23:18:22.328233+0000","last_became_active":"2024-09-11T23:16:25.900726+0000","last_became_peered":"2024-09-11T23:16:25.900726+0000","last_unstale":"2024-09-11T23:18:22.328233+0000","last_undegraded":"2024-09-11T23:18:22.328233+0000","last_fullsized":"2024-09-11T23:18:22.328233+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:57:40.955645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"85'8","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.182688+0000","last_change":"2024-09-11T23:16:25.900461+0000","last_active":"2024-09-11T23:18:22.182688+0000","last_peered":"2024-09-11T23:18:22.182688+0000","last_clean":"2024-09-11T23:18:22.182688+0000","last_became_active":"2024-09-11T23:16:25.900174+0000","last_became_peered":"2024-09-11T23:16:25.900174+0000","last_unstale":"2024-09-11T23:18:22.182688+0000","last_undegraded":"2024-09-11T23:18:22.182688+0000","last_fullsized":"2024-09-11T23:18:22.182688+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:51:50.953198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"85'6","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.184452+0000","last_change":"2024-09-11T23:16:25.907569+0000","last_active":"2024-09-11T23:18:22.184452+0000","last_peered":"2024-09-11T23:18:22.184452+0000","last_clean":"2024-09-11T23:18:22.184452+0000","last_became_active":"2024-09-11T23:16:25.907091+0000","last_became_peered":"2024-09-11T23:16:25.907091+0000","last_unstale":"2024-09-11T23:18:22.184452+0000","last_undegraded":"2024-09-11T23:18:22.184452+0000","last_fullsized":"2024-09-11T23:18:22.184452+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:53:45.961167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"85'2","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.183204+0000","last_change":"2024-09-11T23:16:25.903246+0000","last_active":"2024-09-11T23:18:22.183204+0000","last_peered":"2024-09-11T23:18:22.183204+0000","last_clean":"2024-09-11T23:18:22.183204+0000","last_became_active":"2024-09-11T23:16:25.902907+0000","last_became_peered":"2024-09-11T23:16:25.902907+0000","last_unstale":"2024-09-11T23:18:22.183204+0000","last_undegraded":"2024-09-11T23:18:22.183204+0000","last_fullsized":"2024-09-11T23:18:22.183204+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:20:38.868384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"85'15","reported_seq":34,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328979+0000","last_change":"2024-09-11T23:16:25.906298+0000","last_active":"2024-09-11T23:18:22.328979+0000","last_peered":"2024-09-11T23:18:22.328979+0000","last_clean":"2024-09-11T23:18:22.328979+0000","last_became_active":"2024-09-11T23:16:25.905350+0000","last_became_peered":"2024-09-11T23:16:25.905350+0000","last_unstale":"2024-09-11T23:18:22.328979+0000","last_undegraded":"2024-09-11T23:18:22.328979+0000","last_fullsized":"2024-09-11T23:18:22.328979+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:04:22.122514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"85'6","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328869+0000","last_change":"2024-09-11T23:16:25.902114+0000","last_active":"2024-09-11T23:18:22.328869+0000","last_peered":"2024-09-11T23:18:22.328869+0000","last_clean":"2024-09-11T23:18:22.328869+0000","last_became_active":"2024-09-11T23:16:25.901742+0000","last_became_peered":"2024-09-11T23:16:25.901742+0000","last_unstale":"2024-09-11T23:18:22.328869+0000","last_undegraded":"2024-09-11T23:18:22.328869+0000","last_fullsized":"2024-09-11T23:18:22.328869+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:02:10.640801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"85'4","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.329796+0000","last_change":"2024-09-11T23:16:25.898117+0000","last_active":"2024-09-11T23:18:22.329796+0000","last_peered":"2024-09-11T23:18:22.329796+0000","last_clean":"2024-09-11T23:18:22.329796+0000","last_became_active":"2024-09-11T23:16:25.897818+0000","last_became_peered":"2024-09-11T23:16:25.897818+0000","last_unstale":"2024-09-11T23:18:22.329796+0000","last_undegraded":"2024-09-11T23:18:22.329796+0000","last_fullsized":"2024-09-11T23:18:22.329796+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:27:37.175471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"85'7","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.544057+0000","last_change":"2024-09-11T23:16:25.905600+0000","last_active":"2024-09-11T23:18:22.544057+0000","last_peered":"2024-09-11T23:18:22.544057+0000","last_clean":"2024-09-11T23:18:22.544057+0000","last_became_active":"2024-09-11T23:16:25.903924+0000","last_became_peered":"2024-09-11T23:16:25.903924+0000","last_unstale":"2024-09-11T23:18:22.544057+0000","last_undegraded":"2024-09-11T23:18:22.544057+0000","last_fullsized":"2024-09-11T23:18:22.544057+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:20:29.929079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"85'8","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328449+0000","last_change":"2024-09-11T23:16:25.904763+0000","last_active":"2024-09-11T23:18:22.328449+0000","last_peered":"2024-09-11T23:18:22.328449+0000","last_clean":"2024-09-11T23:18:22.328449+0000","last_became_active":"2024-09-11T23:16:25.904406+0000","last_became_peered":"2024-09-11T23:16:25.904406+0000","last_unstale":"2024-09-11T23:18:22.328449+0000","last_undegraded":"2024-09-11T23:18:22.328449+0000","last_fullsized":"2024-09-11T23:18:22.328449+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:27:38.918017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"85'12","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328598+0000","last_change":"2024-09-11T23:16:25.906573+0000","last_active":"2024-09-11T23:18:22.328598+0000","last_peered":"2024-09-11T23:18:22.328598+0000","last_clean":"2024-09-11T23:18:22.328598+0000","last_became_active":"2024-09-11T23:16:25.904771+0000","last_became_peered":"2024-09-11T23:16:25.904771+0000","last_unstale":"2024-09-11T23:18:22.328598+0000","last_undegraded":"2024-09-11T23:18:22.328598+0000","last_fullsized":"2024-09-11T23:18:22.328598+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:04:17.090648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"85'9","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328406+0000","last_change":"2024-09-11T23:16:25.902123+0000","last_active":"2024-09-11T23:18:22.328406+0000","last_peered":"2024-09-11T23:18:22.328406+0000","last_clean":"2024-09-11T23:18:22.328406+0000","last_became_active":"2024-09-11T23:16:25.901756+0000","last_became_peered":"2024-09-11T23:16:25.901756+0000","last_unstale":"2024-09-11T23:18:22.328406+0000","last_undegraded":"2024-09-11T23:18:22.328406+0000","last_fullsized":"2024-09-11T23:18:22.328406+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:17:37.074976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"85'7","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.295802+0000","last_change":"2024-09-11T23:16:25.904217+0000","last_active":"2024-09-11T23:18:22.295802+0000","last_peered":"2024-09-11T23:18:22.295802+0000","last_clean":"2024-09-11T23:18:22.295802+0000","last_became_active":"2024-09-11T23:16:25.903522+0000","last_became_peered":"2024-09-11T23:16:25.903522+0000","last_unstale":"2024-09-11T23:18:22.295802+0000","last_undegraded":"2024-09-11T23:18:22.295802+0000","last_fullsized":"2024-09-11T23:18:22.295802+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:29:46.634475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"86'7","reported_seq":145,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:27.129184+0000","last_change":"2024-09-11T23:16:23.793696+0000","last_active":"2024-09-11T23:18:27.129184+0000","last_peered":"2024-09-11T23:18:27.129184+0000","last_clean":"2024-09-11T23:18:27.129184+0000","last_became_active":"2024-09-11T23:16:23.793062+0000","last_became_peered":"2024-09-11T23:16:23.793062+0000","last_unstale":"2024-09-11T23:18:27.129184+0000","last_undegraded":"2024-09-11T23:18:27.129184+0000","last_fullsized":"2024-09-11T23:18:27.129184+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:00:25.655817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1323,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":114,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"85'8","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.184453+0000","last_change":"2024-09-11T23:16:25.906673+0000","last_active":"2024-09-11T23:18:22.184453+0000","last_peered":"2024-09-11T23:18:22.184453+0000","last_clean":"2024-09-11T23:18:22.184453+0000","last_became_active":"2024-09-11T23:16:25.906239+0000","last_became_peered":"2024-09-11T23:16:25.906239+0000","last_unstale":"2024-09-11T23:18:22.184453+0000","last_undegraded":"2024-09-11T23:18:22.184453+0000","last_fullsized":"2024-09-11T23:18:22.184453+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:15:54.907269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"85'7","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.302905+0000","last_change":"2024-09-11T23:16:23.793302+0000","last_active":"2024-09-11T23:18:22.302905+0000","last_peered":"2024-09-11T23:18:22.302905+0000","last_clean":"2024-09-11T23:18:22.302905+0000","last_became_active":"2024-09-11T23:16:23.792874+0000","last_became_peered":"2024-09-11T23:16:23.792874+0000","last_unstale":"2024-09-11T23:18:22.302905+0000","last_undegraded":"2024-09-11T23:18:22.302905+0000","last_fullsized":"2024-09-11T23:18:22.302905+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:52:59.558223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"85'3","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328845+0000","last_change":"2024-09-11T23:16:23.792999+0000","last_active":"2024-09-11T23:18:22.328845+0000","last_peered":"2024-09-11T23:18:22.328845+0000","last_clean":"2024-09-11T23:18:22.328845+0000","last_became_active":"2024-09-11T23:16:23.792630+0000","last_became_peered":"2024-09-11T23:16:23.792630+0000","last_unstale":"2024-09-11T23:18:22.328845+0000","last_undegraded":"2024-09-11T23:18:22.328845+0000","last_fullsized":"2024-09-11T23:18:22.328845+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:09:28.001060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"85'8","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.292734+0000","last_change":"2024-09-11T23:16:35.943722+0000","last_active":"2024-09-11T23:18:22.292734+0000","last_peered":"2024-09-11T23:18:22.292734+0000","last_clean":"2024-09-11T23:18:22.292734+0000","last_became_active":"2024-09-11T23:16:35.943355+0000","last_became_peered":"2024-09-11T23:16:35.943355+0000","last_unstale":"2024-09-11T23:18:22.292734+0000","last_undegraded":"2024-09-11T23:18:22.292734+0000","last_fullsized":"2024-09-11T23:18:22.292734+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:26:22.769824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,6],"acting":[9,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"85'3","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.184646+0000","last_change":"2024-09-11T23:16:23.797945+0000","last_active":"2024-09-11T23:18:22.184646+0000","last_peered":"2024-09-11T23:18:22.184646+0000","last_clean":"2024-09-11T23:18:22.184646+0000","last_became_active":"2024-09-11T23:16:23.797572+0000","last_became_peered":"2024-09-11T23:16:23.797572+0000","last_unstale":"2024-09-11T23:18:22.184646+0000","last_undegraded":"2024-09-11T23:18:22.184646+0000","last_fullsized":"2024-09-11T23:18:22.184646+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:04:59.017668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"85'11","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.292877+0000","last_change":"2024-09-11T23:16:25.891587+0000","last_active":"2024-09-11T23:18:22.292877+0000","last_peered":"2024-09-11T23:18:22.292877+0000","last_clean":"2024-09-11T23:18:22.292877+0000","last_became_active":"2024-09-11T23:16:25.891190+0000","last_became_peered":"2024-09-11T23:16:25.891190+0000","last_unstale":"2024-09-11T23:18:22.292877+0000","last_undegraded":"2024-09-11T23:18:22.292877+0000","last_fullsized":"2024-09-11T23:18:22.292877+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:57:05.177843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"85'6","reported_seq":250,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:28.129952+0000","last_change":"2024-09-11T23:16:23.790129+0000","last_active":"2024-09-11T23:18:28.129952+0000","last_peered":"2024-09-11T23:18:28.129952+0000","last_clean":"2024-09-11T23:18:28.129952+0000","last_became_active":"2024-09-11T23:16:23.789720+0000","last_became_peered":"2024-09-11T23:16:23.789720+0000","last_unstale":"2024-09-11T23:18:28.129952+0000","last_undegraded":"2024-09-11T23:18:28.129952+0000","last_fullsized":"2024-09-11T23:18:28.129952+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:33:58.388513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":220,"num_read_kb":220,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"85'10","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.329142+0000","last_change":"2024-09-11T23:16:25.901014+0000","last_active":"2024-09-11T23:18:22.329142+0000","last_peered":"2024-09-11T23:18:22.329142+0000","last_clean":"2024-09-11T23:18:22.329142+0000","last_became_active":"2024-09-11T23:16:25.900280+0000","last_became_peered":"2024-09-11T23:16:25.900280+0000","last_unstale":"2024-09-11T23:18:22.329142+0000","last_undegraded":"2024-09-11T23:18:22.329142+0000","last_fullsized":"2024-09-11T23:18:22.329142+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:14:03.662204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"85'4","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328532+0000","last_change":"2024-09-11T23:16:23.797944+0000","last_active":"2024-09-11T23:18:22.328532+0000","last_peered":"2024-09-11T23:18:22.328532+0000","last_clean":"2024-09-11T23:18:22.328532+0000","last_became_active":"2024-09-11T23:16:23.797520+0000","last_became_peered":"2024-09-11T23:16:23.797520+0000","last_unstale":"2024-09-11T23:18:22.328532+0000","last_undegraded":"2024-09-11T23:18:22.328532+0000","last_fullsized":"2024-09-11T23:18:22.328532+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:29:06.957518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"85'9","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.544713+0000","last_change":"2024-09-11T23:16:25.905379+0000","last_active":"2024-09-11T23:18:22.544713+0000","last_peered":"2024-09-11T23:18:22.544713+0000","last_clean":"2024-09-11T23:18:22.544713+0000","last_became_active":"2024-09-11T23:16:25.905085+0000","last_became_peered":"2024-09-11T23:16:25.905085+0000","last_unstale":"2024-09-11T23:18:22.544713+0000","last_undegraded":"2024-09-11T23:18:22.544713+0000","last_fullsized":"2024-09-11T23:18:22.544713+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:22:35.129736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"85'5","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.184874+0000","last_change":"2024-09-11T23:16:24.075556+0000","last_active":"2024-09-11T23:18:22.184874+0000","last_peered":"2024-09-11T23:18:22.184874+0000","last_clean":"2024-09-11T23:18:22.184874+0000","last_became_active":"2024-09-11T23:16:24.074783+0000","last_became_peered":"2024-09-11T23:16:24.074783+0000","last_unstale":"2024-09-11T23:18:22.184874+0000","last_undegraded":"2024-09-11T23:18:22.184874+0000","last_fullsized":"2024-09-11T23:18:22.184874+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:55:09.962342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"85'11","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328166+0000","last_change":"2024-09-11T23:16:25.907720+0000","last_active":"2024-09-11T23:18:22.328166+0000","last_peered":"2024-09-11T23:18:22.328166+0000","last_clean":"2024-09-11T23:18:22.328166+0000","last_became_active":"2024-09-11T23:16:25.907335+0000","last_became_peered":"2024-09-11T23:16:25.907335+0000","last_unstale":"2024-09-11T23:18:22.328166+0000","last_undegraded":"2024-09-11T23:18:22.328166+0000","last_fullsized":"2024-09-11T23:18:22.328166+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:46:14.519384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"86'38","reported_seq":294,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:28.151652+0000","last_change":"2024-09-11T23:16:23.794321+0000","last_active":"2024-09-11T23:18:28.151652+0000","last_peered":"2024-09-11T23:18:28.151652+0000","last_clean":"2024-09-11T23:18:28.151652+0000","last_became_active":"2024-09-11T23:16:23.793955+0000","last_became_peered":"2024-09-11T23:16:23.793955+0000","last_unstale":"2024-09-11T23:18:28.151652+0000","last_undegraded":"2024-09-11T23:18:28.151652+0000","last_fullsized":"2024-09-11T23:18:28.151652+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:47:46.442656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":66480,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":232,"num_read_kb":180,"num_write":40,"num_write_kb":82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"85'6","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.296053+0000","last_change":"2024-09-11T23:16:25.896598+0000","last_active":"2024-09-11T23:18:22.296053+0000","last_peered":"2024-09-11T23:18:22.296053+0000","last_clean":"2024-09-11T23:18:22.296053+0000","last_became_active":"2024-09-11T23:16:25.896293+0000","last_became_peered":"2024-09-11T23:16:25.896293+0000","last_unstale":"2024-09-11T23:18:22.296053+0000","last_undegraded":"2024-09-11T23:18:22.296053+0000","last_fullsized":"2024-09-11T23:18:22.296053+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:04:11.489644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"85'9","reported_seq":35,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328677+0000","last_change":"2024-09-11T23:16:23.791486+0000","last_active":"2024-09-11T23:18:22.328677+0000","last_peered":"2024-09-11T23:18:22.328677+0000","last_clean":"2024-09-11T23:18:22.328677+0000","last_became_active":"2024-09-11T23:16:23.790786+0000","last_became_peered":"2024-09-11T23:16:23.790786+0000","last_unstale":"2024-09-11T23:18:22.328677+0000","last_undegraded":"2024-09-11T23:18:22.328677+0000","last_fullsized":"2024-09-11T23:18:22.328677+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:03:52.742560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"85'9","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.329171+0000","last_change":"2024-09-11T23:16:25.897382+0000","last_active":"2024-09-11T23:18:22.329171+0000","last_peered":"2024-09-11T23:18:22.329171+0000","last_clean":"2024-09-11T23:18:22.329171+0000","last_became_active":"2024-09-11T23:16:25.897073+0000","last_became_peered":"2024-09-11T23:16:25.897073+0000","last_unstale":"2024-09-11T23:18:22.329171+0000","last_undegraded":"2024-09-11T23:18:22.329171+0000","last_fullsized":"2024-09-11T23:18:22.329171+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:15:59.780100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"85'5","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.184727+0000","last_change":"2024-09-11T23:16:23.793886+0000","last_active":"2024-09-11T23:18:22.184727+0000","last_peered":"2024-09-11T23:18:22.184727+0000","last_clean":"2024-09-11T23:18:22.184727+0000","last_became_active":"2024-09-11T23:16:23.793066+0000","last_became_peered":"2024-09-11T23:16:23.793066+0000","last_unstale":"2024-09-11T23:18:22.184727+0000","last_undegraded":"2024-09-11T23:18:22.184727+0000","last_fullsized":"2024-09-11T23:18:22.184727+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:07:03.882546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"85'15","reported_seq":34,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328552+0000","last_change":"2024-09-11T23:16:25.904575+0000","last_active":"2024-09-11T23:18:22.328552+0000","last_peered":"2024-09-11T23:18:22.328552+0000","last_clean":"2024-09-11T23:18:22.328552+0000","last_became_active":"2024-09-11T23:16:25.904224+0000","last_became_peered":"2024-09-11T23:16:25.904224+0000","last_unstale":"2024-09-11T23:18:22.328552+0000","last_undegraded":"2024-09-11T23:18:22.328552+0000","last_fullsized":"2024-09-11T23:18:22.328552+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:42:13.036931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"85'5","reported_seq":141,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:27.129022+0000","last_change":"2024-09-11T23:16:23.796073+0000","last_active":"2024-09-11T23:18:27.129022+0000","last_peered":"2024-09-11T23:18:27.129022+0000","last_clean":"2024-09-11T23:18:27.129022+0000","last_became_active":"2024-09-11T23:16:23.794850+0000","last_became_peered":"2024-09-11T23:16:23.794850+0000","last_unstale":"2024-09-11T23:18:27.129022+0000","last_undegraded":"2024-09-11T23:18:27.129022+0000","last_fullsized":"2024-09-11T23:18:27.129022+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:57:22.361662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":113,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"85'6","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328221+0000","last_change":"2024-09-11T23:16:25.908700+0000","last_active":"2024-09-11T23:18:22.328221+0000","last_peered":"2024-09-11T23:18:22.328221+0000","last_clean":"2024-09-11T23:18:22.328221+0000","last_became_active":"2024-09-11T23:16:25.908373+0000","last_became_peered":"2024-09-11T23:16:25.908373+0000","last_unstale":"2024-09-11T23:18:22.328221+0000","last_undegraded":"2024-09-11T23:18:22.328221+0000","last_fullsized":"2024-09-11T23:18:22.328221+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:20:18.536079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"85'17","reported_seq":42,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.295792+0000","last_change":"2024-09-11T23:16:24.075927+0000","last_active":"2024-09-11T23:18:22.295792+0000","last_peered":"2024-09-11T23:18:22.295792+0000","last_clean":"2024-09-11T23:18:22.295792+0000","last_became_active":"2024-09-11T23:16:24.075527+0000","last_became_peered":"2024-09-11T23:16:24.075527+0000","last_unstale":"2024-09-11T23:18:22.295792+0000","last_undegraded":"2024-09-11T23:18:22.295792+0000","last_fullsized":"2024-09-11T23:18:22.295792+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:27:54.622059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"85'6","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.184380+0000","last_change":"2024-09-11T23:16:35.947475+0000","last_active":"2024-09-11T23:18:22.184380+0000","last_peered":"2024-09-11T23:18:22.184380+0000","last_clean":"2024-09-11T23:18:22.184380+0000","last_became_active":"2024-09-11T23:16:35.947028+0000","last_became_peered":"2024-09-11T23:16:35.947028+0000","last_unstale":"2024-09-11T23:18:22.184380+0000","last_undegraded":"2024-09-11T23:18:22.184380+0000","last_fullsized":"2024-09-11T23:18:22.184380+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:18:13.479554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'2","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.296198+0000","last_change":"2024-09-11T23:16:23.793298+0000","last_active":"2024-09-11T23:18:22.296198+0000","last_peered":"2024-09-11T23:18:22.296198+0000","last_clean":"2024-09-11T23:18:22.296198+0000","last_became_active":"2024-09-11T23:16:23.792947+0000","last_became_peered":"2024-09-11T23:16:23.792947+0000","last_unstale":"2024-09-11T23:18:22.296198+0000","last_undegraded":"2024-09-11T23:18:22.296198+0000","last_fullsized":"2024-09-11T23:18:22.296198+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:32:21.294560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"85'5","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328871+0000","last_change":"2024-09-11T23:16:25.902348+0000","last_active":"2024-09-11T23:18:22.328871+0000","last_peered":"2024-09-11T23:18:22.328871+0000","last_clean":"2024-09-11T23:18:22.328871+0000","last_became_active":"2024-09-11T23:16:25.901686+0000","last_became_peered":"2024-09-11T23:16:25.901686+0000","last_unstale":"2024-09-11T23:18:22.328871+0000","last_undegraded":"2024-09-11T23:18:22.328871+0000","last_fullsized":"2024-09-11T23:18:22.328871+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:00:15.543003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"85'5","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.330216+0000","last_change":"2024-09-11T23:16:23.790984+0000","last_active":"2024-09-11T23:18:22.330216+0000","last_peered":"2024-09-11T23:18:22.330216+0000","last_clean":"2024-09-11T23:18:22.330216+0000","last_became_active":"2024-09-11T23:16:23.790569+0000","last_became_peered":"2024-09-11T23:16:23.790569+0000","last_unstale":"2024-09-11T23:18:22.330216+0000","last_undegraded":"2024-09-11T23:18:22.330216+0000","last_fullsized":"2024-09-11T23:18:22.330216+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:57:39.675793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"85'9","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328796+0000","last_change":"2024-09-11T23:16:25.899249+0000","last_active":"2024-09-11T23:18:22.328796+0000","last_peered":"2024-09-11T23:18:22.328796+0000","last_clean":"2024-09-11T23:18:22.328796+0000","last_became_active":"2024-09-11T23:16:25.898914+0000","last_became_peered":"2024-09-11T23:16:25.898914+0000","last_unstale":"2024-09-11T23:18:22.328796+0000","last_undegraded":"2024-09-11T23:18:22.328796+0000","last_fullsized":"2024-09-11T23:18:22.328796+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:17:07.545770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"86'41","reported_seq":73,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.429528+0000","last_change":"2024-09-11T23:16:23.787387+0000","last_active":"2024-09-11T23:18:22.429528+0000","last_peered":"2024-09-11T23:18:22.429528+0000","last_clean":"2024-09-11T23:18:22.429528+0000","last_became_active":"2024-09-11T23:16:23.787014+0000","last_became_peered":"2024-09-11T23:16:23.787014+0000","last_unstale":"2024-09-11T23:18:22.429528+0000","last_undegraded":"2024-09-11T23:18:22.429528+0000","last_fullsized":"2024-09-11T23:18:22.429528+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:45:52.004253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"85'7","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328671+0000","last_change":"2024-09-11T23:16:25.908304+0000","last_active":"2024-09-11T23:18:22.328671+0000","last_peered":"2024-09-11T23:18:22.328671+0000","last_clean":"2024-09-11T23:18:22.328671+0000","last_became_active":"2024-09-11T23:16:25.907900+0000","last_became_peered":"2024-09-11T23:16:25.907900+0000","last_unstale":"2024-09-11T23:18:22.328671+0000","last_undegraded":"2024-09-11T23:18:22.328671+0000","last_fullsized":"2024-09-11T23:18:22.328671+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:43:26.247682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"85'7","reported_seq":145,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:27.129070+0000","last_change":"2024-09-11T23:16:24.074610+0000","last_active":"2024-09-11T23:18:27.129070+0000","last_peered":"2024-09-11T23:18:27.129070+0000","last_clean":"2024-09-11T23:18:27.129070+0000","last_became_active":"2024-09-11T23:16:24.074210+0000","last_became_peered":"2024-09-11T23:16:24.074210+0000","last_unstale":"2024-09-11T23:18:27.129070+0000","last_undegraded":"2024-09-11T23:18:27.129070+0000","last_fullsized":"2024-09-11T23:18:27.129070+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:09:37.167050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1704,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":114,"num_read_kb":4,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"85'7","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328842+0000","last_change":"2024-09-11T23:16:25.903120+0000","last_active":"2024-09-11T23:18:22.328842+0000","last_peered":"2024-09-11T23:18:22.328842+0000","last_clean":"2024-09-11T23:18:22.328842+0000","last_became_active":"2024-09-11T23:16:25.902770+0000","last_became_peered":"2024-09-11T23:16:25.902770+0000","last_unstale":"2024-09-11T23:18:22.328842+0000","last_undegraded":"2024-09-11T23:18:22.328842+0000","last_fullsized":"2024-09-11T23:18:22.328842+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:03:30.041088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"86'148","reported_seq":393,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:24.295645+0000","last_change":"2024-09-11T23:15:13.115055+0000","last_active":"2024-09-11T23:18:24.295645+0000","last_peered":"2024-09-11T23:18:24.295645+0000","last_clean":"2024-09-11T23:18:24.295645+0000","last_became_active":"2024-09-11T23:15:13.114480+0000","last_became_peered":"2024-09-11T23:15:13.114480+0000","last_unstale":"2024-09-11T23:18:24.295645+0000","last_undegraded":"2024-09-11T23:18:24.295645+0000","last_fullsized":"2024-09-11T23:18:24.295645+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:11:50.495799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:11:50.495799+0000","last_clean_scrub_stamp":"2024-09-11T23:11:50.495799+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:15:53.681230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"85'6","reported_seq":33,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.187094+0000","last_change":"2024-09-11T23:16:23.793996+0000","last_active":"2024-09-11T23:18:22.187094+0000","last_peered":"2024-09-11T23:18:22.187094+0000","last_clean":"2024-09-11T23:18:22.187094+0000","last_became_active":"2024-09-11T23:16:23.793681+0000","last_became_peered":"2024-09-11T23:16:23.793681+0000","last_unstale":"2024-09-11T23:18:22.187094+0000","last_undegraded":"2024-09-11T23:18:22.187094+0000","last_fullsized":"2024-09-11T23:18:22.187094+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:52:50.463623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"85'8","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.182924+0000","last_change":"2024-09-11T23:16:25.905133+0000","last_active":"2024-09-11T23:18:22.182924+0000","last_peered":"2024-09-11T23:18:22.182924+0000","last_clean":"2024-09-11T23:18:22.182924+0000","last_became_active":"2024-09-11T23:16:25.904819+0000","last_became_peered":"2024-09-11T23:16:25.904819+0000","last_unstale":"2024-09-11T23:18:22.182924+0000","last_undegraded":"2024-09-11T23:18:22.182924+0000","last_fullsized":"2024-09-11T23:18:22.182924+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:51:09.664343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"85'4","reported_seq":35,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328418+0000","last_change":"2024-09-11T23:16:36.228806+0000","last_active":"2024-09-11T23:18:22.328418+0000","last_peered":"2024-09-11T23:18:22.328418+0000","last_clean":"2024-09-11T23:18:22.328418+0000","last_became_active":"2024-09-11T23:16:36.228311+0000","last_became_peered":"2024-09-11T23:16:36.228311+0000","last_unstale":"2024-09-11T23:18:22.328418+0000","last_undegraded":"2024-09-11T23:18:22.328418+0000","last_fullsized":"2024-09-11T23:18:22.328418+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:42:32.856691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4],"acting":[7,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"85'5","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328756+0000","last_change":"2024-09-11T23:16:25.898935+0000","last_active":"2024-09-11T23:18:22.328756+0000","last_peered":"2024-09-11T23:18:22.328756+0000","last_clean":"2024-09-11T23:18:22.328756+0000","last_became_active":"2024-09-11T23:16:25.898618+0000","last_became_peered":"2024-09-11T23:16:25.898618+0000","last_unstale":"2024-09-11T23:18:22.328756+0000","last_undegraded":"2024-09-11T23:18:22.328756+0000","last_fullsized":"2024-09-11T23:18:22.328756+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:13:29.104911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"85'5","reported_seq":141,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:28.129742+0000","last_change":"2024-09-11T23:16:24.073613+0000","last_active":"2024-09-11T23:18:28.129742+0000","last_peered":"2024-09-11T23:18:28.129742+0000","last_clean":"2024-09-11T23:18:28.129742+0000","last_became_active":"2024-09-11T23:16:24.073262+0000","last_became_peered":"2024-09-11T23:16:24.073262+0000","last_unstale":"2024-09-11T23:18:28.129742+0000","last_undegraded":"2024-09-11T23:18:28.129742+0000","last_fullsized":"2024-09-11T23:18:28.129742+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:31:58.653657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":111,"num_read_kb":111,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"85'8","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.184479+0000","last_change":"2024-09-11T23:16:25.903725+0000","last_active":"2024-09-11T23:18:22.184479+0000","last_peered":"2024-09-11T23:18:22.184479+0000","last_clean":"2024-09-11T23:18:22.184479+0000","last_became_active":"2024-09-11T23:16:25.903375+0000","last_became_peered":"2024-09-11T23:16:25.903375+0000","last_unstale":"2024-09-11T23:18:22.184479+0000","last_undegraded":"2024-09-11T23:18:22.184479+0000","last_fullsized":"2024-09-11T23:18:22.184479+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:55:02.779575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"86'6","reported_seq":143,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:28.130111+0000","last_change":"2024-09-11T23:16:23.793932+0000","last_active":"2024-09-11T23:18:28.130111+0000","last_peered":"2024-09-11T23:18:28.130111+0000","last_clean":"2024-09-11T23:18:28.130111+0000","last_became_active":"2024-09-11T23:16:23.793111+0000","last_became_peered":"2024-09-11T23:16:23.793111+0000","last_unstale":"2024-09-11T23:18:28.130111+0000","last_undegraded":"2024-09-11T23:18:28.130111+0000","last_fullsized":"2024-09-11T23:18:28.130111+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:14:09.087222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":112,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"85'7","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.302982+0000","last_change":"2024-09-11T23:16:25.905020+0000","last_active":"2024-09-11T23:18:22.302982+0000","last_peered":"2024-09-11T23:18:22.302982+0000","last_clean":"2024-09-11T23:18:22.302982+0000","last_became_active":"2024-09-11T23:16:25.904675+0000","last_became_peered":"2024-09-11T23:16:25.904675+0000","last_unstale":"2024-09-11T23:18:22.302982+0000","last_undegraded":"2024-09-11T23:18:22.302982+0000","last_fullsized":"2024-09-11T23:18:22.302982+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:13:57.388773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"85'1","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.293261+0000","last_change":"2024-09-11T23:16:24.073630+0000","last_active":"2024-09-11T23:18:22.293261+0000","last_peered":"2024-09-11T23:18:22.293261+0000","last_clean":"2024-09-11T23:18:22.293261+0000","last_became_active":"2024-09-11T23:16:24.073301+0000","last_became_peered":"2024-09-11T23:16:24.073301+0000","last_unstale":"2024-09-11T23:18:22.293261+0000","last_undegraded":"2024-09-11T23:18:22.293261+0000","last_fullsized":"2024-09-11T23:18:22.293261+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:41:59.374581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"85'5","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.303051+0000","last_change":"2024-09-11T23:16:25.894124+0000","last_active":"2024-09-11T23:18:22.303051+0000","last_peered":"2024-09-11T23:18:22.303051+0000","last_clean":"2024-09-11T23:18:22.303051+0000","last_became_active":"2024-09-11T23:16:25.893751+0000","last_became_peered":"2024-09-11T23:16:25.893751+0000","last_unstale":"2024-09-11T23:18:22.303051+0000","last_undegraded":"2024-09-11T23:18:22.303051+0000","last_fullsized":"2024-09-11T23:18:22.303051+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:00:28.265206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"85'4","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.296214+0000","last_change":"2024-09-11T23:16:24.073700+0000","last_active":"2024-09-11T23:18:22.296214+0000","last_peered":"2024-09-11T23:18:22.296214+0000","last_clean":"2024-09-11T23:18:22.296214+0000","last_became_active":"2024-09-11T23:16:24.073283+0000","last_became_peered":"2024-09-11T23:16:24.073283+0000","last_unstale":"2024-09-11T23:18:22.296214+0000","last_undegraded":"2024-09-11T23:18:22.296214+0000","last_fullsized":"2024-09-11T23:18:22.296214+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:41:43.536834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"85'9","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.329920+0000","last_change":"2024-09-11T23:16:25.900532+0000","last_active":"2024-09-11T23:18:22.329920+0000","last_peered":"2024-09-11T23:18:22.329920+0000","last_clean":"2024-09-11T23:18:22.329920+0000","last_became_active":"2024-09-11T23:16:25.899466+0000","last_became_peered":"2024-09-11T23:16:25.899466+0000","last_unstale":"2024-09-11T23:18:22.329920+0000","last_undegraded":"2024-09-11T23:18:22.329920+0000","last_fullsized":"2024-09-11T23:18:22.329920+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:10:01.152323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"86'50","reported_seq":97,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.432609+0000","last_change":"2024-09-11T23:16:23.788789+0000","last_active":"2024-09-11T23:18:22.432609+0000","last_peered":"2024-09-11T23:18:22.432609+0000","last_clean":"2024-09-11T23:18:22.432609+0000","last_became_active":"2024-09-11T23:16:23.788422+0000","last_became_peered":"2024-09-11T23:16:23.788422+0000","last_unstale":"2024-09-11T23:18:22.432609+0000","last_undegraded":"2024-09-11T23:18:22.432609+0000","last_fullsized":"2024-09-11T23:18:22.432609+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":0,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:09:14.310205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":89,"num_write_kb":837,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"85'5","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.183270+0000","last_change":"2024-09-11T23:16:25.895511+0000","last_active":"2024-09-11T23:18:22.183270+0000","last_peered":"2024-09-11T23:18:22.183270+0000","last_clean":"2024-09-11T23:18:22.183270+0000","last_became_active":"2024-09-11T23:16:25.895143+0000","last_became_peered":"2024-09-11T23:16:25.895143+0000","last_unstale":"2024-09-11T23:18:22.183270+0000","last_undegraded":"2024-09-11T23:18:22.183270+0000","last_fullsized":"2024-09-11T23:18:22.183270+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:31:16.498371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"85'7","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.187209+0000","last_change":"2024-09-11T23:16:24.019064+0000","last_active":"2024-09-11T23:18:22.187209+0000","last_peered":"2024-09-11T23:18:22.187209+0000","last_clean":"2024-09-11T23:18:22.187209+0000","last_became_active":"2024-09-11T23:16:24.018658+0000","last_became_peered":"2024-09-11T23:16:24.018658+0000","last_unstale":"2024-09-11T23:18:22.187209+0000","last_undegraded":"2024-09-11T23:18:22.187209+0000","last_fullsized":"2024-09-11T23:18:22.187209+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:17:41.009507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"85'7","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.544339+0000","last_change":"2024-09-11T23:16:25.903553+0000","last_active":"2024-09-11T23:18:22.544339+0000","last_peered":"2024-09-11T23:18:22.544339+0000","last_clean":"2024-09-11T23:18:22.544339+0000","last_became_active":"2024-09-11T23:16:25.903221+0000","last_became_peered":"2024-09-11T23:16:25.903221+0000","last_unstale":"2024-09-11T23:18:22.544339+0000","last_undegraded":"2024-09-11T23:18:22.544339+0000","last_fullsized":"2024-09-11T23:18:22.544339+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:16:51.872517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"83'2","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328140+0000","last_change":"2024-09-11T23:16:24.073784+0000","last_active":"2024-09-11T23:18:22.328140+0000","last_peered":"2024-09-11T23:18:22.328140+0000","last_clean":"2024-09-11T23:18:22.328140+0000","last_became_active":"2024-09-11T23:16:24.073406+0000","last_became_peered":"2024-09-11T23:16:24.073406+0000","last_unstale":"2024-09-11T23:18:22.328140+0000","last_undegraded":"2024-09-11T23:18:22.328140+0000","last_fullsized":"2024-09-11T23:18:22.328140+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:31:24.617025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"85'2","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.295796+0000","last_change":"2024-09-11T23:16:25.894878+0000","last_active":"2024-09-11T23:18:22.295796+0000","last_peered":"2024-09-11T23:18:22.295796+0000","last_clean":"2024-09-11T23:18:22.295796+0000","last_became_active":"2024-09-11T23:16:25.894543+0000","last_became_peered":"2024-09-11T23:16:25.894543+0000","last_unstale":"2024-09-11T23:18:22.295796+0000","last_undegraded":"2024-09-11T23:18:22.295796+0000","last_fullsized":"2024-09-11T23:18:22.295796+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:34:13.435754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"85'7","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.182832+0000","last_change":"2024-09-11T23:16:23.786502+0000","last_active":"2024-09-11T23:18:22.182832+0000","last_peered":"2024-09-11T23:18:22.182832+0000","last_clean":"2024-09-11T23:18:22.182832+0000","last_became_active":"2024-09-11T23:16:23.786182+0000","last_became_peered":"2024-09-11T23:16:23.786182+0000","last_unstale":"2024-09-11T23:18:22.182832+0000","last_undegraded":"2024-09-11T23:18:22.182832+0000","last_fullsized":"2024-09-11T23:18:22.182832+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:59:07.978244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"85'9","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328520+0000","last_change":"2024-09-11T23:16:25.899625+0000","last_active":"2024-09-11T23:18:22.328520+0000","last_peered":"2024-09-11T23:18:22.328520+0000","last_clean":"2024-09-11T23:18:22.328520+0000","last_became_active":"2024-09-11T23:16:25.899265+0000","last_became_peered":"2024-09-11T23:16:25.899265+0000","last_unstale":"2024-09-11T23:18:22.328520+0000","last_undegraded":"2024-09-11T23:18:22.328520+0000","last_fullsized":"2024-09-11T23:18:22.328520+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:36:06.907622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"86'4","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328472+0000","last_change":"2024-09-11T23:16:36.228991+0000","last_active":"2024-09-11T23:18:22.328472+0000","last_peered":"2024-09-11T23:18:22.328472+0000","last_clean":"2024-09-11T23:18:22.328472+0000","last_became_active":"2024-09-11T23:16:36.227701+0000","last_became_peered":"2024-09-11T23:16:36.227701+0000","last_unstale":"2024-09-11T23:18:22.328472+0000","last_undegraded":"2024-09-11T23:18:22.328472+0000","last_fullsized":"2024-09-11T23:18:22.328472+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:37:57.478572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"85'8","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.182913+0000","last_change":"2024-09-11T23:16:25.894803+0000","last_active":"2024-09-11T23:18:22.182913+0000","last_peered":"2024-09-11T23:18:22.182913+0000","last_clean":"2024-09-11T23:18:22.182913+0000","last_became_active":"2024-09-11T23:16:25.894429+0000","last_became_peered":"2024-09-11T23:16:25.894429+0000","last_unstale":"2024-09-11T23:18:22.182913+0000","last_undegraded":"2024-09-11T23:18:22.182913+0000","last_fullsized":"2024-09-11T23:18:22.182913+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:39:15.868910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"83'4","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.295871+0000","last_change":"2024-09-11T23:16:23.794271+0000","last_active":"2024-09-11T23:18:22.295871+0000","last_peered":"2024-09-11T23:18:22.295871+0000","last_clean":"2024-09-11T23:18:22.295871+0000","last_became_active":"2024-09-11T23:16:23.793919+0000","last_became_peered":"2024-09-11T23:16:23.793919+0000","last_unstale":"2024-09-11T23:18:22.295871+0000","last_undegraded":"2024-09-11T23:18:22.295871+0000","last_fullsized":"2024-09-11T23:18:22.295871+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:30:28.907312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"85'5","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328395+0000","last_change":"2024-09-11T23:16:25.900969+0000","last_active":"2024-09-11T23:18:22.328395+0000","last_peered":"2024-09-11T23:18:22.328395+0000","last_clean":"2024-09-11T23:18:22.328395+0000","last_became_active":"2024-09-11T23:16:25.900425+0000","last_became_peered":"2024-09-11T23:16:25.900425+0000","last_unstale":"2024-09-11T23:18:22.328395+0000","last_undegraded":"2024-09-11T23:18:22.328395+0000","last_fullsized":"2024-09-11T23:18:22.328395+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:03:25.185773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"85'15","reported_seq":34,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.184711+0000","last_change":"2024-09-11T23:16:25.911605+0000","last_active":"2024-09-11T23:18:22.184711+0000","last_peered":"2024-09-11T23:18:22.184711+0000","last_clean":"2024-09-11T23:18:22.184711+0000","last_became_active":"2024-09-11T23:16:25.911180+0000","last_became_peered":"2024-09-11T23:16:25.911180+0000","last_unstale":"2024-09-11T23:18:22.184711+0000","last_undegraded":"2024-09-11T23:18:22.184711+0000","last_fullsized":"2024-09-11T23:18:22.184711+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:55:38.062606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"85'5","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328425+0000","last_change":"2024-09-11T23:16:35.949528+0000","last_active":"2024-09-11T23:18:22.328425+0000","last_peered":"2024-09-11T23:18:22.328425+0000","last_clean":"2024-09-11T23:18:22.328425+0000","last_became_active":"2024-09-11T23:16:35.945679+0000","last_became_peered":"2024-09-11T23:16:35.945679+0000","last_unstale":"2024-09-11T23:18:22.328425+0000","last_undegraded":"2024-09-11T23:18:22.328425+0000","last_fullsized":"2024-09-11T23:18:22.328425+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:14:24.706044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"85'9","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328158+0000","last_change":"2024-09-11T23:16:25.900526+0000","last_active":"2024-09-11T23:18:22.328158+0000","last_peered":"2024-09-11T23:18:22.328158+0000","last_clean":"2024-09-11T23:18:22.328158+0000","last_became_active":"2024-09-11T23:16:25.900148+0000","last_became_peered":"2024-09-11T23:16:25.900148+0000","last_unstale":"2024-09-11T23:18:22.328158+0000","last_undegraded":"2024-09-11T23:18:22.328158+0000","last_fullsized":"2024-09-11T23:18:22.328158+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:26:10.582473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"86'13","reported_seq":48,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.296144+0000","last_change":"2024-09-11T23:16:36.233273+0000","last_active":"2024-09-11T23:18:22.296144+0000","last_peered":"2024-09-11T23:18:22.296144+0000","last_clean":"2024-09-11T23:18:22.296144+0000","last_became_active":"2024-09-11T23:16:36.228358+0000","last_became_peered":"2024-09-11T23:16:36.228358+0000","last_unstale":"2024-09-11T23:18:22.296144+0000","last_undegraded":"2024-09-11T23:18:22.296144+0000","last_fullsized":"2024-09-11T23:18:22.296144+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:06:31.225795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"85'6","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.544571+0000","last_change":"2024-09-11T23:16:25.907874+0000","last_active":"2024-09-11T23:18:22.544571+0000","last_peered":"2024-09-11T23:18:22.544571+0000","last_clean":"2024-09-11T23:18:22.544571+0000","last_became_active":"2024-09-11T23:16:25.907491+0000","last_became_peered":"2024-09-11T23:16:25.907491+0000","last_unstale":"2024-09-11T23:18:22.544571+0000","last_undegraded":"2024-09-11T23:18:22.544571+0000","last_fullsized":"2024-09-11T23:18:22.544571+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:09:20.984249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"84'2","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.183326+0000","last_change":"2024-09-11T23:16:24.074355+0000","last_active":"2024-09-11T23:18:22.183326+0000","last_peered":"2024-09-11T23:18:22.183326+0000","last_clean":"2024-09-11T23:18:22.183326+0000","last_became_active":"2024-09-11T23:16:24.074047+0000","last_became_peered":"2024-09-11T23:16:24.074047+0000","last_unstale":"2024-09-11T23:18:22.183326+0000","last_undegraded":"2024-09-11T23:18:22.183326+0000","last_fullsized":"2024-09-11T23:18:22.183326+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:31:58.804420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"85'5","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.182923+0000","last_change":"2024-09-11T23:16:25.907701+0000","last_active":"2024-09-11T23:18:22.182923+0000","last_peered":"2024-09-11T23:18:22.182923+0000","last_clean":"2024-09-11T23:18:22.182923+0000","last_became_active":"2024-09-11T23:16:25.907379+0000","last_became_peered":"2024-09-11T23:16:25.907379+0000","last_unstale":"2024-09-11T23:18:22.182923+0000","last_undegraded":"2024-09-11T23:18:22.182923+0000","last_fullsized":"2024-09-11T23:18:22.182923+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:12:03.639055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"86'14","reported_seq":150,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:28.129607+0000","last_change":"2024-09-11T23:16:23.794293+0000","last_active":"2024-09-11T23:18:28.129607+0000","last_peered":"2024-09-11T23:18:28.129607+0000","last_clean":"2024-09-11T23:18:28.129607+0000","last_became_active":"2024-09-11T23:16:23.793974+0000","last_became_peered":"2024-09-11T23:16:23.793974+0000","last_unstale":"2024-09-11T23:18:28.129607+0000","last_undegraded":"2024-09-11T23:18:28.129607+0000","last_fullsized":"2024-09-11T23:18:28.129607+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:24:39.170188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":111,"num_read_kb":111,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":599346,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1239,"num_read_kb":734,"num_write":372,"num_write_kb":1462,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2174976,"data_stored":1798038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":298,"ondisk_log_size":298,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":11,"up_from":77,"seq":330712481830,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32128,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713024,"statfs":{"total":95995035648,"available":95962136576,"internally_reserved":0,"allocated":962560,"data_stored":459042,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":70,"seq":300647710762,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28844,"kb_used_data":1760,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716308,"statfs":{"total":95995035648,"available":95965499392,"internally_reserved":0,"allocated":1802240,"data_stored":1247035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":65,"seq":279172874288,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28372,"kb_used_data":1344,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716780,"statfs":{"total":95995035648,"available":95965982720,"internally_reserved":0,"allocated":1376256,"data_stored":851072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":57,"seq":244813135925,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28360,"kb_used_data":1336,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716792,"statfs":{"total":95995035648,"available":95965995008,"internally_reserved":0,"allocated":1368064,"data_stored":850868,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":50,"seq":214748364857,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28568,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716584,"statfs":{"total":95995035648,"available":95965782016,"internally_reserved":0,"allocated":1576960,"data_stored":1051455,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528381,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36292,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708860,"statfs":{"total":95995035648,"available":95957872640,"internally_reserved":0,"allocated":966656,"data_stored":457966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790020,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28768,"kb_used_data":1684,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716384,"statfs":{"total":95995035648,"available":95965577216,"internally_reserved":0,"allocated":1724416,"data_stored":1182340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920841,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36292,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708860,"statfs":{"total":95995035648,"available":95957872640,"internally_reserved":0,"allocated":966656,"data_stored":457434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182478,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":41160,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93703992,"statfs":{"total":95995035648,"available":95952887808,"internally_reserved":0,"allocated":1511424,"data_stored":982214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411411,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36380,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708772,"statfs":{"total":95995035648,"available":95957782528,"internally_reserved":0,"allocated":1056768,"data_stored":526421,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574936,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1036288,"data_stored":525359,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738461,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":987136,"data_stored":459446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":68625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":69687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":557056,"data_stored":525246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":180224,"data_stored":135121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":421888,"data_stored":394134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":394338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":258048,"data_stored":199816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2308,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T23:18:33.990 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-11T23:18:34.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:33 smithi076 ceph-mon[35296]: pgmap v8: 97 pgs: 97 active+clean; 1.1 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 0 B/s wr, 19 op/s 2024-09-11T23:18:34.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:33 smithi136 ceph-mon[35121]: pgmap v8: 97 pgs: 97 active+clean; 1.1 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 0 B/s wr, 19 op/s 2024-09-11T23:18:34.256 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:18:34.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:34.761+0000 7f8f006ae640 1 -- 172.21.15.26:0/2516302288 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ef81036e0 msgr2=0x7f8ef8107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:34.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:34.761+0000 7f8f006ae640 1 --2- 172.21.15.26:0/2516302288 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ef81036e0 0x7f8ef8107b90 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f8ef4009930 tx=0x7f8ef402a420 comp rx=0 tx=0).stop 2024-09-11T23:18:34.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:34.761+0000 7f8f006ae640 1 -- 172.21.15.26:0/2516302288 shutdown_connections 2024-09-11T23:18:34.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:34.761+0000 7f8f006ae640 1 --2- 172.21.15.26:0/2516302288 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8ef81082a0 0x7f8ef810ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:34.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:34.761+0000 7f8f006ae640 1 --2- 172.21.15.26:0/2516302288 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ef81036e0 0x7f8ef8107b90 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:34.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:34.761+0000 7f8f006ae640 1 --2- 172.21.15.26:0/2516302288 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8ef8102d30 0x7f8ef8103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:34.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:34.761+0000 7f8f006ae640 1 -- 172.21.15.26:0/2516302288 >> 172.21.15.26:0/2516302288 conn(0x7f8ef80fc7d0 msgr2=0x7f8ef80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:18:34.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:34.761+0000 7f8f006ae640 1 -- 172.21.15.26:0/2516302288 shutdown_connections 2024-09-11T23:18:34.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:34.761+0000 7f8f006ae640 1 -- 172.21.15.26:0/2516302288 wait complete. 2024-09-11T23:18:34.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:34.762+0000 7f8f006ae640 1 Processor -- start 2024-09-11T23:18:34.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:34.762+0000 7f8f006ae640 1 -- start start 2024-09-11T23:18:34.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:34.762+0000 7f8f006ae640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ef8102d30 0x7f8ef81985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:34.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:34.762+0000 7f8f006ae640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8ef81036e0 0x7f8ef8198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:34.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:34.762+0000 7f8f006ae640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8ef81082a0 0x7f8ef819ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:34.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:34.762+0000 7f8f006ae640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f8ef81120a0 con 0x7f8ef8102d30 2024-09-11T23:18:34.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:34.762+0000 7f8f006ae640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f8ef8111f20 con 0x7f8ef81036e0 2024-09-11T23:18:34.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:34.762+0000 7f8f006ae640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f8ef8112220 con 0x7f8ef81082a0 2024-09-11T23:18:34.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:34.762+0000 7f8efe44a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ef8102d30 0x7f8ef81985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:34.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:34.762+0000 7f8efe44a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ef8102d30 0x7f8ef81985c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60598/0 (socket says 172.21.15.26:60598) 2024-09-11T23:18:34.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:34.762+0000 7f8efe44a640 1 -- 172.21.15.26:0/1091536025 learned_addr learned my addr 172.21.15.26:0/1091536025 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:18:34.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:34.762+0000 7f8efec4b640 1 --2- 172.21.15.26:0/1091536025 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8ef81082a0 0x7f8ef819ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:34.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:34.762+0000 7f8efdc49640 1 --2- 172.21.15.26:0/1091536025 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8ef81036e0 0x7f8ef8198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:34.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:34.763+0000 7f8efe44a640 1 -- 172.21.15.26:0/1091536025 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8ef81082a0 msgr2=0x7f8ef819ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:34.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:34.763+0000 7f8efe44a640 1 --2- 172.21.15.26:0/1091536025 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8ef81082a0 0x7f8ef819ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:34.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:34.763+0000 7f8efe44a640 1 -- 172.21.15.26:0/1091536025 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8ef81036e0 msgr2=0x7f8ef8198b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:34.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:34.763+0000 7f8efe44a640 1 --2- 172.21.15.26:0/1091536025 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8ef81036e0 0x7f8ef8198b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:34.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:34.763+0000 7f8efe44a640 1 -- 172.21.15.26:0/1091536025 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ef819d570 con 0x7f8ef8102d30 2024-09-11T23:18:34.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:34.763+0000 7f8efe44a640 1 --2- 172.21.15.26:0/1091536025 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ef8102d30 0x7f8ef81985c0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f8eec00eab0 tx=0x7f8eec00ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:18:34.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:34.763+0000 7f8eeb7fe640 1 -- 172.21.15.26:0/1091536025 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eec00ce40 con 0x7f8ef8102d30 2024-09-11T23:18:34.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:34.763+0000 7f8eeb7fe640 1 -- 172.21.15.26:0/1091536025 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8eec011950 con 0x7f8ef8102d30 2024-09-11T23:18:34.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:34.763+0000 7f8f006ae640 1 -- 172.21.15.26:0/1091536025 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ef819d860 con 0x7f8ef8102d30 2024-09-11T23:18:34.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:34.763+0000 7f8eeb7fe640 1 -- 172.21.15.26:0/1091536025 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eec010ae0 con 0x7f8ef8102d30 2024-09-11T23:18:34.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:34.763+0000 7f8f006ae640 1 -- 172.21.15.26:0/1091536025 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ef81a5140 con 0x7f8ef8102d30 2024-09-11T23:18:34.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:34.764+0000 7f8f006ae640 1 -- 172.21.15.26:0/1091536025 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ef8103110 con 0x7f8ef8102d30 2024-09-11T23:18:34.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:34.765+0000 7f8eeb7fe640 1 -- 172.21.15.26:0/1091536025 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 24) ==== 138009+0+0 (secure 0 0 0) 0x7f8eec011b00 con 0x7f8ef8102d30 2024-09-11T23:18:34.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:34.766+0000 7f8eeb7fe640 1 --2- 172.21.15.26:0/1091536025 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f8ee40a2e00 0x7f8ee40a52c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:34.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:34.766+0000 7f8efdc49640 1 --2- 172.21.15.26:0/1091536025 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f8ee40a2e00 0x7f8ee40a52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:34.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:34.766+0000 7f8eeb7fe640 1 -- 172.21.15.26:0/1091536025 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f8eec0cfbc0 con 0x7f8ef8102d30 2024-09-11T23:18:34.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:34.767+0000 7f8efdc49640 1 --2- 172.21.15.26:0/1091536025 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f8ee40a2e00 0x7f8ee40a52c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f8ef4005940 tx=0x7f8ef40058d0 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:18:34.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:34.769+0000 7f8eeb7fe640 1 -- 172.21.15.26:0/1091536025 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8eec016030 con 0x7f8ef8102d30 2024-09-11T23:18:34.942 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:34 smithi026 ceph-mon[31172]: from='client.15369 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T23:18:34.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:34.941+0000 7f8f006ae640 1 -- 172.21.15.26:0/1091536025 --> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8ef8062630 con 0x7f8ee40a2e00 2024-09-11T23:18:34.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:34.948+0000 7f8eeb7fe640 1 -- 172.21.15.26:0/1091536025 <== mgr.15171 v2:172.21.15.26:6800/1989555790 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+257805 (secure 0 0 0) 0x7f8ef8062630 con 0x7f8ee40a2e00 2024-09-11T23:18:34.950 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-11T23:18:34.951 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-09-11T23:18:34.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:34.951+0000 7f8f006ae640 1 -- 172.21.15.26:0/1091536025 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f8ee40a2e00 msgr2=0x7f8ee40a52c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:34.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:34.951+0000 7f8f006ae640 1 --2- 172.21.15.26:0/1091536025 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f8ee40a2e00 0x7f8ee40a52c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f8ef4005940 tx=0x7f8ef40058d0 comp rx=0 tx=0).stop 2024-09-11T23:18:34.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:34.951+0000 7f8f006ae640 1 -- 172.21.15.26:0/1091536025 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ef8102d30 msgr2=0x7f8ef81985c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:34.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:34.951+0000 7f8f006ae640 1 --2- 172.21.15.26:0/1091536025 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ef8102d30 0x7f8ef81985c0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f8eec00eab0 tx=0x7f8eec00ef70 comp rx=0 tx=0).stop 2024-09-11T23:18:34.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:34.951+0000 7f8f006ae640 1 -- 172.21.15.26:0/1091536025 shutdown_connections 2024-09-11T23:18:34.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:34.951+0000 7f8f006ae640 1 --2- 172.21.15.26:0/1091536025 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f8ee40a2e00 0x7f8ee40a52c0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:34.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:34.951+0000 7f8f006ae640 1 --2- 172.21.15.26:0/1091536025 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8ef81036e0 0x7f8ef8198b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:34.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:34.951+0000 7f8f006ae640 1 --2- 172.21.15.26:0/1091536025 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8ef81082a0 0x7f8ef819ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:34.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:34.951+0000 7f8f006ae640 1 --2- 172.21.15.26:0/1091536025 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ef8102d30 0x7f8ef81985c0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:34.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:34.951+0000 7f8f006ae640 1 -- 172.21.15.26:0/1091536025 >> 172.21.15.26:0/1091536025 conn(0x7f8ef80fc7d0 msgr2=0x7f8ef8105de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:18:34.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:34.951+0000 7f8f006ae640 1 -- 172.21.15.26:0/1091536025 shutdown_connections 2024-09-11T23:18:34.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:34.951+0000 7f8f006ae640 1 -- 172.21.15.26:0/1091536025 wait complete. 2024-09-11T23:18:35.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:34 smithi076 ceph-mon[35296]: from='client.15369 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T23:18:35.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:34 smithi136 ceph-mon[35121]: from='client.15369 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T23:18:35.642 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":9,"stamp":"2024-09-11T23:18:34.199197+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1386580,"num_objects":104,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1781,"num_read_kb":1321,"num_write":701,"num_write_kb":5316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1027,"ondisk_log_size":1027,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":381824,"kb_used_data":15552,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124560000,"statfs":{"total":1151940427776,"available":1151549440000,"internally_reserved":0,"allocated":15925248,"data_stored":9640548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":374453561},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":196632,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":456,"num_read_kb":493,"num_write":146,"num_write_kb":914,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.993892"},"pg_stats":[{"pgid":"3.3f","version":"85'8","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.329092+0000","last_change":"2024-09-11T23:16:25.899106+0000","last_active":"2024-09-11T23:18:22.329092+0000","last_peered":"2024-09-11T23:18:22.329092+0000","last_clean":"2024-09-11T23:18:22.329092+0000","last_became_active":"2024-09-11T23:16:25.898654+0000","last_became_peered":"2024-09-11T23:16:25.898654+0000","last_unstale":"2024-09-11T23:18:22.329092+0000","last_undegraded":"2024-09-11T23:18:22.329092+0000","last_fullsized":"2024-09-11T23:18:22.329092+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:21:41.490905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"85'4","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.544255+0000","last_change":"2024-09-11T23:16:25.895629+0000","last_active":"2024-09-11T23:18:22.544255+0000","last_peered":"2024-09-11T23:18:22.544255+0000","last_clean":"2024-09-11T23:18:22.544255+0000","last_became_active":"2024-09-11T23:16:25.895308+0000","last_became_peered":"2024-09-11T23:16:25.895308+0000","last_unstale":"2024-09-11T23:18:22.544255+0000","last_undegraded":"2024-09-11T23:18:22.544255+0000","last_fullsized":"2024-09-11T23:18:22.544255+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:39:42.549970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"85'7","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.544176+0000","last_change":"2024-09-11T23:16:25.907113+0000","last_active":"2024-09-11T23:18:22.544176+0000","last_peered":"2024-09-11T23:18:22.544176+0000","last_clean":"2024-09-11T23:18:22.544176+0000","last_became_active":"2024-09-11T23:16:25.906803+0000","last_became_peered":"2024-09-11T23:16:25.906803+0000","last_unstale":"2024-09-11T23:18:22.544176+0000","last_undegraded":"2024-09-11T23:18:22.544176+0000","last_fullsized":"2024-09-11T23:18:22.544176+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:36:49.303932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"85'8","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.292802+0000","last_change":"2024-09-11T23:16:35.946167+0000","last_active":"2024-09-11T23:18:22.292802+0000","last_peered":"2024-09-11T23:18:22.292802+0000","last_clean":"2024-09-11T23:18:22.292802+0000","last_became_active":"2024-09-11T23:16:35.945786+0000","last_became_peered":"2024-09-11T23:16:35.945786+0000","last_unstale":"2024-09-11T23:18:22.292802+0000","last_undegraded":"2024-09-11T23:18:22.292802+0000","last_fullsized":"2024-09-11T23:18:22.292802+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:13:03.064910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"85'4","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.295860+0000","last_change":"2024-09-11T23:16:25.897909+0000","last_active":"2024-09-11T23:18:22.295860+0000","last_peered":"2024-09-11T23:18:22.295860+0000","last_clean":"2024-09-11T23:18:22.295860+0000","last_became_active":"2024-09-11T23:16:25.897569+0000","last_became_peered":"2024-09-11T23:16:25.897569+0000","last_unstale":"2024-09-11T23:18:22.295860+0000","last_undegraded":"2024-09-11T23:18:22.295860+0000","last_fullsized":"2024-09-11T23:18:22.295860+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:25:44.727254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"85'6","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.302845+0000","last_change":"2024-09-11T23:16:25.897594+0000","last_active":"2024-09-11T23:18:22.302845+0000","last_peered":"2024-09-11T23:18:22.302845+0000","last_clean":"2024-09-11T23:18:22.302845+0000","last_became_active":"2024-09-11T23:16:25.897223+0000","last_became_peered":"2024-09-11T23:16:25.897223+0000","last_unstale":"2024-09-11T23:18:22.302845+0000","last_undegraded":"2024-09-11T23:18:22.302845+0000","last_fullsized":"2024-09-11T23:18:22.302845+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:10:50.763455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"85'5","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328293+0000","last_change":"2024-09-11T23:16:35.943511+0000","last_active":"2024-09-11T23:18:22.328293+0000","last_peered":"2024-09-11T23:18:22.328293+0000","last_clean":"2024-09-11T23:18:22.328293+0000","last_became_active":"2024-09-11T23:16:35.943075+0000","last_became_peered":"2024-09-11T23:16:35.943075+0000","last_unstale":"2024-09-11T23:18:22.328293+0000","last_undegraded":"2024-09-11T23:18:22.328293+0000","last_fullsized":"2024-09-11T23:18:22.328293+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:53:43.892421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"85'9","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.182677+0000","last_change":"2024-09-11T23:16:25.903380+0000","last_active":"2024-09-11T23:18:22.182677+0000","last_peered":"2024-09-11T23:18:22.182677+0000","last_clean":"2024-09-11T23:18:22.182677+0000","last_became_active":"2024-09-11T23:16:25.903093+0000","last_became_peered":"2024-09-11T23:16:25.903093+0000","last_unstale":"2024-09-11T23:18:22.182677+0000","last_undegraded":"2024-09-11T23:18:22.182677+0000","last_fullsized":"2024-09-11T23:18:22.182677+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:58:50.944623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"85'11","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.329153+0000","last_change":"2024-09-11T23:16:25.908166+0000","last_active":"2024-09-11T23:18:22.329153+0000","last_peered":"2024-09-11T23:18:22.329153+0000","last_clean":"2024-09-11T23:18:22.329153+0000","last_became_active":"2024-09-11T23:16:25.907749+0000","last_became_peered":"2024-09-11T23:16:25.907749+0000","last_unstale":"2024-09-11T23:18:22.329153+0000","last_undegraded":"2024-09-11T23:18:22.329153+0000","last_fullsized":"2024-09-11T23:18:22.329153+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:11:16.705931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"85'16","reported_seq":41,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.303286+0000","last_change":"2024-09-11T23:16:35.944259+0000","last_active":"2024-09-11T23:18:22.303286+0000","last_peered":"2024-09-11T23:18:22.303286+0000","last_clean":"2024-09-11T23:18:22.303286+0000","last_became_active":"2024-09-11T23:16:35.943767+0000","last_became_peered":"2024-09-11T23:16:35.943767+0000","last_unstale":"2024-09-11T23:18:22.303286+0000","last_undegraded":"2024-09-11T23:18:22.303286+0000","last_fullsized":"2024-09-11T23:18:22.303286+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:20:13.127854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"85'6","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328788+0000","last_change":"2024-09-11T23:16:25.903256+0000","last_active":"2024-09-11T23:18:22.328788+0000","last_peered":"2024-09-11T23:18:22.328788+0000","last_clean":"2024-09-11T23:18:22.328788+0000","last_became_active":"2024-09-11T23:16:25.902954+0000","last_became_peered":"2024-09-11T23:16:25.902954+0000","last_unstale":"2024-09-11T23:18:22.328788+0000","last_undegraded":"2024-09-11T23:18:22.328788+0000","last_fullsized":"2024-09-11T23:18:22.328788+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:43:48.651400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"85'7","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.187001+0000","last_change":"2024-09-11T23:16:25.897656+0000","last_active":"2024-09-11T23:18:22.187001+0000","last_peered":"2024-09-11T23:18:22.187001+0000","last_clean":"2024-09-11T23:18:22.187001+0000","last_became_active":"2024-09-11T23:16:25.897103+0000","last_became_peered":"2024-09-11T23:16:25.897103+0000","last_unstale":"2024-09-11T23:18:22.187001+0000","last_undegraded":"2024-09-11T23:18:22.187001+0000","last_fullsized":"2024-09-11T23:18:22.187001+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:14:34.341383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"85'10","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.329625+0000","last_change":"2024-09-11T23:16:25.902800+0000","last_active":"2024-09-11T23:18:22.329625+0000","last_peered":"2024-09-11T23:18:22.329625+0000","last_clean":"2024-09-11T23:18:22.329625+0000","last_became_active":"2024-09-11T23:16:25.902414+0000","last_became_peered":"2024-09-11T23:16:25.902414+0000","last_unstale":"2024-09-11T23:18:22.329625+0000","last_undegraded":"2024-09-11T23:18:22.329625+0000","last_fullsized":"2024-09-11T23:18:22.329625+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:16:32.427749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"85'7","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.543876+0000","last_change":"2024-09-11T23:16:25.894410+0000","last_active":"2024-09-11T23:18:22.543876+0000","last_peered":"2024-09-11T23:18:22.543876+0000","last_clean":"2024-09-11T23:18:22.543876+0000","last_became_active":"2024-09-11T23:16:25.894067+0000","last_became_peered":"2024-09-11T23:16:25.894067+0000","last_unstale":"2024-09-11T23:18:22.543876+0000","last_undegraded":"2024-09-11T23:18:22.543876+0000","last_fullsized":"2024-09-11T23:18:22.543876+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:03:11.913063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"85'7","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.544140+0000","last_change":"2024-09-11T23:16:25.903440+0000","last_active":"2024-09-11T23:18:22.544140+0000","last_peered":"2024-09-11T23:18:22.544140+0000","last_clean":"2024-09-11T23:18:22.544140+0000","last_became_active":"2024-09-11T23:16:25.903122+0000","last_became_peered":"2024-09-11T23:16:25.903122+0000","last_unstale":"2024-09-11T23:18:22.544140+0000","last_undegraded":"2024-09-11T23:18:22.544140+0000","last_fullsized":"2024-09-11T23:18:22.544140+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:03:41.065314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"85'9","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328034+0000","last_change":"2024-09-11T23:16:25.905336+0000","last_active":"2024-09-11T23:18:22.328034+0000","last_peered":"2024-09-11T23:18:22.328034+0000","last_clean":"2024-09-11T23:18:22.328034+0000","last_became_active":"2024-09-11T23:16:25.904699+0000","last_became_peered":"2024-09-11T23:16:25.904699+0000","last_unstale":"2024-09-11T23:18:22.328034+0000","last_undegraded":"2024-09-11T23:18:22.328034+0000","last_fullsized":"2024-09-11T23:18:22.328034+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:16:13.448585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"85'8","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.329144+0000","last_change":"2024-09-11T23:16:25.897977+0000","last_active":"2024-09-11T23:18:22.329144+0000","last_peered":"2024-09-11T23:18:22.329144+0000","last_clean":"2024-09-11T23:18:22.329144+0000","last_became_active":"2024-09-11T23:16:25.897672+0000","last_became_peered":"2024-09-11T23:16:25.897672+0000","last_unstale":"2024-09-11T23:18:22.329144+0000","last_undegraded":"2024-09-11T23:18:22.329144+0000","last_fullsized":"2024-09-11T23:18:22.329144+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:42:56.575066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"85'10","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.293509+0000","last_change":"2024-09-11T23:16:25.897063+0000","last_active":"2024-09-11T23:18:22.293509+0000","last_peered":"2024-09-11T23:18:22.293509+0000","last_clean":"2024-09-11T23:18:22.293509+0000","last_became_active":"2024-09-11T23:16:25.896770+0000","last_became_peered":"2024-09-11T23:16:25.896770+0000","last_unstale":"2024-09-11T23:18:22.293509+0000","last_undegraded":"2024-09-11T23:18:22.293509+0000","last_fullsized":"2024-09-11T23:18:22.293509+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:55:44.735171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"85'7","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328943+0000","last_change":"2024-09-11T23:16:25.903423+0000","last_active":"2024-09-11T23:18:22.328943+0000","last_peered":"2024-09-11T23:18:22.328943+0000","last_clean":"2024-09-11T23:18:22.328943+0000","last_became_active":"2024-09-11T23:16:25.902906+0000","last_became_peered":"2024-09-11T23:16:25.902906+0000","last_unstale":"2024-09-11T23:18:22.328943+0000","last_undegraded":"2024-09-11T23:18:22.328943+0000","last_fullsized":"2024-09-11T23:18:22.328943+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:40:49.326045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"85'8","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328557+0000","last_change":"2024-09-11T23:16:25.896564+0000","last_active":"2024-09-11T23:18:22.328557+0000","last_peered":"2024-09-11T23:18:22.328557+0000","last_clean":"2024-09-11T23:18:22.328557+0000","last_became_active":"2024-09-11T23:16:25.896301+0000","last_became_peered":"2024-09-11T23:16:25.896301+0000","last_unstale":"2024-09-11T23:18:22.328557+0000","last_undegraded":"2024-09-11T23:18:22.328557+0000","last_fullsized":"2024-09-11T23:18:22.328557+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:30:30.074853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"85'11","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.329804+0000","last_change":"2024-09-11T23:16:25.903491+0000","last_active":"2024-09-11T23:18:22.329804+0000","last_peered":"2024-09-11T23:18:22.329804+0000","last_clean":"2024-09-11T23:18:22.329804+0000","last_became_active":"2024-09-11T23:16:25.901340+0000","last_became_peered":"2024-09-11T23:16:25.901340+0000","last_unstale":"2024-09-11T23:18:22.329804+0000","last_undegraded":"2024-09-11T23:18:22.329804+0000","last_fullsized":"2024-09-11T23:18:22.329804+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:41:16.058034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"85'10","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328233+0000","last_change":"2024-09-11T23:16:25.901084+0000","last_active":"2024-09-11T23:18:22.328233+0000","last_peered":"2024-09-11T23:18:22.328233+0000","last_clean":"2024-09-11T23:18:22.328233+0000","last_became_active":"2024-09-11T23:16:25.900726+0000","last_became_peered":"2024-09-11T23:16:25.900726+0000","last_unstale":"2024-09-11T23:18:22.328233+0000","last_undegraded":"2024-09-11T23:18:22.328233+0000","last_fullsized":"2024-09-11T23:18:22.328233+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:57:40.955645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"85'8","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.182688+0000","last_change":"2024-09-11T23:16:25.900461+0000","last_active":"2024-09-11T23:18:22.182688+0000","last_peered":"2024-09-11T23:18:22.182688+0000","last_clean":"2024-09-11T23:18:22.182688+0000","last_became_active":"2024-09-11T23:16:25.900174+0000","last_became_peered":"2024-09-11T23:16:25.900174+0000","last_unstale":"2024-09-11T23:18:22.182688+0000","last_undegraded":"2024-09-11T23:18:22.182688+0000","last_fullsized":"2024-09-11T23:18:22.182688+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:51:50.953198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"85'6","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.184452+0000","last_change":"2024-09-11T23:16:25.907569+0000","last_active":"2024-09-11T23:18:22.184452+0000","last_peered":"2024-09-11T23:18:22.184452+0000","last_clean":"2024-09-11T23:18:22.184452+0000","last_became_active":"2024-09-11T23:16:25.907091+0000","last_became_peered":"2024-09-11T23:16:25.907091+0000","last_unstale":"2024-09-11T23:18:22.184452+0000","last_undegraded":"2024-09-11T23:18:22.184452+0000","last_fullsized":"2024-09-11T23:18:22.184452+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:53:45.961167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"85'2","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.183204+0000","last_change":"2024-09-11T23:16:25.903246+0000","last_active":"2024-09-11T23:18:22.183204+0000","last_peered":"2024-09-11T23:18:22.183204+0000","last_clean":"2024-09-11T23:18:22.183204+0000","last_became_active":"2024-09-11T23:16:25.902907+0000","last_became_peered":"2024-09-11T23:16:25.902907+0000","last_unstale":"2024-09-11T23:18:22.183204+0000","last_undegraded":"2024-09-11T23:18:22.183204+0000","last_fullsized":"2024-09-11T23:18:22.183204+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:20:38.868384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"85'15","reported_seq":34,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328979+0000","last_change":"2024-09-11T23:16:25.906298+0000","last_active":"2024-09-11T23:18:22.328979+0000","last_peered":"2024-09-11T23:18:22.328979+0000","last_clean":"2024-09-11T23:18:22.328979+0000","last_became_active":"2024-09-11T23:16:25.905350+0000","last_became_peered":"2024-09-11T23:16:25.905350+0000","last_unstale":"2024-09-11T23:18:22.328979+0000","last_undegraded":"2024-09-11T23:18:22.328979+0000","last_fullsized":"2024-09-11T23:18:22.328979+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:04:22.122514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"85'6","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328926+0000","last_change":"2024-09-11T23:16:25.902114+0000","last_active":"2024-09-11T23:18:22.328926+0000","last_peered":"2024-09-11T23:18:22.328926+0000","last_clean":"2024-09-11T23:18:22.328926+0000","last_became_active":"2024-09-11T23:16:25.901742+0000","last_became_peered":"2024-09-11T23:16:25.901742+0000","last_unstale":"2024-09-11T23:18:22.328926+0000","last_undegraded":"2024-09-11T23:18:22.328926+0000","last_fullsized":"2024-09-11T23:18:22.328926+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:02:10.640801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"85'4","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.329844+0000","last_change":"2024-09-11T23:16:25.898117+0000","last_active":"2024-09-11T23:18:22.329844+0000","last_peered":"2024-09-11T23:18:22.329844+0000","last_clean":"2024-09-11T23:18:22.329844+0000","last_became_active":"2024-09-11T23:16:25.897818+0000","last_became_peered":"2024-09-11T23:16:25.897818+0000","last_unstale":"2024-09-11T23:18:22.329844+0000","last_undegraded":"2024-09-11T23:18:22.329844+0000","last_fullsized":"2024-09-11T23:18:22.329844+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:27:37.175471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"85'7","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.544057+0000","last_change":"2024-09-11T23:16:25.905600+0000","last_active":"2024-09-11T23:18:22.544057+0000","last_peered":"2024-09-11T23:18:22.544057+0000","last_clean":"2024-09-11T23:18:22.544057+0000","last_became_active":"2024-09-11T23:16:25.903924+0000","last_became_peered":"2024-09-11T23:16:25.903924+0000","last_unstale":"2024-09-11T23:18:22.544057+0000","last_undegraded":"2024-09-11T23:18:22.544057+0000","last_fullsized":"2024-09-11T23:18:22.544057+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:20:29.929079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"85'8","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328507+0000","last_change":"2024-09-11T23:16:25.904763+0000","last_active":"2024-09-11T23:18:22.328507+0000","last_peered":"2024-09-11T23:18:22.328507+0000","last_clean":"2024-09-11T23:18:22.328507+0000","last_became_active":"2024-09-11T23:16:25.904406+0000","last_became_peered":"2024-09-11T23:16:25.904406+0000","last_unstale":"2024-09-11T23:18:22.328507+0000","last_undegraded":"2024-09-11T23:18:22.328507+0000","last_fullsized":"2024-09-11T23:18:22.328507+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:27:38.918017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"85'12","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328649+0000","last_change":"2024-09-11T23:16:25.906573+0000","last_active":"2024-09-11T23:18:22.328649+0000","last_peered":"2024-09-11T23:18:22.328649+0000","last_clean":"2024-09-11T23:18:22.328649+0000","last_became_active":"2024-09-11T23:16:25.904771+0000","last_became_peered":"2024-09-11T23:16:25.904771+0000","last_unstale":"2024-09-11T23:18:22.328649+0000","last_undegraded":"2024-09-11T23:18:22.328649+0000","last_fullsized":"2024-09-11T23:18:22.328649+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:04:17.090648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"85'9","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328406+0000","last_change":"2024-09-11T23:16:25.902123+0000","last_active":"2024-09-11T23:18:22.328406+0000","last_peered":"2024-09-11T23:18:22.328406+0000","last_clean":"2024-09-11T23:18:22.328406+0000","last_became_active":"2024-09-11T23:16:25.901756+0000","last_became_peered":"2024-09-11T23:16:25.901756+0000","last_unstale":"2024-09-11T23:18:22.328406+0000","last_undegraded":"2024-09-11T23:18:22.328406+0000","last_fullsized":"2024-09-11T23:18:22.328406+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:17:37.074976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"85'7","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.295802+0000","last_change":"2024-09-11T23:16:25.904217+0000","last_active":"2024-09-11T23:18:22.295802+0000","last_peered":"2024-09-11T23:18:22.295802+0000","last_clean":"2024-09-11T23:18:22.295802+0000","last_became_active":"2024-09-11T23:16:25.903522+0000","last_became_peered":"2024-09-11T23:16:25.903522+0000","last_unstale":"2024-09-11T23:18:22.295802+0000","last_undegraded":"2024-09-11T23:18:22.295802+0000","last_fullsized":"2024-09-11T23:18:22.295802+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:29:46.634475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"86'7","reported_seq":150,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:32.135914+0000","last_change":"2024-09-11T23:16:23.793696+0000","last_active":"2024-09-11T23:18:32.135914+0000","last_peered":"2024-09-11T23:18:32.135914+0000","last_clean":"2024-09-11T23:18:32.135914+0000","last_became_active":"2024-09-11T23:16:23.793062+0000","last_became_peered":"2024-09-11T23:16:23.793062+0000","last_unstale":"2024-09-11T23:18:32.135914+0000","last_undegraded":"2024-09-11T23:18:32.135914+0000","last_fullsized":"2024-09-11T23:18:32.135914+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:00:25.655817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1323,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":119,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"85'8","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.184453+0000","last_change":"2024-09-11T23:16:25.906673+0000","last_active":"2024-09-11T23:18:22.184453+0000","last_peered":"2024-09-11T23:18:22.184453+0000","last_clean":"2024-09-11T23:18:22.184453+0000","last_became_active":"2024-09-11T23:16:25.906239+0000","last_became_peered":"2024-09-11T23:16:25.906239+0000","last_unstale":"2024-09-11T23:18:22.184453+0000","last_undegraded":"2024-09-11T23:18:22.184453+0000","last_fullsized":"2024-09-11T23:18:22.184453+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:15:54.907269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"85'7","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.302905+0000","last_change":"2024-09-11T23:16:23.793302+0000","last_active":"2024-09-11T23:18:22.302905+0000","last_peered":"2024-09-11T23:18:22.302905+0000","last_clean":"2024-09-11T23:18:22.302905+0000","last_became_active":"2024-09-11T23:16:23.792874+0000","last_became_peered":"2024-09-11T23:16:23.792874+0000","last_unstale":"2024-09-11T23:18:22.302905+0000","last_undegraded":"2024-09-11T23:18:22.302905+0000","last_fullsized":"2024-09-11T23:18:22.302905+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:52:59.558223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"85'3","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328845+0000","last_change":"2024-09-11T23:16:23.792999+0000","last_active":"2024-09-11T23:18:22.328845+0000","last_peered":"2024-09-11T23:18:22.328845+0000","last_clean":"2024-09-11T23:18:22.328845+0000","last_became_active":"2024-09-11T23:16:23.792630+0000","last_became_peered":"2024-09-11T23:16:23.792630+0000","last_unstale":"2024-09-11T23:18:22.328845+0000","last_undegraded":"2024-09-11T23:18:22.328845+0000","last_fullsized":"2024-09-11T23:18:22.328845+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:09:28.001060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"85'8","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.292734+0000","last_change":"2024-09-11T23:16:35.943722+0000","last_active":"2024-09-11T23:18:22.292734+0000","last_peered":"2024-09-11T23:18:22.292734+0000","last_clean":"2024-09-11T23:18:22.292734+0000","last_became_active":"2024-09-11T23:16:35.943355+0000","last_became_peered":"2024-09-11T23:16:35.943355+0000","last_unstale":"2024-09-11T23:18:22.292734+0000","last_undegraded":"2024-09-11T23:18:22.292734+0000","last_fullsized":"2024-09-11T23:18:22.292734+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:26:22.769824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,6],"acting":[9,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"85'3","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.184646+0000","last_change":"2024-09-11T23:16:23.797945+0000","last_active":"2024-09-11T23:18:22.184646+0000","last_peered":"2024-09-11T23:18:22.184646+0000","last_clean":"2024-09-11T23:18:22.184646+0000","last_became_active":"2024-09-11T23:16:23.797572+0000","last_became_peered":"2024-09-11T23:16:23.797572+0000","last_unstale":"2024-09-11T23:18:22.184646+0000","last_undegraded":"2024-09-11T23:18:22.184646+0000","last_fullsized":"2024-09-11T23:18:22.184646+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:04:59.017668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"85'11","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.292877+0000","last_change":"2024-09-11T23:16:25.891587+0000","last_active":"2024-09-11T23:18:22.292877+0000","last_peered":"2024-09-11T23:18:22.292877+0000","last_clean":"2024-09-11T23:18:22.292877+0000","last_became_active":"2024-09-11T23:16:25.891190+0000","last_became_peered":"2024-09-11T23:16:25.891190+0000","last_unstale":"2024-09-11T23:18:22.292877+0000","last_undegraded":"2024-09-11T23:18:22.292877+0000","last_fullsized":"2024-09-11T23:18:22.292877+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:57:05.177843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"85'6","reported_seq":260,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:33.136590+0000","last_change":"2024-09-11T23:16:23.790129+0000","last_active":"2024-09-11T23:18:33.136590+0000","last_peered":"2024-09-11T23:18:33.136590+0000","last_clean":"2024-09-11T23:18:33.136590+0000","last_became_active":"2024-09-11T23:16:23.789720+0000","last_became_peered":"2024-09-11T23:16:23.789720+0000","last_unstale":"2024-09-11T23:18:33.136590+0000","last_undegraded":"2024-09-11T23:18:33.136590+0000","last_fullsized":"2024-09-11T23:18:33.136590+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:33:58.388513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":230,"num_read_kb":230,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"85'10","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.329142+0000","last_change":"2024-09-11T23:16:25.901014+0000","last_active":"2024-09-11T23:18:22.329142+0000","last_peered":"2024-09-11T23:18:22.329142+0000","last_clean":"2024-09-11T23:18:22.329142+0000","last_became_active":"2024-09-11T23:16:25.900280+0000","last_became_peered":"2024-09-11T23:16:25.900280+0000","last_unstale":"2024-09-11T23:18:22.329142+0000","last_undegraded":"2024-09-11T23:18:22.329142+0000","last_fullsized":"2024-09-11T23:18:22.329142+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:14:03.662204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"85'4","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328532+0000","last_change":"2024-09-11T23:16:23.797944+0000","last_active":"2024-09-11T23:18:22.328532+0000","last_peered":"2024-09-11T23:18:22.328532+0000","last_clean":"2024-09-11T23:18:22.328532+0000","last_became_active":"2024-09-11T23:16:23.797520+0000","last_became_peered":"2024-09-11T23:16:23.797520+0000","last_unstale":"2024-09-11T23:18:22.328532+0000","last_undegraded":"2024-09-11T23:18:22.328532+0000","last_fullsized":"2024-09-11T23:18:22.328532+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:29:06.957518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"85'9","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.544713+0000","last_change":"2024-09-11T23:16:25.905379+0000","last_active":"2024-09-11T23:18:22.544713+0000","last_peered":"2024-09-11T23:18:22.544713+0000","last_clean":"2024-09-11T23:18:22.544713+0000","last_became_active":"2024-09-11T23:16:25.905085+0000","last_became_peered":"2024-09-11T23:16:25.905085+0000","last_unstale":"2024-09-11T23:18:22.544713+0000","last_undegraded":"2024-09-11T23:18:22.544713+0000","last_fullsized":"2024-09-11T23:18:22.544713+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:22:35.129736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"85'5","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.184874+0000","last_change":"2024-09-11T23:16:24.075556+0000","last_active":"2024-09-11T23:18:22.184874+0000","last_peered":"2024-09-11T23:18:22.184874+0000","last_clean":"2024-09-11T23:18:22.184874+0000","last_became_active":"2024-09-11T23:16:24.074783+0000","last_became_peered":"2024-09-11T23:16:24.074783+0000","last_unstale":"2024-09-11T23:18:22.184874+0000","last_undegraded":"2024-09-11T23:18:22.184874+0000","last_fullsized":"2024-09-11T23:18:22.184874+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:55:09.962342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"85'11","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328166+0000","last_change":"2024-09-11T23:16:25.907720+0000","last_active":"2024-09-11T23:18:22.328166+0000","last_peered":"2024-09-11T23:18:22.328166+0000","last_clean":"2024-09-11T23:18:22.328166+0000","last_became_active":"2024-09-11T23:16:25.907335+0000","last_became_peered":"2024-09-11T23:16:25.907335+0000","last_unstale":"2024-09-11T23:18:22.328166+0000","last_undegraded":"2024-09-11T23:18:22.328166+0000","last_fullsized":"2024-09-11T23:18:22.328166+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:46:14.519384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"86'38","reported_seq":305,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:33.158862+0000","last_change":"2024-09-11T23:16:23.794321+0000","last_active":"2024-09-11T23:18:33.158862+0000","last_peered":"2024-09-11T23:18:33.158862+0000","last_clean":"2024-09-11T23:18:33.158862+0000","last_became_active":"2024-09-11T23:16:23.793955+0000","last_became_peered":"2024-09-11T23:16:23.793955+0000","last_unstale":"2024-09-11T23:18:33.158862+0000","last_undegraded":"2024-09-11T23:18:33.158862+0000","last_fullsized":"2024-09-11T23:18:33.158862+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:47:46.442656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":66480,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":243,"num_read_kb":186,"num_write":40,"num_write_kb":82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"85'6","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.296053+0000","last_change":"2024-09-11T23:16:25.896598+0000","last_active":"2024-09-11T23:18:22.296053+0000","last_peered":"2024-09-11T23:18:22.296053+0000","last_clean":"2024-09-11T23:18:22.296053+0000","last_became_active":"2024-09-11T23:16:25.896293+0000","last_became_peered":"2024-09-11T23:16:25.896293+0000","last_unstale":"2024-09-11T23:18:22.296053+0000","last_undegraded":"2024-09-11T23:18:22.296053+0000","last_fullsized":"2024-09-11T23:18:22.296053+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:04:11.489644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"85'9","reported_seq":35,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328677+0000","last_change":"2024-09-11T23:16:23.791486+0000","last_active":"2024-09-11T23:18:22.328677+0000","last_peered":"2024-09-11T23:18:22.328677+0000","last_clean":"2024-09-11T23:18:22.328677+0000","last_became_active":"2024-09-11T23:16:23.790786+0000","last_became_peered":"2024-09-11T23:16:23.790786+0000","last_unstale":"2024-09-11T23:18:22.328677+0000","last_undegraded":"2024-09-11T23:18:22.328677+0000","last_fullsized":"2024-09-11T23:18:22.328677+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:03:52.742560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"85'9","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.329171+0000","last_change":"2024-09-11T23:16:25.897382+0000","last_active":"2024-09-11T23:18:22.329171+0000","last_peered":"2024-09-11T23:18:22.329171+0000","last_clean":"2024-09-11T23:18:22.329171+0000","last_became_active":"2024-09-11T23:16:25.897073+0000","last_became_peered":"2024-09-11T23:16:25.897073+0000","last_unstale":"2024-09-11T23:18:22.329171+0000","last_undegraded":"2024-09-11T23:18:22.329171+0000","last_fullsized":"2024-09-11T23:18:22.329171+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:15:59.780100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"85'5","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.184727+0000","last_change":"2024-09-11T23:16:23.793886+0000","last_active":"2024-09-11T23:18:22.184727+0000","last_peered":"2024-09-11T23:18:22.184727+0000","last_clean":"2024-09-11T23:18:22.184727+0000","last_became_active":"2024-09-11T23:16:23.793066+0000","last_became_peered":"2024-09-11T23:16:23.793066+0000","last_unstale":"2024-09-11T23:18:22.184727+0000","last_undegraded":"2024-09-11T23:18:22.184727+0000","last_fullsized":"2024-09-11T23:18:22.184727+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:07:03.882546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"85'15","reported_seq":34,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328552+0000","last_change":"2024-09-11T23:16:25.904575+0000","last_active":"2024-09-11T23:18:22.328552+0000","last_peered":"2024-09-11T23:18:22.328552+0000","last_clean":"2024-09-11T23:18:22.328552+0000","last_became_active":"2024-09-11T23:16:25.904224+0000","last_became_peered":"2024-09-11T23:16:25.904224+0000","last_unstale":"2024-09-11T23:18:22.328552+0000","last_undegraded":"2024-09-11T23:18:22.328552+0000","last_fullsized":"2024-09-11T23:18:22.328552+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:42:13.036931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"85'5","reported_seq":146,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:32.135461+0000","last_change":"2024-09-11T23:16:23.796073+0000","last_active":"2024-09-11T23:18:32.135461+0000","last_peered":"2024-09-11T23:18:32.135461+0000","last_clean":"2024-09-11T23:18:32.135461+0000","last_became_active":"2024-09-11T23:16:23.794850+0000","last_became_peered":"2024-09-11T23:16:23.794850+0000","last_unstale":"2024-09-11T23:18:32.135461+0000","last_undegraded":"2024-09-11T23:18:32.135461+0000","last_fullsized":"2024-09-11T23:18:32.135461+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:57:22.361662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":118,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"85'6","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328221+0000","last_change":"2024-09-11T23:16:25.908700+0000","last_active":"2024-09-11T23:18:22.328221+0000","last_peered":"2024-09-11T23:18:22.328221+0000","last_clean":"2024-09-11T23:18:22.328221+0000","last_became_active":"2024-09-11T23:16:25.908373+0000","last_became_peered":"2024-09-11T23:16:25.908373+0000","last_unstale":"2024-09-11T23:18:22.328221+0000","last_undegraded":"2024-09-11T23:18:22.328221+0000","last_fullsized":"2024-09-11T23:18:22.328221+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:20:18.536079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"85'17","reported_seq":42,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.295792+0000","last_change":"2024-09-11T23:16:24.075927+0000","last_active":"2024-09-11T23:18:22.295792+0000","last_peered":"2024-09-11T23:18:22.295792+0000","last_clean":"2024-09-11T23:18:22.295792+0000","last_became_active":"2024-09-11T23:16:24.075527+0000","last_became_peered":"2024-09-11T23:16:24.075527+0000","last_unstale":"2024-09-11T23:18:22.295792+0000","last_undegraded":"2024-09-11T23:18:22.295792+0000","last_fullsized":"2024-09-11T23:18:22.295792+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:27:54.622059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"85'6","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.184380+0000","last_change":"2024-09-11T23:16:35.947475+0000","last_active":"2024-09-11T23:18:22.184380+0000","last_peered":"2024-09-11T23:18:22.184380+0000","last_clean":"2024-09-11T23:18:22.184380+0000","last_became_active":"2024-09-11T23:16:35.947028+0000","last_became_peered":"2024-09-11T23:16:35.947028+0000","last_unstale":"2024-09-11T23:18:22.184380+0000","last_undegraded":"2024-09-11T23:18:22.184380+0000","last_fullsized":"2024-09-11T23:18:22.184380+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:18:13.479554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'2","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.296198+0000","last_change":"2024-09-11T23:16:23.793298+0000","last_active":"2024-09-11T23:18:22.296198+0000","last_peered":"2024-09-11T23:18:22.296198+0000","last_clean":"2024-09-11T23:18:22.296198+0000","last_became_active":"2024-09-11T23:16:23.792947+0000","last_became_peered":"2024-09-11T23:16:23.792947+0000","last_unstale":"2024-09-11T23:18:22.296198+0000","last_undegraded":"2024-09-11T23:18:22.296198+0000","last_fullsized":"2024-09-11T23:18:22.296198+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:32:21.294560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"85'5","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328871+0000","last_change":"2024-09-11T23:16:25.902348+0000","last_active":"2024-09-11T23:18:22.328871+0000","last_peered":"2024-09-11T23:18:22.328871+0000","last_clean":"2024-09-11T23:18:22.328871+0000","last_became_active":"2024-09-11T23:16:25.901686+0000","last_became_peered":"2024-09-11T23:16:25.901686+0000","last_unstale":"2024-09-11T23:18:22.328871+0000","last_undegraded":"2024-09-11T23:18:22.328871+0000","last_fullsized":"2024-09-11T23:18:22.328871+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:00:15.543003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"85'5","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.330216+0000","last_change":"2024-09-11T23:16:23.790984+0000","last_active":"2024-09-11T23:18:22.330216+0000","last_peered":"2024-09-11T23:18:22.330216+0000","last_clean":"2024-09-11T23:18:22.330216+0000","last_became_active":"2024-09-11T23:16:23.790569+0000","last_became_peered":"2024-09-11T23:16:23.790569+0000","last_unstale":"2024-09-11T23:18:22.330216+0000","last_undegraded":"2024-09-11T23:18:22.330216+0000","last_fullsized":"2024-09-11T23:18:22.330216+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:57:39.675793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"85'9","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328796+0000","last_change":"2024-09-11T23:16:25.899249+0000","last_active":"2024-09-11T23:18:22.328796+0000","last_peered":"2024-09-11T23:18:22.328796+0000","last_clean":"2024-09-11T23:18:22.328796+0000","last_became_active":"2024-09-11T23:16:25.898914+0000","last_became_peered":"2024-09-11T23:16:25.898914+0000","last_unstale":"2024-09-11T23:18:22.328796+0000","last_undegraded":"2024-09-11T23:18:22.328796+0000","last_fullsized":"2024-09-11T23:18:22.328796+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:17:07.545770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"86'67","reported_seq":122,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:31.561400+0000","last_change":"2024-09-11T23:16:23.787387+0000","last_active":"2024-09-11T23:18:31.561400+0000","last_peered":"2024-09-11T23:18:31.561400+0000","last_clean":"2024-09-11T23:18:31.561400+0000","last_became_active":"2024-09-11T23:16:23.787014+0000","last_became_peered":"2024-09-11T23:16:23.787014+0000","last_unstale":"2024-09-11T23:18:31.561400+0000","last_undegraded":"2024-09-11T23:18:31.561400+0000","last_fullsized":"2024-09-11T23:18:31.561400+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":67,"log_dups_size":0,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:45:52.004253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":328302,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":34,"num_read_kb":33,"num_write":76,"num_write_kb":941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"85'7","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328671+0000","last_change":"2024-09-11T23:16:25.908304+0000","last_active":"2024-09-11T23:18:22.328671+0000","last_peered":"2024-09-11T23:18:22.328671+0000","last_clean":"2024-09-11T23:18:22.328671+0000","last_became_active":"2024-09-11T23:16:25.907900+0000","last_became_peered":"2024-09-11T23:16:25.907900+0000","last_unstale":"2024-09-11T23:18:22.328671+0000","last_undegraded":"2024-09-11T23:18:22.328671+0000","last_fullsized":"2024-09-11T23:18:22.328671+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:43:26.247682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"85'7","reported_seq":150,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:32.135824+0000","last_change":"2024-09-11T23:16:24.074610+0000","last_active":"2024-09-11T23:18:32.135824+0000","last_peered":"2024-09-11T23:18:32.135824+0000","last_clean":"2024-09-11T23:18:32.135824+0000","last_became_active":"2024-09-11T23:16:24.074210+0000","last_became_peered":"2024-09-11T23:16:24.074210+0000","last_unstale":"2024-09-11T23:18:32.135824+0000","last_undegraded":"2024-09-11T23:18:32.135824+0000","last_fullsized":"2024-09-11T23:18:32.135824+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:09:37.167050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1704,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":119,"num_read_kb":4,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"85'7","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328842+0000","last_change":"2024-09-11T23:16:25.903120+0000","last_active":"2024-09-11T23:18:22.328842+0000","last_peered":"2024-09-11T23:18:22.328842+0000","last_clean":"2024-09-11T23:18:22.328842+0000","last_became_active":"2024-09-11T23:16:25.902770+0000","last_became_peered":"2024-09-11T23:16:25.902770+0000","last_unstale":"2024-09-11T23:18:22.328842+0000","last_undegraded":"2024-09-11T23:18:22.328842+0000","last_fullsized":"2024-09-11T23:18:22.328842+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:03:30.041088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"86'148","reported_seq":393,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:24.295645+0000","last_change":"2024-09-11T23:15:13.115055+0000","last_active":"2024-09-11T23:18:24.295645+0000","last_peered":"2024-09-11T23:18:24.295645+0000","last_clean":"2024-09-11T23:18:24.295645+0000","last_became_active":"2024-09-11T23:15:13.114480+0000","last_became_peered":"2024-09-11T23:15:13.114480+0000","last_unstale":"2024-09-11T23:18:24.295645+0000","last_undegraded":"2024-09-11T23:18:24.295645+0000","last_fullsized":"2024-09-11T23:18:24.295645+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:11:50.495799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:11:50.495799+0000","last_clean_scrub_stamp":"2024-09-11T23:11:50.495799+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:15:53.681230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"85'6","reported_seq":34,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.187143+0000","last_change":"2024-09-11T23:16:23.793996+0000","last_active":"2024-09-11T23:18:22.187143+0000","last_peered":"2024-09-11T23:18:22.187143+0000","last_clean":"2024-09-11T23:18:22.187143+0000","last_became_active":"2024-09-11T23:16:23.793681+0000","last_became_peered":"2024-09-11T23:16:23.793681+0000","last_unstale":"2024-09-11T23:18:22.187143+0000","last_undegraded":"2024-09-11T23:18:22.187143+0000","last_fullsized":"2024-09-11T23:18:22.187143+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:52:50.463623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"85'8","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.182978+0000","last_change":"2024-09-11T23:16:25.905133+0000","last_active":"2024-09-11T23:18:22.182978+0000","last_peered":"2024-09-11T23:18:22.182978+0000","last_clean":"2024-09-11T23:18:22.182978+0000","last_became_active":"2024-09-11T23:16:25.904819+0000","last_became_peered":"2024-09-11T23:16:25.904819+0000","last_unstale":"2024-09-11T23:18:22.182978+0000","last_undegraded":"2024-09-11T23:18:22.182978+0000","last_fullsized":"2024-09-11T23:18:22.182978+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:51:09.664343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"85'4","reported_seq":35,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328418+0000","last_change":"2024-09-11T23:16:36.228806+0000","last_active":"2024-09-11T23:18:22.328418+0000","last_peered":"2024-09-11T23:18:22.328418+0000","last_clean":"2024-09-11T23:18:22.328418+0000","last_became_active":"2024-09-11T23:16:36.228311+0000","last_became_peered":"2024-09-11T23:16:36.228311+0000","last_unstale":"2024-09-11T23:18:22.328418+0000","last_undegraded":"2024-09-11T23:18:22.328418+0000","last_fullsized":"2024-09-11T23:18:22.328418+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:42:32.856691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4],"acting":[7,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"85'5","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328756+0000","last_change":"2024-09-11T23:16:25.898935+0000","last_active":"2024-09-11T23:18:22.328756+0000","last_peered":"2024-09-11T23:18:22.328756+0000","last_clean":"2024-09-11T23:18:22.328756+0000","last_became_active":"2024-09-11T23:16:25.898618+0000","last_became_peered":"2024-09-11T23:16:25.898618+0000","last_unstale":"2024-09-11T23:18:22.328756+0000","last_undegraded":"2024-09-11T23:18:22.328756+0000","last_fullsized":"2024-09-11T23:18:22.328756+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:13:29.104911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"85'5","reported_seq":146,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:32.134935+0000","last_change":"2024-09-11T23:16:24.073613+0000","last_active":"2024-09-11T23:18:32.134935+0000","last_peered":"2024-09-11T23:18:32.134935+0000","last_clean":"2024-09-11T23:18:32.134935+0000","last_became_active":"2024-09-11T23:16:24.073262+0000","last_became_peered":"2024-09-11T23:16:24.073262+0000","last_unstale":"2024-09-11T23:18:32.134935+0000","last_undegraded":"2024-09-11T23:18:32.134935+0000","last_fullsized":"2024-09-11T23:18:32.134935+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:31:58.653657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":116,"num_read_kb":116,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"85'8","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.184531+0000","last_change":"2024-09-11T23:16:25.903725+0000","last_active":"2024-09-11T23:18:22.184531+0000","last_peered":"2024-09-11T23:18:22.184531+0000","last_clean":"2024-09-11T23:18:22.184531+0000","last_became_active":"2024-09-11T23:16:25.903375+0000","last_became_peered":"2024-09-11T23:16:25.903375+0000","last_unstale":"2024-09-11T23:18:22.184531+0000","last_undegraded":"2024-09-11T23:18:22.184531+0000","last_fullsized":"2024-09-11T23:18:22.184531+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:55:02.779575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"86'6","reported_seq":148,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:32.135224+0000","last_change":"2024-09-11T23:16:23.793932+0000","last_active":"2024-09-11T23:18:32.135224+0000","last_peered":"2024-09-11T23:18:32.135224+0000","last_clean":"2024-09-11T23:18:32.135224+0000","last_became_active":"2024-09-11T23:16:23.793111+0000","last_became_peered":"2024-09-11T23:16:23.793111+0000","last_unstale":"2024-09-11T23:18:32.135224+0000","last_undegraded":"2024-09-11T23:18:32.135224+0000","last_fullsized":"2024-09-11T23:18:32.135224+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:14:09.087222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":117,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"85'7","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.303042+0000","last_change":"2024-09-11T23:16:25.905020+0000","last_active":"2024-09-11T23:18:22.303042+0000","last_peered":"2024-09-11T23:18:22.303042+0000","last_clean":"2024-09-11T23:18:22.303042+0000","last_became_active":"2024-09-11T23:16:25.904675+0000","last_became_peered":"2024-09-11T23:16:25.904675+0000","last_unstale":"2024-09-11T23:18:22.303042+0000","last_undegraded":"2024-09-11T23:18:22.303042+0000","last_fullsized":"2024-09-11T23:18:22.303042+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:13:57.388773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"85'1","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.293308+0000","last_change":"2024-09-11T23:16:24.073630+0000","last_active":"2024-09-11T23:18:22.293308+0000","last_peered":"2024-09-11T23:18:22.293308+0000","last_clean":"2024-09-11T23:18:22.293308+0000","last_became_active":"2024-09-11T23:16:24.073301+0000","last_became_peered":"2024-09-11T23:16:24.073301+0000","last_unstale":"2024-09-11T23:18:22.293308+0000","last_undegraded":"2024-09-11T23:18:22.293308+0000","last_fullsized":"2024-09-11T23:18:22.293308+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:41:59.374581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"85'5","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.303107+0000","last_change":"2024-09-11T23:16:25.894124+0000","last_active":"2024-09-11T23:18:22.303107+0000","last_peered":"2024-09-11T23:18:22.303107+0000","last_clean":"2024-09-11T23:18:22.303107+0000","last_became_active":"2024-09-11T23:16:25.893751+0000","last_became_peered":"2024-09-11T23:16:25.893751+0000","last_unstale":"2024-09-11T23:18:22.303107+0000","last_undegraded":"2024-09-11T23:18:22.303107+0000","last_fullsized":"2024-09-11T23:18:22.303107+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:00:28.265206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"85'4","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.296263+0000","last_change":"2024-09-11T23:16:24.073700+0000","last_active":"2024-09-11T23:18:22.296263+0000","last_peered":"2024-09-11T23:18:22.296263+0000","last_clean":"2024-09-11T23:18:22.296263+0000","last_became_active":"2024-09-11T23:16:24.073283+0000","last_became_peered":"2024-09-11T23:16:24.073283+0000","last_unstale":"2024-09-11T23:18:22.296263+0000","last_undegraded":"2024-09-11T23:18:22.296263+0000","last_fullsized":"2024-09-11T23:18:22.296263+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:41:43.536834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"85'9","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.329964+0000","last_change":"2024-09-11T23:16:25.900532+0000","last_active":"2024-09-11T23:18:22.329964+0000","last_peered":"2024-09-11T23:18:22.329964+0000","last_clean":"2024-09-11T23:18:22.329964+0000","last_became_active":"2024-09-11T23:16:25.899466+0000","last_became_peered":"2024-09-11T23:16:25.899466+0000","last_unstale":"2024-09-11T23:18:22.329964+0000","last_undegraded":"2024-09-11T23:18:22.329964+0000","last_fullsized":"2024-09-11T23:18:22.329964+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:10:01.152323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"86'103","reported_seq":220,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:31.947637+0000","last_change":"2024-09-11T23:16:23.788789+0000","last_active":"2024-09-11T23:18:31.947637+0000","last_peered":"2024-09-11T23:18:31.947637+0000","last_clean":"2024-09-11T23:18:31.947637+0000","last_became_active":"2024-09-11T23:16:23.788422+0000","last_became_peered":"2024-09-11T23:16:23.788422+0000","last_unstale":"2024-09-11T23:18:31.947637+0000","last_undegraded":"2024-09-11T23:18:31.947637+0000","last_fullsized":"2024-09-11T23:18:31.947637+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":103,"log_dups_size":0,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:09:14.310205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":312,"num_read_kb":277,"num_write":188,"num_write_kb":1285,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"85'5","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.183270+0000","last_change":"2024-09-11T23:16:25.895511+0000","last_active":"2024-09-11T23:18:22.183270+0000","last_peered":"2024-09-11T23:18:22.183270+0000","last_clean":"2024-09-11T23:18:22.183270+0000","last_became_active":"2024-09-11T23:16:25.895143+0000","last_became_peered":"2024-09-11T23:16:25.895143+0000","last_unstale":"2024-09-11T23:18:22.183270+0000","last_undegraded":"2024-09-11T23:18:22.183270+0000","last_fullsized":"2024-09-11T23:18:22.183270+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:31:16.498371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"85'7","reported_seq":33,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.187236+0000","last_change":"2024-09-11T23:16:24.019064+0000","last_active":"2024-09-11T23:18:22.187236+0000","last_peered":"2024-09-11T23:18:22.187236+0000","last_clean":"2024-09-11T23:18:22.187236+0000","last_became_active":"2024-09-11T23:16:24.018658+0000","last_became_peered":"2024-09-11T23:16:24.018658+0000","last_unstale":"2024-09-11T23:18:22.187236+0000","last_undegraded":"2024-09-11T23:18:22.187236+0000","last_fullsized":"2024-09-11T23:18:22.187236+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:17:41.009507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"85'7","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.544339+0000","last_change":"2024-09-11T23:16:25.903553+0000","last_active":"2024-09-11T23:18:22.544339+0000","last_peered":"2024-09-11T23:18:22.544339+0000","last_clean":"2024-09-11T23:18:22.544339+0000","last_became_active":"2024-09-11T23:16:25.903221+0000","last_became_peered":"2024-09-11T23:16:25.903221+0000","last_unstale":"2024-09-11T23:18:22.544339+0000","last_undegraded":"2024-09-11T23:18:22.544339+0000","last_fullsized":"2024-09-11T23:18:22.544339+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:16:51.872517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"83'2","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328204+0000","last_change":"2024-09-11T23:16:24.073784+0000","last_active":"2024-09-11T23:18:22.328204+0000","last_peered":"2024-09-11T23:18:22.328204+0000","last_clean":"2024-09-11T23:18:22.328204+0000","last_became_active":"2024-09-11T23:16:24.073406+0000","last_became_peered":"2024-09-11T23:16:24.073406+0000","last_unstale":"2024-09-11T23:18:22.328204+0000","last_undegraded":"2024-09-11T23:18:22.328204+0000","last_fullsized":"2024-09-11T23:18:22.328204+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:31:24.617025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"85'2","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.295796+0000","last_change":"2024-09-11T23:16:25.894878+0000","last_active":"2024-09-11T23:18:22.295796+0000","last_peered":"2024-09-11T23:18:22.295796+0000","last_clean":"2024-09-11T23:18:22.295796+0000","last_became_active":"2024-09-11T23:16:25.894543+0000","last_became_peered":"2024-09-11T23:16:25.894543+0000","last_unstale":"2024-09-11T23:18:22.295796+0000","last_undegraded":"2024-09-11T23:18:22.295796+0000","last_fullsized":"2024-09-11T23:18:22.295796+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:34:13.435754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"85'7","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.182832+0000","last_change":"2024-09-11T23:16:23.786502+0000","last_active":"2024-09-11T23:18:22.182832+0000","last_peered":"2024-09-11T23:18:22.182832+0000","last_clean":"2024-09-11T23:18:22.182832+0000","last_became_active":"2024-09-11T23:16:23.786182+0000","last_became_peered":"2024-09-11T23:16:23.786182+0000","last_unstale":"2024-09-11T23:18:22.182832+0000","last_undegraded":"2024-09-11T23:18:22.182832+0000","last_fullsized":"2024-09-11T23:18:22.182832+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:59:07.978244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"85'9","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328520+0000","last_change":"2024-09-11T23:16:25.899625+0000","last_active":"2024-09-11T23:18:22.328520+0000","last_peered":"2024-09-11T23:18:22.328520+0000","last_clean":"2024-09-11T23:18:22.328520+0000","last_became_active":"2024-09-11T23:16:25.899265+0000","last_became_peered":"2024-09-11T23:16:25.899265+0000","last_unstale":"2024-09-11T23:18:22.328520+0000","last_undegraded":"2024-09-11T23:18:22.328520+0000","last_fullsized":"2024-09-11T23:18:22.328520+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:36:06.907622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"86'4","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328472+0000","last_change":"2024-09-11T23:16:36.228991+0000","last_active":"2024-09-11T23:18:22.328472+0000","last_peered":"2024-09-11T23:18:22.328472+0000","last_clean":"2024-09-11T23:18:22.328472+0000","last_became_active":"2024-09-11T23:16:36.227701+0000","last_became_peered":"2024-09-11T23:16:36.227701+0000","last_unstale":"2024-09-11T23:18:22.328472+0000","last_undegraded":"2024-09-11T23:18:22.328472+0000","last_fullsized":"2024-09-11T23:18:22.328472+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:37:57.478572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"85'8","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.182913+0000","last_change":"2024-09-11T23:16:25.894803+0000","last_active":"2024-09-11T23:18:22.182913+0000","last_peered":"2024-09-11T23:18:22.182913+0000","last_clean":"2024-09-11T23:18:22.182913+0000","last_became_active":"2024-09-11T23:16:25.894429+0000","last_became_peered":"2024-09-11T23:16:25.894429+0000","last_unstale":"2024-09-11T23:18:22.182913+0000","last_undegraded":"2024-09-11T23:18:22.182913+0000","last_fullsized":"2024-09-11T23:18:22.182913+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:39:15.868910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"83'4","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.295871+0000","last_change":"2024-09-11T23:16:23.794271+0000","last_active":"2024-09-11T23:18:22.295871+0000","last_peered":"2024-09-11T23:18:22.295871+0000","last_clean":"2024-09-11T23:18:22.295871+0000","last_became_active":"2024-09-11T23:16:23.793919+0000","last_became_peered":"2024-09-11T23:16:23.793919+0000","last_unstale":"2024-09-11T23:18:22.295871+0000","last_undegraded":"2024-09-11T23:18:22.295871+0000","last_fullsized":"2024-09-11T23:18:22.295871+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:30:28.907312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"85'5","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328395+0000","last_change":"2024-09-11T23:16:25.900969+0000","last_active":"2024-09-11T23:18:22.328395+0000","last_peered":"2024-09-11T23:18:22.328395+0000","last_clean":"2024-09-11T23:18:22.328395+0000","last_became_active":"2024-09-11T23:16:25.900425+0000","last_became_peered":"2024-09-11T23:16:25.900425+0000","last_unstale":"2024-09-11T23:18:22.328395+0000","last_undegraded":"2024-09-11T23:18:22.328395+0000","last_fullsized":"2024-09-11T23:18:22.328395+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:03:25.185773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"85'15","reported_seq":34,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.184711+0000","last_change":"2024-09-11T23:16:25.911605+0000","last_active":"2024-09-11T23:18:22.184711+0000","last_peered":"2024-09-11T23:18:22.184711+0000","last_clean":"2024-09-11T23:18:22.184711+0000","last_became_active":"2024-09-11T23:16:25.911180+0000","last_became_peered":"2024-09-11T23:16:25.911180+0000","last_unstale":"2024-09-11T23:18:22.184711+0000","last_undegraded":"2024-09-11T23:18:22.184711+0000","last_fullsized":"2024-09-11T23:18:22.184711+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:55:38.062606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"85'5","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328425+0000","last_change":"2024-09-11T23:16:35.949528+0000","last_active":"2024-09-11T23:18:22.328425+0000","last_peered":"2024-09-11T23:18:22.328425+0000","last_clean":"2024-09-11T23:18:22.328425+0000","last_became_active":"2024-09-11T23:16:35.945679+0000","last_became_peered":"2024-09-11T23:16:35.945679+0000","last_unstale":"2024-09-11T23:18:22.328425+0000","last_undegraded":"2024-09-11T23:18:22.328425+0000","last_fullsized":"2024-09-11T23:18:22.328425+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:14:24.706044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"85'9","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328158+0000","last_change":"2024-09-11T23:16:25.900526+0000","last_active":"2024-09-11T23:18:22.328158+0000","last_peered":"2024-09-11T23:18:22.328158+0000","last_clean":"2024-09-11T23:18:22.328158+0000","last_became_active":"2024-09-11T23:16:25.900148+0000","last_became_peered":"2024-09-11T23:16:25.900148+0000","last_unstale":"2024-09-11T23:18:22.328158+0000","last_undegraded":"2024-09-11T23:18:22.328158+0000","last_fullsized":"2024-09-11T23:18:22.328158+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:26:10.582473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"86'13","reported_seq":48,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.296144+0000","last_change":"2024-09-11T23:16:36.233273+0000","last_active":"2024-09-11T23:18:22.296144+0000","last_peered":"2024-09-11T23:18:22.296144+0000","last_clean":"2024-09-11T23:18:22.296144+0000","last_became_active":"2024-09-11T23:16:36.228358+0000","last_became_peered":"2024-09-11T23:16:36.228358+0000","last_unstale":"2024-09-11T23:18:22.296144+0000","last_undegraded":"2024-09-11T23:18:22.296144+0000","last_fullsized":"2024-09-11T23:18:22.296144+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:06:31.225795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"85'6","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.544620+0000","last_change":"2024-09-11T23:16:25.907874+0000","last_active":"2024-09-11T23:18:22.544620+0000","last_peered":"2024-09-11T23:18:22.544620+0000","last_clean":"2024-09-11T23:18:22.544620+0000","last_became_active":"2024-09-11T23:16:25.907491+0000","last_became_peered":"2024-09-11T23:16:25.907491+0000","last_unstale":"2024-09-11T23:18:22.544620+0000","last_undegraded":"2024-09-11T23:18:22.544620+0000","last_fullsized":"2024-09-11T23:18:22.544620+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:09:20.984249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"84'2","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.183326+0000","last_change":"2024-09-11T23:16:24.074355+0000","last_active":"2024-09-11T23:18:22.183326+0000","last_peered":"2024-09-11T23:18:22.183326+0000","last_clean":"2024-09-11T23:18:22.183326+0000","last_became_active":"2024-09-11T23:16:24.074047+0000","last_became_peered":"2024-09-11T23:16:24.074047+0000","last_unstale":"2024-09-11T23:18:22.183326+0000","last_undegraded":"2024-09-11T23:18:22.183326+0000","last_fullsized":"2024-09-11T23:18:22.183326+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:31:58.804420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"85'5","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.182974+0000","last_change":"2024-09-11T23:16:25.907701+0000","last_active":"2024-09-11T23:18:22.182974+0000","last_peered":"2024-09-11T23:18:22.182974+0000","last_clean":"2024-09-11T23:18:22.182974+0000","last_became_active":"2024-09-11T23:16:25.907379+0000","last_became_peered":"2024-09-11T23:16:25.907379+0000","last_unstale":"2024-09-11T23:18:22.182974+0000","last_undegraded":"2024-09-11T23:18:22.182974+0000","last_fullsized":"2024-09-11T23:18:22.182974+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:12:03.639055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"86'14","reported_seq":156,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:32.134506+0000","last_change":"2024-09-11T23:16:23.794293+0000","last_active":"2024-09-11T23:18:32.134506+0000","last_peered":"2024-09-11T23:18:32.134506+0000","last_clean":"2024-09-11T23:18:32.134506+0000","last_became_active":"2024-09-11T23:16:23.793974+0000","last_became_peered":"2024-09-11T23:16:23.793974+0000","last_unstale":"2024-09-11T23:18:32.134506+0000","last_undegraded":"2024-09-11T23:18:32.134506+0000","last_fullsized":"2024-09-11T23:18:32.134506+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:24:39.170188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":117,"num_read_kb":117,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":795978,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1544,"num_read_kb":991,"num_write":498,"num_write_kb":2376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2764800,"data_stored":2387934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":377,"ondisk_log_size":377,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":11,"up_from":77,"seq":330712481831,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32128,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713024,"statfs":{"total":95995035648,"available":95962136576,"internally_reserved":0,"allocated":962560,"data_stored":459042,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":70,"seq":300647710763,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":29096,"kb_used_data":1952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716056,"statfs":{"total":95995035648,"available":95965241344,"internally_reserved":0,"allocated":1998848,"data_stored":1443667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":65,"seq":279172874289,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28368,"kb_used_data":1344,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716784,"statfs":{"total":95995035648,"available":95965986816,"internally_reserved":0,"allocated":1376256,"data_stored":851072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":57,"seq":244813135926,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28360,"kb_used_data":1336,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716792,"statfs":{"total":95995035648,"available":95965995008,"internally_reserved":0,"allocated":1368064,"data_stored":850868,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":50,"seq":214748364858,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28568,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716584,"statfs":{"total":95995035648,"available":95965782016,"internally_reserved":0,"allocated":1576960,"data_stored":1051455,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528382,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36292,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708860,"statfs":{"total":95995035648,"available":95957872640,"internally_reserved":0,"allocated":966656,"data_stored":457966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790021,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":29020,"kb_used_data":1876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716132,"statfs":{"total":95995035648,"available":95965319168,"internally_reserved":0,"allocated":1921024,"data_stored":1378972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920842,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36292,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708860,"statfs":{"total":95995035648,"available":95957872640,"internally_reserved":0,"allocated":966656,"data_stored":457434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182479,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":41292,"kb_used_data":1668,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93703860,"statfs":{"total":95995035648,"available":95952752640,"internally_reserved":0,"allocated":1708032,"data_stored":1178846,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411412,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36380,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708772,"statfs":{"total":95995035648,"available":95957782528,"internally_reserved":0,"allocated":1056768,"data_stored":526421,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574937,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1036288,"data_stored":525359,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738462,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":987136,"data_stored":459446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":68625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":69687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":753664,"data_stored":721878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":331753,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":421888,"data_stored":394134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":394338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":454656,"data_stored":396448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2308,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T23:18:35.647 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-11T23:18:35.648 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-11T23:18:35.908 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:18:36.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:35 smithi026 ceph-mon[31172]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 83 KiB/s wr, 54 op/s 2024-09-11T23:18:36.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:35 smithi026 ceph-mon[31172]: from='client.15375 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T23:18:36.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:35 smithi076 ceph-mon[35296]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 83 KiB/s wr, 54 op/s 2024-09-11T23:18:36.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:35 smithi076 ceph-mon[35296]: from='client.15375 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T23:18:36.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:35 smithi136 ceph-mon[35121]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 83 KiB/s wr, 54 op/s 2024-09-11T23:18:36.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:35 smithi136 ceph-mon[35121]: from='client.15375 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T23:18:36.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:36.400+0000 7fcf7c43f640 1 -- 172.21.15.26:0/3113509201 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcf74101de0 msgr2=0x7fcf741021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:36.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:36.400+0000 7fcf7c43f640 1 --2- 172.21.15.26:0/3113509201 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcf74101de0 0x7fcf741021c0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fcf70009900 tx=0x7fcf7002a410 comp rx=0 tx=0).stop 2024-09-11T23:18:36.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:36.400+0000 7fcf7c43f640 1 -- 172.21.15.26:0/3113509201 shutdown_connections 2024-09-11T23:18:36.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:36.400+0000 7fcf7c43f640 1 --2- 172.21.15.26:0/3113509201 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcf74106f10 0x7fcf7410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:36.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:36.400+0000 7fcf7c43f640 1 --2- 172.21.15.26:0/3113509201 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcf74102790 0x7fcf741067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:36.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:36.400+0000 7fcf7c43f640 1 --2- 172.21.15.26:0/3113509201 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcf74101de0 0x7fcf741021c0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:36.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:36.401+0000 7fcf7c43f640 1 -- 172.21.15.26:0/3113509201 >> 172.21.15.26:0/3113509201 conn(0x7fcf740fd8c0 msgr2=0x7fcf740ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:18:36.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:36.401+0000 7fcf7c43f640 1 -- 172.21.15.26:0/3113509201 shutdown_connections 2024-09-11T23:18:36.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:36.401+0000 7fcf7c43f640 1 -- 172.21.15.26:0/3113509201 wait complete. 2024-09-11T23:18:36.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:36.401+0000 7fcf7c43f640 1 Processor -- start 2024-09-11T23:18:36.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:36.401+0000 7fcf7c43f640 1 -- start start 2024-09-11T23:18:36.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:36.401+0000 7fcf7c43f640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcf74101de0 0x7fcf740744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:36.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:36.401+0000 7fcf7c43f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcf74102790 0x7fcf74074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:36.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:36.401+0000 7fcf7c43f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcf74106f10 0x7fcf74074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:36.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:36.401+0000 7fcf7c43f640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fcf74111fc0 con 0x7fcf74106f10 2024-09-11T23:18:36.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:36.401+0000 7fcf7c43f640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fcf74111e40 con 0x7fcf74102790 2024-09-11T23:18:36.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:36.401+0000 7fcf7c43f640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fcf74112140 con 0x7fcf74101de0 2024-09-11T23:18:36.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:36.402+0000 7fcf7a9dc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcf74106f10 0x7fcf74074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:36.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:36.402+0000 7fcf7a9dc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcf74106f10 0x7fcf74074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60614/0 (socket says 172.21.15.26:60614) 2024-09-11T23:18:36.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:36.402+0000 7fcf7a9dc640 1 -- 172.21.15.26:0/2891517857 learned_addr learned my addr 172.21.15.26:0/2891517857 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:18:36.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:36.402+0000 7fcf7a1db640 1 --2- 172.21.15.26:0/2891517857 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcf74101de0 0x7fcf740744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:36.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:36.402+0000 7fcf799da640 1 --2- 172.21.15.26:0/2891517857 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcf74102790 0x7fcf74074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:36.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:36.402+0000 7fcf7a9dc640 1 -- 172.21.15.26:0/2891517857 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcf74101de0 msgr2=0x7fcf740744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:36.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:36.402+0000 7fcf7a9dc640 1 --2- 172.21.15.26:0/2891517857 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcf74101de0 0x7fcf740744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:36.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:36.402+0000 7fcf7a9dc640 1 -- 172.21.15.26:0/2891517857 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcf74102790 msgr2=0x7fcf74074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:36.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:36.402+0000 7fcf7a9dc640 1 --2- 172.21.15.26:0/2891517857 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcf74102790 0x7fcf74074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:36.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:36.402+0000 7fcf7a9dc640 1 -- 172.21.15.26:0/2891517857 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf741a69a0 con 0x7fcf74106f10 2024-09-11T23:18:36.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:36.402+0000 7fcf7a9dc640 1 --2- 172.21.15.26:0/2891517857 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcf74106f10 0x7fcf74074f70 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fcf6c00cad0 tx=0x7fcf6c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:18:36.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:36.402+0000 7fcf6b7fe640 1 -- 172.21.15.26:0/2891517857 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf6c00ae20 con 0x7fcf74106f10 2024-09-11T23:18:36.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:36.402+0000 7fcf7c43f640 1 -- 172.21.15.26:0/2891517857 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf741a6c90 con 0x7fcf74106f10 2024-09-11T23:18:36.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:36.402+0000 7fcf6b7fe640 1 -- 172.21.15.26:0/2891517857 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf6c005020 con 0x7fcf74106f10 2024-09-11T23:18:36.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:36.402+0000 7fcf6b7fe640 1 -- 172.21.15.26:0/2891517857 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf6c00f6f0 con 0x7fcf74106f10 2024-09-11T23:18:36.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:36.402+0000 7fcf7c43f640 1 -- 172.21.15.26:0/2891517857 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf741a7180 con 0x7fcf74106f10 2024-09-11T23:18:36.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:36.403+0000 7fcf7c43f640 1 -- 172.21.15.26:0/2891517857 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf741021c0 con 0x7fcf74106f10 2024-09-11T23:18:36.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:36.404+0000 7fcf6b7fe640 1 -- 172.21.15.26:0/2891517857 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 24) ==== 138009+0+0 (secure 0 0 0) 0x7fcf6c0051d0 con 0x7fcf74106f10 2024-09-11T23:18:36.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:36.405+0000 7fcf6b7fe640 1 --2- 172.21.15.26:0/2891517857 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fcf600a2d80 0x7fcf600a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:36.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:36.406+0000 7fcf7a1db640 1 --2- 172.21.15.26:0/2891517857 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fcf600a2d80 0x7fcf600a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:36.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:36.406+0000 7fcf6b7fe640 1 -- 172.21.15.26:0/2891517857 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fcf6c0cf520 con 0x7fcf74106f10 2024-09-11T23:18:36.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:36.406+0000 7fcf7a1db640 1 --2- 172.21.15.26:0/2891517857 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fcf600a2d80 0x7fcf600a5240 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fcf70000c00 tx=0x7fcf70005df0 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:18:36.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:36.410+0000 7fcf6b7fe640 1 -- 172.21.15.26:0/2891517857 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fcf6c014030 con 0x7fcf74106f10 2024-09-11T23:18:36.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:36.581+0000 7fcf7c43f640 1 -- 172.21.15.26:0/2891517857 --> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fcf7410a160 con 0x7fcf600a2d80 2024-09-11T23:18:36.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:36.590+0000 7fcf6b7fe640 1 -- 172.21.15.26:0/2891517857 <== mgr.15171 v2:172.21.15.26:6800/1989555790 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+257806 (secure 0 0 0) 0x7fcf700042d0 con 0x7fcf600a2d80 2024-09-11T23:18:36.593 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-11T23:18:36.594 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-09-11T23:18:36.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:36.592+0000 7fcf7c43f640 1 -- 172.21.15.26:0/2891517857 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fcf600a2d80 msgr2=0x7fcf600a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:36.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:36.592+0000 7fcf7c43f640 1 --2- 172.21.15.26:0/2891517857 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fcf600a2d80 0x7fcf600a5240 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fcf70000c00 tx=0x7fcf70005df0 comp rx=0 tx=0).stop 2024-09-11T23:18:36.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:36.592+0000 7fcf7c43f640 1 -- 172.21.15.26:0/2891517857 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcf74106f10 msgr2=0x7fcf74074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:36.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:36.592+0000 7fcf7c43f640 1 --2- 172.21.15.26:0/2891517857 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcf74106f10 0x7fcf74074f70 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fcf6c00cad0 tx=0x7fcf6c00cf90 comp rx=0 tx=0).stop 2024-09-11T23:18:36.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:36.592+0000 7fcf7c43f640 1 -- 172.21.15.26:0/2891517857 shutdown_connections 2024-09-11T23:18:36.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:36.593+0000 7fcf7c43f640 1 --2- 172.21.15.26:0/2891517857 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fcf600a2d80 0x7fcf600a5240 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:36.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:36.593+0000 7fcf7c43f640 1 --2- 172.21.15.26:0/2891517857 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcf74102790 0x7fcf74074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:36.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:36.593+0000 7fcf7c43f640 1 --2- 172.21.15.26:0/2891517857 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcf74106f10 0x7fcf74074f70 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:36.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:36.593+0000 7fcf7c43f640 1 --2- 172.21.15.26:0/2891517857 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcf74101de0 0x7fcf740744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:36.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:36.593+0000 7fcf7c43f640 1 -- 172.21.15.26:0/2891517857 >> 172.21.15.26:0/2891517857 conn(0x7fcf740fd8c0 msgr2=0x7fcf740febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:18:36.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:36.593+0000 7fcf7c43f640 1 -- 172.21.15.26:0/2891517857 shutdown_connections 2024-09-11T23:18:36.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:36.593+0000 7fcf7c43f640 1 -- 172.21.15.26:0/2891517857 wait complete. 2024-09-11T23:18:37.329 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":10,"stamp":"2024-09-11T23:18:36.208248+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1386580,"num_objects":104,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1781,"num_read_kb":1321,"num_write":701,"num_write_kb":5316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1027,"ondisk_log_size":1027,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":381824,"kb_used_data":15552,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124560000,"statfs":{"total":1151940427776,"available":1151549440000,"internally_reserved":0,"allocated":15925248,"data_stored":9640548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":374453561},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":196632,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":456,"num_read_kb":493,"num_write":146,"num_write_kb":914,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.011242"},"pg_stats":[{"pgid":"3.3f","version":"85'8","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.329092+0000","last_change":"2024-09-11T23:16:25.899106+0000","last_active":"2024-09-11T23:18:22.329092+0000","last_peered":"2024-09-11T23:18:22.329092+0000","last_clean":"2024-09-11T23:18:22.329092+0000","last_became_active":"2024-09-11T23:16:25.898654+0000","last_became_peered":"2024-09-11T23:16:25.898654+0000","last_unstale":"2024-09-11T23:18:22.329092+0000","last_undegraded":"2024-09-11T23:18:22.329092+0000","last_fullsized":"2024-09-11T23:18:22.329092+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:21:41.490905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"85'4","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.544255+0000","last_change":"2024-09-11T23:16:25.895629+0000","last_active":"2024-09-11T23:18:22.544255+0000","last_peered":"2024-09-11T23:18:22.544255+0000","last_clean":"2024-09-11T23:18:22.544255+0000","last_became_active":"2024-09-11T23:16:25.895308+0000","last_became_peered":"2024-09-11T23:16:25.895308+0000","last_unstale":"2024-09-11T23:18:22.544255+0000","last_undegraded":"2024-09-11T23:18:22.544255+0000","last_fullsized":"2024-09-11T23:18:22.544255+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:39:42.549970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"85'7","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.544176+0000","last_change":"2024-09-11T23:16:25.907113+0000","last_active":"2024-09-11T23:18:22.544176+0000","last_peered":"2024-09-11T23:18:22.544176+0000","last_clean":"2024-09-11T23:18:22.544176+0000","last_became_active":"2024-09-11T23:16:25.906803+0000","last_became_peered":"2024-09-11T23:16:25.906803+0000","last_unstale":"2024-09-11T23:18:22.544176+0000","last_undegraded":"2024-09-11T23:18:22.544176+0000","last_fullsized":"2024-09-11T23:18:22.544176+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:36:49.303932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"85'8","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.292802+0000","last_change":"2024-09-11T23:16:35.946167+0000","last_active":"2024-09-11T23:18:22.292802+0000","last_peered":"2024-09-11T23:18:22.292802+0000","last_clean":"2024-09-11T23:18:22.292802+0000","last_became_active":"2024-09-11T23:16:35.945786+0000","last_became_peered":"2024-09-11T23:16:35.945786+0000","last_unstale":"2024-09-11T23:18:22.292802+0000","last_undegraded":"2024-09-11T23:18:22.292802+0000","last_fullsized":"2024-09-11T23:18:22.292802+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:13:03.064910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"85'4","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.295860+0000","last_change":"2024-09-11T23:16:25.897909+0000","last_active":"2024-09-11T23:18:22.295860+0000","last_peered":"2024-09-11T23:18:22.295860+0000","last_clean":"2024-09-11T23:18:22.295860+0000","last_became_active":"2024-09-11T23:16:25.897569+0000","last_became_peered":"2024-09-11T23:16:25.897569+0000","last_unstale":"2024-09-11T23:18:22.295860+0000","last_undegraded":"2024-09-11T23:18:22.295860+0000","last_fullsized":"2024-09-11T23:18:22.295860+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:25:44.727254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"85'6","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.302845+0000","last_change":"2024-09-11T23:16:25.897594+0000","last_active":"2024-09-11T23:18:22.302845+0000","last_peered":"2024-09-11T23:18:22.302845+0000","last_clean":"2024-09-11T23:18:22.302845+0000","last_became_active":"2024-09-11T23:16:25.897223+0000","last_became_peered":"2024-09-11T23:16:25.897223+0000","last_unstale":"2024-09-11T23:18:22.302845+0000","last_undegraded":"2024-09-11T23:18:22.302845+0000","last_fullsized":"2024-09-11T23:18:22.302845+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:10:50.763455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"85'5","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328293+0000","last_change":"2024-09-11T23:16:35.943511+0000","last_active":"2024-09-11T23:18:22.328293+0000","last_peered":"2024-09-11T23:18:22.328293+0000","last_clean":"2024-09-11T23:18:22.328293+0000","last_became_active":"2024-09-11T23:16:35.943075+0000","last_became_peered":"2024-09-11T23:16:35.943075+0000","last_unstale":"2024-09-11T23:18:22.328293+0000","last_undegraded":"2024-09-11T23:18:22.328293+0000","last_fullsized":"2024-09-11T23:18:22.328293+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:53:43.892421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"85'9","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.182677+0000","last_change":"2024-09-11T23:16:25.903380+0000","last_active":"2024-09-11T23:18:22.182677+0000","last_peered":"2024-09-11T23:18:22.182677+0000","last_clean":"2024-09-11T23:18:22.182677+0000","last_became_active":"2024-09-11T23:16:25.903093+0000","last_became_peered":"2024-09-11T23:16:25.903093+0000","last_unstale":"2024-09-11T23:18:22.182677+0000","last_undegraded":"2024-09-11T23:18:22.182677+0000","last_fullsized":"2024-09-11T23:18:22.182677+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:58:50.944623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"85'11","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.329153+0000","last_change":"2024-09-11T23:16:25.908166+0000","last_active":"2024-09-11T23:18:22.329153+0000","last_peered":"2024-09-11T23:18:22.329153+0000","last_clean":"2024-09-11T23:18:22.329153+0000","last_became_active":"2024-09-11T23:16:25.907749+0000","last_became_peered":"2024-09-11T23:16:25.907749+0000","last_unstale":"2024-09-11T23:18:22.329153+0000","last_undegraded":"2024-09-11T23:18:22.329153+0000","last_fullsized":"2024-09-11T23:18:22.329153+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:11:16.705931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"85'16","reported_seq":41,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.303286+0000","last_change":"2024-09-11T23:16:35.944259+0000","last_active":"2024-09-11T23:18:22.303286+0000","last_peered":"2024-09-11T23:18:22.303286+0000","last_clean":"2024-09-11T23:18:22.303286+0000","last_became_active":"2024-09-11T23:16:35.943767+0000","last_became_peered":"2024-09-11T23:16:35.943767+0000","last_unstale":"2024-09-11T23:18:22.303286+0000","last_undegraded":"2024-09-11T23:18:22.303286+0000","last_fullsized":"2024-09-11T23:18:22.303286+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:20:13.127854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"85'6","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328788+0000","last_change":"2024-09-11T23:16:25.903256+0000","last_active":"2024-09-11T23:18:22.328788+0000","last_peered":"2024-09-11T23:18:22.328788+0000","last_clean":"2024-09-11T23:18:22.328788+0000","last_became_active":"2024-09-11T23:16:25.902954+0000","last_became_peered":"2024-09-11T23:16:25.902954+0000","last_unstale":"2024-09-11T23:18:22.328788+0000","last_undegraded":"2024-09-11T23:18:22.328788+0000","last_fullsized":"2024-09-11T23:18:22.328788+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:43:48.651400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"85'7","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.187001+0000","last_change":"2024-09-11T23:16:25.897656+0000","last_active":"2024-09-11T23:18:22.187001+0000","last_peered":"2024-09-11T23:18:22.187001+0000","last_clean":"2024-09-11T23:18:22.187001+0000","last_became_active":"2024-09-11T23:16:25.897103+0000","last_became_peered":"2024-09-11T23:16:25.897103+0000","last_unstale":"2024-09-11T23:18:22.187001+0000","last_undegraded":"2024-09-11T23:18:22.187001+0000","last_fullsized":"2024-09-11T23:18:22.187001+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:14:34.341383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"85'10","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.329625+0000","last_change":"2024-09-11T23:16:25.902800+0000","last_active":"2024-09-11T23:18:22.329625+0000","last_peered":"2024-09-11T23:18:22.329625+0000","last_clean":"2024-09-11T23:18:22.329625+0000","last_became_active":"2024-09-11T23:16:25.902414+0000","last_became_peered":"2024-09-11T23:16:25.902414+0000","last_unstale":"2024-09-11T23:18:22.329625+0000","last_undegraded":"2024-09-11T23:18:22.329625+0000","last_fullsized":"2024-09-11T23:18:22.329625+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:16:32.427749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"85'7","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.543876+0000","last_change":"2024-09-11T23:16:25.894410+0000","last_active":"2024-09-11T23:18:22.543876+0000","last_peered":"2024-09-11T23:18:22.543876+0000","last_clean":"2024-09-11T23:18:22.543876+0000","last_became_active":"2024-09-11T23:16:25.894067+0000","last_became_peered":"2024-09-11T23:16:25.894067+0000","last_unstale":"2024-09-11T23:18:22.543876+0000","last_undegraded":"2024-09-11T23:18:22.543876+0000","last_fullsized":"2024-09-11T23:18:22.543876+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:03:11.913063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"85'7","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.544140+0000","last_change":"2024-09-11T23:16:25.903440+0000","last_active":"2024-09-11T23:18:22.544140+0000","last_peered":"2024-09-11T23:18:22.544140+0000","last_clean":"2024-09-11T23:18:22.544140+0000","last_became_active":"2024-09-11T23:16:25.903122+0000","last_became_peered":"2024-09-11T23:16:25.903122+0000","last_unstale":"2024-09-11T23:18:22.544140+0000","last_undegraded":"2024-09-11T23:18:22.544140+0000","last_fullsized":"2024-09-11T23:18:22.544140+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:03:41.065314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"85'9","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328034+0000","last_change":"2024-09-11T23:16:25.905336+0000","last_active":"2024-09-11T23:18:22.328034+0000","last_peered":"2024-09-11T23:18:22.328034+0000","last_clean":"2024-09-11T23:18:22.328034+0000","last_became_active":"2024-09-11T23:16:25.904699+0000","last_became_peered":"2024-09-11T23:16:25.904699+0000","last_unstale":"2024-09-11T23:18:22.328034+0000","last_undegraded":"2024-09-11T23:18:22.328034+0000","last_fullsized":"2024-09-11T23:18:22.328034+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:16:13.448585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"85'8","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.329144+0000","last_change":"2024-09-11T23:16:25.897977+0000","last_active":"2024-09-11T23:18:22.329144+0000","last_peered":"2024-09-11T23:18:22.329144+0000","last_clean":"2024-09-11T23:18:22.329144+0000","last_became_active":"2024-09-11T23:16:25.897672+0000","last_became_peered":"2024-09-11T23:16:25.897672+0000","last_unstale":"2024-09-11T23:18:22.329144+0000","last_undegraded":"2024-09-11T23:18:22.329144+0000","last_fullsized":"2024-09-11T23:18:22.329144+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:42:56.575066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"85'10","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.293509+0000","last_change":"2024-09-11T23:16:25.897063+0000","last_active":"2024-09-11T23:18:22.293509+0000","last_peered":"2024-09-11T23:18:22.293509+0000","last_clean":"2024-09-11T23:18:22.293509+0000","last_became_active":"2024-09-11T23:16:25.896770+0000","last_became_peered":"2024-09-11T23:16:25.896770+0000","last_unstale":"2024-09-11T23:18:22.293509+0000","last_undegraded":"2024-09-11T23:18:22.293509+0000","last_fullsized":"2024-09-11T23:18:22.293509+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:55:44.735171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"85'7","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328943+0000","last_change":"2024-09-11T23:16:25.903423+0000","last_active":"2024-09-11T23:18:22.328943+0000","last_peered":"2024-09-11T23:18:22.328943+0000","last_clean":"2024-09-11T23:18:22.328943+0000","last_became_active":"2024-09-11T23:16:25.902906+0000","last_became_peered":"2024-09-11T23:16:25.902906+0000","last_unstale":"2024-09-11T23:18:22.328943+0000","last_undegraded":"2024-09-11T23:18:22.328943+0000","last_fullsized":"2024-09-11T23:18:22.328943+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:40:49.326045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"85'8","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328557+0000","last_change":"2024-09-11T23:16:25.896564+0000","last_active":"2024-09-11T23:18:22.328557+0000","last_peered":"2024-09-11T23:18:22.328557+0000","last_clean":"2024-09-11T23:18:22.328557+0000","last_became_active":"2024-09-11T23:16:25.896301+0000","last_became_peered":"2024-09-11T23:16:25.896301+0000","last_unstale":"2024-09-11T23:18:22.328557+0000","last_undegraded":"2024-09-11T23:18:22.328557+0000","last_fullsized":"2024-09-11T23:18:22.328557+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:30:30.074853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"85'11","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.329804+0000","last_change":"2024-09-11T23:16:25.903491+0000","last_active":"2024-09-11T23:18:22.329804+0000","last_peered":"2024-09-11T23:18:22.329804+0000","last_clean":"2024-09-11T23:18:22.329804+0000","last_became_active":"2024-09-11T23:16:25.901340+0000","last_became_peered":"2024-09-11T23:16:25.901340+0000","last_unstale":"2024-09-11T23:18:22.329804+0000","last_undegraded":"2024-09-11T23:18:22.329804+0000","last_fullsized":"2024-09-11T23:18:22.329804+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:41:16.058034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"85'10","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328233+0000","last_change":"2024-09-11T23:16:25.901084+0000","last_active":"2024-09-11T23:18:22.328233+0000","last_peered":"2024-09-11T23:18:22.328233+0000","last_clean":"2024-09-11T23:18:22.328233+0000","last_became_active":"2024-09-11T23:16:25.900726+0000","last_became_peered":"2024-09-11T23:16:25.900726+0000","last_unstale":"2024-09-11T23:18:22.328233+0000","last_undegraded":"2024-09-11T23:18:22.328233+0000","last_fullsized":"2024-09-11T23:18:22.328233+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:57:40.955645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"85'8","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.182688+0000","last_change":"2024-09-11T23:16:25.900461+0000","last_active":"2024-09-11T23:18:22.182688+0000","last_peered":"2024-09-11T23:18:22.182688+0000","last_clean":"2024-09-11T23:18:22.182688+0000","last_became_active":"2024-09-11T23:16:25.900174+0000","last_became_peered":"2024-09-11T23:16:25.900174+0000","last_unstale":"2024-09-11T23:18:22.182688+0000","last_undegraded":"2024-09-11T23:18:22.182688+0000","last_fullsized":"2024-09-11T23:18:22.182688+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:51:50.953198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"85'6","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.184452+0000","last_change":"2024-09-11T23:16:25.907569+0000","last_active":"2024-09-11T23:18:22.184452+0000","last_peered":"2024-09-11T23:18:22.184452+0000","last_clean":"2024-09-11T23:18:22.184452+0000","last_became_active":"2024-09-11T23:16:25.907091+0000","last_became_peered":"2024-09-11T23:16:25.907091+0000","last_unstale":"2024-09-11T23:18:22.184452+0000","last_undegraded":"2024-09-11T23:18:22.184452+0000","last_fullsized":"2024-09-11T23:18:22.184452+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:53:45.961167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"85'2","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.183204+0000","last_change":"2024-09-11T23:16:25.903246+0000","last_active":"2024-09-11T23:18:22.183204+0000","last_peered":"2024-09-11T23:18:22.183204+0000","last_clean":"2024-09-11T23:18:22.183204+0000","last_became_active":"2024-09-11T23:16:25.902907+0000","last_became_peered":"2024-09-11T23:16:25.902907+0000","last_unstale":"2024-09-11T23:18:22.183204+0000","last_undegraded":"2024-09-11T23:18:22.183204+0000","last_fullsized":"2024-09-11T23:18:22.183204+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:20:38.868384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"85'15","reported_seq":34,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328979+0000","last_change":"2024-09-11T23:16:25.906298+0000","last_active":"2024-09-11T23:18:22.328979+0000","last_peered":"2024-09-11T23:18:22.328979+0000","last_clean":"2024-09-11T23:18:22.328979+0000","last_became_active":"2024-09-11T23:16:25.905350+0000","last_became_peered":"2024-09-11T23:16:25.905350+0000","last_unstale":"2024-09-11T23:18:22.328979+0000","last_undegraded":"2024-09-11T23:18:22.328979+0000","last_fullsized":"2024-09-11T23:18:22.328979+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:04:22.122514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"85'6","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328926+0000","last_change":"2024-09-11T23:16:25.902114+0000","last_active":"2024-09-11T23:18:22.328926+0000","last_peered":"2024-09-11T23:18:22.328926+0000","last_clean":"2024-09-11T23:18:22.328926+0000","last_became_active":"2024-09-11T23:16:25.901742+0000","last_became_peered":"2024-09-11T23:16:25.901742+0000","last_unstale":"2024-09-11T23:18:22.328926+0000","last_undegraded":"2024-09-11T23:18:22.328926+0000","last_fullsized":"2024-09-11T23:18:22.328926+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:02:10.640801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"85'4","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.329844+0000","last_change":"2024-09-11T23:16:25.898117+0000","last_active":"2024-09-11T23:18:22.329844+0000","last_peered":"2024-09-11T23:18:22.329844+0000","last_clean":"2024-09-11T23:18:22.329844+0000","last_became_active":"2024-09-11T23:16:25.897818+0000","last_became_peered":"2024-09-11T23:16:25.897818+0000","last_unstale":"2024-09-11T23:18:22.329844+0000","last_undegraded":"2024-09-11T23:18:22.329844+0000","last_fullsized":"2024-09-11T23:18:22.329844+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:27:37.175471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"85'7","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.544057+0000","last_change":"2024-09-11T23:16:25.905600+0000","last_active":"2024-09-11T23:18:22.544057+0000","last_peered":"2024-09-11T23:18:22.544057+0000","last_clean":"2024-09-11T23:18:22.544057+0000","last_became_active":"2024-09-11T23:16:25.903924+0000","last_became_peered":"2024-09-11T23:16:25.903924+0000","last_unstale":"2024-09-11T23:18:22.544057+0000","last_undegraded":"2024-09-11T23:18:22.544057+0000","last_fullsized":"2024-09-11T23:18:22.544057+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:20:29.929079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"85'8","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328507+0000","last_change":"2024-09-11T23:16:25.904763+0000","last_active":"2024-09-11T23:18:22.328507+0000","last_peered":"2024-09-11T23:18:22.328507+0000","last_clean":"2024-09-11T23:18:22.328507+0000","last_became_active":"2024-09-11T23:16:25.904406+0000","last_became_peered":"2024-09-11T23:16:25.904406+0000","last_unstale":"2024-09-11T23:18:22.328507+0000","last_undegraded":"2024-09-11T23:18:22.328507+0000","last_fullsized":"2024-09-11T23:18:22.328507+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:27:38.918017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"85'12","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328649+0000","last_change":"2024-09-11T23:16:25.906573+0000","last_active":"2024-09-11T23:18:22.328649+0000","last_peered":"2024-09-11T23:18:22.328649+0000","last_clean":"2024-09-11T23:18:22.328649+0000","last_became_active":"2024-09-11T23:16:25.904771+0000","last_became_peered":"2024-09-11T23:16:25.904771+0000","last_unstale":"2024-09-11T23:18:22.328649+0000","last_undegraded":"2024-09-11T23:18:22.328649+0000","last_fullsized":"2024-09-11T23:18:22.328649+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:04:17.090648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"85'9","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328406+0000","last_change":"2024-09-11T23:16:25.902123+0000","last_active":"2024-09-11T23:18:22.328406+0000","last_peered":"2024-09-11T23:18:22.328406+0000","last_clean":"2024-09-11T23:18:22.328406+0000","last_became_active":"2024-09-11T23:16:25.901756+0000","last_became_peered":"2024-09-11T23:16:25.901756+0000","last_unstale":"2024-09-11T23:18:22.328406+0000","last_undegraded":"2024-09-11T23:18:22.328406+0000","last_fullsized":"2024-09-11T23:18:22.328406+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:17:37.074976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"85'7","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.295802+0000","last_change":"2024-09-11T23:16:25.904217+0000","last_active":"2024-09-11T23:18:22.295802+0000","last_peered":"2024-09-11T23:18:22.295802+0000","last_clean":"2024-09-11T23:18:22.295802+0000","last_became_active":"2024-09-11T23:16:25.903522+0000","last_became_peered":"2024-09-11T23:16:25.903522+0000","last_unstale":"2024-09-11T23:18:22.295802+0000","last_undegraded":"2024-09-11T23:18:22.295802+0000","last_fullsized":"2024-09-11T23:18:22.295802+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:29:46.634475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"86'7","reported_seq":150,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:32.135914+0000","last_change":"2024-09-11T23:16:23.793696+0000","last_active":"2024-09-11T23:18:32.135914+0000","last_peered":"2024-09-11T23:18:32.135914+0000","last_clean":"2024-09-11T23:18:32.135914+0000","last_became_active":"2024-09-11T23:16:23.793062+0000","last_became_peered":"2024-09-11T23:16:23.793062+0000","last_unstale":"2024-09-11T23:18:32.135914+0000","last_undegraded":"2024-09-11T23:18:32.135914+0000","last_fullsized":"2024-09-11T23:18:32.135914+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:00:25.655817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1323,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":119,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"85'8","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.184453+0000","last_change":"2024-09-11T23:16:25.906673+0000","last_active":"2024-09-11T23:18:22.184453+0000","last_peered":"2024-09-11T23:18:22.184453+0000","last_clean":"2024-09-11T23:18:22.184453+0000","last_became_active":"2024-09-11T23:16:25.906239+0000","last_became_peered":"2024-09-11T23:16:25.906239+0000","last_unstale":"2024-09-11T23:18:22.184453+0000","last_undegraded":"2024-09-11T23:18:22.184453+0000","last_fullsized":"2024-09-11T23:18:22.184453+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:15:54.907269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"85'7","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.302905+0000","last_change":"2024-09-11T23:16:23.793302+0000","last_active":"2024-09-11T23:18:22.302905+0000","last_peered":"2024-09-11T23:18:22.302905+0000","last_clean":"2024-09-11T23:18:22.302905+0000","last_became_active":"2024-09-11T23:16:23.792874+0000","last_became_peered":"2024-09-11T23:16:23.792874+0000","last_unstale":"2024-09-11T23:18:22.302905+0000","last_undegraded":"2024-09-11T23:18:22.302905+0000","last_fullsized":"2024-09-11T23:18:22.302905+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:52:59.558223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"85'3","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328845+0000","last_change":"2024-09-11T23:16:23.792999+0000","last_active":"2024-09-11T23:18:22.328845+0000","last_peered":"2024-09-11T23:18:22.328845+0000","last_clean":"2024-09-11T23:18:22.328845+0000","last_became_active":"2024-09-11T23:16:23.792630+0000","last_became_peered":"2024-09-11T23:16:23.792630+0000","last_unstale":"2024-09-11T23:18:22.328845+0000","last_undegraded":"2024-09-11T23:18:22.328845+0000","last_fullsized":"2024-09-11T23:18:22.328845+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:09:28.001060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"85'8","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.292734+0000","last_change":"2024-09-11T23:16:35.943722+0000","last_active":"2024-09-11T23:18:22.292734+0000","last_peered":"2024-09-11T23:18:22.292734+0000","last_clean":"2024-09-11T23:18:22.292734+0000","last_became_active":"2024-09-11T23:16:35.943355+0000","last_became_peered":"2024-09-11T23:16:35.943355+0000","last_unstale":"2024-09-11T23:18:22.292734+0000","last_undegraded":"2024-09-11T23:18:22.292734+0000","last_fullsized":"2024-09-11T23:18:22.292734+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:26:22.769824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,6],"acting":[9,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"85'3","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.184646+0000","last_change":"2024-09-11T23:16:23.797945+0000","last_active":"2024-09-11T23:18:22.184646+0000","last_peered":"2024-09-11T23:18:22.184646+0000","last_clean":"2024-09-11T23:18:22.184646+0000","last_became_active":"2024-09-11T23:16:23.797572+0000","last_became_peered":"2024-09-11T23:16:23.797572+0000","last_unstale":"2024-09-11T23:18:22.184646+0000","last_undegraded":"2024-09-11T23:18:22.184646+0000","last_fullsized":"2024-09-11T23:18:22.184646+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:04:59.017668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"85'11","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.292877+0000","last_change":"2024-09-11T23:16:25.891587+0000","last_active":"2024-09-11T23:18:22.292877+0000","last_peered":"2024-09-11T23:18:22.292877+0000","last_clean":"2024-09-11T23:18:22.292877+0000","last_became_active":"2024-09-11T23:16:25.891190+0000","last_became_peered":"2024-09-11T23:16:25.891190+0000","last_unstale":"2024-09-11T23:18:22.292877+0000","last_undegraded":"2024-09-11T23:18:22.292877+0000","last_fullsized":"2024-09-11T23:18:22.292877+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:57:05.177843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"85'6","reported_seq":260,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:33.136590+0000","last_change":"2024-09-11T23:16:23.790129+0000","last_active":"2024-09-11T23:18:33.136590+0000","last_peered":"2024-09-11T23:18:33.136590+0000","last_clean":"2024-09-11T23:18:33.136590+0000","last_became_active":"2024-09-11T23:16:23.789720+0000","last_became_peered":"2024-09-11T23:16:23.789720+0000","last_unstale":"2024-09-11T23:18:33.136590+0000","last_undegraded":"2024-09-11T23:18:33.136590+0000","last_fullsized":"2024-09-11T23:18:33.136590+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:33:58.388513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":230,"num_read_kb":230,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"85'10","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.329142+0000","last_change":"2024-09-11T23:16:25.901014+0000","last_active":"2024-09-11T23:18:22.329142+0000","last_peered":"2024-09-11T23:18:22.329142+0000","last_clean":"2024-09-11T23:18:22.329142+0000","last_became_active":"2024-09-11T23:16:25.900280+0000","last_became_peered":"2024-09-11T23:16:25.900280+0000","last_unstale":"2024-09-11T23:18:22.329142+0000","last_undegraded":"2024-09-11T23:18:22.329142+0000","last_fullsized":"2024-09-11T23:18:22.329142+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:14:03.662204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"85'4","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328532+0000","last_change":"2024-09-11T23:16:23.797944+0000","last_active":"2024-09-11T23:18:22.328532+0000","last_peered":"2024-09-11T23:18:22.328532+0000","last_clean":"2024-09-11T23:18:22.328532+0000","last_became_active":"2024-09-11T23:16:23.797520+0000","last_became_peered":"2024-09-11T23:16:23.797520+0000","last_unstale":"2024-09-11T23:18:22.328532+0000","last_undegraded":"2024-09-11T23:18:22.328532+0000","last_fullsized":"2024-09-11T23:18:22.328532+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:29:06.957518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"85'9","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.544713+0000","last_change":"2024-09-11T23:16:25.905379+0000","last_active":"2024-09-11T23:18:22.544713+0000","last_peered":"2024-09-11T23:18:22.544713+0000","last_clean":"2024-09-11T23:18:22.544713+0000","last_became_active":"2024-09-11T23:16:25.905085+0000","last_became_peered":"2024-09-11T23:16:25.905085+0000","last_unstale":"2024-09-11T23:18:22.544713+0000","last_undegraded":"2024-09-11T23:18:22.544713+0000","last_fullsized":"2024-09-11T23:18:22.544713+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:22:35.129736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"85'5","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.184874+0000","last_change":"2024-09-11T23:16:24.075556+0000","last_active":"2024-09-11T23:18:22.184874+0000","last_peered":"2024-09-11T23:18:22.184874+0000","last_clean":"2024-09-11T23:18:22.184874+0000","last_became_active":"2024-09-11T23:16:24.074783+0000","last_became_peered":"2024-09-11T23:16:24.074783+0000","last_unstale":"2024-09-11T23:18:22.184874+0000","last_undegraded":"2024-09-11T23:18:22.184874+0000","last_fullsized":"2024-09-11T23:18:22.184874+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:55:09.962342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"85'11","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328166+0000","last_change":"2024-09-11T23:16:25.907720+0000","last_active":"2024-09-11T23:18:22.328166+0000","last_peered":"2024-09-11T23:18:22.328166+0000","last_clean":"2024-09-11T23:18:22.328166+0000","last_became_active":"2024-09-11T23:16:25.907335+0000","last_became_peered":"2024-09-11T23:16:25.907335+0000","last_unstale":"2024-09-11T23:18:22.328166+0000","last_undegraded":"2024-09-11T23:18:22.328166+0000","last_fullsized":"2024-09-11T23:18:22.328166+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:46:14.519384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"86'38","reported_seq":305,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:33.158862+0000","last_change":"2024-09-11T23:16:23.794321+0000","last_active":"2024-09-11T23:18:33.158862+0000","last_peered":"2024-09-11T23:18:33.158862+0000","last_clean":"2024-09-11T23:18:33.158862+0000","last_became_active":"2024-09-11T23:16:23.793955+0000","last_became_peered":"2024-09-11T23:16:23.793955+0000","last_unstale":"2024-09-11T23:18:33.158862+0000","last_undegraded":"2024-09-11T23:18:33.158862+0000","last_fullsized":"2024-09-11T23:18:33.158862+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:47:46.442656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":66480,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":243,"num_read_kb":186,"num_write":40,"num_write_kb":82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"85'6","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.296053+0000","last_change":"2024-09-11T23:16:25.896598+0000","last_active":"2024-09-11T23:18:22.296053+0000","last_peered":"2024-09-11T23:18:22.296053+0000","last_clean":"2024-09-11T23:18:22.296053+0000","last_became_active":"2024-09-11T23:16:25.896293+0000","last_became_peered":"2024-09-11T23:16:25.896293+0000","last_unstale":"2024-09-11T23:18:22.296053+0000","last_undegraded":"2024-09-11T23:18:22.296053+0000","last_fullsized":"2024-09-11T23:18:22.296053+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:04:11.489644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"85'9","reported_seq":35,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328677+0000","last_change":"2024-09-11T23:16:23.791486+0000","last_active":"2024-09-11T23:18:22.328677+0000","last_peered":"2024-09-11T23:18:22.328677+0000","last_clean":"2024-09-11T23:18:22.328677+0000","last_became_active":"2024-09-11T23:16:23.790786+0000","last_became_peered":"2024-09-11T23:16:23.790786+0000","last_unstale":"2024-09-11T23:18:22.328677+0000","last_undegraded":"2024-09-11T23:18:22.328677+0000","last_fullsized":"2024-09-11T23:18:22.328677+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:03:52.742560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"85'9","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.329171+0000","last_change":"2024-09-11T23:16:25.897382+0000","last_active":"2024-09-11T23:18:22.329171+0000","last_peered":"2024-09-11T23:18:22.329171+0000","last_clean":"2024-09-11T23:18:22.329171+0000","last_became_active":"2024-09-11T23:16:25.897073+0000","last_became_peered":"2024-09-11T23:16:25.897073+0000","last_unstale":"2024-09-11T23:18:22.329171+0000","last_undegraded":"2024-09-11T23:18:22.329171+0000","last_fullsized":"2024-09-11T23:18:22.329171+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:15:59.780100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"85'5","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.184727+0000","last_change":"2024-09-11T23:16:23.793886+0000","last_active":"2024-09-11T23:18:22.184727+0000","last_peered":"2024-09-11T23:18:22.184727+0000","last_clean":"2024-09-11T23:18:22.184727+0000","last_became_active":"2024-09-11T23:16:23.793066+0000","last_became_peered":"2024-09-11T23:16:23.793066+0000","last_unstale":"2024-09-11T23:18:22.184727+0000","last_undegraded":"2024-09-11T23:18:22.184727+0000","last_fullsized":"2024-09-11T23:18:22.184727+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:07:03.882546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"85'15","reported_seq":34,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328552+0000","last_change":"2024-09-11T23:16:25.904575+0000","last_active":"2024-09-11T23:18:22.328552+0000","last_peered":"2024-09-11T23:18:22.328552+0000","last_clean":"2024-09-11T23:18:22.328552+0000","last_became_active":"2024-09-11T23:16:25.904224+0000","last_became_peered":"2024-09-11T23:16:25.904224+0000","last_unstale":"2024-09-11T23:18:22.328552+0000","last_undegraded":"2024-09-11T23:18:22.328552+0000","last_fullsized":"2024-09-11T23:18:22.328552+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:42:13.036931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"85'5","reported_seq":146,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:32.135461+0000","last_change":"2024-09-11T23:16:23.796073+0000","last_active":"2024-09-11T23:18:32.135461+0000","last_peered":"2024-09-11T23:18:32.135461+0000","last_clean":"2024-09-11T23:18:32.135461+0000","last_became_active":"2024-09-11T23:16:23.794850+0000","last_became_peered":"2024-09-11T23:16:23.794850+0000","last_unstale":"2024-09-11T23:18:32.135461+0000","last_undegraded":"2024-09-11T23:18:32.135461+0000","last_fullsized":"2024-09-11T23:18:32.135461+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:57:22.361662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":118,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"85'6","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328221+0000","last_change":"2024-09-11T23:16:25.908700+0000","last_active":"2024-09-11T23:18:22.328221+0000","last_peered":"2024-09-11T23:18:22.328221+0000","last_clean":"2024-09-11T23:18:22.328221+0000","last_became_active":"2024-09-11T23:16:25.908373+0000","last_became_peered":"2024-09-11T23:16:25.908373+0000","last_unstale":"2024-09-11T23:18:22.328221+0000","last_undegraded":"2024-09-11T23:18:22.328221+0000","last_fullsized":"2024-09-11T23:18:22.328221+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:20:18.536079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"85'17","reported_seq":42,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.295792+0000","last_change":"2024-09-11T23:16:24.075927+0000","last_active":"2024-09-11T23:18:22.295792+0000","last_peered":"2024-09-11T23:18:22.295792+0000","last_clean":"2024-09-11T23:18:22.295792+0000","last_became_active":"2024-09-11T23:16:24.075527+0000","last_became_peered":"2024-09-11T23:16:24.075527+0000","last_unstale":"2024-09-11T23:18:22.295792+0000","last_undegraded":"2024-09-11T23:18:22.295792+0000","last_fullsized":"2024-09-11T23:18:22.295792+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:27:54.622059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"85'6","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.184380+0000","last_change":"2024-09-11T23:16:35.947475+0000","last_active":"2024-09-11T23:18:22.184380+0000","last_peered":"2024-09-11T23:18:22.184380+0000","last_clean":"2024-09-11T23:18:22.184380+0000","last_became_active":"2024-09-11T23:16:35.947028+0000","last_became_peered":"2024-09-11T23:16:35.947028+0000","last_unstale":"2024-09-11T23:18:22.184380+0000","last_undegraded":"2024-09-11T23:18:22.184380+0000","last_fullsized":"2024-09-11T23:18:22.184380+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:18:13.479554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'2","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.296198+0000","last_change":"2024-09-11T23:16:23.793298+0000","last_active":"2024-09-11T23:18:22.296198+0000","last_peered":"2024-09-11T23:18:22.296198+0000","last_clean":"2024-09-11T23:18:22.296198+0000","last_became_active":"2024-09-11T23:16:23.792947+0000","last_became_peered":"2024-09-11T23:16:23.792947+0000","last_unstale":"2024-09-11T23:18:22.296198+0000","last_undegraded":"2024-09-11T23:18:22.296198+0000","last_fullsized":"2024-09-11T23:18:22.296198+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:32:21.294560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"85'5","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328871+0000","last_change":"2024-09-11T23:16:25.902348+0000","last_active":"2024-09-11T23:18:22.328871+0000","last_peered":"2024-09-11T23:18:22.328871+0000","last_clean":"2024-09-11T23:18:22.328871+0000","last_became_active":"2024-09-11T23:16:25.901686+0000","last_became_peered":"2024-09-11T23:16:25.901686+0000","last_unstale":"2024-09-11T23:18:22.328871+0000","last_undegraded":"2024-09-11T23:18:22.328871+0000","last_fullsized":"2024-09-11T23:18:22.328871+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:00:15.543003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"85'5","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.330216+0000","last_change":"2024-09-11T23:16:23.790984+0000","last_active":"2024-09-11T23:18:22.330216+0000","last_peered":"2024-09-11T23:18:22.330216+0000","last_clean":"2024-09-11T23:18:22.330216+0000","last_became_active":"2024-09-11T23:16:23.790569+0000","last_became_peered":"2024-09-11T23:16:23.790569+0000","last_unstale":"2024-09-11T23:18:22.330216+0000","last_undegraded":"2024-09-11T23:18:22.330216+0000","last_fullsized":"2024-09-11T23:18:22.330216+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:57:39.675793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"85'9","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328796+0000","last_change":"2024-09-11T23:16:25.899249+0000","last_active":"2024-09-11T23:18:22.328796+0000","last_peered":"2024-09-11T23:18:22.328796+0000","last_clean":"2024-09-11T23:18:22.328796+0000","last_became_active":"2024-09-11T23:16:25.898914+0000","last_became_peered":"2024-09-11T23:16:25.898914+0000","last_unstale":"2024-09-11T23:18:22.328796+0000","last_undegraded":"2024-09-11T23:18:22.328796+0000","last_fullsized":"2024-09-11T23:18:22.328796+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:17:07.545770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"86'67","reported_seq":122,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:31.561400+0000","last_change":"2024-09-11T23:16:23.787387+0000","last_active":"2024-09-11T23:18:31.561400+0000","last_peered":"2024-09-11T23:18:31.561400+0000","last_clean":"2024-09-11T23:18:31.561400+0000","last_became_active":"2024-09-11T23:16:23.787014+0000","last_became_peered":"2024-09-11T23:16:23.787014+0000","last_unstale":"2024-09-11T23:18:31.561400+0000","last_undegraded":"2024-09-11T23:18:31.561400+0000","last_fullsized":"2024-09-11T23:18:31.561400+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":67,"log_dups_size":0,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:45:52.004253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":328302,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":34,"num_read_kb":33,"num_write":76,"num_write_kb":941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"85'7","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328671+0000","last_change":"2024-09-11T23:16:25.908304+0000","last_active":"2024-09-11T23:18:22.328671+0000","last_peered":"2024-09-11T23:18:22.328671+0000","last_clean":"2024-09-11T23:18:22.328671+0000","last_became_active":"2024-09-11T23:16:25.907900+0000","last_became_peered":"2024-09-11T23:16:25.907900+0000","last_unstale":"2024-09-11T23:18:22.328671+0000","last_undegraded":"2024-09-11T23:18:22.328671+0000","last_fullsized":"2024-09-11T23:18:22.328671+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:43:26.247682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"85'7","reported_seq":150,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:32.135824+0000","last_change":"2024-09-11T23:16:24.074610+0000","last_active":"2024-09-11T23:18:32.135824+0000","last_peered":"2024-09-11T23:18:32.135824+0000","last_clean":"2024-09-11T23:18:32.135824+0000","last_became_active":"2024-09-11T23:16:24.074210+0000","last_became_peered":"2024-09-11T23:16:24.074210+0000","last_unstale":"2024-09-11T23:18:32.135824+0000","last_undegraded":"2024-09-11T23:18:32.135824+0000","last_fullsized":"2024-09-11T23:18:32.135824+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:09:37.167050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1704,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":119,"num_read_kb":4,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"85'7","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328842+0000","last_change":"2024-09-11T23:16:25.903120+0000","last_active":"2024-09-11T23:18:22.328842+0000","last_peered":"2024-09-11T23:18:22.328842+0000","last_clean":"2024-09-11T23:18:22.328842+0000","last_became_active":"2024-09-11T23:16:25.902770+0000","last_became_peered":"2024-09-11T23:16:25.902770+0000","last_unstale":"2024-09-11T23:18:22.328842+0000","last_undegraded":"2024-09-11T23:18:22.328842+0000","last_fullsized":"2024-09-11T23:18:22.328842+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:03:30.041088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"86'148","reported_seq":393,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:24.295645+0000","last_change":"2024-09-11T23:15:13.115055+0000","last_active":"2024-09-11T23:18:24.295645+0000","last_peered":"2024-09-11T23:18:24.295645+0000","last_clean":"2024-09-11T23:18:24.295645+0000","last_became_active":"2024-09-11T23:15:13.114480+0000","last_became_peered":"2024-09-11T23:15:13.114480+0000","last_unstale":"2024-09-11T23:18:24.295645+0000","last_undegraded":"2024-09-11T23:18:24.295645+0000","last_fullsized":"2024-09-11T23:18:24.295645+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:11:50.495799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:11:50.495799+0000","last_clean_scrub_stamp":"2024-09-11T23:11:50.495799+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:15:53.681230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"85'6","reported_seq":34,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.187143+0000","last_change":"2024-09-11T23:16:23.793996+0000","last_active":"2024-09-11T23:18:22.187143+0000","last_peered":"2024-09-11T23:18:22.187143+0000","last_clean":"2024-09-11T23:18:22.187143+0000","last_became_active":"2024-09-11T23:16:23.793681+0000","last_became_peered":"2024-09-11T23:16:23.793681+0000","last_unstale":"2024-09-11T23:18:22.187143+0000","last_undegraded":"2024-09-11T23:18:22.187143+0000","last_fullsized":"2024-09-11T23:18:22.187143+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:52:50.463623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"85'8","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.182978+0000","last_change":"2024-09-11T23:16:25.905133+0000","last_active":"2024-09-11T23:18:22.182978+0000","last_peered":"2024-09-11T23:18:22.182978+0000","last_clean":"2024-09-11T23:18:22.182978+0000","last_became_active":"2024-09-11T23:16:25.904819+0000","last_became_peered":"2024-09-11T23:16:25.904819+0000","last_unstale":"2024-09-11T23:18:22.182978+0000","last_undegraded":"2024-09-11T23:18:22.182978+0000","last_fullsized":"2024-09-11T23:18:22.182978+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:51:09.664343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"85'4","reported_seq":35,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328418+0000","last_change":"2024-09-11T23:16:36.228806+0000","last_active":"2024-09-11T23:18:22.328418+0000","last_peered":"2024-09-11T23:18:22.328418+0000","last_clean":"2024-09-11T23:18:22.328418+0000","last_became_active":"2024-09-11T23:16:36.228311+0000","last_became_peered":"2024-09-11T23:16:36.228311+0000","last_unstale":"2024-09-11T23:18:22.328418+0000","last_undegraded":"2024-09-11T23:18:22.328418+0000","last_fullsized":"2024-09-11T23:18:22.328418+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:42:32.856691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4],"acting":[7,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"85'5","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328756+0000","last_change":"2024-09-11T23:16:25.898935+0000","last_active":"2024-09-11T23:18:22.328756+0000","last_peered":"2024-09-11T23:18:22.328756+0000","last_clean":"2024-09-11T23:18:22.328756+0000","last_became_active":"2024-09-11T23:16:25.898618+0000","last_became_peered":"2024-09-11T23:16:25.898618+0000","last_unstale":"2024-09-11T23:18:22.328756+0000","last_undegraded":"2024-09-11T23:18:22.328756+0000","last_fullsized":"2024-09-11T23:18:22.328756+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:13:29.104911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"85'5","reported_seq":146,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:32.134935+0000","last_change":"2024-09-11T23:16:24.073613+0000","last_active":"2024-09-11T23:18:32.134935+0000","last_peered":"2024-09-11T23:18:32.134935+0000","last_clean":"2024-09-11T23:18:32.134935+0000","last_became_active":"2024-09-11T23:16:24.073262+0000","last_became_peered":"2024-09-11T23:16:24.073262+0000","last_unstale":"2024-09-11T23:18:32.134935+0000","last_undegraded":"2024-09-11T23:18:32.134935+0000","last_fullsized":"2024-09-11T23:18:32.134935+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:31:58.653657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":116,"num_read_kb":116,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"85'8","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.184531+0000","last_change":"2024-09-11T23:16:25.903725+0000","last_active":"2024-09-11T23:18:22.184531+0000","last_peered":"2024-09-11T23:18:22.184531+0000","last_clean":"2024-09-11T23:18:22.184531+0000","last_became_active":"2024-09-11T23:16:25.903375+0000","last_became_peered":"2024-09-11T23:16:25.903375+0000","last_unstale":"2024-09-11T23:18:22.184531+0000","last_undegraded":"2024-09-11T23:18:22.184531+0000","last_fullsized":"2024-09-11T23:18:22.184531+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:55:02.779575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"86'6","reported_seq":148,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:32.135224+0000","last_change":"2024-09-11T23:16:23.793932+0000","last_active":"2024-09-11T23:18:32.135224+0000","last_peered":"2024-09-11T23:18:32.135224+0000","last_clean":"2024-09-11T23:18:32.135224+0000","last_became_active":"2024-09-11T23:16:23.793111+0000","last_became_peered":"2024-09-11T23:16:23.793111+0000","last_unstale":"2024-09-11T23:18:32.135224+0000","last_undegraded":"2024-09-11T23:18:32.135224+0000","last_fullsized":"2024-09-11T23:18:32.135224+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:14:09.087222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":117,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"85'7","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.303042+0000","last_change":"2024-09-11T23:16:25.905020+0000","last_active":"2024-09-11T23:18:22.303042+0000","last_peered":"2024-09-11T23:18:22.303042+0000","last_clean":"2024-09-11T23:18:22.303042+0000","last_became_active":"2024-09-11T23:16:25.904675+0000","last_became_peered":"2024-09-11T23:16:25.904675+0000","last_unstale":"2024-09-11T23:18:22.303042+0000","last_undegraded":"2024-09-11T23:18:22.303042+0000","last_fullsized":"2024-09-11T23:18:22.303042+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:13:57.388773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"85'1","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.293308+0000","last_change":"2024-09-11T23:16:24.073630+0000","last_active":"2024-09-11T23:18:22.293308+0000","last_peered":"2024-09-11T23:18:22.293308+0000","last_clean":"2024-09-11T23:18:22.293308+0000","last_became_active":"2024-09-11T23:16:24.073301+0000","last_became_peered":"2024-09-11T23:16:24.073301+0000","last_unstale":"2024-09-11T23:18:22.293308+0000","last_undegraded":"2024-09-11T23:18:22.293308+0000","last_fullsized":"2024-09-11T23:18:22.293308+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:41:59.374581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"85'5","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.303107+0000","last_change":"2024-09-11T23:16:25.894124+0000","last_active":"2024-09-11T23:18:22.303107+0000","last_peered":"2024-09-11T23:18:22.303107+0000","last_clean":"2024-09-11T23:18:22.303107+0000","last_became_active":"2024-09-11T23:16:25.893751+0000","last_became_peered":"2024-09-11T23:16:25.893751+0000","last_unstale":"2024-09-11T23:18:22.303107+0000","last_undegraded":"2024-09-11T23:18:22.303107+0000","last_fullsized":"2024-09-11T23:18:22.303107+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:00:28.265206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"85'4","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.296263+0000","last_change":"2024-09-11T23:16:24.073700+0000","last_active":"2024-09-11T23:18:22.296263+0000","last_peered":"2024-09-11T23:18:22.296263+0000","last_clean":"2024-09-11T23:18:22.296263+0000","last_became_active":"2024-09-11T23:16:24.073283+0000","last_became_peered":"2024-09-11T23:16:24.073283+0000","last_unstale":"2024-09-11T23:18:22.296263+0000","last_undegraded":"2024-09-11T23:18:22.296263+0000","last_fullsized":"2024-09-11T23:18:22.296263+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:41:43.536834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"85'9","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.329964+0000","last_change":"2024-09-11T23:16:25.900532+0000","last_active":"2024-09-11T23:18:22.329964+0000","last_peered":"2024-09-11T23:18:22.329964+0000","last_clean":"2024-09-11T23:18:22.329964+0000","last_became_active":"2024-09-11T23:16:25.899466+0000","last_became_peered":"2024-09-11T23:16:25.899466+0000","last_unstale":"2024-09-11T23:18:22.329964+0000","last_undegraded":"2024-09-11T23:18:22.329964+0000","last_fullsized":"2024-09-11T23:18:22.329964+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:10:01.152323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"86'103","reported_seq":220,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:31.947637+0000","last_change":"2024-09-11T23:16:23.788789+0000","last_active":"2024-09-11T23:18:31.947637+0000","last_peered":"2024-09-11T23:18:31.947637+0000","last_clean":"2024-09-11T23:18:31.947637+0000","last_became_active":"2024-09-11T23:16:23.788422+0000","last_became_peered":"2024-09-11T23:16:23.788422+0000","last_unstale":"2024-09-11T23:18:31.947637+0000","last_undegraded":"2024-09-11T23:18:31.947637+0000","last_fullsized":"2024-09-11T23:18:31.947637+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":103,"log_dups_size":0,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:09:14.310205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":312,"num_read_kb":277,"num_write":188,"num_write_kb":1285,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"85'5","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.183270+0000","last_change":"2024-09-11T23:16:25.895511+0000","last_active":"2024-09-11T23:18:22.183270+0000","last_peered":"2024-09-11T23:18:22.183270+0000","last_clean":"2024-09-11T23:18:22.183270+0000","last_became_active":"2024-09-11T23:16:25.895143+0000","last_became_peered":"2024-09-11T23:16:25.895143+0000","last_unstale":"2024-09-11T23:18:22.183270+0000","last_undegraded":"2024-09-11T23:18:22.183270+0000","last_fullsized":"2024-09-11T23:18:22.183270+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:31:16.498371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"85'7","reported_seq":33,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.187236+0000","last_change":"2024-09-11T23:16:24.019064+0000","last_active":"2024-09-11T23:18:22.187236+0000","last_peered":"2024-09-11T23:18:22.187236+0000","last_clean":"2024-09-11T23:18:22.187236+0000","last_became_active":"2024-09-11T23:16:24.018658+0000","last_became_peered":"2024-09-11T23:16:24.018658+0000","last_unstale":"2024-09-11T23:18:22.187236+0000","last_undegraded":"2024-09-11T23:18:22.187236+0000","last_fullsized":"2024-09-11T23:18:22.187236+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:17:41.009507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"85'7","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.544339+0000","last_change":"2024-09-11T23:16:25.903553+0000","last_active":"2024-09-11T23:18:22.544339+0000","last_peered":"2024-09-11T23:18:22.544339+0000","last_clean":"2024-09-11T23:18:22.544339+0000","last_became_active":"2024-09-11T23:16:25.903221+0000","last_became_peered":"2024-09-11T23:16:25.903221+0000","last_unstale":"2024-09-11T23:18:22.544339+0000","last_undegraded":"2024-09-11T23:18:22.544339+0000","last_fullsized":"2024-09-11T23:18:22.544339+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:16:51.872517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"83'2","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328204+0000","last_change":"2024-09-11T23:16:24.073784+0000","last_active":"2024-09-11T23:18:22.328204+0000","last_peered":"2024-09-11T23:18:22.328204+0000","last_clean":"2024-09-11T23:18:22.328204+0000","last_became_active":"2024-09-11T23:16:24.073406+0000","last_became_peered":"2024-09-11T23:16:24.073406+0000","last_unstale":"2024-09-11T23:18:22.328204+0000","last_undegraded":"2024-09-11T23:18:22.328204+0000","last_fullsized":"2024-09-11T23:18:22.328204+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:31:24.617025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"85'2","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.295796+0000","last_change":"2024-09-11T23:16:25.894878+0000","last_active":"2024-09-11T23:18:22.295796+0000","last_peered":"2024-09-11T23:18:22.295796+0000","last_clean":"2024-09-11T23:18:22.295796+0000","last_became_active":"2024-09-11T23:16:25.894543+0000","last_became_peered":"2024-09-11T23:16:25.894543+0000","last_unstale":"2024-09-11T23:18:22.295796+0000","last_undegraded":"2024-09-11T23:18:22.295796+0000","last_fullsized":"2024-09-11T23:18:22.295796+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:34:13.435754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"85'7","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.182832+0000","last_change":"2024-09-11T23:16:23.786502+0000","last_active":"2024-09-11T23:18:22.182832+0000","last_peered":"2024-09-11T23:18:22.182832+0000","last_clean":"2024-09-11T23:18:22.182832+0000","last_became_active":"2024-09-11T23:16:23.786182+0000","last_became_peered":"2024-09-11T23:16:23.786182+0000","last_unstale":"2024-09-11T23:18:22.182832+0000","last_undegraded":"2024-09-11T23:18:22.182832+0000","last_fullsized":"2024-09-11T23:18:22.182832+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:59:07.978244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"85'9","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328520+0000","last_change":"2024-09-11T23:16:25.899625+0000","last_active":"2024-09-11T23:18:22.328520+0000","last_peered":"2024-09-11T23:18:22.328520+0000","last_clean":"2024-09-11T23:18:22.328520+0000","last_became_active":"2024-09-11T23:16:25.899265+0000","last_became_peered":"2024-09-11T23:16:25.899265+0000","last_unstale":"2024-09-11T23:18:22.328520+0000","last_undegraded":"2024-09-11T23:18:22.328520+0000","last_fullsized":"2024-09-11T23:18:22.328520+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:36:06.907622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"86'4","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328472+0000","last_change":"2024-09-11T23:16:36.228991+0000","last_active":"2024-09-11T23:18:22.328472+0000","last_peered":"2024-09-11T23:18:22.328472+0000","last_clean":"2024-09-11T23:18:22.328472+0000","last_became_active":"2024-09-11T23:16:36.227701+0000","last_became_peered":"2024-09-11T23:16:36.227701+0000","last_unstale":"2024-09-11T23:18:22.328472+0000","last_undegraded":"2024-09-11T23:18:22.328472+0000","last_fullsized":"2024-09-11T23:18:22.328472+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:37:57.478572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"85'8","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.182913+0000","last_change":"2024-09-11T23:16:25.894803+0000","last_active":"2024-09-11T23:18:22.182913+0000","last_peered":"2024-09-11T23:18:22.182913+0000","last_clean":"2024-09-11T23:18:22.182913+0000","last_became_active":"2024-09-11T23:16:25.894429+0000","last_became_peered":"2024-09-11T23:16:25.894429+0000","last_unstale":"2024-09-11T23:18:22.182913+0000","last_undegraded":"2024-09-11T23:18:22.182913+0000","last_fullsized":"2024-09-11T23:18:22.182913+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:39:15.868910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"83'4","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.295871+0000","last_change":"2024-09-11T23:16:23.794271+0000","last_active":"2024-09-11T23:18:22.295871+0000","last_peered":"2024-09-11T23:18:22.295871+0000","last_clean":"2024-09-11T23:18:22.295871+0000","last_became_active":"2024-09-11T23:16:23.793919+0000","last_became_peered":"2024-09-11T23:16:23.793919+0000","last_unstale":"2024-09-11T23:18:22.295871+0000","last_undegraded":"2024-09-11T23:18:22.295871+0000","last_fullsized":"2024-09-11T23:18:22.295871+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:30:28.907312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"85'5","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328395+0000","last_change":"2024-09-11T23:16:25.900969+0000","last_active":"2024-09-11T23:18:22.328395+0000","last_peered":"2024-09-11T23:18:22.328395+0000","last_clean":"2024-09-11T23:18:22.328395+0000","last_became_active":"2024-09-11T23:16:25.900425+0000","last_became_peered":"2024-09-11T23:16:25.900425+0000","last_unstale":"2024-09-11T23:18:22.328395+0000","last_undegraded":"2024-09-11T23:18:22.328395+0000","last_fullsized":"2024-09-11T23:18:22.328395+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:03:25.185773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"85'15","reported_seq":34,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.184711+0000","last_change":"2024-09-11T23:16:25.911605+0000","last_active":"2024-09-11T23:18:22.184711+0000","last_peered":"2024-09-11T23:18:22.184711+0000","last_clean":"2024-09-11T23:18:22.184711+0000","last_became_active":"2024-09-11T23:16:25.911180+0000","last_became_peered":"2024-09-11T23:16:25.911180+0000","last_unstale":"2024-09-11T23:18:22.184711+0000","last_undegraded":"2024-09-11T23:18:22.184711+0000","last_fullsized":"2024-09-11T23:18:22.184711+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:55:38.062606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"85'5","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328425+0000","last_change":"2024-09-11T23:16:35.949528+0000","last_active":"2024-09-11T23:18:22.328425+0000","last_peered":"2024-09-11T23:18:22.328425+0000","last_clean":"2024-09-11T23:18:22.328425+0000","last_became_active":"2024-09-11T23:16:35.945679+0000","last_became_peered":"2024-09-11T23:16:35.945679+0000","last_unstale":"2024-09-11T23:18:22.328425+0000","last_undegraded":"2024-09-11T23:18:22.328425+0000","last_fullsized":"2024-09-11T23:18:22.328425+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:14:24.706044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"85'9","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.328158+0000","last_change":"2024-09-11T23:16:25.900526+0000","last_active":"2024-09-11T23:18:22.328158+0000","last_peered":"2024-09-11T23:18:22.328158+0000","last_clean":"2024-09-11T23:18:22.328158+0000","last_became_active":"2024-09-11T23:16:25.900148+0000","last_became_peered":"2024-09-11T23:16:25.900148+0000","last_unstale":"2024-09-11T23:18:22.328158+0000","last_undegraded":"2024-09-11T23:18:22.328158+0000","last_fullsized":"2024-09-11T23:18:22.328158+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:26:10.582473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"86'13","reported_seq":48,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.296144+0000","last_change":"2024-09-11T23:16:36.233273+0000","last_active":"2024-09-11T23:18:22.296144+0000","last_peered":"2024-09-11T23:18:22.296144+0000","last_clean":"2024-09-11T23:18:22.296144+0000","last_became_active":"2024-09-11T23:16:36.228358+0000","last_became_peered":"2024-09-11T23:16:36.228358+0000","last_unstale":"2024-09-11T23:18:22.296144+0000","last_undegraded":"2024-09-11T23:18:22.296144+0000","last_fullsized":"2024-09-11T23:18:22.296144+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:06:31.225795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"85'6","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.544620+0000","last_change":"2024-09-11T23:16:25.907874+0000","last_active":"2024-09-11T23:18:22.544620+0000","last_peered":"2024-09-11T23:18:22.544620+0000","last_clean":"2024-09-11T23:18:22.544620+0000","last_became_active":"2024-09-11T23:16:25.907491+0000","last_became_peered":"2024-09-11T23:16:25.907491+0000","last_unstale":"2024-09-11T23:18:22.544620+0000","last_undegraded":"2024-09-11T23:18:22.544620+0000","last_fullsized":"2024-09-11T23:18:22.544620+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:09:20.984249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"84'2","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.183326+0000","last_change":"2024-09-11T23:16:24.074355+0000","last_active":"2024-09-11T23:18:22.183326+0000","last_peered":"2024-09-11T23:18:22.183326+0000","last_clean":"2024-09-11T23:18:22.183326+0000","last_became_active":"2024-09-11T23:16:24.074047+0000","last_became_peered":"2024-09-11T23:16:24.074047+0000","last_unstale":"2024-09-11T23:18:22.183326+0000","last_undegraded":"2024-09-11T23:18:22.183326+0000","last_fullsized":"2024-09-11T23:18:22.183326+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:31:58.804420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"85'5","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:22.182974+0000","last_change":"2024-09-11T23:16:25.907701+0000","last_active":"2024-09-11T23:18:22.182974+0000","last_peered":"2024-09-11T23:18:22.182974+0000","last_clean":"2024-09-11T23:18:22.182974+0000","last_became_active":"2024-09-11T23:16:25.907379+0000","last_became_peered":"2024-09-11T23:16:25.907379+0000","last_unstale":"2024-09-11T23:18:22.182974+0000","last_undegraded":"2024-09-11T23:18:22.182974+0000","last_fullsized":"2024-09-11T23:18:22.182974+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:24.876829+0000","last_clean_scrub_stamp":"2024-09-11T23:16:24.876829+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:12:03.639055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"86'14","reported_seq":156,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T23:18:32.134506+0000","last_change":"2024-09-11T23:16:23.794293+0000","last_active":"2024-09-11T23:18:32.134506+0000","last_peered":"2024-09-11T23:18:32.134506+0000","last_clean":"2024-09-11T23:18:32.134506+0000","last_became_active":"2024-09-11T23:16:23.793974+0000","last_became_peered":"2024-09-11T23:16:23.793974+0000","last_unstale":"2024-09-11T23:18:32.134506+0000","last_undegraded":"2024-09-11T23:18:32.134506+0000","last_fullsized":"2024-09-11T23:18:32.134506+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:16:22.771659+0000","last_clean_scrub_stamp":"2024-09-11T23:16:22.771659+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:24:39.170188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":117,"num_read_kb":117,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":795978,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1544,"num_read_kb":991,"num_write":498,"num_write_kb":2376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2764800,"data_stored":2387934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":377,"ondisk_log_size":377,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":11,"up_from":77,"seq":330712481831,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32128,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713024,"statfs":{"total":95995035648,"available":95962136576,"internally_reserved":0,"allocated":962560,"data_stored":459042,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":70,"seq":300647710763,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":29096,"kb_used_data":1952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716056,"statfs":{"total":95995035648,"available":95965241344,"internally_reserved":0,"allocated":1998848,"data_stored":1443667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":65,"seq":279172874289,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28368,"kb_used_data":1344,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716784,"statfs":{"total":95995035648,"available":95965986816,"internally_reserved":0,"allocated":1376256,"data_stored":851072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":57,"seq":244813135926,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28360,"kb_used_data":1336,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716792,"statfs":{"total":95995035648,"available":95965995008,"internally_reserved":0,"allocated":1368064,"data_stored":850868,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":50,"seq":214748364858,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28568,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716584,"statfs":{"total":95995035648,"available":95965782016,"internally_reserved":0,"allocated":1576960,"data_stored":1051455,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528382,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36292,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708860,"statfs":{"total":95995035648,"available":95957872640,"internally_reserved":0,"allocated":966656,"data_stored":457966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790021,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":29020,"kb_used_data":1876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716132,"statfs":{"total":95995035648,"available":95965319168,"internally_reserved":0,"allocated":1921024,"data_stored":1378972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920842,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36292,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708860,"statfs":{"total":95995035648,"available":95957872640,"internally_reserved":0,"allocated":966656,"data_stored":457434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182479,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":41292,"kb_used_data":1668,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93703860,"statfs":{"total":95995035648,"available":95952752640,"internally_reserved":0,"allocated":1708032,"data_stored":1178846,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411412,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36380,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708772,"statfs":{"total":95995035648,"available":95957782528,"internally_reserved":0,"allocated":1056768,"data_stored":526421,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574937,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1036288,"data_stored":525359,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738462,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":987136,"data_stored":459446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":68625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":69687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":753664,"data_stored":721878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":331753,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":421888,"data_stored":394134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":394338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":454656,"data_stored":396448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2308,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T23:18:37.333 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:18:37.596 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:18:38.000 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:37 smithi026 ceph-mon[31172]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 76 KiB/s wr, 50 op/s 2024-09-11T23:18:38.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:37 smithi026 ceph-mon[31172]: from='client.15381 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T23:18:38.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:37 smithi076 ceph-mon[35296]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 76 KiB/s wr, 50 op/s 2024-09-11T23:18:38.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:37 smithi076 ceph-mon[35296]: from='client.15381 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T23:18:38.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:38.083+0000 7f4e0e4e6640 1 -- 172.21.15.26:0/674162963 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4e08102d40 msgr2=0x7f4e08103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:38.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:38.083+0000 7f4e0e4e6640 1 --2- 172.21.15.26:0/674162963 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4e08102d40 0x7f4e08103120 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f4df8009900 tx=0x7f4df802a480 comp rx=0 tx=0).stop 2024-09-11T23:18:38.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:38.083+0000 7f4e0e4e6640 1 -- 172.21.15.26:0/674162963 shutdown_connections 2024-09-11T23:18:38.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:38.083+0000 7f4e0e4e6640 1 --2- 172.21.15.26:0/674162963 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4e0810c170 0x7f4e0810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:38.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:38.083+0000 7f4e0e4e6640 1 --2- 172.21.15.26:0/674162963 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4e08103660 0x7f4e0810bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:38.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:38.083+0000 7f4e0e4e6640 1 --2- 172.21.15.26:0/674162963 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4e08102d40 0x7f4e08103120 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:38.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:38.083+0000 7f4e0e4e6640 1 -- 172.21.15.26:0/674162963 >> 172.21.15.26:0/674162963 conn(0x7f4e080fc7a0 msgr2=0x7f4e080febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:18:38.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:38.083+0000 7f4e0e4e6640 1 -- 172.21.15.26:0/674162963 shutdown_connections 2024-09-11T23:18:38.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:38.083+0000 7f4e0e4e6640 1 -- 172.21.15.26:0/674162963 wait complete. 2024-09-11T23:18:38.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:38.084+0000 7f4e0e4e6640 1 Processor -- start 2024-09-11T23:18:38.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:38.084+0000 7f4e0e4e6640 1 -- start start 2024-09-11T23:18:38.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:38.084+0000 7f4e0e4e6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4e08102d40 0x7f4e0819a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:38.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:38.084+0000 7f4e0e4e6640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4e08103660 0x7f4e0819abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:38.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:38.084+0000 7f4e0e4e6640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4e0810c170 0x7f4e0819ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:38.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:38.084+0000 7f4e0e4e6640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f4e08111f70 con 0x7f4e08102d40 2024-09-11T23:18:38.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:38.084+0000 7f4e0e4e6640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f4e08111df0 con 0x7f4e08103660 2024-09-11T23:18:38.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:38.084+0000 7f4e0e4e6640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f4e081120f0 con 0x7f4e0810c170 2024-09-11T23:18:38.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:38.084+0000 7f4e07fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4e08102d40 0x7f4e0819a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:38.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:38.084+0000 7f4e07fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4e08102d40 0x7f4e0819a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60634/0 (socket says 172.21.15.26:60634) 2024-09-11T23:18:38.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:38.084+0000 7f4e07fff640 1 -- 172.21.15.26:0/3930636433 learned_addr learned my addr 172.21.15.26:0/3930636433 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:18:38.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:38.084+0000 7f4dfffff640 1 --2- 172.21.15.26:0/3930636433 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4e08103660 0x7f4e0819abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:38.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:38.084+0000 7f4e0ca83640 1 --2- 172.21.15.26:0/3930636433 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4e0810c170 0x7f4e0819ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:38.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:38.084+0000 7f4e07fff640 1 -- 172.21.15.26:0/3930636433 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4e0810c170 msgr2=0x7f4e0819ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:38.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:38.084+0000 7f4e07fff640 1 --2- 172.21.15.26:0/3930636433 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4e0810c170 0x7f4e0819ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:38.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:38.084+0000 7f4e07fff640 1 -- 172.21.15.26:0/3930636433 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4e08103660 msgr2=0x7f4e0819abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:38.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:38.084+0000 7f4e07fff640 1 --2- 172.21.15.26:0/3930636433 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4e08103660 0x7f4e0819abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:38.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:38.084+0000 7f4e07fff640 1 -- 172.21.15.26:0/3930636433 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e0819f620 con 0x7f4e08102d40 2024-09-11T23:18:38.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:38.084+0000 7f4dfffff640 1 --2- 172.21.15.26:0/3930636433 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4e08103660 0x7f4e0819abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:18:38.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:38.085+0000 7f4e0ca83640 1 --2- 172.21.15.26:0/3930636433 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4e0810c170 0x7f4e0819ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:18:38.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:38.085+0000 7f4e07fff640 1 --2- 172.21.15.26:0/3930636433 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4e08102d40 0x7f4e0819a670 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f4df8006010 tx=0x7f4df80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:18:38.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:38.085+0000 7f4e05ffb640 1 -- 172.21.15.26:0/3930636433 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4df803f030 con 0x7f4e08102d40 2024-09-11T23:18:38.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:38.085+0000 7f4e05ffb640 1 -- 172.21.15.26:0/3930636433 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4df8003ce0 con 0x7f4e08102d40 2024-09-11T23:18:38.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:38.085+0000 7f4e0e4e6640 1 -- 172.21.15.26:0/3930636433 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e0819f8b0 con 0x7f4e08102d40 2024-09-11T23:18:38.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:38.085+0000 7f4e05ffb640 1 -- 172.21.15.26:0/3930636433 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4df8035070 con 0x7f4e08102d40 2024-09-11T23:18:38.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:38.085+0000 7f4e0e4e6640 1 -- 172.21.15.26:0/3930636433 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e081a71f0 con 0x7f4e08102d40 2024-09-11T23:18:38.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:38.086+0000 7f4e0e4e6640 1 -- 172.21.15.26:0/3930636433 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e08103120 con 0x7f4e08102d40 2024-09-11T23:18:38.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:38.087+0000 7f4e05ffb640 1 -- 172.21.15.26:0/3930636433 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 24) ==== 138009+0+0 (secure 0 0 0) 0x7f4df8042030 con 0x7f4e08102d40 2024-09-11T23:18:38.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:38.088+0000 7f4e05ffb640 1 --2- 172.21.15.26:0/3930636433 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f4df40a2e00 0x7f4df40a52c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:38.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:38.088+0000 7f4dfffff640 1 --2- 172.21.15.26:0/3930636433 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f4df40a2e00 0x7f4df40a52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:38.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:38.088+0000 7f4e05ffb640 1 -- 172.21.15.26:0/3930636433 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f4df80ef730 con 0x7f4e08102d40 2024-09-11T23:18:38.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:38.088+0000 7f4dfffff640 1 --2- 172.21.15.26:0/3930636433 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f4df40a2e00 0x7f4df40a52c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f4e080febe0 tx=0x7f4df0005f90 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:18:38.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:38.092+0000 7f4e05ffb640 1 -- 172.21.15.26:0/3930636433 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4df80bc2b0 con 0x7f4e08102d40 2024-09-11T23:18:38.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:37 smithi136 ceph-mon[35121]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 76 KiB/s wr, 50 op/s 2024-09-11T23:18:38.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:37 smithi136 ceph-mon[35121]: from='client.15381 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T23:18:38.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:38.305+0000 7f4e0e4e6640 1 -- 172.21.15.26:0/3930636433 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4e081a73c0 con 0x7f4e08102d40 2024-09-11T23:18:38.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:38.306+0000 7f4e05ffb640 1 -- 172.21.15.26:0/3930636433 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11971 (secure 0 0 0) 0x7f4df80bbbb0 con 0x7f4e08102d40 2024-09-11T23:18:38.306 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-11T23:18:38.306 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":29,"btime":"2024-09-11T23:18:21:019324+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14499,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/367659515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":367659515},{"type":"v1","addr":"172.21.15.26:6835","nonce":367659515}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14511,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/2041065299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":2041065299},{"type":"v1","addr":"172.21.15.26:6837","nonce":2041065299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:16:26.544203+0000","modified":"2024-09-11T23:18:20.969151+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24485,"mds_1":24463,"mds_2":24458,"mds_3":24448,"mds_4":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14550":{"gid":14550,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6839/2233068305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":2233068305},{"type":"v1","addr":"172.21.15.26:6839","nonce":2233068305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14562":{"gid":14562,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6841/2984982336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":2984982336},{"type":"v1","addr":"172.21.15.26:6841","nonce":2984982336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6833/2019841455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":2019841455},{"type":"v1","addr":"172.21.15.136:6833","nonce":2019841455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.76:6833/3438147050","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3438147050},{"type":"v1","addr":"172.21.15.76:6833","nonce":3438147050}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6835/3617665975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3617665975},{"type":"v1","addr":"172.21.15.136:6835","nonce":3617665975}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":12,"addr":"172.21.15.76:6835/393308499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":393308499},{"type":"v1","addr":"172.21.15.76:6835","nonce":393308499}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":10,"addr":"172.21.15.136:6837/1633082387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":1633082387},{"type":"v1","addr":"172.21.15.136:6837","nonce":1633082387}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.76:6837/2078978748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":2078978748},{"type":"v1","addr":"172.21.15.76:6837","nonce":2078978748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.136:6839/3211398630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":3211398630},{"type":"v1","addr":"172.21.15.136:6839","nonce":3211398630}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.76:6839/2077637090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6838","nonce":2077637090},{"type":"v1","addr":"172.21.15.76:6839","nonce":2077637090}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24485,"qdb_cluster":[24443,24448,24458,24463,24485]},"id":1}]} 2024-09-11T23:18:38.307 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 29 2024-09-11T23:18:38.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:38.308+0000 7f4e0e4e6640 1 -- 172.21.15.26:0/3930636433 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f4df40a2e00 msgr2=0x7f4df40a52c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:38.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:38.308+0000 7f4e0e4e6640 1 --2- 172.21.15.26:0/3930636433 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f4df40a2e00 0x7f4df40a52c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f4e080febe0 tx=0x7f4df0005f90 comp rx=0 tx=0).stop 2024-09-11T23:18:38.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:38.308+0000 7f4e0e4e6640 1 -- 172.21.15.26:0/3930636433 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4e08102d40 msgr2=0x7f4e0819a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:38.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:38.308+0000 7f4e0e4e6640 1 --2- 172.21.15.26:0/3930636433 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4e08102d40 0x7f4e0819a670 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f4df8006010 tx=0x7f4df80042f0 comp rx=0 tx=0).stop 2024-09-11T23:18:38.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:38.308+0000 7f4e0e4e6640 1 -- 172.21.15.26:0/3930636433 shutdown_connections 2024-09-11T23:18:38.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:38.308+0000 7f4e0e4e6640 1 --2- 172.21.15.26:0/3930636433 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f4df40a2e00 0x7f4df40a52c0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:38.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:38.308+0000 7f4e0e4e6640 1 --2- 172.21.15.26:0/3930636433 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4e08103660 0x7f4e0819abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:38.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:38.308+0000 7f4e0e4e6640 1 --2- 172.21.15.26:0/3930636433 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4e0810c170 0x7f4e0819ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:38.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:38.308+0000 7f4e0e4e6640 1 --2- 172.21.15.26:0/3930636433 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4e08102d40 0x7f4e0819a670 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:38.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:38.308+0000 7f4e0e4e6640 1 -- 172.21.15.26:0/3930636433 >> 172.21.15.26:0/3930636433 conn(0x7f4e080fc7a0 msgr2=0x7f4e08101950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:18:38.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:38.308+0000 7f4e0e4e6640 1 -- 172.21.15.26:0/3930636433 shutdown_connections 2024-09-11T23:18:38.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:38.308+0000 7f4e0e4e6640 1 -- 172.21.15.26:0/3930636433 wait complete. 2024-09-11T23:18:39.001 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:18:39.027 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:38 smithi026 ceph-mon[31172]: from='client.? 172.21.15.26:0/3930636433' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:18:39.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:38 smithi076 ceph-mon[35296]: from='client.? 172.21.15.26:0/3930636433' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:18:39.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:38 smithi136 ceph-mon[35121]: from='client.? 172.21.15.26:0/3930636433' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:18:39.265 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:18:39.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:39.723+0000 7fde07428640 1 -- 172.21.15.26:0/491848997 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde00103780 msgr2=0x7fde00107bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:39.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:39.723+0000 7fde07428640 1 --2- 172.21.15.26:0/491848997 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde00103780 0x7fde00107bc0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fddf0009930 tx=0x7fddf002a420 comp rx=0 tx=0).stop 2024-09-11T23:18:39.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:39.723+0000 7fde07428640 1 -- 172.21.15.26:0/491848997 shutdown_connections 2024-09-11T23:18:39.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:39.723+0000 7fde07428640 1 --2- 172.21.15.26:0/491848997 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fde00108300 0x7fde0010ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:39.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:39.723+0000 7fde07428640 1 --2- 172.21.15.26:0/491848997 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde00103780 0x7fde00107bc0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:39.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:39.723+0000 7fde07428640 1 --2- 172.21.15.26:0/491848997 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fde00102dd0 0x7fde001031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:39.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:39.723+0000 7fde07428640 1 -- 172.21.15.26:0/491848997 >> 172.21.15.26:0/491848997 conn(0x7fde000fc7d0 msgr2=0x7fde000febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:18:39.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:39.723+0000 7fde07428640 1 -- 172.21.15.26:0/491848997 shutdown_connections 2024-09-11T23:18:39.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:39.723+0000 7fde07428640 1 -- 172.21.15.26:0/491848997 wait complete. 2024-09-11T23:18:39.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:39.724+0000 7fde07428640 1 Processor -- start 2024-09-11T23:18:39.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:39.724+0000 7fde07428640 1 -- start start 2024-09-11T23:18:39.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:39.724+0000 7fde07428640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fde00102dd0 0x7fde00198530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:39.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:39.724+0000 7fde07428640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fde00103780 0x7fde00198a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:39.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:39.724+0000 7fde07428640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde00108300 0x7fde0019ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:39.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:39.724+0000 7fde07428640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fde00112070 con 0x7fde00108300 2024-09-11T23:18:39.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:39.724+0000 7fde07428640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fde00111ef0 con 0x7fde00103780 2024-09-11T23:18:39.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:39.724+0000 7fde07428640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fde001121f0 con 0x7fde00102dd0 2024-09-11T23:18:39.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:39.724+0000 7fde059c5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde00108300 0x7fde0019ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:39.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:39.724+0000 7fde051c4640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fde00102dd0 0x7fde00198530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:39.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:39.724+0000 7fde059c5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde00108300 0x7fde0019ce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60660/0 (socket says 172.21.15.26:60660) 2024-09-11T23:18:39.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:39.724+0000 7fde059c5640 1 -- 172.21.15.26:0/105314819 learned_addr learned my addr 172.21.15.26:0/105314819 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:18:39.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:39.724+0000 7fde049c3640 1 --2- 172.21.15.26:0/105314819 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fde00103780 0x7fde00198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:39.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:39.724+0000 7fde051c4640 1 -- 172.21.15.26:0/105314819 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fde00103780 msgr2=0x7fde00198a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:39.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:39.724+0000 7fde051c4640 1 --2- 172.21.15.26:0/105314819 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fde00103780 0x7fde00198a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:39.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:39.724+0000 7fde051c4640 1 -- 172.21.15.26:0/105314819 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde00108300 msgr2=0x7fde0019ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:39.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:39.724+0000 7fde051c4640 1 --2- 172.21.15.26:0/105314819 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde00108300 0x7fde0019ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:39.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:39.724+0000 7fde051c4640 1 -- 172.21.15.26:0/105314819 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde0019d4e0 con 0x7fde00102dd0 2024-09-11T23:18:39.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:39.724+0000 7fde059c5640 1 --2- 172.21.15.26:0/105314819 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde00108300 0x7fde0019ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:18:39.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:39.725+0000 7fde051c4640 1 --2- 172.21.15.26:0/105314819 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fde00102dd0 0x7fde00198530 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fddfc00c970 tx=0x7fddfc00ce30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:18:39.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:39.725+0000 7fddf67fc640 1 -- 172.21.15.26:0/105314819 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddfc007bb0 con 0x7fde00102dd0 2024-09-11T23:18:39.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:39.725+0000 7fde07428640 1 -- 172.21.15.26:0/105314819 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde0019d7d0 con 0x7fde00102dd0 2024-09-11T23:18:39.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:39.725+0000 7fddf67fc640 1 -- 172.21.15.26:0/105314819 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fddfc007d50 con 0x7fde00102dd0 2024-09-11T23:18:39.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:39.725+0000 7fddf67fc640 1 -- 172.21.15.26:0/105314819 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddfc018890 con 0x7fde00102dd0 2024-09-11T23:18:39.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:39.725+0000 7fde07428640 1 -- 172.21.15.26:0/105314819 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde001a50b0 con 0x7fde00102dd0 2024-09-11T23:18:39.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:39.726+0000 7fde07428640 1 -- 172.21.15.26:0/105314819 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde001031b0 con 0x7fde00102dd0 2024-09-11T23:18:39.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:39.728+0000 7fddf67fc640 1 -- 172.21.15.26:0/105314819 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 24) ==== 138009+0+0 (secure 0 0 0) 0x7fddfc018a30 con 0x7fde00102dd0 2024-09-11T23:18:39.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:39.729+0000 7fddf67fc640 1 --2- 172.21.15.26:0/105314819 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fddec0a2e00 0x7fddec0a52c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:39.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:39.729+0000 7fde049c3640 1 --2- 172.21.15.26:0/105314819 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fddec0a2e00 0x7fddec0a52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:39.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:39.729+0000 7fddf67fc640 1 -- 172.21.15.26:0/105314819 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fddfc0cf8a0 con 0x7fde00102dd0 2024-09-11T23:18:39.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:39.729+0000 7fde049c3640 1 --2- 172.21.15.26:0/105314819 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fddec0a2e00 0x7fddec0a52c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fddf0005940 tx=0x7fddf00058d0 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:18:39.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:39.732+0000 7fddf67fc640 1 -- 172.21.15.26:0/105314819 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fddfc014030 con 0x7fde00102dd0 2024-09-11T23:18:39.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:39.947+0000 7fde07428640 1 -- 172.21.15.26:0/105314819 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fde00199890 con 0x7fde00102dd0 2024-09-11T23:18:39.947 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:39 smithi026 ceph-mon[31172]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 76 KiB/s wr, 54 op/s 2024-09-11T23:18:39.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:39.948+0000 7fddf67fc640 1 -- 172.21.15.26:0/105314819 <== mon.1 v2:172.21.15.136:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11971 (secure 0 0 0) 0x7fde00199890 con 0x7fde00102dd0 2024-09-11T23:18:39.948 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-11T23:18:39.948 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":29,"btime":"2024-09-11T23:18:21:019324+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14499,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/367659515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":367659515},{"type":"v1","addr":"172.21.15.26:6835","nonce":367659515}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14511,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/2041065299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":2041065299},{"type":"v1","addr":"172.21.15.26:6837","nonce":2041065299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:16:26.544203+0000","modified":"2024-09-11T23:18:20.969151+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24485,"mds_1":24463,"mds_2":24458,"mds_3":24448,"mds_4":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14550":{"gid":14550,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6839/2233068305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":2233068305},{"type":"v1","addr":"172.21.15.26:6839","nonce":2233068305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14562":{"gid":14562,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6841/2984982336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":2984982336},{"type":"v1","addr":"172.21.15.26:6841","nonce":2984982336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6833/2019841455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":2019841455},{"type":"v1","addr":"172.21.15.136:6833","nonce":2019841455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.76:6833/3438147050","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3438147050},{"type":"v1","addr":"172.21.15.76:6833","nonce":3438147050}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6835/3617665975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3617665975},{"type":"v1","addr":"172.21.15.136:6835","nonce":3617665975}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":12,"addr":"172.21.15.76:6835/393308499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":393308499},{"type":"v1","addr":"172.21.15.76:6835","nonce":393308499}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":10,"addr":"172.21.15.136:6837/1633082387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":1633082387},{"type":"v1","addr":"172.21.15.136:6837","nonce":1633082387}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.76:6837/2078978748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":2078978748},{"type":"v1","addr":"172.21.15.76:6837","nonce":2078978748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.136:6839/3211398630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":3211398630},{"type":"v1","addr":"172.21.15.136:6839","nonce":3211398630}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.76:6839/2077637090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6838","nonce":2077637090},{"type":"v1","addr":"172.21.15.76:6839","nonce":2077637090}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24485,"qdb_cluster":[24443,24448,24458,24463,24485]},"id":1}]} 2024-09-11T23:18:39.949 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 29 2024-09-11T23:18:39.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:39.950+0000 7fde07428640 1 -- 172.21.15.26:0/105314819 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fddec0a2e00 msgr2=0x7fddec0a52c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:39.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:39.950+0000 7fde07428640 1 --2- 172.21.15.26:0/105314819 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fddec0a2e00 0x7fddec0a52c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fddf0005940 tx=0x7fddf00058d0 comp rx=0 tx=0).stop 2024-09-11T23:18:39.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:39.951+0000 7fde07428640 1 -- 172.21.15.26:0/105314819 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fde00102dd0 msgr2=0x7fde00198530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:39.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:39.951+0000 7fde07428640 1 --2- 172.21.15.26:0/105314819 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fde00102dd0 0x7fde00198530 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fddfc00c970 tx=0x7fddfc00ce30 comp rx=0 tx=0).stop 2024-09-11T23:18:39.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:39.951+0000 7fde07428640 1 -- 172.21.15.26:0/105314819 shutdown_connections 2024-09-11T23:18:39.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:39.951+0000 7fde07428640 1 --2- 172.21.15.26:0/105314819 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fddec0a2e00 0x7fddec0a52c0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:39.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:39.951+0000 7fde07428640 1 --2- 172.21.15.26:0/105314819 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fde00103780 0x7fde00198a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:39.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:39.951+0000 7fde07428640 1 --2- 172.21.15.26:0/105314819 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde00108300 0x7fde0019ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:39.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:39.951+0000 7fde07428640 1 --2- 172.21.15.26:0/105314819 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fde00102dd0 0x7fde00198530 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:39.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:39.951+0000 7fde07428640 1 -- 172.21.15.26:0/105314819 >> 172.21.15.26:0/105314819 conn(0x7fde000fc7d0 msgr2=0x7fde00105d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:18:39.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:39.951+0000 7fde07428640 1 -- 172.21.15.26:0/105314819 shutdown_connections 2024-09-11T23:18:39.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:39.951+0000 7fde07428640 1 -- 172.21.15.26:0/105314819 wait complete. 2024-09-11T23:18:40.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:39 smithi076 ceph-mon[35296]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 76 KiB/s wr, 54 op/s 2024-09-11T23:18:40.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:39 smithi136 ceph-mon[35121]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 76 KiB/s wr, 54 op/s 2024-09-11T23:18:40.669 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-11T23:18:40.972 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:18:41.000 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:40 smithi026 ceph-mon[31172]: from='client.? 172.21.15.26:0/105314819' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:18:41.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:40 smithi136 ceph-mon[35121]: from='client.? 172.21.15.26:0/105314819' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:18:41.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:40 smithi076 ceph-mon[35296]: from='client.? 172.21.15.26:0/105314819' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:18:41.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:41.398+0000 7f22c0753640 1 -- 172.21.15.26:0/3706109013 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f22b8100a80 msgr2=0x7f22b8100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:41.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:41.398+0000 7f22c0753640 1 --2- 172.21.15.26:0/3706109013 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f22b8100a80 0x7f22b8100ee0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f22ac009930 tx=0x7f22ac02a420 comp rx=0 tx=0).stop 2024-09-11T23:18:41.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:41.398+0000 7f22c0753640 1 -- 172.21.15.26:0/3706109013 shutdown_connections 2024-09-11T23:18:41.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:41.398+0000 7f22c0753640 1 --2- 172.21.15.26:0/3706109013 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f22b8101420 0x7f22b810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:41.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:41.398+0000 7f22c0753640 1 --2- 172.21.15.26:0/3706109013 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f22b8100a80 0x7f22b8100ee0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:41.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:41.398+0000 7f22c0753640 1 --2- 172.21.15.26:0/3706109013 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f22b8106a80 0x7f22b8106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:41.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:41.398+0000 7f22c0753640 1 -- 172.21.15.26:0/3706109013 >> 172.21.15.26:0/3706109013 conn(0x7f22b80fc7a0 msgr2=0x7f22b80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:18:41.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:41.398+0000 7f22c0753640 1 -- 172.21.15.26:0/3706109013 shutdown_connections 2024-09-11T23:18:41.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:41.398+0000 7f22c0753640 1 -- 172.21.15.26:0/3706109013 wait complete. 2024-09-11T23:18:41.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:41.399+0000 7f22c0753640 1 Processor -- start 2024-09-11T23:18:41.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:41.399+0000 7f22c0753640 1 -- start start 2024-09-11T23:18:41.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:41.399+0000 7f22c0753640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f22b8100a80 0x7f22b819a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:41.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:41.399+0000 7f22c0753640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f22b8101420 0x7f22b819aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:41.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:41.399+0000 7f22c0753640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f22b8106a80 0x7f22b819ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:41.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:41.399+0000 7f22c0753640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f22b8110e80 con 0x7f22b8101420 2024-09-11T23:18:41.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:41.399+0000 7f22c0753640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f22b8110d00 con 0x7f22b8106a80 2024-09-11T23:18:41.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:41.399+0000 7f22c0753640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f22b8111000 con 0x7f22b8100a80 2024-09-11T23:18:41.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:41.399+0000 7f22bdcee640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f22b8101420 0x7f22b819aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:41.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:41.400+0000 7f22bdcee640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f22b8101420 0x7f22b819aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46290/0 (socket says 172.21.15.26:46290) 2024-09-11T23:18:41.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:41.400+0000 7f22bdcee640 1 -- 172.21.15.26:0/3272124602 learned_addr learned my addr 172.21.15.26:0/3272124602 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:18:41.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:41.400+0000 7f22be4ef640 1 --2- 172.21.15.26:0/3272124602 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f22b8100a80 0x7f22b819a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:41.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:41.400+0000 7f22becf0640 1 --2- 172.21.15.26:0/3272124602 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f22b8106a80 0x7f22b819ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:41.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:41.400+0000 7f22be4ef640 1 -- 172.21.15.26:0/3272124602 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f22b8106a80 msgr2=0x7f22b819ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:41.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:41.400+0000 7f22be4ef640 1 --2- 172.21.15.26:0/3272124602 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f22b8106a80 0x7f22b819ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:41.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:41.400+0000 7f22be4ef640 1 -- 172.21.15.26:0/3272124602 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f22b8101420 msgr2=0x7f22b819aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:41.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:41.400+0000 7f22be4ef640 1 --2- 172.21.15.26:0/3272124602 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f22b8101420 0x7f22b819aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:41.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:41.400+0000 7f22be4ef640 1 -- 172.21.15.26:0/3272124602 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22b819f610 con 0x7f22b8100a80 2024-09-11T23:18:41.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:41.400+0000 7f22bdcee640 1 --2- 172.21.15.26:0/3272124602 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f22b8101420 0x7f22b819aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:18:41.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:41.400+0000 7f22be4ef640 1 --2- 172.21.15.26:0/3272124602 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f22b8100a80 0x7f22b819a660 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f22b000d720 tx=0x7f22b000dbe0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:18:41.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:41.400+0000 7f22b77fe640 1 -- 172.21.15.26:0/3272124602 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22b0017070 con 0x7f22b8100a80 2024-09-11T23:18:41.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:41.400+0000 7f22c0753640 1 -- 172.21.15.26:0/3272124602 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22b819f900 con 0x7f22b8100a80 2024-09-11T23:18:41.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:41.401+0000 7f22b77fe640 1 -- 172.21.15.26:0/3272124602 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f22b000de30 con 0x7f22b8100a80 2024-09-11T23:18:41.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:41.401+0000 7f22b77fe640 1 -- 172.21.15.26:0/3272124602 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22b00047a0 con 0x7f22b8100a80 2024-09-11T23:18:41.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:41.401+0000 7f22c0753640 1 -- 172.21.15.26:0/3272124602 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22b81a71e0 con 0x7f22b8100a80 2024-09-11T23:18:41.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:41.401+0000 7f22c0753640 1 -- 172.21.15.26:0/3272124602 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22b8100ee0 con 0x7f22b8100a80 2024-09-11T23:18:41.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:41.403+0000 7f22b77fe640 1 -- 172.21.15.26:0/3272124602 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 24) ==== 138009+0+0 (secure 0 0 0) 0x7f22b00099d0 con 0x7f22b8100a80 2024-09-11T23:18:41.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:41.404+0000 7f22b77fe640 1 --2- 172.21.15.26:0/3272124602 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f22a00a2d80 0x7f22a00a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:41.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:41.404+0000 7f22b77fe640 1 -- 172.21.15.26:0/3272124602 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f22b00cf5f0 con 0x7f22b8100a80 2024-09-11T23:18:41.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:41.404+0000 7f22bdcee640 1 --2- 172.21.15.26:0/3272124602 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f22a00a2d80 0x7f22a00a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:41.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:41.404+0000 7f22bdcee640 1 --2- 172.21.15.26:0/3272124602 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f22a00a2d80 0x7f22a00a5240 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f22b80fed10 tx=0x7f22ac009560 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:18:41.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:41.407+0000 7f22b77fe640 1 -- 172.21.15.26:0/3272124602 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f22b0020020 con 0x7f22b8100a80 2024-09-11T23:18:41.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:41.583+0000 7f22c0753640 1 -- 172.21.15.26:0/3272124602 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f22b81a7440 con 0x7f22b8100a80 2024-09-11T23:18:41.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:41.585+0000 7f22b77fe640 1 -- 172.21.15.26:0/3272124602 <== mon.1 v2:172.21.15.136:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+23109 (secure 0 0 0) 0x7f22b009c050 con 0x7f22b8100a80 2024-09-11T23:18:41.585 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-11T23:18:41.586 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":86,"fsid":"af7e6fc8-7092-11ef-bcea-c7b262605968","created":"2024-09-11T23:08:40.486438+0000","modified":"2024-09-11T23:18:22.177962+0000","last_up_change":"2024-09-11T23:15:35.179446+0000","last_in_change":"2024-09-11T23:15:11.064627+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T23:11:50.494160+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-11T23:16:22.503928+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-11T23:16:24.876554+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a4d62fcf-c31e-41b1-932c-3d87ff640531","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":2014857905},{"type":"v1","addr":"172.21.15.26:6803","nonce":2014857905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":2014857905},{"type":"v1","addr":"172.21.15.26:6805","nonce":2014857905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":2014857905},{"type":"v1","addr":"172.21.15.26:6809","nonce":2014857905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":2014857905},{"type":"v1","addr":"172.21.15.26:6807","nonce":2014857905}]},"public_addr":"172.21.15.26:6803/2014857905","cluster_addr":"172.21.15.26:6805/2014857905","heartbeat_back_addr":"172.21.15.26:6809/2014857905","heartbeat_front_addr":"172.21.15.26:6807/2014857905","state":["exists","up"]},{"osd":1,"uuid":"e0d69112-3c0a-4fc5-8783-e41b4081e0c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":333383087},{"type":"v1","addr":"172.21.15.76:6801","nonce":333383087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":333383087},{"type":"v1","addr":"172.21.15.76:6803","nonce":333383087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":333383087},{"type":"v1","addr":"172.21.15.76:6807","nonce":333383087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":333383087},{"type":"v1","addr":"172.21.15.76:6805","nonce":333383087}]},"public_addr":"172.21.15.76:6801/333383087","cluster_addr":"172.21.15.76:6803/333383087","heartbeat_back_addr":"172.21.15.76:6807/333383087","heartbeat_front_addr":"172.21.15.76:6805/333383087","state":["exists","up"]},{"osd":2,"uuid":"d578429e-bf23-4a1e-890f-7172955c0eb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":1271538911},{"type":"v1","addr":"172.21.15.136:6801","nonce":1271538911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":1271538911},{"type":"v1","addr":"172.21.15.136:6803","nonce":1271538911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":1271538911},{"type":"v1","addr":"172.21.15.136:6807","nonce":1271538911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":1271538911},{"type":"v1","addr":"172.21.15.136:6805","nonce":1271538911}]},"public_addr":"172.21.15.136:6801/1271538911","cluster_addr":"172.21.15.136:6803/1271538911","heartbeat_back_addr":"172.21.15.136:6807/1271538911","heartbeat_front_addr":"172.21.15.136:6805/1271538911","state":["exists","up"]},{"osd":3,"uuid":"3e34623f-0ea5-4b10-b8a9-f8756c35d512","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":1740304151},{"type":"v1","addr":"172.21.15.26:6811","nonce":1740304151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":1740304151},{"type":"v1","addr":"172.21.15.26:6813","nonce":1740304151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1740304151},{"type":"v1","addr":"172.21.15.26:6817","nonce":1740304151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":1740304151},{"type":"v1","addr":"172.21.15.26:6815","nonce":1740304151}]},"public_addr":"172.21.15.26:6811/1740304151","cluster_addr":"172.21.15.26:6813/1740304151","heartbeat_back_addr":"172.21.15.26:6817/1740304151","heartbeat_front_addr":"172.21.15.26:6815/1740304151","state":["exists","up"]},{"osd":4,"uuid":"77607929-f9ed-49a9-abb5-48cd88eec4ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":2702262182},{"type":"v1","addr":"172.21.15.76:6809","nonce":2702262182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":2702262182},{"type":"v1","addr":"172.21.15.76:6811","nonce":2702262182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":2702262182},{"type":"v1","addr":"172.21.15.76:6815","nonce":2702262182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":2702262182},{"type":"v1","addr":"172.21.15.76:6813","nonce":2702262182}]},"public_addr":"172.21.15.76:6809/2702262182","cluster_addr":"172.21.15.76:6811/2702262182","heartbeat_back_addr":"172.21.15.76:6815/2702262182","heartbeat_front_addr":"172.21.15.76:6813/2702262182","state":["exists","up"]},{"osd":5,"uuid":"29e20e37-09a0-4924-8aaa-17939763a7c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":1749229277},{"type":"v1","addr":"172.21.15.136:6809","nonce":1749229277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":1749229277},{"type":"v1","addr":"172.21.15.136:6811","nonce":1749229277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":1749229277},{"type":"v1","addr":"172.21.15.136:6815","nonce":1749229277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":1749229277},{"type":"v1","addr":"172.21.15.136:6813","nonce":1749229277}]},"public_addr":"172.21.15.136:6809/1749229277","cluster_addr":"172.21.15.136:6811/1749229277","heartbeat_back_addr":"172.21.15.136:6815/1749229277","heartbeat_front_addr":"172.21.15.136:6813/1749229277","state":["exists","up"]},{"osd":6,"uuid":"e0ddd1c0-1c46-41e6-b2f4-396bdcd2bf8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1928245624},{"type":"v1","addr":"172.21.15.26:6819","nonce":1928245624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1928245624},{"type":"v1","addr":"172.21.15.26:6821","nonce":1928245624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":1928245624},{"type":"v1","addr":"172.21.15.26:6825","nonce":1928245624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1928245624},{"type":"v1","addr":"172.21.15.26:6823","nonce":1928245624}]},"public_addr":"172.21.15.26:6819/1928245624","cluster_addr":"172.21.15.26:6821/1928245624","heartbeat_back_addr":"172.21.15.26:6825/1928245624","heartbeat_front_addr":"172.21.15.26:6823/1928245624","state":["exists","up"]},{"osd":7,"uuid":"fe031f5c-d915-48fb-91a6-c76f252467fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":1409789455},{"type":"v1","addr":"172.21.15.76:6817","nonce":1409789455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":1409789455},{"type":"v1","addr":"172.21.15.76:6819","nonce":1409789455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":1409789455},{"type":"v1","addr":"172.21.15.76:6823","nonce":1409789455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":1409789455},{"type":"v1","addr":"172.21.15.76:6821","nonce":1409789455}]},"public_addr":"172.21.15.76:6817/1409789455","cluster_addr":"172.21.15.76:6819/1409789455","heartbeat_back_addr":"172.21.15.76:6823/1409789455","heartbeat_front_addr":"172.21.15.76:6821/1409789455","state":["exists","up"]},{"osd":8,"uuid":"18851bde-9e6c-4e05-b4bf-ac5c3073eee7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":543619079},{"type":"v1","addr":"172.21.15.136:6817","nonce":543619079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":543619079},{"type":"v1","addr":"172.21.15.136:6819","nonce":543619079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":543619079},{"type":"v1","addr":"172.21.15.136:6823","nonce":543619079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":543619079},{"type":"v1","addr":"172.21.15.136:6821","nonce":543619079}]},"public_addr":"172.21.15.136:6817/543619079","cluster_addr":"172.21.15.136:6819/543619079","heartbeat_back_addr":"172.21.15.136:6823/543619079","heartbeat_front_addr":"172.21.15.136:6821/543619079","state":["exists","up"]},{"osd":9,"uuid":"52a65eab-8446-47ad-bfe7-5133f7e3eb98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":2201297064},{"type":"v1","addr":"172.21.15.26:6827","nonce":2201297064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":2201297064},{"type":"v1","addr":"172.21.15.26:6829","nonce":2201297064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":2201297064},{"type":"v1","addr":"172.21.15.26:6833","nonce":2201297064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":2201297064},{"type":"v1","addr":"172.21.15.26:6831","nonce":2201297064}]},"public_addr":"172.21.15.26:6827/2201297064","cluster_addr":"172.21.15.26:6829/2201297064","heartbeat_back_addr":"172.21.15.26:6833/2201297064","heartbeat_front_addr":"172.21.15.26:6831/2201297064","state":["exists","up"]},{"osd":10,"uuid":"b758d4bb-da62-4c5f-a992-bb6a145d35d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":3184128096},{"type":"v1","addr":"172.21.15.76:6825","nonce":3184128096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":3184128096},{"type":"v1","addr":"172.21.15.76:6827","nonce":3184128096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":3184128096},{"type":"v1","addr":"172.21.15.76:6831","nonce":3184128096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":3184128096},{"type":"v1","addr":"172.21.15.76:6829","nonce":3184128096}]},"public_addr":"172.21.15.76:6825/3184128096","cluster_addr":"172.21.15.76:6827/3184128096","heartbeat_back_addr":"172.21.15.76:6831/3184128096","heartbeat_front_addr":"172.21.15.76:6829/3184128096","state":["exists","up"]},{"osd":11,"uuid":"8fddda46-e22b-4d0c-ac48-e038fb29dfec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":1960689685},{"type":"v1","addr":"172.21.15.136:6825","nonce":1960689685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":1960689685},{"type":"v1","addr":"172.21.15.136:6827","nonce":1960689685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":1960689685},{"type":"v1","addr":"172.21.15.136:6831","nonce":1960689685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":1960689685},{"type":"v1","addr":"172.21.15.136:6829","nonce":1960689685}]},"public_addr":"172.21.15.136:6825/1960689685","cluster_addr":"172.21.15.136:6827/1960689685","heartbeat_back_addr":"172.21.15.136:6831/1960689685","heartbeat_front_addr":"172.21.15.136:6829/1960689685","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:10:56.656177+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:11:22.234031+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:11:46.464102+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:12:12.353967+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:12:37.836200+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:13:01.728839+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:13:27.527230+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:13:51.981393+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:14:17.035464+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:14:42.805029+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:15:08.208999+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:15:32.800844+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.18","mappings":[{"from":3,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":6}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.26:0/2626831531":"2024-09-12T23:18:22.177934+0000","172.21.15.26:0/3861103130":"2024-09-12T23:09:09.781665+0000","172.21.15.26:0/17142658":"2024-09-12T23:09:09.781665+0000","172.21.15.26:0/2817092653":"2024-09-12T23:09:09.781665+0000","172.21.15.26:6801/110833859":"2024-09-12T23:18:22.177934+0000","172.21.15.26:0/2107281413":"2024-09-12T23:09:33.983310+0000","172.21.15.26:0/883212708":"2024-09-12T23:09:33.983310+0000","172.21.15.26:6800/2673575521":"2024-09-12T23:09:09.781665+0000","172.21.15.26:0/2559066894":"2024-09-12T23:18:22.177934+0000","172.21.15.26:6801/2673575521":"2024-09-12T23:09:09.781665+0000","172.21.15.26:0/3597996003":"2024-09-12T23:09:33.983310+0000","172.21.15.26:6800/677148151":"2024-09-12T23:09:33.983310+0000","172.21.15.26:6801/677148151":"2024-09-12T23:09:33.983310+0000","172.21.15.26:0/1584355185":"2024-09-12T23:18:22.177934+0000","172.21.15.26:6800/110833859":"2024-09-12T23:18:22.177934+0000","172.21.15.26:0/1256290054":"2024-09-12T23:18:22.177934+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T23:18:41.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:41.587+0000 7f22c0753640 1 -- 172.21.15.26:0/3272124602 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f22a00a2d80 msgr2=0x7f22a00a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:41.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:41.587+0000 7f22c0753640 1 --2- 172.21.15.26:0/3272124602 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f22a00a2d80 0x7f22a00a5240 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f22b80fed10 tx=0x7f22ac009560 comp rx=0 tx=0).stop 2024-09-11T23:18:41.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:41.587+0000 7f22c0753640 1 -- 172.21.15.26:0/3272124602 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f22b8100a80 msgr2=0x7f22b819a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:41.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:41.587+0000 7f22c0753640 1 --2- 172.21.15.26:0/3272124602 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f22b8100a80 0x7f22b819a660 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f22b000d720 tx=0x7f22b000dbe0 comp rx=0 tx=0).stop 2024-09-11T23:18:41.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:41.587+0000 7f22c0753640 1 -- 172.21.15.26:0/3272124602 shutdown_connections 2024-09-11T23:18:41.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:41.587+0000 7f22c0753640 1 --2- 172.21.15.26:0/3272124602 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f22a00a2d80 0x7f22a00a5240 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:41.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:41.587+0000 7f22c0753640 1 --2- 172.21.15.26:0/3272124602 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f22b8106a80 0x7f22b819ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:41.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:41.587+0000 7f22c0753640 1 --2- 172.21.15.26:0/3272124602 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f22b8101420 0x7f22b819aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:41.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:41.587+0000 7f22c0753640 1 --2- 172.21.15.26:0/3272124602 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f22b8100a80 0x7f22b819a660 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:41.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:41.587+0000 7f22c0753640 1 -- 172.21.15.26:0/3272124602 >> 172.21.15.26:0/3272124602 conn(0x7f22b80fc7a0 msgr2=0x7f22b806a7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:18:41.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:41.587+0000 7f22c0753640 1 -- 172.21.15.26:0/3272124602 shutdown_connections 2024-09-11T23:18:41.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:41.587+0000 7f22c0753640 1 -- 172.21.15.26:0/3272124602 wait complete. 2024-09-11T23:18:42.241 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:41 smithi136 ceph-mon[35121]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 76 KiB/s wr, 43 op/s 2024-09-11T23:18:42.242 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:41 smithi136 ceph-mon[35121]: scrub summary: idle 2024-09-11T23:18:42.242 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:41 smithi136 ceph-mon[35121]: from='client.? 172.21.15.26:0/3272124602' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T23:18:42.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:41 smithi026 ceph-mon[31172]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 76 KiB/s wr, 43 op/s 2024-09-11T23:18:42.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:41 smithi026 ceph-mon[31172]: scrub summary: idle 2024-09-11T23:18:42.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:41 smithi026 ceph-mon[31172]: from='client.? 172.21.15.26:0/3272124602' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T23:18:42.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:41 smithi076 ceph-mon[35296]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 76 KiB/s wr, 43 op/s 2024-09-11T23:18:42.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:41 smithi076 ceph-mon[35296]: scrub summary: idle 2024-09-11T23:18:42.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:41 smithi076 ceph-mon[35296]: from='client.? 172.21.15.26:0/3272124602' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T23:18:42.333 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph daemon mon.a --format=json config get run_dir 2024-09-11T23:18:42.590 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:18:43.749 INFO:teuthology.orchestra.run.smithi026.stdout:{"run_dir":"/var/run/ceph"} 2024-09-11T23:18:43.750 DEBUG:tasks.cephfs.filesystem:_json_asok output { "run_dir": "/var/run/ceph" } 2024-09-11T23:18:43.750 INFO:tasks.quiescer.fs.[cephfs]:Initializing Quiescer with seed 481 2024-09-11T23:18:43.750 DEBUG:tasks.quiescer:Yielding 2024-09-11T23:18:43.751 INFO:teuthology.run_tasks:Running task workunit... 2024-09-11T23:18:43.767 INFO:tasks.workunit:Pulling workunits from ref 086c737fa75c22d521a3932c59b31c5e1d1c9ef4 2024-09-11T23:18:43.768 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-09-11T23:18:43.768 INFO:tasks.workunit:timeout=3h 2024-09-11T23:18:43.768 INFO:tasks.workunit:cleanup=True 2024-09-11T23:18:43.768 DEBUG:teuthology.orchestra.run.smithi026:> stat -- /home/ubuntu/cephtest/mnt.0 2024-09-11T23:18:43.769 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:18:43.786 INFO:teuthology.orchestra.run.smithi026.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-09-11T23:18:43.786 INFO:teuthology.orchestra.run.smithi026.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-09-11T23:18:43.787 INFO:teuthology.orchestra.run.smithi026.stdout:Device: 78h/120d Inode: 1099511628283 Links: 2 2024-09-11T23:18:43.787 INFO:teuthology.orchestra.run.smithi026.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T23:18:43.787 INFO:teuthology.orchestra.run.smithi026.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-09-11T23:18:43.787 INFO:teuthology.orchestra.run.smithi026.stdout:Access: 2024-09-11 23:16:44.755423662 +0000 2024-09-11T23:18:43.787 INFO:teuthology.orchestra.run.smithi026.stdout:Modify: 2024-09-11 23:16:44.755423662 +0000 2024-09-11T23:18:43.787 INFO:teuthology.orchestra.run.smithi026.stdout:Change: 2024-09-11 23:18:20.223825685 +0000 2024-09-11T23:18:43.787 INFO:teuthology.orchestra.run.smithi026.stdout: Birth: 2024-09-11 23:16:44.755423662 +0000 2024-09-11T23:18:43.787 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-09-11T23:18:43.787 DEBUG:teuthology.orchestra.run.smithi026:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-09-11T23:18:43.861 DEBUG:teuthology.orchestra.run.smithi026:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 086c737fa75c22d521a3932c59b31c5e1d1c9ef4 2024-09-11T23:18:43.917 INFO:tasks.workunit.client.0.smithi026.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-09-11T23:18:44.000 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:43 smithi026 ceph-mon[31172]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 76 KiB/s wr, 40 op/s 2024-09-11T23:18:44.035 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:18:44.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:43 smithi136 ceph-mon[35121]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 76 KiB/s wr, 40 op/s 2024-09-11T23:18:44.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:43 smithi076 ceph-mon[35296]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 76 KiB/s wr, 40 op/s 2024-09-11T23:18:44.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:44.524+0000 7f45294d4640 1 -- 172.21.15.26:0/3044372929 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4524102ef0 msgr2=0x7f452410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:44.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:44.524+0000 7f45294d4640 1 --2- 172.21.15.26:0/3044372929 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4524102ef0 0x7f452410da40 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f451c009900 tx=0x7f451c02a480 comp rx=0 tx=0).stop 2024-09-11T23:18:44.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:44.524+0000 7f45294d4640 1 -- 172.21.15.26:0/3044372929 shutdown_connections 2024-09-11T23:18:44.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:44.524+0000 7f45294d4640 1 --2- 172.21.15.26:0/3044372929 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4524102570 0x7f45241029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:44.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:44.524+0000 7f45294d4640 1 --2- 172.21.15.26:0/3044372929 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4524102ef0 0x7f452410da40 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:44.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:44.524+0000 7f45294d4640 1 --2- 172.21.15.26:0/3044372929 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4524100e20 0x7f4524101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:44.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:44.524+0000 7f45294d4640 1 -- 172.21.15.26:0/3044372929 >> 172.21.15.26:0/3044372929 conn(0x7f45240fc7d0 msgr2=0x7f45240febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:18:44.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:44.524+0000 7f45294d4640 1 -- 172.21.15.26:0/3044372929 shutdown_connections 2024-09-11T23:18:44.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:44.524+0000 7f45294d4640 1 -- 172.21.15.26:0/3044372929 wait complete. 2024-09-11T23:18:44.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:44.525+0000 7f45294d4640 1 Processor -- start 2024-09-11T23:18:44.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:44.525+0000 7f45294d4640 1 -- start start 2024-09-11T23:18:44.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:44.526+0000 7f45294d4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4524100e20 0x7f4524199330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:44.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:44.526+0000 7f45294d4640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4524102570 0x7f4524199870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:44.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:44.526+0000 7f45294d4640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4524102ef0 0x7f45241a43b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:44.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:44.526+0000 7f45294d4640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f4524110f80 con 0x7f4524100e20 2024-09-11T23:18:44.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:44.526+0000 7f45294d4640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f4524110e00 con 0x7f4524102ef0 2024-09-11T23:18:44.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:44.526+0000 7f45294d4640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f4524111100 con 0x7f4524102570 2024-09-11T23:18:44.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:44.526+0000 7f4522ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4524100e20 0x7f4524199330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:44.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:44.526+0000 7f4522ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4524100e20 0x7f4524199330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46308/0 (socket says 172.21.15.26:46308) 2024-09-11T23:18:44.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:44.526+0000 7f4522ffd640 1 -- 172.21.15.26:0/3360387100 learned_addr learned my addr 172.21.15.26:0/3360387100 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:18:44.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:44.526+0000 7f45227fc640 1 --2- 172.21.15.26:0/3360387100 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4524102570 0x7f4524199870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:44.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:44.526+0000 7f45237fe640 1 --2- 172.21.15.26:0/3360387100 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4524102ef0 0x7f45241a43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:44.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:44.526+0000 7f4522ffd640 1 -- 172.21.15.26:0/3360387100 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4524102570 msgr2=0x7f4524199870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:44.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:44.526+0000 7f4522ffd640 1 --2- 172.21.15.26:0/3360387100 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4524102570 0x7f4524199870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:44.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:44.526+0000 7f4522ffd640 1 -- 172.21.15.26:0/3360387100 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4524102ef0 msgr2=0x7f45241a43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:44.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:44.526+0000 7f4522ffd640 1 --2- 172.21.15.26:0/3360387100 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4524102ef0 0x7f45241a43b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:44.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:44.526+0000 7f4522ffd640 1 -- 172.21.15.26:0/3360387100 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45241a48f0 con 0x7f4524100e20 2024-09-11T23:18:44.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:44.526+0000 7f4522ffd640 1 --2- 172.21.15.26:0/3360387100 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4524100e20 0x7f4524199330 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f451400d770 tx=0x7f451400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:18:44.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:44.526+0000 7f450bfff640 1 -- 172.21.15.26:0/3360387100 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4514017070 con 0x7f4524100e20 2024-09-11T23:18:44.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:44.527+0000 7f450bfff640 1 -- 172.21.15.26:0/3360387100 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4514005500 con 0x7f4524100e20 2024-09-11T23:18:44.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:44.527+0000 7f45294d4640 1 -- 172.21.15.26:0/3360387100 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45241a4ac0 con 0x7f4524100e20 2024-09-11T23:18:44.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:44.527+0000 7f450bfff640 1 -- 172.21.15.26:0/3360387100 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4514004e00 con 0x7f4524100e20 2024-09-11T23:18:44.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:44.527+0000 7f45294d4640 1 -- 172.21.15.26:0/3360387100 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45241a4fb0 con 0x7f4524100e20 2024-09-11T23:18:44.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:44.528+0000 7f45294d4640 1 -- 172.21.15.26:0/3360387100 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4524101200 con 0x7f4524100e20 2024-09-11T23:18:44.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:44.529+0000 7f450bfff640 1 -- 172.21.15.26:0/3360387100 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 24) ==== 138009+0+0 (secure 0 0 0) 0x7f4514004fa0 con 0x7f4524100e20 2024-09-11T23:18:44.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:44.529+0000 7f450bfff640 1 --2- 172.21.15.26:0/3360387100 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f450c0a2d80 0x7f450c0a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:44.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:44.530+0000 7f450bfff640 1 -- 172.21.15.26:0/3360387100 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f45140cf060 con 0x7f4524100e20 2024-09-11T23:18:44.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:44.530+0000 7f45227fc640 1 --2- 172.21.15.26:0/3360387100 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f450c0a2d80 0x7f450c0a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:44.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:44.530+0000 7f45227fc640 1 --2- 172.21.15.26:0/3360387100 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f450c0a2d80 0x7f450c0a5240 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f452406ab00 tx=0x7f4518009500 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:18:44.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:44.534+0000 7f450bfff640 1 -- 172.21.15.26:0/3360387100 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f451409bb20 con 0x7f4524100e20 2024-09-11T23:18:44.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:44.747+0000 7f45294d4640 1 -- 172.21.15.26:0/3360387100 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f45241a51e0 con 0x7f4524100e20 2024-09-11T23:18:44.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:44.748+0000 7f450bfff640 1 -- 172.21.15.26:0/3360387100 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11971 (secure 0 0 0) 0x7f451409b420 con 0x7f4524100e20 2024-09-11T23:18:44.748 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-11T23:18:44.748 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":29,"btime":"2024-09-11T23:18:21:019324+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14499,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/367659515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":367659515},{"type":"v1","addr":"172.21.15.26:6835","nonce":367659515}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14511,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/2041065299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":2041065299},{"type":"v1","addr":"172.21.15.26:6837","nonce":2041065299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:16:26.544203+0000","modified":"2024-09-11T23:18:20.969151+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24485,"mds_1":24463,"mds_2":24458,"mds_3":24448,"mds_4":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14550":{"gid":14550,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6839/2233068305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":2233068305},{"type":"v1","addr":"172.21.15.26:6839","nonce":2233068305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14562":{"gid":14562,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6841/2984982336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":2984982336},{"type":"v1","addr":"172.21.15.26:6841","nonce":2984982336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6833/2019841455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":2019841455},{"type":"v1","addr":"172.21.15.136:6833","nonce":2019841455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.76:6833/3438147050","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3438147050},{"type":"v1","addr":"172.21.15.76:6833","nonce":3438147050}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6835/3617665975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3617665975},{"type":"v1","addr":"172.21.15.136:6835","nonce":3617665975}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":12,"addr":"172.21.15.76:6835/393308499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":393308499},{"type":"v1","addr":"172.21.15.76:6835","nonce":393308499}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":10,"addr":"172.21.15.136:6837/1633082387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":1633082387},{"type":"v1","addr":"172.21.15.136:6837","nonce":1633082387}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.76:6837/2078978748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":2078978748},{"type":"v1","addr":"172.21.15.76:6837","nonce":2078978748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.136:6839/3211398630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":3211398630},{"type":"v1","addr":"172.21.15.136:6839","nonce":3211398630}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.76:6839/2077637090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6838","nonce":2077637090},{"type":"v1","addr":"172.21.15.76:6839","nonce":2077637090}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24485,"qdb_cluster":[24443,24448,24458,24463,24485]},"id":1}]} 2024-09-11T23:18:44.748 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 29 2024-09-11T23:18:44.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:44.751+0000 7f45294d4640 1 -- 172.21.15.26:0/3360387100 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f450c0a2d80 msgr2=0x7f450c0a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:44.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:44.751+0000 7f45294d4640 1 --2- 172.21.15.26:0/3360387100 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f450c0a2d80 0x7f450c0a5240 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f452406ab00 tx=0x7f4518009500 comp rx=0 tx=0).stop 2024-09-11T23:18:44.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:44.751+0000 7f45294d4640 1 -- 172.21.15.26:0/3360387100 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4524100e20 msgr2=0x7f4524199330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:44.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:44.751+0000 7f45294d4640 1 --2- 172.21.15.26:0/3360387100 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4524100e20 0x7f4524199330 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f451400d770 tx=0x7f451400dc30 comp rx=0 tx=0).stop 2024-09-11T23:18:44.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:44.751+0000 7f45294d4640 1 -- 172.21.15.26:0/3360387100 shutdown_connections 2024-09-11T23:18:44.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:44.751+0000 7f45294d4640 1 --2- 172.21.15.26:0/3360387100 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f450c0a2d80 0x7f450c0a5240 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:44.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:44.751+0000 7f45294d4640 1 --2- 172.21.15.26:0/3360387100 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4524102ef0 0x7f45241a43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:44.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:44.751+0000 7f45294d4640 1 --2- 172.21.15.26:0/3360387100 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4524102570 0x7f4524199870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:44.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:44.751+0000 7f45294d4640 1 --2- 172.21.15.26:0/3360387100 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4524100e20 0x7f4524199330 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:44.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:44.751+0000 7f45294d4640 1 -- 172.21.15.26:0/3360387100 >> 172.21.15.26:0/3360387100 conn(0x7f45240fc7d0 msgr2=0x7f4524106880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:18:44.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:44.751+0000 7f45294d4640 1 -- 172.21.15.26:0/3360387100 shutdown_connections 2024-09-11T23:18:44.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:44.751+0000 7f45294d4640 1 -- 172.21.15.26:0/3360387100 wait complete. 2024-09-11T23:18:45.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:44 smithi026 ceph-mon[31172]: from='client.? 172.21.15.26:0/3360387100' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:18:45.170 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.170+0000 7f829e9c4640 1 -- 172.21.15.26:0/1454566399 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8298103640 msgr2=0x7f829810bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:45.170 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.170+0000 7f829e9c4640 1 --2- 172.21.15.26:0/1454566399 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8298103640 0x7f829810bb20 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f8280009900 tx=0x7f828002a480 comp rx=0 tx=0).stop 2024-09-11T23:18:45.171 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.170+0000 7f829e9c4640 1 -- 172.21.15.26:0/1454566399 shutdown_connections 2024-09-11T23:18:45.171 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.170+0000 7f829e9c4640 1 --2- 172.21.15.26:0/1454566399 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f829810c0f0 0x7f829810e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:45.171 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.170+0000 7f829e9c4640 1 --2- 172.21.15.26:0/1454566399 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8298103640 0x7f829810bb20 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:45.171 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.170+0000 7f829e9c4640 1 --2- 172.21.15.26:0/1454566399 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8298102d20 0x7f8298103100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:45.171 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.170+0000 7f829e9c4640 1 -- 172.21.15.26:0/1454566399 >> 172.21.15.26:0/1454566399 conn(0x7f8298077990 msgr2=0x7f82980fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:18:45.171 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.170+0000 7f829e9c4640 1 -- 172.21.15.26:0/1454566399 shutdown_connections 2024-09-11T23:18:45.171 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.170+0000 7f829e9c4640 1 -- 172.21.15.26:0/1454566399 wait complete. 2024-09-11T23:18:45.171 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.171+0000 7f829e9c4640 1 Processor -- start 2024-09-11T23:18:45.171 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.171+0000 7f829e9c4640 1 -- start start 2024-09-11T23:18:45.173 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.171+0000 7f829e9c4640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8298102d20 0x7f8298198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:45.173 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.171+0000 7f829e9c4640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8298103640 0x7f82981988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:45.173 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.171+0000 7f829e9c4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f829810c0f0 0x7f829819cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:45.173 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.171+0000 7f829e9c4640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f8298111ec0 con 0x7f829810c0f0 2024-09-11T23:18:45.173 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.171+0000 7f829e9c4640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f8298111d40 con 0x7f8298103640 2024-09-11T23:18:45.173 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.171+0000 7f829e9c4640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f8298112040 con 0x7f8298102d20 2024-09-11T23:18:45.173 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.171+0000 7f829cf61640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f829810c0f0 0x7f829819cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:45.173 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.171+0000 7f829cf61640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f829810c0f0 0x7f829819cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46322/0 (socket says 172.21.15.26:46322) 2024-09-11T23:18:45.173 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.171+0000 7f829cf61640 1 -- 172.21.15.26:0/2274729827 learned_addr learned my addr 172.21.15.26:0/2274729827 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:18:45.173 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.171+0000 7f828ffff640 1 --2- 172.21.15.26:0/2274729827 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8298103640 0x7f82981988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:45.174 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.171+0000 7f8297fff640 1 --2- 172.21.15.26:0/2274729827 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8298102d20 0x7f8298198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:45.174 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.171+0000 7f829cf61640 1 -- 172.21.15.26:0/2274729827 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8298102d20 msgr2=0x7f8298198370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:45.174 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.171+0000 7f829cf61640 1 --2- 172.21.15.26:0/2274729827 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8298102d20 0x7f8298198370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:45.174 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.171+0000 7f829cf61640 1 -- 172.21.15.26:0/2274729827 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8298103640 msgr2=0x7f82981988b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:45.174 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.171+0000 7f829cf61640 1 --2- 172.21.15.26:0/2274729827 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8298103640 0x7f82981988b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:45.174 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.171+0000 7f829cf61640 1 -- 172.21.15.26:0/2274729827 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f829819d360 con 0x7f829810c0f0 2024-09-11T23:18:45.174 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.172+0000 7f829cf61640 1 --2- 172.21.15.26:0/2274729827 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f829810c0f0 0x7f829819cc30 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f829000eaa0 tx=0x7f829000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:18:45.174 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.172+0000 7f8295ffb640 1 -- 172.21.15.26:0/2274729827 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f829000cdb0 con 0x7f829810c0f0 2024-09-11T23:18:45.174 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.172+0000 7f8295ffb640 1 -- 172.21.15.26:0/2274729827 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8290005020 con 0x7f829810c0f0 2024-09-11T23:18:45.174 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.172+0000 7f8295ffb640 1 -- 172.21.15.26:0/2274729827 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82900116b0 con 0x7f829810c0f0 2024-09-11T23:18:45.174 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.172+0000 7f829e9c4640 1 -- 172.21.15.26:0/2274729827 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f829819d650 con 0x7f829810c0f0 2024-09-11T23:18:45.174 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.172+0000 7f829e9c4640 1 -- 172.21.15.26:0/2274729827 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82981a4ee0 con 0x7f829810c0f0 2024-09-11T23:18:45.181 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.174+0000 7f8295ffb640 1 -- 172.21.15.26:0/2274729827 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 24) ==== 138009+0+0 (secure 0 0 0) 0x7f8290005560 con 0x7f829810c0f0 2024-09-11T23:18:45.181 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.174+0000 7f82a1840740 1 Processor -- start 2024-09-11T23:18:45.181 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.174+0000 7f82a1840740 1 -- start start 2024-09-11T23:18:45.181 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.175+0000 7f82a1840740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a7b81999b0 0x55a7b8199d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:45.181 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.175+0000 7f82a1840740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a7b819a2d0 0x55a7b81a8b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:45.181 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.175+0000 7f82a1840740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a7b81a90c0 0x55a7b81ab4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:45.181 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.175+0000 7f82a1840740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55a7b80fe2f0 con 0x55a7b819a2d0 2024-09-11T23:18:45.181 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.175+0000 7f82a1840740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55a7b80a6270 con 0x55a7b81a90c0 2024-09-11T23:18:45.181 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.175+0000 7f82a1840740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55a7b80fa2b0 con 0x55a7b81999b0 2024-09-11T23:18:45.181 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.175+0000 7f8297fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a7b819a2d0 0x55a7b81a8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:45.181 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.175+0000 7f8297fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a7b819a2d0 0x55a7b81a8b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46336/0 (socket says 172.21.15.26:46336) 2024-09-11T23:18:45.181 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.175+0000 7f8297fff640 1 -- 172.21.15.26:0/4116935730 learned_addr learned my addr 172.21.15.26:0/4116935730 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:18:45.182 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.175+0000 7f828ffff640 1 --2- 172.21.15.26:0/4116935730 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a7b81999b0 0x55a7b8199d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:45.182 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.175+0000 7f828ffff640 1 --2- 172.21.15.26:0/4116935730 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a7b81a90c0 0x55a7b81ab4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:45.182 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.175+0000 7f8297fff640 1 -- 172.21.15.26:0/4116935730 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a7b81999b0 msgr2=0x55a7b8199d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:18:45.182 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.175+0000 7f8297fff640 1 --2- 172.21.15.26:0/4116935730 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a7b81999b0 0x55a7b8199d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:45.182 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.175+0000 7f8297fff640 1 -- 172.21.15.26:0/4116935730 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a7b81a90c0 msgr2=0x55a7b81ab4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:18:45.182 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.175+0000 7f8297fff640 1 --2- 172.21.15.26:0/4116935730 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a7b81a90c0 0x55a7b81ab4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:45.182 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.175+0000 7f8297fff640 1 -- 172.21.15.26:0/4116935730 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a7b80a5340 con 0x55a7b819a2d0 2024-09-11T23:18:45.182 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.175+0000 7f8295ffb640 1 --2- 172.21.15.26:0/2274729827 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f82840a2e40 0x7f82840a5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:45.182 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.175+0000 7f828ffff640 1 --2- 172.21.15.26:0/4116935730 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a7b81999b0 0x55a7b8199d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:18:45.182 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.175+0000 7f8297fff640 1 --2- 172.21.15.26:0/4116935730 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a7b819a2d0 0x55a7b81a8b80 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f828800cbf0 tx=0x7f828800e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89cf9d5b938ba7da server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:18:45.182 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.175+0000 7f828ffff640 1 --2- 172.21.15.26:0/2274729827 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f82840a2e40 0x7f82840a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:45.182 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.175+0000 7f8295ffb640 1 -- 172.21.15.26:0/2274729827 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f82900cf830 con 0x7f829810c0f0 2024-09-11T23:18:45.183 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.175+0000 7f828e7fc640 1 -- 172.21.15.26:0/4116935730 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8288010630 con 0x55a7b819a2d0 2024-09-11T23:18:45.183 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.175+0000 7f828e7fc640 1 -- 172.21.15.26:0/4116935730 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8288010cb0 con 0x55a7b819a2d0 2024-09-11T23:18:45.183 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.175+0000 7f828e7fc640 1 -- 172.21.15.26:0/4116935730 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8288019c50 con 0x55a7b819a2d0 2024-09-11T23:18:45.183 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.176+0000 7f82a1840740 1 -- 172.21.15.26:0/4116935730 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a7b819a2d0 msgr2=0x55a7b81a8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:45.183 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.176+0000 7f828ffff640 1 --2- 172.21.15.26:0/2274729827 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f82840a2e40 0x7f82840a5300 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f829810e530 tx=0x7f8280009260 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:18:45.183 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.176+0000 7f82a1840740 1 --2- 172.21.15.26:0/4116935730 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a7b819a2d0 0x55a7b81a8b80 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f828800cbf0 tx=0x7f828800e720 comp rx=0 tx=0).stop 2024-09-11T23:18:45.183 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.176+0000 7f82a1840740 1 -- 172.21.15.26:0/4116935730 shutdown_connections 2024-09-11T23:18:45.183 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.176+0000 7f82a1840740 1 --2- 172.21.15.26:0/4116935730 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a7b81a90c0 0x55a7b81ab4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:45.183 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.176+0000 7f82a1840740 1 --2- 172.21.15.26:0/4116935730 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a7b819a2d0 0x55a7b81a8b80 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:45.183 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.176+0000 7f82a1840740 1 --2- 172.21.15.26:0/4116935730 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a7b81999b0 0x55a7b8199d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:45.183 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.176+0000 7f82a1840740 1 -- 172.21.15.26:0/4116935730 >> 172.21.15.26:0/4116935730 conn(0x55a7b8196060 msgr2=0x55a7b8196440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:18:45.183 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.176+0000 7f82a1840740 1 -- 172.21.15.26:0/4116935730 shutdown_connections 2024-09-11T23:18:45.183 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.176+0000 7f82a1840740 1 -- 172.21.15.26:0/4116935730 wait complete. 2024-09-11T23:18:45.183 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.177+0000 7f82a1840740 1 -- start start 2024-09-11T23:18:45.184 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.177+0000 7f82a1840740 1 Processor -- start 2024-09-11T23:18:45.184 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.177+0000 7f82a1840740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a7b81999b0 0x55a7b81a5fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:45.184 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.177+0000 7f82a1840740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a7b819a2d0 0x55a7b81a64f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:45.184 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.177+0000 7f82a1840740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a7b81a90c0 0x55a7b81a6a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:45.184 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.177+0000 7f82a1840740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55a7b81abc90 con 0x55a7b81999b0 2024-09-11T23:18:45.184 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.177+0000 7f82a1840740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55a7b8087290 con 0x55a7b81a90c0 2024-09-11T23:18:45.184 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.177+0000 7f82a1840740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55a7b81abe10 con 0x55a7b819a2d0 2024-09-11T23:18:45.184 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.177+0000 7f82a1840740 10 client.0 _ll_register_callbacks cb 0x55a7b80b8140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:18:45.184 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.177+0000 7f8297fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a7b81999b0 0x55a7b81a5fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:45.184 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.177+0000 7f8297fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a7b81999b0 0x55a7b81a5fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46346/0 (socket says 172.21.15.26:46346) 2024-09-11T23:18:45.184 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.177+0000 7f8297fff640 1 -- 172.21.15.26:0/1359806336 learned_addr learned my addr 172.21.15.26:0/1359806336 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:18:45.184 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.177+0000 7f828ffff640 1 --2- 172.21.15.26:0/1359806336 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a7b819a2d0 0x55a7b81a64f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:45.184 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.177+0000 7f829cf61640 1 --2- 172.21.15.26:0/1359806336 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a7b81a90c0 0x55a7b81a6a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:45.185 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.177+0000 7f8297fff640 1 -- 172.21.15.26:0/1359806336 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a7b819a2d0 msgr2=0x55a7b81a64f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:18:45.185 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.177+0000 7f8297fff640 1 --2- 172.21.15.26:0/1359806336 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a7b819a2d0 0x55a7b81a64f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:45.185 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.177+0000 7f8297fff640 1 -- 172.21.15.26:0/1359806336 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a7b81a90c0 msgr2=0x55a7b81a6a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:18:45.185 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.177+0000 7f8297fff640 1 --2- 172.21.15.26:0/1359806336 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a7b81a90c0 0x55a7b81a6a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:45.185 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.177+0000 7f8297fff640 1 -- 172.21.15.26:0/1359806336 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a7b7e11210 con 0x55a7b81999b0 2024-09-11T23:18:45.185 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.177+0000 7f828ffff640 1 --2- 172.21.15.26:0/1359806336 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a7b819a2d0 0x55a7b81a64f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:18:45.185 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.177+0000 7f829cf61640 1 --2- 172.21.15.26:0/1359806336 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a7b81a90c0 0x55a7b81a6a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:18:45.185 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.178+0000 7f8297fff640 1 --2- 172.21.15.26:0/1359806336 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a7b81999b0 0x55a7b81a5fb0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f828800cfa0 tx=0x7f8288011fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=289ca3ebe529d32d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:18:45.185 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.178+0000 7f82777fe640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:18:45.185 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.178+0000 7f82777fe640 1 -- 172.21.15.26:0/1359806336 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8288019da0 con 0x55a7b81999b0 2024-09-11T23:18:45.185 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.178+0000 7f82777fe640 1 -- 172.21.15.26:0/1359806336 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f8288005050 con 0x55a7b81999b0 2024-09-11T23:18:45.185 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.178+0000 7f82a1840740 1 -- 172.21.15.26:0/1359806336 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8288010cb0 con 0x55a7b81999b0 2024-09-11T23:18:45.185 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.178+0000 7f82777fe640 1 -- 172.21.15.26:0/1359806336 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8288005250 con 0x55a7b81999b0 2024-09-11T23:18:45.186 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.178+0000 7f82a1840740 10 client.15417 fetch_fsmap learned FSMap version 29 2024-09-11T23:18:45.186 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.178+0000 7f82a1840740 1 -- 172.21.15.26:0/1359806336 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a7b7edd620 con 0x55a7b81999b0 2024-09-11T23:18:45.186 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.178+0000 7f82777fe640 1 -- 172.21.15.26:0/1359806336 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8288010cb0 con 0x55a7b81999b0 2024-09-11T23:18:45.186 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.178+0000 7f82777fe640 1 -- 172.21.15.26:0/1359806336 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f8288004920 con 0x55a7b81999b0 2024-09-11T23:18:45.186 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.178+0000 7f82a1840740 10 client.15417 fetch_fsmap finished waiting for FSMap version 29 2024-09-11T23:18:45.186 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.178+0000 7f82a1840740 10 client.15417 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:18:45.186 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.178+0000 7f82a1840740 20 client.15417 populate_metadata read hostname 'smithi026' 2024-09-11T23:18:45.186 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.178+0000 7f82a1840740 1 --2- 172.21.15.26:0/1359806336 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55a7b81b8f80 0x55a7b81bb370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:45.186 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.178+0000 7f82a1840740 4 client.15417 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:18:45.186 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.178+0000 7f82a1840740 1 -- 172.21.15.26:0/1359806336 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a7b810ba20 con 0x55a7b81b8f80 2024-09-11T23:18:45.186 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.179+0000 7f828ffff640 1 --2- 172.21.15.26:0/1359806336 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55a7b81b8f80 0x55a7b81bb370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:45.186 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.179+0000 7f828ffff640 1 --2- 172.21.15.26:0/1359806336 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55a7b81b8f80 0x55a7b81bb370 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=852da76adbe0386f server_cookie=863d6d4c3cb1d135 in_seq=0 out_seq=0 2024-09-11T23:18:45.186 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.179+0000 7f82777fe640 10 client.15417 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:18:45.187 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.180+0000 7f82777fe640 1 -- 172.21.15.26:0/1359806336 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a7b810ba20 con 0x55a7b81b8f80 2024-09-11T23:18:45.187 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.180+0000 7f82777fe640 10 client.15417 handle_command_reply: tid=0 2024-09-11T23:18:45.187 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.180+0000 7f82a1840740 1 client.15417 shutdown 2024-09-11T23:18:45.187 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.180+0000 7f82a1840740 1 -- 172.21.15.26:0/1359806336 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a7b81999b0 msgr2=0x55a7b81a5fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:45.187 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.180+0000 7f82a1840740 1 --2- 172.21.15.26:0/1359806336 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a7b81999b0 0x55a7b81a5fb0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f828800cfa0 tx=0x7f8288011fb0 comp rx=0 tx=0).stop 2024-09-11T23:18:45.187 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.180+0000 7f82a1840740 1 -- 172.21.15.26:0/1359806336 shutdown_connections 2024-09-11T23:18:45.187 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.180+0000 7f82a1840740 1 --2- 172.21.15.26:0/1359806336 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a7b81a90c0 0x55a7b81a6a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:45.187 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.180+0000 7f82a1840740 1 --2- 172.21.15.26:0/1359806336 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55a7b81b8f80 0x55a7b81bb370 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:45.187 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.180+0000 7f82a1840740 1 --2- 172.21.15.26:0/1359806336 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a7b819a2d0 0x55a7b81a64f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:45.187 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.180+0000 7f82a1840740 1 --2- 172.21.15.26:0/1359806336 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a7b81999b0 0x55a7b81a5fb0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:45.187 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.180+0000 7f82777fe640 0 client.15417 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:18:45.187 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.180+0000 7f82a1840740 1 -- 172.21.15.26:0/1359806336 >> 172.21.15.26:0/1359806336 conn(0x55a7b8196060 msgr2=0x55a7b819bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:18:45.187 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.180+0000 7f82a1840740 1 -- 172.21.15.26:0/1359806336 shutdown_connections 2024-09-11T23:18:45.187 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.180+0000 7f82a1840740 1 -- 172.21.15.26:0/1359806336 wait complete. 2024-09-11T23:18:45.187 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.181+0000 7f82a1840740 20 client.15417 trim_cache size 0 max 16384 2024-09-11T23:18:45.200 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.195+0000 7f82a1840740 1 Processor -- start 2024-09-11T23:18:45.200 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.195+0000 7f82a1840740 1 -- start start 2024-09-11T23:18:45.200 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.195+0000 7f82a1840740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a7b81999b0 0x55a7b81c0b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:45.200 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.195+0000 7f82a1840740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a7b819a2d0 0x55a7b81c1090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:45.200 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.195+0000 7f82a1840740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a7b81a90c0 0x55a7b81a31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:45.200 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.195+0000 7f82a1840740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f82981ab650 con 0x55a7b81a90c0 2024-09-11T23:18:45.200 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.195+0000 7f82a1840740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f82981ad190 con 0x55a7b81999b0 2024-09-11T23:18:45.200 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.195+0000 7f82a1840740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f82981abd20 con 0x55a7b819a2d0 2024-09-11T23:18:45.201 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.195+0000 7f829cf61640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a7b81a90c0 0x55a7b81a31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:45.201 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.195+0000 7f829cf61640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a7b81a90c0 0x55a7b81a31f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46354/0 (socket says 172.21.15.26:46354) 2024-09-11T23:18:45.201 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.195+0000 7f829cf61640 1 -- 172.21.15.26:0/2367591447 learned_addr learned my addr 172.21.15.26:0/2367591447 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:18:45.201 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.195+0000 7f8297fff640 1 --2- 172.21.15.26:0/2367591447 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a7b81999b0 0x55a7b81c0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:45.201 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.195+0000 7f828ffff640 1 --2- 172.21.15.26:0/2367591447 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a7b819a2d0 0x55a7b81c1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:45.201 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.195+0000 7f829cf61640 1 -- 172.21.15.26:0/2367591447 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a7b819a2d0 msgr2=0x55a7b81c1090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:18:45.201 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.195+0000 7f829cf61640 1 --2- 172.21.15.26:0/2367591447 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a7b819a2d0 0x55a7b81c1090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:45.201 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.195+0000 7f829cf61640 1 -- 172.21.15.26:0/2367591447 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a7b81999b0 msgr2=0x55a7b81c0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:18:45.201 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.195+0000 7f829cf61640 1 --2- 172.21.15.26:0/2367591447 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a7b81999b0 0x55a7b81c0b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:45.201 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.195+0000 7f829cf61640 1 -- 172.21.15.26:0/2367591447 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a7b819a100 con 0x55a7b81a90c0 2024-09-11T23:18:45.201 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.195+0000 7f8297fff640 1 --2- 172.21.15.26:0/2367591447 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a7b81999b0 0x55a7b81c0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:18:45.201 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.196+0000 7f829cf61640 1 --2- 172.21.15.26:0/2367591447 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a7b81a90c0 0x55a7b81a31f0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f82900cd590 tx=0x7f829009baf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34300ff8ae118029 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:18:45.202 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.196+0000 7f828dffb640 1 -- 172.21.15.26:0/2367591447 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82900b2070 con 0x55a7b81a90c0 2024-09-11T23:18:45.202 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.196+0000 7f828dffb640 1 -- 172.21.15.26:0/2367591447 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82900a4860 con 0x55a7b81a90c0 2024-09-11T23:18:45.202 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.196+0000 7f828dffb640 1 -- 172.21.15.26:0/2367591447 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f829001f050 con 0x55a7b81a90c0 2024-09-11T23:18:45.202 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.196+0000 7f82a1840740 1 -- 172.21.15.26:0/2367591447 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a7b81a90c0 msgr2=0x55a7b81a31f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:45.202 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.196+0000 7f82a1840740 1 --2- 172.21.15.26:0/2367591447 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a7b81a90c0 0x55a7b81a31f0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f82900cd590 tx=0x7f829009baf0 comp rx=0 tx=0).stop 2024-09-11T23:18:45.202 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.196+0000 7f82a1840740 1 -- 172.21.15.26:0/2367591447 shutdown_connections 2024-09-11T23:18:45.202 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.196+0000 7f82a1840740 1 --2- 172.21.15.26:0/2367591447 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a7b81a90c0 0x55a7b81a31f0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:45.202 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.196+0000 7f82a1840740 1 --2- 172.21.15.26:0/2367591447 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a7b819a2d0 0x55a7b81c1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:45.202 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.196+0000 7f82a1840740 1 --2- 172.21.15.26:0/2367591447 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a7b81999b0 0x55a7b81c0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:45.202 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.196+0000 7f82a1840740 1 -- 172.21.15.26:0/2367591447 >> 172.21.15.26:0/2367591447 conn(0x55a7b8196060 msgr2=0x55a7b81a8030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:18:45.202 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.196+0000 7f82a1840740 1 -- 172.21.15.26:0/2367591447 shutdown_connections 2024-09-11T23:18:45.202 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.196+0000 7f82a1840740 1 -- 172.21.15.26:0/2367591447 wait complete. 2024-09-11T23:18:45.203 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.197+0000 7f82a1840740 1 -- start start 2024-09-11T23:18:45.203 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.197+0000 7f82a1840740 1 Processor -- start 2024-09-11T23:18:45.203 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.197+0000 7f82a1840740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a7b81999b0 0x55a7b81b1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:45.203 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.197+0000 7f82a1840740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a7b819a2d0 0x55a7b81ac830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:45.203 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.197+0000 7f82a1840740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a7b81a90c0 0x55a7b81acd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:45.203 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.197+0000 7f82a1840740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f82981acac0 con 0x55a7b81999b0 2024-09-11T23:18:45.203 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.197+0000 7f82a1840740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f82981ac3f0 con 0x55a7b81a90c0 2024-09-11T23:18:45.203 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.197+0000 7f82a1840740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f82981a9630 con 0x55a7b819a2d0 2024-09-11T23:18:45.203 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.197+0000 7f82a1840740 10 client.0 _ll_register_callbacks cb 0x7f8288014070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:18:45.203 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.197+0000 7f8297fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a7b81999b0 0x55a7b81b1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:45.203 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.197+0000 7f8297fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a7b81999b0 0x55a7b81b1510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46356/0 (socket says 172.21.15.26:46356) 2024-09-11T23:18:45.203 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.197+0000 7f8297fff640 1 -- 172.21.15.26:0/1246800474 learned_addr learned my addr 172.21.15.26:0/1246800474 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:18:45.203 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.197+0000 7f829cf61640 1 --2- 172.21.15.26:0/1246800474 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a7b81a90c0 0x55a7b81acd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:45.204 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.198+0000 7f828ffff640 1 --2- 172.21.15.26:0/1246800474 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a7b819a2d0 0x55a7b81ac830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:45.204 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.198+0000 7f8297fff640 1 -- 172.21.15.26:0/1246800474 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a7b819a2d0 msgr2=0x55a7b81ac830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:18:45.204 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.198+0000 7f8297fff640 1 --2- 172.21.15.26:0/1246800474 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a7b819a2d0 0x55a7b81ac830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:45.204 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.198+0000 7f8297fff640 1 -- 172.21.15.26:0/1246800474 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a7b81a90c0 msgr2=0x55a7b81acd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:18:45.204 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.198+0000 7f8297fff640 1 --2- 172.21.15.26:0/1246800474 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a7b81a90c0 0x55a7b81acd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:45.204 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.198+0000 7f8297fff640 1 -- 172.21.15.26:0/1246800474 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a7b81ad2b0 con 0x55a7b81999b0 2024-09-11T23:18:45.204 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.198+0000 7f829cf61640 1 --2- 172.21.15.26:0/1246800474 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a7b81a90c0 0x55a7b81acd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:18:45.204 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.198+0000 7f8297fff640 1 --2- 172.21.15.26:0/1246800474 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a7b81999b0 0x55a7b81b1510 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f828800cbf0 tx=0x7f82880051e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a6a53c4ab9a731b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:18:45.204 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.198+0000 7f82777fe640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:18:45.204 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.198+0000 7f82777fe640 1 -- 172.21.15.26:0/1246800474 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8288025420 con 0x55a7b81999b0 2024-09-11T23:18:45.204 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.198+0000 7f82a1840740 1 -- 172.21.15.26:0/1246800474 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f82900a4860 con 0x55a7b81999b0 2024-09-11T23:18:45.204 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.198+0000 7f82777fe640 1 -- 172.21.15.26:0/1246800474 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f82880053f0 con 0x55a7b81999b0 2024-09-11T23:18:45.204 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.198+0000 7f82777fe640 1 -- 172.21.15.26:0/1246800474 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f828803c050 con 0x55a7b81999b0 2024-09-11T23:18:45.205 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.198+0000 7f82a1840740 10 client.15423 fetch_fsmap learned FSMap version 29 2024-09-11T23:18:45.205 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.198+0000 7f82a1840740 1 -- 172.21.15.26:0/1246800474 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a7b8223820 con 0x55a7b81999b0 2024-09-11T23:18:45.205 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.198+0000 7f82777fe640 1 -- 172.21.15.26:0/1246800474 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82900a4860 con 0x55a7b81999b0 2024-09-11T23:18:45.205 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.198+0000 7f82777fe640 1 -- 172.21.15.26:0/1246800474 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f8288034020 con 0x55a7b81999b0 2024-09-11T23:18:45.205 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.199+0000 7f82a1840740 10 client.15423 fetch_fsmap finished waiting for FSMap version 29 2024-09-11T23:18:45.205 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.199+0000 7f82a1840740 10 client.15423 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:18:45.205 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.199+0000 7f82a1840740 20 client.15423 populate_metadata read hostname 'smithi026' 2024-09-11T23:18:45.205 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.199+0000 7f82a1840740 1 --2- 172.21.15.26:0/1246800474 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55a7b8223f30 0x55a7b8226350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:45.205 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.199+0000 7f82a1840740 4 client.15423 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T23:18:45.205 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.199+0000 7f82a1840740 1 -- 172.21.15.26:0/1246800474 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a7b8105d70 con 0x55a7b8223f30 2024-09-11T23:18:45.205 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.199+0000 7f828ffff640 1 --2- 172.21.15.26:0/1246800474 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55a7b8223f30 0x55a7b8226350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:45.205 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.199+0000 7f82777fe640 10 client.15423 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:18:45.205 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.199+0000 7f828ffff640 1 --2- 172.21.15.26:0/1246800474 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55a7b8223f30 0x55a7b8226350 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4742ce87c92aa96 server_cookie=e6119ce28462ff1a in_seq=0 out_seq=0 2024-09-11T23:18:45.211 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.211+0000 7f82777fe640 1 -- 172.21.15.26:0/1246800474 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a7b8105d70 con 0x55a7b8223f30 2024-09-11T23:18:45.212 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.211+0000 7f82777fe640 10 client.15423 handle_command_reply: tid=0 2024-09-11T23:18:45.212 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.211+0000 7f82a1840740 1 client.15423 shutdown 2024-09-11T23:18:45.212 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.211+0000 7f82a1840740 1 -- 172.21.15.26:0/1246800474 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a7b81999b0 msgr2=0x55a7b81b1510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:45.212 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.211+0000 7f82a1840740 1 --2- 172.21.15.26:0/1246800474 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a7b81999b0 0x55a7b81b1510 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f828800cbf0 tx=0x7f82880051e0 comp rx=0 tx=0).stop 2024-09-11T23:18:45.212 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.211+0000 7f82a1840740 1 -- 172.21.15.26:0/1246800474 shutdown_connections 2024-09-11T23:18:45.212 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.211+0000 7f82a1840740 1 --2- 172.21.15.26:0/1246800474 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a7b81a90c0 0x55a7b81acd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:45.212 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.211+0000 7f82a1840740 1 --2- 172.21.15.26:0/1246800474 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55a7b8223f30 0x55a7b8226350 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:45.212 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.211+0000 7f82a1840740 1 --2- 172.21.15.26:0/1246800474 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a7b819a2d0 0x55a7b81ac830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:45.212 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.211+0000 7f82a1840740 1 --2- 172.21.15.26:0/1246800474 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a7b81999b0 0x55a7b81b1510 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:45.212 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.211+0000 7f82777fe640 0 client.15423 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:18:45.212 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.211+0000 7f82a1840740 1 -- 172.21.15.26:0/1246800474 >> 172.21.15.26:0/1246800474 conn(0x55a7b8196060 msgr2=0x55a7b819b430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:18:45.212 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.211+0000 7f82a1840740 1 -- 172.21.15.26:0/1246800474 shutdown_connections 2024-09-11T23:18:45.213 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.211+0000 7f82a1840740 1 -- 172.21.15.26:0/1246800474 wait complete. 2024-09-11T23:18:45.213 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.211+0000 7f82a1840740 20 client.15423 trim_cache size 0 max 16384 2024-09-11T23:18:45.213 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-09-11T23:18:45.213 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "message": "", 2024-09-11T23:18:45.213 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-09-11T23:18:45.213 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-09-11T23:18:45.214 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.212+0000 7f829e9c4640 1 -- 172.21.15.26:0/2274729827 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f82840a2e40 msgr2=0x7f82840a5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:45.214 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.212+0000 7f829e9c4640 1 --2- 172.21.15.26:0/2274729827 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f82840a2e40 0x7f82840a5300 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f829810e530 tx=0x7f8280009260 comp rx=0 tx=0).stop 2024-09-11T23:18:45.214 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.212+0000 7f829e9c4640 1 -- 172.21.15.26:0/2274729827 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f829810c0f0 msgr2=0x7f829819cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:45.214 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.212+0000 7f829e9c4640 1 --2- 172.21.15.26:0/2274729827 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f829810c0f0 0x7f829819cc30 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f829000eaa0 tx=0x7f829000ef60 comp rx=0 tx=0).stop 2024-09-11T23:18:45.214 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.212+0000 7f829e9c4640 1 -- 172.21.15.26:0/2274729827 shutdown_connections 2024-09-11T23:18:45.214 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.212+0000 7f829e9c4640 1 --2- 172.21.15.26:0/2274729827 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f82840a2e40 0x7f82840a5300 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:45.214 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.212+0000 7f829e9c4640 1 --2- 172.21.15.26:0/2274729827 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8298103640 0x7f82981988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:45.214 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.212+0000 7f829e9c4640 1 --2- 172.21.15.26:0/2274729827 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f829810c0f0 0x7f829819cc30 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:45.214 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.212+0000 7f829e9c4640 1 --2- 172.21.15.26:0/2274729827 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8298102d20 0x7f8298198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:45.215 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.212+0000 7f829e9c4640 1 -- 172.21.15.26:0/2274729827 >> 172.21.15.26:0/2274729827 conn(0x7f8298077990 msgr2=0x7f82980fd060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:18:45.215 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.212+0000 7f829e9c4640 1 -- 172.21.15.26:0/2274729827 shutdown_connections 2024-09-11T23:18:45.215 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:45.212+0000 7f829e9c4640 1 -- 172.21.15.26:0/2274729827 wait complete. 2024-09-11T23:18:45.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:44 smithi136 ceph-mon[35121]: from='client.? 172.21.15.26:0/3360387100' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:18:45.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:44 smithi076 ceph-mon[35296]: from='client.? 172.21.15.26:0/3360387100' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:18:45.396 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 29, 'flags': 306, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': True}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-11T23:16:26.544203+0000', 'modified': '2024-09-11T23:18:20.969151+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24485, 'mds_1': 24463, 'mds_2': 24458, 'mds_3': 24448, 'mds_4': 24443}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14550': {'gid': 14550, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.26:6839/2233068305', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6838', 'nonce': 2233068305}, {'type': 'v1', 'addr': '172.21.15.26:6839', 'nonce': 2233068305}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14562': {'gid': 14562, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.26:6841/2984982336', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6840', 'nonce': 2984982336}, {'type': 'v1', 'addr': '172.21.15.26:6841', 'nonce': 2984982336}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24403': {'gid': 24403, 'name': 'l', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.136:6833/2019841455', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6832', 'nonce': 2019841455}, {'type': 'v1', 'addr': '172.21.15.136:6833', 'nonce': 2019841455}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24407': {'gid': 24407, 'name': 'k', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.76:6833/3438147050', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6832', 'nonce': 3438147050}, {'type': 'v1', 'addr': '172.21.15.76:6833', 'nonce': 3438147050}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24439': {'gid': 24439, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.136:6835/3617665975', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6834', 'nonce': 3617665975}, {'type': 'v1', 'addr': '172.21.15.136:6835', 'nonce': 3617665975}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24443': {'gid': 24443, 'name': 'e', 'rank': 4, 'incarnation': 26, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.76:6835/393308499', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6834', 'nonce': 393308499}, {'type': 'v1', 'addr': '172.21.15.76:6835', 'nonce': 393308499}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24448': {'gid': 24448, 'name': 'c', 'rank': 3, 'incarnation': 24, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.136:6837/1633082387', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6836', 'nonce': 1633082387}, {'type': 'v1', 'addr': '172.21.15.136:6837', 'nonce': 1633082387}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24458': {'gid': 24458, 'name': 'h', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.76:6837/2078978748', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6836', 'nonce': 2078978748}, {'type': 'v1', 'addr': '172.21.15.76:6837', 'nonce': 2078978748}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24463': {'gid': 24463, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.136:6839/3211398630', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6838', 'nonce': 3211398630}, {'type': 'v1', 'addr': '172.21.15.136:6839', 'nonce': 3211398630}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24485': {'gid': 24485, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.76:6839/2077637090', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6838', 'nonce': 2077637090}, {'type': 'v1', 'addr': '172.21.15.76:6839', 'nonce': 2077637090}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24485, 'qdb_cluster': [24443, 24448, 24458, 24463, 24485]} 2024-09-11T23:18:45.396 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-09-11T23:18:45.396 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph tell mds.e status 2024-09-11T23:18:45.693 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:18:46.072 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:45 smithi026 ceph-mon[31172]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 76 KiB/s wr, 44 op/s 2024-09-11T23:18:46.140 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:45 smithi136 ceph-mon[35121]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 76 KiB/s wr, 44 op/s 2024-09-11T23:18:46.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.155+0000 7f2f8a95a640 1 -- 172.21.15.26:0/2319246366 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2f84100e20 msgr2=0x7f2f84101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:46.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.155+0000 7f2f8a95a640 1 --2- 172.21.15.26:0/2319246366 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2f84100e20 0x7f2f84101200 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f2f74009900 tx=0x7f2f7402a480 comp rx=0 tx=0).stop 2024-09-11T23:18:46.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.155+0000 7f2f8a95a640 1 -- 172.21.15.26:0/2319246366 shutdown_connections 2024-09-11T23:18:46.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.155+0000 7f2f8a95a640 1 --2- 172.21.15.26:0/2319246366 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2f84102570 0x7f2f841029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:46.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.155+0000 7f2f8a95a640 1 --2- 172.21.15.26:0/2319246366 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2f84102ef0 0x7f2f8410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:46.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.155+0000 7f2f8a95a640 1 --2- 172.21.15.26:0/2319246366 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2f84100e20 0x7f2f84101200 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:46.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.155+0000 7f2f8a95a640 1 -- 172.21.15.26:0/2319246366 >> 172.21.15.26:0/2319246366 conn(0x7f2f840fc7d0 msgr2=0x7f2f840febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:18:46.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.155+0000 7f2f8a95a640 1 -- 172.21.15.26:0/2319246366 shutdown_connections 2024-09-11T23:18:46.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.156+0000 7f2f8a95a640 1 -- 172.21.15.26:0/2319246366 wait complete. 2024-09-11T23:18:46.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.156+0000 7f2f8a95a640 1 Processor -- start 2024-09-11T23:18:46.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.156+0000 7f2f8a95a640 1 -- start start 2024-09-11T23:18:46.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.157+0000 7f2f8a95a640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2f84100e20 0x7f2f84199460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:46.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.157+0000 7f2f8a95a640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2f84102570 0x7f2f841999a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:46.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.157+0000 7f2f8a95a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2f84102ef0 0x7f2f84199f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:46.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.157+0000 7f2f8a95a640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f2f84110f50 con 0x7f2f84102ef0 2024-09-11T23:18:46.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.157+0000 7f2f8a95a640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f2f84110dd0 con 0x7f2f84102570 2024-09-11T23:18:46.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.157+0000 7f2f8a95a640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f2f841110d0 con 0x7f2f84100e20 2024-09-11T23:18:46.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.157+0000 7f2f88ef7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2f84102ef0 0x7f2f84199f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:46.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.157+0000 7f2f88ef7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2f84102ef0 0x7f2f84199f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46370/0 (socket says 172.21.15.26:46370) 2024-09-11T23:18:46.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.157+0000 7f2f88ef7640 1 -- 172.21.15.26:0/2356773171 learned_addr learned my addr 172.21.15.26:0/2356773171 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:18:46.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.157+0000 7f2f837fe640 1 --2- 172.21.15.26:0/2356773171 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2f84102570 0x7f2f841999a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:46.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.157+0000 7f2f83fff640 1 --2- 172.21.15.26:0/2356773171 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2f84100e20 0x7f2f84199460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:46.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.157+0000 7f2f88ef7640 1 -- 172.21.15.26:0/2356773171 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2f84100e20 msgr2=0x7f2f84199460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:46.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.157+0000 7f2f88ef7640 1 --2- 172.21.15.26:0/2356773171 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2f84100e20 0x7f2f84199460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:46.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.157+0000 7f2f88ef7640 1 -- 172.21.15.26:0/2356773171 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2f84102570 msgr2=0x7f2f841999a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:46.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.157+0000 7f2f88ef7640 1 --2- 172.21.15.26:0/2356773171 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2f84102570 0x7f2f841999a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:46.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.157+0000 7f2f88ef7640 1 -- 172.21.15.26:0/2356773171 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f841a4860 con 0x7f2f84102ef0 2024-09-11T23:18:46.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.157+0000 7f2f837fe640 1 --2- 172.21.15.26:0/2356773171 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2f84102570 0x7f2f841999a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:18:46.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.157+0000 7f2f88ef7640 1 --2- 172.21.15.26:0/2356773171 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2f84102ef0 0x7f2f84199f20 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f2f7c00eaa0 tx=0x7f2f7c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:18:46.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.157+0000 7f2f817fa640 1 -- 172.21.15.26:0/2356773171 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f7c00cdb0 con 0x7f2f84102ef0 2024-09-11T23:18:46.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.157+0000 7f2f817fa640 1 -- 172.21.15.26:0/2356773171 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2f7c005020 con 0x7f2f84102ef0 2024-09-11T23:18:46.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.157+0000 7f2f817fa640 1 -- 172.21.15.26:0/2356773171 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f7c0116b0 con 0x7f2f84102ef0 2024-09-11T23:18:46.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.157+0000 7f2f8a95a640 1 -- 172.21.15.26:0/2356773171 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f841a4b50 con 0x7f2f84102ef0 2024-09-11T23:18:46.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.158+0000 7f2f8a95a640 1 -- 172.21.15.26:0/2356773171 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f841a5040 con 0x7f2f84102ef0 2024-09-11T23:18:46.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.160+0000 7f2f817fa640 1 -- 172.21.15.26:0/2356773171 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 24) ==== 138009+0+0 (secure 0 0 0) 0x7f2f7c005560 con 0x7f2f84102ef0 2024-09-11T23:18:46.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.160+0000 7f2f8cd33740 1 Processor -- start 2024-09-11T23:18:46.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.160+0000 7f2f8cd33740 1 -- start start 2024-09-11T23:18:46.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.160+0000 7f2f8cd33740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563b4c6029d0 0x563b4c602db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:46.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.160+0000 7f2f8cd33740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563b4c6032f0 0x563b4c60d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:46.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.160+0000 7f2f8cd33740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563b4c60dbb0 0x563b4c60ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:46.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.160+0000 7f2f8cd33740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x563b4c569090 con 0x563b4c6029d0 2024-09-11T23:18:46.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.160+0000 7f2f8cd33740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x563b4c510a70 con 0x563b4c6032f0 2024-09-11T23:18:46.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.160+0000 7f2f8cd33740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x563b4c565050 con 0x563b4c60dbb0 2024-09-11T23:18:46.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.160+0000 7f2f837fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563b4c6029d0 0x563b4c602db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:46.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.160+0000 7f2f837fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563b4c6029d0 0x563b4c602db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46378/0 (socket says 172.21.15.26:46378) 2024-09-11T23:18:46.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.160+0000 7f2f837fe640 1 -- 172.21.15.26:0/2871261847 learned_addr learned my addr 172.21.15.26:0/2871261847 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:18:46.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.161+0000 7f2f817fa640 1 --2- 172.21.15.26:0/2356773171 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f2f6c0a2d80 0x7f2f6c0a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:46.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.161+0000 7f2f83fff640 1 --2- 172.21.15.26:0/2871261847 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563b4c6032f0 0x563b4c60d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:46.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.161+0000 7f2f837fe640 1 --2- 172.21.15.26:0/2871261847 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563b4c60dbb0 0x563b4c60ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:46.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.161+0000 7f2f88ef7640 1 --2- 172.21.15.26:0/2356773171 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f2f6c0a2d80 0x7f2f6c0a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:46.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.161+0000 7f2f837fe640 1 -- 172.21.15.26:0/2871261847 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563b4c60dbb0 msgr2=0x563b4c60ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:18:46.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.161+0000 7f2f837fe640 1 --2- 172.21.15.26:0/2871261847 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563b4c60dbb0 0x563b4c60ffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:46.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.161+0000 7f2f817fa640 1 -- 172.21.15.26:0/2356773171 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f2f7c0cf740 con 0x7f2f84102ef0 2024-09-11T23:18:46.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.161+0000 7f2f837fe640 1 -- 172.21.15.26:0/2871261847 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563b4c6032f0 msgr2=0x563b4c60d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:18:46.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.161+0000 7f2f837fe640 1 --2- 172.21.15.26:0/2871261847 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563b4c6032f0 0x563b4c60d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:46.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.161+0000 7f2f837fe640 1 -- 172.21.15.26:0/2871261847 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b4c51d130 con 0x563b4c6029d0 2024-09-11T23:18:46.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.161+0000 7f2f837fe640 1 --2- 172.21.15.26:0/2871261847 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563b4c6029d0 0x563b4c602db0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f2f78013c00 tx=0x7f2f78009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c784e96d2fe19da7 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:18:46.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.161+0000 7f2f88ef7640 1 --2- 172.21.15.26:0/2356773171 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f2f6c0a2d80 0x7f2f6c0a5240 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f2f7c0cce40 tx=0x7f2f7c0054d0 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:18:46.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.161+0000 7f2f6a7fc640 1 -- 172.21.15.26:0/2871261847 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f78013130 con 0x563b4c6029d0 2024-09-11T23:18:46.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.161+0000 7f2f6a7fc640 1 -- 172.21.15.26:0/2871261847 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2f7800fc50 con 0x563b4c6029d0 2024-09-11T23:18:46.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.161+0000 7f2f6a7fc640 1 -- 172.21.15.26:0/2871261847 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f78005520 con 0x563b4c6029d0 2024-09-11T23:18:46.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.162+0000 7f2f8cd33740 1 -- 172.21.15.26:0/2871261847 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563b4c6029d0 msgr2=0x563b4c602db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:46.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.162+0000 7f2f8cd33740 1 --2- 172.21.15.26:0/2871261847 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563b4c6029d0 0x563b4c602db0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f2f78013c00 tx=0x7f2f78009c00 comp rx=0 tx=0).stop 2024-09-11T23:18:46.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.162+0000 7f2f8cd33740 1 -- 172.21.15.26:0/2871261847 shutdown_connections 2024-09-11T23:18:46.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.162+0000 7f2f8cd33740 1 --2- 172.21.15.26:0/2871261847 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563b4c6032f0 0x563b4c60d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:46.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.162+0000 7f2f8cd33740 1 --2- 172.21.15.26:0/2871261847 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563b4c60dbb0 0x563b4c60ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:46.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.162+0000 7f2f8cd33740 1 --2- 172.21.15.26:0/2871261847 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563b4c6029d0 0x563b4c602db0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:46.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.162+0000 7f2f8cd33740 1 -- 172.21.15.26:0/2871261847 >> 172.21.15.26:0/2871261847 conn(0x563b4c5fef60 msgr2=0x563b4c5ff340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:18:46.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.162+0000 7f2f8cd33740 1 -- 172.21.15.26:0/2871261847 shutdown_connections 2024-09-11T23:18:46.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.162+0000 7f2f8cd33740 1 -- 172.21.15.26:0/2871261847 wait complete. 2024-09-11T23:18:46.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.163+0000 7f2f8cd33740 1 -- start start 2024-09-11T23:18:46.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.163+0000 7f2f8cd33740 1 Processor -- start 2024-09-11T23:18:46.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.163+0000 7f2f8cd33740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563b4c6029d0 0x563b4c60d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:46.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.163+0000 7f2f8cd33740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563b4c6032f0 0x563b4c60a750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:46.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.163+0000 7f2f8cd33740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563b4c60dbb0 0x563b4c60ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:46.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.163+0000 7f2f8cd33740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x563b4c610780 con 0x563b4c6029d0 2024-09-11T23:18:46.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.163+0000 7f2f8cd33740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x563b4c523f20 con 0x563b4c60dbb0 2024-09-11T23:18:46.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.163+0000 7f2f8cd33740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x563b4c610900 con 0x563b4c6032f0 2024-09-11T23:18:46.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.163+0000 7f2f8cd33740 10 client.0 _ll_register_callbacks cb 0x563b4c4ed030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:18:46.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.163+0000 7f2f837fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563b4c6029d0 0x563b4c60d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:46.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.163+0000 7f2f837fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563b4c6029d0 0x563b4c60d400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46380/0 (socket says 172.21.15.26:46380) 2024-09-11T23:18:46.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.163+0000 7f2f837fe640 1 -- 172.21.15.26:0/2971648867 learned_addr learned my addr 172.21.15.26:0/2971648867 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:18:46.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.163+0000 7f2f83fff640 1 --2- 172.21.15.26:0/2971648867 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563b4c6032f0 0x563b4c60a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:46.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.163+0000 7f2f837fe640 1 --2- 172.21.15.26:0/2971648867 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563b4c60dbb0 0x563b4c60ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:46.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.164+0000 7f2f837fe640 1 -- 172.21.15.26:0/2971648867 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563b4c6032f0 msgr2=0x563b4c60a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:18:46.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.164+0000 7f2f837fe640 1 --2- 172.21.15.26:0/2971648867 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563b4c6032f0 0x563b4c60a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:46.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.164+0000 7f2f837fe640 1 -- 172.21.15.26:0/2971648867 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563b4c60dbb0 msgr2=0x563b4c60ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:18:46.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.164+0000 7f2f837fe640 1 --2- 172.21.15.26:0/2971648867 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563b4c60dbb0 0x563b4c60ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:46.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.164+0000 7f2f837fe640 1 -- 172.21.15.26:0/2971648867 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b4c2da2b0 con 0x563b4c6029d0 2024-09-11T23:18:46.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.164+0000 7f2f83fff640 1 --2- 172.21.15.26:0/2971648867 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563b4c6032f0 0x563b4c60a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:18:46.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.164+0000 7f2f837fe640 1 --2- 172.21.15.26:0/2971648867 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563b4c6029d0 0x563b4c60d400 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f2f7800fc20 tx=0x7f2f78009c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4880f14ff0c30218 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:18:46.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.164+0000 7f2f637fe640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:18:46.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.164+0000 7f2f637fe640 1 -- 172.21.15.26:0/2971648867 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f780046c0 con 0x563b4c6029d0 2024-09-11T23:18:46.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.164+0000 7f2f637fe640 1 -- 172.21.15.26:0/2971648867 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f2f78004860 con 0x563b4c6029d0 2024-09-11T23:18:46.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.164+0000 7f2f8cd33740 1 -- 172.21.15.26:0/2971648867 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2f7800fc50 con 0x563b4c6029d0 2024-09-11T23:18:46.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.164+0000 7f2f637fe640 1 -- 172.21.15.26:0/2971648867 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2f78004a60 con 0x563b4c6029d0 2024-09-11T23:18:46.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.164+0000 7f2f8cd33740 10 client.15435 fetch_fsmap learned FSMap version 29 2024-09-11T23:18:46.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.164+0000 7f2f8cd33740 1 -- 172.21.15.26:0/2971648867 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563b4c61c050 con 0x563b4c6029d0 2024-09-11T23:18:46.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.164+0000 7f2f637fe640 1 -- 172.21.15.26:0/2971648867 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2f7800fc50 con 0x563b4c6029d0 2024-09-11T23:18:46.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.165+0000 7f2f637fe640 1 -- 172.21.15.26:0/2971648867 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f2f78010430 con 0x563b4c6029d0 2024-09-11T23:18:46.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.165+0000 7f2f8cd33740 10 client.15435 fetch_fsmap finished waiting for FSMap version 29 2024-09-11T23:18:46.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.165+0000 7f2f8cd33740 10 client.15435 resolve_mds: resolved name 'e' to daemon mds.e 2024-09-11T23:18:46.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.165+0000 7f2f8cd33740 20 client.15435 populate_metadata read hostname 'smithi026' 2024-09-11T23:18:46.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.165+0000 7f2f8cd33740 1 --2- 172.21.15.26:0/2971648867 >> [v2:172.21.15.76:6834/393308499,v1:172.21.15.76:6835/393308499] conn(0x563b4c61e2c0 0x563b4c61f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:46.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.165+0000 7f2f8cd33740 4 client.15435 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:18:46.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.165+0000 7f2f8cd33740 1 -- 172.21.15.26:0/2971648867 --> [v2:172.21.15.76:6834/393308499,v1:172.21.15.76:6835/393308499] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563b4c576600 con 0x563b4c61e2c0 2024-09-11T23:18:46.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.165+0000 7f2f83fff640 1 --2- 172.21.15.26:0/2971648867 >> [v2:172.21.15.76:6834/393308499,v1:172.21.15.76:6835/393308499] conn(0x563b4c61e2c0 0x563b4c61f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:46.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.165+0000 7f2f83fff640 1 --2- 172.21.15.26:0/2971648867 >> [v2:172.21.15.76:6834/393308499,v1:172.21.15.76:6835/393308499] conn(0x563b4c61e2c0 0x563b4c61f730 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=dcd51988e1adef9c server_cookie=36e952c544cdb937 in_seq=0 out_seq=0 2024-09-11T23:18:46.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.165+0000 7f2f637fe640 10 client.15435 ms_handle_connect on v2:172.21.15.76:6834/393308499 2024-09-11T23:18:46.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.166+0000 7f2f637fe640 1 -- 172.21.15.26:0/2971648867 <== mds.4 v2:172.21.15.76:6834/393308499 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563b4c576600 con 0x563b4c61e2c0 2024-09-11T23:18:46.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.166+0000 7f2f637fe640 10 client.15435 handle_command_reply: tid=0 2024-09-11T23:18:46.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.166+0000 7f2f8cd33740 1 client.15435 shutdown 2024-09-11T23:18:46.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.166+0000 7f2f8cd33740 1 -- 172.21.15.26:0/2971648867 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563b4c6029d0 msgr2=0x563b4c60d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:46.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.166+0000 7f2f8cd33740 1 --2- 172.21.15.26:0/2971648867 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563b4c6029d0 0x563b4c60d400 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f2f7800fc20 tx=0x7f2f78009c70 comp rx=0 tx=0).stop 2024-09-11T23:18:46.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.166+0000 7f2f8cd33740 1 -- 172.21.15.26:0/2971648867 shutdown_connections 2024-09-11T23:18:46.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.166+0000 7f2f8cd33740 1 --2- 172.21.15.26:0/2971648867 >> [v2:172.21.15.76:6834/393308499,v1:172.21.15.76:6835/393308499] conn(0x563b4c61e2c0 0x563b4c61f730 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:46.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.166+0000 7f2f637fe640 0 client.15435 ms_handle_reset on v2:172.21.15.76:6834/393308499 2024-09-11T23:18:46.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.166+0000 7f2f8cd33740 1 --2- 172.21.15.26:0/2971648867 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563b4c60dbb0 0x563b4c60ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:46.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.166+0000 7f2f8cd33740 1 --2- 172.21.15.26:0/2971648867 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563b4c6032f0 0x563b4c60a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:46.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.166+0000 7f2f8cd33740 1 --2- 172.21.15.26:0/2971648867 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563b4c6029d0 0x563b4c60d400 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:46.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.167+0000 7f2f8cd33740 1 -- 172.21.15.26:0/2971648867 >> 172.21.15.26:0/2971648867 conn(0x563b4c5fef60 msgr2=0x563b4c604c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:18:46.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.167+0000 7f2f8cd33740 1 -- 172.21.15.26:0/2971648867 shutdown_connections 2024-09-11T23:18:46.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.167+0000 7f2f8cd33740 1 -- 172.21.15.26:0/2971648867 wait complete. 2024-09-11T23:18:46.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.167+0000 7f2f8cd33740 20 client.15435 trim_cache size 0 max 16384 2024-09-11T23:18:46.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.185+0000 7f2f8cd33740 1 Processor -- start 2024-09-11T23:18:46.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.185+0000 7f2f8cd33740 1 -- start start 2024-09-11T23:18:46.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.185+0000 7f2f8cd33740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563b4c6029d0 0x563b4c61f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:46.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.185+0000 7f2f8cd33740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563b4c6032f0 0x563b4c5f9e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:46.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.185+0000 7f2f8cd33740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563b4c60dbb0 0x563b4c60a710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:46.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.185+0000 7f2f8cd33740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x563b4c61da70 con 0x563b4c6032f0 2024-09-11T23:18:46.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.185+0000 7f2f8cd33740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x563b4c61e140 con 0x563b4c6029d0 2024-09-11T23:18:46.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.185+0000 7f2f8cd33740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x563b4c61e810 con 0x563b4c60dbb0 2024-09-11T23:18:46.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.185+0000 7f2f83fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563b4c6032f0 0x563b4c5f9e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:46.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.185+0000 7f2f83fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563b4c6032f0 0x563b4c5f9e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46382/0 (socket says 172.21.15.26:46382) 2024-09-11T23:18:46.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.185+0000 7f2f83fff640 1 -- 172.21.15.26:0/2290094845 learned_addr learned my addr 172.21.15.26:0/2290094845 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:18:46.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.185+0000 7f2f837fe640 1 --2- 172.21.15.26:0/2290094845 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563b4c60dbb0 0x563b4c60a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:46.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.185+0000 7f2f83fff640 1 -- 172.21.15.26:0/2290094845 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563b4c60dbb0 msgr2=0x563b4c60a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:18:46.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.185+0000 7f2f83fff640 1 --2- 172.21.15.26:0/2290094845 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563b4c60dbb0 0x563b4c60a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:46.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.185+0000 7f2f83fff640 1 -- 172.21.15.26:0/2290094845 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563b4c6029d0 msgr2=0x563b4c61f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:18:46.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.185+0000 7f2f83fff640 1 --2- 172.21.15.26:0/2290094845 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563b4c6029d0 0x563b4c61f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:46.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.185+0000 7f2f83fff640 1 -- 172.21.15.26:0/2290094845 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b4c610890 con 0x563b4c6032f0 2024-09-11T23:18:46.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.185+0000 7f2f837fe640 1 --2- 172.21.15.26:0/2290094845 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563b4c60dbb0 0x563b4c60a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:18:46.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.185+0000 7f2f83fff640 1 --2- 172.21.15.26:0/2290094845 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563b4c6032f0 0x563b4c5f9e40 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f2f74009900 tx=0x7f2f74004410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fed53f80c927fad server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:18:46.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.186+0000 7f2f69ffb640 1 -- 172.21.15.26:0/2290094845 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f7404a020 con 0x563b4c6032f0 2024-09-11T23:18:46.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.186+0000 7f2f69ffb640 1 -- 172.21.15.26:0/2290094845 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2f740047a0 con 0x563b4c6032f0 2024-09-11T23:18:46.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.186+0000 7f2f69ffb640 1 -- 172.21.15.26:0/2290094845 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f74035040 con 0x563b4c6032f0 2024-09-11T23:18:46.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.186+0000 7f2f8cd33740 1 -- 172.21.15.26:0/2290094845 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563b4c6032f0 msgr2=0x563b4c5f9e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:46.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.186+0000 7f2f8cd33740 1 --2- 172.21.15.26:0/2290094845 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563b4c6032f0 0x563b4c5f9e40 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f2f74009900 tx=0x7f2f74004410 comp rx=0 tx=0).stop 2024-09-11T23:18:46.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.186+0000 7f2f8cd33740 1 -- 172.21.15.26:0/2290094845 shutdown_connections 2024-09-11T23:18:46.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.186+0000 7f2f8cd33740 1 --2- 172.21.15.26:0/2290094845 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563b4c60dbb0 0x563b4c60a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:46.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.186+0000 7f2f8cd33740 1 --2- 172.21.15.26:0/2290094845 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563b4c6032f0 0x563b4c5f9e40 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:46.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.186+0000 7f2f8cd33740 1 --2- 172.21.15.26:0/2290094845 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563b4c6029d0 0x563b4c61f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:46.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.186+0000 7f2f8cd33740 1 -- 172.21.15.26:0/2290094845 >> 172.21.15.26:0/2290094845 conn(0x563b4c5fef60 msgr2=0x563b4c609060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:18:46.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.186+0000 7f2f8cd33740 1 -- 172.21.15.26:0/2290094845 shutdown_connections 2024-09-11T23:18:46.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.187+0000 7f2f8cd33740 1 -- 172.21.15.26:0/2290094845 wait complete. 2024-09-11T23:18:46.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.187+0000 7f2f8cd33740 1 -- start start 2024-09-11T23:18:46.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.187+0000 7f2f8cd33740 1 Processor -- start 2024-09-11T23:18:46.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.187+0000 7f2f8cd33740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563b4c6029d0 0x563b4c622a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:46.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.187+0000 7f2f8cd33740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563b4c6032f0 0x563b4c622fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:46.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.187+0000 7f2f8cd33740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563b4c60dbb0 0x563b4c6234e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:46.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.187+0000 7f2f8cd33740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x563b4c61f5b0 con 0x563b4c6029d0 2024-09-11T23:18:46.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.187+0000 7f2f8cd33740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x563b4c61eee0 con 0x563b4c60dbb0 2024-09-11T23:18:46.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.187+0000 7f2f8cd33740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x563b4c61fc80 con 0x563b4c6032f0 2024-09-11T23:18:46.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.187+0000 7f2f8cd33740 10 client.0 _ll_register_callbacks cb 0x7f2f74009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:18:46.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.188+0000 7f2f837fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563b4c6029d0 0x563b4c622a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:46.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.188+0000 7f2f837fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563b4c6029d0 0x563b4c622a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46390/0 (socket says 172.21.15.26:46390) 2024-09-11T23:18:46.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.188+0000 7f2f837fe640 1 -- 172.21.15.26:0/1411662249 learned_addr learned my addr 172.21.15.26:0/1411662249 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:18:46.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.188+0000 7f2f83fff640 1 --2- 172.21.15.26:0/1411662249 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563b4c6032f0 0x563b4c622fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:46.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.188+0000 7f2f837fe640 1 -- 172.21.15.26:0/1411662249 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563b4c6032f0 msgr2=0x563b4c622fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:18:46.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.188+0000 7f2f837fe640 1 --2- 172.21.15.26:0/1411662249 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563b4c6032f0 0x563b4c622fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:46.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.188+0000 7f2f837fe640 1 -- 172.21.15.26:0/1411662249 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563b4c60dbb0 msgr2=0x563b4c6234e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:18:46.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.188+0000 7f2f837fe640 1 --2- 172.21.15.26:0/1411662249 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563b4c60dbb0 0x563b4c6234e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:46.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.188+0000 7f2f837fe640 1 -- 172.21.15.26:0/1411662249 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b4c686c60 con 0x563b4c6029d0 2024-09-11T23:18:46.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.188+0000 7f2f83fff640 1 --2- 172.21.15.26:0/1411662249 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563b4c6032f0 0x563b4c622fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:18:46.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.188+0000 7f2f837fe640 1 --2- 172.21.15.26:0/1411662249 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563b4c6029d0 0x563b4c622a60 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f2f78010830 tx=0x7f2f7800fc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a20872cd2932c9a7 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:18:46.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.188+0000 7f2f637fe640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:18:46.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.188+0000 7f2f637fe640 1 -- 172.21.15.26:0/1411662249 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f78010640 con 0x563b4c6029d0 2024-09-11T23:18:46.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.188+0000 7f2f637fe640 1 -- 172.21.15.26:0/1411662249 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f2f78004c00 con 0x563b4c6029d0 2024-09-11T23:18:46.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.188+0000 7f2f8cd33740 1 -- 172.21.15.26:0/1411662249 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2f740047a0 con 0x563b4c6029d0 2024-09-11T23:18:46.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.188+0000 7f2f637fe640 1 -- 172.21.15.26:0/1411662249 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2f780136c0 con 0x563b4c6029d0 2024-09-11T23:18:46.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.188+0000 7f2f8cd33740 10 client.15441 fetch_fsmap learned FSMap version 29 2024-09-11T23:18:46.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.188+0000 7f2f8cd33740 1 -- 172.21.15.26:0/1411662249 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563b4c687ed0 con 0x563b4c6029d0 2024-09-11T23:18:46.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.189+0000 7f2f637fe640 1 -- 172.21.15.26:0/1411662249 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2f740047a0 con 0x563b4c6029d0 2024-09-11T23:18:46.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.189+0000 7f2f637fe640 1 -- 172.21.15.26:0/1411662249 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f2f78004040 con 0x563b4c6029d0 2024-09-11T23:18:46.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.189+0000 7f2f8cd33740 10 client.15441 fetch_fsmap finished waiting for FSMap version 29 2024-09-11T23:18:46.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.189+0000 7f2f8cd33740 10 client.15441 resolve_mds: resolved name 'e' to daemon mds.e 2024-09-11T23:18:46.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.189+0000 7f2f8cd33740 20 client.15441 populate_metadata read hostname 'smithi026' 2024-09-11T23:18:46.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.189+0000 7f2f8cd33740 1 --2- 172.21.15.26:0/1411662249 >> [v2:172.21.15.76:6834/393308499,v1:172.21.15.76:6835/393308499] conn(0x563b4c61e2c0 0x563b4c68b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:46.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.189+0000 7f2f8cd33740 4 client.15441 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-11T23:18:46.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.189+0000 7f2f8cd33740 1 -- 172.21.15.26:0/1411662249 --> [v2:172.21.15.76:6834/393308499,v1:172.21.15.76:6835/393308499] -- command(tid 0: {"prefix": "status"}) -- 0x563b4c570b10 con 0x563b4c61e2c0 2024-09-11T23:18:46.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.189+0000 7f2f83fff640 1 --2- 172.21.15.26:0/1411662249 >> [v2:172.21.15.76:6834/393308499,v1:172.21.15.76:6835/393308499] conn(0x563b4c61e2c0 0x563b4c68b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:46.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.190+0000 7f2f83fff640 1 --2- 172.21.15.26:0/1411662249 >> [v2:172.21.15.76:6834/393308499,v1:172.21.15.76:6835/393308499] conn(0x563b4c61e2c0 0x563b4c68b7a0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=da980bf5fe15c7d2 server_cookie=d4c7d20bc2424638 in_seq=0 out_seq=0 2024-09-11T23:18:46.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.190+0000 7f2f637fe640 10 client.15441 ms_handle_connect on v2:172.21.15.76:6834/393308499 2024-09-11T23:18:46.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.190+0000 7f2f637fe640 1 -- 172.21.15.26:0/1411662249 <== mds.4 v2:172.21.15.76:6834/393308499 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x563b4c570b10 con 0x563b4c61e2c0 2024-09-11T23:18:46.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.190+0000 7f2f637fe640 10 client.15441 handle_command_reply: tid=0 2024-09-11T23:18:46.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.190+0000 7f2f8cd33740 1 client.15441 shutdown 2024-09-11T23:18:46.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.190+0000 7f2f8cd33740 1 -- 172.21.15.26:0/1411662249 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563b4c6029d0 msgr2=0x563b4c622a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:46.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.190+0000 7f2f8cd33740 1 --2- 172.21.15.26:0/1411662249 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563b4c6029d0 0x563b4c622a60 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f2f78010830 tx=0x7f2f7800fc50 comp rx=0 tx=0).stop 2024-09-11T23:18:46.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.190+0000 7f2f8cd33740 1 -- 172.21.15.26:0/1411662249 shutdown_connections 2024-09-11T23:18:46.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.190+0000 7f2f8cd33740 1 --2- 172.21.15.26:0/1411662249 >> [v2:172.21.15.76:6834/393308499,v1:172.21.15.76:6835/393308499] conn(0x563b4c61e2c0 0x563b4c68b7a0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:46.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.190+0000 7f2f8cd33740 1 --2- 172.21.15.26:0/1411662249 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563b4c60dbb0 0x563b4c6234e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:46.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.190+0000 7f2f8cd33740 1 --2- 172.21.15.26:0/1411662249 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563b4c6032f0 0x563b4c622fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:46.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.190+0000 7f2f8cd33740 1 --2- 172.21.15.26:0/1411662249 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563b4c6029d0 0x563b4c622a60 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:46.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.190+0000 7f2f8cd33740 1 -- 172.21.15.26:0/1411662249 >> 172.21.15.26:0/1411662249 conn(0x563b4c5fef60 msgr2=0x563b4c60a710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:18:46.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.190+0000 7f2f637fe640 0 client.15441 ms_handle_reset on v2:172.21.15.76:6834/393308499 2024-09-11T23:18:46.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.190+0000 7f2f8cd33740 1 -- 172.21.15.26:0/1411662249 shutdown_connections 2024-09-11T23:18:46.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.190+0000 7f2f8cd33740 1 -- 172.21.15.26:0/1411662249 wait complete. 2024-09-11T23:18:46.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.190+0000 7f2f8cd33740 20 client.15441 trim_cache size 0 max 16384 2024-09-11T23:18:46.196 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-11T23:18:46.196 INFO:teuthology.orchestra.run.smithi026.stdout: "cluster_fsid": "af7e6fc8-7092-11ef-bcea-c7b262605968", 2024-09-11T23:18:46.196 INFO:teuthology.orchestra.run.smithi026.stdout: "whoami": 4, 2024-09-11T23:18:46.196 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 24443, 2024-09-11T23:18:46.196 INFO:teuthology.orchestra.run.smithi026.stdout: "want_state": "up:active", 2024-09-11T23:18:46.196 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "up:active", 2024-09-11T23:18:46.196 INFO:teuthology.orchestra.run.smithi026.stdout: "fs_name": "cephfs", 2024-09-11T23:18:46.197 INFO:teuthology.orchestra.run.smithi026.stdout: "rank_uptime": 128.22262165699999, 2024-09-11T23:18:46.197 INFO:teuthology.orchestra.run.smithi026.stdout: "mdsmap_epoch": 29, 2024-09-11T23:18:46.197 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch": 86, 2024-09-11T23:18:46.197 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch_barrier": 85, 2024-09-11T23:18:46.197 INFO:teuthology.orchestra.run.smithi026.stdout: "uptime": 170.191787578 2024-09-11T23:18:46.197 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-11T23:18:46.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.191+0000 7f2f8a95a640 1 -- 172.21.15.26:0/2356773171 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f2f6c0a2d80 msgr2=0x7f2f6c0a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:46.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.191+0000 7f2f8a95a640 1 --2- 172.21.15.26:0/2356773171 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f2f6c0a2d80 0x7f2f6c0a5240 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f2f7c0cce40 tx=0x7f2f7c0054d0 comp rx=0 tx=0).stop 2024-09-11T23:18:46.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.191+0000 7f2f8a95a640 1 -- 172.21.15.26:0/2356773171 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2f84102ef0 msgr2=0x7f2f84199f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:46.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.191+0000 7f2f8a95a640 1 --2- 172.21.15.26:0/2356773171 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2f84102ef0 0x7f2f84199f20 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f2f7c00eaa0 tx=0x7f2f7c00ef60 comp rx=0 tx=0).stop 2024-09-11T23:18:46.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.191+0000 7f2f8a95a640 1 -- 172.21.15.26:0/2356773171 shutdown_connections 2024-09-11T23:18:46.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.191+0000 7f2f8a95a640 1 --2- 172.21.15.26:0/2356773171 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f2f6c0a2d80 0x7f2f6c0a5240 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:46.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.191+0000 7f2f8a95a640 1 --2- 172.21.15.26:0/2356773171 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2f84102570 0x7f2f841999a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:46.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.191+0000 7f2f8a95a640 1 --2- 172.21.15.26:0/2356773171 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2f84102ef0 0x7f2f84199f20 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:46.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.191+0000 7f2f8a95a640 1 --2- 172.21.15.26:0/2356773171 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2f84100e20 0x7f2f84199460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:46.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.191+0000 7f2f8a95a640 1 -- 172.21.15.26:0/2356773171 >> 172.21.15.26:0/2356773171 conn(0x7f2f840fc7d0 msgr2=0x7f2f840febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:18:46.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.191+0000 7f2f8a95a640 1 -- 172.21.15.26:0/2356773171 shutdown_connections 2024-09-11T23:18:46.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:46.191+0000 7f2f8a95a640 1 -- 172.21.15.26:0/2356773171 wait complete. 2024-09-11T23:18:46.204 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:45 smithi076 ceph-mon[35296]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 76 KiB/s wr, 44 op/s 2024-09-11T23:18:46.904 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph tell mds.c status 2024-09-11T23:18:47.161 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:18:47.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.614+0000 7fe2fa8da640 1 -- 172.21.15.26:0/524931648 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe2f4103660 msgr2=0x7fe2f410bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:47.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.614+0000 7fe2fa8da640 1 --2- 172.21.15.26:0/524931648 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe2f4103660 0x7fe2f410bb50 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fe2e8009900 tx=0x7fe2e802a480 comp rx=0 tx=0).stop 2024-09-11T23:18:47.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.614+0000 7fe2fa8da640 1 -- 172.21.15.26:0/524931648 shutdown_connections 2024-09-11T23:18:47.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.614+0000 7fe2fa8da640 1 --2- 172.21.15.26:0/524931648 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe2f410c170 0x7fe2f410e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:47.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.614+0000 7fe2fa8da640 1 --2- 172.21.15.26:0/524931648 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe2f4103660 0x7fe2f410bb50 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:47.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.615+0000 7fe2fa8da640 1 --2- 172.21.15.26:0/524931648 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe2f4102d40 0x7fe2f4103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:47.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.615+0000 7fe2fa8da640 1 -- 172.21.15.26:0/524931648 >> 172.21.15.26:0/524931648 conn(0x7fe2f40fc7a0 msgr2=0x7fe2f40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:18:47.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.615+0000 7fe2fa8da640 1 -- 172.21.15.26:0/524931648 shutdown_connections 2024-09-11T23:18:47.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.615+0000 7fe2fa8da640 1 -- 172.21.15.26:0/524931648 wait complete. 2024-09-11T23:18:47.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.615+0000 7fe2fa8da640 1 Processor -- start 2024-09-11T23:18:47.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.615+0000 7fe2fa8da640 1 -- start start 2024-09-11T23:18:47.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.616+0000 7fe2fa8da640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe2f4102d40 0x7fe2f419a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:47.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.616+0000 7fe2fa8da640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe2f4103660 0x7fe2f419aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:47.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.616+0000 7fe2fa8da640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe2f410c170 0x7fe2f419ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:47.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.616+0000 7fe2fa8da640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fe2f4111f70 con 0x7fe2f4102d40 2024-09-11T23:18:47.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.616+0000 7fe2fa8da640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fe2f4111df0 con 0x7fe2f410c170 2024-09-11T23:18:47.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.616+0000 7fe2fa8da640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe2f41120f0 con 0x7fe2f4103660 2024-09-11T23:18:47.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.616+0000 7fe2f3fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe2f4102d40 0x7fe2f419a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:47.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.616+0000 7fe2f3fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe2f4102d40 0x7fe2f419a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46400/0 (socket says 172.21.15.26:46400) 2024-09-11T23:18:47.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.616+0000 7fe2f3fff640 1 -- 172.21.15.26:0/3614923521 learned_addr learned my addr 172.21.15.26:0/3614923521 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:18:47.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.616+0000 7fe2f37fe640 1 --2- 172.21.15.26:0/3614923521 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe2f4103660 0x7fe2f419aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:47.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.616+0000 7fe2f8e77640 1 --2- 172.21.15.26:0/3614923521 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe2f410c170 0x7fe2f419ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:47.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.616+0000 7fe2f3fff640 1 -- 172.21.15.26:0/3614923521 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe2f4103660 msgr2=0x7fe2f419aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:47.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.616+0000 7fe2f3fff640 1 --2- 172.21.15.26:0/3614923521 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe2f4103660 0x7fe2f419aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:47.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.616+0000 7fe2f3fff640 1 -- 172.21.15.26:0/3614923521 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe2f410c170 msgr2=0x7fe2f419ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:47.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.616+0000 7fe2f3fff640 1 --2- 172.21.15.26:0/3614923521 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe2f410c170 0x7fe2f419ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:47.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.616+0000 7fe2f3fff640 1 -- 172.21.15.26:0/3614923521 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2f419f610 con 0x7fe2f4102d40 2024-09-11T23:18:47.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.616+0000 7fe2f8e77640 1 --2- 172.21.15.26:0/3614923521 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe2f410c170 0x7fe2f419ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:18:47.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.616+0000 7fe2f3fff640 1 --2- 172.21.15.26:0/3614923521 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe2f4102d40 0x7fe2f419a660 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fe2e400e9f0 tx=0x7fe2e400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:18:47.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.616+0000 7fe2f17fa640 1 -- 172.21.15.26:0/3614923521 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2e400cd80 con 0x7fe2f4102d40 2024-09-11T23:18:47.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.616+0000 7fe2fa8da640 1 -- 172.21.15.26:0/3614923521 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2f419f900 con 0x7fe2f4102d40 2024-09-11T23:18:47.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.616+0000 7fe2f17fa640 1 -- 172.21.15.26:0/3614923521 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe2e4010910 con 0x7fe2f4102d40 2024-09-11T23:18:47.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.616+0000 7fe2f17fa640 1 -- 172.21.15.26:0/3614923521 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2e40188f0 con 0x7fe2f4102d40 2024-09-11T23:18:47.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.616+0000 7fe2fa8da640 1 -- 172.21.15.26:0/3614923521 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2f41a71e0 con 0x7fe2f4102d40 2024-09-11T23:18:47.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.619+0000 7fe2f17fa640 1 -- 172.21.15.26:0/3614923521 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 24) ==== 138009+0+0 (secure 0 0 0) 0x7fe2e4010ac0 con 0x7fe2f4102d40 2024-09-11T23:18:47.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.619+0000 7fe2fccb3740 1 Processor -- start 2024-09-11T23:18:47.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.619+0000 7fe2fccb3740 1 -- start start 2024-09-11T23:18:47.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.619+0000 7fe2fccb3740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563be528e9d0 0x563be528edb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:47.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.620+0000 7fe2fccb3740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563be528f2f0 0x563be5299670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:47.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.620+0000 7fe2fccb3740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563be5299bb0 0x563be529bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:47.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.620+0000 7fe2fccb3740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x563be51f5090 con 0x563be528e9d0 2024-09-11T23:18:47.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.620+0000 7fe2fccb3740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x563be519ca70 con 0x563be528f2f0 2024-09-11T23:18:47.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.620+0000 7fe2fccb3740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x563be51f1050 con 0x563be5299bb0 2024-09-11T23:18:47.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.620+0000 7fe2f8e77640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563be528e9d0 0x563be528edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:47.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.620+0000 7fe2f8e77640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563be528e9d0 0x563be528edb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46404/0 (socket says 172.21.15.26:46404) 2024-09-11T23:18:47.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.620+0000 7fe2f8e77640 1 -- 172.21.15.26:0/1559453026 learned_addr learned my addr 172.21.15.26:0/1559453026 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:18:47.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.620+0000 7fe2f3fff640 1 --2- 172.21.15.26:0/1559453026 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563be528f2f0 0x563be5299670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:47.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.620+0000 7fe2f37fe640 1 --2- 172.21.15.26:0/1559453026 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563be5299bb0 0x563be529bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:47.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.620+0000 7fe2f8e77640 1 -- 172.21.15.26:0/1559453026 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563be5299bb0 msgr2=0x563be529bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:18:47.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.620+0000 7fe2f8e77640 1 --2- 172.21.15.26:0/1559453026 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563be5299bb0 0x563be529bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:47.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.620+0000 7fe2f17fa640 1 --2- 172.21.15.26:0/3614923521 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fe2dc0a2d80 0x7fe2dc0a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:47.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.620+0000 7fe2f8e77640 1 -- 172.21.15.26:0/1559453026 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563be528f2f0 msgr2=0x563be5299670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:18:47.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.620+0000 7fe2f8e77640 1 --2- 172.21.15.26:0/1559453026 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563be528f2f0 0x563be5299670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:47.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.620+0000 7fe2f8e77640 1 -- 172.21.15.26:0/1559453026 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563be51a9130 con 0x563be528e9d0 2024-09-11T23:18:47.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.620+0000 7fe2f17fa640 1 -- 172.21.15.26:0/3614923521 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fe2e40cfab0 con 0x7fe2f4102d40 2024-09-11T23:18:47.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.620+0000 7fe2f8e77640 1 --2- 172.21.15.26:0/1559453026 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563be528e9d0 0x563be528edb0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fe2ec009fd0 tx=0x7fe2ec010860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=daf30d9caa3cb776 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:18:47.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.620+0000 7fe2f8e77640 1 --2- 172.21.15.26:0/3614923521 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fe2dc0a2d80 0x7fe2dc0a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:47.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.620+0000 7fe2da7fc640 1 -- 172.21.15.26:0/1559453026 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2ec011860 con 0x563be528e9d0 2024-09-11T23:18:47.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.621+0000 7fe2da7fc640 1 -- 172.21.15.26:0/1559453026 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe2ec011c20 con 0x563be528e9d0 2024-09-11T23:18:47.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.621+0000 7fe2da7fc640 1 -- 172.21.15.26:0/1559453026 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2ec01a830 con 0x563be528e9d0 2024-09-11T23:18:47.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.621+0000 7fe2fccb3740 1 -- 172.21.15.26:0/1559453026 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563be528e9d0 msgr2=0x563be528edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:47.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.621+0000 7fe2f8e77640 1 --2- 172.21.15.26:0/3614923521 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fe2dc0a2d80 0x7fe2dc0a5240 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe2ec011010 tx=0x7fe2ec01af30 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:18:47.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.621+0000 7fe2fccb3740 1 --2- 172.21.15.26:0/1559453026 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563be528e9d0 0x563be528edb0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fe2ec009fd0 tx=0x7fe2ec010860 comp rx=0 tx=0).stop 2024-09-11T23:18:47.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.621+0000 7fe2fccb3740 1 -- 172.21.15.26:0/1559453026 shutdown_connections 2024-09-11T23:18:47.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.621+0000 7fe2fccb3740 1 --2- 172.21.15.26:0/1559453026 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563be528f2f0 0x563be5299670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:47.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.621+0000 7fe2fccb3740 1 --2- 172.21.15.26:0/1559453026 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563be5299bb0 0x563be529bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:47.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.621+0000 7fe2fccb3740 1 --2- 172.21.15.26:0/1559453026 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563be528e9d0 0x563be528edb0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:47.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.621+0000 7fe2fccb3740 1 -- 172.21.15.26:0/1559453026 >> 172.21.15.26:0/1559453026 conn(0x563be528af60 msgr2=0x563be528b340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:18:47.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.621+0000 7fe2fccb3740 1 -- 172.21.15.26:0/1559453026 shutdown_connections 2024-09-11T23:18:47.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.621+0000 7fe2fccb3740 1 -- 172.21.15.26:0/1559453026 wait complete. 2024-09-11T23:18:47.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.622+0000 7fe2fccb3740 1 -- start start 2024-09-11T23:18:47.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.622+0000 7fe2fccb3740 1 Processor -- start 2024-09-11T23:18:47.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.622+0000 7fe2fccb3740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563be528e9d0 0x563be52993a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:47.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.622+0000 7fe2fccb3740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563be528f2f0 0x563be52966f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:47.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.622+0000 7fe2fccb3740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563be5299bb0 0x563be5296c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:47.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.622+0000 7fe2fccb3740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x563be529c780 con 0x563be528e9d0 2024-09-11T23:18:47.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.622+0000 7fe2fccb3740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x563be51aff20 con 0x563be5299bb0 2024-09-11T23:18:47.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.622+0000 7fe2fccb3740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x563be529c900 con 0x563be528f2f0 2024-09-11T23:18:47.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.622+0000 7fe2fccb3740 10 client.0 _ll_register_callbacks cb 0x563be5179030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:18:47.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.623+0000 7fe2f3fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563be528e9d0 0x563be52993a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:47.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.623+0000 7fe2f3fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563be528e9d0 0x563be52993a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46416/0 (socket says 172.21.15.26:46416) 2024-09-11T23:18:47.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.623+0000 7fe2f37fe640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563be528f2f0 0x563be52966f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:47.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.623+0000 7fe2f3fff640 1 -- 172.21.15.26:0/1390579174 learned_addr learned my addr 172.21.15.26:0/1390579174 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:18:47.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.623+0000 7fe2f8e77640 1 --2- 172.21.15.26:0/1390579174 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563be5299bb0 0x563be5296c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:47.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.623+0000 7fe2f3fff640 1 -- 172.21.15.26:0/1390579174 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563be528f2f0 msgr2=0x563be52966f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:18:47.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.623+0000 7fe2f3fff640 1 --2- 172.21.15.26:0/1390579174 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563be528f2f0 0x563be52966f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:47.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.623+0000 7fe2f3fff640 1 -- 172.21.15.26:0/1390579174 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563be5299bb0 msgr2=0x563be5296c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:18:47.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.623+0000 7fe2f3fff640 1 --2- 172.21.15.26:0/1390579174 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563be5299bb0 0x563be5296c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:47.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.623+0000 7fe2f3fff640 1 -- 172.21.15.26:0/1390579174 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563be4f662b0 con 0x563be528e9d0 2024-09-11T23:18:47.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.623+0000 7fe2f3fff640 1 --2- 172.21.15.26:0/1390579174 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563be528e9d0 0x563be52993a0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fe2e400e9c0 tx=0x7fe2e40981a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0a638af5f1197be server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:18:47.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.623+0000 7fe2d37fe640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:18:47.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.623+0000 7fe2d37fe640 1 -- 172.21.15.26:0/1390579174 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2e409c510 con 0x563be528e9d0 2024-09-11T23:18:47.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.623+0000 7fe2d37fe640 1 -- 172.21.15.26:0/1390579174 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fe2e409b260 con 0x563be528e9d0 2024-09-11T23:18:47.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.623+0000 7fe2fccb3740 1 -- 172.21.15.26:0/1390579174 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe2ec011c20 con 0x563be528e9d0 2024-09-11T23:18:47.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.623+0000 7fe2d37fe640 1 -- 172.21.15.26:0/1390579174 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe2e409be10 con 0x563be528e9d0 2024-09-11T23:18:47.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.623+0000 7fe2fccb3740 10 client.15453 fetch_fsmap learned FSMap version 29 2024-09-11T23:18:47.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.623+0000 7fe2fccb3740 1 -- 172.21.15.26:0/1390579174 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563be52a8040 con 0x563be528e9d0 2024-09-11T23:18:47.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.623+0000 7fe2d37fe640 1 -- 172.21.15.26:0/1390579174 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe2ec011c20 con 0x563be528e9d0 2024-09-11T23:18:47.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.624+0000 7fe2d37fe640 1 -- 172.21.15.26:0/1390579174 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fe2e409acd0 con 0x563be528e9d0 2024-09-11T23:18:47.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.624+0000 7fe2fccb3740 10 client.15453 fetch_fsmap finished waiting for FSMap version 29 2024-09-11T23:18:47.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.624+0000 7fe2fccb3740 10 client.15453 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-11T23:18:47.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.624+0000 7fe2fccb3740 20 client.15453 populate_metadata read hostname 'smithi026' 2024-09-11T23:18:47.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.624+0000 7fe2fccb3740 1 --2- 172.21.15.26:0/1390579174 >> [v2:172.21.15.136:6836/1633082387,v1:172.21.15.136:6837/1633082387] conn(0x563be52aa5a0 0x563be52aba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:47.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.624+0000 7fe2fccb3740 4 client.15453 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:18:47.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.624+0000 7fe2fccb3740 1 -- 172.21.15.26:0/1390579174 --> [v2:172.21.15.136:6836/1633082387,v1:172.21.15.136:6837/1633082387] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563be5202600 con 0x563be52aa5a0 2024-09-11T23:18:47.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.624+0000 7fe2f37fe640 1 --2- 172.21.15.26:0/1390579174 >> [v2:172.21.15.136:6836/1633082387,v1:172.21.15.136:6837/1633082387] conn(0x563be52aa5a0 0x563be52aba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:47.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.624+0000 7fe2f37fe640 1 --2- 172.21.15.26:0/1390579174 >> [v2:172.21.15.136:6836/1633082387,v1:172.21.15.136:6837/1633082387] conn(0x563be52aa5a0 0x563be52aba10 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d7df1a8591163b3 server_cookie=246db9a82db94b3a in_seq=0 out_seq=0 2024-09-11T23:18:47.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.624+0000 7fe2d37fe640 10 client.15453 ms_handle_connect on v2:172.21.15.136:6836/1633082387 2024-09-11T23:18:47.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.625+0000 7fe2d37fe640 1 -- 172.21.15.26:0/1390579174 <== mds.3 v2:172.21.15.136:6836/1633082387 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563be5202600 con 0x563be52aa5a0 2024-09-11T23:18:47.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.625+0000 7fe2d37fe640 10 client.15453 handle_command_reply: tid=0 2024-09-11T23:18:47.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.625+0000 7fe2fccb3740 1 client.15453 shutdown 2024-09-11T23:18:47.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.625+0000 7fe2fccb3740 1 -- 172.21.15.26:0/1390579174 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563be528e9d0 msgr2=0x563be52993a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:47.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.625+0000 7fe2fccb3740 1 --2- 172.21.15.26:0/1390579174 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563be528e9d0 0x563be52993a0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fe2e400e9c0 tx=0x7fe2e40981a0 comp rx=0 tx=0).stop 2024-09-11T23:18:47.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.625+0000 7fe2fccb3740 1 -- 172.21.15.26:0/1390579174 shutdown_connections 2024-09-11T23:18:47.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.625+0000 7fe2fccb3740 1 --2- 172.21.15.26:0/1390579174 >> [v2:172.21.15.136:6836/1633082387,v1:172.21.15.136:6837/1633082387] conn(0x563be52aa5a0 0x563be52aba10 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:47.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.625+0000 7fe2fccb3740 1 --2- 172.21.15.26:0/1390579174 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563be5299bb0 0x563be5296c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:47.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.625+0000 7fe2fccb3740 1 --2- 172.21.15.26:0/1390579174 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563be528f2f0 0x563be52966f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:47.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.625+0000 7fe2fccb3740 1 --2- 172.21.15.26:0/1390579174 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563be528e9d0 0x563be52993a0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:47.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.625+0000 7fe2d37fe640 0 client.15453 ms_handle_reset on v2:172.21.15.136:6836/1633082387 2024-09-11T23:18:47.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.625+0000 7fe2fccb3740 1 -- 172.21.15.26:0/1390579174 >> 172.21.15.26:0/1390579174 conn(0x563be528af60 msgr2=0x563be5290c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:18:47.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.625+0000 7fe2fccb3740 1 -- 172.21.15.26:0/1390579174 shutdown_connections 2024-09-11T23:18:47.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.625+0000 7fe2fccb3740 1 -- 172.21.15.26:0/1390579174 wait complete. 2024-09-11T23:18:47.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.626+0000 7fe2fccb3740 20 client.15453 trim_cache size 0 max 16384 2024-09-11T23:18:47.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.643+0000 7fe2fccb3740 1 Processor -- start 2024-09-11T23:18:47.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.644+0000 7fe2fccb3740 1 -- start start 2024-09-11T23:18:47.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.644+0000 7fe2fccb3740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563be528e9d0 0x563be52aba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:47.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.644+0000 7fe2fccb3740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563be528f2f0 0x563be52966b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:47.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.644+0000 7fe2fccb3740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563be5299bb0 0x563be5296bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:47.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.644+0000 7fe2fccb3740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x563be52a9d50 con 0x563be528f2f0 2024-09-11T23:18:47.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.644+0000 7fe2fccb3740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x563be52aa420 con 0x563be528e9d0 2024-09-11T23:18:47.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.644+0000 7fe2fccb3740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x563be52aaaf0 con 0x563be5299bb0 2024-09-11T23:18:47.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.644+0000 7fe2f37fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563be528f2f0 0x563be52966b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:47.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.644+0000 7fe2f37fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563be528f2f0 0x563be52966b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46432/0 (socket says 172.21.15.26:46432) 2024-09-11T23:18:47.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.644+0000 7fe2f37fe640 1 -- 172.21.15.26:0/540993983 learned_addr learned my addr 172.21.15.26:0/540993983 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:18:47.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.644+0000 7fe2f3fff640 1 --2- 172.21.15.26:0/540993983 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563be528e9d0 0x563be52aba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:47.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.644+0000 7fe2f8e77640 1 --2- 172.21.15.26:0/540993983 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563be5299bb0 0x563be5296bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:47.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.644+0000 7fe2f37fe640 1 -- 172.21.15.26:0/540993983 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563be5299bb0 msgr2=0x563be5296bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:18:47.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.644+0000 7fe2f37fe640 1 --2- 172.21.15.26:0/540993983 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563be5299bb0 0x563be5296bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:47.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.644+0000 7fe2f37fe640 1 -- 172.21.15.26:0/540993983 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563be528e9d0 msgr2=0x563be52aba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:18:47.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.644+0000 7fe2f37fe640 1 --2- 172.21.15.26:0/540993983 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563be528e9d0 0x563be52aba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:47.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.644+0000 7fe2f37fe640 1 -- 172.21.15.26:0/540993983 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563be5294c40 con 0x563be528f2f0 2024-09-11T23:18:47.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.644+0000 7fe2f3fff640 1 --2- 172.21.15.26:0/540993983 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563be528e9d0 0x563be52aba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:18:47.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.644+0000 7fe2f8e77640 1 --2- 172.21.15.26:0/540993983 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563be5299bb0 0x563be5296bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:18:47.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.644+0000 7fe2f37fe640 1 --2- 172.21.15.26:0/540993983 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563be528f2f0 0x563be52966b0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fe2e802a9a0 tx=0x7fe2e80319a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7318fbd88e1a5463 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:18:47.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.645+0000 7fe2d9ffb640 1 -- 172.21.15.26:0/540993983 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2e8035040 con 0x563be528f2f0 2024-09-11T23:18:47.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.645+0000 7fe2d9ffb640 1 -- 172.21.15.26:0/540993983 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe2e8004240 con 0x563be528f2f0 2024-09-11T23:18:47.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.645+0000 7fe2d9ffb640 1 -- 172.21.15.26:0/540993983 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2e8048630 con 0x563be528f2f0 2024-09-11T23:18:47.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.645+0000 7fe2fccb3740 1 -- 172.21.15.26:0/540993983 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563be528f2f0 msgr2=0x563be52966b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:47.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.645+0000 7fe2fccb3740 1 --2- 172.21.15.26:0/540993983 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563be528f2f0 0x563be52966b0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fe2e802a9a0 tx=0x7fe2e80319a0 comp rx=0 tx=0).stop 2024-09-11T23:18:47.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.645+0000 7fe2fccb3740 1 -- 172.21.15.26:0/540993983 shutdown_connections 2024-09-11T23:18:47.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.645+0000 7fe2fccb3740 1 --2- 172.21.15.26:0/540993983 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563be5299bb0 0x563be5296bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:47.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.645+0000 7fe2fccb3740 1 --2- 172.21.15.26:0/540993983 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563be528f2f0 0x563be52966b0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:47.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.645+0000 7fe2fccb3740 1 --2- 172.21.15.26:0/540993983 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563be528e9d0 0x563be52aba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:47.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.645+0000 7fe2fccb3740 1 -- 172.21.15.26:0/540993983 >> 172.21.15.26:0/540993983 conn(0x563be528af60 msgr2=0x563be5295c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:18:47.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.645+0000 7fe2fccb3740 1 -- 172.21.15.26:0/540993983 shutdown_connections 2024-09-11T23:18:47.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.645+0000 7fe2fccb3740 1 -- 172.21.15.26:0/540993983 wait complete. 2024-09-11T23:18:47.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.646+0000 7fe2fccb3740 1 -- start start 2024-09-11T23:18:47.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.646+0000 7fe2fccb3740 1 Processor -- start 2024-09-11T23:18:47.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.646+0000 7fe2fccb3740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563be528e9d0 0x563be52af310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:47.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.646+0000 7fe2fccb3740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563be528f2f0 0x563be52af850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:47.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.646+0000 7fe2fccb3740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563be5299bb0 0x563be530cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:47.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.646+0000 7fe2fccb3740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x563be52ab890 con 0x563be5299bb0 2024-09-11T23:18:47.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.646+0000 7fe2fccb3740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x563be52ab1c0 con 0x563be528f2f0 2024-09-11T23:18:47.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.646+0000 7fe2fccb3740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x563be52abf60 con 0x563be528e9d0 2024-09-11T23:18:47.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.646+0000 7fe2fccb3740 10 client.0 _ll_register_callbacks cb 0x7fe2e40d3250 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:18:47.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.646+0000 7fe2f8e77640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563be5299bb0 0x563be530cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:47.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.646+0000 7fe2f8e77640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563be5299bb0 0x563be530cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46442/0 (socket says 172.21.15.26:46442) 2024-09-11T23:18:47.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.646+0000 7fe2f8e77640 1 -- 172.21.15.26:0/2736401207 learned_addr learned my addr 172.21.15.26:0/2736401207 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:18:47.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.646+0000 7fe2f3fff640 1 --2- 172.21.15.26:0/2736401207 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563be528e9d0 0x563be52af310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:47.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.646+0000 7fe2f37fe640 1 --2- 172.21.15.26:0/2736401207 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563be528f2f0 0x563be52af850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:47.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.647+0000 7fe2f8e77640 1 -- 172.21.15.26:0/2736401207 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563be528e9d0 msgr2=0x563be52af310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:18:47.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.647+0000 7fe2f8e77640 1 --2- 172.21.15.26:0/2736401207 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563be528e9d0 0x563be52af310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:47.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.647+0000 7fe2f8e77640 1 -- 172.21.15.26:0/2736401207 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563be528f2f0 msgr2=0x563be52af850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:18:47.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.647+0000 7fe2f8e77640 1 --2- 172.21.15.26:0/2736401207 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563be528f2f0 0x563be52af850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:47.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.647+0000 7fe2f8e77640 1 -- 172.21.15.26:0/2736401207 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563be530d2d0 con 0x563be5299bb0 2024-09-11T23:18:47.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.647+0000 7fe2f8e77640 1 --2- 172.21.15.26:0/2736401207 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563be5299bb0 0x563be530cce0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fe2ec010d60 tx=0x7fe2ec019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29ff44fe441899cc server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:18:47.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.647+0000 7fe2d37fe640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:18:47.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.647+0000 7fe2d37fe640 1 -- 172.21.15.26:0/2736401207 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2ec037070 con 0x563be5299bb0 2024-09-11T23:18:47.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.647+0000 7fe2d37fe640 1 -- 172.21.15.26:0/2736401207 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fe2ec00ece0 con 0x563be5299bb0 2024-09-11T23:18:47.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.647+0000 7fe2fccb3740 1 -- 172.21.15.26:0/2736401207 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe2e8004240 con 0x563be5299bb0 2024-09-11T23:18:47.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.647+0000 7fe2d37fe640 1 -- 172.21.15.26:0/2736401207 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe2ec043050 con 0x563be5299bb0 2024-09-11T23:18:47.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.647+0000 7fe2fccb3740 10 client.15459 fetch_fsmap learned FSMap version 29 2024-09-11T23:18:47.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.647+0000 7fe2fccb3740 1 -- 172.21.15.26:0/2736401207 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563be5314690 con 0x563be5299bb0 2024-09-11T23:18:47.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.648+0000 7fe2d37fe640 1 -- 172.21.15.26:0/2736401207 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe2e8004240 con 0x563be5299bb0 2024-09-11T23:18:47.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.648+0000 7fe2d37fe640 1 -- 172.21.15.26:0/2736401207 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fe2ec01c480 con 0x563be5299bb0 2024-09-11T23:18:47.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.648+0000 7fe2fccb3740 10 client.15459 fetch_fsmap finished waiting for FSMap version 29 2024-09-11T23:18:47.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.648+0000 7fe2fccb3740 10 client.15459 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-11T23:18:47.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.648+0000 7fe2fccb3740 20 client.15459 populate_metadata read hostname 'smithi026' 2024-09-11T23:18:47.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.648+0000 7fe2fccb3740 1 --2- 172.21.15.26:0/2736401207 >> [v2:172.21.15.136:6836/1633082387,v1:172.21.15.136:6837/1633082387] conn(0x563be5319010 0x563be5317d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:47.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.648+0000 7fe2fccb3740 4 client.15459 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-11T23:18:47.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.648+0000 7fe2fccb3740 1 -- 172.21.15.26:0/2736401207 --> [v2:172.21.15.136:6836/1633082387,v1:172.21.15.136:6837/1633082387] -- command(tid 0: {"prefix": "status"}) -- 0x563be51fcb10 con 0x563be5319010 2024-09-11T23:18:47.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.648+0000 7fe2f3fff640 1 --2- 172.21.15.26:0/2736401207 >> [v2:172.21.15.136:6836/1633082387,v1:172.21.15.136:6837/1633082387] conn(0x563be5319010 0x563be5317d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:47.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.648+0000 7fe2f3fff640 1 --2- 172.21.15.26:0/2736401207 >> [v2:172.21.15.136:6836/1633082387,v1:172.21.15.136:6837/1633082387] conn(0x563be5319010 0x563be5317d20 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=23fb9170a225443a server_cookie=cd157d8a47774abe in_seq=0 out_seq=0 2024-09-11T23:18:47.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.648+0000 7fe2d37fe640 10 client.15459 ms_handle_connect on v2:172.21.15.136:6836/1633082387 2024-09-11T23:18:47.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.649+0000 7fe2d37fe640 1 -- 172.21.15.26:0/2736401207 <== mds.3 v2:172.21.15.136:6836/1633082387 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x563be51fcb10 con 0x563be5319010 2024-09-11T23:18:47.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.649+0000 7fe2d37fe640 10 client.15459 handle_command_reply: tid=0 2024-09-11T23:18:47.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.649+0000 7fe2fccb3740 1 client.15459 shutdown 2024-09-11T23:18:47.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.649+0000 7fe2fccb3740 1 -- 172.21.15.26:0/2736401207 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563be5299bb0 msgr2=0x563be530cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:47.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.649+0000 7fe2fccb3740 1 --2- 172.21.15.26:0/2736401207 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563be5299bb0 0x563be530cce0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fe2ec010d60 tx=0x7fe2ec019000 comp rx=0 tx=0).stop 2024-09-11T23:18:47.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.649+0000 7fe2fccb3740 1 -- 172.21.15.26:0/2736401207 shutdown_connections 2024-09-11T23:18:47.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.649+0000 7fe2fccb3740 1 --2- 172.21.15.26:0/2736401207 >> [v2:172.21.15.136:6836/1633082387,v1:172.21.15.136:6837/1633082387] conn(0x563be5319010 0x563be5317d20 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:47.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.649+0000 7fe2fccb3740 1 --2- 172.21.15.26:0/2736401207 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563be528f2f0 0x563be52af850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:47.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.649+0000 7fe2fccb3740 1 --2- 172.21.15.26:0/2736401207 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563be5299bb0 0x563be530cce0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:47.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.649+0000 7fe2fccb3740 1 --2- 172.21.15.26:0/2736401207 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563be528e9d0 0x563be52af310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:47.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.649+0000 7fe2fccb3740 1 -- 172.21.15.26:0/2736401207 >> 172.21.15.26:0/2736401207 conn(0x563be528af60 msgr2=0x563be52966b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:18:47.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.649+0000 7fe2d37fe640 0 client.15459 ms_handle_reset on v2:172.21.15.136:6836/1633082387 2024-09-11T23:18:47.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.649+0000 7fe2fccb3740 1 -- 172.21.15.26:0/2736401207 shutdown_connections 2024-09-11T23:18:47.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.649+0000 7fe2fccb3740 1 -- 172.21.15.26:0/2736401207 wait complete. 2024-09-11T23:18:47.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.649+0000 7fe2fccb3740 20 client.15459 trim_cache size 0 max 16384 2024-09-11T23:18:47.661 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-11T23:18:47.661 INFO:teuthology.orchestra.run.smithi026.stdout: "cluster_fsid": "af7e6fc8-7092-11ef-bcea-c7b262605968", 2024-09-11T23:18:47.662 INFO:teuthology.orchestra.run.smithi026.stdout: "whoami": 3, 2024-09-11T23:18:47.662 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 24448, 2024-09-11T23:18:47.662 INFO:teuthology.orchestra.run.smithi026.stdout: "want_state": "up:active", 2024-09-11T23:18:47.662 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "up:active", 2024-09-11T23:18:47.662 INFO:teuthology.orchestra.run.smithi026.stdout: "fs_name": "cephfs", 2024-09-11T23:18:47.662 INFO:teuthology.orchestra.run.smithi026.stdout: "rank_uptime": 130.68632063699999, 2024-09-11T23:18:47.662 INFO:teuthology.orchestra.run.smithi026.stdout: "mdsmap_epoch": 29, 2024-09-11T23:18:47.662 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch": 86, 2024-09-11T23:18:47.662 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch_barrier": 85, 2024-09-11T23:18:47.662 INFO:teuthology.orchestra.run.smithi026.stdout: "uptime": 166.664313773 2024-09-11T23:18:47.662 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-11T23:18:47.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.650+0000 7fe2fa8da640 1 -- 172.21.15.26:0/3614923521 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fe2dc0a2d80 msgr2=0x7fe2dc0a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:47.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.650+0000 7fe2fa8da640 1 --2- 172.21.15.26:0/3614923521 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fe2dc0a2d80 0x7fe2dc0a5240 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe2ec011010 tx=0x7fe2ec01af30 comp rx=0 tx=0).stop 2024-09-11T23:18:47.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.650+0000 7fe2fa8da640 1 -- 172.21.15.26:0/3614923521 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe2f4102d40 msgr2=0x7fe2f419a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:47.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.650+0000 7fe2fa8da640 1 --2- 172.21.15.26:0/3614923521 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe2f4102d40 0x7fe2f419a660 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fe2e400e9f0 tx=0x7fe2e400eeb0 comp rx=0 tx=0).stop 2024-09-11T23:18:47.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.650+0000 7fe2fa8da640 1 -- 172.21.15.26:0/3614923521 shutdown_connections 2024-09-11T23:18:47.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.650+0000 7fe2fa8da640 1 --2- 172.21.15.26:0/3614923521 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fe2dc0a2d80 0x7fe2dc0a5240 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:47.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.650+0000 7fe2fa8da640 1 --2- 172.21.15.26:0/3614923521 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe2f410c170 0x7fe2f419ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:47.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.650+0000 7fe2fa8da640 1 --2- 172.21.15.26:0/3614923521 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe2f4103660 0x7fe2f419aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:47.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.650+0000 7fe2fa8da640 1 --2- 172.21.15.26:0/3614923521 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe2f4102d40 0x7fe2f419a660 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:47.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.650+0000 7fe2fa8da640 1 -- 172.21.15.26:0/3614923521 >> 172.21.15.26:0/3614923521 conn(0x7fe2f40fc7a0 msgr2=0x7fe2f410d490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:18:47.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.650+0000 7fe2fa8da640 1 -- 172.21.15.26:0/3614923521 shutdown_connections 2024-09-11T23:18:47.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:47.650+0000 7fe2fa8da640 1 -- 172.21.15.26:0/3614923521 wait complete. 2024-09-11T23:18:48.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:47 smithi136 ceph-mon[35121]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T23:18:48.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:47 smithi026 ceph-mon[31172]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T23:18:48.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:47 smithi076 ceph-mon[35296]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T23:18:48.349 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph tell mds.h status 2024-09-11T23:18:48.607 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:18:49.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.128+0000 7fccc1217640 1 -- 172.21.15.26:0/2156948967 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fccbc101420 msgr2=0x7fccbc10d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:49.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.128+0000 7fccc1217640 1 --2- 172.21.15.26:0/2156948967 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fccbc101420 0x7fccbc10d950 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fccb4009cd0 tx=0x7fccb402a8e0 comp rx=0 tx=0).stop 2024-09-11T23:18:49.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.128+0000 7fccc1217640 1 -- 172.21.15.26:0/2156948967 shutdown_connections 2024-09-11T23:18:49.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.128+0000 7fccc1217640 1 --2- 172.21.15.26:0/2156948967 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fccbc100a80 0x7fccbc100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:49.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.128+0000 7fccc1217640 1 --2- 172.21.15.26:0/2156948967 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fccbc101420 0x7fccbc10d950 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:49.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.128+0000 7fccc1217640 1 --2- 172.21.15.26:0/2156948967 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fccbc106a80 0x7fccbc106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:49.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.128+0000 7fccc1217640 1 -- 172.21.15.26:0/2156948967 >> 172.21.15.26:0/2156948967 conn(0x7fccbc0fc7a0 msgr2=0x7fccbc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:18:49.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.128+0000 7fccc1217640 1 -- 172.21.15.26:0/2156948967 shutdown_connections 2024-09-11T23:18:49.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.128+0000 7fccc1217640 1 -- 172.21.15.26:0/2156948967 wait complete. 2024-09-11T23:18:49.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.129+0000 7fccc1217640 1 Processor -- start 2024-09-11T23:18:49.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.129+0000 7fccc1217640 1 -- start start 2024-09-11T23:18:49.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.130+0000 7fccc1217640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fccbc100a80 0x7fccbc198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:49.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.130+0000 7fccc1217640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fccbc101420 0x7fccbc1989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:49.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.130+0000 7fccc1217640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fccbc106a80 0x7fccbc19cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:49.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.130+0000 7fccc1217640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fccbc110e80 con 0x7fccbc100a80 2024-09-11T23:18:49.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.130+0000 7fccc1217640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fccbc110d00 con 0x7fccbc106a80 2024-09-11T23:18:49.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.130+0000 7fccc1217640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fccbc111000 con 0x7fccbc101420 2024-09-11T23:18:49.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.130+0000 7fccbad9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fccbc100a80 0x7fccbc198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:49.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.130+0000 7fccbad9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fccbc100a80 0x7fccbc198470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46472/0 (socket says 172.21.15.26:46472) 2024-09-11T23:18:49.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.130+0000 7fccbad9d640 1 -- 172.21.15.26:0/3462228706 learned_addr learned my addr 172.21.15.26:0/3462228706 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:18:49.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.130+0000 7fccba59c640 1 --2- 172.21.15.26:0/3462228706 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fccbc101420 0x7fccbc1989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:49.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.130+0000 7fccbb59e640 1 --2- 172.21.15.26:0/3462228706 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fccbc106a80 0x7fccbc19cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:49.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.130+0000 7fccbad9d640 1 -- 172.21.15.26:0/3462228706 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fccbc101420 msgr2=0x7fccbc1989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:49.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.130+0000 7fccbad9d640 1 --2- 172.21.15.26:0/3462228706 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fccbc101420 0x7fccbc1989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:49.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.130+0000 7fccbad9d640 1 -- 172.21.15.26:0/3462228706 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fccbc106a80 msgr2=0x7fccbc19cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:49.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.130+0000 7fccbad9d640 1 --2- 172.21.15.26:0/3462228706 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fccbc106a80 0x7fccbc19cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:49.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.130+0000 7fccbad9d640 1 -- 172.21.15.26:0/3462228706 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccbc19d420 con 0x7fccbc100a80 2024-09-11T23:18:49.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.130+0000 7fccbb59e640 1 --2- 172.21.15.26:0/3462228706 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fccbc106a80 0x7fccbc19cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:18:49.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.130+0000 7fccbad9d640 1 --2- 172.21.15.26:0/3462228706 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fccbc100a80 0x7fccbc198470 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fccac00e9f0 tx=0x7fccac00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:18:49.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.130+0000 7fcca3fff640 1 -- 172.21.15.26:0/3462228706 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccac00cd80 con 0x7fccbc100a80 2024-09-11T23:18:49.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.130+0000 7fccc1217640 1 -- 172.21.15.26:0/3462228706 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccbc19d710 con 0x7fccbc100a80 2024-09-11T23:18:49.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.130+0000 7fcca3fff640 1 -- 172.21.15.26:0/3462228706 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fccac005500 con 0x7fccbc100a80 2024-09-11T23:18:49.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.130+0000 7fcca3fff640 1 -- 172.21.15.26:0/3462228706 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccac010690 con 0x7fccbc100a80 2024-09-11T23:18:49.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.131+0000 7fccc1217640 1 -- 172.21.15.26:0/3462228706 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fccbc1a4ff0 con 0x7fccbc100a80 2024-09-11T23:18:49.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.133+0000 7fcca3fff640 1 -- 172.21.15.26:0/3462228706 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 24) ==== 138009+0+0 (secure 0 0 0) 0x7fccac0046c0 con 0x7fccbc100a80 2024-09-11T23:18:49.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.133+0000 7fccc35f0740 1 Processor -- start 2024-09-11T23:18:49.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.133+0000 7fccc35f0740 1 -- start start 2024-09-11T23:18:49.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.133+0000 7fcca3fff640 1 --2- 172.21.15.26:0/3462228706 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fcca40a2d80 0x7fcca40a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:49.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.134+0000 7fccc35f0740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a5238529d0 0x55a523852db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:49.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.134+0000 7fccc35f0740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a5238532f0 0x55a52385d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:49.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.134+0000 7fccc35f0740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a52385dbb0 0x55a52385ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:49.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.134+0000 7fccc35f0740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55a5237b9090 con 0x55a52385dbb0 2024-09-11T23:18:49.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.134+0000 7fccc35f0740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55a523760a70 con 0x55a5238529d0 2024-09-11T23:18:49.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.134+0000 7fccc35f0740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55a5237b5050 con 0x55a5238532f0 2024-09-11T23:18:49.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.134+0000 7fccbb59e640 1 --2- 172.21.15.26:0/3462228706 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fcca40a2d80 0x7fcca40a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:49.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.134+0000 7fcca3fff640 1 -- 172.21.15.26:0/3462228706 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fccac0cfaa0 con 0x7fccbc100a80 2024-09-11T23:18:49.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.134+0000 7fccba59c640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a5238532f0 0x55a52385d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:49.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.134+0000 7fccba59c640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a5238532f0 0x55a52385d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:40396/0 (socket says 172.21.15.26:40396) 2024-09-11T23:18:49.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.134+0000 7fccba59c640 1 -- 172.21.15.26:0/1560154423 learned_addr learned my addr 172.21.15.26:0/1560154423 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:18:49.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.134+0000 7fccbb59e640 1 --2- 172.21.15.26:0/1560154423 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a52385dbb0 0x55a52385ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:49.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.134+0000 7fccbad9d640 1 --2- 172.21.15.26:0/1560154423 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a5238529d0 0x55a523852db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:49.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.134+0000 7fccba59c640 1 -- 172.21.15.26:0/1560154423 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a5238529d0 msgr2=0x55a523852db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:18:49.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.134+0000 7fccba59c640 1 --2- 172.21.15.26:0/1560154423 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a5238529d0 0x55a523852db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:49.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.134+0000 7fccba59c640 1 -- 172.21.15.26:0/1560154423 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a52385dbb0 msgr2=0x55a52385ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:18:49.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.134+0000 7fccba59c640 1 --2- 172.21.15.26:0/1560154423 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a52385dbb0 0x55a52385ffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:49.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.134+0000 7fccba59c640 1 -- 172.21.15.26:0/1560154423 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a52376d130 con 0x55a5238532f0 2024-09-11T23:18:49.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.134+0000 7fccbb59e640 1 --2- 172.21.15.26:0/3462228706 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fcca40a2d80 0x7fcca40a5240 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fccb4009a60 tx=0x7fccb40099d0 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:18:49.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.134+0000 7fccba59c640 1 --2- 172.21.15.26:0/1560154423 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a5238532f0 0x55a52385d670 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fccb000bc90 tx=0x7fccb00107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=813ba5b3e10b107d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:18:49.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.135+0000 7fcca17fa640 1 -- 172.21.15.26:0/1560154423 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccb0005220 con 0x55a5238532f0 2024-09-11T23:18:49.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.135+0000 7fcca17fa640 1 -- 172.21.15.26:0/1560154423 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fccb000be40 con 0x55a5238532f0 2024-09-11T23:18:49.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.135+0000 7fcca17fa640 1 -- 172.21.15.26:0/1560154423 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccb0011770 con 0x55a5238532f0 2024-09-11T23:18:49.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.135+0000 7fccc35f0740 1 -- 172.21.15.26:0/1560154423 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a5238532f0 msgr2=0x55a52385d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:49.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.135+0000 7fccc35f0740 1 --2- 172.21.15.26:0/1560154423 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a5238532f0 0x55a52385d670 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fccb000bc90 tx=0x7fccb00107a0 comp rx=0 tx=0).stop 2024-09-11T23:18:49.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.135+0000 7fccc35f0740 1 -- 172.21.15.26:0/1560154423 shutdown_connections 2024-09-11T23:18:49.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.135+0000 7fccc35f0740 1 --2- 172.21.15.26:0/1560154423 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a52385dbb0 0x55a52385ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:49.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.135+0000 7fccc35f0740 1 --2- 172.21.15.26:0/1560154423 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a5238532f0 0x55a52385d670 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:49.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.135+0000 7fccc35f0740 1 --2- 172.21.15.26:0/1560154423 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a5238529d0 0x55a523852db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:49.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.135+0000 7fccc35f0740 1 -- 172.21.15.26:0/1560154423 >> 172.21.15.26:0/1560154423 conn(0x55a52384ef60 msgr2=0x55a52384f340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:18:49.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.135+0000 7fccc35f0740 1 -- 172.21.15.26:0/1560154423 shutdown_connections 2024-09-11T23:18:49.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.135+0000 7fccc35f0740 1 -- 172.21.15.26:0/1560154423 wait complete. 2024-09-11T23:18:49.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.136+0000 7fccc35f0740 1 -- start start 2024-09-11T23:18:49.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.136+0000 7fccc35f0740 1 Processor -- start 2024-09-11T23:18:49.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.136+0000 7fccc35f0740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a5238529d0 0x55a52385d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:49.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.136+0000 7fccc35f0740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a5238532f0 0x55a52385a750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:49.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.136+0000 7fccc35f0740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a52385dbb0 0x55a52385ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:49.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.136+0000 7fccc35f0740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55a523860780 con 0x55a5238529d0 2024-09-11T23:18:49.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.136+0000 7fccc35f0740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55a523773f20 con 0x55a5238532f0 2024-09-11T23:18:49.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.136+0000 7fccc35f0740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55a523860900 con 0x55a52385dbb0 2024-09-11T23:18:49.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.136+0000 7fccc35f0740 10 client.0 _ll_register_callbacks cb 0x55a52373d030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:18:49.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.136+0000 7fccbad9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a5238529d0 0x55a52385d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:49.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.136+0000 7fccbad9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a5238529d0 0x55a52385d400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46482/0 (socket says 172.21.15.26:46482) 2024-09-11T23:18:49.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.136+0000 7fccbad9d640 1 -- 172.21.15.26:0/1681695199 learned_addr learned my addr 172.21.15.26:0/1681695199 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:18:49.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.136+0000 7fccbb59e640 1 --2- 172.21.15.26:0/1681695199 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a52385dbb0 0x55a52385ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:49.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.136+0000 7fccba59c640 1 --2- 172.21.15.26:0/1681695199 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a5238532f0 0x55a52385a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:49.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.137+0000 7fccbad9d640 1 -- 172.21.15.26:0/1681695199 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a52385dbb0 msgr2=0x55a52385ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:18:49.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.137+0000 7fccbad9d640 1 --2- 172.21.15.26:0/1681695199 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a52385dbb0 0x55a52385ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:49.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.137+0000 7fccbad9d640 1 -- 172.21.15.26:0/1681695199 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a5238532f0 msgr2=0x55a52385a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:18:49.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.137+0000 7fccbad9d640 1 --2- 172.21.15.26:0/1681695199 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a5238532f0 0x55a52385a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:49.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.137+0000 7fccbad9d640 1 -- 172.21.15.26:0/1681695199 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a52352a2b0 con 0x55a5238529d0 2024-09-11T23:18:49.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.137+0000 7fccbad9d640 1 --2- 172.21.15.26:0/1681695199 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a5238529d0 0x55a52385d400 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fccac00e9c0 tx=0x7fccac09ba70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acb35a3033016036 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:18:49.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.137+0000 7fcc9eff5640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:18:49.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.137+0000 7fcc9eff5640 1 -- 172.21.15.26:0/1681695199 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccac0a01b0 con 0x55a5238529d0 2024-09-11T23:18:49.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.137+0000 7fcc9eff5640 1 -- 172.21.15.26:0/1681695199 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fccac0bd230 con 0x55a5238529d0 2024-09-11T23:18:49.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.137+0000 7fccc35f0740 1 -- 172.21.15.26:0/1681695199 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a5237c6600 con 0x55a5238529d0 2024-09-11T23:18:49.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.137+0000 7fcc9eff5640 1 -- 172.21.15.26:0/1681695199 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fccac0a4a70 con 0x55a5238529d0 2024-09-11T23:18:49.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.137+0000 7fccc35f0740 10 client.15471 fetch_fsmap learned FSMap version 29 2024-09-11T23:18:49.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.137+0000 7fccc35f0740 1 -- 172.21.15.26:0/1681695199 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a52386c080 con 0x55a5238529d0 2024-09-11T23:18:49.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.137+0000 7fcc9eff5640 1 -- 172.21.15.26:0/1681695199 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55a5237c6600 con 0x55a5238529d0 2024-09-11T23:18:49.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.137+0000 7fcc9eff5640 1 -- 172.21.15.26:0/1681695199 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fccac0cd390 con 0x55a5238529d0 2024-09-11T23:18:49.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.137+0000 7fccc35f0740 10 client.15471 fetch_fsmap finished waiting for FSMap version 29 2024-09-11T23:18:49.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.138+0000 7fccc35f0740 10 client.15471 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-11T23:18:49.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.138+0000 7fccc35f0740 20 client.15471 populate_metadata read hostname 'smithi026' 2024-09-11T23:18:49.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.138+0000 7fccc35f0740 1 --2- 172.21.15.26:0/1681695199 >> [v2:172.21.15.76:6836/2078978748,v1:172.21.15.76:6837/2078978748] conn(0x55a52386e630 0x55a52386faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:49.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.138+0000 7fccc35f0740 4 client.15471 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:18:49.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.138+0000 7fccc35f0740 1 -- 172.21.15.26:0/1681695199 --> [v2:172.21.15.76:6836/2078978748,v1:172.21.15.76:6837/2078978748] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a5237c0b10 con 0x55a52386e630 2024-09-11T23:18:49.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.138+0000 7fccba59c640 1 --2- 172.21.15.26:0/1681695199 >> [v2:172.21.15.76:6836/2078978748,v1:172.21.15.76:6837/2078978748] conn(0x55a52386e630 0x55a52386faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:49.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.138+0000 7fcc9eff5640 10 client.15471 ms_handle_connect on v2:172.21.15.76:6836/2078978748 2024-09-11T23:18:49.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.138+0000 7fccba59c640 1 --2- 172.21.15.26:0/1681695199 >> [v2:172.21.15.76:6836/2078978748,v1:172.21.15.76:6837/2078978748] conn(0x55a52386e630 0x55a52386faa0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9ca0da0e9617218b server_cookie=fc2a628994b0eeb0 in_seq=0 out_seq=0 2024-09-11T23:18:49.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.139+0000 7fcc9eff5640 1 -- 172.21.15.26:0/1681695199 <== mds.2 v2:172.21.15.76:6836/2078978748 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a5237c0b10 con 0x55a52386e630 2024-09-11T23:18:49.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.139+0000 7fcc9eff5640 10 client.15471 handle_command_reply: tid=0 2024-09-11T23:18:49.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.139+0000 7fccc35f0740 1 client.15471 shutdown 2024-09-11T23:18:49.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.139+0000 7fccc35f0740 1 -- 172.21.15.26:0/1681695199 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a5238529d0 msgr2=0x55a52385d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:49.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.139+0000 7fccc35f0740 1 --2- 172.21.15.26:0/1681695199 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a5238529d0 0x55a52385d400 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fccac00e9c0 tx=0x7fccac09ba70 comp rx=0 tx=0).stop 2024-09-11T23:18:49.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.139+0000 7fccc35f0740 1 -- 172.21.15.26:0/1681695199 shutdown_connections 2024-09-11T23:18:49.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.139+0000 7fccc35f0740 1 --2- 172.21.15.26:0/1681695199 >> [v2:172.21.15.76:6836/2078978748,v1:172.21.15.76:6837/2078978748] conn(0x55a52386e630 0x55a52386faa0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:49.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.139+0000 7fccc35f0740 1 --2- 172.21.15.26:0/1681695199 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a5238532f0 0x55a52385a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:49.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.139+0000 7fccc35f0740 1 --2- 172.21.15.26:0/1681695199 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a52385dbb0 0x55a52385ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:49.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.139+0000 7fcc9eff5640 0 client.15471 ms_handle_reset on v2:172.21.15.76:6836/2078978748 2024-09-11T23:18:49.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.139+0000 7fccc35f0740 1 --2- 172.21.15.26:0/1681695199 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a5238529d0 0x55a52385d400 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:49.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.139+0000 7fccc35f0740 1 -- 172.21.15.26:0/1681695199 >> 172.21.15.26:0/1681695199 conn(0x55a52384ef60 msgr2=0x55a523854c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:18:49.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.140+0000 7fccc35f0740 1 -- 172.21.15.26:0/1681695199 shutdown_connections 2024-09-11T23:18:49.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.140+0000 7fccc35f0740 1 -- 172.21.15.26:0/1681695199 wait complete. 2024-09-11T23:18:49.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.140+0000 7fccc35f0740 20 client.15471 trim_cache size 0 max 16384 2024-09-11T23:18:49.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.158+0000 7fccc35f0740 1 Processor -- start 2024-09-11T23:18:49.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.158+0000 7fccc35f0740 1 -- start start 2024-09-11T23:18:49.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.158+0000 7fccc35f0740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a5238529d0 0x55a52386faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:49.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.158+0000 7fccc35f0740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a5238532f0 0x55a523849ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:49.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.158+0000 7fccc35f0740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a52385dbb0 0x55a523859020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:49.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.158+0000 7fccc35f0740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55a52386dde0 con 0x55a52385dbb0 2024-09-11T23:18:49.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.159+0000 7fccc35f0740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55a52386e4b0 con 0x55a5238529d0 2024-09-11T23:18:49.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.159+0000 7fccc35f0740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55a52386eb80 con 0x55a5238532f0 2024-09-11T23:18:49.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.159+0000 7fccbb59e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a52385dbb0 0x55a523859020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:49.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.159+0000 7fccba59c640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a5238532f0 0x55a523849ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:49.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.159+0000 7fccba59c640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a5238532f0 0x55a523849ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:40426/0 (socket says 172.21.15.26:40426) 2024-09-11T23:18:49.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.159+0000 7fccbb59e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a52385dbb0 0x55a523859020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46494/0 (socket says 172.21.15.26:46494) 2024-09-11T23:18:49.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.159+0000 7fccba59c640 1 -- 172.21.15.26:0/301943904 learned_addr learned my addr 172.21.15.26:0/301943904 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:18:49.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.159+0000 7fccbad9d640 1 --2- 172.21.15.26:0/301943904 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a5238529d0 0x55a52386faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:49.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.159+0000 7fccba59c640 1 -- 172.21.15.26:0/301943904 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a5238529d0 msgr2=0x55a52386faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:18:49.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.159+0000 7fccba59c640 1 --2- 172.21.15.26:0/301943904 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a5238529d0 0x55a52386faa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:49.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.159+0000 7fccba59c640 1 -- 172.21.15.26:0/301943904 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a52385dbb0 msgr2=0x55a523859020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:18:49.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.159+0000 7fccba59c640 1 --2- 172.21.15.26:0/301943904 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a52385dbb0 0x55a523859020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:49.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.159+0000 7fccba59c640 1 -- 172.21.15.26:0/301943904 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5238500d0 con 0x55a5238532f0 2024-09-11T23:18:49.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.159+0000 7fccba59c640 1 --2- 172.21.15.26:0/301943904 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a5238532f0 0x55a523849ce0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fccb000bc90 tx=0x7fccb00055e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1c18b268426f6a2e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:18:49.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.159+0000 7fcca0ff9640 1 -- 172.21.15.26:0/301943904 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccb0004220 con 0x55a5238532f0 2024-09-11T23:18:49.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.159+0000 7fcca0ff9640 1 -- 172.21.15.26:0/301943904 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fccb00043c0 con 0x55a5238532f0 2024-09-11T23:18:49.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.159+0000 7fcca0ff9640 1 -- 172.21.15.26:0/301943904 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccb00287c0 con 0x55a5238532f0 2024-09-11T23:18:49.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.160+0000 7fccc35f0740 1 -- 172.21.15.26:0/301943904 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a5238532f0 msgr2=0x55a523849ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:49.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.160+0000 7fccc35f0740 1 --2- 172.21.15.26:0/301943904 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a5238532f0 0x55a523849ce0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fccb000bc90 tx=0x7fccb00055e0 comp rx=0 tx=0).stop 2024-09-11T23:18:49.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.160+0000 7fccc35f0740 1 -- 172.21.15.26:0/301943904 shutdown_connections 2024-09-11T23:18:49.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.160+0000 7fccc35f0740 1 --2- 172.21.15.26:0/301943904 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a52385dbb0 0x55a523859020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:49.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.160+0000 7fccc35f0740 1 --2- 172.21.15.26:0/301943904 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a5238532f0 0x55a523849ce0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:49.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.160+0000 7fccc35f0740 1 --2- 172.21.15.26:0/301943904 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a5238529d0 0x55a52386faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:49.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.160+0000 7fccc35f0740 1 -- 172.21.15.26:0/301943904 >> 172.21.15.26:0/301943904 conn(0x55a52384ef60 msgr2=0x55a52384f340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:18:49.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.160+0000 7fccc35f0740 1 -- 172.21.15.26:0/301943904 shutdown_connections 2024-09-11T23:18:49.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.160+0000 7fccc35f0740 1 -- 172.21.15.26:0/301943904 wait complete. 2024-09-11T23:18:49.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.161+0000 7fccc35f0740 1 -- start start 2024-09-11T23:18:49.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.161+0000 7fccc35f0740 1 Processor -- start 2024-09-11T23:18:49.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.161+0000 7fccc35f0740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a5238529d0 0x55a523873390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:49.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.161+0000 7fccc35f0740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a5238532f0 0x55a5238738d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:49.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.161+0000 7fccc35f0740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a52385dbb0 0x55a5238d0d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:49.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.161+0000 7fccc35f0740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55a52386f920 con 0x55a5238529d0 2024-09-11T23:18:49.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.161+0000 7fccc35f0740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55a52386f250 con 0x55a52385dbb0 2024-09-11T23:18:49.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.161+0000 7fccc35f0740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55a52386fff0 con 0x55a5238532f0 2024-09-11T23:18:49.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.161+0000 7fccc35f0740 10 client.0 _ll_register_callbacks cb 0x7fccac009720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:18:49.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.161+0000 7fccbad9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a5238529d0 0x55a523873390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:49.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.161+0000 7fccbad9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a5238529d0 0x55a523873390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46506/0 (socket says 172.21.15.26:46506) 2024-09-11T23:18:49.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.161+0000 7fccbad9d640 1 -- 172.21.15.26:0/238673185 learned_addr learned my addr 172.21.15.26:0/238673185 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:18:49.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.161+0000 7fccbb59e640 1 --2- 172.21.15.26:0/238673185 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a52385dbb0 0x55a5238d0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:49.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.161+0000 7fccba59c640 1 --2- 172.21.15.26:0/238673185 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a5238532f0 0x55a5238738d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:49.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.161+0000 7fccbad9d640 1 -- 172.21.15.26:0/238673185 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a5238532f0 msgr2=0x55a5238738d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:18:49.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.161+0000 7fccbad9d640 1 --2- 172.21.15.26:0/238673185 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a5238532f0 0x55a5238738d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:49.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.161+0000 7fccbad9d640 1 -- 172.21.15.26:0/238673185 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a52385dbb0 msgr2=0x55a5238d0d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:18:49.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.161+0000 7fccbad9d640 1 --2- 172.21.15.26:0/238673185 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a52385dbb0 0x55a5238d0d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:49.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.161+0000 7fccbad9d640 1 -- 172.21.15.26:0/238673185 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a5238d1410 con 0x55a5238529d0 2024-09-11T23:18:49.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.162+0000 7fccbad9d640 1 --2- 172.21.15.26:0/238673185 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a5238529d0 0x55a523873390 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fccac0cd700 tx=0x7fccac09b620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ad316766584d793 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:18:49.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.162+0000 7fcc9eff5640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:18:49.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.162+0000 7fcc9eff5640 1 -- 172.21.15.26:0/238673185 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccac066060 con 0x55a5238529d0 2024-09-11T23:18:49.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.162+0000 7fcc9eff5640 1 -- 172.21.15.26:0/238673185 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fccac0a45e0 con 0x55a5238529d0 2024-09-11T23:18:49.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.162+0000 7fccc35f0740 1 -- 172.21.15.26:0/238673185 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fccb00043c0 con 0x55a5238529d0 2024-09-11T23:18:49.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.162+0000 7fcc9eff5640 1 -- 172.21.15.26:0/238673185 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fccac0c4050 con 0x55a5238529d0 2024-09-11T23:18:49.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.162+0000 7fccc35f0740 10 client.15477 fetch_fsmap learned FSMap version 29 2024-09-11T23:18:49.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.162+0000 7fccc35f0740 1 -- 172.21.15.26:0/238673185 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a5238d8780 con 0x55a5238529d0 2024-09-11T23:18:49.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.162+0000 7fcc9eff5640 1 -- 172.21.15.26:0/238673185 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fccb00043c0 con 0x55a5238529d0 2024-09-11T23:18:49.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.162+0000 7fcc9eff5640 1 -- 172.21.15.26:0/238673185 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fccac09acc0 con 0x55a5238529d0 2024-09-11T23:18:49.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.162+0000 7fccc35f0740 10 client.15477 fetch_fsmap finished waiting for FSMap version 29 2024-09-11T23:18:49.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.162+0000 7fccc35f0740 10 client.15477 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-11T23:18:49.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.162+0000 7fccc35f0740 20 client.15477 populate_metadata read hostname 'smithi026' 2024-09-11T23:18:49.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.162+0000 7fccc35f0740 1 --2- 172.21.15.26:0/238673185 >> [v2:172.21.15.76:6836/2078978748,v1:172.21.15.76:6837/2078978748] conn(0x55a5238dd2c0 0x55a5238dbfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:49.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.163+0000 7fccc35f0740 4 client.15477 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-11T23:18:49.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.163+0000 7fccc35f0740 1 -- 172.21.15.26:0/238673185 --> [v2:172.21.15.76:6836/2078978748,v1:172.21.15.76:6837/2078978748] -- command(tid 0: {"prefix": "status"}) -- 0x55a52373d0e0 con 0x55a5238dd2c0 2024-09-11T23:18:49.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.163+0000 7fccba59c640 1 --2- 172.21.15.26:0/238673185 >> [v2:172.21.15.76:6836/2078978748,v1:172.21.15.76:6837/2078978748] conn(0x55a5238dd2c0 0x55a5238dbfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:49.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.163+0000 7fccba59c640 1 --2- 172.21.15.26:0/238673185 >> [v2:172.21.15.76:6836/2078978748,v1:172.21.15.76:6837/2078978748] conn(0x55a5238dd2c0 0x55a5238dbfd0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2bd5b43c07017d9b server_cookie=1bca9779b413956d in_seq=0 out_seq=0 2024-09-11T23:18:49.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.163+0000 7fcc9eff5640 10 client.15477 ms_handle_connect on v2:172.21.15.76:6836/2078978748 2024-09-11T23:18:49.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.163+0000 7fcc9eff5640 1 -- 172.21.15.26:0/238673185 <== mds.2 v2:172.21.15.76:6836/2078978748 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55a52373d0e0 con 0x55a5238dd2c0 2024-09-11T23:18:49.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.163+0000 7fcc9eff5640 10 client.15477 handle_command_reply: tid=0 2024-09-11T23:18:49.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.163+0000 7fccc35f0740 1 client.15477 shutdown 2024-09-11T23:18:49.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.163+0000 7fccc35f0740 1 -- 172.21.15.26:0/238673185 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a5238529d0 msgr2=0x55a523873390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:49.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.163+0000 7fccc35f0740 1 --2- 172.21.15.26:0/238673185 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a5238529d0 0x55a523873390 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fccac0cd700 tx=0x7fccac09b620 comp rx=0 tx=0).stop 2024-09-11T23:18:49.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.164+0000 7fccc35f0740 1 -- 172.21.15.26:0/238673185 shutdown_connections 2024-09-11T23:18:49.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.164+0000 7fccc35f0740 1 --2- 172.21.15.26:0/238673185 >> [v2:172.21.15.76:6836/2078978748,v1:172.21.15.76:6837/2078978748] conn(0x55a5238dd2c0 0x55a5238dbfd0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:49.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.164+0000 7fccc35f0740 1 --2- 172.21.15.26:0/238673185 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a52385dbb0 0x55a5238d0d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:49.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.164+0000 7fccc35f0740 1 --2- 172.21.15.26:0/238673185 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a5238532f0 0x55a5238738d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:49.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.164+0000 7fccc35f0740 1 --2- 172.21.15.26:0/238673185 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a5238529d0 0x55a523873390 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:49.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.164+0000 7fccc35f0740 1 -- 172.21.15.26:0/238673185 >> 172.21.15.26:0/238673185 conn(0x55a52384ef60 msgr2=0x55a523849ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:18:49.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.164+0000 7fcc9eff5640 0 client.15477 ms_handle_reset on v2:172.21.15.76:6836/2078978748 2024-09-11T23:18:49.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.164+0000 7fccc35f0740 1 -- 172.21.15.26:0/238673185 shutdown_connections 2024-09-11T23:18:49.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.164+0000 7fccc35f0740 1 -- 172.21.15.26:0/238673185 wait complete. 2024-09-11T23:18:49.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.164+0000 7fccc35f0740 20 client.15477 trim_cache size 0 max 16384 2024-09-11T23:18:49.169 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-11T23:18:49.170 INFO:teuthology.orchestra.run.smithi026.stdout: "cluster_fsid": "af7e6fc8-7092-11ef-bcea-c7b262605968", 2024-09-11T23:18:49.170 INFO:teuthology.orchestra.run.smithi026.stdout: "whoami": 2, 2024-09-11T23:18:49.170 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 24458, 2024-09-11T23:18:49.170 INFO:teuthology.orchestra.run.smithi026.stdout: "want_state": "up:active", 2024-09-11T23:18:49.170 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "up:active", 2024-09-11T23:18:49.170 INFO:teuthology.orchestra.run.smithi026.stdout: "fs_name": "cephfs", 2024-09-11T23:18:49.170 INFO:teuthology.orchestra.run.smithi026.stdout: "rank_uptime": 134.218747186, 2024-09-11T23:18:49.170 INFO:teuthology.orchestra.run.smithi026.stdout: "mdsmap_epoch": 29, 2024-09-11T23:18:49.170 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch": 86, 2024-09-11T23:18:49.170 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch_barrier": 85, 2024-09-11T23:18:49.170 INFO:teuthology.orchestra.run.smithi026.stdout: "uptime": 165.47405925000001 2024-09-11T23:18:49.170 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-11T23:18:49.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.164+0000 7fccc1217640 1 -- 172.21.15.26:0/3462228706 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fcca40a2d80 msgr2=0x7fcca40a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:49.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.164+0000 7fccc1217640 1 --2- 172.21.15.26:0/3462228706 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fcca40a2d80 0x7fcca40a5240 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fccb4009a60 tx=0x7fccb40099d0 comp rx=0 tx=0).stop 2024-09-11T23:18:49.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.164+0000 7fccc1217640 1 -- 172.21.15.26:0/3462228706 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fccbc100a80 msgr2=0x7fccbc198470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:49.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.164+0000 7fccc1217640 1 --2- 172.21.15.26:0/3462228706 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fccbc100a80 0x7fccbc198470 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fccac00e9f0 tx=0x7fccac00edf0 comp rx=0 tx=0).stop 2024-09-11T23:18:49.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.165+0000 7fccc1217640 1 -- 172.21.15.26:0/3462228706 shutdown_connections 2024-09-11T23:18:49.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.165+0000 7fccc1217640 1 --2- 172.21.15.26:0/3462228706 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fcca40a2d80 0x7fcca40a5240 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:49.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.165+0000 7fccc1217640 1 --2- 172.21.15.26:0/3462228706 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fccbc106a80 0x7fccbc19cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:49.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.165+0000 7fccc1217640 1 --2- 172.21.15.26:0/3462228706 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fccbc101420 0x7fccbc1989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:49.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.165+0000 7fccc1217640 1 --2- 172.21.15.26:0/3462228706 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fccbc100a80 0x7fccbc198470 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:49.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.165+0000 7fccc1217640 1 -- 172.21.15.26:0/3462228706 >> 172.21.15.26:0/3462228706 conn(0x7fccbc0fc7a0 msgr2=0x7fccbc06aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:18:49.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.165+0000 7fccc1217640 1 -- 172.21.15.26:0/3462228706 shutdown_connections 2024-09-11T23:18:49.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:49.165+0000 7fccc1217640 1 -- 172.21.15.26:0/3462228706 wait complete. 2024-09-11T23:18:49.962 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph tell mds.i status 2024-09-11T23:18:50.220 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:18:50.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:49 smithi136 ceph-mon[35121]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 2.6 KiB/s wr, 17 op/s 2024-09-11T23:18:50.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:49 smithi026 ceph-mon[31172]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 2.6 KiB/s wr, 17 op/s 2024-09-11T23:18:50.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:49 smithi076 ceph-mon[35296]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 2.6 KiB/s wr, 17 op/s 2024-09-11T23:18:50.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.687+0000 7f3572bfa640 1 -- 172.21.15.26:0/79377165 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f356c106a80 msgr2=0x7f356c106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:50.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.687+0000 7f3572bfa640 1 --2- 172.21.15.26:0/79377165 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f356c106a80 0x7f356c106e60 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f3560009930 tx=0x7f356002a420 comp rx=0 tx=0).stop 2024-09-11T23:18:50.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.687+0000 7f3572bfa640 1 -- 172.21.15.26:0/79377165 shutdown_connections 2024-09-11T23:18:50.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.687+0000 7f3572bfa640 1 --2- 172.21.15.26:0/79377165 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f356c101420 0x7f356c10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:50.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.687+0000 7f3572bfa640 1 --2- 172.21.15.26:0/79377165 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f356c100a80 0x7f356c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:50.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.687+0000 7f3572bfa640 1 --2- 172.21.15.26:0/79377165 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f356c106a80 0x7f356c106e60 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:50.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.687+0000 7f3572bfa640 1 -- 172.21.15.26:0/79377165 >> 172.21.15.26:0/79377165 conn(0x7f356c0fc7a0 msgr2=0x7f356c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:18:50.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.687+0000 7f3572bfa640 1 -- 172.21.15.26:0/79377165 shutdown_connections 2024-09-11T23:18:50.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.687+0000 7f3572bfa640 1 -- 172.21.15.26:0/79377165 wait complete. 2024-09-11T23:18:50.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.688+0000 7f3572bfa640 1 Processor -- start 2024-09-11T23:18:50.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.688+0000 7f3572bfa640 1 -- start start 2024-09-11T23:18:50.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.688+0000 7f3572bfa640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f356c100a80 0x7f356c1079b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:50.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.688+0000 7f3572bfa640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f356c101420 0x7f356c107ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:50.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.688+0000 7f3572bfa640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f356c106a80 0x7f356c108430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:50.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.688+0000 7f3572bfa640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f356c110e80 con 0x7f356c106a80 2024-09-11T23:18:50.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.688+0000 7f3572bfa640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f356c110d00 con 0x7f356c100a80 2024-09-11T23:18:50.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.688+0000 7f3572bfa640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f356c111000 con 0x7f356c101420 2024-09-11T23:18:50.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.688+0000 7f3571197640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f356c106a80 0x7f356c108430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:50.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.688+0000 7f3571197640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f356c106a80 0x7f356c108430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53346/0 (socket says 172.21.15.26:53346) 2024-09-11T23:18:50.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.688+0000 7f3571197640 1 -- 172.21.15.26:0/1243710858 learned_addr learned my addr 172.21.15.26:0/1243710858 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:18:50.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.688+0000 7f356bfff640 1 --2- 172.21.15.26:0/1243710858 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f356c101420 0x7f356c107ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:50.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.688+0000 7f3570996640 1 --2- 172.21.15.26:0/1243710858 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f356c100a80 0x7f356c1079b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:50.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.689+0000 7f3571197640 1 -- 172.21.15.26:0/1243710858 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f356c101420 msgr2=0x7f356c107ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:50.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.689+0000 7f3571197640 1 --2- 172.21.15.26:0/1243710858 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f356c101420 0x7f356c107ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:50.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.689+0000 7f3571197640 1 -- 172.21.15.26:0/1243710858 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f356c100a80 msgr2=0x7f356c1079b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:50.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.689+0000 7f3571197640 1 --2- 172.21.15.26:0/1243710858 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f356c100a80 0x7f356c1079b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:50.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.689+0000 7f3571197640 1 -- 172.21.15.26:0/1243710858 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f356c1aad10 con 0x7f356c106a80 2024-09-11T23:18:50.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.689+0000 7f3571197640 1 --2- 172.21.15.26:0/1243710858 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f356c106a80 0x7f356c108430 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f356400ead0 tx=0x7f356400ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:18:50.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.689+0000 7f3569ffb640 1 -- 172.21.15.26:0/1243710858 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f356400cde0 con 0x7f356c106a80 2024-09-11T23:18:50.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.689+0000 7f3569ffb640 1 -- 172.21.15.26:0/1243710858 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3564005020 con 0x7f356c106a80 2024-09-11T23:18:50.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.689+0000 7f3572bfa640 1 -- 172.21.15.26:0/1243710858 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f356c1ab000 con 0x7f356c106a80 2024-09-11T23:18:50.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.689+0000 7f3569ffb640 1 -- 172.21.15.26:0/1243710858 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35640116d0 con 0x7f356c106a80 2024-09-11T23:18:50.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.689+0000 7f3572bfa640 1 -- 172.21.15.26:0/1243710858 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f356c1ab4c0 con 0x7f356c106a80 2024-09-11T23:18:50.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.691+0000 7f3569ffb640 1 -- 172.21.15.26:0/1243710858 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 24) ==== 138009+0+0 (secure 0 0 0) 0x7f3564005560 con 0x7f356c106a80 2024-09-11T23:18:50.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.692+0000 7f3574fd3740 1 Processor -- start 2024-09-11T23:18:50.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.692+0000 7f3574fd3740 1 -- start start 2024-09-11T23:18:50.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.692+0000 7f3569ffb640 1 --2- 172.21.15.26:0/1243710858 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f355c0a2d80 0x7f355c0a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:50.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.692+0000 7f3574fd3740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559d5a3d29d0 0x559d5a3d2db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:50.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.692+0000 7f3574fd3740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559d5a3d32f0 0x559d5a3dd670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:50.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.692+0000 7f3574fd3740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559d5a3ddbb0 0x559d5a3dffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:50.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.692+0000 7f3574fd3740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x559d5a339090 con 0x559d5a3ddbb0 2024-09-11T23:18:50.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.692+0000 7f3574fd3740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x559d5a2e0a70 con 0x559d5a3d32f0 2024-09-11T23:18:50.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.692+0000 7f3574fd3740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x559d5a335050 con 0x559d5a3d29d0 2024-09-11T23:18:50.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.692+0000 7f356bfff640 1 --2- 172.21.15.26:0/1243710858 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f355c0a2d80 0x7f355c0a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:50.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.692+0000 7f3571197640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559d5a3ddbb0 0x559d5a3dffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:50.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.692+0000 7f3569ffb640 1 -- 172.21.15.26:0/1243710858 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f35640cf890 con 0x7f356c106a80 2024-09-11T23:18:50.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.692+0000 7f3570996640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559d5a3d29d0 0x559d5a3d2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:50.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.692+0000 7f3570996640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559d5a3d29d0 0x559d5a3d2db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:42656/0 (socket says 172.21.15.26:42656) 2024-09-11T23:18:50.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.692+0000 7f3571197640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559d5a3ddbb0 0x559d5a3dffa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53358/0 (socket says 172.21.15.26:53358) 2024-09-11T23:18:50.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.692+0000 7f3570996640 1 -- 172.21.15.26:0/92652410 learned_addr learned my addr 172.21.15.26:0/92652410 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:18:50.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.692+0000 7f356bfff640 1 --2- 172.21.15.26:0/92652410 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559d5a3d32f0 0x559d5a3dd670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:50.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.693+0000 7f3571197640 1 -- 172.21.15.26:0/92652410 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559d5a3d29d0 msgr2=0x559d5a3d2db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:18:50.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.693+0000 7f3571197640 1 --2- 172.21.15.26:0/92652410 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559d5a3d29d0 0x559d5a3d2db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:50.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.693+0000 7f3571197640 1 -- 172.21.15.26:0/92652410 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559d5a3d32f0 msgr2=0x559d5a3dd670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:18:50.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.693+0000 7f3571197640 1 --2- 172.21.15.26:0/92652410 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559d5a3d32f0 0x559d5a3dd670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:50.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.693+0000 7f3571197640 1 -- 172.21.15.26:0/92652410 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d5a2ed130 con 0x559d5a3ddbb0 2024-09-11T23:18:50.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.693+0000 7f3570996640 1 --2- 172.21.15.26:0/92652410 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559d5a3d29d0 0x559d5a3d2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:18:50.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.693+0000 7f356bfff640 1 --2- 172.21.15.26:0/92652410 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559d5a3d32f0 0x559d5a3dd670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:18:50.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.693+0000 7f356bfff640 1 --2- 172.21.15.26:0/1243710858 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f355c0a2d80 0x7f355c0a5240 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f3558004650 tx=0x7f355800bef0 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:18:50.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.693+0000 7f3571197640 1 --2- 172.21.15.26:0/92652410 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559d5a3ddbb0 0x559d5a3dffa0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f35640cced0 tx=0x7f356409c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c49e34924aa5425f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:18:50.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.693+0000 7f3556ffd640 1 -- 172.21.15.26:0/92652410 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f356401f040 con 0x559d5a3ddbb0 2024-09-11T23:18:50.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.693+0000 7f3556ffd640 1 -- 172.21.15.26:0/92652410 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3564099ef0 con 0x559d5a3ddbb0 2024-09-11T23:18:50.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.693+0000 7f3556ffd640 1 -- 172.21.15.26:0/92652410 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35640cd3e0 con 0x559d5a3ddbb0 2024-09-11T23:18:50.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.694+0000 7f3574fd3740 1 -- 172.21.15.26:0/92652410 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559d5a3ddbb0 msgr2=0x559d5a3dffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:50.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.694+0000 7f3574fd3740 1 --2- 172.21.15.26:0/92652410 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559d5a3ddbb0 0x559d5a3dffa0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f35640cced0 tx=0x7f356409c430 comp rx=0 tx=0).stop 2024-09-11T23:18:50.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.694+0000 7f3574fd3740 1 -- 172.21.15.26:0/92652410 shutdown_connections 2024-09-11T23:18:50.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.694+0000 7f3574fd3740 1 --2- 172.21.15.26:0/92652410 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559d5a3d32f0 0x559d5a3dd670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:50.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.694+0000 7f3574fd3740 1 --2- 172.21.15.26:0/92652410 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559d5a3ddbb0 0x559d5a3dffa0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:50.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.694+0000 7f3574fd3740 1 --2- 172.21.15.26:0/92652410 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559d5a3d29d0 0x559d5a3d2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:50.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.694+0000 7f3574fd3740 1 -- 172.21.15.26:0/92652410 >> 172.21.15.26:0/92652410 conn(0x559d5a3cef60 msgr2=0x559d5a3cf340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:18:50.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.694+0000 7f3574fd3740 1 -- 172.21.15.26:0/92652410 shutdown_connections 2024-09-11T23:18:50.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.694+0000 7f3574fd3740 1 -- 172.21.15.26:0/92652410 wait complete. 2024-09-11T23:18:50.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.694+0000 7f3574fd3740 1 -- start start 2024-09-11T23:18:50.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.694+0000 7f3574fd3740 1 Processor -- start 2024-09-11T23:18:50.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.695+0000 7f3574fd3740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559d5a3d29d0 0x559d5a3dd4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:50.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.695+0000 7f3574fd3740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559d5a3d32f0 0x559d5a3da820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:50.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.695+0000 7f3574fd3740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559d5a3ddbb0 0x559d5a3dad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:50.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.695+0000 7f3574fd3740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x559d5a3e0720 con 0x559d5a3d32f0 2024-09-11T23:18:50.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.695+0000 7f3574fd3740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x559d5a2f3f20 con 0x559d5a3d29d0 2024-09-11T23:18:50.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.695+0000 7f3574fd3740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x559d5a3e08a0 con 0x559d5a3ddbb0 2024-09-11T23:18:50.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.695+0000 7f3574fd3740 10 client.0 _ll_register_callbacks cb 0x559d5a2bd030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:18:50.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.695+0000 7f356bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559d5a3d32f0 0x559d5a3da820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:50.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.695+0000 7f356bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559d5a3d32f0 0x559d5a3da820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53360/0 (socket says 172.21.15.26:53360) 2024-09-11T23:18:50.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.695+0000 7f356bfff640 1 -- 172.21.15.26:0/2764513975 learned_addr learned my addr 172.21.15.26:0/2764513975 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:18:50.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.695+0000 7f3571197640 1 --2- 172.21.15.26:0/2764513975 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559d5a3ddbb0 0x559d5a3dad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:50.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.695+0000 7f3570996640 1 --2- 172.21.15.26:0/2764513975 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559d5a3d29d0 0x559d5a3dd4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:50.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.695+0000 7f356bfff640 1 -- 172.21.15.26:0/2764513975 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559d5a3ddbb0 msgr2=0x559d5a3dad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:18:50.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.695+0000 7f356bfff640 1 --2- 172.21.15.26:0/2764513975 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559d5a3ddbb0 0x559d5a3dad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:50.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.695+0000 7f356bfff640 1 -- 172.21.15.26:0/2764513975 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559d5a3d29d0 msgr2=0x559d5a3dd4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:18:50.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.695+0000 7f356bfff640 1 --2- 172.21.15.26:0/2764513975 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559d5a3d29d0 0x559d5a3dd4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:50.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.695+0000 7f356bfff640 1 -- 172.21.15.26:0/2764513975 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d5a0aa2b0 con 0x559d5a3d32f0 2024-09-11T23:18:50.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.695+0000 7f3571197640 1 --2- 172.21.15.26:0/2764513975 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559d5a3ddbb0 0x559d5a3dad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:18:50.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.695+0000 7f3570996640 1 --2- 172.21.15.26:0/2764513975 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559d5a3d29d0 0x559d5a3dd4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:18:50.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.695+0000 7f356bfff640 1 --2- 172.21.15.26:0/2764513975 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559d5a3d32f0 0x559d5a3da820 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f3558003eb0 tx=0x7f3558004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67818c9236630a9e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:18:50.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.695+0000 7f354bfff640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:18:50.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.695+0000 7f354bfff640 1 -- 172.21.15.26:0/2764513975 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3558020850 con 0x559d5a3d32f0 2024-09-11T23:18:50.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.696+0000 7f354bfff640 1 -- 172.21.15.26:0/2764513975 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f35580209f0 con 0x559d5a3d32f0 2024-09-11T23:18:50.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.696+0000 7f3574fd3740 1 -- 172.21.15.26:0/2764513975 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3564099ef0 con 0x559d5a3d32f0 2024-09-11T23:18:50.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.696+0000 7f354bfff640 1 -- 172.21.15.26:0/2764513975 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f355802e050 con 0x559d5a3d32f0 2024-09-11T23:18:50.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.696+0000 7f3574fd3740 10 client.15489 fetch_fsmap learned FSMap version 29 2024-09-11T23:18:50.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.696+0000 7f3574fd3740 1 -- 172.21.15.26:0/2764513975 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559d5a3ec0a0 con 0x559d5a3d32f0 2024-09-11T23:18:50.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.696+0000 7f354bfff640 1 -- 172.21.15.26:0/2764513975 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3564099ef0 con 0x559d5a3d32f0 2024-09-11T23:18:50.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.696+0000 7f354bfff640 1 -- 172.21.15.26:0/2764513975 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f3558029c40 con 0x559d5a3d32f0 2024-09-11T23:18:50.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.697+0000 7f3574fd3740 10 client.15489 fetch_fsmap finished waiting for FSMap version 29 2024-09-11T23:18:50.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.697+0000 7f3574fd3740 10 client.15489 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-11T23:18:50.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.697+0000 7f3574fd3740 20 client.15489 populate_metadata read hostname 'smithi026' 2024-09-11T23:18:50.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.697+0000 7f3574fd3740 1 --2- 172.21.15.26:0/2764513975 >> [v2:172.21.15.136:6838/3211398630,v1:172.21.15.136:6839/3211398630] conn(0x559d5a3ee6e0 0x559d5a3efb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:50.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.697+0000 7f3574fd3740 4 client.15489 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:18:50.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.697+0000 7f3574fd3740 1 -- 172.21.15.26:0/2764513975 --> [v2:172.21.15.136:6838/3211398630,v1:172.21.15.136:6839/3211398630] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559d5a346600 con 0x559d5a3ee6e0 2024-09-11T23:18:50.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.697+0000 7f3570996640 1 --2- 172.21.15.26:0/2764513975 >> [v2:172.21.15.136:6838/3211398630,v1:172.21.15.136:6839/3211398630] conn(0x559d5a3ee6e0 0x559d5a3efb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:50.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.698+0000 7f354bfff640 10 client.15489 ms_handle_connect on v2:172.21.15.136:6838/3211398630 2024-09-11T23:18:50.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.698+0000 7f3570996640 1 --2- 172.21.15.26:0/2764513975 >> [v2:172.21.15.136:6838/3211398630,v1:172.21.15.136:6839/3211398630] conn(0x559d5a3ee6e0 0x559d5a3efb50 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8ab8d06f2c1766ec server_cookie=29febd90fea4ad4a in_seq=0 out_seq=0 2024-09-11T23:18:50.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.698+0000 7f354bfff640 1 -- 172.21.15.26:0/2764513975 <== mds.1 v2:172.21.15.136:6838/3211398630 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559d5a346600 con 0x559d5a3ee6e0 2024-09-11T23:18:50.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.698+0000 7f354bfff640 10 client.15489 handle_command_reply: tid=0 2024-09-11T23:18:50.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.698+0000 7f3574fd3740 1 client.15489 shutdown 2024-09-11T23:18:50.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.699+0000 7f3574fd3740 1 -- 172.21.15.26:0/2764513975 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559d5a3d32f0 msgr2=0x559d5a3da820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:50.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.699+0000 7f3574fd3740 1 --2- 172.21.15.26:0/2764513975 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559d5a3d32f0 0x559d5a3da820 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f3558003eb0 tx=0x7f3558004730 comp rx=0 tx=0).stop 2024-09-11T23:18:50.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.699+0000 7f3574fd3740 1 -- 172.21.15.26:0/2764513975 shutdown_connections 2024-09-11T23:18:50.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.699+0000 7f3574fd3740 1 --2- 172.21.15.26:0/2764513975 >> [v2:172.21.15.136:6838/3211398630,v1:172.21.15.136:6839/3211398630] conn(0x559d5a3ee6e0 0x559d5a3efb50 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:50.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.699+0000 7f3574fd3740 1 --2- 172.21.15.26:0/2764513975 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559d5a3ddbb0 0x559d5a3dad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:50.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.699+0000 7f3574fd3740 1 --2- 172.21.15.26:0/2764513975 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559d5a3d32f0 0x559d5a3da820 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:50.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.699+0000 7f3574fd3740 1 --2- 172.21.15.26:0/2764513975 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559d5a3d29d0 0x559d5a3dd4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:50.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.699+0000 7f354bfff640 0 client.15489 ms_handle_reset on v2:172.21.15.136:6838/3211398630 2024-09-11T23:18:50.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.699+0000 7f3574fd3740 1 -- 172.21.15.26:0/2764513975 >> 172.21.15.26:0/2764513975 conn(0x559d5a3cef60 msgr2=0x559d5a3d4c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:18:50.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.699+0000 7f3574fd3740 1 -- 172.21.15.26:0/2764513975 shutdown_connections 2024-09-11T23:18:50.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.699+0000 7f3574fd3740 1 -- 172.21.15.26:0/2764513975 wait complete. 2024-09-11T23:18:50.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.699+0000 7f3574fd3740 20 client.15489 trim_cache size 0 max 16384 2024-09-11T23:18:50.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.717+0000 7f3574fd3740 1 Processor -- start 2024-09-11T23:18:50.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.717+0000 7f3574fd3740 1 -- start start 2024-09-11T23:18:50.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.717+0000 7f3574fd3740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559d5a3d29d0 0x559d5a3ef480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:50.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.717+0000 7f3574fd3740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559d5a3d32f0 0x559d5a3c9ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:50.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.717+0000 7f3574fd3740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559d5a3ddbb0 0x559d5a3dbf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:50.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.717+0000 7f3574fd3740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x559d5a3ede90 con 0x559d5a3d32f0 2024-09-11T23:18:50.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.717+0000 7f3574fd3740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x559d5a3ee560 con 0x559d5a3d29d0 2024-09-11T23:18:50.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.717+0000 7f3574fd3740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x559d5a3eec30 con 0x559d5a3ddbb0 2024-09-11T23:18:50.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.717+0000 7f356bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559d5a3d32f0 0x559d5a3c9ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:50.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.717+0000 7f356bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559d5a3d32f0 0x559d5a3c9ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53374/0 (socket says 172.21.15.26:53374) 2024-09-11T23:18:50.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.717+0000 7f356bfff640 1 -- 172.21.15.26:0/984550826 learned_addr learned my addr 172.21.15.26:0/984550826 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:18:50.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.717+0000 7f3570996640 1 --2- 172.21.15.26:0/984550826 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559d5a3d29d0 0x559d5a3ef480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:50.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.717+0000 7f3571197640 1 --2- 172.21.15.26:0/984550826 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559d5a3ddbb0 0x559d5a3dbf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:50.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.717+0000 7f356bfff640 1 -- 172.21.15.26:0/984550826 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559d5a3ddbb0 msgr2=0x559d5a3dbf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:18:50.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.717+0000 7f356bfff640 1 --2- 172.21.15.26:0/984550826 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559d5a3ddbb0 0x559d5a3dbf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:50.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.717+0000 7f356bfff640 1 -- 172.21.15.26:0/984550826 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559d5a3d29d0 msgr2=0x559d5a3ef480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:18:50.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.717+0000 7f356bfff640 1 --2- 172.21.15.26:0/984550826 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559d5a3d29d0 0x559d5a3ef480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:50.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.717+0000 7f356bfff640 1 -- 172.21.15.26:0/984550826 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d5a3d00f0 con 0x559d5a3d32f0 2024-09-11T23:18:50.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.718+0000 7f3571197640 1 --2- 172.21.15.26:0/984550826 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559d5a3ddbb0 0x559d5a3dbf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:18:50.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.718+0000 7f356bfff640 1 --2- 172.21.15.26:0/984550826 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559d5a3d32f0 0x559d5a3c9ce0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f3558000c00 tx=0x7f355800b440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea95741e49a2ad6e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:18:50.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.718+0000 7f35567fc640 1 -- 172.21.15.26:0/984550826 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35580042f0 con 0x559d5a3d32f0 2024-09-11T23:18:50.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.718+0000 7f35567fc640 1 -- 172.21.15.26:0/984550826 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3558025070 con 0x559d5a3d32f0 2024-09-11T23:18:50.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.718+0000 7f35567fc640 1 -- 172.21.15.26:0/984550826 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35580047a0 con 0x559d5a3d32f0 2024-09-11T23:18:50.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.719+0000 7f3574fd3740 1 -- 172.21.15.26:0/984550826 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559d5a3d32f0 msgr2=0x559d5a3c9ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:50.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.719+0000 7f3574fd3740 1 --2- 172.21.15.26:0/984550826 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559d5a3d32f0 0x559d5a3c9ce0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f3558000c00 tx=0x7f355800b440 comp rx=0 tx=0).stop 2024-09-11T23:18:50.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.719+0000 7f3574fd3740 1 -- 172.21.15.26:0/984550826 shutdown_connections 2024-09-11T23:18:50.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.719+0000 7f3574fd3740 1 --2- 172.21.15.26:0/984550826 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559d5a3ddbb0 0x559d5a3dbf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:50.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.719+0000 7f3574fd3740 1 --2- 172.21.15.26:0/984550826 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559d5a3d32f0 0x559d5a3c9ce0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:50.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.719+0000 7f3574fd3740 1 --2- 172.21.15.26:0/984550826 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559d5a3d29d0 0x559d5a3ef480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:50.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.719+0000 7f3574fd3740 1 -- 172.21.15.26:0/984550826 >> 172.21.15.26:0/984550826 conn(0x559d5a3cef60 msgr2=0x559d5a3da7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:18:50.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.719+0000 7f3574fd3740 1 -- 172.21.15.26:0/984550826 shutdown_connections 2024-09-11T23:18:50.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.719+0000 7f3574fd3740 1 -- 172.21.15.26:0/984550826 wait complete. 2024-09-11T23:18:50.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.719+0000 7f3574fd3740 1 -- start start 2024-09-11T23:18:50.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.719+0000 7f3574fd3740 1 Processor -- start 2024-09-11T23:18:50.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.719+0000 7f3574fd3740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559d5a3d29d0 0x559d5a3f8220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:50.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.720+0000 7f3574fd3740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559d5a3d32f0 0x559d5a3d7220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:50.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.720+0000 7f3574fd3740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559d5a3ddbb0 0x559d5a3d7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:50.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.720+0000 7f3574fd3740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x559d5a3ef9d0 con 0x559d5a3d32f0 2024-09-11T23:18:50.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.720+0000 7f3574fd3740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x559d5a3ef300 con 0x559d5a3ddbb0 2024-09-11T23:18:50.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.720+0000 7f3574fd3740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x559d5a3f00a0 con 0x559d5a3d29d0 2024-09-11T23:18:50.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.720+0000 7f3574fd3740 10 client.0 _ll_register_callbacks cb 0x7f3558005c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:18:50.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.720+0000 7f356bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559d5a3d32f0 0x559d5a3d7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:50.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.720+0000 7f3570996640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559d5a3d29d0 0x559d5a3f8220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:50.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.720+0000 7f3571197640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559d5a3ddbb0 0x559d5a3d7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:50.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.720+0000 7f3570996640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559d5a3d29d0 0x559d5a3f8220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:42672/0 (socket says 172.21.15.26:42672) 2024-09-11T23:18:50.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.720+0000 7f356bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559d5a3d32f0 0x559d5a3d7220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53388/0 (socket says 172.21.15.26:53388) 2024-09-11T23:18:50.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.720+0000 7f3570996640 1 -- 172.21.15.26:0/588940765 learned_addr learned my addr 172.21.15.26:0/588940765 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:18:50.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.720+0000 7f356bfff640 1 -- 172.21.15.26:0/588940765 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559d5a3d29d0 msgr2=0x559d5a3f8220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:18:50.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.720+0000 7f356bfff640 1 --2- 172.21.15.26:0/588940765 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559d5a3d29d0 0x559d5a3f8220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:50.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.720+0000 7f356bfff640 1 -- 172.21.15.26:0/588940765 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559d5a3ddbb0 msgr2=0x559d5a3d7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:18:50.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.720+0000 7f356bfff640 1 --2- 172.21.15.26:0/588940765 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559d5a3ddbb0 0x559d5a3d7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:50.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.720+0000 7f356bfff640 1 -- 172.21.15.26:0/588940765 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d5a3d7ca0 con 0x559d5a3d32f0 2024-09-11T23:18:50.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.720+0000 7f3571197640 1 --2- 172.21.15.26:0/588940765 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559d5a3ddbb0 0x559d5a3d7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:18:50.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.720+0000 7f356bfff640 1 --2- 172.21.15.26:0/588940765 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559d5a3d32f0 0x559d5a3d7220 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x559d5a3d3e60 tx=0x7f35580040b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97db38161313dfab server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:18:50.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.720+0000 7f354bfff640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:18:50.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.720+0000 7f354bfff640 1 -- 172.21.15.26:0/588940765 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35580055b0 con 0x559d5a3d32f0 2024-09-11T23:18:50.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.720+0000 7f354bfff640 1 -- 172.21.15.26:0/588940765 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f355801f990 con 0x559d5a3d32f0 2024-09-11T23:18:50.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.720+0000 7f3574fd3740 1 -- 172.21.15.26:0/588940765 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3558025070 con 0x559d5a3d32f0 2024-09-11T23:18:50.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.721+0000 7f354bfff640 1 -- 172.21.15.26:0/588940765 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f35580042f0 con 0x559d5a3d32f0 2024-09-11T23:18:50.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.721+0000 7f3574fd3740 10 client.15495 fetch_fsmap learned FSMap version 29 2024-09-11T23:18:50.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.721+0000 7f3574fd3740 1 -- 172.21.15.26:0/588940765 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559d5a3f6aa0 con 0x559d5a3d32f0 2024-09-11T23:18:50.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.721+0000 7f354bfff640 1 -- 172.21.15.26:0/588940765 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3558025070 con 0x559d5a3d32f0 2024-09-11T23:18:50.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.721+0000 7f354bfff640 1 -- 172.21.15.26:0/588940765 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f3558020650 con 0x559d5a3d32f0 2024-09-11T23:18:50.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.721+0000 7f3574fd3740 10 client.15495 fetch_fsmap finished waiting for FSMap version 29 2024-09-11T23:18:50.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.721+0000 7f3574fd3740 10 client.15495 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-11T23:18:50.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.721+0000 7f3574fd3740 20 client.15495 populate_metadata read hostname 'smithi026' 2024-09-11T23:18:50.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.721+0000 7f3574fd3740 1 --2- 172.21.15.26:0/588940765 >> [v2:172.21.15.136:6838/3211398630,v1:172.21.15.136:6839/3211398630] conn(0x559d5a45d190 0x559d5a45bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:50.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.721+0000 7f3574fd3740 4 client.15495 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-11T23:18:50.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.721+0000 7f3574fd3740 1 -- 172.21.15.26:0/588940765 --> [v2:172.21.15.136:6838/3211398630,v1:172.21.15.136:6839/3211398630] -- command(tid 0: {"prefix": "status"}) -- 0x559d5a340b10 con 0x559d5a45d190 2024-09-11T23:18:50.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.721+0000 7f3570996640 1 --2- 172.21.15.26:0/588940765 >> [v2:172.21.15.136:6838/3211398630,v1:172.21.15.136:6839/3211398630] conn(0x559d5a45d190 0x559d5a45bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:50.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.722+0000 7f354bfff640 10 client.15495 ms_handle_connect on v2:172.21.15.136:6838/3211398630 2024-09-11T23:18:50.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.722+0000 7f3570996640 1 --2- 172.21.15.26:0/588940765 >> [v2:172.21.15.136:6838/3211398630,v1:172.21.15.136:6839/3211398630] conn(0x559d5a45d190 0x559d5a45bea0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ee8ecd92087ce90a server_cookie=a36619d2fabd24ca in_seq=0 out_seq=0 2024-09-11T23:18:50.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.722+0000 7f354bfff640 1 -- 172.21.15.26:0/588940765 <== mds.1 v2:172.21.15.136:6838/3211398630 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x559d5a340b10 con 0x559d5a45d190 2024-09-11T23:18:50.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.722+0000 7f354bfff640 10 client.15495 handle_command_reply: tid=0 2024-09-11T23:18:50.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.722+0000 7f3574fd3740 1 client.15495 shutdown 2024-09-11T23:18:50.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.722+0000 7f3574fd3740 1 -- 172.21.15.26:0/588940765 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559d5a3d32f0 msgr2=0x559d5a3d7220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:50.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.722+0000 7f3574fd3740 1 --2- 172.21.15.26:0/588940765 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559d5a3d32f0 0x559d5a3d7220 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x559d5a3d3e60 tx=0x7f35580040b0 comp rx=0 tx=0).stop 2024-09-11T23:18:50.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.722+0000 7f3574fd3740 1 -- 172.21.15.26:0/588940765 shutdown_connections 2024-09-11T23:18:50.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.722+0000 7f3574fd3740 1 --2- 172.21.15.26:0/588940765 >> [v2:172.21.15.136:6838/3211398630,v1:172.21.15.136:6839/3211398630] conn(0x559d5a45d190 0x559d5a45bea0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:50.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.722+0000 7f3574fd3740 1 --2- 172.21.15.26:0/588940765 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559d5a3ddbb0 0x559d5a3d7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:50.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.722+0000 7f354bfff640 0 client.15495 ms_handle_reset on v2:172.21.15.136:6838/3211398630 2024-09-11T23:18:50.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.722+0000 7f3574fd3740 1 --2- 172.21.15.26:0/588940765 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559d5a3d32f0 0x559d5a3d7220 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:50.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.722+0000 7f3574fd3740 1 --2- 172.21.15.26:0/588940765 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559d5a3d29d0 0x559d5a3f8220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:50.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.722+0000 7f3574fd3740 1 -- 172.21.15.26:0/588940765 >> 172.21.15.26:0/588940765 conn(0x559d5a3cef60 msgr2=0x559d5a3cd7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:18:50.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.722+0000 7f3574fd3740 1 -- 172.21.15.26:0/588940765 shutdown_connections 2024-09-11T23:18:50.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.722+0000 7f3574fd3740 1 -- 172.21.15.26:0/588940765 wait complete. 2024-09-11T23:18:50.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.723+0000 7f3574fd3740 20 client.15495 trim_cache size 0 max 16384 2024-09-11T23:18:50.728 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-11T23:18:50.728 INFO:teuthology.orchestra.run.smithi026.stdout: "cluster_fsid": "af7e6fc8-7092-11ef-bcea-c7b262605968", 2024-09-11T23:18:50.728 INFO:teuthology.orchestra.run.smithi026.stdout: "whoami": 1, 2024-09-11T23:18:50.729 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 24463, 2024-09-11T23:18:50.729 INFO:teuthology.orchestra.run.smithi026.stdout: "want_state": "up:active", 2024-09-11T23:18:50.729 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "up:active", 2024-09-11T23:18:50.729 INFO:teuthology.orchestra.run.smithi026.stdout: "fs_name": "cephfs", 2024-09-11T23:18:50.729 INFO:teuthology.orchestra.run.smithi026.stdout: "rank_uptime": 136.79177328399999, 2024-09-11T23:18:50.729 INFO:teuthology.orchestra.run.smithi026.stdout: "mdsmap_epoch": 29, 2024-09-11T23:18:50.729 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch": 86, 2024-09-11T23:18:50.729 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch_barrier": 85, 2024-09-11T23:18:50.729 INFO:teuthology.orchestra.run.smithi026.stdout: "uptime": 164.500495124 2024-09-11T23:18:50.729 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-11T23:18:50.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.723+0000 7f3572bfa640 1 -- 172.21.15.26:0/1243710858 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f355c0a2d80 msgr2=0x7f355c0a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:50.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.723+0000 7f3572bfa640 1 --2- 172.21.15.26:0/1243710858 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f355c0a2d80 0x7f355c0a5240 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f3558004650 tx=0x7f355800bef0 comp rx=0 tx=0).stop 2024-09-11T23:18:50.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.723+0000 7f3572bfa640 1 -- 172.21.15.26:0/1243710858 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f356c106a80 msgr2=0x7f356c108430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:50.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.723+0000 7f3572bfa640 1 --2- 172.21.15.26:0/1243710858 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f356c106a80 0x7f356c108430 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f356400ead0 tx=0x7f356400ef90 comp rx=0 tx=0).stop 2024-09-11T23:18:50.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.723+0000 7f3572bfa640 1 -- 172.21.15.26:0/1243710858 shutdown_connections 2024-09-11T23:18:50.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.723+0000 7f3572bfa640 1 --2- 172.21.15.26:0/1243710858 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f355c0a2d80 0x7f355c0a5240 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:50.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.723+0000 7f3572bfa640 1 --2- 172.21.15.26:0/1243710858 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f356c106a80 0x7f356c108430 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:50.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.723+0000 7f3572bfa640 1 --2- 172.21.15.26:0/1243710858 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f356c101420 0x7f356c107ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:50.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.723+0000 7f3572bfa640 1 --2- 172.21.15.26:0/1243710858 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f356c100a80 0x7f356c1079b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:50.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.723+0000 7f3572bfa640 1 -- 172.21.15.26:0/1243710858 >> 172.21.15.26:0/1243710858 conn(0x7f356c0fc7a0 msgr2=0x7f356c06a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:18:50.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.723+0000 7f3572bfa640 1 -- 172.21.15.26:0/1243710858 shutdown_connections 2024-09-11T23:18:50.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:50.723+0000 7f3572bfa640 1 -- 172.21.15.26:0/1243710858 wait complete. 2024-09-11T23:18:51.421 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph tell mds.b status 2024-09-11T23:18:51.679 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:18:52.062 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:51 smithi026 ceph-mon[31172]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 2.6 KiB/s wr, 13 op/s 2024-09-11T23:18:52.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.146+0000 7f204fde9640 1 -- 172.21.15.26:0/4036018298 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20481027c0 msgr2=0x7f2048106810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:52.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.146+0000 7f204fde9640 1 --2- 172.21.15.26:0/4036018298 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20481027c0 0x7f2048106810 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f203c009900 tx=0x7f203c02a410 comp rx=0 tx=0).stop 2024-09-11T23:18:52.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.147+0000 7f204fde9640 1 -- 172.21.15.26:0/4036018298 shutdown_connections 2024-09-11T23:18:52.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.147+0000 7f204fde9640 1 --2- 172.21.15.26:0/4036018298 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2048106f40 0x7f204810eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:52.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.147+0000 7f204fde9640 1 --2- 172.21.15.26:0/4036018298 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20481027c0 0x7f2048106810 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:52.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.147+0000 7f204fde9640 1 --2- 172.21.15.26:0/4036018298 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2048101e10 0x7f20481021f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:52.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.147+0000 7f204fde9640 1 -- 172.21.15.26:0/4036018298 >> 172.21.15.26:0/4036018298 conn(0x7f20480fd8d0 msgr2=0x7f20480ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:18:52.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.147+0000 7f204fde9640 1 -- 172.21.15.26:0/4036018298 shutdown_connections 2024-09-11T23:18:52.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.147+0000 7f204fde9640 1 -- 172.21.15.26:0/4036018298 wait complete. 2024-09-11T23:18:52.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.147+0000 7f204fde9640 1 Processor -- start 2024-09-11T23:18:52.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.147+0000 7f204fde9640 1 -- start start 2024-09-11T23:18:52.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.148+0000 7f204fde9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2048101e10 0x7f20481983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:52.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.148+0000 7f204fde9640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f20481027c0 0x7f20481988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:52.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.148+0000 7f204fde9640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2048106f40 0x7f204819cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:52.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.148+0000 7f204fde9640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f2048111fa0 con 0x7f2048101e10 2024-09-11T23:18:52.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.148+0000 7f204fde9640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f2048111e20 con 0x7f20481027c0 2024-09-11T23:18:52.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.148+0000 7f204fde9640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f2048112120 con 0x7f2048106f40 2024-09-11T23:18:52.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.148+0000 7f204db85640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2048101e10 0x7f20481983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:52.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.148+0000 7f204db85640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2048101e10 0x7f20481983a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53404/0 (socket says 172.21.15.26:53404) 2024-09-11T23:18:52.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.148+0000 7f204db85640 1 -- 172.21.15.26:0/287077952 learned_addr learned my addr 172.21.15.26:0/287077952 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:18:52.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.148+0000 7f204e386640 1 --2- 172.21.15.26:0/287077952 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2048106f40 0x7f204819cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:52.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.148+0000 7f204d384640 1 --2- 172.21.15.26:0/287077952 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f20481027c0 0x7f20481988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:52.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.148+0000 7f204db85640 1 -- 172.21.15.26:0/287077952 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2048106f40 msgr2=0x7f204819cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:52.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.148+0000 7f204db85640 1 --2- 172.21.15.26:0/287077952 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2048106f40 0x7f204819cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:52.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.148+0000 7f204db85640 1 -- 172.21.15.26:0/287077952 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f20481027c0 msgr2=0x7f20481988e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:52.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.148+0000 7f204db85640 1 --2- 172.21.15.26:0/287077952 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f20481027c0 0x7f20481988e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:52.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.148+0000 7f204db85640 1 -- 172.21.15.26:0/287077952 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f204819d350 con 0x7f2048101e10 2024-09-11T23:18:52.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.148+0000 7f204d384640 1 --2- 172.21.15.26:0/287077952 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f20481027c0 0x7f20481988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:18:52.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.148+0000 7f204db85640 1 --2- 172.21.15.26:0/287077952 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2048101e10 0x7f20481983a0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f204000c920 tx=0x7f204000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:18:52.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.149+0000 7f2046ffd640 1 -- 172.21.15.26:0/287077952 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2040007c70 con 0x7f2048101e10 2024-09-11T23:18:52.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.149+0000 7f204fde9640 1 -- 172.21.15.26:0/287077952 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f204819d640 con 0x7f2048101e10 2024-09-11T23:18:52.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.149+0000 7f2046ffd640 1 -- 172.21.15.26:0/287077952 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2040007e10 con 0x7f2048101e10 2024-09-11T23:18:52.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.149+0000 7f2046ffd640 1 -- 172.21.15.26:0/287077952 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2040018890 con 0x7f2048101e10 2024-09-11T23:18:52.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.149+0000 7f204fde9640 1 -- 172.21.15.26:0/287077952 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20481a4f20 con 0x7f2048101e10 2024-09-11T23:18:52.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.151+0000 7f2046ffd640 1 -- 172.21.15.26:0/287077952 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 24) ==== 138009+0+0 (secure 0 0 0) 0x7f2040018a70 con 0x7f2048101e10 2024-09-11T23:18:52.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.151+0000 7f20521c2740 1 Processor -- start 2024-09-11T23:18:52.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.151+0000 7f20521c2740 1 -- start start 2024-09-11T23:18:52.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.151+0000 7f20521c2740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561b990bb9d0 0x561b990bbdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:52.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.151+0000 7f20521c2740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561b990bc2f0 0x561b990c6670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:52.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.151+0000 7f20521c2740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561b990c6bb0 0x561b990c8fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:52.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.151+0000 7f20521c2740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x561b99022090 con 0x561b990bb9d0 2024-09-11T23:18:52.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.152+0000 7f20521c2740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x561b98fc9a70 con 0x561b990c6bb0 2024-09-11T23:18:52.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.152+0000 7f20521c2740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x561b9901e050 con 0x561b990bc2f0 2024-09-11T23:18:52.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.152+0000 7f204e386640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561b990bb9d0 0x561b990bbdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:52.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.152+0000 7f204e386640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561b990bb9d0 0x561b990bbdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53412/0 (socket says 172.21.15.26:53412) 2024-09-11T23:18:52.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.152+0000 7f204e386640 1 -- 172.21.15.26:0/4208492760 learned_addr learned my addr 172.21.15.26:0/4208492760 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:18:52.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.152+0000 7f2046ffd640 1 --2- 172.21.15.26:0/287077952 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f20300a2d80 0x7f20300a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:52.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.152+0000 7f204db85640 1 --2- 172.21.15.26:0/4208492760 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561b990bc2f0 0x561b990c6670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:52.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.152+0000 7f204d384640 1 --2- 172.21.15.26:0/4208492760 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561b990c6bb0 0x561b990c8fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:52.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.152+0000 7f204e386640 1 -- 172.21.15.26:0/4208492760 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561b990bc2f0 msgr2=0x561b990c6670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:18:52.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.152+0000 7f204e386640 1 --2- 172.21.15.26:0/4208492760 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561b990bc2f0 0x561b990c6670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:52.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.152+0000 7f204e386640 1 -- 172.21.15.26:0/4208492760 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561b990c6bb0 msgr2=0x561b990c8fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:18:52.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.152+0000 7f204e386640 1 --2- 172.21.15.26:0/4208492760 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561b990c6bb0 0x561b990c8fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:52.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.152+0000 7f204e386640 1 -- 172.21.15.26:0/4208492760 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561b98fd6130 con 0x561b990bb9d0 2024-09-11T23:18:52.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.152+0000 7f2046ffd640 1 -- 172.21.15.26:0/287077952 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f20400cf730 con 0x7f2048101e10 2024-09-11T23:18:52.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.152+0000 7f204e386640 1 --2- 172.21.15.26:0/287077952 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f20300a2d80 0x7f20300a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:52.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.152+0000 7f204e386640 1 --2- 172.21.15.26:0/4208492760 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561b990bb9d0 0x561b990bbdb0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f2038010440 tx=0x7f2038010900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd44a486f60c0711 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:18:52.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.152+0000 7f202ffff640 1 -- 172.21.15.26:0/4208492760 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f203801d400 con 0x561b990bb9d0 2024-09-11T23:18:52.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.152+0000 7f202ffff640 1 -- 172.21.15.26:0/4208492760 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20380134c0 con 0x561b990bb9d0 2024-09-11T23:18:52.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.152+0000 7f202ffff640 1 -- 172.21.15.26:0/4208492760 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2038011860 con 0x561b990bb9d0 2024-09-11T23:18:52.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.153+0000 7f20521c2740 1 -- 172.21.15.26:0/4208492760 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561b990bb9d0 msgr2=0x561b990bbdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:52.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.153+0000 7f20521c2740 1 --2- 172.21.15.26:0/4208492760 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561b990bb9d0 0x561b990bbdb0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f2038010440 tx=0x7f2038010900 comp rx=0 tx=0).stop 2024-09-11T23:18:52.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.153+0000 7f204e386640 1 --2- 172.21.15.26:0/287077952 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f20300a2d80 0x7f20300a5240 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f2038021000 tx=0x7f203801e000 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:18:52.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.153+0000 7f20521c2740 1 -- 172.21.15.26:0/4208492760 shutdown_connections 2024-09-11T23:18:52.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.153+0000 7f20521c2740 1 --2- 172.21.15.26:0/4208492760 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561b990c6bb0 0x561b990c8fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:52.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.153+0000 7f20521c2740 1 --2- 172.21.15.26:0/4208492760 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561b990bc2f0 0x561b990c6670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:52.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.153+0000 7f20521c2740 1 --2- 172.21.15.26:0/4208492760 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561b990bb9d0 0x561b990bbdb0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:52.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.153+0000 7f20521c2740 1 -- 172.21.15.26:0/4208492760 >> 172.21.15.26:0/4208492760 conn(0x561b990b7f60 msgr2=0x561b990b8340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:18:52.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.153+0000 7f20521c2740 1 -- 172.21.15.26:0/4208492760 shutdown_connections 2024-09-11T23:18:52.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.153+0000 7f20521c2740 1 -- 172.21.15.26:0/4208492760 wait complete. 2024-09-11T23:18:52.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.154+0000 7f20521c2740 1 -- start start 2024-09-11T23:18:52.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.154+0000 7f20521c2740 1 Processor -- start 2024-09-11T23:18:52.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.154+0000 7f20521c2740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561b990bb9d0 0x561b990c64a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:52.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.154+0000 7f20521c2740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561b990bc2f0 0x561b990c37f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:52.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.154+0000 7f20521c2740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561b990c6bb0 0x561b990c3d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:52.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.154+0000 7f20521c2740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x561b990c9780 con 0x561b990bb9d0 2024-09-11T23:18:52.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.154+0000 7f20521c2740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x561b98fdcf20 con 0x561b990bc2f0 2024-09-11T23:18:52.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.154+0000 7f20521c2740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x561b990c9900 con 0x561b990c6bb0 2024-09-11T23:18:52.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.154+0000 7f20521c2740 10 client.0 _ll_register_callbacks cb 0x561b98fa6030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:18:52.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.154+0000 7f204db85640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561b990bb9d0 0x561b990c64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:52.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.154+0000 7f204db85640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561b990bb9d0 0x561b990c64a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53416/0 (socket says 172.21.15.26:53416) 2024-09-11T23:18:52.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.154+0000 7f204db85640 1 -- 172.21.15.26:0/3336922643 learned_addr learned my addr 172.21.15.26:0/3336922643 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:18:52.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.154+0000 7f204d384640 1 --2- 172.21.15.26:0/3336922643 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561b990bc2f0 0x561b990c37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:52.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.155+0000 7f204e386640 1 --2- 172.21.15.26:0/3336922643 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561b990c6bb0 0x561b990c3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:52.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.155+0000 7f204db85640 1 -- 172.21.15.26:0/3336922643 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561b990c6bb0 msgr2=0x561b990c3d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:18:52.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.155+0000 7f204db85640 1 --2- 172.21.15.26:0/3336922643 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561b990c6bb0 0x561b990c3d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:52.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.155+0000 7f204db85640 1 -- 172.21.15.26:0/3336922643 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561b990bc2f0 msgr2=0x561b990c37f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:18:52.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.155+0000 7f204db85640 1 --2- 172.21.15.26:0/3336922643 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561b990bc2f0 0x561b990c37f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:52.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.155+0000 7f204db85640 1 -- 172.21.15.26:0/3336922643 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561b98d932b0 con 0x561b990bb9d0 2024-09-11T23:18:52.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.155+0000 7f204d384640 1 --2- 172.21.15.26:0/3336922643 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561b990bc2f0 0x561b990c37f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:18:52.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.155+0000 7f204e386640 1 --2- 172.21.15.26:0/3336922643 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561b990c6bb0 0x561b990c3d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:18:52.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.155+0000 7f204db85640 1 --2- 172.21.15.26:0/3336922643 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561b990bb9d0 0x561b990c64a0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f2040097f50 tx=0x7f204009b450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b95d4b02d25d0332 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:18:52.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.155+0000 7f202d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:18:52.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.155+0000 7f202d7fa640 1 -- 172.21.15.26:0/3336922643 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f204009baf0 con 0x561b990bb9d0 2024-09-11T23:18:52.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.155+0000 7f202d7fa640 1 -- 172.21.15.26:0/3336922643 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f204000fb30 con 0x561b990bb9d0 2024-09-11T23:18:52.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.155+0000 7f20521c2740 1 -- 172.21.15.26:0/3336922643 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f20380134c0 con 0x561b990bb9d0 2024-09-11T23:18:52.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.155+0000 7f202d7fa640 1 -- 172.21.15.26:0/3336922643 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f204009fea0 con 0x561b990bb9d0 2024-09-11T23:18:52.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.155+0000 7f20521c2740 10 client.15507 fetch_fsmap learned FSMap version 29 2024-09-11T23:18:52.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.155+0000 7f20521c2740 1 -- 172.21.15.26:0/3336922643 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x561b990d50f0 con 0x561b990bb9d0 2024-09-11T23:18:52.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.155+0000 7f202d7fa640 1 -- 172.21.15.26:0/3336922643 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20380134c0 con 0x561b990bb9d0 2024-09-11T23:18:52.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.155+0000 7f202d7fa640 1 -- 172.21.15.26:0/3336922643 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f20400cfbe0 con 0x561b990bb9d0 2024-09-11T23:18:52.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.155+0000 7f20521c2740 10 client.15507 fetch_fsmap finished waiting for FSMap version 29 2024-09-11T23:18:52.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.156+0000 7f20521c2740 10 client.15507 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-11T23:18:52.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.156+0000 7f20521c2740 20 client.15507 populate_metadata read hostname 'smithi026' 2024-09-11T23:18:52.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.156+0000 7f20521c2740 1 --2- 172.21.15.26:0/3336922643 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x561b990d7720 0x561b990d8b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:52.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.156+0000 7f20521c2740 4 client.15507 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:18:52.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.156+0000 7f20521c2740 1 -- 172.21.15.26:0/3336922643 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561b9902f600 con 0x561b990d7720 2024-09-11T23:18:52.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.156+0000 7f204d384640 1 --2- 172.21.15.26:0/3336922643 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x561b990d7720 0x561b990d8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:52.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.156+0000 7f204d384640 1 --2- 172.21.15.26:0/3336922643 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x561b990d7720 0x561b990d8b90 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=36fe92b52aaa04ad server_cookie=b2093a2e2f731f1b in_seq=0 out_seq=0 2024-09-11T23:18:52.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.156+0000 7f202d7fa640 10 client.15507 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:18:52.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.157+0000 7f202d7fa640 1 -- 172.21.15.26:0/3336922643 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561b9902f600 con 0x561b990d7720 2024-09-11T23:18:52.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.157+0000 7f202d7fa640 10 client.15507 handle_command_reply: tid=0 2024-09-11T23:18:52.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.157+0000 7f20521c2740 1 client.15507 shutdown 2024-09-11T23:18:52.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.157+0000 7f20521c2740 1 -- 172.21.15.26:0/3336922643 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561b990bb9d0 msgr2=0x561b990c64a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:52.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.157+0000 7f20521c2740 1 --2- 172.21.15.26:0/3336922643 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561b990bb9d0 0x561b990c64a0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f2040097f50 tx=0x7f204009b450 comp rx=0 tx=0).stop 2024-09-11T23:18:52.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.157+0000 7f20521c2740 1 -- 172.21.15.26:0/3336922643 shutdown_connections 2024-09-11T23:18:52.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.157+0000 7f20521c2740 1 --2- 172.21.15.26:0/3336922643 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561b990bc2f0 0x561b990c37f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:52.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.157+0000 7f20521c2740 1 --2- 172.21.15.26:0/3336922643 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x561b990d7720 0x561b990d8b90 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:52.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.157+0000 7f20521c2740 1 --2- 172.21.15.26:0/3336922643 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561b990c6bb0 0x561b990c3d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:52.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.157+0000 7f20521c2740 1 --2- 172.21.15.26:0/3336922643 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561b990bb9d0 0x561b990c64a0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:52.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.157+0000 7f20521c2740 1 -- 172.21.15.26:0/3336922643 >> 172.21.15.26:0/3336922643 conn(0x561b990b7f60 msgr2=0x561b990bdc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:18:52.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.157+0000 7f202d7fa640 0 client.15507 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:18:52.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.158+0000 7f20521c2740 1 -- 172.21.15.26:0/3336922643 shutdown_connections 2024-09-11T23:18:52.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.158+0000 7f20521c2740 1 -- 172.21.15.26:0/3336922643 wait complete. 2024-09-11T23:18:52.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.158+0000 7f20521c2740 20 client.15507 trim_cache size 0 max 16384 2024-09-11T23:18:52.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.175+0000 7f20521c2740 1 Processor -- start 2024-09-11T23:18:52.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.176+0000 7f20521c2740 1 -- start start 2024-09-11T23:18:52.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.176+0000 7f20521c2740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561b990bb9d0 0x561b990d8b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:52.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.176+0000 7f20521c2740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561b990bc2f0 0x561b990e0e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:52.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.176+0000 7f20521c2740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561b990c6bb0 0x561b990e13b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:52.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.176+0000 7f20521c2740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x561b990d6ed0 con 0x561b990bb9d0 2024-09-11T23:18:52.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.176+0000 7f20521c2740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x561b990d75a0 con 0x561b990c6bb0 2024-09-11T23:18:52.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.176+0000 7f20521c2740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x561b990d7c70 con 0x561b990bc2f0 2024-09-11T23:18:52.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.176+0000 7f204db85640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561b990bb9d0 0x561b990d8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:52.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.176+0000 7f204db85640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561b990bb9d0 0x561b990d8b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53422/0 (socket says 172.21.15.26:53422) 2024-09-11T23:18:52.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.176+0000 7f204db85640 1 -- 172.21.15.26:0/3204600294 learned_addr learned my addr 172.21.15.26:0/3204600294 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:18:52.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.176+0000 7f204d384640 1 --2- 172.21.15.26:0/3204600294 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561b990bc2f0 0x561b990e0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:52.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.176+0000 7f204e386640 1 --2- 172.21.15.26:0/3204600294 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561b990c6bb0 0x561b990e13b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:52.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.176+0000 7f204db85640 1 -- 172.21.15.26:0/3204600294 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561b990bc2f0 msgr2=0x561b990e0e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:18:52.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.176+0000 7f204db85640 1 --2- 172.21.15.26:0/3204600294 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561b990bc2f0 0x561b990e0e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:52.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.176+0000 7f204db85640 1 -- 172.21.15.26:0/3204600294 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561b990c6bb0 msgr2=0x561b990e13b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:18:52.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.176+0000 7f204db85640 1 --2- 172.21.15.26:0/3204600294 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561b990c6bb0 0x561b990e13b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:52.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.176+0000 7f204db85640 1 -- 172.21.15.26:0/3204600294 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561b990bb750 con 0x561b990bb9d0 2024-09-11T23:18:52.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.176+0000 7f204d384640 1 --2- 172.21.15.26:0/3204600294 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561b990bc2f0 0x561b990e0e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:18:52.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.176+0000 7f204e386640 1 --2- 172.21.15.26:0/3204600294 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561b990c6bb0 0x561b990e13b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:18:52.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.176+0000 7f204db85640 1 --2- 172.21.15.26:0/3204600294 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561b990bb9d0 0x561b990d8b90 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f204000c8f0 tx=0x7f204000ff30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f186e90e5a459393 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:18:52.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.177+0000 7f202f7fe640 1 -- 172.21.15.26:0/3204600294 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2040097d90 con 0x561b990bb9d0 2024-09-11T23:18:52.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.177+0000 7f202f7fe640 1 -- 172.21.15.26:0/3204600294 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20400a4760 con 0x561b990bb9d0 2024-09-11T23:18:52.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.177+0000 7f202f7fe640 1 -- 172.21.15.26:0/3204600294 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20400cd1e0 con 0x561b990bb9d0 2024-09-11T23:18:52.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.177+0000 7f20521c2740 1 -- 172.21.15.26:0/3204600294 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561b990bb9d0 msgr2=0x561b990d8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:52.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.177+0000 7f20521c2740 1 --2- 172.21.15.26:0/3204600294 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561b990bb9d0 0x561b990d8b90 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f204000c8f0 tx=0x7f204000ff30 comp rx=0 tx=0).stop 2024-09-11T23:18:52.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.177+0000 7f20521c2740 1 -- 172.21.15.26:0/3204600294 shutdown_connections 2024-09-11T23:18:52.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.177+0000 7f20521c2740 1 --2- 172.21.15.26:0/3204600294 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561b990c6bb0 0x561b990e13b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:52.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.177+0000 7f20521c2740 1 --2- 172.21.15.26:0/3204600294 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561b990bc2f0 0x561b990e0e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:52.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.177+0000 7f20521c2740 1 --2- 172.21.15.26:0/3204600294 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561b990bb9d0 0x561b990d8b90 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:52.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.177+0000 7f20521c2740 1 -- 172.21.15.26:0/3204600294 >> 172.21.15.26:0/3204600294 conn(0x561b990b7f60 msgr2=0x561b990c20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:18:52.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.177+0000 7f20521c2740 1 -- 172.21.15.26:0/3204600294 shutdown_connections 2024-09-11T23:18:52.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.177+0000 7f20521c2740 1 -- 172.21.15.26:0/3204600294 wait complete. 2024-09-11T23:18:52.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.178+0000 7f20521c2740 1 -- start start 2024-09-11T23:18:52.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.178+0000 7f20521c2740 1 Processor -- start 2024-09-11T23:18:52.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.178+0000 7f20521c2740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561b990bb9d0 0x561b990dc260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:52.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.178+0000 7f20521c2740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561b990bc2f0 0x561b990dc7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:52.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.178+0000 7f20521c2740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561b990c6bb0 0x561b9913fe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:52.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.178+0000 7f20521c2740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x561b990d8a10 con 0x561b990c6bb0 2024-09-11T23:18:52.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.178+0000 7f20521c2740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x561b990d8340 con 0x561b990bb9d0 2024-09-11T23:18:52.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.178+0000 7f20521c2740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x561b990d90e0 con 0x561b990bc2f0 2024-09-11T23:18:52.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.178+0000 7f20521c2740 10 client.0 _ll_register_callbacks cb 0x7f20400ccc60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:18:52.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.178+0000 7f204e386640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561b990c6bb0 0x561b9913fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:52.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.178+0000 7f204e386640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561b990c6bb0 0x561b9913fe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53438/0 (socket says 172.21.15.26:53438) 2024-09-11T23:18:52.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.178+0000 7f204e386640 1 -- 172.21.15.26:0/2883017558 learned_addr learned my addr 172.21.15.26:0/2883017558 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:18:52.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.178+0000 7f204d384640 1 --2- 172.21.15.26:0/2883017558 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561b990bc2f0 0x561b990dc7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:52.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.179+0000 7f204db85640 1 --2- 172.21.15.26:0/2883017558 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561b990bb9d0 0x561b990dc260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:52.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.179+0000 7f204e386640 1 -- 172.21.15.26:0/2883017558 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561b990bc2f0 msgr2=0x561b990dc7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:18:52.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.179+0000 7f204e386640 1 --2- 172.21.15.26:0/2883017558 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561b990bc2f0 0x561b990dc7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:52.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.179+0000 7f204e386640 1 -- 172.21.15.26:0/2883017558 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561b990bb9d0 msgr2=0x561b990dc260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:18:52.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.179+0000 7f204e386640 1 --2- 172.21.15.26:0/2883017558 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561b990bb9d0 0x561b990dc260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:52.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.179+0000 7f204e386640 1 -- 172.21.15.26:0/2883017558 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561b99140360 con 0x561b990c6bb0 2024-09-11T23:18:52.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.179+0000 7f204db85640 1 --2- 172.21.15.26:0/2883017558 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561b990bb9d0 0x561b990dc260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:18:52.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.179+0000 7f204d384640 1 --2- 172.21.15.26:0/2883017558 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561b990bc2f0 0x561b990dc7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:18:52.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.179+0000 7f204e386640 1 --2- 172.21.15.26:0/2883017558 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561b990c6bb0 0x561b9913fe20 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f2038011010 tx=0x7f20380093c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9f9c768f342217d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:18:52.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.179+0000 7f202d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:18:52.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.179+0000 7f202d7fa640 1 -- 172.21.15.26:0/2883017558 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2038009600 con 0x561b990c6bb0 2024-09-11T23:18:52.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.179+0000 7f202d7fa640 1 -- 172.21.15.26:0/2883017558 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f2038023070 con 0x561b990c6bb0 2024-09-11T23:18:52.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.179+0000 7f20521c2740 1 -- 172.21.15.26:0/2883017558 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f20400a4760 con 0x561b990c6bb0 2024-09-11T23:18:52.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.180+0000 7f202d7fa640 1 -- 172.21.15.26:0/2883017558 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2038009a00 con 0x561b990c6bb0 2024-09-11T23:18:52.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.180+0000 7f20521c2740 10 client.15513 fetch_fsmap learned FSMap version 29 2024-09-11T23:18:52.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.180+0000 7f20521c2740 1 -- 172.21.15.26:0/2883017558 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x561b991415b0 con 0x561b990c6bb0 2024-09-11T23:18:52.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.180+0000 7f202d7fa640 1 -- 172.21.15.26:0/2883017558 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20400a4760 con 0x561b990c6bb0 2024-09-11T23:18:52.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.180+0000 7f202d7fa640 1 -- 172.21.15.26:0/2883017558 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f203800f040 con 0x561b990c6bb0 2024-09-11T23:18:52.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.180+0000 7f20521c2740 10 client.15513 fetch_fsmap finished waiting for FSMap version 29 2024-09-11T23:18:52.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.180+0000 7f20521c2740 10 client.15513 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-11T23:18:52.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.180+0000 7f20521c2740 20 client.15513 populate_metadata read hostname 'smithi026' 2024-09-11T23:18:52.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.180+0000 7f20521c2740 1 --2- 172.21.15.26:0/2883017558 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x561b99145fc0 0x561b99144cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:52.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.180+0000 7f20521c2740 4 client.15513 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-11T23:18:52.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.180+0000 7f20521c2740 1 -- 172.21.15.26:0/2883017558 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "status"}) -- 0x561b99029b10 con 0x561b99145fc0 2024-09-11T23:18:52.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.180+0000 7f204db85640 1 --2- 172.21.15.26:0/2883017558 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x561b99145fc0 0x561b99144cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:52.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.181+0000 7f204db85640 1 --2- 172.21.15.26:0/2883017558 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x561b99145fc0 0x561b99144cd0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f3c0c5c51af0c61c server_cookie=7d3051a069be828d in_seq=0 out_seq=0 2024-09-11T23:18:52.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.181+0000 7f202d7fa640 10 client.15513 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:18:52.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.181+0000 7f202d7fa640 1 -- 172.21.15.26:0/2883017558 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x561b99029b10 con 0x561b99145fc0 2024-09-11T23:18:52.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.181+0000 7f202d7fa640 10 client.15513 handle_command_reply: tid=0 2024-09-11T23:18:52.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.181+0000 7f20521c2740 1 client.15513 shutdown 2024-09-11T23:18:52.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.181+0000 7f20521c2740 1 -- 172.21.15.26:0/2883017558 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561b990c6bb0 msgr2=0x561b9913fe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:52.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.181+0000 7f20521c2740 1 --2- 172.21.15.26:0/2883017558 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561b990c6bb0 0x561b9913fe20 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f2038011010 tx=0x7f20380093c0 comp rx=0 tx=0).stop 2024-09-11T23:18:52.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.181+0000 7f20521c2740 1 -- 172.21.15.26:0/2883017558 shutdown_connections 2024-09-11T23:18:52.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.181+0000 7f20521c2740 1 --2- 172.21.15.26:0/2883017558 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x561b99145fc0 0x561b99144cd0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:52.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.181+0000 7f20521c2740 1 --2- 172.21.15.26:0/2883017558 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561b990c6bb0 0x561b9913fe20 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:52.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.181+0000 7f20521c2740 1 --2- 172.21.15.26:0/2883017558 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561b990bc2f0 0x561b990dc7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:52.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.181+0000 7f202d7fa640 0 client.15513 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:18:52.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.181+0000 7f20521c2740 1 --2- 172.21.15.26:0/2883017558 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561b990bb9d0 0x561b990dc260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:52.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.181+0000 7f20521c2740 1 -- 172.21.15.26:0/2883017558 >> 172.21.15.26:0/2883017558 conn(0x561b990b7f60 msgr2=0x561b990c24e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:18:52.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.181+0000 7f20521c2740 1 -- 172.21.15.26:0/2883017558 shutdown_connections 2024-09-11T23:18:52.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.181+0000 7f20521c2740 1 -- 172.21.15.26:0/2883017558 wait complete. 2024-09-11T23:18:52.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.181+0000 7f20521c2740 20 client.15513 trim_cache size 0 max 16384 2024-09-11T23:18:52.196 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-11T23:18:52.196 INFO:teuthology.orchestra.run.smithi026.stdout: "cluster_fsid": "af7e6fc8-7092-11ef-bcea-c7b262605968", 2024-09-11T23:18:52.196 INFO:teuthology.orchestra.run.smithi026.stdout: "whoami": 0, 2024-09-11T23:18:52.196 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 24485, 2024-09-11T23:18:52.196 INFO:teuthology.orchestra.run.smithi026.stdout: "want_state": "up:active", 2024-09-11T23:18:52.197 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "up:active", 2024-09-11T23:18:52.197 INFO:teuthology.orchestra.run.smithi026.stdout: "fs_name": "cephfs", 2024-09-11T23:18:52.197 INFO:teuthology.orchestra.run.smithi026.stdout: "rank_uptime": 145.27606942599999, 2024-09-11T23:18:52.197 INFO:teuthology.orchestra.run.smithi026.stdout: "mdsmap_epoch": 29, 2024-09-11T23:18:52.197 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch": 86, 2024-09-11T23:18:52.197 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch_barrier": 85, 2024-09-11T23:18:52.197 INFO:teuthology.orchestra.run.smithi026.stdout: "uptime": 156.09247495400001 2024-09-11T23:18:52.197 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-11T23:18:52.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.182+0000 7f204fde9640 1 -- 172.21.15.26:0/287077952 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f20300a2d80 msgr2=0x7f20300a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:52.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.182+0000 7f204fde9640 1 --2- 172.21.15.26:0/287077952 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f20300a2d80 0x7f20300a5240 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f2038021000 tx=0x7f203801e000 comp rx=0 tx=0).stop 2024-09-11T23:18:52.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.182+0000 7f204fde9640 1 -- 172.21.15.26:0/287077952 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2048101e10 msgr2=0x7f20481983a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:52.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.182+0000 7f204fde9640 1 --2- 172.21.15.26:0/287077952 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2048101e10 0x7f20481983a0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f204000c920 tx=0x7f204000cde0 comp rx=0 tx=0).stop 2024-09-11T23:18:52.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.182+0000 7f204fde9640 1 -- 172.21.15.26:0/287077952 shutdown_connections 2024-09-11T23:18:52.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.182+0000 7f204fde9640 1 --2- 172.21.15.26:0/287077952 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f20300a2d80 0x7f20300a5240 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:52.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.182+0000 7f204fde9640 1 --2- 172.21.15.26:0/287077952 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f20481027c0 0x7f20481988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:52.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.182+0000 7f204fde9640 1 --2- 172.21.15.26:0/287077952 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2048106f40 0x7f204819cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:52.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.182+0000 7f204fde9640 1 --2- 172.21.15.26:0/287077952 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2048101e10 0x7f20481983a0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:52.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.182+0000 7f204fde9640 1 -- 172.21.15.26:0/287077952 >> 172.21.15.26:0/287077952 conn(0x7f20480fd8d0 msgr2=0x7f20480fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:18:52.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.182+0000 7f204fde9640 1 -- 172.21.15.26:0/287077952 shutdown_connections 2024-09-11T23:18:52.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:18:52.182+0000 7f204fde9640 1 -- 172.21.15.26:0/287077952 wait complete. 2024-09-11T23:18:52.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:51 smithi136 ceph-mon[35121]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 2.6 KiB/s wr, 13 op/s 2024-09-11T23:18:52.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:51 smithi076 ceph-mon[35296]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 2.6 KiB/s wr, 13 op/s 2024-09-11T23:18:52.898 INFO:tasks.quiescer:Ready to start quiesce thrashing; initial delay: 15 sec 2024-09-11T23:18:54.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:53 smithi136 ceph-mon[35121]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 2.6 KiB/s wr, 13 op/s 2024-09-11T23:18:54.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:53 smithi026 ceph-mon[31172]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 2.6 KiB/s wr, 13 op/s 2024-09-11T23:18:54.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:53 smithi076 ceph-mon[35296]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 2.6 KiB/s wr, 13 op/s 2024-09-11T23:18:56.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:55 smithi136 ceph-mon[35121]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 2.6 KiB/s wr, 17 op/s 2024-09-11T23:18:56.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:55 smithi026 ceph-mon[31172]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 2.6 KiB/s wr, 17 op/s 2024-09-11T23:18:56.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:55 smithi076 ceph-mon[35296]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 2.6 KiB/s wr, 17 op/s 2024-09-11T23:18:58.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:57 smithi136 ceph-mon[35121]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 2.6 KiB/s wr, 13 op/s 2024-09-11T23:18:58.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:57 smithi026 ceph-mon[31172]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 2.6 KiB/s wr, 13 op/s 2024-09-11T23:18:58.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:57 smithi076 ceph-mon[35296]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 2.6 KiB/s wr, 13 op/s 2024-09-11T23:18:58.313 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.313+0000 7ff53da80640 1 -- 172.21.15.26:0/3659696811 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff538102cc0 msgr2=0x7ff5381030a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:58.313 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.313+0000 7ff53da80640 1 --2- 172.21.15.26:0/3659696811 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff538102cc0 0x7ff5381030a0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7ff528009900 tx=0x7ff52802a480 comp rx=0 tx=0).stop 2024-09-11T23:18:58.313 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.313+0000 7ff53da80640 1 -- 172.21.15.26:0/3659696811 shutdown_connections 2024-09-11T23:18:58.313 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.313+0000 7ff53da80640 1 --2- 172.21.15.26:0/3659696811 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff5381035e0 0x7ff53810bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:58.313 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.313+0000 7ff53da80640 1 --2- 172.21.15.26:0/3659696811 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff53810c110 0x7ff53810e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:58.313 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.313+0000 7ff53da80640 1 --2- 172.21.15.26:0/3659696811 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff538102cc0 0x7ff5381030a0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:58.313 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.313+0000 7ff53da80640 1 -- 172.21.15.26:0/3659696811 >> 172.21.15.26:0/3659696811 conn(0x7ff5380fc6b0 msgr2=0x7ff5380fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:18:58.313 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.313+0000 7ff53da80640 1 -- 172.21.15.26:0/3659696811 shutdown_connections 2024-09-11T23:18:58.313 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.313+0000 7ff53da80640 1 -- 172.21.15.26:0/3659696811 wait complete. 2024-09-11T23:18:58.315 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.313+0000 7ff53da80640 1 Processor -- start 2024-09-11T23:18:58.315 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.314+0000 7ff53da80640 1 -- start start 2024-09-11T23:18:58.316 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.314+0000 7ff53da80640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff538102cc0 0x7ff538100910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:58.316 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.314+0000 7ff53da80640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff5381035e0 0x7ff538100e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:58.316 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.314+0000 7ff53da80640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff53810c110 0x7ff538101390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:58.316 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.314+0000 7ff53da80640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff538111ef0 con 0x7ff538102cc0 2024-09-11T23:18:58.316 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.314+0000 7ff53da80640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7ff538111d70 con 0x7ff53810c110 2024-09-11T23:18:58.316 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.314+0000 7ff53da80640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ff538112070 con 0x7ff5381035e0 2024-09-11T23:18:58.316 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.314+0000 7ff5377fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff538102cc0 0x7ff538100910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:58.316 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.314+0000 7ff5377fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff538102cc0 0x7ff538100910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53448/0 (socket says 172.21.15.26:53448) 2024-09-11T23:18:58.316 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.314+0000 7ff5377fe640 1 -- 172.21.15.26:0/71349994 learned_addr learned my addr 172.21.15.26:0/71349994 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:18:58.316 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.314+0000 7ff537fff640 1 --2- 172.21.15.26:0/71349994 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff53810c110 0x7ff538101390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:58.316 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.314+0000 7ff536ffd640 1 --2- 172.21.15.26:0/71349994 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff5381035e0 0x7ff538100e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:58.316 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.314+0000 7ff5377fe640 1 -- 172.21.15.26:0/71349994 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff5381035e0 msgr2=0x7ff538100e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:58.316 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.314+0000 7ff5377fe640 1 --2- 172.21.15.26:0/71349994 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff5381035e0 0x7ff538100e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:58.317 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.314+0000 7ff5377fe640 1 -- 172.21.15.26:0/71349994 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff53810c110 msgr2=0x7ff538101390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:58.317 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.314+0000 7ff5377fe640 1 --2- 172.21.15.26:0/71349994 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff53810c110 0x7ff538101390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:58.317 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.314+0000 7ff5377fe640 1 -- 172.21.15.26:0/71349994 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5381a6860 con 0x7ff538102cc0 2024-09-11T23:18:58.317 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.315+0000 7ff5377fe640 1 --2- 172.21.15.26:0/71349994 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff538102cc0 0x7ff538100910 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7ff528006010 tx=0x7ff5280042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:18:58.317 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.315+0000 7ff534ff9640 1 -- 172.21.15.26:0/71349994 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff52803f030 con 0x7ff538102cc0 2024-09-11T23:18:58.317 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.315+0000 7ff534ff9640 1 -- 172.21.15.26:0/71349994 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff528003ce0 con 0x7ff538102cc0 2024-09-11T23:18:58.317 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.315+0000 7ff53da80640 1 -- 172.21.15.26:0/71349994 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5381a6af0 con 0x7ff538102cc0 2024-09-11T23:18:58.317 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.315+0000 7ff534ff9640 1 -- 172.21.15.26:0/71349994 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff528035070 con 0x7ff538102cc0 2024-09-11T23:18:58.317 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.315+0000 7ff53da80640 1 -- 172.21.15.26:0/71349994 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5381a6fe0 con 0x7ff538102cc0 2024-09-11T23:18:58.322 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.317+0000 7ff534ff9640 1 -- 172.21.15.26:0/71349994 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 24) ==== 138009+0+0 (secure 0 0 0) 0x7ff528042030 con 0x7ff538102cc0 2024-09-11T23:18:58.322 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.317+0000 7ff54080e740 1 Processor -- start 2024-09-11T23:18:58.322 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.317+0000 7ff54080e740 1 -- start start 2024-09-11T23:18:58.322 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.318+0000 7ff54080e740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c10e801af0 0x55c10e801ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:58.322 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.318+0000 7ff54080e740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c10e802410 0x55c10e80c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:58.323 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.318+0000 7ff54080e740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c10e80ccb0 0x55c10e80f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:58.323 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.318+0000 7ff54080e740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55c10e7662f0 con 0x55c10e802410 2024-09-11T23:18:58.323 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.318+0000 7ff54080e740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55c10e70e270 con 0x55c10e801af0 2024-09-11T23:18:58.323 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.318+0000 7ff54080e740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55c10e7622b0 con 0x55c10e80ccb0 2024-09-11T23:18:58.323 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.318+0000 7ff5377fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c10e802410 0x55c10e80c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:58.323 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.318+0000 7ff5377fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c10e802410 0x55c10e80c770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53462/0 (socket says 172.21.15.26:53462) 2024-09-11T23:18:58.323 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.318+0000 7ff5377fe640 1 -- 172.21.15.26:0/1191274302 learned_addr learned my addr 172.21.15.26:0/1191274302 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:18:58.323 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.318+0000 7ff536ffd640 1 --2- 172.21.15.26:0/1191274302 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c10e80ccb0 0x55c10e80f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:58.323 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.318+0000 7ff537fff640 1 --2- 172.21.15.26:0/1191274302 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c10e801af0 0x55c10e801ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:58.323 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.318+0000 7ff5377fe640 1 -- 172.21.15.26:0/1191274302 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c10e80ccb0 msgr2=0x55c10e80f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:18:58.323 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.318+0000 7ff5377fe640 1 --2- 172.21.15.26:0/1191274302 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c10e80ccb0 0x55c10e80f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:58.323 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.318+0000 7ff5377fe640 1 -- 172.21.15.26:0/1191274302 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c10e801af0 msgr2=0x55c10e801ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:18:58.323 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.318+0000 7ff5377fe640 1 --2- 172.21.15.26:0/1191274302 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c10e801af0 0x55c10e801ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:58.324 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.318+0000 7ff5377fe640 1 -- 172.21.15.26:0/1191274302 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c10e70d340 con 0x55c10e802410 2024-09-11T23:18:58.324 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.318+0000 7ff536ffd640 1 --2- 172.21.15.26:0/1191274302 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c10e80ccb0 0x55c10e80f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:18:58.324 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.318+0000 7ff534ff9640 1 --2- 172.21.15.26:0/71349994 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7ff5200a2e40 0x7ff5200a5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:58.324 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.319+0000 7ff5377fe640 1 --2- 172.21.15.26:0/1191274302 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c10e802410 0x55c10e80c770 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7ff5280ed200 tx=0x7ff5280f3d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c14be4abb60c7bb server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:18:58.324 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.319+0000 7ff537fff640 1 --2- 172.21.15.26:0/71349994 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7ff5200a2e40 0x7ff5200a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:58.324 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.319+0000 7ff534ff9640 1 -- 172.21.15.26:0/71349994 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7ff5280efbc0 con 0x7ff538102cc0 2024-09-11T23:18:58.324 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.319+0000 7ff51dffb640 1 -- 172.21.15.26:0/1191274302 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5280b82e0 con 0x55c10e802410 2024-09-11T23:18:58.324 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.319+0000 7ff51dffb640 1 -- 172.21.15.26:0/1191274302 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5280bc740 con 0x55c10e802410 2024-09-11T23:18:58.324 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.319+0000 7ff51dffb640 1 -- 172.21.15.26:0/1191274302 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5280044e0 con 0x55c10e802410 2024-09-11T23:18:58.324 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.320+0000 7ff54080e740 1 -- 172.21.15.26:0/1191274302 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c10e802410 msgr2=0x55c10e80c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:58.324 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.320+0000 7ff537fff640 1 --2- 172.21.15.26:0/71349994 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7ff5200a2e40 0x7ff5200a5300 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ff530005fd0 tx=0x7ff530009500 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:18:58.324 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.320+0000 7ff54080e740 1 --2- 172.21.15.26:0/1191274302 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c10e802410 0x55c10e80c770 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7ff5280ed200 tx=0x7ff5280f3d20 comp rx=0 tx=0).stop 2024-09-11T23:18:58.325 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.320+0000 7ff54080e740 1 -- 172.21.15.26:0/1191274302 shutdown_connections 2024-09-11T23:18:58.325 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.320+0000 7ff54080e740 1 --2- 172.21.15.26:0/1191274302 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c10e80ccb0 0x55c10e80f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:58.325 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.320+0000 7ff54080e740 1 --2- 172.21.15.26:0/1191274302 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c10e802410 0x55c10e80c770 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:58.325 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.320+0000 7ff54080e740 1 --2- 172.21.15.26:0/1191274302 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c10e801af0 0x55c10e801ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:58.325 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.320+0000 7ff54080e740 1 -- 172.21.15.26:0/1191274302 >> 172.21.15.26:0/1191274302 conn(0x55c10e7fe060 msgr2=0x55c10e7fe440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:18:58.325 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.320+0000 7ff54080e740 1 -- 172.21.15.26:0/1191274302 shutdown_connections 2024-09-11T23:18:58.325 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.320+0000 7ff54080e740 1 -- 172.21.15.26:0/1191274302 wait complete. 2024-09-11T23:18:58.325 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.320+0000 7ff54080e740 1 -- start start 2024-09-11T23:18:58.325 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.320+0000 7ff54080e740 1 Processor -- start 2024-09-11T23:18:58.325 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.321+0000 7ff54080e740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c10e801af0 0x55c10e809c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:58.325 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.321+0000 7ff54080e740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c10e802410 0x55c10e80a170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:58.325 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.321+0000 7ff54080e740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c10e80ccb0 0x55c10e80a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:58.325 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.321+0000 7ff54080e740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55c10e80f640 con 0x55c10e80ccb0 2024-09-11T23:18:58.326 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.321+0000 7ff54080e740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55c10e6ef290 con 0x55c10e802410 2024-09-11T23:18:58.326 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.321+0000 7ff54080e740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55c10e80f7c0 con 0x55c10e801af0 2024-09-11T23:18:58.326 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.321+0000 7ff54080e740 10 client.0 _ll_register_callbacks cb 0x55c10e720140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:18:58.326 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.321+0000 7ff537fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c10e80ccb0 0x55c10e80a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:58.326 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.321+0000 7ff537fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c10e80ccb0 0x55c10e80a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53464/0 (socket says 172.21.15.26:53464) 2024-09-11T23:18:58.326 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.321+0000 7ff537fff640 1 -- 172.21.15.26:0/492461161 learned_addr learned my addr 172.21.15.26:0/492461161 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:18:58.326 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.321+0000 7ff5377fe640 1 --2- 172.21.15.26:0/492461161 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c10e801af0 0x55c10e809c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:58.326 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.321+0000 7ff536ffd640 1 --2- 172.21.15.26:0/492461161 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c10e802410 0x55c10e80a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:58.326 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.321+0000 7ff537fff640 1 -- 172.21.15.26:0/492461161 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c10e801af0 msgr2=0x55c10e809c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:18:58.326 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.321+0000 7ff537fff640 1 --2- 172.21.15.26:0/492461161 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c10e801af0 0x55c10e809c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:58.326 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.321+0000 7ff537fff640 1 -- 172.21.15.26:0/492461161 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c10e802410 msgr2=0x55c10e80a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:18:58.326 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.321+0000 7ff537fff640 1 --2- 172.21.15.26:0/492461161 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c10e802410 0x55c10e80a170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:58.326 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.321+0000 7ff537fff640 1 -- 172.21.15.26:0/492461161 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c10e479210 con 0x55c10e80ccb0 2024-09-11T23:18:58.327 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.321+0000 7ff537fff640 1 --2- 172.21.15.26:0/492461161 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c10e80ccb0 0x55c10e80a6b0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7ff5300099c0 tx=0x7ff530004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99f3d3d345e2df19 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:18:58.327 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.321+0000 7ff516ffd640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:18:58.327 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.321+0000 7ff516ffd640 1 -- 172.21.15.26:0/492461161 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff530004b20 con 0x55c10e80ccb0 2024-09-11T23:18:58.327 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.322+0000 7ff516ffd640 1 -- 172.21.15.26:0/492461161 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7ff530004cc0 con 0x55c10e80ccb0 2024-09-11T23:18:58.327 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.322+0000 7ff54080e740 1 -- 172.21.15.26:0/492461161 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff5280bc740 con 0x55c10e80ccb0 2024-09-11T23:18:58.327 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.322+0000 7ff516ffd640 1 -- 172.21.15.26:0/492461161 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff53000f7d0 con 0x55c10e80ccb0 2024-09-11T23:18:58.327 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.322+0000 7ff54080e740 10 client.15525 fetch_fsmap learned FSMap version 29 2024-09-11T23:18:58.327 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.322+0000 7ff54080e740 1 -- 172.21.15.26:0/492461161 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c10e545620 con 0x55c10e80ccb0 2024-09-11T23:18:58.327 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.322+0000 7ff516ffd640 1 -- 172.21.15.26:0/492461161 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5280bc740 con 0x55c10e80ccb0 2024-09-11T23:18:58.327 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.322+0000 7ff516ffd640 1 -- 172.21.15.26:0/492461161 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7ff530026070 con 0x55c10e80ccb0 2024-09-11T23:18:58.327 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.322+0000 7ff54080e740 10 client.15525 fetch_fsmap finished waiting for FSMap version 29 2024-09-11T23:18:58.327 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.322+0000 7ff54080e740 10 client.15525 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:18:58.327 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.322+0000 7ff54080e740 20 client.15525 populate_metadata read hostname 'smithi026' 2024-09-11T23:18:58.328 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.322+0000 7ff54080e740 1 --2- 172.21.15.26:0/492461161 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55c10e81cab0 0x55c10e81eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:58.328 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.322+0000 7ff54080e740 4 client.15525 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:18:58.328 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.322+0000 7ff54080e740 1 -- 172.21.15.26:0/492461161 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c10e773a20 con 0x55c10e81cab0 2024-09-11T23:18:58.328 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.323+0000 7ff5377fe640 1 --2- 172.21.15.26:0/492461161 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55c10e81cab0 0x55c10e81eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:58.329 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.323+0000 7ff5377fe640 1 --2- 172.21.15.26:0/492461161 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55c10e81cab0 0x55c10e81eea0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d039b61eb8e3a664 server_cookie=2eeb98869f279dce in_seq=0 out_seq=0 2024-09-11T23:18:58.329 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.323+0000 7ff516ffd640 10 client.15525 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:18:58.329 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.324+0000 7ff516ffd640 1 -- 172.21.15.26:0/492461161 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c10e773a20 con 0x55c10e81cab0 2024-09-11T23:18:58.329 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.324+0000 7ff516ffd640 10 client.15525 handle_command_reply: tid=0 2024-09-11T23:18:58.329 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.324+0000 7ff54080e740 1 client.15525 shutdown 2024-09-11T23:18:58.329 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.324+0000 7ff54080e740 1 -- 172.21.15.26:0/492461161 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c10e80ccb0 msgr2=0x55c10e80a6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:58.329 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.324+0000 7ff54080e740 1 --2- 172.21.15.26:0/492461161 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c10e80ccb0 0x55c10e80a6b0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7ff5300099c0 tx=0x7ff530004ab0 comp rx=0 tx=0).stop 2024-09-11T23:18:58.330 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.324+0000 7ff54080e740 1 -- 172.21.15.26:0/492461161 shutdown_connections 2024-09-11T23:18:58.330 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.324+0000 7ff54080e740 1 --2- 172.21.15.26:0/492461161 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c10e802410 0x55c10e80a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:58.330 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.324+0000 7ff54080e740 1 --2- 172.21.15.26:0/492461161 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55c10e81cab0 0x55c10e81eea0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:58.330 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.324+0000 7ff54080e740 1 --2- 172.21.15.26:0/492461161 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c10e80ccb0 0x55c10e80a6b0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:58.330 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.324+0000 7ff54080e740 1 --2- 172.21.15.26:0/492461161 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c10e801af0 0x55c10e809c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:58.330 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.324+0000 7ff516ffd640 0 client.15525 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:18:58.330 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.324+0000 7ff54080e740 1 -- 172.21.15.26:0/492461161 >> 172.21.15.26:0/492461161 conn(0x55c10e7fe060 msgr2=0x55c10e803dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:18:58.330 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.324+0000 7ff54080e740 1 -- 172.21.15.26:0/492461161 shutdown_connections 2024-09-11T23:18:58.330 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.324+0000 7ff54080e740 1 -- 172.21.15.26:0/492461161 wait complete. 2024-09-11T23:18:58.330 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.324+0000 7ff54080e740 20 client.15525 trim_cache size 0 max 16384 2024-09-11T23:18:58.344 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.339+0000 7ff54080e740 1 Processor -- start 2024-09-11T23:18:58.344 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.339+0000 7ff54080e740 1 -- start start 2024-09-11T23:18:58.344 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.339+0000 7ff54080e740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c10e801af0 0x55c10e80a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:58.344 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.339+0000 7ff54080e740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c10e802410 0x55c10e824790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:58.344 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.339+0000 7ff54080e740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c10e80ccb0 0x55c10e824cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:58.344 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.339+0000 7ff54080e740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff5381afa40 con 0x55c10e802410 2024-09-11T23:18:58.344 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.339+0000 7ff54080e740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7ff5381b1580 con 0x55c10e80ccb0 2024-09-11T23:18:58.344 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.339+0000 7ff54080e740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ff5381b0110 con 0x55c10e801af0 2024-09-11T23:18:58.344 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.339+0000 7ff536ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c10e802410 0x55c10e824790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:58.344 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.339+0000 7ff536ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c10e802410 0x55c10e824790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53470/0 (socket says 172.21.15.26:53470) 2024-09-11T23:18:58.345 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.339+0000 7ff536ffd640 1 -- 172.21.15.26:0/1438581934 learned_addr learned my addr 172.21.15.26:0/1438581934 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:18:58.345 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.339+0000 7ff537fff640 1 --2- 172.21.15.26:0/1438581934 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c10e80ccb0 0x55c10e824cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:58.345 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.339+0000 7ff536ffd640 1 -- 172.21.15.26:0/1438581934 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c10e801af0 msgr2=0x55c10e80a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:18:58.345 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.339+0000 7ff5377fe640 1 --2- 172.21.15.26:0/1438581934 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c10e801af0 0x55c10e80a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:58.345 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.339+0000 7ff536ffd640 1 --2- 172.21.15.26:0/1438581934 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c10e801af0 0x55c10e80a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:58.345 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.339+0000 7ff536ffd640 1 -- 172.21.15.26:0/1438581934 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c10e80ccb0 msgr2=0x55c10e824cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:18:58.345 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.339+0000 7ff536ffd640 1 --2- 172.21.15.26:0/1438581934 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c10e80ccb0 0x55c10e824cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:58.345 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.340+0000 7ff536ffd640 1 -- 172.21.15.26:0/1438581934 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c10e819360 con 0x55c10e802410 2024-09-11T23:18:58.345 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.340+0000 7ff537fff640 1 --2- 172.21.15.26:0/1438581934 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c10e80ccb0 0x55c10e824cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:18:58.345 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.340+0000 7ff5377fe640 1 --2- 172.21.15.26:0/1438581934 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c10e801af0 0x55c10e80a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:18:58.345 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.340+0000 7ff536ffd640 1 --2- 172.21.15.26:0/1438581934 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c10e802410 0x55c10e824790 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7ff5381023a0 tx=0x7ff52c003e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23d7dbab51cea87e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:18:58.345 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.340+0000 7ff51d7fa640 1 -- 172.21.15.26:0/1438581934 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff52c0129c0 con 0x55c10e802410 2024-09-11T23:18:58.345 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.340+0000 7ff51d7fa640 1 -- 172.21.15.26:0/1438581934 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff52c004c80 con 0x55c10e802410 2024-09-11T23:18:58.346 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.340+0000 7ff51d7fa640 1 -- 172.21.15.26:0/1438581934 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff52c005510 con 0x55c10e802410 2024-09-11T23:18:58.346 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.340+0000 7ff54080e740 1 -- 172.21.15.26:0/1438581934 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c10e802410 msgr2=0x55c10e824790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:58.346 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.340+0000 7ff54080e740 1 --2- 172.21.15.26:0/1438581934 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c10e802410 0x55c10e824790 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7ff5381023a0 tx=0x7ff52c003e00 comp rx=0 tx=0).stop 2024-09-11T23:18:58.346 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.340+0000 7ff54080e740 1 -- 172.21.15.26:0/1438581934 shutdown_connections 2024-09-11T23:18:58.346 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.340+0000 7ff54080e740 1 --2- 172.21.15.26:0/1438581934 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c10e80ccb0 0x55c10e824cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:58.346 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.340+0000 7ff54080e740 1 --2- 172.21.15.26:0/1438581934 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c10e802410 0x55c10e824790 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:58.346 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.340+0000 7ff54080e740 1 --2- 172.21.15.26:0/1438581934 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c10e801af0 0x55c10e80a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:58.346 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.340+0000 7ff54080e740 1 -- 172.21.15.26:0/1438581934 >> 172.21.15.26:0/1438581934 conn(0x55c10e7fe060 msgr2=0x55c10e80bcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:18:58.346 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.341+0000 7ff54080e740 1 -- 172.21.15.26:0/1438581934 shutdown_connections 2024-09-11T23:18:58.346 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.341+0000 7ff54080e740 1 -- 172.21.15.26:0/1438581934 wait complete. 2024-09-11T23:18:58.346 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.341+0000 7ff54080e740 1 -- start start 2024-09-11T23:18:58.346 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.341+0000 7ff54080e740 1 Processor -- start 2024-09-11T23:18:58.346 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.341+0000 7ff54080e740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c10e801af0 0x55c10e8107c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:58.347 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.341+0000 7ff54080e740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c10e802410 0x55c10e810d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:58.347 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.341+0000 7ff54080e740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c10e80ccb0 0x55c10e811240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:58.347 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.341+0000 7ff54080e740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff5381b0eb0 con 0x55c10e80ccb0 2024-09-11T23:18:58.347 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.341+0000 7ff54080e740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7ff5381b07e0 con 0x55c10e801af0 2024-09-11T23:18:58.347 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.341+0000 7ff54080e740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ff5381ada20 con 0x55c10e802410 2024-09-11T23:18:58.347 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.341+0000 7ff54080e740 10 client.0 _ll_register_callbacks cb 0x7ff530009e40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:18:58.347 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.342+0000 7ff537fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c10e80ccb0 0x55c10e811240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:58.347 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.342+0000 7ff537fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c10e80ccb0 0x55c10e811240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53484/0 (socket says 172.21.15.26:53484) 2024-09-11T23:18:58.348 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.342+0000 7ff537fff640 1 -- 172.21.15.26:0/4125648602 learned_addr learned my addr 172.21.15.26:0/4125648602 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:18:58.348 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.342+0000 7ff536ffd640 1 --2- 172.21.15.26:0/4125648602 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c10e802410 0x55c10e810d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:58.348 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.342+0000 7ff5377fe640 1 --2- 172.21.15.26:0/4125648602 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c10e801af0 0x55c10e8107c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:58.348 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.342+0000 7ff537fff640 1 -- 172.21.15.26:0/4125648602 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c10e802410 msgr2=0x55c10e810d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:18:58.348 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.342+0000 7ff537fff640 1 --2- 172.21.15.26:0/4125648602 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c10e802410 0x55c10e810d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:58.348 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.342+0000 7ff537fff640 1 -- 172.21.15.26:0/4125648602 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c10e801af0 msgr2=0x55c10e8107c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:18:58.348 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.342+0000 7ff537fff640 1 --2- 172.21.15.26:0/4125648602 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c10e801af0 0x55c10e8107c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:58.348 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.342+0000 7ff537fff640 1 -- 172.21.15.26:0/4125648602 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c10e8867e0 con 0x55c10e80ccb0 2024-09-11T23:18:58.348 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.342+0000 7ff5377fe640 1 --2- 172.21.15.26:0/4125648602 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c10e801af0 0x55c10e8107c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:18:58.348 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.342+0000 7ff537fff640 1 --2- 172.21.15.26:0/4125648602 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c10e80ccb0 0x55c10e811240 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7ff530005a40 tx=0x7ff530009570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4350adf3be84925 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:18:58.348 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.342+0000 7ff516ffd640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:18:58.348 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.342+0000 7ff516ffd640 1 -- 172.21.15.26:0/4125648602 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff53000fb70 con 0x55c10e80ccb0 2024-09-11T23:18:58.349 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.342+0000 7ff516ffd640 1 -- 172.21.15.26:0/4125648602 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7ff53000fd10 con 0x55c10e80ccb0 2024-09-11T23:18:58.349 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.342+0000 7ff54080e740 1 -- 172.21.15.26:0/4125648602 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff52c004c80 con 0x55c10e80ccb0 2024-09-11T23:18:58.349 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.342+0000 7ff516ffd640 1 -- 172.21.15.26:0/4125648602 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff53003f050 con 0x55c10e80ccb0 2024-09-11T23:18:58.349 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.342+0000 7ff54080e740 10 client.15531 fetch_fsmap learned FSMap version 29 2024-09-11T23:18:58.349 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.342+0000 7ff54080e740 1 -- 172.21.15.26:0/4125648602 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c10e887a20 con 0x55c10e80ccb0 2024-09-11T23:18:58.349 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.343+0000 7ff516ffd640 1 -- 172.21.15.26:0/4125648602 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff52c004c80 con 0x55c10e80ccb0 2024-09-11T23:18:58.349 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.343+0000 7ff516ffd640 1 -- 172.21.15.26:0/4125648602 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7ff530026070 con 0x55c10e80ccb0 2024-09-11T23:18:58.349 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.343+0000 7ff54080e740 10 client.15531 fetch_fsmap finished waiting for FSMap version 29 2024-09-11T23:18:58.349 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.343+0000 7ff54080e740 10 client.15531 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:18:58.349 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.343+0000 7ff54080e740 20 client.15531 populate_metadata read hostname 'smithi026' 2024-09-11T23:18:58.349 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.343+0000 7ff54080e740 1 --2- 172.21.15.26:0/4125648602 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55c10e8880b0 0x55c10e88a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:58.349 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.343+0000 7ff54080e740 4 client.15531 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T23:18:58.349 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.343+0000 7ff54080e740 1 -- 172.21.15.26:0/4125648602 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c10e76dd70 con 0x55c10e8880b0 2024-09-11T23:18:58.349 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.343+0000 7ff5377fe640 1 --2- 172.21.15.26:0/4125648602 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55c10e8880b0 0x55c10e88a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:58.350 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.343+0000 7ff5377fe640 1 --2- 172.21.15.26:0/4125648602 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55c10e8880b0 0x55c10e88a4f0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e801d25cff302600 server_cookie=c7fdb710a409023 in_seq=0 out_seq=0 2024-09-11T23:18:58.350 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.343+0000 7ff516ffd640 10 client.15531 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:18:58.351 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.349+0000 7ff516ffd640 1 -- 172.21.15.26:0/4125648602 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c10e76dd70 con 0x55c10e8880b0 2024-09-11T23:18:58.351 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.349+0000 7ff516ffd640 10 client.15531 handle_command_reply: tid=0 2024-09-11T23:18:58.351 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.349+0000 7ff54080e740 1 client.15531 shutdown 2024-09-11T23:18:58.351 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.349+0000 7ff54080e740 1 -- 172.21.15.26:0/4125648602 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c10e80ccb0 msgr2=0x55c10e811240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:58.351 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.349+0000 7ff54080e740 1 --2- 172.21.15.26:0/4125648602 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c10e80ccb0 0x55c10e811240 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7ff530005a40 tx=0x7ff530009570 comp rx=0 tx=0).stop 2024-09-11T23:18:58.351 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.349+0000 7ff54080e740 1 -- 172.21.15.26:0/4125648602 shutdown_connections 2024-09-11T23:18:58.351 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.349+0000 7ff54080e740 1 --2- 172.21.15.26:0/4125648602 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55c10e8880b0 0x55c10e88a4f0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:58.351 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.349+0000 7ff54080e740 1 --2- 172.21.15.26:0/4125648602 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c10e80ccb0 0x55c10e811240 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:58.351 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.349+0000 7ff54080e740 1 --2- 172.21.15.26:0/4125648602 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c10e802410 0x55c10e810d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:58.351 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.349+0000 7ff516ffd640 0 client.15531 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:18:58.352 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.349+0000 7ff54080e740 1 --2- 172.21.15.26:0/4125648602 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c10e801af0 0x55c10e8107c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:58.352 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.349+0000 7ff54080e740 1 -- 172.21.15.26:0/4125648602 >> 172.21.15.26:0/4125648602 conn(0x55c10e7fe060 msgr2=0x55c10e8035a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:18:58.352 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.350+0000 7ff54080e740 1 -- 172.21.15.26:0/4125648602 shutdown_connections 2024-09-11T23:18:58.352 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.350+0000 7ff54080e740 1 -- 172.21.15.26:0/4125648602 wait complete. 2024-09-11T23:18:58.352 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.350+0000 7ff54080e740 20 client.15531 trim_cache size 0 max 16384 2024-09-11T23:18:58.352 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-09-11T23:18:58.352 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "message": "", 2024-09-11T23:18:58.352 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-09-11T23:18:58.352 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-09-11T23:18:58.353 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.350+0000 7ff53da80640 1 -- 172.21.15.26:0/71349994 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7ff5200a2e40 msgr2=0x7ff5200a5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:58.353 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.350+0000 7ff53da80640 1 --2- 172.21.15.26:0/71349994 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7ff5200a2e40 0x7ff5200a5300 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ff530005fd0 tx=0x7ff530009500 comp rx=0 tx=0).stop 2024-09-11T23:18:58.353 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.350+0000 7ff53da80640 1 -- 172.21.15.26:0/71349994 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff538102cc0 msgr2=0x7ff538100910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:58.353 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.350+0000 7ff53da80640 1 --2- 172.21.15.26:0/71349994 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff538102cc0 0x7ff538100910 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7ff528006010 tx=0x7ff5280042f0 comp rx=0 tx=0).stop 2024-09-11T23:18:58.353 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.350+0000 7ff53da80640 1 -- 172.21.15.26:0/71349994 shutdown_connections 2024-09-11T23:18:58.353 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.350+0000 7ff53da80640 1 --2- 172.21.15.26:0/71349994 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7ff5200a2e40 0x7ff5200a5300 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:58.353 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.350+0000 7ff53da80640 1 --2- 172.21.15.26:0/71349994 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff53810c110 0x7ff538101390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:58.354 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.350+0000 7ff53da80640 1 --2- 172.21.15.26:0/71349994 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff5381035e0 0x7ff538100e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:58.354 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.350+0000 7ff53da80640 1 --2- 172.21.15.26:0/71349994 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff538102cc0 0x7ff538100910 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:58.354 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.350+0000 7ff53da80640 1 -- 172.21.15.26:0/71349994 >> 172.21.15.26:0/71349994 conn(0x7ff5380fc6b0 msgr2=0x7ff5380fd2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:18:58.354 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.350+0000 7ff53da80640 1 -- 172.21.15.26:0/71349994 shutdown_connections 2024-09-11T23:18:58.354 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:18:58.350+0000 7ff53da80640 1 -- 172.21.15.26:0/71349994 wait complete. 2024-09-11T23:19:00.225 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:18:59 smithi026 ceph-mon[31172]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 2.6 KiB/s wr, 17 op/s 2024-09-11T23:19:00.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:18:59 smithi136 ceph-mon[35121]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 2.6 KiB/s wr, 17 op/s 2024-09-11T23:19:00.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:18:59 smithi076 ceph-mon[35296]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 2.6 KiB/s wr, 17 op/s 2024-09-11T23:19:00.500 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:19:00 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: 2024-09-11T23:19:00.224+0000 7f7c79ae7640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T23:19:00.501 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:19:00 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: 2024-09-11T23:19:00.224+0000 7f7c79ae7640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T23:19:00.501 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:19:00 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: 2024-09-11T23:19:00.224+0000 7f7c79ae7640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T23:19:00.501 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:19:00 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: 2024-09-11T23:19:00.224+0000 7f7c79ae7640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T23:19:00.501 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:19:00 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: 2024-09-11T23:19:00.224+0000 7f7c79ae7640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T23:19:02.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:19:01 smithi136 ceph-mon[35121]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T23:19:02.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:19:01 smithi026 ceph-mon[31172]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T23:19:02.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:19:01 smithi076 ceph-mon[35296]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T23:19:02.576 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T23:19:02.867 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:19:03.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.377+0000 7fb2eb582640 1 -- 172.21.15.26:0/3742166827 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb2e41036e0 msgr2=0x7fb2e4107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:03.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.377+0000 7fb2eb582640 1 --2- 172.21.15.26:0/3742166827 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb2e41036e0 0x7fb2e4107b90 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fb2d4009930 tx=0x7fb2d402a420 comp rx=0 tx=0).stop 2024-09-11T23:19:03.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.377+0000 7fb2eb582640 1 -- 172.21.15.26:0/3742166827 shutdown_connections 2024-09-11T23:19:03.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.377+0000 7fb2eb582640 1 --2- 172.21.15.26:0/3742166827 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb2e41082a0 0x7fb2e410ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:03.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.377+0000 7fb2eb582640 1 --2- 172.21.15.26:0/3742166827 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb2e41036e0 0x7fb2e4107b90 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:03.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.377+0000 7fb2eb582640 1 --2- 172.21.15.26:0/3742166827 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb2e4102d30 0x7fb2e4103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:03.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.377+0000 7fb2eb582640 1 -- 172.21.15.26:0/3742166827 >> 172.21.15.26:0/3742166827 conn(0x7fb2e40fc7d0 msgr2=0x7fb2e40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:19:03.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.377+0000 7fb2eb582640 1 -- 172.21.15.26:0/3742166827 shutdown_connections 2024-09-11T23:19:03.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.377+0000 7fb2eb582640 1 -- 172.21.15.26:0/3742166827 wait complete. 2024-09-11T23:19:03.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.378+0000 7fb2eb582640 1 Processor -- start 2024-09-11T23:19:03.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.378+0000 7fb2eb582640 1 -- start start 2024-09-11T23:19:03.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.378+0000 7fb2eb582640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb2e4102d30 0x7fb2e41985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:03.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.378+0000 7fb2eb582640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb2e41036e0 0x7fb2e4198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:03.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.378+0000 7fb2eb582640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb2e41082a0 0x7fb2e419ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:03.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.378+0000 7fb2eb582640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb2e41120a0 con 0x7fb2e41036e0 2024-09-11T23:19:03.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.378+0000 7fb2eb582640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fb2e4111f20 con 0x7fb2e41082a0 2024-09-11T23:19:03.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.378+0000 7fb2eb582640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb2e4112220 con 0x7fb2e4102d30 2024-09-11T23:19:03.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.378+0000 7fb2e8b1d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb2e41036e0 0x7fb2e4198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:03.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.378+0000 7fb2e8b1d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb2e41036e0 0x7fb2e4198af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33200/0 (socket says 172.21.15.26:33200) 2024-09-11T23:19:03.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.378+0000 7fb2e8b1d640 1 -- 172.21.15.26:0/4199696979 learned_addr learned my addr 172.21.15.26:0/4199696979 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:19:03.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.378+0000 7fb2e9b1f640 1 --2- 172.21.15.26:0/4199696979 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb2e41082a0 0x7fb2e419ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:03.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.378+0000 7fb2e931e640 1 --2- 172.21.15.26:0/4199696979 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb2e4102d30 0x7fb2e41985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:03.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.379+0000 7fb2e8b1d640 1 -- 172.21.15.26:0/4199696979 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb2e4102d30 msgr2=0x7fb2e41985b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:03.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.379+0000 7fb2e8b1d640 1 --2- 172.21.15.26:0/4199696979 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb2e4102d30 0x7fb2e41985b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:03.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.379+0000 7fb2e8b1d640 1 -- 172.21.15.26:0/4199696979 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb2e41082a0 msgr2=0x7fb2e419ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:03.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.379+0000 7fb2e8b1d640 1 --2- 172.21.15.26:0/4199696979 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb2e41082a0 0x7fb2e419ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:03.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.379+0000 7fb2e8b1d640 1 -- 172.21.15.26:0/4199696979 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2e419d560 con 0x7fb2e41036e0 2024-09-11T23:19:03.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.379+0000 7fb2e931e640 1 --2- 172.21.15.26:0/4199696979 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb2e4102d30 0x7fb2e41985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:19:03.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.379+0000 7fb2e8b1d640 1 --2- 172.21.15.26:0/4199696979 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb2e41036e0 0x7fb2e4198af0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fb2d4005940 tx=0x7fb2d40040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:19:03.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.379+0000 7fb2da7fc640 1 -- 172.21.15.26:0/4199696979 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2d403f030 con 0x7fb2e41036e0 2024-09-11T23:19:03.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.379+0000 7fb2eb582640 1 -- 172.21.15.26:0/4199696979 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2e419d7f0 con 0x7fb2e41036e0 2024-09-11T23:19:03.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.379+0000 7fb2da7fc640 1 -- 172.21.15.26:0/4199696979 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb2d40044c0 con 0x7fb2e41036e0 2024-09-11T23:19:03.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.379+0000 7fb2da7fc640 1 -- 172.21.15.26:0/4199696979 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2d4035070 con 0x7fb2e41036e0 2024-09-11T23:19:03.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.379+0000 7fb2eb582640 1 -- 172.21.15.26:0/4199696979 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2e41a5130 con 0x7fb2e41036e0 2024-09-11T23:19:03.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.382+0000 7fb2da7fc640 1 -- 172.21.15.26:0/4199696979 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 25) ==== 138066+0+0 (secure 0 0 0) 0x7fb2d4042030 con 0x7fb2e41036e0 2024-09-11T23:19:03.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.383+0000 7fb2da7fc640 1 --2- 172.21.15.26:0/4199696979 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fb2d00a2e90 0x7fb2d00a5350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:03.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.383+0000 7fb2e931e640 1 --2- 172.21.15.26:0/4199696979 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fb2d00a2e90 0x7fb2d00a5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:03.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.383+0000 7fb2ed95b740 1 Processor -- start 2024-09-11T23:19:03.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.383+0000 7fb2da7fc640 1 -- 172.21.15.26:0/4199696979 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fb2d40ef670 con 0x7fb2e41036e0 2024-09-11T23:19:03.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.383+0000 7fb2ed95b740 1 -- start start 2024-09-11T23:19:03.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.383+0000 7fb2ed95b740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559c6ea4d4a0 0x559c6ea4d880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:03.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.383+0000 7fb2ed95b740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559c6ea4ddc0 0x559c6ea58140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:03.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.383+0000 7fb2ed95b740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559c6ea58680 0x559c6ea5aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:03.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.383+0000 7fb2ed95b740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x559c6e9b32e0 con 0x559c6ea58680 2024-09-11T23:19:03.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.383+0000 7fb2e931e640 1 --2- 172.21.15.26:0/4199696979 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fb2d00a2e90 0x7fb2d00a5350 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb2e0009c20 tx=0x7fb2e0005f90 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:19:03.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.383+0000 7fb2ed95b740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x559c6e95ace0 con 0x559c6ea4ddc0 2024-09-11T23:19:03.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.384+0000 7fb2ed95b740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x559c6e9af2a0 con 0x559c6ea4d4a0 2024-09-11T23:19:03.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.384+0000 7fb2e8b1d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559c6ea4ddc0 0x559c6ea58140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:03.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.384+0000 7fb2e8b1d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559c6ea4ddc0 0x559c6ea58140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:40666/0 (socket says 172.21.15.26:40666) 2024-09-11T23:19:03.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.384+0000 7fb2e8b1d640 1 -- 172.21.15.26:0/1622995657 learned_addr learned my addr 172.21.15.26:0/1622995657 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:19:03.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.384+0000 7fb2e9b1f640 1 --2- 172.21.15.26:0/1622995657 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559c6ea58680 0x559c6ea5aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:03.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.384+0000 7fb2e931e640 1 --2- 172.21.15.26:0/1622995657 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559c6ea4d4a0 0x559c6ea4d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:03.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.384+0000 7fb2e8b1d640 1 -- 172.21.15.26:0/1622995657 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559c6ea4d4a0 msgr2=0x559c6ea4d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:19:03.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.384+0000 7fb2e8b1d640 1 --2- 172.21.15.26:0/1622995657 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559c6ea4d4a0 0x559c6ea4d880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:03.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.384+0000 7fb2e8b1d640 1 -- 172.21.15.26:0/1622995657 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559c6ea58680 msgr2=0x559c6ea5aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:19:03.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.384+0000 7fb2e8b1d640 1 --2- 172.21.15.26:0/1622995657 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559c6ea58680 0x559c6ea5aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:03.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.384+0000 7fb2e8b1d640 1 -- 172.21.15.26:0/1622995657 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c6e93f440 con 0x559c6ea4ddc0 2024-09-11T23:19:03.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.384+0000 7fb2e931e640 1 --2- 172.21.15.26:0/1622995657 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559c6ea4d4a0 0x559c6ea4d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:19:03.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.384+0000 7fb2e8b1d640 1 --2- 172.21.15.26:0/1622995657 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559c6ea4ddc0 0x559c6ea58140 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb2d40b83f0 tx=0x7fb2d40bb440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d3061bc7200780d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:19:03.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.384+0000 7fb2cf7fe640 1 -- 172.21.15.26:0/1622995657 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2d4049070 con 0x559c6ea4ddc0 2024-09-11T23:19:03.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.384+0000 7fb2cf7fe640 1 -- 172.21.15.26:0/1622995657 <== mon.2 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb2d40bfdd0 con 0x559c6ea4ddc0 2024-09-11T23:19:03.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.385+0000 7fb2cf7fe640 1 -- 172.21.15.26:0/1622995657 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2d40c4200 con 0x559c6ea4ddc0 2024-09-11T23:19:03.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.385+0000 7fb2ed95b740 1 -- 172.21.15.26:0/1622995657 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559c6ea4ddc0 msgr2=0x559c6ea58140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:03.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.385+0000 7fb2ed95b740 1 --2- 172.21.15.26:0/1622995657 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559c6ea4ddc0 0x559c6ea58140 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb2d40b83f0 tx=0x7fb2d40bb440 comp rx=0 tx=0).stop 2024-09-11T23:19:03.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.386+0000 7fb2ed95b740 1 -- 172.21.15.26:0/1622995657 shutdown_connections 2024-09-11T23:19:03.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.386+0000 7fb2ed95b740 1 --2- 172.21.15.26:0/1622995657 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559c6ea4ddc0 0x559c6ea58140 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:03.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.386+0000 7fb2ed95b740 1 --2- 172.21.15.26:0/1622995657 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559c6ea58680 0x559c6ea5aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:03.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.386+0000 7fb2ed95b740 1 --2- 172.21.15.26:0/1622995657 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559c6ea4d4a0 0x559c6ea4d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:03.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.386+0000 7fb2ed95b740 1 -- 172.21.15.26:0/1622995657 >> 172.21.15.26:0/1622995657 conn(0x559c6ea49a30 msgr2=0x559c6ea49e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:19:03.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.386+0000 7fb2ed95b740 1 -- 172.21.15.26:0/1622995657 shutdown_connections 2024-09-11T23:19:03.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.386+0000 7fb2ed95b740 1 -- 172.21.15.26:0/1622995657 wait complete. 2024-09-11T23:19:03.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.386+0000 7fb2ed95b740 1 -- start start 2024-09-11T23:19:03.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.386+0000 7fb2ed95b740 1 Processor -- start 2024-09-11T23:19:03.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.387+0000 7fb2ed95b740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559c6ea4d4a0 0x559c6ea58040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:03.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.387+0000 7fb2ed95b740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559c6ea4ddc0 0x559c6ea552f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:03.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.387+0000 7fb2ed95b740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559c6ea58680 0x559c6ea558e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:03.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.387+0000 7fb2ed95b740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x559c6ea5f420 con 0x559c6ea4ddc0 2024-09-11T23:19:03.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.387+0000 7fb2ed95b740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x559c6e96e050 con 0x559c6ea4d4a0 2024-09-11T23:19:03.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.387+0000 7fb2ed95b740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x559c6ea5f5a0 con 0x559c6ea58680 2024-09-11T23:19:03.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.387+0000 7fb2ed95b740 10 client.0 _ll_register_callbacks cb 0x559c6e9ae160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:19:03.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.387+0000 7fb2e8b1d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559c6ea4ddc0 0x559c6ea552f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:03.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.387+0000 7fb2e8b1d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559c6ea4ddc0 0x559c6ea552f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33210/0 (socket says 172.21.15.26:33210) 2024-09-11T23:19:03.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.387+0000 7fb2e8b1d640 1 -- 172.21.15.26:0/1707233837 learned_addr learned my addr 172.21.15.26:0/1707233837 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:19:03.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.387+0000 7fb2e931e640 1 --2- 172.21.15.26:0/1707233837 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559c6ea4d4a0 0x559c6ea58040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:03.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.387+0000 7fb2e9b1f640 1 --2- 172.21.15.26:0/1707233837 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559c6ea58680 0x559c6ea558e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:03.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.387+0000 7fb2e8b1d640 1 -- 172.21.15.26:0/1707233837 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559c6ea58680 msgr2=0x559c6ea558e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:19:03.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.387+0000 7fb2e8b1d640 1 --2- 172.21.15.26:0/1707233837 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559c6ea58680 0x559c6ea558e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:03.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.387+0000 7fb2e8b1d640 1 -- 172.21.15.26:0/1707233837 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559c6ea4d4a0 msgr2=0x559c6ea58040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:19:03.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.387+0000 7fb2e8b1d640 1 --2- 172.21.15.26:0/1707233837 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559c6ea4d4a0 0x559c6ea58040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:03.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.387+0000 7fb2e8b1d640 1 -- 172.21.15.26:0/1707233837 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c6e7243c0 con 0x559c6ea4ddc0 2024-09-11T23:19:03.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.387+0000 7fb2e8b1d640 1 --2- 172.21.15.26:0/1707233837 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559c6ea4ddc0 0x559c6ea552f0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fb2d40bb6d0 tx=0x7fb2d402a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52de97455bc1a631 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:19:03.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.387+0000 7fb2ccff9640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:19:03.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.388+0000 7fb2ccff9640 1 -- 172.21.15.26:0/1707233837 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2d4049070 con 0x559c6ea4ddc0 2024-09-11T23:19:03.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.388+0000 7fb2ed95b740 1 -- 172.21.15.26:0/1707233837 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb2d40bfdd0 con 0x559c6ea4ddc0 2024-09-11T23:19:03.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.388+0000 7fb2ccff9640 1 -- 172.21.15.26:0/1707233837 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fb2d40c4200 con 0x559c6ea4ddc0 2024-09-11T23:19:03.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.388+0000 7fb2ccff9640 1 -- 172.21.15.26:0/1707233837 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb2d40c4400 con 0x559c6ea4ddc0 2024-09-11T23:19:03.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.388+0000 7fb2ed95b740 10 client.15549 fetch_fsmap learned FSMap version 29 2024-09-11T23:19:03.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.388+0000 7fb2ed95b740 1 -- 172.21.15.26:0/1707233837 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559c6ea5f420 con 0x559c6ea4ddc0 2024-09-11T23:19:03.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.389+0000 7fb2ccff9640 1 -- 172.21.15.26:0/1707233837 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb2d40da2e0 con 0x559c6ea4ddc0 2024-09-11T23:19:03.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.389+0000 7fb2ccff9640 1 -- 172.21.15.26:0/1707233837 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fb2d404c060 con 0x559c6ea4ddc0 2024-09-11T23:19:03.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.389+0000 7fb2ed95b740 10 client.15549 fetch_fsmap finished waiting for FSMap version 29 2024-09-11T23:19:03.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.389+0000 7fb2ed95b740 10 client.15549 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:19:03.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.389+0000 7fb2ed95b740 20 client.15549 populate_metadata read hostname 'smithi026' 2024-09-11T23:19:03.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.389+0000 7fb2ed95b740 1 --2- 172.21.15.26:0/1707233837 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x559c6ea6c0f0 0x559c6ea6e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:03.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.389+0000 7fb2ed95b740 4 client.15549 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:19:03.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.389+0000 7fb2ed95b740 1 -- 172.21.15.26:0/1707233837 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fb2d40bfdd0 con 0x559c6ea6c0f0 2024-09-11T23:19:03.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.389+0000 7fb2e931e640 1 --2- 172.21.15.26:0/1707233837 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x559c6ea6c0f0 0x559c6ea6e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:03.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.390+0000 7fb2e931e640 1 --2- 172.21.15.26:0/1707233837 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x559c6ea6c0f0 0x559c6ea6e510 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=697760eb131bf95b server_cookie=f2ba7ec598cb80b4 in_seq=0 out_seq=0 2024-09-11T23:19:03.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.390+0000 7fb2ccff9640 10 client.15549 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:19:03.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.390+0000 7fb2ccff9640 1 -- 172.21.15.26:0/1707233837 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7fb2d40bfdd0 con 0x559c6ea6c0f0 2024-09-11T23:19:03.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.390+0000 7fb2ccff9640 10 client.15549 handle_command_reply: tid=0 2024-09-11T23:19:03.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.391+0000 7fb2ed95b740 1 client.15549 shutdown 2024-09-11T23:19:03.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.391+0000 7fb2ed95b740 1 -- 172.21.15.26:0/1707233837 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559c6ea4ddc0 msgr2=0x559c6ea552f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:03.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.391+0000 7fb2ed95b740 1 --2- 172.21.15.26:0/1707233837 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559c6ea4ddc0 0x559c6ea552f0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fb2d40bb6d0 tx=0x7fb2d402a420 comp rx=0 tx=0).stop 2024-09-11T23:19:03.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.391+0000 7fb2ed95b740 1 -- 172.21.15.26:0/1707233837 shutdown_connections 2024-09-11T23:19:03.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.391+0000 7fb2ed95b740 1 --2- 172.21.15.26:0/1707233837 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x559c6ea6c0f0 0x559c6ea6e510 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:03.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.391+0000 7fb2ed95b740 1 --2- 172.21.15.26:0/1707233837 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559c6ea58680 0x559c6ea558e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:03.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.391+0000 7fb2ed95b740 1 --2- 172.21.15.26:0/1707233837 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559c6ea4ddc0 0x559c6ea552f0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:03.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.391+0000 7fb2ed95b740 1 --2- 172.21.15.26:0/1707233837 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559c6ea4d4a0 0x559c6ea58040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:03.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.391+0000 7fb2ed95b740 1 -- 172.21.15.26:0/1707233837 >> 172.21.15.26:0/1707233837 conn(0x559c6ea49a30 msgr2=0x559c6ea4f710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:19:03.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.391+0000 7fb2ccff9640 0 client.15549 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:19:03.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.391+0000 7fb2ed95b740 1 -- 172.21.15.26:0/1707233837 shutdown_connections 2024-09-11T23:19:03.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.391+0000 7fb2ed95b740 1 -- 172.21.15.26:0/1707233837 wait complete. 2024-09-11T23:19:03.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.391+0000 7fb2ed95b740 20 client.15549 trim_cache size 0 max 16384 2024-09-11T23:19:03.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.427+0000 7fb2ed95b740 1 Processor -- start 2024-09-11T23:19:03.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.427+0000 7fb2ed95b740 1 -- start start 2024-09-11T23:19:03.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.427+0000 7fb2ed95b740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559c6ea4d4a0 0x559c6ea57260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:03.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.427+0000 7fb2ed95b740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559c6ea4ddc0 0x559c6ea73da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:03.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.427+0000 7fb2ed95b740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559c6ea58680 0x559c6ea742e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:03.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.428+0000 7fb2ed95b740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb2e41ab780 con 0x559c6ea58680 2024-09-11T23:19:03.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.428+0000 7fb2ed95b740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fb2e41ad2c0 con 0x559c6ea4d4a0 2024-09-11T23:19:03.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.428+0000 7fb2ed95b740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb2e41abe50 con 0x559c6ea4ddc0 2024-09-11T23:19:03.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.428+0000 7fb2e9b1f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559c6ea58680 0x559c6ea742e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:03.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.428+0000 7fb2e9b1f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559c6ea58680 0x559c6ea742e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33216/0 (socket says 172.21.15.26:33216) 2024-09-11T23:19:03.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.428+0000 7fb2e9b1f640 1 -- 172.21.15.26:0/337131957 learned_addr learned my addr 172.21.15.26:0/337131957 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:19:03.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.428+0000 7fb2e8b1d640 1 --2- 172.21.15.26:0/337131957 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559c6ea4ddc0 0x559c6ea73da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:03.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.428+0000 7fb2e931e640 1 --2- 172.21.15.26:0/337131957 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559c6ea4d4a0 0x559c6ea57260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:03.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.428+0000 7fb2e9b1f640 1 -- 172.21.15.26:0/337131957 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559c6ea4ddc0 msgr2=0x559c6ea73da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:19:03.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.428+0000 7fb2e9b1f640 1 --2- 172.21.15.26:0/337131957 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559c6ea4ddc0 0x559c6ea73da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:03.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.428+0000 7fb2e9b1f640 1 -- 172.21.15.26:0/337131957 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559c6ea4d4a0 msgr2=0x559c6ea57260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:19:03.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.428+0000 7fb2e9b1f640 1 --2- 172.21.15.26:0/337131957 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559c6ea4d4a0 0x559c6ea57260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:03.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.428+0000 7fb2e9b1f640 1 -- 172.21.15.26:0/337131957 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c6ea542a0 con 0x559c6ea58680 2024-09-11T23:19:03.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.428+0000 7fb2e931e640 1 --2- 172.21.15.26:0/337131957 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559c6ea4d4a0 0x559c6ea57260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:19:03.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.428+0000 7fb2e8b1d640 1 --2- 172.21.15.26:0/337131957 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559c6ea4ddc0 0x559c6ea73da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:19:03.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.429+0000 7fb2e9b1f640 1 --2- 172.21.15.26:0/337131957 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559c6ea58680 0x559c6ea742e0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fb2dc013590 tx=0x7fb2dc009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d75ab7132beb5387 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:19:03.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.429+0000 7fb2ceffd640 1 -- 172.21.15.26:0/337131957 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2dc004b20 con 0x559c6ea58680 2024-09-11T23:19:03.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.429+0000 7fb2ceffd640 1 -- 172.21.15.26:0/337131957 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb2dc004cc0 con 0x559c6ea58680 2024-09-11T23:19:03.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.429+0000 7fb2ceffd640 1 -- 172.21.15.26:0/337131957 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2dc005420 con 0x559c6ea58680 2024-09-11T23:19:03.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.430+0000 7fb2ed95b740 1 -- 172.21.15.26:0/337131957 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559c6ea58680 msgr2=0x559c6ea742e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:03.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.430+0000 7fb2ed95b740 1 --2- 172.21.15.26:0/337131957 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559c6ea58680 0x559c6ea742e0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fb2dc013590 tx=0x7fb2dc009f90 comp rx=0 tx=0).stop 2024-09-11T23:19:03.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.430+0000 7fb2ed95b740 1 -- 172.21.15.26:0/337131957 shutdown_connections 2024-09-11T23:19:03.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.430+0000 7fb2ed95b740 1 --2- 172.21.15.26:0/337131957 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559c6ea58680 0x559c6ea742e0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:03.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.430+0000 7fb2ed95b740 1 --2- 172.21.15.26:0/337131957 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559c6ea4ddc0 0x559c6ea73da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:03.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.430+0000 7fb2ed95b740 1 --2- 172.21.15.26:0/337131957 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559c6ea4d4a0 0x559c6ea57260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:03.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.430+0000 7fb2ed95b740 1 -- 172.21.15.26:0/337131957 >> 172.21.15.26:0/337131957 conn(0x559c6ea49a30 msgr2=0x559c6ea56b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:19:03.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.430+0000 7fb2ed95b740 1 -- 172.21.15.26:0/337131957 shutdown_connections 2024-09-11T23:19:03.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.430+0000 7fb2ed95b740 1 -- 172.21.15.26:0/337131957 wait complete. 2024-09-11T23:19:03.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.430+0000 7fb2ed95b740 1 -- start start 2024-09-11T23:19:03.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.431+0000 7fb2ed95b740 1 Processor -- start 2024-09-11T23:19:03.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.431+0000 7fb2ed95b740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559c6ea4d4a0 0x559c6ea64b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:03.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.431+0000 7fb2ed95b740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559c6ea4ddc0 0x559c6ea61ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:03.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.431+0000 7fb2ed95b740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559c6ea58680 0x559c6ea62410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:03.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.431+0000 7fb2ed95b740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb2e41acbf0 con 0x559c6ea58680 2024-09-11T23:19:03.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.431+0000 7fb2ed95b740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fb2e41ac520 con 0x559c6ea4ddc0 2024-09-11T23:19:03.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.431+0000 7fb2ed95b740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb2e41a9760 con 0x559c6ea4d4a0 2024-09-11T23:19:03.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.431+0000 7fb2e9b1f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559c6ea58680 0x559c6ea62410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:03.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.431+0000 7fb2e8b1d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559c6ea4ddc0 0x559c6ea61ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:03.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.431+0000 7fb2e9b1f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559c6ea58680 0x559c6ea62410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33220/0 (socket says 172.21.15.26:33220) 2024-09-11T23:19:03.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.431+0000 7fb2e9b1f640 1 -- 172.21.15.26:0/3033863063 learned_addr learned my addr 172.21.15.26:0/3033863063 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:19:03.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.431+0000 7fb2e8b1d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559c6ea4ddc0 0x559c6ea61ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:40696/0 (socket says 172.21.15.26:40696) 2024-09-11T23:19:03.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.432+0000 7fb2e931e640 1 --2- 172.21.15.26:0/3033863063 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559c6ea4d4a0 0x559c6ea64b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:03.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.432+0000 7fb2e9b1f640 1 -- 172.21.15.26:0/3033863063 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559c6ea4d4a0 msgr2=0x559c6ea64b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:19:03.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.432+0000 7fb2e9b1f640 1 --2- 172.21.15.26:0/3033863063 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559c6ea4d4a0 0x559c6ea64b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:03.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.432+0000 7fb2e9b1f640 1 -- 172.21.15.26:0/3033863063 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559c6ea4ddc0 msgr2=0x559c6ea61ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:19:03.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.432+0000 7fb2e9b1f640 1 --2- 172.21.15.26:0/3033863063 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559c6ea4ddc0 0x559c6ea61ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:03.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.432+0000 7fb2e9b1f640 1 -- 172.21.15.26:0/3033863063 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c6ea55c50 con 0x559c6ea58680 2024-09-11T23:19:03.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.432+0000 7fb2e931e640 1 --2- 172.21.15.26:0/3033863063 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559c6ea4d4a0 0x559c6ea64b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:19:03.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.432+0000 7fb2ed95b740 10 client.0 _ll_register_callbacks cb 0x7fb2dc009a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:19:03.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.432+0000 7fb2e9b1f640 1 --2- 172.21.15.26:0/3033863063 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559c6ea58680 0x559c6ea62410 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fb2dc004ef0 tx=0x7fb2dc004f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9a1be0acd0309b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:19:03.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.432+0000 7fb2ccff9640 10 client.15555 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:19:03.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.432+0000 7fb2ed95b740 1 -- 172.21.15.26:0/3033863063 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb2dc004cc0 con 0x559c6ea58680 2024-09-11T23:19:03.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.432+0000 7fb2ccff9640 1 -- 172.21.15.26:0/3033863063 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2dc029030 con 0x559c6ea58680 2024-09-11T23:19:03.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.432+0000 7fb2ccff9640 1 -- 172.21.15.26:0/3033863063 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fb2dc019030 con 0x559c6ea58680 2024-09-11T23:19:03.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.433+0000 7fb2ccff9640 1 -- 172.21.15.26:0/3033863063 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb2dc003b80 con 0x559c6ea58680 2024-09-11T23:19:03.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.433+0000 7fb2ed95b740 10 client.15555 fetch_fsmap learned FSMap version 29 2024-09-11T23:19:03.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.433+0000 7fb2ed95b740 1 -- 172.21.15.26:0/3033863063 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({fsmap=29}) -- 0x559c6ead6790 con 0x559c6ea58680 2024-09-11T23:19:03.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.433+0000 7fb2ccff9640 1 -- 172.21.15.26:0/3033863063 <== mon.0 v2:172.21.15.26:3300/0 4 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fb2dc00f040 con 0x559c6ea58680 2024-09-11T23:19:03.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.433+0000 7fb2ed95b740 10 client.15555 fetch_fsmap finished waiting for FSMap version 29 2024-09-11T23:19:03.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.433+0000 7fb2ed95b740 10 client.15555 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:19:03.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.433+0000 7fb2ed95b740 20 client.15555 populate_metadata read hostname 'smithi026' 2024-09-11T23:19:03.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.433+0000 7fb2ed95b740 1 --2- 172.21.15.26:0/3033863063 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x559c6ead6db0 0x559c6ead91d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:03.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.433+0000 7fb2ed95b740 4 client.15555 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T23:19:03.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.433+0000 7fb2ed95b740 1 -- 172.21.15.26:0/3033863063 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7fb2dc004cc0 con 0x559c6ead6db0 2024-09-11T23:19:03.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.433+0000 7fb2e931e640 1 --2- 172.21.15.26:0/3033863063 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x559c6ead6db0 0x559c6ead91d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:03.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.434+0000 7fb2ccff9640 10 client.15555 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:19:03.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.434+0000 7fb2e931e640 1 --2- 172.21.15.26:0/3033863063 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x559c6ead6db0 0x559c6ead91d0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=971f0d5712cfc6b4 server_cookie=47cb0b91a79e1ccc in_seq=0 out_seq=0 2024-09-11T23:19:03.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.434+0000 7fb2ccff9640 1 -- 172.21.15.26:0/3033863063 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7fb2dc004cc0 con 0x559c6ead6db0 2024-09-11T23:19:03.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.434+0000 7fb2ccff9640 10 client.15555 handle_command_reply: tid=0 2024-09-11T23:19:03.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.434+0000 7fb2ed95b740 1 client.15555 shutdown 2024-09-11T23:19:03.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.434+0000 7fb2ed95b740 1 -- 172.21.15.26:0/3033863063 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559c6ea58680 msgr2=0x559c6ea62410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:03.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.434+0000 7fb2ed95b740 1 --2- 172.21.15.26:0/3033863063 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559c6ea58680 0x559c6ea62410 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fb2dc004ef0 tx=0x7fb2dc004f20 comp rx=0 tx=0).stop 2024-09-11T23:19:03.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.434+0000 7fb2ed95b740 1 -- 172.21.15.26:0/3033863063 shutdown_connections 2024-09-11T23:19:03.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.434+0000 7fb2ed95b740 1 --2- 172.21.15.26:0/3033863063 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559c6ea4ddc0 0x559c6ea61ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:03.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.434+0000 7fb2ed95b740 1 --2- 172.21.15.26:0/3033863063 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x559c6ead6db0 0x559c6ead91d0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:03.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.434+0000 7fb2ed95b740 1 --2- 172.21.15.26:0/3033863063 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559c6ea58680 0x559c6ea62410 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:03.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.434+0000 7fb2ed95b740 1 --2- 172.21.15.26:0/3033863063 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559c6ea4d4a0 0x559c6ea64b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:03.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.434+0000 7fb2ccff9640 0 client.15555 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:19:03.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.434+0000 7fb2ed95b740 1 -- 172.21.15.26:0/3033863063 >> 172.21.15.26:0/3033863063 conn(0x559c6ea49a30 msgr2=0x559c6ea57370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:19:03.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.435+0000 7fb2ed95b740 1 -- 172.21.15.26:0/3033863063 shutdown_connections 2024-09-11T23:19:03.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.435+0000 7fb2ed95b740 1 -- 172.21.15.26:0/3033863063 wait complete. 2024-09-11T23:19:03.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.435+0000 7fb2ed95b740 20 client.15555 trim_cache size 0 max 16384 2024-09-11T23:19:03.441 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-11T23:19:03.441 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "no active scrubs running", 2024-09-11T23:19:03.441 INFO:teuthology.orchestra.run.smithi026.stdout: "scrubs": {} 2024-09-11T23:19:03.441 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-11T23:19:03.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.435+0000 7fb2eb582640 1 -- 172.21.15.26:0/4199696979 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fb2d00a2e90 msgr2=0x7fb2d00a5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:03.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.435+0000 7fb2eb582640 1 --2- 172.21.15.26:0/4199696979 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fb2d00a2e90 0x7fb2d00a5350 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb2e0009c20 tx=0x7fb2e0005f90 comp rx=0 tx=0).stop 2024-09-11T23:19:03.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.435+0000 7fb2eb582640 1 -- 172.21.15.26:0/4199696979 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb2e41036e0 msgr2=0x7fb2e4198af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:03.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.435+0000 7fb2eb582640 1 --2- 172.21.15.26:0/4199696979 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb2e41036e0 0x7fb2e4198af0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fb2d4005940 tx=0x7fb2d40040c0 comp rx=0 tx=0).stop 2024-09-11T23:19:03.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.435+0000 7fb2eb582640 1 -- 172.21.15.26:0/4199696979 shutdown_connections 2024-09-11T23:19:03.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.436+0000 7fb2eb582640 1 --2- 172.21.15.26:0/4199696979 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fb2d00a2e90 0x7fb2d00a5350 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:03.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.436+0000 7fb2eb582640 1 --2- 172.21.15.26:0/4199696979 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb2e41082a0 0x7fb2e419ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:03.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.436+0000 7fb2eb582640 1 --2- 172.21.15.26:0/4199696979 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb2e41036e0 0x7fb2e4198af0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:03.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.436+0000 7fb2eb582640 1 --2- 172.21.15.26:0/4199696979 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb2e4102d30 0x7fb2e41985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:03.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.436+0000 7fb2eb582640 1 -- 172.21.15.26:0/4199696979 >> 172.21.15.26:0/4199696979 conn(0x7fb2e40fc7d0 msgr2=0x7fb2e4105de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:19:03.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.436+0000 7fb2eb582640 1 -- 172.21.15.26:0/4199696979 shutdown_connections 2024-09-11T23:19:03.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:03.436+0000 7fb2eb582640 1 -- 172.21.15.26:0/4199696979 wait complete. 2024-09-11T23:19:04.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:19:03 smithi136 ceph-mon[35121]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T23:19:04.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:19:03 smithi136 ceph-mon[35121]: mgrmap e25: x(active, since 40s), standbys: y, z 2024-09-11T23:19:04.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:19:03 smithi026 ceph-mon[31172]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T23:19:04.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:19:03 smithi026 ceph-mon[31172]: mgrmap e25: x(active, since 40s), standbys: y, z 2024-09-11T23:19:04.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:19:03 smithi076 ceph-mon[35296]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T23:19:04.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:19:03 smithi076 ceph-mon[35296]: mgrmap e25: x(active, since 40s), standbys: y, z 2024-09-11T23:19:04.316 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-11T23:19:04.316 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:19:04.595 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:19:05.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:05.099+0000 7fac8d0c3640 1 -- 172.21.15.26:0/2001637956 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fac88102570 msgr2=0x7fac881029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:05.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:05.099+0000 7fac8d0c3640 1 --2- 172.21.15.26:0/2001637956 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fac88102570 0x7fac881029b0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fac7c009900 tx=0x7fac7c02a480 comp rx=0 tx=0).stop 2024-09-11T23:19:05.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:05.099+0000 7fac8d0c3640 1 -- 172.21.15.26:0/2001637956 shutdown_connections 2024-09-11T23:19:05.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:05.099+0000 7fac8d0c3640 1 --2- 172.21.15.26:0/2001637956 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fac88102ef0 0x7fac8810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:05.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:05.099+0000 7fac8d0c3640 1 --2- 172.21.15.26:0/2001637956 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fac88102570 0x7fac881029b0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:05.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:05.099+0000 7fac8d0c3640 1 --2- 172.21.15.26:0/2001637956 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fac88100e20 0x7fac88101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:05.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:05.099+0000 7fac8d0c3640 1 -- 172.21.15.26:0/2001637956 >> 172.21.15.26:0/2001637956 conn(0x7fac880fc7d0 msgr2=0x7fac880febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:19:05.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:05.099+0000 7fac8d0c3640 1 -- 172.21.15.26:0/2001637956 shutdown_connections 2024-09-11T23:19:05.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:05.099+0000 7fac8d0c3640 1 -- 172.21.15.26:0/2001637956 wait complete. 2024-09-11T23:19:05.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:05.100+0000 7fac8d0c3640 1 Processor -- start 2024-09-11T23:19:05.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:05.100+0000 7fac8d0c3640 1 -- start start 2024-09-11T23:19:05.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:05.100+0000 7fac8d0c3640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fac88100e20 0x7fac88199340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:05.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:05.100+0000 7fac8d0c3640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fac88102570 0x7fac88199880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:05.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:05.100+0000 7fac8d0c3640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fac88102ef0 0x7fac881a43c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:05.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:05.100+0000 7fac8d0c3640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fac88110fd0 con 0x7fac88102570 2024-09-11T23:19:05.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:05.100+0000 7fac8d0c3640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fac88110e50 con 0x7fac88102ef0 2024-09-11T23:19:05.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:05.100+0000 7fac8d0c3640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fac88111150 con 0x7fac88100e20 2024-09-11T23:19:05.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:05.100+0000 7fac8659c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fac88102570 0x7fac88199880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:05.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:05.100+0000 7fac8759e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fac88102ef0 0x7fac881a43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:05.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:05.100+0000 7fac8759e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fac88102ef0 0x7fac881a43c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:40706/0 (socket says 172.21.15.26:40706) 2024-09-11T23:19:05.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:05.100+0000 7fac8659c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fac88102570 0x7fac88199880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33232/0 (socket says 172.21.15.26:33232) 2024-09-11T23:19:05.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:05.100+0000 7fac8759e640 1 -- 172.21.15.26:0/3843542096 learned_addr learned my addr 172.21.15.26:0/3843542096 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:19:05.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:05.100+0000 7fac86d9d640 1 --2- 172.21.15.26:0/3843542096 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fac88100e20 0x7fac88199340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:05.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:05.101+0000 7fac8759e640 1 -- 172.21.15.26:0/3843542096 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fac88100e20 msgr2=0x7fac88199340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:05.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:05.101+0000 7fac8759e640 1 --2- 172.21.15.26:0/3843542096 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fac88100e20 0x7fac88199340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:05.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:05.101+0000 7fac8759e640 1 -- 172.21.15.26:0/3843542096 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fac88102570 msgr2=0x7fac88199880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:05.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:05.101+0000 7fac8759e640 1 --2- 172.21.15.26:0/3843542096 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fac88102570 0x7fac88199880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:05.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:05.101+0000 7fac8759e640 1 -- 172.21.15.26:0/3843542096 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac881a4900 con 0x7fac88102ef0 2024-09-11T23:19:05.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:05.101+0000 7fac8759e640 1 --2- 172.21.15.26:0/3843542096 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fac88102ef0 0x7fac881a43c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fac8000eb10 tx=0x7fac8000c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:19:05.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:05.101+0000 7fac6ffff640 1 -- 172.21.15.26:0/3843542096 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac80004b20 con 0x7fac88102ef0 2024-09-11T23:19:05.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:05.101+0000 7fac6ffff640 1 -- 172.21.15.26:0/3843542096 <== mon.2 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fac80004cc0 con 0x7fac88102ef0 2024-09-11T23:19:05.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:05.101+0000 7fac8d0c3640 1 -- 172.21.15.26:0/3843542096 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac881a4ad0 con 0x7fac88102ef0 2024-09-11T23:19:05.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:05.101+0000 7fac6ffff640 1 -- 172.21.15.26:0/3843542096 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac80010760 con 0x7fac88102ef0 2024-09-11T23:19:05.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:05.101+0000 7fac8d0c3640 1 -- 172.21.15.26:0/3843542096 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac881a4f90 con 0x7fac88102ef0 2024-09-11T23:19:05.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:05.102+0000 7fac8d0c3640 1 -- 172.21.15.26:0/3843542096 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac88101200 con 0x7fac88102ef0 2024-09-11T23:19:05.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:05.104+0000 7fac6ffff640 1 -- 172.21.15.26:0/3843542096 <== mon.2 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 25) ==== 138066+0+0 (secure 0 0 0) 0x7fac80010900 con 0x7fac88102ef0 2024-09-11T23:19:05.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:05.105+0000 7fac6ffff640 1 --2- 172.21.15.26:0/3843542096 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fac700a2e10 0x7fac700a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:05.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:05.105+0000 7fac86d9d640 1 --2- 172.21.15.26:0/3843542096 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fac700a2e10 0x7fac700a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:05.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:05.105+0000 7fac6ffff640 1 -- 172.21.15.26:0/3843542096 <== mon.2 v2:172.21.15.76:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fac800d0150 con 0x7fac88102ef0 2024-09-11T23:19:05.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:05.106+0000 7fac86d9d640 1 --2- 172.21.15.26:0/3843542096 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fac700a2e10 0x7fac700a52d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fac78009b60 tx=0x7fac78005f90 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:19:05.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:05.108+0000 7fac6ffff640 1 -- 172.21.15.26:0/3843542096 <== mon.2 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fac8009cb30 con 0x7fac88102ef0 2024-09-11T23:19:05.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:05.326+0000 7fac8d0c3640 1 -- 172.21.15.26:0/3843542096 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fac881a5280 con 0x7fac88102ef0 2024-09-11T23:19:05.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:05.327+0000 7fac6ffff640 1 -- 172.21.15.26:0/3843542096 <== mon.2 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11971 (secure 0 0 0) 0x7fac8009c430 con 0x7fac88102ef0 2024-09-11T23:19:05.328 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-11T23:19:05.328 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":29,"btime":"2024-09-11T23:18:21:019324+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14499,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/367659515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":367659515},{"type":"v1","addr":"172.21.15.26:6835","nonce":367659515}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14511,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/2041065299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":2041065299},{"type":"v1","addr":"172.21.15.26:6837","nonce":2041065299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:16:26.544203+0000","modified":"2024-09-11T23:18:20.969151+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24485,"mds_1":24463,"mds_2":24458,"mds_3":24448,"mds_4":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14550":{"gid":14550,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6839/2233068305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":2233068305},{"type":"v1","addr":"172.21.15.26:6839","nonce":2233068305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14562":{"gid":14562,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6841/2984982336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":2984982336},{"type":"v1","addr":"172.21.15.26:6841","nonce":2984982336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6833/2019841455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":2019841455},{"type":"v1","addr":"172.21.15.136:6833","nonce":2019841455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.76:6833/3438147050","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3438147050},{"type":"v1","addr":"172.21.15.76:6833","nonce":3438147050}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6835/3617665975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3617665975},{"type":"v1","addr":"172.21.15.136:6835","nonce":3617665975}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":12,"addr":"172.21.15.76:6835/393308499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":393308499},{"type":"v1","addr":"172.21.15.76:6835","nonce":393308499}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":10,"addr":"172.21.15.136:6837/1633082387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":1633082387},{"type":"v1","addr":"172.21.15.136:6837","nonce":1633082387}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.76:6837/2078978748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":2078978748},{"type":"v1","addr":"172.21.15.76:6837","nonce":2078978748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.136:6839/3211398630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":3211398630},{"type":"v1","addr":"172.21.15.136:6839","nonce":3211398630}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.76:6839/2077637090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6838","nonce":2077637090},{"type":"v1","addr":"172.21.15.76:6839","nonce":2077637090}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24485,"qdb_cluster":[24443,24448,24458,24463,24485]},"id":1}]} 2024-09-11T23:19:05.328 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 29 2024-09-11T23:19:05.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:05.330+0000 7fac8d0c3640 1 -- 172.21.15.26:0/3843542096 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fac700a2e10 msgr2=0x7fac700a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:05.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:05.330+0000 7fac8d0c3640 1 --2- 172.21.15.26:0/3843542096 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fac700a2e10 0x7fac700a52d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fac78009b60 tx=0x7fac78005f90 comp rx=0 tx=0).stop 2024-09-11T23:19:05.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:05.330+0000 7fac8d0c3640 1 -- 172.21.15.26:0/3843542096 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fac88102ef0 msgr2=0x7fac881a43c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:05.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:05.330+0000 7fac8d0c3640 1 --2- 172.21.15.26:0/3843542096 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fac88102ef0 0x7fac881a43c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fac8000eb10 tx=0x7fac8000c2d0 comp rx=0 tx=0).stop 2024-09-11T23:19:05.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:05.330+0000 7fac8d0c3640 1 -- 172.21.15.26:0/3843542096 shutdown_connections 2024-09-11T23:19:05.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:05.330+0000 7fac8d0c3640 1 --2- 172.21.15.26:0/3843542096 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fac700a2e10 0x7fac700a52d0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:05.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:05.330+0000 7fac8d0c3640 1 --2- 172.21.15.26:0/3843542096 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fac88102ef0 0x7fac881a43c0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:05.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:05.330+0000 7fac8d0c3640 1 --2- 172.21.15.26:0/3843542096 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fac88102570 0x7fac88199880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:05.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:05.330+0000 7fac8d0c3640 1 --2- 172.21.15.26:0/3843542096 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fac88100e20 0x7fac88199340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:05.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:05.330+0000 7fac8d0c3640 1 -- 172.21.15.26:0/3843542096 >> 172.21.15.26:0/3843542096 conn(0x7fac880fc7d0 msgr2=0x7fac88106840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:19:05.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:05.330+0000 7fac8d0c3640 1 -- 172.21.15.26:0/3843542096 shutdown_connections 2024-09-11T23:19:05.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:05.330+0000 7fac8d0c3640 1 -- 172.21.15.26:0/3843542096 wait complete. 2024-09-11T23:19:06.144 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:19:05 smithi026 ceph-mon[31172]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 24 KiB/s wr, 29 op/s 2024-09-11T23:19:06.144 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:19:05 smithi026 ceph-mon[31172]: from='client.? 172.21.15.26:0/3843542096' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:19:06.172 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-11T23:19:06.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:19:05 smithi136 ceph-mon[35121]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 24 KiB/s wr, 29 op/s 2024-09-11T23:19:06.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:19:05 smithi136 ceph-mon[35121]: from='client.? 172.21.15.26:0/3843542096' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:19:06.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:19:05 smithi076 ceph-mon[35296]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 24 KiB/s wr, 29 op/s 2024-09-11T23:19:06.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:19:05 smithi076 ceph-mon[35296]: from='client.? 172.21.15.26:0/3843542096' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:19:06.441 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:19:07.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.024+0000 7feff0916640 1 -- 172.21.15.26:0/747820443 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fefec102df0 msgr2=0x7fefec1031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:07.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.024+0000 7feff0916640 1 --2- 172.21.15.26:0/747820443 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fefec102df0 0x7fefec1031d0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fefe4009ca0 tx=0x7fefe402a8c0 comp rx=0 tx=0).stop 2024-09-11T23:19:07.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.024+0000 7feff0916640 1 -- 172.21.15.26:0/747820443 shutdown_connections 2024-09-11T23:19:07.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.024+0000 7feff0916640 1 --2- 172.21.15.26:0/747820443 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fefec10c2f0 0x7fefec10e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:07.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.024+0000 7feff0916640 1 --2- 172.21.15.26:0/747820443 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fefec1037a0 0x7fefec10bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:07.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.024+0000 7feff0916640 1 --2- 172.21.15.26:0/747820443 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fefec102df0 0x7fefec1031d0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:07.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.024+0000 7feff0916640 1 -- 172.21.15.26:0/747820443 >> 172.21.15.26:0/747820443 conn(0x7fefec0fc7d0 msgr2=0x7fefec0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:19:07.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.024+0000 7feff0916640 1 -- 172.21.15.26:0/747820443 shutdown_connections 2024-09-11T23:19:07.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.024+0000 7feff0916640 1 -- 172.21.15.26:0/747820443 wait complete. 2024-09-11T23:19:07.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.025+0000 7feff0916640 1 Processor -- start 2024-09-11T23:19:07.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.025+0000 7feff0916640 1 -- start start 2024-09-11T23:19:07.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.025+0000 7feff0916640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fefec102df0 0x7fefec100a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:07.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.025+0000 7feff0916640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fefec1037a0 0x7fefec100f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:07.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.025+0000 7feff0916640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fefec10c2f0 0x7fefec1014d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:07.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.025+0000 7feff0916640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fefec1120f0 con 0x7fefec102df0 2024-09-11T23:19:07.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.025+0000 7feff0916640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fefec111f70 con 0x7fefec1037a0 2024-09-11T23:19:07.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.025+0000 7feff0916640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fefec112270 con 0x7fefec10c2f0 2024-09-11T23:19:07.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.025+0000 7fefea59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fefec102df0 0x7fefec100a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:07.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.025+0000 7fefea59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fefec102df0 0x7fefec100a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33248/0 (socket says 172.21.15.26:33248) 2024-09-11T23:19:07.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.025+0000 7fefea59c640 1 -- 172.21.15.26:0/999415133 learned_addr learned my addr 172.21.15.26:0/999415133 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:19:07.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.025+0000 7fefead9d640 1 --2- 172.21.15.26:0/999415133 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fefec10c2f0 0x7fefec1014d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:07.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.026+0000 7fefe9d9b640 1 --2- 172.21.15.26:0/999415133 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fefec1037a0 0x7fefec100f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:07.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.026+0000 7fefea59c640 1 -- 172.21.15.26:0/999415133 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fefec10c2f0 msgr2=0x7fefec1014d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:07.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.026+0000 7fefea59c640 1 --2- 172.21.15.26:0/999415133 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fefec10c2f0 0x7fefec1014d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:07.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.026+0000 7fefea59c640 1 -- 172.21.15.26:0/999415133 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fefec1037a0 msgr2=0x7fefec100f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:07.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.026+0000 7fefea59c640 1 --2- 172.21.15.26:0/999415133 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fefec1037a0 0x7fefec100f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:07.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.026+0000 7fefea59c640 1 -- 172.21.15.26:0/999415133 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefec1a6ad0 con 0x7fefec102df0 2024-09-11T23:19:07.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.026+0000 7fefea59c640 1 --2- 172.21.15.26:0/999415133 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fefec102df0 0x7fefec100a50 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fefe402ade0 tx=0x7fefe40319f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:19:07.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.026+0000 7fefe37fe640 1 -- 172.21.15.26:0/999415133 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefe402fa20 con 0x7fefec102df0 2024-09-11T23:19:07.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.026+0000 7fefe37fe640 1 -- 172.21.15.26:0/999415133 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fefe402fbc0 con 0x7fefec102df0 2024-09-11T23:19:07.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.026+0000 7feff0916640 1 -- 172.21.15.26:0/999415133 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefec1a6d60 con 0x7fefec102df0 2024-09-11T23:19:07.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.026+0000 7fefe37fe640 1 -- 172.21.15.26:0/999415133 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefe40467b0 con 0x7fefec102df0 2024-09-11T23:19:07.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.026+0000 7feff0916640 1 -- 172.21.15.26:0/999415133 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefec1a7240 con 0x7fefec102df0 2024-09-11T23:19:07.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.029+0000 7feff2cef740 1 Processor -- start 2024-09-11T23:19:07.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.029+0000 7feff2cef740 1 -- start start 2024-09-11T23:19:07.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.029+0000 7feff2cef740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b7b12cab40 0x55b7b12c9090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:07.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.030+0000 7feff2cef740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b7b12c95d0 0x55b7b12c9a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:07.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.030+0000 7feff2cef740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b7b12d4370 0x55b7b12d6760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:07.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.030+0000 7feff2cef740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55b7b122f2f0 con 0x55b7b12cab40 2024-09-11T23:19:07.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.030+0000 7feff2cef740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55b7b11d6cf0 con 0x55b7b12c95d0 2024-09-11T23:19:07.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.030+0000 7feff2cef740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55b7b122b2b0 con 0x55b7b12d4370 2024-09-11T23:19:07.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.030+0000 7fefe37fe640 1 -- 172.21.15.26:0/999415133 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 25) ==== 138066+0+0 (secure 0 0 0) 0x7fefe4046950 con 0x7fefec102df0 2024-09-11T23:19:07.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.030+0000 7fefead9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b7b12cab40 0x55b7b12c9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:07.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.030+0000 7fefead9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b7b12cab40 0x55b7b12c9090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33250/0 (socket says 172.21.15.26:33250) 2024-09-11T23:19:07.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.030+0000 7fefead9d640 1 -- 172.21.15.26:0/502943613 learned_addr learned my addr 172.21.15.26:0/502943613 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:19:07.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.030+0000 7fefea59c640 1 --2- 172.21.15.26:0/502943613 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b7b12c95d0 0x55b7b12c9a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:07.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.030+0000 7fefe9d9b640 1 --2- 172.21.15.26:0/502943613 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b7b12d4370 0x55b7b12d6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:07.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.030+0000 7fefead9d640 1 -- 172.21.15.26:0/502943613 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b7b12d4370 msgr2=0x55b7b12d6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:19:07.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.030+0000 7fefead9d640 1 --2- 172.21.15.26:0/502943613 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b7b12d4370 0x55b7b12d6760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:07.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.030+0000 7fefead9d640 1 -- 172.21.15.26:0/502943613 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b7b12c95d0 msgr2=0x55b7b12c9a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:19:07.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.030+0000 7fefead9d640 1 --2- 172.21.15.26:0/502943613 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b7b12c95d0 0x55b7b12c9a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:07.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.030+0000 7fefead9d640 1 -- 172.21.15.26:0/502943613 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b7b11bb3a0 con 0x55b7b12cab40 2024-09-11T23:19:07.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.030+0000 7fefea59c640 1 --2- 172.21.15.26:0/502943613 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b7b12c95d0 0x55b7b12c9a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:19:07.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.030+0000 7fefead9d640 1 --2- 172.21.15.26:0/502943613 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b7b12cab40 0x55b7b12c9090 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fefdc00b6b0 tx=0x7fefdc004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f0c0c8443e41f5c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:19:07.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.031+0000 7fefe0ff9640 1 -- 172.21.15.26:0/502943613 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefdc003b20 con 0x55b7b12cab40 2024-09-11T23:19:07.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.031+0000 7fefe0ff9640 1 -- 172.21.15.26:0/502943613 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fefdc015070 con 0x55b7b12cab40 2024-09-11T23:19:07.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.031+0000 7fefe0ff9640 1 -- 172.21.15.26:0/502943613 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefdc0116a0 con 0x55b7b12cab40 2024-09-11T23:19:07.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.031+0000 7fefe37fe640 1 --2- 172.21.15.26:0/999415133 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fefd80a2e90 0x7fefd80a5350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:07.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.031+0000 7fefea59c640 1 --2- 172.21.15.26:0/999415133 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fefd80a2e90 0x7fefd80a5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:07.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.031+0000 7fefe37fe640 1 -- 172.21.15.26:0/999415133 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fefe40f0b10 con 0x7fefec102df0 2024-09-11T23:19:07.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.031+0000 7feff2cef740 1 -- 172.21.15.26:0/502943613 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b7b12cab40 msgr2=0x55b7b12c9090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:07.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.031+0000 7feff2cef740 1 --2- 172.21.15.26:0/502943613 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b7b12cab40 0x55b7b12c9090 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fefdc00b6b0 tx=0x7fefdc004610 comp rx=0 tx=0).stop 2024-09-11T23:19:07.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.031+0000 7feff2cef740 1 -- 172.21.15.26:0/502943613 shutdown_connections 2024-09-11T23:19:07.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.031+0000 7feff2cef740 1 --2- 172.21.15.26:0/502943613 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b7b12c95d0 0x55b7b12c9a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:07.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.031+0000 7feff2cef740 1 --2- 172.21.15.26:0/502943613 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b7b12d4370 0x55b7b12d6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:07.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.031+0000 7feff2cef740 1 --2- 172.21.15.26:0/502943613 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b7b12cab40 0x55b7b12c9090 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:07.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.031+0000 7feff2cef740 1 -- 172.21.15.26:0/502943613 >> 172.21.15.26:0/502943613 conn(0x55b7b1292310 msgr2=0x55b7b12c5c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:19:07.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.032+0000 7fefea59c640 1 --2- 172.21.15.26:0/999415133 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fefd80a2e90 0x7fefd80a5350 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fefe40ee200 tx=0x7fefe404f000 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:19:07.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.032+0000 7feff2cef740 1 -- 172.21.15.26:0/502943613 shutdown_connections 2024-09-11T23:19:07.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.032+0000 7feff2cef740 1 -- 172.21.15.26:0/502943613 wait complete. 2024-09-11T23:19:07.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.032+0000 7feff2cef740 1 -- start start 2024-09-11T23:19:07.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.032+0000 7feff2cef740 1 Processor -- start 2024-09-11T23:19:07.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.032+0000 7feff2cef740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b7b12c95d0 0x55b7b12d3d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:07.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.032+0000 7feff2cef740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b7b12cab40 0x55b7b12d1020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:07.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.032+0000 7feff2cef740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b7b12d4370 0x55b7b12d1610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:07.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.032+0000 7feff2cef740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55b7b12d7070 con 0x55b7b12d4370 2024-09-11T23:19:07.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.032+0000 7feff2cef740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55b7b11ea060 con 0x55b7b12cab40 2024-09-11T23:19:07.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.032+0000 7feff2cef740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55b7b12d71f0 con 0x55b7b12c95d0 2024-09-11T23:19:07.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.032+0000 7feff2cef740 10 client.0 _ll_register_callbacks cb 0x55b7b0f3d8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:19:07.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.033+0000 7fefead9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b7b12d4370 0x55b7b12d1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:07.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.033+0000 7fefead9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b7b12d4370 0x55b7b12d1610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33254/0 (socket says 172.21.15.26:33254) 2024-09-11T23:19:07.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.033+0000 7fefead9d640 1 -- 172.21.15.26:0/409623667 learned_addr learned my addr 172.21.15.26:0/409623667 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:19:07.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.033+0000 7fefead9d640 1 --2- 172.21.15.26:0/409623667 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b7b12c95d0 0x55b7b12d3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:07.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.033+0000 7fefe9d9b640 1 --2- 172.21.15.26:0/409623667 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b7b12cab40 0x55b7b12d1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:07.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.033+0000 7fefead9d640 1 -- 172.21.15.26:0/409623667 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b7b12c95d0 msgr2=0x55b7b12d3d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:19:07.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.033+0000 7fefead9d640 1 --2- 172.21.15.26:0/409623667 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b7b12c95d0 0x55b7b12d3d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:07.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.033+0000 7fefead9d640 1 -- 172.21.15.26:0/409623667 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b7b12cab40 msgr2=0x55b7b12d1020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:19:07.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.033+0000 7fefead9d640 1 --2- 172.21.15.26:0/409623667 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b7b12cab40 0x55b7b12d1020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:07.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.033+0000 7fefead9d640 1 -- 172.21.15.26:0/409623667 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b7b0fa03c0 con 0x55b7b12d4370 2024-09-11T23:19:07.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.033+0000 7fefead9d640 1 --2- 172.21.15.26:0/409623667 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b7b12d4370 0x55b7b12d1610 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fefdc000c00 tx=0x7fefdc01ad80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=499a9e34e5a82c6b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:19:07.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.033+0000 7fefd1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:19:07.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.033+0000 7fefd1ffb640 1 -- 172.21.15.26:0/409623667 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefdc023040 con 0x55b7b12d4370 2024-09-11T23:19:07.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.033+0000 7fefd1ffb640 1 -- 172.21.15.26:0/409623667 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fefdc003920 con 0x55b7b12d4370 2024-09-11T23:19:07.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.033+0000 7feff2cef740 1 -- 172.21.15.26:0/409623667 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fefdc015070 con 0x55b7b12d4370 2024-09-11T23:19:07.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.034+0000 7fefd1ffb640 1 -- 172.21.15.26:0/409623667 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fefdc030050 con 0x55b7b12d4370 2024-09-11T23:19:07.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.034+0000 7feff2cef740 10 client.15573 fetch_fsmap learned FSMap version 29 2024-09-11T23:19:07.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.034+0000 7feff2cef740 1 -- 172.21.15.26:0/409623667 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b7b12e2b10 con 0x55b7b12d4370 2024-09-11T23:19:07.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.034+0000 7fefd1ffb640 1 -- 172.21.15.26:0/409623667 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fefdc015070 con 0x55b7b12d4370 2024-09-11T23:19:07.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.034+0000 7fefd1ffb640 1 -- 172.21.15.26:0/409623667 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fefdc005050 con 0x55b7b12d4370 2024-09-11T23:19:07.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.034+0000 7feff2cef740 10 client.15573 fetch_fsmap finished waiting for FSMap version 29 2024-09-11T23:19:07.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.034+0000 7feff2cef740 10 client.15573 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-09-11T23:19:07.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.034+0000 7feff2cef740 20 client.15573 populate_metadata read hostname 'smithi026' 2024-09-11T23:19:07.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.034+0000 7feff2cef740 1 --2- 172.21.15.26:0/409623667 >> [v2:172.21.15.76:6834/393308499,v1:172.21.15.76:6835/393308499] conn(0x55b7b12e3ec0 0x55b7b12e62b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:07.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.034+0000 7feff2cef740 4 client.15573 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:19:07.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.034+0000 7feff2cef740 1 -- 172.21.15.26:0/409623667 --> [v2:172.21.15.76:6834/393308499,v1:172.21.15.76:6835/393308499] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b7b123c860 con 0x55b7b12e3ec0 2024-09-11T23:19:07.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.034+0000 7fefe9d9b640 1 --2- 172.21.15.26:0/409623667 >> [v2:172.21.15.76:6834/393308499,v1:172.21.15.76:6835/393308499] conn(0x55b7b12e3ec0 0x55b7b12e62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:07.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.035+0000 7fefe9d9b640 1 --2- 172.21.15.26:0/409623667 >> [v2:172.21.15.76:6834/393308499,v1:172.21.15.76:6835/393308499] conn(0x55b7b12e3ec0 0x55b7b12e62b0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f1aa3afff769d806 server_cookie=2f734942b5d0e8b1 in_seq=0 out_seq=0 2024-09-11T23:19:07.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.035+0000 7fefd1ffb640 10 client.15573 ms_handle_connect on v2:172.21.15.76:6834/393308499 2024-09-11T23:19:07.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.035+0000 7fefd1ffb640 1 -- 172.21.15.26:0/409623667 <== mds.4 v2:172.21.15.76:6834/393308499 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b7b123c860 con 0x55b7b12e3ec0 2024-09-11T23:19:07.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.035+0000 7fefd1ffb640 10 client.15573 handle_command_reply: tid=0 2024-09-11T23:19:07.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.036+0000 7feff2cef740 1 client.15573 shutdown 2024-09-11T23:19:07.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.036+0000 7feff2cef740 1 -- 172.21.15.26:0/409623667 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b7b12d4370 msgr2=0x55b7b12d1610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:07.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.036+0000 7feff2cef740 1 --2- 172.21.15.26:0/409623667 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b7b12d4370 0x55b7b12d1610 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fefdc000c00 tx=0x7fefdc01ad80 comp rx=0 tx=0).stop 2024-09-11T23:19:07.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.036+0000 7feff2cef740 1 -- 172.21.15.26:0/409623667 shutdown_connections 2024-09-11T23:19:07.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.036+0000 7feff2cef740 1 --2- 172.21.15.26:0/409623667 >> [v2:172.21.15.76:6834/393308499,v1:172.21.15.76:6835/393308499] conn(0x55b7b12e3ec0 0x55b7b12e62b0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:07.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.036+0000 7feff2cef740 1 --2- 172.21.15.26:0/409623667 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b7b12cab40 0x55b7b12d1020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:07.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.036+0000 7feff2cef740 1 --2- 172.21.15.26:0/409623667 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b7b12d4370 0x55b7b12d1610 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:07.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.036+0000 7fefd1ffb640 0 client.15573 ms_handle_reset on v2:172.21.15.76:6834/393308499 2024-09-11T23:19:07.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.036+0000 7feff2cef740 1 --2- 172.21.15.26:0/409623667 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b7b12c95d0 0x55b7b12d3d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:07.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.036+0000 7feff2cef740 1 -- 172.21.15.26:0/409623667 >> 172.21.15.26:0/409623667 conn(0x55b7b1292310 msgr2=0x55b7b12cb930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:19:07.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.036+0000 7feff2cef740 1 -- 172.21.15.26:0/409623667 shutdown_connections 2024-09-11T23:19:07.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.036+0000 7feff2cef740 1 -- 172.21.15.26:0/409623667 wait complete. 2024-09-11T23:19:07.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.036+0000 7feff2cef740 20 client.15573 trim_cache size 0 max 16384 2024-09-11T23:19:07.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.062+0000 7feff2cef740 1 Processor -- start 2024-09-11T23:19:07.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.062+0000 7feff2cef740 1 -- start start 2024-09-11T23:19:07.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.062+0000 7feff2cef740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b7b12c95d0 0x55b7b12e0310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:07.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.062+0000 7feff2cef740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b7b12cab40 0x55b7b12eb9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:07.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.062+0000 7feff2cef740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b7b12d4370 0x55b7b12ebf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:07.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.062+0000 7feff2cef740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fefec1afa20 con 0x55b7b12cab40 2024-09-11T23:19:07.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.062+0000 7feff2cef740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fefec1b1560 con 0x55b7b12d4370 2024-09-11T23:19:07.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.062+0000 7feff2cef740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fefec1b00f0 con 0x55b7b12c95d0 2024-09-11T23:19:07.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.063+0000 7fefe9d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b7b12cab40 0x55b7b12eb9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:07.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.063+0000 7fefe9d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b7b12cab40 0x55b7b12eb9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33270/0 (socket says 172.21.15.26:33270) 2024-09-11T23:19:07.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.063+0000 7fefe9d9b640 1 -- 172.21.15.26:0/2323143731 learned_addr learned my addr 172.21.15.26:0/2323143731 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:19:07.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.063+0000 7fefead9d640 1 --2- 172.21.15.26:0/2323143731 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b7b12c95d0 0x55b7b12e0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:07.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.063+0000 7fefead9d640 1 --2- 172.21.15.26:0/2323143731 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b7b12d4370 0x55b7b12ebf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:07.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.063+0000 7fefe9d9b640 1 -- 172.21.15.26:0/2323143731 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b7b12c95d0 msgr2=0x55b7b12e0310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:19:07.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.063+0000 7fefe9d9b640 1 --2- 172.21.15.26:0/2323143731 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b7b12c95d0 0x55b7b12e0310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:07.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.063+0000 7fefe9d9b640 1 -- 172.21.15.26:0/2323143731 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b7b12d4370 msgr2=0x55b7b12ebf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:19:07.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.063+0000 7fefe9d9b640 1 --2- 172.21.15.26:0/2323143731 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b7b12d4370 0x55b7b12ebf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:07.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.063+0000 7fefe9d9b640 1 -- 172.21.15.26:0/2323143731 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b7b12d40e0 con 0x55b7b12cab40 2024-09-11T23:19:07.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.063+0000 7fefead9d640 1 --2- 172.21.15.26:0/2323143731 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b7b12d4370 0x55b7b12ebf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:19:07.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.063+0000 7fefe9d9b640 1 --2- 172.21.15.26:0/2323143731 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b7b12cab40 0x55b7b12eb9f0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fefec1024e0 tx=0x7fefd4004410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f44bf9387e81c08e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:19:07.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.063+0000 7fefd3fff640 1 -- 172.21.15.26:0/2323143731 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefd4020020 con 0x55b7b12cab40 2024-09-11T23:19:07.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.063+0000 7fefd3fff640 1 -- 172.21.15.26:0/2323143731 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fefd4003d20 con 0x55b7b12cab40 2024-09-11T23:19:07.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.063+0000 7fefd3fff640 1 -- 172.21.15.26:0/2323143731 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefd40128b0 con 0x55b7b12cab40 2024-09-11T23:19:07.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.064+0000 7feff2cef740 1 -- 172.21.15.26:0/2323143731 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b7b12cab40 msgr2=0x55b7b12eb9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:07.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.064+0000 7feff2cef740 1 --2- 172.21.15.26:0/2323143731 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b7b12cab40 0x55b7b12eb9f0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fefec1024e0 tx=0x7fefd4004410 comp rx=0 tx=0).stop 2024-09-11T23:19:07.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.064+0000 7feff2cef740 1 -- 172.21.15.26:0/2323143731 shutdown_connections 2024-09-11T23:19:07.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.064+0000 7feff2cef740 1 --2- 172.21.15.26:0/2323143731 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b7b12d4370 0x55b7b12ebf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:07.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.064+0000 7feff2cef740 1 --2- 172.21.15.26:0/2323143731 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b7b12cab40 0x55b7b12eb9f0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:07.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.064+0000 7feff2cef740 1 --2- 172.21.15.26:0/2323143731 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b7b12c95d0 0x55b7b12e0310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:07.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.064+0000 7feff2cef740 1 -- 172.21.15.26:0/2323143731 >> 172.21.15.26:0/2323143731 conn(0x55b7b1292310 msgr2=0x55b7b12d0110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:19:07.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.064+0000 7feff2cef740 1 -- 172.21.15.26:0/2323143731 shutdown_connections 2024-09-11T23:19:07.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.064+0000 7feff2cef740 1 -- 172.21.15.26:0/2323143731 wait complete. 2024-09-11T23:19:07.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.065+0000 7feff2cef740 1 -- start start 2024-09-11T23:19:07.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.065+0000 7feff2cef740 1 Processor -- start 2024-09-11T23:19:07.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.065+0000 7feff2cef740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b7b12c95d0 0x55b7b12dcc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:07.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.065+0000 7feff2cef740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b7b12cab40 0x55b7b12cde40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:07.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.065+0000 7feff2cef740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b7b12d4370 0x55b7b12ce3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:07.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.065+0000 7feff2cef740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fefec1b0e90 con 0x55b7b12c95d0 2024-09-11T23:19:07.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.065+0000 7feff2cef740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fefec1b07c0 con 0x55b7b12d4370 2024-09-11T23:19:07.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.065+0000 7feff2cef740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fefec1ada20 con 0x55b7b12cab40 2024-09-11T23:19:07.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.065+0000 7feff2cef740 10 client.0 _ll_register_callbacks cb 0x7fefdc004510 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:19:07.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.065+0000 7fefead9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b7b12c95d0 0x55b7b12dcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:07.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.065+0000 7fefead9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b7b12c95d0 0x55b7b12dcc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33280/0 (socket says 172.21.15.26:33280) 2024-09-11T23:19:07.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.065+0000 7fefead9d640 1 -- 172.21.15.26:0/3986622913 learned_addr learned my addr 172.21.15.26:0/3986622913 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:19:07.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.065+0000 7fefe9d9b640 1 --2- 172.21.15.26:0/3986622913 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b7b12cab40 0x55b7b12cde40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:07.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.065+0000 7fefead9d640 1 --2- 172.21.15.26:0/3986622913 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b7b12d4370 0x55b7b12ce3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:07.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.065+0000 7fefead9d640 1 -- 172.21.15.26:0/3986622913 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b7b12cab40 msgr2=0x55b7b12cde40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:19:07.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.065+0000 7fefead9d640 1 --2- 172.21.15.26:0/3986622913 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b7b12cab40 0x55b7b12cde40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:07.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.065+0000 7fefead9d640 1 -- 172.21.15.26:0/3986622913 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b7b12d4370 msgr2=0x55b7b12ce3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:19:07.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.065+0000 7fefead9d640 1 --2- 172.21.15.26:0/3986622913 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b7b12d4370 0x55b7b12ce3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:07.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.065+0000 7fefead9d640 1 -- 172.21.15.26:0/3986622913 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b7b12cebf0 con 0x55b7b12c95d0 2024-09-11T23:19:07.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.066+0000 7fefead9d640 1 --2- 172.21.15.26:0/3986622913 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b7b12c95d0 0x55b7b12dcc80 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fefdc014040 tx=0x7fefdc01ad80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cd2f41647e47e69 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:19:07.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.066+0000 7fefd1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:19:07.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.066+0000 7fefd1ffb640 1 -- 172.21.15.26:0/3986622913 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefdc011990 con 0x55b7b12c95d0 2024-09-11T23:19:07.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.066+0000 7fefd1ffb640 1 -- 172.21.15.26:0/3986622913 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fefdc011b30 con 0x55b7b12c95d0 2024-09-11T23:19:07.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.066+0000 7feff2cef740 1 -- 172.21.15.26:0/3986622913 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fefd4003d20 con 0x55b7b12c95d0 2024-09-11T23:19:07.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.066+0000 7fefd1ffb640 1 -- 172.21.15.26:0/3986622913 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fefdc005640 con 0x55b7b12c95d0 2024-09-11T23:19:07.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.066+0000 7feff2cef740 10 client.15579 fetch_fsmap learned FSMap version 29 2024-09-11T23:19:07.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.066+0000 7feff2cef740 1 -- 172.21.15.26:0/3986622913 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b7b134ea60 con 0x55b7b12c95d0 2024-09-11T23:19:07.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.066+0000 7fefd1ffb640 1 -- 172.21.15.26:0/3986622913 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fefd4003d20 con 0x55b7b12c95d0 2024-09-11T23:19:07.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.066+0000 7fefd1ffb640 1 -- 172.21.15.26:0/3986622913 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fefdc0355b0 con 0x55b7b12c95d0 2024-09-11T23:19:07.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.066+0000 7feff2cef740 10 client.15579 fetch_fsmap finished waiting for FSMap version 29 2024-09-11T23:19:07.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.066+0000 7feff2cef740 10 client.15579 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-09-11T23:19:07.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.066+0000 7feff2cef740 20 client.15579 populate_metadata read hostname 'smithi026' 2024-09-11T23:19:07.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.067+0000 7feff2cef740 1 --2- 172.21.15.26:0/3986622913 >> [v2:172.21.15.76:6834/393308499,v1:172.21.15.76:6835/393308499] conn(0x55b7b134f0c0 0x55b7b13514b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:07.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.067+0000 7feff2cef740 4 client.15579 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:19:07.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.067+0000 7feff2cef740 1 -- 172.21.15.26:0/3986622913 --> [v2:172.21.15.76:6834/393308499,v1:172.21.15.76:6835/393308499] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b7b1236d70 con 0x55b7b134f0c0 2024-09-11T23:19:07.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.067+0000 7fefe9d9b640 1 --2- 172.21.15.26:0/3986622913 >> [v2:172.21.15.76:6834/393308499,v1:172.21.15.76:6835/393308499] conn(0x55b7b134f0c0 0x55b7b13514b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:07.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.067+0000 7fefd1ffb640 10 client.15579 ms_handle_connect on v2:172.21.15.76:6834/393308499 2024-09-11T23:19:07.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.067+0000 7fefe9d9b640 1 --2- 172.21.15.26:0/3986622913 >> [v2:172.21.15.76:6834/393308499,v1:172.21.15.76:6835/393308499] conn(0x55b7b134f0c0 0x55b7b13514b0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=228b3813b9f0d6de server_cookie=10d821037760b5b in_seq=0 out_seq=0 2024-09-11T23:19:07.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.067+0000 7fefd1ffb640 1 -- 172.21.15.26:0/3986622913 <== mds.4 v2:172.21.15.76:6834/393308499 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b7b1236d70 con 0x55b7b134f0c0 2024-09-11T23:19:07.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.068+0000 7fefd1ffb640 10 client.15579 handle_command_reply: tid=0 2024-09-11T23:19:07.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.068+0000 7feff2cef740 1 client.15579 shutdown 2024-09-11T23:19:07.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.068+0000 7feff2cef740 1 -- 172.21.15.26:0/3986622913 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b7b12c95d0 msgr2=0x55b7b12dcc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:07.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.068+0000 7feff2cef740 1 --2- 172.21.15.26:0/3986622913 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b7b12c95d0 0x55b7b12dcc80 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fefdc014040 tx=0x7fefdc01ad80 comp rx=0 tx=0).stop 2024-09-11T23:19:07.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.068+0000 7feff2cef740 1 -- 172.21.15.26:0/3986622913 shutdown_connections 2024-09-11T23:19:07.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.068+0000 7feff2cef740 1 --2- 172.21.15.26:0/3986622913 >> [v2:172.21.15.76:6834/393308499,v1:172.21.15.76:6835/393308499] conn(0x55b7b134f0c0 0x55b7b13514b0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:07.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.068+0000 7feff2cef740 1 --2- 172.21.15.26:0/3986622913 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b7b12d4370 0x55b7b12ce3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:07.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.068+0000 7feff2cef740 1 --2- 172.21.15.26:0/3986622913 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b7b12cab40 0x55b7b12cde40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:07.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.068+0000 7feff2cef740 1 --2- 172.21.15.26:0/3986622913 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b7b12c95d0 0x55b7b12dcc80 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:07.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.068+0000 7feff2cef740 1 -- 172.21.15.26:0/3986622913 >> 172.21.15.26:0/3986622913 conn(0x55b7b1292310 msgr2=0x55b7b12d2880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:19:07.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.068+0000 7fefd1ffb640 0 client.15579 ms_handle_reset on v2:172.21.15.76:6834/393308499 2024-09-11T23:19:07.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.068+0000 7feff2cef740 1 -- 172.21.15.26:0/3986622913 shutdown_connections 2024-09-11T23:19:07.199 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2024-09-11T23:19:07.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.068+0000 7feff2cef740 1 -- 172.21.15.26:0/3986622913 wait complete. 2024-09-11T23:19:07.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.068+0000 7feff2cef740 20 client.15579 trim_cache size 0 max 16384 2024-09-11T23:19:07.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.068+0000 7feff0916640 1 -- 172.21.15.26:0/999415133 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fefd80a2e90 msgr2=0x7fefd80a5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:07.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.068+0000 7feff0916640 1 --2- 172.21.15.26:0/999415133 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fefd80a2e90 0x7fefd80a5350 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fefe40ee200 tx=0x7fefe404f000 comp rx=0 tx=0).stop 2024-09-11T23:19:07.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.068+0000 7feff0916640 1 -- 172.21.15.26:0/999415133 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fefec102df0 msgr2=0x7fefec100a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:07.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.068+0000 7feff0916640 1 --2- 172.21.15.26:0/999415133 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fefec102df0 0x7fefec100a50 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fefe402ade0 tx=0x7fefe40319f0 comp rx=0 tx=0).stop 2024-09-11T23:19:07.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.068+0000 7feff0916640 1 -- 172.21.15.26:0/999415133 shutdown_connections 2024-09-11T23:19:07.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.068+0000 7feff0916640 1 --2- 172.21.15.26:0/999415133 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fefd80a2e90 0x7fefd80a5350 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:07.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.068+0000 7feff0916640 1 --2- 172.21.15.26:0/999415133 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fefec1037a0 0x7fefec100f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:07.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.068+0000 7feff0916640 1 --2- 172.21.15.26:0/999415133 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fefec10c2f0 0x7fefec1014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:07.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.069+0000 7feff0916640 1 --2- 172.21.15.26:0/999415133 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fefec102df0 0x7fefec100a50 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:07.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.069+0000 7feff0916640 1 -- 172.21.15.26:0/999415133 >> 172.21.15.26:0/999415133 conn(0x7fefec0fc7d0 msgr2=0x7fefec0fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:19:07.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.069+0000 7feff0916640 1 -- 172.21.15.26:0/999415133 shutdown_connections 2024-09-11T23:19:07.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:07.069+0000 7feff0916640 1 -- 172.21.15.26:0/999415133 wait complete. 2024-09-11T23:19:07.899 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 24.5 2024-09-11T23:19:07.899 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:19:08.111 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:19:07 smithi026 ceph-mon[31172]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 25 op/s 2024-09-11T23:19:08.150 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-11T23:19:08.198 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:19:08.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:19:07 smithi136 ceph-mon[35121]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 25 op/s 2024-09-11T23:19:08.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:19:07 smithi076 ceph-mon[35296]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 25 op/s 2024-09-11T23:19:08.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:08.733+0000 7f7c9c309640 1 -- 172.21.15.26:0/2730616511 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7c94102790 msgr2=0x7f7c941067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:08.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:08.733+0000 7f7c9c309640 1 --2- 172.21.15.26:0/2730616511 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7c94102790 0x7f7c941067e0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f7c84009900 tx=0x7f7c8402a410 comp rx=0 tx=0).stop 2024-09-11T23:19:08.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:08.733+0000 7f7c9c309640 1 -- 172.21.15.26:0/2730616511 shutdown_connections 2024-09-11T23:19:08.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:08.733+0000 7f7c9c309640 1 --2- 172.21.15.26:0/2730616511 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7c94106f10 0x7f7c9410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:08.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:08.733+0000 7f7c9c309640 1 --2- 172.21.15.26:0/2730616511 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7c94102790 0x7f7c941067e0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:08.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:08.733+0000 7f7c9c309640 1 --2- 172.21.15.26:0/2730616511 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7c94101de0 0x7f7c941021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:08.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:08.733+0000 7f7c9c309640 1 -- 172.21.15.26:0/2730616511 >> 172.21.15.26:0/2730616511 conn(0x7f7c940fd8c0 msgr2=0x7f7c940ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:19:08.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:08.733+0000 7f7c9c309640 1 -- 172.21.15.26:0/2730616511 shutdown_connections 2024-09-11T23:19:08.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:08.733+0000 7f7c9c309640 1 -- 172.21.15.26:0/2730616511 wait complete. 2024-09-11T23:19:08.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:08.734+0000 7f7c9c309640 1 Processor -- start 2024-09-11T23:19:08.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:08.734+0000 7f7c9c309640 1 -- start start 2024-09-11T23:19:08.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:08.734+0000 7f7c9c309640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7c94101de0 0x7f7c941984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:08.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:08.734+0000 7f7c9c309640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7c94102790 0x7f7c941989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:08.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:08.734+0000 7f7c9c309640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7c94106f10 0x7f7c9419cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:08.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:08.734+0000 7f7c9c309640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f7c94111f70 con 0x7f7c94102790 2024-09-11T23:19:08.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:08.734+0000 7f7c9c309640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f7c94111df0 con 0x7f7c94101de0 2024-09-11T23:19:08.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:08.734+0000 7f7c9c309640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f7c941120f0 con 0x7f7c94106f10 2024-09-11T23:19:08.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:08.734+0000 7f7c998a4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7c94102790 0x7f7c941989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:08.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:08.734+0000 7f7c998a4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7c94102790 0x7f7c941989f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33306/0 (socket says 172.21.15.26:33306) 2024-09-11T23:19:08.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:08.734+0000 7f7c998a4640 1 -- 172.21.15.26:0/1208627861 learned_addr learned my addr 172.21.15.26:0/1208627861 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:19:08.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:08.734+0000 7f7c9a0a5640 1 --2- 172.21.15.26:0/1208627861 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7c94101de0 0x7f7c941984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:08.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:08.734+0000 7f7c9a8a6640 1 --2- 172.21.15.26:0/1208627861 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7c94106f10 0x7f7c9419cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:08.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:08.734+0000 7f7c998a4640 1 -- 172.21.15.26:0/1208627861 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7c94106f10 msgr2=0x7f7c9419cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:08.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:08.734+0000 7f7c998a4640 1 --2- 172.21.15.26:0/1208627861 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7c94106f10 0x7f7c9419cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:08.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:08.734+0000 7f7c998a4640 1 -- 172.21.15.26:0/1208627861 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7c94101de0 msgr2=0x7f7c941984b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:08.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:08.734+0000 7f7c998a4640 1 --2- 172.21.15.26:0/1208627861 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7c94101de0 0x7f7c941984b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:08.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:08.734+0000 7f7c998a4640 1 -- 172.21.15.26:0/1208627861 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c9419d460 con 0x7f7c94102790 2024-09-11T23:19:08.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:08.735+0000 7f7c998a4640 1 --2- 172.21.15.26:0/1208627861 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7c94102790 0x7f7c941989f0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f7c84006010 tx=0x7f7c840042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:19:08.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:08.735+0000 7f7c8b7fe640 1 -- 172.21.15.26:0/1208627861 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c8403f030 con 0x7f7c94102790 2024-09-11T23:19:08.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:08.735+0000 7f7c8b7fe640 1 -- 172.21.15.26:0/1208627861 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c84003ce0 con 0x7f7c94102790 2024-09-11T23:19:08.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:08.735+0000 7f7c9c309640 1 -- 172.21.15.26:0/1208627861 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c9419d6f0 con 0x7f7c94102790 2024-09-11T23:19:08.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:08.735+0000 7f7c8b7fe640 1 -- 172.21.15.26:0/1208627861 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c84035070 con 0x7f7c94102790 2024-09-11T23:19:08.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:08.735+0000 7f7c9c309640 1 -- 172.21.15.26:0/1208627861 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c941a5030 con 0x7f7c94102790 2024-09-11T23:19:08.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:08.736+0000 7f7c9c309640 1 -- 172.21.15.26:0/1208627861 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c941021c0 con 0x7f7c94102790 2024-09-11T23:19:08.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:08.737+0000 7f7c8b7fe640 1 -- 172.21.15.26:0/1208627861 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 25) ==== 138066+0+0 (secure 0 0 0) 0x7f7c84042030 con 0x7f7c94102790 2024-09-11T23:19:08.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:08.738+0000 7f7c8b7fe640 1 --2- 172.21.15.26:0/1208627861 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f7c800a2e90 0x7f7c800a5350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:08.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:08.738+0000 7f7c9a0a5640 1 --2- 172.21.15.26:0/1208627861 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f7c800a2e90 0x7f7c800a5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:08.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:08.738+0000 7f7c8b7fe640 1 -- 172.21.15.26:0/1208627861 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f7c840efc70 con 0x7f7c94102790 2024-09-11T23:19:08.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:08.739+0000 7f7c9a0a5640 1 --2- 172.21.15.26:0/1208627861 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f7c800a2e90 0x7f7c800a5350 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f7c90005fd0 tx=0x7f7c90005850 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:19:08.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:08.743+0000 7f7c8b7fe640 1 -- 172.21.15.26:0/1208627861 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7c840bc7e0 con 0x7f7c94102790 2024-09-11T23:19:08.789 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:19:08.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:08.980+0000 7f7c9c309640 1 -- 172.21.15.26:0/1208627861 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7c9410a150 con 0x7f7c94102790 2024-09-11T23:19:08.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:08.981+0000 7f7c8b7fe640 1 -- 172.21.15.26:0/1208627861 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11971 (secure 0 0 0) 0x7f7c840bc0e0 con 0x7f7c94102790 2024-09-11T23:19:08.981 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-11T23:19:08.981 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":29,"btime":"2024-09-11T23:18:21:019324+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14499,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/367659515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":367659515},{"type":"v1","addr":"172.21.15.26:6835","nonce":367659515}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14511,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/2041065299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":2041065299},{"type":"v1","addr":"172.21.15.26:6837","nonce":2041065299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:16:26.544203+0000","modified":"2024-09-11T23:18:20.969151+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24485,"mds_1":24463,"mds_2":24458,"mds_3":24448,"mds_4":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14550":{"gid":14550,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6839/2233068305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":2233068305},{"type":"v1","addr":"172.21.15.26:6839","nonce":2233068305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14562":{"gid":14562,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6841/2984982336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":2984982336},{"type":"v1","addr":"172.21.15.26:6841","nonce":2984982336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6833/2019841455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":2019841455},{"type":"v1","addr":"172.21.15.136:6833","nonce":2019841455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.76:6833/3438147050","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3438147050},{"type":"v1","addr":"172.21.15.76:6833","nonce":3438147050}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6835/3617665975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3617665975},{"type":"v1","addr":"172.21.15.136:6835","nonce":3617665975}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":12,"addr":"172.21.15.76:6835/393308499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":393308499},{"type":"v1","addr":"172.21.15.76:6835","nonce":393308499}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":10,"addr":"172.21.15.136:6837/1633082387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":1633082387},{"type":"v1","addr":"172.21.15.136:6837","nonce":1633082387}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.76:6837/2078978748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":2078978748},{"type":"v1","addr":"172.21.15.76:6837","nonce":2078978748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.136:6839/3211398630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":3211398630},{"type":"v1","addr":"172.21.15.136:6839","nonce":3211398630}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.76:6839/2077637090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6838","nonce":2077637090},{"type":"v1","addr":"172.21.15.76:6839","nonce":2077637090}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24485,"qdb_cluster":[24443,24448,24458,24463,24485]},"id":1}]} 2024-09-11T23:19:08.982 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 29 2024-09-11T23:19:08.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:08.983+0000 7f7c9c309640 1 -- 172.21.15.26:0/1208627861 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f7c800a2e90 msgr2=0x7f7c800a5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:08.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:08.983+0000 7f7c9c309640 1 --2- 172.21.15.26:0/1208627861 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f7c800a2e90 0x7f7c800a5350 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f7c90005fd0 tx=0x7f7c90005850 comp rx=0 tx=0).stop 2024-09-11T23:19:08.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:08.983+0000 7f7c9c309640 1 -- 172.21.15.26:0/1208627861 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7c94102790 msgr2=0x7f7c941989f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:08.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:08.983+0000 7f7c9c309640 1 --2- 172.21.15.26:0/1208627861 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7c94102790 0x7f7c941989f0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f7c84006010 tx=0x7f7c840042f0 comp rx=0 tx=0).stop 2024-09-11T23:19:08.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:08.983+0000 7f7c9c309640 1 -- 172.21.15.26:0/1208627861 shutdown_connections 2024-09-11T23:19:08.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:08.983+0000 7f7c9c309640 1 --2- 172.21.15.26:0/1208627861 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f7c800a2e90 0x7f7c800a5350 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:08.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:08.983+0000 7f7c9c309640 1 --2- 172.21.15.26:0/1208627861 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7c94106f10 0x7f7c9419cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:08.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:08.983+0000 7f7c9c309640 1 --2- 172.21.15.26:0/1208627861 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7c94102790 0x7f7c941989f0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:08.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:08.983+0000 7f7c9c309640 1 --2- 172.21.15.26:0/1208627861 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7c94101de0 0x7f7c941984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:08.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:08.983+0000 7f7c9c309640 1 -- 172.21.15.26:0/1208627861 >> 172.21.15.26:0/1208627861 conn(0x7f7c940fd8c0 msgr2=0x7f7c940fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:19:08.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:08.983+0000 7f7c9c309640 1 -- 172.21.15.26:0/1208627861 shutdown_connections 2024-09-11T23:19:08.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:08.983+0000 7f7c9c309640 1 -- 172.21.15.26:0/1208627861 wait complete. 2024-09-11T23:19:09.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.630+0000 7f216c7f6640 1 -- 172.21.15.26:0/2230120646 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2164100e20 msgr2=0x7f2164101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:09.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.630+0000 7f216c7f6640 1 --2- 172.21.15.26:0/2230120646 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2164100e20 0x7f2164101200 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f2160009900 tx=0x7f216002a480 comp rx=0 tx=0).stop 2024-09-11T23:19:09.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.630+0000 7f216c7f6640 1 -- 172.21.15.26:0/2230120646 shutdown_connections 2024-09-11T23:19:09.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.630+0000 7f216c7f6640 1 --2- 172.21.15.26:0/2230120646 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2164102ef0 0x7f216410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:09.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.630+0000 7f216c7f6640 1 --2- 172.21.15.26:0/2230120646 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2164102570 0x7f21641029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:09.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.630+0000 7f216c7f6640 1 --2- 172.21.15.26:0/2230120646 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2164100e20 0x7f2164101200 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:09.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.630+0000 7f216c7f6640 1 -- 172.21.15.26:0/2230120646 >> 172.21.15.26:0/2230120646 conn(0x7f21640fc7d0 msgr2=0x7f21640febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:19:09.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.630+0000 7f216c7f6640 1 -- 172.21.15.26:0/2230120646 shutdown_connections 2024-09-11T23:19:09.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.630+0000 7f216c7f6640 1 -- 172.21.15.26:0/2230120646 wait complete. 2024-09-11T23:19:09.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.630+0000 7f216c7f6640 1 Processor -- start 2024-09-11T23:19:09.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.631+0000 7f216c7f6640 1 -- start start 2024-09-11T23:19:09.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.631+0000 7f216c7f6640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2164100e20 0x7f21641984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:09.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.631+0000 7f216c7f6640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2164102570 0x7f21641989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:09.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.631+0000 7f216c7f6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2164102ef0 0x7f216419cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:09.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.631+0000 7f216c7f6640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f2164110f80 con 0x7f2164102ef0 2024-09-11T23:19:09.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.631+0000 7f216c7f6640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f2164110e00 con 0x7f2164100e20 2024-09-11T23:19:09.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.631+0000 7f216c7f6640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f2164111100 con 0x7f2164102570 2024-09-11T23:19:09.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.631+0000 7f216ad93640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2164102ef0 0x7f216419cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:09.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.631+0000 7f216ad93640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2164102ef0 0x7f216419cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33322/0 (socket says 172.21.15.26:33322) 2024-09-11T23:19:09.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.631+0000 7f216ad93640 1 -- 172.21.15.26:0/4220639429 learned_addr learned my addr 172.21.15.26:0/4220639429 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:19:09.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.631+0000 7f216a592640 1 --2- 172.21.15.26:0/4220639429 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2164100e20 0x7f21641984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:09.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.631+0000 7f2169d91640 1 --2- 172.21.15.26:0/4220639429 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2164102570 0x7f21641989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:09.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.631+0000 7f216ad93640 1 -- 172.21.15.26:0/4220639429 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2164102570 msgr2=0x7f21641989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:09.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.631+0000 7f216ad93640 1 --2- 172.21.15.26:0/4220639429 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2164102570 0x7f21641989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:09.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.631+0000 7f216ad93640 1 -- 172.21.15.26:0/4220639429 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2164100e20 msgr2=0x7f21641984b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:09.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.631+0000 7f216ad93640 1 --2- 172.21.15.26:0/4220639429 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2164100e20 0x7f21641984b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:09.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.631+0000 7f216ad93640 1 -- 172.21.15.26:0/4220639429 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f216419d460 con 0x7f2164102ef0 2024-09-11T23:19:09.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.631+0000 7f216a592640 1 --2- 172.21.15.26:0/4220639429 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2164100e20 0x7f21641984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:19:09.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.632+0000 7f216ad93640 1 --2- 172.21.15.26:0/4220639429 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2164102ef0 0x7f216419cd80 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f215c00cce0 tx=0x7f215c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:19:09.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.632+0000 7f21577fe640 1 -- 172.21.15.26:0/4220639429 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f215c00f450 con 0x7f2164102ef0 2024-09-11T23:19:09.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.632+0000 7f216c7f6640 1 -- 172.21.15.26:0/4220639429 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f216419d750 con 0x7f2164102ef0 2024-09-11T23:19:09.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.632+0000 7f21577fe640 1 -- 172.21.15.26:0/4220639429 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f215c00fad0 con 0x7f2164102ef0 2024-09-11T23:19:09.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.632+0000 7f21577fe640 1 -- 172.21.15.26:0/4220639429 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f215c018c10 con 0x7f2164102ef0 2024-09-11T23:19:09.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.632+0000 7f216c7f6640 1 -- 172.21.15.26:0/4220639429 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21641a5030 con 0x7f2164102ef0 2024-09-11T23:19:09.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.634+0000 7f21577fe640 1 -- 172.21.15.26:0/4220639429 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 25) ==== 138066+0+0 (secure 0 0 0) 0x7f215c00f5f0 con 0x7f2164102ef0 2024-09-11T23:19:09.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.634+0000 7f216ebcf740 1 Processor -- start 2024-09-11T23:19:09.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.635+0000 7f216ebcf740 1 -- start start 2024-09-11T23:19:09.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.635+0000 7f216ebcf740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557421774b40 0x557421773090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:09.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.635+0000 7f216ebcf740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5574217735d0 0x557421773a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:09.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.635+0000 7f216ebcf740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557421782780 0x557421784b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:09.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.635+0000 7f216ebcf740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5574216d92f0 con 0x557421782780 2024-09-11T23:19:09.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.635+0000 7f216ebcf740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x557421680cf0 con 0x557421774b40 2024-09-11T23:19:09.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.635+0000 7f216ebcf740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5574216d52b0 con 0x5574217735d0 2024-09-11T23:19:09.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.635+0000 7f2169d91640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557421782780 0x557421784b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:09.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.635+0000 7f2169d91640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557421782780 0x557421784b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33338/0 (socket says 172.21.15.26:33338) 2024-09-11T23:19:09.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.635+0000 7f216a592640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5574217735d0 0x557421773a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:09.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.635+0000 7f2169d91640 1 -- 172.21.15.26:0/845897396 learned_addr learned my addr 172.21.15.26:0/845897396 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:19:09.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.635+0000 7f2169d91640 1 --2- 172.21.15.26:0/845897396 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557421774b40 0x557421773090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:09.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.635+0000 7f21577fe640 1 --2- 172.21.15.26:0/4220639429 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f21500a2e10 0x7f21500a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:09.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.636+0000 7f216ad93640 1 --2- 172.21.15.26:0/4220639429 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f21500a2e10 0x7f21500a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:09.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.636+0000 7f2169d91640 1 -- 172.21.15.26:0/845897396 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5574217735d0 msgr2=0x557421773a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:19:09.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.636+0000 7f2169d91640 1 --2- 172.21.15.26:0/845897396 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5574217735d0 0x557421773a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:09.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.636+0000 7f2169d91640 1 -- 172.21.15.26:0/845897396 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557421774b40 msgr2=0x557421773090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:19:09.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.636+0000 7f2169d91640 1 --2- 172.21.15.26:0/845897396 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557421774b40 0x557421773090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:09.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.636+0000 7f2169d91640 1 -- 172.21.15.26:0/845897396 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5574216653a0 con 0x557421782780 2024-09-11T23:19:09.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.636+0000 7f21577fe640 1 -- 172.21.15.26:0/4220639429 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f215c0cfe90 con 0x7f2164102ef0 2024-09-11T23:19:09.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.636+0000 7f2169d91640 1 --2- 172.21.15.26:0/845897396 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557421782780 0x557421784b70 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f21580149c0 tx=0x7f215800c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=203691833c3cbd35 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:19:09.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.636+0000 7f216ad93640 1 --2- 172.21.15.26:0/4220639429 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f21500a2e10 0x7f21500a52d0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f215c0cd4d0 tx=0x7f215c00af30 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:19:09.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.636+0000 7f2154ff9640 1 -- 172.21.15.26:0/845897396 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21580142d0 con 0x557421782780 2024-09-11T23:19:09.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.636+0000 7f2154ff9640 1 -- 172.21.15.26:0/845897396 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2158005500 con 0x557421782780 2024-09-11T23:19:09.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.636+0000 7f2154ff9640 1 -- 172.21.15.26:0/845897396 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f215800f9c0 con 0x557421782780 2024-09-11T23:19:09.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.637+0000 7f216ebcf740 1 -- 172.21.15.26:0/845897396 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557421782780 msgr2=0x557421784b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:09.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.637+0000 7f216ebcf740 1 --2- 172.21.15.26:0/845897396 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557421782780 0x557421784b70 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f21580149c0 tx=0x7f215800c4d0 comp rx=0 tx=0).stop 2024-09-11T23:19:09.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.637+0000 7f216ebcf740 1 -- 172.21.15.26:0/845897396 shutdown_connections 2024-09-11T23:19:09.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.637+0000 7f216ebcf740 1 --2- 172.21.15.26:0/845897396 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557421782780 0x557421784b70 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:09.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.637+0000 7f216ebcf740 1 --2- 172.21.15.26:0/845897396 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5574217735d0 0x557421773a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:09.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.637+0000 7f216ebcf740 1 --2- 172.21.15.26:0/845897396 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557421774b40 0x557421773090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:09.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.637+0000 7f216ebcf740 1 -- 172.21.15.26:0/845897396 >> 172.21.15.26:0/845897396 conn(0x55742173c310 msgr2=0x55742176fc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:19:09.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.637+0000 7f216ebcf740 1 -- 172.21.15.26:0/845897396 shutdown_connections 2024-09-11T23:19:09.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.637+0000 7f216ebcf740 1 -- 172.21.15.26:0/845897396 wait complete. 2024-09-11T23:19:09.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.638+0000 7f216ebcf740 1 -- start start 2024-09-11T23:19:09.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.638+0000 7f216ebcf740 1 Processor -- start 2024-09-11T23:19:09.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.638+0000 7f216ebcf740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5574217735d0 0x5574217821c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:09.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.638+0000 7f216ebcf740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557421774b40 0x55742177f470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:09.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.638+0000 7f216ebcf740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557421782780 0x55742177fc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:09.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.638+0000 7f216ebcf740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x557421785480 con 0x5574217735d0 2024-09-11T23:19:09.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.638+0000 7f216ebcf740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x557421694060 con 0x557421782780 2024-09-11T23:19:09.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.638+0000 7f216ebcf740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x557421785600 con 0x557421774b40 2024-09-11T23:19:09.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.638+0000 7f216ebcf740 10 client.0 _ll_register_callbacks cb 0x5574213e78c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:19:09.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.638+0000 7f2169d91640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5574217735d0 0x5574217821c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:09.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.638+0000 7f2169d91640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5574217735d0 0x5574217821c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33346/0 (socket says 172.21.15.26:33346) 2024-09-11T23:19:09.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.638+0000 7f2169d91640 1 -- 172.21.15.26:0/1964706330 learned_addr learned my addr 172.21.15.26:0/1964706330 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:19:09.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.638+0000 7f2169d91640 1 --2- 172.21.15.26:0/1964706330 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557421782780 0x55742177fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:09.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.638+0000 7f216a592640 1 --2- 172.21.15.26:0/1964706330 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557421774b40 0x55742177f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:09.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.638+0000 7f2169d91640 1 -- 172.21.15.26:0/1964706330 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557421774b40 msgr2=0x55742177f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:19:09.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.638+0000 7f2169d91640 1 --2- 172.21.15.26:0/1964706330 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557421774b40 0x55742177f470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:09.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.638+0000 7f2169d91640 1 -- 172.21.15.26:0/1964706330 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557421782780 msgr2=0x55742177fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:19:09.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.638+0000 7f2169d91640 1 --2- 172.21.15.26:0/1964706330 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557421782780 0x55742177fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:09.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.638+0000 7f2169d91640 1 -- 172.21.15.26:0/1964706330 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55742144a3c0 con 0x5574217735d0 2024-09-11T23:19:09.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.638+0000 7f216a592640 1 --2- 172.21.15.26:0/1964706330 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557421774b40 0x55742177f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:19:09.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.639+0000 7f2169d91640 1 --2- 172.21.15.26:0/1964706330 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5574217735d0 0x5574217821c0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f21580058c0 tx=0x7f2158005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42b1c090bcece647 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:19:09.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.639+0000 7f214dffb640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:19:09.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.639+0000 7f214dffb640 1 -- 172.21.15.26:0/1964706330 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f215802a020 con 0x5574217735d0 2024-09-11T23:19:09.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.639+0000 7f214dffb640 1 -- 172.21.15.26:0/1964706330 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f2158005000 con 0x5574217735d0 2024-09-11T23:19:09.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.639+0000 7f216ebcf740 1 -- 172.21.15.26:0/1964706330 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2158005500 con 0x5574217735d0 2024-09-11T23:19:09.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.639+0000 7f214dffb640 1 -- 172.21.15.26:0/1964706330 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2158005200 con 0x5574217735d0 2024-09-11T23:19:09.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.639+0000 7f216ebcf740 10 client.15597 fetch_fsmap learned FSMap version 29 2024-09-11T23:19:09.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.639+0000 7f216ebcf740 1 -- 172.21.15.26:0/1964706330 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x557421776310 con 0x5574217735d0 2024-09-11T23:19:09.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.639+0000 7f214dffb640 1 -- 172.21.15.26:0/1964706330 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2158005500 con 0x5574217735d0 2024-09-11T23:19:09.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.639+0000 7f214dffb640 1 -- 172.21.15.26:0/1964706330 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f21580126b0 con 0x5574217735d0 2024-09-11T23:19:09.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.639+0000 7f216ebcf740 10 client.15597 fetch_fsmap finished waiting for FSMap version 29 2024-09-11T23:19:09.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.639+0000 7f216ebcf740 10 client.15597 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-11T23:19:09.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.639+0000 7f216ebcf740 20 client.15597 populate_metadata read hostname 'smithi026' 2024-09-11T23:19:09.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.639+0000 7f216ebcf740 1 --2- 172.21.15.26:0/1964706330 >> [v2:172.21.15.136:6836/1633082387,v1:172.21.15.136:6837/1633082387] conn(0x5574217776c0 0x557421779af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:09.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.639+0000 7f216ebcf740 4 client.15597 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:19:09.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.639+0000 7f216ebcf740 1 -- 172.21.15.26:0/1964706330 --> [v2:172.21.15.136:6836/1633082387,v1:172.21.15.136:6837/1633082387] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5574216e6860 con 0x5574217776c0 2024-09-11T23:19:09.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.640+0000 7f216a592640 1 --2- 172.21.15.26:0/1964706330 >> [v2:172.21.15.136:6836/1633082387,v1:172.21.15.136:6837/1633082387] conn(0x5574217776c0 0x557421779af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:09.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.640+0000 7f216a592640 1 --2- 172.21.15.26:0/1964706330 >> [v2:172.21.15.136:6836/1633082387,v1:172.21.15.136:6837/1633082387] conn(0x5574217776c0 0x557421779af0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=fd4a8f6cdef61275 server_cookie=83e9262aefef1aa2 in_seq=0 out_seq=0 2024-09-11T23:19:09.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.640+0000 7f214dffb640 10 client.15597 ms_handle_connect on v2:172.21.15.136:6836/1633082387 2024-09-11T23:19:09.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.641+0000 7f214dffb640 1 -- 172.21.15.26:0/1964706330 <== mds.3 v2:172.21.15.136:6836/1633082387 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5574216e6860 con 0x5574217776c0 2024-09-11T23:19:09.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.641+0000 7f214dffb640 10 client.15597 handle_command_reply: tid=0 2024-09-11T23:19:09.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.641+0000 7f216ebcf740 1 client.15597 shutdown 2024-09-11T23:19:09.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.641+0000 7f216ebcf740 1 -- 172.21.15.26:0/1964706330 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5574217735d0 msgr2=0x5574217821c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:09.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.641+0000 7f216ebcf740 1 --2- 172.21.15.26:0/1964706330 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5574217735d0 0x5574217821c0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f21580058c0 tx=0x7f2158005850 comp rx=0 tx=0).stop 2024-09-11T23:19:09.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.641+0000 7f216ebcf740 1 -- 172.21.15.26:0/1964706330 shutdown_connections 2024-09-11T23:19:09.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.641+0000 7f216ebcf740 1 --2- 172.21.15.26:0/1964706330 >> [v2:172.21.15.136:6836/1633082387,v1:172.21.15.136:6837/1633082387] conn(0x5574217776c0 0x557421779af0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:09.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.641+0000 7f216ebcf740 1 --2- 172.21.15.26:0/1964706330 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557421782780 0x55742177fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:09.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.641+0000 7f216ebcf740 1 --2- 172.21.15.26:0/1964706330 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557421774b40 0x55742177f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:09.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.641+0000 7f214dffb640 0 client.15597 ms_handle_reset on v2:172.21.15.136:6836/1633082387 2024-09-11T23:19:09.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.641+0000 7f216ebcf740 1 --2- 172.21.15.26:0/1964706330 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5574217735d0 0x5574217821c0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:09.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.641+0000 7f216ebcf740 1 -- 172.21.15.26:0/1964706330 >> 172.21.15.26:0/1964706330 conn(0x55742173c310 msgr2=0x557421775960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:19:09.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.641+0000 7f216ebcf740 1 -- 172.21.15.26:0/1964706330 shutdown_connections 2024-09-11T23:19:09.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.641+0000 7f216ebcf740 1 -- 172.21.15.26:0/1964706330 wait complete. 2024-09-11T23:19:09.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.642+0000 7f216ebcf740 20 client.15597 trim_cache size 0 max 16384 2024-09-11T23:19:09.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.655+0000 7f216ebcf740 1 Processor -- start 2024-09-11T23:19:09.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.655+0000 7f216ebcf740 1 -- start start 2024-09-11T23:19:09.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.656+0000 7f216ebcf740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5574217735d0 0x55742177e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:09.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.656+0000 7f216ebcf740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557421774b40 0x55742177aac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:09.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.656+0000 7f216ebcf740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557421782780 0x55742179e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:09.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.656+0000 7f216ebcf740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f21641ab630 con 0x557421774b40 2024-09-11T23:19:09.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.656+0000 7f216ebcf740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f21641ad170 con 0x5574217735d0 2024-09-11T23:19:09.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.656+0000 7f216ebcf740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f21641abd00 con 0x557421782780 2024-09-11T23:19:09.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.656+0000 7f216a592640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557421774b40 0x55742177aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:09.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.656+0000 7f216a592640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557421774b40 0x55742177aac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33350/0 (socket says 172.21.15.26:33350) 2024-09-11T23:19:09.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.656+0000 7f216a592640 1 -- 172.21.15.26:0/2644530142 learned_addr learned my addr 172.21.15.26:0/2644530142 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:19:09.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.656+0000 7f2169d91640 1 --2- 172.21.15.26:0/2644530142 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5574217735d0 0x55742177e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:09.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.656+0000 7f2169d91640 1 --2- 172.21.15.26:0/2644530142 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557421782780 0x55742179e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:09.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.656+0000 7f216a592640 1 -- 172.21.15.26:0/2644530142 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557421782780 msgr2=0x55742179e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:19:09.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.656+0000 7f216a592640 1 --2- 172.21.15.26:0/2644530142 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557421782780 0x55742179e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:09.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.656+0000 7f216a592640 1 -- 172.21.15.26:0/2644530142 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5574217735d0 msgr2=0x55742177e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:19:09.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.656+0000 7f216a592640 1 --2- 172.21.15.26:0/2644530142 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5574217735d0 0x55742177e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:09.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.656+0000 7f216a592640 1 -- 172.21.15.26:0/2644530142 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557421774780 con 0x557421774b40 2024-09-11T23:19:09.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.656+0000 7f2169d91640 1 --2- 172.21.15.26:0/2644530142 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557421782780 0x55742179e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:19:09.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.656+0000 7f216a592640 1 --2- 172.21.15.26:0/2644530142 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557421774b40 0x55742177aac0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f2160009900 tx=0x7f21600058b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a8c723f4b114ace server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:19:09.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.656+0000 7f214ffff640 1 -- 172.21.15.26:0/2644530142 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2160035040 con 0x557421774b40 2024-09-11T23:19:09.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.656+0000 7f214ffff640 1 -- 172.21.15.26:0/2644530142 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2160047070 con 0x557421774b40 2024-09-11T23:19:09.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.657+0000 7f214ffff640 1 -- 172.21.15.26:0/2644530142 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21600035f0 con 0x557421774b40 2024-09-11T23:19:09.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.657+0000 7f216ebcf740 1 -- 172.21.15.26:0/2644530142 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557421774b40 msgr2=0x55742177aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:09.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.657+0000 7f216ebcf740 1 --2- 172.21.15.26:0/2644530142 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557421774b40 0x55742177aac0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f2160009900 tx=0x7f21600058b0 comp rx=0 tx=0).stop 2024-09-11T23:19:09.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.657+0000 7f216ebcf740 1 -- 172.21.15.26:0/2644530142 shutdown_connections 2024-09-11T23:19:09.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.657+0000 7f216ebcf740 1 --2- 172.21.15.26:0/2644530142 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557421782780 0x55742179e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:09.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.657+0000 7f216ebcf740 1 --2- 172.21.15.26:0/2644530142 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557421774b40 0x55742177aac0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:09.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.657+0000 7f216ebcf740 1 --2- 172.21.15.26:0/2644530142 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5574217735d0 0x55742177e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:09.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.657+0000 7f216ebcf740 1 -- 172.21.15.26:0/2644530142 >> 172.21.15.26:0/2644530142 conn(0x55742173c310 msgr2=0x557421775930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:19:09.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.657+0000 7f216ebcf740 1 -- 172.21.15.26:0/2644530142 shutdown_connections 2024-09-11T23:19:09.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.657+0000 7f216ebcf740 1 -- 172.21.15.26:0/2644530142 wait complete. 2024-09-11T23:19:09.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.658+0000 7f216ebcf740 1 -- start start 2024-09-11T23:19:09.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.658+0000 7f216ebcf740 1 Processor -- start 2024-09-11T23:19:09.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.658+0000 7f216ebcf740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5574217735d0 0x55742178aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:09.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.658+0000 7f216ebcf740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557421774b40 0x55742178afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:09.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.658+0000 7f216ebcf740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557421782780 0x55742177bae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:09.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.658+0000 7f216ebcf740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f21641acaa0 con 0x557421774b40 2024-09-11T23:19:09.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.658+0000 7f216ebcf740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f21641ac3d0 con 0x557421782780 2024-09-11T23:19:09.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.658+0000 7f216ebcf740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f21641a9610 con 0x5574217735d0 2024-09-11T23:19:09.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.658+0000 7f216ebcf740 10 client.0 _ll_register_callbacks cb 0x7f215800cf30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:19:09.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.658+0000 7f216a592640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557421774b40 0x55742178afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:09.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.658+0000 7f216a592640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557421774b40 0x55742178afb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33368/0 (socket says 172.21.15.26:33368) 2024-09-11T23:19:09.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.658+0000 7f216a592640 1 -- 172.21.15.26:0/3722815396 learned_addr learned my addr 172.21.15.26:0/3722815396 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:19:09.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.658+0000 7f2169d91640 1 --2- 172.21.15.26:0/3722815396 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557421782780 0x55742177bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:09.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.658+0000 7f2169d91640 1 --2- 172.21.15.26:0/3722815396 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5574217735d0 0x55742178aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:09.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.659+0000 7f216a592640 1 -- 172.21.15.26:0/3722815396 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5574217735d0 msgr2=0x55742178aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:19:09.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.659+0000 7f216a592640 1 --2- 172.21.15.26:0/3722815396 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5574217735d0 0x55742178aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:09.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.659+0000 7f216a592640 1 -- 172.21.15.26:0/3722815396 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557421782780 msgr2=0x55742177bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:19:09.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.659+0000 7f216a592640 1 --2- 172.21.15.26:0/3722815396 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557421782780 0x55742177bae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:09.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.659+0000 7f216a592640 1 -- 172.21.15.26:0/3722815396 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55742177c180 con 0x557421774b40 2024-09-11T23:19:09.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.659+0000 7f216a592640 1 --2- 172.21.15.26:0/3722815396 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557421774b40 0x55742178afb0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f216002a9a0 tx=0x7f2160004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1766c099d842d42c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:19:09.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.659+0000 7f214dffb640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:19:09.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.659+0000 7f214dffb640 1 -- 172.21.15.26:0/3722815396 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f216004f020 con 0x557421774b40 2024-09-11T23:19:09.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.659+0000 7f214dffb640 1 -- 172.21.15.26:0/3722815396 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f2160031750 con 0x557421774b40 2024-09-11T23:19:09.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.659+0000 7f216ebcf740 1 -- 172.21.15.26:0/3722815396 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2160047070 con 0x557421774b40 2024-09-11T23:19:09.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.659+0000 7f214dffb640 1 -- 172.21.15.26:0/3722815396 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2160035070 con 0x557421774b40 2024-09-11T23:19:09.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.659+0000 7f216ebcf740 10 client.15603 fetch_fsmap learned FSMap version 29 2024-09-11T23:19:09.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.659+0000 7f216ebcf740 1 -- 172.21.15.26:0/3722815396 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x557421801070 con 0x557421774b40 2024-09-11T23:19:09.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.660+0000 7f214dffb640 1 -- 172.21.15.26:0/3722815396 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2160047070 con 0x557421774b40 2024-09-11T23:19:09.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.660+0000 7f214dffb640 1 -- 172.21.15.26:0/3722815396 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f2160003bf0 con 0x557421774b40 2024-09-11T23:19:09.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.660+0000 7f216ebcf740 10 client.15603 fetch_fsmap finished waiting for FSMap version 29 2024-09-11T23:19:09.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.660+0000 7f216ebcf740 10 client.15603 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-11T23:19:09.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.660+0000 7f216ebcf740 20 client.15603 populate_metadata read hostname 'smithi026' 2024-09-11T23:19:09.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.660+0000 7f216ebcf740 1 --2- 172.21.15.26:0/3722815396 >> [v2:172.21.15.136:6836/1633082387,v1:172.21.15.136:6837/1633082387] conn(0x5574218017b0 0x557421803bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:09.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.660+0000 7f216ebcf740 4 client.15603 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:19:09.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.660+0000 7f216ebcf740 1 -- 172.21.15.26:0/3722815396 --> [v2:172.21.15.136:6836/1633082387,v1:172.21.15.136:6837/1633082387] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5574216e0d70 con 0x5574218017b0 2024-09-11T23:19:09.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.660+0000 7f2169d91640 1 --2- 172.21.15.26:0/3722815396 >> [v2:172.21.15.136:6836/1633082387,v1:172.21.15.136:6837/1633082387] conn(0x5574218017b0 0x557421803bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:09.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.661+0000 7f2169d91640 1 --2- 172.21.15.26:0/3722815396 >> [v2:172.21.15.136:6836/1633082387,v1:172.21.15.136:6837/1633082387] conn(0x5574218017b0 0x557421803bd0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a7dff57221590257 server_cookie=a4d66b11dec79dd4 in_seq=0 out_seq=0 2024-09-11T23:19:09.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.661+0000 7f214dffb640 10 client.15603 ms_handle_connect on v2:172.21.15.136:6836/1633082387 2024-09-11T23:19:09.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.661+0000 7f214dffb640 1 -- 172.21.15.26:0/3722815396 <== mds.3 v2:172.21.15.136:6836/1633082387 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5574216e0d70 con 0x5574218017b0 2024-09-11T23:19:09.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.661+0000 7f214dffb640 10 client.15603 handle_command_reply: tid=0 2024-09-11T23:19:09.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.661+0000 7f216ebcf740 1 client.15603 shutdown 2024-09-11T23:19:09.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.661+0000 7f216ebcf740 1 -- 172.21.15.26:0/3722815396 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557421774b40 msgr2=0x55742178afb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:09.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.661+0000 7f216ebcf740 1 --2- 172.21.15.26:0/3722815396 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557421774b40 0x55742178afb0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f216002a9a0 tx=0x7f2160004360 comp rx=0 tx=0).stop 2024-09-11T23:19:09.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.661+0000 7f216ebcf740 1 -- 172.21.15.26:0/3722815396 shutdown_connections 2024-09-11T23:19:09.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.661+0000 7f216ebcf740 1 --2- 172.21.15.26:0/3722815396 >> [v2:172.21.15.136:6836/1633082387,v1:172.21.15.136:6837/1633082387] conn(0x5574218017b0 0x557421803bd0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:09.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.661+0000 7f216ebcf740 1 --2- 172.21.15.26:0/3722815396 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557421782780 0x55742177bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:09.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.661+0000 7f216ebcf740 1 --2- 172.21.15.26:0/3722815396 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557421774b40 0x55742178afb0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:09.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.661+0000 7f216ebcf740 1 --2- 172.21.15.26:0/3722815396 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5574217735d0 0x55742178aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:09.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.661+0000 7f214dffb640 0 client.15603 ms_handle_reset on v2:172.21.15.136:6836/1633082387 2024-09-11T23:19:09.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.661+0000 7f216ebcf740 1 -- 172.21.15.26:0/3722815396 >> 172.21.15.26:0/3722815396 conn(0x55742173c310 msgr2=0x557421775080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:19:09.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.662+0000 7f216ebcf740 1 -- 172.21.15.26:0/3722815396 shutdown_connections 2024-09-11T23:19:09.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.662+0000 7f216ebcf740 1 -- 172.21.15.26:0/3722815396 wait complete. 2024-09-11T23:19:09.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.662+0000 7f216ebcf740 20 client.15603 trim_cache size 0 max 16384 2024-09-11T23:19:09.678 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2024-09-11T23:19:09.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.662+0000 7f216c7f6640 1 -- 172.21.15.26:0/4220639429 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f21500a2e10 msgr2=0x7f21500a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:09.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.662+0000 7f216c7f6640 1 --2- 172.21.15.26:0/4220639429 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f21500a2e10 0x7f21500a52d0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f215c0cd4d0 tx=0x7f215c00af30 comp rx=0 tx=0).stop 2024-09-11T23:19:09.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.662+0000 7f216c7f6640 1 -- 172.21.15.26:0/4220639429 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2164102ef0 msgr2=0x7f216419cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:09.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.662+0000 7f216c7f6640 1 --2- 172.21.15.26:0/4220639429 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2164102ef0 0x7f216419cd80 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f215c00cce0 tx=0x7f215c00a620 comp rx=0 tx=0).stop 2024-09-11T23:19:09.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.662+0000 7f216c7f6640 1 -- 172.21.15.26:0/4220639429 shutdown_connections 2024-09-11T23:19:09.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.662+0000 7f216c7f6640 1 --2- 172.21.15.26:0/4220639429 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f21500a2e10 0x7f21500a52d0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:09.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.662+0000 7f216c7f6640 1 --2- 172.21.15.26:0/4220639429 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2164102ef0 0x7f216419cd80 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:09.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.662+0000 7f216c7f6640 1 --2- 172.21.15.26:0/4220639429 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2164102570 0x7f21641989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:09.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.662+0000 7f216c7f6640 1 --2- 172.21.15.26:0/4220639429 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2164100e20 0x7f21641984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:09.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.662+0000 7f216c7f6640 1 -- 172.21.15.26:0/4220639429 >> 172.21.15.26:0/4220639429 conn(0x7f21640fc7d0 msgr2=0x7f216406a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:19:09.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.663+0000 7f216c7f6640 1 -- 172.21.15.26:0/4220639429 shutdown_connections 2024-09-11T23:19:09.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:09.663+0000 7f216c7f6640 1 -- 172.21.15.26:0/4220639429 wait complete. 2024-09-11T23:19:10.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:19:09 smithi136 ceph-mon[35121]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 24 KiB/s wr, 29 op/s 2024-09-11T23:19:10.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:19:09 smithi136 ceph-mon[35121]: from='client.? 172.21.15.26:0/1208627861' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:19:10.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:19:09 smithi026 ceph-mon[31172]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 24 KiB/s wr, 29 op/s 2024-09-11T23:19:10.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:19:09 smithi026 ceph-mon[31172]: from='client.? 172.21.15.26:0/1208627861' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:19:10.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:19:09 smithi076 ceph-mon[35296]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 24 KiB/s wr, 29 op/s 2024-09-11T23:19:10.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:19:09 smithi076 ceph-mon[35296]: from='client.? 172.21.15.26:0/1208627861' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:19:10.395 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-11T23:19:10.917 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24485 quiesce db / --timeout 90 --expiration 144.5 --await-for 45.0' 2024-09-11T23:19:10.917 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24485 quiesce db / --timeout 90 --expiration 144.5 --await-for 45.0 2024-09-11T23:19:11.002 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:19:11.453 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.453+0000 7f533f3c0640 1 -- 172.21.15.26:0/1644592819 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f533810c0d0 msgr2=0x7f533810e550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:11.453 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.453+0000 7f533f3c0640 1 --2- 172.21.15.26:0/1644592819 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f533810c0d0 0x7f533810e550 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f5330009900 tx=0x7f533002a480 comp rx=0 tx=0).stop 2024-09-11T23:19:11.453 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.453+0000 7f533f3c0640 1 -- 172.21.15.26:0/1644592819 shutdown_connections 2024-09-11T23:19:11.453 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.453+0000 7f533f3c0640 1 --2- 172.21.15.26:0/1644592819 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5338103620 0x7f533810bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:11.453 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.453+0000 7f533f3c0640 1 --2- 172.21.15.26:0/1644592819 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f533810c0d0 0x7f533810e550 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:11.453 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.453+0000 7f533f3c0640 1 --2- 172.21.15.26:0/1644592819 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5338102d00 0x7f53381030e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:11.453 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.453+0000 7f533f3c0640 1 -- 172.21.15.26:0/1644592819 >> 172.21.15.26:0/1644592819 conn(0x7f5338077990 msgr2=0x7f53380fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:19:11.454 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.453+0000 7f533f3c0640 1 -- 172.21.15.26:0/1644592819 shutdown_connections 2024-09-11T23:19:11.454 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.453+0000 7f533f3c0640 1 -- 172.21.15.26:0/1644592819 wait complete. 2024-09-11T23:19:11.456 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.454+0000 7f533f3c0640 1 Processor -- start 2024-09-11T23:19:11.456 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.454+0000 7f533f3c0640 1 -- start start 2024-09-11T23:19:11.456 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.454+0000 7f533f3c0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5338102d00 0x7f53381982c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:11.456 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.454+0000 7f533f3c0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5338103620 0x7f5338198800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:11.456 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.454+0000 7f533f3c0640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f533810c0d0 0x7f533819cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:11.456 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.454+0000 7f533f3c0640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f5338111eb0 con 0x7f5338102d00 2024-09-11T23:19:11.456 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.454+0000 7f533f3c0640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f5338111d30 con 0x7f5338103620 2024-09-11T23:19:11.456 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.454+0000 7f533f3c0640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f5338112030 con 0x7f533810c0d0 2024-09-11T23:19:11.456 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.454+0000 7f533d15c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5338102d00 0x7f53381982c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:11.456 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.454+0000 7f533d15c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5338102d00 0x7f53381982c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54626/0 (socket says 172.21.15.26:54626) 2024-09-11T23:19:11.456 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.454+0000 7f533d15c640 1 -- 172.21.15.26:0/3261703972 learned_addr learned my addr 172.21.15.26:0/3261703972 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:19:11.457 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.454+0000 7f533d95d640 1 --2- 172.21.15.26:0/3261703972 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f533810c0d0 0x7f533819cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:11.457 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.455+0000 7f533d15c640 1 -- 172.21.15.26:0/3261703972 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f533810c0d0 msgr2=0x7f533819cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:11.457 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.455+0000 7f533d15c640 1 --2- 172.21.15.26:0/3261703972 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f533810c0d0 0x7f533819cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:11.457 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.455+0000 7f533c95b640 1 --2- 172.21.15.26:0/3261703972 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5338103620 0x7f5338198800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:11.457 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.455+0000 7f533d15c640 1 -- 172.21.15.26:0/3261703972 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5338103620 msgr2=0x7f5338198800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:11.457 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.455+0000 7f533d15c640 1 --2- 172.21.15.26:0/3261703972 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5338103620 0x7f5338198800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:11.457 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.455+0000 7f533d15c640 1 -- 172.21.15.26:0/3261703972 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f533819d2b0 con 0x7f5338102d00 2024-09-11T23:19:11.457 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.455+0000 7f533c95b640 1 --2- 172.21.15.26:0/3261703972 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5338103620 0x7f5338198800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:19:11.457 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.455+0000 7f533d15c640 1 --2- 172.21.15.26:0/3261703972 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5338102d00 0x7f53381982c0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f532800eab0 tx=0x7f532800ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:19:11.457 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.455+0000 7f53367fc640 1 -- 172.21.15.26:0/3261703972 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f532800ce40 con 0x7f5338102d00 2024-09-11T23:19:11.457 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.455+0000 7f53367fc640 1 -- 172.21.15.26:0/3261703972 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5328011950 con 0x7f5338102d00 2024-09-11T23:19:11.457 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.455+0000 7f53367fc640 1 -- 172.21.15.26:0/3261703972 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5328010ae0 con 0x7f5338102d00 2024-09-11T23:19:11.457 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.455+0000 7f533f3c0640 1 -- 172.21.15.26:0/3261703972 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f533819d5a0 con 0x7f5338102d00 2024-09-11T23:19:11.458 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.455+0000 7f533f3c0640 1 -- 172.21.15.26:0/3261703972 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53381a4e30 con 0x7f5338102d00 2024-09-11T23:19:11.460 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.457+0000 7f53367fc640 1 -- 172.21.15.26:0/3261703972 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 25) ==== 138066+0+0 (secure 0 0 0) 0x7f5328011b00 con 0x7f5338102d00 2024-09-11T23:19:11.460 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.458+0000 7f534222e740 1 Processor -- start 2024-09-11T23:19:11.460 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.458+0000 7f534222e740 1 -- start start 2024-09-11T23:19:11.460 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.458+0000 7f534222e740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cae960aaf0 0x55cae960aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:11.460 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.458+0000 7f534222e740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cae960b410 0x55cae9615770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:11.460 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.458+0000 7f534222e740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cae9615cb0 0x55cae96180a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:11.460 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.458+0000 7f534222e740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55cae956f2f0 con 0x55cae960aaf0 2024-09-11T23:19:11.460 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.458+0000 7f534222e740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55cae9517270 con 0x55cae960b410 2024-09-11T23:19:11.460 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.458+0000 7f534222e740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55cae956b2b0 con 0x55cae9615cb0 2024-09-11T23:19:11.460 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.459+0000 7f53367fc640 1 --2- 172.21.15.26:0/3261703972 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f53200a2ec0 0x7f53200a5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:11.461 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.459+0000 7f533d95d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cae960aaf0 0x55cae960aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:11.461 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.459+0000 7f533d95d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cae960aaf0 0x55cae960aed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54636/0 (socket says 172.21.15.26:54636) 2024-09-11T23:19:11.461 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.459+0000 7f533d95d640 1 -- 172.21.15.26:0/185970260 learned_addr learned my addr 172.21.15.26:0/185970260 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:19:11.461 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.459+0000 7f533d15c640 1 --2- 172.21.15.26:0/185970260 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cae960b410 0x55cae9615770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:11.461 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.459+0000 7f53367fc640 1 -- 172.21.15.26:0/3261703972 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f53280cff40 con 0x7f5338102d00 2024-09-11T23:19:11.461 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.459+0000 7f533d95d640 1 --2- 172.21.15.26:0/3261703972 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f53200a2ec0 0x7f53200a5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:11.461 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.459+0000 7f533c95b640 1 --2- 172.21.15.26:0/185970260 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cae9615cb0 0x55cae96180a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:11.461 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.459+0000 7f533d95d640 1 -- 172.21.15.26:0/185970260 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cae9615cb0 msgr2=0x55cae96180a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:19:11.461 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.459+0000 7f533d95d640 1 --2- 172.21.15.26:0/185970260 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cae9615cb0 0x55cae96180a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:11.461 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.459+0000 7f533d95d640 1 -- 172.21.15.26:0/185970260 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cae960b410 msgr2=0x55cae9615770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:19:11.461 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.459+0000 7f533d95d640 1 --2- 172.21.15.26:0/185970260 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cae960b410 0x55cae9615770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:11.461 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.459+0000 7f533d95d640 1 -- 172.21.15.26:0/185970260 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cae9516340 con 0x55cae960aaf0 2024-09-11T23:19:11.461 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.459+0000 7f533d95d640 1 --2- 172.21.15.26:0/185970260 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cae960aaf0 0x55cae960aed0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f5330033000 tx=0x7f53300315b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81fb6e09bbfefc1e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:19:11.461 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.460+0000 7f531f7fe640 1 -- 172.21.15.26:0/185970260 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f533004c070 con 0x55cae960aaf0 2024-09-11T23:19:11.462 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.460+0000 7f531f7fe640 1 -- 172.21.15.26:0/185970260 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5330003ad0 con 0x55cae960aaf0 2024-09-11T23:19:11.462 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.460+0000 7f531f7fe640 1 -- 172.21.15.26:0/185970260 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5330035040 con 0x55cae960aaf0 2024-09-11T23:19:11.466 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.460+0000 7f533d95d640 1 --2- 172.21.15.26:0/3261703972 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f53200a2ec0 0x7f53200a5380 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f5330049000 tx=0x7f533004a000 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:19:11.466 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.460+0000 7f534222e740 1 -- 172.21.15.26:0/185970260 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cae960aaf0 msgr2=0x55cae960aed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:11.466 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.460+0000 7f534222e740 1 --2- 172.21.15.26:0/185970260 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cae960aaf0 0x55cae960aed0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f5330033000 tx=0x7f53300315b0 comp rx=0 tx=0).stop 2024-09-11T23:19:11.466 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.461+0000 7f534222e740 1 -- 172.21.15.26:0/185970260 shutdown_connections 2024-09-11T23:19:11.466 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.461+0000 7f534222e740 1 --2- 172.21.15.26:0/185970260 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cae960b410 0x55cae9615770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:11.466 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.461+0000 7f534222e740 1 --2- 172.21.15.26:0/185970260 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cae9615cb0 0x55cae96180a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:11.466 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.461+0000 7f534222e740 1 --2- 172.21.15.26:0/185970260 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cae960aaf0 0x55cae960aed0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:11.466 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.461+0000 7f534222e740 1 -- 172.21.15.26:0/185970260 >> 172.21.15.26:0/185970260 conn(0x55cae9607060 msgr2=0x55cae9607440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:19:11.466 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.461+0000 7f534222e740 1 -- 172.21.15.26:0/185970260 shutdown_connections 2024-09-11T23:19:11.466 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.461+0000 7f534222e740 1 -- 172.21.15.26:0/185970260 wait complete. 2024-09-11T23:19:11.466 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.461+0000 7f534222e740 1 -- start start 2024-09-11T23:19:11.466 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.461+0000 7f534222e740 1 Processor -- start 2024-09-11T23:19:11.467 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.461+0000 7f534222e740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cae960aaf0 0x55cae9612b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:11.467 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.461+0000 7f534222e740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cae960b410 0x55cae96130a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:11.467 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.461+0000 7f534222e740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cae9615cb0 0x55cae96135e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:11.467 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.461+0000 7f534222e740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55cae9618880 con 0x55cae960aaf0 2024-09-11T23:19:11.467 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.461+0000 7f534222e740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55cae94f8290 con 0x55cae9615cb0 2024-09-11T23:19:11.467 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.461+0000 7f534222e740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55cae9618a00 con 0x55cae960b410 2024-09-11T23:19:11.467 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.462+0000 7f534222e740 10 client.0 _ll_register_callbacks cb 0x55cae9529140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:19:11.467 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.462+0000 7f533d15c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cae960aaf0 0x55cae9612b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:11.467 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.462+0000 7f533d15c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cae960aaf0 0x55cae9612b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54638/0 (socket says 172.21.15.26:54638) 2024-09-11T23:19:11.467 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.462+0000 7f533d15c640 1 -- 172.21.15.26:0/865704161 learned_addr learned my addr 172.21.15.26:0/865704161 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:19:11.467 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.462+0000 7f533c95b640 1 --2- 172.21.15.26:0/865704161 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cae960b410 0x55cae96130a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:11.467 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.462+0000 7f533d15c640 1 -- 172.21.15.26:0/865704161 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cae960b410 msgr2=0x55cae96130a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:19:11.467 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.462+0000 7f533d95d640 1 --2- 172.21.15.26:0/865704161 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cae9615cb0 0x55cae96135e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:11.468 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.462+0000 7f533d15c640 1 --2- 172.21.15.26:0/865704161 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cae960b410 0x55cae96130a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:11.468 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.462+0000 7f533d15c640 1 -- 172.21.15.26:0/865704161 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cae9615cb0 msgr2=0x55cae96135e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:19:11.468 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.462+0000 7f533d15c640 1 --2- 172.21.15.26:0/865704161 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cae9615cb0 0x55cae96135e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:11.468 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.462+0000 7f533d15c640 1 -- 172.21.15.26:0/865704161 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cae9282210 con 0x55cae960aaf0 2024-09-11T23:19:11.468 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.462+0000 7f533d95d640 1 --2- 172.21.15.26:0/865704161 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cae9615cb0 0x55cae96135e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:19:11.468 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.462+0000 7f533d15c640 1 --2- 172.21.15.26:0/865704161 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cae960aaf0 0x55cae9612b60 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f53280a4900 tx=0x7f532809bdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0a5dd1c901ad5d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:19:11.468 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.462+0000 7f531cff9640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:19:11.468 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.462+0000 7f531cff9640 1 -- 172.21.15.26:0/865704161 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53280a0570 con 0x55cae960aaf0 2024-09-11T23:19:11.468 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.462+0000 7f534222e740 1 -- 172.21.15.26:0/865704161 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5330003ad0 con 0x55cae960aaf0 2024-09-11T23:19:11.468 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.462+0000 7f531cff9640 1 -- 172.21.15.26:0/865704161 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f53280202b0 con 0x55cae960aaf0 2024-09-11T23:19:11.468 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.463+0000 7f531cff9640 1 -- 172.21.15.26:0/865704161 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f53280204b0 con 0x55cae960aaf0 2024-09-11T23:19:11.468 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.463+0000 7f534222e740 10 client.15615 fetch_fsmap learned FSMap version 29 2024-09-11T23:19:11.468 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.463+0000 7f534222e740 1 -- 172.21.15.26:0/865704161 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55cae934e620 con 0x55cae960aaf0 2024-09-11T23:19:11.469 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.463+0000 7f531cff9640 1 -- 172.21.15.26:0/865704161 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5330003ad0 con 0x55cae960aaf0 2024-09-11T23:19:11.469 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.463+0000 7f531cff9640 1 -- 172.21.15.26:0/865704161 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f532800f040 con 0x55cae960aaf0 2024-09-11T23:19:11.469 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.463+0000 7f534222e740 10 client.15615 fetch_fsmap finished waiting for FSMap version 29 2024-09-11T23:19:11.469 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.463+0000 7f534222e740 10 client.15615 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:19:11.469 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.463+0000 7f534222e740 20 client.15615 populate_metadata read hostname 'smithi026' 2024-09-11T23:19:11.469 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.463+0000 7f534222e740 1 --2- 172.21.15.26:0/865704161 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55cae9625c70 0x55cae9628080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:11.472 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.463+0000 7f534222e740 4 client.15615 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:19:11.472 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.463+0000 7f534222e740 1 -- 172.21.15.26:0/865704161 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cae957ca20 con 0x55cae9625c70 2024-09-11T23:19:11.472 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.463+0000 7f533c95b640 1 --2- 172.21.15.26:0/865704161 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55cae9625c70 0x55cae9628080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:11.473 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.464+0000 7f533c95b640 1 --2- 172.21.15.26:0/865704161 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55cae9625c70 0x55cae9628080 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a022e86e27146d3 server_cookie=1393b7f2bfd4dafa in_seq=0 out_seq=0 2024-09-11T23:19:11.473 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.464+0000 7f531cff9640 10 client.15615 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:19:11.473 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.465+0000 7f531cff9640 1 -- 172.21.15.26:0/865704161 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55cae957ca20 con 0x55cae9625c70 2024-09-11T23:19:11.473 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.465+0000 7f531cff9640 10 client.15615 handle_command_reply: tid=0 2024-09-11T23:19:11.473 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.465+0000 7f534222e740 1 client.15615 shutdown 2024-09-11T23:19:11.473 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.465+0000 7f534222e740 1 -- 172.21.15.26:0/865704161 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cae960aaf0 msgr2=0x55cae9612b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:11.473 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.465+0000 7f534222e740 1 --2- 172.21.15.26:0/865704161 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cae960aaf0 0x55cae9612b60 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f53280a4900 tx=0x7f532809bdc0 comp rx=0 tx=0).stop 2024-09-11T23:19:11.473 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.465+0000 7f534222e740 1 -- 172.21.15.26:0/865704161 shutdown_connections 2024-09-11T23:19:11.473 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.465+0000 7f534222e740 1 --2- 172.21.15.26:0/865704161 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cae9615cb0 0x55cae96135e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:11.473 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.465+0000 7f534222e740 1 --2- 172.21.15.26:0/865704161 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55cae9625c70 0x55cae9628080 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:11.473 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.465+0000 7f534222e740 1 --2- 172.21.15.26:0/865704161 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cae960b410 0x55cae96130a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:11.473 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.465+0000 7f534222e740 1 --2- 172.21.15.26:0/865704161 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cae960aaf0 0x55cae9612b60 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:11.473 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.465+0000 7f531cff9640 0 client.15615 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:19:11.473 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.465+0000 7f534222e740 1 -- 172.21.15.26:0/865704161 >> 172.21.15.26:0/865704161 conn(0x55cae9607060 msgr2=0x55cae960cdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:19:11.474 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.465+0000 7f534222e740 1 -- 172.21.15.26:0/865704161 shutdown_connections 2024-09-11T23:19:11.474 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.465+0000 7f534222e740 1 -- 172.21.15.26:0/865704161 wait complete. 2024-09-11T23:19:11.474 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.465+0000 7f534222e740 20 client.15615 trim_cache size 0 max 16384 2024-09-11T23:19:11.481 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.480+0000 7f534222e740 1 Processor -- start 2024-09-11T23:19:11.481 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.480+0000 7f534222e740 1 -- start start 2024-09-11T23:19:11.482 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.480+0000 7f534222e740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cae960aaf0 0x55cae9613410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:11.482 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.480+0000 7f534222e740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cae960b410 0x55cae962d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:11.482 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.480+0000 7f534222e740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cae9615cb0 0x55cae962ded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:11.482 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.480+0000 7f534222e740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f53381ad7f0 con 0x55cae960aaf0 2024-09-11T23:19:11.482 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.480+0000 7f534222e740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f53381af330 con 0x55cae960b410 2024-09-11T23:19:11.482 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.480+0000 7f534222e740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f53381adec0 con 0x55cae9615cb0 2024-09-11T23:19:11.482 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.480+0000 7f533d15c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cae960aaf0 0x55cae9613410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:11.482 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.480+0000 7f533d15c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cae960aaf0 0x55cae9613410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54646/0 (socket says 172.21.15.26:54646) 2024-09-11T23:19:11.482 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.480+0000 7f533d15c640 1 -- 172.21.15.26:0/2070934424 learned_addr learned my addr 172.21.15.26:0/2070934424 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:19:11.482 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.480+0000 7f533d95d640 1 --2- 172.21.15.26:0/2070934424 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cae9615cb0 0x55cae962ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:11.482 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.480+0000 7f533c95b640 1 --2- 172.21.15.26:0/2070934424 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cae960b410 0x55cae962d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:11.482 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.480+0000 7f533d15c640 1 -- 172.21.15.26:0/2070934424 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cae9615cb0 msgr2=0x55cae962ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:19:11.482 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.480+0000 7f533d15c640 1 --2- 172.21.15.26:0/2070934424 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cae9615cb0 0x55cae962ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:11.483 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.480+0000 7f533d15c640 1 -- 172.21.15.26:0/2070934424 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cae960b410 msgr2=0x55cae962d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:19:11.483 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.480+0000 7f533d15c640 1 --2- 172.21.15.26:0/2070934424 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cae960b410 0x55cae962d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:11.483 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.480+0000 7f533d15c640 1 -- 172.21.15.26:0/2070934424 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cae9606e90 con 0x55cae960aaf0 2024-09-11T23:19:11.483 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.480+0000 7f533c95b640 1 --2- 172.21.15.26:0/2070934424 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cae960b410 0x55cae962d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:19:11.483 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.480+0000 7f533d15c640 1 --2- 172.21.15.26:0/2070934424 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cae960aaf0 0x55cae9613410 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f532800ea80 tx=0x7f532809be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc173d1162fafd65 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:19:11.483 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.481+0000 7f531effd640 1 -- 172.21.15.26:0/2070934424 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f532809bf90 con 0x55cae960aaf0 2024-09-11T23:19:11.483 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.481+0000 7f531effd640 1 -- 172.21.15.26:0/2070934424 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5328065020 con 0x55cae960aaf0 2024-09-11T23:19:11.483 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.481+0000 7f531effd640 1 -- 172.21.15.26:0/2070934424 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53280a4a50 con 0x55cae960aaf0 2024-09-11T23:19:11.483 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.481+0000 7f534222e740 1 -- 172.21.15.26:0/2070934424 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cae960aaf0 msgr2=0x55cae9613410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:11.483 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.481+0000 7f534222e740 1 --2- 172.21.15.26:0/2070934424 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cae960aaf0 0x55cae9613410 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f532800ea80 tx=0x7f532809be30 comp rx=0 tx=0).stop 2024-09-11T23:19:11.483 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.481+0000 7f534222e740 1 -- 172.21.15.26:0/2070934424 shutdown_connections 2024-09-11T23:19:11.483 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.481+0000 7f534222e740 1 --2- 172.21.15.26:0/2070934424 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cae960b410 0x55cae962d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:11.483 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.481+0000 7f534222e740 1 --2- 172.21.15.26:0/2070934424 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cae9615cb0 0x55cae962ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:11.483 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.481+0000 7f534222e740 1 --2- 172.21.15.26:0/2070934424 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cae960aaf0 0x55cae9613410 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:11.484 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.481+0000 7f534222e740 1 -- 172.21.15.26:0/2070934424 >> 172.21.15.26:0/2070934424 conn(0x55cae9607060 msgr2=0x55cae9614be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:19:11.484 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.481+0000 7f534222e740 1 -- 172.21.15.26:0/2070934424 shutdown_connections 2024-09-11T23:19:11.484 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.481+0000 7f534222e740 1 -- 172.21.15.26:0/2070934424 wait complete. 2024-09-11T23:19:11.484 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.482+0000 7f534222e740 1 -- start start 2024-09-11T23:19:11.484 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.482+0000 7f534222e740 1 Processor -- start 2024-09-11T23:19:11.487 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.482+0000 7f534222e740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cae960aaf0 0x55cae9619a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:11.487 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.482+0000 7f534222e740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cae960b410 0x55cae9619fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:11.487 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.482+0000 7f534222e740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cae9615cb0 0x55cae961a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:11.487 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.482+0000 7f534222e740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f53381aec60 con 0x55cae9615cb0 2024-09-11T23:19:11.487 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.482+0000 7f534222e740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f53381ae590 con 0x55cae960aaf0 2024-09-11T23:19:11.487 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.482+0000 7f534222e740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f53381ab7d0 con 0x55cae960b410 2024-09-11T23:19:11.487 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.482+0000 7f534222e740 10 client.0 _ll_register_callbacks cb 0x7f5328098460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:19:11.487 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.482+0000 7f533d95d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cae9615cb0 0x55cae961a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:11.487 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.482+0000 7f533d95d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cae9615cb0 0x55cae961a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54652/0 (socket says 172.21.15.26:54652) 2024-09-11T23:19:11.487 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.482+0000 7f533d95d640 1 -- 172.21.15.26:0/2962769391 learned_addr learned my addr 172.21.15.26:0/2962769391 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:19:11.487 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.482+0000 7f533d15c640 1 --2- 172.21.15.26:0/2962769391 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cae960aaf0 0x55cae9619a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:11.487 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.482+0000 7f533c95b640 1 --2- 172.21.15.26:0/2962769391 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cae960b410 0x55cae9619fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:11.488 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.483+0000 7f533d95d640 1 -- 172.21.15.26:0/2962769391 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cae960b410 msgr2=0x55cae9619fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:19:11.488 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.483+0000 7f533d95d640 1 --2- 172.21.15.26:0/2962769391 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cae960b410 0x55cae9619fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:11.488 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.483+0000 7f533d95d640 1 -- 172.21.15.26:0/2962769391 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cae960aaf0 msgr2=0x55cae9619a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:19:11.488 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.483+0000 7f533d95d640 1 --2- 172.21.15.26:0/2962769391 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cae960aaf0 0x55cae9619a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:11.488 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.483+0000 7f533d95d640 1 -- 172.21.15.26:0/2962769391 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cae968f9e0 con 0x55cae9615cb0 2024-09-11T23:19:11.488 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.483+0000 7f533d15c640 1 --2- 172.21.15.26:0/2962769391 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cae960aaf0 0x55cae9619a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:19:11.488 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.483+0000 7f533d95d640 1 --2- 172.21.15.26:0/2962769391 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cae9615cb0 0x55cae961a4e0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f5330033000 tx=0x7f533003d7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b63429ca5c797c1f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:19:11.488 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.483+0000 7f531cff9640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:19:11.488 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.483+0000 7f531cff9640 1 -- 172.21.15.26:0/2962769391 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f533005f060 con 0x55cae9615cb0 2024-09-11T23:19:11.488 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.483+0000 7f531cff9640 1 -- 172.21.15.26:0/2962769391 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f533003d9f0 con 0x55cae9615cb0 2024-09-11T23:19:11.488 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.483+0000 7f534222e740 1 -- 172.21.15.26:0/2962769391 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5328065020 con 0x55cae9615cb0 2024-09-11T23:19:11.488 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.483+0000 7f531cff9640 1 -- 172.21.15.26:0/2962769391 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f5330035070 con 0x55cae9615cb0 2024-09-11T23:19:11.488 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.483+0000 7f534222e740 10 client.15621 fetch_fsmap learned FSMap version 29 2024-09-11T23:19:11.488 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.483+0000 7f534222e740 1 -- 172.21.15.26:0/2962769391 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55cae9690c40 con 0x55cae9615cb0 2024-09-11T23:19:11.489 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.484+0000 7f531cff9640 1 -- 172.21.15.26:0/2962769391 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5328065020 con 0x55cae9615cb0 2024-09-11T23:19:11.489 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.484+0000 7f531cff9640 1 -- 172.21.15.26:0/2962769391 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f5330059030 con 0x55cae9615cb0 2024-09-11T23:19:11.489 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.484+0000 7f534222e740 10 client.15621 fetch_fsmap finished waiting for FSMap version 29 2024-09-11T23:19:11.489 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.484+0000 7f534222e740 10 client.15621 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:19:11.489 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.484+0000 7f534222e740 20 client.15621 populate_metadata read hostname 'smithi026' 2024-09-11T23:19:11.489 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.484+0000 7f534222e740 1 --2- 172.21.15.26:0/2962769391 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55cae9625c70 0x55cae9692f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:11.489 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.484+0000 7f534222e740 4 client.15621 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T23:19:11.489 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.484+0000 7f534222e740 1 -- 172.21.15.26:0/2962769391 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55cae9576d70 con 0x55cae9625c70 2024-09-11T23:19:11.489 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.484+0000 7f533d15c640 1 --2- 172.21.15.26:0/2962769391 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55cae9625c70 0x55cae9692f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:11.489 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.484+0000 7f533d15c640 1 --2- 172.21.15.26:0/2962769391 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55cae9625c70 0x55cae9692f20 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=169ed931d30fb74f server_cookie=b5a9e8f97557cc27 in_seq=0 out_seq=0 2024-09-11T23:19:11.489 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.484+0000 7f531cff9640 10 client.15621 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:19:11.492 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.492+0000 7f531cff9640 1 -- 172.21.15.26:0/2962769391 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55cae9576d70 con 0x55cae9625c70 2024-09-11T23:19:11.492 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.492+0000 7f531cff9640 10 client.15621 handle_command_reply: tid=0 2024-09-11T23:19:11.492 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.492+0000 7f534222e740 1 client.15621 shutdown 2024-09-11T23:19:11.492 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.492+0000 7f534222e740 1 -- 172.21.15.26:0/2962769391 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cae9615cb0 msgr2=0x55cae961a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:11.492 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.492+0000 7f534222e740 1 --2- 172.21.15.26:0/2962769391 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cae9615cb0 0x55cae961a4e0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f5330033000 tx=0x7f533003d7f0 comp rx=0 tx=0).stop 2024-09-11T23:19:11.493 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.492+0000 7f534222e740 1 -- 172.21.15.26:0/2962769391 shutdown_connections 2024-09-11T23:19:11.493 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.492+0000 7f534222e740 1 --2- 172.21.15.26:0/2962769391 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55cae9625c70 0x55cae9692f20 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:11.493 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.492+0000 7f534222e740 1 --2- 172.21.15.26:0/2962769391 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cae9615cb0 0x55cae961a4e0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:11.493 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.492+0000 7f534222e740 1 --2- 172.21.15.26:0/2962769391 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cae960b410 0x55cae9619fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:11.493 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.492+0000 7f531cff9640 0 client.15621 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:19:11.493 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.492+0000 7f534222e740 1 --2- 172.21.15.26:0/2962769391 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cae960aaf0 0x55cae9619a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:11.493 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.492+0000 7f534222e740 1 -- 172.21.15.26:0/2962769391 >> 172.21.15.26:0/2962769391 conn(0x55cae9607060 msgr2=0x55cae96059a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:19:11.493 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.492+0000 7f534222e740 1 -- 172.21.15.26:0/2962769391 shutdown_connections 2024-09-11T23:19:11.493 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.492+0000 7f534222e740 1 -- 172.21.15.26:0/2962769391 wait complete. 2024-09-11T23:19:11.493 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.492+0000 7f534222e740 20 client.15621 trim_cache size 0 max 16384 2024-09-11T23:19:11.494 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-09-11T23:19:11.494 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "message": "", 2024-09-11T23:19:11.494 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-09-11T23:19:11.494 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-09-11T23:19:11.494 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.493+0000 7f533f3c0640 1 -- 172.21.15.26:0/3261703972 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f53200a2ec0 msgr2=0x7f53200a5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:11.494 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.493+0000 7f533f3c0640 1 --2- 172.21.15.26:0/3261703972 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f53200a2ec0 0x7f53200a5380 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f5330049000 tx=0x7f533004a000 comp rx=0 tx=0).stop 2024-09-11T23:19:11.494 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.493+0000 7f533f3c0640 1 -- 172.21.15.26:0/3261703972 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5338102d00 msgr2=0x7f53381982c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:11.495 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.493+0000 7f533f3c0640 1 --2- 172.21.15.26:0/3261703972 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5338102d00 0x7f53381982c0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f532800eab0 tx=0x7f532800ef70 comp rx=0 tx=0).stop 2024-09-11T23:19:11.495 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.493+0000 7f533f3c0640 1 -- 172.21.15.26:0/3261703972 shutdown_connections 2024-09-11T23:19:11.495 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.493+0000 7f533f3c0640 1 --2- 172.21.15.26:0/3261703972 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f53200a2ec0 0x7f53200a5380 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:11.495 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.493+0000 7f533f3c0640 1 --2- 172.21.15.26:0/3261703972 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5338103620 0x7f5338198800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:11.495 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.493+0000 7f533f3c0640 1 --2- 172.21.15.26:0/3261703972 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f533810c0d0 0x7f533819cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:11.495 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.493+0000 7f533f3c0640 1 --2- 172.21.15.26:0/3261703972 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5338102d00 0x7f53381982c0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:11.495 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.493+0000 7f533f3c0640 1 -- 172.21.15.26:0/3261703972 >> 172.21.15.26:0/3261703972 conn(0x7f5338077990 msgr2=0x7f53380fcf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:19:11.495 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.493+0000 7f533f3c0640 1 -- 172.21.15.26:0/3261703972 shutdown_connections 2024-09-11T23:19:11.495 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:11.493+0000 7f533f3c0640 1 -- 172.21.15.26:0/3261703972 wait complete. 2024-09-11T23:19:11.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.585+0000 7fee66b56640 1 -- 172.21.15.26:0/2037772313 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee60102570 msgr2=0x7fee601029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:11.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.585+0000 7fee66b56640 1 --2- 172.21.15.26:0/2037772313 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee60102570 0x7fee601029b0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fee4c009900 tx=0x7fee4c02a480 comp rx=0 tx=0).stop 2024-09-11T23:19:11.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.585+0000 7fee66b56640 1 -- 172.21.15.26:0/2037772313 shutdown_connections 2024-09-11T23:19:11.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.585+0000 7fee66b56640 1 --2- 172.21.15.26:0/2037772313 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fee60102ef0 0x7fee6010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:11.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.585+0000 7fee66b56640 1 --2- 172.21.15.26:0/2037772313 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee60102570 0x7fee601029b0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:11.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.585+0000 7fee66b56640 1 --2- 172.21.15.26:0/2037772313 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fee60100e20 0x7fee60101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:11.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.585+0000 7fee66b56640 1 -- 172.21.15.26:0/2037772313 >> 172.21.15.26:0/2037772313 conn(0x7fee600fc7d0 msgr2=0x7fee600febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:19:11.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.585+0000 7fee66b56640 1 -- 172.21.15.26:0/2037772313 shutdown_connections 2024-09-11T23:19:11.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.586+0000 7fee66b56640 1 -- 172.21.15.26:0/2037772313 wait complete. 2024-09-11T23:19:11.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.586+0000 7fee66b56640 1 Processor -- start 2024-09-11T23:19:11.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.586+0000 7fee66b56640 1 -- start start 2024-09-11T23:19:11.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.586+0000 7fee66b56640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee60100e20 0x7fee60198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:11.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.586+0000 7fee66b56640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fee60102570 0x7fee60198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:11.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.586+0000 7fee66b56640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fee60102ef0 0x7fee6019ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:11.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.586+0000 7fee66b56640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fee60110f80 con 0x7fee60100e20 2024-09-11T23:19:11.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.586+0000 7fee66b56640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fee60110e00 con 0x7fee60102570 2024-09-11T23:19:11.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.586+0000 7fee66b56640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fee60111100 con 0x7fee60102ef0 2024-09-11T23:19:11.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.587+0000 7fee648f2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee60100e20 0x7fee60198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:11.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.587+0000 7fee648f2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee60100e20 0x7fee60198560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54668/0 (socket says 172.21.15.26:54668) 2024-09-11T23:19:11.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.587+0000 7fee648f2640 1 -- 172.21.15.26:0/1316992569 learned_addr learned my addr 172.21.15.26:0/1316992569 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:19:11.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.587+0000 7fee650f3640 1 --2- 172.21.15.26:0/1316992569 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fee60102ef0 0x7fee6019ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:11.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.587+0000 7fee5ffff640 1 --2- 172.21.15.26:0/1316992569 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fee60102570 0x7fee60198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:11.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.587+0000 7fee648f2640 1 -- 172.21.15.26:0/1316992569 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fee60102ef0 msgr2=0x7fee6019ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:11.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.587+0000 7fee648f2640 1 --2- 172.21.15.26:0/1316992569 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fee60102ef0 0x7fee6019ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:11.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.587+0000 7fee648f2640 1 -- 172.21.15.26:0/1316992569 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fee60102570 msgr2=0x7fee60198aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:11.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.587+0000 7fee648f2640 1 --2- 172.21.15.26:0/1316992569 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fee60102570 0x7fee60198aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:11.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.587+0000 7fee648f2640 1 -- 172.21.15.26:0/1316992569 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee6019d510 con 0x7fee60100e20 2024-09-11T23:19:11.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.587+0000 7fee5ffff640 1 --2- 172.21.15.26:0/1316992569 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fee60102570 0x7fee60198aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:19:11.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.587+0000 7fee648f2640 1 --2- 172.21.15.26:0/1316992569 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee60100e20 0x7fee60198560 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fee5400d950 tx=0x7fee5400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:19:11.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.587+0000 7fee5dffb640 1 -- 172.21.15.26:0/1316992569 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee54017070 con 0x7fee60100e20 2024-09-11T23:19:11.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.587+0000 7fee5dffb640 1 -- 172.21.15.26:0/1316992569 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee54009c30 con 0x7fee60100e20 2024-09-11T23:19:11.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.587+0000 7fee66b56640 1 -- 172.21.15.26:0/1316992569 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee6019d800 con 0x7fee60100e20 2024-09-11T23:19:11.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.587+0000 7fee5dffb640 1 -- 172.21.15.26:0/1316992569 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee5400f6b0 con 0x7fee60100e20 2024-09-11T23:19:11.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.588+0000 7fee66b56640 1 -- 172.21.15.26:0/1316992569 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee601a50e0 con 0x7fee60100e20 2024-09-11T23:19:11.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.590+0000 7fee5dffb640 1 -- 172.21.15.26:0/1316992569 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 25) ==== 138066+0+0 (secure 0 0 0) 0x7fee540109d0 con 0x7fee60100e20 2024-09-11T23:19:11.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.590+0000 7fee68f2f740 1 Processor -- start 2024-09-11T23:19:11.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.591+0000 7fee5dffb640 1 --2- 172.21.15.26:0/1316992569 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fee500a2e10 0x7fee500a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:11.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.591+0000 7fee68f2f740 1 -- start start 2024-09-11T23:19:11.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.591+0000 7fee650f3640 1 --2- 172.21.15.26:0/1316992569 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fee500a2e10 0x7fee500a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:11.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.591+0000 7fee5dffb640 1 -- 172.21.15.26:0/1316992569 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fee540cf390 con 0x7fee60100e20 2024-09-11T23:19:11.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.591+0000 7fee68f2f740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fcfa772b40 0x55fcfa771090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:11.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.591+0000 7fee68f2f740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fcfa7715d0 0x55fcfa771a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:11.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.591+0000 7fee68f2f740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fcfa780780 0x55fcfa782b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:11.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.591+0000 7fee68f2f740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55fcfa6d72f0 con 0x55fcfa780780 2024-09-11T23:19:11.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.591+0000 7fee68f2f740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55fcfa67ecf0 con 0x55fcfa7715d0 2024-09-11T23:19:11.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.591+0000 7fee68f2f740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55fcfa6d32b0 con 0x55fcfa772b40 2024-09-11T23:19:11.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.591+0000 7fee650f3640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fcfa780780 0x55fcfa782b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:11.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.591+0000 7fee650f3640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fcfa780780 0x55fcfa782b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54678/0 (socket says 172.21.15.26:54678) 2024-09-11T23:19:11.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.591+0000 7fee650f3640 1 -- 172.21.15.26:0/600125885 learned_addr learned my addr 172.21.15.26:0/600125885 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:19:11.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.591+0000 7fee5ffff640 1 --2- 172.21.15.26:0/600125885 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fcfa7715d0 0x55fcfa771a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:11.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.591+0000 7fee648f2640 1 --2- 172.21.15.26:0/600125885 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fcfa772b40 0x55fcfa771090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:11.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.591+0000 7fee5ffff640 1 -- 172.21.15.26:0/600125885 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fcfa772b40 msgr2=0x55fcfa771090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:19:11.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.591+0000 7fee5ffff640 1 --2- 172.21.15.26:0/600125885 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fcfa772b40 0x55fcfa771090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:11.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.591+0000 7fee5ffff640 1 -- 172.21.15.26:0/600125885 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fcfa780780 msgr2=0x55fcfa782b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:19:11.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.591+0000 7fee5ffff640 1 --2- 172.21.15.26:0/600125885 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fcfa780780 0x55fcfa782b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:11.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.591+0000 7fee5ffff640 1 -- 172.21.15.26:0/600125885 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fcfa6633a0 con 0x55fcfa7715d0 2024-09-11T23:19:11.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.591+0000 7fee650f3640 1 --2- 172.21.15.26:0/1316992569 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fee500a2e10 0x7fee500a52d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fee58014820 tx=0x7fee58014c20 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:19:11.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.592+0000 7fee5ffff640 1 --2- 172.21.15.26:0/600125885 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fcfa7715d0 0x55fcfa771a10 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fee4c000c00 tx=0x7fee4c009680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c916f3f25f7afa5b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:19:11.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.592+0000 7fee4affd640 1 -- 172.21.15.26:0/600125885 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee4c03f070 con 0x55fcfa7715d0 2024-09-11T23:19:11.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.592+0000 7fee4affd640 1 -- 172.21.15.26:0/600125885 <== mon.2 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee4c005520 con 0x55fcfa7715d0 2024-09-11T23:19:11.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.592+0000 7fee4affd640 1 -- 172.21.15.26:0/600125885 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee4c035030 con 0x55fcfa7715d0 2024-09-11T23:19:11.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.593+0000 7fee68f2f740 1 -- 172.21.15.26:0/600125885 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fcfa7715d0 msgr2=0x55fcfa771a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:11.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.593+0000 7fee68f2f740 1 --2- 172.21.15.26:0/600125885 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fcfa7715d0 0x55fcfa771a10 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fee4c000c00 tx=0x7fee4c009680 comp rx=0 tx=0).stop 2024-09-11T23:19:11.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.593+0000 7fee68f2f740 1 -- 172.21.15.26:0/600125885 shutdown_connections 2024-09-11T23:19:11.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.593+0000 7fee68f2f740 1 --2- 172.21.15.26:0/600125885 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fcfa7715d0 0x55fcfa771a10 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:11.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.593+0000 7fee68f2f740 1 --2- 172.21.15.26:0/600125885 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fcfa780780 0x55fcfa782b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:11.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.593+0000 7fee68f2f740 1 --2- 172.21.15.26:0/600125885 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fcfa772b40 0x55fcfa771090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:11.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.593+0000 7fee68f2f740 1 -- 172.21.15.26:0/600125885 >> 172.21.15.26:0/600125885 conn(0x55fcfa73a310 msgr2=0x55fcfa76dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:19:11.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.593+0000 7fee68f2f740 1 -- 172.21.15.26:0/600125885 shutdown_connections 2024-09-11T23:19:11.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.593+0000 7fee68f2f740 1 -- 172.21.15.26:0/600125885 wait complete. 2024-09-11T23:19:11.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.593+0000 7fee68f2f740 1 -- start start 2024-09-11T23:19:11.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.593+0000 7fee68f2f740 1 Processor -- start 2024-09-11T23:19:11.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.594+0000 7fee68f2f740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fcfa7715d0 0x55fcfa780110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:11.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.594+0000 7fee68f2f740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fcfa772b40 0x55fcfa77d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:11.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.594+0000 7fee68f2f740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fcfa780780 0x55fcfa77d940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:11.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.594+0000 7fee68f2f740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55fcfa783420 con 0x55fcfa772b40 2024-09-11T23:19:11.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.594+0000 7fee68f2f740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55fcfa692060 con 0x55fcfa780780 2024-09-11T23:19:11.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.594+0000 7fee68f2f740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55fcfa7835a0 con 0x55fcfa7715d0 2024-09-11T23:19:11.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.594+0000 7fee68f2f740 10 client.0 _ll_register_callbacks cb 0x55fcfa3e58c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:19:11.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.594+0000 7fee5ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fcfa772b40 0x55fcfa77d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:11.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.594+0000 7fee5ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fcfa772b40 0x55fcfa77d3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54688/0 (socket says 172.21.15.26:54688) 2024-09-11T23:19:11.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.594+0000 7fee5ffff640 1 -- 172.21.15.26:0/4064234264 learned_addr learned my addr 172.21.15.26:0/4064234264 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:19:11.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.594+0000 7fee648f2640 1 --2- 172.21.15.26:0/4064234264 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fcfa7715d0 0x55fcfa780110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:11.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.594+0000 7fee650f3640 1 --2- 172.21.15.26:0/4064234264 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fcfa780780 0x55fcfa77d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:11.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.594+0000 7fee5ffff640 1 -- 172.21.15.26:0/4064234264 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fcfa7715d0 msgr2=0x55fcfa780110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:19:11.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.594+0000 7fee5ffff640 1 --2- 172.21.15.26:0/4064234264 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fcfa7715d0 0x55fcfa780110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:11.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.594+0000 7fee5ffff640 1 -- 172.21.15.26:0/4064234264 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fcfa780780 msgr2=0x55fcfa77d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:19:11.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.594+0000 7fee5ffff640 1 --2- 172.21.15.26:0/4064234264 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fcfa780780 0x55fcfa77d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:11.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.594+0000 7fee5ffff640 1 -- 172.21.15.26:0/4064234264 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fcfa4483c0 con 0x55fcfa772b40 2024-09-11T23:19:11.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.594+0000 7fee5ffff640 1 --2- 172.21.15.26:0/4064234264 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fcfa772b40 0x55fcfa77d3c0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fee4c02a4c0 tx=0x7fee4c0092e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b84267e13e84722 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:19:11.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.595+0000 7fee3ffff640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:19:11.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.595+0000 7fee3ffff640 1 -- 172.21.15.26:0/4064234264 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee4c04c070 con 0x55fcfa772b40 2024-09-11T23:19:11.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.595+0000 7fee3ffff640 1 -- 172.21.15.26:0/4064234264 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fee4c004f40 con 0x55fcfa772b40 2024-09-11T23:19:11.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.595+0000 7fee68f2f740 1 -- 172.21.15.26:0/4064234264 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee4c005520 con 0x55fcfa772b40 2024-09-11T23:19:11.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.595+0000 7fee3ffff640 1 -- 172.21.15.26:0/4064234264 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fee4c058050 con 0x55fcfa772b40 2024-09-11T23:19:11.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.595+0000 7fee68f2f740 10 client.15633 fetch_fsmap learned FSMap version 29 2024-09-11T23:19:11.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.595+0000 7fee68f2f740 1 -- 172.21.15.26:0/4064234264 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55fcfa78ee50 con 0x55fcfa772b40 2024-09-11T23:19:11.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.595+0000 7fee3ffff640 1 -- 172.21.15.26:0/4064234264 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee4c005520 con 0x55fcfa772b40 2024-09-11T23:19:11.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.595+0000 7fee3ffff640 1 -- 172.21.15.26:0/4064234264 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fee4c031540 con 0x55fcfa772b40 2024-09-11T23:19:11.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.595+0000 7fee68f2f740 10 client.15633 fetch_fsmap finished waiting for FSMap version 29 2024-09-11T23:19:11.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.595+0000 7fee68f2f740 10 client.15633 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-11T23:19:11.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.595+0000 7fee68f2f740 20 client.15633 populate_metadata read hostname 'smithi026' 2024-09-11T23:19:11.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.595+0000 7fee68f2f740 1 --2- 172.21.15.26:0/4064234264 >> [v2:172.21.15.76:6836/2078978748,v1:172.21.15.76:6837/2078978748] conn(0x55fcfa7901a0 0x55fcfa792590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:11.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.595+0000 7fee68f2f740 4 client.15633 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:19:11.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.595+0000 7fee68f2f740 1 -- 172.21.15.26:0/4064234264 --> [v2:172.21.15.76:6836/2078978748,v1:172.21.15.76:6837/2078978748] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fcfa6e4860 con 0x55fcfa7901a0 2024-09-11T23:19:11.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.596+0000 7fee648f2640 1 --2- 172.21.15.26:0/4064234264 >> [v2:172.21.15.76:6836/2078978748,v1:172.21.15.76:6837/2078978748] conn(0x55fcfa7901a0 0x55fcfa792590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:11.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.596+0000 7fee648f2640 1 --2- 172.21.15.26:0/4064234264 >> [v2:172.21.15.76:6836/2078978748,v1:172.21.15.76:6837/2078978748] conn(0x55fcfa7901a0 0x55fcfa792590 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3d54b86d94de5c02 server_cookie=e5149d62c482a316 in_seq=0 out_seq=0 2024-09-11T23:19:11.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.596+0000 7fee3ffff640 10 client.15633 ms_handle_connect on v2:172.21.15.76:6836/2078978748 2024-09-11T23:19:11.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.597+0000 7fee3ffff640 1 -- 172.21.15.26:0/4064234264 <== mds.2 v2:172.21.15.76:6836/2078978748 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fcfa6e4860 con 0x55fcfa7901a0 2024-09-11T23:19:11.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.597+0000 7fee3ffff640 10 client.15633 handle_command_reply: tid=0 2024-09-11T23:19:11.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.597+0000 7fee68f2f740 1 client.15633 shutdown 2024-09-11T23:19:11.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.597+0000 7fee68f2f740 1 -- 172.21.15.26:0/4064234264 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fcfa772b40 msgr2=0x55fcfa77d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:11.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.597+0000 7fee68f2f740 1 --2- 172.21.15.26:0/4064234264 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fcfa772b40 0x55fcfa77d3c0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fee4c02a4c0 tx=0x7fee4c0092e0 comp rx=0 tx=0).stop 2024-09-11T23:19:11.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.597+0000 7fee68f2f740 1 -- 172.21.15.26:0/4064234264 shutdown_connections 2024-09-11T23:19:11.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.597+0000 7fee68f2f740 1 --2- 172.21.15.26:0/4064234264 >> [v2:172.21.15.76:6836/2078978748,v1:172.21.15.76:6837/2078978748] conn(0x55fcfa7901a0 0x55fcfa792590 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:11.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.597+0000 7fee68f2f740 1 --2- 172.21.15.26:0/4064234264 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fcfa780780 0x55fcfa77d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:11.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.597+0000 7fee68f2f740 1 --2- 172.21.15.26:0/4064234264 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fcfa772b40 0x55fcfa77d3c0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:11.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.597+0000 7fee68f2f740 1 --2- 172.21.15.26:0/4064234264 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fcfa7715d0 0x55fcfa780110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:11.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.597+0000 7fee3ffff640 0 client.15633 ms_handle_reset on v2:172.21.15.76:6836/2078978748 2024-09-11T23:19:11.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.597+0000 7fee68f2f740 1 -- 172.21.15.26:0/4064234264 >> 172.21.15.26:0/4064234264 conn(0x55fcfa73a310 msgr2=0x55fcfa773960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:19:11.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.597+0000 7fee68f2f740 1 -- 172.21.15.26:0/4064234264 shutdown_connections 2024-09-11T23:19:11.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.597+0000 7fee68f2f740 1 -- 172.21.15.26:0/4064234264 wait complete. 2024-09-11T23:19:11.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.597+0000 7fee68f2f740 20 client.15633 trim_cache size 0 max 16384 2024-09-11T23:19:11.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.610+0000 7fee68f2f740 1 Processor -- start 2024-09-11T23:19:11.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.610+0000 7fee68f2f740 1 -- start start 2024-09-11T23:19:11.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.611+0000 7fee68f2f740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fcfa7715d0 0x55fcfa76de10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:11.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.611+0000 7fee68f2f740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fcfa772b40 0x55fcfa78c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:11.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.611+0000 7fee68f2f740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fcfa780780 0x55fcfa797b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:11.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.611+0000 7fee68f2f740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fee601ab6b0 con 0x55fcfa7715d0 2024-09-11T23:19:11.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.611+0000 7fee68f2f740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fee601ad1f0 con 0x55fcfa772b40 2024-09-11T23:19:11.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.611+0000 7fee68f2f740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fee601abd80 con 0x55fcfa780780 2024-09-11T23:19:11.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.611+0000 7fee648f2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fcfa7715d0 0x55fcfa76de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:11.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.611+0000 7fee648f2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fcfa7715d0 0x55fcfa76de10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54696/0 (socket says 172.21.15.26:54696) 2024-09-11T23:19:11.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.611+0000 7fee648f2640 1 -- 172.21.15.26:0/3228510695 learned_addr learned my addr 172.21.15.26:0/3228510695 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:19:11.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.611+0000 7fee5ffff640 1 --2- 172.21.15.26:0/3228510695 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fcfa772b40 0x55fcfa78c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:11.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.611+0000 7fee650f3640 1 --2- 172.21.15.26:0/3228510695 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fcfa780780 0x55fcfa797b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:11.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.611+0000 7fee648f2640 1 -- 172.21.15.26:0/3228510695 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fcfa780780 msgr2=0x55fcfa797b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:19:11.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.611+0000 7fee648f2640 1 --2- 172.21.15.26:0/3228510695 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fcfa780780 0x55fcfa797b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:11.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.611+0000 7fee648f2640 1 -- 172.21.15.26:0/3228510695 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fcfa772b40 msgr2=0x55fcfa78c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:19:11.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.611+0000 7fee648f2640 1 --2- 172.21.15.26:0/3228510695 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fcfa772b40 0x55fcfa78c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:11.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.611+0000 7fee648f2640 1 -- 172.21.15.26:0/3228510695 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fcfa783590 con 0x55fcfa7715d0 2024-09-11T23:19:11.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.611+0000 7fee650f3640 1 --2- 172.21.15.26:0/3228510695 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fcfa780780 0x55fcfa797b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:19:11.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.611+0000 7fee648f2640 1 --2- 172.21.15.26:0/3228510695 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fcfa7715d0 0x55fcfa76de10 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fee5409b370 tx=0x7fee54000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93466dd89c93ecb1 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:19:11.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.611+0000 7fee4a7fc640 1 -- 172.21.15.26:0/3228510695 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee54020040 con 0x55fcfa7715d0 2024-09-11T23:19:11.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.611+0000 7fee4a7fc640 1 -- 172.21.15.26:0/3228510695 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee5409fa20 con 0x55fcfa7715d0 2024-09-11T23:19:11.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.611+0000 7fee4a7fc640 1 -- 172.21.15.26:0/3228510695 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee54084bb0 con 0x55fcfa7715d0 2024-09-11T23:19:11.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.612+0000 7fee68f2f740 1 -- 172.21.15.26:0/3228510695 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fcfa7715d0 msgr2=0x55fcfa76de10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:11.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.612+0000 7fee68f2f740 1 --2- 172.21.15.26:0/3228510695 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fcfa7715d0 0x55fcfa76de10 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fee5409b370 tx=0x7fee54000bc0 comp rx=0 tx=0).stop 2024-09-11T23:19:11.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.612+0000 7fee68f2f740 1 -- 172.21.15.26:0/3228510695 shutdown_connections 2024-09-11T23:19:11.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.612+0000 7fee68f2f740 1 --2- 172.21.15.26:0/3228510695 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fcfa772b40 0x55fcfa78c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:11.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.612+0000 7fee68f2f740 1 --2- 172.21.15.26:0/3228510695 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fcfa780780 0x55fcfa797b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:11.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.612+0000 7fee68f2f740 1 --2- 172.21.15.26:0/3228510695 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fcfa7715d0 0x55fcfa76de10 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:11.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.612+0000 7fee68f2f740 1 -- 172.21.15.26:0/3228510695 >> 172.21.15.26:0/3228510695 conn(0x55fcfa73a310 msgr2=0x55fcfa76d700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:19:11.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.612+0000 7fee68f2f740 1 -- 172.21.15.26:0/3228510695 shutdown_connections 2024-09-11T23:19:11.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.612+0000 7fee68f2f740 1 -- 172.21.15.26:0/3228510695 wait complete. 2024-09-11T23:19:11.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.613+0000 7fee68f2f740 1 -- start start 2024-09-11T23:19:11.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.613+0000 7fee68f2f740 1 Processor -- start 2024-09-11T23:19:11.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.613+0000 7fee68f2f740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fcfa7715d0 0x55fcfa788830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:11.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.613+0000 7fee68f2f740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fcfa772b40 0x55fcfa788d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:11.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.613+0000 7fee68f2f740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fcfa780780 0x55fcfa785800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:11.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.613+0000 7fee68f2f740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fee601acb20 con 0x55fcfa772b40 2024-09-11T23:19:11.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.613+0000 7fee68f2f740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fee601ac450 con 0x55fcfa7715d0 2024-09-11T23:19:11.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.613+0000 7fee68f2f740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fee601a9690 con 0x55fcfa780780 2024-09-11T23:19:11.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.613+0000 7fee68f2f740 10 client.0 _ll_register_callbacks cb 0x7fee58009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:19:11.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.613+0000 7fee5ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fcfa772b40 0x55fcfa788d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:11.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.613+0000 7fee5ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fcfa772b40 0x55fcfa788d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54706/0 (socket says 172.21.15.26:54706) 2024-09-11T23:19:11.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.613+0000 7fee5ffff640 1 -- 172.21.15.26:0/1323598109 learned_addr learned my addr 172.21.15.26:0/1323598109 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:19:11.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.614+0000 7fee650f3640 1 --2- 172.21.15.26:0/1323598109 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fcfa780780 0x55fcfa785800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:11.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.614+0000 7fee648f2640 1 --2- 172.21.15.26:0/1323598109 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fcfa7715d0 0x55fcfa788830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:11.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.614+0000 7fee5ffff640 1 -- 172.21.15.26:0/1323598109 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fcfa780780 msgr2=0x55fcfa785800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:19:11.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.614+0000 7fee5ffff640 1 --2- 172.21.15.26:0/1323598109 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fcfa780780 0x55fcfa785800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:11.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.614+0000 7fee5ffff640 1 -- 172.21.15.26:0/1323598109 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fcfa7715d0 msgr2=0x55fcfa788830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:19:11.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.614+0000 7fee5ffff640 1 --2- 172.21.15.26:0/1323598109 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fcfa7715d0 0x55fcfa788830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:11.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.614+0000 7fee5ffff640 1 -- 172.21.15.26:0/1323598109 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fcfa785d40 con 0x55fcfa772b40 2024-09-11T23:19:11.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.614+0000 7fee648f2640 1 --2- 172.21.15.26:0/1323598109 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fcfa7715d0 0x55fcfa788830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:19:11.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.614+0000 7fee5ffff640 1 --2- 172.21.15.26:0/1323598109 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fcfa772b40 0x55fcfa788d70 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fee4c000c00 tx=0x7fee4c004bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a55d7694a0d06ba server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:19:11.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.614+0000 7fee3ffff640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:19:11.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.614+0000 7fee3ffff640 1 -- 172.21.15.26:0/1323598109 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee4c04c040 con 0x55fcfa772b40 2024-09-11T23:19:11.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.614+0000 7fee3ffff640 1 -- 172.21.15.26:0/1323598109 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fee4c031d80 con 0x55fcfa772b40 2024-09-11T23:19:11.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.614+0000 7fee68f2f740 1 -- 172.21.15.26:0/1323598109 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee5409fa20 con 0x55fcfa772b40 2024-09-11T23:19:11.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.614+0000 7fee3ffff640 1 -- 172.21.15.26:0/1323598109 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fee4c03f050 con 0x55fcfa772b40 2024-09-11T23:19:11.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.614+0000 7fee68f2f740 10 client.15639 fetch_fsmap learned FSMap version 29 2024-09-11T23:19:11.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.614+0000 7fee68f2f740 1 -- 172.21.15.26:0/1323598109 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55fcfa7fb300 con 0x55fcfa772b40 2024-09-11T23:19:11.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.615+0000 7fee3ffff640 1 -- 172.21.15.26:0/1323598109 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee5409fa20 con 0x55fcfa772b40 2024-09-11T23:19:11.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.615+0000 7fee3ffff640 1 -- 172.21.15.26:0/1323598109 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fee4c0444e0 con 0x55fcfa772b40 2024-09-11T23:19:11.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.615+0000 7fee68f2f740 10 client.15639 fetch_fsmap finished waiting for FSMap version 29 2024-09-11T23:19:11.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.615+0000 7fee68f2f740 10 client.15639 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-11T23:19:11.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.615+0000 7fee68f2f740 20 client.15639 populate_metadata read hostname 'smithi026' 2024-09-11T23:19:11.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.615+0000 7fee68f2f740 1 --2- 172.21.15.26:0/1323598109 >> [v2:172.21.15.76:6836/2078978748,v1:172.21.15.76:6837/2078978748] conn(0x55fcfa7fb990 0x55fcfa7fddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:11.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.615+0000 7fee68f2f740 4 client.15639 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:19:11.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.615+0000 7fee68f2f740 1 -- 172.21.15.26:0/1323598109 --> [v2:172.21.15.76:6836/2078978748,v1:172.21.15.76:6837/2078978748] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fcfa6ded70 con 0x55fcfa7fb990 2024-09-11T23:19:11.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.615+0000 7fee648f2640 1 --2- 172.21.15.26:0/1323598109 >> [v2:172.21.15.76:6836/2078978748,v1:172.21.15.76:6837/2078978748] conn(0x55fcfa7fb990 0x55fcfa7fddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:11.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.615+0000 7fee648f2640 1 --2- 172.21.15.26:0/1323598109 >> [v2:172.21.15.76:6836/2078978748,v1:172.21.15.76:6837/2078978748] conn(0x55fcfa7fb990 0x55fcfa7fddb0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=200f17727d5cc122 server_cookie=3aab7c06b60a2e81 in_seq=0 out_seq=0 2024-09-11T23:19:11.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.615+0000 7fee3ffff640 10 client.15639 ms_handle_connect on v2:172.21.15.76:6836/2078978748 2024-09-11T23:19:11.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.616+0000 7fee3ffff640 1 -- 172.21.15.26:0/1323598109 <== mds.2 v2:172.21.15.76:6836/2078978748 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fcfa6ded70 con 0x55fcfa7fb990 2024-09-11T23:19:11.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.616+0000 7fee3ffff640 10 client.15639 handle_command_reply: tid=0 2024-09-11T23:19:11.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.616+0000 7fee68f2f740 1 client.15639 shutdown 2024-09-11T23:19:11.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.616+0000 7fee68f2f740 1 -- 172.21.15.26:0/1323598109 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fcfa772b40 msgr2=0x55fcfa788d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:11.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.616+0000 7fee68f2f740 1 --2- 172.21.15.26:0/1323598109 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fcfa772b40 0x55fcfa788d70 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fee4c000c00 tx=0x7fee4c004bc0 comp rx=0 tx=0).stop 2024-09-11T23:19:11.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.616+0000 7fee68f2f740 1 -- 172.21.15.26:0/1323598109 shutdown_connections 2024-09-11T23:19:11.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.616+0000 7fee68f2f740 1 --2- 172.21.15.26:0/1323598109 >> [v2:172.21.15.76:6836/2078978748,v1:172.21.15.76:6837/2078978748] conn(0x55fcfa7fb990 0x55fcfa7fddb0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:11.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.616+0000 7fee68f2f740 1 --2- 172.21.15.26:0/1323598109 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fcfa780780 0x55fcfa785800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:11.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.616+0000 7fee68f2f740 1 --2- 172.21.15.26:0/1323598109 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fcfa772b40 0x55fcfa788d70 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:11.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.616+0000 7fee68f2f740 1 --2- 172.21.15.26:0/1323598109 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fcfa7715d0 0x55fcfa788830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:11.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.616+0000 7fee3ffff640 0 client.15639 ms_handle_reset on v2:172.21.15.76:6836/2078978748 2024-09-11T23:19:11.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.616+0000 7fee68f2f740 1 -- 172.21.15.26:0/1323598109 >> 172.21.15.26:0/1323598109 conn(0x55fcfa73a310 msgr2=0x55fcfa77ec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:19:11.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.616+0000 7fee68f2f740 1 -- 172.21.15.26:0/1323598109 shutdown_connections 2024-09-11T23:19:11.641 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2024-09-11T23:19:11.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.616+0000 7fee68f2f740 1 -- 172.21.15.26:0/1323598109 wait complete. 2024-09-11T23:19:11.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.616+0000 7fee68f2f740 20 client.15639 trim_cache size 0 max 16384 2024-09-11T23:19:11.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.617+0000 7fee66b56640 1 -- 172.21.15.26:0/1316992569 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fee500a2e10 msgr2=0x7fee500a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:11.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.617+0000 7fee66b56640 1 --2- 172.21.15.26:0/1316992569 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fee500a2e10 0x7fee500a52d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fee58014820 tx=0x7fee58014c20 comp rx=0 tx=0).stop 2024-09-11T23:19:11.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.617+0000 7fee66b56640 1 -- 172.21.15.26:0/1316992569 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee60100e20 msgr2=0x7fee60198560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:11.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.617+0000 7fee66b56640 1 --2- 172.21.15.26:0/1316992569 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee60100e20 0x7fee60198560 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fee5400d950 tx=0x7fee5400de10 comp rx=0 tx=0).stop 2024-09-11T23:19:11.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.617+0000 7fee66b56640 1 -- 172.21.15.26:0/1316992569 shutdown_connections 2024-09-11T23:19:11.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.617+0000 7fee66b56640 1 --2- 172.21.15.26:0/1316992569 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fee500a2e10 0x7fee500a52d0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:11.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.617+0000 7fee66b56640 1 --2- 172.21.15.26:0/1316992569 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fee60102570 0x7fee60198aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:11.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.617+0000 7fee66b56640 1 --2- 172.21.15.26:0/1316992569 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fee60102ef0 0x7fee6019ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:11.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.617+0000 7fee66b56640 1 --2- 172.21.15.26:0/1316992569 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee60100e20 0x7fee60198560 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:11.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.617+0000 7fee66b56640 1 -- 172.21.15.26:0/1316992569 >> 172.21.15.26:0/1316992569 conn(0x7fee600fc7d0 msgr2=0x7fee6006a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:19:11.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.617+0000 7fee66b56640 1 -- 172.21.15.26:0/1316992569 shutdown_connections 2024-09-11T23:19:11.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:11.617+0000 7fee66b56640 1 -- 172.21.15.26:0/1316992569 wait complete. 2024-09-11T23:19:12.225 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:19:11 smithi026 ceph-mon[31172]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 25 op/s 2024-09-11T23:19:12.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:19:11 smithi136 ceph-mon[35121]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 25 op/s 2024-09-11T23:19:12.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:19:11 smithi076 ceph-mon[35296]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 25 op/s 2024-09-11T23:19:14.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:19:13 smithi136 ceph-mon[35121]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 25 op/s 2024-09-11T23:19:14.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:19:13 smithi026 ceph-mon[31172]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 25 op/s 2024-09-11T23:19:14.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:19:13 smithi076 ceph-mon[35296]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 25 op/s 2024-09-11T23:19:14.482 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-11T23:19:14.485 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:19:16.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:19:15 smithi136 ceph-mon[35121]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 25 KiB/s wr, 31 op/s 2024-09-11T23:19:16.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:19:15 smithi026 ceph-mon[31172]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 25 KiB/s wr, 31 op/s 2024-09-11T23:19:16.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:19:15 smithi076 ceph-mon[35296]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 25 KiB/s wr, 31 op/s 2024-09-11T23:19:17.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.043+0000 7f50864cd640 1 -- 172.21.15.26:0/297387473 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5080108290 msgr2=0x7f508010ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:17.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.043+0000 7f50864cd640 1 --2- 172.21.15.26:0/297387473 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5080108290 0x7f508010ec30 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f5078009cd0 tx=0x7f507802a8e0 comp rx=0 tx=0).stop 2024-09-11T23:19:17.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.043+0000 7f50864cd640 1 -- 172.21.15.26:0/297387473 shutdown_connections 2024-09-11T23:19:17.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.043+0000 7f50864cd640 1 --2- 172.21.15.26:0/297387473 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5080103740 0x7f5080107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:17.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.043+0000 7f50864cd640 1 --2- 172.21.15.26:0/297387473 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5080108290 0x7f508010ec30 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:17.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.043+0000 7f50864cd640 1 --2- 172.21.15.26:0/297387473 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5080102d90 0x7f5080103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:17.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.043+0000 7f50864cd640 1 -- 172.21.15.26:0/297387473 >> 172.21.15.26:0/297387473 conn(0x7f50800fc7d0 msgr2=0x7f50800febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:19:17.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.043+0000 7f50864cd640 1 -- 172.21.15.26:0/297387473 shutdown_connections 2024-09-11T23:19:17.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.043+0000 7f50864cd640 1 -- 172.21.15.26:0/297387473 wait complete. 2024-09-11T23:19:17.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.044+0000 7f50864cd640 1 Processor -- start 2024-09-11T23:19:17.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.044+0000 7f50864cd640 1 -- start start 2024-09-11T23:19:17.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.044+0000 7f50864cd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5080102d90 0x7f50801009f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:17.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.044+0000 7f50864cd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5080103740 0x7f5080100f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:17.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.044+0000 7f50864cd640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5080108290 0x7f5080101470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:17.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.044+0000 7f50864cd640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f5080112090 con 0x7f5080102d90 2024-09-11T23:19:17.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.044+0000 7f50864cd640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f5080111f10 con 0x7f5080103740 2024-09-11T23:19:17.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.044+0000 7f50864cd640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f5080112210 con 0x7f5080108290 2024-09-11T23:19:17.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.045+0000 7f5084a6a640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5080108290 0x7f5080101470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:17.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.045+0000 7f5084a6a640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5080108290 0x7f5080101470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:45784/0 (socket says 172.21.15.26:45784) 2024-09-11T23:19:17.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.045+0000 7f5084a6a640 1 -- 172.21.15.26:0/1234599386 learned_addr learned my addr 172.21.15.26:0/1234599386 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:19:17.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.045+0000 7f507ffff640 1 --2- 172.21.15.26:0/1234599386 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5080102d90 0x7f50801009f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:17.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.045+0000 7f5084a6a640 1 -- 172.21.15.26:0/1234599386 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5080103740 msgr2=0x7f5080100f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:17.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.045+0000 7f5084a6a640 1 --2- 172.21.15.26:0/1234599386 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5080103740 0x7f5080100f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:17.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.045+0000 7f5084a6a640 1 -- 172.21.15.26:0/1234599386 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5080102d90 msgr2=0x7f50801009f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:17.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.046+0000 7f5084a6a640 1 --2- 172.21.15.26:0/1234599386 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5080102d90 0x7f50801009f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:17.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.046+0000 7f5084a6a640 1 -- 172.21.15.26:0/1234599386 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50801a6a00 con 0x7f5080108290 2024-09-11T23:19:17.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.046+0000 7f507ffff640 1 --2- 172.21.15.26:0/1234599386 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5080102d90 0x7f50801009f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:19:17.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.046+0000 7f5084a6a640 1 --2- 172.21.15.26:0/1234599386 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5080108290 0x7f5080101470 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f5078005cc0 tx=0x7f5078003970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:19:17.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.046+0000 7f507d7fa640 1 -- 172.21.15.26:0/1234599386 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5078004040 con 0x7f5080108290 2024-09-11T23:19:17.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.046+0000 7f50864cd640 1 -- 172.21.15.26:0/1234599386 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50801a6c90 con 0x7f5080108290 2024-09-11T23:19:17.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.047+0000 7f50864cd640 1 -- 172.21.15.26:0/1234599386 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50801a7220 con 0x7f5080108290 2024-09-11T23:19:17.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.047+0000 7f507d7fa640 1 -- 172.21.15.26:0/1234599386 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f50780046c0 con 0x7f5080108290 2024-09-11T23:19:17.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.047+0000 7f507d7fa640 1 -- 172.21.15.26:0/1234599386 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5078046480 con 0x7f5080108290 2024-09-11T23:19:17.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.049+0000 7f50888a6740 1 Processor -- start 2024-09-11T23:19:17.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.050+0000 7f507d7fa640 1 -- 172.21.15.26:0/1234599386 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 25) ==== 138066+0+0 (secure 0 0 0) 0x7f50780041e0 con 0x7f5080108290 2024-09-11T23:19:17.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.050+0000 7f50888a6740 1 -- start start 2024-09-11T23:19:17.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.050+0000 7f50888a6740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d41a2e1f70 0x55d41a2e2350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:17.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.050+0000 7f50888a6740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d41a2e2890 0x55d41a2ecc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:17.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.050+0000 7f50888a6740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d41a2ed150 0x55d41a2ef540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:17.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.050+0000 7f50888a6740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55d41a23f2a0 con 0x55d41a2e1f70 2024-09-11T23:19:17.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.050+0000 7f50888a6740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55d41a1e6b10 con 0x55d41a2ed150 2024-09-11T23:19:17.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.050+0000 7f50888a6740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55d41a23b260 con 0x55d41a2e2890 2024-09-11T23:19:17.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.050+0000 7f507f7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d41a2e1f70 0x55d41a2e2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:17.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.051+0000 7f507f7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d41a2e1f70 0x55d41a2e2350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54720/0 (socket says 172.21.15.26:54720) 2024-09-11T23:19:17.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.051+0000 7f507f7fe640 1 -- 172.21.15.26:0/2327506312 learned_addr learned my addr 172.21.15.26:0/2327506312 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:19:17.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.051+0000 7f507d7fa640 1 --2- 172.21.15.26:0/1234599386 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f50680a2e90 0x7f50680a5350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:17.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.051+0000 7f507ffff640 1 --2- 172.21.15.26:0/2327506312 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d41a2e2890 0x55d41a2ecc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:17.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.051+0000 7f507f7fe640 1 -- 172.21.15.26:0/2327506312 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d41a2e2890 msgr2=0x55d41a2ecc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:19:17.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.051+0000 7f507f7fe640 1 --2- 172.21.15.26:0/2327506312 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d41a2e2890 0x55d41a2ecc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:17.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.051+0000 7f5084a6a640 1 --2- 172.21.15.26:0/1234599386 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f50680a2e90 0x7f50680a5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:17.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.051+0000 7f507f7fe640 1 -- 172.21.15.26:0/2327506312 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d41a2ed150 msgr2=0x55d41a2ef540 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T23:19:17.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.051+0000 7f507f7fe640 1 --2- 172.21.15.26:0/2327506312 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d41a2ed150 0x55d41a2ef540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:17.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.051+0000 7f507f7fe640 1 -- 172.21.15.26:0/2327506312 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d419fb0b90 con 0x55d41a2e1f70 2024-09-11T23:19:17.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.051+0000 7f507d7fa640 1 -- 172.21.15.26:0/1234599386 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f507802e030 con 0x7f5080108290 2024-09-11T23:19:17.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.051+0000 7f507f7fe640 1 --2- 172.21.15.26:0/2327506312 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d41a2e1f70 0x55d41a2e2350 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f507400b9c0 tx=0x7f507400bdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=265b41fe4926b810 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:19:17.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.052+0000 7f50667fc640 1 -- 172.21.15.26:0/2327506312 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5074016070 con 0x55d41a2e1f70 2024-09-11T23:19:17.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.052+0000 7f50667fc640 1 -- 172.21.15.26:0/2327506312 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5074005500 con 0x55d41a2e1f70 2024-09-11T23:19:17.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.052+0000 7f50667fc640 1 -- 172.21.15.26:0/2327506312 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5074004de0 con 0x55d41a2e1f70 2024-09-11T23:19:17.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.052+0000 7f5084a6a640 1 --2- 172.21.15.26:0/1234599386 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f50680a2e90 0x7f50680a5350 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f50780ee4f0 tx=0x7f507803d040 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:19:17.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.052+0000 7f50888a6740 1 -- 172.21.15.26:0/2327506312 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d41a2e1f70 msgr2=0x55d41a2e2350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:17.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.052+0000 7f50888a6740 1 --2- 172.21.15.26:0/2327506312 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d41a2e1f70 0x55d41a2e2350 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f507400b9c0 tx=0x7f507400bdc0 comp rx=0 tx=0).stop 2024-09-11T23:19:17.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.053+0000 7f50888a6740 1 -- 172.21.15.26:0/2327506312 shutdown_connections 2024-09-11T23:19:17.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.053+0000 7f50888a6740 1 --2- 172.21.15.26:0/2327506312 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d41a2ed150 0x55d41a2ef540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:17.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.053+0000 7f50888a6740 1 --2- 172.21.15.26:0/2327506312 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d41a2e2890 0x55d41a2ecc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:17.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.053+0000 7f50888a6740 1 --2- 172.21.15.26:0/2327506312 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d41a2e1f70 0x55d41a2e2350 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:17.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.053+0000 7f50888a6740 1 -- 172.21.15.26:0/2327506312 >> 172.21.15.26:0/2327506312 conn(0x55d41a2de500 msgr2=0x55d41a2de8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:19:17.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.053+0000 7f50888a6740 1 -- 172.21.15.26:0/2327506312 shutdown_connections 2024-09-11T23:19:17.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.053+0000 7f50888a6740 1 -- 172.21.15.26:0/2327506312 wait complete. 2024-09-11T23:19:17.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.053+0000 7f50888a6740 1 -- start start 2024-09-11T23:19:17.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.053+0000 7f50888a6740 1 Processor -- start 2024-09-11T23:19:17.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.054+0000 7f50888a6740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d41a2e1f70 0x55d41a2ea0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:17.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.054+0000 7f50888a6740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d41a2e2890 0x55d41a2ea600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:17.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.054+0000 7f50888a6740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d41a2ed150 0x55d41a2eab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:17.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.054+0000 7f50888a6740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55d41a2efd20 con 0x55d41a2e2890 2024-09-11T23:19:17.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.054+0000 7f50888a6740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55d41a1c76e0 con 0x55d41a2e1f70 2024-09-11T23:19:17.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.054+0000 7f50888a6740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55d41a2efea0 con 0x55d41a2ed150 2024-09-11T23:19:17.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.054+0000 7f50888a6740 10 client.0 _ll_register_callbacks cb 0x55d41a1eba80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:19:17.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.054+0000 7f507ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d41a2e2890 0x55d41a2ea600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:17.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.054+0000 7f507ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d41a2e2890 0x55d41a2ea600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54732/0 (socket says 172.21.15.26:54732) 2024-09-11T23:19:17.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.054+0000 7f507ffff640 1 -- 172.21.15.26:0/3102867091 learned_addr learned my addr 172.21.15.26:0/3102867091 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:19:17.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.054+0000 7f507f7fe640 1 --2- 172.21.15.26:0/3102867091 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d41a2e1f70 0x55d41a2ea0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:17.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.054+0000 7f507ffff640 1 -- 172.21.15.26:0/3102867091 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d41a2ed150 msgr2=0x55d41a2eab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:19:17.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.054+0000 7f507ffff640 1 --2- 172.21.15.26:0/3102867091 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d41a2ed150 0x55d41a2eab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:17.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.054+0000 7f507ffff640 1 -- 172.21.15.26:0/3102867091 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d41a2e1f70 msgr2=0x55d41a2ea0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:19:17.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.054+0000 7f507ffff640 1 --2- 172.21.15.26:0/3102867091 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d41a2e1f70 0x55d41a2ea0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:17.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.054+0000 7f507ffff640 1 -- 172.21.15.26:0/3102867091 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d41a2f92c0 con 0x55d41a2e2890 2024-09-11T23:19:17.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.055+0000 7f507ffff640 1 --2- 172.21.15.26:0/3102867091 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d41a2e2890 0x55d41a2ea600 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f5070004490 tx=0x7f50700039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e05fce79e27eea4 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:19:17.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.055+0000 7f505f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:19:17.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.055+0000 7f505f7fe640 1 -- 172.21.15.26:0/3102867091 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5070011770 con 0x55d41a2e2890 2024-09-11T23:19:17.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.055+0000 7f50888a6740 1 -- 172.21.15.26:0/3102867091 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5074005500 con 0x55d41a2e2890 2024-09-11T23:19:17.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.055+0000 7f505f7fe640 1 -- 172.21.15.26:0/3102867091 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f5070011910 con 0x55d41a2e2890 2024-09-11T23:19:17.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.056+0000 7f505f7fe640 1 -- 172.21.15.26:0/3102867091 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f5070011b10 con 0x55d41a2e2890 2024-09-11T23:19:17.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.056+0000 7f50888a6740 10 client.15651 fetch_fsmap learned FSMap version 29 2024-09-11T23:19:17.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.056+0000 7f50888a6740 1 -- 172.21.15.26:0/3102867091 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d41a2fbd00 con 0x55d41a2e2890 2024-09-11T23:19:17.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.056+0000 7f505f7fe640 1 -- 172.21.15.26:0/3102867091 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5074005500 con 0x55d41a2e2890 2024-09-11T23:19:17.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.056+0000 7f505f7fe640 1 -- 172.21.15.26:0/3102867091 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f5070010440 con 0x55d41a2e2890 2024-09-11T23:19:17.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.056+0000 7f50888a6740 10 client.15651 fetch_fsmap finished waiting for FSMap version 29 2024-09-11T23:19:17.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.056+0000 7f50888a6740 10 client.15651 resolve_mds: validated gid 24485 aka daemon mds.b 2024-09-11T23:19:17.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.057+0000 7f50888a6740 20 client.15651 populate_metadata read hostname 'smithi026' 2024-09-11T23:19:17.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.057+0000 7f50888a6740 1 --2- 172.21.15.26:0/3102867091 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55d41a2fd600 0x55d41a2ffa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:17.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.057+0000 7f50888a6740 4 client.15651 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:19:17.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.057+0000 7f50888a6740 1 -- 172.21.15.26:0/3102867091 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d41a24c9d0 con 0x55d41a2fd600 2024-09-11T23:19:17.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.057+0000 7f507f7fe640 1 --2- 172.21.15.26:0/3102867091 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55d41a2fd600 0x55d41a2ffa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:17.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.058+0000 7f507f7fe640 1 --2- 172.21.15.26:0/3102867091 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55d41a2fd600 0x55d41a2ffa80 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fdf6c0e2ae0bc142 server_cookie=523da7ed52c2afd in_seq=0 out_seq=0 2024-09-11T23:19:17.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.058+0000 7f505f7fe640 10 client.15651 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:19:17.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.058+0000 7f505f7fe640 1 -- 172.21.15.26:0/3102867091 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d41a24c9d0 con 0x55d41a2fd600 2024-09-11T23:19:17.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.058+0000 7f505f7fe640 10 client.15651 handle_command_reply: tid=0 2024-09-11T23:19:17.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.058+0000 7f50888a6740 1 client.15651 shutdown 2024-09-11T23:19:17.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.059+0000 7f50888a6740 1 -- 172.21.15.26:0/3102867091 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d41a2e2890 msgr2=0x55d41a2ea600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:17.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.059+0000 7f50888a6740 1 --2- 172.21.15.26:0/3102867091 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d41a2e2890 0x55d41a2ea600 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f5070004490 tx=0x7f50700039c0 comp rx=0 tx=0).stop 2024-09-11T23:19:17.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.059+0000 7f50888a6740 1 -- 172.21.15.26:0/3102867091 shutdown_connections 2024-09-11T23:19:17.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.059+0000 7f50888a6740 1 --2- 172.21.15.26:0/3102867091 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55d41a2fd600 0x55d41a2ffa80 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:17.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.059+0000 7f505f7fe640 0 client.15651 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:19:17.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.059+0000 7f50888a6740 1 --2- 172.21.15.26:0/3102867091 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d41a2ed150 0x55d41a2eab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:17.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.059+0000 7f50888a6740 1 --2- 172.21.15.26:0/3102867091 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d41a2e2890 0x55d41a2ea600 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:17.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.060+0000 7f50888a6740 1 --2- 172.21.15.26:0/3102867091 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d41a2e1f70 0x55d41a2ea0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:17.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.060+0000 7f50888a6740 1 -- 172.21.15.26:0/3102867091 >> 172.21.15.26:0/3102867091 conn(0x55d41a2de500 msgr2=0x55d41a2e4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:19:17.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.060+0000 7f50888a6740 1 -- 172.21.15.26:0/3102867091 shutdown_connections 2024-09-11T23:19:17.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.060+0000 7f50888a6740 1 -- 172.21.15.26:0/3102867091 wait complete. 2024-09-11T23:19:17.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.060+0000 7f50888a6740 20 client.15651 trim_cache size 0 max 16384 2024-09-11T23:19:17.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.083+0000 7f50888a6740 1 Processor -- start 2024-09-11T23:19:17.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.084+0000 7f50888a6740 1 -- start start 2024-09-11T23:19:17.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.084+0000 7f50888a6740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d41a2e1f70 0x55d41a2d4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:17.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.084+0000 7f50888a6740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d41a2e2890 0x55d41a2e89b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:17.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.084+0000 7f50888a6740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d41a2ed150 0x55d41a2ea630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:17.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.084+0000 7f50888a6740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f50801ab500 con 0x55d41a2e2890 2024-09-11T23:19:17.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.085+0000 7f50888a6740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f50801ad040 con 0x55d41a2ed150 2024-09-11T23:19:17.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.085+0000 7f50888a6740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f50801abbd0 con 0x55d41a2e1f70 2024-09-11T23:19:17.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.085+0000 7f507ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d41a2e2890 0x55d41a2e89b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:17.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.085+0000 7f507ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d41a2e2890 0x55d41a2e89b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54744/0 (socket says 172.21.15.26:54744) 2024-09-11T23:19:17.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.085+0000 7f507ffff640 1 -- 172.21.15.26:0/2149629418 learned_addr learned my addr 172.21.15.26:0/2149629418 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:19:17.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.085+0000 7f507f7fe640 1 --2- 172.21.15.26:0/2149629418 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d41a2e1f70 0x55d41a2d4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:17.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.085+0000 7f507f7fe640 1 --2- 172.21.15.26:0/2149629418 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d41a2ed150 0x55d41a2ea630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:17.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.085+0000 7f507ffff640 1 -- 172.21.15.26:0/2149629418 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d41a2e1f70 msgr2=0x55d41a2d4cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:19:17.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.085+0000 7f507ffff640 1 --2- 172.21.15.26:0/2149629418 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d41a2e1f70 0x55d41a2d4cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:17.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.085+0000 7f507ffff640 1 -- 172.21.15.26:0/2149629418 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d41a2ed150 msgr2=0x55d41a2ea630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:19:17.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.085+0000 7f507ffff640 1 --2- 172.21.15.26:0/2149629418 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d41a2ed150 0x55d41a2ea630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:17.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.085+0000 7f507ffff640 1 -- 172.21.15.26:0/2149629418 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d41a2d1260 con 0x55d41a2e2890 2024-09-11T23:19:17.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.085+0000 7f507ffff640 1 --2- 172.21.15.26:0/2149629418 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d41a2e2890 0x55d41a2e89b0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f507000f3e0 tx=0x7f5070001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0765be0133cf0f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:19:17.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.086+0000 7f5065ffb640 1 -- 172.21.15.26:0/2149629418 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f507000ea70 con 0x55d41a2e2890 2024-09-11T23:19:17.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.086+0000 7f5065ffb640 1 -- 172.21.15.26:0/2149629418 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f50700223a0 con 0x55d41a2e2890 2024-09-11T23:19:17.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.086+0000 7f5065ffb640 1 -- 172.21.15.26:0/2149629418 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f507000ea70 con 0x55d41a2e2890 2024-09-11T23:19:17.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.087+0000 7f50888a6740 1 -- 172.21.15.26:0/2149629418 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d41a2e2890 msgr2=0x55d41a2e89b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:17.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.087+0000 7f50888a6740 1 --2- 172.21.15.26:0/2149629418 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d41a2e2890 0x55d41a2e89b0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f507000f3e0 tx=0x7f5070001e50 comp rx=0 tx=0).stop 2024-09-11T23:19:17.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.087+0000 7f50888a6740 1 -- 172.21.15.26:0/2149629418 shutdown_connections 2024-09-11T23:19:17.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.087+0000 7f50888a6740 1 --2- 172.21.15.26:0/2149629418 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d41a2ed150 0x55d41a2ea630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:17.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.087+0000 7f50888a6740 1 --2- 172.21.15.26:0/2149629418 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d41a2e2890 0x55d41a2e89b0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:17.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.087+0000 7f50888a6740 1 --2- 172.21.15.26:0/2149629418 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d41a2e1f70 0x55d41a2d4cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:17.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.087+0000 7f50888a6740 1 -- 172.21.15.26:0/2149629418 >> 172.21.15.26:0/2149629418 conn(0x55d41a2de500 msgr2=0x55d41a2ec140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:19:17.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.087+0000 7f50888a6740 1 -- 172.21.15.26:0/2149629418 shutdown_connections 2024-09-11T23:19:17.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.088+0000 7f50888a6740 1 -- 172.21.15.26:0/2149629418 wait complete. 2024-09-11T23:19:17.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.088+0000 7f50888a6740 1 -- start start 2024-09-11T23:19:17.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.089+0000 7f50888a6740 1 Processor -- start 2024-09-11T23:19:17.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.089+0000 7f50888a6740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d41a2e1f70 0x55d41a3030f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:17.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.089+0000 7f50888a6740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d41a2e2890 0x55d41a303630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:17.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.089+0000 7f50888a6740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d41a2ed150 0x55d41a303b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:17.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.089+0000 7f50888a6740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f50801ac970 con 0x55d41a2ed150 2024-09-11T23:19:17.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.089+0000 7f50888a6740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f50801ac2a0 con 0x55d41a2e1f70 2024-09-11T23:19:17.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.089+0000 7f50888a6740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f50801a94e0 con 0x55d41a2e2890 2024-09-11T23:19:17.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.089+0000 7f50888a6740 10 client.0 _ll_register_callbacks cb 0x7f50700040d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:19:17.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.089+0000 7f507f7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d41a2e1f70 0x55d41a3030f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:17.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.089+0000 7f507ffff640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d41a2e2890 0x55d41a303630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:17.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.089+0000 7f507ffff640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d41a2e2890 0x55d41a303630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:45832/0 (socket says 172.21.15.26:45832) 2024-09-11T23:19:17.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.090+0000 7f507ffff640 1 -- 172.21.15.26:0/1403878626 learned_addr learned my addr 172.21.15.26:0/1403878626 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:19:17.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.090+0000 7f507f7fe640 1 --2- 172.21.15.26:0/1403878626 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d41a2ed150 0x55d41a303b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:17.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.090+0000 7f507ffff640 1 -- 172.21.15.26:0/1403878626 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d41a2e1f70 msgr2=0x55d41a3030f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:19:17.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.090+0000 7f507ffff640 1 --2- 172.21.15.26:0/1403878626 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d41a2e1f70 0x55d41a3030f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:17.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.090+0000 7f507ffff640 1 -- 172.21.15.26:0/1403878626 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d41a2ed150 msgr2=0x55d41a303b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:19:17.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.090+0000 7f507ffff640 1 --2- 172.21.15.26:0/1403878626 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d41a2ed150 0x55d41a303b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:17.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.090+0000 7f507ffff640 1 -- 172.21.15.26:0/1403878626 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d41a3673e0 con 0x55d41a2e2890 2024-09-11T23:19:17.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.090+0000 7f507ffff640 1 --2- 172.21.15.26:0/1403878626 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d41a2e2890 0x55d41a303630 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f507000dbe0 tx=0x7f50700195a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ec095a8f840f603b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:19:17.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.090+0000 7f505f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-09-11T23:19:17.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.091+0000 7f505f7fe640 1 -- 172.21.15.26:0/1403878626 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f507000ea70 con 0x55d41a2e2890 2024-09-11T23:19:17.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.091+0000 7f505f7fe640 1 -- 172.21.15.26:0/1403878626 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f5070017040 con 0x55d41a2e2890 2024-09-11T23:19:17.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.091+0000 7f50888a6740 1 -- 172.21.15.26:0/1403878626 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f50700223a0 con 0x55d41a2e2890 2024-09-11T23:19:17.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.091+0000 7f505f7fe640 1 -- 172.21.15.26:0/1403878626 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f5070022e70 con 0x55d41a2e2890 2024-09-11T23:19:17.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.091+0000 7f50888a6740 10 client.25291 fetch_fsmap learned FSMap version 29 2024-09-11T23:19:17.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.091+0000 7f50888a6740 1 -- 172.21.15.26:0/1403878626 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d41a3686a0 con 0x55d41a2e2890 2024-09-11T23:19:17.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.092+0000 7f505f7fe640 1 -- 172.21.15.26:0/1403878626 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f50700223a0 con 0x55d41a2e2890 2024-09-11T23:19:17.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.093+0000 7f505f7fe640 1 -- 172.21.15.26:0/1403878626 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f507002a070 con 0x55d41a2e2890 2024-09-11T23:19:17.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.093+0000 7f50888a6740 10 client.25291 fetch_fsmap finished waiting for FSMap version 29 2024-09-11T23:19:17.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.093+0000 7f50888a6740 10 client.25291 resolve_mds: validated gid 24485 aka daemon mds.b 2024-09-11T23:19:17.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.093+0000 7f50888a6740 20 client.25291 populate_metadata read hostname 'smithi026' 2024-09-11T23:19:17.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.093+0000 7f50888a6740 1 --2- 172.21.15.26:0/1403878626 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55d41a2fd600 0x55d41a36ad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:17.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.093+0000 7f50888a6740 4 client.25291 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 144.5, "await_for": 45.0}] 2024-09-11T23:19:17.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.093+0000 7f50888a6740 1 -- 172.21.15.26:0/1403878626 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 144.5, "await_for": 45.0}) -- 0x55d41a246d20 con 0x55d41a2fd600 2024-09-11T23:19:17.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.094+0000 7f507f7fe640 1 --2- 172.21.15.26:0/1403878626 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55d41a2fd600 0x55d41a36ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:17.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.094+0000 7f505f7fe640 10 client.25291 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:19:17.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.094+0000 7f507f7fe640 1 --2- 172.21.15.26:0/1403878626 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55d41a2fd600 0x55d41a36ad50 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=42ece10f0fa50695 server_cookie=46dccdb6f9196b2f in_seq=0 out_seq=0 2024-09-11T23:19:17.100 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:19:17.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.102+0000 7f505f7fe640 1 -- 172.21.15.26:0/1403878626 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x55d41a246d20 con 0x55d41a2fd600 2024-09-11T23:19:17.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.102+0000 7f505f7fe640 10 client.25291 handle_command_reply: tid=0 2024-09-11T23:19:17.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.103+0000 7f50888a6740 1 client.25291 shutdown 2024-09-11T23:19:17.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.103+0000 7f50888a6740 1 -- 172.21.15.26:0/1403878626 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d41a2e2890 msgr2=0x55d41a303630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:17.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.103+0000 7f50888a6740 1 --2- 172.21.15.26:0/1403878626 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d41a2e2890 0x55d41a303630 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f507000dbe0 tx=0x7f50700195a0 comp rx=0 tx=0).stop 2024-09-11T23:19:17.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.103+0000 7f50888a6740 1 -- 172.21.15.26:0/1403878626 shutdown_connections 2024-09-11T23:19:17.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.103+0000 7f50888a6740 1 --2- 172.21.15.26:0/1403878626 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55d41a2fd600 0x55d41a36ad50 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:17.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.103+0000 7f50888a6740 1 --2- 172.21.15.26:0/1403878626 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d41a2ed150 0x55d41a303b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:17.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.103+0000 7f50888a6740 1 --2- 172.21.15.26:0/1403878626 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d41a2e2890 0x55d41a303630 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:17.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.103+0000 7f50888a6740 1 --2- 172.21.15.26:0/1403878626 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d41a2e1f70 0x55d41a3030f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:17.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.103+0000 7f50888a6740 1 -- 172.21.15.26:0/1403878626 >> 172.21.15.26:0/1403878626 conn(0x55d41a2de500 msgr2=0x55d41a2eb920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:19:17.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.103+0000 7f505f7fe640 0 client.25291 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:19:17.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.103+0000 7f50888a6740 1 -- 172.21.15.26:0/1403878626 shutdown_connections 2024-09-11T23:19:17.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.103+0000 7f50888a6740 1 -- 172.21.15.26:0/1403878626 wait complete. 2024-09-11T23:19:17.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.103+0000 7f50888a6740 20 client.25291 trim_cache size 0 max 16384 2024-09-11T23:19:17.106 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-11T23:19:17.106 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 29, 2024-09-11T23:19:17.106 INFO:teuthology.orchestra.run.smithi026.stdout: "leader": 24485, 2024-09-11T23:19:17.106 INFO:teuthology.orchestra.run.smithi026.stdout: "set_version": 2, 2024-09-11T23:19:17.106 INFO:teuthology.orchestra.run.smithi026.stdout: "sets": { 2024-09-11T23:19:17.106 INFO:teuthology.orchestra.run.smithi026.stdout: "37f40e7f": { 2024-09-11T23:19:17.107 INFO:teuthology.orchestra.run.smithi026.stdout: "version": 2, 2024-09-11T23:19:17.107 INFO:teuthology.orchestra.run.smithi026.stdout: "age_ref": 0.0, 2024-09-11T23:19:17.107 INFO:teuthology.orchestra.run.smithi026.stdout: "state": { 2024-09-11T23:19:17.107 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "QUIESCED", 2024-09-11T23:19:17.107 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.0 2024-09-11T23:19:17.107 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:19:17.107 INFO:teuthology.orchestra.run.smithi026.stdout: "timeout": 90.0, 2024-09-11T23:19:17.107 INFO:teuthology.orchestra.run.smithi026.stdout: "expiration": 144.5, 2024-09-11T23:19:17.107 INFO:teuthology.orchestra.run.smithi026.stdout: "members": { 2024-09-11T23:19:17.107 INFO:teuthology.orchestra.run.smithi026.stdout: "file:/": { 2024-09-11T23:19:17.107 INFO:teuthology.orchestra.run.smithi026.stdout: "excluded": false, 2024-09-11T23:19:17.107 INFO:teuthology.orchestra.run.smithi026.stdout: "state": { 2024-09-11T23:19:17.107 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "QUIESCED", 2024-09-11T23:19:17.108 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.0 2024-09-11T23:19:17.108 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:19:17.108 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:19:17.108 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:19:17.108 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:19:17.108 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:19:17.108 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-11T23:19:17.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.104+0000 7f50864cd640 1 -- 172.21.15.26:0/1234599386 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f50680a2e90 msgr2=0x7f50680a5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:17.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.104+0000 7f50864cd640 1 --2- 172.21.15.26:0/1234599386 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f50680a2e90 0x7f50680a5350 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f50780ee4f0 tx=0x7f507803d040 comp rx=0 tx=0).stop 2024-09-11T23:19:17.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.104+0000 7f50864cd640 1 -- 172.21.15.26:0/1234599386 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5080108290 msgr2=0x7f5080101470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:17.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.104+0000 7f50864cd640 1 --2- 172.21.15.26:0/1234599386 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5080108290 0x7f5080101470 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f5078005cc0 tx=0x7f5078003970 comp rx=0 tx=0).stop 2024-09-11T23:19:17.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.105+0000 7f50864cd640 1 -- 172.21.15.26:0/1234599386 shutdown_connections 2024-09-11T23:19:17.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.105+0000 7f50864cd640 1 --2- 172.21.15.26:0/1234599386 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f50680a2e90 0x7f50680a5350 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:17.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.105+0000 7f50864cd640 1 --2- 172.21.15.26:0/1234599386 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5080103740 0x7f5080100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:17.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.105+0000 7f50864cd640 1 --2- 172.21.15.26:0/1234599386 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5080108290 0x7f5080101470 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:17.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.105+0000 7f50864cd640 1 --2- 172.21.15.26:0/1234599386 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5080102d90 0x7f50801009f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:17.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.105+0000 7f50864cd640 1 -- 172.21.15.26:0/1234599386 >> 172.21.15.26:0/1234599386 conn(0x7f50800fc7d0 msgr2=0x7f5080106350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:19:17.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.105+0000 7f50864cd640 1 -- 172.21.15.26:0/1234599386 shutdown_connections 2024-09-11T23:19:17.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:17.105+0000 7f50864cd640 1 -- 172.21.15.26:0/1234599386 wait complete. 2024-09-11T23:19:18.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:19:17 smithi136 ceph-mon[35121]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-09-11T23:19:18.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:19:17 smithi026 ceph-mon[31172]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-09-11T23:19:18.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:19:17 smithi076 ceph-mon[35296]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-09-11T23:19:18.328 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '37f40e7f', quiesce took 10.4 seconds. Will release after: 14.1 2024-09-11T23:19:18.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.664+0000 7f47a3464640 1 -- 172.21.15.26:0/3037115517 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f479c103740 msgr2=0x7f479c107b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:18.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.664+0000 7f47a3464640 1 --2- 172.21.15.26:0/3037115517 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f479c103740 0x7f479c107b80 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f478c009930 tx=0x7f478c02a420 comp rx=0 tx=0).stop 2024-09-11T23:19:18.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.664+0000 7f47a3464640 1 -- 172.21.15.26:0/3037115517 shutdown_connections 2024-09-11T23:19:18.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.664+0000 7f47a3464640 1 --2- 172.21.15.26:0/3037115517 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f479c108290 0x7f479c10ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:18.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.664+0000 7f47a3464640 1 --2- 172.21.15.26:0/3037115517 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f479c103740 0x7f479c107b80 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:18.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.664+0000 7f47a3464640 1 --2- 172.21.15.26:0/3037115517 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f479c102d90 0x7f479c103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:18.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.664+0000 7f47a3464640 1 -- 172.21.15.26:0/3037115517 >> 172.21.15.26:0/3037115517 conn(0x7f479c0fc7d0 msgr2=0x7f479c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:19:18.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.665+0000 7f47a3464640 1 -- 172.21.15.26:0/3037115517 shutdown_connections 2024-09-11T23:19:18.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.665+0000 7f47a3464640 1 -- 172.21.15.26:0/3037115517 wait complete. 2024-09-11T23:19:18.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.665+0000 7f47a3464640 1 Processor -- start 2024-09-11T23:19:18.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.666+0000 7f47a3464640 1 -- start start 2024-09-11T23:19:18.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.666+0000 7f47a3464640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f479c102d90 0x7f479c198610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:18.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.666+0000 7f47a3464640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f479c103740 0x7f479c198b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:18.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.666+0000 7f47a3464640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f479c108290 0x7f479c19cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:18.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.666+0000 7f47a3464640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f479c112090 con 0x7f479c103740 2024-09-11T23:19:18.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.666+0000 7f47a3464640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f479c111f10 con 0x7f479c108290 2024-09-11T23:19:18.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.666+0000 7f47a3464640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f479c112210 con 0x7f479c102d90 2024-09-11T23:19:18.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.666+0000 7f47a09ff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f479c103740 0x7f479c198b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:18.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.666+0000 7f47a09ff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f479c103740 0x7f479c198b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54772/0 (socket says 172.21.15.26:54772) 2024-09-11T23:19:18.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.666+0000 7f47a09ff640 1 -- 172.21.15.26:0/1072676574 learned_addr learned my addr 172.21.15.26:0/1072676574 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:19:18.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.666+0000 7f47a1a01640 1 --2- 172.21.15.26:0/1072676574 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f479c108290 0x7f479c19cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:18.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.666+0000 7f47a1200640 1 --2- 172.21.15.26:0/1072676574 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f479c102d90 0x7f479c198610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:18.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.666+0000 7f47a09ff640 1 -- 172.21.15.26:0/1072676574 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f479c102d90 msgr2=0x7f479c198610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:18.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.666+0000 7f47a09ff640 1 --2- 172.21.15.26:0/1072676574 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f479c102d90 0x7f479c198610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:18.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.666+0000 7f47a09ff640 1 -- 172.21.15.26:0/1072676574 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f479c108290 msgr2=0x7f479c19cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:18.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.666+0000 7f47a09ff640 1 --2- 172.21.15.26:0/1072676574 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f479c108290 0x7f479c19cee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:18.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.666+0000 7f47a09ff640 1 -- 172.21.15.26:0/1072676574 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f479c19d5c0 con 0x7f479c103740 2024-09-11T23:19:18.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.667+0000 7f47a09ff640 1 --2- 172.21.15.26:0/1072676574 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f479c103740 0x7f479c198b50 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f478c005940 tx=0x7f478c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:19:18.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.667+0000 7f47927fc640 1 -- 172.21.15.26:0/1072676574 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f478c03f030 con 0x7f479c103740 2024-09-11T23:19:18.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.667+0000 7f47a3464640 1 -- 172.21.15.26:0/1072676574 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f479c19d850 con 0x7f479c103740 2024-09-11T23:19:18.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.667+0000 7f47a3464640 1 -- 172.21.15.26:0/1072676574 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f479c100920 con 0x7f479c103740 2024-09-11T23:19:18.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.667+0000 7f47927fc640 1 -- 172.21.15.26:0/1072676574 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f478c003ce0 con 0x7f479c103740 2024-09-11T23:19:18.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.667+0000 7f47927fc640 1 -- 172.21.15.26:0/1072676574 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f478c035070 con 0x7f479c103740 2024-09-11T23:19:18.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.671+0000 7f47927fc640 1 -- 172.21.15.26:0/1072676574 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 25) ==== 138066+0+0 (secure 0 0 0) 0x7f478c042030 con 0x7f479c103740 2024-09-11T23:19:18.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.671+0000 7f47a583d740 1 Processor -- start 2024-09-11T23:19:18.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.671+0000 7f47a583d740 1 -- start start 2024-09-11T23:19:18.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.671+0000 7f47a583d740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f2cdb46b40 0x55f2cdb45090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:18.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.671+0000 7f47a583d740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f2cdb455d0 0x55f2cdb45a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:18.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.671+0000 7f47a583d740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f2cdb50370 0x55f2cdb52760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:18.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.671+0000 7f47a583d740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55f2cdaab2f0 con 0x55f2cdb46b40 2024-09-11T23:19:18.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.671+0000 7f47a583d740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55f2cda52cf0 con 0x55f2cdb50370 2024-09-11T23:19:18.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.671+0000 7f47a583d740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55f2cdaa72b0 con 0x55f2cdb455d0 2024-09-11T23:19:18.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.671+0000 7f47a1a01640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f2cdb46b40 0x55f2cdb45090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:18.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.671+0000 7f47a1a01640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f2cdb46b40 0x55f2cdb45090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54786/0 (socket says 172.21.15.26:54786) 2024-09-11T23:19:18.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.671+0000 7f47a1a01640 1 -- 172.21.15.26:0/1950674358 learned_addr learned my addr 172.21.15.26:0/1950674358 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:19:18.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.671+0000 7f47a1200640 1 --2- 172.21.15.26:0/1950674358 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f2cdb455d0 0x55f2cdb45a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:18.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.672+0000 7f47a09ff640 1 --2- 172.21.15.26:0/1950674358 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f2cdb50370 0x55f2cdb52760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:18.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.672+0000 7f47a1200640 1 -- 172.21.15.26:0/1950674358 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f2cdb50370 msgr2=0x55f2cdb52760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:19:18.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.672+0000 7f47a1200640 1 --2- 172.21.15.26:0/1950674358 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f2cdb50370 0x55f2cdb52760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:18.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.672+0000 7f47a1200640 1 -- 172.21.15.26:0/1950674358 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f2cdb46b40 msgr2=0x55f2cdb45090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:19:18.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.672+0000 7f47a1200640 1 --2- 172.21.15.26:0/1950674358 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f2cdb46b40 0x55f2cdb45090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:18.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.672+0000 7f47a1200640 1 -- 172.21.15.26:0/1950674358 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f2cda373a0 con 0x55f2cdb455d0 2024-09-11T23:19:18.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.672+0000 7f47a1a01640 1 --2- 172.21.15.26:0/1950674358 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f2cdb46b40 0x55f2cdb45090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:19:18.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.672+0000 7f47a1200640 1 --2- 172.21.15.26:0/1950674358 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f2cdb455d0 0x55f2cdb45a10 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f4798004630 tx=0x7f479800bb20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=826c6c7a2edc1a97 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:19:18.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.672+0000 7f47927fc640 1 --2- 172.21.15.26:0/1072676574 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f47880a2f10 0x7f47880a53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:18.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.673+0000 7f47877fe640 1 -- 172.21.15.26:0/1950674358 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4798003bd0 con 0x55f2cdb455d0 2024-09-11T23:19:18.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.673+0000 7f47a1a01640 1 --2- 172.21.15.26:0/1072676574 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f47880a2f10 0x7f47880a53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:18.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.673+0000 7f47927fc640 1 -- 172.21.15.26:0/1072676574 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f478c0efab0 con 0x7f479c103740 2024-09-11T23:19:18.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.673+0000 7f47877fe640 1 -- 172.21.15.26:0/1950674358 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4798015070 con 0x55f2cdb455d0 2024-09-11T23:19:18.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.673+0000 7f47877fe640 1 -- 172.21.15.26:0/1950674358 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4798004d30 con 0x55f2cdb455d0 2024-09-11T23:19:18.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.674+0000 7f47a583d740 1 -- 172.21.15.26:0/1950674358 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f2cdb455d0 msgr2=0x55f2cdb45a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:18.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.674+0000 7f47a1a01640 1 --2- 172.21.15.26:0/1072676574 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f47880a2f10 0x7f47880a53d0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f4794005f40 tx=0x7f4794009600 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:19:18.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.674+0000 7f47a583d740 1 --2- 172.21.15.26:0/1950674358 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f2cdb455d0 0x55f2cdb45a10 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f4798004630 tx=0x7f479800bb20 comp rx=0 tx=0).stop 2024-09-11T23:19:18.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.674+0000 7f47a583d740 1 -- 172.21.15.26:0/1950674358 shutdown_connections 2024-09-11T23:19:18.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.674+0000 7f47a583d740 1 --2- 172.21.15.26:0/1950674358 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f2cdb50370 0x55f2cdb52760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:18.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.674+0000 7f47a583d740 1 --2- 172.21.15.26:0/1950674358 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f2cdb455d0 0x55f2cdb45a10 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:18.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.674+0000 7f47a583d740 1 --2- 172.21.15.26:0/1950674358 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f2cdb46b40 0x55f2cdb45090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:18.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.674+0000 7f47a583d740 1 -- 172.21.15.26:0/1950674358 >> 172.21.15.26:0/1950674358 conn(0x55f2cdb0e310 msgr2=0x55f2cdb41c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:19:18.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.674+0000 7f47a583d740 1 -- 172.21.15.26:0/1950674358 shutdown_connections 2024-09-11T23:19:18.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.674+0000 7f47a583d740 1 -- 172.21.15.26:0/1950674358 wait complete. 2024-09-11T23:19:18.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.675+0000 7f47a583d740 1 -- start start 2024-09-11T23:19:18.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.675+0000 7f47a583d740 1 Processor -- start 2024-09-11T23:19:18.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.676+0000 7f47a583d740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f2cdb455d0 0x55f2cdb4fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:18.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.676+0000 7f47a583d740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f2cdb46b40 0x55f2cdb4cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:18.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.676+0000 7f47a583d740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f2cdb50370 0x55f2cdb4d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:18.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.676+0000 7f47a1200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f2cdb455d0 0x55f2cdb4fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:18.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.676+0000 7f47a1200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f2cdb455d0 0x55f2cdb4fd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54796/0 (socket says 172.21.15.26:54796) 2024-09-11T23:19:18.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.676+0000 7f47a1200640 1 -- 172.21.15.26:0/3888895791 learned_addr learned my addr 172.21.15.26:0/3888895791 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:19:18.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.676+0000 7f47a583d740 1 -- 172.21.15.26:0/3888895791 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55f2cdb53070 con 0x55f2cdb455d0 2024-09-11T23:19:18.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.676+0000 7f47a583d740 1 -- 172.21.15.26:0/3888895791 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55f2cda66060 con 0x55f2cdb46b40 2024-09-11T23:19:18.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.676+0000 7f47a583d740 1 -- 172.21.15.26:0/3888895791 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55f2cdb531f0 con 0x55f2cdb50370 2024-09-11T23:19:18.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.676+0000 7f47a09ff640 1 --2- 172.21.15.26:0/3888895791 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f2cdb46b40 0x55f2cdb4cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:18.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.676+0000 7f47a1a01640 1 --2- 172.21.15.26:0/3888895791 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f2cdb50370 0x55f2cdb4d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:18.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.677+0000 7f47a583d740 10 client.0 _ll_register_callbacks cb 0x55f2cd7b98c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:19:18.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.677+0000 7f47a1200640 1 -- 172.21.15.26:0/3888895791 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f2cdb50370 msgr2=0x55f2cdb4d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:19:18.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.677+0000 7f47a1200640 1 --2- 172.21.15.26:0/3888895791 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f2cdb50370 0x55f2cdb4d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:18.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.677+0000 7f47a1200640 1 -- 172.21.15.26:0/3888895791 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f2cdb46b40 msgr2=0x55f2cdb4cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:19:18.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.677+0000 7f47a1200640 1 --2- 172.21.15.26:0/3888895791 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f2cdb46b40 0x55f2cdb4cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:18.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.677+0000 7f47a1200640 1 -- 172.21.15.26:0/3888895791 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f2cd81c3c0 con 0x55f2cdb455d0 2024-09-11T23:19:18.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.677+0000 7f47a09ff640 1 --2- 172.21.15.26:0/3888895791 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f2cdb46b40 0x55f2cdb4cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:19:18.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.677+0000 7f47a1200640 1 --2- 172.21.15.26:0/3888895791 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f2cdb455d0 0x55f2cdb4fd00 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f4798006010 tx=0x7f479800bfa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a9754c42021786b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:19:18.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.677+0000 7f4784ff9640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:19:18.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.678+0000 7f4784ff9640 1 -- 172.21.15.26:0/3888895791 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4798028060 con 0x55f2cdb455d0 2024-09-11T23:19:18.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.678+0000 7f4784ff9640 1 -- 172.21.15.26:0/3888895791 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f4798011640 con 0x55f2cdb455d0 2024-09-11T23:19:18.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.678+0000 7f47a583d740 1 -- 172.21.15.26:0/3888895791 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4798015070 con 0x55f2cdb455d0 2024-09-11T23:19:18.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.678+0000 7f4784ff9640 1 -- 172.21.15.26:0/3888895791 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4798003e60 con 0x55f2cdb455d0 2024-09-11T23:19:18.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.678+0000 7f47a583d740 10 client.15669 fetch_fsmap learned FSMap version 29 2024-09-11T23:19:18.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.678+0000 7f47a583d740 1 -- 172.21.15.26:0/3888895791 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f2cdb67290 con 0x55f2cdb455d0 2024-09-11T23:19:18.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.678+0000 7f4784ff9640 1 -- 172.21.15.26:0/3888895791 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4798015070 con 0x55f2cdb455d0 2024-09-11T23:19:18.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.678+0000 7f4784ff9640 1 -- 172.21.15.26:0/3888895791 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f4798022070 con 0x55f2cdb455d0 2024-09-11T23:19:18.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.679+0000 7f47a583d740 10 client.15669 fetch_fsmap finished waiting for FSMap version 29 2024-09-11T23:19:18.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.679+0000 7f47a583d740 10 client.15669 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-11T23:19:18.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.679+0000 7f47a583d740 20 client.15669 populate_metadata read hostname 'smithi026' 2024-09-11T23:19:18.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.679+0000 7f47a583d740 1 --2- 172.21.15.26:0/3888895791 >> [v2:172.21.15.136:6838/3211398630,v1:172.21.15.136:6839/3211398630] conn(0x55f2cdb68690 0x55f2cdb6aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:18.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.679+0000 7f47a583d740 4 client.15669 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:19:18.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.679+0000 7f47a583d740 1 -- 172.21.15.26:0/3888895791 --> [v2:172.21.15.136:6838/3211398630,v1:172.21.15.136:6839/3211398630] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f2cdab8860 con 0x55f2cdb68690 2024-09-11T23:19:18.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.679+0000 7f47a09ff640 1 --2- 172.21.15.26:0/3888895791 >> [v2:172.21.15.136:6838/3211398630,v1:172.21.15.136:6839/3211398630] conn(0x55f2cdb68690 0x55f2cdb6aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:18.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.679+0000 7f47a09ff640 1 --2- 172.21.15.26:0/3888895791 >> [v2:172.21.15.136:6838/3211398630,v1:172.21.15.136:6839/3211398630] conn(0x55f2cdb68690 0x55f2cdb6aa80 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9d8abf5e8a0b5b9b server_cookie=d4946be666218eac in_seq=0 out_seq=0 2024-09-11T23:19:18.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.679+0000 7f4784ff9640 10 client.15669 ms_handle_connect on v2:172.21.15.136:6838/3211398630 2024-09-11T23:19:18.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.680+0000 7f4784ff9640 1 -- 172.21.15.26:0/3888895791 <== mds.1 v2:172.21.15.136:6838/3211398630 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f2cdab8860 con 0x55f2cdb68690 2024-09-11T23:19:18.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.680+0000 7f4784ff9640 10 client.15669 handle_command_reply: tid=0 2024-09-11T23:19:18.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.680+0000 7f47a583d740 1 client.15669 shutdown 2024-09-11T23:19:18.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.680+0000 7f47a583d740 1 -- 172.21.15.26:0/3888895791 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f2cdb455d0 msgr2=0x55f2cdb4fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:18.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.680+0000 7f47a583d740 1 --2- 172.21.15.26:0/3888895791 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f2cdb455d0 0x55f2cdb4fd00 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f4798006010 tx=0x7f479800bfa0 comp rx=0 tx=0).stop 2024-09-11T23:19:18.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.680+0000 7f47a583d740 1 -- 172.21.15.26:0/3888895791 shutdown_connections 2024-09-11T23:19:18.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.680+0000 7f47a583d740 1 --2- 172.21.15.26:0/3888895791 >> [v2:172.21.15.136:6838/3211398630,v1:172.21.15.136:6839/3211398630] conn(0x55f2cdb68690 0x55f2cdb6aa80 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:18.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.680+0000 7f47a583d740 1 --2- 172.21.15.26:0/3888895791 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f2cdb46b40 0x55f2cdb4cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:18.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.680+0000 7f47a583d740 1 --2- 172.21.15.26:0/3888895791 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f2cdb50370 0x55f2cdb4d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:18.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.680+0000 7f47a583d740 1 --2- 172.21.15.26:0/3888895791 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f2cdb455d0 0x55f2cdb4fd00 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:18.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.680+0000 7f47a583d740 1 -- 172.21.15.26:0/3888895791 >> 172.21.15.26:0/3888895791 conn(0x55f2cdb0e310 msgr2=0x55f2cdb47960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:19:18.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.680+0000 7f4784ff9640 0 client.15669 ms_handle_reset on v2:172.21.15.136:6838/3211398630 2024-09-11T23:19:18.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.681+0000 7f47a583d740 1 -- 172.21.15.26:0/3888895791 shutdown_connections 2024-09-11T23:19:18.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.681+0000 7f47a583d740 1 -- 172.21.15.26:0/3888895791 wait complete. 2024-09-11T23:19:18.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.681+0000 7f47a583d740 20 client.15669 trim_cache size 0 max 16384 2024-09-11T23:19:18.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.706+0000 7f47a583d740 1 Processor -- start 2024-09-11T23:19:18.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.706+0000 7f47a583d740 1 -- start start 2024-09-11T23:19:18.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.707+0000 7f47a583d740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f2cdb46b40 0x55f2cdb64ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:18.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.707+0000 7f47a583d740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f2cdb50370 0x55f2cdb4bf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:18.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.707+0000 7f47a583d740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f2cdb41f20 0x55f2cdb70050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:18.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.707+0000 7f47a583d740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f479c1ab790 con 0x55f2cdb50370 2024-09-11T23:19:18.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.707+0000 7f47a583d740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f479c1ad2d0 con 0x55f2cdb41f20 2024-09-11T23:19:18.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.707+0000 7f47a583d740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f479c1abe60 con 0x55f2cdb46b40 2024-09-11T23:19:18.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.707+0000 7f47a09ff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f2cdb50370 0x55f2cdb4bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:18.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.707+0000 7f47a09ff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f2cdb50370 0x55f2cdb4bf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54800/0 (socket says 172.21.15.26:54800) 2024-09-11T23:19:18.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.707+0000 7f47a1200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f2cdb46b40 0x55f2cdb64ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:18.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.707+0000 7f47a09ff640 1 -- 172.21.15.26:0/1209009872 learned_addr learned my addr 172.21.15.26:0/1209009872 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:19:18.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.707+0000 7f47a1a01640 1 --2- 172.21.15.26:0/1209009872 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f2cdb41f20 0x55f2cdb70050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:18.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.707+0000 7f47a1a01640 1 -- 172.21.15.26:0/1209009872 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f2cdb46b40 msgr2=0x55f2cdb64ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:19:18.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.707+0000 7f47a1a01640 1 --2- 172.21.15.26:0/1209009872 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f2cdb46b40 0x55f2cdb64ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:18.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.707+0000 7f47a1a01640 1 -- 172.21.15.26:0/1209009872 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f2cdb50370 msgr2=0x55f2cdb4bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:19:18.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.707+0000 7f47a1a01640 1 --2- 172.21.15.26:0/1209009872 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f2cdb50370 0x55f2cdb4bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:18.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.707+0000 7f47a1a01640 1 -- 172.21.15.26:0/1209009872 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f2cdb42dd0 con 0x55f2cdb41f20 2024-09-11T23:19:18.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.707+0000 7f47a09ff640 1 --2- 172.21.15.26:0/1209009872 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f2cdb50370 0x55f2cdb4bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:19:18.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.708+0000 7f47a1a01640 1 --2- 172.21.15.26:0/1209009872 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f2cdb41f20 0x55f2cdb70050 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f4794005f10 tx=0x7f4794016000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e63678fc6babecc4 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:19:18.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.708+0000 7f4786ffd640 1 -- 172.21.15.26:0/1209009872 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4794004b20 con 0x55f2cdb41f20 2024-09-11T23:19:18.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.708+0000 7f4786ffd640 1 -- 172.21.15.26:0/1209009872 <== mon.2 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4794004cc0 con 0x55f2cdb41f20 2024-09-11T23:19:18.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.708+0000 7f4786ffd640 1 -- 172.21.15.26:0/1209009872 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f479401e770 con 0x55f2cdb41f20 2024-09-11T23:19:18.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.709+0000 7f47a583d740 1 -- 172.21.15.26:0/1209009872 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f2cdb41f20 msgr2=0x55f2cdb70050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:18.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.709+0000 7f47a583d740 1 --2- 172.21.15.26:0/1209009872 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f2cdb41f20 0x55f2cdb70050 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f4794005f10 tx=0x7f4794016000 comp rx=0 tx=0).stop 2024-09-11T23:19:18.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.709+0000 7f47a583d740 1 -- 172.21.15.26:0/1209009872 shutdown_connections 2024-09-11T23:19:18.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.709+0000 7f47a583d740 1 --2- 172.21.15.26:0/1209009872 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f2cdb41f20 0x55f2cdb70050 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:18.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.709+0000 7f47a583d740 1 --2- 172.21.15.26:0/1209009872 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f2cdb50370 0x55f2cdb4bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:18.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.709+0000 7f47a583d740 1 --2- 172.21.15.26:0/1209009872 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f2cdb46b40 0x55f2cdb64ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:18.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.709+0000 7f47a583d740 1 -- 172.21.15.26:0/1209009872 >> 172.21.15.26:0/1209009872 conn(0x55f2cdb0e310 msgr2=0x55f2cdb4b880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:19:18.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.709+0000 7f47a583d740 1 -- 172.21.15.26:0/1209009872 shutdown_connections 2024-09-11T23:19:18.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.709+0000 7f47a583d740 1 -- 172.21.15.26:0/1209009872 wait complete. 2024-09-11T23:19:18.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.709+0000 7f47a583d740 1 -- start start 2024-09-11T23:19:18.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.710+0000 7f47a583d740 1 Processor -- start 2024-09-11T23:19:18.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.710+0000 7f47a583d740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f2cdb41f20 0x55f2cdb614f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:18.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.710+0000 7f47a583d740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f2cdb46b40 0x55f2cdb5c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:18.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.710+0000 7f47a583d740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f2cdb50370 0x55f2cdb5cd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:18.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.710+0000 7f47a583d740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f479c1acc00 con 0x55f2cdb41f20 2024-09-11T23:19:18.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.710+0000 7f47a583d740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f479c1ac530 con 0x55f2cdb50370 2024-09-11T23:19:18.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.710+0000 7f47a583d740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f479c1a9770 con 0x55f2cdb46b40 2024-09-11T23:19:18.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.710+0000 7f47a583d740 10 client.0 _ll_register_callbacks cb 0x7f4794009a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:19:18.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.710+0000 7f47a1a01640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f2cdb50370 0x55f2cdb5cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:18.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.710+0000 7f47a1a01640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f2cdb50370 0x55f2cdb5cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:55954/0 (socket says 172.21.15.26:55954) 2024-09-11T23:19:18.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.710+0000 7f47a1a01640 1 -- 172.21.15.26:0/100215972 learned_addr learned my addr 172.21.15.26:0/100215972 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:19:18.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.711+0000 7f47a1200640 1 --2- 172.21.15.26:0/100215972 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f2cdb41f20 0x55f2cdb614f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:18.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.711+0000 7f47a09ff640 1 --2- 172.21.15.26:0/100215972 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f2cdb46b40 0x55f2cdb5c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:18.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.711+0000 7f47a1a01640 1 -- 172.21.15.26:0/100215972 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f2cdb46b40 msgr2=0x55f2cdb5c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:19:18.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.711+0000 7f47a1a01640 1 --2- 172.21.15.26:0/100215972 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f2cdb46b40 0x55f2cdb5c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:18.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.711+0000 7f47a1a01640 1 -- 172.21.15.26:0/100215972 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f2cdb41f20 msgr2=0x55f2cdb614f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:19:18.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.711+0000 7f47a1a01640 1 --2- 172.21.15.26:0/100215972 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f2cdb41f20 0x55f2cdb614f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:18.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.711+0000 7f47a1a01640 1 -- 172.21.15.26:0/100215972 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f2cdb5d5b0 con 0x55f2cdb50370 2024-09-11T23:19:18.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.711+0000 7f47a1a01640 1 --2- 172.21.15.26:0/100215972 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f2cdb50370 0x55f2cdb5cd80 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f47940099d0 tx=0x7f479401fba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ccfe2d50d2acef2d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:19:18.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.711+0000 7f4784ff9640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-11T23:19:18.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.711+0000 7f4784ff9640 1 -- 172.21.15.26:0/100215972 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4794011e90 con 0x55f2cdb50370 2024-09-11T23:19:18.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.711+0000 7f47a583d740 1 -- 172.21.15.26:0/100215972 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4794004cc0 con 0x55f2cdb50370 2024-09-11T23:19:18.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.711+0000 7f4784ff9640 1 -- 172.21.15.26:0/100215972 <== mon.2 v2:172.21.15.76:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f4794028600 con 0x55f2cdb50370 2024-09-11T23:19:18.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.712+0000 7f4784ff9640 1 -- 172.21.15.26:0/100215972 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4794040050 con 0x55f2cdb50370 2024-09-11T23:19:18.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.712+0000 7f47a583d740 10 client.25277 fetch_fsmap learned FSMap version 29 2024-09-11T23:19:18.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.712+0000 7f47a583d740 1 -- 172.21.15.26:0/100215972 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f2cdbd30b0 con 0x55f2cdb50370 2024-09-11T23:19:18.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.712+0000 7f4784ff9640 1 -- 172.21.15.26:0/100215972 <== mon.2 v2:172.21.15.76:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4794004cc0 con 0x55f2cdb50370 2024-09-11T23:19:18.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.712+0000 7f4784ff9640 1 -- 172.21.15.26:0/100215972 <== mon.2 v2:172.21.15.76:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f4794031070 con 0x55f2cdb50370 2024-09-11T23:19:18.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.712+0000 7f47a583d740 10 client.25277 fetch_fsmap finished waiting for FSMap version 29 2024-09-11T23:19:18.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.712+0000 7f47a583d740 10 client.25277 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-11T23:19:18.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.712+0000 7f47a583d740 20 client.25277 populate_metadata read hostname 'smithi026' 2024-09-11T23:19:18.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.712+0000 7f47a583d740 1 --2- 172.21.15.26:0/100215972 >> [v2:172.21.15.136:6838/3211398630,v1:172.21.15.136:6839/3211398630] conn(0x55f2cdbd3780 0x55f2cdbd5ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:18.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.712+0000 7f47a583d740 4 client.25277 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:19:18.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.712+0000 7f47a583d740 1 -- 172.21.15.26:0/100215972 --> [v2:172.21.15.136:6838/3211398630,v1:172.21.15.136:6839/3211398630] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f2cdab2d70 con 0x55f2cdbd3780 2024-09-11T23:19:18.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.713+0000 7f47a1200640 1 --2- 172.21.15.26:0/100215972 >> [v2:172.21.15.136:6838/3211398630,v1:172.21.15.136:6839/3211398630] conn(0x55f2cdbd3780 0x55f2cdbd5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:18.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.713+0000 7f4784ff9640 10 client.25277 ms_handle_connect on v2:172.21.15.136:6838/3211398630 2024-09-11T23:19:18.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.713+0000 7f47a1200640 1 --2- 172.21.15.26:0/100215972 >> [v2:172.21.15.136:6838/3211398630,v1:172.21.15.136:6839/3211398630] conn(0x55f2cdbd3780 0x55f2cdbd5ba0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a2f5ee1a73f0d5dc server_cookie=4a90d33daa17f1 in_seq=0 out_seq=0 2024-09-11T23:19:18.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.713+0000 7f4784ff9640 1 -- 172.21.15.26:0/100215972 <== mds.1 v2:172.21.15.136:6838/3211398630 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f2cdab2d70 con 0x55f2cdbd3780 2024-09-11T23:19:18.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.713+0000 7f4784ff9640 10 client.25277 handle_command_reply: tid=0 2024-09-11T23:19:18.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.713+0000 7f47a583d740 1 client.25277 shutdown 2024-09-11T23:19:18.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.713+0000 7f47a583d740 1 -- 172.21.15.26:0/100215972 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f2cdb50370 msgr2=0x55f2cdb5cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:18.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.713+0000 7f47a583d740 1 --2- 172.21.15.26:0/100215972 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f2cdb50370 0x55f2cdb5cd80 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f47940099d0 tx=0x7f479401fba0 comp rx=0 tx=0).stop 2024-09-11T23:19:18.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.714+0000 7f47a583d740 1 -- 172.21.15.26:0/100215972 shutdown_connections 2024-09-11T23:19:18.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.714+0000 7f47a583d740 1 --2- 172.21.15.26:0/100215972 >> [v2:172.21.15.136:6838/3211398630,v1:172.21.15.136:6839/3211398630] conn(0x55f2cdbd3780 0x55f2cdbd5ba0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:18.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.714+0000 7f47a583d740 1 --2- 172.21.15.26:0/100215972 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f2cdb50370 0x55f2cdb5cd80 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:18.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.714+0000 7f47a583d740 1 --2- 172.21.15.26:0/100215972 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f2cdb46b40 0x55f2cdb5c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:18.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.714+0000 7f47a583d740 1 --2- 172.21.15.26:0/100215972 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f2cdb41f20 0x55f2cdb614f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:18.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.714+0000 7f47a583d740 1 -- 172.21.15.26:0/100215972 >> 172.21.15.26:0/100215972 conn(0x55f2cdb0e310 msgr2=0x55f2cdb4c0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:19:18.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.714+0000 7f4784ff9640 0 client.25277 ms_handle_reset on v2:172.21.15.136:6838/3211398630 2024-09-11T23:19:18.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.714+0000 7f47a583d740 1 -- 172.21.15.26:0/100215972 shutdown_connections 2024-09-11T23:19:18.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.714+0000 7f47a583d740 1 -- 172.21.15.26:0/100215972 wait complete. 2024-09-11T23:19:18.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.714+0000 7f47a583d740 20 client.25277 trim_cache size 0 max 16384 2024-09-11T23:19:18.720 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2024-09-11T23:19:18.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.715+0000 7f47a3464640 1 -- 172.21.15.26:0/1072676574 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f47880a2f10 msgr2=0x7f47880a53d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:18.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.715+0000 7f47a3464640 1 --2- 172.21.15.26:0/1072676574 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f47880a2f10 0x7f47880a53d0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f4794005f40 tx=0x7f4794009600 comp rx=0 tx=0).stop 2024-09-11T23:19:18.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.715+0000 7f47a3464640 1 -- 172.21.15.26:0/1072676574 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f479c103740 msgr2=0x7f479c198b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:18.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.715+0000 7f47a3464640 1 --2- 172.21.15.26:0/1072676574 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f479c103740 0x7f479c198b50 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f478c005940 tx=0x7f478c0042f0 comp rx=0 tx=0).stop 2024-09-11T23:19:18.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.715+0000 7f47a3464640 1 -- 172.21.15.26:0/1072676574 shutdown_connections 2024-09-11T23:19:18.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.715+0000 7f47a3464640 1 --2- 172.21.15.26:0/1072676574 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f47880a2f10 0x7f47880a53d0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:18.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.715+0000 7f47a3464640 1 --2- 172.21.15.26:0/1072676574 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f479c108290 0x7f479c19cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:18.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.715+0000 7f47a3464640 1 --2- 172.21.15.26:0/1072676574 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f479c103740 0x7f479c198b50 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:18.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.715+0000 7f47a3464640 1 --2- 172.21.15.26:0/1072676574 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f479c102d90 0x7f479c198610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:18.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.715+0000 7f47a3464640 1 -- 172.21.15.26:0/1072676574 >> 172.21.15.26:0/1072676574 conn(0x7f479c0fc7d0 msgr2=0x7f479c105e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:19:18.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.715+0000 7f47a3464640 1 -- 172.21.15.26:0/1072676574 shutdown_connections 2024-09-11T23:19:18.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:18.715+0000 7f47a3464640 1 -- 172.21.15.26:0/1072676574 wait complete. 2024-09-11T23:19:19.574 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-11T23:19:19.863 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:19:20.163 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:19:19 smithi026 ceph-mon[31172]: pgmap v31: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 2.6 KiB/s wr, 14 op/s 2024-09-11T23:19:20.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:19:19 smithi136 ceph-mon[35121]: pgmap v31: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 2.6 KiB/s wr, 14 op/s 2024-09-11T23:19:20.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:19:19 smithi076 ceph-mon[35296]: pgmap v31: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 2.6 KiB/s wr, 14 op/s 2024-09-11T23:19:20.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.436+0000 7f08602d8640 1 -- 172.21.15.26:0/1891531770 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f08581027a0 msgr2=0x7f08581067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:20.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.436+0000 7f08602d8640 1 --2- 172.21.15.26:0/1891531770 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f08581027a0 0x7f08581067f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f0848009900 tx=0x7f084802a410 comp rx=0 tx=0).stop 2024-09-11T23:19:20.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.436+0000 7f08602d8640 1 -- 172.21.15.26:0/1891531770 shutdown_connections 2024-09-11T23:19:20.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.436+0000 7f08602d8640 1 --2- 172.21.15.26:0/1891531770 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f08581027a0 0x7f08581067f0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:20.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.436+0000 7f08602d8640 1 --2- 172.21.15.26:0/1891531770 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0858106f20 0x7f085810eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:20.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.436+0000 7f08602d8640 1 --2- 172.21.15.26:0/1891531770 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0858101df0 0x7f08581021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:20.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.436+0000 7f08602d8640 1 -- 172.21.15.26:0/1891531770 >> 172.21.15.26:0/1891531770 conn(0x7f08580fd8f0 msgr2=0x7f08580ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:19:20.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.436+0000 7f08602d8640 1 -- 172.21.15.26:0/1891531770 shutdown_connections 2024-09-11T23:19:20.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.436+0000 7f08602d8640 1 -- 172.21.15.26:0/1891531770 wait complete. 2024-09-11T23:19:20.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.437+0000 7f08602d8640 1 Processor -- start 2024-09-11T23:19:20.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.437+0000 7f08602d8640 1 -- start start 2024-09-11T23:19:20.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.438+0000 7f08602d8640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0858101df0 0x7f08580744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:20.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.438+0000 7f08602d8640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f08581027a0 0x7f0858074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:20.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.438+0000 7f08602d8640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0858106f20 0x7f0858074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:20.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.438+0000 7f085e875640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0858106f20 0x7f0858074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:20.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.438+0000 7f085e875640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0858106f20 0x7f0858074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54820/0 (socket says 172.21.15.26:54820) 2024-09-11T23:19:20.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.438+0000 7f085e875640 1 -- 172.21.15.26:0/3773958062 learned_addr learned my addr 172.21.15.26:0/3773958062 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:19:20.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.438+0000 7f08602d8640 1 -- 172.21.15.26:0/3773958062 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f0858111f80 con 0x7f0858106f20 2024-09-11T23:19:20.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.438+0000 7f08602d8640 1 -- 172.21.15.26:0/3773958062 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f0858111e00 con 0x7f0858101df0 2024-09-11T23:19:20.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.438+0000 7f08602d8640 1 -- 172.21.15.26:0/3773958062 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f0858112100 con 0x7f08581027a0 2024-09-11T23:19:20.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.438+0000 7f085e074640 1 --2- 172.21.15.26:0/3773958062 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0858101df0 0x7f08580744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:20.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.438+0000 7f085d873640 1 --2- 172.21.15.26:0/3773958062 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f08581027a0 0x7f0858074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:20.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.438+0000 7f085e074640 1 -- 172.21.15.26:0/3773958062 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f08581027a0 msgr2=0x7f0858074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:20.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.438+0000 7f085e074640 1 --2- 172.21.15.26:0/3773958062 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f08581027a0 0x7f0858074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:20.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.438+0000 7f085e074640 1 -- 172.21.15.26:0/3773958062 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0858106f20 msgr2=0x7f0858074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:20.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.438+0000 7f085e074640 1 --2- 172.21.15.26:0/3773958062 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0858106f20 0x7f0858074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:20.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.438+0000 7f085e074640 1 -- 172.21.15.26:0/3773958062 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08581a68f0 con 0x7f0858101df0 2024-09-11T23:19:20.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.439+0000 7f085e875640 1 --2- 172.21.15.26:0/3773958062 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0858106f20 0x7f0858074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:19:20.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.439+0000 7f085e074640 1 --2- 172.21.15.26:0/3773958062 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0858101df0 0x7f08580744f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f085400d720 tx=0x7f085400dbe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:19:20.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.439+0000 7f084f7fe640 1 -- 172.21.15.26:0/3773958062 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0854017070 con 0x7f0858101df0 2024-09-11T23:19:20.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.439+0000 7f08602d8640 1 -- 172.21.15.26:0/3773958062 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08581a6be0 con 0x7f0858101df0 2024-09-11T23:19:20.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.439+0000 7f084f7fe640 1 -- 172.21.15.26:0/3773958062 <== mon.2 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f085400de30 con 0x7f0858101df0 2024-09-11T23:19:20.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.440+0000 7f084f7fe640 1 -- 172.21.15.26:0/3773958062 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08540047a0 con 0x7f0858101df0 2024-09-11T23:19:20.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.440+0000 7f08602d8640 1 -- 172.21.15.26:0/3773958062 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08581a7120 con 0x7f0858101df0 2024-09-11T23:19:20.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.443+0000 7f084f7fe640 1 -- 172.21.15.26:0/3773958062 <== mon.2 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 25) ==== 138066+0+0 (secure 0 0 0) 0x7f08540099d0 con 0x7f0858101df0 2024-09-11T23:19:20.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.443+0000 7f08626b1740 1 Processor -- start 2024-09-11T23:19:20.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.444+0000 7f08626b1740 1 -- start start 2024-09-11T23:19:20.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.444+0000 7f08626b1740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56166e1eeb40 0x56166e1ed090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:20.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.444+0000 7f08626b1740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56166e1ed5d0 0x56166e1eda10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:20.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.444+0000 7f085e875640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56166e1eeb40 0x56166e1ed090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:20.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.444+0000 7f085e875640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56166e1eeb40 0x56166e1ed090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54836/0 (socket says 172.21.15.26:54836) 2024-09-11T23:19:20.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.444+0000 7f085e875640 1 -- 172.21.15.26:0/4255193147 learned_addr learned my addr 172.21.15.26:0/4255193147 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:19:20.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.444+0000 7f085e074640 1 --2- 172.21.15.26:0/4255193147 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56166e1ed5d0 0x56166e1eda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:20.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.444+0000 7f08626b1740 1 --2- 172.21.15.26:0/4255193147 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56166e1f8370 0x56166e1fa760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:20.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.445+0000 7f08626b1740 1 -- 172.21.15.26:0/4255193147 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x56166e1532f0 con 0x56166e1eeb40 2024-09-11T23:19:20.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.445+0000 7f08626b1740 1 -- 172.21.15.26:0/4255193147 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x56166e0facf0 con 0x56166e1f8370 2024-09-11T23:19:20.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.445+0000 7f08626b1740 1 -- 172.21.15.26:0/4255193147 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x56166e14f2b0 con 0x56166e1ed5d0 2024-09-11T23:19:20.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.445+0000 7f085d873640 1 --2- 172.21.15.26:0/4255193147 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56166e1f8370 0x56166e1fa760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:20.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.445+0000 7f085e875640 1 -- 172.21.15.26:0/4255193147 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56166e1ed5d0 msgr2=0x56166e1eda10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:19:20.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.445+0000 7f085e875640 1 --2- 172.21.15.26:0/4255193147 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56166e1ed5d0 0x56166e1eda10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:20.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.445+0000 7f085e875640 1 -- 172.21.15.26:0/4255193147 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56166e1f8370 msgr2=0x56166e1fa760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:19:20.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.445+0000 7f085e875640 1 --2- 172.21.15.26:0/4255193147 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56166e1f8370 0x56166e1fa760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:20.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.445+0000 7f085e875640 1 -- 172.21.15.26:0/4255193147 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56166e0df3a0 con 0x56166e1eeb40 2024-09-11T23:19:20.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.445+0000 7f085e875640 1 --2- 172.21.15.26:0/4255193147 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56166e1eeb40 0x56166e1ed090 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f085000f390 tx=0x7f085000f850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a9e873b31079b31 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:19:20.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.445+0000 7f084cdea640 1 -- 172.21.15.26:0/4255193147 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0850011700 con 0x56166e1eeb40 2024-09-11T23:19:20.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.446+0000 7f084cdea640 1 -- 172.21.15.26:0/4255193147 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0850011d80 con 0x56166e1eeb40 2024-09-11T23:19:20.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.446+0000 7f084cdea640 1 -- 172.21.15.26:0/4255193147 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08500103d0 con 0x56166e1eeb40 2024-09-11T23:19:20.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.446+0000 7f084f7fe640 1 --2- 172.21.15.26:0/3773958062 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f08440a2e90 0x7f08440a5350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:20.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.446+0000 7f08626b1740 1 -- 172.21.15.26:0/4255193147 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56166e1eeb40 msgr2=0x56166e1ed090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:20.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.446+0000 7f08626b1740 1 --2- 172.21.15.26:0/4255193147 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56166e1eeb40 0x56166e1ed090 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f085000f390 tx=0x7f085000f850 comp rx=0 tx=0).stop 2024-09-11T23:19:20.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.447+0000 7f08626b1740 1 -- 172.21.15.26:0/4255193147 shutdown_connections 2024-09-11T23:19:20.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.447+0000 7f08626b1740 1 --2- 172.21.15.26:0/4255193147 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56166e1f8370 0x56166e1fa760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:20.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.447+0000 7f08626b1740 1 --2- 172.21.15.26:0/4255193147 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56166e1ed5d0 0x56166e1eda10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:20.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.447+0000 7f08626b1740 1 --2- 172.21.15.26:0/4255193147 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56166e1eeb40 0x56166e1ed090 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:20.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.447+0000 7f08626b1740 1 -- 172.21.15.26:0/4255193147 >> 172.21.15.26:0/4255193147 conn(0x56166e1b6310 msgr2=0x56166e1e9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:19:20.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.447+0000 7f084f7fe640 1 -- 172.21.15.26:0/3773958062 <== mon.2 v2:172.21.15.76:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f08540cf3a0 con 0x7f0858101df0 2024-09-11T23:19:20.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.448+0000 7f085e074640 1 --2- 172.21.15.26:0/3773958062 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f08440a2e90 0x7f08440a5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:20.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.448+0000 7f08626b1740 1 -- 172.21.15.26:0/4255193147 shutdown_connections 2024-09-11T23:19:20.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.448+0000 7f08626b1740 1 -- 172.21.15.26:0/4255193147 wait complete. 2024-09-11T23:19:20.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.448+0000 7f085e074640 1 --2- 172.21.15.26:0/3773958062 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f08440a2e90 0x7f08440a5350 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f08540cc9e0 tx=0x7f08540d2eb0 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:19:20.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.448+0000 7f08626b1740 1 -- start start 2024-09-11T23:19:20.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.448+0000 7f08626b1740 1 Processor -- start 2024-09-11T23:19:20.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.449+0000 7f08626b1740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56166e1ed5d0 0x56166e1f7da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:20.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.449+0000 7f08626b1740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56166e1eeb40 0x56166e1f5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:20.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.449+0000 7f08626b1740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56166e1f8370 0x56166e1f57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:20.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.449+0000 7f08626b1740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x56166e1fb070 con 0x56166e1eeb40 2024-09-11T23:19:20.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.449+0000 7f08626b1740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x56166e10e060 con 0x56166e1ed5d0 2024-09-11T23:19:20.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.449+0000 7f08626b1740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x56166e1fb1f0 con 0x56166e1f8370 2024-09-11T23:19:20.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.449+0000 7f08626b1740 10 client.0 _ll_register_callbacks cb 0x56166de618c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:19:20.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.449+0000 7f085d873640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56166e1eeb40 0x56166e1f5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:20.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.449+0000 7f085e875640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56166e1ed5d0 0x56166e1f7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:20.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.449+0000 7f085e875640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56166e1f8370 0x56166e1f57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:20.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.449+0000 7f085d873640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56166e1eeb40 0x56166e1f5050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54848/0 (socket says 172.21.15.26:54848) 2024-09-11T23:19:20.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.449+0000 7f085e875640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56166e1ed5d0 0x56166e1f7da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:55986/0 (socket says 172.21.15.26:55986) 2024-09-11T23:19:20.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.449+0000 7f085d873640 1 -- 172.21.15.26:0/3809138900 learned_addr learned my addr 172.21.15.26:0/3809138900 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:19:20.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.450+0000 7f085d873640 1 -- 172.21.15.26:0/3809138900 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56166e1f8370 msgr2=0x56166e1f57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:19:20.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.450+0000 7f085d873640 1 --2- 172.21.15.26:0/3809138900 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56166e1f8370 0x56166e1f57e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:20.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.450+0000 7f085d873640 1 -- 172.21.15.26:0/3809138900 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56166e1ed5d0 msgr2=0x56166e1f7da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:19:20.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.450+0000 7f085d873640 1 --2- 172.21.15.26:0/3809138900 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56166e1ed5d0 0x56166e1f7da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:20.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.450+0000 7f085d873640 1 -- 172.21.15.26:0/3809138900 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56166dec43c0 con 0x56166e1eeb40 2024-09-11T23:19:20.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.450+0000 7f085e875640 1 --2- 172.21.15.26:0/3809138900 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56166e1ed5d0 0x56166e1f7da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:19:20.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.450+0000 7f085d873640 1 --2- 172.21.15.26:0/3809138900 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56166e1eeb40 0x56166e1f5050 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f0848005940 tx=0x7f0848031840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dc5f2684644a06f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:19:20.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.450+0000 7f0841ffb640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:19:20.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.451+0000 7f0841ffb640 1 -- 172.21.15.26:0/3809138900 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f084803f070 con 0x56166e1eeb40 2024-09-11T23:19:20.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.451+0000 7f0841ffb640 1 -- 172.21.15.26:0/3809138900 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f0848004450 con 0x56166e1eeb40 2024-09-11T23:19:20.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.451+0000 7f08626b1740 1 -- 172.21.15.26:0/3809138900 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0850011d80 con 0x56166e1eeb40 2024-09-11T23:19:20.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.451+0000 7f0841ffb640 1 -- 172.21.15.26:0/3809138900 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0848035070 con 0x56166e1eeb40 2024-09-11T23:19:20.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.451+0000 7f08626b1740 10 client.15684 fetch_fsmap learned FSMap version 29 2024-09-11T23:19:20.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.451+0000 7f08626b1740 1 -- 172.21.15.26:0/3809138900 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56166e206b50 con 0x56166e1eeb40 2024-09-11T23:19:20.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.451+0000 7f0841ffb640 1 -- 172.21.15.26:0/3809138900 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0850011d80 con 0x56166e1eeb40 2024-09-11T23:19:20.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.452+0000 7f0841ffb640 1 -- 172.21.15.26:0/3809138900 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f0848004900 con 0x56166e1eeb40 2024-09-11T23:19:20.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.452+0000 7f08626b1740 10 client.15684 fetch_fsmap finished waiting for FSMap version 29 2024-09-11T23:19:20.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.452+0000 7f08626b1740 10 client.15684 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:19:20.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.452+0000 7f08626b1740 20 client.15684 populate_metadata read hostname 'smithi026' 2024-09-11T23:19:20.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.452+0000 7f08626b1740 1 --2- 172.21.15.26:0/3809138900 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x56166e207f60 0x56166e20a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:20.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.452+0000 7f08626b1740 4 client.15684 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:19:20.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.452+0000 7f08626b1740 1 -- 172.21.15.26:0/3809138900 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56166e160860 con 0x56166e207f60 2024-09-11T23:19:20.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.453+0000 7f085e875640 1 --2- 172.21.15.26:0/3809138900 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x56166e207f60 0x56166e20a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:20.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.453+0000 7f085e875640 1 --2- 172.21.15.26:0/3809138900 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x56166e207f60 0x56166e20a350 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=555ec9562e6ed4ac server_cookie=502cf778afa38b2b in_seq=0 out_seq=0 2024-09-11T23:19:20.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.453+0000 7f0841ffb640 10 client.15684 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:19:20.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.454+0000 7f0841ffb640 1 -- 172.21.15.26:0/3809138900 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56166e160860 con 0x56166e207f60 2024-09-11T23:19:20.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.454+0000 7f0841ffb640 10 client.15684 handle_command_reply: tid=0 2024-09-11T23:19:20.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.454+0000 7f08626b1740 1 client.15684 shutdown 2024-09-11T23:19:20.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.454+0000 7f08626b1740 1 -- 172.21.15.26:0/3809138900 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56166e1eeb40 msgr2=0x56166e1f5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:20.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.454+0000 7f08626b1740 1 --2- 172.21.15.26:0/3809138900 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56166e1eeb40 0x56166e1f5050 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f0848005940 tx=0x7f0848031840 comp rx=0 tx=0).stop 2024-09-11T23:19:20.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.454+0000 7f08626b1740 1 -- 172.21.15.26:0/3809138900 shutdown_connections 2024-09-11T23:19:20.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.454+0000 7f08626b1740 1 --2- 172.21.15.26:0/3809138900 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x56166e207f60 0x56166e20a350 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:20.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.454+0000 7f08626b1740 1 --2- 172.21.15.26:0/3809138900 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56166e1f8370 0x56166e1f57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:20.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.454+0000 7f0841ffb640 0 client.15684 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:19:20.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.454+0000 7f08626b1740 1 --2- 172.21.15.26:0/3809138900 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56166e1eeb40 0x56166e1f5050 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:20.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.454+0000 7f08626b1740 1 --2- 172.21.15.26:0/3809138900 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56166e1ed5d0 0x56166e1f7da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:20.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.454+0000 7f08626b1740 1 -- 172.21.15.26:0/3809138900 >> 172.21.15.26:0/3809138900 conn(0x56166e1b6310 msgr2=0x56166e1ef960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:19:20.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.454+0000 7f08626b1740 1 -- 172.21.15.26:0/3809138900 shutdown_connections 2024-09-11T23:19:20.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.454+0000 7f08626b1740 1 -- 172.21.15.26:0/3809138900 wait complete. 2024-09-11T23:19:20.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.455+0000 7f08626b1740 20 client.15684 trim_cache size 0 max 16384 2024-09-11T23:19:20.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.481+0000 7f08626b1740 1 Processor -- start 2024-09-11T23:19:20.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.481+0000 7f08626b1740 1 -- start start 2024-09-11T23:19:20.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.481+0000 7f08626b1740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56166e1ed5d0 0x56166e1ee300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:20.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.481+0000 7f08626b1740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56166e1f8370 0x56166e1f3920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:20.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.481+0000 7f08626b1740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56166e207f60 0x56166e1f3e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:20.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.481+0000 7f08626b1740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f08581b3d20 con 0x56166e1ed5d0 2024-09-11T23:19:20.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.481+0000 7f08626b1740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f08581b5860 con 0x56166e207f60 2024-09-11T23:19:20.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.481+0000 7f08626b1740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f08581b43f0 con 0x56166e1f8370 2024-09-11T23:19:20.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.481+0000 7f085e875640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56166e1ed5d0 0x56166e1ee300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:20.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.481+0000 7f085e875640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56166e1ed5d0 0x56166e1ee300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54854/0 (socket says 172.21.15.26:54854) 2024-09-11T23:19:20.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.481+0000 7f085e875640 1 -- 172.21.15.26:0/3239427348 learned_addr learned my addr 172.21.15.26:0/3239427348 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:19:20.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.481+0000 7f085d873640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56166e1f8370 0x56166e1f3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:20.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.482+0000 7f085d873640 1 -- 172.21.15.26:0/3239427348 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56166e207f60 msgr2=0x56166e1f3e60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T23:19:20.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.482+0000 7f085d873640 1 --2- 172.21.15.26:0/3239427348 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56166e207f60 0x56166e1f3e60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:20.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.482+0000 7f085d873640 1 -- 172.21.15.26:0/3239427348 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56166e1ed5d0 msgr2=0x56166e1ee300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:19:20.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.482+0000 7f085d873640 1 --2- 172.21.15.26:0/3239427348 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56166e1ed5d0 0x56166e1ee300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:20.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.482+0000 7f085d873640 1 -- 172.21.15.26:0/3239427348 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56166e1faad0 con 0x56166e1f8370 2024-09-11T23:19:20.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.482+0000 7f085e875640 1 --2- 172.21.15.26:0/3239427348 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56166e1ed5d0 0x56166e1ee300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:19:20.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.482+0000 7f085d873640 1 --2- 172.21.15.26:0/3239427348 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56166e1f8370 0x56166e1f3920 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x56166e1f6140 tx=0x7f084802a410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c98a683b155bb146 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:19:20.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.482+0000 7f0843fff640 1 -- 172.21.15.26:0/3239427348 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0848000e60 con 0x56166e1f8370 2024-09-11T23:19:20.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.482+0000 7f0843fff640 1 -- 172.21.15.26:0/3239427348 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08480318b0 con 0x56166e1f8370 2024-09-11T23:19:20.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.482+0000 7f0843fff640 1 -- 172.21.15.26:0/3239427348 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f084803f070 con 0x56166e1f8370 2024-09-11T23:19:20.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.483+0000 7f08626b1740 1 -- 172.21.15.26:0/3239427348 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56166e1f8370 msgr2=0x56166e1f3920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:20.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.483+0000 7f08626b1740 1 --2- 172.21.15.26:0/3239427348 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56166e1f8370 0x56166e1f3920 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x56166e1f6140 tx=0x7f084802a410 comp rx=0 tx=0).stop 2024-09-11T23:19:20.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.483+0000 7f08626b1740 1 -- 172.21.15.26:0/3239427348 shutdown_connections 2024-09-11T23:19:20.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.483+0000 7f08626b1740 1 --2- 172.21.15.26:0/3239427348 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56166e207f60 0x56166e1f3e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:20.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.483+0000 7f08626b1740 1 --2- 172.21.15.26:0/3239427348 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56166e1f8370 0x56166e1f3920 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:20.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.483+0000 7f08626b1740 1 --2- 172.21.15.26:0/3239427348 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56166e1ed5d0 0x56166e1ee300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:20.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.483+0000 7f08626b1740 1 -- 172.21.15.26:0/3239427348 >> 172.21.15.26:0/3239427348 conn(0x56166e1b6310 msgr2=0x56166e1e4a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:19:20.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.484+0000 7f08626b1740 1 -- 172.21.15.26:0/3239427348 shutdown_connections 2024-09-11T23:19:20.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.484+0000 7f08626b1740 1 -- 172.21.15.26:0/3239427348 wait complete. 2024-09-11T23:19:20.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.484+0000 7f08626b1740 1 -- start start 2024-09-11T23:19:20.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.484+0000 7f08626b1740 1 Processor -- start 2024-09-11T23:19:20.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.484+0000 7f08626b1740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56166e1ed5d0 0x56166e200a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:20.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.484+0000 7f08626b1740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56166e1f8370 0x56166e1fbd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:20.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.484+0000 7f08626b1740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56166e207f60 0x56166e1fc280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:20.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.485+0000 7f085d873640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56166e1f8370 0x56166e1fbd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:20.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.485+0000 7f085d873640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56166e1f8370 0x56166e1fbd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:45938/0 (socket says 172.21.15.26:45938) 2024-09-11T23:19:20.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.485+0000 7f085e875640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56166e207f60 0x56166e1fc280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:20.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.485+0000 7f085e875640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56166e207f60 0x56166e1fc280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54866/0 (socket says 172.21.15.26:54866) 2024-09-11T23:19:20.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.485+0000 7f08626b1740 1 -- 172.21.15.26:0/3749777161 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f08581b5190 con 0x56166e207f60 2024-09-11T23:19:20.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.485+0000 7f085d873640 1 -- 172.21.15.26:0/3749777161 learned_addr learned my addr 172.21.15.26:0/3749777161 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:19:20.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.485+0000 7f08626b1740 1 -- 172.21.15.26:0/3749777161 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f08581b4ac0 con 0x56166e1ed5d0 2024-09-11T23:19:20.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.485+0000 7f08626b1740 1 -- 172.21.15.26:0/3749777161 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f08581b1ef0 con 0x56166e1f8370 2024-09-11T23:19:20.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.485+0000 7f085e875640 1 -- 172.21.15.26:0/3749777161 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56166e1f8370 msgr2=0x56166e1fbd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:19:20.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.485+0000 7f085e875640 1 --2- 172.21.15.26:0/3749777161 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56166e1f8370 0x56166e1fbd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:20.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.485+0000 7f085e875640 1 -- 172.21.15.26:0/3749777161 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56166e1ed5d0 msgr2=0x56166e200a10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T23:19:20.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.485+0000 7f085e875640 1 --2- 172.21.15.26:0/3749777161 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56166e1ed5d0 0x56166e200a10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:20.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.485+0000 7f085e875640 1 -- 172.21.15.26:0/3749777161 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56166e1fc8c0 con 0x56166e207f60 2024-09-11T23:19:20.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.485+0000 7f085d873640 1 --2- 172.21.15.26:0/3749777161 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56166e1f8370 0x56166e1fbd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:19:20.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.486+0000 7f085e875640 1 --2- 172.21.15.26:0/3749777161 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56166e207f60 0x56166e1fc280 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x56166e1e5180 tx=0x7f085001bb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56ac73adcd0cd9cd server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:19:20.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.486+0000 7f0841ffb640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:19:20.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.486+0000 7f08626b1740 10 client.0 _ll_register_callbacks cb 0x7f08500093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:19:20.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.486+0000 7f08626b1740 1 -- 172.21.15.26:0/3749777161 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f08480318b0 con 0x56166e207f60 2024-09-11T23:19:20.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.486+0000 7f0841ffb640 1 -- 172.21.15.26:0/3749777161 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0850025040 con 0x56166e207f60 2024-09-11T23:19:20.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.486+0000 7f0841ffb640 1 -- 172.21.15.26:0/3749777161 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f0850004c00 con 0x56166e207f60 2024-09-11T23:19:20.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.486+0000 7f0841ffb640 1 -- 172.21.15.26:0/3749777161 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0850004e00 con 0x56166e207f60 2024-09-11T23:19:20.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.486+0000 7f08626b1740 10 client.15690 fetch_fsmap learned FSMap version 29 2024-09-11T23:19:20.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.486+0000 7f08626b1740 1 -- 172.21.15.26:0/3749777161 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({fsmap=29}) -- 0x56166e283770 con 0x56166e207f60 2024-09-11T23:19:20.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.487+0000 7f0841ffb640 1 -- 172.21.15.26:0/3749777161 <== mon.0 v2:172.21.15.26:3300/0 4 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f0850011500 con 0x56166e207f60 2024-09-11T23:19:20.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.487+0000 7f08626b1740 10 client.15690 fetch_fsmap finished waiting for FSMap version 29 2024-09-11T23:19:20.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.487+0000 7f08626b1740 10 client.15690 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:19:20.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.487+0000 7f08626b1740 20 client.15690 populate_metadata read hostname 'smithi026' 2024-09-11T23:19:20.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.487+0000 7f08626b1740 1 --2- 172.21.15.26:0/3749777161 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x56166e283ab0 0x56166e285ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:20.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.487+0000 7f08626b1740 4 client.15690 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:19:20.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.487+0000 7f08626b1740 1 -- 172.21.15.26:0/3749777161 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56166e15ad70 con 0x56166e283ab0 2024-09-11T23:19:20.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.487+0000 7f085d873640 1 --2- 172.21.15.26:0/3749777161 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x56166e283ab0 0x56166e285ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:20.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.487+0000 7f085d873640 1 --2- 172.21.15.26:0/3749777161 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x56166e283ab0 0x56166e285ea0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee8903be90895761 server_cookie=aee7c551a8fa7ea8 in_seq=0 out_seq=0 2024-09-11T23:19:20.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.488+0000 7f0841ffb640 10 client.15690 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:19:20.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.488+0000 7f0841ffb640 1 -- 172.21.15.26:0/3749777161 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56166e15ad70 con 0x56166e283ab0 2024-09-11T23:19:20.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.488+0000 7f0841ffb640 10 client.15690 handle_command_reply: tid=0 2024-09-11T23:19:20.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.488+0000 7f08626b1740 1 client.15690 shutdown 2024-09-11T23:19:20.494 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2024-09-11T23:19:20.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.488+0000 7f08626b1740 1 -- 172.21.15.26:0/3749777161 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56166e207f60 msgr2=0x56166e1fc280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:20.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.488+0000 7f08626b1740 1 --2- 172.21.15.26:0/3749777161 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56166e207f60 0x56166e1fc280 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x56166e1e5180 tx=0x7f085001bb40 comp rx=0 tx=0).stop 2024-09-11T23:19:20.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.488+0000 7f08626b1740 1 -- 172.21.15.26:0/3749777161 shutdown_connections 2024-09-11T23:19:20.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.488+0000 7f08626b1740 1 --2- 172.21.15.26:0/3749777161 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x56166e283ab0 0x56166e285ea0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:20.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.488+0000 7f08626b1740 1 --2- 172.21.15.26:0/3749777161 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56166e207f60 0x56166e1fc280 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:20.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.488+0000 7f08626b1740 1 --2- 172.21.15.26:0/3749777161 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56166e1f8370 0x56166e1fbd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:20.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.488+0000 7f08626b1740 1 --2- 172.21.15.26:0/3749777161 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56166e1ed5d0 0x56166e200a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:20.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.488+0000 7f08626b1740 1 -- 172.21.15.26:0/3749777161 >> 172.21.15.26:0/3749777161 conn(0x56166e1b6310 msgr2=0x56166e1f38e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:19:20.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.488+0000 7f0841ffb640 0 client.15690 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:19:20.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.488+0000 7f08626b1740 1 -- 172.21.15.26:0/3749777161 shutdown_connections 2024-09-11T23:19:20.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.488+0000 7f08626b1740 1 -- 172.21.15.26:0/3749777161 wait complete. 2024-09-11T23:19:20.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.488+0000 7f08626b1740 20 client.15690 trim_cache size 0 max 16384 2024-09-11T23:19:20.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.489+0000 7f08602d8640 1 -- 172.21.15.26:0/3773958062 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f08440a2e90 msgr2=0x7f08440a5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:20.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.489+0000 7f08602d8640 1 --2- 172.21.15.26:0/3773958062 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f08440a2e90 0x7f08440a5350 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f08540cc9e0 tx=0x7f08540d2eb0 comp rx=0 tx=0).stop 2024-09-11T23:19:20.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.489+0000 7f08602d8640 1 -- 172.21.15.26:0/3773958062 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0858101df0 msgr2=0x7f08580744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:20.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.489+0000 7f08602d8640 1 --2- 172.21.15.26:0/3773958062 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0858101df0 0x7f08580744f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f085400d720 tx=0x7f085400dbe0 comp rx=0 tx=0).stop 2024-09-11T23:19:20.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.489+0000 7f08602d8640 1 -- 172.21.15.26:0/3773958062 shutdown_connections 2024-09-11T23:19:20.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.489+0000 7f08602d8640 1 --2- 172.21.15.26:0/3773958062 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f08440a2e90 0x7f08440a5350 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:20.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.489+0000 7f08602d8640 1 --2- 172.21.15.26:0/3773958062 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0858106f20 0x7f0858074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:20.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.489+0000 7f08602d8640 1 --2- 172.21.15.26:0/3773958062 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f08581027a0 0x7f0858074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:20.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.489+0000 7f08602d8640 1 --2- 172.21.15.26:0/3773958062 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0858101df0 0x7f08580744f0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:20.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.489+0000 7f08602d8640 1 -- 172.21.15.26:0/3773958062 >> 172.21.15.26:0/3773958062 conn(0x7f08580fd8f0 msgr2=0x7f08580fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:19:20.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.490+0000 7f08602d8640 1 -- 172.21.15.26:0/3773958062 shutdown_connections 2024-09-11T23:19:20.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:20.490+0000 7f08602d8640 1 -- 172.21.15.26:0/3773958062 wait complete. 2024-09-11T23:19:22.237 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-11T23:19:22.237 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-11T23:19:22.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:19:21 smithi136 ceph-mon[35121]: pgmap v32: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 2.5 KiB/s wr, 10 op/s 2024-09-11T23:19:22.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:19:21 smithi026 ceph-mon[31172]: pgmap v32: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 2.5 KiB/s wr, 10 op/s 2024-09-11T23:19:22.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:19:21 smithi076 ceph-mon[35296]: pgmap v32: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 2.5 KiB/s wr, 10 op/s 2024-09-11T23:19:22.513 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:19:23.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.026+0000 7f4a1dab9640 1 -- 172.21.15.26:0/3321803815 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4a18102570 msgr2=0x7f4a181029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:23.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.026+0000 7f4a1dab9640 1 --2- 172.21.15.26:0/3321803815 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4a18102570 0x7f4a181029b0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f4a0c009900 tx=0x7f4a0c02a480 comp rx=0 tx=0).stop 2024-09-11T23:19:23.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.026+0000 7f4a1dab9640 1 -- 172.21.15.26:0/3321803815 shutdown_connections 2024-09-11T23:19:23.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.026+0000 7f4a1dab9640 1 --2- 172.21.15.26:0/3321803815 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4a18102ef0 0x7f4a1810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:23.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.026+0000 7f4a1dab9640 1 --2- 172.21.15.26:0/3321803815 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4a18102570 0x7f4a181029b0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:23.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.026+0000 7f4a1dab9640 1 --2- 172.21.15.26:0/3321803815 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4a18100e20 0x7f4a18101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:23.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.026+0000 7f4a1dab9640 1 -- 172.21.15.26:0/3321803815 >> 172.21.15.26:0/3321803815 conn(0x7f4a180fc7d0 msgr2=0x7f4a180febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:19:23.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.026+0000 7f4a1dab9640 1 -- 172.21.15.26:0/3321803815 shutdown_connections 2024-09-11T23:19:23.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.027+0000 7f4a1dab9640 1 -- 172.21.15.26:0/3321803815 wait complete. 2024-09-11T23:19:23.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.027+0000 7f4a1dab9640 1 Processor -- start 2024-09-11T23:19:23.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.028+0000 7f4a1dab9640 1 -- start start 2024-09-11T23:19:23.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.028+0000 7f4a1dab9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4a18100e20 0x7f4a181993e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:23.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.028+0000 7f4a1dab9640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4a18102570 0x7f4a18199920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:23.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.028+0000 7f4a1dab9640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4a18102ef0 0x7f4a181a4460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:23.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.028+0000 7f4a1dab9640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f4a18110f80 con 0x7f4a18100e20 2024-09-11T23:19:23.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.028+0000 7f4a1dab9640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f4a18110e00 con 0x7f4a18102ef0 2024-09-11T23:19:23.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.028+0000 7f4a1dab9640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f4a18111100 con 0x7f4a18102570 2024-09-11T23:19:23.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.028+0000 7f4a177fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4a18100e20 0x7f4a181993e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:23.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.028+0000 7f4a177fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4a18100e20 0x7f4a181993e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40612/0 (socket says 172.21.15.26:40612) 2024-09-11T23:19:23.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.028+0000 7f4a177fe640 1 -- 172.21.15.26:0/3987711717 learned_addr learned my addr 172.21.15.26:0/3987711717 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:19:23.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.028+0000 7f4a16ffd640 1 --2- 172.21.15.26:0/3987711717 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4a18102570 0x7f4a18199920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:23.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.029+0000 7f4a17fff640 1 --2- 172.21.15.26:0/3987711717 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4a18102ef0 0x7f4a181a4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:23.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.029+0000 7f4a16ffd640 1 -- 172.21.15.26:0/3987711717 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4a18102ef0 msgr2=0x7f4a181a4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:23.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.029+0000 7f4a16ffd640 1 --2- 172.21.15.26:0/3987711717 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4a18102ef0 0x7f4a181a4460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:23.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.029+0000 7f4a16ffd640 1 -- 172.21.15.26:0/3987711717 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4a18100e20 msgr2=0x7f4a181993e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:23.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.029+0000 7f4a16ffd640 1 --2- 172.21.15.26:0/3987711717 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4a18100e20 0x7f4a181993e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:23.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.029+0000 7f4a16ffd640 1 -- 172.21.15.26:0/3987711717 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a181a49a0 con 0x7f4a18102570 2024-09-11T23:19:23.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.029+0000 7f4a16ffd640 1 --2- 172.21.15.26:0/3987711717 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4a18102570 0x7f4a18199920 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f4a0c005940 tx=0x7f4a0c003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:19:23.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.029+0000 7f4a14ff9640 1 -- 172.21.15.26:0/3987711717 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a0c03f070 con 0x7f4a18102570 2024-09-11T23:19:23.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.029+0000 7f4a1dab9640 1 -- 172.21.15.26:0/3987711717 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a181a4b70 con 0x7f4a18102570 2024-09-11T23:19:23.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.030+0000 7f4a14ff9640 1 -- 172.21.15.26:0/3987711717 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a0c031c30 con 0x7f4a18102570 2024-09-11T23:19:23.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.030+0000 7f4a14ff9640 1 -- 172.21.15.26:0/3987711717 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a0c035030 con 0x7f4a18102570 2024-09-11T23:19:23.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.030+0000 7f4a1dab9640 1 -- 172.21.15.26:0/3987711717 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a181a5020 con 0x7f4a18102570 2024-09-11T23:19:23.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.033+0000 7f4a14ff9640 1 -- 172.21.15.26:0/3987711717 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 25) ==== 138066+0+0 (secure 0 0 0) 0x7f4a0c042070 con 0x7f4a18102570 2024-09-11T23:19:23.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.034+0000 7f4a1fe92740 1 Processor -- start 2024-09-11T23:19:23.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.034+0000 7f4a1fe92740 1 -- start start 2024-09-11T23:19:23.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.034+0000 7f4a1fe92740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5618600bea40 0x5618600bee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:23.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.034+0000 7f4a1fe92740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5618600bf360 0x5618600cdc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:23.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.034+0000 7f4a17fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5618600bea40 0x5618600bee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:23.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.034+0000 7f4a17fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5618600bea40 0x5618600bee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40624/0 (socket says 172.21.15.26:40624) 2024-09-11T23:19:23.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.034+0000 7f4a14ff9640 1 --2- 172.21.15.26:0/3987711717 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f4a000a2e10 0x7f4a000a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:23.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.034+0000 7f4a177fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5618600bf360 0x5618600cdc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:23.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.034+0000 7f4a177fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5618600bf360 0x5618600cdc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:60964/0 (socket says 172.21.15.26:60964) 2024-09-11T23:19:23.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.035+0000 7f4a14ff9640 1 -- 172.21.15.26:0/3987711717 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f4a0c0efdf0 con 0x7f4a18102570 2024-09-11T23:19:23.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.034+0000 7f4a1fe92740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5618600ce150 0x5618600d0540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:23.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.035+0000 7f4a1fe92740 1 -- 172.21.15.26:0/1482240435 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x561860024280 con 0x5618600bea40 2024-09-11T23:19:23.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.035+0000 7f4a1fe92740 1 -- 172.21.15.26:0/1482240435 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x56185ffcbb10 con 0x5618600bf360 2024-09-11T23:19:23.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.035+0000 7f4a17fff640 1 -- 172.21.15.26:0/1482240435 learned_addr learned my addr 172.21.15.26:0/1482240435 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:19:23.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.035+0000 7f4a1fe92740 1 -- 172.21.15.26:0/1482240435 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x561860020240 con 0x5618600ce150 2024-09-11T23:19:23.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.035+0000 7f4a17fff640 1 --2- 172.21.15.26:0/3987711717 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f4a000a2e10 0x7f4a000a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:23.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.035+0000 7f4a16ffd640 1 --2- 172.21.15.26:0/1482240435 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5618600ce150 0x5618600d0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:23.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.035+0000 7f4a17fff640 1 -- 172.21.15.26:0/1482240435 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5618600ce150 msgr2=0x5618600d0540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:19:23.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.035+0000 7f4a17fff640 1 --2- 172.21.15.26:0/1482240435 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5618600ce150 0x5618600d0540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:23.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.035+0000 7f4a17fff640 1 -- 172.21.15.26:0/1482240435 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5618600bf360 msgr2=0x5618600cdc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:19:23.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.035+0000 7f4a17fff640 1 --2- 172.21.15.26:0/1482240435 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5618600bf360 0x5618600cdc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:23.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.035+0000 7f4a17fff640 1 -- 172.21.15.26:0/1482240435 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56185ffd81f0 con 0x5618600bea40 2024-09-11T23:19:23.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.036+0000 7f4a17fff640 1 --2- 172.21.15.26:0/1482240435 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5618600bea40 0x5618600bee20 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f4a100139f0 tx=0x7f4a10013eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1d25f1f4d800552 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:19:23.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.036+0000 7f49fdffb640 1 -- 172.21.15.26:0/1482240435 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a10011e40 con 0x5618600bea40 2024-09-11T23:19:23.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.036+0000 7f49fdffb640 1 -- 172.21.15.26:0/1482240435 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a10022070 con 0x5618600bea40 2024-09-11T23:19:23.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.036+0000 7f49fdffb640 1 -- 172.21.15.26:0/1482240435 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a1001c800 con 0x5618600bea40 2024-09-11T23:19:23.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.037+0000 7f4a1fe92740 1 -- 172.21.15.26:0/1482240435 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5618600bea40 msgr2=0x5618600bee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:23.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.037+0000 7f4a17fff640 1 --2- 172.21.15.26:0/3987711717 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f4a000a2e10 0x7f4a000a52d0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f4a10021000 tx=0x7f4a1001ceb0 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:19:23.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.037+0000 7f4a1fe92740 1 --2- 172.21.15.26:0/1482240435 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5618600bea40 0x5618600bee20 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f4a100139f0 tx=0x7f4a10013eb0 comp rx=0 tx=0).stop 2024-09-11T23:19:23.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.037+0000 7f4a1fe92740 1 -- 172.21.15.26:0/1482240435 shutdown_connections 2024-09-11T23:19:23.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.037+0000 7f4a1fe92740 1 --2- 172.21.15.26:0/1482240435 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5618600bf360 0x5618600cdc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:23.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.037+0000 7f4a1fe92740 1 --2- 172.21.15.26:0/1482240435 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5618600ce150 0x5618600d0540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:23.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.037+0000 7f4a1fe92740 1 --2- 172.21.15.26:0/1482240435 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5618600bea40 0x5618600bee20 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:23.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.037+0000 7f4a1fe92740 1 -- 172.21.15.26:0/1482240435 >> 172.21.15.26:0/1482240435 conn(0x5618600bb0f0 msgr2=0x5618600bb4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:19:23.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.038+0000 7f4a1fe92740 1 -- 172.21.15.26:0/1482240435 shutdown_connections 2024-09-11T23:19:23.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.038+0000 7f4a1fe92740 1 -- 172.21.15.26:0/1482240435 wait complete. 2024-09-11T23:19:23.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.038+0000 7f4a1fe92740 1 -- start start 2024-09-11T23:19:23.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.038+0000 7f4a1fe92740 1 Processor -- start 2024-09-11T23:19:23.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.039+0000 7f4a1fe92740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5618600bea40 0x5618600cd910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:23.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.039+0000 7f4a1fe92740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5618600bf360 0x5618600c6cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:23.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.039+0000 7f4a1fe92740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5618600ce150 0x5618600c71f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:23.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.039+0000 7f4a1fe92740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5618600d0d20 con 0x5618600ce150 2024-09-11T23:19:23.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.039+0000 7f4a1fe92740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x56185ffdef10 con 0x5618600bea40 2024-09-11T23:19:23.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.039+0000 7f4a1fe92740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5618600d0ea0 con 0x5618600bf360 2024-09-11T23:19:23.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.039+0000 7f4a1fe92740 10 client.0 _ll_register_callbacks cb 0x56185ffa7a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:19:23.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.039+0000 7f4a17fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5618600ce150 0x5618600c71f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:23.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.039+0000 7f4a17fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5618600ce150 0x5618600c71f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40626/0 (socket says 172.21.15.26:40626) 2024-09-11T23:19:23.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.039+0000 7f4a17fff640 1 -- 172.21.15.26:0/1509843964 learned_addr learned my addr 172.21.15.26:0/1509843964 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:19:23.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.039+0000 7f4a177fe640 1 --2- 172.21.15.26:0/1509843964 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5618600bea40 0x5618600cd910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:23.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.039+0000 7f4a16ffd640 1 --2- 172.21.15.26:0/1509843964 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5618600bf360 0x5618600c6cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:23.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.040+0000 7f4a17fff640 1 -- 172.21.15.26:0/1509843964 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5618600bf360 msgr2=0x5618600c6cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:19:23.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.040+0000 7f4a17fff640 1 --2- 172.21.15.26:0/1509843964 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5618600bf360 0x5618600c6cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:23.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.040+0000 7f4a17fff640 1 -- 172.21.15.26:0/1509843964 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5618600bea40 msgr2=0x5618600cd910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:19:23.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.040+0000 7f4a17fff640 1 --2- 172.21.15.26:0/1509843964 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5618600bea40 0x5618600cd910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:23.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.040+0000 7f4a17fff640 1 -- 172.21.15.26:0/1509843964 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56185fd956a0 con 0x5618600ce150 2024-09-11T23:19:23.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.040+0000 7f4a16ffd640 1 --2- 172.21.15.26:0/1509843964 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5618600bf360 0x5618600c6cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:19:23.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.040+0000 7f4a17fff640 1 --2- 172.21.15.26:0/1509843964 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5618600ce150 0x5618600c71f0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f4a10005cc0 tx=0x7f4a10009940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=382baad1950951f9 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:19:23.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.040+0000 7f49f6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:19:23.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.040+0000 7f49f6ffd640 1 -- 172.21.15.26:0/1509843964 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a10035060 con 0x5618600ce150 2024-09-11T23:19:23.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.040+0000 7f49f6ffd640 1 -- 172.21.15.26:0/1509843964 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f4a10013250 con 0x5618600ce150 2024-09-11T23:19:23.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.040+0000 7f4a1fe92740 1 -- 172.21.15.26:0/1509843964 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a10022070 con 0x5618600ce150 2024-09-11T23:19:23.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.040+0000 7f49f6ffd640 1 -- 172.21.15.26:0/1509843964 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4a1001c500 con 0x5618600ce150 2024-09-11T23:19:23.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.040+0000 7f4a1fe92740 10 client.15702 fetch_fsmap learned FSMap version 29 2024-09-11T23:19:23.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.040+0000 7f4a1fe92740 1 -- 172.21.15.26:0/1509843964 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5618600cc2a0 con 0x5618600ce150 2024-09-11T23:19:23.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.041+0000 7f49f6ffd640 1 -- 172.21.15.26:0/1509843964 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a10003e50 con 0x5618600ce150 2024-09-11T23:19:23.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.041+0000 7f49f6ffd640 1 -- 172.21.15.26:0/1509843964 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f4a1000d8e0 con 0x5618600ce150 2024-09-11T23:19:23.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.041+0000 7f4a1fe92740 10 client.15702 fetch_fsmap finished waiting for FSMap version 29 2024-09-11T23:19:23.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.041+0000 7f4a1fe92740 10 client.15702 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:19:23.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.041+0000 7f4a1fe92740 20 client.15702 populate_metadata read hostname 'smithi026' 2024-09-11T23:19:23.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.041+0000 7f4a1fe92740 1 --2- 172.21.15.26:0/1509843964 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5618600e2290 0x5618600e4680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:23.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.041+0000 7f4a1fe92740 4 client.15702 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:19:23.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.041+0000 7f4a1fe92740 1 -- 172.21.15.26:0/1509843964 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f4a10022070 con 0x5618600e2290 2024-09-11T23:19:23.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.041+0000 7f4a177fe640 1 --2- 172.21.15.26:0/1509843964 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5618600e2290 0x5618600e4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:23.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.042+0000 7f49f6ffd640 10 client.15702 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:19:23.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.042+0000 7f4a177fe640 1 --2- 172.21.15.26:0/1509843964 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5618600e2290 0x5618600e4680 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a915cbb8878f0776 server_cookie=5f5307038a987b89 in_seq=0 out_seq=0 2024-09-11T23:19:23.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.043+0000 7f49f6ffd640 1 -- 172.21.15.26:0/1509843964 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f4a10022070 con 0x5618600e2290 2024-09-11T23:19:23.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.043+0000 7f49f6ffd640 10 client.15702 handle_command_reply: tid=0 2024-09-11T23:19:23.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.043+0000 7f4a1fe92740 1 client.15702 shutdown 2024-09-11T23:19:23.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.043+0000 7f4a1fe92740 1 -- 172.21.15.26:0/1509843964 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5618600ce150 msgr2=0x5618600c71f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:23.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.043+0000 7f4a1fe92740 1 --2- 172.21.15.26:0/1509843964 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5618600ce150 0x5618600c71f0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f4a10005cc0 tx=0x7f4a10009940 comp rx=0 tx=0).stop 2024-09-11T23:19:23.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.044+0000 7f4a1fe92740 1 -- 172.21.15.26:0/1509843964 shutdown_connections 2024-09-11T23:19:23.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.044+0000 7f4a1fe92740 1 --2- 172.21.15.26:0/1509843964 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5618600e2290 0x5618600e4680 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:23.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.044+0000 7f49f6ffd640 0 client.15702 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:19:23.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.044+0000 7f4a1fe92740 1 --2- 172.21.15.26:0/1509843964 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5618600ce150 0x5618600c71f0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:23.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.044+0000 7f4a1fe92740 1 --2- 172.21.15.26:0/1509843964 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5618600bf360 0x5618600c6cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:23.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.044+0000 7f4a1fe92740 1 --2- 172.21.15.26:0/1509843964 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5618600bea40 0x5618600cd910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:23.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.044+0000 7f4a1fe92740 1 -- 172.21.15.26:0/1509843964 >> 172.21.15.26:0/1509843964 conn(0x5618600bb0f0 msgr2=0x5618600c0e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:19:23.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.044+0000 7f4a1fe92740 1 -- 172.21.15.26:0/1509843964 shutdown_connections 2024-09-11T23:19:23.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.044+0000 7f4a1fe92740 1 -- 172.21.15.26:0/1509843964 wait complete. 2024-09-11T23:19:23.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.044+0000 7f4a1fe92740 20 client.15702 trim_cache size 0 max 16384 2024-09-11T23:19:23.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.061+0000 7f4a1fe92740 1 Processor -- start 2024-09-11T23:19:23.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.061+0000 7f4a1fe92740 1 -- start start 2024-09-11T23:19:23.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.062+0000 7f4a1fe92740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5618600bea40 0x5618600e9ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:23.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.062+0000 7f4a1fe92740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5618600bf360 0x5618600ea220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:23.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.062+0000 7f4a1fe92740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5618600ce150 0x5618600c3f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:23.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.062+0000 7f4a1fe92740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f4a181ab620 con 0x5618600bea40 2024-09-11T23:19:23.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.062+0000 7f4a1fe92740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f4a181ad160 con 0x5618600ce150 2024-09-11T23:19:23.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.062+0000 7f4a1fe92740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f4a181abcf0 con 0x5618600bf360 2024-09-11T23:19:23.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.062+0000 7f4a177fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5618600bea40 0x5618600e9ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:23.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.062+0000 7f4a177fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5618600bea40 0x5618600e9ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40632/0 (socket says 172.21.15.26:40632) 2024-09-11T23:19:23.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.062+0000 7f4a177fe640 1 -- 172.21.15.26:0/797222176 learned_addr learned my addr 172.21.15.26:0/797222176 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:19:23.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.062+0000 7f4a177fe640 1 -- 172.21.15.26:0/797222176 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5618600bf360 msgr2=0x5618600ea220 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T23:19:23.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.062+0000 7f4a17fff640 1 --2- 172.21.15.26:0/797222176 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5618600ce150 0x5618600c3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:23.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.062+0000 7f4a177fe640 1 --2- 172.21.15.26:0/797222176 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5618600bf360 0x5618600ea220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:23.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.062+0000 7f4a177fe640 1 -- 172.21.15.26:0/797222176 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5618600ce150 msgr2=0x5618600c3f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:19:23.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.062+0000 7f4a177fe640 1 --2- 172.21.15.26:0/797222176 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5618600ce150 0x5618600c3f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:23.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.062+0000 7f4a177fe640 1 -- 172.21.15.26:0/797222176 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5618600c7ff0 con 0x5618600bea40 2024-09-11T23:19:23.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.062+0000 7f4a17fff640 1 --2- 172.21.15.26:0/797222176 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5618600ce150 0x5618600c3f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:19:23.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.063+0000 7f4a177fe640 1 --2- 172.21.15.26:0/797222176 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5618600bea40 0x5618600e9ce0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f4a0800bc40 tx=0x7f4a0800bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1f682d716389ae2 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:19:23.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.063+0000 7f49fd7fa640 1 -- 172.21.15.26:0/797222176 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a08005020 con 0x5618600bea40 2024-09-11T23:19:23.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.063+0000 7f49fd7fa640 1 -- 172.21.15.26:0/797222176 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a080047a0 con 0x5618600bea40 2024-09-11T23:19:23.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.063+0000 7f49fd7fa640 1 -- 172.21.15.26:0/797222176 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a0801d850 con 0x5618600bea40 2024-09-11T23:19:23.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.064+0000 7f4a1fe92740 1 -- 172.21.15.26:0/797222176 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5618600bea40 msgr2=0x5618600e9ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:23.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.064+0000 7f4a1fe92740 1 --2- 172.21.15.26:0/797222176 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5618600bea40 0x5618600e9ce0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f4a0800bc40 tx=0x7f4a0800bf90 comp rx=0 tx=0).stop 2024-09-11T23:19:23.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.064+0000 7f4a1fe92740 1 -- 172.21.15.26:0/797222176 shutdown_connections 2024-09-11T23:19:23.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.064+0000 7f4a1fe92740 1 --2- 172.21.15.26:0/797222176 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5618600ce150 0x5618600c3f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:23.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.064+0000 7f4a1fe92740 1 --2- 172.21.15.26:0/797222176 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5618600bf360 0x5618600ea220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:23.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.064+0000 7f4a1fe92740 1 --2- 172.21.15.26:0/797222176 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5618600bea40 0x5618600e9ce0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:23.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.064+0000 7f4a1fe92740 1 -- 172.21.15.26:0/797222176 >> 172.21.15.26:0/797222176 conn(0x5618600bb0f0 msgr2=0x5618600c0dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:19:23.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.064+0000 7f4a1fe92740 1 -- 172.21.15.26:0/797222176 shutdown_connections 2024-09-11T23:19:23.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.064+0000 7f4a1fe92740 1 -- 172.21.15.26:0/797222176 wait complete. 2024-09-11T23:19:23.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.065+0000 7f4a1fe92740 1 -- start start 2024-09-11T23:19:23.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.065+0000 7f4a1fe92740 1 Processor -- start 2024-09-11T23:19:23.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.066+0000 7f4a1fe92740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5618600bea40 0x5618600daf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:23.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.066+0000 7f4a1fe92740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5618600bf360 0x5618600db460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:23.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.066+0000 7f4a1fe92740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5618600ce150 0x56186014bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:23.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.066+0000 7f4a177fe640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5618600bea40 0x5618600daf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:23.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.066+0000 7f4a177fe640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5618600bea40 0x5618600daf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:49666/0 (socket says 172.21.15.26:49666) 2024-09-11T23:19:23.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.066+0000 7f4a17fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5618600ce150 0x56186014bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:23.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.066+0000 7f4a17fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5618600ce150 0x56186014bcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40638/0 (socket says 172.21.15.26:40638) 2024-09-11T23:19:23.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.066+0000 7f4a177fe640 1 -- 172.21.15.26:0/863074031 learned_addr learned my addr 172.21.15.26:0/863074031 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:19:23.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.066+0000 7f4a16ffd640 1 --2- 172.21.15.26:0/863074031 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5618600bf360 0x5618600db460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:23.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.066+0000 7f4a1fe92740 1 -- 172.21.15.26:0/863074031 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f4a181aca90 con 0x5618600ce150 2024-09-11T23:19:23.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.066+0000 7f4a1fe92740 1 -- 172.21.15.26:0/863074031 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f4a181ac3c0 con 0x5618600bf360 2024-09-11T23:19:23.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.066+0000 7f4a1fe92740 1 -- 172.21.15.26:0/863074031 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f4a181a9620 con 0x5618600bea40 2024-09-11T23:19:23.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.066+0000 7f4a16ffd640 1 -- 172.21.15.26:0/863074031 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5618600bea40 msgr2=0x5618600daf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:19:23.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.066+0000 7f4a16ffd640 1 --2- 172.21.15.26:0/863074031 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5618600bea40 0x5618600daf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:23.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.066+0000 7f4a16ffd640 1 -- 172.21.15.26:0/863074031 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5618600ce150 msgr2=0x56186014bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:19:23.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.066+0000 7f4a16ffd640 1 --2- 172.21.15.26:0/863074031 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5618600ce150 0x56186014bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:23.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.066+0000 7f4a16ffd640 1 -- 172.21.15.26:0/863074031 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56186014c1f0 con 0x5618600bf360 2024-09-11T23:19:23.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.067+0000 7f4a1fe92740 10 client.0 _ll_register_callbacks cb 0x7f4a100137f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:19:23.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.067+0000 7f4a1fe92740 1 -- 172.21.15.26:0/863074031 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a080047a0 con 0x5618600bf360 2024-09-11T23:19:23.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.067+0000 7f4a16ffd640 1 --2- 172.21.15.26:0/863074031 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5618600bf360 0x5618600db460 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f4a0c0ed370 tx=0x7f4a0c0045e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=29f024820219c1e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:19:23.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.067+0000 7f49f6ffd640 10 client.25301 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-11T23:19:23.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.067+0000 7f49f6ffd640 1 -- 172.21.15.26:0/863074031 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a0c0d4070 con 0x5618600bf360 2024-09-11T23:19:23.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.067+0000 7f49f6ffd640 1 -- 172.21.15.26:0/863074031 <== mon.2 v2:172.21.15.76:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f4a0c03b4f0 con 0x5618600bf360 2024-09-11T23:19:23.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.067+0000 7f49f6ffd640 1 -- 172.21.15.26:0/863074031 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4a0c049050 con 0x5618600bf360 2024-09-11T23:19:23.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.067+0000 7f4a1fe92740 10 client.25301 fetch_fsmap learned FSMap version 29 2024-09-11T23:19:23.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.067+0000 7f4a1fe92740 1 -- 172.21.15.26:0/863074031 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({fsmap=29}) -- 0x56186014d2b0 con 0x5618600bf360 2024-09-11T23:19:23.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.068+0000 7f49f6ffd640 1 -- 172.21.15.26:0/863074031 <== mon.2 v2:172.21.15.76:3300/0 4 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f4a0c0c4920 con 0x5618600bf360 2024-09-11T23:19:23.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.068+0000 7f4a1fe92740 10 client.25301 fetch_fsmap finished waiting for FSMap version 29 2024-09-11T23:19:23.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.068+0000 7f4a1fe92740 10 client.25301 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:19:23.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.068+0000 7f4a1fe92740 20 client.25301 populate_metadata read hostname 'smithi026' 2024-09-11T23:19:23.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.068+0000 7f4a1fe92740 1 --2- 172.21.15.26:0/863074031 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5618600e2290 0x56186014f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:23.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.068+0000 7f4a1fe92740 4 client.25301 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-11T23:19:23.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.068+0000 7f4a1fe92740 1 -- 172.21.15.26:0/863074031 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5618600317f0 con 0x5618600e2290 2024-09-11T23:19:23.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.068+0000 7f4a177fe640 1 --2- 172.21.15.26:0/863074031 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5618600e2290 0x56186014f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:23.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.068+0000 7f4a177fe640 1 --2- 172.21.15.26:0/863074031 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5618600e2290 0x56186014f9d0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e01e2d6d39fba088 server_cookie=636a0ac5002b9a2a in_seq=0 out_seq=0 2024-09-11T23:19:23.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.069+0000 7f49f6ffd640 10 client.25301 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:19:23.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.070+0000 7f49f6ffd640 1 -- 172.21.15.26:0/863074031 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5618600317f0 con 0x5618600e2290 2024-09-11T23:19:23.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.070+0000 7f49f6ffd640 10 client.25301 handle_command_reply: tid=0 2024-09-11T23:19:23.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.070+0000 7f4a1fe92740 1 client.25301 shutdown 2024-09-11T23:19:23.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.070+0000 7f4a1fe92740 1 -- 172.21.15.26:0/863074031 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5618600bf360 msgr2=0x5618600db460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:23.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.070+0000 7f4a1fe92740 1 --2- 172.21.15.26:0/863074031 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5618600bf360 0x5618600db460 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f4a0c0ed370 tx=0x7f4a0c0045e0 comp rx=0 tx=0).stop 2024-09-11T23:19:23.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.070+0000 7f4a1fe92740 1 -- 172.21.15.26:0/863074031 shutdown_connections 2024-09-11T23:19:23.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.070+0000 7f4a1fe92740 1 --2- 172.21.15.26:0/863074031 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5618600bf360 0x5618600db460 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:23.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.070+0000 7f4a1fe92740 1 --2- 172.21.15.26:0/863074031 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5618600e2290 0x56186014f9d0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:23.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.070+0000 7f49f6ffd640 0 client.25301 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:19:23.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.070+0000 7f4a1fe92740 1 --2- 172.21.15.26:0/863074031 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5618600ce150 0x56186014bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:23.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.070+0000 7f4a1fe92740 1 --2- 172.21.15.26:0/863074031 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5618600bea40 0x5618600daf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:23.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.070+0000 7f4a1fe92740 1 -- 172.21.15.26:0/863074031 >> 172.21.15.26:0/863074031 conn(0x5618600bb0f0 msgr2=0x5618600cc520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:19:23.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.070+0000 7f4a1fe92740 1 -- 172.21.15.26:0/863074031 shutdown_connections 2024-09-11T23:19:23.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.070+0000 7f4a1fe92740 1 -- 172.21.15.26:0/863074031 wait complete. 2024-09-11T23:19:23.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.071+0000 7f4a1fe92740 20 client.25301 trim_cache size 0 max 16384 2024-09-11T23:19:23.075 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-11T23:19:23.075 INFO:teuthology.orchestra.run.smithi026.stdout: "return_code": 0, 2024-09-11T23:19:23.075 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_tag": "6adeb6f6-995b-4377-9a31-baeabff7f707", 2024-09-11T23:19:23.075 INFO:teuthology.orchestra.run.smithi026.stdout: "mode": "asynchronous" 2024-09-11T23:19:23.075 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-11T23:19:23.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.071+0000 7f4a1dab9640 1 -- 172.21.15.26:0/3987711717 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f4a000a2e10 msgr2=0x7f4a000a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:23.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.071+0000 7f4a1dab9640 1 --2- 172.21.15.26:0/3987711717 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f4a000a2e10 0x7f4a000a52d0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f4a10021000 tx=0x7f4a1001ceb0 comp rx=0 tx=0).stop 2024-09-11T23:19:23.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.071+0000 7f4a1dab9640 1 -- 172.21.15.26:0/3987711717 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4a18102570 msgr2=0x7f4a18199920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:23.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.071+0000 7f4a1dab9640 1 --2- 172.21.15.26:0/3987711717 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4a18102570 0x7f4a18199920 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f4a0c005940 tx=0x7f4a0c003c10 comp rx=0 tx=0).stop 2024-09-11T23:19:23.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.072+0000 7f4a1dab9640 1 -- 172.21.15.26:0/3987711717 shutdown_connections 2024-09-11T23:19:23.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.072+0000 7f4a1dab9640 1 --2- 172.21.15.26:0/3987711717 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f4a000a2e10 0x7f4a000a52d0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:23.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.072+0000 7f4a1dab9640 1 --2- 172.21.15.26:0/3987711717 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4a18102ef0 0x7f4a181a4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:23.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.072+0000 7f4a1dab9640 1 --2- 172.21.15.26:0/3987711717 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4a18102570 0x7f4a18199920 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:23.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.072+0000 7f4a1dab9640 1 --2- 172.21.15.26:0/3987711717 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4a18100e20 0x7f4a181993e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:23.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.072+0000 7f4a1dab9640 1 -- 172.21.15.26:0/3987711717 >> 172.21.15.26:0/3987711717 conn(0x7f4a180fc7d0 msgr2=0x7f4a180febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:19:23.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.072+0000 7f4a1dab9640 1 -- 172.21.15.26:0/3987711717 shutdown_connections 2024-09-11T23:19:23.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:23.072+0000 7f4a1dab9640 1 -- 172.21.15.26:0/3987711717 wait complete. 2024-09-11T23:19:23.790 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T23:19:24.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:19:23 smithi026 ceph-mon[31172]: pgmap v33: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 2.5 KiB/s wr, 10 op/s 2024-09-11T23:19:24.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:19:23 smithi026 ceph-mon[31172]: scrub queued for path: / 2024-09-11T23:19:24.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:19:23 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:19:24.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:19:23 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:19:24.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:19:23 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:19:24.117 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:19:24.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:19:23 smithi136 ceph-mon[35121]: pgmap v33: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 2.5 KiB/s wr, 10 op/s 2024-09-11T23:19:24.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:19:23 smithi136 ceph-mon[35121]: scrub queued for path: / 2024-09-11T23:19:24.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:19:23 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:19:24.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:19:23 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:19:24.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:19:23 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:19:24.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:19:23 smithi076 ceph-mon[35296]: pgmap v33: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 2.5 KiB/s wr, 10 op/s 2024-09-11T23:19:24.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:19:23 smithi076 ceph-mon[35296]: scrub queued for path: / 2024-09-11T23:19:24.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:19:23 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:19:24.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:19:23 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:19:24.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:19:23 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:19:24.633 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.631+0000 7f337175b640 1 -- 172.21.15.26:0/2878286526 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3364099000 msgr2=0x7f33640a0b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:24.633 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.631+0000 7f337175b640 1 --2- 172.21.15.26:0/2878286526 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3364099000 0x7f33640a0b50 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f336c050280 tx=0x7f336c097cc0 comp rx=0 tx=0).stop 2024-09-11T23:19:24.633 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.632+0000 7f337175b640 1 -- 172.21.15.26:0/2878286526 shutdown_connections 2024-09-11T23:19:24.633 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.632+0000 7f337175b640 1 --2- 172.21.15.26:0/2878286526 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3364099000 0x7f33640a0b50 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:24.633 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.632+0000 7f337175b640 1 --2- 172.21.15.26:0/2878286526 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3364094cc0 0x7f33640988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:24.633 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.632+0000 7f337175b640 1 --2- 172.21.15.26:0/2878286526 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3364094310 0x7f33640946f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:24.633 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.632+0000 7f337175b640 1 -- 172.21.15.26:0/2878286526 >> 172.21.15.26:0/2878286526 conn(0x7f3364009990 msgr2=0x7f3364009da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:19:24.633 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.632+0000 7f337175b640 1 -- 172.21.15.26:0/2878286526 shutdown_connections 2024-09-11T23:19:24.634 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.632+0000 7f337175b640 1 -- 172.21.15.26:0/2878286526 wait complete. 2024-09-11T23:19:24.634 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.633+0000 7f337175b640 1 Processor -- start 2024-09-11T23:19:24.634 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.633+0000 7f337175b640 1 -- start start 2024-09-11T23:19:24.635 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.634+0000 7f337175b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3364094310 0x7f336412f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:24.635 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.634+0000 7f337175b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3364094cc0 0x7f336412fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:24.635 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.634+0000 7f337175b640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3364099000 0x7f33641298b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:24.635 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.634+0000 7f337175b640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f33640a3700 con 0x7f3364094cc0 2024-09-11T23:19:24.635 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.634+0000 7f336bfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3364094310 0x7f336412f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:24.636 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.634+0000 7f336bfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3364094310 0x7f336412f700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:32776/0 (socket says 172.21.15.26:32776) 2024-09-11T23:19:24.636 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.634+0000 7f336bfff640 1 -- 172.21.15.26:0/1378006403 learned_addr learned my addr 172.21.15.26:0/1378006403 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:19:24.636 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.634+0000 7f3370f5a640 1 --2- 172.21.15.26:0/1378006403 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3364099000 0x7f33641298b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:24.636 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.634+0000 7f337175b640 1 -- 172.21.15.26:0/1378006403 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f33640a3580 con 0x7f3364094310 2024-09-11T23:19:24.636 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.634+0000 7f337175b640 1 -- 172.21.15.26:0/1378006403 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f33640a3880 con 0x7f3364099000 2024-09-11T23:19:24.636 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.634+0000 7f336b7fe640 1 --2- 172.21.15.26:0/1378006403 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3364094cc0 0x7f336412fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:24.637 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.635+0000 7f3370f5a640 1 -- 172.21.15.26:0/1378006403 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3364094310 msgr2=0x7f336412f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:24.637 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.635+0000 7f3370f5a640 1 --2- 172.21.15.26:0/1378006403 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3364094310 0x7f336412f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:24.637 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.635+0000 7f3370f5a640 1 -- 172.21.15.26:0/1378006403 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3364094cc0 msgr2=0x7f336412fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:24.637 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.635+0000 7f3370f5a640 1 --2- 172.21.15.26:0/1378006403 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3364094cc0 0x7f336412fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:24.637 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.635+0000 7f3370f5a640 1 -- 172.21.15.26:0/1378006403 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f336412a030 con 0x7f3364099000 2024-09-11T23:19:24.637 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.635+0000 7f3370f5a640 1 --2- 172.21.15.26:0/1378006403 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3364099000 0x7f33641298b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f336c000b60 tx=0x7f336c06ba50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:19:24.638 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.635+0000 7f33697fa640 1 -- 172.21.15.26:0/1378006403 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f336c098a80 con 0x7f3364099000 2024-09-11T23:19:24.638 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.635+0000 7f33697fa640 1 -- 172.21.15.26:0/1378006403 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f336c063260 con 0x7f3364099000 2024-09-11T23:19:24.638 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.635+0000 7f337175b640 1 -- 172.21.15.26:0/1378006403 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f336412a2c0 con 0x7f3364099000 2024-09-11T23:19:24.638 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.635+0000 7f33697fa640 1 -- 172.21.15.26:0/1378006403 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f336c0a0650 con 0x7f3364099000 2024-09-11T23:19:24.638 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.636+0000 7f337175b640 1 -- 172.21.15.26:0/1378006403 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33641365c0 con 0x7f3364099000 2024-09-11T23:19:24.640 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.639+0000 7f3375835740 1 Processor -- start 2024-09-11T23:19:24.641 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.641+0000 7f33697fa640 1 -- 172.21.15.26:0/1378006403 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 25) ==== 138066+0+0 (secure 0 0 0) 0x7f336c062630 con 0x7f3364099000 2024-09-11T23:19:24.642 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.641+0000 7f3375835740 1 -- start start 2024-09-11T23:19:24.642 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.641+0000 7f3375835740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a9cc1c6cc0 0x55a9cc1c70a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:24.642 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.641+0000 7f3375835740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a9cc1c75e0 0x55a9cc1d1940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:24.642 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.641+0000 7f3375835740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a9cc1d1e80 0x55a9cc1d4270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:24.642 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.641+0000 7f3375835740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55a9cc12b2f0 con 0x55a9cc1d1e80 2024-09-11T23:19:24.643 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.641+0000 7f3375835740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55a9cc0d3270 con 0x55a9cc1c6cc0 2024-09-11T23:19:24.643 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.641+0000 7f3375835740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55a9cc1272b0 con 0x55a9cc1c75e0 2024-09-11T23:19:24.644 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.642+0000 7f33697fa640 1 --2- 172.21.15.26:0/1378006403 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f335c0a2ea0 0x7f335c0a5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:24.644 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.642+0000 7f33697fa640 1 -- 172.21.15.26:0/1378006403 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f336c15ae70 con 0x7f3364099000 2024-09-11T23:19:24.644 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.643+0000 7f336b7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a9cc1c6cc0 0x55a9cc1c70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:24.644 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.643+0000 7f336b7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a9cc1c6cc0 0x55a9cc1c70a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:32792/0 (socket says 172.21.15.26:32792) 2024-09-11T23:19:24.644 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.643+0000 7f336b7fe640 1 -- 172.21.15.26:0/1816864504 learned_addr learned my addr 172.21.15.26:0/1816864504 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:19:24.644 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.643+0000 7f336bfff640 1 --2- 172.21.15.26:0/1816864504 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a9cc1c75e0 0x55a9cc1d1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:24.644 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.643+0000 7f3370f5a640 1 --2- 172.21.15.26:0/1378006403 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f335c0a2ea0 0x7f335c0a5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:24.645 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.643+0000 7f336bfff640 1 -- 172.21.15.26:0/1816864504 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a9cc1c6cc0 msgr2=0x55a9cc1c70a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:19:24.645 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.643+0000 7f336bfff640 1 --2- 172.21.15.26:0/1816864504 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a9cc1c6cc0 0x55a9cc1c70a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:24.645 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.643+0000 7f336bfff640 1 -- 172.21.15.26:0/1816864504 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a9cc1d1e80 msgr2=0x55a9cc1d4270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:19:24.645 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.643+0000 7f336bfff640 1 --2- 172.21.15.26:0/1816864504 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a9cc1d1e80 0x55a9cc1d4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:24.645 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.643+0000 7f336bfff640 1 -- 172.21.15.26:0/1816864504 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a9cc0d2340 con 0x55a9cc1c75e0 2024-09-11T23:19:24.645 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.644+0000 7f336bfff640 1 --2- 172.21.15.26:0/1816864504 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a9cc1c75e0 0x55a9cc1d1940 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3360010ec0 tx=0x7f3360005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f4364bd8608582b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:19:24.645 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.644+0000 7f3370f5a640 1 --2- 172.21.15.26:0/1378006403 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f335c0a2ea0 0x7f335c0a5360 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f336c158560 tx=0x7f336c068050 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:19:24.645 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.644+0000 7f33567fc640 1 -- 172.21.15.26:0/1816864504 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f336001e070 con 0x55a9cc1c75e0 2024-09-11T23:19:24.646 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.644+0000 7f33567fc640 1 -- 172.21.15.26:0/1816864504 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3360019920 con 0x55a9cc1c75e0 2024-09-11T23:19:24.646 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.644+0000 7f33567fc640 1 -- 172.21.15.26:0/1816864504 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f336000fa60 con 0x55a9cc1c75e0 2024-09-11T23:19:24.646 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.645+0000 7f3375835740 1 -- 172.21.15.26:0/1816864504 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a9cc1c75e0 msgr2=0x55a9cc1d1940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:24.646 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.645+0000 7f3375835740 1 --2- 172.21.15.26:0/1816864504 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a9cc1c75e0 0x55a9cc1d1940 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3360010ec0 tx=0x7f3360005f90 comp rx=0 tx=0).stop 2024-09-11T23:19:24.646 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.645+0000 7f3375835740 1 -- 172.21.15.26:0/1816864504 shutdown_connections 2024-09-11T23:19:24.646 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.645+0000 7f3375835740 1 --2- 172.21.15.26:0/1816864504 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a9cc1d1e80 0x55a9cc1d4270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:24.647 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.645+0000 7f3375835740 1 --2- 172.21.15.26:0/1816864504 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a9cc1c75e0 0x55a9cc1d1940 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:24.647 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.645+0000 7f3375835740 1 --2- 172.21.15.26:0/1816864504 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a9cc1c6cc0 0x55a9cc1c70a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:24.647 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.645+0000 7f3375835740 1 -- 172.21.15.26:0/1816864504 >> 172.21.15.26:0/1816864504 conn(0x55a9cc1c3230 msgr2=0x55a9cc1c3610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:19:24.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.645+0000 7f1ae9575640 1 -- 172.21.15.26:0/4046282018 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ae4102570 msgr2=0x7f1ae41029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:24.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.645+0000 7f1ae9575640 1 --2- 172.21.15.26:0/4046282018 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ae4102570 0x7f1ae41029b0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f1ad8009900 tx=0x7f1ad802a4d0 comp rx=0 tx=0).stop 2024-09-11T23:19:24.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.645+0000 7f1ae9575640 1 -- 172.21.15.26:0/4046282018 shutdown_connections 2024-09-11T23:19:24.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.645+0000 7f1ae9575640 1 --2- 172.21.15.26:0/4046282018 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1ae4102ef0 0x7f1ae410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:24.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.645+0000 7f1ae9575640 1 --2- 172.21.15.26:0/4046282018 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ae4102570 0x7f1ae41029b0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:24.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.645+0000 7f1ae9575640 1 --2- 172.21.15.26:0/4046282018 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1ae4100e20 0x7f1ae4101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:24.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.645+0000 7f1ae9575640 1 -- 172.21.15.26:0/4046282018 >> 172.21.15.26:0/4046282018 conn(0x7f1ae40fc7d0 msgr2=0x7f1ae40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:19:24.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.646+0000 7f1ae9575640 1 -- 172.21.15.26:0/4046282018 shutdown_connections 2024-09-11T23:19:24.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.646+0000 7f1ae9575640 1 -- 172.21.15.26:0/4046282018 wait complete. 2024-09-11T23:19:24.648 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.646+0000 7f3375835740 1 -- 172.21.15.26:0/1816864504 shutdown_connections 2024-09-11T23:19:24.649 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.646+0000 7f3375835740 1 -- 172.21.15.26:0/1816864504 wait complete. 2024-09-11T23:19:24.649 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.647+0000 7f3375835740 1 -- start start 2024-09-11T23:19:24.649 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.647+0000 7f3375835740 1 Processor -- start 2024-09-11T23:19:24.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.646+0000 7f1ae9575640 1 Processor -- start 2024-09-11T23:19:24.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.647+0000 7f1ae9575640 1 -- start start 2024-09-11T23:19:24.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.647+0000 7f1ae9575640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1ae4100e20 0x7f1ae410e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:24.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.647+0000 7f1ae9575640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1ae4102ef0 0x7f1ae410e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:24.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.647+0000 7f1ae9575640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ae410fe60 0x7f1ae410ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:24.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.647+0000 7f1ae9575640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f1ae41120e0 con 0x7f1ae410fe60 2024-09-11T23:19:24.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.647+0000 7f1ae9575640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f1ae4111f60 con 0x7f1ae4102ef0 2024-09-11T23:19:24.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.647+0000 7f1ae9575640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f1ae4112260 con 0x7f1ae4100e20 2024-09-11T23:19:24.651 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.647+0000 7f3375835740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a9cc1c6cc0 0x55a9cc1ced50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:24.651 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.647+0000 7f3375835740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a9cc1c75e0 0x55a9cc1cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:24.651 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.648+0000 7f3375835740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a9cc1d1e80 0x55a9cc1cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:24.651 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.648+0000 7f3375835740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55a9cc1dd260 con 0x55a9cc1c75e0 2024-09-11T23:19:24.651 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.648+0000 7f3375835740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55a9cc0b4290 con 0x55a9cc1c6cc0 2024-09-11T23:19:24.651 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.648+0000 7f3375835740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55a9cc1dd3e0 con 0x55a9cc1d1e80 2024-09-11T23:19:24.651 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.648+0000 7f3375835740 10 client.0 _ll_register_callbacks cb 0x55a9cc0e5140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:19:24.651 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.648+0000 7f336bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a9cc1c75e0 0x55a9cc1cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:24.651 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.648+0000 7f336bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a9cc1c75e0 0x55a9cc1cf290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40704/0 (socket says 172.21.15.26:40704) 2024-09-11T23:19:24.652 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.648+0000 7f336bfff640 1 -- 172.21.15.26:0/3046052216 learned_addr learned my addr 172.21.15.26:0/3046052216 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:19:24.652 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.648+0000 7f336b7fe640 1 --2- 172.21.15.26:0/3046052216 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a9cc1c6cc0 0x55a9cc1ced50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:24.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.648+0000 7f1ae2ffd640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1ae4100e20 0x7f1ae410e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:24.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.648+0000 7f1ae27fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1ae4102ef0 0x7f1ae410e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:24.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.648+0000 7f1ae37fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ae410fe60 0x7f1ae410ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:24.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.648+0000 7f1ae27fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1ae4102ef0 0x7f1ae410e710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:32816/0 (socket says 172.21.15.26:32816) 2024-09-11T23:19:24.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.648+0000 7f1ae27fc640 1 -- 172.21.15.26:0/1890726096 learned_addr learned my addr 172.21.15.26:0/1890726096 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:19:24.653 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.648+0000 7f336bfff640 1 -- 172.21.15.26:0/3046052216 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a9cc1d1e80 msgr2=0x55a9cc1cf7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:19:24.653 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.648+0000 7f336bfff640 1 --2- 172.21.15.26:0/3046052216 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a9cc1d1e80 0x55a9cc1cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:24.653 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.648+0000 7f336bfff640 1 -- 172.21.15.26:0/3046052216 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a9cc1c6cc0 msgr2=0x55a9cc1ced50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:19:24.653 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.648+0000 7f336bfff640 1 --2- 172.21.15.26:0/3046052216 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a9cc1c6cc0 0x55a9cc1ced50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:24.653 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.648+0000 7f336bfff640 1 -- 172.21.15.26:0/3046052216 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a9cbe3e210 con 0x55a9cc1c75e0 2024-09-11T23:19:24.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.648+0000 7f1ae27fc640 1 -- 172.21.15.26:0/1890726096 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1ae4100e20 msgr2=0x7f1ae410e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:24.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.649+0000 7f1ae27fc640 1 --2- 172.21.15.26:0/1890726096 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1ae4100e20 0x7f1ae410e1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:24.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.649+0000 7f1ae27fc640 1 -- 172.21.15.26:0/1890726096 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ae410fe60 msgr2=0x7f1ae410ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:24.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.649+0000 7f1ae27fc640 1 --2- 172.21.15.26:0/1890726096 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ae410fe60 0x7f1ae410ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:24.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.649+0000 7f1ae27fc640 1 -- 172.21.15.26:0/1890726096 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ae419c900 con 0x7f1ae4102ef0 2024-09-11T23:19:24.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.649+0000 7f1ae37fe640 1 --2- 172.21.15.26:0/1890726096 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ae410fe60 0x7f1ae410ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:19:24.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.649+0000 7f1ae2ffd640 1 --2- 172.21.15.26:0/1890726096 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1ae4100e20 0x7f1ae410e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:19:24.655 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.649+0000 7f336bfff640 1 --2- 172.21.15.26:0/3046052216 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a9cc1c75e0 0x55a9cc1cf290 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f336000f640 tx=0x7f3360010720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bb345fc1ea38317 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:19:24.655 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.649+0000 7f334b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:19:24.655 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.649+0000 7f334b7fe640 1 -- 172.21.15.26:0/3046052216 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f336002a060 con 0x55a9cc1c75e0 2024-09-11T23:19:24.655 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.649+0000 7f334b7fe640 1 -- 172.21.15.26:0/3046052216 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f3360005030 con 0x55a9cc1c75e0 2024-09-11T23:19:24.655 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.649+0000 7f3375835740 1 -- 172.21.15.26:0/3046052216 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3360019920 con 0x55a9cc1c75e0 2024-09-11T23:19:24.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.649+0000 7f1ae27fc640 1 --2- 172.21.15.26:0/1890726096 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1ae4102ef0 0x7f1ae410e710 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f1ad802aa40 tx=0x7f1ad8031b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:19:24.655 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.650+0000 7f334b7fe640 1 -- 172.21.15.26:0/3046052216 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f336001e070 con 0x55a9cc1c75e0 2024-09-11T23:19:24.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.650+0000 7f1acbfff640 1 -- 172.21.15.26:0/1890726096 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ad8040030 con 0x7f1ae4102ef0 2024-09-11T23:19:24.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.650+0000 7f1ae9575640 1 -- 172.21.15.26:0/1890726096 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ae419cb90 con 0x7f1ae4102ef0 2024-09-11T23:19:24.656 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.650+0000 7f3375835740 10 client.15720 fetch_fsmap learned FSMap version 29 2024-09-11T23:19:24.656 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.650+0000 7f3375835740 1 -- 172.21.15.26:0/3046052216 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a9cbf0a620 con 0x55a9cc1c75e0 2024-09-11T23:19:24.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.650+0000 7f1ae9575640 1 -- 172.21.15.26:0/1890726096 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ae419d090 con 0x7f1ae4102ef0 2024-09-11T23:19:24.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.651+0000 7f1acbfff640 1 -- 172.21.15.26:0/1890726096 <== mon.2 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ad803c920 con 0x7f1ae4102ef0 2024-09-11T23:19:24.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.651+0000 7f1acbfff640 1 -- 172.21.15.26:0/1890726096 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ad80050a0 con 0x7f1ae4102ef0 2024-09-11T23:19:24.658 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.652+0000 7f334b7fe640 1 -- 172.21.15.26:0/3046052216 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3360019920 con 0x55a9cc1c75e0 2024-09-11T23:19:24.658 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.652+0000 7f334b7fe640 1 -- 172.21.15.26:0/3046052216 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f3360004680 con 0x55a9cc1c75e0 2024-09-11T23:19:24.658 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.653+0000 7f3375835740 10 client.15720 fetch_fsmap finished waiting for FSMap version 29 2024-09-11T23:19:24.658 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.653+0000 7f3375835740 10 client.15720 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:19:24.658 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.653+0000 7f3375835740 20 client.15720 populate_metadata read hostname 'smithi026' 2024-09-11T23:19:24.658 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.653+0000 7f3375835740 1 --2- 172.21.15.26:0/3046052216 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55a9cc1ea660 0x55a9cc1eca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:24.658 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.653+0000 7f3375835740 4 client.15720 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:19:24.658 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.653+0000 7f3375835740 1 -- 172.21.15.26:0/3046052216 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a9cc138a20 con 0x55a9cc1ea660 2024-09-11T23:19:24.659 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.653+0000 7f336b7fe640 1 --2- 172.21.15.26:0/3046052216 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55a9cc1ea660 0x55a9cc1eca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:24.659 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.654+0000 7f336b7fe640 1 --2- 172.21.15.26:0/3046052216 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55a9cc1ea660 0x55a9cc1eca50 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c745ef9463e860e server_cookie=93dbdd073fab1de1 in_seq=0 out_seq=0 2024-09-11T23:19:24.659 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.654+0000 7f334b7fe640 10 client.15720 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:19:24.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.654+0000 7f1acbfff640 1 -- 172.21.15.26:0/1890726096 <== mon.2 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 25) ==== 138066+0+0 (secure 0 0 0) 0x7f1ad8043030 con 0x7f1ae4102ef0 2024-09-11T23:19:24.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.654+0000 7f1aeb94e740 1 Processor -- start 2024-09-11T23:19:24.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.654+0000 7f1aeb94e740 1 -- start start 2024-09-11T23:19:24.659 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.654+0000 7f334b7fe640 1 -- 172.21.15.26:0/3046052216 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a9cc138a20 con 0x55a9cc1ea660 2024-09-11T23:19:24.659 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.654+0000 7f334b7fe640 10 client.15720 handle_command_reply: tid=0 2024-09-11T23:19:24.660 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.655+0000 7f3375835740 1 client.15720 shutdown 2024-09-11T23:19:24.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.654+0000 7f1aeb94e740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56451bf6e4a0 0x56451bf6e880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:24.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.654+0000 7f1aeb94e740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56451bf6edc0 0x56451bf79140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:24.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.654+0000 7f1aeb94e740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56451bf79680 0x56451bf7ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:24.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.654+0000 7f1aeb94e740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x56451bed42e0 con 0x56451bf79680 2024-09-11T23:19:24.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.654+0000 7f1aeb94e740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x56451be7bce0 con 0x56451bf6edc0 2024-09-11T23:19:24.661 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.655+0000 7f3375835740 1 -- 172.21.15.26:0/3046052216 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a9cc1c75e0 msgr2=0x55a9cc1cf290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:24.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.654+0000 7f1aeb94e740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x56451bed02a0 con 0x56451bf6e4a0 2024-09-11T23:19:24.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.655+0000 7f1ae37fe640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56451bf6e4a0 0x56451bf6e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:24.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.655+0000 7f1ae37fe640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56451bf6e4a0 0x56451bf6e880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:49752/0 (socket says 172.21.15.26:49752) 2024-09-11T23:19:24.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.655+0000 7f1ae37fe640 1 -- 172.21.15.26:0/1497413362 learned_addr learned my addr 172.21.15.26:0/1497413362 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:19:24.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.655+0000 7f1ae2ffd640 1 --2- 172.21.15.26:0/1497413362 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56451bf6edc0 0x56451bf79140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:24.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.655+0000 7f1ae27fc640 1 --2- 172.21.15.26:0/1497413362 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56451bf79680 0x56451bf7ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:24.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.655+0000 7f1ae37fe640 1 -- 172.21.15.26:0/1497413362 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56451bf6edc0 msgr2=0x56451bf79140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:19:24.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.655+0000 7f1ae37fe640 1 --2- 172.21.15.26:0/1497413362 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56451bf6edc0 0x56451bf79140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:24.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.655+0000 7f1ae37fe640 1 -- 172.21.15.26:0/1497413362 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56451bf79680 msgr2=0x56451bf7ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:19:24.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.655+0000 7f1ae37fe640 1 --2- 172.21.15.26:0/1497413362 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56451bf79680 0x56451bf7ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:24.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.655+0000 7f1ae37fe640 1 -- 172.21.15.26:0/1497413362 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56451be60440 con 0x56451bf6e4a0 2024-09-11T23:19:24.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.655+0000 7f1ae27fc640 1 --2- 172.21.15.26:0/1497413362 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56451bf79680 0x56451bf7ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:19:24.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.655+0000 7f1acbfff640 1 --2- 172.21.15.26:0/1890726096 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f1acc0a2ed0 0x7f1acc0a5390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:24.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.656+0000 7f1acbfff640 1 -- 172.21.15.26:0/1890726096 <== mon.2 v2:172.21.15.76:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f1ad80efc40 con 0x7f1ae4102ef0 2024-09-11T23:19:24.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.656+0000 7f1ae37fe640 1 --2- 172.21.15.26:0/1497413362 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56451bf6e4a0 0x56451bf6e880 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f1adc009ea0 tx=0x7f1adc009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=91a2cb686727a6a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:19:24.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.656+0000 7f1ac97fa640 1 -- 172.21.15.26:0/1497413362 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1adc016070 con 0x56451bf6e4a0 2024-09-11T23:19:24.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.656+0000 7f1ac97fa640 1 -- 172.21.15.26:0/1497413362 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1adc00ee80 con 0x56451bf6e4a0 2024-09-11T23:19:24.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.656+0000 7f1ac97fa640 1 -- 172.21.15.26:0/1497413362 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1adc0117c0 con 0x56451bf6e4a0 2024-09-11T23:19:24.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.657+0000 7f1ae2ffd640 1 --2- 172.21.15.26:0/1890726096 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f1acc0a2ed0 0x7f1acc0a5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:24.663 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.655+0000 7f3375835740 1 --2- 172.21.15.26:0/3046052216 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a9cc1c75e0 0x55a9cc1cf290 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f336000f640 tx=0x7f3360010720 comp rx=0 tx=0).stop 2024-09-11T23:19:24.663 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.657+0000 7f3375835740 1 -- 172.21.15.26:0/3046052216 shutdown_connections 2024-09-11T23:19:24.664 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.657+0000 7f3375835740 1 --2- 172.21.15.26:0/3046052216 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55a9cc1ea660 0x55a9cc1eca50 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:24.664 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.657+0000 7f3375835740 1 --2- 172.21.15.26:0/3046052216 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a9cc1d1e80 0x55a9cc1cf7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:24.664 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.657+0000 7f334b7fe640 0 client.15720 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:19:24.664 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.657+0000 7f3375835740 1 --2- 172.21.15.26:0/3046052216 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a9cc1c75e0 0x55a9cc1cf290 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:24.664 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.657+0000 7f3375835740 1 --2- 172.21.15.26:0/3046052216 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a9cc1c6cc0 0x55a9cc1ced50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:24.664 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.657+0000 7f3375835740 1 -- 172.21.15.26:0/3046052216 >> 172.21.15.26:0/3046052216 conn(0x55a9cc1c3230 msgr2=0x55a9cc1c8f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:19:24.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.657+0000 7f1aeb94e740 1 -- 172.21.15.26:0/1497413362 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56451bf6e4a0 msgr2=0x56451bf6e880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:24.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.657+0000 7f1aeb94e740 1 --2- 172.21.15.26:0/1497413362 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56451bf6e4a0 0x56451bf6e880 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f1adc009ea0 tx=0x7f1adc009f90 comp rx=0 tx=0).stop 2024-09-11T23:19:24.665 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.658+0000 7f3375835740 1 -- 172.21.15.26:0/3046052216 shutdown_connections 2024-09-11T23:19:24.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.658+0000 7f1aeb94e740 1 -- 172.21.15.26:0/1497413362 shutdown_connections 2024-09-11T23:19:24.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.658+0000 7f1aeb94e740 1 --2- 172.21.15.26:0/1497413362 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56451bf6edc0 0x56451bf79140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:24.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.658+0000 7f1aeb94e740 1 --2- 172.21.15.26:0/1497413362 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56451bf79680 0x56451bf7ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:24.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.658+0000 7f1aeb94e740 1 --2- 172.21.15.26:0/1497413362 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56451bf6e4a0 0x56451bf6e880 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:24.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.658+0000 7f1aeb94e740 1 -- 172.21.15.26:0/1497413362 >> 172.21.15.26:0/1497413362 conn(0x56451bf6aa30 msgr2=0x56451bf6ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:19:24.666 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.658+0000 7f3375835740 1 -- 172.21.15.26:0/3046052216 wait complete. 2024-09-11T23:19:24.666 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.658+0000 7f3375835740 20 client.15720 trim_cache size 0 max 16384 2024-09-11T23:19:24.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.658+0000 7f1ae2ffd640 1 --2- 172.21.15.26:0/1890726096 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f1acc0a2ed0 0x7f1acc0a5390 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x56451bf6b890 tx=0x7f1ad4005dd0 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:19:24.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.658+0000 7f1aeb94e740 1 -- 172.21.15.26:0/1497413362 shutdown_connections 2024-09-11T23:19:24.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.658+0000 7f1aeb94e740 1 -- 172.21.15.26:0/1497413362 wait complete. 2024-09-11T23:19:24.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.659+0000 7f1aeb94e740 1 -- start start 2024-09-11T23:19:24.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.659+0000 7f1aeb94e740 1 Processor -- start 2024-09-11T23:19:24.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.660+0000 7f1aeb94e740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56451bf6e4a0 0x56451bf78f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:24.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.660+0000 7f1aeb94e740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56451bf6edc0 0x56451bf761d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:24.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.660+0000 7f1aeb94e740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56451bf79680 0x56451bf76710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:24.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.660+0000 7f1aeb94e740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x56451bf7c2a0 con 0x56451bf6edc0 2024-09-11T23:19:24.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.660+0000 7f1aeb94e740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x56451be8f050 con 0x56451bf6e4a0 2024-09-11T23:19:24.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.660+0000 7f1aeb94e740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x56451bf7c420 con 0x56451bf79680 2024-09-11T23:19:24.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.660+0000 7f1ae27fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56451bf6edc0 0x56451bf761d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:24.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.660+0000 7f1aeb94e740 10 client.0 _ll_register_callbacks cb 0x56451becf160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:19:24.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.660+0000 7f1ae27fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56451bf6edc0 0x56451bf761d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40724/0 (socket says 172.21.15.26:40724) 2024-09-11T23:19:24.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.660+0000 7f1ae27fc640 1 -- 172.21.15.26:0/2300543444 learned_addr learned my addr 172.21.15.26:0/2300543444 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:19:24.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.660+0000 7f1ae27fc640 1 -- 172.21.15.26:0/2300543444 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56451bf79680 msgr2=0x56451bf76710 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T23:19:24.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.660+0000 7f1ae27fc640 1 --2- 172.21.15.26:0/2300543444 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56451bf79680 0x56451bf76710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:24.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.660+0000 7f1ae27fc640 1 -- 172.21.15.26:0/2300543444 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56451bf6e4a0 msgr2=0x56451bf78f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:19:24.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.660+0000 7f1ae2ffd640 1 --2- 172.21.15.26:0/2300543444 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56451bf6e4a0 0x56451bf78f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:24.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.661+0000 7f1ae27fc640 1 --2- 172.21.15.26:0/2300543444 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56451bf6e4a0 0x56451bf78f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:24.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.661+0000 7f1ae27fc640 1 -- 172.21.15.26:0/2300543444 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56451bc453c0 con 0x56451bf6edc0 2024-09-11T23:19:24.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.661+0000 7f1ae2ffd640 1 --2- 172.21.15.26:0/2300543444 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56451bf6e4a0 0x56451bf78f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:19:24.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.661+0000 7f1ae27fc640 1 --2- 172.21.15.26:0/2300543444 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56451bf6edc0 0x56451bf761d0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f1ad80bbcf0 tx=0x7f1ad803ce20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=981468b5e77dce26 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:19:24.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.661+0000 7f1ac6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:19:24.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.661+0000 7f1ac6ff5640 1 -- 172.21.15.26:0/2300543444 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ad804a040 con 0x56451bf6edc0 2024-09-11T23:19:24.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.661+0000 7f1ac6ff5640 1 -- 172.21.15.26:0/2300543444 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f1ad803cad0 con 0x56451bf6edc0 2024-09-11T23:19:24.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.661+0000 7f1aeb94e740 1 -- 172.21.15.26:0/2300543444 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56451bee1850 con 0x56451bf6edc0 2024-09-11T23:19:24.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.661+0000 7f1ac6ff5640 1 -- 172.21.15.26:0/2300543444 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1ad80c03a0 con 0x56451bf6edc0 2024-09-11T23:19:24.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.662+0000 7f1aeb94e740 10 client.15723 fetch_fsmap learned FSMap version 29 2024-09-11T23:19:24.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.662+0000 7f1aeb94e740 1 -- 172.21.15.26:0/2300543444 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56451bf87b60 con 0x56451bf6edc0 2024-09-11T23:19:24.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.662+0000 7f1ac6ff5640 1 -- 172.21.15.26:0/2300543444 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ad80c4c60 con 0x56451bf6edc0 2024-09-11T23:19:24.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.662+0000 7f1ac6ff5640 1 -- 172.21.15.26:0/2300543444 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f1ad80c47d0 con 0x56451bf6edc0 2024-09-11T23:19:24.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.662+0000 7f1aeb94e740 10 client.15723 fetch_fsmap finished waiting for FSMap version 29 2024-09-11T23:19:24.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.662+0000 7f1aeb94e740 10 client.15723 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:19:24.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.662+0000 7f1aeb94e740 20 client.15723 populate_metadata read hostname 'smithi026' 2024-09-11T23:19:24.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.662+0000 7f1aeb94e740 1 --2- 172.21.15.26:0/2300543444 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x56451bf88f70 0x56451bf8b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:24.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.662+0000 7f1aeb94e740 4 client.15723 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:19:24.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.662+0000 7f1aeb94e740 1 -- 172.21.15.26:0/2300543444 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56451bee1850 con 0x56451bf88f70 2024-09-11T23:19:24.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.663+0000 7f1ae2ffd640 1 --2- 172.21.15.26:0/2300543444 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x56451bf88f70 0x56451bf8b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:24.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.663+0000 7f1ae2ffd640 1 --2- 172.21.15.26:0/2300543444 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x56451bf88f70 0x56451bf8b3f0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ddaebdaf7f97171c server_cookie=2077d41b342dc568 in_seq=0 out_seq=0 2024-09-11T23:19:24.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.663+0000 7f1ac6ff5640 10 client.15723 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:19:24.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.664+0000 7f1ac6ff5640 1 -- 172.21.15.26:0/2300543444 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56451bee1850 con 0x56451bf88f70 2024-09-11T23:19:24.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.664+0000 7f1ac6ff5640 10 client.15723 handle_command_reply: tid=0 2024-09-11T23:19:24.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.664+0000 7f1aeb94e740 1 client.15723 shutdown 2024-09-11T23:19:24.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.664+0000 7f1aeb94e740 1 -- 172.21.15.26:0/2300543444 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56451bf6edc0 msgr2=0x56451bf761d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:24.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.664+0000 7f1aeb94e740 1 --2- 172.21.15.26:0/2300543444 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56451bf6edc0 0x56451bf761d0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f1ad80bbcf0 tx=0x7f1ad803ce20 comp rx=0 tx=0).stop 2024-09-11T23:19:24.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.665+0000 7f1aeb94e740 1 -- 172.21.15.26:0/2300543444 shutdown_connections 2024-09-11T23:19:24.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.665+0000 7f1aeb94e740 1 --2- 172.21.15.26:0/2300543444 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x56451bf88f70 0x56451bf8b3f0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:24.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.665+0000 7f1ac6ff5640 0 client.15723 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:19:24.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.665+0000 7f1aeb94e740 1 --2- 172.21.15.26:0/2300543444 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56451bf79680 0x56451bf76710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:24.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.665+0000 7f1aeb94e740 1 --2- 172.21.15.26:0/2300543444 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56451bf6edc0 0x56451bf761d0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:24.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.665+0000 7f1aeb94e740 1 --2- 172.21.15.26:0/2300543444 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56451bf6e4a0 0x56451bf78f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:24.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.665+0000 7f1aeb94e740 1 -- 172.21.15.26:0/2300543444 >> 172.21.15.26:0/2300543444 conn(0x56451bf6aa30 msgr2=0x56451bf69b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:19:24.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.665+0000 7f1aeb94e740 1 -- 172.21.15.26:0/2300543444 shutdown_connections 2024-09-11T23:19:24.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.665+0000 7f1aeb94e740 1 -- 172.21.15.26:0/2300543444 wait complete. 2024-09-11T23:19:24.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.665+0000 7f1aeb94e740 20 client.15723 trim_cache size 0 max 16384 2024-09-11T23:19:24.691 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.690+0000 7f3375835740 1 Processor -- start 2024-09-11T23:19:24.691 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.690+0000 7f3375835740 1 -- start start 2024-09-11T23:19:24.693 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.691+0000 7f3375835740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a9cc1c6cc0 0x55a9cc1f2140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:24.693 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.691+0000 7f3375835740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a9cc1c75e0 0x55a9cc1f2680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:24.693 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.691+0000 7f3375835740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a9cc1d1e80 0x55a9cc1cbf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:24.693 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.691+0000 7f3375835740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f336413ef80 con 0x55a9cc1c75e0 2024-09-11T23:19:24.693 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.691+0000 7f336b7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a9cc1c6cc0 0x55a9cc1f2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:24.693 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.691+0000 7f336bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a9cc1c75e0 0x55a9cc1f2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:24.693 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.691+0000 7f336bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a9cc1c75e0 0x55a9cc1f2680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40732/0 (socket says 172.21.15.26:40732) 2024-09-11T23:19:24.693 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.691+0000 7f336bfff640 1 -- 172.21.15.26:0/2203986263 learned_addr learned my addr 172.21.15.26:0/2203986263 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:19:24.693 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.692+0000 7f3375835740 1 -- 172.21.15.26:0/2203986263 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f3364140ac0 con 0x55a9cc1c6cc0 2024-09-11T23:19:24.694 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.692+0000 7f3375835740 1 -- 172.21.15.26:0/2203986263 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f336413f650 con 0x55a9cc1d1e80 2024-09-11T23:19:24.694 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.692+0000 7f336b7fe640 1 --2- 172.21.15.26:0/2203986263 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a9cc1d1e80 0x55a9cc1cbf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:24.694 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.692+0000 7f336b7fe640 1 -- 172.21.15.26:0/2203986263 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a9cc1c6cc0 msgr2=0x55a9cc1f2140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:19:24.694 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.692+0000 7f336b7fe640 1 --2- 172.21.15.26:0/2203986263 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a9cc1c6cc0 0x55a9cc1f2140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:24.694 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.692+0000 7f336b7fe640 1 -- 172.21.15.26:0/2203986263 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a9cc1c75e0 msgr2=0x55a9cc1f2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:19:24.694 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.692+0000 7f336b7fe640 1 --2- 172.21.15.26:0/2203986263 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a9cc1c75e0 0x55a9cc1f2680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:24.694 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.692+0000 7f336b7fe640 1 -- 172.21.15.26:0/2203986263 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a9cc1c7410 con 0x55a9cc1d1e80 2024-09-11T23:19:24.695 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.692+0000 7f336b7fe640 1 --2- 172.21.15.26:0/2203986263 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a9cc1d1e80 0x55a9cc1cbf90 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f3358009530 tx=0x7f3358009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=346d5f321f54d387 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:19:24.695 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.693+0000 7f3355ffb640 1 -- 172.21.15.26:0/2203986263 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3358000e60 con 0x55a9cc1d1e80 2024-09-11T23:19:24.695 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.693+0000 7f3355ffb640 1 -- 172.21.15.26:0/2203986263 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3358005040 con 0x55a9cc1d1e80 2024-09-11T23:19:24.695 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.693+0000 7f3355ffb640 1 -- 172.21.15.26:0/2203986263 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3358004ab0 con 0x55a9cc1d1e80 2024-09-11T23:19:24.696 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.694+0000 7f3375835740 1 -- 172.21.15.26:0/2203986263 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a9cc1d1e80 msgr2=0x55a9cc1cbf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:24.696 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.694+0000 7f3375835740 1 --2- 172.21.15.26:0/2203986263 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a9cc1d1e80 0x55a9cc1cbf90 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f3358009530 tx=0x7f3358009260 comp rx=0 tx=0).stop 2024-09-11T23:19:24.696 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.694+0000 7f3375835740 1 -- 172.21.15.26:0/2203986263 shutdown_connections 2024-09-11T23:19:24.696 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.694+0000 7f3375835740 1 --2- 172.21.15.26:0/2203986263 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a9cc1d1e80 0x55a9cc1cbf90 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:24.696 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.694+0000 7f3375835740 1 --2- 172.21.15.26:0/2203986263 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a9cc1c75e0 0x55a9cc1f2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:24.696 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.694+0000 7f3375835740 1 --2- 172.21.15.26:0/2203986263 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a9cc1c6cc0 0x55a9cc1f2140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:24.696 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.694+0000 7f3375835740 1 -- 172.21.15.26:0/2203986263 >> 172.21.15.26:0/2203986263 conn(0x55a9cc1c3230 msgr2=0x55a9cc1d0dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:19:24.696 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.694+0000 7f3375835740 1 -- 172.21.15.26:0/2203986263 shutdown_connections 2024-09-11T23:19:24.696 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.694+0000 7f3375835740 1 -- 172.21.15.26:0/2203986263 wait complete. 2024-09-11T23:19:24.696 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.694+0000 7f3375835740 1 -- start start 2024-09-11T23:19:24.696 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.695+0000 7f3375835740 1 Processor -- start 2024-09-11T23:19:24.699 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.695+0000 7f3375835740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a9cc1c6cc0 0x55a9cc1de870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:24.699 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.695+0000 7f3375835740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a9cc1c75e0 0x55a9cc1dedb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:24.699 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.695+0000 7f3375835740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a9cc1d1e80 0x55a9cc254110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:24.699 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.696+0000 7f336b7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a9cc1d1e80 0x55a9cc254110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:24.699 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.696+0000 7f336b7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a9cc1c6cc0 0x55a9cc1de870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:24.699 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.696+0000 7f336b7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a9cc1c6cc0 0x55a9cc1de870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:32860/0 (socket says 172.21.15.26:32860) 2024-09-11T23:19:24.699 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.696+0000 7f336bfff640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a9cc1c75e0 0x55a9cc1dedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:24.699 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.696+0000 7f336b7fe640 1 -- 172.21.15.26:0/2363791673 learned_addr learned my addr 172.21.15.26:0/2363791673 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:19:24.700 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.696+0000 7f3375835740 1 -- 172.21.15.26:0/2363791673 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f33641403f0 con 0x55a9cc1d1e80 2024-09-11T23:19:24.700 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.696+0000 7f3375835740 1 -- 172.21.15.26:0/2363791673 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f336413fd20 con 0x55a9cc1c6cc0 2024-09-11T23:19:24.700 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.696+0000 7f3375835740 1 -- 172.21.15.26:0/2363791673 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f336413cf60 con 0x55a9cc1c75e0 2024-09-11T23:19:24.700 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.696+0000 7f336bfff640 1 -- 172.21.15.26:0/2363791673 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a9cc1c6cc0 msgr2=0x55a9cc1de870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:19:24.700 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.696+0000 7f336bfff640 1 --2- 172.21.15.26:0/2363791673 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a9cc1c6cc0 0x55a9cc1de870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:24.700 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.696+0000 7f336bfff640 1 -- 172.21.15.26:0/2363791673 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a9cc1d1e80 msgr2=0x55a9cc254110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:19:24.700 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.696+0000 7f336bfff640 1 --2- 172.21.15.26:0/2363791673 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a9cc1d1e80 0x55a9cc254110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:24.700 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.696+0000 7f336bfff640 1 -- 172.21.15.26:0/2363791673 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a9cc254650 con 0x55a9cc1c75e0 2024-09-11T23:19:24.700 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.696+0000 7f336b7fe640 1 --2- 172.21.15.26:0/2363791673 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a9cc1d1e80 0x55a9cc254110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:19:24.700 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.697+0000 7f3375835740 10 client.0 _ll_register_callbacks cb 0x7f336001c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:19:24.700 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.697+0000 7f3375835740 1 -- 172.21.15.26:0/2363791673 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3358005040 con 0x55a9cc1c75e0 2024-09-11T23:19:24.700 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.697+0000 7f336bfff640 1 --2- 172.21.15.26:0/2363791673 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a9cc1c75e0 0x55a9cc1dedb0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f3360004a50 tx=0x7f3360004930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=714b38c7bdce7e45 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:19:24.700 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.697+0000 7f334b7fe640 10 client.25363 ms_handle_connect on v2:172.21.15.136:3300/0 2024-09-11T23:19:24.701 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.697+0000 7f334b7fe640 1 -- 172.21.15.26:0/2363791673 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f336003c060 con 0x55a9cc1c75e0 2024-09-11T23:19:24.701 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.697+0000 7f334b7fe640 1 -- 172.21.15.26:0/2363791673 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f33600050d0 con 0x55a9cc1c75e0 2024-09-11T23:19:24.701 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.697+0000 7f334b7fe640 1 -- 172.21.15.26:0/2363791673 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f3360005350 con 0x55a9cc1c75e0 2024-09-11T23:19:24.701 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.697+0000 7f3375835740 10 client.25363 fetch_fsmap learned FSMap version 29 2024-09-11T23:19:24.701 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.697+0000 7f3375835740 1 -- 172.21.15.26:0/2363791673 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({fsmap=29}) -- 0x55a9cc2554b0 con 0x55a9cc1c75e0 2024-09-11T23:19:24.701 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.698+0000 7f334b7fe640 1 -- 172.21.15.26:0/2363791673 <== mon.1 v2:172.21.15.136:3300/0 4 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f3360019ce0 con 0x55a9cc1c75e0 2024-09-11T23:19:24.701 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.698+0000 7f3375835740 10 client.25363 fetch_fsmap finished waiting for FSMap version 29 2024-09-11T23:19:24.701 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.698+0000 7f3375835740 10 client.25363 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:19:24.701 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.698+0000 7f3375835740 20 client.25363 populate_metadata read hostname 'smithi026' 2024-09-11T23:19:24.701 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.698+0000 7f3375835740 1 --2- 172.21.15.26:0/2363791673 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55a9cc1ea660 0x55a9cc257ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:24.701 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.698+0000 7f3375835740 4 client.25363 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T23:19:24.701 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.698+0000 7f3375835740 1 -- 172.21.15.26:0/2363791673 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a9cc132d70 con 0x55a9cc1ea660 2024-09-11T23:19:24.702 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.698+0000 7f336b7fe640 1 --2- 172.21.15.26:0/2363791673 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55a9cc1ea660 0x55a9cc257ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:24.702 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.699+0000 7f334b7fe640 10 client.25363 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:19:24.702 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.699+0000 7f336b7fe640 1 --2- 172.21.15.26:0/2363791673 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55a9cc1ea660 0x55a9cc257ca0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=30205e4c799d1c47 server_cookie=a211fa1e0c6291 in_seq=0 out_seq=0 2024-09-11T23:19:24.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.700+0000 7f1aeb94e740 1 Processor -- start 2024-09-11T23:19:24.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.701+0000 7f1aeb94e740 1 -- start start 2024-09-11T23:19:24.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.701+0000 7f1aeb94e740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56451bf6e4a0 0x56451bf74aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:24.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.701+0000 7f1aeb94e740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56451bf6edc0 0x56451bf71720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:24.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.701+0000 7f1ae27fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56451bf6edc0 0x56451bf71720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:24.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.701+0000 7f1ae27fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56451bf6edc0 0x56451bf71720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40756/0 (socket says 172.21.15.26:40756) 2024-09-11T23:19:24.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.701+0000 7f1ae27fc640 1 -- 172.21.15.26:0/1441472982 learned_addr learned my addr 172.21.15.26:0/1441472982 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:19:24.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.701+0000 7f1aeb94e740 1 --2- 172.21.15.26:0/1441472982 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56451bf79680 0x56451bf78140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:24.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.701+0000 7f1aeb94e740 1 -- 172.21.15.26:0/1441472982 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f1ae41bcd50 con 0x56451bf6edc0 2024-09-11T23:19:24.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.701+0000 7f1aeb94e740 1 -- 172.21.15.26:0/1441472982 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f1ae41be890 con 0x56451bf6e4a0 2024-09-11T23:19:24.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.701+0000 7f1aeb94e740 1 -- 172.21.15.26:0/1441472982 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f1ae41bd420 con 0x56451bf79680 2024-09-11T23:19:24.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.702+0000 7f1ae2ffd640 1 --2- 172.21.15.26:0/1441472982 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56451bf6e4a0 0x56451bf74aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:24.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.702+0000 7f1ae27fc640 1 -- 172.21.15.26:0/1441472982 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56451bf79680 msgr2=0x56451bf78140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:19:24.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.702+0000 7f1ae27fc640 1 --2- 172.21.15.26:0/1441472982 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56451bf79680 0x56451bf78140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:24.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.702+0000 7f1ae27fc640 1 -- 172.21.15.26:0/1441472982 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56451bf6e4a0 msgr2=0x56451bf74aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:19:24.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.702+0000 7f1ae27fc640 1 --2- 172.21.15.26:0/1441472982 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56451bf6e4a0 0x56451bf74aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:24.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.702+0000 7f1ae27fc640 1 -- 172.21.15.26:0/1441472982 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56451bf75180 con 0x56451bf6edc0 2024-09-11T23:19:24.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.703+0000 7f1ae27fc640 1 --2- 172.21.15.26:0/1441472982 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56451bf6edc0 0x56451bf71720 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f1ad80b89c0 tx=0x7f1ad80ed620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=421bc44ad17a2771 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:19:24.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.703+0000 7f1ac8ff9640 1 -- 172.21.15.26:0/1441472982 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ad804a070 con 0x56451bf6edc0 2024-09-11T23:19:24.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.703+0000 7f1ac8ff9640 1 -- 172.21.15.26:0/1441472982 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ad80b8660 con 0x56451bf6edc0 2024-09-11T23:19:24.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.703+0000 7f1ac8ff9640 1 -- 172.21.15.26:0/1441472982 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ad80bb530 con 0x56451bf6edc0 2024-09-11T23:19:24.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.706+0000 7f1aeb94e740 1 -- 172.21.15.26:0/1441472982 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56451bf6edc0 msgr2=0x56451bf71720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:24.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.706+0000 7f1aeb94e740 1 --2- 172.21.15.26:0/1441472982 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56451bf6edc0 0x56451bf71720 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f1ad80b89c0 tx=0x7f1ad80ed620 comp rx=0 tx=0).stop 2024-09-11T23:19:24.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.706+0000 7f1aeb94e740 1 -- 172.21.15.26:0/1441472982 shutdown_connections 2024-09-11T23:19:24.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.706+0000 7f1aeb94e740 1 --2- 172.21.15.26:0/1441472982 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56451bf79680 0x56451bf78140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:24.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.706+0000 7f1aeb94e740 1 --2- 172.21.15.26:0/1441472982 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56451bf6edc0 0x56451bf71720 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:24.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.706+0000 7f1aeb94e740 1 --2- 172.21.15.26:0/1441472982 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56451bf6e4a0 0x56451bf74aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:24.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.706+0000 7f1aeb94e740 1 -- 172.21.15.26:0/1441472982 >> 172.21.15.26:0/1441472982 conn(0x56451bf6aa30 msgr2=0x56451bf77a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:19:24.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.706+0000 7f1aeb94e740 1 -- 172.21.15.26:0/1441472982 shutdown_connections 2024-09-11T23:19:24.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.706+0000 7f1aeb94e740 1 -- 172.21.15.26:0/1441472982 wait complete. 2024-09-11T23:19:24.708 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.707+0000 7f334b7fe640 1 -- 172.21.15.26:0/2363791673 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a9cc132d70 con 0x55a9cc1ea660 2024-09-11T23:19:24.708 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.707+0000 7f334b7fe640 10 client.25363 handle_command_reply: tid=0 2024-09-11T23:19:24.709 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.707+0000 7f3375835740 1 client.25363 shutdown 2024-09-11T23:19:24.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.707+0000 7f1aeb94e740 1 -- start start 2024-09-11T23:19:24.709 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.707+0000 7f3375835740 1 -- 172.21.15.26:0/2363791673 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a9cc1c75e0 msgr2=0x55a9cc1dedb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:24.709 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.707+0000 7f3375835740 1 --2- 172.21.15.26:0/2363791673 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a9cc1c75e0 0x55a9cc1dedb0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f3360004a50 tx=0x7f3360004930 comp rx=0 tx=0).stop 2024-09-11T23:19:24.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.707+0000 7f1aeb94e740 1 Processor -- start 2024-09-11T23:19:24.710 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.707+0000 7f3375835740 1 -- 172.21.15.26:0/2363791673 shutdown_connections 2024-09-11T23:19:24.711 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.707+0000 7f3375835740 1 --2- 172.21.15.26:0/2363791673 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55a9cc1ea660 0x55a9cc257ca0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:24.711 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.707+0000 7f334b7fe640 0 client.25363 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:19:24.711 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.707+0000 7f3375835740 1 --2- 172.21.15.26:0/2363791673 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a9cc1d1e80 0x55a9cc254110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:24.711 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.707+0000 7f3375835740 1 --2- 172.21.15.26:0/2363791673 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a9cc1c75e0 0x55a9cc1dedb0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:24.711 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.707+0000 7f3375835740 1 --2- 172.21.15.26:0/2363791673 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a9cc1c6cc0 0x55a9cc1de870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:24.711 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.707+0000 7f3375835740 1 -- 172.21.15.26:0/2363791673 >> 172.21.15.26:0/2363791673 conn(0x55a9cc1c3230 msgr2=0x55a9cc1c8720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:19:24.711 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.708+0000 7f3375835740 1 -- 172.21.15.26:0/2363791673 shutdown_connections 2024-09-11T23:19:24.711 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.708+0000 7f3375835740 1 -- 172.21.15.26:0/2363791673 wait complete. 2024-09-11T23:19:24.711 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.708+0000 7f3375835740 20 client.25363 trim_cache size 0 max 16384 2024-09-11T23:19:24.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.708+0000 7f1aeb94e740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56451bf6e4a0 0x56451bf70c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:24.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.708+0000 7f1aeb94e740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56451bf6edc0 0x56451bf71150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:24.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.708+0000 7f1aeb94e740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56451bf79680 0x56451bf71690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:24.712 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-09-11T23:19:24.712 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "message": "", 2024-09-11T23:19:24.712 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-09-11T23:19:24.712 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-09-11T23:19:24.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.708+0000 7f1ae2ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56451bf6e4a0 0x56451bf70c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:24.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.708+0000 7f1ae2ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56451bf6e4a0 0x56451bf70c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:32888/0 (socket says 172.21.15.26:32888) 2024-09-11T23:19:24.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.708+0000 7f1ae27fc640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56451bf6edc0 0x56451bf71150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:24.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.708+0000 7f1ae37fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56451bf79680 0x56451bf71690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:24.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.708+0000 7f1ae27fc640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56451bf6edc0 0x56451bf71150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:49794/0 (socket says 172.21.15.26:49794) 2024-09-11T23:19:24.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.708+0000 7f1ae37fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56451bf79680 0x56451bf71690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40762/0 (socket says 172.21.15.26:40762) 2024-09-11T23:19:24.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.708+0000 7f1ae2ffd640 1 -- 172.21.15.26:0/3051726351 learned_addr learned my addr 172.21.15.26:0/3051726351 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:19:24.714 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.709+0000 7f337175b640 1 -- 172.21.15.26:0/1378006403 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f335c0a2ea0 msgr2=0x7f335c0a5360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:24.714 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.709+0000 7f337175b640 1 --2- 172.21.15.26:0/1378006403 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f335c0a2ea0 0x7f335c0a5360 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f336c158560 tx=0x7f336c068050 comp rx=0 tx=0).stop 2024-09-11T23:19:24.714 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.709+0000 7f337175b640 1 -- 172.21.15.26:0/1378006403 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3364099000 msgr2=0x7f33641298b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:24.714 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.709+0000 7f337175b640 1 --2- 172.21.15.26:0/1378006403 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3364099000 0x7f33641298b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f336c000b60 tx=0x7f336c06ba50 comp rx=0 tx=0).stop 2024-09-11T23:19:24.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.708+0000 7f1aeb94e740 1 -- 172.21.15.26:0/3051726351 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f1ae41be1c0 con 0x56451bf79680 2024-09-11T23:19:24.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.708+0000 7f1aeb94e740 1 -- 172.21.15.26:0/3051726351 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f1ae41bdaf0 con 0x56451bf6e4a0 2024-09-11T23:19:24.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.708+0000 7f1aeb94e740 1 -- 172.21.15.26:0/3051726351 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f1ae41029b0 con 0x56451bf6edc0 2024-09-11T23:19:24.715 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.709+0000 7f337175b640 1 -- 172.21.15.26:0/1378006403 shutdown_connections 2024-09-11T23:19:24.715 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.709+0000 7f337175b640 1 --2- 172.21.15.26:0/1378006403 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f335c0a2ea0 0x7f335c0a5360 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:24.716 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.709+0000 7f337175b640 1 --2- 172.21.15.26:0/1378006403 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3364099000 0x7f33641298b0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:24.716 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.709+0000 7f337175b640 1 --2- 172.21.15.26:0/1378006403 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3364094cc0 0x7f336412fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:24.716 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.709+0000 7f337175b640 1 --2- 172.21.15.26:0/1378006403 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3364094310 0x7f336412f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:24.716 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.709+0000 7f337175b640 1 -- 172.21.15.26:0/1378006403 >> 172.21.15.26:0/1378006403 conn(0x7f3364009990 msgr2=0x7f3364091e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:19:24.716 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.709+0000 7f337175b640 1 -- 172.21.15.26:0/1378006403 shutdown_connections 2024-09-11T23:19:24.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.709+0000 7f1ae2ffd640 1 -- 172.21.15.26:0/3051726351 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56451bf6edc0 msgr2=0x56451bf71150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:19:24.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.709+0000 7f1ae2ffd640 1 --2- 172.21.15.26:0/3051726351 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56451bf6edc0 0x56451bf71150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:24.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.709+0000 7f1ae2ffd640 1 -- 172.21.15.26:0/3051726351 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56451bf79680 msgr2=0x56451bf71690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:19:24.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.709+0000 7f1ae2ffd640 1 --2- 172.21.15.26:0/3051726351 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56451bf79680 0x56451bf71690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:24.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.709+0000 7f1ae2ffd640 1 -- 172.21.15.26:0/3051726351 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56451bf7fa70 con 0x56451bf6e4a0 2024-09-11T23:19:24.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.709+0000 7f1ae37fe640 1 --2- 172.21.15.26:0/3051726351 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56451bf79680 0x56451bf71690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:19:24.717 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:24.709+0000 7f337175b640 1 -- 172.21.15.26:0/1378006403 wait complete. 2024-09-11T23:19:24.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.709+0000 7f1aeb94e740 10 client.0 _ll_register_callbacks cb 0x7f1ad80b7b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:19:24.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.709+0000 7f1aeb94e740 1 -- 172.21.15.26:0/3051726351 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ad80b8660 con 0x56451bf6e4a0 2024-09-11T23:19:24.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.709+0000 7f1ae2ffd640 1 --2- 172.21.15.26:0/3051726351 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56451bf6e4a0 0x56451bf70c10 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x56451bf8bc20 tx=0x7f1ad40046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cf6ad614a071c4ac server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:19:24.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.709+0000 7f1ac6ff5640 10 client.25328 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-11T23:19:24.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.710+0000 7f1ac6ff5640 1 -- 172.21.15.26:0/3051726351 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ad401c620 con 0x56451bf6e4a0 2024-09-11T23:19:24.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.710+0000 7f1ac6ff5640 1 -- 172.21.15.26:0/3051726351 <== mon.2 v2:172.21.15.76:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f1ad401c7c0 con 0x56451bf6e4a0 2024-09-11T23:19:24.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.710+0000 7f1ac6ff5640 1 -- 172.21.15.26:0/3051726351 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1ad401c9c0 con 0x56451bf6e4a0 2024-09-11T23:19:24.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.710+0000 7f1aeb94e740 10 client.25328 fetch_fsmap learned FSMap version 29 2024-09-11T23:19:24.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.710+0000 7f1aeb94e740 1 -- 172.21.15.26:0/3051726351 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({fsmap=29}) -- 0x56451bff3c80 con 0x56451bf6e4a0 2024-09-11T23:19:24.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.710+0000 7f1ac6ff5640 1 -- 172.21.15.26:0/3051726351 <== mon.2 v2:172.21.15.76:3300/0 4 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f1ad401cb60 con 0x56451bf6e4a0 2024-09-11T23:19:24.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.710+0000 7f1aeb94e740 10 client.25328 fetch_fsmap finished waiting for FSMap version 29 2024-09-11T23:19:24.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.710+0000 7f1aeb94e740 10 client.25328 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:19:24.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.710+0000 7f1aeb94e740 20 client.25328 populate_metadata read hostname 'smithi026' 2024-09-11T23:19:24.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.710+0000 7f1aeb94e740 1 --2- 172.21.15.26:0/3051726351 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x56451bff4290 0x56451bff6720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:24.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.711+0000 7f1aeb94e740 4 client.25328 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T23:19:24.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.711+0000 7f1aeb94e740 1 -- 172.21.15.26:0/3051726351 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56451bedbd60 con 0x56451bff4290 2024-09-11T23:19:24.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.711+0000 7f1ae27fc640 1 --2- 172.21.15.26:0/3051726351 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x56451bff4290 0x56451bff6720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:24.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.711+0000 7f1ac6ff5640 10 client.25328 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:19:24.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.711+0000 7f1ae27fc640 1 --2- 172.21.15.26:0/3051726351 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x56451bff4290 0x56451bff6720 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=49c2e71e8e9cbc93 server_cookie=3801816175a5076e in_seq=0 out_seq=0 2024-09-11T23:19:24.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.712+0000 7f1ac6ff5640 1 -- 172.21.15.26:0/3051726351 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x56451bedbd60 con 0x56451bff4290 2024-09-11T23:19:24.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.712+0000 7f1ac6ff5640 10 client.25328 handle_command_reply: tid=0 2024-09-11T23:19:24.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.712+0000 7f1aeb94e740 1 client.25328 shutdown 2024-09-11T23:19:24.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.712+0000 7f1aeb94e740 1 -- 172.21.15.26:0/3051726351 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56451bf6e4a0 msgr2=0x56451bf70c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:24.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.712+0000 7f1aeb94e740 1 --2- 172.21.15.26:0/3051726351 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56451bf6e4a0 0x56451bf70c10 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x56451bf8bc20 tx=0x7f1ad40046c0 comp rx=0 tx=0).stop 2024-09-11T23:19:24.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.713+0000 7f1aeb94e740 1 -- 172.21.15.26:0/3051726351 shutdown_connections 2024-09-11T23:19:24.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.713+0000 7f1aeb94e740 1 --2- 172.21.15.26:0/3051726351 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x56451bff4290 0x56451bff6720 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:24.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.713+0000 7f1ac6ff5640 0 client.25328 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:19:24.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.713+0000 7f1aeb94e740 1 --2- 172.21.15.26:0/3051726351 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56451bf79680 0x56451bf71690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:24.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.713+0000 7f1aeb94e740 1 --2- 172.21.15.26:0/3051726351 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56451bf6edc0 0x56451bf71150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:24.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.713+0000 7f1aeb94e740 1 --2- 172.21.15.26:0/3051726351 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56451bf6e4a0 0x56451bf70c10 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:24.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.713+0000 7f1aeb94e740 1 -- 172.21.15.26:0/3051726351 >> 172.21.15.26:0/3051726351 conn(0x56451bf6aa30 msgr2=0x56451bf77a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:19:24.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.713+0000 7f1aeb94e740 1 -- 172.21.15.26:0/3051726351 shutdown_connections 2024-09-11T23:19:24.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.713+0000 7f1aeb94e740 1 -- 172.21.15.26:0/3051726351 wait complete. 2024-09-11T23:19:24.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.713+0000 7f1aeb94e740 20 client.25328 trim_cache size 0 max 16384 2024-09-11T23:19:24.722 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-11T23:19:24.722 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-11T23:19:24.722 INFO:teuthology.orchestra.run.smithi026.stdout: "scrubs": { 2024-09-11T23:19:24.722 INFO:teuthology.orchestra.run.smithi026.stdout: "6adeb6f6-995b-4377-9a31-baeabff7f707": { 2024-09-11T23:19:24.722 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/", 2024-09-11T23:19:24.722 INFO:teuthology.orchestra.run.smithi026.stdout: "tag": "6adeb6f6-995b-4377-9a31-baeabff7f707", 2024-09-11T23:19:24.722 INFO:teuthology.orchestra.run.smithi026.stdout: "options": "recursive,force" 2024-09-11T23:19:24.722 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:19:24.722 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:19:24.723 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-11T23:19:24.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.714+0000 7f1ae9575640 1 -- 172.21.15.26:0/1890726096 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f1acc0a2ed0 msgr2=0x7f1acc0a5390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:24.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.714+0000 7f1ae9575640 1 --2- 172.21.15.26:0/1890726096 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f1acc0a2ed0 0x7f1acc0a5390 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x56451bf6b890 tx=0x7f1ad4005dd0 comp rx=0 tx=0).stop 2024-09-11T23:19:24.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.714+0000 7f1ae9575640 1 -- 172.21.15.26:0/1890726096 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1ae4102ef0 msgr2=0x7f1ae410e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:24.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.714+0000 7f1ae9575640 1 --2- 172.21.15.26:0/1890726096 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1ae4102ef0 0x7f1ae410e710 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f1ad802aa40 tx=0x7f1ad8031b20 comp rx=0 tx=0).stop 2024-09-11T23:19:24.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.714+0000 7f1ae9575640 1 -- 172.21.15.26:0/1890726096 shutdown_connections 2024-09-11T23:19:24.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.714+0000 7f1ae9575640 1 --2- 172.21.15.26:0/1890726096 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f1acc0a2ed0 0x7f1acc0a5390 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:24.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.714+0000 7f1ae9575640 1 --2- 172.21.15.26:0/1890726096 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1ae4102ef0 0x7f1ae410e710 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:24.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.714+0000 7f1ae9575640 1 --2- 172.21.15.26:0/1890726096 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ae410fe60 0x7f1ae410ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:24.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.714+0000 7f1ae9575640 1 --2- 172.21.15.26:0/1890726096 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1ae4100e20 0x7f1ae410e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:24.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.714+0000 7f1ae9575640 1 -- 172.21.15.26:0/1890726096 >> 172.21.15.26:0/1890726096 conn(0x7f1ae40fc7d0 msgr2=0x7f1ae40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:19:24.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.715+0000 7f1ae9575640 1 -- 172.21.15.26:0/1890726096 shutdown_connections 2024-09-11T23:19:24.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:24.715+0000 7f1ae9575640 1 -- 172.21.15.26:0/1890726096 wait complete. 2024-09-11T23:19:25.617 INFO:tasks.cephfs.filesystem:scrub status for tag:6adeb6f6-995b-4377-9a31-baeabff7f707 - {'path': '/', 'tag': '6adeb6f6-995b-4377-9a31-baeabff7f707', 'options': 'recursive,force'} 2024-09-11T23:19:26.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:19:25 smithi136 ceph-mon[35121]: pgmap v34: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 2.5 KiB/s wr, 15 op/s 2024-09-11T23:19:26.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:19:25 smithi026 ceph-mon[31172]: pgmap v34: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 2.5 KiB/s wr, 15 op/s 2024-09-11T23:19:26.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:19:25 smithi076 ceph-mon[35296]: pgmap v34: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 2.5 KiB/s wr, 15 op/s 2024-09-11T23:19:27.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:19:26 smithi136 ceph-mon[35121]: scrub summary: idle 2024-09-11T23:19:27.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:19:26 smithi026 ceph-mon[31172]: scrub summary: idle 2024-09-11T23:19:27.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:19:26 smithi076 ceph-mon[35296]: scrub summary: idle 2024-09-11T23:19:28.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:19:27 smithi136 ceph-mon[35121]: pgmap v35: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.3 KiB/s wr, 9 op/s 2024-09-11T23:19:28.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:19:27 smithi026 ceph-mon[31172]: pgmap v35: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.3 KiB/s wr, 9 op/s 2024-09-11T23:19:28.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:19:27 smithi076 ceph-mon[35296]: pgmap v35: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.3 KiB/s wr, 9 op/s 2024-09-11T23:19:30.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:19:29 smithi136 ceph-mon[35121]: pgmap v36: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 2.6 KiB/s wr, 15 op/s 2024-09-11T23:19:30.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:19:29 smithi026 ceph-mon[31172]: pgmap v36: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 2.6 KiB/s wr, 15 op/s 2024-09-11T23:19:30.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:19:29 smithi076 ceph-mon[35296]: pgmap v36: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 2.6 KiB/s wr, 15 op/s 2024-09-11T23:19:31.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:19:30 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:19:31.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:19:30 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:19:31.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:19:30 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:19:32.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:19:31 smithi136 ceph-mon[35121]: pgmap v37: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-09-11T23:19:32.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:19:31 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:19:32.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:19:31 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:19:32.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:19:31 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:19:32.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:19:31 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:19:32.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:19:31 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:19:32.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:19:31 smithi026 ceph-mon[31172]: pgmap v37: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-09-11T23:19:32.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:19:31 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:19:32.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:19:31 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:19:32.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:19:31 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:19:32.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:19:31 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:19:32.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:19:31 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:19:32.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:19:31 smithi076 ceph-mon[35296]: pgmap v37: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-09-11T23:19:32.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:19:31 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:19:32.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:19:31 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:19:32.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:19:31 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:19:32.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:19:31 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:19:32.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:19:31 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:19:32.429 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:19:32.760 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:19:33.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:33.260+0000 7f5c047d1640 1 -- 172.21.15.26:0/2189077767 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5bfc102570 msgr2=0x7f5bfc1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:33.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:33.260+0000 7f5c047d1640 1 --2- 172.21.15.26:0/2189077767 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5bfc102570 0x7f5bfc1029b0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f5bec009900 tx=0x7f5bec02a480 comp rx=0 tx=0).stop 2024-09-11T23:19:33.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:33.260+0000 7f5c047d1640 1 -- 172.21.15.26:0/2189077767 shutdown_connections 2024-09-11T23:19:33.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:33.260+0000 7f5c047d1640 1 --2- 172.21.15.26:0/2189077767 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5bfc102ef0 0x7f5bfc10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:33.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:33.260+0000 7f5c047d1640 1 --2- 172.21.15.26:0/2189077767 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5bfc102570 0x7f5bfc1029b0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:33.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:33.260+0000 7f5c047d1640 1 --2- 172.21.15.26:0/2189077767 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5bfc100e20 0x7f5bfc101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:33.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:33.260+0000 7f5c047d1640 1 -- 172.21.15.26:0/2189077767 >> 172.21.15.26:0/2189077767 conn(0x7f5bfc0fc7d0 msgr2=0x7f5bfc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:19:33.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:33.260+0000 7f5c047d1640 1 -- 172.21.15.26:0/2189077767 shutdown_connections 2024-09-11T23:19:33.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:33.260+0000 7f5c047d1640 1 -- 172.21.15.26:0/2189077767 wait complete. 2024-09-11T23:19:33.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:33.261+0000 7f5c047d1640 1 Processor -- start 2024-09-11T23:19:33.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:33.261+0000 7f5c047d1640 1 -- start start 2024-09-11T23:19:33.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:33.261+0000 7f5c047d1640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5bfc100e20 0x7f5bfc199360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:33.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:33.261+0000 7f5c047d1640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5bfc102570 0x7f5bfc1998a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:33.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:33.261+0000 7f5c047d1640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5bfc102ef0 0x7f5bfc1a43e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:33.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:33.261+0000 7f5c01d6c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5bfc102570 0x7f5bfc1998a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:33.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:33.261+0000 7f5c01d6c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5bfc102570 0x7f5bfc1998a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52468/0 (socket says 172.21.15.26:52468) 2024-09-11T23:19:33.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:33.262+0000 7f5c047d1640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f5bfc110f50 con 0x7f5bfc102570 2024-09-11T23:19:33.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:33.262+0000 7f5c047d1640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f5bfc110dd0 con 0x7f5bfc102ef0 2024-09-11T23:19:33.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:33.262+0000 7f5c047d1640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f5bfc1110d0 con 0x7f5bfc100e20 2024-09-11T23:19:33.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:33.262+0000 7f5c01d6c640 1 -- 172.21.15.26:0/332391003 learned_addr learned my addr 172.21.15.26:0/332391003 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:19:33.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:33.262+0000 7f5c0256d640 1 --2- 172.21.15.26:0/332391003 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5bfc100e20 0x7f5bfc199360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:33.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:33.262+0000 7f5c02d6e640 1 --2- 172.21.15.26:0/332391003 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5bfc102ef0 0x7f5bfc1a43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:33.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:33.262+0000 7f5c01d6c640 1 -- 172.21.15.26:0/332391003 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5bfc100e20 msgr2=0x7f5bfc199360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:33.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:33.262+0000 7f5c01d6c640 1 --2- 172.21.15.26:0/332391003 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5bfc100e20 0x7f5bfc199360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:33.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:33.262+0000 7f5c01d6c640 1 -- 172.21.15.26:0/332391003 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5bfc102ef0 msgr2=0x7f5bfc1a43e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:33.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:33.262+0000 7f5c01d6c640 1 --2- 172.21.15.26:0/332391003 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5bfc102ef0 0x7f5bfc1a43e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:33.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:33.262+0000 7f5c01d6c640 1 -- 172.21.15.26:0/332391003 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bfc1a4920 con 0x7f5bfc102570 2024-09-11T23:19:33.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:33.262+0000 7f5c02d6e640 1 --2- 172.21.15.26:0/332391003 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5bfc102ef0 0x7f5bfc1a43e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:19:33.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:33.262+0000 7f5c01d6c640 1 --2- 172.21.15.26:0/332391003 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5bfc102570 0x7f5bfc1998a0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f5bec006010 tx=0x7f5bec003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:19:33.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:33.262+0000 7f5bf37fe640 1 -- 172.21.15.26:0/332391003 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bec03f070 con 0x7f5bfc102570 2024-09-11T23:19:33.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:33.263+0000 7f5c047d1640 1 -- 172.21.15.26:0/332391003 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bfc1a4af0 con 0x7f5bfc102570 2024-09-11T23:19:33.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:33.263+0000 7f5bf37fe640 1 -- 172.21.15.26:0/332391003 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5bec031c30 con 0x7f5bfc102570 2024-09-11T23:19:33.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:33.263+0000 7f5c047d1640 1 -- 172.21.15.26:0/332391003 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bfc1a4fd0 con 0x7f5bfc102570 2024-09-11T23:19:33.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:33.263+0000 7f5bf37fe640 1 -- 172.21.15.26:0/332391003 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bec035030 con 0x7f5bfc102570 2024-09-11T23:19:33.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:33.265+0000 7f5c047d1640 1 -- 172.21.15.26:0/332391003 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5bfc101200 con 0x7f5bfc102570 2024-09-11T23:19:33.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:33.266+0000 7f5bf37fe640 1 -- 172.21.15.26:0/332391003 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 25) ==== 138066+0+0 (secure 0 0 0) 0x7f5bec042070 con 0x7f5bfc102570 2024-09-11T23:19:33.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:33.267+0000 7f5bf37fe640 1 --2- 172.21.15.26:0/332391003 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f5be80a2e90 0x7f5be80a5350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:33.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:33.267+0000 7f5bf37fe640 1 -- 172.21.15.26:0/332391003 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f5bec0efe50 con 0x7f5bfc102570 2024-09-11T23:19:33.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:33.267+0000 7f5c0256d640 1 --2- 172.21.15.26:0/332391003 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f5be80a2e90 0x7f5be80a5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:33.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:33.268+0000 7f5c0256d640 1 --2- 172.21.15.26:0/332391003 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f5be80a2e90 0x7f5be80a5350 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f5bf8005fd0 tx=0x7f5bf8005850 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:19:33.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:33.272+0000 7f5bf37fe640 1 -- 172.21.15.26:0/332391003 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5bec0bc8a0 con 0x7f5bfc102570 2024-09-11T23:19:33.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:33.699+0000 7f5c047d1640 1 -- 172.21.15.26:0/332391003 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5bfc109090 con 0x7f5bfc102570 2024-09-11T23:19:33.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:33.700+0000 7f5bf37fe640 1 -- 172.21.15.26:0/332391003 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11971 (secure 0 0 0) 0x7f5bec0bc1a0 con 0x7f5bfc102570 2024-09-11T23:19:33.701 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-11T23:19:33.701 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 29 2024-09-11T23:19:33.702 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":29,"btime":"2024-09-11T23:18:21:019324+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14499,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/367659515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":367659515},{"type":"v1","addr":"172.21.15.26:6835","nonce":367659515}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14511,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/2041065299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":2041065299},{"type":"v1","addr":"172.21.15.26:6837","nonce":2041065299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:16:26.544203+0000","modified":"2024-09-11T23:18:20.969151+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24485,"mds_1":24463,"mds_2":24458,"mds_3":24448,"mds_4":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14550":{"gid":14550,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6839/2233068305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":2233068305},{"type":"v1","addr":"172.21.15.26:6839","nonce":2233068305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14562":{"gid":14562,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6841/2984982336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":2984982336},{"type":"v1","addr":"172.21.15.26:6841","nonce":2984982336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6833/2019841455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":2019841455},{"type":"v1","addr":"172.21.15.136:6833","nonce":2019841455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.76:6833/3438147050","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3438147050},{"type":"v1","addr":"172.21.15.76:6833","nonce":3438147050}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6835/3617665975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3617665975},{"type":"v1","addr":"172.21.15.136:6835","nonce":3617665975}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":12,"addr":"172.21.15.76:6835/393308499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":393308499},{"type":"v1","addr":"172.21.15.76:6835","nonce":393308499}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":10,"addr":"172.21.15.136:6837/1633082387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":1633082387},{"type":"v1","addr":"172.21.15.136:6837","nonce":1633082387}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.76:6837/2078978748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":2078978748},{"type":"v1","addr":"172.21.15.76:6837","nonce":2078978748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.136:6839/3211398630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":3211398630},{"type":"v1","addr":"172.21.15.136:6839","nonce":3211398630}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.76:6839/2077637090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6838","nonce":2077637090},{"type":"v1","addr":"172.21.15.76:6839","nonce":2077637090}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24485,"qdb_cluster":[24443,24448,24458,24463,24485]},"id":1}]} 2024-09-11T23:19:33.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:33.703+0000 7f5c047d1640 1 -- 172.21.15.26:0/332391003 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f5be80a2e90 msgr2=0x7f5be80a5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:33.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:33.703+0000 7f5c047d1640 1 --2- 172.21.15.26:0/332391003 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f5be80a2e90 0x7f5be80a5350 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f5bf8005fd0 tx=0x7f5bf8005850 comp rx=0 tx=0).stop 2024-09-11T23:19:33.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:33.703+0000 7f5c047d1640 1 -- 172.21.15.26:0/332391003 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5bfc102570 msgr2=0x7f5bfc1998a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:33.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:33.703+0000 7f5c047d1640 1 --2- 172.21.15.26:0/332391003 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5bfc102570 0x7f5bfc1998a0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f5bec006010 tx=0x7f5bec003c10 comp rx=0 tx=0).stop 2024-09-11T23:19:33.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:33.703+0000 7f5c047d1640 1 -- 172.21.15.26:0/332391003 shutdown_connections 2024-09-11T23:19:33.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:33.703+0000 7f5c047d1640 1 --2- 172.21.15.26:0/332391003 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f5be80a2e90 0x7f5be80a5350 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:33.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:33.703+0000 7f5c047d1640 1 --2- 172.21.15.26:0/332391003 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5bfc102ef0 0x7f5bfc1a43e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:33.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:33.703+0000 7f5c047d1640 1 --2- 172.21.15.26:0/332391003 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5bfc102570 0x7f5bfc1998a0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:33.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:33.703+0000 7f5c047d1640 1 --2- 172.21.15.26:0/332391003 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5bfc100e20 0x7f5bfc199360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:33.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:33.703+0000 7f5c047d1640 1 -- 172.21.15.26:0/332391003 >> 172.21.15.26:0/332391003 conn(0x7f5bfc0fc7d0 msgr2=0x7f5bfc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:19:33.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:33.703+0000 7f5c047d1640 1 -- 172.21.15.26:0/332391003 shutdown_connections 2024-09-11T23:19:33.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:33.703+0000 7f5c047d1640 1 -- 172.21.15.26:0/332391003 wait complete. 2024-09-11T23:19:34.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:19:33 smithi026 ceph-mon[31172]: pgmap v38: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-09-11T23:19:34.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:19:33 smithi026 ceph-mon[31172]: from='client.? 172.21.15.26:0/332391003' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:19:34.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:19:33 smithi136 ceph-mon[35121]: pgmap v38: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-09-11T23:19:34.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:19:33 smithi136 ceph-mon[35121]: from='client.? 172.21.15.26:0/332391003' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:19:34.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:19:33 smithi076 ceph-mon[35296]: pgmap v38: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-09-11T23:19:34.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:19:33 smithi076 ceph-mon[35296]: from='client.? 172.21.15.26:0/332391003' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:19:34.533 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24485 quiesce db --set-id 37f40e7f --release --await' 2024-09-11T23:19:34.533 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24485 quiesce db --set-id 37f40e7f --release --await 2024-09-11T23:19:34.941 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:19:35.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.532+0000 7f3a1db00640 1 -- 172.21.15.26:0/3071771944 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a181082a0 msgr2=0x7f3a1810ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:35.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.532+0000 7f3a1db00640 1 --2- 172.21.15.26:0/3071771944 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a181082a0 0x7f3a1810ec40 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f3a10009cd0 tx=0x7f3a1002a8e0 comp rx=0 tx=0).stop 2024-09-11T23:19:35.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.532+0000 7f3a1db00640 1 -- 172.21.15.26:0/3071771944 shutdown_connections 2024-09-11T23:19:35.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.532+0000 7f3a1db00640 1 --2- 172.21.15.26:0/3071771944 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a181082a0 0x7f3a1810ec40 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:35.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.532+0000 7f3a1db00640 1 --2- 172.21.15.26:0/3071771944 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3a181036e0 0x7f3a18107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:35.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.532+0000 7f3a1db00640 1 --2- 172.21.15.26:0/3071771944 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3a18102d30 0x7f3a18103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:35.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.532+0000 7f3a1db00640 1 -- 172.21.15.26:0/3071771944 >> 172.21.15.26:0/3071771944 conn(0x7f3a180fc7d0 msgr2=0x7f3a180febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:19:35.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.533+0000 7f3a1db00640 1 -- 172.21.15.26:0/3071771944 shutdown_connections 2024-09-11T23:19:35.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.533+0000 7f3a1db00640 1 -- 172.21.15.26:0/3071771944 wait complete. 2024-09-11T23:19:35.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.533+0000 7f3a1db00640 1 Processor -- start 2024-09-11T23:19:35.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.534+0000 7f3a1db00640 1 -- start start 2024-09-11T23:19:35.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.534+0000 7f3a1db00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3a18102d30 0x7f3a181985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:35.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.534+0000 7f3a1db00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a181036e0 0x7f3a18198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:35.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.534+0000 7f3a1db00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3a181082a0 0x7f3a1819ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:35.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.534+0000 7f3a1db00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f3a181120a0 con 0x7f3a181036e0 2024-09-11T23:19:35.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.534+0000 7f3a1db00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f3a18111f20 con 0x7f3a18102d30 2024-09-11T23:19:35.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.534+0000 7f3a1db00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f3a18112220 con 0x7f3a181082a0 2024-09-11T23:19:35.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.534+0000 7f3a16ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a181036e0 0x7f3a18198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:35.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.534+0000 7f3a16ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a181036e0 0x7f3a18198ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52496/0 (socket says 172.21.15.26:52496) 2024-09-11T23:19:35.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.534+0000 7f3a16ffd640 1 -- 172.21.15.26:0/405804834 learned_addr learned my addr 172.21.15.26:0/405804834 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:19:35.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.534+0000 7f3a17fff640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3a181082a0 0x7f3a1819ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:35.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.535+0000 7f3a16ffd640 1 -- 172.21.15.26:0/405804834 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3a181082a0 msgr2=0x7f3a1819ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:35.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.535+0000 7f3a16ffd640 1 --2- 172.21.15.26:0/405804834 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3a181082a0 0x7f3a1819ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:35.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.535+0000 7f3a16ffd640 1 -- 172.21.15.26:0/405804834 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3a18102d30 msgr2=0x7f3a181985a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:35.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.535+0000 7f3a16ffd640 1 --2- 172.21.15.26:0/405804834 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3a18102d30 0x7f3a181985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:35.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.535+0000 7f3a16ffd640 1 -- 172.21.15.26:0/405804834 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a1819d550 con 0x7f3a181036e0 2024-09-11T23:19:35.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.535+0000 7f3a16ffd640 1 --2- 172.21.15.26:0/405804834 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a181036e0 0x7f3a18198ae0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f3a0c00d750 tx=0x7f3a0c00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:19:35.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.535+0000 7f3a14ff9640 1 -- 172.21.15.26:0/405804834 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a0c017070 con 0x7f3a181036e0 2024-09-11T23:19:35.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.535+0000 7f3a14ff9640 1 -- 172.21.15.26:0/405804834 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a0c005500 con 0x7f3a181036e0 2024-09-11T23:19:35.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.535+0000 7f3a1db00640 1 -- 172.21.15.26:0/405804834 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a1819d840 con 0x7f3a181036e0 2024-09-11T23:19:35.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.535+0000 7f3a1db00640 1 -- 172.21.15.26:0/405804834 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a181a5120 con 0x7f3a181036e0 2024-09-11T23:19:35.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.536+0000 7f3a14ff9640 1 -- 172.21.15.26:0/405804834 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a0c004e00 con 0x7f3a181036e0 2024-09-11T23:19:35.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.539+0000 7f3a1fed9740 1 Processor -- start 2024-09-11T23:19:35.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.539+0000 7f3a14ff9640 1 -- 172.21.15.26:0/405804834 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 25) ==== 138066+0+0 (secure 0 0 0) 0x7f3a0c010460 con 0x7f3a181036e0 2024-09-11T23:19:35.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.539+0000 7f3a1fed9740 1 -- start start 2024-09-11T23:19:35.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.539+0000 7f3a1fed9740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564bfdaa0f20 0x564bfdaa1300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:35.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.539+0000 7f3a1fed9740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564bfdaa1840 0x564bfdab0110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:35.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.539+0000 7f3a1fed9740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564bfdab0650 0x564bfdab2a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:35.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.539+0000 7f3a1fed9740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x564bfda06620 con 0x564bfdab0650 2024-09-11T23:19:35.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.539+0000 7f3a1fed9740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x564bfd9ade50 con 0x564bfdaa0f20 2024-09-11T23:19:35.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.539+0000 7f3a1fed9740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x564bfda025e0 con 0x564bfdaa1840 2024-09-11T23:19:35.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.540+0000 7f3a177fe640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564bfdaa1840 0x564bfdab0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:35.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.540+0000 7f3a177fe640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564bfdaa1840 0x564bfdab0110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:58958/0 (socket says 172.21.15.26:58958) 2024-09-11T23:19:35.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.540+0000 7f3a177fe640 1 -- 172.21.15.26:0/1860463830 learned_addr learned my addr 172.21.15.26:0/1860463830 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:19:35.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.540+0000 7f3a17fff640 1 --2- 172.21.15.26:0/1860463830 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564bfdaa0f20 0x564bfdaa1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:35.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.540+0000 7f3a16ffd640 1 --2- 172.21.15.26:0/1860463830 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564bfdab0650 0x564bfdab2a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:35.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.540+0000 7f3a177fe640 1 -- 172.21.15.26:0/1860463830 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564bfdaa0f20 msgr2=0x564bfdaa1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:19:35.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.540+0000 7f3a177fe640 1 --2- 172.21.15.26:0/1860463830 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564bfdaa0f20 0x564bfdaa1300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:35.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.540+0000 7f3a177fe640 1 -- 172.21.15.26:0/1860463830 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564bfdab0650 msgr2=0x564bfdab2a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:19:35.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.540+0000 7f3a177fe640 1 --2- 172.21.15.26:0/1860463830 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564bfdab0650 0x564bfdab2a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:35.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.540+0000 7f3a177fe640 1 -- 172.21.15.26:0/1860463830 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564bfd99ed40 con 0x564bfdaa1840 2024-09-11T23:19:35.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.540+0000 7f3a177fe640 1 --2- 172.21.15.26:0/1860463830 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564bfdaa1840 0x564bfdab0110 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f3a08009c40 tx=0x7f3a0800e720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ea9c3a5f430b3f3f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:19:35.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.541+0000 7f39fdffb640 1 -- 172.21.15.26:0/1860463830 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a08005020 con 0x564bfdaa1840 2024-09-11T23:19:35.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.541+0000 7f39fdffb640 1 -- 172.21.15.26:0/1860463830 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a080047a0 con 0x564bfdaa1840 2024-09-11T23:19:35.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.541+0000 7f3a14ff9640 1 --2- 172.21.15.26:0/405804834 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f3a000a2e90 0x7f3a000a5350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:35.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.541+0000 7f3a14ff9640 1 -- 172.21.15.26:0/405804834 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f3a0c0cf000 con 0x7f3a181036e0 2024-09-11T23:19:35.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.541+0000 7f3a17fff640 1 --2- 172.21.15.26:0/405804834 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f3a000a2e90 0x7f3a000a5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:35.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.541+0000 7f39fdffb640 1 -- 172.21.15.26:0/1860463830 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a08010720 con 0x564bfdaa1840 2024-09-11T23:19:35.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.542+0000 7f3a1fed9740 1 -- 172.21.15.26:0/1860463830 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564bfdaa1840 msgr2=0x564bfdab0110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:35.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.542+0000 7f3a1fed9740 1 --2- 172.21.15.26:0/1860463830 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564bfdaa1840 0x564bfdab0110 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f3a08009c40 tx=0x7f3a0800e720 comp rx=0 tx=0).stop 2024-09-11T23:19:35.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.542+0000 7f3a1fed9740 1 -- 172.21.15.26:0/1860463830 shutdown_connections 2024-09-11T23:19:35.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.542+0000 7f3a1fed9740 1 --2- 172.21.15.26:0/1860463830 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564bfdab0650 0x564bfdab2a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:35.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.542+0000 7f3a1fed9740 1 --2- 172.21.15.26:0/1860463830 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564bfdaa1840 0x564bfdab0110 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:35.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.542+0000 7f3a1fed9740 1 --2- 172.21.15.26:0/1860463830 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564bfdaa0f20 0x564bfdaa1300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:35.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.542+0000 7f3a1fed9740 1 -- 172.21.15.26:0/1860463830 >> 172.21.15.26:0/1860463830 conn(0x564bfda9d5f0 msgr2=0x564bfda9d9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:19:35.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.542+0000 7f3a1fed9740 1 -- 172.21.15.26:0/1860463830 shutdown_connections 2024-09-11T23:19:35.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.542+0000 7f3a1fed9740 1 -- 172.21.15.26:0/1860463830 wait complete. 2024-09-11T23:19:35.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.542+0000 7f3a17fff640 1 --2- 172.21.15.26:0/405804834 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f3a000a2e90 0x7f3a000a5350 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3a10004ab0 tx=0x7f3a10004a40 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:19:35.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.542+0000 7f3a1fed9740 1 -- start start 2024-09-11T23:19:35.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.543+0000 7f3a1fed9740 1 Processor -- start 2024-09-11T23:19:35.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.543+0000 7f3a1fed9740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564bfdaa0f20 0x564bfdaad4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:35.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.543+0000 7f3a1fed9740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564bfdaa1840 0x564bfdaad9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:35.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.543+0000 7f3a1fed9740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564bfdab0650 0x564bfdaadf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:35.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.543+0000 7f3a1fed9740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x564bfdab3270 con 0x564bfdab0650 2024-09-11T23:19:35.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.543+0000 7f3a1fed9740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x564bfd98ef10 con 0x564bfdaa0f20 2024-09-11T23:19:35.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.543+0000 7f3a1fed9740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x564bfdab33f0 con 0x564bfdaa1840 2024-09-11T23:19:35.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.543+0000 7f3a1fed9740 10 client.0 _ll_register_callbacks cb 0x564bfd9b2db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:19:35.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.543+0000 7f3a177fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564bfdaa0f20 0x564bfdaad4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:35.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.543+0000 7f3a177fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564bfdaa0f20 0x564bfdaad4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:55068/0 (socket says 172.21.15.26:55068) 2024-09-11T23:19:35.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.543+0000 7f3a177fe640 1 -- 172.21.15.26:0/1756126685 learned_addr learned my addr 172.21.15.26:0/1756126685 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:19:35.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.543+0000 7f3a16ffd640 1 --2- 172.21.15.26:0/1756126685 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564bfdaa1840 0x564bfdaad9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:35.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.543+0000 7f3a17fff640 1 --2- 172.21.15.26:0/1756126685 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564bfdab0650 0x564bfdaadf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:35.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.544+0000 7f3a177fe640 1 -- 172.21.15.26:0/1756126685 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564bfdaa1840 msgr2=0x564bfdaad9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:19:35.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.544+0000 7f3a177fe640 1 --2- 172.21.15.26:0/1756126685 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564bfdaa1840 0x564bfdaad9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:35.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.544+0000 7f3a177fe640 1 -- 172.21.15.26:0/1756126685 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564bfdab0650 msgr2=0x564bfdaadf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:19:35.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.544+0000 7f3a177fe640 1 --2- 172.21.15.26:0/1756126685 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564bfdab0650 0x564bfdaadf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:35.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.544+0000 7f3a177fe640 1 -- 172.21.15.26:0/1756126685 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564bfd715580 con 0x564bfdaa0f20 2024-09-11T23:19:35.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.544+0000 7f3a17fff640 1 --2- 172.21.15.26:0/1756126685 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564bfdab0650 0x564bfdaadf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:19:35.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.544+0000 7f3a177fe640 1 --2- 172.21.15.26:0/1756126685 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564bfdaa0f20 0x564bfdaad4b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f3a08004f60 tx=0x7f3a08004c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a33abd3fbdd4e77a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:19:35.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.544+0000 7f39f6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-11T23:19:35.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.544+0000 7f39f6ffd640 1 -- 172.21.15.26:0/1756126685 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a0801d040 con 0x564bfdaa0f20 2024-09-11T23:19:35.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.544+0000 7f3a1fed9740 1 -- 172.21.15.26:0/1756126685 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3a080047a0 con 0x564bfdaa0f20 2024-09-11T23:19:35.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.544+0000 7f39f6ffd640 1 -- 172.21.15.26:0/1756126685 <== mon.2 v2:172.21.15.76:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f3a08004250 con 0x564bfdaa0f20 2024-09-11T23:19:35.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.544+0000 7f39f6ffd640 1 -- 172.21.15.26:0/1756126685 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f3a08030050 con 0x564bfdaa0f20 2024-09-11T23:19:35.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.544+0000 7f3a1fed9740 10 client.25343 fetch_fsmap learned FSMap version 29 2024-09-11T23:19:35.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.544+0000 7f3a1fed9740 1 -- 172.21.15.26:0/1756126685 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564bfd7df960 con 0x564bfdaa0f20 2024-09-11T23:19:35.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.545+0000 7f39f6ffd640 1 -- 172.21.15.26:0/1756126685 <== mon.2 v2:172.21.15.76:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a080047a0 con 0x564bfdaa0f20 2024-09-11T23:19:35.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.545+0000 7f39f6ffd640 1 -- 172.21.15.26:0/1756126685 <== mon.2 v2:172.21.15.76:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f3a080039d0 con 0x564bfdaa0f20 2024-09-11T23:19:35.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.545+0000 7f3a1fed9740 10 client.25343 fetch_fsmap finished waiting for FSMap version 29 2024-09-11T23:19:35.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.545+0000 7f3a1fed9740 10 client.25343 resolve_mds: validated gid 24485 aka daemon mds.b 2024-09-11T23:19:35.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.545+0000 7f3a1fed9740 20 client.25343 populate_metadata read hostname 'smithi026' 2024-09-11T23:19:35.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.545+0000 7f3a1fed9740 1 --2- 172.21.15.26:0/1756126685 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x564bfdac00b0 0x564bfdac24a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:35.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.545+0000 7f3a1fed9740 4 client.25343 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:19:35.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.545+0000 7f3a1fed9740 1 -- 172.21.15.26:0/1756126685 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564bfda13b90 con 0x564bfdac00b0 2024-09-11T23:19:35.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.545+0000 7f3a16ffd640 1 --2- 172.21.15.26:0/1756126685 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x564bfdac00b0 0x564bfdac24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:35.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.546+0000 7f39f6ffd640 10 client.25343 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:19:35.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.546+0000 7f3a16ffd640 1 --2- 172.21.15.26:0/1756126685 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x564bfdac00b0 0x564bfdac24a0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ac0ea28d398c313 server_cookie=d3efd949ff891746 in_seq=0 out_seq=0 2024-09-11T23:19:35.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.546+0000 7f39f6ffd640 1 -- 172.21.15.26:0/1756126685 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564bfda13b90 con 0x564bfdac00b0 2024-09-11T23:19:35.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.546+0000 7f39f6ffd640 10 client.25343 handle_command_reply: tid=0 2024-09-11T23:19:35.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.546+0000 7f3a1fed9740 1 client.25343 shutdown 2024-09-11T23:19:35.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.546+0000 7f3a1fed9740 1 -- 172.21.15.26:0/1756126685 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564bfdaa0f20 msgr2=0x564bfdaad4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:35.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.546+0000 7f3a1fed9740 1 --2- 172.21.15.26:0/1756126685 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564bfdaa0f20 0x564bfdaad4b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f3a08004f60 tx=0x7f3a08004c00 comp rx=0 tx=0).stop 2024-09-11T23:19:35.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.547+0000 7f3a1fed9740 1 -- 172.21.15.26:0/1756126685 shutdown_connections 2024-09-11T23:19:35.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.547+0000 7f3a1fed9740 1 --2- 172.21.15.26:0/1756126685 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x564bfdac00b0 0x564bfdac24a0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:35.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.547+0000 7f3a1fed9740 1 --2- 172.21.15.26:0/1756126685 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564bfdab0650 0x564bfdaadf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:35.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.547+0000 7f3a1fed9740 1 --2- 172.21.15.26:0/1756126685 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564bfdaa1840 0x564bfdaad9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:35.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.547+0000 7f3a1fed9740 1 --2- 172.21.15.26:0/1756126685 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564bfdaa0f20 0x564bfdaad4b0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:35.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.547+0000 7f39f6ffd640 0 client.25343 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:19:35.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.547+0000 7f3a1fed9740 1 -- 172.21.15.26:0/1756126685 >> 172.21.15.26:0/1756126685 conn(0x564bfda9d5f0 msgr2=0x564bfdaa3350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:19:35.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.547+0000 7f3a1fed9740 1 -- 172.21.15.26:0/1756126685 shutdown_connections 2024-09-11T23:19:35.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.547+0000 7f3a1fed9740 1 -- 172.21.15.26:0/1756126685 wait complete. 2024-09-11T23:19:35.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.547+0000 7f3a1fed9740 20 client.25343 trim_cache size 0 max 16384 2024-09-11T23:19:35.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.565+0000 7f3a1fed9740 1 Processor -- start 2024-09-11T23:19:35.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.565+0000 7f3a1fed9740 1 -- start start 2024-09-11T23:19:35.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.565+0000 7f3a1fed9740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564bfdaa0f20 0x564bfdaaed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:35.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.565+0000 7f3a1fed9740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564bfdaa1840 0x564bfdaa30d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:35.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.565+0000 7f3a1fed9740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564bfdab0650 0x564bfdaa3610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:35.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.565+0000 7f3a1fed9740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f3a181ab730 con 0x564bfdaa0f20 2024-09-11T23:19:35.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.565+0000 7f3a1fed9740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f3a181ad270 con 0x564bfdaa1840 2024-09-11T23:19:35.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.565+0000 7f3a1fed9740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f3a181abe00 con 0x564bfdab0650 2024-09-11T23:19:35.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.565+0000 7f3a177fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564bfdaa0f20 0x564bfdaaed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:35.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.565+0000 7f3a17fff640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564bfdab0650 0x564bfdaa3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:35.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.565+0000 7f3a17fff640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564bfdab0650 0x564bfdaa3610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:58972/0 (socket says 172.21.15.26:58972) 2024-09-11T23:19:35.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.565+0000 7f3a17fff640 1 -- 172.21.15.26:0/2412719324 learned_addr learned my addr 172.21.15.26:0/2412719324 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:19:35.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.565+0000 7f3a177fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564bfdaa0f20 0x564bfdaaed10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52526/0 (socket says 172.21.15.26:52526) 2024-09-11T23:19:35.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.565+0000 7f3a16ffd640 1 --2- 172.21.15.26:0/2412719324 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564bfdaa1840 0x564bfdaa30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:35.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.565+0000 7f3a177fe640 1 -- 172.21.15.26:0/2412719324 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564bfdab0650 msgr2=0x564bfdaa3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:19:35.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.565+0000 7f3a177fe640 1 --2- 172.21.15.26:0/2412719324 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564bfdab0650 0x564bfdaa3610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:35.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.565+0000 7f3a177fe640 1 -- 172.21.15.26:0/2412719324 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564bfdaa1840 msgr2=0x564bfdaa30d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:19:35.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.565+0000 7f3a177fe640 1 --2- 172.21.15.26:0/2412719324 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564bfdaa1840 0x564bfdaa30d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:35.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.565+0000 7f3a177fe640 1 -- 172.21.15.26:0/2412719324 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564bfda9c610 con 0x564bfdaa0f20 2024-09-11T23:19:35.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.565+0000 7f3a16ffd640 1 --2- 172.21.15.26:0/2412719324 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564bfdaa1840 0x564bfdaa30d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:19:35.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.566+0000 7f3a177fe640 1 --2- 172.21.15.26:0/2412719324 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564bfdaa0f20 0x564bfdaaed10 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f3a0800f010 tx=0x7f3a08005a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae2033724fa53666 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:19:35.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.566+0000 7f39fd7fa640 1 -- 172.21.15.26:0/2412719324 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a080184b0 con 0x564bfdaa0f20 2024-09-11T23:19:35.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.566+0000 7f39fd7fa640 1 -- 172.21.15.26:0/2412719324 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a08039070 con 0x564bfdaa0f20 2024-09-11T23:19:35.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.566+0000 7f39fd7fa640 1 -- 172.21.15.26:0/2412719324 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a08004510 con 0x564bfdaa0f20 2024-09-11T23:19:35.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.567+0000 7f3a1fed9740 1 -- 172.21.15.26:0/2412719324 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564bfdaa0f20 msgr2=0x564bfdaaed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:35.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.567+0000 7f3a1fed9740 1 --2- 172.21.15.26:0/2412719324 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564bfdaa0f20 0x564bfdaaed10 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f3a0800f010 tx=0x7f3a08005a70 comp rx=0 tx=0).stop 2024-09-11T23:19:35.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.567+0000 7f3a1fed9740 1 -- 172.21.15.26:0/2412719324 shutdown_connections 2024-09-11T23:19:35.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.567+0000 7f3a1fed9740 1 --2- 172.21.15.26:0/2412719324 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564bfdaa1840 0x564bfdaa30d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:35.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.567+0000 7f3a1fed9740 1 --2- 172.21.15.26:0/2412719324 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564bfdab0650 0x564bfdaa3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:35.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.567+0000 7f3a1fed9740 1 --2- 172.21.15.26:0/2412719324 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564bfdaa0f20 0x564bfdaaed10 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:35.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.567+0000 7f3a1fed9740 1 -- 172.21.15.26:0/2412719324 >> 172.21.15.26:0/2412719324 conn(0x564bfda9d5f0 msgr2=0x564bfdaa29c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:19:35.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.567+0000 7f3a1fed9740 1 -- 172.21.15.26:0/2412719324 shutdown_connections 2024-09-11T23:19:35.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.567+0000 7f3a1fed9740 1 -- 172.21.15.26:0/2412719324 wait complete. 2024-09-11T23:19:35.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.568+0000 7f3a1fed9740 1 -- start start 2024-09-11T23:19:35.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.568+0000 7f3a1fed9740 1 Processor -- start 2024-09-11T23:19:35.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.569+0000 7f3a1fed9740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564bfdaa0f20 0x564bfdab4b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:35.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.569+0000 7f3a1fed9740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564bfdaa1840 0x564bfdab5070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:35.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.569+0000 7f3a1fed9740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564bfdab0650 0x564bfdb29b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:35.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.569+0000 7f3a177fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564bfdaa0f20 0x564bfdab4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:35.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.569+0000 7f3a1fed9740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f3a181acba0 con 0x564bfdaa0f20 2024-09-11T23:19:35.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.569+0000 7f3a1fed9740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f3a181ac4d0 con 0x564bfdaa1840 2024-09-11T23:19:35.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.569+0000 7f3a1fed9740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f3a181a9710 con 0x564bfdab0650 2024-09-11T23:19:35.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.569+0000 7f3a177fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564bfdaa0f20 0x564bfdab4b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52530/0 (socket says 172.21.15.26:52530) 2024-09-11T23:19:35.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.569+0000 7f3a177fe640 1 -- 172.21.15.26:0/3433242796 learned_addr learned my addr 172.21.15.26:0/3433242796 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:19:35.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.569+0000 7f3a1fed9740 10 client.0 _ll_register_callbacks cb 0x7f3a08009b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:19:35.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.569+0000 7f3a17fff640 1 --2- 172.21.15.26:0/3433242796 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564bfdab0650 0x564bfdb29b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:35.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.569+0000 7f3a16ffd640 1 --2- 172.21.15.26:0/3433242796 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564bfdaa1840 0x564bfdab5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:35.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.569+0000 7f3a177fe640 1 -- 172.21.15.26:0/3433242796 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564bfdab0650 msgr2=0x564bfdb29b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:19:35.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.569+0000 7f3a177fe640 1 --2- 172.21.15.26:0/3433242796 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564bfdab0650 0x564bfdb29b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:35.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.569+0000 7f3a177fe640 1 -- 172.21.15.26:0/3433242796 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564bfdaa1840 msgr2=0x564bfdab5070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:19:35.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.569+0000 7f3a177fe640 1 --2- 172.21.15.26:0/3433242796 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564bfdaa1840 0x564bfdab5070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:35.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.569+0000 7f3a177fe640 1 -- 172.21.15.26:0/3433242796 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564bfdb2a0a0 con 0x564bfdaa0f20 2024-09-11T23:19:35.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.569+0000 7f3a17fff640 1 --2- 172.21.15.26:0/3433242796 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564bfdab0650 0x564bfdb29b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:19:35.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.569+0000 7f3a16ffd640 1 --2- 172.21.15.26:0/3433242796 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564bfdaa1840 0x564bfdab5070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:19:35.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.570+0000 7f3a177fe640 1 --2- 172.21.15.26:0/3433242796 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564bfdaa0f20 0x564bfdab4b30 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f3a08001eb0 tx=0x7f3a08004e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73f7efd7e5689ba3 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:19:35.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.570+0000 7f39f6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:19:35.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.570+0000 7f39f6ffd640 1 -- 172.21.15.26:0/3433242796 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a08043020 con 0x564bfdaa0f20 2024-09-11T23:19:35.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.570+0000 7f3a1fed9740 1 -- 172.21.15.26:0/3433242796 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3a08039070 con 0x564bfdaa0f20 2024-09-11T23:19:35.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.570+0000 7f39f6ffd640 1 -- 172.21.15.26:0/3433242796 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f3a08004450 con 0x564bfdaa0f20 2024-09-11T23:19:35.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.570+0000 7f39f6ffd640 1 -- 172.21.15.26:0/3433242796 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f3a08018950 con 0x564bfdaa0f20 2024-09-11T23:19:35.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.570+0000 7f3a1fed9740 10 client.15759 fetch_fsmap learned FSMap version 29 2024-09-11T23:19:35.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.571+0000 7f3a1fed9740 1 -- 172.21.15.26:0/3433242796 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564bfdb2b340 con 0x564bfdaa0f20 2024-09-11T23:19:35.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.571+0000 7f39f6ffd640 1 -- 172.21.15.26:0/3433242796 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a08039070 con 0x564bfdaa0f20 2024-09-11T23:19:35.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.571+0000 7f39f6ffd640 1 -- 172.21.15.26:0/3433242796 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f3a08004fb0 con 0x564bfdaa0f20 2024-09-11T23:19:35.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.571+0000 7f3a1fed9740 10 client.15759 fetch_fsmap finished waiting for FSMap version 29 2024-09-11T23:19:35.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.571+0000 7f3a1fed9740 10 client.15759 resolve_mds: validated gid 24485 aka daemon mds.b 2024-09-11T23:19:35.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.571+0000 7f3a1fed9740 20 client.15759 populate_metadata read hostname 'smithi026' 2024-09-11T23:19:35.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.571+0000 7f3a1fed9740 1 --2- 172.21.15.26:0/3433242796 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x564bfdb2b9a0 0x564bfdb2dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:35.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.571+0000 7f3a1fed9740 4 client.15759 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "37f40e7f", "release": true, "await": true}] 2024-09-11T23:19:35.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.571+0000 7f3a1fed9740 1 -- 172.21.15.26:0/3433242796 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "quiesce db", "set_id": "37f40e7f", "release": true, "await": true}) -- 0x564bfda0e0a0 con 0x564bfdb2b9a0 2024-09-11T23:19:35.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.572+0000 7f3a16ffd640 1 --2- 172.21.15.26:0/3433242796 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x564bfdb2b9a0 0x564bfdb2dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:35.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.572+0000 7f3a16ffd640 1 --2- 172.21.15.26:0/3433242796 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x564bfdb2b9a0 0x564bfdb2dd90 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d98a0bc785f1edcc server_cookie=63ed4a0acf7c7547 in_seq=0 out_seq=0 2024-09-11T23:19:35.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.572+0000 7f39f6ffd640 10 client.15759 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:19:35.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.574+0000 7f39f6ffd640 1 -- 172.21.15.26:0/3433242796 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x564bfda0e0a0 con 0x564bfdb2b9a0 2024-09-11T23:19:35.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.574+0000 7f39f6ffd640 10 client.15759 handle_command_reply: tid=0 2024-09-11T23:19:35.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.574+0000 7f3a1fed9740 1 client.15759 shutdown 2024-09-11T23:19:35.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.574+0000 7f3a1fed9740 1 -- 172.21.15.26:0/3433242796 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564bfdaa0f20 msgr2=0x564bfdab4b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:35.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.574+0000 7f3a1fed9740 1 --2- 172.21.15.26:0/3433242796 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564bfdaa0f20 0x564bfdab4b30 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f3a08001eb0 tx=0x7f3a08004e30 comp rx=0 tx=0).stop 2024-09-11T23:19:35.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.574+0000 7f3a1fed9740 1 -- 172.21.15.26:0/3433242796 shutdown_connections 2024-09-11T23:19:35.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.574+0000 7f3a1fed9740 1 --2- 172.21.15.26:0/3433242796 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564bfdaa1840 0x564bfdab5070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:35.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.574+0000 7f3a1fed9740 1 --2- 172.21.15.26:0/3433242796 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x564bfdb2b9a0 0x564bfdb2dd90 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:35.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.574+0000 7f3a1fed9740 1 --2- 172.21.15.26:0/3433242796 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564bfdab0650 0x564bfdb29b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:35.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.574+0000 7f3a1fed9740 1 --2- 172.21.15.26:0/3433242796 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564bfdaa0f20 0x564bfdab4b30 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:35.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.574+0000 7f39f6ffd640 0 client.15759 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:19:35.578 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-11T23:19:35.578 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 29, 2024-09-11T23:19:35.578 INFO:teuthology.orchestra.run.smithi026.stdout: "leader": 24485, 2024-09-11T23:19:35.578 INFO:teuthology.orchestra.run.smithi026.stdout: "set_version": 4, 2024-09-11T23:19:35.579 INFO:teuthology.orchestra.run.smithi026.stdout: "sets": { 2024-09-11T23:19:35.579 INFO:teuthology.orchestra.run.smithi026.stdout: "37f40e7f": { 2024-09-11T23:19:35.579 INFO:teuthology.orchestra.run.smithi026.stdout: "version": 4, 2024-09-11T23:19:35.579 INFO:teuthology.orchestra.run.smithi026.stdout: "age_ref": 0.0, 2024-09-11T23:19:35.579 INFO:teuthology.orchestra.run.smithi026.stdout: "state": { 2024-09-11T23:19:35.579 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "RELEASED", 2024-09-11T23:19:35.579 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.0 2024-09-11T23:19:35.579 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:19:35.579 INFO:teuthology.orchestra.run.smithi026.stdout: "timeout": 90.0, 2024-09-11T23:19:35.579 INFO:teuthology.orchestra.run.smithi026.stdout: "expiration": 144.5, 2024-09-11T23:19:35.579 INFO:teuthology.orchestra.run.smithi026.stdout: "members": { 2024-09-11T23:19:35.579 INFO:teuthology.orchestra.run.smithi026.stdout: "file:/": { 2024-09-11T23:19:35.579 INFO:teuthology.orchestra.run.smithi026.stdout: "excluded": false, 2024-09-11T23:19:35.579 INFO:teuthology.orchestra.run.smithi026.stdout: "state": { 2024-09-11T23:19:35.580 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "RELEASED", 2024-09-11T23:19:35.580 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.0 2024-09-11T23:19:35.580 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:19:35.580 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:19:35.580 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:19:35.580 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:19:35.580 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:19:35.580 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-11T23:19:35.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.574+0000 7f3a1fed9740 1 -- 172.21.15.26:0/3433242796 >> 172.21.15.26:0/3433242796 conn(0x564bfda9d5f0 msgr2=0x564bfdabbd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:19:35.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.574+0000 7f3a1fed9740 1 -- 172.21.15.26:0/3433242796 shutdown_connections 2024-09-11T23:19:35.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.574+0000 7f3a1fed9740 1 -- 172.21.15.26:0/3433242796 wait complete. 2024-09-11T23:19:35.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.575+0000 7f3a1fed9740 20 client.15759 trim_cache size 0 max 16384 2024-09-11T23:19:35.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.575+0000 7f3a1db00640 1 -- 172.21.15.26:0/405804834 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f3a000a2e90 msgr2=0x7f3a000a5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:35.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.575+0000 7f3a1db00640 1 --2- 172.21.15.26:0/405804834 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f3a000a2e90 0x7f3a000a5350 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3a10004ab0 tx=0x7f3a10004a40 comp rx=0 tx=0).stop 2024-09-11T23:19:35.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.575+0000 7f3a1db00640 1 -- 172.21.15.26:0/405804834 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a181036e0 msgr2=0x7f3a18198ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:35.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.575+0000 7f3a1db00640 1 --2- 172.21.15.26:0/405804834 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a181036e0 0x7f3a18198ae0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f3a0c00d750 tx=0x7f3a0c00dc10 comp rx=0 tx=0).stop 2024-09-11T23:19:35.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.575+0000 7f3a1db00640 1 -- 172.21.15.26:0/405804834 shutdown_connections 2024-09-11T23:19:35.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.575+0000 7f3a1db00640 1 --2- 172.21.15.26:0/405804834 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f3a000a2e90 0x7f3a000a5350 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:35.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.575+0000 7f3a1db00640 1 --2- 172.21.15.26:0/405804834 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3a181082a0 0x7f3a1819ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:35.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.575+0000 7f3a1db00640 1 --2- 172.21.15.26:0/405804834 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a181036e0 0x7f3a18198ae0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:35.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.575+0000 7f3a1db00640 1 --2- 172.21.15.26:0/405804834 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3a18102d30 0x7f3a181985a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:35.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.575+0000 7f3a1db00640 1 -- 172.21.15.26:0/405804834 >> 172.21.15.26:0/405804834 conn(0x7f3a180fc7d0 msgr2=0x7f3a18106380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:19:35.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.576+0000 7f3a1db00640 1 -- 172.21.15.26:0/405804834 shutdown_connections 2024-09-11T23:19:35.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:35.576+0000 7f3a1db00640 1 -- 172.21.15.26:0/405804834 wait complete. 2024-09-11T23:19:36.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:19:35 smithi136 ceph-mon[35121]: pgmap v39: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 1.2 KiB/s wr, 15 op/s 2024-09-11T23:19:36.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:19:35 smithi026 ceph-mon[31172]: pgmap v39: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 1.2 KiB/s wr, 15 op/s 2024-09-11T23:19:36.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:19:35 smithi076 ceph-mon[35296]: pgmap v39: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 1.2 KiB/s wr, 15 op/s 2024-09-11T23:19:36.494 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '37f40e7f', total seconds elapsed: 28.6 2024-09-11T23:19:37.811 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.810+0000 7efef8a5e640 1 -- 172.21.15.26:0/3407047967 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efef4101210 msgr2=0x7efef410ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:37.811 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.810+0000 7efef8a5e640 1 --2- 172.21.15.26:0/3407047967 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efef4101210 0x7efef410ded0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7efee8009900 tx=0x7efee802a410 comp rx=0 tx=0).stop 2024-09-11T23:19:37.811 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.810+0000 7efef8a5e640 1 -- 172.21.15.26:0/3407047967 shutdown_connections 2024-09-11T23:19:37.811 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.810+0000 7efef8a5e640 1 --2- 172.21.15.26:0/3407047967 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efef410e410 0x7efef4110800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:37.811 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.810+0000 7efef8a5e640 1 --2- 172.21.15.26:0/3407047967 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efef4101210 0x7efef410ded0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:37.811 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.810+0000 7efef8a5e640 1 --2- 172.21.15.26:0/3407047967 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efef4100860 0x7efef4100c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:37.811 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.810+0000 7efef8a5e640 1 -- 172.21.15.26:0/3407047967 >> 172.21.15.26:0/3407047967 conn(0x7efef4077990 msgr2=0x7efef40fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:19:37.811 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.810+0000 7efef8a5e640 1 -- 172.21.15.26:0/3407047967 shutdown_connections 2024-09-11T23:19:37.811 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.810+0000 7efef8a5e640 1 -- 172.21.15.26:0/3407047967 wait complete. 2024-09-11T23:19:37.813 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.811+0000 7efef8a5e640 1 Processor -- start 2024-09-11T23:19:37.814 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.811+0000 7efef8a5e640 1 -- start start 2024-09-11T23:19:37.814 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.812+0000 7efef8a5e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efef4100860 0x7efef419a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:37.814 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.812+0000 7efef8a5e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efef4101210 0x7efef419aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:37.814 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.812+0000 7efef8a5e640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efef410e410 0x7efef419ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:37.814 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.812+0000 7efef8a5e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7efef41140e0 con 0x7efef4101210 2024-09-11T23:19:37.814 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.812+0000 7efef8a5e640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7efef4113f60 con 0x7efef4100860 2024-09-11T23:19:37.814 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.812+0000 7efef8a5e640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7efef4114260 con 0x7efef410e410 2024-09-11T23:19:37.814 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.812+0000 7efef1d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efef4101210 0x7efef419aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:37.814 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.812+0000 7efef1d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efef4101210 0x7efef419aaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52554/0 (socket says 172.21.15.26:52554) 2024-09-11T23:19:37.814 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.812+0000 7efef1d9b640 1 -- 172.21.15.26:0/1132388294 learned_addr learned my addr 172.21.15.26:0/1132388294 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:19:37.814 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.812+0000 7efef2d9d640 1 --2- 172.21.15.26:0/1132388294 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efef410e410 0x7efef419ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:37.814 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.812+0000 7efef259c640 1 --2- 172.21.15.26:0/1132388294 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efef4100860 0x7efef419a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:37.814 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.812+0000 7efef1d9b640 1 -- 172.21.15.26:0/1132388294 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efef410e410 msgr2=0x7efef419ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:37.815 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.812+0000 7efef1d9b640 1 --2- 172.21.15.26:0/1132388294 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efef410e410 0x7efef419ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:37.815 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.812+0000 7efef1d9b640 1 -- 172.21.15.26:0/1132388294 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efef4100860 msgr2=0x7efef419a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:37.815 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.812+0000 7efef1d9b640 1 --2- 172.21.15.26:0/1132388294 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efef4100860 0x7efef419a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:37.815 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.812+0000 7efef1d9b640 1 -- 172.21.15.26:0/1132388294 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efef419f5a0 con 0x7efef4101210 2024-09-11T23:19:37.815 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.813+0000 7efef1d9b640 1 --2- 172.21.15.26:0/1132388294 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efef4101210 0x7efef419aaf0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7efee8004770 tx=0x7efee8003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:19:37.815 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.813+0000 7efee37fe640 1 -- 172.21.15.26:0/1132388294 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efee803f070 con 0x7efef4101210 2024-09-11T23:19:37.815 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.813+0000 7efef8a5e640 1 -- 172.21.15.26:0/1132388294 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efef419f830 con 0x7efef4101210 2024-09-11T23:19:37.815 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.813+0000 7efee37fe640 1 -- 172.21.15.26:0/1132388294 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efee8031c20 con 0x7efef4101210 2024-09-11T23:19:37.815 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.814+0000 7efee37fe640 1 -- 172.21.15.26:0/1132388294 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efee8035030 con 0x7efef4101210 2024-09-11T23:19:37.816 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.814+0000 7efef8a5e640 1 -- 172.21.15.26:0/1132388294 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efef41a7120 con 0x7efef4101210 2024-09-11T23:19:37.816 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.816+0000 7efee37fe640 1 -- 172.21.15.26:0/1132388294 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 25) ==== 138066+0+0 (secure 0 0 0) 0x7efee8042070 con 0x7efef4101210 2024-09-11T23:19:37.816 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.817+0000 7efefb7fc740 1 Processor -- start 2024-09-11T23:19:37.818 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.817+0000 7efefb7fc740 1 -- start start 2024-09-11T23:19:37.818 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.817+0000 7efee37fe640 1 --2- 172.21.15.26:0/1132388294 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7efedc0a2f40 0x7efedc0a5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:37.818 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.817+0000 7efefb7fc740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ecae384af0 0x55ecae384ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:37.819 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.817+0000 7efefb7fc740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ecae385410 0x55ecae38f770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:37.819 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.817+0000 7efefb7fc740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ecae38fcb0 0x55ecae3920a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:37.819 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.818+0000 7efefb7fc740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55ecae2e92f0 con 0x55ecae385410 2024-09-11T23:19:37.819 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.818+0000 7efefb7fc740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55ecae291270 con 0x55ecae384af0 2024-09-11T23:19:37.819 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.818+0000 7efefb7fc740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55ecae2e52b0 con 0x55ecae38fcb0 2024-09-11T23:19:37.819 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.818+0000 7efef2d9d640 1 --2- 172.21.15.26:0/1132388294 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7efedc0a2f40 0x7efedc0a5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:37.819 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.818+0000 7efee37fe640 1 -- 172.21.15.26:0/1132388294 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7efee80efc60 con 0x7efef4101210 2024-09-11T23:19:37.819 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.818+0000 7efef1d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ecae385410 0x55ecae38f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:37.819 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.818+0000 7efef1d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ecae385410 0x55ecae38f770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52568/0 (socket says 172.21.15.26:52568) 2024-09-11T23:19:37.819 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.818+0000 7efef1d9b640 1 -- 172.21.15.26:0/1624322994 learned_addr learned my addr 172.21.15.26:0/1624322994 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:19:37.819 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.818+0000 7efef259c640 1 --2- 172.21.15.26:0/1624322994 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ecae384af0 0x55ecae384ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:37.819 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.818+0000 7efef2d9d640 1 --2- 172.21.15.26:0/1624322994 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ecae38fcb0 0x55ecae3920a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:37.819 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.818+0000 7efef1d9b640 1 -- 172.21.15.26:0/1624322994 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ecae38fcb0 msgr2=0x55ecae3920a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:19:37.820 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.818+0000 7efef1d9b640 1 --2- 172.21.15.26:0/1624322994 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ecae38fcb0 0x55ecae3920a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:37.820 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.818+0000 7efef1d9b640 1 -- 172.21.15.26:0/1624322994 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ecae384af0 msgr2=0x55ecae384ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:19:37.820 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.818+0000 7efef1d9b640 1 --2- 172.21.15.26:0/1624322994 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ecae384af0 0x55ecae384ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:37.820 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.818+0000 7efef1d9b640 1 -- 172.21.15.26:0/1624322994 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ecae290340 con 0x55ecae385410 2024-09-11T23:19:37.820 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.818+0000 7efef2d9d640 1 --2- 172.21.15.26:0/1624322994 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ecae38fcb0 0x55ecae3920a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:19:37.820 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.818+0000 7efef2d9d640 1 --2- 172.21.15.26:0/1132388294 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7efedc0a2f40 0x7efedc0a5400 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7efeec005fd0 tx=0x7efeec011b60 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:19:37.820 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.818+0000 7efef1d9b640 1 --2- 172.21.15.26:0/1624322994 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ecae385410 0x55ecae38f770 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7efee80bc050 tx=0x7efee802a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce33de01259c134a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:19:37.824 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.819+0000 7efee0ff9640 1 -- 172.21.15.26:0/1624322994 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efee8049070 con 0x55ecae385410 2024-09-11T23:19:37.824 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.819+0000 7efee0ff9640 1 -- 172.21.15.26:0/1624322994 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efee80dc0d0 con 0x55ecae385410 2024-09-11T23:19:37.824 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.819+0000 7efee0ff9640 1 -- 172.21.15.26:0/1624322994 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efee8031740 con 0x55ecae385410 2024-09-11T23:19:37.824 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.820+0000 7efefb7fc740 1 -- 172.21.15.26:0/1624322994 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ecae385410 msgr2=0x55ecae38f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:37.824 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.820+0000 7efefb7fc740 1 --2- 172.21.15.26:0/1624322994 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ecae385410 0x55ecae38f770 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7efee80bc050 tx=0x7efee802a410 comp rx=0 tx=0).stop 2024-09-11T23:19:37.825 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.820+0000 7efefb7fc740 1 -- 172.21.15.26:0/1624322994 shutdown_connections 2024-09-11T23:19:37.825 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.820+0000 7efefb7fc740 1 --2- 172.21.15.26:0/1624322994 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ecae38fcb0 0x55ecae3920a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:37.825 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.820+0000 7efefb7fc740 1 --2- 172.21.15.26:0/1624322994 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ecae385410 0x55ecae38f770 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:37.825 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.820+0000 7efefb7fc740 1 --2- 172.21.15.26:0/1624322994 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ecae384af0 0x55ecae384ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:37.825 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.820+0000 7efefb7fc740 1 -- 172.21.15.26:0/1624322994 >> 172.21.15.26:0/1624322994 conn(0x55ecae381060 msgr2=0x55ecae381440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:19:37.825 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.820+0000 7efefb7fc740 1 -- 172.21.15.26:0/1624322994 shutdown_connections 2024-09-11T23:19:37.825 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.820+0000 7efefb7fc740 1 -- 172.21.15.26:0/1624322994 wait complete. 2024-09-11T23:19:37.825 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.820+0000 7efefb7fc740 1 -- start start 2024-09-11T23:19:37.825 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.821+0000 7efefb7fc740 1 Processor -- start 2024-09-11T23:19:37.825 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.821+0000 7efefb7fc740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ecae384af0 0x55ecae38cb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:37.825 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.821+0000 7efefb7fc740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ecae385410 0x55ecae38d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:37.825 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.821+0000 7efefb7fc740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ecae38fcb0 0x55ecae38d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:37.826 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.821+0000 7efefb7fc740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55ecae392640 con 0x55ecae384af0 2024-09-11T23:19:37.826 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.821+0000 7efefb7fc740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55ecae272290 con 0x55ecae385410 2024-09-11T23:19:37.826 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.821+0000 7efefb7fc740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55ecae3927c0 con 0x55ecae38fcb0 2024-09-11T23:19:37.826 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.821+0000 7efef259c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ecae384af0 0x55ecae38cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:37.826 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.821+0000 7efef259c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ecae384af0 0x55ecae38cb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52574/0 (socket says 172.21.15.26:52574) 2024-09-11T23:19:37.826 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.821+0000 7efef259c640 1 -- 172.21.15.26:0/1924341192 learned_addr learned my addr 172.21.15.26:0/1924341192 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:19:37.826 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.821+0000 7efefb7fc740 10 client.0 _ll_register_callbacks cb 0x55ecae2a3140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:19:37.826 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.821+0000 7efef1d9b640 1 --2- 172.21.15.26:0/1924341192 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ecae385410 0x55ecae38d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:37.826 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.821+0000 7efef2d9d640 1 --2- 172.21.15.26:0/1924341192 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ecae38fcb0 0x55ecae38d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:37.826 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.821+0000 7efef259c640 1 -- 172.21.15.26:0/1924341192 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ecae38fcb0 msgr2=0x55ecae38d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:19:37.826 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.821+0000 7efef259c640 1 --2- 172.21.15.26:0/1924341192 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ecae38fcb0 0x55ecae38d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:37.826 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.821+0000 7efef259c640 1 -- 172.21.15.26:0/1924341192 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ecae385410 msgr2=0x55ecae38d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:19:37.826 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.821+0000 7efef259c640 1 --2- 172.21.15.26:0/1924341192 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ecae385410 0x55ecae38d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:37.826 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.822+0000 7efef259c640 1 -- 172.21.15.26:0/1924341192 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ecadffc210 con 0x55ecae384af0 2024-09-11T23:19:37.827 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.822+0000 7efef259c640 1 --2- 172.21.15.26:0/1924341192 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ecae384af0 0x55ecae38cb60 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7efee400ba90 tx=0x7efee400bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fc5d43aaacb555d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:19:37.827 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.822+0000 7efed9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:19:37.827 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.822+0000 7efed9ffb640 1 -- 172.21.15.26:0/1924341192 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efee4005020 con 0x55ecae384af0 2024-09-11T23:19:37.827 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.822+0000 7efed9ffb640 1 -- 172.21.15.26:0/1924341192 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7efee40051c0 con 0x55ecae384af0 2024-09-11T23:19:37.827 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.822+0000 7efefb7fc740 1 -- 172.21.15.26:0/1924341192 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efee80dc0d0 con 0x55ecae384af0 2024-09-11T23:19:37.827 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.822+0000 7efed9ffb640 1 -- 172.21.15.26:0/1924341192 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7efee40053c0 con 0x55ecae384af0 2024-09-11T23:19:37.827 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.822+0000 7efefb7fc740 10 client.15771 fetch_fsmap learned FSMap version 29 2024-09-11T23:19:37.827 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.822+0000 7efefb7fc740 1 -- 172.21.15.26:0/1924341192 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ecae0c8620 con 0x55ecae384af0 2024-09-11T23:19:37.827 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.823+0000 7efed9ffb640 1 -- 172.21.15.26:0/1924341192 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efee80dc0d0 con 0x55ecae384af0 2024-09-11T23:19:37.827 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.823+0000 7efed9ffb640 1 -- 172.21.15.26:0/1924341192 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7efee401b6a0 con 0x55ecae384af0 2024-09-11T23:19:37.827 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.823+0000 7efefb7fc740 10 client.15771 fetch_fsmap finished waiting for FSMap version 29 2024-09-11T23:19:37.827 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.823+0000 7efefb7fc740 10 client.15771 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:19:37.828 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.823+0000 7efefb7fc740 20 client.15771 populate_metadata read hostname 'smithi026' 2024-09-11T23:19:37.828 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.823+0000 7efefb7fc740 1 --2- 172.21.15.26:0/1924341192 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55ecae39f920 0x55ecae3a1d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:37.828 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.823+0000 7efefb7fc740 4 client.15771 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:19:37.828 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.823+0000 7efefb7fc740 1 -- 172.21.15.26:0/1924341192 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ecae2f6a20 con 0x55ecae39f920 2024-09-11T23:19:37.828 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.824+0000 7efef1d9b640 1 --2- 172.21.15.26:0/1924341192 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55ecae39f920 0x55ecae3a1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:37.828 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.824+0000 7efef1d9b640 1 --2- 172.21.15.26:0/1924341192 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55ecae39f920 0x55ecae3a1d10 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ba6519819d60431 server_cookie=7cb0dcbc0c15ca7d in_seq=0 out_seq=0 2024-09-11T23:19:37.828 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.824+0000 7efed9ffb640 10 client.15771 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:19:37.830 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.825+0000 7efed9ffb640 1 -- 172.21.15.26:0/1924341192 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ecae2f6a20 con 0x55ecae39f920 2024-09-11T23:19:37.830 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.825+0000 7efed9ffb640 10 client.15771 handle_command_reply: tid=0 2024-09-11T23:19:37.830 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.825+0000 7efefb7fc740 1 client.15771 shutdown 2024-09-11T23:19:37.830 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.825+0000 7efefb7fc740 1 -- 172.21.15.26:0/1924341192 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ecae384af0 msgr2=0x55ecae38cb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:37.830 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.825+0000 7efefb7fc740 1 --2- 172.21.15.26:0/1924341192 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ecae384af0 0x55ecae38cb60 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7efee400ba90 tx=0x7efee400bf50 comp rx=0 tx=0).stop 2024-09-11T23:19:37.830 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.825+0000 7efefb7fc740 1 -- 172.21.15.26:0/1924341192 shutdown_connections 2024-09-11T23:19:37.830 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.825+0000 7efefb7fc740 1 --2- 172.21.15.26:0/1924341192 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ecae385410 0x55ecae38d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:37.830 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.826+0000 7efefb7fc740 1 --2- 172.21.15.26:0/1924341192 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55ecae39f920 0x55ecae3a1d10 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:37.830 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.826+0000 7efed9ffb640 0 client.15771 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:19:37.830 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.826+0000 7efefb7fc740 1 --2- 172.21.15.26:0/1924341192 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ecae38fcb0 0x55ecae38d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:37.830 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.826+0000 7efefb7fc740 1 --2- 172.21.15.26:0/1924341192 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ecae384af0 0x55ecae38cb60 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:37.831 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.826+0000 7efefb7fc740 1 -- 172.21.15.26:0/1924341192 >> 172.21.15.26:0/1924341192 conn(0x55ecae381060 msgr2=0x55ecae386dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:19:37.831 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.826+0000 7efefb7fc740 1 -- 172.21.15.26:0/1924341192 shutdown_connections 2024-09-11T23:19:37.831 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.826+0000 7efefb7fc740 1 -- 172.21.15.26:0/1924341192 wait complete. 2024-09-11T23:19:37.831 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.826+0000 7efefb7fc740 20 client.15771 trim_cache size 0 max 16384 2024-09-11T23:19:37.854 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.853+0000 7efefb7fc740 1 Processor -- start 2024-09-11T23:19:37.854 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.853+0000 7efefb7fc740 1 -- start start 2024-09-11T23:19:37.854 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.853+0000 7efefb7fc740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ecae384af0 0x55ecae38d410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:37.854 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.853+0000 7efefb7fc740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ecae385410 0x55ecae3a7600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:37.854 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.853+0000 7efefb7fc740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ecae38fcb0 0x55ecae3a7b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:37.854 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.853+0000 7efefb7fc740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7efef41afa10 con 0x55ecae385410 2024-09-11T23:19:37.854 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.853+0000 7efefb7fc740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7efef41b1550 con 0x55ecae38fcb0 2024-09-11T23:19:37.855 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.853+0000 7efefb7fc740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7efef41b00e0 con 0x55ecae384af0 2024-09-11T23:19:37.855 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.854+0000 7efef1d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ecae385410 0x55ecae3a7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:37.855 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.854+0000 7efef1d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ecae385410 0x55ecae3a7600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52582/0 (socket says 172.21.15.26:52582) 2024-09-11T23:19:37.855 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.854+0000 7efef259c640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ecae384af0 0x55ecae38d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:37.855 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.854+0000 7efef1d9b640 1 -- 172.21.15.26:0/3899529207 learned_addr learned my addr 172.21.15.26:0/3899529207 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:19:37.855 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.854+0000 7efef2d9d640 1 --2- 172.21.15.26:0/3899529207 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ecae38fcb0 0x55ecae3a7b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:37.855 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.854+0000 7efef259c640 1 -- 172.21.15.26:0/3899529207 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ecae38fcb0 msgr2=0x55ecae3a7b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:19:37.855 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.854+0000 7efef259c640 1 --2- 172.21.15.26:0/3899529207 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ecae38fcb0 0x55ecae3a7b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:37.855 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.854+0000 7efef259c640 1 -- 172.21.15.26:0/3899529207 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ecae385410 msgr2=0x55ecae3a7600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:19:37.855 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.854+0000 7efef259c640 1 --2- 172.21.15.26:0/3899529207 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ecae385410 0x55ecae3a7600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:37.855 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.854+0000 7efef259c640 1 -- 172.21.15.26:0/3899529207 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ecae380e90 con 0x55ecae384af0 2024-09-11T23:19:37.855 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.854+0000 7efef1d9b640 1 --2- 172.21.15.26:0/3899529207 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ecae385410 0x55ecae3a7600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:19:37.855 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.854+0000 7efef259c640 1 --2- 172.21.15.26:0/3899529207 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ecae384af0 0x55ecae38d410 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7efee40045b0 tx=0x7efee4003f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=18b54d236ae1d1c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:19:37.859 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.855+0000 7efedbfff640 1 -- 172.21.15.26:0/3899529207 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efee4031070 con 0x55ecae384af0 2024-09-11T23:19:37.859 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.855+0000 7efedbfff640 1 -- 172.21.15.26:0/3899529207 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efee4005560 con 0x55ecae384af0 2024-09-11T23:19:37.860 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.855+0000 7efedbfff640 1 -- 172.21.15.26:0/3899529207 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efee4028040 con 0x55ecae384af0 2024-09-11T23:19:37.860 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.856+0000 7efefb7fc740 1 -- 172.21.15.26:0/3899529207 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ecae384af0 msgr2=0x55ecae38d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:37.860 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.856+0000 7efefb7fc740 1 --2- 172.21.15.26:0/3899529207 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ecae384af0 0x55ecae38d410 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7efee40045b0 tx=0x7efee4003f40 comp rx=0 tx=0).stop 2024-09-11T23:19:37.860 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.856+0000 7efefb7fc740 1 -- 172.21.15.26:0/3899529207 shutdown_connections 2024-09-11T23:19:37.860 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.856+0000 7efefb7fc740 1 --2- 172.21.15.26:0/3899529207 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ecae38fcb0 0x55ecae3a7b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:37.860 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.856+0000 7efefb7fc740 1 --2- 172.21.15.26:0/3899529207 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ecae385410 0x55ecae3a7600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:37.860 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.856+0000 7efefb7fc740 1 --2- 172.21.15.26:0/3899529207 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ecae384af0 0x55ecae38d410 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:37.860 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.856+0000 7efefb7fc740 1 -- 172.21.15.26:0/3899529207 >> 172.21.15.26:0/3899529207 conn(0x55ecae381060 msgr2=0x55ecae38ebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:19:37.860 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.856+0000 7efefb7fc740 1 -- 172.21.15.26:0/3899529207 shutdown_connections 2024-09-11T23:19:37.860 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.856+0000 7efefb7fc740 1 -- 172.21.15.26:0/3899529207 wait complete. 2024-09-11T23:19:37.860 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.856+0000 7efefb7fc740 1 -- start start 2024-09-11T23:19:37.861 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.856+0000 7efefb7fc740 1 Processor -- start 2024-09-11T23:19:37.861 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.857+0000 7efefb7fc740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ecae384af0 0x55ecae398310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:37.861 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.857+0000 7efef259c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ecae384af0 0x55ecae398310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:37.861 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.857+0000 7efef259c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ecae384af0 0x55ecae398310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52584/0 (socket says 172.21.15.26:52584) 2024-09-11T23:19:37.861 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.857+0000 7efef259c640 1 -- 172.21.15.26:0/2063769077 learned_addr learned my addr 172.21.15.26:0/2063769077 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:19:37.861 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.857+0000 7efefb7fc740 1 --2- 172.21.15.26:0/2063769077 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ecae385410 0x55ecae393630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:37.861 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.857+0000 7efefb7fc740 1 --2- 172.21.15.26:0/2063769077 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ecae38fcb0 0x55ecae393dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:37.861 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.857+0000 7efefb7fc740 1 -- 172.21.15.26:0/2063769077 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7efef41b0e80 con 0x55ecae384af0 2024-09-11T23:19:37.861 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.857+0000 7efefb7fc740 1 -- 172.21.15.26:0/2063769077 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7efef41b07b0 con 0x55ecae38fcb0 2024-09-11T23:19:37.861 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.857+0000 7efefb7fc740 1 -- 172.21.15.26:0/2063769077 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7efef41ad9f0 con 0x55ecae385410 2024-09-11T23:19:37.861 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.857+0000 7efef1d9b640 1 --2- 172.21.15.26:0/2063769077 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ecae385410 0x55ecae393630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:37.861 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.857+0000 7efef2d9d640 1 --2- 172.21.15.26:0/2063769077 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ecae38fcb0 0x55ecae393dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:37.861 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.857+0000 7efef259c640 1 -- 172.21.15.26:0/2063769077 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ecae385410 msgr2=0x55ecae393630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:19:37.862 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.857+0000 7efef259c640 1 --2- 172.21.15.26:0/2063769077 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ecae385410 0x55ecae393630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:37.862 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.857+0000 7efef259c640 1 -- 172.21.15.26:0/2063769077 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ecae38fcb0 msgr2=0x55ecae393dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:19:37.862 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.857+0000 7efef259c640 1 --2- 172.21.15.26:0/2063769077 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ecae38fcb0 0x55ecae393dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:37.862 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.857+0000 7efef259c640 1 -- 172.21.15.26:0/2063769077 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ecae4095d0 con 0x55ecae384af0 2024-09-11T23:19:37.862 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.857+0000 7efef1d9b640 1 --2- 172.21.15.26:0/2063769077 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ecae385410 0x55ecae393630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:19:37.862 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.857+0000 7efef2d9d640 1 --2- 172.21.15.26:0/2063769077 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ecae38fcb0 0x55ecae393dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:19:37.862 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.858+0000 7efef259c640 1 --2- 172.21.15.26:0/2063769077 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ecae384af0 0x55ecae398310 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7efee401ba40 tx=0x7efee4005710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3595e7b932e9a263 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:19:37.862 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.858+0000 7efed9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:19:37.862 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.858+0000 7efefb7fc740 10 client.0 _ll_register_callbacks cb 0x7efee400b670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:19:37.862 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.858+0000 7efefb7fc740 1 -- 172.21.15.26:0/2063769077 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efee4005560 con 0x55ecae384af0 2024-09-11T23:19:37.862 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.858+0000 7efed9ffb640 1 -- 172.21.15.26:0/2063769077 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efee4031070 con 0x55ecae384af0 2024-09-11T23:19:37.862 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.858+0000 7efed9ffb640 1 -- 172.21.15.26:0/2063769077 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7efee400b390 con 0x55ecae384af0 2024-09-11T23:19:37.862 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.858+0000 7efed9ffb640 1 -- 172.21.15.26:0/2063769077 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7efee40119e0 con 0x55ecae384af0 2024-09-11T23:19:37.863 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.858+0000 7efefb7fc740 10 client.15777 fetch_fsmap learned FSMap version 29 2024-09-11T23:19:37.863 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.858+0000 7efefb7fc740 1 -- 172.21.15.26:0/2063769077 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({fsmap=29}) -- 0x55ecae40a3f0 con 0x55ecae384af0 2024-09-11T23:19:37.863 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.859+0000 7efed9ffb640 1 -- 172.21.15.26:0/2063769077 <== mon.0 v2:172.21.15.26:3300/0 4 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7efee4011b80 con 0x55ecae384af0 2024-09-11T23:19:37.863 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.859+0000 7efefb7fc740 10 client.15777 fetch_fsmap finished waiting for FSMap version 29 2024-09-11T23:19:37.863 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.859+0000 7efefb7fc740 10 client.15777 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:19:37.863 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.859+0000 7efefb7fc740 20 client.15777 populate_metadata read hostname 'smithi026' 2024-09-11T23:19:37.863 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.859+0000 7efefb7fc740 1 --2- 172.21.15.26:0/2063769077 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55ecae39f920 0x55ecae40cc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:37.863 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.859+0000 7efefb7fc740 4 client.15777 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T23:19:37.863 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.859+0000 7efefb7fc740 1 -- 172.21.15.26:0/2063769077 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ecae2f0d70 con 0x55ecae39f920 2024-09-11T23:19:37.863 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.859+0000 7efef1d9b640 1 --2- 172.21.15.26:0/2063769077 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55ecae39f920 0x55ecae40cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:37.863 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.860+0000 7efef1d9b640 1 --2- 172.21.15.26:0/2063769077 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55ecae39f920 0x55ecae40cc00 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f1d9bff256337ed server_cookie=9c42201f818fd36e in_seq=0 out_seq=0 2024-09-11T23:19:37.863 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.860+0000 7efed9ffb640 10 client.15777 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:19:37.864 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.863+0000 7efed9ffb640 1 -- 172.21.15.26:0/2063769077 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ecae2f0d70 con 0x55ecae39f920 2024-09-11T23:19:37.865 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.863+0000 7efed9ffb640 10 client.15777 handle_command_reply: tid=0 2024-09-11T23:19:37.865 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.863+0000 7efefb7fc740 1 client.15777 shutdown 2024-09-11T23:19:37.865 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.864+0000 7efefb7fc740 1 -- 172.21.15.26:0/2063769077 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ecae384af0 msgr2=0x55ecae398310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:37.865 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.864+0000 7efefb7fc740 1 --2- 172.21.15.26:0/2063769077 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ecae384af0 0x55ecae398310 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7efee401ba40 tx=0x7efee4005710 comp rx=0 tx=0).stop 2024-09-11T23:19:37.865 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.864+0000 7efefb7fc740 1 -- 172.21.15.26:0/2063769077 shutdown_connections 2024-09-11T23:19:37.865 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.864+0000 7efefb7fc740 1 --2- 172.21.15.26:0/2063769077 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ecae38fcb0 0x55ecae393dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:37.865 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.864+0000 7efefb7fc740 1 --2- 172.21.15.26:0/2063769077 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55ecae39f920 0x55ecae40cc00 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:37.865 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.864+0000 7efefb7fc740 1 --2- 172.21.15.26:0/2063769077 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ecae385410 0x55ecae393630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:37.865 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.864+0000 7efefb7fc740 1 --2- 172.21.15.26:0/2063769077 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ecae384af0 0x55ecae398310 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:37.865 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.864+0000 7efefb7fc740 1 -- 172.21.15.26:0/2063769077 >> 172.21.15.26:0/2063769077 conn(0x55ecae381060 msgr2=0x55ecae386550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:19:37.865 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.864+0000 7efed9ffb640 0 client.15777 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:19:37.865 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.864+0000 7efefb7fc740 1 -- 172.21.15.26:0/2063769077 shutdown_connections 2024-09-11T23:19:37.865 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.864+0000 7efefb7fc740 1 -- 172.21.15.26:0/2063769077 wait complete. 2024-09-11T23:19:37.866 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.864+0000 7efefb7fc740 20 client.15777 trim_cache size 0 max 16384 2024-09-11T23:19:37.866 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-09-11T23:19:37.866 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "message": "", 2024-09-11T23:19:37.866 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-09-11T23:19:37.866 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-09-11T23:19:37.867 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.865+0000 7efef8a5e640 1 -- 172.21.15.26:0/1132388294 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7efedc0a2f40 msgr2=0x7efedc0a5400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:37.867 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.865+0000 7efef8a5e640 1 --2- 172.21.15.26:0/1132388294 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7efedc0a2f40 0x7efedc0a5400 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7efeec005fd0 tx=0x7efeec011b60 comp rx=0 tx=0).stop 2024-09-11T23:19:37.867 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.865+0000 7efef8a5e640 1 -- 172.21.15.26:0/1132388294 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efef4101210 msgr2=0x7efef419aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:37.867 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.865+0000 7efef8a5e640 1 --2- 172.21.15.26:0/1132388294 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efef4101210 0x7efef419aaf0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7efee8004770 tx=0x7efee8003b90 comp rx=0 tx=0).stop 2024-09-11T23:19:37.867 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.865+0000 7efef8a5e640 1 -- 172.21.15.26:0/1132388294 shutdown_connections 2024-09-11T23:19:37.867 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.865+0000 7efef8a5e640 1 --2- 172.21.15.26:0/1132388294 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7efedc0a2f40 0x7efedc0a5400 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:37.867 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.865+0000 7efef8a5e640 1 --2- 172.21.15.26:0/1132388294 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efef410e410 0x7efef419ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:37.867 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.865+0000 7efef8a5e640 1 --2- 172.21.15.26:0/1132388294 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efef4101210 0x7efef419aaf0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:37.867 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.865+0000 7efef8a5e640 1 --2- 172.21.15.26:0/1132388294 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efef4100860 0x7efef419a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:37.868 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.865+0000 7efef8a5e640 1 -- 172.21.15.26:0/1132388294 >> 172.21.15.26:0/1132388294 conn(0x7efef4077990 msgr2=0x7efef40ff540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:19:37.868 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.865+0000 7efef8a5e640 1 -- 172.21.15.26:0/1132388294 shutdown_connections 2024-09-11T23:19:37.868 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:37.865+0000 7efef8a5e640 1 -- 172.21.15.26:0/1132388294 wait complete. 2024-09-11T23:19:38.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:19:37 smithi136 ceph-mon[35121]: pgmap v40: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-09-11T23:19:38.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:19:37 smithi026 ceph-mon[31172]: pgmap v40: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-09-11T23:19:38.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:19:37 smithi076 ceph-mon[35296]: pgmap v40: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-09-11T23:19:40.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:19:39 smithi136 ceph-mon[35121]: pgmap v41: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.5 KiB/s wr, 14 op/s 2024-09-11T23:19:40.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:19:39 smithi026 ceph-mon[31172]: pgmap v41: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.5 KiB/s wr, 14 op/s 2024-09-11T23:19:40.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:19:39 smithi076 ceph-mon[35296]: pgmap v41: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.5 KiB/s wr, 14 op/s 2024-09-11T23:19:42.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:19:41 smithi136 ceph-mon[35121]: pgmap v42: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-09-11T23:19:42.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:19:41 smithi026 ceph-mon[31172]: pgmap v42: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-09-11T23:19:42.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:19:41 smithi076 ceph-mon[35296]: pgmap v42: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-09-11T23:19:43.179 INFO:tasks.workunit.client.0.smithi026.stderr:Updating files: 95% (12349/12903) Updating files: 96% (12387/12903) Updating files: 97% (12516/12903) Updating files: 98% (12645/12903) Updating files: 99% (12774/12903) Updating files: 100% (12903/12903) Updating files: 100% (12903/12903), done. 2024-09-11T23:19:43.644 INFO:tasks.workunit.client.0.smithi026.stderr:Note: switching to '086c737fa75c22d521a3932c59b31c5e1d1c9ef4'. 2024-09-11T23:19:43.644 INFO:tasks.workunit.client.0.smithi026.stderr: 2024-09-11T23:19:43.645 INFO:tasks.workunit.client.0.smithi026.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-09-11T23:19:43.645 INFO:tasks.workunit.client.0.smithi026.stderr:changes and commit them, and you can discard any commits you make in this 2024-09-11T23:19:43.645 INFO:tasks.workunit.client.0.smithi026.stderr:state without impacting any branches by switching back to a branch. 2024-09-11T23:19:43.645 INFO:tasks.workunit.client.0.smithi026.stderr: 2024-09-11T23:19:43.645 INFO:tasks.workunit.client.0.smithi026.stderr:If you want to create a new branch to retain commits you create, you may 2024-09-11T23:19:43.645 INFO:tasks.workunit.client.0.smithi026.stderr:do so (now or later) by using -c with the switch command. Example: 2024-09-11T23:19:43.645 INFO:tasks.workunit.client.0.smithi026.stderr: 2024-09-11T23:19:43.645 INFO:tasks.workunit.client.0.smithi026.stderr: git switch -c 2024-09-11T23:19:43.645 INFO:tasks.workunit.client.0.smithi026.stderr: 2024-09-11T23:19:43.645 INFO:tasks.workunit.client.0.smithi026.stderr:Or undo this operation with: 2024-09-11T23:19:43.645 INFO:tasks.workunit.client.0.smithi026.stderr: 2024-09-11T23:19:43.645 INFO:tasks.workunit.client.0.smithi026.stderr: git switch - 2024-09-11T23:19:43.645 INFO:tasks.workunit.client.0.smithi026.stderr: 2024-09-11T23:19:43.646 INFO:tasks.workunit.client.0.smithi026.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-09-11T23:19:43.646 INFO:tasks.workunit.client.0.smithi026.stderr: 2024-09-11T23:19:43.646 INFO:tasks.workunit.client.0.smithi026.stderr:HEAD is now at 086c737fa75 Merge pull request #59733 from zdover23/wip-doc-2024-09-11-backport-59635-to-squid 2024-09-11T23:19:43.650 DEBUG:teuthology.orchestra.run.smithi026:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-09-11T23:19:43.716 INFO:tasks.workunit.client.0.smithi026.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-09-11T23:19:43.718 INFO:tasks.workunit.client.0.smithi026.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-11T23:19:43.718 INFO:tasks.workunit.client.0.smithi026.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-09-11T23:19:44.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:19:43 smithi136 ceph-mon[35121]: pgmap v43: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-09-11T23:19:44.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:19:43 smithi026 ceph-mon[31172]: pgmap v43: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-09-11T23:19:44.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:19:43 smithi076 ceph-mon[35296]: pgmap v43: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-09-11T23:19:44.433 INFO:tasks.workunit.client.0.smithi026.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-09-11T23:19:44.503 INFO:tasks.workunit.client.0.smithi026.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-09-11T23:19:44.543 INFO:tasks.workunit.client.0.smithi026.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-11T23:19:44.545 INFO:tasks.workunit.client.0.smithi026.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-11T23:19:44.545 INFO:tasks.workunit.client.0.smithi026.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-09-11T23:19:44.585 INFO:tasks.workunit.client.0.smithi026.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-11T23:19:44.588 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-09-11T23:19:44.588 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-09-11T23:19:44.645 INFO:tasks.workunit:Running workunits matching suites/blogbench.sh on client.0... 2024-09-11T23:19:44.646 INFO:tasks.workunit:Running workunit suites/blogbench.sh... 2024-09-11T23:19:44.647 DEBUG:teuthology.orchestra.run.smithi026:workunit test suites/blogbench.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=086c737fa75c22d521a3932c59b31c5e1d1c9ef4 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/blogbench.sh 2024-09-11T23:19:44.709 INFO:tasks.workunit.client.0.smithi026.stdout:getting blogbench 2024-09-11T23:19:44.709 INFO:tasks.workunit.client.0.smithi026.stderr:+ echo 'getting blogbench' 2024-09-11T23:19:44.709 INFO:tasks.workunit.client.0.smithi026.stderr:+ wget http://download.ceph.com/qa/blogbench-1.0.tar.bz2 2024-09-11T23:19:44.728 INFO:tasks.workunit.client.0.smithi026.stderr:--2024-09-11 23:19:44-- http://download.ceph.com/qa/blogbench-1.0.tar.bz2 2024-09-11T23:19:44.729 INFO:tasks.workunit.client.0.smithi026.stderr:Resolving download.ceph.com (download.ceph.com)... 158.69.68.124, 2607:5300:201:2000::3:58a1 2024-09-11T23:19:44.750 INFO:tasks.workunit.client.0.smithi026.stderr:Connecting to download.ceph.com (download.ceph.com)|158.69.68.124|:80... connected. 2024-09-11T23:19:44.771 INFO:tasks.workunit.client.0.smithi026.stderr:HTTP request sent, awaiting response... 200 OK 2024-09-11T23:19:44.771 INFO:tasks.workunit.client.0.smithi026.stderr:Length: 103355 (101K) [application/octet-stream] 2024-09-11T23:19:44.772 INFO:tasks.workunit.client.0.smithi026.stderr:Saving to: ‘blogbench-1.0.tar.bz2’ 2024-09-11T23:19:44.774 INFO:tasks.workunit.client.0.smithi026.stderr: 2024-09-11T23:19:44.813 INFO:tasks.workunit.client.0.smithi026.stderr: 0K .......... .......... .......... .......... .......... 49% 1.23M 0s 2024-09-11T23:19:44.832 INFO:tasks.workunit.client.0.smithi026.stderr: 50K .......... .......... .......... .......... .......... 99% 2.39M 0s 2024-09-11T23:19:44.833 INFO:tasks.workunit.client.0.smithi026.stderr: 100K 100% 1.74T=0.06s 2024-09-11T23:19:44.833 INFO:tasks.workunit.client.0.smithi026.stderr: 2024-09-11T23:19:44.833 INFO:tasks.workunit.client.0.smithi026.stderr:2024-09-11 23:19:44 (1.64 MB/s) - ‘blogbench-1.0.tar.bz2’ saved [103355/103355] 2024-09-11T23:19:44.833 INFO:tasks.workunit.client.0.smithi026.stderr: 2024-09-11T23:19:44.833 INFO:tasks.workunit.client.0.smithi026.stderr:+ tar -xvf blogbench-1.0.tar.bz2 2024-09-11T23:19:44.870 INFO:tasks.workunit.client.0.smithi026.stdout:blogbench-1.0/ 2024-09-11T23:19:44.871 INFO:tasks.workunit.client.0.smithi026.stdout:blogbench-1.0/m4/ 2024-09-11T23:19:44.872 INFO:tasks.workunit.client.0.smithi026.stdout:blogbench-1.0/m4/acx_pthread.m4 2024-09-11T23:19:44.873 INFO:tasks.workunit.client.0.smithi026.stdout:blogbench-1.0/man/ 2024-09-11T23:19:44.874 INFO:tasks.workunit.client.0.smithi026.stdout:blogbench-1.0/man/blogbench.8 2024-09-11T23:19:44.875 INFO:tasks.workunit.client.0.smithi026.stdout:blogbench-1.0/man/Makefile.am 2024-09-11T23:19:44.876 INFO:tasks.workunit.client.0.smithi026.stdout:blogbench-1.0/man/Makefile.in 2024-09-11T23:19:44.877 INFO:tasks.workunit.client.0.smithi026.stdout:blogbench-1.0/src/ 2024-09-11T23:19:44.878 INFO:tasks.workunit.client.0.smithi026.stdout:blogbench-1.0/src/mysnprintf.c 2024-09-11T23:19:44.879 INFO:tasks.workunit.client.0.smithi026.stdout:blogbench-1.0/src/mysnprintf.h 2024-09-11T23:19:44.880 INFO:tasks.workunit.client.0.smithi026.stdout:blogbench-1.0/src/blogbench.c 2024-09-11T23:19:44.881 INFO:tasks.workunit.client.0.smithi026.stdout:blogbench-1.0/src/blogbench.h 2024-09-11T23:19:44.882 INFO:tasks.workunit.client.0.smithi026.stdout:blogbench-1.0/src/fakesnprintf.c 2024-09-11T23:19:44.883 INFO:tasks.workunit.client.0.smithi026.stdout:blogbench-1.0/src/fakesnprintf.h 2024-09-11T23:19:44.884 INFO:tasks.workunit.client.0.smithi026.stdout:blogbench-1.0/src/blogbench_p.h 2024-09-11T23:19:44.885 INFO:tasks.workunit.client.0.smithi026.stdout:blogbench-1.0/src/bsd-getopt_long.c 2024-09-11T23:19:44.886 INFO:tasks.workunit.client.0.smithi026.stdout:blogbench-1.0/src/bsd-getopt_long.h 2024-09-11T23:19:44.887 INFO:tasks.workunit.client.0.smithi026.stdout:blogbench-1.0/src/Makefile.am 2024-09-11T23:19:44.888 INFO:tasks.workunit.client.0.smithi026.stdout:blogbench-1.0/src/Makefile.in 2024-09-11T23:19:44.889 INFO:tasks.workunit.client.0.smithi026.stdout:blogbench-1.0/src/process.c 2024-09-11T23:19:44.890 INFO:tasks.workunit.client.0.smithi026.stdout:blogbench-1.0/src/globals.h 2024-09-11T23:19:44.891 INFO:tasks.workunit.client.0.smithi026.stdout:blogbench-1.0/src/commenter.c 2024-09-11T23:19:44.892 INFO:tasks.workunit.client.0.smithi026.stdout:blogbench-1.0/src/rewriter.c 2024-09-11T23:19:44.893 INFO:tasks.workunit.client.0.smithi026.stdout:blogbench-1.0/src/reader.c 2024-09-11T23:19:44.894 INFO:tasks.workunit.client.0.smithi026.stdout:blogbench-1.0/src/writer.c 2024-09-11T23:19:44.895 INFO:tasks.workunit.client.0.smithi026.stdout:blogbench-1.0/src/helpers.c 2024-09-11T23:19:44.896 INFO:tasks.workunit.client.0.smithi026.stdout:blogbench-1.0/NEWS 2024-09-11T23:19:44.897 INFO:tasks.workunit.client.0.smithi026.stdout:blogbench-1.0/compile 2024-09-11T23:19:44.898 INFO:tasks.workunit.client.0.smithi026.stdout:blogbench-1.0/depcomp 2024-09-11T23:19:44.899 INFO:tasks.workunit.client.0.smithi026.stdout:blogbench-1.0/aclocal.m4 2024-09-11T23:19:44.900 INFO:tasks.workunit.client.0.smithi026.stdout:blogbench-1.0/README 2024-09-11T23:19:44.901 INFO:tasks.workunit.client.0.smithi026.stdout:blogbench-1.0/configure 2024-09-11T23:19:44.904 INFO:tasks.workunit.client.0.smithi026.stdout:blogbench-1.0/configure.ac 2024-09-11T23:19:44.905 INFO:tasks.workunit.client.0.smithi026.stdout:blogbench-1.0/config.guess 2024-09-11T23:19:44.906 INFO:tasks.workunit.client.0.smithi026.stdout:blogbench-1.0/install-sh 2024-09-11T23:19:44.907 INFO:tasks.workunit.client.0.smithi026.stdout:blogbench-1.0/config.sub 2024-09-11T23:19:44.908 INFO:tasks.workunit.client.0.smithi026.stdout:blogbench-1.0/missing 2024-09-11T23:19:44.909 INFO:tasks.workunit.client.0.smithi026.stdout:blogbench-1.0/mkinstalldirs 2024-09-11T23:19:44.910 INFO:tasks.workunit.client.0.smithi026.stdout:blogbench-1.0/Makefile.am 2024-09-11T23:19:44.911 INFO:tasks.workunit.client.0.smithi026.stdout:blogbench-1.0/Makefile.in 2024-09-11T23:19:44.911 INFO:tasks.workunit.client.0.smithi026.stdout:blogbench-1.0/config.h.in 2024-09-11T23:19:44.912 INFO:tasks.workunit.client.0.smithi026.stdout:blogbench-1.0/AUTHORS 2024-09-11T23:19:44.913 INFO:tasks.workunit.client.0.smithi026.stdout:blogbench-1.0/INSTALL 2024-09-11T23:19:44.914 INFO:tasks.workunit.client.0.smithi026.stdout:blogbench-1.0/ChangeLog 2024-09-11T23:19:44.915 INFO:tasks.workunit.client.0.smithi026.stdout:blogbench-1.0/COPYING 2024-09-11T23:19:44.917 INFO:tasks.workunit.client.0.smithi026.stderr:+ cd blogbench-1.0/ 2024-09-11T23:19:44.917 INFO:tasks.workunit.client.0.smithi026.stdout:making blogbench 2024-09-11T23:19:44.917 INFO:tasks.workunit.client.0.smithi026.stderr:+ echo 'making blogbench' 2024-09-11T23:19:44.917 INFO:tasks.workunit.client.0.smithi026.stderr:+ ./configure 2024-09-11T23:19:45.022 INFO:tasks.workunit.client.0.smithi026.stdout:checking for a BSD-compatible install... /usr/bin/install -c 2024-09-11T23:19:46.029 INFO:tasks.workunit.client.0.smithi026.stdout:checking whether build environment is sane... yes 2024-09-11T23:19:46.038 INFO:tasks.workunit.client.0.smithi026.stdout:checking for gawk... gawk 2024-09-11T23:19:46.054 INFO:tasks.workunit.client.0.smithi026.stdout:checking whether make sets $(MAKE)... yes 2024-09-11T23:19:46.073 INFO:tasks.workunit.client.0.smithi026.stdout:checking for gcc... gcc 2024-09-11T23:19:46.121 INFO:tasks.workunit.client.0.smithi026.stdout:checking for C compiler default output file name... a.out 2024-09-11T23:19:46.123 INFO:tasks.workunit.client.0.smithi026.stdout:checking whether the C compiler works... yes 2024-09-11T23:19:46.126 INFO:tasks.workunit.client.0.smithi026.stdout:checking whether we are cross compiling... no 2024-09-11T23:19:46.163 INFO:tasks.workunit.client.0.smithi026.stdout:checking for suffix of executables... 2024-09-11T23:19:46.206 INFO:tasks.workunit.client.0.smithi026.stdout:checking for suffix of object files... o 2024-09-11T23:19:46.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:19:45 smithi136 ceph-mon[35121]: pgmap v44: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-09-11T23:19:46.243 INFO:tasks.workunit.client.0.smithi026.stdout:checking whether we are using the GNU C compiler... yes 2024-09-11T23:19:46.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:19:45 smithi026 ceph-mon[31172]: pgmap v44: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-09-11T23:19:46.281 INFO:tasks.workunit.client.0.smithi026.stdout:checking whether gcc accepts -g... yes 2024-09-11T23:19:46.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:19:45 smithi076 ceph-mon[35296]: pgmap v44: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-09-11T23:19:46.349 INFO:tasks.workunit.client.0.smithi026.stdout:checking for gcc option to accept ANSI C... none needed 2024-09-11T23:19:46.401 INFO:tasks.workunit.client.0.smithi026.stdout:checking for style of include used by make... GNU 2024-09-11T23:19:46.503 INFO:tasks.workunit.client.0.smithi026.stdout:checking dependency style of gcc... gcc3 2024-09-11T23:19:46.503 INFO:tasks.workunit.client.0.smithi026.stdout:checking for a BSD-compatible install... /usr/bin/install -c 2024-09-11T23:19:46.559 INFO:tasks.workunit.client.0.smithi026.stdout:checking how to run the C preprocessor... gcc -E 2024-09-11T23:19:46.601 INFO:tasks.workunit.client.0.smithi026.stdout:checking for egrep... grep -E 2024-09-11T23:19:46.610 INFO:tasks.workunit.client.0.smithi026.stdout:checking for AIX... no 2024-09-11T23:19:46.686 INFO:tasks.workunit.client.0.smithi026.stdout:checking for library containing strerror... none required 2024-09-11T23:19:46.834 INFO:tasks.workunit.client.0.smithi026.stdout:checking for ANSI C header files... yes 2024-09-11T23:19:46.883 INFO:tasks.workunit.client.0.smithi026.stdout:checking whether time.h and sys/time.h may both be included... yes 2024-09-11T23:19:46.931 INFO:tasks.workunit.client.0.smithi026.stdout:checking for sys/types.h... yes 2024-09-11T23:19:46.980 INFO:tasks.workunit.client.0.smithi026.stdout:checking for sys/stat.h... yes 2024-09-11T23:19:47.030 INFO:tasks.workunit.client.0.smithi026.stdout:checking for stdlib.h... yes 2024-09-11T23:19:47.081 INFO:tasks.workunit.client.0.smithi026.stdout:checking for string.h... yes 2024-09-11T23:19:47.188 INFO:tasks.workunit.client.0.smithi026.stdout:checking for memory.h... yes 2024-09-11T23:19:47.238 INFO:tasks.workunit.client.0.smithi026.stdout:checking for strings.h... yes 2024-09-11T23:19:47.288 INFO:tasks.workunit.client.0.smithi026.stdout:checking for inttypes.h... yes 2024-09-11T23:19:47.336 INFO:tasks.workunit.client.0.smithi026.stdout:checking for stdint.h... yes 2024-09-11T23:19:47.386 INFO:tasks.workunit.client.0.smithi026.stdout:checking for unistd.h... yes 2024-09-11T23:19:47.398 INFO:tasks.workunit.client.0.smithi026.stdout:checking for memory.h... (cached) yes 2024-09-11T23:19:47.403 INFO:tasks.workunit.client.0.smithi026.stdout:checking for strings.h... (cached) yes 2024-09-11T23:19:47.408 INFO:tasks.workunit.client.0.smithi026.stdout:checking for unistd.h... (cached) yes 2024-09-11T23:19:47.453 INFO:tasks.workunit.client.0.smithi026.stdout:checking locale.h usability... yes 2024-09-11T23:19:47.472 INFO:tasks.workunit.client.0.smithi026.stdout:checking locale.h presence... yes 2024-09-11T23:19:47.473 INFO:tasks.workunit.client.0.smithi026.stdout:checking for locale.h... yes 2024-09-11T23:19:47.488 INFO:tasks.workunit.client.0.smithi026.stdout:checking for stdlib.h... (cached) yes 2024-09-11T23:19:47.493 INFO:tasks.workunit.client.0.smithi026.stdout:checking for string.h... (cached) yes 2024-09-11T23:19:47.547 INFO:tasks.workunit.client.0.smithi026.stdout:checking sys/socket.h usability... yes 2024-09-11T23:19:47.569 INFO:tasks.workunit.client.0.smithi026.stdout:checking sys/socket.h presence... yes 2024-09-11T23:19:47.570 INFO:tasks.workunit.client.0.smithi026.stdout:checking for sys/socket.h... yes 2024-09-11T23:19:47.623 INFO:tasks.workunit.client.0.smithi026.stdout:checking sys/time.h usability... yes 2024-09-11T23:19:47.643 INFO:tasks.workunit.client.0.smithi026.stdout:checking sys/time.h presence... yes 2024-09-11T23:19:47.644 INFO:tasks.workunit.client.0.smithi026.stdout:checking for sys/time.h... yes 2024-09-11T23:19:47.698 INFO:tasks.workunit.client.0.smithi026.stdout:checking fcntl.h usability... yes 2024-09-11T23:19:47.718 INFO:tasks.workunit.client.0.smithi026.stdout:checking fcntl.h presence... yes 2024-09-11T23:19:47.719 INFO:tasks.workunit.client.0.smithi026.stdout:checking for fcntl.h... yes 2024-09-11T23:19:47.772 INFO:tasks.workunit.client.0.smithi026.stdout:checking sys/fcntl.h usability... yes 2024-09-11T23:19:47.793 INFO:tasks.workunit.client.0.smithi026.stdout:checking sys/fcntl.h presence... yes 2024-09-11T23:19:47.793 INFO:tasks.workunit.client.0.smithi026.stdout:checking for sys/fcntl.h... yes 2024-09-11T23:19:47.905 INFO:tasks.workunit.client.0.smithi026.stdout:checking sys/param.h usability... yes 2024-09-11T23:19:47.932 INFO:tasks.workunit.client.0.smithi026.stdout:checking sys/param.h presence... yes 2024-09-11T23:19:47.932 INFO:tasks.workunit.client.0.smithi026.stdout:checking for sys/param.h... yes 2024-09-11T23:19:47.981 INFO:tasks.workunit.client.0.smithi026.stdout:checking ioctl.h usability... no 2024-09-11T23:19:48.001 INFO:tasks.workunit.client.0.smithi026.stdout:checking ioctl.h presence... no 2024-09-11T23:19:48.002 INFO:tasks.workunit.client.0.smithi026.stdout:checking for ioctl.h... no 2024-09-11T23:19:48.061 INFO:tasks.workunit.client.0.smithi026.stdout:checking sys/ioctl.h usability... yes 2024-09-11T23:19:48.082 INFO:tasks.workunit.client.0.smithi026.stdout:checking sys/ioctl.h presence... yes 2024-09-11T23:19:48.082 INFO:tasks.workunit.client.0.smithi026.stdout:checking for sys/ioctl.h... yes 2024-09-11T23:19:48.136 INFO:tasks.workunit.client.0.smithi026.stdout:checking return type of signal handlers... void 2024-09-11T23:19:48.180 INFO:tasks.workunit.client.0.smithi026.stdout:checking for size_t... yes 2024-09-11T23:19:48.219 INFO:tasks.workunit.client.0.smithi026.stdout:checking for dev_t... yes 2024-09-11T23:19:48.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:19:47 smithi136 ceph-mon[35121]: pgmap v45: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-09-11T23:19:48.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:19:47 smithi026 ceph-mon[31172]: pgmap v45: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-09-11T23:19:48.263 INFO:tasks.workunit.client.0.smithi026.stdout:checking for short... yes 2024-09-11T23:19:48.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:19:47 smithi076 ceph-mon[35296]: pgmap v45: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-09-11T23:19:48.342 INFO:tasks.workunit.client.0.smithi026.stdout:checking size of short... 2 2024-09-11T23:19:48.387 INFO:tasks.workunit.client.0.smithi026.stdout:checking for int... yes 2024-09-11T23:19:48.456 INFO:tasks.workunit.client.0.smithi026.stdout:checking size of int... 4 2024-09-11T23:19:48.501 INFO:tasks.workunit.client.0.smithi026.stdout:checking for long... yes 2024-09-11T23:19:48.570 INFO:tasks.workunit.client.0.smithi026.stdout:checking size of long... 8 2024-09-11T23:19:48.615 INFO:tasks.workunit.client.0.smithi026.stdout:checking for long long... yes 2024-09-11T23:19:48.685 INFO:tasks.workunit.client.0.smithi026.stdout:checking size of long long... 8 2024-09-11T23:19:48.728 INFO:tasks.workunit.client.0.smithi026.stdout:checking for ssize_t... yes 2024-09-11T23:19:48.768 INFO:tasks.workunit.client.0.smithi026.stdout:checking for sig_atomic_t... yes 2024-09-11T23:19:48.822 INFO:tasks.workunit.client.0.smithi026.stdout:checking whether gcc needs -traditional... no 2024-09-11T23:19:48.857 INFO:tasks.workunit.client.0.smithi026.stdout:checking for an ANSI C-conforming const... yes 2024-09-11T23:19:48.881 INFO:tasks.workunit.client.0.smithi026.stdout:checking for inline... inline 2024-09-11T23:19:48.942 INFO:tasks.workunit.client.0.smithi026.stdout:checking for working alloca.h... yes 2024-09-11T23:19:48.990 INFO:tasks.workunit.client.0.smithi026.stdout:checking for alloca... yes 2024-09-11T23:19:49.060 INFO:tasks.workunit.client.0.smithi026.stdout:checking for working memcmp... yes 2024-09-11T23:19:49.117 INFO:tasks.workunit.client.0.smithi026.stdout:checking for strftime... yes 2024-09-11T23:19:49.191 INFO:tasks.workunit.client.0.smithi026.stdout:checking whether lstat dereferences a symlink specified with a trailing slash... yes 2024-09-11T23:19:49.253 INFO:tasks.workunit.client.0.smithi026.stdout:checking whether stat accepts an empty string... no 2024-09-11T23:19:49.309 INFO:tasks.workunit.client.0.smithi026.stdout:checking for vprintf... yes 2024-09-11T23:19:49.368 INFO:tasks.workunit.client.0.smithi026.stdout:checking for _doprnt... no 2024-09-11T23:19:49.390 INFO:tasks.workunit.client.0.smithi026.stdout:checking build system type... x86_64-unknown-linux-gnu 2024-09-11T23:19:49.402 INFO:tasks.workunit.client.0.smithi026.stdout:checking host system type... x86_64-unknown-linux-gnu 2024-09-11T23:19:49.458 INFO:tasks.workunit.client.0.smithi026.stdout:checking for the pthreads library -lpthreads... no 2024-09-11T23:19:49.512 INFO:tasks.workunit.client.0.smithi026.stdout:checking whether pthreads work without any flags... yes 2024-09-11T23:19:49.558 INFO:tasks.workunit.client.0.smithi026.stdout:checking for joinable pthread attribute... PTHREAD_CREATE_JOINABLE 2024-09-11T23:19:49.558 INFO:tasks.workunit.client.0.smithi026.stdout:checking if more special flags are required for pthreads... no 2024-09-11T23:19:49.558 INFO:tasks.workunit.client.0.smithi026.stdout:checking for cc_r... gcc 2024-09-11T23:19:49.633 INFO:tasks.workunit.client.0.smithi026.stdout:checking for setrlimit... yes 2024-09-11T23:19:49.697 INFO:tasks.workunit.client.0.smithi026.stdout:checking for setproctitle... no 2024-09-11T23:19:49.756 INFO:tasks.workunit.client.0.smithi026.stdout:checking for getopt_long... yes 2024-09-11T23:19:49.818 INFO:tasks.workunit.client.0.smithi026.stdout:checking for strtoull... yes 2024-09-11T23:19:49.883 INFO:tasks.workunit.client.0.smithi026.stdout:checking for strtoq... yes 2024-09-11T23:19:49.971 INFO:tasks.workunit.client.0.smithi026.stdout:checking for snprintf... yes 2024-09-11T23:19:50.048 INFO:tasks.workunit.client.0.smithi026.stdout:checking for vsnprintf... yes 2024-09-11T23:19:50.124 INFO:tasks.workunit.client.0.smithi026.stdout:checking for vfprintf... yes 2024-09-11T23:19:50.190 INFO:tasks.workunit.client.0.smithi026.stdout:checking for gethostname... yes 2024-09-11T23:19:50.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:19:49 smithi136 ceph-mon[35121]: pgmap v46: 97 pgs: 97 active+clean; 2.7 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 115 KiB/s wr, 17 op/s 2024-09-11T23:19:50.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:19:49 smithi026 ceph-mon[31172]: pgmap v46: 97 pgs: 97 active+clean; 2.7 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 115 KiB/s wr, 17 op/s 2024-09-11T23:19:50.251 INFO:tasks.workunit.client.0.smithi026.stdout:checking for setlocale... yes 2024-09-11T23:19:50.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:19:49 smithi076 ceph-mon[35296]: pgmap v46: 97 pgs: 97 active+clean; 2.7 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 115 KiB/s wr, 17 op/s 2024-09-11T23:19:50.312 INFO:tasks.workunit.client.0.smithi026.stdout:checking for sync... yes 2024-09-11T23:19:50.378 INFO:tasks.workunit.client.0.smithi026.stdout:checking whether snprintf is C99 conformant... done 2024-09-11T23:19:50.389 INFO:tasks.workunit.client.0.smithi026.stdout:configure: creating ./config.status 2024-09-11T23:19:50.540 INFO:tasks.workunit.client.0.smithi026.stdout:config.status: creating Makefile 2024-09-11T23:19:50.557 INFO:tasks.workunit.client.0.smithi026.stdout:config.status: creating src/Makefile 2024-09-11T23:19:50.576 INFO:tasks.workunit.client.0.smithi026.stdout:config.status: creating man/Makefile 2024-09-11T23:19:50.587 INFO:tasks.workunit.client.0.smithi026.stdout:config.status: creating config.h 2024-09-11T23:19:50.639 INFO:tasks.workunit.client.0.smithi026.stdout:config.status: executing depfiles commands 2024-09-11T23:19:50.770 INFO:tasks.workunit.client.0.smithi026.stderr:+ make 2024-09-11T23:19:50.779 INFO:tasks.workunit.client.0.smithi026.stdout:make all-recursive 2024-09-11T23:19:50.782 INFO:tasks.workunit.client.0.smithi026.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0' 2024-09-11T23:19:50.784 INFO:tasks.workunit.client.0.smithi026.stdout:Making all in src 2024-09-11T23:19:50.791 INFO:tasks.workunit.client.0.smithi026.stdout:make[2]: Entering directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0/src' 2024-09-11T23:19:50.791 INFO:tasks.workunit.client.0.smithi026.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT blogbench.o -MD -MP -MF ".deps/blogbench.Tpo" -c -o blogbench.o blogbench.c; \ 2024-09-11T23:19:50.791 INFO:tasks.workunit.client.0.smithi026.stdout:then mv -f ".deps/blogbench.Tpo" ".deps/blogbench.Po"; else rm -f ".deps/blogbench.Tpo"; exit 1; fi 2024-09-11T23:19:50.884 INFO:tasks.workunit.client.0.smithi026.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT process.o -MD -MP -MF ".deps/process.Tpo" -c -o process.o process.c; \ 2024-09-11T23:19:50.884 INFO:tasks.workunit.client.0.smithi026.stdout:then mv -f ".deps/process.Tpo" ".deps/process.Po"; else rm -f ".deps/process.Tpo"; exit 1; fi 2024-09-11T23:19:50.955 INFO:tasks.workunit.client.0.smithi026.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT writer.o -MD -MP -MF ".deps/writer.Tpo" -c -o writer.o writer.c; \ 2024-09-11T23:19:50.955 INFO:tasks.workunit.client.0.smithi026.stdout:then mv -f ".deps/writer.Tpo" ".deps/writer.Po"; else rm -f ".deps/writer.Tpo"; exit 1; fi 2024-09-11T23:19:50.960 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.960+0000 7fdeb5dd6640 1 -- 172.21.15.26:0/2859282413 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdeb01009c0 msgr2=0x7fdeb010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:50.960 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.960+0000 7fdeb5dd6640 1 --2- 172.21.15.26:0/2859282413 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdeb01009c0 0x7fdeb010ea90 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fdea8009ca0 tx=0x7fdea802a8c0 comp rx=0 tx=0).stop 2024-09-11T23:19:50.961 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.960+0000 7fdeb5dd6640 1 -- 172.21.15.26:0/2859282413 shutdown_connections 2024-09-11T23:19:50.961 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.960+0000 7fdeb5dd6640 1 --2- 172.21.15.26:0/2859282413 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdeb01009c0 0x7fdeb010ea90 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:50.961 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.960+0000 7fdeb5dd6640 1 --2- 172.21.15.26:0/2859282413 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdeb0100040 0x7fdeb0100480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:50.961 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.960+0000 7fdeb5dd6640 1 --2- 172.21.15.26:0/2859282413 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdeb01039d0 0x7fdeb0103db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:50.961 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.960+0000 7fdeb5dd6640 1 -- 172.21.15.26:0/2859282413 >> 172.21.15.26:0/2859282413 conn(0x7fdeb00770f0 msgr2=0x7fdeb00fe480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:19:50.961 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.960+0000 7fdeb5dd6640 1 -- 172.21.15.26:0/2859282413 shutdown_connections 2024-09-11T23:19:50.961 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.960+0000 7fdeb5dd6640 1 -- 172.21.15.26:0/2859282413 wait complete. 2024-09-11T23:19:50.961 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.961+0000 7fdeb5dd6640 1 Processor -- start 2024-09-11T23:19:50.961 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.961+0000 7fdeb5dd6640 1 -- start start 2024-09-11T23:19:50.963 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.961+0000 7fdeb5dd6640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdeb0100040 0x7fdeb0198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:50.963 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.961+0000 7fdeb5dd6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdeb01009c0 0x7fdeb0198860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:50.963 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.961+0000 7fdeb5dd6640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdeb01039d0 0x7fdeb019cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:50.963 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.961+0000 7fdeb5dd6640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fdeb0111fb0 con 0x7fdeb01009c0 2024-09-11T23:19:50.963 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.961+0000 7fdeb5dd6640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fdeb0111e30 con 0x7fdeb0100040 2024-09-11T23:19:50.963 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.961+0000 7fdeb5dd6640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fdeb0112130 con 0x7fdeb01039d0 2024-09-11T23:19:50.963 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.961+0000 7fdea6ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdeb01009c0 0x7fdeb0198860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:50.963 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.961+0000 7fdea6ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdeb01009c0 0x7fdeb0198860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54184/0 (socket says 172.21.15.26:54184) 2024-09-11T23:19:50.964 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.961+0000 7fdea6ffd640 1 -- 172.21.15.26:0/3408866650 learned_addr learned my addr 172.21.15.26:0/3408866650 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:19:50.964 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.961+0000 7fdeaffff640 1 --2- 172.21.15.26:0/3408866650 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdeb01039d0 0x7fdeb019cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:50.964 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.961+0000 7fdeaf7fe640 1 --2- 172.21.15.26:0/3408866650 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdeb0100040 0x7fdeb0198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:50.964 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.962+0000 7fdea6ffd640 1 -- 172.21.15.26:0/3408866650 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdeb01039d0 msgr2=0x7fdeb019cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:50.964 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.962+0000 7fdea6ffd640 1 --2- 172.21.15.26:0/3408866650 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdeb01039d0 0x7fdeb019cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:50.964 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.962+0000 7fdea6ffd640 1 -- 172.21.15.26:0/3408866650 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdeb0100040 msgr2=0x7fdeb0198320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:50.964 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.962+0000 7fdea6ffd640 1 --2- 172.21.15.26:0/3408866650 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdeb0100040 0x7fdeb0198320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:50.964 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.962+0000 7fdea6ffd640 1 -- 172.21.15.26:0/3408866650 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdeb019d310 con 0x7fdeb01009c0 2024-09-11T23:19:50.964 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.962+0000 7fdeaf7fe640 1 --2- 172.21.15.26:0/3408866650 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdeb0100040 0x7fdeb0198320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:19:50.964 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.962+0000 7fdea6ffd640 1 --2- 172.21.15.26:0/3408866650 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdeb01009c0 0x7fdeb0198860 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fde9800d950 tx=0x7fde9800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:19:50.964 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.962+0000 7fdead7fa640 1 -- 172.21.15.26:0/3408866650 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde98017070 con 0x7fdeb01009c0 2024-09-11T23:19:50.964 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.962+0000 7fdeb5dd6640 1 -- 172.21.15.26:0/3408866650 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdeb019d600 con 0x7fdeb01009c0 2024-09-11T23:19:50.964 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.962+0000 7fdead7fa640 1 -- 172.21.15.26:0/3408866650 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde98009c30 con 0x7fdeb01009c0 2024-09-11T23:19:50.965 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.962+0000 7fdead7fa640 1 -- 172.21.15.26:0/3408866650 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde9800f6b0 con 0x7fdeb01009c0 2024-09-11T23:19:50.965 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.962+0000 7fdeb5dd6640 1 -- 172.21.15.26:0/3408866650 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdeb01a4e90 con 0x7fdeb01009c0 2024-09-11T23:19:50.970 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.964+0000 7fdeb8c4a740 1 Processor -- start 2024-09-11T23:19:50.970 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.965+0000 7fdeb8c4a740 1 -- start start 2024-09-11T23:19:50.970 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.965+0000 7fdeb8c4a740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564fb4317af0 0x564fb4317ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:50.970 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.965+0000 7fdeb8c4a740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564fb4318410 0x564fb4322770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:50.971 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.965+0000 7fdeb8c4a740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564fb4322cb0 0x564fb43250a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:50.971 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.965+0000 7fdeb8c4a740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x564fb427c2f0 con 0x564fb4322cb0 2024-09-11T23:19:50.971 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.965+0000 7fdeb8c4a740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x564fb4224270 con 0x564fb4318410 2024-09-11T23:19:50.971 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.965+0000 7fdeb8c4a740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x564fb42782b0 con 0x564fb4317af0 2024-09-11T23:19:50.971 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.965+0000 7fdeaf7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564fb4318410 0x564fb4322770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:50.971 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.965+0000 7fdeaf7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564fb4318410 0x564fb4322770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:39490/0 (socket says 172.21.15.26:39490) 2024-09-11T23:19:50.971 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.965+0000 7fdeaf7fe640 1 -- 172.21.15.26:0/1768799841 learned_addr learned my addr 172.21.15.26:0/1768799841 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:19:50.971 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.965+0000 7fdead7fa640 1 -- 172.21.15.26:0/3408866650 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 25) ==== 138066+0+0 (secure 0 0 0) 0x7fde980109d0 con 0x7fdeb01009c0 2024-09-11T23:19:50.971 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.965+0000 7fdeaffff640 1 --2- 172.21.15.26:0/1768799841 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564fb4317af0 0x564fb4317ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:50.971 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.965+0000 7fdeaf7fe640 1 -- 172.21.15.26:0/1768799841 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564fb4317af0 msgr2=0x564fb4317ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:19:50.971 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.965+0000 7fdeaf7fe640 1 --2- 172.21.15.26:0/1768799841 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564fb4317af0 0x564fb4317ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:50.971 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.965+0000 7fdeaf7fe640 1 -- 172.21.15.26:0/1768799841 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564fb4322cb0 msgr2=0x564fb43250a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:19:50.971 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.965+0000 7fdeaf7fe640 1 --2- 172.21.15.26:0/1768799841 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564fb4322cb0 0x564fb43250a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:50.972 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.965+0000 7fdeaf7fe640 1 -- 172.21.15.26:0/1768799841 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564fb4223340 con 0x564fb4318410 2024-09-11T23:19:50.972 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.966+0000 7fdeaf7fe640 1 --2- 172.21.15.26:0/1768799841 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564fb4318410 0x564fb4322770 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fdea0009d60 tx=0x7fdea00117b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b3c8ee6fca33a04 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:19:50.972 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.966+0000 7fdea5ffb640 1 -- 172.21.15.26:0/1768799841 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdea0012650 con 0x564fb4318410 2024-09-11T23:19:50.972 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.966+0000 7fdea5ffb640 1 -- 172.21.15.26:0/1768799841 <== mon.2 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdea0009e50 con 0x564fb4318410 2024-09-11T23:19:50.972 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.966+0000 7fdea5ffb640 1 -- 172.21.15.26:0/1768799841 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdea000fd10 con 0x564fb4318410 2024-09-11T23:19:50.972 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.966+0000 7fdead7fa640 1 --2- 172.21.15.26:0/3408866650 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fde9c0a2ec0 0x7fde9c0a5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:50.972 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.966+0000 7fdeaffff640 1 --2- 172.21.15.26:0/3408866650 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fde9c0a2ec0 0x7fde9c0a5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:50.972 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.966+0000 7fdead7fa640 1 -- 172.21.15.26:0/3408866650 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fde980cf370 con 0x7fdeb01009c0 2024-09-11T23:19:50.972 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.967+0000 7fdeaffff640 1 --2- 172.21.15.26:0/3408866650 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fde9c0a2ec0 0x7fde9c0a5380 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fdea8005fe0 tx=0x7fdea8005f30 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:19:50.972 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.967+0000 7fdeb8c4a740 1 -- 172.21.15.26:0/1768799841 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564fb4318410 msgr2=0x564fb4322770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:50.972 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.967+0000 7fdeb8c4a740 1 --2- 172.21.15.26:0/1768799841 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564fb4318410 0x564fb4322770 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fdea0009d60 tx=0x7fdea00117b0 comp rx=0 tx=0).stop 2024-09-11T23:19:50.972 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.967+0000 7fdeb8c4a740 1 -- 172.21.15.26:0/1768799841 shutdown_connections 2024-09-11T23:19:50.972 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.967+0000 7fdeb8c4a740 1 --2- 172.21.15.26:0/1768799841 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564fb4318410 0x564fb4322770 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:50.973 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.967+0000 7fdeb8c4a740 1 --2- 172.21.15.26:0/1768799841 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564fb4322cb0 0x564fb43250a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:50.973 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.967+0000 7fdeb8c4a740 1 --2- 172.21.15.26:0/1768799841 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564fb4317af0 0x564fb4317ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:50.973 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.967+0000 7fdeb8c4a740 1 -- 172.21.15.26:0/1768799841 >> 172.21.15.26:0/1768799841 conn(0x564fb4314060 msgr2=0x564fb4314440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:19:50.973 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.967+0000 7fdeb8c4a740 1 -- 172.21.15.26:0/1768799841 shutdown_connections 2024-09-11T23:19:50.973 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.967+0000 7fdeb8c4a740 1 -- 172.21.15.26:0/1768799841 wait complete. 2024-09-11T23:19:50.973 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.967+0000 7fdeb8c4a740 1 -- start start 2024-09-11T23:19:50.973 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.967+0000 7fdeb8c4a740 1 Processor -- start 2024-09-11T23:19:50.973 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.968+0000 7fdeb8c4a740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564fb4317af0 0x564fb431fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:50.973 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.968+0000 7fdeb8c4a740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564fb4318410 0x564fb43200c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:50.973 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.968+0000 7fdeb8c4a740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564fb4322cb0 0x564fb4320600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:50.973 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.968+0000 7fdeb8c4a740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x564fb4325880 con 0x564fb4322cb0 2024-09-11T23:19:50.973 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.968+0000 7fdeb8c4a740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x564fb4205290 con 0x564fb4317af0 2024-09-11T23:19:50.973 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.968+0000 7fdeb8c4a740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x564fb4325a00 con 0x564fb4318410 2024-09-11T23:19:50.974 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.968+0000 7fdeb8c4a740 10 client.0 _ll_register_callbacks cb 0x564fb4236140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:19:50.974 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.968+0000 7fdeaffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564fb4322cb0 0x564fb4320600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:50.974 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.968+0000 7fdeaf7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564fb4317af0 0x564fb431fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:50.974 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.968+0000 7fdeaf7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564fb4317af0 0x564fb431fb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:39494/0 (socket says 172.21.15.26:39494) 2024-09-11T23:19:50.974 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.968+0000 7fdeaffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564fb4322cb0 0x564fb4320600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54202/0 (socket says 172.21.15.26:54202) 2024-09-11T23:19:50.974 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.968+0000 7fdeaf7fe640 1 -- 172.21.15.26:0/480384813 learned_addr learned my addr 172.21.15.26:0/480384813 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:19:50.974 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.968+0000 7fdea6ffd640 1 --2- 172.21.15.26:0/480384813 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564fb4318410 0x564fb43200c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:50.974 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.968+0000 7fdeaf7fe640 1 -- 172.21.15.26:0/480384813 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564fb4318410 msgr2=0x564fb43200c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:19:50.974 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.968+0000 7fdeaf7fe640 1 --2- 172.21.15.26:0/480384813 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564fb4318410 0x564fb43200c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:50.974 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.968+0000 7fdeaf7fe640 1 -- 172.21.15.26:0/480384813 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564fb4322cb0 msgr2=0x564fb4320600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:19:50.974 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.968+0000 7fdeaf7fe640 1 --2- 172.21.15.26:0/480384813 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564fb4322cb0 0x564fb4320600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:50.974 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.968+0000 7fdeaf7fe640 1 -- 172.21.15.26:0/480384813 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564fb3f8f210 con 0x564fb4317af0 2024-09-11T23:19:50.975 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.968+0000 7fdea6ffd640 1 --2- 172.21.15.26:0/480384813 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564fb4318410 0x564fb43200c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:19:50.975 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.969+0000 7fdeaf7fe640 1 --2- 172.21.15.26:0/480384813 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564fb4317af0 0x564fb431fb80 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fdea00095f0 tx=0x7fdea000f670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=53f67acde6184645 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:19:50.975 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.969+0000 7fde8effd640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-11T23:19:50.975 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.969+0000 7fde8effd640 1 -- 172.21.15.26:0/480384813 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdea0028030 con 0x564fb4317af0 2024-09-11T23:19:50.975 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.969+0000 7fde8effd640 1 -- 172.21.15.26:0/480384813 <== mon.2 v2:172.21.15.76:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fdea0005250 con 0x564fb4317af0 2024-09-11T23:19:50.975 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.969+0000 7fdeb8c4a740 1 -- 172.21.15.26:0/480384813 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdea0009e50 con 0x564fb4317af0 2024-09-11T23:19:50.975 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.969+0000 7fde8effd640 1 -- 172.21.15.26:0/480384813 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fdea0032050 con 0x564fb4317af0 2024-09-11T23:19:50.975 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.969+0000 7fdeb8c4a740 10 client.25376 fetch_fsmap learned FSMap version 29 2024-09-11T23:19:50.975 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.969+0000 7fdeb8c4a740 1 -- 172.21.15.26:0/480384813 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564fb405b620 con 0x564fb4317af0 2024-09-11T23:19:50.975 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.969+0000 7fde8effd640 1 -- 172.21.15.26:0/480384813 <== mon.2 v2:172.21.15.76:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdea0009e50 con 0x564fb4317af0 2024-09-11T23:19:50.975 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.969+0000 7fde8effd640 1 -- 172.21.15.26:0/480384813 <== mon.2 v2:172.21.15.76:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fdea0012550 con 0x564fb4317af0 2024-09-11T23:19:50.975 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.970+0000 7fdeb8c4a740 10 client.25376 fetch_fsmap finished waiting for FSMap version 29 2024-09-11T23:19:50.975 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.970+0000 7fdeb8c4a740 10 client.25376 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:19:50.975 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.970+0000 7fdeb8c4a740 20 client.25376 populate_metadata read hostname 'smithi026' 2024-09-11T23:19:50.976 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.970+0000 7fdeb8c4a740 1 --2- 172.21.15.26:0/480384813 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x564fb4332b90 0x564fb4334f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:50.976 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.970+0000 7fdeb8c4a740 4 client.25376 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:19:50.976 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.970+0000 7fdeb8c4a740 1 -- 172.21.15.26:0/480384813 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564fb4289a20 con 0x564fb4332b90 2024-09-11T23:19:50.976 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.970+0000 7fdea6ffd640 1 --2- 172.21.15.26:0/480384813 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x564fb4332b90 0x564fb4334f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:50.976 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.970+0000 7fdea6ffd640 1 --2- 172.21.15.26:0/480384813 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x564fb4332b90 0x564fb4334f80 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5529a354ec9f1386 server_cookie=e4802927a21ed870 in_seq=0 out_seq=0 2024-09-11T23:19:50.976 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.970+0000 7fde8effd640 10 client.25376 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:19:50.977 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.971+0000 7fde8effd640 1 -- 172.21.15.26:0/480384813 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564fb4289a20 con 0x564fb4332b90 2024-09-11T23:19:50.977 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.971+0000 7fde8effd640 10 client.25376 handle_command_reply: tid=0 2024-09-11T23:19:50.977 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.971+0000 7fdeb8c4a740 1 client.25376 shutdown 2024-09-11T23:19:50.977 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.971+0000 7fdeb8c4a740 1 -- 172.21.15.26:0/480384813 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564fb4317af0 msgr2=0x564fb431fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:50.977 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.971+0000 7fdeb8c4a740 1 --2- 172.21.15.26:0/480384813 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564fb4317af0 0x564fb431fb80 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fdea00095f0 tx=0x7fdea000f670 comp rx=0 tx=0).stop 2024-09-11T23:19:50.977 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.971+0000 7fdeb8c4a740 1 -- 172.21.15.26:0/480384813 shutdown_connections 2024-09-11T23:19:50.978 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.971+0000 7fdeb8c4a740 1 --2- 172.21.15.26:0/480384813 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x564fb4332b90 0x564fb4334f80 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:50.978 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.971+0000 7fdeb8c4a740 1 --2- 172.21.15.26:0/480384813 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564fb4322cb0 0x564fb4320600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:50.978 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.971+0000 7fdeb8c4a740 1 --2- 172.21.15.26:0/480384813 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564fb4318410 0x564fb43200c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:50.978 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.971+0000 7fde8effd640 0 client.25376 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:19:50.978 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.971+0000 7fdeb8c4a740 1 --2- 172.21.15.26:0/480384813 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564fb4317af0 0x564fb431fb80 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:50.978 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.971+0000 7fdeb8c4a740 1 -- 172.21.15.26:0/480384813 >> 172.21.15.26:0/480384813 conn(0x564fb4314060 msgr2=0x564fb4319d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:19:50.978 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.971+0000 7fdeb8c4a740 1 -- 172.21.15.26:0/480384813 shutdown_connections 2024-09-11T23:19:50.978 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.971+0000 7fdeb8c4a740 1 -- 172.21.15.26:0/480384813 wait complete. 2024-09-11T23:19:50.978 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.971+0000 7fdeb8c4a740 20 client.25376 trim_cache size 0 max 16384 2024-09-11T23:19:50.987 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.986+0000 7fdeb8c4a740 1 Processor -- start 2024-09-11T23:19:50.987 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.986+0000 7fdeb8c4a740 1 -- start start 2024-09-11T23:19:50.987 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.986+0000 7fdeb8c4a740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564fb4317af0 0x564fb431eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:50.987 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.986+0000 7fdeb8c4a740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564fb4318410 0x564fb431b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:50.987 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.986+0000 7fdeb8c4a740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564fb4322cb0 0x564fb43200b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:50.987 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.986+0000 7fdeb8c4a740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fdeb01ad7c0 con 0x564fb4322cb0 2024-09-11T23:19:50.988 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.986+0000 7fdeb8c4a740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fdeb01af300 con 0x564fb4317af0 2024-09-11T23:19:50.988 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.986+0000 7fdeb8c4a740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fdeb01ade90 con 0x564fb4318410 2024-09-11T23:19:50.988 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.986+0000 7fdeaffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564fb4322cb0 0x564fb43200b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:50.988 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.986+0000 7fdeaffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564fb4322cb0 0x564fb43200b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54204/0 (socket says 172.21.15.26:54204) 2024-09-11T23:19:50.988 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.986+0000 7fdeaffff640 1 -- 172.21.15.26:0/3540900895 learned_addr learned my addr 172.21.15.26:0/3540900895 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:19:50.988 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.986+0000 7fdea6ffd640 1 --2- 172.21.15.26:0/3540900895 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564fb4318410 0x564fb431b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:50.988 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.987+0000 7fdeaf7fe640 1 --2- 172.21.15.26:0/3540900895 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564fb4317af0 0x564fb431eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:50.988 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.987+0000 7fdeaffff640 1 -- 172.21.15.26:0/3540900895 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564fb4318410 msgr2=0x564fb431b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:19:50.988 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.987+0000 7fdeaffff640 1 --2- 172.21.15.26:0/3540900895 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564fb4318410 0x564fb431b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:50.988 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.987+0000 7fdeaffff640 1 -- 172.21.15.26:0/3540900895 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564fb4317af0 msgr2=0x564fb431eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:19:50.988 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.987+0000 7fdeaffff640 1 --2- 172.21.15.26:0/3540900895 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564fb4317af0 0x564fb431eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:50.988 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.987+0000 7fdeaffff640 1 -- 172.21.15.26:0/3540900895 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564fb430f2d0 con 0x564fb4322cb0 2024-09-11T23:19:50.988 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.987+0000 7fdeaffff640 1 --2- 172.21.15.26:0/3540900895 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564fb4322cb0 0x564fb43200b0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fdea8035000 tx=0x7fdea8041000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4d376953ad23283 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:19:50.989 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.987+0000 7fdea57fa640 1 -- 172.21.15.26:0/3540900895 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdea8003af0 con 0x564fb4322cb0 2024-09-11T23:19:50.989 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.987+0000 7fdea57fa640 1 -- 172.21.15.26:0/3540900895 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdea8036050 con 0x564fb4322cb0 2024-09-11T23:19:50.989 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.987+0000 7fdea57fa640 1 -- 172.21.15.26:0/3540900895 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdea8004430 con 0x564fb4322cb0 2024-09-11T23:19:50.992 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.988+0000 7fdeb8c4a740 1 -- 172.21.15.26:0/3540900895 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564fb4322cb0 msgr2=0x564fb43200b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:50.992 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.988+0000 7fdeb8c4a740 1 --2- 172.21.15.26:0/3540900895 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564fb4322cb0 0x564fb43200b0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fdea8035000 tx=0x7fdea8041000 comp rx=0 tx=0).stop 2024-09-11T23:19:50.992 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.988+0000 7fdeb8c4a740 1 -- 172.21.15.26:0/3540900895 shutdown_connections 2024-09-11T23:19:50.992 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.988+0000 7fdeb8c4a740 1 --2- 172.21.15.26:0/3540900895 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564fb4322cb0 0x564fb43200b0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:50.993 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.988+0000 7fdeb8c4a740 1 --2- 172.21.15.26:0/3540900895 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564fb4318410 0x564fb431b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:50.993 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.988+0000 7fdeb8c4a740 1 --2- 172.21.15.26:0/3540900895 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564fb4317af0 0x564fb431eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:50.993 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.988+0000 7fdeb8c4a740 1 -- 172.21.15.26:0/3540900895 >> 172.21.15.26:0/3540900895 conn(0x564fb4314060 msgr2=0x564fb4321c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:19:50.993 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.988+0000 7fdeb8c4a740 1 -- 172.21.15.26:0/3540900895 shutdown_connections 2024-09-11T23:19:50.993 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.988+0000 7fdeb8c4a740 1 -- 172.21.15.26:0/3540900895 wait complete. 2024-09-11T23:19:50.993 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.988+0000 7fdeb8c4a740 1 -- start start 2024-09-11T23:19:50.993 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.989+0000 7fdeb8c4a740 1 Processor -- start 2024-09-11T23:19:50.993 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.989+0000 7fdeb8c4a740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564fb4317af0 0x564fb4325c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:50.993 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.989+0000 7fdeb8c4a740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564fb4318410 0x564fb4326190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:50.993 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.989+0000 7fdeb8c4a740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564fb4322cb0 0x564fb43266d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:50.993 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.989+0000 7fdeb8c4a740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fdeb01aec30 con 0x564fb4317af0 2024-09-11T23:19:50.993 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.989+0000 7fdeb8c4a740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fdeb01ae560 con 0x564fb4322cb0 2024-09-11T23:19:50.994 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.989+0000 7fdeb8c4a740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fdeb01ab7a0 con 0x564fb4318410 2024-09-11T23:19:50.994 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.989+0000 7fdeb8c4a740 10 client.0 _ll_register_callbacks cb 0x7fdea0009bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:19:50.994 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.989+0000 7fdeaf7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564fb4317af0 0x564fb4325c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:50.994 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.989+0000 7fdeaf7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564fb4317af0 0x564fb4325c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54212/0 (socket says 172.21.15.26:54212) 2024-09-11T23:19:50.994 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.989+0000 7fdeaf7fe640 1 -- 172.21.15.26:0/1202360325 learned_addr learned my addr 172.21.15.26:0/1202360325 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:19:50.994 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.989+0000 7fdeaffff640 1 --2- 172.21.15.26:0/1202360325 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564fb4322cb0 0x564fb43266d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:50.994 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.989+0000 7fdea6ffd640 1 --2- 172.21.15.26:0/1202360325 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564fb4318410 0x564fb4326190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:50.994 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.989+0000 7fdeaf7fe640 1 -- 172.21.15.26:0/1202360325 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564fb4318410 msgr2=0x564fb4326190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:19:50.994 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.989+0000 7fdeaf7fe640 1 --2- 172.21.15.26:0/1202360325 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564fb4318410 0x564fb4326190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:50.994 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.989+0000 7fdeaf7fe640 1 -- 172.21.15.26:0/1202360325 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564fb4322cb0 msgr2=0x564fb43266d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:19:50.994 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.989+0000 7fdeaf7fe640 1 --2- 172.21.15.26:0/1202360325 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564fb4322cb0 0x564fb43266d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:50.994 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.989+0000 7fdeaf7fe640 1 -- 172.21.15.26:0/1202360325 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564fb439b540 con 0x564fb4317af0 2024-09-11T23:19:50.994 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.989+0000 7fdea6ffd640 1 --2- 172.21.15.26:0/1202360325 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564fb4318410 0x564fb4326190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:19:50.994 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.989+0000 7fdeaf7fe640 1 --2- 172.21.15.26:0/1202360325 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564fb4317af0 0x564fb4325c50 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fdea00095f0 tx=0x7fdea0012e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e68773da73740b43 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:19:50.995 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.989+0000 7fde8effd640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:19:50.995 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.990+0000 7fde8effd640 1 -- 172.21.15.26:0/1202360325 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdea0028070 con 0x564fb4317af0 2024-09-11T23:19:50.995 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.990+0000 7fde8effd640 1 -- 172.21.15.26:0/1202360325 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fdea000fe00 con 0x564fb4317af0 2024-09-11T23:19:50.995 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.990+0000 7fdeb8c4a740 1 -- 172.21.15.26:0/1202360325 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdea8036050 con 0x564fb4317af0 2024-09-11T23:19:50.995 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.990+0000 7fde8effd640 1 -- 172.21.15.26:0/1202360325 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fdea0011cb0 con 0x564fb4317af0 2024-09-11T23:19:50.995 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.990+0000 7fdeb8c4a740 10 client.15792 fetch_fsmap learned FSMap version 29 2024-09-11T23:19:50.995 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.990+0000 7fdeb8c4a740 1 -- 172.21.15.26:0/1202360325 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564fb439c830 con 0x564fb4317af0 2024-09-11T23:19:50.995 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.990+0000 7fde8effd640 1 -- 172.21.15.26:0/1202360325 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdea8036050 con 0x564fb4317af0 2024-09-11T23:19:50.995 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.990+0000 7fde8effd640 1 -- 172.21.15.26:0/1202360325 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fdea0004a20 con 0x564fb4317af0 2024-09-11T23:19:50.995 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.990+0000 7fdeb8c4a740 10 client.15792 fetch_fsmap finished waiting for FSMap version 29 2024-09-11T23:19:50.995 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.990+0000 7fdeb8c4a740 10 client.15792 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:19:50.995 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.990+0000 7fdeb8c4a740 20 client.15792 populate_metadata read hostname 'smithi026' 2024-09-11T23:19:50.995 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.990+0000 7fdeb8c4a740 1 --2- 172.21.15.26:0/1202360325 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x564fb439cf20 0x564fb439f340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:50.995 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.990+0000 7fdeb8c4a740 4 client.15792 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T23:19:50.995 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.990+0000 7fdeb8c4a740 1 -- 172.21.15.26:0/1202360325 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564fb4283d70 con 0x564fb439cf20 2024-09-11T23:19:50.996 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.991+0000 7fdea6ffd640 1 --2- 172.21.15.26:0/1202360325 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x564fb439cf20 0x564fb439f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:50.996 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.991+0000 7fde8effd640 10 client.15792 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:19:50.996 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:50.991+0000 7fdea6ffd640 1 --2- 172.21.15.26:0/1202360325 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x564fb439cf20 0x564fb439f340 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=684f5d33c3a3fea0 server_cookie=2409451e290cfcef in_seq=0 out_seq=0 2024-09-11T23:19:51.035 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:51.035+0000 7fde8effd640 1 -- 172.21.15.26:0/1202360325 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564fb4283d70 con 0x564fb439cf20 2024-09-11T23:19:51.035 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:51.035+0000 7fde8effd640 10 client.15792 handle_command_reply: tid=0 2024-09-11T23:19:51.035 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:51.035+0000 7fdeb8c4a740 1 client.15792 shutdown 2024-09-11T23:19:51.035 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:51.035+0000 7fdeb8c4a740 1 -- 172.21.15.26:0/1202360325 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564fb4317af0 msgr2=0x564fb4325c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:51.035 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:51.035+0000 7fdeb8c4a740 1 --2- 172.21.15.26:0/1202360325 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564fb4317af0 0x564fb4325c50 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fdea00095f0 tx=0x7fdea0012e10 comp rx=0 tx=0).stop 2024-09-11T23:19:51.035 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:51.035+0000 7fdeb8c4a740 1 -- 172.21.15.26:0/1202360325 shutdown_connections 2024-09-11T23:19:51.035 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:51.035+0000 7fdeb8c4a740 1 --2- 172.21.15.26:0/1202360325 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564fb4322cb0 0x564fb43266d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:51.035 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:51.035+0000 7fdeb8c4a740 1 --2- 172.21.15.26:0/1202360325 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x564fb439cf20 0x564fb439f340 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:51.036 INFO:tasks.workunit.client.0.smithi026.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT rewriter.o -MD -MP -MF ".deps/rewriter.Tpo" -c -o rewriter.o rewriter.c; \ 2024-09-11T23:19:51.036 INFO:tasks.workunit.client.0.smithi026.stdout:then mv -f ".deps/rewriter.Tpo" ".deps/rewriter.Po"; else rm -f ".deps/rewriter.Tpo"; exit 1; fi 2024-09-11T23:19:51.036 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:51.035+0000 7fdeb8c4a740 1 --2- 172.21.15.26:0/1202360325 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564fb4318410 0x564fb4326190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:51.036 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:51.035+0000 7fdeb8c4a740 1 --2- 172.21.15.26:0/1202360325 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564fb4317af0 0x564fb4325c50 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:51.036 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:51.035+0000 7fde8effd640 0 client.15792 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:19:51.036 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:51.035+0000 7fdeb8c4a740 1 -- 172.21.15.26:0/1202360325 >> 172.21.15.26:0/1202360325 conn(0x564fb4314060 msgr2=0x564fb4319550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:19:51.037 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:51.035+0000 7fdeb8c4a740 1 -- 172.21.15.26:0/1202360325 shutdown_connections 2024-09-11T23:19:51.037 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:51.035+0000 7fdeb8c4a740 1 -- 172.21.15.26:0/1202360325 wait complete. 2024-09-11T23:19:51.037 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:51.035+0000 7fdeb8c4a740 20 client.15792 trim_cache size 0 max 16384 2024-09-11T23:19:51.037 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-09-11T23:19:51.037 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "message": "", 2024-09-11T23:19:51.037 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-09-11T23:19:51.037 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-09-11T23:19:51.038 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:51.036+0000 7fdeb5dd6640 1 -- 172.21.15.26:0/3408866650 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fde9c0a2ec0 msgr2=0x7fde9c0a5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:51.038 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:51.036+0000 7fdeb5dd6640 1 --2- 172.21.15.26:0/3408866650 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fde9c0a2ec0 0x7fde9c0a5380 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fdea8005fe0 tx=0x7fdea8005f30 comp rx=0 tx=0).stop 2024-09-11T23:19:51.038 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:51.036+0000 7fdeb5dd6640 1 -- 172.21.15.26:0/3408866650 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdeb01009c0 msgr2=0x7fdeb0198860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:51.038 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:51.036+0000 7fdeb5dd6640 1 --2- 172.21.15.26:0/3408866650 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdeb01009c0 0x7fdeb0198860 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fde9800d950 tx=0x7fde9800de10 comp rx=0 tx=0).stop 2024-09-11T23:19:51.038 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:51.036+0000 7fdeb5dd6640 1 -- 172.21.15.26:0/3408866650 shutdown_connections 2024-09-11T23:19:51.038 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:51.036+0000 7fdeb5dd6640 1 --2- 172.21.15.26:0/3408866650 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fde9c0a2ec0 0x7fde9c0a5380 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:51.038 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:51.036+0000 7fdeb5dd6640 1 --2- 172.21.15.26:0/3408866650 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdeb01039d0 0x7fdeb019cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:51.038 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:51.036+0000 7fdeb5dd6640 1 --2- 172.21.15.26:0/3408866650 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdeb01009c0 0x7fdeb0198860 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:51.038 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:51.036+0000 7fdeb5dd6640 1 --2- 172.21.15.26:0/3408866650 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdeb0100040 0x7fdeb0198320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:51.039 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:51.036+0000 7fdeb5dd6640 1 -- 172.21.15.26:0/3408866650 >> 172.21.15.26:0/3408866650 conn(0x7fdeb00770f0 msgr2=0x7fdeb0105870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:19:51.039 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:51.036+0000 7fdeb5dd6640 1 -- 172.21.15.26:0/3408866650 shutdown_connections 2024-09-11T23:19:51.039 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:19:51.036+0000 7fdeb5dd6640 1 -- 172.21.15.26:0/3408866650 wait complete. 2024-09-11T23:19:51.086 INFO:tasks.workunit.client.0.smithi026.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT reader.o -MD -MP -MF ".deps/reader.Tpo" -c -o reader.o reader.c; \ 2024-09-11T23:19:51.086 INFO:tasks.workunit.client.0.smithi026.stdout:then mv -f ".deps/reader.Tpo" ".deps/reader.Po"; else rm -f ".deps/reader.Tpo"; exit 1; fi 2024-09-11T23:19:51.145 INFO:tasks.workunit.client.0.smithi026.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT commenter.o -MD -MP -MF ".deps/commenter.Tpo" -c -o commenter.o commenter.c; \ 2024-09-11T23:19:51.145 INFO:tasks.workunit.client.0.smithi026.stdout:then mv -f ".deps/commenter.Tpo" ".deps/commenter.Po"; else rm -f ".deps/commenter.Tpo"; exit 1; fi 2024-09-11T23:19:51.224 INFO:tasks.workunit.client.0.smithi026.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT helpers.o -MD -MP -MF ".deps/helpers.Tpo" -c -o helpers.o helpers.c; \ 2024-09-11T23:19:51.224 INFO:tasks.workunit.client.0.smithi026.stdout:then mv -f ".deps/helpers.Tpo" ".deps/helpers.Po"; else rm -f ".deps/helpers.Tpo"; exit 1; fi 2024-09-11T23:19:51.332 INFO:tasks.workunit.client.0.smithi026.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT bsd-getopt_long.o -MD -MP -MF ".deps/bsd-getopt_long.Tpo" -c -o bsd-getopt_long.o bsd-getopt_long.c; \ 2024-09-11T23:19:51.332 INFO:tasks.workunit.client.0.smithi026.stdout:then mv -f ".deps/bsd-getopt_long.Tpo" ".deps/bsd-getopt_long.Po"; else rm -f ".deps/bsd-getopt_long.Tpo"; exit 1; fi 2024-09-11T23:19:51.400 INFO:tasks.workunit.client.0.smithi026.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT fakesnprintf.o -MD -MP -MF ".deps/fakesnprintf.Tpo" -c -o fakesnprintf.o fakesnprintf.c; \ 2024-09-11T23:19:51.400 INFO:tasks.workunit.client.0.smithi026.stdout:then mv -f ".deps/fakesnprintf.Tpo" ".deps/fakesnprintf.Po"; else rm -f ".deps/fakesnprintf.Tpo"; exit 1; fi 2024-09-11T23:19:51.461 INFO:tasks.workunit.client.0.smithi026.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT mysnprintf.o -MD -MP -MF ".deps/mysnprintf.Tpo" -c -o mysnprintf.o mysnprintf.c; \ 2024-09-11T23:19:51.461 INFO:tasks.workunit.client.0.smithi026.stdout:then mv -f ".deps/mysnprintf.Tpo" ".deps/mysnprintf.Po"; else rm -f ".deps/mysnprintf.Tpo"; exit 1; fi 2024-09-11T23:19:51.515 INFO:tasks.workunit.client.0.smithi026.stdout:gcc -g -O2 -o blogbench blogbench.o process.o writer.o rewriter.o reader.o commenter.o helpers.o bsd-getopt_long.o fakesnprintf.o mysnprintf.o 2024-09-11T23:19:51.537 INFO:tasks.workunit.client.0.smithi026.stdout:make[2]: Leaving directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0/src' 2024-09-11T23:19:51.538 INFO:tasks.workunit.client.0.smithi026.stdout:Making all in man 2024-09-11T23:19:51.545 INFO:tasks.workunit.client.0.smithi026.stdout:make[2]: Entering directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0/man' 2024-09-11T23:19:51.545 INFO:tasks.workunit.client.0.smithi026.stdout:make[2]: Nothing to be done for 'all'. 2024-09-11T23:19:51.545 INFO:tasks.workunit.client.0.smithi026.stdout:make[2]: Leaving directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0/man' 2024-09-11T23:19:51.546 INFO:tasks.workunit.client.0.smithi026.stdout:make[2]: Entering directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0' 2024-09-11T23:19:51.547 INFO:tasks.workunit.client.0.smithi026.stdout:make[2]: Leaving directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0' 2024-09-11T23:19:51.548 INFO:tasks.workunit.client.0.smithi026.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0' 2024-09-11T23:19:51.548 INFO:tasks.workunit.client.0.smithi026.stderr:+ cd src 2024-09-11T23:19:51.549 INFO:tasks.workunit.client.0.smithi026.stderr:+ mkdir blogtest_in 2024-09-11T23:19:51.552 INFO:tasks.workunit.client.0.smithi026.stdout:running blogbench 2024-09-11T23:19:51.553 INFO:tasks.workunit.client.0.smithi026.stderr:+ echo 'running blogbench' 2024-09-11T23:19:51.553 INFO:tasks.workunit.client.0.smithi026.stderr:+ ./blogbench -d blogtest_in 2024-09-11T23:19:51.553 INFO:tasks.workunit.client.0.smithi026.stdout: 2024-09-11T23:19:51.553 INFO:tasks.workunit.client.0.smithi026.stdout:Frequency = 10 secs 2024-09-11T23:19:51.553 INFO:tasks.workunit.client.0.smithi026.stdout:Scratch dir = [blogtest_in] 2024-09-11T23:19:52.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:19:51 smithi136 ceph-mon[35121]: pgmap v47: 97 pgs: 97 active+clean; 2.7 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 115 KiB/s wr, 13 op/s 2024-09-11T23:19:52.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:19:51 smithi026 ceph-mon[31172]: pgmap v47: 97 pgs: 97 active+clean; 2.7 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 115 KiB/s wr, 13 op/s 2024-09-11T23:19:52.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:19:51 smithi076 ceph-mon[35296]: pgmap v47: 97 pgs: 97 active+clean; 2.7 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 115 KiB/s wr, 13 op/s 2024-09-11T23:19:53.573 INFO:tasks.workunit.client.0.smithi026.stdout:Spawning 3 writers... 2024-09-11T23:19:53.573 INFO:tasks.workunit.client.0.smithi026.stdout:Spawning 1 rewriters... 2024-09-11T23:19:53.573 INFO:tasks.workunit.client.0.smithi026.stdout:Spawning 5 commenters... 2024-09-11T23:19:53.573 INFO:tasks.workunit.client.0.smithi026.stdout:Spawning 100 readers... 2024-09-11T23:19:53.576 INFO:tasks.workunit.client.0.smithi026.stdout:Benchmarking for 30 iterations. 2024-09-11T23:19:53.576 INFO:tasks.workunit.client.0.smithi026.stdout:The test will run during 5 minutes. 2024-09-11T23:19:53.576 INFO:tasks.workunit.client.0.smithi026.stdout: 2024-09-11T23:19:53.576 INFO:tasks.workunit.client.0.smithi026.stdout: Nb blogs R articles W articles R pictures W pictures R comments W comments 2024-09-11T23:19:54.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:19:53 smithi136 ceph-mon[35121]: pgmap v48: 97 pgs: 97 active+clean; 2.7 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 115 KiB/s wr, 13 op/s 2024-09-11T23:19:54.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:19:53 smithi026 ceph-mon[31172]: pgmap v48: 97 pgs: 97 active+clean; 2.7 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 115 KiB/s wr, 13 op/s 2024-09-11T23:19:54.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:19:53 smithi076 ceph-mon[35296]: pgmap v48: 97 pgs: 97 active+clean; 2.7 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 115 KiB/s wr, 13 op/s 2024-09-11T23:19:55.619 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T23:19:55.901 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:19:56.201 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:19:55 smithi026 ceph-mon[31172]: pgmap v49: 97 pgs: 97 active+clean; 5.7 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 443 KiB/s wr, 91 op/s 2024-09-11T23:19:56.201 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:19:55 smithi026 ceph-mon[31172]: mgrmap e26: x(active, since 92s), standbys: y, z 2024-09-11T23:19:56.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:19:55 smithi136 ceph-mon[35121]: pgmap v49: 97 pgs: 97 active+clean; 5.7 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 443 KiB/s wr, 91 op/s 2024-09-11T23:19:56.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:19:55 smithi136 ceph-mon[35121]: mgrmap e26: x(active, since 92s), standbys: y, z 2024-09-11T23:19:56.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:19:55 smithi076 ceph-mon[35296]: pgmap v49: 97 pgs: 97 active+clean; 5.7 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 443 KiB/s wr, 91 op/s 2024-09-11T23:19:56.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:19:55 smithi076 ceph-mon[35296]: mgrmap e26: x(active, since 92s), standbys: y, z 2024-09-11T23:19:56.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.411+0000 7fc9f11ed640 1 -- 172.21.15.26:0/534449368 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc9ec1082a0 msgr2=0x7fc9ec10ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:56.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.411+0000 7fc9f11ed640 1 --2- 172.21.15.26:0/534449368 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc9ec1082a0 0x7fc9ec10ec40 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fc9e4009930 tx=0x7fc9e402a420 comp rx=0 tx=0).stop 2024-09-11T23:19:56.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.412+0000 7fc9f11ed640 1 -- 172.21.15.26:0/534449368 shutdown_connections 2024-09-11T23:19:56.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.412+0000 7fc9f11ed640 1 --2- 172.21.15.26:0/534449368 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc9ec1036e0 0x7fc9ec107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:56.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.412+0000 7fc9f11ed640 1 --2- 172.21.15.26:0/534449368 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc9ec1082a0 0x7fc9ec10ec40 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:56.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.412+0000 7fc9f11ed640 1 --2- 172.21.15.26:0/534449368 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc9ec102d30 0x7fc9ec103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:56.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.412+0000 7fc9f11ed640 1 -- 172.21.15.26:0/534449368 >> 172.21.15.26:0/534449368 conn(0x7fc9ec0fc7d0 msgr2=0x7fc9ec0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:19:56.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.412+0000 7fc9f11ed640 1 -- 172.21.15.26:0/534449368 shutdown_connections 2024-09-11T23:19:56.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.412+0000 7fc9f11ed640 1 -- 172.21.15.26:0/534449368 wait complete. 2024-09-11T23:19:56.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.412+0000 7fc9f11ed640 1 Processor -- start 2024-09-11T23:19:56.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.413+0000 7fc9f11ed640 1 -- start start 2024-09-11T23:19:56.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.413+0000 7fc9f11ed640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc9ec102d30 0x7fc9ec19a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:56.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.413+0000 7fc9f11ed640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc9ec1036e0 0x7fc9ec19ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:56.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.413+0000 7fc9f11ed640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc9ec1082a0 0x7fc9ec19f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:56.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.413+0000 7fc9f11ed640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc9ec1120a0 con 0x7fc9ec1082a0 2024-09-11T23:19:56.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.413+0000 7fc9f11ed640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fc9ec111f20 con 0x7fc9ec102d30 2024-09-11T23:19:56.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.413+0000 7fc9f11ed640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fc9ec112220 con 0x7fc9ec1036e0 2024-09-11T23:19:56.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.413+0000 7fc9eb59e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc9ec1082a0 0x7fc9ec19f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:56.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.413+0000 7fc9ead9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc9ec102d30 0x7fc9ec19a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:56.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.413+0000 7fc9ead9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc9ec102d30 0x7fc9ec19a740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:39528/0 (socket says 172.21.15.26:39528) 2024-09-11T23:19:56.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.413+0000 7fc9eb59e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc9ec1082a0 0x7fc9ec19f010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54236/0 (socket says 172.21.15.26:54236) 2024-09-11T23:19:56.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.413+0000 7fc9ead9d640 1 -- 172.21.15.26:0/2322938857 learned_addr learned my addr 172.21.15.26:0/2322938857 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:19:56.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.413+0000 7fc9ea59c640 1 --2- 172.21.15.26:0/2322938857 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc9ec1036e0 0x7fc9ec19ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:56.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.413+0000 7fc9ead9d640 1 -- 172.21.15.26:0/2322938857 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc9ec1036e0 msgr2=0x7fc9ec19ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:56.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.413+0000 7fc9ead9d640 1 --2- 172.21.15.26:0/2322938857 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc9ec1036e0 0x7fc9ec19ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:56.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.413+0000 7fc9ead9d640 1 -- 172.21.15.26:0/2322938857 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc9ec1082a0 msgr2=0x7fc9ec19f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:56.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.413+0000 7fc9ead9d640 1 --2- 172.21.15.26:0/2322938857 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc9ec1082a0 0x7fc9ec19f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:56.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.413+0000 7fc9ead9d640 1 -- 172.21.15.26:0/2322938857 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9ec19f6f0 con 0x7fc9ec102d30 2024-09-11T23:19:56.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.413+0000 7fc9eb59e640 1 --2- 172.21.15.26:0/2322938857 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc9ec1082a0 0x7fc9ec19f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:19:56.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.414+0000 7fc9ead9d640 1 --2- 172.21.15.26:0/2322938857 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc9ec102d30 0x7fc9ec19a740 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc9dc00d950 tx=0x7fc9dc00de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:19:56.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.414+0000 7fc9d3fff640 1 -- 172.21.15.26:0/2322938857 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9dc017030 con 0x7fc9ec102d30 2024-09-11T23:19:56.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.414+0000 7fc9f11ed640 1 -- 172.21.15.26:0/2322938857 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9ec19f9e0 con 0x7fc9ec102d30 2024-09-11T23:19:56.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.414+0000 7fc9d3fff640 1 -- 172.21.15.26:0/2322938857 <== mon.2 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc9dc004020 con 0x7fc9ec102d30 2024-09-11T23:19:56.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.414+0000 7fc9d3fff640 1 -- 172.21.15.26:0/2322938857 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9dc0047a0 con 0x7fc9ec102d30 2024-09-11T23:19:56.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.414+0000 7fc9f11ed640 1 -- 172.21.15.26:0/2322938857 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9ec1a72c0 con 0x7fc9ec102d30 2024-09-11T23:19:56.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.416+0000 7fc9d3fff640 1 -- 172.21.15.26:0/2322938857 <== mon.2 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fc9dc0041d0 con 0x7fc9ec102d30 2024-09-11T23:19:56.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.416+0000 7fc9f35c6740 1 Processor -- start 2024-09-11T23:19:56.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.417+0000 7fc9f35c6740 1 -- start start 2024-09-11T23:19:56.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.417+0000 7fc9f35c6740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556a129524a0 0x556a12952880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:56.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.417+0000 7fc9f35c6740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556a12952dc0 0x556a1295d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:56.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.417+0000 7fc9f35c6740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556a1295d680 0x556a1295fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:56.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.417+0000 7fc9f35c6740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x556a128b82e0 con 0x556a1295d680 2024-09-11T23:19:56.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.417+0000 7fc9f35c6740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x556a1285fce0 con 0x556a129524a0 2024-09-11T23:19:56.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.417+0000 7fc9f35c6740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x556a128b42a0 con 0x556a12952dc0 2024-09-11T23:19:56.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.417+0000 7fc9eb59e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556a129524a0 0x556a12952880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:56.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.417+0000 7fc9ea59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556a1295d680 0x556a1295fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:56.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.417+0000 7fc9eb59e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556a129524a0 0x556a12952880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:39542/0 (socket says 172.21.15.26:39542) 2024-09-11T23:19:56.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.417+0000 7fc9eb59e640 1 -- 172.21.15.26:0/980380954 learned_addr learned my addr 172.21.15.26:0/980380954 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:19:56.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.417+0000 7fc9ead9d640 1 --2- 172.21.15.26:0/980380954 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556a12952dc0 0x556a1295d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:56.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.417+0000 7fc9d3fff640 1 --2- 172.21.15.26:0/2322938857 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fc9d40a2cf0 0x7fc9d40a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:56.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.417+0000 7fc9eb59e640 1 -- 172.21.15.26:0/980380954 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556a12952dc0 msgr2=0x556a1295d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:19:56.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.417+0000 7fc9eb59e640 1 --2- 172.21.15.26:0/980380954 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556a12952dc0 0x556a1295d140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:56.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.417+0000 7fc9eb59e640 1 -- 172.21.15.26:0/980380954 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556a1295d680 msgr2=0x556a1295fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:19:56.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.417+0000 7fc9eb59e640 1 --2- 172.21.15.26:0/980380954 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556a1295d680 0x556a1295fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:56.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.417+0000 7fc9eb59e640 1 -- 172.21.15.26:0/980380954 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a12844440 con 0x556a129524a0 2024-09-11T23:19:56.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.417+0000 7fc9ea59c640 1 --2- 172.21.15.26:0/980380954 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556a1295d680 0x556a1295fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:19:56.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.417+0000 7fc9ead9d640 1 --2- 172.21.15.26:0/980380954 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556a12952dc0 0x556a1295d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:19:56.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.417+0000 7fc9d3fff640 1 -- 172.21.15.26:0/2322938857 <== mon.2 v2:172.21.15.76:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fc9dc0ced70 con 0x7fc9ec102d30 2024-09-11T23:19:56.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.417+0000 7fc9eb59e640 1 --2- 172.21.15.26:0/2322938857 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fc9d40a2cf0 0x7fc9d40a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:56.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.418+0000 7fc9eb59e640 1 --2- 172.21.15.26:0/980380954 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556a129524a0 0x556a12952880 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc9e4009930 tx=0x7fc9e40315b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5ac69806dd34eabc server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:19:56.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.418+0000 7fc9d17fa640 1 -- 172.21.15.26:0/980380954 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9e404d070 con 0x556a129524a0 2024-09-11T23:19:56.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.418+0000 7fc9eb59e640 1 --2- 172.21.15.26:0/2322938857 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fc9d40a2cf0 0x7fc9d40a51b0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fc9e404a000 tx=0x7fc9e4047000 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:19:56.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.418+0000 7fc9d17fa640 1 -- 172.21.15.26:0/980380954 <== mon.2 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc9e40035f0 con 0x556a129524a0 2024-09-11T23:19:56.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.418+0000 7fc9d17fa640 1 -- 172.21.15.26:0/980380954 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9e4035040 con 0x556a129524a0 2024-09-11T23:19:56.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.419+0000 7fc9f35c6740 1 -- 172.21.15.26:0/980380954 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556a129524a0 msgr2=0x556a12952880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:56.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.419+0000 7fc9f35c6740 1 --2- 172.21.15.26:0/980380954 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556a129524a0 0x556a12952880 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc9e4009930 tx=0x7fc9e40315b0 comp rx=0 tx=0).stop 2024-09-11T23:19:56.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.419+0000 7fc9f35c6740 1 -- 172.21.15.26:0/980380954 shutdown_connections 2024-09-11T23:19:56.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.419+0000 7fc9f35c6740 1 --2- 172.21.15.26:0/980380954 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556a1295d680 0x556a1295fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:56.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.419+0000 7fc9f35c6740 1 --2- 172.21.15.26:0/980380954 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556a12952dc0 0x556a1295d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:56.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.419+0000 7fc9f35c6740 1 --2- 172.21.15.26:0/980380954 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556a129524a0 0x556a12952880 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:56.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.419+0000 7fc9f35c6740 1 -- 172.21.15.26:0/980380954 >> 172.21.15.26:0/980380954 conn(0x556a1294ea30 msgr2=0x556a1294ee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:19:56.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.419+0000 7fc9f35c6740 1 -- 172.21.15.26:0/980380954 shutdown_connections 2024-09-11T23:19:56.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.419+0000 7fc9f35c6740 1 -- 172.21.15.26:0/980380954 wait complete. 2024-09-11T23:19:56.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.419+0000 7fc9f35c6740 1 -- start start 2024-09-11T23:19:56.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.419+0000 7fc9f35c6740 1 Processor -- start 2024-09-11T23:19:56.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.419+0000 7fc9f35c6740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556a129524a0 0x556a1295cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:56.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.419+0000 7fc9f35c6740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556a12952dc0 0x556a1295a220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:56.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.419+0000 7fc9f35c6740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556a1295d680 0x556a1295a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:56.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.419+0000 7fc9f35c6740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x556a12960250 con 0x556a1295d680 2024-09-11T23:19:56.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.419+0000 7fc9f35c6740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x556a12873050 con 0x556a12952dc0 2024-09-11T23:19:56.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.419+0000 7fc9f35c6740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x556a129603d0 con 0x556a129524a0 2024-09-11T23:19:56.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.419+0000 7fc9f35c6740 10 client.0 _ll_register_callbacks cb 0x556a128b3160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:19:56.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.419+0000 7fc9eb59e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556a1295d680 0x556a1295a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:56.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.419+0000 7fc9ea59c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556a12952dc0 0x556a1295a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:56.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.419+0000 7fc9eb59e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556a1295d680 0x556a1295a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54250/0 (socket says 172.21.15.26:54250) 2024-09-11T23:19:56.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.419+0000 7fc9eb59e640 1 -- 172.21.15.26:0/2287050396 learned_addr learned my addr 172.21.15.26:0/2287050396 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:19:56.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.420+0000 7fc9ead9d640 1 --2- 172.21.15.26:0/2287050396 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556a129524a0 0x556a1295cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:56.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.420+0000 7fc9eb59e640 1 -- 172.21.15.26:0/2287050396 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556a129524a0 msgr2=0x556a1295cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:19:56.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.420+0000 7fc9eb59e640 1 --2- 172.21.15.26:0/2287050396 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556a129524a0 0x556a1295cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:56.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.420+0000 7fc9eb59e640 1 -- 172.21.15.26:0/2287050396 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556a12952dc0 msgr2=0x556a1295a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:19:56.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.420+0000 7fc9eb59e640 1 --2- 172.21.15.26:0/2287050396 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556a12952dc0 0x556a1295a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:56.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.420+0000 7fc9eb59e640 1 -- 172.21.15.26:0/2287050396 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a126293c0 con 0x556a1295d680 2024-09-11T23:19:56.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.420+0000 7fc9ea59c640 1 --2- 172.21.15.26:0/2287050396 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556a12952dc0 0x556a1295a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:19:56.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.420+0000 7fc9eb59e640 1 --2- 172.21.15.26:0/2287050396 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556a1295d680 0x556a1295a760 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fc9e40096b0 tx=0x7fc9e4003f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d010066fd7f09249 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:19:56.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.420+0000 7fc9ceff5640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:19:56.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.420+0000 7fc9ceff5640 1 -- 172.21.15.26:0/2287050396 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9e4055070 con 0x556a1295d680 2024-09-11T23:19:56.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.420+0000 7fc9ceff5640 1 -- 172.21.15.26:0/2287050396 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fc9e40037a0 con 0x556a1295d680 2024-09-11T23:19:56.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.420+0000 7fc9f35c6740 1 -- 172.21.15.26:0/2287050396 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc9e40035f0 con 0x556a1295d680 2024-09-11T23:19:56.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.420+0000 7fc9ceff5640 1 -- 172.21.15.26:0/2287050396 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc9e4061050 con 0x556a1295d680 2024-09-11T23:19:56.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.420+0000 7fc9f35c6740 10 client.15804 fetch_fsmap learned FSMap version 29 2024-09-11T23:19:56.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.420+0000 7fc9f35c6740 1 -- 172.21.15.26:0/2287050396 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556a12960250 con 0x556a1295d680 2024-09-11T23:19:56.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.420+0000 7fc9ceff5640 1 -- 172.21.15.26:0/2287050396 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc9e40035f0 con 0x556a1295d680 2024-09-11T23:19:56.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.421+0000 7fc9ceff5640 1 -- 172.21.15.26:0/2287050396 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fc9e40477c0 con 0x556a1295d680 2024-09-11T23:19:56.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.421+0000 7fc9f35c6740 10 client.15804 fetch_fsmap finished waiting for FSMap version 29 2024-09-11T23:19:56.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.421+0000 7fc9f35c6740 10 client.15804 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:19:56.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.421+0000 7fc9f35c6740 20 client.15804 populate_metadata read hostname 'smithi026' 2024-09-11T23:19:56.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.421+0000 7fc9f35c6740 1 --2- 172.21.15.26:0/2287050396 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x556a1296cd40 0x556a1296f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:56.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.421+0000 7fc9f35c6740 4 client.15804 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:19:56.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.421+0000 7fc9f35c6740 1 -- 172.21.15.26:0/2287050396 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556a128c5850 con 0x556a1296cd40 2024-09-11T23:19:56.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.421+0000 7fc9ead9d640 1 --2- 172.21.15.26:0/2287050396 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x556a1296cd40 0x556a1296f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:56.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.421+0000 7fc9ead9d640 1 --2- 172.21.15.26:0/2287050396 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x556a1296cd40 0x556a1296f130 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=93d5ba811a3c1b1 server_cookie=2ab0150771a999c1 in_seq=0 out_seq=0 2024-09-11T23:19:56.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.421+0000 7fc9ceff5640 10 client.15804 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:19:56.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.425+0000 7fc9ceff5640 1 -- 172.21.15.26:0/2287050396 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556a128c5850 con 0x556a1296cd40 2024-09-11T23:19:56.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.425+0000 7fc9ceff5640 10 client.15804 handle_command_reply: tid=0 2024-09-11T23:19:56.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.425+0000 7fc9f35c6740 1 client.15804 shutdown 2024-09-11T23:19:56.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.425+0000 7fc9f35c6740 1 -- 172.21.15.26:0/2287050396 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556a1295d680 msgr2=0x556a1295a760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:56.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.425+0000 7fc9f35c6740 1 --2- 172.21.15.26:0/2287050396 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556a1295d680 0x556a1295a760 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fc9e40096b0 tx=0x7fc9e4003f70 comp rx=0 tx=0).stop 2024-09-11T23:19:56.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.425+0000 7fc9f35c6740 1 -- 172.21.15.26:0/2287050396 shutdown_connections 2024-09-11T23:19:56.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.425+0000 7fc9f35c6740 1 --2- 172.21.15.26:0/2287050396 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556a12952dc0 0x556a1295a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:56.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.425+0000 7fc9f35c6740 1 --2- 172.21.15.26:0/2287050396 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x556a1296cd40 0x556a1296f130 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:56.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.425+0000 7fc9f35c6740 1 --2- 172.21.15.26:0/2287050396 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556a1295d680 0x556a1295a760 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:56.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.425+0000 7fc9f35c6740 1 --2- 172.21.15.26:0/2287050396 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556a129524a0 0x556a1295cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:56.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.425+0000 7fc9f35c6740 1 -- 172.21.15.26:0/2287050396 >> 172.21.15.26:0/2287050396 conn(0x556a1294ea30 msgr2=0x556a12954710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:19:56.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.425+0000 7fc9ceff5640 0 client.15804 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:19:56.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.425+0000 7fc9f35c6740 1 -- 172.21.15.26:0/2287050396 shutdown_connections 2024-09-11T23:19:56.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.425+0000 7fc9f35c6740 1 -- 172.21.15.26:0/2287050396 wait complete. 2024-09-11T23:19:56.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.426+0000 7fc9f35c6740 20 client.15804 trim_cache size 0 max 16384 2024-09-11T23:19:56.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.444+0000 7fc9f35c6740 1 Processor -- start 2024-09-11T23:19:56.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.444+0000 7fc9f35c6740 1 -- start start 2024-09-11T23:19:56.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.444+0000 7fc9f35c6740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556a129524a0 0x556a1295c550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:56.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.444+0000 7fc9f35c6740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556a12952dc0 0x556a129748c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:56.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.444+0000 7fc9f35c6740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556a1295d680 0x556a12974e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:56.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.444+0000 7fc9f35c6740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc9ec1afb70 con 0x556a12952dc0 2024-09-11T23:19:56.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.444+0000 7fc9f35c6740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fc9ec1b16b0 con 0x556a1295d680 2024-09-11T23:19:56.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.444+0000 7fc9f35c6740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fc9ec1b0240 con 0x556a129524a0 2024-09-11T23:19:56.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.444+0000 7fc9ea59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556a12952dc0 0x556a129748c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:56.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.444+0000 7fc9ea59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556a12952dc0 0x556a129748c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54256/0 (socket says 172.21.15.26:54256) 2024-09-11T23:19:56.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.444+0000 7fc9ea59c640 1 -- 172.21.15.26:0/3843577214 learned_addr learned my addr 172.21.15.26:0/3843577214 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:19:56.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.444+0000 7fc9eb59e640 1 --2- 172.21.15.26:0/3843577214 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556a1295d680 0x556a12974e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:56.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.444+0000 7fc9ead9d640 1 --2- 172.21.15.26:0/3843577214 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556a129524a0 0x556a1295c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:56.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.445+0000 7fc9ea59c640 1 -- 172.21.15.26:0/3843577214 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556a129524a0 msgr2=0x556a1295c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:19:56.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.445+0000 7fc9ea59c640 1 --2- 172.21.15.26:0/3843577214 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556a129524a0 0x556a1295c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:56.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.445+0000 7fc9ea59c640 1 -- 172.21.15.26:0/3843577214 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556a1295d680 msgr2=0x556a12974e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:19:56.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.445+0000 7fc9ea59c640 1 --2- 172.21.15.26:0/3843577214 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556a1295d680 0x556a12974e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:56.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.445+0000 7fc9ea59c640 1 -- 172.21.15.26:0/3843577214 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a1295d2e0 con 0x556a12952dc0 2024-09-11T23:19:56.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.445+0000 7fc9ea59c640 1 --2- 172.21.15.26:0/3843577214 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556a12952dc0 0x556a129748c0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fc9ec0697d0 tx=0x7fc9e0011c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c7e5e8fe6ed71b1 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:19:56.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.445+0000 7fc9d0ff9640 1 -- 172.21.15.26:0/3843577214 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9e0004b20 con 0x556a12952dc0 2024-09-11T23:19:56.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.445+0000 7fc9d0ff9640 1 -- 172.21.15.26:0/3843577214 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc9e0003e50 con 0x556a12952dc0 2024-09-11T23:19:56.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.445+0000 7fc9d0ff9640 1 -- 172.21.15.26:0/3843577214 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9e0004290 con 0x556a12952dc0 2024-09-11T23:19:56.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.446+0000 7fc9f35c6740 1 -- 172.21.15.26:0/3843577214 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556a12952dc0 msgr2=0x556a129748c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:56.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.446+0000 7fc9f35c6740 1 --2- 172.21.15.26:0/3843577214 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556a12952dc0 0x556a129748c0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fc9ec0697d0 tx=0x7fc9e0011c90 comp rx=0 tx=0).stop 2024-09-11T23:19:56.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.446+0000 7fc9f35c6740 1 -- 172.21.15.26:0/3843577214 shutdown_connections 2024-09-11T23:19:56.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.446+0000 7fc9f35c6740 1 --2- 172.21.15.26:0/3843577214 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556a1295d680 0x556a12974e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:56.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.446+0000 7fc9f35c6740 1 --2- 172.21.15.26:0/3843577214 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556a12952dc0 0x556a129748c0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:56.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.446+0000 7fc9f35c6740 1 --2- 172.21.15.26:0/3843577214 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556a129524a0 0x556a1295c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:56.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.446+0000 7fc9f35c6740 1 -- 172.21.15.26:0/3843577214 >> 172.21.15.26:0/3843577214 conn(0x556a1294ea30 msgr2=0x556a129591e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:19:56.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.446+0000 7fc9f35c6740 1 -- 172.21.15.26:0/3843577214 shutdown_connections 2024-09-11T23:19:56.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.446+0000 7fc9f35c6740 1 -- 172.21.15.26:0/3843577214 wait complete. 2024-09-11T23:19:56.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.446+0000 7fc9f35c6740 1 -- start start 2024-09-11T23:19:56.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.446+0000 7fc9f35c6740 1 Processor -- start 2024-09-11T23:19:56.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.447+0000 7fc9f35c6740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556a129524a0 0x556a129cfcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:56.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.447+0000 7fc9f35c6740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556a12952dc0 0x556a129d01f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:56.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.447+0000 7fc9f35c6740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556a1295d680 0x556a12962c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:56.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.447+0000 7fc9f35c6740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc9ec1b0fe0 con 0x556a12952dc0 2024-09-11T23:19:56.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.447+0000 7fc9f35c6740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fc9ec1b0910 con 0x556a1295d680 2024-09-11T23:19:56.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.447+0000 7fc9f35c6740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fc9ec1adb50 con 0x556a129524a0 2024-09-11T23:19:56.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.447+0000 7fc9f35c6740 10 client.0 _ll_register_callbacks cb 0x7fc9e404b070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:19:56.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.447+0000 7fc9ea59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556a12952dc0 0x556a129d01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:56.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.447+0000 7fc9ead9d640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556a129524a0 0x556a129cfcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:56.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.447+0000 7fc9ea59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556a12952dc0 0x556a129d01f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54260/0 (socket says 172.21.15.26:54260) 2024-09-11T23:19:56.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.447+0000 7fc9ead9d640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556a129524a0 0x556a129cfcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:55604/0 (socket says 172.21.15.26:55604) 2024-09-11T23:19:56.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.447+0000 7fc9ea59c640 1 -- 172.21.15.26:0/1309760917 learned_addr learned my addr 172.21.15.26:0/1309760917 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:19:56.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.447+0000 7fc9eb59e640 1 --2- 172.21.15.26:0/1309760917 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556a1295d680 0x556a12962c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:56.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.447+0000 7fc9ead9d640 1 -- 172.21.15.26:0/1309760917 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556a1295d680 msgr2=0x556a12962c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:19:56.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.447+0000 7fc9ead9d640 1 --2- 172.21.15.26:0/1309760917 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556a1295d680 0x556a12962c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:56.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.447+0000 7fc9ead9d640 1 -- 172.21.15.26:0/1309760917 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556a12952dc0 msgr2=0x556a129d01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:19:56.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.447+0000 7fc9ead9d640 1 --2- 172.21.15.26:0/1309760917 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556a12952dc0 0x556a129d01f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:56.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.447+0000 7fc9ead9d640 1 -- 172.21.15.26:0/1309760917 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a12963410 con 0x556a129524a0 2024-09-11T23:19:56.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.447+0000 7fc9ead9d640 1 --2- 172.21.15.26:0/1309760917 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556a129524a0 0x556a129cfcb0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc9dc0cc4d0 tx=0x7fc9dc0bb040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8df603576cd51af4 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:19:56.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.447+0000 7fc9ceff5640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-09-11T23:19:56.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.448+0000 7fc9ceff5640 1 -- 172.21.15.26:0/1309760917 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9dc09b2a0 con 0x556a129524a0 2024-09-11T23:19:56.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.448+0000 7fc9ceff5640 1 -- 172.21.15.26:0/1309760917 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fc9dc0cc750 con 0x556a129524a0 2024-09-11T23:19:56.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.448+0000 7fc9f35c6740 1 -- 172.21.15.26:0/1309760917 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc9e0003e50 con 0x556a129524a0 2024-09-11T23:19:56.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.448+0000 7fc9ceff5640 1 -- 172.21.15.26:0/1309760917 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc9dc0cc950 con 0x556a129524a0 2024-09-11T23:19:56.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.448+0000 7fc9f35c6740 10 client.25444 fetch_fsmap learned FSMap version 29 2024-09-11T23:19:56.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.448+0000 7fc9f35c6740 1 -- 172.21.15.26:0/1309760917 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556a129d7420 con 0x556a129524a0 2024-09-11T23:19:56.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.448+0000 7fc9ceff5640 1 -- 172.21.15.26:0/1309760917 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc9e0003e50 con 0x556a129524a0 2024-09-11T23:19:56.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.448+0000 7fc9ceff5640 1 -- 172.21.15.26:0/1309760917 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fc9dc0bb1d0 con 0x556a129524a0 2024-09-11T23:19:56.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.448+0000 7fc9f35c6740 10 client.25444 fetch_fsmap finished waiting for FSMap version 29 2024-09-11T23:19:56.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.448+0000 7fc9f35c6740 10 client.25444 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:19:56.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.448+0000 7fc9f35c6740 20 client.25444 populate_metadata read hostname 'smithi026' 2024-09-11T23:19:56.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.448+0000 7fc9f35c6740 1 --2- 172.21.15.26:0/1309760917 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x556a129d7b20 0x556a129d9f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:56.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.448+0000 7fc9f35c6740 4 client.25444 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T23:19:56.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.448+0000 7fc9f35c6740 1 -- 172.21.15.26:0/1309760917 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556a128bfd60 con 0x556a129d7b20 2024-09-11T23:19:56.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.448+0000 7fc9ea59c640 1 --2- 172.21.15.26:0/1309760917 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x556a129d7b20 0x556a129d9f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:56.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.449+0000 7fc9ea59c640 1 --2- 172.21.15.26:0/1309760917 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x556a129d7b20 0x556a129d9f60 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dafc844e04b599a9 server_cookie=abd2619352c16c93 in_seq=0 out_seq=0 2024-09-11T23:19:56.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.449+0000 7fc9ceff5640 10 client.25444 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:19:56.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.454+0000 7fc9ceff5640 1 -- 172.21.15.26:0/1309760917 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x556a128bfd60 con 0x556a129d7b20 2024-09-11T23:19:56.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.454+0000 7fc9ceff5640 10 client.25444 handle_command_reply: tid=0 2024-09-11T23:19:56.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.454+0000 7fc9f35c6740 1 client.25444 shutdown 2024-09-11T23:19:56.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.455+0000 7fc9f35c6740 1 -- 172.21.15.26:0/1309760917 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556a129524a0 msgr2=0x556a129cfcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:56.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.455+0000 7fc9f35c6740 1 --2- 172.21.15.26:0/1309760917 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556a129524a0 0x556a129cfcb0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc9dc0cc4d0 tx=0x7fc9dc0bb040 comp rx=0 tx=0).stop 2024-09-11T23:19:56.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.455+0000 7fc9f35c6740 1 -- 172.21.15.26:0/1309760917 shutdown_connections 2024-09-11T23:19:56.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.455+0000 7fc9f35c6740 1 --2- 172.21.15.26:0/1309760917 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556a1295d680 0x556a12962c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:56.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.455+0000 7fc9f35c6740 1 --2- 172.21.15.26:0/1309760917 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x556a129d7b20 0x556a129d9f60 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:56.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.455+0000 7fc9f35c6740 1 --2- 172.21.15.26:0/1309760917 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556a12952dc0 0x556a129d01f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:56.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.455+0000 7fc9f35c6740 1 --2- 172.21.15.26:0/1309760917 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556a129524a0 0x556a129cfcb0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:56.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.455+0000 7fc9ceff5640 0 client.25444 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:19:56.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.455+0000 7fc9f35c6740 1 -- 172.21.15.26:0/1309760917 >> 172.21.15.26:0/1309760917 conn(0x556a1294ea30 msgr2=0x556a129589c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:19:56.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.455+0000 7fc9f35c6740 1 -- 172.21.15.26:0/1309760917 shutdown_connections 2024-09-11T23:19:56.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.455+0000 7fc9f35c6740 1 -- 172.21.15.26:0/1309760917 wait complete. 2024-09-11T23:19:56.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.455+0000 7fc9f35c6740 20 client.25444 trim_cache size 0 max 16384 2024-09-11T23:19:56.456 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-11T23:19:56.456 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "no active scrubs running", 2024-09-11T23:19:56.456 INFO:teuthology.orchestra.run.smithi026.stdout: "scrubs": {} 2024-09-11T23:19:56.456 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-11T23:19:56.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.456+0000 7fc9f11ed640 1 -- 172.21.15.26:0/2322938857 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fc9d40a2cf0 msgr2=0x7fc9d40a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:56.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.456+0000 7fc9f11ed640 1 --2- 172.21.15.26:0/2322938857 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fc9d40a2cf0 0x7fc9d40a51b0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fc9e404a000 tx=0x7fc9e4047000 comp rx=0 tx=0).stop 2024-09-11T23:19:56.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.456+0000 7fc9f11ed640 1 -- 172.21.15.26:0/2322938857 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc9ec102d30 msgr2=0x7fc9ec19a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:56.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.456+0000 7fc9f11ed640 1 --2- 172.21.15.26:0/2322938857 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc9ec102d30 0x7fc9ec19a740 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc9dc00d950 tx=0x7fc9dc00de10 comp rx=0 tx=0).stop 2024-09-11T23:19:56.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.456+0000 7fc9f11ed640 1 -- 172.21.15.26:0/2322938857 shutdown_connections 2024-09-11T23:19:56.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.456+0000 7fc9f11ed640 1 --2- 172.21.15.26:0/2322938857 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fc9d40a2cf0 0x7fc9d40a51b0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:56.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.456+0000 7fc9f11ed640 1 --2- 172.21.15.26:0/2322938857 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc9ec1082a0 0x7fc9ec19f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:56.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.456+0000 7fc9f11ed640 1 --2- 172.21.15.26:0/2322938857 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc9ec1036e0 0x7fc9ec19ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:56.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.456+0000 7fc9f11ed640 1 --2- 172.21.15.26:0/2322938857 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc9ec102d30 0x7fc9ec19a740 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:56.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.456+0000 7fc9f11ed640 1 -- 172.21.15.26:0/2322938857 >> 172.21.15.26:0/2322938857 conn(0x7fc9ec0fc7d0 msgr2=0x7fc9ec0fe670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:19:56.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.456+0000 7fc9f11ed640 1 -- 172.21.15.26:0/2322938857 shutdown_connections 2024-09-11T23:19:56.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:56.456+0000 7fc9f11ed640 1 -- 172.21.15.26:0/2322938857 wait complete. 2024-09-11T23:19:57.250 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-11T23:19:57.250 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:19:57.528 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:19:58.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:58.034+0000 7f23d19f2640 1 -- 172.21.15.26:0/2242456811 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f23cc106f00 msgr2=0x7f23cc10ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:58.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:58.034+0000 7f23d19f2640 1 --2- 172.21.15.26:0/2242456811 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f23cc106f00 0x7f23cc10ea80 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f23bc009be0 tx=0x7f23bc02a6c0 comp rx=0 tx=0).stop 2024-09-11T23:19:58.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:58.034+0000 7f23d19f2640 1 -- 172.21.15.26:0/2242456811 shutdown_connections 2024-09-11T23:19:58.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:58.034+0000 7f23d19f2640 1 --2- 172.21.15.26:0/2242456811 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f23cc106f00 0x7f23cc10ea80 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:58.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:58.034+0000 7f23d19f2640 1 --2- 172.21.15.26:0/2242456811 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f23cc102780 0x7f23cc1067d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:58.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:58.034+0000 7f23d19f2640 1 --2- 172.21.15.26:0/2242456811 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23cc101dd0 0x7f23cc1021b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:58.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:58.034+0000 7f23d19f2640 1 -- 172.21.15.26:0/2242456811 >> 172.21.15.26:0/2242456811 conn(0x7f23cc0fd8b0 msgr2=0x7f23cc0ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:19:58.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:58.034+0000 7f23d19f2640 1 -- 172.21.15.26:0/2242456811 shutdown_connections 2024-09-11T23:19:58.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:58.034+0000 7f23d19f2640 1 -- 172.21.15.26:0/2242456811 wait complete. 2024-09-11T23:19:58.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:58.035+0000 7f23d19f2640 1 Processor -- start 2024-09-11T23:19:58.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:58.035+0000 7f23d19f2640 1 -- start start 2024-09-11T23:19:58.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:58.035+0000 7f23d19f2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23cc101dd0 0x7f23cc198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:58.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:58.035+0000 7f23d19f2640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f23cc102780 0x7f23cc198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:58.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:58.035+0000 7f23d19f2640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f23cc106f00 0x7f23cc19cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:58.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:58.035+0000 7f23d19f2640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f23cc111f10 con 0x7f23cc101dd0 2024-09-11T23:19:58.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:58.035+0000 7f23d19f2640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f23cc111d90 con 0x7f23cc106f00 2024-09-11T23:19:58.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:58.035+0000 7f23d19f2640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f23cc112090 con 0x7f23cc102780 2024-09-11T23:19:58.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:58.036+0000 7f23ca7fc640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f23cc102780 0x7f23cc198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:58.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:58.036+0000 7f23ca7fc640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f23cc102780 0x7f23cc198950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:55620/0 (socket says 172.21.15.26:55620) 2024-09-11T23:19:58.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:58.036+0000 7f23ca7fc640 1 -- 172.21.15.26:0/3496207556 learned_addr learned my addr 172.21.15.26:0/3496207556 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:19:58.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:58.036+0000 7f23caffd640 1 --2- 172.21.15.26:0/3496207556 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23cc101dd0 0x7f23cc198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:58.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:58.036+0000 7f23cb7fe640 1 --2- 172.21.15.26:0/3496207556 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f23cc106f00 0x7f23cc19cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:58.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:58.036+0000 7f23ca7fc640 1 -- 172.21.15.26:0/3496207556 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f23cc106f00 msgr2=0x7f23cc19cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:58.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:58.036+0000 7f23ca7fc640 1 --2- 172.21.15.26:0/3496207556 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f23cc106f00 0x7f23cc19cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:58.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:58.036+0000 7f23ca7fc640 1 -- 172.21.15.26:0/3496207556 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23cc101dd0 msgr2=0x7f23cc198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:58.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:58.036+0000 7f23ca7fc640 1 --2- 172.21.15.26:0/3496207556 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23cc101dd0 0x7f23cc198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:58.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:58.036+0000 7f23ca7fc640 1 -- 172.21.15.26:0/3496207556 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23cc19d3c0 con 0x7f23cc102780 2024-09-11T23:19:58.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:58.036+0000 7f23ca7fc640 1 --2- 172.21.15.26:0/3496207556 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f23cc102780 0x7f23cc198950 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f23b400e9f0 tx=0x7f23b400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:19:58.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:58.036+0000 7f23d09f0640 1 -- 172.21.15.26:0/3496207556 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23b400cd80 con 0x7f23cc102780 2024-09-11T23:19:58.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:58.037+0000 7f23d19f2640 1 -- 172.21.15.26:0/3496207556 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23cc19d6b0 con 0x7f23cc102780 2024-09-11T23:19:58.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:58.037+0000 7f23d09f0640 1 -- 172.21.15.26:0/3496207556 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f23b4010910 con 0x7f23cc102780 2024-09-11T23:19:58.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:58.037+0000 7f23d09f0640 1 -- 172.21.15.26:0/3496207556 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23b4018990 con 0x7f23cc102780 2024-09-11T23:19:58.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:58.037+0000 7f23d19f2640 1 -- 172.21.15.26:0/3496207556 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23cc1a4f90 con 0x7f23cc102780 2024-09-11T23:19:58.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:58.038+0000 7f23d19f2640 1 -- 172.21.15.26:0/3496207556 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23cc1021b0 con 0x7f23cc102780 2024-09-11T23:19:58.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:58.039+0000 7f23d09f0640 1 -- 172.21.15.26:0/3496207556 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f23b4010ac0 con 0x7f23cc102780 2024-09-11T23:19:58.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:58.040+0000 7f23d09f0640 1 --2- 172.21.15.26:0/3496207556 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f23b80a2cf0 0x7f23b80a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:58.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:58.040+0000 7f23d09f0640 1 -- 172.21.15.26:0/3496207556 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f23b40cf6d0 con 0x7f23cc102780 2024-09-11T23:19:58.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:58.040+0000 7f23caffd640 1 --2- 172.21.15.26:0/3496207556 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f23b80a2cf0 0x7f23b80a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:58.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:58.040+0000 7f23caffd640 1 --2- 172.21.15.26:0/3496207556 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f23b80a2cf0 0x7f23b80a51b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f23c4009530 tx=0x7f23c4009260 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:19:58.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:58.046+0000 7f23d09f0640 1 -- 172.21.15.26:0/3496207556 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f23b409bcb0 con 0x7f23cc102780 2024-09-11T23:19:58.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:19:57 smithi136 ceph-mon[35121]: pgmap v50: 97 pgs: 97 active+clean; 5.7 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 443 KiB/s wr, 86 op/s 2024-09-11T23:19:58.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:19:57 smithi136 ceph-mon[35121]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:19:58.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:19:57 smithi026 ceph-mon[31172]: pgmap v50: 97 pgs: 97 active+clean; 5.7 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 443 KiB/s wr, 86 op/s 2024-09-11T23:19:58.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:19:57 smithi026 ceph-mon[31172]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:19:58.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:58.302+0000 7f23d19f2640 1 -- 172.21.15.26:0/3496207556 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f23cc10a0e0 con 0x7f23cc102780 2024-09-11T23:19:58.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:19:57 smithi076 ceph-mon[35296]: pgmap v50: 97 pgs: 97 active+clean; 5.7 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 443 KiB/s wr, 86 op/s 2024-09-11T23:19:58.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:19:57 smithi076 ceph-mon[35296]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:19:58.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:58.303+0000 7f23d09f0640 1 -- 172.21.15.26:0/3496207556 <== mon.1 v2:172.21.15.136:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11972 (secure 0 0 0) 0x7f23b409ffb0 con 0x7f23cc102780 2024-09-11T23:19:58.303 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-11T23:19:58.304 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":30,"btime":"2024-09-11T23:19:56:946565+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14499,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/367659515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":367659515},{"type":"v1","addr":"172.21.15.26:6835","nonce":367659515}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14511,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/2041065299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":2041065299},{"type":"v1","addr":"172.21.15.26:6837","nonce":2041065299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:16:26.544203+0000","modified":"2024-09-11T23:19:56.097901+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24485,"mds_1":24463,"mds_2":24458,"mds_3":24448,"mds_4":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14550":{"gid":14550,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6839/2233068305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":2233068305},{"type":"v1","addr":"172.21.15.26:6839","nonce":2233068305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14562":{"gid":14562,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6841/2984982336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":2984982336},{"type":"v1","addr":"172.21.15.26:6841","nonce":2984982336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6833/2019841455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":2019841455},{"type":"v1","addr":"172.21.15.136:6833","nonce":2019841455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.76:6833/3438147050","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3438147050},{"type":"v1","addr":"172.21.15.76:6833","nonce":3438147050}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6835/3617665975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3617665975},{"type":"v1","addr":"172.21.15.136:6835","nonce":3617665975}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":12,"addr":"172.21.15.76:6835/393308499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":393308499},{"type":"v1","addr":"172.21.15.76:6835","nonce":393308499}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":10,"addr":"172.21.15.136:6837/1633082387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":1633082387},{"type":"v1","addr":"172.21.15.136:6837","nonce":1633082387}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.76:6837/2078978748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":2078978748},{"type":"v1","addr":"172.21.15.76:6837","nonce":2078978748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.136:6839/3211398630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":3211398630},{"type":"v1","addr":"172.21.15.136:6839","nonce":3211398630}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.76:6839/2077637090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6838","nonce":2077637090},{"type":"v1","addr":"172.21.15.76:6839","nonce":2077637090}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24485,"qdb_cluster":[24485,24463,24458,24448,24443]},"id":1}]} 2024-09-11T23:19:58.304 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 30 2024-09-11T23:19:58.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:58.305+0000 7f23d19f2640 1 -- 172.21.15.26:0/3496207556 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f23b80a2cf0 msgr2=0x7f23b80a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:58.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:58.305+0000 7f23d19f2640 1 --2- 172.21.15.26:0/3496207556 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f23b80a2cf0 0x7f23b80a51b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f23c4009530 tx=0x7f23c4009260 comp rx=0 tx=0).stop 2024-09-11T23:19:58.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:58.305+0000 7f23d19f2640 1 -- 172.21.15.26:0/3496207556 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f23cc102780 msgr2=0x7f23cc198950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:58.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:58.305+0000 7f23d19f2640 1 --2- 172.21.15.26:0/3496207556 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f23cc102780 0x7f23cc198950 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f23b400e9f0 tx=0x7f23b400eeb0 comp rx=0 tx=0).stop 2024-09-11T23:19:58.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:58.305+0000 7f23d19f2640 1 -- 172.21.15.26:0/3496207556 shutdown_connections 2024-09-11T23:19:58.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:58.305+0000 7f23d19f2640 1 --2- 172.21.15.26:0/3496207556 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f23b80a2cf0 0x7f23b80a51b0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:58.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:58.305+0000 7f23d19f2640 1 --2- 172.21.15.26:0/3496207556 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f23cc106f00 0x7f23cc19cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:58.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:58.305+0000 7f23d19f2640 1 --2- 172.21.15.26:0/3496207556 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f23cc102780 0x7f23cc198950 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:58.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:58.305+0000 7f23d19f2640 1 --2- 172.21.15.26:0/3496207556 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23cc101dd0 0x7f23cc198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:58.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:58.305+0000 7f23d19f2640 1 -- 172.21.15.26:0/3496207556 >> 172.21.15.26:0/3496207556 conn(0x7f23cc0fd8b0 msgr2=0x7f23cc0fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:19:58.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:58.305+0000 7f23d19f2640 1 -- 172.21.15.26:0/3496207556 shutdown_connections 2024-09-11T23:19:58.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:58.305+0000 7f23d19f2640 1 -- 172.21.15.26:0/3496207556 wait complete. 2024-09-11T23:19:58.989 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:19:58 smithi026 ceph-mon[31172]: from='client.? 172.21.15.26:0/3496207556' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:19:59.016 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-11T23:19:59.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:19:58 smithi136 ceph-mon[35121]: from='client.? 172.21.15.26:0/3496207556' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:19:59.285 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:19:59.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:19:58 smithi076 ceph-mon[35296]: from='client.? 172.21.15.26:0/3496207556' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:19:59.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.778+0000 7ff9960b6640 1 -- 172.21.15.26:0/2642784807 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff990101420 msgr2=0x7ff99010d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:59.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.778+0000 7ff9960b6640 1 --2- 172.21.15.26:0/2642784807 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff990101420 0x7ff99010d950 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7ff988009cd0 tx=0x7ff98802a8e0 comp rx=0 tx=0).stop 2024-09-11T23:19:59.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.778+0000 7ff9960b6640 1 -- 172.21.15.26:0/2642784807 shutdown_connections 2024-09-11T23:19:59.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.778+0000 7ff9960b6640 1 --2- 172.21.15.26:0/2642784807 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff990100a80 0x7ff990100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:59.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.778+0000 7ff9960b6640 1 --2- 172.21.15.26:0/2642784807 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff990101420 0x7ff99010d950 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:59.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.778+0000 7ff9960b6640 1 --2- 172.21.15.26:0/2642784807 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff990106a80 0x7ff990106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:59.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.778+0000 7ff9960b6640 1 -- 172.21.15.26:0/2642784807 >> 172.21.15.26:0/2642784807 conn(0x7ff9900fc7a0 msgr2=0x7ff9900febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:19:59.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.778+0000 7ff9960b6640 1 -- 172.21.15.26:0/2642784807 shutdown_connections 2024-09-11T23:19:59.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.778+0000 7ff9960b6640 1 -- 172.21.15.26:0/2642784807 wait complete. 2024-09-11T23:19:59.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.778+0000 7ff9960b6640 1 Processor -- start 2024-09-11T23:19:59.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.779+0000 7ff9960b6640 1 -- start start 2024-09-11T23:19:59.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.779+0000 7ff9960b6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff990100a80 0x7ff990197330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:59.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.779+0000 7ff9960b6640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff990101420 0x7ff990197870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:59.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.779+0000 7ff9960b6640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff990106a80 0x7ff99019bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:59.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.779+0000 7ff9960b6640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff990110ed0 con 0x7ff990100a80 2024-09-11T23:19:59.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.779+0000 7ff9960b6640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7ff990110d50 con 0x7ff990101420 2024-09-11T23:19:59.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.779+0000 7ff9960b6640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ff990111050 con 0x7ff990106a80 2024-09-11T23:19:59.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.779+0000 7ff98effd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff990101420 0x7ff990197870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:59.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.779+0000 7ff98f7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff990100a80 0x7ff990197330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:59.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.779+0000 7ff98f7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff990100a80 0x7ff990197330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54302/0 (socket says 172.21.15.26:54302) 2024-09-11T23:19:59.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.779+0000 7ff98effd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff990101420 0x7ff990197870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:39600/0 (socket says 172.21.15.26:39600) 2024-09-11T23:19:59.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.779+0000 7ff98f7fe640 1 -- 172.21.15.26:0/2783431837 learned_addr learned my addr 172.21.15.26:0/2783431837 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:19:59.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.779+0000 7ff98ffff640 1 --2- 172.21.15.26:0/2783431837 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff990106a80 0x7ff99019bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:59.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.779+0000 7ff98effd640 1 -- 172.21.15.26:0/2783431837 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff990106a80 msgr2=0x7ff99019bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:59.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.779+0000 7ff98effd640 1 --2- 172.21.15.26:0/2783431837 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff990106a80 0x7ff99019bc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:59.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.779+0000 7ff98effd640 1 -- 172.21.15.26:0/2783431837 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff990100a80 msgr2=0x7ff990197330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:59.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.779+0000 7ff98effd640 1 --2- 172.21.15.26:0/2783431837 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff990100a80 0x7ff990197330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:59.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.779+0000 7ff98effd640 1 -- 172.21.15.26:0/2783431837 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff99019c2e0 con 0x7ff990101420 2024-09-11T23:19:59.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.780+0000 7ff98ffff640 1 --2- 172.21.15.26:0/2783431837 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff990106a80 0x7ff99019bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:19:59.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.780+0000 7ff98f7fe640 1 --2- 172.21.15.26:0/2783431837 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff990100a80 0x7ff990197330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:19:59.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.780+0000 7ff98effd640 1 --2- 172.21.15.26:0/2783431837 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff990101420 0x7ff990197870 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ff984005fd0 tx=0x7ff98400dde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:19:59.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.780+0000 7ff98cff9640 1 -- 172.21.15.26:0/2783431837 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff984009c30 con 0x7ff990101420 2024-09-11T23:19:59.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.780+0000 7ff98cff9640 1 -- 172.21.15.26:0/2783431837 <== mon.2 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff984009dd0 con 0x7ff990101420 2024-09-11T23:19:59.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.780+0000 7ff9960b6640 1 -- 172.21.15.26:0/2783431837 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff99019c5d0 con 0x7ff990101420 2024-09-11T23:19:59.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.780+0000 7ff98cff9640 1 -- 172.21.15.26:0/2783431837 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff984010460 con 0x7ff990101420 2024-09-11T23:19:59.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.780+0000 7ff9960b6640 1 -- 172.21.15.26:0/2783431837 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9901a3eb0 con 0x7ff990101420 2024-09-11T23:19:59.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.782+0000 7ff98cff9640 1 -- 172.21.15.26:0/2783431837 <== mon.2 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7ff984003ef0 con 0x7ff990101420 2024-09-11T23:19:59.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.783+0000 7ff99848f740 1 Processor -- start 2024-09-11T23:19:59.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.783+0000 7ff99848f740 1 -- start start 2024-09-11T23:19:59.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.783+0000 7ff99848f740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559051aa1a20 0x559051aa0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:59.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.783+0000 7ff99848f740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559051aa05d0 0x559051aa0a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:59.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.783+0000 7ff99848f740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559051aaf970 0x559051ab1d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:59.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.783+0000 7ff99848f740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x559051a062f0 con 0x559051aa05d0 2024-09-11T23:19:59.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.783+0000 7ff99848f740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5590519adcf0 con 0x559051aaf970 2024-09-11T23:19:59.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.783+0000 7ff99848f740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x559051a022b0 con 0x559051aa1a20 2024-09-11T23:19:59.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.783+0000 7ff98ffff640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559051aa1a20 0x559051aa0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:59.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.783+0000 7ff98ffff640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559051aa1a20 0x559051aa0090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:55636/0 (socket says 172.21.15.26:55636) 2024-09-11T23:19:59.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.783+0000 7ff98f7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559051aa05d0 0x559051aa0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:59.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.783+0000 7ff98ffff640 1 -- 172.21.15.26:0/1696165600 learned_addr learned my addr 172.21.15.26:0/1696165600 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:19:59.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.783+0000 7ff98cff9640 1 --2- 172.21.15.26:0/2783431837 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7ff9780a2cf0 0x7ff9780a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:59.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.783+0000 7ff98effd640 1 --2- 172.21.15.26:0/1696165600 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559051aaf970 0x559051ab1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:59.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.783+0000 7ff98ffff640 1 --2- 172.21.15.26:0/2783431837 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7ff9780a2cf0 0x7ff9780a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:59.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.783+0000 7ff98ffff640 1 -- 172.21.15.26:0/1696165600 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559051aaf970 msgr2=0x559051ab1d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:19:59.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.783+0000 7ff98ffff640 1 --2- 172.21.15.26:0/1696165600 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559051aaf970 0x559051ab1d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:59.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.783+0000 7ff98ffff640 1 -- 172.21.15.26:0/1696165600 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559051aa05d0 msgr2=0x559051aa0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:19:59.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.783+0000 7ff98ffff640 1 --2- 172.21.15.26:0/1696165600 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559051aa05d0 0x559051aa0a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:59.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.784+0000 7ff98ffff640 1 -- 172.21.15.26:0/1696165600 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5590519923a0 con 0x559051aa1a20 2024-09-11T23:19:59.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.784+0000 7ff98cff9640 1 -- 172.21.15.26:0/2783431837 <== mon.2 v2:172.21.15.76:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7ff9840cef80 con 0x7ff990101420 2024-09-11T23:19:59.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.784+0000 7ff98ffff640 1 --2- 172.21.15.26:0/1696165600 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559051aa1a20 0x559051aa0090 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff988031860 tx=0x7ff988031d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cf97e5eabc92dab6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:19:59.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.784+0000 7ff98ffff640 1 --2- 172.21.15.26:0/2783431837 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7ff9780a2cf0 0x7ff9780a51b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ff988048000 tx=0x7ff98804b000 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:19:59.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.784+0000 7ff975ffb640 1 -- 172.21.15.26:0/1696165600 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff988050070 con 0x559051aa1a20 2024-09-11T23:19:59.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.784+0000 7ff975ffb640 1 -- 172.21.15.26:0/1696165600 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff988036040 con 0x559051aa1a20 2024-09-11T23:19:59.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.784+0000 7ff975ffb640 1 -- 172.21.15.26:0/1696165600 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff988030bb0 con 0x559051aa1a20 2024-09-11T23:19:59.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.785+0000 7ff99848f740 1 -- 172.21.15.26:0/1696165600 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559051aa1a20 msgr2=0x559051aa0090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:59.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.785+0000 7ff99848f740 1 --2- 172.21.15.26:0/1696165600 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559051aa1a20 0x559051aa0090 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff988031860 tx=0x7ff988031d10 comp rx=0 tx=0).stop 2024-09-11T23:19:59.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.785+0000 7ff99848f740 1 -- 172.21.15.26:0/1696165600 shutdown_connections 2024-09-11T23:19:59.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.785+0000 7ff99848f740 1 --2- 172.21.15.26:0/1696165600 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559051aaf970 0x559051ab1d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:59.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.785+0000 7ff99848f740 1 --2- 172.21.15.26:0/1696165600 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559051aa05d0 0x559051aa0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:59.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.785+0000 7ff99848f740 1 --2- 172.21.15.26:0/1696165600 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559051aa1a20 0x559051aa0090 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:59.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.785+0000 7ff99848f740 1 -- 172.21.15.26:0/1696165600 >> 172.21.15.26:0/1696165600 conn(0x559051a69310 msgr2=0x559051a9cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:19:59.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.785+0000 7ff99848f740 1 -- 172.21.15.26:0/1696165600 shutdown_connections 2024-09-11T23:19:59.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.785+0000 7ff99848f740 1 -- 172.21.15.26:0/1696165600 wait complete. 2024-09-11T23:19:59.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.785+0000 7ff99848f740 1 -- start start 2024-09-11T23:19:59.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.786+0000 7ff99848f740 1 Processor -- start 2024-09-11T23:19:59.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.786+0000 7ff99848f740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559051aa05d0 0x559051aaf2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:59.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.786+0000 7ff99848f740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559051aa1a20 0x559051aac590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:59.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.786+0000 7ff99848f740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559051aaf970 0x559051aacaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:59.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.786+0000 7ff99848f740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x559051ab24e0 con 0x559051aa05d0 2024-09-11T23:19:59.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.786+0000 7ff99848f740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5590519c1060 con 0x559051aa1a20 2024-09-11T23:19:59.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.786+0000 7ff99848f740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x559051ab2660 con 0x559051aaf970 2024-09-11T23:19:59.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.786+0000 7ff99848f740 10 client.0 _ll_register_callbacks cb 0x5590517148c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:19:59.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.786+0000 7ff98f7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559051aa05d0 0x559051aaf2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:59.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.786+0000 7ff98effd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559051aa1a20 0x559051aac590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:59.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.786+0000 7ff98f7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559051aa05d0 0x559051aaf2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54326/0 (socket says 172.21.15.26:54326) 2024-09-11T23:19:59.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.786+0000 7ff98ffff640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559051aaf970 0x559051aacaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:59.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.786+0000 7ff98f7fe640 1 -- 172.21.15.26:0/4141479198 learned_addr learned my addr 172.21.15.26:0/4141479198 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:19:59.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.786+0000 7ff98f7fe640 1 -- 172.21.15.26:0/4141479198 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559051aaf970 msgr2=0x559051aacaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:19:59.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.786+0000 7ff98f7fe640 1 --2- 172.21.15.26:0/4141479198 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559051aaf970 0x559051aacaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:59.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.786+0000 7ff98f7fe640 1 -- 172.21.15.26:0/4141479198 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559051aa1a20 msgr2=0x559051aac590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:19:59.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.786+0000 7ff98f7fe640 1 --2- 172.21.15.26:0/4141479198 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559051aa1a20 0x559051aac590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:59.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.786+0000 7ff98f7fe640 1 -- 172.21.15.26:0/4141479198 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5590517773c0 con 0x559051aa05d0 2024-09-11T23:19:59.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.786+0000 7ff98effd640 1 --2- 172.21.15.26:0/4141479198 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559051aa1a20 0x559051aac590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:19:59.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.787+0000 7ff98f7fe640 1 --2- 172.21.15.26:0/4141479198 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559051aa05d0 0x559051aaf2e0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7ff980012d20 tx=0x7ff98000f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10a3ebc3b10783bb server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:19:59.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.787+0000 7ff96effd640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:19:59.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.787+0000 7ff96effd640 1 -- 172.21.15.26:0/4141479198 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff98000fe20 con 0x559051aa05d0 2024-09-11T23:19:59.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.787+0000 7ff99848f740 1 -- 172.21.15.26:0/4141479198 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff988036040 con 0x559051aa05d0 2024-09-11T23:19:59.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.787+0000 7ff96effd640 1 -- 172.21.15.26:0/4141479198 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7ff980005020 con 0x559051aa05d0 2024-09-11T23:19:59.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.787+0000 7ff96effd640 1 -- 172.21.15.26:0/4141479198 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff980005220 con 0x559051aa05d0 2024-09-11T23:19:59.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.787+0000 7ff99848f740 10 client.15828 fetch_fsmap learned FSMap version 30 2024-09-11T23:19:59.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.787+0000 7ff99848f740 1 -- 172.21.15.26:0/4141479198 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559051abdfa0 con 0x559051aa05d0 2024-09-11T23:19:59.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.787+0000 7ff96effd640 1 -- 172.21.15.26:0/4141479198 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff988036040 con 0x559051aa05d0 2024-09-11T23:19:59.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.787+0000 7ff96effd640 1 -- 172.21.15.26:0/4141479198 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7ff980025570 con 0x559051aa05d0 2024-09-11T23:19:59.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.787+0000 7ff99848f740 10 client.15828 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:19:59.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.787+0000 7ff99848f740 10 client.15828 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-09-11T23:19:59.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.787+0000 7ff99848f740 20 client.15828 populate_metadata read hostname 'smithi026' 2024-09-11T23:19:59.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.787+0000 7ff99848f740 1 --2- 172.21.15.26:0/4141479198 >> [v2:172.21.15.76:6834/393308499,v1:172.21.15.76:6835/393308499] conn(0x559051abf3b0 0x559051ac17a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:59.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.788+0000 7ff99848f740 4 client.15828 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:19:59.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.788+0000 7ff99848f740 1 -- 172.21.15.26:0/4141479198 --> [v2:172.21.15.76:6834/393308499,v1:172.21.15.76:6835/393308499] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559051a13860 con 0x559051abf3b0 2024-09-11T23:19:59.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.788+0000 7ff98effd640 1 --2- 172.21.15.26:0/4141479198 >> [v2:172.21.15.76:6834/393308499,v1:172.21.15.76:6835/393308499] conn(0x559051abf3b0 0x559051ac17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:59.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.788+0000 7ff98effd640 1 --2- 172.21.15.26:0/4141479198 >> [v2:172.21.15.76:6834/393308499,v1:172.21.15.76:6835/393308499] conn(0x559051abf3b0 0x559051ac17a0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d61dbca79c25ab09 server_cookie=626d7a41793688a4 in_seq=0 out_seq=0 2024-09-11T23:19:59.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.788+0000 7ff96effd640 10 client.15828 ms_handle_connect on v2:172.21.15.76:6834/393308499 2024-09-11T23:19:59.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.789+0000 7ff96effd640 1 -- 172.21.15.26:0/4141479198 <== mds.4 v2:172.21.15.76:6834/393308499 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559051a13860 con 0x559051abf3b0 2024-09-11T23:19:59.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.789+0000 7ff96effd640 10 client.15828 handle_command_reply: tid=0 2024-09-11T23:19:59.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.789+0000 7ff99848f740 1 client.15828 shutdown 2024-09-11T23:19:59.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.789+0000 7ff99848f740 1 -- 172.21.15.26:0/4141479198 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559051aa05d0 msgr2=0x559051aaf2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:59.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.789+0000 7ff99848f740 1 --2- 172.21.15.26:0/4141479198 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559051aa05d0 0x559051aaf2e0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7ff980012d20 tx=0x7ff98000f790 comp rx=0 tx=0).stop 2024-09-11T23:19:59.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.789+0000 7ff99848f740 1 -- 172.21.15.26:0/4141479198 shutdown_connections 2024-09-11T23:19:59.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.789+0000 7ff99848f740 1 --2- 172.21.15.26:0/4141479198 >> [v2:172.21.15.76:6834/393308499,v1:172.21.15.76:6835/393308499] conn(0x559051abf3b0 0x559051ac17a0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:59.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.789+0000 7ff99848f740 1 --2- 172.21.15.26:0/4141479198 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559051aa1a20 0x559051aac590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:59.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.789+0000 7ff99848f740 1 --2- 172.21.15.26:0/4141479198 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559051aaf970 0x559051aacaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:59.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.789+0000 7ff99848f740 1 --2- 172.21.15.26:0/4141479198 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559051aa05d0 0x559051aaf2e0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:59.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.789+0000 7ff99848f740 1 -- 172.21.15.26:0/4141479198 >> 172.21.15.26:0/4141479198 conn(0x559051a69310 msgr2=0x559051aa29c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:19:59.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.789+0000 7ff96effd640 0 client.15828 ms_handle_reset on v2:172.21.15.76:6834/393308499 2024-09-11T23:19:59.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.789+0000 7ff99848f740 1 -- 172.21.15.26:0/4141479198 shutdown_connections 2024-09-11T23:19:59.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.789+0000 7ff99848f740 1 -- 172.21.15.26:0/4141479198 wait complete. 2024-09-11T23:19:59.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.789+0000 7ff99848f740 20 client.15828 trim_cache size 0 max 16384 2024-09-11T23:19:59.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.805+0000 7ff99848f740 1 Processor -- start 2024-09-11T23:19:59.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.805+0000 7ff99848f740 1 -- start start 2024-09-11T23:19:59.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.805+0000 7ff99848f740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559051aa05d0 0x559051abb740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:59.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.805+0000 7ff99848f740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559051aa1a20 0x559051aa9930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:59.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.805+0000 7ff99848f740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559051aaf970 0x559051aa9e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:59.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.805+0000 7ff99848f740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff9901ab6d0 con 0x559051aa1a20 2024-09-11T23:19:59.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.805+0000 7ff99848f740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7ff9901ad240 con 0x559051aaf970 2024-09-11T23:19:59.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.805+0000 7ff99848f740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ff9901abda0 con 0x559051aa05d0 2024-09-11T23:19:59.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.805+0000 7ff98effd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559051aa1a20 0x559051aa9930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:59.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.805+0000 7ff98effd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559051aa1a20 0x559051aa9930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54332/0 (socket says 172.21.15.26:54332) 2024-09-11T23:19:59.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.805+0000 7ff98effd640 1 -- 172.21.15.26:0/3081076353 learned_addr learned my addr 172.21.15.26:0/3081076353 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:19:59.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.805+0000 7ff98f7fe640 1 --2- 172.21.15.26:0/3081076353 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559051aa05d0 0x559051abb740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:59.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.805+0000 7ff98ffff640 1 --2- 172.21.15.26:0/3081076353 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559051aaf970 0x559051aa9e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:59.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.805+0000 7ff98effd640 1 -- 172.21.15.26:0/3081076353 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559051aa05d0 msgr2=0x559051abb740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:19:59.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.805+0000 7ff98effd640 1 --2- 172.21.15.26:0/3081076353 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559051aa05d0 0x559051abb740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:59.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.805+0000 7ff98effd640 1 -- 172.21.15.26:0/3081076353 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559051aaf970 msgr2=0x559051aa9e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:19:59.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.805+0000 7ff98effd640 1 --2- 172.21.15.26:0/3081076353 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559051aaf970 0x559051aa9e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:59.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.805+0000 7ff98effd640 1 -- 172.21.15.26:0/3081076353 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559051aaf650 con 0x559051aa1a20 2024-09-11T23:19:59.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.806+0000 7ff98effd640 1 --2- 172.21.15.26:0/3081076353 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559051aa1a20 0x559051aa9930 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7ff98400bfd0 tx=0x7ff98409b460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92d344638d131169 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:19:59.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.806+0000 7ff9757fa640 1 -- 172.21.15.26:0/3081076353 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff984020040 con 0x559051aa1a20 2024-09-11T23:19:59.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.806+0000 7ff9757fa640 1 -- 172.21.15.26:0/3081076353 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff984099600 con 0x559051aa1a20 2024-09-11T23:19:59.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.806+0000 7ff9757fa640 1 -- 172.21.15.26:0/3081076353 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9840d10e0 con 0x559051aa1a20 2024-09-11T23:19:59.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.807+0000 7ff99848f740 1 -- 172.21.15.26:0/3081076353 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559051aa1a20 msgr2=0x559051aa9930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:59.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.807+0000 7ff99848f740 1 --2- 172.21.15.26:0/3081076353 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559051aa1a20 0x559051aa9930 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7ff98400bfd0 tx=0x7ff98409b460 comp rx=0 tx=0).stop 2024-09-11T23:19:59.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.807+0000 7ff99848f740 1 -- 172.21.15.26:0/3081076353 shutdown_connections 2024-09-11T23:19:59.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.807+0000 7ff99848f740 1 --2- 172.21.15.26:0/3081076353 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559051aaf970 0x559051aa9e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:59.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.807+0000 7ff99848f740 1 --2- 172.21.15.26:0/3081076353 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559051aa1a20 0x559051aa9930 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:59.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.807+0000 7ff99848f740 1 --2- 172.21.15.26:0/3081076353 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559051aa05d0 0x559051abb740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:59.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.807+0000 7ff99848f740 1 -- 172.21.15.26:0/3081076353 >> 172.21.15.26:0/3081076353 conn(0x559051a69310 msgr2=0x559051aaae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:19:59.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.807+0000 7ff99848f740 1 -- 172.21.15.26:0/3081076353 shutdown_connections 2024-09-11T23:19:59.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.807+0000 7ff99848f740 1 -- 172.21.15.26:0/3081076353 wait complete. 2024-09-11T23:19:59.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.807+0000 7ff99848f740 1 -- start start 2024-09-11T23:19:59.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.807+0000 7ff99848f740 1 Processor -- start 2024-09-11T23:19:59.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.808+0000 7ff99848f740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559051aa05d0 0x559051ab78b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:59.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.808+0000 7ff99848f740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559051aa1a20 0x559051ab7df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:59.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.808+0000 7ff99848f740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559051aaf970 0x559051ac6060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:59.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.808+0000 7ff99848f740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff9901acb40 con 0x559051aa05d0 2024-09-11T23:19:59.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.808+0000 7ff99848f740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7ff9901ac470 con 0x559051aa1a20 2024-09-11T23:19:59.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.808+0000 7ff99848f740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ff9901a96b0 con 0x559051aaf970 2024-09-11T23:19:59.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.808+0000 7ff99848f740 10 client.0 _ll_register_callbacks cb 0x7ff980009d10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:19:59.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.808+0000 7ff98f7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559051aa05d0 0x559051ab78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:59.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.808+0000 7ff98ffff640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559051aaf970 0x559051ac6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:59.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.808+0000 7ff98f7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559051aa05d0 0x559051ab78b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54342/0 (socket says 172.21.15.26:54342) 2024-09-11T23:19:59.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.808+0000 7ff98ffff640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559051aaf970 0x559051ac6060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:55668/0 (socket says 172.21.15.26:55668) 2024-09-11T23:19:59.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.808+0000 7ff98f7fe640 1 -- 172.21.15.26:0/2371690366 learned_addr learned my addr 172.21.15.26:0/2371690366 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:19:59.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.808+0000 7ff98effd640 1 --2- 172.21.15.26:0/2371690366 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559051aa1a20 0x559051ab7df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:59.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.808+0000 7ff98f7fe640 1 -- 172.21.15.26:0/2371690366 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559051aaf970 msgr2=0x559051ac6060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:19:59.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.808+0000 7ff98f7fe640 1 --2- 172.21.15.26:0/2371690366 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559051aaf970 0x559051ac6060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:59.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.808+0000 7ff98f7fe640 1 -- 172.21.15.26:0/2371690366 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559051aa1a20 msgr2=0x559051ab7df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:19:59.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.808+0000 7ff98f7fe640 1 --2- 172.21.15.26:0/2371690366 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559051aa1a20 0x559051ab7df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:59.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.808+0000 7ff98f7fe640 1 -- 172.21.15.26:0/2371690366 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559051ac65a0 con 0x559051aa05d0 2024-09-11T23:19:59.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.808+0000 7ff98effd640 1 --2- 172.21.15.26:0/2371690366 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559051aa1a20 0x559051ab7df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:19:59.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.808+0000 7ff98f7fe640 1 --2- 172.21.15.26:0/2371690366 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559051aa05d0 0x559051ab78b0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7ff9800059f0 tx=0x7ff98000fc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61897007060fa351 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:19:59.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.808+0000 7ff96effd640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:19:59.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.809+0000 7ff96effd640 1 -- 172.21.15.26:0/2371690366 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff980025680 con 0x559051aa05d0 2024-09-11T23:19:59.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.809+0000 7ff96effd640 1 -- 172.21.15.26:0/2371690366 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7ff9800053c0 con 0x559051aa05d0 2024-09-11T23:19:59.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.809+0000 7ff99848f740 1 -- 172.21.15.26:0/2371690366 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff984099600 con 0x559051aa05d0 2024-09-11T23:19:59.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.809+0000 7ff96effd640 1 -- 172.21.15.26:0/2371690366 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff980036050 con 0x559051aa05d0 2024-09-11T23:19:59.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.809+0000 7ff99848f740 10 client.15834 fetch_fsmap learned FSMap version 30 2024-09-11T23:19:59.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.809+0000 7ff99848f740 1 -- 172.21.15.26:0/2371690366 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559051b297a0 con 0x559051aa05d0 2024-09-11T23:19:59.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.809+0000 7ff96effd640 1 -- 172.21.15.26:0/2371690366 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff984099600 con 0x559051aa05d0 2024-09-11T23:19:59.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.809+0000 7ff96effd640 1 -- 172.21.15.26:0/2371690366 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7ff980004680 con 0x559051aa05d0 2024-09-11T23:19:59.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.809+0000 7ff99848f740 10 client.15834 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:19:59.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.809+0000 7ff99848f740 10 client.15834 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-09-11T23:19:59.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.809+0000 7ff99848f740 20 client.15834 populate_metadata read hostname 'smithi026' 2024-09-11T23:19:59.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.809+0000 7ff99848f740 1 --2- 172.21.15.26:0/2371690366 >> [v2:172.21.15.76:6834/393308499,v1:172.21.15.76:6835/393308499] conn(0x559051b29ef0 0x559051b2c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:59.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.809+0000 7ff99848f740 4 client.15834 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:19:59.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.809+0000 7ff99848f740 1 -- 172.21.15.26:0/2371690366 --> [v2:172.21.15.76:6834/393308499,v1:172.21.15.76:6835/393308499] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559051a0dd70 con 0x559051b29ef0 2024-09-11T23:19:59.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.810+0000 7ff98effd640 1 --2- 172.21.15.26:0/2371690366 >> [v2:172.21.15.76:6834/393308499,v1:172.21.15.76:6835/393308499] conn(0x559051b29ef0 0x559051b2c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:59.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.810+0000 7ff96effd640 10 client.15834 ms_handle_connect on v2:172.21.15.76:6834/393308499 2024-09-11T23:19:59.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.810+0000 7ff98effd640 1 --2- 172.21.15.26:0/2371690366 >> [v2:172.21.15.76:6834/393308499,v1:172.21.15.76:6835/393308499] conn(0x559051b29ef0 0x559051b2c310 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=5bdd3a4842c75749 server_cookie=6973167e5fc8f07a in_seq=0 out_seq=0 2024-09-11T23:19:59.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.810+0000 7ff96effd640 1 -- 172.21.15.26:0/2371690366 <== mds.4 v2:172.21.15.76:6834/393308499 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559051a0dd70 con 0x559051b29ef0 2024-09-11T23:19:59.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.810+0000 7ff96effd640 10 client.15834 handle_command_reply: tid=0 2024-09-11T23:19:59.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.810+0000 7ff99848f740 1 client.15834 shutdown 2024-09-11T23:19:59.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.810+0000 7ff99848f740 1 -- 172.21.15.26:0/2371690366 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559051aa05d0 msgr2=0x559051ab78b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:59.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.810+0000 7ff99848f740 1 --2- 172.21.15.26:0/2371690366 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559051aa05d0 0x559051ab78b0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7ff9800059f0 tx=0x7ff98000fc90 comp rx=0 tx=0).stop 2024-09-11T23:19:59.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.810+0000 7ff99848f740 1 -- 172.21.15.26:0/2371690366 shutdown_connections 2024-09-11T23:19:59.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.810+0000 7ff99848f740 1 --2- 172.21.15.26:0/2371690366 >> [v2:172.21.15.76:6834/393308499,v1:172.21.15.76:6835/393308499] conn(0x559051b29ef0 0x559051b2c310 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:59.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.810+0000 7ff99848f740 1 --2- 172.21.15.26:0/2371690366 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559051aa1a20 0x559051ab7df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:59.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.810+0000 7ff99848f740 1 --2- 172.21.15.26:0/2371690366 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559051aaf970 0x559051ac6060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:59.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.810+0000 7ff96effd640 0 client.15834 ms_handle_reset on v2:172.21.15.76:6834/393308499 2024-09-11T23:19:59.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.810+0000 7ff99848f740 1 --2- 172.21.15.26:0/2371690366 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559051aa05d0 0x559051ab78b0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:59.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.810+0000 7ff99848f740 1 -- 172.21.15.26:0/2371690366 >> 172.21.15.26:0/2371690366 conn(0x559051a69310 msgr2=0x559051aa9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:19:59.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.810+0000 7ff99848f740 1 -- 172.21.15.26:0/2371690366 shutdown_connections 2024-09-11T23:19:59.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.811+0000 7ff99848f740 1 -- 172.21.15.26:0/2371690366 wait complete. 2024-09-11T23:19:59.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.811+0000 7ff99848f740 20 client.15834 trim_cache size 0 max 16384 2024-09-11T23:19:59.816 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2024-09-11T23:19:59.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.811+0000 7ff9960b6640 1 -- 172.21.15.26:0/2783431837 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7ff9780a2cf0 msgr2=0x7ff9780a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:59.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.811+0000 7ff9960b6640 1 --2- 172.21.15.26:0/2783431837 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7ff9780a2cf0 0x7ff9780a51b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ff988048000 tx=0x7ff98804b000 comp rx=0 tx=0).stop 2024-09-11T23:19:59.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.811+0000 7ff9960b6640 1 -- 172.21.15.26:0/2783431837 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff990101420 msgr2=0x7ff990197870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:59.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.811+0000 7ff9960b6640 1 --2- 172.21.15.26:0/2783431837 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff990101420 0x7ff990197870 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ff984005fd0 tx=0x7ff98400dde0 comp rx=0 tx=0).stop 2024-09-11T23:19:59.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.811+0000 7ff9960b6640 1 -- 172.21.15.26:0/2783431837 shutdown_connections 2024-09-11T23:19:59.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.811+0000 7ff9960b6640 1 --2- 172.21.15.26:0/2783431837 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7ff9780a2cf0 0x7ff9780a51b0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:59.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.811+0000 7ff9960b6640 1 --2- 172.21.15.26:0/2783431837 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff990101420 0x7ff990197870 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:59.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.811+0000 7ff9960b6640 1 --2- 172.21.15.26:0/2783431837 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff990106a80 0x7ff99019bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:59.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.811+0000 7ff9960b6640 1 --2- 172.21.15.26:0/2783431837 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff990100a80 0x7ff990197330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:59.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.811+0000 7ff9960b6640 1 -- 172.21.15.26:0/2783431837 >> 172.21.15.26:0/2783431837 conn(0x7ff9900fc7a0 msgr2=0x7ff9900fceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:19:59.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.811+0000 7ff9960b6640 1 -- 172.21.15.26:0/2783431837 shutdown_connections 2024-09-11T23:19:59.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:19:59.811+0000 7ff9960b6640 1 -- 172.21.15.26:0/2783431837 wait complete. 2024-09-11T23:20:00.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:19:59 smithi136 ceph-mon[35121]: pgmap v51: 97 pgs: 97 active+clean; 14 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 124 op/s 2024-09-11T23:20:00.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:19:59 smithi026 ceph-mon[31172]: pgmap v51: 97 pgs: 97 active+clean; 14 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 124 op/s 2024-09-11T23:20:00.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:19:59 smithi076 ceph-mon[35296]: pgmap v51: 97 pgs: 97 active+clean; 14 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 124 op/s 2024-09-11T23:20:00.534 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-11T23:20:00.821 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:20:01.235 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:00 smithi026 ceph-mon[31172]: overall HEALTH_OK 2024-09-11T23:20:01.238 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:00 smithi136 ceph-mon[35121]: overall HEALTH_OK 2024-09-11T23:20:01.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:00 smithi076 ceph-mon[35296]: overall HEALTH_OK 2024-09-11T23:20:01.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.322+0000 7f0a7f6e7640 1 -- 172.21.15.26:0/3379331429 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a78102570 msgr2=0x7f0a781029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:01.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.322+0000 7f0a7f6e7640 1 --2- 172.21.15.26:0/3379331429 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a78102570 0x7f0a781029b0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f0a68009900 tx=0x7f0a6802a480 comp rx=0 tx=0).stop 2024-09-11T23:20:01.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.322+0000 7f0a7f6e7640 1 -- 172.21.15.26:0/3379331429 shutdown_connections 2024-09-11T23:20:01.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.322+0000 7f0a7f6e7640 1 --2- 172.21.15.26:0/3379331429 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0a78102ef0 0x7f0a7810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:01.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.322+0000 7f0a7f6e7640 1 --2- 172.21.15.26:0/3379331429 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a78102570 0x7f0a781029b0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:01.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.322+0000 7f0a7f6e7640 1 --2- 172.21.15.26:0/3379331429 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0a78100e20 0x7f0a78101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:01.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.322+0000 7f0a7f6e7640 1 -- 172.21.15.26:0/3379331429 >> 172.21.15.26:0/3379331429 conn(0x7f0a780fc7d0 msgr2=0x7f0a780febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:01.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.322+0000 7f0a7f6e7640 1 -- 172.21.15.26:0/3379331429 shutdown_connections 2024-09-11T23:20:01.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.322+0000 7f0a7f6e7640 1 -- 172.21.15.26:0/3379331429 wait complete. 2024-09-11T23:20:01.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.322+0000 7f0a7f6e7640 1 Processor -- start 2024-09-11T23:20:01.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.323+0000 7f0a7f6e7640 1 -- start start 2024-09-11T23:20:01.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.323+0000 7f0a7f6e7640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0a78100e20 0x7f0a781984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:01.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.323+0000 7f0a7f6e7640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0a78102570 0x7f0a78198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:01.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.323+0000 7f0a7f6e7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a78102ef0 0x7f0a7819cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:01.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.323+0000 7f0a7f6e7640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f0a78110f50 con 0x7f0a78102ef0 2024-09-11T23:20:01.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.323+0000 7f0a7f6e7640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f0a78110dd0 con 0x7f0a78100e20 2024-09-11T23:20:01.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.323+0000 7f0a7f6e7640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f0a781110d0 con 0x7f0a78102570 2024-09-11T23:20:01.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.323+0000 7f0a7cc82640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0a78102570 0x7f0a78198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:01.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.323+0000 7f0a7cc82640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0a78102570 0x7f0a78198a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:54488/0 (socket says 172.21.15.26:54488) 2024-09-11T23:20:01.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.323+0000 7f0a7cc82640 1 -- 172.21.15.26:0/2867408698 learned_addr learned my addr 172.21.15.26:0/2867408698 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:20:01.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.323+0000 7f0a7dc84640 1 --2- 172.21.15.26:0/2867408698 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a78102ef0 0x7f0a7819cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:01.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.323+0000 7f0a7d483640 1 --2- 172.21.15.26:0/2867408698 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0a78100e20 0x7f0a781984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:01.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.323+0000 7f0a7cc82640 1 -- 172.21.15.26:0/2867408698 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0a78100e20 msgr2=0x7f0a781984f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:01.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.323+0000 7f0a7cc82640 1 --2- 172.21.15.26:0/2867408698 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0a78100e20 0x7f0a781984f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:01.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.323+0000 7f0a7cc82640 1 -- 172.21.15.26:0/2867408698 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a78102ef0 msgr2=0x7f0a7819cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:01.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.323+0000 7f0a7cc82640 1 --2- 172.21.15.26:0/2867408698 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a78102ef0 0x7f0a7819cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:01.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.323+0000 7f0a7cc82640 1 -- 172.21.15.26:0/2867408698 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a7819d4a0 con 0x7f0a78102570 2024-09-11T23:20:01.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.323+0000 7f0a7dc84640 1 --2- 172.21.15.26:0/2867408698 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a78102ef0 0x7f0a7819cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:20:01.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.324+0000 7f0a7cc82640 1 --2- 172.21.15.26:0/2867408698 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0a78102570 0x7f0a78198a30 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f0a68006010 tx=0x7f0a68003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:01.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.324+0000 7f0a6e7fc640 1 -- 172.21.15.26:0/2867408698 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a6803f070 con 0x7f0a78102570 2024-09-11T23:20:01.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.324+0000 7f0a6e7fc640 1 -- 172.21.15.26:0/2867408698 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a68031c30 con 0x7f0a78102570 2024-09-11T23:20:01.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.324+0000 7f0a7f6e7640 1 -- 172.21.15.26:0/2867408698 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a7819d730 con 0x7f0a78102570 2024-09-11T23:20:01.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.324+0000 7f0a6e7fc640 1 -- 172.21.15.26:0/2867408698 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a68035030 con 0x7f0a78102570 2024-09-11T23:20:01.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.324+0000 7f0a7f6e7640 1 -- 172.21.15.26:0/2867408698 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a781a5070 con 0x7f0a78102570 2024-09-11T23:20:01.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.327+0000 7f0a81ac0740 1 Processor -- start 2024-09-11T23:20:01.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.327+0000 7f0a81ac0740 1 -- start start 2024-09-11T23:20:01.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.327+0000 7f0a6e7fc640 1 -- 172.21.15.26:0/2867408698 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f0a68042070 con 0x7f0a78102570 2024-09-11T23:20:01.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.327+0000 7f0a81ac0740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a299e23b40 0x55a299e22090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:01.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.327+0000 7f0a81ac0740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a299e225d0 0x55a299e22a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:01.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.327+0000 7f0a81ac0740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a299e31780 0x55a299e33b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:01.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.327+0000 7f0a81ac0740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55a299d882f0 con 0x55a299e31780 2024-09-11T23:20:01.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.327+0000 7f0a81ac0740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55a299d2fcf0 con 0x55a299e225d0 2024-09-11T23:20:01.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.327+0000 7f0a81ac0740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55a299d842b0 con 0x55a299e23b40 2024-09-11T23:20:01.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.327+0000 7f0a7d483640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a299e225d0 0x55a299e22a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:01.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.327+0000 7f0a7dc84640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a299e23b40 0x55a299e22090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:01.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.327+0000 7f0a7cc82640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a299e31780 0x55a299e33b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:01.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.327+0000 7f0a7dc84640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a299e23b40 0x55a299e22090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:54500/0 (socket says 172.21.15.26:54500) 2024-09-11T23:20:01.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.328+0000 7f0a7dc84640 1 -- 172.21.15.26:0/272326828 learned_addr learned my addr 172.21.15.26:0/272326828 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:20:01.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.327+0000 7f0a7d483640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a299e225d0 0x55a299e22a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:60560/0 (socket says 172.21.15.26:60560) 2024-09-11T23:20:01.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.328+0000 7f0a7d483640 1 -- 172.21.15.26:0/272326828 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a299e23b40 msgr2=0x55a299e22090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:20:01.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.328+0000 7f0a7d483640 1 --2- 172.21.15.26:0/272326828 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a299e23b40 0x55a299e22090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:01.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.328+0000 7f0a7d483640 1 -- 172.21.15.26:0/272326828 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a299e31780 msgr2=0x55a299e33b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:20:01.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.328+0000 7f0a7d483640 1 --2- 172.21.15.26:0/272326828 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a299e31780 0x55a299e33b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:01.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.328+0000 7f0a7d483640 1 -- 172.21.15.26:0/272326828 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a299d143a0 con 0x55a299e225d0 2024-09-11T23:20:01.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.328+0000 7f0a7d483640 1 --2- 172.21.15.26:0/272326828 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a299e225d0 0x55a299e22a10 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f0a74009c20 tx=0x7f0a74011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8988a0f12bdc3148 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:01.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.328+0000 7f0a637fe640 1 -- 172.21.15.26:0/272326828 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a74004050 con 0x55a299e225d0 2024-09-11T23:20:01.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.328+0000 7f0a6e7fc640 1 --2- 172.21.15.26:0/2867408698 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f0a640a2e40 0x7f0a640a5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:01.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.328+0000 7f0a637fe640 1 -- 172.21.15.26:0/272326828 <== mon.2 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a74005500 con 0x55a299e225d0 2024-09-11T23:20:01.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.328+0000 7f0a637fe640 1 -- 172.21.15.26:0/272326828 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a74004dd0 con 0x55a299e225d0 2024-09-11T23:20:01.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.328+0000 7f0a7dc84640 1 --2- 172.21.15.26:0/2867408698 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f0a640a2e40 0x7f0a640a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:01.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.329+0000 7f0a6e7fc640 1 -- 172.21.15.26:0/2867408698 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f0a680ef820 con 0x7f0a78102570 2024-09-11T23:20:01.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.329+0000 7f0a7dc84640 1 --2- 172.21.15.26:0/2867408698 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f0a640a2e40 0x7f0a640a5300 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f0a70009fd0 tx=0x7f0a70009500 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:01.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.329+0000 7f0a81ac0740 1 -- 172.21.15.26:0/272326828 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a299e225d0 msgr2=0x55a299e22a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:01.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.329+0000 7f0a81ac0740 1 --2- 172.21.15.26:0/272326828 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a299e225d0 0x55a299e22a10 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f0a74009c20 tx=0x7f0a74011810 comp rx=0 tx=0).stop 2024-09-11T23:20:01.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.329+0000 7f0a81ac0740 1 -- 172.21.15.26:0/272326828 shutdown_connections 2024-09-11T23:20:01.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.329+0000 7f0a81ac0740 1 --2- 172.21.15.26:0/272326828 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a299e225d0 0x55a299e22a10 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:01.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.329+0000 7f0a81ac0740 1 --2- 172.21.15.26:0/272326828 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a299e31780 0x55a299e33b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:01.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.329+0000 7f0a81ac0740 1 --2- 172.21.15.26:0/272326828 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a299e23b40 0x55a299e22090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:01.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.329+0000 7f0a81ac0740 1 -- 172.21.15.26:0/272326828 >> 172.21.15.26:0/272326828 conn(0x55a299deb310 msgr2=0x55a299e1ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:01.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.329+0000 7f0a81ac0740 1 -- 172.21.15.26:0/272326828 shutdown_connections 2024-09-11T23:20:01.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.329+0000 7f0a81ac0740 1 -- 172.21.15.26:0/272326828 wait complete. 2024-09-11T23:20:01.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.329+0000 7f0a81ac0740 1 -- start start 2024-09-11T23:20:01.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.330+0000 7f0a81ac0740 1 Processor -- start 2024-09-11T23:20:01.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.330+0000 7f0a81ac0740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a299e225d0 0x55a299e310a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:01.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.330+0000 7f0a81ac0740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a299e23b40 0x55a299e2e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:01.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.330+0000 7f0a81ac0740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a299e31780 0x55a299e2e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:01.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.330+0000 7f0a81ac0740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55a299e34480 con 0x55a299e31780 2024-09-11T23:20:01.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.330+0000 7f0a81ac0740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55a299d43060 con 0x55a299e225d0 2024-09-11T23:20:01.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.330+0000 7f0a81ac0740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55a299e34600 con 0x55a299e23b40 2024-09-11T23:20:01.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.330+0000 7f0a81ac0740 10 client.0 _ll_register_callbacks cb 0x55a299a968c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:20:01.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.330+0000 7f0a7d483640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a299e225d0 0x55a299e310a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:01.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.330+0000 7f0a7dc84640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a299e31780 0x55a299e2e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:01.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.330+0000 7f0a7d483640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a299e225d0 0x55a299e310a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:60568/0 (socket says 172.21.15.26:60568) 2024-09-11T23:20:01.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.330+0000 7f0a7d483640 1 -- 172.21.15.26:0/1745568118 learned_addr learned my addr 172.21.15.26:0/1745568118 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:20:01.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.330+0000 7f0a7dc84640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a299e31780 0x55a299e2e890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48248/0 (socket says 172.21.15.26:48248) 2024-09-11T23:20:01.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.330+0000 7f0a7cc82640 1 --2- 172.21.15.26:0/1745568118 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a299e23b40 0x55a299e2e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:01.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.330+0000 7f0a7d483640 1 -- 172.21.15.26:0/1745568118 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a299e23b40 msgr2=0x55a299e2e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:20:01.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.330+0000 7f0a7d483640 1 --2- 172.21.15.26:0/1745568118 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a299e23b40 0x55a299e2e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:01.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.330+0000 7f0a7d483640 1 -- 172.21.15.26:0/1745568118 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a299e31780 msgr2=0x55a299e2e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:20:01.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.331+0000 7f0a7d483640 1 --2- 172.21.15.26:0/1745568118 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a299e31780 0x55a299e2e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:01.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.331+0000 7f0a7d483640 1 -- 172.21.15.26:0/1745568118 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a299af93c0 con 0x55a299e225d0 2024-09-11T23:20:01.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.331+0000 7f0a7d483640 1 --2- 172.21.15.26:0/1745568118 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a299e225d0 0x55a299e310a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f0a7401b040 tx=0x7f0a74005710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2da634b3bf03c882 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:01.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.331+0000 7f0a60ff9640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-11T23:20:01.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.331+0000 7f0a60ff9640 1 -- 172.21.15.26:0/1745568118 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a74029070 con 0x55a299e225d0 2024-09-11T23:20:01.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.331+0000 7f0a60ff9640 1 -- 172.21.15.26:0/1745568118 <== mon.2 v2:172.21.15.76:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f0a74004d60 con 0x55a299e225d0 2024-09-11T23:20:01.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.331+0000 7f0a81ac0740 1 -- 172.21.15.26:0/1745568118 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0a74005500 con 0x55a299e225d0 2024-09-11T23:20:01.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.331+0000 7f0a60ff9640 1 -- 172.21.15.26:0/1745568118 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f0a74035050 con 0x55a299e225d0 2024-09-11T23:20:01.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.331+0000 7f0a81ac0740 10 client.25436 fetch_fsmap learned FSMap version 30 2024-09-11T23:20:01.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.331+0000 7f0a81ac0740 1 -- 172.21.15.26:0/1745568118 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a299e3fe40 con 0x55a299e225d0 2024-09-11T23:20:01.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.331+0000 7f0a60ff9640 1 -- 172.21.15.26:0/1745568118 <== mon.2 v2:172.21.15.76:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a74005500 con 0x55a299e225d0 2024-09-11T23:20:01.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.332+0000 7f0a60ff9640 1 -- 172.21.15.26:0/1745568118 <== mon.2 v2:172.21.15.76:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7f0a74003d30 con 0x55a299e225d0 2024-09-11T23:20:01.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.332+0000 7f0a81ac0740 10 client.25436 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:20:01.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.332+0000 7f0a81ac0740 10 client.25436 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-11T23:20:01.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.332+0000 7f0a81ac0740 20 client.25436 populate_metadata read hostname 'smithi026' 2024-09-11T23:20:01.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.332+0000 7f0a81ac0740 1 --2- 172.21.15.26:0/1745568118 >> [v2:172.21.15.136:6836/1633082387,v1:172.21.15.136:6837/1633082387] conn(0x55a299e41190 0x55a299e43580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:01.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.332+0000 7f0a81ac0740 4 client.25436 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:20:01.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.332+0000 7f0a81ac0740 1 -- 172.21.15.26:0/1745568118 --> [v2:172.21.15.136:6836/1633082387,v1:172.21.15.136:6837/1633082387] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a299d95860 con 0x55a299e41190 2024-09-11T23:20:01.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.332+0000 7f0a7cc82640 1 --2- 172.21.15.26:0/1745568118 >> [v2:172.21.15.136:6836/1633082387,v1:172.21.15.136:6837/1633082387] conn(0x55a299e41190 0x55a299e43580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:01.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.333+0000 7f0a60ff9640 10 client.25436 ms_handle_connect on v2:172.21.15.136:6836/1633082387 2024-09-11T23:20:01.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.333+0000 7f0a7cc82640 1 --2- 172.21.15.26:0/1745568118 >> [v2:172.21.15.136:6836/1633082387,v1:172.21.15.136:6837/1633082387] conn(0x55a299e41190 0x55a299e43580 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=cfb7b5ea0d34b932 server_cookie=26bdd6f3d7df67ec in_seq=0 out_seq=0 2024-09-11T23:20:01.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.334+0000 7f0a60ff9640 1 -- 172.21.15.26:0/1745568118 <== mds.3 v2:172.21.15.136:6836/1633082387 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a299d95860 con 0x55a299e41190 2024-09-11T23:20:01.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.334+0000 7f0a60ff9640 10 client.25436 handle_command_reply: tid=0 2024-09-11T23:20:01.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.334+0000 7f0a81ac0740 1 client.25436 shutdown 2024-09-11T23:20:01.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.334+0000 7f0a81ac0740 1 -- 172.21.15.26:0/1745568118 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a299e225d0 msgr2=0x55a299e310a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:01.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.334+0000 7f0a81ac0740 1 --2- 172.21.15.26:0/1745568118 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a299e225d0 0x55a299e310a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f0a7401b040 tx=0x7f0a74005710 comp rx=0 tx=0).stop 2024-09-11T23:20:01.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.334+0000 7f0a81ac0740 1 -- 172.21.15.26:0/1745568118 shutdown_connections 2024-09-11T23:20:01.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.334+0000 7f0a81ac0740 1 --2- 172.21.15.26:0/1745568118 >> [v2:172.21.15.136:6836/1633082387,v1:172.21.15.136:6837/1633082387] conn(0x55a299e41190 0x55a299e43580 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:01.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.334+0000 7f0a81ac0740 1 --2- 172.21.15.26:0/1745568118 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a299e31780 0x55a299e2e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:01.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.334+0000 7f0a81ac0740 1 --2- 172.21.15.26:0/1745568118 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a299e23b40 0x55a299e2e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:01.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.334+0000 7f0a60ff9640 0 client.25436 ms_handle_reset on v2:172.21.15.136:6836/1633082387 2024-09-11T23:20:01.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.334+0000 7f0a81ac0740 1 --2- 172.21.15.26:0/1745568118 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a299e225d0 0x55a299e310a0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:01.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.334+0000 7f0a81ac0740 1 -- 172.21.15.26:0/1745568118 >> 172.21.15.26:0/1745568118 conn(0x55a299deb310 msgr2=0x55a299e24960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:01.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.335+0000 7f0a81ac0740 1 -- 172.21.15.26:0/1745568118 shutdown_connections 2024-09-11T23:20:01.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.335+0000 7f0a81ac0740 1 -- 172.21.15.26:0/1745568118 wait complete. 2024-09-11T23:20:01.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.336+0000 7f0a81ac0740 20 client.25436 trim_cache size 0 max 16384 2024-09-11T23:20:01.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.364+0000 7f0a81ac0740 1 Processor -- start 2024-09-11T23:20:01.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.364+0000 7f0a81ac0740 1 -- start start 2024-09-11T23:20:01.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.364+0000 7f0a81ac0740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a299e23b40 0x55a299e3d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:01.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.364+0000 7f0a81ac0740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a299e31780 0x55a299e32b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:01.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.364+0000 7f0a81ac0740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a299e2e980 0x55a299e330b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:01.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.364+0000 7f0a81ac0740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f0a781ab680 con 0x55a299e2e980 2024-09-11T23:20:01.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.364+0000 7f0a81ac0740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f0a781ad1f0 con 0x55a299e23b40 2024-09-11T23:20:01.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.364+0000 7f0a81ac0740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f0a781abd50 con 0x55a299e31780 2024-09-11T23:20:01.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.365+0000 7f0a7dc84640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a299e2e980 0x55a299e330b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:01.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.365+0000 7f0a7dc84640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a299e2e980 0x55a299e330b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48254/0 (socket says 172.21.15.26:48254) 2024-09-11T23:20:01.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.365+0000 7f0a7dc84640 1 -- 172.21.15.26:0/3658361096 learned_addr learned my addr 172.21.15.26:0/3658361096 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:20:01.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.365+0000 7f0a7cc82640 1 --2- 172.21.15.26:0/3658361096 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a299e31780 0x55a299e32b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:01.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.365+0000 7f0a7dc84640 1 -- 172.21.15.26:0/3658361096 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a299e31780 msgr2=0x55a299e32b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:20:01.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.365+0000 7f0a7dc84640 1 --2- 172.21.15.26:0/3658361096 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a299e31780 0x55a299e32b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:01.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.365+0000 7f0a7dc84640 1 -- 172.21.15.26:0/3658361096 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a299e23b40 msgr2=0x55a299e3d720 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T23:20:01.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.365+0000 7f0a7dc84640 1 --2- 172.21.15.26:0/3658361096 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a299e23b40 0x55a299e3d720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:01.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.365+0000 7f0a7dc84640 1 -- 172.21.15.26:0/3658361096 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a299e335f0 con 0x55a299e2e980 2024-09-11T23:20:01.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.366+0000 7f0a7dc84640 1 --2- 172.21.15.26:0/3658361096 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a299e2e980 0x55a299e330b0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f0a70011a00 tx=0x7f0a70016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=758e00c625734c0f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:01.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.368+0000 7f0a62ffd640 1 -- 172.21.15.26:0/3658361096 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a7001c610 con 0x55a299e2e980 2024-09-11T23:20:01.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.368+0000 7f0a62ffd640 1 -- 172.21.15.26:0/3658361096 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a7002d070 con 0x55a299e2e980 2024-09-11T23:20:01.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.368+0000 7f0a62ffd640 1 -- 172.21.15.26:0/3658361096 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a70028d60 con 0x55a299e2e980 2024-09-11T23:20:01.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.372+0000 7f0a81ac0740 1 -- 172.21.15.26:0/3658361096 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a299e2e980 msgr2=0x55a299e330b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:01.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.372+0000 7f0a81ac0740 1 --2- 172.21.15.26:0/3658361096 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a299e2e980 0x55a299e330b0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f0a70011a00 tx=0x7f0a70016000 comp rx=0 tx=0).stop 2024-09-11T23:20:01.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.373+0000 7f0a81ac0740 1 -- 172.21.15.26:0/3658361096 shutdown_connections 2024-09-11T23:20:01.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.373+0000 7f0a81ac0740 1 --2- 172.21.15.26:0/3658361096 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a299e2e980 0x55a299e330b0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:01.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.373+0000 7f0a81ac0740 1 --2- 172.21.15.26:0/3658361096 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a299e31780 0x55a299e32b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:01.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.373+0000 7f0a81ac0740 1 --2- 172.21.15.26:0/3658361096 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a299e23b40 0x55a299e3d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:01.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.373+0000 7f0a81ac0740 1 -- 172.21.15.26:0/3658361096 >> 172.21.15.26:0/3658361096 conn(0x55a299deb310 msgr2=0x55a299e2d440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:01.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.376+0000 7f0a81ac0740 1 -- 172.21.15.26:0/3658361096 shutdown_connections 2024-09-11T23:20:01.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.376+0000 7f0a81ac0740 1 -- 172.21.15.26:0/3658361096 wait complete. 2024-09-11T23:20:01.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.377+0000 7f0a81ac0740 1 -- start start 2024-09-11T23:20:01.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.377+0000 7f0a81ac0740 1 Processor -- start 2024-09-11T23:20:01.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.378+0000 7f0a81ac0740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a299e23b40 0x55a299e24b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:01.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.378+0000 7f0a81ac0740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a299e2e980 0x55a299e25040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:01.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.378+0000 7f0a81ac0740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a299e31780 0x55a299e25580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:01.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.378+0000 7f0a7d483640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a299e23b40 0x55a299e24b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:01.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.378+0000 7f0a7d483640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a299e23b40 0x55a299e24b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:54520/0 (socket says 172.21.15.26:54520) 2024-09-11T23:20:01.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.378+0000 7f0a7cc82640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a299e2e980 0x55a299e25040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:01.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.378+0000 7f0a7d483640 1 -- 172.21.15.26:0/2919756628 learned_addr learned my addr 172.21.15.26:0/2919756628 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:20:01.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.378+0000 7f0a81ac0740 1 -- 172.21.15.26:0/2919756628 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f0a781acaf0 con 0x55a299e31780 2024-09-11T23:20:01.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.378+0000 7f0a81ac0740 1 -- 172.21.15.26:0/2919756628 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f0a781ac420 con 0x55a299e2e980 2024-09-11T23:20:01.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.378+0000 7f0a81ac0740 1 -- 172.21.15.26:0/2919756628 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f0a781a9660 con 0x55a299e23b40 2024-09-11T23:20:01.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.378+0000 7f0a7dc84640 1 --2- 172.21.15.26:0/2919756628 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a299e31780 0x55a299e25580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:01.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.378+0000 7f0a81ac0740 10 client.0 _ll_register_callbacks cb 0x7f0a680f5420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:20:01.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.379+0000 7f0a7cc82640 1 -- 172.21.15.26:0/2919756628 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a299e23b40 msgr2=0x55a299e24b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:20:01.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.379+0000 7f0a7cc82640 1 --2- 172.21.15.26:0/2919756628 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a299e23b40 0x55a299e24b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:01.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.379+0000 7f0a7cc82640 1 -- 172.21.15.26:0/2919756628 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a299e31780 msgr2=0x55a299e25580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:20:01.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.379+0000 7f0a7cc82640 1 --2- 172.21.15.26:0/2919756628 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a299e31780 0x55a299e25580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:01.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.379+0000 7f0a7cc82640 1 -- 172.21.15.26:0/2919756628 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a299e39b20 con 0x55a299e2e980 2024-09-11T23:20:01.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.379+0000 7f0a7cc82640 1 --2- 172.21.15.26:0/2919756628 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a299e2e980 0x55a299e25040 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f0a680ecf80 tx=0x7f0a6802a480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d88f82a009589ad server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:01.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.379+0000 7f0a60ff9640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-11T23:20:01.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.379+0000 7f0a60ff9640 1 -- 172.21.15.26:0/2919756628 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a68031de0 con 0x55a299e2e980 2024-09-11T23:20:01.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.379+0000 7f0a60ff9640 1 -- 172.21.15.26:0/2919756628 <== mon.2 v2:172.21.15.76:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f0a680b8470 con 0x55a299e2e980 2024-09-11T23:20:01.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.380+0000 7f0a81ac0740 1 -- 172.21.15.26:0/2919756628 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0a7002d070 con 0x55a299e2e980 2024-09-11T23:20:01.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.381+0000 7f0a60ff9640 1 -- 172.21.15.26:0/2919756628 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f0a680bbe00 con 0x55a299e2e980 2024-09-11T23:20:01.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.381+0000 7f0a81ac0740 10 client.25439 fetch_fsmap learned FSMap version 30 2024-09-11T23:20:01.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.381+0000 7f0a81ac0740 1 -- 172.21.15.26:0/2919756628 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a299ea82f0 con 0x55a299e2e980 2024-09-11T23:20:01.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.382+0000 7f0a60ff9640 1 -- 172.21.15.26:0/2919756628 <== mon.2 v2:172.21.15.76:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a7002d070 con 0x55a299e2e980 2024-09-11T23:20:01.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.382+0000 7f0a60ff9640 1 -- 172.21.15.26:0/2919756628 <== mon.2 v2:172.21.15.76:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7f0a680c4530 con 0x55a299e2e980 2024-09-11T23:20:01.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.382+0000 7f0a81ac0740 10 client.25439 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:20:01.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.382+0000 7f0a81ac0740 10 client.25439 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-11T23:20:01.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.382+0000 7f0a81ac0740 20 client.25439 populate_metadata read hostname 'smithi026' 2024-09-11T23:20:01.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.382+0000 7f0a81ac0740 1 --2- 172.21.15.26:0/2919756628 >> [v2:172.21.15.136:6836/1633082387,v1:172.21.15.136:6837/1633082387] conn(0x55a299ea8b10 0x55a299eaaf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:01.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.382+0000 7f0a81ac0740 4 client.25439 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:20:01.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.382+0000 7f0a81ac0740 1 -- 172.21.15.26:0/2919756628 --> [v2:172.21.15.136:6836/1633082387,v1:172.21.15.136:6837/1633082387] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a299d8fd70 con 0x55a299ea8b10 2024-09-11T23:20:01.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.382+0000 7f0a7d483640 1 --2- 172.21.15.26:0/2919756628 >> [v2:172.21.15.136:6836/1633082387,v1:172.21.15.136:6837/1633082387] conn(0x55a299ea8b10 0x55a299eaaf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:01.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.383+0000 7f0a60ff9640 10 client.25439 ms_handle_connect on v2:172.21.15.136:6836/1633082387 2024-09-11T23:20:01.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.383+0000 7f0a7d483640 1 --2- 172.21.15.26:0/2919756628 >> [v2:172.21.15.136:6836/1633082387,v1:172.21.15.136:6837/1633082387] conn(0x55a299ea8b10 0x55a299eaaf30 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=1ef2523a722015ce server_cookie=cc80560bc9242414 in_seq=0 out_seq=0 2024-09-11T23:20:01.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.383+0000 7f0a60ff9640 1 -- 172.21.15.26:0/2919756628 <== mds.3 v2:172.21.15.136:6836/1633082387 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a299d8fd70 con 0x55a299ea8b10 2024-09-11T23:20:01.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.383+0000 7f0a60ff9640 10 client.25439 handle_command_reply: tid=0 2024-09-11T23:20:01.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.383+0000 7f0a81ac0740 1 client.25439 shutdown 2024-09-11T23:20:01.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.384+0000 7f0a81ac0740 1 -- 172.21.15.26:0/2919756628 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a299e2e980 msgr2=0x55a299e25040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:01.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.384+0000 7f0a81ac0740 1 --2- 172.21.15.26:0/2919756628 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a299e2e980 0x55a299e25040 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f0a680ecf80 tx=0x7f0a6802a480 comp rx=0 tx=0).stop 2024-09-11T23:20:01.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.384+0000 7f0a81ac0740 1 -- 172.21.15.26:0/2919756628 shutdown_connections 2024-09-11T23:20:01.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.384+0000 7f0a81ac0740 1 --2- 172.21.15.26:0/2919756628 >> [v2:172.21.15.136:6836/1633082387,v1:172.21.15.136:6837/1633082387] conn(0x55a299ea8b10 0x55a299eaaf30 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:01.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.384+0000 7f0a81ac0740 1 --2- 172.21.15.26:0/2919756628 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a299e2e980 0x55a299e25040 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:01.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.384+0000 7f0a60ff9640 0 client.25439 ms_handle_reset on v2:172.21.15.136:6836/1633082387 2024-09-11T23:20:01.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.384+0000 7f0a81ac0740 1 --2- 172.21.15.26:0/2919756628 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a299e31780 0x55a299e25580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:01.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.384+0000 7f0a81ac0740 1 --2- 172.21.15.26:0/2919756628 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a299e23b40 0x55a299e24b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:01.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.384+0000 7f0a81ac0740 1 -- 172.21.15.26:0/2919756628 >> 172.21.15.26:0/2919756628 conn(0x55a299deb310 msgr2=0x55a299e2fbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:01.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.385+0000 7f0a81ac0740 1 -- 172.21.15.26:0/2919756628 shutdown_connections 2024-09-11T23:20:01.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.385+0000 7f0a81ac0740 1 -- 172.21.15.26:0/2919756628 wait complete. 2024-09-11T23:20:01.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.387+0000 7f0a81ac0740 20 client.25439 trim_cache size 0 max 16384 2024-09-11T23:20:01.389 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2024-09-11T23:20:01.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.389+0000 7f0a7f6e7640 1 -- 172.21.15.26:0/2867408698 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f0a640a2e40 msgr2=0x7f0a640a5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:01.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.389+0000 7f0a7f6e7640 1 --2- 172.21.15.26:0/2867408698 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f0a640a2e40 0x7f0a640a5300 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f0a70009fd0 tx=0x7f0a70009500 comp rx=0 tx=0).stop 2024-09-11T23:20:01.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.389+0000 7f0a7f6e7640 1 -- 172.21.15.26:0/2867408698 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0a78102570 msgr2=0x7f0a78198a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:01.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.389+0000 7f0a7f6e7640 1 --2- 172.21.15.26:0/2867408698 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0a78102570 0x7f0a78198a30 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f0a68006010 tx=0x7f0a68003c10 comp rx=0 tx=0).stop 2024-09-11T23:20:01.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.389+0000 7f0a7f6e7640 1 -- 172.21.15.26:0/2867408698 shutdown_connections 2024-09-11T23:20:01.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.389+0000 7f0a7f6e7640 1 --2- 172.21.15.26:0/2867408698 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f0a640a2e40 0x7f0a640a5300 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:01.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.389+0000 7f0a7f6e7640 1 --2- 172.21.15.26:0/2867408698 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a78102ef0 0x7f0a7819cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:01.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.390+0000 7f0a7f6e7640 1 --2- 172.21.15.26:0/2867408698 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0a78102570 0x7f0a78198a30 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:01.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.390+0000 7f0a7f6e7640 1 --2- 172.21.15.26:0/2867408698 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0a78100e20 0x7f0a781984f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:01.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.390+0000 7f0a7f6e7640 1 -- 172.21.15.26:0/2867408698 >> 172.21.15.26:0/2867408698 conn(0x7f0a780fc7d0 msgr2=0x7f0a7806a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:01.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.391+0000 7f0a7f6e7640 1 -- 172.21.15.26:0/2867408698 shutdown_connections 2024-09-11T23:20:01.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:01.391+0000 7f0a7f6e7640 1 -- 172.21.15.26:0/2867408698 wait complete. 2024-09-11T23:20:02.216 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:01 smithi026 ceph-mon[31172]: pgmap v52: 97 pgs: 97 active+clean; 14 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 115 op/s 2024-09-11T23:20:02.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:01 smithi136 ceph-mon[35121]: pgmap v52: 97 pgs: 97 active+clean; 14 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 115 op/s 2024-09-11T23:20:02.244 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-11T23:20:02.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:01 smithi076 ceph-mon[35296]: pgmap v52: 97 pgs: 97 active+clean; 14 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 115 op/s 2024-09-11T23:20:02.580 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:20:03.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.059+0000 7fab6012a640 1 -- 172.21.15.26:0/3268111660 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fab58100a80 msgr2=0x7fab58100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:03.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.059+0000 7fab6012a640 1 --2- 172.21.15.26:0/3268111660 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fab58100a80 0x7fab58100ee0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fab48009930 tx=0x7fab4802a420 comp rx=0 tx=0).stop 2024-09-11T23:20:03.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.059+0000 7fab6012a640 1 -- 172.21.15.26:0/3268111660 shutdown_connections 2024-09-11T23:20:03.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.059+0000 7fab6012a640 1 --2- 172.21.15.26:0/3268111660 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fab58101420 0x7fab5810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:03.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.059+0000 7fab6012a640 1 --2- 172.21.15.26:0/3268111660 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fab58100a80 0x7fab58100ee0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:03.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.059+0000 7fab6012a640 1 --2- 172.21.15.26:0/3268111660 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fab58106a80 0x7fab58106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:03.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.059+0000 7fab6012a640 1 -- 172.21.15.26:0/3268111660 >> 172.21.15.26:0/3268111660 conn(0x7fab580fc7a0 msgr2=0x7fab580febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:03.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.059+0000 7fab6012a640 1 -- 172.21.15.26:0/3268111660 shutdown_connections 2024-09-11T23:20:03.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.060+0000 7fab6012a640 1 -- 172.21.15.26:0/3268111660 wait complete. 2024-09-11T23:20:03.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.060+0000 7fab6012a640 1 Processor -- start 2024-09-11T23:20:03.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.060+0000 7fab6012a640 1 -- start start 2024-09-11T23:20:03.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.060+0000 7fab6012a640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fab58100a80 0x7fab5819a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:03.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.060+0000 7fab6012a640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fab58101420 0x7fab5819abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:03.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.060+0000 7fab6012a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fab58106a80 0x7fab5819ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:03.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.061+0000 7fab6012a640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fab58110e80 con 0x7fab58106a80 2024-09-11T23:20:03.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.061+0000 7fab6012a640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fab58110d00 con 0x7fab58100a80 2024-09-11T23:20:03.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.061+0000 7fab6012a640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fab58111000 con 0x7fab58101420 2024-09-11T23:20:03.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.061+0000 7fab5dec6640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fab58100a80 0x7fab5819a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:03.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.061+0000 7fab5dec6640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fab58100a80 0x7fab5819a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:60598/0 (socket says 172.21.15.26:60598) 2024-09-11T23:20:03.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.061+0000 7fab5dec6640 1 -- 172.21.15.26:0/3295115293 learned_addr learned my addr 172.21.15.26:0/3295115293 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:20:03.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.061+0000 7fab5e6c7640 1 --2- 172.21.15.26:0/3295115293 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fab58106a80 0x7fab5819ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:03.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.061+0000 7fab5d6c5640 1 --2- 172.21.15.26:0/3295115293 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fab58101420 0x7fab5819abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:03.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.061+0000 7fab5dec6640 1 -- 172.21.15.26:0/3295115293 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fab58101420 msgr2=0x7fab5819abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:03.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.061+0000 7fab5dec6640 1 --2- 172.21.15.26:0/3295115293 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fab58101420 0x7fab5819abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:03.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.061+0000 7fab5dec6640 1 -- 172.21.15.26:0/3295115293 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fab58106a80 msgr2=0x7fab5819ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:03.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.061+0000 7fab5dec6640 1 --2- 172.21.15.26:0/3295115293 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fab58106a80 0x7fab5819ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:03.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.061+0000 7fab5dec6640 1 -- 172.21.15.26:0/3295115293 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab5819f630 con 0x7fab58100a80 2024-09-11T23:20:03.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.061+0000 7fab5e6c7640 1 --2- 172.21.15.26:0/3295115293 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fab58106a80 0x7fab5819ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:20:03.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.061+0000 7fab5dec6640 1 --2- 172.21.15.26:0/3295115293 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fab58100a80 0x7fab5819a680 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fab5400ebc0 tx=0x7fab5400c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:03.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.061+0000 7fab4effd640 1 -- 172.21.15.26:0/3295115293 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab54010630 con 0x7fab58100a80 2024-09-11T23:20:03.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.061+0000 7fab6012a640 1 -- 172.21.15.26:0/3295115293 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab5819f920 con 0x7fab58100a80 2024-09-11T23:20:03.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.061+0000 7fab4effd640 1 -- 172.21.15.26:0/3295115293 <== mon.2 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab54010cb0 con 0x7fab58100a80 2024-09-11T23:20:03.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.061+0000 7fab4effd640 1 -- 172.21.15.26:0/3295115293 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab54018bd0 con 0x7fab58100a80 2024-09-11T23:20:03.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.062+0000 7fab6012a640 1 -- 172.21.15.26:0/3295115293 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab581a7200 con 0x7fab58100a80 2024-09-11T23:20:03.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.064+0000 7fab62503740 1 Processor -- start 2024-09-11T23:20:03.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.064+0000 7fab4effd640 1 -- 172.21.15.26:0/3295115293 <== mon.2 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fab540107d0 con 0x7fab58100a80 2024-09-11T23:20:03.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.064+0000 7fab62503740 1 -- start start 2024-09-11T23:20:03.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.065+0000 7fab62503740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556993cb8b40 0x556993cb7090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:03.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.065+0000 7fab62503740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556993cb75d0 0x556993cb7a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:03.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.065+0000 7fab62503740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556993cc2370 0x556993cc4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:03.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.065+0000 7fab62503740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x556993c1d2f0 con 0x556993cb8b40 2024-09-11T23:20:03.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.065+0000 7fab62503740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x556993bc4cf0 con 0x556993cc2370 2024-09-11T23:20:03.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.065+0000 7fab62503740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x556993c192b0 con 0x556993cb75d0 2024-09-11T23:20:03.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.065+0000 7fab5dec6640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556993cb75d0 0x556993cb7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:03.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.065+0000 7fab5dec6640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556993cb75d0 0x556993cb7a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:54540/0 (socket says 172.21.15.26:54540) 2024-09-11T23:20:03.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.065+0000 7fab5e6c7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556993cb8b40 0x556993cb7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:03.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.065+0000 7fab5dec6640 1 -- 172.21.15.26:0/2890241719 learned_addr learned my addr 172.21.15.26:0/2890241719 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:20:03.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.065+0000 7fab5d6c5640 1 --2- 172.21.15.26:0/2890241719 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556993cc2370 0x556993cc4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:03.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.065+0000 7fab5dec6640 1 -- 172.21.15.26:0/2890241719 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556993cc2370 msgr2=0x556993cc4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:20:03.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.065+0000 7fab5dec6640 1 --2- 172.21.15.26:0/2890241719 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556993cc2370 0x556993cc4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:03.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.065+0000 7fab5dec6640 1 -- 172.21.15.26:0/2890241719 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556993cb8b40 msgr2=0x556993cb7090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:20:03.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.065+0000 7fab5dec6640 1 --2- 172.21.15.26:0/2890241719 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556993cb8b40 0x556993cb7090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:03.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.065+0000 7fab5dec6640 1 -- 172.21.15.26:0/2890241719 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556993ba93a0 con 0x556993cb75d0 2024-09-11T23:20:03.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.066+0000 7fab5dec6640 1 --2- 172.21.15.26:0/2890241719 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556993cb75d0 0x556993cb7a10 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fab540cd040 tx=0x7fab540d7650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3124cd5f09bbb02b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:03.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.066+0000 7fab4effd640 1 --2- 172.21.15.26:0/3295115293 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fab440a2cf0 0x7fab440a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:03.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.066+0000 7fab43fff640 1 -- 172.21.15.26:0/2890241719 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab540de070 con 0x556993cb75d0 2024-09-11T23:20:03.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.066+0000 7fab5e6c7640 1 --2- 172.21.15.26:0/3295115293 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fab440a2cf0 0x7fab440a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:03.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.066+0000 7fab43fff640 1 -- 172.21.15.26:0/2890241719 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab5409bf80 con 0x556993cb75d0 2024-09-11T23:20:03.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.066+0000 7fab43fff640 1 -- 172.21.15.26:0/2890241719 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab5409bd50 con 0x556993cb75d0 2024-09-11T23:20:03.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.066+0000 7fab4effd640 1 -- 172.21.15.26:0/3295115293 <== mon.2 v2:172.21.15.76:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fab540cfa00 con 0x7fab58100a80 2024-09-11T23:20:03.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.067+0000 7fab62503740 1 -- 172.21.15.26:0/2890241719 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556993cb75d0 msgr2=0x556993cb7a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:03.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.067+0000 7fab5e6c7640 1 --2- 172.21.15.26:0/3295115293 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fab440a2cf0 0x7fab440a51b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fab50012640 tx=0x7fab50005f90 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:03.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.067+0000 7fab62503740 1 --2- 172.21.15.26:0/2890241719 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556993cb75d0 0x556993cb7a10 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fab540cd040 tx=0x7fab540d7650 comp rx=0 tx=0).stop 2024-09-11T23:20:03.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.067+0000 7fab62503740 1 -- 172.21.15.26:0/2890241719 shutdown_connections 2024-09-11T23:20:03.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.067+0000 7fab62503740 1 --2- 172.21.15.26:0/2890241719 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556993cc2370 0x556993cc4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:03.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.067+0000 7fab62503740 1 --2- 172.21.15.26:0/2890241719 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556993cb75d0 0x556993cb7a10 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:03.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.067+0000 7fab62503740 1 --2- 172.21.15.26:0/2890241719 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556993cb8b40 0x556993cb7090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:03.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.067+0000 7fab62503740 1 -- 172.21.15.26:0/2890241719 >> 172.21.15.26:0/2890241719 conn(0x556993c80310 msgr2=0x556993cb3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:03.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.067+0000 7fab62503740 1 -- 172.21.15.26:0/2890241719 shutdown_connections 2024-09-11T23:20:03.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.067+0000 7fab62503740 1 -- 172.21.15.26:0/2890241719 wait complete. 2024-09-11T23:20:03.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.067+0000 7fab62503740 1 -- start start 2024-09-11T23:20:03.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.067+0000 7fab62503740 1 Processor -- start 2024-09-11T23:20:03.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.068+0000 7fab62503740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556993cb75d0 0x556993cc1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:03.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.068+0000 7fab62503740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556993cb8b40 0x556993cbef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:03.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.068+0000 7fab62503740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556993cc2370 0x556993cbf4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:03.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.068+0000 7fab62503740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x556993cc5070 con 0x556993cc2370 2024-09-11T23:20:03.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.068+0000 7fab62503740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x556993bd8060 con 0x556993cb75d0 2024-09-11T23:20:03.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.068+0000 7fab62503740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x556993cc51f0 con 0x556993cb8b40 2024-09-11T23:20:03.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.068+0000 7fab62503740 10 client.0 _ll_register_callbacks cb 0x55699392b8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:20:03.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.068+0000 7fab5e6c7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556993cc2370 0x556993cbf4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:03.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.068+0000 7fab5e6c7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556993cc2370 0x556993cbf4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48294/0 (socket says 172.21.15.26:48294) 2024-09-11T23:20:03.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.068+0000 7fab5e6c7640 1 -- 172.21.15.26:0/4235786632 learned_addr learned my addr 172.21.15.26:0/4235786632 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:20:03.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.068+0000 7fab5dec6640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556993cb75d0 0x556993cc1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:03.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.068+0000 7fab5d6c5640 1 --2- 172.21.15.26:0/4235786632 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556993cb8b40 0x556993cbef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:03.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.068+0000 7fab5e6c7640 1 -- 172.21.15.26:0/4235786632 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556993cb8b40 msgr2=0x556993cbef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:20:03.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.068+0000 7fab5e6c7640 1 --2- 172.21.15.26:0/4235786632 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556993cb8b40 0x556993cbef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:03.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.068+0000 7fab5e6c7640 1 -- 172.21.15.26:0/4235786632 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556993cb75d0 msgr2=0x556993cc1ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:20:03.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.068+0000 7fab5e6c7640 1 --2- 172.21.15.26:0/4235786632 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556993cb75d0 0x556993cc1ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:03.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.068+0000 7fab5e6c7640 1 -- 172.21.15.26:0/4235786632 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55699398e3c0 con 0x556993cc2370 2024-09-11T23:20:03.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.068+0000 7fab5d6c5640 1 --2- 172.21.15.26:0/4235786632 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556993cb8b40 0x556993cbef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:20:03.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.068+0000 7fab5dec6640 1 --2- 172.21.15.26:0/4235786632 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556993cb75d0 0x556993cc1ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:20:03.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.069+0000 7fab5e6c7640 1 --2- 172.21.15.26:0/4235786632 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556993cc2370 0x556993cbf4f0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fab500035f0 tx=0x7fab50004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edf47400e009d182 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:03.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.069+0000 7fab417fa640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:20:03.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.069+0000 7fab417fa640 1 -- 172.21.15.26:0/4235786632 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab5000f850 con 0x556993cc2370 2024-09-11T23:20:03.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.069+0000 7fab417fa640 1 -- 172.21.15.26:0/4235786632 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fab5000f9f0 con 0x556993cc2370 2024-09-11T23:20:03.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.069+0000 7fab62503740 1 -- 172.21.15.26:0/4235786632 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab5409bf80 con 0x556993cc2370 2024-09-11T23:20:03.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.069+0000 7fab417fa640 1 -- 172.21.15.26:0/4235786632 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fab5000fbf0 con 0x556993cc2370 2024-09-11T23:20:03.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.069+0000 7fab62503740 10 client.15864 fetch_fsmap learned FSMap version 31 2024-09-11T23:20:03.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.069+0000 7fab62503740 1 -- 172.21.15.26:0/4235786632 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x556993cd0b30 con 0x556993cc2370 2024-09-11T23:20:03.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.069+0000 7fab417fa640 1 -- 172.21.15.26:0/4235786632 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab5409bf80 con 0x556993cc2370 2024-09-11T23:20:03.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.069+0000 7fab417fa640 1 -- 172.21.15.26:0/4235786632 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7fab50027070 con 0x556993cc2370 2024-09-11T23:20:03.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.070+0000 7fab62503740 10 client.15864 fetch_fsmap finished waiting for FSMap version 31 2024-09-11T23:20:03.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.070+0000 7fab62503740 10 client.15864 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-11T23:20:03.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.070+0000 7fab62503740 20 client.15864 populate_metadata read hostname 'smithi026' 2024-09-11T23:20:03.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.070+0000 7fab62503740 1 --2- 172.21.15.26:0/4235786632 >> [v2:172.21.15.76:6836/2078978748,v1:172.21.15.76:6837/2078978748] conn(0x556993cd1ff0 0x556993cd4440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:03.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.070+0000 7fab62503740 4 client.15864 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:20:03.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.070+0000 7fab62503740 1 -- 172.21.15.26:0/4235786632 --> [v2:172.21.15.76:6836/2078978748,v1:172.21.15.76:6837/2078978748] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556993c2a860 con 0x556993cd1ff0 2024-09-11T23:20:03.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.070+0000 7fab5dec6640 1 --2- 172.21.15.26:0/4235786632 >> [v2:172.21.15.76:6836/2078978748,v1:172.21.15.76:6837/2078978748] conn(0x556993cd1ff0 0x556993cd4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:03.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.070+0000 7fab417fa640 10 client.15864 ms_handle_connect on v2:172.21.15.76:6836/2078978748 2024-09-11T23:20:03.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.070+0000 7fab5dec6640 1 --2- 172.21.15.26:0/4235786632 >> [v2:172.21.15.76:6836/2078978748,v1:172.21.15.76:6837/2078978748] conn(0x556993cd1ff0 0x556993cd4440 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d6a6e19f71b54346 server_cookie=8be5032ea0d925d in_seq=0 out_seq=0 2024-09-11T23:20:03.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.071+0000 7fab417fa640 1 -- 172.21.15.26:0/4235786632 <== mds.2 v2:172.21.15.76:6836/2078978748 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556993c2a860 con 0x556993cd1ff0 2024-09-11T23:20:03.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.071+0000 7fab417fa640 10 client.15864 handle_command_reply: tid=0 2024-09-11T23:20:03.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.071+0000 7fab62503740 1 client.15864 shutdown 2024-09-11T23:20:03.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.071+0000 7fab62503740 1 -- 172.21.15.26:0/4235786632 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556993cc2370 msgr2=0x556993cbf4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:03.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.071+0000 7fab62503740 1 --2- 172.21.15.26:0/4235786632 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556993cc2370 0x556993cbf4f0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fab500035f0 tx=0x7fab50004730 comp rx=0 tx=0).stop 2024-09-11T23:20:03.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.071+0000 7fab62503740 1 -- 172.21.15.26:0/4235786632 shutdown_connections 2024-09-11T23:20:03.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.071+0000 7fab62503740 1 --2- 172.21.15.26:0/4235786632 >> [v2:172.21.15.76:6836/2078978748,v1:172.21.15.76:6837/2078978748] conn(0x556993cd1ff0 0x556993cd4440 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:03.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.071+0000 7fab62503740 1 --2- 172.21.15.26:0/4235786632 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556993cc2370 0x556993cbf4f0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:03.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.071+0000 7fab62503740 1 --2- 172.21.15.26:0/4235786632 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556993cb8b40 0x556993cbef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:03.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.071+0000 7fab417fa640 0 client.15864 ms_handle_reset on v2:172.21.15.76:6836/2078978748 2024-09-11T23:20:03.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.071+0000 7fab62503740 1 --2- 172.21.15.26:0/4235786632 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556993cb75d0 0x556993cc1ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:03.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.071+0000 7fab62503740 1 -- 172.21.15.26:0/4235786632 >> 172.21.15.26:0/4235786632 conn(0x556993c80310 msgr2=0x556993cb9960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:03.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.071+0000 7fab62503740 1 -- 172.21.15.26:0/4235786632 shutdown_connections 2024-09-11T23:20:03.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.071+0000 7fab62503740 1 -- 172.21.15.26:0/4235786632 wait complete. 2024-09-11T23:20:03.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.071+0000 7fab62503740 20 client.15864 trim_cache size 0 max 16384 2024-09-11T23:20:03.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.087+0000 7fab62503740 1 Processor -- start 2024-09-11T23:20:03.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.087+0000 7fab62503740 1 -- start start 2024-09-11T23:20:03.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.087+0000 7fab62503740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556993cb75d0 0x556993cb3e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:03.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.087+0000 7fab62503740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556993cb8b40 0x556993cce310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:03.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.087+0000 7fab62503740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556993cc2370 0x556993cba120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:03.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.087+0000 7fab62503740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fab581afaa0 con 0x556993cb75d0 2024-09-11T23:20:03.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.087+0000 7fab62503740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fab581b16d0 con 0x556993cb8b40 2024-09-11T23:20:03.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.087+0000 7fab62503740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fab581b0170 con 0x556993cc2370 2024-09-11T23:20:03.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.087+0000 7fab5dec6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556993cb75d0 0x556993cb3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:03.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.087+0000 7fab5dec6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556993cb75d0 0x556993cb3e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48298/0 (socket says 172.21.15.26:48298) 2024-09-11T23:20:03.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.087+0000 7fab5dec6640 1 -- 172.21.15.26:0/3864081909 learned_addr learned my addr 172.21.15.26:0/3864081909 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:20:03.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.087+0000 7fab5e6c7640 1 --2- 172.21.15.26:0/3864081909 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556993cc2370 0x556993cba120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:03.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.087+0000 7fab5d6c5640 1 --2- 172.21.15.26:0/3864081909 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556993cb8b40 0x556993cce310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:03.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.088+0000 7fab5dec6640 1 -- 172.21.15.26:0/3864081909 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556993cc2370 msgr2=0x556993cba120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:20:03.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.088+0000 7fab5dec6640 1 --2- 172.21.15.26:0/3864081909 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556993cc2370 0x556993cba120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:03.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.088+0000 7fab5dec6640 1 -- 172.21.15.26:0/3864081909 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556993cb8b40 msgr2=0x556993cce310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:20:03.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.088+0000 7fab5dec6640 1 --2- 172.21.15.26:0/3864081909 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556993cb8b40 0x556993cce310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:03.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.088+0000 7fab5dec6640 1 -- 172.21.15.26:0/3864081909 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556993cc2050 con 0x556993cb75d0 2024-09-11T23:20:03.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.088+0000 7fab5e6c7640 1 --2- 172.21.15.26:0/3864081909 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556993cc2370 0x556993cba120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:20:03.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.088+0000 7fab5dec6640 1 --2- 172.21.15.26:0/3864081909 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556993cb75d0 0x556993cb3e10 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fab54098390 tx=0x7fab5400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85615d926109168f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:03.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.088+0000 7fab437fe640 1 -- 172.21.15.26:0/3864081909 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab54023a90 con 0x556993cb75d0 2024-09-11T23:20:03.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.088+0000 7fab437fe640 1 -- 172.21.15.26:0/3864081909 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab54018420 con 0x556993cb75d0 2024-09-11T23:20:03.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.088+0000 7fab437fe640 1 -- 172.21.15.26:0/3864081909 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab54023c80 con 0x556993cb75d0 2024-09-11T23:20:03.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.089+0000 7fab62503740 1 -- 172.21.15.26:0/3864081909 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556993cb75d0 msgr2=0x556993cb3e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:03.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.089+0000 7fab62503740 1 --2- 172.21.15.26:0/3864081909 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556993cb75d0 0x556993cb3e10 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fab54098390 tx=0x7fab5400cf90 comp rx=0 tx=0).stop 2024-09-11T23:20:03.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.089+0000 7fab62503740 1 -- 172.21.15.26:0/3864081909 shutdown_connections 2024-09-11T23:20:03.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.089+0000 7fab62503740 1 --2- 172.21.15.26:0/3864081909 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556993cb8b40 0x556993cce310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:03.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.089+0000 7fab62503740 1 --2- 172.21.15.26:0/3864081909 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556993cc2370 0x556993cba120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:03.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.089+0000 7fab62503740 1 --2- 172.21.15.26:0/3864081909 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556993cb75d0 0x556993cb3e10 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:03.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.089+0000 7fab62503740 1 -- 172.21.15.26:0/3864081909 >> 172.21.15.26:0/3864081909 conn(0x556993c80310 msgr2=0x556993cb3700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:03.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.089+0000 7fab62503740 1 -- 172.21.15.26:0/3864081909 shutdown_connections 2024-09-11T23:20:03.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.089+0000 7fab62503740 1 -- 172.21.15.26:0/3864081909 wait complete. 2024-09-11T23:20:03.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.089+0000 7fab62503740 1 -- start start 2024-09-11T23:20:03.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.090+0000 7fab62503740 1 Processor -- start 2024-09-11T23:20:03.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.090+0000 7fab62503740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556993cb75d0 0x556993cca660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:03.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.090+0000 7fab62503740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556993cb8b40 0x556993ccaba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:03.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.090+0000 7fab62503740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556993cc2370 0x556993cbb7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:03.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.090+0000 7fab62503740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fab581b0f40 con 0x556993cc2370 2024-09-11T23:20:03.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.090+0000 7fab62503740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fab581b0840 con 0x556993cb8b40 2024-09-11T23:20:03.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.090+0000 7fab62503740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fab581ada80 con 0x556993cb75d0 2024-09-11T23:20:03.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.090+0000 7fab62503740 10 client.0 _ll_register_callbacks cb 0x7fab50009e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:20:03.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.090+0000 7fab5dec6640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556993cb75d0 0x556993cca660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:03.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.090+0000 7fab5e6c7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556993cc2370 0x556993cbb7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:03.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.090+0000 7fab5dec6640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556993cb75d0 0x556993cca660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:54566/0 (socket says 172.21.15.26:54566) 2024-09-11T23:20:03.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.090+0000 7fab5dec6640 1 -- 172.21.15.26:0/1130189551 learned_addr learned my addr 172.21.15.26:0/1130189551 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:20:03.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.090+0000 7fab5d6c5640 1 --2- 172.21.15.26:0/1130189551 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556993cb8b40 0x556993ccaba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:03.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.091+0000 7fab5d6c5640 1 -- 172.21.15.26:0/1130189551 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556993cb75d0 msgr2=0x556993cca660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:20:03.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.091+0000 7fab5d6c5640 1 --2- 172.21.15.26:0/1130189551 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556993cb75d0 0x556993cca660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:03.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.091+0000 7fab5d6c5640 1 -- 172.21.15.26:0/1130189551 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556993cc2370 msgr2=0x556993cbb7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:20:03.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.091+0000 7fab5d6c5640 1 --2- 172.21.15.26:0/1130189551 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556993cc2370 0x556993cbb7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:03.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.091+0000 7fab5d6c5640 1 -- 172.21.15.26:0/1130189551 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556993cbbe60 con 0x556993cb8b40 2024-09-11T23:20:03.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.091+0000 7fab5dec6640 1 --2- 172.21.15.26:0/1130189551 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556993cb75d0 0x556993cca660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:20:03.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.091+0000 7fab5e6c7640 1 --2- 172.21.15.26:0/1130189551 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556993cc2370 0x556993cbb7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:20:03.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.091+0000 7fab5d6c5640 1 --2- 172.21.15.26:0/1130189551 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556993cb8b40 0x556993ccaba0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fab48005940 tx=0x7fab48004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f69e8455e78e11f7 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:03.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.091+0000 7fab417fa640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-11T23:20:03.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.091+0000 7fab417fa640 1 -- 172.21.15.26:0/1130189551 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab4804b020 con 0x556993cb8b40 2024-09-11T23:20:03.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.091+0000 7fab62503740 1 -- 172.21.15.26:0/1130189551 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab54018420 con 0x556993cb8b40 2024-09-11T23:20:03.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.091+0000 7fab417fa640 1 -- 172.21.15.26:0/1130189551 <== mon.2 v2:172.21.15.76:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fab48005220 con 0x556993cb8b40 2024-09-11T23:20:03.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.091+0000 7fab417fa640 1 -- 172.21.15.26:0/1130189551 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fab48035040 con 0x556993cb8b40 2024-09-11T23:20:03.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.091+0000 7fab62503740 10 client.25457 fetch_fsmap learned FSMap version 31 2024-09-11T23:20:03.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.091+0000 7fab62503740 1 -- 172.21.15.26:0/1130189551 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x556993d3c740 con 0x556993cb8b40 2024-09-11T23:20:03.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.092+0000 7fab417fa640 1 -- 172.21.15.26:0/1130189551 <== mon.2 v2:172.21.15.76:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab54018420 con 0x556993cb8b40 2024-09-11T23:20:03.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.092+0000 7fab417fa640 1 -- 172.21.15.26:0/1130189551 <== mon.2 v2:172.21.15.76:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7fab48003710 con 0x556993cb8b40 2024-09-11T23:20:03.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.092+0000 7fab62503740 10 client.25457 fetch_fsmap finished waiting for FSMap version 31 2024-09-11T23:20:03.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.092+0000 7fab62503740 10 client.25457 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-11T23:20:03.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.092+0000 7fab62503740 20 client.25457 populate_metadata read hostname 'smithi026' 2024-09-11T23:20:03.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.092+0000 7fab62503740 1 --2- 172.21.15.26:0/1130189551 >> [v2:172.21.15.76:6836/2078978748,v1:172.21.15.76:6837/2078978748] conn(0x556993cd1ff0 0x556993d3eee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:03.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.092+0000 7fab62503740 4 client.25457 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:20:03.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.092+0000 7fab62503740 1 -- 172.21.15.26:0/1130189551 --> [v2:172.21.15.76:6836/2078978748,v1:172.21.15.76:6837/2078978748] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556993c24d70 con 0x556993cd1ff0 2024-09-11T23:20:03.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.092+0000 7fab5dec6640 1 --2- 172.21.15.26:0/1130189551 >> [v2:172.21.15.76:6836/2078978748,v1:172.21.15.76:6837/2078978748] conn(0x556993cd1ff0 0x556993d3eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:03.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.092+0000 7fab5dec6640 1 --2- 172.21.15.26:0/1130189551 >> [v2:172.21.15.76:6836/2078978748,v1:172.21.15.76:6837/2078978748] conn(0x556993cd1ff0 0x556993d3eee0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cd2cb14aada7c1fb server_cookie=b2a05dae324de2b6 in_seq=0 out_seq=0 2024-09-11T23:20:03.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.092+0000 7fab417fa640 10 client.25457 ms_handle_connect on v2:172.21.15.76:6836/2078978748 2024-09-11T23:20:03.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.092+0000 7fab417fa640 1 -- 172.21.15.26:0/1130189551 <== mds.2 v2:172.21.15.76:6836/2078978748 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556993c24d70 con 0x556993cd1ff0 2024-09-11T23:20:03.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.092+0000 7fab417fa640 10 client.25457 handle_command_reply: tid=0 2024-09-11T23:20:03.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.093+0000 7fab62503740 1 client.25457 shutdown 2024-09-11T23:20:03.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.093+0000 7fab62503740 1 -- 172.21.15.26:0/1130189551 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556993cb8b40 msgr2=0x556993ccaba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:03.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.093+0000 7fab62503740 1 --2- 172.21.15.26:0/1130189551 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556993cb8b40 0x556993ccaba0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fab48005940 tx=0x7fab48004270 comp rx=0 tx=0).stop 2024-09-11T23:20:03.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.093+0000 7fab62503740 1 -- 172.21.15.26:0/1130189551 shutdown_connections 2024-09-11T23:20:03.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.093+0000 7fab62503740 1 --2- 172.21.15.26:0/1130189551 >> [v2:172.21.15.76:6836/2078978748,v1:172.21.15.76:6837/2078978748] conn(0x556993cd1ff0 0x556993d3eee0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:03.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.093+0000 7fab62503740 1 --2- 172.21.15.26:0/1130189551 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556993cb8b40 0x556993ccaba0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:03.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.093+0000 7fab62503740 1 --2- 172.21.15.26:0/1130189551 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556993cc2370 0x556993cbb7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:03.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.093+0000 7fab62503740 1 --2- 172.21.15.26:0/1130189551 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556993cb75d0 0x556993cca660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:03.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.093+0000 7fab417fa640 0 client.25457 ms_handle_reset on v2:172.21.15.76:6836/2078978748 2024-09-11T23:20:03.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.093+0000 7fab62503740 1 -- 172.21.15.26:0/1130189551 >> 172.21.15.26:0/1130189551 conn(0x556993c80310 msgr2=0x556993cb24e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:03.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.093+0000 7fab62503740 1 -- 172.21.15.26:0/1130189551 shutdown_connections 2024-09-11T23:20:03.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.093+0000 7fab62503740 1 -- 172.21.15.26:0/1130189551 wait complete. 2024-09-11T23:20:03.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.093+0000 7fab62503740 20 client.25457 trim_cache size 0 max 16384 2024-09-11T23:20:03.100 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2024-09-11T23:20:03.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.093+0000 7fab6012a640 1 -- 172.21.15.26:0/3295115293 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fab440a2cf0 msgr2=0x7fab440a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:03.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.094+0000 7fab6012a640 1 --2- 172.21.15.26:0/3295115293 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fab440a2cf0 0x7fab440a51b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fab50012640 tx=0x7fab50005f90 comp rx=0 tx=0).stop 2024-09-11T23:20:03.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.094+0000 7fab6012a640 1 -- 172.21.15.26:0/3295115293 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fab58100a80 msgr2=0x7fab5819a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:03.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.094+0000 7fab6012a640 1 --2- 172.21.15.26:0/3295115293 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fab58100a80 0x7fab5819a680 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fab5400ebc0 tx=0x7fab5400c6a0 comp rx=0 tx=0).stop 2024-09-11T23:20:03.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.094+0000 7fab6012a640 1 -- 172.21.15.26:0/3295115293 shutdown_connections 2024-09-11T23:20:03.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.094+0000 7fab6012a640 1 --2- 172.21.15.26:0/3295115293 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fab440a2cf0 0x7fab440a51b0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:03.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.094+0000 7fab6012a640 1 --2- 172.21.15.26:0/3295115293 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fab58106a80 0x7fab5819ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:03.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.094+0000 7fab6012a640 1 --2- 172.21.15.26:0/3295115293 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fab58101420 0x7fab5819abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:03.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.094+0000 7fab6012a640 1 --2- 172.21.15.26:0/3295115293 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fab58100a80 0x7fab5819a680 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:03.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.094+0000 7fab6012a640 1 -- 172.21.15.26:0/3295115293 >> 172.21.15.26:0/3295115293 conn(0x7fab580fc7a0 msgr2=0x7fab5806a790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:03.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.094+0000 7fab6012a640 1 -- 172.21.15.26:0/3295115293 shutdown_connections 2024-09-11T23:20:03.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:03.094+0000 7fab6012a640 1 -- 172.21.15.26:0/3295115293 wait complete. 2024-09-11T23:20:03.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:02 smithi136 ceph-mon[35121]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:20:03.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:02 smithi136 ceph-mon[35121]: pgmap v53: 97 pgs: 97 active+clean; 14 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 115 op/s 2024-09-11T23:20:03.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:02 smithi026 ceph-mon[31172]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:20:03.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:02 smithi026 ceph-mon[31172]: pgmap v53: 97 pgs: 97 active+clean; 14 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 115 op/s 2024-09-11T23:20:03.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:02 smithi076 ceph-mon[35296]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:20:03.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:02 smithi076 ceph-mon[35296]: pgmap v53: 97 pgs: 97 active+clean; 14 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 115 op/s 2024-09-11T23:20:03.576 INFO:tasks.workunit.client.0.smithi026.stdout: 18 142639 1013 77900 921 35088 753 2024-09-11T23:20:03.877 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-11T23:20:04.140 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.139+0000 7f075f2d0640 1 -- 172.21.15.26:0/2555816029 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f075810c0d0 msgr2=0x7f075810e570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:04.140 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.139+0000 7f075f2d0640 1 --2- 172.21.15.26:0/2555816029 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f075810c0d0 0x7f075810e570 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f0750009be0 tx=0x7f075002a6c0 comp rx=0 tx=0).stop 2024-09-11T23:20:04.140 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.139+0000 7f075f2d0640 1 -- 172.21.15.26:0/2555816029 shutdown_connections 2024-09-11T23:20:04.140 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.139+0000 7f075f2d0640 1 --2- 172.21.15.26:0/2555816029 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f075810c0d0 0x7f075810e570 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:04.140 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.139+0000 7f075f2d0640 1 --2- 172.21.15.26:0/2555816029 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07581035c0 0x7f075810baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:04.140 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.139+0000 7f075f2d0640 1 --2- 172.21.15.26:0/2555816029 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0758102ca0 0x7f0758103080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:04.140 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.139+0000 7f075f2d0640 1 -- 172.21.15.26:0/2555816029 >> 172.21.15.26:0/2555816029 conn(0x7f07580fc6b0 msgr2=0x7f07580fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:04.140 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.139+0000 7f075f2d0640 1 -- 172.21.15.26:0/2555816029 shutdown_connections 2024-09-11T23:20:04.140 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.139+0000 7f075f2d0640 1 -- 172.21.15.26:0/2555816029 wait complete. 2024-09-11T23:20:04.140 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.140+0000 7f075f2d0640 1 Processor -- start 2024-09-11T23:20:04.140 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.140+0000 7f075f2d0640 1 -- start start 2024-09-11T23:20:04.142 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.140+0000 7f075f2d0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0758102ca0 0x7f075819a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:04.142 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.140+0000 7f075f2d0640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f07581035c0 0x7f075819aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:04.142 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.140+0000 7f075f2d0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f075810c0d0 0x7f075819eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:04.142 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.140+0000 7f075f2d0640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f0758111e80 con 0x7f075810c0d0 2024-09-11T23:20:04.142 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.140+0000 7f075f2d0640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f0758111d00 con 0x7f0758102ca0 2024-09-11T23:20:04.142 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.140+0000 7f075f2d0640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f0758112000 con 0x7f07581035c0 2024-09-11T23:20:04.142 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.141+0000 7f075c86b640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f07581035c0 0x7f075819aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:04.143 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.141+0000 7f075d86d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f075810c0d0 0x7f075819eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:04.143 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.141+0000 7f075d06c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0758102ca0 0x7f075819a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:04.143 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.141+0000 7f075c86b640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f07581035c0 0x7f075819aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:54590/0 (socket says 172.21.15.26:54590) 2024-09-11T23:20:04.143 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.141+0000 7f075c86b640 1 -- 172.21.15.26:0/2670677157 learned_addr learned my addr 172.21.15.26:0/2670677157 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:20:04.143 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.141+0000 7f075c86b640 1 -- 172.21.15.26:0/2670677157 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0758102ca0 msgr2=0x7f075819a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:04.143 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.141+0000 7f075c86b640 1 --2- 172.21.15.26:0/2670677157 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0758102ca0 0x7f075819a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:04.143 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.141+0000 7f075c86b640 1 -- 172.21.15.26:0/2670677157 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f075810c0d0 msgr2=0x7f075819eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:04.143 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.141+0000 7f075c86b640 1 --2- 172.21.15.26:0/2670677157 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f075810c0d0 0x7f075819eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:04.143 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.141+0000 7f075c86b640 1 -- 172.21.15.26:0/2670677157 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f075819f4d0 con 0x7f07581035c0 2024-09-11T23:20:04.143 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.141+0000 7f075d06c640 1 --2- 172.21.15.26:0/2670677157 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0758102ca0 0x7f075819a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:20:04.144 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.141+0000 7f075c86b640 1 --2- 172.21.15.26:0/2670677157 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f07581035c0 0x7f075819aa20 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f074800e9a0 tx=0x7f074800ee60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:04.144 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.141+0000 7f074e7fc640 1 -- 172.21.15.26:0/2670677157 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f074800cd80 con 0x7f07581035c0 2024-09-11T23:20:04.144 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.141+0000 7f075f2d0640 1 -- 172.21.15.26:0/2670677157 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f075819f7c0 con 0x7f07581035c0 2024-09-11T23:20:04.144 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.141+0000 7f074e7fc640 1 -- 172.21.15.26:0/2670677157 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0748010910 con 0x7f07581035c0 2024-09-11T23:20:04.144 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.141+0000 7f074e7fc640 1 -- 172.21.15.26:0/2670677157 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0748018a40 con 0x7f07581035c0 2024-09-11T23:20:04.144 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.142+0000 7f075f2d0640 1 -- 172.21.15.26:0/2670677157 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07581a7050 con 0x7f07581035c0 2024-09-11T23:20:04.145 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.144+0000 7f074e7fc640 1 -- 172.21.15.26:0/2670677157 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f0748018be0 con 0x7f07581035c0 2024-09-11T23:20:04.145 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.144+0000 7f076206d740 1 Processor -- start 2024-09-11T23:20:04.145 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.145+0000 7f076206d740 1 -- start start 2024-09-11T23:20:04.146 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.145+0000 7f076206d740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558b656d0af0 0x558b656d0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:04.146 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.145+0000 7f076206d740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558b656d1410 0x558b656db770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:04.146 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.145+0000 7f076206d740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558b656dbcb0 0x558b656de0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:04.146 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.145+0000 7f076206d740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x558b656352f0 con 0x558b656dbcb0 2024-09-11T23:20:04.146 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.145+0000 7f076206d740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x558b655dd270 con 0x558b656d0af0 2024-09-11T23:20:04.146 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.145+0000 7f076206d740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x558b656312b0 con 0x558b656d1410 2024-09-11T23:20:04.146 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.145+0000 7f075d06c640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558b656d1410 0x558b656db770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:04.146 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.145+0000 7f075d06c640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558b656d1410 0x558b656db770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:54604/0 (socket says 172.21.15.26:54604) 2024-09-11T23:20:04.146 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.145+0000 7f075d06c640 1 -- 172.21.15.26:0/2533948580 learned_addr learned my addr 172.21.15.26:0/2533948580 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:20:04.146 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.145+0000 7f074e7fc640 1 --2- 172.21.15.26:0/2670677157 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f07440a2da0 0x7f07440a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:04.146 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.145+0000 7f075c86b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558b656dbcb0 0x558b656de0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:04.146 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.145+0000 7f075d06c640 1 -- 172.21.15.26:0/2533948580 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558b656d0af0 msgr2=0x558b656d0ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:20:04.146 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.145+0000 7f075d06c640 1 --2- 172.21.15.26:0/2533948580 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558b656d0af0 0x558b656d0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:04.147 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.146+0000 7f075d06c640 1 -- 172.21.15.26:0/2533948580 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558b656dbcb0 msgr2=0x558b656de0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:20:04.147 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.146+0000 7f075d06c640 1 --2- 172.21.15.26:0/2533948580 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558b656dbcb0 0x558b656de0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:04.147 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.146+0000 7f075d06c640 1 -- 172.21.15.26:0/2533948580 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b655dc340 con 0x558b656d1410 2024-09-11T23:20:04.150 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.146+0000 7f075d86d640 1 --2- 172.21.15.26:0/2670677157 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f07440a2da0 0x7f07440a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:04.150 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.146+0000 7f075d06c640 1 --2- 172.21.15.26:0/2533948580 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558b656d1410 0x558b656db770 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f07540104d0 tx=0x7f0754010990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bfd25e7e992d1faa server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:04.150 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.146+0000 7f074e7fc640 1 -- 172.21.15.26:0/2670677157 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f07480cf7e0 con 0x7f07581035c0 2024-09-11T23:20:04.151 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.146+0000 7f07437fe640 1 -- 172.21.15.26:0/2533948580 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0754004220 con 0x558b656d1410 2024-09-11T23:20:04.151 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.146+0000 7f07437fe640 1 -- 172.21.15.26:0/2533948580 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f07540043c0 con 0x558b656d1410 2024-09-11T23:20:04.151 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.146+0000 7f07437fe640 1 -- 172.21.15.26:0/2533948580 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0754011bd0 con 0x558b656d1410 2024-09-11T23:20:04.151 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.147+0000 7f076206d740 1 -- 172.21.15.26:0/2533948580 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558b656d1410 msgr2=0x558b656db770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:04.151 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.147+0000 7f075d86d640 1 --2- 172.21.15.26:0/2670677157 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f07440a2da0 0x7f07440a5260 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f0750005cc0 tx=0x7f0750005c30 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:04.151 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.147+0000 7f076206d740 1 --2- 172.21.15.26:0/2533948580 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558b656d1410 0x558b656db770 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f07540104d0 tx=0x7f0754010990 comp rx=0 tx=0).stop 2024-09-11T23:20:04.151 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.147+0000 7f076206d740 1 -- 172.21.15.26:0/2533948580 shutdown_connections 2024-09-11T23:20:04.151 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.147+0000 7f076206d740 1 --2- 172.21.15.26:0/2533948580 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558b656dbcb0 0x558b656de0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:04.151 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.147+0000 7f076206d740 1 --2- 172.21.15.26:0/2533948580 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558b656d1410 0x558b656db770 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:04.151 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.147+0000 7f076206d740 1 --2- 172.21.15.26:0/2533948580 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558b656d0af0 0x558b656d0ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:04.151 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.147+0000 7f076206d740 1 -- 172.21.15.26:0/2533948580 >> 172.21.15.26:0/2533948580 conn(0x558b656cd060 msgr2=0x558b656cd440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:04.151 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.147+0000 7f076206d740 1 -- 172.21.15.26:0/2533948580 shutdown_connections 2024-09-11T23:20:04.151 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.147+0000 7f076206d740 1 -- 172.21.15.26:0/2533948580 wait complete. 2024-09-11T23:20:04.152 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.148+0000 7f076206d740 1 -- start start 2024-09-11T23:20:04.152 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.148+0000 7f076206d740 1 Processor -- start 2024-09-11T23:20:04.152 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.149+0000 7f076206d740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558b656d0af0 0x558b656d8bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:04.152 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.149+0000 7f076206d740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558b656d1410 0x558b656d9130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:04.152 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.149+0000 7f076206d740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558b656dbcb0 0x558b656d9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:04.152 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.149+0000 7f076206d740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x558b656de880 con 0x558b656d1410 2024-09-11T23:20:04.152 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.149+0000 7f076206d740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x558b655be290 con 0x558b656dbcb0 2024-09-11T23:20:04.152 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.149+0000 7f076206d740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x558b656dea00 con 0x558b656d0af0 2024-09-11T23:20:04.152 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.149+0000 7f076206d740 10 client.0 _ll_register_callbacks cb 0x558b655ef140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:20:04.152 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.149+0000 7f075c86b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558b656d1410 0x558b656d9130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:04.152 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.149+0000 7f075d06c640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558b656d0af0 0x558b656d8bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:04.152 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.149+0000 7f075c86b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558b656d1410 0x558b656d9130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48352/0 (socket says 172.21.15.26:48352) 2024-09-11T23:20:04.152 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.149+0000 7f075c86b640 1 -- 172.21.15.26:0/1370250094 learned_addr learned my addr 172.21.15.26:0/1370250094 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:20:04.152 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.149+0000 7f075d86d640 1 --2- 172.21.15.26:0/1370250094 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558b656dbcb0 0x558b656d9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:04.153 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.149+0000 7f075c86b640 1 -- 172.21.15.26:0/1370250094 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558b656d0af0 msgr2=0x558b656d8bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:20:04.153 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.149+0000 7f075c86b640 1 --2- 172.21.15.26:0/1370250094 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558b656d0af0 0x558b656d8bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:04.153 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.149+0000 7f075c86b640 1 -- 172.21.15.26:0/1370250094 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558b656dbcb0 msgr2=0x558b656d9670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:20:04.153 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.149+0000 7f075c86b640 1 --2- 172.21.15.26:0/1370250094 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558b656dbcb0 0x558b656d9670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:04.153 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.149+0000 7f075c86b640 1 -- 172.21.15.26:0/1370250094 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b65348210 con 0x558b656d1410 2024-09-11T23:20:04.153 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.149+0000 7f075d06c640 1 --2- 172.21.15.26:0/1370250094 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558b656d0af0 0x558b656d8bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:20:04.153 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.149+0000 7f075c86b640 1 --2- 172.21.15.26:0/1370250094 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558b656d1410 0x558b656d9130 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f07480058c0 tx=0x7f0748009620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c928a385716ff3eb server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:04.153 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.149+0000 7f0740ff9640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:20:04.153 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.150+0000 7f0740ff9640 1 -- 172.21.15.26:0/1370250094 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f074809b610 con 0x558b656d1410 2024-09-11T23:20:04.153 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.150+0000 7f0740ff9640 1 -- 172.21.15.26:0/1370250094 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f07480cd160 con 0x558b656d1410 2024-09-11T23:20:04.153 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.150+0000 7f076206d740 1 -- 172.21.15.26:0/1370250094 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f07540043c0 con 0x558b656d1410 2024-09-11T23:20:04.153 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.150+0000 7f0740ff9640 1 -- 172.21.15.26:0/1370250094 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f07480cd360 con 0x558b656d1410 2024-09-11T23:20:04.153 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.150+0000 7f076206d740 10 client.15882 fetch_fsmap learned FSMap version 31 2024-09-11T23:20:04.154 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.150+0000 7f076206d740 1 -- 172.21.15.26:0/1370250094 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x558b65414620 con 0x558b656d1410 2024-09-11T23:20:04.154 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.150+0000 7f0740ff9640 1 -- 172.21.15.26:0/1370250094 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f07540043c0 con 0x558b656d1410 2024-09-11T23:20:04.154 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.150+0000 7f0740ff9640 1 -- 172.21.15.26:0/1370250094 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7f07480cfbd0 con 0x558b656d1410 2024-09-11T23:20:04.154 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.150+0000 7f076206d740 10 client.15882 fetch_fsmap finished waiting for FSMap version 31 2024-09-11T23:20:04.154 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.150+0000 7f076206d740 10 client.15882 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:20:04.154 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.150+0000 7f076206d740 20 client.15882 populate_metadata read hostname 'smithi026' 2024-09-11T23:20:04.154 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.150+0000 7f076206d740 1 --2- 172.21.15.26:0/1370250094 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x558b656ebca0 0x558b656ee0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:04.154 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.150+0000 7f076206d740 4 client.15882 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:20:04.154 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.150+0000 7f076206d740 1 -- 172.21.15.26:0/1370250094 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558b65642a20 con 0x558b656ebca0 2024-09-11T23:20:04.154 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.151+0000 7f075d06c640 1 --2- 172.21.15.26:0/1370250094 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x558b656ebca0 0x558b656ee0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:04.155 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.151+0000 7f075d06c640 1 --2- 172.21.15.26:0/1370250094 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x558b656ebca0 0x558b656ee0f0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75e7aebc23647787 server_cookie=54075b690de88424 in_seq=0 out_seq=0 2024-09-11T23:20:04.156 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.151+0000 7f0740ff9640 10 client.15882 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:20:04.156 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.152+0000 7f0740ff9640 1 -- 172.21.15.26:0/1370250094 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558b65642a20 con 0x558b656ebca0 2024-09-11T23:20:04.156 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.152+0000 7f0740ff9640 10 client.15882 handle_command_reply: tid=0 2024-09-11T23:20:04.156 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.152+0000 7f076206d740 1 client.15882 shutdown 2024-09-11T23:20:04.156 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.152+0000 7f076206d740 1 -- 172.21.15.26:0/1370250094 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558b656d1410 msgr2=0x558b656d9130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:04.156 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.152+0000 7f076206d740 1 --2- 172.21.15.26:0/1370250094 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558b656d1410 0x558b656d9130 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f07480058c0 tx=0x7f0748009620 comp rx=0 tx=0).stop 2024-09-11T23:20:04.156 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.152+0000 7f076206d740 1 -- 172.21.15.26:0/1370250094 shutdown_connections 2024-09-11T23:20:04.156 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.152+0000 7f076206d740 1 --2- 172.21.15.26:0/1370250094 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558b656dbcb0 0x558b656d9670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:04.156 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.152+0000 7f076206d740 1 --2- 172.21.15.26:0/1370250094 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x558b656ebca0 0x558b656ee0f0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:04.156 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.152+0000 7f076206d740 1 --2- 172.21.15.26:0/1370250094 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558b656d1410 0x558b656d9130 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:04.156 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.152+0000 7f076206d740 1 --2- 172.21.15.26:0/1370250094 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558b656d0af0 0x558b656d8bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:04.156 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.152+0000 7f076206d740 1 -- 172.21.15.26:0/1370250094 >> 172.21.15.26:0/1370250094 conn(0x558b656cd060 msgr2=0x558b656d2d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:04.156 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.152+0000 7f0740ff9640 0 client.15882 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:20:04.157 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.152+0000 7f076206d740 1 -- 172.21.15.26:0/1370250094 shutdown_connections 2024-09-11T23:20:04.157 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.152+0000 7f076206d740 1 -- 172.21.15.26:0/1370250094 wait complete. 2024-09-11T23:20:04.157 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.152+0000 7f076206d740 20 client.15882 trim_cache size 0 max 16384 2024-09-11T23:20:04.157 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:20:04.168 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.167+0000 7f076206d740 1 Processor -- start 2024-09-11T23:20:04.168 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.167+0000 7f076206d740 1 -- start start 2024-09-11T23:20:04.169 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.167+0000 7f076206d740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558b656d0af0 0x558b656d3560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:04.169 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.167+0000 7f076206d740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558b656d1410 0x558b656d94a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:04.169 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.167+0000 7f076206d740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558b656dbcb0 0x558b656dab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:04.169 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.168+0000 7f076206d740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f07581afa30 con 0x558b656dbcb0 2024-09-11T23:20:04.169 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.168+0000 7f076206d740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f07581b1660 con 0x558b656d1410 2024-09-11T23:20:04.169 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.168+0000 7f076206d740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f07581b0100 con 0x558b656d0af0 2024-09-11T23:20:04.169 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.168+0000 7f075d06c640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558b656d0af0 0x558b656d3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:04.169 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.168+0000 7f075d86d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558b656dbcb0 0x558b656dab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:04.169 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.168+0000 7f075d06c640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558b656d0af0 0x558b656d3560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:54614/0 (socket says 172.21.15.26:54614) 2024-09-11T23:20:04.169 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.168+0000 7f075d06c640 1 -- 172.21.15.26:0/2595430154 learned_addr learned my addr 172.21.15.26:0/2595430154 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:20:04.169 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.168+0000 7f075c86b640 1 --2- 172.21.15.26:0/2595430154 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558b656d1410 0x558b656d94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:04.169 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.168+0000 7f075d06c640 1 -- 172.21.15.26:0/2595430154 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558b656d1410 msgr2=0x558b656d94a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:20:04.169 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.168+0000 7f075d06c640 1 --2- 172.21.15.26:0/2595430154 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558b656d1410 0x558b656d94a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:04.170 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.168+0000 7f075d06c640 1 -- 172.21.15.26:0/2595430154 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558b656dbcb0 msgr2=0x558b656dab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:20:04.170 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.168+0000 7f075d06c640 1 --2- 172.21.15.26:0/2595430154 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558b656dbcb0 0x558b656dab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:04.170 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.168+0000 7f075d06c640 1 -- 172.21.15.26:0/2595430154 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b656ce1d0 con 0x558b656d0af0 2024-09-11T23:20:04.170 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.168+0000 7f075c86b640 1 --2- 172.21.15.26:0/2595430154 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558b656d1410 0x558b656d94a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:20:04.170 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.168+0000 7f075d06c640 1 --2- 172.21.15.26:0/2595430154 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558b656d0af0 0x558b656d3560 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f07540104d0 tx=0x7f07540051d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3016bc6f89003b5b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:04.170 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.168+0000 7f0742ffd640 1 -- 172.21.15.26:0/2595430154 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07540054c0 con 0x558b656d0af0 2024-09-11T23:20:04.170 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.168+0000 7f0742ffd640 1 -- 172.21.15.26:0/2595430154 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0754004660 con 0x558b656d0af0 2024-09-11T23:20:04.170 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.168+0000 7f0742ffd640 1 -- 172.21.15.26:0/2595430154 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0754003850 con 0x558b656d0af0 2024-09-11T23:20:04.173 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.169+0000 7f076206d740 1 -- 172.21.15.26:0/2595430154 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558b656d0af0 msgr2=0x558b656d3560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:04.173 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.169+0000 7f076206d740 1 --2- 172.21.15.26:0/2595430154 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558b656d0af0 0x558b656d3560 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f07540104d0 tx=0x7f07540051d0 comp rx=0 tx=0).stop 2024-09-11T23:20:04.173 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.169+0000 7f076206d740 1 -- 172.21.15.26:0/2595430154 shutdown_connections 2024-09-11T23:20:04.173 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.169+0000 7f076206d740 1 --2- 172.21.15.26:0/2595430154 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558b656d1410 0x558b656d94a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:04.174 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.169+0000 7f076206d740 1 --2- 172.21.15.26:0/2595430154 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558b656dbcb0 0x558b656dab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:04.174 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.169+0000 7f076206d740 1 --2- 172.21.15.26:0/2595430154 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558b656d0af0 0x558b656d3560 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:04.174 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.169+0000 7f076206d740 1 -- 172.21.15.26:0/2595430154 >> 172.21.15.26:0/2595430154 conn(0x558b656cd060 msgr2=0x558b656da450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:04.174 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.169+0000 7f076206d740 1 -- 172.21.15.26:0/2595430154 shutdown_connections 2024-09-11T23:20:04.174 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.170+0000 7f076206d740 1 -- 172.21.15.26:0/2595430154 wait complete. 2024-09-11T23:20:04.174 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.170+0000 7f076206d740 1 -- start start 2024-09-11T23:20:04.174 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.170+0000 7f076206d740 1 Processor -- start 2024-09-11T23:20:04.174 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.170+0000 7f076206d740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558b656d0af0 0x558b656ded70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:04.174 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.170+0000 7f076206d740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558b656d1410 0x558b656df2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:04.174 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.170+0000 7f076206d740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558b656dbcb0 0x558b65754980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:04.174 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.170+0000 7f076206d740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f07581b0ed0 con 0x558b656d1410 2024-09-11T23:20:04.174 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.170+0000 7f076206d740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f07581b07d0 con 0x558b656dbcb0 2024-09-11T23:20:04.174 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.170+0000 7f076206d740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f07581ada10 con 0x558b656d0af0 2024-09-11T23:20:04.175 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.170+0000 7f076206d740 10 client.0 _ll_register_callbacks cb 0x7f074800ef90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:20:04.175 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.171+0000 7f075c86b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558b656d1410 0x558b656df2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:04.175 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.171+0000 7f075d06c640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558b656d0af0 0x558b656ded70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:04.175 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.171+0000 7f075d06c640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558b656d0af0 0x558b656ded70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:54616/0 (socket says 172.21.15.26:54616) 2024-09-11T23:20:04.175 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.171+0000 7f075d06c640 1 -- 172.21.15.26:0/3132400089 learned_addr learned my addr 172.21.15.26:0/3132400089 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:20:04.175 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.171+0000 7f075d86d640 1 --2- 172.21.15.26:0/3132400089 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558b656dbcb0 0x558b65754980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:04.175 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.171+0000 7f075c86b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558b656d1410 0x558b656df2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48380/0 (socket says 172.21.15.26:48380) 2024-09-11T23:20:04.175 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.171+0000 7f075d06c640 1 -- 172.21.15.26:0/3132400089 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558b656dbcb0 msgr2=0x558b65754980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:20:04.175 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.171+0000 7f075d06c640 1 --2- 172.21.15.26:0/3132400089 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558b656dbcb0 0x558b65754980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:04.175 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.171+0000 7f075d06c640 1 -- 172.21.15.26:0/3132400089 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558b656d1410 msgr2=0x558b656df2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:20:04.175 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.171+0000 7f075d06c640 1 --2- 172.21.15.26:0/3132400089 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558b656d1410 0x558b656df2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:04.175 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.171+0000 7f075d06c640 1 -- 172.21.15.26:0/3132400089 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b65754ec0 con 0x558b656d0af0 2024-09-11T23:20:04.175 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.171+0000 7f075c86b640 1 --2- 172.21.15.26:0/3132400089 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558b656d1410 0x558b656df2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:20:04.175 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.171+0000 7f075d06c640 1 --2- 172.21.15.26:0/3132400089 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558b656d0af0 0x558b656ded70 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f0754005040 tx=0x7f0754005070 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3f76bf271c253c51 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:04.176 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.171+0000 7f0740ff9640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-09-11T23:20:04.176 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.171+0000 7f0740ff9640 1 -- 172.21.15.26:0/3132400089 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0754038020 con 0x558b656d0af0 2024-09-11T23:20:04.176 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.171+0000 7f0740ff9640 1 -- 172.21.15.26:0/3132400089 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f0754004020 con 0x558b656d0af0 2024-09-11T23:20:04.176 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.171+0000 7f076206d740 1 -- 172.21.15.26:0/3132400089 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0754004660 con 0x558b656d0af0 2024-09-11T23:20:04.176 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.172+0000 7f0740ff9640 1 -- 172.21.15.26:0/3132400089 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f0754004240 con 0x558b656d0af0 2024-09-11T23:20:04.176 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.172+0000 7f076206d740 10 client.25522 fetch_fsmap learned FSMap version 31 2024-09-11T23:20:04.176 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.172+0000 7f076206d740 1 -- 172.21.15.26:0/3132400089 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x558b65756070 con 0x558b656d0af0 2024-09-11T23:20:04.176 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.172+0000 7f0740ff9640 1 -- 172.21.15.26:0/3132400089 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0754004660 con 0x558b656d0af0 2024-09-11T23:20:04.176 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.172+0000 7f0740ff9640 1 -- 172.21.15.26:0/3132400089 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7f0754003b20 con 0x558b656d0af0 2024-09-11T23:20:04.176 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.172+0000 7f076206d740 10 client.25522 fetch_fsmap finished waiting for FSMap version 31 2024-09-11T23:20:04.176 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.172+0000 7f076206d740 10 client.25522 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:20:04.176 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.172+0000 7f076206d740 20 client.25522 populate_metadata read hostname 'smithi026' 2024-09-11T23:20:04.176 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.172+0000 7f076206d740 1 --2- 172.21.15.26:0/3132400089 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x558b65756610 0x558b65758a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:04.176 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.172+0000 7f076206d740 4 client.25522 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T23:20:04.177 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.172+0000 7f076206d740 1 -- 172.21.15.26:0/3132400089 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558b6563cd70 con 0x558b65756610 2024-09-11T23:20:04.177 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.172+0000 7f075c86b640 1 --2- 172.21.15.26:0/3132400089 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x558b65756610 0x558b65758a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:04.177 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.172+0000 7f0740ff9640 10 client.25522 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:20:04.177 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.172+0000 7f075c86b640 1 --2- 172.21.15.26:0/3132400089 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x558b65756610 0x558b65758a30 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c606e58245733931 server_cookie=e4a37a39bed07dc8 in_seq=0 out_seq=0 2024-09-11T23:20:04.302 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.302+0000 7f0740ff9640 1 -- 172.21.15.26:0/3132400089 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558b6563cd70 con 0x558b65756610 2024-09-11T23:20:04.303 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.302+0000 7f0740ff9640 10 client.25522 handle_command_reply: tid=0 2024-09-11T23:20:04.303 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.302+0000 7f076206d740 1 client.25522 shutdown 2024-09-11T23:20:04.303 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.302+0000 7f076206d740 1 -- 172.21.15.26:0/3132400089 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558b656d0af0 msgr2=0x558b656ded70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:04.303 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.302+0000 7f076206d740 1 --2- 172.21.15.26:0/3132400089 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558b656d0af0 0x558b656ded70 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f0754005040 tx=0x7f0754005070 comp rx=0 tx=0).stop 2024-09-11T23:20:04.303 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.302+0000 7f076206d740 1 -- 172.21.15.26:0/3132400089 shutdown_connections 2024-09-11T23:20:04.303 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.302+0000 7f076206d740 1 --2- 172.21.15.26:0/3132400089 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558b656dbcb0 0x558b65754980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:04.303 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.302+0000 7f076206d740 1 --2- 172.21.15.26:0/3132400089 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x558b65756610 0x558b65758a30 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:04.303 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.302+0000 7f076206d740 1 --2- 172.21.15.26:0/3132400089 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558b656d1410 0x558b656df2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:04.303 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.302+0000 7f076206d740 1 --2- 172.21.15.26:0/3132400089 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558b656d0af0 0x558b656ded70 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:04.303 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.302+0000 7f0740ff9640 0 client.25522 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:20:04.303 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.302+0000 7f076206d740 1 -- 172.21.15.26:0/3132400089 >> 172.21.15.26:0/3132400089 conn(0x558b656cd060 msgr2=0x558b656cc1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:04.303 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.302+0000 7f076206d740 1 -- 172.21.15.26:0/3132400089 shutdown_connections 2024-09-11T23:20:04.304 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.302+0000 7f076206d740 1 -- 172.21.15.26:0/3132400089 wait complete. 2024-09-11T23:20:04.304 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.302+0000 7f076206d740 20 client.25522 trim_cache size 0 max 16384 2024-09-11T23:20:04.304 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-09-11T23:20:04.304 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "message": "", 2024-09-11T23:20:04.304 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-09-11T23:20:04.304 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-09-11T23:20:04.305 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.303+0000 7f075f2d0640 1 -- 172.21.15.26:0/2670677157 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f07440a2da0 msgr2=0x7f07440a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:04.305 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.303+0000 7f075f2d0640 1 --2- 172.21.15.26:0/2670677157 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f07440a2da0 0x7f07440a5260 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f0750005cc0 tx=0x7f0750005c30 comp rx=0 tx=0).stop 2024-09-11T23:20:04.305 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.303+0000 7f075f2d0640 1 -- 172.21.15.26:0/2670677157 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f07581035c0 msgr2=0x7f075819aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:04.305 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.303+0000 7f075f2d0640 1 --2- 172.21.15.26:0/2670677157 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f07581035c0 0x7f075819aa20 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f074800e9a0 tx=0x7f074800ee60 comp rx=0 tx=0).stop 2024-09-11T23:20:04.305 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.303+0000 7f075f2d0640 1 -- 172.21.15.26:0/2670677157 shutdown_connections 2024-09-11T23:20:04.305 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.303+0000 7f075f2d0640 1 --2- 172.21.15.26:0/2670677157 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f07440a2da0 0x7f07440a5260 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:04.305 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.303+0000 7f075f2d0640 1 --2- 172.21.15.26:0/2670677157 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f075810c0d0 0x7f075819eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:04.306 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.303+0000 7f075f2d0640 1 --2- 172.21.15.26:0/2670677157 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f07581035c0 0x7f075819aa20 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:04.306 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.303+0000 7f075f2d0640 1 --2- 172.21.15.26:0/2670677157 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0758102ca0 0x7f075819a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:04.306 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.303+0000 7f075f2d0640 1 -- 172.21.15.26:0/2670677157 >> 172.21.15.26:0/2670677157 conn(0x7f07580fc6b0 msgr2=0x7f075810d340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:04.306 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.303+0000 7f075f2d0640 1 -- 172.21.15.26:0/2670677157 shutdown_connections 2024-09-11T23:20:04.306 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:04.303+0000 7f075f2d0640 1 -- 172.21.15.26:0/2670677157 wait complete. 2024-09-11T23:20:04.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.637+0000 7fbe4e636640 1 -- 172.21.15.26:0/1262569354 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe48106f40 msgr2=0x7fbe4810eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:04.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.637+0000 7fbe4e636640 1 --2- 172.21.15.26:0/1262569354 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe48106f40 0x7fbe4810eac0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fbe40009ca0 tx=0x7fbe4002a8c0 comp rx=0 tx=0).stop 2024-09-11T23:20:04.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.637+0000 7fbe4e636640 1 -- 172.21.15.26:0/1262569354 shutdown_connections 2024-09-11T23:20:04.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.637+0000 7fbe4e636640 1 --2- 172.21.15.26:0/1262569354 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbe481027c0 0x7fbe48106810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:04.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.637+0000 7fbe4e636640 1 --2- 172.21.15.26:0/1262569354 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe48106f40 0x7fbe4810eac0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:04.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.637+0000 7fbe4e636640 1 --2- 172.21.15.26:0/1262569354 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbe48101e10 0x7fbe481021f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:04.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.637+0000 7fbe4e636640 1 -- 172.21.15.26:0/1262569354 >> 172.21.15.26:0/1262569354 conn(0x7fbe480fd8d0 msgr2=0x7fbe480ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:04.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.638+0000 7fbe4e636640 1 -- 172.21.15.26:0/1262569354 shutdown_connections 2024-09-11T23:20:04.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.638+0000 7fbe4e636640 1 -- 172.21.15.26:0/1262569354 wait complete. 2024-09-11T23:20:04.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.638+0000 7fbe4e636640 1 Processor -- start 2024-09-11T23:20:04.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.639+0000 7fbe4e636640 1 -- start start 2024-09-11T23:20:04.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.639+0000 7fbe4e636640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe48101e10 0x7fbe480744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:04.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.639+0000 7fbe4e636640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbe481027c0 0x7fbe48074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:04.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.639+0000 7fbe4e636640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbe48106f40 0x7fbe48074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:04.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.639+0000 7fbe4e636640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fbe48111fa0 con 0x7fbe48101e10 2024-09-11T23:20:04.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.639+0000 7fbe4e636640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fbe48111e20 con 0x7fbe481027c0 2024-09-11T23:20:04.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.639+0000 7fbe4e636640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fbe48112120 con 0x7fbe48106f40 2024-09-11T23:20:04.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.639+0000 7fbe47fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe48101e10 0x7fbe480744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:04.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.639+0000 7fbe477fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbe481027c0 0x7fbe48074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:04.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.639+0000 7fbe4cbd3640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbe48106f40 0x7fbe48074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:04.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.639+0000 7fbe47fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe48101e10 0x7fbe480744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48398/0 (socket says 172.21.15.26:48398) 2024-09-11T23:20:04.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.639+0000 7fbe47fff640 1 -- 172.21.15.26:0/24769772 learned_addr learned my addr 172.21.15.26:0/24769772 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:20:04.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.639+0000 7fbe477fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbe481027c0 0x7fbe48074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:60678/0 (socket says 172.21.15.26:60678) 2024-09-11T23:20:04.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.639+0000 7fbe4cbd3640 1 -- 172.21.15.26:0/24769772 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbe481027c0 msgr2=0x7fbe48074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:04.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.639+0000 7fbe4cbd3640 1 --2- 172.21.15.26:0/24769772 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbe481027c0 0x7fbe48074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:04.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.639+0000 7fbe4cbd3640 1 -- 172.21.15.26:0/24769772 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe48101e10 msgr2=0x7fbe480744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:04.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.640+0000 7fbe4cbd3640 1 --2- 172.21.15.26:0/24769772 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe48101e10 0x7fbe480744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:04.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.640+0000 7fbe4cbd3640 1 -- 172.21.15.26:0/24769772 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe481a68c0 con 0x7fbe48106f40 2024-09-11T23:20:04.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.640+0000 7fbe47fff640 1 --2- 172.21.15.26:0/24769772 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe48101e10 0x7fbe480744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:20:04.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.640+0000 7fbe4cbd3640 1 --2- 172.21.15.26:0/24769772 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbe48106f40 0x7fbe48074f70 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fbe4002ade0 tx=0x7fbe400319f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:04.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.640+0000 7fbe457fa640 1 -- 172.21.15.26:0/24769772 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe40003b70 con 0x7fbe48106f40 2024-09-11T23:20:04.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.640+0000 7fbe457fa640 1 -- 172.21.15.26:0/24769772 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe40003d10 con 0x7fbe48106f40 2024-09-11T23:20:04.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.640+0000 7fbe4e636640 1 -- 172.21.15.26:0/24769772 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe481a6b50 con 0x7fbe48106f40 2024-09-11T23:20:04.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.640+0000 7fbe457fa640 1 -- 172.21.15.26:0/24769772 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe4002fc40 con 0x7fbe48106f40 2024-09-11T23:20:04.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.640+0000 7fbe4e636640 1 -- 172.21.15.26:0/24769772 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe481a7090 con 0x7fbe48106f40 2024-09-11T23:20:04.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.642+0000 7fbe457fa640 1 -- 172.21.15.26:0/24769772 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fbe40004050 con 0x7fbe48106f40 2024-09-11T23:20:04.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.643+0000 7fbe50a0f740 1 Processor -- start 2024-09-11T23:20:04.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.643+0000 7fbe50a0f740 1 -- start start 2024-09-11T23:20:04.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.643+0000 7fbe457fa640 1 --2- 172.21.15.26:0/24769772 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fbe300a2cf0 0x7fbe300a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:04.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.643+0000 7fbe477fe640 1 --2- 172.21.15.26:0/24769772 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fbe300a2cf0 0x7fbe300a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:04.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.643+0000 7fbe457fa640 1 -- 172.21.15.26:0/24769772 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fbe400f0280 con 0x7fbe48106f40 2024-09-11T23:20:04.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.643+0000 7fbe50a0f740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556091784b40 0x556091783090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:04.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.643+0000 7fbe50a0f740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5560917835d0 0x556091783a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:04.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.643+0000 7fbe50a0f740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55609178e370 0x556091790760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:04.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.643+0000 7fbe50a0f740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5560916e92f0 con 0x556091784b40 2024-09-11T23:20:04.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.643+0000 7fbe50a0f740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x556091690cf0 con 0x5560917835d0 2024-09-11T23:20:04.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.643+0000 7fbe50a0f740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5560916e52b0 con 0x55609178e370 2024-09-11T23:20:04.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.643+0000 7fbe47fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556091784b40 0x556091783090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:04.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.643+0000 7fbe47fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556091784b40 0x556091783090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48414/0 (socket says 172.21.15.26:48414) 2024-09-11T23:20:04.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.644+0000 7fbe477fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5560917835d0 0x556091783a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:04.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.644+0000 7fbe47fff640 1 -- 172.21.15.26:0/2886177496 learned_addr learned my addr 172.21.15.26:0/2886177496 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:20:04.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.644+0000 7fbe4cbd3640 1 --2- 172.21.15.26:0/2886177496 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55609178e370 0x556091790760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:04.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.644+0000 7fbe47fff640 1 -- 172.21.15.26:0/2886177496 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55609178e370 msgr2=0x556091790760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:20:04.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.644+0000 7fbe47fff640 1 --2- 172.21.15.26:0/2886177496 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55609178e370 0x556091790760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:04.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.644+0000 7fbe47fff640 1 -- 172.21.15.26:0/2886177496 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5560917835d0 msgr2=0x556091783a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:20:04.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.644+0000 7fbe47fff640 1 --2- 172.21.15.26:0/2886177496 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5560917835d0 0x556091783a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:04.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.644+0000 7fbe47fff640 1 -- 172.21.15.26:0/2886177496 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5560916753a0 con 0x556091784b40 2024-09-11T23:20:04.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.644+0000 7fbe477fe640 1 --2- 172.21.15.26:0/24769772 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fbe300a2cf0 0x7fbe300a51b0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fbe48075f80 tx=0x7fbe3c011810 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:04.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.644+0000 7fbe47fff640 1 --2- 172.21.15.26:0/2886177496 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556091784b40 0x556091783090 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fbe38011650 tx=0x7fbe38009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd20f79faa419581 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:04.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.644+0000 7fbe2e7fc640 1 -- 172.21.15.26:0/2886177496 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe3801e070 con 0x556091784b40 2024-09-11T23:20:04.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.644+0000 7fbe2e7fc640 1 -- 172.21.15.26:0/2886177496 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe3800fb20 con 0x556091784b40 2024-09-11T23:20:04.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.644+0000 7fbe2e7fc640 1 -- 172.21.15.26:0/2886177496 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe38019c90 con 0x556091784b40 2024-09-11T23:20:04.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.645+0000 7fbe50a0f740 1 -- 172.21.15.26:0/2886177496 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556091784b40 msgr2=0x556091783090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:04.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.645+0000 7fbe50a0f740 1 --2- 172.21.15.26:0/2886177496 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556091784b40 0x556091783090 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fbe38011650 tx=0x7fbe38009f90 comp rx=0 tx=0).stop 2024-09-11T23:20:04.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.645+0000 7fbe50a0f740 1 -- 172.21.15.26:0/2886177496 shutdown_connections 2024-09-11T23:20:04.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.645+0000 7fbe50a0f740 1 --2- 172.21.15.26:0/2886177496 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5560917835d0 0x556091783a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:04.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.645+0000 7fbe50a0f740 1 --2- 172.21.15.26:0/2886177496 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55609178e370 0x556091790760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:04.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.645+0000 7fbe50a0f740 1 --2- 172.21.15.26:0/2886177496 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556091784b40 0x556091783090 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:04.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.645+0000 7fbe50a0f740 1 -- 172.21.15.26:0/2886177496 >> 172.21.15.26:0/2886177496 conn(0x55609174c310 msgr2=0x55609177fc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:04.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.645+0000 7fbe50a0f740 1 -- 172.21.15.26:0/2886177496 shutdown_connections 2024-09-11T23:20:04.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.645+0000 7fbe50a0f740 1 -- 172.21.15.26:0/2886177496 wait complete. 2024-09-11T23:20:04.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.645+0000 7fbe50a0f740 1 -- start start 2024-09-11T23:20:04.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.646+0000 7fbe50a0f740 1 Processor -- start 2024-09-11T23:20:04.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.646+0000 7fbe50a0f740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5560917835d0 0x55609178dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:04.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.646+0000 7fbe50a0f740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556091784b40 0x55609178afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:04.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.646+0000 7fbe50a0f740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55609178e370 0x55609178b570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:04.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.646+0000 7fbe50a0f740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x556091791070 con 0x5560917835d0 2024-09-11T23:20:04.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.646+0000 7fbe50a0f740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5560916a4060 con 0x556091784b40 2024-09-11T23:20:04.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.646+0000 7fbe50a0f740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5560917911f0 con 0x55609178e370 2024-09-11T23:20:04.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.646+0000 7fbe50a0f740 10 client.0 _ll_register_callbacks cb 0x5560913f78c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:20:04.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.646+0000 7fbe47fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5560917835d0 0x55609178dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:04.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.646+0000 7fbe47fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5560917835d0 0x55609178dcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48426/0 (socket says 172.21.15.26:48426) 2024-09-11T23:20:04.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.646+0000 7fbe47fff640 1 -- 172.21.15.26:0/3212037303 learned_addr learned my addr 172.21.15.26:0/3212037303 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:20:04.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.646+0000 7fbe4cbd3640 1 --2- 172.21.15.26:0/3212037303 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55609178e370 0x55609178b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:04.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.646+0000 7fbe477fe640 1 --2- 172.21.15.26:0/3212037303 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556091784b40 0x55609178afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:04.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.646+0000 7fbe4cbd3640 1 -- 172.21.15.26:0/3212037303 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556091784b40 msgr2=0x55609178afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:20:04.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.646+0000 7fbe4cbd3640 1 --2- 172.21.15.26:0/3212037303 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556091784b40 0x55609178afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:04.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.646+0000 7fbe4cbd3640 1 -- 172.21.15.26:0/3212037303 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5560917835d0 msgr2=0x55609178dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:20:04.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.646+0000 7fbe4cbd3640 1 --2- 172.21.15.26:0/3212037303 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5560917835d0 0x55609178dcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:04.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.646+0000 7fbe4cbd3640 1 -- 172.21.15.26:0/3212037303 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55609145a3c0 con 0x55609178e370 2024-09-11T23:20:04.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.646+0000 7fbe47fff640 1 --2- 172.21.15.26:0/3212037303 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5560917835d0 0x55609178dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:20:04.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.646+0000 7fbe477fe640 1 --2- 172.21.15.26:0/3212037303 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556091784b40 0x55609178afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:20:04.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.647+0000 7fbe4cbd3640 1 --2- 172.21.15.26:0/3212037303 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55609178e370 0x55609178b570 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fbe400edaf0 tx=0x7fbe400bc4c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e506ce902faf22af server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:04.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.647+0000 7fbe277fe640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-09-11T23:20:04.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.647+0000 7fbe277fe640 1 -- 172.21.15.26:0/3212037303 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe400c0c70 con 0x55609178e370 2024-09-11T23:20:04.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.647+0000 7fbe277fe640 1 -- 172.21.15.26:0/3212037303 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fbe400a5e00 con 0x55609178e370 2024-09-11T23:20:04.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.647+0000 7fbe50a0f740 1 -- 172.21.15.26:0/3212037303 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbe3800fb20 con 0x55609178e370 2024-09-11T23:20:04.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.647+0000 7fbe277fe640 1 -- 172.21.15.26:0/3212037303 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fbe400c5530 con 0x55609178e370 2024-09-11T23:20:04.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.647+0000 7fbe50a0f740 10 client.25534 fetch_fsmap learned FSMap version 31 2024-09-11T23:20:04.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.647+0000 7fbe50a0f740 1 -- 172.21.15.26:0/3212037303 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x556091791070 con 0x55609178e370 2024-09-11T23:20:04.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.647+0000 7fbe277fe640 1 -- 172.21.15.26:0/3212037303 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe40036040 con 0x55609178e370 2024-09-11T23:20:04.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.647+0000 7fbe277fe640 1 -- 172.21.15.26:0/3212037303 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7fbe400b8f30 con 0x55609178e370 2024-09-11T23:20:04.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.647+0000 7fbe50a0f740 10 client.25534 fetch_fsmap finished waiting for FSMap version 31 2024-09-11T23:20:04.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.647+0000 7fbe50a0f740 10 client.25534 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-11T23:20:04.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.647+0000 7fbe50a0f740 20 client.25534 populate_metadata read hostname 'smithi026' 2024-09-11T23:20:04.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.647+0000 7fbe50a0f740 1 --2- 172.21.15.26:0/3212037303 >> [v2:172.21.15.136:6838/3211398630,v1:172.21.15.136:6839/3211398630] conn(0x55609179df40 0x5560917a0380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:04.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.647+0000 7fbe50a0f740 4 client.25534 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:20:04.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.647+0000 7fbe50a0f740 1 -- 172.21.15.26:0/3212037303 --> [v2:172.21.15.136:6838/3211398630,v1:172.21.15.136:6839/3211398630] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fbe3800fb20 con 0x55609179df40 2024-09-11T23:20:04.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.648+0000 7fbe47fff640 1 --2- 172.21.15.26:0/3212037303 >> [v2:172.21.15.136:6838/3211398630,v1:172.21.15.136:6839/3211398630] conn(0x55609179df40 0x5560917a0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:04.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.648+0000 7fbe47fff640 1 --2- 172.21.15.26:0/3212037303 >> [v2:172.21.15.136:6838/3211398630,v1:172.21.15.136:6839/3211398630] conn(0x55609179df40 0x5560917a0380 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=43d23116a9e2002b server_cookie=508e224b04a3a3e8 in_seq=0 out_seq=0 2024-09-11T23:20:04.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.648+0000 7fbe277fe640 10 client.25534 ms_handle_connect on v2:172.21.15.136:6838/3211398630 2024-09-11T23:20:04.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.650+0000 7fbe277fe640 1 -- 172.21.15.26:0/3212037303 <== mds.1 v2:172.21.15.136:6838/3211398630 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7fbe3800fb20 con 0x55609179df40 2024-09-11T23:20:04.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.650+0000 7fbe277fe640 10 client.25534 handle_command_reply: tid=0 2024-09-11T23:20:04.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.650+0000 7fbe50a0f740 1 client.25534 shutdown 2024-09-11T23:20:04.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.650+0000 7fbe50a0f740 1 -- 172.21.15.26:0/3212037303 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55609178e370 msgr2=0x55609178b570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:04.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.650+0000 7fbe50a0f740 1 --2- 172.21.15.26:0/3212037303 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55609178e370 0x55609178b570 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fbe400edaf0 tx=0x7fbe400bc4c0 comp rx=0 tx=0).stop 2024-09-11T23:20:04.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.650+0000 7fbe50a0f740 1 -- 172.21.15.26:0/3212037303 shutdown_connections 2024-09-11T23:20:04.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.650+0000 7fbe50a0f740 1 --2- 172.21.15.26:0/3212037303 >> [v2:172.21.15.136:6838/3211398630,v1:172.21.15.136:6839/3211398630] conn(0x55609179df40 0x5560917a0380 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:04.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.650+0000 7fbe50a0f740 1 --2- 172.21.15.26:0/3212037303 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556091784b40 0x55609178afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:04.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.650+0000 7fbe277fe640 0 client.25534 ms_handle_reset on v2:172.21.15.136:6838/3211398630 2024-09-11T23:20:04.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.650+0000 7fbe50a0f740 1 --2- 172.21.15.26:0/3212037303 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55609178e370 0x55609178b570 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:04.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.650+0000 7fbe50a0f740 1 --2- 172.21.15.26:0/3212037303 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5560917835d0 0x55609178dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:04.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.650+0000 7fbe50a0f740 1 -- 172.21.15.26:0/3212037303 >> 172.21.15.26:0/3212037303 conn(0x55609174c310 msgr2=0x556091785960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:04.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.650+0000 7fbe50a0f740 1 -- 172.21.15.26:0/3212037303 shutdown_connections 2024-09-11T23:20:04.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.650+0000 7fbe50a0f740 1 -- 172.21.15.26:0/3212037303 wait complete. 2024-09-11T23:20:04.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.650+0000 7fbe50a0f740 20 client.25534 trim_cache size 0 max 16384 2024-09-11T23:20:04.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.666+0000 7fbe50a0f740 1 Processor -- start 2024-09-11T23:20:04.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.666+0000 7fbe50a0f740 1 -- start start 2024-09-11T23:20:04.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.666+0000 7fbe50a0f740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5560917835d0 0x55609179a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:04.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.666+0000 7fbe50a0f740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556091784b40 0x5560917a5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:04.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.666+0000 7fbe50a0f740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55609178e370 0x5560917a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:04.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.666+0000 7fbe50a0f740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fbe481afa20 con 0x55609178e370 2024-09-11T23:20:04.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.666+0000 7fbe50a0f740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fbe481b1650 con 0x5560917835d0 2024-09-11T23:20:04.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.666+0000 7fbe50a0f740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fbe481b00f0 con 0x556091784b40 2024-09-11T23:20:04.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.666+0000 7fbe47fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5560917835d0 0x55609179a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:04.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.666+0000 7fbe477fe640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556091784b40 0x5560917a5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:04.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.666+0000 7fbe4cbd3640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55609178e370 0x5560917a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:04.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.666+0000 7fbe47fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5560917835d0 0x55609179a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:60708/0 (socket says 172.21.15.26:60708) 2024-09-11T23:20:04.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.666+0000 7fbe47fff640 1 -- 172.21.15.26:0/4083871232 learned_addr learned my addr 172.21.15.26:0/4083871232 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:20:04.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.666+0000 7fbe477fe640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556091784b40 0x5560917a5c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:54654/0 (socket says 172.21.15.26:54654) 2024-09-11T23:20:04.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.667+0000 7fbe47fff640 1 -- 172.21.15.26:0/4083871232 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556091784b40 msgr2=0x5560917a5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:20:04.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.667+0000 7fbe47fff640 1 --2- 172.21.15.26:0/4083871232 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556091784b40 0x5560917a5c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:04.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.667+0000 7fbe47fff640 1 -- 172.21.15.26:0/4083871232 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55609178e370 msgr2=0x5560917a61d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:20:04.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.667+0000 7fbe47fff640 1 --2- 172.21.15.26:0/4083871232 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55609178e370 0x5560917a61d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:04.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.667+0000 7fbe47fff640 1 -- 172.21.15.26:0/4083871232 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556091780730 con 0x5560917835d0 2024-09-11T23:20:04.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.667+0000 7fbe4cbd3640 1 --2- 172.21.15.26:0/4083871232 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55609178e370 0x5560917a61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:20:04.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.667+0000 7fbe47fff640 1 --2- 172.21.15.26:0/4083871232 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5560917835d0 0x55609179a310 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fbe380197a0 tx=0x7fbe380198e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=823ba71eabc53b72 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:04.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.667+0000 7fbe2dffb640 1 -- 172.21.15.26:0/4083871232 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe3801e040 con 0x5560917835d0 2024-09-11T23:20:04.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.667+0000 7fbe2dffb640 1 -- 172.21.15.26:0/4083871232 <== mon.2 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe380199a0 con 0x5560917835d0 2024-09-11T23:20:04.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.667+0000 7fbe2dffb640 1 -- 172.21.15.26:0/4083871232 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe38005350 con 0x5560917835d0 2024-09-11T23:20:04.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.668+0000 7fbe50a0f740 1 -- 172.21.15.26:0/4083871232 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5560917835d0 msgr2=0x55609179a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:04.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.668+0000 7fbe50a0f740 1 --2- 172.21.15.26:0/4083871232 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5560917835d0 0x55609179a310 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fbe380197a0 tx=0x7fbe380198e0 comp rx=0 tx=0).stop 2024-09-11T23:20:04.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.668+0000 7fbe50a0f740 1 -- 172.21.15.26:0/4083871232 shutdown_connections 2024-09-11T23:20:04.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.668+0000 7fbe50a0f740 1 --2- 172.21.15.26:0/4083871232 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55609178e370 0x5560917a61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:04.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.668+0000 7fbe50a0f740 1 --2- 172.21.15.26:0/4083871232 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556091784b40 0x5560917a5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:04.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.668+0000 7fbe50a0f740 1 --2- 172.21.15.26:0/4083871232 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5560917835d0 0x55609179a310 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:04.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.668+0000 7fbe50a0f740 1 -- 172.21.15.26:0/4083871232 >> 172.21.15.26:0/4083871232 conn(0x55609174c310 msgr2=0x55609178af90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:04.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.668+0000 7fbe50a0f740 1 -- 172.21.15.26:0/4083871232 shutdown_connections 2024-09-11T23:20:04.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.668+0000 7fbe50a0f740 1 -- 172.21.15.26:0/4083871232 wait complete. 2024-09-11T23:20:04.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.668+0000 7fbe50a0f740 1 -- start start 2024-09-11T23:20:04.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.669+0000 7fbe50a0f740 1 Processor -- start 2024-09-11T23:20:04.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.669+0000 7fbe50a0f740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5560917835d0 0x556091792420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:04.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.669+0000 7fbe50a0f740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556091784b40 0x556091792960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:04.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.669+0000 7fbe50a0f740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55609178e370 0x556091792ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:04.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.669+0000 7fbe50a0f740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fbe481b0ec0 con 0x55609178e370 2024-09-11T23:20:04.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.669+0000 7fbe50a0f740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fbe481b07c0 con 0x5560917835d0 2024-09-11T23:20:04.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.669+0000 7fbe50a0f740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fbe481ada00 con 0x556091784b40 2024-09-11T23:20:04.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.669+0000 7fbe50a0f740 10 client.0 _ll_register_callbacks cb 0x7fbe40009990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:20:04.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.669+0000 7fbe47fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5560917835d0 0x556091792420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:04.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.669+0000 7fbe47fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5560917835d0 0x556091792420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:60718/0 (socket says 172.21.15.26:60718) 2024-09-11T23:20:04.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.669+0000 7fbe47fff640 1 -- 172.21.15.26:0/77236408 learned_addr learned my addr 172.21.15.26:0/77236408 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:20:04.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.669+0000 7fbe477fe640 1 --2- 172.21.15.26:0/77236408 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556091784b40 0x556091792960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:04.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.669+0000 7fbe4cbd3640 1 --2- 172.21.15.26:0/77236408 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55609178e370 0x556091792ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:04.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.669+0000 7fbe47fff640 1 -- 172.21.15.26:0/77236408 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556091784b40 msgr2=0x556091792960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:20:04.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.669+0000 7fbe47fff640 1 --2- 172.21.15.26:0/77236408 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556091784b40 0x556091792960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:04.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.669+0000 7fbe47fff640 1 -- 172.21.15.26:0/77236408 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55609178e370 msgr2=0x556091792ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:20:04.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.669+0000 7fbe47fff640 1 --2- 172.21.15.26:0/77236408 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55609178e370 0x556091792ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:04.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.669+0000 7fbe47fff640 1 -- 172.21.15.26:0/77236408 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556091807eb0 con 0x5560917835d0 2024-09-11T23:20:04.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.670+0000 7fbe47fff640 1 --2- 172.21.15.26:0/77236408 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5560917835d0 0x556091792420 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fbe38019970 tx=0x7fbe380050b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4a3d8c81f4b27654 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:04.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.670+0000 7fbe277fe640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-11T23:20:04.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.670+0000 7fbe277fe640 1 -- 172.21.15.26:0/77236408 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe3801e040 con 0x5560917835d0 2024-09-11T23:20:04.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.670+0000 7fbe277fe640 1 -- 172.21.15.26:0/77236408 <== mon.2 v2:172.21.15.76:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fbe3802a070 con 0x5560917835d0 2024-09-11T23:20:04.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.670+0000 7fbe50a0f740 1 -- 172.21.15.26:0/77236408 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbe380199a0 con 0x5560917835d0 2024-09-11T23:20:04.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.670+0000 7fbe277fe640 1 -- 172.21.15.26:0/77236408 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fbe38019b50 con 0x5560917835d0 2024-09-11T23:20:04.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.670+0000 7fbe50a0f740 10 client.25490 fetch_fsmap learned FSMap version 31 2024-09-11T23:20:04.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.670+0000 7fbe50a0f740 1 -- 172.21.15.26:0/77236408 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x556091809190 con 0x5560917835d0 2024-09-11T23:20:04.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.670+0000 7fbe277fe640 1 -- 172.21.15.26:0/77236408 <== mon.2 v2:172.21.15.76:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe380199a0 con 0x5560917835d0 2024-09-11T23:20:04.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.670+0000 7fbe277fe640 1 -- 172.21.15.26:0/77236408 <== mon.2 v2:172.21.15.76:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7fbe380035f0 con 0x5560917835d0 2024-09-11T23:20:04.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.670+0000 7fbe50a0f740 10 client.25490 fetch_fsmap finished waiting for FSMap version 31 2024-09-11T23:20:04.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.670+0000 7fbe50a0f740 10 client.25490 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-11T23:20:04.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.671+0000 7fbe50a0f740 20 client.25490 populate_metadata read hostname 'smithi026' 2024-09-11T23:20:04.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.671+0000 7fbe50a0f740 1 --2- 172.21.15.26:0/77236408 >> [v2:172.21.15.136:6838/3211398630,v1:172.21.15.136:6839/3211398630] conn(0x556091809710 0x55609180bb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:04.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.671+0000 7fbe50a0f740 4 client.25490 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:20:04.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.671+0000 7fbe50a0f740 1 -- 172.21.15.26:0/77236408 --> [v2:172.21.15.136:6838/3211398630,v1:172.21.15.136:6839/3211398630] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5560916f6860 con 0x556091809710 2024-09-11T23:20:04.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.671+0000 7fbe477fe640 1 --2- 172.21.15.26:0/77236408 >> [v2:172.21.15.136:6838/3211398630,v1:172.21.15.136:6839/3211398630] conn(0x556091809710 0x55609180bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:04.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.671+0000 7fbe277fe640 10 client.25490 ms_handle_connect on v2:172.21.15.136:6838/3211398630 2024-09-11T23:20:04.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.671+0000 7fbe477fe640 1 --2- 172.21.15.26:0/77236408 >> [v2:172.21.15.136:6838/3211398630,v1:172.21.15.136:6839/3211398630] conn(0x556091809710 0x55609180bb00 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f3ab412591dad16b server_cookie=697000f8cae125ba in_seq=0 out_seq=0 2024-09-11T23:20:04.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.674+0000 7fbe277fe640 1 -- 172.21.15.26:0/77236408 <== mds.1 v2:172.21.15.136:6838/3211398630 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5560916f6860 con 0x556091809710 2024-09-11T23:20:04.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.674+0000 7fbe277fe640 10 client.25490 handle_command_reply: tid=0 2024-09-11T23:20:04.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.674+0000 7fbe50a0f740 1 client.25490 shutdown 2024-09-11T23:20:04.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.674+0000 7fbe50a0f740 1 -- 172.21.15.26:0/77236408 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5560917835d0 msgr2=0x556091792420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:04.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.674+0000 7fbe50a0f740 1 --2- 172.21.15.26:0/77236408 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5560917835d0 0x556091792420 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fbe38019970 tx=0x7fbe380050b0 comp rx=0 tx=0).stop 2024-09-11T23:20:04.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.674+0000 7fbe50a0f740 1 -- 172.21.15.26:0/77236408 shutdown_connections 2024-09-11T23:20:04.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.674+0000 7fbe50a0f740 1 --2- 172.21.15.26:0/77236408 >> [v2:172.21.15.136:6838/3211398630,v1:172.21.15.136:6839/3211398630] conn(0x556091809710 0x55609180bb00 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:04.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.674+0000 7fbe50a0f740 1 --2- 172.21.15.26:0/77236408 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55609178e370 0x556091792ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:04.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.674+0000 7fbe50a0f740 1 --2- 172.21.15.26:0/77236408 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556091784b40 0x556091792960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:04.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.674+0000 7fbe50a0f740 1 --2- 172.21.15.26:0/77236408 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5560917835d0 0x556091792420 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:04.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.674+0000 7fbe277fe640 0 client.25490 ms_handle_reset on v2:172.21.15.136:6838/3211398630 2024-09-11T23:20:04.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.674+0000 7fbe50a0f740 1 -- 172.21.15.26:0/77236408 >> 172.21.15.26:0/77236408 conn(0x55609174c310 msgr2=0x55609177e4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:04.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.674+0000 7fbe50a0f740 1 -- 172.21.15.26:0/77236408 shutdown_connections 2024-09-11T23:20:04.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.674+0000 7fbe50a0f740 1 -- 172.21.15.26:0/77236408 wait complete. 2024-09-11T23:20:04.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.674+0000 7fbe50a0f740 20 client.25490 trim_cache size 0 max 16384 2024-09-11T23:20:04.683 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2024-09-11T23:20:04.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.675+0000 7fbe4e636640 1 -- 172.21.15.26:0/24769772 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fbe300a2cf0 msgr2=0x7fbe300a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:04.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.675+0000 7fbe4e636640 1 --2- 172.21.15.26:0/24769772 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fbe300a2cf0 0x7fbe300a51b0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fbe48075f80 tx=0x7fbe3c011810 comp rx=0 tx=0).stop 2024-09-11T23:20:04.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.675+0000 7fbe4e636640 1 -- 172.21.15.26:0/24769772 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbe48106f40 msgr2=0x7fbe48074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:04.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.675+0000 7fbe4e636640 1 --2- 172.21.15.26:0/24769772 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbe48106f40 0x7fbe48074f70 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fbe4002ade0 tx=0x7fbe400319f0 comp rx=0 tx=0).stop 2024-09-11T23:20:04.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.675+0000 7fbe4e636640 1 -- 172.21.15.26:0/24769772 shutdown_connections 2024-09-11T23:20:04.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.675+0000 7fbe4e636640 1 --2- 172.21.15.26:0/24769772 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fbe300a2cf0 0x7fbe300a51b0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:04.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.675+0000 7fbe4e636640 1 --2- 172.21.15.26:0/24769772 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbe481027c0 0x7fbe48074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:04.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.675+0000 7fbe4e636640 1 --2- 172.21.15.26:0/24769772 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbe48106f40 0x7fbe48074f70 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:04.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.675+0000 7fbe4e636640 1 --2- 172.21.15.26:0/24769772 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe48101e10 0x7fbe480744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:04.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.675+0000 7fbe4e636640 1 -- 172.21.15.26:0/24769772 >> 172.21.15.26:0/24769772 conn(0x7fbe480fd8d0 msgr2=0x7fbe480fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:04.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.675+0000 7fbe4e636640 1 -- 172.21.15.26:0/24769772 shutdown_connections 2024-09-11T23:20:04.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:04.675+0000 7fbe4e636640 1 -- 172.21.15.26:0/24769772 wait complete. 2024-09-11T23:20:05.413 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-11T23:20:05.553 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:05 smithi076 ceph-mon[35296]: pgmap v54: 97 pgs: 97 active+clean; 57 MiB data, 563 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.9 MiB/s wr, 328 op/s 2024-09-11T23:20:05.694 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:20:05.729 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:05 smithi026 ceph-mon[31172]: pgmap v54: 97 pgs: 97 active+clean; 57 MiB data, 563 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.9 MiB/s wr, 328 op/s 2024-09-11T23:20:05.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:05 smithi136 ceph-mon[35121]: pgmap v54: 97 pgs: 97 active+clean; 57 MiB data, 563 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.9 MiB/s wr, 328 op/s 2024-09-11T23:20:06.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.178+0000 7ff84e860640 1 -- 172.21.15.26:0/1666142145 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff848109110 msgr2=0x7ff848110c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:06.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.178+0000 7ff84e860640 1 --2- 172.21.15.26:0/1666142145 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff848109110 0x7ff848110c90 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7ff840009ca0 tx=0x7ff84002a8c0 comp rx=0 tx=0).stop 2024-09-11T23:20:06.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.178+0000 7ff84e860640 1 -- 172.21.15.26:0/1666142145 shutdown_connections 2024-09-11T23:20:06.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.178+0000 7ff84e860640 1 --2- 172.21.15.26:0/1666142145 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff848109110 0x7ff848110c90 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:06.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.178+0000 7ff84e860640 1 --2- 172.21.15.26:0/1666142145 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff848104990 0x7ff8481089e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:06.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.178+0000 7ff84e860640 1 --2- 172.21.15.26:0/1666142145 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff848103fe0 0x7ff8481043c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:06.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.178+0000 7ff84e860640 1 -- 172.21.15.26:0/1666142145 >> 172.21.15.26:0/1666142145 conn(0x7ff8480ffa20 msgr2=0x7ff848101e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:06.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.178+0000 7ff84e860640 1 -- 172.21.15.26:0/1666142145 shutdown_connections 2024-09-11T23:20:06.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.178+0000 7ff84e860640 1 -- 172.21.15.26:0/1666142145 wait complete. 2024-09-11T23:20:06.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.178+0000 7ff84e860640 1 Processor -- start 2024-09-11T23:20:06.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.178+0000 7ff84e860640 1 -- start start 2024-09-11T23:20:06.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.179+0000 7ff84e860640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff848103fe0 0x7ff84819a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:06.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.179+0000 7ff84e860640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff848104990 0x7ff84819ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:06.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.179+0000 7ff84e860640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff848109110 0x7ff84819eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:06.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.179+0000 7ff84e860640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff848114140 con 0x7ff848103fe0 2024-09-11T23:20:06.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.179+0000 7ff84e860640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7ff848113fc0 con 0x7ff848104990 2024-09-11T23:20:06.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.179+0000 7ff84e860640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ff8481142c0 con 0x7ff848109110 2024-09-11T23:20:06.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.179+0000 7ff847fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff848103fe0 0x7ff84819a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:06.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.179+0000 7ff8477fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff848104990 0x7ff84819ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:06.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.179+0000 7ff847fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff848103fe0 0x7ff84819a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48462/0 (socket says 172.21.15.26:48462) 2024-09-11T23:20:06.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.179+0000 7ff847fff640 1 -- 172.21.15.26:0/239833266 learned_addr learned my addr 172.21.15.26:0/239833266 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:20:06.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.179+0000 7ff84cdfd640 1 --2- 172.21.15.26:0/239833266 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff848109110 0x7ff84819eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:06.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.179+0000 7ff8477fe640 1 -- 172.21.15.26:0/239833266 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff848109110 msgr2=0x7ff84819eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:06.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.179+0000 7ff8477fe640 1 --2- 172.21.15.26:0/239833266 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff848109110 0x7ff84819eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:06.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.179+0000 7ff8477fe640 1 -- 172.21.15.26:0/239833266 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff848103fe0 msgr2=0x7ff84819a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:06.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.179+0000 7ff8477fe640 1 --2- 172.21.15.26:0/239833266 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff848103fe0 0x7ff84819a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:06.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.179+0000 7ff8477fe640 1 -- 172.21.15.26:0/239833266 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff84819f5b0 con 0x7ff848104990 2024-09-11T23:20:06.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.180+0000 7ff8477fe640 1 --2- 172.21.15.26:0/239833266 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff848104990 0x7ff84819ab40 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff83c00e9f0 tx=0x7ff83c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:06.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.180+0000 7ff8457fa640 1 -- 172.21.15.26:0/239833266 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff83c00ccc0 con 0x7ff848104990 2024-09-11T23:20:06.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.180+0000 7ff84e860640 1 -- 172.21.15.26:0/239833266 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff84819f8a0 con 0x7ff848104990 2024-09-11T23:20:06.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.180+0000 7ff8457fa640 1 -- 172.21.15.26:0/239833266 <== mon.2 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff83c005020 con 0x7ff848104990 2024-09-11T23:20:06.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.180+0000 7ff8457fa640 1 -- 172.21.15.26:0/239833266 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff83c0049e0 con 0x7ff848104990 2024-09-11T23:20:06.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.180+0000 7ff84e860640 1 -- 172.21.15.26:0/239833266 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8481a7180 con 0x7ff848104990 2024-09-11T23:20:06.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.182+0000 7ff8457fa640 1 -- 172.21.15.26:0/239833266 <== mon.2 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7ff83c005560 con 0x7ff848104990 2024-09-11T23:20:06.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.183+0000 7ff850c39740 1 Processor -- start 2024-09-11T23:20:06.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.183+0000 7ff850c39740 1 -- start start 2024-09-11T23:20:06.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.183+0000 7ff8457fa640 1 --2- 172.21.15.26:0/239833266 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7ff8300a2cf0 0x7ff8300a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:06.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.183+0000 7ff850c39740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b9064fbb40 0x55b9064fa090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:06.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.183+0000 7ff850c39740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b9064fa5d0 0x55b9064faa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:06.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.183+0000 7ff850c39740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b906505370 0x55b906507760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:06.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.183+0000 7ff850c39740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55b9064602f0 con 0x55b9064fbb40 2024-09-11T23:20:06.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.183+0000 7ff850c39740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55b906407cf0 con 0x55b906505370 2024-09-11T23:20:06.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.183+0000 7ff850c39740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55b90645c2b0 con 0x55b9064fa5d0 2024-09-11T23:20:06.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.183+0000 7ff847fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b9064fbb40 0x55b9064fa090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:06.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.183+0000 7ff847fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b9064fbb40 0x55b9064fa090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48464/0 (socket says 172.21.15.26:48464) 2024-09-11T23:20:06.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.183+0000 7ff8457fa640 1 -- 172.21.15.26:0/239833266 <== mon.2 v2:172.21.15.76:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7ff83c0cf390 con 0x7ff848104990 2024-09-11T23:20:06.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.183+0000 7ff847fff640 1 -- 172.21.15.26:0/3014035758 learned_addr learned my addr 172.21.15.26:0/3014035758 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:20:06.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.183+0000 7ff84cdfd640 1 --2- 172.21.15.26:0/239833266 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7ff8300a2cf0 0x7ff8300a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:06.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.183+0000 7ff84cdfd640 1 --2- 172.21.15.26:0/3014035758 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b906505370 0x55b906507760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:06.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.183+0000 7ff8477fe640 1 --2- 172.21.15.26:0/3014035758 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b9064fa5d0 0x55b9064faa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:06.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.184+0000 7ff847fff640 1 -- 172.21.15.26:0/3014035758 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b9064fa5d0 msgr2=0x55b9064faa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:20:06.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.184+0000 7ff847fff640 1 --2- 172.21.15.26:0/3014035758 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b9064fa5d0 0x55b9064faa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:06.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.184+0000 7ff847fff640 1 -- 172.21.15.26:0/3014035758 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b906505370 msgr2=0x55b906507760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:20:06.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.184+0000 7ff847fff640 1 --2- 172.21.15.26:0/3014035758 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b906505370 0x55b906507760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:06.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.184+0000 7ff847fff640 1 -- 172.21.15.26:0/3014035758 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b9063ec3a0 con 0x55b9064fbb40 2024-09-11T23:20:06.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.184+0000 7ff847fff640 1 --2- 172.21.15.26:0/3014035758 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b9064fbb40 0x55b9064fa090 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7ff838009dc0 tx=0x7ff838011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a403626324619ae server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:06.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.184+0000 7ff84cdfd640 1 --2- 172.21.15.26:0/239833266 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7ff8300a2cf0 0x7ff8300a51b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff840005fe0 tx=0x7ff84002a8c0 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:06.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.184+0000 7ff82e7fc640 1 -- 172.21.15.26:0/3014035758 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff838005220 con 0x55b9064fbb40 2024-09-11T23:20:06.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.184+0000 7ff82e7fc640 1 -- 172.21.15.26:0/3014035758 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff838004c80 con 0x55b9064fbb40 2024-09-11T23:20:06.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.184+0000 7ff82e7fc640 1 -- 172.21.15.26:0/3014035758 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8380198d0 con 0x55b9064fbb40 2024-09-11T23:20:06.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.185+0000 7ff850c39740 1 -- 172.21.15.26:0/3014035758 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b9064fbb40 msgr2=0x55b9064fa090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:06.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.185+0000 7ff850c39740 1 --2- 172.21.15.26:0/3014035758 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b9064fbb40 0x55b9064fa090 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7ff838009dc0 tx=0x7ff838011810 comp rx=0 tx=0).stop 2024-09-11T23:20:06.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.185+0000 7ff850c39740 1 -- 172.21.15.26:0/3014035758 shutdown_connections 2024-09-11T23:20:06.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.185+0000 7ff850c39740 1 --2- 172.21.15.26:0/3014035758 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b906505370 0x55b906507760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:06.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.185+0000 7ff850c39740 1 --2- 172.21.15.26:0/3014035758 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b9064fa5d0 0x55b9064faa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:06.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.185+0000 7ff850c39740 1 --2- 172.21.15.26:0/3014035758 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b9064fbb40 0x55b9064fa090 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:06.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.185+0000 7ff850c39740 1 -- 172.21.15.26:0/3014035758 >> 172.21.15.26:0/3014035758 conn(0x55b9064c3310 msgr2=0x55b9064f6c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:06.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.185+0000 7ff850c39740 1 -- 172.21.15.26:0/3014035758 shutdown_connections 2024-09-11T23:20:06.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.185+0000 7ff850c39740 1 -- 172.21.15.26:0/3014035758 wait complete. 2024-09-11T23:20:06.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.185+0000 7ff850c39740 1 -- start start 2024-09-11T23:20:06.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.186+0000 7ff850c39740 1 Processor -- start 2024-09-11T23:20:06.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.186+0000 7ff850c39740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b9064fa5d0 0x55b906504cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:06.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.186+0000 7ff850c39740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b9064fbb40 0x55b906501f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:06.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.186+0000 7ff850c39740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b906505370 0x55b9065024d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:06.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.186+0000 7ff850c39740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55b906508070 con 0x55b9064fa5d0 2024-09-11T23:20:06.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.186+0000 7ff850c39740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55b90641b060 con 0x55b9064fbb40 2024-09-11T23:20:06.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.186+0000 7ff850c39740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55b9065081f0 con 0x55b906505370 2024-09-11T23:20:06.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.186+0000 7ff850c39740 10 client.0 _ll_register_callbacks cb 0x55b90616e8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:20:06.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.186+0000 7ff847fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b9064fa5d0 0x55b906504cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:06.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.186+0000 7ff847fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b9064fa5d0 0x55b906504cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48472/0 (socket says 172.21.15.26:48472) 2024-09-11T23:20:06.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.186+0000 7ff847fff640 1 -- 172.21.15.26:0/2117630665 learned_addr learned my addr 172.21.15.26:0/2117630665 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:20:06.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.186+0000 7ff84cdfd640 1 --2- 172.21.15.26:0/2117630665 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b906505370 0x55b9065024d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:06.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.186+0000 7ff8477fe640 1 --2- 172.21.15.26:0/2117630665 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b9064fbb40 0x55b906501f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:06.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.186+0000 7ff847fff640 1 -- 172.21.15.26:0/2117630665 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b906505370 msgr2=0x55b9065024d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:20:06.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.186+0000 7ff847fff640 1 --2- 172.21.15.26:0/2117630665 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b906505370 0x55b9065024d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:06.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.186+0000 7ff847fff640 1 -- 172.21.15.26:0/2117630665 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b9064fbb40 msgr2=0x55b906501f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:20:06.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.186+0000 7ff847fff640 1 --2- 172.21.15.26:0/2117630665 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b9064fbb40 0x55b906501f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:06.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.186+0000 7ff847fff640 1 -- 172.21.15.26:0/2117630665 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b9061d13c0 con 0x55b9064fa5d0 2024-09-11T23:20:06.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.187+0000 7ff84cdfd640 1 --2- 172.21.15.26:0/2117630665 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b906505370 0x55b9065024d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:20:06.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.187+0000 7ff8477fe640 1 --2- 172.21.15.26:0/2117630665 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b9064fbb40 0x55b906501f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:20:06.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.187+0000 7ff847fff640 1 --2- 172.21.15.26:0/2117630665 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b9064fa5d0 0x55b906504cc0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7ff83800a010 tx=0x7ff8380042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b7dfe7f8ba77f44 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:06.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.187+0000 7ff8277fe640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:20:06.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.187+0000 7ff8277fe640 1 -- 172.21.15.26:0/2117630665 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff838027070 con 0x55b9064fa5d0 2024-09-11T23:20:06.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.187+0000 7ff8277fe640 1 -- 172.21.15.26:0/2117630665 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7ff838005240 con 0x55b9064fa5d0 2024-09-11T23:20:06.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.187+0000 7ff850c39740 1 -- 172.21.15.26:0/2117630665 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff838004c80 con 0x55b9064fa5d0 2024-09-11T23:20:06.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.187+0000 7ff8277fe640 1 -- 172.21.15.26:0/2117630665 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7ff8380099d0 con 0x55b9064fa5d0 2024-09-11T23:20:06.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.187+0000 7ff850c39740 10 client.15918 fetch_fsmap learned FSMap version 31 2024-09-11T23:20:06.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.187+0000 7ff850c39740 1 -- 172.21.15.26:0/2117630665 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55b906513a80 con 0x55b9064fa5d0 2024-09-11T23:20:06.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.187+0000 7ff8277fe640 1 -- 172.21.15.26:0/2117630665 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff838004c80 con 0x55b9064fa5d0 2024-09-11T23:20:06.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.188+0000 7ff8277fe640 1 -- 172.21.15.26:0/2117630665 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7ff8380195d0 con 0x55b9064fa5d0 2024-09-11T23:20:06.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.188+0000 7ff850c39740 10 client.15918 fetch_fsmap finished waiting for FSMap version 31 2024-09-11T23:20:06.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.188+0000 7ff850c39740 10 client.15918 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:20:06.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.188+0000 7ff850c39740 20 client.15918 populate_metadata read hostname 'smithi026' 2024-09-11T23:20:06.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.188+0000 7ff850c39740 1 --2- 172.21.15.26:0/2117630665 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55b906514e30 0x55b906517220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:06.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.188+0000 7ff850c39740 4 client.15918 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:20:06.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.188+0000 7ff850c39740 1 -- 172.21.15.26:0/2117630665 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b90646d860 con 0x55b906514e30 2024-09-11T23:20:06.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.188+0000 7ff8477fe640 1 --2- 172.21.15.26:0/2117630665 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55b906514e30 0x55b906517220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:06.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.188+0000 7ff8277fe640 10 client.15918 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:20:06.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.188+0000 7ff8477fe640 1 --2- 172.21.15.26:0/2117630665 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55b906514e30 0x55b906517220 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa7ae3c92a186f29 server_cookie=1480a58ec1e85f05 in_seq=0 out_seq=0 2024-09-11T23:20:06.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.189+0000 7ff8277fe640 1 -- 172.21.15.26:0/2117630665 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b90646d860 con 0x55b906514e30 2024-09-11T23:20:06.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.189+0000 7ff8277fe640 10 client.15918 handle_command_reply: tid=0 2024-09-11T23:20:06.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.189+0000 7ff850c39740 1 client.15918 shutdown 2024-09-11T23:20:06.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.189+0000 7ff850c39740 1 -- 172.21.15.26:0/2117630665 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b9064fa5d0 msgr2=0x55b906504cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:06.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.189+0000 7ff850c39740 1 --2- 172.21.15.26:0/2117630665 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b9064fa5d0 0x55b906504cc0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7ff83800a010 tx=0x7ff8380042f0 comp rx=0 tx=0).stop 2024-09-11T23:20:06.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.189+0000 7ff850c39740 1 -- 172.21.15.26:0/2117630665 shutdown_connections 2024-09-11T23:20:06.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.189+0000 7ff850c39740 1 --2- 172.21.15.26:0/2117630665 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b9064fbb40 0x55b906501f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:06.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.189+0000 7ff850c39740 1 --2- 172.21.15.26:0/2117630665 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55b906514e30 0x55b906517220 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:06.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.189+0000 7ff850c39740 1 --2- 172.21.15.26:0/2117630665 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b906505370 0x55b9065024d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:06.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.189+0000 7ff850c39740 1 --2- 172.21.15.26:0/2117630665 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b9064fa5d0 0x55b906504cc0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:06.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.189+0000 7ff8277fe640 0 client.15918 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:20:06.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.189+0000 7ff850c39740 1 -- 172.21.15.26:0/2117630665 >> 172.21.15.26:0/2117630665 conn(0x55b9064c3310 msgr2=0x55b9064fc960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:06.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.189+0000 7ff850c39740 1 -- 172.21.15.26:0/2117630665 shutdown_connections 2024-09-11T23:20:06.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.190+0000 7ff850c39740 1 -- 172.21.15.26:0/2117630665 wait complete. 2024-09-11T23:20:06.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.190+0000 7ff850c39740 20 client.15918 trim_cache size 0 max 16384 2024-09-11T23:20:06.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.204+0000 7ff850c39740 1 Processor -- start 2024-09-11T23:20:06.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.204+0000 7ff850c39740 1 -- start start 2024-09-11T23:20:06.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.204+0000 7ff850c39740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b9064fa5d0 0x55b906501720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:06.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.204+0000 7ff850c39740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b9064fbb40 0x55b9064f5c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:06.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.204+0000 7ff850c39740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b906505370 0x55b9064f6150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:06.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.204+0000 7ff850c39740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff8481afae0 con 0x55b906505370 2024-09-11T23:20:06.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.204+0000 7ff850c39740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7ff8481b1710 con 0x55b9064fbb40 2024-09-11T23:20:06.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.204+0000 7ff850c39740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ff8481b01b0 con 0x55b9064fa5d0 2024-09-11T23:20:06.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.204+0000 7ff84cdfd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b906505370 0x55b9064f6150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:06.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.204+0000 7ff84cdfd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b906505370 0x55b9064f6150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48484/0 (socket says 172.21.15.26:48484) 2024-09-11T23:20:06.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.204+0000 7ff84cdfd640 1 -- 172.21.15.26:0/824151701 learned_addr learned my addr 172.21.15.26:0/824151701 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:20:06.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.204+0000 7ff847fff640 1 --2- 172.21.15.26:0/824151701 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b9064fa5d0 0x55b906501720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:06.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.204+0000 7ff8477fe640 1 --2- 172.21.15.26:0/824151701 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b9064fbb40 0x55b9064f5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:06.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.204+0000 7ff8477fe640 1 -- 172.21.15.26:0/824151701 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b9064fa5d0 msgr2=0x55b906501720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:20:06.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.205+0000 7ff8477fe640 1 --2- 172.21.15.26:0/824151701 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b9064fa5d0 0x55b906501720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:06.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.205+0000 7ff8477fe640 1 -- 172.21.15.26:0/824151701 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b906505370 msgr2=0x55b9064f6150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:20:06.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.205+0000 7ff8477fe640 1 --2- 172.21.15.26:0/824151701 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b906505370 0x55b9064f6150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:06.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.205+0000 7ff8477fe640 1 -- 172.21.15.26:0/824151701 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b906517850 con 0x55b9064fbb40 2024-09-11T23:20:06.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.205+0000 7ff84cdfd640 1 --2- 172.21.15.26:0/824151701 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b906505370 0x55b9064f6150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:20:06.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.205+0000 7ff847fff640 1 --2- 172.21.15.26:0/824151701 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b9064fa5d0 0x55b906501720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:20:06.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.205+0000 7ff8477fe640 1 --2- 172.21.15.26:0/824151701 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b9064fbb40 0x55b9064f5c10 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ff83c0cca90 tx=0x7ff83c0096e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7e208d1f9da1f4df server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:06.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.205+0000 7ff82dffb640 1 -- 172.21.15.26:0/824151701 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff83c01c040 con 0x55b9064fbb40 2024-09-11T23:20:06.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.205+0000 7ff82dffb640 1 -- 172.21.15.26:0/824151701 <== mon.2 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff83c0b3020 con 0x55b9064fbb40 2024-09-11T23:20:06.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.205+0000 7ff82dffb640 1 -- 172.21.15.26:0/824151701 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff83c099b60 con 0x55b9064fbb40 2024-09-11T23:20:06.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.206+0000 7ff850c39740 1 -- 172.21.15.26:0/824151701 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b9064fbb40 msgr2=0x55b9064f5c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:06.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.206+0000 7ff850c39740 1 --2- 172.21.15.26:0/824151701 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b9064fbb40 0x55b9064f5c10 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ff83c0cca90 tx=0x7ff83c0096e0 comp rx=0 tx=0).stop 2024-09-11T23:20:06.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.206+0000 7ff850c39740 1 -- 172.21.15.26:0/824151701 shutdown_connections 2024-09-11T23:20:06.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.206+0000 7ff850c39740 1 --2- 172.21.15.26:0/824151701 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b9064fbb40 0x55b9064f5c10 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:06.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.206+0000 7ff850c39740 1 --2- 172.21.15.26:0/824151701 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b906505370 0x55b9064f6150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:06.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.206+0000 7ff850c39740 1 --2- 172.21.15.26:0/824151701 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b9064fa5d0 0x55b906501720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:06.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.206+0000 7ff850c39740 1 -- 172.21.15.26:0/824151701 >> 172.21.15.26:0/824151701 conn(0x55b9064c3310 msgr2=0x55b9064f5500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:06.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.206+0000 7ff850c39740 1 -- 172.21.15.26:0/824151701 shutdown_connections 2024-09-11T23:20:06.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.206+0000 7ff850c39740 1 -- 172.21.15.26:0/824151701 wait complete. 2024-09-11T23:20:06.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.206+0000 7ff850c39740 1 -- start start 2024-09-11T23:20:06.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.207+0000 7ff850c39740 1 Processor -- start 2024-09-11T23:20:06.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.207+0000 7ff850c39740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b9064fa5d0 0x55b90650dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:06.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.207+0000 7ff850c39740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b9064fbb40 0x55b906508fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:06.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.207+0000 7ff850c39740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b906505370 0x55b9065095d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:06.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.207+0000 7ff850c39740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff8481b0f80 con 0x55b9064fbb40 2024-09-11T23:20:06.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.207+0000 7ff850c39740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7ff8481b0880 con 0x55b906505370 2024-09-11T23:20:06.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.207+0000 7ff850c39740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ff8481adac0 con 0x55b9064fa5d0 2024-09-11T23:20:06.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.207+0000 7ff850c39740 10 client.0 _ll_register_callbacks cb 0x7ff83c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:20:06.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.207+0000 7ff847fff640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b9064fa5d0 0x55b90650dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:06.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.207+0000 7ff847fff640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b9064fa5d0 0x55b90650dcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:54716/0 (socket says 172.21.15.26:54716) 2024-09-11T23:20:06.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.207+0000 7ff847fff640 1 -- 172.21.15.26:0/81728419 learned_addr learned my addr 172.21.15.26:0/81728419 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:20:06.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.207+0000 7ff8477fe640 1 --2- 172.21.15.26:0/81728419 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b9064fbb40 0x55b906508fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:06.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.207+0000 7ff84cdfd640 1 --2- 172.21.15.26:0/81728419 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b906505370 0x55b9065095d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:06.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.208+0000 7ff847fff640 1 -- 172.21.15.26:0/81728419 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b906505370 msgr2=0x55b9065095d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:20:06.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.208+0000 7ff847fff640 1 --2- 172.21.15.26:0/81728419 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b906505370 0x55b9065095d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:06.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.208+0000 7ff847fff640 1 -- 172.21.15.26:0/81728419 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b9064fbb40 msgr2=0x55b906508fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:20:06.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.208+0000 7ff847fff640 1 --2- 172.21.15.26:0/81728419 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b9064fbb40 0x55b906508fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:06.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.208+0000 7ff847fff640 1 -- 172.21.15.26:0/81728419 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b906509dd0 con 0x55b9064fa5d0 2024-09-11T23:20:06.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.208+0000 7ff8477fe640 1 --2- 172.21.15.26:0/81728419 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b9064fbb40 0x55b906508fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:20:06.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.208+0000 7ff84cdfd640 1 --2- 172.21.15.26:0/81728419 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b906505370 0x55b9065095d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:20:06.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.208+0000 7ff847fff640 1 --2- 172.21.15.26:0/81728419 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b9064fa5d0 0x55b90650dcb0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ff83800a010 tx=0x7ff838004c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2037fb5f75d0eace server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:06.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.208+0000 7ff8277fe640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-09-11T23:20:06.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.208+0000 7ff8277fe640 1 -- 172.21.15.26:0/81728419 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8380099d0 con 0x55b9064fa5d0 2024-09-11T23:20:06.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.208+0000 7ff8277fe640 1 -- 172.21.15.26:0/81728419 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7ff838005040 con 0x55b9064fa5d0 2024-09-11T23:20:06.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.208+0000 7ff850c39740 1 -- 172.21.15.26:0/81728419 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff83c0b3020 con 0x55b9064fa5d0 2024-09-11T23:20:06.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.208+0000 7ff8277fe640 1 -- 172.21.15.26:0/81728419 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7ff838005280 con 0x55b9064fa5d0 2024-09-11T23:20:06.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.208+0000 7ff850c39740 10 client.25558 fetch_fsmap learned FSMap version 31 2024-09-11T23:20:06.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.208+0000 7ff850c39740 1 -- 172.21.15.26:0/81728419 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55b906590b40 con 0x55b9064fa5d0 2024-09-11T23:20:06.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.209+0000 7ff8277fe640 1 -- 172.21.15.26:0/81728419 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff83c0b3020 con 0x55b9064fa5d0 2024-09-11T23:20:06.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.209+0000 7ff8277fe640 1 -- 172.21.15.26:0/81728419 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7ff838003ad0 con 0x55b9064fa5d0 2024-09-11T23:20:06.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.209+0000 7ff850c39740 10 client.25558 fetch_fsmap finished waiting for FSMap version 31 2024-09-11T23:20:06.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.209+0000 7ff850c39740 10 client.25558 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:20:06.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.209+0000 7ff850c39740 20 client.25558 populate_metadata read hostname 'smithi026' 2024-09-11T23:20:06.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.209+0000 7ff850c39740 1 --2- 172.21.15.26:0/81728419 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55b906591340 0x55b906593760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:06.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.209+0000 7ff850c39740 4 client.25558 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:20:06.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.209+0000 7ff850c39740 1 -- 172.21.15.26:0/81728419 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b906467d70 con 0x55b906591340 2024-09-11T23:20:06.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.209+0000 7ff8477fe640 1 --2- 172.21.15.26:0/81728419 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55b906591340 0x55b906593760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:06.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.210+0000 7ff8277fe640 10 client.25558 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:20:06.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.210+0000 7ff8477fe640 1 --2- 172.21.15.26:0/81728419 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55b906591340 0x55b906593760 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b61b6f0aaddfb78 server_cookie=495b28695a08bb31 in_seq=0 out_seq=0 2024-09-11T23:20:06.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.210+0000 7ff8277fe640 1 -- 172.21.15.26:0/81728419 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b906467d70 con 0x55b906591340 2024-09-11T23:20:06.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.210+0000 7ff8277fe640 10 client.25558 handle_command_reply: tid=0 2024-09-11T23:20:06.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.210+0000 7ff850c39740 1 client.25558 shutdown 2024-09-11T23:20:06.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.210+0000 7ff850c39740 1 -- 172.21.15.26:0/81728419 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b9064fa5d0 msgr2=0x55b90650dcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:06.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.210+0000 7ff850c39740 1 --2- 172.21.15.26:0/81728419 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b9064fa5d0 0x55b90650dcb0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ff83800a010 tx=0x7ff838004c00 comp rx=0 tx=0).stop 2024-09-11T23:20:06.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.210+0000 7ff850c39740 1 -- 172.21.15.26:0/81728419 shutdown_connections 2024-09-11T23:20:06.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.210+0000 7ff850c39740 1 --2- 172.21.15.26:0/81728419 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b906505370 0x55b9065095d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:06.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.210+0000 7ff850c39740 1 --2- 172.21.15.26:0/81728419 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55b906591340 0x55b906593760 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:06.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.210+0000 7ff850c39740 1 --2- 172.21.15.26:0/81728419 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b9064fbb40 0x55b906508fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:06.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.210+0000 7ff850c39740 1 --2- 172.21.15.26:0/81728419 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b9064fa5d0 0x55b90650dcb0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:06.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.210+0000 7ff8277fe640 0 client.25558 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:20:06.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.210+0000 7ff850c39740 1 -- 172.21.15.26:0/81728419 >> 172.21.15.26:0/81728419 conn(0x55b9064c3310 msgr2=0x55b9064fce90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:06.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.210+0000 7ff850c39740 1 -- 172.21.15.26:0/81728419 shutdown_connections 2024-09-11T23:20:06.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.210+0000 7ff850c39740 1 -- 172.21.15.26:0/81728419 wait complete. 2024-09-11T23:20:06.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.210+0000 7ff850c39740 20 client.25558 trim_cache size 0 max 16384 2024-09-11T23:20:06.229 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2024-09-11T23:20:06.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.211+0000 7ff84e860640 1 -- 172.21.15.26:0/239833266 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7ff8300a2cf0 msgr2=0x7ff8300a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:06.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.211+0000 7ff84e860640 1 --2- 172.21.15.26:0/239833266 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7ff8300a2cf0 0x7ff8300a51b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff840005fe0 tx=0x7ff84002a8c0 comp rx=0 tx=0).stop 2024-09-11T23:20:06.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.211+0000 7ff84e860640 1 -- 172.21.15.26:0/239833266 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff848104990 msgr2=0x7ff84819ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:06.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.211+0000 7ff84e860640 1 --2- 172.21.15.26:0/239833266 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff848104990 0x7ff84819ab40 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff83c00e9f0 tx=0x7ff83c00eeb0 comp rx=0 tx=0).stop 2024-09-11T23:20:06.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.211+0000 7ff84e860640 1 -- 172.21.15.26:0/239833266 shutdown_connections 2024-09-11T23:20:06.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.211+0000 7ff84e860640 1 --2- 172.21.15.26:0/239833266 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7ff8300a2cf0 0x7ff8300a51b0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:06.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.211+0000 7ff84e860640 1 --2- 172.21.15.26:0/239833266 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff848104990 0x7ff84819ab40 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:06.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.211+0000 7ff84e860640 1 --2- 172.21.15.26:0/239833266 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff848109110 0x7ff84819eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:06.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.211+0000 7ff84e860640 1 --2- 172.21.15.26:0/239833266 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff848103fe0 0x7ff84819a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:06.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.211+0000 7ff84e860640 1 -- 172.21.15.26:0/239833266 >> 172.21.15.26:0/239833266 conn(0x7ff8480ffa20 msgr2=0x7ff848100d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:06.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.211+0000 7ff84e860640 1 -- 172.21.15.26:0/239833266 shutdown_connections 2024-09-11T23:20:06.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:06.211+0000 7ff84e860640 1 -- 172.21.15.26:0/239833266 wait complete. 2024-09-11T23:20:07.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:07 smithi026 ceph-mon[31172]: pgmap v55: 97 pgs: 97 active+clean; 57 MiB data, 563 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.6 MiB/s wr, 250 op/s 2024-09-11T23:20:07.501 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:07 smithi026 ceph-mon[31172]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:20:07.553 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:07 smithi076 ceph-mon[35296]: pgmap v55: 97 pgs: 97 active+clean; 57 MiB data, 563 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.6 MiB/s wr, 250 op/s 2024-09-11T23:20:07.553 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:07 smithi076 ceph-mon[35296]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:20:07.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:07 smithi136 ceph-mon[35121]: pgmap v55: 97 pgs: 97 active+clean; 57 MiB data, 563 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.6 MiB/s wr, 250 op/s 2024-09-11T23:20:07.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:07 smithi136 ceph-mon[35121]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:20:08.081 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-11T23:20:08.081 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-11T23:20:08.350 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:20:08.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.820+0000 7f72a159a640 1 -- 172.21.15.26:0/171016890 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72940319b0 con 0x7f72a4101de0 2024-09-11T23:20:08.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.821+0000 7f72a8f27640 1 -- 172.21.15.26:0/171016890 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f72a4101de0 msgr2=0x7f72a41021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:08.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.821+0000 7f72a8f27640 1 --2- 172.21.15.26:0/171016890 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f72a4101de0 0x7f72a41021c0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f7294009900 tx=0x7f729402a410 comp rx=0 tx=0).stop 2024-09-11T23:20:08.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.821+0000 7f72a8f27640 1 -- 172.21.15.26:0/171016890 shutdown_connections 2024-09-11T23:20:08.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.821+0000 7f72a8f27640 1 --2- 172.21.15.26:0/171016890 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f72a4106f10 0x7f72a410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:08.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.821+0000 7f72a8f27640 1 --2- 172.21.15.26:0/171016890 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f72a4102790 0x7f72a41067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:08.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.821+0000 7f72a8f27640 1 --2- 172.21.15.26:0/171016890 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f72a4101de0 0x7f72a41021c0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:08.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.821+0000 7f72a8f27640 1 -- 172.21.15.26:0/171016890 >> 172.21.15.26:0/171016890 conn(0x7f72a40fd8c0 msgr2=0x7f72a40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:08.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.821+0000 7f72a8f27640 1 -- 172.21.15.26:0/171016890 shutdown_connections 2024-09-11T23:20:08.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.821+0000 7f72a8f27640 1 -- 172.21.15.26:0/171016890 wait complete. 2024-09-11T23:20:08.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.822+0000 7f72a8f27640 1 Processor -- start 2024-09-11T23:20:08.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.822+0000 7f72a8f27640 1 -- start start 2024-09-11T23:20:08.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.822+0000 7f72a8f27640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f72a4101de0 0x7f72a40744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:08.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.822+0000 7f72a8f27640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f72a4102790 0x7f72a4074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:08.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.822+0000 7f72a8f27640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f72a4106f10 0x7f72a4074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:08.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.822+0000 7f72a8f27640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f72a4111f70 con 0x7f72a4101de0 2024-09-11T23:20:08.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.822+0000 7f72a8f27640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f72a4111df0 con 0x7f72a4106f10 2024-09-11T23:20:08.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.822+0000 7f72a8f27640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f72a41120f0 con 0x7f72a4102790 2024-09-11T23:20:08.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.822+0000 7f72a259c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f72a4101de0 0x7f72a40744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:08.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.822+0000 7f72a259c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f72a4101de0 0x7f72a40744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48516/0 (socket says 172.21.15.26:48516) 2024-09-11T23:20:08.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.822+0000 7f72a259c640 1 -- 172.21.15.26:0/3106063161 learned_addr learned my addr 172.21.15.26:0/3106063161 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:20:08.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.822+0000 7f72a1d9b640 1 --2- 172.21.15.26:0/3106063161 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f72a4102790 0x7f72a4074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:08.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.822+0000 7f72a2d9d640 1 --2- 172.21.15.26:0/3106063161 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f72a4106f10 0x7f72a4074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:08.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.822+0000 7f72a259c640 1 -- 172.21.15.26:0/3106063161 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f72a4102790 msgr2=0x7f72a4074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:08.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.822+0000 7f72a259c640 1 --2- 172.21.15.26:0/3106063161 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f72a4102790 0x7f72a4074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:08.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.822+0000 7f72a259c640 1 -- 172.21.15.26:0/3106063161 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f72a4106f10 msgr2=0x7f72a4074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:08.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.822+0000 7f72a259c640 1 --2- 172.21.15.26:0/3106063161 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f72a4106f10 0x7f72a4074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:08.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.822+0000 7f72a259c640 1 -- 172.21.15.26:0/3106063161 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72a41a6970 con 0x7f72a4101de0 2024-09-11T23:20:08.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.823+0000 7f72a259c640 1 --2- 172.21.15.26:0/3106063161 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f72a4101de0 0x7f72a40744f0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f7294006010 tx=0x7f72940042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:08.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.823+0000 7f72937fe640 1 -- 172.21.15.26:0/3106063161 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f729403f030 con 0x7f72a4101de0 2024-09-11T23:20:08.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.823+0000 7f72a8f27640 1 -- 172.21.15.26:0/3106063161 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72a41a6c00 con 0x7f72a4101de0 2024-09-11T23:20:08.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.823+0000 7f72937fe640 1 -- 172.21.15.26:0/3106063161 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7294003ce0 con 0x7f72a4101de0 2024-09-11T23:20:08.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.823+0000 7f72937fe640 1 -- 172.21.15.26:0/3106063161 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7294035070 con 0x7f72a4101de0 2024-09-11T23:20:08.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.823+0000 7f72a8f27640 1 -- 172.21.15.26:0/3106063161 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72a41a70e0 con 0x7f72a4101de0 2024-09-11T23:20:08.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.826+0000 7f72ab300740 1 Processor -- start 2024-09-11T23:20:08.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.826+0000 7f72937fe640 1 -- 172.21.15.26:0/3106063161 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f7294042030 con 0x7f72a4101de0 2024-09-11T23:20:08.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.826+0000 7f72ab300740 1 -- start start 2024-09-11T23:20:08.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.826+0000 7f72ab300740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561d9ec2ab60 0x561d9ec2af40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:08.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.826+0000 7f72ab300740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561d9ec2b480 0x561d9ec35800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:08.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.826+0000 7f72ab300740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561d9ec35d40 0x561d9ec38130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:08.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.826+0000 7f72ab300740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x561d9eb90280 con 0x561d9ec2b480 2024-09-11T23:20:08.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.826+0000 7f72ab300740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x561d9eb37b10 con 0x561d9ec2ab60 2024-09-11T23:20:08.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.826+0000 7f72ab300740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x561d9eb8c240 con 0x561d9ec35d40 2024-09-11T23:20:08.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.827+0000 7f72a259c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561d9ec2b480 0x561d9ec35800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:08.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.827+0000 7f72a1d9b640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561d9ec35d40 0x561d9ec38130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:08.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.827+0000 7f72a259c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561d9ec2b480 0x561d9ec35800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48526/0 (socket says 172.21.15.26:48526) 2024-09-11T23:20:08.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.827+0000 7f72a259c640 1 -- 172.21.15.26:0/3163818308 learned_addr learned my addr 172.21.15.26:0/3163818308 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:20:08.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.827+0000 7f72a2d9d640 1 --2- 172.21.15.26:0/3163818308 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561d9ec2ab60 0x561d9ec2af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:08.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.827+0000 7f72a2d9d640 1 -- 172.21.15.26:0/3163818308 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561d9ec35d40 msgr2=0x561d9ec38130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:20:08.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.827+0000 7f72a2d9d640 1 --2- 172.21.15.26:0/3163818308 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561d9ec35d40 0x561d9ec38130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:08.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.827+0000 7f72a2d9d640 1 -- 172.21.15.26:0/3163818308 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561d9ec2b480 msgr2=0x561d9ec35800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:20:08.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.827+0000 7f72a2d9d640 1 --2- 172.21.15.26:0/3163818308 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561d9ec2b480 0x561d9ec35800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:08.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.827+0000 7f72a2d9d640 1 -- 172.21.15.26:0/3163818308 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d9eb441f0 con 0x561d9ec2ab60 2024-09-11T23:20:08.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.827+0000 7f72937fe640 1 --2- 172.21.15.26:0/3106063161 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f728c0a2cf0 0x7f728c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:08.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.827+0000 7f72a1d9b640 1 --2- 172.21.15.26:0/3163818308 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561d9ec35d40 0x561d9ec38130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:20:08.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.827+0000 7f72a1d9b640 1 --2- 172.21.15.26:0/3106063161 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f728c0a2cf0 0x7f728c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:08.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.827+0000 7f72937fe640 1 -- 172.21.15.26:0/3106063161 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f72940efa20 con 0x7f72a4101de0 2024-09-11T23:20:08.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.827+0000 7f72a2d9d640 1 --2- 172.21.15.26:0/3163818308 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561d9ec2ab60 0x561d9ec2af40 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f729c009da0 tx=0x7f729c010740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=20a210cddd72a8ac server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:08.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.827+0000 7f7290ff9640 1 -- 172.21.15.26:0/3163818308 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f729c016070 con 0x561d9ec2ab60 2024-09-11T23:20:08.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.827+0000 7f7290ff9640 1 -- 172.21.15.26:0/3163818308 <== mon.2 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f729c010e50 con 0x561d9ec2ab60 2024-09-11T23:20:08.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.827+0000 7f7290ff9640 1 -- 172.21.15.26:0/3163818308 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f729c012740 con 0x561d9ec2ab60 2024-09-11T23:20:08.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.828+0000 7f72ab300740 1 -- 172.21.15.26:0/3163818308 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561d9ec2ab60 msgr2=0x561d9ec2af40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:08.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.828+0000 7f72ab300740 1 --2- 172.21.15.26:0/3163818308 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561d9ec2ab60 0x561d9ec2af40 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f729c009da0 tx=0x7f729c010740 comp rx=0 tx=0).stop 2024-09-11T23:20:08.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.828+0000 7f72a1d9b640 1 --2- 172.21.15.26:0/3106063161 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f728c0a2cf0 0x7f728c0a51b0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x561d9ec28470 tx=0x7f7298005f90 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:08.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.828+0000 7f72ab300740 1 -- 172.21.15.26:0/3163818308 shutdown_connections 2024-09-11T23:20:08.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.828+0000 7f72ab300740 1 --2- 172.21.15.26:0/3163818308 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561d9ec35d40 0x561d9ec38130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:08.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.828+0000 7f72ab300740 1 --2- 172.21.15.26:0/3163818308 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561d9ec2b480 0x561d9ec35800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:08.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.828+0000 7f72ab300740 1 --2- 172.21.15.26:0/3163818308 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561d9ec2ab60 0x561d9ec2af40 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:08.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.828+0000 7f72ab300740 1 -- 172.21.15.26:0/3163818308 >> 172.21.15.26:0/3163818308 conn(0x561d9ec270f0 msgr2=0x561d9ec274d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:08.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.828+0000 7f72ab300740 1 -- 172.21.15.26:0/3163818308 shutdown_connections 2024-09-11T23:20:08.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.828+0000 7f72ab300740 1 -- 172.21.15.26:0/3163818308 wait complete. 2024-09-11T23:20:08.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.829+0000 7f72ab300740 1 -- start start 2024-09-11T23:20:08.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.829+0000 7f72ab300740 1 Processor -- start 2024-09-11T23:20:08.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.829+0000 7f72ab300740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561d9ec2ab60 0x561d9ec32c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:08.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.829+0000 7f72ab300740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561d9ec2b480 0x561d9ec33150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:08.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.829+0000 7f72ab300740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561d9ec35d40 0x561d9ec33690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:08.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.829+0000 7f72ab300740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x561d9ec38910 con 0x561d9ec35d40 2024-09-11T23:20:08.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.829+0000 7f72ab300740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x561d9eb4af10 con 0x561d9ec2b480 2024-09-11T23:20:08.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.829+0000 7f72ab300740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x561d9ec38a90 con 0x561d9ec2ab60 2024-09-11T23:20:08.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.829+0000 7f72ab300740 10 client.0 _ll_register_callbacks cb 0x561d9eb13a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:20:08.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.829+0000 7f72a2d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561d9ec35d40 0x561d9ec33690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:08.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.829+0000 7f72a2d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561d9ec35d40 0x561d9ec33690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48532/0 (socket says 172.21.15.26:48532) 2024-09-11T23:20:08.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.829+0000 7f72a2d9d640 1 -- 172.21.15.26:0/3588931222 learned_addr learned my addr 172.21.15.26:0/3588931222 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:20:08.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.829+0000 7f72a259c640 1 --2- 172.21.15.26:0/3588931222 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561d9ec2ab60 0x561d9ec32c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:08.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.829+0000 7f72a1d9b640 1 --2- 172.21.15.26:0/3588931222 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561d9ec2b480 0x561d9ec33150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:08.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.830+0000 7f72a2d9d640 1 -- 172.21.15.26:0/3588931222 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561d9ec2ab60 msgr2=0x561d9ec32c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:20:08.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.830+0000 7f72a2d9d640 1 --2- 172.21.15.26:0/3588931222 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561d9ec2ab60 0x561d9ec32c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:08.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.830+0000 7f72a2d9d640 1 -- 172.21.15.26:0/3588931222 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561d9ec2b480 msgr2=0x561d9ec33150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:20:08.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.830+0000 7f72a2d9d640 1 --2- 172.21.15.26:0/3588931222 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561d9ec2b480 0x561d9ec33150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:08.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.830+0000 7f72a2d9d640 1 -- 172.21.15.26:0/3588931222 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d9e9016a0 con 0x561d9ec35d40 2024-09-11T23:20:08.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.830+0000 7f72a2d9d640 1 --2- 172.21.15.26:0/3588931222 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561d9ec35d40 0x561d9ec33690 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f729c000f80 tx=0x7f729c013700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42e92d5ea99fc40c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:08.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.830+0000 7f7289ffb640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:20:08.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.830+0000 7f7289ffb640 1 -- 172.21.15.26:0/3588931222 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f729c028060 con 0x561d9ec35d40 2024-09-11T23:20:08.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.830+0000 7f7289ffb640 1 -- 172.21.15.26:0/3588931222 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f729c013a80 con 0x561d9ec35d40 2024-09-11T23:20:08.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.830+0000 7f72ab300740 1 -- 172.21.15.26:0/3588931222 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f729c010e50 con 0x561d9ec35d40 2024-09-11T23:20:08.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.830+0000 7f7289ffb640 1 -- 172.21.15.26:0/3588931222 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f729c016050 con 0x561d9ec35d40 2024-09-11T23:20:08.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.830+0000 7f72ab300740 10 client.15936 fetch_fsmap learned FSMap version 32 2024-09-11T23:20:08.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.830+0000 7f72ab300740 1 -- 172.21.15.26:0/3588931222 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x561d9ec44540 con 0x561d9ec35d40 2024-09-11T23:20:08.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.830+0000 7f7289ffb640 1 -- 172.21.15.26:0/3588931222 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f729c010e50 con 0x561d9ec35d40 2024-09-11T23:20:08.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.831+0000 7f7289ffb640 1 -- 172.21.15.26:0/3588931222 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 32) ==== 8394+0+0 (secure 0 0 0) 0x7f729c00f040 con 0x561d9ec35d40 2024-09-11T23:20:08.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.831+0000 7f72ab300740 10 client.15936 fetch_fsmap finished waiting for FSMap version 32 2024-09-11T23:20:08.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.831+0000 7f72ab300740 10 client.15936 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:20:08.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.831+0000 7f72ab300740 20 client.15936 populate_metadata read hostname 'smithi026' 2024-09-11T23:20:08.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.831+0000 7f72ab300740 1 --2- 172.21.15.26:0/3588931222 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x561d9ec45a00 0x561d9ec47df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:08.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.831+0000 7f72ab300740 4 client.15936 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:20:08.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.831+0000 7f72ab300740 1 -- 172.21.15.26:0/3588931222 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561d9eb9d7f0 con 0x561d9ec45a00 2024-09-11T23:20:08.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.831+0000 7f72a259c640 1 --2- 172.21.15.26:0/3588931222 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x561d9ec45a00 0x561d9ec47df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:08.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.831+0000 7f72a259c640 1 --2- 172.21.15.26:0/3588931222 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x561d9ec45a00 0x561d9ec47df0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=364cc5297b15f57e server_cookie=39beeedde56d64b4 in_seq=0 out_seq=0 2024-09-11T23:20:08.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.831+0000 7f7289ffb640 10 client.15936 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:20:08.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.832+0000 7f7289ffb640 1 -- 172.21.15.26:0/3588931222 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561d9eb9d7f0 con 0x561d9ec45a00 2024-09-11T23:20:08.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.832+0000 7f7289ffb640 10 client.15936 handle_command_reply: tid=0 2024-09-11T23:20:08.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.832+0000 7f72ab300740 1 client.15936 shutdown 2024-09-11T23:20:08.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.832+0000 7f72ab300740 1 -- 172.21.15.26:0/3588931222 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561d9ec35d40 msgr2=0x561d9ec33690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:08.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.832+0000 7f72ab300740 1 --2- 172.21.15.26:0/3588931222 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561d9ec35d40 0x561d9ec33690 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f729c000f80 tx=0x7f729c013700 comp rx=0 tx=0).stop 2024-09-11T23:20:08.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.832+0000 7f72ab300740 1 -- 172.21.15.26:0/3588931222 shutdown_connections 2024-09-11T23:20:08.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.832+0000 7f72ab300740 1 --2- 172.21.15.26:0/3588931222 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561d9ec2b480 0x561d9ec33150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:08.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.832+0000 7f72ab300740 1 --2- 172.21.15.26:0/3588931222 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x561d9ec45a00 0x561d9ec47df0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:08.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.832+0000 7f7289ffb640 0 client.15936 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:20:08.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.832+0000 7f72ab300740 1 --2- 172.21.15.26:0/3588931222 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561d9ec35d40 0x561d9ec33690 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:08.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.832+0000 7f72ab300740 1 --2- 172.21.15.26:0/3588931222 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561d9ec2ab60 0x561d9ec32c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:08.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.832+0000 7f72ab300740 1 -- 172.21.15.26:0/3588931222 >> 172.21.15.26:0/3588931222 conn(0x561d9ec270f0 msgr2=0x561d9ec2ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:08.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.832+0000 7f72ab300740 1 -- 172.21.15.26:0/3588931222 shutdown_connections 2024-09-11T23:20:08.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.832+0000 7f72ab300740 1 -- 172.21.15.26:0/3588931222 wait complete. 2024-09-11T23:20:08.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.833+0000 7f72ab300740 20 client.15936 trim_cache size 0 max 16384 2024-09-11T23:20:08.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.850+0000 7f72ab300740 1 Processor -- start 2024-09-11T23:20:08.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.850+0000 7f72ab300740 1 -- start start 2024-09-11T23:20:08.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.851+0000 7f72ab300740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561d9ec2ab60 0x561d9ec2d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:08.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.851+0000 7f72ab300740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561d9ec2b480 0x561d9ec4d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:08.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.851+0000 7f72ab300740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561d9ec35d40 0x561d9ec4d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:08.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.851+0000 7f72ab300740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f72a41afa90 con 0x561d9ec2b480 2024-09-11T23:20:08.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.851+0000 7f72ab300740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f72a41b1750 con 0x561d9ec35d40 2024-09-11T23:20:08.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.851+0000 7f72ab300740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f72a41b0160 con 0x561d9ec2ab60 2024-09-11T23:20:08.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.851+0000 7f72a1d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561d9ec2b480 0x561d9ec4d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:08.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.851+0000 7f72a259c640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561d9ec2ab60 0x561d9ec2d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:08.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.851+0000 7f72a1d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561d9ec2b480 0x561d9ec4d4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48544/0 (socket says 172.21.15.26:48544) 2024-09-11T23:20:08.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.851+0000 7f72a1d9b640 1 -- 172.21.15.26:0/892734812 learned_addr learned my addr 172.21.15.26:0/892734812 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:20:08.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.851+0000 7f72a2d9d640 1 --2- 172.21.15.26:0/892734812 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561d9ec35d40 0x561d9ec4d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:08.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.851+0000 7f72a259c640 1 -- 172.21.15.26:0/892734812 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561d9ec35d40 msgr2=0x561d9ec4d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:20:08.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.851+0000 7f72a259c640 1 --2- 172.21.15.26:0/892734812 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561d9ec35d40 0x561d9ec4d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:08.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.851+0000 7f72a259c640 1 -- 172.21.15.26:0/892734812 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561d9ec2b480 msgr2=0x561d9ec4d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:20:08.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.851+0000 7f72a259c640 1 --2- 172.21.15.26:0/892734812 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561d9ec2b480 0x561d9ec4d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:08.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.851+0000 7f72a259c640 1 -- 172.21.15.26:0/892734812 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d9ec2b2b0 con 0x561d9ec2ab60 2024-09-11T23:20:08.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.851+0000 7f72a259c640 1 --2- 172.21.15.26:0/892734812 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561d9ec2ab60 0x561d9ec2d5c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f72940ed780 tx=0x7f72940f34c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5cc2edbb10011073 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:08.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.852+0000 7f728bfff640 1 -- 172.21.15.26:0/892734812 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7294049040 con 0x561d9ec2ab60 2024-09-11T23:20:08.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.852+0000 7f728bfff640 1 -- 172.21.15.26:0/892734812 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72940f8040 con 0x561d9ec2ab60 2024-09-11T23:20:08.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.852+0000 7f728bfff640 1 -- 172.21.15.26:0/892734812 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72940044e0 con 0x561d9ec2ab60 2024-09-11T23:20:08.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.852+0000 7f72ab300740 1 -- 172.21.15.26:0/892734812 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561d9ec2ab60 msgr2=0x561d9ec2d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:08.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.852+0000 7f72ab300740 1 --2- 172.21.15.26:0/892734812 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561d9ec2ab60 0x561d9ec2d5c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f72940ed780 tx=0x7f72940f34c0 comp rx=0 tx=0).stop 2024-09-11T23:20:08.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.852+0000 7f72ab300740 1 -- 172.21.15.26:0/892734812 shutdown_connections 2024-09-11T23:20:08.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.852+0000 7f72ab300740 1 --2- 172.21.15.26:0/892734812 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561d9ec35d40 0x561d9ec4d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:08.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.852+0000 7f72ab300740 1 --2- 172.21.15.26:0/892734812 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561d9ec2b480 0x561d9ec4d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:08.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.852+0000 7f72ab300740 1 --2- 172.21.15.26:0/892734812 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561d9ec2ab60 0x561d9ec2d5c0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:08.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.852+0000 7f72ab300740 1 -- 172.21.15.26:0/892734812 >> 172.21.15.26:0/892734812 conn(0x561d9ec270f0 msgr2=0x561d9ec34b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:08.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.852+0000 7f72ab300740 1 -- 172.21.15.26:0/892734812 shutdown_connections 2024-09-11T23:20:08.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.853+0000 7f72ab300740 1 -- 172.21.15.26:0/892734812 wait complete. 2024-09-11T23:20:08.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.853+0000 7f72ab300740 1 -- start start 2024-09-11T23:20:08.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.853+0000 7f72ab300740 1 Processor -- start 2024-09-11T23:20:08.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.853+0000 7f72ab300740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561d9ec2ab60 0x561d9ec3e460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:08.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.853+0000 7f72ab300740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561d9ec2b480 0x561d9ec2f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:08.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.853+0000 7f72ab300740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561d9ec35d40 0x561d9ec2fef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:08.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.853+0000 7f72ab300740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f72a41b0fc0 con 0x561d9ec2b480 2024-09-11T23:20:08.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.853+0000 7f72ab300740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f72a41b0830 con 0x561d9ec35d40 2024-09-11T23:20:08.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.853+0000 7f72ab300740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f72a41ada70 con 0x561d9ec2ab60 2024-09-11T23:20:08.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.853+0000 7f72a259c640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561d9ec2ab60 0x561d9ec3e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:08.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.853+0000 7f72a259c640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561d9ec2ab60 0x561d9ec3e460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:54772/0 (socket says 172.21.15.26:54772) 2024-09-11T23:20:08.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.853+0000 7f72ab300740 10 client.0 _ll_register_callbacks cb 0x7f72940f4040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:20:08.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.853+0000 7f72a259c640 1 -- 172.21.15.26:0/1503086504 learned_addr learned my addr 172.21.15.26:0/1503086504 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:20:08.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.854+0000 7f72a1d9b640 1 --2- 172.21.15.26:0/1503086504 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561d9ec2b480 0x561d9ec2f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:08.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.854+0000 7f72a2d9d640 1 --2- 172.21.15.26:0/1503086504 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561d9ec35d40 0x561d9ec2fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:08.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.854+0000 7f72a259c640 1 -- 172.21.15.26:0/1503086504 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561d9ec35d40 msgr2=0x561d9ec2fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:20:08.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.854+0000 7f72a259c640 1 --2- 172.21.15.26:0/1503086504 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561d9ec35d40 0x561d9ec2fef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:08.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.854+0000 7f72a259c640 1 -- 172.21.15.26:0/1503086504 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561d9ec2b480 msgr2=0x561d9ec2f9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:20:08.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.854+0000 7f72a259c640 1 --2- 172.21.15.26:0/1503086504 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561d9ec2b480 0x561d9ec2f9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:08.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.854+0000 7f72a259c640 1 -- 172.21.15.26:0/1503086504 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d9ec30720 con 0x561d9ec2ab60 2024-09-11T23:20:08.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.854+0000 7f72a259c640 1 --2- 172.21.15.26:0/1503086504 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561d9ec2ab60 0x561d9ec3e460 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f7294005ab0 tx=0x7f7294031a60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=acf28ddf9cb91fb2 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:08.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.854+0000 7f7289ffb640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-09-11T23:20:08.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.854+0000 7f7289ffb640 1 -- 172.21.15.26:0/1503086504 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7294049070 con 0x561d9ec2ab60 2024-09-11T23:20:08.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.854+0000 7f72ab300740 1 -- 172.21.15.26:0/1503086504 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f72940f8040 con 0x561d9ec2ab60 2024-09-11T23:20:08.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.854+0000 7f7289ffb640 1 -- 172.21.15.26:0/1503086504 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f72940b8680 con 0x561d9ec2ab60 2024-09-11T23:20:08.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.854+0000 7f7289ffb640 1 -- 172.21.15.26:0/1503086504 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f72940044e0 con 0x561d9ec2ab60 2024-09-11T23:20:08.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.854+0000 7f72ab300740 10 client.25576 fetch_fsmap learned FSMap version 32 2024-09-11T23:20:08.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.854+0000 7f72ab300740 1 -- 172.21.15.26:0/1503086504 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x561d9ecb0510 con 0x561d9ec2ab60 2024-09-11T23:20:08.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.855+0000 7f7289ffb640 1 -- 172.21.15.26:0/1503086504 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72940f8040 con 0x561d9ec2ab60 2024-09-11T23:20:08.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.855+0000 7f7289ffb640 1 -- 172.21.15.26:0/1503086504 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 32) ==== 8394+0+0 (secure 0 0 0) 0x7f7294079030 con 0x561d9ec2ab60 2024-09-11T23:20:08.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.855+0000 7f72ab300740 10 client.25576 fetch_fsmap finished waiting for FSMap version 32 2024-09-11T23:20:08.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.855+0000 7f72ab300740 10 client.25576 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:20:08.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.855+0000 7f72ab300740 20 client.25576 populate_metadata read hostname 'smithi026' 2024-09-11T23:20:08.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.855+0000 7f72ab300740 1 --2- 172.21.15.26:0/1503086504 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x561d9ec45a00 0x561d9ecb2b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:08.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.855+0000 7f72ab300740 4 client.25576 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-11T23:20:08.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.855+0000 7f72ab300740 1 -- 172.21.15.26:0/1503086504 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x561d9eb97d00 con 0x561d9ec45a00 2024-09-11T23:20:08.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.855+0000 7f72a1d9b640 1 --2- 172.21.15.26:0/1503086504 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x561d9ec45a00 0x561d9ecb2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:08.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.855+0000 7f72a1d9b640 1 --2- 172.21.15.26:0/1503086504 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x561d9ec45a00 0x561d9ecb2b00 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f655dd78252a3b80 server_cookie=8615f223fd43013 in_seq=0 out_seq=0 2024-09-11T23:20:08.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.856+0000 7f7289ffb640 10 client.25576 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:20:08.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.856+0000 7f7289ffb640 1 -- 172.21.15.26:0/1503086504 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x561d9eb97d00 con 0x561d9ec45a00 2024-09-11T23:20:08.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.856+0000 7f7289ffb640 10 client.25576 handle_command_reply: tid=0 2024-09-11T23:20:08.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.856+0000 7f72ab300740 1 client.25576 shutdown 2024-09-11T23:20:08.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.856+0000 7f72ab300740 1 -- 172.21.15.26:0/1503086504 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561d9ec2ab60 msgr2=0x561d9ec3e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:08.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.856+0000 7f72ab300740 1 --2- 172.21.15.26:0/1503086504 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561d9ec2ab60 0x561d9ec3e460 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f7294005ab0 tx=0x7f7294031a60 comp rx=0 tx=0).stop 2024-09-11T23:20:08.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.856+0000 7f72ab300740 1 -- 172.21.15.26:0/1503086504 shutdown_connections 2024-09-11T23:20:08.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.856+0000 7f72ab300740 1 --2- 172.21.15.26:0/1503086504 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561d9ec35d40 0x561d9ec2fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:08.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.856+0000 7f72ab300740 1 --2- 172.21.15.26:0/1503086504 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x561d9ec45a00 0x561d9ecb2b00 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:08.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.856+0000 7f72ab300740 1 --2- 172.21.15.26:0/1503086504 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561d9ec2b480 0x561d9ec2f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:08.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.856+0000 7f72ab300740 1 --2- 172.21.15.26:0/1503086504 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561d9ec2ab60 0x561d9ec3e460 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:08.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.856+0000 7f7289ffb640 0 client.25576 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:20:08.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.856+0000 7f72ab300740 1 -- 172.21.15.26:0/1503086504 >> 172.21.15.26:0/1503086504 conn(0x561d9ec270f0 msgr2=0x561d9ec32b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:08.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.856+0000 7f72ab300740 1 -- 172.21.15.26:0/1503086504 shutdown_connections 2024-09-11T23:20:08.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.856+0000 7f72ab300740 1 -- 172.21.15.26:0/1503086504 wait complete. 2024-09-11T23:20:08.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.857+0000 7f72ab300740 20 client.25576 trim_cache size 0 max 16384 2024-09-11T23:20:08.864 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-11T23:20:08.864 INFO:teuthology.orchestra.run.smithi026.stdout: "return_code": 0, 2024-09-11T23:20:08.864 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_tag": "a4be7cc5-004d-4c94-bbfa-1fe107d6af79", 2024-09-11T23:20:08.864 INFO:teuthology.orchestra.run.smithi026.stdout: "mode": "asynchronous" 2024-09-11T23:20:08.864 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-11T23:20:08.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.857+0000 7f72a8f27640 1 -- 172.21.15.26:0/3106063161 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f728c0a2cf0 msgr2=0x7f728c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:08.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.857+0000 7f72a8f27640 1 --2- 172.21.15.26:0/3106063161 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f728c0a2cf0 0x7f728c0a51b0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x561d9ec28470 tx=0x7f7298005f90 comp rx=0 tx=0).stop 2024-09-11T23:20:08.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.857+0000 7f72a8f27640 1 -- 172.21.15.26:0/3106063161 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f72a4101de0 msgr2=0x7f72a40744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:08.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.857+0000 7f72a8f27640 1 --2- 172.21.15.26:0/3106063161 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f72a4101de0 0x7f72a40744f0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f7294006010 tx=0x7f72940042f0 comp rx=0 tx=0).stop 2024-09-11T23:20:08.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.857+0000 7f72a8f27640 1 -- 172.21.15.26:0/3106063161 shutdown_connections 2024-09-11T23:20:08.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.857+0000 7f72a8f27640 1 --2- 172.21.15.26:0/3106063161 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f728c0a2cf0 0x7f728c0a51b0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:08.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.857+0000 7f72a8f27640 1 --2- 172.21.15.26:0/3106063161 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f72a4106f10 0x7f72a4074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:08.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.857+0000 7f72a8f27640 1 --2- 172.21.15.26:0/3106063161 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f72a4102790 0x7f72a4074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:08.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.857+0000 7f72a8f27640 1 --2- 172.21.15.26:0/3106063161 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f72a4101de0 0x7f72a40744f0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:08.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.857+0000 7f72a8f27640 1 -- 172.21.15.26:0/3106063161 >> 172.21.15.26:0/3106063161 conn(0x7f72a40fd8c0 msgr2=0x7f72a40fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:08.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.857+0000 7f72a8f27640 1 -- 172.21.15.26:0/3106063161 shutdown_connections 2024-09-11T23:20:08.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:08.857+0000 7f72a8f27640 1 -- 172.21.15.26:0/3106063161 wait complete. 2024-09-11T23:20:09.553 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:09 smithi076 ceph-mon[35296]: pgmap v56: 97 pgs: 97 active+clean; 59 MiB data, 577 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 5.5 MiB/s wr, 326 op/s 2024-09-11T23:20:09.553 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:09 smithi076 ceph-mon[35296]: scrub queued for path: / 2024-09-11T23:20:09.553 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:09 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:09.553 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:09 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:09.553 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:09 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:09.594 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:09 smithi026 ceph-mon[31172]: pgmap v56: 97 pgs: 97 active+clean; 59 MiB data, 577 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 5.5 MiB/s wr, 326 op/s 2024-09-11T23:20:09.594 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:09 smithi026 ceph-mon[31172]: scrub queued for path: / 2024-09-11T23:20:09.594 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:09 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:09.594 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:09 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:09.594 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:09 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:09.620 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:09 smithi136 ceph-mon[35121]: pgmap v56: 97 pgs: 97 active+clean; 59 MiB data, 577 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 5.5 MiB/s wr, 326 op/s 2024-09-11T23:20:09.620 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:09 smithi136 ceph-mon[35121]: scrub queued for path: / 2024-09-11T23:20:09.620 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:09 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:09.620 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:09 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:09.620 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:09 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:09.621 INFO:journalctl@ceph.mds.i.smithi136.stdout:Sep 11 23:20:09 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mds-i[65341]: 2024-09-11T23:20:09.533+0000 7ff35e42b640 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000020b [3,head] /volumes/qa/sv_1/9fc8f023-d646-4fec-98c7-1f3d972f60a9/client.0/tmp/blogbench-1.0/src/blogbench.c auth v50 ap=1 s=2117 n(v0 rc2024-09-11T23:19:44.881504+0000 b2117 1=1+0) caps={15180=pAsLsXsFscr/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x555c908c3600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000020b:[<0x10000000208/blogbench.c v50>,<0x10000000201/src v1914>,<0x10000000006/blogbench-1.0 v1716>,<0x10000000005/tmp v18>,<0x100000001fb/client.0 v16>,<0x100000001fa/9fc8f023-d646-4fec-98c7-1f3d972f60a9 v32>,<0x10000000001/sv_1 v55>,<0x10000000000/qa v43>,<0x1/volumes v42>]//[]","memoryvalue":"(3)0x1000000020b:[<0x10000000208/blogbench.c v50>,<0x10000000201/src v1989>,<0x10000000006/blogbench-1.0 v1743>,<0x10000000005/tmp v35>,<0x100000001fb/client.0 v31>,<0x100000001fa/9fc8f023-d646-4fec-98c7-1f3d972f60a9 v44>,<0x10000000001/sv_1 v67>,<0x10000000000/qa v33>,<0x1/volumes v32>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-11T23:20:09.621 INFO:journalctl@ceph.mds.i.smithi136.stdout:Sep 11 23:20:09 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mds-i[65341]: 2024-09-11T23:20:09.535+0000 7ff35e42b640 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000379 [...3,head] /volumes/qa/sv_1/9fc8f023-d646-4fec-98c7-1f3d972f60a9/client.0/tmp/blogbench-1.0/src/.deps/ auth{0=1,2=1,3=1,4=1} v266 ap=1 f(v0 m2024-09-11T23:19:51.511400+0000 10=10+0) n(v0 rc2024-09-11T23:19:51.511400+0000 b79855 11=10+1) (inest mix) (ifile excl) caps={15180=pAsLsXsFsx/pAsLsXsFsx@2},l=15180 | lock=0 importingcaps=0 importing=0 dirfrag=1 caps=1 stickydirs=0 dirwaiter=0 replicated=1 dirty=1 authpin=1 discoverbase=0 scrubqueue=0 0x555c903b7080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(2)0x10000000379:[<0x10000000208/.deps v120>,<0x10000000201/src v1914>,<0x10000000006/blogbench-1.0 v1716>,<0x10000000005/tmp v18>,<0x100000001fb/client.0 v16>,<0x100000001fa/9fc8f023-d646-4fec-98c7-1f3d972f60a9 v32>,<0x10000000001/sv_1 v55>,<0x10000000000/qa v43>,<0x1/volumes v42>]//[]","memoryvalue":"(2)0x10000000379:[<0x10000000208/.deps v266>,<0x10000000201/src v1989>,<0x10000000006/blogbench-1.0 v1743>,<0x10000000005/tmp v35>,<0x100000001fb/client.0 v31>,<0x100000001fa/9fc8f023-d646-4fec-98c7-1f3d972f60a9 v44>,<0x10000000001/sv_1 v67>,<0x10000000000/qa v33>,<0x1/volumes v32>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-09-11T23:19:51.511400+0000 10=10+0)","ondisk_value.rstat":"n(v0 rc2024-09-11T23:19:51.511400+0000 b79855 11=10+1)","memory_value.dirstat":"f(v0 m2024-09-11T23:19:51.511400+0000 10=10+0)","memory_value.rstat":"n(v0 rc2024-09-11T23:19:51.511400+0000 b79855 11=10+1)","error_str":""},"return_code":0} 2024-09-11T23:20:09.621 INFO:journalctl@ceph.mds.i.smithi136.stdout:Sep 11 23:20:09 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mds-i[65341]: 2024-09-11T23:20:09.536+0000 7ff35e42b640 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000387 [3,head] /volumes/qa/sv_1/9fc8f023-d646-4fec-98c7-1f3d972f60a9/client.0/tmp/blogbench-1.0/src/process.o auth{0=1} v252 ap=1 s=18416 n(v0 rc2024-09-11T23:19:50.951409+0000 b18416 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={15180=0-4194304@2} caps={15180=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15180 | importingcaps=0 caps=1 replicated=1 dirty=1 authpin=1 scrubqueue=0 0x555c908c4100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000387:[<0x10000000208/process.o v112>,<0x10000000201/src v1914>,<0x10000000006/blogbench-1.0 v1716>,<0x10000000005/tmp v18>,<0x100000001fb/client.0 v16>,<0x100000001fa/9fc8f023-d646-4fec-98c7-1f3d972f60a9 v32>,<0x10000000001/sv_1 v55>,<0x10000000000/qa v43>,<0x1/volumes v42>]//[]","memoryvalue":"(3)0x10000000387:[<0x10000000208/process.o v252>,<0x10000000201/src v1989>,<0x10000000006/blogbench-1.0 v1743>,<0x10000000005/tmp v35>,<0x100000001fb/client.0 v31>,<0x100000001fa/9fc8f023-d646-4fec-98c7-1f3d972f60a9 v44>,<0x10000000001/sv_1 v67>,<0x10000000000/qa v33>,<0x1/volumes v32>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-11T23:20:09.621 INFO:journalctl@ceph.mds.i.smithi136.stdout:Sep 11 23:20:09 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mds-i[65341]: 2024-09-11T23:20:09.541+0000 7ff35e42b640 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000215 [3,head] /volumes/qa/sv_1/9fc8f023-d646-4fec-98c7-1f3d972f60a9/client.0/tmp/blogbench-1.0/src/globals.h auth v54 ap=1 s=1419 n(v0 rc2024-09-11T23:19:44.892504+0000 b1419 1=1+0) caps={15180=pAsLsXsFscr/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x555c908c5180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000215:[<0x10000000208/globals.h v54>,<0x10000000201/src v1914>,<0x10000000006/blogbench-1.0 v1716>,<0x10000000005/tmp v18>,<0x100000001fb/client.0 v16>,<0x100000001fa/9fc8f023-d646-4fec-98c7-1f3d972f60a9 v32>,<0x10000000001/sv_1 v55>,<0x10000000000/qa v43>,<0x1/volumes v42>]//[]","memoryvalue":"(3)0x10000000215:[<0x10000000208/globals.h v54>,<0x10000000201/src v1989>,<0x10000000006/blogbench-1.0 v1743>,<0x10000000005/tmp v35>,<0x100000001fb/client.0 v31>,<0x100000001fa/9fc8f023-d646-4fec-98c7-1f3d972f60a9 v44>,<0x10000000001/sv_1 v67>,<0x10000000000/qa v33>,<0x1/volumes v32>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-11T23:20:09.621 INFO:journalctl@ceph.mds.i.smithi136.stdout:Sep 11 23:20:09 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mds-i[65341]: 2024-09-11T23:20:09.542+0000 7ff35e42b640 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000219 [3,head] /volumes/qa/sv_1/9fc8f023-d646-4fec-98c7-1f3d972f60a9/client.0/tmp/blogbench-1.0/src/writer.c auth v52 ap=1 s=1910 n(v0 rc2005-02-11T14:29:28.000000+0000 b1910 1=1+0) caps={15180=pAsLsXsFscr/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x555c908c5700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000219:[<0x10000000208/writer.c v52>,<0x10000000201/src v1914>,<0x10000000006/blogbench-1.0 v1716>,<0x10000000005/tmp v18>,<0x100000001fb/client.0 v16>,<0x100000001fa/9fc8f023-d646-4fec-98c7-1f3d972f60a9 v32>,<0x10000000001/sv_1 v55>,<0x10000000000/qa v43>,<0x1/volumes v42>]//[]","memoryvalue":"(3)0x10000000219:[<0x10000000208/writer.c v52>,<0x10000000201/src v1989>,<0x10000000006/blogbench-1.0 v1743>,<0x10000000005/tmp v35>,<0x100000001fb/client.0 v31>,<0x100000001fa/9fc8f023-d646-4fec-98c7-1f3d972f60a9 v44>,<0x10000000001/sv_1 v67>,<0x10000000000/qa v33>,<0x1/volumes v32>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-11T23:20:09.622 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T23:20:09.895 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:20:09.989 INFO:journalctl@ceph.mds.i.smithi136.stdout:Sep 11 23:20:09 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mds-i[65341]: 2024-09-11T23:20:09.620+0000 7ff35e42b640 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000209 [3,head] /volumes/qa/sv_1/9fc8f023-d646-4fec-98c7-1f3d972f60a9/client.0/tmp/blogbench-1.0/src/mysnprintf.c auth v48 ap=1 s=729 n(v0 rc2024-09-11T23:19:44.879504+0000 b729 1=1+0) caps={15180=pAsLsXsFscr/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x555c90928000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000209:[<0x10000000208/mysnprintf.c v48>,<0x10000000201/src v1914>,<0x10000000006/blogbench-1.0 v1716>,<0x10000000005/tmp v18>,<0x100000001fb/client.0 v16>,<0x100000001fa/9fc8f023-d646-4fec-98c7-1f3d972f60a9 v32>,<0x10000000001/sv_1 v55>,<0x10000000000/qa v43>,<0x1/volumes v42>]//[]","memoryvalue":"(3)0x10000000209:[<0x10000000208/mysnprintf.c v48>,<0x10000000201/src v1989>,<0x10000000006/blogbench-1.0 v1743>,<0x10000000005/tmp v35>,<0x100000001fb/client.0 v31>,<0x100000001fa/9fc8f023-d646-4fec-98c7-1f3d972f60a9 v44>,<0x10000000001/sv_1 v67>,<0x10000000000/qa v33>,<0x1/volumes v32>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-11T23:20:09.990 INFO:journalctl@ceph.mds.i.smithi136.stdout:Sep 11 23:20:09 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mds-i[65341]: 2024-09-11T23:20:09.630+0000 7ff35e42b640 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000211 [3,head] /volumes/qa/sv_1/9fc8f023-d646-4fec-98c7-1f3d972f60a9/client.0/tmp/blogbench-1.0/src/bsd-getopt_long.h auth v40 ap=1 s=4944 n(v0 rc2024-09-11T23:19:44.888504+0000 b4944 1=1+0) caps={15180=pAsLsXsFscr/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x555c90928580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000211:[<0x10000000208/bsd-getopt_long.h v40>,<0x10000000201/src v1914>,<0x10000000006/blogbench-1.0 v1716>,<0x10000000005/tmp v18>,<0x100000001fb/client.0 v16>,<0x100000001fa/9fc8f023-d646-4fec-98c7-1f3d972f60a9 v32>,<0x10000000001/sv_1 v55>,<0x10000000000/qa v43>,<0x1/volumes v42>]//[]","memoryvalue":"(3)0x10000000211:[<0x10000000208/bsd-getopt_long.h v40>,<0x10000000201/src v1989>,<0x10000000006/blogbench-1.0 v1743>,<0x10000000005/tmp v35>,<0x100000001fb/client.0 v31>,<0x100000001fa/9fc8f023-d646-4fec-98c7-1f3d972f60a9 v44>,<0x10000000001/sv_1 v67>,<0x10000000000/qa v33>,<0x1/volumes v32>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-11T23:20:09.990 INFO:journalctl@ceph.mds.i.smithi136.stdout:Sep 11 23:20:09 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mds-i[65341]: 2024-09-11T23:20:09.630+0000 7ff35e42b640 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000216 [3,head] /volumes/qa/sv_1/9fc8f023-d646-4fec-98c7-1f3d972f60a9/client.0/tmp/blogbench-1.0/src/commenter.c auth v46 ap=1 s=939 n(v0 rc2024-09-11T23:19:44.893504+0000 b939 1=1+0) caps={15180=pAsLsXsFscr/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x555c90928b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000216:[<0x10000000208/commenter.c v46>,<0x10000000201/src v1914>,<0x10000000006/blogbench-1.0 v1716>,<0x10000000005/tmp v18>,<0x100000001fb/client.0 v16>,<0x100000001fa/9fc8f023-d646-4fec-98c7-1f3d972f60a9 v32>,<0x10000000001/sv_1 v55>,<0x10000000000/qa v43>,<0x1/volumes v42>]//[]","memoryvalue":"(3)0x10000000216:[<0x10000000208/commenter.c v46>,<0x10000000201/src v1989>,<0x10000000006/blogbench-1.0 v1743>,<0x10000000005/tmp v35>,<0x100000001fb/client.0 v31>,<0x100000001fa/9fc8f023-d646-4fec-98c7-1f3d972f60a9 v44>,<0x10000000001/sv_1 v67>,<0x10000000000/qa v33>,<0x1/volumes v32>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-11T23:20:09.990 INFO:journalctl@ceph.mds.i.smithi136.stdout:Sep 11 23:20:09 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mds-i[65341]: 2024-09-11T23:20:09.632+0000 7ff35e42b640 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000021a [3,head] /volumes/qa/sv_1/9fc8f023-d646-4fec-98c7-1f3d972f60a9/client.0/tmp/blogbench-1.0/src/helpers.c auth v66 ap=1 s=4200 n(v0 rc2024-09-11T23:19:44.897504+0000 b4200 1=1+0) caps={15180=pAsLsXsFscr/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x555c90929080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000021a:[<0x10000000208/helpers.c v66>,<0x10000000201/src v1914>,<0x10000000006/blogbench-1.0 v1716>,<0x10000000005/tmp v18>,<0x100000001fb/client.0 v16>,<0x100000001fa/9fc8f023-d646-4fec-98c7-1f3d972f60a9 v32>,<0x10000000001/sv_1 v55>,<0x10000000000/qa v43>,<0x1/volumes v42>]//[]","memoryvalue":"(3)0x1000000021a:[<0x10000000208/helpers.c v66>,<0x10000000201/src v1989>,<0x10000000006/blogbench-1.0 v1743>,<0x10000000005/tmp v35>,<0x100000001fb/client.0 v31>,<0x100000001fa/9fc8f023-d646-4fec-98c7-1f3d972f60a9 v44>,<0x10000000001/sv_1 v67>,<0x10000000000/qa v33>,<0x1/volumes v32>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-11T23:20:09.990 INFO:journalctl@ceph.mds.i.smithi136.stdout:Sep 11 23:20:09 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mds-i[65341]: 2024-09-11T23:20:09.633+0000 7ff35e42b640 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000020a [3,head] /volumes/qa/sv_1/9fc8f023-d646-4fec-98c7-1f3d972f60a9/client.0/tmp/blogbench-1.0/src/mysnprintf.h auth v70 ap=1 s=696 n(v0 rc2024-09-11T23:19:44.880504+0000 b696 1=1+0) caps={15180=pAsLsXsFscr/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x555c90929600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000020a:[<0x10000000208/mysnprintf.h v70>,<0x10000000201/src v1914>,<0x10000000006/blogbench-1.0 v1716>,<0x10000000005/tmp v18>,<0x100000001fb/client.0 v16>,<0x100000001fa/9fc8f023-d646-4fec-98c7-1f3d972f60a9 v32>,<0x10000000001/sv_1 v55>,<0x10000000000/qa v43>,<0x1/volumes v42>]//[]","memoryvalue":"(3)0x1000000020a:[<0x10000000208/mysnprintf.h v70>,<0x10000000201/src v1989>,<0x10000000006/blogbench-1.0 v1743>,<0x10000000005/tmp v35>,<0x100000001fb/client.0 v31>,<0x100000001fa/9fc8f023-d646-4fec-98c7-1f3d972f60a9 v44>,<0x10000000001/sv_1 v67>,<0x10000000000/qa v33>,<0x1/volumes v32>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-11T23:20:09.990 INFO:journalctl@ceph.mds.i.smithi136.stdout:Sep 11 23:20:09 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mds-i[65341]: 2024-09-11T23:20:09.634+0000 7ff35e42b640 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000020f [3,head] /volumes/qa/sv_1/9fc8f023-d646-4fec-98c7-1f3d972f60a9/client.0/tmp/blogbench-1.0/src/blogbench_p.h auth v58 ap=1 s=460 n(v0 rc2024-09-11T23:19:44.886504+0000 b460 1=1+0) caps={15180=pAsLsXsFscr/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x555c9092a100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000020f:[<0x10000000208/blogbench_p.h v58>,<0x10000000201/src v1914>,<0x10000000006/blogbench-1.0 v1716>,<0x10000000005/tmp v18>,<0x100000001fb/client.0 v16>,<0x100000001fa/9fc8f023-d646-4fec-98c7-1f3d972f60a9 v32>,<0x10000000001/sv_1 v55>,<0x10000000000/qa v43>,<0x1/volumes v42>]//[]","memoryvalue":"(3)0x1000000020f:[<0x10000000208/blogbench_p.h v58>,<0x10000000201/src v1989>,<0x10000000006/blogbench-1.0 v1743>,<0x10000000005/tmp v35>,<0x100000001fb/client.0 v31>,<0x100000001fa/9fc8f023-d646-4fec-98c7-1f3d972f60a9 v44>,<0x10000000001/sv_1 v67>,<0x10000000000/qa v33>,<0x1/volumes v32>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-11T23:20:09.990 INFO:journalctl@ceph.mds.i.smithi136.stdout:Sep 11 23:20:09 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mds-i[65341]: 2024-09-11T23:20:09.636+0000 7ff35e42b640 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000020c [3,head] /volumes/qa/sv_1/9fc8f023-d646-4fec-98c7-1f3d972f60a9/client.0/tmp/blogbench-1.0/src/blogbench.h auth v62 ap=1 s=3764 n(v0 rc2024-09-11T23:19:44.882504+0000 b3764 1=1+0) caps={15180=pAsLsXsFscr/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x555c9092ac00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000020c:[<0x10000000208/blogbench.h v62>,<0x10000000201/src v1914>,<0x10000000006/blogbench-1.0 v1716>,<0x10000000005/tmp v18>,<0x100000001fb/client.0 v16>,<0x100000001fa/9fc8f023-d646-4fec-98c7-1f3d972f60a9 v32>,<0x10000000001/sv_1 v55>,<0x10000000000/qa v43>,<0x1/volumes v42>]//[]","memoryvalue":"(3)0x1000000020c:[<0x10000000208/blogbench.h v62>,<0x10000000201/src v1989>,<0x10000000006/blogbench-1.0 v1743>,<0x10000000005/tmp v35>,<0x100000001fb/client.0 v31>,<0x100000001fa/9fc8f023-d646-4fec-98c7-1f3d972f60a9 v44>,<0x10000000001/sv_1 v67>,<0x10000000000/qa v33>,<0x1/volumes v32>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-11T23:20:09.990 INFO:journalctl@ceph.mds.i.smithi136.stdout:Sep 11 23:20:09 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mds-i[65341]: 2024-09-11T23:20:09.637+0000 7ff35e42b640 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000217 [3,head] /volumes/qa/sv_1/9fc8f023-d646-4fec-98c7-1f3d972f60a9/client.0/tmp/blogbench-1.0/src/rewriter.c auth v60 ap=1 s=1581 n(v0 rc2005-02-11T12:50:44.000000+0000 b1581 1=1+0) caps={15180=pAsLsXsFscr/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x555c9092b180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000217:[<0x10000000208/rewriter.c v60>,<0x10000000201/src v1914>,<0x10000000006/blogbench-1.0 v1716>,<0x10000000005/tmp v18>,<0x100000001fb/client.0 v16>,<0x100000001fa/9fc8f023-d646-4fec-98c7-1f3d972f60a9 v32>,<0x10000000001/sv_1 v55>,<0x10000000000/qa v43>,<0x1/volumes v42>]//[]","memoryvalue":"(3)0x10000000217:[<0x10000000208/rewriter.c v60>,<0x10000000201/src v1989>,<0x10000000006/blogbench-1.0 v1743>,<0x10000000005/tmp v35>,<0x100000001fb/client.0 v31>,<0x100000001fa/9fc8f023-d646-4fec-98c7-1f3d972f60a9 v44>,<0x10000000001/sv_1 v67>,<0x10000000000/qa v33>,<0x1/volumes v32>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-11T23:20:09.990 INFO:journalctl@ceph.mds.i.smithi136.stdout:Sep 11 23:20:09 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mds-i[65341]: 2024-09-11T23:20:09.639+0000 7ff35e42b640 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000385 [3,head] /volumes/qa/sv_1/9fc8f023-d646-4fec-98c7-1f3d972f60a9/client.0/tmp/blogbench-1.0/src/blogbench.o auth{0=1} v264 ap=1 s=18944 n(v0 rc2024-09-11T23:19:50.880410+0000 b18944 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={15180=0-4194304@2} caps={15180=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15180 | importingcaps=0 caps=1 replicated=1 dirty=1 authpin=1 scrubqueue=0 0x555c9092b700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000385:[<0x10000000208/blogbench.o v100>,<0x10000000201/src v1914>,<0x10000000006/blogbench-1.0 v1716>,<0x10000000005/tmp v18>,<0x100000001fb/client.0 v16>,<0x100000001fa/9fc8f023-d646-4fec-98c7-1f3d972f60a9 v32>,<0x10000000001/sv_1 v55>,<0x10000000000/qa v43>,<0x1/volumes v42>]//[]","memoryvalue":"(3)0x10000000385:[<0x10000000208/blogbench.o v264>,<0x10000000201/src v1989>,<0x10000000006/blogbench-1.0 v1743>,<0x10000000005/tmp v35>,<0x100000001fb/client.0 v31>,<0x100000001fa/9fc8f023-d646-4fec-98c7-1f3d972f60a9 v44>,<0x10000000001/sv_1 v67>,<0x10000000000/qa v33>,<0x1/volumes v32>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-11T23:20:09.990 INFO:journalctl@ceph.mds.i.smithi136.stdout:Sep 11 23:20:09 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mds-i[65341]: 2024-09-11T23:20:09.639+0000 7ff35e42b640 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000371 [3,head] /volumes/qa/sv_1/9fc8f023-d646-4fec-98c7-1f3d972f60a9/client.0/tmp/blogbench-1.0/src/Makefile auth v246 ap=1 s=13306 n(v0 rc2024-09-11T23:19:50.566415+0000 b13306 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={15180=0-4194304@2} caps={15180=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15180 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x555c905a0000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000371:[<0x10000000208/Makefile v74>,<0x10000000201/src v1914>,<0x10000000006/blogbench-1.0 v1716>,<0x10000000005/tmp v18>,<0x100000001fb/client.0 v16>,<0x100000001fa/9fc8f023-d646-4fec-98c7-1f3d972f60a9 v32>,<0x10000000001/sv_1 v55>,<0x10000000000/qa v43>,<0x1/volumes v42>]//[]","memoryvalue":"(3)0x10000000371:[<0x10000000208/Makefile v246>,<0x10000000201/src v1989>,<0x10000000006/blogbench-1.0 v1743>,<0x10000000005/tmp v35>,<0x100000001fb/client.0 v31>,<0x100000001fa/9fc8f023-d646-4fec-98c7-1f3d972f60a9 v44>,<0x10000000001/sv_1 v67>,<0x10000000000/qa v33>,<0x1/volumes v32>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-11T23:20:10.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.424+0000 7f4e1968a640 1 -- 172.21.15.26:0/1285647762 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4e141036e0 msgr2=0x7f4e14107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:10.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.424+0000 7f4e1968a640 1 --2- 172.21.15.26:0/1285647762 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4e141036e0 0x7f4e14107b90 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f4e08009cd0 tx=0x7f4e0802a8e0 comp rx=0 tx=0).stop 2024-09-11T23:20:10.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.424+0000 7f4e1968a640 1 -- 172.21.15.26:0/1285647762 shutdown_connections 2024-09-11T23:20:10.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.424+0000 7f4e1968a640 1 --2- 172.21.15.26:0/1285647762 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4e141082a0 0x7f4e1410ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:10.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.424+0000 7f4e1968a640 1 --2- 172.21.15.26:0/1285647762 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4e141036e0 0x7f4e14107b90 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:10.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.424+0000 7f4e1968a640 1 --2- 172.21.15.26:0/1285647762 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4e14102d30 0x7f4e14103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:10.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.424+0000 7f4e1968a640 1 -- 172.21.15.26:0/1285647762 >> 172.21.15.26:0/1285647762 conn(0x7f4e140fc7d0 msgr2=0x7f4e140febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:10.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.424+0000 7f4e1968a640 1 -- 172.21.15.26:0/1285647762 shutdown_connections 2024-09-11T23:20:10.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.424+0000 7f4e1968a640 1 -- 172.21.15.26:0/1285647762 wait complete. 2024-09-11T23:20:10.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.424+0000 7f4e1968a640 1 Processor -- start 2024-09-11T23:20:10.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.424+0000 7f4e1968a640 1 -- start start 2024-09-11T23:20:10.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.425+0000 7f4e1968a640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4e14102d30 0x7f4e141985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:10.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.425+0000 7f4e1968a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4e141036e0 0x7f4e14198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:10.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.425+0000 7f4e1968a640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4e141082a0 0x7f4e1419ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:10.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.425+0000 7f4e1968a640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f4e141120a0 con 0x7f4e141036e0 2024-09-11T23:20:10.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.425+0000 7f4e1968a640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f4e14111f20 con 0x7f4e141082a0 2024-09-11T23:20:10.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.425+0000 7f4e1968a640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f4e14112220 con 0x7f4e14102d30 2024-09-11T23:20:10.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.425+0000 7f4e127fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4e141036e0 0x7f4e14198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:10.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.425+0000 7f4e12ffd640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4e14102d30 0x7f4e141985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:10.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.425+0000 7f4e127fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4e141036e0 0x7f4e14198b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48578/0 (socket says 172.21.15.26:48578) 2024-09-11T23:20:10.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.425+0000 7f4e127fc640 1 -- 172.21.15.26:0/1538687858 learned_addr learned my addr 172.21.15.26:0/1538687858 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:20:10.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.425+0000 7f4e137fe640 1 --2- 172.21.15.26:0/1538687858 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4e141082a0 0x7f4e1419ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:10.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.425+0000 7f4e127fc640 1 -- 172.21.15.26:0/1538687858 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4e14102d30 msgr2=0x7f4e141985c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:10.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.425+0000 7f4e127fc640 1 --2- 172.21.15.26:0/1538687858 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4e14102d30 0x7f4e141985c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:10.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.425+0000 7f4e127fc640 1 -- 172.21.15.26:0/1538687858 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4e141082a0 msgr2=0x7f4e1419ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:10.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.425+0000 7f4e127fc640 1 --2- 172.21.15.26:0/1538687858 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4e141082a0 0x7f4e1419ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:10.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.425+0000 7f4e127fc640 1 -- 172.21.15.26:0/1538687858 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e1419d570 con 0x7f4e141036e0 2024-09-11T23:20:10.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.425+0000 7f4e127fc640 1 --2- 172.21.15.26:0/1538687858 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4e141036e0 0x7f4e14198b00 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f4e08000f80 tx=0x7f4e08031e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:10.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.426+0000 7f4dfbfff640 1 -- 172.21.15.26:0/1538687858 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e08004040 con 0x7f4e141036e0 2024-09-11T23:20:10.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.426+0000 7f4e1968a640 1 -- 172.21.15.26:0/1538687858 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e1419d800 con 0x7f4e141036e0 2024-09-11T23:20:10.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.426+0000 7f4dfbfff640 1 -- 172.21.15.26:0/1538687858 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e080046c0 con 0x7f4e141036e0 2024-09-11T23:20:10.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.426+0000 7f4dfbfff640 1 -- 172.21.15.26:0/1538687858 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e08030e40 con 0x7f4e141036e0 2024-09-11T23:20:10.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.426+0000 7f4e1968a640 1 -- 172.21.15.26:0/1538687858 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e141a5140 con 0x7f4e141036e0 2024-09-11T23:20:10.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.429+0000 7f4dfbfff640 1 -- 172.21.15.26:0/1538687858 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f4e080305a0 con 0x7f4e141036e0 2024-09-11T23:20:10.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.430+0000 7f4dfbfff640 1 --2- 172.21.15.26:0/1538687858 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f4dfc0a2d70 0x7f4dfc0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:10.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.430+0000 7f4e1ba63740 1 Processor -- start 2024-09-11T23:20:10.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.430+0000 7f4e1ba63740 1 -- start start 2024-09-11T23:20:10.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.430+0000 7f4e12ffd640 1 --2- 172.21.15.26:0/1538687858 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f4dfc0a2d70 0x7f4dfc0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:10.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.430+0000 7f4dfbfff640 1 -- 172.21.15.26:0/1538687858 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f4e0802e030 con 0x7f4e141036e0 2024-09-11T23:20:10.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.430+0000 7f4e1ba63740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c59f0b64a0 0x55c59f0b6880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:10.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.430+0000 7f4e1ba63740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c59f0b6dc0 0x55c59f0c1140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:10.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.430+0000 7f4e1ba63740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c59f0c1680 0x55c59f0c3a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:10.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.430+0000 7f4e1ba63740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55c59f01c2e0 con 0x55c59f0c1680 2024-09-11T23:20:10.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.430+0000 7f4e1ba63740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55c59efc3ce0 con 0x55c59f0b6dc0 2024-09-11T23:20:10.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.430+0000 7f4e1ba63740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55c59f0182a0 con 0x55c59f0b64a0 2024-09-11T23:20:10.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.430+0000 7f4e137fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c59f0c1680 0x55c59f0c3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:10.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.431+0000 7f4e137fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c59f0c1680 0x55c59f0c3a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48590/0 (socket says 172.21.15.26:48590) 2024-09-11T23:20:10.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.431+0000 7f4e127fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c59f0b6dc0 0x55c59f0c1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:10.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.431+0000 7f4e137fe640 1 -- 172.21.15.26:0/1036864489 learned_addr learned my addr 172.21.15.26:0/1036864489 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:20:10.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.431+0000 7f4e12ffd640 1 --2- 172.21.15.26:0/1036864489 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c59f0b64a0 0x55c59f0b6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:10.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.431+0000 7f4e12ffd640 1 --2- 172.21.15.26:0/1538687858 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f4dfc0a2d70 0x7f4dfc0a5230 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f4e04009c20 tx=0x7f4e04011810 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:10.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.431+0000 7f4e137fe640 1 -- 172.21.15.26:0/1036864489 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c59f0b64a0 msgr2=0x55c59f0b6880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:20:10.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.431+0000 7f4e137fe640 1 --2- 172.21.15.26:0/1036864489 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c59f0b64a0 0x55c59f0b6880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:10.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.431+0000 7f4e137fe640 1 -- 172.21.15.26:0/1036864489 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c59f0b6dc0 msgr2=0x55c59f0c1140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:20:10.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.431+0000 7f4e137fe640 1 --2- 172.21.15.26:0/1036864489 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c59f0b6dc0 0x55c59f0c1140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:10.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.431+0000 7f4e137fe640 1 -- 172.21.15.26:0/1036864489 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c59efa8440 con 0x55c59f0c1680 2024-09-11T23:20:10.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.431+0000 7f4e12ffd640 1 --2- 172.21.15.26:0/1036864489 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c59f0b64a0 0x55c59f0b6880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:20:10.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.431+0000 7f4e137fe640 1 --2- 172.21.15.26:0/1036864489 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c59f0c1680 0x55c59f0c3a70 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f4e0c00b6b0 tx=0x7f4e0c004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfefc4712cd7af61 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:10.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.431+0000 7f4df97fa640 1 -- 172.21.15.26:0/1036864489 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e0c003b20 con 0x55c59f0c1680 2024-09-11T23:20:10.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.431+0000 7f4df97fa640 1 -- 172.21.15.26:0/1036864489 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e0c015070 con 0x55c59f0c1680 2024-09-11T23:20:10.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.431+0000 7f4df97fa640 1 -- 172.21.15.26:0/1036864489 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e0c0106a0 con 0x55c59f0c1680 2024-09-11T23:20:10.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.432+0000 7f4e1ba63740 1 -- 172.21.15.26:0/1036864489 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c59f0c1680 msgr2=0x55c59f0c3a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:10.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.432+0000 7f4e1ba63740 1 --2- 172.21.15.26:0/1036864489 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c59f0c1680 0x55c59f0c3a70 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f4e0c00b6b0 tx=0x7f4e0c004610 comp rx=0 tx=0).stop 2024-09-11T23:20:10.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.432+0000 7f4e1ba63740 1 -- 172.21.15.26:0/1036864489 shutdown_connections 2024-09-11T23:20:10.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.432+0000 7f4e1ba63740 1 --2- 172.21.15.26:0/1036864489 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c59f0b6dc0 0x55c59f0c1140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:10.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.432+0000 7f4e1ba63740 1 --2- 172.21.15.26:0/1036864489 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c59f0c1680 0x55c59f0c3a70 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:10.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.432+0000 7f4e1ba63740 1 --2- 172.21.15.26:0/1036864489 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c59f0b64a0 0x55c59f0b6880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:10.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.432+0000 7f4e1ba63740 1 -- 172.21.15.26:0/1036864489 >> 172.21.15.26:0/1036864489 conn(0x55c59f0b2a30 msgr2=0x55c59f0b2e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:10.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.432+0000 7f4e1ba63740 1 -- 172.21.15.26:0/1036864489 shutdown_connections 2024-09-11T23:20:10.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.432+0000 7f4e1ba63740 1 -- 172.21.15.26:0/1036864489 wait complete. 2024-09-11T23:20:10.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.432+0000 7f4e1ba63740 1 -- start start 2024-09-11T23:20:10.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.433+0000 7f4e1ba63740 1 Processor -- start 2024-09-11T23:20:10.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.433+0000 7f4e1ba63740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c59f0b64a0 0x55c59f0c0f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:10.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.433+0000 7f4e1ba63740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c59f0b6dc0 0x55c59f0be220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:10.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.433+0000 7f4e1ba63740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c59f0c1680 0x55c59f0be760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:10.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.433+0000 7f4e1ba63740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55c59f0c4250 con 0x55c59f0b6dc0 2024-09-11T23:20:10.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.433+0000 7f4e1ba63740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55c59efd7050 con 0x55c59f0b64a0 2024-09-11T23:20:10.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.433+0000 7f4e1ba63740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55c59f0c43d0 con 0x55c59f0c1680 2024-09-11T23:20:10.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.433+0000 7f4e1ba63740 10 client.0 _ll_register_callbacks cb 0x55c59f017160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:20:10.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.433+0000 7f4e127fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c59f0b6dc0 0x55c59f0be220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:10.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.433+0000 7f4e12ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c59f0b64a0 0x55c59f0c0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:10.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.433+0000 7f4e127fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c59f0b6dc0 0x55c59f0be220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48598/0 (socket says 172.21.15.26:48598) 2024-09-11T23:20:10.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.433+0000 7f4e127fc640 1 -- 172.21.15.26:0/1085353009 learned_addr learned my addr 172.21.15.26:0/1085353009 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:20:10.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.433+0000 7f4e137fe640 1 --2- 172.21.15.26:0/1085353009 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c59f0c1680 0x55c59f0be760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:10.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.433+0000 7f4e127fc640 1 -- 172.21.15.26:0/1085353009 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c59f0c1680 msgr2=0x55c59f0be760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:20:10.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.433+0000 7f4e127fc640 1 --2- 172.21.15.26:0/1085353009 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c59f0c1680 0x55c59f0be760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:10.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.433+0000 7f4e127fc640 1 -- 172.21.15.26:0/1085353009 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c59f0b64a0 msgr2=0x55c59f0c0f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:20:10.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.433+0000 7f4e127fc640 1 --2- 172.21.15.26:0/1085353009 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c59f0b64a0 0x55c59f0c0f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:10.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.433+0000 7f4e127fc640 1 -- 172.21.15.26:0/1085353009 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c59ed8d3c0 con 0x55c59f0b6dc0 2024-09-11T23:20:10.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.433+0000 7f4e127fc640 1 --2- 172.21.15.26:0/1085353009 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c59f0b6dc0 0x55c59f0be220 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f4e080b9720 tx=0x7f4e080b90e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d20dfc5f60333a0f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:10.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.433+0000 7f4df6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:20:10.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.434+0000 7f4df6ff5640 1 -- 172.21.15.26:0/1085353009 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e080d9030 con 0x55c59f0b6dc0 2024-09-11T23:20:10.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.434+0000 7f4df6ff5640 1 -- 172.21.15.26:0/1085353009 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f4e0809bb70 con 0x55c59f0b6dc0 2024-09-11T23:20:10.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.434+0000 7f4e1ba63740 1 -- 172.21.15.26:0/1085353009 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4e0c015070 con 0x55c59f0b6dc0 2024-09-11T23:20:10.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.434+0000 7f4df6ff5640 1 -- 172.21.15.26:0/1085353009 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f4e080ee300 con 0x55c59f0b6dc0 2024-09-11T23:20:10.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.434+0000 7f4e1ba63740 10 client.15954 fetch_fsmap learned FSMap version 32 2024-09-11T23:20:10.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.434+0000 7f4e1ba63740 1 -- 172.21.15.26:0/1085353009 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55c59f0cfbc0 con 0x55c59f0b6dc0 2024-09-11T23:20:10.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.434+0000 7f4df6ff5640 1 -- 172.21.15.26:0/1085353009 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e0c015070 con 0x55c59f0b6dc0 2024-09-11T23:20:10.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.434+0000 7f4df6ff5640 1 -- 172.21.15.26:0/1085353009 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 32) ==== 8394+0+0 (secure 0 0 0) 0x7f4e0804e070 con 0x55c59f0b6dc0 2024-09-11T23:20:10.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.434+0000 7f4e1ba63740 10 client.15954 fetch_fsmap finished waiting for FSMap version 32 2024-09-11T23:20:10.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.434+0000 7f4e1ba63740 10 client.15954 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:20:10.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.434+0000 7f4e1ba63740 20 client.15954 populate_metadata read hostname 'smithi026' 2024-09-11T23:20:10.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.434+0000 7f4e1ba63740 1 --2- 172.21.15.26:0/1085353009 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55c59f0d10a0 0x55c59f0d3520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:10.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.434+0000 7f4e1ba63740 4 client.15954 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:20:10.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.434+0000 7f4e1ba63740 1 -- 172.21.15.26:0/1085353009 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c59f029850 con 0x55c59f0d10a0 2024-09-11T23:20:10.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.435+0000 7f4e12ffd640 1 --2- 172.21.15.26:0/1085353009 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55c59f0d10a0 0x55c59f0d3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:10.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.435+0000 7f4e12ffd640 1 --2- 172.21.15.26:0/1085353009 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55c59f0d10a0 0x55c59f0d3520 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a754399ef7d7bdca server_cookie=be7360199789e610 in_seq=0 out_seq=0 2024-09-11T23:20:10.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.435+0000 7f4df6ff5640 10 client.15954 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:20:10.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.436+0000 7f4df6ff5640 1 -- 172.21.15.26:0/1085353009 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c59f029850 con 0x55c59f0d10a0 2024-09-11T23:20:10.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.436+0000 7f4df6ff5640 10 client.15954 handle_command_reply: tid=0 2024-09-11T23:20:10.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.436+0000 7f4e1ba63740 1 client.15954 shutdown 2024-09-11T23:20:10.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.436+0000 7f4e1ba63740 1 -- 172.21.15.26:0/1085353009 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c59f0b6dc0 msgr2=0x55c59f0be220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:10.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.436+0000 7f4e1ba63740 1 --2- 172.21.15.26:0/1085353009 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c59f0b6dc0 0x55c59f0be220 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f4e080b9720 tx=0x7f4e080b90e0 comp rx=0 tx=0).stop 2024-09-11T23:20:10.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.436+0000 7f4e1ba63740 1 -- 172.21.15.26:0/1085353009 shutdown_connections 2024-09-11T23:20:10.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.436+0000 7f4e1ba63740 1 --2- 172.21.15.26:0/1085353009 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55c59f0d10a0 0x55c59f0d3520 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:10.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.436+0000 7f4e1ba63740 1 --2- 172.21.15.26:0/1085353009 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c59f0c1680 0x55c59f0be760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:10.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.436+0000 7f4e1ba63740 1 --2- 172.21.15.26:0/1085353009 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c59f0b6dc0 0x55c59f0be220 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:10.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.436+0000 7f4df6ff5640 0 client.15954 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:20:10.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.436+0000 7f4e1ba63740 1 --2- 172.21.15.26:0/1085353009 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c59f0b64a0 0x55c59f0c0f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:10.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.436+0000 7f4e1ba63740 1 -- 172.21.15.26:0/1085353009 >> 172.21.15.26:0/1085353009 conn(0x55c59f0b2a30 msgr2=0x55c59f0b8710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:10.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.436+0000 7f4e1ba63740 1 -- 172.21.15.26:0/1085353009 shutdown_connections 2024-09-11T23:20:10.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.436+0000 7f4e1ba63740 1 -- 172.21.15.26:0/1085353009 wait complete. 2024-09-11T23:20:10.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.436+0000 7f4e1ba63740 20 client.15954 trim_cache size 0 max 16384 2024-09-11T23:20:10.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.454+0000 7f4e1ba63740 1 Processor -- start 2024-09-11T23:20:10.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.454+0000 7f4e1ba63740 1 -- start start 2024-09-11T23:20:10.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.454+0000 7f4e1ba63740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c59f0b64a0 0x55c59f0cd4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:10.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.454+0000 7f4e1ba63740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c59f0b6dc0 0x55c59f0b97f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:10.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.454+0000 7f4e1ba63740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c59f0c1680 0x55c59f0d8e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:10.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.454+0000 7f4e1ba63740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f4e141ad870 con 0x55c59f0b64a0 2024-09-11T23:20:10.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.454+0000 7f4e1ba63740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f4e141af530 con 0x55c59f0c1680 2024-09-11T23:20:10.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.454+0000 7f4e1ba63740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f4e141adf40 con 0x55c59f0b6dc0 2024-09-11T23:20:10.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.454+0000 7f4e12ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c59f0b64a0 0x55c59f0cd4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:10.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.454+0000 7f4e12ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c59f0b64a0 0x55c59f0cd4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48604/0 (socket says 172.21.15.26:48604) 2024-09-11T23:20:10.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.454+0000 7f4e12ffd640 1 -- 172.21.15.26:0/776263007 learned_addr learned my addr 172.21.15.26:0/776263007 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:20:10.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.454+0000 7f4e137fe640 1 --2- 172.21.15.26:0/776263007 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c59f0c1680 0x55c59f0d8e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:10.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.454+0000 7f4e127fc640 1 --2- 172.21.15.26:0/776263007 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c59f0b6dc0 0x55c59f0b97f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:10.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.454+0000 7f4e12ffd640 1 -- 172.21.15.26:0/776263007 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c59f0b6dc0 msgr2=0x55c59f0b97f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:20:10.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.454+0000 7f4e12ffd640 1 --2- 172.21.15.26:0/776263007 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c59f0b6dc0 0x55c59f0b97f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:10.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.454+0000 7f4e12ffd640 1 -- 172.21.15.26:0/776263007 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c59f0c1680 msgr2=0x55c59f0d8e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:20:10.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.454+0000 7f4e12ffd640 1 --2- 172.21.15.26:0/776263007 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c59f0c1680 0x55c59f0d8e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:10.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.454+0000 7f4e12ffd640 1 -- 172.21.15.26:0/776263007 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c59f0c12e0 con 0x55c59f0b64a0 2024-09-11T23:20:10.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.454+0000 7f4e137fe640 1 --2- 172.21.15.26:0/776263007 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c59f0c1680 0x55c59f0d8e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:20:10.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.454+0000 7f4e12ffd640 1 --2- 172.21.15.26:0/776263007 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c59f0b64a0 0x55c59f0cd4b0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x55c59f0d3da0 tx=0x7f4e0401c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=855737e32cd1a4ef server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:10.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.455+0000 7f4df8ff9640 1 -- 172.21.15.26:0/776263007 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e04017040 con 0x55c59f0b64a0 2024-09-11T23:20:10.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.455+0000 7f4df8ff9640 1 -- 172.21.15.26:0/776263007 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e04019050 con 0x55c59f0b64a0 2024-09-11T23:20:10.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.455+0000 7f4df8ff9640 1 -- 172.21.15.26:0/776263007 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e04004c30 con 0x55c59f0b64a0 2024-09-11T23:20:10.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.455+0000 7f4e1ba63740 1 -- 172.21.15.26:0/776263007 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c59f0b64a0 msgr2=0x55c59f0cd4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:10.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.455+0000 7f4e1ba63740 1 --2- 172.21.15.26:0/776263007 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c59f0b64a0 0x55c59f0cd4b0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x55c59f0d3da0 tx=0x7f4e0401c000 comp rx=0 tx=0).stop 2024-09-11T23:20:10.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.455+0000 7f4e1ba63740 1 -- 172.21.15.26:0/776263007 shutdown_connections 2024-09-11T23:20:10.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.455+0000 7f4e1ba63740 1 --2- 172.21.15.26:0/776263007 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c59f0c1680 0x55c59f0d8e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:10.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.455+0000 7f4e1ba63740 1 --2- 172.21.15.26:0/776263007 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c59f0b6dc0 0x55c59f0b97f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:10.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.455+0000 7f4e1ba63740 1 --2- 172.21.15.26:0/776263007 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c59f0b64a0 0x55c59f0cd4b0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:10.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.455+0000 7f4e1ba63740 1 -- 172.21.15.26:0/776263007 >> 172.21.15.26:0/776263007 conn(0x55c59f0b2a30 msgr2=0x55c59f0b90e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:10.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.456+0000 7f4e1ba63740 1 -- 172.21.15.26:0/776263007 shutdown_connections 2024-09-11T23:20:10.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.456+0000 7f4e1ba63740 1 -- 172.21.15.26:0/776263007 wait complete. 2024-09-11T23:20:10.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.456+0000 7f4e1ba63740 1 -- start start 2024-09-11T23:20:10.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.456+0000 7f4e1ba63740 1 Processor -- start 2024-09-11T23:20:10.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.456+0000 7f4e1ba63740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c59f0b64a0 0x55c59f0c9ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:10.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.456+0000 7f4e1ba63740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c59f0b6dc0 0x55c59f0c51f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:10.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.456+0000 7f4e1ba63740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c59f0c1680 0x55c59f0c5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:10.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.456+0000 7f4e1ba63740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f4e141aeda0 con 0x55c59f0b6dc0 2024-09-11T23:20:10.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.456+0000 7f4e1ba63740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f4e141ae610 con 0x55c59f0b64a0 2024-09-11T23:20:10.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.456+0000 7f4e1ba63740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f4e141ab850 con 0x55c59f0c1680 2024-09-11T23:20:10.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.456+0000 7f4e1ba63740 10 client.0 _ll_register_callbacks cb 0x7f4e08003ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:20:10.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.456+0000 7f4e127fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c59f0b6dc0 0x55c59f0c51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:10.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.456+0000 7f4e127fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c59f0b6dc0 0x55c59f0c51f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48614/0 (socket says 172.21.15.26:48614) 2024-09-11T23:20:10.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.456+0000 7f4e127fc640 1 -- 172.21.15.26:0/3628068709 learned_addr learned my addr 172.21.15.26:0/3628068709 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:20:10.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.456+0000 7f4e137fe640 1 --2- 172.21.15.26:0/3628068709 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c59f0c1680 0x55c59f0c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:10.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.457+0000 7f4e12ffd640 1 --2- 172.21.15.26:0/3628068709 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c59f0b64a0 0x55c59f0c9ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:10.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.457+0000 7f4e127fc640 1 -- 172.21.15.26:0/3628068709 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c59f0c1680 msgr2=0x55c59f0c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:20:10.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.457+0000 7f4e127fc640 1 --2- 172.21.15.26:0/3628068709 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c59f0c1680 0x55c59f0c5980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:10.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.457+0000 7f4e127fc640 1 -- 172.21.15.26:0/3628068709 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c59f0b64a0 msgr2=0x55c59f0c9ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:20:10.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.457+0000 7f4e127fc640 1 --2- 172.21.15.26:0/3628068709 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c59f0b64a0 0x55c59f0c9ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:10.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.457+0000 7f4e127fc640 1 -- 172.21.15.26:0/3628068709 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c59f13f210 con 0x55c59f0b6dc0 2024-09-11T23:20:10.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.457+0000 7f4e127fc640 1 --2- 172.21.15.26:0/3628068709 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c59f0b6dc0 0x55c59f0c51f0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f4e080bb000 tx=0x7f4e08004640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43daf71a29bf07b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:10.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.457+0000 7f4df6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:20:10.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.457+0000 7f4df6ff5640 1 -- 172.21.15.26:0/3628068709 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e080d9050 con 0x55c59f0b6dc0 2024-09-11T23:20:10.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.457+0000 7f4df6ff5640 1 -- 172.21.15.26:0/3628068709 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f4e080b8ca0 con 0x55c59f0b6dc0 2024-09-11T23:20:10.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.457+0000 7f4e1ba63740 1 -- 172.21.15.26:0/3628068709 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4e04019050 con 0x55c59f0b6dc0 2024-09-11T23:20:10.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.457+0000 7f4df6ff5640 1 -- 172.21.15.26:0/3628068709 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f4e080bc890 con 0x55c59f0b6dc0 2024-09-11T23:20:10.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.457+0000 7f4e1ba63740 10 client.15960 fetch_fsmap learned FSMap version 32 2024-09-11T23:20:10.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.457+0000 7f4e1ba63740 1 -- 172.21.15.26:0/3628068709 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55c59f140370 con 0x55c59f0b6dc0 2024-09-11T23:20:10.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.457+0000 7f4df6ff5640 1 -- 172.21.15.26:0/3628068709 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e04019050 con 0x55c59f0b6dc0 2024-09-11T23:20:10.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.458+0000 7f4df6ff5640 1 -- 172.21.15.26:0/3628068709 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 32) ==== 8394+0+0 (secure 0 0 0) 0x7f4e0804e040 con 0x55c59f0b6dc0 2024-09-11T23:20:10.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.458+0000 7f4e1ba63740 10 client.15960 fetch_fsmap finished waiting for FSMap version 32 2024-09-11T23:20:10.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.458+0000 7f4e1ba63740 10 client.15960 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:20:10.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.458+0000 7f4e1ba63740 20 client.15960 populate_metadata read hostname 'smithi026' 2024-09-11T23:20:10.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.458+0000 7f4e1ba63740 1 --2- 172.21.15.26:0/3628068709 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55c59f140990 0x55c59f142db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:10.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.458+0000 7f4e1ba63740 4 client.15960 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T23:20:10.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.458+0000 7f4e1ba63740 1 -- 172.21.15.26:0/3628068709 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c59f023d60 con 0x55c59f140990 2024-09-11T23:20:10.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.458+0000 7f4e12ffd640 1 --2- 172.21.15.26:0/3628068709 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55c59f140990 0x55c59f142db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:10.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.458+0000 7f4e12ffd640 1 --2- 172.21.15.26:0/3628068709 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55c59f140990 0x55c59f142db0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c368c8c71b0b3774 server_cookie=39ba73a59913dcda in_seq=0 out_seq=0 2024-09-11T23:20:10.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.458+0000 7f4df6ff5640 10 client.15960 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:20:10.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.458+0000 7f4df6ff5640 1 -- 172.21.15.26:0/3628068709 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55c59f023d60 con 0x55c59f140990 2024-09-11T23:20:10.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.458+0000 7f4df6ff5640 10 client.15960 handle_command_reply: tid=0 2024-09-11T23:20:10.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.458+0000 7f4e1ba63740 1 client.15960 shutdown 2024-09-11T23:20:10.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.458+0000 7f4e1ba63740 1 -- 172.21.15.26:0/3628068709 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c59f0b6dc0 msgr2=0x55c59f0c51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:10.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.458+0000 7f4e1ba63740 1 --2- 172.21.15.26:0/3628068709 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c59f0b6dc0 0x55c59f0c51f0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f4e080bb000 tx=0x7f4e08004640 comp rx=0 tx=0).stop 2024-09-11T23:20:10.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.459+0000 7f4e1ba63740 1 -- 172.21.15.26:0/3628068709 shutdown_connections 2024-09-11T23:20:10.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.459+0000 7f4e1ba63740 1 --2- 172.21.15.26:0/3628068709 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55c59f140990 0x55c59f142db0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:10.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.459+0000 7f4e1ba63740 1 --2- 172.21.15.26:0/3628068709 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c59f0c1680 0x55c59f0c5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:10.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.459+0000 7f4e1ba63740 1 --2- 172.21.15.26:0/3628068709 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c59f0b6dc0 0x55c59f0c51f0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:10.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.459+0000 7f4e1ba63740 1 --2- 172.21.15.26:0/3628068709 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c59f0b64a0 0x55c59f0c9ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:10.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.459+0000 7f4e1ba63740 1 -- 172.21.15.26:0/3628068709 >> 172.21.15.26:0/3628068709 conn(0x55c59f0b2a30 msgr2=0x55c59f0b8710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:10.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.459+0000 7f4df6ff5640 0 client.15960 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:20:10.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.459+0000 7f4e1ba63740 1 -- 172.21.15.26:0/3628068709 shutdown_connections 2024-09-11T23:20:10.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.459+0000 7f4e1ba63740 1 -- 172.21.15.26:0/3628068709 wait complete. 2024-09-11T23:20:10.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.459+0000 7f4e1ba63740 20 client.15960 trim_cache size 0 max 16384 2024-09-11T23:20:10.466 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-11T23:20:10.466 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-11T23:20:10.466 INFO:teuthology.orchestra.run.smithi026.stdout: "scrubs": { 2024-09-11T23:20:10.466 INFO:teuthology.orchestra.run.smithi026.stdout: "a4be7cc5-004d-4c94-bbfa-1fe107d6af79": { 2024-09-11T23:20:10.466 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/", 2024-09-11T23:20:10.466 INFO:teuthology.orchestra.run.smithi026.stdout: "tag": "a4be7cc5-004d-4c94-bbfa-1fe107d6af79", 2024-09-11T23:20:10.466 INFO:teuthology.orchestra.run.smithi026.stdout: "options": "recursive,force" 2024-09-11T23:20:10.466 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:20:10.466 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:20:10.466 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-11T23:20:10.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.459+0000 7f4e1968a640 1 -- 172.21.15.26:0/1538687858 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f4dfc0a2d70 msgr2=0x7f4dfc0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:10.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.459+0000 7f4e1968a640 1 --2- 172.21.15.26:0/1538687858 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f4dfc0a2d70 0x7f4dfc0a5230 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f4e04009c20 tx=0x7f4e04011810 comp rx=0 tx=0).stop 2024-09-11T23:20:10.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.459+0000 7f4e1968a640 1 -- 172.21.15.26:0/1538687858 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4e141036e0 msgr2=0x7f4e14198b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:10.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.459+0000 7f4e1968a640 1 --2- 172.21.15.26:0/1538687858 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4e141036e0 0x7f4e14198b00 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f4e08000f80 tx=0x7f4e08031e60 comp rx=0 tx=0).stop 2024-09-11T23:20:10.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.459+0000 7f4e1968a640 1 -- 172.21.15.26:0/1538687858 shutdown_connections 2024-09-11T23:20:10.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.459+0000 7f4e1968a640 1 --2- 172.21.15.26:0/1538687858 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f4dfc0a2d70 0x7f4dfc0a5230 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:10.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.459+0000 7f4e1968a640 1 --2- 172.21.15.26:0/1538687858 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4e141082a0 0x7f4e1419ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:10.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.459+0000 7f4e1968a640 1 --2- 172.21.15.26:0/1538687858 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4e141036e0 0x7f4e14198b00 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:10.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.459+0000 7f4e1968a640 1 --2- 172.21.15.26:0/1538687858 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4e14102d30 0x7f4e141985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:10.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.459+0000 7f4e1968a640 1 -- 172.21.15.26:0/1538687858 >> 172.21.15.26:0/1538687858 conn(0x7f4e140fc7d0 msgr2=0x7f4e14105de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:10.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.460+0000 7f4e1968a640 1 -- 172.21.15.26:0/1538687858 shutdown_connections 2024-09-11T23:20:10.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:10.460+0000 7f4e1968a640 1 -- 172.21.15.26:0/1538687858 wait complete. 2024-09-11T23:20:10.553 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:10 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:10.553 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:10 smithi076 ceph-mon[35296]: Scrub error on inode 0x1000000020b (/volumes/qa/sv_1/9fc8f023-d646-4fec-98c7-1f3d972f60a9/client.0/tmp/blogbench-1.0/src/blogbench.c) see mds.i log and `damage ls` output for details 2024-09-11T23:20:10.553 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:10 smithi076 ceph-mon[35296]: Scrub error on inode 0x10000000379 (/volumes/qa/sv_1/9fc8f023-d646-4fec-98c7-1f3d972f60a9/client.0/tmp/blogbench-1.0/src/.deps) see mds.i log and `damage ls` output for details 2024-09-11T23:20:10.553 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:10 smithi076 ceph-mon[35296]: Scrub error on inode 0x10000000387 (/volumes/qa/sv_1/9fc8f023-d646-4fec-98c7-1f3d972f60a9/client.0/tmp/blogbench-1.0/src/process.o) see mds.i log and `damage ls` output for details 2024-09-11T23:20:10.553 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:10 smithi076 ceph-mon[35296]: Scrub error on inode 0x10000000215 (/volumes/qa/sv_1/9fc8f023-d646-4fec-98c7-1f3d972f60a9/client.0/tmp/blogbench-1.0/src/globals.h) see mds.i log and `damage ls` output for details 2024-09-11T23:20:10.553 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:10 smithi076 ceph-mon[35296]: Scrub error on inode 0x10000000219 (/volumes/qa/sv_1/9fc8f023-d646-4fec-98c7-1f3d972f60a9/client.0/tmp/blogbench-1.0/src/writer.c) see mds.i log and `damage ls` output for details 2024-09-11T23:20:10.553 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:10 smithi076 ceph-mon[35296]: Scrub error on inode 0x10000000209 (/volumes/qa/sv_1/9fc8f023-d646-4fec-98c7-1f3d972f60a9/client.0/tmp/blogbench-1.0/src/mysnprintf.c) see mds.i log and `damage ls` output for details 2024-09-11T23:20:10.553 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:10 smithi076 ceph-mon[35296]: Scrub error on inode 0x10000000211 (/volumes/qa/sv_1/9fc8f023-d646-4fec-98c7-1f3d972f60a9/client.0/tmp/blogbench-1.0/src/bsd-getopt_long.h) see mds.i log and `damage ls` output for details 2024-09-11T23:20:10.553 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:10 smithi076 ceph-mon[35296]: Scrub error on inode 0x10000000216 (/volumes/qa/sv_1/9fc8f023-d646-4fec-98c7-1f3d972f60a9/client.0/tmp/blogbench-1.0/src/commenter.c) see mds.i log and `damage ls` output for details 2024-09-11T23:20:10.553 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:10 smithi076 ceph-mon[35296]: Scrub error on inode 0x1000000021a (/volumes/qa/sv_1/9fc8f023-d646-4fec-98c7-1f3d972f60a9/client.0/tmp/blogbench-1.0/src/helpers.c) see mds.i log and `damage ls` output for details 2024-09-11T23:20:10.553 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:10 smithi076 ceph-mon[35296]: Scrub error on inode 0x1000000020a (/volumes/qa/sv_1/9fc8f023-d646-4fec-98c7-1f3d972f60a9/client.0/tmp/blogbench-1.0/src/mysnprintf.h) see mds.i log and `damage ls` output for details 2024-09-11T23:20:10.554 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:10 smithi076 ceph-mon[35296]: Scrub error on inode 0x1000000020f (/volumes/qa/sv_1/9fc8f023-d646-4fec-98c7-1f3d972f60a9/client.0/tmp/blogbench-1.0/src/blogbench_p.h) see mds.i log and `damage ls` output for details 2024-09-11T23:20:10.554 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:10 smithi076 ceph-mon[35296]: Scrub error on inode 0x1000000020c (/volumes/qa/sv_1/9fc8f023-d646-4fec-98c7-1f3d972f60a9/client.0/tmp/blogbench-1.0/src/blogbench.h) see mds.i log and `damage ls` output for details 2024-09-11T23:20:10.554 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:10 smithi076 ceph-mon[35296]: Scrub error on inode 0x10000000217 (/volumes/qa/sv_1/9fc8f023-d646-4fec-98c7-1f3d972f60a9/client.0/tmp/blogbench-1.0/src/rewriter.c) see mds.i log and `damage ls` output for details 2024-09-11T23:20:10.554 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:10 smithi076 ceph-mon[35296]: Scrub error on inode 0x10000000385 (/volumes/qa/sv_1/9fc8f023-d646-4fec-98c7-1f3d972f60a9/client.0/tmp/blogbench-1.0/src/blogbench.o) see mds.i log and `damage ls` output for details 2024-09-11T23:20:10.554 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:10 smithi076 ceph-mon[35296]: Scrub error on inode 0x10000000371 (/volumes/qa/sv_1/9fc8f023-d646-4fec-98c7-1f3d972f60a9/client.0/tmp/blogbench-1.0/src/Makefile) see mds.i log and `damage ls` output for details 2024-09-11T23:20:10.739 INFO:journalctl@ceph.mds.i.smithi136.stdout:Sep 11 23:20:10 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mds-i[65341]: 2024-09-11T23:20:10.339+0000 7ff35e42b640 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000213 [3,head] /volumes/qa/sv_1/9fc8f023-d646-4fec-98c7-1f3d972f60a9/client.0/tmp/blogbench-1.0/src/Makefile.in auth v72 ap=1 s=13775 n(v0 rc2024-09-11T23:19:44.890504+0000 b13775 1=1+0) caps={15180=pAsLsXsFscr/pFscr@1},l=15180 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x555c905a0580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000213:[<0x10000000208/Makefile.in v72>,<0x10000000201/src v1914>,<0x10000000006/blogbench-1.0 v1716>,<0x10000000005/tmp v18>,<0x100000001fb/client.0 v16>,<0x100000001fa/9fc8f023-d646-4fec-98c7-1f3d972f60a9 v32>,<0x10000000001/sv_1 v55>,<0x10000000000/qa v43>,<0x1/volumes v42>]//[]","memoryvalue":"(3)0x10000000213:[<0x10000000208/Makefile.in v72>,<0x10000000201/src v1989>,<0x10000000006/blogbench-1.0 v1743>,<0x10000000005/tmp v35>,<0x100000001fb/client.0 v31>,<0x100000001fa/9fc8f023-d646-4fec-98c7-1f3d972f60a9 v44>,<0x10000000001/sv_1 v67>,<0x10000000000/qa v33>,<0x1/volumes v32>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-11T23:20:10.739 INFO:journalctl@ceph.mds.i.smithi136.stdout:Sep 11 23:20:10 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mds-i[65341]: 2024-09-11T23:20:10.341+0000 7ff35e42b640 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000020d [3,head] /volumes/qa/sv_1/9fc8f023-d646-4fec-98c7-1f3d972f60a9/client.0/tmp/blogbench-1.0/src/fakesnprintf.c auth v44 ap=1 s=10682 n(v0 rc2024-09-11T23:19:44.884504+0000 b10682 1=1+0) caps={15180=pAsLsXsFscr/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x555c905a0b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000020d:[<0x10000000208/fakesnprintf.c v44>,<0x10000000201/src v1914>,<0x10000000006/blogbench-1.0 v1716>,<0x10000000005/tmp v18>,<0x100000001fb/client.0 v16>,<0x100000001fa/9fc8f023-d646-4fec-98c7-1f3d972f60a9 v32>,<0x10000000001/sv_1 v55>,<0x10000000000/qa v43>,<0x1/volumes v42>]//[]","memoryvalue":"(3)0x1000000020d:[<0x10000000208/fakesnprintf.c v44>,<0x10000000201/src v1989>,<0x10000000006/blogbench-1.0 v1743>,<0x10000000005/tmp v35>,<0x100000001fb/client.0 v31>,<0x100000001fa/9fc8f023-d646-4fec-98c7-1f3d972f60a9 v44>,<0x10000000001/sv_1 v67>,<0x10000000000/qa v33>,<0x1/volumes v32>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-11T23:20:10.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:10 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:10.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:10 smithi136 ceph-mon[35121]: Scrub error on inode 0x1000000020b (/volumes/qa/sv_1/9fc8f023-d646-4fec-98c7-1f3d972f60a9/client.0/tmp/blogbench-1.0/src/blogbench.c) see mds.i log and `damage ls` output for details 2024-09-11T23:20:10.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:10 smithi136 ceph-mon[35121]: Scrub error on inode 0x10000000379 (/volumes/qa/sv_1/9fc8f023-d646-4fec-98c7-1f3d972f60a9/client.0/tmp/blogbench-1.0/src/.deps) see mds.i log and `damage ls` output for details 2024-09-11T23:20:10.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:10 smithi136 ceph-mon[35121]: Scrub error on inode 0x10000000387 (/volumes/qa/sv_1/9fc8f023-d646-4fec-98c7-1f3d972f60a9/client.0/tmp/blogbench-1.0/src/process.o) see mds.i log and `damage ls` output for details 2024-09-11T23:20:10.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:10 smithi136 ceph-mon[35121]: Scrub error on inode 0x10000000215 (/volumes/qa/sv_1/9fc8f023-d646-4fec-98c7-1f3d972f60a9/client.0/tmp/blogbench-1.0/src/globals.h) see mds.i log and `damage ls` output for details 2024-09-11T23:20:10.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:10 smithi136 ceph-mon[35121]: Scrub error on inode 0x10000000219 (/volumes/qa/sv_1/9fc8f023-d646-4fec-98c7-1f3d972f60a9/client.0/tmp/blogbench-1.0/src/writer.c) see mds.i log and `damage ls` output for details 2024-09-11T23:20:10.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:10 smithi136 ceph-mon[35121]: Scrub error on inode 0x10000000209 (/volumes/qa/sv_1/9fc8f023-d646-4fec-98c7-1f3d972f60a9/client.0/tmp/blogbench-1.0/src/mysnprintf.c) see mds.i log and `damage ls` output for details 2024-09-11T23:20:10.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:10 smithi136 ceph-mon[35121]: Scrub error on inode 0x10000000211 (/volumes/qa/sv_1/9fc8f023-d646-4fec-98c7-1f3d972f60a9/client.0/tmp/blogbench-1.0/src/bsd-getopt_long.h) see mds.i log and `damage ls` output for details 2024-09-11T23:20:10.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:10 smithi136 ceph-mon[35121]: Scrub error on inode 0x10000000216 (/volumes/qa/sv_1/9fc8f023-d646-4fec-98c7-1f3d972f60a9/client.0/tmp/blogbench-1.0/src/commenter.c) see mds.i log and `damage ls` output for details 2024-09-11T23:20:10.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:10 smithi136 ceph-mon[35121]: Scrub error on inode 0x1000000021a (/volumes/qa/sv_1/9fc8f023-d646-4fec-98c7-1f3d972f60a9/client.0/tmp/blogbench-1.0/src/helpers.c) see mds.i log and `damage ls` output for details 2024-09-11T23:20:10.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:10 smithi136 ceph-mon[35121]: Scrub error on inode 0x1000000020a (/volumes/qa/sv_1/9fc8f023-d646-4fec-98c7-1f3d972f60a9/client.0/tmp/blogbench-1.0/src/mysnprintf.h) see mds.i log and `damage ls` output for details 2024-09-11T23:20:10.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:10 smithi136 ceph-mon[35121]: Scrub error on inode 0x1000000020f (/volumes/qa/sv_1/9fc8f023-d646-4fec-98c7-1f3d972f60a9/client.0/tmp/blogbench-1.0/src/blogbench_p.h) see mds.i log and `damage ls` output for details 2024-09-11T23:20:10.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:10 smithi136 ceph-mon[35121]: Scrub error on inode 0x1000000020c (/volumes/qa/sv_1/9fc8f023-d646-4fec-98c7-1f3d972f60a9/client.0/tmp/blogbench-1.0/src/blogbench.h) see mds.i log and `damage ls` output for details 2024-09-11T23:20:10.741 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:10 smithi136 ceph-mon[35121]: Scrub error on inode 0x10000000217 (/volumes/qa/sv_1/9fc8f023-d646-4fec-98c7-1f3d972f60a9/client.0/tmp/blogbench-1.0/src/rewriter.c) see mds.i log and `damage ls` output for details 2024-09-11T23:20:10.741 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:10 smithi136 ceph-mon[35121]: Scrub error on inode 0x10000000385 (/volumes/qa/sv_1/9fc8f023-d646-4fec-98c7-1f3d972f60a9/client.0/tmp/blogbench-1.0/src/blogbench.o) see mds.i log and `damage ls` output for details 2024-09-11T23:20:10.741 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:10 smithi136 ceph-mon[35121]: Scrub error on inode 0x10000000371 (/volumes/qa/sv_1/9fc8f023-d646-4fec-98c7-1f3d972f60a9/client.0/tmp/blogbench-1.0/src/Makefile) see mds.i log and `damage ls` output for details 2024-09-11T23:20:10.742 INFO:journalctl@ceph.mds.i.smithi136.stdout:Sep 11 23:20:10 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mds-i[65341]: 2024-09-11T23:20:10.343+0000 7ff35e42b640 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000212 [3,head] /volumes/qa/sv_1/9fc8f023-d646-4fec-98c7-1f3d972f60a9/client.0/tmp/blogbench-1.0/src/Makefile.am auth v42 ap=1 s=299 n(v0 rc2024-09-11T23:19:44.889504+0000 b299 1=1+0) caps={15180=pAsLsXsFscr/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x555c905a1080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000212:[<0x10000000208/Makefile.am v42>,<0x10000000201/src v1914>,<0x10000000006/blogbench-1.0 v1716>,<0x10000000005/tmp v18>,<0x100000001fb/client.0 v16>,<0x100000001fa/9fc8f023-d646-4fec-98c7-1f3d972f60a9 v32>,<0x10000000001/sv_1 v55>,<0x10000000000/qa v43>,<0x1/volumes v42>]//[]","memoryvalue":"(3)0x10000000212:[<0x10000000208/Makefile.am v42>,<0x10000000201/src v1989>,<0x10000000006/blogbench-1.0 v1743>,<0x10000000005/tmp v35>,<0x100000001fb/client.0 v31>,<0x100000001fa/9fc8f023-d646-4fec-98c7-1f3d972f60a9 v44>,<0x10000000001/sv_1 v67>,<0x10000000000/qa v33>,<0x1/volumes v32>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-11T23:20:10.742 INFO:journalctl@ceph.mds.i.smithi136.stdout:Sep 11 23:20:10 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mds-i[65341]: 2024-09-11T23:20:10.343+0000 7ff35e42b640 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000210 [3,head] /volumes/qa/sv_1/9fc8f023-d646-4fec-98c7-1f3d972f60a9/client.0/tmp/blogbench-1.0/src/bsd-getopt_long.c auth v64 ap=1 s=18847 n(v0 rc2024-09-11T23:19:44.887504+0000 b18847 1=1+0) caps={15180=pAsLsXsFscr/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x555c905a1600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000210:[<0x10000000208/bsd-getopt_long.c v64>,<0x10000000201/src v1914>,<0x10000000006/blogbench-1.0 v1716>,<0x10000000005/tmp v18>,<0x100000001fb/client.0 v16>,<0x100000001fa/9fc8f023-d646-4fec-98c7-1f3d972f60a9 v32>,<0x10000000001/sv_1 v55>,<0x10000000000/qa v43>,<0x1/volumes v42>]//[]","memoryvalue":"(3)0x10000000210:[<0x10000000208/bsd-getopt_long.c v64>,<0x10000000201/src v1989>,<0x10000000006/blogbench-1.0 v1743>,<0x10000000005/tmp v35>,<0x100000001fb/client.0 v31>,<0x100000001fa/9fc8f023-d646-4fec-98c7-1f3d972f60a9 v44>,<0x10000000001/sv_1 v67>,<0x10000000000/qa v33>,<0x1/volumes v32>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-11T23:20:10.742 INFO:journalctl@ceph.mds.i.smithi136.stdout:Sep 11 23:20:10 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mds-i[65341]: 2024-09-11T23:20:10.349+0000 7ff35e42b640 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000218 [3,head] /volumes/qa/sv_1/9fc8f023-d646-4fec-98c7-1f3d972f60a9/client.0/tmp/blogbench-1.0/src/reader.c auth v68 ap=1 s=1605 n(v0 rc2024-09-11T23:19:44.895504+0000 b1605 1=1+0) caps={15180=pAsLsXsFscr/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x555c905a2100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000218:[<0x10000000208/reader.c v68>,<0x10000000201/src v1914>,<0x10000000006/blogbench-1.0 v1716>,<0x10000000005/tmp v18>,<0x100000001fb/client.0 v16>,<0x100000001fa/9fc8f023-d646-4fec-98c7-1f3d972f60a9 v32>,<0x10000000001/sv_1 v55>,<0x10000000000/qa v43>,<0x1/volumes v42>]//[]","memoryvalue":"(3)0x10000000218:[<0x10000000208/reader.c v68>,<0x10000000201/src v1989>,<0x10000000006/blogbench-1.0 v1743>,<0x10000000005/tmp v35>,<0x100000001fb/client.0 v31>,<0x100000001fa/9fc8f023-d646-4fec-98c7-1f3d972f60a9 v44>,<0x10000000001/sv_1 v67>,<0x10000000000/qa v33>,<0x1/volumes v32>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-11T23:20:10.742 INFO:journalctl@ceph.mds.i.smithi136.stdout:Sep 11 23:20:10 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mds-i[65341]: 2024-09-11T23:20:10.350+0000 7ff35e42b640 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000214 [3,head] /volumes/qa/sv_1/9fc8f023-d646-4fec-98c7-1f3d972f60a9/client.0/tmp/blogbench-1.0/src/process.c auth v56 ap=1 s=4649 n(v0 rc2024-09-11T23:19:44.891504+0000 b4649 1=1+0) caps={15180=pAsLsXsFscr/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x555c905a2680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000214:[<0x10000000208/process.c v56>,<0x10000000201/src v1914>,<0x10000000006/blogbench-1.0 v1716>,<0x10000000005/tmp v18>,<0x100000001fb/client.0 v16>,<0x100000001fa/9fc8f023-d646-4fec-98c7-1f3d972f60a9 v32>,<0x10000000001/sv_1 v55>,<0x10000000000/qa v43>,<0x1/volumes v42>]//[]","memoryvalue":"(3)0x10000000214:[<0x10000000208/process.c v56>,<0x10000000201/src v1989>,<0x10000000006/blogbench-1.0 v1743>,<0x10000000005/tmp v35>,<0x100000001fb/client.0 v31>,<0x100000001fa/9fc8f023-d646-4fec-98c7-1f3d972f60a9 v44>,<0x10000000001/sv_1 v67>,<0x10000000000/qa v33>,<0x1/volumes v32>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-11T23:20:10.742 INFO:journalctl@ceph.mds.i.smithi136.stdout:Sep 11 23:20:10 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mds-i[65341]: 2024-09-11T23:20:10.414+0000 7ff35e42b640 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000020e [3,head] /volumes/qa/sv_1/9fc8f023-d646-4fec-98c7-1f3d972f60a9/client.0/tmp/blogbench-1.0/src/fakesnprintf.h auth v38 ap=1 s=606 n(v0 rc2024-09-11T23:19:44.885504+0000 b606 1=1+0) caps={15180=pAsLsXsFscr/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x555c905a3180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000020e:[<0x10000000208/fakesnprintf.h v38>,<0x10000000201/src v1914>,<0x10000000006/blogbench-1.0 v1716>,<0x10000000005/tmp v18>,<0x100000001fb/client.0 v16>,<0x100000001fa/9fc8f023-d646-4fec-98c7-1f3d972f60a9 v32>,<0x10000000001/sv_1 v55>,<0x10000000000/qa v43>,<0x1/volumes v42>]//[]","memoryvalue":"(3)0x1000000020e:[<0x10000000208/fakesnprintf.h v38>,<0x10000000201/src v1989>,<0x10000000006/blogbench-1.0 v1743>,<0x10000000005/tmp v35>,<0x100000001fb/client.0 v31>,<0x100000001fa/9fc8f023-d646-4fec-98c7-1f3d972f60a9 v44>,<0x10000000001/sv_1 v67>,<0x10000000000/qa v33>,<0x1/volumes v32>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-11T23:20:10.742 INFO:journalctl@ceph.mds.i.smithi136.stdout:Sep 11 23:20:10 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mds-i[65341]: 2024-09-11T23:20:10.429+0000 7ff35e42b640 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000384 [3,head] /volumes/qa/sv_1/9fc8f023-d646-4fec-98c7-1f3d972f60a9/client.0/tmp/blogbench-1.0/src/.deps/blogbench.Po auth v120 ap=1 s=8504 n(v0 rc2024-09-11T23:19:50.881410+0000 b8504 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={15180=0-4194304@2} caps={15180=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=15180 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x555c90358000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000384:[<0x10000000379/blogbench.Po v26>,<0x10000000208/.deps v120>,<0x10000000201/src v1914>,<0x10000000006/blogbench-1.0 v1716>,<0x10000000005/tmp v18>,<0x100000001fb/client.0 v16>,<0x100000001fa/9fc8f023-d646-4fec-98c7-1f3d972f60a9 v32>,<0x10000000001/sv_1 v55>,<0x10000000000/qa v43>,<0x1/volumes v42>]//[]","memoryvalue":"(3)0x10000000384:[<0x10000000379/blogbench.Po v120>,<0x10000000208/.deps v266>,<0x10000000201/src v1989>,<0x10000000006/blogbench-1.0 v1743>,<0x10000000005/tmp v35>,<0x100000001fb/client.0 v31>,<0x100000001fa/9fc8f023-d646-4fec-98c7-1f3d972f60a9 v44>,<0x10000000001/sv_1 v67>,<0x10000000000/qa v33>,<0x1/volumes v32>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-11T23:20:10.742 INFO:journalctl@ceph.mds.i.smithi136.stdout:Sep 11 23:20:10 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mds-i[65341]: 2024-09-11T23:20:10.430+0000 7ff35e42b640 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000386 [3,head] /volumes/qa/sv_1/9fc8f023-d646-4fec-98c7-1f3d972f60a9/client.0/tmp/blogbench-1.0/src/.deps/process.Po auth v126 ap=1 s=8360 n(v0 rc2024-09-11T23:19:50.952409+0000 b8360 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={15180=0-4194304@2} caps={15180=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=15180 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x555c90358580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000386:[<0x10000000379/process.Po v35>,<0x10000000208/.deps v120>,<0x10000000201/src v1914>,<0x10000000006/blogbench-1.0 v1716>,<0x10000000005/tmp v18>,<0x100000001fb/client.0 v16>,<0x100000001fa/9fc8f023-d646-4fec-98c7-1f3d972f60a9 v32>,<0x10000000001/sv_1 v55>,<0x10000000000/qa v43>,<0x1/volumes v42>]//[]","memoryvalue":"(3)0x10000000386:[<0x10000000379/process.Po v126>,<0x10000000208/.deps v266>,<0x10000000201/src v1989>,<0x10000000006/blogbench-1.0 v1743>,<0x10000000005/tmp v35>,<0x100000001fb/client.0 v31>,<0x100000001fa/9fc8f023-d646-4fec-98c7-1f3d972f60a9 v44>,<0x10000000001/sv_1 v67>,<0x10000000000/qa v33>,<0x1/volumes v32>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-11T23:20:10.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:10 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:10.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:10 smithi026 ceph-mon[31172]: Scrub error on inode 0x1000000020b (/volumes/qa/sv_1/9fc8f023-d646-4fec-98c7-1f3d972f60a9/client.0/tmp/blogbench-1.0/src/blogbench.c) see mds.i log and `damage ls` output for details 2024-09-11T23:20:10.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:10 smithi026 ceph-mon[31172]: Scrub error on inode 0x10000000379 (/volumes/qa/sv_1/9fc8f023-d646-4fec-98c7-1f3d972f60a9/client.0/tmp/blogbench-1.0/src/.deps) see mds.i log and `damage ls` output for details 2024-09-11T23:20:10.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:10 smithi026 ceph-mon[31172]: Scrub error on inode 0x10000000387 (/volumes/qa/sv_1/9fc8f023-d646-4fec-98c7-1f3d972f60a9/client.0/tmp/blogbench-1.0/src/process.o) see mds.i log and `damage ls` output for details 2024-09-11T23:20:10.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:10 smithi026 ceph-mon[31172]: Scrub error on inode 0x10000000215 (/volumes/qa/sv_1/9fc8f023-d646-4fec-98c7-1f3d972f60a9/client.0/tmp/blogbench-1.0/src/globals.h) see mds.i log and `damage ls` output for details 2024-09-11T23:20:10.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:10 smithi026 ceph-mon[31172]: Scrub error on inode 0x10000000219 (/volumes/qa/sv_1/9fc8f023-d646-4fec-98c7-1f3d972f60a9/client.0/tmp/blogbench-1.0/src/writer.c) see mds.i log and `damage ls` output for details 2024-09-11T23:20:10.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:10 smithi026 ceph-mon[31172]: Scrub error on inode 0x10000000209 (/volumes/qa/sv_1/9fc8f023-d646-4fec-98c7-1f3d972f60a9/client.0/tmp/blogbench-1.0/src/mysnprintf.c) see mds.i log and `damage ls` output for details 2024-09-11T23:20:10.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:10 smithi026 ceph-mon[31172]: Scrub error on inode 0x10000000211 (/volumes/qa/sv_1/9fc8f023-d646-4fec-98c7-1f3d972f60a9/client.0/tmp/blogbench-1.0/src/bsd-getopt_long.h) see mds.i log and `damage ls` output for details 2024-09-11T23:20:10.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:10 smithi026 ceph-mon[31172]: Scrub error on inode 0x10000000216 (/volumes/qa/sv_1/9fc8f023-d646-4fec-98c7-1f3d972f60a9/client.0/tmp/blogbench-1.0/src/commenter.c) see mds.i log and `damage ls` output for details 2024-09-11T23:20:10.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:10 smithi026 ceph-mon[31172]: Scrub error on inode 0x1000000021a (/volumes/qa/sv_1/9fc8f023-d646-4fec-98c7-1f3d972f60a9/client.0/tmp/blogbench-1.0/src/helpers.c) see mds.i log and `damage ls` output for details 2024-09-11T23:20:10.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:10 smithi026 ceph-mon[31172]: Scrub error on inode 0x1000000020a (/volumes/qa/sv_1/9fc8f023-d646-4fec-98c7-1f3d972f60a9/client.0/tmp/blogbench-1.0/src/mysnprintf.h) see mds.i log and `damage ls` output for details 2024-09-11T23:20:10.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:10 smithi026 ceph-mon[31172]: Scrub error on inode 0x1000000020f (/volumes/qa/sv_1/9fc8f023-d646-4fec-98c7-1f3d972f60a9/client.0/tmp/blogbench-1.0/src/blogbench_p.h) see mds.i log and `damage ls` output for details 2024-09-11T23:20:10.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:10 smithi026 ceph-mon[31172]: Scrub error on inode 0x1000000020c (/volumes/qa/sv_1/9fc8f023-d646-4fec-98c7-1f3d972f60a9/client.0/tmp/blogbench-1.0/src/blogbench.h) see mds.i log and `damage ls` output for details 2024-09-11T23:20:10.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:10 smithi026 ceph-mon[31172]: Scrub error on inode 0x10000000217 (/volumes/qa/sv_1/9fc8f023-d646-4fec-98c7-1f3d972f60a9/client.0/tmp/blogbench-1.0/src/rewriter.c) see mds.i log and `damage ls` output for details 2024-09-11T23:20:10.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:10 smithi026 ceph-mon[31172]: Scrub error on inode 0x10000000385 (/volumes/qa/sv_1/9fc8f023-d646-4fec-98c7-1f3d972f60a9/client.0/tmp/blogbench-1.0/src/blogbench.o) see mds.i log and `damage ls` output for details 2024-09-11T23:20:10.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:10 smithi026 ceph-mon[31172]: Scrub error on inode 0x10000000371 (/volumes/qa/sv_1/9fc8f023-d646-4fec-98c7-1f3d972f60a9/client.0/tmp/blogbench-1.0/src/Makefile) see mds.i log and `damage ls` output for details 2024-09-11T23:20:11.189 INFO:tasks.cephfs.filesystem:scrub status for tag:a4be7cc5-004d-4c94-bbfa-1fe107d6af79 - {'path': '/', 'tag': 'a4be7cc5-004d-4c94-bbfa-1fe107d6af79', 'options': 'recursive,force'} 2024-09-11T23:20:11.501 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:11 smithi026 ceph-mon[31172]: pgmap v57: 97 pgs: 97 active+clean; 59 MiB data, 577 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.7 MiB/s wr, 288 op/s 2024-09-11T23:20:11.501 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:11 smithi026 ceph-mon[31172]: Scrub error on inode 0x10000000213 (/volumes/qa/sv_1/9fc8f023-d646-4fec-98c7-1f3d972f60a9/client.0/tmp/blogbench-1.0/src/Makefile.in) see mds.i log and `damage ls` output for details 2024-09-11T23:20:11.501 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:11 smithi026 ceph-mon[31172]: Scrub error on inode 0x1000000020d (/volumes/qa/sv_1/9fc8f023-d646-4fec-98c7-1f3d972f60a9/client.0/tmp/blogbench-1.0/src/fakesnprintf.c) see mds.i log and `damage ls` output for details 2024-09-11T23:20:11.501 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:11 smithi026 ceph-mon[31172]: Scrub error on inode 0x10000000212 (/volumes/qa/sv_1/9fc8f023-d646-4fec-98c7-1f3d972f60a9/client.0/tmp/blogbench-1.0/src/Makefile.am) see mds.i log and `damage ls` output for details 2024-09-11T23:20:11.501 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:11 smithi026 ceph-mon[31172]: Scrub error on inode 0x10000000210 (/volumes/qa/sv_1/9fc8f023-d646-4fec-98c7-1f3d972f60a9/client.0/tmp/blogbench-1.0/src/bsd-getopt_long.c) see mds.i log and `damage ls` output for details 2024-09-11T23:20:11.501 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:11 smithi026 ceph-mon[31172]: Scrub error on inode 0x10000000218 (/volumes/qa/sv_1/9fc8f023-d646-4fec-98c7-1f3d972f60a9/client.0/tmp/blogbench-1.0/src/reader.c) see mds.i log and `damage ls` output for details 2024-09-11T23:20:11.501 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:11 smithi026 ceph-mon[31172]: Scrub error on inode 0x10000000214 (/volumes/qa/sv_1/9fc8f023-d646-4fec-98c7-1f3d972f60a9/client.0/tmp/blogbench-1.0/src/process.c) see mds.i log and `damage ls` output for details 2024-09-11T23:20:11.501 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:11 smithi026 ceph-mon[31172]: Scrub error on inode 0x1000000020e (/volumes/qa/sv_1/9fc8f023-d646-4fec-98c7-1f3d972f60a9/client.0/tmp/blogbench-1.0/src/fakesnprintf.h) see mds.i log and `damage ls` output for details 2024-09-11T23:20:11.501 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:11 smithi026 ceph-mon[31172]: Scrub error on inode 0x10000000384 (/volumes/qa/sv_1/9fc8f023-d646-4fec-98c7-1f3d972f60a9/client.0/tmp/blogbench-1.0/src/.deps/blogbench.Po) see mds.i log and `damage ls` output for details 2024-09-11T23:20:11.501 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:11 smithi026 ceph-mon[31172]: Scrub error on inode 0x10000000386 (/volumes/qa/sv_1/9fc8f023-d646-4fec-98c7-1f3d972f60a9/client.0/tmp/blogbench-1.0/src/.deps/process.Po) see mds.i log and `damage ls` output for details 2024-09-11T23:20:11.501 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:11 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:11.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:11 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:11.553 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:11 smithi076 ceph-mon[35296]: pgmap v57: 97 pgs: 97 active+clean; 59 MiB data, 577 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.7 MiB/s wr, 288 op/s 2024-09-11T23:20:11.553 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:11 smithi076 ceph-mon[35296]: Scrub error on inode 0x10000000213 (/volumes/qa/sv_1/9fc8f023-d646-4fec-98c7-1f3d972f60a9/client.0/tmp/blogbench-1.0/src/Makefile.in) see mds.i log and `damage ls` output for details 2024-09-11T23:20:11.553 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:11 smithi076 ceph-mon[35296]: Scrub error on inode 0x1000000020d (/volumes/qa/sv_1/9fc8f023-d646-4fec-98c7-1f3d972f60a9/client.0/tmp/blogbench-1.0/src/fakesnprintf.c) see mds.i log and `damage ls` output for details 2024-09-11T23:20:11.553 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:11 smithi076 ceph-mon[35296]: Scrub error on inode 0x10000000212 (/volumes/qa/sv_1/9fc8f023-d646-4fec-98c7-1f3d972f60a9/client.0/tmp/blogbench-1.0/src/Makefile.am) see mds.i log and `damage ls` output for details 2024-09-11T23:20:11.553 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:11 smithi076 ceph-mon[35296]: Scrub error on inode 0x10000000210 (/volumes/qa/sv_1/9fc8f023-d646-4fec-98c7-1f3d972f60a9/client.0/tmp/blogbench-1.0/src/bsd-getopt_long.c) see mds.i log and `damage ls` output for details 2024-09-11T23:20:11.553 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:11 smithi076 ceph-mon[35296]: Scrub error on inode 0x10000000218 (/volumes/qa/sv_1/9fc8f023-d646-4fec-98c7-1f3d972f60a9/client.0/tmp/blogbench-1.0/src/reader.c) see mds.i log and `damage ls` output for details 2024-09-11T23:20:11.553 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:11 smithi076 ceph-mon[35296]: Scrub error on inode 0x10000000214 (/volumes/qa/sv_1/9fc8f023-d646-4fec-98c7-1f3d972f60a9/client.0/tmp/blogbench-1.0/src/process.c) see mds.i log and `damage ls` output for details 2024-09-11T23:20:11.553 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:11 smithi076 ceph-mon[35296]: Scrub error on inode 0x1000000020e (/volumes/qa/sv_1/9fc8f023-d646-4fec-98c7-1f3d972f60a9/client.0/tmp/blogbench-1.0/src/fakesnprintf.h) see mds.i log and `damage ls` output for details 2024-09-11T23:20:11.553 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:11 smithi076 ceph-mon[35296]: Scrub error on inode 0x10000000384 (/volumes/qa/sv_1/9fc8f023-d646-4fec-98c7-1f3d972f60a9/client.0/tmp/blogbench-1.0/src/.deps/blogbench.Po) see mds.i log and `damage ls` output for details 2024-09-11T23:20:11.553 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:11 smithi076 ceph-mon[35296]: Scrub error on inode 0x10000000386 (/volumes/qa/sv_1/9fc8f023-d646-4fec-98c7-1f3d972f60a9/client.0/tmp/blogbench-1.0/src/.deps/process.Po) see mds.i log and `damage ls` output for details 2024-09-11T23:20:11.554 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:11 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:11.554 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:11 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:11.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:11 smithi136 ceph-mon[35121]: pgmap v57: 97 pgs: 97 active+clean; 59 MiB data, 577 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.7 MiB/s wr, 288 op/s 2024-09-11T23:20:11.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:11 smithi136 ceph-mon[35121]: Scrub error on inode 0x10000000213 (/volumes/qa/sv_1/9fc8f023-d646-4fec-98c7-1f3d972f60a9/client.0/tmp/blogbench-1.0/src/Makefile.in) see mds.i log and `damage ls` output for details 2024-09-11T23:20:11.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:11 smithi136 ceph-mon[35121]: Scrub error on inode 0x1000000020d (/volumes/qa/sv_1/9fc8f023-d646-4fec-98c7-1f3d972f60a9/client.0/tmp/blogbench-1.0/src/fakesnprintf.c) see mds.i log and `damage ls` output for details 2024-09-11T23:20:11.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:11 smithi136 ceph-mon[35121]: Scrub error on inode 0x10000000212 (/volumes/qa/sv_1/9fc8f023-d646-4fec-98c7-1f3d972f60a9/client.0/tmp/blogbench-1.0/src/Makefile.am) see mds.i log and `damage ls` output for details 2024-09-11T23:20:11.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:11 smithi136 ceph-mon[35121]: Scrub error on inode 0x10000000210 (/volumes/qa/sv_1/9fc8f023-d646-4fec-98c7-1f3d972f60a9/client.0/tmp/blogbench-1.0/src/bsd-getopt_long.c) see mds.i log and `damage ls` output for details 2024-09-11T23:20:11.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:11 smithi136 ceph-mon[35121]: Scrub error on inode 0x10000000218 (/volumes/qa/sv_1/9fc8f023-d646-4fec-98c7-1f3d972f60a9/client.0/tmp/blogbench-1.0/src/reader.c) see mds.i log and `damage ls` output for details 2024-09-11T23:20:11.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:11 smithi136 ceph-mon[35121]: Scrub error on inode 0x10000000214 (/volumes/qa/sv_1/9fc8f023-d646-4fec-98c7-1f3d972f60a9/client.0/tmp/blogbench-1.0/src/process.c) see mds.i log and `damage ls` output for details 2024-09-11T23:20:11.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:11 smithi136 ceph-mon[35121]: Scrub error on inode 0x1000000020e (/volumes/qa/sv_1/9fc8f023-d646-4fec-98c7-1f3d972f60a9/client.0/tmp/blogbench-1.0/src/fakesnprintf.h) see mds.i log and `damage ls` output for details 2024-09-11T23:20:11.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:11 smithi136 ceph-mon[35121]: Scrub error on inode 0x10000000384 (/volumes/qa/sv_1/9fc8f023-d646-4fec-98c7-1f3d972f60a9/client.0/tmp/blogbench-1.0/src/.deps/blogbench.Po) see mds.i log and `damage ls` output for details 2024-09-11T23:20:11.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:11 smithi136 ceph-mon[35121]: Scrub error on inode 0x10000000386 (/volumes/qa/sv_1/9fc8f023-d646-4fec-98c7-1f3d972f60a9/client.0/tmp/blogbench-1.0/src/.deps/process.Po) see mds.i log and `damage ls` output for details 2024-09-11T23:20:11.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:11 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:11.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:11 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:12.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:12 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:12.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:12 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:12.750 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:12 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:12.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:12 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:12.802 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:12 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:12.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:12 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:13.576 INFO:tasks.workunit.client.0.smithi026.stdout: 50 18206 1582 10406 1512 2915 605 2024-09-11T23:20:13.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:13 smithi136 ceph-mon[35121]: pgmap v58: 97 pgs: 97 active+clean; 59 MiB data, 577 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.7 MiB/s wr, 288 op/s 2024-09-11T23:20:13.750 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:13 smithi026 ceph-mon[31172]: pgmap v58: 97 pgs: 97 active+clean; 59 MiB data, 577 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.7 MiB/s wr, 288 op/s 2024-09-11T23:20:13.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:13 smithi076 ceph-mon[35296]: pgmap v58: 97 pgs: 97 active+clean; 59 MiB data, 577 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.7 MiB/s wr, 288 op/s 2024-09-11T23:20:14.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:14 smithi136 ceph-mon[35121]: Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-09-11T23:20:14.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:14 smithi136 ceph-mon[35121]: mds.? [v2:172.21.15.136:6838/3211398630,v1:172.21.15.136:6839/3211398630] up:active 2024-09-11T23:20:14.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:14 smithi136 ceph-mon[35121]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:20:14.750 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:14 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mon-a[31168]: 2024-09-11T23:20:14.304+0000 7f5d93536640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-09-11T23:20:14.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:14 smithi026 ceph-mon[31172]: Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-09-11T23:20:14.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:14 smithi026 ceph-mon[31172]: mds.? [v2:172.21.15.136:6838/3211398630,v1:172.21.15.136:6839/3211398630] up:active 2024-09-11T23:20:14.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:14 smithi026 ceph-mon[31172]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:20:14.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:14 smithi076 ceph-mon[35296]: Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-09-11T23:20:14.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:14 smithi076 ceph-mon[35296]: mds.? [v2:172.21.15.136:6838/3211398630,v1:172.21.15.136:6839/3211398630] up:active 2024-09-11T23:20:14.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:14 smithi076 ceph-mon[35296]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:20:15.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:15 smithi136 ceph-mon[35121]: pgmap v59: 97 pgs: 97 active+clean; 68 MiB data, 609 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 5.6 MiB/s wr, 329 op/s 2024-09-11T23:20:15.750 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:15 smithi026 ceph-mon[31172]: pgmap v59: 97 pgs: 97 active+clean; 68 MiB data, 609 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 5.6 MiB/s wr, 329 op/s 2024-09-11T23:20:15.802 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:15 smithi076 ceph-mon[35296]: pgmap v59: 97 pgs: 97 active+clean; 68 MiB data, 609 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 5.6 MiB/s wr, 329 op/s 2024-09-11T23:20:16.501 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:16 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:16.501 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:16 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:16.501 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:16 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:16.501 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:16 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:16.501 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:16 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:16.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:16 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:16.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:16 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:16.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:16 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:16.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:16 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:16.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:16 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:16.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:16 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:16.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:16 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:16.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:16 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:16.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:16 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:16.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:16 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:16.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:16 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:16.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:16 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:16.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:16 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:16.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:16 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:16.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:16 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:16.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:16 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:16.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:16 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:16.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:16 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:16.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:16 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:16.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:16 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:16.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:16 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:16.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:16 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:16.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:16 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:16.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:16 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:16.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:16 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:16.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:16 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:16.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:16 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:16.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:16 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:16.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:16 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:16.504 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:16 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:16.504 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:16 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:16.504 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:16 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:16.504 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:16 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:16.504 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:16 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:16.504 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:16 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:16.504 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:16 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:16.504 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:16 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:16.504 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:16 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:16.504 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:16 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:16.504 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:16 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:16.504 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:16 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:16.504 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:16 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:16.504 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:16 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:16.504 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:16 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:16.505 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:16 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:16.505 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:16 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:16.505 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:16 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:16.505 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:16 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:16.505 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:16 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:16.505 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:16 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:16.505 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:16 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:16.505 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:16 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:16.505 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:16 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:16.505 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:16 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:16.506 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:16 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:16.506 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:16 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:16.506 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:16 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:16.506 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:16 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:16.506 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:16 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:16.506 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:16 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:16.506 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:16 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:16.506 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:16 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:16.506 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:16 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:16.506 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:16 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:16.506 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:16 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:16.506 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:16 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:16.506 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:16 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:16.506 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:16 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:16.507 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:16 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:16.507 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:16 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:16.507 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:16 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:16.507 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:16 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:16.507 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:16 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:16.507 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:16 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:16.507 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:16 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:16.507 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:16 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:16.507 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:16 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:16.507 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:16 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:16.507 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:16 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:16.507 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:16 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:16.507 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:16 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:16.507 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:16 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:16.507 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:16 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:16.508 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:16 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:16.508 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:16 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:16.508 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:16 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:16.508 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:16 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:16.641 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:16 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:16.641 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:16 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:16.641 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:16 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:16.641 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:16 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:16.641 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:16 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:16.641 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:16 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:16.641 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:16 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:16.641 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:16 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:16.642 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:16 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:16.642 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:16 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:16.642 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:16 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:16.642 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:16 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:16.642 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:16 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:16.642 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:16 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:16.642 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:16 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:16.642 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:16 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:16.642 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:16 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:16.642 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:16 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:16.642 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:16 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:16.642 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:16 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:16.643 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:16 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:16.643 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:16 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:16.643 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:16 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:16.643 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:16 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:16.643 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:16 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:16.643 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:16 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:16.643 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:16 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:16.643 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:16 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:16.643 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:16 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:16.643 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:16 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:16.644 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:16 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:16.644 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:16 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:16.644 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:16 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:16.644 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:16 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:16.644 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:16 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:16.644 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:16 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:16.644 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:16 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:16.644 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:16 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:16.644 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:16 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:16.645 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:16 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:16.645 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:16 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:16.645 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:16 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:16.645 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:16 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:16.645 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:16 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:16.645 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:16 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:16.645 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:16 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:16.645 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:16 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:16.645 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:16 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:16.645 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:16 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:16.645 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:16 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:16.645 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:16 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:16.645 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:16 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:16.645 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:16 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:16.645 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:16 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:16.646 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:16 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:16.646 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:16 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:16.646 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:16 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:16.646 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:16 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:16.646 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:16 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:16.646 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:16 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:16.646 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:16 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:16.646 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:16 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:16.646 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:16 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:16.646 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:16 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:16.646 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:16 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:16.646 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:16 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:16.647 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:16 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:16.647 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:16 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:16.647 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:16 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:16.647 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:16 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:16.647 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:16 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:16.647 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:16 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:16.647 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:16 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:16.647 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:16 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:16.647 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:16 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:16.647 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:16 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:16.647 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:16 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:16.647 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:16 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:16.647 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:16 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:16.648 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:16 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:16.648 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:16 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:16.648 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:16 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:16.648 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:16 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:16.648 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:16 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:16.648 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:16 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:16.648 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:16 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:16.648 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:16 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:16.648 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:16 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:16.648 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:16 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:16.648 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:16 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:16.648 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:16 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:16.648 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:16 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:16.704 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:16 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:16.704 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:16 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:16.704 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:16 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:16.704 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:16 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:16.704 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:16 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:16.704 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:16 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:16.704 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:16 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:16.704 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:16 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:16.704 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:16 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:16.704 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:16 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:16.704 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:16 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:16.705 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:16 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:16.705 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:16 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:16.705 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:16 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:16.705 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:16 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:16.705 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:16 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:16.705 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:16 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:16.705 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:16 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:16.705 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:16 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:16.705 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:16 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:16.705 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:16 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:16.705 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:16 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:16.705 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:16 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:16.705 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:16 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:16.705 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:16 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:16.706 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:16 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:16.706 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:16 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:16.706 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:16 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:16.706 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:16 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:16.706 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:16 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:16.706 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:16 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:16.706 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:16 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:16.706 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:16 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:16.706 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:16 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:16.706 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:16 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:16.706 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:16 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:16.706 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:16 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:16.706 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:16 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:16.707 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:16 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:16.707 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:16 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:16.707 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:16 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:16.707 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:16 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:16.707 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:16 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:16.707 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:16 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:16.707 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:16 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:16.707 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:16 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:16.707 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:16 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:16.707 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:16 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:16.707 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:16 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:16.707 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:16 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:16.707 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:16 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:16.707 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:16 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:16.708 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:16 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:16.708 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:16 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:16.708 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:16 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:16.708 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:16 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:16.708 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:16 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:16.708 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:16 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:16.708 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:16 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:16.708 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:16 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:16.708 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:16 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:16.709 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:16 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:16.709 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:16 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:16.709 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:16 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:16.709 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:16 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:16.709 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:16 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:16.709 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:16 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:16.709 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:16 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:16.709 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:16 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:16.709 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:16 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:16.709 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:16 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:16.709 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:16 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:16.709 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:16 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:16.709 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:16 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:16.710 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:16 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:16.710 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:16 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:16.710 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:16 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:16.710 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:16 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:16.710 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:16 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:16.710 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:16 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:16.710 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:16 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:16.710 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:16 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:16.710 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:16 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:16.710 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:16 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:16.710 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:16 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:16.710 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:16 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:16.710 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:16 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:16.710 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:16 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:16.710 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:16 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:16.711 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:16 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:16.711 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:16 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:16.711 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:16 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:17.407 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.406+0000 7fc82e4a6640 1 -- 172.21.15.26:0/3810124189 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc8281035a0 msgr2=0x7fc82810ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:17.408 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.406+0000 7fc82e4a6640 1 --2- 172.21.15.26:0/3810124189 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc8281035a0 0x7fc82810ba80 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fc81c009900 tx=0x7fc81c02a480 comp rx=0 tx=0).stop 2024-09-11T23:20:17.408 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.406+0000 7fc82e4a6640 1 -- 172.21.15.26:0/3810124189 shutdown_connections 2024-09-11T23:20:17.408 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.406+0000 7fc82e4a6640 1 --2- 172.21.15.26:0/3810124189 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc82810c0d0 0x7fc82810e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:17.408 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.406+0000 7fc82e4a6640 1 --2- 172.21.15.26:0/3810124189 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc8281035a0 0x7fc82810ba80 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:17.408 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.406+0000 7fc82e4a6640 1 --2- 172.21.15.26:0/3810124189 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc828102c80 0x7fc828103060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:17.408 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.406+0000 7fc82e4a6640 1 -- 172.21.15.26:0/3810124189 >> 172.21.15.26:0/3810124189 conn(0x7fc828077990 msgr2=0x7fc8280fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:17.408 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.406+0000 7fc82e4a6640 1 -- 172.21.15.26:0/3810124189 shutdown_connections 2024-09-11T23:20:17.408 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.406+0000 7fc82e4a6640 1 -- 172.21.15.26:0/3810124189 wait complete. 2024-09-11T23:20:17.408 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.407+0000 7fc82e4a6640 1 Processor -- start 2024-09-11T23:20:17.408 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.407+0000 7fc82e4a6640 1 -- start start 2024-09-11T23:20:17.410 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.407+0000 7fc82e4a6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc828102c80 0x7fc8281008d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:17.410 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.407+0000 7fc82e4a6640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc8281035a0 0x7fc828100e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:17.410 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.407+0000 7fc82e4a6640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc82810c0d0 0x7fc828101350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:17.410 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.407+0000 7fc82e4a6640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc828111eb0 con 0x7fc828102c80 2024-09-11T23:20:17.410 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.407+0000 7fc82e4a6640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fc828111d30 con 0x7fc8281035a0 2024-09-11T23:20:17.410 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.407+0000 7fc82e4a6640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fc828112030 con 0x7fc82810c0d0 2024-09-11T23:20:17.411 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.407+0000 7fc827fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc828102c80 0x7fc8281008d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:17.411 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.407+0000 7fc827fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc828102c80 0x7fc8281008d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58976/0 (socket says 172.21.15.26:58976) 2024-09-11T23:20:17.411 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.407+0000 7fc827fff640 1 -- 172.21.15.26:0/355663473 learned_addr learned my addr 172.21.15.26:0/355663473 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:20:17.411 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.407+0000 7fc8277fe640 1 --2- 172.21.15.26:0/355663473 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc8281035a0 0x7fc828100e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:17.411 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.407+0000 7fc82ca43640 1 --2- 172.21.15.26:0/355663473 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc82810c0d0 0x7fc828101350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:17.411 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.408+0000 7fc8277fe640 1 -- 172.21.15.26:0/355663473 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc82810c0d0 msgr2=0x7fc828101350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:17.411 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.408+0000 7fc8277fe640 1 --2- 172.21.15.26:0/355663473 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc82810c0d0 0x7fc828101350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:17.411 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.408+0000 7fc8277fe640 1 -- 172.21.15.26:0/355663473 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc828102c80 msgr2=0x7fc8281008d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:17.411 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.408+0000 7fc8277fe640 1 --2- 172.21.15.26:0/355663473 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc828102c80 0x7fc8281008d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:17.411 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.408+0000 7fc8277fe640 1 -- 172.21.15.26:0/355663473 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8281a6840 con 0x7fc8281035a0 2024-09-11T23:20:17.411 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.408+0000 7fc827fff640 1 --2- 172.21.15.26:0/355663473 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc828102c80 0x7fc8281008d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:20:17.411 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.408+0000 7fc8277fe640 1 --2- 172.21.15.26:0/355663473 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc8281035a0 0x7fc828100e10 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fc81c005940 tx=0x7fc81c004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:17.411 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.408+0000 7fc8257fa640 1 -- 172.21.15.26:0/355663473 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc81c03f030 con 0x7fc8281035a0 2024-09-11T23:20:17.412 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.408+0000 7fc8257fa640 1 -- 172.21.15.26:0/355663473 <== mon.2 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc81c004440 con 0x7fc8281035a0 2024-09-11T23:20:17.412 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.408+0000 7fc82e4a6640 1 -- 172.21.15.26:0/355663473 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8281a6ad0 con 0x7fc8281035a0 2024-09-11T23:20:17.412 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.408+0000 7fc8257fa640 1 -- 172.21.15.26:0/355663473 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc81c035070 con 0x7fc8281035a0 2024-09-11T23:20:17.412 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.408+0000 7fc82e4a6640 1 -- 172.21.15.26:0/355663473 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8281a6fc0 con 0x7fc8281035a0 2024-09-11T23:20:17.419 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.410+0000 7fc8257fa640 1 -- 172.21.15.26:0/355663473 <== mon.2 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fc81c042030 con 0x7fc8281035a0 2024-09-11T23:20:17.419 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.411+0000 7fc831238740 1 Processor -- start 2024-09-11T23:20:17.419 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.411+0000 7fc831238740 1 -- start start 2024-09-11T23:20:17.419 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.411+0000 7fc831238740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d01871aaf0 0x55d01871aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:17.419 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.411+0000 7fc831238740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d01871b410 0x55d018725770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:17.419 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.411+0000 7fc831238740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d018725cb0 0x55d0187280a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:17.419 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.411+0000 7fc831238740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55d01867f2f0 con 0x55d01871aaf0 2024-09-11T23:20:17.419 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.411+0000 7fc831238740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55d018627270 con 0x55d018725cb0 2024-09-11T23:20:17.419 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.411+0000 7fc831238740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55d01867b2b0 con 0x55d01871b410 2024-09-11T23:20:17.420 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.411+0000 7fc82ca43640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d01871aaf0 0x55d01871aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:17.420 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.411+0000 7fc827fff640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d01871b410 0x55d018725770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:17.420 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.411+0000 7fc8257fa640 1 --2- 172.21.15.26:0/355663473 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fc8100a2da0 0x7fc8100a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:17.420 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.411+0000 7fc827fff640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d01871b410 0x55d018725770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:37808/0 (socket says 172.21.15.26:37808) 2024-09-11T23:20:17.420 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.411+0000 7fc8277fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d018725cb0 0x55d0187280a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:17.420 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.411+0000 7fc827fff640 1 -- 172.21.15.26:0/1719605183 learned_addr learned my addr 172.21.15.26:0/1719605183 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:20:17.420 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.411+0000 7fc82ca43640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d01871aaf0 0x55d01871aed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58982/0 (socket says 172.21.15.26:58982) 2024-09-11T23:20:17.420 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.411+0000 7fc82ca43640 1 --2- 172.21.15.26:0/355663473 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fc8100a2da0 0x7fc8100a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:17.420 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.411+0000 7fc827fff640 1 -- 172.21.15.26:0/1719605183 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d018725cb0 msgr2=0x55d0187280a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:20:17.420 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.411+0000 7fc827fff640 1 --2- 172.21.15.26:0/1719605183 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d018725cb0 0x55d0187280a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:17.420 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.411+0000 7fc827fff640 1 -- 172.21.15.26:0/1719605183 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d01871aaf0 msgr2=0x55d01871aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:20:17.420 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.411+0000 7fc827fff640 1 --2- 172.21.15.26:0/1719605183 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d01871aaf0 0x55d01871aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:17.421 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.411+0000 7fc827fff640 1 -- 172.21.15.26:0/1719605183 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d018626340 con 0x55d01871b410 2024-09-11T23:20:17.421 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.411+0000 7fc8257fa640 1 -- 172.21.15.26:0/355663473 <== mon.2 v2:172.21.15.76:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fc81c0ef6f0 con 0x7fc8281035a0 2024-09-11T23:20:17.421 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.412+0000 7fc827fff640 1 --2- 172.21.15.26:0/1719605183 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d01871b410 0x55d018725770 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc818011650 tx=0x7fc818009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ac87a01fc62725c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:17.421 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.412+0000 7fc82ca43640 1 --2- 172.21.15.26:0/355663473 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fc8100a2da0 0x7fc8100a5260 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fc820009fd0 tx=0x7fc820004ab0 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:17.421 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.412+0000 7fc80e7fc640 1 -- 172.21.15.26:0/1719605183 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc81801e070 con 0x55d01871b410 2024-09-11T23:20:17.421 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.412+0000 7fc80e7fc640 1 -- 172.21.15.26:0/1719605183 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc81800fba0 con 0x55d01871b410 2024-09-11T23:20:17.421 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.412+0000 7fc80e7fc640 1 -- 172.21.15.26:0/1719605183 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc818019d20 con 0x55d01871b410 2024-09-11T23:20:17.421 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.412+0000 7fc831238740 1 -- 172.21.15.26:0/1719605183 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d01871b410 msgr2=0x55d018725770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:17.421 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.412+0000 7fc831238740 1 --2- 172.21.15.26:0/1719605183 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d01871b410 0x55d018725770 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc818011650 tx=0x7fc818009f90 comp rx=0 tx=0).stop 2024-09-11T23:20:17.421 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.412+0000 7fc831238740 1 -- 172.21.15.26:0/1719605183 shutdown_connections 2024-09-11T23:20:17.421 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.412+0000 7fc831238740 1 --2- 172.21.15.26:0/1719605183 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d018725cb0 0x55d0187280a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:17.421 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.412+0000 7fc831238740 1 --2- 172.21.15.26:0/1719605183 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d01871b410 0x55d018725770 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:17.421 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.412+0000 7fc831238740 1 --2- 172.21.15.26:0/1719605183 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d01871aaf0 0x55d01871aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:17.421 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.412+0000 7fc831238740 1 -- 172.21.15.26:0/1719605183 >> 172.21.15.26:0/1719605183 conn(0x55d018717060 msgr2=0x55d018717440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:17.422 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.412+0000 7fc831238740 1 -- 172.21.15.26:0/1719605183 shutdown_connections 2024-09-11T23:20:17.422 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.413+0000 7fc831238740 1 -- 172.21.15.26:0/1719605183 wait complete. 2024-09-11T23:20:17.422 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.413+0000 7fc831238740 1 -- start start 2024-09-11T23:20:17.422 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.413+0000 7fc831238740 1 Processor -- start 2024-09-11T23:20:17.422 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.413+0000 7fc831238740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d01871aaf0 0x55d018722c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:17.422 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.413+0000 7fc831238740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d01871b410 0x55d018723140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:17.422 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.413+0000 7fc831238740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d018725cb0 0x55d018723680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:17.422 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.413+0000 7fc831238740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55d018728880 con 0x55d01871aaf0 2024-09-11T23:20:17.422 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.413+0000 7fc831238740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55d018608290 con 0x55d01871b410 2024-09-11T23:20:17.422 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.413+0000 7fc831238740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55d018728a00 con 0x55d018725cb0 2024-09-11T23:20:17.422 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.413+0000 7fc831238740 10 client.0 _ll_register_callbacks cb 0x55d018639140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:20:17.422 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.413+0000 7fc827fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d01871aaf0 0x55d018722c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:17.422 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.413+0000 7fc827fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d01871aaf0 0x55d018722c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58984/0 (socket says 172.21.15.26:58984) 2024-09-11T23:20:17.423 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.413+0000 7fc827fff640 1 -- 172.21.15.26:0/2275746024 learned_addr learned my addr 172.21.15.26:0/2275746024 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:20:17.423 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.413+0000 7fc82ca43640 1 --2- 172.21.15.26:0/2275746024 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d018725cb0 0x55d018723680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:17.423 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.414+0000 7fc8277fe640 1 --2- 172.21.15.26:0/2275746024 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d01871b410 0x55d018723140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:17.423 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.414+0000 7fc82ca43640 1 -- 172.21.15.26:0/2275746024 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d01871b410 msgr2=0x55d018723140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:20:17.423 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.414+0000 7fc82ca43640 1 --2- 172.21.15.26:0/2275746024 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d01871b410 0x55d018723140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:17.423 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.414+0000 7fc82ca43640 1 -- 172.21.15.26:0/2275746024 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d01871aaf0 msgr2=0x55d018722c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:20:17.423 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.414+0000 7fc82ca43640 1 --2- 172.21.15.26:0/2275746024 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d01871aaf0 0x55d018722c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:17.423 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.414+0000 7fc82ca43640 1 -- 172.21.15.26:0/2275746024 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d018392210 con 0x55d018725cb0 2024-09-11T23:20:17.423 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.414+0000 7fc827fff640 1 --2- 172.21.15.26:0/2275746024 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d01871aaf0 0x55d018722c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:20:17.423 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.414+0000 7fc8277fe640 1 --2- 172.21.15.26:0/2275746024 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d01871b410 0x55d018723140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:20:17.423 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.414+0000 7fc82ca43640 1 --2- 172.21.15.26:0/2275746024 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d018725cb0 0x55d018723680 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc820005fd0 tx=0x7fc820004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a47f8af550fc3fbe server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:17.423 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.414+0000 7fc8077fe640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-09-11T23:20:17.423 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.414+0000 7fc8077fe640 1 -- 172.21.15.26:0/2275746024 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc820025070 con 0x55d018725cb0 2024-09-11T23:20:17.423 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.414+0000 7fc831238740 1 -- 172.21.15.26:0/2275746024 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc81800fba0 con 0x55d018725cb0 2024-09-11T23:20:17.424 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.414+0000 7fc8077fe640 1 -- 172.21.15.26:0/2275746024 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fc820010dc0 con 0x55d018725cb0 2024-09-11T23:20:17.424 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.414+0000 7fc8077fe640 1 -- 172.21.15.26:0/2275746024 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fc820004b20 con 0x55d018725cb0 2024-09-11T23:20:17.424 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.414+0000 7fc831238740 10 client.25606 fetch_fsmap learned FSMap version 33 2024-09-11T23:20:17.424 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.414+0000 7fc831238740 1 -- 172.21.15.26:0/2275746024 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55d01845e620 con 0x55d018725cb0 2024-09-11T23:20:17.424 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.415+0000 7fc8077fe640 1 -- 172.21.15.26:0/2275746024 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc81800fba0 con 0x55d018725cb0 2024-09-11T23:20:17.424 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.415+0000 7fc8077fe640 1 -- 172.21.15.26:0/2275746024 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 33) ==== 8394+0+0 (secure 0 0 0) 0x7fc820032470 con 0x55d018725cb0 2024-09-11T23:20:17.424 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.415+0000 7fc831238740 10 client.25606 fetch_fsmap finished waiting for FSMap version 33 2024-09-11T23:20:17.424 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.415+0000 7fc831238740 10 client.25606 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:20:17.424 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.415+0000 7fc831238740 20 client.25606 populate_metadata read hostname 'smithi026' 2024-09-11T23:20:17.424 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.415+0000 7fc831238740 1 --2- 172.21.15.26:0/2275746024 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55d018735d20 0x55d018738190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:17.424 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.415+0000 7fc831238740 4 client.25606 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:20:17.424 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.415+0000 7fc831238740 1 -- 172.21.15.26:0/2275746024 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d01868ca20 con 0x55d018735d20 2024-09-11T23:20:17.424 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.415+0000 7fc827fff640 1 --2- 172.21.15.26:0/2275746024 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55d018735d20 0x55d018738190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:17.425 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.415+0000 7fc8077fe640 10 client.25606 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:20:17.425 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.415+0000 7fc827fff640 1 --2- 172.21.15.26:0/2275746024 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55d018735d20 0x55d018738190 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=10ef3f0352675489 server_cookie=54fab99e5d772115 in_seq=0 out_seq=0 2024-09-11T23:20:17.425 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.416+0000 7fc8077fe640 1 -- 172.21.15.26:0/2275746024 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d01868ca20 con 0x55d018735d20 2024-09-11T23:20:17.425 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.416+0000 7fc8077fe640 10 client.25606 handle_command_reply: tid=0 2024-09-11T23:20:17.425 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.416+0000 7fc831238740 1 client.25606 shutdown 2024-09-11T23:20:17.425 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.416+0000 7fc831238740 1 -- 172.21.15.26:0/2275746024 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d018725cb0 msgr2=0x55d018723680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:17.425 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.416+0000 7fc831238740 1 --2- 172.21.15.26:0/2275746024 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d018725cb0 0x55d018723680 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc820005fd0 tx=0x7fc820004a40 comp rx=0 tx=0).stop 2024-09-11T23:20:17.425 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.416+0000 7fc831238740 1 -- 172.21.15.26:0/2275746024 shutdown_connections 2024-09-11T23:20:17.425 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.416+0000 7fc831238740 1 --2- 172.21.15.26:0/2275746024 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d01871b410 0x55d018723140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:17.425 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.416+0000 7fc831238740 1 --2- 172.21.15.26:0/2275746024 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55d018735d20 0x55d018738190 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:17.425 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.416+0000 7fc831238740 1 --2- 172.21.15.26:0/2275746024 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d018725cb0 0x55d018723680 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:17.425 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.416+0000 7fc8077fe640 0 client.25606 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:20:17.425 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.416+0000 7fc831238740 1 --2- 172.21.15.26:0/2275746024 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d01871aaf0 0x55d018722c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:17.426 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.416+0000 7fc831238740 1 -- 172.21.15.26:0/2275746024 >> 172.21.15.26:0/2275746024 conn(0x55d018717060 msgr2=0x55d01871cdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:17.426 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.416+0000 7fc831238740 1 -- 172.21.15.26:0/2275746024 shutdown_connections 2024-09-11T23:20:17.426 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.416+0000 7fc831238740 1 -- 172.21.15.26:0/2275746024 wait complete. 2024-09-11T23:20:17.426 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.417+0000 7fc831238740 20 client.25606 trim_cache size 0 max 16384 2024-09-11T23:20:17.439 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.433+0000 7fc831238740 1 Processor -- start 2024-09-11T23:20:17.439 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.433+0000 7fc831238740 1 -- start start 2024-09-11T23:20:17.439 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.433+0000 7fc831238740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d01871aaf0 0x55d0187234b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:17.439 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.433+0000 7fc831238740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d01871b410 0x55d01873da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:17.439 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.433+0000 7fc831238740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d018725cb0 0x55d01873dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:17.439 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.433+0000 7fc831238740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc8281af9a0 con 0x55d018725cb0 2024-09-11T23:20:17.440 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.433+0000 7fc831238740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fc8281b1660 con 0x55d01871aaf0 2024-09-11T23:20:17.440 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.433+0000 7fc831238740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fc8281b0070 con 0x55d01871b410 2024-09-11T23:20:17.440 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.433+0000 7fc82ca43640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d018725cb0 0x55d01873dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:17.440 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.433+0000 7fc827fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d01871aaf0 0x55d0187234b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:17.440 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.433+0000 7fc82ca43640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d018725cb0 0x55d01873dfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58994/0 (socket says 172.21.15.26:58994) 2024-09-11T23:20:17.440 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.433+0000 7fc82ca43640 1 -- 172.21.15.26:0/306086423 learned_addr learned my addr 172.21.15.26:0/306086423 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:20:17.440 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.433+0000 7fc8277fe640 1 --2- 172.21.15.26:0/306086423 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d01871b410 0x55d01873da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:17.440 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.433+0000 7fc82ca43640 1 -- 172.21.15.26:0/306086423 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d01871b410 msgr2=0x55d01873da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:20:17.440 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.434+0000 7fc82ca43640 1 --2- 172.21.15.26:0/306086423 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d01871b410 0x55d01873da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:17.441 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.434+0000 7fc82ca43640 1 -- 172.21.15.26:0/306086423 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d01871aaf0 msgr2=0x55d0187234b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:20:17.441 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.434+0000 7fc82ca43640 1 --2- 172.21.15.26:0/306086423 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d01871aaf0 0x55d0187234b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:17.441 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.434+0000 7fc82ca43640 1 -- 172.21.15.26:0/306086423 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d01871a920 con 0x55d018725cb0 2024-09-11T23:20:17.441 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.434+0000 7fc827fff640 1 --2- 172.21.15.26:0/306086423 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d01871aaf0 0x55d0187234b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:20:17.441 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.434+0000 7fc82ca43640 1 --2- 172.21.15.26:0/306086423 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d018725cb0 0x55d01873dfa0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fc820032840 tx=0x7fc820009d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d75213373db608 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:17.441 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.434+0000 7fc80dffb640 1 -- 172.21.15.26:0/306086423 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc820025070 con 0x55d018725cb0 2024-09-11T23:20:17.441 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.434+0000 7fc80dffb640 1 -- 172.21.15.26:0/306086423 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc820010bc0 con 0x55d018725cb0 2024-09-11T23:20:17.441 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.434+0000 7fc80dffb640 1 -- 172.21.15.26:0/306086423 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc82002a920 con 0x55d018725cb0 2024-09-11T23:20:17.441 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.435+0000 7fc831238740 1 -- 172.21.15.26:0/306086423 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d018725cb0 msgr2=0x55d01873dfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:17.442 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.435+0000 7fc831238740 1 --2- 172.21.15.26:0/306086423 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d018725cb0 0x55d01873dfa0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fc820032840 tx=0x7fc820009d70 comp rx=0 tx=0).stop 2024-09-11T23:20:17.442 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.435+0000 7fc831238740 1 -- 172.21.15.26:0/306086423 shutdown_connections 2024-09-11T23:20:17.442 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.435+0000 7fc831238740 1 --2- 172.21.15.26:0/306086423 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d018725cb0 0x55d01873dfa0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:17.442 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.435+0000 7fc831238740 1 --2- 172.21.15.26:0/306086423 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d01871b410 0x55d01873da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:17.442 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.435+0000 7fc831238740 1 --2- 172.21.15.26:0/306086423 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d01871aaf0 0x55d0187234b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:17.442 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.435+0000 7fc831238740 1 -- 172.21.15.26:0/306086423 >> 172.21.15.26:0/306086423 conn(0x55d018717060 msgr2=0x55d018724c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:17.442 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.435+0000 7fc831238740 1 -- 172.21.15.26:0/306086423 shutdown_connections 2024-09-11T23:20:17.442 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.435+0000 7fc831238740 1 -- 172.21.15.26:0/306086423 wait complete. 2024-09-11T23:20:17.442 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.435+0000 7fc831238740 1 -- start start 2024-09-11T23:20:17.443 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.435+0000 7fc831238740 1 Processor -- start 2024-09-11T23:20:17.443 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.436+0000 7fc831238740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d01871aaf0 0x55d01872e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:17.443 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.436+0000 7fc831238740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d01871b410 0x55d018729480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:17.443 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.436+0000 7fc831238740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d018725cb0 0x55d0187299c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:17.443 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.436+0000 7fc831238740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc8281b0ed0 con 0x55d01871aaf0 2024-09-11T23:20:17.443 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.436+0000 7fc831238740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fc8281b0740 con 0x55d018725cb0 2024-09-11T23:20:17.443 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.436+0000 7fc831238740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fc8281ad980 con 0x55d01871b410 2024-09-11T23:20:17.443 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.436+0000 7fc831238740 10 client.0 _ll_register_callbacks cb 0x7fc820009ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:20:17.443 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.436+0000 7fc827fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d01871aaf0 0x55d01872e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:17.443 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.436+0000 7fc827fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d01871aaf0 0x55d01872e110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59008/0 (socket says 172.21.15.26:59008) 2024-09-11T23:20:17.444 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.436+0000 7fc827fff640 1 -- 172.21.15.26:0/3711791972 learned_addr learned my addr 172.21.15.26:0/3711791972 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:20:17.444 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.436+0000 7fc8277fe640 1 --2- 172.21.15.26:0/3711791972 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d01871b410 0x55d018729480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:17.444 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.436+0000 7fc82ca43640 1 --2- 172.21.15.26:0/3711791972 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d018725cb0 0x55d0187299c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:17.444 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.436+0000 7fc827fff640 1 -- 172.21.15.26:0/3711791972 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d01871b410 msgr2=0x55d018729480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:20:17.444 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.436+0000 7fc827fff640 1 --2- 172.21.15.26:0/3711791972 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d01871b410 0x55d018729480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:17.444 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.436+0000 7fc827fff640 1 -- 172.21.15.26:0/3711791972 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d018725cb0 msgr2=0x55d0187299c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:20:17.444 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.436+0000 7fc827fff640 1 --2- 172.21.15.26:0/3711791972 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d018725cb0 0x55d0187299c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:17.444 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.436+0000 7fc827fff640 1 -- 172.21.15.26:0/3711791972 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d018729f00 con 0x55d01871aaf0 2024-09-11T23:20:17.444 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.436+0000 7fc82ca43640 1 --2- 172.21.15.26:0/3711791972 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d018725cb0 0x55d0187299c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:20:17.444 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.436+0000 7fc827fff640 1 --2- 172.21.15.26:0/3711791972 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d01871aaf0 0x55d01872e110 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fc818005fd0 tx=0x7fc818009280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=904e5dc9d0a7d9f5 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:17.444 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.436+0000 7fc8077fe640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:20:17.444 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.436+0000 7fc8077fe640 1 -- 172.21.15.26:0/3711791972 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc81801e040 con 0x55d01871aaf0 2024-09-11T23:20:17.445 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.436+0000 7fc8077fe640 1 -- 172.21.15.26:0/3711791972 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fc8180199f0 con 0x55d01871aaf0 2024-09-11T23:20:17.445 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.436+0000 7fc831238740 1 -- 172.21.15.26:0/3711791972 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc820010bc0 con 0x55d01871aaf0 2024-09-11T23:20:17.445 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.437+0000 7fc8077fe640 1 -- 172.21.15.26:0/3711791972 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fc818019bf0 con 0x55d01871aaf0 2024-09-11T23:20:17.445 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.437+0000 7fc831238740 10 client.15978 fetch_fsmap learned FSMap version 33 2024-09-11T23:20:17.445 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.437+0000 7fc831238740 1 -- 172.21.15.26:0/3711791972 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55d0187a0770 con 0x55d01871aaf0 2024-09-11T23:20:17.445 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.437+0000 7fc8077fe640 1 -- 172.21.15.26:0/3711791972 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc820010bc0 con 0x55d01871aaf0 2024-09-11T23:20:17.445 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.437+0000 7fc8077fe640 1 -- 172.21.15.26:0/3711791972 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 33) ==== 8394+0+0 (secure 0 0 0) 0x7fc8180053d0 con 0x55d01871aaf0 2024-09-11T23:20:17.445 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.437+0000 7fc831238740 10 client.15978 fetch_fsmap finished waiting for FSMap version 33 2024-09-11T23:20:17.445 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.437+0000 7fc831238740 10 client.15978 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:20:17.445 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.437+0000 7fc831238740 20 client.15978 populate_metadata read hostname 'smithi026' 2024-09-11T23:20:17.445 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.437+0000 7fc831238740 1 --2- 172.21.15.26:0/3711791972 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55d0187a0de0 0x55d0187a3200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:17.445 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.437+0000 7fc831238740 4 client.15978 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T23:20:17.445 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.437+0000 7fc831238740 1 -- 172.21.15.26:0/3711791972 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d018686d70 con 0x55d0187a0de0 2024-09-11T23:20:17.446 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.437+0000 7fc8277fe640 1 --2- 172.21.15.26:0/3711791972 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55d0187a0de0 0x55d0187a3200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:17.446 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.438+0000 7fc8077fe640 10 client.15978 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:20:17.446 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.438+0000 7fc8277fe640 1 --2- 172.21.15.26:0/3711791972 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55d0187a0de0 0x55d0187a3200 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e9c8b9b94b2a727 server_cookie=5f30ab78ebd1e437 in_seq=0 out_seq=0 2024-09-11T23:20:17.539 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.538+0000 7fc8077fe640 1 -- 172.21.15.26:0/3711791972 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d018686d70 con 0x55d0187a0de0 2024-09-11T23:20:17.539 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.538+0000 7fc8077fe640 10 client.15978 handle_command_reply: tid=0 2024-09-11T23:20:17.540 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.538+0000 7fc831238740 1 client.15978 shutdown 2024-09-11T23:20:17.540 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.538+0000 7fc831238740 1 -- 172.21.15.26:0/3711791972 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d01871aaf0 msgr2=0x55d01872e110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:17.540 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.538+0000 7fc831238740 1 --2- 172.21.15.26:0/3711791972 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d01871aaf0 0x55d01872e110 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fc818005fd0 tx=0x7fc818009280 comp rx=0 tx=0).stop 2024-09-11T23:20:17.540 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.538+0000 7fc831238740 1 -- 172.21.15.26:0/3711791972 shutdown_connections 2024-09-11T23:20:17.540 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.538+0000 7fc831238740 1 --2- 172.21.15.26:0/3711791972 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d018725cb0 0x55d0187299c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:17.540 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.538+0000 7fc831238740 1 --2- 172.21.15.26:0/3711791972 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55d0187a0de0 0x55d0187a3200 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:17.540 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.538+0000 7fc831238740 1 --2- 172.21.15.26:0/3711791972 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d01871b410 0x55d018729480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:17.540 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.538+0000 7fc8077fe640 0 client.15978 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:20:17.540 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.538+0000 7fc831238740 1 --2- 172.21.15.26:0/3711791972 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d01871aaf0 0x55d01872e110 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:17.540 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.538+0000 7fc831238740 1 -- 172.21.15.26:0/3711791972 >> 172.21.15.26:0/3711791972 conn(0x55d018717060 msgr2=0x55d01871c550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:17.540 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.538+0000 7fc831238740 1 -- 172.21.15.26:0/3711791972 shutdown_connections 2024-09-11T23:20:17.540 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.538+0000 7fc831238740 1 -- 172.21.15.26:0/3711791972 wait complete. 2024-09-11T23:20:17.541 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.538+0000 7fc831238740 20 client.15978 trim_cache size 0 max 16384 2024-09-11T23:20:17.541 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-09-11T23:20:17.541 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "message": "", 2024-09-11T23:20:17.541 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-09-11T23:20:17.541 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-09-11T23:20:17.542 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.539+0000 7fc82e4a6640 1 -- 172.21.15.26:0/355663473 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fc8100a2da0 msgr2=0x7fc8100a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:17.542 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.539+0000 7fc82e4a6640 1 --2- 172.21.15.26:0/355663473 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fc8100a2da0 0x7fc8100a5260 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fc820009fd0 tx=0x7fc820004ab0 comp rx=0 tx=0).stop 2024-09-11T23:20:17.542 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.539+0000 7fc82e4a6640 1 -- 172.21.15.26:0/355663473 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc8281035a0 msgr2=0x7fc828100e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:17.542 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.539+0000 7fc82e4a6640 1 --2- 172.21.15.26:0/355663473 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc8281035a0 0x7fc828100e10 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fc81c005940 tx=0x7fc81c004040 comp rx=0 tx=0).stop 2024-09-11T23:20:17.542 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.539+0000 7fc82e4a6640 1 -- 172.21.15.26:0/355663473 shutdown_connections 2024-09-11T23:20:17.542 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.539+0000 7fc82e4a6640 1 --2- 172.21.15.26:0/355663473 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fc8100a2da0 0x7fc8100a5260 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:17.542 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.539+0000 7fc82e4a6640 1 --2- 172.21.15.26:0/355663473 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc8281035a0 0x7fc828100e10 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:17.542 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.539+0000 7fc82e4a6640 1 --2- 172.21.15.26:0/355663473 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc82810c0d0 0x7fc828101350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:17.542 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.539+0000 7fc82e4a6640 1 --2- 172.21.15.26:0/355663473 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc828102c80 0x7fc8281008d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:17.542 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.539+0000 7fc82e4a6640 1 -- 172.21.15.26:0/355663473 >> 172.21.15.26:0/355663473 conn(0x7fc828077990 msgr2=0x7fc8280fcfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:17.543 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.539+0000 7fc82e4a6640 1 -- 172.21.15.26:0/355663473 shutdown_connections 2024-09-11T23:20:17.543 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:17.539+0000 7fc82e4a6640 1 -- 172.21.15.26:0/355663473 wait complete. 2024-09-11T23:20:17.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:17 smithi136 ceph-mon[35121]: pgmap v60: 97 pgs: 97 active+clean; 68 MiB data, 609 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 117 op/s 2024-09-11T23:20:17.750 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:17 smithi026 ceph-mon[31172]: pgmap v60: 97 pgs: 97 active+clean; 68 MiB data, 609 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 117 op/s 2024-09-11T23:20:17.802 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:17 smithi076 ceph-mon[35296]: pgmap v60: 97 pgs: 97 active+clean; 68 MiB data, 609 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 117 op/s 2024-09-11T23:20:18.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:18.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:18.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:18.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.741 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:18.741 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.741 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:18.741 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.741 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:18.741 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.742 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:18.742 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.742 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:18.742 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.742 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.743 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:18.743 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.743 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:18.743 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.743 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:18.743 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.744 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:18.744 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.744 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:18.744 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.744 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:18.744 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.745 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:18.745 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.745 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:18.745 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.745 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:18.745 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.746 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:18.746 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.746 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:18.746 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.746 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:18.746 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.747 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:18.747 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.747 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:18.747 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.747 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:18.747 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.747 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:18.748 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.748 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:18.748 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.748 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:18.748 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.750 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:18.751 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.751 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:18.751 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.751 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:18.751 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.752 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:18.752 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.752 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:18.752 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.752 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:18.753 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.753 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:18.753 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.753 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:18.754 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.754 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:18.754 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.754 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:18.754 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.754 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:18.755 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.755 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:18.755 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.755 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:18.755 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.756 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:18.756 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.756 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:18.756 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.756 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:18.756 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.757 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:18.757 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.757 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:18.757 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.757 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:18.757 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.758 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:18.758 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.758 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:18.758 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.758 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:18.759 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.759 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:18.759 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.759 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:18.759 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.760 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:18.760 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.760 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:18.760 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.760 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:18.761 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.761 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:18.761 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.761 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:18.761 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.761 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:18.762 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.762 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:18.762 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.762 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:18.762 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:18.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:18.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:18.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.764 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:18.764 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.764 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:18.764 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.764 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:18.764 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.764 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:18.764 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.765 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:18.765 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.765 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:18.765 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.765 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:18.765 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.765 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:18.765 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.766 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:18.766 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.766 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:18.766 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.766 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:18.766 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.766 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:18.766 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.766 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:18.766 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.766 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:18.766 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.766 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:18.766 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.766 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:18.767 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.767 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:18.767 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.767 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:18.767 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.767 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:18.767 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.767 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:18.767 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.767 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:18.767 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.767 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:18.767 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.768 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:18.768 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.768 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:18.768 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.768 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:18.768 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.768 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:18.768 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.768 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:18.768 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.768 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:18.768 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.768 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:18.768 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.769 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:18.769 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.769 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:18.769 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.769 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:18.769 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.769 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:18.769 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.769 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:18.769 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.769 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:18.769 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.770 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:18.770 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.770 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:18.770 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.770 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:18.770 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.771 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:18.771 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.771 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:18.771 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.771 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:18.771 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.772 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:18.772 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.772 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:18.772 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.772 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:18.772 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.772 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:18.772 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.772 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:18.772 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.772 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:18.772 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.772 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:18.772 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.773 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:18.773 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.773 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:18.773 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.773 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:18.774 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.774 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:18.774 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.774 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:18.774 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.774 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:18.774 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.774 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:18.774 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.774 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:18.774 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.774 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:18.774 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.775 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:18.775 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.775 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:18.775 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.775 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:18.775 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.775 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:18.775 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.775 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:18.775 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.775 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:18.775 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.775 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:18.775 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.775 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:18.776 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.776 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:18.776 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.776 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:18.776 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.776 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:18.776 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.776 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:18.776 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.776 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:18.776 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.776 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:18.776 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.776 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:18.777 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.777 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:18.777 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.777 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:18.777 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.777 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:18.777 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.777 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:18.777 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.777 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:18.777 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.777 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:18.777 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.778 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:18.778 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.778 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:18.778 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.778 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:18.778 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.778 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:18.778 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.778 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:18.778 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.778 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:18.778 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.778 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:18.778 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.778 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:18.779 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.779 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:18.779 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.779 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:18.779 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.779 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:18.779 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.779 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:18.779 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.779 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:18.779 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.779 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:18.779 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.779 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:18.780 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.780 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:18.780 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.780 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:18.780 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.780 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:18.780 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.780 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:18.780 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.780 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:18.780 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.781 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:18.781 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.781 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:18.781 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.781 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:18.781 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.781 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:18.781 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.781 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:18.781 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.781 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:18.781 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.781 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:18.781 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:18 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.783 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:18.783 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:18.783 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.783 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:18.783 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.783 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:18.783 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.783 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:18.783 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.783 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:18.783 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.783 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:18.784 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.784 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:18.784 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.784 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.784 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:18.784 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.784 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:18.784 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.784 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:18.784 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.784 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:18.784 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.784 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:18.785 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.785 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:18.785 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.785 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:18.785 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.785 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:18.785 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.785 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:18.785 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.786 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:18.786 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.786 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:18.786 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.786 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:18.786 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.786 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:18.786 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.786 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:18.786 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.786 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:18.786 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.786 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:18.787 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.787 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:18.787 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.787 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:18.787 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.787 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:18.787 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.787 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:18.787 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.787 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:18.787 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.787 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:18.787 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.788 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:18.788 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.788 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:18.788 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.788 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:18.788 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.788 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:18.788 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.788 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:18.788 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.788 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:18.788 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.788 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:18.788 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.789 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:18.789 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.789 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:18.789 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.789 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:18.789 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.789 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:18.789 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.789 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:18.789 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.790 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:18.790 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.790 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:18.790 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.790 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:18.790 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.790 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:18.790 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.790 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:18.790 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.790 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:18.791 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.791 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:18.791 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.791 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:18.791 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.791 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:18.791 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.791 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:18.791 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.791 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:18.791 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.791 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:18.791 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.792 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:18.792 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.792 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:18.792 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.792 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:18.792 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.792 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:18.792 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.792 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:18.792 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.792 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:18.793 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.793 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:18.793 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.793 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:18.793 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.793 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:18.793 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.793 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:18.793 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.793 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:18.793 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.793 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:18.793 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.794 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:18.794 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.794 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:18.794 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.794 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:18.794 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.794 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:18.794 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.794 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:18.794 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.794 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:18.794 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.794 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:18.795 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.795 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:18.795 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.795 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:18.795 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.795 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:18.795 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.795 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:18.795 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.795 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:18.795 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.795 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:18.795 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.795 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:18.796 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.796 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:18.796 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.796 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:18.796 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.796 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:18.796 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.796 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:18.796 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.797 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:18.797 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.797 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:18.797 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.797 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:18.797 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.797 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:18.797 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.797 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:18.797 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.797 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:18.798 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.798 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:18.798 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.798 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:18.798 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.798 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:18.798 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.798 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:18.798 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.799 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:18.799 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.799 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:18.799 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.799 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:18.799 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.799 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:18.799 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.800 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:18.800 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.800 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:18.800 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.800 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:18.800 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.800 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:18.800 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.801 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:18.801 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.801 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:18.801 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.801 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:18.801 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.802 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:18.802 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.802 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:18.802 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.802 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:18.802 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.802 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:18.802 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.802 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:18.802 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.802 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:18.803 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.803 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:18.803 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.803 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:18.803 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.803 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:18.803 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.803 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:18.803 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.803 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:18.803 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.803 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:18.803 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.804 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:18.804 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.804 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:18.804 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.804 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:18.804 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.804 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:18.804 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.804 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:18.804 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.804 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:18.805 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.805 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:18.805 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.805 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:18.805 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.805 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:18.805 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.805 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:18.805 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.805 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:18.805 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.805 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:18.805 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.806 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:18.806 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.806 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:18.806 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.806 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:18.806 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.806 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:18.806 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.806 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:18.806 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.806 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:18.806 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.806 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:18.806 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.806 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:18.807 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.807 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:18.807 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.807 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:18.807 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.807 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:18.807 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.807 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:18.807 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.807 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:18.807 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.807 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:18.807 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.807 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:18.808 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.808 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:18.808 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.808 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:18.808 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.808 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:18.808 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.808 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:18.808 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.808 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:18.808 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.808 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:18.808 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.808 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:18.808 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.809 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:18.809 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.809 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:18.809 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.809 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:18.809 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.809 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:18.809 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.809 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:18.809 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.809 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:18.809 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.809 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:18.810 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.810 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:18.810 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.810 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:18.810 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.810 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:18.810 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.810 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:18.810 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.810 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:18.810 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.810 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:18.810 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.810 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:18.810 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.811 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:18.811 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:18 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.812 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:18.812 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:18.812 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.812 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:18.812 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.812 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:18.812 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.812 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:18.812 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.812 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:18.813 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.813 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:18.813 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.813 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:18.813 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.813 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.813 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:18.813 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.813 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:18.813 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.813 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:18.813 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.813 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:18.814 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.814 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:18.814 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.814 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:18.814 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.814 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:18.814 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.814 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:18.814 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.814 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:18.814 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.814 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:18.814 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.814 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:18.815 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.815 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:18.815 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.815 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:18.815 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.815 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:18.815 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.815 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:18.815 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.815 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:18.815 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.815 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:18.815 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.815 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:18.815 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.816 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:18.816 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.816 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:18.816 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.816 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:18.816 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.816 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:18.816 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.816 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:18.816 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.816 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:18.816 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.816 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:18.817 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.817 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:18.817 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.817 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:18.817 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.817 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:18.817 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.817 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:18.817 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.817 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:18.817 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.817 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:18.817 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.817 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:18.818 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.818 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:18.818 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.818 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:18.818 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.818 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:18.818 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.818 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:18.818 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.818 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:18.818 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.818 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:18.818 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.818 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:18.819 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.819 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:18.819 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.819 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:18.819 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.819 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:18.819 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.819 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:18.819 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.819 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:18.819 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.819 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:18.819 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.819 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:18.819 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.820 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:18.820 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.820 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:18.820 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.820 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:18.820 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.820 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:18.820 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.820 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:18.820 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.820 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:18.820 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.820 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:18.821 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.821 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:18.821 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.821 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:18.821 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.821 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:18.821 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.821 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:18.821 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.821 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:18.821 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.821 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:18.822 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.822 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:18.822 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.822 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:18.822 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.822 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:18.822 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.822 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:18.822 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.822 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:18.822 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.823 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:18.823 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.823 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:18.823 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.823 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:18.823 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.823 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:18.823 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.823 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:18.823 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.823 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:18.823 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.823 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:18.823 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.824 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:18.824 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.824 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:18.824 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.824 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:18.824 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.824 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:18.824 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.824 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:18.824 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.824 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:18.824 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.824 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:18.824 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.825 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:18.825 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.825 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:18.825 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.825 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:18.825 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.825 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:18.825 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.825 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:18.825 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.825 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:18.825 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.826 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:18.826 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.826 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:18.826 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.826 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:18.826 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.826 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:18.826 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.826 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:18.826 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.826 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:18.826 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.826 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:18.826 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.827 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:18.827 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.827 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:18.827 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.827 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:18.827 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.827 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:18.827 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.827 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:18.827 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.827 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:18.827 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.828 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:18.828 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.828 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:18.828 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.828 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:18.828 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.828 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:18.828 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.828 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:18.828 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.828 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:18.828 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.828 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:18.829 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.829 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:18.829 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.829 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:18.829 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.829 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:18.829 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.829 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:18.829 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.829 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:18.829 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.829 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:18.829 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.829 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:18.830 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.830 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:18.830 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.830 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:18.830 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.830 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:18.830 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.830 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:18.830 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.830 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:18.830 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.830 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:18.830 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.831 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:18.831 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.831 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:18.831 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.831 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:18.831 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.831 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:18.831 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.831 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:18.831 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.831 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:18.831 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.831 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:18.831 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.832 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:18.832 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.832 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:18.832 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.832 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:18.832 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.832 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:18.832 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.832 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:18.832 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.832 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:18.832 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.832 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:18.832 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.833 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:18.833 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.833 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:18.833 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.833 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:18.833 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.833 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:18.833 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.833 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:18.833 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.833 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:18.833 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.833 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:18.834 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.834 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:18.834 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.834 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:18.834 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.834 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:18.834 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.834 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:18.834 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.834 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:18.834 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.834 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:18.834 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.834 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:18.834 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.835 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:18.835 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.835 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:18.835 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.835 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:18.835 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.835 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:18.835 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.835 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:18.835 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.835 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:18.835 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.835 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:18.835 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.836 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:18.836 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.836 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:18.836 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.836 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:18.836 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.836 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:18.836 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.836 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:18.836 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:18.836 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:18.836 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:18 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: pgmap v61: 97 pgs: 97 active+clean; 78 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.7 MiB/s wr, 279 op/s 2024-09-11T23:20:19.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.741 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.741 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.741 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.741 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.741 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.741 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.741 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.741 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.741 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.741 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.741 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.741 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.741 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.741 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.742 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.742 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.742 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.742 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.742 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.742 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.742 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.742 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.742 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.742 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.742 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.742 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.742 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.743 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.743 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.743 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.743 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.743 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.743 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.743 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.743 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.743 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.745 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.745 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.745 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.745 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.745 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.745 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.745 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.745 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.745 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.745 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.745 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.746 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.746 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.746 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.746 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.746 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.746 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.746 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.746 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.746 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.746 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.746 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.746 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.746 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.746 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.747 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.747 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.747 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.747 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.747 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.747 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.747 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.747 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.747 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.747 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.747 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.748 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.748 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.748 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.748 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.748 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.748 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.748 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.748 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.748 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.748 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.748 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.748 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.748 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.749 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.749 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.749 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.749 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.749 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.749 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.749 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.749 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.749 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.749 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.749 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.749 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.749 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.749 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.750 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.750 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.750 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.750 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.750 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.750 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.750 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.750 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.750 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.750 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.750 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.750 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.750 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.750 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.751 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.751 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.751 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.751 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.751 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.751 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.751 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.751 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.751 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.751 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.751 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.751 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.751 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.752 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.752 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.752 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.752 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.752 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.752 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.752 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.752 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.752 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.752 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.752 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.752 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.752 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.752 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.753 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.753 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.753 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.753 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.753 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.753 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.753 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.753 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.753 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.753 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.753 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.753 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.753 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.753 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.754 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.754 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.754 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.754 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.754 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.754 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.754 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.754 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.754 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.754 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.754 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.754 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.755 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.755 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.755 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.755 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.755 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.755 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.755 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.755 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.756 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.756 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.756 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.756 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.756 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.756 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.756 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.756 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.756 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.756 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.756 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.756 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.756 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.757 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.757 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.757 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.757 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.757 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.757 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.757 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.757 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.757 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.757 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.758 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.758 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.758 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.758 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.758 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.758 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.758 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.758 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.758 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.758 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.758 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.758 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.759 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.759 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.759 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.759 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.759 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.759 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.759 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.759 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.759 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.759 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.759 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.759 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.759 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.759 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.760 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.760 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.760 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.760 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.760 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.760 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.760 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.760 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.760 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.760 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.760 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.760 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.760 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.760 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.761 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.761 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.761 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.761 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.761 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.761 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.761 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.761 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.761 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.761 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.761 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.761 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.761 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.762 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.762 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.762 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.762 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.762 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.762 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.762 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.762 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.762 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.762 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.762 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.762 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.762 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.762 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.762 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.764 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.764 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.764 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.764 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.764 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.764 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.764 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.764 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.764 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.764 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.764 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.764 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.764 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.764 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.765 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.765 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.765 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.765 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.765 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.765 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.765 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.765 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.765 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.765 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.766 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.766 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.766 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.766 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.766 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.766 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.767 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.767 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.767 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.767 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.767 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.767 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.768 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.768 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.768 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.768 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.768 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.768 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.769 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.769 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.769 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.769 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.769 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.769 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.770 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.770 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.770 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.770 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.770 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.770 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.771 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.771 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.771 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.771 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.771 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.771 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.771 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.772 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.772 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.772 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.772 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.772 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.772 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.773 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.773 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.773 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.773 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.773 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.774 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.774 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.774 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.774 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.774 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.774 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.775 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.775 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.775 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.775 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.775 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.776 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.776 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.776 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.776 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.776 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.776 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.777 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.777 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.777 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.777 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.777 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.777 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.777 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.778 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.778 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.778 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.778 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.778 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.778 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.779 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.779 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.779 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.779 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.779 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.779 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.780 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.780 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.780 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.780 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.780 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.781 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.781 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.781 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.781 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.781 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.781 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.782 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.782 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.782 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.782 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.782 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.782 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.784 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.784 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.784 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.784 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.784 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.784 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.785 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.785 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.785 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.785 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.785 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.785 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.786 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.786 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.786 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.786 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.786 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.786 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.787 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.787 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.787 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.787 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.787 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.787 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.788 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.788 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.788 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.788 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.788 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.788 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.789 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.789 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.789 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.789 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.789 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.790 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.790 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.790 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.790 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.790 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.790 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.791 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.791 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.791 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.791 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.791 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.791 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.791 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.792 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.792 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.792 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.792 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.792 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.792 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.793 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.793 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.793 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.793 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.793 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.793 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.794 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.794 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.794 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.794 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.794 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.794 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.795 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.795 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.795 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.795 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.795 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.795 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.796 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.796 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.796 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.796 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.796 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.797 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.797 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.797 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.797 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.797 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.798 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.798 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.798 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.798 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.798 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.798 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.799 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.799 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.799 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.799 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.799 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.799 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.800 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.800 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.800 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.800 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.800 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.801 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.801 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.801 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.801 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.801 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.801 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.802 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.802 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.802 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.802 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.802 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.802 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.803 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.803 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.803 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.803 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.803 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.803 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.804 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.804 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.804 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.804 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.804 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.804 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.805 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.805 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.805 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.805 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.805 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.805 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.806 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.806 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.806 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.806 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.806 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.807 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.807 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.807 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.807 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.807 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.807 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.808 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.808 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.808 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.808 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.808 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.808 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.809 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.809 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.809 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.809 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.809 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.809 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.810 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.814 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.815 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.815 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.815 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.815 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.815 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.815 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.815 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.815 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.816 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.816 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.816 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.816 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.816 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.816 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.816 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.816 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.816 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.816 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.817 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.817 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.817 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.817 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.817 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.817 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.817 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.817 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.817 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.818 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.818 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.818 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.818 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.818 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.818 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.818 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.818 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.818 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.818 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.819 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.819 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.819 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.819 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.819 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.819 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.819 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.819 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.819 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.820 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.820 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.820 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.820 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.820 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.820 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.820 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.820 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.820 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.821 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.821 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.821 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.821 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.821 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.821 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.821 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.821 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.821 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.821 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.822 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.822 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.822 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.822 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.822 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.822 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.822 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.822 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.822 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.822 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.823 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.823 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.823 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.823 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.823 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.823 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.823 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.823 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.823 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.824 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.824 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.824 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.824 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.824 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.824 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.824 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.824 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.824 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.824 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.825 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.825 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.825 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.825 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.825 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.825 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.826 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.826 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.826 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.826 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.826 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.826 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.827 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.827 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.827 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.827 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.827 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.827 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.827 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.827 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.828 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.828 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.828 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.828 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.828 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.828 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.828 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.828 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.828 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.828 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.828 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.828 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.828 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.828 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.829 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.829 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.829 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.829 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.829 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.829 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.829 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.829 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.829 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.829 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.829 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.829 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.829 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.829 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.830 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.830 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.830 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.830 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.830 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.830 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.830 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.830 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.830 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.830 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.830 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.830 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.830 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.830 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.831 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.831 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.831 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.831 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.831 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.831 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.831 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.831 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.831 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.831 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.831 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.831 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.831 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.831 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.831 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.832 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.832 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.832 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.832 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.832 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.832 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.832 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.832 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.832 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.832 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.832 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.832 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.832 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.832 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.833 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.833 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.833 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.833 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.833 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.833 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.833 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.833 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.833 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.833 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.833 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.833 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.833 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.833 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.834 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.834 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.834 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.834 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.834 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.834 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.834 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.834 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.834 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.834 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.834 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.834 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.834 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.834 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.834 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.835 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.835 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.835 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.835 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.835 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.835 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.835 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.835 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.835 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.835 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.835 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.835 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.835 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.835 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.835 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.836 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.836 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.836 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.836 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.836 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.836 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.836 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.836 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.836 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.836 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.836 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.836 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.836 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.836 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:19.837 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:19 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.839 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: pgmap v61: 97 pgs: 97 active+clean; 78 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.7 MiB/s wr, 279 op/s 2024-09-11T23:20:19.840 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.840 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.840 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.840 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.840 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.840 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.840 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.840 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.840 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.840 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.840 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.840 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.840 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.841 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.841 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.841 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.841 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.841 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.841 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.841 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.841 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.841 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.841 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.841 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.841 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.841 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.841 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.842 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.842 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.842 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.842 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.842 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.842 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.842 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.842 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.842 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.842 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.842 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.842 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.842 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.842 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.843 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.843 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.843 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.843 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.843 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.843 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.843 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.843 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.843 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.843 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.843 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.843 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.843 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.843 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.844 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.844 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.844 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.844 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.844 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.844 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.844 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.844 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.844 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.844 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.844 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.844 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.844 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.844 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.845 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.845 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.845 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.845 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.845 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.845 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.845 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.845 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.845 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.845 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.845 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.845 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.845 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.845 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.845 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.846 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.846 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.846 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.846 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.846 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.846 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.846 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.846 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.846 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.846 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.846 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.846 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.846 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.846 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.847 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.847 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.847 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.847 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.847 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.847 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.847 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.847 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.847 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.847 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.847 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.847 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.847 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.847 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.848 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.848 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.848 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.848 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.848 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.848 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.848 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.848 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.848 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.848 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.848 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.848 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.848 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.848 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.849 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.849 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.849 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.849 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.849 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.849 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.849 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.849 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.849 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.849 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.849 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.849 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.849 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.849 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.850 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.850 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.850 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.850 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.850 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.850 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.850 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.850 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.850 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.850 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.850 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.850 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.850 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.850 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.850 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.851 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.851 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.851 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.851 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.851 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.851 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.851 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.851 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.851 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.851 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.851 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.851 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.851 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.851 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.852 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.852 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.852 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.852 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.852 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.852 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.852 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.852 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.852 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.852 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.852 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.852 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.852 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.852 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.853 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.853 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.853 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.853 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.853 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.853 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.853 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.853 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.853 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.853 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.853 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.853 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.853 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.853 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.853 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.854 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.854 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.854 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.854 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.854 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.854 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.854 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.854 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.854 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.854 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.854 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.854 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.854 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.854 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.855 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.855 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.855 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.855 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.855 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.855 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.855 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.855 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.855 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.855 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.855 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.855 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.855 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.855 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.856 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.856 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.856 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.856 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.856 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.856 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.856 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.856 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.856 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.856 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.856 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.856 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.856 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.856 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.857 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.857 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.857 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.857 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.857 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.857 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.857 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.857 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.857 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.858 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.858 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.858 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.858 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.858 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.858 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.858 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.858 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.858 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.858 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.858 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.858 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.858 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.858 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.858 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.859 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.859 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.859 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.859 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.859 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.859 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.859 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.859 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.859 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.859 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.859 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.859 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.859 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.859 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.860 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.860 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.860 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.860 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.860 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.860 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.860 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.860 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.860 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.860 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.860 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.860 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.860 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.861 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.861 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.861 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.861 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.861 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.861 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.861 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.861 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.861 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.861 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.861 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.861 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.861 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.861 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.861 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.862 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.862 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.862 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.862 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.862 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.862 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.862 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.862 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.862 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.862 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.862 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.862 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.862 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.863 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.863 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.863 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.863 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.863 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.863 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.863 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.863 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.863 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.863 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.863 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.863 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.863 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.863 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.864 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.864 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.864 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.864 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.864 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.864 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.864 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.864 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.864 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.864 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.864 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.864 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.864 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.864 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.865 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.865 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.865 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.865 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.865 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.865 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.865 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.865 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.865 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.865 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.865 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.865 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.865 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.865 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.866 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.866 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.866 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.866 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.866 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.866 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.866 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.866 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.866 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.866 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.866 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.866 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.866 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.866 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.867 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.867 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.867 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.867 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.867 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.867 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.867 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.867 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.867 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.867 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.867 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.867 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.867 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.867 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.867 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.868 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.868 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.868 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.868 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.868 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.868 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.868 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.868 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.868 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.868 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.868 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.868 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.868 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.868 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.869 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.869 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.869 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.869 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.869 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.869 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.869 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.869 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.869 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.869 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.869 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.869 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.869 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.869 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.870 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.870 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.870 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.870 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.870 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.870 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.870 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.870 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.870 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.870 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.870 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.870 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.870 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.870 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.871 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.871 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.871 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.871 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.871 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.871 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.871 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.871 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.871 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.871 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.871 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.871 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.871 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.871 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.872 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.872 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.872 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.872 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.872 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.872 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.872 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.872 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.872 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.872 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.872 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.872 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.872 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.872 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.873 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.873 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.873 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.873 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.873 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.873 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.873 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.873 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.873 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.873 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.873 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.873 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.873 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.873 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.874 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.874 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.874 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.874 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.874 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.874 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.874 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.874 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.874 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.874 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.874 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.874 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.874 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.874 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.875 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.875 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.875 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.875 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.875 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.875 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.875 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.875 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.875 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.875 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.875 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.875 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.875 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.875 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.875 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.876 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.876 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.876 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.876 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.876 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.876 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.876 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.876 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.876 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.876 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.876 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.876 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.876 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.876 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.877 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.877 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.877 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.877 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.877 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.877 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.877 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.877 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.877 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.877 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.877 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.877 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.877 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.877 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.878 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.878 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.878 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.878 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.878 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.878 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.878 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.878 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.878 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.878 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.878 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.878 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.878 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.878 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.879 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.879 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.879 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.879 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.879 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.879 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.879 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.879 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.879 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.879 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.879 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.879 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.879 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.879 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.879 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.880 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.880 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.880 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.880 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.880 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.880 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.880 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.880 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.880 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.880 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.880 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.880 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.880 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.880 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.881 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.881 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.881 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.881 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.881 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.881 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.881 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.881 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.881 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.881 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.881 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.881 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.881 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.881 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.882 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.882 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.882 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.882 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.882 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.882 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.882 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.882 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.882 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.882 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.882 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.882 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.882 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.882 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.883 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.883 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.883 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.883 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.883 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.883 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.883 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.883 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.883 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.883 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.883 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.883 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.883 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.883 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.884 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.884 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.884 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.884 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.884 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.884 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.884 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.884 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.884 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.884 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.884 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.884 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.884 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.884 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.885 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.885 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.885 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.885 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.885 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.885 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.885 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.885 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.885 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.885 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.885 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.885 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.885 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.886 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.886 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.886 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.886 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.886 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.886 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.886 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.886 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.886 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.886 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.886 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.886 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.886 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.886 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.886 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.887 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.887 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.887 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.887 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.887 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.887 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.887 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.887 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.887 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.887 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.887 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.887 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.887 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.887 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.887 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.888 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.888 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.888 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.888 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.888 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.888 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.888 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.888 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.888 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.888 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.888 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.888 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.888 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.888 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.889 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.889 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.889 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.889 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.889 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.889 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.889 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.889 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.889 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.889 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.889 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.889 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.889 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.889 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.890 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.890 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.890 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.890 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.890 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.890 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.890 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.890 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.890 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.890 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.890 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.890 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.890 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.890 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.890 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.891 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.891 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.891 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.891 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.891 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.891 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.891 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.891 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.891 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.891 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.891 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.891 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.891 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.891 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.891 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.892 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.892 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.892 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.892 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.892 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.892 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.892 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.892 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.892 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.892 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.892 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.892 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.892 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.893 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.893 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.893 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.893 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.893 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.893 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.893 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.893 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.893 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.893 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.893 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.893 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.893 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.893 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.894 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.894 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.894 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.894 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.894 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.894 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.894 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.894 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.894 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.894 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.894 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.894 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.894 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.894 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.895 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.895 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.895 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.895 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.895 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.895 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.895 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.895 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.895 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.895 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.895 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.895 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.895 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.895 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.895 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.896 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.896 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.896 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.896 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.896 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.896 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.896 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.896 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.896 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.896 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.896 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.896 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.896 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.896 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.896 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.897 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.897 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.897 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.897 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.897 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.897 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.897 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.897 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.897 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.897 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.897 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.897 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.897 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.897 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.898 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.898 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.898 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:19.898 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:19 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.898 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: pgmap v61: 97 pgs: 97 active+clean; 78 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.7 MiB/s wr, 279 op/s 2024-09-11T23:20:19.898 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.898 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.898 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.898 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.898 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.898 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.898 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.898 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.899 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.899 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.899 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.899 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.899 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.899 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.899 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.899 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.899 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.899 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.899 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.899 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.899 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.899 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.900 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.900 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.900 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.900 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.900 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.900 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.900 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.900 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.900 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.900 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.900 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.900 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.900 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.900 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.901 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.901 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.901 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.901 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.901 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.901 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.901 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.901 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.901 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.901 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.901 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.901 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.901 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.901 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.902 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.902 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.902 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.902 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.902 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.902 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.902 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.902 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.902 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.902 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.902 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.902 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.902 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.902 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.903 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.903 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.903 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.903 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.903 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.903 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.903 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.903 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.903 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.903 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.903 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.903 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.903 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.903 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.903 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.904 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.904 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.904 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.904 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.904 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.904 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.904 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.904 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.904 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.904 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.904 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.904 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.904 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.904 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.905 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.905 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.905 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.905 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.905 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.905 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.905 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.905 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.905 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.905 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.905 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.905 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.905 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.905 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.906 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.906 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.906 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.906 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.906 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.906 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.906 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.906 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.906 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.906 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.906 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.906 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.906 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.906 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.907 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.907 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.907 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.907 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.907 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.907 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.907 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.907 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.907 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.907 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.907 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.907 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.907 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.907 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.908 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.908 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.908 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.908 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.908 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.908 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.908 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.908 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.908 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.908 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.908 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.908 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.908 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.908 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.909 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.909 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.909 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.909 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.909 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.909 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.909 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.909 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.909 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.909 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.909 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.909 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.909 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.909 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.909 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.910 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.910 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.910 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.910 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.910 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.910 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.910 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.910 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.910 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.910 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.910 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.910 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.910 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.910 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.911 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.911 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.911 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.911 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.911 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.911 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.911 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.911 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.911 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.911 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.911 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.911 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.911 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.911 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.911 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.912 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.912 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.912 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.912 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.912 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.912 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.912 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.912 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.912 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.912 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.912 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.912 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.912 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.912 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.913 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.913 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.913 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.913 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.913 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.913 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.913 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.913 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.913 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.913 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.913 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.913 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.913 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.913 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.914 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.914 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.914 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.914 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.914 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.914 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.914 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.914 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.914 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.914 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.914 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.914 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.914 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.914 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.915 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.915 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.915 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.915 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.915 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.915 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.915 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.915 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.915 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.915 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.915 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.915 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.915 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.915 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.915 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.916 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.916 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.916 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.916 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.916 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.916 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.916 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.916 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.916 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.916 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.916 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.916 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.916 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.916 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.917 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.917 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.917 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.917 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.917 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.917 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.917 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.917 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.917 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.917 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.917 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.917 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.917 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.917 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.917 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.918 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.918 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.918 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.918 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.918 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.918 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.918 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.918 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.918 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.918 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.918 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.918 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.918 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.918 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.919 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.919 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.919 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.919 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.919 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.919 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.919 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.919 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.919 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.919 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.919 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.919 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.919 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.919 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.919 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.920 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.920 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.920 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.920 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.920 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.920 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.920 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.920 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.920 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.920 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.920 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.920 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.920 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.921 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.921 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.921 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.921 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.921 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.921 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.921 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.921 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.921 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.921 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.921 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.921 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.921 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.921 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.922 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.922 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.922 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.922 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.922 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.922 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.922 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.922 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.922 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.922 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.922 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.922 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.922 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.922 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.922 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.923 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.923 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.923 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.923 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.923 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.923 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.923 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.923 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.923 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.923 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.923 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.923 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.923 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.923 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.924 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.924 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.924 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.924 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.924 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.924 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.924 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.924 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.924 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.924 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.924 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.924 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.924 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.924 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.925 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.925 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.925 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.925 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.925 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.925 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.925 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.925 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.925 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.925 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.925 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.925 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.925 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.926 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.926 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.926 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.926 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.926 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.926 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.926 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.926 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.926 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.926 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.926 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.926 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.926 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.926 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.926 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.927 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.927 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.927 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.927 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.927 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.927 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.927 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.927 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.927 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.927 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.927 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.927 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.927 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.927 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.928 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.928 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.928 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.928 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.928 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.928 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.928 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.928 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.928 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.928 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.928 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.928 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.928 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.928 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.929 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.929 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.929 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.929 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.929 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.929 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.929 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.929 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.929 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.929 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.929 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.929 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.929 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.929 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.929 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.930 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.930 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.930 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.930 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.930 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.930 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.930 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.930 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.930 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.930 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.930 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.930 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.930 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.930 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.931 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.931 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.931 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.931 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.931 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.931 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.931 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.931 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.931 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.931 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.931 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.931 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.931 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.931 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.932 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.932 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.932 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.932 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.932 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.932 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.932 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.932 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.932 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.932 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.932 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.932 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.932 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.932 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.933 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.933 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.933 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.933 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.933 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.933 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.933 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.933 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.933 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.933 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.933 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.933 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.933 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.933 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.933 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.934 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.934 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.934 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.934 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.934 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.934 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.934 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.934 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.934 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.934 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.934 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.934 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.934 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.934 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.934 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.936 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.936 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.936 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.936 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.936 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.936 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.936 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.936 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.936 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.936 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.936 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.936 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.936 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.936 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.937 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.937 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.937 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.937 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.937 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.937 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.937 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.937 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.937 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.937 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.937 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.937 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.937 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.937 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.938 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.938 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.938 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.938 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.938 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.938 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.938 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.938 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.938 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.938 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.938 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.938 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.938 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.938 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.939 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.939 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.939 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.939 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.939 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.939 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.939 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.939 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.939 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.939 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.939 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.939 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.939 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.939 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.939 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.940 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.940 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.940 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.940 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.940 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.940 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.940 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.940 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.940 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.940 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.940 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.940 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.940 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.940 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.941 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.941 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.941 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.941 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.941 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.941 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.941 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.941 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.941 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.941 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.941 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.941 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.941 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.941 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.942 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.942 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.942 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.942 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.942 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.942 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.942 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.942 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.942 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.942 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.942 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.942 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.942 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.942 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.942 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.943 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.943 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.943 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.943 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.943 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.943 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.943 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.943 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.943 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.943 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.943 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.943 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.943 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.943 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.944 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.944 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.944 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.944 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.944 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.944 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.944 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.944 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.944 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.944 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.944 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.944 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.944 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.944 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.945 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.945 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.945 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.945 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.945 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.945 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.945 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.945 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.945 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.945 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.945 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.945 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.945 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.945 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.946 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.946 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.946 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.946 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.946 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.946 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.946 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.946 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.946 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.946 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.946 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.946 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.946 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.946 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.946 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.947 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.947 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.947 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.947 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.947 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.947 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.947 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.947 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.947 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.947 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.947 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.947 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.947 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.947 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.948 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.948 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.948 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.948 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.948 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.948 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.948 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.948 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.948 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.948 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.948 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.948 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.948 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.948 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.948 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.949 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.949 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.949 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.949 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.949 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.949 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.949 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.949 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.949 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.949 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.949 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.949 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.949 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.949 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.950 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.950 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.950 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.950 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.950 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.950 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.950 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.950 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.950 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.950 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.950 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.950 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.950 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.950 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.950 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.951 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.951 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.951 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.951 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.951 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.951 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.951 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.951 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.951 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.951 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.951 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.951 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.951 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.951 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.954 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.954 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.954 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.954 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.954 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.954 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.954 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.954 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.954 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.954 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.954 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.954 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.954 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.954 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.954 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.955 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.955 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.955 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.955 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.955 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.955 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.955 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.955 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.955 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.955 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.955 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:19.955 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:19.955 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:19 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.741 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.741 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.741 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.741 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.741 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.741 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.741 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.741 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.741 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.741 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.741 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.741 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.741 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.742 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.742 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.742 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.742 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.742 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.742 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.742 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.742 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.742 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.742 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.742 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.742 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.742 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.742 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.743 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.743 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.743 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.743 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.743 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.743 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.743 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.743 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.743 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.743 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.743 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.743 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.744 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.745 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.745 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.745 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.745 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.745 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.745 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.745 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.745 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.746 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.746 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.746 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.746 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.746 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.746 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.746 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.746 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.746 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.746 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.746 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.746 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.746 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.746 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.747 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.747 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.747 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.747 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.747 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.747 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.747 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.747 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.747 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.747 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.747 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.747 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.748 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.748 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.748 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.748 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.748 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.748 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.748 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.748 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.748 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.748 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.748 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.749 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.749 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.749 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.749 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.749 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.749 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.749 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.749 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.749 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.749 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.749 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.749 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.749 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.749 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.750 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.750 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.750 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.750 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.750 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.750 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.750 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.750 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.750 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.750 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.750 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.750 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.750 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.750 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.751 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.751 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.751 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.751 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.751 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.751 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.751 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.751 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.751 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.751 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.751 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.751 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.752 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.752 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.752 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.752 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.752 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.752 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.752 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.752 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.752 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.752 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.752 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.752 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.752 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.753 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.753 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.753 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.753 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.753 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.753 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.753 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.753 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.753 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.753 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.753 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.753 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.753 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.754 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.754 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.754 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.754 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.754 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.754 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.754 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.754 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.754 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.754 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.754 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.754 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.754 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.754 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.754 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.755 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.755 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.755 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.755 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.755 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.755 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.755 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.755 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.755 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.755 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.755 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.755 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.755 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.756 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.756 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.756 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.756 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.756 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.756 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.756 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.756 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.756 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.756 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.756 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.756 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.756 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.756 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.757 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.757 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.757 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.757 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.757 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.757 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.757 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.757 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.757 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.757 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.757 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.757 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.757 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.758 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.758 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.758 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.758 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.758 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.758 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.758 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.758 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.759 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.759 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.759 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.759 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.759 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.759 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.759 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.759 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.759 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.759 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.759 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.759 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.759 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.760 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.760 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.760 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.760 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.760 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.760 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.760 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.760 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.760 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.760 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.761 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.761 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.761 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.761 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.761 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.761 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.761 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.761 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.761 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.761 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.761 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.761 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.761 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.762 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.762 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.762 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.762 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.762 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.762 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.762 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.762 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.762 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.762 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.762 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.762 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.762 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.762 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.764 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.764 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.764 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.764 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.764 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.764 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.764 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.764 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.764 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.764 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.764 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.764 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.764 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.764 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.765 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.765 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.765 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.765 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.765 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.765 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.765 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.765 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.765 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.765 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.765 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.765 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.765 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.766 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.766 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.766 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.766 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.766 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.766 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.766 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.766 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.766 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.766 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.766 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.766 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.766 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.766 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.767 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.767 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.767 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.767 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.767 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.767 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.767 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.767 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.767 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.767 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.767 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.767 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.767 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.767 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.768 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.768 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.768 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.768 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.768 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.768 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.768 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.768 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.768 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.768 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.768 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.768 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.769 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.769 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.769 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.769 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.769 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.770 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.770 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.770 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.770 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.770 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.770 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.770 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.770 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.770 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.770 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.770 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.770 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.771 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.771 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.771 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.771 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.771 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.771 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.771 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.771 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.771 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.771 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.771 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.771 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.771 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.771 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.772 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.772 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.772 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.772 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.772 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.772 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.772 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.772 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.772 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.772 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.772 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.772 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.773 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.773 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.773 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.773 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: active paths [/] 2024-09-11T23:20:20.773 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:20 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.774 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.775 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.775 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.775 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.775 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.775 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.775 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.775 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.775 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.775 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.775 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.775 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.775 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.775 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.776 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.776 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.776 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.776 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.776 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.776 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.776 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.776 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.776 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.776 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.776 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.776 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.776 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.776 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.777 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.777 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.777 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.777 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.777 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.777 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.777 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.777 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.777 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.777 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.777 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.777 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.777 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.778 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.778 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.778 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.778 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.778 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.778 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.778 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.778 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.778 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.778 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.778 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.778 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.778 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.779 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.779 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.779 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.779 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.779 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.779 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.779 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.779 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.779 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.779 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.779 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.779 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.779 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.779 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.779 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.780 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.780 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.780 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.780 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.780 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.780 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.780 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.780 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.781 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.781 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.781 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.781 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.781 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.781 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.782 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.782 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.782 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.782 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.782 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.783 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.783 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.783 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.783 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.783 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.783 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.783 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.783 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.784 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.784 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.784 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.784 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.784 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.784 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.785 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.785 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.785 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.785 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.785 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.786 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.786 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.786 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.786 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.786 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.786 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.787 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.787 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.787 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.787 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.787 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.787 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.787 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.787 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.788 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.788 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.788 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.788 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.788 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.788 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.788 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.788 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.789 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.789 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.789 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.789 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.789 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.789 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.789 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.790 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.790 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.790 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.790 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.790 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.790 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.791 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.791 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.791 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.791 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.791 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.791 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.792 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.792 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.792 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.792 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.792 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.792 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.793 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.793 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.793 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.793 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.793 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.793 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.793 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.793 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.794 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.794 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.794 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.794 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.794 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.794 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.794 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.794 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.794 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.795 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.795 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.795 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.795 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.795 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.795 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.795 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.795 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.795 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.796 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.796 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.796 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.796 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.796 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.796 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.796 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.796 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.797 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.797 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.797 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.797 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.797 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.797 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.798 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.798 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.798 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.798 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.798 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.798 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.799 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.799 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.799 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.799 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.799 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.799 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.799 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.799 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.799 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.799 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.799 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.799 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.799 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.799 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.800 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.800 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.800 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.800 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.800 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.800 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.800 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.800 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.800 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.800 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.800 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.800 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.801 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.801 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.801 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.801 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.801 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.801 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.801 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.801 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.801 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.801 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.801 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.801 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.802 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.802 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.802 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.802 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.802 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.802 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.802 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.802 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.802 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.802 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.802 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.802 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.802 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.802 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.802 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.803 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.803 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.803 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.803 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.803 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.803 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.803 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.803 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.803 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.803 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.803 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.803 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.803 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.803 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.804 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.804 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.804 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.804 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.804 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.804 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.804 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.804 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.804 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.804 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.804 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.804 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.804 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.804 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.804 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.805 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.805 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.805 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.805 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.805 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.805 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.805 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.805 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.805 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.805 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.805 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.805 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.805 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.806 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.806 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.806 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.806 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.806 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.806 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.806 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.806 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.806 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.806 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.806 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.807 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.807 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.807 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.807 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.807 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.807 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.807 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.807 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.807 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.807 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.807 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.807 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.807 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.807 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.807 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.808 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.808 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.808 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.808 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.808 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.808 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.808 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.808 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.808 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.808 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.808 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.808 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.808 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.808 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.809 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.809 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.809 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.809 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.809 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.809 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.809 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.810 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.810 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.810 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.810 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.810 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.810 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.810 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.810 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.810 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.810 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.811 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.811 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.811 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.811 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.811 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.811 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.811 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.811 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.811 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.811 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.811 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.811 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.811 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.811 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.811 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.812 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.812 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.812 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.812 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.812 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.812 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.812 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.812 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.812 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.812 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.812 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.812 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.812 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.812 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.813 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.813 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.813 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.813 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.813 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.813 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.813 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.813 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.813 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.813 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.813 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.813 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.813 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.813 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.813 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.814 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.814 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.814 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.814 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.814 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.814 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.814 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.814 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.814 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: active paths [/] 2024-09-11T23:20:20.814 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:20 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.815 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.815 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.816 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.816 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.816 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.816 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.816 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.816 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.816 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.816 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.816 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.816 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.816 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.816 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.816 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.816 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.817 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.817 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.817 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.817 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.817 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.817 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.817 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.817 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.817 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.817 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.817 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.817 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.817 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.817 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.818 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.818 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.818 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.818 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.818 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.818 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.818 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.818 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.818 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.818 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.818 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.818 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.818 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.818 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.819 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.819 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.819 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.819 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.819 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.819 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.819 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.819 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.819 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.819 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.819 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.820 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.820 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.820 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.820 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.820 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.820 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.820 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.820 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.820 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.820 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.821 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.821 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.821 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.821 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.821 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.821 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.821 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.821 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.821 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.821 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.821 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.821 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.821 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.821 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.822 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.822 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.822 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.822 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.822 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.822 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.822 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.822 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.822 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.822 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.822 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.822 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.822 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.822 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.822 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.822 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.823 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.823 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.823 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.823 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.823 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.823 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.823 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.823 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.823 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.823 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.823 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.823 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.823 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.823 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.823 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.824 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.824 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.824 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.824 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.824 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.824 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.824 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.824 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.824 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.824 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.824 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.824 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.824 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.824 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.824 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.825 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.825 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.825 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.825 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.825 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.825 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.825 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.825 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.825 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.825 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.825 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.825 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.825 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.825 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.826 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.826 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.826 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.826 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.826 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.826 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.826 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.826 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.826 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.826 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.826 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.826 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.826 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.826 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.827 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.827 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.827 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.827 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.827 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.827 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.827 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.827 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.827 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.827 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.827 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.827 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.827 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.827 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.827 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.828 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.828 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.828 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.828 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.828 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.828 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.828 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.828 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.828 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.828 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.828 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.828 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.828 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.828 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.829 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.829 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.829 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.829 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.829 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.829 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.829 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.829 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.829 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.829 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.829 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.829 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.829 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.830 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.830 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.830 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.830 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.830 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.830 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.830 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.830 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.830 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.830 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.830 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.830 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.830 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.830 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.831 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.831 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.831 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.831 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.831 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.831 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.831 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.831 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.831 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.831 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.831 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.832 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.832 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.832 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.832 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.832 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.832 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.832 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.832 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.832 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.832 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.832 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.832 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.832 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.832 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.833 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.833 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.833 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.833 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.833 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.833 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.833 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.833 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.833 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.833 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.833 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.833 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.833 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.833 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.834 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.834 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.834 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.834 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.834 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.834 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.834 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.834 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.834 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.834 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.834 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.834 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.834 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.835 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.835 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.835 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.835 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.835 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.835 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.835 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.835 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.835 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.835 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.835 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.835 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.835 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.835 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.836 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.836 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.836 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.836 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.836 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.836 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.836 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.836 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.836 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.836 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.836 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.836 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.836 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.836 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.837 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.837 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.837 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.837 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.837 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.837 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.837 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.837 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.837 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.837 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.837 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.837 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.837 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.838 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.838 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.838 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.838 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.838 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.838 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.838 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.838 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.838 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.838 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.838 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.838 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.838 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.838 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.838 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.839 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.839 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.839 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.839 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.839 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.839 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.839 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.839 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.839 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.839 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.839 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.839 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.839 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.839 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.839 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.839 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.840 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.840 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.840 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.840 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.840 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.840 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.840 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.840 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.840 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.840 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.840 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.840 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.841 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.841 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.841 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.841 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.841 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.841 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.841 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.841 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.841 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.842 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.842 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.842 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.842 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.842 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.842 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.842 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.843 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.843 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.843 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.843 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.843 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.843 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.843 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.843 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.843 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.843 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.843 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.843 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.843 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.843 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.843 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.844 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.844 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.844 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.844 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.844 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.844 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.844 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.844 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.844 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.844 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.844 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.844 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.844 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.844 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.845 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.845 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.845 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.845 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.845 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.845 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.845 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.845 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.845 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.845 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.845 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.845 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.845 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.846 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:20.846 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: active paths [/] 2024-09-11T23:20:20.846 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:20 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:21.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:21 smithi136 ceph-mon[35121]: pgmap v62: 97 pgs: 97 active+clean; 78 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.9 MiB/s wr, 202 op/s 2024-09-11T23:20:21.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:21 smithi136 ceph-mon[35121]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:21.750 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:21 smithi026 ceph-mon[31172]: pgmap v62: 97 pgs: 97 active+clean; 78 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.9 MiB/s wr, 202 op/s 2024-09-11T23:20:21.750 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:21 smithi026 ceph-mon[31172]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:21.802 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:21 smithi076 ceph-mon[35296]: pgmap v62: 97 pgs: 97 active+clean; 78 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.9 MiB/s wr, 202 op/s 2024-09-11T23:20:21.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:21 smithi076 ceph-mon[35296]: scrub summary: idle+waiting paths [/] 2024-09-11T23:20:21.995 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 13.4 2024-09-11T23:20:21.996 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:20:22.312 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:20:23.576 INFO:tasks.workunit.client.0.smithi026.stdout: 62 17375 704 9856 533 1930 351 2024-09-11T23:20:23.659 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:23 smithi026 ceph-mon[31172]: pgmap v63: 97 pgs: 97 active+clean; 78 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.9 MiB/s wr, 202 op/s 2024-09-11T23:20:23.660 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:23 smithi026 ceph-mon[31172]: from='client.? 172.21.15.26:0/3792977388' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:20:23.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:22.878+0000 7f80955ac640 1 -- 172.21.15.26:0/1974382189 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8090102d30 msgr2=0x7f8090103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:23.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:22.878+0000 7f80955ac640 1 --2- 172.21.15.26:0/1974382189 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8090102d30 0x7f8090103110 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f8080009900 tx=0x7f808002a480 comp rx=0 tx=0).stop 2024-09-11T23:20:23.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:22.878+0000 7f80955ac640 1 -- 172.21.15.26:0/1974382189 shutdown_connections 2024-09-11T23:20:23.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:22.878+0000 7f80955ac640 1 --2- 172.21.15.26:0/1974382189 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8090103650 0x7f809010bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:23.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:22.878+0000 7f80955ac640 1 --2- 172.21.15.26:0/1974382189 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f809010c160 0x7f809010e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:23.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:22.878+0000 7f80955ac640 1 --2- 172.21.15.26:0/1974382189 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8090102d30 0x7f8090103110 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:23.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:22.878+0000 7f80955ac640 1 -- 172.21.15.26:0/1974382189 >> 172.21.15.26:0/1974382189 conn(0x7f80900fc790 msgr2=0x7f80900febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:23.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:22.878+0000 7f80955ac640 1 -- 172.21.15.26:0/1974382189 shutdown_connections 2024-09-11T23:20:23.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:22.878+0000 7f80955ac640 1 -- 172.21.15.26:0/1974382189 wait complete. 2024-09-11T23:20:23.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:22.879+0000 7f80955ac640 1 Processor -- start 2024-09-11T23:20:23.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:22.879+0000 7f80955ac640 1 -- start start 2024-09-11T23:20:23.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:22.879+0000 7f80955ac640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8090102d30 0x7f80901984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:23.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:22.879+0000 7f80955ac640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8090103650 0x7f8090198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:23.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:22.879+0000 7f80955ac640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f809010c160 0x7f809019cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:23.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:22.879+0000 7f80955ac640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f8090111f60 con 0x7f809010c160 2024-09-11T23:20:23.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:22.879+0000 7f80955ac640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f8090111de0 con 0x7f8090102d30 2024-09-11T23:20:23.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:22.879+0000 7f80955ac640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f80901120e0 con 0x7f8090103650 2024-09-11T23:20:23.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:22.879+0000 7f808f7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f809010c160 0x7f809019cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:23.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:22.879+0000 7f808f7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f809010c160 0x7f809019cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34638/0 (socket says 172.21.15.26:34638) 2024-09-11T23:20:23.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:22.879+0000 7f808f7fe640 1 -- 172.21.15.26:0/3792977388 learned_addr learned my addr 172.21.15.26:0/3792977388 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:20:23.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:22.879+0000 7f8087fff640 1 --2- 172.21.15.26:0/3792977388 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8090103650 0x7f8090198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:23.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:22.879+0000 7f808effd640 1 --2- 172.21.15.26:0/3792977388 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8090102d30 0x7f80901984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:23.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:22.880+0000 7f808f7fe640 1 -- 172.21.15.26:0/3792977388 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8090103650 msgr2=0x7f8090198a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:23.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:22.880+0000 7f808f7fe640 1 --2- 172.21.15.26:0/3792977388 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8090103650 0x7f8090198a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:23.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:22.880+0000 7f808f7fe640 1 -- 172.21.15.26:0/3792977388 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8090102d30 msgr2=0x7f80901984d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:23.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:22.880+0000 7f808f7fe640 1 --2- 172.21.15.26:0/3792977388 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8090102d30 0x7f80901984d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:23.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:22.880+0000 7f808f7fe640 1 -- 172.21.15.26:0/3792977388 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f809019d480 con 0x7f809010c160 2024-09-11T23:20:23.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:22.880+0000 7f808f7fe640 1 --2- 172.21.15.26:0/3792977388 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f809010c160 0x7f809019cda0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f808800eb10 tx=0x7f808800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:23.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:22.880+0000 7f808cff9640 1 -- 172.21.15.26:0/3792977388 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8088004b20 con 0x7f809010c160 2024-09-11T23:20:23.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:22.880+0000 7f808cff9640 1 -- 172.21.15.26:0/3792977388 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8088004cc0 con 0x7f809010c160 2024-09-11T23:20:23.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:22.880+0000 7f80955ac640 1 -- 172.21.15.26:0/3792977388 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f809019d770 con 0x7f809010c160 2024-09-11T23:20:23.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:22.880+0000 7f808cff9640 1 -- 172.21.15.26:0/3792977388 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8088010760 con 0x7f809010c160 2024-09-11T23:20:23.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:22.880+0000 7f80955ac640 1 -- 172.21.15.26:0/3792977388 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80901a5050 con 0x7f809010c160 2024-09-11T23:20:23.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:22.881+0000 7f80955ac640 1 -- 172.21.15.26:0/3792977388 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8090103110 con 0x7f809010c160 2024-09-11T23:20:23.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:22.882+0000 7f808cff9640 1 -- 172.21.15.26:0/3792977388 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f8088005020 con 0x7f809010c160 2024-09-11T23:20:23.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:22.883+0000 7f808cff9640 1 --2- 172.21.15.26:0/3792977388 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f807c0a2d70 0x7f807c0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:23.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:22.883+0000 7f808effd640 1 --2- 172.21.15.26:0/3792977388 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f807c0a2d70 0x7f807c0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:23.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:22.883+0000 7f808cff9640 1 -- 172.21.15.26:0/3792977388 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f80880cf8d0 con 0x7f809010c160 2024-09-11T23:20:23.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:22.883+0000 7f808effd640 1 --2- 172.21.15.26:0/3792977388 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f807c0a2d70 0x7f807c0a5230 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f8080009230 tx=0x7f80800058d0 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:23.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:22.888+0000 7f808cff9640 1 -- 172.21.15.26:0/3792977388 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f808809c500 con 0x7f809010c160 2024-09-11T23:20:23.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:23.127+0000 7f80955ac640 1 -- 172.21.15.26:0/3792977388 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8090107260 con 0x7f809010c160 2024-09-11T23:20:23.664 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-11T23:20:23.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:23.128+0000 7f808cff9640 1 -- 172.21.15.26:0/3792977388 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+11986 (secure 0 0 0) 0x7f808809be00 con 0x7f809010c160 2024-09-11T23:20:23.664 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":33,"btime":"2024-09-11T23:20:14:305449+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14499,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/367659515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":367659515},{"type":"v1","addr":"172.21.15.26:6835","nonce":367659515}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14511,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/2041065299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":2041065299},{"type":"v1","addr":"172.21.15.26:6837","nonce":2041065299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":33,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:16:26.544203+0000","modified":"2024-09-11T23:20:13.958301+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24485,"mds_1":24463,"mds_2":24458,"mds_3":24448,"mds_4":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14550":{"gid":14550,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6839/2233068305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":2233068305},{"type":"v1","addr":"172.21.15.26:6839","nonce":2233068305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14562":{"gid":14562,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6841/2984982336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":2984982336},{"type":"v1","addr":"172.21.15.26:6841","nonce":2984982336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6833/2019841455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":2019841455},{"type":"v1","addr":"172.21.15.136:6833","nonce":2019841455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.76:6833/3438147050","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3438147050},{"type":"v1","addr":"172.21.15.76:6833","nonce":3438147050}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6835/3617665975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3617665975},{"type":"v1","addr":"172.21.15.136:6835","nonce":3617665975}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":12,"addr":"172.21.15.76:6835/393308499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":393308499},{"type":"v1","addr":"172.21.15.76:6835","nonce":393308499}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":10,"addr":"172.21.15.136:6837/1633082387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":1633082387},{"type":"v1","addr":"172.21.15.136:6837","nonce":1633082387}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.76:6837/2078978748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":2078978748},{"type":"v1","addr":"172.21.15.76:6837","nonce":2078978748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":63,"addr":"172.21.15.136:6839/3211398630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":3211398630},{"type":"v1","addr":"172.21.15.136:6839","nonce":3211398630}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.76:6839/2077637090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6838","nonce":2077637090},{"type":"v1","addr":"172.21.15.76:6839","nonce":2077637090}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24485,"qdb_cluster":[24443,24448,24458,24463,24485]},"id":1}]} 2024-09-11T23:20:23.665 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 33 2024-09-11T23:20:23.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:23.130+0000 7f80955ac640 1 -- 172.21.15.26:0/3792977388 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f807c0a2d70 msgr2=0x7f807c0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:23.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:23.130+0000 7f80955ac640 1 --2- 172.21.15.26:0/3792977388 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f807c0a2d70 0x7f807c0a5230 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f8080009230 tx=0x7f80800058d0 comp rx=0 tx=0).stop 2024-09-11T23:20:23.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:23.130+0000 7f80955ac640 1 -- 172.21.15.26:0/3792977388 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f809010c160 msgr2=0x7f809019cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:23.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:23.130+0000 7f80955ac640 1 --2- 172.21.15.26:0/3792977388 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f809010c160 0x7f809019cda0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f808800eb10 tx=0x7f808800c2d0 comp rx=0 tx=0).stop 2024-09-11T23:20:23.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:23.130+0000 7f80955ac640 1 -- 172.21.15.26:0/3792977388 shutdown_connections 2024-09-11T23:20:23.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:23.130+0000 7f80955ac640 1 --2- 172.21.15.26:0/3792977388 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f807c0a2d70 0x7f807c0a5230 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:23.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:23.130+0000 7f80955ac640 1 --2- 172.21.15.26:0/3792977388 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f809010c160 0x7f809019cda0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:23.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:23.130+0000 7f80955ac640 1 --2- 172.21.15.26:0/3792977388 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8090103650 0x7f8090198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:23.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:23.130+0000 7f80955ac640 1 --2- 172.21.15.26:0/3792977388 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8090102d30 0x7f80901984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:23.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:23.130+0000 7f80955ac640 1 -- 172.21.15.26:0/3792977388 >> 172.21.15.26:0/3792977388 conn(0x7f80900fc790 msgr2=0x7f80900fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:23.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:23.130+0000 7f80955ac640 1 -- 172.21.15.26:0/3792977388 shutdown_connections 2024-09-11T23:20:23.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:23.130+0000 7f80955ac640 1 -- 172.21.15.26:0/3792977388 wait complete. 2024-09-11T23:20:23.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:23 smithi136 ceph-mon[35121]: pgmap v63: 97 pgs: 97 active+clean; 78 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.9 MiB/s wr, 202 op/s 2024-09-11T23:20:23.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:23 smithi136 ceph-mon[35121]: from='client.? 172.21.15.26:0/3792977388' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:20:23.802 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:23 smithi076 ceph-mon[35296]: pgmap v63: 97 pgs: 97 active+clean; 78 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.9 MiB/s wr, 202 op/s 2024-09-11T23:20:23.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:23 smithi076 ceph-mon[35296]: from='client.? 172.21.15.26:0/3792977388' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:20:24.512 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24485 quiesce db / --timeout 90 --expiration 133.4 --await-for 45.0' 2024-09-11T23:20:24.513 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24485 quiesce db / --timeout 90 --expiration 133.4 --await-for 45.0 2024-09-11T23:20:24.836 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:20:25.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.316+0000 7f30da6ed640 1 -- 172.21.15.26:0/281136192 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f30d4102ef0 msgr2=0x7f30d410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:25.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.316+0000 7f30da6ed640 1 --2- 172.21.15.26:0/281136192 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f30d4102ef0 0x7f30d410da40 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f30cc009ca0 tx=0x7f30cc02a950 comp rx=0 tx=0).stop 2024-09-11T23:20:25.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.316+0000 7f30da6ed640 1 -- 172.21.15.26:0/281136192 shutdown_connections 2024-09-11T23:20:25.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.316+0000 7f30da6ed640 1 --2- 172.21.15.26:0/281136192 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f30d4102ef0 0x7f30d410da40 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:25.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.316+0000 7f30da6ed640 1 --2- 172.21.15.26:0/281136192 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f30d4102570 0x7f30d41029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:25.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.316+0000 7f30da6ed640 1 --2- 172.21.15.26:0/281136192 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f30d4100e20 0x7f30d4101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:25.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.316+0000 7f30da6ed640 1 -- 172.21.15.26:0/281136192 >> 172.21.15.26:0/281136192 conn(0x7f30d40fc7d0 msgr2=0x7f30d40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:25.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.316+0000 7f30da6ed640 1 -- 172.21.15.26:0/281136192 shutdown_connections 2024-09-11T23:20:25.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.316+0000 7f30da6ed640 1 -- 172.21.15.26:0/281136192 wait complete. 2024-09-11T23:20:25.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.317+0000 7f30da6ed640 1 Processor -- start 2024-09-11T23:20:25.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.317+0000 7f30da6ed640 1 -- start start 2024-09-11T23:20:25.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.317+0000 7f30da6ed640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f30d4100e20 0x7f30d40fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:25.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.317+0000 7f30da6ed640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f30d4102570 0x7f30d40fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:25.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.317+0000 7f30da6ed640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f30d4102ef0 0x7f30d40fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:25.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.317+0000 7f30da6ed640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f30d4110f80 con 0x7f30d4100e20 2024-09-11T23:20:25.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.317+0000 7f30da6ed640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f30d4110e00 con 0x7f30d4102570 2024-09-11T23:20:25.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.317+0000 7f30da6ed640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f30d4111100 con 0x7f30d4102ef0 2024-09-11T23:20:25.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.317+0000 7f30d3fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f30d4100e20 0x7f30d40fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:25.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.317+0000 7f30d3fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f30d4100e20 0x7f30d40fcc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34650/0 (socket says 172.21.15.26:34650) 2024-09-11T23:20:25.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.317+0000 7f30d3fff640 1 -- 172.21.15.26:0/292909952 learned_addr learned my addr 172.21.15.26:0/292909952 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:20:25.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.317+0000 7f30d8c8a640 1 --2- 172.21.15.26:0/292909952 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f30d4102ef0 0x7f30d40fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:25.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.317+0000 7f30d37fe640 1 --2- 172.21.15.26:0/292909952 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f30d4102570 0x7f30d40fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:25.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.317+0000 7f30d3fff640 1 -- 172.21.15.26:0/292909952 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f30d4102ef0 msgr2=0x7f30d40fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:25.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.317+0000 7f30d3fff640 1 --2- 172.21.15.26:0/292909952 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f30d4102ef0 0x7f30d40fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:25.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.317+0000 7f30d3fff640 1 -- 172.21.15.26:0/292909952 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f30d4102570 msgr2=0x7f30d40fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:25.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.317+0000 7f30d3fff640 1 --2- 172.21.15.26:0/292909952 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f30d4102570 0x7f30d40fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:25.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.317+0000 7f30d3fff640 1 -- 172.21.15.26:0/292909952 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30d41a6890 con 0x7f30d4100e20 2024-09-11T23:20:25.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.318+0000 7f30d3fff640 1 --2- 172.21.15.26:0/292909952 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f30d4100e20 0x7f30d40fcc80 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f30c400c920 tx=0x7f30c400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:25.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.318+0000 7f30d17fa640 1 -- 172.21.15.26:0/292909952 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30c4007c70 con 0x7f30d4100e20 2024-09-11T23:20:25.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.318+0000 7f30d17fa640 1 -- 172.21.15.26:0/292909952 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f30c4007e10 con 0x7f30d4100e20 2024-09-11T23:20:25.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.318+0000 7f30da6ed640 1 -- 172.21.15.26:0/292909952 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30d41a6b80 con 0x7f30d4100e20 2024-09-11T23:20:25.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.318+0000 7f30d17fa640 1 -- 172.21.15.26:0/292909952 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30c4018890 con 0x7f30d4100e20 2024-09-11T23:20:25.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.318+0000 7f30da6ed640 1 -- 172.21.15.26:0/292909952 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30d41a7110 con 0x7f30d4100e20 2024-09-11T23:20:25.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.320+0000 7f30d17fa640 1 -- 172.21.15.26:0/292909952 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f30c4018a70 con 0x7f30d4100e20 2024-09-11T23:20:25.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.321+0000 7f30dcac6740 1 Processor -- start 2024-09-11T23:20:25.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.321+0000 7f30dcac6740 1 -- start start 2024-09-11T23:20:25.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.321+0000 7f30dcac6740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e06f7cf760 0x55e06f7cfb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:25.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.321+0000 7f30d17fa640 1 --2- 172.21.15.26:0/292909952 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f30bc0a2d70 0x7f30bc0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:25.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.321+0000 7f30dcac6740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e06f7d0080 0x55e06f7da400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:25.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.321+0000 7f30dcac6740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e06f7da940 0x55e06f7dcd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:25.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.321+0000 7f30dcac6740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55e06f7352a0 con 0x55e06f7cf760 2024-09-11T23:20:25.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.321+0000 7f30dcac6740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55e06f6dcb10 con 0x55e06f7da940 2024-09-11T23:20:25.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.321+0000 7f30dcac6740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55e06f731260 con 0x55e06f7d0080 2024-09-11T23:20:25.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.321+0000 7f30d3fff640 1 --2- 172.21.15.26:0/292909952 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f30bc0a2d70 0x7f30bc0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:25.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.321+0000 7f30d8c8a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e06f7cf760 0x55e06f7cfb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:25.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.321+0000 7f30d8c8a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e06f7cf760 0x55e06f7cfb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34666/0 (socket says 172.21.15.26:34666) 2024-09-11T23:20:25.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.321+0000 7f30d8c8a640 1 -- 172.21.15.26:0/1518011776 learned_addr learned my addr 172.21.15.26:0/1518011776 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:20:25.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.321+0000 7f30d37fe640 1 --2- 172.21.15.26:0/1518011776 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e06f7d0080 0x55e06f7da400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:25.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.321+0000 7f30d17fa640 1 -- 172.21.15.26:0/292909952 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f30c40cf500 con 0x7f30d4100e20 2024-09-11T23:20:25.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.321+0000 7f30d8c8a640 1 --2- 172.21.15.26:0/1518011776 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e06f7da940 0x55e06f7dcd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:25.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.322+0000 7f30d37fe640 1 -- 172.21.15.26:0/1518011776 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e06f7da940 msgr2=0x55e06f7dcd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:20:25.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.322+0000 7f30d37fe640 1 --2- 172.21.15.26:0/1518011776 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e06f7da940 0x55e06f7dcd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:25.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.322+0000 7f30d37fe640 1 -- 172.21.15.26:0/1518011776 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e06f7cf760 msgr2=0x55e06f7cfb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:20:25.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.322+0000 7f30d37fe640 1 --2- 172.21.15.26:0/1518011776 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e06f7cf760 0x55e06f7cfb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:25.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.322+0000 7f30d37fe640 1 -- 172.21.15.26:0/1518011776 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e06f4a6b90 con 0x55e06f7d0080 2024-09-11T23:20:25.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.322+0000 7f30d8c8a640 1 --2- 172.21.15.26:0/1518011776 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e06f7cf760 0x55e06f7cfb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:20:25.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.322+0000 7f30d3fff640 1 --2- 172.21.15.26:0/292909952 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f30bc0a2d70 0x7f30bc0a5230 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f30c4097e90 tx=0x7f30c4097cd0 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:25.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.322+0000 7f30d37fe640 1 --2- 172.21.15.26:0/1518011776 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e06f7d0080 0x55e06f7da400 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f30c800bb30 tx=0x7f30c800f360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1f73c05052df2aaf server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:25.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.322+0000 7f30ba7fc640 1 -- 172.21.15.26:0/1518011776 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30c8015070 con 0x55e06f7d0080 2024-09-11T23:20:25.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.322+0000 7f30ba7fc640 1 -- 172.21.15.26:0/1518011776 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f30c800fd90 con 0x55e06f7d0080 2024-09-11T23:20:25.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.322+0000 7f30ba7fc640 1 -- 172.21.15.26:0/1518011776 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30c8011440 con 0x55e06f7d0080 2024-09-11T23:20:25.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.323+0000 7f30dcac6740 1 -- 172.21.15.26:0/1518011776 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e06f7d0080 msgr2=0x55e06f7da400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:25.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.323+0000 7f30dcac6740 1 --2- 172.21.15.26:0/1518011776 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e06f7d0080 0x55e06f7da400 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f30c800bb30 tx=0x7f30c800f360 comp rx=0 tx=0).stop 2024-09-11T23:20:25.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.323+0000 7f30dcac6740 1 -- 172.21.15.26:0/1518011776 shutdown_connections 2024-09-11T23:20:25.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.323+0000 7f30dcac6740 1 --2- 172.21.15.26:0/1518011776 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e06f7da940 0x55e06f7dcd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:25.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.323+0000 7f30dcac6740 1 --2- 172.21.15.26:0/1518011776 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e06f7d0080 0x55e06f7da400 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:25.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.323+0000 7f30dcac6740 1 --2- 172.21.15.26:0/1518011776 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e06f7cf760 0x55e06f7cfb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:25.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.323+0000 7f30dcac6740 1 -- 172.21.15.26:0/1518011776 >> 172.21.15.26:0/1518011776 conn(0x55e06f7cbcf0 msgr2=0x55e06f7cc0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:25.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.323+0000 7f30dcac6740 1 -- 172.21.15.26:0/1518011776 shutdown_connections 2024-09-11T23:20:25.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.323+0000 7f30dcac6740 1 -- 172.21.15.26:0/1518011776 wait complete. 2024-09-11T23:20:25.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.323+0000 7f30dcac6740 1 -- start start 2024-09-11T23:20:25.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.324+0000 7f30dcac6740 1 Processor -- start 2024-09-11T23:20:25.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.324+0000 7f30dcac6740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e06f7cf760 0x55e06f7d77b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:25.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.324+0000 7f30dcac6740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e06f7d0080 0x55e06f7d7cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:25.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.324+0000 7f30dcac6740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e06f7da940 0x55e06f7d8230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:25.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.324+0000 7f30dcac6740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55e06f7dd510 con 0x55e06f7da940 2024-09-11T23:20:25.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.324+0000 7f30dcac6740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55e06f6bd6e0 con 0x55e06f7cf760 2024-09-11T23:20:25.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.324+0000 7f30dcac6740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55e06f7dd690 con 0x55e06f7d0080 2024-09-11T23:20:25.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.324+0000 7f30dcac6740 10 client.0 _ll_register_callbacks cb 0x55e06f6e1a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:20:25.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.324+0000 7f30d8c8a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e06f7da940 0x55e06f7d8230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:25.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.324+0000 7f30d37fe640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e06f7d0080 0x55e06f7d7cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:25.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.324+0000 7f30d8c8a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e06f7da940 0x55e06f7d8230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34682/0 (socket says 172.21.15.26:34682) 2024-09-11T23:20:25.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.324+0000 7f30d8c8a640 1 -- 172.21.15.26:0/1717813663 learned_addr learned my addr 172.21.15.26:0/1717813663 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:20:25.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.324+0000 7f30d37fe640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e06f7d0080 0x55e06f7d7cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:44564/0 (socket says 172.21.15.26:44564) 2024-09-11T23:20:25.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.324+0000 7f30d8c8a640 1 --2- 172.21.15.26:0/1717813663 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e06f7cf760 0x55e06f7d77b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:25.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.324+0000 7f30d8c8a640 1 -- 172.21.15.26:0/1717813663 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e06f7d0080 msgr2=0x55e06f7d7cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:20:25.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.324+0000 7f30d8c8a640 1 --2- 172.21.15.26:0/1717813663 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e06f7d0080 0x55e06f7d7cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:25.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.324+0000 7f30d8c8a640 1 -- 172.21.15.26:0/1717813663 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e06f7cf760 msgr2=0x55e06f7d77b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:20:25.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.324+0000 7f30d8c8a640 1 --2- 172.21.15.26:0/1717813663 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e06f7cf760 0x55e06f7d77b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:25.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.324+0000 7f30d8c8a640 1 -- 172.21.15.26:0/1717813663 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e06f7e69b0 con 0x55e06f7da940 2024-09-11T23:20:25.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.324+0000 7f30d8c8a640 1 --2- 172.21.15.26:0/1717813663 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e06f7da940 0x55e06f7d8230 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f30cc005e30 tx=0x7f30cc005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31fa5df61074c1ac server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:25.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.324+0000 7f30b37fe640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:20:25.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.325+0000 7f30b37fe640 1 -- 172.21.15.26:0/1717813663 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30cc043040 con 0x55e06f7da940 2024-09-11T23:20:25.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.325+0000 7f30b37fe640 1 -- 172.21.15.26:0/1717813663 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f30cc003b80 con 0x55e06f7da940 2024-09-11T23:20:25.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.325+0000 7f30dcac6740 1 -- 172.21.15.26:0/1717813663 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f30c800fd90 con 0x55e06f7da940 2024-09-11T23:20:25.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.325+0000 7f30b37fe640 1 -- 172.21.15.26:0/1717813663 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f30cc034040 con 0x55e06f7da940 2024-09-11T23:20:25.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.325+0000 7f30dcac6740 10 client.15996 fetch_fsmap learned FSMap version 33 2024-09-11T23:20:25.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.325+0000 7f30dcac6740 1 -- 172.21.15.26:0/1717813663 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55e06f7e9340 con 0x55e06f7da940 2024-09-11T23:20:25.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.325+0000 7f30b37fe640 1 -- 172.21.15.26:0/1717813663 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f30c800fd90 con 0x55e06f7da940 2024-09-11T23:20:25.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.325+0000 7f30b37fe640 1 -- 172.21.15.26:0/1717813663 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 33) ==== 8394+0+0 (secure 0 0 0) 0x7f30cc049040 con 0x55e06f7da940 2024-09-11T23:20:25.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.325+0000 7f30dcac6740 10 client.15996 fetch_fsmap finished waiting for FSMap version 33 2024-09-11T23:20:25.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.325+0000 7f30dcac6740 10 client.15996 resolve_mds: validated gid 24485 aka daemon mds.b 2024-09-11T23:20:25.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.325+0000 7f30dcac6740 20 client.15996 populate_metadata read hostname 'smithi026' 2024-09-11T23:20:25.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.325+0000 7f30dcac6740 1 --2- 172.21.15.26:0/1717813663 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55e06f7ea8a0 0x55e06f7eccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:25.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.325+0000 7f30dcac6740 4 client.15996 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:20:25.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.325+0000 7f30dcac6740 1 -- 172.21.15.26:0/1717813663 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e06f7429d0 con 0x55e06f7ea8a0 2024-09-11T23:20:25.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.326+0000 7f30d37fe640 1 --2- 172.21.15.26:0/1717813663 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55e06f7ea8a0 0x55e06f7eccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:25.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.326+0000 7f30d37fe640 1 --2- 172.21.15.26:0/1717813663 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55e06f7ea8a0 0x55e06f7eccf0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=88fc3a84aadd0fdb server_cookie=d28dd5a2f1559295 in_seq=0 out_seq=0 2024-09-11T23:20:25.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.326+0000 7f30b37fe640 10 client.15996 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:20:25.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.327+0000 7f30b37fe640 1 -- 172.21.15.26:0/1717813663 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e06f7429d0 con 0x55e06f7ea8a0 2024-09-11T23:20:25.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.327+0000 7f30b37fe640 10 client.15996 handle_command_reply: tid=0 2024-09-11T23:20:25.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.327+0000 7f30dcac6740 1 client.15996 shutdown 2024-09-11T23:20:25.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.327+0000 7f30dcac6740 1 -- 172.21.15.26:0/1717813663 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e06f7da940 msgr2=0x55e06f7d8230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:25.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.327+0000 7f30dcac6740 1 --2- 172.21.15.26:0/1717813663 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e06f7da940 0x55e06f7d8230 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f30cc005e30 tx=0x7f30cc005c50 comp rx=0 tx=0).stop 2024-09-11T23:20:25.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.327+0000 7f30dcac6740 1 -- 172.21.15.26:0/1717813663 shutdown_connections 2024-09-11T23:20:25.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.327+0000 7f30dcac6740 1 --2- 172.21.15.26:0/1717813663 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55e06f7ea8a0 0x55e06f7eccf0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:25.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.327+0000 7f30dcac6740 1 --2- 172.21.15.26:0/1717813663 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e06f7da940 0x55e06f7d8230 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:25.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.327+0000 7f30dcac6740 1 --2- 172.21.15.26:0/1717813663 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e06f7d0080 0x55e06f7d7cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:25.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.327+0000 7f30dcac6740 1 --2- 172.21.15.26:0/1717813663 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e06f7cf760 0x55e06f7d77b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:25.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.327+0000 7f30dcac6740 1 -- 172.21.15.26:0/1717813663 >> 172.21.15.26:0/1717813663 conn(0x55e06f7cbcf0 msgr2=0x55e06f7cae70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:25.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.327+0000 7f30b37fe640 0 client.15996 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:20:25.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.328+0000 7f30dcac6740 1 -- 172.21.15.26:0/1717813663 shutdown_connections 2024-09-11T23:20:25.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.328+0000 7f30dcac6740 1 -- 172.21.15.26:0/1717813663 wait complete. 2024-09-11T23:20:25.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.328+0000 7f30dcac6740 20 client.15996 trim_cache size 0 max 16384 2024-09-11T23:20:25.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.347+0000 7f30dcac6740 1 Processor -- start 2024-09-11T23:20:25.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.347+0000 7f30dcac6740 1 -- start start 2024-09-11T23:20:25.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.347+0000 7f30dcac6740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e06f7cf760 0x55e06f7d18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:25.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.347+0000 7f30dcac6740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e06f7d0080 0x55e06f7f2600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:25.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.347+0000 7f30dcac6740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e06f7da940 0x55e06f7f2b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:25.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.347+0000 7f30dcac6740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f30d41af9c0 con 0x55e06f7d0080 2024-09-11T23:20:25.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.347+0000 7f30dcac6740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f30d41b1680 con 0x55e06f7cf760 2024-09-11T23:20:25.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.347+0000 7f30dcac6740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f30d41b0090 con 0x55e06f7da940 2024-09-11T23:20:25.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.348+0000 7f30d37fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e06f7d0080 0x55e06f7f2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:25.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.348+0000 7f30d37fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e06f7d0080 0x55e06f7f2600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34696/0 (socket says 172.21.15.26:34696) 2024-09-11T23:20:25.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.348+0000 7f30d37fe640 1 -- 172.21.15.26:0/2175203319 learned_addr learned my addr 172.21.15.26:0/2175203319 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:20:25.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.348+0000 7f30d8c8a640 1 --2- 172.21.15.26:0/2175203319 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e06f7da940 0x55e06f7f2b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:25.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.348+0000 7f30d8c8a640 1 --2- 172.21.15.26:0/2175203319 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e06f7cf760 0x55e06f7d18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:25.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.348+0000 7f30d37fe640 1 -- 172.21.15.26:0/2175203319 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e06f7da940 msgr2=0x55e06f7f2b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:20:25.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.348+0000 7f30d37fe640 1 --2- 172.21.15.26:0/2175203319 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e06f7da940 0x55e06f7f2b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:25.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.348+0000 7f30d37fe640 1 -- 172.21.15.26:0/2175203319 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e06f7cf760 msgr2=0x55e06f7d18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:20:25.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.348+0000 7f30d37fe640 1 --2- 172.21.15.26:0/2175203319 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e06f7cf760 0x55e06f7d18c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:25.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.348+0000 7f30d37fe640 1 -- 172.21.15.26:0/2175203319 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e06f7c7260 con 0x55e06f7d0080 2024-09-11T23:20:25.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.348+0000 7f30d37fe640 1 --2- 172.21.15.26:0/2175203319 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e06f7d0080 0x55e06f7f2600 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f30c800bb30 tx=0x7f30c800b4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70f77032c5f243dd server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:25.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.348+0000 7f30b9ffb640 1 -- 172.21.15.26:0/2175203319 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30c8015040 con 0x55e06f7d0080 2024-09-11T23:20:25.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.348+0000 7f30b9ffb640 1 -- 172.21.15.26:0/2175203319 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f30c8004680 con 0x55e06f7d0080 2024-09-11T23:20:25.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.348+0000 7f30b9ffb640 1 -- 172.21.15.26:0/2175203319 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30c8003d90 con 0x55e06f7d0080 2024-09-11T23:20:25.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.349+0000 7f30dcac6740 1 -- 172.21.15.26:0/2175203319 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e06f7d0080 msgr2=0x55e06f7f2600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:25.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.349+0000 7f30dcac6740 1 --2- 172.21.15.26:0/2175203319 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e06f7d0080 0x55e06f7f2600 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f30c800bb30 tx=0x7f30c800b4d0 comp rx=0 tx=0).stop 2024-09-11T23:20:25.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.349+0000 7f30dcac6740 1 -- 172.21.15.26:0/2175203319 shutdown_connections 2024-09-11T23:20:25.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.349+0000 7f30dcac6740 1 --2- 172.21.15.26:0/2175203319 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e06f7da940 0x55e06f7f2b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:25.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.349+0000 7f30dcac6740 1 --2- 172.21.15.26:0/2175203319 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e06f7d0080 0x55e06f7f2600 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:25.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.349+0000 7f30dcac6740 1 --2- 172.21.15.26:0/2175203319 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e06f7cf760 0x55e06f7d18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:25.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.349+0000 7f30dcac6740 1 -- 172.21.15.26:0/2175203319 >> 172.21.15.26:0/2175203319 conn(0x55e06f7cbcf0 msgr2=0x55e06f7d11b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:25.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.349+0000 7f30dcac6740 1 -- 172.21.15.26:0/2175203319 shutdown_connections 2024-09-11T23:20:25.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.349+0000 7f30dcac6740 1 -- 172.21.15.26:0/2175203319 wait complete. 2024-09-11T23:20:25.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.349+0000 7f30dcac6740 1 -- start start 2024-09-11T23:20:25.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.350+0000 7f30dcac6740 1 Processor -- start 2024-09-11T23:20:25.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.350+0000 7f30dcac6740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e06f7cf760 0x55e06f7e29d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:25.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.350+0000 7f30dcac6740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e06f7d0080 0x55e06f7e2f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:25.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.350+0000 7f30dcac6740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e06f7da940 0x55e06f7dee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:25.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.350+0000 7f30dcac6740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f30d41b0ef0 con 0x55e06f7cf760 2024-09-11T23:20:25.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.350+0000 7f30dcac6740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f30d41b0760 con 0x55e06f7d0080 2024-09-11T23:20:25.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.350+0000 7f30dcac6740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f30d41ad9a0 con 0x55e06f7da940 2024-09-11T23:20:25.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.350+0000 7f30dcac6740 10 client.0 _ll_register_callbacks cb 0x7f30cc02f800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:20:25.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.350+0000 7f30d8c8a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e06f7cf760 0x55e06f7e29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:25.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.350+0000 7f30d8c8a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e06f7cf760 0x55e06f7e29d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34702/0 (socket says 172.21.15.26:34702) 2024-09-11T23:20:25.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.350+0000 7f30d8c8a640 1 -- 172.21.15.26:0/2965765331 learned_addr learned my addr 172.21.15.26:0/2965765331 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:20:25.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.350+0000 7f30d37fe640 1 --2- 172.21.15.26:0/2965765331 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e06f7d0080 0x55e06f7e2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:25.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.350+0000 7f30d8c8a640 1 --2- 172.21.15.26:0/2965765331 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e06f7da940 0x55e06f7dee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:25.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.350+0000 7f30d8c8a640 1 -- 172.21.15.26:0/2965765331 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e06f7da940 msgr2=0x55e06f7dee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:20:25.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.350+0000 7f30d8c8a640 1 --2- 172.21.15.26:0/2965765331 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e06f7da940 0x55e06f7dee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:25.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.350+0000 7f30d8c8a640 1 -- 172.21.15.26:0/2965765331 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e06f7d0080 msgr2=0x55e06f7e2f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:20:25.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.350+0000 7f30d8c8a640 1 --2- 172.21.15.26:0/2965765331 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e06f7d0080 0x55e06f7e2f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:25.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.350+0000 7f30d8c8a640 1 -- 172.21.15.26:0/2965765331 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e06f7df630 con 0x55e06f7cf760 2024-09-11T23:20:25.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.351+0000 7f30d8c8a640 1 --2- 172.21.15.26:0/2965765331 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e06f7cf760 0x55e06f7e29d0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f30cc03c5f0 tx=0x7f30cc005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=425930acd1b1a204 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:25.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.351+0000 7f30b37fe640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:20:25.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.351+0000 7f30b37fe640 1 -- 172.21.15.26:0/2965765331 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30cc043040 con 0x55e06f7cf760 2024-09-11T23:20:25.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.351+0000 7f30dcac6740 1 -- 172.21.15.26:0/2965765331 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f30c8004680 con 0x55e06f7cf760 2024-09-11T23:20:25.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.351+0000 7f30b37fe640 1 -- 172.21.15.26:0/2965765331 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f30cc003d80 con 0x55e06f7cf760 2024-09-11T23:20:25.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.351+0000 7f30b37fe640 1 -- 172.21.15.26:0/2965765331 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f30cc004330 con 0x55e06f7cf760 2024-09-11T23:20:25.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.351+0000 7f30dcac6740 10 client.16002 fetch_fsmap learned FSMap version 33 2024-09-11T23:20:25.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.351+0000 7f30dcac6740 1 -- 172.21.15.26:0/2965765331 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55e06f8560c0 con 0x55e06f7cf760 2024-09-11T23:20:25.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.351+0000 7f30b37fe640 1 -- 172.21.15.26:0/2965765331 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f30c8004680 con 0x55e06f7cf760 2024-09-11T23:20:25.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.351+0000 7f30b37fe640 1 -- 172.21.15.26:0/2965765331 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 33) ==== 8394+0+0 (secure 0 0 0) 0x7f30cc030440 con 0x55e06f7cf760 2024-09-11T23:20:25.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.352+0000 7f30dcac6740 10 client.16002 fetch_fsmap finished waiting for FSMap version 33 2024-09-11T23:20:25.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.352+0000 7f30dcac6740 10 client.16002 resolve_mds: validated gid 24485 aka daemon mds.b 2024-09-11T23:20:25.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.352+0000 7f30dcac6740 20 client.16002 populate_metadata read hostname 'smithi026' 2024-09-11T23:20:25.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.352+0000 7f30dcac6740 1 --2- 172.21.15.26:0/2965765331 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55e06f856800 0x55e06f858c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:25.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.352+0000 7f30dcac6740 4 client.16002 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 133.4, "await_for": 45.0}] 2024-09-11T23:20:25.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.352+0000 7f30dcac6740 1 -- 172.21.15.26:0/2965765331 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 133.4, "await_for": 45.0}) -- 0x55e06f73cd20 con 0x55e06f856800 2024-09-11T23:20:25.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.352+0000 7f30d37fe640 1 --2- 172.21.15.26:0/2965765331 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55e06f856800 0x55e06f858c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:25.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.352+0000 7f30d37fe640 1 --2- 172.21.15.26:0/2965765331 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55e06f856800 0x55e06f858c20 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cada660065571e36 server_cookie=5ff77ebf63680ae4 in_seq=0 out_seq=0 2024-09-11T23:20:25.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:25.352+0000 7f30b37fe640 10 client.16002 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:20:25.501 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:25 smithi026 ceph-mon[31172]: pgmap v64: 97 pgs: 97 active+clean; 153 MiB data, 904 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 8.5 MiB/s wr, 538 op/s 2024-09-11T23:20:25.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:25 smithi136 ceph-mon[35121]: pgmap v64: 97 pgs: 97 active+clean; 153 MiB data, 904 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 8.5 MiB/s wr, 538 op/s 2024-09-11T23:20:25.802 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:25 smithi076 ceph-mon[35296]: pgmap v64: 97 pgs: 97 active+clean; 153 MiB data, 904 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 8.5 MiB/s wr, 538 op/s 2024-09-11T23:20:27.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:27 smithi136 ceph-mon[35121]: pgmap v65: 97 pgs: 97 active+clean; 153 MiB data, 904 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 7.6 MiB/s wr, 497 op/s 2024-09-11T23:20:27.750 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:27 smithi026 ceph-mon[31172]: pgmap v65: 97 pgs: 97 active+clean; 153 MiB data, 904 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 7.6 MiB/s wr, 497 op/s 2024-09-11T23:20:27.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:27 smithi076 ceph-mon[35296]: pgmap v65: 97 pgs: 97 active+clean; 153 MiB data, 904 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 7.6 MiB/s wr, 497 op/s 2024-09-11T23:20:29.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:29 smithi136 ceph-mon[35121]: pgmap v66: 97 pgs: 97 active+clean; 223 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 14 MiB/s wr, 776 op/s 2024-09-11T23:20:29.750 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:29 smithi026 ceph-mon[31172]: pgmap v66: 97 pgs: 97 active+clean; 223 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 14 MiB/s wr, 776 op/s 2024-09-11T23:20:29.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:29 smithi076 ceph-mon[35296]: pgmap v66: 97 pgs: 97 active+clean; 223 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 14 MiB/s wr, 776 op/s 2024-09-11T23:20:30.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:30.157+0000 7f30b37fe640 1 -- 172.21.15.26:0/2965765331 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x55e06f73cd20 con 0x55e06f856800 2024-09-11T23:20:30.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:30.157+0000 7f30b37fe640 10 client.16002 handle_command_reply: tid=0 2024-09-11T23:20:30.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:30.157+0000 7f30dcac6740 1 client.16002 shutdown 2024-09-11T23:20:30.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:30.157+0000 7f30dcac6740 1 -- 172.21.15.26:0/2965765331 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e06f7cf760 msgr2=0x55e06f7e29d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:30.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:30.157+0000 7f30dcac6740 1 --2- 172.21.15.26:0/2965765331 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e06f7cf760 0x55e06f7e29d0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f30cc03c5f0 tx=0x7f30cc005c50 comp rx=0 tx=0).stop 2024-09-11T23:20:30.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:30.157+0000 7f30dcac6740 1 -- 172.21.15.26:0/2965765331 shutdown_connections 2024-09-11T23:20:30.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:30.157+0000 7f30dcac6740 1 --2- 172.21.15.26:0/2965765331 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e06f7d0080 0x55e06f7e2f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:30.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:30.157+0000 7f30dcac6740 1 --2- 172.21.15.26:0/2965765331 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55e06f856800 0x55e06f858c20 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:30.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:30.157+0000 7f30dcac6740 1 --2- 172.21.15.26:0/2965765331 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e06f7da940 0x55e06f7dee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:30.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:30.157+0000 7f30dcac6740 1 --2- 172.21.15.26:0/2965765331 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e06f7cf760 0x55e06f7e29d0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:30.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:30.157+0000 7f30dcac6740 1 -- 172.21.15.26:0/2965765331 >> 172.21.15.26:0/2965765331 conn(0x55e06f7cbcf0 msgr2=0x55e06f7cae70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:30.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:30.157+0000 7f30b37fe640 0 client.16002 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:20:30.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:30.157+0000 7f30dcac6740 1 -- 172.21.15.26:0/2965765331 shutdown_connections 2024-09-11T23:20:30.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:30.157+0000 7f30dcac6740 1 -- 172.21.15.26:0/2965765331 wait complete. 2024-09-11T23:20:30.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:30.157+0000 7f30dcac6740 20 client.16002 trim_cache size 0 max 16384 2024-09-11T23:20:30.160 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-11T23:20:30.160 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 33, 2024-09-11T23:20:30.160 INFO:teuthology.orchestra.run.smithi026.stdout: "leader": 24485, 2024-09-11T23:20:30.160 INFO:teuthology.orchestra.run.smithi026.stdout: "set_version": 6, 2024-09-11T23:20:30.160 INFO:teuthology.orchestra.run.smithi026.stdout: "sets": { 2024-09-11T23:20:30.160 INFO:teuthology.orchestra.run.smithi026.stdout: "a6703ea8": { 2024-09-11T23:20:30.160 INFO:teuthology.orchestra.run.smithi026.stdout: "version": 6, 2024-09-11T23:20:30.160 INFO:teuthology.orchestra.run.smithi026.stdout: "age_ref": 0.0, 2024-09-11T23:20:30.160 INFO:teuthology.orchestra.run.smithi026.stdout: "state": { 2024-09-11T23:20:30.160 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "QUIESCED", 2024-09-11T23:20:30.161 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.0 2024-09-11T23:20:30.161 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:20:30.161 INFO:teuthology.orchestra.run.smithi026.stdout: "timeout": 90.0, 2024-09-11T23:20:30.161 INFO:teuthology.orchestra.run.smithi026.stdout: "expiration": 133.4, 2024-09-11T23:20:30.161 INFO:teuthology.orchestra.run.smithi026.stdout: "members": { 2024-09-11T23:20:30.161 INFO:teuthology.orchestra.run.smithi026.stdout: "file:/": { 2024-09-11T23:20:30.161 INFO:teuthology.orchestra.run.smithi026.stdout: "excluded": false, 2024-09-11T23:20:30.161 INFO:teuthology.orchestra.run.smithi026.stdout: "state": { 2024-09-11T23:20:30.161 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "QUIESCED", 2024-09-11T23:20:30.161 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.0 2024-09-11T23:20:30.161 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:20:30.161 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:20:30.161 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:20:30.161 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:20:30.162 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:20:30.162 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-11T23:20:30.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:30.158+0000 7f30da6ed640 1 -- 172.21.15.26:0/292909952 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f30bc0a2d70 msgr2=0x7f30bc0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:30.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:30.158+0000 7f30da6ed640 1 --2- 172.21.15.26:0/292909952 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f30bc0a2d70 0x7f30bc0a5230 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f30c4097e90 tx=0x7f30c4097cd0 comp rx=0 tx=0).stop 2024-09-11T23:20:30.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:30.158+0000 7f30da6ed640 1 -- 172.21.15.26:0/292909952 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f30d4100e20 msgr2=0x7f30d40fcc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:30.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:30.158+0000 7f30da6ed640 1 --2- 172.21.15.26:0/292909952 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f30d4100e20 0x7f30d40fcc80 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f30c400c920 tx=0x7f30c400cde0 comp rx=0 tx=0).stop 2024-09-11T23:20:30.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:30.158+0000 7f30da6ed640 1 -- 172.21.15.26:0/292909952 shutdown_connections 2024-09-11T23:20:30.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:30.158+0000 7f30da6ed640 1 --2- 172.21.15.26:0/292909952 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f30bc0a2d70 0x7f30bc0a5230 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:30.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:30.158+0000 7f30da6ed640 1 --2- 172.21.15.26:0/292909952 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f30d4102570 0x7f30d40fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:30.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:30.158+0000 7f30da6ed640 1 --2- 172.21.15.26:0/292909952 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f30d4102ef0 0x7f30d40fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:30.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:30.158+0000 7f30da6ed640 1 --2- 172.21.15.26:0/292909952 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f30d4100e20 0x7f30d40fcc80 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:30.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:30.158+0000 7f30da6ed640 1 -- 172.21.15.26:0/292909952 >> 172.21.15.26:0/292909952 conn(0x7f30d40fc7d0 msgr2=0x7f30d406a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:30.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:30.158+0000 7f30da6ed640 1 -- 172.21.15.26:0/292909952 shutdown_connections 2024-09-11T23:20:30.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:30.158+0000 7f30da6ed640 1 -- 172.21.15.26:0/292909952 wait complete. 2024-09-11T23:20:30.632 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.631+0000 7f8f02442640 1 -- 172.21.15.26:0/1299473356 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8efc1009f0 msgr2=0x7f8efc100e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:30.632 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.631+0000 7f8f02442640 1 --2- 172.21.15.26:0/1299473356 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8efc1009f0 0x7f8efc100e50 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f8ef0009930 tx=0x7f8ef002a420 comp rx=0 tx=0).stop 2024-09-11T23:20:30.632 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.631+0000 7f8f02442640 1 -- 172.21.15.26:0/1299473356 shutdown_connections 2024-09-11T23:20:30.632 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.631+0000 7f8f02442640 1 --2- 172.21.15.26:0/1299473356 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8efc101390 0x7f8efc10d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:30.632 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.631+0000 7f8f02442640 1 --2- 172.21.15.26:0/1299473356 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8efc1009f0 0x7f8efc100e50 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:30.633 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.631+0000 7f8f02442640 1 --2- 172.21.15.26:0/1299473356 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8efc106a00 0x7f8efc106de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:30.633 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.631+0000 7f8f02442640 1 -- 172.21.15.26:0/1299473356 >> 172.21.15.26:0/1299473356 conn(0x7f8efc0fc6b0 msgr2=0x7f8efc0fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:30.633 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.631+0000 7f8f02442640 1 -- 172.21.15.26:0/1299473356 shutdown_connections 2024-09-11T23:20:30.633 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.631+0000 7f8f02442640 1 -- 172.21.15.26:0/1299473356 wait complete. 2024-09-11T23:20:30.635 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.632+0000 7f8f02442640 1 Processor -- start 2024-09-11T23:20:30.635 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.632+0000 7f8f02442640 1 -- start start 2024-09-11T23:20:30.635 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.632+0000 7f8f02442640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8efc1009f0 0x7f8efc1972f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:30.635 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.632+0000 7f8f02442640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8efc101390 0x7f8efc197830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:30.635 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.632+0000 7f8f02442640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8efc106a00 0x7f8efc19bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:30.635 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.632+0000 7f8f02442640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f8efc110e40 con 0x7f8efc101390 2024-09-11T23:20:30.635 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.632+0000 7f8f02442640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f8efc110cc0 con 0x7f8efc106a00 2024-09-11T23:20:30.635 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.632+0000 7f8f02442640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f8efc110fc0 con 0x7f8efc1009f0 2024-09-11T23:20:30.635 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.632+0000 7f8efb7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8efc101390 0x7f8efc197830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:30.635 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.632+0000 7f8efb7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8efc101390 0x7f8efc197830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33882/0 (socket says 172.21.15.26:33882) 2024-09-11T23:20:30.636 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.632+0000 7f8efbfff640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8efc1009f0 0x7f8efc1972f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:30.636 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.632+0000 7f8efb7fe640 1 -- 172.21.15.26:0/2718213092 learned_addr learned my addr 172.21.15.26:0/2718213092 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:20:30.636 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.633+0000 7f8f009df640 1 --2- 172.21.15.26:0/2718213092 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8efc106a00 0x7f8efc19bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:30.636 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.633+0000 7f8efb7fe640 1 -- 172.21.15.26:0/2718213092 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8efc1009f0 msgr2=0x7f8efc1972f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:30.636 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.633+0000 7f8efb7fe640 1 --2- 172.21.15.26:0/2718213092 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8efc1009f0 0x7f8efc1972f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:30.636 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.633+0000 7f8efb7fe640 1 -- 172.21.15.26:0/2718213092 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8efc106a00 msgr2=0x7f8efc19bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:30.636 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.633+0000 7f8efb7fe640 1 --2- 172.21.15.26:0/2718213092 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8efc106a00 0x7f8efc19bbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:30.636 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.633+0000 7f8efb7fe640 1 -- 172.21.15.26:0/2718213092 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8efc19c2e0 con 0x7f8efc101390 2024-09-11T23:20:30.636 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.633+0000 7f8efbfff640 1 --2- 172.21.15.26:0/2718213092 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8efc1009f0 0x7f8efc1972f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:20:30.636 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.633+0000 7f8efb7fe640 1 --2- 172.21.15.26:0/2718213092 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8efc101390 0x7f8efc197830 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f8ef0009230 tx=0x7f8ef0004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:30.636 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.633+0000 7f8ef97fa640 1 -- 172.21.15.26:0/2718213092 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ef003f030 con 0x7f8efc101390 2024-09-11T23:20:30.636 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.633+0000 7f8f02442640 1 -- 172.21.15.26:0/2718213092 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8efc19c570 con 0x7f8efc101390 2024-09-11T23:20:30.636 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.633+0000 7f8ef97fa640 1 -- 172.21.15.26:0/2718213092 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ef0004440 con 0x7f8efc101390 2024-09-11T23:20:30.636 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.633+0000 7f8ef97fa640 1 -- 172.21.15.26:0/2718213092 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ef0035070 con 0x7f8efc101390 2024-09-11T23:20:30.637 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.633+0000 7f8f02442640 1 -- 172.21.15.26:0/2718213092 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8efc1a3e60 con 0x7f8efc101390 2024-09-11T23:20:30.641 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.636+0000 7f8f051d3740 1 Processor -- start 2024-09-11T23:20:30.642 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.636+0000 7f8ef97fa640 1 -- 172.21.15.26:0/2718213092 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f8ef0042030 con 0x7f8efc101390 2024-09-11T23:20:30.642 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.636+0000 7f8f051d3740 1 -- start start 2024-09-11T23:20:30.642 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.636+0000 7f8f051d3740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a63b49d990 0x55a63b49dd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:30.642 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.636+0000 7f8f051d3740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a63b49e2b0 0x55a63b4aa980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:30.642 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.636+0000 7f8f051d3740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a63b4aaec0 0x55a63b4ad2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:30.642 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.636+0000 7f8f051d3740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55a63b4022f0 con 0x55a63b49d990 2024-09-11T23:20:30.642 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.636+0000 7f8f051d3740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55a63b3aa270 con 0x55a63b4aaec0 2024-09-11T23:20:30.642 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.636+0000 7f8f051d3740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55a63b3fe2b0 con 0x55a63b49e2b0 2024-09-11T23:20:30.642 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.637+0000 7f8f009df640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a63b49d990 0x55a63b49dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:30.642 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.637+0000 7f8f009df640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a63b49d990 0x55a63b49dd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33894/0 (socket says 172.21.15.26:33894) 2024-09-11T23:20:30.642 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.637+0000 7f8efbfff640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a63b49e2b0 0x55a63b4aa980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:30.642 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.637+0000 7f8f009df640 1 -- 172.21.15.26:0/2723573257 learned_addr learned my addr 172.21.15.26:0/2723573257 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:20:30.643 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.637+0000 7f8efb7fe640 1 --2- 172.21.15.26:0/2723573257 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a63b4aaec0 0x55a63b4ad2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:30.643 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.637+0000 7f8f009df640 1 -- 172.21.15.26:0/2723573257 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a63b49e2b0 msgr2=0x55a63b4aa980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:20:30.643 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.637+0000 7f8f009df640 1 --2- 172.21.15.26:0/2723573257 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a63b49e2b0 0x55a63b4aa980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:30.643 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.637+0000 7f8f009df640 1 -- 172.21.15.26:0/2723573257 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a63b4aaec0 msgr2=0x55a63b4ad2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:20:30.643 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.637+0000 7f8f009df640 1 --2- 172.21.15.26:0/2723573257 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a63b4aaec0 0x55a63b4ad2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:30.643 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.637+0000 7f8f009df640 1 -- 172.21.15.26:0/2723573257 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a63b3a9340 con 0x55a63b49d990 2024-09-11T23:20:30.643 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.637+0000 7f8efbfff640 1 --2- 172.21.15.26:0/2723573257 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a63b49e2b0 0x55a63b4aa980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:20:30.643 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.637+0000 7f8ef97fa640 1 --2- 172.21.15.26:0/2718213092 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f8ee40a2da0 0x7f8ee40a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:30.643 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.637+0000 7f8f009df640 1 --2- 172.21.15.26:0/2723573257 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a63b49d990 0x55a63b49dd70 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f8ef4009fc0 tx=0x7f8ef4010860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=348ffd7be056730f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:30.643 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.637+0000 7f8efbfff640 1 --2- 172.21.15.26:0/2718213092 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f8ee40a2da0 0x7f8ee40a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:30.643 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.637+0000 7f8ef97fa640 1 -- 172.21.15.26:0/2718213092 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f8ef00ef7d0 con 0x7f8efc101390 2024-09-11T23:20:30.643 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.637+0000 7f8ee27fc640 1 -- 172.21.15.26:0/2723573257 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ef4003b70 con 0x55a63b49d990 2024-09-11T23:20:30.643 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.637+0000 7f8ee27fc640 1 -- 172.21.15.26:0/2723573257 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ef4003d10 con 0x55a63b49d990 2024-09-11T23:20:30.643 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.637+0000 7f8ee27fc640 1 -- 172.21.15.26:0/2723573257 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ef4011a40 con 0x55a63b49d990 2024-09-11T23:20:30.644 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.638+0000 7f8f051d3740 1 -- 172.21.15.26:0/2723573257 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a63b49d990 msgr2=0x55a63b49dd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:30.644 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.638+0000 7f8efbfff640 1 --2- 172.21.15.26:0/2718213092 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f8ee40a2da0 0x7f8ee40a5260 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f8eec009fd0 tx=0x7f8eec005f90 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:30.644 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.638+0000 7f8f051d3740 1 --2- 172.21.15.26:0/2723573257 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a63b49d990 0x55a63b49dd70 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f8ef4009fc0 tx=0x7f8ef4010860 comp rx=0 tx=0).stop 2024-09-11T23:20:30.644 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.638+0000 7f8f051d3740 1 -- 172.21.15.26:0/2723573257 shutdown_connections 2024-09-11T23:20:30.644 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.638+0000 7f8f051d3740 1 --2- 172.21.15.26:0/2723573257 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a63b4aaec0 0x55a63b4ad2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:30.644 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.638+0000 7f8f051d3740 1 --2- 172.21.15.26:0/2723573257 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a63b49e2b0 0x55a63b4aa980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:30.644 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.638+0000 7f8f051d3740 1 --2- 172.21.15.26:0/2723573257 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a63b49d990 0x55a63b49dd70 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:30.644 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.638+0000 7f8f051d3740 1 -- 172.21.15.26:0/2723573257 >> 172.21.15.26:0/2723573257 conn(0x55a63b49a060 msgr2=0x55a63b49a440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:30.644 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.638+0000 7f8f051d3740 1 -- 172.21.15.26:0/2723573257 shutdown_connections 2024-09-11T23:20:30.644 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.638+0000 7f8f051d3740 1 -- 172.21.15.26:0/2723573257 wait complete. 2024-09-11T23:20:30.644 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.638+0000 7f8f051d3740 1 -- start start 2024-09-11T23:20:30.644 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.639+0000 7f8f051d3740 1 Processor -- start 2024-09-11T23:20:30.645 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.639+0000 7f8f051d3740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a63b49d990 0x55a63b4a7ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:30.645 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.639+0000 7f8f051d3740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a63b49e2b0 0x55a63b4a83e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:30.645 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.639+0000 7f8f051d3740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a63b4aaec0 0x55a63b4a8920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:30.645 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.639+0000 7f8f051d3740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55a63b4adaf0 con 0x55a63b49e2b0 2024-09-11T23:20:30.645 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.639+0000 7f8f051d3740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55a63b38b290 con 0x55a63b4aaec0 2024-09-11T23:20:30.645 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.639+0000 7f8f051d3740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55a63b4adc70 con 0x55a63b49d990 2024-09-11T23:20:30.645 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.639+0000 7f8efb7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a63b49e2b0 0x55a63b4a83e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:30.645 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.639+0000 7f8efbfff640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a63b49d990 0x55a63b4a7ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:30.645 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.639+0000 7f8f051d3740 10 client.0 _ll_register_callbacks cb 0x55a63b3bc140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:20:30.645 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.639+0000 7f8efb7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a63b49e2b0 0x55a63b4a83e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33902/0 (socket says 172.21.15.26:33902) 2024-09-11T23:20:30.645 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.639+0000 7f8efb7fe640 1 -- 172.21.15.26:0/237178993 learned_addr learned my addr 172.21.15.26:0/237178993 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:20:30.645 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.639+0000 7f8f009df640 1 --2- 172.21.15.26:0/237178993 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a63b4aaec0 0x55a63b4a8920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:30.645 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.639+0000 7f8efb7fe640 1 -- 172.21.15.26:0/237178993 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a63b49d990 msgr2=0x55a63b4a7ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:20:30.646 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.639+0000 7f8efb7fe640 1 --2- 172.21.15.26:0/237178993 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a63b49d990 0x55a63b4a7ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:30.646 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.639+0000 7f8efb7fe640 1 -- 172.21.15.26:0/237178993 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a63b4aaec0 msgr2=0x55a63b4a8920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:20:30.646 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.639+0000 7f8efb7fe640 1 --2- 172.21.15.26:0/237178993 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a63b4aaec0 0x55a63b4a8920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:30.646 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.639+0000 7f8efb7fe640 1 -- 172.21.15.26:0/237178993 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a63b115210 con 0x55a63b49e2b0 2024-09-11T23:20:30.646 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.640+0000 7f8efb7fe640 1 --2- 172.21.15.26:0/237178993 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a63b49e2b0 0x55a63b4a83e0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f8ef0009900 tx=0x7f8ef0005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51c7c1f8dc01d776 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:30.646 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.640+0000 7f8edb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:20:30.646 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.640+0000 7f8edb7fe640 1 -- 172.21.15.26:0/237178993 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ef0049070 con 0x55a63b49e2b0 2024-09-11T23:20:30.646 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.640+0000 7f8f051d3740 1 -- 172.21.15.26:0/237178993 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ef4003d10 con 0x55a63b49e2b0 2024-09-11T23:20:30.646 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.640+0000 7f8edb7fe640 1 -- 172.21.15.26:0/237178993 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f8ef00b8370 con 0x55a63b49e2b0 2024-09-11T23:20:30.646 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.640+0000 7f8edb7fe640 1 -- 172.21.15.26:0/237178993 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f8ef00bc400 con 0x55a63b49e2b0 2024-09-11T23:20:30.646 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.640+0000 7f8f051d3740 10 client.16014 fetch_fsmap learned FSMap version 33 2024-09-11T23:20:30.646 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.640+0000 7f8f051d3740 1 -- 172.21.15.26:0/237178993 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55a63b1e1620 con 0x55a63b49e2b0 2024-09-11T23:20:30.647 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.640+0000 7f8edb7fe640 1 -- 172.21.15.26:0/237178993 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ef4003d10 con 0x55a63b49e2b0 2024-09-11T23:20:30.647 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.640+0000 7f8edb7fe640 1 -- 172.21.15.26:0/237178993 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 33) ==== 8394+0+0 (secure 0 0 0) 0x7f8ef0031bd0 con 0x55a63b49e2b0 2024-09-11T23:20:30.647 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.640+0000 7f8f051d3740 10 client.16014 fetch_fsmap finished waiting for FSMap version 33 2024-09-11T23:20:30.647 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.640+0000 7f8f051d3740 10 client.16014 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:20:30.647 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.640+0000 7f8f051d3740 20 client.16014 populate_metadata read hostname 'smithi026' 2024-09-11T23:20:30.647 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.640+0000 7f8f051d3740 1 --2- 172.21.15.26:0/237178993 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55a63b4a1b90 0x55a63b4a1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:30.647 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.640+0000 7f8f051d3740 4 client.16014 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:20:30.647 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.640+0000 7f8f051d3740 1 -- 172.21.15.26:0/237178993 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a63b40fa20 con 0x55a63b4a1b90 2024-09-11T23:20:30.647 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.641+0000 7f8efbfff640 1 --2- 172.21.15.26:0/237178993 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55a63b4a1b90 0x55a63b4a1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:30.648 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.641+0000 7f8efbfff640 1 --2- 172.21.15.26:0/237178993 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55a63b4a1b90 0x55a63b4a1fa0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=723c58b666d8c5ad server_cookie=16ca488519955e79 in_seq=0 out_seq=0 2024-09-11T23:20:30.649 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.641+0000 7f8edb7fe640 10 client.16014 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:20:30.649 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.642+0000 7f8edb7fe640 1 -- 172.21.15.26:0/237178993 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a63b40fa20 con 0x55a63b4a1b90 2024-09-11T23:20:30.649 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.642+0000 7f8edb7fe640 10 client.16014 handle_command_reply: tid=0 2024-09-11T23:20:30.649 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.642+0000 7f8f051d3740 1 client.16014 shutdown 2024-09-11T23:20:30.649 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.642+0000 7f8f051d3740 1 -- 172.21.15.26:0/237178993 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a63b49e2b0 msgr2=0x55a63b4a83e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:30.649 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.642+0000 7f8f051d3740 1 --2- 172.21.15.26:0/237178993 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a63b49e2b0 0x55a63b4a83e0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f8ef0009900 tx=0x7f8ef0005c00 comp rx=0 tx=0).stop 2024-09-11T23:20:30.649 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.642+0000 7f8f051d3740 1 -- 172.21.15.26:0/237178993 shutdown_connections 2024-09-11T23:20:30.649 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.642+0000 7f8f051d3740 1 --2- 172.21.15.26:0/237178993 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a63b4aaec0 0x55a63b4a8920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:30.649 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.642+0000 7f8f051d3740 1 --2- 172.21.15.26:0/237178993 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55a63b4a1b90 0x55a63b4a1fa0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:30.649 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.642+0000 7f8f051d3740 1 --2- 172.21.15.26:0/237178993 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a63b49e2b0 0x55a63b4a83e0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:30.649 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.642+0000 7f8edb7fe640 0 client.16014 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:20:30.650 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.642+0000 7f8f051d3740 1 --2- 172.21.15.26:0/237178993 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a63b49d990 0x55a63b4a7ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:30.650 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.642+0000 7f8f051d3740 1 -- 172.21.15.26:0/237178993 >> 172.21.15.26:0/237178993 conn(0x55a63b49a060 msgr2=0x55a63b49fdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:30.650 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.642+0000 7f8f051d3740 1 -- 172.21.15.26:0/237178993 shutdown_connections 2024-09-11T23:20:30.650 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.642+0000 7f8f051d3740 1 -- 172.21.15.26:0/237178993 wait complete. 2024-09-11T23:20:30.650 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.642+0000 7f8f051d3740 20 client.16014 trim_cache size 0 max 16384 2024-09-11T23:20:30.658 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.656+0000 7f8f051d3740 1 Processor -- start 2024-09-11T23:20:30.658 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.657+0000 7f8f051d3740 1 -- start start 2024-09-11T23:20:30.658 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.657+0000 7f8f051d3740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a63b49d990 0x55a63b4beb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:30.659 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.657+0000 7f8f051d3740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a63b49e2b0 0x55a63b4bf080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:30.659 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.657+0000 7f8f051d3740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a63b4aaec0 0x55a63b4a5100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:30.659 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.657+0000 7f8f051d3740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f8efc1ab620 con 0x55a63b49e2b0 2024-09-11T23:20:30.659 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.657+0000 7f8f051d3740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f8efc1ad2e0 con 0x55a63b49d990 2024-09-11T23:20:30.659 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.657+0000 7f8f051d3740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f8efc1abcf0 con 0x55a63b4aaec0 2024-09-11T23:20:30.659 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.657+0000 7f8efb7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a63b49e2b0 0x55a63b4bf080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:30.659 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.657+0000 7f8efb7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a63b49e2b0 0x55a63b4bf080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33918/0 (socket says 172.21.15.26:33918) 2024-09-11T23:20:30.659 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.657+0000 7f8efb7fe640 1 -- 172.21.15.26:0/1290185302 learned_addr learned my addr 172.21.15.26:0/1290185302 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:20:30.659 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.657+0000 7f8f009df640 1 --2- 172.21.15.26:0/1290185302 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a63b4aaec0 0x55a63b4a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:30.659 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.657+0000 7f8efbfff640 1 --2- 172.21.15.26:0/1290185302 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a63b49d990 0x55a63b4beb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:30.659 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.657+0000 7f8efb7fe640 1 -- 172.21.15.26:0/1290185302 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a63b4aaec0 msgr2=0x55a63b4a5100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:20:30.659 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.657+0000 7f8efb7fe640 1 --2- 172.21.15.26:0/1290185302 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a63b4aaec0 0x55a63b4a5100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:30.659 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.657+0000 7f8efb7fe640 1 -- 172.21.15.26:0/1290185302 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a63b49d990 msgr2=0x55a63b4beb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:20:30.660 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.657+0000 7f8efb7fe640 1 --2- 172.21.15.26:0/1290185302 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a63b49d990 0x55a63b4beb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:30.660 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.657+0000 7f8efb7fe640 1 -- 172.21.15.26:0/1290185302 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a63b4a2310 con 0x55a63b49e2b0 2024-09-11T23:20:30.660 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.657+0000 7f8efb7fe640 1 --2- 172.21.15.26:0/1290185302 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a63b49e2b0 0x55a63b4bf080 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f8ef003b510 tx=0x7f8ef0003f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11095dabb7a42933 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:30.660 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.657+0000 7f8ee1ffb640 1 -- 172.21.15.26:0/1290185302 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ef0049070 con 0x55a63b49e2b0 2024-09-11T23:20:30.660 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.657+0000 7f8ee1ffb640 1 -- 172.21.15.26:0/1290185302 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ef00efad0 con 0x55a63b49e2b0 2024-09-11T23:20:30.660 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.657+0000 7f8ee1ffb640 1 -- 172.21.15.26:0/1290185302 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ef00b8370 con 0x55a63b49e2b0 2024-09-11T23:20:30.663 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.658+0000 7f8f051d3740 1 -- 172.21.15.26:0/1290185302 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a63b49e2b0 msgr2=0x55a63b4bf080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:30.663 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.658+0000 7f8f051d3740 1 --2- 172.21.15.26:0/1290185302 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a63b49e2b0 0x55a63b4bf080 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f8ef003b510 tx=0x7f8ef0003f30 comp rx=0 tx=0).stop 2024-09-11T23:20:30.663 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.658+0000 7f8f051d3740 1 -- 172.21.15.26:0/1290185302 shutdown_connections 2024-09-11T23:20:30.664 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.658+0000 7f8f051d3740 1 --2- 172.21.15.26:0/1290185302 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a63b4aaec0 0x55a63b4a5100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:30.664 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.658+0000 7f8f051d3740 1 --2- 172.21.15.26:0/1290185302 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a63b49e2b0 0x55a63b4bf080 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:30.664 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.658+0000 7f8f051d3740 1 --2- 172.21.15.26:0/1290185302 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a63b49d990 0x55a63b4beb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:30.664 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.658+0000 7f8f051d3740 1 -- 172.21.15.26:0/1290185302 >> 172.21.15.26:0/1290185302 conn(0x55a63b49a060 msgr2=0x55a63b4a9f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:30.664 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.658+0000 7f8f051d3740 1 -- 172.21.15.26:0/1290185302 shutdown_connections 2024-09-11T23:20:30.664 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.658+0000 7f8f051d3740 1 -- 172.21.15.26:0/1290185302 wait complete. 2024-09-11T23:20:30.664 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.658+0000 7f8f051d3740 1 -- start start 2024-09-11T23:20:30.664 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.659+0000 7f8f051d3740 1 Processor -- start 2024-09-11T23:20:30.664 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.659+0000 7f8f051d3740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a63b49d990 0x55a63b4bf310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:30.664 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.659+0000 7f8f051d3740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a63b49e2b0 0x55a63b4bc650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:30.664 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.659+0000 7f8f051d3740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a63b4aaec0 0x55a63b4bcbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:30.664 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.659+0000 7f8f051d3740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f8efc1acb50 con 0x55a63b49e2b0 2024-09-11T23:20:30.664 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.659+0000 7f8f051d3740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f8efc1ac3c0 con 0x55a63b49d990 2024-09-11T23:20:30.665 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.659+0000 7f8f051d3740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f8efc1a9600 con 0x55a63b4aaec0 2024-09-11T23:20:30.665 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.659+0000 7f8f051d3740 10 client.0 _ll_register_callbacks cb 0x7f8ef003b460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:20:30.665 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.659+0000 7f8efb7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a63b49e2b0 0x55a63b4bc650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:30.665 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.659+0000 7f8efb7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a63b49e2b0 0x55a63b4bc650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33924/0 (socket says 172.21.15.26:33924) 2024-09-11T23:20:30.665 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.659+0000 7f8efbfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a63b49d990 0x55a63b4bf310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:30.665 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.659+0000 7f8efb7fe640 1 -- 172.21.15.26:0/3591648452 learned_addr learned my addr 172.21.15.26:0/3591648452 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:20:30.665 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.659+0000 7f8f009df640 1 --2- 172.21.15.26:0/3591648452 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a63b4aaec0 0x55a63b4bcbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:30.665 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.659+0000 7f8efb7fe640 1 -- 172.21.15.26:0/3591648452 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a63b4aaec0 msgr2=0x55a63b4bcbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:20:30.665 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.659+0000 7f8efb7fe640 1 --2- 172.21.15.26:0/3591648452 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a63b4aaec0 0x55a63b4bcbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:30.665 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.659+0000 7f8efb7fe640 1 -- 172.21.15.26:0/3591648452 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a63b49d990 msgr2=0x55a63b4bf310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:20:30.665 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.659+0000 7f8efb7fe640 1 --2- 172.21.15.26:0/3591648452 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a63b49d990 0x55a63b4bf310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:30.665 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.659+0000 7f8efb7fe640 1 -- 172.21.15.26:0/3591648452 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a63b4bd3e0 con 0x55a63b49e2b0 2024-09-11T23:20:30.665 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.659+0000 7f8efbfff640 1 --2- 172.21.15.26:0/3591648452 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a63b49d990 0x55a63b4bf310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:20:30.666 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.659+0000 7f8efb7fe640 1 --2- 172.21.15.26:0/3591648452 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a63b49e2b0 0x55a63b4bc650 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f8ef00b86d0 tx=0x7f8ef00b8620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb64665e2b6b124a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:30.666 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.659+0000 7f8edb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:20:30.666 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.660+0000 7f8edb7fe640 1 -- 172.21.15.26:0/3591648452 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ef009e020 con 0x55a63b49e2b0 2024-09-11T23:20:30.666 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.660+0000 7f8edb7fe640 1 -- 172.21.15.26:0/3591648452 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f8ef00bb150 con 0x55a63b49e2b0 2024-09-11T23:20:30.666 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.660+0000 7f8f051d3740 1 -- 172.21.15.26:0/3591648452 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ef00efad0 con 0x55a63b49e2b0 2024-09-11T23:20:30.666 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.660+0000 7f8edb7fe640 1 -- 172.21.15.26:0/3591648452 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f8ef00c9050 con 0x55a63b49e2b0 2024-09-11T23:20:30.666 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.660+0000 7f8f051d3740 10 client.16020 fetch_fsmap learned FSMap version 33 2024-09-11T23:20:30.666 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.660+0000 7f8f051d3740 1 -- 172.21.15.26:0/3591648452 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55a63b4bd3e0 con 0x55a63b49e2b0 2024-09-11T23:20:30.666 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.660+0000 7f8edb7fe640 1 -- 172.21.15.26:0/3591648452 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ef00efad0 con 0x55a63b49e2b0 2024-09-11T23:20:30.666 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.660+0000 7f8edb7fe640 1 -- 172.21.15.26:0/3591648452 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 33) ==== 8394+0+0 (secure 0 0 0) 0x7f8ef00c4c50 con 0x55a63b49e2b0 2024-09-11T23:20:30.666 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.660+0000 7f8f051d3740 10 client.16020 fetch_fsmap finished waiting for FSMap version 33 2024-09-11T23:20:30.666 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.660+0000 7f8f051d3740 10 client.16020 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:20:30.666 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.660+0000 7f8f051d3740 20 client.16020 populate_metadata read hostname 'smithi026' 2024-09-11T23:20:30.666 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.660+0000 7f8f051d3740 1 --2- 172.21.15.26:0/3591648452 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55a63b524050 0x55a63b526470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:30.667 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.660+0000 7f8f051d3740 4 client.16020 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T23:20:30.667 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.660+0000 7f8f051d3740 1 -- 172.21.15.26:0/3591648452 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a63b409d70 con 0x55a63b524050 2024-09-11T23:20:30.667 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.660+0000 7f8efbfff640 1 --2- 172.21.15.26:0/3591648452 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55a63b524050 0x55a63b526470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:30.667 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.661+0000 7f8edb7fe640 10 client.16020 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:20:30.667 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:30.661+0000 7f8efbfff640 1 --2- 172.21.15.26:0/3591648452 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55a63b524050 0x55a63b526470 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2549c3c506810f7e server_cookie=32d81586457ac3d5 in_seq=0 out_seq=0 2024-09-11T23:20:30.934 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'a6703ea8', quiesce took 8.9 seconds. Will release after: 4.5 2024-09-11T23:20:31.356 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:31.355+0000 7f8edb7fe640 1 -- 172.21.15.26:0/3591648452 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a63b409d70 con 0x55a63b524050 2024-09-11T23:20:31.357 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:31.355+0000 7f8edb7fe640 10 client.16020 handle_command_reply: tid=0 2024-09-11T23:20:31.357 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:31.355+0000 7f8f051d3740 1 client.16020 shutdown 2024-09-11T23:20:31.357 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:31.355+0000 7f8f051d3740 1 -- 172.21.15.26:0/3591648452 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a63b49e2b0 msgr2=0x55a63b4bc650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:31.357 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:31.355+0000 7f8f051d3740 1 --2- 172.21.15.26:0/3591648452 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a63b49e2b0 0x55a63b4bc650 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f8ef00b86d0 tx=0x7f8ef00b8620 comp rx=0 tx=0).stop 2024-09-11T23:20:31.357 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:31.355+0000 7f8f051d3740 1 -- 172.21.15.26:0/3591648452 shutdown_connections 2024-09-11T23:20:31.357 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:31.355+0000 7f8f051d3740 1 --2- 172.21.15.26:0/3591648452 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55a63b524050 0x55a63b526470 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:31.357 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:31.355+0000 7f8f051d3740 1 --2- 172.21.15.26:0/3591648452 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a63b4aaec0 0x55a63b4bcbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:31.357 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:31.355+0000 7f8edb7fe640 0 client.16020 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:20:31.357 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:31.355+0000 7f8f051d3740 1 --2- 172.21.15.26:0/3591648452 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a63b49e2b0 0x55a63b4bc650 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:31.357 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:31.355+0000 7f8f051d3740 1 --2- 172.21.15.26:0/3591648452 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a63b49d990 0x55a63b4bf310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:31.357 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:31.355+0000 7f8f051d3740 1 -- 172.21.15.26:0/3591648452 >> 172.21.15.26:0/3591648452 conn(0x55a63b49a060 msgr2=0x55a63b4b66e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:31.357 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:31.355+0000 7f8f051d3740 1 -- 172.21.15.26:0/3591648452 shutdown_connections 2024-09-11T23:20:31.357 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:31.355+0000 7f8f051d3740 1 -- 172.21.15.26:0/3591648452 wait complete. 2024-09-11T23:20:31.358 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:31.355+0000 7f8f051d3740 20 client.16020 trim_cache size 0 max 16384 2024-09-11T23:20:31.358 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-09-11T23:20:31.358 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "message": "", 2024-09-11T23:20:31.358 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-09-11T23:20:31.358 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-09-11T23:20:31.359 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:31.356+0000 7f8f02442640 1 -- 172.21.15.26:0/2718213092 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f8ee40a2da0 msgr2=0x7f8ee40a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:31.359 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:31.356+0000 7f8f02442640 1 --2- 172.21.15.26:0/2718213092 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f8ee40a2da0 0x7f8ee40a5260 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f8eec009fd0 tx=0x7f8eec005f90 comp rx=0 tx=0).stop 2024-09-11T23:20:31.359 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:31.356+0000 7f8f02442640 1 -- 172.21.15.26:0/2718213092 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8efc101390 msgr2=0x7f8efc197830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:31.359 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:31.356+0000 7f8f02442640 1 --2- 172.21.15.26:0/2718213092 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8efc101390 0x7f8efc197830 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f8ef0009230 tx=0x7f8ef0004040 comp rx=0 tx=0).stop 2024-09-11T23:20:31.359 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:31.356+0000 7f8f02442640 1 -- 172.21.15.26:0/2718213092 shutdown_connections 2024-09-11T23:20:31.359 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:31.356+0000 7f8f02442640 1 --2- 172.21.15.26:0/2718213092 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f8ee40a2da0 0x7f8ee40a5260 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:31.359 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:31.356+0000 7f8f02442640 1 --2- 172.21.15.26:0/2718213092 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8efc106a00 0x7f8efc19bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:31.359 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:31.356+0000 7f8f02442640 1 --2- 172.21.15.26:0/2718213092 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8efc101390 0x7f8efc197830 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:31.360 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:31.356+0000 7f8f02442640 1 --2- 172.21.15.26:0/2718213092 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8efc1009f0 0x7f8efc1972f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:31.360 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:31.356+0000 7f8f02442640 1 -- 172.21.15.26:0/2718213092 >> 172.21.15.26:0/2718213092 conn(0x7f8efc0fc6b0 msgr2=0x7f8efc0fce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:31.360 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:31.356+0000 7f8f02442640 1 -- 172.21.15.26:0/2718213092 shutdown_connections 2024-09-11T23:20:31.360 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:31.356+0000 7f8f02442640 1 -- 172.21.15.26:0/2718213092 wait complete. 2024-09-11T23:20:31.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:31 smithi136 ceph-mon[35121]: pgmap v67: 97 pgs: 97 active+clean; 223 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 13 MiB/s wr, 615 op/s 2024-09-11T23:20:31.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:31 smithi136 ceph-mon[35121]: scrub summary: idle 2024-09-11T23:20:31.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:31 smithi136 ceph-mon[35121]: scrub summary: idle 2024-09-11T23:20:31.750 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:31 smithi026 ceph-mon[31172]: pgmap v67: 97 pgs: 97 active+clean; 223 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 13 MiB/s wr, 615 op/s 2024-09-11T23:20:31.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:31 smithi026 ceph-mon[31172]: scrub summary: idle 2024-09-11T23:20:31.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:31 smithi026 ceph-mon[31172]: scrub summary: idle 2024-09-11T23:20:31.802 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:31 smithi076 ceph-mon[35296]: pgmap v67: 97 pgs: 97 active+clean; 223 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 13 MiB/s wr, 615 op/s 2024-09-11T23:20:31.802 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:31 smithi076 ceph-mon[35296]: scrub summary: idle 2024-09-11T23:20:31.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:31 smithi076 ceph-mon[35296]: scrub summary: idle 2024-09-11T23:20:32.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:32 smithi136 ceph-mon[35121]: scrub summary: idle 2024-09-11T23:20:32.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:32 smithi136 ceph-mon[35121]: scrub summary: idle 2024-09-11T23:20:32.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:32 smithi136 ceph-mon[35121]: scrub summary: idle 2024-09-11T23:20:32.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:32 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:20:32.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:32 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:20:32.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:32 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:20:32.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:32 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:20:32.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:32 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:20:32.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:32 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:20:32.740 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:32 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:20:32.741 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:32 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:20:32.750 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:32 smithi026 ceph-mon[31172]: scrub summary: idle 2024-09-11T23:20:32.750 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:32 smithi026 ceph-mon[31172]: scrub summary: idle 2024-09-11T23:20:32.750 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:32 smithi026 ceph-mon[31172]: scrub summary: idle 2024-09-11T23:20:32.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:32 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:20:32.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:32 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:20:32.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:32 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:20:32.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:32 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:20:32.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:32 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:20:32.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:32 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:20:32.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:32 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:20:32.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:32 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:20:32.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:32 smithi076 ceph-mon[35296]: scrub summary: idle 2024-09-11T23:20:32.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:32 smithi076 ceph-mon[35296]: scrub summary: idle 2024-09-11T23:20:32.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:32 smithi076 ceph-mon[35296]: scrub summary: idle 2024-09-11T23:20:32.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:32 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:20:32.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:32 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:20:32.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:32 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:20:32.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:32 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:20:32.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:32 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:20:32.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:32 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:20:32.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:32 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:20:32.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:32 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:20:33.576 INFO:tasks.workunit.client.0.smithi026.stdout: 62 16144 20 9597 19 2411 25 2024-09-11T23:20:34.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:33 smithi136 ceph-mon[35121]: pgmap v68: 97 pgs: 97 active+clean; 223 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 13 MiB/s wr, 615 op/s 2024-09-11T23:20:34.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:33 smithi026 ceph-mon[31172]: pgmap v68: 97 pgs: 97 active+clean; 223 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 13 MiB/s wr, 615 op/s 2024-09-11T23:20:34.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:33 smithi076 ceph-mon[35296]: pgmap v68: 97 pgs: 97 active+clean; 223 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 13 MiB/s wr, 615 op/s 2024-09-11T23:20:35.436 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:20:35.703 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:20:36.095 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:35 smithi026 ceph-mon[31172]: pgmap v69: 97 pgs: 97 active+clean; 227 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 13 MiB/s wr, 679 op/s 2024-09-11T23:20:36.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:36.179+0000 7f7be7fa5640 1 -- 172.21.15.26:0/4113226949 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7be010c170 msgr2=0x7f7be010e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:36.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:36.179+0000 7f7be7fa5640 1 --2- 172.21.15.26:0/4113226949 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7be010c170 0x7f7be010e610 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f7bd8009ca0 tx=0x7f7bd802a950 comp rx=0 tx=0).stop 2024-09-11T23:20:36.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:36.179+0000 7f7be7fa5640 1 -- 172.21.15.26:0/4113226949 shutdown_connections 2024-09-11T23:20:36.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:36.179+0000 7f7be7fa5640 1 --2- 172.21.15.26:0/4113226949 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7be0103660 0x7f7be010bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:36.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:36.179+0000 7f7be7fa5640 1 --2- 172.21.15.26:0/4113226949 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7be010c170 0x7f7be010e610 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:36.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:36.179+0000 7f7be7fa5640 1 --2- 172.21.15.26:0/4113226949 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7be0102d40 0x7f7be0103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:36.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:36.179+0000 7f7be7fa5640 1 -- 172.21.15.26:0/4113226949 >> 172.21.15.26:0/4113226949 conn(0x7f7be00fc7a0 msgr2=0x7f7be00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:36.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:36.179+0000 7f7be7fa5640 1 -- 172.21.15.26:0/4113226949 shutdown_connections 2024-09-11T23:20:36.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:36.179+0000 7f7be7fa5640 1 -- 172.21.15.26:0/4113226949 wait complete. 2024-09-11T23:20:36.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:36.180+0000 7f7be7fa5640 1 Processor -- start 2024-09-11T23:20:36.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:36.180+0000 7f7be7fa5640 1 -- start start 2024-09-11T23:20:36.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:36.180+0000 7f7be7fa5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7be0102d40 0x7f7be0198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:36.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:36.180+0000 7f7be7fa5640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7be0103660 0x7f7be01988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:36.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:36.180+0000 7f7be7fa5640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7be010c170 0x7f7be019cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:36.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:36.180+0000 7f7be7fa5640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f7be0111f70 con 0x7f7be0102d40 2024-09-11T23:20:36.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:36.180+0000 7f7be7fa5640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f7be0111df0 con 0x7f7be010c170 2024-09-11T23:20:36.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:36.180+0000 7f7be7fa5640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f7be01120f0 con 0x7f7be0103660 2024-09-11T23:20:36.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:36.180+0000 7f7be5d41640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7be0102d40 0x7f7be0198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:36.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:36.180+0000 7f7be5d41640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7be0102d40 0x7f7be0198370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33956/0 (socket says 172.21.15.26:33956) 2024-09-11T23:20:36.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:36.180+0000 7f7be5d41640 1 -- 172.21.15.26:0/2308133916 learned_addr learned my addr 172.21.15.26:0/2308133916 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:20:36.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:36.180+0000 7f7be6542640 1 --2- 172.21.15.26:0/2308133916 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7be010c170 0x7f7be019cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:36.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:36.180+0000 7f7be5d41640 1 -- 172.21.15.26:0/2308133916 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7be0103660 msgr2=0x7f7be01988b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:36.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:36.180+0000 7f7be5540640 1 --2- 172.21.15.26:0/2308133916 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7be0103660 0x7f7be01988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:36.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:36.180+0000 7f7be5d41640 1 --2- 172.21.15.26:0/2308133916 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7be0103660 0x7f7be01988b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:36.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:36.180+0000 7f7be5d41640 1 -- 172.21.15.26:0/2308133916 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7be010c170 msgr2=0x7f7be019cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:36.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:36.180+0000 7f7be5d41640 1 --2- 172.21.15.26:0/2308133916 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7be010c170 0x7f7be019cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:36.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:36.180+0000 7f7be5d41640 1 -- 172.21.15.26:0/2308133916 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7be019d320 con 0x7f7be0102d40 2024-09-11T23:20:36.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:36.181+0000 7f7be6542640 1 --2- 172.21.15.26:0/2308133916 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7be010c170 0x7f7be019cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:20:36.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:36.181+0000 7f7be5540640 1 --2- 172.21.15.26:0/2308133916 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7be0103660 0x7f7be01988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:20:36.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:36.181+0000 7f7be5d41640 1 --2- 172.21.15.26:0/2308133916 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7be0102d40 0x7f7be0198370 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f7bdc00c920 tx=0x7f7bdc00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:36.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:36.181+0000 7f7bd6ffd640 1 -- 172.21.15.26:0/2308133916 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bdc007c70 con 0x7f7be0102d40 2024-09-11T23:20:36.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:36.181+0000 7f7be7fa5640 1 -- 172.21.15.26:0/2308133916 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7be019d610 con 0x7f7be0102d40 2024-09-11T23:20:36.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:36.181+0000 7f7bd6ffd640 1 -- 172.21.15.26:0/2308133916 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7bdc007e10 con 0x7f7be0102d40 2024-09-11T23:20:36.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:36.181+0000 7f7bd6ffd640 1 -- 172.21.15.26:0/2308133916 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bdc018890 con 0x7f7be0102d40 2024-09-11T23:20:36.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:36.181+0000 7f7be7fa5640 1 -- 172.21.15.26:0/2308133916 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7be01a4ef0 con 0x7f7be0102d40 2024-09-11T23:20:36.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:36.182+0000 7f7be7fa5640 1 -- 172.21.15.26:0/2308133916 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7be0103120 con 0x7f7be0102d40 2024-09-11T23:20:36.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:36.183+0000 7f7bd6ffd640 1 -- 172.21.15.26:0/2308133916 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f7bdc018a70 con 0x7f7be0102d40 2024-09-11T23:20:36.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:36.184+0000 7f7bd6ffd640 1 --2- 172.21.15.26:0/2308133916 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f7bcc0a2cf0 0x7f7bcc0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:36.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:36.184+0000 7f7be5540640 1 --2- 172.21.15.26:0/2308133916 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f7bcc0a2cf0 0x7f7bcc0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:36.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:36.184+0000 7f7bd6ffd640 1 -- 172.21.15.26:0/2308133916 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f7bdc0cf6a0 con 0x7f7be0102d40 2024-09-11T23:20:36.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:36.184+0000 7f7be5540640 1 --2- 172.21.15.26:0/2308133916 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f7bcc0a2cf0 0x7f7bcc0a51b0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f7bd0005fd0 tx=0x7f7bd0005dc0 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:36.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:36.188+0000 7f7bd6ffd640 1 -- 172.21.15.26:0/2308133916 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7bdc014030 con 0x7f7be0102d40 2024-09-11T23:20:36.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:35 smithi136 ceph-mon[35121]: pgmap v69: 97 pgs: 97 active+clean; 227 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 13 MiB/s wr, 679 op/s 2024-09-11T23:20:36.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:35 smithi076 ceph-mon[35296]: pgmap v69: 97 pgs: 97 active+clean; 227 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 13 MiB/s wr, 679 op/s 2024-09-11T23:20:36.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:36.402+0000 7f7be7fa5640 1 -- 172.21.15.26:0/2308133916 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7be01a5220 con 0x7f7be0102d40 2024-09-11T23:20:36.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:36.403+0000 7f7bd6ffd640 1 -- 172.21.15.26:0/2308133916 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+11986 (secure 0 0 0) 0x7f7bdc00de20 con 0x7f7be0102d40 2024-09-11T23:20:36.405 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-11T23:20:36.405 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":33,"btime":"2024-09-11T23:20:14:305449+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14499,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/367659515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":367659515},{"type":"v1","addr":"172.21.15.26:6835","nonce":367659515}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14511,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/2041065299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":2041065299},{"type":"v1","addr":"172.21.15.26:6837","nonce":2041065299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":33,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:16:26.544203+0000","modified":"2024-09-11T23:20:13.958301+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24485,"mds_1":24463,"mds_2":24458,"mds_3":24448,"mds_4":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14550":{"gid":14550,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6839/2233068305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":2233068305},{"type":"v1","addr":"172.21.15.26:6839","nonce":2233068305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14562":{"gid":14562,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6841/2984982336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":2984982336},{"type":"v1","addr":"172.21.15.26:6841","nonce":2984982336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6833/2019841455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":2019841455},{"type":"v1","addr":"172.21.15.136:6833","nonce":2019841455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.76:6833/3438147050","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3438147050},{"type":"v1","addr":"172.21.15.76:6833","nonce":3438147050}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6835/3617665975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3617665975},{"type":"v1","addr":"172.21.15.136:6835","nonce":3617665975}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":12,"addr":"172.21.15.76:6835/393308499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":393308499},{"type":"v1","addr":"172.21.15.76:6835","nonce":393308499}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":10,"addr":"172.21.15.136:6837/1633082387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":1633082387},{"type":"v1","addr":"172.21.15.136:6837","nonce":1633082387}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.76:6837/2078978748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":2078978748},{"type":"v1","addr":"172.21.15.76:6837","nonce":2078978748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":63,"addr":"172.21.15.136:6839/3211398630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":3211398630},{"type":"v1","addr":"172.21.15.136:6839","nonce":3211398630}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.76:6839/2077637090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6838","nonce":2077637090},{"type":"v1","addr":"172.21.15.76:6839","nonce":2077637090}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24485,"qdb_cluster":[24443,24448,24458,24463,24485]},"id":1}]} 2024-09-11T23:20:36.405 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 33 2024-09-11T23:20:36.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:36.405+0000 7f7be7fa5640 1 -- 172.21.15.26:0/2308133916 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f7bcc0a2cf0 msgr2=0x7f7bcc0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:36.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:36.405+0000 7f7be7fa5640 1 --2- 172.21.15.26:0/2308133916 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f7bcc0a2cf0 0x7f7bcc0a51b0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f7bd0005fd0 tx=0x7f7bd0005dc0 comp rx=0 tx=0).stop 2024-09-11T23:20:36.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:36.405+0000 7f7be7fa5640 1 -- 172.21.15.26:0/2308133916 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7be0102d40 msgr2=0x7f7be0198370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:36.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:36.405+0000 7f7be7fa5640 1 --2- 172.21.15.26:0/2308133916 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7be0102d40 0x7f7be0198370 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f7bdc00c920 tx=0x7f7bdc00cde0 comp rx=0 tx=0).stop 2024-09-11T23:20:36.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:36.405+0000 7f7be7fa5640 1 -- 172.21.15.26:0/2308133916 shutdown_connections 2024-09-11T23:20:36.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:36.405+0000 7f7be7fa5640 1 --2- 172.21.15.26:0/2308133916 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f7bcc0a2cf0 0x7f7bcc0a51b0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:36.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:36.405+0000 7f7be7fa5640 1 --2- 172.21.15.26:0/2308133916 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7be010c170 0x7f7be019cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:36.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:36.405+0000 7f7be7fa5640 1 --2- 172.21.15.26:0/2308133916 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7be0103660 0x7f7be01988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:36.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:36.405+0000 7f7be7fa5640 1 --2- 172.21.15.26:0/2308133916 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7be0102d40 0x7f7be0198370 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:36.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:36.405+0000 7f7be7fa5640 1 -- 172.21.15.26:0/2308133916 >> 172.21.15.26:0/2308133916 conn(0x7f7be00fc7a0 msgr2=0x7f7be00fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:36.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:36.405+0000 7f7be7fa5640 1 -- 172.21.15.26:0/2308133916 shutdown_connections 2024-09-11T23:20:36.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:36.405+0000 7f7be7fa5640 1 -- 172.21.15.26:0/2308133916 wait complete. 2024-09-11T23:20:37.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:36 smithi026 ceph-mon[31172]: from='client.? 172.21.15.26:0/2308133916' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:20:37.192 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24485 quiesce db --set-id a6703ea8 --release --await' 2024-09-11T23:20:37.193 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24485 quiesce db --set-id a6703ea8 --release --await 2024-09-11T23:20:37.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:36 smithi136 ceph-mon[35121]: from='client.? 172.21.15.26:0/2308133916' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:20:37.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:36 smithi076 ceph-mon[35296]: from='client.? 172.21.15.26:0/2308133916' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:20:37.452 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:20:37.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.908+0000 7fe87c42f640 1 -- 172.21.15.26:0/2450144630 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe874103660 msgr2=0x7fe87410bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:37.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.908+0000 7fe87c42f640 1 --2- 172.21.15.26:0/2450144630 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe874103660 0x7fe87410bb50 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fe864009900 tx=0x7fe86402a480 comp rx=0 tx=0).stop 2024-09-11T23:20:37.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.908+0000 7fe87c42f640 1 -- 172.21.15.26:0/2450144630 shutdown_connections 2024-09-11T23:20:37.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.908+0000 7fe87c42f640 1 --2- 172.21.15.26:0/2450144630 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe87410c170 0x7fe87410e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:37.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.908+0000 7fe87c42f640 1 --2- 172.21.15.26:0/2450144630 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe874103660 0x7fe87410bb50 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:37.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.908+0000 7fe87c42f640 1 --2- 172.21.15.26:0/2450144630 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe874102d40 0x7fe874103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:37.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.908+0000 7fe87c42f640 1 -- 172.21.15.26:0/2450144630 >> 172.21.15.26:0/2450144630 conn(0x7fe8740fc7a0 msgr2=0x7fe8740febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:37.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.908+0000 7fe87c42f640 1 -- 172.21.15.26:0/2450144630 shutdown_connections 2024-09-11T23:20:37.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.908+0000 7fe87c42f640 1 -- 172.21.15.26:0/2450144630 wait complete. 2024-09-11T23:20:37.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.909+0000 7fe87c42f640 1 Processor -- start 2024-09-11T23:20:37.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.909+0000 7fe87c42f640 1 -- start start 2024-09-11T23:20:37.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.909+0000 7fe87c42f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe874102d40 0x7fe874198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:37.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.909+0000 7fe87c42f640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe874103660 0x7fe874198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:37.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.909+0000 7fe87c42f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe87410c170 0x7fe87419cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:37.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.909+0000 7fe87c42f640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fe874111f70 con 0x7fe87410c170 2024-09-11T23:20:37.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.909+0000 7fe87c42f640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fe874111df0 con 0x7fe874102d40 2024-09-11T23:20:37.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.909+0000 7fe87c42f640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe8741120f0 con 0x7fe874103660 2024-09-11T23:20:37.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.909+0000 7fe87a9cc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe87410c170 0x7fe87419cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:37.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.909+0000 7fe87a9cc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe87410c170 0x7fe87419cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33966/0 (socket says 172.21.15.26:33966) 2024-09-11T23:20:37.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.909+0000 7fe87a1cb640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe874102d40 0x7fe874198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:37.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.909+0000 7fe87a9cc640 1 -- 172.21.15.26:0/2810550390 learned_addr learned my addr 172.21.15.26:0/2810550390 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:20:37.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.910+0000 7fe8799ca640 1 --2- 172.21.15.26:0/2810550390 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe874103660 0x7fe874198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:37.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.910+0000 7fe87a9cc640 1 -- 172.21.15.26:0/2810550390 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe874103660 msgr2=0x7fe874198990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:37.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.910+0000 7fe87a9cc640 1 --2- 172.21.15.26:0/2810550390 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe874103660 0x7fe874198990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:37.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.910+0000 7fe87a9cc640 1 -- 172.21.15.26:0/2810550390 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe874102d40 msgr2=0x7fe874198450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:37.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.910+0000 7fe87a9cc640 1 --2- 172.21.15.26:0/2810550390 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe874102d40 0x7fe874198450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:37.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.910+0000 7fe87a9cc640 1 -- 172.21.15.26:0/2810550390 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe87419d400 con 0x7fe87410c170 2024-09-11T23:20:37.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.910+0000 7fe87a9cc640 1 --2- 172.21.15.26:0/2810550390 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe87410c170 0x7fe87419cd20 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fe86c00eaa0 tx=0x7fe86c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:37.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.910+0000 7fe86b7fe640 1 -- 172.21.15.26:0/2810550390 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe86c00cdb0 con 0x7fe87410c170 2024-09-11T23:20:37.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.910+0000 7fe86b7fe640 1 -- 172.21.15.26:0/2810550390 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe86c005020 con 0x7fe87410c170 2024-09-11T23:20:37.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.910+0000 7fe87c42f640 1 -- 172.21.15.26:0/2810550390 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe87419d6f0 con 0x7fe87410c170 2024-09-11T23:20:37.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.910+0000 7fe86b7fe640 1 -- 172.21.15.26:0/2810550390 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe86c0116b0 con 0x7fe87410c170 2024-09-11T23:20:37.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.910+0000 7fe87c42f640 1 -- 172.21.15.26:0/2810550390 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8741a4fd0 con 0x7fe87410c170 2024-09-11T23:20:37.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.912+0000 7fe86b7fe640 1 -- 172.21.15.26:0/2810550390 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fe86c005560 con 0x7fe87410c170 2024-09-11T23:20:37.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.913+0000 7fe87e808740 1 Processor -- start 2024-09-11T23:20:37.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.913+0000 7fe86b7fe640 1 --2- 172.21.15.26:0/2810550390 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fe8600a2d70 0x7fe8600a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:37.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.913+0000 7fe87e808740 1 -- start start 2024-09-11T23:20:37.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.913+0000 7fe8799ca640 1 --2- 172.21.15.26:0/2810550390 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fe8600a2d70 0x7fe8600a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:37.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.913+0000 7fe87e808740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a5589e2060 0x55a5589e2440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:37.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.914+0000 7fe86b7fe640 1 -- 172.21.15.26:0/2810550390 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fe86c0cf5e0 con 0x7fe87410c170 2024-09-11T23:20:37.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.914+0000 7fe87e808740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a5589e2980 0x55a5589ecd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:37.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.914+0000 7fe87e808740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a5589ed240 0x55a5589ef630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:37.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.914+0000 7fe87e808740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55a558947620 con 0x55a5589e2060 2024-09-11T23:20:37.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.914+0000 7fe87e808740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55a5588eee50 con 0x55a5589ed240 2024-09-11T23:20:37.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.914+0000 7fe87e808740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55a5589435e0 con 0x55a5589e2980 2024-09-11T23:20:37.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.914+0000 7fe87a1cb640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a5589e2060 0x55a5589e2440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:37.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.914+0000 7fe87a1cb640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a5589e2060 0x55a5589e2440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33970/0 (socket says 172.21.15.26:33970) 2024-09-11T23:20:37.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.914+0000 7fe87a9cc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a5589ed240 0x55a5589ef630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:37.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.914+0000 7fe87a1cb640 1 -- 172.21.15.26:0/3304147007 learned_addr learned my addr 172.21.15.26:0/3304147007 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:20:37.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.914+0000 7fe8799ca640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a5589e2980 0x55a5589ecd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:37.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.914+0000 7fe87a1cb640 1 -- 172.21.15.26:0/3304147007 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a5589e2980 msgr2=0x55a5589ecd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:20:37.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.914+0000 7fe87a1cb640 1 --2- 172.21.15.26:0/3304147007 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a5589e2980 0x55a5589ecd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:37.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.914+0000 7fe87a1cb640 1 -- 172.21.15.26:0/3304147007 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a5589ed240 msgr2=0x55a5589ef630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:20:37.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.914+0000 7fe87a1cb640 1 --2- 172.21.15.26:0/3304147007 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a5589ed240 0x55a5589ef630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:37.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.914+0000 7fe87a1cb640 1 -- 172.21.15.26:0/3304147007 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5588dfd40 con 0x55a5589e2060 2024-09-11T23:20:37.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.914+0000 7fe8799ca640 1 --2- 172.21.15.26:0/2810550390 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fe8600a2d70 0x7fe8600a5230 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe87410d640 tx=0x7fe8640315b0 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:37.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.914+0000 7fe87a1cb640 1 --2- 172.21.15.26:0/3304147007 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a5589e2060 0x55a5589e2440 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fe87000bc90 tx=0x7fe870012b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38c76737f3763bba server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:37.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.914+0000 7fe868dea640 1 -- 172.21.15.26:0/3304147007 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe870004030 con 0x55a5589e2060 2024-09-11T23:20:37.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.915+0000 7fe868dea640 1 -- 172.21.15.26:0/3304147007 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe87000de00 con 0x55a5589e2060 2024-09-11T23:20:37.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.915+0000 7fe868dea640 1 -- 172.21.15.26:0/3304147007 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe870003c70 con 0x55a5589e2060 2024-09-11T23:20:37.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.915+0000 7fe87e808740 1 -- 172.21.15.26:0/3304147007 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a5589e2060 msgr2=0x55a5589e2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:37.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.915+0000 7fe87e808740 1 --2- 172.21.15.26:0/3304147007 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a5589e2060 0x55a5589e2440 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fe87000bc90 tx=0x7fe870012b30 comp rx=0 tx=0).stop 2024-09-11T23:20:37.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.915+0000 7fe87e808740 1 -- 172.21.15.26:0/3304147007 shutdown_connections 2024-09-11T23:20:37.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.915+0000 7fe87e808740 1 --2- 172.21.15.26:0/3304147007 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a5589ed240 0x55a5589ef630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:37.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.915+0000 7fe87e808740 1 --2- 172.21.15.26:0/3304147007 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a5589e2980 0x55a5589ecd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:37.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.915+0000 7fe87e808740 1 --2- 172.21.15.26:0/3304147007 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a5589e2060 0x55a5589e2440 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:37.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.915+0000 7fe87e808740 1 -- 172.21.15.26:0/3304147007 >> 172.21.15.26:0/3304147007 conn(0x55a5589de5f0 msgr2=0x55a5589de9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:37.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.915+0000 7fe87e808740 1 -- 172.21.15.26:0/3304147007 shutdown_connections 2024-09-11T23:20:37.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.915+0000 7fe87e808740 1 -- 172.21.15.26:0/3304147007 wait complete. 2024-09-11T23:20:37.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.916+0000 7fe87e808740 1 -- start start 2024-09-11T23:20:37.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.916+0000 7fe87e808740 1 Processor -- start 2024-09-11T23:20:37.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.916+0000 7fe87e808740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a5589e2060 0x55a5589ea040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:37.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.916+0000 7fe87e808740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a5589e2980 0x55a5589ea580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:37.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.916+0000 7fe87e808740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a5589ed240 0x55a5589eaac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:37.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.916+0000 7fe87e808740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55a5589efe10 con 0x55a5589e2060 2024-09-11T23:20:37.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.916+0000 7fe87e808740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55a5588cff10 con 0x55a5589e2980 2024-09-11T23:20:37.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.916+0000 7fe87e808740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55a5589eff90 con 0x55a5589ed240 2024-09-11T23:20:37.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.916+0000 7fe87e808740 10 client.0 _ll_register_callbacks cb 0x55a5588f3db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:20:37.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.916+0000 7fe87a1cb640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a5589e2060 0x55a5589ea040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:37.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.916+0000 7fe87a1cb640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a5589e2060 0x55a5589ea040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33978/0 (socket says 172.21.15.26:33978) 2024-09-11T23:20:37.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.916+0000 7fe87a1cb640 1 -- 172.21.15.26:0/1009123786 learned_addr learned my addr 172.21.15.26:0/1009123786 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:20:37.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.916+0000 7fe87a9cc640 1 --2- 172.21.15.26:0/1009123786 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a5589ed240 0x55a5589eaac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:37.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.916+0000 7fe8799ca640 1 --2- 172.21.15.26:0/1009123786 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a5589e2980 0x55a5589ea580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:37.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.916+0000 7fe87a1cb640 1 -- 172.21.15.26:0/1009123786 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a5589ed240 msgr2=0x55a5589eaac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:20:37.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.917+0000 7fe87a1cb640 1 --2- 172.21.15.26:0/1009123786 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a5589ed240 0x55a5589eaac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:37.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.917+0000 7fe87a1cb640 1 -- 172.21.15.26:0/1009123786 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a5589e2980 msgr2=0x55a5589ea580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:20:37.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.917+0000 7fe87a1cb640 1 --2- 172.21.15.26:0/1009123786 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a5589e2980 0x55a5589ea580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:37.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.917+0000 7fe87a1cb640 1 -- 172.21.15.26:0/1009123786 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a558656580 con 0x55a5589e2060 2024-09-11T23:20:37.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.917+0000 7fe8799ca640 1 --2- 172.21.15.26:0/1009123786 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a5589e2980 0x55a5589ea580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:20:37.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.917+0000 7fe87a1cb640 1 --2- 172.21.15.26:0/1009123786 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a5589e2060 0x55a5589ea040 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fe87000dc00 tx=0x7fe87000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d73d77ccac3ae2f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:37.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.917+0000 7fe85dffb640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:20:37.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.917+0000 7fe85dffb640 1 -- 172.21.15.26:0/1009123786 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe870029020 con 0x55a5589e2060 2024-09-11T23:20:37.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.917+0000 7fe85dffb640 1 -- 172.21.15.26:0/1009123786 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fe870024650 con 0x55a5589e2060 2024-09-11T23:20:37.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.917+0000 7fe87e808740 1 -- 172.21.15.26:0/1009123786 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe87000de00 con 0x55a5589e2060 2024-09-11T23:20:37.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.917+0000 7fe85dffb640 1 -- 172.21.15.26:0/1009123786 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fe870034050 con 0x55a5589e2060 2024-09-11T23:20:37.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.917+0000 7fe87e808740 10 client.16038 fetch_fsmap learned FSMap version 33 2024-09-11T23:20:37.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.917+0000 7fe87e808740 1 -- 172.21.15.26:0/1009123786 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55a558720960 con 0x55a5589e2060 2024-09-11T23:20:37.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.917+0000 7fe85dffb640 1 -- 172.21.15.26:0/1009123786 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe87000de00 con 0x55a5589e2060 2024-09-11T23:20:37.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.917+0000 7fe85dffb640 1 -- 172.21.15.26:0/1009123786 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 33) ==== 8394+0+0 (secure 0 0 0) 0x7fe87001b5c0 con 0x55a5589e2060 2024-09-11T23:20:37.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.918+0000 7fe87e808740 10 client.16038 fetch_fsmap finished waiting for FSMap version 33 2024-09-11T23:20:37.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.918+0000 7fe87e808740 10 client.16038 resolve_mds: validated gid 24485 aka daemon mds.b 2024-09-11T23:20:37.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.918+0000 7fe87e808740 20 client.16038 populate_metadata read hostname 'smithi026' 2024-09-11T23:20:37.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.918+0000 7fe87e808740 1 --2- 172.21.15.26:0/1009123786 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55a5589fcc70 0x55a5589ff060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:37.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.918+0000 7fe87e808740 4 client.16038 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:20:37.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.918+0000 7fe87e808740 1 -- 172.21.15.26:0/1009123786 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a558954b90 con 0x55a5589fcc70 2024-09-11T23:20:37.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.918+0000 7fe8799ca640 1 --2- 172.21.15.26:0/1009123786 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55a5589fcc70 0x55a5589ff060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:37.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.918+0000 7fe8799ca640 1 --2- 172.21.15.26:0/1009123786 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55a5589fcc70 0x55a5589ff060 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e916df4c94e472d7 server_cookie=61e8cf95ef872e63 in_seq=0 out_seq=0 2024-09-11T23:20:37.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.918+0000 7fe85dffb640 10 client.16038 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:20:37.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.919+0000 7fe85dffb640 1 -- 172.21.15.26:0/1009123786 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a558954b90 con 0x55a5589fcc70 2024-09-11T23:20:37.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.919+0000 7fe85dffb640 10 client.16038 handle_command_reply: tid=0 2024-09-11T23:20:37.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.919+0000 7fe87e808740 1 client.16038 shutdown 2024-09-11T23:20:37.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.919+0000 7fe87e808740 1 -- 172.21.15.26:0/1009123786 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a5589e2060 msgr2=0x55a5589ea040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:37.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.919+0000 7fe87e808740 1 --2- 172.21.15.26:0/1009123786 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a5589e2060 0x55a5589ea040 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fe87000dc00 tx=0x7fe87000dc30 comp rx=0 tx=0).stop 2024-09-11T23:20:37.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.919+0000 7fe87e808740 1 -- 172.21.15.26:0/1009123786 shutdown_connections 2024-09-11T23:20:37.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.919+0000 7fe87e808740 1 --2- 172.21.15.26:0/1009123786 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a5589e2980 0x55a5589ea580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:37.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.919+0000 7fe87e808740 1 --2- 172.21.15.26:0/1009123786 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55a5589fcc70 0x55a5589ff060 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:37.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.919+0000 7fe87e808740 1 --2- 172.21.15.26:0/1009123786 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a5589ed240 0x55a5589eaac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:37.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.919+0000 7fe87e808740 1 --2- 172.21.15.26:0/1009123786 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a5589e2060 0x55a5589ea040 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:37.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.919+0000 7fe85dffb640 0 client.16038 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:20:37.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.919+0000 7fe87e808740 1 -- 172.21.15.26:0/1009123786 >> 172.21.15.26:0/1009123786 conn(0x55a5589de5f0 msgr2=0x55a5589dd750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:37.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.919+0000 7fe87e808740 1 -- 172.21.15.26:0/1009123786 shutdown_connections 2024-09-11T23:20:37.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.919+0000 7fe87e808740 1 -- 172.21.15.26:0/1009123786 wait complete. 2024-09-11T23:20:37.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.919+0000 7fe87e808740 20 client.16038 trim_cache size 0 max 16384 2024-09-11T23:20:37.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.937+0000 7fe87e808740 1 Processor -- start 2024-09-11T23:20:37.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.937+0000 7fe87e808740 1 -- start start 2024-09-11T23:20:37.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.937+0000 7fe87e808740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a5589e2060 0x55a5589e92c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:37.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.937+0000 7fe87e808740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a5589e2980 0x55a5589ea710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:37.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.937+0000 7fe87e808740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a5589ed240 0x55a5589eac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:37.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.937+0000 7fe87e808740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fe8741ab7c0 con 0x55a5589e2980 2024-09-11T23:20:37.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.937+0000 7fe87e808740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fe8741ad480 con 0x55a5589e2060 2024-09-11T23:20:37.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.937+0000 7fe87e808740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe8741abe90 con 0x55a5589ed240 2024-09-11T23:20:37.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.937+0000 7fe8799ca640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a5589e2980 0x55a5589ea710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:37.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.937+0000 7fe8799ca640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a5589e2980 0x55a5589ea710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33994/0 (socket says 172.21.15.26:33994) 2024-09-11T23:20:37.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.937+0000 7fe8799ca640 1 -- 172.21.15.26:0/1882757443 learned_addr learned my addr 172.21.15.26:0/1882757443 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:20:37.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.937+0000 7fe87a1cb640 1 --2- 172.21.15.26:0/1882757443 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a5589e2060 0x55a5589e92c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:37.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.937+0000 7fe8799ca640 1 -- 172.21.15.26:0/1882757443 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a5589ed240 msgr2=0x55a5589eac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:20:37.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.937+0000 7fe8799ca640 1 --2- 172.21.15.26:0/1882757443 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a5589ed240 0x55a5589eac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:37.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.937+0000 7fe8799ca640 1 -- 172.21.15.26:0/1882757443 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a5589e2060 msgr2=0x55a5589e92c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:20:37.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.937+0000 7fe8799ca640 1 --2- 172.21.15.26:0/1882757443 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a5589e2060 0x55a5589e92c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:37.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.937+0000 7fe8799ca640 1 -- 172.21.15.26:0/1882757443 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5589e27b0 con 0x55a5589e2980 2024-09-11T23:20:37.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.937+0000 7fe87a1cb640 1 --2- 172.21.15.26:0/1882757443 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a5589e2060 0x55a5589e92c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:20:37.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.937+0000 7fe8799ca640 1 --2- 172.21.15.26:0/1882757443 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a5589e2980 0x55a5589ea710 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fe864005ab0 tx=0x7fe864046000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c121e87fadc9c9f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:37.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.938+0000 7fe85ffff640 1 -- 172.21.15.26:0/1882757443 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe864035050 con 0x55a5589e2980 2024-09-11T23:20:37.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.938+0000 7fe85ffff640 1 -- 172.21.15.26:0/1882757443 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe864005520 con 0x55a5589e2980 2024-09-11T23:20:37.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.938+0000 7fe85ffff640 1 -- 172.21.15.26:0/1882757443 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8640035f0 con 0x55a5589e2980 2024-09-11T23:20:37.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.938+0000 7fe87e808740 1 -- 172.21.15.26:0/1882757443 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a5589e2980 msgr2=0x55a5589ea710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:37.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.938+0000 7fe87e808740 1 --2- 172.21.15.26:0/1882757443 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a5589e2980 0x55a5589ea710 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fe864005ab0 tx=0x7fe864046000 comp rx=0 tx=0).stop 2024-09-11T23:20:37.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.938+0000 7fe87e808740 1 -- 172.21.15.26:0/1882757443 shutdown_connections 2024-09-11T23:20:37.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.938+0000 7fe87e808740 1 --2- 172.21.15.26:0/1882757443 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a5589ed240 0x55a5589eac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:37.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.938+0000 7fe87e808740 1 --2- 172.21.15.26:0/1882757443 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a5589e2980 0x55a5589ea710 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:37.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.938+0000 7fe87e808740 1 --2- 172.21.15.26:0/1882757443 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a5589e2060 0x55a5589e92c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:37.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.938+0000 7fe87e808740 1 -- 172.21.15.26:0/1882757443 >> 172.21.15.26:0/1882757443 conn(0x55a5589de5f0 msgr2=0x55a5589ea000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:37.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.938+0000 7fe87e808740 1 -- 172.21.15.26:0/1882757443 shutdown_connections 2024-09-11T23:20:37.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.938+0000 7fe87e808740 1 -- 172.21.15.26:0/1882757443 wait complete. 2024-09-11T23:20:37.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.939+0000 7fe87e808740 1 -- start start 2024-09-11T23:20:37.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.939+0000 7fe87e808740 1 Processor -- start 2024-09-11T23:20:37.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.939+0000 7fe87e808740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a5589e2060 0x55a5589f51e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:37.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.939+0000 7fe87e808740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a5589e2980 0x55a5589f5720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:37.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.939+0000 7fe87e808740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a5589ed240 0x55a5589f21b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:37.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.939+0000 7fe87e808740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fe8741accf0 con 0x55a5589ed240 2024-09-11T23:20:37.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.939+0000 7fe87e808740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fe8741ac560 con 0x55a5589e2980 2024-09-11T23:20:37.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.939+0000 7fe87e808740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe8741a97a0 con 0x55a5589e2060 2024-09-11T23:20:37.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.939+0000 7fe87e808740 10 client.0 _ll_register_callbacks cb 0x7fe87000bb60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:20:37.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.939+0000 7fe87a9cc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a5589ed240 0x55a5589f21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:37.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.939+0000 7fe87a9cc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a5589ed240 0x55a5589f21b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33996/0 (socket says 172.21.15.26:33996) 2024-09-11T23:20:37.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.939+0000 7fe87a9cc640 1 -- 172.21.15.26:0/46070750 learned_addr learned my addr 172.21.15.26:0/46070750 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:20:37.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.939+0000 7fe87a1cb640 1 --2- 172.21.15.26:0/46070750 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a5589e2060 0x55a5589f51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:37.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.939+0000 7fe8799ca640 1 --2- 172.21.15.26:0/46070750 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a5589e2980 0x55a5589f5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:37.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.939+0000 7fe87a9cc640 1 -- 172.21.15.26:0/46070750 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a5589e2060 msgr2=0x55a5589f51e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:20:37.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.939+0000 7fe87a9cc640 1 --2- 172.21.15.26:0/46070750 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a5589e2060 0x55a5589f51e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:37.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.939+0000 7fe87a9cc640 1 -- 172.21.15.26:0/46070750 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a5589e2980 msgr2=0x55a5589f5720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:20:37.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.939+0000 7fe87a9cc640 1 --2- 172.21.15.26:0/46070750 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a5589e2980 0x55a5589f5720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:37.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.940+0000 7fe87a9cc640 1 -- 172.21.15.26:0/46070750 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a5589f26f0 con 0x55a5589ed240 2024-09-11T23:20:37.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.940+0000 7fe8799ca640 1 --2- 172.21.15.26:0/46070750 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a5589e2980 0x55a5589f5720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:20:37.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.940+0000 7fe87a9cc640 1 --2- 172.21.15.26:0/46070750 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a5589ed240 0x55a5589f21b0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fe86c00efd0 tx=0x7fe86c09c2f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18ef262895a81c6f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:37.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.940+0000 7fe85dffb640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:20:37.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.940+0000 7fe85dffb640 1 -- 172.21.15.26:0/46070750 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe86c01f040 con 0x55a5589ed240 2024-09-11T23:20:37.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.940+0000 7fe85dffb640 1 -- 172.21.15.26:0/46070750 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fe86c098120 con 0x55a5589ed240 2024-09-11T23:20:37.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.940+0000 7fe87e808740 1 -- 172.21.15.26:0/46070750 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe864005520 con 0x55a5589ed240 2024-09-11T23:20:37.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.940+0000 7fe85dffb640 1 -- 172.21.15.26:0/46070750 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fe86c0cf110 con 0x55a5589ed240 2024-09-11T23:20:37.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.940+0000 7fe87e808740 10 client.16044 fetch_fsmap learned FSMap version 33 2024-09-11T23:20:37.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.940+0000 7fe87e808740 1 -- 172.21.15.26:0/46070750 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55a558a70650 con 0x55a5589ed240 2024-09-11T23:20:37.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.940+0000 7fe85dffb640 1 -- 172.21.15.26:0/46070750 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe86c0cf2b0 con 0x55a5589ed240 2024-09-11T23:20:37.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.940+0000 7fe85dffb640 1 -- 172.21.15.26:0/46070750 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 33) ==== 8394+0+0 (secure 0 0 0) 0x7fe86c00f040 con 0x55a5589ed240 2024-09-11T23:20:37.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.940+0000 7fe87e808740 10 client.16044 fetch_fsmap finished waiting for FSMap version 33 2024-09-11T23:20:37.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.940+0000 7fe87e808740 10 client.16044 resolve_mds: validated gid 24485 aka daemon mds.b 2024-09-11T23:20:37.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.940+0000 7fe87e808740 20 client.16044 populate_metadata read hostname 'smithi026' 2024-09-11T23:20:37.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.940+0000 7fe87e808740 1 --2- 172.21.15.26:0/46070750 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55a558a70de0 0x55a558a73200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:37.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.940+0000 7fe87e808740 4 client.16044 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "a6703ea8", "release": true, "await": true}] 2024-09-11T23:20:37.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.940+0000 7fe87e808740 1 -- 172.21.15.26:0/46070750 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "quiesce db", "set_id": "a6703ea8", "release": true, "await": true}) -- 0x7fe864005520 con 0x55a558a70de0 2024-09-11T23:20:37.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.941+0000 7fe87a1cb640 1 --2- 172.21.15.26:0/46070750 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55a558a70de0 0x55a558a73200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:37.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.941+0000 7fe87a1cb640 1 --2- 172.21.15.26:0/46070750 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55a558a70de0 0x55a558a73200 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a759684f203b4caf server_cookie=1975329954c588a3 in_seq=0 out_seq=0 2024-09-11T23:20:37.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:37.941+0000 7fe85dffb640 10 client.16044 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:20:38.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:38.222+0000 7fe85dffb640 1 -- 172.21.15.26:0/46070750 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x7fe864005520 con 0x55a558a70de0 2024-09-11T23:20:38.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:38.222+0000 7fe85dffb640 10 client.16044 handle_command_reply: tid=0 2024-09-11T23:20:38.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:38.223+0000 7fe87e808740 1 client.16044 shutdown 2024-09-11T23:20:38.224 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:37 smithi026 ceph-mon[31172]: pgmap v70: 97 pgs: 97 active+clean; 227 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 6.7 MiB/s wr, 343 op/s 2024-09-11T23:20:38.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:38.223+0000 7fe87e808740 1 -- 172.21.15.26:0/46070750 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a5589ed240 msgr2=0x55a5589f21b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:38.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:38.223+0000 7fe87e808740 1 --2- 172.21.15.26:0/46070750 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a5589ed240 0x55a5589f21b0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fe86c00efd0 tx=0x7fe86c09c2f0 comp rx=0 tx=0).stop 2024-09-11T23:20:38.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:38.223+0000 7fe87e808740 1 -- 172.21.15.26:0/46070750 shutdown_connections 2024-09-11T23:20:38.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:38.223+0000 7fe87e808740 1 --2- 172.21.15.26:0/46070750 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a5589e2980 0x55a5589f5720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:38.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:38.223+0000 7fe87e808740 1 --2- 172.21.15.26:0/46070750 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55a558a70de0 0x55a558a73200 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:38.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:38.223+0000 7fe87e808740 1 --2- 172.21.15.26:0/46070750 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a5589ed240 0x55a5589f21b0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:38.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:38.223+0000 7fe87e808740 1 --2- 172.21.15.26:0/46070750 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a5589e2060 0x55a5589f51e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:38.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:38.223+0000 7fe87e808740 1 -- 172.21.15.26:0/46070750 >> 172.21.15.26:0/46070750 conn(0x55a5589de5f0 msgr2=0x55a5589e42d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:38.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:38.223+0000 7fe85dffb640 0 client.16044 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:20:38.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:38.223+0000 7fe87e808740 1 -- 172.21.15.26:0/46070750 shutdown_connections 2024-09-11T23:20:38.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:38.223+0000 7fe87e808740 1 -- 172.21.15.26:0/46070750 wait complete. 2024-09-11T23:20:38.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:38.223+0000 7fe87e808740 20 client.16044 trim_cache size 0 max 16384 2024-09-11T23:20:38.226 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-11T23:20:38.226 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 33, 2024-09-11T23:20:38.227 INFO:teuthology.orchestra.run.smithi026.stdout: "leader": 24485, 2024-09-11T23:20:38.227 INFO:teuthology.orchestra.run.smithi026.stdout: "set_version": 8, 2024-09-11T23:20:38.227 INFO:teuthology.orchestra.run.smithi026.stdout: "sets": { 2024-09-11T23:20:38.227 INFO:teuthology.orchestra.run.smithi026.stdout: "a6703ea8": { 2024-09-11T23:20:38.227 INFO:teuthology.orchestra.run.smithi026.stdout: "version": 8, 2024-09-11T23:20:38.227 INFO:teuthology.orchestra.run.smithi026.stdout: "age_ref": 0.0, 2024-09-11T23:20:38.227 INFO:teuthology.orchestra.run.smithi026.stdout: "state": { 2024-09-11T23:20:38.227 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "RELEASED", 2024-09-11T23:20:38.227 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.0 2024-09-11T23:20:38.227 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:20:38.227 INFO:teuthology.orchestra.run.smithi026.stdout: "timeout": 90.0, 2024-09-11T23:20:38.227 INFO:teuthology.orchestra.run.smithi026.stdout: "expiration": 133.4, 2024-09-11T23:20:38.227 INFO:teuthology.orchestra.run.smithi026.stdout: "members": { 2024-09-11T23:20:38.228 INFO:teuthology.orchestra.run.smithi026.stdout: "file:/": { 2024-09-11T23:20:38.228 INFO:teuthology.orchestra.run.smithi026.stdout: "excluded": false, 2024-09-11T23:20:38.228 INFO:teuthology.orchestra.run.smithi026.stdout: "state": { 2024-09-11T23:20:38.228 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "RELEASED", 2024-09-11T23:20:38.228 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.0 2024-09-11T23:20:38.228 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:20:38.228 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:20:38.228 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:20:38.228 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:20:38.228 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:20:38.228 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-11T23:20:38.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:38.224+0000 7fe87c42f640 1 -- 172.21.15.26:0/2810550390 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fe8600a2d70 msgr2=0x7fe8600a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:38.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:38.224+0000 7fe87c42f640 1 --2- 172.21.15.26:0/2810550390 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fe8600a2d70 0x7fe8600a5230 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe87410d640 tx=0x7fe8640315b0 comp rx=0 tx=0).stop 2024-09-11T23:20:38.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:38.224+0000 7fe87c42f640 1 -- 172.21.15.26:0/2810550390 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe87410c170 msgr2=0x7fe87419cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:38.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:38.224+0000 7fe87c42f640 1 --2- 172.21.15.26:0/2810550390 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe87410c170 0x7fe87419cd20 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fe86c00eaa0 tx=0x7fe86c00ef60 comp rx=0 tx=0).stop 2024-09-11T23:20:38.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:38.224+0000 7fe87c42f640 1 -- 172.21.15.26:0/2810550390 shutdown_connections 2024-09-11T23:20:38.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:38.224+0000 7fe87c42f640 1 --2- 172.21.15.26:0/2810550390 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fe8600a2d70 0x7fe8600a5230 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:38.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:38.224+0000 7fe87c42f640 1 --2- 172.21.15.26:0/2810550390 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe87410c170 0x7fe87419cd20 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:38.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:38.224+0000 7fe87c42f640 1 --2- 172.21.15.26:0/2810550390 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe874103660 0x7fe874198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:38.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:38.224+0000 7fe87c42f640 1 --2- 172.21.15.26:0/2810550390 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe874102d40 0x7fe874198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:38.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:38.224+0000 7fe87c42f640 1 -- 172.21.15.26:0/2810550390 >> 172.21.15.26:0/2810550390 conn(0x7fe8740fc7a0 msgr2=0x7fe8740fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:38.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:38.224+0000 7fe87c42f640 1 -- 172.21.15.26:0/2810550390 shutdown_connections 2024-09-11T23:20:38.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:38.224+0000 7fe87c42f640 1 -- 172.21.15.26:0/2810550390 wait complete. 2024-09-11T23:20:38.238 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:37 smithi136 ceph-mon[35121]: pgmap v70: 97 pgs: 97 active+clean; 227 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 6.7 MiB/s wr, 343 op/s 2024-09-11T23:20:38.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:37 smithi076 ceph-mon[35296]: pgmap v70: 97 pgs: 97 active+clean; 227 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 6.7 MiB/s wr, 343 op/s 2024-09-11T23:20:39.004 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'a6703ea8', total seconds elapsed: 17.0 2024-09-11T23:20:40.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:39 smithi136 ceph-mon[35121]: pgmap v71: 97 pgs: 97 active+clean; 228 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 6.8 MiB/s wr, 389 op/s 2024-09-11T23:20:40.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:39 smithi026 ceph-mon[31172]: pgmap v71: 97 pgs: 97 active+clean; 228 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 6.8 MiB/s wr, 389 op/s 2024-09-11T23:20:40.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:39 smithi076 ceph-mon[35296]: pgmap v71: 97 pgs: 97 active+clean; 228 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 6.8 MiB/s wr, 389 op/s 2024-09-11T23:20:41.191 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T23:20:41.504 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:20:42.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.095+0000 7f7b791f9640 1 -- 172.21.15.26:0/1701534926 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b74101de0 msgr2=0x7f7b741021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:42.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.095+0000 7f7b791f9640 1 --2- 172.21.15.26:0/1701534926 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b74101de0 0x7f7b741021c0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f7b64009900 tx=0x7f7b6402a410 comp rx=0 tx=0).stop 2024-09-11T23:20:42.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.095+0000 7f7b791f9640 1 -- 172.21.15.26:0/1701534926 shutdown_connections 2024-09-11T23:20:42.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.095+0000 7f7b791f9640 1 --2- 172.21.15.26:0/1701534926 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7b74106f10 0x7f7b7410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:42.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.095+0000 7f7b791f9640 1 --2- 172.21.15.26:0/1701534926 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7b74102790 0x7f7b741067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:42.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.095+0000 7f7b791f9640 1 --2- 172.21.15.26:0/1701534926 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b74101de0 0x7f7b741021c0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:42.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.095+0000 7f7b791f9640 1 -- 172.21.15.26:0/1701534926 >> 172.21.15.26:0/1701534926 conn(0x7f7b740fd8c0 msgr2=0x7f7b740ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:42.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.095+0000 7f7b791f9640 1 -- 172.21.15.26:0/1701534926 shutdown_connections 2024-09-11T23:20:42.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.095+0000 7f7b791f9640 1 -- 172.21.15.26:0/1701534926 wait complete. 2024-09-11T23:20:42.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.096+0000 7f7b791f9640 1 Processor -- start 2024-09-11T23:20:42.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.096+0000 7f7b791f9640 1 -- start start 2024-09-11T23:20:42.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.096+0000 7f7b791f9640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7b74101de0 0x7f7b740744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:42.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.096+0000 7f7b791f9640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7b74102790 0x7f7b74074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:42.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.096+0000 7f7b791f9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b74106f10 0x7f7b74074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:42.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.096+0000 7f7b791f9640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f7b74111f70 con 0x7f7b74106f10 2024-09-11T23:20:42.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.096+0000 7f7b791f9640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f7b74111df0 con 0x7f7b74102790 2024-09-11T23:20:42.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.096+0000 7f7b791f9640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f7b741120f0 con 0x7f7b74101de0 2024-09-11T23:20:42.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.096+0000 7f7b72d9d640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7b74101de0 0x7f7b740744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:42.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.096+0000 7f7b7359e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b74106f10 0x7f7b74074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:42.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.096+0000 7f7b7259c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7b74102790 0x7f7b74074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:42.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.096+0000 7f7b72d9d640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7b74101de0 0x7f7b740744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:50516/0 (socket says 172.21.15.26:50516) 2024-09-11T23:20:42.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.096+0000 7f7b72d9d640 1 -- 172.21.15.26:0/3548184564 learned_addr learned my addr 172.21.15.26:0/3548184564 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:20:42.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.096+0000 7f7b7259c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7b74102790 0x7f7b74074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:60790/0 (socket says 172.21.15.26:60790) 2024-09-11T23:20:42.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.097+0000 7f7b72d9d640 1 -- 172.21.15.26:0/3548184564 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7b74102790 msgr2=0x7f7b74074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:42.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.097+0000 7f7b72d9d640 1 --2- 172.21.15.26:0/3548184564 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7b74102790 0x7f7b74074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:42.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.097+0000 7f7b72d9d640 1 -- 172.21.15.26:0/3548184564 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b74106f10 msgr2=0x7f7b74074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:42.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.097+0000 7f7b72d9d640 1 --2- 172.21.15.26:0/3548184564 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b74106f10 0x7f7b74074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:42.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.097+0000 7f7b72d9d640 1 -- 172.21.15.26:0/3548184564 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b741a6970 con 0x7f7b74101de0 2024-09-11T23:20:42.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.097+0000 7f7b72d9d640 1 --2- 172.21.15.26:0/3548184564 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7b74101de0 0x7f7b740744f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f7b64006010 tx=0x7f7b640042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:42.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.097+0000 7f7b5bfff640 1 -- 172.21.15.26:0/3548184564 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b6403f030 con 0x7f7b74101de0 2024-09-11T23:20:42.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.097+0000 7f7b791f9640 1 -- 172.21.15.26:0/3548184564 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b741a6c00 con 0x7f7b74101de0 2024-09-11T23:20:42.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.097+0000 7f7b5bfff640 1 -- 172.21.15.26:0/3548184564 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b64003ce0 con 0x7f7b74101de0 2024-09-11T23:20:42.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.097+0000 7f7b5bfff640 1 -- 172.21.15.26:0/3548184564 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b64035070 con 0x7f7b74101de0 2024-09-11T23:20:42.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.097+0000 7f7b791f9640 1 -- 172.21.15.26:0/3548184564 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b741a7130 con 0x7f7b74101de0 2024-09-11T23:20:42.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.100+0000 7f7b7b5d2740 1 Processor -- start 2024-09-11T23:20:42.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.100+0000 7f7b5bfff640 1 -- 172.21.15.26:0/3548184564 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f7b64042030 con 0x7f7b74101de0 2024-09-11T23:20:42.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.100+0000 7f7b7b5d2740 1 -- start start 2024-09-11T23:20:42.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.100+0000 7f7b7b5d2740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b2efc014a0 0x55b2efc01880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:42.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.100+0000 7f7b7b5d2740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b2efc01dc0 0x55b2efc0c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:42.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.100+0000 7f7b7b5d2740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b2efc0c680 0x55b2efc0ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:42.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.100+0000 7f7b7b5d2740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55b2efb672e0 con 0x55b2efc014a0 2024-09-11T23:20:42.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.100+0000 7f7b7b5d2740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55b2efb0ece0 con 0x55b2efc0c680 2024-09-11T23:20:42.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.100+0000 7f7b7b5d2740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55b2efb632a0 con 0x55b2efc01dc0 2024-09-11T23:20:42.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.101+0000 7f7b7359e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b2efc014a0 0x55b2efc01880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:42.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.101+0000 7f7b7359e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b2efc014a0 0x55b2efc01880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55638/0 (socket says 172.21.15.26:55638) 2024-09-11T23:20:42.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.101+0000 7f7b72d9d640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b2efc01dc0 0x55b2efc0c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:42.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.101+0000 7f7b7359e640 1 -- 172.21.15.26:0/857806613 learned_addr learned my addr 172.21.15.26:0/857806613 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:20:42.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.101+0000 7f7b7259c640 1 --2- 172.21.15.26:0/857806613 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b2efc0c680 0x55b2efc0ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:42.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.101+0000 7f7b7359e640 1 -- 172.21.15.26:0/857806613 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b2efc01dc0 msgr2=0x55b2efc0c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:20:42.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.101+0000 7f7b7359e640 1 --2- 172.21.15.26:0/857806613 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b2efc01dc0 0x55b2efc0c140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:42.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.101+0000 7f7b7359e640 1 -- 172.21.15.26:0/857806613 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b2efc0c680 msgr2=0x55b2efc0ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:20:42.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.101+0000 7f7b7359e640 1 --2- 172.21.15.26:0/857806613 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b2efc0c680 0x55b2efc0ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:42.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.101+0000 7f7b7359e640 1 -- 172.21.15.26:0/857806613 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b2efaf3440 con 0x55b2efc014a0 2024-09-11T23:20:42.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.101+0000 7f7b5bfff640 1 --2- 172.21.15.26:0/3548184564 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f7b5c0a2cf0 0x7f7b5c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:42.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.101+0000 7f7b7359e640 1 --2- 172.21.15.26:0/857806613 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b2efc014a0 0x55b2efc01880 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f7b6c009f70 tx=0x7f7b6c010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fdceecc8410b311 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:42.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.101+0000 7f7b72d9d640 1 --2- 172.21.15.26:0/3548184564 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f7b5c0a2cf0 0x7f7b5c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:42.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.101+0000 7f7b5bfff640 1 -- 172.21.15.26:0/3548184564 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f7b640ef970 con 0x7f7b74101de0 2024-09-11T23:20:42.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.101+0000 7f7b597fa640 1 -- 172.21.15.26:0/857806613 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b6c016070 con 0x55b2efc014a0 2024-09-11T23:20:42.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.101+0000 7f7b597fa640 1 -- 172.21.15.26:0/857806613 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b6c004d20 con 0x55b2efc014a0 2024-09-11T23:20:42.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.101+0000 7f7b597fa640 1 -- 172.21.15.26:0/857806613 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b6c012910 con 0x55b2efc014a0 2024-09-11T23:20:42.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.102+0000 7f7b7b5d2740 1 -- 172.21.15.26:0/857806613 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b2efc014a0 msgr2=0x55b2efc01880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:42.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.102+0000 7f7b72d9d640 1 --2- 172.21.15.26:0/3548184564 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f7b5c0a2cf0 0x7f7b5c0a51b0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f7b640ed180 tx=0x7f7b640f5550 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:42.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.102+0000 7f7b7b5d2740 1 --2- 172.21.15.26:0/857806613 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b2efc014a0 0x55b2efc01880 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f7b6c009f70 tx=0x7f7b6c010800 comp rx=0 tx=0).stop 2024-09-11T23:20:42.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.102+0000 7f7b7b5d2740 1 -- 172.21.15.26:0/857806613 shutdown_connections 2024-09-11T23:20:42.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.102+0000 7f7b7b5d2740 1 --2- 172.21.15.26:0/857806613 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b2efc0c680 0x55b2efc0ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:42.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.102+0000 7f7b7b5d2740 1 --2- 172.21.15.26:0/857806613 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b2efc01dc0 0x55b2efc0c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:42.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.102+0000 7f7b7b5d2740 1 --2- 172.21.15.26:0/857806613 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b2efc014a0 0x55b2efc01880 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:42.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.102+0000 7f7b7b5d2740 1 -- 172.21.15.26:0/857806613 >> 172.21.15.26:0/857806613 conn(0x55b2efbfda30 msgr2=0x55b2efbfde10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:42.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.102+0000 7f7b7b5d2740 1 -- 172.21.15.26:0/857806613 shutdown_connections 2024-09-11T23:20:42.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.102+0000 7f7b7b5d2740 1 -- 172.21.15.26:0/857806613 wait complete. 2024-09-11T23:20:42.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.103+0000 7f7b7b5d2740 1 -- start start 2024-09-11T23:20:42.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.103+0000 7f7b7b5d2740 1 Processor -- start 2024-09-11T23:20:42.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.103+0000 7f7b7b5d2740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b2efc014a0 0x55b2efc0bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:42.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.103+0000 7f7b7b5d2740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b2efc01dc0 0x55b2efc09270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:42.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.103+0000 7f7b7b5d2740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b2efc0c680 0x55b2efc097b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:42.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.103+0000 7f7b7b5d2740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55b2efc0f250 con 0x55b2efc014a0 2024-09-11T23:20:42.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.103+0000 7f7b7b5d2740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55b2efb22050 con 0x55b2efc0c680 2024-09-11T23:20:42.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.103+0000 7f7b7b5d2740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55b2efc0f3d0 con 0x55b2efc01dc0 2024-09-11T23:20:42.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.103+0000 7f7b7b5d2740 10 client.0 _ll_register_callbacks cb 0x55b2efb62160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:20:42.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.103+0000 7f7b7359e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b2efc014a0 0x55b2efc0bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:42.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.103+0000 7f7b7359e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b2efc014a0 0x55b2efc0bfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55642/0 (socket says 172.21.15.26:55642) 2024-09-11T23:20:42.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.103+0000 7f7b7259c640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b2efc01dc0 0x55b2efc09270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:42.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.103+0000 7f7b7359e640 1 -- 172.21.15.26:0/1424346973 learned_addr learned my addr 172.21.15.26:0/1424346973 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:20:42.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.103+0000 7f7b7359e640 1 --2- 172.21.15.26:0/1424346973 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b2efc0c680 0x55b2efc097b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:42.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.103+0000 7f7b7359e640 1 -- 172.21.15.26:0/1424346973 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b2efc01dc0 msgr2=0x55b2efc09270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:20:42.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.103+0000 7f7b7359e640 1 --2- 172.21.15.26:0/1424346973 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b2efc01dc0 0x55b2efc09270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:42.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.103+0000 7f7b7359e640 1 -- 172.21.15.26:0/1424346973 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b2efc0c680 msgr2=0x55b2efc097b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:20:42.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.103+0000 7f7b7359e640 1 --2- 172.21.15.26:0/1424346973 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b2efc0c680 0x55b2efc097b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:42.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.103+0000 7f7b7359e640 1 -- 172.21.15.26:0/1424346973 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b2ef8d83c0 con 0x55b2efc014a0 2024-09-11T23:20:42.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.104+0000 7f7b7359e640 1 --2- 172.21.15.26:0/1424346973 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b2efc014a0 0x55b2efc0bfc0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f7b6c006010 tx=0x7f7b6c012590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a54779a4d9eb0dc6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:42.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.104+0000 7f7b56ff5640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:20:42.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.104+0000 7f7b56ff5640 1 -- 172.21.15.26:0/1424346973 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b6c016070 con 0x55b2efc014a0 2024-09-11T23:20:42.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.104+0000 7f7b56ff5640 1 -- 172.21.15.26:0/1424346973 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f7b6c012d50 con 0x55b2efc014a0 2024-09-11T23:20:42.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.104+0000 7f7b7b5d2740 1 -- 172.21.15.26:0/1424346973 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b6c004d20 con 0x55b2efc014a0 2024-09-11T23:20:42.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.104+0000 7f7b56ff5640 1 -- 172.21.15.26:0/1424346973 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f7b6c004ed0 con 0x55b2efc014a0 2024-09-11T23:20:42.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.104+0000 7f7b7b5d2740 10 client.16056 fetch_fsmap learned FSMap version 33 2024-09-11T23:20:42.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.104+0000 7f7b7b5d2740 1 -- 172.21.15.26:0/1424346973 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55b2efc1ac00 con 0x55b2efc014a0 2024-09-11T23:20:42.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.104+0000 7f7b56ff5640 1 -- 172.21.15.26:0/1424346973 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b6c004d20 con 0x55b2efc014a0 2024-09-11T23:20:42.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.105+0000 7f7b56ff5640 1 -- 172.21.15.26:0/1424346973 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 33) ==== 8394+0+0 (secure 0 0 0) 0x7f7b6c00f040 con 0x55b2efc014a0 2024-09-11T23:20:42.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.105+0000 7f7b7b5d2740 10 client.16056 fetch_fsmap finished waiting for FSMap version 33 2024-09-11T23:20:42.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.105+0000 7f7b7b5d2740 10 client.16056 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:20:42.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.105+0000 7f7b7b5d2740 20 client.16056 populate_metadata read hostname 'smithi026' 2024-09-11T23:20:42.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.105+0000 7f7b7b5d2740 1 --2- 172.21.15.26:0/1424346973 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55b2efc1c010 0x55b2efc1e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:42.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.105+0000 7f7b7b5d2740 4 client.16056 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:20:42.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.105+0000 7f7b7b5d2740 1 -- 172.21.15.26:0/1424346973 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b2efb74850 con 0x55b2efc1c010 2024-09-11T23:20:42.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.105+0000 7f7b7259c640 1 --2- 172.21.15.26:0/1424346973 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55b2efc1c010 0x55b2efc1e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:42.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.105+0000 7f7b7259c640 1 --2- 172.21.15.26:0/1424346973 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55b2efc1c010 0x55b2efc1e400 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cacfb6cc6a98dd23 server_cookie=99b9ee6e14c39f06 in_seq=0 out_seq=0 2024-09-11T23:20:42.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.105+0000 7f7b56ff5640 10 client.16056 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:20:42.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.106+0000 7f7b56ff5640 1 -- 172.21.15.26:0/1424346973 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b2efb74850 con 0x55b2efc1c010 2024-09-11T23:20:42.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.106+0000 7f7b56ff5640 10 client.16056 handle_command_reply: tid=0 2024-09-11T23:20:42.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.106+0000 7f7b7b5d2740 1 client.16056 shutdown 2024-09-11T23:20:42.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.106+0000 7f7b7b5d2740 1 -- 172.21.15.26:0/1424346973 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b2efc014a0 msgr2=0x55b2efc0bfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:42.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.106+0000 7f7b7b5d2740 1 --2- 172.21.15.26:0/1424346973 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b2efc014a0 0x55b2efc0bfc0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f7b6c006010 tx=0x7f7b6c012590 comp rx=0 tx=0).stop 2024-09-11T23:20:42.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.106+0000 7f7b7b5d2740 1 -- 172.21.15.26:0/1424346973 shutdown_connections 2024-09-11T23:20:42.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.106+0000 7f7b7b5d2740 1 --2- 172.21.15.26:0/1424346973 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b2efc0c680 0x55b2efc097b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:42.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.106+0000 7f7b7b5d2740 1 --2- 172.21.15.26:0/1424346973 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55b2efc1c010 0x55b2efc1e400 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:42.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.106+0000 7f7b7b5d2740 1 --2- 172.21.15.26:0/1424346973 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b2efc01dc0 0x55b2efc09270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:42.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.106+0000 7f7b56ff5640 0 client.16056 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:20:42.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.106+0000 7f7b7b5d2740 1 --2- 172.21.15.26:0/1424346973 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b2efc014a0 0x55b2efc0bfc0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:42.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.106+0000 7f7b7b5d2740 1 -- 172.21.15.26:0/1424346973 >> 172.21.15.26:0/1424346973 conn(0x55b2efbfda30 msgr2=0x55b2efc03710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:42.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.106+0000 7f7b7b5d2740 1 -- 172.21.15.26:0/1424346973 shutdown_connections 2024-09-11T23:20:42.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.106+0000 7f7b7b5d2740 1 -- 172.21.15.26:0/1424346973 wait complete. 2024-09-11T23:20:42.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.106+0000 7f7b7b5d2740 20 client.16056 trim_cache size 0 max 16384 2024-09-11T23:20:42.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.124+0000 7f7b7b5d2740 1 Processor -- start 2024-09-11T23:20:42.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.124+0000 7f7b7b5d2740 1 -- start start 2024-09-11T23:20:42.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.125+0000 7f7b7b5d2740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b2efc014a0 0x55b2efc23a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:42.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.125+0000 7f7b7b5d2740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b2efc01dc0 0x55b2efc23fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:42.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.125+0000 7f7b7b5d2740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b2efc0c680 0x55b2efc064d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:42.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.125+0000 7f7b7b5d2740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f7b741afa40 con 0x55b2efc014a0 2024-09-11T23:20:42.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.125+0000 7f7b7b5d2740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f7b741b1700 con 0x55b2efc0c680 2024-09-11T23:20:42.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.125+0000 7f7b7b5d2740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f7b741b0110 con 0x55b2efc01dc0 2024-09-11T23:20:42.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.125+0000 7f7b7359e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b2efc014a0 0x55b2efc23a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:42.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.125+0000 7f7b7259c640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b2efc01dc0 0x55b2efc23fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:42.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.125+0000 7f7b7359e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b2efc014a0 0x55b2efc23a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55650/0 (socket says 172.21.15.26:55650) 2024-09-11T23:20:42.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.125+0000 7f7b7359e640 1 -- 172.21.15.26:0/3641230267 learned_addr learned my addr 172.21.15.26:0/3641230267 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:20:42.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.125+0000 7f7b7359e640 1 --2- 172.21.15.26:0/3641230267 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b2efc0c680 0x55b2efc064d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:42.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.125+0000 7f7b7259c640 1 -- 172.21.15.26:0/3641230267 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b2efc0c680 msgr2=0x55b2efc064d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:20:42.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.125+0000 7f7b7259c640 1 --2- 172.21.15.26:0/3641230267 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b2efc0c680 0x55b2efc064d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:42.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.125+0000 7f7b7259c640 1 -- 172.21.15.26:0/3641230267 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b2efc014a0 msgr2=0x55b2efc23a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:20:42.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.125+0000 7f7b7259c640 1 --2- 172.21.15.26:0/3641230267 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b2efc014a0 0x55b2efc23a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:42.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.125+0000 7f7b7259c640 1 -- 172.21.15.26:0/3641230267 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b2efbfeae0 con 0x55b2efc01dc0 2024-09-11T23:20:42.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.125+0000 7f7b7359e640 1 --2- 172.21.15.26:0/3641230267 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b2efc014a0 0x55b2efc23a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:20:42.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.125+0000 7f7b7259c640 1 --2- 172.21.15.26:0/3641230267 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b2efc01dc0 0x55b2efc23fd0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x55b2efc1ea30 tx=0x7f7b68005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3346870be97ee55d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:42.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.126+0000 7f7b58ff9640 1 -- 172.21.15.26:0/3641230267 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b68010430 con 0x55b2efc01dc0 2024-09-11T23:20:42.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.126+0000 7f7b58ff9640 1 -- 172.21.15.26:0/3641230267 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b68010ab0 con 0x55b2efc01dc0 2024-09-11T23:20:42.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.126+0000 7f7b58ff9640 1 -- 172.21.15.26:0/3641230267 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b6801f9e0 con 0x55b2efc01dc0 2024-09-11T23:20:42.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.127+0000 7f7b7b5d2740 1 -- 172.21.15.26:0/3641230267 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b2efc01dc0 msgr2=0x55b2efc23fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:42.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.127+0000 7f7b7b5d2740 1 --2- 172.21.15.26:0/3641230267 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b2efc01dc0 0x55b2efc23fd0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x55b2efc1ea30 tx=0x7f7b68005f90 comp rx=0 tx=0).stop 2024-09-11T23:20:42.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.127+0000 7f7b7b5d2740 1 -- 172.21.15.26:0/3641230267 shutdown_connections 2024-09-11T23:20:42.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.127+0000 7f7b7b5d2740 1 --2- 172.21.15.26:0/3641230267 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b2efc0c680 0x55b2efc064d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:42.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.127+0000 7f7b7b5d2740 1 --2- 172.21.15.26:0/3641230267 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b2efc01dc0 0x55b2efc23fd0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:42.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.127+0000 7f7b7b5d2740 1 --2- 172.21.15.26:0/3641230267 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b2efc014a0 0x55b2efc23a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:42.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.127+0000 7f7b7b5d2740 1 -- 172.21.15.26:0/3641230267 >> 172.21.15.26:0/3641230267 conn(0x55b2efbfda30 msgr2=0x55b2efc0b2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:42.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.127+0000 7f7b7b5d2740 1 -- 172.21.15.26:0/3641230267 shutdown_connections 2024-09-11T23:20:42.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.128+0000 7f7b7b5d2740 1 -- 172.21.15.26:0/3641230267 wait complete. 2024-09-11T23:20:42.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.128+0000 7f7b7b5d2740 1 -- start start 2024-09-11T23:20:42.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.129+0000 7f7b7b5d2740 1 Processor -- start 2024-09-11T23:20:42.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.129+0000 7f7b7b5d2740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b2efc014a0 0x55b2efc14b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:42.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.129+0000 7f7b7b5d2740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b2efc01dc0 0x55b2efc11f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:42.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.129+0000 7f7b7b5d2740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b2efc0c680 0x55b2efc12450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:42.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.129+0000 7f7b7b5d2740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f7b741b0f70 con 0x55b2efc0c680 2024-09-11T23:20:42.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.129+0000 7f7b7b5d2740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f7b741b07e0 con 0x55b2efc01dc0 2024-09-11T23:20:42.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.129+0000 7f7b7b5d2740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f7b741ada20 con 0x55b2efc014a0 2024-09-11T23:20:42.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.129+0000 7f7b7b5d2740 10 client.0 _ll_register_callbacks cb 0x7f7b68009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:20:42.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.129+0000 7f7b7359e640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b2efc014a0 0x55b2efc14b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:42.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.129+0000 7f7b7359e640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b2efc014a0 0x55b2efc14b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:50548/0 (socket says 172.21.15.26:50548) 2024-09-11T23:20:42.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.129+0000 7f7b7359e640 1 -- 172.21.15.26:0/81602860 learned_addr learned my addr 172.21.15.26:0/81602860 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:20:42.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.130+0000 7f7b7359e640 1 --2- 172.21.15.26:0/81602860 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b2efc0c680 0x55b2efc12450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:42.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.130+0000 7f7b7259c640 1 --2- 172.21.15.26:0/81602860 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b2efc01dc0 0x55b2efc11f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:42.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.130+0000 7f7b7359e640 1 -- 172.21.15.26:0/81602860 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b2efc01dc0 msgr2=0x55b2efc11f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:20:42.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.130+0000 7f7b7359e640 1 --2- 172.21.15.26:0/81602860 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b2efc01dc0 0x55b2efc11f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:42.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.130+0000 7f7b7359e640 1 -- 172.21.15.26:0/81602860 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b2efc0c680 msgr2=0x55b2efc12450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:20:42.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.130+0000 7f7b7359e640 1 --2- 172.21.15.26:0/81602860 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b2efc0c680 0x55b2efc12450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:42.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.130+0000 7f7b7359e640 1 -- 172.21.15.26:0/81602860 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b2efc12a10 con 0x55b2efc014a0 2024-09-11T23:20:42.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.130+0000 7f7b7359e640 1 --2- 172.21.15.26:0/81602860 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b2efc014a0 0x55b2efc14b20 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f7b6c0339a0 tx=0x7f7b6c0128a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=15b628d27d8bdc45 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:42.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.130+0000 7f7b56ff5640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-09-11T23:20:42.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.130+0000 7f7b56ff5640 1 -- 172.21.15.26:0/81602860 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b6c016030 con 0x55b2efc014a0 2024-09-11T23:20:42.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.130+0000 7f7b56ff5640 1 -- 172.21.15.26:0/81602860 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f7b6c005070 con 0x55b2efc014a0 2024-09-11T23:20:42.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.131+0000 7f7b7b5d2740 1 -- 172.21.15.26:0/81602860 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b68010ab0 con 0x55b2efc014a0 2024-09-11T23:20:42.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.131+0000 7f7b56ff5640 1 -- 172.21.15.26:0/81602860 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f7b6c012b50 con 0x55b2efc014a0 2024-09-11T23:20:42.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.131+0000 7f7b7b5d2740 10 client.25687 fetch_fsmap learned FSMap version 33 2024-09-11T23:20:42.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.131+0000 7f7b7b5d2740 1 -- 172.21.15.26:0/81602860 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55b2efc86830 con 0x55b2efc014a0 2024-09-11T23:20:42.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.131+0000 7f7b56ff5640 1 -- 172.21.15.26:0/81602860 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b68010ab0 con 0x55b2efc014a0 2024-09-11T23:20:42.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.131+0000 7f7b56ff5640 1 -- 172.21.15.26:0/81602860 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 33) ==== 8394+0+0 (secure 0 0 0) 0x7f7b6c01b570 con 0x55b2efc014a0 2024-09-11T23:20:42.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.131+0000 7f7b7b5d2740 10 client.25687 fetch_fsmap finished waiting for FSMap version 33 2024-09-11T23:20:42.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.131+0000 7f7b7b5d2740 10 client.25687 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:20:42.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.131+0000 7f7b7b5d2740 20 client.25687 populate_metadata read hostname 'smithi026' 2024-09-11T23:20:42.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.131+0000 7f7b7b5d2740 1 --2- 172.21.15.26:0/81602860 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55b2efc1c010 0x55b2efc88f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:42.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.131+0000 7f7b7b5d2740 4 client.25687 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T23:20:42.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.131+0000 7f7b7b5d2740 1 -- 172.21.15.26:0/81602860 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b2efb6ed60 con 0x55b2efc1c010 2024-09-11T23:20:42.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.132+0000 7f7b7259c640 1 --2- 172.21.15.26:0/81602860 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55b2efc1c010 0x55b2efc88f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:42.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.132+0000 7f7b7259c640 1 --2- 172.21.15.26:0/81602860 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55b2efc1c010 0x55b2efc88f90 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d06c88f049760d4d server_cookie=7219ea713e779698 in_seq=0 out_seq=0 2024-09-11T23:20:42.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.132+0000 7f7b56ff5640 10 client.25687 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:20:42.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.132+0000 7f7b56ff5640 1 -- 172.21.15.26:0/81602860 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55b2efb6ed60 con 0x55b2efc1c010 2024-09-11T23:20:42.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.132+0000 7f7b56ff5640 10 client.25687 handle_command_reply: tid=0 2024-09-11T23:20:42.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.132+0000 7f7b7b5d2740 1 client.25687 shutdown 2024-09-11T23:20:42.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.132+0000 7f7b7b5d2740 1 -- 172.21.15.26:0/81602860 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b2efc014a0 msgr2=0x55b2efc14b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:42.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.132+0000 7f7b7b5d2740 1 --2- 172.21.15.26:0/81602860 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b2efc014a0 0x55b2efc14b20 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f7b6c0339a0 tx=0x7f7b6c0128a0 comp rx=0 tx=0).stop 2024-09-11T23:20:42.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.132+0000 7f7b7b5d2740 1 -- 172.21.15.26:0/81602860 shutdown_connections 2024-09-11T23:20:42.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.132+0000 7f7b7b5d2740 1 --2- 172.21.15.26:0/81602860 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b2efc01dc0 0x55b2efc11f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:42.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.132+0000 7f7b7b5d2740 1 --2- 172.21.15.26:0/81602860 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55b2efc1c010 0x55b2efc88f90 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:42.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.132+0000 7f7b7b5d2740 1 --2- 172.21.15.26:0/81602860 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b2efc0c680 0x55b2efc12450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:42.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.132+0000 7f7b7b5d2740 1 --2- 172.21.15.26:0/81602860 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b2efc014a0 0x55b2efc14b20 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:42.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.132+0000 7f7b56ff5640 0 client.25687 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:20:42.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.132+0000 7f7b7b5d2740 1 -- 172.21.15.26:0/81602860 >> 172.21.15.26:0/81602860 conn(0x55b2efbfda30 msgr2=0x55b2efc0b2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:42.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.133+0000 7f7b7b5d2740 1 -- 172.21.15.26:0/81602860 shutdown_connections 2024-09-11T23:20:42.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.133+0000 7f7b7b5d2740 1 -- 172.21.15.26:0/81602860 wait complete. 2024-09-11T23:20:42.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.133+0000 7f7b7b5d2740 20 client.25687 trim_cache size 0 max 16384 2024-09-11T23:20:42.137 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-11T23:20:42.137 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "no active scrubs running", 2024-09-11T23:20:42.137 INFO:teuthology.orchestra.run.smithi026.stdout: "scrubs": {} 2024-09-11T23:20:42.137 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-11T23:20:42.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.133+0000 7f7b791f9640 1 -- 172.21.15.26:0/3548184564 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f7b5c0a2cf0 msgr2=0x7f7b5c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:42.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.133+0000 7f7b791f9640 1 --2- 172.21.15.26:0/3548184564 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f7b5c0a2cf0 0x7f7b5c0a51b0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f7b640ed180 tx=0x7f7b640f5550 comp rx=0 tx=0).stop 2024-09-11T23:20:42.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.133+0000 7f7b791f9640 1 -- 172.21.15.26:0/3548184564 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7b74101de0 msgr2=0x7f7b740744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:42.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.133+0000 7f7b791f9640 1 --2- 172.21.15.26:0/3548184564 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7b74101de0 0x7f7b740744f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f7b64006010 tx=0x7f7b640042f0 comp rx=0 tx=0).stop 2024-09-11T23:20:42.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.133+0000 7f7b791f9640 1 -- 172.21.15.26:0/3548184564 shutdown_connections 2024-09-11T23:20:42.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.133+0000 7f7b791f9640 1 --2- 172.21.15.26:0/3548184564 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f7b5c0a2cf0 0x7f7b5c0a51b0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:42.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.133+0000 7f7b791f9640 1 --2- 172.21.15.26:0/3548184564 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7b74102790 0x7f7b74074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:42.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.133+0000 7f7b791f9640 1 --2- 172.21.15.26:0/3548184564 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b74106f10 0x7f7b74074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:42.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.133+0000 7f7b791f9640 1 --2- 172.21.15.26:0/3548184564 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7b74101de0 0x7f7b740744f0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:42.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.133+0000 7f7b791f9640 1 -- 172.21.15.26:0/3548184564 >> 172.21.15.26:0/3548184564 conn(0x7f7b740fd8c0 msgr2=0x7f7b740fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:42.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.133+0000 7f7b791f9640 1 -- 172.21.15.26:0/3548184564 shutdown_connections 2024-09-11T23:20:42.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:42.133+0000 7f7b791f9640 1 -- 172.21.15.26:0/3548184564 wait complete. 2024-09-11T23:20:42.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:41 smithi136 ceph-mon[35121]: pgmap v72: 97 pgs: 97 active+clean; 228 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 818 KiB/s wr, 109 op/s 2024-09-11T23:20:42.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:41 smithi026 ceph-mon[31172]: pgmap v72: 97 pgs: 97 active+clean; 228 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 818 KiB/s wr, 109 op/s 2024-09-11T23:20:42.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:41 smithi076 ceph-mon[35296]: pgmap v72: 97 pgs: 97 active+clean; 228 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 818 KiB/s wr, 109 op/s 2024-09-11T23:20:43.011 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-11T23:20:43.011 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:20:43.288 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:20:43.576 INFO:tasks.workunit.client.0.smithi026.stdout: 67 261983 309 163034 101 44914 350 2024-09-11T23:20:43.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:43.967+0000 7ff902b66640 1 -- 172.21.15.26:0/2127809717 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff8fc074620 msgr2=0x7ff8fc074a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:43.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:43.967+0000 7ff902b66640 1 --2- 172.21.15.26:0/2127809717 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff8fc074620 0x7ff8fc074a00 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7ff8f0009900 tx=0x7ff8f002a480 comp rx=0 tx=0).stop 2024-09-11T23:20:43.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:43.967+0000 7ff902b66640 1 -- 172.21.15.26:0/2127809717 shutdown_connections 2024-09-11T23:20:43.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:43.967+0000 7ff902b66640 1 --2- 172.21.15.26:0/2127809717 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff8fc10e3c0 0x7ff8fc110810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:43.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:43.967+0000 7ff902b66640 1 --2- 172.21.15.26:0/2127809717 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff8fc074f40 0x7ff8fc10de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:43.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:43.968+0000 7ff902b66640 1 --2- 172.21.15.26:0/2127809717 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff8fc074620 0x7ff8fc074a00 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:43.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:43.968+0000 7ff902b66640 1 -- 172.21.15.26:0/2127809717 >> 172.21.15.26:0/2127809717 conn(0x7ff8fc0fd8c0 msgr2=0x7ff8fc0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:43.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:43.968+0000 7ff902b66640 1 -- 172.21.15.26:0/2127809717 shutdown_connections 2024-09-11T23:20:43.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:43.968+0000 7ff902b66640 1 -- 172.21.15.26:0/2127809717 wait complete. 2024-09-11T23:20:43.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:43.968+0000 7ff902b66640 1 Processor -- start 2024-09-11T23:20:43.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:43.968+0000 7ff902b66640 1 -- start start 2024-09-11T23:20:43.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:43.968+0000 7ff902b66640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff8fc074620 0x7ff8fc19a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:43.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:43.969+0000 7ff902b66640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff8fc074f40 0x7ff8fc19ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:43.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:43.969+0000 7ff902b66640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff8fc10e3c0 0x7ff8fc19efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:43.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:43.969+0000 7ff902b66640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff8fc114150 con 0x7ff8fc10e3c0 2024-09-11T23:20:43.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:43.969+0000 7ff902b66640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7ff8fc113fd0 con 0x7ff8fc074620 2024-09-11T23:20:43.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:43.969+0000 7ff902b66640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ff8fc1142d0 con 0x7ff8fc074f40 2024-09-11T23:20:43.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:43.969+0000 7ff900902640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff8fc074620 0x7ff8fc19a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:43.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:43.969+0000 7ff901103640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff8fc10e3c0 0x7ff8fc19efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:43.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:43.969+0000 7ff8fbfff640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff8fc074f40 0x7ff8fc19ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:43.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:43.969+0000 7ff900902640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff8fc074620 0x7ff8fc19a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:60836/0 (socket says 172.21.15.26:60836) 2024-09-11T23:20:43.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:43.969+0000 7ff900902640 1 -- 172.21.15.26:0/1061437214 learned_addr learned my addr 172.21.15.26:0/1061437214 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:20:43.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:43.969+0000 7ff901103640 1 -- 172.21.15.26:0/1061437214 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff8fc074f40 msgr2=0x7ff8fc19ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:43.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:43.969+0000 7ff901103640 1 --2- 172.21.15.26:0/1061437214 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff8fc074f40 0x7ff8fc19ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:43.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:43.969+0000 7ff901103640 1 -- 172.21.15.26:0/1061437214 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff8fc074620 msgr2=0x7ff8fc19a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:43.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:43.969+0000 7ff901103640 1 --2- 172.21.15.26:0/1061437214 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff8fc074620 0x7ff8fc19a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:43.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:43.969+0000 7ff901103640 1 -- 172.21.15.26:0/1061437214 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8fc19f6c0 con 0x7ff8fc10e3c0 2024-09-11T23:20:43.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:43.969+0000 7ff8fbfff640 1 --2- 172.21.15.26:0/1061437214 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff8fc074f40 0x7ff8fc19ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:20:43.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:43.969+0000 7ff900902640 1 --2- 172.21.15.26:0/1061437214 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff8fc074620 0x7ff8fc19a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:20:43.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:43.969+0000 7ff901103640 1 --2- 172.21.15.26:0/1061437214 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff8fc10e3c0 0x7ff8fc19efe0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7ff8f4009fd0 tx=0x7ff8f400ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:43.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:43.969+0000 7ff8f9ffb640 1 -- 172.21.15.26:0/1061437214 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8f400ce20 con 0x7ff8fc10e3c0 2024-09-11T23:20:43.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:43.970+0000 7ff902b66640 1 -- 172.21.15.26:0/1061437214 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8fc19f9b0 con 0x7ff8fc10e3c0 2024-09-11T23:20:43.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:43.970+0000 7ff8f9ffb640 1 -- 172.21.15.26:0/1061437214 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff8f4005020 con 0x7ff8fc10e3c0 2024-09-11T23:20:43.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:43.970+0000 7ff8f9ffb640 1 -- 172.21.15.26:0/1061437214 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8f4010720 con 0x7ff8fc10e3c0 2024-09-11T23:20:43.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:43.970+0000 7ff902b66640 1 -- 172.21.15.26:0/1061437214 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8fc1a7290 con 0x7ff8fc10e3c0 2024-09-11T23:20:43.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:43.971+0000 7ff902b66640 1 -- 172.21.15.26:0/1061437214 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8fc074a00 con 0x7ff8fc10e3c0 2024-09-11T23:20:43.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:43.972+0000 7ff8f9ffb640 1 -- 172.21.15.26:0/1061437214 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7ff8f4005540 con 0x7ff8fc10e3c0 2024-09-11T23:20:43.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:43.973+0000 7ff8f9ffb640 1 --2- 172.21.15.26:0/1061437214 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7ff8ec0a2d70 0x7ff8ec0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:43.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:43.973+0000 7ff8f9ffb640 1 -- 172.21.15.26:0/1061437214 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7ff8f40cfa50 con 0x7ff8fc10e3c0 2024-09-11T23:20:43.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:43.973+0000 7ff900902640 1 --2- 172.21.15.26:0/1061437214 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7ff8ec0a2d70 0x7ff8ec0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:43.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:43.973+0000 7ff900902640 1 --2- 172.21.15.26:0/1061437214 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7ff8ec0a2d70 0x7ff8ec0a5230 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ff8f00096c0 tx=0x7ff8f0005a20 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:43.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:43.977+0000 7ff8f9ffb640 1 -- 172.21.15.26:0/1061437214 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff8f409c680 con 0x7ff8fc10e3c0 2024-09-11T23:20:44.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:43 smithi136 ceph-mon[35121]: pgmap v73: 97 pgs: 97 active+clean; 228 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 818 KiB/s wr, 109 op/s 2024-09-11T23:20:44.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:43 smithi026 ceph-mon[31172]: pgmap v73: 97 pgs: 97 active+clean; 228 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 818 KiB/s wr, 109 op/s 2024-09-11T23:20:44.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:43 smithi076 ceph-mon[35296]: pgmap v73: 97 pgs: 97 active+clean; 228 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 818 KiB/s wr, 109 op/s 2024-09-11T23:20:44.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:44.385+0000 7ff902b66640 1 -- 172.21.15.26:0/1061437214 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff8fc02d7c0 con 0x7ff8fc10e3c0 2024-09-11T23:20:44.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:44.392+0000 7ff8f9ffb640 1 -- 172.21.15.26:0/1061437214 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+11986 (secure 0 0 0) 0x7ff8f409bf80 con 0x7ff8fc10e3c0 2024-09-11T23:20:44.393 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-11T23:20:44.393 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":33,"btime":"2024-09-11T23:20:14:305449+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14499,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/367659515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":367659515},{"type":"v1","addr":"172.21.15.26:6835","nonce":367659515}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14511,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/2041065299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":2041065299},{"type":"v1","addr":"172.21.15.26:6837","nonce":2041065299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":33,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:16:26.544203+0000","modified":"2024-09-11T23:20:13.958301+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24485,"mds_1":24463,"mds_2":24458,"mds_3":24448,"mds_4":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14550":{"gid":14550,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6839/2233068305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":2233068305},{"type":"v1","addr":"172.21.15.26:6839","nonce":2233068305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14562":{"gid":14562,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6841/2984982336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":2984982336},{"type":"v1","addr":"172.21.15.26:6841","nonce":2984982336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6833/2019841455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":2019841455},{"type":"v1","addr":"172.21.15.136:6833","nonce":2019841455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.76:6833/3438147050","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3438147050},{"type":"v1","addr":"172.21.15.76:6833","nonce":3438147050}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6835/3617665975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3617665975},{"type":"v1","addr":"172.21.15.136:6835","nonce":3617665975}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":12,"addr":"172.21.15.76:6835/393308499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":393308499},{"type":"v1","addr":"172.21.15.76:6835","nonce":393308499}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":10,"addr":"172.21.15.136:6837/1633082387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":1633082387},{"type":"v1","addr":"172.21.15.136:6837","nonce":1633082387}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.76:6837/2078978748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":2078978748},{"type":"v1","addr":"172.21.15.76:6837","nonce":2078978748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":63,"addr":"172.21.15.136:6839/3211398630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":3211398630},{"type":"v1","addr":"172.21.15.136:6839","nonce":3211398630}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.76:6839/2077637090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6838","nonce":2077637090},{"type":"v1","addr":"172.21.15.76:6839","nonce":2077637090}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24485,"qdb_cluster":[24443,24448,24458,24463,24485]},"id":1}]} 2024-09-11T23:20:44.393 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 33 2024-09-11T23:20:44.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:44.401+0000 7ff8e77fe640 1 -- 172.21.15.26:0/1061437214 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7ff8ec0a2d70 msgr2=0x7ff8ec0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:44.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:44.401+0000 7ff8e77fe640 1 --2- 172.21.15.26:0/1061437214 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7ff8ec0a2d70 0x7ff8ec0a5230 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ff8f00096c0 tx=0x7ff8f0005a20 comp rx=0 tx=0).stop 2024-09-11T23:20:44.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:44.401+0000 7ff8e77fe640 1 -- 172.21.15.26:0/1061437214 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff8fc10e3c0 msgr2=0x7ff8fc19efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:44.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:44.401+0000 7ff8e77fe640 1 --2- 172.21.15.26:0/1061437214 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff8fc10e3c0 0x7ff8fc19efe0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7ff8f4009fd0 tx=0x7ff8f400ef10 comp rx=0 tx=0).stop 2024-09-11T23:20:44.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:44.401+0000 7ff8e77fe640 1 -- 172.21.15.26:0/1061437214 shutdown_connections 2024-09-11T23:20:44.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:44.401+0000 7ff8e77fe640 1 --2- 172.21.15.26:0/1061437214 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7ff8ec0a2d70 0x7ff8ec0a5230 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:44.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:44.401+0000 7ff8e77fe640 1 --2- 172.21.15.26:0/1061437214 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff8fc10e3c0 0x7ff8fc19efe0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:44.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:44.401+0000 7ff8e77fe640 1 --2- 172.21.15.26:0/1061437214 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff8fc074f40 0x7ff8fc19ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:44.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:44.401+0000 7ff8e77fe640 1 --2- 172.21.15.26:0/1061437214 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff8fc074620 0x7ff8fc19a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:44.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:44.401+0000 7ff8e77fe640 1 -- 172.21.15.26:0/1061437214 >> 172.21.15.26:0/1061437214 conn(0x7ff8fc0fd8c0 msgr2=0x7ff8fc0fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:44.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:44.403+0000 7ff8e77fe640 1 -- 172.21.15.26:0/1061437214 shutdown_connections 2024-09-11T23:20:44.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:44.403+0000 7ff8e77fe640 1 -- 172.21.15.26:0/1061437214 wait complete. 2024-09-11T23:20:44.525 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.524+0000 7f1e80766640 1 -- 172.21.15.26:0/4158532556 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1e78103660 msgr2=0x7f1e7810bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:44.525 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.525+0000 7f1e80766640 1 --2- 172.21.15.26:0/4158532556 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1e78103660 0x7f1e7810bb40 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f1e68009900 tx=0x7f1e6802a410 comp rx=0 tx=0).stop 2024-09-11T23:20:44.525 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.525+0000 7f1e80766640 1 -- 172.21.15.26:0/4158532556 shutdown_connections 2024-09-11T23:20:44.525 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.525+0000 7f1e80766640 1 --2- 172.21.15.26:0/4158532556 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1e7810c110 0x7f1e7810e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:44.526 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.525+0000 7f1e80766640 1 --2- 172.21.15.26:0/4158532556 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1e78103660 0x7f1e7810bb40 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:44.526 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.525+0000 7f1e80766640 1 --2- 172.21.15.26:0/4158532556 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1e78102d40 0x7f1e78103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:44.526 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.525+0000 7f1e80766640 1 -- 172.21.15.26:0/4158532556 >> 172.21.15.26:0/4158532556 conn(0x7f1e780fc6b0 msgr2=0x7f1e780fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:44.526 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.525+0000 7f1e80766640 1 -- 172.21.15.26:0/4158532556 shutdown_connections 2024-09-11T23:20:44.526 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.525+0000 7f1e80766640 1 -- 172.21.15.26:0/4158532556 wait complete. 2024-09-11T23:20:44.526 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.526+0000 7f1e80766640 1 Processor -- start 2024-09-11T23:20:44.526 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.527+0000 7f1e80766640 1 -- start start 2024-09-11T23:20:44.528 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.527+0000 7f1e80766640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1e78102d40 0x7f1e78198200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:44.528 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.527+0000 7f1e80766640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1e78103660 0x7f1e78198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:44.528 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.527+0000 7f1e80766640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1e7810c110 0x7f1e7819cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:44.528 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.528+0000 7f1e7e502640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1e78102d40 0x7f1e78198200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:44.528 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.528+0000 7f1e7e502640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1e78102d40 0x7f1e78198200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:50584/0 (socket says 172.21.15.26:50584) 2024-09-11T23:20:44.528 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.528+0000 7f1e7e502640 1 -- 172.21.15.26:0/3747852682 learned_addr learned my addr 172.21.15.26:0/3747852682 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:20:44.528 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.528+0000 7f1e7dd01640 1 --2- 172.21.15.26:0/3747852682 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1e78103660 0x7f1e78198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:44.528 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.528+0000 7f1e7ed03640 1 --2- 172.21.15.26:0/3747852682 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1e7810c110 0x7f1e7819cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:44.529 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.528+0000 7f1e80766640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f1e78111f10 con 0x7f1e7810c110 2024-09-11T23:20:44.529 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.528+0000 7f1e80766640 1 -- 172.21.15.26:0/3747852682 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f1e78111d90 con 0x7f1e78103660 2024-09-11T23:20:44.529 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.528+0000 7f1e80766640 1 -- 172.21.15.26:0/3747852682 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f1e78112090 con 0x7f1e78102d40 2024-09-11T23:20:44.530 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.528+0000 7f1e7e502640 1 -- 172.21.15.26:0/3747852682 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1e78103660 msgr2=0x7f1e78198740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:44.530 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.528+0000 7f1e7e502640 1 --2- 172.21.15.26:0/3747852682 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1e78103660 0x7f1e78198740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:44.530 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.528+0000 7f1e7e502640 1 -- 172.21.15.26:0/3747852682 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1e7810c110 msgr2=0x7f1e7819cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:44.530 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.528+0000 7f1e7e502640 1 --2- 172.21.15.26:0/3747852682 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1e7810c110 0x7f1e7819cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:44.530 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.528+0000 7f1e7e502640 1 -- 172.21.15.26:0/3747852682 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e7819d1f0 con 0x7f1e78102d40 2024-09-11T23:20:44.530 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.529+0000 7f1e7e502640 1 --2- 172.21.15.26:0/3747852682 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1e78102d40 0x7f1e78198200 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f1e7400e9f0 tx=0x7f1e7400edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:44.530 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.529+0000 7f1e6f7fe640 1 -- 172.21.15.26:0/3747852682 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e7400cd80 con 0x7f1e78102d40 2024-09-11T23:20:44.530 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.529+0000 7f1e6f7fe640 1 -- 172.21.15.26:0/3747852682 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e74005500 con 0x7f1e78102d40 2024-09-11T23:20:44.530 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.529+0000 7f1e6f7fe640 1 -- 172.21.15.26:0/3747852682 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e74010690 con 0x7f1e78102d40 2024-09-11T23:20:44.530 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.529+0000 7f1e80766640 1 -- 172.21.15.26:0/3747852682 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e7819d4e0 con 0x7f1e78102d40 2024-09-11T23:20:44.530 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.529+0000 7f1e80766640 1 -- 172.21.15.26:0/3747852682 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e781008c0 con 0x7f1e78102d40 2024-09-11T23:20:44.533 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.533+0000 7f1e6f7fe640 1 -- 172.21.15.26:0/3747852682 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f1e740046c0 con 0x7f1e78102d40 2024-09-11T23:20:44.535 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.534+0000 7f1e6f7fe640 1 --2- 172.21.15.26:0/3747852682 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f1e640a7230 0x7f1e640a96f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:44.535 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.535+0000 7f1e7dd01640 1 --2- 172.21.15.26:0/3747852682 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f1e640a7230 0x7f1e640a96f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:44.538 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.538+0000 7f1e6f7fe640 1 -- 172.21.15.26:0/3747852682 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f1e740cf820 con 0x7f1e78102d40 2024-09-11T23:20:44.538 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.538+0000 7f1e835de740 1 Processor -- start 2024-09-11T23:20:44.538 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.539+0000 7f1e835de740 1 -- start start 2024-09-11T23:20:44.539 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.539+0000 7f1e7dd01640 1 --2- 172.21.15.26:0/3747852682 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f1e640a7230 0x7f1e640a96f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f1e7810d790 tx=0x7f1e6800b040 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:44.539 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.539+0000 7f1e835de740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ffc931eaf0 0x55ffc931eed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:44.539 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.539+0000 7f1e835de740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ffc931f410 0x55ffc9329770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:44.539 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.539+0000 7f1e835de740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ffc9329cb0 0x55ffc932c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:44.540 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.540+0000 7f1e7e502640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ffc931eaf0 0x55ffc931eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:44.540 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.540+0000 7f1e7e502640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ffc931eaf0 0x55ffc931eed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:60852/0 (socket says 172.21.15.26:60852) 2024-09-11T23:20:44.540 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.540+0000 7f1e7dd01640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ffc931f410 0x55ffc9329770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:44.540 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.540+0000 7f1e7e502640 1 -- 172.21.15.26:0/2171139989 learned_addr learned my addr 172.21.15.26:0/2171139989 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:20:44.540 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.540+0000 7f1e835de740 1 -- 172.21.15.26:0/2171139989 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55ffc92832f0 con 0x55ffc931f410 2024-09-11T23:20:44.541 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.540+0000 7f1e835de740 1 -- 172.21.15.26:0/2171139989 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55ffc922b270 con 0x55ffc931eaf0 2024-09-11T23:20:44.541 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.540+0000 7f1e835de740 1 -- 172.21.15.26:0/2171139989 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55ffc927f2b0 con 0x55ffc9329cb0 2024-09-11T23:20:44.541 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.540+0000 7f1e7ed03640 1 --2- 172.21.15.26:0/2171139989 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ffc9329cb0 0x55ffc932c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:44.541 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.540+0000 7f1e7ed03640 1 -- 172.21.15.26:0/2171139989 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ffc931eaf0 msgr2=0x55ffc931eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:20:44.541 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.540+0000 7f1e7ed03640 1 --2- 172.21.15.26:0/2171139989 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ffc931eaf0 0x55ffc931eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:44.541 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.540+0000 7f1e7ed03640 1 -- 172.21.15.26:0/2171139989 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ffc931f410 msgr2=0x55ffc9329770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:20:44.541 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.540+0000 7f1e7ed03640 1 --2- 172.21.15.26:0/2171139989 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ffc931f410 0x55ffc9329770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:44.541 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.540+0000 7f1e7ed03640 1 -- 172.21.15.26:0/2171139989 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ffc922a340 con 0x55ffc9329cb0 2024-09-11T23:20:44.541 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.541+0000 7f1e7dd01640 1 --2- 172.21.15.26:0/2171139989 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ffc931f410 0x55ffc9329770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:20:44.541 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.541+0000 7f1e7ed03640 1 --2- 172.21.15.26:0/2171139989 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ffc9329cb0 0x55ffc932c0a0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f1e70017fa0 tx=0x7f1e70005bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ec4adb8f1c0168d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:44.542 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.541+0000 7f1e6cff9640 1 -- 172.21.15.26:0/2171139989 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e7000eb80 con 0x55ffc9329cb0 2024-09-11T23:20:44.542 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.541+0000 7f1e6cff9640 1 -- 172.21.15.26:0/2171139989 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e70003e50 con 0x55ffc9329cb0 2024-09-11T23:20:44.542 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.541+0000 7f1e6cff9640 1 -- 172.21.15.26:0/2171139989 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e700203c0 con 0x55ffc9329cb0 2024-09-11T23:20:44.542 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.542+0000 7f1e835de740 1 -- 172.21.15.26:0/2171139989 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ffc9329cb0 msgr2=0x55ffc932c0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:44.542 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.542+0000 7f1e835de740 1 --2- 172.21.15.26:0/2171139989 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ffc9329cb0 0x55ffc932c0a0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f1e70017fa0 tx=0x7f1e70005bd0 comp rx=0 tx=0).stop 2024-09-11T23:20:44.542 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.542+0000 7f1e835de740 1 -- 172.21.15.26:0/2171139989 shutdown_connections 2024-09-11T23:20:44.542 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.542+0000 7f1e835de740 1 --2- 172.21.15.26:0/2171139989 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ffc9329cb0 0x55ffc932c0a0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:44.542 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.542+0000 7f1e835de740 1 --2- 172.21.15.26:0/2171139989 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ffc931f410 0x55ffc9329770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:44.543 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.542+0000 7f1e835de740 1 --2- 172.21.15.26:0/2171139989 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ffc931eaf0 0x55ffc931eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:44.543 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.542+0000 7f1e835de740 1 -- 172.21.15.26:0/2171139989 >> 172.21.15.26:0/2171139989 conn(0x55ffc931b060 msgr2=0x55ffc931b440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:44.543 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.544+0000 7f1e835de740 1 -- 172.21.15.26:0/2171139989 shutdown_connections 2024-09-11T23:20:44.544 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.544+0000 7f1e835de740 1 -- 172.21.15.26:0/2171139989 wait complete. 2024-09-11T23:20:44.544 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.544+0000 7f1e835de740 1 -- start start 2024-09-11T23:20:44.545 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.544+0000 7f1e835de740 1 Processor -- start 2024-09-11T23:20:44.545 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.545+0000 7f1e835de740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ffc931eaf0 0x55ffc9326b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:44.546 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.545+0000 7f1e835de740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ffc931f410 0x55ffc93270b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:44.546 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.545+0000 7f1e835de740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ffc9329cb0 0x55ffc93275f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:44.546 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.545+0000 7f1e835de740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55ffc932c880 con 0x55ffc9329cb0 2024-09-11T23:20:44.546 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.545+0000 7f1e835de740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55ffc920c290 con 0x55ffc931eaf0 2024-09-11T23:20:44.546 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.545+0000 7f1e835de740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55ffc932ca00 con 0x55ffc931f410 2024-09-11T23:20:44.546 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.545+0000 7f1e835de740 10 client.0 _ll_register_callbacks cb 0x55ffc923d140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:20:44.546 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.545+0000 7f1e7dd01640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ffc931f410 0x55ffc93270b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:44.546 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.545+0000 7f1e7dd01640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ffc931f410 0x55ffc93270b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:50602/0 (socket says 172.21.15.26:50602) 2024-09-11T23:20:44.546 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.545+0000 7f1e7dd01640 1 -- 172.21.15.26:0/1255818515 learned_addr learned my addr 172.21.15.26:0/1255818515 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:20:44.546 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.545+0000 7f1e7e502640 1 --2- 172.21.15.26:0/1255818515 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ffc931eaf0 0x55ffc9326b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:44.546 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.546+0000 7f1e7dd01640 1 -- 172.21.15.26:0/1255818515 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ffc931eaf0 msgr2=0x55ffc9326b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:20:44.547 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.546+0000 7f1e7dd01640 1 --2- 172.21.15.26:0/1255818515 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ffc931eaf0 0x55ffc9326b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:44.547 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.546+0000 7f1e7ed03640 1 --2- 172.21.15.26:0/1255818515 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ffc9329cb0 0x55ffc93275f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:44.547 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.546+0000 7f1e7dd01640 1 -- 172.21.15.26:0/1255818515 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ffc9329cb0 msgr2=0x55ffc93275f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:20:44.547 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.546+0000 7f1e7dd01640 1 --2- 172.21.15.26:0/1255818515 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ffc9329cb0 0x55ffc93275f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:44.547 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.546+0000 7f1e7dd01640 1 -- 172.21.15.26:0/1255818515 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ffc8f96210 con 0x55ffc931f410 2024-09-11T23:20:44.547 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.546+0000 7f1e7ed03640 1 --2- 172.21.15.26:0/1255818515 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ffc9329cb0 0x55ffc93275f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:20:44.549 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.546+0000 7f1e7dd01640 1 --2- 172.21.15.26:0/1255818515 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ffc931f410 0x55ffc93270b0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f1e68033000 tx=0x7f1e68005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=29be800d0c9858e0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:44.549 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.546+0000 7f1e61ffb640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-09-11T23:20:44.549 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.546+0000 7f1e61ffb640 1 -- 172.21.15.26:0/1255818515 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e6804f070 con 0x55ffc931f410 2024-09-11T23:20:44.549 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.546+0000 7f1e61ffb640 1 -- 172.21.15.26:0/1255818515 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f1e68003d00 con 0x55ffc931f410 2024-09-11T23:20:44.549 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.546+0000 7f1e835de740 1 -- 172.21.15.26:0/1255818515 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e70003e50 con 0x55ffc931f410 2024-09-11T23:20:44.549 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.547+0000 7f1e61ffb640 1 -- 172.21.15.26:0/1255818515 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f1e68059050 con 0x55ffc931f410 2024-09-11T23:20:44.549 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.547+0000 7f1e835de740 10 client.25705 fetch_fsmap learned FSMap version 33 2024-09-11T23:20:44.549 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.547+0000 7f1e835de740 1 -- 172.21.15.26:0/1255818515 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55ffc9062620 con 0x55ffc931f410 2024-09-11T23:20:44.550 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.547+0000 7f1e61ffb640 1 -- 172.21.15.26:0/1255818515 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e70003e50 con 0x55ffc931f410 2024-09-11T23:20:44.550 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.547+0000 7f1e61ffb640 1 -- 172.21.15.26:0/1255818515 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 33) ==== 8394+0+0 (secure 0 0 0) 0x7f1e68054740 con 0x55ffc931f410 2024-09-11T23:20:44.550 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.547+0000 7f1e835de740 10 client.25705 fetch_fsmap finished waiting for FSMap version 33 2024-09-11T23:20:44.550 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.547+0000 7f1e835de740 10 client.25705 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:20:44.550 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.547+0000 7f1e835de740 20 client.25705 populate_metadata read hostname 'smithi026' 2024-09-11T23:20:44.550 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.547+0000 7f1e835de740 1 --2- 172.21.15.26:0/1255818515 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55ffc934acf0 0x55ffc934d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:44.550 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.547+0000 7f1e835de740 4 client.25705 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:20:44.550 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.547+0000 7f1e835de740 1 -- 172.21.15.26:0/1255818515 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ffc9290a20 con 0x55ffc934acf0 2024-09-11T23:20:44.550 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.548+0000 7f1e7e502640 1 --2- 172.21.15.26:0/1255818515 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55ffc934acf0 0x55ffc934d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:44.550 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.548+0000 7f1e7e502640 1 --2- 172.21.15.26:0/1255818515 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55ffc934acf0 0x55ffc934d150 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b3cd709c1a106f4 server_cookie=50a0d74a9caab456 in_seq=0 out_seq=0 2024-09-11T23:20:44.550 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.548+0000 7f1e61ffb640 10 client.25705 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:20:44.551 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.549+0000 7f1e61ffb640 1 -- 172.21.15.26:0/1255818515 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ffc9290a20 con 0x55ffc934acf0 2024-09-11T23:20:44.551 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.549+0000 7f1e61ffb640 10 client.25705 handle_command_reply: tid=0 2024-09-11T23:20:44.551 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.549+0000 7f1e835de740 1 client.25705 shutdown 2024-09-11T23:20:44.552 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.549+0000 7f1e835de740 1 -- 172.21.15.26:0/1255818515 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ffc931f410 msgr2=0x55ffc93270b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:44.552 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.549+0000 7f1e835de740 1 --2- 172.21.15.26:0/1255818515 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ffc931f410 0x55ffc93270b0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f1e68033000 tx=0x7f1e68005fd0 comp rx=0 tx=0).stop 2024-09-11T23:20:44.552 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.549+0000 7f1e835de740 1 -- 172.21.15.26:0/1255818515 shutdown_connections 2024-09-11T23:20:44.552 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.549+0000 7f1e835de740 1 --2- 172.21.15.26:0/1255818515 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55ffc934acf0 0x55ffc934d150 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:44.552 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.549+0000 7f1e835de740 1 --2- 172.21.15.26:0/1255818515 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ffc9329cb0 0x55ffc93275f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:44.552 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.549+0000 7f1e835de740 1 --2- 172.21.15.26:0/1255818515 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ffc931f410 0x55ffc93270b0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:44.552 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.549+0000 7f1e835de740 1 --2- 172.21.15.26:0/1255818515 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ffc931eaf0 0x55ffc9326b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:44.552 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.549+0000 7f1e61ffb640 0 client.25705 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:20:44.552 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.549+0000 7f1e835de740 1 -- 172.21.15.26:0/1255818515 >> 172.21.15.26:0/1255818515 conn(0x55ffc931b060 msgr2=0x55ffc9320dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:44.552 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.550+0000 7f1e835de740 1 -- 172.21.15.26:0/1255818515 shutdown_connections 2024-09-11T23:20:44.553 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.550+0000 7f1e835de740 1 -- 172.21.15.26:0/1255818515 wait complete. 2024-09-11T23:20:44.553 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.550+0000 7f1e835de740 20 client.25705 trim_cache size 0 max 16384 2024-09-11T23:20:44.572 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.570+0000 7f1e835de740 1 Processor -- start 2024-09-11T23:20:44.572 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.570+0000 7f1e835de740 1 -- start start 2024-09-11T23:20:44.572 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.571+0000 7f1e835de740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ffc931eaf0 0x55ffc9325420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:44.572 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.571+0000 7f1e835de740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ffc931f410 0x55ffc9327420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:44.572 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.571+0000 7f1e835de740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ffc9329cb0 0x55ffc9352a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:44.572 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.571+0000 7f1e835de740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f1e780724a0 con 0x55ffc931f410 2024-09-11T23:20:44.572 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.571+0000 7f1e835de740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f1e78074160 con 0x55ffc931eaf0 2024-09-11T23:20:44.572 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.571+0000 7f1e835de740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f1e78072b70 con 0x55ffc9329cb0 2024-09-11T23:20:44.572 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.571+0000 7f1e7dd01640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ffc931f410 0x55ffc9327420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:44.572 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.571+0000 7f1e7dd01640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ffc931f410 0x55ffc9327420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55738/0 (socket says 172.21.15.26:55738) 2024-09-11T23:20:44.573 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.571+0000 7f1e7dd01640 1 -- 172.21.15.26:0/2386780403 learned_addr learned my addr 172.21.15.26:0/2386780403 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:20:44.573 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.571+0000 7f1e7e502640 1 --2- 172.21.15.26:0/2386780403 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ffc931eaf0 0x55ffc9325420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:44.573 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.571+0000 7f1e7ed03640 1 --2- 172.21.15.26:0/2386780403 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ffc9329cb0 0x55ffc9352a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:44.573 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.571+0000 7f1e7dd01640 1 -- 172.21.15.26:0/2386780403 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ffc9329cb0 msgr2=0x55ffc9352a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:20:44.573 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.571+0000 7f1e7dd01640 1 --2- 172.21.15.26:0/2386780403 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ffc9329cb0 0x55ffc9352a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:44.573 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.571+0000 7f1e7dd01640 1 -- 172.21.15.26:0/2386780403 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ffc931eaf0 msgr2=0x55ffc9325420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:20:44.573 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.571+0000 7f1e7dd01640 1 --2- 172.21.15.26:0/2386780403 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ffc931eaf0 0x55ffc9325420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:44.573 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.571+0000 7f1e7dd01640 1 -- 172.21.15.26:0/2386780403 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ffc931bcf0 con 0x55ffc931f410 2024-09-11T23:20:44.573 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.572+0000 7f1e7dd01640 1 --2- 172.21.15.26:0/2386780403 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ffc931f410 0x55ffc9327420 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f1e6800b010 tx=0x7f1e68000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e757df33d6a19ed2 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:44.573 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.572+0000 7f1e63fff640 1 -- 172.21.15.26:0/2386780403 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e6804f050 con 0x55ffc931f410 2024-09-11T23:20:44.573 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.572+0000 7f1e63fff640 1 -- 172.21.15.26:0/2386780403 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e68062070 con 0x55ffc931f410 2024-09-11T23:20:44.574 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.572+0000 7f1e63fff640 1 -- 172.21.15.26:0/2386780403 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e68035040 con 0x55ffc931f410 2024-09-11T23:20:44.577 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.573+0000 7f1e835de740 1 -- 172.21.15.26:0/2386780403 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ffc931f410 msgr2=0x55ffc9327420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:44.577 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.573+0000 7f1e835de740 1 --2- 172.21.15.26:0/2386780403 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ffc931f410 0x55ffc9327420 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f1e6800b010 tx=0x7f1e68000bc0 comp rx=0 tx=0).stop 2024-09-11T23:20:44.578 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.573+0000 7f1e835de740 1 -- 172.21.15.26:0/2386780403 shutdown_connections 2024-09-11T23:20:44.578 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.573+0000 7f1e835de740 1 --2- 172.21.15.26:0/2386780403 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ffc9329cb0 0x55ffc9352a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:44.578 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.573+0000 7f1e835de740 1 --2- 172.21.15.26:0/2386780403 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ffc931f410 0x55ffc9327420 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:44.578 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.573+0000 7f1e835de740 1 --2- 172.21.15.26:0/2386780403 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ffc931eaf0 0x55ffc9325420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:44.578 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.573+0000 7f1e835de740 1 -- 172.21.15.26:0/2386780403 >> 172.21.15.26:0/2386780403 conn(0x55ffc931b060 msgr2=0x55ffc9328bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:44.578 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.573+0000 7f1e835de740 1 -- 172.21.15.26:0/2386780403 shutdown_connections 2024-09-11T23:20:44.578 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.573+0000 7f1e835de740 1 -- 172.21.15.26:0/2386780403 wait complete. 2024-09-11T23:20:44.578 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.574+0000 7f1e835de740 1 -- start start 2024-09-11T23:20:44.578 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.574+0000 7f1e835de740 1 Processor -- start 2024-09-11T23:20:44.578 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.574+0000 7f1e835de740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ffc931eaf0 0x55ffc9332540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:44.578 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.574+0000 7f1e835de740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ffc9329cb0 0x55ffc932d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:44.579 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.575+0000 7f1e835de740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ffc932deb0 0x55ffc93b4a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:44.579 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.575+0000 7f1e7e502640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ffc931eaf0 0x55ffc9332540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:44.579 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.575+0000 7f1e7e502640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ffc931eaf0 0x55ffc9332540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:50612/0 (socket says 172.21.15.26:50612) 2024-09-11T23:20:44.579 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.575+0000 7f1e7dd01640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ffc9329cb0 0x55ffc932d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:44.579 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.575+0000 7f1e7dd01640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ffc9329cb0 0x55ffc932d7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55744/0 (socket says 172.21.15.26:55744) 2024-09-11T23:20:44.579 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.575+0000 7f1e835de740 1 -- 172.21.15.26:0/4093953391 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f1e780739d0 con 0x55ffc9329cb0 2024-09-11T23:20:44.579 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.575+0000 7f1e7e502640 1 -- 172.21.15.26:0/4093953391 learned_addr learned my addr 172.21.15.26:0/4093953391 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:20:44.579 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.575+0000 7f1e835de740 1 -- 172.21.15.26:0/4093953391 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f1e78073240 con 0x55ffc932deb0 2024-09-11T23:20:44.579 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.575+0000 7f1e835de740 1 -- 172.21.15.26:0/4093953391 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f1e78070480 con 0x55ffc931eaf0 2024-09-11T23:20:44.579 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.575+0000 7f1e7ed03640 1 --2- 172.21.15.26:0/4093953391 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ffc932deb0 0x55ffc93b4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:44.579 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.575+0000 7f1e835de740 10 client.0 _ll_register_callbacks cb 0x7f1e74005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:20:44.580 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.575+0000 7f1e7e502640 1 -- 172.21.15.26:0/4093953391 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ffc932deb0 msgr2=0x55ffc93b4a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:20:44.580 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.575+0000 7f1e7e502640 1 --2- 172.21.15.26:0/4093953391 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ffc932deb0 0x55ffc93b4a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:44.580 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.575+0000 7f1e7e502640 1 -- 172.21.15.26:0/4093953391 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ffc9329cb0 msgr2=0x55ffc932d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:20:44.580 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.575+0000 7f1e7e502640 1 --2- 172.21.15.26:0/4093953391 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ffc9329cb0 0x55ffc932d7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:44.580 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.575+0000 7f1e7e502640 1 -- 172.21.15.26:0/4093953391 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ffc93b4f70 con 0x55ffc931eaf0 2024-09-11T23:20:44.580 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.575+0000 7f1e835de740 1 -- 172.21.15.26:0/4093953391 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e68062070 con 0x55ffc931eaf0 2024-09-11T23:20:44.580 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.575+0000 7f1e7e502640 1 --2- 172.21.15.26:0/4093953391 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ffc931eaf0 0x55ffc9332540 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f1e7400c6a0 tx=0x7f1e7409c580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a615b8b71fcfc173 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:44.580 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.575+0000 7f1e61ffb640 10 client.25711 ms_handle_connect on v2:172.21.15.136:3300/0 2024-09-11T23:20:44.580 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.576+0000 7f1e61ffb640 1 -- 172.21.15.26:0/4093953391 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e740a9060 con 0x55ffc931eaf0 2024-09-11T23:20:44.580 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.576+0000 7f1e61ffb640 1 -- 172.21.15.26:0/4093953391 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f1e740983c0 con 0x55ffc931eaf0 2024-09-11T23:20:44.581 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.576+0000 7f1e61ffb640 1 -- 172.21.15.26:0/4093953391 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f1e740a49c0 con 0x55ffc931eaf0 2024-09-11T23:20:44.581 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.576+0000 7f1e835de740 10 client.25711 fetch_fsmap learned FSMap version 33 2024-09-11T23:20:44.581 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.576+0000 7f1e835de740 1 -- 172.21.15.26:0/4093953391 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({fsmap=33}) -- 0x55ffc93b5e60 con 0x55ffc931eaf0 2024-09-11T23:20:44.581 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.576+0000 7f1e61ffb640 1 -- 172.21.15.26:0/4093953391 <== mon.1 v2:172.21.15.136:3300/0 4 ==== fsmap(e 33) ==== 8394+0+0 (secure 0 0 0) 0x7f1e740b4040 con 0x55ffc931eaf0 2024-09-11T23:20:44.581 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.576+0000 7f1e835de740 10 client.25711 fetch_fsmap finished waiting for FSMap version 33 2024-09-11T23:20:44.581 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.576+0000 7f1e835de740 10 client.25711 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:20:44.581 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.576+0000 7f1e835de740 20 client.25711 populate_metadata read hostname 'smithi026' 2024-09-11T23:20:44.581 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.576+0000 7f1e835de740 1 --2- 172.21.15.26:0/4093953391 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55ffc93b6630 0x55ffc93b8a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:44.581 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.576+0000 7f1e835de740 4 client.25711 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T23:20:44.581 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.576+0000 7f1e835de740 1 -- 172.21.15.26:0/4093953391 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ffc928ad70 con 0x55ffc93b6630 2024-09-11T23:20:44.581 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.576+0000 7f1e7dd01640 1 --2- 172.21.15.26:0/4093953391 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55ffc93b6630 0x55ffc93b8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:44.581 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.577+0000 7f1e7dd01640 1 --2- 172.21.15.26:0/4093953391 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55ffc93b6630 0x55ffc93b8a50 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf9451d346c1b357 server_cookie=bd99ba2178b1c1db in_seq=0 out_seq=0 2024-09-11T23:20:44.582 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:44.577+0000 7f1e61ffb640 10 client.25711 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:20:45.221 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:44 smithi026 ceph-mon[31172]: from='client.? 172.21.15.26:0/1061437214' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:20:45.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:44 smithi136 ceph-mon[35121]: from='client.? 172.21.15.26:0/1061437214' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:20:45.257 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-11T23:20:45.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:44 smithi076 ceph-mon[35296]: from='client.? 172.21.15.26:0/1061437214' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:20:45.326 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:45.325+0000 7f1e61ffb640 1 -- 172.21.15.26:0/4093953391 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ffc928ad70 con 0x55ffc93b6630 2024-09-11T23:20:45.327 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:45.325+0000 7f1e61ffb640 10 client.25711 handle_command_reply: tid=0 2024-09-11T23:20:45.327 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:45.326+0000 7f1e835de740 1 client.25711 shutdown 2024-09-11T23:20:45.327 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:45.326+0000 7f1e835de740 1 -- 172.21.15.26:0/4093953391 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ffc931eaf0 msgr2=0x55ffc9332540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:45.327 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:45.326+0000 7f1e835de740 1 --2- 172.21.15.26:0/4093953391 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ffc931eaf0 0x55ffc9332540 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f1e7400c6a0 tx=0x7f1e7409c580 comp rx=0 tx=0).stop 2024-09-11T23:20:45.327 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:45.326+0000 7f1e835de740 1 -- 172.21.15.26:0/4093953391 shutdown_connections 2024-09-11T23:20:45.327 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:45.326+0000 7f1e835de740 1 --2- 172.21.15.26:0/4093953391 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ffc932deb0 0x55ffc93b4a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:45.327 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:45.326+0000 7f1e835de740 1 --2- 172.21.15.26:0/4093953391 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55ffc93b6630 0x55ffc93b8a50 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:45.327 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:45.326+0000 7f1e835de740 1 --2- 172.21.15.26:0/4093953391 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ffc9329cb0 0x55ffc932d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:45.327 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:45.326+0000 7f1e61ffb640 0 client.25711 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:20:45.327 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:45.326+0000 7f1e835de740 1 --2- 172.21.15.26:0/4093953391 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ffc931eaf0 0x55ffc9332540 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:45.327 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:45.326+0000 7f1e835de740 1 -- 172.21.15.26:0/4093953391 >> 172.21.15.26:0/4093953391 conn(0x55ffc931b060 msgr2=0x55ffc93205a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:45.327 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:45.326+0000 7f1e835de740 1 -- 172.21.15.26:0/4093953391 shutdown_connections 2024-09-11T23:20:45.327 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:45.326+0000 7f1e835de740 1 -- 172.21.15.26:0/4093953391 wait complete. 2024-09-11T23:20:45.328 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:45.326+0000 7f1e835de740 20 client.25711 trim_cache size 0 max 16384 2024-09-11T23:20:45.328 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-09-11T23:20:45.328 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "message": "", 2024-09-11T23:20:45.328 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-09-11T23:20:45.328 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-09-11T23:20:45.329 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:45.327+0000 7f1e80766640 1 -- 172.21.15.26:0/3747852682 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f1e640a7230 msgr2=0x7f1e640a96f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:45.329 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:45.327+0000 7f1e80766640 1 --2- 172.21.15.26:0/3747852682 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f1e640a7230 0x7f1e640a96f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f1e7810d790 tx=0x7f1e6800b040 comp rx=0 tx=0).stop 2024-09-11T23:20:45.329 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:45.327+0000 7f1e80766640 1 -- 172.21.15.26:0/3747852682 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1e78102d40 msgr2=0x7f1e78198200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:45.329 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:45.327+0000 7f1e80766640 1 --2- 172.21.15.26:0/3747852682 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1e78102d40 0x7f1e78198200 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f1e7400e9f0 tx=0x7f1e7400edf0 comp rx=0 tx=0).stop 2024-09-11T23:20:45.329 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:45.327+0000 7f1e80766640 1 -- 172.21.15.26:0/3747852682 shutdown_connections 2024-09-11T23:20:45.329 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:45.327+0000 7f1e80766640 1 --2- 172.21.15.26:0/3747852682 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f1e640a7230 0x7f1e640a96f0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:45.329 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:45.327+0000 7f1e80766640 1 --2- 172.21.15.26:0/3747852682 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1e78103660 0x7f1e78198740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:45.329 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:45.327+0000 7f1e80766640 1 --2- 172.21.15.26:0/3747852682 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1e7810c110 0x7f1e7819cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:45.329 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:45.327+0000 7f1e80766640 1 --2- 172.21.15.26:0/3747852682 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1e78102d40 0x7f1e78198200 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:45.330 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:45.327+0000 7f1e80766640 1 -- 172.21.15.26:0/3747852682 >> 172.21.15.26:0/3747852682 conn(0x7f1e780fc6b0 msgr2=0x7f1e780fd290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:45.330 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:45.327+0000 7f1e80766640 1 -- 172.21.15.26:0/3747852682 shutdown_connections 2024-09-11T23:20:45.330 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:45.327+0000 7f1e80766640 1 -- 172.21.15.26:0/3747852682 wait complete. 2024-09-11T23:20:45.658 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:20:46.100 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:45 smithi026 ceph-mon[31172]: pgmap v74: 97 pgs: 97 active+clean; 232 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.2 MiB/s wr, 151 op/s 2024-09-11T23:20:46.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.191+0000 7f550d3aa640 1 -- 172.21.15.26:0/2935631821 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5508102790 msgr2=0x7f55081067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:46.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.191+0000 7f550d3aa640 1 --2- 172.21.15.26:0/2935631821 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5508102790 0x7f55081067e0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f54f8009900 tx=0x7f54f802a410 comp rx=0 tx=0).stop 2024-09-11T23:20:46.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.192+0000 7f550d3aa640 1 -- 172.21.15.26:0/2935631821 shutdown_connections 2024-09-11T23:20:46.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.192+0000 7f550d3aa640 1 --2- 172.21.15.26:0/2935631821 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5508106f10 0x7f550810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:46.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.192+0000 7f550d3aa640 1 --2- 172.21.15.26:0/2935631821 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5508102790 0x7f55081067e0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:46.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.192+0000 7f550d3aa640 1 --2- 172.21.15.26:0/2935631821 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5508101de0 0x7f55081021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:46.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.192+0000 7f550d3aa640 1 -- 172.21.15.26:0/2935631821 >> 172.21.15.26:0/2935631821 conn(0x7f55080fd8c0 msgr2=0x7f55080ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:46.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.192+0000 7f550d3aa640 1 -- 172.21.15.26:0/2935631821 shutdown_connections 2024-09-11T23:20:46.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.192+0000 7f550d3aa640 1 -- 172.21.15.26:0/2935631821 wait complete. 2024-09-11T23:20:46.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.192+0000 7f550d3aa640 1 Processor -- start 2024-09-11T23:20:46.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.192+0000 7f550d3aa640 1 -- start start 2024-09-11T23:20:46.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.193+0000 7f550d3aa640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5508101de0 0x7f550819a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:46.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.193+0000 7f550d3aa640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5508102790 0x7f550819ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:46.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.193+0000 7f550d3aa640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5508106f10 0x7f550819ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:46.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.193+0000 7f550d3aa640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f5508111f70 con 0x7f5508106f10 2024-09-11T23:20:46.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.193+0000 7f550d3aa640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f5508111df0 con 0x7f5508101de0 2024-09-11T23:20:46.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.193+0000 7f550d3aa640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f55081120f0 con 0x7f5508102790 2024-09-11T23:20:46.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.193+0000 7f55077fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5508106f10 0x7f550819ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:46.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.193+0000 7f55077fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5508106f10 0x7f550819ef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55762/0 (socket says 172.21.15.26:55762) 2024-09-11T23:20:46.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.193+0000 7f55077fe640 1 -- 172.21.15.26:0/4097086420 learned_addr learned my addr 172.21.15.26:0/4097086420 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:20:46.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.193+0000 7f55067fc640 1 --2- 172.21.15.26:0/4097086420 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5508102790 0x7f550819ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:46.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.193+0000 7f5506ffd640 1 --2- 172.21.15.26:0/4097086420 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5508101de0 0x7f550819a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:46.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.193+0000 7f55077fe640 1 -- 172.21.15.26:0/4097086420 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5508102790 msgr2=0x7f550819ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:46.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.193+0000 7f55077fe640 1 --2- 172.21.15.26:0/4097086420 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5508102790 0x7f550819ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:46.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.193+0000 7f55077fe640 1 -- 172.21.15.26:0/4097086420 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5508101de0 msgr2=0x7f550819a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:46.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.193+0000 7f55077fe640 1 --2- 172.21.15.26:0/4097086420 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5508101de0 0x7f550819a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:46.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.193+0000 7f55077fe640 1 -- 172.21.15.26:0/4097086420 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f550819f5f0 con 0x7f5508106f10 2024-09-11T23:20:46.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.193+0000 7f55077fe640 1 --2- 172.21.15.26:0/4097086420 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5508106f10 0x7f550819ef10 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f550000eb10 tx=0x7f550000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:46.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.193+0000 7f54fffff640 1 -- 172.21.15.26:0/4097086420 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5500004b20 con 0x7f5508106f10 2024-09-11T23:20:46.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.193+0000 7f550d3aa640 1 -- 172.21.15.26:0/4097086420 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f550819f8e0 con 0x7f5508106f10 2024-09-11T23:20:46.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.193+0000 7f54fffff640 1 -- 172.21.15.26:0/4097086420 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5500004cc0 con 0x7f5508106f10 2024-09-11T23:20:46.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.194+0000 7f54fffff640 1 -- 172.21.15.26:0/4097086420 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5500010760 con 0x7f5508106f10 2024-09-11T23:20:46.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.194+0000 7f550d3aa640 1 -- 172.21.15.26:0/4097086420 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55081a71c0 con 0x7f5508106f10 2024-09-11T23:20:46.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.196+0000 7f54fffff640 1 -- 172.21.15.26:0/4097086420 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f5500005020 con 0x7f5508106f10 2024-09-11T23:20:46.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.196+0000 7f550f783740 1 Processor -- start 2024-09-11T23:20:46.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.196+0000 7f550f783740 1 -- start start 2024-09-11T23:20:46.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.197+0000 7f550f783740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556b2f837b40 0x556b2f836090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:46.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.197+0000 7f550f783740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556b2f8365d0 0x556b2f836a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:46.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.197+0000 7f550f783740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556b2f841370 0x556b2f843760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:46.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.197+0000 7f550f783740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x556b2f79c2f0 con 0x556b2f837b40 2024-09-11T23:20:46.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.197+0000 7f550f783740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x556b2f743cf0 con 0x556b2f8365d0 2024-09-11T23:20:46.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.197+0000 7f550f783740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x556b2f7982b0 con 0x556b2f841370 2024-09-11T23:20:46.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.197+0000 7f55067fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556b2f837b40 0x556b2f836090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:46.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.197+0000 7f54fffff640 1 --2- 172.21.15.26:0/4097086420 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f54f40a2cf0 0x7f54f40a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:46.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.197+0000 7f55067fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556b2f837b40 0x556b2f836090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55778/0 (socket says 172.21.15.26:55778) 2024-09-11T23:20:46.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.197+0000 7f55067fc640 1 -- 172.21.15.26:0/1356702694 learned_addr learned my addr 172.21.15.26:0/1356702694 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:20:46.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.197+0000 7f5506ffd640 1 --2- 172.21.15.26:0/1356702694 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556b2f8365d0 0x556b2f836a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:46.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.197+0000 7f55067fc640 1 --2- 172.21.15.26:0/1356702694 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556b2f841370 0x556b2f843760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:46.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.197+0000 7f55077fe640 1 --2- 172.21.15.26:0/4097086420 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f54f40a2cf0 0x7f54f40a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:46.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.197+0000 7f5506ffd640 1 -- 172.21.15.26:0/1356702694 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556b2f841370 msgr2=0x556b2f843760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:20:46.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.197+0000 7f5506ffd640 1 --2- 172.21.15.26:0/1356702694 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556b2f841370 0x556b2f843760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:46.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.197+0000 7f5506ffd640 1 -- 172.21.15.26:0/1356702694 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556b2f837b40 msgr2=0x556b2f836090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:20:46.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.197+0000 7f5506ffd640 1 --2- 172.21.15.26:0/1356702694 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556b2f837b40 0x556b2f836090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:46.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.197+0000 7f5506ffd640 1 -- 172.21.15.26:0/1356702694 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b2f7283a0 con 0x556b2f8365d0 2024-09-11T23:20:46.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.197+0000 7f55067fc640 1 --2- 172.21.15.26:0/1356702694 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556b2f841370 0x556b2f843760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:20:46.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.197+0000 7f54fffff640 1 -- 172.21.15.26:0/4097086420 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f55000cfbb0 con 0x7f5508106f10 2024-09-11T23:20:46.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.198+0000 7f5506ffd640 1 --2- 172.21.15.26:0/1356702694 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556b2f8365d0 0x556b2f836a10 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f54f0009b70 tx=0x7f54f00117b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d996a64d7ca92a95 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:46.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.198+0000 7f55077fe640 1 --2- 172.21.15.26:0/4097086420 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f54f40a2cf0 0x7f54f40a51b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f550006f0c0 tx=0x7f550006f0f0 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:46.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.198+0000 7f54fd7fa640 1 -- 172.21.15.26:0/1356702694 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54f0009e60 con 0x556b2f8365d0 2024-09-11T23:20:46.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.198+0000 7f54fd7fa640 1 -- 172.21.15.26:0/1356702694 <== mon.2 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54f0012930 con 0x556b2f8365d0 2024-09-11T23:20:46.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.198+0000 7f54fd7fa640 1 -- 172.21.15.26:0/1356702694 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54f000fad0 con 0x556b2f8365d0 2024-09-11T23:20:46.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.198+0000 7f550f783740 1 -- 172.21.15.26:0/1356702694 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556b2f8365d0 msgr2=0x556b2f836a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:46.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.198+0000 7f550f783740 1 --2- 172.21.15.26:0/1356702694 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556b2f8365d0 0x556b2f836a10 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f54f0009b70 tx=0x7f54f00117b0 comp rx=0 tx=0).stop 2024-09-11T23:20:46.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.199+0000 7f550f783740 1 -- 172.21.15.26:0/1356702694 shutdown_connections 2024-09-11T23:20:46.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.199+0000 7f550f783740 1 --2- 172.21.15.26:0/1356702694 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556b2f8365d0 0x556b2f836a10 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:46.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.199+0000 7f550f783740 1 --2- 172.21.15.26:0/1356702694 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556b2f841370 0x556b2f843760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:46.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.199+0000 7f550f783740 1 --2- 172.21.15.26:0/1356702694 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556b2f837b40 0x556b2f836090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:46.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.199+0000 7f550f783740 1 -- 172.21.15.26:0/1356702694 >> 172.21.15.26:0/1356702694 conn(0x556b2f7ff310 msgr2=0x556b2f832c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:46.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.199+0000 7f550f783740 1 -- 172.21.15.26:0/1356702694 shutdown_connections 2024-09-11T23:20:46.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.199+0000 7f550f783740 1 -- 172.21.15.26:0/1356702694 wait complete. 2024-09-11T23:20:46.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.199+0000 7f550f783740 1 -- start start 2024-09-11T23:20:46.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.199+0000 7f550f783740 1 Processor -- start 2024-09-11T23:20:46.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.199+0000 7f550f783740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556b2f8365d0 0x556b2f840d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:46.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.199+0000 7f550f783740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556b2f837b40 0x556b2f83dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:46.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.199+0000 7f550f783740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556b2f841370 0x556b2f83e560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:46.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.199+0000 7f550f783740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x556b2f844070 con 0x556b2f837b40 2024-09-11T23:20:46.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.199+0000 7f550f783740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x556b2f757060 con 0x556b2f8365d0 2024-09-11T23:20:46.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.199+0000 7f550f783740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x556b2f8441f0 con 0x556b2f841370 2024-09-11T23:20:46.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.199+0000 7f550f783740 10 client.0 _ll_register_callbacks cb 0x556b2f4aa8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:20:46.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.199+0000 7f55067fc640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556b2f841370 0x556b2f83e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:46.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.199+0000 7f55067fc640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556b2f841370 0x556b2f83e560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:50658/0 (socket says 172.21.15.26:50658) 2024-09-11T23:20:46.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.199+0000 7f55067fc640 1 -- 172.21.15.26:0/3531032344 learned_addr learned my addr 172.21.15.26:0/3531032344 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:20:46.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.200+0000 7f55067fc640 1 --2- 172.21.15.26:0/3531032344 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556b2f8365d0 0x556b2f840d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:46.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.200+0000 7f5506ffd640 1 --2- 172.21.15.26:0/3531032344 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556b2f837b40 0x556b2f83dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:46.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.200+0000 7f55067fc640 1 -- 172.21.15.26:0/3531032344 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556b2f8365d0 msgr2=0x556b2f840d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:20:46.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.200+0000 7f55067fc640 1 --2- 172.21.15.26:0/3531032344 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556b2f8365d0 0x556b2f840d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:46.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.200+0000 7f55067fc640 1 -- 172.21.15.26:0/3531032344 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556b2f837b40 msgr2=0x556b2f83dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:20:46.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.200+0000 7f55067fc640 1 --2- 172.21.15.26:0/3531032344 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556b2f837b40 0x556b2f83dfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:46.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.200+0000 7f55067fc640 1 -- 172.21.15.26:0/3531032344 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b2f50d3c0 con 0x556b2f841370 2024-09-11T23:20:46.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.200+0000 7f55067fc640 1 --2- 172.21.15.26:0/3531032344 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556b2f841370 0x556b2f83e560 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f54f8006010 tx=0x7f54f80058d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4fe1e015acf71f40 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:46.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.200+0000 7f54ee7fc640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-09-11T23:20:46.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.200+0000 7f54ee7fc640 1 -- 172.21.15.26:0/3531032344 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54f8035040 con 0x556b2f841370 2024-09-11T23:20:46.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.200+0000 7f550f783740 1 -- 172.21.15.26:0/3531032344 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54f0012930 con 0x556b2f841370 2024-09-11T23:20:46.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.200+0000 7f54ee7fc640 1 -- 172.21.15.26:0/3531032344 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f54f8004320 con 0x556b2f841370 2024-09-11T23:20:46.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.200+0000 7f54ee7fc640 1 -- 172.21.15.26:0/3531032344 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f54f8005020 con 0x556b2f841370 2024-09-11T23:20:46.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.200+0000 7f550f783740 10 client.25723 fetch_fsmap learned FSMap version 33 2024-09-11T23:20:46.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.201+0000 7f550f783740 1 -- 172.21.15.26:0/3531032344 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x556b2f84fac0 con 0x556b2f841370 2024-09-11T23:20:46.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.201+0000 7f54ee7fc640 1 -- 172.21.15.26:0/3531032344 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54f0012930 con 0x556b2f841370 2024-09-11T23:20:46.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.201+0000 7f54ee7fc640 1 -- 172.21.15.26:0/3531032344 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 33) ==== 8394+0+0 (secure 0 0 0) 0x7f54f8004750 con 0x556b2f841370 2024-09-11T23:20:46.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.201+0000 7f550f783740 10 client.25723 fetch_fsmap finished waiting for FSMap version 33 2024-09-11T23:20:46.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.201+0000 7f550f783740 10 client.25723 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-09-11T23:20:46.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.201+0000 7f550f783740 20 client.25723 populate_metadata read hostname 'smithi026' 2024-09-11T23:20:46.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.201+0000 7f550f783740 1 --2- 172.21.15.26:0/3531032344 >> [v2:172.21.15.76:6834/393308499,v1:172.21.15.76:6835/393308499] conn(0x556b2f850ed0 0x556b2f8532c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:46.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.201+0000 7f550f783740 4 client.25723 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:20:46.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.201+0000 7f550f783740 1 -- 172.21.15.26:0/3531032344 --> [v2:172.21.15.76:6834/393308499,v1:172.21.15.76:6835/393308499] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556b2f7a9860 con 0x556b2f850ed0 2024-09-11T23:20:46.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.201+0000 7f5506ffd640 1 --2- 172.21.15.26:0/3531032344 >> [v2:172.21.15.76:6834/393308499,v1:172.21.15.76:6835/393308499] conn(0x556b2f850ed0 0x556b2f8532c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:46.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.202+0000 7f5506ffd640 1 --2- 172.21.15.26:0/3531032344 >> [v2:172.21.15.76:6834/393308499,v1:172.21.15.76:6835/393308499] conn(0x556b2f850ed0 0x556b2f8532c0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a2e7ccc9b4cf0309 server_cookie=5bfbb247e2fdc5c3 in_seq=0 out_seq=0 2024-09-11T23:20:46.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.202+0000 7f54ee7fc640 10 client.25723 ms_handle_connect on v2:172.21.15.76:6834/393308499 2024-09-11T23:20:46.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.202+0000 7f54ee7fc640 1 -- 172.21.15.26:0/3531032344 <== mds.4 v2:172.21.15.76:6834/393308499 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556b2f7a9860 con 0x556b2f850ed0 2024-09-11T23:20:46.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.202+0000 7f54ee7fc640 10 client.25723 handle_command_reply: tid=0 2024-09-11T23:20:46.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.202+0000 7f550f783740 1 client.25723 shutdown 2024-09-11T23:20:46.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.202+0000 7f550f783740 1 -- 172.21.15.26:0/3531032344 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556b2f841370 msgr2=0x556b2f83e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:46.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.202+0000 7f550f783740 1 --2- 172.21.15.26:0/3531032344 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556b2f841370 0x556b2f83e560 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f54f8006010 tx=0x7f54f80058d0 comp rx=0 tx=0).stop 2024-09-11T23:20:46.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.202+0000 7f550f783740 1 -- 172.21.15.26:0/3531032344 shutdown_connections 2024-09-11T23:20:46.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.202+0000 7f550f783740 1 --2- 172.21.15.26:0/3531032344 >> [v2:172.21.15.76:6834/393308499,v1:172.21.15.76:6835/393308499] conn(0x556b2f850ed0 0x556b2f8532c0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:46.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.202+0000 7f550f783740 1 --2- 172.21.15.26:0/3531032344 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556b2f841370 0x556b2f83e560 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:46.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.202+0000 7f550f783740 1 --2- 172.21.15.26:0/3531032344 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556b2f837b40 0x556b2f83dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:46.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.202+0000 7f550f783740 1 --2- 172.21.15.26:0/3531032344 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556b2f8365d0 0x556b2f840d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:46.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.202+0000 7f54ee7fc640 0 client.25723 ms_handle_reset on v2:172.21.15.76:6834/393308499 2024-09-11T23:20:46.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.202+0000 7f550f783740 1 -- 172.21.15.26:0/3531032344 >> 172.21.15.26:0/3531032344 conn(0x556b2f7ff310 msgr2=0x556b2f838930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:46.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.203+0000 7f550f783740 1 -- 172.21.15.26:0/3531032344 shutdown_connections 2024-09-11T23:20:46.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.203+0000 7f550f783740 1 -- 172.21.15.26:0/3531032344 wait complete. 2024-09-11T23:20:46.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.203+0000 7f550f783740 20 client.25723 trim_cache size 0 max 16384 2024-09-11T23:20:46.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.216+0000 7f550f783740 1 Processor -- start 2024-09-11T23:20:46.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.216+0000 7f550f783740 1 -- start start 2024-09-11T23:20:46.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.216+0000 7f550f783740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556b2f8365d0 0x556b2f832e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:46.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.216+0000 7f550f783740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556b2f837b40 0x556b2f84d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:46.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.216+0000 7f550f783740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556b2f841370 0x556b2f83d880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:46.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.216+0000 7f550f783740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f55081afbd0 con 0x556b2f837b40 2024-09-11T23:20:46.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.216+0000 7f550f783740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f55081b1890 con 0x556b2f841370 2024-09-11T23:20:46.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.216+0000 7f550f783740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f55081b02a0 con 0x556b2f8365d0 2024-09-11T23:20:46.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.217+0000 7f5506ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556b2f837b40 0x556b2f84d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:46.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.217+0000 7f5506ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556b2f837b40 0x556b2f84d310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55798/0 (socket says 172.21.15.26:55798) 2024-09-11T23:20:46.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.217+0000 7f5506ffd640 1 -- 172.21.15.26:0/1573438791 learned_addr learned my addr 172.21.15.26:0/1573438791 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:20:46.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.217+0000 7f55067fc640 1 --2- 172.21.15.26:0/1573438791 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556b2f8365d0 0x556b2f832e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:46.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.217+0000 7f55067fc640 1 --2- 172.21.15.26:0/1573438791 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556b2f841370 0x556b2f83d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:46.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.217+0000 7f5506ffd640 1 -- 172.21.15.26:0/1573438791 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556b2f8365d0 msgr2=0x556b2f832e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:20:46.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.217+0000 7f5506ffd640 1 --2- 172.21.15.26:0/1573438791 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556b2f8365d0 0x556b2f832e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:46.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.217+0000 7f5506ffd640 1 -- 172.21.15.26:0/1573438791 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556b2f841370 msgr2=0x556b2f83d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:20:46.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.217+0000 7f5506ffd640 1 --2- 172.21.15.26:0/1573438791 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556b2f841370 0x556b2f83d880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:46.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.217+0000 7f5506ffd640 1 -- 172.21.15.26:0/1573438791 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b2f83cf70 con 0x556b2f837b40 2024-09-11T23:20:46.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.217+0000 7f5506ffd640 1 --2- 172.21.15.26:0/1573438791 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556b2f837b40 0x556b2f84d310 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x556b2f833ac0 tx=0x7f54f00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53fb6da56fcabf15 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:46.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.217+0000 7f54fcff9640 1 -- 172.21.15.26:0/1573438791 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54f0009ea0 con 0x556b2f837b40 2024-09-11T23:20:46.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.218+0000 7f54fcff9640 1 -- 172.21.15.26:0/1573438791 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54f0029070 con 0x556b2f837b40 2024-09-11T23:20:46.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.218+0000 7f54fcff9640 1 -- 172.21.15.26:0/1573438791 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54f000fe60 con 0x556b2f837b40 2024-09-11T23:20:46.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.218+0000 7f550f783740 1 -- 172.21.15.26:0/1573438791 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556b2f837b40 msgr2=0x556b2f84d310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:46.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.218+0000 7f550f783740 1 --2- 172.21.15.26:0/1573438791 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556b2f837b40 0x556b2f84d310 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x556b2f833ac0 tx=0x7f54f00057f0 comp rx=0 tx=0).stop 2024-09-11T23:20:46.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.218+0000 7f550f783740 1 -- 172.21.15.26:0/1573438791 shutdown_connections 2024-09-11T23:20:46.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.218+0000 7f550f783740 1 --2- 172.21.15.26:0/1573438791 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556b2f841370 0x556b2f83d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:46.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.218+0000 7f550f783740 1 --2- 172.21.15.26:0/1573438791 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556b2f837b40 0x556b2f84d310 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:46.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.218+0000 7f550f783740 1 --2- 172.21.15.26:0/1573438791 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556b2f8365d0 0x556b2f832e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:46.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.218+0000 7f550f783740 1 -- 172.21.15.26:0/1573438791 >> 172.21.15.26:0/1573438791 conn(0x556b2f7ff310 msgr2=0x556b2f832700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:46.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.218+0000 7f550f783740 1 -- 172.21.15.26:0/1573438791 shutdown_connections 2024-09-11T23:20:46.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.218+0000 7f550f783740 1 -- 172.21.15.26:0/1573438791 wait complete. 2024-09-11T23:20:46.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.219+0000 7f550f783740 1 -- start start 2024-09-11T23:20:46.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.219+0000 7f550f783740 1 Processor -- start 2024-09-11T23:20:46.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.219+0000 7f550f783740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556b2f8365d0 0x556b2f849970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:46.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.219+0000 7f550f783740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556b2f837b40 0x556b2f83aae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:46.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.219+0000 7f550f783740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556b2f841370 0x556b2f83b020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:46.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.219+0000 7f550f783740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f55081b1100 con 0x556b2f841370 2024-09-11T23:20:46.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.219+0000 7f550f783740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f55081b0970 con 0x556b2f8365d0 2024-09-11T23:20:46.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.219+0000 7f550f783740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f55081adbb0 con 0x556b2f837b40 2024-09-11T23:20:46.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.219+0000 7f550f783740 10 client.0 _ll_register_callbacks cb 0x7f54f8009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:20:46.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.219+0000 7f55067fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556b2f841370 0x556b2f83b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:46.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.219+0000 7f5506ffd640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556b2f837b40 0x556b2f83aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:46.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.219+0000 7f55067fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556b2f841370 0x556b2f83b020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55806/0 (socket says 172.21.15.26:55806) 2024-09-11T23:20:46.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.219+0000 7f5506ffd640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556b2f837b40 0x556b2f83aae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:50674/0 (socket says 172.21.15.26:50674) 2024-09-11T23:20:46.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.219+0000 7f55067fc640 1 -- 172.21.15.26:0/2826217346 learned_addr learned my addr 172.21.15.26:0/2826217346 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:20:46.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.219+0000 7f55067fc640 1 --2- 172.21.15.26:0/2826217346 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556b2f8365d0 0x556b2f849970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:46.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.220+0000 7f5506ffd640 1 -- 172.21.15.26:0/2826217346 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556b2f8365d0 msgr2=0x556b2f849970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:20:46.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.220+0000 7f5506ffd640 1 --2- 172.21.15.26:0/2826217346 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556b2f8365d0 0x556b2f849970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:46.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.220+0000 7f5506ffd640 1 -- 172.21.15.26:0/2826217346 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556b2f841370 msgr2=0x556b2f83b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:20:46.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.220+0000 7f5506ffd640 1 --2- 172.21.15.26:0/2826217346 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556b2f841370 0x556b2f83b020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:46.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.220+0000 7f5506ffd640 1 -- 172.21.15.26:0/2826217346 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b2f83b5e0 con 0x556b2f837b40 2024-09-11T23:20:46.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.220+0000 7f55067fc640 1 --2- 172.21.15.26:0/2826217346 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556b2f841370 0x556b2f83b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:20:46.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.220+0000 7f5506ffd640 1 --2- 172.21.15.26:0/2826217346 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556b2f837b40 0x556b2f83aae0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f54f0000c00 tx=0x7f54f0011d60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ff2a63b6f34b4fc5 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:46.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.220+0000 7f54ee7fc640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-09-11T23:20:46.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.220+0000 7f54ee7fc640 1 -- 172.21.15.26:0/2826217346 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54f000fae0 con 0x556b2f837b40 2024-09-11T23:20:46.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.220+0000 7f550f783740 1 -- 172.21.15.26:0/2826217346 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54f0029070 con 0x556b2f837b40 2024-09-11T23:20:46.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.220+0000 7f54ee7fc640 1 -- 172.21.15.26:0/2826217346 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f54f000fc80 con 0x556b2f837b40 2024-09-11T23:20:46.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.220+0000 7f54ee7fc640 1 -- 172.21.15.26:0/2826217346 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f54f000fe80 con 0x556b2f837b40 2024-09-11T23:20:46.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.220+0000 7f550f783740 10 client.25729 fetch_fsmap learned FSMap version 33 2024-09-11T23:20:46.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.220+0000 7f550f783740 1 -- 172.21.15.26:0/2826217346 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x556b2f8bb910 con 0x556b2f837b40 2024-09-11T23:20:46.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.221+0000 7f54ee7fc640 1 -- 172.21.15.26:0/2826217346 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54f0029070 con 0x556b2f837b40 2024-09-11T23:20:46.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.221+0000 7f54ee7fc640 1 -- 172.21.15.26:0/2826217346 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 33) ==== 8394+0+0 (secure 0 0 0) 0x7f54f0003b20 con 0x556b2f837b40 2024-09-11T23:20:46.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.221+0000 7f550f783740 10 client.25729 fetch_fsmap finished waiting for FSMap version 33 2024-09-11T23:20:46.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.221+0000 7f550f783740 10 client.25729 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-09-11T23:20:46.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.221+0000 7f550f783740 20 client.25729 populate_metadata read hostname 'smithi026' 2024-09-11T23:20:46.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.221+0000 7f550f783740 1 --2- 172.21.15.26:0/2826217346 >> [v2:172.21.15.76:6834/393308499,v1:172.21.15.76:6835/393308499] conn(0x556b2f8bbe40 0x556b2f8be230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:46.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.221+0000 7f550f783740 4 client.25729 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:20:46.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.221+0000 7f550f783740 1 -- 172.21.15.26:0/2826217346 --> [v2:172.21.15.76:6834/393308499,v1:172.21.15.76:6835/393308499] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556b2f7a3d70 con 0x556b2f8bbe40 2024-09-11T23:20:46.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.221+0000 7f55067fc640 1 --2- 172.21.15.26:0/2826217346 >> [v2:172.21.15.76:6834/393308499,v1:172.21.15.76:6835/393308499] conn(0x556b2f8bbe40 0x556b2f8be230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:46.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.221+0000 7f55067fc640 1 --2- 172.21.15.26:0/2826217346 >> [v2:172.21.15.76:6834/393308499,v1:172.21.15.76:6835/393308499] conn(0x556b2f8bbe40 0x556b2f8be230 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c9dde68a6720e317 server_cookie=4c8a9c7e8e274d2d in_seq=0 out_seq=0 2024-09-11T23:20:46.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.221+0000 7f54ee7fc640 10 client.25729 ms_handle_connect on v2:172.21.15.76:6834/393308499 2024-09-11T23:20:46.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.222+0000 7f54ee7fc640 1 -- 172.21.15.26:0/2826217346 <== mds.4 v2:172.21.15.76:6834/393308499 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556b2f7a3d70 con 0x556b2f8bbe40 2024-09-11T23:20:46.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.222+0000 7f54ee7fc640 10 client.25729 handle_command_reply: tid=0 2024-09-11T23:20:46.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.222+0000 7f550f783740 1 client.25729 shutdown 2024-09-11T23:20:46.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.222+0000 7f550f783740 1 -- 172.21.15.26:0/2826217346 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556b2f837b40 msgr2=0x556b2f83aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:46.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.222+0000 7f550f783740 1 --2- 172.21.15.26:0/2826217346 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556b2f837b40 0x556b2f83aae0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f54f0000c00 tx=0x7f54f0011d60 comp rx=0 tx=0).stop 2024-09-11T23:20:46.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.222+0000 7f550f783740 1 -- 172.21.15.26:0/2826217346 shutdown_connections 2024-09-11T23:20:46.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.222+0000 7f550f783740 1 --2- 172.21.15.26:0/2826217346 >> [v2:172.21.15.76:6834/393308499,v1:172.21.15.76:6835/393308499] conn(0x556b2f8bbe40 0x556b2f8be230 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:46.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.222+0000 7f550f783740 1 --2- 172.21.15.26:0/2826217346 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556b2f841370 0x556b2f83b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:46.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.222+0000 7f54ee7fc640 0 client.25729 ms_handle_reset on v2:172.21.15.76:6834/393308499 2024-09-11T23:20:46.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.222+0000 7f550f783740 1 --2- 172.21.15.26:0/2826217346 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556b2f837b40 0x556b2f83aae0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:46.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.222+0000 7f550f783740 1 --2- 172.21.15.26:0/2826217346 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556b2f8365d0 0x556b2f849970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:46.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.222+0000 7f550f783740 1 -- 172.21.15.26:0/2826217346 >> 172.21.15.26:0/2826217346 conn(0x556b2f7ff310 msgr2=0x556b2f838930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:46.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.222+0000 7f550f783740 1 -- 172.21.15.26:0/2826217346 shutdown_connections 2024-09-11T23:20:46.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.222+0000 7f550f783740 1 -- 172.21.15.26:0/2826217346 wait complete. 2024-09-11T23:20:46.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.222+0000 7f550f783740 20 client.25729 trim_cache size 0 max 16384 2024-09-11T23:20:46.228 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2024-09-11T23:20:46.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.222+0000 7f550d3aa640 1 -- 172.21.15.26:0/4097086420 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f54f40a2cf0 msgr2=0x7f54f40a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:46.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.222+0000 7f550d3aa640 1 --2- 172.21.15.26:0/4097086420 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f54f40a2cf0 0x7f54f40a51b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f550006f0c0 tx=0x7f550006f0f0 comp rx=0 tx=0).stop 2024-09-11T23:20:46.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.222+0000 7f550d3aa640 1 -- 172.21.15.26:0/4097086420 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5508106f10 msgr2=0x7f550819ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:46.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.222+0000 7f550d3aa640 1 --2- 172.21.15.26:0/4097086420 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5508106f10 0x7f550819ef10 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f550000eb10 tx=0x7f550000c2d0 comp rx=0 tx=0).stop 2024-09-11T23:20:46.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.223+0000 7f550d3aa640 1 -- 172.21.15.26:0/4097086420 shutdown_connections 2024-09-11T23:20:46.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.223+0000 7f550d3aa640 1 --2- 172.21.15.26:0/4097086420 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f54f40a2cf0 0x7f54f40a51b0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:46.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.223+0000 7f550d3aa640 1 --2- 172.21.15.26:0/4097086420 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5508106f10 0x7f550819ef10 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:46.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.223+0000 7f550d3aa640 1 --2- 172.21.15.26:0/4097086420 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5508102790 0x7f550819ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:46.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.223+0000 7f550d3aa640 1 --2- 172.21.15.26:0/4097086420 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5508101de0 0x7f550819a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:46.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.223+0000 7f550d3aa640 1 -- 172.21.15.26:0/4097086420 >> 172.21.15.26:0/4097086420 conn(0x7f55080fd8c0 msgr2=0x7f55080feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:46.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.223+0000 7f550d3aa640 1 -- 172.21.15.26:0/4097086420 shutdown_connections 2024-09-11T23:20:46.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:46.223+0000 7f550d3aa640 1 -- 172.21.15.26:0/4097086420 wait complete. 2024-09-11T23:20:46.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:45 smithi136 ceph-mon[35121]: pgmap v74: 97 pgs: 97 active+clean; 232 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.2 MiB/s wr, 151 op/s 2024-09-11T23:20:46.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:45 smithi076 ceph-mon[35296]: pgmap v74: 97 pgs: 97 active+clean; 232 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.2 MiB/s wr, 151 op/s 2024-09-11T23:20:47.105 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:46 smithi026 ceph-mon[31172]: pgmap v75: 97 pgs: 97 active+clean; 232 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 466 KiB/s wr, 86 op/s 2024-09-11T23:20:47.106 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-11T23:20:47.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:46 smithi136 ceph-mon[35121]: pgmap v75: 97 pgs: 97 active+clean; 232 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 466 KiB/s wr, 86 op/s 2024-09-11T23:20:47.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:46 smithi076 ceph-mon[35296]: pgmap v75: 97 pgs: 97 active+clean; 232 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 466 KiB/s wr, 86 op/s 2024-09-11T23:20:47.460 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:20:48.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.003+0000 7f06f27a9640 1 -- 172.21.15.26:0/3845437340 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f06ec106f10 msgr2=0x7f06ec10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:48.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.003+0000 7f06f27a9640 1 --2- 172.21.15.26:0/3845437340 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f06ec106f10 0x7f06ec10ea90 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f06e4009ca0 tx=0x7f06e402a8c0 comp rx=0 tx=0).stop 2024-09-11T23:20:48.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.003+0000 7f06f27a9640 1 -- 172.21.15.26:0/3845437340 shutdown_connections 2024-09-11T23:20:48.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.003+0000 7f06f27a9640 1 --2- 172.21.15.26:0/3845437340 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f06ec102790 0x7f06ec1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:48.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.003+0000 7f06f27a9640 1 --2- 172.21.15.26:0/3845437340 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f06ec106f10 0x7f06ec10ea90 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:48.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.003+0000 7f06f27a9640 1 --2- 172.21.15.26:0/3845437340 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f06ec101de0 0x7f06ec1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:48.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.003+0000 7f06f27a9640 1 -- 172.21.15.26:0/3845437340 >> 172.21.15.26:0/3845437340 conn(0x7f06ec0fd8c0 msgr2=0x7f06ec0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:48.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.003+0000 7f06f27a9640 1 -- 172.21.15.26:0/3845437340 shutdown_connections 2024-09-11T23:20:48.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.003+0000 7f06f27a9640 1 -- 172.21.15.26:0/3845437340 wait complete. 2024-09-11T23:20:48.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.004+0000 7f06f27a9640 1 Processor -- start 2024-09-11T23:20:48.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.004+0000 7f06f27a9640 1 -- start start 2024-09-11T23:20:48.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.004+0000 7f06f27a9640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f06ec101de0 0x7f06ec0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:48.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.004+0000 7f06f27a9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f06ec102790 0x7f06ec074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:48.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.004+0000 7f06f27a9640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f06ec106f10 0x7f06ec074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:48.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.004+0000 7f06f27a9640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f06ec111f70 con 0x7f06ec102790 2024-09-11T23:20:48.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.004+0000 7f06f27a9640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f06ec111df0 con 0x7f06ec101de0 2024-09-11T23:20:48.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.004+0000 7f06f27a9640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f06ec1120f0 con 0x7f06ec106f10 2024-09-11T23:20:48.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.004+0000 7f06ebfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f06ec101de0 0x7f06ec0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:48.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.004+0000 7f06ebfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f06ec101de0 0x7f06ec0744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:60968/0 (socket says 172.21.15.26:60968) 2024-09-11T23:20:48.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.004+0000 7f06ebfff640 1 -- 172.21.15.26:0/4256901606 learned_addr learned my addr 172.21.15.26:0/4256901606 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:20:48.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.004+0000 7f06eb7fe640 1 --2- 172.21.15.26:0/4256901606 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f06ec102790 0x7f06ec074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:48.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.004+0000 7f06f0d46640 1 --2- 172.21.15.26:0/4256901606 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f06ec106f10 0x7f06ec074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:48.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.005+0000 7f06ebfff640 1 -- 172.21.15.26:0/4256901606 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f06ec106f10 msgr2=0x7f06ec074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:48.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.005+0000 7f06ebfff640 1 --2- 172.21.15.26:0/4256901606 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f06ec106f10 0x7f06ec074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:48.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.005+0000 7f06ebfff640 1 -- 172.21.15.26:0/4256901606 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f06ec102790 msgr2=0x7f06ec074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:48.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.005+0000 7f06ebfff640 1 --2- 172.21.15.26:0/4256901606 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f06ec102790 0x7f06ec074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:48.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.005+0000 7f06ebfff640 1 -- 172.21.15.26:0/4256901606 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06ec1a68b0 con 0x7f06ec101de0 2024-09-11T23:20:48.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.005+0000 7f06ebfff640 1 --2- 172.21.15.26:0/4256901606 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f06ec101de0 0x7f06ec0744f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f06dc00e9f0 tx=0x7f06dc00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:48.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.005+0000 7f06e97fa640 1 -- 172.21.15.26:0/4256901606 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06dc00cd80 con 0x7f06ec101de0 2024-09-11T23:20:48.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.005+0000 7f06f27a9640 1 -- 172.21.15.26:0/4256901606 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06ec1a6ba0 con 0x7f06ec101de0 2024-09-11T23:20:48.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.005+0000 7f06e97fa640 1 -- 172.21.15.26:0/4256901606 <== mon.2 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06dc010910 con 0x7f06ec101de0 2024-09-11T23:20:48.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.005+0000 7f06e97fa640 1 -- 172.21.15.26:0/4256901606 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06dc0188f0 con 0x7f06ec101de0 2024-09-11T23:20:48.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.005+0000 7f06f27a9640 1 -- 172.21.15.26:0/4256901606 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06ec1a70e0 con 0x7f06ec101de0 2024-09-11T23:20:48.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.008+0000 7f06e97fa640 1 -- 172.21.15.26:0/4256901606 <== mon.2 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f06dc010ac0 con 0x7f06ec101de0 2024-09-11T23:20:48.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.009+0000 7f06f4b82740 1 Processor -- start 2024-09-11T23:20:48.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.009+0000 7f06f4b82740 1 -- start start 2024-09-11T23:20:48.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.009+0000 7f06f4b82740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f89a61fb40 0x55f89a61e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:48.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.009+0000 7f06f4b82740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f89a61e5d0 0x55f89a61ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:48.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.009+0000 7f06f4b82740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f89a629370 0x55f89a62b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:48.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.009+0000 7f06f4b82740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55f89a5842f0 con 0x55f89a629370 2024-09-11T23:20:48.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.009+0000 7f06f4b82740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55f89a52bcf0 con 0x55f89a61e5d0 2024-09-11T23:20:48.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.009+0000 7f06f4b82740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55f89a5802b0 con 0x55f89a61fb40 2024-09-11T23:20:48.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.009+0000 7f06f0d46640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f89a61fb40 0x55f89a61e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:48.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.009+0000 7f06f0d46640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f89a61fb40 0x55f89a61e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:50710/0 (socket says 172.21.15.26:50710) 2024-09-11T23:20:48.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.009+0000 7f06f0d46640 1 -- 172.21.15.26:0/217164457 learned_addr learned my addr 172.21.15.26:0/217164457 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:20:48.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.009+0000 7f06ebfff640 1 --2- 172.21.15.26:0/217164457 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f89a61e5d0 0x55f89a61ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:48.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.010+0000 7f06f0d46640 1 -- 172.21.15.26:0/217164457 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f89a61e5d0 msgr2=0x55f89a61ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:20:48.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.010+0000 7f06f0d46640 1 --2- 172.21.15.26:0/217164457 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f89a61e5d0 0x55f89a61ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:48.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.010+0000 7f06f0d46640 1 -- 172.21.15.26:0/217164457 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f89a629370 msgr2=0x55f89a62b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:20:48.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.010+0000 7f06e97fa640 1 --2- 172.21.15.26:0/4256901606 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f06d40a2cf0 0x7f06d40a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:48.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.010+0000 7f06ebfff640 1 --2- 172.21.15.26:0/4256901606 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f06d40a2cf0 0x7f06d40a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:48.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.009+0000 7f06eb7fe640 1 --2- 172.21.15.26:0/217164457 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f89a629370 0x55f89a62b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:48.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.010+0000 7f06f0d46640 1 --2- 172.21.15.26:0/217164457 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f89a629370 0x55f89a62b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:48.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.010+0000 7f06f0d46640 1 -- 172.21.15.26:0/217164457 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f89a5103a0 con 0x55f89a61fb40 2024-09-11T23:20:48.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.010+0000 7f06eb7fe640 1 --2- 172.21.15.26:0/217164457 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f89a629370 0x55f89a62b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:20:48.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.010+0000 7f06e97fa640 1 -- 172.21.15.26:0/4256901606 <== mon.2 v2:172.21.15.76:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f06dc0cf7f0 con 0x7f06ec101de0 2024-09-11T23:20:48.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.010+0000 7f06f0d46640 1 --2- 172.21.15.26:0/217164457 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f89a61fb40 0x55f89a61e090 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f06e4009ca0 tx=0x7f06e4005f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3178bf6296d77004 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:48.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.011+0000 7f06d27fc640 1 -- 172.21.15.26:0/217164457 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06e4031db0 con 0x55f89a61fb40 2024-09-11T23:20:48.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.011+0000 7f06ebfff640 1 --2- 172.21.15.26:0/4256901606 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f06d40a2cf0 0x7f06d40a51b0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f06dc0cd000 tx=0x7f06dc0d5550 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:48.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.011+0000 7f06d27fc640 1 -- 172.21.15.26:0/217164457 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06e4036040 con 0x55f89a61fb40 2024-09-11T23:20:48.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.011+0000 7f06d27fc640 1 -- 172.21.15.26:0/217164457 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06e4004260 con 0x55f89a61fb40 2024-09-11T23:20:48.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.012+0000 7f06f4b82740 1 -- 172.21.15.26:0/217164457 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f89a61fb40 msgr2=0x55f89a61e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:48.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.012+0000 7f06f4b82740 1 --2- 172.21.15.26:0/217164457 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f89a61fb40 0x55f89a61e090 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f06e4009ca0 tx=0x7f06e4005f10 comp rx=0 tx=0).stop 2024-09-11T23:20:48.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.012+0000 7f06f4b82740 1 -- 172.21.15.26:0/217164457 shutdown_connections 2024-09-11T23:20:48.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.012+0000 7f06f4b82740 1 --2- 172.21.15.26:0/217164457 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f89a61e5d0 0x55f89a61ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:48.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.012+0000 7f06f4b82740 1 --2- 172.21.15.26:0/217164457 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f89a629370 0x55f89a62b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:48.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.012+0000 7f06f4b82740 1 --2- 172.21.15.26:0/217164457 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f89a61fb40 0x55f89a61e090 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:48.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.012+0000 7f06f4b82740 1 -- 172.21.15.26:0/217164457 >> 172.21.15.26:0/217164457 conn(0x55f89a5e7310 msgr2=0x55f89a61ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:48.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.012+0000 7f06f4b82740 1 -- 172.21.15.26:0/217164457 shutdown_connections 2024-09-11T23:20:48.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.012+0000 7f06f4b82740 1 -- 172.21.15.26:0/217164457 wait complete. 2024-09-11T23:20:48.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.012+0000 7f06f4b82740 1 -- start start 2024-09-11T23:20:48.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.013+0000 7f06f4b82740 1 Processor -- start 2024-09-11T23:20:48.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.013+0000 7f06f4b82740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f89a61e5d0 0x55f89a628cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:48.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.013+0000 7f06f4b82740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f89a61fb40 0x55f89a625f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:48.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.013+0000 7f06f4b82740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f89a629370 0x55f89a6264a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:48.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.013+0000 7f06f4b82740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55f89a62be30 con 0x55f89a61fb40 2024-09-11T23:20:48.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.013+0000 7f06f4b82740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55f89a53f060 con 0x55f89a61e5d0 2024-09-11T23:20:48.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.013+0000 7f06f4b82740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55f89a62bfb0 con 0x55f89a629370 2024-09-11T23:20:48.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.013+0000 7f06f4b82740 10 client.0 _ll_register_callbacks cb 0x55f89a2928c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:20:48.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.013+0000 7f06eb7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f89a61fb40 0x55f89a625f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:48.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.013+0000 7f06f0d46640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f89a61e5d0 0x55f89a628cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:48.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.013+0000 7f06f0d46640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f89a61e5d0 0x55f89a628cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:60994/0 (socket says 172.21.15.26:60994) 2024-09-11T23:20:48.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.013+0000 7f06f0d46640 1 -- 172.21.15.26:0/1503982751 learned_addr learned my addr 172.21.15.26:0/1503982751 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:20:48.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.013+0000 7f06eb7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f89a61fb40 0x55f89a625f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55854/0 (socket says 172.21.15.26:55854) 2024-09-11T23:20:48.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.013+0000 7f06f0d46640 1 --2- 172.21.15.26:0/1503982751 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f89a629370 0x55f89a6264a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:48.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.014+0000 7f06f0d46640 1 -- 172.21.15.26:0/1503982751 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f89a629370 msgr2=0x55f89a6264a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:20:48.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.014+0000 7f06f0d46640 1 --2- 172.21.15.26:0/1503982751 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f89a629370 0x55f89a6264a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:48.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.014+0000 7f06f0d46640 1 -- 172.21.15.26:0/1503982751 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f89a61fb40 msgr2=0x55f89a625f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:20:48.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.014+0000 7f06f0d46640 1 --2- 172.21.15.26:0/1503982751 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f89a61fb40 0x55f89a625f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:48.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.014+0000 7f06f0d46640 1 -- 172.21.15.26:0/1503982751 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f89a2f53c0 con 0x55f89a61e5d0 2024-09-11T23:20:48.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.014+0000 7f06f0d46640 1 --2- 172.21.15.26:0/1503982751 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f89a61e5d0 0x55f89a628cb0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f06e402f470 tx=0x7f06e4031940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=57caf99e3e69a3f8 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:48.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.014+0000 7f06cb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-11T23:20:48.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.014+0000 7f06cb7fe640 1 -- 172.21.15.26:0/1503982751 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06e405f030 con 0x55f89a61e5d0 2024-09-11T23:20:48.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.014+0000 7f06cb7fe640 1 -- 172.21.15.26:0/1503982751 <== mon.2 v2:172.21.15.76:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f06e402f610 con 0x55f89a61e5d0 2024-09-11T23:20:48.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.014+0000 7f06f4b82740 1 -- 172.21.15.26:0/1503982751 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f06e4036040 con 0x55f89a61e5d0 2024-09-11T23:20:48.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.014+0000 7f06cb7fe640 1 -- 172.21.15.26:0/1503982751 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f06e402f810 con 0x55f89a61e5d0 2024-09-11T23:20:48.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.014+0000 7f06f4b82740 10 client.25688 fetch_fsmap learned FSMap version 33 2024-09-11T23:20:48.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.014+0000 7f06f4b82740 1 -- 172.21.15.26:0/1503982751 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55f89a62be30 con 0x55f89a61e5d0 2024-09-11T23:20:48.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.015+0000 7f06cb7fe640 1 -- 172.21.15.26:0/1503982751 <== mon.2 v2:172.21.15.76:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06e4036040 con 0x55f89a61e5d0 2024-09-11T23:20:48.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.015+0000 7f06cb7fe640 1 -- 172.21.15.26:0/1503982751 <== mon.2 v2:172.21.15.76:3300/0 5 ==== fsmap(e 33) ==== 8394+0+0 (secure 0 0 0) 0x7f06e4030a70 con 0x55f89a61e5d0 2024-09-11T23:20:48.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.015+0000 7f06f4b82740 10 client.25688 fetch_fsmap finished waiting for FSMap version 33 2024-09-11T23:20:48.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.015+0000 7f06f4b82740 10 client.25688 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-11T23:20:48.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.015+0000 7f06f4b82740 20 client.25688 populate_metadata read hostname 'smithi026' 2024-09-11T23:20:48.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.015+0000 7f06f4b82740 1 --2- 172.21.15.26:0/1503982751 >> [v2:172.21.15.136:6836/1633082387,v1:172.21.15.136:6837/1633082387] conn(0x55f89a638920 0x55f89a63ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:48.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.015+0000 7f06f4b82740 4 client.25688 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:20:48.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.015+0000 7f06f4b82740 1 -- 172.21.15.26:0/1503982751 --> [v2:172.21.15.136:6836/1633082387,v1:172.21.15.136:6837/1633082387] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f89a591860 con 0x55f89a638920 2024-09-11T23:20:48.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.015+0000 7f06eb7fe640 1 --2- 172.21.15.26:0/1503982751 >> [v2:172.21.15.136:6836/1633082387,v1:172.21.15.136:6837/1633082387] conn(0x55f89a638920 0x55f89a63ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:48.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.016+0000 7f06eb7fe640 1 --2- 172.21.15.26:0/1503982751 >> [v2:172.21.15.136:6836/1633082387,v1:172.21.15.136:6837/1633082387] conn(0x55f89a638920 0x55f89a63ad10 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=9de55833807046e4 server_cookie=e2ed3e24ff64a09a in_seq=0 out_seq=0 2024-09-11T23:20:48.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.016+0000 7f06cb7fe640 10 client.25688 ms_handle_connect on v2:172.21.15.136:6836/1633082387 2024-09-11T23:20:48.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.016+0000 7f06cb7fe640 1 -- 172.21.15.26:0/1503982751 <== mds.3 v2:172.21.15.136:6836/1633082387 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f89a591860 con 0x55f89a638920 2024-09-11T23:20:48.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.016+0000 7f06cb7fe640 10 client.25688 handle_command_reply: tid=0 2024-09-11T23:20:48.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.016+0000 7f06f4b82740 1 client.25688 shutdown 2024-09-11T23:20:48.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.017+0000 7f06f4b82740 1 -- 172.21.15.26:0/1503982751 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f89a61e5d0 msgr2=0x55f89a628cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:48.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.017+0000 7f06f4b82740 1 --2- 172.21.15.26:0/1503982751 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f89a61e5d0 0x55f89a628cb0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f06e402f470 tx=0x7f06e4031940 comp rx=0 tx=0).stop 2024-09-11T23:20:48.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.017+0000 7f06f4b82740 1 -- 172.21.15.26:0/1503982751 shutdown_connections 2024-09-11T23:20:48.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.017+0000 7f06f4b82740 1 --2- 172.21.15.26:0/1503982751 >> [v2:172.21.15.136:6836/1633082387,v1:172.21.15.136:6837/1633082387] conn(0x55f89a638920 0x55f89a63ad10 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:48.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.017+0000 7f06f4b82740 1 --2- 172.21.15.26:0/1503982751 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f89a629370 0x55f89a6264a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:48.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.017+0000 7f06f4b82740 1 --2- 172.21.15.26:0/1503982751 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f89a61fb40 0x55f89a625f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:48.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.017+0000 7f06f4b82740 1 --2- 172.21.15.26:0/1503982751 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f89a61e5d0 0x55f89a628cb0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:48.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.017+0000 7f06f4b82740 1 -- 172.21.15.26:0/1503982751 >> 172.21.15.26:0/1503982751 conn(0x55f89a5e7310 msgr2=0x55f89a620960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:48.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.017+0000 7f06cb7fe640 0 client.25688 ms_handle_reset on v2:172.21.15.136:6836/1633082387 2024-09-11T23:20:48.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.017+0000 7f06f4b82740 1 -- 172.21.15.26:0/1503982751 shutdown_connections 2024-09-11T23:20:48.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.017+0000 7f06f4b82740 1 -- 172.21.15.26:0/1503982751 wait complete. 2024-09-11T23:20:48.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.017+0000 7f06f4b82740 20 client.25688 trim_cache size 0 max 16384 2024-09-11T23:20:48.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.033+0000 7f06f4b82740 1 Processor -- start 2024-09-11T23:20:48.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.033+0000 7f06f4b82740 1 -- start start 2024-09-11T23:20:48.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.033+0000 7f06f4b82740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f89a61e5d0 0x55f89a635100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:48.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.033+0000 7f06f4b82740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f89a61fb40 0x55f89a6210e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:48.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.033+0000 7f06f4b82740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f89a629370 0x55f89a621620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:48.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.033+0000 7f06f4b82740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f06ec1ab5b0 con 0x55f89a61e5d0 2024-09-11T23:20:48.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.033+0000 7f06f4b82740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f06ec1ad270 con 0x55f89a61fb40 2024-09-11T23:20:48.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.033+0000 7f06f4b82740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f06ec1abc80 con 0x55f89a629370 2024-09-11T23:20:48.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.033+0000 7f06f0d46640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f89a61e5d0 0x55f89a635100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:48.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.033+0000 7f06f0d46640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f89a61e5d0 0x55f89a635100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55864/0 (socket says 172.21.15.26:55864) 2024-09-11T23:20:48.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.033+0000 7f06f0d46640 1 -- 172.21.15.26:0/1789860807 learned_addr learned my addr 172.21.15.26:0/1789860807 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:20:48.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.033+0000 7f06eb7fe640 1 --2- 172.21.15.26:0/1789860807 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f89a61fb40 0x55f89a6210e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:48.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.033+0000 7f06f0d46640 1 --2- 172.21.15.26:0/1789860807 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f89a629370 0x55f89a621620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:48.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.033+0000 7f06f0d46640 1 -- 172.21.15.26:0/1789860807 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f89a629370 msgr2=0x55f89a621620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:20:48.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.033+0000 7f06f0d46640 1 --2- 172.21.15.26:0/1789860807 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f89a629370 0x55f89a621620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:48.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.033+0000 7f06f0d46640 1 -- 172.21.15.26:0/1789860807 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f89a61fb40 msgr2=0x55f89a6210e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:20:48.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.033+0000 7f06f0d46640 1 --2- 172.21.15.26:0/1789860807 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f89a61fb40 0x55f89a6210e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:48.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.033+0000 7f06f0d46640 1 -- 172.21.15.26:0/1789860807 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f89a61b960 con 0x55f89a61e5d0 2024-09-11T23:20:48.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.033+0000 7f06eb7fe640 1 --2- 172.21.15.26:0/1789860807 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f89a61fb40 0x55f89a6210e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:20:48.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.034+0000 7f06f0d46640 1 --2- 172.21.15.26:0/1789860807 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f89a61e5d0 0x55f89a635100 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f06e4002230 tx=0x7f06e4005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a21cc907973c1c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:48.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.034+0000 7f06d1ffb640 1 -- 172.21.15.26:0/1789860807 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06e405f040 con 0x55f89a61e5d0 2024-09-11T23:20:48.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.034+0000 7f06d1ffb640 1 -- 172.21.15.26:0/1789860807 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06e4036040 con 0x55f89a61e5d0 2024-09-11T23:20:48.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.034+0000 7f06d1ffb640 1 -- 172.21.15.26:0/1789860807 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06e402f9b0 con 0x55f89a61e5d0 2024-09-11T23:20:48.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.035+0000 7f06f4b82740 1 -- 172.21.15.26:0/1789860807 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f89a61e5d0 msgr2=0x55f89a635100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:48.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.035+0000 7f06f4b82740 1 --2- 172.21.15.26:0/1789860807 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f89a61e5d0 0x55f89a635100 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f06e4002230 tx=0x7f06e4005ef0 comp rx=0 tx=0).stop 2024-09-11T23:20:48.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.035+0000 7f06f4b82740 1 -- 172.21.15.26:0/1789860807 shutdown_connections 2024-09-11T23:20:48.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.035+0000 7f06f4b82740 1 --2- 172.21.15.26:0/1789860807 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f89a61fb40 0x55f89a6210e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:48.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.035+0000 7f06f4b82740 1 --2- 172.21.15.26:0/1789860807 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f89a629370 0x55f89a621620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:48.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.035+0000 7f06f4b82740 1 --2- 172.21.15.26:0/1789860807 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f89a61e5d0 0x55f89a635100 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:48.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.035+0000 7f06f4b82740 1 -- 172.21.15.26:0/1789860807 >> 172.21.15.26:0/1789860807 conn(0x55f89a5e7310 msgr2=0x55f89a625050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:48.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.035+0000 7f06f4b82740 1 -- 172.21.15.26:0/1789860807 shutdown_connections 2024-09-11T23:20:48.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.035+0000 7f06f4b82740 1 -- 172.21.15.26:0/1789860807 wait complete. 2024-09-11T23:20:48.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.035+0000 7f06f4b82740 1 -- start start 2024-09-11T23:20:48.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.036+0000 7f06f4b82740 1 Processor -- start 2024-09-11T23:20:48.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.036+0000 7f06f4b82740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f89a61e5d0 0x55f89a631760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:48.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.036+0000 7f06f4b82740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f89a61fb40 0x55f89a62cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:48.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.036+0000 7f06f4b82740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f89a629370 0x55f89a62d070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:48.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.036+0000 7f06f4b82740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f06ec1acae0 con 0x55f89a61fb40 2024-09-11T23:20:48.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.036+0000 7f06f4b82740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f06ec1ac350 con 0x55f89a61e5d0 2024-09-11T23:20:48.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.036+0000 7f06f4b82740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f06ec1a9590 con 0x55f89a629370 2024-09-11T23:20:48.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.036+0000 7f06f4b82740 10 client.0 _ll_register_callbacks cb 0x7f06dc0d5420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:20:48.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.036+0000 7f06f0d46640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f89a61e5d0 0x55f89a631760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:48.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.036+0000 7f06eb7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f89a61fb40 0x55f89a62cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:48.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.036+0000 7f06f0d46640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f89a61e5d0 0x55f89a631760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:32786/0 (socket says 172.21.15.26:32786) 2024-09-11T23:20:48.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.036+0000 7f06f0d46640 1 -- 172.21.15.26:0/1493834309 learned_addr learned my addr 172.21.15.26:0/1493834309 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:20:48.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.036+0000 7f06f0d46640 1 --2- 172.21.15.26:0/1493834309 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f89a629370 0x55f89a62d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:48.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.037+0000 7f06f0d46640 1 -- 172.21.15.26:0/1493834309 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f89a629370 msgr2=0x55f89a62d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:20:48.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.037+0000 7f06f0d46640 1 --2- 172.21.15.26:0/1493834309 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f89a629370 0x55f89a62d070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:48.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.037+0000 7f06f0d46640 1 -- 172.21.15.26:0/1493834309 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f89a61fb40 msgr2=0x55f89a62cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:20:48.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.037+0000 7f06f0d46640 1 --2- 172.21.15.26:0/1493834309 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f89a61fb40 0x55f89a62cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:48.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.037+0000 7f06f0d46640 1 -- 172.21.15.26:0/1493834309 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f89a62d630 con 0x55f89a61e5d0 2024-09-11T23:20:48.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.037+0000 7f06f0d46640 1 --2- 172.21.15.26:0/1493834309 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f89a61e5d0 0x55f89a631760 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f06e402ade0 tx=0x7f06e40098a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=13b13abc6249c5d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:48.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.037+0000 7f06cb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-11T23:20:48.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.037+0000 7f06cb7fe640 1 -- 172.21.15.26:0/1493834309 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06e4066450 con 0x55f89a61e5d0 2024-09-11T23:20:48.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.037+0000 7f06f4b82740 1 -- 172.21.15.26:0/1493834309 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f06e4036040 con 0x55f89a61e5d0 2024-09-11T23:20:48.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.037+0000 7f06cb7fe640 1 -- 172.21.15.26:0/1493834309 <== mon.2 v2:172.21.15.76:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f06e402f9b0 con 0x55f89a61e5d0 2024-09-11T23:20:48.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.037+0000 7f06cb7fe640 1 -- 172.21.15.26:0/1493834309 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f06e402fbb0 con 0x55f89a61e5d0 2024-09-11T23:20:48.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.037+0000 7f06f4b82740 10 client.25694 fetch_fsmap learned FSMap version 33 2024-09-11T23:20:48.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.037+0000 7f06f4b82740 1 -- 172.21.15.26:0/1493834309 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55f89a6a32d0 con 0x55f89a61e5d0 2024-09-11T23:20:48.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.038+0000 7f06cb7fe640 1 -- 172.21.15.26:0/1493834309 <== mon.2 v2:172.21.15.76:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06e4036040 con 0x55f89a61e5d0 2024-09-11T23:20:48.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.038+0000 7f06cb7fe640 1 -- 172.21.15.26:0/1493834309 <== mon.2 v2:172.21.15.76:3300/0 5 ==== fsmap(e 33) ==== 8394+0+0 (secure 0 0 0) 0x7f06e4030860 con 0x55f89a61e5d0 2024-09-11T23:20:48.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.038+0000 7f06f4b82740 10 client.25694 fetch_fsmap finished waiting for FSMap version 33 2024-09-11T23:20:48.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.038+0000 7f06f4b82740 10 client.25694 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-11T23:20:48.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.038+0000 7f06f4b82740 20 client.25694 populate_metadata read hostname 'smithi026' 2024-09-11T23:20:48.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.038+0000 7f06f4b82740 1 --2- 172.21.15.26:0/1493834309 >> [v2:172.21.15.136:6836/1633082387,v1:172.21.15.136:6837/1633082387] conn(0x55f89a6a3660 0x55f89a6a5a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:48.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.038+0000 7f06f4b82740 4 client.25694 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:20:48.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.038+0000 7f06f4b82740 1 -- 172.21.15.26:0/1493834309 --> [v2:172.21.15.136:6836/1633082387,v1:172.21.15.136:6837/1633082387] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f89a58bd70 con 0x55f89a6a3660 2024-09-11T23:20:48.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.038+0000 7f06eb7fe640 1 --2- 172.21.15.26:0/1493834309 >> [v2:172.21.15.136:6836/1633082387,v1:172.21.15.136:6837/1633082387] conn(0x55f89a6a3660 0x55f89a6a5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:48.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.039+0000 7f06eb7fe640 1 --2- 172.21.15.26:0/1493834309 >> [v2:172.21.15.136:6836/1633082387,v1:172.21.15.136:6837/1633082387] conn(0x55f89a6a3660 0x55f89a6a5a50 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=faf425cb79ed5fa server_cookie=8cb750417b111351 in_seq=0 out_seq=0 2024-09-11T23:20:48.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.039+0000 7f06cb7fe640 10 client.25694 ms_handle_connect on v2:172.21.15.136:6836/1633082387 2024-09-11T23:20:48.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.041+0000 7f06cb7fe640 1 -- 172.21.15.26:0/1493834309 <== mds.3 v2:172.21.15.136:6836/1633082387 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f89a58bd70 con 0x55f89a6a3660 2024-09-11T23:20:48.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.041+0000 7f06cb7fe640 10 client.25694 handle_command_reply: tid=0 2024-09-11T23:20:48.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.041+0000 7f06f4b82740 1 client.25694 shutdown 2024-09-11T23:20:48.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.041+0000 7f06f4b82740 1 -- 172.21.15.26:0/1493834309 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f89a61e5d0 msgr2=0x55f89a631760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:48.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.041+0000 7f06f4b82740 1 --2- 172.21.15.26:0/1493834309 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f89a61e5d0 0x55f89a631760 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f06e402ade0 tx=0x7f06e40098a0 comp rx=0 tx=0).stop 2024-09-11T23:20:48.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.041+0000 7f06f4b82740 1 -- 172.21.15.26:0/1493834309 shutdown_connections 2024-09-11T23:20:48.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.041+0000 7f06f4b82740 1 --2- 172.21.15.26:0/1493834309 >> [v2:172.21.15.136:6836/1633082387,v1:172.21.15.136:6837/1633082387] conn(0x55f89a6a3660 0x55f89a6a5a50 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:48.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.041+0000 7f06f4b82740 1 --2- 172.21.15.26:0/1493834309 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f89a629370 0x55f89a62d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:48.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.041+0000 7f06f4b82740 1 --2- 172.21.15.26:0/1493834309 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f89a61fb40 0x55f89a62cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:48.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.041+0000 7f06f4b82740 1 --2- 172.21.15.26:0/1493834309 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f89a61e5d0 0x55f89a631760 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:48.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.041+0000 7f06f4b82740 1 -- 172.21.15.26:0/1493834309 >> 172.21.15.26:0/1493834309 conn(0x55f89a5e7310 msgr2=0x55f89a6194e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:48.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.041+0000 7f06cb7fe640 0 client.25694 ms_handle_reset on v2:172.21.15.136:6836/1633082387 2024-09-11T23:20:48.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.041+0000 7f06f4b82740 1 -- 172.21.15.26:0/1493834309 shutdown_connections 2024-09-11T23:20:48.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.042+0000 7f06f4b82740 1 -- 172.21.15.26:0/1493834309 wait complete. 2024-09-11T23:20:48.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.042+0000 7f06f4b82740 20 client.25694 trim_cache size 0 max 16384 2024-09-11T23:20:48.066 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2024-09-11T23:20:48.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.043+0000 7f06f27a9640 1 -- 172.21.15.26:0/4256901606 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f06d40a2cf0 msgr2=0x7f06d40a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:48.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.043+0000 7f06f27a9640 1 --2- 172.21.15.26:0/4256901606 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f06d40a2cf0 0x7f06d40a51b0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f06dc0cd000 tx=0x7f06dc0d5550 comp rx=0 tx=0).stop 2024-09-11T23:20:48.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.043+0000 7f06f27a9640 1 -- 172.21.15.26:0/4256901606 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f06ec101de0 msgr2=0x7f06ec0744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:48.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.043+0000 7f06f27a9640 1 --2- 172.21.15.26:0/4256901606 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f06ec101de0 0x7f06ec0744f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f06dc00e9f0 tx=0x7f06dc00eeb0 comp rx=0 tx=0).stop 2024-09-11T23:20:48.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.043+0000 7f06f27a9640 1 -- 172.21.15.26:0/4256901606 shutdown_connections 2024-09-11T23:20:48.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.043+0000 7f06f27a9640 1 --2- 172.21.15.26:0/4256901606 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f06d40a2cf0 0x7f06d40a51b0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:48.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.043+0000 7f06f27a9640 1 --2- 172.21.15.26:0/4256901606 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f06ec106f10 0x7f06ec074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:48.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.043+0000 7f06f27a9640 1 --2- 172.21.15.26:0/4256901606 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f06ec102790 0x7f06ec074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:48.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.043+0000 7f06f27a9640 1 --2- 172.21.15.26:0/4256901606 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f06ec101de0 0x7f06ec0744f0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:48.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.043+0000 7f06f27a9640 1 -- 172.21.15.26:0/4256901606 >> 172.21.15.26:0/4256901606 conn(0x7f06ec0fd8c0 msgr2=0x7f06ec0feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:48.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.043+0000 7f06f27a9640 1 -- 172.21.15.26:0/4256901606 shutdown_connections 2024-09-11T23:20:48.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:48.043+0000 7f06f27a9640 1 -- 172.21.15.26:0/4256901606 wait complete. 2024-09-11T23:20:48.865 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-11T23:20:49.153 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:20:49.527 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:49 smithi026 ceph-mon[31172]: pgmap v76: 97 pgs: 97 active+clean; 240 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.3 MiB/s wr, 182 op/s 2024-09-11T23:20:49.552 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:49 smithi076 ceph-mon[35296]: pgmap v76: 97 pgs: 97 active+clean; 240 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.3 MiB/s wr, 182 op/s 2024-09-11T23:20:49.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:49 smithi136 ceph-mon[35121]: pgmap v76: 97 pgs: 97 active+clean; 240 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.3 MiB/s wr, 182 op/s 2024-09-11T23:20:49.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.743+0000 7fe89d130640 1 -- 172.21.15.26:0/654142057 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe8981036e0 msgr2=0x7fe898107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:49.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.743+0000 7fe89d130640 1 --2- 172.21.15.26:0/654142057 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe8981036e0 0x7fe898107b90 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fe88c009930 tx=0x7fe88c02a420 comp rx=0 tx=0).stop 2024-09-11T23:20:49.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.743+0000 7fe89d130640 1 -- 172.21.15.26:0/654142057 shutdown_connections 2024-09-11T23:20:49.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.743+0000 7fe89d130640 1 --2- 172.21.15.26:0/654142057 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe8981082a0 0x7fe89810ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:49.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.743+0000 7fe89d130640 1 --2- 172.21.15.26:0/654142057 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe8981036e0 0x7fe898107b90 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:49.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.743+0000 7fe89d130640 1 --2- 172.21.15.26:0/654142057 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe898102d30 0x7fe898103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:49.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.743+0000 7fe89d130640 1 -- 172.21.15.26:0/654142057 >> 172.21.15.26:0/654142057 conn(0x7fe8980fc7d0 msgr2=0x7fe8980febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:49.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.743+0000 7fe89d130640 1 -- 172.21.15.26:0/654142057 shutdown_connections 2024-09-11T23:20:49.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.743+0000 7fe89d130640 1 -- 172.21.15.26:0/654142057 wait complete. 2024-09-11T23:20:49.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.744+0000 7fe89d130640 1 Processor -- start 2024-09-11T23:20:49.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.744+0000 7fe89d130640 1 -- start start 2024-09-11T23:20:49.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.744+0000 7fe89d130640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe898102d30 0x7fe8981985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:49.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.744+0000 7fe89d130640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe8981036e0 0x7fe898198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:49.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.744+0000 7fe89d130640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe8981082a0 0x7fe89819ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:49.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.744+0000 7fe89d130640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fe898112060 con 0x7fe8981036e0 2024-09-11T23:20:49.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.744+0000 7fe89d130640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fe898111ee0 con 0x7fe8981082a0 2024-09-11T23:20:49.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.744+0000 7fe89d130640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe8981121e0 con 0x7fe898102d30 2024-09-11T23:20:49.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.744+0000 7fe89659c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe8981036e0 0x7fe898198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:49.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.744+0000 7fe89659c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe8981036e0 0x7fe898198ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55890/0 (socket says 172.21.15.26:55890) 2024-09-11T23:20:49.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.744+0000 7fe89659c640 1 -- 172.21.15.26:0/2592388268 learned_addr learned my addr 172.21.15.26:0/2592388268 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:20:49.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.744+0000 7fe896d9d640 1 --2- 172.21.15.26:0/2592388268 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe898102d30 0x7fe8981985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:49.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.744+0000 7fe89759e640 1 --2- 172.21.15.26:0/2592388268 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe8981082a0 0x7fe89819ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:49.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.744+0000 7fe89659c640 1 -- 172.21.15.26:0/2592388268 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe898102d30 msgr2=0x7fe8981985a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:49.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.744+0000 7fe89659c640 1 --2- 172.21.15.26:0/2592388268 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe898102d30 0x7fe8981985a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:49.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.744+0000 7fe89659c640 1 -- 172.21.15.26:0/2592388268 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe8981082a0 msgr2=0x7fe89819ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:49.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.744+0000 7fe89659c640 1 --2- 172.21.15.26:0/2592388268 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe8981082a0 0x7fe89819ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:49.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.744+0000 7fe89659c640 1 -- 172.21.15.26:0/2592388268 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe89819d550 con 0x7fe8981036e0 2024-09-11T23:20:49.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.745+0000 7fe89659c640 1 --2- 172.21.15.26:0/2592388268 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe8981036e0 0x7fe898198ae0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fe88c005940 tx=0x7fe88c031b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:49.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.745+0000 7fe87ffff640 1 -- 172.21.15.26:0/2592388268 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe88c03f030 con 0x7fe8981036e0 2024-09-11T23:20:49.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.745+0000 7fe87ffff640 1 -- 172.21.15.26:0/2592388268 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe88c0043d0 con 0x7fe8981036e0 2024-09-11T23:20:49.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.745+0000 7fe89d130640 1 -- 172.21.15.26:0/2592388268 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe89819d7e0 con 0x7fe8981036e0 2024-09-11T23:20:49.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.745+0000 7fe87ffff640 1 -- 172.21.15.26:0/2592388268 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe88c035070 con 0x7fe8981036e0 2024-09-11T23:20:49.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.745+0000 7fe89d130640 1 -- 172.21.15.26:0/2592388268 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8981a5120 con 0x7fe8981036e0 2024-09-11T23:20:49.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.747+0000 7fe87ffff640 1 -- 172.21.15.26:0/2592388268 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fe88c042030 con 0x7fe8981036e0 2024-09-11T23:20:49.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.748+0000 7fe87ffff640 1 --2- 172.21.15.26:0/2592388268 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fe8800a2cf0 0x7fe8800a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:49.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.748+0000 7fe896d9d640 1 --2- 172.21.15.26:0/2592388268 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fe8800a2cf0 0x7fe8800a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:49.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.748+0000 7fe87ffff640 1 -- 172.21.15.26:0/2592388268 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fe88c0ef750 con 0x7fe8981036e0 2024-09-11T23:20:49.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.749+0000 7fe89f509740 1 Processor -- start 2024-09-11T23:20:49.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.749+0000 7fe896d9d640 1 --2- 172.21.15.26:0/2592388268 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fe8800a2cf0 0x7fe8800a51b0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fe8880095f0 tx=0x7fe888005f90 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:49.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.749+0000 7fe89f509740 1 -- start start 2024-09-11T23:20:49.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.749+0000 7fe89f509740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eae147eb40 0x55eae147d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:49.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.749+0000 7fe89f509740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55eae147d5d0 0x55eae147da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:49.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.749+0000 7fe89f509740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eae1488370 0x55eae148a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:49.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.749+0000 7fe89f509740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55eae13e32f0 con 0x55eae147eb40 2024-09-11T23:20:49.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.749+0000 7fe89f509740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55eae138acf0 con 0x55eae147d5d0 2024-09-11T23:20:49.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.749+0000 7fe89f509740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55eae13df2b0 con 0x55eae1488370 2024-09-11T23:20:49.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.749+0000 7fe896d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eae147eb40 0x55eae147d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:49.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.749+0000 7fe89659c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55eae147d5d0 0x55eae147da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:49.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.749+0000 7fe896d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eae147eb40 0x55eae147d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55898/0 (socket says 172.21.15.26:55898) 2024-09-11T23:20:49.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.749+0000 7fe896d9d640 1 -- 172.21.15.26:0/463524460 learned_addr learned my addr 172.21.15.26:0/463524460 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:20:49.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.749+0000 7fe89759e640 1 --2- 172.21.15.26:0/463524460 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eae1488370 0x55eae148a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:49.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.749+0000 7fe89659c640 1 -- 172.21.15.26:0/463524460 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eae1488370 msgr2=0x55eae148a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:20:49.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.749+0000 7fe89659c640 1 --2- 172.21.15.26:0/463524460 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eae1488370 0x55eae148a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:49.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.749+0000 7fe89659c640 1 -- 172.21.15.26:0/463524460 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eae147eb40 msgr2=0x55eae147d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:20:49.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.749+0000 7fe89659c640 1 --2- 172.21.15.26:0/463524460 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eae147eb40 0x55eae147d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:49.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.749+0000 7fe89659c640 1 -- 172.21.15.26:0/463524460 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eae136f3a0 con 0x55eae147d5d0 2024-09-11T23:20:49.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.750+0000 7fe89659c640 1 --2- 172.21.15.26:0/463524460 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55eae147d5d0 0x55eae147da10 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe88c0ece50 tx=0x7fe88c0bb6f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=baa9c6b955c72ac8 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:49.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.750+0000 7fe87d7fa640 1 -- 172.21.15.26:0/463524460 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe88c049040 con 0x55eae147d5d0 2024-09-11T23:20:49.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.750+0000 7fe87d7fa640 1 -- 172.21.15.26:0/463524460 <== mon.2 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe88c003cd0 con 0x55eae147d5d0 2024-09-11T23:20:49.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.750+0000 7fe87d7fa640 1 -- 172.21.15.26:0/463524460 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe88c031df0 con 0x55eae147d5d0 2024-09-11T23:20:49.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.751+0000 7fe89f509740 1 -- 172.21.15.26:0/463524460 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55eae147d5d0 msgr2=0x55eae147da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:49.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.751+0000 7fe89f509740 1 --2- 172.21.15.26:0/463524460 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55eae147d5d0 0x55eae147da10 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe88c0ece50 tx=0x7fe88c0bb6f0 comp rx=0 tx=0).stop 2024-09-11T23:20:49.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.751+0000 7fe89f509740 1 -- 172.21.15.26:0/463524460 shutdown_connections 2024-09-11T23:20:49.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.751+0000 7fe89f509740 1 --2- 172.21.15.26:0/463524460 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55eae147d5d0 0x55eae147da10 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:49.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.751+0000 7fe89f509740 1 --2- 172.21.15.26:0/463524460 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eae1488370 0x55eae148a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:49.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.751+0000 7fe89f509740 1 --2- 172.21.15.26:0/463524460 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eae147eb40 0x55eae147d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:49.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.751+0000 7fe89f509740 1 -- 172.21.15.26:0/463524460 >> 172.21.15.26:0/463524460 conn(0x55eae1446310 msgr2=0x55eae1479c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:49.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.751+0000 7fe89f509740 1 -- 172.21.15.26:0/463524460 shutdown_connections 2024-09-11T23:20:49.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.751+0000 7fe89f509740 1 -- 172.21.15.26:0/463524460 wait complete. 2024-09-11T23:20:49.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.751+0000 7fe89f509740 1 -- start start 2024-09-11T23:20:49.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.751+0000 7fe89f509740 1 Processor -- start 2024-09-11T23:20:49.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.752+0000 7fe89f509740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eae147d5d0 0x55eae1487da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:49.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.752+0000 7fe89f509740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55eae147eb40 0x55eae1485050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:49.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.752+0000 7fe89f509740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eae1488370 0x55eae14857e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:49.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.752+0000 7fe89f509740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55eae148ae30 con 0x55eae147d5d0 2024-09-11T23:20:49.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.752+0000 7fe89f509740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55eae139e060 con 0x55eae147eb40 2024-09-11T23:20:49.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.752+0000 7fe89f509740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55eae148afb0 con 0x55eae1488370 2024-09-11T23:20:49.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.752+0000 7fe89f509740 10 client.0 _ll_register_callbacks cb 0x55eae10f18c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:20:49.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.752+0000 7fe896d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eae147d5d0 0x55eae1487da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:49.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.752+0000 7fe896d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eae147d5d0 0x55eae1487da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55902/0 (socket says 172.21.15.26:55902) 2024-09-11T23:20:49.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.752+0000 7fe896d9d640 1 -- 172.21.15.26:0/2931539127 learned_addr learned my addr 172.21.15.26:0/2931539127 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:20:49.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.752+0000 7fe89659c640 1 --2- 172.21.15.26:0/2931539127 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55eae147eb40 0x55eae1485050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:49.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.752+0000 7fe89759e640 1 --2- 172.21.15.26:0/2931539127 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eae1488370 0x55eae14857e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:49.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.752+0000 7fe896d9d640 1 -- 172.21.15.26:0/2931539127 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eae1488370 msgr2=0x55eae14857e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:20:49.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.752+0000 7fe896d9d640 1 --2- 172.21.15.26:0/2931539127 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eae1488370 0x55eae14857e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:49.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.752+0000 7fe896d9d640 1 -- 172.21.15.26:0/2931539127 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55eae147eb40 msgr2=0x55eae1485050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:20:49.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.752+0000 7fe896d9d640 1 --2- 172.21.15.26:0/2931539127 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55eae147eb40 0x55eae1485050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:49.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.752+0000 7fe896d9d640 1 -- 172.21.15.26:0/2931539127 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eae11543c0 con 0x55eae147d5d0 2024-09-11T23:20:49.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.752+0000 7fe89659c640 1 --2- 172.21.15.26:0/2931539127 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55eae147eb40 0x55eae1485050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:20:49.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.752+0000 7fe896d9d640 1 --2- 172.21.15.26:0/2931539127 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eae147d5d0 0x55eae1487da0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fe8880097b0 tx=0x7fe888004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cba6e0f343724b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:49.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.752+0000 7fe87aff5640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:20:49.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.752+0000 7fe87aff5640 1 -- 172.21.15.26:0/2931539127 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe88802b060 con 0x55eae147d5d0 2024-09-11T23:20:49.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.752+0000 7fe87aff5640 1 -- 172.21.15.26:0/2931539127 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fe88801da70 con 0x55eae147d5d0 2024-09-11T23:20:49.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.752+0000 7fe89f509740 1 -- 172.21.15.26:0/2931539127 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe88c003cd0 con 0x55eae147d5d0 2024-09-11T23:20:49.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.753+0000 7fe87aff5640 1 -- 172.21.15.26:0/2931539127 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fe88801dc70 con 0x55eae147d5d0 2024-09-11T23:20:49.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.753+0000 7fe89f509740 10 client.16128 fetch_fsmap learned FSMap version 33 2024-09-11T23:20:49.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.753+0000 7fe89f509740 1 -- 172.21.15.26:0/2931539127 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55eae1496890 con 0x55eae147d5d0 2024-09-11T23:20:49.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.753+0000 7fe87aff5640 1 -- 172.21.15.26:0/2931539127 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe88c003cd0 con 0x55eae147d5d0 2024-09-11T23:20:49.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.753+0000 7fe87aff5640 1 -- 172.21.15.26:0/2931539127 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 33) ==== 8394+0+0 (secure 0 0 0) 0x7fe88801c400 con 0x55eae147d5d0 2024-09-11T23:20:49.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.753+0000 7fe89f509740 10 client.16128 fetch_fsmap finished waiting for FSMap version 33 2024-09-11T23:20:49.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.753+0000 7fe89f509740 10 client.16128 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-11T23:20:49.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.753+0000 7fe89f509740 20 client.16128 populate_metadata read hostname 'smithi026' 2024-09-11T23:20:49.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.753+0000 7fe89f509740 1 --2- 172.21.15.26:0/2931539127 >> [v2:172.21.15.76:6836/2078978748,v1:172.21.15.76:6837/2078978748] conn(0x55eae1497d50 0x55eae149a180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:49.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.753+0000 7fe89f509740 4 client.16128 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:20:49.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.753+0000 7fe89f509740 1 -- 172.21.15.26:0/2931539127 --> [v2:172.21.15.76:6836/2078978748,v1:172.21.15.76:6837/2078978748] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eae13f0860 con 0x55eae1497d50 2024-09-11T23:20:49.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.753+0000 7fe89659c640 1 --2- 172.21.15.26:0/2931539127 >> [v2:172.21.15.76:6836/2078978748,v1:172.21.15.76:6837/2078978748] conn(0x55eae1497d50 0x55eae149a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:49.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.754+0000 7fe89659c640 1 --2- 172.21.15.26:0/2931539127 >> [v2:172.21.15.76:6836/2078978748,v1:172.21.15.76:6837/2078978748] conn(0x55eae1497d50 0x55eae149a180 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f330c580412da43c server_cookie=87dc71171399cc8d in_seq=0 out_seq=0 2024-09-11T23:20:49.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.754+0000 7fe87aff5640 10 client.16128 ms_handle_connect on v2:172.21.15.76:6836/2078978748 2024-09-11T23:20:49.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.754+0000 7fe87aff5640 1 -- 172.21.15.26:0/2931539127 <== mds.2 v2:172.21.15.76:6836/2078978748 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55eae13f0860 con 0x55eae1497d50 2024-09-11T23:20:49.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.754+0000 7fe87aff5640 10 client.16128 handle_command_reply: tid=0 2024-09-11T23:20:49.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.754+0000 7fe89f509740 1 client.16128 shutdown 2024-09-11T23:20:49.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.754+0000 7fe89f509740 1 -- 172.21.15.26:0/2931539127 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eae147d5d0 msgr2=0x55eae1487da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:49.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.754+0000 7fe89f509740 1 --2- 172.21.15.26:0/2931539127 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eae147d5d0 0x55eae1487da0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fe8880097b0 tx=0x7fe888004730 comp rx=0 tx=0).stop 2024-09-11T23:20:49.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.755+0000 7fe89f509740 1 -- 172.21.15.26:0/2931539127 shutdown_connections 2024-09-11T23:20:49.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.755+0000 7fe89f509740 1 --2- 172.21.15.26:0/2931539127 >> [v2:172.21.15.76:6836/2078978748,v1:172.21.15.76:6837/2078978748] conn(0x55eae1497d50 0x55eae149a180 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:49.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.755+0000 7fe89f509740 1 --2- 172.21.15.26:0/2931539127 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55eae147eb40 0x55eae1485050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:49.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.755+0000 7fe89f509740 1 --2- 172.21.15.26:0/2931539127 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eae1488370 0x55eae14857e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:49.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.755+0000 7fe89f509740 1 --2- 172.21.15.26:0/2931539127 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eae147d5d0 0x55eae1487da0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:49.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.755+0000 7fe87aff5640 0 client.16128 ms_handle_reset on v2:172.21.15.76:6836/2078978748 2024-09-11T23:20:49.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.755+0000 7fe89f509740 1 -- 172.21.15.26:0/2931539127 >> 172.21.15.26:0/2931539127 conn(0x55eae1446310 msgr2=0x55eae147f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:49.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.755+0000 7fe89f509740 1 -- 172.21.15.26:0/2931539127 shutdown_connections 2024-09-11T23:20:49.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.755+0000 7fe89f509740 1 -- 172.21.15.26:0/2931539127 wait complete. 2024-09-11T23:20:49.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.755+0000 7fe89f509740 20 client.16128 trim_cache size 0 max 16384 2024-09-11T23:20:49.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.768+0000 7fe89f509740 1 Processor -- start 2024-09-11T23:20:49.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.768+0000 7fe89f509740 1 -- start start 2024-09-11T23:20:49.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.768+0000 7fe89f509740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55eae147d5d0 0x55eae1479e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:49.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.768+0000 7fe89f509740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eae147eb40 0x55eae149fa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:49.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.768+0000 7fe89f509740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eae1488370 0x55eae149ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:49.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.768+0000 7fe89f509740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fe8981ad900 con 0x55eae1488370 2024-09-11T23:20:49.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.768+0000 7fe89f509740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fe8981af5c0 con 0x55eae147d5d0 2024-09-11T23:20:49.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.768+0000 7fe89f509740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe8981adfd0 con 0x55eae147eb40 2024-09-11T23:20:49.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.768+0000 7fe896d9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55eae147d5d0 0x55eae1479e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:49.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.768+0000 7fe89659c640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eae147eb40 0x55eae149fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:49.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.768+0000 7fe896d9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55eae147d5d0 0x55eae1479e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:32852/0 (socket says 172.21.15.26:32852) 2024-09-11T23:20:49.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.768+0000 7fe896d9d640 1 -- 172.21.15.26:0/3305705025 learned_addr learned my addr 172.21.15.26:0/3305705025 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:20:49.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.768+0000 7fe89759e640 1 --2- 172.21.15.26:0/3305705025 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eae1488370 0x55eae149ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:49.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.768+0000 7fe896d9d640 1 -- 172.21.15.26:0/3305705025 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eae147eb40 msgr2=0x55eae149fa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:20:49.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.768+0000 7fe896d9d640 1 --2- 172.21.15.26:0/3305705025 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eae147eb40 0x55eae149fa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:49.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.768+0000 7fe896d9d640 1 -- 172.21.15.26:0/3305705025 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eae1488370 msgr2=0x55eae149ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:20:49.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.768+0000 7fe896d9d640 1 --2- 172.21.15.26:0/3305705025 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eae1488370 0x55eae149ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:49.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.769+0000 7fe896d9d640 1 -- 172.21.15.26:0/3305705025 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eae1479510 con 0x55eae147d5d0 2024-09-11T23:20:49.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.769+0000 7fe89659c640 1 --2- 172.21.15.26:0/3305705025 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eae147eb40 0x55eae149fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:20:49.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.769+0000 7fe896d9d640 1 --2- 172.21.15.26:0/3305705025 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55eae147d5d0 0x55eae1479e10 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fe888000c00 tx=0x7fe8880046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=617a3fb5bbf1f74 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:49.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.769+0000 7fe87cff9640 1 -- 172.21.15.26:0/3305705025 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe88801da70 con 0x55eae147d5d0 2024-09-11T23:20:49.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.769+0000 7fe87cff9640 1 -- 172.21.15.26:0/3305705025 <== mon.2 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe888004510 con 0x55eae147d5d0 2024-09-11T23:20:49.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.769+0000 7fe87cff9640 1 -- 172.21.15.26:0/3305705025 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8880035f0 con 0x55eae147d5d0 2024-09-11T23:20:49.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.770+0000 7fe89f509740 1 -- 172.21.15.26:0/3305705025 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55eae147d5d0 msgr2=0x55eae1479e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:49.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.770+0000 7fe89f509740 1 --2- 172.21.15.26:0/3305705025 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55eae147d5d0 0x55eae1479e10 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fe888000c00 tx=0x7fe8880046c0 comp rx=0 tx=0).stop 2024-09-11T23:20:49.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.770+0000 7fe89f509740 1 -- 172.21.15.26:0/3305705025 shutdown_connections 2024-09-11T23:20:49.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.770+0000 7fe89f509740 1 --2- 172.21.15.26:0/3305705025 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eae1488370 0x55eae149ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:49.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.770+0000 7fe89f509740 1 --2- 172.21.15.26:0/3305705025 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eae147eb40 0x55eae149fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:49.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.770+0000 7fe89f509740 1 --2- 172.21.15.26:0/3305705025 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55eae147d5d0 0x55eae1479e10 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:49.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.770+0000 7fe89f509740 1 -- 172.21.15.26:0/3305705025 >> 172.21.15.26:0/3305705025 conn(0x55eae1446310 msgr2=0x55eae1479700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:49.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.770+0000 7fe89f509740 1 -- 172.21.15.26:0/3305705025 shutdown_connections 2024-09-11T23:20:49.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.770+0000 7fe89f509740 1 -- 172.21.15.26:0/3305705025 wait complete. 2024-09-11T23:20:49.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.771+0000 7fe89f509740 1 -- start start 2024-09-11T23:20:49.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.771+0000 7fe89f509740 1 Processor -- start 2024-09-11T23:20:49.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.771+0000 7fe89f509740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eae147d5d0 0x55eae148c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:49.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.771+0000 7fe89f509740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55eae147eb40 0x55eae148c690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:49.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.771+0000 7fe89f509740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eae1488370 0x55eae148cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:49.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.771+0000 7fe89f509740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fe8981aee30 con 0x55eae147d5d0 2024-09-11T23:20:49.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.771+0000 7fe89f509740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fe8981ae6a0 con 0x55eae147eb40 2024-09-11T23:20:49.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.771+0000 7fe89f509740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe8981ab8e0 con 0x55eae1488370 2024-09-11T23:20:49.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.771+0000 7fe89f509740 10 client.0 _ll_register_callbacks cb 0x7fe88800e3f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:20:49.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.771+0000 7fe896d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eae147d5d0 0x55eae148c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:49.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.771+0000 7fe89659c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55eae147eb40 0x55eae148c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:49.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.771+0000 7fe896d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eae147d5d0 0x55eae148c150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55906/0 (socket says 172.21.15.26:55906) 2024-09-11T23:20:49.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.771+0000 7fe896d9d640 1 -- 172.21.15.26:0/2165385458 learned_addr learned my addr 172.21.15.26:0/2165385458 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:20:49.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.771+0000 7fe89659c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55eae147eb40 0x55eae148c690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:32866/0 (socket says 172.21.15.26:32866) 2024-09-11T23:20:49.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.772+0000 7fe89759e640 1 --2- 172.21.15.26:0/2165385458 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eae1488370 0x55eae148cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:49.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.772+0000 7fe89659c640 1 -- 172.21.15.26:0/2165385458 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eae1488370 msgr2=0x55eae148cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:20:49.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.772+0000 7fe89659c640 1 --2- 172.21.15.26:0/2165385458 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eae1488370 0x55eae148cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:49.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.772+0000 7fe89659c640 1 -- 172.21.15.26:0/2165385458 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eae147d5d0 msgr2=0x55eae148c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:20:49.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.772+0000 7fe89659c640 1 --2- 172.21.15.26:0/2165385458 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eae147d5d0 0x55eae148c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:49.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.772+0000 7fe89659c640 1 -- 172.21.15.26:0/2165385458 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eae1506030 con 0x55eae147eb40 2024-09-11T23:20:49.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.772+0000 7fe89759e640 1 --2- 172.21.15.26:0/2165385458 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eae1488370 0x55eae148cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:20:49.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.772+0000 7fe89659c640 1 --2- 172.21.15.26:0/2165385458 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55eae147eb40 0x55eae148c690 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x55eae147aac0 tx=0x7fe88c03b460 comp rx=0 tx=0).ready entity=mon.2 client_cookie=816c19273a208a92 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:49.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.772+0000 7fe87aff5640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-11T23:20:49.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.772+0000 7fe87aff5640 1 -- 172.21.15.26:0/2165385458 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe88c0d8070 con 0x55eae147eb40 2024-09-11T23:20:49.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.772+0000 7fe87aff5640 1 -- 172.21.15.26:0/2165385458 <== mon.2 v2:172.21.15.76:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fe88c003cd0 con 0x55eae147eb40 2024-09-11T23:20:49.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.772+0000 7fe89f509740 1 -- 172.21.15.26:0/2165385458 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe888004510 con 0x55eae147eb40 2024-09-11T23:20:49.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.772+0000 7fe87aff5640 1 -- 172.21.15.26:0/2165385458 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fe88c049040 con 0x55eae147eb40 2024-09-11T23:20:49.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.772+0000 7fe89f509740 10 client.25712 fetch_fsmap learned FSMap version 33 2024-09-11T23:20:49.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.772+0000 7fe89f509740 1 -- 172.21.15.26:0/2165385458 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55eae15072f0 con 0x55eae147eb40 2024-09-11T23:20:49.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.773+0000 7fe87aff5640 1 -- 172.21.15.26:0/2165385458 <== mon.2 v2:172.21.15.76:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe888004510 con 0x55eae147eb40 2024-09-11T23:20:49.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.773+0000 7fe87aff5640 1 -- 172.21.15.26:0/2165385458 <== mon.2 v2:172.21.15.76:3300/0 5 ==== fsmap(e 33) ==== 8394+0+0 (secure 0 0 0) 0x7fe88c0da340 con 0x55eae147eb40 2024-09-11T23:20:49.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.773+0000 7fe89f509740 10 client.25712 fetch_fsmap finished waiting for FSMap version 33 2024-09-11T23:20:49.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.773+0000 7fe89f509740 10 client.25712 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-11T23:20:49.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.773+0000 7fe89f509740 20 client.25712 populate_metadata read hostname 'smithi026' 2024-09-11T23:20:49.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.773+0000 7fe89f509740 1 --2- 172.21.15.26:0/2165385458 >> [v2:172.21.15.76:6836/2078978748,v1:172.21.15.76:6837/2078978748] conn(0x55eae15076f0 0x55eae1509ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:49.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.773+0000 7fe89f509740 4 client.25712 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:20:49.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.773+0000 7fe89f509740 1 -- 172.21.15.26:0/2165385458 --> [v2:172.21.15.76:6836/2078978748,v1:172.21.15.76:6837/2078978748] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55eae13ead70 con 0x55eae15076f0 2024-09-11T23:20:49.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.773+0000 7fe896d9d640 1 --2- 172.21.15.26:0/2165385458 >> [v2:172.21.15.76:6836/2078978748,v1:172.21.15.76:6837/2078978748] conn(0x55eae15076f0 0x55eae1509ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:49.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.773+0000 7fe87aff5640 10 client.25712 ms_handle_connect on v2:172.21.15.76:6836/2078978748 2024-09-11T23:20:49.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.774+0000 7fe896d9d640 1 --2- 172.21.15.26:0/2165385458 >> [v2:172.21.15.76:6836/2078978748,v1:172.21.15.76:6837/2078978748] conn(0x55eae15076f0 0x55eae1509ae0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ff56c6d8b6da5dc7 server_cookie=b67376e5d33ad8d3 in_seq=0 out_seq=0 2024-09-11T23:20:49.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.774+0000 7fe87aff5640 1 -- 172.21.15.26:0/2165385458 <== mds.2 v2:172.21.15.76:6836/2078978748 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55eae13ead70 con 0x55eae15076f0 2024-09-11T23:20:49.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.774+0000 7fe87aff5640 10 client.25712 handle_command_reply: tid=0 2024-09-11T23:20:49.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.774+0000 7fe89f509740 1 client.25712 shutdown 2024-09-11T23:20:49.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.774+0000 7fe89f509740 1 -- 172.21.15.26:0/2165385458 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55eae147eb40 msgr2=0x55eae148c690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:49.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.774+0000 7fe89f509740 1 --2- 172.21.15.26:0/2165385458 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55eae147eb40 0x55eae148c690 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x55eae147aac0 tx=0x7fe88c03b460 comp rx=0 tx=0).stop 2024-09-11T23:20:49.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.774+0000 7fe89f509740 1 -- 172.21.15.26:0/2165385458 shutdown_connections 2024-09-11T23:20:49.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.774+0000 7fe89f509740 1 --2- 172.21.15.26:0/2165385458 >> [v2:172.21.15.76:6836/2078978748,v1:172.21.15.76:6837/2078978748] conn(0x55eae15076f0 0x55eae1509ae0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:49.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.774+0000 7fe89f509740 1 --2- 172.21.15.26:0/2165385458 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55eae147eb40 0x55eae148c690 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:49.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.774+0000 7fe89f509740 1 --2- 172.21.15.26:0/2165385458 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eae1488370 0x55eae148cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:49.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.774+0000 7fe89f509740 1 --2- 172.21.15.26:0/2165385458 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eae147d5d0 0x55eae148c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:49.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.774+0000 7fe87aff5640 0 client.25712 ms_handle_reset on v2:172.21.15.76:6836/2078978748 2024-09-11T23:20:49.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.774+0000 7fe89f509740 1 -- 172.21.15.26:0/2165385458 >> 172.21.15.26:0/2165385458 conn(0x55eae1446310 msgr2=0x55eae14784e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:49.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.775+0000 7fe89f509740 1 -- 172.21.15.26:0/2165385458 shutdown_connections 2024-09-11T23:20:49.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.775+0000 7fe89f509740 1 -- 172.21.15.26:0/2165385458 wait complete. 2024-09-11T23:20:49.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.775+0000 7fe89f509740 20 client.25712 trim_cache size 0 max 16384 2024-09-11T23:20:49.791 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2024-09-11T23:20:49.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.775+0000 7fe89d130640 1 -- 172.21.15.26:0/2592388268 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fe8800a2cf0 msgr2=0x7fe8800a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:49.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.775+0000 7fe89d130640 1 --2- 172.21.15.26:0/2592388268 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fe8800a2cf0 0x7fe8800a51b0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fe8880095f0 tx=0x7fe888005f90 comp rx=0 tx=0).stop 2024-09-11T23:20:49.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.775+0000 7fe89d130640 1 -- 172.21.15.26:0/2592388268 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe8981036e0 msgr2=0x7fe898198ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:49.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.775+0000 7fe89d130640 1 --2- 172.21.15.26:0/2592388268 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe8981036e0 0x7fe898198ae0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fe88c005940 tx=0x7fe88c031b50 comp rx=0 tx=0).stop 2024-09-11T23:20:49.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.775+0000 7fe89d130640 1 -- 172.21.15.26:0/2592388268 shutdown_connections 2024-09-11T23:20:49.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.775+0000 7fe89d130640 1 --2- 172.21.15.26:0/2592388268 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fe8800a2cf0 0x7fe8800a51b0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:49.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.775+0000 7fe89d130640 1 --2- 172.21.15.26:0/2592388268 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe8981082a0 0x7fe89819ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:49.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.775+0000 7fe89d130640 1 --2- 172.21.15.26:0/2592388268 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe8981036e0 0x7fe898198ae0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:49.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.775+0000 7fe89d130640 1 --2- 172.21.15.26:0/2592388268 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe898102d30 0x7fe8981985a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:49.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.775+0000 7fe89d130640 1 -- 172.21.15.26:0/2592388268 >> 172.21.15.26:0/2592388268 conn(0x7fe8980fc7d0 msgr2=0x7fe898105de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:49.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.776+0000 7fe89d130640 1 -- 172.21.15.26:0/2592388268 shutdown_connections 2024-09-11T23:20:49.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:49.776+0000 7fe89d130640 1 -- 172.21.15.26:0/2592388268 wait complete. 2024-09-11T23:20:50.485 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-11T23:20:50.775 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:20:51.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.493+0000 7f0e40ed4640 1 -- 172.21.15.26:0/1499743927 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0e3c10ea10 msgr2=0x7f0e3c115360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:51.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.493+0000 7f0e40ed4640 1 --2- 172.21.15.26:0/1499743927 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0e3c10ea10 0x7f0e3c115360 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f0e3000af30 tx=0x7f0e3002e660 comp rx=0 tx=0).stop 2024-09-11T23:20:51.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.497+0000 7f0e40ed4640 1 -- 172.21.15.26:0/1499743927 shutdown_connections 2024-09-11T23:20:51.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.497+0000 7f0e40ed4640 1 --2- 172.21.15.26:0/1499743927 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0e3c109e00 0x7f0e3c10e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:51.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.497+0000 7f0e40ed4640 1 --2- 172.21.15.26:0/1499743927 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0e3c10ea10 0x7f0e3c115360 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:51.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.497+0000 7f0e40ed4640 1 --2- 172.21.15.26:0/1499743927 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0e3c109450 0x7f0e3c109830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:51.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.497+0000 7f0e40ed4640 1 -- 172.21.15.26:0/1499743927 >> 172.21.15.26:0/1499743927 conn(0x7f0e3c073660 msgr2=0x7f0e3c073a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:51.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.499+0000 7f0e40ed4640 1 -- 172.21.15.26:0/1499743927 shutdown_connections 2024-09-11T23:20:51.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.500+0000 7f0e40ed4640 1 -- 172.21.15.26:0/1499743927 wait complete. 2024-09-11T23:20:51.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.501+0000 7f0e40ed4640 1 Processor -- start 2024-09-11T23:20:51.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.504+0000 7f0e40ed4640 1 -- start start 2024-09-11T23:20:51.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.505+0000 7f0e40ed4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0e3c109450 0x7f0e3c11efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:51.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.506+0000 7f0e40ed4640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0e3c109e00 0x7f0e3c118050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:51.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.506+0000 7f0e40ed4640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0e3c10ea10 0x7f0e3c118590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:51.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.507+0000 7f0e40ed4640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f0e3c120c50 con 0x7f0e3c109450 2024-09-11T23:20:51.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.507+0000 7f0e40ed4640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f0e3c120ad0 con 0x7f0e3c10ea10 2024-09-11T23:20:51.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.507+0000 7f0e40ed4640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f0e3c120dd0 con 0x7f0e3c109e00 2024-09-11T23:20:51.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.507+0000 7f0e3b7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0e3c109450 0x7f0e3c11efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:51.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.507+0000 7f0e3b7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0e3c109450 0x7f0e3c11efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50118/0 (socket says 172.21.15.26:50118) 2024-09-11T23:20:51.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.507+0000 7f0e3b7fe640 1 -- 172.21.15.26:0/1272179804 learned_addr learned my addr 172.21.15.26:0/1272179804 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:20:51.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.508+0000 7f0e3b7fe640 1 -- 172.21.15.26:0/1272179804 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0e3c109e00 msgr2=0x7f0e3c118050 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-11T23:20:51.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.508+0000 7f0e3b7fe640 1 --2- 172.21.15.26:0/1272179804 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0e3c109e00 0x7f0e3c118050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:51.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.508+0000 7f0e3b7fe640 1 -- 172.21.15.26:0/1272179804 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0e3c10ea10 msgr2=0x7f0e3c118590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-11T23:20:51.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.508+0000 7f0e3b7fe640 1 --2- 172.21.15.26:0/1272179804 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0e3c10ea10 0x7f0e3c118590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:51.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.508+0000 7f0e3b7fe640 1 -- 172.21.15.26:0/1272179804 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e3c118c00 con 0x7f0e3c109450 2024-09-11T23:20:51.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.510+0000 7f0e3b7fe640 1 --2- 172.21.15.26:0/1272179804 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0e3c109450 0x7f0e3c11efc0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f0e3400ba50 tx=0x7f0e3400bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:51.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.513+0000 7f0e38ff9640 1 -- 172.21.15.26:0/1272179804 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e34019070 con 0x7f0e3c109450 2024-09-11T23:20:51.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.513+0000 7f0e38ff9640 1 -- 172.21.15.26:0/1272179804 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e3400cc10 con 0x7f0e3c109450 2024-09-11T23:20:51.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.513+0000 7f0e38ff9640 1 -- 172.21.15.26:0/1272179804 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e34007770 con 0x7f0e3c109450 2024-09-11T23:20:51.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.514+0000 7f0e40ed4640 1 -- 172.21.15.26:0/1272179804 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e3c11caf0 con 0x7f0e3c109450 2024-09-11T23:20:51.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.514+0000 7f0e40ed4640 1 -- 172.21.15.26:0/1272179804 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e3c11cfb0 con 0x7f0e3c109450 2024-09-11T23:20:51.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.519+0000 7f0e4450f740 1 Processor -- start 2024-09-11T23:20:51.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.519+0000 7f0e4450f740 1 -- start start 2024-09-11T23:20:51.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.528+0000 7f0e4450f740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fc60711fe0 0x55fc60710530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:51.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.528+0000 7f0e4450f740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fc60710a70 0x55fc60710eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:51.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.528+0000 7f0e4450f740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fc6071b810 0x55fc6071dc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:51.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.528+0000 7f0e4450f740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55fc606762f0 con 0x55fc6071b810 2024-09-11T23:20:51.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.528+0000 7f0e4450f740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55fc6061dcf0 con 0x55fc60711fe0 2024-09-11T23:20:51.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.528+0000 7f0e4450f740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55fc606722b0 con 0x55fc60710a70 2024-09-11T23:20:51.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.531+0000 7f0e3affd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fc6071b810 0x55fc6071dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:51.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.531+0000 7f0e3affd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fc6071b810 0x55fc6071dc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50122/0 (socket says 172.21.15.26:50122) 2024-09-11T23:20:51.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.531+0000 7f0e3affd640 1 -- 172.21.15.26:0/2728417787 learned_addr learned my addr 172.21.15.26:0/2728417787 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:20:51.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.531+0000 7f0e3b7fe640 1 --2- 172.21.15.26:0/2728417787 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fc60710a70 0x55fc60710eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:51.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.532+0000 7f0e3affd640 1 -- 172.21.15.26:0/2728417787 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fc60710a70 msgr2=0x55fc60710eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:20:51.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.532+0000 7f0e3affd640 1 --2- 172.21.15.26:0/2728417787 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fc60710a70 0x55fc60710eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:51.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.532+0000 7f0e3affd640 1 -- 172.21.15.26:0/2728417787 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fc60711fe0 msgr2=0x55fc60710530 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T23:20:51.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.532+0000 7f0e3affd640 1 --2- 172.21.15.26:0/2728417787 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fc60711fe0 0x55fc60710530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:51.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.532+0000 7f0e3affd640 1 -- 172.21.15.26:0/2728417787 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc606023a0 con 0x55fc6071b810 2024-09-11T23:20:51.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.532+0000 7f0e3affd640 1 --2- 172.21.15.26:0/2728417787 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fc6071b810 0x55fc6071dc00 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f0e2c0096f0 tx=0x7f0e2c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2153d9a20a9b7450 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:51.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.533+0000 7f0e38ff9640 1 -- 172.21.15.26:0/1272179804 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f0e34007910 con 0x7f0e3c109450 2024-09-11T23:20:51.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.534+0000 7f0e38ff9640 1 --2- 172.21.15.26:0/1272179804 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f0e240a2f00 0x7f0e240a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:51.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.534+0000 7f0e38ff9640 1 -- 172.21.15.26:0/1272179804 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f0e340ce9c0 con 0x7f0e3c109450 2024-09-11T23:20:51.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.535+0000 7f0e217fa640 1 -- 172.21.15.26:0/2728417787 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e2c015020 con 0x55fc6071b810 2024-09-11T23:20:51.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.535+0000 7f0e217fa640 1 -- 172.21.15.26:0/2728417787 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e2c00bdb0 con 0x55fc6071b810 2024-09-11T23:20:51.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.536+0000 7f0e4450f740 1 -- 172.21.15.26:0/2728417787 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fc6071b810 msgr2=0x55fc6071dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:51.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.536+0000 7f0e4450f740 1 --2- 172.21.15.26:0/2728417787 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fc6071b810 0x55fc6071dc00 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f0e2c0096f0 tx=0x7f0e2c009bb0 comp rx=0 tx=0).stop 2024-09-11T23:20:51.538 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:51 smithi026 ceph-mon[31172]: pgmap v77: 97 pgs: 97 active+clean; 240 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.2 MiB/s wr, 136 op/s 2024-09-11T23:20:51.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.537+0000 7f0e3bfff640 1 --2- 172.21.15.26:0/1272179804 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f0e240a2f00 0x7f0e240a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:51.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.543+0000 7f0e4450f740 1 -- 172.21.15.26:0/2728417787 shutdown_connections 2024-09-11T23:20:51.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.543+0000 7f0e4450f740 1 --2- 172.21.15.26:0/2728417787 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fc6071b810 0x55fc6071dc00 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:51.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.543+0000 7f0e4450f740 1 --2- 172.21.15.26:0/2728417787 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fc60710a70 0x55fc60710eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:51.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.543+0000 7f0e4450f740 1 --2- 172.21.15.26:0/2728417787 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fc60711fe0 0x55fc60710530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:51.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.543+0000 7f0e4450f740 1 -- 172.21.15.26:0/2728417787 >> 172.21.15.26:0/2728417787 conn(0x55fc606d9310 msgr2=0x55fc6070d100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:51.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.545+0000 7f0e3bfff640 1 --2- 172.21.15.26:0/1272179804 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f0e240a2f00 0x7f0e240a53c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f0e30008330 tx=0x7f0e30005fd0 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:51.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.549+0000 7f0e4450f740 1 -- 172.21.15.26:0/2728417787 shutdown_connections 2024-09-11T23:20:51.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.551+0000 7f0e4450f740 1 -- 172.21.15.26:0/2728417787 wait complete. 2024-09-11T23:20:51.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.552+0000 7f0e4450f740 1 -- start start 2024-09-11T23:20:51.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.552+0000 7f0e4450f740 1 Processor -- start 2024-09-11T23:20:51.552 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:51 smithi076 ceph-mon[35296]: pgmap v77: 97 pgs: 97 active+clean; 240 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.2 MiB/s wr, 136 op/s 2024-09-11T23:20:51.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.554+0000 7f0e4450f740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fc60710a70 0x55fc6071aea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:51.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.554+0000 7f0e4450f740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fc60711fe0 0x55fc60718150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:51.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.554+0000 7f0e4450f740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fc6071b810 0x55fc60718690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:51.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.554+0000 7f0e4450f740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55fc6071e490 con 0x55fc6071b810 2024-09-11T23:20:51.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.554+0000 7f0e4450f740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55fc60631060 con 0x55fc60711fe0 2024-09-11T23:20:51.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.554+0000 7f0e4450f740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55fc6071e610 con 0x55fc60710a70 2024-09-11T23:20:51.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.554+0000 7f0e4450f740 10 client.0 _ll_register_callbacks cb 0x55fc603848c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:20:51.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.555+0000 7f0e3b7fe640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fc60710a70 0x55fc6071aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:51.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.555+0000 7f0e3b7fe640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fc60710a70 0x55fc6071aea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:59954/0 (socket says 172.21.15.26:59954) 2024-09-11T23:20:51.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.555+0000 7f0e3b7fe640 1 -- 172.21.15.26:0/3639372127 learned_addr learned my addr 172.21.15.26:0/3639372127 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:20:51.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.556+0000 7f0e3affd640 1 --2- 172.21.15.26:0/3639372127 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fc60711fe0 0x55fc60718150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:51.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.556+0000 7f0e3b7fe640 1 -- 172.21.15.26:0/3639372127 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fc60711fe0 msgr2=0x55fc60718150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:20:51.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.556+0000 7f0e3b7fe640 1 --2- 172.21.15.26:0/3639372127 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fc60711fe0 0x55fc60718150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:51.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.556+0000 7f0e3b7fe640 1 -- 172.21.15.26:0/3639372127 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fc6071b810 msgr2=0x55fc60718690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:20:51.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.557+0000 7f0e3b7fe640 1 --2- 172.21.15.26:0/3639372127 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fc6071b810 0x55fc60718690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:51.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.557+0000 7f0e3b7fe640 1 -- 172.21.15.26:0/3639372127 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc603e73c0 con 0x55fc60710a70 2024-09-11T23:20:51.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.557+0000 7f0e3b7fe640 1 --2- 172.21.15.26:0/3639372127 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fc60710a70 0x55fc6071aea0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f0e3409aca0 tx=0x7f0e3409aba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ae877b32c73775f5 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:51.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.560+0000 7f0e1f7f6640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-09-11T23:20:51.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.560+0000 7f0e1f7f6640 1 -- 172.21.15.26:0/3639372127 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e3409afa0 con 0x55fc60710a70 2024-09-11T23:20:51.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.560+0000 7f0e4450f740 1 -- 172.21.15.26:0/3639372127 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0e2c00bdb0 con 0x55fc60710a70 2024-09-11T23:20:51.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.560+0000 7f0e1f7f6640 1 -- 172.21.15.26:0/3639372127 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f0e34097610 con 0x55fc60710a70 2024-09-11T23:20:51.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.561+0000 7f0e1f7f6640 1 -- 172.21.15.26:0/3639372127 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f0e3409f350 con 0x55fc60710a70 2024-09-11T23:20:51.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.562+0000 7f0e4450f740 10 client.25771 fetch_fsmap learned FSMap version 33 2024-09-11T23:20:51.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.562+0000 7f0e4450f740 1 -- 172.21.15.26:0/3639372127 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55fc60729c60 con 0x55fc60710a70 2024-09-11T23:20:51.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.568+0000 7f0e1f7f6640 1 -- 172.21.15.26:0/3639372127 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e2c00bdb0 con 0x55fc60710a70 2024-09-11T23:20:51.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.568+0000 7f0e1f7f6640 1 -- 172.21.15.26:0/3639372127 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 33) ==== 8394+0+0 (secure 0 0 0) 0x7f0e3409b200 con 0x55fc60710a70 2024-09-11T23:20:51.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.569+0000 7f0e4450f740 10 client.25771 fetch_fsmap finished waiting for FSMap version 33 2024-09-11T23:20:51.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.569+0000 7f0e4450f740 10 client.25771 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-11T23:20:51.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.569+0000 7f0e4450f740 20 client.25771 populate_metadata read hostname 'smithi026' 2024-09-11T23:20:51.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.569+0000 7f0e4450f740 1 --2- 172.21.15.26:0/3639372127 >> [v2:172.21.15.136:6838/3211398630,v1:172.21.15.136:6839/3211398630] conn(0x55fc6072b120 0x55fc6072d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:51.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.569+0000 7f0e4450f740 4 client.25771 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:20:51.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.569+0000 7f0e4450f740 1 -- 172.21.15.26:0/3639372127 --> [v2:172.21.15.136:6838/3211398630,v1:172.21.15.136:6839/3211398630] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fc60683860 con 0x55fc6072b120 2024-09-11T23:20:51.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.572+0000 7f0e3affd640 1 --2- 172.21.15.26:0/3639372127 >> [v2:172.21.15.136:6838/3211398630,v1:172.21.15.136:6839/3211398630] conn(0x55fc6072b120 0x55fc6072d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:51.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.577+0000 7f0e3affd640 1 --2- 172.21.15.26:0/3639372127 >> [v2:172.21.15.136:6838/3211398630,v1:172.21.15.136:6839/3211398630] conn(0x55fc6072b120 0x55fc6072d5e0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2300b5467af82115 server_cookie=c3b1c5fc09b68811 in_seq=0 out_seq=0 2024-09-11T23:20:51.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.578+0000 7f0e1f7f6640 10 client.25771 ms_handle_connect on v2:172.21.15.136:6838/3211398630 2024-09-11T23:20:51.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.583+0000 7f0e1f7f6640 1 -- 172.21.15.26:0/3639372127 <== mds.1 v2:172.21.15.136:6838/3211398630 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fc60683860 con 0x55fc6072b120 2024-09-11T23:20:51.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.583+0000 7f0e1f7f6640 10 client.25771 handle_command_reply: tid=0 2024-09-11T23:20:51.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.584+0000 7f0e4450f740 1 client.25771 shutdown 2024-09-11T23:20:51.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.592+0000 7f0e4450f740 1 -- 172.21.15.26:0/3639372127 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fc60710a70 msgr2=0x55fc6071aea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:51.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.592+0000 7f0e4450f740 1 --2- 172.21.15.26:0/3639372127 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fc60710a70 0x55fc6071aea0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f0e3409aca0 tx=0x7f0e3409aba0 comp rx=0 tx=0).stop 2024-09-11T23:20:51.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.598+0000 7f0e4450f740 1 -- 172.21.15.26:0/3639372127 shutdown_connections 2024-09-11T23:20:51.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.598+0000 7f0e4450f740 1 --2- 172.21.15.26:0/3639372127 >> [v2:172.21.15.136:6838/3211398630,v1:172.21.15.136:6839/3211398630] conn(0x55fc6072b120 0x55fc6072d5e0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:51.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.598+0000 7f0e4450f740 1 --2- 172.21.15.26:0/3639372127 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fc60711fe0 0x55fc60718150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:51.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.598+0000 7f0e4450f740 1 --2- 172.21.15.26:0/3639372127 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fc6071b810 0x55fc60718690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:51.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.598+0000 7f0e4450f740 1 --2- 172.21.15.26:0/3639372127 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fc60710a70 0x55fc6071aea0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:51.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.598+0000 7f0e4450f740 1 -- 172.21.15.26:0/3639372127 >> 172.21.15.26:0/3639372127 conn(0x55fc606d9310 msgr2=0x55fc60712e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:51.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.600+0000 7f0e1f7f6640 0 client.25771 ms_handle_reset on v2:172.21.15.136:6838/3211398630 2024-09-11T23:20:51.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.604+0000 7f0e4450f740 1 -- 172.21.15.26:0/3639372127 shutdown_connections 2024-09-11T23:20:51.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.605+0000 7f0e4450f740 1 -- 172.21.15.26:0/3639372127 wait complete. 2024-09-11T23:20:51.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.605+0000 7f0e4450f740 20 client.25771 trim_cache size 0 max 16384 2024-09-11T23:20:51.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.637+0000 7f0e4450f740 1 Processor -- start 2024-09-11T23:20:51.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.637+0000 7f0e4450f740 1 -- start start 2024-09-11T23:20:51.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.637+0000 7f0e4450f740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fc60710a70 0x55fc60716a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:51.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.637+0000 7f0e4450f740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fc60711fe0 0x55fc60716f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:51.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.637+0000 7f0e3b7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fc60710a70 0x55fc60716a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:51.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.637+0000 7f0e3b7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fc60710a70 0x55fc60716a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:45302/0 (socket says 172.21.15.26:45302) 2024-09-11T23:20:51.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.637+0000 7f0e4450f740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fc6071b810 0x55fc60732eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:51.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.637+0000 7f0e3b7fe640 1 -- 172.21.15.26:0/2764390448 learned_addr learned my addr 172.21.15.26:0/2764390448 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:20:51.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.637+0000 7f0e4450f740 1 -- 172.21.15.26:0/2764390448 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f0e3c1c4e90 con 0x55fc6071b810 2024-09-11T23:20:51.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.637+0000 7f0e4450f740 1 -- 172.21.15.26:0/2764390448 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f0e3c1c6b50 con 0x55fc60710a70 2024-09-11T23:20:51.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.637+0000 7f0e4450f740 1 -- 172.21.15.26:0/2764390448 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f0e3c1c5560 con 0x55fc60711fe0 2024-09-11T23:20:51.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.638+0000 7f0e3affd640 1 --2- 172.21.15.26:0/2764390448 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fc60711fe0 0x55fc60716f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:51.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.638+0000 7f0e3affd640 1 -- 172.21.15.26:0/2764390448 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fc60710a70 msgr2=0x55fc60716a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:20:51.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.638+0000 7f0e3affd640 1 --2- 172.21.15.26:0/2764390448 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fc60710a70 0x55fc60716a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:51.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.638+0000 7f0e3affd640 1 -- 172.21.15.26:0/2764390448 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fc6071b810 msgr2=0x55fc60732eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:20:51.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.638+0000 7f0e3affd640 1 --2- 172.21.15.26:0/2764390448 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fc6071b810 0x55fc60732eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:51.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.638+0000 7f0e3affd640 1 -- 172.21.15.26:0/2764390448 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc606e6bf0 con 0x55fc60711fe0 2024-09-11T23:20:51.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.638+0000 7f0e3b7fe640 1 --2- 172.21.15.26:0/2764390448 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fc60710a70 0x55fc60716a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:20:51.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.639+0000 7f0e3affd640 1 --2- 172.21.15.26:0/2764390448 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fc60711fe0 0x55fc60716f60 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f0e2c00bfa0 tx=0x7f0e2c0090e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c438a5f9b552c56 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:51.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.642+0000 7f0e20ff9640 1 -- 172.21.15.26:0/2764390448 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e2c015040 con 0x55fc60711fe0 2024-09-11T23:20:51.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.642+0000 7f0e20ff9640 1 -- 172.21.15.26:0/2764390448 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e2c011900 con 0x55fc60711fe0 2024-09-11T23:20:51.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.646+0000 7f0e20ff9640 1 -- 172.21.15.26:0/2764390448 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e2c0043e0 con 0x55fc60711fe0 2024-09-11T23:20:51.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.647+0000 7f0e4450f740 1 -- 172.21.15.26:0/2764390448 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fc60711fe0 msgr2=0x55fc60716f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:51.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.647+0000 7f0e4450f740 1 --2- 172.21.15.26:0/2764390448 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fc60711fe0 0x55fc60716f60 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f0e2c00bfa0 tx=0x7f0e2c0090e0 comp rx=0 tx=0).stop 2024-09-11T23:20:51.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.650+0000 7f0e4450f740 1 -- 172.21.15.26:0/2764390448 shutdown_connections 2024-09-11T23:20:51.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.650+0000 7f0e4450f740 1 --2- 172.21.15.26:0/2764390448 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fc6071b810 0x55fc60732eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:51.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.650+0000 7f0e4450f740 1 --2- 172.21.15.26:0/2764390448 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fc60711fe0 0x55fc60716f60 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:51.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.650+0000 7f0e4450f740 1 --2- 172.21.15.26:0/2764390448 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fc60710a70 0x55fc60716a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:51.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.650+0000 7f0e4450f740 1 -- 172.21.15.26:0/2764390448 >> 172.21.15.26:0/2764390448 conn(0x55fc606d9310 msgr2=0x55fc6070cba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:51.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.652+0000 7f0e4450f740 1 -- 172.21.15.26:0/2764390448 shutdown_connections 2024-09-11T23:20:51.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.652+0000 7f0e4450f740 1 -- 172.21.15.26:0/2764390448 wait complete. 2024-09-11T23:20:51.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.652+0000 7f0e4450f740 1 -- start start 2024-09-11T23:20:51.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.652+0000 7f0e4450f740 1 Processor -- start 2024-09-11T23:20:51.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.653+0000 7f0e4450f740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fc60710a70 0x55fc60723dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:51.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.653+0000 7f0e4450f740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fc60711fe0 0x55fc60714c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:51.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.653+0000 7f0e4450f740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fc6071b810 0x55fc60715140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:51.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.653+0000 7f0e4450f740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f0e3c1c63c0 con 0x55fc60711fe0 2024-09-11T23:20:51.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.653+0000 7f0e4450f740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f0e3c1c5c30 con 0x55fc6071b810 2024-09-11T23:20:51.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.653+0000 7f0e4450f740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f0e3c1c2e70 con 0x55fc60710a70 2024-09-11T23:20:51.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.653+0000 7f0e4450f740 10 client.0 _ll_register_callbacks cb 0x7f0e340972f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:20:51.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.653+0000 7f0e3bfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fc6071b810 0x55fc60715140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:51.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.653+0000 7f0e3bfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fc6071b810 0x55fc60715140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:45310/0 (socket says 172.21.15.26:45310) 2024-09-11T23:20:51.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.653+0000 7f0e3bfff640 1 -- 172.21.15.26:0/2524711893 learned_addr learned my addr 172.21.15.26:0/2524711893 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:20:51.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.654+0000 7f0e3b7fe640 1 --2- 172.21.15.26:0/2524711893 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fc60710a70 0x55fc60723dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:51.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.654+0000 7f0e3b7fe640 1 -- 172.21.15.26:0/2524711893 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fc6071b810 msgr2=0x55fc60715140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:20:51.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.654+0000 7f0e3b7fe640 1 --2- 172.21.15.26:0/2524711893 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fc6071b810 0x55fc60715140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:51.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.654+0000 7f0e3b7fe640 1 -- 172.21.15.26:0/2524711893 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fc60711fe0 msgr2=0x55fc60714c00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T23:20:51.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.654+0000 7f0e3affd640 1 --2- 172.21.15.26:0/2524711893 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fc60711fe0 0x55fc60714c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:51.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.654+0000 7f0e3b7fe640 1 --2- 172.21.15.26:0/2524711893 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fc60711fe0 0x55fc60714c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:51.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.655+0000 7f0e3b7fe640 1 -- 172.21.15.26:0/2524711893 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc60715700 con 0x55fc60710a70 2024-09-11T23:20:51.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.655+0000 7f0e3affd640 1 --2- 172.21.15.26:0/2524711893 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fc60711fe0 0x55fc60714c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:20:51.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.655+0000 7f0e3b7fe640 1 --2- 172.21.15.26:0/2524711893 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fc60710a70 0x55fc60723dc0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f0e3400c7d0 tx=0x7f0e3409ac10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=64614952cca7a28 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:51.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.655+0000 7f0e1f7f6640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-09-11T23:20:51.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.655+0000 7f0e1f7f6640 1 -- 172.21.15.26:0/2524711893 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e340a3a10 con 0x55fc60710a70 2024-09-11T23:20:51.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.656+0000 7f0e4450f740 1 -- 172.21.15.26:0/2524711893 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0e2c011900 con 0x55fc60710a70 2024-09-11T23:20:51.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.656+0000 7f0e1f7f6640 1 -- 172.21.15.26:0/2524711893 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f0e34097610 con 0x55fc60710a70 2024-09-11T23:20:51.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.656+0000 7f0e1f7f6640 1 -- 172.21.15.26:0/2524711893 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f0e3409afa0 con 0x55fc60710a70 2024-09-11T23:20:51.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.657+0000 7f0e4450f740 10 client.25777 fetch_fsmap learned FSMap version 33 2024-09-11T23:20:51.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.657+0000 7f0e4450f740 1 -- 172.21.15.26:0/2524711893 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55fc60795b80 con 0x55fc60710a70 2024-09-11T23:20:51.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.661+0000 7f0e1f7f6640 1 -- 172.21.15.26:0/2524711893 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e2c011900 con 0x55fc60710a70 2024-09-11T23:20:51.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.661+0000 7f0e1f7f6640 1 -- 172.21.15.26:0/2524711893 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 33) ==== 8394+0+0 (secure 0 0 0) 0x7f0e340b8060 con 0x55fc60710a70 2024-09-11T23:20:51.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.661+0000 7f0e4450f740 10 client.25777 fetch_fsmap finished waiting for FSMap version 33 2024-09-11T23:20:51.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.661+0000 7f0e4450f740 10 client.25777 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-11T23:20:51.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.661+0000 7f0e4450f740 20 client.25777 populate_metadata read hostname 'smithi026' 2024-09-11T23:20:51.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.661+0000 7f0e4450f740 1 --2- 172.21.15.26:0/2524711893 >> [v2:172.21.15.136:6838/3211398630,v1:172.21.15.136:6839/3211398630] conn(0x55fc607961e0 0x55fc60798600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:51.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.661+0000 7f0e4450f740 4 client.25777 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:20:51.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.661+0000 7f0e4450f740 1 -- 172.21.15.26:0/2524711893 --> [v2:172.21.15.136:6838/3211398630,v1:172.21.15.136:6839/3211398630] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fc6067dd70 con 0x55fc607961e0 2024-09-11T23:20:51.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.662+0000 7f0e3affd640 1 --2- 172.21.15.26:0/2524711893 >> [v2:172.21.15.136:6838/3211398630,v1:172.21.15.136:6839/3211398630] conn(0x55fc607961e0 0x55fc60798600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:51.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.668+0000 7f0e3affd640 1 --2- 172.21.15.26:0/2524711893 >> [v2:172.21.15.136:6838/3211398630,v1:172.21.15.136:6839/3211398630] conn(0x55fc607961e0 0x55fc60798600 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3a1d0c45a9ae1d25 server_cookie=d14ea7bfc09724b0 in_seq=0 out_seq=0 2024-09-11T23:20:51.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.668+0000 7f0e1f7f6640 10 client.25777 ms_handle_connect on v2:172.21.15.136:6838/3211398630 2024-09-11T23:20:51.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.673+0000 7f0e1f7f6640 1 -- 172.21.15.26:0/2524711893 <== mds.1 v2:172.21.15.136:6838/3211398630 1 ==== command_reply(tid 0: 0 ) ==== 8+0+5281 (crc 0 0 0) 0x55fc6067dd70 con 0x55fc607961e0 2024-09-11T23:20:51.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.673+0000 7f0e1f7f6640 10 client.25777 handle_command_reply: tid=0 2024-09-11T23:20:51.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.673+0000 7f0e4450f740 1 client.25777 shutdown 2024-09-11T23:20:51.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.678+0000 7f0e4450f740 1 -- 172.21.15.26:0/2524711893 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fc60710a70 msgr2=0x55fc60723dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:51.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.678+0000 7f0e4450f740 1 --2- 172.21.15.26:0/2524711893 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fc60710a70 0x55fc60723dc0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f0e3400c7d0 tx=0x7f0e3409ac10 comp rx=0 tx=0).stop 2024-09-11T23:20:51.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.678+0000 7f0e4450f740 1 -- 172.21.15.26:0/2524711893 shutdown_connections 2024-09-11T23:20:51.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.678+0000 7f0e4450f740 1 --2- 172.21.15.26:0/2524711893 >> [v2:172.21.15.136:6838/3211398630,v1:172.21.15.136:6839/3211398630] conn(0x55fc607961e0 0x55fc60798600 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:51.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.678+0000 7f0e4450f740 1 --2- 172.21.15.26:0/2524711893 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fc6071b810 0x55fc60715140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:51.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.678+0000 7f0e4450f740 1 --2- 172.21.15.26:0/2524711893 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fc60711fe0 0x55fc60714c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:51.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.678+0000 7f0e4450f740 1 --2- 172.21.15.26:0/2524711893 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fc60710a70 0x55fc60723dc0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:51.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.678+0000 7f0e1f7f6640 0 client.25777 ms_handle_reset on v2:172.21.15.136:6838/3211398630 2024-09-11T23:20:51.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.678+0000 7f0e4450f740 1 -- 172.21.15.26:0/2524711893 >> 172.21.15.26:0/2524711893 conn(0x55fc606d9310 msgr2=0x55fc607199b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:51.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.678+0000 7f0e4450f740 1 -- 172.21.15.26:0/2524711893 shutdown_connections 2024-09-11T23:20:51.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.678+0000 7f0e4450f740 1 -- 172.21.15.26:0/2524711893 wait complete. 2024-09-11T23:20:51.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.679+0000 7f0e4450f740 20 client.25777 trim_cache size 0 max 16384 2024-09-11T23:20:51.681 INFO:teuthology.orchestra.run.smithi026.stdout:[ 2024-09-11T23:20:51.681 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-11T23:20:51.682 INFO:teuthology.orchestra.run.smithi026.stdout: "damage_type": "backtrace", 2024-09-11T23:20:51.682 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 59548436, 2024-09-11T23:20:51.682 INFO:teuthology.orchestra.run.smithi026.stdout: "ino": 1099511628312, 2024-09-11T23:20:51.682 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/volumes/qa/sv_1/9fc8f023-d646-4fec-98c7-1f3d972f60a9/client.0/tmp/blogbench-1.0/src/reader.c" 2024-09-11T23:20:51.682 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:20:51.682 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-11T23:20:51.682 INFO:teuthology.orchestra.run.smithi026.stdout: "damage_type": "backtrace", 2024-09-11T23:20:51.682 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 60730685, 2024-09-11T23:20:51.682 INFO:teuthology.orchestra.run.smithi026.stdout: "ino": 1099511628678, 2024-09-11T23:20:51.682 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/volumes/qa/sv_1/9fc8f023-d646-4fec-98c7-1f3d972f60a9/client.0/tmp/blogbench-1.0/src/.deps/process.Po" 2024-09-11T23:20:51.682 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:20:51.682 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-11T23:20:51.682 INFO:teuthology.orchestra.run.smithi026.stdout: "damage_type": "backtrace", 2024-09-11T23:20:51.683 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 88304087, 2024-09-11T23:20:51.683 INFO:teuthology.orchestra.run.smithi026.stdout: "ino": 1099511628307, 2024-09-11T23:20:51.683 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/volumes/qa/sv_1/9fc8f023-d646-4fec-98c7-1f3d972f60a9/client.0/tmp/blogbench-1.0/src/Makefile.in" 2024-09-11T23:20:51.683 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:20:51.683 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-11T23:20:51.683 INFO:teuthology.orchestra.run.smithi026.stdout: "damage_type": "backtrace", 2024-09-11T23:20:51.683 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 195604997, 2024-09-11T23:20:51.683 INFO:teuthology.orchestra.run.smithi026.stdout: "ino": 1099511628300, 2024-09-11T23:20:51.683 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/volumes/qa/sv_1/9fc8f023-d646-4fec-98c7-1f3d972f60a9/client.0/tmp/blogbench-1.0/src/blogbench.h" 2024-09-11T23:20:51.683 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:20:51.683 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-11T23:20:51.683 INFO:teuthology.orchestra.run.smithi026.stdout: "damage_type": "backtrace", 2024-09-11T23:20:51.683 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 373890108, 2024-09-11T23:20:51.683 INFO:teuthology.orchestra.run.smithi026.stdout: "ino": 1099511628299, 2024-09-11T23:20:51.684 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/volumes/qa/sv_1/9fc8f023-d646-4fec-98c7-1f3d972f60a9/client.0/tmp/blogbench-1.0/src/blogbench.c" 2024-09-11T23:20:51.684 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:20:51.684 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-11T23:20:51.684 INFO:teuthology.orchestra.run.smithi026.stdout: "damage_type": "backtrace", 2024-09-11T23:20:51.684 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 553887758, 2024-09-11T23:20:51.684 INFO:teuthology.orchestra.run.smithi026.stdout: "ino": 1099511628306, 2024-09-11T23:20:51.684 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/volumes/qa/sv_1/9fc8f023-d646-4fec-98c7-1f3d972f60a9/client.0/tmp/blogbench-1.0/src/Makefile.am" 2024-09-11T23:20:51.684 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:20:51.684 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-11T23:20:51.684 INFO:teuthology.orchestra.run.smithi026.stdout: "damage_type": "backtrace", 2024-09-11T23:20:51.684 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 671366844, 2024-09-11T23:20:51.684 INFO:teuthology.orchestra.run.smithi026.stdout: "ino": 1099511628302, 2024-09-11T23:20:51.684 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/volumes/qa/sv_1/9fc8f023-d646-4fec-98c7-1f3d972f60a9/client.0/tmp/blogbench-1.0/src/fakesnprintf.h" 2024-09-11T23:20:51.684 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:20:51.685 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-11T23:20:51.685 INFO:teuthology.orchestra.run.smithi026.stdout: "damage_type": "backtrace", 2024-09-11T23:20:51.685 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 900911283, 2024-09-11T23:20:51.685 INFO:teuthology.orchestra.run.smithi026.stdout: "ino": 1099511628301, 2024-09-11T23:20:51.685 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/volumes/qa/sv_1/9fc8f023-d646-4fec-98c7-1f3d972f60a9/client.0/tmp/blogbench-1.0/src/fakesnprintf.c" 2024-09-11T23:20:51.685 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:20:51.685 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-11T23:20:51.685 INFO:teuthology.orchestra.run.smithi026.stdout: "damage_type": "backtrace", 2024-09-11T23:20:51.685 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 926345818, 2024-09-11T23:20:51.685 INFO:teuthology.orchestra.run.smithi026.stdout: "ino": 1099511628303, 2024-09-11T23:20:51.685 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/volumes/qa/sv_1/9fc8f023-d646-4fec-98c7-1f3d972f60a9/client.0/tmp/blogbench-1.0/src/blogbench_p.h" 2024-09-11T23:20:51.685 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:20:51.685 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-11T23:20:51.686 INFO:teuthology.orchestra.run.smithi026.stdout: "damage_type": "backtrace", 2024-09-11T23:20:51.686 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 1256918086, 2024-09-11T23:20:51.686 INFO:teuthology.orchestra.run.smithi026.stdout: "ino": 1099511628657, 2024-09-11T23:20:51.686 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/volumes/qa/sv_1/9fc8f023-d646-4fec-98c7-1f3d972f60a9/client.0/tmp/blogbench-1.0/src/Makefile" 2024-09-11T23:20:51.686 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:20:51.686 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-11T23:20:51.686 INFO:teuthology.orchestra.run.smithi026.stdout: "damage_type": "backtrace", 2024-09-11T23:20:51.686 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 1482542569, 2024-09-11T23:20:51.686 INFO:teuthology.orchestra.run.smithi026.stdout: "ino": 1099511628308, 2024-09-11T23:20:51.686 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/volumes/qa/sv_1/9fc8f023-d646-4fec-98c7-1f3d972f60a9/client.0/tmp/blogbench-1.0/src/process.c" 2024-09-11T23:20:51.686 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:20:51.686 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-11T23:20:51.686 INFO:teuthology.orchestra.run.smithi026.stdout: "damage_type": "backtrace", 2024-09-11T23:20:51.686 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 1599535364, 2024-09-11T23:20:51.687 INFO:teuthology.orchestra.run.smithi026.stdout: "ino": 1099511628313, 2024-09-11T23:20:51.687 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/volumes/qa/sv_1/9fc8f023-d646-4fec-98c7-1f3d972f60a9/client.0/tmp/blogbench-1.0/src/writer.c" 2024-09-11T23:20:51.687 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:20:51.687 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-11T23:20:51.687 INFO:teuthology.orchestra.run.smithi026.stdout: "damage_type": "backtrace", 2024-09-11T23:20:51.687 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 1756046412, 2024-09-11T23:20:51.687 INFO:teuthology.orchestra.run.smithi026.stdout: "ino": 1099511628679, 2024-09-11T23:20:51.687 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/volumes/qa/sv_1/9fc8f023-d646-4fec-98c7-1f3d972f60a9/client.0/tmp/blogbench-1.0/src/process.o" 2024-09-11T23:20:51.687 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:20:51.687 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-11T23:20:51.687 INFO:teuthology.orchestra.run.smithi026.stdout: "damage_type": "backtrace", 2024-09-11T23:20:51.687 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 2032523121, 2024-09-11T23:20:51.687 INFO:teuthology.orchestra.run.smithi026.stdout: "ino": 1099511628665, 2024-09-11T23:20:51.687 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/volumes/qa/sv_1/9fc8f023-d646-4fec-98c7-1f3d972f60a9/client.0/tmp/blogbench-1.0/src/.deps" 2024-09-11T23:20:51.688 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:20:51.688 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-11T23:20:51.688 INFO:teuthology.orchestra.run.smithi026.stdout: "damage_type": "backtrace", 2024-09-11T23:20:51.688 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 2197745511, 2024-09-11T23:20:51.688 INFO:teuthology.orchestra.run.smithi026.stdout: "ino": 1099511628305, 2024-09-11T23:20:51.688 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/volumes/qa/sv_1/9fc8f023-d646-4fec-98c7-1f3d972f60a9/client.0/tmp/blogbench-1.0/src/bsd-getopt_long.h" 2024-09-11T23:20:51.688 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:20:51.688 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-11T23:20:51.688 INFO:teuthology.orchestra.run.smithi026.stdout: "damage_type": "backtrace", 2024-09-11T23:20:51.688 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 2239312215, 2024-09-11T23:20:51.688 INFO:teuthology.orchestra.run.smithi026.stdout: "ino": 1099511628309, 2024-09-11T23:20:51.688 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/volumes/qa/sv_1/9fc8f023-d646-4fec-98c7-1f3d972f60a9/client.0/tmp/blogbench-1.0/src/globals.h" 2024-09-11T23:20:51.688 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:20:51.688 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-11T23:20:51.689 INFO:teuthology.orchestra.run.smithi026.stdout: "damage_type": "backtrace", 2024-09-11T23:20:51.689 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 2442947403, 2024-09-11T23:20:51.689 INFO:teuthology.orchestra.run.smithi026.stdout: "ino": 1099511628311, 2024-09-11T23:20:51.689 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/volumes/qa/sv_1/9fc8f023-d646-4fec-98c7-1f3d972f60a9/client.0/tmp/blogbench-1.0/src/rewriter.c" 2024-09-11T23:20:51.689 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:20:51.689 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-11T23:20:51.689 INFO:teuthology.orchestra.run.smithi026.stdout: "damage_type": "backtrace", 2024-09-11T23:20:51.689 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 2585114484, 2024-09-11T23:20:51.689 INFO:teuthology.orchestra.run.smithi026.stdout: "ino": 1099511628677, 2024-09-11T23:20:51.689 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/volumes/qa/sv_1/9fc8f023-d646-4fec-98c7-1f3d972f60a9/client.0/tmp/blogbench-1.0/src/blogbench.o" 2024-09-11T23:20:51.689 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:20:51.689 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-11T23:20:51.689 INFO:teuthology.orchestra.run.smithi026.stdout: "damage_type": "backtrace", 2024-09-11T23:20:51.689 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 2754559038, 2024-09-11T23:20:51.690 INFO:teuthology.orchestra.run.smithi026.stdout: "ino": 1099511628298, 2024-09-11T23:20:51.690 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/volumes/qa/sv_1/9fc8f023-d646-4fec-98c7-1f3d972f60a9/client.0/tmp/blogbench-1.0/src/mysnprintf.h" 2024-09-11T23:20:51.690 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:20:51.690 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-11T23:20:51.690 INFO:teuthology.orchestra.run.smithi026.stdout: "damage_type": "backtrace", 2024-09-11T23:20:51.690 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 3160277156, 2024-09-11T23:20:51.690 INFO:teuthology.orchestra.run.smithi026.stdout: "ino": 1099511628314, 2024-09-11T23:20:51.690 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/volumes/qa/sv_1/9fc8f023-d646-4fec-98c7-1f3d972f60a9/client.0/tmp/blogbench-1.0/src/helpers.c" 2024-09-11T23:20:51.690 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:20:51.690 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-11T23:20:51.690 INFO:teuthology.orchestra.run.smithi026.stdout: "damage_type": "backtrace", 2024-09-11T23:20:51.690 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 3611183181, 2024-09-11T23:20:51.690 INFO:teuthology.orchestra.run.smithi026.stdout: "ino": 1099511628310, 2024-09-11T23:20:51.690 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/volumes/qa/sv_1/9fc8f023-d646-4fec-98c7-1f3d972f60a9/client.0/tmp/blogbench-1.0/src/commenter.c" 2024-09-11T23:20:51.691 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:20:51.691 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-11T23:20:51.691 INFO:teuthology.orchestra.run.smithi026.stdout: "damage_type": "backtrace", 2024-09-11T23:20:51.691 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 3728318696, 2024-09-11T23:20:51.691 INFO:teuthology.orchestra.run.smithi026.stdout: "ino": 1099511628304, 2024-09-11T23:20:51.691 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/volumes/qa/sv_1/9fc8f023-d646-4fec-98c7-1f3d972f60a9/client.0/tmp/blogbench-1.0/src/bsd-getopt_long.c" 2024-09-11T23:20:51.691 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:20:51.691 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-11T23:20:51.691 INFO:teuthology.orchestra.run.smithi026.stdout: "damage_type": "backtrace", 2024-09-11T23:20:51.691 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 3916915527, 2024-09-11T23:20:51.691 INFO:teuthology.orchestra.run.smithi026.stdout: "ino": 1099511628676, 2024-09-11T23:20:51.691 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/volumes/qa/sv_1/9fc8f023-d646-4fec-98c7-1f3d972f60a9/client.0/tmp/blogbench-1.0/src/.deps/blogbench.Po" 2024-09-11T23:20:51.691 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:20:51.691 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-11T23:20:51.692 INFO:teuthology.orchestra.run.smithi026.stdout: "damage_type": "backtrace", 2024-09-11T23:20:51.692 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 4228120609, 2024-09-11T23:20:51.692 INFO:teuthology.orchestra.run.smithi026.stdout: "ino": 1099511628297, 2024-09-11T23:20:51.692 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/volumes/qa/sv_1/9fc8f023-d646-4fec-98c7-1f3d972f60a9/client.0/tmp/blogbench-1.0/src/mysnprintf.c" 2024-09-11T23:20:51.692 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:20:51.692 INFO:teuthology.orchestra.run.smithi026.stdout:] 2024-09-11T23:20:51.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.685+0000 7f0e227fc640 1 -- 172.21.15.26:0/1272179804 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f0e240a2f00 msgr2=0x7f0e240a53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:51.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.685+0000 7f0e227fc640 1 --2- 172.21.15.26:0/1272179804 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f0e240a2f00 0x7f0e240a53c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f0e30008330 tx=0x7f0e30005fd0 comp rx=0 tx=0).stop 2024-09-11T23:20:51.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.685+0000 7f0e227fc640 1 -- 172.21.15.26:0/1272179804 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0e3c109450 msgr2=0x7f0e3c11efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:51.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.685+0000 7f0e227fc640 1 --2- 172.21.15.26:0/1272179804 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0e3c109450 0x7f0e3c11efc0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f0e3400ba50 tx=0x7f0e3400bf10 comp rx=0 tx=0).stop 2024-09-11T23:20:51.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.685+0000 7f0e227fc640 1 -- 172.21.15.26:0/1272179804 shutdown_connections 2024-09-11T23:20:51.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.685+0000 7f0e227fc640 1 --2- 172.21.15.26:0/1272179804 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f0e240a2f00 0x7f0e240a53c0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:51.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.685+0000 7f0e227fc640 1 --2- 172.21.15.26:0/1272179804 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0e3c10ea10 0x7f0e3c118590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:51.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.685+0000 7f0e227fc640 1 --2- 172.21.15.26:0/1272179804 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0e3c109e00 0x7f0e3c118050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:51.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.685+0000 7f0e227fc640 1 --2- 172.21.15.26:0/1272179804 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0e3c109450 0x7f0e3c11efc0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:51.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.685+0000 7f0e227fc640 1 -- 172.21.15.26:0/1272179804 >> 172.21.15.26:0/1272179804 conn(0x7f0e3c073660 msgr2=0x7f0e3c10a610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:51.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.690+0000 7f0e227fc640 1 -- 172.21.15.26:0/1272179804 shutdown_connections 2024-09-11T23:20:51.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:51.691+0000 7f0e227fc640 1 -- 172.21.15.26:0/1272179804 wait complete. 2024-09-11T23:20:51.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:51 smithi136 ceph-mon[35121]: pgmap v77: 97 pgs: 97 active+clean; 240 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.2 MiB/s wr, 136 op/s 2024-09-11T23:20:52.701 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-11T23:20:53.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:53 smithi026 ceph-mon[31172]: pgmap v78: 97 pgs: 97 active+clean; 240 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.2 MiB/s wr, 136 op/s 2024-09-11T23:20:53.530 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:20:53.552 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:53 smithi076 ceph-mon[35296]: pgmap v78: 97 pgs: 97 active+clean; 240 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.2 MiB/s wr, 136 op/s 2024-09-11T23:20:53.577 INFO:tasks.workunit.client.0.smithi026.stdout: 84 940335 931 576326 871 226637 1761 2024-09-11T23:20:53.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:53 smithi136 ceph-mon[35121]: pgmap v78: 97 pgs: 97 active+clean; 240 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.2 MiB/s wr, 136 op/s 2024-09-11T23:20:54.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.106+0000 7f65801c9640 1 -- 172.21.15.26:0/3788923956 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6578102570 msgr2=0x7f65781029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:54.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.106+0000 7f65801c9640 1 --2- 172.21.15.26:0/3788923956 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6578102570 0x7f65781029b0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f6568009890 tx=0x7f656802a410 comp rx=0 tx=0).stop 2024-09-11T23:20:54.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.106+0000 7f65801c9640 1 -- 172.21.15.26:0/3788923956 shutdown_connections 2024-09-11T23:20:54.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.106+0000 7f65801c9640 1 --2- 172.21.15.26:0/3788923956 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6578102ef0 0x7f657810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:54.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.106+0000 7f65801c9640 1 --2- 172.21.15.26:0/3788923956 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6578102570 0x7f65781029b0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:54.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.106+0000 7f65801c9640 1 --2- 172.21.15.26:0/3788923956 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6578100e20 0x7f6578101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:54.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.106+0000 7f65801c9640 1 -- 172.21.15.26:0/3788923956 >> 172.21.15.26:0/3788923956 conn(0x7f65780fc7d0 msgr2=0x7f65780febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:54.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.106+0000 7f65801c9640 1 -- 172.21.15.26:0/3788923956 shutdown_connections 2024-09-11T23:20:54.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.106+0000 7f65801c9640 1 -- 172.21.15.26:0/3788923956 wait complete. 2024-09-11T23:20:54.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.107+0000 7f65801c9640 1 Processor -- start 2024-09-11T23:20:54.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.107+0000 7f65801c9640 1 -- start start 2024-09-11T23:20:54.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.107+0000 7f65801c9640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6578100e20 0x7f65781a0450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:54.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.107+0000 7f65801c9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6578102570 0x7f65781a0990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:54.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.108+0000 7f65801c9640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6578102ef0 0x7f657819a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:54.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.108+0000 7f657df65640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6578100e20 0x7f65781a0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:54.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.108+0000 7f657df65640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6578100e20 0x7f65781a0450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:45322/0 (socket says 172.21.15.26:45322) 2024-09-11T23:20:54.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.108+0000 7f657df65640 1 -- 172.21.15.26:0/454794238 learned_addr learned my addr 172.21.15.26:0/454794238 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:20:54.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.108+0000 7f657d764640 1 --2- 172.21.15.26:0/454794238 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6578102570 0x7f65781a0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:54.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.108+0000 7f65801c9640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f6578110fd0 con 0x7f6578102570 2024-09-11T23:20:54.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.108+0000 7f65801c9640 1 -- 172.21.15.26:0/454794238 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f6578110e50 con 0x7f6578100e20 2024-09-11T23:20:54.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.108+0000 7f65801c9640 1 -- 172.21.15.26:0/454794238 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f6578111150 con 0x7f6578102ef0 2024-09-11T23:20:54.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.108+0000 7f657e766640 1 --2- 172.21.15.26:0/454794238 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6578102ef0 0x7f657819a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:54.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.108+0000 7f657d764640 1 -- 172.21.15.26:0/454794238 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6578102ef0 msgr2=0x7f657819a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:54.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.108+0000 7f657d764640 1 --2- 172.21.15.26:0/454794238 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6578102ef0 0x7f657819a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:54.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.108+0000 7f657d764640 1 -- 172.21.15.26:0/454794238 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6578100e20 msgr2=0x7f65781a0450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:54.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.108+0000 7f657d764640 1 --2- 172.21.15.26:0/454794238 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6578100e20 0x7f65781a0450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:54.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.108+0000 7f657d764640 1 -- 172.21.15.26:0/454794238 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f657819ad40 con 0x7f6578102570 2024-09-11T23:20:54.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.109+0000 7f657d764640 1 --2- 172.21.15.26:0/454794238 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6578102570 0x7f65781a0990 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f6568009890 tx=0x7f6568030920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:54.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.109+0000 7f656effd640 1 -- 172.21.15.26:0/454794238 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6568005590 con 0x7f6578102570 2024-09-11T23:20:54.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.109+0000 7f656effd640 1 -- 172.21.15.26:0/454794238 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f65680318e0 con 0x7f6578102570 2024-09-11T23:20:54.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.109+0000 7f656effd640 1 -- 172.21.15.26:0/454794238 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f656803c420 con 0x7f6578102570 2024-09-11T23:20:54.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.109+0000 7f65801c9640 1 -- 172.21.15.26:0/454794238 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f657819afd0 con 0x7f6578102570 2024-09-11T23:20:54.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.110+0000 7f65801c9640 1 -- 172.21.15.26:0/454794238 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f657819b440 con 0x7f6578102570 2024-09-11T23:20:54.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.112+0000 7f656effd640 1 -- 172.21.15.26:0/454794238 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f6568039040 con 0x7f6578102570 2024-09-11T23:20:54.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.113+0000 7f656effd640 1 --2- 172.21.15.26:0/454794238 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f65640a2d70 0x7f65640a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:54.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.113+0000 7f656effd640 1 -- 172.21.15.26:0/454794238 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f65680f02c0 con 0x7f6578102570 2024-09-11T23:20:54.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.114+0000 7f65825a2740 1 Processor -- start 2024-09-11T23:20:54.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.114+0000 7f65825a2740 1 -- start start 2024-09-11T23:20:54.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.114+0000 7f65825a2740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5644dc0c2b40 0x5644dc0c1090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:54.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.114+0000 7f65825a2740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5644dc0c15d0 0x5644dc0c1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:54.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.114+0000 7f65825a2740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5644dc0cc370 0x5644dc0ce760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:54.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.114+0000 7f65825a2740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5644dc0272f0 con 0x5644dc0c2b40 2024-09-11T23:20:54.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.114+0000 7f65825a2740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5644dbfcecf0 con 0x5644dc0cc370 2024-09-11T23:20:54.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.114+0000 7f65825a2740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5644dc0232b0 con 0x5644dc0c15d0 2024-09-11T23:20:54.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.114+0000 7f657d764640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5644dc0c15d0 0x5644dc0c1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:54.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.114+0000 7f657d764640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5644dc0c15d0 0x5644dc0c1a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:60002/0 (socket says 172.21.15.26:60002) 2024-09-11T23:20:54.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.114+0000 7f657d764640 1 -- 172.21.15.26:0/4127246120 learned_addr learned my addr 172.21.15.26:0/4127246120 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:20:54.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.114+0000 7f657e766640 1 --2- 172.21.15.26:0/4127246120 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5644dc0cc370 0x5644dc0ce760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:54.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.115+0000 7f657d764640 1 -- 172.21.15.26:0/4127246120 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5644dc0cc370 msgr2=0x5644dc0ce760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:20:54.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.115+0000 7f657d764640 1 --2- 172.21.15.26:0/4127246120 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5644dc0cc370 0x5644dc0ce760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:54.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.115+0000 7f657d764640 1 -- 172.21.15.26:0/4127246120 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5644dc0c2b40 msgr2=0x5644dc0c1090 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T23:20:54.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.115+0000 7f657d764640 1 --2- 172.21.15.26:0/4127246120 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5644dc0c2b40 0x5644dc0c1090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:54.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.115+0000 7f657d764640 1 -- 172.21.15.26:0/4127246120 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5644dbfb33a0 con 0x5644dc0c15d0 2024-09-11T23:20:54.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.115+0000 7f657d764640 1 --2- 172.21.15.26:0/4127246120 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5644dc0c15d0 0x5644dc0c1a10 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f65680b91f0 tx=0x7f6568030a10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=618f5e1510979d63 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:54.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.115+0000 7f6563fff640 1 -- 172.21.15.26:0/4127246120 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65680bcf20 con 0x5644dc0c15d0 2024-09-11T23:20:54.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.115+0000 7f6563fff640 1 -- 172.21.15.26:0/4127246120 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f65680bc820 con 0x5644dc0c15d0 2024-09-11T23:20:54.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.115+0000 7f6563fff640 1 -- 172.21.15.26:0/4127246120 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65680edc40 con 0x5644dc0c15d0 2024-09-11T23:20:54.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.116+0000 7f657df65640 1 --2- 172.21.15.26:0/454794238 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f65640a2d70 0x7f65640a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:54.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.116+0000 7f65825a2740 1 -- 172.21.15.26:0/4127246120 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5644dc0c15d0 msgr2=0x5644dc0c1a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:54.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.116+0000 7f65825a2740 1 --2- 172.21.15.26:0/4127246120 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5644dc0c15d0 0x5644dc0c1a10 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f65680b91f0 tx=0x7f6568030a10 comp rx=0 tx=0).stop 2024-09-11T23:20:54.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.116+0000 7f65825a2740 1 -- 172.21.15.26:0/4127246120 shutdown_connections 2024-09-11T23:20:54.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.116+0000 7f65825a2740 1 --2- 172.21.15.26:0/4127246120 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5644dc0cc370 0x5644dc0ce760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:54.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.116+0000 7f65825a2740 1 --2- 172.21.15.26:0/4127246120 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5644dc0c15d0 0x5644dc0c1a10 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:54.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.116+0000 7f65825a2740 1 --2- 172.21.15.26:0/4127246120 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5644dc0c2b40 0x5644dc0c1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:54.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.116+0000 7f65825a2740 1 -- 172.21.15.26:0/4127246120 >> 172.21.15.26:0/4127246120 conn(0x5644dc08a310 msgr2=0x5644dc0bdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:54.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.117+0000 7f657df65640 1 --2- 172.21.15.26:0/454794238 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f65640a2d70 0x7f65640a5230 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f6574005c80 tx=0x7f6574009260 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:54.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.117+0000 7f65825a2740 1 -- 172.21.15.26:0/4127246120 shutdown_connections 2024-09-11T23:20:54.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.117+0000 7f65825a2740 1 -- 172.21.15.26:0/4127246120 wait complete. 2024-09-11T23:20:54.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.117+0000 7f65825a2740 1 -- start start 2024-09-11T23:20:54.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.117+0000 7f65825a2740 1 Processor -- start 2024-09-11T23:20:54.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.118+0000 7f65825a2740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5644dc0c15d0 0x5644dc0cbd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:54.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.118+0000 7f65825a2740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5644dc0c2b40 0x5644dc0c9000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:54.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.118+0000 7f65825a2740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5644dc0cc370 0x5644dc0c95d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:54.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.118+0000 7f65825a2740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5644dc0cee30 con 0x5644dc0cc370 2024-09-11T23:20:54.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.118+0000 7f65825a2740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5644dbfe2060 con 0x5644dc0c15d0 2024-09-11T23:20:54.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.118+0000 7f65825a2740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5644dc0cefb0 con 0x5644dc0c2b40 2024-09-11T23:20:54.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.118+0000 7f65825a2740 10 client.0 _ll_register_callbacks cb 0x5644dbd358c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:20:54.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.118+0000 7f657df65640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5644dc0c15d0 0x5644dc0cbd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:54.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.118+0000 7f657df65640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5644dc0c15d0 0x5644dc0cbd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:45344/0 (socket says 172.21.15.26:45344) 2024-09-11T23:20:54.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.118+0000 7f657df65640 1 -- 172.21.15.26:0/2033067745 learned_addr learned my addr 172.21.15.26:0/2033067745 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:20:54.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.118+0000 7f657e766640 1 --2- 172.21.15.26:0/2033067745 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5644dc0cc370 0x5644dc0c95d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:54.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.118+0000 7f657d764640 1 --2- 172.21.15.26:0/2033067745 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5644dc0c2b40 0x5644dc0c9000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:54.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.118+0000 7f657df65640 1 -- 172.21.15.26:0/2033067745 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5644dc0c2b40 msgr2=0x5644dc0c9000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:20:54.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.118+0000 7f657df65640 1 --2- 172.21.15.26:0/2033067745 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5644dc0c2b40 0x5644dc0c9000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:54.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.118+0000 7f657df65640 1 -- 172.21.15.26:0/2033067745 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5644dc0cc370 msgr2=0x5644dc0c95d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:20:54.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.118+0000 7f657df65640 1 --2- 172.21.15.26:0/2033067745 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5644dc0cc370 0x5644dc0c95d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:54.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.118+0000 7f657df65640 1 -- 172.21.15.26:0/2033067745 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644dbd983c0 con 0x5644dc0c15d0 2024-09-11T23:20:54.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.119+0000 7f657df65640 1 --2- 172.21.15.26:0/2033067745 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5644dc0c15d0 0x5644dc0cbd50 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f6574004610 tx=0x7f6574004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c9c5174eade947a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:54.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.119+0000 7f65617fa640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-11T23:20:54.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.119+0000 7f65617fa640 1 -- 172.21.15.26:0/2033067745 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f657401d810 con 0x5644dc0c15d0 2024-09-11T23:20:54.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.119+0000 7f65617fa640 1 -- 172.21.15.26:0/2033067745 <== mon.2 v2:172.21.15.76:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f657401d9b0 con 0x5644dc0c15d0 2024-09-11T23:20:54.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.119+0000 7f65825a2740 1 -- 172.21.15.26:0/2033067745 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f65680bc820 con 0x5644dc0c15d0 2024-09-11T23:20:54.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.119+0000 7f65617fa640 1 -- 172.21.15.26:0/2033067745 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f657401dbb0 con 0x5644dc0c15d0 2024-09-11T23:20:54.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.119+0000 7f65825a2740 10 client.25730 fetch_fsmap learned FSMap version 33 2024-09-11T23:20:54.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.119+0000 7f65825a2740 1 -- 172.21.15.26:0/2033067745 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5644dc0e30b0 con 0x5644dc0c15d0 2024-09-11T23:20:54.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.120+0000 7f65617fa640 1 -- 172.21.15.26:0/2033067745 <== mon.2 v2:172.21.15.76:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f65680bc820 con 0x5644dc0c15d0 2024-09-11T23:20:54.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.120+0000 7f65617fa640 1 -- 172.21.15.26:0/2033067745 <== mon.2 v2:172.21.15.76:3300/0 5 ==== fsmap(e 33) ==== 8394+0+0 (secure 0 0 0) 0x7f657401a030 con 0x5644dc0c15d0 2024-09-11T23:20:54.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.120+0000 7f65825a2740 10 client.25730 fetch_fsmap finished waiting for FSMap version 33 2024-09-11T23:20:54.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.120+0000 7f65825a2740 10 client.25730 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:20:54.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.120+0000 7f65825a2740 20 client.25730 populate_metadata read hostname 'smithi026' 2024-09-11T23:20:54.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.120+0000 7f65825a2740 1 --2- 172.21.15.26:0/2033067745 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5644dc0e4570 0x5644dc0e69b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:54.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.120+0000 7f65825a2740 4 client.25730 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:20:54.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.120+0000 7f65825a2740 1 -- 172.21.15.26:0/2033067745 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5644dc034860 con 0x5644dc0e4570 2024-09-11T23:20:54.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.121+0000 7f657d764640 1 --2- 172.21.15.26:0/2033067745 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5644dc0e4570 0x5644dc0e69b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:54.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.122+0000 7f65617fa640 10 client.25730 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:20:54.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.122+0000 7f657d764640 1 --2- 172.21.15.26:0/2033067745 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5644dc0e4570 0x5644dc0e69b0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6455f7807e81fbcf server_cookie=b071d931430fdefd in_seq=0 out_seq=0 2024-09-11T23:20:54.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.124+0000 7f65617fa640 1 -- 172.21.15.26:0/2033067745 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5644dc034860 con 0x5644dc0e4570 2024-09-11T23:20:54.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.124+0000 7f65617fa640 10 client.25730 handle_command_reply: tid=0 2024-09-11T23:20:54.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.124+0000 7f65825a2740 1 client.25730 shutdown 2024-09-11T23:20:54.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.124+0000 7f65825a2740 1 -- 172.21.15.26:0/2033067745 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5644dc0c15d0 msgr2=0x5644dc0cbd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:54.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.124+0000 7f65825a2740 1 --2- 172.21.15.26:0/2033067745 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5644dc0c15d0 0x5644dc0cbd50 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f6574004610 tx=0x7f6574004730 comp rx=0 tx=0).stop 2024-09-11T23:20:54.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.125+0000 7f65825a2740 1 -- 172.21.15.26:0/2033067745 shutdown_connections 2024-09-11T23:20:54.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.125+0000 7f65825a2740 1 --2- 172.21.15.26:0/2033067745 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5644dc0e4570 0x5644dc0e69b0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:54.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.125+0000 7f65825a2740 1 --2- 172.21.15.26:0/2033067745 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5644dc0cc370 0x5644dc0c95d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:54.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.125+0000 7f65825a2740 1 --2- 172.21.15.26:0/2033067745 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5644dc0c2b40 0x5644dc0c9000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:54.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.125+0000 7f65825a2740 1 --2- 172.21.15.26:0/2033067745 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5644dc0c15d0 0x5644dc0cbd50 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:54.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.125+0000 7f65825a2740 1 -- 172.21.15.26:0/2033067745 >> 172.21.15.26:0/2033067745 conn(0x5644dc08a310 msgr2=0x5644dc0c3930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:54.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.125+0000 7f65617fa640 0 client.25730 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:20:54.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.125+0000 7f65825a2740 1 -- 172.21.15.26:0/2033067745 shutdown_connections 2024-09-11T23:20:54.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.125+0000 7f65825a2740 1 -- 172.21.15.26:0/2033067745 wait complete. 2024-09-11T23:20:54.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.127+0000 7f65825a2740 20 client.25730 trim_cache size 0 max 16384 2024-09-11T23:20:54.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.143+0000 7f65825a2740 1 Processor -- start 2024-09-11T23:20:54.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.144+0000 7f65825a2740 1 -- start start 2024-09-11T23:20:54.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.144+0000 7f65825a2740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5644dc0c15d0 0x5644dc0bde10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:54.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.144+0000 7f65825a2740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5644dc0c2b40 0x5644dc0e08a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:54.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.144+0000 7f657df65640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5644dc0c15d0 0x5644dc0bde10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:54.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.144+0000 7f657d764640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5644dc0c2b40 0x5644dc0e08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:54.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.144+0000 7f657df65640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5644dc0c15d0 0x5644dc0bde10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:60018/0 (socket says 172.21.15.26:60018) 2024-09-11T23:20:54.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.144+0000 7f657d764640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5644dc0c2b40 0x5644dc0e08a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50184/0 (socket says 172.21.15.26:50184) 2024-09-11T23:20:54.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.144+0000 7f657df65640 1 -- 172.21.15.26:0/1989815710 learned_addr learned my addr 172.21.15.26:0/1989815710 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:20:54.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.144+0000 7f65825a2740 1 --2- 172.21.15.26:0/1989815710 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5644dc0cc370 0x5644dc0c88c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:54.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.144+0000 7f65825a2740 1 -- 172.21.15.26:0/1989815710 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f65781ab7a0 con 0x5644dc0c2b40 2024-09-11T23:20:54.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.144+0000 7f65825a2740 1 -- 172.21.15.26:0/1989815710 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f65781ad460 con 0x5644dc0cc370 2024-09-11T23:20:54.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.144+0000 7f65825a2740 1 -- 172.21.15.26:0/1989815710 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f65781abe70 con 0x5644dc0c15d0 2024-09-11T23:20:54.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.145+0000 7f657e766640 1 --2- 172.21.15.26:0/1989815710 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5644dc0cc370 0x5644dc0c88c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:54.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.145+0000 7f657df65640 1 -- 172.21.15.26:0/1989815710 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5644dc0cc370 msgr2=0x5644dc0c88c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:20:54.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.145+0000 7f657df65640 1 --2- 172.21.15.26:0/1989815710 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5644dc0cc370 0x5644dc0c88c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:54.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.145+0000 7f657df65640 1 -- 172.21.15.26:0/1989815710 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5644dc0c2b40 msgr2=0x5644dc0e08a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:20:54.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.145+0000 7f657df65640 1 --2- 172.21.15.26:0/1989815710 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5644dc0c2b40 0x5644dc0e08a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:54.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.145+0000 7f657df65640 1 -- 172.21.15.26:0/1989815710 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5644dc0c7fb0 con 0x5644dc0c15d0 2024-09-11T23:20:54.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.145+0000 7f657d764640 1 --2- 172.21.15.26:0/1989815710 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5644dc0c2b40 0x5644dc0e08a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:20:54.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.145+0000 7f657df65640 1 --2- 172.21.15.26:0/1989815710 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5644dc0c15d0 0x5644dc0bde10 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f65740045e0 tx=0x7f65740095a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d1fa3f6bc0ce0118 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:54.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.146+0000 7f65637fe640 1 -- 172.21.15.26:0/1989815710 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6574034070 con 0x5644dc0c15d0 2024-09-11T23:20:54.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.146+0000 7f65637fe640 1 -- 172.21.15.26:0/1989815710 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6574022070 con 0x5644dc0c15d0 2024-09-11T23:20:54.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.147+0000 7f65637fe640 1 -- 172.21.15.26:0/1989815710 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6574027470 con 0x5644dc0c15d0 2024-09-11T23:20:54.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.147+0000 7f65825a2740 1 -- 172.21.15.26:0/1989815710 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5644dc0c15d0 msgr2=0x5644dc0bde10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:54.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.147+0000 7f65825a2740 1 --2- 172.21.15.26:0/1989815710 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5644dc0c15d0 0x5644dc0bde10 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f65740045e0 tx=0x7f65740095a0 comp rx=0 tx=0).stop 2024-09-11T23:20:54.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.147+0000 7f65825a2740 1 -- 172.21.15.26:0/1989815710 shutdown_connections 2024-09-11T23:20:54.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.147+0000 7f65825a2740 1 --2- 172.21.15.26:0/1989815710 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5644dc0cc370 0x5644dc0c88c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:54.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.147+0000 7f65825a2740 1 --2- 172.21.15.26:0/1989815710 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5644dc0c2b40 0x5644dc0e08a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:54.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.147+0000 7f65825a2740 1 --2- 172.21.15.26:0/1989815710 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5644dc0c15d0 0x5644dc0bde10 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:54.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.147+0000 7f65825a2740 1 -- 172.21.15.26:0/1989815710 >> 172.21.15.26:0/1989815710 conn(0x5644dc08a310 msgr2=0x5644dc0bd700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:54.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.147+0000 7f65825a2740 1 -- 172.21.15.26:0/1989815710 shutdown_connections 2024-09-11T23:20:54.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.147+0000 7f65825a2740 1 -- 172.21.15.26:0/1989815710 wait complete. 2024-09-11T23:20:54.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.148+0000 7f65825a2740 1 -- start start 2024-09-11T23:20:54.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.148+0000 7f65825a2740 1 Processor -- start 2024-09-11T23:20:54.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.148+0000 7f65825a2740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5644dc0c15d0 0x5644dc0d7f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:54.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.148+0000 7f65825a2740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5644dc0c2b40 0x5644dc0d84d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:54.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.148+0000 7f65825a2740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5644dc0cc370 0x5644dc14d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:54.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.148+0000 7f65825a2740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f65781accd0 con 0x5644dc0cc370 2024-09-11T23:20:54.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.149+0000 7f65825a2740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f65781ac540 con 0x5644dc0c15d0 2024-09-11T23:20:54.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.149+0000 7f65825a2740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f65781a97e0 con 0x5644dc0c2b40 2024-09-11T23:20:54.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.149+0000 7f65825a2740 10 client.0 _ll_register_callbacks cb 0x7f6574009d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:20:54.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.149+0000 7f657e766640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5644dc0cc370 0x5644dc14d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:54.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.149+0000 7f657e766640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5644dc0cc370 0x5644dc14d240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50186/0 (socket says 172.21.15.26:50186) 2024-09-11T23:20:54.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.149+0000 7f657e766640 1 -- 172.21.15.26:0/278436015 learned_addr learned my addr 172.21.15.26:0/278436015 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:20:54.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.149+0000 7f657d764640 1 --2- 172.21.15.26:0/278436015 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5644dc0c2b40 0x5644dc0d84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:54.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.149+0000 7f657df65640 1 --2- 172.21.15.26:0/278436015 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5644dc0c15d0 0x5644dc0d7f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:54.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.149+0000 7f657e766640 1 -- 172.21.15.26:0/278436015 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5644dc0c2b40 msgr2=0x5644dc0d84d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:20:54.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.149+0000 7f657e766640 1 --2- 172.21.15.26:0/278436015 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5644dc0c2b40 0x5644dc0d84d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:54.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.149+0000 7f657e766640 1 -- 172.21.15.26:0/278436015 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5644dc0c15d0 msgr2=0x5644dc0d7f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:20:54.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.149+0000 7f657e766640 1 --2- 172.21.15.26:0/278436015 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5644dc0c15d0 0x5644dc0d7f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:54.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.149+0000 7f657e766640 1 -- 172.21.15.26:0/278436015 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644dc14d800 con 0x5644dc0cc370 2024-09-11T23:20:54.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.149+0000 7f657df65640 1 --2- 172.21.15.26:0/278436015 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5644dc0c15d0 0x5644dc0d7f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:20:54.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.150+0000 7f657e766640 1 --2- 172.21.15.26:0/278436015 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5644dc0cc370 0x5644dc14d240 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f6570004330 tx=0x7f65700047f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cceaece4f3250be1 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:54.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.150+0000 7f65617fa640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:20:54.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.150+0000 7f65617fa640 1 -- 172.21.15.26:0/278436015 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6570022060 con 0x5644dc0cc370 2024-09-11T23:20:54.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.150+0000 7f65617fa640 1 -- 172.21.15.26:0/278436015 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f6570012bb0 con 0x5644dc0cc370 2024-09-11T23:20:54.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.150+0000 7f65825a2740 1 -- 172.21.15.26:0/278436015 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6574022070 con 0x5644dc0cc370 2024-09-11T23:20:54.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.150+0000 7f65617fa640 1 -- 172.21.15.26:0/278436015 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f6570012db0 con 0x5644dc0cc370 2024-09-11T23:20:54.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.150+0000 7f65825a2740 10 client.16158 fetch_fsmap learned FSMap version 33 2024-09-11T23:20:54.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.150+0000 7f65825a2740 1 -- 172.21.15.26:0/278436015 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5644dc14eb40 con 0x5644dc0cc370 2024-09-11T23:20:54.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.151+0000 7f65617fa640 1 -- 172.21.15.26:0/278436015 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6574022070 con 0x5644dc0cc370 2024-09-11T23:20:54.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.151+0000 7f65617fa640 1 -- 172.21.15.26:0/278436015 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 33) ==== 8394+0+0 (secure 0 0 0) 0x7f6570025510 con 0x5644dc0cc370 2024-09-11T23:20:54.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.151+0000 7f65825a2740 10 client.16158 fetch_fsmap finished waiting for FSMap version 33 2024-09-11T23:20:54.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.151+0000 7f65825a2740 10 client.16158 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:20:54.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.151+0000 7f65825a2740 20 client.16158 populate_metadata read hostname 'smithi026' 2024-09-11T23:20:54.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.151+0000 7f65825a2740 1 --2- 172.21.15.26:0/278436015 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5644dc14f380 0x5644dc1517a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:54.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.151+0000 7f65825a2740 4 client.16158 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:20:54.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.151+0000 7f65825a2740 1 -- 172.21.15.26:0/278436015 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5644dc02ed70 con 0x5644dc14f380 2024-09-11T23:20:54.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.151+0000 7f657df65640 1 --2- 172.21.15.26:0/278436015 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5644dc14f380 0x5644dc1517a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:54.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.152+0000 7f65617fa640 10 client.16158 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:20:54.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.152+0000 7f657df65640 1 --2- 172.21.15.26:0/278436015 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5644dc14f380 0x5644dc1517a0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=782fb1ae25b76b78 server_cookie=e9022443f33108cd in_seq=0 out_seq=0 2024-09-11T23:20:54.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.152+0000 7f65617fa640 1 -- 172.21.15.26:0/278436015 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5644dc02ed70 con 0x5644dc14f380 2024-09-11T23:20:54.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.152+0000 7f65617fa640 10 client.16158 handle_command_reply: tid=0 2024-09-11T23:20:54.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.152+0000 7f65825a2740 1 client.16158 shutdown 2024-09-11T23:20:54.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.152+0000 7f65825a2740 1 -- 172.21.15.26:0/278436015 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5644dc0cc370 msgr2=0x5644dc14d240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:54.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.152+0000 7f65825a2740 1 --2- 172.21.15.26:0/278436015 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5644dc0cc370 0x5644dc14d240 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f6570004330 tx=0x7f65700047f0 comp rx=0 tx=0).stop 2024-09-11T23:20:54.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.152+0000 7f65825a2740 1 -- 172.21.15.26:0/278436015 shutdown_connections 2024-09-11T23:20:54.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.152+0000 7f65825a2740 1 --2- 172.21.15.26:0/278436015 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5644dc14f380 0x5644dc1517a0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:54.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.152+0000 7f65825a2740 1 --2- 172.21.15.26:0/278436015 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5644dc0cc370 0x5644dc14d240 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:54.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.152+0000 7f65825a2740 1 --2- 172.21.15.26:0/278436015 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5644dc0c2b40 0x5644dc0d84d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:54.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.152+0000 7f65825a2740 1 --2- 172.21.15.26:0/278436015 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5644dc0c15d0 0x5644dc0d7f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:54.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.153+0000 7f65825a2740 1 -- 172.21.15.26:0/278436015 >> 172.21.15.26:0/278436015 conn(0x5644dc08a310 msgr2=0x5644dc0cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:54.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.153+0000 7f65617fa640 0 client.16158 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:20:54.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.153+0000 7f65825a2740 1 -- 172.21.15.26:0/278436015 shutdown_connections 2024-09-11T23:20:54.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.153+0000 7f65825a2740 1 -- 172.21.15.26:0/278436015 wait complete. 2024-09-11T23:20:54.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.153+0000 7f65825a2740 20 client.16158 trim_cache size 0 max 16384 2024-09-11T23:20:54.155 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2024-09-11T23:20:54.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.154+0000 7f65801c9640 1 -- 172.21.15.26:0/454794238 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f65640a2d70 msgr2=0x7f65640a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:54.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.154+0000 7f65801c9640 1 --2- 172.21.15.26:0/454794238 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f65640a2d70 0x7f65640a5230 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f6574005c80 tx=0x7f6574009260 comp rx=0 tx=0).stop 2024-09-11T23:20:54.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.155+0000 7f65801c9640 1 -- 172.21.15.26:0/454794238 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6578102570 msgr2=0x7f65781a0990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:54.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.155+0000 7f65801c9640 1 --2- 172.21.15.26:0/454794238 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6578102570 0x7f65781a0990 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f6568009890 tx=0x7f6568030920 comp rx=0 tx=0).stop 2024-09-11T23:20:54.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.155+0000 7f65801c9640 1 -- 172.21.15.26:0/454794238 shutdown_connections 2024-09-11T23:20:54.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.155+0000 7f65801c9640 1 --2- 172.21.15.26:0/454794238 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f65640a2d70 0x7f65640a5230 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:54.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.156+0000 7f65801c9640 1 --2- 172.21.15.26:0/454794238 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6578102ef0 0x7f657819a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:54.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.156+0000 7f65801c9640 1 --2- 172.21.15.26:0/454794238 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6578102570 0x7f65781a0990 secure :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f6568009890 tx=0x7f6568030920 comp rx=0 tx=0).stop 2024-09-11T23:20:54.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.156+0000 7f65801c9640 1 --2- 172.21.15.26:0/454794238 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6578100e20 0x7f65781a0450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:54.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.156+0000 7f65801c9640 1 -- 172.21.15.26:0/454794238 >> 172.21.15.26:0/454794238 conn(0x7f65780fc7d0 msgr2=0x7f657806a0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:54.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.156+0000 7f65801c9640 1 -- 172.21.15.26:0/454794238 shutdown_connections 2024-09-11T23:20:54.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:20:54.156+0000 7f65801c9640 1 -- 172.21.15.26:0/454794238 wait complete. 2024-09-11T23:20:55.121 ERROR:tasks.fwd_scrub:rank 1 damaged: [{'damage_type': 'backtrace', 'id': 59548436, 'ino': 1099511628312, 'path': '/volumes/qa/sv_1/9fc8f023-d646-4fec-98c7-1f3d972f60a9/client.0/tmp/blogbench-1.0/src/reader.c'}, {'damage_type': 'backtrace', 'id': 60730685, 'ino': 1099511628678, 'path': '/volumes/qa/sv_1/9fc8f023-d646-4fec-98c7-1f3d972f60a9/client.0/tmp/blogbench-1.0/src/.deps/process.Po'}, {'damage_type': 'backtrace', 'id': 88304087, 'ino': 1099511628307, 'path': '/volumes/qa/sv_1/9fc8f023-d646-4fec-98c7-1f3d972f60a9/client.0/tmp/blogbench-1.0/src/Makefile.in'}, {'damage_type': 'backtrace', 'id': 195604997, 'ino': 1099511628300, 'path': '/volumes/qa/sv_1/9fc8f023-d646-4fec-98c7-1f3d972f60a9/client.0/tmp/blogbench-1.0/src/blogbench.h'}, {'damage_type': 'backtrace', 'id': 373890108, 'ino': 1099511628299, 'path': '/volumes/qa/sv_1/9fc8f023-d646-4fec-98c7-1f3d972f60a9/client.0/tmp/blogbench-1.0/src/blogbench.c'}, {'damage_type': 'backtrace', 'id': 553887758, 'ino': 1099511628306, 'path': '/volumes/qa/sv_1/9fc8f023-d646-4fec-98c7-1f3d972f60a9/client.0/tmp/blogbench-1.0/src/Makefile.am'}, {'damage_type': 'backtrace', 'id': 671366844, 'ino': 1099511628302, 'path': '/volumes/qa/sv_1/9fc8f023-d646-4fec-98c7-1f3d972f60a9/client.0/tmp/blogbench-1.0/src/fakesnprintf.h'}, {'damage_type': 'backtrace', 'id': 900911283, 'ino': 1099511628301, 'path': '/volumes/qa/sv_1/9fc8f023-d646-4fec-98c7-1f3d972f60a9/client.0/tmp/blogbench-1.0/src/fakesnprintf.c'}, {'damage_type': 'backtrace', 'id': 926345818, 'ino': 1099511628303, 'path': '/volumes/qa/sv_1/9fc8f023-d646-4fec-98c7-1f3d972f60a9/client.0/tmp/blogbench-1.0/src/blogbench_p.h'}, {'damage_type': 'backtrace', 'id': 1256918086, 'ino': 1099511628657, 'path': '/volumes/qa/sv_1/9fc8f023-d646-4fec-98c7-1f3d972f60a9/client.0/tmp/blogbench-1.0/src/Makefile'}, {'damage_type': 'backtrace', 'id': 1482542569, 'ino': 1099511628308, 'path': '/volumes/qa/sv_1/9fc8f023-d646-4fec-98c7-1f3d972f60a9/client.0/tmp/blogbench-1.0/src/process.c'}, {'damage_type': 'backtrace', 'id': 1599535364, 'ino': 1099511628313, 'path': '/volumes/qa/sv_1/9fc8f023-d646-4fec-98c7-1f3d972f60a9/client.0/tmp/blogbench-1.0/src/writer.c'}, {'damage_type': 'backtrace', 'id': 1756046412, 'ino': 1099511628679, 'path': '/volumes/qa/sv_1/9fc8f023-d646-4fec-98c7-1f3d972f60a9/client.0/tmp/blogbench-1.0/src/process.o'}, {'damage_type': 'backtrace', 'id': 2032523121, 'ino': 1099511628665, 'path': '/volumes/qa/sv_1/9fc8f023-d646-4fec-98c7-1f3d972f60a9/client.0/tmp/blogbench-1.0/src/.deps'}, {'damage_type': 'backtrace', 'id': 2197745511, 'ino': 1099511628305, 'path': '/volumes/qa/sv_1/9fc8f023-d646-4fec-98c7-1f3d972f60a9/client.0/tmp/blogbench-1.0/src/bsd-getopt_long.h'}, {'damage_type': 'backtrace', 'id': 2239312215, 'ino': 1099511628309, 'path': '/volumes/qa/sv_1/9fc8f023-d646-4fec-98c7-1f3d972f60a9/client.0/tmp/blogbench-1.0/src/globals.h'}, {'damage_type': 'backtrace', 'id': 2442947403, 'ino': 1099511628311, 'path': '/volumes/qa/sv_1/9fc8f023-d646-4fec-98c7-1f3d972f60a9/client.0/tmp/blogbench-1.0/src/rewriter.c'}, {'damage_type': 'backtrace', 'id': 2585114484, 'ino': 1099511628677, 'path': '/volumes/qa/sv_1/9fc8f023-d646-4fec-98c7-1f3d972f60a9/client.0/tmp/blogbench-1.0/src/blogbench.o'}, {'damage_type': 'backtrace', 'id': 2754559038, 'ino': 1099511628298, 'path': '/volumes/qa/sv_1/9fc8f023-d646-4fec-98c7-1f3d972f60a9/client.0/tmp/blogbench-1.0/src/mysnprintf.h'}, {'damage_type': 'backtrace', 'id': 3160277156, 'ino': 1099511628314, 'path': '/volumes/qa/sv_1/9fc8f023-d646-4fec-98c7-1f3d972f60a9/client.0/tmp/blogbench-1.0/src/helpers.c'}, {'damage_type': 'backtrace', 'id': 3611183181, 'ino': 1099511628310, 'path': '/volumes/qa/sv_1/9fc8f023-d646-4fec-98c7-1f3d972f60a9/client.0/tmp/blogbench-1.0/src/commenter.c'}, {'damage_type': 'backtrace', 'id': 3728318696, 'ino': 1099511628304, 'path': '/volumes/qa/sv_1/9fc8f023-d646-4fec-98c7-1f3d972f60a9/client.0/tmp/blogbench-1.0/src/bsd-getopt_long.c'}, {'damage_type': 'backtrace', 'id': 3916915527, 'ino': 1099511628676, 'path': '/volumes/qa/sv_1/9fc8f023-d646-4fec-98c7-1f3d972f60a9/client.0/tmp/blogbench-1.0/src/.deps/blogbench.Po'}, {'damage_type': 'backtrace', 'id': 4228120609, 'ino': 1099511628297, 'path': '/volumes/qa/sv_1/9fc8f023-d646-4fec-98c7-1f3d972f60a9/client.0/tmp/blogbench-1.0/src/mysnprintf.c'}] 2024-09-11T23:20:55.121 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph_086c737fa75c22d521a3932c59b31c5e1d1c9ef4/qa/tasks/fwd_scrub.py", line 35, in _run self.do_scrub() File "/home/teuthworker/src/git.ceph.com_ceph_086c737fa75c22d521a3932c59b31c5e1d1c9ef4/qa/tasks/fwd_scrub.py", line 48, in do_scrub self._scrub() File "/home/teuthworker/src/git.ceph.com_ceph_086c737fa75c22d521a3932c59b31c5e1d1c9ef4/qa/tasks/fwd_scrub.py", line 70, in _scrub self._check_damage() File "/home/teuthworker/src/git.ceph.com_ceph_086c737fa75c22d521a3932c59b31c5e1d1c9ef4/qa/tasks/fwd_scrub.py", line 81, in _check_damage raise RuntimeError(f"rank damage found: {types}") RuntimeError: rank damage found: {'backtrace'} 2024-09-11T23:20:55.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:55 smithi136 ceph-mon[35121]: pgmap v79: 97 pgs: 97 active+clean; 250 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.3 MiB/s wr, 197 op/s 2024-09-11T23:20:55.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:55 smithi026 ceph-mon[31172]: pgmap v79: 97 pgs: 97 active+clean; 250 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.3 MiB/s wr, 197 op/s 2024-09-11T23:20:55.802 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:55 smithi076 ceph-mon[35296]: pgmap v79: 97 pgs: 97 active+clean; 250 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.3 MiB/s wr, 197 op/s 2024-09-11T23:20:57.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:57 smithi136 ceph-mon[35121]: pgmap v80: 97 pgs: 97 active+clean; 250 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 156 op/s 2024-09-11T23:20:57.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:57 smithi026 ceph-mon[31172]: pgmap v80: 97 pgs: 97 active+clean; 250 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 156 op/s 2024-09-11T23:20:57.802 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:57 smithi076 ceph-mon[35296]: pgmap v80: 97 pgs: 97 active+clean; 250 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 156 op/s 2024-09-11T23:20:58.764 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.764+0000 7fea9839f640 1 -- 172.21.15.26:0/2807000359 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fea901026f0 msgr2=0x7fea9010d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:58.764 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.764+0000 7fea9839f640 1 --2- 172.21.15.26:0/2807000359 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fea901026f0 0x7fea9010d110 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fea88009cd0 tx=0x7fea8802a8e0 comp rx=0 tx=0).stop 2024-09-11T23:20:58.764 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.764+0000 7fea9839f640 1 -- 172.21.15.26:0/2807000359 shutdown_connections 2024-09-11T23:20:58.764 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.764+0000 7fea9839f640 1 --2- 172.21.15.26:0/2807000359 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fea901026f0 0x7fea9010d110 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:58.764 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.764+0000 7fea9839f640 1 --2- 172.21.15.26:0/2807000359 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fea90101d70 0x7fea901021b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:58.764 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.764+0000 7fea9839f640 1 --2- 172.21.15.26:0/2807000359 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fea90100490 0x7fea90100870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:58.764 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.764+0000 7fea9839f640 1 -- 172.21.15.26:0/2807000359 >> 172.21.15.26:0/2807000359 conn(0x7fea900770f0 msgr2=0x7fea900fe480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:58.765 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.764+0000 7fea9839f640 1 -- 172.21.15.26:0/2807000359 shutdown_connections 2024-09-11T23:20:58.765 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.764+0000 7fea9839f640 1 -- 172.21.15.26:0/2807000359 wait complete. 2024-09-11T23:20:58.765 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.765+0000 7fea9839f640 1 Processor -- start 2024-09-11T23:20:58.765 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.765+0000 7fea9839f640 1 -- start start 2024-09-11T23:20:58.765 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.766+0000 7fea9839f640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fea90100490 0x7fea90196150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:58.766 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.766+0000 7fea9839f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fea90101d70 0x7fea90196690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:58.766 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.766+0000 7fea9839f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fea901026f0 0x7fea9019aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:58.766 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.766+0000 7fea9839f640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fea9010fe20 con 0x7fea90101d70 2024-09-11T23:20:58.766 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.766+0000 7fea9839f640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fea9010fca0 con 0x7fea901026f0 2024-09-11T23:20:58.766 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.766+0000 7fea9839f640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fea9010ffa0 con 0x7fea90100490 2024-09-11T23:20:58.767 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.766+0000 7fea9613b640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fea90100490 0x7fea90196150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:58.767 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.766+0000 7fea9593a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fea90101d70 0x7fea90196690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:58.767 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.766+0000 7fea9593a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fea90101d70 0x7fea90196690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50198/0 (socket says 172.21.15.26:50198) 2024-09-11T23:20:58.767 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.766+0000 7fea9593a640 1 -- 172.21.15.26:0/2330652035 learned_addr learned my addr 172.21.15.26:0/2330652035 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:20:58.768 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.766+0000 7fea9613b640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fea90100490 0x7fea90196150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:60040/0 (socket says 172.21.15.26:60040) 2024-09-11T23:20:58.768 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.766+0000 7fea9693c640 1 --2- 172.21.15.26:0/2330652035 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fea901026f0 0x7fea9019aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:58.768 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.766+0000 7fea9593a640 1 -- 172.21.15.26:0/2330652035 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fea90100490 msgr2=0x7fea90196150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:58.768 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.766+0000 7fea9593a640 1 --2- 172.21.15.26:0/2330652035 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fea90100490 0x7fea90196150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:58.768 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.766+0000 7fea9593a640 1 -- 172.21.15.26:0/2330652035 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fea901026f0 msgr2=0x7fea9019aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:58.768 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.766+0000 7fea9593a640 1 --2- 172.21.15.26:0/2330652035 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fea901026f0 0x7fea9019aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:58.768 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.767+0000 7fea9593a640 1 -- 172.21.15.26:0/2330652035 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea9019b140 con 0x7fea90101d70 2024-09-11T23:20:58.768 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.767+0000 7fea9593a640 1 --2- 172.21.15.26:0/2330652035 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fea90101d70 0x7fea90196690 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fea8000e9f0 tx=0x7fea8000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:58.768 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.767+0000 7fea877fe640 1 -- 172.21.15.26:0/2330652035 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea8000cd80 con 0x7fea90101d70 2024-09-11T23:20:58.768 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.767+0000 7fea877fe640 1 -- 172.21.15.26:0/2330652035 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea80010910 con 0x7fea90101d70 2024-09-11T23:20:58.768 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.767+0000 7fea877fe640 1 -- 172.21.15.26:0/2330652035 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea80018990 con 0x7fea90101d70 2024-09-11T23:20:58.768 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.767+0000 7fea9839f640 1 -- 172.21.15.26:0/2330652035 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea9019b430 con 0x7fea90101d70 2024-09-11T23:20:58.768 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.767+0000 7fea9839f640 1 -- 172.21.15.26:0/2330652035 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea901a70d0 con 0x7fea90101d70 2024-09-11T23:20:58.770 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.770+0000 7fea877fe640 1 -- 172.21.15.26:0/2330652035 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fea80010ac0 con 0x7fea90101d70 2024-09-11T23:20:58.771 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.771+0000 7fea877fe640 1 --2- 172.21.15.26:0/2330652035 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fea7c0a2e20 0x7fea7c0a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:58.772 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.772+0000 7fea877fe640 1 -- 172.21.15.26:0/2330652035 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fea800cf6b0 con 0x7fea90101d70 2024-09-11T23:20:58.772 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.772+0000 7fea9613b640 1 --2- 172.21.15.26:0/2330652035 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fea7c0a2e20 0x7fea7c0a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:58.772 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.772+0000 7fea9b21b740 1 Processor -- start 2024-09-11T23:20:58.772 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.773+0000 7fea9b21b740 1 -- start start 2024-09-11T23:20:58.774 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.773+0000 7fea9613b640 1 --2- 172.21.15.26:0/2330652035 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fea7c0a2e20 0x7fea7c0a52e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fea8c0095f0 tx=0x7fea8c005f90 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:58.774 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.773+0000 7fea9b21b740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f70585caf0 0x55f70585ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:58.774 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.773+0000 7fea9b21b740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f70585d410 0x55f705867770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:58.775 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.773+0000 7fea9b21b740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f705867cb0 0x55f70586a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:58.775 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.773+0000 7fea9593a640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f70585d410 0x55f705867770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:58.775 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.773+0000 7fea9b21b740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55f7057c12f0 con 0x55f705867cb0 2024-09-11T23:20:58.775 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.773+0000 7fea9593a640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f70585d410 0x55f705867770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:60056/0 (socket says 172.21.15.26:60056) 2024-09-11T23:20:58.775 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.773+0000 7fea9593a640 1 -- 172.21.15.26:0/1964391655 learned_addr learned my addr 172.21.15.26:0/1964391655 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:20:58.775 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.773+0000 7fea9b21b740 1 -- 172.21.15.26:0/1964391655 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55f705769270 con 0x55f70585caf0 2024-09-11T23:20:58.775 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.773+0000 7fea9b21b740 1 -- 172.21.15.26:0/1964391655 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55f7057bd2b0 con 0x55f70585d410 2024-09-11T23:20:58.775 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.773+0000 7fea9613b640 1 --2- 172.21.15.26:0/1964391655 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f70585caf0 0x55f70585ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:58.775 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.773+0000 7fea9693c640 1 --2- 172.21.15.26:0/1964391655 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f705867cb0 0x55f70586a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:58.775 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.774+0000 7fea9613b640 1 -- 172.21.15.26:0/1964391655 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f70585d410 msgr2=0x55f705867770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:20:58.775 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.774+0000 7fea9613b640 1 --2- 172.21.15.26:0/1964391655 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f70585d410 0x55f705867770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:58.775 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.774+0000 7fea9613b640 1 -- 172.21.15.26:0/1964391655 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f705867cb0 msgr2=0x55f70586a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:20:58.775 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.774+0000 7fea9613b640 1 --2- 172.21.15.26:0/1964391655 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f705867cb0 0x55f70586a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:58.776 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.774+0000 7fea9613b640 1 -- 172.21.15.26:0/1964391655 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f705768340 con 0x55f70585caf0 2024-09-11T23:20:58.776 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.774+0000 7fea9593a640 1 --2- 172.21.15.26:0/1964391655 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f70585d410 0x55f705867770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:20:58.776 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.774+0000 7fea9693c640 1 --2- 172.21.15.26:0/1964391655 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f705867cb0 0x55f70586a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:20:58.776 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.774+0000 7fea9613b640 1 --2- 172.21.15.26:0/1964391655 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f70585caf0 0x55f70585ced0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fea8c0118d0 tx=0x7fea8c014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3919fde0ce7157c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:58.776 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.774+0000 7fea84bfe640 1 -- 172.21.15.26:0/1964391655 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea8c026430 con 0x55f70585caf0 2024-09-11T23:20:58.776 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.775+0000 7fea84bfe640 1 -- 172.21.15.26:0/1964391655 <== mon.2 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea8c01c8f0 con 0x55f70585caf0 2024-09-11T23:20:58.776 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.775+0000 7fea84bfe640 1 -- 172.21.15.26:0/1964391655 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea8c01d950 con 0x55f70585caf0 2024-09-11T23:20:58.779 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.776+0000 7fea9b21b740 1 -- 172.21.15.26:0/1964391655 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f70585caf0 msgr2=0x55f70585ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:58.779 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.776+0000 7fea9b21b740 1 --2- 172.21.15.26:0/1964391655 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f70585caf0 0x55f70585ced0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fea8c0118d0 tx=0x7fea8c014000 comp rx=0 tx=0).stop 2024-09-11T23:20:58.780 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.776+0000 7fea9b21b740 1 -- 172.21.15.26:0/1964391655 shutdown_connections 2024-09-11T23:20:58.780 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.776+0000 7fea9b21b740 1 --2- 172.21.15.26:0/1964391655 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f705867cb0 0x55f70586a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:58.780 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.776+0000 7fea9b21b740 1 --2- 172.21.15.26:0/1964391655 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f70585d410 0x55f705867770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:58.780 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.776+0000 7fea9b21b740 1 --2- 172.21.15.26:0/1964391655 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f70585caf0 0x55f70585ced0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:58.780 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.776+0000 7fea9b21b740 1 -- 172.21.15.26:0/1964391655 >> 172.21.15.26:0/1964391655 conn(0x55f705859060 msgr2=0x55f705859440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:58.780 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.776+0000 7fea9b21b740 1 -- 172.21.15.26:0/1964391655 shutdown_connections 2024-09-11T23:20:58.780 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.776+0000 7fea9b21b740 1 -- 172.21.15.26:0/1964391655 wait complete. 2024-09-11T23:20:58.780 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.777+0000 7fea9b21b740 1 -- start start 2024-09-11T23:20:58.780 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.777+0000 7fea9b21b740 1 Processor -- start 2024-09-11T23:20:58.780 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.777+0000 7fea9b21b740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f70585caf0 0x55f705864bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:58.780 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.777+0000 7fea9b21b740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f70585d410 0x55f705865110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:58.780 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.777+0000 7fea9b21b740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f705867cb0 0x55f705865650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:58.781 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.777+0000 7fea9b21b740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55f70586a880 con 0x55f70585d410 2024-09-11T23:20:58.781 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.777+0000 7fea9b21b740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55f70574a290 con 0x55f705867cb0 2024-09-11T23:20:58.781 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.777+0000 7fea9b21b740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55f70586aa00 con 0x55f70585caf0 2024-09-11T23:20:58.781 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.777+0000 7fea9b21b740 10 client.0 _ll_register_callbacks cb 0x55f70577b140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:20:58.781 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.778+0000 7fea9593a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f70585d410 0x55f705865110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:58.781 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.778+0000 7fea9613b640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f70585caf0 0x55f705864bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:58.781 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.778+0000 7fea9693c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f705867cb0 0x55f705865650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:58.781 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.778+0000 7fea9613b640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f70585caf0 0x55f705864bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:60070/0 (socket says 172.21.15.26:60070) 2024-09-11T23:20:58.781 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.778+0000 7fea9613b640 1 -- 172.21.15.26:0/3945907435 learned_addr learned my addr 172.21.15.26:0/3945907435 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:20:58.781 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.778+0000 7fea9693c640 1 -- 172.21.15.26:0/3945907435 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f70585caf0 msgr2=0x55f705864bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:20:58.781 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.778+0000 7fea9693c640 1 --2- 172.21.15.26:0/3945907435 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f70585caf0 0x55f705864bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:58.781 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.778+0000 7fea9693c640 1 -- 172.21.15.26:0/3945907435 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f70585d410 msgr2=0x55f705865110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:20:58.781 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.778+0000 7fea9693c640 1 --2- 172.21.15.26:0/3945907435 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f70585d410 0x55f705865110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:58.782 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.778+0000 7fea9693c640 1 -- 172.21.15.26:0/3945907435 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f7054d4210 con 0x55f705867cb0 2024-09-11T23:20:58.782 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.778+0000 7fea9693c640 1 --2- 172.21.15.26:0/3945907435 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f705867cb0 0x55f705865650 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fea88009ca0 tx=0x7fea88031af0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e56efc761d491996 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:58.782 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.778+0000 7fea79ffb640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-11T23:20:58.782 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.778+0000 7fea79ffb640 1 -- 172.21.15.26:0/3945907435 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea88003b70 con 0x55f705867cb0 2024-09-11T23:20:58.782 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.779+0000 7fea79ffb640 1 -- 172.21.15.26:0/3945907435 <== mon.2 v2:172.21.15.76:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fea88003d10 con 0x55f705867cb0 2024-09-11T23:20:58.782 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.779+0000 7fea9b21b740 1 -- 172.21.15.26:0/3945907435 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea8c01c8f0 con 0x55f705867cb0 2024-09-11T23:20:58.782 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.779+0000 7fea79ffb640 1 -- 172.21.15.26:0/3945907435 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fea88053050 con 0x55f705867cb0 2024-09-11T23:20:58.782 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.779+0000 7fea9b21b740 10 client.25748 fetch_fsmap learned FSMap version 33 2024-09-11T23:20:58.782 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.779+0000 7fea9b21b740 1 -- 172.21.15.26:0/3945907435 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55f7055a0620 con 0x55f705867cb0 2024-09-11T23:20:58.782 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.779+0000 7fea79ffb640 1 -- 172.21.15.26:0/3945907435 <== mon.2 v2:172.21.15.76:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea8c01c8f0 con 0x55f705867cb0 2024-09-11T23:20:58.782 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.779+0000 7fea79ffb640 1 -- 172.21.15.26:0/3945907435 <== mon.2 v2:172.21.15.76:3300/0 5 ==== fsmap(e 33) ==== 8394+0+0 (secure 0 0 0) 0x7fea8804e030 con 0x55f705867cb0 2024-09-11T23:20:58.782 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.779+0000 7fea9b21b740 10 client.25748 fetch_fsmap finished waiting for FSMap version 33 2024-09-11T23:20:58.782 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.779+0000 7fea9b21b740 10 client.25748 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:20:58.782 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.779+0000 7fea9b21b740 20 client.25748 populate_metadata read hostname 'smithi026' 2024-09-11T23:20:58.783 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.779+0000 7fea9b21b740 1 --2- 172.21.15.26:0/3945907435 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55f705877c00 0x55f70587a040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:58.783 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.779+0000 7fea9b21b740 4 client.25748 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:20:58.783 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.779+0000 7fea9b21b740 1 -- 172.21.15.26:0/3945907435 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f7057cea20 con 0x55f705877c00 2024-09-11T23:20:58.783 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.780+0000 7fea9613b640 1 --2- 172.21.15.26:0/3945907435 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55f705877c00 0x55f70587a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:58.784 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.780+0000 7fea9613b640 1 --2- 172.21.15.26:0/3945907435 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55f705877c00 0x55f70587a040 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a1e76ab4b881e8f0 server_cookie=a3cc9b368c947676 in_seq=0 out_seq=0 2024-09-11T23:20:58.784 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.780+0000 7fea79ffb640 10 client.25748 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:20:58.784 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.781+0000 7fea79ffb640 1 -- 172.21.15.26:0/3945907435 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f7057cea20 con 0x55f705877c00 2024-09-11T23:20:58.784 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.781+0000 7fea79ffb640 10 client.25748 handle_command_reply: tid=0 2024-09-11T23:20:58.784 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.781+0000 7fea9b21b740 1 client.25748 shutdown 2024-09-11T23:20:58.784 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.781+0000 7fea9b21b740 1 -- 172.21.15.26:0/3945907435 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f705867cb0 msgr2=0x55f705865650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:58.784 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.781+0000 7fea9b21b740 1 --2- 172.21.15.26:0/3945907435 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f705867cb0 0x55f705865650 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fea88009ca0 tx=0x7fea88031af0 comp rx=0 tx=0).stop 2024-09-11T23:20:58.785 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.781+0000 7fea9b21b740 1 -- 172.21.15.26:0/3945907435 shutdown_connections 2024-09-11T23:20:58.785 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.781+0000 7fea9b21b740 1 --2- 172.21.15.26:0/3945907435 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f705867cb0 0x55f705865650 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:58.785 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.781+0000 7fea9b21b740 1 --2- 172.21.15.26:0/3945907435 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55f705877c00 0x55f70587a040 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:58.785 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.781+0000 7fea9b21b740 1 --2- 172.21.15.26:0/3945907435 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f70585d410 0x55f705865110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:58.785 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.781+0000 7fea9b21b740 1 --2- 172.21.15.26:0/3945907435 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f70585caf0 0x55f705864bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:58.785 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.781+0000 7fea79ffb640 0 client.25748 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:20:58.785 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.781+0000 7fea9b21b740 1 -- 172.21.15.26:0/3945907435 >> 172.21.15.26:0/3945907435 conn(0x55f705859060 msgr2=0x55f70585ed70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:58.785 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.781+0000 7fea9b21b740 1 -- 172.21.15.26:0/3945907435 shutdown_connections 2024-09-11T23:20:58.785 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.781+0000 7fea9b21b740 1 -- 172.21.15.26:0/3945907435 wait complete. 2024-09-11T23:20:58.785 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.781+0000 7fea9b21b740 20 client.25748 trim_cache size 0 max 16384 2024-09-11T23:20:58.798 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.797+0000 7fea9b21b740 1 Processor -- start 2024-09-11T23:20:58.798 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.797+0000 7fea9b21b740 1 -- start start 2024-09-11T23:20:58.798 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.797+0000 7fea9b21b740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f70585caf0 0x55f70585f560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:58.798 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.797+0000 7fea9b21b740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f70585d410 0x55f705865480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:58.798 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.797+0000 7fea9b21b740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f705867cb0 0x55f70587f950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:58.798 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.797+0000 7fea9b21b740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fea901af950 con 0x55f70585d410 2024-09-11T23:20:58.798 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.797+0000 7fea9b21b740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fea901b1610 con 0x55f70585caf0 2024-09-11T23:20:58.799 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.797+0000 7fea9b21b740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fea901b0020 con 0x55f705867cb0 2024-09-11T23:20:58.799 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.797+0000 7fea9693c640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f705867cb0 0x55f70587f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:58.799 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.797+0000 7fea9693c640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f705867cb0 0x55f70587f950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:60078/0 (socket says 172.21.15.26:60078) 2024-09-11T23:20:58.799 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.797+0000 7fea9693c640 1 -- 172.21.15.26:0/2617565477 learned_addr learned my addr 172.21.15.26:0/2617565477 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:20:58.799 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.797+0000 7fea9613b640 1 --2- 172.21.15.26:0/2617565477 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f70585caf0 0x55f70585f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:58.799 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.798+0000 7fea9593a640 1 --2- 172.21.15.26:0/2617565477 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f70585d410 0x55f705865480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:58.799 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.798+0000 7fea9613b640 1 -- 172.21.15.26:0/2617565477 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f705867cb0 msgr2=0x55f70587f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:20:58.799 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.798+0000 7fea9613b640 1 --2- 172.21.15.26:0/2617565477 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f705867cb0 0x55f70587f950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:58.799 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.798+0000 7fea9613b640 1 -- 172.21.15.26:0/2617565477 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f70585d410 msgr2=0x55f705865480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:20:58.799 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.798+0000 7fea9613b640 1 --2- 172.21.15.26:0/2617565477 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f70585d410 0x55f705865480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:58.799 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.798+0000 7fea9613b640 1 -- 172.21.15.26:0/2617565477 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f70585c920 con 0x55f70585caf0 2024-09-11T23:20:58.799 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.798+0000 7fea9613b640 1 --2- 172.21.15.26:0/2617565477 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f70585caf0 0x55f70585f560 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fea8c01d5b0 tx=0x7fea8c00eee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df72c809d38e6695 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:58.800 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.798+0000 7fea7bfff640 1 -- 172.21.15.26:0/2617565477 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea8c02b040 con 0x55f70585caf0 2024-09-11T23:20:58.800 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.798+0000 7fea7bfff640 1 -- 172.21.15.26:0/2617565477 <== mon.2 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea8c01c650 con 0x55f70585caf0 2024-09-11T23:20:58.800 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.798+0000 7fea7bfff640 1 -- 172.21.15.26:0/2617565477 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea8c003850 con 0x55f70585caf0 2024-09-11T23:20:58.803 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.799+0000 7fea9b21b740 1 -- 172.21.15.26:0/2617565477 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f70585caf0 msgr2=0x55f70585f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:58.803 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.799+0000 7fea9b21b740 1 --2- 172.21.15.26:0/2617565477 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f70585caf0 0x55f70585f560 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fea8c01d5b0 tx=0x7fea8c00eee0 comp rx=0 tx=0).stop 2024-09-11T23:20:58.803 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.799+0000 7fea9b21b740 1 -- 172.21.15.26:0/2617565477 shutdown_connections 2024-09-11T23:20:58.803 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.799+0000 7fea9b21b740 1 --2- 172.21.15.26:0/2617565477 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f705867cb0 0x55f70587f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:58.803 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.799+0000 7fea9b21b740 1 --2- 172.21.15.26:0/2617565477 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f70585d410 0x55f705865480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:58.803 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.799+0000 7fea9b21b740 1 --2- 172.21.15.26:0/2617565477 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f70585caf0 0x55f70585f560 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:58.803 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.799+0000 7fea9b21b740 1 -- 172.21.15.26:0/2617565477 >> 172.21.15.26:0/2617565477 conn(0x55f705859060 msgr2=0x55f705866c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:58.804 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.799+0000 7fea9b21b740 1 -- 172.21.15.26:0/2617565477 shutdown_connections 2024-09-11T23:20:58.804 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.799+0000 7fea9b21b740 1 -- 172.21.15.26:0/2617565477 wait complete. 2024-09-11T23:20:58.804 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.799+0000 7fea9b21b740 1 -- start start 2024-09-11T23:20:58.804 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.800+0000 7fea9b21b740 1 Processor -- start 2024-09-11T23:20:58.804 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.800+0000 7fea9b21b740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f70585caf0 0x55f70586fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:58.804 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.800+0000 7fea9b21b740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f70585d410 0x55f7058703b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:58.804 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.800+0000 7fea9b21b740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f705867cb0 0x55f70586b250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:58.804 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.800+0000 7fea9b21b740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fea901b0e80 con 0x55f70585d410 2024-09-11T23:20:58.804 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.800+0000 7fea9b21b740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fea901b06f0 con 0x55f705867cb0 2024-09-11T23:20:58.804 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.800+0000 7fea9b21b740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fea901ad930 con 0x55f70585caf0 2024-09-11T23:20:58.804 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.800+0000 7fea9b21b740 10 client.0 _ll_register_callbacks cb 0x7fea88009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:20:58.804 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.800+0000 7fea9593a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f70585d410 0x55f7058703b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:58.805 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.800+0000 7fea9693c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f705867cb0 0x55f70586b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:58.805 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.800+0000 7fea9613b640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f70585caf0 0x55f70586fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:58.805 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.800+0000 7fea9593a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f70585d410 0x55f7058703b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50230/0 (socket says 172.21.15.26:50230) 2024-09-11T23:20:58.805 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.800+0000 7fea9593a640 1 -- 172.21.15.26:0/223274297 learned_addr learned my addr 172.21.15.26:0/223274297 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:20:58.805 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.800+0000 7fea9693c640 1 -- 172.21.15.26:0/223274297 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f70585caf0 msgr2=0x55f70586fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:20:58.805 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.800+0000 7fea9693c640 1 --2- 172.21.15.26:0/223274297 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f70585caf0 0x55f70586fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:58.805 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.800+0000 7fea9693c640 1 -- 172.21.15.26:0/223274297 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f70585d410 msgr2=0x55f7058703b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:20:58.805 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.800+0000 7fea9693c640 1 --2- 172.21.15.26:0/223274297 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f70585d410 0x55f7058703b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:58.805 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.800+0000 7fea9693c640 1 -- 172.21.15.26:0/223274297 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f7058542d0 con 0x55f705867cb0 2024-09-11T23:20:58.805 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.800+0000 7fea9593a640 1 --2- 172.21.15.26:0/223274297 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f70585d410 0x55f7058703b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:20:58.805 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.801+0000 7fea79ffb640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-11T23:20:58.805 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.801+0000 7fea9693c640 1 --2- 172.21.15.26:0/223274297 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f705867cb0 0x55f70586b250 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fea880307d0 tx=0x7fea88031bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=20f6ff82ec264855 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:58.805 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.801+0000 7fea79ffb640 1 -- 172.21.15.26:0/223274297 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea8805b020 con 0x55f705867cb0 2024-09-11T23:20:58.806 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.801+0000 7fea79ffb640 1 -- 172.21.15.26:0/223274297 <== mon.2 v2:172.21.15.76:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fea88003b80 con 0x55f705867cb0 2024-09-11T23:20:58.806 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.801+0000 7fea9b21b740 1 -- 172.21.15.26:0/223274297 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea8c01c650 con 0x55f705867cb0 2024-09-11T23:20:58.806 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.802+0000 7fea79ffb640 1 -- 172.21.15.26:0/223274297 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fea88066050 con 0x55f705867cb0 2024-09-11T23:20:58.806 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.802+0000 7fea9b21b740 10 client.25754 fetch_fsmap learned FSMap version 33 2024-09-11T23:20:58.806 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.802+0000 7fea9b21b740 1 -- 172.21.15.26:0/223274297 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55f7058e22a0 con 0x55f705867cb0 2024-09-11T23:20:58.806 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.802+0000 7fea79ffb640 1 -- 172.21.15.26:0/223274297 <== mon.2 v2:172.21.15.76:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea8c01c650 con 0x55f705867cb0 2024-09-11T23:20:58.806 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.802+0000 7fea79ffb640 1 -- 172.21.15.26:0/223274297 <== mon.2 v2:172.21.15.76:3300/0 5 ==== fsmap(e 33) ==== 8394+0+0 (secure 0 0 0) 0x7fea8804e040 con 0x55f705867cb0 2024-09-11T23:20:58.806 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.802+0000 7fea9b21b740 10 client.25754 fetch_fsmap finished waiting for FSMap version 33 2024-09-11T23:20:58.806 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.802+0000 7fea9b21b740 10 client.25754 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:20:58.806 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.802+0000 7fea9b21b740 20 client.25754 populate_metadata read hostname 'smithi026' 2024-09-11T23:20:58.806 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.802+0000 7fea9b21b740 1 --2- 172.21.15.26:0/223274297 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55f7058e2ad0 0x55f7058e4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:58.806 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.802+0000 7fea9b21b740 4 client.25754 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T23:20:58.806 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.802+0000 7fea9b21b740 1 -- 172.21.15.26:0/223274297 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f7057c8d70 con 0x55f7058e2ad0 2024-09-11T23:20:58.806 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.802+0000 7fea9613b640 1 --2- 172.21.15.26:0/223274297 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55f7058e2ad0 0x55f7058e4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:58.807 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.803+0000 7fea79ffb640 10 client.25754 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:20:58.807 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:58.803+0000 7fea9613b640 1 --2- 172.21.15.26:0/223274297 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55f7058e2ad0 0x55f7058e4ef0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=155024c8abfdd49e server_cookie=7350e2b63966aac9 in_seq=0 out_seq=0 2024-09-11T23:20:59.355 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:59.354+0000 7fea79ffb640 1 -- 172.21.15.26:0/223274297 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f7057c8d70 con 0x55f7058e2ad0 2024-09-11T23:20:59.355 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:59.354+0000 7fea79ffb640 10 client.25754 handle_command_reply: tid=0 2024-09-11T23:20:59.355 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:59.354+0000 7fea9b21b740 1 client.25754 shutdown 2024-09-11T23:20:59.355 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:59.354+0000 7fea9b21b740 1 -- 172.21.15.26:0/223274297 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f705867cb0 msgr2=0x55f70586b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:59.355 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:59.355+0000 7fea9b21b740 1 --2- 172.21.15.26:0/223274297 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f705867cb0 0x55f70586b250 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fea880307d0 tx=0x7fea88031bd0 comp rx=0 tx=0).stop 2024-09-11T23:20:59.355 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:59.355+0000 7fea9b21b740 1 -- 172.21.15.26:0/223274297 shutdown_connections 2024-09-11T23:20:59.356 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:59.355+0000 7fea9b21b740 1 --2- 172.21.15.26:0/223274297 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f705867cb0 0x55f70586b250 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:59.356 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:59.355+0000 7fea9b21b740 1 --2- 172.21.15.26:0/223274297 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55f7058e2ad0 0x55f7058e4ef0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:59.356 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:59.355+0000 7fea9b21b740 1 --2- 172.21.15.26:0/223274297 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f70585d410 0x55f7058703b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:59.356 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:59.355+0000 7fea9b21b740 1 --2- 172.21.15.26:0/223274297 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f70585caf0 0x55f70586fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:59.356 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:59.355+0000 7fea9b21b740 1 -- 172.21.15.26:0/223274297 >> 172.21.15.26:0/223274297 conn(0x55f705859060 msgr2=0x55f705866430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:59.356 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:59.355+0000 7fea79ffb640 0 client.25754 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:20:59.356 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:59.355+0000 7fea9b21b740 1 -- 172.21.15.26:0/223274297 shutdown_connections 2024-09-11T23:20:59.356 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:59.355+0000 7fea9b21b740 1 -- 172.21.15.26:0/223274297 wait complete. 2024-09-11T23:20:59.356 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:59.355+0000 7fea9b21b740 20 client.25754 trim_cache size 0 max 16384 2024-09-11T23:20:59.356 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-09-11T23:20:59.357 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "message": "", 2024-09-11T23:20:59.357 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-09-11T23:20:59.357 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-09-11T23:20:59.357 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:59.356+0000 7fea9839f640 1 -- 172.21.15.26:0/2330652035 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fea7c0a2e20 msgr2=0x7fea7c0a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:59.357 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:59.356+0000 7fea9839f640 1 --2- 172.21.15.26:0/2330652035 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fea7c0a2e20 0x7fea7c0a52e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fea8c0095f0 tx=0x7fea8c005f90 comp rx=0 tx=0).stop 2024-09-11T23:20:59.357 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:59.356+0000 7fea9839f640 1 -- 172.21.15.26:0/2330652035 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fea90101d70 msgr2=0x7fea90196690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:59.358 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:59.356+0000 7fea9839f640 1 --2- 172.21.15.26:0/2330652035 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fea90101d70 0x7fea90196690 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fea8000e9f0 tx=0x7fea8000eeb0 comp rx=0 tx=0).stop 2024-09-11T23:20:59.358 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:59.356+0000 7fea9839f640 1 -- 172.21.15.26:0/2330652035 shutdown_connections 2024-09-11T23:20:59.358 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:59.356+0000 7fea9839f640 1 --2- 172.21.15.26:0/2330652035 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fea7c0a2e20 0x7fea7c0a52e0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:59.358 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:59.356+0000 7fea9839f640 1 --2- 172.21.15.26:0/2330652035 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fea901026f0 0x7fea9019aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:59.358 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:59.356+0000 7fea9839f640 1 --2- 172.21.15.26:0/2330652035 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fea90101d70 0x7fea90196690 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:59.358 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:59.356+0000 7fea9839f640 1 --2- 172.21.15.26:0/2330652035 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fea90100490 0x7fea90196150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:59.358 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:59.356+0000 7fea9839f640 1 -- 172.21.15.26:0/2330652035 >> 172.21.15.26:0/2330652035 conn(0x7fea900770f0 msgr2=0x7fea90103ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:59.358 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:59.356+0000 7fea9839f640 1 -- 172.21.15.26:0/2330652035 shutdown_connections 2024-09-11T23:20:59.358 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:20:59.356+0000 7fea9839f640 1 -- 172.21.15.26:0/2330652035 wait complete. 2024-09-11T23:20:59.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:20:59 smithi136 ceph-mon[35121]: pgmap v81: 97 pgs: 97 active+clean; 337 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 9.8 MiB/s wr, 656 op/s 2024-09-11T23:20:59.750 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:20:59 smithi026 ceph-mon[31172]: pgmap v81: 97 pgs: 97 active+clean; 337 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 9.8 MiB/s wr, 656 op/s 2024-09-11T23:20:59.802 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:20:59 smithi076 ceph-mon[35296]: pgmap v81: 97 pgs: 97 active+clean; 337 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 9.8 MiB/s wr, 656 op/s 2024-09-11T23:21:01.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:21:01 smithi136 ceph-mon[35121]: pgmap v82: 97 pgs: 97 active+clean; 337 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 8.9 MiB/s wr, 560 op/s 2024-09-11T23:21:01.750 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:21:01 smithi026 ceph-mon[31172]: pgmap v82: 97 pgs: 97 active+clean; 337 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 8.9 MiB/s wr, 560 op/s 2024-09-11T23:21:01.802 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:21:01 smithi076 ceph-mon[35296]: pgmap v82: 97 pgs: 97 active+clean; 337 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 8.9 MiB/s wr, 560 op/s 2024-09-11T23:21:03.577 INFO:tasks.workunit.client.0.smithi026.stdout: 101 1288320 751 806950 806 387814 1191 2024-09-11T23:21:03.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:21:03 smithi136 ceph-mon[35121]: pgmap v83: 97 pgs: 97 active+clean; 337 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 8.9 MiB/s wr, 560 op/s 2024-09-11T23:21:03.750 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:21:03 smithi026 ceph-mon[31172]: pgmap v83: 97 pgs: 97 active+clean; 337 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 8.9 MiB/s wr, 560 op/s 2024-09-11T23:21:03.802 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:21:03 smithi076 ceph-mon[35296]: pgmap v83: 97 pgs: 97 active+clean; 337 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 8.9 MiB/s wr, 560 op/s 2024-09-11T23:21:03.891 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 24.7 2024-09-11T23:21:03.891 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:21:04.160 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:21:04.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:04.617+0000 7ff4daa8d640 1 -- 172.21.15.26:0/3040676733 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff4d4103720 msgr2=0x7ff4d4107b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:04.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:04.617+0000 7ff4daa8d640 1 --2- 172.21.15.26:0/3040676733 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff4d4103720 0x7ff4d4107b60 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7ff4c4009930 tx=0x7ff4c402a420 comp rx=0 tx=0).stop 2024-09-11T23:21:04.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:04.618+0000 7ff4daa8d640 1 -- 172.21.15.26:0/3040676733 shutdown_connections 2024-09-11T23:21:04.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:04.618+0000 7ff4daa8d640 1 --2- 172.21.15.26:0/3040676733 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff4d41082a0 0x7ff4d410ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:04.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:04.618+0000 7ff4daa8d640 1 --2- 172.21.15.26:0/3040676733 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff4d4103720 0x7ff4d4107b60 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:04.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:04.618+0000 7ff4daa8d640 1 --2- 172.21.15.26:0/3040676733 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff4d4102d70 0x7ff4d4103150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:04.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:04.618+0000 7ff4daa8d640 1 -- 172.21.15.26:0/3040676733 >> 172.21.15.26:0/3040676733 conn(0x7ff4d40fc7d0 msgr2=0x7ff4d40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:21:04.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:04.618+0000 7ff4daa8d640 1 -- 172.21.15.26:0/3040676733 shutdown_connections 2024-09-11T23:21:04.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:04.618+0000 7ff4daa8d640 1 -- 172.21.15.26:0/3040676733 wait complete. 2024-09-11T23:21:04.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:04.618+0000 7ff4daa8d640 1 Processor -- start 2024-09-11T23:21:04.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:04.618+0000 7ff4daa8d640 1 -- start start 2024-09-11T23:21:04.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:04.619+0000 7ff4daa8d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff4d4102d70 0x7ff4d41a0560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:04.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:04.619+0000 7ff4daa8d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff4d4103720 0x7ff4d41a0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:04.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:04.619+0000 7ff4daa8d640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff4d41082a0 0x7ff4d419a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:04.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:04.619+0000 7ff4daa8d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff4d4112060 con 0x7ff4d4103720 2024-09-11T23:21:04.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:04.619+0000 7ff4daa8d640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7ff4d4111ee0 con 0x7ff4d4102d70 2024-09-11T23:21:04.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:04.619+0000 7ff4daa8d640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ff4d41121e0 con 0x7ff4d41082a0 2024-09-11T23:21:04.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:04.619+0000 7ff4d902a640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff4d41082a0 0x7ff4d419a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:04.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:04.619+0000 7ff4cbfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff4d4103720 0x7ff4d41a0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:04.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:04.619+0000 7ff4d902a640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff4d41082a0 0x7ff4d419a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:36140/0 (socket says 172.21.15.26:36140) 2024-09-11T23:21:04.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:04.619+0000 7ff4d902a640 1 -- 172.21.15.26:0/2186557937 learned_addr learned my addr 172.21.15.26:0/2186557937 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:21:04.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:04.619+0000 7ff4d8829640 1 --2- 172.21.15.26:0/2186557937 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff4d4102d70 0x7ff4d41a0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:04.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:04.619+0000 7ff4cbfff640 1 -- 172.21.15.26:0/2186557937 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff4d41082a0 msgr2=0x7ff4d419a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:04.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:04.619+0000 7ff4cbfff640 1 --2- 172.21.15.26:0/2186557937 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff4d41082a0 0x7ff4d419a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:04.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:04.619+0000 7ff4cbfff640 1 -- 172.21.15.26:0/2186557937 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff4d4102d70 msgr2=0x7ff4d41a0560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:04.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:04.619+0000 7ff4cbfff640 1 --2- 172.21.15.26:0/2186557937 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff4d4102d70 0x7ff4d41a0560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:04.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:04.619+0000 7ff4cbfff640 1 -- 172.21.15.26:0/2186557937 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4d419aea0 con 0x7ff4d4103720 2024-09-11T23:21:04.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:04.619+0000 7ff4d902a640 1 --2- 172.21.15.26:0/2186557937 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff4d41082a0 0x7ff4d419a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:21:04.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:04.620+0000 7ff4cbfff640 1 --2- 172.21.15.26:0/2186557937 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff4d4103720 0x7ff4d41a0aa0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7ff4c402a940 tx=0x7ff4c40040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:21:04.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:04.620+0000 7ff4c9ffb640 1 -- 172.21.15.26:0/2186557937 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4c403f070 con 0x7ff4d4103720 2024-09-11T23:21:04.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:04.620+0000 7ff4daa8d640 1 -- 172.21.15.26:0/2186557937 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4d419b130 con 0x7ff4d4103720 2024-09-11T23:21:04.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:04.620+0000 7ff4c9ffb640 1 -- 172.21.15.26:0/2186557937 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff4c40044c0 con 0x7ff4d4103720 2024-09-11T23:21:04.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:04.620+0000 7ff4c9ffb640 1 -- 172.21.15.26:0/2186557937 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4c4035070 con 0x7ff4d4103720 2024-09-11T23:21:04.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:04.620+0000 7ff4daa8d640 1 -- 172.21.15.26:0/2186557937 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4d41a7310 con 0x7ff4d4103720 2024-09-11T23:21:04.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:04.621+0000 7ff4daa8d640 1 -- 172.21.15.26:0/2186557937 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4d406a810 con 0x7ff4d4103720 2024-09-11T23:21:04.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:04.622+0000 7ff4c9ffb640 1 -- 172.21.15.26:0/2186557937 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7ff4c403b460 con 0x7ff4d4103720 2024-09-11T23:21:04.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:04.623+0000 7ff4c9ffb640 1 --2- 172.21.15.26:0/2186557937 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7ff4bc0a2d70 0x7ff4bc0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:04.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:04.623+0000 7ff4c9ffb640 1 -- 172.21.15.26:0/2186557937 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7ff4c40ef800 con 0x7ff4d4103720 2024-09-11T23:21:04.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:04.623+0000 7ff4d8829640 1 --2- 172.21.15.26:0/2186557937 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7ff4bc0a2d70 0x7ff4bc0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:04.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:04.624+0000 7ff4d8829640 1 --2- 172.21.15.26:0/2186557937 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7ff4bc0a2d70 0x7ff4bc0a5230 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ff4d0005d00 tx=0x7ff4d000b410 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:21:04.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:04.627+0000 7ff4c9ffb640 1 -- 172.21.15.26:0/2186557937 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff4c40bbde0 con 0x7ff4d4103720 2024-09-11T23:21:04.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:04.854+0000 7ff4daa8d640 1 -- 172.21.15.26:0/2186557937 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff4d419bb60 con 0x7ff4d4103720 2024-09-11T23:21:04.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:04.855+0000 7ff4c9ffb640 1 -- 172.21.15.26:0/2186557937 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+11986 (secure 0 0 0) 0x7ff4c40c00e0 con 0x7ff4d4103720 2024-09-11T23:21:04.855 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-11T23:21:04.855 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":33,"btime":"2024-09-11T23:20:14:305449+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14499,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/367659515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":367659515},{"type":"v1","addr":"172.21.15.26:6835","nonce":367659515}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14511,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/2041065299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":2041065299},{"type":"v1","addr":"172.21.15.26:6837","nonce":2041065299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":33,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:16:26.544203+0000","modified":"2024-09-11T23:20:13.958301+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24485,"mds_1":24463,"mds_2":24458,"mds_3":24448,"mds_4":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14550":{"gid":14550,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6839/2233068305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":2233068305},{"type":"v1","addr":"172.21.15.26:6839","nonce":2233068305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14562":{"gid":14562,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6841/2984982336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":2984982336},{"type":"v1","addr":"172.21.15.26:6841","nonce":2984982336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6833/2019841455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":2019841455},{"type":"v1","addr":"172.21.15.136:6833","nonce":2019841455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.76:6833/3438147050","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3438147050},{"type":"v1","addr":"172.21.15.76:6833","nonce":3438147050}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6835/3617665975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3617665975},{"type":"v1","addr":"172.21.15.136:6835","nonce":3617665975}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":12,"addr":"172.21.15.76:6835/393308499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":393308499},{"type":"v1","addr":"172.21.15.76:6835","nonce":393308499}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":10,"addr":"172.21.15.136:6837/1633082387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":1633082387},{"type":"v1","addr":"172.21.15.136:6837","nonce":1633082387}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.76:6837/2078978748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":2078978748},{"type":"v1","addr":"172.21.15.76:6837","nonce":2078978748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":63,"addr":"172.21.15.136:6839/3211398630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":3211398630},{"type":"v1","addr":"172.21.15.136:6839","nonce":3211398630}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.76:6839/2077637090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6838","nonce":2077637090},{"type":"v1","addr":"172.21.15.76:6839","nonce":2077637090}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24485,"qdb_cluster":[24443,24448,24458,24463,24485]},"id":1}]} 2024-09-11T23:21:04.856 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 33 2024-09-11T23:21:04.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:04.857+0000 7ff4daa8d640 1 -- 172.21.15.26:0/2186557937 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7ff4bc0a2d70 msgr2=0x7ff4bc0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:04.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:04.857+0000 7ff4daa8d640 1 --2- 172.21.15.26:0/2186557937 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7ff4bc0a2d70 0x7ff4bc0a5230 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ff4d0005d00 tx=0x7ff4d000b410 comp rx=0 tx=0).stop 2024-09-11T23:21:04.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:04.857+0000 7ff4daa8d640 1 -- 172.21.15.26:0/2186557937 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff4d4103720 msgr2=0x7ff4d41a0aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:04.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:04.857+0000 7ff4daa8d640 1 --2- 172.21.15.26:0/2186557937 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff4d4103720 0x7ff4d41a0aa0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7ff4c402a940 tx=0x7ff4c40040c0 comp rx=0 tx=0).stop 2024-09-11T23:21:04.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:04.857+0000 7ff4daa8d640 1 -- 172.21.15.26:0/2186557937 shutdown_connections 2024-09-11T23:21:04.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:04.857+0000 7ff4daa8d640 1 --2- 172.21.15.26:0/2186557937 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7ff4bc0a2d70 0x7ff4bc0a5230 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:04.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:04.857+0000 7ff4daa8d640 1 --2- 172.21.15.26:0/2186557937 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff4d41082a0 0x7ff4d419a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:04.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:04.857+0000 7ff4daa8d640 1 --2- 172.21.15.26:0/2186557937 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff4d4103720 0x7ff4d41a0aa0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:04.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:04.857+0000 7ff4daa8d640 1 --2- 172.21.15.26:0/2186557937 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff4d4102d70 0x7ff4d41a0560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:04.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:04.857+0000 7ff4daa8d640 1 -- 172.21.15.26:0/2186557937 >> 172.21.15.26:0/2186557937 conn(0x7ff4d40fc7d0 msgr2=0x7ff4d4100a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:21:04.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:04.857+0000 7ff4daa8d640 1 -- 172.21.15.26:0/2186557937 shutdown_connections 2024-09-11T23:21:04.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:04.857+0000 7ff4daa8d640 1 -- 172.21.15.26:0/2186557937 wait complete. 2024-09-11T23:21:05.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:21:05 smithi026 ceph-mon[31172]: pgmap v84: 97 pgs: 97 active+clean; 347 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 10 MiB/s wr, 760 op/s 2024-09-11T23:21:05.501 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:21:05 smithi026 ceph-mon[31172]: from='client.? 172.21.15.26:0/2186557937' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:21:05.649 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24485 quiesce db / --timeout 90 --expiration 144.7 --await-for 45.0' 2024-09-11T23:21:05.650 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24485 quiesce db / --timeout 90 --expiration 144.7 --await-for 45.0 2024-09-11T23:21:05.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:21:05 smithi136 ceph-mon[35121]: pgmap v84: 97 pgs: 97 active+clean; 347 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 10 MiB/s wr, 760 op/s 2024-09-11T23:21:05.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:21:05 smithi136 ceph-mon[35121]: from='client.? 172.21.15.26:0/2186557937' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:21:05.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:21:05 smithi076 ceph-mon[35296]: pgmap v84: 97 pgs: 97 active+clean; 347 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 10 MiB/s wr, 760 op/s 2024-09-11T23:21:05.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:21:05 smithi076 ceph-mon[35296]: from='client.? 172.21.15.26:0/2186557937' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:21:05.922 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:21:06.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.431+0000 7fd35e13a640 1 -- 172.21.15.26:0/331142388 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd358108290 msgr2=0x7fd35810ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:06.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.431+0000 7fd35e13a640 1 --2- 172.21.15.26:0/331142388 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd358108290 0x7fd35810ec30 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fd350009cd0 tx=0x7fd35002a8e0 comp rx=0 tx=0).stop 2024-09-11T23:21:06.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.432+0000 7fd35e13a640 1 -- 172.21.15.26:0/331142388 shutdown_connections 2024-09-11T23:21:06.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.432+0000 7fd35e13a640 1 --2- 172.21.15.26:0/331142388 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd358108290 0x7fd35810ec30 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:06.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.432+0000 7fd35e13a640 1 --2- 172.21.15.26:0/331142388 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd358103740 0x7fd358107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:06.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.432+0000 7fd35e13a640 1 --2- 172.21.15.26:0/331142388 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd358102d90 0x7fd358103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:06.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.432+0000 7fd35e13a640 1 -- 172.21.15.26:0/331142388 >> 172.21.15.26:0/331142388 conn(0x7fd3580fc7d0 msgr2=0x7fd3580febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:21:06.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.432+0000 7fd35e13a640 1 -- 172.21.15.26:0/331142388 shutdown_connections 2024-09-11T23:21:06.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.432+0000 7fd35e13a640 1 -- 172.21.15.26:0/331142388 wait complete. 2024-09-11T23:21:06.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.433+0000 7fd35e13a640 1 Processor -- start 2024-09-11T23:21:06.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.433+0000 7fd35e13a640 1 -- start start 2024-09-11T23:21:06.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.433+0000 7fd35e13a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd358102d90 0x7fd3581985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:06.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.433+0000 7fd35e13a640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd358103740 0x7fd358198b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:06.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.433+0000 7fd35e13a640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd358108290 0x7fd35819ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:06.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.433+0000 7fd35e13a640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd358112090 con 0x7fd358102d90 2024-09-11T23:21:06.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.433+0000 7fd35e13a640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fd358111f10 con 0x7fd358108290 2024-09-11T23:21:06.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.433+0000 7fd35e13a640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd358112210 con 0x7fd358103740 2024-09-11T23:21:06.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.433+0000 7fd3577fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd358102d90 0x7fd3581985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:06.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.433+0000 7fd3577fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd358102d90 0x7fd3581985e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48830/0 (socket says 172.21.15.26:48830) 2024-09-11T23:21:06.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.433+0000 7fd3577fe640 1 -- 172.21.15.26:0/2097500553 learned_addr learned my addr 172.21.15.26:0/2097500553 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:21:06.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.433+0000 7fd34ffff640 1 --2- 172.21.15.26:0/2097500553 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd358103740 0x7fd358198b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:06.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.433+0000 7fd357fff640 1 --2- 172.21.15.26:0/2097500553 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd358108290 0x7fd35819ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:06.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.433+0000 7fd3577fe640 1 -- 172.21.15.26:0/2097500553 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd358103740 msgr2=0x7fd358198b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:06.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.433+0000 7fd3577fe640 1 --2- 172.21.15.26:0/2097500553 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd358103740 0x7fd358198b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:06.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.433+0000 7fd3577fe640 1 -- 172.21.15.26:0/2097500553 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd358108290 msgr2=0x7fd35819ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:06.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.433+0000 7fd3577fe640 1 --2- 172.21.15.26:0/2097500553 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd358108290 0x7fd35819ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:06.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.433+0000 7fd3577fe640 1 -- 172.21.15.26:0/2097500553 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd35819d590 con 0x7fd358102d90 2024-09-11T23:21:06.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.434+0000 7fd3577fe640 1 --2- 172.21.15.26:0/2097500553 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd358102d90 0x7fd3581985e0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fd34800d950 tx=0x7fd34800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:21:06.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.434+0000 7fd3557fa640 1 -- 172.21.15.26:0/2097500553 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd348017070 con 0x7fd358102d90 2024-09-11T23:21:06.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.434+0000 7fd3557fa640 1 -- 172.21.15.26:0/2097500553 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd348009c30 con 0x7fd358102d90 2024-09-11T23:21:06.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.434+0000 7fd35e13a640 1 -- 172.21.15.26:0/2097500553 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd35819d880 con 0x7fd358102d90 2024-09-11T23:21:06.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.434+0000 7fd3557fa640 1 -- 172.21.15.26:0/2097500553 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd34800f6b0 con 0x7fd358102d90 2024-09-11T23:21:06.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.434+0000 7fd35e13a640 1 -- 172.21.15.26:0/2097500553 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3581a5160 con 0x7fd358102d90 2024-09-11T23:21:06.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.436+0000 7fd3557fa640 1 -- 172.21.15.26:0/2097500553 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fd3480109d0 con 0x7fd358102d90 2024-09-11T23:21:06.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.436+0000 7fd360513740 1 Processor -- start 2024-09-11T23:21:06.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.436+0000 7fd360513740 1 -- start start 2024-09-11T23:21:06.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.437+0000 7fd360513740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560bef9c2640 0x560bef9c2a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:06.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.437+0000 7fd360513740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560bef9c2f60 0x560bef9d1810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:06.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.437+0000 7fd360513740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560bef9d1d50 0x560bef9d4140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:06.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.437+0000 7fd360513740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x560bef9282a0 con 0x560bef9c2f60 2024-09-11T23:21:06.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.437+0000 7fd360513740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x560bef8cfb10 con 0x560bef9d1d50 2024-09-11T23:21:06.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.437+0000 7fd360513740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x560bef924260 con 0x560bef9c2640 2024-09-11T23:21:06.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.437+0000 7fd3577fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560bef9c2f60 0x560bef9d1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:06.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.437+0000 7fd357fff640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560bef9c2640 0x560bef9c2a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:06.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.437+0000 7fd3577fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560bef9c2f60 0x560bef9d1810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48844/0 (socket says 172.21.15.26:48844) 2024-09-11T23:21:06.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.437+0000 7fd3577fe640 1 -- 172.21.15.26:0/2340119171 learned_addr learned my addr 172.21.15.26:0/2340119171 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:21:06.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.437+0000 7fd34ffff640 1 --2- 172.21.15.26:0/2340119171 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560bef9d1d50 0x560bef9d4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:06.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.437+0000 7fd3577fe640 1 -- 172.21.15.26:0/2340119171 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560bef9c2640 msgr2=0x560bef9c2a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:21:06.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.437+0000 7fd3577fe640 1 --2- 172.21.15.26:0/2340119171 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560bef9c2640 0x560bef9c2a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:06.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.437+0000 7fd3577fe640 1 -- 172.21.15.26:0/2340119171 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560bef9d1d50 msgr2=0x560bef9d4140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:21:06.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.437+0000 7fd3577fe640 1 --2- 172.21.15.26:0/2340119171 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560bef9d1d50 0x560bef9d4140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:06.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.437+0000 7fd3557fa640 1 --2- 172.21.15.26:0/2097500553 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fd3440a2d70 0x7fd3440a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:06.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.437+0000 7fd3577fe640 1 -- 172.21.15.26:0/2340119171 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560bef699b90 con 0x560bef9c2f60 2024-09-11T23:21:06.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.437+0000 7fd34ffff640 1 --2- 172.21.15.26:0/2340119171 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560bef9d1d50 0x560bef9d4140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:21:06.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.437+0000 7fd357fff640 1 --2- 172.21.15.26:0/2097500553 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fd3440a2d70 0x7fd3440a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:06.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.437+0000 7fd3577fe640 1 --2- 172.21.15.26:0/2340119171 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560bef9c2f60 0x560bef9d1810 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fd3480cc650 tx=0x7fd3480978c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dd899b7d746eda3 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:21:06.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.438+0000 7fd3557fa640 1 -- 172.21.15.26:0/2097500553 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fd3480cef80 con 0x7fd358102d90 2024-09-11T23:21:06.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.438+0000 7fd34dffb640 1 -- 172.21.15.26:0/2340119171 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd348020020 con 0x560bef9c2f60 2024-09-11T23:21:06.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.438+0000 7fd34dffb640 1 -- 172.21.15.26:0/2340119171 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd348009de0 con 0x560bef9c2f60 2024-09-11T23:21:06.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.438+0000 7fd34dffb640 1 -- 172.21.15.26:0/2340119171 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd34809b330 con 0x560bef9c2f60 2024-09-11T23:21:06.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.438+0000 7fd360513740 1 -- 172.21.15.26:0/2340119171 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560bef9c2f60 msgr2=0x560bef9d1810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:06.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.438+0000 7fd360513740 1 --2- 172.21.15.26:0/2340119171 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560bef9c2f60 0x560bef9d1810 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fd3480cc650 tx=0x7fd3480978c0 comp rx=0 tx=0).stop 2024-09-11T23:21:06.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.438+0000 7fd357fff640 1 --2- 172.21.15.26:0/2097500553 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fd3440a2d70 0x7fd3440a5230 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fd350004ab0 tx=0x7fd350004a40 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:21:06.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.439+0000 7fd360513740 1 -- 172.21.15.26:0/2340119171 shutdown_connections 2024-09-11T23:21:06.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.439+0000 7fd360513740 1 --2- 172.21.15.26:0/2340119171 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560bef9d1d50 0x560bef9d4140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:06.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.439+0000 7fd360513740 1 --2- 172.21.15.26:0/2340119171 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560bef9c2f60 0x560bef9d1810 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:06.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.439+0000 7fd360513740 1 --2- 172.21.15.26:0/2340119171 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560bef9c2640 0x560bef9c2a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:06.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.439+0000 7fd360513740 1 -- 172.21.15.26:0/2340119171 >> 172.21.15.26:0/2340119171 conn(0x560bef9becf0 msgr2=0x560bef9bf0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:21:06.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.439+0000 7fd360513740 1 -- 172.21.15.26:0/2340119171 shutdown_connections 2024-09-11T23:21:06.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.439+0000 7fd360513740 1 -- 172.21.15.26:0/2340119171 wait complete. 2024-09-11T23:21:06.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.439+0000 7fd360513740 1 -- start start 2024-09-11T23:21:06.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.439+0000 7fd360513740 1 Processor -- start 2024-09-11T23:21:06.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.439+0000 7fd360513740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560bef9c2640 0x560bef9ced20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:06.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.439+0000 7fd360513740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560bef9c2f60 0x560bef9cf260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:06.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.439+0000 7fd360513740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560bef9d1d50 0x560bef9cf7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:06.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.439+0000 7fd360513740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x560bef9d48c0 con 0x560bef9d1d50 2024-09-11T23:21:06.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.439+0000 7fd360513740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x560bef8b06e0 con 0x560bef9c2f60 2024-09-11T23:21:06.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.439+0000 7fd360513740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x560bef9d4a40 con 0x560bef9c2640 2024-09-11T23:21:06.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.439+0000 7fd360513740 10 client.0 _ll_register_callbacks cb 0x560bef8d4a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:21:06.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.440+0000 7fd3577fe640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560bef9c2640 0x560bef9ced20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:06.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.440+0000 7fd34ffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560bef9c2f60 0x560bef9cf260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:06.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.440+0000 7fd357fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560bef9d1d50 0x560bef9cf7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:06.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.440+0000 7fd3577fe640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560bef9c2640 0x560bef9ced20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:36194/0 (socket says 172.21.15.26:36194) 2024-09-11T23:21:06.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.440+0000 7fd3577fe640 1 -- 172.21.15.26:0/1909095365 learned_addr learned my addr 172.21.15.26:0/1909095365 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:21:06.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.440+0000 7fd34ffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560bef9c2f60 0x560bef9cf260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:56270/0 (socket says 172.21.15.26:56270) 2024-09-11T23:21:06.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.440+0000 7fd34ffff640 1 -- 172.21.15.26:0/1909095365 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560bef9c2640 msgr2=0x560bef9ced20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:21:06.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.440+0000 7fd34ffff640 1 --2- 172.21.15.26:0/1909095365 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560bef9c2640 0x560bef9ced20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:06.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.440+0000 7fd34ffff640 1 -- 172.21.15.26:0/1909095365 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560bef9d1d50 msgr2=0x560bef9cf7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:21:06.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.440+0000 7fd34ffff640 1 --2- 172.21.15.26:0/1909095365 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560bef9d1d50 0x560bef9cf7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:06.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.440+0000 7fd34ffff640 1 -- 172.21.15.26:0/1909095365 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560bef9dded0 con 0x560bef9c2f60 2024-09-11T23:21:06.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.440+0000 7fd3577fe640 1 --2- 172.21.15.26:0/1909095365 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560bef9c2640 0x560bef9ced20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:21:06.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.440+0000 7fd357fff640 1 --2- 172.21.15.26:0/1909095365 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560bef9d1d50 0x560bef9cf7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:21:06.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.440+0000 7fd34ffff640 1 --2- 172.21.15.26:0/1909095365 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560bef9c2f60 0x560bef9cf260 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x560bef9bfde0 tx=0x7fd340012aa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f00fb67f6ef1c7bc server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:21:06.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.440+0000 7fd336ffd640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-11T23:21:06.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.440+0000 7fd336ffd640 1 -- 172.21.15.26:0/1909095365 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd340004050 con 0x560bef9c2f60 2024-09-11T23:21:06.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.440+0000 7fd360513740 1 -- 172.21.15.26:0/1909095365 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd348009de0 con 0x560bef9c2f60 2024-09-11T23:21:06.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.440+0000 7fd336ffd640 1 -- 172.21.15.26:0/1909095365 <== mon.2 v2:172.21.15.76:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fd3400041f0 con 0x560bef9c2f60 2024-09-11T23:21:06.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.440+0000 7fd336ffd640 1 -- 172.21.15.26:0/1909095365 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fd3400043f0 con 0x560bef9c2f60 2024-09-11T23:21:06.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.440+0000 7fd360513740 10 client.25772 fetch_fsmap learned FSMap version 33 2024-09-11T23:21:06.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.440+0000 7fd360513740 1 -- 172.21.15.26:0/1909095365 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x560bef9e0910 con 0x560bef9c2f60 2024-09-11T23:21:06.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.441+0000 7fd336ffd640 1 -- 172.21.15.26:0/1909095365 <== mon.2 v2:172.21.15.76:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd348009de0 con 0x560bef9c2f60 2024-09-11T23:21:06.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.441+0000 7fd336ffd640 1 -- 172.21.15.26:0/1909095365 <== mon.2 v2:172.21.15.76:3300/0 5 ==== fsmap(e 33) ==== 8394+0+0 (secure 0 0 0) 0x7fd34000f8e0 con 0x560bef9c2f60 2024-09-11T23:21:06.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.441+0000 7fd360513740 10 client.25772 fetch_fsmap finished waiting for FSMap version 33 2024-09-11T23:21:06.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.441+0000 7fd360513740 10 client.25772 resolve_mds: validated gid 24485 aka daemon mds.b 2024-09-11T23:21:06.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.441+0000 7fd360513740 20 client.25772 populate_metadata read hostname 'smithi026' 2024-09-11T23:21:06.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.441+0000 7fd360513740 1 --2- 172.21.15.26:0/1909095365 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x560bef9e1e70 0x560bef9e42b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:06.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.441+0000 7fd360513740 4 client.25772 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:21:06.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.441+0000 7fd360513740 1 -- 172.21.15.26:0/1909095365 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560bef9359d0 con 0x560bef9e1e70 2024-09-11T23:21:06.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.441+0000 7fd3577fe640 1 --2- 172.21.15.26:0/1909095365 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x560bef9e1e70 0x560bef9e42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:06.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.442+0000 7fd336ffd640 10 client.25772 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:21:06.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.442+0000 7fd3577fe640 1 --2- 172.21.15.26:0/1909095365 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x560bef9e1e70 0x560bef9e42b0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=87880a224a84aa64 server_cookie=21459e37be16c7f8 in_seq=0 out_seq=0 2024-09-11T23:21:06.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.442+0000 7fd336ffd640 1 -- 172.21.15.26:0/1909095365 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560bef9359d0 con 0x560bef9e1e70 2024-09-11T23:21:06.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.442+0000 7fd336ffd640 10 client.25772 handle_command_reply: tid=0 2024-09-11T23:21:06.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.442+0000 7fd360513740 1 client.25772 shutdown 2024-09-11T23:21:06.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.442+0000 7fd360513740 1 -- 172.21.15.26:0/1909095365 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560bef9c2f60 msgr2=0x560bef9cf260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:06.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.442+0000 7fd360513740 1 --2- 172.21.15.26:0/1909095365 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560bef9c2f60 0x560bef9cf260 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x560bef9bfde0 tx=0x7fd340012aa0 comp rx=0 tx=0).stop 2024-09-11T23:21:06.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.443+0000 7fd360513740 1 -- 172.21.15.26:0/1909095365 shutdown_connections 2024-09-11T23:21:06.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.443+0000 7fd360513740 1 --2- 172.21.15.26:0/1909095365 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560bef9c2f60 0x560bef9cf260 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:06.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.443+0000 7fd360513740 1 --2- 172.21.15.26:0/1909095365 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x560bef9e1e70 0x560bef9e42b0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:06.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.443+0000 7fd360513740 1 --2- 172.21.15.26:0/1909095365 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560bef9d1d50 0x560bef9cf7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:06.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.443+0000 7fd360513740 1 --2- 172.21.15.26:0/1909095365 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560bef9c2640 0x560bef9ced20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:06.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.443+0000 7fd336ffd640 0 client.25772 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:21:06.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.443+0000 7fd360513740 1 -- 172.21.15.26:0/1909095365 >> 172.21.15.26:0/1909095365 conn(0x560bef9becf0 msgr2=0x560bef9c4a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:21:06.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.443+0000 7fd360513740 1 -- 172.21.15.26:0/1909095365 shutdown_connections 2024-09-11T23:21:06.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.443+0000 7fd360513740 1 -- 172.21.15.26:0/1909095365 wait complete. 2024-09-11T23:21:06.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.443+0000 7fd360513740 20 client.25772 trim_cache size 0 max 16384 2024-09-11T23:21:06.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.472+0000 7fd360513740 1 Processor -- start 2024-09-11T23:21:06.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.472+0000 7fd360513740 1 -- start start 2024-09-11T23:21:06.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.472+0000 7fd360513740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560bef9c2640 0x560bef9bde70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:06.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.472+0000 7fd360513740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560bef9c2f60 0x560bef9e9bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:06.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.472+0000 7fd360513740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560bef9d1d50 0x560bef9ea100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:06.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.472+0000 7fd360513740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd3581ab710 con 0x560bef9c2f60 2024-09-11T23:21:06.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.472+0000 7fd360513740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fd3581ad3d0 con 0x560bef9d1d50 2024-09-11T23:21:06.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.472+0000 7fd360513740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd3581abde0 con 0x560bef9c2640 2024-09-11T23:21:06.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.473+0000 7fd3577fe640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560bef9c2640 0x560bef9bde70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:06.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.473+0000 7fd34ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560bef9c2f60 0x560bef9e9bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:06.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.473+0000 7fd3577fe640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560bef9c2640 0x560bef9bde70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:36202/0 (socket says 172.21.15.26:36202) 2024-09-11T23:21:06.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.473+0000 7fd3577fe640 1 -- 172.21.15.26:0/129302185 learned_addr learned my addr 172.21.15.26:0/129302185 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:21:06.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.473+0000 7fd357fff640 1 --2- 172.21.15.26:0/129302185 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560bef9d1d50 0x560bef9ea100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:06.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.473+0000 7fd34ffff640 1 -- 172.21.15.26:0/129302185 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560bef9c2640 msgr2=0x560bef9bde70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:21:06.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.473+0000 7fd34ffff640 1 --2- 172.21.15.26:0/129302185 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560bef9c2640 0x560bef9bde70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:06.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.473+0000 7fd34ffff640 1 -- 172.21.15.26:0/129302185 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560bef9d1d50 msgr2=0x560bef9ea100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:21:06.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.473+0000 7fd34ffff640 1 --2- 172.21.15.26:0/129302185 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560bef9d1d50 0x560bef9ea100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:06.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.473+0000 7fd34ffff640 1 -- 172.21.15.26:0/129302185 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560bef9ba260 con 0x560bef9c2f60 2024-09-11T23:21:06.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.474+0000 7fd34ffff640 1 --2- 172.21.15.26:0/129302185 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560bef9c2f60 0x560bef9e9bc0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fd34000fec0 tx=0x7fd3400092e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f56224dca73f4b13 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:21:06.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.474+0000 7fd34d7fa640 1 -- 172.21.15.26:0/129302185 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd340029040 con 0x560bef9c2f60 2024-09-11T23:21:06.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.475+0000 7fd34d7fa640 1 -- 172.21.15.26:0/129302185 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd340009da0 con 0x560bef9c2f60 2024-09-11T23:21:06.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.475+0000 7fd34d7fa640 1 -- 172.21.15.26:0/129302185 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd340010990 con 0x560bef9c2f60 2024-09-11T23:21:06.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.476+0000 7fd360513740 1 -- 172.21.15.26:0/129302185 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560bef9c2f60 msgr2=0x560bef9e9bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:06.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.476+0000 7fd360513740 1 --2- 172.21.15.26:0/129302185 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560bef9c2f60 0x560bef9e9bc0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fd34000fec0 tx=0x7fd3400092e0 comp rx=0 tx=0).stop 2024-09-11T23:21:06.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.476+0000 7fd360513740 1 -- 172.21.15.26:0/129302185 shutdown_connections 2024-09-11T23:21:06.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.476+0000 7fd360513740 1 --2- 172.21.15.26:0/129302185 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560bef9d1d50 0x560bef9ea100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:06.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.476+0000 7fd360513740 1 --2- 172.21.15.26:0/129302185 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560bef9c2f60 0x560bef9e9bc0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:06.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.476+0000 7fd360513740 1 --2- 172.21.15.26:0/129302185 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560bef9c2640 0x560bef9bde70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:06.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.476+0000 7fd360513740 1 -- 172.21.15.26:0/129302185 >> 172.21.15.26:0/129302185 conn(0x560bef9becf0 msgr2=0x560bef9cde90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:21:06.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.476+0000 7fd360513740 1 -- 172.21.15.26:0/129302185 shutdown_connections 2024-09-11T23:21:06.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.476+0000 7fd360513740 1 -- 172.21.15.26:0/129302185 wait complete. 2024-09-11T23:21:06.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.477+0000 7fd360513740 1 -- start start 2024-09-11T23:21:06.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.478+0000 7fd360513740 1 Processor -- start 2024-09-11T23:21:06.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.478+0000 7fd360513740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560bef9c2640 0x560bef9da4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:06.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.478+0000 7fd360513740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560bef9c2f60 0x560bef9d5870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:06.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.478+0000 7fd3577fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560bef9c2640 0x560bef9da4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:06.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.478+0000 7fd3577fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560bef9c2640 0x560bef9da4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48876/0 (socket says 172.21.15.26:48876) 2024-09-11T23:21:06.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.478+0000 7fd3577fe640 1 -- 172.21.15.26:0/4225889498 learned_addr learned my addr 172.21.15.26:0/4225889498 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:21:06.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.478+0000 7fd360513740 1 --2- 172.21.15.26:0/4225889498 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560bef9d1d50 0x560bef9d5e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:06.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.479+0000 7fd360513740 1 -- 172.21.15.26:0/4225889498 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd3581acc40 con 0x560bef9c2640 2024-09-11T23:21:06.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.479+0000 7fd360513740 1 -- 172.21.15.26:0/4225889498 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fd3581ac4b0 con 0x560bef9c2f60 2024-09-11T23:21:06.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.479+0000 7fd360513740 1 -- 172.21.15.26:0/4225889498 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd3581a96f0 con 0x560bef9d1d50 2024-09-11T23:21:06.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.479+0000 7fd34ffff640 1 --2- 172.21.15.26:0/4225889498 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560bef9c2f60 0x560bef9d5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:06.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.479+0000 7fd357fff640 1 --2- 172.21.15.26:0/4225889498 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560bef9d1d50 0x560bef9d5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:06.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.479+0000 7fd34ffff640 1 -- 172.21.15.26:0/4225889498 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560bef9d1d50 msgr2=0x560bef9d5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:21:06.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.479+0000 7fd34ffff640 1 --2- 172.21.15.26:0/4225889498 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560bef9d1d50 0x560bef9d5e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:06.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.479+0000 7fd34ffff640 1 -- 172.21.15.26:0/4225889498 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560bef9c2640 msgr2=0x560bef9da4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:21:06.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.479+0000 7fd34ffff640 1 --2- 172.21.15.26:0/4225889498 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560bef9c2640 0x560bef9da4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:06.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.479+0000 7fd34ffff640 1 -- 172.21.15.26:0/4225889498 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560bef9d6640 con 0x560bef9c2f60 2024-09-11T23:21:06.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.479+0000 7fd360513740 10 client.0 _ll_register_callbacks cb 0x7fd340009cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:21:06.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.479+0000 7fd3577fe640 1 --2- 172.21.15.26:0/4225889498 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560bef9c2640 0x560bef9da4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:21:06.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.479+0000 7fd34ffff640 1 --2- 172.21.15.26:0/4225889498 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560bef9c2f60 0x560bef9d5870 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x560bef9d4ab0 tx=0x7fd340005f30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=931be0a14a03f5a0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:21:06.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.480+0000 7fd360513740 1 -- 172.21.15.26:0/4225889498 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd340009da0 con 0x560bef9c2f60 2024-09-11T23:21:06.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.480+0000 7fd336ffd640 10 client.25775 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-11T23:21:06.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.480+0000 7fd336ffd640 1 -- 172.21.15.26:0/4225889498 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd340029040 con 0x560bef9c2f60 2024-09-11T23:21:06.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.480+0000 7fd336ffd640 1 -- 172.21.15.26:0/4225889498 <== mon.2 v2:172.21.15.76:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fd34003bcd0 con 0x560bef9c2f60 2024-09-11T23:21:06.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.480+0000 7fd336ffd640 1 -- 172.21.15.26:0/4225889498 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fd34000f4e0 con 0x560bef9c2f60 2024-09-11T23:21:06.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.480+0000 7fd360513740 10 client.25775 fetch_fsmap learned FSMap version 33 2024-09-11T23:21:06.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.480+0000 7fd360513740 1 -- 172.21.15.26:0/4225889498 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({fsmap=33}) -- 0x560bef9c62b0 con 0x560bef9c2f60 2024-09-11T23:21:06.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.481+0000 7fd336ffd640 1 -- 172.21.15.26:0/4225889498 <== mon.2 v2:172.21.15.76:3300/0 4 ==== fsmap(e 33) ==== 8394+0+0 (secure 0 0 0) 0x7fd34000f680 con 0x560bef9c2f60 2024-09-11T23:21:06.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.481+0000 7fd360513740 10 client.25775 fetch_fsmap finished waiting for FSMap version 33 2024-09-11T23:21:06.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.481+0000 7fd360513740 10 client.25775 resolve_mds: validated gid 24485 aka daemon mds.b 2024-09-11T23:21:06.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.481+0000 7fd360513740 20 client.25775 populate_metadata read hostname 'smithi026' 2024-09-11T23:21:06.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.481+0000 7fd360513740 1 --2- 172.21.15.26:0/4225889498 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x560bef9c6970 0x560bef9c8e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:06.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.481+0000 7fd360513740 4 client.25775 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 144.7, "await_for": 45.0}] 2024-09-11T23:21:06.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.481+0000 7fd360513740 1 -- 172.21.15.26:0/4225889498 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 144.7, "await_for": 45.0}) -- 0x560bef92fd20 con 0x560bef9c6970 2024-09-11T23:21:06.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.482+0000 7fd3577fe640 1 --2- 172.21.15.26:0/4225889498 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x560bef9c6970 0x560bef9c8e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:06.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.482+0000 7fd3577fe640 1 --2- 172.21.15.26:0/4225889498 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x560bef9c6970 0x560bef9c8e10 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f79e08b1722bd2d server_cookie=ddd888fe6eb5491d in_seq=0 out_seq=0 2024-09-11T23:21:06.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:06.482+0000 7fd336ffd640 10 client.25775 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:21:07.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:21:07 smithi136 ceph-mon[35121]: pgmap v85: 97 pgs: 97 active+clean; 347 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 9.0 MiB/s wr, 699 op/s 2024-09-11T23:21:07.750 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:21:07 smithi026 ceph-mon[31172]: pgmap v85: 97 pgs: 97 active+clean; 347 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 9.0 MiB/s wr, 699 op/s 2024-09-11T23:21:07.802 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:21:07 smithi076 ceph-mon[35296]: pgmap v85: 97 pgs: 97 active+clean; 347 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 9.0 MiB/s wr, 699 op/s 2024-09-11T23:21:09.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:21:09 smithi136 ceph-mon[35121]: pgmap v86: 97 pgs: 97 active+clean; 360 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 10 MiB/s wr, 815 op/s 2024-09-11T23:21:09.750 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:21:09 smithi026 ceph-mon[31172]: pgmap v86: 97 pgs: 97 active+clean; 360 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 10 MiB/s wr, 815 op/s 2024-09-11T23:21:09.802 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:21:09 smithi076 ceph-mon[35296]: pgmap v86: 97 pgs: 97 active+clean; 360 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 10 MiB/s wr, 815 op/s 2024-09-11T23:21:11.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:21:11 smithi136 ceph-mon[35121]: pgmap v87: 97 pgs: 97 active+clean; 360 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.3 MiB/s wr, 316 op/s 2024-09-11T23:21:11.750 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:21:11 smithi026 ceph-mon[31172]: pgmap v87: 97 pgs: 97 active+clean; 360 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.3 MiB/s wr, 316 op/s 2024-09-11T23:21:11.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:21:11 smithi076 ceph-mon[35296]: pgmap v87: 97 pgs: 97 active+clean; 360 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.3 MiB/s wr, 316 op/s 2024-09-11T23:21:12.457 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.457+0000 7f559f85c640 1 -- 172.21.15.26:0/3187001161 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5598107b70 msgr2=0x7f5598107f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:12.457 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.457+0000 7f559f85c640 1 --2- 172.21.15.26:0/3187001161 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5598107b70 0x7f5598107f50 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f5594009930 tx=0x7f559402a420 comp rx=0 tx=0).stop 2024-09-11T23:21:12.457 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.457+0000 7f559f85c640 1 -- 172.21.15.26:0/3187001161 shutdown_connections 2024-09-11T23:21:12.458 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.457+0000 7f559f85c640 1 --2- 172.21.15.26:0/3187001161 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5598102470 0x7f559810e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:12.458 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.457+0000 7f559f85c640 1 --2- 172.21.15.26:0/3187001161 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5598101ad0 0x7f5598101f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:12.458 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.457+0000 7f559f85c640 1 --2- 172.21.15.26:0/3187001161 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5598107b70 0x7f5598107f50 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:12.458 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.457+0000 7f559f85c640 1 -- 172.21.15.26:0/3187001161 >> 172.21.15.26:0/3187001161 conn(0x7f55980fd7d0 msgr2=0x7f55980ffbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:21:12.458 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.457+0000 7f559f85c640 1 -- 172.21.15.26:0/3187001161 shutdown_connections 2024-09-11T23:21:12.458 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.457+0000 7f559f85c640 1 -- 172.21.15.26:0/3187001161 wait complete. 2024-09-11T23:21:12.458 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.457+0000 7f559f85c640 1 Processor -- start 2024-09-11T23:21:12.459 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.457+0000 7f559f85c640 1 -- start start 2024-09-11T23:21:12.460 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.458+0000 7f559f85c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5598101ad0 0x7f55980744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:12.460 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.458+0000 7f559f85c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5598102470 0x7f5598074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:12.460 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.458+0000 7f559f85c640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5598107b70 0x7f5598074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:12.460 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.458+0000 7f559f85c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f5598111f20 con 0x7f5598102470 2024-09-11T23:21:12.460 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.458+0000 7f559f85c640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f5598111da0 con 0x7f5598101ad0 2024-09-11T23:21:12.460 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.458+0000 7f559f85c640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f55981120a0 con 0x7f5598107b70 2024-09-11T23:21:12.460 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.458+0000 7f559cdf7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5598102470 0x7f5598074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:12.460 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.458+0000 7f559cdf7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5598102470 0x7f5598074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36290/0 (socket says 172.21.15.26:36290) 2024-09-11T23:21:12.460 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.458+0000 7f559d5f8640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5598101ad0 0x7f55980744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:12.460 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.458+0000 7f559cdf7640 1 -- 172.21.15.26:0/957724630 learned_addr learned my addr 172.21.15.26:0/957724630 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:21:12.460 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.458+0000 7f559ddf9640 1 --2- 172.21.15.26:0/957724630 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5598107b70 0x7f5598074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:12.460 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.458+0000 7f559cdf7640 1 -- 172.21.15.26:0/957724630 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5598107b70 msgr2=0x7f5598074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:12.461 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.458+0000 7f559cdf7640 1 --2- 172.21.15.26:0/957724630 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5598107b70 0x7f5598074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:12.461 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.458+0000 7f559cdf7640 1 -- 172.21.15.26:0/957724630 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5598101ad0 msgr2=0x7f55980744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:12.461 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.458+0000 7f559cdf7640 1 --2- 172.21.15.26:0/957724630 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5598101ad0 0x7f55980744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:12.461 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.458+0000 7f559cdf7640 1 -- 172.21.15.26:0/957724630 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55981a68f0 con 0x7f5598102470 2024-09-11T23:21:12.461 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.458+0000 7f559cdf7640 1 --2- 172.21.15.26:0/957724630 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5598102470 0x7f5598074a30 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f558800e9f0 tx=0x7f558800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:21:12.461 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.459+0000 7f558e7fc640 1 -- 172.21.15.26:0/957724630 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f558800cd80 con 0x7f5598102470 2024-09-11T23:21:12.461 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.459+0000 7f559f85c640 1 -- 172.21.15.26:0/957724630 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55981a6be0 con 0x7f5598102470 2024-09-11T23:21:12.461 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.459+0000 7f558e7fc640 1 -- 172.21.15.26:0/957724630 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5588005500 con 0x7f5598102470 2024-09-11T23:21:12.461 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.459+0000 7f558e7fc640 1 -- 172.21.15.26:0/957724630 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5588010690 con 0x7f5598102470 2024-09-11T23:21:12.461 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.459+0000 7f559f85c640 1 -- 172.21.15.26:0/957724630 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55981a7130 con 0x7f5598102470 2024-09-11T23:21:12.463 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.462+0000 7f558e7fc640 1 -- 172.21.15.26:0/957724630 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f55880046c0 con 0x7f5598102470 2024-09-11T23:21:12.463 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.462+0000 7f55a25ed740 1 Processor -- start 2024-09-11T23:21:12.464 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.462+0000 7f55a25ed740 1 -- start start 2024-09-11T23:21:12.464 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.462+0000 7f55a25ed740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c0f1ae0af0 0x55c0f1ae0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:12.464 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.462+0000 7f55a25ed740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c0f1ae1410 0x55c0f1aeb770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:12.464 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.462+0000 7f55a25ed740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c0f1aebcb0 0x55c0f1aee0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:12.464 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.462+0000 7f55a25ed740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55c0f1a452f0 con 0x55c0f1aebcb0 2024-09-11T23:21:12.464 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.462+0000 7f55a25ed740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55c0f19ed270 con 0x55c0f1ae0af0 2024-09-11T23:21:12.464 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.462+0000 7f55a25ed740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55c0f1a412b0 con 0x55c0f1ae1410 2024-09-11T23:21:12.464 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.462+0000 7f559ddf9640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c0f1ae0af0 0x55c0f1ae0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:12.464 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.462+0000 7f559d5f8640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c0f1ae1410 0x55c0f1aeb770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:12.464 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.462+0000 7f559ddf9640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c0f1ae0af0 0x55c0f1ae0ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:59292/0 (socket says 172.21.15.26:59292) 2024-09-11T23:21:12.464 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.462+0000 7f559ddf9640 1 -- 172.21.15.26:0/119855821 learned_addr learned my addr 172.21.15.26:0/119855821 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:21:12.464 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.463+0000 7f559cdf7640 1 --2- 172.21.15.26:0/119855821 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c0f1aebcb0 0x55c0f1aee0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:12.465 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.463+0000 7f559d5f8640 1 -- 172.21.15.26:0/119855821 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c0f1ae0af0 msgr2=0x55c0f1ae0ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:21:12.465 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.463+0000 7f559d5f8640 1 --2- 172.21.15.26:0/119855821 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c0f1ae0af0 0x55c0f1ae0ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:12.465 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.463+0000 7f559d5f8640 1 -- 172.21.15.26:0/119855821 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c0f1aebcb0 msgr2=0x55c0f1aee0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:21:12.465 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.463+0000 7f559d5f8640 1 --2- 172.21.15.26:0/119855821 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c0f1aebcb0 0x55c0f1aee0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:12.465 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.463+0000 7f559d5f8640 1 -- 172.21.15.26:0/119855821 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c0f19ec340 con 0x55c0f1ae1410 2024-09-11T23:21:12.465 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.463+0000 7f559cdf7640 1 --2- 172.21.15.26:0/119855821 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c0f1aebcb0 0x55c0f1aee0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:21:12.465 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.463+0000 7f558e7fc640 1 --2- 172.21.15.26:0/957724630 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f55840a2e20 0x7f55840a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:12.465 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.463+0000 7f559ddf9640 1 --2- 172.21.15.26:0/957724630 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f55840a2e20 0x7f55840a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:12.465 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.463+0000 7f559d5f8640 1 --2- 172.21.15.26:0/119855821 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c0f1ae1410 0x55c0f1aeb770 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f5594005940 tx=0x7f5594005bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=66153c9ee02a5245 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:21:12.465 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.463+0000 7f558e7fc640 1 -- 172.21.15.26:0/957724630 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f55880cf7c0 con 0x7f5598102470 2024-09-11T23:21:12.465 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.463+0000 7f55837fe640 1 -- 172.21.15.26:0/119855821 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5594040070 con 0x55c0f1ae1410 2024-09-11T23:21:12.465 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.463+0000 7f55837fe640 1 -- 172.21.15.26:0/119855821 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5594031870 con 0x55c0f1ae1410 2024-09-11T23:21:12.465 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.463+0000 7f55837fe640 1 -- 172.21.15.26:0/119855821 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5594035040 con 0x55c0f1ae1410 2024-09-11T23:21:12.470 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.464+0000 7f55a25ed740 1 -- 172.21.15.26:0/119855821 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c0f1ae1410 msgr2=0x55c0f1aeb770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:12.470 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.464+0000 7f55a25ed740 1 --2- 172.21.15.26:0/119855821 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c0f1ae1410 0x55c0f1aeb770 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f5594005940 tx=0x7f5594005bd0 comp rx=0 tx=0).stop 2024-09-11T23:21:12.470 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.464+0000 7f559ddf9640 1 --2- 172.21.15.26:0/957724630 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f55840a2e20 0x7f55840a52e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f5590011630 tx=0x7f5590009500 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:21:12.470 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.464+0000 7f55a25ed740 1 -- 172.21.15.26:0/119855821 shutdown_connections 2024-09-11T23:21:12.470 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.464+0000 7f55a25ed740 1 --2- 172.21.15.26:0/119855821 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c0f1aebcb0 0x55c0f1aee0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:12.470 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.464+0000 7f55a25ed740 1 --2- 172.21.15.26:0/119855821 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c0f1ae1410 0x55c0f1aeb770 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:12.470 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.464+0000 7f55a25ed740 1 --2- 172.21.15.26:0/119855821 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c0f1ae0af0 0x55c0f1ae0ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:12.470 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.464+0000 7f55a25ed740 1 -- 172.21.15.26:0/119855821 >> 172.21.15.26:0/119855821 conn(0x55c0f1add060 msgr2=0x55c0f1add440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:21:12.471 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.464+0000 7f55a25ed740 1 -- 172.21.15.26:0/119855821 shutdown_connections 2024-09-11T23:21:12.471 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.464+0000 7f55a25ed740 1 -- 172.21.15.26:0/119855821 wait complete. 2024-09-11T23:21:12.471 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.465+0000 7f55a25ed740 1 -- start start 2024-09-11T23:21:12.471 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.465+0000 7f55a25ed740 1 Processor -- start 2024-09-11T23:21:12.471 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.465+0000 7f55a25ed740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c0f1ae0af0 0x55c0f1ae8b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:12.471 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.465+0000 7f55a25ed740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c0f1ae1410 0x55c0f1ae9050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:12.471 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.465+0000 7f55a25ed740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c0f1aebcb0 0x55c0f1ae9590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:12.471 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.465+0000 7f55a25ed740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55c0f1aee880 con 0x55c0f1ae1410 2024-09-11T23:21:12.471 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.465+0000 7f55a25ed740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55c0f19ce290 con 0x55c0f1aebcb0 2024-09-11T23:21:12.471 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.465+0000 7f55a25ed740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55c0f1aeea00 con 0x55c0f1ae0af0 2024-09-11T23:21:12.471 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.465+0000 7f55a25ed740 10 client.0 _ll_register_callbacks cb 0x55c0f19ff140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:21:12.471 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.465+0000 7f559cdf7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c0f1ae1410 0x55c0f1ae9050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:12.471 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.465+0000 7f559d5f8640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c0f1ae0af0 0x55c0f1ae8b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:12.471 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.465+0000 7f559cdf7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c0f1ae1410 0x55c0f1ae9050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36302/0 (socket says 172.21.15.26:36302) 2024-09-11T23:21:12.472 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.465+0000 7f559cdf7640 1 -- 172.21.15.26:0/3575948370 learned_addr learned my addr 172.21.15.26:0/3575948370 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:21:12.472 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.465+0000 7f559ddf9640 1 --2- 172.21.15.26:0/3575948370 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c0f1aebcb0 0x55c0f1ae9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:12.472 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.466+0000 7f559cdf7640 1 -- 172.21.15.26:0/3575948370 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c0f1ae0af0 msgr2=0x55c0f1ae8b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:21:12.472 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.466+0000 7f559cdf7640 1 --2- 172.21.15.26:0/3575948370 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c0f1ae0af0 0x55c0f1ae8b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:12.472 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.466+0000 7f559cdf7640 1 -- 172.21.15.26:0/3575948370 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c0f1aebcb0 msgr2=0x55c0f1ae9590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:21:12.472 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.466+0000 7f559cdf7640 1 --2- 172.21.15.26:0/3575948370 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c0f1aebcb0 0x55c0f1ae9590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:12.472 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.466+0000 7f559cdf7640 1 -- 172.21.15.26:0/3575948370 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c0f1758210 con 0x55c0f1ae1410 2024-09-11T23:21:12.472 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.466+0000 7f559d5f8640 1 --2- 172.21.15.26:0/3575948370 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c0f1ae0af0 0x55c0f1ae8b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:21:12.472 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.466+0000 7f559cdf7640 1 --2- 172.21.15.26:0/3575948370 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c0f1ae1410 0x55c0f1ae9050 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f558809b9f0 tx=0x7f558809b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f36ae4378fb22a5d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:21:12.472 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.466+0000 7f5580ff9640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:21:12.472 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.466+0000 7f5580ff9640 1 -- 172.21.15.26:0/3575948370 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55880a46d0 con 0x55c0f1ae1410 2024-09-11T23:21:12.472 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.466+0000 7f5580ff9640 1 -- 172.21.15.26:0/3575948370 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f558809b140 con 0x55c0f1ae1410 2024-09-11T23:21:12.472 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.466+0000 7f55a25ed740 1 -- 172.21.15.26:0/3575948370 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5594031870 con 0x55c0f1ae1410 2024-09-11T23:21:12.473 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.466+0000 7f5580ff9640 1 -- 172.21.15.26:0/3575948370 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f55880a00a0 con 0x55c0f1ae1410 2024-09-11T23:21:12.473 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.466+0000 7f55a25ed740 10 client.16209 fetch_fsmap learned FSMap version 33 2024-09-11T23:21:12.473 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.466+0000 7f55a25ed740 1 -- 172.21.15.26:0/3575948370 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55c0f1824620 con 0x55c0f1ae1410 2024-09-11T23:21:12.473 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.466+0000 7f5580ff9640 1 -- 172.21.15.26:0/3575948370 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5594031870 con 0x55c0f1ae1410 2024-09-11T23:21:12.473 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.467+0000 7f5580ff9640 1 -- 172.21.15.26:0/3575948370 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 33) ==== 8394+0+0 (secure 0 0 0) 0x7f558809acd0 con 0x55c0f1ae1410 2024-09-11T23:21:12.473 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.467+0000 7f55a25ed740 10 client.16209 fetch_fsmap finished waiting for FSMap version 33 2024-09-11T23:21:12.473 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.467+0000 7f55a25ed740 10 client.16209 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:21:12.473 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.467+0000 7f55a25ed740 20 client.16209 populate_metadata read hostname 'smithi026' 2024-09-11T23:21:12.473 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.467+0000 7f55a25ed740 1 --2- 172.21.15.26:0/3575948370 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55c0f1afbfd0 0x55c0f1afe470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:12.473 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.467+0000 7f55a25ed740 4 client.16209 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:21:12.473 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.467+0000 7f55a25ed740 1 -- 172.21.15.26:0/3575948370 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c0f1a52a20 con 0x55c0f1afbfd0 2024-09-11T23:21:12.473 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.467+0000 7f559d5f8640 1 --2- 172.21.15.26:0/3575948370 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55c0f1afbfd0 0x55c0f1afe470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:12.474 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.467+0000 7f5580ff9640 10 client.16209 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:21:12.474 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.467+0000 7f559d5f8640 1 --2- 172.21.15.26:0/3575948370 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55c0f1afbfd0 0x55c0f1afe470 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f85bf6c0da04a0a server_cookie=31bbbbe0f169285a in_seq=0 out_seq=0 2024-09-11T23:21:12.474 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.468+0000 7f5580ff9640 1 -- 172.21.15.26:0/3575948370 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c0f1a52a20 con 0x55c0f1afbfd0 2024-09-11T23:21:12.474 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.468+0000 7f5580ff9640 10 client.16209 handle_command_reply: tid=0 2024-09-11T23:21:12.474 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.468+0000 7f55a25ed740 1 client.16209 shutdown 2024-09-11T23:21:12.474 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.468+0000 7f55a25ed740 1 -- 172.21.15.26:0/3575948370 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c0f1ae1410 msgr2=0x55c0f1ae9050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:12.474 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.468+0000 7f55a25ed740 1 --2- 172.21.15.26:0/3575948370 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c0f1ae1410 0x55c0f1ae9050 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f558809b9f0 tx=0x7f558809b8f0 comp rx=0 tx=0).stop 2024-09-11T23:21:12.474 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.468+0000 7f55a25ed740 1 -- 172.21.15.26:0/3575948370 shutdown_connections 2024-09-11T23:21:12.474 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.468+0000 7f55a25ed740 1 --2- 172.21.15.26:0/3575948370 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c0f1aebcb0 0x55c0f1ae9590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:12.474 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.468+0000 7f55a25ed740 1 --2- 172.21.15.26:0/3575948370 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55c0f1afbfd0 0x55c0f1afe470 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:12.474 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.468+0000 7f55a25ed740 1 --2- 172.21.15.26:0/3575948370 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c0f1ae1410 0x55c0f1ae9050 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:12.474 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.468+0000 7f55a25ed740 1 --2- 172.21.15.26:0/3575948370 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c0f1ae0af0 0x55c0f1ae8b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:12.475 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.468+0000 7f5580ff9640 0 client.16209 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:21:12.475 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.468+0000 7f55a25ed740 1 -- 172.21.15.26:0/3575948370 >> 172.21.15.26:0/3575948370 conn(0x55c0f1add060 msgr2=0x55c0f1ae2d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:21:12.475 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.469+0000 7f55a25ed740 1 -- 172.21.15.26:0/3575948370 shutdown_connections 2024-09-11T23:21:12.475 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.469+0000 7f55a25ed740 1 -- 172.21.15.26:0/3575948370 wait complete. 2024-09-11T23:21:12.475 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.469+0000 7f55a25ed740 20 client.16209 trim_cache size 0 max 16384 2024-09-11T23:21:12.484 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.483+0000 7f55a25ed740 1 Processor -- start 2024-09-11T23:21:12.484 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.483+0000 7f55a25ed740 1 -- start start 2024-09-11T23:21:12.484 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.483+0000 7f55a25ed740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c0f1ae0af0 0x55c0f1ae93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:12.484 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.483+0000 7f55a25ed740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c0f1ae1410 0x55c0f1ae2c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:12.484 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.483+0000 7f55a25ed740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c0f1aebcb0 0x55c0f1b03d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:12.485 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.483+0000 7f55a25ed740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f55981afa40 con 0x55c0f1ae0af0 2024-09-11T23:21:12.485 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.483+0000 7f55a25ed740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f55981b1700 con 0x55c0f1ae1410 2024-09-11T23:21:12.485 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.483+0000 7f55a25ed740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f55981b0110 con 0x55c0f1aebcb0 2024-09-11T23:21:12.485 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.484+0000 7f559d5f8640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c0f1ae0af0 0x55c0f1ae93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:12.485 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.484+0000 7f559d5f8640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c0f1ae0af0 0x55c0f1ae93c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36308/0 (socket says 172.21.15.26:36308) 2024-09-11T23:21:12.485 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.484+0000 7f559d5f8640 1 -- 172.21.15.26:0/4246469891 learned_addr learned my addr 172.21.15.26:0/4246469891 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:21:12.485 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.484+0000 7f559cdf7640 1 --2- 172.21.15.26:0/4246469891 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c0f1ae1410 0x55c0f1ae2c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:12.485 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.484+0000 7f559ddf9640 1 --2- 172.21.15.26:0/4246469891 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c0f1aebcb0 0x55c0f1b03d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:12.485 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.484+0000 7f559d5f8640 1 -- 172.21.15.26:0/4246469891 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c0f1aebcb0 msgr2=0x55c0f1b03d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:21:12.485 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.484+0000 7f559d5f8640 1 --2- 172.21.15.26:0/4246469891 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c0f1aebcb0 0x55c0f1b03d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:12.485 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.484+0000 7f559d5f8640 1 -- 172.21.15.26:0/4246469891 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c0f1ae1410 msgr2=0x55c0f1ae2c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:21:12.485 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.484+0000 7f559d5f8640 1 --2- 172.21.15.26:0/4246469891 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c0f1ae1410 0x55c0f1ae2c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:12.485 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.484+0000 7f559d5f8640 1 -- 172.21.15.26:0/4246469891 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c0f1adce90 con 0x55c0f1ae0af0 2024-09-11T23:21:12.485 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.484+0000 7f559d5f8640 1 --2- 172.21.15.26:0/4246469891 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c0f1ae0af0 0x55c0f1ae93c0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f5594005940 tx=0x7f5594000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6510bc4bc732c03c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:21:12.486 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.484+0000 7f5582ffd640 1 -- 172.21.15.26:0/4246469891 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5594054020 con 0x55c0f1ae0af0 2024-09-11T23:21:12.486 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.484+0000 7f5582ffd640 1 -- 172.21.15.26:0/4246469891 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5594004230 con 0x55c0f1ae0af0 2024-09-11T23:21:12.486 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.484+0000 7f5582ffd640 1 -- 172.21.15.26:0/4246469891 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5594040070 con 0x55c0f1ae0af0 2024-09-11T23:21:12.488 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.485+0000 7f55a25ed740 1 -- 172.21.15.26:0/4246469891 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c0f1ae0af0 msgr2=0x55c0f1ae93c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:12.489 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.485+0000 7f55a25ed740 1 --2- 172.21.15.26:0/4246469891 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c0f1ae0af0 0x55c0f1ae93c0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f5594005940 tx=0x7f5594000bc0 comp rx=0 tx=0).stop 2024-09-11T23:21:12.489 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.485+0000 7f55a25ed740 1 -- 172.21.15.26:0/4246469891 shutdown_connections 2024-09-11T23:21:12.489 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.485+0000 7f55a25ed740 1 --2- 172.21.15.26:0/4246469891 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c0f1ae1410 0x55c0f1ae2c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:12.489 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.485+0000 7f55a25ed740 1 --2- 172.21.15.26:0/4246469891 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c0f1aebcb0 0x55c0f1b03d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:12.489 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.485+0000 7f55a25ed740 1 --2- 172.21.15.26:0/4246469891 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c0f1ae0af0 0x55c0f1ae93c0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:12.489 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.485+0000 7f55a25ed740 1 -- 172.21.15.26:0/4246469891 >> 172.21.15.26:0/4246469891 conn(0x55c0f1add060 msgr2=0x55c0f1ae2550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:21:12.489 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.485+0000 7f55a25ed740 1 -- 172.21.15.26:0/4246469891 shutdown_connections 2024-09-11T23:21:12.489 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.485+0000 7f55a25ed740 1 -- 172.21.15.26:0/4246469891 wait complete. 2024-09-11T23:21:12.489 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.485+0000 7f55a25ed740 1 -- start start 2024-09-11T23:21:12.489 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.485+0000 7f55a25ed740 1 Processor -- start 2024-09-11T23:21:12.489 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.486+0000 7f55a25ed740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c0f1ae0af0 0x55c0f1af44d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:12.489 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.486+0000 7f55a25ed740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c0f1ae1410 0x55c0f1ae58f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:12.490 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.486+0000 7f55a25ed740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c0f1aebcb0 0x55c0f1ae5ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:12.490 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.486+0000 7f55a25ed740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f55981b0f70 con 0x55c0f1aebcb0 2024-09-11T23:21:12.490 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.486+0000 7f55a25ed740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f55981b07e0 con 0x55c0f1ae0af0 2024-09-11T23:21:12.490 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.486+0000 7f55a25ed740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f55981ada20 con 0x55c0f1ae1410 2024-09-11T23:21:12.490 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.486+0000 7f55a25ed740 10 client.0 _ll_register_callbacks cb 0x7f558800e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:21:12.490 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.486+0000 7f559ddf9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c0f1aebcb0 0x55c0f1ae5ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:12.490 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.486+0000 7f559d5f8640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c0f1ae0af0 0x55c0f1af44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:12.490 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.486+0000 7f559cdf7640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c0f1ae1410 0x55c0f1ae58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:12.490 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.486+0000 7f559ddf9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c0f1aebcb0 0x55c0f1ae5ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36318/0 (socket says 172.21.15.26:36318) 2024-09-11T23:21:12.490 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.486+0000 7f559ddf9640 1 -- 172.21.15.26:0/4109739666 learned_addr learned my addr 172.21.15.26:0/4109739666 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:21:12.491 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.486+0000 7f559ddf9640 1 -- 172.21.15.26:0/4109739666 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c0f1ae1410 msgr2=0x55c0f1ae58f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:21:12.491 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.486+0000 7f559ddf9640 1 --2- 172.21.15.26:0/4109739666 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c0f1ae1410 0x55c0f1ae58f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:12.491 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.486+0000 7f559ddf9640 1 -- 172.21.15.26:0/4109739666 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c0f1ae0af0 msgr2=0x55c0f1af44d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:21:12.491 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.486+0000 7f559ddf9640 1 --2- 172.21.15.26:0/4109739666 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c0f1ae0af0 0x55c0f1af44d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:12.491 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.486+0000 7f559ddf9640 1 -- 172.21.15.26:0/4109739666 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c0f1ae3940 con 0x55c0f1aebcb0 2024-09-11T23:21:12.491 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.486+0000 7f559cdf7640 1 --2- 172.21.15.26:0/4109739666 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c0f1ae1410 0x55c0f1ae58f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:21:12.491 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.486+0000 7f559d5f8640 1 --2- 172.21.15.26:0/4109739666 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c0f1ae0af0 0x55c0f1af44d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:21:12.491 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.487+0000 7f559ddf9640 1 --2- 172.21.15.26:0/4109739666 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c0f1aebcb0 0x55c0f1ae5ee0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f55900041a0 tx=0x7f5590004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c00f2d8748d573c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:21:12.491 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.487+0000 7f5580ff9640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:21:12.491 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.487+0000 7f5580ff9640 1 -- 172.21.15.26:0/4109739666 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55900047e0 con 0x55c0f1aebcb0 2024-09-11T23:21:12.491 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.487+0000 7f55a25ed740 1 -- 172.21.15.26:0/4109739666 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5594004230 con 0x55c0f1aebcb0 2024-09-11T23:21:12.491 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.487+0000 7f5580ff9640 1 -- 172.21.15.26:0/4109739666 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f559000fc70 con 0x55c0f1aebcb0 2024-09-11T23:21:12.491 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.487+0000 7f5580ff9640 1 -- 172.21.15.26:0/4109739666 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f5590004b20 con 0x55c0f1aebcb0 2024-09-11T23:21:12.492 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.487+0000 7f55a25ed740 10 client.16215 fetch_fsmap learned FSMap version 33 2024-09-11T23:21:12.492 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.487+0000 7f55a25ed740 1 -- 172.21.15.26:0/4109739666 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55c0f1b66cd0 con 0x55c0f1aebcb0 2024-09-11T23:21:12.492 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.487+0000 7f5580ff9640 1 -- 172.21.15.26:0/4109739666 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5594004230 con 0x55c0f1aebcb0 2024-09-11T23:21:12.492 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.488+0000 7f5580ff9640 1 -- 172.21.15.26:0/4109739666 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 33) ==== 8394+0+0 (secure 0 0 0) 0x7f5590028420 con 0x55c0f1aebcb0 2024-09-11T23:21:12.492 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.488+0000 7f55a25ed740 10 client.16215 fetch_fsmap finished waiting for FSMap version 33 2024-09-11T23:21:12.492 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.488+0000 7f55a25ed740 10 client.16215 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:21:12.492 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.488+0000 7f55a25ed740 20 client.16215 populate_metadata read hostname 'smithi026' 2024-09-11T23:21:12.492 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.488+0000 7f55a25ed740 1 --2- 172.21.15.26:0/4109739666 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55c0f1b67530 0x55c0f1b69950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:12.492 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.488+0000 7f55a25ed740 4 client.16215 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T23:21:12.492 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.488+0000 7f55a25ed740 1 -- 172.21.15.26:0/4109739666 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c0f1a4cd70 con 0x55c0f1b67530 2024-09-11T23:21:12.492 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.488+0000 7f559d5f8640 1 --2- 172.21.15.26:0/4109739666 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55c0f1b67530 0x55c0f1b69950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:12.492 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.488+0000 7f5580ff9640 10 client.16215 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:21:12.492 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.488+0000 7f559d5f8640 1 --2- 172.21.15.26:0/4109739666 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55c0f1b67530 0x55c0f1b69950 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d629ba6b428358a server_cookie=7e804507346894ac in_seq=0 out_seq=0 2024-09-11T23:21:12.649 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.648+0000 7f5580ff9640 1 -- 172.21.15.26:0/4109739666 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c0f1a4cd70 con 0x55c0f1b67530 2024-09-11T23:21:12.649 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.648+0000 7f5580ff9640 10 client.16215 handle_command_reply: tid=0 2024-09-11T23:21:12.649 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.648+0000 7f55a25ed740 1 client.16215 shutdown 2024-09-11T23:21:12.649 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.648+0000 7f55a25ed740 1 -- 172.21.15.26:0/4109739666 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c0f1aebcb0 msgr2=0x55c0f1ae5ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:12.649 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.648+0000 7f55a25ed740 1 --2- 172.21.15.26:0/4109739666 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c0f1aebcb0 0x55c0f1ae5ee0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f55900041a0 tx=0x7f5590004a40 comp rx=0 tx=0).stop 2024-09-11T23:21:12.649 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.648+0000 7f55a25ed740 1 -- 172.21.15.26:0/4109739666 shutdown_connections 2024-09-11T23:21:12.650 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.648+0000 7f55a25ed740 1 --2- 172.21.15.26:0/4109739666 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55c0f1b67530 0x55c0f1b69950 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:12.650 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.648+0000 7f55a25ed740 1 --2- 172.21.15.26:0/4109739666 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c0f1aebcb0 0x55c0f1ae5ee0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:12.650 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.648+0000 7f55a25ed740 1 --2- 172.21.15.26:0/4109739666 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c0f1ae1410 0x55c0f1ae58f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:12.650 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.648+0000 7f5580ff9640 0 client.16215 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:21:12.650 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.648+0000 7f55a25ed740 1 --2- 172.21.15.26:0/4109739666 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c0f1ae0af0 0x55c0f1af44d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:12.650 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.648+0000 7f55a25ed740 1 -- 172.21.15.26:0/4109739666 >> 172.21.15.26:0/4109739666 conn(0x55c0f1add060 msgr2=0x55c0f1adc1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:21:12.650 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.649+0000 7f55a25ed740 1 -- 172.21.15.26:0/4109739666 shutdown_connections 2024-09-11T23:21:12.650 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.649+0000 7f55a25ed740 1 -- 172.21.15.26:0/4109739666 wait complete. 2024-09-11T23:21:12.650 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.649+0000 7f55a25ed740 20 client.16215 trim_cache size 0 max 16384 2024-09-11T23:21:12.651 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-09-11T23:21:12.651 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "message": "", 2024-09-11T23:21:12.651 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-09-11T23:21:12.651 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-09-11T23:21:12.651 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.649+0000 7f559f85c640 1 -- 172.21.15.26:0/957724630 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f55840a2e20 msgr2=0x7f55840a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:12.652 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.649+0000 7f559f85c640 1 --2- 172.21.15.26:0/957724630 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f55840a2e20 0x7f55840a52e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f5590011630 tx=0x7f5590009500 comp rx=0 tx=0).stop 2024-09-11T23:21:12.652 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.649+0000 7f559f85c640 1 -- 172.21.15.26:0/957724630 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5598102470 msgr2=0x7f5598074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:12.652 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.649+0000 7f559f85c640 1 --2- 172.21.15.26:0/957724630 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5598102470 0x7f5598074a30 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f558800e9f0 tx=0x7f558800edf0 comp rx=0 tx=0).stop 2024-09-11T23:21:12.652 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.649+0000 7f559f85c640 1 -- 172.21.15.26:0/957724630 shutdown_connections 2024-09-11T23:21:12.652 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.649+0000 7f559f85c640 1 --2- 172.21.15.26:0/957724630 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f55840a2e20 0x7f55840a52e0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:12.652 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.649+0000 7f559f85c640 1 --2- 172.21.15.26:0/957724630 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5598107b70 0x7f5598074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:12.652 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.649+0000 7f559f85c640 1 --2- 172.21.15.26:0/957724630 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5598102470 0x7f5598074a30 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:12.652 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.649+0000 7f559f85c640 1 --2- 172.21.15.26:0/957724630 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5598101ad0 0x7f55980744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:12.652 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.649+0000 7f559f85c640 1 -- 172.21.15.26:0/957724630 >> 172.21.15.26:0/957724630 conn(0x7f55980fd7d0 msgr2=0x7f55980fdfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:21:12.652 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.650+0000 7f559f85c640 1 -- 172.21.15.26:0/957724630 shutdown_connections 2024-09-11T23:21:12.652 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:12.650+0000 7f559f85c640 1 -- 172.21.15.26:0/957724630 wait complete. 2024-09-11T23:21:13.577 INFO:tasks.workunit.client.0.smithi026.stdout: 117 29195 706 18420 574 8673 104 2024-09-11T23:21:13.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:21:13 smithi136 ceph-mon[35121]: pgmap v88: 97 pgs: 97 active+clean; 360 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.3 MiB/s wr, 316 op/s 2024-09-11T23:21:13.750 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:21:13 smithi026 ceph-mon[31172]: pgmap v88: 97 pgs: 97 active+clean; 360 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.3 MiB/s wr, 316 op/s 2024-09-11T23:21:13.802 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:21:13 smithi076 ceph-mon[35296]: pgmap v88: 97 pgs: 97 active+clean; 360 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.3 MiB/s wr, 316 op/s 2024-09-11T23:21:15.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:21:15 smithi136 ceph-mon[35121]: pgmap v89: 97 pgs: 97 active+clean; 374 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.0 MiB/s wr, 645 op/s 2024-09-11T23:21:15.750 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:21:15 smithi026 ceph-mon[31172]: pgmap v89: 97 pgs: 97 active+clean; 374 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.0 MiB/s wr, 645 op/s 2024-09-11T23:21:15.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:21:15 smithi076 ceph-mon[35296]: pgmap v89: 97 pgs: 97 active+clean; 374 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.0 MiB/s wr, 645 op/s 2024-09-11T23:21:16.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:16.960+0000 7fd336ffd640 1 -- 172.21.15.26:0/4225889498 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x560bef92fd20 con 0x560bef9c6970 2024-09-11T23:21:16.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:16.960+0000 7fd336ffd640 10 client.25775 handle_command_reply: tid=0 2024-09-11T23:21:16.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:16.960+0000 7fd360513740 1 client.25775 shutdown 2024-09-11T23:21:16.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:16.961+0000 7fd360513740 1 -- 172.21.15.26:0/4225889498 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560bef9c2f60 msgr2=0x560bef9d5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:16.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:16.961+0000 7fd360513740 1 --2- 172.21.15.26:0/4225889498 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560bef9c2f60 0x560bef9d5870 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x560bef9d4ab0 tx=0x7fd340005f30 comp rx=0 tx=0).stop 2024-09-11T23:21:16.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:16.961+0000 7fd360513740 1 -- 172.21.15.26:0/4225889498 shutdown_connections 2024-09-11T23:21:16.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:16.961+0000 7fd360513740 1 --2- 172.21.15.26:0/4225889498 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560bef9c2f60 0x560bef9d5870 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:16.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:16.961+0000 7fd360513740 1 --2- 172.21.15.26:0/4225889498 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x560bef9c6970 0x560bef9c8e10 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:16.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:16.961+0000 7fd360513740 1 --2- 172.21.15.26:0/4225889498 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560bef9d1d50 0x560bef9d5e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:16.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:16.961+0000 7fd360513740 1 --2- 172.21.15.26:0/4225889498 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560bef9c2640 0x560bef9da4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:16.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:16.961+0000 7fd336ffd640 0 client.25775 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:21:16.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:16.961+0000 7fd360513740 1 -- 172.21.15.26:0/4225889498 >> 172.21.15.26:0/4225889498 conn(0x560bef9becf0 msgr2=0x560bef9bde70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:21:16.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:16.961+0000 7fd360513740 1 -- 172.21.15.26:0/4225889498 shutdown_connections 2024-09-11T23:21:16.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:16.961+0000 7fd360513740 1 -- 172.21.15.26:0/4225889498 wait complete. 2024-09-11T23:21:16.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:16.962+0000 7fd360513740 20 client.25775 trim_cache size 0 max 16384 2024-09-11T23:21:16.963 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-11T23:21:16.963 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 33, 2024-09-11T23:21:16.963 INFO:teuthology.orchestra.run.smithi026.stdout: "leader": 24485, 2024-09-11T23:21:16.963 INFO:teuthology.orchestra.run.smithi026.stdout: "set_version": 10, 2024-09-11T23:21:16.963 INFO:teuthology.orchestra.run.smithi026.stdout: "sets": { 2024-09-11T23:21:16.963 INFO:teuthology.orchestra.run.smithi026.stdout: "6529ce3d": { 2024-09-11T23:21:16.963 INFO:teuthology.orchestra.run.smithi026.stdout: "version": 10, 2024-09-11T23:21:16.964 INFO:teuthology.orchestra.run.smithi026.stdout: "age_ref": 0.0, 2024-09-11T23:21:16.964 INFO:teuthology.orchestra.run.smithi026.stdout: "state": { 2024-09-11T23:21:16.964 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "QUIESCED", 2024-09-11T23:21:16.964 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.0 2024-09-11T23:21:16.964 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:21:16.964 INFO:teuthology.orchestra.run.smithi026.stdout: "timeout": 90.0, 2024-09-11T23:21:16.964 INFO:teuthology.orchestra.run.smithi026.stdout: "expiration": 144.7, 2024-09-11T23:21:16.964 INFO:teuthology.orchestra.run.smithi026.stdout: "members": { 2024-09-11T23:21:16.964 INFO:teuthology.orchestra.run.smithi026.stdout: "file:/": { 2024-09-11T23:21:16.964 INFO:teuthology.orchestra.run.smithi026.stdout: "excluded": false, 2024-09-11T23:21:16.964 INFO:teuthology.orchestra.run.smithi026.stdout: "state": { 2024-09-11T23:21:16.964 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "QUIESCED", 2024-09-11T23:21:16.964 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.0 2024-09-11T23:21:16.965 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:21:16.965 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:21:16.965 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:21:16.965 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:21:16.965 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:21:16.965 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-11T23:21:16.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:16.962+0000 7fd35e13a640 1 -- 172.21.15.26:0/2097500553 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fd3440a2d70 msgr2=0x7fd3440a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:16.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:16.962+0000 7fd35e13a640 1 --2- 172.21.15.26:0/2097500553 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fd3440a2d70 0x7fd3440a5230 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fd350004ab0 tx=0x7fd350004a40 comp rx=0 tx=0).stop 2024-09-11T23:21:16.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:16.962+0000 7fd35e13a640 1 -- 172.21.15.26:0/2097500553 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd358102d90 msgr2=0x7fd3581985e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:16.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:16.962+0000 7fd35e13a640 1 --2- 172.21.15.26:0/2097500553 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd358102d90 0x7fd3581985e0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fd34800d950 tx=0x7fd34800de10 comp rx=0 tx=0).stop 2024-09-11T23:21:16.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:16.962+0000 7fd35e13a640 1 -- 172.21.15.26:0/2097500553 shutdown_connections 2024-09-11T23:21:16.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:16.962+0000 7fd35e13a640 1 --2- 172.21.15.26:0/2097500553 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fd3440a2d70 0x7fd3440a5230 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:16.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:16.962+0000 7fd35e13a640 1 --2- 172.21.15.26:0/2097500553 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd358108290 0x7fd35819ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:16.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:16.962+0000 7fd35e13a640 1 --2- 172.21.15.26:0/2097500553 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd358103740 0x7fd358198b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:16.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:16.962+0000 7fd35e13a640 1 --2- 172.21.15.26:0/2097500553 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd358102d90 0x7fd3581985e0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:16.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:16.962+0000 7fd35e13a640 1 -- 172.21.15.26:0/2097500553 >> 172.21.15.26:0/2097500553 conn(0x7fd3580fc7d0 msgr2=0x7fd358106370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:21:16.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:16.963+0000 7fd35e13a640 1 -- 172.21.15.26:0/2097500553 shutdown_connections 2024-09-11T23:21:16.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:16.963+0000 7fd35e13a640 1 -- 172.21.15.26:0/2097500553 wait complete. 2024-09-11T23:21:17.604 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:21:17 smithi076 ceph-mon[35296]: pgmap v90: 97 pgs: 97 active+clean; 374 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.9 MiB/s wr, 446 op/s 2024-09-11T23:21:17.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:21:17 smithi136 ceph-mon[35121]: pgmap v90: 97 pgs: 97 active+clean; 374 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.9 MiB/s wr, 446 op/s 2024-09-11T23:21:17.750 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:21:17 smithi026 ceph-mon[31172]: pgmap v90: 97 pgs: 97 active+clean; 374 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.9 MiB/s wr, 446 op/s 2024-09-11T23:21:17.786 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '6529ce3d', quiesce took 13.9 seconds. Will release after: 10.799999999999999 2024-09-11T23:21:19.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:21:19 smithi136 ceph-mon[35121]: pgmap v91: 97 pgs: 97 active+clean; 387 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.1 MiB/s wr, 563 op/s 2024-09-11T23:21:19.750 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:21:19 smithi026 ceph-mon[31172]: pgmap v91: 97 pgs: 97 active+clean; 387 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.1 MiB/s wr, 563 op/s 2024-09-11T23:21:19.802 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:21:19 smithi076 ceph-mon[35296]: pgmap v91: 97 pgs: 97 active+clean; 387 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.1 MiB/s wr, 563 op/s 2024-09-11T23:21:21.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:21:21 smithi136 ceph-mon[35121]: pgmap v92: 97 pgs: 97 active+clean; 387 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.9 MiB/s wr, 447 op/s 2024-09-11T23:21:21.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:21:21 smithi026 ceph-mon[31172]: pgmap v92: 97 pgs: 97 active+clean; 387 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.9 MiB/s wr, 447 op/s 2024-09-11T23:21:21.802 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:21:21 smithi076 ceph-mon[35296]: pgmap v92: 97 pgs: 97 active+clean; 387 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.9 MiB/s wr, 447 op/s 2024-09-11T23:21:23.577 INFO:tasks.workunit.client.0.smithi026.stdout: 117 14953 0 10062 0 6231 0 2024-09-11T23:21:23.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:21:23 smithi136 ceph-mon[35121]: pgmap v93: 97 pgs: 97 active+clean; 387 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.9 MiB/s wr, 447 op/s 2024-09-11T23:21:23.750 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:21:23 smithi026 ceph-mon[31172]: pgmap v93: 97 pgs: 97 active+clean; 387 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.9 MiB/s wr, 447 op/s 2024-09-11T23:21:23.802 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:21:23 smithi076 ceph-mon[35296]: pgmap v93: 97 pgs: 97 active+clean; 387 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.9 MiB/s wr, 447 op/s 2024-09-11T23:21:25.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:21:25 smithi136 ceph-mon[35121]: pgmap v94: 97 pgs: 97 active+clean; 399 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 4.0 MiB/s wr, 622 op/s 2024-09-11T23:21:25.750 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:21:25 smithi026 ceph-mon[31172]: pgmap v94: 97 pgs: 97 active+clean; 399 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 4.0 MiB/s wr, 622 op/s 2024-09-11T23:21:25.758 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.755+0000 7f80581cf640 1 -- 172.21.15.26:0/4245036435 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f80501069c0 msgr2=0x7f8050106da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:25.758 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.755+0000 7f80581cf640 1 --2- 172.21.15.26:0/4245036435 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f80501069c0 0x7f8050106da0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f804c009930 tx=0x7f804c02a420 comp rx=0 tx=0).stop 2024-09-11T23:21:25.759 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.755+0000 7f80581cf640 1 -- 172.21.15.26:0/4245036435 shutdown_connections 2024-09-11T23:21:25.759 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.755+0000 7f80581cf640 1 --2- 172.21.15.26:0/4245036435 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8050101350 0x7f805010d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:25.759 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.755+0000 7f80581cf640 1 --2- 172.21.15.26:0/4245036435 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f80501009b0 0x7f8050100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:25.759 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.755+0000 7f80581cf640 1 --2- 172.21.15.26:0/4245036435 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f80501069c0 0x7f8050106da0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:25.759 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.755+0000 7f80581cf640 1 -- 172.21.15.26:0/4245036435 >> 172.21.15.26:0/4245036435 conn(0x7f8050077990 msgr2=0x7f80500fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:21:25.759 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.756+0000 7f80581cf640 1 -- 172.21.15.26:0/4245036435 shutdown_connections 2024-09-11T23:21:25.759 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.756+0000 7f80581cf640 1 -- 172.21.15.26:0/4245036435 wait complete. 2024-09-11T23:21:25.759 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.756+0000 7f80581cf640 1 Processor -- start 2024-09-11T23:21:25.759 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.756+0000 7f80581cf640 1 -- start start 2024-09-11T23:21:25.759 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.756+0000 7f80581cf640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f80501009b0 0x7f8050198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:25.759 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.756+0000 7f80581cf640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8050101350 0x7f80501988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:25.759 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.756+0000 7f80581cf640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f80501069c0 0x7f805019cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:25.759 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.756+0000 7f80581cf640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f8050110dc0 con 0x7f80501069c0 2024-09-11T23:21:25.760 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.756+0000 7f80581cf640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f8050110c40 con 0x7f80501009b0 2024-09-11T23:21:25.760 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.756+0000 7f80581cf640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f8050110f40 con 0x7f8050101350 2024-09-11T23:21:25.760 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.757+0000 7f8055f6b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f80501009b0 0x7f8050198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:25.760 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.757+0000 7f805676c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f80501069c0 0x7f805019cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:25.760 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.757+0000 7f805576a640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8050101350 0x7f80501988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:25.760 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.757+0000 7f8055f6b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f80501009b0 0x7f8050198370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:38110/0 (socket says 172.21.15.26:38110) 2024-09-11T23:21:25.760 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.757+0000 7f805576a640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8050101350 0x7f80501988b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:36548/0 (socket says 172.21.15.26:36548) 2024-09-11T23:21:25.760 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.757+0000 7f8055f6b640 1 -- 172.21.15.26:0/3574868048 learned_addr learned my addr 172.21.15.26:0/3574868048 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:21:25.760 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.757+0000 7f805676c640 1 -- 172.21.15.26:0/3574868048 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8050101350 msgr2=0x7f80501988b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:25.760 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.757+0000 7f805676c640 1 --2- 172.21.15.26:0/3574868048 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8050101350 0x7f80501988b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:25.760 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.757+0000 7f805676c640 1 -- 172.21.15.26:0/3574868048 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f80501009b0 msgr2=0x7f8050198370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:25.760 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.757+0000 7f805676c640 1 --2- 172.21.15.26:0/3574868048 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f80501009b0 0x7f8050198370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:25.760 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.757+0000 7f805676c640 1 -- 172.21.15.26:0/3574868048 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f805019d360 con 0x7f80501069c0 2024-09-11T23:21:25.761 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.757+0000 7f805576a640 1 --2- 172.21.15.26:0/3574868048 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8050101350 0x7f80501988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:21:25.761 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.757+0000 7f8055f6b640 1 --2- 172.21.15.26:0/3574868048 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f80501009b0 0x7f8050198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:21:25.761 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.757+0000 7f805676c640 1 --2- 172.21.15.26:0/3574868048 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f80501069c0 0x7f805019cc30 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f804800da10 tx=0x7f804800ded0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:21:25.761 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.757+0000 7f8046ffd640 1 -- 172.21.15.26:0/3574868048 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f804800bd70 con 0x7f80501069c0 2024-09-11T23:21:25.761 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.757+0000 7f8046ffd640 1 -- 172.21.15.26:0/3574868048 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8048003e50 con 0x7f80501069c0 2024-09-11T23:21:25.761 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.757+0000 7f80581cf640 1 -- 172.21.15.26:0/3574868048 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f805019d650 con 0x7f80501069c0 2024-09-11T23:21:25.761 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.757+0000 7f8046ffd640 1 -- 172.21.15.26:0/3574868048 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80480106a0 con 0x7f80501069c0 2024-09-11T23:21:25.761 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.757+0000 7f80581cf640 1 -- 172.21.15.26:0/3574868048 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80500fcd10 con 0x7f80501069c0 2024-09-11T23:21:25.764 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.761+0000 7f805b044740 1 Processor -- start 2024-09-11T23:21:25.764 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.761+0000 7f8046ffd640 1 -- 172.21.15.26:0/3574868048 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f8048003970 con 0x7f80501069c0 2024-09-11T23:21:25.764 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.761+0000 7f805b044740 1 -- start start 2024-09-11T23:21:25.764 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.761+0000 7f805b044740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559f7720caf0 0x559f7720ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:25.764 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.761+0000 7f805b044740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559f7720d410 0x559f77217770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:25.765 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.761+0000 7f805b044740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559f77217cb0 0x559f7721a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:25.765 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.761+0000 7f805b044740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x559f771712f0 con 0x559f7720caf0 2024-09-11T23:21:25.765 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.761+0000 7f805b044740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x559f77119270 con 0x559f77217cb0 2024-09-11T23:21:25.765 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.761+0000 7f805b044740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x559f7716d2b0 con 0x559f7720d410 2024-09-11T23:21:25.765 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.761+0000 7f805576a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559f7720caf0 0x559f7720ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:25.765 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.761+0000 7f805576a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559f7720caf0 0x559f7720ced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43540/0 (socket says 172.21.15.26:43540) 2024-09-11T23:21:25.765 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.761+0000 7f8055f6b640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559f7720d410 0x559f77217770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:25.765 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.761+0000 7f805576a640 1 -- 172.21.15.26:0/2995302627 learned_addr learned my addr 172.21.15.26:0/2995302627 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:21:25.765 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.761+0000 7f805576a640 1 --2- 172.21.15.26:0/2995302627 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559f77217cb0 0x559f7721a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:25.765 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.762+0000 7f805576a640 1 -- 172.21.15.26:0/2995302627 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559f7720d410 msgr2=0x559f77217770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:21:25.765 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.762+0000 7f805576a640 1 --2- 172.21.15.26:0/2995302627 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559f7720d410 0x559f77217770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:25.766 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.762+0000 7f805576a640 1 -- 172.21.15.26:0/2995302627 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559f77217cb0 msgr2=0x559f7721a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:21:25.766 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.762+0000 7f805576a640 1 --2- 172.21.15.26:0/2995302627 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559f77217cb0 0x559f7721a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:25.766 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.762+0000 7f8046ffd640 1 --2- 172.21.15.26:0/3574868048 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f803c0a2da0 0x7f803c0a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:25.766 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.762+0000 7f805576a640 1 -- 172.21.15.26:0/2995302627 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f77118340 con 0x559f7720caf0 2024-09-11T23:21:25.766 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.762+0000 7f8055f6b640 1 --2- 172.21.15.26:0/2995302627 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559f7720d410 0x559f77217770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:21:25.766 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.762+0000 7f805676c640 1 --2- 172.21.15.26:0/3574868048 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f803c0a2da0 0x7f803c0a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:25.766 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.762+0000 7f805576a640 1 --2- 172.21.15.26:0/2995302627 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559f7720caf0 0x559f7720ced0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f8040013a20 tx=0x7f8040009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2185cc0ba8178382 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:21:25.766 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.762+0000 7f8046ffd640 1 -- 172.21.15.26:0/3574868048 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f80480cf780 con 0x7f80501069c0 2024-09-11T23:21:25.766 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.762+0000 7f803bfff640 1 -- 172.21.15.26:0/2995302627 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8040004030 con 0x559f7720caf0 2024-09-11T23:21:25.766 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.762+0000 7f803bfff640 1 -- 172.21.15.26:0/2995302627 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f804000fe60 con 0x559f7720caf0 2024-09-11T23:21:25.766 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.762+0000 7f803bfff640 1 -- 172.21.15.26:0/2995302627 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8040003aa0 con 0x559f7720caf0 2024-09-11T23:21:25.766 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.763+0000 7f805b044740 1 -- 172.21.15.26:0/2995302627 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559f7720caf0 msgr2=0x559f7720ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:25.766 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.763+0000 7f805b044740 1 --2- 172.21.15.26:0/2995302627 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559f7720caf0 0x559f7720ced0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f8040013a20 tx=0x7f8040009c00 comp rx=0 tx=0).stop 2024-09-11T23:21:25.766 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.763+0000 7f805676c640 1 --2- 172.21.15.26:0/3574868048 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f803c0a2da0 0x7f803c0a5260 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f80480ccf90 tx=0x7f8048091140 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:21:25.766 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.763+0000 7f805b044740 1 -- 172.21.15.26:0/2995302627 shutdown_connections 2024-09-11T23:21:25.767 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.763+0000 7f805b044740 1 --2- 172.21.15.26:0/2995302627 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559f77217cb0 0x559f7721a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:25.767 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.763+0000 7f805b044740 1 --2- 172.21.15.26:0/2995302627 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559f7720d410 0x559f77217770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:25.767 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.763+0000 7f805b044740 1 --2- 172.21.15.26:0/2995302627 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559f7720caf0 0x559f7720ced0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:25.767 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.763+0000 7f805b044740 1 -- 172.21.15.26:0/2995302627 >> 172.21.15.26:0/2995302627 conn(0x559f77209060 msgr2=0x559f77209440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:21:25.767 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.763+0000 7f805b044740 1 -- 172.21.15.26:0/2995302627 shutdown_connections 2024-09-11T23:21:25.767 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.763+0000 7f805b044740 1 -- 172.21.15.26:0/2995302627 wait complete. 2024-09-11T23:21:25.767 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.763+0000 7f805b044740 1 -- start start 2024-09-11T23:21:25.767 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.763+0000 7f805b044740 1 Processor -- start 2024-09-11T23:21:25.770 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.764+0000 7f805b044740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559f7720caf0 0x559f77214b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:25.770 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.764+0000 7f805b044740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559f7720d410 0x559f77215040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:25.770 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.764+0000 7f805b044740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559f77217cb0 0x559f77215580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:25.770 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.764+0000 7f805b044740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x559f7721a880 con 0x559f77217cb0 2024-09-11T23:21:25.770 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.764+0000 7f805b044740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x559f770fa290 con 0x559f7720d410 2024-09-11T23:21:25.770 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.764+0000 7f805b044740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x559f7721aa00 con 0x559f7720caf0 2024-09-11T23:21:25.771 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.764+0000 7f805b044740 10 client.0 _ll_register_callbacks cb 0x559f7712b140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:21:25.771 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.764+0000 7f805576a640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559f7720caf0 0x559f77214b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:25.771 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.764+0000 7f805576a640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559f7720caf0 0x559f77214b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:36560/0 (socket says 172.21.15.26:36560) 2024-09-11T23:21:25.771 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.764+0000 7f805576a640 1 -- 172.21.15.26:0/551317423 learned_addr learned my addr 172.21.15.26:0/551317423 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:21:25.771 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.764+0000 7f8055f6b640 1 --2- 172.21.15.26:0/551317423 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559f7720d410 0x559f77215040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:25.771 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.764+0000 7f805576a640 1 --2- 172.21.15.26:0/551317423 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559f77217cb0 0x559f77215580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:25.771 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.764+0000 7f805576a640 1 -- 172.21.15.26:0/551317423 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559f7720d410 msgr2=0x559f77215040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:21:25.771 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.764+0000 7f805576a640 1 --2- 172.21.15.26:0/551317423 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559f7720d410 0x559f77215040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:25.771 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.764+0000 7f805576a640 1 -- 172.21.15.26:0/551317423 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559f77217cb0 msgr2=0x559f77215580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:21:25.771 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.764+0000 7f805576a640 1 --2- 172.21.15.26:0/551317423 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559f77217cb0 0x559f77215580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:25.771 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.764+0000 7f805576a640 1 -- 172.21.15.26:0/551317423 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f76e84210 con 0x559f7720caf0 2024-09-11T23:21:25.771 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.764+0000 7f8055f6b640 1 --2- 172.21.15.26:0/551317423 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559f7720d410 0x559f77215040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:21:25.771 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.764+0000 7f805576a640 1 --2- 172.21.15.26:0/551317423 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559f7720caf0 0x559f77214b00 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f8040009dc0 tx=0x7f8040005850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5306b67bef538f9b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:21:25.772 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.764+0000 7f80397fa640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-09-11T23:21:25.772 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.765+0000 7f80397fa640 1 -- 172.21.15.26:0/551317423 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8040005260 con 0x559f7720caf0 2024-09-11T23:21:25.772 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.765+0000 7f805b044740 1 -- 172.21.15.26:0/551317423 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f804000fe60 con 0x559f7720caf0 2024-09-11T23:21:25.772 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.765+0000 7f80397fa640 1 -- 172.21.15.26:0/551317423 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f8040005400 con 0x559f7720caf0 2024-09-11T23:21:25.772 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.765+0000 7f80397fa640 1 -- 172.21.15.26:0/551317423 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f8040035050 con 0x559f7720caf0 2024-09-11T23:21:25.772 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.765+0000 7f805b044740 10 client.25861 fetch_fsmap learned FSMap version 33 2024-09-11T23:21:25.772 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.765+0000 7f805b044740 1 -- 172.21.15.26:0/551317423 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x559f76f50620 con 0x559f7720caf0 2024-09-11T23:21:25.772 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.765+0000 7f80397fa640 1 -- 172.21.15.26:0/551317423 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f804000fe60 con 0x559f7720caf0 2024-09-11T23:21:25.772 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.765+0000 7f80397fa640 1 -- 172.21.15.26:0/551317423 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 33) ==== 8394+0+0 (secure 0 0 0) 0x7f804001e070 con 0x559f7720caf0 2024-09-11T23:21:25.772 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.765+0000 7f805b044740 10 client.25861 fetch_fsmap finished waiting for FSMap version 33 2024-09-11T23:21:25.772 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.765+0000 7f805b044740 10 client.25861 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:21:25.772 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.765+0000 7f805b044740 20 client.25861 populate_metadata read hostname 'smithi026' 2024-09-11T23:21:25.772 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.766+0000 7f805b044740 1 --2- 172.21.15.26:0/551317423 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x559f77227a80 0x559f77229ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:25.772 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.766+0000 7f805b044740 4 client.25861 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:21:25.773 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.766+0000 7f805b044740 1 -- 172.21.15.26:0/551317423 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559f7717ea20 con 0x559f77227a80 2024-09-11T23:21:25.773 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.766+0000 7f8055f6b640 1 --2- 172.21.15.26:0/551317423 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x559f77227a80 0x559f77229ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:25.773 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.766+0000 7f8055f6b640 1 --2- 172.21.15.26:0/551317423 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x559f77227a80 0x559f77229ea0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=902c65c5e4ec1caa server_cookie=62d8f7749959d2d0 in_seq=0 out_seq=0 2024-09-11T23:21:25.773 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.766+0000 7f80397fa640 10 client.25861 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:21:25.773 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.767+0000 7f80397fa640 1 -- 172.21.15.26:0/551317423 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559f7717ea20 con 0x559f77227a80 2024-09-11T23:21:25.773 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.767+0000 7f80397fa640 10 client.25861 handle_command_reply: tid=0 2024-09-11T23:21:25.773 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.767+0000 7f805b044740 1 client.25861 shutdown 2024-09-11T23:21:25.773 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.767+0000 7f805b044740 1 -- 172.21.15.26:0/551317423 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559f7720caf0 msgr2=0x559f77214b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:25.773 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.767+0000 7f805b044740 1 --2- 172.21.15.26:0/551317423 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559f7720caf0 0x559f77214b00 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f8040009dc0 tx=0x7f8040005850 comp rx=0 tx=0).stop 2024-09-11T23:21:25.773 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.767+0000 7f805b044740 1 -- 172.21.15.26:0/551317423 shutdown_connections 2024-09-11T23:21:25.773 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.767+0000 7f805b044740 1 --2- 172.21.15.26:0/551317423 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559f7720d410 0x559f77215040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:25.773 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.767+0000 7f805b044740 1 --2- 172.21.15.26:0/551317423 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x559f77227a80 0x559f77229ea0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:25.773 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.767+0000 7f805b044740 1 --2- 172.21.15.26:0/551317423 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559f77217cb0 0x559f77215580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:25.774 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.767+0000 7f805b044740 1 --2- 172.21.15.26:0/551317423 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559f7720caf0 0x559f77214b00 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:25.774 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.767+0000 7f80397fa640 0 client.25861 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:21:25.774 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.767+0000 7f805b044740 1 -- 172.21.15.26:0/551317423 >> 172.21.15.26:0/551317423 conn(0x559f77209060 msgr2=0x559f7720ed70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:21:25.774 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.767+0000 7f805b044740 1 -- 172.21.15.26:0/551317423 shutdown_connections 2024-09-11T23:21:25.774 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.767+0000 7f805b044740 1 -- 172.21.15.26:0/551317423 wait complete. 2024-09-11T23:21:25.774 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.767+0000 7f805b044740 20 client.25861 trim_cache size 0 max 16384 2024-09-11T23:21:25.785 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.782+0000 7f805b044740 1 Processor -- start 2024-09-11T23:21:25.785 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.782+0000 7f805b044740 1 -- start start 2024-09-11T23:21:25.786 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.782+0000 7f805b044740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559f7720caf0 0x559f77209440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:25.786 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.782+0000 7f805b044740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559f7720d410 0x559f772153b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:25.786 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.782+0000 7f805b044740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559f77217cb0 0x559f7722f750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:25.786 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.782+0000 7f805b044740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f80501af990 con 0x559f7720caf0 2024-09-11T23:21:25.786 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.782+0000 7f805b044740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f80501b1650 con 0x559f7720d410 2024-09-11T23:21:25.786 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.782+0000 7f805b044740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f80501b0060 con 0x559f77217cb0 2024-09-11T23:21:25.786 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.783+0000 7f805576a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559f7720caf0 0x559f77209440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:25.786 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.783+0000 7f805576a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559f7720caf0 0x559f77209440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43556/0 (socket says 172.21.15.26:43556) 2024-09-11T23:21:25.786 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.783+0000 7f805576a640 1 -- 172.21.15.26:0/3181705774 learned_addr learned my addr 172.21.15.26:0/3181705774 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:21:25.786 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.783+0000 7f805576a640 1 --2- 172.21.15.26:0/3181705774 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559f77217cb0 0x559f7722f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:25.786 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.783+0000 7f8055f6b640 1 --2- 172.21.15.26:0/3181705774 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559f7720d410 0x559f772153b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:25.787 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.783+0000 7f805576a640 1 -- 172.21.15.26:0/3181705774 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559f77217cb0 msgr2=0x559f7722f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:21:25.787 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.783+0000 7f805576a640 1 --2- 172.21.15.26:0/3181705774 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559f77217cb0 0x559f7722f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:25.787 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.783+0000 7f805576a640 1 -- 172.21.15.26:0/3181705774 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559f7720d410 msgr2=0x559f772153b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:21:25.787 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.783+0000 7f805576a640 1 --2- 172.21.15.26:0/3181705774 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559f7720d410 0x559f772153b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:25.787 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.783+0000 7f805576a640 1 -- 172.21.15.26:0/3181705774 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f77213a90 con 0x559f7720caf0 2024-09-11T23:21:25.787 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.783+0000 7f8055f6b640 1 --2- 172.21.15.26:0/3181705774 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559f7720d410 0x559f772153b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:21:25.787 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.783+0000 7f805576a640 1 --2- 172.21.15.26:0/3181705774 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559f7720caf0 0x559f77209440 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f804000fd30 tx=0x7f8040001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc4324f5c7566601 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:21:25.787 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.783+0000 7f803b7fe640 1 -- 172.21.15.26:0/3181705774 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8040009b50 con 0x559f7720caf0 2024-09-11T23:21:25.787 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.783+0000 7f803b7fe640 1 -- 172.21.15.26:0/3181705774 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f804000fdd0 con 0x559f7720caf0 2024-09-11T23:21:25.787 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.783+0000 7f803b7fe640 1 -- 172.21.15.26:0/3181705774 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8040010e40 con 0x559f7720caf0 2024-09-11T23:21:25.787 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.784+0000 7f805b044740 1 -- 172.21.15.26:0/3181705774 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559f7720caf0 msgr2=0x559f77209440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:25.787 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.784+0000 7f805b044740 1 --2- 172.21.15.26:0/3181705774 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559f7720caf0 0x559f77209440 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f804000fd30 tx=0x7f8040001e50 comp rx=0 tx=0).stop 2024-09-11T23:21:25.788 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.784+0000 7f805b044740 1 -- 172.21.15.26:0/3181705774 shutdown_connections 2024-09-11T23:21:25.788 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.784+0000 7f805b044740 1 --2- 172.21.15.26:0/3181705774 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559f7720d410 0x559f772153b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:25.788 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.784+0000 7f805b044740 1 --2- 172.21.15.26:0/3181705774 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559f77217cb0 0x559f7722f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:25.788 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.784+0000 7f805b044740 1 --2- 172.21.15.26:0/3181705774 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559f7720caf0 0x559f77209440 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:25.788 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.784+0000 7f805b044740 1 -- 172.21.15.26:0/3181705774 >> 172.21.15.26:0/3181705774 conn(0x559f77209060 msgr2=0x559f77216b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:21:25.788 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.784+0000 7f805b044740 1 -- 172.21.15.26:0/3181705774 shutdown_connections 2024-09-11T23:21:25.788 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.784+0000 7f805b044740 1 -- 172.21.15.26:0/3181705774 wait complete. 2024-09-11T23:21:25.788 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.784+0000 7f805b044740 1 -- start start 2024-09-11T23:21:25.788 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.785+0000 7f805b044740 1 Processor -- start 2024-09-11T23:21:25.791 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.785+0000 7f805b044740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559f7720caf0 0x559f772112f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:25.791 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.785+0000 7f805b044740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559f7720d410 0x559f77211830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:25.791 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.785+0000 7f805b044740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559f77217cb0 0x559f7721ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:25.791 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.785+0000 7f805b044740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f80501b0ec0 con 0x559f77217cb0 2024-09-11T23:21:25.791 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.785+0000 7f805b044740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f80501b0730 con 0x559f7720d410 2024-09-11T23:21:25.791 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.785+0000 7f805b044740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f80501ad970 con 0x559f7720caf0 2024-09-11T23:21:25.791 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.785+0000 7f805b044740 10 client.0 _ll_register_callbacks cb 0x7f804c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:21:25.791 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.785+0000 7f805576a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559f77217cb0 0x559f7721ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:25.791 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.785+0000 7f8055f6b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559f7720d410 0x559f77211830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:25.791 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.785+0000 7f805576a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559f77217cb0 0x559f7721ecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43566/0 (socket says 172.21.15.26:43566) 2024-09-11T23:21:25.791 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.785+0000 7f805576a640 1 -- 172.21.15.26:0/2488907229 learned_addr learned my addr 172.21.15.26:0/2488907229 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:21:25.792 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.785+0000 7f8055f6b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559f7720d410 0x559f77211830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:38152/0 (socket says 172.21.15.26:38152) 2024-09-11T23:21:25.792 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.785+0000 7f805576a640 1 --2- 172.21.15.26:0/2488907229 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559f7720caf0 0x559f772112f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:25.792 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.785+0000 7f805576a640 1 -- 172.21.15.26:0/2488907229 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559f7720caf0 msgr2=0x559f772112f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:21:25.792 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.785+0000 7f805576a640 1 --2- 172.21.15.26:0/2488907229 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559f7720caf0 0x559f772112f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:25.792 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.785+0000 7f805576a640 1 -- 172.21.15.26:0/2488907229 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559f7720d410 msgr2=0x559f77211830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:21:25.792 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.785+0000 7f805576a640 1 --2- 172.21.15.26:0/2488907229 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559f7720d410 0x559f77211830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:25.792 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.785+0000 7f805576a640 1 -- 172.21.15.26:0/2488907229 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f7721f200 con 0x559f77217cb0 2024-09-11T23:21:25.792 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.786+0000 7f805576a640 1 --2- 172.21.15.26:0/2488907229 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559f77217cb0 0x559f7721ecc0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f8040013040 tx=0x7f8040009e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7ae85bba6e54a12 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:21:25.792 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.786+0000 7f80397fa640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:21:25.792 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.786+0000 7f80397fa640 1 -- 172.21.15.26:0/2488907229 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8040010880 con 0x559f77217cb0 2024-09-11T23:21:25.792 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.786+0000 7f80397fa640 1 -- 172.21.15.26:0/2488907229 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f804003f070 con 0x559f77217cb0 2024-09-11T23:21:25.792 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.786+0000 7f805b044740 1 -- 172.21.15.26:0/2488907229 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f804000fdd0 con 0x559f77217cb0 2024-09-11T23:21:25.792 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.786+0000 7f80397fa640 1 -- 172.21.15.26:0/2488907229 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f8040010bc0 con 0x559f77217cb0 2024-09-11T23:21:25.793 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.786+0000 7f805b044740 10 client.16233 fetch_fsmap learned FSMap version 33 2024-09-11T23:21:25.793 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.786+0000 7f805b044740 1 -- 172.21.15.26:0/2488907229 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x559f77220470 con 0x559f77217cb0 2024-09-11T23:21:25.793 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.786+0000 7f80397fa640 1 -- 172.21.15.26:0/2488907229 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f804000fdd0 con 0x559f77217cb0 2024-09-11T23:21:25.793 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.786+0000 7f80397fa640 1 -- 172.21.15.26:0/2488907229 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 33) ==== 8394+0+0 (secure 0 0 0) 0x7f804001e040 con 0x559f77217cb0 2024-09-11T23:21:25.793 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.786+0000 7f805b044740 10 client.16233 fetch_fsmap finished waiting for FSMap version 33 2024-09-11T23:21:25.793 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.786+0000 7f805b044740 10 client.16233 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:21:25.793 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.786+0000 7f805b044740 20 client.16233 populate_metadata read hostname 'smithi026' 2024-09-11T23:21:25.793 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.786+0000 7f805b044740 1 --2- 172.21.15.26:0/2488907229 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x559f77292720 0x559f77294b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:25.793 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.786+0000 7f805b044740 4 client.16233 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T23:21:25.793 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.786+0000 7f805b044740 1 -- 172.21.15.26:0/2488907229 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559f77178d70 con 0x559f77292720 2024-09-11T23:21:25.793 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.787+0000 7f8055f6b640 1 --2- 172.21.15.26:0/2488907229 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x559f77292720 0x559f77294b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:25.793 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.787+0000 7f80397fa640 10 client.16233 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:21:25.793 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.787+0000 7f8055f6b640 1 --2- 172.21.15.26:0/2488907229 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x559f77292720 0x559f77294b10 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3da78b300ea8d010 server_cookie=60ef4922f7997725 in_seq=0 out_seq=0 2024-09-11T23:21:25.802 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:21:25 smithi076 ceph-mon[35296]: pgmap v94: 97 pgs: 97 active+clean; 399 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 4.0 MiB/s wr, 622 op/s 2024-09-11T23:21:25.869 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.867+0000 7f80397fa640 1 -- 172.21.15.26:0/2488907229 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559f77178d70 con 0x559f77292720 2024-09-11T23:21:25.870 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.867+0000 7f80397fa640 10 client.16233 handle_command_reply: tid=0 2024-09-11T23:21:25.870 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.867+0000 7f805b044740 1 client.16233 shutdown 2024-09-11T23:21:25.870 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.867+0000 7f805b044740 1 -- 172.21.15.26:0/2488907229 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559f77217cb0 msgr2=0x559f7721ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:25.870 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.867+0000 7f805b044740 1 --2- 172.21.15.26:0/2488907229 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559f77217cb0 0x559f7721ecc0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f8040013040 tx=0x7f8040009e50 comp rx=0 tx=0).stop 2024-09-11T23:21:25.870 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.868+0000 7f805b044740 1 -- 172.21.15.26:0/2488907229 shutdown_connections 2024-09-11T23:21:25.870 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.868+0000 7f805b044740 1 --2- 172.21.15.26:0/2488907229 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559f7720d410 0x559f77211830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:25.870 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.868+0000 7f805b044740 1 --2- 172.21.15.26:0/2488907229 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x559f77292720 0x559f77294b10 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:25.870 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.868+0000 7f805b044740 1 --2- 172.21.15.26:0/2488907229 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559f77217cb0 0x559f7721ecc0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:25.870 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.868+0000 7f80397fa640 0 client.16233 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:21:25.870 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.868+0000 7f805b044740 1 --2- 172.21.15.26:0/2488907229 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559f7720caf0 0x559f772112f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:25.870 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.868+0000 7f805b044740 1 -- 172.21.15.26:0/2488907229 >> 172.21.15.26:0/2488907229 conn(0x559f77209060 msgr2=0x559f772079a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:21:25.871 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.868+0000 7f805b044740 1 -- 172.21.15.26:0/2488907229 shutdown_connections 2024-09-11T23:21:25.871 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.868+0000 7f805b044740 1 -- 172.21.15.26:0/2488907229 wait complete. 2024-09-11T23:21:25.871 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.868+0000 7f805b044740 20 client.16233 trim_cache size 0 max 16384 2024-09-11T23:21:25.871 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-09-11T23:21:25.871 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "message": "", 2024-09-11T23:21:25.871 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-09-11T23:21:25.871 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-09-11T23:21:25.872 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.868+0000 7f80581cf640 1 -- 172.21.15.26:0/3574868048 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f803c0a2da0 msgr2=0x7f803c0a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:25.872 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.868+0000 7f80581cf640 1 --2- 172.21.15.26:0/3574868048 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f803c0a2da0 0x7f803c0a5260 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f80480ccf90 tx=0x7f8048091140 comp rx=0 tx=0).stop 2024-09-11T23:21:25.872 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.868+0000 7f80581cf640 1 -- 172.21.15.26:0/3574868048 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f80501069c0 msgr2=0x7f805019cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:25.872 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.868+0000 7f80581cf640 1 --2- 172.21.15.26:0/3574868048 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f80501069c0 0x7f805019cc30 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f804800da10 tx=0x7f804800ded0 comp rx=0 tx=0).stop 2024-09-11T23:21:25.872 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.868+0000 7f80581cf640 1 -- 172.21.15.26:0/3574868048 shutdown_connections 2024-09-11T23:21:25.872 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.868+0000 7f80581cf640 1 --2- 172.21.15.26:0/3574868048 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f803c0a2da0 0x7f803c0a5260 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:25.872 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.868+0000 7f80581cf640 1 --2- 172.21.15.26:0/3574868048 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f80501069c0 0x7f805019cc30 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:25.873 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.868+0000 7f80581cf640 1 --2- 172.21.15.26:0/3574868048 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8050101350 0x7f80501988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:25.873 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.868+0000 7f80581cf640 1 --2- 172.21.15.26:0/3574868048 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f80501009b0 0x7f8050198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:25.873 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.868+0000 7f80581cf640 1 -- 172.21.15.26:0/3574868048 >> 172.21.15.26:0/3574868048 conn(0x7f8050077990 msgr2=0x7f8050069c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:21:25.873 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.869+0000 7f80581cf640 1 -- 172.21.15.26:0/3574868048 shutdown_connections 2024-09-11T23:21:25.873 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:25.869+0000 7f80581cf640 1 -- 172.21.15.26:0/3574868048 wait complete. 2024-09-11T23:21:27.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:21:27 smithi136 ceph-mon[35121]: pgmap v95: 97 pgs: 97 active+clean; 399 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.2 MiB/s wr, 292 op/s 2024-09-11T23:21:27.750 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:21:27 smithi026 ceph-mon[31172]: pgmap v95: 97 pgs: 97 active+clean; 399 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.2 MiB/s wr, 292 op/s 2024-09-11T23:21:27.802 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:21:27 smithi076 ceph-mon[35296]: pgmap v95: 97 pgs: 97 active+clean; 399 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.2 MiB/s wr, 292 op/s 2024-09-11T23:21:28.587 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:21:28.864 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:21:29.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:29.343+0000 7f00f5e50640 1 -- 172.21.15.26:0/3619912361 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f00f0103660 msgr2=0x7f00f010bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:29.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:29.343+0000 7f00f5e50640 1 --2- 172.21.15.26:0/3619912361 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f00f0103660 0x7f00f010bb50 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f00e4009900 tx=0x7f00e402a480 comp rx=0 tx=0).stop 2024-09-11T23:21:29.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:29.344+0000 7f00f5e50640 1 -- 172.21.15.26:0/3619912361 shutdown_connections 2024-09-11T23:21:29.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:29.344+0000 7f00f5e50640 1 --2- 172.21.15.26:0/3619912361 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f00f010c170 0x7f00f010e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:29.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:29.344+0000 7f00f5e50640 1 --2- 172.21.15.26:0/3619912361 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f00f0103660 0x7f00f010bb50 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:29.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:29.344+0000 7f00f5e50640 1 --2- 172.21.15.26:0/3619912361 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f00f0102d40 0x7f00f0103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:29.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:29.344+0000 7f00f5e50640 1 -- 172.21.15.26:0/3619912361 >> 172.21.15.26:0/3619912361 conn(0x7f00f00fc7a0 msgr2=0x7f00f00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:21:29.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:29.344+0000 7f00f5e50640 1 -- 172.21.15.26:0/3619912361 shutdown_connections 2024-09-11T23:21:29.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:29.344+0000 7f00f5e50640 1 -- 172.21.15.26:0/3619912361 wait complete. 2024-09-11T23:21:29.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:29.344+0000 7f00f5e50640 1 Processor -- start 2024-09-11T23:21:29.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:29.344+0000 7f00f5e50640 1 -- start start 2024-09-11T23:21:29.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:29.345+0000 7f00f5e50640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f00f0102d40 0x7f00f0198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:29.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:29.345+0000 7f00f5e50640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f00f0103660 0x7f00f0198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:29.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:29.345+0000 7f00f5e50640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f00f010c170 0x7f00f019cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:29.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:29.345+0000 7f00f5e50640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f00f0111fc0 con 0x7f00f010c170 2024-09-11T23:21:29.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:29.345+0000 7f00f5e50640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f00f0111e40 con 0x7f00f0102d40 2024-09-11T23:21:29.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:29.345+0000 7f00f5e50640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f00f0112140 con 0x7f00f0103660 2024-09-11T23:21:29.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:29.345+0000 7f00effff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f00f010c170 0x7f00f019cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:29.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:29.345+0000 7f00effff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f00f010c170 0x7f00f019cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43582/0 (socket says 172.21.15.26:43582) 2024-09-11T23:21:29.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:29.345+0000 7f00effff640 1 -- 172.21.15.26:0/2653921669 learned_addr learned my addr 172.21.15.26:0/2653921669 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:21:29.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:29.345+0000 7f00eeffd640 1 --2- 172.21.15.26:0/2653921669 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f00f0103660 0x7f00f0198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:29.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:29.345+0000 7f00ef7fe640 1 --2- 172.21.15.26:0/2653921669 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f00f0102d40 0x7f00f0198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:29.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:29.345+0000 7f00effff640 1 -- 172.21.15.26:0/2653921669 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f00f0103660 msgr2=0x7f00f0198990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:29.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:29.345+0000 7f00effff640 1 --2- 172.21.15.26:0/2653921669 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f00f0103660 0x7f00f0198990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:29.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:29.345+0000 7f00effff640 1 -- 172.21.15.26:0/2653921669 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f00f0102d40 msgr2=0x7f00f0198450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:29.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:29.345+0000 7f00effff640 1 --2- 172.21.15.26:0/2653921669 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f00f0102d40 0x7f00f0198450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:29.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:29.345+0000 7f00effff640 1 -- 172.21.15.26:0/2653921669 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00f019d400 con 0x7f00f010c170 2024-09-11T23:21:29.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:29.345+0000 7f00effff640 1 --2- 172.21.15.26:0/2653921669 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f00f010c170 0x7f00f019cd20 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f00e800eb10 tx=0x7f00e800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:21:29.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:29.345+0000 7f00ecff9640 1 -- 172.21.15.26:0/2653921669 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00e8004b20 con 0x7f00f010c170 2024-09-11T23:21:29.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:29.345+0000 7f00ecff9640 1 -- 172.21.15.26:0/2653921669 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00e8004cc0 con 0x7f00f010c170 2024-09-11T23:21:29.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:29.345+0000 7f00f5e50640 1 -- 172.21.15.26:0/2653921669 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00f019d6f0 con 0x7f00f010c170 2024-09-11T23:21:29.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:29.345+0000 7f00ecff9640 1 -- 172.21.15.26:0/2653921669 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00e8010760 con 0x7f00f010c170 2024-09-11T23:21:29.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:29.346+0000 7f00f5e50640 1 -- 172.21.15.26:0/2653921669 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00f01a4fd0 con 0x7f00f010c170 2024-09-11T23:21:29.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:29.347+0000 7f00f5e50640 1 -- 172.21.15.26:0/2653921669 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00f0103120 con 0x7f00f010c170 2024-09-11T23:21:29.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:29.348+0000 7f00ecff9640 1 -- 172.21.15.26:0/2653921669 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f00e8005020 con 0x7f00f010c170 2024-09-11T23:21:29.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:29.348+0000 7f00ecff9640 1 --2- 172.21.15.26:0/2653921669 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f00d80a2cf0 0x7f00d80a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:29.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:29.348+0000 7f00ef7fe640 1 --2- 172.21.15.26:0/2653921669 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f00d80a2cf0 0x7f00d80a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:29.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:29.348+0000 7f00ecff9640 1 -- 172.21.15.26:0/2653921669 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f00e80cfa70 con 0x7f00f010c170 2024-09-11T23:21:29.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:29.349+0000 7f00ef7fe640 1 --2- 172.21.15.26:0/2653921669 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f00d80a2cf0 0x7f00d80a51b0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f00e0005d00 tx=0x7f00e000b410 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:21:29.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:29.352+0000 7f00ecff9640 1 -- 172.21.15.26:0/2653921669 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f00e809c6a0 con 0x7f00f010c170 2024-09-11T23:21:29.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:29.564+0000 7f00f5e50640 1 -- 172.21.15.26:0/2653921669 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f00f002d7c0 con 0x7f00f010c170 2024-09-11T23:21:29.566 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:21:29 smithi026 ceph-mon[31172]: pgmap v96: 97 pgs: 97 active+clean; 405 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 2.8 MiB/s wr, 383 op/s 2024-09-11T23:21:29.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:29.565+0000 7f00ecff9640 1 -- 172.21.15.26:0/2653921669 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+11986 (secure 0 0 0) 0x7f00e809bfa0 con 0x7f00f010c170 2024-09-11T23:21:29.566 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-11T23:21:29.567 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":33,"btime":"2024-09-11T23:20:14:305449+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14499,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/367659515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":367659515},{"type":"v1","addr":"172.21.15.26:6835","nonce":367659515}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14511,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/2041065299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":2041065299},{"type":"v1","addr":"172.21.15.26:6837","nonce":2041065299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":33,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:16:26.544203+0000","modified":"2024-09-11T23:20:13.958301+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24485,"mds_1":24463,"mds_2":24458,"mds_3":24448,"mds_4":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14550":{"gid":14550,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6839/2233068305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":2233068305},{"type":"v1","addr":"172.21.15.26:6839","nonce":2233068305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14562":{"gid":14562,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6841/2984982336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":2984982336},{"type":"v1","addr":"172.21.15.26:6841","nonce":2984982336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6833/2019841455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":2019841455},{"type":"v1","addr":"172.21.15.136:6833","nonce":2019841455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.76:6833/3438147050","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3438147050},{"type":"v1","addr":"172.21.15.76:6833","nonce":3438147050}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6835/3617665975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3617665975},{"type":"v1","addr":"172.21.15.136:6835","nonce":3617665975}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":12,"addr":"172.21.15.76:6835/393308499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":393308499},{"type":"v1","addr":"172.21.15.76:6835","nonce":393308499}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":10,"addr":"172.21.15.136:6837/1633082387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":1633082387},{"type":"v1","addr":"172.21.15.136:6837","nonce":1633082387}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.76:6837/2078978748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":2078978748},{"type":"v1","addr":"172.21.15.76:6837","nonce":2078978748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":63,"addr":"172.21.15.136:6839/3211398630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":3211398630},{"type":"v1","addr":"172.21.15.136:6839","nonce":3211398630}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.76:6839/2077637090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6838","nonce":2077637090},{"type":"v1","addr":"172.21.15.76:6839","nonce":2077637090}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24485,"qdb_cluster":[24443,24448,24458,24463,24485]},"id":1}]} 2024-09-11T23:21:29.567 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 33 2024-09-11T23:21:29.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:29.567+0000 7f00f5e50640 1 -- 172.21.15.26:0/2653921669 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f00d80a2cf0 msgr2=0x7f00d80a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:29.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:29.567+0000 7f00f5e50640 1 --2- 172.21.15.26:0/2653921669 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f00d80a2cf0 0x7f00d80a51b0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f00e0005d00 tx=0x7f00e000b410 comp rx=0 tx=0).stop 2024-09-11T23:21:29.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:29.567+0000 7f00f5e50640 1 -- 172.21.15.26:0/2653921669 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f00f010c170 msgr2=0x7f00f019cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:29.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:29.567+0000 7f00f5e50640 1 --2- 172.21.15.26:0/2653921669 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f00f010c170 0x7f00f019cd20 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f00e800eb10 tx=0x7f00e800c2d0 comp rx=0 tx=0).stop 2024-09-11T23:21:29.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:29.567+0000 7f00f5e50640 1 -- 172.21.15.26:0/2653921669 shutdown_connections 2024-09-11T23:21:29.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:29.567+0000 7f00f5e50640 1 --2- 172.21.15.26:0/2653921669 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f00d80a2cf0 0x7f00d80a51b0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:29.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:29.567+0000 7f00f5e50640 1 --2- 172.21.15.26:0/2653921669 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f00f010c170 0x7f00f019cd20 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:29.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:29.567+0000 7f00f5e50640 1 --2- 172.21.15.26:0/2653921669 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f00f0103660 0x7f00f0198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:29.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:29.567+0000 7f00f5e50640 1 --2- 172.21.15.26:0/2653921669 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f00f0102d40 0x7f00f0198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:29.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:29.567+0000 7f00f5e50640 1 -- 172.21.15.26:0/2653921669 >> 172.21.15.26:0/2653921669 conn(0x7f00f00fc7a0 msgr2=0x7f00f00fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:21:29.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:29.567+0000 7f00f5e50640 1 -- 172.21.15.26:0/2653921669 shutdown_connections 2024-09-11T23:21:29.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:29.567+0000 7f00f5e50640 1 -- 172.21.15.26:0/2653921669 wait complete. 2024-09-11T23:21:29.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:21:29 smithi136 ceph-mon[35121]: pgmap v96: 97 pgs: 97 active+clean; 405 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 2.8 MiB/s wr, 383 op/s 2024-09-11T23:21:29.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:21:29 smithi076 ceph-mon[35296]: pgmap v96: 97 pgs: 97 active+clean; 405 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 2.8 MiB/s wr, 383 op/s 2024-09-11T23:21:30.340 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24485 quiesce db --set-id 6529ce3d --release --await' 2024-09-11T23:21:30.346 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24485 quiesce db --set-id 6529ce3d --release --await 2024-09-11T23:21:30.607 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:21:30.728 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:21:30 smithi026 ceph-mon[31172]: from='client.? 172.21.15.26:0/2653921669' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:21:30.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:21:30 smithi136 ceph-mon[35121]: from='client.? 172.21.15.26:0/2653921669' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:21:30.802 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:21:30 smithi076 ceph-mon[35296]: from='client.? 172.21.15.26:0/2653921669' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:21:31.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.081+0000 7f54b2b9a640 1 -- 172.21.15.26:0/2034391865 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f54ac100ca0 msgr2=0x7f54ac106840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:31.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.081+0000 7f54b2b9a640 1 --2- 172.21.15.26:0/2034391865 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f54ac100ca0 0x7f54ac106840 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f5498009930 tx=0x7f549802a420 comp rx=0 tx=0).stop 2024-09-11T23:21:31.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.081+0000 7f54b2b9a640 1 -- 172.21.15.26:0/2034391865 shutdown_connections 2024-09-11T23:21:31.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.081+0000 7f54b2b9a640 1 --2- 172.21.15.26:0/2034391865 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f54ac106f50 0x7f54ac10eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:31.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.081+0000 7f54b2b9a640 1 --2- 172.21.15.26:0/2034391865 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f54ac100ca0 0x7f54ac106840 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:31.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.081+0000 7f54b2b9a640 1 --2- 172.21.15.26:0/2034391865 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f54ac1002f0 0x7f54ac1006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:31.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.081+0000 7f54b2b9a640 1 -- 172.21.15.26:0/2034391865 >> 172.21.15.26:0/2034391865 conn(0x7f54ac077210 msgr2=0x7f54ac0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:21:31.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.081+0000 7f54b2b9a640 1 -- 172.21.15.26:0/2034391865 shutdown_connections 2024-09-11T23:21:31.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.081+0000 7f54b2b9a640 1 -- 172.21.15.26:0/2034391865 wait complete. 2024-09-11T23:21:31.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.082+0000 7f54b2b9a640 1 Processor -- start 2024-09-11T23:21:31.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.082+0000 7f54b2b9a640 1 -- start start 2024-09-11T23:21:31.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.082+0000 7f54b2b9a640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f54ac1002f0 0x7f54ac198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:31.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.082+0000 7f54b2b9a640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f54ac100ca0 0x7f54ac198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:31.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.082+0000 7f54b2b9a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f54ac106f50 0x7f54ac19cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:31.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.082+0000 7f54b2b9a640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f54ac111ff0 con 0x7f54ac106f50 2024-09-11T23:21:31.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.082+0000 7f54b2b9a640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f54ac111e70 con 0x7f54ac1002f0 2024-09-11T23:21:31.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.082+0000 7f54b2b9a640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f54ac112170 con 0x7f54ac100ca0 2024-09-11T23:21:31.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.082+0000 7f54b1137640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f54ac106f50 0x7f54ac19cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:31.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.082+0000 7f54b1137640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f54ac106f50 0x7f54ac19cdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52020/0 (socket says 172.21.15.26:52020) 2024-09-11T23:21:31.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.082+0000 7f54b1137640 1 -- 172.21.15.26:0/2293998557 learned_addr learned my addr 172.21.15.26:0/2293998557 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:21:31.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.083+0000 7f54b0936640 1 --2- 172.21.15.26:0/2293998557 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f54ac1002f0 0x7f54ac198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:31.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.083+0000 7f54abfff640 1 --2- 172.21.15.26:0/2293998557 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f54ac100ca0 0x7f54ac198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:31.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.083+0000 7f54b1137640 1 -- 172.21.15.26:0/2293998557 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f54ac100ca0 msgr2=0x7f54ac198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:31.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.083+0000 7f54b1137640 1 --2- 172.21.15.26:0/2293998557 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f54ac100ca0 0x7f54ac198a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:31.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.083+0000 7f54b1137640 1 -- 172.21.15.26:0/2293998557 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f54ac1002f0 msgr2=0x7f54ac198500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:31.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.083+0000 7f54b1137640 1 --2- 172.21.15.26:0/2293998557 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f54ac1002f0 0x7f54ac198500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:31.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.083+0000 7f54b1137640 1 -- 172.21.15.26:0/2293998557 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54ac19d4b0 con 0x7f54ac106f50 2024-09-11T23:21:31.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.083+0000 7f54abfff640 1 --2- 172.21.15.26:0/2293998557 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f54ac100ca0 0x7f54ac198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:21:31.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.083+0000 7f54b1137640 1 --2- 172.21.15.26:0/2293998557 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f54ac106f50 0x7f54ac19cdd0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f54a400cad0 tx=0x7f54a400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:21:31.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.083+0000 7f54a9ffb640 1 -- 172.21.15.26:0/2293998557 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54a400ae20 con 0x7f54ac106f50 2024-09-11T23:21:31.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.083+0000 7f54a9ffb640 1 -- 172.21.15.26:0/2293998557 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54a4005020 con 0x7f54ac106f50 2024-09-11T23:21:31.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.083+0000 7f54a9ffb640 1 -- 172.21.15.26:0/2293998557 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54a400f6f0 con 0x7f54ac106f50 2024-09-11T23:21:31.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.083+0000 7f54b2b9a640 1 -- 172.21.15.26:0/2293998557 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54ac19d7a0 con 0x7f54ac106f50 2024-09-11T23:21:31.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.083+0000 7f54b2b9a640 1 -- 172.21.15.26:0/2293998557 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54ac1011a0 con 0x7f54ac106f50 2024-09-11T23:21:31.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.086+0000 7f54a9ffb640 1 -- 172.21.15.26:0/2293998557 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f54a40051d0 con 0x7f54ac106f50 2024-09-11T23:21:31.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.086+0000 7f54b4f73740 1 Processor -- start 2024-09-11T23:21:31.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.086+0000 7f54b4f73740 1 -- start start 2024-09-11T23:21:31.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.086+0000 7f54b4f73740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bc09842060 0x55bc09842440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:31.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.086+0000 7f54b4f73740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bc09842980 0x55bc0984cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:31.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.086+0000 7f54b4f73740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bc0984d240 0x55bc0984f630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:31.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.086+0000 7f54b4f73740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55bc097a7620 con 0x55bc09842060 2024-09-11T23:21:31.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.086+0000 7f54b4f73740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55bc0974ee50 con 0x55bc09842980 2024-09-11T23:21:31.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.086+0000 7f54b4f73740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55bc097a35e0 con 0x55bc0984d240 2024-09-11T23:21:31.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.086+0000 7f54a9ffb640 1 --2- 172.21.15.26:0/2293998557 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f549c0a7100 0x7f549c0a95c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:31.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.086+0000 7f54abfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bc09842060 0x55bc09842440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:31.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.086+0000 7f54abfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bc09842060 0x55bc09842440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52036/0 (socket says 172.21.15.26:52036) 2024-09-11T23:21:31.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.086+0000 7f54abfff640 1 -- 172.21.15.26:0/3385090778 learned_addr learned my addr 172.21.15.26:0/3385090778 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:21:31.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.086+0000 7f54b1137640 1 --2- 172.21.15.26:0/2293998557 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f549c0a7100 0x7f549c0a95c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:31.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.087+0000 7f54b0936640 1 --2- 172.21.15.26:0/3385090778 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bc09842980 0x55bc0984cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:31.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.087+0000 7f54abfff640 1 --2- 172.21.15.26:0/3385090778 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bc0984d240 0x55bc0984f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:31.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.087+0000 7f54a9ffb640 1 -- 172.21.15.26:0/2293998557 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f54a40cf830 con 0x7f54ac106f50 2024-09-11T23:21:31.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.087+0000 7f54abfff640 1 -- 172.21.15.26:0/3385090778 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bc0984d240 msgr2=0x55bc0984f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:21:31.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.087+0000 7f54abfff640 1 --2- 172.21.15.26:0/3385090778 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bc0984d240 0x55bc0984f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:31.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.087+0000 7f54abfff640 1 -- 172.21.15.26:0/3385090778 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bc09842980 msgr2=0x55bc0984cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:21:31.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.087+0000 7f54abfff640 1 --2- 172.21.15.26:0/3385090778 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bc09842980 0x55bc0984cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:31.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.087+0000 7f54abfff640 1 -- 172.21.15.26:0/3385090778 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc0973fd40 con 0x55bc09842060 2024-09-11T23:21:31.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.087+0000 7f54abfff640 1 --2- 172.21.15.26:0/3385090778 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bc09842060 0x55bc09842440 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x55bc0983f970 tx=0x7f5498031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3881d4994552be22 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:21:31.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.087+0000 7f54b1137640 1 --2- 172.21.15.26:0/2293998557 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f549c0a7100 0x7f549c0a95c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f54a40986d0 tx=0x7f54a406ed70 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:21:31.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.087+0000 7f5496ffd640 1 -- 172.21.15.26:0/3385090778 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5498035070 con 0x55bc09842060 2024-09-11T23:21:31.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.087+0000 7f5496ffd640 1 -- 172.21.15.26:0/3385090778 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54980046c0 con 0x55bc09842060 2024-09-11T23:21:31.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.087+0000 7f5496ffd640 1 -- 172.21.15.26:0/3385090778 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54980039e0 con 0x55bc09842060 2024-09-11T23:21:31.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.088+0000 7f54b4f73740 1 -- 172.21.15.26:0/3385090778 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bc09842060 msgr2=0x55bc09842440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:31.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.088+0000 7f54b4f73740 1 --2- 172.21.15.26:0/3385090778 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bc09842060 0x55bc09842440 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x55bc0983f970 tx=0x7f5498031400 comp rx=0 tx=0).stop 2024-09-11T23:21:31.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.088+0000 7f54b4f73740 1 -- 172.21.15.26:0/3385090778 shutdown_connections 2024-09-11T23:21:31.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.088+0000 7f54b4f73740 1 --2- 172.21.15.26:0/3385090778 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bc09842980 0x55bc0984cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:31.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.088+0000 7f54b4f73740 1 --2- 172.21.15.26:0/3385090778 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bc0984d240 0x55bc0984f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:31.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.088+0000 7f54b4f73740 1 --2- 172.21.15.26:0/3385090778 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bc09842060 0x55bc09842440 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:31.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.088+0000 7f54b4f73740 1 -- 172.21.15.26:0/3385090778 >> 172.21.15.26:0/3385090778 conn(0x55bc0983e5f0 msgr2=0x55bc0983e9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:21:31.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.088+0000 7f54b4f73740 1 -- 172.21.15.26:0/3385090778 shutdown_connections 2024-09-11T23:21:31.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.088+0000 7f54b4f73740 1 -- 172.21.15.26:0/3385090778 wait complete. 2024-09-11T23:21:31.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.088+0000 7f54b4f73740 1 -- start start 2024-09-11T23:21:31.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.088+0000 7f54b4f73740 1 Processor -- start 2024-09-11T23:21:31.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.089+0000 7f54b4f73740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bc09842060 0x55bc0984a210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:31.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.089+0000 7f54b4f73740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bc09842980 0x55bc0984a750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:31.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.089+0000 7f54b4f73740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bc0984d240 0x55bc0984ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:31.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.089+0000 7f54b4f73740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55bc0984fdb0 con 0x55bc09842060 2024-09-11T23:21:31.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.089+0000 7f54b4f73740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55bc0972ff10 con 0x55bc0984d240 2024-09-11T23:21:31.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.089+0000 7f54b4f73740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55bc0984ff30 con 0x55bc09842980 2024-09-11T23:21:31.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.089+0000 7f54b4f73740 10 client.0 _ll_register_callbacks cb 0x55bc09753db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:21:31.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.089+0000 7f54abfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bc09842060 0x55bc0984a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:31.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.089+0000 7f54abfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bc09842060 0x55bc0984a210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52040/0 (socket says 172.21.15.26:52040) 2024-09-11T23:21:31.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.089+0000 7f54abfff640 1 -- 172.21.15.26:0/1510346142 learned_addr learned my addr 172.21.15.26:0/1510346142 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:21:31.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.089+0000 7f54abfff640 1 -- 172.21.15.26:0/1510346142 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bc09842980 msgr2=0x55bc0984a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:21:31.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.089+0000 7f54abfff640 1 --2- 172.21.15.26:0/1510346142 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bc09842980 0x55bc0984a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:31.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.089+0000 7f54abfff640 1 -- 172.21.15.26:0/1510346142 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bc0984d240 msgr2=0x55bc0984ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:21:31.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.089+0000 7f54abfff640 1 --2- 172.21.15.26:0/1510346142 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bc0984d240 0x55bc0984ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:31.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.089+0000 7f54abfff640 1 -- 172.21.15.26:0/1510346142 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc094b6580 con 0x55bc09842060 2024-09-11T23:21:31.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.089+0000 7f54abfff640 1 --2- 172.21.15.26:0/1510346142 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bc09842060 0x55bc0984a210 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f54980058b0 tx=0x7f54980058e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a27816dc6a6be36f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:21:31.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.089+0000 7f548bfff640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:21:31.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.090+0000 7f548bfff640 1 -- 172.21.15.26:0/1510346142 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5498035070 con 0x55bc09842060 2024-09-11T23:21:31.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.090+0000 7f548bfff640 1 -- 172.21.15.26:0/1510346142 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f54980054d0 con 0x55bc09842060 2024-09-11T23:21:31.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.090+0000 7f54b4f73740 1 -- 172.21.15.26:0/1510346142 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54980046c0 con 0x55bc09842060 2024-09-11T23:21:31.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.090+0000 7f548bfff640 1 -- 172.21.15.26:0/1510346142 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f5498003760 con 0x55bc09842060 2024-09-11T23:21:31.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.090+0000 7f54b4f73740 10 client.16251 fetch_fsmap learned FSMap version 33 2024-09-11T23:21:31.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.090+0000 7f54b4f73740 1 -- 172.21.15.26:0/1510346142 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55bc09580960 con 0x55bc09842060 2024-09-11T23:21:31.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.090+0000 7f548bfff640 1 -- 172.21.15.26:0/1510346142 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54980046c0 con 0x55bc09842060 2024-09-11T23:21:31.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.090+0000 7f548bfff640 1 -- 172.21.15.26:0/1510346142 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 33) ==== 8394+0+0 (secure 0 0 0) 0x7f54980319f0 con 0x55bc09842060 2024-09-11T23:21:31.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.090+0000 7f54b4f73740 10 client.16251 fetch_fsmap finished waiting for FSMap version 33 2024-09-11T23:21:31.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.090+0000 7f54b4f73740 10 client.16251 resolve_mds: validated gid 24485 aka daemon mds.b 2024-09-11T23:21:31.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.090+0000 7f54b4f73740 20 client.16251 populate_metadata read hostname 'smithi026' 2024-09-11T23:21:31.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.090+0000 7f54b4f73740 1 --2- 172.21.15.26:0/1510346142 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55bc0985ccb0 0x55bc0985f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:31.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.090+0000 7f54b4f73740 4 client.16251 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:21:31.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.090+0000 7f54b4f73740 1 -- 172.21.15.26:0/1510346142 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bc097b4b90 con 0x55bc0985ccb0 2024-09-11T23:21:31.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.091+0000 7f54b0936640 1 --2- 172.21.15.26:0/1510346142 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55bc0985ccb0 0x55bc0985f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:31.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.091+0000 7f54b0936640 1 --2- 172.21.15.26:0/1510346142 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55bc0985ccb0 0x55bc0985f0a0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e7ef98f4268ad293 server_cookie=8e865d33f48aeac5 in_seq=0 out_seq=0 2024-09-11T23:21:31.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.091+0000 7f548bfff640 10 client.16251 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:21:31.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.092+0000 7f548bfff640 1 -- 172.21.15.26:0/1510346142 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bc097b4b90 con 0x55bc0985ccb0 2024-09-11T23:21:31.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.092+0000 7f548bfff640 10 client.16251 handle_command_reply: tid=0 2024-09-11T23:21:31.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.092+0000 7f54b4f73740 1 client.16251 shutdown 2024-09-11T23:21:31.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.092+0000 7f54b4f73740 1 -- 172.21.15.26:0/1510346142 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bc09842060 msgr2=0x55bc0984a210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:31.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.092+0000 7f54b4f73740 1 --2- 172.21.15.26:0/1510346142 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bc09842060 0x55bc0984a210 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f54980058b0 tx=0x7f54980058e0 comp rx=0 tx=0).stop 2024-09-11T23:21:31.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.092+0000 7f54b4f73740 1 -- 172.21.15.26:0/1510346142 shutdown_connections 2024-09-11T23:21:31.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.092+0000 7f54b4f73740 1 --2- 172.21.15.26:0/1510346142 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bc0984d240 0x55bc0984ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:31.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.092+0000 7f54b4f73740 1 --2- 172.21.15.26:0/1510346142 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55bc0985ccb0 0x55bc0985f0a0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:31.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.092+0000 7f54b4f73740 1 --2- 172.21.15.26:0/1510346142 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bc09842980 0x55bc0984a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:31.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.092+0000 7f54b4f73740 1 --2- 172.21.15.26:0/1510346142 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bc09842060 0x55bc0984a210 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:31.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.092+0000 7f54b4f73740 1 -- 172.21.15.26:0/1510346142 >> 172.21.15.26:0/1510346142 conn(0x55bc0983e5f0 msgr2=0x55bc09844320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:21:31.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.092+0000 7f548bfff640 0 client.16251 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:21:31.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.092+0000 7f54b4f73740 1 -- 172.21.15.26:0/1510346142 shutdown_connections 2024-09-11T23:21:31.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.092+0000 7f54b4f73740 1 -- 172.21.15.26:0/1510346142 wait complete. 2024-09-11T23:21:31.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.092+0000 7f54b4f73740 20 client.16251 trim_cache size 0 max 16384 2024-09-11T23:21:31.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.109+0000 7f54b4f73740 1 Processor -- start 2024-09-11T23:21:31.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.109+0000 7f54b4f73740 1 -- start start 2024-09-11T23:21:31.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.109+0000 7f54b4f73740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bc0983e5f0 0x55bc09844ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:31.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.109+0000 7f54b4f73740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bc09842060 0x55bc0984c180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:31.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.109+0000 7f54b4f73740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bc09842980 0x55bc09847490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:31.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.109+0000 7f54b4f73740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f54ac1ab720 con 0x55bc09842980 2024-09-11T23:21:31.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.109+0000 7f54b4f73740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f54ac1ad3e0 con 0x55bc0983e5f0 2024-09-11T23:21:31.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.109+0000 7f54b4f73740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f54ac1abdf0 con 0x55bc09842060 2024-09-11T23:21:31.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.110+0000 7f54abfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bc09842980 0x55bc09847490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:31.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.110+0000 7f54abfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bc09842980 0x55bc09847490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52046/0 (socket says 172.21.15.26:52046) 2024-09-11T23:21:31.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.110+0000 7f54abfff640 1 -- 172.21.15.26:0/3738177964 learned_addr learned my addr 172.21.15.26:0/3738177964 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:21:31.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.110+0000 7f54b0936640 1 --2- 172.21.15.26:0/3738177964 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bc09842060 0x55bc0984c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:31.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.110+0000 7f54abfff640 1 --2- 172.21.15.26:0/3738177964 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bc0983e5f0 0x55bc09844ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:31.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.110+0000 7f54abfff640 1 -- 172.21.15.26:0/3738177964 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bc09842060 msgr2=0x55bc0984c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:21:31.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.110+0000 7f54abfff640 1 --2- 172.21.15.26:0/3738177964 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bc09842060 0x55bc0984c180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:31.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.110+0000 7f54abfff640 1 -- 172.21.15.26:0/3738177964 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bc0983e5f0 msgr2=0x55bc09844ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:21:31.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.110+0000 7f54abfff640 1 --2- 172.21.15.26:0/3738177964 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bc0983e5f0 0x55bc09844ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:31.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.110+0000 7f54abfff640 1 -- 172.21.15.26:0/3738177964 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc0984cb80 con 0x55bc09842980 2024-09-11T23:21:31.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.110+0000 7f54abfff640 1 --2- 172.21.15.26:0/3738177964 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bc09842980 0x55bc09847490 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f5498001eb0 tx=0x7f54980043c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e1f9751df921ad9 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:21:31.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.110+0000 7f54967fc640 1 -- 172.21.15.26:0/3738177964 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5498035040 con 0x55bc09842980 2024-09-11T23:21:31.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.110+0000 7f54967fc640 1 -- 172.21.15.26:0/3738177964 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54980046c0 con 0x55bc09842980 2024-09-11T23:21:31.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.110+0000 7f54967fc640 1 -- 172.21.15.26:0/3738177964 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5498031810 con 0x55bc09842980 2024-09-11T23:21:31.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.111+0000 7f54b4f73740 1 -- 172.21.15.26:0/3738177964 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bc09842980 msgr2=0x55bc09847490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:31.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.111+0000 7f54b4f73740 1 --2- 172.21.15.26:0/3738177964 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bc09842980 0x55bc09847490 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f5498001eb0 tx=0x7f54980043c0 comp rx=0 tx=0).stop 2024-09-11T23:21:31.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.111+0000 7f54b4f73740 1 -- 172.21.15.26:0/3738177964 shutdown_connections 2024-09-11T23:21:31.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.111+0000 7f54b4f73740 1 --2- 172.21.15.26:0/3738177964 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bc09842980 0x55bc09847490 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:31.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.111+0000 7f54b4f73740 1 --2- 172.21.15.26:0/3738177964 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bc09842060 0x55bc0984c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:31.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.111+0000 7f54b4f73740 1 --2- 172.21.15.26:0/3738177964 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bc0983e5f0 0x55bc09844ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:31.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.111+0000 7f54b4f73740 1 -- 172.21.15.26:0/3738177964 >> 172.21.15.26:0/3738177964 conn(0x55bc0984adf0 msgr2=0x55bc0984ba70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:21:31.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.111+0000 7f54b4f73740 1 -- 172.21.15.26:0/3738177964 shutdown_connections 2024-09-11T23:21:31.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.111+0000 7f54b4f73740 1 -- 172.21.15.26:0/3738177964 wait complete. 2024-09-11T23:21:31.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.111+0000 7f54b4f73740 1 -- start start 2024-09-11T23:21:31.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.111+0000 7f54b4f73740 1 Processor -- start 2024-09-11T23:21:31.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.112+0000 7f54b4f73740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bc0983e5f0 0x55bc09861db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:31.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.112+0000 7f54b4f73740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bc09842060 0x55bc098622f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:31.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.112+0000 7f54b4f73740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bc09842980 0x55bc09860210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:31.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.112+0000 7f54b4f73740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f54ac1acc50 con 0x55bc0983e5f0 2024-09-11T23:21:31.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.112+0000 7f54b4f73740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f54ac1ac4c0 con 0x55bc09842060 2024-09-11T23:21:31.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.112+0000 7f54b4f73740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f54ac1a9700 con 0x55bc09842980 2024-09-11T23:21:31.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.112+0000 7f54b4f73740 10 client.0 _ll_register_callbacks cb 0x7f5498004c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:21:31.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.112+0000 7f54abfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bc0983e5f0 0x55bc09861db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:31.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.112+0000 7f54abfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bc0983e5f0 0x55bc09861db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52062/0 (socket says 172.21.15.26:52062) 2024-09-11T23:21:31.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.112+0000 7f54abfff640 1 -- 172.21.15.26:0/1907334928 learned_addr learned my addr 172.21.15.26:0/1907334928 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:21:31.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.112+0000 7f54b0936640 1 --2- 172.21.15.26:0/1907334928 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bc09842060 0x55bc098622f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:31.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.112+0000 7f54abfff640 1 --2- 172.21.15.26:0/1907334928 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bc09842980 0x55bc09860210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:31.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.112+0000 7f54abfff640 1 -- 172.21.15.26:0/1907334928 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bc09842980 msgr2=0x55bc09860210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:21:31.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.112+0000 7f54abfff640 1 --2- 172.21.15.26:0/1907334928 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bc09842980 0x55bc09860210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:31.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.112+0000 7f54abfff640 1 -- 172.21.15.26:0/1907334928 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bc09842060 msgr2=0x55bc098622f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:21:31.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.112+0000 7f54abfff640 1 --2- 172.21.15.26:0/1907334928 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bc09842060 0x55bc098622f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:31.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.112+0000 7f54abfff640 1 -- 172.21.15.26:0/1907334928 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc09860a40 con 0x55bc0983e5f0 2024-09-11T23:21:31.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.112+0000 7f54abfff640 1 --2- 172.21.15.26:0/1907334928 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bc0983e5f0 0x55bc09861db0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f5498031f30 tx=0x7f5498005570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=801fb51242d4dc2b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:21:31.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.112+0000 7f548bfff640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:21:31.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.113+0000 7f548bfff640 1 -- 172.21.15.26:0/1907334928 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5498035040 con 0x55bc0983e5f0 2024-09-11T23:21:31.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.113+0000 7f548bfff640 1 -- 172.21.15.26:0/1907334928 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f54980317c0 con 0x55bc0983e5f0 2024-09-11T23:21:31.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.113+0000 7f54b4f73740 1 -- 172.21.15.26:0/1907334928 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54980046c0 con 0x55bc0983e5f0 2024-09-11T23:21:31.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.113+0000 7f548bfff640 1 -- 172.21.15.26:0/1907334928 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f549806e050 con 0x55bc0983e5f0 2024-09-11T23:21:31.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.113+0000 7f54b4f73740 10 client.16257 fetch_fsmap learned FSMap version 33 2024-09-11T23:21:31.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.113+0000 7f54b4f73740 1 -- 172.21.15.26:0/1907334928 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55bc098c8120 con 0x55bc0983e5f0 2024-09-11T23:21:31.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.113+0000 7f548bfff640 1 -- 172.21.15.26:0/1907334928 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54980046c0 con 0x55bc0983e5f0 2024-09-11T23:21:31.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.113+0000 7f548bfff640 1 -- 172.21.15.26:0/1907334928 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 33) ==== 8394+0+0 (secure 0 0 0) 0x7f5498031a50 con 0x55bc0983e5f0 2024-09-11T23:21:31.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.113+0000 7f54b4f73740 10 client.16257 fetch_fsmap finished waiting for FSMap version 33 2024-09-11T23:21:31.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.113+0000 7f54b4f73740 10 client.16257 resolve_mds: validated gid 24485 aka daemon mds.b 2024-09-11T23:21:31.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.113+0000 7f54b4f73740 20 client.16257 populate_metadata read hostname 'smithi026' 2024-09-11T23:21:31.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.113+0000 7f54b4f73740 1 --2- 172.21.15.26:0/1907334928 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55bc0984d240 0x55bc098ca670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:31.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.113+0000 7f54b4f73740 4 client.16257 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "6529ce3d", "release": true, "await": true}] 2024-09-11T23:21:31.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.113+0000 7f54b4f73740 1 -- 172.21.15.26:0/1907334928 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "quiesce db", "set_id": "6529ce3d", "release": true, "await": true}) -- 0x55bc097af0a0 con 0x55bc0984d240 2024-09-11T23:21:31.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.114+0000 7f54b0936640 1 --2- 172.21.15.26:0/1907334928 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55bc0984d240 0x55bc098ca670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:31.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.114+0000 7f54b0936640 1 --2- 172.21.15.26:0/1907334928 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55bc0984d240 0x55bc098ca670 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5227c3d018b364fe server_cookie=74e41882027b5bb9 in_seq=0 out_seq=0 2024-09-11T23:21:31.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.114+0000 7f548bfff640 10 client.16257 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:21:31.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.659+0000 7f548bfff640 1 -- 172.21.15.26:0/1907334928 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55bc097af0a0 con 0x55bc0984d240 2024-09-11T23:21:31.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.659+0000 7f548bfff640 10 client.16257 handle_command_reply: tid=0 2024-09-11T23:21:31.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.659+0000 7f54b4f73740 1 client.16257 shutdown 2024-09-11T23:21:31.660 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:21:31 smithi026 ceph-mon[31172]: pgmap v97: 97 pgs: 97 active+clean; 405 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 1.6 MiB/s wr, 266 op/s 2024-09-11T23:21:31.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.659+0000 7f54b4f73740 1 -- 172.21.15.26:0/1907334928 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bc0983e5f0 msgr2=0x55bc09861db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:31.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.659+0000 7f54b4f73740 1 --2- 172.21.15.26:0/1907334928 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bc0983e5f0 0x55bc09861db0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f5498031f30 tx=0x7f5498005570 comp rx=0 tx=0).stop 2024-09-11T23:21:31.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.659+0000 7f54b4f73740 1 -- 172.21.15.26:0/1907334928 shutdown_connections 2024-09-11T23:21:31.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.659+0000 7f54b4f73740 1 --2- 172.21.15.26:0/1907334928 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bc09842060 0x55bc098622f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:31.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.659+0000 7f54b4f73740 1 --2- 172.21.15.26:0/1907334928 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55bc0984d240 0x55bc098ca670 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:31.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.659+0000 7f54b4f73740 1 --2- 172.21.15.26:0/1907334928 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bc09842980 0x55bc09860210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:31.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.659+0000 7f548bfff640 0 client.16257 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:21:31.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.659+0000 7f54b4f73740 1 --2- 172.21.15.26:0/1907334928 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bc0983e5f0 0x55bc09861db0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:31.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.659+0000 7f54b4f73740 1 -- 172.21.15.26:0/1907334928 >> 172.21.15.26:0/1907334928 conn(0x55bc0984adf0 msgr2=0x55bc098589a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:21:31.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.659+0000 7f54b4f73740 1 -- 172.21.15.26:0/1907334928 shutdown_connections 2024-09-11T23:21:31.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.659+0000 7f54b4f73740 1 -- 172.21.15.26:0/1907334928 wait complete. 2024-09-11T23:21:31.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.659+0000 7f54b4f73740 20 client.16257 trim_cache size 0 max 16384 2024-09-11T23:21:31.662 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-11T23:21:31.662 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 33, 2024-09-11T23:21:31.662 INFO:teuthology.orchestra.run.smithi026.stdout: "leader": 24485, 2024-09-11T23:21:31.662 INFO:teuthology.orchestra.run.smithi026.stdout: "set_version": 12, 2024-09-11T23:21:31.662 INFO:teuthology.orchestra.run.smithi026.stdout: "sets": { 2024-09-11T23:21:31.662 INFO:teuthology.orchestra.run.smithi026.stdout: "6529ce3d": { 2024-09-11T23:21:31.662 INFO:teuthology.orchestra.run.smithi026.stdout: "version": 12, 2024-09-11T23:21:31.662 INFO:teuthology.orchestra.run.smithi026.stdout: "age_ref": 0.0, 2024-09-11T23:21:31.662 INFO:teuthology.orchestra.run.smithi026.stdout: "state": { 2024-09-11T23:21:31.662 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "RELEASED", 2024-09-11T23:21:31.663 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.0 2024-09-11T23:21:31.663 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:21:31.663 INFO:teuthology.orchestra.run.smithi026.stdout: "timeout": 90.0, 2024-09-11T23:21:31.663 INFO:teuthology.orchestra.run.smithi026.stdout: "expiration": 144.7, 2024-09-11T23:21:31.663 INFO:teuthology.orchestra.run.smithi026.stdout: "members": { 2024-09-11T23:21:31.663 INFO:teuthology.orchestra.run.smithi026.stdout: "file:/": { 2024-09-11T23:21:31.663 INFO:teuthology.orchestra.run.smithi026.stdout: "excluded": false, 2024-09-11T23:21:31.663 INFO:teuthology.orchestra.run.smithi026.stdout: "state": { 2024-09-11T23:21:31.663 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "RELEASED", 2024-09-11T23:21:31.663 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.0 2024-09-11T23:21:31.663 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:21:31.663 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:21:31.663 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:21:31.664 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:21:31.664 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:21:31.664 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-11T23:21:31.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.659+0000 7f54b2b9a640 1 -- 172.21.15.26:0/2293998557 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f549c0a7100 msgr2=0x7f549c0a95c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:31.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.659+0000 7f54b2b9a640 1 --2- 172.21.15.26:0/2293998557 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f549c0a7100 0x7f549c0a95c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f54a40986d0 tx=0x7f54a406ed70 comp rx=0 tx=0).stop 2024-09-11T23:21:31.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.659+0000 7f54b2b9a640 1 -- 172.21.15.26:0/2293998557 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f54ac106f50 msgr2=0x7f54ac19cdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:31.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.659+0000 7f54b2b9a640 1 --2- 172.21.15.26:0/2293998557 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f54ac106f50 0x7f54ac19cdd0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f54a400cad0 tx=0x7f54a400cf90 comp rx=0 tx=0).stop 2024-09-11T23:21:31.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.659+0000 7f54b2b9a640 1 -- 172.21.15.26:0/2293998557 shutdown_connections 2024-09-11T23:21:31.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.659+0000 7f54b2b9a640 1 --2- 172.21.15.26:0/2293998557 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f549c0a7100 0x7f549c0a95c0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:31.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.659+0000 7f54b2b9a640 1 --2- 172.21.15.26:0/2293998557 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f54ac106f50 0x7f54ac19cdd0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:31.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.659+0000 7f54b2b9a640 1 --2- 172.21.15.26:0/2293998557 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f54ac100ca0 0x7f54ac198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:31.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.659+0000 7f54b2b9a640 1 --2- 172.21.15.26:0/2293998557 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f54ac1002f0 0x7f54ac198500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:31.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.659+0000 7f54b2b9a640 1 -- 172.21.15.26:0/2293998557 >> 172.21.15.26:0/2293998557 conn(0x7f54ac077210 msgr2=0x7f54ac1052a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:21:31.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.660+0000 7f54b2b9a640 1 -- 172.21.15.26:0/2293998557 shutdown_connections 2024-09-11T23:21:31.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:21:31.660+0000 7f54b2b9a640 1 -- 172.21.15.26:0/2293998557 wait complete. 2024-09-11T23:21:31.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:21:31 smithi136 ceph-mon[35121]: pgmap v97: 97 pgs: 97 active+clean; 405 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 1.6 MiB/s wr, 266 op/s 2024-09-11T23:21:31.802 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:21:31 smithi076 ceph-mon[35296]: pgmap v97: 97 pgs: 97 active+clean; 405 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 1.6 MiB/s wr, 266 op/s 2024-09-11T23:21:32.505 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '6529ce3d', total seconds elapsed: 28.6 2024-09-11T23:21:33.577 INFO:tasks.workunit.client.0.smithi026.stdout: 119 356440 211 224084 69 105393 88 2024-09-11T23:21:33.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:21:33 smithi136 ceph-mon[35121]: pgmap v98: 97 pgs: 97 active+clean; 405 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 1.6 MiB/s wr, 266 op/s 2024-09-11T23:21:33.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:21:33 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:21:33.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:21:33 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:21:33.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:21:33 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:21:33.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:21:33 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:21:33.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:21:33 smithi026 ceph-mon[31172]: pgmap v98: 97 pgs: 97 active+clean; 405 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 1.6 MiB/s wr, 266 op/s 2024-09-11T23:21:33.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:21:33 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:21:33.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:21:33 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:21:33.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:21:33 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:21:33.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:21:33 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:21:33.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:21:33 smithi076 ceph-mon[35296]: pgmap v98: 97 pgs: 97 active+clean; 405 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 1.6 MiB/s wr, 266 op/s 2024-09-11T23:21:33.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:21:33 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:21:33.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:21:33 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:21:33.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:21:33 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:21:33.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:21:33 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:21:35.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:21:35 smithi136 ceph-mon[35121]: pgmap v99: 97 pgs: 97 active+clean; 410 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.0 MiB/s wr, 317 op/s 2024-09-11T23:21:35.750 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:21:35 smithi026 ceph-mon[31172]: pgmap v99: 97 pgs: 97 active+clean; 410 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.0 MiB/s wr, 317 op/s 2024-09-11T23:21:35.802 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:21:35 smithi076 ceph-mon[35296]: pgmap v99: 97 pgs: 97 active+clean; 410 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.0 MiB/s wr, 317 op/s 2024-09-11T23:21:37.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:21:37 smithi136 ceph-mon[35121]: pgmap v100: 97 pgs: 97 active+clean; 410 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 978 KiB/s wr, 142 op/s 2024-09-11T23:21:37.750 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:21:37 smithi026 ceph-mon[31172]: pgmap v100: 97 pgs: 97 active+clean; 410 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 978 KiB/s wr, 142 op/s 2024-09-11T23:21:37.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:21:37 smithi076 ceph-mon[35296]: pgmap v100: 97 pgs: 97 active+clean; 410 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 978 KiB/s wr, 142 op/s 2024-09-11T23:21:39.014 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.012+0000 7ffa08c74640 1 -- 172.21.15.26:0/2470981914 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffa04102d20 msgr2=0x7ffa04103100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:39.014 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.012+0000 7ffa08c74640 1 --2- 172.21.15.26:0/2470981914 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffa04102d20 0x7ffa04103100 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7ff9f4009900 tx=0x7ff9f402a480 comp rx=0 tx=0).stop 2024-09-11T23:21:39.014 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.013+0000 7ffa08c74640 1 -- 172.21.15.26:0/2470981914 shutdown_connections 2024-09-11T23:21:39.014 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.013+0000 7ffa08c74640 1 --2- 172.21.15.26:0/2470981914 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffa0410c0f0 0x7ffa0410e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:39.014 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.013+0000 7ffa08c74640 1 --2- 172.21.15.26:0/2470981914 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffa04103640 0x7ffa0410bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:39.014 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.013+0000 7ffa08c74640 1 --2- 172.21.15.26:0/2470981914 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffa04102d20 0x7ffa04103100 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:39.014 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.013+0000 7ffa08c74640 1 -- 172.21.15.26:0/2470981914 >> 172.21.15.26:0/2470981914 conn(0x7ffa04077990 msgr2=0x7ffa040fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:21:39.015 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.014+0000 7ffa08c74640 1 -- 172.21.15.26:0/2470981914 shutdown_connections 2024-09-11T23:21:39.015 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.014+0000 7ffa08c74640 1 -- 172.21.15.26:0/2470981914 wait complete. 2024-09-11T23:21:39.017 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.016+0000 7ffa08c74640 1 Processor -- start 2024-09-11T23:21:39.017 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.016+0000 7ffa08c74640 1 -- start start 2024-09-11T23:21:39.017 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.016+0000 7ffa08c74640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffa04102d20 0x7ffa04198160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:39.017 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.016+0000 7ffa08c74640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffa04103640 0x7ffa041986a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:39.017 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.016+0000 7ffa08c74640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffa0410c0f0 0x7ffa0419ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:39.018 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.016+0000 7ffa01d9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffa04103640 0x7ffa041986a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:39.018 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.016+0000 7ffa01d9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffa04103640 0x7ffa041986a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:39498/0 (socket says 172.21.15.26:39498) 2024-09-11T23:21:39.019 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.016+0000 7ffa0259c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffa04102d20 0x7ffa04198160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:39.019 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.017+0000 7ffa0259c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffa04102d20 0x7ffa04198160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52066/0 (socket says 172.21.15.26:52066) 2024-09-11T23:21:39.019 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.017+0000 7ffa01d9b640 1 -- 172.21.15.26:0/2097732104 learned_addr learned my addr 172.21.15.26:0/2097732104 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:21:39.019 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.017+0000 7ffa08c74640 1 -- 172.21.15.26:0/2097732104 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ffa04111ec0 con 0x7ffa04102d20 2024-09-11T23:21:39.019 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.017+0000 7ffa08c74640 1 -- 172.21.15.26:0/2097732104 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7ffa04111d40 con 0x7ffa04103640 2024-09-11T23:21:39.019 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.017+0000 7ffa08c74640 1 -- 172.21.15.26:0/2097732104 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ffa04112040 con 0x7ffa0410c0f0 2024-09-11T23:21:39.019 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.017+0000 7ffa02d9d640 1 --2- 172.21.15.26:0/2097732104 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffa0410c0f0 0x7ffa0419ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:39.019 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.017+0000 7ffa01d9b640 1 -- 172.21.15.26:0/2097732104 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffa0410c0f0 msgr2=0x7ffa0419ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:39.019 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.017+0000 7ffa01d9b640 1 --2- 172.21.15.26:0/2097732104 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffa0410c0f0 0x7ffa0419ca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:39.019 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.017+0000 7ffa01d9b640 1 -- 172.21.15.26:0/2097732104 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffa04102d20 msgr2=0x7ffa04198160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:39.019 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.017+0000 7ffa01d9b640 1 --2- 172.21.15.26:0/2097732104 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffa04102d20 0x7ffa04198160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:39.019 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.017+0000 7ffa01d9b640 1 -- 172.21.15.26:0/2097732104 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa0419d150 con 0x7ffa04103640 2024-09-11T23:21:39.019 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.017+0000 7ffa01d9b640 1 --2- 172.21.15.26:0/2097732104 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffa04103640 0x7ffa041986a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7ff9f800e9f0 tx=0x7ff9f800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:21:39.019 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.017+0000 7ff9f37fe640 1 -- 172.21.15.26:0/2097732104 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9f800cd80 con 0x7ffa04103640 2024-09-11T23:21:39.020 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.017+0000 7ff9f37fe640 1 -- 172.21.15.26:0/2097732104 <== mon.2 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9f8010910 con 0x7ffa04103640 2024-09-11T23:21:39.020 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.017+0000 7ffa08c74640 1 -- 172.21.15.26:0/2097732104 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa0419d440 con 0x7ffa04103640 2024-09-11T23:21:39.020 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.018+0000 7ff9f37fe640 1 -- 172.21.15.26:0/2097732104 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9f8018990 con 0x7ffa04103640 2024-09-11T23:21:39.020 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.018+0000 7ffa08c74640 1 -- 172.21.15.26:0/2097732104 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa041008a0 con 0x7ffa04103640 2024-09-11T23:21:39.021 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.020+0000 7ff9f37fe640 1 -- 172.21.15.26:0/2097732104 <== mon.2 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7ff9f8018b30 con 0x7ffa04103640 2024-09-11T23:21:39.022 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.021+0000 7ffa0ba17740 1 Processor -- start 2024-09-11T23:21:39.023 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.022+0000 7ffa0ba17740 1 -- start start 2024-09-11T23:21:39.023 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.022+0000 7ff9f37fe640 1 --2- 172.21.15.26:0/2097732104 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7ff9ec0ab5c0 0x7ff9ec0ada80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:39.023 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.022+0000 7ffa0ba17740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5577a2d48af0 0x5577a2d48ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:39.023 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.022+0000 7ffa0ba17740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5577a2d49410 0x5577a2d53770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:39.023 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.022+0000 7ffa0ba17740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5577a2d53cb0 0x5577a2d560a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:39.023 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.022+0000 7ffa0ba17740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5577a2cad2f0 con 0x5577a2d49410 2024-09-11T23:21:39.023 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.022+0000 7ffa0ba17740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5577a2c55270 con 0x5577a2d48af0 2024-09-11T23:21:39.023 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.022+0000 7ffa0ba17740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5577a2ca92b0 con 0x5577a2d53cb0 2024-09-11T23:21:39.024 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.022+0000 7ffa02d9d640 1 --2- 172.21.15.26:0/2097732104 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7ff9ec0ab5c0 0x7ff9ec0ada80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:39.024 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.022+0000 7ff9f37fe640 1 -- 172.21.15.26:0/2097732104 <== mon.2 v2:172.21.15.76:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7ff9f80cf840 con 0x7ffa04103640 2024-09-11T23:21:39.025 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.022+0000 7ffa0259c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5577a2d48af0 0x5577a2d48ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:39.025 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.022+0000 7ffa0259c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5577a2d48af0 0x5577a2d48ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:39512/0 (socket says 172.21.15.26:39512) 2024-09-11T23:21:39.025 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.022+0000 7ffa0259c640 1 -- 172.21.15.26:0/4054813264 learned_addr learned my addr 172.21.15.26:0/4054813264 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:21:39.025 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.023+0000 7ffa01d9b640 1 --2- 172.21.15.26:0/4054813264 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5577a2d49410 0x5577a2d53770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:39.025 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.023+0000 7ffa0259c640 1 -- 172.21.15.26:0/4054813264 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5577a2d53cb0 msgr2=0x5577a2d560a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T23:21:39.025 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.023+0000 7ffa0259c640 1 --2- 172.21.15.26:0/4054813264 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5577a2d53cb0 0x5577a2d560a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:39.025 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.023+0000 7ffa0259c640 1 -- 172.21.15.26:0/4054813264 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5577a2d49410 msgr2=0x5577a2d53770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:21:39.025 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.023+0000 7ffa0259c640 1 --2- 172.21.15.26:0/4054813264 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5577a2d49410 0x5577a2d53770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:39.025 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.023+0000 7ffa0259c640 1 -- 172.21.15.26:0/4054813264 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5577a2c54340 con 0x5577a2d48af0 2024-09-11T23:21:39.025 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.023+0000 7ffa01d9b640 1 --2- 172.21.15.26:0/4054813264 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5577a2d49410 0x5577a2d53770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:21:39.025 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.023+0000 7ffa0259c640 1 --2- 172.21.15.26:0/4054813264 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5577a2d48af0 0x5577a2d48ed0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ff9f40314d0 tx=0x7ff9f4000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c0cc8d4ee7735a71 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:21:39.025 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.023+0000 7ffa02d9d640 1 --2- 172.21.15.26:0/2097732104 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7ff9ec0ab5c0 0x7ff9ec0ada80 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x5577a2d463e0 tx=0x7ff9fc009500 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:21:39.025 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.024+0000 7ff9f0ff9640 1 -- 172.21.15.26:0/4054813264 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9f4035040 con 0x5577a2d48af0 2024-09-11T23:21:39.026 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.024+0000 7ff9f0ff9640 1 -- 172.21.15.26:0/4054813264 <== mon.2 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9f4005020 con 0x5577a2d48af0 2024-09-11T23:21:39.026 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.024+0000 7ff9f0ff9640 1 -- 172.21.15.26:0/4054813264 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9f4004ba0 con 0x5577a2d48af0 2024-09-11T23:21:39.027 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.024+0000 7ffa0ba17740 1 -- 172.21.15.26:0/4054813264 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5577a2d48af0 msgr2=0x5577a2d48ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:39.028 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.024+0000 7ffa0ba17740 1 --2- 172.21.15.26:0/4054813264 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5577a2d48af0 0x5577a2d48ed0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ff9f40314d0 tx=0x7ff9f4000bc0 comp rx=0 tx=0).stop 2024-09-11T23:21:39.028 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.025+0000 7ffa0ba17740 1 -- 172.21.15.26:0/4054813264 shutdown_connections 2024-09-11T23:21:39.028 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.025+0000 7ffa0ba17740 1 --2- 172.21.15.26:0/4054813264 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5577a2d53cb0 0x5577a2d560a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:39.028 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.025+0000 7ffa0ba17740 1 --2- 172.21.15.26:0/4054813264 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5577a2d49410 0x5577a2d53770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:39.028 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.025+0000 7ffa0ba17740 1 --2- 172.21.15.26:0/4054813264 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5577a2d48af0 0x5577a2d48ed0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:39.028 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.025+0000 7ffa0ba17740 1 -- 172.21.15.26:0/4054813264 >> 172.21.15.26:0/4054813264 conn(0x5577a2d45060 msgr2=0x5577a2d45440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:21:39.028 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.025+0000 7ffa0ba17740 1 -- 172.21.15.26:0/4054813264 shutdown_connections 2024-09-11T23:21:39.028 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.025+0000 7ffa0ba17740 1 -- 172.21.15.26:0/4054813264 wait complete. 2024-09-11T23:21:39.028 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.026+0000 7ffa0ba17740 1 -- start start 2024-09-11T23:21:39.028 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.026+0000 7ffa0ba17740 1 Processor -- start 2024-09-11T23:21:39.028 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.026+0000 7ffa0ba17740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5577a2d48af0 0x5577a2d50ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:39.028 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.026+0000 7ffa0ba17740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5577a2d49410 0x5577a2d510e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:39.029 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.026+0000 7ffa0ba17740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5577a2d53cb0 0x5577a2d51620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:39.029 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.026+0000 7ffa0ba17740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5577a2d56880 con 0x5577a2d49410 2024-09-11T23:21:39.029 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.026+0000 7ffa0ba17740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5577a2c36290 con 0x5577a2d48af0 2024-09-11T23:21:39.029 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.026+0000 7ffa0ba17740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5577a2d56a00 con 0x5577a2d53cb0 2024-09-11T23:21:39.029 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.026+0000 7ffa0ba17740 10 client.0 _ll_register_callbacks cb 0x5577a2c67140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:21:39.029 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.027+0000 7ffa0259c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5577a2d48af0 0x5577a2d50ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:39.029 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.027+0000 7ffa01d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5577a2d49410 0x5577a2d510e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:39.029 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.027+0000 7ffa01d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5577a2d49410 0x5577a2d510e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52084/0 (socket says 172.21.15.26:52084) 2024-09-11T23:21:39.029 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.027+0000 7ffa01d9b640 1 -- 172.21.15.26:0/1065532151 learned_addr learned my addr 172.21.15.26:0/1065532151 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:21:39.030 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.027+0000 7ffa02d9d640 1 --2- 172.21.15.26:0/1065532151 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5577a2d53cb0 0x5577a2d51620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:39.030 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.027+0000 7ffa01d9b640 1 -- 172.21.15.26:0/1065532151 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5577a2d53cb0 msgr2=0x5577a2d51620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:21:39.030 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.027+0000 7ffa01d9b640 1 --2- 172.21.15.26:0/1065532151 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5577a2d53cb0 0x5577a2d51620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:39.031 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.027+0000 7ffa01d9b640 1 -- 172.21.15.26:0/1065532151 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5577a2d48af0 msgr2=0x5577a2d50ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:21:39.031 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.027+0000 7ffa01d9b640 1 --2- 172.21.15.26:0/1065532151 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5577a2d48af0 0x5577a2d50ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:39.031 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.028+0000 7ffa01d9b640 1 -- 172.21.15.26:0/1065532151 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5577a29c0210 con 0x5577a2d49410 2024-09-11T23:21:39.031 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.028+0000 7ffa01d9b640 1 --2- 172.21.15.26:0/1065532151 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5577a2d49410 0x5577a2d510e0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7ff9f80cd050 tx=0x7ff9f809beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b68aeeaddc72b79 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:21:39.031 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.028+0000 7ff9e9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:21:39.031 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.028+0000 7ff9e9ffb640 1 -- 172.21.15.26:0/1065532151 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9f80cd220 con 0x5577a2d49410 2024-09-11T23:21:39.031 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.028+0000 7ffa0ba17740 1 -- 172.21.15.26:0/1065532151 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9f4005020 con 0x5577a2d49410 2024-09-11T23:21:39.031 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.028+0000 7ff9e9ffb640 1 -- 172.21.15.26:0/1065532151 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7ff9f80983e0 con 0x5577a2d49410 2024-09-11T23:21:39.031 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.029+0000 7ff9e9ffb640 1 -- 172.21.15.26:0/1065532151 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7ff9f80ba050 con 0x5577a2d49410 2024-09-11T23:21:39.031 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.029+0000 7ffa0ba17740 10 client.16266 fetch_fsmap learned FSMap version 33 2024-09-11T23:21:39.031 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.029+0000 7ffa0ba17740 1 -- 172.21.15.26:0/1065532151 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5577a2a8c620 con 0x5577a2d49410 2024-09-11T23:21:39.032 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.030+0000 7ff9e9ffb640 1 -- 172.21.15.26:0/1065532151 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9f4005020 con 0x5577a2d49410 2024-09-11T23:21:39.032 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.030+0000 7ff9e9ffb640 1 -- 172.21.15.26:0/1065532151 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 33) ==== 8394+0+0 (secure 0 0 0) 0x7ff9f809ac30 con 0x5577a2d49410 2024-09-11T23:21:39.032 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.031+0000 7ffa0ba17740 10 client.16266 fetch_fsmap finished waiting for FSMap version 33 2024-09-11T23:21:39.032 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.031+0000 7ffa0ba17740 10 client.16266 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:21:39.032 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.031+0000 7ffa0ba17740 20 client.16266 populate_metadata read hostname 'smithi026' 2024-09-11T23:21:39.032 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.031+0000 7ffa0ba17740 1 --2- 172.21.15.26:0/1065532151 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5577a2d63d20 0x5577a2d66160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:39.033 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.031+0000 7ffa0ba17740 4 client.16266 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:21:39.033 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.031+0000 7ffa0ba17740 1 -- 172.21.15.26:0/1065532151 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5577a2cbaa20 con 0x5577a2d63d20 2024-09-11T23:21:39.033 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.032+0000 7ffa0259c640 1 --2- 172.21.15.26:0/1065532151 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5577a2d63d20 0x5577a2d66160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:39.033 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.032+0000 7ffa0259c640 1 --2- 172.21.15.26:0/1065532151 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5577a2d63d20 0x5577a2d66160 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c539645960fb15b server_cookie=64dc1ca54a4a6665 in_seq=0 out_seq=0 2024-09-11T23:21:39.033 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.032+0000 7ff9e9ffb640 10 client.16266 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:21:39.034 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.033+0000 7ff9e9ffb640 1 -- 172.21.15.26:0/1065532151 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5577a2cbaa20 con 0x5577a2d63d20 2024-09-11T23:21:39.034 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.033+0000 7ff9e9ffb640 10 client.16266 handle_command_reply: tid=0 2024-09-11T23:21:39.034 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.034+0000 7ffa0ba17740 1 client.16266 shutdown 2024-09-11T23:21:39.036 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.034+0000 7ffa0ba17740 1 -- 172.21.15.26:0/1065532151 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5577a2d49410 msgr2=0x5577a2d510e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:39.036 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.034+0000 7ffa0ba17740 1 --2- 172.21.15.26:0/1065532151 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5577a2d49410 0x5577a2d510e0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7ff9f80cd050 tx=0x7ff9f809beb0 comp rx=0 tx=0).stop 2024-09-11T23:21:39.036 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.034+0000 7ffa0ba17740 1 -- 172.21.15.26:0/1065532151 shutdown_connections 2024-09-11T23:21:39.036 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.034+0000 7ffa0ba17740 1 --2- 172.21.15.26:0/1065532151 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5577a2d63d20 0x5577a2d66160 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:39.036 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.034+0000 7ffa0ba17740 1 --2- 172.21.15.26:0/1065532151 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5577a2d53cb0 0x5577a2d51620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:39.036 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.034+0000 7ffa0ba17740 1 --2- 172.21.15.26:0/1065532151 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5577a2d49410 0x5577a2d510e0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:39.036 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.034+0000 7ff9e9ffb640 0 client.16266 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:21:39.036 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.034+0000 7ffa0ba17740 1 --2- 172.21.15.26:0/1065532151 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5577a2d48af0 0x5577a2d50ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:39.036 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.035+0000 7ffa0ba17740 1 -- 172.21.15.26:0/1065532151 >> 172.21.15.26:0/1065532151 conn(0x5577a2d45060 msgr2=0x5577a2d4adc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:21:39.037 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.035+0000 7ffa0ba17740 1 -- 172.21.15.26:0/1065532151 shutdown_connections 2024-09-11T23:21:39.037 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.035+0000 7ffa0ba17740 1 -- 172.21.15.26:0/1065532151 wait complete. 2024-09-11T23:21:39.037 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.035+0000 7ffa0ba17740 20 client.16266 trim_cache size 0 max 16384 2024-09-11T23:21:39.069 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.067+0000 7ffa0ba17740 1 Processor -- start 2024-09-11T23:21:39.069 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.067+0000 7ffa0ba17740 1 -- start start 2024-09-11T23:21:39.069 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.067+0000 7ffa0ba17740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5577a2d48af0 0x5577a2d51450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:39.069 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.067+0000 7ffa0ba17740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5577a2d49410 0x5577a2d6ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:39.069 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.067+0000 7ffa0ba17740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5577a2d53cb0 0x5577a2d6bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:39.069 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.067+0000 7ffa0ba17740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ffa041ab630 con 0x5577a2d48af0 2024-09-11T23:21:39.070 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.067+0000 7ffa0ba17740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7ffa041ad2f0 con 0x5577a2d53cb0 2024-09-11T23:21:39.070 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.067+0000 7ffa0ba17740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ffa041abd00 con 0x5577a2d49410 2024-09-11T23:21:39.070 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.067+0000 7ffa0259c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5577a2d48af0 0x5577a2d51450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:39.070 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.067+0000 7ffa0259c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5577a2d48af0 0x5577a2d51450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52094/0 (socket says 172.21.15.26:52094) 2024-09-11T23:21:39.070 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.067+0000 7ffa0259c640 1 -- 172.21.15.26:0/3146222237 learned_addr learned my addr 172.21.15.26:0/3146222237 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:21:39.070 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.067+0000 7ffa02d9d640 1 --2- 172.21.15.26:0/3146222237 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5577a2d53cb0 0x5577a2d6bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:39.070 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.067+0000 7ffa01d9b640 1 --2- 172.21.15.26:0/3146222237 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5577a2d49410 0x5577a2d6ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:39.070 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.068+0000 7ffa0259c640 1 -- 172.21.15.26:0/3146222237 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5577a2d49410 msgr2=0x5577a2d6ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:21:39.070 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.068+0000 7ffa0259c640 1 --2- 172.21.15.26:0/3146222237 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5577a2d49410 0x5577a2d6ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:39.070 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.068+0000 7ffa0259c640 1 -- 172.21.15.26:0/3146222237 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5577a2d53cb0 msgr2=0x5577a2d6bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:21:39.070 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.068+0000 7ffa0259c640 1 --2- 172.21.15.26:0/3146222237 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5577a2d53cb0 0x5577a2d6bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:39.070 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.068+0000 7ffa0259c640 1 -- 172.21.15.26:0/3146222237 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5577a2d44b00 con 0x5577a2d48af0 2024-09-11T23:21:39.070 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.068+0000 7ffa0259c640 1 --2- 172.21.15.26:0/3146222237 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5577a2d48af0 0x5577a2d51450 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7ff9f402a9a0 tx=0x7ff9f4000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9795cf7b7a7c933b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:21:39.071 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.068+0000 7ff9ebfff640 1 -- 172.21.15.26:0/3146222237 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9f4035040 con 0x5577a2d48af0 2024-09-11T23:21:39.071 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.068+0000 7ff9ebfff640 1 -- 172.21.15.26:0/3146222237 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9f40035f0 con 0x5577a2d48af0 2024-09-11T23:21:39.071 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.068+0000 7ff9ebfff640 1 -- 172.21.15.26:0/3146222237 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9f40319f0 con 0x5577a2d48af0 2024-09-11T23:21:39.074 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.069+0000 7ffa0ba17740 1 -- 172.21.15.26:0/3146222237 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5577a2d48af0 msgr2=0x5577a2d51450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:39.074 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.069+0000 7ffa0ba17740 1 --2- 172.21.15.26:0/3146222237 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5577a2d48af0 0x5577a2d51450 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7ff9f402a9a0 tx=0x7ff9f4000bc0 comp rx=0 tx=0).stop 2024-09-11T23:21:39.074 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.069+0000 7ffa0ba17740 1 -- 172.21.15.26:0/3146222237 shutdown_connections 2024-09-11T23:21:39.074 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.069+0000 7ffa0ba17740 1 --2- 172.21.15.26:0/3146222237 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5577a2d53cb0 0x5577a2d6bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:39.074 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.069+0000 7ffa0ba17740 1 --2- 172.21.15.26:0/3146222237 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5577a2d49410 0x5577a2d6ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:39.074 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.069+0000 7ffa0ba17740 1 --2- 172.21.15.26:0/3146222237 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5577a2d48af0 0x5577a2d51450 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:39.074 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.069+0000 7ffa0ba17740 1 -- 172.21.15.26:0/3146222237 >> 172.21.15.26:0/3146222237 conn(0x5577a2d45060 msgr2=0x5577a2d52c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:21:39.074 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.069+0000 7ffa0ba17740 1 -- 172.21.15.26:0/3146222237 shutdown_connections 2024-09-11T23:21:39.074 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.069+0000 7ffa0ba17740 1 -- 172.21.15.26:0/3146222237 wait complete. 2024-09-11T23:21:39.075 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.069+0000 7ffa0ba17740 1 -- start start 2024-09-11T23:21:39.075 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.069+0000 7ffa0ba17740 1 Processor -- start 2024-09-11T23:21:39.075 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.070+0000 7ffa0ba17740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5577a2d48af0 0x5577a2d5c520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:39.075 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.070+0000 7ffa0ba17740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5577a2d49410 0x5577a2d57840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:39.075 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.070+0000 7ffa0ba17740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5577a2d53cb0 0x5577a2d57e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:39.075 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.070+0000 7ffa0ba17740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ffa041acb60 con 0x5577a2d49410 2024-09-11T23:21:39.075 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.070+0000 7ffa0ba17740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7ffa041ac3d0 con 0x5577a2d48af0 2024-09-11T23:21:39.075 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.070+0000 7ffa0ba17740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ffa041a9610 con 0x5577a2d53cb0 2024-09-11T23:21:39.075 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.070+0000 7ffa0ba17740 10 client.0 _ll_register_callbacks cb 0x7ff9f8098010 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:21:39.075 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.070+0000 7ffa01d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5577a2d49410 0x5577a2d57840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:39.075 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.070+0000 7ffa0259c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5577a2d48af0 0x5577a2d5c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:39.076 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.070+0000 7ffa0259c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5577a2d48af0 0x5577a2d5c520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:39542/0 (socket says 172.21.15.26:39542) 2024-09-11T23:21:39.076 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.070+0000 7ffa01d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5577a2d49410 0x5577a2d57840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52108/0 (socket says 172.21.15.26:52108) 2024-09-11T23:21:39.076 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.070+0000 7ffa0259c640 1 -- 172.21.15.26:0/3905729933 learned_addr learned my addr 172.21.15.26:0/3905729933 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:21:39.076 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.070+0000 7ffa02d9d640 1 --2- 172.21.15.26:0/3905729933 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5577a2d53cb0 0x5577a2d57e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:39.076 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.070+0000 7ffa01d9b640 1 -- 172.21.15.26:0/3905729933 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5577a2d53cb0 msgr2=0x5577a2d57e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:21:39.076 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.070+0000 7ffa01d9b640 1 --2- 172.21.15.26:0/3905729933 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5577a2d53cb0 0x5577a2d57e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:39.076 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.070+0000 7ffa01d9b640 1 -- 172.21.15.26:0/3905729933 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5577a2d48af0 msgr2=0x5577a2d5c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:21:39.076 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.070+0000 7ffa01d9b640 1 --2- 172.21.15.26:0/3905729933 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5577a2d48af0 0x5577a2d5c520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:39.076 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.070+0000 7ffa01d9b640 1 -- 172.21.15.26:0/3905729933 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5577a2d58660 con 0x5577a2d49410 2024-09-11T23:21:39.076 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.070+0000 7ffa01d9b640 1 --2- 172.21.15.26:0/3905729933 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5577a2d49410 0x5577a2d57840 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7ff9f809bd00 tx=0x7ff9f8018420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=127b28c2dd75e5fa server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:21:39.076 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.070+0000 7ff9e9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:21:39.076 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.070+0000 7ff9e9ffb640 1 -- 172.21.15.26:0/3905729933 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9f80a9080 con 0x5577a2d49410 2024-09-11T23:21:39.076 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.070+0000 7ffa0ba17740 1 -- 172.21.15.26:0/3905729933 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9f40035f0 con 0x5577a2d49410 2024-09-11T23:21:39.077 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.070+0000 7ff9e9ffb640 1 -- 172.21.15.26:0/3905729933 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7ff9f80983e0 con 0x5577a2d49410 2024-09-11T23:21:39.077 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.071+0000 7ff9e9ffb640 1 -- 172.21.15.26:0/3905729933 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7ff9f809b1c0 con 0x5577a2d49410 2024-09-11T23:21:39.077 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.071+0000 7ffa0ba17740 10 client.16272 fetch_fsmap learned FSMap version 33 2024-09-11T23:21:39.077 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.071+0000 7ffa0ba17740 1 -- 172.21.15.26:0/3905729933 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5577a2dceaf0 con 0x5577a2d49410 2024-09-11T23:21:39.077 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.071+0000 7ff9e9ffb640 1 -- 172.21.15.26:0/3905729933 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9f40035f0 con 0x5577a2d49410 2024-09-11T23:21:39.077 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.071+0000 7ff9e9ffb640 1 -- 172.21.15.26:0/3905729933 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 33) ==== 8394+0+0 (secure 0 0 0) 0x7ff9f80a12b0 con 0x5577a2d49410 2024-09-11T23:21:39.077 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.071+0000 7ffa0ba17740 10 client.16272 fetch_fsmap finished waiting for FSMap version 33 2024-09-11T23:21:39.077 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.071+0000 7ffa0ba17740 10 client.16272 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:21:39.077 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.071+0000 7ffa0ba17740 20 client.16272 populate_metadata read hostname 'smithi026' 2024-09-11T23:21:39.077 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.071+0000 7ffa0ba17740 1 --2- 172.21.15.26:0/3905729933 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5577a2d63d20 0x5577a2dd10f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:39.077 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.071+0000 7ffa0ba17740 4 client.16272 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T23:21:39.077 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.071+0000 7ffa0ba17740 1 -- 172.21.15.26:0/3905729933 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5577a2cb4d70 con 0x5577a2d63d20 2024-09-11T23:21:39.077 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.071+0000 7ffa0259c640 1 --2- 172.21.15.26:0/3905729933 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5577a2d63d20 0x5577a2dd10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:39.077 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.072+0000 7ffa0259c640 1 --2- 172.21.15.26:0/3905729933 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5577a2d63d20 0x5577a2dd10f0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=61e724b6949bacec server_cookie=39c7245b9b2c5529 in_seq=0 out_seq=0 2024-09-11T23:21:39.078 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.072+0000 7ff9e9ffb640 10 client.16272 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:21:39.272 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.271+0000 7ff9e9ffb640 1 -- 172.21.15.26:0/3905729933 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5577a2cb4d70 con 0x5577a2d63d20 2024-09-11T23:21:39.272 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.271+0000 7ff9e9ffb640 10 client.16272 handle_command_reply: tid=0 2024-09-11T23:21:39.272 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.271+0000 7ffa0ba17740 1 client.16272 shutdown 2024-09-11T23:21:39.273 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.271+0000 7ffa0ba17740 1 -- 172.21.15.26:0/3905729933 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5577a2d49410 msgr2=0x5577a2d57840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:39.273 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.271+0000 7ffa0ba17740 1 --2- 172.21.15.26:0/3905729933 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5577a2d49410 0x5577a2d57840 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7ff9f809bd00 tx=0x7ff9f8018420 comp rx=0 tx=0).stop 2024-09-11T23:21:39.273 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.271+0000 7ffa0ba17740 1 -- 172.21.15.26:0/3905729933 shutdown_connections 2024-09-11T23:21:39.273 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.271+0000 7ffa0ba17740 1 --2- 172.21.15.26:0/3905729933 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5577a2d63d20 0x5577a2dd10f0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:39.273 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.271+0000 7ffa0ba17740 1 --2- 172.21.15.26:0/3905729933 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5577a2d53cb0 0x5577a2d57e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:39.273 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.271+0000 7ffa0ba17740 1 --2- 172.21.15.26:0/3905729933 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5577a2d49410 0x5577a2d57840 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:39.273 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.271+0000 7ff9e9ffb640 0 client.16272 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:21:39.273 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.271+0000 7ffa0ba17740 1 --2- 172.21.15.26:0/3905729933 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5577a2d48af0 0x5577a2d5c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:39.273 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.271+0000 7ffa0ba17740 1 -- 172.21.15.26:0/3905729933 >> 172.21.15.26:0/3905729933 conn(0x5577a2d45060 msgr2=0x5577a2d4a520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:21:39.273 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.271+0000 7ffa0ba17740 1 -- 172.21.15.26:0/3905729933 shutdown_connections 2024-09-11T23:21:39.273 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.271+0000 7ffa0ba17740 1 -- 172.21.15.26:0/3905729933 wait complete. 2024-09-11T23:21:39.273 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.271+0000 7ffa0ba17740 20 client.16272 trim_cache size 0 max 16384 2024-09-11T23:21:39.274 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-09-11T23:21:39.274 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "message": "", 2024-09-11T23:21:39.274 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-09-11T23:21:39.274 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-09-11T23:21:39.275 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.272+0000 7ffa08c74640 1 -- 172.21.15.26:0/2097732104 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7ff9ec0ab5c0 msgr2=0x7ff9ec0ada80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:39.275 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.272+0000 7ffa08c74640 1 --2- 172.21.15.26:0/2097732104 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7ff9ec0ab5c0 0x7ff9ec0ada80 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x5577a2d463e0 tx=0x7ff9fc009500 comp rx=0 tx=0).stop 2024-09-11T23:21:39.275 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.272+0000 7ffa08c74640 1 -- 172.21.15.26:0/2097732104 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffa04103640 msgr2=0x7ffa041986a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:39.275 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.272+0000 7ffa08c74640 1 --2- 172.21.15.26:0/2097732104 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffa04103640 0x7ffa041986a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7ff9f800e9f0 tx=0x7ff9f800eeb0 comp rx=0 tx=0).stop 2024-09-11T23:21:39.275 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.272+0000 7ffa08c74640 1 -- 172.21.15.26:0/2097732104 shutdown_connections 2024-09-11T23:21:39.275 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.272+0000 7ffa08c74640 1 --2- 172.21.15.26:0/2097732104 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7ff9ec0ab5c0 0x7ff9ec0ada80 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:39.275 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.272+0000 7ffa08c74640 1 --2- 172.21.15.26:0/2097732104 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffa04103640 0x7ffa041986a0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:39.275 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.272+0000 7ffa08c74640 1 --2- 172.21.15.26:0/2097732104 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffa0410c0f0 0x7ffa0419ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:39.275 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.272+0000 7ffa08c74640 1 --2- 172.21.15.26:0/2097732104 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffa04102d20 0x7ffa04198160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:39.275 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.272+0000 7ffa08c74640 1 -- 172.21.15.26:0/2097732104 >> 172.21.15.26:0/2097732104 conn(0x7ffa04077990 msgr2=0x7ffa040fcd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:21:39.275 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.272+0000 7ffa08c74640 1 -- 172.21.15.26:0/2097732104 shutdown_connections 2024-09-11T23:21:39.276 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:39.272+0000 7ffa08c74640 1 -- 172.21.15.26:0/2097732104 wait complete. 2024-09-11T23:21:39.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:21:39 smithi136 ceph-mon[35121]: pgmap v101: 97 pgs: 97 active+clean; 460 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.4 MiB/s wr, 389 op/s 2024-09-11T23:21:39.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:21:39 smithi026 ceph-mon[31172]: pgmap v101: 97 pgs: 97 active+clean; 460 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.4 MiB/s wr, 389 op/s 2024-09-11T23:21:39.802 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:21:39 smithi076 ceph-mon[35296]: pgmap v101: 97 pgs: 97 active+clean; 460 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.4 MiB/s wr, 389 op/s 2024-09-11T23:21:41.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:21:41 smithi136 ceph-mon[35121]: pgmap v102: 97 pgs: 97 active+clean; 460 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.9 MiB/s wr, 298 op/s 2024-09-11T23:21:41.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:21:41 smithi026 ceph-mon[31172]: pgmap v102: 97 pgs: 97 active+clean; 460 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.9 MiB/s wr, 298 op/s 2024-09-11T23:21:41.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:21:41 smithi076 ceph-mon[35296]: pgmap v102: 97 pgs: 97 active+clean; 460 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.9 MiB/s wr, 298 op/s 2024-09-11T23:21:43.577 INFO:tasks.workunit.client.0.smithi026.stdout: 146 1667069 1625 1028642 1420 420057 290 2024-09-11T23:21:43.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:21:43 smithi136 ceph-mon[35121]: pgmap v103: 97 pgs: 97 active+clean; 460 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.9 MiB/s wr, 298 op/s 2024-09-11T23:21:43.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:21:43 smithi026 ceph-mon[31172]: pgmap v103: 97 pgs: 97 active+clean; 460 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.9 MiB/s wr, 298 op/s 2024-09-11T23:21:43.802 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:21:43 smithi076 ceph-mon[35296]: pgmap v103: 97 pgs: 97 active+clean; 460 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.9 MiB/s wr, 298 op/s 2024-09-11T23:21:45.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:21:45 smithi136 ceph-mon[35121]: pgmap v104: 97 pgs: 97 active+clean; 475 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 6.4 MiB/s wr, 483 op/s 2024-09-11T23:21:45.750 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:21:45 smithi026 ceph-mon[31172]: pgmap v104: 97 pgs: 97 active+clean; 475 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 6.4 MiB/s wr, 483 op/s 2024-09-11T23:21:45.802 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:21:45 smithi076 ceph-mon[35296]: pgmap v104: 97 pgs: 97 active+clean; 475 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 6.4 MiB/s wr, 483 op/s 2024-09-11T23:21:47.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:21:47 smithi136 ceph-mon[35121]: pgmap v105: 97 pgs: 97 active+clean; 475 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.9 MiB/s wr, 431 op/s 2024-09-11T23:21:47.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:21:47 smithi026 ceph-mon[31172]: pgmap v105: 97 pgs: 97 active+clean; 475 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.9 MiB/s wr, 431 op/s 2024-09-11T23:21:47.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:21:47 smithi076 ceph-mon[35296]: pgmap v105: 97 pgs: 97 active+clean; 475 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.9 MiB/s wr, 431 op/s 2024-09-11T23:21:49.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:21:49 smithi136 ceph-mon[35121]: pgmap v106: 97 pgs: 97 active+clean; 489 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 7.3 MiB/s wr, 513 op/s 2024-09-11T23:21:49.750 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:21:49 smithi026 ceph-mon[31172]: pgmap v106: 97 pgs: 97 active+clean; 489 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 7.3 MiB/s wr, 513 op/s 2024-09-11T23:21:49.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:21:49 smithi076 ceph-mon[35296]: pgmap v106: 97 pgs: 97 active+clean; 489 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 7.3 MiB/s wr, 513 op/s 2024-09-11T23:21:51.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:21:51 smithi136 ceph-mon[35121]: pgmap v107: 97 pgs: 97 active+clean; 489 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.8 MiB/s wr, 266 op/s 2024-09-11T23:21:51.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:21:51 smithi026 ceph-mon[31172]: pgmap v107: 97 pgs: 97 active+clean; 489 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.8 MiB/s wr, 266 op/s 2024-09-11T23:21:51.802 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:21:51 smithi076 ceph-mon[35296]: pgmap v107: 97 pgs: 97 active+clean; 489 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.8 MiB/s wr, 266 op/s 2024-09-11T23:21:53.577 INFO:tasks.workunit.client.0.smithi026.stdout: 187 2335461 2213 1505936 2293 482622 109 2024-09-11T23:21:53.699 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:53.699+0000 7f3e3ffff640 1 -- 172.21.15.26:0/2884722780 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3e401095d0 msgr2=0x7f3e40109a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:53.699 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:53.699+0000 7f3e3ffff640 1 --2- 172.21.15.26:0/2884722780 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3e401095d0 0x7f3e40109a30 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f3e38009ca0 tx=0x7f3e3802a8b0 comp rx=0 tx=0).stop 2024-09-11T23:21:53.699 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:53.699+0000 7f3e3ffff640 1 -- 172.21.15.26:0/2884722780 shutdown_connections 2024-09-11T23:21:53.699 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:53.699+0000 7f3e3ffff640 1 --2- 172.21.15.26:0/2884722780 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3e4011b140 0x7f3e4011d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:53.699 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:53.699+0000 7f3e3ffff640 1 --2- 172.21.15.26:0/2884722780 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3e401095d0 0x7f3e40109a30 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:53.700 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:53.699+0000 7f3e3ffff640 1 --2- 172.21.15.26:0/2884722780 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3e401091f0 0x7f3e40110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:53.700 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:53.699+0000 7f3e3ffff640 1 -- 172.21.15.26:0/2884722780 >> 172.21.15.26:0/2884722780 conn(0x7f3e4006cbf0 msgr2=0x7f3e4006d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:21:53.700 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:53.699+0000 7f3e3ffff640 1 -- 172.21.15.26:0/2884722780 shutdown_connections 2024-09-11T23:21:53.700 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:53.699+0000 7f3e3ffff640 1 -- 172.21.15.26:0/2884722780 wait complete. 2024-09-11T23:21:53.701 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:53.701+0000 7f3e3ffff640 1 Processor -- start 2024-09-11T23:21:53.701 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:53.701+0000 7f3e3ffff640 1 -- start start 2024-09-11T23:21:53.702 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:53.702+0000 7f3e3ffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3e401091f0 0x7f3e4011a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:53.702 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:53.702+0000 7f3e3ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3e4011b140 0x7f3e4011aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:53.702 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:53.702+0000 7f3e3ffff640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3e401135b0 0x7f3e40113a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:53.702 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:53.702+0000 7f3e3ffff640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f3e40120b50 con 0x7f3e4011b140 2024-09-11T23:21:53.702 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:53.702+0000 7f3e3ffff640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f3e401209d0 con 0x7f3e401091f0 2024-09-11T23:21:53.702 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:53.702+0000 7f3e3ffff640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f3e40120cd0 con 0x7f3e401135b0 2024-09-11T23:21:53.703 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:53.702+0000 7f3e3effd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3e401091f0 0x7f3e4011a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:53.703 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:53.702+0000 7f3e3effd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3e401091f0 0x7f3e4011a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:39862/0 (socket says 172.21.15.26:39862) 2024-09-11T23:21:53.703 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:53.702+0000 7f3e3effd640 1 -- 172.21.15.26:0/100453806 learned_addr learned my addr 172.21.15.26:0/100453806 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:21:53.703 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:53.702+0000 7f3e3e7fc640 1 --2- 172.21.15.26:0/100453806 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3e4011b140 0x7f3e4011aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:53.703 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:53.703+0000 7f3e3effd640 1 -- 172.21.15.26:0/100453806 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3e401135b0 msgr2=0x7f3e40113a60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-11T23:21:53.703 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:53.703+0000 7f3e3effd640 1 --2- 172.21.15.26:0/100453806 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3e401135b0 0x7f3e40113a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:53.703 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:53.703+0000 7f3e3effd640 1 -- 172.21.15.26:0/100453806 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3e4011b140 msgr2=0x7f3e4011aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:53.703 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:53.703+0000 7f3e3effd640 1 --2- 172.21.15.26:0/100453806 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3e4011b140 0x7f3e4011aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:53.703 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:53.703+0000 7f3e3effd640 1 -- 172.21.15.26:0/100453806 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e40114320 con 0x7f3e401091f0 2024-09-11T23:21:53.704 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:53.703+0000 7f3e3e7fc640 1 --2- 172.21.15.26:0/100453806 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3e4011b140 0x7f3e4011aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:21:53.704 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:53.703+0000 7f3e3effd640 1 --2- 172.21.15.26:0/100453806 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3e401091f0 0x7f3e4011a4c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f3e3000e9f0 tx=0x7f3e3000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:21:53.715 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:53.715+0000 7f3e2ffff640 1 -- 172.21.15.26:0/100453806 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e3000cd80 con 0x7f3e401091f0 2024-09-11T23:21:53.718 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:53.716+0000 7f3e3ffff640 1 -- 172.21.15.26:0/100453806 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e40118050 con 0x7f3e401091f0 2024-09-11T23:21:53.718 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:53.716+0000 7f3e3ffff640 1 -- 172.21.15.26:0/100453806 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e401185a0 con 0x7f3e401091f0 2024-09-11T23:21:53.721 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:53.721+0000 7f3e48478740 1 Processor -- start 2024-09-11T23:21:53.721 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:53.721+0000 7f3e2ffff640 1 -- 172.21.15.26:0/100453806 <== mon.2 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e30010910 con 0x7f3e401091f0 2024-09-11T23:21:53.721 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:53.721+0000 7f3e2ffff640 1 -- 172.21.15.26:0/100453806 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e30018980 con 0x7f3e401091f0 2024-09-11T23:21:53.722 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:53.722+0000 7f3e48478740 1 -- start start 2024-09-11T23:21:53.725 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:53.724+0000 7f3e48478740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560aa1af1cc0 0x560aa1af20a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:53.725 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:53.724+0000 7f3e48478740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560aa1af25e0 0x560aa1afc940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:53.725 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:53.724+0000 7f3e48478740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560aa1afce80 0x560aa1aff270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:53.725 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:53.724+0000 7f3e48478740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x560aa1a562f0 con 0x560aa1af1cc0 2024-09-11T23:21:53.725 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:53.724+0000 7f3e48478740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x560aa19fe270 con 0x560aa1af25e0 2024-09-11T23:21:53.725 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:53.724+0000 7f3e48478740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x560aa1a522b0 con 0x560aa1afce80 2024-09-11T23:21:53.725 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:53.724+0000 7f3e3e7fc640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560aa1afce80 0x560aa1aff270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:53.725 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:53.724+0000 7f3e3e7fc640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560aa1afce80 0x560aa1aff270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:51324/0 (socket says 172.21.15.26:51324) 2024-09-11T23:21:53.725 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:53.724+0000 7f3e3e7fc640 1 -- 172.21.15.26:0/2330853649 learned_addr learned my addr 172.21.15.26:0/2330853649 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:21:53.725 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:53.725+0000 7f3e3effd640 1 --2- 172.21.15.26:0/2330853649 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560aa1af25e0 0x560aa1afc940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:53.725 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:53.725+0000 7f3e3e7fc640 1 -- 172.21.15.26:0/2330853649 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560aa1af25e0 msgr2=0x560aa1afc940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:21:53.725 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:53.725+0000 7f3e3e7fc640 1 --2- 172.21.15.26:0/2330853649 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560aa1af25e0 0x560aa1afc940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:53.726 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:53.725+0000 7f3e3e7fc640 1 -- 172.21.15.26:0/2330853649 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560aa1af1cc0 msgr2=0x560aa1af20a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T23:21:53.726 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:53.725+0000 7f3e3f7fe640 1 --2- 172.21.15.26:0/2330853649 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560aa1af1cc0 0x560aa1af20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:53.726 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:53.725+0000 7f3e3e7fc640 1 --2- 172.21.15.26:0/2330853649 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560aa1af1cc0 0x560aa1af20a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:53.726 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:53.725+0000 7f3e3e7fc640 1 -- 172.21.15.26:0/2330853649 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560aa19fd340 con 0x560aa1afce80 2024-09-11T23:21:53.726 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:53.726+0000 7f3e3f7fe640 1 --2- 172.21.15.26:0/2330853649 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560aa1af1cc0 0x560aa1af20a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:21:53.726 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:53.726+0000 7f3e2ffff640 1 -- 172.21.15.26:0/100453806 <== mon.2 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f3e30018be0 con 0x7f3e401091f0 2024-09-11T23:21:53.726 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:53.726+0000 7f3e3e7fc640 1 --2- 172.21.15.26:0/2330853649 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560aa1afce80 0x560aa1aff270 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f3e3802ae20 tx=0x7f3e380304b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=77a4e549f700e85c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:21:53.729 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:53.729+0000 7f3e2ffff640 1 --2- 172.21.15.26:0/100453806 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f3e280a2f00 0x7f3e280a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:53.730 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:53.730+0000 7f3e3f7fe640 1 --2- 172.21.15.26:0/100453806 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f3e280a2f00 0x7f3e280a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:53.731 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:53.731+0000 7f3e3f7fe640 1 --2- 172.21.15.26:0/100453806 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f3e280a2f00 0x7f3e280a53c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f3e3400a860 tx=0x7f3e34008040 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:21:53.738 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:21:53 smithi136 ceph-mon[35121]: pgmap v108: 97 pgs: 97 active+clean; 489 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.8 MiB/s wr, 266 op/s 2024-09-11T23:21:53.739 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:53.740+0000 7f3e2ffff640 1 -- 172.21.15.26:0/100453806 <== mon.2 v2:172.21.15.76:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f3e300d0880 con 0x7f3e401091f0 2024-09-11T23:21:53.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:21:53 smithi026 ceph-mon[31172]: pgmap v108: 97 pgs: 97 active+clean; 489 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.8 MiB/s wr, 266 op/s 2024-09-11T23:21:53.768 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:53.768+0000 7f3e2d7fa640 1 -- 172.21.15.26:0/2330853649 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e38003970 con 0x560aa1afce80 2024-09-11T23:21:53.768 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:53.768+0000 7f3e2d7fa640 1 -- 172.21.15.26:0/2330853649 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e38036040 con 0x560aa1afce80 2024-09-11T23:21:53.770 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:53.771+0000 7f3e48478740 1 -- 172.21.15.26:0/2330853649 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560aa1afce80 msgr2=0x560aa1aff270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:53.771 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:53.771+0000 7f3e48478740 1 --2- 172.21.15.26:0/2330853649 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560aa1afce80 0x560aa1aff270 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f3e3802ae20 tx=0x7f3e380304b0 comp rx=0 tx=0).stop 2024-09-11T23:21:53.771 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:53.771+0000 7f3e48478740 1 -- 172.21.15.26:0/2330853649 shutdown_connections 2024-09-11T23:21:53.771 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:53.771+0000 7f3e48478740 1 --2- 172.21.15.26:0/2330853649 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560aa1af25e0 0x560aa1afc940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:53.771 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:53.771+0000 7f3e48478740 1 --2- 172.21.15.26:0/2330853649 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560aa1afce80 0x560aa1aff270 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:53.771 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:53.771+0000 7f3e48478740 1 --2- 172.21.15.26:0/2330853649 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560aa1af1cc0 0x560aa1af20a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:53.771 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:53.771+0000 7f3e48478740 1 -- 172.21.15.26:0/2330853649 >> 172.21.15.26:0/2330853649 conn(0x560aa1aee230 msgr2=0x560aa1aee610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:21:53.772 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:53.771+0000 7f3e48478740 1 -- 172.21.15.26:0/2330853649 shutdown_connections 2024-09-11T23:21:53.772 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:53.772+0000 7f3e48478740 1 -- 172.21.15.26:0/2330853649 wait complete. 2024-09-11T23:21:53.773 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:53.773+0000 7f3e48478740 1 -- start start 2024-09-11T23:21:53.773 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:53.773+0000 7f3e48478740 1 Processor -- start 2024-09-11T23:21:53.775 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:53.774+0000 7f3e48478740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560aa1af1cc0 0x560aa1aff1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:53.775 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:53.774+0000 7f3e48478740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560aa1af25e0 0x560aa1afc440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:53.775 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:53.774+0000 7f3e48478740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560aa1afc9d0 0x560aa1afce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:53.776 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:53.774+0000 7f3e48478740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x560aa1affaa0 con 0x560aa1afc9d0 2024-09-11T23:21:53.776 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:53.774+0000 7f3e48478740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x560aa19df290 con 0x560aa1af25e0 2024-09-11T23:21:53.776 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:53.774+0000 7f3e48478740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x560aa1affc20 con 0x560aa1af1cc0 2024-09-11T23:21:53.776 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:53.774+0000 7f3e3effd640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560aa1af1cc0 0x560aa1aff1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:53.776 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:53.775+0000 7f3e3effd640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560aa1af1cc0 0x560aa1aff1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:51328/0 (socket says 172.21.15.26:51328) 2024-09-11T23:21:53.776 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:53.775+0000 7f3e3e7fc640 1 --2- 172.21.15.26:0/85957848 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560aa1af25e0 0x560aa1afc440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:53.776 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:53.775+0000 7f3e3effd640 1 -- 172.21.15.26:0/85957848 learned_addr learned my addr 172.21.15.26:0/85957848 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:21:53.776 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:53.775+0000 7f3e3effd640 1 -- 172.21.15.26:0/85957848 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560aa1af25e0 msgr2=0x560aa1afc440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:21:53.776 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:53.775+0000 7f3e3effd640 1 --2- 172.21.15.26:0/85957848 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560aa1af25e0 0x560aa1afc440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:53.776 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:53.775+0000 7f3e3effd640 1 -- 172.21.15.26:0/85957848 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560aa1afc9d0 msgr2=0x560aa1afce10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T23:21:53.776 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:53.775+0000 7f3e3effd640 1 --2- 172.21.15.26:0/85957848 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560aa1afc9d0 0x560aa1afce10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:53.776 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:53.775+0000 7f3e3effd640 1 -- 172.21.15.26:0/85957848 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560aa1769210 con 0x560aa1af1cc0 2024-09-11T23:21:53.776 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:53.775+0000 7f3e3effd640 1 --2- 172.21.15.26:0/85957848 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560aa1af1cc0 0x560aa1aff1a0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f3e3009cce0 tx=0x7f3e3009cc10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6d43fc395fba0d72 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:21:53.777 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:53.776+0000 7f3e48478740 10 client.0 _ll_register_callbacks cb 0x560aa1a10140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:21:53.777 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:53.776+0000 7f3e48478740 1 -- 172.21.15.26:0/85957848 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x560aa1a63a20 con 0x560aa1af1cc0 2024-09-11T23:21:53.802 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:21:53 smithi076 ceph-mon[35296]: pgmap v108: 97 pgs: 97 active+clean; 489 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.8 MiB/s wr, 266 op/s 2024-09-11T23:21:53.823 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:53.823+0000 7f3e267fc640 10 client.25903 ms_handle_connect on v2:172.21.15.136:3300/0 2024-09-11T23:21:53.823 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:53.823+0000 7f3e267fc640 1 -- 172.21.15.26:0/85957848 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e300ce410 con 0x560aa1af1cc0 2024-09-11T23:21:53.823 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:53.823+0000 7f3e267fc640 1 -- 172.21.15.26:0/85957848 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f3e3009c3a0 con 0x560aa1af1cc0 2024-09-11T23:21:53.823 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:53.823+0000 7f3e267fc640 1 -- 172.21.15.26:0/85957848 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f3e3009cd50 con 0x560aa1af1cc0 2024-09-11T23:21:53.823 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:53.823+0000 7f3e48478740 10 client.25903 fetch_fsmap learned FSMap version 33 2024-09-11T23:21:53.823 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:53.823+0000 7f3e48478740 1 -- 172.21.15.26:0/85957848 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({fsmap=33}) -- 0x560aa1835620 con 0x560aa1af1cc0 2024-09-11T23:21:53.824 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:53.824+0000 7f3e267fc640 1 -- 172.21.15.26:0/85957848 <== mon.1 v2:172.21.15.136:3300/0 4 ==== fsmap(e 33) ==== 8394+0+0 (secure 0 0 0) 0x7f3e30010430 con 0x560aa1af1cc0 2024-09-11T23:21:53.824 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:53.824+0000 7f3e48478740 10 client.25903 fetch_fsmap finished waiting for FSMap version 33 2024-09-11T23:21:53.824 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:53.824+0000 7f3e48478740 10 client.25903 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:21:53.824 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:53.824+0000 7f3e48478740 20 client.25903 populate_metadata read hostname 'smithi026' 2024-09-11T23:21:53.824 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:53.824+0000 7f3e48478740 1 --2- 172.21.15.26:0/85957848 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x560aa1b0d410 0x560aa1b0f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:53.825 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:53.824+0000 7f3e48478740 4 client.25903 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:21:53.825 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:53.824+0000 7f3e48478740 1 -- 172.21.15.26:0/85957848 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560aa1a5dd70 con 0x560aa1b0d410 2024-09-11T23:21:53.825 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:53.824+0000 7f3e3e7fc640 1 --2- 172.21.15.26:0/85957848 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x560aa1b0d410 0x560aa1b0f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:53.825 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:53.825+0000 7f3e3e7fc640 1 --2- 172.21.15.26:0/85957848 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x560aa1b0d410 0x560aa1b0f8d0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c13c323fa939bec server_cookie=105f92d623c4464e in_seq=0 out_seq=0 2024-09-11T23:21:53.825 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:53.825+0000 7f3e267fc640 10 client.25903 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:21:53.826 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:53.826+0000 7f3e267fc640 1 -- 172.21.15.26:0/85957848 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560aa1a5dd70 con 0x560aa1b0d410 2024-09-11T23:21:53.826 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:53.826+0000 7f3e267fc640 10 client.25903 handle_command_reply: tid=0 2024-09-11T23:21:53.826 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:53.826+0000 7f3e48478740 1 client.25903 shutdown 2024-09-11T23:21:53.829 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:53.829+0000 7f3e48478740 1 -- 172.21.15.26:0/85957848 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560aa1af1cc0 msgr2=0x560aa1aff1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:53.829 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:53.829+0000 7f3e48478740 1 --2- 172.21.15.26:0/85957848 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560aa1af1cc0 0x560aa1aff1a0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f3e3009cce0 tx=0x7f3e3009cc10 comp rx=0 tx=0).stop 2024-09-11T23:21:53.829 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:53.829+0000 7f3e48478740 1 -- 172.21.15.26:0/85957848 shutdown_connections 2024-09-11T23:21:53.829 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:53.829+0000 7f3e48478740 1 --2- 172.21.15.26:0/85957848 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560aa1af25e0 0x560aa1afc440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:53.829 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:53.829+0000 7f3e48478740 1 --2- 172.21.15.26:0/85957848 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x560aa1b0d410 0x560aa1b0f8d0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:53.829 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:53.829+0000 7f3e48478740 1 --2- 172.21.15.26:0/85957848 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560aa1afc9d0 0x560aa1afce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:53.829 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:53.829+0000 7f3e48478740 1 --2- 172.21.15.26:0/85957848 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560aa1af1cc0 0x560aa1aff1a0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:53.829 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:53.829+0000 7f3e48478740 1 -- 172.21.15.26:0/85957848 >> 172.21.15.26:0/85957848 conn(0x560aa1aee230 msgr2=0x560aa1af3f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:21:53.830 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:53.829+0000 7f3e267fc640 0 client.25903 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:21:53.830 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:53.829+0000 7f3e48478740 1 -- 172.21.15.26:0/85957848 shutdown_connections 2024-09-11T23:21:53.830 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:53.830+0000 7f3e48478740 1 -- 172.21.15.26:0/85957848 wait complete. 2024-09-11T23:21:53.830 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:53.830+0000 7f3e48478740 20 client.25903 trim_cache size 0 max 16384 2024-09-11T23:21:54.115 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:54.115+0000 7f3e48478740 1 Processor -- start 2024-09-11T23:21:54.115 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:54.115+0000 7f3e48478740 1 -- start start 2024-09-11T23:21:54.118 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:54.118+0000 7f3e48478740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560aa1af1cc0 0x560aa1aed150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:54.119 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:54.119+0000 7f3e48478740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560aa1af25e0 0x560aa1b151e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:54.119 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:54.119+0000 7f3e3effd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560aa1af1cc0 0x560aa1aed150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:54.119 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:54.119+0000 7f3e3effd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560aa1af1cc0 0x560aa1aed150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:39874/0 (socket says 172.21.15.26:39874) 2024-09-11T23:21:54.120 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:54.120+0000 7f3e3e7fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560aa1af25e0 0x560aa1b151e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:54.120 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:54.120+0000 7f3e3e7fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560aa1af25e0 0x560aa1b151e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38332/0 (socket says 172.21.15.26:38332) 2024-09-11T23:21:54.120 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:54.121+0000 7f3e3effd640 1 -- 172.21.15.26:0/1407042201 learned_addr learned my addr 172.21.15.26:0/1407042201 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:21:54.121 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:54.121+0000 7f3e48478740 1 --2- 172.21.15.26:0/1407042201 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560aa1afc9d0 0x560aa1b15720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:54.122 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:54.121+0000 7f3e48478740 1 -- 172.21.15.26:0/1407042201 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f3e2000e850 con 0x560aa1af25e0 2024-09-11T23:21:54.122 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:54.121+0000 7f3e3f7fe640 1 --2- 172.21.15.26:0/1407042201 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560aa1afc9d0 0x560aa1b15720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:54.122 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:54.122+0000 7f3e48478740 1 -- 172.21.15.26:0/1407042201 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f3e20010510 con 0x560aa1af1cc0 2024-09-11T23:21:54.122 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:54.123+0000 7f3e48478740 1 -- 172.21.15.26:0/1407042201 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f3e2000ef20 con 0x560aa1afc9d0 2024-09-11T23:21:54.124 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:54.123+0000 7f3e3effd640 1 -- 172.21.15.26:0/1407042201 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560aa1afc9d0 msgr2=0x560aa1b15720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:21:54.124 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:54.123+0000 7f3e3effd640 1 --2- 172.21.15.26:0/1407042201 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560aa1afc9d0 0x560aa1b15720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:54.124 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:54.123+0000 7f3e3effd640 1 -- 172.21.15.26:0/1407042201 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560aa1af25e0 msgr2=0x560aa1b151e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:21:54.124 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:54.124+0000 7f3e3effd640 1 --2- 172.21.15.26:0/1407042201 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560aa1af25e0 0x560aa1b151e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:54.124 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:54.124+0000 7f3e3effd640 1 -- 172.21.15.26:0/1407042201 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560aa1af2410 con 0x560aa1af1cc0 2024-09-11T23:21:54.124 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:54.124+0000 7f3e3e7fc640 1 --2- 172.21.15.26:0/1407042201 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560aa1af25e0 0x560aa1b151e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:21:54.124 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:54.124+0000 7f3e3effd640 1 --2- 172.21.15.26:0/1407042201 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560aa1af1cc0 0x560aa1aed150 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f3e300991b0 tx=0x7f3e30019000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=133cdffeb6389219 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:21:54.124 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:54.124+0000 7f3e3f7fe640 1 --2- 172.21.15.26:0/1407042201 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560aa1afc9d0 0x560aa1b15720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:21:54.134 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:54.134+0000 7f3e2cff9640 1 -- 172.21.15.26:0/1407042201 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e300a1150 con 0x560aa1af1cc0 2024-09-11T23:21:54.134 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:54.134+0000 7f3e2cff9640 1 -- 172.21.15.26:0/1407042201 <== mon.2 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x560aa1a63a20 con 0x560aa1af1cc0 2024-09-11T23:21:54.134 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:54.134+0000 7f3e2cff9640 1 -- 172.21.15.26:0/1407042201 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e3009c400 con 0x560aa1af1cc0 2024-09-11T23:21:54.138 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:54.138+0000 7f3e48478740 1 -- 172.21.15.26:0/1407042201 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560aa1af1cc0 msgr2=0x560aa1aed150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:54.138 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:54.138+0000 7f3e48478740 1 --2- 172.21.15.26:0/1407042201 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560aa1af1cc0 0x560aa1aed150 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f3e300991b0 tx=0x7f3e30019000 comp rx=0 tx=0).stop 2024-09-11T23:21:54.139 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:54.138+0000 7f3e48478740 1 -- 172.21.15.26:0/1407042201 shutdown_connections 2024-09-11T23:21:54.139 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:54.139+0000 7f3e48478740 1 --2- 172.21.15.26:0/1407042201 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560aa1afc9d0 0x560aa1b15720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:54.139 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:54.139+0000 7f3e48478740 1 --2- 172.21.15.26:0/1407042201 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560aa1af25e0 0x560aa1b151e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:54.139 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:54.139+0000 7f3e48478740 1 --2- 172.21.15.26:0/1407042201 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560aa1af1cc0 0x560aa1aed150 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:54.139 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:54.139+0000 7f3e48478740 1 -- 172.21.15.26:0/1407042201 >> 172.21.15.26:0/1407042201 conn(0x560aa1aee230 msgr2=0x560aa1aeca40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:21:54.144 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:54.144+0000 7f3e48478740 1 -- 172.21.15.26:0/1407042201 shutdown_connections 2024-09-11T23:21:54.144 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:54.144+0000 7f3e48478740 1 -- 172.21.15.26:0/1407042201 wait complete. 2024-09-11T23:21:54.145 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:54.145+0000 7f3e48478740 1 -- start start 2024-09-11T23:21:54.145 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:54.145+0000 7f3e48478740 1 Processor -- start 2024-09-11T23:21:54.154 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:54.154+0000 7f3e48478740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560aa1af1cc0 0x560aa1b05280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:54.155 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:54.155+0000 7f3e3effd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560aa1af1cc0 0x560aa1b05280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:54.155 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:54.155+0000 7f3e3effd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560aa1af1cc0 0x560aa1b05280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:39884/0 (socket says 172.21.15.26:39884) 2024-09-11T23:21:54.155 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:54.155+0000 7f3e48478740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560aa1af25e0 0x560aa1b057c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:54.156 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:54.155+0000 7f3e3effd640 1 -- 172.21.15.26:0/434858993 learned_addr learned my addr 172.21.15.26:0/434858993 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:21:54.156 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:54.156+0000 7f3e48478740 1 --2- 172.21.15.26:0/434858993 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560aa1afc9d0 0x560aa1af8e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:54.156 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:54.156+0000 7f3e3e7fc640 1 --2- 172.21.15.26:0/434858993 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560aa1af25e0 0x560aa1b057c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:54.156 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:54.156+0000 7f3e48478740 1 -- 172.21.15.26:0/434858993 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f3e2000fd80 con 0x560aa1af25e0 2024-09-11T23:21:54.156 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:54.156+0000 7f3e48478740 1 -- 172.21.15.26:0/434858993 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f3e2000f5f0 con 0x560aa1af1cc0 2024-09-11T23:21:54.156 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:54.156+0000 7f3e48478740 1 -- 172.21.15.26:0/434858993 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f3e2000c830 con 0x560aa1afc9d0 2024-09-11T23:21:54.157 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:54.156+0000 7f3e3f7fe640 1 --2- 172.21.15.26:0/434858993 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560aa1afc9d0 0x560aa1af8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:54.157 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:54.156+0000 7f3e48478740 10 client.0 _ll_register_callbacks cb 0x7f3e300990e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:21:54.157 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:54.156+0000 7f3e3f7fe640 1 -- 172.21.15.26:0/434858993 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560aa1af1cc0 msgr2=0x560aa1b05280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:21:54.157 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:54.156+0000 7f3e3f7fe640 1 --2- 172.21.15.26:0/434858993 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560aa1af1cc0 0x560aa1b05280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:54.157 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:54.156+0000 7f3e3f7fe640 1 -- 172.21.15.26:0/434858993 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560aa1af25e0 msgr2=0x560aa1b057c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:21:54.157 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:54.156+0000 7f3e3f7fe640 1 --2- 172.21.15.26:0/434858993 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560aa1af25e0 0x560aa1b057c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:54.157 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:54.156+0000 7f3e3f7fe640 1 -- 172.21.15.26:0/434858993 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560aa1af9680 con 0x560aa1afc9d0 2024-09-11T23:21:54.157 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:54.157+0000 7f3e3e7fc640 1 --2- 172.21.15.26:0/434858993 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560aa1af25e0 0x560aa1b057c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:21:54.157 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:54.157+0000 7f3e3effd640 1 --2- 172.21.15.26:0/434858993 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560aa1af1cc0 0x560aa1b05280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:21:54.157 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:54.157+0000 7f3e3f7fe640 1 --2- 172.21.15.26:0/434858993 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560aa1afc9d0 0x560aa1af8e50 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f3e34006fd0 tx=0x7f3e340046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=16fa7ef8d3627833 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:21:54.162 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:54.162+0000 7f3e48478740 1 -- 172.21.15.26:0/434858993 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x560aa1a63a20 con 0x560aa1afc9d0 2024-09-11T23:21:54.185 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:54.184+0000 7f3e267fc640 10 client.25906 ms_handle_connect on v2:172.21.15.136:3300/0 2024-09-11T23:21:54.185 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:54.184+0000 7f3e267fc640 1 -- 172.21.15.26:0/434858993 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e3401e610 con 0x560aa1afc9d0 2024-09-11T23:21:54.185 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:54.184+0000 7f3e267fc640 1 -- 172.21.15.26:0/434858993 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f3e3401e7b0 con 0x560aa1afc9d0 2024-09-11T23:21:54.185 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:54.185+0000 7f3e267fc640 1 -- 172.21.15.26:0/434858993 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f3e3402c050 con 0x560aa1afc9d0 2024-09-11T23:21:54.185 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:54.185+0000 7f3e48478740 10 client.25906 fetch_fsmap learned FSMap version 33 2024-09-11T23:21:54.185 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:54.185+0000 7f3e48478740 1 -- 172.21.15.26:0/434858993 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({fsmap=33}) -- 0x560aa1b77ca0 con 0x560aa1afc9d0 2024-09-11T23:21:54.196 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:54.196+0000 7f3e267fc640 1 -- 172.21.15.26:0/434858993 <== mon.1 v2:172.21.15.136:3300/0 4 ==== fsmap(e 33) ==== 8394+0+0 (secure 0 0 0) 0x7f3e3401e9f0 con 0x560aa1afc9d0 2024-09-11T23:21:54.196 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:54.196+0000 7f3e48478740 10 client.25906 fetch_fsmap finished waiting for FSMap version 33 2024-09-11T23:21:54.197 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:54.196+0000 7f3e48478740 10 client.25906 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:21:54.197 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:54.196+0000 7f3e48478740 20 client.25906 populate_metadata read hostname 'smithi026' 2024-09-11T23:21:54.197 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:54.196+0000 7f3e48478740 1 --2- 172.21.15.26:0/434858993 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x560aa1b78350 0x560aa1b7a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:54.197 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:54.196+0000 7f3e48478740 4 client.25906 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T23:21:54.197 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:54.196+0000 7f3e48478740 1 -- 172.21.15.26:0/434858993 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560aa19da900 con 0x560aa1b78350 2024-09-11T23:21:54.197 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:54.197+0000 7f3e3effd640 1 --2- 172.21.15.26:0/434858993 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x560aa1b78350 0x560aa1b7a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:54.197 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:54.197+0000 7f3e3effd640 1 --2- 172.21.15.26:0/434858993 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x560aa1b78350 0x560aa1b7a770 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b3474c904eccd85b server_cookie=8047887a2f863064 in_seq=0 out_seq=0 2024-09-11T23:21:54.198 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:54.197+0000 7f3e267fc640 10 client.25906 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:21:54.624 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:54.624+0000 7f3e267fc640 1 -- 172.21.15.26:0/434858993 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560aa19da900 con 0x560aa1b78350 2024-09-11T23:21:54.624 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:54.624+0000 7f3e267fc640 10 client.25906 handle_command_reply: tid=0 2024-09-11T23:21:54.625 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:54.624+0000 7f3e48478740 1 client.25906 shutdown 2024-09-11T23:21:54.625 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:54.624+0000 7f3e48478740 1 -- 172.21.15.26:0/434858993 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560aa1afc9d0 msgr2=0x560aa1af8e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:54.625 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:54.624+0000 7f3e48478740 1 --2- 172.21.15.26:0/434858993 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560aa1afc9d0 0x560aa1af8e50 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f3e34006fd0 tx=0x7f3e340046c0 comp rx=0 tx=0).stop 2024-09-11T23:21:54.625 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:54.624+0000 7f3e48478740 1 -- 172.21.15.26:0/434858993 shutdown_connections 2024-09-11T23:21:54.625 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:54.624+0000 7f3e48478740 1 --2- 172.21.15.26:0/434858993 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x560aa1b78350 0x560aa1b7a770 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:54.625 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:54.624+0000 7f3e48478740 1 --2- 172.21.15.26:0/434858993 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560aa1afc9d0 0x560aa1af8e50 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:54.625 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:54.624+0000 7f3e48478740 1 --2- 172.21.15.26:0/434858993 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560aa1af25e0 0x560aa1b057c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:54.625 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:54.624+0000 7f3e48478740 1 --2- 172.21.15.26:0/434858993 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560aa1af1cc0 0x560aa1b05280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:54.625 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:54.624+0000 7f3e48478740 1 -- 172.21.15.26:0/434858993 >> 172.21.15.26:0/434858993 conn(0x560aa1aee230 msgr2=0x560aa1af3f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:21:54.625 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:54.624+0000 7f3e267fc640 0 client.25906 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:21:54.625 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:54.624+0000 7f3e48478740 1 -- 172.21.15.26:0/434858993 shutdown_connections 2024-09-11T23:21:54.625 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:54.624+0000 7f3e48478740 1 -- 172.21.15.26:0/434858993 wait complete. 2024-09-11T23:21:54.626 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:54.624+0000 7f3e48478740 20 client.25906 trim_cache size 0 max 16384 2024-09-11T23:21:54.626 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-09-11T23:21:54.626 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "message": "", 2024-09-11T23:21:54.626 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-09-11T23:21:54.626 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-09-11T23:21:54.649 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:54.649+0000 7f3e3ffff640 1 -- 172.21.15.26:0/100453806 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f3e280a2f00 msgr2=0x7f3e280a53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:54.649 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:54.649+0000 7f3e3ffff640 1 --2- 172.21.15.26:0/100453806 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f3e280a2f00 0x7f3e280a53c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f3e3400a860 tx=0x7f3e34008040 comp rx=0 tx=0).stop 2024-09-11T23:21:54.650 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:54.649+0000 7f3e3ffff640 1 -- 172.21.15.26:0/100453806 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3e401091f0 msgr2=0x7f3e4011a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:54.650 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:54.649+0000 7f3e3ffff640 1 --2- 172.21.15.26:0/100453806 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3e401091f0 0x7f3e4011a4c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f3e3000e9f0 tx=0x7f3e3000eeb0 comp rx=0 tx=0).stop 2024-09-11T23:21:54.650 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:54.649+0000 7f3e3ffff640 1 -- 172.21.15.26:0/100453806 shutdown_connections 2024-09-11T23:21:54.650 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:54.649+0000 7f3e3ffff640 1 --2- 172.21.15.26:0/100453806 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f3e280a2f00 0x7f3e280a53c0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:54.650 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:54.649+0000 7f3e3ffff640 1 --2- 172.21.15.26:0/100453806 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3e401135b0 0x7f3e40113a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:54.650 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:54.649+0000 7f3e3ffff640 1 --2- 172.21.15.26:0/100453806 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3e4011b140 0x7f3e4011aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:54.650 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:54.649+0000 7f3e3ffff640 1 --2- 172.21.15.26:0/100453806 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3e401091f0 0x7f3e4011a4c0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:54.650 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:54.649+0000 7f3e3ffff640 1 -- 172.21.15.26:0/100453806 >> 172.21.15.26:0/100453806 conn(0x7f3e4006cbf0 msgr2=0x7f3e400705a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:21:54.650 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:54.649+0000 7f3e3ffff640 1 -- 172.21.15.26:0/100453806 shutdown_connections 2024-09-11T23:21:54.650 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:21:54.649+0000 7f3e3ffff640 1 -- 172.21.15.26:0/100453806 wait complete. 2024-09-11T23:21:55.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:21:55 smithi136 ceph-mon[35121]: pgmap v109: 97 pgs: 97 active+clean; 584 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 11 MiB/s wr, 621 op/s 2024-09-11T23:21:55.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:21:55 smithi026 ceph-mon[31172]: pgmap v109: 97 pgs: 97 active+clean; 584 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 11 MiB/s wr, 621 op/s 2024-09-11T23:21:55.802 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:21:55 smithi076 ceph-mon[35296]: pgmap v109: 97 pgs: 97 active+clean; 584 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 11 MiB/s wr, 621 op/s 2024-09-11T23:21:57.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:21:57 smithi136 ceph-mon[35121]: pgmap v110: 97 pgs: 97 active+clean; 584 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 9.5 MiB/s wr, 436 op/s 2024-09-11T23:21:57.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:21:57 smithi026 ceph-mon[31172]: pgmap v110: 97 pgs: 97 active+clean; 584 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 9.5 MiB/s wr, 436 op/s 2024-09-11T23:21:57.802 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:21:57 smithi076 ceph-mon[35296]: pgmap v110: 97 pgs: 97 active+clean; 584 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 9.5 MiB/s wr, 436 op/s 2024-09-11T23:21:59.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:21:59 smithi136 ceph-mon[35121]: pgmap v111: 97 pgs: 97 active+clean; 672 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 18 MiB/s wr, 804 op/s 2024-09-11T23:21:59.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:21:59 smithi026 ceph-mon[31172]: pgmap v111: 97 pgs: 97 active+clean; 672 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 18 MiB/s wr, 804 op/s 2024-09-11T23:21:59.802 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:21:59 smithi076 ceph-mon[35296]: pgmap v111: 97 pgs: 97 active+clean; 672 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 18 MiB/s wr, 804 op/s 2024-09-11T23:22:01.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:22:01 smithi136 ceph-mon[35121]: pgmap v112: 97 pgs: 97 active+clean; 701 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 19 MiB/s wr, 837 op/s 2024-09-11T23:22:01.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:22:01 smithi026 ceph-mon[31172]: pgmap v112: 97 pgs: 97 active+clean; 701 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 19 MiB/s wr, 837 op/s 2024-09-11T23:22:01.802 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:22:01 smithi076 ceph-mon[35296]: pgmap v112: 97 pgs: 97 active+clean; 701 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 19 MiB/s wr, 837 op/s 2024-09-11T23:22:03.578 INFO:tasks.workunit.client.0.smithi026.stdout: 235 2159971 1942 1445032 2823 397732 1062 2024-09-11T23:22:03.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:22:03 smithi136 ceph-mon[35121]: pgmap v113: 97 pgs: 97 active+clean; 701 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 19 MiB/s wr, 837 op/s 2024-09-11T23:22:03.756 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:22:03 smithi026 ceph-mon[31172]: pgmap v113: 97 pgs: 97 active+clean; 701 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 19 MiB/s wr, 837 op/s 2024-09-11T23:22:03.802 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:22:03 smithi076 ceph-mon[35296]: pgmap v113: 97 pgs: 97 active+clean; 701 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 19 MiB/s wr, 837 op/s 2024-09-11T23:22:05.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:22:05 smithi136 ceph-mon[35121]: pgmap v114: 97 pgs: 97 active+clean; 732 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 22 MiB/s wr, 970 op/s 2024-09-11T23:22:05.750 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:22:05 smithi026 ceph-mon[31172]: pgmap v114: 97 pgs: 97 active+clean; 732 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 22 MiB/s wr, 970 op/s 2024-09-11T23:22:05.802 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:22:05 smithi076 ceph-mon[35296]: pgmap v114: 97 pgs: 97 active+clean; 732 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 22 MiB/s wr, 970 op/s 2024-09-11T23:22:07.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:22:07 smithi136 ceph-mon[35121]: pgmap v115: 97 pgs: 97 active+clean; 732 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 14 MiB/s wr, 616 op/s 2024-09-11T23:22:07.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:22:07 smithi026 ceph-mon[31172]: pgmap v115: 97 pgs: 97 active+clean; 732 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 14 MiB/s wr, 616 op/s 2024-09-11T23:22:07.802 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:22:07 smithi076 ceph-mon[35296]: pgmap v115: 97 pgs: 97 active+clean; 732 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 14 MiB/s wr, 616 op/s 2024-09-11T23:22:07.831 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.830+0000 7f20772a6640 1 -- 172.21.15.26:0/169066219 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f207010c0c0 msgr2=0x7f207010e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:07.831 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.830+0000 7f20772a6640 1 --2- 172.21.15.26:0/169066219 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f207010c0c0 0x7f207010e580 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f2068009be0 tx=0x7f206802a750 comp rx=0 tx=0).stop 2024-09-11T23:22:07.831 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.830+0000 7f20772a6640 1 -- 172.21.15.26:0/169066219 shutdown_connections 2024-09-11T23:22:07.831 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.830+0000 7f20772a6640 1 --2- 172.21.15.26:0/169066219 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f207010c0c0 0x7f207010e580 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:07.831 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.830+0000 7f20772a6640 1 --2- 172.21.15.26:0/169066219 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f20701035c0 0x7f207010baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:07.831 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.830+0000 7f20772a6640 1 --2- 172.21.15.26:0/169066219 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2070102ca0 0x7f2070103080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:07.831 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.830+0000 7f20772a6640 1 -- 172.21.15.26:0/169066219 >> 172.21.15.26:0/169066219 conn(0x7f20700fc6f0 msgr2=0x7f20700feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:22:07.831 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.831+0000 7f20772a6640 1 -- 172.21.15.26:0/169066219 shutdown_connections 2024-09-11T23:22:07.831 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.831+0000 7f20772a6640 1 -- 172.21.15.26:0/169066219 wait complete. 2024-09-11T23:22:07.833 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.831+0000 7f20772a6640 1 Processor -- start 2024-09-11T23:22:07.833 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.832+0000 7f20772a6640 1 -- start start 2024-09-11T23:22:07.834 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.832+0000 7f20772a6640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2070102ca0 0x7f2070198110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:07.834 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.832+0000 7f20772a6640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f20701035c0 0x7f2070198650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:07.834 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.832+0000 7f20772a6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f207010c0c0 0x7f207019c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:07.834 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.832+0000 7f20772a6640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f2070111ee0 con 0x7f207010c0c0 2024-09-11T23:22:07.834 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.832+0000 7f20772a6640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f2070111d60 con 0x7f2070102ca0 2024-09-11T23:22:07.834 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.832+0000 7f20772a6640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f2070112060 con 0x7f20701035c0 2024-09-11T23:22:07.834 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.832+0000 7f2075843640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f207010c0c0 0x7f207019c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:07.834 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.832+0000 7f2074841640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f20701035c0 0x7f2070198650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:07.834 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.832+0000 7f2075843640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f207010c0c0 0x7f207019c9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60178/0 (socket says 172.21.15.26:60178) 2024-09-11T23:22:07.834 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.832+0000 7f2075843640 1 -- 172.21.15.26:0/2380868792 learned_addr learned my addr 172.21.15.26:0/2380868792 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:22:07.834 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.832+0000 7f2075042640 1 --2- 172.21.15.26:0/2380868792 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2070102ca0 0x7f2070198110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:07.834 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.832+0000 7f2074841640 1 -- 172.21.15.26:0/2380868792 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2070102ca0 msgr2=0x7f2070198110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:07.835 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.832+0000 7f2074841640 1 --2- 172.21.15.26:0/2380868792 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2070102ca0 0x7f2070198110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:07.835 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.832+0000 7f2074841640 1 -- 172.21.15.26:0/2380868792 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f207010c0c0 msgr2=0x7f207019c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:07.835 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.832+0000 7f2074841640 1 --2- 172.21.15.26:0/2380868792 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f207010c0c0 0x7f207019c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:07.835 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.832+0000 7f2074841640 1 -- 172.21.15.26:0/2380868792 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f207019d100 con 0x7f20701035c0 2024-09-11T23:22:07.835 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.833+0000 7f2074841640 1 --2- 172.21.15.26:0/2380868792 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f20701035c0 0x7f2070198650 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f2060005fd0 tx=0x7f206000ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:22:07.835 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.833+0000 7f20667fc640 1 -- 172.21.15.26:0/2380868792 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f206000cd00 con 0x7f20701035c0 2024-09-11T23:22:07.835 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.833+0000 7f20667fc640 1 -- 172.21.15.26:0/2380868792 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20600035f0 con 0x7f20701035c0 2024-09-11T23:22:07.835 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.833+0000 7f20772a6640 1 -- 172.21.15.26:0/2380868792 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f207019d3f0 con 0x7f20701035c0 2024-09-11T23:22:07.835 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.833+0000 7f20667fc640 1 -- 172.21.15.26:0/2380868792 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2060010690 con 0x7f20701035c0 2024-09-11T23:22:07.835 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.833+0000 7f20772a6640 1 -- 172.21.15.26:0/2380868792 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20701a4c80 con 0x7f20701035c0 2024-09-11T23:22:07.837 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.836+0000 7f20667fc640 1 -- 172.21.15.26:0/2380868792 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f2060010830 con 0x7f20701035c0 2024-09-11T23:22:07.838 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.836+0000 7f207a039740 1 Processor -- start 2024-09-11T23:22:07.838 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.836+0000 7f207a039740 1 -- start start 2024-09-11T23:22:07.838 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.836+0000 7f207a039740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5557612a3af0 0x5557612a3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:07.838 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.836+0000 7f207a039740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5557612a4410 0x5557612ae770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:07.838 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.836+0000 7f207a039740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5557612aecb0 0x5557612b10a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:07.838 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.836+0000 7f207a039740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5557612082f0 con 0x5557612aecb0 2024-09-11T23:22:07.838 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.836+0000 7f207a039740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5557611b0270 con 0x5557612a3af0 2024-09-11T23:22:07.838 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.836+0000 7f207a039740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5557612042b0 con 0x5557612a4410 2024-09-11T23:22:07.838 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.837+0000 7f2075843640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5557612a3af0 0x5557612a3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:07.838 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.837+0000 7f2074841640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5557612aecb0 0x5557612b10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:07.838 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.837+0000 7f2075843640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5557612a3af0 0x5557612a3ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:43766/0 (socket says 172.21.15.26:43766) 2024-09-11T23:22:07.839 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.837+0000 7f2075843640 1 -- 172.21.15.26:0/4012119745 learned_addr learned my addr 172.21.15.26:0/4012119745 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:22:07.839 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.837+0000 7f2075042640 1 --2- 172.21.15.26:0/4012119745 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5557612a4410 0x5557612ae770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:07.839 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.837+0000 7f20667fc640 1 --2- 172.21.15.26:0/2380868792 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f205c0a2da0 0x7f205c0a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:07.839 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.837+0000 7f2074841640 1 -- 172.21.15.26:0/4012119745 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5557612a4410 msgr2=0x5557612ae770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:22:07.839 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.837+0000 7f2074841640 1 --2- 172.21.15.26:0/4012119745 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5557612a4410 0x5557612ae770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:07.839 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.837+0000 7f2074841640 1 -- 172.21.15.26:0/4012119745 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5557612a3af0 msgr2=0x5557612a3ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:22:07.839 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.837+0000 7f2074841640 1 --2- 172.21.15.26:0/4012119745 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5557612a3af0 0x5557612a3ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:07.839 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.837+0000 7f2074841640 1 -- 172.21.15.26:0/4012119745 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557611af340 con 0x5557612aecb0 2024-09-11T23:22:07.839 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.837+0000 7f2075042640 1 --2- 172.21.15.26:0/4012119745 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5557612a4410 0x5557612ae770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:22:07.839 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.837+0000 7f2075843640 1 --2- 172.21.15.26:0/4012119745 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5557612a3af0 0x5557612a3ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:22:07.839 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.837+0000 7f2075843640 1 --2- 172.21.15.26:0/2380868792 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f205c0a2da0 0x7f205c0a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:07.839 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.837+0000 7f20667fc640 1 -- 172.21.15.26:0/2380868792 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f20600cf5e0 con 0x7f20701035c0 2024-09-11T23:22:07.839 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.837+0000 7f2074841640 1 --2- 172.21.15.26:0/4012119745 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5557612aecb0 0x5557612b10a0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f20600ccd40 tx=0x7f2060021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43547cd6b24f252a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:22:07.840 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.838+0000 7f205b7fe640 1 -- 172.21.15.26:0/4012119745 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f206009b650 con 0x5557612aecb0 2024-09-11T23:22:07.840 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.838+0000 7f205b7fe640 1 -- 172.21.15.26:0/4012119745 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f206009c210 con 0x5557612aecb0 2024-09-11T23:22:07.840 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.838+0000 7f205b7fe640 1 -- 172.21.15.26:0/4012119745 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20600037a0 con 0x5557612aecb0 2024-09-11T23:22:07.840 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.838+0000 7f2075843640 1 --2- 172.21.15.26:0/2380868792 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f205c0a2da0 0x7f205c0a5260 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f206800b230 tx=0x7f2068005fd0 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:22:07.844 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.838+0000 7f207a039740 1 -- 172.21.15.26:0/4012119745 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5557612aecb0 msgr2=0x5557612b10a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:07.844 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.838+0000 7f207a039740 1 --2- 172.21.15.26:0/4012119745 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5557612aecb0 0x5557612b10a0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f20600ccd40 tx=0x7f2060021000 comp rx=0 tx=0).stop 2024-09-11T23:22:07.845 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.838+0000 7f207a039740 1 -- 172.21.15.26:0/4012119745 shutdown_connections 2024-09-11T23:22:07.845 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.838+0000 7f207a039740 1 --2- 172.21.15.26:0/4012119745 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5557612aecb0 0x5557612b10a0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:07.845 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.838+0000 7f207a039740 1 --2- 172.21.15.26:0/4012119745 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5557612a4410 0x5557612ae770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:07.845 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.839+0000 7f207a039740 1 --2- 172.21.15.26:0/4012119745 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5557612a3af0 0x5557612a3ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:07.845 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.839+0000 7f207a039740 1 -- 172.21.15.26:0/4012119745 >> 172.21.15.26:0/4012119745 conn(0x5557612a0060 msgr2=0x5557612a0440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:22:07.845 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.839+0000 7f207a039740 1 -- 172.21.15.26:0/4012119745 shutdown_connections 2024-09-11T23:22:07.845 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.839+0000 7f207a039740 1 -- 172.21.15.26:0/4012119745 wait complete. 2024-09-11T23:22:07.845 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.839+0000 7f207a039740 1 -- start start 2024-09-11T23:22:07.845 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.839+0000 7f207a039740 1 Processor -- start 2024-09-11T23:22:07.845 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.839+0000 7f207a039740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5557612a3af0 0x5557612abb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:07.845 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.839+0000 7f207a039740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5557612a4410 0x5557612ac0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:07.845 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.839+0000 7f207a039740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5557612aecb0 0x5557612ac610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:07.846 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.839+0000 7f207a039740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5557612b1880 con 0x5557612a3af0 2024-09-11T23:22:07.846 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.839+0000 7f207a039740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x555761191290 con 0x5557612a4410 2024-09-11T23:22:07.846 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.839+0000 7f207a039740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5557612b1a00 con 0x5557612aecb0 2024-09-11T23:22:07.846 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.840+0000 7f207a039740 10 client.0 _ll_register_callbacks cb 0x5557611c2140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:22:07.846 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.840+0000 7f2075042640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5557612a3af0 0x5557612abb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:07.846 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.840+0000 7f2074841640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5557612a4410 0x5557612ac0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:07.846 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.840+0000 7f2074841640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5557612a4410 0x5557612ac0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:43780/0 (socket says 172.21.15.26:43780) 2024-09-11T23:22:07.846 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.840+0000 7f2074841640 1 -- 172.21.15.26:0/1108304290 learned_addr learned my addr 172.21.15.26:0/1108304290 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:22:07.846 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.840+0000 7f2075843640 1 --2- 172.21.15.26:0/1108304290 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5557612aecb0 0x5557612ac610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:07.846 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.840+0000 7f2075042640 1 -- 172.21.15.26:0/1108304290 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5557612aecb0 msgr2=0x5557612ac610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:22:07.846 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.840+0000 7f2075042640 1 --2- 172.21.15.26:0/1108304290 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5557612aecb0 0x5557612ac610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:07.846 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.840+0000 7f2075042640 1 -- 172.21.15.26:0/1108304290 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5557612a4410 msgr2=0x5557612ac0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:22:07.847 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.840+0000 7f2075042640 1 --2- 172.21.15.26:0/1108304290 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5557612a4410 0x5557612ac0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:07.847 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.840+0000 7f2075042640 1 -- 172.21.15.26:0/1108304290 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555760f1b210 con 0x5557612a3af0 2024-09-11T23:22:07.847 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.840+0000 7f2074841640 1 --2- 172.21.15.26:0/1108304290 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5557612a4410 0x5557612ac0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:22:07.847 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.840+0000 7f2075042640 1 --2- 172.21.15.26:0/1108304290 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5557612a3af0 0x5557612abb90 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f206c0126e0 tx=0x7f206c012ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48b66a2ddd114ad5 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:22:07.847 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.840+0000 7f2058ff9640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:22:07.847 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.840+0000 7f2058ff9640 1 -- 172.21.15.26:0/1108304290 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f206c004030 con 0x5557612a3af0 2024-09-11T23:22:07.847 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.840+0000 7f2058ff9640 1 -- 172.21.15.26:0/1108304290 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f206c0041d0 con 0x5557612a3af0 2024-09-11T23:22:07.847 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.840+0000 7f207a039740 1 -- 172.21.15.26:0/1108304290 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f206009c210 con 0x5557612a3af0 2024-09-11T23:22:07.847 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.841+0000 7f2058ff9640 1 -- 172.21.15.26:0/1108304290 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f206c004410 con 0x5557612a3af0 2024-09-11T23:22:07.847 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.841+0000 7f207a039740 10 client.16293 fetch_fsmap learned FSMap version 33 2024-09-11T23:22:07.847 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.841+0000 7f207a039740 1 -- 172.21.15.26:0/1108304290 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x555760fe7620 con 0x5557612a3af0 2024-09-11T23:22:07.847 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.841+0000 7f2058ff9640 1 -- 172.21.15.26:0/1108304290 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f206009c210 con 0x5557612a3af0 2024-09-11T23:22:07.847 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.841+0000 7f2058ff9640 1 -- 172.21.15.26:0/1108304290 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 33) ==== 8394+0+0 (secure 0 0 0) 0x7f206c0039f0 con 0x5557612a3af0 2024-09-11T23:22:07.847 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.841+0000 7f207a039740 10 client.16293 fetch_fsmap finished waiting for FSMap version 33 2024-09-11T23:22:07.848 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.841+0000 7f207a039740 10 client.16293 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:22:07.848 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.841+0000 7f207a039740 20 client.16293 populate_metadata read hostname 'smithi026' 2024-09-11T23:22:07.848 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.841+0000 7f207a039740 1 --2- 172.21.15.26:0/1108304290 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5557612c30b0 0x5557612c5500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:07.848 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.841+0000 7f207a039740 4 client.16293 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:22:07.848 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.841+0000 7f207a039740 1 -- 172.21.15.26:0/1108304290 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555761215a20 con 0x5557612c30b0 2024-09-11T23:22:07.848 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.842+0000 7f2074841640 1 --2- 172.21.15.26:0/1108304290 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5557612c30b0 0x5557612c5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:07.848 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.842+0000 7f2074841640 1 --2- 172.21.15.26:0/1108304290 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5557612c30b0 0x5557612c5500 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc973992a6305041 server_cookie=41425a06d140a2ea in_seq=0 out_seq=0 2024-09-11T23:22:07.848 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.842+0000 7f2058ff9640 10 client.16293 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:22:07.848 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.844+0000 7f2058ff9640 1 -- 172.21.15.26:0/1108304290 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555761215a20 con 0x5557612c30b0 2024-09-11T23:22:07.848 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.844+0000 7f2058ff9640 10 client.16293 handle_command_reply: tid=0 2024-09-11T23:22:07.848 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.844+0000 7f207a039740 1 client.16293 shutdown 2024-09-11T23:22:07.849 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.844+0000 7f207a039740 1 -- 172.21.15.26:0/1108304290 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5557612a3af0 msgr2=0x5557612abb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:07.849 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.844+0000 7f207a039740 1 --2- 172.21.15.26:0/1108304290 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5557612a3af0 0x5557612abb90 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f206c0126e0 tx=0x7f206c012ae0 comp rx=0 tx=0).stop 2024-09-11T23:22:07.849 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.844+0000 7f207a039740 1 -- 172.21.15.26:0/1108304290 shutdown_connections 2024-09-11T23:22:07.849 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.844+0000 7f207a039740 1 --2- 172.21.15.26:0/1108304290 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5557612a4410 0x5557612ac0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:07.849 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.844+0000 7f207a039740 1 --2- 172.21.15.26:0/1108304290 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5557612c30b0 0x5557612c5500 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:07.849 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.844+0000 7f207a039740 1 --2- 172.21.15.26:0/1108304290 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5557612aecb0 0x5557612ac610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:07.849 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.844+0000 7f207a039740 1 --2- 172.21.15.26:0/1108304290 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5557612a3af0 0x5557612abb90 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:07.849 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.844+0000 7f207a039740 1 -- 172.21.15.26:0/1108304290 >> 172.21.15.26:0/1108304290 conn(0x5557612a0060 msgr2=0x5557612a5dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:22:07.849 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.844+0000 7f2058ff9640 0 client.16293 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:22:07.849 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.844+0000 7f207a039740 1 -- 172.21.15.26:0/1108304290 shutdown_connections 2024-09-11T23:22:07.849 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.844+0000 7f207a039740 1 -- 172.21.15.26:0/1108304290 wait complete. 2024-09-11T23:22:07.849 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.845+0000 7f207a039740 20 client.16293 trim_cache size 0 max 16384 2024-09-11T23:22:07.873 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.872+0000 7f207a039740 1 Processor -- start 2024-09-11T23:22:07.873 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.872+0000 7f207a039740 1 -- start start 2024-09-11T23:22:07.873 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.872+0000 7f207a039740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5557612a4410 0x5557612ac440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:07.874 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.872+0000 7f207a039740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5557612aecb0 0x5557612cae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:07.874 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.872+0000 7f207a039740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5557612a0ab0 0x5557612cb350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:07.874 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.872+0000 7f207a039740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f20701ab550 con 0x5557612a4410 2024-09-11T23:22:07.874 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.872+0000 7f207a039740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f20701ad210 con 0x5557612a0ab0 2024-09-11T23:22:07.874 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.872+0000 7f207a039740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f20701abc20 con 0x5557612aecb0 2024-09-11T23:22:07.874 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.872+0000 7f2075843640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5557612a0ab0 0x5557612cb350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:07.874 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.873+0000 7f2075042640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5557612a4410 0x5557612ac440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:07.874 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.873+0000 7f2075843640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5557612a0ab0 0x5557612cb350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:43788/0 (socket says 172.21.15.26:43788) 2024-09-11T23:22:07.874 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.873+0000 7f2075843640 1 -- 172.21.15.26:0/671557560 learned_addr learned my addr 172.21.15.26:0/671557560 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:22:07.874 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.873+0000 7f2074841640 1 --2- 172.21.15.26:0/671557560 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5557612aecb0 0x5557612cae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:07.874 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.873+0000 7f2075042640 1 -- 172.21.15.26:0/671557560 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5557612aecb0 msgr2=0x5557612cae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:22:07.874 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.873+0000 7f2075042640 1 --2- 172.21.15.26:0/671557560 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5557612aecb0 0x5557612cae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:07.874 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.873+0000 7f2075042640 1 -- 172.21.15.26:0/671557560 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5557612a0ab0 msgr2=0x5557612cb350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:22:07.875 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.873+0000 7f2075042640 1 --2- 172.21.15.26:0/671557560 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5557612a0ab0 0x5557612cb350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:07.875 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.873+0000 7f2075042640 1 -- 172.21.15.26:0/671557560 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557612a4240 con 0x5557612a4410 2024-09-11T23:22:07.875 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.873+0000 7f2075042640 1 --2- 172.21.15.26:0/671557560 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5557612a4410 0x5557612ac440 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f206c003e90 tx=0x7f206c00f3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3f83424e8717c06 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:22:07.875 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.873+0000 7f205affd640 1 -- 172.21.15.26:0/671557560 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f206c000e60 con 0x5557612a4410 2024-09-11T23:22:07.875 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.874+0000 7f205affd640 1 -- 172.21.15.26:0/671557560 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f206c003ad0 con 0x5557612a4410 2024-09-11T23:22:07.875 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.874+0000 7f205affd640 1 -- 172.21.15.26:0/671557560 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f206c029070 con 0x5557612a4410 2024-09-11T23:22:07.879 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.874+0000 7f207a039740 1 -- 172.21.15.26:0/671557560 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5557612a4410 msgr2=0x5557612ac440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:07.879 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.874+0000 7f207a039740 1 --2- 172.21.15.26:0/671557560 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5557612a4410 0x5557612ac440 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f206c003e90 tx=0x7f206c00f3c0 comp rx=0 tx=0).stop 2024-09-11T23:22:07.879 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.874+0000 7f207a039740 1 -- 172.21.15.26:0/671557560 shutdown_connections 2024-09-11T23:22:07.879 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.874+0000 7f207a039740 1 --2- 172.21.15.26:0/671557560 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5557612a0ab0 0x5557612cb350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:07.879 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.874+0000 7f207a039740 1 --2- 172.21.15.26:0/671557560 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5557612aecb0 0x5557612cae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:07.879 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.874+0000 7f207a039740 1 --2- 172.21.15.26:0/671557560 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5557612a4410 0x5557612ac440 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:07.879 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.874+0000 7f207a039740 1 -- 172.21.15.26:0/671557560 >> 172.21.15.26:0/671557560 conn(0x5557612a0060 msgr2=0x5557612aac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:22:07.879 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.874+0000 7f207a039740 1 -- 172.21.15.26:0/671557560 shutdown_connections 2024-09-11T23:22:07.879 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.874+0000 7f207a039740 1 -- 172.21.15.26:0/671557560 wait complete. 2024-09-11T23:22:07.879 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.875+0000 7f207a039740 1 -- start start 2024-09-11T23:22:07.880 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.875+0000 7f207a039740 1 Processor -- start 2024-09-11T23:22:07.880 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.875+0000 7f207a039740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5557612a0ab0 0x5557612b2230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:07.880 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.875+0000 7f207a039740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5557612a4410 0x5557612b2770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:07.880 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.875+0000 7f207a039740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5557612aecb0 0x55576132bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:07.880 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.875+0000 7f207a039740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f20701aca80 con 0x5557612a0ab0 2024-09-11T23:22:07.880 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.875+0000 7f207a039740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f20701ac2f0 con 0x5557612a4410 2024-09-11T23:22:07.880 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.875+0000 7f207a039740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f20701a9530 con 0x5557612aecb0 2024-09-11T23:22:07.880 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.875+0000 7f207a039740 10 client.0 _ll_register_callbacks cb 0x7f206c009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:22:07.880 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.875+0000 7f2075042640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5557612a0ab0 0x5557612b2230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:07.880 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.876+0000 7f2074841640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5557612a4410 0x5557612b2770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:07.880 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.876+0000 7f2075843640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5557612aecb0 0x55576132bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:07.880 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.876+0000 7f2074841640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5557612a4410 0x5557612b2770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:43792/0 (socket says 172.21.15.26:43792) 2024-09-11T23:22:07.880 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.876+0000 7f2075042640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5557612a0ab0 0x5557612b2230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60208/0 (socket says 172.21.15.26:60208) 2024-09-11T23:22:07.881 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.876+0000 7f2074841640 1 -- 172.21.15.26:0/3641836909 learned_addr learned my addr 172.21.15.26:0/3641836909 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:22:07.881 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.876+0000 7f2074841640 1 -- 172.21.15.26:0/3641836909 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5557612aecb0 msgr2=0x55576132bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:22:07.881 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.876+0000 7f2074841640 1 --2- 172.21.15.26:0/3641836909 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5557612aecb0 0x55576132bdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:07.881 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.876+0000 7f2074841640 1 -- 172.21.15.26:0/3641836909 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5557612a0ab0 msgr2=0x5557612b2230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:22:07.881 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.876+0000 7f2074841640 1 --2- 172.21.15.26:0/3641836909 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5557612a0ab0 0x5557612b2230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:07.881 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.876+0000 7f2074841640 1 -- 172.21.15.26:0/3641836909 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55576132c360 con 0x5557612a4410 2024-09-11T23:22:07.881 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.876+0000 7f2075843640 1 --2- 172.21.15.26:0/3641836909 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5557612aecb0 0x55576132bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:22:07.881 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.876+0000 7f2075042640 1 --2- 172.21.15.26:0/3641836909 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5557612a0ab0 0x5557612b2230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:22:07.881 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.876+0000 7f2074841640 1 --2- 172.21.15.26:0/3641836909 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5557612a4410 0x5557612b2770 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f206009b810 tx=0x7f20600a47d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=73e230893e3a235c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:22:07.881 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.876+0000 7f2058ff9640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-11T23:22:07.881 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.876+0000 7f2058ff9640 1 -- 172.21.15.26:0/3641836909 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f206009b470 con 0x5557612a4410 2024-09-11T23:22:07.881 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.876+0000 7f2058ff9640 1 -- 172.21.15.26:0/3641836909 <== mon.2 v2:172.21.15.76:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f2060097920 con 0x5557612a4410 2024-09-11T23:22:07.881 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.876+0000 7f207a039740 1 -- 172.21.15.26:0/3641836909 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f206c003ad0 con 0x5557612a4410 2024-09-11T23:22:07.882 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.877+0000 7f2058ff9640 1 -- 172.21.15.26:0/3641836909 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f206009b650 con 0x5557612a4410 2024-09-11T23:22:07.882 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.877+0000 7f207a039740 10 client.25886 fetch_fsmap learned FSMap version 33 2024-09-11T23:22:07.882 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.877+0000 7f207a039740 1 -- 172.21.15.26:0/3641836909 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55576132d6f0 con 0x5557612a4410 2024-09-11T23:22:07.882 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.877+0000 7f2058ff9640 1 -- 172.21.15.26:0/3641836909 <== mon.2 v2:172.21.15.76:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f206c003ad0 con 0x5557612a4410 2024-09-11T23:22:07.882 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.877+0000 7f2058ff9640 1 -- 172.21.15.26:0/3641836909 <== mon.2 v2:172.21.15.76:3300/0 5 ==== fsmap(e 33) ==== 8394+0+0 (secure 0 0 0) 0x7f2060098180 con 0x5557612a4410 2024-09-11T23:22:07.882 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.877+0000 7f207a039740 10 client.25886 fetch_fsmap finished waiting for FSMap version 33 2024-09-11T23:22:07.882 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.877+0000 7f207a039740 10 client.25886 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:22:07.882 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.877+0000 7f207a039740 20 client.25886 populate_metadata read hostname 'smithi026' 2024-09-11T23:22:07.882 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.877+0000 7f207a039740 1 --2- 172.21.15.26:0/3641836909 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55576132db40 0x55576132ff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:07.882 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.877+0000 7f207a039740 4 client.25886 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T23:22:07.882 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.877+0000 7f207a039740 1 -- 172.21.15.26:0/3641836909 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55576120fd70 con 0x55576132db40 2024-09-11T23:22:07.882 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.877+0000 7f2075042640 1 --2- 172.21.15.26:0/3641836909 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55576132db40 0x55576132ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:07.883 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.878+0000 7f2075042640 1 --2- 172.21.15.26:0/3641836909 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55576132db40 0x55576132ff30 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=793b057f3ee199f4 server_cookie=231eefb67b9c5594 in_seq=0 out_seq=0 2024-09-11T23:22:07.883 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:07.878+0000 7f2058ff9640 10 client.25886 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:22:09.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:22:09 smithi136 ceph-mon[35121]: pgmap v116: 97 pgs: 97 active+clean; 743 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 15 MiB/s wr, 669 op/s 2024-09-11T23:22:09.750 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:22:09 smithi026 ceph-mon[31172]: pgmap v116: 97 pgs: 97 active+clean; 743 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 15 MiB/s wr, 669 op/s 2024-09-11T23:22:09.802 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:22:09 smithi076 ceph-mon[35296]: pgmap v116: 97 pgs: 97 active+clean; 743 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 15 MiB/s wr, 669 op/s 2024-09-11T23:22:10.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:22:10 smithi136 ceph-mon[35121]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:22:10.750 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:22:10 smithi026 ceph-mon[31172]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:22:10.802 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:22:10 smithi076 ceph-mon[35296]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:22:11.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:22:11 smithi136 ceph-mon[35121]: pgmap v117: 97 pgs: 97 active+clean; 752 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 7.4 MiB/s wr, 318 op/s 2024-09-11T23:22:11.750 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:22:11 smithi026 ceph-mon[31172]: pgmap v117: 97 pgs: 97 active+clean; 752 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 7.4 MiB/s wr, 318 op/s 2024-09-11T23:22:11.802 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:22:11 smithi076 ceph-mon[35296]: pgmap v117: 97 pgs: 97 active+clean; 752 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 7.4 MiB/s wr, 318 op/s 2024-09-11T23:22:12.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:22:12 smithi136 ceph-mon[35121]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:22:12.750 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:22:12 smithi026 ceph-mon[31172]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:22:12.802 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:22:12 smithi076 ceph-mon[35296]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:22:13.577 INFO:tasks.workunit.client.0.smithi026.stdout: 290 497160 2454 338574 2711 94251 2105 2024-09-11T23:22:13.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:22:13 smithi136 ceph-mon[35121]: pgmap v118: 97 pgs: 97 active+clean; 752 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.9 MiB/s wr, 203 op/s 2024-09-11T23:22:13.750 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:22:13 smithi026 ceph-mon[31172]: pgmap v118: 97 pgs: 97 active+clean; 752 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.9 MiB/s wr, 203 op/s 2024-09-11T23:22:13.802 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:22:13 smithi076 ceph-mon[35296]: pgmap v118: 97 pgs: 97 active+clean; 752 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.9 MiB/s wr, 203 op/s 2024-09-11T23:22:15.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:22:15 smithi136 ceph-mon[35121]: pgmap v119: 97 pgs: 97 active+clean; 774 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 7.0 MiB/s wr, 626 op/s 2024-09-11T23:22:15.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:22:15 smithi136 ceph-mon[35121]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:22:15.750 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:22:15 smithi026 ceph-mon[31172]: pgmap v119: 97 pgs: 97 active+clean; 774 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 7.0 MiB/s wr, 626 op/s 2024-09-11T23:22:15.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:22:15 smithi026 ceph-mon[31172]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:22:15.802 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:22:15 smithi076 ceph-mon[35296]: pgmap v119: 97 pgs: 97 active+clean; 774 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 7.0 MiB/s wr, 626 op/s 2024-09-11T23:22:15.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:22:15 smithi076 ceph-mon[35296]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:22:16.706 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:16.706+0000 7f2058ff9640 1 -- 172.21.15.26:0/3641836909 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55576120fd70 con 0x55576132db40 2024-09-11T23:22:16.707 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:16.706+0000 7f2058ff9640 10 client.25886 handle_command_reply: tid=0 2024-09-11T23:22:16.707 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:16.706+0000 7f207a039740 1 client.25886 shutdown 2024-09-11T23:22:16.707 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:16.706+0000 7f207a039740 1 -- 172.21.15.26:0/3641836909 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5557612a4410 msgr2=0x5557612b2770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:16.707 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:16.706+0000 7f207a039740 1 --2- 172.21.15.26:0/3641836909 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5557612a4410 0x5557612b2770 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f206009b810 tx=0x7f20600a47d0 comp rx=0 tx=0).stop 2024-09-11T23:22:16.707 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:16.706+0000 7f207a039740 1 -- 172.21.15.26:0/3641836909 shutdown_connections 2024-09-11T23:22:16.707 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:16.706+0000 7f207a039740 1 --2- 172.21.15.26:0/3641836909 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5557612a4410 0x5557612b2770 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:16.707 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:16.706+0000 7f207a039740 1 --2- 172.21.15.26:0/3641836909 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55576132db40 0x55576132ff30 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:16.707 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:16.706+0000 7f207a039740 1 --2- 172.21.15.26:0/3641836909 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5557612aecb0 0x55576132bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:16.707 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:16.706+0000 7f207a039740 1 --2- 172.21.15.26:0/3641836909 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5557612a0ab0 0x5557612b2230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:16.707 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:16.706+0000 7f2058ff9640 0 client.25886 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:22:16.707 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:16.706+0000 7f207a039740 1 -- 172.21.15.26:0/3641836909 >> 172.21.15.26:0/3641836909 conn(0x5557612a0060 msgr2=0x55576129e9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:22:16.707 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:16.706+0000 7f207a039740 1 -- 172.21.15.26:0/3641836909 shutdown_connections 2024-09-11T23:22:16.708 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:16.706+0000 7f207a039740 1 -- 172.21.15.26:0/3641836909 wait complete. 2024-09-11T23:22:16.708 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:16.706+0000 7f207a039740 20 client.25886 trim_cache size 0 max 16384 2024-09-11T23:22:16.708 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-09-11T23:22:16.708 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "message": "", 2024-09-11T23:22:16.708 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-09-11T23:22:16.708 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-09-11T23:22:16.709 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:16.707+0000 7f20772a6640 1 -- 172.21.15.26:0/2380868792 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f205c0a2da0 msgr2=0x7f205c0a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:16.709 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:16.707+0000 7f20772a6640 1 --2- 172.21.15.26:0/2380868792 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f205c0a2da0 0x7f205c0a5260 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f206800b230 tx=0x7f2068005fd0 comp rx=0 tx=0).stop 2024-09-11T23:22:16.709 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:16.707+0000 7f20772a6640 1 -- 172.21.15.26:0/2380868792 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f20701035c0 msgr2=0x7f2070198650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:16.709 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:16.707+0000 7f20772a6640 1 --2- 172.21.15.26:0/2380868792 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f20701035c0 0x7f2070198650 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f2060005fd0 tx=0x7f206000ee50 comp rx=0 tx=0).stop 2024-09-11T23:22:16.709 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:16.707+0000 7f20772a6640 1 -- 172.21.15.26:0/2380868792 shutdown_connections 2024-09-11T23:22:16.709 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:16.707+0000 7f20772a6640 1 --2- 172.21.15.26:0/2380868792 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f205c0a2da0 0x7f205c0a5260 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:16.709 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:16.707+0000 7f20772a6640 1 --2- 172.21.15.26:0/2380868792 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f207010c0c0 0x7f207019c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:16.709 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:16.707+0000 7f20772a6640 1 --2- 172.21.15.26:0/2380868792 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f20701035c0 0x7f2070198650 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:16.710 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:16.707+0000 7f20772a6640 1 --2- 172.21.15.26:0/2380868792 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2070102ca0 0x7f2070198110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:16.710 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:16.707+0000 7f20772a6640 1 -- 172.21.15.26:0/2380868792 >> 172.21.15.26:0/2380868792 conn(0x7f20700fc6f0 msgr2=0x7f207010d940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:22:16.710 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:16.708+0000 7f20772a6640 1 -- 172.21.15.26:0/2380868792 shutdown_connections 2024-09-11T23:22:16.710 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:16.708+0000 7f20772a6640 1 -- 172.21.15.26:0/2380868792 wait complete. 2024-09-11T23:22:17.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:22:17 smithi136 ceph-mon[35121]: pgmap v120: 97 pgs: 97 active+clean; 774 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.0 MiB/s wr, 493 op/s 2024-09-11T23:22:17.750 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:22:17 smithi026 ceph-mon[31172]: pgmap v120: 97 pgs: 97 active+clean; 774 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.0 MiB/s wr, 493 op/s 2024-09-11T23:22:17.802 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:22:17 smithi076 ceph-mon[35296]: pgmap v120: 97 pgs: 97 active+clean; 774 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.0 MiB/s wr, 493 op/s 2024-09-11T23:22:18.378 INFO:tasks.quiescer.fs.[cephfs]:Total duration (40.3) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-11T23:22:18.378 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 20.15 2024-09-11T23:22:18.379 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:22:19.491 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:22:19 smithi026 ceph-mon[31172]: pgmap v121: 97 pgs: 97 active+clean; 788 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 5.3 MiB/s wr, 535 op/s 2024-09-11T23:22:19.739 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:22:19 smithi136 ceph-mon[35121]: pgmap v121: 97 pgs: 97 active+clean; 788 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 5.3 MiB/s wr, 535 op/s 2024-09-11T23:22:19.802 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:22:19 smithi076 ceph-mon[35296]: pgmap v121: 97 pgs: 97 active+clean; 788 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 5.3 MiB/s wr, 535 op/s 2024-09-11T23:22:21.466 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:22:21.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:22:21 smithi026 ceph-mon[31172]: pgmap v122: 97 pgs: 97 active+clean; 832 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 8.4 MiB/s wr, 651 op/s 2024-09-11T23:22:21.802 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:22:21 smithi076 ceph-mon[35296]: pgmap v122: 97 pgs: 97 active+clean; 832 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 8.4 MiB/s wr, 651 op/s 2024-09-11T23:22:21.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:22:21 smithi136 ceph-mon[35121]: pgmap v122: 97 pgs: 97 active+clean; 832 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 8.4 MiB/s wr, 651 op/s 2024-09-11T23:22:22.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:22.616+0000 7f9e1a53b640 1 -- 172.21.15.26:0/155302566 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9e14112850 msgr2=0x7f9e14114c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:22.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:22.616+0000 7f9e1a53b640 1 --2- 172.21.15.26:0/155302566 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9e14112850 0x7f9e14114c40 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f9e0c01a910 tx=0x7f9e0c03b430 comp rx=0 tx=0).stop 2024-09-11T23:22:22.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:22.618+0000 7f9e1a53b640 1 -- 172.21.15.26:0/155302566 shutdown_connections 2024-09-11T23:22:22.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:22.618+0000 7f9e1a53b640 1 --2- 172.21.15.26:0/155302566 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9e14112850 0x7f9e14114c40 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:22.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:22.618+0000 7f9e1a53b640 1 --2- 172.21.15.26:0/155302566 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9e14077040 0x7f9e14112310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:22.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:22.618+0000 7f9e1a53b640 1 --2- 172.21.15.26:0/155302566 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9e14076720 0x7f9e14076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:22.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:22.618+0000 7f9e1a53b640 1 -- 172.21.15.26:0/155302566 >> 172.21.15.26:0/155302566 conn(0x7f9e140ff870 msgr2=0x7f9e14101c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:22:22.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:22.621+0000 7f9e1a53b640 1 -- 172.21.15.26:0/155302566 shutdown_connections 2024-09-11T23:22:22.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:22.621+0000 7f9e1a53b640 1 -- 172.21.15.26:0/155302566 wait complete. 2024-09-11T23:22:22.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:22.622+0000 7f9e1a53b640 1 Processor -- start 2024-09-11T23:22:22.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:22.623+0000 7f9e1a53b640 1 -- start start 2024-09-11T23:22:22.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:22.623+0000 7f9e1a53b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9e14076720 0x7f9e14111f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:22.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:22.624+0000 7f9e1a53b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9e14077040 0x7f9e1410dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:22.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:22.624+0000 7f9e1a53b640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9e14112850 0x7f9e1410e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:22.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:22.624+0000 7f9e1a53b640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f9e14106450 con 0x7f9e14076720 2024-09-11T23:22:22.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:22.624+0000 7f9e19539640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9e14076720 0x7f9e14111f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:22.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:22.624+0000 7f9e19539640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9e14076720 0x7f9e14111f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43796/0 (socket says 172.21.15.26:43796) 2024-09-11T23:22:22.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:22.624+0000 7f9e19539640 1 -- 172.21.15.26:0/1271524545 learned_addr learned my addr 172.21.15.26:0/1271524545 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:22:22.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:22.624+0000 7f9e18d38640 1 --2- 172.21.15.26:0/1271524545 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9e14077040 0x7f9e1410dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:22.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:22.624+0000 7f9e19d3a640 1 --2- 172.21.15.26:0/1271524545 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9e14112850 0x7f9e1410e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:22.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:22.624+0000 7f9e1a53b640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f9e141062d0 con 0x7f9e14077040 2024-09-11T23:22:22.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:22.624+0000 7f9e1a53b640 1 -- 172.21.15.26:0/1271524545 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f9e141065d0 con 0x7f9e14112850 2024-09-11T23:22:22.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:22.627+0000 7f9e19539640 1 -- 172.21.15.26:0/1271524545 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9e14112850 msgr2=0x7f9e1410e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:22.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:22.627+0000 7f9e19539640 1 --2- 172.21.15.26:0/1271524545 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9e14112850 0x7f9e1410e4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:22.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:22.627+0000 7f9e19539640 1 -- 172.21.15.26:0/1271524545 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9e14077040 msgr2=0x7f9e1410dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:22.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:22.627+0000 7f9e19539640 1 --2- 172.21.15.26:0/1271524545 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9e14077040 0x7f9e1410dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:22.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:22.627+0000 7f9e19539640 1 -- 172.21.15.26:0/1271524545 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e1410ec20 con 0x7f9e14076720 2024-09-11T23:22:22.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:22.627+0000 7f9e19539640 1 --2- 172.21.15.26:0/1271524545 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9e14076720 0x7f9e14111f10 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f9e0000c890 tx=0x7f9e0000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:22:22.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:22.628+0000 7f9e19d3a640 1 --2- 172.21.15.26:0/1271524545 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9e14112850 0x7f9e1410e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:22:22.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:22.628+0000 7f9e127fc640 1 -- 172.21.15.26:0/1271524545 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e00004220 con 0x7f9e14076720 2024-09-11T23:22:22.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:22.628+0000 7f9e127fc640 1 -- 172.21.15.26:0/1271524545 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e000043c0 con 0x7f9e14076720 2024-09-11T23:22:22.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:22.628+0000 7f9e127fc640 1 -- 172.21.15.26:0/1271524545 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e000049c0 con 0x7f9e14076720 2024-09-11T23:22:22.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:22.628+0000 7f9e1a53b640 1 -- 172.21.15.26:0/1271524545 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e141a2780 con 0x7f9e14076720 2024-09-11T23:22:22.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:22.628+0000 7f9e1a53b640 1 -- 172.21.15.26:0/1271524545 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e141a2b20 con 0x7f9e14076720 2024-09-11T23:22:22.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:22.633+0000 7f9e1a53b640 1 -- 172.21.15.26:0/1271524545 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e14077e60 con 0x7f9e14076720 2024-09-11T23:22:22.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:22.638+0000 7f9e127fc640 1 -- 172.21.15.26:0/1271524545 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f9e00004b60 con 0x7f9e14076720 2024-09-11T23:22:22.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:22.639+0000 7f9e127fc640 1 --2- 172.21.15.26:0/1271524545 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f9e040a2f00 0x7f9e040a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:22.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:22.639+0000 7f9e18d38640 1 --2- 172.21.15.26:0/1271524545 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f9e040a2f00 0x7f9e040a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:22.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:22.640+0000 7f9e127fc640 1 -- 172.21.15.26:0/1271524545 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f9e000cf380 con 0x7f9e14076720 2024-09-11T23:22:22.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:22.641+0000 7f9e18d38640 1 --2- 172.21.15.26:0/1271524545 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f9e040a2f00 0x7f9e040a53c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f9e1410f770 tx=0x7f9e0800a590 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:22:22.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:22.647+0000 7f9e127fc640 1 -- 172.21.15.26:0/1271524545 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9e00014030 con 0x7f9e14076720 2024-09-11T23:22:23.167 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-11T23:22:23.168 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":36,"btime":"2024-09-11T23:22:14:471550+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14499,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/367659515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":367659515},{"type":"v1","addr":"172.21.15.26:6835","nonce":367659515}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14511,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/2041065299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":2041065299},{"type":"v1","addr":"172.21.15.26:6837","nonce":2041065299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":36,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:16:26.544203+0000","modified":"2024-09-11T23:22:13.708769+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24485,"mds_1":24463,"mds_2":24458,"mds_3":24448,"mds_4":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14550":{"gid":14550,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6839/2233068305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":2233068305},{"type":"v1","addr":"172.21.15.26:6839","nonce":2233068305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14562":{"gid":14562,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6841/2984982336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":2984982336},{"type":"v1","addr":"172.21.15.26:6841","nonce":2984982336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6833/2019841455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":2019841455},{"type":"v1","addr":"172.21.15.136:6833","nonce":2019841455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.76:6833/3438147050","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3438147050},{"type":"v1","addr":"172.21.15.76:6833","nonce":3438147050}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6835/3617665975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3617665975},{"type":"v1","addr":"172.21.15.136:6835","nonce":3617665975}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":12,"addr":"172.21.15.76:6835/393308499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":393308499},{"type":"v1","addr":"172.21.15.76:6835","nonce":393308499}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":10,"addr":"172.21.15.136:6837/1633082387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":1633082387},{"type":"v1","addr":"172.21.15.136:6837","nonce":1633082387}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.76:6837/2078978748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":2078978748},{"type":"v1","addr":"172.21.15.76:6837","nonce":2078978748}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":63,"addr":"172.21.15.136:6839/3211398630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":3211398630},{"type":"v1","addr":"172.21.15.136:6839","nonce":3211398630}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.76:6839/2077637090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6838","nonce":2077637090},{"type":"v1","addr":"172.21.15.76:6839","nonce":2077637090}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24485,"qdb_cluster":[24485,24463,24458,24448,24443]},"id":1}]} 2024-09-11T23:22:23.168 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 36 2024-09-11T23:22:23.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:23.164+0000 7f9e1a53b640 1 -- 172.21.15.26:0/1271524545 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9e141a2f10 con 0x7f9e14076720 2024-09-11T23:22:23.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:23.166+0000 7f9e127fc640 1 -- 172.21.15.26:0/1271524545 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+11987 (secure 0 0 0) 0x7f9e0009bf50 con 0x7f9e14076720 2024-09-11T23:22:23.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:23.179+0000 7f9dfffff640 1 -- 172.21.15.26:0/1271524545 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f9e040a2f00 msgr2=0x7f9e040a53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:23.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:23.179+0000 7f9dfffff640 1 --2- 172.21.15.26:0/1271524545 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f9e040a2f00 0x7f9e040a53c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f9e1410f770 tx=0x7f9e0800a590 comp rx=0 tx=0).stop 2024-09-11T23:22:23.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:23.179+0000 7f9dfffff640 1 -- 172.21.15.26:0/1271524545 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9e14076720 msgr2=0x7f9e14111f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:23.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:23.179+0000 7f9dfffff640 1 --2- 172.21.15.26:0/1271524545 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9e14076720 0x7f9e14111f10 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f9e0000c890 tx=0x7f9e0000cd50 comp rx=0 tx=0).stop 2024-09-11T23:22:23.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:23.179+0000 7f9dfffff640 1 -- 172.21.15.26:0/1271524545 shutdown_connections 2024-09-11T23:22:23.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:23.179+0000 7f9dfffff640 1 --2- 172.21.15.26:0/1271524545 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f9e040a2f00 0x7f9e040a53c0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:23.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:23.179+0000 7f9dfffff640 1 --2- 172.21.15.26:0/1271524545 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9e14077040 0x7f9e1410dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:23.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:23.179+0000 7f9dfffff640 1 --2- 172.21.15.26:0/1271524545 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9e14112850 0x7f9e1410e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:23.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:23.179+0000 7f9dfffff640 1 --2- 172.21.15.26:0/1271524545 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9e14076720 0x7f9e14111f10 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:23.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:23.180+0000 7f9dfffff640 1 -- 172.21.15.26:0/1271524545 >> 172.21.15.26:0/1271524545 conn(0x7f9e140ff870 msgr2=0x7f9e14101c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:22:23.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:23.180+0000 7f9dfffff640 1 -- 172.21.15.26:0/1271524545 shutdown_connections 2024-09-11T23:22:23.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:23.182+0000 7f9dfffff640 1 -- 172.21.15.26:0/1271524545 wait complete. 2024-09-11T23:22:23.501 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:22:23 smithi026 ceph-mon[31172]: pgmap v123: 97 pgs: 97 active+clean; 832 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 7.6 MiB/s wr, 633 op/s 2024-09-11T23:22:23.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:22:23 smithi026 ceph-mon[31172]: from='client.? 172.21.15.26:0/1271524545' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:22:23.577 INFO:tasks.workunit.client.0.smithi026.stdout: 332 1286949 2041 880352 2218 272366 1000 2024-09-11T23:22:23.802 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:22:23 smithi076 ceph-mon[35296]: pgmap v123: 97 pgs: 97 active+clean; 832 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 7.6 MiB/s wr, 633 op/s 2024-09-11T23:22:23.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:22:23 smithi076 ceph-mon[35296]: from='client.? 172.21.15.26:0/1271524545' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:22:23.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:22:23 smithi136 ceph-mon[35121]: pgmap v123: 97 pgs: 97 active+clean; 832 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 7.6 MiB/s wr, 633 op/s 2024-09-11T23:22:23.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:22:23 smithi136 ceph-mon[35121]: from='client.? 172.21.15.26:0/1271524545' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:22:25.426 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24485 quiesce db / --timeout 90 --expiration 140.15 --await-for 45.0' 2024-09-11T23:22:25.426 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24485 quiesce db / --timeout 90 --expiration 140.15 --await-for 45.0 2024-09-11T23:22:25.755 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:22:25 smithi026 ceph-mon[31172]: pgmap v124: 97 pgs: 97 active+clean; 937 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 17 MiB/s wr, 982 op/s 2024-09-11T23:22:25.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:22:25 smithi076 ceph-mon[35296]: pgmap v124: 97 pgs: 97 active+clean; 937 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 17 MiB/s wr, 982 op/s 2024-09-11T23:22:25.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:22:25 smithi136 ceph-mon[35121]: pgmap v124: 97 pgs: 97 active+clean; 937 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 17 MiB/s wr, 982 op/s 2024-09-11T23:22:27.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:22:27 smithi076 ceph-mon[35296]: pgmap v125: 97 pgs: 97 active+clean; 950 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 16 MiB/s wr, 608 op/s 2024-09-11T23:22:27.803 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:22:27 smithi076 ceph-mon[35296]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:22:27.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:22:27 smithi136 ceph-mon[35121]: pgmap v125: 97 pgs: 97 active+clean; 950 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 16 MiB/s wr, 608 op/s 2024-09-11T23:22:27.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:22:27 smithi136 ceph-mon[35121]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:22:28.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:22:27 smithi026 ceph-mon[31172]: pgmap v125: 97 pgs: 97 active+clean; 950 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 16 MiB/s wr, 608 op/s 2024-09-11T23:22:28.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:22:27 smithi026 ceph-mon[31172]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:22:29.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:22:29 smithi136 ceph-mon[35121]: pgmap v126: 97 pgs: 97 active+clean; 986 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 19 MiB/s wr, 718 op/s 2024-09-11T23:22:30.000 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:22:29 smithi026 ceph-mon[31172]: pgmap v126: 97 pgs: 97 active+clean; 986 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 19 MiB/s wr, 718 op/s 2024-09-11T23:22:30.052 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:22:29 smithi076 ceph-mon[35296]: pgmap v126: 97 pgs: 97 active+clean; 986 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 19 MiB/s wr, 718 op/s 2024-09-11T23:22:31.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:22:31 smithi136 ceph-mon[35121]: pgmap v127: 97 pgs: 97 active+clean; 1.1 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 31 MiB/s wr, 1.29k op/s 2024-09-11T23:22:32.000 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:22:31 smithi026 ceph-mon[31172]: pgmap v127: 97 pgs: 97 active+clean; 1.1 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 31 MiB/s wr, 1.29k op/s 2024-09-11T23:22:32.052 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:22:31 smithi076 ceph-mon[35296]: pgmap v127: 97 pgs: 97 active+clean; 1.1 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 31 MiB/s wr, 1.29k op/s 2024-09-11T23:22:32.540 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:32.539+0000 7fc3b005c640 1 -- 172.21.15.26:0/3878913464 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc3a81091f0 msgr2=0x7fc3a8110250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:32.541 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:32.539+0000 7fc3b005c640 1 --2- 172.21.15.26:0/3878913464 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc3a81091f0 0x7fc3a8110250 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fc3a0009ca0 tx=0x7fc3a002a8f0 comp rx=0 tx=0).stop 2024-09-11T23:22:32.544 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:32.543+0000 7fc3b005c640 1 -- 172.21.15.26:0/3878913464 shutdown_connections 2024-09-11T23:22:32.544 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:32.543+0000 7fc3b005c640 1 --2- 172.21.15.26:0/3878913464 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc3a811b140 0x7fc3a811d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:32.544 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:32.543+0000 7fc3b005c640 1 --2- 172.21.15.26:0/3878913464 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc3a81095d0 0x7fc3a8109a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:32.544 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:32.543+0000 7fc3b005c640 1 --2- 172.21.15.26:0/3878913464 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc3a81091f0 0x7fc3a8110250 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:32.544 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:32.543+0000 7fc3b005c640 1 -- 172.21.15.26:0/3878913464 >> 172.21.15.26:0/3878913464 conn(0x7fc3a806cc60 msgr2=0x7fc3a806d070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:22:32.545 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:32.543+0000 7fc3b005c640 1 -- 172.21.15.26:0/3878913464 shutdown_connections 2024-09-11T23:22:32.545 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:22:32.547 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:32.547+0000 7fc3b005c640 1 -- 172.21.15.26:0/3878913464 wait complete. 2024-09-11T23:22:32.549 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:32.548+0000 7fc3b005c640 1 Processor -- start 2024-09-11T23:22:32.549 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:32.548+0000 7fc3b005c640 1 -- start start 2024-09-11T23:22:32.549 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:32.548+0000 7fc3b005c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc3a81091f0 0x7fc3a81a7090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:32.549 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:32.548+0000 7fc3b005c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc3a81095d0 0x7fc3a81a75d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:32.549 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:32.548+0000 7fc3b005c640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc3a811b140 0x7fc3a81ab950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:32.550 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:32.548+0000 7fc3b005c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc3a8120ba0 con 0x7fc3a81095d0 2024-09-11T23:22:32.550 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:32.548+0000 7fc3b005c640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fc3a8120a20 con 0x7fc3a81091f0 2024-09-11T23:22:32.550 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:32.548+0000 7fc3b005c640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fc3a8120d20 con 0x7fc3a811b140 2024-09-11T23:22:32.551 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:32.549+0000 7fc3addf8640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc3a81091f0 0x7fc3a81a7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:32.551 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:32.549+0000 7fc3addf8640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc3a81091f0 0x7fc3a81a7090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:45138/0 (socket says 172.21.15.26:45138) 2024-09-11T23:22:32.551 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:32.549+0000 7fc3addf8640 1 -- 172.21.15.26:0/2973489930 learned_addr learned my addr 172.21.15.26:0/2973489930 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:22:32.551 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:32.549+0000 7fc3ad5f7640 1 --2- 172.21.15.26:0/2973489930 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc3a81095d0 0x7fc3a81a75d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:32.551 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:32.549+0000 7fc3addf8640 1 -- 172.21.15.26:0/2973489930 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc3a811b140 msgr2=0x7fc3a81ab950 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-11T23:22:32.551 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:32.549+0000 7fc3addf8640 1 --2- 172.21.15.26:0/2973489930 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc3a811b140 0x7fc3a81ab950 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:32.552 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:32.549+0000 7fc3addf8640 1 -- 172.21.15.26:0/2973489930 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc3a81095d0 msgr2=0x7fc3a81a75d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:32.552 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:32.549+0000 7fc3addf8640 1 --2- 172.21.15.26:0/2973489930 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc3a81095d0 0x7fc3a81a75d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:32.552 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:32.549+0000 7fc3addf8640 1 -- 172.21.15.26:0/2973489930 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3a81ac0d0 con 0x7fc3a81091f0 2024-09-11T23:22:32.552 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:32.549+0000 7fc3ad5f7640 1 --2- 172.21.15.26:0/2973489930 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc3a81095d0 0x7fc3a81a75d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:22:32.552 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:32.550+0000 7fc3addf8640 1 --2- 172.21.15.26:0/2973489930 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc3a81091f0 0x7fc3a81a7090 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fc3a002f860 tx=0x7fc3a002f890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:22:32.590 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:32.589+0000 7fc39effd640 1 -- 172.21.15.26:0/2973489930 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3a002fe40 con 0x7fc3a81091f0 2024-09-11T23:22:32.590 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:32.589+0000 7fc3b005c640 1 -- 172.21.15.26:0/2973489930 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3a81ac360 con 0x7fc3a81091f0 2024-09-11T23:22:32.590 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:32.589+0000 7fc3b005c640 1 -- 172.21.15.26:0/2973489930 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3a81b3c00 con 0x7fc3a81091f0 2024-09-11T23:22:32.590 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:32.589+0000 7fc39effd640 1 -- 172.21.15.26:0/2973489930 <== mon.2 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3a0036040 con 0x7fc3a81091f0 2024-09-11T23:22:32.590 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:32.589+0000 7fc39effd640 1 -- 172.21.15.26:0/2973489930 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3a00458b0 con 0x7fc3a81091f0 2024-09-11T23:22:32.596 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:32.595+0000 7fc39effd640 1 -- 172.21.15.26:0/2973489930 <== mon.2 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fc3a0043020 con 0x7fc3a81091f0 2024-09-11T23:22:32.596 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:32.595+0000 7fc3b2ded740 1 Processor -- start 2024-09-11T23:22:32.597 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:32.596+0000 7fc3b2ded740 1 -- start start 2024-09-11T23:22:32.598 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:32.596+0000 7fc3b2ded740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561397f18cc0 0x561397f190a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:32.598 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:32.597+0000 7fc3b2ded740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561397f195e0 0x561397f23940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:32.598 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:32.597+0000 7fc3b2ded740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561397f23e80 0x561397f26270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:32.598 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:32.597+0000 7fc3b2ded740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x561397e7d2f0 con 0x561397f23e80 2024-09-11T23:22:32.598 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:32.597+0000 7fc3b2ded740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x561397e25270 con 0x561397f195e0 2024-09-11T23:22:32.598 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:32.597+0000 7fc3b2ded740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x561397e792b0 con 0x561397f18cc0 2024-09-11T23:22:32.598 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:32.597+0000 7fc3ad5f7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561397f23e80 0x561397f26270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:32.599 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:32.597+0000 7fc3ad5f7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561397f23e80 0x561397f26270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57104/0 (socket says 172.21.15.26:57104) 2024-09-11T23:22:32.599 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:32.597+0000 7fc3ad5f7640 1 -- 172.21.15.26:0/3567309841 learned_addr learned my addr 172.21.15.26:0/3567309841 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:22:32.599 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:32.597+0000 7fc3ae5f9640 1 --2- 172.21.15.26:0/3567309841 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561397f18cc0 0x561397f190a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:32.600 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:32.597+0000 7fc3addf8640 1 --2- 172.21.15.26:0/3567309841 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561397f195e0 0x561397f23940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:32.600 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:32.598+0000 7fc3addf8640 1 -- 172.21.15.26:0/3567309841 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561397f18cc0 msgr2=0x561397f190a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:22:32.600 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:32.598+0000 7fc3addf8640 1 --2- 172.21.15.26:0/3567309841 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561397f18cc0 0x561397f190a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:32.600 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:32.598+0000 7fc3addf8640 1 -- 172.21.15.26:0/3567309841 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561397f23e80 msgr2=0x561397f26270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:22:32.601 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:32.600+0000 7fc3addf8640 1 --2- 172.21.15.26:0/3567309841 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561397f23e80 0x561397f26270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:32.601 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:32.600+0000 7fc3addf8640 1 -- 172.21.15.26:0/3567309841 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561397e24340 con 0x561397f195e0 2024-09-11T23:22:32.601 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:32.600+0000 7fc3ad5f7640 1 --2- 172.21.15.26:0/3567309841 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561397f23e80 0x561397f26270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:22:32.601 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:32.601+0000 7fc3addf8640 1 --2- 172.21.15.26:0/3567309841 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561397f195e0 0x561397f23940 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fc3a00ed7b0 tx=0x7fc3a00f3f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=845bd7be41f4ea3b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:22:32.602 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:32.601+0000 7fc39effd640 1 --2- 172.21.15.26:0/2973489930 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fc3940a2f00 0x7fc3940a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:32.602 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:32.602+0000 7fc3ae5f9640 1 --2- 172.21.15.26:0/2973489930 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fc3940a2f00 0x7fc3940a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:32.603 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:32.603+0000 7fc3ae5f9640 1 --2- 172.21.15.26:0/2973489930 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fc3940a2f00 0x7fc3940a53c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fc3a806af00 tx=0x7fc398006f50 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:22:32.604 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:32.603+0000 7fc39effd640 1 -- 172.21.15.26:0/2973489930 <== mon.2 v2:172.21.15.76:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fc3a00f00b0 con 0x7fc3a81091f0 2024-09-11T23:22:32.664 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:32.663+0000 7fc393fff640 1 -- 172.21.15.26:0/3567309841 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3a00f22e0 con 0x561397f195e0 2024-09-11T23:22:32.664 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:32.663+0000 7fc393fff640 1 -- 172.21.15.26:0/3567309841 <== mon.2 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3a00fd030 con 0x561397f195e0 2024-09-11T23:22:32.669 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:32.668+0000 7fc3b2ded740 1 -- 172.21.15.26:0/3567309841 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561397f195e0 msgr2=0x561397f23940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:32.669 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:32.668+0000 7fc3b2ded740 1 --2- 172.21.15.26:0/3567309841 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561397f195e0 0x561397f23940 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fc3a00ed7b0 tx=0x7fc3a00f3f90 comp rx=0 tx=0).stop 2024-09-11T23:22:32.669 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:32.668+0000 7fc3b2ded740 1 -- 172.21.15.26:0/3567309841 shutdown_connections 2024-09-11T23:22:32.669 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:32.668+0000 7fc3b2ded740 1 --2- 172.21.15.26:0/3567309841 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561397f195e0 0x561397f23940 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:32.669 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:32.668+0000 7fc3b2ded740 1 --2- 172.21.15.26:0/3567309841 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561397f23e80 0x561397f26270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:32.669 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:32.668+0000 7fc3b2ded740 1 --2- 172.21.15.26:0/3567309841 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561397f18cc0 0x561397f190a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:32.669 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:32.668+0000 7fc3b2ded740 1 -- 172.21.15.26:0/3567309841 >> 172.21.15.26:0/3567309841 conn(0x561397f15230 msgr2=0x561397f15610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:22:32.670 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:32.669+0000 7fc3b2ded740 1 -- 172.21.15.26:0/3567309841 shutdown_connections 2024-09-11T23:22:32.670 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:32.669+0000 7fc3b2ded740 1 -- 172.21.15.26:0/3567309841 wait complete. 2024-09-11T23:22:32.670 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:32.669+0000 7fc3b2ded740 1 -- start start 2024-09-11T23:22:32.670 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:32.669+0000 7fc3b2ded740 1 Processor -- start 2024-09-11T23:22:32.671 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:32.670+0000 7fc3b2ded740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561397f18cc0 0x561397f23920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:32.671 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:32.670+0000 7fc3b2ded740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561397f23e80 0x561397f20bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:32.672 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:32.670+0000 7fc3b2ded740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561397f212b0 0x561397f2fd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:32.672 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:32.670+0000 7fc3b2ded740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x561397f26810 con 0x561397f23e80 2024-09-11T23:22:32.672 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:32.670+0000 7fc3b2ded740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x561397e06290 con 0x561397f18cc0 2024-09-11T23:22:32.672 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:32.670+0000 7fc3b2ded740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x561397f26990 con 0x561397f212b0 2024-09-11T23:22:32.672 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:32.670+0000 7fc3b2ded740 10 client.0 _ll_register_callbacks cb 0x561397e37140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:22:32.672 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:32.671+0000 7fc3ad5f7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561397f23e80 0x561397f20bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:32.672 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:32.671+0000 7fc3ad5f7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561397f23e80 0x561397f20bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57112/0 (socket says 172.21.15.26:57112) 2024-09-11T23:22:32.672 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:32.671+0000 7fc3ad5f7640 1 -- 172.21.15.26:0/1526976596 learned_addr learned my addr 172.21.15.26:0/1526976596 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:22:32.673 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:32.671+0000 7fc3ae5f9640 1 --2- 172.21.15.26:0/1526976596 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561397f212b0 0x561397f2fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:32.673 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:32.671+0000 7fc3addf8640 1 --2- 172.21.15.26:0/1526976596 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561397f18cc0 0x561397f23920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:32.673 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:32.671+0000 7fc3ae5f9640 1 -- 172.21.15.26:0/1526976596 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561397f18cc0 msgr2=0x561397f23920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:22:32.673 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:32.671+0000 7fc3ae5f9640 1 --2- 172.21.15.26:0/1526976596 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561397f18cc0 0x561397f23920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:32.673 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:32.671+0000 7fc3ae5f9640 1 -- 172.21.15.26:0/1526976596 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561397f23e80 msgr2=0x561397f20bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:22:32.673 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:32.672+0000 7fc3ae5f9640 1 --2- 172.21.15.26:0/1526976596 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561397f23e80 0x561397f20bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:32.673 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:32.672+0000 7fc3ae5f9640 1 -- 172.21.15.26:0/1526976596 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561397b90210 con 0x561397f212b0 2024-09-11T23:22:32.673 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:32.672+0000 7fc3ad5f7640 1 --2- 172.21.15.26:0/1526976596 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561397f23e80 0x561397f20bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:22:32.673 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:32.672+0000 7fc3ae5f9640 1 --2- 172.21.15.26:0/1526976596 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561397f212b0 0x561397f2fd60 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fc3980156b0 tx=0x7fc398004770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=64bd486d7ac8a7bb server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:22:32.693 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:32.692+0000 7fc3917fa640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-09-11T23:22:32.693 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:32.692+0000 7fc3917fa640 1 -- 172.21.15.26:0/1526976596 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc398023020 con 0x561397f212b0 2024-09-11T23:22:32.693 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:32.692+0000 7fc3b2ded740 1 -- 172.21.15.26:0/1526976596 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x561397e8aa20 con 0x561397f212b0 2024-09-11T23:22:32.693 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:32.692+0000 7fc3917fa640 1 -- 172.21.15.26:0/1526976596 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fc398013450 con 0x561397f212b0 2024-09-11T23:22:32.694 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:32.693+0000 7fc3917fa640 1 -- 172.21.15.26:0/1526976596 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fc39802e050 con 0x561397f212b0 2024-09-11T23:22:32.694 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:32.693+0000 7fc3b2ded740 10 client.25930 fetch_fsmap learned FSMap version 38 2024-09-11T23:22:32.694 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:32.693+0000 7fc3b2ded740 1 -- 172.21.15.26:0/1526976596 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x561397c5c620 con 0x561397f212b0 2024-09-11T23:22:32.695 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:32.695+0000 7fc3917fa640 1 -- 172.21.15.26:0/1526976596 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x561397e8aa20 con 0x561397f212b0 2024-09-11T23:22:32.697 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:32.696+0000 7fc3917fa640 1 -- 172.21.15.26:0/1526976596 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 38) ==== 8414+0+0 (secure 0 0 0) 0x7fc39801e030 con 0x561397f212b0 2024-09-11T23:22:32.697 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:32.696+0000 7fc3b2ded740 10 client.25930 fetch_fsmap finished waiting for FSMap version 38 2024-09-11T23:22:32.697 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:32.696+0000 7fc3b2ded740 10 client.25930 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:22:32.697 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:32.696+0000 7fc3b2ded740 20 client.25930 populate_metadata read hostname 'smithi026' 2024-09-11T23:22:32.697 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:32.696+0000 7fc3b2ded740 1 --2- 172.21.15.26:0/1526976596 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x561397f45490 0x561397f478d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:32.698 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:32.696+0000 7fc3b2ded740 4 client.25930 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:22:32.698 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:32.696+0000 7fc3b2ded740 1 -- 172.21.15.26:0/1526976596 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561397e84d70 con 0x561397f45490 2024-09-11T23:22:32.698 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:32.697+0000 7fc3addf8640 1 --2- 172.21.15.26:0/1526976596 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x561397f45490 0x561397f478d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:32.698 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:32.697+0000 7fc3addf8640 1 --2- 172.21.15.26:0/1526976596 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x561397f45490 0x561397f478d0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=72ff83e47e5bcbc1 server_cookie=d4fe95f65857e7b9 in_seq=0 out_seq=0 2024-09-11T23:22:32.703 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:32.702+0000 7fc3917fa640 10 client.25930 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:22:32.703 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:32.702+0000 7fc3917fa640 1 -- 172.21.15.26:0/1526976596 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561397e84d70 con 0x561397f45490 2024-09-11T23:22:32.703 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:32.702+0000 7fc3917fa640 10 client.25930 handle_command_reply: tid=0 2024-09-11T23:22:32.703 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:32.702+0000 7fc3b2ded740 1 client.25930 shutdown 2024-09-11T23:22:32.706 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:32.706+0000 7fc3b2ded740 1 -- 172.21.15.26:0/1526976596 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561397f212b0 msgr2=0x561397f2fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:32.707 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:32.706+0000 7fc3b2ded740 1 --2- 172.21.15.26:0/1526976596 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561397f212b0 0x561397f2fd60 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fc3980156b0 tx=0x7fc398004770 comp rx=0 tx=0).stop 2024-09-11T23:22:32.713 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:32.712+0000 7fc3b2ded740 1 -- 172.21.15.26:0/1526976596 shutdown_connections 2024-09-11T23:22:32.713 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:32.712+0000 7fc3b2ded740 1 --2- 172.21.15.26:0/1526976596 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x561397f45490 0x561397f478d0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:32.713 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:32.712+0000 7fc3b2ded740 1 --2- 172.21.15.26:0/1526976596 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561397f212b0 0x561397f2fd60 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:32.713 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:32.712+0000 7fc3b2ded740 1 --2- 172.21.15.26:0/1526976596 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561397f23e80 0x561397f20bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:32.713 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:32.712+0000 7fc3b2ded740 1 --2- 172.21.15.26:0/1526976596 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561397f18cc0 0x561397f23920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:32.713 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:32.712+0000 7fc3b2ded740 1 -- 172.21.15.26:0/1526976596 >> 172.21.15.26:0/1526976596 conn(0x561397f15230 msgr2=0x561397f1a720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:22:32.723 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:32.722+0000 7fc3917fa640 0 client.25930 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:22:32.723 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:32.722+0000 7fc3b2ded740 1 -- 172.21.15.26:0/1526976596 shutdown_connections 2024-09-11T23:22:32.723 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:32.722+0000 7fc3b2ded740 1 -- 172.21.15.26:0/1526976596 wait complete. 2024-09-11T23:22:32.726 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:32.726+0000 7fc3b2ded740 20 client.25930 trim_cache size 0 max 16384 2024-09-11T23:22:32.968 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:32.967+0000 7fc3b2ded740 1 Processor -- start 2024-09-11T23:22:32.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:22:32 smithi136 ceph-mon[35121]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:22:32.989 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:32.989+0000 7fc3b2ded740 1 -- start start 2024-09-11T23:22:32.990 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:32.989+0000 7fc3b2ded740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561397f18cc0 0x561397f20b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:32.991 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:32.990+0000 7fc3addf8640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561397f18cc0 0x561397f20b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:32.991 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:32.990+0000 7fc3addf8640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561397f18cc0 0x561397f20b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57122/0 (socket says 172.21.15.26:57122) 2024-09-11T23:22:32.991 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:32.990+0000 7fc3addf8640 1 -- 172.21.15.26:0/2245529796 learned_addr learned my addr 172.21.15.26:0/2245529796 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:22:32.991 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:32.990+0000 7fc3b2ded740 1 --2- 172.21.15.26:0/2245529796 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561397f212b0 0x561397f2fb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:32.992 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:32.990+0000 7fc3b2ded740 1 --2- 172.21.15.26:0/2245529796 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561397f23e80 0x561397f25270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:32.992 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:32.991+0000 7fc3ad5f7640 1 --2- 172.21.15.26:0/2245529796 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561397f212b0 0x561397f2fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:32.992 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:32.991+0000 7fc3b2ded740 1 -- 172.21.15.26:0/2245529796 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc38c00e720 con 0x561397f18cc0 2024-09-11T23:22:32.992 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:32.991+0000 7fc3b2ded740 1 -- 172.21.15.26:0/2245529796 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fc38c0104d0 con 0x561397f23e80 2024-09-11T23:22:32.992 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:32.991+0000 7fc3ae5f9640 1 --2- 172.21.15.26:0/2245529796 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561397f23e80 0x561397f25270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:32.992 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:32.991+0000 7fc3b2ded740 1 -- 172.21.15.26:0/2245529796 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fc38c00eeb0 con 0x561397f212b0 2024-09-11T23:22:32.993 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:32.992+0000 7fc3addf8640 1 -- 172.21.15.26:0/2245529796 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561397f212b0 msgr2=0x561397f2fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:22:32.994 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:32.992+0000 7fc3addf8640 1 --2- 172.21.15.26:0/2245529796 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561397f212b0 0x561397f2fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:32.994 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:32.992+0000 7fc3addf8640 1 -- 172.21.15.26:0/2245529796 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561397f23e80 msgr2=0x561397f25270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:22:32.994 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:32.992+0000 7fc3addf8640 1 --2- 172.21.15.26:0/2245529796 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561397f23e80 0x561397f25270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:32.994 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:32.992+0000 7fc3addf8640 1 -- 172.21.15.26:0/2245529796 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561397f1b830 con 0x561397f18cc0 2024-09-11T23:22:32.994 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:32.992+0000 7fc3ad5f7640 1 --2- 172.21.15.26:0/2245529796 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561397f212b0 0x561397f2fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:22:32.994 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:32.993+0000 7fc3addf8640 1 --2- 172.21.15.26:0/2245529796 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561397f18cc0 0x561397f20b70 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fc3a002f830 tx=0x7fc3a002a8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a289cfd681338b6b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:22:33.000 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:22:32 smithi026 ceph-mon[31172]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:22:33.015 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:33.014+0000 7fc3937fe640 1 -- 172.21.15.26:0/2245529796 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3a00d5070 con 0x561397f18cc0 2024-09-11T23:22:33.015 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:33.014+0000 7fc3937fe640 1 -- 172.21.15.26:0/2245529796 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3a00fd040 con 0x561397f18cc0 2024-09-11T23:22:33.015 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:33.014+0000 7fc3937fe640 1 -- 172.21.15.26:0/2245529796 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3a00bbee0 con 0x561397f18cc0 2024-09-11T23:22:33.019 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:33.018+0000 7fc3b2ded740 1 -- 172.21.15.26:0/2245529796 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561397f18cc0 msgr2=0x561397f20b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:33.019 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:33.018+0000 7fc3b2ded740 1 --2- 172.21.15.26:0/2245529796 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561397f18cc0 0x561397f20b70 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fc3a002f830 tx=0x7fc3a002a8f0 comp rx=0 tx=0).stop 2024-09-11T23:22:33.021 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:33.021+0000 7fc3b2ded740 1 -- 172.21.15.26:0/2245529796 shutdown_connections 2024-09-11T23:22:33.022 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:33.021+0000 7fc3b2ded740 1 --2- 172.21.15.26:0/2245529796 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561397f23e80 0x561397f25270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:33.022 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:33.021+0000 7fc3b2ded740 1 --2- 172.21.15.26:0/2245529796 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561397f212b0 0x561397f2fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:33.022 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:33.021+0000 7fc3b2ded740 1 --2- 172.21.15.26:0/2245529796 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561397f18cc0 0x561397f20b70 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:33.022 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:33.021+0000 7fc3b2ded740 1 -- 172.21.15.26:0/2245529796 >> 172.21.15.26:0/2245529796 conn(0x561397f15230 msgr2=0x561397f20460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:22:33.026 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:33.025+0000 7fc3b2ded740 1 -- 172.21.15.26:0/2245529796 shutdown_connections 2024-09-11T23:22:33.026 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:33.025+0000 7fc3b2ded740 1 -- 172.21.15.26:0/2245529796 wait complete. 2024-09-11T23:22:33.026 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:33.025+0000 7fc3b2ded740 1 -- start start 2024-09-11T23:22:33.027 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:33.026+0000 7fc3b2ded740 1 Processor -- start 2024-09-11T23:22:33.028 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:33.026+0000 7fc3b2ded740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561397f18cc0 0x561397f4c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:33.028 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:33.027+0000 7fc3addf8640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561397f18cc0 0x561397f4c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:33.029 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:33.027+0000 7fc3addf8640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561397f18cc0 0x561397f4c1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:43768/0 (socket says 172.21.15.26:43768) 2024-09-11T23:22:33.029 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:33.027+0000 7fc3b2ded740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561397f212b0 0x561397f4c710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:33.029 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:33.027+0000 7fc3ad5f7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561397f212b0 0x561397f4c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:33.029 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:33.027+0000 7fc3ad5f7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561397f212b0 0x561397f4c710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57124/0 (socket says 172.21.15.26:57124) 2024-09-11T23:22:33.029 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:33.027+0000 7fc3addf8640 1 -- 172.21.15.26:0/3160012103 learned_addr learned my addr 172.21.15.26:0/3160012103 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:22:33.029 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:33.027+0000 7fc3b2ded740 1 --2- 172.21.15.26:0/3160012103 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561397f23e80 0x561397f4cc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:33.029 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:33.027+0000 7fc3b2ded740 1 -- 172.21.15.26:0/3160012103 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc38c00fdd0 con 0x561397f212b0 2024-09-11T23:22:33.029 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:33.027+0000 7fc3b2ded740 1 -- 172.21.15.26:0/3160012103 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fc38c00f640 con 0x561397f23e80 2024-09-11T23:22:33.031 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:33.027+0000 7fc3b2ded740 1 -- 172.21.15.26:0/3160012103 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fc38c00c7b0 con 0x561397f18cc0 2024-09-11T23:22:33.032 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:33.028+0000 7fc3ae5f9640 1 --2- 172.21.15.26:0/3160012103 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561397f23e80 0x561397f4cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:33.032 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:33.028+0000 7fc3ae5f9640 1 -- 172.21.15.26:0/3160012103 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561397f18cc0 msgr2=0x561397f4c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:22:33.032 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:33.028+0000 7fc3ae5f9640 1 --2- 172.21.15.26:0/3160012103 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561397f18cc0 0x561397f4c1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:33.032 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:33.028+0000 7fc3ae5f9640 1 -- 172.21.15.26:0/3160012103 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561397f212b0 msgr2=0x561397f4c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:22:33.032 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:33.028+0000 7fc3ae5f9640 1 --2- 172.21.15.26:0/3160012103 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561397f212b0 0x561397f4c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:33.032 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:33.028+0000 7fc3b2ded740 10 client.0 _ll_register_callbacks cb 0x7fc39800eed0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:22:33.032 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:33.028+0000 7fc3ae5f9640 1 -- 172.21.15.26:0/3160012103 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561397f2acf0 con 0x561397f23e80 2024-09-11T23:22:33.032 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:33.028+0000 7fc3b2ded740 1 -- 172.21.15.26:0/3160012103 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc3a00fd040 con 0x561397f23e80 2024-09-11T23:22:33.032 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:33.029+0000 7fc3ad5f7640 1 --2- 172.21.15.26:0/3160012103 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561397f212b0 0x561397f4c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:22:33.032 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:33.029+0000 7fc3ae5f9640 1 --2- 172.21.15.26:0/3160012103 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561397f23e80 0x561397f4cc50 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fc398017010 tx=0x7fc39801fd60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c3ec46668a5e9d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:22:33.052 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:22:32 smithi076 ceph-mon[35296]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:22:33.059 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:33.058+0000 7fc3917fa640 10 client.25910 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-11T23:22:33.059 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:33.058+0000 7fc3917fa640 1 -- 172.21.15.26:0/3160012103 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc398036020 con 0x561397f23e80 2024-09-11T23:22:33.059 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:33.058+0000 7fc3917fa640 1 -- 172.21.15.26:0/3160012103 <== mon.2 v2:172.21.15.76:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fc398004020 con 0x561397f23e80 2024-09-11T23:22:33.060 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:33.058+0000 7fc3917fa640 1 -- 172.21.15.26:0/3160012103 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fc398023070 con 0x561397f23e80 2024-09-11T23:22:33.060 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:33.059+0000 7fc3b2ded740 10 client.25910 fetch_fsmap learned FSMap version 38 2024-09-11T23:22:33.060 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:33.059+0000 7fc3b2ded740 1 -- 172.21.15.26:0/3160012103 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({fsmap=38}) -- 0x561397f2bdb0 con 0x561397f23e80 2024-09-11T23:22:33.061 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:33.060+0000 7fc3917fa640 1 -- 172.21.15.26:0/3160012103 <== mon.2 v2:172.21.15.76:3300/0 4 ==== fsmap(e 38) ==== 8414+0+0 (secure 0 0 0) 0x7fc398039020 con 0x561397f23e80 2024-09-11T23:22:33.064 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:33.063+0000 7fc3b2ded740 10 client.25910 fetch_fsmap finished waiting for FSMap version 38 2024-09-11T23:22:33.065 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:33.063+0000 7fc3b2ded740 10 client.25910 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:22:33.065 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:33.063+0000 7fc3b2ded740 20 client.25910 populate_metadata read hostname 'smithi026' 2024-09-11T23:22:33.065 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:33.063+0000 7fc3b2ded740 1 --2- 172.21.15.26:0/3160012103 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x561397f2c400 0x561397fb21c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:33.065 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:33.064+0000 7fc3b2ded740 4 client.25910 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T23:22:33.065 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:33.064+0000 7fc3b2ded740 1 -- 172.21.15.26:0/3160012103 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561397e01900 con 0x561397f2c400 2024-09-11T23:22:33.067 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:33.067+0000 7fc3addf8640 1 --2- 172.21.15.26:0/3160012103 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x561397f2c400 0x561397fb21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:33.069 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:33.068+0000 7fc3addf8640 1 --2- 172.21.15.26:0/3160012103 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x561397f2c400 0x561397fb21c0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20d6d82feebb7136 server_cookie=804b30428f7049dd in_seq=0 out_seq=0 2024-09-11T23:22:33.069 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:33.069+0000 7fc3917fa640 10 client.25910 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:22:33.577 INFO:tasks.workunit.client.0.smithi026.stdout: 357 1858059 1892 1279875 1438 377283 258 2024-09-11T23:22:33.913 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:22:33 smithi026 ceph-mon[31172]: pgmap v128: 97 pgs: 97 active+clean; 1.1 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 27 MiB/s wr, 1.12k op/s 2024-09-11T23:22:33.914 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:22:33 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:22:33.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:22:33 smithi136 ceph-mon[35121]: pgmap v128: 97 pgs: 97 active+clean; 1.1 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 27 MiB/s wr, 1.12k op/s 2024-09-11T23:22:33.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:22:33 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:22:34.052 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:22:33 smithi076 ceph-mon[35296]: pgmap v128: 97 pgs: 97 active+clean; 1.1 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 27 MiB/s wr, 1.12k op/s 2024-09-11T23:22:34.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:22:33 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:22:34.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.131+0000 7fccabd4f640 1 -- 172.21.15.26:0/599606999 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcca4109830 msgr2=0x7fcca4109c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:34.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.131+0000 7fccabd4f640 1 --2- 172.21.15.26:0/599606999 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcca4109830 0x7fcca4109c90 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fcca0009900 tx=0x7fcca002a440 comp rx=0 tx=0).stop 2024-09-11T23:22:34.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.133+0000 7fccabd4f640 1 -- 172.21.15.26:0/599606999 shutdown_connections 2024-09-11T23:22:34.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.133+0000 7fccabd4f640 1 --2- 172.21.15.26:0/599606999 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcca411b140 0x7fcca411d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:34.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.133+0000 7fccabd4f640 1 --2- 172.21.15.26:0/599606999 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcca4109830 0x7fcca4109c90 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:34.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.133+0000 7fccabd4f640 1 --2- 172.21.15.26:0/599606999 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcca4109450 0x7fcca4110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:34.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.133+0000 7fccabd4f640 1 -- 172.21.15.26:0/599606999 >> 172.21.15.26:0/599606999 conn(0x7fcca406e140 msgr2=0x7fcca406e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:22:34.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.135+0000 7fccabd4f640 1 -- 172.21.15.26:0/599606999 shutdown_connections 2024-09-11T23:22:34.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.139+0000 7fccabd4f640 1 -- 172.21.15.26:0/599606999 wait complete. 2024-09-11T23:22:34.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.139+0000 7fccabd4f640 1 Processor -- start 2024-09-11T23:22:34.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.145+0000 7fccabd4f640 1 -- start start 2024-09-11T23:22:34.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.145+0000 7fccabd4f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcca4109450 0x7fcca41a7250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:34.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.145+0000 7fccabd4f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcca4109830 0x7fcca41a7790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:34.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.145+0000 7fccabd4f640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcca411b140 0x7fcca41abb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:34.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.145+0000 7fccabd4f640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fcca4120df0 con 0x7fcca4109450 2024-09-11T23:22:34.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.145+0000 7fccabd4f640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fcca4120c70 con 0x7fcca4109830 2024-09-11T23:22:34.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.145+0000 7fccabd4f640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fcca4120f70 con 0x7fcca411b140 2024-09-11T23:22:34.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.146+0000 7fcca92ea640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcca4109830 0x7fcca41a7790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:34.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.146+0000 7fcca92ea640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcca4109830 0x7fcca41a7790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:45182/0 (socket says 172.21.15.26:45182) 2024-09-11T23:22:34.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.146+0000 7fcca92ea640 1 -- 172.21.15.26:0/2964718668 learned_addr learned my addr 172.21.15.26:0/2964718668 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:22:34.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.147+0000 7fcca92ea640 1 -- 172.21.15.26:0/2964718668 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcca411b140 msgr2=0x7fcca41abb20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-11T23:22:34.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.147+0000 7fcca92ea640 1 --2- 172.21.15.26:0/2964718668 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcca411b140 0x7fcca41abb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:34.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.147+0000 7fcca92ea640 1 -- 172.21.15.26:0/2964718668 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcca4109450 msgr2=0x7fcca41a7250 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-11T23:22:34.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.147+0000 7fcca92ea640 1 --2- 172.21.15.26:0/2964718668 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcca4109450 0x7fcca41a7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:34.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.147+0000 7fcca92ea640 1 -- 172.21.15.26:0/2964718668 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcca41ac200 con 0x7fcca4109830 2024-09-11T23:22:34.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.148+0000 7fcca92ea640 1 --2- 172.21.15.26:0/2964718668 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcca4109830 0x7fcca41a7790 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fcca0005ab0 tx=0x7fcca00052b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:22:34.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.150+0000 7fcc9affd640 1 -- 172.21.15.26:0/2964718668 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcca0040030 con 0x7fcca4109830 2024-09-11T23:22:34.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.150+0000 7fcc9affd640 1 -- 172.21.15.26:0/2964718668 <== mon.2 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcca0004a30 con 0x7fcca4109830 2024-09-11T23:22:34.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.150+0000 7fccabd4f640 1 -- 172.21.15.26:0/2964718668 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcca41ac490 con 0x7fcca4109830 2024-09-11T23:22:34.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.151+0000 7fcc9affd640 1 -- 172.21.15.26:0/2964718668 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcca003c940 con 0x7fcca4109830 2024-09-11T23:22:34.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.154+0000 7fccabd4f640 1 -- 172.21.15.26:0/2964718668 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcca41b3dd0 con 0x7fcca4109830 2024-09-11T23:22:34.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.159+0000 7fccae128740 1 Processor -- start 2024-09-11T23:22:34.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.159+0000 7fccae128740 1 -- start start 2024-09-11T23:22:34.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.163+0000 7fccae128740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ac4de10850 0x55ac4de10c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:34.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.163+0000 7fccae128740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ac4de11170 0x55ac4de1b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:34.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.163+0000 7fccae128740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ac4de1ba30 0x55ac4de1de20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:34.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.163+0000 7fccae128740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55ac4dd75160 con 0x55ac4de1ba30 2024-09-11T23:22:34.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.163+0000 7fccae128740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55ac4dd1c720 con 0x55ac4de11170 2024-09-11T23:22:34.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.163+0000 7fccae128740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55ac4dd71120 con 0x55ac4de10850 2024-09-11T23:22:34.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.164+0000 7fccaa2ec640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ac4de10850 0x55ac4de10c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:34.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.164+0000 7fccaa2ec640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ac4de10850 0x55ac4de10c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:43776/0 (socket says 172.21.15.26:43776) 2024-09-11T23:22:34.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.164+0000 7fccaa2ec640 1 -- 172.21.15.26:0/3673510559 learned_addr learned my addr 172.21.15.26:0/3673510559 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:22:34.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.164+0000 7fcc9affd640 1 -- 172.21.15.26:0/2964718668 <== mon.2 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fcca0043030 con 0x7fcca4109830 2024-09-11T23:22:34.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.165+0000 7fcca9aeb640 1 --2- 172.21.15.26:0/3673510559 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ac4de11170 0x55ac4de1b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:34.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.165+0000 7fcca9aeb640 1 -- 172.21.15.26:0/3673510559 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ac4de10850 msgr2=0x55ac4de10c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:22:34.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.165+0000 7fcca9aeb640 1 --2- 172.21.15.26:0/3673510559 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ac4de10850 0x55ac4de10c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:34.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.165+0000 7fcca9aeb640 1 -- 172.21.15.26:0/3673510559 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ac4de1ba30 msgr2=0x55ac4de1de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:22:34.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.165+0000 7fcca9aeb640 1 --2- 172.21.15.26:0/3673510559 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ac4de1ba30 0x55ac4de1de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:34.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.165+0000 7fcca9aeb640 1 -- 172.21.15.26:0/3673510559 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac4dae6ba0 con 0x55ac4de11170 2024-09-11T23:22:34.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.165+0000 7fcca9aeb640 1 --2- 172.21.15.26:0/3673510559 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ac4de11170 0x55ac4de1b4f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fcc9400a930 tx=0x7fcc9400adf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=68309aa07d2e8703 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:22:34.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.166+0000 7fcc9affd640 1 --2- 172.21.15.26:0/2964718668 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fcc900a2ec0 0x7fcc900a5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:34.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.166+0000 7fcc9affd640 1 -- 172.21.15.26:0/2964718668 <== mon.2 v2:172.21.15.76:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fcca00ef690 con 0x7fcca4109830 2024-09-11T23:22:34.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.166+0000 7fcc8ffff640 1 -- 172.21.15.26:0/3673510559 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc9400cd80 con 0x55ac4de11170 2024-09-11T23:22:34.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.166+0000 7fcc8ffff640 1 -- 172.21.15.26:0/3673510559 <== mon.2 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc9400f930 con 0x55ac4de11170 2024-09-11T23:22:34.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.166+0000 7fcc8ffff640 1 -- 172.21.15.26:0/3673510559 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc94018950 con 0x55ac4de11170 2024-09-11T23:22:34.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.168+0000 7fccae128740 1 -- 172.21.15.26:0/3673510559 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ac4de11170 msgr2=0x55ac4de1b4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:34.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.168+0000 7fccae128740 1 --2- 172.21.15.26:0/3673510559 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ac4de11170 0x55ac4de1b4f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fcc9400a930 tx=0x7fcc9400adf0 comp rx=0 tx=0).stop 2024-09-11T23:22:34.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.168+0000 7fccae128740 1 -- 172.21.15.26:0/3673510559 shutdown_connections 2024-09-11T23:22:34.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.168+0000 7fccae128740 1 --2- 172.21.15.26:0/3673510559 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ac4de11170 0x55ac4de1b4f0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:34.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.168+0000 7fccae128740 1 --2- 172.21.15.26:0/3673510559 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ac4de1ba30 0x55ac4de1de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:34.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.168+0000 7fccae128740 1 --2- 172.21.15.26:0/3673510559 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ac4de10850 0x55ac4de10c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:34.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.168+0000 7fccae128740 1 -- 172.21.15.26:0/3673510559 >> 172.21.15.26:0/3673510559 conn(0x55ac4de0cde0 msgr2=0x55ac4de0d1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:22:34.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.170+0000 7fccae128740 1 -- 172.21.15.26:0/3673510559 shutdown_connections 2024-09-11T23:22:34.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.170+0000 7fccaa2ec640 1 --2- 172.21.15.26:0/2964718668 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fcc900a2ec0 0x7fcc900a5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:34.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.171+0000 7fccaa2ec640 1 --2- 172.21.15.26:0/2964718668 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fcc900a2ec0 0x7fcc900a5380 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fcc9c005c40 tx=0x7fcc9c009570 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:22:34.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.171+0000 7fccae128740 1 -- 172.21.15.26:0/3673510559 wait complete. 2024-09-11T23:22:34.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.171+0000 7fccae128740 1 -- start start 2024-09-11T23:22:34.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.171+0000 7fccae128740 1 Processor -- start 2024-09-11T23:22:34.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.172+0000 7fccae128740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ac4de10850 0x55ac4de18950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:34.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.172+0000 7fccae128740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ac4de11170 0x55ac4de18e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:34.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.172+0000 7fccae128740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ac4de1ba30 0x55ac4de193d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:34.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.172+0000 7fccae128740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55ac4de1e600 con 0x55ac4de11170 2024-09-11T23:22:34.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.172+0000 7fccae128740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55ac4dcfd900 con 0x55ac4de1ba30 2024-09-11T23:22:34.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.172+0000 7fccae128740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55ac4de1e780 con 0x55ac4de10850 2024-09-11T23:22:34.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.172+0000 7fccae128740 10 client.0 _ll_register_callbacks cb 0x55ac4dcfb780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:22:34.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.172+0000 7fcca9aeb640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ac4de10850 0x55ac4de18950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:34.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.172+0000 7fcca9aeb640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ac4de10850 0x55ac4de18950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:43786/0 (socket says 172.21.15.26:43786) 2024-09-11T23:22:34.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.172+0000 7fcca9aeb640 1 -- 172.21.15.26:0/543819426 learned_addr learned my addr 172.21.15.26:0/543819426 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:22:34.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.172+0000 7fccaa2ec640 1 --2- 172.21.15.26:0/543819426 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ac4de1ba30 0x55ac4de193d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:34.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.173+0000 7fcca9aeb640 1 -- 172.21.15.26:0/543819426 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ac4de1ba30 msgr2=0x55ac4de193d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:22:34.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.173+0000 7fcca9aeb640 1 --2- 172.21.15.26:0/543819426 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ac4de1ba30 0x55ac4de193d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:34.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.173+0000 7fcca9aeb640 1 -- 172.21.15.26:0/543819426 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ac4de11170 msgr2=0x55ac4de18e90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T23:22:34.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.173+0000 7fcca9aeb640 1 --2- 172.21.15.26:0/543819426 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ac4de11170 0x55ac4de18e90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:34.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.173+0000 7fcca9aeb640 1 -- 172.21.15.26:0/543819426 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac4dd3dc50 con 0x55ac4de10850 2024-09-11T23:22:34.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.173+0000 7fcca9aeb640 1 --2- 172.21.15.26:0/543819426 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ac4de10850 0x55ac4de18950 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fcc9400d010 tx=0x7fcc9400a4e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1449d388249613c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:22:34.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.173+0000 7fcc8d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-09-11T23:22:34.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.175+0000 7fcc8d7fa640 1 -- 172.21.15.26:0/543819426 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc9401d070 con 0x55ac4de10850 2024-09-11T23:22:34.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.175+0000 7fccae128740 1 -- 172.21.15.26:0/543819426 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc9400f930 con 0x55ac4de10850 2024-09-11T23:22:34.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.175+0000 7fcc8d7fa640 1 -- 172.21.15.26:0/543819426 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fcc94018670 con 0x55ac4de10850 2024-09-11T23:22:34.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.175+0000 7fcc8d7fa640 1 -- 172.21.15.26:0/543819426 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fcc94018870 con 0x55ac4de10850 2024-09-11T23:22:34.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.176+0000 7fccae128740 10 client.25942 fetch_fsmap learned FSMap version 38 2024-09-11T23:22:34.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.176+0000 7fccae128740 1 -- 172.21.15.26:0/543819426 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55ac4de2a320 con 0x55ac4de10850 2024-09-11T23:22:34.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.176+0000 7fcc8d7fa640 1 -- 172.21.15.26:0/543819426 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc9400f930 con 0x55ac4de10850 2024-09-11T23:22:34.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.177+0000 7fcc8d7fa640 1 -- 172.21.15.26:0/543819426 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 38) ==== 8414+0+0 (secure 0 0 0) 0x7fcc9402b860 con 0x55ac4de10850 2024-09-11T23:22:34.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.177+0000 7fccae128740 10 client.25942 fetch_fsmap finished waiting for FSMap version 38 2024-09-11T23:22:34.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.177+0000 7fccae128740 10 client.25942 resolve_mds: validated gid 24485 aka daemon mds.b 2024-09-11T23:22:34.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.177+0000 7fccae128740 20 client.25942 populate_metadata read hostname 'smithi026' 2024-09-11T23:22:34.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.177+0000 7fccae128740 1 --2- 172.21.15.26:0/543819426 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55ac4de2baa0 0x55ac4de2def0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:34.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.177+0000 7fccae128740 4 client.25942 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:22:34.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.177+0000 7fccae128740 1 -- 172.21.15.26:0/543819426 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ac4dd82890 con 0x55ac4de2baa0 2024-09-11T23:22:34.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.183+0000 7fcca92ea640 1 --2- 172.21.15.26:0/543819426 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55ac4de2baa0 0x55ac4de2def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:34.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.194+0000 7fcca92ea640 1 --2- 172.21.15.26:0/543819426 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55ac4de2baa0 0x55ac4de2def0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a9a5c254dc640b1 server_cookie=a72570437947b6a7 in_seq=0 out_seq=0 2024-09-11T23:22:34.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.194+0000 7fcc8d7fa640 10 client.25942 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:22:34.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.200+0000 7fcc8d7fa640 1 -- 172.21.15.26:0/543819426 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ac4dd82890 con 0x55ac4de2baa0 2024-09-11T23:22:34.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.200+0000 7fcc8d7fa640 10 client.25942 handle_command_reply: tid=0 2024-09-11T23:22:34.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.200+0000 7fccae128740 1 client.25942 shutdown 2024-09-11T23:22:34.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.200+0000 7fccae128740 1 -- 172.21.15.26:0/543819426 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ac4de10850 msgr2=0x55ac4de18950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:34.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.200+0000 7fccae128740 1 --2- 172.21.15.26:0/543819426 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ac4de10850 0x55ac4de18950 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fcc9400d010 tx=0x7fcc9400a4e0 comp rx=0 tx=0).stop 2024-09-11T23:22:34.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.200+0000 7fccae128740 1 -- 172.21.15.26:0/543819426 shutdown_connections 2024-09-11T23:22:34.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.200+0000 7fccae128740 1 --2- 172.21.15.26:0/543819426 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ac4de1ba30 0x55ac4de193d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:34.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.200+0000 7fccae128740 1 --2- 172.21.15.26:0/543819426 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55ac4de2baa0 0x55ac4de2def0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:34.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.200+0000 7fccae128740 1 --2- 172.21.15.26:0/543819426 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ac4de11170 0x55ac4de18e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:34.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.200+0000 7fccae128740 1 --2- 172.21.15.26:0/543819426 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ac4de10850 0x55ac4de18950 secure :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fcc9400d010 tx=0x7fcc9400a4e0 comp rx=0 tx=0).stop 2024-09-11T23:22:34.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.200+0000 7fccae128740 1 -- 172.21.15.26:0/543819426 >> 172.21.15.26:0/543819426 conn(0x55ac4de0cde0 msgr2=0x55ac4de12ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:22:34.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.200+0000 7fcc8d7fa640 0 client.25942 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:22:34.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.201+0000 7fccae128740 1 -- 172.21.15.26:0/543819426 shutdown_connections 2024-09-11T23:22:34.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.202+0000 7fccae128740 1 -- 172.21.15.26:0/543819426 wait complete. 2024-09-11T23:22:34.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.204+0000 7fccae128740 20 client.25942 trim_cache size 0 max 16384 2024-09-11T23:22:34.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.255+0000 7fccae128740 1 Processor -- start 2024-09-11T23:22:34.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.256+0000 7fccae128740 1 -- start start 2024-09-11T23:22:34.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.256+0000 7fccae128740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ac4de11170 0x55ac4de191f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:34.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.256+0000 7fccae128740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ac4de1ba30 0x55ac4de0bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:34.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.256+0000 7fccae128740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ac4de0c2f0 0x55ac4de337d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:34.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.256+0000 7fccae128740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fcca41bc810 con 0x55ac4de1ba30 2024-09-11T23:22:34.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.256+0000 7fccae128740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fcca41be5c0 con 0x55ac4de0c2f0 2024-09-11T23:22:34.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.256+0000 7fccae128740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fcca41bcfa0 con 0x55ac4de11170 2024-09-11T23:22:34.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.258+0000 7fcca9aeb640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ac4de11170 0x55ac4de191f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:34.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.260+0000 7fccaa2ec640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ac4de0c2f0 0x55ac4de337d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:34.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.260+0000 7fccaa2ec640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ac4de0c2f0 0x55ac4de337d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:45210/0 (socket says 172.21.15.26:45210) 2024-09-11T23:22:34.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.260+0000 7fccaa2ec640 1 -- 172.21.15.26:0/664454583 learned_addr learned my addr 172.21.15.26:0/664454583 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:22:34.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.260+0000 7fccaa2ec640 1 -- 172.21.15.26:0/664454583 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ac4de11170 msgr2=0x55ac4de191f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:22:34.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.260+0000 7fccaa2ec640 1 --2- 172.21.15.26:0/664454583 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ac4de11170 0x55ac4de191f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:34.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.260+0000 7fccaa2ec640 1 -- 172.21.15.26:0/664454583 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ac4de1ba30 msgr2=0x55ac4de0bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:22:34.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.260+0000 7fccaa2ec640 1 --2- 172.21.15.26:0/664454583 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ac4de1ba30 0x55ac4de0bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:34.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.260+0000 7fccaa2ec640 1 -- 172.21.15.26:0/664454583 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac4de083f0 con 0x55ac4de0c2f0 2024-09-11T23:22:34.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.260+0000 7fccaa2ec640 1 --2- 172.21.15.26:0/664454583 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ac4de0c2f0 0x55ac4de337d0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fcc9c0126e0 tx=0x7fcc9c014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=65bdf2009bbfe640 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:22:34.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.262+0000 7fcc8f7fe640 1 -- 172.21.15.26:0/664454583 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc9c003c30 con 0x55ac4de0c2f0 2024-09-11T23:22:34.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.262+0000 7fcc8f7fe640 1 -- 172.21.15.26:0/664454583 <== mon.2 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc9c024070 con 0x55ac4de0c2f0 2024-09-11T23:22:34.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.263+0000 7fcc8f7fe640 1 -- 172.21.15.26:0/664454583 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc9c01fac0 con 0x55ac4de0c2f0 2024-09-11T23:22:34.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.264+0000 7fccae128740 1 -- 172.21.15.26:0/664454583 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ac4de0c2f0 msgr2=0x55ac4de337d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:34.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.264+0000 7fccae128740 1 --2- 172.21.15.26:0/664454583 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ac4de0c2f0 0x55ac4de337d0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fcc9c0126e0 tx=0x7fcc9c014000 comp rx=0 tx=0).stop 2024-09-11T23:22:34.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.265+0000 7fccae128740 1 -- 172.21.15.26:0/664454583 shutdown_connections 2024-09-11T23:22:34.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.265+0000 7fccae128740 1 --2- 172.21.15.26:0/664454583 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ac4de0c2f0 0x55ac4de337d0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:34.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.265+0000 7fccae128740 1 --2- 172.21.15.26:0/664454583 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ac4de1ba30 0x55ac4de0bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:34.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.265+0000 7fccae128740 1 --2- 172.21.15.26:0/664454583 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ac4de11170 0x55ac4de191f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:34.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.265+0000 7fccae128740 1 -- 172.21.15.26:0/664454583 >> 172.21.15.26:0/664454583 conn(0x55ac4de0cde0 msgr2=0x55ac4de0b6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:22:34.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.268+0000 7fccae128740 1 -- 172.21.15.26:0/664454583 shutdown_connections 2024-09-11T23:22:34.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.268+0000 7fccae128740 1 -- 172.21.15.26:0/664454583 wait complete. 2024-09-11T23:22:34.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.269+0000 7fccae128740 1 -- start start 2024-09-11T23:22:34.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.269+0000 7fccae128740 1 Processor -- start 2024-09-11T23:22:34.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.270+0000 7fccae128740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ac4de0c2f0 0x55ac4de1fa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:34.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.270+0000 7fccae128740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ac4de11170 0x55ac4de1ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:34.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.270+0000 7fccae128740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ac4de1ba30 0x55ac4de204f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:34.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.270+0000 7fcca9aeb640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ac4de0c2f0 0x55ac4de1fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:34.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.270+0000 7fcca9aeb640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ac4de0c2f0 0x55ac4de1fa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:45224/0 (socket says 172.21.15.26:45224) 2024-09-11T23:22:34.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.271+0000 7fcca9aeb640 1 -- 172.21.15.26:0/3207599342 learned_addr learned my addr 172.21.15.26:0/3207599342 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:22:34.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.271+0000 7fccae128740 1 -- 172.21.15.26:0/3207599342 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fcca41bdec0 con 0x55ac4de1ba30 2024-09-11T23:22:34.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.271+0000 7fccae128740 1 -- 172.21.15.26:0/3207599342 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fcca41bd730 con 0x55ac4de0c2f0 2024-09-11T23:22:34.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.271+0000 7fccae128740 1 -- 172.21.15.26:0/3207599342 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fcca41ba7f0 con 0x55ac4de11170 2024-09-11T23:22:34.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.271+0000 7fcca92ea640 1 --2- 172.21.15.26:0/3207599342 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ac4de11170 0x55ac4de1ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:34.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.271+0000 7fccae128740 10 client.0 _ll_register_callbacks cb 0x7fcca4110870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:22:34.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.271+0000 7fccaa2ec640 1 --2- 172.21.15.26:0/3207599342 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ac4de1ba30 0x55ac4de204f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:34.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.272+0000 7fcca9aeb640 1 -- 172.21.15.26:0/3207599342 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ac4de11170 msgr2=0x55ac4de1ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:22:34.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.272+0000 7fcca9aeb640 1 --2- 172.21.15.26:0/3207599342 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ac4de11170 0x55ac4de1ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:34.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.272+0000 7fcca9aeb640 1 -- 172.21.15.26:0/3207599342 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ac4de1ba30 msgr2=0x55ac4de204f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:22:34.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.272+0000 7fcca9aeb640 1 --2- 172.21.15.26:0/3207599342 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ac4de1ba30 0x55ac4de204f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:34.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.272+0000 7fcca9aeb640 1 -- 172.21.15.26:0/3207599342 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac4de0d990 con 0x55ac4de0c2f0 2024-09-11T23:22:34.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.272+0000 7fcca92ea640 1 --2- 172.21.15.26:0/3207599342 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ac4de11170 0x55ac4de1ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:22:34.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.272+0000 7fcca9aeb640 1 --2- 172.21.15.26:0/3207599342 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ac4de0c2f0 0x55ac4de1fa50 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fcc9400ae30 tx=0x7fcc9400c750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9f73bc0850af230f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:22:34.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.272+0000 7fcc8d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-11T23:22:34.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.273+0000 7fcc8d7fa640 1 -- 172.21.15.26:0/3207599342 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc9401d070 con 0x55ac4de0c2f0 2024-09-11T23:22:34.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.273+0000 7fcc8d7fa640 1 -- 172.21.15.26:0/3207599342 <== mon.2 v2:172.21.15.76:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fcc9400f660 con 0x55ac4de0c2f0 2024-09-11T23:22:34.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.273+0000 7fccae128740 1 -- 172.21.15.26:0/3207599342 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc9c024070 con 0x55ac4de0c2f0 2024-09-11T23:22:34.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.273+0000 7fcc8d7fa640 1 -- 172.21.15.26:0/3207599342 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fcc9400f860 con 0x55ac4de0c2f0 2024-09-11T23:22:34.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.275+0000 7fccae128740 10 client.25925 fetch_fsmap learned FSMap version 38 2024-09-11T23:22:34.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.275+0000 7fccae128740 1 -- 172.21.15.26:0/3207599342 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55ac4dea7b90 con 0x55ac4de0c2f0 2024-09-11T23:22:34.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.276+0000 7fcc8d7fa640 1 -- 172.21.15.26:0/3207599342 <== mon.2 v2:172.21.15.76:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc9c024070 con 0x55ac4de0c2f0 2024-09-11T23:22:34.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.276+0000 7fcc8d7fa640 1 -- 172.21.15.26:0/3207599342 <== mon.2 v2:172.21.15.76:3300/0 5 ==== fsmap(e 38) ==== 8414+0+0 (secure 0 0 0) 0x7fcc94004c10 con 0x55ac4de0c2f0 2024-09-11T23:22:34.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.276+0000 7fccae128740 10 client.25925 fetch_fsmap finished waiting for FSMap version 38 2024-09-11T23:22:34.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.276+0000 7fccae128740 10 client.25925 resolve_mds: validated gid 24485 aka daemon mds.b 2024-09-11T23:22:34.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.276+0000 7fccae128740 20 client.25925 populate_metadata read hostname 'smithi026' 2024-09-11T23:22:34.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.276+0000 7fccae128740 1 --2- 172.21.15.26:0/3207599342 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55ac4dea8230 0x55ac4deaa650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:34.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.277+0000 7fccae128740 4 client.25925 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.15, "await_for": 45.0}] 2024-09-11T23:22:34.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.277+0000 7fccae128740 1 -- 172.21.15.26:0/3207599342 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.15, "await_for": 45.0}) -- 0x55ac4dd7cbe0 con 0x55ac4dea8230 2024-09-11T23:22:34.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.277+0000 7fcca92ea640 1 --2- 172.21.15.26:0/3207599342 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55ac4dea8230 0x55ac4deaa650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:34.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.281+0000 7fcca92ea640 1 --2- 172.21.15.26:0/3207599342 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55ac4dea8230 0x55ac4deaa650 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=154865e533123c53 server_cookie=790270683a3bbc14 in_seq=0 out_seq=0 2024-09-11T23:22:34.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:34.283+0000 7fcc8d7fa640 10 client.25925 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:22:34.930 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:34.928+0000 7fc3917fa640 1 -- 172.21.15.26:0/3160012103 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561397e01900 con 0x561397f2c400 2024-09-11T23:22:34.930 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:34.928+0000 7fc3917fa640 10 client.25910 handle_command_reply: tid=0 2024-09-11T23:22:34.930 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:34.928+0000 7fc3b2ded740 1 client.25910 shutdown 2024-09-11T23:22:34.930 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:34.928+0000 7fc3b2ded740 1 -- 172.21.15.26:0/3160012103 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561397f23e80 msgr2=0x561397f4cc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:34.931 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:34.928+0000 7fc3b2ded740 1 --2- 172.21.15.26:0/3160012103 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561397f23e80 0x561397f4cc50 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fc398017010 tx=0x7fc39801fd60 comp rx=0 tx=0).stop 2024-09-11T23:22:34.931 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:34.929+0000 7fc3b2ded740 1 -- 172.21.15.26:0/3160012103 shutdown_connections 2024-09-11T23:22:34.931 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:34.929+0000 7fc3b2ded740 1 --2- 172.21.15.26:0/3160012103 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561397f23e80 0x561397f4cc50 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:34.931 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:34.929+0000 7fc3b2ded740 1 --2- 172.21.15.26:0/3160012103 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x561397f2c400 0x561397fb21c0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:34.931 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:34.929+0000 7fc3b2ded740 1 --2- 172.21.15.26:0/3160012103 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561397f212b0 0x561397f4c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:34.931 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:34.929+0000 7fc3917fa640 0 client.25910 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:22:34.931 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:34.929+0000 7fc3b2ded740 1 --2- 172.21.15.26:0/3160012103 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561397f18cc0 0x561397f4c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:34.931 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:34.929+0000 7fc3b2ded740 1 -- 172.21.15.26:0/3160012103 >> 172.21.15.26:0/3160012103 conn(0x561397f15230 msgr2=0x561397f1f470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:22:34.931 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:34.929+0000 7fc3b2ded740 1 -- 172.21.15.26:0/3160012103 shutdown_connections 2024-09-11T23:22:34.931 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:34.929+0000 7fc3b2ded740 1 -- 172.21.15.26:0/3160012103 wait complete. 2024-09-11T23:22:34.931 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:34.929+0000 7fc3b2ded740 20 client.25910 trim_cache size 0 max 16384 2024-09-11T23:22:34.932 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-09-11T23:22:34.932 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "message": "", 2024-09-11T23:22:34.932 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-09-11T23:22:34.932 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-09-11T23:22:34.933 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:34.930+0000 7fc3b005c640 1 -- 172.21.15.26:0/2973489930 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fc3940a2f00 msgr2=0x7fc3940a53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:34.933 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:34.930+0000 7fc3b005c640 1 --2- 172.21.15.26:0/2973489930 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fc3940a2f00 0x7fc3940a53c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fc3a806af00 tx=0x7fc398006f50 comp rx=0 tx=0).stop 2024-09-11T23:22:34.933 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:34.930+0000 7fc3b005c640 1 -- 172.21.15.26:0/2973489930 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc3a81091f0 msgr2=0x7fc3a81a7090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:34.933 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:34.930+0000 7fc3b005c640 1 --2- 172.21.15.26:0/2973489930 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc3a81091f0 0x7fc3a81a7090 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fc3a002f860 tx=0x7fc3a002f890 comp rx=0 tx=0).stop 2024-09-11T23:22:34.933 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:34.930+0000 7fc3b005c640 1 -- 172.21.15.26:0/2973489930 shutdown_connections 2024-09-11T23:22:34.934 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:34.930+0000 7fc3b005c640 1 --2- 172.21.15.26:0/2973489930 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fc3940a2f00 0x7fc3940a53c0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:34.934 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:34.930+0000 7fc3b005c640 1 --2- 172.21.15.26:0/2973489930 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc3a811b140 0x7fc3a81ab950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:34.934 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:34.930+0000 7fc3b005c640 1 --2- 172.21.15.26:0/2973489930 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc3a81095d0 0x7fc3a81a75d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:34.934 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:34.930+0000 7fc3b005c640 1 --2- 172.21.15.26:0/2973489930 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc3a81091f0 0x7fc3a81a7090 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:34.934 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:34.930+0000 7fc3b005c640 1 -- 172.21.15.26:0/2973489930 >> 172.21.15.26:0/2973489930 conn(0x7fc3a806cc60 msgr2=0x7fc3a811b8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:22:34.934 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:34.930+0000 7fc3b005c640 1 -- 172.21.15.26:0/2973489930 shutdown_connections 2024-09-11T23:22:34.934 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:34.930+0000 7fc3b005c640 1 -- 172.21.15.26:0/2973489930 wait complete. 2024-09-11T23:22:35.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:22:35 smithi136 ceph-mon[35121]: pgmap v129: 97 pgs: 97 active+clean; 1.1 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 29 MiB/s wr, 1.28k op/s 2024-09-11T23:22:35.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:22:35 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:22:35.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:22:35 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:22:35.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:22:35 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:22:36.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:22:35 smithi026 ceph-mon[31172]: pgmap v129: 97 pgs: 97 active+clean; 1.1 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 29 MiB/s wr, 1.28k op/s 2024-09-11T23:22:36.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:22:35 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:22:36.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:22:35 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:22:36.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:22:35 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:22:36.052 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:22:35 smithi076 ceph-mon[35296]: pgmap v129: 97 pgs: 97 active+clean; 1.1 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 29 MiB/s wr, 1.28k op/s 2024-09-11T23:22:36.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:22:35 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:22:36.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:22:35 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:22:36.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:22:35 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:22:37.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:22:37 smithi136 ceph-mon[35121]: pgmap v130: 97 pgs: 97 active+clean; 1.1 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 20 MiB/s wr, 1.05k op/s 2024-09-11T23:22:38.000 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:22:37 smithi026 ceph-mon[31172]: pgmap v130: 97 pgs: 97 active+clean; 1.1 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 20 MiB/s wr, 1.05k op/s 2024-09-11T23:22:38.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:22:37 smithi076 ceph-mon[35296]: pgmap v130: 97 pgs: 97 active+clean; 1.1 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 20 MiB/s wr, 1.05k op/s 2024-09-11T23:22:39.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:22:39 smithi136 ceph-mon[35121]: pgmap v131: 97 pgs: 97 active+clean; 1.1 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 19 MiB/s wr, 1.18k op/s 2024-09-11T23:22:40.000 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:22:39 smithi026 ceph-mon[31172]: pgmap v131: 97 pgs: 97 active+clean; 1.1 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 19 MiB/s wr, 1.18k op/s 2024-09-11T23:22:40.052 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:22:39 smithi076 ceph-mon[35296]: pgmap v131: 97 pgs: 97 active+clean; 1.1 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 19 MiB/s wr, 1.18k op/s 2024-09-11T23:22:41.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:22:41 smithi136 ceph-mon[35121]: pgmap v132: 97 pgs: 97 active+clean; 1.2 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 20 MiB/s wr, 1.90k op/s 2024-09-11T23:22:42.000 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:22:41 smithi026 ceph-mon[31172]: pgmap v132: 97 pgs: 97 active+clean; 1.2 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 20 MiB/s wr, 1.90k op/s 2024-09-11T23:22:42.052 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:22:41 smithi076 ceph-mon[35296]: pgmap v132: 97 pgs: 97 active+clean; 1.2 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 20 MiB/s wr, 1.90k op/s 2024-09-11T23:22:43.577 INFO:tasks.workunit.client.0.smithi026.stdout: 362 141631 71 94339 149 25056 7 2024-09-11T23:22:43.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:22:43 smithi136 ceph-mon[35121]: pgmap v133: 97 pgs: 97 active+clean; 1.2 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 6.4 MiB/s wr, 1.28k op/s 2024-09-11T23:22:44.000 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:22:43 smithi026 ceph-mon[31172]: pgmap v133: 97 pgs: 97 active+clean; 1.2 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 6.4 MiB/s wr, 1.28k op/s 2024-09-11T23:22:44.052 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:22:43 smithi076 ceph-mon[35296]: pgmap v133: 97 pgs: 97 active+clean; 1.2 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 6.4 MiB/s wr, 1.28k op/s 2024-09-11T23:22:45.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:22:45 smithi136 ceph-mon[35121]: pgmap v134: 97 pgs: 97 active+clean; 1.2 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 12 MiB/s wr, 1.76k op/s 2024-09-11T23:22:46.000 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:22:45 smithi026 ceph-mon[31172]: pgmap v134: 97 pgs: 97 active+clean; 1.2 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 12 MiB/s wr, 1.76k op/s 2024-09-11T23:22:46.052 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:22:45 smithi076 ceph-mon[35296]: pgmap v134: 97 pgs: 97 active+clean; 1.2 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 12 MiB/s wr, 1.76k op/s 2024-09-11T23:22:47.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:22:47 smithi136 ceph-mon[35121]: pgmap v135: 97 pgs: 97 active+clean; 1.2 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 12 MiB/s wr, 1.64k op/s 2024-09-11T23:22:48.000 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:22:47 smithi026 ceph-mon[31172]: pgmap v135: 97 pgs: 97 active+clean; 1.2 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 12 MiB/s wr, 1.64k op/s 2024-09-11T23:22:48.035 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.033+0000 7ff4b39c0640 1 -- 172.21.15.26:0/2499651316 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff4ac10c0f0 msgr2=0x7ff4ac10e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:48.035 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.033+0000 7ff4b39c0640 1 --2- 172.21.15.26:0/2499651316 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff4ac10c0f0 0x7ff4ac10e560 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7ff4a4009ca0 tx=0x7ff4a402a950 comp rx=0 tx=0).stop 2024-09-11T23:22:48.035 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.033+0000 7ff4b39c0640 1 -- 172.21.15.26:0/2499651316 shutdown_connections 2024-09-11T23:22:48.035 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.033+0000 7ff4b39c0640 1 --2- 172.21.15.26:0/2499651316 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff4ac103640 0x7ff4ac10bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:48.035 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.033+0000 7ff4b39c0640 1 --2- 172.21.15.26:0/2499651316 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff4ac10c0f0 0x7ff4ac10e560 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:48.036 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.033+0000 7ff4b39c0640 1 --2- 172.21.15.26:0/2499651316 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff4ac102d20 0x7ff4ac103100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:48.036 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.033+0000 7ff4b39c0640 1 -- 172.21.15.26:0/2499651316 >> 172.21.15.26:0/2499651316 conn(0x7ff4ac077990 msgr2=0x7ff4ac0fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:22:48.036 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.033+0000 7ff4b39c0640 1 -- 172.21.15.26:0/2499651316 shutdown_connections 2024-09-11T23:22:48.036 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.033+0000 7ff4b39c0640 1 -- 172.21.15.26:0/2499651316 wait complete. 2024-09-11T23:22:48.036 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.034+0000 7ff4b39c0640 1 Processor -- start 2024-09-11T23:22:48.036 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.034+0000 7ff4b39c0640 1 -- start start 2024-09-11T23:22:48.036 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.034+0000 7ff4b39c0640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff4ac102d20 0x7ff4ac198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:48.036 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.034+0000 7ff4b39c0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff4ac103640 0x7ff4ac198890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:48.036 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.034+0000 7ff4b39c0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff4ac10c0f0 0x7ff4ac19cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:48.036 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.034+0000 7ff4b39c0640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff4ac111ec0 con 0x7ff4ac10c0f0 2024-09-11T23:22:48.036 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.034+0000 7ff4b39c0640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7ff4ac111d40 con 0x7ff4ac103640 2024-09-11T23:22:48.037 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.034+0000 7ff4b39c0640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ff4ac112040 con 0x7ff4ac102d20 2024-09-11T23:22:48.037 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.034+0000 7ff4b1f5d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff4ac10c0f0 0x7ff4ac19cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:48.037 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.034+0000 7ff4b1f5d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff4ac10c0f0 0x7ff4ac19cc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46264/0 (socket says 172.21.15.26:46264) 2024-09-11T23:22:48.037 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.034+0000 7ff4b0f5b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff4ac103640 0x7ff4ac198890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:48.037 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.034+0000 7ff4b175c640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff4ac102d20 0x7ff4ac198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:48.037 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.034+0000 7ff4b1f5d640 1 -- 172.21.15.26:0/3103898987 learned_addr learned my addr 172.21.15.26:0/3103898987 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:22:48.037 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.035+0000 7ff4b1f5d640 1 -- 172.21.15.26:0/3103898987 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff4ac102d20 msgr2=0x7ff4ac198350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:48.037 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.035+0000 7ff4b1f5d640 1 --2- 172.21.15.26:0/3103898987 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff4ac102d20 0x7ff4ac198350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:48.037 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.035+0000 7ff4b1f5d640 1 -- 172.21.15.26:0/3103898987 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff4ac103640 msgr2=0x7ff4ac198890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:48.037 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.035+0000 7ff4b1f5d640 1 --2- 172.21.15.26:0/3103898987 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff4ac103640 0x7ff4ac198890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:48.037 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.035+0000 7ff4b1f5d640 1 -- 172.21.15.26:0/3103898987 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4ac19d340 con 0x7ff4ac10c0f0 2024-09-11T23:22:48.037 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.035+0000 7ff4b0f5b640 1 --2- 172.21.15.26:0/3103898987 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff4ac103640 0x7ff4ac198890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:22:48.038 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.035+0000 7ff4b1f5d640 1 --2- 172.21.15.26:0/3103898987 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff4ac10c0f0 0x7ff4ac19cc10 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7ff4a4004af0 tx=0x7ff4a402f3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:22:48.038 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.035+0000 7ff4a27fc640 1 -- 172.21.15.26:0/3103898987 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4a4034040 con 0x7ff4ac10c0f0 2024-09-11T23:22:48.038 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.035+0000 7ff4b39c0640 1 -- 172.21.15.26:0/3103898987 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4ac19d5d0 con 0x7ff4ac10c0f0 2024-09-11T23:22:48.038 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.035+0000 7ff4a27fc640 1 -- 172.21.15.26:0/3103898987 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff4a4030920 con 0x7ff4ac10c0f0 2024-09-11T23:22:48.038 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.035+0000 7ff4a27fc640 1 -- 172.21.15.26:0/3103898987 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4a4004670 con 0x7ff4ac10c0f0 2024-09-11T23:22:48.038 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.035+0000 7ff4b39c0640 1 -- 172.21.15.26:0/3103898987 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4ac1a4ec0 con 0x7ff4ac10c0f0 2024-09-11T23:22:48.044 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.038+0000 7ff4b682e740 1 Processor -- start 2024-09-11T23:22:48.044 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.038+0000 7ff4a27fc640 1 -- 172.21.15.26:0/3103898987 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7ff4a4037020 con 0x7ff4ac10c0f0 2024-09-11T23:22:48.044 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.039+0000 7ff4b682e740 1 -- start start 2024-09-11T23:22:48.044 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.039+0000 7ff4b682e740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d88db06af0 0x55d88db06ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:48.045 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.039+0000 7ff4b682e740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d88db07410 0x55d88db11770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:48.045 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.039+0000 7ff4b682e740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d88db11cb0 0x55d88db140a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:48.045 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.039+0000 7ff4b682e740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55d88da6b2f0 con 0x55d88db07410 2024-09-11T23:22:48.045 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.039+0000 7ff4b682e740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55d88da13270 con 0x55d88db06af0 2024-09-11T23:22:48.045 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.039+0000 7ff4b682e740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55d88da672b0 con 0x55d88db11cb0 2024-09-11T23:22:48.045 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.039+0000 7ff4b175c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d88db07410 0x55d88db11770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:48.045 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.039+0000 7ff4b175c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d88db07410 0x55d88db11770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46270/0 (socket says 172.21.15.26:46270) 2024-09-11T23:22:48.045 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.039+0000 7ff4b175c640 1 -- 172.21.15.26:0/3421086286 learned_addr learned my addr 172.21.15.26:0/3421086286 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:22:48.045 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.039+0000 7ff4b0f5b640 1 --2- 172.21.15.26:0/3421086286 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d88db06af0 0x55d88db06ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:48.045 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.039+0000 7ff4b0f5b640 1 --2- 172.21.15.26:0/3421086286 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d88db11cb0 0x55d88db140a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:48.045 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.039+0000 7ff4b175c640 1 -- 172.21.15.26:0/3421086286 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d88db11cb0 msgr2=0x55d88db140a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:22:48.046 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.039+0000 7ff4b175c640 1 --2- 172.21.15.26:0/3421086286 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d88db11cb0 0x55d88db140a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:48.046 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.039+0000 7ff4b175c640 1 -- 172.21.15.26:0/3421086286 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d88db06af0 msgr2=0x55d88db06ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:22:48.046 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.039+0000 7ff4b175c640 1 --2- 172.21.15.26:0/3421086286 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d88db06af0 0x55d88db06ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:48.046 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.039+0000 7ff4b175c640 1 -- 172.21.15.26:0/3421086286 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d88da12340 con 0x55d88db07410 2024-09-11T23:22:48.046 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.039+0000 7ff4a27fc640 1 --2- 172.21.15.26:0/3103898987 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7ff4980a2da0 0x7ff4980a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:48.046 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.039+0000 7ff4b175c640 1 --2- 172.21.15.26:0/3421086286 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d88db07410 0x55d88db11770 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7ff4a8009d20 tx=0x7ff4a8011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6a417262349db1b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:22:48.046 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.039+0000 7ff4b0f5b640 1 --2- 172.21.15.26:0/3103898987 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7ff4980a2da0 0x7ff4980a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:48.046 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.040+0000 7ff4a27fc640 1 -- 172.21.15.26:0/3103898987 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7ff4a40effd0 con 0x7ff4ac10c0f0 2024-09-11T23:22:48.046 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.040+0000 7ff4977fe640 1 -- 172.21.15.26:0/3421086286 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4a801e070 con 0x55d88db07410 2024-09-11T23:22:48.046 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.040+0000 7ff4977fe640 1 -- 172.21.15.26:0/3421086286 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff4a8009e10 con 0x55d88db07410 2024-09-11T23:22:48.047 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.040+0000 7ff4977fe640 1 -- 172.21.15.26:0/3421086286 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4a8018910 con 0x55d88db07410 2024-09-11T23:22:48.047 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.040+0000 7ff4b682e740 1 -- 172.21.15.26:0/3421086286 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d88db07410 msgr2=0x55d88db11770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:48.047 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.040+0000 7ff4b0f5b640 1 --2- 172.21.15.26:0/3103898987 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7ff4980a2da0 0x7ff4980a5260 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x55d88db043e0 tx=0x7ff49c0096c0 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:22:48.047 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.040+0000 7ff4b682e740 1 --2- 172.21.15.26:0/3421086286 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d88db07410 0x55d88db11770 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7ff4a8009d20 tx=0x7ff4a8011810 comp rx=0 tx=0).stop 2024-09-11T23:22:48.047 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.040+0000 7ff4b682e740 1 -- 172.21.15.26:0/3421086286 shutdown_connections 2024-09-11T23:22:48.047 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.040+0000 7ff4b682e740 1 --2- 172.21.15.26:0/3421086286 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d88db11cb0 0x55d88db140a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:48.047 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.040+0000 7ff4b682e740 1 --2- 172.21.15.26:0/3421086286 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d88db07410 0x55d88db11770 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:48.047 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.040+0000 7ff4b682e740 1 --2- 172.21.15.26:0/3421086286 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d88db06af0 0x55d88db06ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:48.047 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.040+0000 7ff4b682e740 1 -- 172.21.15.26:0/3421086286 >> 172.21.15.26:0/3421086286 conn(0x55d88db03060 msgr2=0x55d88db03440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:22:48.047 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.041+0000 7ff4b682e740 1 -- 172.21.15.26:0/3421086286 shutdown_connections 2024-09-11T23:22:48.047 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.041+0000 7ff4b682e740 1 -- 172.21.15.26:0/3421086286 wait complete. 2024-09-11T23:22:48.048 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.041+0000 7ff4b682e740 1 -- start start 2024-09-11T23:22:48.048 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.041+0000 7ff4b682e740 1 Processor -- start 2024-09-11T23:22:48.048 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.041+0000 7ff4b682e740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d88db06af0 0x55d88db0eb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:48.048 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.041+0000 7ff4b682e740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d88db07410 0x55d88db0f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:48.048 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.041+0000 7ff4b682e740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d88db11cb0 0x55d88db0f610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:48.048 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.041+0000 7ff4b682e740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55d88db14880 con 0x55d88db06af0 2024-09-11T23:22:48.048 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.041+0000 7ff4b682e740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55d88d9f4290 con 0x55d88db11cb0 2024-09-11T23:22:48.048 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.041+0000 7ff4b682e740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55d88db14a00 con 0x55d88db07410 2024-09-11T23:22:48.048 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.041+0000 7ff4b175c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d88db06af0 0x55d88db0eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:48.048 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.041+0000 7ff4b682e740 10 client.0 _ll_register_callbacks cb 0x55d88da25140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:22:48.048 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.042+0000 7ff4b175c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d88db06af0 0x55d88db0eb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46278/0 (socket says 172.21.15.26:46278) 2024-09-11T23:22:48.049 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.042+0000 7ff4b175c640 1 -- 172.21.15.26:0/2746074003 learned_addr learned my addr 172.21.15.26:0/2746074003 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:22:48.049 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.042+0000 7ff4b0f5b640 1 --2- 172.21.15.26:0/2746074003 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d88db07410 0x55d88db0f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:48.049 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.042+0000 7ff4b1f5d640 1 --2- 172.21.15.26:0/2746074003 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d88db11cb0 0x55d88db0f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:48.049 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.042+0000 7ff4b175c640 1 -- 172.21.15.26:0/2746074003 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d88db07410 msgr2=0x55d88db0f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:22:48.049 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.042+0000 7ff4b175c640 1 --2- 172.21.15.26:0/2746074003 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d88db07410 0x55d88db0f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:48.049 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.042+0000 7ff4b175c640 1 -- 172.21.15.26:0/2746074003 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d88db11cb0 msgr2=0x55d88db0f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:22:48.049 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.042+0000 7ff4b175c640 1 --2- 172.21.15.26:0/2746074003 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d88db11cb0 0x55d88db0f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:48.049 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.042+0000 7ff4b175c640 1 -- 172.21.15.26:0/2746074003 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d88d77e210 con 0x55d88db06af0 2024-09-11T23:22:48.049 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.042+0000 7ff4b175c640 1 --2- 172.21.15.26:0/2746074003 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d88db06af0 0x55d88db0eb90 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7ff4a8018530 tx=0x7ff4a80188c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b364ae60e058d03e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:22:48.049 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.042+0000 7ff494ff9640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:22:48.049 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.042+0000 7ff494ff9640 1 -- 172.21.15.26:0/2746074003 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4a8027060 con 0x55d88db06af0 2024-09-11T23:22:48.050 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.042+0000 7ff4b682e740 1 -- 172.21.15.26:0/2746074003 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff4a8009e10 con 0x55d88db06af0 2024-09-11T23:22:48.050 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.042+0000 7ff494ff9640 1 -- 172.21.15.26:0/2746074003 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7ff4a8018a30 con 0x55d88db06af0 2024-09-11T23:22:48.050 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.042+0000 7ff494ff9640 1 -- 172.21.15.26:0/2746074003 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7ff4a801e040 con 0x55d88db06af0 2024-09-11T23:22:48.050 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.042+0000 7ff4b682e740 10 client.16338 fetch_fsmap learned FSMap version 38 2024-09-11T23:22:48.050 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.042+0000 7ff4b682e740 1 -- 172.21.15.26:0/2746074003 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55d88d84a620 con 0x55d88db06af0 2024-09-11T23:22:48.050 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.043+0000 7ff494ff9640 1 -- 172.21.15.26:0/2746074003 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff4a8009e10 con 0x55d88db06af0 2024-09-11T23:22:48.050 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.043+0000 7ff494ff9640 1 -- 172.21.15.26:0/2746074003 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 38) ==== 8414+0+0 (secure 0 0 0) 0x7ff4a80047c0 con 0x55d88db06af0 2024-09-11T23:22:48.050 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.043+0000 7ff4b682e740 10 client.16338 fetch_fsmap finished waiting for FSMap version 38 2024-09-11T23:22:48.050 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.043+0000 7ff4b682e740 10 client.16338 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:22:48.050 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.043+0000 7ff4b682e740 20 client.16338 populate_metadata read hostname 'smithi026' 2024-09-11T23:22:48.051 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.043+0000 7ff4b682e740 1 --2- 172.21.15.26:0/2746074003 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55d88db21c20 0x55d88db24010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:48.051 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.043+0000 7ff4b682e740 4 client.16338 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:22:48.051 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.043+0000 7ff4b682e740 1 -- 172.21.15.26:0/2746074003 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d88da78a20 con 0x55d88db21c20 2024-09-11T23:22:48.051 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.043+0000 7ff4b0f5b640 1 --2- 172.21.15.26:0/2746074003 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55d88db21c20 0x55d88db24010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:48.051 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.044+0000 7ff4b0f5b640 1 --2- 172.21.15.26:0/2746074003 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55d88db21c20 0x55d88db24010 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de5ece3c4af40793 server_cookie=ce85a7e2978ed34d in_seq=0 out_seq=0 2024-09-11T23:22:48.051 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.044+0000 7ff494ff9640 10 client.16338 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:22:48.052 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.046+0000 7ff494ff9640 1 -- 172.21.15.26:0/2746074003 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d88da78a20 con 0x55d88db21c20 2024-09-11T23:22:48.053 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.046+0000 7ff494ff9640 10 client.16338 handle_command_reply: tid=0 2024-09-11T23:22:48.053 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.046+0000 7ff4b682e740 1 client.16338 shutdown 2024-09-11T23:22:48.053 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.046+0000 7ff4b682e740 1 -- 172.21.15.26:0/2746074003 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d88db06af0 msgr2=0x55d88db0eb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:48.053 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.046+0000 7ff4b682e740 1 --2- 172.21.15.26:0/2746074003 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d88db06af0 0x55d88db0eb90 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7ff4a8018530 tx=0x7ff4a80188c0 comp rx=0 tx=0).stop 2024-09-11T23:22:48.053 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.046+0000 7ff4b682e740 1 -- 172.21.15.26:0/2746074003 shutdown_connections 2024-09-11T23:22:48.053 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.046+0000 7ff4b682e740 1 --2- 172.21.15.26:0/2746074003 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d88db11cb0 0x55d88db0f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:48.053 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.046+0000 7ff4b682e740 1 --2- 172.21.15.26:0/2746074003 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55d88db21c20 0x55d88db24010 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:48.053 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.046+0000 7ff4b682e740 1 --2- 172.21.15.26:0/2746074003 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d88db07410 0x55d88db0f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:48.053 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.047+0000 7ff4b682e740 1 --2- 172.21.15.26:0/2746074003 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d88db06af0 0x55d88db0eb90 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:48.053 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.047+0000 7ff494ff9640 0 client.16338 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:22:48.054 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.047+0000 7ff4b682e740 1 -- 172.21.15.26:0/2746074003 >> 172.21.15.26:0/2746074003 conn(0x55d88db03060 msgr2=0x55d88db08dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:22:48.054 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.047+0000 7ff4b682e740 1 -- 172.21.15.26:0/2746074003 shutdown_connections 2024-09-11T23:22:48.054 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.047+0000 7ff4b682e740 1 -- 172.21.15.26:0/2746074003 wait complete. 2024-09-11T23:22:48.054 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.047+0000 7ff4b682e740 20 client.16338 trim_cache size 0 max 16384 2024-09-11T23:22:48.054 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:22:47 smithi076 ceph-mon[35296]: pgmap v135: 97 pgs: 97 active+clean; 1.2 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 12 MiB/s wr, 1.64k op/s 2024-09-11T23:22:48.063 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.062+0000 7ff4b682e740 1 Processor -- start 2024-09-11T23:22:48.063 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.062+0000 7ff4b682e740 1 -- start start 2024-09-11T23:22:48.063 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.062+0000 7ff4b682e740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d88db06af0 0x55d88db0f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:48.063 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.062+0000 7ff4b682e740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d88db07410 0x55d88db08cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:48.063 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.062+0000 7ff4b682e740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d88db11cb0 0x55d88db297d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:48.063 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.062+0000 7ff4b682e740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff4ac1ad720 con 0x55d88db06af0 2024-09-11T23:22:48.063 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.062+0000 7ff4b682e740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7ff4ac1af4d0 con 0x55d88db11cb0 2024-09-11T23:22:48.063 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.062+0000 7ff4b682e740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ff4ac1adeb0 con 0x55d88db07410 2024-09-11T23:22:48.064 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.062+0000 7ff4b175c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d88db06af0 0x55d88db0f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:48.064 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.062+0000 7ff4b175c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d88db06af0 0x55d88db0f440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46280/0 (socket says 172.21.15.26:46280) 2024-09-11T23:22:48.064 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.062+0000 7ff4b175c640 1 -- 172.21.15.26:0/1066356849 learned_addr learned my addr 172.21.15.26:0/1066356849 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:22:48.064 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.062+0000 7ff4b1f5d640 1 --2- 172.21.15.26:0/1066356849 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d88db11cb0 0x55d88db297d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:48.064 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.062+0000 7ff4b0f5b640 1 --2- 172.21.15.26:0/1066356849 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d88db07410 0x55d88db08cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:48.064 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.062+0000 7ff4b175c640 1 -- 172.21.15.26:0/1066356849 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d88db07410 msgr2=0x55d88db08cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:22:48.064 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.062+0000 7ff4b175c640 1 --2- 172.21.15.26:0/1066356849 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d88db07410 0x55d88db08cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:48.064 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.062+0000 7ff4b175c640 1 -- 172.21.15.26:0/1066356849 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d88db11cb0 msgr2=0x55d88db297d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:22:48.064 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.062+0000 7ff4b175c640 1 --2- 172.21.15.26:0/1066356849 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d88db11cb0 0x55d88db297d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:48.064 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.063+0000 7ff4b175c640 1 -- 172.21.15.26:0/1066356849 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d88db02e90 con 0x55d88db06af0 2024-09-11T23:22:48.064 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.063+0000 7ff4b175c640 1 --2- 172.21.15.26:0/1066356849 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d88db06af0 0x55d88db0f440 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7ff4a80059f0 tx=0x7ff4a8009660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eccfe25ed4f1d182 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:22:48.064 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.063+0000 7ff496ffd640 1 -- 172.21.15.26:0/1066356849 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4a8004630 con 0x55d88db06af0 2024-09-11T23:22:48.064 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.063+0000 7ff496ffd640 1 -- 172.21.15.26:0/1066356849 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff4a80098f0 con 0x55d88db06af0 2024-09-11T23:22:48.065 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.063+0000 7ff496ffd640 1 -- 172.21.15.26:0/1066356849 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4a8003b60 con 0x55d88db06af0 2024-09-11T23:22:48.068 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.064+0000 7ff4b682e740 1 -- 172.21.15.26:0/1066356849 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d88db06af0 msgr2=0x55d88db0f440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:48.068 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.064+0000 7ff4b682e740 1 --2- 172.21.15.26:0/1066356849 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d88db06af0 0x55d88db0f440 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7ff4a80059f0 tx=0x7ff4a8009660 comp rx=0 tx=0).stop 2024-09-11T23:22:48.068 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.064+0000 7ff4b682e740 1 -- 172.21.15.26:0/1066356849 shutdown_connections 2024-09-11T23:22:48.068 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.064+0000 7ff4b682e740 1 --2- 172.21.15.26:0/1066356849 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d88db11cb0 0x55d88db297d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:48.068 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.064+0000 7ff4b682e740 1 --2- 172.21.15.26:0/1066356849 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d88db07410 0x55d88db08cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:48.068 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.064+0000 7ff4b682e740 1 --2- 172.21.15.26:0/1066356849 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d88db06af0 0x55d88db0f440 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:48.068 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.064+0000 7ff4b682e740 1 -- 172.21.15.26:0/1066356849 >> 172.21.15.26:0/1066356849 conn(0x55d88db03060 msgr2=0x55d88db085a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:22:48.068 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.064+0000 7ff4b682e740 1 -- 172.21.15.26:0/1066356849 shutdown_connections 2024-09-11T23:22:48.068 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.064+0000 7ff4b682e740 1 -- 172.21.15.26:0/1066356849 wait complete. 2024-09-11T23:22:48.069 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.064+0000 7ff4b682e740 1 -- start start 2024-09-11T23:22:48.069 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.064+0000 7ff4b682e740 1 Processor -- start 2024-09-11T23:22:48.069 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.064+0000 7ff4b682e740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d88db06af0 0x55d88db1a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:48.069 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.064+0000 7ff4b682e740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d88db07410 0x55d88db14c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:48.069 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.065+0000 7ff4b682e740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d88db11cb0 0x55d88db15410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:48.069 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.065+0000 7ff4b682e740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff4ac1aedd0 con 0x55d88db07410 2024-09-11T23:22:48.069 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.065+0000 7ff4b682e740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7ff4ac1ae640 con 0x55d88db11cb0 2024-09-11T23:22:48.069 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.065+0000 7ff4b682e740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ff4ac1ab700 con 0x55d88db06af0 2024-09-11T23:22:48.069 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.065+0000 7ff4b682e740 10 client.0 _ll_register_callbacks cb 0x7ff4a801c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:22:48.069 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.065+0000 7ff4b0f5b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d88db07410 0x55d88db14c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:48.069 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.065+0000 7ff4b0f5b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d88db07410 0x55d88db14c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46292/0 (socket says 172.21.15.26:46292) 2024-09-11T23:22:48.070 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.065+0000 7ff4b175c640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d88db06af0 0x55d88db1a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:48.070 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.065+0000 7ff4b0f5b640 1 -- 172.21.15.26:0/2316134916 learned_addr learned my addr 172.21.15.26:0/2316134916 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:22:48.070 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.065+0000 7ff4b1f5d640 1 --2- 172.21.15.26:0/2316134916 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d88db11cb0 0x55d88db15410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:48.070 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.065+0000 7ff4b0f5b640 1 -- 172.21.15.26:0/2316134916 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d88db06af0 msgr2=0x55d88db1a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:22:48.070 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.065+0000 7ff4b0f5b640 1 --2- 172.21.15.26:0/2316134916 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d88db06af0 0x55d88db1a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:48.070 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.065+0000 7ff4b0f5b640 1 -- 172.21.15.26:0/2316134916 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d88db11cb0 msgr2=0x55d88db15410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:22:48.070 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.065+0000 7ff4b0f5b640 1 --2- 172.21.15.26:0/2316134916 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d88db11cb0 0x55d88db15410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:48.070 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.065+0000 7ff4b0f5b640 1 -- 172.21.15.26:0/2316134916 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d88db8a790 con 0x55d88db07410 2024-09-11T23:22:48.070 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.065+0000 7ff4b175c640 1 --2- 172.21.15.26:0/2316134916 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d88db06af0 0x55d88db1a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:22:48.070 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.065+0000 7ff4b0f5b640 1 --2- 172.21.15.26:0/2316134916 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d88db07410 0x55d88db14c80 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7ff49c009730 tx=0x7ff49c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ae8ae4c98758f94 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:22:48.070 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.065+0000 7ff494ff9640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:22:48.071 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.065+0000 7ff494ff9640 1 -- 172.21.15.26:0/2316134916 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff49c011e60 con 0x55d88db07410 2024-09-11T23:22:48.071 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.065+0000 7ff494ff9640 1 -- 172.21.15.26:0/2316134916 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7ff49c01a070 con 0x55d88db07410 2024-09-11T23:22:48.071 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.065+0000 7ff4b682e740 1 -- 172.21.15.26:0/2316134916 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff4a80098f0 con 0x55d88db07410 2024-09-11T23:22:48.071 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.065+0000 7ff494ff9640 1 -- 172.21.15.26:0/2316134916 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7ff49c01ea10 con 0x55d88db07410 2024-09-11T23:22:48.071 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.065+0000 7ff4b682e740 10 client.16344 fetch_fsmap learned FSMap version 38 2024-09-11T23:22:48.071 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.065+0000 7ff4b682e740 1 -- 172.21.15.26:0/2316134916 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55d88db8ba10 con 0x55d88db07410 2024-09-11T23:22:48.071 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.066+0000 7ff494ff9640 1 -- 172.21.15.26:0/2316134916 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff4a80098f0 con 0x55d88db07410 2024-09-11T23:22:48.071 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.066+0000 7ff494ff9640 1 -- 172.21.15.26:0/2316134916 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 38) ==== 8414+0+0 (secure 0 0 0) 0x7ff49c01d070 con 0x55d88db07410 2024-09-11T23:22:48.071 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.066+0000 7ff4b682e740 10 client.16344 fetch_fsmap finished waiting for FSMap version 38 2024-09-11T23:22:48.071 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.066+0000 7ff4b682e740 10 client.16344 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:22:48.071 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.066+0000 7ff4b682e740 20 client.16344 populate_metadata read hostname 'smithi026' 2024-09-11T23:22:48.072 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.066+0000 7ff4b682e740 1 --2- 172.21.15.26:0/2316134916 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55d88db8c1d0 0x55d88db8e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:48.072 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.066+0000 7ff4b682e740 4 client.16344 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T23:22:48.072 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.066+0000 7ff4b682e740 1 -- 172.21.15.26:0/2316134916 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d88da72d70 con 0x55d88db8c1d0 2024-09-11T23:22:48.072 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.066+0000 7ff4b175c640 1 --2- 172.21.15.26:0/2316134916 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55d88db8c1d0 0x55d88db8e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:48.072 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.066+0000 7ff494ff9640 10 client.16344 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:22:48.072 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:48.066+0000 7ff4b175c640 1 --2- 172.21.15.26:0/2316134916 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55d88db8c1d0 0x55d88db8e5f0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=89679b88bc8c2771 server_cookie=8ea3f51da1d550e2 in_seq=0 out_seq=0 2024-09-11T23:22:50.000 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:22:49 smithi026 ceph-mon[31172]: pgmap v136: 97 pgs: 97 active+clean; 1.3 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 12 MiB/s wr, 1.62k op/s 2024-09-11T23:22:50.003 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:22:49 smithi136 ceph-mon[35121]: pgmap v136: 97 pgs: 97 active+clean; 1.3 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 12 MiB/s wr, 1.62k op/s 2024-09-11T23:22:50.054 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:22:49 smithi076 ceph-mon[35296]: pgmap v136: 97 pgs: 97 active+clean; 1.3 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 12 MiB/s wr, 1.62k op/s 2024-09-11T23:22:50.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:50.312+0000 7fcc8d7fa640 1 -- 172.21.15.26:0/3207599342 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55ac4dd7cbe0 con 0x55ac4dea8230 2024-09-11T23:22:50.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:50.312+0000 7fcc8d7fa640 10 client.25925 handle_command_reply: tid=0 2024-09-11T23:22:50.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:50.312+0000 7fccae128740 1 client.25925 shutdown 2024-09-11T23:22:50.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:50.312+0000 7fccae128740 1 -- 172.21.15.26:0/3207599342 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ac4de0c2f0 msgr2=0x55ac4de1fa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:50.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:50.312+0000 7fccae128740 1 --2- 172.21.15.26:0/3207599342 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ac4de0c2f0 0x55ac4de1fa50 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fcc9400ae30 tx=0x7fcc9400c750 comp rx=0 tx=0).stop 2024-09-11T23:22:50.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:50.312+0000 7fccae128740 1 -- 172.21.15.26:0/3207599342 shutdown_connections 2024-09-11T23:22:50.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:50.312+0000 7fccae128740 1 --2- 172.21.15.26:0/3207599342 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55ac4dea8230 0x55ac4deaa650 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:50.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:50.312+0000 7fccae128740 1 --2- 172.21.15.26:0/3207599342 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ac4de1ba30 0x55ac4de204f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:50.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:50.312+0000 7fccae128740 1 --2- 172.21.15.26:0/3207599342 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ac4de11170 0x55ac4de1ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:50.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:50.312+0000 7fccae128740 1 --2- 172.21.15.26:0/3207599342 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ac4de0c2f0 0x55ac4de1fa50 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:50.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:50.312+0000 7fcc8d7fa640 0 client.25925 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:22:50.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:50.312+0000 7fccae128740 1 -- 172.21.15.26:0/3207599342 >> 172.21.15.26:0/3207599342 conn(0x55ac4de0cde0 msgr2=0x55ac4de26890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:22:50.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:50.312+0000 7fccae128740 1 -- 172.21.15.26:0/3207599342 shutdown_connections 2024-09-11T23:22:50.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:50.312+0000 7fccae128740 1 -- 172.21.15.26:0/3207599342 wait complete. 2024-09-11T23:22:50.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:50.312+0000 7fccae128740 20 client.25925 trim_cache size 0 max 16384 2024-09-11T23:22:50.314 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-11T23:22:50.314 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 38, 2024-09-11T23:22:50.314 INFO:teuthology.orchestra.run.smithi026.stdout: "leader": 24485, 2024-09-11T23:22:50.314 INFO:teuthology.orchestra.run.smithi026.stdout: "set_version": 14, 2024-09-11T23:22:50.314 INFO:teuthology.orchestra.run.smithi026.stdout: "sets": { 2024-09-11T23:22:50.314 INFO:teuthology.orchestra.run.smithi026.stdout: "82d019e7": { 2024-09-11T23:22:50.314 INFO:teuthology.orchestra.run.smithi026.stdout: "version": 14, 2024-09-11T23:22:50.314 INFO:teuthology.orchestra.run.smithi026.stdout: "age_ref": 0.0, 2024-09-11T23:22:50.314 INFO:teuthology.orchestra.run.smithi026.stdout: "state": { 2024-09-11T23:22:50.314 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "QUIESCED", 2024-09-11T23:22:50.314 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.0 2024-09-11T23:22:50.315 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:22:50.315 INFO:teuthology.orchestra.run.smithi026.stdout: "timeout": 90.0, 2024-09-11T23:22:50.315 INFO:teuthology.orchestra.run.smithi026.stdout: "expiration": 140.2, 2024-09-11T23:22:50.315 INFO:teuthology.orchestra.run.smithi026.stdout: "members": { 2024-09-11T23:22:50.315 INFO:teuthology.orchestra.run.smithi026.stdout: "file:/": { 2024-09-11T23:22:50.315 INFO:teuthology.orchestra.run.smithi026.stdout: "excluded": false, 2024-09-11T23:22:50.315 INFO:teuthology.orchestra.run.smithi026.stdout: "state": { 2024-09-11T23:22:50.315 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "QUIESCED", 2024-09-11T23:22:50.315 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.0 2024-09-11T23:22:50.315 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:22:50.315 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:22:50.315 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:22:50.315 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:22:50.316 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:22:50.316 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-11T23:22:50.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:50.313+0000 7fccabd4f640 1 -- 172.21.15.26:0/2964718668 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fcc900a2ec0 msgr2=0x7fcc900a5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:50.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:50.313+0000 7fccabd4f640 1 --2- 172.21.15.26:0/2964718668 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fcc900a2ec0 0x7fcc900a5380 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fcc9c005c40 tx=0x7fcc9c009570 comp rx=0 tx=0).stop 2024-09-11T23:22:50.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:50.313+0000 7fccabd4f640 1 -- 172.21.15.26:0/2964718668 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcca4109830 msgr2=0x7fcca41a7790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:50.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:50.313+0000 7fccabd4f640 1 --2- 172.21.15.26:0/2964718668 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcca4109830 0x7fcca41a7790 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fcca0005ab0 tx=0x7fcca00052b0 comp rx=0 tx=0).stop 2024-09-11T23:22:50.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:50.313+0000 7fccabd4f640 1 -- 172.21.15.26:0/2964718668 shutdown_connections 2024-09-11T23:22:50.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:50.313+0000 7fccabd4f640 1 --2- 172.21.15.26:0/2964718668 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fcc900a2ec0 0x7fcc900a5380 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:50.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:50.313+0000 7fccabd4f640 1 --2- 172.21.15.26:0/2964718668 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcca4109830 0x7fcca41a7790 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:50.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:50.313+0000 7fccabd4f640 1 --2- 172.21.15.26:0/2964718668 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcca411b140 0x7fcca41abb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:50.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:50.313+0000 7fccabd4f640 1 --2- 172.21.15.26:0/2964718668 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcca4109450 0x7fcca41a7250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:50.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:50.313+0000 7fccabd4f640 1 -- 172.21.15.26:0/2964718668 >> 172.21.15.26:0/2964718668 conn(0x7fcca406e140 msgr2=0x7fcca411be80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:22:50.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:50.313+0000 7fccabd4f640 1 -- 172.21.15.26:0/2964718668 shutdown_connections 2024-09-11T23:22:50.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:50.313+0000 7fccabd4f640 1 -- 172.21.15.26:0/2964718668 wait complete. 2024-09-11T23:22:51.172 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '82d019e7', quiesce took 32.8 seconds. Will release after: -12.649999999999999 2024-09-11T23:22:51.172 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:22:51.489 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:22:51.892 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:22:51 smithi026 ceph-mon[31172]: pgmap v137: 97 pgs: 97 active+clean; 1.3 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 17 MiB/s wr, 1.61k op/s 2024-09-11T23:22:51.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:22:51 smithi136 ceph-mon[35121]: pgmap v137: 97 pgs: 97 active+clean; 1.3 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 17 MiB/s wr, 1.61k op/s 2024-09-11T23:22:52.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:51.985+0000 7ff006ccd640 1 -- 172.21.15.26:0/1277382737 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff000102d90 msgr2=0x7ff000103170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:52.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:51.985+0000 7ff006ccd640 1 --2- 172.21.15.26:0/1277382737 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff000102d90 0x7ff000103170 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7feff4009930 tx=0x7feff402a420 comp rx=0 tx=0).stop 2024-09-11T23:22:52.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:51.985+0000 7ff006ccd640 1 -- 172.21.15.26:0/1277382737 shutdown_connections 2024-09-11T23:22:52.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:51.985+0000 7ff006ccd640 1 --2- 172.21.15.26:0/1277382737 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff000103740 0x7ff000107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:52.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:51.985+0000 7ff006ccd640 1 --2- 172.21.15.26:0/1277382737 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff000108290 0x7ff00010ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:52.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:51.985+0000 7ff006ccd640 1 --2- 172.21.15.26:0/1277382737 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff000102d90 0x7ff000103170 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:52.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:51.985+0000 7ff006ccd640 1 -- 172.21.15.26:0/1277382737 >> 172.21.15.26:0/1277382737 conn(0x7ff0000fc7d0 msgr2=0x7ff0000febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:22:52.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:51.985+0000 7ff006ccd640 1 -- 172.21.15.26:0/1277382737 shutdown_connections 2024-09-11T23:22:52.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:51.985+0000 7ff006ccd640 1 -- 172.21.15.26:0/1277382737 wait complete. 2024-09-11T23:22:52.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:51.986+0000 7ff006ccd640 1 Processor -- start 2024-09-11T23:22:52.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:51.986+0000 7ff006ccd640 1 -- start start 2024-09-11T23:22:52.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:51.986+0000 7ff006ccd640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff000102d90 0x7ff0001009f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:52.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:51.986+0000 7ff006ccd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff000103740 0x7ff000100f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:52.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:51.986+0000 7ff006ccd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff000108290 0x7ff000101470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:52.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:51.986+0000 7ff006ccd640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff000112090 con 0x7ff000108290 2024-09-11T23:22:52.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:51.986+0000 7ff006ccd640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7ff000111f10 con 0x7ff000103740 2024-09-11T23:22:52.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:51.986+0000 7ff006ccd640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ff000112210 con 0x7ff000102d90 2024-09-11T23:22:52.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:51.986+0000 7ff004a69640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff000102d90 0x7ff0001009f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:52.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:51.986+0000 7ff00526a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff000108290 0x7ff000101470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:52.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:51.986+0000 7ff004a69640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff000102d90 0x7ff0001009f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:56840/0 (socket says 172.21.15.26:56840) 2024-09-11T23:22:52.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:51.986+0000 7feffffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff000103740 0x7ff000100f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:52.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:51.986+0000 7ff004a69640 1 -- 172.21.15.26:0/2588096 learned_addr learned my addr 172.21.15.26:0/2588096 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:22:52.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:51.987+0000 7ff004a69640 1 -- 172.21.15.26:0/2588096 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff000103740 msgr2=0x7ff000100f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:52.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:51.987+0000 7ff004a69640 1 --2- 172.21.15.26:0/2588096 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff000103740 0x7ff000100f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:52.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:51.987+0000 7ff004a69640 1 -- 172.21.15.26:0/2588096 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff000108290 msgr2=0x7ff000101470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:52.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:51.987+0000 7ff004a69640 1 --2- 172.21.15.26:0/2588096 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff000108290 0x7ff000101470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:52.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:51.987+0000 7ff004a69640 1 -- 172.21.15.26:0/2588096 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0001a6ab0 con 0x7ff000102d90 2024-09-11T23:22:52.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:51.987+0000 7feffffff640 1 --2- 172.21.15.26:0/2588096 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff000103740 0x7ff000100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:22:52.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:51.987+0000 7ff00526a640 1 --2- 172.21.15.26:0/2588096 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff000108290 0x7ff000101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:22:52.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:51.987+0000 7ff004a69640 1 --2- 172.21.15.26:0/2588096 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff000102d90 0x7ff0001009f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7feff4009230 tx=0x7feff4004040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:22:52.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:51.987+0000 7feffdffb640 1 -- 172.21.15.26:0/2588096 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feff403f030 con 0x7ff000102d90 2024-09-11T23:22:52.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:51.987+0000 7feffdffb640 1 -- 172.21.15.26:0/2588096 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feff4004440 con 0x7ff000102d90 2024-09-11T23:22:52.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:51.987+0000 7feffdffb640 1 -- 172.21.15.26:0/2588096 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feff4035070 con 0x7ff000102d90 2024-09-11T23:22:52.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:51.987+0000 7ff006ccd640 1 -- 172.21.15.26:0/2588096 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0001a6d40 con 0x7ff000102d90 2024-09-11T23:22:52.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:51.988+0000 7ff006ccd640 1 -- 172.21.15.26:0/2588096 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0001a7170 con 0x7ff000102d90 2024-09-11T23:22:52.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:51.989+0000 7ff006ccd640 1 -- 172.21.15.26:0/2588096 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff00006a7f0 con 0x7ff000102d90 2024-09-11T23:22:52.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:51.990+0000 7feffdffb640 1 -- 172.21.15.26:0/2588096 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7feff4042030 con 0x7ff000102d90 2024-09-11T23:22:52.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:51.991+0000 7feffdffb640 1 --2- 172.21.15.26:0/2588096 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7feff00a2cf0 0x7feff00a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:52.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:51.991+0000 7feffffff640 1 --2- 172.21.15.26:0/2588096 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7feff00a2cf0 0x7feff00a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:52.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:51.991+0000 7feffdffb640 1 -- 172.21.15.26:0/2588096 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7feff40ef730 con 0x7ff000102d90 2024-09-11T23:22:52.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:51.992+0000 7feffffff640 1 --2- 172.21.15.26:0/2588096 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7feff00a2cf0 0x7feff00a51b0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff000102480 tx=0x7fefec005f90 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:22:52.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:51.994+0000 7feffdffb640 1 -- 172.21.15.26:0/2588096 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7feff40bbd10 con 0x7ff000102d90 2024-09-11T23:22:52.052 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:22:51 smithi076 ceph-mon[35296]: pgmap v137: 97 pgs: 97 active+clean; 1.3 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 17 MiB/s wr, 1.61k op/s 2024-09-11T23:22:52.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:52.233+0000 7ff006ccd640 1 -- 172.21.15.26:0/2588096 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff000103170 con 0x7ff000102d90 2024-09-11T23:22:52.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:52.234+0000 7feffdffb640 1 -- 172.21.15.26:0/2588096 <== mon.1 v2:172.21.15.136:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) ==== 94+0+11994 (secure 0 0 0) 0x7ff000103170 con 0x7ff000102d90 2024-09-11T23:22:52.234 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-11T23:22:52.234 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 38 2024-09-11T23:22:52.235 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":38,"btime":"2024-09-11T23:22:31:575477+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14499,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/367659515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":367659515},{"type":"v1","addr":"172.21.15.26:6835","nonce":367659515}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14511,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/2041065299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":2041065299},{"type":"v1","addr":"172.21.15.26:6837","nonce":2041065299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":38,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:16:26.544203+0000","modified":"2024-09-11T23:22:31.018594+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24485,"mds_1":24463,"mds_2":24458,"mds_3":24448,"mds_4":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14550":{"gid":14550,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6839/2233068305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":2233068305},{"type":"v1","addr":"172.21.15.26:6839","nonce":2233068305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14562":{"gid":14562,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6841/2984982336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":2984982336},{"type":"v1","addr":"172.21.15.26:6841","nonce":2984982336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6833/2019841455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":2019841455},{"type":"v1","addr":"172.21.15.136:6833","nonce":2019841455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.76:6833/3438147050","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3438147050},{"type":"v1","addr":"172.21.15.76:6833","nonce":3438147050}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6835/3617665975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3617665975},{"type":"v1","addr":"172.21.15.136:6835","nonce":3617665975}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":12,"addr":"172.21.15.76:6835/393308499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":393308499},{"type":"v1","addr":"172.21.15.76:6835","nonce":393308499}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":10,"addr":"172.21.15.136:6837/1633082387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":1633082387},{"type":"v1","addr":"172.21.15.136:6837","nonce":1633082387}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.76:6837/2078978748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":2078978748},{"type":"v1","addr":"172.21.15.76:6837","nonce":2078978748}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":63,"addr":"172.21.15.136:6839/3211398630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":3211398630},{"type":"v1","addr":"172.21.15.136:6839","nonce":3211398630}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.76:6839/2077637090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6838","nonce":2077637090},{"type":"v1","addr":"172.21.15.76:6839","nonce":2077637090}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24485,"qdb_cluster":[24485,24463,24458,24448,24443]},"id":1}]} 2024-09-11T23:22:52.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:52.236+0000 7ff006ccd640 1 -- 172.21.15.26:0/2588096 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7feff00a2cf0 msgr2=0x7feff00a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:52.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:52.236+0000 7ff006ccd640 1 --2- 172.21.15.26:0/2588096 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7feff00a2cf0 0x7feff00a51b0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff000102480 tx=0x7fefec005f90 comp rx=0 tx=0).stop 2024-09-11T23:22:52.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:52.236+0000 7ff006ccd640 1 -- 172.21.15.26:0/2588096 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff000102d90 msgr2=0x7ff0001009f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:52.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:52.236+0000 7ff006ccd640 1 --2- 172.21.15.26:0/2588096 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff000102d90 0x7ff0001009f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7feff4009230 tx=0x7feff4004040 comp rx=0 tx=0).stop 2024-09-11T23:22:52.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:52.236+0000 7ff006ccd640 1 -- 172.21.15.26:0/2588096 shutdown_connections 2024-09-11T23:22:52.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:52.236+0000 7ff006ccd640 1 --2- 172.21.15.26:0/2588096 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7feff00a2cf0 0x7feff00a51b0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:52.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:52.236+0000 7ff006ccd640 1 --2- 172.21.15.26:0/2588096 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff000103740 0x7ff000100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:52.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:52.236+0000 7ff006ccd640 1 --2- 172.21.15.26:0/2588096 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff000108290 0x7ff000101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:52.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:52.236+0000 7ff006ccd640 1 --2- 172.21.15.26:0/2588096 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff000102d90 0x7ff0001009f0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:52.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:52.236+0000 7ff006ccd640 1 -- 172.21.15.26:0/2588096 >> 172.21.15.26:0/2588096 conn(0x7ff0000fc7d0 msgr2=0x7ff000106370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:22:52.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:52.237+0000 7ff006ccd640 1 -- 172.21.15.26:0/2588096 shutdown_connections 2024-09-11T23:22:52.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:52.237+0000 7ff006ccd640 1 -- 172.21.15.26:0/2588096 wait complete. 2024-09-11T23:22:52.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:22:52 smithi136 ceph-mon[35121]: from='client.? 172.21.15.26:0/2588096' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:22:52.989 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:22:52 smithi026 ceph-mon[31172]: from='client.? 172.21.15.26:0/2588096' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:22:53.025 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24485 quiesce db --set-id 82d019e7 --release --await' 2024-09-11T23:22:53.025 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24485 quiesce db --set-id 82d019e7 --release --await 2024-09-11T23:22:53.052 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:22:52 smithi076 ceph-mon[35296]: from='client.? 172.21.15.26:0/2588096' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:22:53.326 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:22:53.578 INFO:tasks.workunit.client.0.smithi026.stdout: 362 19856 0 13680 0 3302 0 2024-09-11T23:22:53.750 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:22:53 smithi026 ceph-mon[31172]: pgmap v138: 97 pgs: 97 active+clean; 1.3 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 13 MiB/s wr, 781 op/s 2024-09-11T23:22:53.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.876+0000 7fc9057fe640 1 -- 172.21.15.26:0/985748521 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc900100930 msgr2=0x7fc900100d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:53.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.876+0000 7fc9057fe640 1 --2- 172.21.15.26:0/985748521 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc900100930 0x7fc900100d10 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fc8f0009900 tx=0x7fc8f002a410 comp rx=0 tx=0).stop 2024-09-11T23:22:53.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.877+0000 7fc9057fe640 1 -- 172.21.15.26:0/985748521 shutdown_connections 2024-09-11T23:22:53.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.877+0000 7fc9057fe640 1 --2- 172.21.15.26:0/985748521 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc9001012e0 0x7fc90010dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:53.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.877+0000 7fc9057fe640 1 --2- 172.21.15.26:0/985748521 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc90010e4e0 0x7fc9001108d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:53.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.877+0000 7fc9057fe640 1 --2- 172.21.15.26:0/985748521 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc900100930 0x7fc900100d10 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:53.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.877+0000 7fc9057fe640 1 -- 172.21.15.26:0/985748521 >> 172.21.15.26:0/985748521 conn(0x7fc9000fc7a0 msgr2=0x7fc9000febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:22:53.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.877+0000 7fc9057fe640 1 -- 172.21.15.26:0/985748521 shutdown_connections 2024-09-11T23:22:53.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.877+0000 7fc9057fe640 1 -- 172.21.15.26:0/985748521 wait complete. 2024-09-11T23:22:53.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.877+0000 7fc9057fe640 1 Processor -- start 2024-09-11T23:22:53.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.877+0000 7fc9057fe640 1 -- start start 2024-09-11T23:22:53.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.877+0000 7fc9057fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc900100930 0x7fc90019a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:53.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.877+0000 7fc9057fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc9001012e0 0x7fc90019aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:53.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.878+0000 7fc9057fe640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc90010e4e0 0x7fc90019ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:53.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.878+0000 7fc9057fe640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc9001141c0 con 0x7fc9001012e0 2024-09-11T23:22:53.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.878+0000 7fc9057fe640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fc900114040 con 0x7fc900100930 2024-09-11T23:22:53.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.878+0000 7fc9057fe640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fc900114340 con 0x7fc90010e4e0 2024-09-11T23:22:53.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.878+0000 7fc8f7fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc9001012e0 0x7fc90019aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:53.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.878+0000 7fc8f7fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc9001012e0 0x7fc90019aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60964/0 (socket says 172.21.15.26:60964) 2024-09-11T23:22:53.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.878+0000 7fc8f7fff640 1 -- 172.21.15.26:0/2123065275 learned_addr learned my addr 172.21.15.26:0/2123065275 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:22:53.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.878+0000 7fc8feffd640 1 --2- 172.21.15.26:0/2123065275 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc900100930 0x7fc90019a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:53.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.878+0000 7fc8ff7fe640 1 --2- 172.21.15.26:0/2123065275 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc90010e4e0 0x7fc90019ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:53.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.878+0000 7fc8f7fff640 1 -- 172.21.15.26:0/2123065275 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc90010e4e0 msgr2=0x7fc90019ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:53.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.878+0000 7fc8f7fff640 1 --2- 172.21.15.26:0/2123065275 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc90010e4e0 0x7fc90019ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:53.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.878+0000 7fc8f7fff640 1 -- 172.21.15.26:0/2123065275 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc900100930 msgr2=0x7fc90019a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:53.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.878+0000 7fc8f7fff640 1 --2- 172.21.15.26:0/2123065275 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc900100930 0x7fc90019a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:53.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.878+0000 7fc8f7fff640 1 -- 172.21.15.26:0/2123065275 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc90019f610 con 0x7fc9001012e0 2024-09-11T23:22:53.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.878+0000 7fc8ff7fe640 1 --2- 172.21.15.26:0/2123065275 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc90010e4e0 0x7fc90019ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:22:53.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.878+0000 7fc8f7fff640 1 --2- 172.21.15.26:0/2123065275 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc9001012e0 0x7fc90019aba0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fc8e800e9f0 tx=0x7fc8e800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:22:53.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.878+0000 7fc8fcff9640 1 -- 172.21.15.26:0/2123065275 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8e800cd80 con 0x7fc9001012e0 2024-09-11T23:22:53.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.879+0000 7fc8fcff9640 1 -- 172.21.15.26:0/2123065275 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc8e8010910 con 0x7fc9001012e0 2024-09-11T23:22:53.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.879+0000 7fc9057fe640 1 -- 172.21.15.26:0/2123065275 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc90019f900 con 0x7fc9001012e0 2024-09-11T23:22:53.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.879+0000 7fc8fcff9640 1 -- 172.21.15.26:0/2123065275 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8e8018990 con 0x7fc9001012e0 2024-09-11T23:22:53.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.879+0000 7fc9057fe640 1 -- 172.21.15.26:0/2123065275 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9001a71e0 con 0x7fc9001012e0 2024-09-11T23:22:53.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.881+0000 7fc8fcff9640 1 -- 172.21.15.26:0/2123065275 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fc8e8010ac0 con 0x7fc9001012e0 2024-09-11T23:22:53.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.881+0000 7fc907bd7740 1 Processor -- start 2024-09-11T23:22:53.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.881+0000 7fc907bd7740 1 -- start start 2024-09-11T23:22:53.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.882+0000 7fc907bd7740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5637fdbd0060 0x5637fdbd0440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:53.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.882+0000 7fc907bd7740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5637fdbd0980 0x5637fdbdad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:53.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.882+0000 7fc907bd7740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5637fdbdb240 0x5637fdbdd630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:53.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.882+0000 7fc907bd7740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5637fdb35620 con 0x5637fdbd0060 2024-09-11T23:22:53.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.882+0000 7fc907bd7740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5637fdadce50 con 0x5637fdbd0980 2024-09-11T23:22:53.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.882+0000 7fc907bd7740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5637fdb315e0 con 0x5637fdbdb240 2024-09-11T23:22:53.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.882+0000 7fc8ff7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5637fdbd0060 0x5637fdbd0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:53.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.882+0000 7fc8ff7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5637fdbd0060 0x5637fdbd0440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60972/0 (socket says 172.21.15.26:60972) 2024-09-11T23:22:53.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.882+0000 7fc8ff7fe640 1 -- 172.21.15.26:0/1130453850 learned_addr learned my addr 172.21.15.26:0/1130453850 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:22:53.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.882+0000 7fc8feffd640 1 --2- 172.21.15.26:0/1130453850 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5637fdbd0980 0x5637fdbdad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:53.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.882+0000 7fc8f7fff640 1 --2- 172.21.15.26:0/1130453850 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5637fdbdb240 0x5637fdbdd630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:53.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.882+0000 7fc8ff7fe640 1 -- 172.21.15.26:0/1130453850 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5637fdbdb240 msgr2=0x5637fdbdd630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:22:53.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.882+0000 7fc8ff7fe640 1 --2- 172.21.15.26:0/1130453850 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5637fdbdb240 0x5637fdbdd630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:53.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.882+0000 7fc8ff7fe640 1 -- 172.21.15.26:0/1130453850 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5637fdbd0980 msgr2=0x5637fdbdad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:22:53.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.882+0000 7fc8ff7fe640 1 --2- 172.21.15.26:0/1130453850 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5637fdbd0980 0x5637fdbdad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:53.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.882+0000 7fc8ff7fe640 1 -- 172.21.15.26:0/1130453850 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5637fdacdd40 con 0x5637fdbd0060 2024-09-11T23:22:53.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.883+0000 7fc8ff7fe640 1 --2- 172.21.15.26:0/1130453850 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5637fdbd0060 0x5637fdbd0440 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fc8f8009ee0 tx=0x7fc8f800f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b770170b6245006 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:22:53.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.883+0000 7fc8fcff9640 1 --2- 172.21.15.26:0/2123065275 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fc8ec0a2cf0 0x7fc8ec0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:53.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.883+0000 7fc8f57fa640 1 -- 172.21.15.26:0/1130453850 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8f8011640 con 0x5637fdbd0060 2024-09-11T23:22:53.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.883+0000 7fc8feffd640 1 --2- 172.21.15.26:0/2123065275 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fc8ec0a2cf0 0x7fc8ec0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:53.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.883+0000 7fc8fcff9640 1 -- 172.21.15.26:0/2123065275 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fc8e80cf820 con 0x7fc9001012e0 2024-09-11T23:22:53.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.883+0000 7fc8f57fa640 1 -- 172.21.15.26:0/1130453850 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc8f8011cc0 con 0x5637fdbd0060 2024-09-11T23:22:53.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.883+0000 7fc8f57fa640 1 -- 172.21.15.26:0/1130453850 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8f801bde0 con 0x5637fdbd0060 2024-09-11T23:22:53.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.884+0000 7fc907bd7740 1 -- 172.21.15.26:0/1130453850 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5637fdbd0060 msgr2=0x5637fdbd0440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:53.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.884+0000 7fc8feffd640 1 --2- 172.21.15.26:0/2123065275 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fc8ec0a2cf0 0x7fc8ec0a51b0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fc8f0000c00 tx=0x7fc8f0005b50 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:22:53.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.884+0000 7fc907bd7740 1 --2- 172.21.15.26:0/1130453850 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5637fdbd0060 0x5637fdbd0440 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fc8f8009ee0 tx=0x7fc8f800f730 comp rx=0 tx=0).stop 2024-09-11T23:22:53.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.884+0000 7fc907bd7740 1 -- 172.21.15.26:0/1130453850 shutdown_connections 2024-09-11T23:22:53.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.884+0000 7fc907bd7740 1 --2- 172.21.15.26:0/1130453850 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5637fdbd0980 0x5637fdbdad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:53.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.884+0000 7fc907bd7740 1 --2- 172.21.15.26:0/1130453850 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5637fdbdb240 0x5637fdbdd630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:53.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.884+0000 7fc907bd7740 1 --2- 172.21.15.26:0/1130453850 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5637fdbd0060 0x5637fdbd0440 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:53.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.884+0000 7fc907bd7740 1 -- 172.21.15.26:0/1130453850 >> 172.21.15.26:0/1130453850 conn(0x5637fdbcc5f0 msgr2=0x5637fdbcc9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:22:53.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.884+0000 7fc907bd7740 1 -- 172.21.15.26:0/1130453850 shutdown_connections 2024-09-11T23:22:53.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.884+0000 7fc907bd7740 1 -- 172.21.15.26:0/1130453850 wait complete. 2024-09-11T23:22:53.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.884+0000 7fc907bd7740 1 -- start start 2024-09-11T23:22:53.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.885+0000 7fc907bd7740 1 Processor -- start 2024-09-11T23:22:53.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.885+0000 7fc907bd7740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5637fdbd0060 0x5637fdbd81c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:53.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.885+0000 7fc907bd7740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5637fdbd0980 0x5637fdbd8700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:53.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.885+0000 7fc907bd7740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5637fdbdb240 0x5637fdbd8c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:53.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.885+0000 7fc907bd7740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5637fdbdde10 con 0x5637fdbdb240 2024-09-11T23:22:53.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.885+0000 7fc907bd7740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5637fdabdf10 con 0x5637fdbd0980 2024-09-11T23:22:53.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.885+0000 7fc907bd7740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5637fdbddf90 con 0x5637fdbd0060 2024-09-11T23:22:53.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.885+0000 7fc907bd7740 10 client.0 _ll_register_callbacks cb 0x5637fdae1db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:22:53.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.885+0000 7fc8feffd640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5637fdbd0060 0x5637fdbd81c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:53.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.885+0000 7fc8ff7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5637fdbdb240 0x5637fdbd8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:53.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.885+0000 7fc8feffd640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5637fdbd0060 0x5637fdbd81c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:56874/0 (socket says 172.21.15.26:56874) 2024-09-11T23:22:53.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.885+0000 7fc8feffd640 1 -- 172.21.15.26:0/4294564744 learned_addr learned my addr 172.21.15.26:0/4294564744 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:22:53.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.885+0000 7fc8ff7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5637fdbdb240 0x5637fdbd8c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60984/0 (socket says 172.21.15.26:60984) 2024-09-11T23:22:53.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.885+0000 7fc8f7fff640 1 --2- 172.21.15.26:0/4294564744 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5637fdbd0980 0x5637fdbd8700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:53.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.886+0000 7fc8ff7fe640 1 -- 172.21.15.26:0/4294564744 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5637fdbd0060 msgr2=0x5637fdbd81c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:22:53.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.886+0000 7fc8ff7fe640 1 --2- 172.21.15.26:0/4294564744 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5637fdbd0060 0x5637fdbd81c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:53.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.886+0000 7fc8ff7fe640 1 -- 172.21.15.26:0/4294564744 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5637fdbd0980 msgr2=0x5637fdbd8700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:22:53.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.886+0000 7fc8ff7fe640 1 --2- 172.21.15.26:0/4294564744 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5637fdbd0980 0x5637fdbd8700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:53.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.886+0000 7fc8ff7fe640 1 -- 172.21.15.26:0/4294564744 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5637fd844580 con 0x5637fdbdb240 2024-09-11T23:22:53.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.886+0000 7fc8ff7fe640 1 --2- 172.21.15.26:0/4294564744 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5637fdbdb240 0x5637fdbd8c40 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fc8f8000f80 tx=0x7fc8f8009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4c7f3dab398a6ff server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:22:53.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.886+0000 7fc8e67fc640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:22:53.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.886+0000 7fc8e67fc640 1 -- 172.21.15.26:0/4294564744 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8f801be30 con 0x5637fdbdb240 2024-09-11T23:22:53.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.886+0000 7fc8e67fc640 1 -- 172.21.15.26:0/4294564744 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fc8f801b410 con 0x5637fdbdb240 2024-09-11T23:22:53.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.886+0000 7fc907bd7740 1 -- 172.21.15.26:0/4294564744 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc8f8011cc0 con 0x5637fdbdb240 2024-09-11T23:22:53.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.886+0000 7fc8e67fc640 1 -- 172.21.15.26:0/4294564744 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fc8f801b610 con 0x5637fdbdb240 2024-09-11T23:22:53.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.886+0000 7fc907bd7740 10 client.16362 fetch_fsmap learned FSMap version 38 2024-09-11T23:22:53.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.886+0000 7fc907bd7740 1 -- 172.21.15.26:0/4294564744 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5637fd90e960 con 0x5637fdbdb240 2024-09-11T23:22:53.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.887+0000 7fc8e67fc640 1 -- 172.21.15.26:0/4294564744 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc8f8011cc0 con 0x5637fdbdb240 2024-09-11T23:22:53.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.887+0000 7fc8e67fc640 1 -- 172.21.15.26:0/4294564744 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 38) ==== 8414+0+0 (secure 0 0 0) 0x7fc8f8011540 con 0x5637fdbdb240 2024-09-11T23:22:53.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.887+0000 7fc907bd7740 10 client.16362 fetch_fsmap finished waiting for FSMap version 38 2024-09-11T23:22:53.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.887+0000 7fc907bd7740 10 client.16362 resolve_mds: validated gid 24485 aka daemon mds.b 2024-09-11T23:22:53.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.887+0000 7fc907bd7740 20 client.16362 populate_metadata read hostname 'smithi026' 2024-09-11T23:22:53.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.887+0000 7fc907bd7740 1 --2- 172.21.15.26:0/4294564744 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5637fdbeadd0 0x5637fdbed1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:53.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.887+0000 7fc907bd7740 4 client.16362 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:22:53.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.887+0000 7fc907bd7740 1 -- 172.21.15.26:0/4294564744 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5637fdb42b90 con 0x5637fdbeadd0 2024-09-11T23:22:53.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.887+0000 7fc8feffd640 1 --2- 172.21.15.26:0/4294564744 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5637fdbeadd0 0x5637fdbed1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:53.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.888+0000 7fc8e67fc640 10 client.16362 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:22:53.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.888+0000 7fc8feffd640 1 --2- 172.21.15.26:0/4294564744 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5637fdbeadd0 0x5637fdbed1c0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=30ef0be078315b6 server_cookie=9ecc933a167becd8 in_seq=0 out_seq=0 2024-09-11T23:22:53.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.890+0000 7fc8e67fc640 1 -- 172.21.15.26:0/4294564744 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5637fdb42b90 con 0x5637fdbeadd0 2024-09-11T23:22:53.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.890+0000 7fc8e67fc640 10 client.16362 handle_command_reply: tid=0 2024-09-11T23:22:53.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.890+0000 7fc907bd7740 1 client.16362 shutdown 2024-09-11T23:22:53.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.890+0000 7fc907bd7740 1 -- 172.21.15.26:0/4294564744 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5637fdbdb240 msgr2=0x5637fdbd8c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:53.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.890+0000 7fc907bd7740 1 --2- 172.21.15.26:0/4294564744 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5637fdbdb240 0x5637fdbd8c40 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fc8f8000f80 tx=0x7fc8f8009cd0 comp rx=0 tx=0).stop 2024-09-11T23:22:53.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.890+0000 7fc907bd7740 1 -- 172.21.15.26:0/4294564744 shutdown_connections 2024-09-11T23:22:53.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.890+0000 7fc907bd7740 1 --2- 172.21.15.26:0/4294564744 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5637fdbd0980 0x5637fdbd8700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:53.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.890+0000 7fc907bd7740 1 --2- 172.21.15.26:0/4294564744 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5637fdbeadd0 0x5637fdbed1c0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:53.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.890+0000 7fc8e67fc640 0 client.16362 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:22:53.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.890+0000 7fc907bd7740 1 --2- 172.21.15.26:0/4294564744 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5637fdbdb240 0x5637fdbd8c40 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:53.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.890+0000 7fc907bd7740 1 --2- 172.21.15.26:0/4294564744 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5637fdbd0060 0x5637fdbd81c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:53.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.890+0000 7fc907bd7740 1 -- 172.21.15.26:0/4294564744 >> 172.21.15.26:0/4294564744 conn(0x5637fdbcc5f0 msgr2=0x5637fdbcb750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:22:53.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.890+0000 7fc907bd7740 1 -- 172.21.15.26:0/4294564744 shutdown_connections 2024-09-11T23:22:53.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.890+0000 7fc907bd7740 1 -- 172.21.15.26:0/4294564744 wait complete. 2024-09-11T23:22:53.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.890+0000 7fc907bd7740 20 client.16362 trim_cache size 0 max 16384 2024-09-11T23:22:53.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.909+0000 7fc907bd7740 1 Processor -- start 2024-09-11T23:22:53.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.909+0000 7fc907bd7740 1 -- start start 2024-09-11T23:22:53.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.910+0000 7fc907bd7740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5637fdbd0060 0x5637fdbda130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:53.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.910+0000 7fc907bd7740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5637fdbd0980 0x5637fdbf2790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:53.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.910+0000 7fc907bd7740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5637fdbdb240 0x5637fdbf2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:53.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.910+0000 7fc907bd7740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc9001afb20 con 0x5637fdbd0980 2024-09-11T23:22:53.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.910+0000 7fc907bd7740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fc9001b18d0 con 0x5637fdbdb240 2024-09-11T23:22:53.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.910+0000 7fc907bd7740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fc9001b02b0 con 0x5637fdbd0060 2024-09-11T23:22:53.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.910+0000 7fc8feffd640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5637fdbd0060 0x5637fdbda130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:53.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.910+0000 7fc8f7fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5637fdbd0980 0x5637fdbf2790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:53.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.910+0000 7fc8feffd640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5637fdbd0060 0x5637fdbda130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:56884/0 (socket says 172.21.15.26:56884) 2024-09-11T23:22:53.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.910+0000 7fc8feffd640 1 -- 172.21.15.26:0/4268908099 learned_addr learned my addr 172.21.15.26:0/4268908099 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:22:53.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.910+0000 7fc8ff7fe640 1 --2- 172.21.15.26:0/4268908099 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5637fdbdb240 0x5637fdbf2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:53.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.910+0000 7fc8feffd640 1 -- 172.21.15.26:0/4268908099 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5637fdbdb240 msgr2=0x5637fdbf2cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:22:53.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.910+0000 7fc8feffd640 1 --2- 172.21.15.26:0/4268908099 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5637fdbdb240 0x5637fdbf2cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:53.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.910+0000 7fc8feffd640 1 -- 172.21.15.26:0/4268908099 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5637fdbd0980 msgr2=0x5637fdbf2790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:22:53.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.910+0000 7fc8feffd640 1 --2- 172.21.15.26:0/4268908099 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5637fdbd0980 0x5637fdbf2790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:53.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.910+0000 7fc8feffd640 1 -- 172.21.15.26:0/4268908099 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5637fdbcd6a0 con 0x5637fdbd0060 2024-09-11T23:22:53.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.911+0000 7fc8feffd640 1 --2- 172.21.15.26:0/4268908099 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5637fdbd0060 0x5637fdbda130 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fc8f0009230 tx=0x7fc8f003f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d7d0bc6432c1506a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:22:53.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.911+0000 7fc8f4ff9640 1 -- 172.21.15.26:0/4268908099 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8f0035040 con 0x5637fdbd0060 2024-09-11T23:22:53.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.911+0000 7fc8f4ff9640 1 -- 172.21.15.26:0/4268908099 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc8f005a070 con 0x5637fdbd0060 2024-09-11T23:22:53.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.911+0000 7fc8f4ff9640 1 -- 172.21.15.26:0/4268908099 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8f0032bc0 con 0x5637fdbd0060 2024-09-11T23:22:53.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.912+0000 7fc907bd7740 1 -- 172.21.15.26:0/4268908099 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5637fdbd0060 msgr2=0x5637fdbda130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:53.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.912+0000 7fc907bd7740 1 --2- 172.21.15.26:0/4268908099 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5637fdbd0060 0x5637fdbda130 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fc8f0009230 tx=0x7fc8f003f000 comp rx=0 tx=0).stop 2024-09-11T23:22:53.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.912+0000 7fc907bd7740 1 -- 172.21.15.26:0/4268908099 shutdown_connections 2024-09-11T23:22:53.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.912+0000 7fc907bd7740 1 --2- 172.21.15.26:0/4268908099 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5637fdbdb240 0x5637fdbf2cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:53.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.912+0000 7fc907bd7740 1 --2- 172.21.15.26:0/4268908099 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5637fdbd0980 0x5637fdbf2790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:53.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.912+0000 7fc907bd7740 1 --2- 172.21.15.26:0/4268908099 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5637fdbd0060 0x5637fdbda130 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:53.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.912+0000 7fc907bd7740 1 -- 172.21.15.26:0/4268908099 >> 172.21.15.26:0/4268908099 conn(0x5637fdbcc5f0 msgr2=0x5637fdbd9a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:22:53.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.912+0000 7fc907bd7740 1 -- 172.21.15.26:0/4268908099 shutdown_connections 2024-09-11T23:22:53.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.912+0000 7fc907bd7740 1 -- 172.21.15.26:0/4268908099 wait complete. 2024-09-11T23:22:53.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.912+0000 7fc907bd7740 1 -- start start 2024-09-11T23:22:53.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.912+0000 7fc907bd7740 1 Processor -- start 2024-09-11T23:22:53.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.913+0000 7fc907bd7740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5637fdbd0060 0x5637fdbe31e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:53.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.913+0000 7fc907bd7740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5637fdbd0980 0x5637fdbe3720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:53.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.913+0000 7fc907bd7740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5637fdbdb240 0x5637fdbe01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:53.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.913+0000 7fc907bd7740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc9001b11d0 con 0x5637fdbd0980 2024-09-11T23:22:53.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.913+0000 7fc907bd7740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fc9001b0a40 con 0x5637fdbdb240 2024-09-11T23:22:53.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.913+0000 7fc907bd7740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fc9001adb00 con 0x5637fdbd0060 2024-09-11T23:22:53.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.913+0000 7fc907bd7740 10 client.0 _ll_register_callbacks cb 0x7fc8f8009de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:22:53.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.913+0000 7fc8f7fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5637fdbd0980 0x5637fdbe3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:53.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.913+0000 7fc8f7fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5637fdbd0980 0x5637fdbe3720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:32768/0 (socket says 172.21.15.26:32768) 2024-09-11T23:22:53.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.913+0000 7fc8f7fff640 1 -- 172.21.15.26:0/1022624512 learned_addr learned my addr 172.21.15.26:0/1022624512 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:22:53.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.913+0000 7fc8ff7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5637fdbdb240 0x5637fdbe01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:53.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.913+0000 7fc8feffd640 1 --2- 172.21.15.26:0/1022624512 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5637fdbd0060 0x5637fdbe31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:53.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.913+0000 7fc8f7fff640 1 -- 172.21.15.26:0/1022624512 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5637fdbd0060 msgr2=0x5637fdbe31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:22:53.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.913+0000 7fc8f7fff640 1 --2- 172.21.15.26:0/1022624512 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5637fdbd0060 0x5637fdbe31e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:53.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.913+0000 7fc8f7fff640 1 -- 172.21.15.26:0/1022624512 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5637fdbdb240 msgr2=0x5637fdbe01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:22:53.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.913+0000 7fc8f7fff640 1 --2- 172.21.15.26:0/1022624512 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5637fdbdb240 0x5637fdbe01b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:53.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.913+0000 7fc8f7fff640 1 -- 172.21.15.26:0/1022624512 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5637fdbe06f0 con 0x5637fdbd0980 2024-09-11T23:22:53.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.913+0000 7fc8ff7fe640 1 --2- 172.21.15.26:0/1022624512 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5637fdbdb240 0x5637fdbe01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:22:53.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.913+0000 7fc8f7fff640 1 --2- 172.21.15.26:0/1022624512 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5637fdbd0980 0x5637fdbe3720 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fc8e800c6a0 tx=0x7fc8e809c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba3a1f7692e987f7 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:22:53.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.913+0000 7fc8e67fc640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:22:53.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.914+0000 7fc8e67fc640 1 -- 172.21.15.26:0/1022624512 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8e8018d30 con 0x5637fdbd0980 2024-09-11T23:22:53.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.914+0000 7fc8e67fc640 1 -- 172.21.15.26:0/1022624512 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fc8e809b1a0 con 0x5637fdbd0980 2024-09-11T23:22:53.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.914+0000 7fc907bd7740 1 -- 172.21.15.26:0/1022624512 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc8f005a070 con 0x5637fdbd0980 2024-09-11T23:22:53.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.914+0000 7fc8e67fc640 1 -- 172.21.15.26:0/1022624512 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fc8e80a49c0 con 0x5637fdbd0980 2024-09-11T23:22:53.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.914+0000 7fc907bd7740 10 client.16368 fetch_fsmap learned FSMap version 38 2024-09-11T23:22:53.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.914+0000 7fc907bd7740 1 -- 172.21.15.26:0/1022624512 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5637fdc56010 con 0x5637fdbd0980 2024-09-11T23:22:53.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.914+0000 7fc8e67fc640 1 -- 172.21.15.26:0/1022624512 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc8f005a070 con 0x5637fdbd0980 2024-09-11T23:22:53.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.914+0000 7fc8e67fc640 1 -- 172.21.15.26:0/1022624512 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 38) ==== 8414+0+0 (secure 0 0 0) 0x7fc8e809ac10 con 0x5637fdbd0980 2024-09-11T23:22:53.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.914+0000 7fc907bd7740 10 client.16368 fetch_fsmap finished waiting for FSMap version 38 2024-09-11T23:22:53.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.914+0000 7fc907bd7740 10 client.16368 resolve_mds: validated gid 24485 aka daemon mds.b 2024-09-11T23:22:53.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.914+0000 7fc907bd7740 20 client.16368 populate_metadata read hostname 'smithi026' 2024-09-11T23:22:53.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.914+0000 7fc907bd7740 1 --2- 172.21.15.26:0/1022624512 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5637fdbeadd0 0x5637fdc587b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:53.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.914+0000 7fc907bd7740 4 client.16368 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "82d019e7", "release": true, "await": true}] 2024-09-11T23:22:53.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.914+0000 7fc907bd7740 1 -- 172.21.15.26:0/1022624512 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "quiesce db", "set_id": "82d019e7", "release": true, "await": true}) -- 0x5637fdbc7950 con 0x5637fdbeadd0 2024-09-11T23:22:53.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.915+0000 7fc8feffd640 1 --2- 172.21.15.26:0/1022624512 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5637fdbeadd0 0x5637fdc587b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:53.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.915+0000 7fc8feffd640 1 --2- 172.21.15.26:0/1022624512 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5637fdbeadd0 0x5637fdc587b0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f3f414926b34684 server_cookie=87a5c4e389adaaf2 in_seq=0 out_seq=0 2024-09-11T23:22:53.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:53.915+0000 7fc8e67fc640 10 client.16368 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:22:53.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:22:53 smithi136 ceph-mon[35121]: pgmap v138: 97 pgs: 97 active+clean; 1.3 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 13 MiB/s wr, 781 op/s 2024-09-11T23:22:54.052 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:22:53 smithi076 ceph-mon[35296]: pgmap v138: 97 pgs: 97 active+clean; 1.3 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 13 MiB/s wr, 781 op/s 2024-09-11T23:22:54.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:54.819+0000 7fc8e67fc640 1 -- 172.21.15.26:0/1022624512 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5637fdbc7950 con 0x5637fdbeadd0 2024-09-11T23:22:54.822 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-11T23:22:54.822 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 38, 2024-09-11T23:22:54.822 INFO:teuthology.orchestra.run.smithi026.stdout: "leader": 24485, 2024-09-11T23:22:54.822 INFO:teuthology.orchestra.run.smithi026.stdout: "set_version": 16, 2024-09-11T23:22:54.822 INFO:teuthology.orchestra.run.smithi026.stdout: "sets": { 2024-09-11T23:22:54.823 INFO:teuthology.orchestra.run.smithi026.stdout: "82d019e7": { 2024-09-11T23:22:54.823 INFO:teuthology.orchestra.run.smithi026.stdout: "version": 16, 2024-09-11T23:22:54.823 INFO:teuthology.orchestra.run.smithi026.stdout: "age_ref": 0.0, 2024-09-11T23:22:54.823 INFO:teuthology.orchestra.run.smithi026.stdout: "state": { 2024-09-11T23:22:54.823 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "RELEASED", 2024-09-11T23:22:54.823 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.0 2024-09-11T23:22:54.823 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:22:54.823 INFO:teuthology.orchestra.run.smithi026.stdout: "timeout": 90.0, 2024-09-11T23:22:54.823 INFO:teuthology.orchestra.run.smithi026.stdout: "expiration": 140.2, 2024-09-11T23:22:54.823 INFO:teuthology.orchestra.run.smithi026.stdout: "members": { 2024-09-11T23:22:54.823 INFO:teuthology.orchestra.run.smithi026.stdout: "file:/": { 2024-09-11T23:22:54.823 INFO:teuthology.orchestra.run.smithi026.stdout: "excluded": false, 2024-09-11T23:22:54.823 INFO:teuthology.orchestra.run.smithi026.stdout: "state": { 2024-09-11T23:22:54.824 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "RELEASED", 2024-09-11T23:22:54.824 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.0 2024-09-11T23:22:54.824 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:22:54.824 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:22:54.824 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:22:54.824 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:22:54.824 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:22:54.824 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-11T23:22:54.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:54.820+0000 7fc8e67fc640 10 client.16368 handle_command_reply: tid=0 2024-09-11T23:22:54.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:54.820+0000 7fc907bd7740 1 client.16368 shutdown 2024-09-11T23:22:54.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:54.820+0000 7fc907bd7740 1 -- 172.21.15.26:0/1022624512 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5637fdbd0980 msgr2=0x5637fdbe3720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:54.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:54.820+0000 7fc907bd7740 1 --2- 172.21.15.26:0/1022624512 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5637fdbd0980 0x5637fdbe3720 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fc8e800c6a0 tx=0x7fc8e809c510 comp rx=0 tx=0).stop 2024-09-11T23:22:54.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:54.820+0000 7fc907bd7740 1 -- 172.21.15.26:0/1022624512 shutdown_connections 2024-09-11T23:22:54.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:54.820+0000 7fc907bd7740 1 --2- 172.21.15.26:0/1022624512 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5637fdbdb240 0x5637fdbe01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:54.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:54.820+0000 7fc907bd7740 1 --2- 172.21.15.26:0/1022624512 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5637fdbeadd0 0x5637fdc587b0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:54.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:54.821+0000 7fc907bd7740 1 --2- 172.21.15.26:0/1022624512 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5637fdbd0980 0x5637fdbe3720 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:54.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:54.821+0000 7fc8e67fc640 0 client.16368 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:22:54.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:54.821+0000 7fc907bd7740 1 --2- 172.21.15.26:0/1022624512 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5637fdbd0060 0x5637fdbe31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:54.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:54.821+0000 7fc907bd7740 1 -- 172.21.15.26:0/1022624512 >> 172.21.15.26:0/1022624512 conn(0x5637fdbcc5f0 msgr2=0x5637fdbd2320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:22:54.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:54.821+0000 7fc907bd7740 1 -- 172.21.15.26:0/1022624512 shutdown_connections 2024-09-11T23:22:54.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:54.821+0000 7fc907bd7740 1 -- 172.21.15.26:0/1022624512 wait complete. 2024-09-11T23:22:54.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:54.821+0000 7fc907bd7740 20 client.16368 trim_cache size 0 max 16384 2024-09-11T23:22:54.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:54.823+0000 7fc9057fe640 1 -- 172.21.15.26:0/2123065275 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fc8ec0a2cf0 msgr2=0x7fc8ec0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:54.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:54.823+0000 7fc9057fe640 1 --2- 172.21.15.26:0/2123065275 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fc8ec0a2cf0 0x7fc8ec0a51b0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fc8f0000c00 tx=0x7fc8f0005b50 comp rx=0 tx=0).stop 2024-09-11T23:22:54.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:54.823+0000 7fc9057fe640 1 -- 172.21.15.26:0/2123065275 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc9001012e0 msgr2=0x7fc90019aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:54.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:54.823+0000 7fc9057fe640 1 --2- 172.21.15.26:0/2123065275 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc9001012e0 0x7fc90019aba0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fc8e800e9f0 tx=0x7fc8e800eeb0 comp rx=0 tx=0).stop 2024-09-11T23:22:54.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:54.823+0000 7fc9057fe640 1 -- 172.21.15.26:0/2123065275 shutdown_connections 2024-09-11T23:22:54.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:54.823+0000 7fc9057fe640 1 --2- 172.21.15.26:0/2123065275 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fc8ec0a2cf0 0x7fc8ec0a51b0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:54.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:54.823+0000 7fc9057fe640 1 --2- 172.21.15.26:0/2123065275 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc90010e4e0 0x7fc90019ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:54.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:54.823+0000 7fc9057fe640 1 --2- 172.21.15.26:0/2123065275 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc9001012e0 0x7fc90019aba0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:54.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:54.823+0000 7fc9057fe640 1 --2- 172.21.15.26:0/2123065275 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc900100930 0x7fc90019a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:54.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:54.823+0000 7fc9057fe640 1 -- 172.21.15.26:0/2123065275 >> 172.21.15.26:0/2123065275 conn(0x7fc9000fc7a0 msgr2=0x7fc9000feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:22:54.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:54.823+0000 7fc9057fe640 1 -- 172.21.15.26:0/2123065275 shutdown_connections 2024-09-11T23:22:54.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:54.823+0000 7fc9057fe640 1 -- 172.21.15.26:0/2123065275 wait complete. 2024-09-11T23:22:55.201 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:55.200+0000 7ff494ff9640 1 -- 172.21.15.26:0/2316134916 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d88da72d70 con 0x55d88db8c1d0 2024-09-11T23:22:55.201 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:55.200+0000 7ff494ff9640 10 client.16344 handle_command_reply: tid=0 2024-09-11T23:22:55.202 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:55.200+0000 7ff4b682e740 1 client.16344 shutdown 2024-09-11T23:22:55.202 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:55.201+0000 7ff4b682e740 1 -- 172.21.15.26:0/2316134916 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d88db07410 msgr2=0x55d88db14c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:55.202 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:55.201+0000 7ff4b682e740 1 --2- 172.21.15.26:0/2316134916 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d88db07410 0x55d88db14c80 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7ff49c009730 tx=0x7ff49c004730 comp rx=0 tx=0).stop 2024-09-11T23:22:55.202 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:55.201+0000 7ff4b682e740 1 -- 172.21.15.26:0/2316134916 shutdown_connections 2024-09-11T23:22:55.202 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:55.201+0000 7ff4b682e740 1 --2- 172.21.15.26:0/2316134916 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d88db11cb0 0x55d88db15410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:55.202 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:55.201+0000 7ff4b682e740 1 --2- 172.21.15.26:0/2316134916 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55d88db8c1d0 0x55d88db8e5f0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:55.202 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:55.201+0000 7ff4b682e740 1 --2- 172.21.15.26:0/2316134916 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d88db07410 0x55d88db14c80 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:55.202 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:55.201+0000 7ff4b682e740 1 --2- 172.21.15.26:0/2316134916 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d88db06af0 0x55d88db1a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:55.202 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:55.201+0000 7ff494ff9640 0 client.16344 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:22:55.202 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:55.201+0000 7ff4b682e740 1 -- 172.21.15.26:0/2316134916 >> 172.21.15.26:0/2316134916 conn(0x55d88db03060 msgr2=0x55d88db08dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:22:55.202 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:55.201+0000 7ff4b682e740 1 -- 172.21.15.26:0/2316134916 shutdown_connections 2024-09-11T23:22:55.202 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:55.201+0000 7ff4b682e740 1 -- 172.21.15.26:0/2316134916 wait complete. 2024-09-11T23:22:55.202 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:55.201+0000 7ff4b682e740 20 client.16344 trim_cache size 0 max 16384 2024-09-11T23:22:55.203 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-09-11T23:22:55.203 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "message": "", 2024-09-11T23:22:55.203 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-09-11T23:22:55.203 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-09-11T23:22:55.204 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:55.202+0000 7ff4b39c0640 1 -- 172.21.15.26:0/3103898987 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7ff4980a2da0 msgr2=0x7ff4980a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:55.204 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:55.202+0000 7ff4b39c0640 1 --2- 172.21.15.26:0/3103898987 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7ff4980a2da0 0x7ff4980a5260 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x55d88db043e0 tx=0x7ff49c0096c0 comp rx=0 tx=0).stop 2024-09-11T23:22:55.204 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:55.202+0000 7ff4b39c0640 1 -- 172.21.15.26:0/3103898987 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff4ac10c0f0 msgr2=0x7ff4ac19cc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:55.204 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:55.202+0000 7ff4b39c0640 1 --2- 172.21.15.26:0/3103898987 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff4ac10c0f0 0x7ff4ac19cc10 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7ff4a4004af0 tx=0x7ff4a402f3a0 comp rx=0 tx=0).stop 2024-09-11T23:22:55.204 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:55.202+0000 7ff4b39c0640 1 -- 172.21.15.26:0/3103898987 shutdown_connections 2024-09-11T23:22:55.204 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:55.202+0000 7ff4b39c0640 1 --2- 172.21.15.26:0/3103898987 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7ff4980a2da0 0x7ff4980a5260 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:55.204 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:55.202+0000 7ff4b39c0640 1 --2- 172.21.15.26:0/3103898987 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff4ac103640 0x7ff4ac198890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:55.204 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:55.202+0000 7ff4b39c0640 1 --2- 172.21.15.26:0/3103898987 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff4ac10c0f0 0x7ff4ac19cc10 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:55.205 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:55.202+0000 7ff4b39c0640 1 --2- 172.21.15.26:0/3103898987 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff4ac102d20 0x7ff4ac198350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:55.205 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:55.202+0000 7ff4b39c0640 1 -- 172.21.15.26:0/3103898987 >> 172.21.15.26:0/3103898987 conn(0x7ff4ac077990 msgr2=0x7ff4ac0fcee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:22:55.205 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:55.202+0000 7ff4b39c0640 1 -- 172.21.15.26:0/3103898987 shutdown_connections 2024-09-11T23:22:55.205 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:22:55.202+0000 7ff4b39c0640 1 -- 172.21.15.26:0/3103898987 wait complete. 2024-09-11T23:22:55.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:22:55 smithi026 ceph-mon[31172]: pgmap v139: 97 pgs: 97 active+clean; 1.3 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 15 MiB/s wr, 1.03k op/s 2024-09-11T23:22:55.687 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '82d019e7', total seconds elapsed: 37.3 2024-09-11T23:22:55.687 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 20.15 2024-09-11T23:22:55.687 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:22:55.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:22:55 smithi136 ceph-mon[35121]: pgmap v139: 97 pgs: 97 active+clean; 1.3 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 15 MiB/s wr, 1.03k op/s 2024-09-11T23:22:56.027 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:22:56.052 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:22:55 smithi076 ceph-mon[35296]: pgmap v139: 97 pgs: 97 active+clean; 1.3 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 15 MiB/s wr, 1.03k op/s 2024-09-11T23:22:56.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:56.561+0000 7f414503f640 1 -- 172.21.15.26:0/3790947847 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4140106f10 msgr2=0x7f414010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:56.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:56.561+0000 7f414503f640 1 --2- 172.21.15.26:0/3790947847 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4140106f10 0x7f414010ea90 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f4138009be0 tx=0x7f413802a6c0 comp rx=0 tx=0).stop 2024-09-11T23:22:56.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:56.561+0000 7f414503f640 1 -- 172.21.15.26:0/3790947847 shutdown_connections 2024-09-11T23:22:56.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:56.561+0000 7f414503f640 1 --2- 172.21.15.26:0/3790947847 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4140102790 0x7f41401067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:56.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:56.561+0000 7f414503f640 1 --2- 172.21.15.26:0/3790947847 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4140106f10 0x7f414010ea90 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:56.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:56.561+0000 7f414503f640 1 --2- 172.21.15.26:0/3790947847 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4140101de0 0x7f41401021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:56.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:56.561+0000 7f414503f640 1 -- 172.21.15.26:0/3790947847 >> 172.21.15.26:0/3790947847 conn(0x7f41400fd880 msgr2=0x7f41400ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:22:56.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:56.561+0000 7f414503f640 1 -- 172.21.15.26:0/3790947847 shutdown_connections 2024-09-11T23:22:56.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:56.561+0000 7f414503f640 1 -- 172.21.15.26:0/3790947847 wait complete. 2024-09-11T23:22:56.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:56.562+0000 7f414503f640 1 Processor -- start 2024-09-11T23:22:56.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:56.563+0000 7f414503f640 1 -- start start 2024-09-11T23:22:56.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:56.563+0000 7f414503f640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4140101de0 0x7f4140198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:56.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:56.563+0000 7f414503f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4140102790 0x7f41401988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:56.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:56.563+0000 7f414503f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4140106f10 0x7f414019cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:56.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:56.563+0000 7f414503f640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f4140111f20 con 0x7f4140106f10 2024-09-11T23:22:56.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:56.563+0000 7f414503f640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f4140111da0 con 0x7f4140102790 2024-09-11T23:22:56.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:56.563+0000 7f414503f640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f41401120a0 con 0x7f4140101de0 2024-09-11T23:22:56.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:56.563+0000 7f413659c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4140102790 0x7f41401988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:56.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:56.563+0000 7f413ed9d640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4140101de0 0x7f4140198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:56.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:56.563+0000 7f413f59e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4140106f10 0x7f414019cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:56.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:56.563+0000 7f413659c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4140102790 0x7f41401988a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:44912/0 (socket says 172.21.15.26:44912) 2024-09-11T23:22:56.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:56.563+0000 7f413659c640 1 -- 172.21.15.26:0/2293008465 learned_addr learned my addr 172.21.15.26:0/2293008465 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:22:56.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:56.563+0000 7f413ed9d640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4140101de0 0x7f4140198360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:56894/0 (socket says 172.21.15.26:56894) 2024-09-11T23:22:56.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:56.564+0000 7f413ed9d640 1 -- 172.21.15.26:0/2293008465 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4140102790 msgr2=0x7f41401988a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:56.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:56.564+0000 7f413ed9d640 1 --2- 172.21.15.26:0/2293008465 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4140102790 0x7f41401988a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:56.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:56.564+0000 7f413ed9d640 1 -- 172.21.15.26:0/2293008465 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4140106f10 msgr2=0x7f414019cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:56.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:56.564+0000 7f413ed9d640 1 --2- 172.21.15.26:0/2293008465 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4140106f10 0x7f414019cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:56.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:56.564+0000 7f413ed9d640 1 -- 172.21.15.26:0/2293008465 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f414019d310 con 0x7f4140101de0 2024-09-11T23:22:56.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:56.564+0000 7f413f59e640 1 --2- 172.21.15.26:0/2293008465 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4140106f10 0x7f414019cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:22:56.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:56.564+0000 7f413659c640 1 --2- 172.21.15.26:0/2293008465 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4140102790 0x7f41401988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:22:56.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:56.564+0000 7f413ed9d640 1 --2- 172.21.15.26:0/2293008465 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4140101de0 0x7f4140198360 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f413000e790 tx=0x7f413000ec50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:22:56.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:56.564+0000 7f413cd99640 1 -- 172.21.15.26:0/2293008465 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f413000eea0 con 0x7f4140101de0 2024-09-11T23:22:56.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:56.564+0000 7f413cd99640 1 -- 172.21.15.26:0/2293008465 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f413000ccc0 con 0x7f4140101de0 2024-09-11T23:22:56.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:56.564+0000 7f414503f640 1 -- 172.21.15.26:0/2293008465 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f414019d600 con 0x7f4140101de0 2024-09-11T23:22:56.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:56.564+0000 7f413cd99640 1 -- 172.21.15.26:0/2293008465 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4130018780 con 0x7f4140101de0 2024-09-11T23:22:56.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:56.564+0000 7f414503f640 1 -- 172.21.15.26:0/2293008465 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4140074420 con 0x7f4140101de0 2024-09-11T23:22:56.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:56.565+0000 7f414503f640 1 -- 172.21.15.26:0/2293008465 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41401021c0 con 0x7f4140101de0 2024-09-11T23:22:56.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:56.567+0000 7f413cd99640 1 -- 172.21.15.26:0/2293008465 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f4130010430 con 0x7f4140101de0 2024-09-11T23:22:56.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:56.568+0000 7f413cd99640 1 --2- 172.21.15.26:0/2293008465 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f412c0a2d70 0x7f412c0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:56.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:56.568+0000 7f413cd99640 1 -- 172.21.15.26:0/2293008465 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f41300cf880 con 0x7f4140101de0 2024-09-11T23:22:56.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:56.568+0000 7f413659c640 1 --2- 172.21.15.26:0/2293008465 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f412c0a2d70 0x7f412c0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:56.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:56.568+0000 7f413659c640 1 --2- 172.21.15.26:0/2293008465 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f412c0a2d70 0x7f412c0a5230 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f41400fdd10 tx=0x7f4128005f90 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:22:56.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:56.571+0000 7f413cd99640 1 -- 172.21.15.26:0/2293008465 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f413009be60 con 0x7f4140101de0 2024-09-11T23:22:56.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:56.838+0000 7f414503f640 1 -- 172.21.15.26:0/2293008465 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f414010a0f0 con 0x7f4140101de0 2024-09-11T23:22:56.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:56.838+0000 7f413cd99640 1 -- 172.21.15.26:0/2293008465 <== mon.1 v2:172.21.15.136:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) ==== 94+0+11994 (secure 0 0 0) 0x7f41300a0160 con 0x7f4140101de0 2024-09-11T23:22:56.839 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-11T23:22:56.839 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":38,"btime":"2024-09-11T23:22:31:575477+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14499,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/367659515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":367659515},{"type":"v1","addr":"172.21.15.26:6835","nonce":367659515}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14511,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/2041065299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":2041065299},{"type":"v1","addr":"172.21.15.26:6837","nonce":2041065299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":38,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:16:26.544203+0000","modified":"2024-09-11T23:22:31.018594+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24485,"mds_1":24463,"mds_2":24458,"mds_3":24448,"mds_4":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14550":{"gid":14550,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6839/2233068305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":2233068305},{"type":"v1","addr":"172.21.15.26:6839","nonce":2233068305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14562":{"gid":14562,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6841/2984982336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":2984982336},{"type":"v1","addr":"172.21.15.26:6841","nonce":2984982336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6833/2019841455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":2019841455},{"type":"v1","addr":"172.21.15.136:6833","nonce":2019841455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.76:6833/3438147050","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3438147050},{"type":"v1","addr":"172.21.15.76:6833","nonce":3438147050}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6835/3617665975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3617665975},{"type":"v1","addr":"172.21.15.136:6835","nonce":3617665975}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":12,"addr":"172.21.15.76:6835/393308499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":393308499},{"type":"v1","addr":"172.21.15.76:6835","nonce":393308499}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":10,"addr":"172.21.15.136:6837/1633082387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":1633082387},{"type":"v1","addr":"172.21.15.136:6837","nonce":1633082387}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.76:6837/2078978748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":2078978748},{"type":"v1","addr":"172.21.15.76:6837","nonce":2078978748}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":63,"addr":"172.21.15.136:6839/3211398630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":3211398630},{"type":"v1","addr":"172.21.15.136:6839","nonce":3211398630}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.76:6839/2077637090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6838","nonce":2077637090},{"type":"v1","addr":"172.21.15.76:6839","nonce":2077637090}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24485,"qdb_cluster":[24485,24463,24458,24448,24443]},"id":1}]} 2024-09-11T23:22:56.839 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 38 2024-09-11T23:22:56.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:56.841+0000 7f414503f640 1 -- 172.21.15.26:0/2293008465 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f412c0a2d70 msgr2=0x7f412c0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:56.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:56.841+0000 7f414503f640 1 --2- 172.21.15.26:0/2293008465 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f412c0a2d70 0x7f412c0a5230 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f41400fdd10 tx=0x7f4128005f90 comp rx=0 tx=0).stop 2024-09-11T23:22:56.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:56.841+0000 7f414503f640 1 -- 172.21.15.26:0/2293008465 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4140101de0 msgr2=0x7f4140198360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:56.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:56.841+0000 7f414503f640 1 --2- 172.21.15.26:0/2293008465 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4140101de0 0x7f4140198360 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f413000e790 tx=0x7f413000ec50 comp rx=0 tx=0).stop 2024-09-11T23:22:56.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:56.841+0000 7f414503f640 1 -- 172.21.15.26:0/2293008465 shutdown_connections 2024-09-11T23:22:56.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:56.841+0000 7f414503f640 1 --2- 172.21.15.26:0/2293008465 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f412c0a2d70 0x7f412c0a5230 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:56.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:56.841+0000 7f414503f640 1 --2- 172.21.15.26:0/2293008465 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4140102790 0x7f41401988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:56.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:56.841+0000 7f414503f640 1 --2- 172.21.15.26:0/2293008465 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4140106f10 0x7f414019cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:56.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:56.841+0000 7f414503f640 1 --2- 172.21.15.26:0/2293008465 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4140101de0 0x7f4140198360 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:56.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:56.841+0000 7f414503f640 1 -- 172.21.15.26:0/2293008465 >> 172.21.15.26:0/2293008465 conn(0x7f41400fd880 msgr2=0x7f41400feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:22:56.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:56.841+0000 7f414503f640 1 -- 172.21.15.26:0/2293008465 shutdown_connections 2024-09-11T23:22:56.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:56.841+0000 7f414503f640 1 -- 172.21.15.26:0/2293008465 wait complete. 2024-09-11T23:22:57.708 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24485 quiesce db / --timeout 90 --expiration 140.15 --await-for 45.0' 2024-09-11T23:22:57.708 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24485 quiesce db / --timeout 90 --expiration 140.15 --await-for 45.0 2024-09-11T23:22:57.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:22:57 smithi136 ceph-mon[35121]: pgmap v140: 97 pgs: 97 active+clean; 1.3 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 8.8 MiB/s wr, 604 op/s 2024-09-11T23:22:57.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:22:57 smithi136 ceph-mon[35121]: from='client.? 172.21.15.26:0/2293008465' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:22:58.000 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:22:57 smithi026 ceph-mon[31172]: pgmap v140: 97 pgs: 97 active+clean; 1.3 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 8.8 MiB/s wr, 604 op/s 2024-09-11T23:22:58.000 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:22:57 smithi026 ceph-mon[31172]: from='client.? 172.21.15.26:0/2293008465' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:22:58.040 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:22:58.052 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:22:57 smithi076 ceph-mon[35296]: pgmap v140: 97 pgs: 97 active+clean; 1.3 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 8.8 MiB/s wr, 604 op/s 2024-09-11T23:22:58.052 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:22:57 smithi076 ceph-mon[35296]: from='client.? 172.21.15.26:0/2293008465' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:22:58.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.559+0000 7f60d87ba640 1 -- 172.21.15.26:0/2203131165 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f60d010e3c0 msgr2=0x7f60d0110810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:58.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.559+0000 7f60d87ba640 1 --2- 172.21.15.26:0/2203131165 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f60d010e3c0 0x7f60d0110810 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f60c8009be0 tx=0x7f60c802a750 comp rx=0 tx=0).stop 2024-09-11T23:22:58.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.559+0000 7f60d87ba640 1 -- 172.21.15.26:0/2203131165 shutdown_connections 2024-09-11T23:22:58.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.559+0000 7f60d87ba640 1 --2- 172.21.15.26:0/2203131165 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f60d010e3c0 0x7f60d0110810 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:58.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.559+0000 7f60d87ba640 1 --2- 172.21.15.26:0/2203131165 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f60d0074f40 0x7f60d010de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:58.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.559+0000 7f60d87ba640 1 --2- 172.21.15.26:0/2203131165 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f60d0074620 0x7f60d0074a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:58.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.559+0000 7f60d87ba640 1 -- 172.21.15.26:0/2203131165 >> 172.21.15.26:0/2203131165 conn(0x7f60d00fd8e0 msgr2=0x7f60d00ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:22:58.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.559+0000 7f60d87ba640 1 -- 172.21.15.26:0/2203131165 shutdown_connections 2024-09-11T23:22:58.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.559+0000 7f60d87ba640 1 -- 172.21.15.26:0/2203131165 wait complete. 2024-09-11T23:22:58.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.560+0000 7f60d87ba640 1 Processor -- start 2024-09-11T23:22:58.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.560+0000 7f60d87ba640 1 -- start start 2024-09-11T23:22:58.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.560+0000 7f60d87ba640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f60d0074620 0x7f60d019a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:58.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.560+0000 7f60d87ba640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f60d0074f40 0x7f60d019ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:58.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.560+0000 7f60d87ba640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f60d010e3c0 0x7f60d019eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:58.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.560+0000 7f60d87ba640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f60d01140f0 con 0x7f60d010e3c0 2024-09-11T23:22:58.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.560+0000 7f60d87ba640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f60d0113f70 con 0x7f60d0074f40 2024-09-11T23:22:58.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.560+0000 7f60d87ba640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f60d0114270 con 0x7f60d0074620 2024-09-11T23:22:58.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.561+0000 7f60d6556640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f60d0074620 0x7f60d019a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:58.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.561+0000 7f60d6d57640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f60d010e3c0 0x7f60d019eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:58.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.561+0000 7f60d5d55640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f60d0074f40 0x7f60d019ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:58.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.561+0000 7f60d6556640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f60d0074620 0x7f60d019a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:56912/0 (socket says 172.21.15.26:56912) 2024-09-11T23:22:58.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.561+0000 7f60d6556640 1 -- 172.21.15.26:0/2553936592 learned_addr learned my addr 172.21.15.26:0/2553936592 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:22:58.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.561+0000 7f60d6556640 1 -- 172.21.15.26:0/2553936592 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f60d0074f40 msgr2=0x7f60d019ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:58.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.561+0000 7f60d6556640 1 --2- 172.21.15.26:0/2553936592 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f60d0074f40 0x7f60d019ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:58.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.561+0000 7f60d6556640 1 -- 172.21.15.26:0/2553936592 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f60d010e3c0 msgr2=0x7f60d019eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:58.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.561+0000 7f60d6556640 1 --2- 172.21.15.26:0/2553936592 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f60d010e3c0 0x7f60d019eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:58.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.561+0000 7f60d6556640 1 -- 172.21.15.26:0/2553936592 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60d019f580 con 0x7f60d0074620 2024-09-11T23:22:58.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.561+0000 7f60d6556640 1 --2- 172.21.15.26:0/2553936592 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f60d0074620 0x7f60d019a5d0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f60cc00ea60 tx=0x7f60cc00ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:22:58.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.561+0000 7f60c77fe640 1 -- 172.21.15.26:0/2553936592 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60cc00cde0 con 0x7f60d0074620 2024-09-11T23:22:58.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.561+0000 7f60c77fe640 1 -- 172.21.15.26:0/2553936592 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f60cc0035f0 con 0x7f60d0074620 2024-09-11T23:22:58.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.561+0000 7f60d87ba640 1 -- 172.21.15.26:0/2553936592 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60d019f870 con 0x7f60d0074620 2024-09-11T23:22:58.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.561+0000 7f60c77fe640 1 -- 172.21.15.26:0/2553936592 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60cc010670 con 0x7f60d0074620 2024-09-11T23:22:58.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.561+0000 7f60d87ba640 1 -- 172.21.15.26:0/2553936592 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60d01a7150 con 0x7f60d0074620 2024-09-11T23:22:58.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.564+0000 7f60c77fe640 1 -- 172.21.15.26:0/2553936592 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f60cc0037a0 con 0x7f60d0074620 2024-09-11T23:22:58.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.564+0000 7f60dab93740 1 Processor -- start 2024-09-11T23:22:58.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.565+0000 7f60dab93740 1 -- start start 2024-09-11T23:22:58.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.565+0000 7f60dab93740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5649dfe08770 0x5649dfe08b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:58.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.565+0000 7f60dab93740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5649dfe09090 0x5649dfe1bd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:58.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.565+0000 7f60dab93740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5649dfe1c240 0x5649dfe1e630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:58.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.565+0000 7f60dab93740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5649dfd6d160 con 0x5649dfe08770 2024-09-11T23:22:58.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.565+0000 7f60dab93740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5649dfd14720 con 0x5649dfe1c240 2024-09-11T23:22:58.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.565+0000 7f60dab93740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5649dfd69120 con 0x5649dfe09090 2024-09-11T23:22:58.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.565+0000 7f60c77fe640 1 --2- 172.21.15.26:0/2553936592 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f60bc0a2d70 0x7f60bc0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:58.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.565+0000 7f60d6d57640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5649dfe08770 0x5649dfe08b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:58.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.565+0000 7f60d6556640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5649dfe09090 0x5649dfe1bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:58.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.565+0000 7f60d6d57640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5649dfe08770 0x5649dfe08b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:32818/0 (socket says 172.21.15.26:32818) 2024-09-11T23:22:58.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.565+0000 7f60d6d57640 1 -- 172.21.15.26:0/3012646578 learned_addr learned my addr 172.21.15.26:0/3012646578 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:22:58.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.565+0000 7f60d5d55640 1 --2- 172.21.15.26:0/3012646578 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5649dfe1c240 0x5649dfe1e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:58.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.565+0000 7f60d6d57640 1 --2- 172.21.15.26:0/2553936592 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f60bc0a2d70 0x7f60bc0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:58.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.566+0000 7f60c77fe640 1 -- 172.21.15.26:0/2553936592 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f60cc0cf6b0 con 0x7f60d0074620 2024-09-11T23:22:58.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.566+0000 7f60d6556640 1 -- 172.21.15.26:0/3012646578 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5649dfe1c240 msgr2=0x5649dfe1e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:22:58.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.566+0000 7f60d6556640 1 --2- 172.21.15.26:0/3012646578 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5649dfe1c240 0x5649dfe1e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:58.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.566+0000 7f60d6556640 1 -- 172.21.15.26:0/3012646578 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5649dfe08770 msgr2=0x5649dfe08b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:22:58.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.566+0000 7f60d6556640 1 --2- 172.21.15.26:0/3012646578 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5649dfe08770 0x5649dfe08b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:58.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.566+0000 7f60d6556640 1 -- 172.21.15.26:0/3012646578 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5649dfadeba0 con 0x5649dfe09090 2024-09-11T23:22:58.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.566+0000 7f60d6556640 1 --2- 172.21.15.26:0/3012646578 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5649dfe09090 0x5649dfe1bd00 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f60cc0cce10 tx=0x7f60cc09b540 comp rx=0 tx=0).ready entity=mon.1 client_cookie=84b413d402358af2 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:22:58.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.566+0000 7f60d6d57640 1 --2- 172.21.15.26:0/2553936592 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f60bc0a2d70 0x7f60bc0a5230 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f60c8033000 tx=0x7f60c8009500 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:22:58.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.566+0000 7f60c4ff9640 1 -- 172.21.15.26:0/3012646578 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60cc0b2020 con 0x5649dfe09090 2024-09-11T23:22:58.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.566+0000 7f60c4ff9640 1 -- 172.21.15.26:0/3012646578 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f60cc09c2e0 con 0x5649dfe09090 2024-09-11T23:22:58.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.566+0000 7f60c4ff9640 1 -- 172.21.15.26:0/3012646578 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60cc0a3310 con 0x5649dfe09090 2024-09-11T23:22:58.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.567+0000 7f60dab93740 1 -- 172.21.15.26:0/3012646578 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5649dfe09090 msgr2=0x5649dfe1bd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:58.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.567+0000 7f60dab93740 1 --2- 172.21.15.26:0/3012646578 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5649dfe09090 0x5649dfe1bd00 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f60cc0cce10 tx=0x7f60cc09b540 comp rx=0 tx=0).stop 2024-09-11T23:22:58.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.567+0000 7f60dab93740 1 -- 172.21.15.26:0/3012646578 shutdown_connections 2024-09-11T23:22:58.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.567+0000 7f60dab93740 1 --2- 172.21.15.26:0/3012646578 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5649dfe1c240 0x5649dfe1e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:58.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.567+0000 7f60dab93740 1 --2- 172.21.15.26:0/3012646578 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5649dfe09090 0x5649dfe1bd00 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:58.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.567+0000 7f60dab93740 1 --2- 172.21.15.26:0/3012646578 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5649dfe08770 0x5649dfe08b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:58.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.567+0000 7f60dab93740 1 -- 172.21.15.26:0/3012646578 >> 172.21.15.26:0/3012646578 conn(0x5649dfe04de0 msgr2=0x5649dfe051c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:22:58.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.568+0000 7f60dab93740 1 -- 172.21.15.26:0/3012646578 shutdown_connections 2024-09-11T23:22:58.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.568+0000 7f60dab93740 1 -- 172.21.15.26:0/3012646578 wait complete. 2024-09-11T23:22:58.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.568+0000 7f60dab93740 1 -- start start 2024-09-11T23:22:58.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.568+0000 7f60dab93740 1 Processor -- start 2024-09-11T23:22:58.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.569+0000 7f60dab93740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5649dfe08770 0x5649dfe19250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:58.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.569+0000 7f60dab93740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5649dfe09090 0x5649dfe19790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:58.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.569+0000 7f60dab93740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5649dfe1c240 0x5649dfe19cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:58.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.569+0000 7f60dab93740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5649dfe1ed50 con 0x5649dfe09090 2024-09-11T23:22:58.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.569+0000 7f60dab93740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5649dfcf5900 con 0x5649dfe08770 2024-09-11T23:22:58.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.569+0000 7f60dab93740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5649dfe1eed0 con 0x5649dfe1c240 2024-09-11T23:22:58.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.569+0000 7f60dab93740 10 client.0 _ll_register_callbacks cb 0x5649dfcf3780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:22:58.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.569+0000 7f60d6d57640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5649dfe1c240 0x5649dfe19cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:58.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.569+0000 7f60d6d57640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5649dfe1c240 0x5649dfe19cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:56926/0 (socket says 172.21.15.26:56926) 2024-09-11T23:22:58.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.569+0000 7f60d6d57640 1 -- 172.21.15.26:0/1277217422 learned_addr learned my addr 172.21.15.26:0/1277217422 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:22:58.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.569+0000 7f60d5d55640 1 --2- 172.21.15.26:0/1277217422 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5649dfe09090 0x5649dfe19790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:58.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.569+0000 7f60d6556640 1 --2- 172.21.15.26:0/1277217422 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5649dfe08770 0x5649dfe19250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:58.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.570+0000 7f60d6d57640 1 -- 172.21.15.26:0/1277217422 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5649dfe08770 msgr2=0x5649dfe19250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:22:58.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.570+0000 7f60d6d57640 1 --2- 172.21.15.26:0/1277217422 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5649dfe08770 0x5649dfe19250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:58.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.570+0000 7f60d6d57640 1 -- 172.21.15.26:0/1277217422 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5649dfe09090 msgr2=0x5649dfe19790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:22:58.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.570+0000 7f60d6d57640 1 --2- 172.21.15.26:0/1277217422 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5649dfe09090 0x5649dfe19790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:58.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.570+0000 7f60d6d57640 1 -- 172.21.15.26:0/1277217422 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5649dfd35c50 con 0x5649dfe1c240 2024-09-11T23:22:58.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.570+0000 7f60d6d57640 1 --2- 172.21.15.26:0/1277217422 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5649dfe1c240 0x5649dfe19cd0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f60c8003c20 tx=0x7f60c802a750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=20762f47dc5be5c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:22:58.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.570+0000 7f60b9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-09-11T23:22:58.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.570+0000 7f60b9ffb640 1 -- 172.21.15.26:0/1277217422 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60c8036040 con 0x5649dfe1c240 2024-09-11T23:22:58.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.570+0000 7f60b9ffb640 1 -- 172.21.15.26:0/1277217422 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f60c8004240 con 0x5649dfe1c240 2024-09-11T23:22:58.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.570+0000 7f60dab93740 1 -- 172.21.15.26:0/1277217422 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f60cc09c2e0 con 0x5649dfe1c240 2024-09-11T23:22:58.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.571+0000 7f60b9ffb640 1 -- 172.21.15.26:0/1277217422 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f60c8004440 con 0x5649dfe1c240 2024-09-11T23:22:58.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.571+0000 7f60dab93740 10 client.26002 fetch_fsmap learned FSMap version 38 2024-09-11T23:22:58.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.571+0000 7f60dab93740 1 -- 172.21.15.26:0/1277217422 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5649dfe2ab60 con 0x5649dfe1c240 2024-09-11T23:22:58.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.571+0000 7f60b9ffb640 1 -- 172.21.15.26:0/1277217422 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f60cc09c2e0 con 0x5649dfe1c240 2024-09-11T23:22:58.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.571+0000 7f60b9ffb640 1 -- 172.21.15.26:0/1277217422 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 38) ==== 8414+0+0 (secure 0 0 0) 0x7f60c8031b80 con 0x5649dfe1c240 2024-09-11T23:22:58.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.571+0000 7f60dab93740 10 client.26002 fetch_fsmap finished waiting for FSMap version 38 2024-09-11T23:22:58.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.571+0000 7f60dab93740 10 client.26002 resolve_mds: validated gid 24485 aka daemon mds.b 2024-09-11T23:22:58.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.571+0000 7f60dab93740 20 client.26002 populate_metadata read hostname 'smithi026' 2024-09-11T23:22:58.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.571+0000 7f60dab93740 1 --2- 172.21.15.26:0/1277217422 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5649dfe2c0c0 0x5649dfe2e520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:58.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.571+0000 7f60dab93740 4 client.26002 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:22:58.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.571+0000 7f60dab93740 1 -- 172.21.15.26:0/1277217422 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5649dfd7a890 con 0x5649dfe2c0c0 2024-09-11T23:22:58.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.571+0000 7f60d6556640 1 --2- 172.21.15.26:0/1277217422 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5649dfe2c0c0 0x5649dfe2e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:58.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.572+0000 7f60d6556640 1 --2- 172.21.15.26:0/1277217422 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5649dfe2c0c0 0x5649dfe2e520 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d4b97302639f269 server_cookie=a36f15a41631e23d in_seq=0 out_seq=0 2024-09-11T23:22:58.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.572+0000 7f60b9ffb640 10 client.26002 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:22:58.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.573+0000 7f60b9ffb640 1 -- 172.21.15.26:0/1277217422 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5649dfd7a890 con 0x5649dfe2c0c0 2024-09-11T23:22:58.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.573+0000 7f60b9ffb640 10 client.26002 handle_command_reply: tid=0 2024-09-11T23:22:58.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.573+0000 7f60dab93740 1 client.26002 shutdown 2024-09-11T23:22:58.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.574+0000 7f60dab93740 1 -- 172.21.15.26:0/1277217422 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5649dfe1c240 msgr2=0x5649dfe19cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:58.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.574+0000 7f60dab93740 1 --2- 172.21.15.26:0/1277217422 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5649dfe1c240 0x5649dfe19cd0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f60c8003c20 tx=0x7f60c802a750 comp rx=0 tx=0).stop 2024-09-11T23:22:58.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.574+0000 7f60dab93740 1 -- 172.21.15.26:0/1277217422 shutdown_connections 2024-09-11T23:22:58.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.574+0000 7f60dab93740 1 --2- 172.21.15.26:0/1277217422 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5649dfe2c0c0 0x5649dfe2e520 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:58.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.574+0000 7f60dab93740 1 --2- 172.21.15.26:0/1277217422 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5649dfe1c240 0x5649dfe19cd0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:58.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.574+0000 7f60dab93740 1 --2- 172.21.15.26:0/1277217422 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5649dfe09090 0x5649dfe19790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:58.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.574+0000 7f60b9ffb640 0 client.26002 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:22:58.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.574+0000 7f60dab93740 1 --2- 172.21.15.26:0/1277217422 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5649dfe08770 0x5649dfe19250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:58.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.574+0000 7f60dab93740 1 -- 172.21.15.26:0/1277217422 >> 172.21.15.26:0/1277217422 conn(0x5649dfe04de0 msgr2=0x5649dfe0aa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:22:58.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.574+0000 7f60dab93740 1 -- 172.21.15.26:0/1277217422 shutdown_connections 2024-09-11T23:22:58.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.574+0000 7f60dab93740 1 -- 172.21.15.26:0/1277217422 wait complete. 2024-09-11T23:22:58.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.574+0000 7f60dab93740 20 client.26002 trim_cache size 0 max 16384 2024-09-11T23:22:58.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.595+0000 7f60dab93740 1 Processor -- start 2024-09-11T23:22:58.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.595+0000 7f60dab93740 1 -- start start 2024-09-11T23:22:58.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.595+0000 7f60dab93740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5649dfe08770 0x5649dfe19920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:58.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.595+0000 7f60dab93740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5649dfe09090 0x5649dfe33df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:58.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.595+0000 7f60dab93740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5649dfe1c240 0x5649dfe34330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:58.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.595+0000 7f60dab93740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f60d01ab750 con 0x5649dfe08770 2024-09-11T23:22:58.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.595+0000 7f60d6556640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5649dfe08770 0x5649dfe19920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:58.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.595+0000 7f60dab93740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f60d01ad500 con 0x5649dfe09090 2024-09-11T23:22:58.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.595+0000 7f60dab93740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f60d01abee0 con 0x5649dfe1c240 2024-09-11T23:22:58.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.596+0000 7f60d6556640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5649dfe08770 0x5649dfe19920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:32844/0 (socket says 172.21.15.26:32844) 2024-09-11T23:22:58.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.596+0000 7f60d6556640 1 -- 172.21.15.26:0/3952552178 learned_addr learned my addr 172.21.15.26:0/3952552178 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:22:58.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.596+0000 7f60d5d55640 1 --2- 172.21.15.26:0/3952552178 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5649dfe09090 0x5649dfe33df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:58.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.596+0000 7f60d6d57640 1 --2- 172.21.15.26:0/3952552178 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5649dfe1c240 0x5649dfe34330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:58.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.596+0000 7f60d6556640 1 -- 172.21.15.26:0/3952552178 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5649dfe1c240 msgr2=0x5649dfe34330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:22:58.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.596+0000 7f60d6556640 1 --2- 172.21.15.26:0/3952552178 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5649dfe1c240 0x5649dfe34330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:58.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.596+0000 7f60d6556640 1 -- 172.21.15.26:0/3952552178 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5649dfe09090 msgr2=0x5649dfe33df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:22:58.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.596+0000 7f60d6556640 1 --2- 172.21.15.26:0/3952552178 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5649dfe09090 0x5649dfe33df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:58.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.596+0000 7f60d6556640 1 -- 172.21.15.26:0/3952552178 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5649dfe003f0 con 0x5649dfe08770 2024-09-11T23:22:58.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.596+0000 7f60d5d55640 1 --2- 172.21.15.26:0/3952552178 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5649dfe09090 0x5649dfe33df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:22:58.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.596+0000 7f60d6d57640 1 --2- 172.21.15.26:0/3952552178 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5649dfe1c240 0x5649dfe34330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:22:58.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.596+0000 7f60d6556640 1 --2- 172.21.15.26:0/3952552178 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5649dfe08770 0x5649dfe19920 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f60cc09b8e0 tx=0x7f60cc09a320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d9d18b0f30d1cc9 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:22:58.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.597+0000 7f60bbfff640 1 -- 172.21.15.26:0/3952552178 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60cc0b2040 con 0x5649dfe08770 2024-09-11T23:22:58.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.597+0000 7f60bbfff640 1 -- 172.21.15.26:0/3952552178 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f60cc09c2e0 con 0x5649dfe08770 2024-09-11T23:22:58.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.597+0000 7f60bbfff640 1 -- 172.21.15.26:0/3952552178 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60cc09bbe0 con 0x5649dfe08770 2024-09-11T23:22:58.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.597+0000 7f60dab93740 1 -- 172.21.15.26:0/3952552178 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5649dfe08770 msgr2=0x5649dfe19920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:58.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.597+0000 7f60dab93740 1 --2- 172.21.15.26:0/3952552178 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5649dfe08770 0x5649dfe19920 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f60cc09b8e0 tx=0x7f60cc09a320 comp rx=0 tx=0).stop 2024-09-11T23:22:58.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.597+0000 7f60dab93740 1 -- 172.21.15.26:0/3952552178 shutdown_connections 2024-09-11T23:22:58.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.597+0000 7f60dab93740 1 --2- 172.21.15.26:0/3952552178 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5649dfe09090 0x5649dfe33df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:58.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.597+0000 7f60dab93740 1 --2- 172.21.15.26:0/3952552178 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5649dfe1c240 0x5649dfe34330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:58.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.597+0000 7f60dab93740 1 --2- 172.21.15.26:0/3952552178 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5649dfe08770 0x5649dfe19920 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:58.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.597+0000 7f60dab93740 1 -- 172.21.15.26:0/3952552178 >> 172.21.15.26:0/3952552178 conn(0x5649dfe04de0 msgr2=0x5649dfe19210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:22:58.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.598+0000 7f60dab93740 1 -- 172.21.15.26:0/3952552178 shutdown_connections 2024-09-11T23:22:58.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.598+0000 7f60dab93740 1 -- 172.21.15.26:0/3952552178 wait complete. 2024-09-11T23:22:58.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.598+0000 7f60dab93740 1 -- start start 2024-09-11T23:22:58.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.598+0000 7f60dab93740 1 Processor -- start 2024-09-11T23:22:58.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.598+0000 7f60dab93740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5649dfe08770 0x5649dfe24770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:58.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.598+0000 7f60dab93740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5649dfe09090 0x5649dfe21130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:58.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.598+0000 7f60dab93740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5649dfe1c240 0x5649dfe21670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:58.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.598+0000 7f60dab93740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f60d01ace00 con 0x5649dfe09090 2024-09-11T23:22:58.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.598+0000 7f60dab93740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f60d01ac670 con 0x5649dfe1c240 2024-09-11T23:22:58.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.598+0000 7f60dab93740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f60d01a9730 con 0x5649dfe08770 2024-09-11T23:22:58.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.599+0000 7f60dab93740 10 client.0 _ll_register_callbacks cb 0x7f60cc005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:22:58.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.599+0000 7f60d5d55640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5649dfe09090 0x5649dfe21130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:58.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.599+0000 7f60d5d55640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5649dfe09090 0x5649dfe21130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:32852/0 (socket says 172.21.15.26:32852) 2024-09-11T23:22:58.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.599+0000 7f60d6556640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5649dfe08770 0x5649dfe24770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:58.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.599+0000 7f60d5d55640 1 -- 172.21.15.26:0/1371623105 learned_addr learned my addr 172.21.15.26:0/1371623105 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:22:58.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.599+0000 7f60d6d57640 1 --2- 172.21.15.26:0/1371623105 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5649dfe1c240 0x5649dfe21670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:58.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.599+0000 7f60d6556640 1 -- 172.21.15.26:0/1371623105 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5649dfe1c240 msgr2=0x5649dfe21670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:22:58.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.599+0000 7f60d6556640 1 --2- 172.21.15.26:0/1371623105 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5649dfe1c240 0x5649dfe21670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:58.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.599+0000 7f60d6556640 1 -- 172.21.15.26:0/1371623105 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5649dfe09090 msgr2=0x5649dfe21130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:22:58.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.599+0000 7f60d6556640 1 --2- 172.21.15.26:0/1371623105 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5649dfe09090 0x5649dfe21130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:58.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.599+0000 7f60d6556640 1 -- 172.21.15.26:0/1371623105 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5649dfe05a70 con 0x5649dfe08770 2024-09-11T23:22:58.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.599+0000 7f60d6556640 1 --2- 172.21.15.26:0/1371623105 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5649dfe08770 0x5649dfe24770 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x5649dfdf9680 tx=0x7f60cc09b540 comp rx=0 tx=0).ready entity=mon.1 client_cookie=66751a9b0a78b325 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:22:58.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.599+0000 7f60b9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-09-11T23:22:58.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.599+0000 7f60b9ffb640 1 -- 172.21.15.26:0/1371623105 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60cc0b2070 con 0x5649dfe08770 2024-09-11T23:22:58.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.599+0000 7f60b9ffb640 1 -- 172.21.15.26:0/1371623105 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f60cc099ee0 con 0x5649dfe08770 2024-09-11T23:22:58.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.599+0000 7f60dab93740 1 -- 172.21.15.26:0/1371623105 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f60cc09c2e0 con 0x5649dfe08770 2024-09-11T23:22:58.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.600+0000 7f60b9ffb640 1 -- 172.21.15.26:0/1371623105 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f60cc0a3740 con 0x5649dfe08770 2024-09-11T23:22:58.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.600+0000 7f60dab93740 10 client.26008 fetch_fsmap learned FSMap version 38 2024-09-11T23:22:58.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.600+0000 7f60dab93740 1 -- 172.21.15.26:0/1371623105 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5649dfe97aa0 con 0x5649dfe08770 2024-09-11T23:22:58.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.600+0000 7f60b9ffb640 1 -- 172.21.15.26:0/1371623105 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f60cc09c2e0 con 0x5649dfe08770 2024-09-11T23:22:58.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.600+0000 7f60b9ffb640 1 -- 172.21.15.26:0/1371623105 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 38) ==== 8414+0+0 (secure 0 0 0) 0x7f60cc098250 con 0x5649dfe08770 2024-09-11T23:22:58.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.600+0000 7f60dab93740 10 client.26008 fetch_fsmap finished waiting for FSMap version 38 2024-09-11T23:22:58.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.600+0000 7f60dab93740 10 client.26008 resolve_mds: validated gid 24485 aka daemon mds.b 2024-09-11T23:22:58.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.600+0000 7f60dab93740 20 client.26008 populate_metadata read hostname 'smithi026' 2024-09-11T23:22:58.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.600+0000 7f60dab93740 1 --2- 172.21.15.26:0/1371623105 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5649dfe97e00 0x5649dfe9a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:58.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.600+0000 7f60dab93740 4 client.26008 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.15, "await_for": 45.0}] 2024-09-11T23:22:58.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.600+0000 7f60dab93740 1 -- 172.21.15.26:0/1371623105 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.15, "await_for": 45.0}) -- 0x5649dfd74be0 con 0x5649dfe97e00 2024-09-11T23:22:58.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.600+0000 7f60d5d55640 1 --2- 172.21.15.26:0/1371623105 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5649dfe97e00 0x5649dfe9a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:58.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.601+0000 7f60b9ffb640 10 client.26008 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:22:58.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:22:58.601+0000 7f60d5d55640 1 --2- 172.21.15.26:0/1371623105 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5649dfe97e00 0x5649dfe9a1f0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b17ef7fac79be2d0 server_cookie=c31e95a08d6bba87 in_seq=0 out_seq=0 2024-09-11T23:22:59.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:22:59 smithi136 ceph-mon[35121]: pgmap v141: 97 pgs: 97 active+clean; 1.3 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 7.7 MiB/s wr, 620 op/s 2024-09-11T23:23:00.000 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:22:59 smithi026 ceph-mon[31172]: pgmap v141: 97 pgs: 97 active+clean; 1.3 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 7.7 MiB/s wr, 620 op/s 2024-09-11T23:23:00.052 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:22:59 smithi076 ceph-mon[35296]: pgmap v141: 97 pgs: 97 active+clean; 1.3 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 7.7 MiB/s wr, 620 op/s 2024-09-11T23:23:01.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:23:01 smithi136 ceph-mon[35121]: pgmap v142: 97 pgs: 97 active+clean; 1.3 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 9.5 MiB/s wr, 860 op/s 2024-09-11T23:23:02.000 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:23:01 smithi026 ceph-mon[31172]: pgmap v142: 97 pgs: 97 active+clean; 1.3 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 9.5 MiB/s wr, 860 op/s 2024-09-11T23:23:02.052 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:23:01 smithi076 ceph-mon[35296]: pgmap v142: 97 pgs: 97 active+clean; 1.3 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 9.5 MiB/s wr, 860 op/s 2024-09-11T23:23:02.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:23:02 smithi136 ceph-mon[35121]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:23:03.000 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:23:02 smithi026 ceph-mon[31172]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:23:03.052 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:23:02 smithi076 ceph-mon[35296]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:23:03.578 INFO:tasks.workunit.client.0.smithi026.stdout: 365 52239 182 35388 136 10044 46 2024-09-11T23:23:03.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:23:03 smithi136 ceph-mon[35121]: pgmap v143: 97 pgs: 97 active+clean; 1.3 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 4.7 MiB/s wr, 694 op/s 2024-09-11T23:23:04.000 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:23:03 smithi026 ceph-mon[31172]: pgmap v143: 97 pgs: 97 active+clean; 1.3 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 4.7 MiB/s wr, 694 op/s 2024-09-11T23:23:04.052 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:23:03 smithi076 ceph-mon[35296]: pgmap v143: 97 pgs: 97 active+clean; 1.3 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 4.7 MiB/s wr, 694 op/s 2024-09-11T23:23:05.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:23:05 smithi136 ceph-mon[35121]: pgmap v144: 97 pgs: 97 active+clean; 1.4 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 8.9 MiB/s wr, 884 op/s 2024-09-11T23:23:06.000 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:23:05 smithi026 ceph-mon[31172]: pgmap v144: 97 pgs: 97 active+clean; 1.4 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 8.9 MiB/s wr, 884 op/s 2024-09-11T23:23:06.052 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:23:05 smithi076 ceph-mon[35296]: pgmap v144: 97 pgs: 97 active+clean; 1.4 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 8.9 MiB/s wr, 884 op/s 2024-09-11T23:23:06.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:06.932+0000 7f60b9ffb640 1 -- 172.21.15.26:0/1371623105 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5649dfd74be0 con 0x5649dfe97e00 2024-09-11T23:23:06.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:06.932+0000 7f60b9ffb640 10 client.26008 handle_command_reply: tid=0 2024-09-11T23:23:06.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:06.932+0000 7f60dab93740 1 client.26008 shutdown 2024-09-11T23:23:06.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:06.933+0000 7f60dab93740 1 -- 172.21.15.26:0/1371623105 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5649dfe08770 msgr2=0x5649dfe24770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:23:06.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:06.933+0000 7f60dab93740 1 --2- 172.21.15.26:0/1371623105 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5649dfe08770 0x5649dfe24770 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x5649dfdf9680 tx=0x7f60cc09b540 comp rx=0 tx=0).stop 2024-09-11T23:23:06.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:06.933+0000 7f60dab93740 1 -- 172.21.15.26:0/1371623105 shutdown_connections 2024-09-11T23:23:06.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:06.933+0000 7f60dab93740 1 --2- 172.21.15.26:0/1371623105 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5649dfe1c240 0x5649dfe21670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:06.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:06.933+0000 7f60dab93740 1 --2- 172.21.15.26:0/1371623105 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5649dfe97e00 0x5649dfe9a1f0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:06.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:06.933+0000 7f60dab93740 1 --2- 172.21.15.26:0/1371623105 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5649dfe09090 0x5649dfe21130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:06.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:06.933+0000 7f60dab93740 1 --2- 172.21.15.26:0/1371623105 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5649dfe08770 0x5649dfe24770 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:06.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:06.933+0000 7f60b9ffb640 0 client.26008 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:23:06.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:06.933+0000 7f60dab93740 1 -- 172.21.15.26:0/1371623105 >> 172.21.15.26:0/1371623105 conn(0x5649dfe04de0 msgr2=0x5649dfe03ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:23:06.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:06.933+0000 7f60dab93740 1 -- 172.21.15.26:0/1371623105 shutdown_connections 2024-09-11T23:23:06.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:06.933+0000 7f60dab93740 1 -- 172.21.15.26:0/1371623105 wait complete. 2024-09-11T23:23:06.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:06.933+0000 7f60dab93740 20 client.26008 trim_cache size 0 max 16384 2024-09-11T23:23:06.935 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-11T23:23:06.935 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 39, 2024-09-11T23:23:06.935 INFO:teuthology.orchestra.run.smithi026.stdout: "leader": 24485, 2024-09-11T23:23:06.935 INFO:teuthology.orchestra.run.smithi026.stdout: "set_version": 18, 2024-09-11T23:23:06.935 INFO:teuthology.orchestra.run.smithi026.stdout: "sets": { 2024-09-11T23:23:06.935 INFO:teuthology.orchestra.run.smithi026.stdout: "e975f3df": { 2024-09-11T23:23:06.935 INFO:teuthology.orchestra.run.smithi026.stdout: "version": 18, 2024-09-11T23:23:06.936 INFO:teuthology.orchestra.run.smithi026.stdout: "age_ref": 0.0, 2024-09-11T23:23:06.936 INFO:teuthology.orchestra.run.smithi026.stdout: "state": { 2024-09-11T23:23:06.936 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "QUIESCED", 2024-09-11T23:23:06.936 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.0 2024-09-11T23:23:06.936 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:23:06.936 INFO:teuthology.orchestra.run.smithi026.stdout: "timeout": 90.0, 2024-09-11T23:23:06.936 INFO:teuthology.orchestra.run.smithi026.stdout: "expiration": 140.2, 2024-09-11T23:23:06.936 INFO:teuthology.orchestra.run.smithi026.stdout: "members": { 2024-09-11T23:23:06.936 INFO:teuthology.orchestra.run.smithi026.stdout: "file:/": { 2024-09-11T23:23:06.936 INFO:teuthology.orchestra.run.smithi026.stdout: "excluded": false, 2024-09-11T23:23:06.936 INFO:teuthology.orchestra.run.smithi026.stdout: "state": { 2024-09-11T23:23:06.936 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "QUIESCED", 2024-09-11T23:23:06.936 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.0 2024-09-11T23:23:06.937 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:23:06.937 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:23:06.937 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:23:06.937 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:23:06.937 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:23:06.937 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-11T23:23:06.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:06.934+0000 7f60d87ba640 1 -- 172.21.15.26:0/2553936592 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f60bc0a2d70 msgr2=0x7f60bc0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:23:06.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:06.934+0000 7f60d87ba640 1 --2- 172.21.15.26:0/2553936592 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f60bc0a2d70 0x7f60bc0a5230 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f60c8033000 tx=0x7f60c8009500 comp rx=0 tx=0).stop 2024-09-11T23:23:06.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:06.934+0000 7f60d87ba640 1 -- 172.21.15.26:0/2553936592 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f60d0074620 msgr2=0x7f60d019a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:23:06.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:06.934+0000 7f60d87ba640 1 --2- 172.21.15.26:0/2553936592 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f60d0074620 0x7f60d019a5d0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f60cc00ea60 tx=0x7f60cc00ef20 comp rx=0 tx=0).stop 2024-09-11T23:23:06.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:06.934+0000 7f60d87ba640 1 -- 172.21.15.26:0/2553936592 shutdown_connections 2024-09-11T23:23:06.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:06.934+0000 7f60d87ba640 1 --2- 172.21.15.26:0/2553936592 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f60bc0a2d70 0x7f60bc0a5230 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:06.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:06.934+0000 7f60d87ba640 1 --2- 172.21.15.26:0/2553936592 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f60d0074f40 0x7f60d019ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:06.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:06.934+0000 7f60d87ba640 1 --2- 172.21.15.26:0/2553936592 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f60d010e3c0 0x7f60d019eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:06.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:06.934+0000 7f60d87ba640 1 --2- 172.21.15.26:0/2553936592 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f60d0074620 0x7f60d019a5d0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:06.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:06.934+0000 7f60d87ba640 1 -- 172.21.15.26:0/2553936592 >> 172.21.15.26:0/2553936592 conn(0x7f60d00fd8e0 msgr2=0x7f60d00fefc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:23:06.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:06.934+0000 7f60d87ba640 1 -- 172.21.15.26:0/2553936592 shutdown_connections 2024-09-11T23:23:06.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:06.934+0000 7f60d87ba640 1 -- 172.21.15.26:0/2553936592 wait complete. 2024-09-11T23:23:07.874 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'e975f3df', quiesce took 12.2 seconds. Will release after: 7.949999999999999 2024-09-11T23:23:07.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:23:07 smithi136 ceph-mon[35121]: pgmap v145: 97 pgs: 97 active+clean; 1.4 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 8.5 MiB/s wr, 691 op/s 2024-09-11T23:23:08.000 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:23:07 smithi026 ceph-mon[31172]: pgmap v145: 97 pgs: 97 active+clean; 1.4 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 8.5 MiB/s wr, 691 op/s 2024-09-11T23:23:08.052 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:23:07 smithi076 ceph-mon[35296]: pgmap v145: 97 pgs: 97 active+clean; 1.4 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 8.5 MiB/s wr, 691 op/s 2024-09-11T23:23:08.313 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.313+0000 7f05dbed0640 1 -- 172.21.15.26:0/3294957304 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05d4101350 msgr2=0x7f05d410d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:23:08.313 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.313+0000 7f05dbed0640 1 --2- 172.21.15.26:0/3294957304 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05d4101350 0x7f05d410d880 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f05cc009930 tx=0x7f05cc02a420 comp rx=0 tx=0).stop 2024-09-11T23:23:08.313 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.313+0000 7f05dbed0640 1 -- 172.21.15.26:0/3294957304 shutdown_connections 2024-09-11T23:23:08.313 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.313+0000 7f05dbed0640 1 --2- 172.21.15.26:0/3294957304 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f05d41009b0 0x7f05d4100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:08.314 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.313+0000 7f05dbed0640 1 --2- 172.21.15.26:0/3294957304 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05d4101350 0x7f05d410d880 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:08.314 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.313+0000 7f05dbed0640 1 --2- 172.21.15.26:0/3294957304 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f05d41069c0 0x7f05d4106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:08.314 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.313+0000 7f05dbed0640 1 -- 172.21.15.26:0/3294957304 >> 172.21.15.26:0/3294957304 conn(0x7f05d4077990 msgr2=0x7f05d40fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:23:08.314 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.313+0000 7f05dbed0640 1 -- 172.21.15.26:0/3294957304 shutdown_connections 2024-09-11T23:23:08.314 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.313+0000 7f05dbed0640 1 -- 172.21.15.26:0/3294957304 wait complete. 2024-09-11T23:23:08.314 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.313+0000 7f05dbed0640 1 Processor -- start 2024-09-11T23:23:08.314 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.313+0000 7f05dbed0640 1 -- start start 2024-09-11T23:23:08.316 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.314+0000 7f05dbed0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05d41009b0 0x7f05d4199270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:08.316 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.314+0000 7f05dbed0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f05d4101350 0x7f05d41997b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:08.316 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.314+0000 7f05dbed0640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f05d41069c0 0x7f05d41a42f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:08.316 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.314+0000 7f05dbed0640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f05d4110e00 con 0x7f05d41009b0 2024-09-11T23:23:08.316 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.314+0000 7f05dbed0640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f05d4110c80 con 0x7f05d4101350 2024-09-11T23:23:08.316 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.314+0000 7f05dbed0640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f05d4110f80 con 0x7f05d41069c0 2024-09-11T23:23:08.316 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.314+0000 7f05d9c6c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05d41009b0 0x7f05d4199270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:08.316 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.314+0000 7f05d9c6c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05d41009b0 0x7f05d4199270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50026/0 (socket says 172.21.15.26:50026) 2024-09-11T23:23:08.316 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.314+0000 7f05d9c6c640 1 -- 172.21.15.26:0/2760325570 learned_addr learned my addr 172.21.15.26:0/2760325570 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:23:08.316 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.314+0000 7f05d946b640 1 --2- 172.21.15.26:0/2760325570 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f05d4101350 0x7f05d41997b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:08.317 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.314+0000 7f05d9c6c640 1 -- 172.21.15.26:0/2760325570 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f05d41069c0 msgr2=0x7f05d41a42f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:23:08.317 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.314+0000 7f05d9c6c640 1 --2- 172.21.15.26:0/2760325570 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f05d41069c0 0x7f05d41a42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:08.317 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.314+0000 7f05d9c6c640 1 -- 172.21.15.26:0/2760325570 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f05d4101350 msgr2=0x7f05d41997b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:23:08.317 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.314+0000 7f05d9c6c640 1 --2- 172.21.15.26:0/2760325570 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f05d4101350 0x7f05d41997b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:08.317 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.314+0000 7f05d9c6c640 1 -- 172.21.15.26:0/2760325570 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05d41a4830 con 0x7f05d41009b0 2024-09-11T23:23:08.317 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.314+0000 7f05d9c6c640 1 --2- 172.21.15.26:0/2760325570 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05d41009b0 0x7f05d4199270 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f05c400ba50 tx=0x7f05c400bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:23:08.317 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.314+0000 7f05d2ffd640 1 -- 172.21.15.26:0/2760325570 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05c4019070 con 0x7f05d41009b0 2024-09-11T23:23:08.317 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.314+0000 7f05d2ffd640 1 -- 172.21.15.26:0/2760325570 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f05c400cbe0 con 0x7f05d41009b0 2024-09-11T23:23:08.317 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.314+0000 7f05dbed0640 1 -- 172.21.15.26:0/2760325570 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05d41a4a00 con 0x7f05d41009b0 2024-09-11T23:23:08.317 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.315+0000 7f05d2ffd640 1 -- 172.21.15.26:0/2760325570 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05c40076e0 con 0x7f05d41009b0 2024-09-11T23:23:08.317 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.315+0000 7f05dbed0640 1 -- 172.21.15.26:0/2760325570 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05d41a4ed0 con 0x7f05d41009b0 2024-09-11T23:23:08.322 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.317+0000 7f05d2ffd640 1 -- 172.21.15.26:0/2760325570 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f05c401c070 con 0x7f05d41009b0 2024-09-11T23:23:08.322 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.317+0000 7f05dec69740 1 Processor -- start 2024-09-11T23:23:08.322 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.317+0000 7f05dec69740 1 -- start start 2024-09-11T23:23:08.322 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.317+0000 7f05dec69740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d65a69caf0 0x55d65a69ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:08.322 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.317+0000 7f05dec69740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d65a69d410 0x55d65a6a7770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:08.323 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.317+0000 7f05dec69740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d65a6a7cb0 0x55d65a6aa0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:08.323 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.317+0000 7f05dec69740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55d65a6012f0 con 0x55d65a69caf0 2024-09-11T23:23:08.323 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.317+0000 7f05dec69740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55d65a5a9270 con 0x55d65a69d410 2024-09-11T23:23:08.323 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.317+0000 7f05dec69740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55d65a5fd2b0 con 0x55d65a6a7cb0 2024-09-11T23:23:08.323 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.318+0000 7f05da46d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d65a69caf0 0x55d65a69ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:08.323 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.318+0000 7f05d9c6c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d65a69d410 0x55d65a6a7770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:08.323 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.318+0000 7f05da46d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d65a69caf0 0x55d65a69ced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50038/0 (socket says 172.21.15.26:50038) 2024-09-11T23:23:08.323 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.318+0000 7f05da46d640 1 -- 172.21.15.26:0/147641710 learned_addr learned my addr 172.21.15.26:0/147641710 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:23:08.323 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.318+0000 7f05d946b640 1 --2- 172.21.15.26:0/147641710 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d65a6a7cb0 0x55d65a6aa0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:08.323 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.318+0000 7f05d2ffd640 1 --2- 172.21.15.26:0/2760325570 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f05bc0a2da0 0x7f05bc0a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:08.323 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.318+0000 7f05da46d640 1 -- 172.21.15.26:0/147641710 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d65a6a7cb0 msgr2=0x55d65a6aa0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:23:08.323 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.318+0000 7f05da46d640 1 --2- 172.21.15.26:0/147641710 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d65a6a7cb0 0x55d65a6aa0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:08.324 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.318+0000 7f05da46d640 1 -- 172.21.15.26:0/147641710 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d65a69d410 msgr2=0x55d65a6a7770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:23:08.324 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.318+0000 7f05da46d640 1 --2- 172.21.15.26:0/147641710 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d65a69d410 0x55d65a6a7770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:08.324 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.318+0000 7f05da46d640 1 -- 172.21.15.26:0/147641710 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d65a5a8340 con 0x55d65a69caf0 2024-09-11T23:23:08.324 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.318+0000 7f05d2ffd640 1 -- 172.21.15.26:0/2760325570 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f05c40ce930 con 0x7f05d41009b0 2024-09-11T23:23:08.324 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.318+0000 7f05da46d640 1 --2- 172.21.15.26:0/2760325570 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f05bc0a2da0 0x7f05bc0a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:08.324 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.318+0000 7f05da46d640 1 --2- 172.21.15.26:0/147641710 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d65a69caf0 0x55d65a69ced0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f05cc009a50 tx=0x7f05cc005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e292bc8214e4be1f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:23:08.324 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.318+0000 7f05bbfff640 1 -- 172.21.15.26:0/147641710 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05cc035040 con 0x55d65a69caf0 2024-09-11T23:23:08.324 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.318+0000 7f05bbfff640 1 -- 172.21.15.26:0/147641710 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f05cc040030 con 0x55d65a69caf0 2024-09-11T23:23:08.324 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.318+0000 7f05bbfff640 1 -- 172.21.15.26:0/147641710 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05cc045620 con 0x55d65a69caf0 2024-09-11T23:23:08.324 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.319+0000 7f05dec69740 1 -- 172.21.15.26:0/147641710 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d65a69caf0 msgr2=0x55d65a69ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:23:08.324 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.319+0000 7f05da46d640 1 --2- 172.21.15.26:0/2760325570 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f05bc0a2da0 0x7f05bc0a5260 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f05cc043000 tx=0x7f05cc045900 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:23:08.324 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.319+0000 7f05dec69740 1 --2- 172.21.15.26:0/147641710 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d65a69caf0 0x55d65a69ced0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f05cc009a50 tx=0x7f05cc005dc0 comp rx=0 tx=0).stop 2024-09-11T23:23:08.325 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.319+0000 7f05dec69740 1 -- 172.21.15.26:0/147641710 shutdown_connections 2024-09-11T23:23:08.325 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.319+0000 7f05dec69740 1 --2- 172.21.15.26:0/147641710 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d65a69d410 0x55d65a6a7770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:08.325 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.319+0000 7f05dec69740 1 --2- 172.21.15.26:0/147641710 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d65a6a7cb0 0x55d65a6aa0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:08.325 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.319+0000 7f05dec69740 1 --2- 172.21.15.26:0/147641710 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d65a69caf0 0x55d65a69ced0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:08.325 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.319+0000 7f05dec69740 1 -- 172.21.15.26:0/147641710 >> 172.21.15.26:0/147641710 conn(0x55d65a699060 msgr2=0x55d65a699440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:23:08.325 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.319+0000 7f05dec69740 1 -- 172.21.15.26:0/147641710 shutdown_connections 2024-09-11T23:23:08.325 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.319+0000 7f05dec69740 1 -- 172.21.15.26:0/147641710 wait complete. 2024-09-11T23:23:08.325 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.320+0000 7f05dec69740 1 -- start start 2024-09-11T23:23:08.325 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.320+0000 7f05dec69740 1 Processor -- start 2024-09-11T23:23:08.325 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.320+0000 7f05dec69740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d65a69caf0 0x55d65a6a4af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:08.326 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.320+0000 7f05dec69740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d65a69d410 0x55d65a6a5030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:08.326 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.320+0000 7f05dec69740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d65a6a7cb0 0x55d65a6a5570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:08.326 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.320+0000 7f05dec69740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55d65a6aa880 con 0x55d65a6a7cb0 2024-09-11T23:23:08.326 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.320+0000 7f05dec69740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55d65a58a290 con 0x55d65a69caf0 2024-09-11T23:23:08.326 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.320+0000 7f05dec69740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55d65a6aaa00 con 0x55d65a69d410 2024-09-11T23:23:08.326 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.320+0000 7f05dec69740 10 client.0 _ll_register_callbacks cb 0x55d65a5bb140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:23:08.326 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.320+0000 7f05da46d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d65a6a7cb0 0x55d65a6a5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:08.326 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.320+0000 7f05da46d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d65a6a7cb0 0x55d65a6a5570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50052/0 (socket says 172.21.15.26:50052) 2024-09-11T23:23:08.326 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.320+0000 7f05da46d640 1 -- 172.21.15.26:0/2835416567 learned_addr learned my addr 172.21.15.26:0/2835416567 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:23:08.327 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.320+0000 7f05d9c6c640 1 --2- 172.21.15.26:0/2835416567 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d65a69caf0 0x55d65a6a4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:08.327 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.320+0000 7f05d946b640 1 --2- 172.21.15.26:0/2835416567 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d65a69d410 0x55d65a6a5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:08.327 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.321+0000 7f05da46d640 1 -- 172.21.15.26:0/2835416567 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d65a69d410 msgr2=0x55d65a6a5030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:23:08.327 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.321+0000 7f05da46d640 1 --2- 172.21.15.26:0/2835416567 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d65a69d410 0x55d65a6a5030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:08.327 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.321+0000 7f05da46d640 1 -- 172.21.15.26:0/2835416567 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d65a69caf0 msgr2=0x55d65a6a4af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:23:08.327 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.321+0000 7f05da46d640 1 --2- 172.21.15.26:0/2835416567 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d65a69caf0 0x55d65a6a4af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:08.327 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.321+0000 7f05da46d640 1 -- 172.21.15.26:0/2835416567 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d65a314210 con 0x55d65a6a7cb0 2024-09-11T23:23:08.327 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.321+0000 7f05d9c6c640 1 --2- 172.21.15.26:0/2835416567 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d65a69caf0 0x55d65a6a4af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:23:08.327 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.321+0000 7f05da46d640 1 --2- 172.21.15.26:0/2835416567 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d65a6a7cb0 0x55d65a6a5570 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f05cc009a20 tx=0x7f05cc03e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=482991a5218cd950 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:23:08.327 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.321+0000 7f05b97fa640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:23:08.328 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.321+0000 7f05b97fa640 1 -- 172.21.15.26:0/2835416567 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05cc035070 con 0x55d65a6a7cb0 2024-09-11T23:23:08.328 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.321+0000 7f05b97fa640 1 -- 172.21.15.26:0/2835416567 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f05cc0315b0 con 0x55d65a6a7cb0 2024-09-11T23:23:08.328 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.321+0000 7f05dec69740 1 -- 172.21.15.26:0/2835416567 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f05cc040030 con 0x55d65a6a7cb0 2024-09-11T23:23:08.328 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.321+0000 7f05b97fa640 1 -- 172.21.15.26:0/2835416567 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f05cc061050 con 0x55d65a6a7cb0 2024-09-11T23:23:08.328 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.321+0000 7f05dec69740 10 client.16404 fetch_fsmap learned FSMap version 39 2024-09-11T23:23:08.328 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.321+0000 7f05dec69740 1 -- 172.21.15.26:0/2835416567 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55d65a3e0620 con 0x55d65a6a7cb0 2024-09-11T23:23:08.328 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.321+0000 7f05b97fa640 1 -- 172.21.15.26:0/2835416567 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f05cc040030 con 0x55d65a6a7cb0 2024-09-11T23:23:08.328 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.321+0000 7f05b97fa640 1 -- 172.21.15.26:0/2835416567 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 39) ==== 8410+0+0 (secure 0 0 0) 0x7f05cc005050 con 0x55d65a6a7cb0 2024-09-11T23:23:08.328 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.321+0000 7f05dec69740 10 client.16404 fetch_fsmap finished waiting for FSMap version 39 2024-09-11T23:23:08.328 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.321+0000 7f05dec69740 10 client.16404 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:23:08.329 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.321+0000 7f05dec69740 20 client.16404 populate_metadata read hostname 'smithi026' 2024-09-11T23:23:08.329 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.321+0000 7f05dec69740 1 --2- 172.21.15.26:0/2835416567 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55d65a6b79e0 0x55d65a6b9dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:08.329 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.321+0000 7f05dec69740 4 client.16404 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:23:08.329 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.321+0000 7f05dec69740 1 -- 172.21.15.26:0/2835416567 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d65a60ea20 con 0x55d65a6b79e0 2024-09-11T23:23:08.490 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.490+0000 7f05d9c6c640 1 --2- 172.21.15.26:0/2835416567 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55d65a6b79e0 0x55d65a6b9dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:08.490 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.491+0000 7f05b97fa640 10 client.16404 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:23:08.490 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.491+0000 7f05d9c6c640 1 --2- 172.21.15.26:0/2835416567 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55d65a6b79e0 0x55d65a6b9dd0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a6f82c69428b3d4 server_cookie=ce3578b5f83d6a9e in_seq=0 out_seq=0 2024-09-11T23:23:08.493 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.492+0000 7f05b97fa640 1 -- 172.21.15.26:0/2835416567 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d65a60ea20 con 0x55d65a6b79e0 2024-09-11T23:23:08.493 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.492+0000 7f05b97fa640 10 client.16404 handle_command_reply: tid=0 2024-09-11T23:23:08.493 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.492+0000 7f05dec69740 1 client.16404 shutdown 2024-09-11T23:23:08.493 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.492+0000 7f05dec69740 1 -- 172.21.15.26:0/2835416567 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d65a6a7cb0 msgr2=0x55d65a6a5570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:23:08.493 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.492+0000 7f05dec69740 1 --2- 172.21.15.26:0/2835416567 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d65a6a7cb0 0x55d65a6a5570 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f05cc009a20 tx=0x7f05cc03e000 comp rx=0 tx=0).stop 2024-09-11T23:23:08.493 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.493+0000 7f05dec69740 1 -- 172.21.15.26:0/2835416567 shutdown_connections 2024-09-11T23:23:08.493 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.493+0000 7f05dec69740 1 --2- 172.21.15.26:0/2835416567 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55d65a6b79e0 0x55d65a6b9dd0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:08.493 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.493+0000 7f05dec69740 1 --2- 172.21.15.26:0/2835416567 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d65a6a7cb0 0x55d65a6a5570 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:08.493 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.493+0000 7f05dec69740 1 --2- 172.21.15.26:0/2835416567 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d65a69d410 0x55d65a6a5030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:08.493 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.493+0000 7f05b97fa640 0 client.16404 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:23:08.493 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.493+0000 7f05dec69740 1 --2- 172.21.15.26:0/2835416567 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d65a69caf0 0x55d65a6a4af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:08.493 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.493+0000 7f05dec69740 1 -- 172.21.15.26:0/2835416567 >> 172.21.15.26:0/2835416567 conn(0x55d65a699060 msgr2=0x55d65a69edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:23:08.494 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.493+0000 7f05dec69740 1 -- 172.21.15.26:0/2835416567 shutdown_connections 2024-09-11T23:23:08.494 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.493+0000 7f05dec69740 1 -- 172.21.15.26:0/2835416567 wait complete. 2024-09-11T23:23:08.494 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.493+0000 7f05dec69740 20 client.16404 trim_cache size 0 max 16384 2024-09-11T23:23:08.514 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.513+0000 7f05dec69740 1 Processor -- start 2024-09-11T23:23:08.514 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.513+0000 7f05dec69740 1 -- start start 2024-09-11T23:23:08.514 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.513+0000 7f05dec69740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d65a69caf0 0x55d65a6a53a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:08.514 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.513+0000 7f05dec69740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d65a69d410 0x55d65a6bf480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:08.514 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.513+0000 7f05dec69740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d65a6a7cb0 0x55d65a6bf9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:08.514 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.513+0000 7f05dec69740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f05d41ad830 con 0x55d65a6a7cb0 2024-09-11T23:23:08.514 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.513+0000 7f05dec69740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f05d41af5b0 con 0x55d65a69d410 2024-09-11T23:23:08.515 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.513+0000 7f05dec69740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f05d41adfc0 con 0x55d65a69caf0 2024-09-11T23:23:08.515 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.513+0000 7f05da46d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d65a6a7cb0 0x55d65a6bf9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:08.515 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.513+0000 7f05da46d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d65a6a7cb0 0x55d65a6bf9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50056/0 (socket says 172.21.15.26:50056) 2024-09-11T23:23:08.515 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.513+0000 7f05d946b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d65a69d410 0x55d65a6bf480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:08.515 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.513+0000 7f05da46d640 1 -- 172.21.15.26:0/3401434041 learned_addr learned my addr 172.21.15.26:0/3401434041 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:23:08.515 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.513+0000 7f05d9c6c640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d65a69caf0 0x55d65a6a53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:08.515 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.514+0000 7f05da46d640 1 -- 172.21.15.26:0/3401434041 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d65a69caf0 msgr2=0x55d65a6a53a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:23:08.515 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.514+0000 7f05da46d640 1 --2- 172.21.15.26:0/3401434041 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d65a69caf0 0x55d65a6a53a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:08.515 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.514+0000 7f05da46d640 1 -- 172.21.15.26:0/3401434041 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d65a69d410 msgr2=0x55d65a6bf480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:23:08.515 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.514+0000 7f05da46d640 1 --2- 172.21.15.26:0/3401434041 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d65a69d410 0x55d65a6bf480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:08.515 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.514+0000 7f05da46d640 1 -- 172.21.15.26:0/3401434041 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d65a698e90 con 0x55d65a6a7cb0 2024-09-11T23:23:08.515 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.514+0000 7f05da46d640 1 --2- 172.21.15.26:0/3401434041 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d65a6a7cb0 0x55d65a6bf9c0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f05cc02a940 tx=0x7f05cc005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e368851838672e2 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:23:08.515 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.514+0000 7f05bb7fe640 1 -- 172.21.15.26:0/3401434041 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05cc035030 con 0x55d65a6a7cb0 2024-09-11T23:23:08.516 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.514+0000 7f05bb7fe640 1 -- 172.21.15.26:0/3401434041 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f05cc040040 con 0x55d65a6a7cb0 2024-09-11T23:23:08.516 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.514+0000 7f05bb7fe640 1 -- 172.21.15.26:0/3401434041 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05cc003850 con 0x55d65a6a7cb0 2024-09-11T23:23:08.519 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.515+0000 7f05dec69740 1 -- 172.21.15.26:0/3401434041 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d65a6a7cb0 msgr2=0x55d65a6bf9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:23:08.519 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.515+0000 7f05dec69740 1 --2- 172.21.15.26:0/3401434041 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d65a6a7cb0 0x55d65a6bf9c0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f05cc02a940 tx=0x7f05cc005780 comp rx=0 tx=0).stop 2024-09-11T23:23:08.519 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.515+0000 7f05dec69740 1 -- 172.21.15.26:0/3401434041 shutdown_connections 2024-09-11T23:23:08.519 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.515+0000 7f05dec69740 1 --2- 172.21.15.26:0/3401434041 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d65a69d410 0x55d65a6bf480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:08.519 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.515+0000 7f05dec69740 1 --2- 172.21.15.26:0/3401434041 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d65a6a7cb0 0x55d65a6bf9c0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:08.519 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.515+0000 7f05dec69740 1 --2- 172.21.15.26:0/3401434041 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d65a69caf0 0x55d65a6a53a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:08.519 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.515+0000 7f05dec69740 1 -- 172.21.15.26:0/3401434041 >> 172.21.15.26:0/3401434041 conn(0x55d65a699060 msgr2=0x55d65a6a6b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:23:08.519 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.515+0000 7f05dec69740 1 -- 172.21.15.26:0/3401434041 shutdown_connections 2024-09-11T23:23:08.519 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.515+0000 7f05dec69740 1 -- 172.21.15.26:0/3401434041 wait complete. 2024-09-11T23:23:08.519 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.515+0000 7f05dec69740 1 -- start start 2024-09-11T23:23:08.519 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.515+0000 7f05dec69740 1 Processor -- start 2024-09-11T23:23:08.520 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.516+0000 7f05dec69740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d65a69caf0 0x55d65a6abae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:08.520 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.516+0000 7f05dec69740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d65a69d410 0x55d65a6ac020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:08.520 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.516+0000 7f05dec69740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d65a6a7cb0 0x55d65a6ac560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:08.520 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.516+0000 7f05dec69740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f05d41aeee0 con 0x55d65a69caf0 2024-09-11T23:23:08.520 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.516+0000 7f05dec69740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f05d41ae750 con 0x55d65a69d410 2024-09-11T23:23:08.520 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.516+0000 7f05dec69740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f05d41ab810 con 0x55d65a6a7cb0 2024-09-11T23:23:08.520 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.516+0000 7f05dec69740 10 client.0 _ll_register_callbacks cb 0x7f05c40d4420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:23:08.520 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.516+0000 7f05d9c6c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d65a69caf0 0x55d65a6abae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:08.520 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.516+0000 7f05da46d640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d65a6a7cb0 0x55d65a6ac560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:08.520 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.516+0000 7f05d946b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d65a69d410 0x55d65a6ac020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:08.520 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.516+0000 7f05d9c6c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d65a69caf0 0x55d65a6abae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50058/0 (socket says 172.21.15.26:50058) 2024-09-11T23:23:08.520 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.516+0000 7f05d9c6c640 1 -- 172.21.15.26:0/446983473 learned_addr learned my addr 172.21.15.26:0/446983473 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:23:08.520 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.517+0000 7f05d946b640 1 -- 172.21.15.26:0/446983473 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d65a6a7cb0 msgr2=0x55d65a6ac560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:23:08.521 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.517+0000 7f05d946b640 1 --2- 172.21.15.26:0/446983473 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d65a6a7cb0 0x55d65a6ac560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:08.521 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.517+0000 7f05d946b640 1 -- 172.21.15.26:0/446983473 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d65a69caf0 msgr2=0x55d65a6abae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:23:08.521 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.517+0000 7f05d946b640 1 --2- 172.21.15.26:0/446983473 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d65a69caf0 0x55d65a6abae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:08.521 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.517+0000 7f05d946b640 1 -- 172.21.15.26:0/446983473 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d65a6942d0 con 0x55d65a69d410 2024-09-11T23:23:08.521 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.517+0000 7f05d9c6c640 1 --2- 172.21.15.26:0/446983473 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d65a69caf0 0x55d65a6abae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:23:08.521 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.517+0000 7f05d946b640 1 --2- 172.21.15.26:0/446983473 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d65a69d410 0x55d65a6ac020 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x55d65a6a4360 tx=0x7f05c8010a80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8defd41840ddf412 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:23:08.521 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.517+0000 7f05b97fa640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-11T23:23:08.521 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.517+0000 7f05b97fa640 1 -- 172.21.15.26:0/446983473 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05c8005020 con 0x55d65a69d410 2024-09-11T23:23:08.521 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.517+0000 7f05b97fa640 1 -- 172.21.15.26:0/446983473 <== mon.2 v2:172.21.15.76:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f05c80051c0 con 0x55d65a69d410 2024-09-11T23:23:08.521 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.517+0000 7f05dec69740 1 -- 172.21.15.26:0/446983473 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f05cc040040 con 0x55d65a69d410 2024-09-11T23:23:08.521 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.518+0000 7f05b97fa640 1 -- 172.21.15.26:0/446983473 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f05c802a050 con 0x55d65a69d410 2024-09-11T23:23:08.521 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.518+0000 7f05dec69740 10 client.26009 fetch_fsmap learned FSMap version 39 2024-09-11T23:23:08.521 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.518+0000 7f05dec69740 1 -- 172.21.15.26:0/446983473 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55d65a699cf0 con 0x55d65a69d410 2024-09-11T23:23:08.522 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.518+0000 7f05b97fa640 1 -- 172.21.15.26:0/446983473 <== mon.2 v2:172.21.15.76:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f05cc040040 con 0x55d65a69d410 2024-09-11T23:23:08.522 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.518+0000 7f05b97fa640 1 -- 172.21.15.26:0/446983473 <== mon.2 v2:172.21.15.76:3300/0 5 ==== fsmap(e 39) ==== 8410+0+0 (secure 0 0 0) 0x7f05c800f9b0 con 0x55d65a69d410 2024-09-11T23:23:08.522 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.518+0000 7f05dec69740 10 client.26009 fetch_fsmap finished waiting for FSMap version 39 2024-09-11T23:23:08.522 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.518+0000 7f05dec69740 10 client.26009 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:23:08.522 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.518+0000 7f05dec69740 20 client.26009 populate_metadata read hostname 'smithi026' 2024-09-11T23:23:08.522 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.518+0000 7f05dec69740 1 --2- 172.21.15.26:0/446983473 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55d65a72b510 0x55d65a72d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:08.522 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.518+0000 7f05dec69740 4 client.26009 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T23:23:08.522 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.518+0000 7f05dec69740 1 -- 172.21.15.26:0/446983473 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d65a608d70 con 0x55d65a72b510 2024-09-11T23:23:08.522 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.519+0000 7f05d9c6c640 1 --2- 172.21.15.26:0/446983473 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55d65a72b510 0x55d65a72d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:08.522 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.519+0000 7f05b97fa640 10 client.26009 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:23:08.522 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:08.519+0000 7f05d9c6c640 1 --2- 172.21.15.26:0/446983473 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55d65a72b510 0x55d65a72d930 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=11de96c2117d2e0f server_cookie=e61c7c2d17caaa2e in_seq=0 out_seq=0 2024-09-11T23:23:09.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:23:09 smithi136 ceph-mon[35121]: pgmap v146: 97 pgs: 97 active+clean; 1.4 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 9.0 MiB/s wr, 672 op/s 2024-09-11T23:23:10.000 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:23:09 smithi026 ceph-mon[31172]: pgmap v146: 97 pgs: 97 active+clean; 1.4 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 9.0 MiB/s wr, 672 op/s 2024-09-11T23:23:10.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:23:09 smithi076 ceph-mon[35296]: pgmap v146: 97 pgs: 97 active+clean; 1.4 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 9.0 MiB/s wr, 672 op/s 2024-09-11T23:23:11.355 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:11.354+0000 7f05b97fa640 1 -- 172.21.15.26:0/446983473 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d65a608d70 con 0x55d65a72b510 2024-09-11T23:23:11.355 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:11.354+0000 7f05b97fa640 10 client.26009 handle_command_reply: tid=0 2024-09-11T23:23:11.355 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:11.354+0000 7f05dec69740 1 client.26009 shutdown 2024-09-11T23:23:11.356 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:11.355+0000 7f05dec69740 1 -- 172.21.15.26:0/446983473 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d65a69d410 msgr2=0x55d65a6ac020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:23:11.356 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:11.355+0000 7f05dec69740 1 --2- 172.21.15.26:0/446983473 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d65a69d410 0x55d65a6ac020 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x55d65a6a4360 tx=0x7f05c8010a80 comp rx=0 tx=0).stop 2024-09-11T23:23:11.356 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:11.355+0000 7f05dec69740 1 -- 172.21.15.26:0/446983473 shutdown_connections 2024-09-11T23:23:11.356 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:11.355+0000 7f05dec69740 1 --2- 172.21.15.26:0/446983473 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d65a69d410 0x55d65a6ac020 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:11.357 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:11.355+0000 7f05dec69740 1 --2- 172.21.15.26:0/446983473 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55d65a72b510 0x55d65a72d930 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:11.357 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:11.355+0000 7f05dec69740 1 --2- 172.21.15.26:0/446983473 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d65a6a7cb0 0x55d65a6ac560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:11.357 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:11.355+0000 7f05dec69740 1 --2- 172.21.15.26:0/446983473 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d65a69caf0 0x55d65a6abae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:11.357 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:11.355+0000 7f05dec69740 1 -- 172.21.15.26:0/446983473 >> 172.21.15.26:0/446983473 conn(0x55d65a699060 msgr2=0x55d65a69e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:23:11.357 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:11.355+0000 7f05b97fa640 0 client.26009 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:23:11.357 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:11.356+0000 7f05dec69740 1 -- 172.21.15.26:0/446983473 shutdown_connections 2024-09-11T23:23:11.357 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:11.356+0000 7f05dec69740 1 -- 172.21.15.26:0/446983473 wait complete. 2024-09-11T23:23:11.357 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:11.356+0000 7f05dec69740 20 client.26009 trim_cache size 0 max 16384 2024-09-11T23:23:11.357 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-09-11T23:23:11.358 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "message": "", 2024-09-11T23:23:11.358 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-09-11T23:23:11.358 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-09-11T23:23:11.359 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:11.357+0000 7f05dbed0640 1 -- 172.21.15.26:0/2760325570 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f05bc0a2da0 msgr2=0x7f05bc0a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:23:11.359 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:11.357+0000 7f05dbed0640 1 --2- 172.21.15.26:0/2760325570 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f05bc0a2da0 0x7f05bc0a5260 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f05cc043000 tx=0x7f05cc045900 comp rx=0 tx=0).stop 2024-09-11T23:23:11.359 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:11.357+0000 7f05dbed0640 1 -- 172.21.15.26:0/2760325570 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05d41009b0 msgr2=0x7f05d4199270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:23:11.359 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:11.357+0000 7f05dbed0640 1 --2- 172.21.15.26:0/2760325570 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05d41009b0 0x7f05d4199270 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f05c400ba50 tx=0x7f05c400bf10 comp rx=0 tx=0).stop 2024-09-11T23:23:11.359 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:11.357+0000 7f05dbed0640 1 -- 172.21.15.26:0/2760325570 shutdown_connections 2024-09-11T23:23:11.359 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:11.357+0000 7f05dbed0640 1 --2- 172.21.15.26:0/2760325570 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f05bc0a2da0 0x7f05bc0a5260 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:11.359 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:11.357+0000 7f05dbed0640 1 --2- 172.21.15.26:0/2760325570 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f05d4101350 0x7f05d41997b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:11.359 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:11.357+0000 7f05dbed0640 1 --2- 172.21.15.26:0/2760325570 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f05d41069c0 0x7f05d41a42f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:11.359 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:11.357+0000 7f05dbed0640 1 --2- 172.21.15.26:0/2760325570 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05d41009b0 0x7f05d4199270 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:11.359 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:11.357+0000 7f05dbed0640 1 -- 172.21.15.26:0/2760325570 >> 172.21.15.26:0/2760325570 conn(0x7f05d4077990 msgr2=0x7f05d41037f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:23:11.360 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:11.357+0000 7f05dbed0640 1 -- 172.21.15.26:0/2760325570 shutdown_connections 2024-09-11T23:23:11.360 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:11.357+0000 7f05dbed0640 1 -- 172.21.15.26:0/2760325570 wait complete. 2024-09-11T23:23:11.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:23:11 smithi136 ceph-mon[35121]: pgmap v147: 97 pgs: 97 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 849 op/s 2024-09-11T23:23:12.000 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:23:11 smithi026 ceph-mon[31172]: pgmap v147: 97 pgs: 97 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 849 op/s 2024-09-11T23:23:12.052 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:23:11 smithi076 ceph-mon[35296]: pgmap v147: 97 pgs: 97 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 849 op/s 2024-09-11T23:23:13.578 INFO:tasks.workunit.client.0.smithi026.stdout: 365 59198 0 38898 0 12229 1 2024-09-11T23:23:13.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:23:13 smithi136 ceph-mon[35121]: pgmap v148: 97 pgs: 97 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 8.0 MiB/s wr, 509 op/s 2024-09-11T23:23:14.000 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:23:13 smithi026 ceph-mon[31172]: pgmap v148: 97 pgs: 97 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 8.0 MiB/s wr, 509 op/s 2024-09-11T23:23:14.052 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:23:13 smithi076 ceph-mon[35296]: pgmap v148: 97 pgs: 97 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 8.0 MiB/s wr, 509 op/s 2024-09-11T23:23:15.825 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:23:15.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:23:15 smithi136 ceph-mon[35121]: pgmap v149: 97 pgs: 97 active+clean; 1.4 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 8.6 MiB/s wr, 930 op/s 2024-09-11T23:23:16.000 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:23:15 smithi026 ceph-mon[31172]: pgmap v149: 97 pgs: 97 active+clean; 1.4 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 8.6 MiB/s wr, 930 op/s 2024-09-11T23:23:16.052 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:23:15 smithi076 ceph-mon[35296]: pgmap v149: 97 pgs: 97 active+clean; 1.4 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 8.6 MiB/s wr, 930 op/s 2024-09-11T23:23:16.503 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:23:17.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:17.297+0000 7f350f5ba640 1 -- 172.21.15.26:0/2332910752 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3508109830 msgr2=0x7f3508109c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:23:17.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:17.297+0000 7f350f5ba640 1 --2- 172.21.15.26:0/2332910752 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3508109830 0x7f3508109c90 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f34f8009900 tx=0x7f34f802a440 comp rx=0 tx=0).stop 2024-09-11T23:23:17.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:17.298+0000 7f350f5ba640 1 -- 172.21.15.26:0/2332910752 shutdown_connections 2024-09-11T23:23:17.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:17.298+0000 7f350f5ba640 1 --2- 172.21.15.26:0/2332910752 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f350811b140 0x7f350811d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:17.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:17.298+0000 7f350f5ba640 1 --2- 172.21.15.26:0/2332910752 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3508109830 0x7f3508109c90 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:17.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:17.298+0000 7f350f5ba640 1 --2- 172.21.15.26:0/2332910752 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3508109450 0x7f3508110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:17.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:17.298+0000 7f350f5ba640 1 -- 172.21.15.26:0/2332910752 >> 172.21.15.26:0/2332910752 conn(0x7f350806e140 msgr2=0x7f350806e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:23:17.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:17.306+0000 7f350f5ba640 1 -- 172.21.15.26:0/2332910752 shutdown_connections 2024-09-11T23:23:17.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:17.309+0000 7f350f5ba640 1 -- 172.21.15.26:0/2332910752 wait complete. 2024-09-11T23:23:17.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:17.310+0000 7f350f5ba640 1 Processor -- start 2024-09-11T23:23:17.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:17.315+0000 7f350f5ba640 1 -- start start 2024-09-11T23:23:17.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:17.316+0000 7f350f5ba640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3508109450 0x7f35081a7050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:17.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:17.316+0000 7f350f5ba640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3508109830 0x7f35081a7590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:17.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:17.316+0000 7f350d356640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3508109450 0x7f35081a7050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:17.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:17.316+0000 7f350d356640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3508109450 0x7f35081a7050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:55704/0 (socket says 172.21.15.26:55704) 2024-09-11T23:23:17.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:17.317+0000 7f350cb55640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3508109830 0x7f35081a7590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:17.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:17.317+0000 7f350cb55640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3508109830 0x7f35081a7590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47842/0 (socket says 172.21.15.26:47842) 2024-09-11T23:23:17.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:17.317+0000 7f350d356640 1 -- 172.21.15.26:0/1032247444 learned_addr learned my addr 172.21.15.26:0/1032247444 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:23:17.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:17.317+0000 7f350f5ba640 1 --2- 172.21.15.26:0/1032247444 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f350811b140 0x7f35081ab920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:17.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:17.317+0000 7f350f5ba640 1 -- 172.21.15.26:0/1032247444 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f3508120df0 con 0x7f3508109830 2024-09-11T23:23:17.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:17.317+0000 7f350f5ba640 1 -- 172.21.15.26:0/1032247444 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f3508120c70 con 0x7f350811b140 2024-09-11T23:23:17.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:17.317+0000 7f350f5ba640 1 -- 172.21.15.26:0/1032247444 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f3508120f70 con 0x7f3508109450 2024-09-11T23:23:17.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:17.318+0000 7f350d356640 1 -- 172.21.15.26:0/1032247444 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f350811b140 msgr2=0x7f35081ab920 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-11T23:23:17.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:17.318+0000 7f350d356640 1 --2- 172.21.15.26:0/1032247444 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f350811b140 0x7f35081ab920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:17.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:17.318+0000 7f350d356640 1 -- 172.21.15.26:0/1032247444 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3508109830 msgr2=0x7f35081a7590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:23:17.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:17.318+0000 7f350d356640 1 --2- 172.21.15.26:0/1032247444 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3508109830 0x7f35081a7590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:17.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:17.318+0000 7f350d356640 1 -- 172.21.15.26:0/1032247444 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35081ac000 con 0x7f3508109450 2024-09-11T23:23:17.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:17.319+0000 7f350cb55640 1 --2- 172.21.15.26:0/1032247444 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3508109830 0x7f35081a7590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:23:17.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:17.326+0000 7f350d356640 1 --2- 172.21.15.26:0/1032247444 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3508109450 0x7f35081a7050 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f350400c890 tx=0x7f350400cd50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:23:17.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:17.329+0000 7f34fe7fc640 1 -- 172.21.15.26:0/1032247444 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3504004220 con 0x7f3508109450 2024-09-11T23:23:17.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:17.329+0000 7f350f5ba640 1 -- 172.21.15.26:0/1032247444 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35081ac2f0 con 0x7f3508109450 2024-09-11T23:23:17.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:17.329+0000 7f350f5ba640 1 -- 172.21.15.26:0/1032247444 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35081b3bd0 con 0x7f3508109450 2024-09-11T23:23:17.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:17.331+0000 7f34fe7fc640 1 -- 172.21.15.26:0/1032247444 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35040043c0 con 0x7f3508109450 2024-09-11T23:23:17.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:17.331+0000 7f34fe7fc640 1 -- 172.21.15.26:0/1032247444 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35040049e0 con 0x7f3508109450 2024-09-11T23:23:17.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:17.331+0000 7f350f5ba640 1 -- 172.21.15.26:0/1032247444 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35081a83b0 con 0x7f3508109450 2024-09-11T23:23:17.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:17.342+0000 7f34fe7fc640 1 -- 172.21.15.26:0/1032247444 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f35040079b0 con 0x7f3508109450 2024-09-11T23:23:17.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:17.344+0000 7f34fe7fc640 1 --2- 172.21.15.26:0/1032247444 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f34f40a2f00 0x7f34f40a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:17.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:17.344+0000 7f34fe7fc640 1 -- 172.21.15.26:0/1032247444 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f35040cf470 con 0x7f3508109450 2024-09-11T23:23:17.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:17.345+0000 7f34fe7fc640 1 -- 172.21.15.26:0/1032247444 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3504014030 con 0x7f3508109450 2024-09-11T23:23:17.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:17.345+0000 7f350cb55640 1 --2- 172.21.15.26:0/1032247444 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f34f40a2f00 0x7f34f40a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:17.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:17.346+0000 7f350cb55640 1 --2- 172.21.15.26:0/1032247444 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f34f40a2f00 0x7f34f40a53c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f34f80096c0 tx=0x7f34f8000bc0 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:23:17.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:17.834+0000 7f350f5ba640 1 -- 172.21.15.26:0/1032247444 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f350810b320 con 0x7f3508109450 2024-09-11T23:23:17.836 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-11T23:23:17.836 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":39,"btime":"2024-09-11T23:23:01:650738+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14499,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/367659515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":367659515},{"type":"v1","addr":"172.21.15.26:6835","nonce":367659515}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14511,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/2041065299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":2041065299},{"type":"v1","addr":"172.21.15.26:6837","nonce":2041065299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":39,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:16:26.544203+0000","modified":"2024-09-11T23:23:01.110910+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24485,"mds_1":24463,"mds_2":24458,"mds_3":24448,"mds_4":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14550":{"gid":14550,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6839/2233068305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":2233068305},{"type":"v1","addr":"172.21.15.26:6839","nonce":2233068305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14562":{"gid":14562,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6841/2984982336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":2984982336},{"type":"v1","addr":"172.21.15.26:6841","nonce":2984982336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6833/2019841455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":2019841455},{"type":"v1","addr":"172.21.15.136:6833","nonce":2019841455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.76:6833/3438147050","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3438147050},{"type":"v1","addr":"172.21.15.76:6833","nonce":3438147050}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6835/3617665975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3617665975},{"type":"v1","addr":"172.21.15.136:6835","nonce":3617665975}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":12,"addr":"172.21.15.76:6835/393308499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":393308499},{"type":"v1","addr":"172.21.15.76:6835","nonce":393308499}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":10,"addr":"172.21.15.136:6837/1633082387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":1633082387},{"type":"v1","addr":"172.21.15.136:6837","nonce":1633082387}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.76:6837/2078978748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":2078978748},{"type":"v1","addr":"172.21.15.76:6837","nonce":2078978748}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":63,"addr":"172.21.15.136:6839/3211398630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":3211398630},{"type":"v1","addr":"172.21.15.136:6839","nonce":3211398630}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.76:6839/2077637090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6838","nonce":2077637090},{"type":"v1","addr":"172.21.15.76:6839","nonce":2077637090}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24485,"qdb_cluster":[24443,24448,24458,24463,24485]},"id":1}]} 2024-09-11T23:23:17.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:17.835+0000 7f34fe7fc640 1 -- 172.21.15.26:0/1032247444 <== mon.1 v2:172.21.15.136:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+11993 (secure 0 0 0) 0x7f350409c0c0 con 0x7f3508109450 2024-09-11T23:23:17.837 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 39 2024-09-11T23:23:17.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:17.846+0000 7f34ebfff640 1 -- 172.21.15.26:0/1032247444 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f34f40a2f00 msgr2=0x7f34f40a53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:23:17.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:17.846+0000 7f34ebfff640 1 --2- 172.21.15.26:0/1032247444 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f34f40a2f00 0x7f34f40a53c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f34f80096c0 tx=0x7f34f8000bc0 comp rx=0 tx=0).stop 2024-09-11T23:23:17.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:17.846+0000 7f34ebfff640 1 -- 172.21.15.26:0/1032247444 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3508109450 msgr2=0x7f35081a7050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:23:17.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:17.846+0000 7f34ebfff640 1 --2- 172.21.15.26:0/1032247444 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3508109450 0x7f35081a7050 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f350400c890 tx=0x7f350400cd50 comp rx=0 tx=0).stop 2024-09-11T23:23:17.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:17.851+0000 7f34ebfff640 1 -- 172.21.15.26:0/1032247444 shutdown_connections 2024-09-11T23:23:17.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:17.851+0000 7f34ebfff640 1 --2- 172.21.15.26:0/1032247444 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f34f40a2f00 0x7f34f40a53c0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:17.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:17.851+0000 7f34ebfff640 1 --2- 172.21.15.26:0/1032247444 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f350811b140 0x7f35081ab920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:17.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:17.851+0000 7f34ebfff640 1 --2- 172.21.15.26:0/1032247444 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3508109830 0x7f35081a7590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:17.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:17.851+0000 7f34ebfff640 1 --2- 172.21.15.26:0/1032247444 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3508109450 0x7f35081a7050 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:17.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:17.851+0000 7f34ebfff640 1 -- 172.21.15.26:0/1032247444 >> 172.21.15.26:0/1032247444 conn(0x7f350806e140 msgr2=0x7f350811bee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:23:17.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:17.858+0000 7f34ebfff640 1 -- 172.21.15.26:0/1032247444 shutdown_connections 2024-09-11T23:23:17.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:17.863+0000 7f34ebfff640 1 -- 172.21.15.26:0/1032247444 wait complete. 2024-09-11T23:23:17.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:23:17 smithi136 ceph-mon[35121]: pgmap v150: 97 pgs: 97 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 4.4 MiB/s wr, 850 op/s 2024-09-11T23:23:18.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:23:17 smithi026 ceph-mon[31172]: pgmap v150: 97 pgs: 97 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 4.4 MiB/s wr, 850 op/s 2024-09-11T23:23:18.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:23:17 smithi076 ceph-mon[35296]: pgmap v150: 97 pgs: 97 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 4.4 MiB/s wr, 850 op/s 2024-09-11T23:23:18.920 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24485 quiesce db --set-id e975f3df --release --await' 2024-09-11T23:23:18.920 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24485 quiesce db --set-id e975f3df --release --await 2024-09-11T23:23:18.943 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:23:18 smithi026 ceph-mon[31172]: from='client.? 172.21.15.26:0/1032247444' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:23:18.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:23:18 smithi136 ceph-mon[35121]: from='client.? 172.21.15.26:0/1032247444' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:23:19.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:23:18 smithi076 ceph-mon[35296]: from='client.? 172.21.15.26:0/1032247444' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:23:19.183 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:23:19.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.689+0000 7f674c67e640 1 -- 172.21.15.26:0/466119938 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6744108300 msgr2=0x7f674410ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:23:19.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.689+0000 7f674c67e640 1 --2- 172.21.15.26:0/466119938 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6744108300 0x7f674410ec70 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f673c009cd0 tx=0x7f673c02a8d0 comp rx=0 tx=0).stop 2024-09-11T23:23:19.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.689+0000 7f674c67e640 1 -- 172.21.15.26:0/466119938 shutdown_connections 2024-09-11T23:23:19.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.689+0000 7f674c67e640 1 --2- 172.21.15.26:0/466119938 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6744103780 0x7f6744107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:19.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.689+0000 7f674c67e640 1 --2- 172.21.15.26:0/466119938 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6744108300 0x7f674410ec70 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:19.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.689+0000 7f674c67e640 1 --2- 172.21.15.26:0/466119938 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6744102dd0 0x7f67441031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:19.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.689+0000 7f674c67e640 1 -- 172.21.15.26:0/466119938 >> 172.21.15.26:0/466119938 conn(0x7f67440fc7d0 msgr2=0x7f67440febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:23:19.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.689+0000 7f674c67e640 1 -- 172.21.15.26:0/466119938 shutdown_connections 2024-09-11T23:23:19.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.689+0000 7f674c67e640 1 -- 172.21.15.26:0/466119938 wait complete. 2024-09-11T23:23:19.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.690+0000 7f674c67e640 1 Processor -- start 2024-09-11T23:23:19.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.690+0000 7f674c67e640 1 -- start start 2024-09-11T23:23:19.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.690+0000 7f674c67e640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6744102dd0 0x7f67441985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:19.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.690+0000 7f674c67e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6744103780 0x7f6744198b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:19.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.690+0000 7f674c67e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6744108300 0x7f674419cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:19.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.690+0000 7f674c67e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f67441120d0 con 0x7f6744108300 2024-09-11T23:23:19.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.690+0000 7f674c67e640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f6744111f50 con 0x7f6744103780 2024-09-11T23:23:19.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.690+0000 7f674c67e640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f6744112250 con 0x7f6744102dd0 2024-09-11T23:23:19.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.690+0000 7f674ac1b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6744108300 0x7f674419cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:19.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.690+0000 7f674ac1b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6744108300 0x7f674419cec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47852/0 (socket says 172.21.15.26:47852) 2024-09-11T23:23:19.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.690+0000 7f674ac1b640 1 -- 172.21.15.26:0/286841578 learned_addr learned my addr 172.21.15.26:0/286841578 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:23:19.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.690+0000 7f6749c19640 1 --2- 172.21.15.26:0/286841578 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6744103780 0x7f6744198b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:19.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.690+0000 7f674a41a640 1 --2- 172.21.15.26:0/286841578 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6744102dd0 0x7f67441985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:19.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.690+0000 7f674ac1b640 1 -- 172.21.15.26:0/286841578 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6744102dd0 msgr2=0x7f67441985f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:23:19.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.690+0000 7f674ac1b640 1 --2- 172.21.15.26:0/286841578 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6744102dd0 0x7f67441985f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:19.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.690+0000 7f674ac1b640 1 -- 172.21.15.26:0/286841578 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6744103780 msgr2=0x7f6744198b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:23:19.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.690+0000 7f674ac1b640 1 --2- 172.21.15.26:0/286841578 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6744103780 0x7f6744198b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:19.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.690+0000 7f674ac1b640 1 -- 172.21.15.26:0/286841578 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f674419d5a0 con 0x7f6744108300 2024-09-11T23:23:19.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.690+0000 7f674a41a640 1 --2- 172.21.15.26:0/286841578 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6744102dd0 0x7f67441985f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:23:19.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.691+0000 7f674ac1b640 1 --2- 172.21.15.26:0/286841578 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6744108300 0x7f674419cec0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f673c009ca0 tx=0x7f673c02f420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:23:19.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.691+0000 7f67377fe640 1 -- 172.21.15.26:0/286841578 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f673c031860 con 0x7f6744108300 2024-09-11T23:23:19.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.691+0000 7f67377fe640 1 -- 172.21.15.26:0/286841578 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f673c003e50 con 0x7f6744108300 2024-09-11T23:23:19.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.691+0000 7f67377fe640 1 -- 172.21.15.26:0/286841578 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f673c0307f0 con 0x7f6744108300 2024-09-11T23:23:19.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.691+0000 7f674c67e640 1 -- 172.21.15.26:0/286841578 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f674419d830 con 0x7f6744108300 2024-09-11T23:23:19.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.691+0000 7f674c67e640 1 -- 172.21.15.26:0/286841578 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67441a5170 con 0x7f6744108300 2024-09-11T23:23:19.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.694+0000 7f67377fe640 1 -- 172.21.15.26:0/286841578 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f673c0041d0 con 0x7f6744108300 2024-09-11T23:23:19.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.694+0000 7f674ea57740 1 Processor -- start 2024-09-11T23:23:19.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.694+0000 7f674ea57740 1 -- start start 2024-09-11T23:23:19.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.695+0000 7f674ea57740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5631e72b8f40 0x5631e72b9320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:19.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.695+0000 7f674ea57740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5631e72b9860 0x5631e72c8110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:19.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.695+0000 7f674ea57740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5631e72c8650 0x5631e72caa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:19.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.695+0000 7f674ea57740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5631e721e620 con 0x5631e72c8650 2024-09-11T23:23:19.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.695+0000 7f674ea57740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5631e71c5e50 con 0x5631e72b9860 2024-09-11T23:23:19.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.695+0000 7f674ea57740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5631e721a5e0 con 0x5631e72b8f40 2024-09-11T23:23:19.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.695+0000 7f6749c19640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5631e72c8650 0x5631e72caa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:19.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.695+0000 7f674a41a640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5631e72b9860 0x5631e72c8110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:19.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.695+0000 7f6749c19640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5631e72c8650 0x5631e72caa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47864/0 (socket says 172.21.15.26:47864) 2024-09-11T23:23:19.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.695+0000 7f6749c19640 1 -- 172.21.15.26:0/3118006695 learned_addr learned my addr 172.21.15.26:0/3118006695 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:23:19.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.695+0000 7f6749c19640 1 --2- 172.21.15.26:0/3118006695 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5631e72b8f40 0x5631e72b9320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:19.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.695+0000 7f6749c19640 1 -- 172.21.15.26:0/3118006695 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5631e72b8f40 msgr2=0x5631e72b9320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:23:19.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.695+0000 7f6749c19640 1 --2- 172.21.15.26:0/3118006695 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5631e72b8f40 0x5631e72b9320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:19.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.695+0000 7f6749c19640 1 -- 172.21.15.26:0/3118006695 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5631e72b9860 msgr2=0x5631e72c8110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:23:19.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.695+0000 7f6749c19640 1 --2- 172.21.15.26:0/3118006695 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5631e72b9860 0x5631e72c8110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:19.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.695+0000 7f67377fe640 1 --2- 172.21.15.26:0/286841578 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f67300a2cf0 0x7f67300a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:19.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.695+0000 7f6749c19640 1 -- 172.21.15.26:0/3118006695 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5631e71b6d40 con 0x5631e72c8650 2024-09-11T23:23:19.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.695+0000 7f67377fe640 1 -- 172.21.15.26:0/286841578 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f673c0f0320 con 0x7f6744108300 2024-09-11T23:23:19.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.695+0000 7f6749c19640 1 --2- 172.21.15.26:0/286841578 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f67300a2cf0 0x7f67300a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:19.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.696+0000 7f6749c19640 1 --2- 172.21.15.26:0/3118006695 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5631e72c8650 0x5631e72caa40 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f6738011630 tx=0x7f673800a5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80111545c2d4824 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:23:19.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.696+0000 7f6749c19640 1 --2- 172.21.15.26:0/286841578 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f67300a2cf0 0x7f67300a51b0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f6738023040 tx=0x7f673801f000 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:23:19.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.698+0000 7f6734ff9640 1 -- 172.21.15.26:0/3118006695 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f673800fd90 con 0x5631e72c8650 2024-09-11T23:23:19.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.698+0000 7f6734ff9640 1 -- 172.21.15.26:0/3118006695 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6738021de0 con 0x5631e72c8650 2024-09-11T23:23:19.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.698+0000 7f6734ff9640 1 -- 172.21.15.26:0/3118006695 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6738020d80 con 0x5631e72c8650 2024-09-11T23:23:19.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.698+0000 7f674ea57740 1 -- 172.21.15.26:0/3118006695 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5631e72c8650 msgr2=0x5631e72caa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:23:19.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.698+0000 7f674ea57740 1 --2- 172.21.15.26:0/3118006695 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5631e72c8650 0x5631e72caa40 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f6738011630 tx=0x7f673800a5e0 comp rx=0 tx=0).stop 2024-09-11T23:23:19.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.698+0000 7f674ea57740 1 -- 172.21.15.26:0/3118006695 shutdown_connections 2024-09-11T23:23:19.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.698+0000 7f674ea57740 1 --2- 172.21.15.26:0/3118006695 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5631e72b9860 0x5631e72c8110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:19.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.698+0000 7f674ea57740 1 --2- 172.21.15.26:0/3118006695 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5631e72c8650 0x5631e72caa40 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:19.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.698+0000 7f674ea57740 1 --2- 172.21.15.26:0/3118006695 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5631e72b8f40 0x5631e72b9320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:19.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.698+0000 7f674ea57740 1 -- 172.21.15.26:0/3118006695 >> 172.21.15.26:0/3118006695 conn(0x5631e72b55f0 msgr2=0x5631e72b59d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:23:19.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.699+0000 7f674ea57740 1 -- 172.21.15.26:0/3118006695 shutdown_connections 2024-09-11T23:23:19.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.699+0000 7f674ea57740 1 -- 172.21.15.26:0/3118006695 wait complete. 2024-09-11T23:23:19.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.699+0000 7f674ea57740 1 -- start start 2024-09-11T23:23:19.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.699+0000 7f674ea57740 1 Processor -- start 2024-09-11T23:23:19.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.699+0000 7f674ea57740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5631e72b8f40 0x5631e72c55c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:19.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.699+0000 7f674ea57740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5631e72b9860 0x5631e72c5b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:19.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.699+0000 7f674ea57740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5631e72c8650 0x5631e72c6040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:19.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.699+0000 7f674ea57740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5631e72cb220 con 0x5631e72c8650 2024-09-11T23:23:19.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.699+0000 7f674ea57740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5631e71a6f10 con 0x5631e72b8f40 2024-09-11T23:23:19.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.699+0000 7f674ea57740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5631e72cb3a0 con 0x5631e72b9860 2024-09-11T23:23:19.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.699+0000 7f674ea57740 10 client.0 _ll_register_callbacks cb 0x5631e71cadb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:23:19.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.699+0000 7f674ac1b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5631e72c8650 0x5631e72c6040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:19.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.699+0000 7f674ac1b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5631e72c8650 0x5631e72c6040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47870/0 (socket says 172.21.15.26:47870) 2024-09-11T23:23:19.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.699+0000 7f674ac1b640 1 -- 172.21.15.26:0/2297994679 learned_addr learned my addr 172.21.15.26:0/2297994679 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:23:19.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.699+0000 7f674a41a640 1 --2- 172.21.15.26:0/2297994679 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5631e72b8f40 0x5631e72c55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:19.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.699+0000 7f6749c19640 1 --2- 172.21.15.26:0/2297994679 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5631e72b9860 0x5631e72c5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:19.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.700+0000 7f674ac1b640 1 -- 172.21.15.26:0/2297994679 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5631e72b9860 msgr2=0x5631e72c5b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:23:19.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.700+0000 7f674ac1b640 1 --2- 172.21.15.26:0/2297994679 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5631e72b9860 0x5631e72c5b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:19.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.700+0000 7f674ac1b640 1 -- 172.21.15.26:0/2297994679 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5631e72b8f40 msgr2=0x5631e72c55c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:23:19.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.700+0000 7f674ac1b640 1 --2- 172.21.15.26:0/2297994679 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5631e72b8f40 0x5631e72c55c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:19.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.700+0000 7f674ac1b640 1 -- 172.21.15.26:0/2297994679 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5631e6f2d580 con 0x5631e72c8650 2024-09-11T23:23:19.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.700+0000 7f674ac1b640 1 --2- 172.21.15.26:0/2297994679 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5631e72c8650 0x5631e72c6040 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f673c031040 tx=0x7f673c0bc7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c5709b17b1f6300 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:23:19.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.700+0000 7f672dffb640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:23:19.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.700+0000 7f672dffb640 1 -- 172.21.15.26:0/2297994679 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f673c0bc960 con 0x5631e72c8650 2024-09-11T23:23:19.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.700+0000 7f672dffb640 1 -- 172.21.15.26:0/2297994679 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f673c0ef070 con 0x5631e72c8650 2024-09-11T23:23:19.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.700+0000 7f674ea57740 1 -- 172.21.15.26:0/2297994679 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6738021de0 con 0x5631e72c8650 2024-09-11T23:23:19.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.700+0000 7f672dffb640 1 -- 172.21.15.26:0/2297994679 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f673c0c0d10 con 0x5631e72c8650 2024-09-11T23:23:19.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.700+0000 7f674ea57740 10 client.16428 fetch_fsmap learned FSMap version 39 2024-09-11T23:23:19.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.700+0000 7f674ea57740 1 -- 172.21.15.26:0/2297994679 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5631e6ff7960 con 0x5631e72c8650 2024-09-11T23:23:19.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.700+0000 7f672dffb640 1 -- 172.21.15.26:0/2297994679 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6738021de0 con 0x5631e72c8650 2024-09-11T23:23:19.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.701+0000 7f672dffb640 1 -- 172.21.15.26:0/2297994679 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 39) ==== 8410+0+0 (secure 0 0 0) 0x7f673c0db170 con 0x5631e72c8650 2024-09-11T23:23:19.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.701+0000 7f674ea57740 10 client.16428 fetch_fsmap finished waiting for FSMap version 39 2024-09-11T23:23:19.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.701+0000 7f674ea57740 10 client.16428 resolve_mds: validated gid 24485 aka daemon mds.b 2024-09-11T23:23:19.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.701+0000 7f674ea57740 20 client.16428 populate_metadata read hostname 'smithi026' 2024-09-11T23:23:19.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.701+0000 7f674ea57740 1 --2- 172.21.15.26:0/2297994679 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5631e72d8290 0x5631e72da680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:19.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.701+0000 7f674ea57740 4 client.16428 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:23:19.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.701+0000 7f674ea57740 1 -- 172.21.15.26:0/2297994679 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5631e722bb90 con 0x5631e72d8290 2024-09-11T23:23:19.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.701+0000 7f674a41a640 1 --2- 172.21.15.26:0/2297994679 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5631e72d8290 0x5631e72da680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:19.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.701+0000 7f674a41a640 1 --2- 172.21.15.26:0/2297994679 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5631e72d8290 0x5631e72da680 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e3b8a40bcf55098b server_cookie=d7904564a33a2bcd in_seq=0 out_seq=0 2024-09-11T23:23:19.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.701+0000 7f672dffb640 10 client.16428 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:23:19.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.702+0000 7f672dffb640 1 -- 172.21.15.26:0/2297994679 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5631e722bb90 con 0x5631e72d8290 2024-09-11T23:23:19.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.702+0000 7f672dffb640 10 client.16428 handle_command_reply: tid=0 2024-09-11T23:23:19.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.702+0000 7f674ea57740 1 client.16428 shutdown 2024-09-11T23:23:19.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.702+0000 7f674ea57740 1 -- 172.21.15.26:0/2297994679 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5631e72c8650 msgr2=0x5631e72c6040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:23:19.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.702+0000 7f674ea57740 1 --2- 172.21.15.26:0/2297994679 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5631e72c8650 0x5631e72c6040 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f673c031040 tx=0x7f673c0bc7b0 comp rx=0 tx=0).stop 2024-09-11T23:23:19.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.702+0000 7f674ea57740 1 -- 172.21.15.26:0/2297994679 shutdown_connections 2024-09-11T23:23:19.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.702+0000 7f674ea57740 1 --2- 172.21.15.26:0/2297994679 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5631e72d8290 0x5631e72da680 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:19.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.702+0000 7f674ea57740 1 --2- 172.21.15.26:0/2297994679 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5631e72c8650 0x5631e72c6040 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:19.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.702+0000 7f672dffb640 0 client.16428 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:23:19.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.702+0000 7f674ea57740 1 --2- 172.21.15.26:0/2297994679 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5631e72b9860 0x5631e72c5b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:19.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.702+0000 7f674ea57740 1 --2- 172.21.15.26:0/2297994679 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5631e72b8f40 0x5631e72c55c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:19.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.702+0000 7f674ea57740 1 -- 172.21.15.26:0/2297994679 >> 172.21.15.26:0/2297994679 conn(0x5631e72b55f0 msgr2=0x5631e72bb350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:23:19.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.702+0000 7f674ea57740 1 -- 172.21.15.26:0/2297994679 shutdown_connections 2024-09-11T23:23:19.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.703+0000 7f674ea57740 1 -- 172.21.15.26:0/2297994679 wait complete. 2024-09-11T23:23:19.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.703+0000 7f674ea57740 20 client.16428 trim_cache size 0 max 16384 2024-09-11T23:23:19.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.720+0000 7f674ea57740 1 Processor -- start 2024-09-11T23:23:19.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.720+0000 7f674ea57740 1 -- start start 2024-09-11T23:23:19.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.720+0000 7f674ea57740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5631e72b8f40 0x5631e72c5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:19.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.720+0000 7f674ea57740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5631e72b9860 0x5631e72dfe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:19.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.720+0000 7f674ea57740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5631e72c8650 0x5631e72e0350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:19.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.720+0000 7f674ea57740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f67441ab4e0 con 0x5631e72b9860 2024-09-11T23:23:19.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.720+0000 7f674ea57740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f67441ad260 con 0x5631e72c8650 2024-09-11T23:23:19.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.720+0000 7f674ea57740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f67441abc70 con 0x5631e72b8f40 2024-09-11T23:23:19.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.720+0000 7f6749c19640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5631e72b9860 0x5631e72dfe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:19.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.720+0000 7f6749c19640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5631e72b9860 0x5631e72dfe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47872/0 (socket says 172.21.15.26:47872) 2024-09-11T23:23:19.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.720+0000 7f6749c19640 1 -- 172.21.15.26:0/3709103786 learned_addr learned my addr 172.21.15.26:0/3709103786 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:23:19.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.720+0000 7f674ac1b640 1 --2- 172.21.15.26:0/3709103786 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5631e72c8650 0x5631e72e0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:19.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.720+0000 7f674a41a640 1 --2- 172.21.15.26:0/3709103786 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5631e72b8f40 0x5631e72c5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:19.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.721+0000 7f6749c19640 1 -- 172.21.15.26:0/3709103786 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5631e72b8f40 msgr2=0x5631e72c5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:23:19.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.721+0000 7f6749c19640 1 --2- 172.21.15.26:0/3709103786 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5631e72b8f40 0x5631e72c5c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:19.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.721+0000 7f6749c19640 1 -- 172.21.15.26:0/3709103786 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5631e72c8650 msgr2=0x5631e72e0350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:23:19.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.721+0000 7f6749c19640 1 --2- 172.21.15.26:0/3709103786 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5631e72c8650 0x5631e72e0350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:19.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.721+0000 7f6749c19640 1 -- 172.21.15.26:0/3709103786 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5631e72c11e0 con 0x5631e72b9860 2024-09-11T23:23:19.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.721+0000 7f6749c19640 1 --2- 172.21.15.26:0/3709103786 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5631e72b9860 0x5631e72dfe10 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f6738020a50 tx=0x7f6738020a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a61b793892fbab68 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:23:19.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.721+0000 7f672ffff640 1 -- 172.21.15.26:0/3709103786 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6738020440 con 0x5631e72b9860 2024-09-11T23:23:19.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.721+0000 7f672ffff640 1 -- 172.21.15.26:0/3709103786 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6738015040 con 0x5631e72b9860 2024-09-11T23:23:19.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.721+0000 7f672ffff640 1 -- 172.21.15.26:0/3709103786 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6738004240 con 0x5631e72b9860 2024-09-11T23:23:19.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.722+0000 7f674ea57740 1 -- 172.21.15.26:0/3709103786 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5631e72b9860 msgr2=0x5631e72dfe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:23:19.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.722+0000 7f674ea57740 1 --2- 172.21.15.26:0/3709103786 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5631e72b9860 0x5631e72dfe10 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f6738020a50 tx=0x7f6738020a80 comp rx=0 tx=0).stop 2024-09-11T23:23:19.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.722+0000 7f674ea57740 1 -- 172.21.15.26:0/3709103786 shutdown_connections 2024-09-11T23:23:19.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.722+0000 7f674ea57740 1 --2- 172.21.15.26:0/3709103786 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5631e72c8650 0x5631e72e0350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:19.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.722+0000 7f674ea57740 1 --2- 172.21.15.26:0/3709103786 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5631e72b9860 0x5631e72dfe10 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:19.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.722+0000 7f674ea57740 1 --2- 172.21.15.26:0/3709103786 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5631e72b8f40 0x5631e72c5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:19.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.722+0000 7f674ea57740 1 -- 172.21.15.26:0/3709103786 >> 172.21.15.26:0/3709103786 conn(0x5631e72b55f0 msgr2=0x5631e72c5580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:23:19.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.722+0000 7f674ea57740 1 -- 172.21.15.26:0/3709103786 shutdown_connections 2024-09-11T23:23:19.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.722+0000 7f674ea57740 1 -- 172.21.15.26:0/3709103786 wait complete. 2024-09-11T23:23:19.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.722+0000 7f674ea57740 1 -- start start 2024-09-11T23:23:19.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.722+0000 7f674ea57740 1 Processor -- start 2024-09-11T23:23:19.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.723+0000 7f674ea57740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5631e72b8f40 0x5631e72d0de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:19.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.723+0000 7f674ea57740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5631e72b9860 0x5631e72cd600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:19.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.723+0000 7f674ea57740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5631e72c8650 0x5631e72cdb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:19.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.723+0000 7f674ea57740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f67441acb90 con 0x5631e72b8f40 2024-09-11T23:23:19.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.723+0000 7f674ea57740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f67441ac400 con 0x5631e72c8650 2024-09-11T23:23:19.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.723+0000 7f674ea57740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f67441a94c0 con 0x5631e72b9860 2024-09-11T23:23:19.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.723+0000 7f674ea57740 10 client.0 _ll_register_callbacks cb 0x7f673c0b8520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:23:19.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.723+0000 7f674a41a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5631e72b8f40 0x5631e72d0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:19.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.723+0000 7f674a41a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5631e72b8f40 0x5631e72d0de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47874/0 (socket says 172.21.15.26:47874) 2024-09-11T23:23:19.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.723+0000 7f674a41a640 1 -- 172.21.15.26:0/4168840743 learned_addr learned my addr 172.21.15.26:0/4168840743 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:23:19.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.723+0000 7f674ac1b640 1 --2- 172.21.15.26:0/4168840743 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5631e72c8650 0x5631e72cdb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:19.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.723+0000 7f6749c19640 1 --2- 172.21.15.26:0/4168840743 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5631e72b9860 0x5631e72cd600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:19.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.723+0000 7f674a41a640 1 -- 172.21.15.26:0/4168840743 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5631e72b9860 msgr2=0x5631e72cd600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:23:19.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.723+0000 7f674a41a640 1 --2- 172.21.15.26:0/4168840743 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5631e72b9860 0x5631e72cd600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:19.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.723+0000 7f674a41a640 1 -- 172.21.15.26:0/4168840743 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5631e72c8650 msgr2=0x5631e72cdb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:23:19.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.723+0000 7f674a41a640 1 --2- 172.21.15.26:0/4168840743 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5631e72c8650 0x5631e72cdb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:19.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.723+0000 7f674a41a640 1 -- 172.21.15.26:0/4168840743 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5631e72ce370 con 0x5631e72b8f40 2024-09-11T23:23:19.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.724+0000 7f674a41a640 1 --2- 172.21.15.26:0/4168840743 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5631e72b8f40 0x5631e72d0de0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f6740010860 tx=0x7f6740010d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a393d0dbec9da8a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:23:19.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.724+0000 7f672dffb640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:23:19.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.724+0000 7f672dffb640 1 -- 172.21.15.26:0/4168840743 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f674000fd10 con 0x5631e72b8f40 2024-09-11T23:23:19.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.724+0000 7f672dffb640 1 -- 172.21.15.26:0/4168840743 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f6740005020 con 0x5631e72b8f40 2024-09-11T23:23:19.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.724+0000 7f674ea57740 1 -- 172.21.15.26:0/4168840743 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6738015040 con 0x5631e72b8f40 2024-09-11T23:23:19.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.724+0000 7f672dffb640 1 -- 172.21.15.26:0/4168840743 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f6740005220 con 0x5631e72b8f40 2024-09-11T23:23:19.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.724+0000 7f674ea57740 10 client.16434 fetch_fsmap learned FSMap version 39 2024-09-11T23:23:19.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.724+0000 7f674ea57740 1 -- 172.21.15.26:0/4168840743 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5631e7343ef0 con 0x5631e72b8f40 2024-09-11T23:23:19.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.724+0000 7f672dffb640 1 -- 172.21.15.26:0/4168840743 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6738015040 con 0x5631e72b8f40 2024-09-11T23:23:19.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.724+0000 7f672dffb640 1 -- 172.21.15.26:0/4168840743 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 39) ==== 8410+0+0 (secure 0 0 0) 0x7f674001c3b0 con 0x5631e72b8f40 2024-09-11T23:23:19.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.724+0000 7f674ea57740 10 client.16434 fetch_fsmap finished waiting for FSMap version 39 2024-09-11T23:23:19.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.724+0000 7f674ea57740 10 client.16434 resolve_mds: validated gid 24485 aka daemon mds.b 2024-09-11T23:23:19.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.724+0000 7f674ea57740 20 client.16434 populate_metadata read hostname 'smithi026' 2024-09-11T23:23:19.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.724+0000 7f674ea57740 1 --2- 172.21.15.26:0/4168840743 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5631e72d8290 0x5631e7346620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:19.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.725+0000 7f674ea57740 4 client.16434 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "e975f3df", "release": true, "await": true}] 2024-09-11T23:23:19.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.725+0000 7f674ea57740 1 -- 172.21.15.26:0/4168840743 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "quiesce db", "set_id": "e975f3df", "release": true, "await": true}) -- 0x5631e72260a0 con 0x5631e72d8290 2024-09-11T23:23:19.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.725+0000 7f6749c19640 1 --2- 172.21.15.26:0/4168840743 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5631e72d8290 0x5631e7346620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:19.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.725+0000 7f6749c19640 1 --2- 172.21.15.26:0/4168840743 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5631e72d8290 0x5631e7346620 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a80e0c99c49bbd3 server_cookie=a99596401122e9a4 in_seq=0 out_seq=0 2024-09-11T23:23:19.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:19.725+0000 7f672dffb640 10 client.16434 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:23:19.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:23:19 smithi136 ceph-mon[35121]: pgmap v151: 97 pgs: 97 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 3.1 MiB/s wr, 820 op/s 2024-09-11T23:23:20.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:23:19 smithi026 ceph-mon[31172]: pgmap v151: 97 pgs: 97 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 3.1 MiB/s wr, 820 op/s 2024-09-11T23:23:20.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:23:19 smithi076 ceph-mon[35296]: pgmap v151: 97 pgs: 97 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 3.1 MiB/s wr, 820 op/s 2024-09-11T23:23:20.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:20.520+0000 7f672dffb640 1 -- 172.21.15.26:0/4168840743 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5631e72260a0 con 0x5631e72d8290 2024-09-11T23:23:20.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:20.520+0000 7f672dffb640 10 client.16434 handle_command_reply: tid=0 2024-09-11T23:23:20.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:20.520+0000 7f674ea57740 1 client.16434 shutdown 2024-09-11T23:23:20.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:20.521+0000 7f674ea57740 1 -- 172.21.15.26:0/4168840743 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5631e72b8f40 msgr2=0x5631e72d0de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:23:20.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:20.521+0000 7f674ea57740 1 --2- 172.21.15.26:0/4168840743 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5631e72b8f40 0x5631e72d0de0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f6740010860 tx=0x7f6740010d20 comp rx=0 tx=0).stop 2024-09-11T23:23:20.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:20.521+0000 7f674ea57740 1 -- 172.21.15.26:0/4168840743 shutdown_connections 2024-09-11T23:23:20.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:20.521+0000 7f674ea57740 1 --2- 172.21.15.26:0/4168840743 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5631e72c8650 0x5631e72cdb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:20.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:20.521+0000 7f674ea57740 1 --2- 172.21.15.26:0/4168840743 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5631e72d8290 0x5631e7346620 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:20.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:20.521+0000 7f674ea57740 1 --2- 172.21.15.26:0/4168840743 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5631e72b9860 0x5631e72cd600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:20.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:20.521+0000 7f674ea57740 1 --2- 172.21.15.26:0/4168840743 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5631e72b8f40 0x5631e72d0de0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:20.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:20.521+0000 7f674ea57740 1 -- 172.21.15.26:0/4168840743 >> 172.21.15.26:0/4168840743 conn(0x5631e72b55f0 msgr2=0x5631e72c5da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:23:20.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:20.521+0000 7f672dffb640 0 client.16434 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:23:20.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:20.521+0000 7f674ea57740 1 -- 172.21.15.26:0/4168840743 shutdown_connections 2024-09-11T23:23:20.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:20.521+0000 7f674ea57740 1 -- 172.21.15.26:0/4168840743 wait complete. 2024-09-11T23:23:20.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:20.521+0000 7f674ea57740 20 client.16434 trim_cache size 0 max 16384 2024-09-11T23:23:20.522 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-11T23:23:20.522 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 39, 2024-09-11T23:23:20.523 INFO:teuthology.orchestra.run.smithi026.stdout: "leader": 24485, 2024-09-11T23:23:20.523 INFO:teuthology.orchestra.run.smithi026.stdout: "set_version": 20, 2024-09-11T23:23:20.523 INFO:teuthology.orchestra.run.smithi026.stdout: "sets": { 2024-09-11T23:23:20.523 INFO:teuthology.orchestra.run.smithi026.stdout: "e975f3df": { 2024-09-11T23:23:20.523 INFO:teuthology.orchestra.run.smithi026.stdout: "version": 20, 2024-09-11T23:23:20.523 INFO:teuthology.orchestra.run.smithi026.stdout: "age_ref": 0.0, 2024-09-11T23:23:20.523 INFO:teuthology.orchestra.run.smithi026.stdout: "state": { 2024-09-11T23:23:20.523 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "RELEASED", 2024-09-11T23:23:20.523 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.0 2024-09-11T23:23:20.523 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:23:20.523 INFO:teuthology.orchestra.run.smithi026.stdout: "timeout": 90.0, 2024-09-11T23:23:20.523 INFO:teuthology.orchestra.run.smithi026.stdout: "expiration": 140.2, 2024-09-11T23:23:20.523 INFO:teuthology.orchestra.run.smithi026.stdout: "members": { 2024-09-11T23:23:20.523 INFO:teuthology.orchestra.run.smithi026.stdout: "file:/": { 2024-09-11T23:23:20.523 INFO:teuthology.orchestra.run.smithi026.stdout: "excluded": false, 2024-09-11T23:23:20.524 INFO:teuthology.orchestra.run.smithi026.stdout: "state": { 2024-09-11T23:23:20.524 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "RELEASED", 2024-09-11T23:23:20.524 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.0 2024-09-11T23:23:20.524 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:23:20.524 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:23:20.524 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:23:20.524 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:23:20.524 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:23:20.524 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-11T23:23:20.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:20.522+0000 7f674c67e640 1 -- 172.21.15.26:0/286841578 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f67300a2cf0 msgr2=0x7f67300a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:23:20.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:20.522+0000 7f674c67e640 1 --2- 172.21.15.26:0/286841578 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f67300a2cf0 0x7f67300a51b0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f6738023040 tx=0x7f673801f000 comp rx=0 tx=0).stop 2024-09-11T23:23:20.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:20.522+0000 7f674c67e640 1 -- 172.21.15.26:0/286841578 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6744108300 msgr2=0x7f674419cec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:23:20.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:20.522+0000 7f674c67e640 1 --2- 172.21.15.26:0/286841578 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6744108300 0x7f674419cec0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f673c009ca0 tx=0x7f673c02f420 comp rx=0 tx=0).stop 2024-09-11T23:23:20.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:20.522+0000 7f674c67e640 1 -- 172.21.15.26:0/286841578 shutdown_connections 2024-09-11T23:23:20.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:20.522+0000 7f674c67e640 1 --2- 172.21.15.26:0/286841578 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f67300a2cf0 0x7f67300a51b0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:20.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:20.522+0000 7f674c67e640 1 --2- 172.21.15.26:0/286841578 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6744103780 0x7f6744198b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:20.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:20.522+0000 7f674c67e640 1 --2- 172.21.15.26:0/286841578 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6744108300 0x7f674419cec0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:20.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:20.522+0000 7f674c67e640 1 --2- 172.21.15.26:0/286841578 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6744102dd0 0x7f67441985f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:20.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:20.522+0000 7f674c67e640 1 -- 172.21.15.26:0/286841578 >> 172.21.15.26:0/286841578 conn(0x7f67440fc7d0 msgr2=0x7f67440fe4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:23:20.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:20.522+0000 7f674c67e640 1 -- 172.21.15.26:0/286841578 shutdown_connections 2024-09-11T23:23:20.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:23:20.522+0000 7f674c67e640 1 -- 172.21.15.26:0/286841578 wait complete. 2024-09-11T23:23:21.434 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'e975f3df', total seconds elapsed: 25.7 2024-09-11T23:23:21.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:23:21 smithi136 ceph-mon[35121]: pgmap v152: 97 pgs: 97 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 2.7 MiB/s wr, 931 op/s 2024-09-11T23:23:22.000 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:23:21 smithi026 ceph-mon[31172]: pgmap v152: 97 pgs: 97 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 2.7 MiB/s wr, 931 op/s 2024-09-11T23:23:22.052 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:23:21 smithi076 ceph-mon[35296]: pgmap v152: 97 pgs: 97 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 2.7 MiB/s wr, 931 op/s 2024-09-11T23:23:23.578 INFO:tasks.workunit.client.0.smithi026.stdout: 375 1667788 762 1145602 514 318228 549 2024-09-11T23:23:23.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:23:23 smithi136 ceph-mon[35121]: pgmap v153: 97 pgs: 97 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 883 KiB/s wr, 699 op/s 2024-09-11T23:23:24.000 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:23:23 smithi026 ceph-mon[31172]: pgmap v153: 97 pgs: 97 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 883 KiB/s wr, 699 op/s 2024-09-11T23:23:24.052 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:23:23 smithi076 ceph-mon[35296]: pgmap v153: 97 pgs: 97 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 883 KiB/s wr, 699 op/s 2024-09-11T23:23:24.922 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:24.921+0000 7f037df5f640 1 -- 172.21.15.26:0/3765746961 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f03781091f0 msgr2=0x7f0378109630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:23:24.922 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:24.921+0000 7f037df5f640 1 --2- 172.21.15.26:0/3765746961 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f03781091f0 0x7f0378109630 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f0370009ca0 tx=0x7f037002a8f0 comp rx=0 tx=0).stop 2024-09-11T23:23:24.922 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:24.921+0000 7f037df5f640 1 -- 172.21.15.26:0/3765746961 shutdown_connections 2024-09-11T23:23:24.922 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:24.921+0000 7f037df5f640 1 --2- 172.21.15.26:0/3765746961 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f03781091f0 0x7f0378109630 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:24.922 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:24.921+0000 7f037df5f640 1 --2- 172.21.15.26:0/3765746961 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0378109b70 0x7f037807ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:24.922 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:24.921+0000 7f037df5f640 1 --2- 172.21.15.26:0/3765746961 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f037810d610 0x7f037810d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:24.922 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:24.921+0000 7f037df5f640 1 -- 172.21.15.26:0/3765746961 >> 172.21.15.26:0/3765746961 conn(0x7f037806cc60 msgr2=0x7f037806d070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:23:24.922 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:24.922+0000 7f037df5f640 1 -- 172.21.15.26:0/3765746961 shutdown_connections 2024-09-11T23:23:24.923 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:24.922+0000 7f037df5f640 1 -- 172.21.15.26:0/3765746961 wait complete. 2024-09-11T23:23:24.924 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:24.923+0000 7f037df5f640 1 Processor -- start 2024-09-11T23:23:24.924 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:24.923+0000 7f037df5f640 1 -- start start 2024-09-11T23:23:24.924 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:24.923+0000 7f037df5f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f03781091f0 0x7f0378080810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:24.924 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:24.923+0000 7f037df5f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0378109b70 0x7f0378080d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:24.924 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:24.923+0000 7f037df5f640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f037810d610 0x7f0378081290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:24.924 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:24.923+0000 7f037df5f640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f037807e5e0 con 0x7f0378109b70 2024-09-11T23:23:24.924 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:24.923+0000 7f037df5f640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f037807e460 con 0x7f03781091f0 2024-09-11T23:23:24.924 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:24.923+0000 7f037df5f640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f037807e760 con 0x7f037810d610 2024-09-11T23:23:24.924 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:24.924+0000 7f03777fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f03781091f0 0x7f0378080810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:24.925 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:24.924+0000 7f03777fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f03781091f0 0x7f0378080810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:44388/0 (socket says 172.21.15.26:44388) 2024-09-11T23:23:24.925 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:24.924+0000 7f03777fe640 1 -- 172.21.15.26:0/740456954 learned_addr learned my addr 172.21.15.26:0/740456954 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:23:24.925 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:24.924+0000 7f0377fff640 1 --2- 172.21.15.26:0/740456954 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f037810d610 0x7f0378081290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:24.925 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:24.924+0000 7f0376ffd640 1 --2- 172.21.15.26:0/740456954 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0378109b70 0x7f0378080d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:24.925 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:24.924+0000 7f03777fe640 1 -- 172.21.15.26:0/740456954 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f037810d610 msgr2=0x7f0378081290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:23:24.925 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:24.924+0000 7f03777fe640 1 --2- 172.21.15.26:0/740456954 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f037810d610 0x7f0378081290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:24.925 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:24.924+0000 7f03777fe640 1 -- 172.21.15.26:0/740456954 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0378109b70 msgr2=0x7f0378080d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:23:24.926 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:24.924+0000 7f03777fe640 1 --2- 172.21.15.26:0/740456954 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0378109b70 0x7f0378080d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:24.926 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:24.924+0000 7f03777fe640 1 -- 172.21.15.26:0/740456954 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03781c03b0 con 0x7f03781091f0 2024-09-11T23:23:24.926 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:24.924+0000 7f0376ffd640 1 --2- 172.21.15.26:0/740456954 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0378109b70 0x7f0378080d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:23:24.926 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:24.925+0000 7f03777fe640 1 --2- 172.21.15.26:0/740456954 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f03781091f0 0x7f0378080810 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f036800b450 tx=0x7f036800b910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:23:24.945 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:24.945+0000 7f0374ff9640 1 -- 172.21.15.26:0/740456954 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0368015020 con 0x7f03781091f0 2024-09-11T23:23:24.945 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:24.945+0000 7f0374ff9640 1 -- 172.21.15.26:0/740456954 <== mon.2 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0368011d70 con 0x7f03781091f0 2024-09-11T23:23:24.945 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:24.945+0000 7f037df5f640 1 -- 172.21.15.26:0/740456954 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03781c0580 con 0x7f03781091f0 2024-09-11T23:23:24.945 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:24.945+0000 7f037df5f640 1 -- 172.21.15.26:0/740456954 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03781c09f0 con 0x7f03781091f0 2024-09-11T23:23:24.946 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:24.946+0000 7f0374ff9640 1 -- 172.21.15.26:0/740456954 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f036800f340 con 0x7f03781091f0 2024-09-11T23:23:24.952 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:24.953+0000 7f0374ff9640 1 -- 172.21.15.26:0/740456954 <== mon.2 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f0368018020 con 0x7f03781091f0 2024-09-11T23:23:24.954 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:24.954+0000 7f0374ff9640 1 --2- 172.21.15.26:0/740456954 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f03600a2f00 0x7f03600a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:24.954 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:24.954+0000 7f0374ff9640 1 -- 172.21.15.26:0/740456954 <== mon.2 v2:172.21.15.76:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f03680cfe00 con 0x7f03781091f0 2024-09-11T23:23:24.954 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:24.954+0000 7f0376ffd640 1 --2- 172.21.15.26:0/740456954 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f03600a2f00 0x7f03600a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:24.955 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:24.956+0000 7f0376ffd640 1 --2- 172.21.15.26:0/740456954 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f03600a2f00 0x7f03600a53c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f0378082060 tx=0x7f037002f4b0 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:23:24.961 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:24.960+0000 7f0380dd5740 1 Processor -- start 2024-09-11T23:23:24.961 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:24.960+0000 7f0380dd5740 1 -- start start 2024-09-11T23:23:24.961 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:24.960+0000 7f0380dd5740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563fa5bfecc0 0x563fa5bff0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:24.961 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:24.960+0000 7f0380dd5740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563fa5bff5e0 0x563fa5c09940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:24.961 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:24.960+0000 7f0380dd5740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563fa5c09e80 0x563fa5c0c270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:24.961 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:24.961+0000 7f0380dd5740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x563fa5b632f0 con 0x563fa5bff5e0 2024-09-11T23:23:24.961 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:24.961+0000 7f0380dd5740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x563fa5b0b270 con 0x563fa5c09e80 2024-09-11T23:23:24.961 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:24.961+0000 7f0380dd5740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x563fa5b5f2b0 con 0x563fa5bfecc0 2024-09-11T23:23:24.961 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:24.961+0000 7f03777fe640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563fa5bfecc0 0x563fa5bff0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:24.961 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:24.961+0000 7f0377fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563fa5c09e80 0x563fa5c0c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:24.962 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:24.961+0000 7f03777fe640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563fa5bfecc0 0x563fa5bff0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:46484/0 (socket says 172.21.15.26:46484) 2024-09-11T23:23:24.962 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:24.961+0000 7f03777fe640 1 -- 172.21.15.26:0/3913196855 learned_addr learned my addr 172.21.15.26:0/3913196855 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:23:24.962 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:24.961+0000 7f0376ffd640 1 --2- 172.21.15.26:0/3913196855 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563fa5bff5e0 0x563fa5c09940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:24.962 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:24.961+0000 7f03777fe640 1 -- 172.21.15.26:0/3913196855 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563fa5c09e80 msgr2=0x563fa5c0c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:23:24.962 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:24.961+0000 7f03777fe640 1 --2- 172.21.15.26:0/3913196855 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563fa5c09e80 0x563fa5c0c270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:24.963 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:24.961+0000 7f03777fe640 1 -- 172.21.15.26:0/3913196855 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563fa5bff5e0 msgr2=0x563fa5c09940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:23:24.963 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:24.961+0000 7f03777fe640 1 --2- 172.21.15.26:0/3913196855 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563fa5bff5e0 0x563fa5c09940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:24.963 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:24.961+0000 7f03777fe640 1 -- 172.21.15.26:0/3913196855 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563fa5b0a340 con 0x563fa5bfecc0 2024-09-11T23:23:24.963 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:24.961+0000 7f0376ffd640 1 --2- 172.21.15.26:0/3913196855 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563fa5bff5e0 0x563fa5c09940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:23:24.963 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:24.962+0000 7f03777fe640 1 --2- 172.21.15.26:0/3913196855 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563fa5bfecc0 0x563fa5bff0a0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f03680a4b20 tx=0x7f036809cae0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=53c11219021362bc server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:23:24.997 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:24.997+0000 7f035dffb640 1 -- 172.21.15.26:0/3913196855 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03680b5070 con 0x563fa5bfecc0 2024-09-11T23:23:24.997 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:24.997+0000 7f035dffb640 1 -- 172.21.15.26:0/3913196855 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f03680a5050 con 0x563fa5bfecc0 2024-09-11T23:23:24.998 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:24.998+0000 7f0380dd5740 1 -- 172.21.15.26:0/3913196855 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563fa5bfecc0 msgr2=0x563fa5bff0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:23:24.998 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:24.998+0000 7f0380dd5740 1 --2- 172.21.15.26:0/3913196855 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563fa5bfecc0 0x563fa5bff0a0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f03680a4b20 tx=0x7f036809cae0 comp rx=0 tx=0).stop 2024-09-11T23:23:24.998 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:24.998+0000 7f0380dd5740 1 -- 172.21.15.26:0/3913196855 shutdown_connections 2024-09-11T23:23:24.998 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:24.998+0000 7f0380dd5740 1 --2- 172.21.15.26:0/3913196855 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563fa5c09e80 0x563fa5c0c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:24.998 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:24.998+0000 7f0380dd5740 1 --2- 172.21.15.26:0/3913196855 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563fa5bff5e0 0x563fa5c09940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:24.998 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:24.998+0000 7f0380dd5740 1 --2- 172.21.15.26:0/3913196855 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563fa5bfecc0 0x563fa5bff0a0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:24.998 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:24.998+0000 7f0380dd5740 1 -- 172.21.15.26:0/3913196855 >> 172.21.15.26:0/3913196855 conn(0x563fa5bfb230 msgr2=0x563fa5bfb610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:23:24.999 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:25.000+0000 7f0380dd5740 1 -- 172.21.15.26:0/3913196855 shutdown_connections 2024-09-11T23:23:25.001 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:25.000+0000 7f0380dd5740 1 -- 172.21.15.26:0/3913196855 wait complete. 2024-09-11T23:23:25.001 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:25.000+0000 7f0380dd5740 1 -- start start 2024-09-11T23:23:25.001 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:25.001+0000 7f0380dd5740 1 Processor -- start 2024-09-11T23:23:25.002 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:25.001+0000 7f0380dd5740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563fa5bfecc0 0x563fa5c06d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:25.002 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:25.001+0000 7f0380dd5740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563fa5bff5e0 0x563fa5c072d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:25.002 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:25.002+0000 7f0380dd5740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563fa5c09e80 0x563fa5c07810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:25.003 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:25.002+0000 7f0380dd5740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x563fa5c0caa0 con 0x563fa5bff5e0 2024-09-11T23:23:25.003 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:25.002+0000 7f0380dd5740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x563fa5aec290 con 0x563fa5bfecc0 2024-09-11T23:23:25.003 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:25.002+0000 7f0380dd5740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x563fa5c0cc20 con 0x563fa5c09e80 2024-09-11T23:23:25.003 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:25.002+0000 7f0380dd5740 10 client.0 _ll_register_callbacks cb 0x563fa5b1d140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:23:25.003 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:25.002+0000 7f0376ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563fa5bff5e0 0x563fa5c072d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:25.003 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:25.002+0000 7f03777fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563fa5bfecc0 0x563fa5c06d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:25.003 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:25.002+0000 7f03777fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563fa5bfecc0 0x563fa5c06d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:44416/0 (socket says 172.21.15.26:44416) 2024-09-11T23:23:25.003 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:25.002+0000 7f03777fe640 1 -- 172.21.15.26:0/2363919708 learned_addr learned my addr 172.21.15.26:0/2363919708 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:23:25.003 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:25.002+0000 7f0377fff640 1 --2- 172.21.15.26:0/2363919708 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563fa5c09e80 0x563fa5c07810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:25.003 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:25.003+0000 7f03777fe640 1 -- 172.21.15.26:0/2363919708 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563fa5c09e80 msgr2=0x563fa5c07810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:23:25.003 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:25.003+0000 7f03777fe640 1 --2- 172.21.15.26:0/2363919708 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563fa5c09e80 0x563fa5c07810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:25.003 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:25.003+0000 7f03777fe640 1 -- 172.21.15.26:0/2363919708 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563fa5bff5e0 msgr2=0x563fa5c072d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:23:25.004 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:25.003+0000 7f03777fe640 1 --2- 172.21.15.26:0/2363919708 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563fa5bff5e0 0x563fa5c072d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:25.004 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:25.003+0000 7f03777fe640 1 -- 172.21.15.26:0/2363919708 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563fa5876210 con 0x563fa5bfecc0 2024-09-11T23:23:25.004 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:25.003+0000 7f0376ffd640 1 --2- 172.21.15.26:0/2363919708 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563fa5bff5e0 0x563fa5c072d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:23:25.004 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:25.003+0000 7f03777fe640 1 --2- 172.21.15.26:0/2363919708 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563fa5bfecc0 0x563fa5c06d90 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f0368010a00 tx=0x7f0368004700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=20357fe7afeb13c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:23:25.012 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:25.012+0000 7f0356ffd640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-11T23:23:25.012 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:25.012+0000 7f0356ffd640 1 -- 172.21.15.26:0/2363919708 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03680b5040 con 0x563fa5bfecc0 2024-09-11T23:23:25.012 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:25.012+0000 7f0380dd5740 1 -- 172.21.15.26:0/2363919708 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f03680a5050 con 0x563fa5bfecc0 2024-09-11T23:23:25.012 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:25.012+0000 7f0356ffd640 1 -- 172.21.15.26:0/2363919708 <== mon.2 v2:172.21.15.76:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f036809ba30 con 0x563fa5bfecc0 2024-09-11T23:23:25.013 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:25.012+0000 7f0356ffd640 1 -- 172.21.15.26:0/2363919708 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f0368011890 con 0x563fa5bfecc0 2024-09-11T23:23:25.013 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:25.012+0000 7f0380dd5740 10 client.26039 fetch_fsmap learned FSMap version 39 2024-09-11T23:23:25.013 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:25.012+0000 7f0380dd5740 1 -- 172.21.15.26:0/2363919708 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x563fa5942620 con 0x563fa5bfecc0 2024-09-11T23:23:25.013 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:25.013+0000 7f0356ffd640 1 -- 172.21.15.26:0/2363919708 <== mon.2 v2:172.21.15.76:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f03680a5050 con 0x563fa5bfecc0 2024-09-11T23:23:25.014 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:25.015+0000 7f0356ffd640 1 -- 172.21.15.26:0/2363919708 <== mon.2 v2:172.21.15.76:3300/0 5 ==== fsmap(e 39) ==== 8410+0+0 (secure 0 0 0) 0x7f03680a7030 con 0x563fa5bfecc0 2024-09-11T23:23:25.015 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:25.015+0000 7f0380dd5740 10 client.26039 fetch_fsmap finished waiting for FSMap version 39 2024-09-11T23:23:25.015 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:25.015+0000 7f0380dd5740 10 client.26039 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:23:25.015 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:25.015+0000 7f0380dd5740 20 client.26039 populate_metadata read hostname 'smithi026' 2024-09-11T23:23:25.016 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:25.015+0000 7f0380dd5740 1 --2- 172.21.15.26:0/2363919708 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x563fa5c22500 0x563fa5c248f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:25.016 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:25.015+0000 7f0380dd5740 4 client.26039 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:23:25.016 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:25.015+0000 7f0380dd5740 1 -- 172.21.15.26:0/2363919708 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563fa5b70a20 con 0x563fa5c22500 2024-09-11T23:23:25.016 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:25.015+0000 7f0376ffd640 1 --2- 172.21.15.26:0/2363919708 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x563fa5c22500 0x563fa5c248f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:25.016 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:25.016+0000 7f0376ffd640 1 --2- 172.21.15.26:0/2363919708 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x563fa5c22500 0x563fa5c248f0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=438e09c93611fafe server_cookie=39be40bf0c2132bd in_seq=0 out_seq=0 2024-09-11T23:23:25.022 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:25.023+0000 7f0356ffd640 10 client.26039 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:23:25.023 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:25.023+0000 7f0356ffd640 1 -- 172.21.15.26:0/2363919708 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563fa5b70a20 con 0x563fa5c22500 2024-09-11T23:23:25.023 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:25.023+0000 7f0356ffd640 10 client.26039 handle_command_reply: tid=0 2024-09-11T23:23:25.023 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:25.023+0000 7f0380dd5740 1 client.26039 shutdown 2024-09-11T23:23:25.025 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:25.025+0000 7f0380dd5740 1 -- 172.21.15.26:0/2363919708 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563fa5bfecc0 msgr2=0x563fa5c06d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:23:25.025 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:25.025+0000 7f0380dd5740 1 --2- 172.21.15.26:0/2363919708 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563fa5bfecc0 0x563fa5c06d90 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f0368010a00 tx=0x7f0368004700 comp rx=0 tx=0).stop 2024-09-11T23:23:25.025 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:25.025+0000 7f0380dd5740 1 -- 172.21.15.26:0/2363919708 shutdown_connections 2024-09-11T23:23:25.025 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:25.025+0000 7f0380dd5740 1 --2- 172.21.15.26:0/2363919708 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x563fa5c22500 0x563fa5c248f0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:25.025 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:25.025+0000 7f0380dd5740 1 --2- 172.21.15.26:0/2363919708 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563fa5c09e80 0x563fa5c07810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:25.026 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:25.025+0000 7f0380dd5740 1 --2- 172.21.15.26:0/2363919708 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563fa5bff5e0 0x563fa5c072d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:25.026 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:25.025+0000 7f0380dd5740 1 --2- 172.21.15.26:0/2363919708 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563fa5bfecc0 0x563fa5c06d90 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:25.026 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:25.025+0000 7f0356ffd640 0 client.26039 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:23:25.026 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:25.025+0000 7f0380dd5740 1 -- 172.21.15.26:0/2363919708 >> 172.21.15.26:0/2363919708 conn(0x563fa5bfb230 msgr2=0x563fa5c00f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:23:25.029 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:25.029+0000 7f0380dd5740 1 -- 172.21.15.26:0/2363919708 shutdown_connections 2024-09-11T23:23:25.029 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:25.029+0000 7f0380dd5740 1 -- 172.21.15.26:0/2363919708 wait complete. 2024-09-11T23:23:25.029 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:25.029+0000 7f0380dd5740 20 client.26039 trim_cache size 0 max 16384 2024-09-11T23:23:25.086 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:25.086+0000 7f0380dd5740 1 Processor -- start 2024-09-11T23:23:25.090 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:25.090+0000 7f0380dd5740 1 -- start start 2024-09-11T23:23:25.091 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:25.091+0000 7f0380dd5740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563fa5bfecc0 0x563fa5c03fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:25.091 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:25.091+0000 7f0380dd5740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563fa5bff5e0 0x563fa5c04510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:25.091 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:25.091+0000 7f0380dd5740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563fa5c09e80 0x563fa5c085f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:25.091 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:25.091+0000 7f0380dd5740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f036400e730 con 0x563fa5c09e80 2024-09-11T23:23:25.092 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:25.092+0000 7f03777fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563fa5bfecc0 0x563fa5c03fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:25.092 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:25.092+0000 7f0380dd5740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f03640104b0 con 0x563fa5bfecc0 2024-09-11T23:23:25.092 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:25.092+0000 7f0380dd5740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f036400eec0 con 0x563fa5bff5e0 2024-09-11T23:23:25.093 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:25.092+0000 7f03777fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563fa5bfecc0 0x563fa5c03fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:44430/0 (socket says 172.21.15.26:44430) 2024-09-11T23:23:25.093 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:25.092+0000 7f03777fe640 1 -- 172.21.15.26:0/60246281 learned_addr learned my addr 172.21.15.26:0/60246281 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:23:25.093 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:25.092+0000 7f0376ffd640 1 --2- 172.21.15.26:0/60246281 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563fa5bff5e0 0x563fa5c04510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:25.093 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:25.092+0000 7f0377fff640 1 --2- 172.21.15.26:0/60246281 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563fa5c09e80 0x563fa5c085f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:25.093 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:25.092+0000 7f03777fe640 1 -- 172.21.15.26:0/60246281 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563fa5bff5e0 msgr2=0x563fa5c04510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:23:25.093 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:25.092+0000 7f03777fe640 1 --2- 172.21.15.26:0/60246281 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563fa5bff5e0 0x563fa5c04510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:25.093 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:25.092+0000 7f03777fe640 1 -- 172.21.15.26:0/60246281 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563fa5c09e80 msgr2=0x563fa5c085f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:23:25.093 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:25.092+0000 7f03777fe640 1 --2- 172.21.15.26:0/60246281 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563fa5c09e80 0x563fa5c085f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:25.093 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:25.092+0000 7f03777fe640 1 -- 172.21.15.26:0/60246281 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563fa5bfc2e0 con 0x563fa5bfecc0 2024-09-11T23:23:25.093 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:25.092+0000 7f0376ffd640 1 --2- 172.21.15.26:0/60246281 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563fa5bff5e0 0x563fa5c04510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:23:25.093 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:25.092+0000 7f0377fff640 1 --2- 172.21.15.26:0/60246281 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563fa5c09e80 0x563fa5c085f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:23:25.093 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:25.093+0000 7f03777fe640 1 --2- 172.21.15.26:0/60246281 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563fa5bfecc0 0x563fa5c03fd0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f036809a690 tx=0x7f0368004770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ed385826e76935bd server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:23:25.096 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:25.096+0000 7f035d7fa640 1 -- 172.21.15.26:0/60246281 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03680b5040 con 0x563fa5bfecc0 2024-09-11T23:23:25.096 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:25.096+0000 7f035d7fa640 1 -- 172.21.15.26:0/60246281 <== mon.2 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f03680a5050 con 0x563fa5bfecc0 2024-09-11T23:23:25.097 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:25.097+0000 7f0380dd5740 1 -- 172.21.15.26:0/60246281 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563fa5bfecc0 msgr2=0x563fa5c03fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:23:25.097 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:25.097+0000 7f0380dd5740 1 --2- 172.21.15.26:0/60246281 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563fa5bfecc0 0x563fa5c03fd0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f036809a690 tx=0x7f0368004770 comp rx=0 tx=0).stop 2024-09-11T23:23:25.098 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:25.098+0000 7f0380dd5740 1 -- 172.21.15.26:0/60246281 shutdown_connections 2024-09-11T23:23:25.098 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:25.098+0000 7f0380dd5740 1 --2- 172.21.15.26:0/60246281 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563fa5c09e80 0x563fa5c085f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:25.098 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:25.098+0000 7f0380dd5740 1 --2- 172.21.15.26:0/60246281 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563fa5bff5e0 0x563fa5c04510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:25.098 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:25.098+0000 7f0380dd5740 1 --2- 172.21.15.26:0/60246281 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563fa5bfecc0 0x563fa5c03fd0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:25.098 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:25.098+0000 7f0380dd5740 1 -- 172.21.15.26:0/60246281 >> 172.21.15.26:0/60246281 conn(0x563fa5bfb230 msgr2=0x563fa5c02020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:23:25.105 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:25.106+0000 7f0380dd5740 1 -- 172.21.15.26:0/60246281 shutdown_connections 2024-09-11T23:23:25.106 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:25.106+0000 7f0380dd5740 1 -- 172.21.15.26:0/60246281 wait complete. 2024-09-11T23:23:25.107 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:25.106+0000 7f0380dd5740 1 -- start start 2024-09-11T23:23:25.107 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:25.107+0000 7f0380dd5740 1 Processor -- start 2024-09-11T23:23:25.108 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:25.107+0000 7f0380dd5740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563fa5bfecc0 0x563fa5c27ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:25.108 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:25.107+0000 7f0380dd5740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563fa5bff5e0 0x563fa5c281e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:25.108 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:25.107+0000 7f0380dd5740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563fa5c09e80 0x563fa5c25230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:25.108 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:25.107+0000 7f0380dd5740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f036400fde0 con 0x563fa5bff5e0 2024-09-11T23:23:25.108 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:25.107+0000 7f0380dd5740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f036400f650 con 0x563fa5bfecc0 2024-09-11T23:23:25.108 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:25.107+0000 7f0380dd5740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f036400c7c0 con 0x563fa5c09e80 2024-09-11T23:23:25.108 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:25.107+0000 7f0380dd5740 10 client.0 _ll_register_callbacks cb 0x7f03680b3070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:23:25.108 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:25.108+0000 7f03777fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563fa5bfecc0 0x563fa5c27ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:25.108 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:25.108+0000 7f03777fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563fa5bfecc0 0x563fa5c27ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:44442/0 (socket says 172.21.15.26:44442) 2024-09-11T23:23:25.108 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:25.108+0000 7f03777fe640 1 -- 172.21.15.26:0/3743468418 learned_addr learned my addr 172.21.15.26:0/3743468418 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:23:25.109 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:25.108+0000 7f0376ffd640 1 --2- 172.21.15.26:0/3743468418 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563fa5bff5e0 0x563fa5c281e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:25.109 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:25.108+0000 7f0377fff640 1 --2- 172.21.15.26:0/3743468418 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563fa5c09e80 0x563fa5c25230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:25.109 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:25.108+0000 7f03777fe640 1 -- 172.21.15.26:0/3743468418 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563fa5c09e80 msgr2=0x563fa5c25230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:23:25.109 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:25.108+0000 7f03777fe640 1 --2- 172.21.15.26:0/3743468418 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563fa5c09e80 0x563fa5c25230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:25.109 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:25.108+0000 7f03777fe640 1 -- 172.21.15.26:0/3743468418 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563fa5bff5e0 msgr2=0x563fa5c281e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:23:25.109 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:25.108+0000 7f03777fe640 1 --2- 172.21.15.26:0/3743468418 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563fa5bff5e0 0x563fa5c281e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:25.109 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:25.108+0000 7f03777fe640 1 -- 172.21.15.26:0/3743468418 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563fa5c06b60 con 0x563fa5bfecc0 2024-09-11T23:23:25.109 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:25.108+0000 7f0376ffd640 1 --2- 172.21.15.26:0/3743468418 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563fa5bff5e0 0x563fa5c281e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:23:25.109 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:25.108+0000 7f03777fe640 1 --2- 172.21.15.26:0/3743468418 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563fa5bfecc0 0x563fa5c27ca0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f036809c0e0 tx=0x7f036809cae0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f216cf21a699cea4 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:23:25.119 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:25.119+0000 7f0356ffd640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-11T23:23:25.119 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:25.119+0000 7f0356ffd640 1 -- 172.21.15.26:0/3743468418 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03680b5040 con 0x563fa5bfecc0 2024-09-11T23:23:25.119 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:25.119+0000 7f0380dd5740 1 -- 172.21.15.26:0/3743468418 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f03680a5050 con 0x563fa5bfecc0 2024-09-11T23:23:25.119 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:25.119+0000 7f0356ffd640 1 -- 172.21.15.26:0/3743468418 <== mon.2 v2:172.21.15.76:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f0368098130 con 0x563fa5bfecc0 2024-09-11T23:23:25.119 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:25.119+0000 7f0356ffd640 1 -- 172.21.15.26:0/3743468418 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f036809b530 con 0x563fa5bfecc0 2024-09-11T23:23:25.119 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:25.119+0000 7f0380dd5740 10 client.26045 fetch_fsmap learned FSMap version 39 2024-09-11T23:23:25.119 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:25.119+0000 7f0380dd5740 1 -- 172.21.15.26:0/3743468418 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x563fa5c06b60 con 0x563fa5bfecc0 2024-09-11T23:23:25.119 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:25.119+0000 7f0356ffd640 1 -- 172.21.15.26:0/3743468418 <== mon.2 v2:172.21.15.76:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f03680a5050 con 0x563fa5bfecc0 2024-09-11T23:23:25.121 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:25.121+0000 7f0356ffd640 1 -- 172.21.15.26:0/3743468418 <== mon.2 v2:172.21.15.76:3300/0 5 ==== fsmap(e 39) ==== 8410+0+0 (secure 0 0 0) 0x7f03680a7070 con 0x563fa5bfecc0 2024-09-11T23:23:25.121 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:25.121+0000 7f0380dd5740 10 client.26045 fetch_fsmap finished waiting for FSMap version 39 2024-09-11T23:23:25.121 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:25.121+0000 7f0380dd5740 10 client.26045 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:23:25.121 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:25.121+0000 7f0380dd5740 20 client.26045 populate_metadata read hostname 'smithi026' 2024-09-11T23:23:25.121 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:25.121+0000 7f0380dd5740 1 --2- 172.21.15.26:0/3743468418 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x563fa5c0dc10 0x563fa5c10000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:25.121 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:25.121+0000 7f0380dd5740 4 client.26045 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T23:23:25.121 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:25.121+0000 7f0380dd5740 1 -- 172.21.15.26:0/3743468418 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563fa5b6ad70 con 0x563fa5c0dc10 2024-09-11T23:23:25.121 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:25.121+0000 7f0376ffd640 1 --2- 172.21.15.26:0/3743468418 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x563fa5c0dc10 0x563fa5c10000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:25.122 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:25.122+0000 7f0356ffd640 10 client.26045 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:23:25.122 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:25.122+0000 7f0376ffd640 1 --2- 172.21.15.26:0/3743468418 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x563fa5c0dc10 0x563fa5c10000 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=38db34db1b6a15ec server_cookie=baa76f44a2594599 in_seq=0 out_seq=0 2024-09-11T23:23:25.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:23:25 smithi136 ceph-mon[35121]: pgmap v154: 97 pgs: 97 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 1.8 MiB/s wr, 723 op/s 2024-09-11T23:23:26.000 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:23:25 smithi026 ceph-mon[31172]: pgmap v154: 97 pgs: 97 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 1.8 MiB/s wr, 723 op/s 2024-09-11T23:23:26.052 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:23:25 smithi076 ceph-mon[35296]: pgmap v154: 97 pgs: 97 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 1.8 MiB/s wr, 723 op/s 2024-09-11T23:23:27.006 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:27.006+0000 7f0356ffd640 1 -- 172.21.15.26:0/3743468418 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563fa5b6ad70 con 0x563fa5c0dc10 2024-09-11T23:23:27.007 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:27.006+0000 7f0356ffd640 10 client.26045 handle_command_reply: tid=0 2024-09-11T23:23:27.007 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:27.007+0000 7f0380dd5740 1 client.26045 shutdown 2024-09-11T23:23:27.008 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:27.007+0000 7f0380dd5740 1 -- 172.21.15.26:0/3743468418 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563fa5bfecc0 msgr2=0x563fa5c27ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:23:27.008 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:27.007+0000 7f0380dd5740 1 --2- 172.21.15.26:0/3743468418 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563fa5bfecc0 0x563fa5c27ca0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f036809c0e0 tx=0x7f036809cae0 comp rx=0 tx=0).stop 2024-09-11T23:23:27.008 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:27.007+0000 7f0380dd5740 1 -- 172.21.15.26:0/3743468418 shutdown_connections 2024-09-11T23:23:27.008 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:27.007+0000 7f0380dd5740 1 --2- 172.21.15.26:0/3743468418 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x563fa5c0dc10 0x563fa5c10000 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:27.008 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:27.007+0000 7f0380dd5740 1 --2- 172.21.15.26:0/3743468418 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563fa5c09e80 0x563fa5c25230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:27.008 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:27.007+0000 7f0380dd5740 1 --2- 172.21.15.26:0/3743468418 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563fa5bff5e0 0x563fa5c281e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:27.008 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:27.007+0000 7f0356ffd640 0 client.26045 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:23:27.008 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:27.007+0000 7f0380dd5740 1 --2- 172.21.15.26:0/3743468418 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563fa5bfecc0 0x563fa5c27ca0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:27.008 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:27.007+0000 7f0380dd5740 1 -- 172.21.15.26:0/3743468418 >> 172.21.15.26:0/3743468418 conn(0x563fa5bfb230 msgr2=0x563fa5bf9a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:23:27.008 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:27.007+0000 7f0380dd5740 1 -- 172.21.15.26:0/3743468418 shutdown_connections 2024-09-11T23:23:27.008 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:27.008+0000 7f0380dd5740 1 -- 172.21.15.26:0/3743468418 wait complete. 2024-09-11T23:23:27.010 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:27.011+0000 7f0380dd5740 20 client.26045 trim_cache size 0 max 16384 2024-09-11T23:23:27.011 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-09-11T23:23:27.011 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "message": "", 2024-09-11T23:23:27.011 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-09-11T23:23:27.011 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-09-11T23:23:27.026 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:27.026+0000 7f037df5f640 1 -- 172.21.15.26:0/740456954 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f03600a2f00 msgr2=0x7f03600a53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:23:27.026 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:27.026+0000 7f037df5f640 1 --2- 172.21.15.26:0/740456954 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f03600a2f00 0x7f03600a53c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f0378082060 tx=0x7f037002f4b0 comp rx=0 tx=0).stop 2024-09-11T23:23:27.027 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:27.027+0000 7f037df5f640 1 -- 172.21.15.26:0/740456954 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f03781091f0 msgr2=0x7f0378080810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:23:27.027 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:27.027+0000 7f037df5f640 1 --2- 172.21.15.26:0/740456954 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f03781091f0 0x7f0378080810 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f036800b450 tx=0x7f036800b910 comp rx=0 tx=0).stop 2024-09-11T23:23:27.028 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:27.027+0000 7f037df5f640 1 -- 172.21.15.26:0/740456954 shutdown_connections 2024-09-11T23:23:27.028 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:27.027+0000 7f037df5f640 1 --2- 172.21.15.26:0/740456954 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f03600a2f00 0x7f03600a53c0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:27.028 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:27.027+0000 7f037df5f640 1 --2- 172.21.15.26:0/740456954 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f037810d610 0x7f0378081290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:27.028 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:27.027+0000 7f037df5f640 1 --2- 172.21.15.26:0/740456954 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0378109b70 0x7f0378080d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:27.028 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:27.027+0000 7f037df5f640 1 --2- 172.21.15.26:0/740456954 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f03781091f0 0x7f0378080810 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:27.028 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:27.027+0000 7f037df5f640 1 -- 172.21.15.26:0/740456954 >> 172.21.15.26:0/740456954 conn(0x7f037806cc60 msgr2=0x7f0378082e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:23:27.028 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:27.028+0000 7f037df5f640 1 -- 172.21.15.26:0/740456954 shutdown_connections 2024-09-11T23:23:27.029 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:27.028+0000 7f037df5f640 1 -- 172.21.15.26:0/740456954 wait complete. 2024-09-11T23:23:27.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:23:27 smithi136 ceph-mon[35121]: pgmap v155: 97 pgs: 97 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 1.1 MiB/s wr, 306 op/s 2024-09-11T23:23:28.000 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:23:27 smithi026 ceph-mon[31172]: pgmap v155: 97 pgs: 97 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 1.1 MiB/s wr, 306 op/s 2024-09-11T23:23:28.052 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:23:27 smithi076 ceph-mon[35296]: pgmap v155: 97 pgs: 97 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 1.1 MiB/s wr, 306 op/s 2024-09-11T23:23:29.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:23:29 smithi136 ceph-mon[35121]: pgmap v156: 97 pgs: 97 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.3 MiB/s wr, 296 op/s 2024-09-11T23:23:30.000 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:23:29 smithi026 ceph-mon[31172]: pgmap v156: 97 pgs: 97 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.3 MiB/s wr, 296 op/s 2024-09-11T23:23:30.052 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:23:29 smithi076 ceph-mon[35296]: pgmap v156: 97 pgs: 97 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.3 MiB/s wr, 296 op/s 2024-09-11T23:23:31.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:23:31 smithi136 ceph-mon[35121]: pgmap v157: 97 pgs: 97 active+clean; 1.5 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 7.7 MiB/s wr, 678 op/s 2024-09-11T23:23:32.000 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:23:31 smithi026 ceph-mon[31172]: pgmap v157: 97 pgs: 97 active+clean; 1.5 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 7.7 MiB/s wr, 678 op/s 2024-09-11T23:23:32.052 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:23:31 smithi076 ceph-mon[35296]: pgmap v157: 97 pgs: 97 active+clean; 1.5 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 7.7 MiB/s wr, 678 op/s 2024-09-11T23:23:33.578 INFO:tasks.workunit.client.0.smithi026.stdout: 412 1903730 2208 1298060 1829 367061 1389 2024-09-11T23:23:33.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:23:33 smithi136 ceph-mon[35121]: pgmap v158: 97 pgs: 97 active+clean; 1.5 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 7.5 MiB/s wr, 539 op/s 2024-09-11T23:23:34.000 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:23:33 smithi026 ceph-mon[31172]: pgmap v158: 97 pgs: 97 active+clean; 1.5 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 7.5 MiB/s wr, 539 op/s 2024-09-11T23:23:34.052 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:23:33 smithi076 ceph-mon[35296]: pgmap v158: 97 pgs: 97 active+clean; 1.5 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 7.5 MiB/s wr, 539 op/s 2024-09-11T23:23:35.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:23:35 smithi136 ceph-mon[35121]: pgmap v159: 97 pgs: 97 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 9.8 MiB/s wr, 711 op/s 2024-09-11T23:23:35.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:23:35 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:23:36.000 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:23:35 smithi026 ceph-mon[31172]: pgmap v159: 97 pgs: 97 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 9.8 MiB/s wr, 711 op/s 2024-09-11T23:23:36.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:23:35 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:23:36.052 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:23:35 smithi076 ceph-mon[35296]: pgmap v159: 97 pgs: 97 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 9.8 MiB/s wr, 711 op/s 2024-09-11T23:23:36.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:23:35 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:23:38.009 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:23:37 smithi026 ceph-mon[31172]: pgmap v160: 97 pgs: 97 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 9.0 MiB/s wr, 726 op/s 2024-09-11T23:23:38.052 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:23:37 smithi076 ceph-mon[35296]: pgmap v160: 97 pgs: 97 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 9.0 MiB/s wr, 726 op/s 2024-09-11T23:23:38.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:23:37 smithi136 ceph-mon[35121]: pgmap v160: 97 pgs: 97 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 9.0 MiB/s wr, 726 op/s 2024-09-11T23:23:40.052 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:23:39 smithi076 ceph-mon[35296]: pgmap v161: 97 pgs: 97 active+clean; 1.5 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 9.9 MiB/s wr, 765 op/s 2024-09-11T23:23:40.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:23:39 smithi136 ceph-mon[35121]: pgmap v161: 97 pgs: 97 active+clean; 1.5 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 9.9 MiB/s wr, 765 op/s 2024-09-11T23:23:40.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:23:39 smithi026 ceph-mon[31172]: pgmap v161: 97 pgs: 97 active+clean; 1.5 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 9.9 MiB/s wr, 765 op/s 2024-09-11T23:23:41.364 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.362+0000 7fe12b265640 1 -- 172.21.15.26:0/4024268741 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe1241095d0 msgr2=0x7fe124109a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:23:41.364 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.362+0000 7fe12b265640 1 --2- 172.21.15.26:0/4024268741 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe1241095d0 0x7fe124109a10 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fe11c009ca0 tx=0x7fe11c02a8f0 comp rx=0 tx=0).stop 2024-09-11T23:23:41.371 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.369+0000 7fe12b265640 1 -- 172.21.15.26:0/4024268741 shutdown_connections 2024-09-11T23:23:41.371 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.369+0000 7fe12b265640 1 --2- 172.21.15.26:0/4024268741 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe12411b140 0x7fe12411d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:41.371 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.369+0000 7fe12b265640 1 --2- 172.21.15.26:0/4024268741 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe1241095d0 0x7fe124109a10 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:41.371 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.369+0000 7fe12b265640 1 --2- 172.21.15.26:0/4024268741 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe1241091f0 0x7fe124110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:41.371 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.369+0000 7fe12b265640 1 -- 172.21.15.26:0/4024268741 >> 172.21.15.26:0/4024268741 conn(0x7fe12406cc60 msgr2=0x7fe12406d070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:23:41.371 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.370+0000 7fe12b265640 1 -- 172.21.15.26:0/4024268741 shutdown_connections 2024-09-11T23:23:41.376 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.370+0000 7fe12b265640 1 -- 172.21.15.26:0/4024268741 wait complete. 2024-09-11T23:23:41.379 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.378+0000 7fe12b265640 1 Processor -- start 2024-09-11T23:23:41.380 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.378+0000 7fe12b265640 1 -- start start 2024-09-11T23:23:41.380 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.379+0000 7fe12b265640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe1241091f0 0x7fe124114920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:41.380 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.379+0000 7fe12b265640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe1241095d0 0x7fe124114e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:41.380 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.379+0000 7fe12b265640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe12411b140 0x7fe12411a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:41.380 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.379+0000 7fe12b265640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fe124110f30 con 0x7fe1241091f0 2024-09-11T23:23:41.380 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.379+0000 7fe12b265640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fe124110db0 con 0x7fe1241095d0 2024-09-11T23:23:41.381 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.379+0000 7fe12b265640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe1241110b0 con 0x7fe12411b140 2024-09-11T23:23:41.384 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.382+0000 7fe129802640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe12411b140 0x7fe12411a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:41.385 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.383+0000 7fe129802640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe12411b140 0x7fe12411a130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:56280/0 (socket says 172.21.15.26:56280) 2024-09-11T23:23:41.385 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.383+0000 7fe129802640 1 -- 172.21.15.26:0/710918410 learned_addr learned my addr 172.21.15.26:0/710918410 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:23:41.385 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.383+0000 7fe128800640 1 --2- 172.21.15.26:0/710918410 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe1241095d0 0x7fe124114e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:41.385 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.383+0000 7fe129802640 1 -- 172.21.15.26:0/710918410 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe1241095d0 msgr2=0x7fe124114e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:23:41.385 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.383+0000 7fe129802640 1 --2- 172.21.15.26:0/710918410 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe1241095d0 0x7fe124114e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:41.385 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.383+0000 7fe129802640 1 -- 172.21.15.26:0/710918410 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe1241091f0 msgr2=0x7fe124114920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:23:41.385 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.383+0000 7fe129802640 1 --2- 172.21.15.26:0/710918410 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe1241091f0 0x7fe124114920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:41.385 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.383+0000 7fe129802640 1 -- 172.21.15.26:0/710918410 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe12411a9d0 con 0x7fe12411b140 2024-09-11T23:23:41.385 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.384+0000 7fe128800640 1 --2- 172.21.15.26:0/710918410 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe1241095d0 0x7fe124114e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:23:41.385 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.384+0000 7fe129802640 1 --2- 172.21.15.26:0/710918410 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe12411b140 0x7fe12411a130 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fe11400b3c0 tx=0x7fe11400b880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:23:41.396 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.395+0000 7fe11a7fc640 1 -- 172.21.15.26:0/710918410 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe114011830 con 0x7fe12411b140 2024-09-11T23:23:41.402 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.395+0000 7fe12b265640 1 -- 172.21.15.26:0/710918410 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe12411acc0 con 0x7fe12411b140 2024-09-11T23:23:41.402 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.395+0000 7fe12b265640 1 -- 172.21.15.26:0/710918410 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1241bc560 con 0x7fe12411b140 2024-09-11T23:23:41.402 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.399+0000 7fe12e001740 1 Processor -- start 2024-09-11T23:23:41.402 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.399+0000 7fe12e001740 1 -- start start 2024-09-11T23:23:41.402 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.400+0000 7fe12e001740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5583704c8cf0 0x5583704c90d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:41.402 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.400+0000 7fe12e001740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5583704c9610 0x5583704d3970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:41.402 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.400+0000 7fe12e001740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5583704d3eb0 0x5583704d62a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:41.402 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.400+0000 7fe12e001740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55837042d2f0 con 0x5583704d3eb0 2024-09-11T23:23:41.402 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.400+0000 7fe12e001740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5583703d5270 con 0x5583704c9610 2024-09-11T23:23:41.402 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.400+0000 7fe12e001740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5583704292b0 con 0x5583704c8cf0 2024-09-11T23:23:41.403 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.401+0000 7fe129001640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5583704c9610 0x5583704d3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:41.403 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.401+0000 7fe129001640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5583704c9610 0x5583704d3970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:33572/0 (socket says 172.21.15.26:33572) 2024-09-11T23:23:41.403 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.401+0000 7fe129001640 1 -- 172.21.15.26:0/142623659 learned_addr learned my addr 172.21.15.26:0/142623659 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:23:41.403 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.401+0000 7fe11a7fc640 1 -- 172.21.15.26:0/710918410 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe114016070 con 0x7fe12411b140 2024-09-11T23:23:41.404 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.402+0000 7fe128800640 1 --2- 172.21.15.26:0/142623659 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5583704c8cf0 0x5583704c90d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:41.404 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.402+0000 7fe128800640 1 --2- 172.21.15.26:0/142623659 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5583704d3eb0 0x5583704d62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:41.405 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.402+0000 7fe129001640 1 -- 172.21.15.26:0/142623659 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5583704c8cf0 msgr2=0x5583704c90d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:23:41.405 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.402+0000 7fe129001640 1 --2- 172.21.15.26:0/142623659 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5583704c8cf0 0x5583704c90d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:41.406 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.402+0000 7fe129001640 1 -- 172.21.15.26:0/142623659 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5583704d3eb0 msgr2=0x5583704d62a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:23:41.406 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.402+0000 7fe129001640 1 --2- 172.21.15.26:0/142623659 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5583704d3eb0 0x5583704d62a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:41.406 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.402+0000 7fe129001640 1 -- 172.21.15.26:0/142623659 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5583703d4340 con 0x5583704c9610 2024-09-11T23:23:41.406 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.405+0000 7fe11a7fc640 1 -- 172.21.15.26:0/710918410 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe11400f2e0 con 0x7fe12411b140 2024-09-11T23:23:41.406 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.405+0000 7fe11a7fc640 1 -- 172.21.15.26:0/710918410 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fe11400f540 con 0x7fe12411b140 2024-09-11T23:23:41.406 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.405+0000 7fe129001640 1 --2- 172.21.15.26:0/142623659 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5583704c9610 0x5583704d3970 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fe1200097c0 tx=0x7fe120009c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7795b905aef750a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:23:41.412 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.411+0000 7fe11a7fc640 1 --2- 172.21.15.26:0/710918410 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fe1100a2f30 0x7fe1100a53f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:41.417 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.416+0000 7fe11a7fc640 1 -- 172.21.15.26:0/710918410 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fe1140cfa10 con 0x7fe12411b140 2024-09-11T23:23:41.417 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.417+0000 7fe128800640 1 --2- 172.21.15.26:0/710918410 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fe1100a2f30 0x7fe1100a53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:41.431 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.430+0000 7fe10f7fe640 1 -- 172.21.15.26:0/142623659 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe120015020 con 0x5583704c9610 2024-09-11T23:23:41.431 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.431+0000 7fe128800640 1 --2- 172.21.15.26:0/710918410 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fe1100a2f30 0x7fe1100a53f0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x5583704c65e0 tx=0x7fe11c000f40 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:23:41.432 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.431+0000 7fe10f7fe640 1 -- 172.21.15.26:0/142623659 <== mon.2 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe120011420 con 0x5583704c9610 2024-09-11T23:23:41.433 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.432+0000 7fe12e001740 1 -- 172.21.15.26:0/142623659 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5583704c9610 msgr2=0x5583704d3970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:23:41.433 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.432+0000 7fe12e001740 1 --2- 172.21.15.26:0/142623659 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5583704c9610 0x5583704d3970 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fe1200097c0 tx=0x7fe120009c80 comp rx=0 tx=0).stop 2024-09-11T23:23:41.434 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.433+0000 7fe12e001740 1 -- 172.21.15.26:0/142623659 shutdown_connections 2024-09-11T23:23:41.434 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.433+0000 7fe12e001740 1 --2- 172.21.15.26:0/142623659 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5583704c9610 0x5583704d3970 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:41.434 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.433+0000 7fe12e001740 1 --2- 172.21.15.26:0/142623659 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5583704d3eb0 0x5583704d62a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:41.434 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.433+0000 7fe12e001740 1 --2- 172.21.15.26:0/142623659 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5583704c8cf0 0x5583704c90d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:41.434 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.433+0000 7fe12e001740 1 -- 172.21.15.26:0/142623659 >> 172.21.15.26:0/142623659 conn(0x5583704c5260 msgr2=0x5583704c5640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:23:41.452 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.452+0000 7fe12e001740 1 -- 172.21.15.26:0/142623659 shutdown_connections 2024-09-11T23:23:41.453 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.452+0000 7fe12e001740 1 -- 172.21.15.26:0/142623659 wait complete. 2024-09-11T23:23:41.453 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.452+0000 7fe12e001740 1 -- start start 2024-09-11T23:23:41.453 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.453+0000 7fe12e001740 1 Processor -- start 2024-09-11T23:23:41.454 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.453+0000 7fe12e001740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5583704c8cf0 0x5583704d3880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:41.455 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.454+0000 7fe12e001740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5583704d3eb0 0x5583704d0b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:41.455 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.454+0000 7fe12e001740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5583704d10b0 0x5583704d14f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:41.455 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.454+0000 7fe12e001740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5583704f8ae0 con 0x5583704d10b0 2024-09-11T23:23:41.455 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.454+0000 7fe12e001740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5583703b6290 con 0x5583704c8cf0 2024-09-11T23:23:41.455 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.454+0000 7fe12e001740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5583704f8c60 con 0x5583704d3eb0 2024-09-11T23:23:41.455 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.454+0000 7fe12e001740 10 client.0 _ll_register_callbacks cb 0x5583703e7140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:23:41.456 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.454+0000 7fe129001640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5583704c8cf0 0x5583704d3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:41.456 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.454+0000 7fe129001640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5583704c8cf0 0x5583704d3880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:33582/0 (socket says 172.21.15.26:33582) 2024-09-11T23:23:41.456 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.454+0000 7fe129001640 1 -- 172.21.15.26:0/548201747 learned_addr learned my addr 172.21.15.26:0/548201747 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:23:41.456 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.454+0000 7fe128800640 1 --2- 172.21.15.26:0/548201747 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5583704d3eb0 0x5583704d0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:41.456 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.455+0000 7fe129001640 1 -- 172.21.15.26:0/548201747 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5583704d3eb0 msgr2=0x5583704d0b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:23:41.456 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.455+0000 7fe129001640 1 --2- 172.21.15.26:0/548201747 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5583704d3eb0 0x5583704d0b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:41.456 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.455+0000 7fe129001640 1 -- 172.21.15.26:0/548201747 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5583704d10b0 msgr2=0x5583704d14f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:23:41.456 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.455+0000 7fe129001640 1 --2- 172.21.15.26:0/548201747 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5583704d10b0 0x5583704d14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:41.457 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.455+0000 7fe129001640 1 -- 172.21.15.26:0/548201747 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558370140210 con 0x5583704c8cf0 2024-09-11T23:23:41.457 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.455+0000 7fe129001640 1 --2- 172.21.15.26:0/548201747 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5583704c8cf0 0x5583704d3880 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fe12000bb00 tx=0x7fe12000bb30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=caebf381e5d34a37 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:23:41.464 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.463+0000 7fe10cff9640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-11T23:23:41.464 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.463+0000 7fe10cff9640 1 -- 172.21.15.26:0/548201747 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe120010a40 con 0x5583704c8cf0 2024-09-11T23:23:41.464 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.463+0000 7fe12e001740 1 -- 172.21.15.26:0/548201747 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55837043aa20 con 0x5583704c8cf0 2024-09-11T23:23:41.465 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.464+0000 7fe10cff9640 1 -- 172.21.15.26:0/548201747 <== mon.2 v2:172.21.15.76:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fe120010be0 con 0x5583704c8cf0 2024-09-11T23:23:41.466 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.465+0000 7fe10cff9640 1 -- 172.21.15.26:0/548201747 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fe12002b050 con 0x5583704c8cf0 2024-09-11T23:23:41.466 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.465+0000 7fe12e001740 10 client.26051 fetch_fsmap learned FSMap version 39 2024-09-11T23:23:41.466 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.465+0000 7fe12e001740 1 -- 172.21.15.26:0/548201747 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55837020c620 con 0x5583704c8cf0 2024-09-11T23:23:41.474 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.473+0000 7fe10cff9640 1 -- 172.21.15.26:0/548201747 <== mon.2 v2:172.21.15.76:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55837043aa20 con 0x5583704c8cf0 2024-09-11T23:23:41.475 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.475+0000 7fe10cff9640 1 -- 172.21.15.26:0/548201747 <== mon.2 v2:172.21.15.76:3300/0 5 ==== fsmap(e 39) ==== 8410+0+0 (secure 0 0 0) 0x7fe120011420 con 0x5583704c8cf0 2024-09-11T23:23:41.476 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.475+0000 7fe12e001740 10 client.26051 fetch_fsmap finished waiting for FSMap version 39 2024-09-11T23:23:41.476 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.475+0000 7fe12e001740 10 client.26051 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:23:41.476 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.475+0000 7fe12e001740 20 client.26051 populate_metadata read hostname 'smithi026' 2024-09-11T23:23:41.476 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.475+0000 7fe12e001740 1 --2- 172.21.15.26:0/548201747 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x558370506560 0x558370508a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:41.476 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.475+0000 7fe12e001740 4 client.26051 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:23:41.476 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.475+0000 7fe12e001740 1 -- 172.21.15.26:0/548201747 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558370434d70 con 0x558370506560 2024-09-11T23:23:41.478 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.477+0000 7fe128800640 1 --2- 172.21.15.26:0/548201747 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x558370506560 0x558370508a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:41.481 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.480+0000 7fe128800640 1 --2- 172.21.15.26:0/548201747 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x558370506560 0x558370508a20 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe40f46cbbe117a server_cookie=70eacbc3ca25eeba in_seq=0 out_seq=0 2024-09-11T23:23:41.481 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.480+0000 7fe10cff9640 10 client.26051 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:23:41.486 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.485+0000 7fe10cff9640 1 -- 172.21.15.26:0/548201747 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558370434d70 con 0x558370506560 2024-09-11T23:23:41.486 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.485+0000 7fe10cff9640 10 client.26051 handle_command_reply: tid=0 2024-09-11T23:23:41.486 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.485+0000 7fe12e001740 1 client.26051 shutdown 2024-09-11T23:23:41.487 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.486+0000 7fe12e001740 1 -- 172.21.15.26:0/548201747 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5583704c8cf0 msgr2=0x5583704d3880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:23:41.487 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.486+0000 7fe12e001740 1 --2- 172.21.15.26:0/548201747 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5583704c8cf0 0x5583704d3880 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fe12000bb00 tx=0x7fe12000bb30 comp rx=0 tx=0).stop 2024-09-11T23:23:41.487 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.486+0000 7fe12e001740 1 -- 172.21.15.26:0/548201747 shutdown_connections 2024-09-11T23:23:41.487 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.486+0000 7fe12e001740 1 --2- 172.21.15.26:0/548201747 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x558370506560 0x558370508a20 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:41.487 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.486+0000 7fe12e001740 1 --2- 172.21.15.26:0/548201747 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5583704d10b0 0x5583704d14f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:41.487 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.486+0000 7fe12e001740 1 --2- 172.21.15.26:0/548201747 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5583704d3eb0 0x5583704d0b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:41.487 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.486+0000 7fe12e001740 1 --2- 172.21.15.26:0/548201747 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5583704c8cf0 0x5583704d3880 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:41.487 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.486+0000 7fe12e001740 1 -- 172.21.15.26:0/548201747 >> 172.21.15.26:0/548201747 conn(0x5583704c5260 msgr2=0x5583704ca750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:23:41.487 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.486+0000 7fe10cff9640 0 client.26051 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:23:41.504 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.503+0000 7fe12e001740 1 -- 172.21.15.26:0/548201747 shutdown_connections 2024-09-11T23:23:41.504 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.503+0000 7fe12e001740 1 -- 172.21.15.26:0/548201747 wait complete. 2024-09-11T23:23:41.504 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.503+0000 7fe12e001740 20 client.26051 trim_cache size 0 max 16384 2024-09-11T23:23:41.782 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.781+0000 7fe12e001740 1 Processor -- start 2024-09-11T23:23:41.782 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.781+0000 7fe12e001740 1 -- start start 2024-09-11T23:23:41.789 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.789+0000 7fe12e001740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5583704c5260 0x5583704cf3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:41.791 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.790+0000 7fe129001640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5583704c5260 0x5583704cf3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:41.791 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.790+0000 7fe129001640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5583704c5260 0x5583704cf3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:33596/0 (socket says 172.21.15.26:33596) 2024-09-11T23:23:41.791 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.790+0000 7fe12e001740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5583704c8cf0 0x5583704cf910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:41.791 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.790+0000 7fe129001640 1 -- 172.21.15.26:0/2439934752 learned_addr learned my addr 172.21.15.26:0/2439934752 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:23:41.792 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.791+0000 7fe12e001740 1 --2- 172.21.15.26:0/2439934752 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5583704d10b0 0x5583704d52a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:41.792 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.791+0000 7fe12e001740 1 -- 172.21.15.26:0/2439934752 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fe10800e750 con 0x5583704c8cf0 2024-09-11T23:23:41.792 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.791+0000 7fe12e001740 1 -- 172.21.15.26:0/2439934752 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fe1080104d0 con 0x5583704c5260 2024-09-11T23:23:41.792 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.791+0000 7fe12e001740 1 -- 172.21.15.26:0/2439934752 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe10800eee0 con 0x5583704d10b0 2024-09-11T23:23:41.792 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.791+0000 7fe129802640 1 --2- 172.21.15.26:0/2439934752 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5583704d10b0 0x5583704d52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:41.793 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.792+0000 7fe129802640 1 -- 172.21.15.26:0/2439934752 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5583704c5260 msgr2=0x5583704cf3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:23:41.793 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.792+0000 7fe129802640 1 --2- 172.21.15.26:0/2439934752 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5583704c5260 0x5583704cf3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:41.793 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.792+0000 7fe129802640 1 -- 172.21.15.26:0/2439934752 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5583704c8cf0 msgr2=0x5583704cf910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:23:41.793 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.792+0000 7fe129802640 1 --2- 172.21.15.26:0/2439934752 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5583704c8cf0 0x5583704cf910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:41.793 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.792+0000 7fe129802640 1 -- 172.21.15.26:0/2439934752 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5583704c99a0 con 0x5583704d10b0 2024-09-11T23:23:41.793 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.792+0000 7fe129802640 1 --2- 172.21.15.26:0/2439934752 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5583704d10b0 0x5583704d52a0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fe1140cd2a0 tx=0x7fe11409b9d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7f3b778ab55e79ef server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:23:41.812 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.811+0000 7fe10effd640 1 -- 172.21.15.26:0/2439934752 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1140a0420 con 0x5583704d10b0 2024-09-11T23:23:41.812 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.811+0000 7fe10effd640 1 -- 172.21.15.26:0/2439934752 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe1140a4ce0 con 0x5583704d10b0 2024-09-11T23:23:41.812 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.811+0000 7fe10effd640 1 -- 172.21.15.26:0/2439934752 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe11409c530 con 0x5583704d10b0 2024-09-11T23:23:41.813 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.812+0000 7fe12e001740 1 -- 172.21.15.26:0/2439934752 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5583704d10b0 msgr2=0x5583704d52a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:23:41.813 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.813+0000 7fe12e001740 1 --2- 172.21.15.26:0/2439934752 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5583704d10b0 0x5583704d52a0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fe1140cd2a0 tx=0x7fe11409b9d0 comp rx=0 tx=0).stop 2024-09-11T23:23:41.816 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.815+0000 7fe12e001740 1 -- 172.21.15.26:0/2439934752 shutdown_connections 2024-09-11T23:23:41.817 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.815+0000 7fe12e001740 1 --2- 172.21.15.26:0/2439934752 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5583704d10b0 0x5583704d52a0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:41.817 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.816+0000 7fe12e001740 1 --2- 172.21.15.26:0/2439934752 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5583704c8cf0 0x5583704cf910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:41.817 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.816+0000 7fe12e001740 1 --2- 172.21.15.26:0/2439934752 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5583704c5260 0x5583704cf3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:41.817 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.816+0000 7fe12e001740 1 -- 172.21.15.26:0/2439934752 >> 172.21.15.26:0/2439934752 conn(0x5583704d6400 msgr2=0x5583704cbd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:23:41.824 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.823+0000 7fe12e001740 1 -- 172.21.15.26:0/2439934752 shutdown_connections 2024-09-11T23:23:41.824 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.824+0000 7fe12e001740 1 -- 172.21.15.26:0/2439934752 wait complete. 2024-09-11T23:23:41.825 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.824+0000 7fe12e001740 1 -- start start 2024-09-11T23:23:41.825 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.825+0000 7fe12e001740 1 Processor -- start 2024-09-11T23:23:41.826 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.826+0000 7fe12e001740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5583704c5260 0x5583704fe630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:41.827 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.826+0000 7fe12e001740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5583704c8cf0 0x5583704d4d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:41.827 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.826+0000 7fe12e001740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5583704d10b0 0x5583704d5280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:41.827 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.826+0000 7fe12e001740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fe10800fe00 con 0x5583704c5260 2024-09-11T23:23:41.827 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.826+0000 7fe12e001740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fe10800f670 con 0x5583704c8cf0 2024-09-11T23:23:41.827 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.826+0000 7fe12e001740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe10800c7e0 con 0x5583704d10b0 2024-09-11T23:23:41.827 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.827+0000 7fe12e001740 10 client.0 _ll_register_callbacks cb 0x7fe120009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:23:41.828 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.827+0000 7fe128800640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5583704c8cf0 0x5583704d4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:41.828 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.827+0000 7fe129802640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5583704d10b0 0x5583704d5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:41.829 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.827+0000 7fe129001640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5583704c5260 0x5583704fe630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:41.829 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.827+0000 7fe129001640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5583704c5260 0x5583704fe630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40058/0 (socket says 172.21.15.26:40058) 2024-09-11T23:23:41.829 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.827+0000 7fe129001640 1 -- 172.21.15.26:0/2476054084 learned_addr learned my addr 172.21.15.26:0/2476054084 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:23:41.829 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.827+0000 7fe128800640 1 -- 172.21.15.26:0/2476054084 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5583704d10b0 msgr2=0x5583704d5280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:23:41.829 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.827+0000 7fe128800640 1 --2- 172.21.15.26:0/2476054084 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5583704d10b0 0x5583704d5280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:41.829 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.827+0000 7fe128800640 1 -- 172.21.15.26:0/2476054084 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5583704c5260 msgr2=0x5583704fe630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:23:41.829 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.827+0000 7fe128800640 1 --2- 172.21.15.26:0/2476054084 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5583704c5260 0x5583704fe630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:41.829 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.827+0000 7fe128800640 1 -- 172.21.15.26:0/2476054084 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5583704d5ab0 con 0x5583704c8cf0 2024-09-11T23:23:41.829 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.828+0000 7fe128800640 1 --2- 172.21.15.26:0/2476054084 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5583704c8cf0 0x5583704d4d40 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fe11c02f4b0 tx=0x7fe11c02a8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1056900d04bc9d2b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:23:41.859 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.858+0000 7fe10cff9640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-11T23:23:41.859 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.859+0000 7fe10cff9640 1 -- 172.21.15.26:0/2476054084 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe11c02fa10 con 0x5583704c8cf0 2024-09-11T23:23:41.867 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.867+0000 7fe12e001740 1 -- 172.21.15.26:0/2476054084 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe1140a4ce0 con 0x5583704c8cf0 2024-09-11T23:23:41.868 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.867+0000 7fe10cff9640 1 -- 172.21.15.26:0/2476054084 <== mon.2 v2:172.21.15.76:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fe11c02fbb0 con 0x5583704c8cf0 2024-09-11T23:23:41.869 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.868+0000 7fe10cff9640 1 -- 172.21.15.26:0/2476054084 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fe11c02fdf0 con 0x5583704c8cf0 2024-09-11T23:23:41.869 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.868+0000 7fe12e001740 10 client.26057 fetch_fsmap learned FSMap version 40 2024-09-11T23:23:41.869 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.868+0000 7fe12e001740 1 -- 172.21.15.26:0/2476054084 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x558370571300 con 0x5583704c8cf0 2024-09-11T23:23:41.869 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.869+0000 7fe10cff9640 1 -- 172.21.15.26:0/2476054084 <== mon.2 v2:172.21.15.76:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe11c036040 con 0x5583704c8cf0 2024-09-11T23:23:41.871 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.870+0000 7fe10cff9640 1 -- 172.21.15.26:0/2476054084 <== mon.2 v2:172.21.15.76:3300/0 5 ==== fsmap(e 40) ==== 8414+0+0 (secure 0 0 0) 0x7fe11c0543a0 con 0x5583704c8cf0 2024-09-11T23:23:41.871 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.870+0000 7fe12e001740 10 client.26057 fetch_fsmap finished waiting for FSMap version 40 2024-09-11T23:23:41.871 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.870+0000 7fe12e001740 10 client.26057 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:23:41.871 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.870+0000 7fe12e001740 20 client.26057 populate_metadata read hostname 'smithi026' 2024-09-11T23:23:41.871 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.870+0000 7fe12e001740 1 --2- 172.21.15.26:0/2476054084 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5583704d3eb0 0x5583705739e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:41.872 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.870+0000 7fe12e001740 4 client.26057 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T23:23:41.872 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.871+0000 7fe12e001740 1 -- 172.21.15.26:0/2476054084 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7fe1140a4ce0 con 0x5583704d3eb0 2024-09-11T23:23:41.873 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.872+0000 7fe129001640 1 --2- 172.21.15.26:0/2476054084 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5583704d3eb0 0x5583705739e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:41.878 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.877+0000 7fe129001640 1 --2- 172.21.15.26:0/2476054084 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5583704d3eb0 0x5583705739e0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b4fb29f2d73ab76 server_cookie=6827ba3eefcbc64b in_seq=0 out_seq=0 2024-09-11T23:23:41.878 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:41.877+0000 7fe10cff9640 10 client.26057 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:23:42.052 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:23:41 smithi076 ceph-mon[35296]: pgmap v162: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 10 MiB/s wr, 784 op/s 2024-09-11T23:23:42.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:23:41 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:23:42.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:23:41 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:23:42.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:23:41 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:23:42.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:23:41 smithi136 ceph-mon[35121]: pgmap v162: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 10 MiB/s wr, 784 op/s 2024-09-11T23:23:42.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:23:41 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:23:42.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:23:41 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:23:42.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:23:41 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:23:42.256 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:23:41 smithi026 ceph-mon[31172]: pgmap v162: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 10 MiB/s wr, 784 op/s 2024-09-11T23:23:42.257 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:23:41 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:23:42.257 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:23:41 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:23:42.257 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:23:41 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:23:43.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:23:42 smithi136 ceph-mon[35121]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:23:43.271 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:23:42 smithi026 ceph-mon[31172]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:23:43.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:23:42 smithi076 ceph-mon[35296]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:23:43.578 INFO:tasks.workunit.client.0.smithi026.stdout: 435 1954388 1530 1329846 1361 416190 2708 2024-09-11T23:23:44.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:23:43 smithi136 ceph-mon[35121]: pgmap v163: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.8 MiB/s wr, 372 op/s 2024-09-11T23:23:44.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:23:43 smithi026 ceph-mon[31172]: pgmap v163: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.8 MiB/s wr, 372 op/s 2024-09-11T23:23:44.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:23:43 smithi076 ceph-mon[35296]: pgmap v163: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.8 MiB/s wr, 372 op/s 2024-09-11T23:23:46.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:23:45 smithi136 ceph-mon[35121]: pgmap v164: 97 pgs: 97 active+clean; 1.6 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 13 MiB/s wr, 770 op/s 2024-09-11T23:23:46.421 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:23:45 smithi076 ceph-mon[35296]: pgmap v164: 97 pgs: 97 active+clean; 1.6 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 13 MiB/s wr, 770 op/s 2024-09-11T23:23:46.421 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:23:45 smithi026 ceph-mon[31172]: pgmap v164: 97 pgs: 97 active+clean; 1.6 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 13 MiB/s wr, 770 op/s 2024-09-11T23:23:48.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:23:47 smithi136 ceph-mon[35121]: pgmap v165: 97 pgs: 97 active+clean; 1.7 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 12 MiB/s wr, 703 op/s 2024-09-11T23:23:48.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:23:47 smithi136 ceph-mon[35121]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:23:48.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:23:47 smithi026 ceph-mon[31172]: pgmap v165: 97 pgs: 97 active+clean; 1.7 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 12 MiB/s wr, 703 op/s 2024-09-11T23:23:48.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:23:47 smithi026 ceph-mon[31172]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:23:48.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:23:47 smithi076 ceph-mon[35296]: pgmap v165: 97 pgs: 97 active+clean; 1.7 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 12 MiB/s wr, 703 op/s 2024-09-11T23:23:48.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:23:47 smithi076 ceph-mon[35296]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:23:49.136 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:49.135+0000 7fe10cff9640 1 -- 172.21.15.26:0/2476054084 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7fe1140a4ce0 con 0x5583704d3eb0 2024-09-11T23:23:49.136 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:49.135+0000 7fe10cff9640 10 client.26057 handle_command_reply: tid=0 2024-09-11T23:23:49.139 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:49.138+0000 7fe12e001740 1 client.26057 shutdown 2024-09-11T23:23:49.140 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:49.139+0000 7fe12e001740 1 -- 172.21.15.26:0/2476054084 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5583704c8cf0 msgr2=0x5583704d4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:23:49.140 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:49.139+0000 7fe12e001740 1 --2- 172.21.15.26:0/2476054084 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5583704c8cf0 0x5583704d4d40 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fe11c02f4b0 tx=0x7fe11c02a8f0 comp rx=0 tx=0).stop 2024-09-11T23:23:49.141 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:49.139+0000 7fe12e001740 1 -- 172.21.15.26:0/2476054084 shutdown_connections 2024-09-11T23:23:49.141 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:49.139+0000 7fe12e001740 1 --2- 172.21.15.26:0/2476054084 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5583704c8cf0 0x5583704d4d40 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:49.141 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:49.139+0000 7fe12e001740 1 --2- 172.21.15.26:0/2476054084 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5583704d3eb0 0x5583705739e0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:49.141 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:49.139+0000 7fe12e001740 1 --2- 172.21.15.26:0/2476054084 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5583704d10b0 0x5583704d5280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:49.141 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:49.139+0000 7fe12e001740 1 --2- 172.21.15.26:0/2476054084 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5583704c5260 0x5583704fe630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:49.141 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:49.139+0000 7fe12e001740 1 -- 172.21.15.26:0/2476054084 >> 172.21.15.26:0/2476054084 conn(0x5583704d6400 msgr2=0x5583704cf240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:23:49.141 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:49.140+0000 7fe10cff9640 0 client.26057 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:23:49.141 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:49.140+0000 7fe12e001740 1 -- 172.21.15.26:0/2476054084 shutdown_connections 2024-09-11T23:23:49.141 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:49.140+0000 7fe12e001740 1 -- 172.21.15.26:0/2476054084 wait complete. 2024-09-11T23:23:49.142 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:49.140+0000 7fe12e001740 20 client.26057 trim_cache size 0 max 16384 2024-09-11T23:23:49.142 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-09-11T23:23:49.142 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "message": "", 2024-09-11T23:23:49.142 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-09-11T23:23:49.142 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-09-11T23:23:49.158 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:49.157+0000 7fe12b265640 1 -- 172.21.15.26:0/710918410 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fe1100a2f30 msgr2=0x7fe1100a53f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:23:49.158 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:49.157+0000 7fe12b265640 1 --2- 172.21.15.26:0/710918410 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fe1100a2f30 0x7fe1100a53f0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x5583704c65e0 tx=0x7fe11c000f40 comp rx=0 tx=0).stop 2024-09-11T23:23:49.159 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:49.157+0000 7fe12b265640 1 -- 172.21.15.26:0/710918410 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe12411b140 msgr2=0x7fe12411a130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:23:49.159 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:49.157+0000 7fe12b265640 1 --2- 172.21.15.26:0/710918410 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe12411b140 0x7fe12411a130 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fe11400b3c0 tx=0x7fe11400b880 comp rx=0 tx=0).stop 2024-09-11T23:23:49.159 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:49.158+0000 7fe12b265640 1 -- 172.21.15.26:0/710918410 shutdown_connections 2024-09-11T23:23:49.159 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:49.158+0000 7fe12b265640 1 --2- 172.21.15.26:0/710918410 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fe1100a2f30 0x7fe1100a53f0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:49.159 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:49.158+0000 7fe12b265640 1 --2- 172.21.15.26:0/710918410 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe1241095d0 0x7fe124114e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:49.159 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:49.158+0000 7fe12b265640 1 --2- 172.21.15.26:0/710918410 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe12411b140 0x7fe12411a130 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:49.159 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:49.158+0000 7fe12b265640 1 --2- 172.21.15.26:0/710918410 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe1241091f0 0x7fe124114920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:49.159 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:49.158+0000 7fe12b265640 1 -- 172.21.15.26:0/710918410 >> 172.21.15.26:0/710918410 conn(0x7fe12406cc60 msgr2=0x7fe124070780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:23:49.159 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:49.158+0000 7fe12b265640 1 -- 172.21.15.26:0/710918410 shutdown_connections 2024-09-11T23:23:49.159 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:23:49.158+0000 7fe12b265640 1 -- 172.21.15.26:0/710918410 wait complete. 2024-09-11T23:23:50.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:23:49 smithi136 ceph-mon[35121]: pgmap v166: 97 pgs: 97 active+clean; 1.7 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 13 MiB/s wr, 805 op/s 2024-09-11T23:23:50.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:23:49 smithi026 ceph-mon[31172]: pgmap v166: 97 pgs: 97 active+clean; 1.7 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 13 MiB/s wr, 805 op/s 2024-09-11T23:23:50.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:23:49 smithi076 ceph-mon[35296]: pgmap v166: 97 pgs: 97 active+clean; 1.7 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 13 MiB/s wr, 805 op/s 2024-09-11T23:23:52.238 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:23:51 smithi136 ceph-mon[35121]: pgmap v167: 97 pgs: 97 active+clean; 1.8 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 19 MiB/s wr, 1.50k op/s 2024-09-11T23:23:52.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:23:51 smithi026 ceph-mon[31172]: pgmap v167: 97 pgs: 97 active+clean; 1.8 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 19 MiB/s wr, 1.50k op/s 2024-09-11T23:23:52.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:23:51 smithi076 ceph-mon[35296]: pgmap v167: 97 pgs: 97 active+clean; 1.8 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 19 MiB/s wr, 1.50k op/s 2024-09-11T23:23:53.578 INFO:tasks.workunit.client.0.smithi026.stdout: 472 1944646 1880 1330646 1933 444442 1254 2024-09-11T23:23:54.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:23:53 smithi136 ceph-mon[35121]: pgmap v168: 97 pgs: 97 active+clean; 1.8 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 18 MiB/s wr, 1.38k op/s 2024-09-11T23:23:54.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:23:53 smithi026 ceph-mon[31172]: pgmap v168: 97 pgs: 97 active+clean; 1.8 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 18 MiB/s wr, 1.38k op/s 2024-09-11T23:23:54.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:23:53 smithi076 ceph-mon[35296]: pgmap v168: 97 pgs: 97 active+clean; 1.8 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 18 MiB/s wr, 1.38k op/s 2024-09-11T23:23:56.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:23:55 smithi136 ceph-mon[35121]: pgmap v169: 97 pgs: 97 active+clean; 1.8 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 19 MiB/s wr, 1.53k op/s 2024-09-11T23:23:56.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:23:55 smithi026 ceph-mon[31172]: pgmap v169: 97 pgs: 97 active+clean; 1.8 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 19 MiB/s wr, 1.53k op/s 2024-09-11T23:23:56.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:23:55 smithi076 ceph-mon[35296]: pgmap v169: 97 pgs: 97 active+clean; 1.8 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 19 MiB/s wr, 1.53k op/s 2024-09-11T23:23:58.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:23:57 smithi136 ceph-mon[35121]: pgmap v170: 97 pgs: 97 active+clean; 1.8 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 12 MiB/s wr, 1.16k op/s 2024-09-11T23:23:58.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:23:57 smithi026 ceph-mon[31172]: pgmap v170: 97 pgs: 97 active+clean; 1.8 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 12 MiB/s wr, 1.16k op/s 2024-09-11T23:23:58.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:23:57 smithi076 ceph-mon[35296]: pgmap v170: 97 pgs: 97 active+clean; 1.8 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 12 MiB/s wr, 1.16k op/s 2024-09-11T23:24:00.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:23:59 smithi136 ceph-mon[35121]: pgmap v171: 97 pgs: 97 active+clean; 1.8 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 11 MiB/s wr, 1.10k op/s 2024-09-11T23:24:00.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:23:59 smithi026 ceph-mon[31172]: pgmap v171: 97 pgs: 97 active+clean; 1.8 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 11 MiB/s wr, 1.10k op/s 2024-09-11T23:24:00.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:23:59 smithi076 ceph-mon[35296]: pgmap v171: 97 pgs: 97 active+clean; 1.8 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 11 MiB/s wr, 1.10k op/s 2024-09-11T23:24:02.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:24:01 smithi136 ceph-mon[35121]: pgmap v172: 97 pgs: 97 active+clean; 1.7 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 11 MiB/s wr, 1.07k op/s 2024-09-11T23:24:02.254 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:24:01 smithi026 ceph-mon[31172]: pgmap v172: 97 pgs: 97 active+clean; 1.7 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 11 MiB/s wr, 1.07k op/s 2024-09-11T23:24:02.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:24:01 smithi076 ceph-mon[35296]: pgmap v172: 97 pgs: 97 active+clean; 1.7 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 11 MiB/s wr, 1.07k op/s 2024-09-11T23:24:03.579 INFO:tasks.workunit.client.0.smithi026.stdout: 501 2128823 1833 1442307 1547 514519 2903 2024-09-11T23:24:04.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:24:03 smithi136 ceph-mon[35121]: pgmap v173: 97 pgs: 97 active+clean; 1.7 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.0 MiB/s wr, 333 op/s 2024-09-11T23:24:04.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:24:03 smithi026 ceph-mon[31172]: pgmap v173: 97 pgs: 97 active+clean; 1.7 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.0 MiB/s wr, 333 op/s 2024-09-11T23:24:04.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:24:03 smithi076 ceph-mon[35296]: pgmap v173: 97 pgs: 97 active+clean; 1.7 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.0 MiB/s wr, 333 op/s 2024-09-11T23:24:05.304 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.303+0000 7fa7c2c3c640 1 -- 172.21.15.26:0/1643008481 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa7bc1091f0 msgr2=0x7fa7bc110250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:24:05.304 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.303+0000 7fa7c2c3c640 1 --2- 172.21.15.26:0/1643008481 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa7bc1091f0 0x7fa7bc110250 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fa7ac009900 tx=0x7fa7ac02a400 comp rx=0 tx=0).stop 2024-09-11T23:24:05.304 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.304+0000 7fa7c2c3c640 1 -- 172.21.15.26:0/1643008481 shutdown_connections 2024-09-11T23:24:05.304 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.304+0000 7fa7c2c3c640 1 --2- 172.21.15.26:0/1643008481 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa7bc11b140 0x7fa7bc11d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:05.304 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.304+0000 7fa7c2c3c640 1 --2- 172.21.15.26:0/1643008481 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa7bc1095d0 0x7fa7bc109a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:05.304 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.304+0000 7fa7c2c3c640 1 --2- 172.21.15.26:0/1643008481 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa7bc1091f0 0x7fa7bc110250 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:05.304 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.304+0000 7fa7c2c3c640 1 -- 172.21.15.26:0/1643008481 >> 172.21.15.26:0/1643008481 conn(0x7fa7bc06cc60 msgr2=0x7fa7bc06d070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:24:05.310 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.310+0000 7fa7c2c3c640 1 -- 172.21.15.26:0/1643008481 shutdown_connections 2024-09-11T23:24:05.312 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.311+0000 7fa7c2c3c640 1 -- 172.21.15.26:0/1643008481 wait complete. 2024-09-11T23:24:05.313 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.313+0000 7fa7c2c3c640 1 Processor -- start 2024-09-11T23:24:05.313 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.314+0000 7fa7c2c3c640 1 -- start start 2024-09-11T23:24:05.314 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.314+0000 7fa7c2c3c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa7bc1091f0 0x7fa7bc1a7070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:05.314 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.315+0000 7fa7c2c3c640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa7bc1095d0 0x7fa7bc1a75b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:05.315 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.315+0000 7fa7c2c3c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa7bc11b140 0x7fa7bc1ab930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:05.316 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.315+0000 7fa7bbfff640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa7bc1095d0 0x7fa7bc1a75b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:05.316 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.315+0000 7fa7c2c3c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fa7bc120b60 con 0x7fa7bc11b140 2024-09-11T23:24:05.316 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.315+0000 7fa7c2c3c640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fa7bc1209e0 con 0x7fa7bc1091f0 2024-09-11T23:24:05.316 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.315+0000 7fa7c2c3c640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fa7bc120ce0 con 0x7fa7bc1095d0 2024-09-11T23:24:05.316 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.315+0000 7fa7c11d9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa7bc11b140 0x7fa7bc1ab930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:05.316 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.315+0000 7fa7c11d9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa7bc11b140 0x7fa7bc1ab930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44620/0 (socket says 172.21.15.26:44620) 2024-09-11T23:24:05.316 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.315+0000 7fa7c11d9640 1 -- 172.21.15.26:0/2408499968 learned_addr learned my addr 172.21.15.26:0/2408499968 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:24:05.317 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.317+0000 7fa7bbfff640 1 -- 172.21.15.26:0/2408499968 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa7bc1091f0 msgr2=0x7fa7bc1a7070 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-11T23:24:05.317 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.317+0000 7fa7bbfff640 1 --2- 172.21.15.26:0/2408499968 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa7bc1091f0 0x7fa7bc1a7070 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:05.317 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.317+0000 7fa7bbfff640 1 -- 172.21.15.26:0/2408499968 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa7bc11b140 msgr2=0x7fa7bc1ab930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:24:05.318 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.318+0000 7fa7bbfff640 1 --2- 172.21.15.26:0/2408499968 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa7bc11b140 0x7fa7bc1ab930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:05.318 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.318+0000 7fa7bbfff640 1 -- 172.21.15.26:0/2408499968 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7bc1ac060 con 0x7fa7bc1095d0 2024-09-11T23:24:05.318 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.318+0000 7fa7c11d9640 1 --2- 172.21.15.26:0/2408499968 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa7bc11b140 0x7fa7bc1ab930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:24:05.318 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.318+0000 7fa7bbfff640 1 --2- 172.21.15.26:0/2408499968 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa7bc1095d0 0x7fa7bc1a75b0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fa7b000e9f0 tx=0x7fa7b000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:24:05.349 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.350+0000 7fa7b9ffb640 1 -- 172.21.15.26:0/2408499968 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7b000ccc0 con 0x7fa7bc1095d0 2024-09-11T23:24:05.357 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.357+0000 7fa7b9ffb640 1 -- 172.21.15.26:0/2408499968 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7b0010910 con 0x7fa7bc1095d0 2024-09-11T23:24:05.357 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.357+0000 7fa7c2c3c640 1 -- 172.21.15.26:0/2408499968 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7bc1ac350 con 0x7fa7bc1095d0 2024-09-11T23:24:05.357 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.357+0000 7fa7c2c3c640 1 -- 172.21.15.26:0/2408499968 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7bc1b3be0 con 0x7fa7bc1095d0 2024-09-11T23:24:05.365 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.362+0000 7fa7c59d6740 1 Processor -- start 2024-09-11T23:24:05.365 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.364+0000 7fa7c59d6740 1 -- start start 2024-09-11T23:24:05.365 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.364+0000 7fa7c59d6740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c29f8bccc0 0x55c29f8bd0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:05.365 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.364+0000 7fa7c59d6740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c29f8bd5e0 0x55c29f8c7940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:05.365 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.364+0000 7fa7c59d6740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c29f8c7e80 0x55c29f8ca270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:05.365 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.364+0000 7fa7c59d6740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55c29f8212f0 con 0x55c29f8c7e80 2024-09-11T23:24:05.365 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.364+0000 7fa7c59d6740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55c29f7c9270 con 0x55c29f8bccc0 2024-09-11T23:24:05.365 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.364+0000 7fa7c59d6740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55c29f81d2b0 con 0x55c29f8bd5e0 2024-09-11T23:24:05.366 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.364+0000 7fa7c09d8640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c29f8bd5e0 0x55c29f8c7940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:05.366 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.364+0000 7fa7c09d8640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c29f8bd5e0 0x55c29f8c7940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:56090/0 (socket says 172.21.15.26:56090) 2024-09-11T23:24:05.366 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.364+0000 7fa7c11d9640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c29f8bccc0 0x55c29f8bd0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:05.366 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.364+0000 7fa7c09d8640 1 -- 172.21.15.26:0/1269666155 learned_addr learned my addr 172.21.15.26:0/1269666155 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:24:05.366 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.365+0000 7fa7c09d8640 1 -- 172.21.15.26:0/1269666155 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c29f8bccc0 msgr2=0x55c29f8bd0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:24:05.366 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.365+0000 7fa7c09d8640 1 --2- 172.21.15.26:0/1269666155 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c29f8bccc0 0x55c29f8bd0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:05.366 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.365+0000 7fa7c09d8640 1 -- 172.21.15.26:0/1269666155 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c29f8c7e80 msgr2=0x55c29f8ca270 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T23:24:05.367 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.365+0000 7fa7bbfff640 1 --2- 172.21.15.26:0/1269666155 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c29f8c7e80 0x55c29f8ca270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:05.367 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.365+0000 7fa7c09d8640 1 --2- 172.21.15.26:0/1269666155 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c29f8c7e80 0x55c29f8ca270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:05.367 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.365+0000 7fa7c09d8640 1 -- 172.21.15.26:0/1269666155 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c29f7c8340 con 0x55c29f8bd5e0 2024-09-11T23:24:05.367 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.366+0000 7fa7bbfff640 1 --2- 172.21.15.26:0/1269666155 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c29f8c7e80 0x55c29f8ca270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:24:05.367 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.366+0000 7fa7c09d8640 1 --2- 172.21.15.26:0/1269666155 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c29f8bd5e0 0x55c29f8c7940 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fa7ac009230 tx=0x7fa7ac031400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1535256c5383ae45 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:24:05.371 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.371+0000 7fa7b9ffb640 1 -- 172.21.15.26:0/2408499968 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7b0018980 con 0x7fa7bc1095d0 2024-09-11T23:24:05.372 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.372+0000 7fa7b9ffb640 1 -- 172.21.15.26:0/2408499968 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fa7b0018be0 con 0x7fa7bc1095d0 2024-09-11T23:24:05.373 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.374+0000 7fa7b9ffb640 1 --2- 172.21.15.26:0/2408499968 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fa7a40a2f30 0x7fa7a40a53f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:05.376 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.374+0000 7fa7c11d9640 1 --2- 172.21.15.26:0/2408499968 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fa7a40a2f30 0x7fa7a40a53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:05.377 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.375+0000 7fa7c11d9640 1 --2- 172.21.15.26:0/2408499968 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fa7a40a2f30 0x7fa7a40a53f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fa7b4011960 tx=0x7fa7b4009720 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:24:05.383 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.384+0000 7fa7b9ffb640 1 -- 172.21.15.26:0/2408499968 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fa7b00d07a0 con 0x7fa7bc1095d0 2024-09-11T23:24:05.391 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.391+0000 7fa7a2ffd640 1 -- 172.21.15.26:0/1269666155 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7ac03e030 con 0x55c29f8bd5e0 2024-09-11T23:24:05.391 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.391+0000 7fa7a2ffd640 1 -- 172.21.15.26:0/1269666155 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7ac0035f0 con 0x55c29f8bd5e0 2024-09-11T23:24:05.392 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.392+0000 7fa7c59d6740 1 -- 172.21.15.26:0/1269666155 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c29f8bd5e0 msgr2=0x55c29f8c7940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:24:05.392 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.392+0000 7fa7c59d6740 1 --2- 172.21.15.26:0/1269666155 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c29f8bd5e0 0x55c29f8c7940 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fa7ac009230 tx=0x7fa7ac031400 comp rx=0 tx=0).stop 2024-09-11T23:24:05.393 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.393+0000 7fa7c59d6740 1 -- 172.21.15.26:0/1269666155 shutdown_connections 2024-09-11T23:24:05.393 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.393+0000 7fa7c59d6740 1 --2- 172.21.15.26:0/1269666155 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c29f8c7e80 0x55c29f8ca270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:05.393 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.393+0000 7fa7c59d6740 1 --2- 172.21.15.26:0/1269666155 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c29f8bd5e0 0x55c29f8c7940 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:05.393 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.393+0000 7fa7c59d6740 1 --2- 172.21.15.26:0/1269666155 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c29f8bccc0 0x55c29f8bd0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:05.393 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.393+0000 7fa7c59d6740 1 -- 172.21.15.26:0/1269666155 >> 172.21.15.26:0/1269666155 conn(0x55c29f8b9230 msgr2=0x55c29f8b9610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:24:05.394 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.394+0000 7fa7c59d6740 1 -- 172.21.15.26:0/1269666155 shutdown_connections 2024-09-11T23:24:05.394 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.395+0000 7fa7c59d6740 1 -- 172.21.15.26:0/1269666155 wait complete. 2024-09-11T23:24:05.396 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.396+0000 7fa7c59d6740 1 -- start start 2024-09-11T23:24:05.396 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.396+0000 7fa7c59d6740 1 Processor -- start 2024-09-11T23:24:05.397 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.396+0000 7fa7c59d6740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c29f8bccc0 0x55c29f8c4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:05.397 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.396+0000 7fa7c59d6740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c29f8bd5e0 0x55c29f8c51f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:05.397 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.396+0000 7fa7c59d6740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c29f8c7e80 0x55c29f8c5730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:05.397 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.396+0000 7fa7c59d6740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55c29f8caaa0 con 0x55c29f8c7e80 2024-09-11T23:24:05.397 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.396+0000 7fa7c59d6740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55c29f7aa290 con 0x55c29f8bccc0 2024-09-11T23:24:05.398 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.397+0000 7fa7c59d6740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55c29f8cac20 con 0x55c29f8bd5e0 2024-09-11T23:24:05.398 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.397+0000 7fa7c11d9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c29f8c7e80 0x55c29f8c5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:05.398 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.397+0000 7fa7c11d9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c29f8c7e80 0x55c29f8c5730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44646/0 (socket says 172.21.15.26:44646) 2024-09-11T23:24:05.398 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.397+0000 7fa7c11d9640 1 -- 172.21.15.26:0/3473970033 learned_addr learned my addr 172.21.15.26:0/3473970033 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:24:05.398 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.397+0000 7fa7bbfff640 1 --2- 172.21.15.26:0/3473970033 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c29f8bd5e0 0x55c29f8c51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:05.398 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.397+0000 7fa7c09d8640 1 --2- 172.21.15.26:0/3473970033 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c29f8bccc0 0x55c29f8c4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:05.399 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.397+0000 7fa7c59d6740 10 client.0 _ll_register_callbacks cb 0x55c29f7db140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:24:05.399 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.398+0000 7fa7c11d9640 1 -- 172.21.15.26:0/3473970033 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c29f8bd5e0 msgr2=0x55c29f8c51f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:24:05.400 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.398+0000 7fa7c11d9640 1 --2- 172.21.15.26:0/3473970033 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c29f8bd5e0 0x55c29f8c51f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:05.400 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.398+0000 7fa7c11d9640 1 -- 172.21.15.26:0/3473970033 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c29f8bccc0 msgr2=0x55c29f8c4cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:24:05.400 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.398+0000 7fa7c11d9640 1 --2- 172.21.15.26:0/3473970033 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c29f8bccc0 0x55c29f8c4cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:05.400 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.399+0000 7fa7c11d9640 1 -- 172.21.15.26:0/3473970033 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c29f534210 con 0x55c29f8c7e80 2024-09-11T23:24:05.400 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.399+0000 7fa7c11d9640 1 --2- 172.21.15.26:0/3473970033 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c29f8c7e80 0x55c29f8c5730 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fa7b4010e30 tx=0x7fa7b4004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ee232ae4d53abed server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:24:05.400 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.399+0000 7fa7bbfff640 1 --2- 172.21.15.26:0/3473970033 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c29f8bd5e0 0x55c29f8c51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:24:05.400 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.400+0000 7fa7c09d8640 1 --2- 172.21.15.26:0/3473970033 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c29f8bccc0 0x55c29f8c4cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:24:05.400 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.400+0000 7fa7c59d6740 1 -- 172.21.15.26:0/3473970033 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa7ac0035f0 con 0x55c29f8c7e80 2024-09-11T23:24:05.441 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.441+0000 7fa7a07f8640 10 client.16461 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:24:05.441 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.441+0000 7fa7a07f8640 1 -- 172.21.15.26:0/3473970033 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7b402c060 con 0x55c29f8c7e80 2024-09-11T23:24:05.441 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.441+0000 7fa7a07f8640 1 -- 172.21.15.26:0/3473970033 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fa7b4017030 con 0x55c29f8c7e80 2024-09-11T23:24:05.441 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.441+0000 7fa7a07f8640 1 -- 172.21.15.26:0/3473970033 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fa7b4010e60 con 0x55c29f8c7e80 2024-09-11T23:24:05.445 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.445+0000 7fa7c59d6740 10 client.16461 fetch_fsmap learned FSMap version 41 2024-09-11T23:24:05.445 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.445+0000 7fa7c59d6740 1 -- 172.21.15.26:0/3473970033 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({fsmap=41}) -- 0x55c29f600620 con 0x55c29f8c7e80 2024-09-11T23:24:05.447 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.448+0000 7fa7a07f8640 1 -- 172.21.15.26:0/3473970033 <== mon.0 v2:172.21.15.26:3300/0 4 ==== fsmap(e 41) ==== 8426+0+0 (secure 0 0 0) 0x7fa7b401c3f0 con 0x55c29f8c7e80 2024-09-11T23:24:05.448 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.448+0000 7fa7c59d6740 10 client.16461 fetch_fsmap finished waiting for FSMap version 41 2024-09-11T23:24:05.448 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.448+0000 7fa7c59d6740 10 client.16461 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:24:05.448 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.448+0000 7fa7c59d6740 20 client.16461 populate_metadata read hostname 'smithi026' 2024-09-11T23:24:05.448 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.448+0000 7fa7c59d6740 1 --2- 172.21.15.26:0/3473970033 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55c29f8e91e0 0x55c29f8eb6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:05.448 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.448+0000 7fa7c59d6740 4 client.16461 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:24:05.449 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.448+0000 7fa7c59d6740 1 -- 172.21.15.26:0/3473970033 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c29f82ea20 con 0x55c29f8e91e0 2024-09-11T23:24:05.449 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.448+0000 7fa7c09d8640 1 --2- 172.21.15.26:0/3473970033 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55c29f8e91e0 0x55c29f8eb6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:05.449 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.449+0000 7fa7c09d8640 1 --2- 172.21.15.26:0/3473970033 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55c29f8e91e0 0x55c29f8eb6a0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a0234010d3fba7b8 server_cookie=be56a4a3db4e53d6 in_seq=0 out_seq=0 2024-09-11T23:24:05.457 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.457+0000 7fa7a07f8640 10 client.16461 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:24:05.457 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.457+0000 7fa7a07f8640 1 -- 172.21.15.26:0/3473970033 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c29f82ea20 con 0x55c29f8e91e0 2024-09-11T23:24:05.457 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.457+0000 7fa7a07f8640 10 client.16461 handle_command_reply: tid=0 2024-09-11T23:24:05.457 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.457+0000 7fa7c59d6740 1 client.16461 shutdown 2024-09-11T23:24:05.459 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.457+0000 7fa7c59d6740 1 -- 172.21.15.26:0/3473970033 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c29f8c7e80 msgr2=0x55c29f8c5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:24:05.459 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.457+0000 7fa7c59d6740 1 --2- 172.21.15.26:0/3473970033 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c29f8c7e80 0x55c29f8c5730 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fa7b4010e30 tx=0x7fa7b4004ab0 comp rx=0 tx=0).stop 2024-09-11T23:24:05.459 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.458+0000 7fa7c59d6740 1 -- 172.21.15.26:0/3473970033 shutdown_connections 2024-09-11T23:24:05.459 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.458+0000 7fa7c59d6740 1 --2- 172.21.15.26:0/3473970033 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55c29f8e91e0 0x55c29f8eb6a0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:05.459 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.458+0000 7fa7c59d6740 1 --2- 172.21.15.26:0/3473970033 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c29f8c7e80 0x55c29f8c5730 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:05.460 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.458+0000 7fa7c59d6740 1 --2- 172.21.15.26:0/3473970033 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c29f8bd5e0 0x55c29f8c51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:05.460 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.458+0000 7fa7c59d6740 1 --2- 172.21.15.26:0/3473970033 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c29f8bccc0 0x55c29f8c4cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:05.460 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.458+0000 7fa7c59d6740 1 -- 172.21.15.26:0/3473970033 >> 172.21.15.26:0/3473970033 conn(0x55c29f8b9230 msgr2=0x55c29f8bef40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:24:05.460 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.458+0000 7fa7a07f8640 0 client.16461 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:24:05.460 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.458+0000 7fa7c59d6740 1 -- 172.21.15.26:0/3473970033 shutdown_connections 2024-09-11T23:24:05.461 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.458+0000 7fa7c59d6740 1 -- 172.21.15.26:0/3473970033 wait complete. 2024-09-11T23:24:05.461 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.458+0000 7fa7c59d6740 20 client.16461 trim_cache size 0 max 16384 2024-09-11T23:24:05.753 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.753+0000 7fa7c59d6740 1 Processor -- start 2024-09-11T23:24:05.753 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.753+0000 7fa7c59d6740 1 -- start start 2024-09-11T23:24:05.758 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.758+0000 7fa7c59d6740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c29f8bccc0 0x55c29f8f0f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:05.758 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.758+0000 7fa7c59d6740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c29f8bd5e0 0x55c29f8f14b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:05.758 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.758+0000 7fa7c09d8640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c29f8bccc0 0x55c29f8f0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:05.758 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.758+0000 7fa7c09d8640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c29f8bccc0 0x55c29f8f0f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:56106/0 (socket says 172.21.15.26:56106) 2024-09-11T23:24:05.759 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.758+0000 7fa7c59d6740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c29f8c7e80 0x55c29f8c1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:05.759 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.759+0000 7fa7bbfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c29f8bd5e0 0x55c29f8f14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:05.759 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.759+0000 7fa7bbfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c29f8bd5e0 0x55c29f8f14b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44654/0 (socket says 172.21.15.26:44654) 2024-09-11T23:24:05.814 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.814+0000 7fa7c09d8640 1 -- 172.21.15.26:0/1017118544 learned_addr learned my addr 172.21.15.26:0/1017118544 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:24:05.815 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.814+0000 7fa7c59d6740 1 -- 172.21.15.26:0/1017118544 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fa7a800e850 con 0x55c29f8bd5e0 2024-09-11T23:24:05.815 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.814+0000 7fa7c59d6740 1 -- 172.21.15.26:0/1017118544 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fa7a8010690 con 0x55c29f8c7e80 2024-09-11T23:24:05.815 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.814+0000 7fa7c59d6740 1 -- 172.21.15.26:0/1017118544 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fa7a800efe0 con 0x55c29f8bccc0 2024-09-11T23:24:05.815 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.815+0000 7fa7c11d9640 1 --2- 172.21.15.26:0/1017118544 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c29f8c7e80 0x55c29f8c1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:05.815 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.815+0000 7fa7bbfff640 1 -- 172.21.15.26:0/1017118544 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c29f8bccc0 msgr2=0x55c29f8f0f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:24:05.815 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.815+0000 7fa7bbfff640 1 --2- 172.21.15.26:0/1017118544 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c29f8bccc0 0x55c29f8f0f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:05.816 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.815+0000 7fa7bbfff640 1 -- 172.21.15.26:0/1017118544 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c29f8c7e80 msgr2=0x55c29f8c1ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:24:05.816 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.815+0000 7fa7bbfff640 1 --2- 172.21.15.26:0/1017118544 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c29f8c7e80 0x55c29f8c1ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:05.816 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.815+0000 7fa7bbfff640 1 -- 172.21.15.26:0/1017118544 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c29f8c31e0 con 0x55c29f8bd5e0 2024-09-11T23:24:05.816 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.816+0000 7fa7bbfff640 1 --2- 172.21.15.26:0/1017118544 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c29f8bd5e0 0x55c29f8f14b0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fa7b00cdf20 tx=0x7fa7b000cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=892e6164e3412e74 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:24:05.818 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.818+0000 7fa7a27fc640 1 -- 172.21.15.26:0/1017118544 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7b00a10a0 con 0x55c29f8bd5e0 2024-09-11T23:24:05.818 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.818+0000 7fa7a27fc640 1 -- 172.21.15.26:0/1017118544 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7b00a5960 con 0x55c29f8bd5e0 2024-09-11T23:24:05.818 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.818+0000 7fa7a27fc640 1 -- 172.21.15.26:0/1017118544 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7b00d0de0 con 0x55c29f8bd5e0 2024-09-11T23:24:05.818 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.818+0000 7fa7c09d8640 1 --2- 172.21.15.26:0/1017118544 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c29f8bccc0 0x55c29f8f0f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:24:05.819 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.819+0000 7fa7c59d6740 1 -- 172.21.15.26:0/1017118544 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c29f8bd5e0 msgr2=0x55c29f8f14b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:24:05.820 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.820+0000 7fa7c59d6740 1 --2- 172.21.15.26:0/1017118544 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c29f8bd5e0 0x55c29f8f14b0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fa7b00cdf20 tx=0x7fa7b000cf20 comp rx=0 tx=0).stop 2024-09-11T23:24:05.824 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.824+0000 7fa7c59d6740 1 -- 172.21.15.26:0/1017118544 shutdown_connections 2024-09-11T23:24:05.824 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.824+0000 7fa7c59d6740 1 --2- 172.21.15.26:0/1017118544 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c29f8c7e80 0x55c29f8c1ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:05.824 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.824+0000 7fa7c59d6740 1 --2- 172.21.15.26:0/1017118544 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c29f8bd5e0 0x55c29f8f14b0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:05.824 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.824+0000 7fa7c59d6740 1 --2- 172.21.15.26:0/1017118544 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c29f8bccc0 0x55c29f8f0f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:05.824 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.824+0000 7fa7c59d6740 1 -- 172.21.15.26:0/1017118544 >> 172.21.15.26:0/1017118544 conn(0x55c29f8b9230 msgr2=0x55c29f8bff20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:24:05.829 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.830+0000 7fa7c59d6740 1 -- 172.21.15.26:0/1017118544 shutdown_connections 2024-09-11T23:24:05.830 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.830+0000 7fa7c59d6740 1 -- 172.21.15.26:0/1017118544 wait complete. 2024-09-11T23:24:05.830 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.831+0000 7fa7c59d6740 1 -- start start 2024-09-11T23:24:05.831 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.831+0000 7fa7c59d6740 1 Processor -- start 2024-09-11T23:24:05.831 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.831+0000 7fa7c59d6740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c29f8bccc0 0x55c29f94c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:05.832 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.832+0000 7fa7c59d6740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c29f8bd5e0 0x55c29f94c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:05.832 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.832+0000 7fa7c09d8640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c29f8bccc0 0x55c29f94c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:05.832 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.832+0000 7fa7c59d6740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c29f8c7e80 0x55c29f8cb4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:05.832 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.832+0000 7fa7c09d8640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c29f8bccc0 0x55c29f94c240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:56510/0 (socket says 172.21.15.26:56510) 2024-09-11T23:24:05.832 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.832+0000 7fa7bbfff640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c29f8bd5e0 0x55c29f94c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:05.832 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.832+0000 7fa7bbfff640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c29f8bd5e0 0x55c29f94c780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:56122/0 (socket says 172.21.15.26:56122) 2024-09-11T23:24:05.832 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.832+0000 7fa7c11d9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c29f8c7e80 0x55c29f8cb4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:05.833 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.832+0000 7fa7c11d9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c29f8c7e80 0x55c29f8cb4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44660/0 (socket says 172.21.15.26:44660) 2024-09-11T23:24:05.833 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.832+0000 7fa7c59d6740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fa7a800ff00 con 0x55c29f8c7e80 2024-09-11T23:24:05.834 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.832+0000 7fa7c59d6740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fa7a800f770 con 0x55c29f8bccc0 2024-09-11T23:24:05.834 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.832+0000 7fa7c59d6740 1 -- 172.21.15.26:0/2737613560 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fa7a800c830 con 0x55c29f8bd5e0 2024-09-11T23:24:05.834 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.832+0000 7fa7c09d8640 1 -- 172.21.15.26:0/2737613560 learned_addr learned my addr 172.21.15.26:0/2737613560 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:24:05.834 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.832+0000 7fa7c59d6740 10 client.0 _ll_register_callbacks cb 0x7fa7b4011560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:24:05.834 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.833+0000 7fa7bbfff640 1 -- 172.21.15.26:0/2737613560 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c29f8bccc0 msgr2=0x55c29f94c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:24:05.834 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.833+0000 7fa7bbfff640 1 --2- 172.21.15.26:0/2737613560 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c29f8bccc0 0x55c29f94c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:05.834 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.833+0000 7fa7bbfff640 1 -- 172.21.15.26:0/2737613560 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c29f8c7e80 msgr2=0x55c29f8cb4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:24:05.834 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.833+0000 7fa7bbfff640 1 --2- 172.21.15.26:0/2737613560 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c29f8c7e80 0x55c29f8cb4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:05.834 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.833+0000 7fa7bbfff640 1 -- 172.21.15.26:0/2737613560 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c29f8cbcf0 con 0x55c29f8bd5e0 2024-09-11T23:24:05.834 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.833+0000 7fa7c11d9640 1 --2- 172.21.15.26:0/2737613560 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c29f8c7e80 0x55c29f8cb4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:24:05.834 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.834+0000 7fa7bbfff640 1 --2- 172.21.15.26:0/2737613560 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c29f8bd5e0 0x55c29f94c780 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fa7b00cdf50 tx=0x7fa7b00a5700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3fab949734a33ee7 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:24:05.871 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.871+0000 7fa7a07f8640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-09-11T23:24:05.872 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.871+0000 7fa7a07f8640 1 -- 172.21.15.26:0/2737613560 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7b0010ac0 con 0x55c29f8bd5e0 2024-09-11T23:24:05.872 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.872+0000 7fa7a07f8640 1 -- 172.21.15.26:0/2737613560 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fa7b0010c60 con 0x55c29f8bd5e0 2024-09-11T23:24:05.872 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.872+0000 7fa7c59d6740 1 -- 172.21.15.26:0/2737613560 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa7b00a5960 con 0x55c29f8bd5e0 2024-09-11T23:24:05.878 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.878+0000 7fa7a07f8640 1 -- 172.21.15.26:0/2737613560 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fa7b00a5bb0 con 0x55c29f8bd5e0 2024-09-11T23:24:05.878 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.878+0000 7fa7c59d6740 10 client.26077 fetch_fsmap learned FSMap version 41 2024-09-11T23:24:05.878 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.878+0000 7fa7c59d6740 1 -- 172.21.15.26:0/2737613560 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55c29f9539a0 con 0x55c29f8bd5e0 2024-09-11T23:24:05.880 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.881+0000 7fa7a07f8640 1 -- 172.21.15.26:0/2737613560 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7b009c140 con 0x55c29f8bd5e0 2024-09-11T23:24:05.882 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.882+0000 7fa7a07f8640 1 -- 172.21.15.26:0/2737613560 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 41) ==== 8426+0+0 (secure 0 0 0) 0x7fa7b009bbb0 con 0x55c29f8bd5e0 2024-09-11T23:24:05.883 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.882+0000 7fa7c59d6740 10 client.26077 fetch_fsmap finished waiting for FSMap version 41 2024-09-11T23:24:05.883 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.882+0000 7fa7c59d6740 10 client.26077 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:24:05.883 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.882+0000 7fa7c59d6740 20 client.26077 populate_metadata read hostname 'smithi026' 2024-09-11T23:24:05.883 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.882+0000 7fa7c59d6740 1 --2- 172.21.15.26:0/2737613560 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55c29f953fc0 0x55c29f9563e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:05.883 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.883+0000 7fa7c59d6740 4 client.26077 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T23:24:05.883 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.883+0000 7fa7c59d6740 1 -- 172.21.15.26:0/2737613560 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7fa7b00a5960 con 0x55c29f953fc0 2024-09-11T23:24:05.883 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.883+0000 7fa7c09d8640 1 --2- 172.21.15.26:0/2737613560 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55c29f953fc0 0x55c29f9563e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:05.884 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.883+0000 7fa7c09d8640 1 --2- 172.21.15.26:0/2737613560 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55c29f953fc0 0x55c29f9563e0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e496d20e41199ea8 server_cookie=51fc8ecaf0148260 in_seq=0 out_seq=0 2024-09-11T23:24:05.886 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:05.886+0000 7fa7a07f8640 10 client.26077 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:24:06.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:24:05 smithi136 ceph-mon[35121]: pgmap v174: 97 pgs: 97 active+clean; 1.8 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 11 MiB/s wr, 600 op/s 2024-09-11T23:24:06.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:24:05 smithi026 ceph-mon[31172]: pgmap v174: 97 pgs: 97 active+clean; 1.8 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 11 MiB/s wr, 600 op/s 2024-09-11T23:24:06.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:24:05 smithi076 ceph-mon[35296]: pgmap v174: 97 pgs: 97 active+clean; 1.8 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 11 MiB/s wr, 600 op/s 2024-09-11T23:24:07.954 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:24:07 smithi026 ceph-mon[31172]: pgmap v175: 97 pgs: 97 active+clean; 1.8 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 10 MiB/s wr, 507 op/s 2024-09-11T23:24:08.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:24:07 smithi136 ceph-mon[35121]: pgmap v175: 97 pgs: 97 active+clean; 1.8 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 10 MiB/s wr, 507 op/s 2024-09-11T23:24:08.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:24:07 smithi076 ceph-mon[35296]: pgmap v175: 97 pgs: 97 active+clean; 1.8 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 10 MiB/s wr, 507 op/s 2024-09-11T23:24:10.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:24:09 smithi136 ceph-mon[35121]: pgmap v176: 97 pgs: 97 active+clean; 1.9 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 13 MiB/s wr, 673 op/s 2024-09-11T23:24:10.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:24:09 smithi026 ceph-mon[31172]: pgmap v176: 97 pgs: 97 active+clean; 1.9 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 13 MiB/s wr, 673 op/s 2024-09-11T23:24:10.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:24:09 smithi076 ceph-mon[35296]: pgmap v176: 97 pgs: 97 active+clean; 1.9 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 13 MiB/s wr, 673 op/s 2024-09-11T23:24:11.451 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:11.451+0000 7fa7a07f8640 1 -- 172.21.15.26:0/2737613560 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7fa7b00a5960 con 0x55c29f953fc0 2024-09-11T23:24:11.451 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:11.451+0000 7fa7a07f8640 10 client.26077 handle_command_reply: tid=0 2024-09-11T23:24:11.451 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:11.451+0000 7fa7c59d6740 1 client.26077 shutdown 2024-09-11T23:24:11.451 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:11.451+0000 7fa7c59d6740 1 -- 172.21.15.26:0/2737613560 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c29f8bd5e0 msgr2=0x55c29f94c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:24:11.452 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:11.451+0000 7fa7c59d6740 1 --2- 172.21.15.26:0/2737613560 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c29f8bd5e0 0x55c29f94c780 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fa7b00cdf50 tx=0x7fa7b00a5700 comp rx=0 tx=0).stop 2024-09-11T23:24:11.452 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:11.451+0000 7fa7c59d6740 1 -- 172.21.15.26:0/2737613560 shutdown_connections 2024-09-11T23:24:11.452 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:11.451+0000 7fa7c59d6740 1 --2- 172.21.15.26:0/2737613560 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55c29f953fc0 0x55c29f9563e0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:11.452 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:11.451+0000 7fa7c59d6740 1 --2- 172.21.15.26:0/2737613560 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c29f8c7e80 0x55c29f8cb4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:11.452 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:11.451+0000 7fa7c59d6740 1 --2- 172.21.15.26:0/2737613560 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c29f8bd5e0 0x55c29f94c780 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:11.452 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:11.451+0000 7fa7c59d6740 1 --2- 172.21.15.26:0/2737613560 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c29f8bccc0 0x55c29f94c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:11.452 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:11.451+0000 7fa7c59d6740 1 -- 172.21.15.26:0/2737613560 >> 172.21.15.26:0/2737613560 conn(0x55c29f8b9230 msgr2=0x55c29f8b77b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:24:11.452 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:11.451+0000 7fa7a07f8640 0 client.26077 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:24:11.452 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:11.451+0000 7fa7c59d6740 1 -- 172.21.15.26:0/2737613560 shutdown_connections 2024-09-11T23:24:11.452 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:11.451+0000 7fa7c59d6740 1 -- 172.21.15.26:0/2737613560 wait complete. 2024-09-11T23:24:11.452 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:11.451+0000 7fa7c59d6740 20 client.26077 trim_cache size 0 max 16384 2024-09-11T23:24:11.453 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-09-11T23:24:11.453 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "message": "", 2024-09-11T23:24:11.453 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-09-11T23:24:11.453 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-09-11T23:24:11.454 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:11.452+0000 7fa7c2c3c640 1 -- 172.21.15.26:0/2408499968 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fa7a40a2f30 msgr2=0x7fa7a40a53f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:24:11.454 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:11.452+0000 7fa7c2c3c640 1 --2- 172.21.15.26:0/2408499968 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fa7a40a2f30 0x7fa7a40a53f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fa7b4011960 tx=0x7fa7b4009720 comp rx=0 tx=0).stop 2024-09-11T23:24:11.454 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:11.452+0000 7fa7c2c3c640 1 -- 172.21.15.26:0/2408499968 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa7bc1095d0 msgr2=0x7fa7bc1a75b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:24:11.454 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:11.452+0000 7fa7c2c3c640 1 --2- 172.21.15.26:0/2408499968 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa7bc1095d0 0x7fa7bc1a75b0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fa7b000e9f0 tx=0x7fa7b000eeb0 comp rx=0 tx=0).stop 2024-09-11T23:24:11.454 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:11.453+0000 7fa7c2c3c640 1 -- 172.21.15.26:0/2408499968 shutdown_connections 2024-09-11T23:24:11.454 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:11.453+0000 7fa7c2c3c640 1 --2- 172.21.15.26:0/2408499968 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fa7a40a2f30 0x7fa7a40a53f0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:11.454 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:11.453+0000 7fa7c2c3c640 1 --2- 172.21.15.26:0/2408499968 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa7bc11b140 0x7fa7bc1ab930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:11.454 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:11.453+0000 7fa7c2c3c640 1 --2- 172.21.15.26:0/2408499968 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa7bc1095d0 0x7fa7bc1a75b0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:11.454 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:11.453+0000 7fa7c2c3c640 1 --2- 172.21.15.26:0/2408499968 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa7bc1091f0 0x7fa7bc1a7070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:11.454 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:11.453+0000 7fa7c2c3c640 1 -- 172.21.15.26:0/2408499968 >> 172.21.15.26:0/2408499968 conn(0x7fa7bc06cc60 msgr2=0x7fa7bc11b7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:24:11.454 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:11.453+0000 7fa7c2c3c640 1 -- 172.21.15.26:0/2408499968 shutdown_connections 2024-09-11T23:24:11.455 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:11.453+0000 7fa7c2c3c640 1 -- 172.21.15.26:0/2408499968 wait complete. 2024-09-11T23:24:12.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:24:11 smithi136 ceph-mon[35121]: pgmap v177: 97 pgs: 97 active+clean; 1.9 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 19 MiB/s wr, 1.29k op/s 2024-09-11T23:24:12.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:24:11 smithi026 ceph-mon[31172]: pgmap v177: 97 pgs: 97 active+clean; 1.9 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 19 MiB/s wr, 1.29k op/s 2024-09-11T23:24:12.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:24:11 smithi076 ceph-mon[35296]: pgmap v177: 97 pgs: 97 active+clean; 1.9 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 19 MiB/s wr, 1.29k op/s 2024-09-11T23:24:13.579 INFO:tasks.workunit.client.0.smithi026.stdout: 517 1403582 708 949507 591 368338 1256 2024-09-11T23:24:14.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:24:13 smithi136 ceph-mon[35121]: pgmap v178: 97 pgs: 97 active+clean; 1.9 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 17 MiB/s wr, 1.18k op/s 2024-09-11T23:24:14.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:24:13 smithi026 ceph-mon[31172]: pgmap v178: 97 pgs: 97 active+clean; 1.9 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 17 MiB/s wr, 1.18k op/s 2024-09-11T23:24:14.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:24:13 smithi076 ceph-mon[35296]: pgmap v178: 97 pgs: 97 active+clean; 1.9 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 17 MiB/s wr, 1.18k op/s 2024-09-11T23:24:16.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:24:15 smithi136 ceph-mon[35121]: pgmap v179: 97 pgs: 97 active+clean; 2.0 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 25 MiB/s wr, 1.55k op/s 2024-09-11T23:24:16.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:24:15 smithi026 ceph-mon[31172]: pgmap v179: 97 pgs: 97 active+clean; 2.0 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 25 MiB/s wr, 1.55k op/s 2024-09-11T23:24:16.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:24:15 smithi076 ceph-mon[35296]: pgmap v179: 97 pgs: 97 active+clean; 2.0 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 25 MiB/s wr, 1.55k op/s 2024-09-11T23:24:18.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:24:17 smithi136 ceph-mon[35121]: pgmap v180: 97 pgs: 97 active+clean; 2.0 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 21 MiB/s wr, 1.43k op/s 2024-09-11T23:24:18.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:24:17 smithi026 ceph-mon[31172]: pgmap v180: 97 pgs: 97 active+clean; 2.0 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 21 MiB/s wr, 1.43k op/s 2024-09-11T23:24:18.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:24:17 smithi076 ceph-mon[35296]: pgmap v180: 97 pgs: 97 active+clean; 2.0 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 21 MiB/s wr, 1.43k op/s 2024-09-11T23:24:19.980 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:24:19 smithi026 ceph-mon[31172]: pgmap v181: 97 pgs: 97 active+clean; 2.0 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 21 MiB/s wr, 1.41k op/s 2024-09-11T23:24:20.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:24:19 smithi136 ceph-mon[35121]: pgmap v181: 97 pgs: 97 active+clean; 2.0 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 21 MiB/s wr, 1.41k op/s 2024-09-11T23:24:20.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:24:19 smithi076 ceph-mon[35296]: pgmap v181: 97 pgs: 97 active+clean; 2.0 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 21 MiB/s wr, 1.41k op/s 2024-09-11T23:24:22.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:24:21 smithi136 ceph-mon[35121]: pgmap v182: 97 pgs: 97 active+clean; 2.0 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 20 MiB/s wr, 1.29k op/s 2024-09-11T23:24:22.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:24:21 smithi026 ceph-mon[31172]: pgmap v182: 97 pgs: 97 active+clean; 2.0 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 20 MiB/s wr, 1.29k op/s 2024-09-11T23:24:22.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:24:21 smithi076 ceph-mon[35296]: pgmap v182: 97 pgs: 97 active+clean; 2.0 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 20 MiB/s wr, 1.29k op/s 2024-09-11T23:24:23.579 INFO:tasks.workunit.client.0.smithi026.stdout: 551 1980129 1683 1337532 1973 556918 2822 2024-09-11T23:24:24.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:24:23 smithi136 ceph-mon[35121]: pgmap v183: 97 pgs: 97 active+clean; 2.0 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 13 MiB/s wr, 634 op/s 2024-09-11T23:24:24.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:24:23 smithi026 ceph-mon[31172]: pgmap v183: 97 pgs: 97 active+clean; 2.0 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 13 MiB/s wr, 634 op/s 2024-09-11T23:24:24.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:24:23 smithi076 ceph-mon[35296]: pgmap v183: 97 pgs: 97 active+clean; 2.0 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 13 MiB/s wr, 634 op/s 2024-09-11T23:24:26.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:24:25 smithi136 ceph-mon[35121]: pgmap v184: 97 pgs: 97 active+clean; 2.1 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 15 MiB/s wr, 744 op/s 2024-09-11T23:24:26.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:24:25 smithi026 ceph-mon[31172]: pgmap v184: 97 pgs: 97 active+clean; 2.1 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 15 MiB/s wr, 744 op/s 2024-09-11T23:24:26.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:24:25 smithi076 ceph-mon[35296]: pgmap v184: 97 pgs: 97 active+clean; 2.1 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 15 MiB/s wr, 744 op/s 2024-09-11T23:24:26.714 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:26.714+0000 7f3756e53640 1 -- 172.21.15.26:0/2202612636 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f37501091f0 msgr2=0x7f3750110250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:24:26.714 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:26.714+0000 7f3756e53640 1 --2- 172.21.15.26:0/2202612636 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f37501091f0 0x7f3750110250 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f3748009ca0 tx=0x7f374802a8f0 comp rx=0 tx=0).stop 2024-09-11T23:24:26.716 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:26.716+0000 7f3756e53640 1 -- 172.21.15.26:0/2202612636 shutdown_connections 2024-09-11T23:24:26.716 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:26.716+0000 7f3756e53640 1 --2- 172.21.15.26:0/2202612636 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f375011b140 0x7f375011d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:26.716 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:26.716+0000 7f3756e53640 1 --2- 172.21.15.26:0/2202612636 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f37501095d0 0x7f3750109a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:26.716 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:26.716+0000 7f3756e53640 1 --2- 172.21.15.26:0/2202612636 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f37501091f0 0x7f3750110250 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:26.717 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:26.716+0000 7f3756e53640 1 -- 172.21.15.26:0/2202612636 >> 172.21.15.26:0/2202612636 conn(0x7f375006cc60 msgr2=0x7f375006d070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:24:26.717 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:26.716+0000 7f3756e53640 1 -- 172.21.15.26:0/2202612636 shutdown_connections 2024-09-11T23:24:26.718 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:26.716+0000 7f3756e53640 1 -- 172.21.15.26:0/2202612636 wait complete. 2024-09-11T23:24:26.718 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:26.719+0000 7f3756e53640 1 Processor -- start 2024-09-11T23:24:26.719 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:26.719+0000 7f3756e53640 1 -- start start 2024-09-11T23:24:26.719 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:26.719+0000 7f3756e53640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f37501091f0 0x7f3750118750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:26.719 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:26.719+0000 7f3756e53640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f37501095d0 0x7f37501117d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:26.719 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:26.719+0000 7f3756e53640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f375011b140 0x7f3750111d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:26.719 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:26.719+0000 7f3756e53640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f3750120b60 con 0x7f37501095d0 2024-09-11T23:24:26.719 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:26.719+0000 7f3756e53640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f37501209e0 con 0x7f37501091f0 2024-09-11T23:24:26.720 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:26.719+0000 7f3756e53640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f3750120ce0 con 0x7f375011b140 2024-09-11T23:24:26.720 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:26.720+0000 7f37553f0640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f375011b140 0x7f3750111d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:26.720 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:26.720+0000 7f37553f0640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f375011b140 0x7f3750111d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:34342/0 (socket says 172.21.15.26:34342) 2024-09-11T23:24:26.720 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:26.720+0000 7f37553f0640 1 -- 172.21.15.26:0/1969579524 learned_addr learned my addr 172.21.15.26:0/1969579524 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:24:26.721 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:26.721+0000 7f374ffff640 1 --2- 172.21.15.26:0/1969579524 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f37501095d0 0x7f37501117d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:26.721 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:26.721+0000 7f37553f0640 1 -- 172.21.15.26:0/1969579524 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f37501091f0 msgr2=0x7f3750118750 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-11T23:24:26.721 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:26.721+0000 7f37553f0640 1 --2- 172.21.15.26:0/1969579524 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f37501091f0 0x7f3750118750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:26.721 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:26.721+0000 7f37553f0640 1 -- 172.21.15.26:0/1969579524 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f37501095d0 msgr2=0x7f37501117d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:24:26.721 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:26.721+0000 7f37553f0640 1 --2- 172.21.15.26:0/1969579524 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f37501095d0 0x7f37501117d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:26.721 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:26.721+0000 7f37553f0640 1 -- 172.21.15.26:0/1969579524 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3750112350 con 0x7f375011b140 2024-09-11T23:24:26.721 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:26.721+0000 7f374ffff640 1 --2- 172.21.15.26:0/1969579524 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f37501095d0 0x7f37501117d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:24:26.725 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:26.726+0000 7f37553f0640 1 --2- 172.21.15.26:0/1969579524 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f375011b140 0x7f3750111d10 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f374400e970 tx=0x7f374400ee30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:24:26.750 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:26.750+0000 7f374dffb640 1 -- 172.21.15.26:0/1969579524 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f374400ccc0 con 0x7f375011b140 2024-09-11T23:24:26.755 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:26.750+0000 7f3756e53640 1 -- 172.21.15.26:0/1969579524 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3750116270 con 0x7f375011b140 2024-09-11T23:24:26.755 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:26.750+0000 7f3756e53640 1 -- 172.21.15.26:0/1969579524 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37501167c0 con 0x7f375011b140 2024-09-11T23:24:26.755 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:26.755+0000 7f3759bec740 1 Processor -- start 2024-09-11T23:24:26.756 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:26.755+0000 7f3759bec740 1 -- start start 2024-09-11T23:24:26.756 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:26.755+0000 7f3759bec740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55571aa98630 0x55571aa96df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:26.756 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:26.756+0000 7f3759bec740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55571aa97330 0x55571aa97770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:26.756 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:26.756+0000 7f3759bec740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55571aaa1c80 0x55571aaa4070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:26.756 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:26.756+0000 7f3759bec740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55571a9fb2f0 con 0x55571aa98630 2024-09-11T23:24:26.756 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:26.756+0000 7f3759bec740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55571a9a3270 con 0x55571aa97330 2024-09-11T23:24:26.756 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:26.756+0000 7f3759bec740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55571a9f72b0 con 0x55571aaa1c80 2024-09-11T23:24:26.757 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:26.756+0000 7f374ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55571aa98630 0x55571aa96df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:26.757 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:26.757+0000 7f374ffff640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55571aaa1c80 0x55571aaa4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:26.757 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:26.757+0000 7f374ffff640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55571aaa1c80 0x55571aaa4070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:34358/0 (socket says 172.21.15.26:34358) 2024-09-11T23:24:26.757 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:26.757+0000 7f374ffff640 1 -- 172.21.15.26:0/3264464083 learned_addr learned my addr 172.21.15.26:0/3264464083 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:24:26.760 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:26.760+0000 7f374ffff640 1 -- 172.21.15.26:0/3264464083 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55571aa97330 msgr2=0x55571aa97770 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T23:24:26.760 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:26.760+0000 7f374ffff640 1 --2- 172.21.15.26:0/3264464083 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55571aa97330 0x55571aa97770 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:26.761 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:26.760+0000 7f374ffff640 1 -- 172.21.15.26:0/3264464083 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55571aa98630 msgr2=0x55571aa96df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:24:26.761 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:26.760+0000 7f374ffff640 1 --2- 172.21.15.26:0/3264464083 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55571aa98630 0x55571aa96df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:26.761 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:26.760+0000 7f374ffff640 1 -- 172.21.15.26:0/3264464083 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55571a9a2340 con 0x55571aaa1c80 2024-09-11T23:24:26.761 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:26.761+0000 7f374ffff640 1 --2- 172.21.15.26:0/3264464083 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55571aaa1c80 0x55571aaa4070 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f3740012900 tx=0x7f374000df90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=52adae2646536609 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:24:26.761 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:26.761+0000 7f374dffb640 1 -- 172.21.15.26:0/1969579524 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3744005500 con 0x7f375011b140 2024-09-11T23:24:26.761 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:26.761+0000 7f374dffb640 1 -- 172.21.15.26:0/1969579524 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3744010690 con 0x7f375011b140 2024-09-11T23:24:26.762 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:26.763+0000 7f374dffb640 1 -- 172.21.15.26:0/1969579524 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f37440108f0 con 0x7f375011b140 2024-09-11T23:24:26.764 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:26.764+0000 7f374dffb640 1 --2- 172.21.15.26:0/1969579524 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f37380a2f30 0x7f37380a53f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:26.775 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:26.775+0000 7f3754bef640 1 --2- 172.21.15.26:0/1969579524 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f37380a2f30 0x7f37380a53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:26.776 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:26.776+0000 7f374dffb640 1 -- 172.21.15.26:0/1969579524 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f37440d0800 con 0x7f375011b140 2024-09-11T23:24:26.779 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:26.780+0000 7f3754bef640 1 --2- 172.21.15.26:0/1969579524 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f37380a2f30 0x7f37380a53f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f3748004af0 tx=0x7f3748000f40 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:24:26.784 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:26.784+0000 7f3756e53640 1 -- 172.21.15.26:0/3264464083 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3740004220 con 0x55571aaa1c80 2024-09-11T23:24:26.784 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:26.784+0000 7f3756e53640 1 -- 172.21.15.26:0/3264464083 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3740010e60 con 0x55571aaa1c80 2024-09-11T23:24:26.785 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:26.785+0000 7f3759bec740 1 -- 172.21.15.26:0/3264464083 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55571aaa1c80 msgr2=0x55571aaa4070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:24:26.785 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:26.785+0000 7f3759bec740 1 --2- 172.21.15.26:0/3264464083 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55571aaa1c80 0x55571aaa4070 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f3740012900 tx=0x7f374000df90 comp rx=0 tx=0).stop 2024-09-11T23:24:26.786 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:26.786+0000 7f3759bec740 1 -- 172.21.15.26:0/3264464083 shutdown_connections 2024-09-11T23:24:26.786 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:26.786+0000 7f3759bec740 1 --2- 172.21.15.26:0/3264464083 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55571aa97330 0x55571aa97770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:26.786 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:26.786+0000 7f3759bec740 1 --2- 172.21.15.26:0/3264464083 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55571aaa1c80 0x55571aaa4070 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:26.786 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:26.786+0000 7f3759bec740 1 --2- 172.21.15.26:0/3264464083 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55571aa98630 0x55571aa96df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:26.786 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:26.786+0000 7f3759bec740 1 -- 172.21.15.26:0/3264464083 >> 172.21.15.26:0/3264464083 conn(0x55571aa93570 msgr2=0x55571aa93950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:24:26.787 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:26.787+0000 7f3759bec740 1 -- 172.21.15.26:0/3264464083 shutdown_connections 2024-09-11T23:24:26.788 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:26.787+0000 7f3759bec740 1 -- 172.21.15.26:0/3264464083 wait complete. 2024-09-11T23:24:26.788 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:26.788+0000 7f3759bec740 1 -- start start 2024-09-11T23:24:26.788 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:26.788+0000 7f3759bec740 1 Processor -- start 2024-09-11T23:24:26.789 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:26.788+0000 7f3759bec740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55571aa97330 0x55571aa9eb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:26.790 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:26.788+0000 7f3759bec740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55571aa98630 0x55571aa9f070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:26.790 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:26.788+0000 7f3759bec740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55571aaa1c80 0x55571aa9f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:26.790 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:26.788+0000 7f3759bec740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55571aaa4b60 con 0x55571aaa1c80 2024-09-11T23:24:26.790 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:26.788+0000 7f3759bec740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55571a984290 con 0x55571aa97330 2024-09-11T23:24:26.790 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:26.788+0000 7f3759bec740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55571aaa4ce0 con 0x55571aa98630 2024-09-11T23:24:26.791 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:26.789+0000 7f3759bec740 10 client.0 _ll_register_callbacks cb 0x55571a9b5140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:24:26.791 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:26.789+0000 7f374ffff640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55571aa98630 0x55571aa9f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:26.791 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:26.789+0000 7f374ffff640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55571aa98630 0x55571aa9f070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:34360/0 (socket says 172.21.15.26:34360) 2024-09-11T23:24:26.791 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:26.789+0000 7f374ffff640 1 -- 172.21.15.26:0/4070015933 learned_addr learned my addr 172.21.15.26:0/4070015933 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:24:26.791 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:26.790+0000 7f3754bef640 1 --2- 172.21.15.26:0/4070015933 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55571aa97330 0x55571aa9eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:26.792 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:26.790+0000 7f374ffff640 1 -- 172.21.15.26:0/4070015933 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55571aa97330 msgr2=0x55571aa9eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:24:26.792 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:26.790+0000 7f374ffff640 1 --2- 172.21.15.26:0/4070015933 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55571aa97330 0x55571aa9eb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:26.792 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:26.790+0000 7f374ffff640 1 -- 172.21.15.26:0/4070015933 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55571aaa1c80 msgr2=0x55571aa9f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:24:26.792 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:26.790+0000 7f374ffff640 1 --2- 172.21.15.26:0/4070015933 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55571aaa1c80 0x55571aa9f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:26.793 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:26.790+0000 7f374ffff640 1 -- 172.21.15.26:0/4070015933 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55571a70e210 con 0x55571aa98630 2024-09-11T23:24:26.793 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:26.790+0000 7f374ffff640 1 --2- 172.21.15.26:0/4070015933 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55571aa98630 0x55571aa9f070 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f3740001eb0 tx=0x7f3740012680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d30bf82ba177680b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:24:26.818 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:26.818+0000 7f37347f8640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-09-11T23:24:26.818 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:26.818+0000 7f37347f8640 1 -- 172.21.15.26:0/4070015933 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3740010b90 con 0x55571aa98630 2024-09-11T23:24:26.818 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:26.818+0000 7f37347f8640 1 -- 172.21.15.26:0/4070015933 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f37400049c0 con 0x55571aa98630 2024-09-11T23:24:26.819 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:26.818+0000 7f3759bec740 1 -- 172.21.15.26:0/4070015933 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3740010e60 con 0x55571aa98630 2024-09-11T23:24:26.832 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:26.832+0000 7f37347f8640 1 -- 172.21.15.26:0/4070015933 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f3740004bc0 con 0x55571aa98630 2024-09-11T23:24:26.832 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:26.832+0000 7f3759bec740 10 client.26089 fetch_fsmap learned FSMap version 41 2024-09-11T23:24:26.832 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:26.832+0000 7f3759bec740 1 -- 172.21.15.26:0/4070015933 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55571a7da620 con 0x55571aa98630 2024-09-11T23:24:26.850 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:26.850+0000 7f37347f8640 1 -- 172.21.15.26:0/4070015933 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3740010e60 con 0x55571aa98630 2024-09-11T23:24:26.850 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:26.850+0000 7f37347f8640 1 -- 172.21.15.26:0/4070015933 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 41) ==== 8426+0+0 (secure 0 0 0) 0x7f37400319c0 con 0x55571aa98630 2024-09-11T23:24:26.850 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:26.850+0000 7f3759bec740 10 client.26089 fetch_fsmap finished waiting for FSMap version 41 2024-09-11T23:24:26.850 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:26.850+0000 7f3759bec740 10 client.26089 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:24:26.850 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:26.850+0000 7f3759bec740 20 client.26089 populate_metadata read hostname 'smithi026' 2024-09-11T23:24:26.850 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:26.850+0000 7f3759bec740 1 --2- 172.21.15.26:0/4070015933 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55571aab23f0 0x55571aab4870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:26.851 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:26.850+0000 7f3759bec740 4 client.26089 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:24:26.851 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:26.850+0000 7f3759bec740 1 -- 172.21.15.26:0/4070015933 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55571aa08a20 con 0x55571aab23f0 2024-09-11T23:24:26.851 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:26.851+0000 7f3754bef640 1 --2- 172.21.15.26:0/4070015933 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55571aab23f0 0x55571aab4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:26.852 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:26.852+0000 7f3754bef640 1 --2- 172.21.15.26:0/4070015933 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55571aab23f0 0x55571aab4870 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=712a9f5a85df1bfc server_cookie=a184d8a4ab457feb in_seq=0 out_seq=0 2024-09-11T23:24:26.857 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:26.857+0000 7f37347f8640 10 client.26089 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:24:26.857 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:26.857+0000 7f37347f8640 1 -- 172.21.15.26:0/4070015933 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55571aa08a20 con 0x55571aab23f0 2024-09-11T23:24:26.857 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:26.857+0000 7f37347f8640 10 client.26089 handle_command_reply: tid=0 2024-09-11T23:24:26.857 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:26.857+0000 7f3759bec740 1 client.26089 shutdown 2024-09-11T23:24:26.858 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:26.857+0000 7f3759bec740 1 -- 172.21.15.26:0/4070015933 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55571aa98630 msgr2=0x55571aa9f070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:24:26.858 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:26.857+0000 7f3759bec740 1 --2- 172.21.15.26:0/4070015933 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55571aa98630 0x55571aa9f070 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f3740001eb0 tx=0x7f3740012680 comp rx=0 tx=0).stop 2024-09-11T23:24:26.858 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:26.858+0000 7f3759bec740 1 -- 172.21.15.26:0/4070015933 shutdown_connections 2024-09-11T23:24:26.858 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:26.858+0000 7f3759bec740 1 --2- 172.21.15.26:0/4070015933 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55571aab23f0 0x55571aab4870 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:26.858 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:26.858+0000 7f3759bec740 1 --2- 172.21.15.26:0/4070015933 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55571aaa1c80 0x55571aa9f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:26.858 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:26.858+0000 7f3759bec740 1 --2- 172.21.15.26:0/4070015933 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55571aa98630 0x55571aa9f070 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:26.859 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:26.858+0000 7f3759bec740 1 --2- 172.21.15.26:0/4070015933 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55571aa97330 0x55571aa9eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:26.859 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:26.858+0000 7f3759bec740 1 -- 172.21.15.26:0/4070015933 >> 172.21.15.26:0/4070015933 conn(0x55571aa93570 msgr2=0x55571aa992d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:24:26.859 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:26.858+0000 7f37347f8640 0 client.26089 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:24:26.859 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:26.858+0000 7f3759bec740 1 -- 172.21.15.26:0/4070015933 shutdown_connections 2024-09-11T23:24:26.859 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:26.858+0000 7f3759bec740 1 -- 172.21.15.26:0/4070015933 wait complete. 2024-09-11T23:24:26.859 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:26.858+0000 7f3759bec740 20 client.26089 trim_cache size 0 max 16384 2024-09-11T23:24:27.122 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:27.122+0000 7f3759bec740 1 Processor -- start 2024-09-11T23:24:27.122 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:27.122+0000 7f3759bec740 1 -- start start 2024-09-11T23:24:27.122 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:27.123+0000 7f3759bec740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55571aa97330 0x55571aa9f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:27.123 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:27.123+0000 7f3759bec740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55571aa98630 0x55571aaa0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:27.123 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:27.123+0000 7f3754bef640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55571aa97330 0x55571aa9f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:27.123 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:27.123+0000 7f3754bef640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55571aa97330 0x55571aa9f3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:51418/0 (socket says 172.21.15.26:51418) 2024-09-11T23:24:27.124 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:27.124+0000 7f374ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55571aa98630 0x55571aaa0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:27.125 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:27.124+0000 7f374ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55571aa98630 0x55571aaa0aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46204/0 (socket says 172.21.15.26:46204) 2024-09-11T23:24:27.125 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:27.124+0000 7f3754bef640 1 -- 172.21.15.26:0/2419285941 learned_addr learned my addr 172.21.15.26:0/2419285941 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:24:27.125 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:27.124+0000 7f3759bec740 1 --2- 172.21.15.26:0/2419285941 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55571aaa1c80 0x55571aaba180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:27.125 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:27.124+0000 7f3759bec740 1 -- 172.21.15.26:0/2419285941 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f373c00e8c0 con 0x55571aa98630 2024-09-11T23:24:27.125 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:27.124+0000 7f37553f0640 1 --2- 172.21.15.26:0/2419285941 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55571aaa1c80 0x55571aaba180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:27.125 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:27.125+0000 7f3759bec740 1 -- 172.21.15.26:0/2419285941 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f373c010700 con 0x55571aa97330 2024-09-11T23:24:27.125 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:27.125+0000 7f3759bec740 1 -- 172.21.15.26:0/2419285941 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f373c00f050 con 0x55571aaa1c80 2024-09-11T23:24:27.126 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:27.126+0000 7f374ffff640 1 -- 172.21.15.26:0/2419285941 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55571aaa1c80 msgr2=0x55571aaba180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:24:27.126 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:27.126+0000 7f374ffff640 1 --2- 172.21.15.26:0/2419285941 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55571aaa1c80 0x55571aaba180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:27.126 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:27.126+0000 7f374ffff640 1 -- 172.21.15.26:0/2419285941 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55571aa97330 msgr2=0x55571aa9f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:24:27.126 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:27.126+0000 7f374ffff640 1 --2- 172.21.15.26:0/2419285941 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55571aa97330 0x55571aa9f3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:27.126 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:27.126+0000 7f374ffff640 1 -- 172.21.15.26:0/2419285941 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55571aa933a0 con 0x55571aa98630 2024-09-11T23:24:27.126 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:27.126+0000 7f374ffff640 1 --2- 172.21.15.26:0/2419285941 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55571aa98630 0x55571aaa0aa0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f3740010740 tx=0x7f374000db00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=600c1e75f718993a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:24:27.166 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:27.166+0000 7f37367fc640 1 -- 172.21.15.26:0/2419285941 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3740036070 con 0x55571aa98630 2024-09-11T23:24:27.166 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:27.166+0000 7f37367fc640 1 -- 172.21.15.26:0/2419285941 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f374002a660 con 0x55571aa98630 2024-09-11T23:24:27.166 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:27.166+0000 7f37367fc640 1 -- 172.21.15.26:0/2419285941 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3740021030 con 0x55571aa98630 2024-09-11T23:24:27.168 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:27.168+0000 7f3759bec740 1 -- 172.21.15.26:0/2419285941 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55571aa98630 msgr2=0x55571aaa0aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:24:27.168 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:27.168+0000 7f3759bec740 1 --2- 172.21.15.26:0/2419285941 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55571aa98630 0x55571aaa0aa0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f3740010740 tx=0x7f374000db00 comp rx=0 tx=0).stop 2024-09-11T23:24:27.168 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:27.168+0000 7f3759bec740 1 -- 172.21.15.26:0/2419285941 shutdown_connections 2024-09-11T23:24:27.169 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:27.169+0000 7f3759bec740 1 --2- 172.21.15.26:0/2419285941 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55571aaa1c80 0x55571aaba180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:27.169 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:27.169+0000 7f3759bec740 1 --2- 172.21.15.26:0/2419285941 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55571aa98630 0x55571aaa0aa0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:27.169 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:27.169+0000 7f3759bec740 1 --2- 172.21.15.26:0/2419285941 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55571aa97330 0x55571aa9f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:27.169 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:27.169+0000 7f3759bec740 1 -- 172.21.15.26:0/2419285941 >> 172.21.15.26:0/2419285941 conn(0x55571aa93570 msgr2=0x55571aaa0390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:24:27.170 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:27.171+0000 7f3759bec740 1 -- 172.21.15.26:0/2419285941 shutdown_connections 2024-09-11T23:24:27.171 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:27.171+0000 7f3759bec740 1 -- 172.21.15.26:0/2419285941 wait complete. 2024-09-11T23:24:27.171 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:27.172+0000 7f3759bec740 1 -- start start 2024-09-11T23:24:27.172 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:27.172+0000 7f3759bec740 1 Processor -- start 2024-09-11T23:24:27.173 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:27.173+0000 7f3759bec740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55571aa97330 0x55571aaa55f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:27.173 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:27.173+0000 7f3759bec740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55571aa98630 0x55571aaa5b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:27.174 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:27.173+0000 7f3754bef640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55571aa97330 0x55571aaa55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:27.174 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:27.173+0000 7f3754bef640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55571aa97330 0x55571aaa55f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:51420/0 (socket says 172.21.15.26:51420) 2024-09-11T23:24:27.174 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:27.174+0000 7f3754bef640 1 -- 172.21.15.26:0/4228390686 learned_addr learned my addr 172.21.15.26:0/4228390686 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:24:27.174 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:27.174+0000 7f3759bec740 1 --2- 172.21.15.26:0/4228390686 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55571aaa1c80 0x55571ab1b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:27.174 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:27.174+0000 7f374ffff640 1 --2- 172.21.15.26:0/4228390686 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55571aa98630 0x55571aaa5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:27.175 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:27.174+0000 7f37553f0640 1 --2- 172.21.15.26:0/4228390686 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55571aaa1c80 0x55571ab1b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:27.175 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:27.175+0000 7f3759bec740 1 -- 172.21.15.26:0/4228390686 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f373c00ff70 con 0x55571aaa1c80 2024-09-11T23:24:27.175 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:27.175+0000 7f3759bec740 1 -- 172.21.15.26:0/4228390686 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f373c00f7e0 con 0x55571aa97330 2024-09-11T23:24:27.175 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:27.175+0000 7f3759bec740 1 -- 172.21.15.26:0/4228390686 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f373c00c8a0 con 0x55571aa98630 2024-09-11T23:24:27.176 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:27.176+0000 7f374ffff640 1 -- 172.21.15.26:0/4228390686 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55571aa97330 msgr2=0x55571aaa55f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:24:27.177 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:27.176+0000 7f374ffff640 1 --2- 172.21.15.26:0/4228390686 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55571aa97330 0x55571aaa55f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:27.177 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:27.176+0000 7f374ffff640 1 -- 172.21.15.26:0/4228390686 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55571aaa1c80 msgr2=0x55571ab1b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:24:27.177 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:27.176+0000 7f374ffff640 1 --2- 172.21.15.26:0/4228390686 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55571aaa1c80 0x55571ab1b140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:27.177 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:27.176+0000 7f374ffff640 1 -- 172.21.15.26:0/4228390686 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55571ab1b680 con 0x55571aa98630 2024-09-11T23:24:27.177 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:27.176+0000 7f3754bef640 1 --2- 172.21.15.26:0/4228390686 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55571aa97330 0x55571aaa55f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:24:27.177 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:27.176+0000 7f37553f0640 1 --2- 172.21.15.26:0/4228390686 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55571aaa1c80 0x55571ab1b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:24:27.177 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:27.177+0000 7f374ffff640 1 --2- 172.21.15.26:0/4228390686 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55571aa98630 0x55571aaa5b30 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f37400058c0 tx=0x7f374000d720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4741c92442f2e018 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:24:27.178 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:27.177+0000 7f3759bec740 10 client.0 _ll_register_callbacks cb 0x7f374000de40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:24:27.178 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:27.177+0000 7f3759bec740 1 -- 172.21.15.26:0/4228390686 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f374002a660 con 0x55571aa98630 2024-09-11T23:24:27.201 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:27.201+0000 7f37347f8640 10 client.26095 ms_handle_connect on v2:172.21.15.136:3300/0 2024-09-11T23:24:27.201 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:27.201+0000 7f37347f8640 1 -- 172.21.15.26:0/4228390686 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3740036030 con 0x55571aa98630 2024-09-11T23:24:27.202 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:27.201+0000 7f37347f8640 1 -- 172.21.15.26:0/4228390686 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f3740024030 con 0x55571aa98630 2024-09-11T23:24:27.202 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:27.201+0000 7f37347f8640 1 -- 172.21.15.26:0/4228390686 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f3740021070 con 0x55571aa98630 2024-09-11T23:24:27.202 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:27.201+0000 7f3759bec740 10 client.26095 fetch_fsmap learned FSMap version 41 2024-09-11T23:24:27.202 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:27.201+0000 7f3759bec740 1 -- 172.21.15.26:0/4228390686 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({fsmap=41}) -- 0x55571ab1c570 con 0x55571aa98630 2024-09-11T23:24:27.206 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:27.205+0000 7f37347f8640 1 -- 172.21.15.26:0/4228390686 <== mon.1 v2:172.21.15.136:3300/0 4 ==== fsmap(e 41) ==== 8426+0+0 (secure 0 0 0) 0x7f374002a300 con 0x55571aa98630 2024-09-11T23:24:27.206 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:27.206+0000 7f3759bec740 10 client.26095 fetch_fsmap finished waiting for FSMap version 41 2024-09-11T23:24:27.206 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:27.206+0000 7f3759bec740 10 client.26095 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:24:27.206 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:27.206+0000 7f3759bec740 20 client.26095 populate_metadata read hostname 'smithi026' 2024-09-11T23:24:27.206 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:27.206+0000 7f3759bec740 1 --2- 172.21.15.26:0/4228390686 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55571ab1cce0 0x55571ab1f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:27.206 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:27.206+0000 7f3759bec740 4 client.26095 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T23:24:27.206 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:27.206+0000 7f3759bec740 1 -- 172.21.15.26:0/4228390686 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f374002a660 con 0x55571ab1cce0 2024-09-11T23:24:27.207 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:27.206+0000 7f3754bef640 1 --2- 172.21.15.26:0/4228390686 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55571ab1cce0 0x55571ab1f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:27.207 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:27.207+0000 7f3754bef640 1 --2- 172.21.15.26:0/4228390686 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55571ab1cce0 0x55571ab1f1a0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b5f714ec0923980f server_cookie=b5abc55869a61c9f in_seq=0 out_seq=0 2024-09-11T23:24:27.209 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:27.210+0000 7f37347f8640 10 client.26095 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:24:28.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:24:27 smithi026 ceph-mon[31172]: pgmap v185: 97 pgs: 97 active+clean; 2.1 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 7.1 MiB/s wr, 405 op/s 2024-09-11T23:24:28.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:24:27 smithi076 ceph-mon[35296]: pgmap v185: 97 pgs: 97 active+clean; 2.1 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 7.1 MiB/s wr, 405 op/s 2024-09-11T23:24:28.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:24:27 smithi136 ceph-mon[35121]: pgmap v185: 97 pgs: 97 active+clean; 2.1 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 7.1 MiB/s wr, 405 op/s 2024-09-11T23:24:30.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:24:30 smithi076 ceph-mon[35296]: pgmap v186: 97 pgs: 97 active+clean; 2.1 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 5.7 MiB/s wr, 322 op/s 2024-09-11T23:24:30.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:24:30 smithi136 ceph-mon[35121]: pgmap v186: 97 pgs: 97 active+clean; 2.1 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 5.7 MiB/s wr, 322 op/s 2024-09-11T23:24:30.504 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:24:30 smithi026 ceph-mon[31172]: pgmap v186: 97 pgs: 97 active+clean; 2.1 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 5.7 MiB/s wr, 322 op/s 2024-09-11T23:24:32.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:24:32 smithi076 ceph-mon[35296]: pgmap v187: 97 pgs: 97 active+clean; 2.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 9.6 MiB/s wr, 590 op/s 2024-09-11T23:24:32.435 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:32.433+0000 7f37347f8640 1 -- 172.21.15.26:0/4228390686 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f374002a660 con 0x55571ab1cce0 2024-09-11T23:24:32.435 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:32.433+0000 7f37347f8640 10 client.26095 handle_command_reply: tid=0 2024-09-11T23:24:32.435 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:32.433+0000 7f3759bec740 1 client.26095 shutdown 2024-09-11T23:24:32.435 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:32.434+0000 7f3759bec740 1 -- 172.21.15.26:0/4228390686 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55571aa98630 msgr2=0x55571aaa5b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:24:32.435 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:32.434+0000 7f3759bec740 1 --2- 172.21.15.26:0/4228390686 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55571aa98630 0x55571aaa5b30 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f37400058c0 tx=0x7f374000d720 comp rx=0 tx=0).stop 2024-09-11T23:24:32.436 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:32.434+0000 7f3759bec740 1 -- 172.21.15.26:0/4228390686 shutdown_connections 2024-09-11T23:24:32.436 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:32.434+0000 7f3759bec740 1 --2- 172.21.15.26:0/4228390686 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55571ab1cce0 0x55571ab1f1a0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:32.436 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:32.434+0000 7f3759bec740 1 --2- 172.21.15.26:0/4228390686 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55571aaa1c80 0x55571ab1b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:32.436 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:32.434+0000 7f3759bec740 1 --2- 172.21.15.26:0/4228390686 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55571aa98630 0x55571aaa5b30 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:32.436 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:32.434+0000 7f37347f8640 0 client.26095 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:24:32.436 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:32.434+0000 7f3759bec740 1 --2- 172.21.15.26:0/4228390686 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55571aa97330 0x55571aaa55f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:32.436 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:32.434+0000 7f3759bec740 1 -- 172.21.15.26:0/4228390686 >> 172.21.15.26:0/4228390686 conn(0x55571aa93570 msgr2=0x55571aa926d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:24:32.436 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:32.434+0000 7f3759bec740 1 -- 172.21.15.26:0/4228390686 shutdown_connections 2024-09-11T23:24:32.436 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:32.434+0000 7f3759bec740 1 -- 172.21.15.26:0/4228390686 wait complete. 2024-09-11T23:24:32.436 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:32.435+0000 7f3759bec740 20 client.26095 trim_cache size 0 max 16384 2024-09-11T23:24:32.437 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-09-11T23:24:32.437 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "message": "", 2024-09-11T23:24:32.437 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-09-11T23:24:32.437 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-09-11T23:24:32.460 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:32.459+0000 7f37377fe640 1 -- 172.21.15.26:0/1969579524 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f37380a2f30 msgr2=0x7f37380a53f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:24:32.469 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:32.468+0000 7f37377fe640 1 --2- 172.21.15.26:0/1969579524 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f37380a2f30 0x7f37380a53f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f3748004af0 tx=0x7f3748000f40 comp rx=0 tx=0).stop 2024-09-11T23:24:32.469 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:32.469+0000 7f37377fe640 1 -- 172.21.15.26:0/1969579524 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f375011b140 msgr2=0x7f3750111d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:24:32.469 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:32.469+0000 7f37377fe640 1 --2- 172.21.15.26:0/1969579524 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f375011b140 0x7f3750111d10 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f374400e970 tx=0x7f374400ee30 comp rx=0 tx=0).stop 2024-09-11T23:24:32.478 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:32.477+0000 7f37377fe640 1 -- 172.21.15.26:0/1969579524 shutdown_connections 2024-09-11T23:24:32.478 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:32.477+0000 7f37377fe640 1 --2- 172.21.15.26:0/1969579524 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f37380a2f30 0x7f37380a53f0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:32.478 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:32.477+0000 7f37377fe640 1 --2- 172.21.15.26:0/1969579524 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f375011b140 0x7f3750111d10 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:32.478 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:32.477+0000 7f37377fe640 1 --2- 172.21.15.26:0/1969579524 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f37501095d0 0x7f37501117d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:32.478 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:32.477+0000 7f37377fe640 1 --2- 172.21.15.26:0/1969579524 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f37501091f0 0x7f3750118750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:32.478 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:32.477+0000 7f37377fe640 1 -- 172.21.15.26:0/1969579524 >> 172.21.15.26:0/1969579524 conn(0x7f375006cc60 msgr2=0x7f375011b700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:24:32.478 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:32.477+0000 7f37377fe640 1 -- 172.21.15.26:0/1969579524 shutdown_connections 2024-09-11T23:24:32.478 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:32.477+0000 7f37377fe640 1 -- 172.21.15.26:0/1969579524 wait complete. 2024-09-11T23:24:32.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:24:32 smithi136 ceph-mon[35121]: pgmap v187: 97 pgs: 97 active+clean; 2.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 9.6 MiB/s wr, 590 op/s 2024-09-11T23:24:32.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:24:32 smithi026 ceph-mon[31172]: pgmap v187: 97 pgs: 97 active+clean; 2.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 9.6 MiB/s wr, 590 op/s 2024-09-11T23:24:33.579 INFO:tasks.workunit.client.0.smithi026.stdout: 577 1699090 1689 1155105 1414 505296 1863 2024-09-11T23:24:34.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:24:34 smithi076 ceph-mon[35296]: pgmap v188: 97 pgs: 97 active+clean; 2.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 9.1 MiB/s wr, 553 op/s 2024-09-11T23:24:34.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:24:34 smithi136 ceph-mon[35121]: pgmap v188: 97 pgs: 97 active+clean; 2.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 9.1 MiB/s wr, 553 op/s 2024-09-11T23:24:34.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:24:34 smithi026 ceph-mon[31172]: pgmap v188: 97 pgs: 97 active+clean; 2.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 9.1 MiB/s wr, 553 op/s 2024-09-11T23:24:36.278 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 20.1 2024-09-11T23:24:36.279 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:24:36.302 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:24:36 smithi026 ceph-mon[31172]: pgmap v189: 97 pgs: 97 active+clean; 2.2 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 17 MiB/s wr, 1.21k op/s 2024-09-11T23:24:36.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:24:36 smithi076 ceph-mon[35296]: pgmap v189: 97 pgs: 97 active+clean; 2.2 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 17 MiB/s wr, 1.21k op/s 2024-09-11T23:24:36.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:24:36 smithi136 ceph-mon[35121]: pgmap v189: 97 pgs: 97 active+clean; 2.2 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 17 MiB/s wr, 1.21k op/s 2024-09-11T23:24:36.601 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:24:38.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:38.032+0000 7fa0489aa640 1 -- 172.21.15.26:0/2696705038 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa044108290 msgr2=0x7fa04410ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:24:38.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:38.032+0000 7fa0489aa640 1 --2- 172.21.15.26:0/2696705038 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa044108290 0x7fa04410ec30 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fa03c009cd0 tx=0x7fa03c02a8e0 comp rx=0 tx=0).stop 2024-09-11T23:24:38.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:38.032+0000 7fa0489aa640 1 -- 172.21.15.26:0/2696705038 shutdown_connections 2024-09-11T23:24:38.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:38.032+0000 7fa0489aa640 1 --2- 172.21.15.26:0/2696705038 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa044103740 0x7fa044107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:38.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:38.032+0000 7fa0489aa640 1 --2- 172.21.15.26:0/2696705038 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa044108290 0x7fa04410ec30 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:38.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:38.032+0000 7fa0489aa640 1 --2- 172.21.15.26:0/2696705038 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa044102d90 0x7fa044103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:38.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:38.032+0000 7fa0489aa640 1 -- 172.21.15.26:0/2696705038 >> 172.21.15.26:0/2696705038 conn(0x7fa0440fc7d0 msgr2=0x7fa0440febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:24:38.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:38.032+0000 7fa0489aa640 1 -- 172.21.15.26:0/2696705038 shutdown_connections 2024-09-11T23:24:38.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:38.032+0000 7fa0489aa640 1 -- 172.21.15.26:0/2696705038 wait complete. 2024-09-11T23:24:38.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:38.034+0000 7fa0489aa640 1 Processor -- start 2024-09-11T23:24:38.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:38.034+0000 7fa0489aa640 1 -- start start 2024-09-11T23:24:38.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:38.034+0000 7fa0489aa640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa044102d90 0x7fa044198540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:38.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:38.034+0000 7fa0489aa640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa044103740 0x7fa044198a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:38.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:38.034+0000 7fa0489aa640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa044108290 0x7fa04419ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:38.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:38.034+0000 7fa0489aa640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fa044112090 con 0x7fa044102d90 2024-09-11T23:24:38.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:38.034+0000 7fa0489aa640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fa044111f10 con 0x7fa044108290 2024-09-11T23:24:38.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:38.034+0000 7fa0489aa640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fa044112210 con 0x7fa044103740 2024-09-11T23:24:38.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:38.034+0000 7fa042d9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa044108290 0x7fa04419ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:38.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:38.034+0000 7fa041d9b640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa044103740 0x7fa044198a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:38.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:38.034+0000 7fa04259c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa044102d90 0x7fa044198540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:38.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:38.034+0000 7fa042d9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa044108290 0x7fa04419ce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:39358/0 (socket says 172.21.15.26:39358) 2024-09-11T23:24:38.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:38.034+0000 7fa042d9d640 1 -- 172.21.15.26:0/3321955561 learned_addr learned my addr 172.21.15.26:0/3321955561 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:24:38.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:38.035+0000 7fa042d9d640 1 -- 172.21.15.26:0/3321955561 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa044103740 msgr2=0x7fa044198a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:24:38.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:38.035+0000 7fa042d9d640 1 --2- 172.21.15.26:0/3321955561 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa044103740 0x7fa044198a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:38.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:38.035+0000 7fa042d9d640 1 -- 172.21.15.26:0/3321955561 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa044102d90 msgr2=0x7fa044198540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:24:38.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:38.035+0000 7fa042d9d640 1 --2- 172.21.15.26:0/3321955561 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa044102d90 0x7fa044198540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:38.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:38.035+0000 7fa042d9d640 1 -- 172.21.15.26:0/3321955561 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa04419d4f0 con 0x7fa044108290 2024-09-11T23:24:38.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:38.035+0000 7fa041d9b640 1 --2- 172.21.15.26:0/3321955561 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa044103740 0x7fa044198a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:24:38.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:38.035+0000 7fa042d9d640 1 --2- 172.21.15.26:0/3321955561 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa044108290 0x7fa04419ce10 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fa03c005cc0 tx=0x7fa03c0093d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:24:38.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:38.035+0000 7fa0337fe640 1 -- 172.21.15.26:0/3321955561 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa03c003e00 con 0x7fa044108290 2024-09-11T23:24:38.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:38.035+0000 7fa0489aa640 1 -- 172.21.15.26:0/3321955561 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa04419d780 con 0x7fa044108290 2024-09-11T23:24:38.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:38.035+0000 7fa0337fe640 1 -- 172.21.15.26:0/3321955561 <== mon.2 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa03c004520 con 0x7fa044108290 2024-09-11T23:24:38.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:38.035+0000 7fa0337fe640 1 -- 172.21.15.26:0/3321955561 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa03c030db0 con 0x7fa044108290 2024-09-11T23:24:38.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:38.035+0000 7fa0489aa640 1 -- 172.21.15.26:0/3321955561 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0441a50c0 con 0x7fa044108290 2024-09-11T23:24:38.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:38.036+0000 7fa0489aa640 1 -- 172.21.15.26:0/3321955561 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa04406a7f0 con 0x7fa044108290 2024-09-11T23:24:38.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:38.038+0000 7fa0337fe640 1 -- 172.21.15.26:0/3321955561 <== mon.2 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fa03c0046d0 con 0x7fa044108290 2024-09-11T23:24:38.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:38.038+0000 7fa0337fe640 1 --2- 172.21.15.26:0/3321955561 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fa02c0a2cf0 0x7fa02c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:38.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:38.039+0000 7fa04259c640 1 --2- 172.21.15.26:0/3321955561 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fa02c0a2cf0 0x7fa02c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:38.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:38.039+0000 7fa0337fe640 1 -- 172.21.15.26:0/3321955561 <== mon.2 v2:172.21.15.76:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fa03c02e030 con 0x7fa044108290 2024-09-11T23:24:38.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:38.039+0000 7fa04259c640 1 --2- 172.21.15.26:0/3321955561 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fa02c0a2cf0 0x7fa02c0a51b0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fa034009bd0 tx=0x7fa034005f90 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:24:38.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:38.046+0000 7fa0337fe640 1 -- 172.21.15.26:0/3321955561 <== mon.2 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa03c04b060 con 0x7fa044108290 2024-09-11T23:24:38.254 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:24:38 smithi026 ceph-mon[31172]: pgmap v190: 97 pgs: 97 active+clean; 2.2 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 18 MiB/s wr, 1.44k op/s 2024-09-11T23:24:38.305 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:24:38 smithi076 ceph-mon[35296]: pgmap v190: 97 pgs: 97 active+clean; 2.2 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 18 MiB/s wr, 1.44k op/s 2024-09-11T23:24:38.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:38.379+0000 7fa0489aa640 1 -- 172.21.15.26:0/3321955561 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa04410a2a0 con 0x7fa044108290 2024-09-11T23:24:38.382 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-11T23:24:38.382 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":41,"btime":"2024-09-11T23:23:46:852133+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14499,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/367659515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":367659515},{"type":"v1","addr":"172.21.15.26:6835","nonce":367659515}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14511,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/2041065299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":2041065299},{"type":"v1","addr":"172.21.15.26:6837","nonce":2041065299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":41,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:16:26.544203+0000","modified":"2024-09-11T23:23:46.128951+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24485,"mds_1":24463,"mds_2":24458,"mds_3":24448,"mds_4":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14550":{"gid":14550,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6839/2233068305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":2233068305},{"type":"v1","addr":"172.21.15.26:6839","nonce":2233068305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14562":{"gid":14562,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6841/2984982336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":2984982336},{"type":"v1","addr":"172.21.15.26:6841","nonce":2984982336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6833/2019841455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":2019841455},{"type":"v1","addr":"172.21.15.136:6833","nonce":2019841455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.76:6833/3438147050","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3438147050},{"type":"v1","addr":"172.21.15.76:6833","nonce":3438147050}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6835/3617665975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3617665975},{"type":"v1","addr":"172.21.15.136:6835","nonce":3617665975}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":12,"addr":"172.21.15.76:6835/393308499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":393308499},{"type":"v1","addr":"172.21.15.76:6835","nonce":393308499}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":10,"addr":"172.21.15.136:6837/1633082387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":1633082387},{"type":"v1","addr":"172.21.15.136:6837","nonce":1633082387}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.76:6837/2078978748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":2078978748},{"type":"v1","addr":"172.21.15.76:6837","nonce":2078978748}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":63,"addr":"172.21.15.136:6839/3211398630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":3211398630},{"type":"v1","addr":"172.21.15.136:6839","nonce":3211398630}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.76:6839/2077637090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6838","nonce":2077637090},{"type":"v1","addr":"172.21.15.76:6839","nonce":2077637090}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24485,"qdb_cluster":[24443,24448,24458,24463,24485]},"id":1}]} 2024-09-11T23:24:38.382 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 41 2024-09-11T23:24:38.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:38.380+0000 7fa0337fe640 1 -- 172.21.15.26:0/3321955561 <== mon.2 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) ==== 94+0+12000 (secure 0 0 0) 0x7fa03c0bd810 con 0x7fa044108290 2024-09-11T23:24:38.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:38.390+0000 7fa0317fa640 1 -- 172.21.15.26:0/3321955561 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fa02c0a2cf0 msgr2=0x7fa02c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:24:38.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:38.390+0000 7fa0317fa640 1 --2- 172.21.15.26:0/3321955561 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fa02c0a2cf0 0x7fa02c0a51b0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fa034009bd0 tx=0x7fa034005f90 comp rx=0 tx=0).stop 2024-09-11T23:24:38.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:38.390+0000 7fa0317fa640 1 -- 172.21.15.26:0/3321955561 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa044108290 msgr2=0x7fa04419ce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:24:38.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:38.390+0000 7fa0317fa640 1 --2- 172.21.15.26:0/3321955561 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa044108290 0x7fa04419ce10 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fa03c005cc0 tx=0x7fa03c0093d0 comp rx=0 tx=0).stop 2024-09-11T23:24:38.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:38.392+0000 7fa0317fa640 1 -- 172.21.15.26:0/3321955561 shutdown_connections 2024-09-11T23:24:38.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:38.392+0000 7fa0317fa640 1 --2- 172.21.15.26:0/3321955561 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fa02c0a2cf0 0x7fa02c0a51b0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:38.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:38.392+0000 7fa0317fa640 1 --2- 172.21.15.26:0/3321955561 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa044108290 0x7fa04419ce10 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:38.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:38.392+0000 7fa0317fa640 1 --2- 172.21.15.26:0/3321955561 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa044103740 0x7fa044198a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:38.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:38.392+0000 7fa0317fa640 1 --2- 172.21.15.26:0/3321955561 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa044102d90 0x7fa044198540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:38.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:38.392+0000 7fa0317fa640 1 -- 172.21.15.26:0/3321955561 >> 172.21.15.26:0/3321955561 conn(0x7fa0440fc7d0 msgr2=0x7fa044106350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:24:38.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:38.395+0000 7fa0317fa640 1 -- 172.21.15.26:0/3321955561 shutdown_connections 2024-09-11T23:24:38.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:38.398+0000 7fa0317fa640 1 -- 172.21.15.26:0/3321955561 wait complete. 2024-09-11T23:24:38.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:24:38 smithi136 ceph-mon[35121]: pgmap v190: 97 pgs: 97 active+clean; 2.2 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 18 MiB/s wr, 1.44k op/s 2024-09-11T23:24:39.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:24:39 smithi076 ceph-mon[35296]: from='client.? 172.21.15.26:0/3321955561' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:24:39.468 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:24:39 smithi026 ceph-mon[31172]: from='client.? 172.21.15.26:0/3321955561' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:24:39.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:24:39 smithi136 ceph-mon[35121]: from='client.? 172.21.15.26:0/3321955561' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:24:39.710 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24485 quiesce db / --timeout 90 --expiration 140.1 --await-for 45.0' 2024-09-11T23:24:39.711 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24485 quiesce db / --timeout 90 --expiration 140.1 --await-for 45.0 2024-09-11T23:24:40.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:24:40 smithi136 ceph-mon[35121]: pgmap v191: 97 pgs: 97 active+clean; 2.2 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 18 MiB/s wr, 1.41k op/s 2024-09-11T23:24:40.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:24:40 smithi026 ceph-mon[31172]: pgmap v191: 97 pgs: 97 active+clean; 2.2 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 18 MiB/s wr, 1.41k op/s 2024-09-11T23:24:40.552 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:24:40 smithi076 ceph-mon[35296]: pgmap v191: 97 pgs: 97 active+clean; 2.2 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 18 MiB/s wr, 1.41k op/s 2024-09-11T23:24:41.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:24:41 smithi136 ceph-mon[35121]: pgmap v192: 97 pgs: 97 active+clean; 2.3 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 24 MiB/s wr, 1.68k op/s 2024-09-11T23:24:42.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:24:41 smithi026 ceph-mon[31172]: pgmap v192: 97 pgs: 97 active+clean; 2.3 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 24 MiB/s wr, 1.68k op/s 2024-09-11T23:24:42.052 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:24:41 smithi076 ceph-mon[35296]: pgmap v192: 97 pgs: 97 active+clean; 2.3 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 24 MiB/s wr, 1.68k op/s 2024-09-11T23:24:43.579 INFO:tasks.workunit.client.0.smithi026.stdout: 598 1512319 1061 1029246 1043 447542 1448 2024-09-11T23:24:43.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:24:43 smithi136 ceph-mon[35121]: pgmap v193: 97 pgs: 97 active+clean; 2.3 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 19 MiB/s wr, 1.40k op/s 2024-09-11T23:24:43.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:24:43 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:24:44.000 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:24:43 smithi026 ceph-mon[31172]: pgmap v193: 97 pgs: 97 active+clean; 2.3 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 19 MiB/s wr, 1.40k op/s 2024-09-11T23:24:44.000 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:24:43 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:24:44.052 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:24:43 smithi076 ceph-mon[35296]: pgmap v193: 97 pgs: 97 active+clean; 2.3 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 19 MiB/s wr, 1.40k op/s 2024-09-11T23:24:44.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:24:43 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:24:44.295 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:24:45.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:45.871+0000 7f7abf72a640 1 -- 172.21.15.26:0/748795767 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7ab8109450 msgr2=0x7f7ab8110250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:24:45.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:45.871+0000 7f7abf72a640 1 --2- 172.21.15.26:0/748795767 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7ab8109450 0x7f7ab8110250 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f7ab0009ca0 tx=0x7f7ab002a8f0 comp rx=0 tx=0).stop 2024-09-11T23:24:45.874 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:24:45 smithi026 ceph-mon[31172]: pgmap v194: 97 pgs: 97 active+clean; 2.3 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 20 MiB/s wr, 1.41k op/s 2024-09-11T23:24:45.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:45.876+0000 7f7abf72a640 1 -- 172.21.15.26:0/748795767 shutdown_connections 2024-09-11T23:24:45.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:45.876+0000 7f7abf72a640 1 --2- 172.21.15.26:0/748795767 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7ab811b140 0x7f7ab811d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:45.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:45.876+0000 7f7abf72a640 1 --2- 172.21.15.26:0/748795767 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7ab8109830 0x7f7ab8109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:45.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:45.876+0000 7f7abf72a640 1 --2- 172.21.15.26:0/748795767 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7ab8109450 0x7f7ab8110250 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:45.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:45.876+0000 7f7abf72a640 1 -- 172.21.15.26:0/748795767 >> 172.21.15.26:0/748795767 conn(0x7f7ab806e140 msgr2=0x7f7ab806e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:24:45.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:45.882+0000 7f7abf72a640 1 -- 172.21.15.26:0/748795767 shutdown_connections 2024-09-11T23:24:45.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:45.886+0000 7f7abf72a640 1 -- 172.21.15.26:0/748795767 wait complete. 2024-09-11T23:24:45.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:45.887+0000 7f7abf72a640 1 Processor -- start 2024-09-11T23:24:45.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:45.888+0000 7f7abf72a640 1 -- start start 2024-09-11T23:24:45.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:45.889+0000 7f7abf72a640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7ab8109450 0x7f7ab81a7470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:45.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:45.889+0000 7f7abf72a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7ab8109830 0x7f7ab81a79b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:45.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:45.889+0000 7f7abf72a640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7ab811b140 0x7f7ab81abd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:45.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:45.889+0000 7f7abf72a640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f7ab8120ed0 con 0x7f7ab8109830 2024-09-11T23:24:45.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:45.889+0000 7f7abf72a640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f7ab8120d50 con 0x7f7ab811b140 2024-09-11T23:24:45.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:45.889+0000 7f7abf72a640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f7ab8121050 con 0x7f7ab8109450 2024-09-11T23:24:45.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:45.889+0000 7f7abd4c6640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7ab8109450 0x7f7ab81a7470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:45.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:45.889+0000 7f7abd4c6640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7ab8109450 0x7f7ab81a7470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:38118/0 (socket says 172.21.15.26:38118) 2024-09-11T23:24:45.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:45.889+0000 7f7abd4c6640 1 -- 172.21.15.26:0/582185800 learned_addr learned my addr 172.21.15.26:0/582185800 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:24:45.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:45.890+0000 7f7abd4c6640 1 -- 172.21.15.26:0/582185800 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7ab811b140 msgr2=0x7f7ab81abd40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-11T23:24:45.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:45.890+0000 7f7abd4c6640 1 --2- 172.21.15.26:0/582185800 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7ab811b140 0x7f7ab81abd40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:45.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:45.890+0000 7f7abd4c6640 1 -- 172.21.15.26:0/582185800 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7ab8109830 msgr2=0x7f7ab81a79b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-11T23:24:45.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:45.890+0000 7f7abd4c6640 1 --2- 172.21.15.26:0/582185800 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7ab8109830 0x7f7ab81a79b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:45.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:45.890+0000 7f7abd4c6640 1 -- 172.21.15.26:0/582185800 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ab81ac420 con 0x7f7ab8109450 2024-09-11T23:24:45.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:45.891+0000 7f7abd4c6640 1 --2- 172.21.15.26:0/582185800 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7ab8109450 0x7f7ab81a7470 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f7ab0006010 tx=0x7f7ab0031730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:24:45.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:45.892+0000 7f7aae7fc640 1 -- 172.21.15.26:0/582185800 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ab0004040 con 0x7f7ab8109450 2024-09-11T23:24:45.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:45.892+0000 7f7abf72a640 1 -- 172.21.15.26:0/582185800 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ab81ac6b0 con 0x7f7ab8109450 2024-09-11T23:24:45.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:45.892+0000 7f7abf72a640 1 -- 172.21.15.26:0/582185800 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ab81b3ff0 con 0x7f7ab8109450 2024-09-11T23:24:45.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:45.894+0000 7f7aae7fc640 1 -- 172.21.15.26:0/582185800 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ab002d070 con 0x7f7ab8109450 2024-09-11T23:24:45.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:45.894+0000 7f7aae7fc640 1 -- 172.21.15.26:0/582185800 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ab002fac0 con 0x7f7ab8109450 2024-09-11T23:24:45.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:45.899+0000 7f7aae7fc640 1 -- 172.21.15.26:0/582185800 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f7ab002fca0 con 0x7f7ab8109450 2024-09-11T23:24:45.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:45.901+0000 7f7aae7fc640 1 --2- 172.21.15.26:0/582185800 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f7aa40a2d70 0x7f7aa40a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:45.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:45.901+0000 7f7aae7fc640 1 -- 172.21.15.26:0/582185800 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f7ab00f1bd0 con 0x7f7ab8109450 2024-09-11T23:24:45.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:45.903+0000 7f7ac1b03740 1 Processor -- start 2024-09-11T23:24:45.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:45.904+0000 7f7abccc5640 1 --2- 172.21.15.26:0/582185800 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f7aa40a2d70 0x7f7aa40a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:45.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:45.905+0000 7f7abccc5640 1 --2- 172.21.15.26:0/582185800 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f7aa40a2d70 0x7f7aa40a5230 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f7ab811d8d0 tx=0x7f7aa8006bd0 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:24:45.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:45.908+0000 7f7ac1b03740 1 -- start start 2024-09-11T23:24:45.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:45.908+0000 7f7ac1b03740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559f5ed82760 0x559f5ed82b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:45.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:45.908+0000 7f7ac1b03740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559f5ed83080 0x559f5ed8d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:45.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:45.908+0000 7f7ac1b03740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559f5ed8d940 0x559f5ed8fd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:45.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:45.909+0000 7f7abd4c6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559f5ed82760 0x559f5ed82b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:45.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:45.909+0000 7f7abd4c6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559f5ed82760 0x559f5ed82b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59812/0 (socket says 172.21.15.26:59812) 2024-09-11T23:24:45.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:45.909+0000 7f7ac1b03740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x559f5ece82a0 con 0x559f5ed82760 2024-09-11T23:24:45.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:45.909+0000 7f7ac1b03740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x559f5ec8fb10 con 0x559f5ed8d940 2024-09-11T23:24:45.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:45.909+0000 7f7ac1b03740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x559f5ece4260 con 0x559f5ed83080 2024-09-11T23:24:45.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:45.909+0000 7f7abdcc7640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559f5ed8d940 0x559f5ed8fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:45.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:45.909+0000 7f7abdcc7640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559f5ed8d940 0x559f5ed8fd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:40760/0 (socket says 172.21.15.26:40760) 2024-09-11T23:24:45.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:45.909+0000 7f7abdcc7640 1 -- 172.21.15.26:0/914459119 learned_addr learned my addr 172.21.15.26:0/914459119 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:24:45.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:45.910+0000 7f7abdcc7640 1 -- 172.21.15.26:0/914459119 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559f5ed83080 msgr2=0x559f5ed8d400 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T23:24:45.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:45.910+0000 7f7abdcc7640 1 --2- 172.21.15.26:0/914459119 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559f5ed83080 0x559f5ed8d400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:45.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:45.910+0000 7f7abdcc7640 1 -- 172.21.15.26:0/914459119 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559f5ed82760 msgr2=0x559f5ed82b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:24:45.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:45.910+0000 7f7abdcc7640 1 --2- 172.21.15.26:0/914459119 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559f5ed82760 0x559f5ed82b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:45.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:45.910+0000 7f7abdcc7640 1 -- 172.21.15.26:0/914459119 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f5ea59b90 con 0x559f5ed8d940 2024-09-11T23:24:45.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:45.910+0000 7f7abdcc7640 1 --2- 172.21.15.26:0/914459119 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559f5ed8d940 0x559f5ed8fd30 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f7ab40096e0 tx=0x7f7ab4009ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f0a657f4af1f6d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:24:45.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:45.912+0000 7f7aa37fe640 1 -- 172.21.15.26:0/914459119 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ab4015020 con 0x559f5ed8d940 2024-09-11T23:24:45.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:45.912+0000 7f7aa37fe640 1 -- 172.21.15.26:0/914459119 <== mon.2 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ab400bda0 con 0x559f5ed8d940 2024-09-11T23:24:45.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:45.914+0000 7f7ac1b03740 1 -- 172.21.15.26:0/914459119 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559f5ed8d940 msgr2=0x559f5ed8fd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:24:45.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:45.915+0000 7f7ac1b03740 1 --2- 172.21.15.26:0/914459119 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559f5ed8d940 0x559f5ed8fd30 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f7ab40096e0 tx=0x7f7ab4009ba0 comp rx=0 tx=0).stop 2024-09-11T23:24:45.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:45.920+0000 7f7ac1b03740 1 -- 172.21.15.26:0/914459119 shutdown_connections 2024-09-11T23:24:45.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:45.920+0000 7f7ac1b03740 1 --2- 172.21.15.26:0/914459119 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559f5ed8d940 0x559f5ed8fd30 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:45.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:45.920+0000 7f7ac1b03740 1 --2- 172.21.15.26:0/914459119 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559f5ed83080 0x559f5ed8d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:45.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:45.920+0000 7f7ac1b03740 1 --2- 172.21.15.26:0/914459119 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559f5ed82760 0x559f5ed82b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:45.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:45.920+0000 7f7ac1b03740 1 -- 172.21.15.26:0/914459119 >> 172.21.15.26:0/914459119 conn(0x559f5ed7ecf0 msgr2=0x559f5ed7f0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:24:45.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:45.923+0000 7f7ac1b03740 1 -- 172.21.15.26:0/914459119 shutdown_connections 2024-09-11T23:24:45.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:45.924+0000 7f7ac1b03740 1 -- 172.21.15.26:0/914459119 wait complete. 2024-09-11T23:24:45.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:45.925+0000 7f7ac1b03740 1 -- start start 2024-09-11T23:24:45.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:45.926+0000 7f7ac1b03740 1 Processor -- start 2024-09-11T23:24:45.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:45.929+0000 7f7ac1b03740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559f5ed82760 0x559f5ed8a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:45.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:45.929+0000 7f7ac1b03740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559f5ed83080 0x559f5ed8ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:45.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:45.930+0000 7f7abd4c6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559f5ed82760 0x559f5ed8a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:45.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:45.930+0000 7f7abd4c6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559f5ed82760 0x559f5ed8a7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59826/0 (socket says 172.21.15.26:59826) 2024-09-11T23:24:45.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:45.930+0000 7f7ac1b03740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559f5ed8d940 0x559f5ed8b260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:45.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:45.931+0000 7f7abccc5640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559f5ed83080 0x559f5ed8ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:45.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:45.931+0000 7f7abccc5640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559f5ed83080 0x559f5ed8ad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:38132/0 (socket says 172.21.15.26:38132) 2024-09-11T23:24:45.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:45.932+0000 7f7ac1b03740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x559f5ed90560 con 0x559f5ed82760 2024-09-11T23:24:45.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:45.932+0000 7f7ac1b03740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x559f5ec706e0 con 0x559f5ed8d940 2024-09-11T23:24:45.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:45.932+0000 7f7ac1b03740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x559f5ed906e0 con 0x559f5ed83080 2024-09-11T23:24:45.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:45.932+0000 7f7abd4c6640 1 -- 172.21.15.26:0/2356336707 learned_addr learned my addr 172.21.15.26:0/2356336707 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:24:45.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:45.932+0000 7f7ac1b03740 10 client.0 _ll_register_callbacks cb 0x559f5ec94a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:24:45.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:45.932+0000 7f7abdcc7640 1 --2- 172.21.15.26:0/2356336707 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559f5ed8d940 0x559f5ed8b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:45.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:45.933+0000 7f7abdcc7640 1 -- 172.21.15.26:0/2356336707 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559f5ed83080 msgr2=0x559f5ed8ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:24:45.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:45.933+0000 7f7abdcc7640 1 --2- 172.21.15.26:0/2356336707 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559f5ed83080 0x559f5ed8ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:45.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:45.933+0000 7f7abdcc7640 1 -- 172.21.15.26:0/2356336707 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559f5ed82760 msgr2=0x559f5ed8a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:24:45.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:45.933+0000 7f7abdcc7640 1 --2- 172.21.15.26:0/2356336707 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559f5ed82760 0x559f5ed8a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:45.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:45.933+0000 7f7abdcc7640 1 -- 172.21.15.26:0/2356336707 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f5ed99b00 con 0x559f5ed8d940 2024-09-11T23:24:45.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:45.933+0000 7f7abdcc7640 1 --2- 172.21.15.26:0/2356336707 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559f5ed8d940 0x559f5ed8b260 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f7ab4016000 tx=0x7f7ab4011a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c25a3bb8daa804ad server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:24:45.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:45.933+0000 7f7aa0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-11T23:24:45.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:45.934+0000 7f7aa0ff9640 1 -- 172.21.15.26:0/2356336707 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ab4027060 con 0x559f5ed8d940 2024-09-11T23:24:45.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:45.934+0000 7f7aa0ff9640 1 -- 172.21.15.26:0/2356336707 <== mon.2 v2:172.21.15.76:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f7ab4011630 con 0x559f5ed8d940 2024-09-11T23:24:45.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:45.935+0000 7f7ac1b03740 1 -- 172.21.15.26:0/2356336707 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ab400bda0 con 0x559f5ed8d940 2024-09-11T23:24:45.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:45.936+0000 7f7aa0ff9640 1 -- 172.21.15.26:0/2356336707 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f7ab400b660 con 0x559f5ed8d940 2024-09-11T23:24:45.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:45.937+0000 7f7ac1b03740 10 client.26090 fetch_fsmap learned FSMap version 41 2024-09-11T23:24:45.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:45.937+0000 7f7ac1b03740 1 -- 172.21.15.26:0/2356336707 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x559f5ed9c3e0 con 0x559f5ed8d940 2024-09-11T23:24:45.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:45.939+0000 7f7aa0ff9640 1 -- 172.21.15.26:0/2356336707 <== mon.2 v2:172.21.15.76:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ab4010840 con 0x559f5ed8d940 2024-09-11T23:24:45.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:45.940+0000 7f7aa0ff9640 1 -- 172.21.15.26:0/2356336707 <== mon.2 v2:172.21.15.76:3300/0 5 ==== fsmap(e 41) ==== 8426+0+0 (secure 0 0 0) 0x7f7ab4004200 con 0x559f5ed8d940 2024-09-11T23:24:45.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:45.940+0000 7f7ac1b03740 10 client.26090 fetch_fsmap finished waiting for FSMap version 41 2024-09-11T23:24:45.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:45.940+0000 7f7ac1b03740 10 client.26090 resolve_mds: validated gid 24485 aka daemon mds.b 2024-09-11T23:24:45.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:45.940+0000 7f7ac1b03740 20 client.26090 populate_metadata read hostname 'smithi026' 2024-09-11T23:24:45.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:45.940+0000 7f7ac1b03740 1 --2- 172.21.15.26:0/2356336707 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x559f5ed9d730 0x559f5ed9fb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:45.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:45.940+0000 7f7ac1b03740 4 client.26090 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:24:45.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:45.940+0000 7f7ac1b03740 1 -- 172.21.15.26:0/2356336707 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f7ab400bda0 con 0x559f5ed9d730 2024-09-11T23:24:45.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:45.940+0000 7f7abd4c6640 1 --2- 172.21.15.26:0/2356336707 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x559f5ed9d730 0x559f5ed9fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:45.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:45.942+0000 7f7abd4c6640 1 --2- 172.21.15.26:0/2356336707 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x559f5ed9d730 0x559f5ed9fb20 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e7e629784fc424b1 server_cookie=3599765485ea2c34 in_seq=0 out_seq=0 2024-09-11T23:24:45.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:45.942+0000 7f7aa0ff9640 10 client.26090 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:24:45.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:45.944+0000 7f7aa0ff9640 1 -- 172.21.15.26:0/2356336707 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f7ab400bda0 con 0x559f5ed9d730 2024-09-11T23:24:45.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:45.944+0000 7f7aa0ff9640 10 client.26090 handle_command_reply: tid=0 2024-09-11T23:24:45.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:45.944+0000 7f7ac1b03740 1 client.26090 shutdown 2024-09-11T23:24:45.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:45.945+0000 7f7ac1b03740 1 -- 172.21.15.26:0/2356336707 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559f5ed8d940 msgr2=0x559f5ed8b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:24:45.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:45.945+0000 7f7ac1b03740 1 --2- 172.21.15.26:0/2356336707 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559f5ed8d940 0x559f5ed8b260 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f7ab4016000 tx=0x7f7ab4011a20 comp rx=0 tx=0).stop 2024-09-11T23:24:45.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:45.945+0000 7f7ac1b03740 1 -- 172.21.15.26:0/2356336707 shutdown_connections 2024-09-11T23:24:45.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:45.945+0000 7f7ac1b03740 1 --2- 172.21.15.26:0/2356336707 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559f5ed8d940 0x559f5ed8b260 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:45.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:45.945+0000 7f7ac1b03740 1 --2- 172.21.15.26:0/2356336707 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x559f5ed9d730 0x559f5ed9fb20 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:45.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:45.945+0000 7f7ac1b03740 1 --2- 172.21.15.26:0/2356336707 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559f5ed83080 0x559f5ed8ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:45.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:45.945+0000 7f7aa0ff9640 0 client.26090 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:24:45.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:45.945+0000 7f7ac1b03740 1 --2- 172.21.15.26:0/2356336707 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559f5ed82760 0x559f5ed8a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:45.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:45.945+0000 7f7ac1b03740 1 -- 172.21.15.26:0/2356336707 >> 172.21.15.26:0/2356336707 conn(0x559f5ed7ecf0 msgr2=0x559f5ed84a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:24:45.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:45.947+0000 7f7ac1b03740 1 -- 172.21.15.26:0/2356336707 shutdown_connections 2024-09-11T23:24:45.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:45.947+0000 7f7ac1b03740 1 -- 172.21.15.26:0/2356336707 wait complete. 2024-09-11T23:24:45.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:45.948+0000 7f7ac1b03740 20 client.26090 trim_cache size 0 max 16384 2024-09-11T23:24:45.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:24:45 smithi136 ceph-mon[35121]: pgmap v194: 97 pgs: 97 active+clean; 2.3 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 20 MiB/s wr, 1.41k op/s 2024-09-11T23:24:45.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:45.998+0000 7f7ac1b03740 1 Processor -- start 2024-09-11T23:24:46.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:45.998+0000 7f7ac1b03740 1 -- start start 2024-09-11T23:24:46.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:45.999+0000 7f7ac1b03740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559f5ed82760 0x559f5ed8c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:46.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:45.999+0000 7f7ac1b03740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559f5ed83080 0x559f5ed8b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:46.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:45.999+0000 7f7ac1b03740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559f5ed8d940 0x559f5eda5350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:46.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:45.999+0000 7f7ac1b03740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f7a9c00e870 con 0x559f5ed83080 2024-09-11T23:24:46.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:45.999+0000 7f7ac1b03740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f7a9c0106b0 con 0x559f5ed8d940 2024-09-11T23:24:46.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:45.999+0000 7f7ac1b03740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f7a9c00f000 con 0x559f5ed82760 2024-09-11T23:24:46.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:45.999+0000 7f7abdcc7640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559f5ed8d940 0x559f5eda5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:46.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:45.999+0000 7f7abdcc7640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559f5ed8d940 0x559f5eda5350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:40768/0 (socket says 172.21.15.26:40768) 2024-09-11T23:24:46.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:45.999+0000 7f7abdcc7640 1 -- 172.21.15.26:0/154419884 learned_addr learned my addr 172.21.15.26:0/154419884 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:24:46.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:45.999+0000 7f7abd4c6640 1 --2- 172.21.15.26:0/154419884 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559f5ed82760 0x559f5ed8c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:46.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:45.999+0000 7f7abdcc7640 1 -- 172.21.15.26:0/154419884 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559f5ed82760 msgr2=0x559f5ed8c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:24:46.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:45.999+0000 7f7abdcc7640 1 --2- 172.21.15.26:0/154419884 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559f5ed82760 0x559f5ed8c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:46.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:45.999+0000 7f7abdcc7640 1 -- 172.21.15.26:0/154419884 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559f5ed83080 msgr2=0x559f5ed8b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:24:46.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:45.999+0000 7f7abdcc7640 1 --2- 172.21.15.26:0/154419884 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559f5ed83080 0x559f5ed8b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:46.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:45.999+0000 7f7abdcc7640 1 -- 172.21.15.26:0/154419884 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f5ed7eb20 con 0x559f5ed8d940 2024-09-11T23:24:46.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:46.000+0000 7f7abdcc7640 1 --2- 172.21.15.26:0/154419884 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559f5ed8d940 0x559f5eda5350 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f7ab4006de0 tx=0x7f7ab4011b70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b85d30847ac69e1a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:24:46.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:46.001+0000 7f7aa2ffd640 1 -- 172.21.15.26:0/154419884 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ab4027040 con 0x559f5ed8d940 2024-09-11T23:24:46.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:46.001+0000 7f7aa2ffd640 1 -- 172.21.15.26:0/154419884 <== mon.2 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ab400b4b0 con 0x559f5ed8d940 2024-09-11T23:24:46.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:46.001+0000 7f7aa2ffd640 1 -- 172.21.15.26:0/154419884 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ab4015040 con 0x559f5ed8d940 2024-09-11T23:24:46.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:46.002+0000 7f7ac1b03740 1 -- 172.21.15.26:0/154419884 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559f5ed8d940 msgr2=0x559f5eda5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:24:46.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:46.002+0000 7f7ac1b03740 1 --2- 172.21.15.26:0/154419884 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559f5ed8d940 0x559f5eda5350 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f7ab4006de0 tx=0x7f7ab4011b70 comp rx=0 tx=0).stop 2024-09-11T23:24:46.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:46.002+0000 7f7ac1b03740 1 -- 172.21.15.26:0/154419884 shutdown_connections 2024-09-11T23:24:46.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:46.003+0000 7f7ac1b03740 1 --2- 172.21.15.26:0/154419884 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559f5ed8d940 0x559f5eda5350 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:46.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:46.003+0000 7f7ac1b03740 1 --2- 172.21.15.26:0/154419884 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559f5ed83080 0x559f5ed8b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:46.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:46.003+0000 7f7ac1b03740 1 --2- 172.21.15.26:0/154419884 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559f5ed82760 0x559f5ed8c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:46.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:46.003+0000 7f7ac1b03740 1 -- 172.21.15.26:0/154419884 >> 172.21.15.26:0/154419884 conn(0x559f5ed7ecf0 msgr2=0x559f5ed849d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:24:46.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:46.006+0000 7f7ac1b03740 1 -- 172.21.15.26:0/154419884 shutdown_connections 2024-09-11T23:24:46.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:46.006+0000 7f7ac1b03740 1 -- 172.21.15.26:0/154419884 wait complete. 2024-09-11T23:24:46.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:46.007+0000 7f7ac1b03740 1 -- start start 2024-09-11T23:24:46.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:46.007+0000 7f7ac1b03740 1 Processor -- start 2024-09-11T23:24:46.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:46.008+0000 7f7ac1b03740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559f5ed82760 0x559f5ed96040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:46.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:46.008+0000 7f7ac1b03740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559f5ed83080 0x559f5ed929e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:46.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:46.008+0000 7f7ac1b03740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559f5ed8d940 0x559f5ed92f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:46.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:46.008+0000 7f7ac1b03740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f7a9c00ff20 con 0x559f5ed8d940 2024-09-11T23:24:46.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:46.008+0000 7f7ac1b03740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f7a9c00f790 con 0x559f5ed82760 2024-09-11T23:24:46.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:46.008+0000 7f7ac1b03740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f7a9c00c850 con 0x559f5ed83080 2024-09-11T23:24:46.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:46.008+0000 7f7ac1b03740 10 client.0 _ll_register_callbacks cb 0x7f7ab4006bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:24:46.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:46.008+0000 7f7abccc5640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559f5ed83080 0x559f5ed929e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:46.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:46.009+0000 7f7abccc5640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559f5ed83080 0x559f5ed929e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:38150/0 (socket says 172.21.15.26:38150) 2024-09-11T23:24:46.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:46.009+0000 7f7abccc5640 1 -- 172.21.15.26:0/1485906251 learned_addr learned my addr 172.21.15.26:0/1485906251 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:24:46.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:46.009+0000 7f7abd4c6640 1 --2- 172.21.15.26:0/1485906251 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559f5ed82760 0x559f5ed96040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:46.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:46.009+0000 7f7abccc5640 1 -- 172.21.15.26:0/1485906251 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559f5ed82760 msgr2=0x559f5ed96040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:24:46.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:46.009+0000 7f7abccc5640 1 --2- 172.21.15.26:0/1485906251 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559f5ed82760 0x559f5ed96040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:46.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:46.009+0000 7f7abccc5640 1 -- 172.21.15.26:0/1485906251 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559f5ed8d940 msgr2=0x559f5ed92f20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T23:24:46.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:46.009+0000 7f7abccc5640 1 --2- 172.21.15.26:0/1485906251 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559f5ed8d940 0x559f5ed92f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:46.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:46.009+0000 7f7abccc5640 1 -- 172.21.15.26:0/1485906251 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f5ed93610 con 0x559f5ed83080 2024-09-11T23:24:46.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:46.010+0000 7f7abccc5640 1 --2- 172.21.15.26:0/1485906251 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559f5ed83080 0x559f5ed929e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x559f5ed7fb50 tx=0x7f7aa80046e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5f0d61b9d2e1d5dd server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:24:46.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:46.010+0000 7f7aa0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-09-11T23:24:46.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:46.010+0000 7f7aa0ff9640 1 -- 172.21.15.26:0/1485906251 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aa801f670 con 0x559f5ed83080 2024-09-11T23:24:46.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:46.011+0000 7f7ac1b03740 1 -- 172.21.15.26:0/1485906251 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ab400b4b0 con 0x559f5ed83080 2024-09-11T23:24:46.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:46.011+0000 7f7aa0ff9640 1 -- 172.21.15.26:0/1485906251 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f7aa801f810 con 0x559f5ed83080 2024-09-11T23:24:46.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:46.011+0000 7f7aa0ff9640 1 -- 172.21.15.26:0/1485906251 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f7aa802d050 con 0x559f5ed83080 2024-09-11T23:24:46.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:46.011+0000 7f7ac1b03740 10 client.26113 fetch_fsmap learned FSMap version 41 2024-09-11T23:24:46.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:46.011+0000 7f7ac1b03740 1 -- 172.21.15.26:0/1485906251 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x559f5ee1a2e0 con 0x559f5ed83080 2024-09-11T23:24:46.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:46.012+0000 7f7aa0ff9640 1 -- 172.21.15.26:0/1485906251 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ab400b4b0 con 0x559f5ed83080 2024-09-11T23:24:46.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:46.013+0000 7f7aa0ff9640 1 -- 172.21.15.26:0/1485906251 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 41) ==== 8426+0+0 (secure 0 0 0) 0x7f7aa8012070 con 0x559f5ed83080 2024-09-11T23:24:46.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:46.013+0000 7f7ac1b03740 10 client.26113 fetch_fsmap finished waiting for FSMap version 41 2024-09-11T23:24:46.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:46.013+0000 7f7ac1b03740 10 client.26113 resolve_mds: validated gid 24485 aka daemon mds.b 2024-09-11T23:24:46.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:46.013+0000 7f7ac1b03740 20 client.26113 populate_metadata read hostname 'smithi026' 2024-09-11T23:24:46.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:46.013+0000 7f7ac1b03740 1 --2- 172.21.15.26:0/1485906251 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x559f5ee1abf0 0x559f5ee1d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:46.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:46.013+0000 7f7ac1b03740 4 client.26113 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.1, "await_for": 45.0}] 2024-09-11T23:24:46.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:46.013+0000 7f7ac1b03740 1 -- 172.21.15.26:0/1485906251 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.1, "await_for": 45.0}) -- 0x559f5ecf59d0 con 0x559f5ee1abf0 2024-09-11T23:24:46.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:46.014+0000 7f7abd4c6640 1 --2- 172.21.15.26:0/1485906251 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x559f5ee1abf0 0x559f5ee1d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:46.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:46.017+0000 7f7aa0ff9640 10 client.26113 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:24:46.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:24:46.030+0000 7f7abd4c6640 1 --2- 172.21.15.26:0/1485906251 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x559f5ee1abf0 0x559f5ee1d010 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b829ec9cb04a8db server_cookie=1a35430154af7f78 in_seq=0 out_seq=0 2024-09-11T23:24:46.052 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:24:45 smithi076 ceph-mon[35296]: pgmap v194: 97 pgs: 97 active+clean; 2.3 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 20 MiB/s wr, 1.41k op/s 2024-09-11T23:24:46.183 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.181+0000 7fe05f6ba640 1 -- 172.21.15.26:0/2688559484 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe058102480 msgr2=0x7fe05810e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:24:46.183 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.181+0000 7fe05f6ba640 1 --2- 172.21.15.26:0/2688559484 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe058102480 0x7fe05810e9b0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fe050009cd0 tx=0x7fe05002a8e0 comp rx=0 tx=0).stop 2024-09-11T23:24:46.183 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.181+0000 7fe05f6ba640 1 -- 172.21.15.26:0/2688559484 shutdown_connections 2024-09-11T23:24:46.183 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.181+0000 7fe05f6ba640 1 --2- 172.21.15.26:0/2688559484 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe058101ae0 0x7fe058101f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:46.183 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.181+0000 7fe05f6ba640 1 --2- 172.21.15.26:0/2688559484 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe058102480 0x7fe05810e9b0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:46.183 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.181+0000 7fe05f6ba640 1 --2- 172.21.15.26:0/2688559484 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe058107b80 0x7fe058107f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:46.183 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.181+0000 7fe05f6ba640 1 -- 172.21.15.26:0/2688559484 >> 172.21.15.26:0/2688559484 conn(0x7fe0580fd7e0 msgr2=0x7fe0580ffc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:24:46.183 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.181+0000 7fe05f6ba640 1 -- 172.21.15.26:0/2688559484 shutdown_connections 2024-09-11T23:24:46.183 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.182+0000 7fe05f6ba640 1 -- 172.21.15.26:0/2688559484 wait complete. 2024-09-11T23:24:46.184 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.182+0000 7fe05f6ba640 1 Processor -- start 2024-09-11T23:24:46.185 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.182+0000 7fe05f6ba640 1 -- start start 2024-09-11T23:24:46.185 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.182+0000 7fe05f6ba640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe058101ae0 0x7fe058198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:46.185 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.182+0000 7fe05f6ba640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe058102480 0x7fe0581988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:46.186 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.183+0000 7fe05f6ba640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe058107b80 0x7fe05819cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:46.186 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.183+0000 7fe05f6ba640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fe058111f30 con 0x7fe058107b80 2024-09-11T23:24:46.186 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.183+0000 7fe05f6ba640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fe058111db0 con 0x7fe058101ae0 2024-09-11T23:24:46.186 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.183+0000 7fe05f6ba640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe0581120b0 con 0x7fe058102480 2024-09-11T23:24:46.186 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.183+0000 7fe05d456640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe058101ae0 0x7fe058198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:46.186 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.183+0000 7fe05d456640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe058101ae0 0x7fe058198370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:40786/0 (socket says 172.21.15.26:40786) 2024-09-11T23:24:46.186 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.183+0000 7fe05d456640 1 -- 172.21.15.26:0/3818007373 learned_addr learned my addr 172.21.15.26:0/3818007373 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:24:46.186 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.183+0000 7fe05dc57640 1 --2- 172.21.15.26:0/3818007373 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe058107b80 0x7fe05819cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:46.186 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.183+0000 7fe05cc55640 1 --2- 172.21.15.26:0/3818007373 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe058102480 0x7fe0581988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:46.186 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.183+0000 7fe05d456640 1 -- 172.21.15.26:0/3818007373 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe058102480 msgr2=0x7fe0581988b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:24:46.186 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.183+0000 7fe05d456640 1 --2- 172.21.15.26:0/3818007373 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe058102480 0x7fe0581988b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:46.186 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.183+0000 7fe05d456640 1 -- 172.21.15.26:0/3818007373 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe058107b80 msgr2=0x7fe05819cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:24:46.186 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.183+0000 7fe05d456640 1 --2- 172.21.15.26:0/3818007373 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe058107b80 0x7fe05819cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:46.187 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.183+0000 7fe05d456640 1 -- 172.21.15.26:0/3818007373 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe05819d360 con 0x7fe058101ae0 2024-09-11T23:24:46.187 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.183+0000 7fe05d456640 1 --2- 172.21.15.26:0/3818007373 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe058101ae0 0x7fe058198370 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fe05400e9f0 tx=0x7fe05400edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:24:46.187 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.183+0000 7fe04e7fc640 1 -- 172.21.15.26:0/3818007373 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe05400cd80 con 0x7fe058101ae0 2024-09-11T23:24:46.187 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.183+0000 7fe05f6ba640 1 -- 172.21.15.26:0/3818007373 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe05819d650 con 0x7fe058101ae0 2024-09-11T23:24:46.187 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.183+0000 7fe04e7fc640 1 -- 172.21.15.26:0/3818007373 <== mon.2 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe054005500 con 0x7fe058101ae0 2024-09-11T23:24:46.187 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.183+0000 7fe04e7fc640 1 -- 172.21.15.26:0/3818007373 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe054010690 con 0x7fe058101ae0 2024-09-11T23:24:46.187 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.184+0000 7fe05f6ba640 1 -- 172.21.15.26:0/3818007373 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0581a4ee0 con 0x7fe058101ae0 2024-09-11T23:24:46.192 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.186+0000 7fe04e7fc640 1 -- 172.21.15.26:0/3818007373 <== mon.2 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fe0540046c0 con 0x7fe058101ae0 2024-09-11T23:24:46.192 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.186+0000 7fe062458740 1 Processor -- start 2024-09-11T23:24:46.192 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.186+0000 7fe062458740 1 -- start start 2024-09-11T23:24:46.192 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.186+0000 7fe062458740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5649a63009b0 0x5649a6300d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:46.192 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.186+0000 7fe062458740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5649a63012d0 0x5649a630fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:46.192 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.186+0000 7fe062458740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5649a63100c0 0x5649a63124b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:46.192 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.186+0000 7fe062458740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5649a62652f0 con 0x5649a63100c0 2024-09-11T23:24:46.192 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.186+0000 7fe062458740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5649a620d270 con 0x5649a63012d0 2024-09-11T23:24:46.192 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.186+0000 7fe062458740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5649a62612b0 con 0x5649a63009b0 2024-09-11T23:24:46.193 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.187+0000 7fe05dc57640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5649a63009b0 0x5649a6300d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:46.193 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.187+0000 7fe05cc55640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5649a63100c0 0x5649a63124b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:46.193 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.187+0000 7fe05dc57640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5649a63009b0 0x5649a6300d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:38190/0 (socket says 172.21.15.26:38190) 2024-09-11T23:24:46.193 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.187+0000 7fe05dc57640 1 -- 172.21.15.26:0/3877083595 learned_addr learned my addr 172.21.15.26:0/3877083595 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:24:46.193 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.187+0000 7fe05d456640 1 --2- 172.21.15.26:0/3877083595 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5649a63012d0 0x5649a630fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:46.193 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.187+0000 7fe05dc57640 1 -- 172.21.15.26:0/3877083595 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5649a63012d0 msgr2=0x5649a630fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:24:46.193 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.187+0000 7fe05dc57640 1 --2- 172.21.15.26:0/3877083595 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5649a63012d0 0x5649a630fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:46.193 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.187+0000 7fe05dc57640 1 -- 172.21.15.26:0/3877083595 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5649a63100c0 msgr2=0x5649a63124b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:24:46.193 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.187+0000 7fe05dc57640 1 --2- 172.21.15.26:0/3877083595 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5649a63100c0 0x5649a63124b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:46.193 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.187+0000 7fe05dc57640 1 -- 172.21.15.26:0/3877083595 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5649a620c340 con 0x5649a63009b0 2024-09-11T23:24:46.193 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.187+0000 7fe04e7fc640 1 --2- 172.21.15.26:0/3818007373 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fe0440a2da0 0x7fe0440a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:46.193 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.187+0000 7fe04e7fc640 1 -- 172.21.15.26:0/3818007373 <== mon.2 v2:172.21.15.76:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fe0540cf890 con 0x7fe058101ae0 2024-09-11T23:24:46.193 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.187+0000 7fe05d456640 1 --2- 172.21.15.26:0/3818007373 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fe0440a2da0 0x7fe0440a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:46.194 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.187+0000 7fe05dc57640 1 --2- 172.21.15.26:0/3877083595 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5649a63009b0 0x5649a6300d90 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fe050009cd0 tx=0x7fe050009a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3b9f367bb4e7f44d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:24:46.194 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.188+0000 7fe0437fe640 1 -- 172.21.15.26:0/3877083595 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe050048070 con 0x5649a63009b0 2024-09-11T23:24:46.194 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.188+0000 7fe0437fe640 1 -- 172.21.15.26:0/3877083595 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe050036070 con 0x5649a63009b0 2024-09-11T23:24:46.194 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.188+0000 7fe0437fe640 1 -- 172.21.15.26:0/3877083595 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe050004440 con 0x5649a63009b0 2024-09-11T23:24:46.194 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.188+0000 7fe062458740 1 -- 172.21.15.26:0/3877083595 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5649a63009b0 msgr2=0x5649a6300d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:24:46.194 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.188+0000 7fe062458740 1 --2- 172.21.15.26:0/3877083595 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5649a63009b0 0x5649a6300d90 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fe050009cd0 tx=0x7fe050009a20 comp rx=0 tx=0).stop 2024-09-11T23:24:46.194 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.189+0000 7fe05d456640 1 --2- 172.21.15.26:0/3818007373 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fe0440a2da0 0x7fe0440a5260 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fe0540cd0a0 tx=0x7fe054004a40 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:24:46.194 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.189+0000 7fe062458740 1 -- 172.21.15.26:0/3877083595 shutdown_connections 2024-09-11T23:24:46.194 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.189+0000 7fe062458740 1 --2- 172.21.15.26:0/3877083595 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5649a63012d0 0x5649a630fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:46.194 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.189+0000 7fe062458740 1 --2- 172.21.15.26:0/3877083595 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5649a63100c0 0x5649a63124b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:46.194 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.189+0000 7fe062458740 1 --2- 172.21.15.26:0/3877083595 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5649a63009b0 0x5649a6300d90 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:46.194 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.189+0000 7fe062458740 1 -- 172.21.15.26:0/3877083595 >> 172.21.15.26:0/3877083595 conn(0x5649a62fd060 msgr2=0x5649a62fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:24:46.194 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.189+0000 7fe062458740 1 -- 172.21.15.26:0/3877083595 shutdown_connections 2024-09-11T23:24:46.194 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.189+0000 7fe062458740 1 -- 172.21.15.26:0/3877083595 wait complete. 2024-09-11T23:24:46.195 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.189+0000 7fe062458740 1 -- start start 2024-09-11T23:24:46.195 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.189+0000 7fe062458740 1 Processor -- start 2024-09-11T23:24:46.195 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.190+0000 7fe062458740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5649a63009b0 0x5649a630d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:46.195 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.190+0000 7fe062458740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5649a63012d0 0x5649a630d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:46.195 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.190+0000 7fe062458740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5649a63100c0 0x5649a630db00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:46.195 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.190+0000 7fe062458740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5649a6312c30 con 0x5649a63009b0 2024-09-11T23:24:46.195 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.190+0000 7fe062458740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5649a61ee290 con 0x5649a63100c0 2024-09-11T23:24:46.195 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.190+0000 7fe062458740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5649a6312db0 con 0x5649a63012d0 2024-09-11T23:24:46.195 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.190+0000 7fe062458740 10 client.0 _ll_register_callbacks cb 0x5649a621f140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:24:46.195 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.190+0000 7fe05dc57640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5649a63009b0 0x5649a630d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:46.195 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.190+0000 7fe05cc55640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5649a63012d0 0x5649a630d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:46.195 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.190+0000 7fe05cc55640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5649a63012d0 0x5649a630d5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:38192/0 (socket says 172.21.15.26:38192) 2024-09-11T23:24:46.196 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.190+0000 7fe05dc57640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5649a63009b0 0x5649a630d080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59880/0 (socket says 172.21.15.26:59880) 2024-09-11T23:24:46.196 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.190+0000 7fe05cc55640 1 -- 172.21.15.26:0/1477072456 learned_addr learned my addr 172.21.15.26:0/1477072456 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:24:46.196 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.190+0000 7fe05dc57640 1 --2- 172.21.15.26:0/1477072456 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5649a63100c0 0x5649a630db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:46.196 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.190+0000 7fe05cc55640 1 -- 172.21.15.26:0/1477072456 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5649a63100c0 msgr2=0x5649a630db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:24:46.196 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.190+0000 7fe05cc55640 1 --2- 172.21.15.26:0/1477072456 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5649a63100c0 0x5649a630db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:46.196 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.190+0000 7fe05cc55640 1 -- 172.21.15.26:0/1477072456 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5649a63009b0 msgr2=0x5649a630d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:24:46.196 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.190+0000 7fe05cc55640 1 --2- 172.21.15.26:0/1477072456 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5649a63009b0 0x5649a630d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:46.196 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.190+0000 7fe05cc55640 1 -- 172.21.15.26:0/1477072456 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5649a5f78210 con 0x5649a63012d0 2024-09-11T23:24:46.196 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.190+0000 7fe05dc57640 1 --2- 172.21.15.26:0/1477072456 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5649a63009b0 0x5649a630d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:24:46.196 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.190+0000 7fe05cc55640 1 --2- 172.21.15.26:0/1477072456 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5649a63012d0 0x5649a630d5c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fe048012700 tx=0x7fe048012bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4d244ff0616882bf server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:24:46.196 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.190+0000 7fe040ff9640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-09-11T23:24:46.196 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.190+0000 7fe040ff9640 1 -- 172.21.15.26:0/1477072456 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe048011bb0 con 0x5649a63012d0 2024-09-11T23:24:46.196 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.191+0000 7fe062458740 1 -- 172.21.15.26:0/1477072456 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe050036070 con 0x5649a63012d0 2024-09-11T23:24:46.197 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.191+0000 7fe040ff9640 1 -- 172.21.15.26:0/1477072456 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fe048011d50 con 0x5649a63012d0 2024-09-11T23:24:46.197 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.191+0000 7fe040ff9640 1 -- 172.21.15.26:0/1477072456 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fe04800f460 con 0x5649a63012d0 2024-09-11T23:24:46.197 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.191+0000 7fe062458740 10 client.26125 fetch_fsmap learned FSMap version 41 2024-09-11T23:24:46.197 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.191+0000 7fe062458740 1 -- 172.21.15.26:0/1477072456 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5649a6044620 con 0x5649a63012d0 2024-09-11T23:24:46.197 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.191+0000 7fe040ff9640 1 -- 172.21.15.26:0/1477072456 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe050036070 con 0x5649a63012d0 2024-09-11T23:24:46.198 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.191+0000 7fe040ff9640 1 -- 172.21.15.26:0/1477072456 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 41) ==== 8426+0+0 (secure 0 0 0) 0x7fe04801c6f0 con 0x5649a63012d0 2024-09-11T23:24:46.199 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.191+0000 7fe062458740 10 client.26125 fetch_fsmap finished waiting for FSMap version 41 2024-09-11T23:24:46.199 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.191+0000 7fe062458740 10 client.26125 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:24:46.199 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.191+0000 7fe062458740 20 client.26125 populate_metadata read hostname 'smithi026' 2024-09-11T23:24:46.199 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.191+0000 7fe062458740 1 --2- 172.21.15.26:0/1477072456 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5649a6320170 0x5649a63225f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:46.199 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.191+0000 7fe062458740 4 client.26125 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:24:46.199 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.191+0000 7fe062458740 1 -- 172.21.15.26:0/1477072456 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5649a6272a20 con 0x5649a6320170 2024-09-11T23:24:46.199 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.192+0000 7fe05dc57640 1 --2- 172.21.15.26:0/1477072456 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5649a6320170 0x5649a63225f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:46.199 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.192+0000 7fe040ff9640 10 client.26125 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:24:46.199 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.192+0000 7fe05dc57640 1 --2- 172.21.15.26:0/1477072456 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5649a6320170 0x5649a63225f0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b4d37492f9c9b22a server_cookie=ba66b277ef05cb14 in_seq=0 out_seq=0 2024-09-11T23:24:46.199 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.193+0000 7fe040ff9640 1 -- 172.21.15.26:0/1477072456 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5649a6272a20 con 0x5649a6320170 2024-09-11T23:24:46.199 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.193+0000 7fe040ff9640 10 client.26125 handle_command_reply: tid=0 2024-09-11T23:24:46.199 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.193+0000 7fe062458740 1 client.26125 shutdown 2024-09-11T23:24:46.200 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.193+0000 7fe062458740 1 -- 172.21.15.26:0/1477072456 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5649a63012d0 msgr2=0x5649a630d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:24:46.200 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.193+0000 7fe062458740 1 --2- 172.21.15.26:0/1477072456 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5649a63012d0 0x5649a630d5c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fe048012700 tx=0x7fe048012bc0 comp rx=0 tx=0).stop 2024-09-11T23:24:46.200 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.193+0000 7fe062458740 1 -- 172.21.15.26:0/1477072456 shutdown_connections 2024-09-11T23:24:46.200 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.193+0000 7fe062458740 1 --2- 172.21.15.26:0/1477072456 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5649a63100c0 0x5649a630db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:46.200 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.193+0000 7fe062458740 1 --2- 172.21.15.26:0/1477072456 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5649a6320170 0x5649a63225f0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:46.200 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.193+0000 7fe062458740 1 --2- 172.21.15.26:0/1477072456 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5649a63012d0 0x5649a630d5c0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:46.200 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.193+0000 7fe062458740 1 --2- 172.21.15.26:0/1477072456 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5649a63009b0 0x5649a630d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:46.200 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.193+0000 7fe040ff9640 0 client.26125 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:24:46.200 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.193+0000 7fe062458740 1 -- 172.21.15.26:0/1477072456 >> 172.21.15.26:0/1477072456 conn(0x5649a62fd060 msgr2=0x5649a62fc1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:24:46.200 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.193+0000 7fe062458740 1 -- 172.21.15.26:0/1477072456 shutdown_connections 2024-09-11T23:24:46.200 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.193+0000 7fe062458740 1 -- 172.21.15.26:0/1477072456 wait complete. 2024-09-11T23:24:46.200 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.193+0000 7fe062458740 20 client.26125 trim_cache size 0 max 16384 2024-09-11T23:24:46.210 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.208+0000 7fe062458740 1 Processor -- start 2024-09-11T23:24:46.211 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.208+0000 7fe062458740 1 -- start start 2024-09-11T23:24:46.211 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.208+0000 7fe062458740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5649a63009b0 0x5649a6327ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:46.211 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.208+0000 7fe062458740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5649a63012d0 0x5649a6328400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:46.211 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.208+0000 7fe062458740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5649a63100c0 0x5649a62fb9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:46.211 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.208+0000 7fe062458740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fe0581ab5c0 con 0x5649a63009b0 2024-09-11T23:24:46.211 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.208+0000 7fe062458740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fe0581ad400 con 0x5649a63012d0 2024-09-11T23:24:46.211 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.208+0000 7fe062458740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe0581abd50 con 0x5649a63100c0 2024-09-11T23:24:46.211 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.209+0000 7fe05dc57640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5649a63009b0 0x5649a6327ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:46.211 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.209+0000 7fe05cc55640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5649a63012d0 0x5649a6328400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:46.211 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.209+0000 7fe05dc57640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5649a63009b0 0x5649a6327ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59886/0 (socket says 172.21.15.26:59886) 2024-09-11T23:24:46.211 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.209+0000 7fe05dc57640 1 -- 172.21.15.26:0/3767702245 learned_addr learned my addr 172.21.15.26:0/3767702245 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:24:46.211 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.209+0000 7fe05cc55640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5649a63012d0 0x5649a6328400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:40808/0 (socket says 172.21.15.26:40808) 2024-09-11T23:24:46.212 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.209+0000 7fe05dc57640 1 --2- 172.21.15.26:0/3767702245 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5649a63100c0 0x5649a62fb9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:46.212 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.209+0000 7fe05dc57640 1 -- 172.21.15.26:0/3767702245 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5649a63100c0 msgr2=0x5649a62fb9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:24:46.212 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.209+0000 7fe05dc57640 1 --2- 172.21.15.26:0/3767702245 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5649a63100c0 0x5649a62fb9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:46.212 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.209+0000 7fe05dc57640 1 -- 172.21.15.26:0/3767702245 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5649a63012d0 msgr2=0x5649a6328400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:24:46.212 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.209+0000 7fe05dc57640 1 --2- 172.21.15.26:0/3767702245 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5649a63012d0 0x5649a6328400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:46.212 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.209+0000 7fe05dc57640 1 -- 172.21.15.26:0/3767702245 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5649a62fcd30 con 0x5649a63009b0 2024-09-11T23:24:46.212 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.209+0000 7fe05cc55640 1 --2- 172.21.15.26:0/3767702245 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5649a63012d0 0x5649a6328400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:24:46.212 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.209+0000 7fe05dc57640 1 --2- 172.21.15.26:0/3767702245 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5649a63009b0 0x5649a6327ec0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fe050003f30 tx=0x7fe05002a8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45e9db0ce0b2056c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:24:46.212 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.209+0000 7fe042ffd640 1 -- 172.21.15.26:0/3767702245 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe050048040 con 0x5649a63009b0 2024-09-11T23:24:46.212 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.209+0000 7fe042ffd640 1 -- 172.21.15.26:0/3767702245 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe050036070 con 0x5649a63009b0 2024-09-11T23:24:46.212 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.209+0000 7fe042ffd640 1 -- 172.21.15.26:0/3767702245 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe050030b50 con 0x5649a63009b0 2024-09-11T23:24:46.215 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.210+0000 7fe062458740 1 -- 172.21.15.26:0/3767702245 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5649a63009b0 msgr2=0x5649a6327ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:24:46.215 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.210+0000 7fe062458740 1 --2- 172.21.15.26:0/3767702245 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5649a63009b0 0x5649a6327ec0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fe050003f30 tx=0x7fe05002a8e0 comp rx=0 tx=0).stop 2024-09-11T23:24:46.215 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.210+0000 7fe062458740 1 -- 172.21.15.26:0/3767702245 shutdown_connections 2024-09-11T23:24:46.216 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.210+0000 7fe062458740 1 --2- 172.21.15.26:0/3767702245 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5649a63012d0 0x5649a6328400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:46.216 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.210+0000 7fe062458740 1 --2- 172.21.15.26:0/3767702245 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5649a63100c0 0x5649a62fb9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:46.216 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.210+0000 7fe062458740 1 --2- 172.21.15.26:0/3767702245 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5649a63009b0 0x5649a6327ec0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:46.216 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.210+0000 7fe062458740 1 -- 172.21.15.26:0/3767702245 >> 172.21.15.26:0/3767702245 conn(0x5649a62fd060 msgr2=0x5649a630c150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:24:46.216 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.210+0000 7fe062458740 1 -- 172.21.15.26:0/3767702245 shutdown_connections 2024-09-11T23:24:46.216 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.210+0000 7fe062458740 1 -- 172.21.15.26:0/3767702245 wait complete. 2024-09-11T23:24:46.216 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.211+0000 7fe062458740 1 -- start start 2024-09-11T23:24:46.216 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.211+0000 7fe062458740 1 Processor -- start 2024-09-11T23:24:46.216 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.211+0000 7fe062458740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5649a63009b0 0x5649a63185c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:46.216 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.211+0000 7fe062458740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5649a63012d0 0x5649a6313930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:46.216 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.211+0000 7fe062458740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5649a63100c0 0x5649a6313e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:46.216 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.211+0000 7fe062458740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fe0581acc70 con 0x5649a63012d0 2024-09-11T23:24:46.216 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.211+0000 7fe062458740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fe0581ac4e0 con 0x5649a63100c0 2024-09-11T23:24:46.217 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.211+0000 7fe062458740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe0581a95c0 con 0x5649a63009b0 2024-09-11T23:24:46.217 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.211+0000 7fe062458740 10 client.0 _ll_register_callbacks cb 0x7fe0480096c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:24:46.217 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.211+0000 7fe05cc55640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5649a63012d0 0x5649a6313930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:46.217 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.211+0000 7fe05cc55640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5649a63012d0 0x5649a6313930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59898/0 (socket says 172.21.15.26:59898) 2024-09-11T23:24:46.217 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.211+0000 7fe05cc55640 1 -- 172.21.15.26:0/3455284014 learned_addr learned my addr 172.21.15.26:0/3455284014 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:24:46.217 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.211+0000 7fe05dc57640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5649a63009b0 0x5649a63185c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:46.217 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.211+0000 7fe05dc57640 1 --2- 172.21.15.26:0/3455284014 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5649a63100c0 0x5649a6313e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:46.217 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.212+0000 7fe05cc55640 1 -- 172.21.15.26:0/3455284014 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5649a63009b0 msgr2=0x5649a63185c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:24:46.217 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.212+0000 7fe05cc55640 1 --2- 172.21.15.26:0/3455284014 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5649a63009b0 0x5649a63185c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:46.217 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.212+0000 7fe05cc55640 1 -- 172.21.15.26:0/3455284014 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5649a63100c0 msgr2=0x5649a6313e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:24:46.217 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.212+0000 7fe05cc55640 1 --2- 172.21.15.26:0/3455284014 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5649a63100c0 0x5649a6313e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:46.217 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.212+0000 7fe05cc55640 1 -- 172.21.15.26:0/3455284014 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5649a6314460 con 0x5649a63012d0 2024-09-11T23:24:46.217 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.212+0000 7fe05cc55640 1 --2- 172.21.15.26:0/3455284014 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5649a63012d0 0x5649a6313930 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x5649a630e6c0 tx=0x7fe048011de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29c4e719552d465e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:24:46.218 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.212+0000 7fe040ff9640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:24:46.218 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.212+0000 7fe040ff9640 1 -- 172.21.15.26:0/3455284014 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe04801cdd0 con 0x5649a63012d0 2024-09-11T23:24:46.218 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.212+0000 7fe040ff9640 1 -- 172.21.15.26:0/3455284014 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fe04801c630 con 0x5649a63012d0 2024-09-11T23:24:46.218 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.212+0000 7fe062458740 1 -- 172.21.15.26:0/3455284014 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe050036070 con 0x5649a63012d0 2024-09-11T23:24:46.218 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.212+0000 7fe040ff9640 1 -- 172.21.15.26:0/3455284014 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fe04801c830 con 0x5649a63012d0 2024-09-11T23:24:46.218 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.212+0000 7fe062458740 10 client.16512 fetch_fsmap learned FSMap version 41 2024-09-11T23:24:46.218 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.212+0000 7fe062458740 1 -- 172.21.15.26:0/3455284014 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5649a638acd0 con 0x5649a63012d0 2024-09-11T23:24:46.218 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.212+0000 7fe040ff9640 1 -- 172.21.15.26:0/3455284014 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe050036070 con 0x5649a63012d0 2024-09-11T23:24:46.218 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.213+0000 7fe040ff9640 1 -- 172.21.15.26:0/3455284014 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 41) ==== 8426+0+0 (secure 0 0 0) 0x7fe048004980 con 0x5649a63012d0 2024-09-11T23:24:46.218 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.213+0000 7fe062458740 10 client.16512 fetch_fsmap finished waiting for FSMap version 41 2024-09-11T23:24:46.218 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.213+0000 7fe062458740 10 client.16512 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:24:46.218 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.213+0000 7fe062458740 20 client.16512 populate_metadata read hostname 'smithi026' 2024-09-11T23:24:46.218 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.213+0000 7fe062458740 1 --2- 172.21.15.26:0/3455284014 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5649a638b590 0x5649a638d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:46.218 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.213+0000 7fe062458740 4 client.16512 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T23:24:46.219 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.213+0000 7fe062458740 1 -- 172.21.15.26:0/3455284014 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5649a626cd70 con 0x5649a638b590 2024-09-11T23:24:46.219 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.213+0000 7fe05dc57640 1 --2- 172.21.15.26:0/3455284014 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5649a638b590 0x5649a638d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:46.219 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.213+0000 7fe040ff9640 10 client.16512 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:24:46.219 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:46.213+0000 7fe05dc57640 1 --2- 172.21.15.26:0/3455284014 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5649a638b590 0x5649a638d9b0 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=78ad713c23167505 server_cookie=24751b81f6ee2246 in_seq=0 out_seq=0 2024-09-11T23:24:47.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:24:47 smithi136 ceph-mon[35121]: pgmap v195: 97 pgs: 97 active+clean; 2.3 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 13 MiB/s wr, 769 op/s 2024-09-11T23:24:48.000 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:24:47 smithi026 ceph-mon[31172]: pgmap v195: 97 pgs: 97 active+clean; 2.3 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 13 MiB/s wr, 769 op/s 2024-09-11T23:24:48.052 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:24:47 smithi076 ceph-mon[35296]: pgmap v195: 97 pgs: 97 active+clean; 2.3 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 13 MiB/s wr, 769 op/s 2024-09-11T23:24:49.494 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:49.492+0000 7fe040ff9640 1 -- 172.21.15.26:0/3455284014 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5649a626cd70 con 0x5649a638b590 2024-09-11T23:24:49.494 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:49.492+0000 7fe040ff9640 10 client.16512 handle_command_reply: tid=0 2024-09-11T23:24:49.494 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:49.492+0000 7fe062458740 1 client.16512 shutdown 2024-09-11T23:24:49.495 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:49.492+0000 7fe062458740 1 -- 172.21.15.26:0/3455284014 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5649a63012d0 msgr2=0x5649a6313930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:24:49.495 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:49.492+0000 7fe062458740 1 --2- 172.21.15.26:0/3455284014 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5649a63012d0 0x5649a6313930 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x5649a630e6c0 tx=0x7fe048011de0 comp rx=0 tx=0).stop 2024-09-11T23:24:49.495 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:49.492+0000 7fe062458740 1 -- 172.21.15.26:0/3455284014 shutdown_connections 2024-09-11T23:24:49.495 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:49.492+0000 7fe062458740 1 --2- 172.21.15.26:0/3455284014 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5649a63100c0 0x5649a6313e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:49.495 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:49.492+0000 7fe062458740 1 --2- 172.21.15.26:0/3455284014 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5649a638b590 0x5649a638d9b0 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:49.495 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:49.492+0000 7fe062458740 1 --2- 172.21.15.26:0/3455284014 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5649a63012d0 0x5649a6313930 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:49.495 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:49.492+0000 7fe062458740 1 --2- 172.21.15.26:0/3455284014 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5649a63009b0 0x5649a63185c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:49.495 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:49.492+0000 7fe062458740 1 -- 172.21.15.26:0/3455284014 >> 172.21.15.26:0/3455284014 conn(0x5649a62fd060 msgr2=0x5649a630d040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:24:49.495 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:49.492+0000 7fe040ff9640 0 client.16512 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:24:49.495 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:49.492+0000 7fe062458740 1 -- 172.21.15.26:0/3455284014 shutdown_connections 2024-09-11T23:24:49.495 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:49.492+0000 7fe062458740 1 -- 172.21.15.26:0/3455284014 wait complete. 2024-09-11T23:24:49.495 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:49.492+0000 7fe062458740 20 client.16512 trim_cache size 0 max 16384 2024-09-11T23:24:49.496 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-09-11T23:24:49.496 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "message": "", 2024-09-11T23:24:49.496 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-09-11T23:24:49.496 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-09-11T23:24:49.497 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:49.493+0000 7fe05f6ba640 1 -- 172.21.15.26:0/3818007373 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fe0440a2da0 msgr2=0x7fe0440a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:24:49.497 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:49.493+0000 7fe05f6ba640 1 --2- 172.21.15.26:0/3818007373 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fe0440a2da0 0x7fe0440a5260 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fe0540cd0a0 tx=0x7fe054004a40 comp rx=0 tx=0).stop 2024-09-11T23:24:49.497 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:49.493+0000 7fe05f6ba640 1 -- 172.21.15.26:0/3818007373 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe058101ae0 msgr2=0x7fe058198370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:24:49.497 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:49.493+0000 7fe05f6ba640 1 --2- 172.21.15.26:0/3818007373 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe058101ae0 0x7fe058198370 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fe05400e9f0 tx=0x7fe05400edf0 comp rx=0 tx=0).stop 2024-09-11T23:24:49.497 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:49.493+0000 7fe05f6ba640 1 -- 172.21.15.26:0/3818007373 shutdown_connections 2024-09-11T23:24:49.497 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:49.493+0000 7fe05f6ba640 1 --2- 172.21.15.26:0/3818007373 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fe0440a2da0 0x7fe0440a5260 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:49.497 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:49.493+0000 7fe05f6ba640 1 --2- 172.21.15.26:0/3818007373 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe058107b80 0x7fe05819cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:49.497 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:49.493+0000 7fe05f6ba640 1 --2- 172.21.15.26:0/3818007373 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe058102480 0x7fe0581988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:49.497 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:49.493+0000 7fe05f6ba640 1 --2- 172.21.15.26:0/3818007373 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe058101ae0 0x7fe058198370 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:49.498 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:49.493+0000 7fe05f6ba640 1 -- 172.21.15.26:0/3818007373 >> 172.21.15.26:0/3818007373 conn(0x7fe0580fd7e0 msgr2=0x7fe0580fde70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:24:49.498 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:49.493+0000 7fe05f6ba640 1 -- 172.21.15.26:0/3818007373 shutdown_connections 2024-09-11T23:24:49.498 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:24:49.493+0000 7fe05f6ba640 1 -- 172.21.15.26:0/3818007373 wait complete. 2024-09-11T23:24:49.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:24:49 smithi136 ceph-mon[35121]: pgmap v196: 97 pgs: 97 active+clean; 2.3 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 8.7 MiB/s wr, 430 op/s 2024-09-11T23:24:50.000 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:24:49 smithi026 ceph-mon[31172]: pgmap v196: 97 pgs: 97 active+clean; 2.3 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 8.7 MiB/s wr, 430 op/s 2024-09-11T23:24:50.052 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:24:49 smithi076 ceph-mon[35296]: pgmap v196: 97 pgs: 97 active+clean; 2.3 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 8.7 MiB/s wr, 430 op/s 2024-09-11T23:24:50.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:24:50 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:24:50.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:24:50 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:24:50.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:24:50 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:24:51.000 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:24:50 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:24:51.000 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:24:50 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:24:51.000 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:24:50 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:24:51.052 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:24:50 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:24:51.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:24:50 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:24:51.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:24:50 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:24:51.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:24:51 smithi136 ceph-mon[35121]: pgmap v197: 97 pgs: 97 active+clean; 2.3 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 10 MiB/s wr, 826 op/s 2024-09-11T23:24:52.000 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:24:51 smithi026 ceph-mon[31172]: pgmap v197: 97 pgs: 97 active+clean; 2.3 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 10 MiB/s wr, 826 op/s 2024-09-11T23:24:52.052 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:24:51 smithi076 ceph-mon[35296]: pgmap v197: 97 pgs: 97 active+clean; 2.3 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 10 MiB/s wr, 826 op/s 2024-09-11T23:24:53.579 INFO:tasks.workunit.client.0.smithi026.stdout: 613 546227 873 373683 964 162518 427 2024-09-11T23:24:53.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:24:53 smithi136 ceph-mon[35121]: pgmap v198: 97 pgs: 97 active+clean; 2.3 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.4 MiB/s wr, 520 op/s 2024-09-11T23:24:54.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:24:53 smithi026 ceph-mon[31172]: pgmap v198: 97 pgs: 97 active+clean; 2.3 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.4 MiB/s wr, 520 op/s 2024-09-11T23:24:54.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:24:53 smithi076 ceph-mon[35296]: pgmap v198: 97 pgs: 97 active+clean; 2.3 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.4 MiB/s wr, 520 op/s 2024-09-11T23:24:55.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:24:55 smithi136 ceph-mon[35121]: pgmap v199: 97 pgs: 97 active+clean; 2.3 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.0 MiB/s wr, 725 op/s 2024-09-11T23:24:56.000 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:24:55 smithi026 ceph-mon[31172]: pgmap v199: 97 pgs: 97 active+clean; 2.3 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.0 MiB/s wr, 725 op/s 2024-09-11T23:24:56.052 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:24:55 smithi076 ceph-mon[35296]: pgmap v199: 97 pgs: 97 active+clean; 2.3 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.0 MiB/s wr, 725 op/s 2024-09-11T23:24:57.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:24:57 smithi136 ceph-mon[35121]: pgmap v200: 97 pgs: 97 active+clean; 2.3 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.8 MiB/s wr, 684 op/s 2024-09-11T23:24:58.000 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:24:57 smithi026 ceph-mon[31172]: pgmap v200: 97 pgs: 97 active+clean; 2.3 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.8 MiB/s wr, 684 op/s 2024-09-11T23:24:58.052 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:24:57 smithi076 ceph-mon[35296]: pgmap v200: 97 pgs: 97 active+clean; 2.3 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.8 MiB/s wr, 684 op/s 2024-09-11T23:24:59.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:24:59 smithi136 ceph-mon[35121]: pgmap v201: 97 pgs: 97 active+clean; 2.3 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.6 MiB/s wr, 670 op/s 2024-09-11T23:25:00.000 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:24:59 smithi026 ceph-mon[31172]: pgmap v201: 97 pgs: 97 active+clean; 2.3 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.6 MiB/s wr, 670 op/s 2024-09-11T23:25:00.052 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:24:59 smithi076 ceph-mon[35296]: pgmap v201: 97 pgs: 97 active+clean; 2.3 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.6 MiB/s wr, 670 op/s 2024-09-11T23:25:01.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:25:01 smithi136 ceph-mon[35121]: pgmap v202: 97 pgs: 97 active+clean; 2.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 6.0 MiB/s wr, 1.01k op/s 2024-09-11T23:25:02.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:25:01 smithi026 ceph-mon[31172]: pgmap v202: 97 pgs: 97 active+clean; 2.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 6.0 MiB/s wr, 1.01k op/s 2024-09-11T23:25:02.052 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:25:01 smithi076 ceph-mon[35296]: pgmap v202: 97 pgs: 97 active+clean; 2.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 6.0 MiB/s wr, 1.01k op/s 2024-09-11T23:25:02.602 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.602+0000 7f70f3f18640 1 -- 172.21.15.26:0/1298155816 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70ec10e410 msgr2=0x7f70ec110800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:25:02.602 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.602+0000 7f70f3f18640 1 --2- 172.21.15.26:0/1298155816 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70ec10e410 0x7f70ec110800 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f70e4009ca0 tx=0x7f70e402a8c0 comp rx=0 tx=0).stop 2024-09-11T23:25:02.602 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.602+0000 7f70f3f18640 1 -- 172.21.15.26:0/1298155816 shutdown_connections 2024-09-11T23:25:02.602 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.602+0000 7f70f3f18640 1 --2- 172.21.15.26:0/1298155816 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f70ec101210 0x7f70ec10ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:02.602 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.602+0000 7f70f3f18640 1 --2- 172.21.15.26:0/1298155816 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70ec10e410 0x7f70ec110800 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:02.602 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.602+0000 7f70f3f18640 1 --2- 172.21.15.26:0/1298155816 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f70ec100860 0x7f70ec100c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:02.602 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.602+0000 7f70f3f18640 1 -- 172.21.15.26:0/1298155816 >> 172.21.15.26:0/1298155816 conn(0x7f70ec077990 msgr2=0x7f70ec0fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:25:02.603 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.602+0000 7f70f3f18640 1 -- 172.21.15.26:0/1298155816 shutdown_connections 2024-09-11T23:25:02.603 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.602+0000 7f70f3f18640 1 -- 172.21.15.26:0/1298155816 wait complete. 2024-09-11T23:25:02.604 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.603+0000 7f70f3f18640 1 Processor -- start 2024-09-11T23:25:02.604 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.603+0000 7f70f3f18640 1 -- start start 2024-09-11T23:25:02.604 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.603+0000 7f70f3f18640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f70ec100860 0x7f70ec19a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:02.605 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.603+0000 7f70f3f18640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f70ec101210 0x7f70ec19aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:02.605 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.603+0000 7f70f3f18640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70ec10e410 0x7f70ec19edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:02.605 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.603+0000 7f70f3f18640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f70ec114060 con 0x7f70ec10e410 2024-09-11T23:25:02.605 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.603+0000 7f70f3f18640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f70ec113ee0 con 0x7f70ec101210 2024-09-11T23:25:02.605 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.603+0000 7f70f3f18640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f70ec1141e0 con 0x7f70ec100860 2024-09-11T23:25:02.605 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.603+0000 7f70f24b5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70ec10e410 0x7f70ec19edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:02.605 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.603+0000 7f70f1cb4640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f70ec100860 0x7f70ec19a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:02.605 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.603+0000 7f70f14b3640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f70ec101210 0x7f70ec19aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:02.605 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.603+0000 7f70f1cb4640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f70ec100860 0x7f70ec19a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:49510/0 (socket says 172.21.15.26:49510) 2024-09-11T23:25:02.605 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.603+0000 7f70f1cb4640 1 -- 172.21.15.26:0/207931450 learned_addr learned my addr 172.21.15.26:0/207931450 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:25:02.605 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.603+0000 7f70f1cb4640 1 -- 172.21.15.26:0/207931450 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f70ec101210 msgr2=0x7f70ec19aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:25:02.605 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.603+0000 7f70f1cb4640 1 --2- 172.21.15.26:0/207931450 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f70ec101210 0x7f70ec19aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:02.605 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.604+0000 7f70f1cb4640 1 -- 172.21.15.26:0/207931450 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70ec10e410 msgr2=0x7f70ec19edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:25:02.606 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.604+0000 7f70f1cb4640 1 --2- 172.21.15.26:0/207931450 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70ec10e410 0x7f70ec19edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:02.606 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.604+0000 7f70f1cb4640 1 -- 172.21.15.26:0/207931450 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70ec19f4e0 con 0x7f70ec100860 2024-09-11T23:25:02.606 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.604+0000 7f70f24b5640 1 --2- 172.21.15.26:0/207931450 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70ec10e410 0x7f70ec19edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:25:02.606 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.604+0000 7f70f1cb4640 1 --2- 172.21.15.26:0/207931450 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f70ec100860 0x7f70ec19a4f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f70e800e9f0 tx=0x7f70e800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:25:02.607 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.605+0000 7f70e2ffd640 1 -- 172.21.15.26:0/207931450 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70e800cd80 con 0x7f70ec100860 2024-09-11T23:25:02.607 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.605+0000 7f70f3f18640 1 -- 172.21.15.26:0/207931450 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70ec19f7d0 con 0x7f70ec100860 2024-09-11T23:25:02.607 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.605+0000 7f70f3f18640 1 -- 172.21.15.26:0/207931450 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70ec1a7060 con 0x7f70ec100860 2024-09-11T23:25:02.607 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.606+0000 7f70e2ffd640 1 -- 172.21.15.26:0/207931450 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70e8010910 con 0x7f70ec100860 2024-09-11T23:25:02.607 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.606+0000 7f70e2ffd640 1 -- 172.21.15.26:0/207931450 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70e80188f0 con 0x7f70ec100860 2024-09-11T23:25:02.608 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.608+0000 7f70f6cb2740 1 Processor -- start 2024-09-11T23:25:02.609 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.608+0000 7f70f6cb2740 1 -- start start 2024-09-11T23:25:02.609 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.609+0000 7f70f6cb2740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561f4752caf0 0x561f4752ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:02.609 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.609+0000 7f70f6cb2740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561f4752d410 0x561f47537770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:02.609 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.609+0000 7f70f6cb2740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561f47537cb0 0x561f4753a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:02.609 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.609+0000 7f70f6cb2740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x561f474912f0 con 0x561f4752d410 2024-09-11T23:25:02.610 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.609+0000 7f70f6cb2740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x561f47439270 con 0x561f4752caf0 2024-09-11T23:25:02.610 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.609+0000 7f70f6cb2740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x561f4748d2b0 con 0x561f47537cb0 2024-09-11T23:25:02.610 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.609+0000 7f70f14b3640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561f47537cb0 0x561f4753a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:02.610 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.609+0000 7f70f14b3640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561f47537cb0 0x561f4753a0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:49514/0 (socket says 172.21.15.26:49514) 2024-09-11T23:25:02.610 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.609+0000 7f70f14b3640 1 -- 172.21.15.26:0/1336379813 learned_addr learned my addr 172.21.15.26:0/1336379813 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:25:02.610 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.609+0000 7f70f24b5640 1 --2- 172.21.15.26:0/1336379813 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561f4752caf0 0x561f4752ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:02.610 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.609+0000 7f70e2ffd640 1 -- 172.21.15.26:0/207931450 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f70e8018bd0 con 0x7f70ec100860 2024-09-11T23:25:02.610 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.609+0000 7f70f14b3640 1 -- 172.21.15.26:0/1336379813 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561f4752caf0 msgr2=0x561f4752ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:25:02.610 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.609+0000 7f70f14b3640 1 --2- 172.21.15.26:0/1336379813 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561f4752caf0 0x561f4752ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:02.610 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.609+0000 7f70f14b3640 1 -- 172.21.15.26:0/1336379813 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561f4752d410 msgr2=0x561f47537770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:25:02.610 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.609+0000 7f70f14b3640 1 --2- 172.21.15.26:0/1336379813 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561f4752d410 0x561f47537770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:02.610 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.609+0000 7f70f14b3640 1 -- 172.21.15.26:0/1336379813 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f47438340 con 0x561f47537cb0 2024-09-11T23:25:02.612 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.610+0000 7f70f14b3640 1 --2- 172.21.15.26:0/1336379813 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561f47537cb0 0x561f4753a0a0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f70dc00bc10 tx=0x7f70dc0107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c034f492a8ae99 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:25:02.612 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.610+0000 7f70d7fff640 1 -- 172.21.15.26:0/1336379813 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70dc005020 con 0x561f47537cb0 2024-09-11T23:25:02.612 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.610+0000 7f70d7fff640 1 -- 172.21.15.26:0/1336379813 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70dc0047a0 con 0x561f47537cb0 2024-09-11T23:25:02.612 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.610+0000 7f70e2ffd640 1 --2- 172.21.15.26:0/207931450 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f70d80a2e20 0x7f70d80a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:02.612 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.611+0000 7f70f24b5640 1 --2- 172.21.15.26:0/207931450 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f70d80a2e20 0x7f70d80a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:02.612 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.611+0000 7f70e2ffd640 1 -- 172.21.15.26:0/207931450 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f70e80cf650 con 0x7f70ec100860 2024-09-11T23:25:02.612 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.611+0000 7f70d7fff640 1 -- 172.21.15.26:0/1336379813 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70dc0117a0 con 0x561f47537cb0 2024-09-11T23:25:02.613 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.611+0000 7f70f6cb2740 1 -- 172.21.15.26:0/1336379813 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561f47537cb0 msgr2=0x561f4753a0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:25:02.613 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.611+0000 7f70f24b5640 1 --2- 172.21.15.26:0/207931450 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f70d80a2e20 0x7f70d80a52e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f70e4031f80 tx=0x7f70e4004a60 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:25:02.613 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.611+0000 7f70f6cb2740 1 --2- 172.21.15.26:0/1336379813 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561f47537cb0 0x561f4753a0a0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f70dc00bc10 tx=0x7f70dc0107a0 comp rx=0 tx=0).stop 2024-09-11T23:25:02.613 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.611+0000 7f70f6cb2740 1 -- 172.21.15.26:0/1336379813 shutdown_connections 2024-09-11T23:25:02.613 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.611+0000 7f70f6cb2740 1 --2- 172.21.15.26:0/1336379813 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561f47537cb0 0x561f4753a0a0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:02.613 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.611+0000 7f70f6cb2740 1 --2- 172.21.15.26:0/1336379813 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561f4752d410 0x561f47537770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:02.613 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.611+0000 7f70f6cb2740 1 --2- 172.21.15.26:0/1336379813 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561f4752caf0 0x561f4752ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:02.613 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.611+0000 7f70f6cb2740 1 -- 172.21.15.26:0/1336379813 >> 172.21.15.26:0/1336379813 conn(0x561f47529060 msgr2=0x561f47529440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:25:02.613 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.612+0000 7f70f6cb2740 1 -- 172.21.15.26:0/1336379813 shutdown_connections 2024-09-11T23:25:02.613 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.612+0000 7f70f6cb2740 1 -- 172.21.15.26:0/1336379813 wait complete. 2024-09-11T23:25:02.613 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.612+0000 7f70f6cb2740 1 -- start start 2024-09-11T23:25:02.613 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.612+0000 7f70f6cb2740 1 Processor -- start 2024-09-11T23:25:02.616 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.613+0000 7f70f6cb2740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561f4752caf0 0x561f47534ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:02.616 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.613+0000 7f70f6cb2740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561f4752d410 0x561f47535000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:02.616 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.613+0000 7f70f6cb2740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561f47537cb0 0x561f47535540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:02.616 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.613+0000 7f70f6cb2740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x561f4753a8d0 con 0x561f4752d410 2024-09-11T23:25:02.616 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.613+0000 7f70f6cb2740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x561f4741a290 con 0x561f47537cb0 2024-09-11T23:25:02.617 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.613+0000 7f70f1cb4640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561f4752caf0 0x561f47534ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:02.617 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.613+0000 7f70f6cb2740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x561f4753aa50 con 0x561f4752caf0 2024-09-11T23:25:02.617 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.613+0000 7f70f1cb4640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561f4752caf0 0x561f47534ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:49526/0 (socket says 172.21.15.26:49526) 2024-09-11T23:25:02.617 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.613+0000 7f70f1cb4640 1 -- 172.21.15.26:0/1333050544 learned_addr learned my addr 172.21.15.26:0/1333050544 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:25:02.617 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.613+0000 7f70f14b3640 1 --2- 172.21.15.26:0/1333050544 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561f4752d410 0x561f47535000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:02.617 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.613+0000 7f70f6cb2740 10 client.0 _ll_register_callbacks cb 0x561f4744b140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:25:02.617 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.613+0000 7f70f24b5640 1 --2- 172.21.15.26:0/1333050544 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561f47537cb0 0x561f47535540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:02.617 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.613+0000 7f70f1cb4640 1 -- 172.21.15.26:0/1333050544 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561f47537cb0 msgr2=0x561f47535540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:25:02.617 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.613+0000 7f70f1cb4640 1 --2- 172.21.15.26:0/1333050544 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561f47537cb0 0x561f47535540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:02.617 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.613+0000 7f70f1cb4640 1 -- 172.21.15.26:0/1333050544 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561f4752d410 msgr2=0x561f47535000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:25:02.617 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.613+0000 7f70f1cb4640 1 --2- 172.21.15.26:0/1333050544 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561f4752d410 0x561f47535000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:02.617 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.613+0000 7f70f1cb4640 1 -- 172.21.15.26:0/1333050544 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f471a4210 con 0x561f4752caf0 2024-09-11T23:25:02.617 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.613+0000 7f70f24b5640 1 --2- 172.21.15.26:0/1333050544 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561f47537cb0 0x561f47535540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:25:02.618 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.613+0000 7f70f1cb4640 1 --2- 172.21.15.26:0/1333050544 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561f4752caf0 0x561f47534ac0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f70e80cd3b0 tx=0x7f70e809bc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7339bd085a6dbc65 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:25:02.618 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.613+0000 7f70d57fa640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-09-11T23:25:02.618 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.614+0000 7f70d57fa640 1 -- 172.21.15.26:0/1333050544 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70e8086040 con 0x561f4752caf0 2024-09-11T23:25:02.618 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.614+0000 7f70d57fa640 1 -- 172.21.15.26:0/1333050544 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f70e809b280 con 0x561f4752caf0 2024-09-11T23:25:02.618 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.614+0000 7f70f6cb2740 1 -- 172.21.15.26:0/1333050544 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f70dc0047a0 con 0x561f4752caf0 2024-09-11T23:25:02.618 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.614+0000 7f70d57fa640 1 -- 172.21.15.26:0/1333050544 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f70e8010430 con 0x561f4752caf0 2024-09-11T23:25:02.618 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.614+0000 7f70f6cb2740 10 client.26143 fetch_fsmap learned FSMap version 41 2024-09-11T23:25:02.618 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.614+0000 7f70f6cb2740 1 -- 172.21.15.26:0/1333050544 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x561f47270620 con 0x561f4752caf0 2024-09-11T23:25:02.618 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.614+0000 7f70d57fa640 1 -- 172.21.15.26:0/1333050544 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70dc0047a0 con 0x561f4752caf0 2024-09-11T23:25:02.618 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.614+0000 7f70d57fa640 1 -- 172.21.15.26:0/1333050544 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 41) ==== 8426+0+0 (secure 0 0 0) 0x7f70e8010ac0 con 0x561f4752caf0 2024-09-11T23:25:02.618 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.614+0000 7f70f6cb2740 10 client.26143 fetch_fsmap finished waiting for FSMap version 41 2024-09-11T23:25:02.618 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.614+0000 7f70f6cb2740 10 client.26143 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:25:02.618 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.614+0000 7f70f6cb2740 20 client.26143 populate_metadata read hostname 'smithi026' 2024-09-11T23:25:02.618 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.614+0000 7f70f6cb2740 1 --2- 172.21.15.26:0/1333050544 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x561f47547c80 0x561f4754a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:02.619 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.614+0000 7f70f6cb2740 4 client.26143 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:25:02.619 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.614+0000 7f70f6cb2740 1 -- 172.21.15.26:0/1333050544 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561f4749ea20 con 0x561f47547c80 2024-09-11T23:25:02.619 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.615+0000 7f70f14b3640 1 --2- 172.21.15.26:0/1333050544 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x561f47547c80 0x561f4754a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:02.619 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.615+0000 7f70d57fa640 10 client.26143 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:25:02.619 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.615+0000 7f70f14b3640 1 --2- 172.21.15.26:0/1333050544 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x561f47547c80 0x561f4754a100 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e3ca396e53e4160 server_cookie=7bf17b5265eaa4ba in_seq=0 out_seq=0 2024-09-11T23:25:02.620 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.618+0000 7f70d57fa640 1 -- 172.21.15.26:0/1333050544 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561f4749ea20 con 0x561f47547c80 2024-09-11T23:25:02.620 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.618+0000 7f70d57fa640 10 client.26143 handle_command_reply: tid=0 2024-09-11T23:25:02.620 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.618+0000 7f70f6cb2740 1 client.26143 shutdown 2024-09-11T23:25:02.620 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.618+0000 7f70f6cb2740 1 -- 172.21.15.26:0/1333050544 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561f4752caf0 msgr2=0x561f47534ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:25:02.620 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.618+0000 7f70f6cb2740 1 --2- 172.21.15.26:0/1333050544 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561f4752caf0 0x561f47534ac0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f70e80cd3b0 tx=0x7f70e809bc30 comp rx=0 tx=0).stop 2024-09-11T23:25:02.620 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.618+0000 7f70f6cb2740 1 -- 172.21.15.26:0/1333050544 shutdown_connections 2024-09-11T23:25:02.620 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.618+0000 7f70f6cb2740 1 --2- 172.21.15.26:0/1333050544 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561f47537cb0 0x561f47535540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:02.621 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.618+0000 7f70f6cb2740 1 --2- 172.21.15.26:0/1333050544 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x561f47547c80 0x561f4754a100 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:02.621 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.618+0000 7f70f6cb2740 1 --2- 172.21.15.26:0/1333050544 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561f4752d410 0x561f47535000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:02.621 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.618+0000 7f70d57fa640 0 client.26143 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:25:02.621 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.618+0000 7f70f6cb2740 1 --2- 172.21.15.26:0/1333050544 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561f4752caf0 0x561f47534ac0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:02.621 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.618+0000 7f70f6cb2740 1 -- 172.21.15.26:0/1333050544 >> 172.21.15.26:0/1333050544 conn(0x561f47529060 msgr2=0x561f4752edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:25:02.621 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.618+0000 7f70f6cb2740 1 -- 172.21.15.26:0/1333050544 shutdown_connections 2024-09-11T23:25:02.621 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.618+0000 7f70f6cb2740 1 -- 172.21.15.26:0/1333050544 wait complete. 2024-09-11T23:25:02.621 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.618+0000 7f70f6cb2740 20 client.26143 trim_cache size 0 max 16384 2024-09-11T23:25:02.637 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.636+0000 7f70f6cb2740 1 Processor -- start 2024-09-11T23:25:02.637 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.636+0000 7f70f6cb2740 1 -- start start 2024-09-11T23:25:02.638 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.636+0000 7f70f6cb2740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561f4752caf0 0x561f4752ec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:02.638 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.636+0000 7f70f6cb2740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561f4752d410 0x561f4754f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:02.638 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.636+0000 7f70f6cb2740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561f47537cb0 0x561f4754ff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:02.638 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.637+0000 7f70f6cb2740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f70d000e750 con 0x561f4752caf0 2024-09-11T23:25:02.638 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.637+0000 7f70f6cb2740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f70d0010590 con 0x561f4752d410 2024-09-11T23:25:02.638 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.637+0000 7f70f6cb2740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f70d000eee0 con 0x561f47537cb0 2024-09-11T23:25:02.638 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.637+0000 7f70f1cb4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561f4752caf0 0x561f4752ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:02.638 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.637+0000 7f70f1cb4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561f4752caf0 0x561f4752ec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37116/0 (socket says 172.21.15.26:37116) 2024-09-11T23:25:02.638 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.637+0000 7f70f1cb4640 1 -- 172.21.15.26:0/1271599923 learned_addr learned my addr 172.21.15.26:0/1271599923 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:25:02.638 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.637+0000 7f70f14b3640 1 --2- 172.21.15.26:0/1271599923 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561f4752d410 0x561f4754f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:02.638 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.637+0000 7f70f24b5640 1 --2- 172.21.15.26:0/1271599923 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561f47537cb0 0x561f4754ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:02.638 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.637+0000 7f70f1cb4640 1 -- 172.21.15.26:0/1271599923 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561f47537cb0 msgr2=0x561f4754ff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:25:02.639 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.637+0000 7f70f1cb4640 1 --2- 172.21.15.26:0/1271599923 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561f47537cb0 0x561f4754ff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:02.639 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.637+0000 7f70f1cb4640 1 -- 172.21.15.26:0/1271599923 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561f4752d410 msgr2=0x561f4754f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:25:02.639 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.637+0000 7f70f1cb4640 1 --2- 172.21.15.26:0/1271599923 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561f4752d410 0x561f4754f9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:02.639 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.637+0000 7f70f1cb4640 1 -- 172.21.15.26:0/1271599923 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f47528e90 con 0x561f4752caf0 2024-09-11T23:25:02.639 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.637+0000 7f70f24b5640 1 --2- 172.21.15.26:0/1271599923 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561f47537cb0 0x561f4754ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:25:02.639 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.637+0000 7f70f1cb4640 1 --2- 172.21.15.26:0/1271599923 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561f4752caf0 0x561f4752ec60 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f70e8098600 tx=0x7f70e8000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab93db94c21d339c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:25:02.639 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.637+0000 7f70d77fe640 1 -- 172.21.15.26:0/1271599923 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70e80b8030 con 0x561f4752caf0 2024-09-11T23:25:02.639 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.638+0000 7f70d77fe640 1 -- 172.21.15.26:0/1271599923 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70e80105d0 con 0x561f4752caf0 2024-09-11T23:25:02.639 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.638+0000 7f70d77fe640 1 -- 172.21.15.26:0/1271599923 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70e8086070 con 0x561f4752caf0 2024-09-11T23:25:02.642 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.638+0000 7f70f6cb2740 1 -- 172.21.15.26:0/1271599923 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561f4752caf0 msgr2=0x561f4752ec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:25:02.642 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.638+0000 7f70f6cb2740 1 --2- 172.21.15.26:0/1271599923 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561f4752caf0 0x561f4752ec60 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f70e8098600 tx=0x7f70e8000bc0 comp rx=0 tx=0).stop 2024-09-11T23:25:02.642 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.639+0000 7f70f6cb2740 1 -- 172.21.15.26:0/1271599923 shutdown_connections 2024-09-11T23:25:02.642 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.639+0000 7f70f6cb2740 1 --2- 172.21.15.26:0/1271599923 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561f4752d410 0x561f4754f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:02.642 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.639+0000 7f70f6cb2740 1 --2- 172.21.15.26:0/1271599923 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561f47537cb0 0x561f4754ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:02.642 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.639+0000 7f70f6cb2740 1 --2- 172.21.15.26:0/1271599923 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561f4752caf0 0x561f4752ec60 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:02.642 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.639+0000 7f70f6cb2740 1 -- 172.21.15.26:0/1271599923 >> 172.21.15.26:0/1271599923 conn(0x561f47529060 msgr2=0x561f4752e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:25:02.642 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.639+0000 7f70f6cb2740 1 -- 172.21.15.26:0/1271599923 shutdown_connections 2024-09-11T23:25:02.642 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.639+0000 7f70f6cb2740 1 -- 172.21.15.26:0/1271599923 wait complete. 2024-09-11T23:25:02.643 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.639+0000 7f70f6cb2740 1 -- start start 2024-09-11T23:25:02.643 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.639+0000 7f70f6cb2740 1 Processor -- start 2024-09-11T23:25:02.643 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.640+0000 7f70f6cb2740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561f4752caf0 0x561f47540650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:02.643 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.640+0000 7f70f6cb2740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561f4752d410 0x561f4753d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:02.643 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.640+0000 7f70f6cb2740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561f47537cb0 0x561f4753dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:02.643 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.640+0000 7f70f6cb2740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f70d000fe00 con 0x561f4752caf0 2024-09-11T23:25:02.643 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.640+0000 7f70f1cb4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561f4752caf0 0x561f47540650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:02.643 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.640+0000 7f70f6cb2740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f70d000f670 con 0x561f47537cb0 2024-09-11T23:25:02.643 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.640+0000 7f70f1cb4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561f4752caf0 0x561f47540650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37128/0 (socket says 172.21.15.26:37128) 2024-09-11T23:25:02.643 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.640+0000 7f70f1cb4640 1 -- 172.21.15.26:0/1644949124 learned_addr learned my addr 172.21.15.26:0/1644949124 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:25:02.643 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.640+0000 7f70f6cb2740 1 -- 172.21.15.26:0/1644949124 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f70d000c730 con 0x561f4752d410 2024-09-11T23:25:02.643 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.640+0000 7f70f6cb2740 10 client.0 _ll_register_callbacks cb 0x7f70e8009700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:25:02.643 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.640+0000 7f70f24b5640 1 --2- 172.21.15.26:0/1644949124 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561f47537cb0 0x561f4753dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:02.644 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.640+0000 7f70f1cb4640 1 -- 172.21.15.26:0/1644949124 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561f4752d410 msgr2=0x561f4753d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:25:02.644 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.640+0000 7f70f1cb4640 1 --2- 172.21.15.26:0/1644949124 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561f4752d410 0x561f4753d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:02.644 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.640+0000 7f70f1cb4640 1 -- 172.21.15.26:0/1644949124 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561f47537cb0 msgr2=0x561f4753dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:25:02.644 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.640+0000 7f70f1cb4640 1 --2- 172.21.15.26:0/1644949124 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561f47537cb0 0x561f4753dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:02.644 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.640+0000 7f70f1cb4640 1 -- 172.21.15.26:0/1644949124 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f4753e7d0 con 0x561f4752caf0 2024-09-11T23:25:02.644 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.641+0000 7f70f1cb4640 1 --2- 172.21.15.26:0/1644949124 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561f4752caf0 0x561f47540650 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f70e8099ee0 tx=0x7f70e8010b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce51193fa3602ef9 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:25:02.644 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.641+0000 7f70d57fa640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:25:02.644 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.641+0000 7f70d57fa640 1 -- 172.21.15.26:0/1644949124 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70e80b8070 con 0x561f4752caf0 2024-09-11T23:25:02.644 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.641+0000 7f70d57fa640 1 -- 172.21.15.26:0/1644949124 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f70e809b280 con 0x561f4752caf0 2024-09-11T23:25:02.644 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.641+0000 7f70f6cb2740 1 -- 172.21.15.26:0/1644949124 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f70e80105d0 con 0x561f4752caf0 2024-09-11T23:25:02.644 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.641+0000 7f70d57fa640 1 -- 172.21.15.26:0/1644949124 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f70e8086050 con 0x561f4752caf0 2024-09-11T23:25:02.644 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.641+0000 7f70f6cb2740 10 client.16527 fetch_fsmap learned FSMap version 41 2024-09-11T23:25:02.644 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.641+0000 7f70f6cb2740 1 -- 172.21.15.26:0/1644949124 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x561f475b2af0 con 0x561f4752caf0 2024-09-11T23:25:02.644 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.641+0000 7f70d57fa640 1 -- 172.21.15.26:0/1644949124 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70e80105d0 con 0x561f4752caf0 2024-09-11T23:25:02.645 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.641+0000 7f70d57fa640 1 -- 172.21.15.26:0/1644949124 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 41) ==== 8426+0+0 (secure 0 0 0) 0x7f70e80a6b40 con 0x561f4752caf0 2024-09-11T23:25:02.645 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.641+0000 7f70f6cb2740 10 client.16527 fetch_fsmap finished waiting for FSMap version 41 2024-09-11T23:25:02.645 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.642+0000 7f70f6cb2740 10 client.16527 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:25:02.645 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.642+0000 7f70f6cb2740 20 client.16527 populate_metadata read hostname 'smithi026' 2024-09-11T23:25:02.645 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.642+0000 7f70f6cb2740 1 --2- 172.21.15.26:0/1644949124 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x561f47547c80 0x561f475b4f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:02.645 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.642+0000 7f70f6cb2740 4 client.16527 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T23:25:02.645 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.642+0000 7f70f6cb2740 1 -- 172.21.15.26:0/1644949124 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561f47498d70 con 0x561f47547c80 2024-09-11T23:25:02.645 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.642+0000 7f70f14b3640 1 --2- 172.21.15.26:0/1644949124 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x561f47547c80 0x561f475b4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:02.645 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.643+0000 7f70d57fa640 10 client.16527 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:25:02.645 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:02.643+0000 7f70f14b3640 1 --2- 172.21.15.26:0/1644949124 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x561f47547c80 0x561f475b4f90 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e89fabaaf32fbc server_cookie=b5f89e24d2cd6d8b in_seq=0 out_seq=0 2024-09-11T23:25:03.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:25:03 smithi136 ceph-mon[35121]: pgmap v203: 97 pgs: 97 active+clean; 2.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 5.2 MiB/s wr, 718 op/s 2024-09-11T23:25:04.000 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:25:03 smithi026 ceph-mon[31172]: pgmap v203: 97 pgs: 97 active+clean; 2.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 5.2 MiB/s wr, 718 op/s 2024-09-11T23:25:04.033 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:04.033+0000 7f70d57fa640 1 -- 172.21.15.26:0/1644949124 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561f47498d70 con 0x561f47547c80 2024-09-11T23:25:04.033 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:04.033+0000 7f70d57fa640 10 client.16527 handle_command_reply: tid=0 2024-09-11T23:25:04.033 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:04.033+0000 7f70f6cb2740 1 client.16527 shutdown 2024-09-11T23:25:04.035 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:04.034+0000 7f70f6cb2740 1 -- 172.21.15.26:0/1644949124 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561f4752caf0 msgr2=0x561f47540650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:25:04.035 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:04.034+0000 7f70f6cb2740 1 --2- 172.21.15.26:0/1644949124 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561f4752caf0 0x561f47540650 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f70e8099ee0 tx=0x7f70e8010b30 comp rx=0 tx=0).stop 2024-09-11T23:25:04.035 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:04.034+0000 7f70f6cb2740 1 -- 172.21.15.26:0/1644949124 shutdown_connections 2024-09-11T23:25:04.035 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:04.034+0000 7f70f6cb2740 1 --2- 172.21.15.26:0/1644949124 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561f47537cb0 0x561f4753dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:04.035 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:04.034+0000 7f70f6cb2740 1 --2- 172.21.15.26:0/1644949124 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x561f47547c80 0x561f475b4f90 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:04.035 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:04.034+0000 7f70d57fa640 0 client.16527 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:25:04.035 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:04.034+0000 7f70f6cb2740 1 --2- 172.21.15.26:0/1644949124 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561f4752d410 0x561f4753d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:04.035 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:04.034+0000 7f70f6cb2740 1 --2- 172.21.15.26:0/1644949124 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561f4752caf0 0x561f47540650 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:04.035 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:04.034+0000 7f70f6cb2740 1 -- 172.21.15.26:0/1644949124 >> 172.21.15.26:0/1644949124 conn(0x561f47529060 msgr2=0x561f4752ed70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:25:04.035 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:04.035+0000 7f70f6cb2740 1 -- 172.21.15.26:0/1644949124 shutdown_connections 2024-09-11T23:25:04.035 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:04.035+0000 7f70f6cb2740 1 -- 172.21.15.26:0/1644949124 wait complete. 2024-09-11T23:25:04.036 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-09-11T23:25:04.036 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "message": "", 2024-09-11T23:25:04.036 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-09-11T23:25:04.036 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-09-11T23:25:04.037 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:04.035+0000 7f70f6cb2740 20 client.16527 trim_cache size 0 max 16384 2024-09-11T23:25:04.037 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:04.036+0000 7f70f3f18640 1 -- 172.21.15.26:0/207931450 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f70d80a2e20 msgr2=0x7f70d80a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:25:04.037 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:04.036+0000 7f70f3f18640 1 --2- 172.21.15.26:0/207931450 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f70d80a2e20 0x7f70d80a52e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f70e4031f80 tx=0x7f70e4004a60 comp rx=0 tx=0).stop 2024-09-11T23:25:04.037 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:04.036+0000 7f70f3f18640 1 -- 172.21.15.26:0/207931450 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f70ec100860 msgr2=0x7f70ec19a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:25:04.037 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:04.036+0000 7f70f3f18640 1 --2- 172.21.15.26:0/207931450 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f70ec100860 0x7f70ec19a4f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f70e800e9f0 tx=0x7f70e800eeb0 comp rx=0 tx=0).stop 2024-09-11T23:25:04.037 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:04.036+0000 7f70f3f18640 1 -- 172.21.15.26:0/207931450 shutdown_connections 2024-09-11T23:25:04.037 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:04.036+0000 7f70f3f18640 1 --2- 172.21.15.26:0/207931450 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f70d80a2e20 0x7f70d80a52e0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:04.037 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:04.036+0000 7f70f3f18640 1 --2- 172.21.15.26:0/207931450 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f70ec101210 0x7f70ec19aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:04.037 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:04.036+0000 7f70f3f18640 1 --2- 172.21.15.26:0/207931450 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70ec10e410 0x7f70ec19edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:04.037 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:04.036+0000 7f70f3f18640 1 --2- 172.21.15.26:0/207931450 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f70ec100860 0x7f70ec19a4f0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:04.038 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:04.036+0000 7f70f3f18640 1 -- 172.21.15.26:0/207931450 >> 172.21.15.26:0/207931450 conn(0x7f70ec077990 msgr2=0x7f70ec0ff540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:25:04.038 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:04.036+0000 7f70f3f18640 1 -- 172.21.15.26:0/207931450 shutdown_connections 2024-09-11T23:25:04.038 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:04.036+0000 7f70f3f18640 1 -- 172.21.15.26:0/207931450 wait complete. 2024-09-11T23:25:04.052 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:25:03 smithi076 ceph-mon[35296]: pgmap v203: 97 pgs: 97 active+clean; 2.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 5.2 MiB/s wr, 718 op/s 2024-09-11T23:25:05.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:25:05 smithi136 ceph-mon[35121]: pgmap v204: 97 pgs: 97 active+clean; 2.4 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 9.6 MiB/s wr, 1.22k op/s 2024-09-11T23:25:06.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:25:05 smithi026 ceph-mon[31172]: pgmap v204: 97 pgs: 97 active+clean; 2.4 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 9.6 MiB/s wr, 1.22k op/s 2024-09-11T23:25:06.052 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:25:05 smithi076 ceph-mon[35296]: pgmap v204: 97 pgs: 97 active+clean; 2.4 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 9.6 MiB/s wr, 1.22k op/s 2024-09-11T23:25:07.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:25:07 smithi136 ceph-mon[35121]: pgmap v205: 97 pgs: 97 active+clean; 2.4 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 11 MiB/s wr, 1.09k op/s 2024-09-11T23:25:07.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:25:07 smithi136 ceph-mon[35121]: osdmap e87: 12 total, 12 up, 12 in 2024-09-11T23:25:08.000 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:25:07 smithi026 ceph-mon[31172]: pgmap v205: 97 pgs: 97 active+clean; 2.4 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 11 MiB/s wr, 1.09k op/s 2024-09-11T23:25:08.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:25:07 smithi026 ceph-mon[31172]: osdmap e87: 12 total, 12 up, 12 in 2024-09-11T23:25:08.052 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:25:07 smithi076 ceph-mon[35296]: pgmap v205: 97 pgs: 97 active+clean; 2.4 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 11 MiB/s wr, 1.09k op/s 2024-09-11T23:25:08.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:25:07 smithi076 ceph-mon[35296]: osdmap e87: 12 total, 12 up, 12 in 2024-09-11T23:25:09.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:25:09 smithi136 ceph-mon[35121]: pgmap v207: 97 pgs: 97 active+clean; 2.4 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 13 MiB/s wr, 1.30k op/s 2024-09-11T23:25:10.000 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:25:09 smithi026 ceph-mon[31172]: pgmap v207: 97 pgs: 97 active+clean; 2.4 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 13 MiB/s wr, 1.30k op/s 2024-09-11T23:25:10.052 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:25:09 smithi076 ceph-mon[35296]: pgmap v207: 97 pgs: 97 active+clean; 2.4 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 13 MiB/s wr, 1.30k op/s 2024-09-11T23:25:11.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:25:11 smithi136 ceph-mon[35121]: pgmap v208: 97 pgs: 21 active+clean+snaptrim, 36 active+clean+snaptrim_wait, 40 active+clean; 2.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 15 MiB/s wr, 1.34k op/s 2024-09-11T23:25:12.000 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:25:11 smithi026 ceph-mon[31172]: pgmap v208: 97 pgs: 21 active+clean+snaptrim, 36 active+clean+snaptrim_wait, 40 active+clean; 2.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 15 MiB/s wr, 1.34k op/s 2024-09-11T23:25:12.052 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:25:11 smithi076 ceph-mon[35296]: pgmap v208: 97 pgs: 21 active+clean+snaptrim, 36 active+clean+snaptrim_wait, 40 active+clean; 2.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 15 MiB/s wr, 1.34k op/s 2024-09-11T23:25:13.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:25:13 smithi136 ceph-mon[35121]: pgmap v209: 97 pgs: 23 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 33 active+clean; 2.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 14 MiB/s wr, 1.22k op/s 2024-09-11T23:25:14.000 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:25:13 smithi026 ceph-mon[31172]: pgmap v209: 97 pgs: 23 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 33 active+clean; 2.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 14 MiB/s wr, 1.22k op/s 2024-09-11T23:25:14.052 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:25:13 smithi076 ceph-mon[35296]: pgmap v209: 97 pgs: 23 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 33 active+clean; 2.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 14 MiB/s wr, 1.22k op/s 2024-09-11T23:25:14.794 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-11T23:25:14.794 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 41, 2024-09-11T23:25:14.794 INFO:teuthology.orchestra.run.smithi026.stdout: "leader": 24485, 2024-09-11T23:25:14.794 INFO:teuthology.orchestra.run.smithi026.stdout: "set_version": 22, 2024-09-11T23:25:14.794 INFO:teuthology.orchestra.run.smithi026.stdout: "sets": { 2024-09-11T23:25:14.794 INFO:teuthology.orchestra.run.smithi026.stdout: "27fb6b2d": { 2024-09-11T23:25:14.794 INFO:teuthology.orchestra.run.smithi026.stdout: "version": 22, 2024-09-11T23:25:14.794 INFO:teuthology.orchestra.run.smithi026.stdout: "age_ref": 0.0, 2024-09-11T23:25:14.794 INFO:teuthology.orchestra.run.smithi026.stdout: "state": { 2024-09-11T23:25:14.794 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "QUIESCED", 2024-09-11T23:25:14.794 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.0 2024-09-11T23:25:14.795 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:25:14.795 INFO:teuthology.orchestra.run.smithi026.stdout: "timeout": 90.0, 2024-09-11T23:25:14.795 INFO:teuthology.orchestra.run.smithi026.stdout: "expiration": 140.1, 2024-09-11T23:25:14.795 INFO:teuthology.orchestra.run.smithi026.stdout: "members": { 2024-09-11T23:25:14.795 INFO:teuthology.orchestra.run.smithi026.stdout: "file:/": { 2024-09-11T23:25:14.795 INFO:teuthology.orchestra.run.smithi026.stdout: "excluded": false, 2024-09-11T23:25:14.795 INFO:teuthology.orchestra.run.smithi026.stdout: "state": { 2024-09-11T23:25:14.795 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "QUIESCED", 2024-09-11T23:25:14.795 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.0 2024-09-11T23:25:14.795 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:25:14.795 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:25:14.795 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:25:14.795 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:25:14.795 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:25:14.796 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-11T23:25:14.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:14.793+0000 7f7aa0ff9640 1 -- 172.21.15.26:0/1485906251 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x559f5ecf59d0 con 0x559f5ee1abf0 2024-09-11T23:25:14.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:14.793+0000 7f7aa0ff9640 10 client.26113 handle_command_reply: tid=0 2024-09-11T23:25:14.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:14.793+0000 7f7ac1b03740 1 client.26113 shutdown 2024-09-11T23:25:14.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:14.793+0000 7f7ac1b03740 1 -- 172.21.15.26:0/1485906251 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559f5ed83080 msgr2=0x559f5ed929e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:25:14.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:14.793+0000 7f7ac1b03740 1 --2- 172.21.15.26:0/1485906251 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559f5ed83080 0x559f5ed929e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x559f5ed7fb50 tx=0x7f7aa80046e0 comp rx=0 tx=0).stop 2024-09-11T23:25:14.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:14.793+0000 7f7ac1b03740 1 -- 172.21.15.26:0/1485906251 shutdown_connections 2024-09-11T23:25:14.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:14.793+0000 7f7ac1b03740 1 --2- 172.21.15.26:0/1485906251 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x559f5ee1abf0 0x559f5ee1d010 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:14.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:14.793+0000 7f7ac1b03740 1 --2- 172.21.15.26:0/1485906251 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559f5ed8d940 0x559f5ed92f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:14.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:14.793+0000 7f7ac1b03740 1 --2- 172.21.15.26:0/1485906251 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559f5ed83080 0x559f5ed929e0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:14.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:14.793+0000 7f7ac1b03740 1 --2- 172.21.15.26:0/1485906251 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559f5ed82760 0x559f5ed96040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:14.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:14.793+0000 7f7aa0ff9640 0 client.26113 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:25:14.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:14.793+0000 7f7ac1b03740 1 -- 172.21.15.26:0/1485906251 >> 172.21.15.26:0/1485906251 conn(0x559f5ed7ecf0 msgr2=0x559f5ed841b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:25:14.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:14.793+0000 7f7ac1b03740 1 -- 172.21.15.26:0/1485906251 shutdown_connections 2024-09-11T23:25:14.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:14.793+0000 7f7ac1b03740 1 -- 172.21.15.26:0/1485906251 wait complete. 2024-09-11T23:25:14.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:14.793+0000 7f7ac1b03740 20 client.26113 trim_cache size 0 max 16384 2024-09-11T23:25:14.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:14.794+0000 7f7abf72a640 1 -- 172.21.15.26:0/582185800 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f7aa40a2d70 msgr2=0x7f7aa40a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:25:14.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:14.794+0000 7f7abf72a640 1 --2- 172.21.15.26:0/582185800 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f7aa40a2d70 0x7f7aa40a5230 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f7ab811d8d0 tx=0x7f7aa8006bd0 comp rx=0 tx=0).stop 2024-09-11T23:25:14.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:14.794+0000 7f7abf72a640 1 -- 172.21.15.26:0/582185800 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7ab8109450 msgr2=0x7f7ab81a7470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:25:14.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:14.794+0000 7f7abf72a640 1 --2- 172.21.15.26:0/582185800 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7ab8109450 0x7f7ab81a7470 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f7ab0006010 tx=0x7f7ab0031730 comp rx=0 tx=0).stop 2024-09-11T23:25:14.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:14.794+0000 7f7abf72a640 1 -- 172.21.15.26:0/582185800 shutdown_connections 2024-09-11T23:25:14.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:14.794+0000 7f7abf72a640 1 --2- 172.21.15.26:0/582185800 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f7aa40a2d70 0x7f7aa40a5230 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:14.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:14.794+0000 7f7abf72a640 1 --2- 172.21.15.26:0/582185800 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7ab811b140 0x7f7ab81abd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:14.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:14.794+0000 7f7abf72a640 1 --2- 172.21.15.26:0/582185800 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7ab8109830 0x7f7ab81a79b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:14.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:14.794+0000 7f7abf72a640 1 --2- 172.21.15.26:0/582185800 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7ab8109450 0x7f7ab81a7470 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:14.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:14.794+0000 7f7abf72a640 1 -- 172.21.15.26:0/582185800 >> 172.21.15.26:0/582185800 conn(0x7f7ab806e140 msgr2=0x7f7ab811bdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:25:14.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:14.794+0000 7f7abf72a640 1 -- 172.21.15.26:0/582185800 shutdown_connections 2024-09-11T23:25:14.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:14.794+0000 7f7abf72a640 1 -- 172.21.15.26:0/582185800 wait complete. 2024-09-11T23:25:15.717 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '27fb6b2d', quiesce took 39.4 seconds. Will release after: -19.299999999999997 2024-09-11T23:25:15.717 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:25:15.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:25:15 smithi136 ceph-mon[35121]: pgmap v210: 97 pgs: 6 active+clean+snaptrim, 15 active+clean+snaptrim_wait, 76 active+clean; 2.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 14 MiB/s wr, 750 op/s 2024-09-11T23:25:16.000 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:25:15 smithi026 ceph-mon[31172]: pgmap v210: 97 pgs: 6 active+clean+snaptrim, 15 active+clean+snaptrim_wait, 76 active+clean; 2.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 14 MiB/s wr, 750 op/s 2024-09-11T23:25:16.050 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:25:16.052 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:25:15 smithi076 ceph-mon[35296]: pgmap v210: 97 pgs: 6 active+clean+snaptrim, 15 active+clean+snaptrim_wait, 76 active+clean; 2.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 14 MiB/s wr, 750 op/s 2024-09-11T23:25:16.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:16.542+0000 7f63dddb4640 1 -- 172.21.15.26:0/172597883 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63d8100e20 msgr2=0x7f63d8101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:25:16.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:16.542+0000 7f63dddb4640 1 --2- 172.21.15.26:0/172597883 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63d8100e20 0x7f63d8101200 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f63c8009900 tx=0x7f63c802a480 comp rx=0 tx=0).stop 2024-09-11T23:25:16.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:16.542+0000 7f63dddb4640 1 -- 172.21.15.26:0/172597883 shutdown_connections 2024-09-11T23:25:16.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:16.542+0000 7f63dddb4640 1 --2- 172.21.15.26:0/172597883 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f63d8102570 0x7f63d81029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:16.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:16.542+0000 7f63dddb4640 1 --2- 172.21.15.26:0/172597883 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f63d8102ef0 0x7f63d810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:16.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:16.542+0000 7f63dddb4640 1 --2- 172.21.15.26:0/172597883 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63d8100e20 0x7f63d8101200 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:16.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:16.542+0000 7f63dddb4640 1 -- 172.21.15.26:0/172597883 >> 172.21.15.26:0/172597883 conn(0x7f63d80fc7d0 msgr2=0x7f63d80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:25:16.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:16.542+0000 7f63dddb4640 1 -- 172.21.15.26:0/172597883 shutdown_connections 2024-09-11T23:25:16.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:16.542+0000 7f63dddb4640 1 -- 172.21.15.26:0/172597883 wait complete. 2024-09-11T23:25:16.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:16.543+0000 7f63dddb4640 1 Processor -- start 2024-09-11T23:25:16.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:16.543+0000 7f63dddb4640 1 -- start start 2024-09-11T23:25:16.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:16.543+0000 7f63dddb4640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f63d8100e20 0x7f63d81992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:16.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:16.543+0000 7f63dddb4640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f63d8102570 0x7f63d8199830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:16.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:16.543+0000 7f63dddb4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63d8102ef0 0x7f63d81a4370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:16.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:16.543+0000 7f63dddb4640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f63d8110f80 con 0x7f63d8102ef0 2024-09-11T23:25:16.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:16.543+0000 7f63dddb4640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f63d8110e00 con 0x7f63d8102570 2024-09-11T23:25:16.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:16.543+0000 7f63dddb4640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f63d8111100 con 0x7f63d8100e20 2024-09-11T23:25:16.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:16.543+0000 7f63ceffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f63d8102570 0x7f63d8199830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:16.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:16.543+0000 7f63d77fe640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f63d8100e20 0x7f63d81992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:16.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:16.544+0000 7f63ceffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f63d8102570 0x7f63d8199830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:38546/0 (socket says 172.21.15.26:38546) 2024-09-11T23:25:16.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:16.544+0000 7f63d77fe640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f63d8100e20 0x7f63d81992f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:55142/0 (socket says 172.21.15.26:55142) 2024-09-11T23:25:16.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:16.544+0000 7f63ceffd640 1 -- 172.21.15.26:0/893183842 learned_addr learned my addr 172.21.15.26:0/893183842 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:25:16.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:16.544+0000 7f63d7fff640 1 --2- 172.21.15.26:0/893183842 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63d8102ef0 0x7f63d81a4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:16.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:16.544+0000 7f63ceffd640 1 -- 172.21.15.26:0/893183842 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f63d8100e20 msgr2=0x7f63d81992f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:25:16.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:16.544+0000 7f63ceffd640 1 --2- 172.21.15.26:0/893183842 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f63d8100e20 0x7f63d81992f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:16.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:16.544+0000 7f63ceffd640 1 -- 172.21.15.26:0/893183842 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63d8102ef0 msgr2=0x7f63d81a4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:25:16.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:16.544+0000 7f63ceffd640 1 --2- 172.21.15.26:0/893183842 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63d8102ef0 0x7f63d81a4370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:16.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:16.544+0000 7f63ceffd640 1 -- 172.21.15.26:0/893183842 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63d81a48b0 con 0x7f63d8102570 2024-09-11T23:25:16.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:16.544+0000 7f63d77fe640 1 --2- 172.21.15.26:0/893183842 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f63d8100e20 0x7f63d81992f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:25:16.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:16.544+0000 7f63ceffd640 1 --2- 172.21.15.26:0/893183842 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f63d8102570 0x7f63d8199830 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f63c000e9f0 tx=0x7f63c000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:25:16.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:16.544+0000 7f63d57fa640 1 -- 172.21.15.26:0/893183842 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63c000cd80 con 0x7f63d8102570 2024-09-11T23:25:16.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:16.544+0000 7f63dddb4640 1 -- 172.21.15.26:0/893183842 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63d81a4a80 con 0x7f63d8102570 2024-09-11T23:25:16.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:16.544+0000 7f63d57fa640 1 -- 172.21.15.26:0/893183842 <== mon.2 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63c0010910 con 0x7f63d8102570 2024-09-11T23:25:16.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:16.544+0000 7f63d57fa640 1 -- 172.21.15.26:0/893183842 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63c00188f0 con 0x7f63d8102570 2024-09-11T23:25:16.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:16.544+0000 7f63dddb4640 1 -- 172.21.15.26:0/893183842 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63d81a5010 con 0x7f63d8102570 2024-09-11T23:25:16.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:16.545+0000 7f63dddb4640 1 -- 172.21.15.26:0/893183842 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63d8101200 con 0x7f63d8102570 2024-09-11T23:25:16.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:16.547+0000 7f63d57fa640 1 -- 172.21.15.26:0/893183842 <== mon.2 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f63c0010ac0 con 0x7f63d8102570 2024-09-11T23:25:16.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:16.547+0000 7f63d57fa640 1 --2- 172.21.15.26:0/893183842 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f63c40a2cf0 0x7f63c40a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:16.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:16.548+0000 7f63d77fe640 1 --2- 172.21.15.26:0/893183842 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f63c40a2cf0 0x7f63c40a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:16.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:16.548+0000 7f63d57fa640 1 -- 172.21.15.26:0/893183842 <== mon.2 v2:172.21.15.76:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8586+0+0 (secure 0 0 0) 0x7f63c00cfd40 con 0x7f63d8102570 2024-09-11T23:25:16.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:16.548+0000 7f63d77fe640 1 --2- 172.21.15.26:0/893183842 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f63c40a2cf0 0x7f63c40a51b0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f63c8009230 tx=0x7f63c80058d0 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:25:16.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:16.551+0000 7f63d57fa640 1 -- 172.21.15.26:0/893183842 <== mon.2 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f63c009c270 con 0x7f63d8102570 2024-09-11T23:25:16.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:16.784+0000 7f63dddb4640 1 -- 172.21.15.26:0/893183842 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f63d806a5a0 con 0x7f63d8102570 2024-09-11T23:25:16.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:16.785+0000 7f63d57fa640 1 -- 172.21.15.26:0/893183842 <== mon.2 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) ==== 94+0+12000 (secure 0 0 0) 0x7f63d806a5a0 con 0x7f63d8102570 2024-09-11T23:25:16.785 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-11T23:25:16.785 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":41,"btime":"2024-09-11T23:23:46:852133+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14499,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/367659515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":367659515},{"type":"v1","addr":"172.21.15.26:6835","nonce":367659515}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14511,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/2041065299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":2041065299},{"type":"v1","addr":"172.21.15.26:6837","nonce":2041065299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":41,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:16:26.544203+0000","modified":"2024-09-11T23:23:46.128951+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24485,"mds_1":24463,"mds_2":24458,"mds_3":24448,"mds_4":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14550":{"gid":14550,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6839/2233068305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":2233068305},{"type":"v1","addr":"172.21.15.26:6839","nonce":2233068305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14562":{"gid":14562,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6841/2984982336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":2984982336},{"type":"v1","addr":"172.21.15.26:6841","nonce":2984982336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6833/2019841455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":2019841455},{"type":"v1","addr":"172.21.15.136:6833","nonce":2019841455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.76:6833/3438147050","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3438147050},{"type":"v1","addr":"172.21.15.76:6833","nonce":3438147050}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6835/3617665975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3617665975},{"type":"v1","addr":"172.21.15.136:6835","nonce":3617665975}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":12,"addr":"172.21.15.76:6835/393308499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":393308499},{"type":"v1","addr":"172.21.15.76:6835","nonce":393308499}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":10,"addr":"172.21.15.136:6837/1633082387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":1633082387},{"type":"v1","addr":"172.21.15.136:6837","nonce":1633082387}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.76:6837/2078978748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":2078978748},{"type":"v1","addr":"172.21.15.76:6837","nonce":2078978748}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":63,"addr":"172.21.15.136:6839/3211398630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":3211398630},{"type":"v1","addr":"172.21.15.136:6839","nonce":3211398630}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.76:6839/2077637090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6838","nonce":2077637090},{"type":"v1","addr":"172.21.15.76:6839","nonce":2077637090}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24485,"qdb_cluster":[24443,24448,24458,24463,24485]},"id":1}]} 2024-09-11T23:25:16.786 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 41 2024-09-11T23:25:16.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:16.787+0000 7f63dddb4640 1 -- 172.21.15.26:0/893183842 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f63c40a2cf0 msgr2=0x7f63c40a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:25:16.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:16.787+0000 7f63dddb4640 1 --2- 172.21.15.26:0/893183842 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f63c40a2cf0 0x7f63c40a51b0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f63c8009230 tx=0x7f63c80058d0 comp rx=0 tx=0).stop 2024-09-11T23:25:16.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:16.787+0000 7f63dddb4640 1 -- 172.21.15.26:0/893183842 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f63d8102570 msgr2=0x7f63d8199830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:25:16.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:16.787+0000 7f63dddb4640 1 --2- 172.21.15.26:0/893183842 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f63d8102570 0x7f63d8199830 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f63c000e9f0 tx=0x7f63c000eeb0 comp rx=0 tx=0).stop 2024-09-11T23:25:16.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:16.787+0000 7f63dddb4640 1 -- 172.21.15.26:0/893183842 shutdown_connections 2024-09-11T23:25:16.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:16.787+0000 7f63dddb4640 1 --2- 172.21.15.26:0/893183842 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f63c40a2cf0 0x7f63c40a51b0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:16.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:16.787+0000 7f63dddb4640 1 --2- 172.21.15.26:0/893183842 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f63d8102570 0x7f63d8199830 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:16.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:16.787+0000 7f63dddb4640 1 --2- 172.21.15.26:0/893183842 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63d8102ef0 0x7f63d81a4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:16.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:16.787+0000 7f63dddb4640 1 --2- 172.21.15.26:0/893183842 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f63d8100e20 0x7f63d81992f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:16.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:16.787+0000 7f63dddb4640 1 -- 172.21.15.26:0/893183842 >> 172.21.15.26:0/893183842 conn(0x7f63d80fc7d0 msgr2=0x7f63d80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:25:16.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:16.787+0000 7f63dddb4640 1 -- 172.21.15.26:0/893183842 shutdown_connections 2024-09-11T23:25:16.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:16.788+0000 7f63dddb4640 1 -- 172.21.15.26:0/893183842 wait complete. 2024-09-11T23:25:17.140 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.140+0000 7f0fb42fa640 1 -- 172.21.15.26:0/3314404952 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0fac1069c0 msgr2=0x7f0fac106da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:25:17.140 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.140+0000 7f0fb42fa640 1 --2- 172.21.15.26:0/3314404952 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0fac1069c0 0x7f0fac106da0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f0fa8009930 tx=0x7f0fa802a420 comp rx=0 tx=0).stop 2024-09-11T23:25:17.141 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.140+0000 7f0fb42fa640 1 -- 172.21.15.26:0/3314404952 shutdown_connections 2024-09-11T23:25:17.141 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.140+0000 7f0fb42fa640 1 --2- 172.21.15.26:0/3314404952 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0fac1009b0 0x7f0fac100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:17.141 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.140+0000 7f0fb42fa640 1 --2- 172.21.15.26:0/3314404952 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0fac101350 0x7f0fac10d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:17.141 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.140+0000 7f0fb42fa640 1 --2- 172.21.15.26:0/3314404952 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0fac1069c0 0x7f0fac106da0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:17.141 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.140+0000 7f0fb42fa640 1 -- 172.21.15.26:0/3314404952 >> 172.21.15.26:0/3314404952 conn(0x7f0fac077990 msgr2=0x7f0fac0fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:25:17.141 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.140+0000 7f0fb42fa640 1 -- 172.21.15.26:0/3314404952 shutdown_connections 2024-09-11T23:25:17.141 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.140+0000 7f0fb42fa640 1 -- 172.21.15.26:0/3314404952 wait complete. 2024-09-11T23:25:17.141 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.141+0000 7f0fb42fa640 1 Processor -- start 2024-09-11T23:25:17.141 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.141+0000 7f0fb42fa640 1 -- start start 2024-09-11T23:25:17.143 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.141+0000 7f0fb42fa640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0fac1009b0 0x7f0fac198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:17.143 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.141+0000 7f0fb42fa640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0fac101350 0x7f0fac198840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:17.143 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.141+0000 7f0fb42fa640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0fac1069c0 0x7f0fac19cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:17.143 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.141+0000 7f0fb42fa640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f0fac110e00 con 0x7f0fac101350 2024-09-11T23:25:17.143 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.141+0000 7f0fb42fa640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f0fac110c80 con 0x7f0fac1009b0 2024-09-11T23:25:17.143 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.141+0000 7f0fb42fa640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f0fac110f80 con 0x7f0fac1069c0 2024-09-11T23:25:17.143 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.141+0000 7f0fb1895640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0fac101350 0x7f0fac198840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:17.143 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.141+0000 7f0fb2096640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0fac1009b0 0x7f0fac198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:17.144 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.141+0000 7f0fb2096640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0fac1009b0 0x7f0fac198300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:38564/0 (socket says 172.21.15.26:38564) 2024-09-11T23:25:17.144 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.141+0000 7f0fb2096640 1 -- 172.21.15.26:0/1231602857 learned_addr learned my addr 172.21.15.26:0/1231602857 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:25:17.144 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.141+0000 7f0fb2897640 1 --2- 172.21.15.26:0/1231602857 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0fac1069c0 0x7f0fac19cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:17.144 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.141+0000 7f0fb1895640 1 -- 172.21.15.26:0/1231602857 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0fac1069c0 msgr2=0x7f0fac19cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:25:17.144 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.141+0000 7f0fb1895640 1 --2- 172.21.15.26:0/1231602857 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0fac1069c0 0x7f0fac19cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:17.144 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.141+0000 7f0fb1895640 1 -- 172.21.15.26:0/1231602857 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0fac1009b0 msgr2=0x7f0fac198300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:25:17.144 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.141+0000 7f0fb1895640 1 --2- 172.21.15.26:0/1231602857 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0fac1009b0 0x7f0fac198300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:17.144 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.141+0000 7f0fb1895640 1 -- 172.21.15.26:0/1231602857 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fac19d2f0 con 0x7f0fac101350 2024-09-11T23:25:17.144 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.141+0000 7f0fb2096640 1 --2- 172.21.15.26:0/1231602857 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0fac1009b0 0x7f0fac198300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:25:17.144 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.142+0000 7f0fb1895640 1 --2- 172.21.15.26:0/1231602857 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0fac101350 0x7f0fac198840 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f0f9c00e9f0 tx=0x7f0f9c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:25:17.144 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.142+0000 7f0fa37fe640 1 -- 172.21.15.26:0/1231602857 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f9c00ccc0 con 0x7f0fac101350 2024-09-11T23:25:17.144 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.142+0000 7f0fa37fe640 1 -- 172.21.15.26:0/1231602857 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f9c010910 con 0x7f0fac101350 2024-09-11T23:25:17.144 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.142+0000 7f0fb42fa640 1 -- 172.21.15.26:0/1231602857 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0fac19d5e0 con 0x7f0fac101350 2024-09-11T23:25:17.145 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.142+0000 7f0fa37fe640 1 -- 172.21.15.26:0/1231602857 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f9c018960 con 0x7f0fac101350 2024-09-11T23:25:17.145 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.142+0000 7f0fb42fa640 1 -- 172.21.15.26:0/1231602857 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0fac1a4e70 con 0x7f0fac101350 2024-09-11T23:25:17.149 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.144+0000 7f0fa37fe640 1 -- 172.21.15.26:0/1231602857 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f0f9c010ac0 con 0x7f0fac101350 2024-09-11T23:25:17.150 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.145+0000 7f0fb7094740 1 Processor -- start 2024-09-11T23:25:17.150 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.145+0000 7f0fb7094740 1 -- start start 2024-09-11T23:25:17.150 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.145+0000 7f0fb7094740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5578087819f0 0x557808781dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:17.150 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.145+0000 7f0fb7094740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557808782310 0x557808790b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:17.150 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.145+0000 7f0fb7094740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5578087910c0 0x5578087934b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:17.150 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.145+0000 7f0fb7094740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5578086e62f0 con 0x5578087819f0 2024-09-11T23:25:17.150 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.145+0000 7f0fb7094740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55780868e270 con 0x5578087910c0 2024-09-11T23:25:17.150 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.145+0000 7f0fb7094740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5578086e22b0 con 0x557808782310 2024-09-11T23:25:17.150 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.145+0000 7f0fb1895640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5578087910c0 0x5578087934b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:17.150 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.145+0000 7f0fb1895640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5578087910c0 0x5578087934b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:38570/0 (socket says 172.21.15.26:38570) 2024-09-11T23:25:17.150 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.145+0000 7f0fb1895640 1 -- 172.21.15.26:0/2173738536 learned_addr learned my addr 172.21.15.26:0/2173738536 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:25:17.150 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.145+0000 7f0fb2897640 1 --2- 172.21.15.26:0/2173738536 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5578087819f0 0x557808781dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:17.151 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.145+0000 7f0fa37fe640 1 --2- 172.21.15.26:0/1231602857 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f0f980a2e20 0x7f0f980a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:17.151 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.146+0000 7f0fb2096640 1 --2- 172.21.15.26:0/2173738536 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557808782310 0x557808790b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:17.151 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.146+0000 7f0fb2897640 1 --2- 172.21.15.26:0/1231602857 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f0f980a2e20 0x7f0f980a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:17.151 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.146+0000 7f0fb1895640 1 -- 172.21.15.26:0/2173738536 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557808782310 msgr2=0x557808790b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:25:17.151 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.146+0000 7f0fb1895640 1 --2- 172.21.15.26:0/2173738536 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557808782310 0x557808790b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:17.151 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.146+0000 7f0fb1895640 1 -- 172.21.15.26:0/2173738536 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5578087819f0 msgr2=0x557808781dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:25:17.151 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.146+0000 7f0fb1895640 1 --2- 172.21.15.26:0/2173738536 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5578087819f0 0x557808781dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:17.151 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.146+0000 7f0fb1895640 1 -- 172.21.15.26:0/2173738536 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55780868d340 con 0x5578087910c0 2024-09-11T23:25:17.151 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.146+0000 7f0fb2096640 1 --2- 172.21.15.26:0/2173738536 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557808782310 0x557808790b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:25:17.151 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.146+0000 7f0fa37fe640 1 -- 172.21.15.26:0/1231602857 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8586+0+0 (secure 0 0 0) 0x7f0f9c0cf7a0 con 0x7f0fac101350 2024-09-11T23:25:17.151 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.146+0000 7f0fb1895640 1 --2- 172.21.15.26:0/2173738536 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5578087910c0 0x5578087934b0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f0f9c09ba30 tx=0x7f0f9c09b7c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fb75ef9e09bb40a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:25:17.151 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.146+0000 7f0fb2897640 1 --2- 172.21.15.26:0/1231602857 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f0f980a2e20 0x7f0f980a52e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f0fa4009fd0 tx=0x7f0fa4004ab0 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:25:17.151 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.146+0000 7f0fa0bfe640 1 -- 172.21.15.26:0/2173738536 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f9c0a0030 con 0x5578087910c0 2024-09-11T23:25:17.152 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.146+0000 7f0fa0bfe640 1 -- 172.21.15.26:0/2173738536 <== mon.2 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f9c0a48f0 con 0x5578087910c0 2024-09-11T23:25:17.152 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.146+0000 7f0fa0bfe640 1 -- 172.21.15.26:0/2173738536 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f9c09b180 con 0x5578087910c0 2024-09-11T23:25:17.152 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.147+0000 7f0fb7094740 1 -- 172.21.15.26:0/2173738536 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5578087910c0 msgr2=0x5578087934b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:25:17.152 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.147+0000 7f0fb7094740 1 --2- 172.21.15.26:0/2173738536 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5578087910c0 0x5578087934b0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f0f9c09ba30 tx=0x7f0f9c09b7c0 comp rx=0 tx=0).stop 2024-09-11T23:25:17.152 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.147+0000 7f0fb7094740 1 -- 172.21.15.26:0/2173738536 shutdown_connections 2024-09-11T23:25:17.152 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.147+0000 7f0fb7094740 1 --2- 172.21.15.26:0/2173738536 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5578087910c0 0x5578087934b0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:17.152 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.147+0000 7f0fb7094740 1 --2- 172.21.15.26:0/2173738536 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557808782310 0x557808790b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:17.152 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.147+0000 7f0fb7094740 1 --2- 172.21.15.26:0/2173738536 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5578087819f0 0x557808781dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:17.152 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.147+0000 7f0fb7094740 1 -- 172.21.15.26:0/2173738536 >> 172.21.15.26:0/2173738536 conn(0x55780877e060 msgr2=0x55780877e440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:25:17.152 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.147+0000 7f0fb7094740 1 -- 172.21.15.26:0/2173738536 shutdown_connections 2024-09-11T23:25:17.152 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.147+0000 7f0fb7094740 1 -- 172.21.15.26:0/2173738536 wait complete. 2024-09-11T23:25:17.152 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.148+0000 7f0fb7094740 1 -- start start 2024-09-11T23:25:17.152 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.148+0000 7f0fb7094740 1 Processor -- start 2024-09-11T23:25:17.152 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.148+0000 7f0fb7094740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5578087819f0 0x55780878e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:17.153 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.148+0000 7f0fb7094740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557808782310 0x55780878e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:17.153 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.148+0000 7f0fb7094740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5578087910c0 0x55780878eac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:17.153 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.148+0000 7f0fb7094740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x557808793c60 con 0x5578087819f0 2024-09-11T23:25:17.153 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.148+0000 7f0fb7094740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55780866f290 con 0x557808782310 2024-09-11T23:25:17.153 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.148+0000 7f0fb7094740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x557808793de0 con 0x5578087910c0 2024-09-11T23:25:17.153 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.148+0000 7f0fb7094740 10 client.0 _ll_register_callbacks cb 0x5578086a0140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:25:17.153 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.148+0000 7f0fb2096640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5578087819f0 0x55780878e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:17.153 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.148+0000 7f0fb1895640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557808782310 0x55780878e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:17.153 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.148+0000 7f0fb2096640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5578087819f0 0x55780878e040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46968/0 (socket says 172.21.15.26:46968) 2024-09-11T23:25:17.154 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.148+0000 7f0fb2096640 1 -- 172.21.15.26:0/2043808487 learned_addr learned my addr 172.21.15.26:0/2043808487 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:25:17.154 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.148+0000 7f0fb1895640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557808782310 0x55780878e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:38578/0 (socket says 172.21.15.26:38578) 2024-09-11T23:25:17.154 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.148+0000 7f0fb2897640 1 --2- 172.21.15.26:0/2043808487 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5578087910c0 0x55780878eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:17.154 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.149+0000 7f0fb1895640 1 -- 172.21.15.26:0/2043808487 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5578087910c0 msgr2=0x55780878eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:25:17.154 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.149+0000 7f0fb1895640 1 --2- 172.21.15.26:0/2043808487 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5578087910c0 0x55780878eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:17.154 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.149+0000 7f0fb1895640 1 -- 172.21.15.26:0/2043808487 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5578087819f0 msgr2=0x55780878e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:25:17.154 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.149+0000 7f0fb1895640 1 --2- 172.21.15.26:0/2043808487 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5578087819f0 0x55780878e040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:17.154 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.149+0000 7f0fb1895640 1 -- 172.21.15.26:0/2043808487 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578083f9210 con 0x557808782310 2024-09-11T23:25:17.154 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.149+0000 7f0fb1895640 1 --2- 172.21.15.26:0/2043808487 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557808782310 0x55780878e580 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x557808782960 tx=0x7f0f9c010660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=517378eda243d74c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:25:17.154 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.149+0000 7f0f95ffb640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-11T23:25:17.154 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.149+0000 7f0f95ffb640 1 -- 172.21.15.26:0/2043808487 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f9c0a0030 con 0x557808782310 2024-09-11T23:25:17.154 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.149+0000 7f0f95ffb640 1 -- 172.21.15.26:0/2043808487 <== mon.2 v2:172.21.15.76:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8586+0+0 (secure 0 0 0) 0x7f0f9c04b2f0 con 0x557808782310 2024-09-11T23:25:17.154 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.149+0000 7f0fb7094740 1 -- 172.21.15.26:0/2043808487 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0f9c0a48f0 con 0x557808782310 2024-09-11T23:25:17.155 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.149+0000 7f0f95ffb640 1 -- 172.21.15.26:0/2043808487 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f0f9c09b180 con 0x557808782310 2024-09-11T23:25:17.155 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.149+0000 7f0fb7094740 10 client.26150 fetch_fsmap learned FSMap version 41 2024-09-11T23:25:17.155 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.149+0000 7f0fb7094740 1 -- 172.21.15.26:0/2043808487 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5578084c5620 con 0x557808782310 2024-09-11T23:25:17.155 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.150+0000 7f0f95ffb640 1 -- 172.21.15.26:0/2043808487 <== mon.2 v2:172.21.15.76:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f9c0a48f0 con 0x557808782310 2024-09-11T23:25:17.155 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.150+0000 7f0f95ffb640 1 -- 172.21.15.26:0/2043808487 <== mon.2 v2:172.21.15.76:3300/0 5 ==== fsmap(e 41) ==== 8426+0+0 (secure 0 0 0) 0x7f0f9c09abf0 con 0x557808782310 2024-09-11T23:25:17.155 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.150+0000 7f0fb7094740 10 client.26150 fetch_fsmap finished waiting for FSMap version 41 2024-09-11T23:25:17.155 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.150+0000 7f0fb7094740 10 client.26150 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:25:17.155 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.150+0000 7f0fb7094740 20 client.26150 populate_metadata read hostname 'smithi026' 2024-09-11T23:25:17.155 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.150+0000 7f0fb7094740 1 --2- 172.21.15.26:0/2043808487 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5578087a0f30 0x5578087a3320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:17.155 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.150+0000 7f0fb7094740 4 client.26150 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:25:17.155 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.150+0000 7f0fb7094740 1 -- 172.21.15.26:0/2043808487 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5578086f3a20 con 0x5578087a0f30 2024-09-11T23:25:17.155 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.150+0000 7f0fb2096640 1 --2- 172.21.15.26:0/2043808487 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5578087a0f30 0x5578087a3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:17.156 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.150+0000 7f0fb2096640 1 --2- 172.21.15.26:0/2043808487 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5578087a0f30 0x5578087a3320 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=976a0acf8831b556 server_cookie=2d0328d5a0d084b in_seq=0 out_seq=0 2024-09-11T23:25:17.157 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.150+0000 7f0f95ffb640 10 client.26150 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:25:17.157 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.151+0000 7f0f95ffb640 1 -- 172.21.15.26:0/2043808487 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5578086f3a20 con 0x5578087a0f30 2024-09-11T23:25:17.157 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.151+0000 7f0f95ffb640 10 client.26150 handle_command_reply: tid=0 2024-09-11T23:25:17.157 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.151+0000 7f0fb7094740 1 client.26150 shutdown 2024-09-11T23:25:17.157 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.151+0000 7f0fb7094740 1 -- 172.21.15.26:0/2043808487 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557808782310 msgr2=0x55780878e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:25:17.157 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.151+0000 7f0fb7094740 1 --2- 172.21.15.26:0/2043808487 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557808782310 0x55780878e580 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x557808782960 tx=0x7f0f9c010660 comp rx=0 tx=0).stop 2024-09-11T23:25:17.157 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.151+0000 7f0fb7094740 1 -- 172.21.15.26:0/2043808487 shutdown_connections 2024-09-11T23:25:17.157 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.151+0000 7f0fb7094740 1 --2- 172.21.15.26:0/2043808487 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557808782310 0x55780878e580 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:17.157 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.151+0000 7f0fb7094740 1 --2- 172.21.15.26:0/2043808487 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5578087a0f30 0x5578087a3320 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:17.157 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.151+0000 7f0fb7094740 1 --2- 172.21.15.26:0/2043808487 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5578087910c0 0x55780878eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:17.157 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.151+0000 7f0fb7094740 1 --2- 172.21.15.26:0/2043808487 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5578087819f0 0x55780878e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:17.157 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.151+0000 7f0fb7094740 1 -- 172.21.15.26:0/2043808487 >> 172.21.15.26:0/2043808487 conn(0x55780877e060 msgr2=0x557808783dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:25:17.157 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.151+0000 7f0f95ffb640 0 client.26150 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:25:17.158 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.151+0000 7f0fb7094740 1 -- 172.21.15.26:0/2043808487 shutdown_connections 2024-09-11T23:25:17.158 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.151+0000 7f0fb7094740 1 -- 172.21.15.26:0/2043808487 wait complete. 2024-09-11T23:25:17.158 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.151+0000 7f0fb7094740 20 client.26150 trim_cache size 0 max 16384 2024-09-11T23:25:17.169 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.168+0000 7f0fb7094740 1 Processor -- start 2024-09-11T23:25:17.169 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.168+0000 7f0fb7094740 1 -- start start 2024-09-11T23:25:17.169 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.168+0000 7f0fb7094740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5578087819f0 0x5578087a88f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:17.169 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.168+0000 7f0fb7094740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557808782310 0x5578087a8e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:17.169 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.168+0000 7f0fb7094740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5578087910c0 0x55780878b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:17.169 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.168+0000 7f0fb7094740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f0fac1ab680 con 0x5578087819f0 2024-09-11T23:25:17.169 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.168+0000 7f0fb7094740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f0fac1ad4c0 con 0x557808782310 2024-09-11T23:25:17.169 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.168+0000 7f0fb7094740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f0fac1abe10 con 0x5578087910c0 2024-09-11T23:25:17.169 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.168+0000 7f0fb2096640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5578087819f0 0x5578087a88f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:17.170 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.168+0000 7f0fb2897640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5578087910c0 0x55780878b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:17.170 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.168+0000 7f0fb2897640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5578087910c0 0x55780878b280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:55174/0 (socket says 172.21.15.26:55174) 2024-09-11T23:25:17.170 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.168+0000 7f0fb2096640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5578087819f0 0x5578087a88f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46982/0 (socket says 172.21.15.26:46982) 2024-09-11T23:25:17.170 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.168+0000 7f0fb2897640 1 -- 172.21.15.26:0/2223378143 learned_addr learned my addr 172.21.15.26:0/2223378143 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:25:17.170 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.168+0000 7f0fb1895640 1 --2- 172.21.15.26:0/2223378143 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557808782310 0x5578087a8e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:17.170 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.169+0000 7f0fb2096640 1 -- 172.21.15.26:0/2223378143 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5578087910c0 msgr2=0x55780878b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:25:17.170 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.169+0000 7f0fb2096640 1 --2- 172.21.15.26:0/2223378143 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5578087910c0 0x55780878b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:17.170 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.169+0000 7f0fb2096640 1 -- 172.21.15.26:0/2223378143 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557808782310 msgr2=0x5578087a8e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:25:17.170 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.169+0000 7f0fb2096640 1 --2- 172.21.15.26:0/2223378143 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557808782310 0x5578087a8e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:17.170 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.169+0000 7f0fb2096640 1 -- 172.21.15.26:0/2223378143 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557808782140 con 0x5578087819f0 2024-09-11T23:25:17.170 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.169+0000 7f0fb2897640 1 --2- 172.21.15.26:0/2223378143 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5578087910c0 0x55780878b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:25:17.170 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.169+0000 7f0fb2096640 1 --2- 172.21.15.26:0/2223378143 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5578087819f0 0x5578087a88f0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x5578087a3890 tx=0x7f0fa80329d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e2970f74ccbab69 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:25:17.170 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.169+0000 7f0f97fff640 1 -- 172.21.15.26:0/2223378143 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fa8035040 con 0x5578087819f0 2024-09-11T23:25:17.170 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.169+0000 7f0f97fff640 1 -- 172.21.15.26:0/2223378143 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0fa8004770 con 0x5578087819f0 2024-09-11T23:25:17.171 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.169+0000 7f0f97fff640 1 -- 172.21.15.26:0/2223378143 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fa80486b0 con 0x5578087819f0 2024-09-11T23:25:17.173 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.170+0000 7f0fb7094740 1 -- 172.21.15.26:0/2223378143 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5578087819f0 msgr2=0x5578087a88f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:25:17.174 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.170+0000 7f0fb7094740 1 --2- 172.21.15.26:0/2223378143 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5578087819f0 0x5578087a88f0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x5578087a3890 tx=0x7f0fa80329d0 comp rx=0 tx=0).stop 2024-09-11T23:25:17.174 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.170+0000 7f0fb7094740 1 -- 172.21.15.26:0/2223378143 shutdown_connections 2024-09-11T23:25:17.174 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.170+0000 7f0fb7094740 1 --2- 172.21.15.26:0/2223378143 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557808782310 0x5578087a8e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:17.174 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.170+0000 7f0fb7094740 1 --2- 172.21.15.26:0/2223378143 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5578087910c0 0x55780878b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:17.174 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.170+0000 7f0fb7094740 1 --2- 172.21.15.26:0/2223378143 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5578087819f0 0x5578087a88f0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:17.174 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.170+0000 7f0fb7094740 1 -- 172.21.15.26:0/2223378143 >> 172.21.15.26:0/2223378143 conn(0x55780877e060 msgr2=0x557808783dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:25:17.174 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.170+0000 7f0fb7094740 1 -- 172.21.15.26:0/2223378143 shutdown_connections 2024-09-11T23:25:17.174 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.170+0000 7f0fb7094740 1 -- 172.21.15.26:0/2223378143 wait complete. 2024-09-11T23:25:17.174 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.170+0000 7f0fb7094740 1 -- start start 2024-09-11T23:25:17.174 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.170+0000 7f0fb7094740 1 Processor -- start 2024-09-11T23:25:17.174 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.171+0000 7f0fb7094740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5578087819f0 0x5578087953b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:17.174 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.171+0000 7f0fb7094740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557808782310 0x5578087958f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:17.175 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.171+0000 7f0fb7094740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5578087910c0 0x557808799130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:17.175 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.171+0000 7f0fb7094740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f0fac1acd30 con 0x557808782310 2024-09-11T23:25:17.175 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.171+0000 7f0fb7094740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f0fac1ac5a0 con 0x5578087910c0 2024-09-11T23:25:17.175 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.171+0000 7f0fb7094740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f0fac1a96a0 con 0x5578087819f0 2024-09-11T23:25:17.175 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.171+0000 7f0fb7094740 10 client.0 _ll_register_callbacks cb 0x7f0fa4009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:25:17.175 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.171+0000 7f0fb1895640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557808782310 0x5578087958f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:17.175 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.171+0000 7f0fb2096640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5578087819f0 0x5578087953b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:17.175 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.171+0000 7f0fb1895640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557808782310 0x5578087958f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46994/0 (socket says 172.21.15.26:46994) 2024-09-11T23:25:17.175 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.171+0000 7f0fb1895640 1 -- 172.21.15.26:0/1699198839 learned_addr learned my addr 172.21.15.26:0/1699198839 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:25:17.175 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.171+0000 7f0fb2897640 1 --2- 172.21.15.26:0/1699198839 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5578087910c0 0x557808799130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:17.175 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.171+0000 7f0fb1895640 1 -- 172.21.15.26:0/1699198839 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5578087819f0 msgr2=0x5578087953b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:25:17.175 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.171+0000 7f0fb1895640 1 --2- 172.21.15.26:0/1699198839 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5578087819f0 0x5578087953b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:17.175 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.171+0000 7f0fb1895640 1 -- 172.21.15.26:0/1699198839 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5578087910c0 msgr2=0x557808799130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:25:17.175 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.171+0000 7f0fb1895640 1 --2- 172.21.15.26:0/1699198839 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5578087910c0 0x557808799130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:17.176 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.171+0000 7f0fb1895640 1 -- 172.21.15.26:0/1699198839 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557808799940 con 0x557808782310 2024-09-11T23:25:17.176 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.171+0000 7f0fb1895640 1 --2- 172.21.15.26:0/1699198839 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557808782310 0x5578087958f0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f0f9c04b0b0 tx=0x7f0f9c09a240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7af976c13416689 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:25:17.176 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.171+0000 7f0f95ffb640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:25:17.176 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.172+0000 7f0f95ffb640 1 -- 172.21.15.26:0/1699198839 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f9c097fd0 con 0x557808782310 2024-09-11T23:25:17.176 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.172+0000 7f0f95ffb640 1 -- 172.21.15.26:0/1699198839 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8586+0+0 (secure 0 0 0) 0x7f0f9c018b00 con 0x557808782310 2024-09-11T23:25:17.176 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.172+0000 7f0fb7094740 1 -- 172.21.15.26:0/1699198839 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0fa8004770 con 0x557808782310 2024-09-11T23:25:17.176 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.172+0000 7f0f95ffb640 1 -- 172.21.15.26:0/1699198839 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f0f9c0a48f0 con 0x557808782310 2024-09-11T23:25:17.176 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.172+0000 7f0fb7094740 10 client.16551 fetch_fsmap learned FSMap version 41 2024-09-11T23:25:17.176 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.172+0000 7f0fb7094740 1 -- 172.21.15.26:0/1699198839 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x557808799940 con 0x557808782310 2024-09-11T23:25:17.176 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.172+0000 7f0f95ffb640 1 -- 172.21.15.26:0/1699198839 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0fa8004770 con 0x557808782310 2024-09-11T23:25:17.176 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.172+0000 7f0f95ffb640 1 -- 172.21.15.26:0/1699198839 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 41) ==== 8426+0+0 (secure 0 0 0) 0x7f0f9c0ba020 con 0x557808782310 2024-09-11T23:25:17.176 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.172+0000 7f0fb7094740 10 client.16551 fetch_fsmap finished waiting for FSMap version 41 2024-09-11T23:25:17.176 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.172+0000 7f0fb7094740 10 client.16551 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:25:17.177 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.172+0000 7f0fb7094740 20 client.16551 populate_metadata read hostname 'smithi026' 2024-09-11T23:25:17.177 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.172+0000 7f0fb7094740 1 --2- 172.21.15.26:0/1699198839 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5578088101c0 0x5578088125e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:17.177 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.172+0000 7f0fb7094740 4 client.16551 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T23:25:17.177 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.172+0000 7f0fb7094740 1 -- 172.21.15.26:0/1699198839 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5578086edd70 con 0x5578088101c0 2024-09-11T23:25:17.177 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.172+0000 7f0fb2096640 1 --2- 172.21.15.26:0/1699198839 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5578088101c0 0x5578088125e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:17.177 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.173+0000 7f0fb2096640 1 --2- 172.21.15.26:0/1699198839 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5578088101c0 0x5578088125e0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d398afb64c1097ca server_cookie=f9b89b48a6d71064 in_seq=0 out_seq=0 2024-09-11T23:25:17.177 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.173+0000 7f0f95ffb640 10 client.16551 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:25:17.504 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24485 quiesce db --set-id 27fb6b2d --release --await' 2024-09-11T23:25:17.504 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24485 quiesce db --set-id 27fb6b2d --release --await 2024-09-11T23:25:17.505 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.504+0000 7f0f95ffb640 1 -- 172.21.15.26:0/1699198839 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5578086edd70 con 0x5578088101c0 2024-09-11T23:25:17.506 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.504+0000 7f0f95ffb640 10 client.16551 handle_command_reply: tid=0 2024-09-11T23:25:17.506 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.504+0000 7f0fb7094740 1 client.16551 shutdown 2024-09-11T23:25:17.506 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.505+0000 7f0fb7094740 1 -- 172.21.15.26:0/1699198839 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557808782310 msgr2=0x5578087958f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:25:17.506 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.505+0000 7f0fb7094740 1 --2- 172.21.15.26:0/1699198839 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557808782310 0x5578087958f0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f0f9c04b0b0 tx=0x7f0f9c09a240 comp rx=0 tx=0).stop 2024-09-11T23:25:17.506 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.505+0000 7f0fb7094740 1 -- 172.21.15.26:0/1699198839 shutdown_connections 2024-09-11T23:25:17.506 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.505+0000 7f0fb7094740 1 --2- 172.21.15.26:0/1699198839 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5578087910c0 0x557808799130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:17.506 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.505+0000 7f0fb7094740 1 --2- 172.21.15.26:0/1699198839 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5578088101c0 0x5578088125e0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:17.506 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.505+0000 7f0fb7094740 1 --2- 172.21.15.26:0/1699198839 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557808782310 0x5578087958f0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:17.506 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.505+0000 7f0f95ffb640 0 client.16551 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:25:17.506 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.505+0000 7f0fb7094740 1 --2- 172.21.15.26:0/1699198839 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5578087819f0 0x5578087953b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:17.506 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.505+0000 7f0fb7094740 1 -- 172.21.15.26:0/1699198839 >> 172.21.15.26:0/1699198839 conn(0x55780877e060 msgr2=0x557808783530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:25:17.506 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.505+0000 7f0fb7094740 1 -- 172.21.15.26:0/1699198839 shutdown_connections 2024-09-11T23:25:17.506 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.505+0000 7f0fb7094740 1 -- 172.21.15.26:0/1699198839 wait complete. 2024-09-11T23:25:17.507 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.505+0000 7f0fb7094740 20 client.16551 trim_cache size 0 max 16384 2024-09-11T23:25:17.507 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-09-11T23:25:17.507 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "message": "", 2024-09-11T23:25:17.507 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-09-11T23:25:17.507 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-09-11T23:25:17.508 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.506+0000 7f0fb42fa640 1 -- 172.21.15.26:0/1231602857 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f0f980a2e20 msgr2=0x7f0f980a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:25:17.508 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.506+0000 7f0fb42fa640 1 --2- 172.21.15.26:0/1231602857 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f0f980a2e20 0x7f0f980a52e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f0fa4009fd0 tx=0x7f0fa4004ab0 comp rx=0 tx=0).stop 2024-09-11T23:25:17.508 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.506+0000 7f0fb42fa640 1 -- 172.21.15.26:0/1231602857 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0fac101350 msgr2=0x7f0fac198840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:25:17.508 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.506+0000 7f0fb42fa640 1 --2- 172.21.15.26:0/1231602857 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0fac101350 0x7f0fac198840 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f0f9c00e9f0 tx=0x7f0f9c00eeb0 comp rx=0 tx=0).stop 2024-09-11T23:25:17.508 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.506+0000 7f0fb42fa640 1 -- 172.21.15.26:0/1231602857 shutdown_connections 2024-09-11T23:25:17.508 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.506+0000 7f0fb42fa640 1 --2- 172.21.15.26:0/1231602857 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f0f980a2e20 0x7f0f980a52e0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:17.509 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.506+0000 7f0fb42fa640 1 --2- 172.21.15.26:0/1231602857 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0fac1069c0 0x7f0fac19cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:17.509 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.506+0000 7f0fb42fa640 1 --2- 172.21.15.26:0/1231602857 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0fac101350 0x7f0fac198840 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:17.509 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.506+0000 7f0fb42fa640 1 --2- 172.21.15.26:0/1231602857 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0fac1009b0 0x7f0fac198300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:17.509 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.506+0000 7f0fb42fa640 1 -- 172.21.15.26:0/1231602857 >> 172.21.15.26:0/1231602857 conn(0x7f0fac077990 msgr2=0x7f0fac069a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:25:17.509 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.506+0000 7f0fb42fa640 1 -- 172.21.15.26:0/1231602857 shutdown_connections 2024-09-11T23:25:17.509 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:17.506+0000 7f0fb42fa640 1 -- 172.21.15.26:0/1231602857 wait complete. 2024-09-11T23:25:17.787 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:25:17.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:25:17 smithi136 ceph-mon[35121]: pgmap v211: 97 pgs: 97 active+clean; 2.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 13 MiB/s wr, 711 op/s 2024-09-11T23:25:17.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:25:17 smithi136 ceph-mon[35121]: from='client.? 172.21.15.26:0/893183842' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:25:18.000 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:25:17 smithi026 ceph-mon[31172]: pgmap v211: 97 pgs: 97 active+clean; 2.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 13 MiB/s wr, 711 op/s 2024-09-11T23:25:18.000 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:25:17 smithi026 ceph-mon[31172]: from='client.? 172.21.15.26:0/893183842' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:25:18.052 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:25:17 smithi076 ceph-mon[35296]: pgmap v211: 97 pgs: 97 active+clean; 2.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 13 MiB/s wr, 711 op/s 2024-09-11T23:25:18.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:25:17 smithi076 ceph-mon[35296]: from='client.? 172.21.15.26:0/893183842' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:25:18.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.307+0000 7f2b73309640 1 -- 172.21.15.26:0/3406805205 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b6c102ef0 msgr2=0x7f2b6c10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:25:18.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.307+0000 7f2b73309640 1 --2- 172.21.15.26:0/3406805205 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b6c102ef0 0x7f2b6c10da40 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f2b64009ca0 tx=0x7f2b6402a950 comp rx=0 tx=0).stop 2024-09-11T23:25:18.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.307+0000 7f2b73309640 1 -- 172.21.15.26:0/3406805205 shutdown_connections 2024-09-11T23:25:18.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.307+0000 7f2b73309640 1 --2- 172.21.15.26:0/3406805205 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b6c102ef0 0x7f2b6c10da40 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:18.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.307+0000 7f2b73309640 1 --2- 172.21.15.26:0/3406805205 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2b6c102570 0x7f2b6c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:18.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.307+0000 7f2b73309640 1 --2- 172.21.15.26:0/3406805205 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2b6c100e20 0x7f2b6c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:18.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.307+0000 7f2b73309640 1 -- 172.21.15.26:0/3406805205 >> 172.21.15.26:0/3406805205 conn(0x7f2b6c0fc7d0 msgr2=0x7f2b6c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:25:18.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.307+0000 7f2b73309640 1 -- 172.21.15.26:0/3406805205 shutdown_connections 2024-09-11T23:25:18.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.307+0000 7f2b73309640 1 -- 172.21.15.26:0/3406805205 wait complete. 2024-09-11T23:25:18.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.308+0000 7f2b73309640 1 Processor -- start 2024-09-11T23:25:18.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.308+0000 7f2b73309640 1 -- start start 2024-09-11T23:25:18.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.308+0000 7f2b73309640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2b6c100e20 0x7f2b6c1970e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:18.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.308+0000 7f2b73309640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b6c102570 0x7f2b6c197620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:18.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.308+0000 7f2b73309640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2b6c102ef0 0x7f2b6c197b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:18.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.308+0000 7f2b73309640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f2b6c110f80 con 0x7f2b6c102570 2024-09-11T23:25:18.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.308+0000 7f2b73309640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f2b6c110e00 con 0x7f2b6c100e20 2024-09-11T23:25:18.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.308+0000 7f2b73309640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f2b6c111100 con 0x7f2b6c102ef0 2024-09-11T23:25:18.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.308+0000 7f2b708a4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b6c102570 0x7f2b6c197620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:18.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.308+0000 7f2b710a5640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2b6c100e20 0x7f2b6c1970e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:18.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.308+0000 7f2b710a5640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2b6c100e20 0x7f2b6c1970e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:38604/0 (socket says 172.21.15.26:38604) 2024-09-11T23:25:18.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.308+0000 7f2b710a5640 1 -- 172.21.15.26:0/672001877 learned_addr learned my addr 172.21.15.26:0/672001877 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:25:18.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.308+0000 7f2b708a4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b6c102570 0x7f2b6c197620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47016/0 (socket says 172.21.15.26:47016) 2024-09-11T23:25:18.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.308+0000 7f2b718a6640 1 --2- 172.21.15.26:0/672001877 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2b6c102ef0 0x7f2b6c197b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:18.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.309+0000 7f2b710a5640 1 -- 172.21.15.26:0/672001877 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2b6c102ef0 msgr2=0x7f2b6c197b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:25:18.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.309+0000 7f2b710a5640 1 --2- 172.21.15.26:0/672001877 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2b6c102ef0 0x7f2b6c197b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:18.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.309+0000 7f2b710a5640 1 -- 172.21.15.26:0/672001877 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b6c102570 msgr2=0x7f2b6c197620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:25:18.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.309+0000 7f2b710a5640 1 --2- 172.21.15.26:0/672001877 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b6c102570 0x7f2b6c197620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:18.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.309+0000 7f2b710a5640 1 -- 172.21.15.26:0/672001877 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b6c1a6950 con 0x7f2b6c100e20 2024-09-11T23:25:18.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.309+0000 7f2b708a4640 1 --2- 172.21.15.26:0/672001877 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b6c102570 0x7f2b6c197620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:25:18.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.309+0000 7f2b710a5640 1 --2- 172.21.15.26:0/672001877 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2b6c100e20 0x7f2b6c1970e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f2b6800e9f0 tx=0x7f2b6800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:25:18.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.309+0000 7f2b627fc640 1 -- 172.21.15.26:0/672001877 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b6800cd80 con 0x7f2b6c100e20 2024-09-11T23:25:18.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.309+0000 7f2b627fc640 1 -- 172.21.15.26:0/672001877 <== mon.2 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b68010910 con 0x7f2b6c100e20 2024-09-11T23:25:18.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.309+0000 7f2b73309640 1 -- 172.21.15.26:0/672001877 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b6c1a6c40 con 0x7f2b6c100e20 2024-09-11T23:25:18.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.309+0000 7f2b627fc640 1 -- 172.21.15.26:0/672001877 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b68018990 con 0x7f2b6c100e20 2024-09-11T23:25:18.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.309+0000 7f2b73309640 1 -- 172.21.15.26:0/672001877 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b6c1a7180 con 0x7f2b6c100e20 2024-09-11T23:25:18.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.312+0000 7f2b756e2740 1 Processor -- start 2024-09-11T23:25:18.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.312+0000 7f2b756e2740 1 -- start start 2024-09-11T23:25:18.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.312+0000 7f2b756e2740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560e857c8060 0x560e857c8440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:18.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.313+0000 7f2b756e2740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560e857c8980 0x560e857d2d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:18.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.313+0000 7f2b756e2740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560e857d3240 0x560e857d5630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:18.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.313+0000 7f2b756e2740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x560e8572d620 con 0x560e857c8060 2024-09-11T23:25:18.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.313+0000 7f2b756e2740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x560e856d4e50 con 0x560e857d3240 2024-09-11T23:25:18.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.313+0000 7f2b756e2740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x560e857295e0 con 0x560e857c8980 2024-09-11T23:25:18.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.313+0000 7f2b718a6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560e857c8060 0x560e857c8440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:18.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.313+0000 7f2b708a4640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560e857d3240 0x560e857d5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:18.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.313+0000 7f2b718a6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560e857c8060 0x560e857c8440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47024/0 (socket says 172.21.15.26:47024) 2024-09-11T23:25:18.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.313+0000 7f2b718a6640 1 -- 172.21.15.26:0/2968938996 learned_addr learned my addr 172.21.15.26:0/2968938996 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:25:18.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.313+0000 7f2b627fc640 1 -- 172.21.15.26:0/672001877 <== mon.2 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f2b68010ac0 con 0x7f2b6c100e20 2024-09-11T23:25:18.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.313+0000 7f2b710a5640 1 --2- 172.21.15.26:0/2968938996 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560e857c8980 0x560e857d2d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:18.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.313+0000 7f2b708a4640 1 -- 172.21.15.26:0/2968938996 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560e857c8980 msgr2=0x560e857d2d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:25:18.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.313+0000 7f2b708a4640 1 --2- 172.21.15.26:0/2968938996 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560e857c8980 0x560e857d2d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:18.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.313+0000 7f2b708a4640 1 -- 172.21.15.26:0/2968938996 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560e857c8060 msgr2=0x560e857c8440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:25:18.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.313+0000 7f2b708a4640 1 --2- 172.21.15.26:0/2968938996 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560e857c8060 0x560e857c8440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:18.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.313+0000 7f2b708a4640 1 -- 172.21.15.26:0/2968938996 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e856c5d40 con 0x560e857d3240 2024-09-11T23:25:18.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.313+0000 7f2b710a5640 1 --2- 172.21.15.26:0/2968938996 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560e857c8980 0x560e857d2d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:25:18.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.313+0000 7f2b718a6640 1 --2- 172.21.15.26:0/2968938996 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560e857c8060 0x560e857c8440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:25:18.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.313+0000 7f2b708a4640 1 --2- 172.21.15.26:0/2968938996 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560e857d3240 0x560e857d5630 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f2b5c0058c0 tx=0x7f2b5c00f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bb8d7daa9e37a201 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:25:18.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.314+0000 7f2b577fe640 1 -- 172.21.15.26:0/2968938996 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b5c01d070 con 0x560e857d3240 2024-09-11T23:25:18.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.314+0000 7f2b577fe640 1 -- 172.21.15.26:0/2968938996 <== mon.2 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b5c010d20 con 0x560e857d3240 2024-09-11T23:25:18.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.314+0000 7f2b577fe640 1 -- 172.21.15.26:0/2968938996 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b5c016340 con 0x560e857d3240 2024-09-11T23:25:18.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.314+0000 7f2b627fc640 1 --2- 172.21.15.26:0/672001877 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f2b580a2cf0 0x7f2b580a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:18.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.314+0000 7f2b718a6640 1 --2- 172.21.15.26:0/672001877 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f2b580a2cf0 0x7f2b580a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:18.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.314+0000 7f2b627fc640 1 -- 172.21.15.26:0/672001877 <== mon.2 v2:172.21.15.76:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8586+0+0 (secure 0 0 0) 0x7f2b680cf7e0 con 0x7f2b6c100e20 2024-09-11T23:25:18.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.315+0000 7f2b756e2740 1 -- 172.21.15.26:0/2968938996 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560e857d3240 msgr2=0x560e857d5630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:25:18.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.315+0000 7f2b756e2740 1 --2- 172.21.15.26:0/2968938996 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560e857d3240 0x560e857d5630 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f2b5c0058c0 tx=0x7f2b5c00f790 comp rx=0 tx=0).stop 2024-09-11T23:25:18.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.315+0000 7f2b718a6640 1 --2- 172.21.15.26:0/672001877 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f2b580a2cf0 0x7f2b580a51b0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f2b64009ca0 tx=0x7f2b64005c50 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:25:18.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.315+0000 7f2b756e2740 1 -- 172.21.15.26:0/2968938996 shutdown_connections 2024-09-11T23:25:18.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.315+0000 7f2b756e2740 1 --2- 172.21.15.26:0/2968938996 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560e857d3240 0x560e857d5630 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:18.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.315+0000 7f2b756e2740 1 --2- 172.21.15.26:0/2968938996 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560e857c8980 0x560e857d2d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:18.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.315+0000 7f2b756e2740 1 --2- 172.21.15.26:0/2968938996 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560e857c8060 0x560e857c8440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:18.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.315+0000 7f2b756e2740 1 -- 172.21.15.26:0/2968938996 >> 172.21.15.26:0/2968938996 conn(0x560e857c45f0 msgr2=0x560e857c49d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:25:18.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.315+0000 7f2b756e2740 1 -- 172.21.15.26:0/2968938996 shutdown_connections 2024-09-11T23:25:18.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.315+0000 7f2b756e2740 1 -- 172.21.15.26:0/2968938996 wait complete. 2024-09-11T23:25:18.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.315+0000 7f2b756e2740 1 -- start start 2024-09-11T23:25:18.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.315+0000 7f2b756e2740 1 Processor -- start 2024-09-11T23:25:18.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.316+0000 7f2b756e2740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560e857c8060 0x560e857d0110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:18.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.316+0000 7f2b756e2740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560e857c8980 0x560e857d0650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:18.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.316+0000 7f2b756e2740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560e857d3240 0x560e857d0b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:18.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.316+0000 7f2b756e2740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x560e857d5e10 con 0x560e857c8980 2024-09-11T23:25:18.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.316+0000 7f2b756e2740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x560e856b5f10 con 0x560e857c8060 2024-09-11T23:25:18.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.316+0000 7f2b756e2740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x560e857d5f90 con 0x560e857d3240 2024-09-11T23:25:18.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.316+0000 7f2b756e2740 10 client.0 _ll_register_callbacks cb 0x560e856d9db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:25:18.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.316+0000 7f2b708a4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560e857c8980 0x560e857d0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:18.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.316+0000 7f2b708a4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560e857c8980 0x560e857d0650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47034/0 (socket says 172.21.15.26:47034) 2024-09-11T23:25:18.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.316+0000 7f2b710a5640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560e857c8060 0x560e857d0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:18.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.316+0000 7f2b708a4640 1 -- 172.21.15.26:0/118501276 learned_addr learned my addr 172.21.15.26:0/118501276 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:25:18.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.316+0000 7f2b718a6640 1 --2- 172.21.15.26:0/118501276 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560e857d3240 0x560e857d0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:18.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.316+0000 7f2b708a4640 1 -- 172.21.15.26:0/118501276 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560e857d3240 msgr2=0x560e857d0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:25:18.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.316+0000 7f2b708a4640 1 --2- 172.21.15.26:0/118501276 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560e857d3240 0x560e857d0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:18.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.316+0000 7f2b708a4640 1 -- 172.21.15.26:0/118501276 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560e857c8060 msgr2=0x560e857d0110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:25:18.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.316+0000 7f2b708a4640 1 --2- 172.21.15.26:0/118501276 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560e857c8060 0x560e857d0110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:18.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.316+0000 7f2b708a4640 1 -- 172.21.15.26:0/118501276 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e8543c580 con 0x560e857c8980 2024-09-11T23:25:18.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.316+0000 7f2b718a6640 1 --2- 172.21.15.26:0/118501276 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560e857d3240 0x560e857d0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:25:18.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.316+0000 7f2b710a5640 1 --2- 172.21.15.26:0/118501276 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560e857c8060 0x560e857d0110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:25:18.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.317+0000 7f2b708a4640 1 --2- 172.21.15.26:0/118501276 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560e857c8980 0x560e857d0650 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f2b5c01b040 tx=0x7f2b5c018650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebef680c3575fb9d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:25:18.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.317+0000 7f2b54ff9640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:25:18.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.317+0000 7f2b54ff9640 1 -- 172.21.15.26:0/118501276 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b5c01d050 con 0x560e857c8980 2024-09-11T23:25:18.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.317+0000 7f2b54ff9640 1 -- 172.21.15.26:0/118501276 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8586+0+0 (secure 0 0 0) 0x7f2b5c010a50 con 0x560e857c8980 2024-09-11T23:25:18.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.317+0000 7f2b756e2740 1 -- 172.21.15.26:0/118501276 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b5c010d20 con 0x560e857c8980 2024-09-11T23:25:18.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.317+0000 7f2b54ff9640 1 -- 172.21.15.26:0/118501276 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f2b5c032050 con 0x560e857c8980 2024-09-11T23:25:18.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.317+0000 7f2b756e2740 10 client.16563 fetch_fsmap learned FSMap version 41 2024-09-11T23:25:18.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.317+0000 7f2b756e2740 1 -- 172.21.15.26:0/118501276 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x560e85506960 con 0x560e857c8980 2024-09-11T23:25:18.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.317+0000 7f2b54ff9640 1 -- 172.21.15.26:0/118501276 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b5c010d20 con 0x560e857c8980 2024-09-11T23:25:18.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.317+0000 7f2b54ff9640 1 -- 172.21.15.26:0/118501276 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 41) ==== 8426+0+0 (secure 0 0 0) 0x7f2b5c004a70 con 0x560e857c8980 2024-09-11T23:25:18.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.318+0000 7f2b756e2740 10 client.16563 fetch_fsmap finished waiting for FSMap version 41 2024-09-11T23:25:18.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.318+0000 7f2b756e2740 10 client.16563 resolve_mds: validated gid 24485 aka daemon mds.b 2024-09-11T23:25:18.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.318+0000 7f2b756e2740 20 client.16563 populate_metadata read hostname 'smithi026' 2024-09-11T23:25:18.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.318+0000 7f2b756e2740 1 --2- 172.21.15.26:0/118501276 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x560e857e2bf0 0x560e857e4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:18.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.318+0000 7f2b756e2740 4 client.16563 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:25:18.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.318+0000 7f2b756e2740 1 -- 172.21.15.26:0/118501276 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560e8573ab90 con 0x560e857e2bf0 2024-09-11T23:25:18.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.318+0000 7f2b710a5640 1 --2- 172.21.15.26:0/118501276 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x560e857e2bf0 0x560e857e4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:18.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.318+0000 7f2b710a5640 1 --2- 172.21.15.26:0/118501276 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x560e857e2bf0 0x560e857e4fe0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2f39333debff460 server_cookie=27a2af2b34aa4f62 in_seq=0 out_seq=0 2024-09-11T23:25:18.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.318+0000 7f2b54ff9640 10 client.16563 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:25:18.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.319+0000 7f2b54ff9640 1 -- 172.21.15.26:0/118501276 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560e8573ab90 con 0x560e857e2bf0 2024-09-11T23:25:18.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.319+0000 7f2b54ff9640 10 client.16563 handle_command_reply: tid=0 2024-09-11T23:25:18.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.319+0000 7f2b756e2740 1 client.16563 shutdown 2024-09-11T23:25:18.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.319+0000 7f2b756e2740 1 -- 172.21.15.26:0/118501276 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560e857c8980 msgr2=0x560e857d0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:25:18.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.319+0000 7f2b756e2740 1 --2- 172.21.15.26:0/118501276 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560e857c8980 0x560e857d0650 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f2b5c01b040 tx=0x7f2b5c018650 comp rx=0 tx=0).stop 2024-09-11T23:25:18.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.319+0000 7f2b756e2740 1 -- 172.21.15.26:0/118501276 shutdown_connections 2024-09-11T23:25:18.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.319+0000 7f2b756e2740 1 --2- 172.21.15.26:0/118501276 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x560e857e2bf0 0x560e857e4fe0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:18.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.319+0000 7f2b756e2740 1 --2- 172.21.15.26:0/118501276 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560e857d3240 0x560e857d0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:18.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.319+0000 7f2b756e2740 1 --2- 172.21.15.26:0/118501276 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560e857c8980 0x560e857d0650 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:18.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.319+0000 7f2b756e2740 1 --2- 172.21.15.26:0/118501276 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560e857c8060 0x560e857d0110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:18.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.319+0000 7f2b54ff9640 0 client.16563 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:25:18.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.319+0000 7f2b756e2740 1 -- 172.21.15.26:0/118501276 >> 172.21.15.26:0/118501276 conn(0x560e857c45f0 msgr2=0x560e857ca320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:25:18.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.319+0000 7f2b756e2740 1 -- 172.21.15.26:0/118501276 shutdown_connections 2024-09-11T23:25:18.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.319+0000 7f2b756e2740 1 -- 172.21.15.26:0/118501276 wait complete. 2024-09-11T23:25:18.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.319+0000 7f2b756e2740 20 client.16563 trim_cache size 0 max 16384 2024-09-11T23:25:18.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.338+0000 7f2b756e2740 1 Processor -- start 2024-09-11T23:25:18.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.338+0000 7f2b756e2740 1 -- start start 2024-09-11T23:25:18.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.339+0000 7f2b756e2740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560e857c8060 0x560e857c49d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:18.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.339+0000 7f2b756e2740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560e857c8980 0x560e857ca1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:18.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.339+0000 7f2b756e2740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560e857d3240 0x560e857ea700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:18.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.339+0000 7f2b756e2740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f2b6c1afb90 con 0x560e857d3240 2024-09-11T23:25:18.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.339+0000 7f2b756e2740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f2b6c1b19d0 con 0x560e857c8980 2024-09-11T23:25:18.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.339+0000 7f2b756e2740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f2b6c1b0320 con 0x560e857c8060 2024-09-11T23:25:18.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.339+0000 7f2b718a6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560e857d3240 0x560e857ea700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:18.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.339+0000 7f2b718a6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560e857d3240 0x560e857ea700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47036/0 (socket says 172.21.15.26:47036) 2024-09-11T23:25:18.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.339+0000 7f2b718a6640 1 -- 172.21.15.26:0/7091596 learned_addr learned my addr 172.21.15.26:0/7091596 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:25:18.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.339+0000 7f2b708a4640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560e857c8980 0x560e857ca1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:18.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.339+0000 7f2b710a5640 1 --2- 172.21.15.26:0/7091596 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560e857c8060 0x560e857c49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:18.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.339+0000 7f2b710a5640 1 -- 172.21.15.26:0/7091596 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560e857c8980 msgr2=0x560e857ca1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:25:18.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.339+0000 7f2b710a5640 1 --2- 172.21.15.26:0/7091596 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560e857c8980 0x560e857ca1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:18.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.339+0000 7f2b710a5640 1 -- 172.21.15.26:0/7091596 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560e857d3240 msgr2=0x560e857ea700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:25:18.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.339+0000 7f2b710a5640 1 --2- 172.21.15.26:0/7091596 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560e857d3240 0x560e857ea700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:18.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.339+0000 7f2b710a5640 1 -- 172.21.15.26:0/7091596 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e857d2050 con 0x560e857c8060 2024-09-11T23:25:18.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.339+0000 7f2b718a6640 1 --2- 172.21.15.26:0/7091596 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560e857d3240 0x560e857ea700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:25:18.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.340+0000 7f2b710a5640 1 --2- 172.21.15.26:0/7091596 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560e857c8060 0x560e857c49d0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f2b680cce90 tx=0x7f2b68000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8636fa30ededba61 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:25:18.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.340+0000 7f2b56ffd640 1 -- 172.21.15.26:0/7091596 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b680cd110 con 0x560e857c8060 2024-09-11T23:25:18.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.340+0000 7f2b56ffd640 1 -- 172.21.15.26:0/7091596 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b680b1070 con 0x560e857c8060 2024-09-11T23:25:18.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.340+0000 7f2b56ffd640 1 -- 172.21.15.26:0/7091596 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b68018d90 con 0x560e857c8060 2024-09-11T23:25:18.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.341+0000 7f2b756e2740 1 -- 172.21.15.26:0/7091596 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560e857c8060 msgr2=0x560e857c49d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:25:18.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.341+0000 7f2b756e2740 1 --2- 172.21.15.26:0/7091596 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560e857c8060 0x560e857c49d0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f2b680cce90 tx=0x7f2b68000bc0 comp rx=0 tx=0).stop 2024-09-11T23:25:18.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.341+0000 7f2b756e2740 1 -- 172.21.15.26:0/7091596 shutdown_connections 2024-09-11T23:25:18.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.341+0000 7f2b756e2740 1 --2- 172.21.15.26:0/7091596 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560e857c8980 0x560e857ca1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:18.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.341+0000 7f2b756e2740 1 --2- 172.21.15.26:0/7091596 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560e857d3240 0x560e857ea700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:18.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.341+0000 7f2b756e2740 1 --2- 172.21.15.26:0/7091596 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560e857c8060 0x560e857c49d0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:18.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.341+0000 7f2b756e2740 1 -- 172.21.15.26:0/7091596 >> 172.21.15.26:0/7091596 conn(0x560e857c45f0 msgr2=0x560e857c9ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:25:18.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.341+0000 7f2b756e2740 1 -- 172.21.15.26:0/7091596 shutdown_connections 2024-09-11T23:25:18.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.341+0000 7f2b756e2740 1 -- 172.21.15.26:0/7091596 wait complete. 2024-09-11T23:25:18.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.341+0000 7f2b756e2740 1 -- start start 2024-09-11T23:25:18.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.341+0000 7f2b756e2740 1 Processor -- start 2024-09-11T23:25:18.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.342+0000 7f2b756e2740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560e857c8060 0x560e857cd240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:18.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.342+0000 7f2b756e2740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560e857c8980 0x560e857cd780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:18.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.342+0000 7f2b756e2740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560e857d3240 0x560e857cdcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:18.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.342+0000 7f2b756e2740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f2b6c1b1240 con 0x560e857c8980 2024-09-11T23:25:18.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.342+0000 7f2b756e2740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f2b6c1b0ab0 con 0x560e857d3240 2024-09-11T23:25:18.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.342+0000 7f2b756e2740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f2b6c1adb90 con 0x560e857c8060 2024-09-11T23:25:18.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.342+0000 7f2b756e2740 10 client.0 _ll_register_callbacks cb 0x7f2b5c00adb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:25:18.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.342+0000 7f2b708a4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560e857c8980 0x560e857cd780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:18.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.342+0000 7f2b708a4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560e857c8980 0x560e857cd780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47038/0 (socket says 172.21.15.26:47038) 2024-09-11T23:25:18.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.342+0000 7f2b708a4640 1 -- 172.21.15.26:0/1937954532 learned_addr learned my addr 172.21.15.26:0/1937954532 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:25:18.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.342+0000 7f2b710a5640 1 --2- 172.21.15.26:0/1937954532 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560e857c8060 0x560e857cd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:18.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.342+0000 7f2b718a6640 1 --2- 172.21.15.26:0/1937954532 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560e857d3240 0x560e857cdcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:18.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.342+0000 7f2b708a4640 1 -- 172.21.15.26:0/1937954532 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560e857c8060 msgr2=0x560e857cd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:25:18.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.342+0000 7f2b708a4640 1 --2- 172.21.15.26:0/1937954532 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560e857c8060 0x560e857cd240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:18.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.342+0000 7f2b708a4640 1 -- 172.21.15.26:0/1937954532 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560e857d3240 msgr2=0x560e857cdcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:25:18.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.342+0000 7f2b708a4640 1 --2- 172.21.15.26:0/1937954532 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560e857d3240 0x560e857cdcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:18.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.342+0000 7f2b708a4640 1 -- 172.21.15.26:0/1937954532 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e8584c7a0 con 0x560e857c8980 2024-09-11T23:25:18.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.342+0000 7f2b718a6640 1 --2- 172.21.15.26:0/1937954532 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560e857d3240 0x560e857cdcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:25:18.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.343+0000 7f2b708a4640 1 --2- 172.21.15.26:0/1937954532 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560e857c8980 0x560e857cd780 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f2b5c004ec0 tx=0x7f2b5c024690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bda50c98c72abef2 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:25:18.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.343+0000 7f2b54ff9640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:25:18.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.343+0000 7f2b54ff9640 1 -- 172.21.15.26:0/1937954532 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b5c01d040 con 0x560e857c8980 2024-09-11T23:25:18.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.343+0000 7f2b54ff9640 1 -- 172.21.15.26:0/1937954532 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8586+0+0 (secure 0 0 0) 0x7f2b5c004020 con 0x560e857c8980 2024-09-11T23:25:18.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.343+0000 7f2b756e2740 1 -- 172.21.15.26:0/1937954532 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b680b1070 con 0x560e857c8980 2024-09-11T23:25:18.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.343+0000 7f2b54ff9640 1 -- 172.21.15.26:0/1937954532 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f2b5c004260 con 0x560e857c8980 2024-09-11T23:25:18.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.343+0000 7f2b756e2740 10 client.16569 fetch_fsmap learned FSMap version 41 2024-09-11T23:25:18.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.343+0000 7f2b756e2740 1 -- 172.21.15.26:0/1937954532 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x560e8584dac0 con 0x560e857c8980 2024-09-11T23:25:18.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.344+0000 7f2b54ff9640 1 -- 172.21.15.26:0/1937954532 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b680b1070 con 0x560e857c8980 2024-09-11T23:25:18.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.344+0000 7f2b54ff9640 1 -- 172.21.15.26:0/1937954532 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 41) ==== 8426+0+0 (secure 0 0 0) 0x7f2b5c024b90 con 0x560e857c8980 2024-09-11T23:25:18.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.344+0000 7f2b756e2740 10 client.16569 fetch_fsmap finished waiting for FSMap version 41 2024-09-11T23:25:18.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.344+0000 7f2b756e2740 10 client.16569 resolve_mds: validated gid 24485 aka daemon mds.b 2024-09-11T23:25:18.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.344+0000 7f2b756e2740 20 client.16569 populate_metadata read hostname 'smithi026' 2024-09-11T23:25:18.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.344+0000 7f2b756e2740 1 --2- 172.21.15.26:0/1937954532 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x560e8584e2d0 0x560e858506f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:18.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.344+0000 7f2b756e2740 4 client.16569 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "27fb6b2d", "release": true, "await": true}] 2024-09-11T23:25:18.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.344+0000 7f2b756e2740 1 -- 172.21.15.26:0/1937954532 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "quiesce db", "set_id": "27fb6b2d", "release": true, "await": true}) -- 0x560e857350a0 con 0x560e8584e2d0 2024-09-11T23:25:18.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.344+0000 7f2b710a5640 1 --2- 172.21.15.26:0/1937954532 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x560e8584e2d0 0x560e858506f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:18.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.345+0000 7f2b710a5640 1 --2- 172.21.15.26:0/1937954532 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x560e8584e2d0 0x560e858506f0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=64a264b7b07cb303 server_cookie=bcfeefc3966569b6 in_seq=0 out_seq=0 2024-09-11T23:25:18.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:18.345+0000 7f2b54ff9640 10 client.16569 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:25:19.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:25:19 smithi136 ceph-mon[35121]: pgmap v212: 97 pgs: 97 active+clean; 2.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 11 MiB/s wr, 612 op/s 2024-09-11T23:25:20.000 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:25:19 smithi026 ceph-mon[31172]: pgmap v212: 97 pgs: 97 active+clean; 2.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 11 MiB/s wr, 612 op/s 2024-09-11T23:25:20.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:25:19 smithi076 ceph-mon[35296]: pgmap v212: 97 pgs: 97 active+clean; 2.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 11 MiB/s wr, 612 op/s 2024-09-11T23:25:20.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:20.128+0000 7f2b54ff9640 1 -- 172.21.15.26:0/1937954532 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x560e857350a0 con 0x560e8584e2d0 2024-09-11T23:25:20.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:20.128+0000 7f2b54ff9640 10 client.16569 handle_command_reply: tid=0 2024-09-11T23:25:20.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:20.128+0000 7f2b756e2740 1 client.16569 shutdown 2024-09-11T23:25:20.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:20.128+0000 7f2b756e2740 1 -- 172.21.15.26:0/1937954532 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560e857c8980 msgr2=0x560e857cd780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:25:20.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:20.128+0000 7f2b756e2740 1 --2- 172.21.15.26:0/1937954532 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560e857c8980 0x560e857cd780 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f2b5c004ec0 tx=0x7f2b5c024690 comp rx=0 tx=0).stop 2024-09-11T23:25:20.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:20.128+0000 7f2b756e2740 1 -- 172.21.15.26:0/1937954532 shutdown_connections 2024-09-11T23:25:20.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:20.128+0000 7f2b756e2740 1 --2- 172.21.15.26:0/1937954532 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560e857d3240 0x560e857cdcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:20.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:20.128+0000 7f2b756e2740 1 --2- 172.21.15.26:0/1937954532 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x560e8584e2d0 0x560e858506f0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:20.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:20.128+0000 7f2b756e2740 1 --2- 172.21.15.26:0/1937954532 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560e857c8980 0x560e857cd780 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:20.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:20.128+0000 7f2b54ff9640 0 client.16569 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:25:20.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:20.128+0000 7f2b756e2740 1 --2- 172.21.15.26:0/1937954532 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560e857c8060 0x560e857cd240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:20.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:20.128+0000 7f2b756e2740 1 -- 172.21.15.26:0/1937954532 >> 172.21.15.26:0/1937954532 conn(0x560e857c45f0 msgr2=0x560e857c3750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:25:20.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:20.128+0000 7f2b756e2740 1 -- 172.21.15.26:0/1937954532 shutdown_connections 2024-09-11T23:25:20.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:20.128+0000 7f2b756e2740 1 -- 172.21.15.26:0/1937954532 wait complete. 2024-09-11T23:25:20.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:20.128+0000 7f2b756e2740 20 client.16569 trim_cache size 0 max 16384 2024-09-11T23:25:20.129 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-11T23:25:20.130 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 41, 2024-09-11T23:25:20.130 INFO:teuthology.orchestra.run.smithi026.stdout: "leader": 24485, 2024-09-11T23:25:20.130 INFO:teuthology.orchestra.run.smithi026.stdout: "set_version": 24, 2024-09-11T23:25:20.130 INFO:teuthology.orchestra.run.smithi026.stdout: "sets": { 2024-09-11T23:25:20.130 INFO:teuthology.orchestra.run.smithi026.stdout: "27fb6b2d": { 2024-09-11T23:25:20.130 INFO:teuthology.orchestra.run.smithi026.stdout: "version": 24, 2024-09-11T23:25:20.130 INFO:teuthology.orchestra.run.smithi026.stdout: "age_ref": 0.0, 2024-09-11T23:25:20.130 INFO:teuthology.orchestra.run.smithi026.stdout: "state": { 2024-09-11T23:25:20.130 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "RELEASED", 2024-09-11T23:25:20.130 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.0 2024-09-11T23:25:20.130 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:25:20.130 INFO:teuthology.orchestra.run.smithi026.stdout: "timeout": 90.0, 2024-09-11T23:25:20.130 INFO:teuthology.orchestra.run.smithi026.stdout: "expiration": 140.1, 2024-09-11T23:25:20.131 INFO:teuthology.orchestra.run.smithi026.stdout: "members": { 2024-09-11T23:25:20.131 INFO:teuthology.orchestra.run.smithi026.stdout: "file:/": { 2024-09-11T23:25:20.131 INFO:teuthology.orchestra.run.smithi026.stdout: "excluded": false, 2024-09-11T23:25:20.131 INFO:teuthology.orchestra.run.smithi026.stdout: "state": { 2024-09-11T23:25:20.131 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "RELEASED", 2024-09-11T23:25:20.131 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.0 2024-09-11T23:25:20.131 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:25:20.131 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:25:20.131 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:25:20.131 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:25:20.131 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:25:20.131 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-11T23:25:20.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:20.129+0000 7f2b73309640 1 -- 172.21.15.26:0/672001877 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f2b580a2cf0 msgr2=0x7f2b580a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:25:20.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:20.129+0000 7f2b73309640 1 --2- 172.21.15.26:0/672001877 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f2b580a2cf0 0x7f2b580a51b0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f2b64009ca0 tx=0x7f2b64005c50 comp rx=0 tx=0).stop 2024-09-11T23:25:20.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:20.129+0000 7f2b73309640 1 -- 172.21.15.26:0/672001877 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2b6c100e20 msgr2=0x7f2b6c1970e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:25:20.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:20.129+0000 7f2b73309640 1 --2- 172.21.15.26:0/672001877 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2b6c100e20 0x7f2b6c1970e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f2b6800e9f0 tx=0x7f2b6800eeb0 comp rx=0 tx=0).stop 2024-09-11T23:25:20.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:20.129+0000 7f2b73309640 1 -- 172.21.15.26:0/672001877 shutdown_connections 2024-09-11T23:25:20.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:20.129+0000 7f2b73309640 1 --2- 172.21.15.26:0/672001877 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f2b580a2cf0 0x7f2b580a51b0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:20.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:20.129+0000 7f2b73309640 1 --2- 172.21.15.26:0/672001877 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2b6c102ef0 0x7f2b6c197b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:20.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:20.129+0000 7f2b73309640 1 --2- 172.21.15.26:0/672001877 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b6c102570 0x7f2b6c197620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:20.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:20.129+0000 7f2b73309640 1 --2- 172.21.15.26:0/672001877 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2b6c100e20 0x7f2b6c1970e0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:20.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:20.129+0000 7f2b73309640 1 -- 172.21.15.26:0/672001877 >> 172.21.15.26:0/672001877 conn(0x7f2b6c0fc7d0 msgr2=0x7f2b6c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:25:20.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:20.129+0000 7f2b73309640 1 -- 172.21.15.26:0/672001877 shutdown_connections 2024-09-11T23:25:20.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:20.129+0000 7f2b73309640 1 -- 172.21.15.26:0/672001877 wait complete. 2024-09-11T23:25:20.955 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '27fb6b2d', total seconds elapsed: 44.7 2024-09-11T23:25:21.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:25:21 smithi136 ceph-mon[35121]: pgmap v213: 97 pgs: 97 active+clean; 2.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 14 MiB/s wr, 977 op/s 2024-09-11T23:25:21.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:25:21 smithi136 ceph-mon[35121]: osdmap e88: 12 total, 12 up, 12 in 2024-09-11T23:25:21.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:25:21 smithi136 ceph-mon[35121]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:25:22.000 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:25:21 smithi026 ceph-mon[31172]: pgmap v213: 97 pgs: 97 active+clean; 2.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 14 MiB/s wr, 977 op/s 2024-09-11T23:25:22.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:25:21 smithi026 ceph-mon[31172]: osdmap e88: 12 total, 12 up, 12 in 2024-09-11T23:25:22.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:25:21 smithi026 ceph-mon[31172]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:25:22.052 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:25:21 smithi076 ceph-mon[35296]: pgmap v213: 97 pgs: 97 active+clean; 2.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 14 MiB/s wr, 977 op/s 2024-09-11T23:25:22.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:25:21 smithi076 ceph-mon[35296]: osdmap e88: 12 total, 12 up, 12 in 2024-09-11T23:25:22.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:25:21 smithi076 ceph-mon[35296]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:25:23.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:25:23 smithi136 ceph-mon[35121]: pgmap v215: 97 pgs: 97 active+clean; 2.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 10 MiB/s wr, 758 op/s 2024-09-11T23:25:24.000 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:25:23 smithi026 ceph-mon[31172]: pgmap v215: 97 pgs: 97 active+clean; 2.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 10 MiB/s wr, 758 op/s 2024-09-11T23:25:24.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:25:23 smithi076 ceph-mon[35296]: pgmap v215: 97 pgs: 97 active+clean; 2.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 10 MiB/s wr, 758 op/s 2024-09-11T23:25:25.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:25:25 smithi136 ceph-mon[35121]: pgmap v216: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 7.9 MiB/s wr, 717 op/s 2024-09-11T23:25:26.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:25:25 smithi026 ceph-mon[31172]: pgmap v216: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 7.9 MiB/s wr, 717 op/s 2024-09-11T23:25:26.052 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:25:25 smithi076 ceph-mon[35296]: pgmap v216: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 7.9 MiB/s wr, 717 op/s 2024-09-11T23:25:27.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:25:27 smithi136 ceph-mon[35121]: pgmap v217: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 6.5 MiB/s wr, 667 op/s 2024-09-11T23:25:28.000 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:25:27 smithi026 ceph-mon[31172]: pgmap v217: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 6.5 MiB/s wr, 667 op/s 2024-09-11T23:25:28.052 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:25:27 smithi076 ceph-mon[35296]: pgmap v217: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 6.5 MiB/s wr, 667 op/s 2024-09-11T23:25:29.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:25:29 smithi136 ceph-mon[35121]: pgmap v218: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 6.5 MiB/s wr, 667 op/s 2024-09-11T23:25:30.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:25:29 smithi026 ceph-mon[31172]: pgmap v218: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 6.5 MiB/s wr, 667 op/s 2024-09-11T23:25:30.052 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:25:29 smithi076 ceph-mon[35296]: pgmap v218: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 6.5 MiB/s wr, 667 op/s 2024-09-11T23:25:30.612 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.611+0000 7f2972d5b640 1 -- 172.21.15.26:0/94489815 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f296c1009b0 msgr2=0x7f296c100e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:25:30.612 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.611+0000 7f2972d5b640 1 --2- 172.21.15.26:0/94489815 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f296c1009b0 0x7f296c100e10 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f2960009930 tx=0x7f296002a420 comp rx=0 tx=0).stop 2024-09-11T23:25:30.612 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.611+0000 7f2972d5b640 1 -- 172.21.15.26:0/94489815 shutdown_connections 2024-09-11T23:25:30.612 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.611+0000 7f2972d5b640 1 --2- 172.21.15.26:0/94489815 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f296c101350 0x7f296c10d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:30.612 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.611+0000 7f2972d5b640 1 --2- 172.21.15.26:0/94489815 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f296c1009b0 0x7f296c100e10 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:30.612 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.611+0000 7f2972d5b640 1 --2- 172.21.15.26:0/94489815 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f296c1069c0 0x7f296c106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:30.612 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.611+0000 7f2972d5b640 1 -- 172.21.15.26:0/94489815 >> 172.21.15.26:0/94489815 conn(0x7f296c077990 msgr2=0x7f296c0fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:25:30.612 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.611+0000 7f2972d5b640 1 -- 172.21.15.26:0/94489815 shutdown_connections 2024-09-11T23:25:30.612 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.611+0000 7f2972d5b640 1 -- 172.21.15.26:0/94489815 wait complete. 2024-09-11T23:25:30.612 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.612+0000 7f2972d5b640 1 Processor -- start 2024-09-11T23:25:30.613 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.612+0000 7f2972d5b640 1 -- start start 2024-09-11T23:25:30.615 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.612+0000 7f2972d5b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f296c1009b0 0x7f296c0fcde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:30.615 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.612+0000 7f2972d5b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f296c101350 0x7f296c0fd320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:30.615 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.612+0000 7f2972d5b640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f296c1069c0 0x7f296c0fd860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:30.615 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.612+0000 7f2972d5b640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f296c110e50 con 0x7f296c101350 2024-09-11T23:25:30.615 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.612+0000 7f2972d5b640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f296c110cd0 con 0x7f296c1009b0 2024-09-11T23:25:30.615 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.612+0000 7f2972d5b640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f296c110fd0 con 0x7f296c1069c0 2024-09-11T23:25:30.615 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.612+0000 7f296bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f296c101350 0x7f296c0fd320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:30.615 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.612+0000 7f296bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f296c101350 0x7f296c0fd320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40052/0 (socket says 172.21.15.26:40052) 2024-09-11T23:25:30.615 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.612+0000 7f296bfff640 1 -- 172.21.15.26:0/1135481063 learned_addr learned my addr 172.21.15.26:0/1135481063 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:25:30.615 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.613+0000 7f2970af7640 1 --2- 172.21.15.26:0/1135481063 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f296c1009b0 0x7f296c0fcde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:30.615 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.613+0000 7f29712f8640 1 --2- 172.21.15.26:0/1135481063 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f296c1069c0 0x7f296c0fd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:30.616 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.613+0000 7f296bfff640 1 -- 172.21.15.26:0/1135481063 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f296c1069c0 msgr2=0x7f296c0fd860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:25:30.616 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.613+0000 7f296bfff640 1 --2- 172.21.15.26:0/1135481063 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f296c1069c0 0x7f296c0fd860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:30.616 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.613+0000 7f296bfff640 1 -- 172.21.15.26:0/1135481063 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f296c1009b0 msgr2=0x7f296c0fcde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:25:30.616 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.613+0000 7f296bfff640 1 --2- 172.21.15.26:0/1135481063 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f296c1009b0 0x7f296c0fcde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:30.616 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.613+0000 7f296bfff640 1 -- 172.21.15.26:0/1135481063 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f296c1a6880 con 0x7f296c101350 2024-09-11T23:25:30.616 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.613+0000 7f29712f8640 1 --2- 172.21.15.26:0/1135481063 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f296c1069c0 0x7f296c0fd860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:25:30.616 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.613+0000 7f296bfff640 1 --2- 172.21.15.26:0/1135481063 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f296c101350 0x7f296c0fd320 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f296002a940 tx=0x7f2960003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:25:30.616 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.613+0000 7f2969ffb640 1 -- 172.21.15.26:0/1135481063 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f296003f070 con 0x7f296c101350 2024-09-11T23:25:30.616 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.613+0000 7f2969ffb640 1 -- 172.21.15.26:0/1135481063 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2960031c90 con 0x7f296c101350 2024-09-11T23:25:30.616 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.613+0000 7f2972d5b640 1 -- 172.21.15.26:0/1135481063 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f296c1a6b10 con 0x7f296c101350 2024-09-11T23:25:30.616 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.613+0000 7f2969ffb640 1 -- 172.21.15.26:0/1135481063 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2960035030 con 0x7f296c101350 2024-09-11T23:25:30.616 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.613+0000 7f2972d5b640 1 -- 172.21.15.26:0/1135481063 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f296c1a7050 con 0x7f296c101350 2024-09-11T23:25:30.623 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.616+0000 7f2969ffb640 1 -- 172.21.15.26:0/1135481063 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f2960042070 con 0x7f296c101350 2024-09-11T23:25:30.623 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.616+0000 7f2975bce740 1 Processor -- start 2024-09-11T23:25:30.623 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.616+0000 7f2975bce740 1 -- start start 2024-09-11T23:25:30.623 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.616+0000 7f2975bce740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b3dbf9eaf0 0x55b3dbf9eed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:30.623 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.616+0000 7f2975bce740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b3dbf9f410 0x55b3dbfa9770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:30.624 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.616+0000 7f2975bce740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b3dbfa9cb0 0x55b3dbfac0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:30.624 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.616+0000 7f2975bce740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55b3dbf032f0 con 0x55b3dbf9f410 2024-09-11T23:25:30.624 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.616+0000 7f2975bce740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55b3dbeab270 con 0x55b3dbf9eaf0 2024-09-11T23:25:30.624 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.616+0000 7f2975bce740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55b3dbeff2b0 con 0x55b3dbfa9cb0 2024-09-11T23:25:30.624 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.617+0000 7f2970af7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b3dbf9f410 0x55b3dbfa9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:30.624 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.617+0000 7f2970af7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b3dbf9f410 0x55b3dbfa9770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40062/0 (socket says 172.21.15.26:40062) 2024-09-11T23:25:30.624 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.617+0000 7f2970af7640 1 -- 172.21.15.26:0/3883251147 learned_addr learned my addr 172.21.15.26:0/3883251147 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:25:30.624 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.617+0000 7f2969ffb640 1 --2- 172.21.15.26:0/1135481063 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f29540a2e20 0x7f29540a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:30.624 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.617+0000 7f29712f8640 1 --2- 172.21.15.26:0/3883251147 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b3dbf9eaf0 0x55b3dbf9eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:30.624 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.617+0000 7f296bfff640 1 --2- 172.21.15.26:0/3883251147 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b3dbfa9cb0 0x55b3dbfac0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:30.624 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.617+0000 7f29712f8640 1 --2- 172.21.15.26:0/1135481063 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f29540a2e20 0x7f29540a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:30.625 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.617+0000 7f2969ffb640 1 -- 172.21.15.26:0/1135481063 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8542+0+0 (secure 0 0 0) 0x7f29600efc80 con 0x7f296c101350 2024-09-11T23:25:30.625 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.617+0000 7f2970af7640 1 -- 172.21.15.26:0/3883251147 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b3dbfa9cb0 msgr2=0x55b3dbfac0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:25:30.625 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.617+0000 7f2970af7640 1 --2- 172.21.15.26:0/3883251147 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b3dbfa9cb0 0x55b3dbfac0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:30.625 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.617+0000 7f2970af7640 1 -- 172.21.15.26:0/3883251147 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b3dbf9eaf0 msgr2=0x55b3dbf9eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:25:30.625 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.617+0000 7f2970af7640 1 --2- 172.21.15.26:0/3883251147 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b3dbf9eaf0 0x55b3dbf9eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:30.625 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.617+0000 7f2970af7640 1 -- 172.21.15.26:0/3883251147 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b3dbeaa340 con 0x55b3dbf9f410 2024-09-11T23:25:30.625 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.617+0000 7f296bfff640 1 --2- 172.21.15.26:0/3883251147 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b3dbfa9cb0 0x55b3dbfac0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:25:30.625 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.617+0000 7f2970af7640 1 --2- 172.21.15.26:0/3883251147 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b3dbf9f410 0x55b3dbfa9770 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f295c009fa0 tx=0x7f295c010860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7650240b359e0f6e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:25:30.625 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.617+0000 7f29712f8640 1 --2- 172.21.15.26:0/1135481063 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f29540a2e20 0x7f29540a52e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f2964005fd0 tx=0x7f2964005850 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:25:30.625 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.617+0000 7f2952ffd640 1 -- 172.21.15.26:0/3883251147 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f295c003b70 con 0x55b3dbf9f410 2024-09-11T23:25:30.626 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.617+0000 7f2952ffd640 1 -- 172.21.15.26:0/3883251147 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f295c003d10 con 0x55b3dbf9f410 2024-09-11T23:25:30.626 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.618+0000 7f2952ffd640 1 -- 172.21.15.26:0/3883251147 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f295c011ac0 con 0x55b3dbf9f410 2024-09-11T23:25:30.626 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.618+0000 7f2975bce740 1 -- 172.21.15.26:0/3883251147 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b3dbf9f410 msgr2=0x55b3dbfa9770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:25:30.626 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.618+0000 7f2975bce740 1 --2- 172.21.15.26:0/3883251147 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b3dbf9f410 0x55b3dbfa9770 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f295c009fa0 tx=0x7f295c010860 comp rx=0 tx=0).stop 2024-09-11T23:25:30.626 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.618+0000 7f2975bce740 1 -- 172.21.15.26:0/3883251147 shutdown_connections 2024-09-11T23:25:30.626 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.618+0000 7f2975bce740 1 --2- 172.21.15.26:0/3883251147 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b3dbfa9cb0 0x55b3dbfac0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:30.626 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.618+0000 7f2975bce740 1 --2- 172.21.15.26:0/3883251147 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b3dbf9f410 0x55b3dbfa9770 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:30.626 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.618+0000 7f2975bce740 1 --2- 172.21.15.26:0/3883251147 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b3dbf9eaf0 0x55b3dbf9eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:30.626 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.618+0000 7f2975bce740 1 -- 172.21.15.26:0/3883251147 >> 172.21.15.26:0/3883251147 conn(0x55b3dbf9b060 msgr2=0x55b3dbf9b440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:25:30.626 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.618+0000 7f2975bce740 1 -- 172.21.15.26:0/3883251147 shutdown_connections 2024-09-11T23:25:30.626 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.618+0000 7f2975bce740 1 -- 172.21.15.26:0/3883251147 wait complete. 2024-09-11T23:25:30.626 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.619+0000 7f2975bce740 1 -- start start 2024-09-11T23:25:30.626 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.619+0000 7f2975bce740 1 Processor -- start 2024-09-11T23:25:30.627 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.619+0000 7f2975bce740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b3dbf9eaf0 0x55b3dbfa6aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:30.627 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.619+0000 7f2975bce740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b3dbf9f410 0x55b3dbfa6fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:30.627 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.619+0000 7f2975bce740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b3dbfa9cb0 0x55b3dbfa7520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:30.627 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.619+0000 7f2975bce740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55b3dbfac880 con 0x55b3dbf9eaf0 2024-09-11T23:25:30.627 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.619+0000 7f2975bce740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55b3dbe8c290 con 0x55b3dbfa9cb0 2024-09-11T23:25:30.627 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.619+0000 7f2975bce740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55b3dbfaca00 con 0x55b3dbf9f410 2024-09-11T23:25:30.627 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.619+0000 7f2975bce740 10 client.0 _ll_register_callbacks cb 0x55b3dbebd140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:25:30.627 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.619+0000 7f2970af7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b3dbf9eaf0 0x55b3dbfa6aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:30.627 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.619+0000 7f2970af7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b3dbf9eaf0 0x55b3dbfa6aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40076/0 (socket says 172.21.15.26:40076) 2024-09-11T23:25:30.628 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.619+0000 7f2970af7640 1 -- 172.21.15.26:0/7477943 learned_addr learned my addr 172.21.15.26:0/7477943 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:25:30.628 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.620+0000 7f296bfff640 1 --2- 172.21.15.26:0/7477943 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b3dbf9f410 0x55b3dbfa6fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:30.628 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.620+0000 7f29712f8640 1 --2- 172.21.15.26:0/7477943 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b3dbfa9cb0 0x55b3dbfa7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:30.628 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.620+0000 7f2970af7640 1 -- 172.21.15.26:0/7477943 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b3dbf9f410 msgr2=0x55b3dbfa6fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:25:30.628 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.620+0000 7f2970af7640 1 --2- 172.21.15.26:0/7477943 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b3dbf9f410 0x55b3dbfa6fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:30.628 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.620+0000 7f2970af7640 1 -- 172.21.15.26:0/7477943 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b3dbfa9cb0 msgr2=0x55b3dbfa7520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:25:30.628 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.620+0000 7f2970af7640 1 --2- 172.21.15.26:0/7477943 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b3dbfa9cb0 0x55b3dbfa7520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:30.628 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.620+0000 7f2970af7640 1 -- 172.21.15.26:0/7477943 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b3dbc16210 con 0x55b3dbf9eaf0 2024-09-11T23:25:30.628 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.620+0000 7f2970af7640 1 --2- 172.21.15.26:0/7477943 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b3dbf9eaf0 0x55b3dbfa6aa0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f295c011500 tx=0x7f295c011530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=799fd0b77002008b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:25:30.628 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.620+0000 7f294bfff640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:25:30.628 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.620+0000 7f294bfff640 1 -- 172.21.15.26:0/7477943 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f295c027020 con 0x55b3dbf9eaf0 2024-09-11T23:25:30.628 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.620+0000 7f294bfff640 1 -- 172.21.15.26:0/7477943 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8542+0+0 (secure 0 0 0) 0x7f295c003a30 con 0x55b3dbf9eaf0 2024-09-11T23:25:30.629 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.620+0000 7f2975bce740 1 -- 172.21.15.26:0/7477943 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f295c003d10 con 0x55b3dbf9eaf0 2024-09-11T23:25:30.629 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.620+0000 7f294bfff640 1 -- 172.21.15.26:0/7477943 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f295c032050 con 0x55b3dbf9eaf0 2024-09-11T23:25:30.629 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.620+0000 7f2975bce740 10 client.16581 fetch_fsmap learned FSMap version 42 2024-09-11T23:25:30.629 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.620+0000 7f2975bce740 1 -- 172.21.15.26:0/7477943 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55b3dbce2620 con 0x55b3dbf9eaf0 2024-09-11T23:25:30.629 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.621+0000 7f294bfff640 1 -- 172.21.15.26:0/7477943 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f295c003d10 con 0x55b3dbf9eaf0 2024-09-11T23:25:30.629 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.621+0000 7f294bfff640 1 -- 172.21.15.26:0/7477943 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 42) ==== 8422+0+0 (secure 0 0 0) 0x7f295c004000 con 0x55b3dbf9eaf0 2024-09-11T23:25:30.629 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.621+0000 7f2975bce740 10 client.16581 fetch_fsmap finished waiting for FSMap version 42 2024-09-11T23:25:30.629 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.621+0000 7f2975bce740 10 client.16581 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:25:30.629 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.621+0000 7f2975bce740 20 client.16581 populate_metadata read hostname 'smithi026' 2024-09-11T23:25:30.629 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.621+0000 7f2975bce740 1 --2- 172.21.15.26:0/7477943 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55b3dbfb9b10 0x55b3dbfbbf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:30.629 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.621+0000 7f2975bce740 4 client.16581 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:25:30.629 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.621+0000 7f2975bce740 1 -- 172.21.15.26:0/7477943 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b3dbf10a20 con 0x55b3dbfb9b10 2024-09-11T23:25:30.629 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.621+0000 7f296bfff640 1 --2- 172.21.15.26:0/7477943 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55b3dbfb9b10 0x55b3dbfbbf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:30.629 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.621+0000 7f294bfff640 10 client.16581 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:25:30.630 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.621+0000 7f296bfff640 1 --2- 172.21.15.26:0/7477943 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55b3dbfb9b10 0x55b3dbfbbf00 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=512ec364e5828411 server_cookie=a0d6937a4b80c16f in_seq=0 out_seq=0 2024-09-11T23:25:30.630 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.622+0000 7f294bfff640 1 -- 172.21.15.26:0/7477943 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b3dbf10a20 con 0x55b3dbfb9b10 2024-09-11T23:25:30.630 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.622+0000 7f294bfff640 10 client.16581 handle_command_reply: tid=0 2024-09-11T23:25:30.630 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.622+0000 7f2975bce740 1 client.16581 shutdown 2024-09-11T23:25:30.630 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.622+0000 7f2975bce740 1 -- 172.21.15.26:0/7477943 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b3dbf9eaf0 msgr2=0x55b3dbfa6aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:25:30.630 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.622+0000 7f2975bce740 1 --2- 172.21.15.26:0/7477943 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b3dbf9eaf0 0x55b3dbfa6aa0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f295c011500 tx=0x7f295c011530 comp rx=0 tx=0).stop 2024-09-11T23:25:30.630 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.622+0000 7f2975bce740 1 -- 172.21.15.26:0/7477943 shutdown_connections 2024-09-11T23:25:30.630 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.622+0000 7f2975bce740 1 --2- 172.21.15.26:0/7477943 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b3dbfa9cb0 0x55b3dbfa7520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:30.630 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.622+0000 7f2975bce740 1 --2- 172.21.15.26:0/7477943 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55b3dbfb9b10 0x55b3dbfbbf00 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:30.630 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.622+0000 7f2975bce740 1 --2- 172.21.15.26:0/7477943 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b3dbf9f410 0x55b3dbfa6fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:30.630 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.622+0000 7f2975bce740 1 --2- 172.21.15.26:0/7477943 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b3dbf9eaf0 0x55b3dbfa6aa0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:30.630 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.622+0000 7f294bfff640 0 client.16581 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:25:30.631 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.622+0000 7f2975bce740 1 -- 172.21.15.26:0/7477943 >> 172.21.15.26:0/7477943 conn(0x55b3dbf9b060 msgr2=0x55b3dbfa0dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:25:30.631 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.623+0000 7f2975bce740 1 -- 172.21.15.26:0/7477943 shutdown_connections 2024-09-11T23:25:30.631 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.623+0000 7f2975bce740 1 -- 172.21.15.26:0/7477943 wait complete. 2024-09-11T23:25:30.631 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.623+0000 7f2975bce740 20 client.16581 trim_cache size 0 max 16384 2024-09-11T23:25:30.639 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.637+0000 7f2975bce740 1 Processor -- start 2024-09-11T23:25:30.639 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.637+0000 7f2975bce740 1 -- start start 2024-09-11T23:25:30.639 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.637+0000 7f2975bce740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b3dbf9eaf0 0x55b3dbfa7350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:30.639 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.637+0000 7f2975bce740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b3dbf9f410 0x55b3dbfc15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:30.639 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.637+0000 7f2975bce740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b3dbfa9cb0 0x55b3dbfc1b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:30.639 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.637+0000 7f2975bce740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f296c1afac0 con 0x55b3dbfa9cb0 2024-09-11T23:25:30.639 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.637+0000 7f2975bce740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f296c1b18d0 con 0x55b3dbf9eaf0 2024-09-11T23:25:30.640 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.637+0000 7f2975bce740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f296c1b0250 con 0x55b3dbf9f410 2024-09-11T23:25:30.640 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.638+0000 7f29712f8640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b3dbfa9cb0 0x55b3dbfc1b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:30.640 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.638+0000 7f29712f8640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b3dbfa9cb0 0x55b3dbfc1b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40086/0 (socket says 172.21.15.26:40086) 2024-09-11T23:25:30.640 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.638+0000 7f29712f8640 1 -- 172.21.15.26:0/2043108610 learned_addr learned my addr 172.21.15.26:0/2043108610 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:25:30.640 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.638+0000 7f2970af7640 1 --2- 172.21.15.26:0/2043108610 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b3dbf9eaf0 0x55b3dbfa7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:30.640 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.638+0000 7f296bfff640 1 --2- 172.21.15.26:0/2043108610 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b3dbf9f410 0x55b3dbfc15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:30.640 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.638+0000 7f29712f8640 1 -- 172.21.15.26:0/2043108610 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b3dbf9f410 msgr2=0x55b3dbfc15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:25:30.640 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.638+0000 7f29712f8640 1 --2- 172.21.15.26:0/2043108610 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b3dbf9f410 0x55b3dbfc15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:30.640 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.638+0000 7f29712f8640 1 -- 172.21.15.26:0/2043108610 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b3dbf9eaf0 msgr2=0x55b3dbfa7350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:25:30.640 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.638+0000 7f29712f8640 1 --2- 172.21.15.26:0/2043108610 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b3dbf9eaf0 0x55b3dbfa7350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:30.640 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.638+0000 7f29712f8640 1 -- 172.21.15.26:0/2043108610 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b3dbf9ae90 con 0x55b3dbfa9cb0 2024-09-11T23:25:30.640 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.638+0000 7f2970af7640 1 --2- 172.21.15.26:0/2043108610 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b3dbf9eaf0 0x55b3dbfa7350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:25:30.640 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.638+0000 7f29712f8640 1 --2- 172.21.15.26:0/2043108610 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b3dbfa9cb0 0x55b3dbfc1b00 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f296400cf50 tx=0x7f296400c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e685945d7d8c7a1f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:25:30.641 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.638+0000 7f29527fc640 1 -- 172.21.15.26:0/2043108610 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2964014250 con 0x55b3dbfa9cb0 2024-09-11T23:25:30.641 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.638+0000 7f29527fc640 1 -- 172.21.15.26:0/2043108610 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2964005500 con 0x55b3dbfa9cb0 2024-09-11T23:25:30.641 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.638+0000 7f29527fc640 1 -- 172.21.15.26:0/2043108610 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2964013460 con 0x55b3dbfa9cb0 2024-09-11T23:25:30.641 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.639+0000 7f2975bce740 1 -- 172.21.15.26:0/2043108610 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b3dbfa9cb0 msgr2=0x55b3dbfc1b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:25:30.641 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.639+0000 7f2975bce740 1 --2- 172.21.15.26:0/2043108610 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b3dbfa9cb0 0x55b3dbfc1b00 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f296400cf50 tx=0x7f296400c4b0 comp rx=0 tx=0).stop 2024-09-11T23:25:30.641 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.639+0000 7f2975bce740 1 -- 172.21.15.26:0/2043108610 shutdown_connections 2024-09-11T23:25:30.641 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.639+0000 7f2975bce740 1 --2- 172.21.15.26:0/2043108610 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b3dbfa9cb0 0x55b3dbfc1b00 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:30.641 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.639+0000 7f2975bce740 1 --2- 172.21.15.26:0/2043108610 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b3dbf9f410 0x55b3dbfc15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:30.641 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.639+0000 7f2975bce740 1 --2- 172.21.15.26:0/2043108610 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b3dbf9eaf0 0x55b3dbfa7350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:30.641 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.639+0000 7f2975bce740 1 -- 172.21.15.26:0/2043108610 >> 172.21.15.26:0/2043108610 conn(0x55b3dbf9b060 msgr2=0x55b3dbfa5b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:25:30.641 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.639+0000 7f2975bce740 1 -- 172.21.15.26:0/2043108610 shutdown_connections 2024-09-11T23:25:30.641 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.639+0000 7f2975bce740 1 -- 172.21.15.26:0/2043108610 wait complete. 2024-09-11T23:25:30.641 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.639+0000 7f2975bce740 1 -- start start 2024-09-11T23:25:30.641 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.639+0000 7f2975bce740 1 Processor -- start 2024-09-11T23:25:30.644 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.640+0000 7f2975bce740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b3dbf9eaf0 0x55b3dbfb1c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:30.644 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.640+0000 7f2975bce740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b3dbf9f410 0x55b3dbfb21d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:30.644 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.640+0000 7f2975bce740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b3dbfa9cb0 0x55b3dbfa2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:30.644 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.640+0000 7f2975bce740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f296c1b1140 con 0x55b3dbfa9cb0 2024-09-11T23:25:30.644 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.640+0000 7f2975bce740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f296c1b09b0 con 0x55b3dbf9eaf0 2024-09-11T23:25:30.644 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.640+0000 7f2975bce740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f296c1adac0 con 0x55b3dbf9f410 2024-09-11T23:25:30.645 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.640+0000 7f2975bce740 10 client.0 _ll_register_callbacks cb 0x7f296400cc60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:25:30.645 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.640+0000 7f29712f8640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b3dbfa9cb0 0x55b3dbfa2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:30.645 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.640+0000 7f29712f8640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b3dbfa9cb0 0x55b3dbfa2cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40090/0 (socket says 172.21.15.26:40090) 2024-09-11T23:25:30.645 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.640+0000 7f29712f8640 1 -- 172.21.15.26:0/535280134 learned_addr learned my addr 172.21.15.26:0/535280134 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:25:30.645 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.640+0000 7f2970af7640 1 --2- 172.21.15.26:0/535280134 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b3dbf9eaf0 0x55b3dbfb1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:30.645 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.640+0000 7f296bfff640 1 --2- 172.21.15.26:0/535280134 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b3dbf9f410 0x55b3dbfb21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:30.645 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.640+0000 7f29712f8640 1 -- 172.21.15.26:0/535280134 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b3dbf9f410 msgr2=0x55b3dbfb21d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:25:30.645 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.640+0000 7f29712f8640 1 --2- 172.21.15.26:0/535280134 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b3dbf9f410 0x55b3dbfb21d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:30.645 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.640+0000 7f29712f8640 1 -- 172.21.15.26:0/535280134 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b3dbf9eaf0 msgr2=0x55b3dbfb1c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:25:30.645 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.640+0000 7f29712f8640 1 --2- 172.21.15.26:0/535280134 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b3dbf9eaf0 0x55b3dbfb1c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:30.645 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.640+0000 7f29712f8640 1 -- 172.21.15.26:0/535280134 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b3dbfa4920 con 0x55b3dbfa9cb0 2024-09-11T23:25:30.645 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.640+0000 7f29712f8640 1 --2- 172.21.15.26:0/535280134 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b3dbfa9cb0 0x55b3dbfa2cf0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f2964011ce0 tx=0x7f296400c360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85cdf1e39051ab0e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:25:30.645 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.640+0000 7f294bfff640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:25:30.646 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.641+0000 7f294bfff640 1 -- 172.21.15.26:0/535280134 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2964004c40 con 0x55b3dbfa9cb0 2024-09-11T23:25:30.646 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.641+0000 7f294bfff640 1 -- 172.21.15.26:0/535280134 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8542+0+0 (secure 0 0 0) 0x7f2964004de0 con 0x55b3dbfa9cb0 2024-09-11T23:25:30.646 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.641+0000 7f2975bce740 1 -- 172.21.15.26:0/535280134 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2964005500 con 0x55b3dbfa9cb0 2024-09-11T23:25:30.646 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.641+0000 7f294bfff640 1 -- 172.21.15.26:0/535280134 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f296403f050 con 0x55b3dbfa9cb0 2024-09-11T23:25:30.646 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.641+0000 7f2975bce740 10 client.16587 fetch_fsmap learned FSMap version 42 2024-09-11T23:25:30.646 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.641+0000 7f2975bce740 1 -- 172.21.15.26:0/535280134 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55b3dc023a00 con 0x55b3dbfa9cb0 2024-09-11T23:25:30.646 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.641+0000 7f294bfff640 1 -- 172.21.15.26:0/535280134 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2964005500 con 0x55b3dbfa9cb0 2024-09-11T23:25:30.646 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.641+0000 7f294bfff640 1 -- 172.21.15.26:0/535280134 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 42) ==== 8422+0+0 (secure 0 0 0) 0x7f2964030070 con 0x55b3dbfa9cb0 2024-09-11T23:25:30.646 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.641+0000 7f2975bce740 10 client.16587 fetch_fsmap finished waiting for FSMap version 42 2024-09-11T23:25:30.646 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.641+0000 7f2975bce740 10 client.16587 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:25:30.646 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.641+0000 7f2975bce740 20 client.16587 populate_metadata read hostname 'smithi026' 2024-09-11T23:25:30.646 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.641+0000 7f2975bce740 1 --2- 172.21.15.26:0/535280134 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55b3dc024110 0x55b3dc026530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:30.646 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.641+0000 7f2975bce740 4 client.16587 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T23:25:30.647 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.641+0000 7f2975bce740 1 -- 172.21.15.26:0/535280134 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b3dbf0ad70 con 0x55b3dc024110 2024-09-11T23:25:30.647 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.642+0000 7f2970af7640 1 --2- 172.21.15.26:0/535280134 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55b3dc024110 0x55b3dc026530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:30.647 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.642+0000 7f2970af7640 1 --2- 172.21.15.26:0/535280134 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55b3dc024110 0x55b3dc026530 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=899148eb1c51ad95 server_cookie=29956863f99c06dc in_seq=0 out_seq=0 2024-09-11T23:25:30.647 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.642+0000 7f294bfff640 10 client.16587 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:25:30.682 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.682+0000 7f294bfff640 1 -- 172.21.15.26:0/535280134 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b3dbf0ad70 con 0x55b3dc024110 2024-09-11T23:25:30.683 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.682+0000 7f294bfff640 10 client.16587 handle_command_reply: tid=0 2024-09-11T23:25:30.683 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.682+0000 7f2975bce740 1 client.16587 shutdown 2024-09-11T23:25:30.683 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.682+0000 7f2975bce740 1 -- 172.21.15.26:0/535280134 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b3dbfa9cb0 msgr2=0x55b3dbfa2cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:25:30.683 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.682+0000 7f2975bce740 1 --2- 172.21.15.26:0/535280134 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b3dbfa9cb0 0x55b3dbfa2cf0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f2964011ce0 tx=0x7f296400c360 comp rx=0 tx=0).stop 2024-09-11T23:25:30.683 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.682+0000 7f2975bce740 1 -- 172.21.15.26:0/535280134 shutdown_connections 2024-09-11T23:25:30.683 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.682+0000 7f2975bce740 1 --2- 172.21.15.26:0/535280134 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55b3dc024110 0x55b3dc026530 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:30.683 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.682+0000 7f2975bce740 1 --2- 172.21.15.26:0/535280134 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b3dbfa9cb0 0x55b3dbfa2cf0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:30.683 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.682+0000 7f2975bce740 1 --2- 172.21.15.26:0/535280134 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b3dbf9f410 0x55b3dbfb21d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:30.683 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.682+0000 7f2975bce740 1 --2- 172.21.15.26:0/535280134 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b3dbf9eaf0 0x55b3dbfb1c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:30.683 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.682+0000 7f294bfff640 0 client.16587 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:25:30.683 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.682+0000 7f2975bce740 1 -- 172.21.15.26:0/535280134 >> 172.21.15.26:0/535280134 conn(0x55b3dbf9b060 msgr2=0x55b3dbf999a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:25:30.683 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.682+0000 7f2975bce740 1 -- 172.21.15.26:0/535280134 shutdown_connections 2024-09-11T23:25:30.683 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.682+0000 7f2975bce740 1 -- 172.21.15.26:0/535280134 wait complete. 2024-09-11T23:25:30.684 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.683+0000 7f2975bce740 20 client.16587 trim_cache size 0 max 16384 2024-09-11T23:25:30.684 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-09-11T23:25:30.684 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "message": "", 2024-09-11T23:25:30.684 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-09-11T23:25:30.684 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-09-11T23:25:30.685 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.683+0000 7f2972d5b640 1 -- 172.21.15.26:0/1135481063 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f29540a2e20 msgr2=0x7f29540a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:25:30.685 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.683+0000 7f2972d5b640 1 --2- 172.21.15.26:0/1135481063 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f29540a2e20 0x7f29540a52e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f2964005fd0 tx=0x7f2964005850 comp rx=0 tx=0).stop 2024-09-11T23:25:30.685 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.683+0000 7f2972d5b640 1 -- 172.21.15.26:0/1135481063 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f296c101350 msgr2=0x7f296c0fd320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:25:30.685 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.683+0000 7f2972d5b640 1 --2- 172.21.15.26:0/1135481063 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f296c101350 0x7f296c0fd320 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f296002a940 tx=0x7f2960003b90 comp rx=0 tx=0).stop 2024-09-11T23:25:30.685 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.683+0000 7f2972d5b640 1 -- 172.21.15.26:0/1135481063 shutdown_connections 2024-09-11T23:25:30.685 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.683+0000 7f2972d5b640 1 --2- 172.21.15.26:0/1135481063 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f29540a2e20 0x7f29540a52e0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:30.685 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.683+0000 7f2972d5b640 1 --2- 172.21.15.26:0/1135481063 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f296c1069c0 0x7f296c0fd860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:30.685 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.683+0000 7f2972d5b640 1 --2- 172.21.15.26:0/1135481063 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f296c101350 0x7f296c0fd320 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:30.686 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.683+0000 7f2972d5b640 1 --2- 172.21.15.26:0/1135481063 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f296c1009b0 0x7f296c0fcde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:30.686 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.683+0000 7f2972d5b640 1 -- 172.21.15.26:0/1135481063 >> 172.21.15.26:0/1135481063 conn(0x7f296c077990 msgr2=0x7f296c0699e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:25:30.686 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.683+0000 7f2972d5b640 1 -- 172.21.15.26:0/1135481063 shutdown_connections 2024-09-11T23:25:30.686 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:30.683+0000 7f2972d5b640 1 -- 172.21.15.26:0/1135481063 wait complete. 2024-09-11T23:25:31.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:25:31 smithi136 ceph-mon[35121]: pgmap v219: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.7 MiB/s wr, 236 op/s 2024-09-11T23:25:31.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:25:31 smithi136 ceph-mon[35121]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:25:32.000 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:25:31 smithi026 ceph-mon[31172]: pgmap v219: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.7 MiB/s wr, 236 op/s 2024-09-11T23:25:32.000 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:25:31 smithi026 ceph-mon[31172]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:25:32.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:25:31 smithi076 ceph-mon[35296]: pgmap v219: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.7 MiB/s wr, 236 op/s 2024-09-11T23:25:32.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:25:31 smithi076 ceph-mon[35296]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:25:33.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:25:33 smithi136 ceph-mon[35121]: pgmap v220: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.1 MiB/s wr, 171 op/s 2024-09-11T23:25:34.000 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:25:33 smithi026 ceph-mon[31172]: pgmap v220: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.1 MiB/s wr, 171 op/s 2024-09-11T23:25:34.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:25:33 smithi076 ceph-mon[35296]: pgmap v220: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.1 MiB/s wr, 171 op/s 2024-09-11T23:25:35.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:25:35 smithi136 ceph-mon[35121]: pgmap v221: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.1 MiB/s wr, 188 op/s 2024-09-11T23:25:36.000 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:25:35 smithi026 ceph-mon[31172]: pgmap v221: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.1 MiB/s wr, 188 op/s 2024-09-11T23:25:36.052 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:25:35 smithi076 ceph-mon[35296]: pgmap v221: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.1 MiB/s wr, 188 op/s 2024-09-11T23:25:36.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:25:36 smithi136 ceph-mon[35121]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:25:37.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:25:36 smithi026 ceph-mon[31172]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:25:37.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:25:36 smithi076 ceph-mon[35296]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:25:37.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:25:37 smithi136 ceph-mon[35121]: pgmap v222: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 501 KiB/s rd, 366 KiB/s wr, 81 op/s 2024-09-11T23:25:37.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:25:37 smithi136 ceph-mon[35121]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:25:38.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:25:37 smithi026 ceph-mon[31172]: pgmap v222: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 501 KiB/s rd, 366 KiB/s wr, 81 op/s 2024-09-11T23:25:38.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:25:37 smithi026 ceph-mon[31172]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:25:38.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:25:37 smithi076 ceph-mon[35296]: pgmap v222: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 501 KiB/s rd, 366 KiB/s wr, 81 op/s 2024-09-11T23:25:38.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:25:37 smithi076 ceph-mon[35296]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:25:39.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:25:39 smithi136 ceph-mon[35121]: pgmap v223: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 182 KiB/s wr, 54 op/s 2024-09-11T23:25:40.000 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:25:39 smithi026 ceph-mon[31172]: pgmap v223: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 182 KiB/s wr, 54 op/s 2024-09-11T23:25:40.052 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:25:39 smithi076 ceph-mon[35296]: pgmap v223: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 182 KiB/s wr, 54 op/s 2024-09-11T23:25:41.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:25:41 smithi136 ceph-mon[35121]: pgmap v224: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 373 KiB/s wr, 73 op/s 2024-09-11T23:25:41.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:25:41 smithi136 ceph-mon[35121]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:25:42.000 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:25:41 smithi026 ceph-mon[31172]: pgmap v224: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 373 KiB/s wr, 73 op/s 2024-09-11T23:25:42.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:25:41 smithi026 ceph-mon[31172]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:25:42.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:25:41 smithi076 ceph-mon[35296]: pgmap v224: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 373 KiB/s wr, 73 op/s 2024-09-11T23:25:42.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:25:41 smithi076 ceph-mon[35296]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:25:43.786 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.783+0000 7fc2bfd4b640 1 -- 172.21.15.26:0/3794878915 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc2b8100890 msgr2=0x7fc2b8100c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:25:43.787 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.783+0000 7fc2bfd4b640 1 --2- 172.21.15.26:0/3794878915 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc2b8100890 0x7fc2b8100c70 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fc2b0009900 tx=0x7fc2b002a410 comp rx=0 tx=0).stop 2024-09-11T23:25:43.787 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.783+0000 7fc2bfd4b640 1 -- 172.21.15.26:0/3794878915 shutdown_connections 2024-09-11T23:25:43.787 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.783+0000 7fc2bfd4b640 1 --2- 172.21.15.26:0/3794878915 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc2b810e440 0x7fc2b8110830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:43.787 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.783+0000 7fc2bfd4b640 1 --2- 172.21.15.26:0/3794878915 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc2b8101240 0x7fc2b810de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:43.787 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.783+0000 7fc2bfd4b640 1 --2- 172.21.15.26:0/3794878915 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc2b8100890 0x7fc2b8100c70 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:43.787 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.783+0000 7fc2bfd4b640 1 -- 172.21.15.26:0/3794878915 >> 172.21.15.26:0/3794878915 conn(0x7fc2b8077990 msgr2=0x7fc2b80fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:25:43.787 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.783+0000 7fc2bfd4b640 1 -- 172.21.15.26:0/3794878915 shutdown_connections 2024-09-11T23:25:43.787 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.783+0000 7fc2bfd4b640 1 -- 172.21.15.26:0/3794878915 wait complete. 2024-09-11T23:25:43.787 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.784+0000 7fc2bfd4b640 1 Processor -- start 2024-09-11T23:25:43.787 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.784+0000 7fc2bfd4b640 1 -- start start 2024-09-11T23:25:43.787 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.784+0000 7fc2bfd4b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc2b8100890 0x7fc2b819a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:43.788 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.784+0000 7fc2bfd4b640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc2b8101240 0x7fc2b819ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:43.788 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.784+0000 7fc2bfd4b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc2b810e440 0x7fc2b819eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:43.788 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.784+0000 7fc2bfd4b640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc2b8114120 con 0x7fc2b8100890 2024-09-11T23:25:43.788 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.784+0000 7fc2bfd4b640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fc2b8113fa0 con 0x7fc2b810e440 2024-09-11T23:25:43.788 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.784+0000 7fc2bfd4b640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fc2b81142a0 con 0x7fc2b8101240 2024-09-11T23:25:43.788 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.784+0000 7fc2bdae7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc2b8100890 0x7fc2b819a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:43.788 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.785+0000 7fc2bdae7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc2b8100890 0x7fc2b819a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34084/0 (socket says 172.21.15.26:34084) 2024-09-11T23:25:43.788 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.785+0000 7fc2bdae7640 1 -- 172.21.15.26:0/4279714040 learned_addr learned my addr 172.21.15.26:0/4279714040 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:25:43.788 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.785+0000 7fc2be2e8640 1 --2- 172.21.15.26:0/4279714040 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc2b810e440 0x7fc2b819eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:43.788 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.785+0000 7fc2bd2e6640 1 --2- 172.21.15.26:0/4279714040 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc2b8101240 0x7fc2b819ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:43.788 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.785+0000 7fc2bdae7640 1 -- 172.21.15.26:0/4279714040 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc2b8101240 msgr2=0x7fc2b819ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:25:43.788 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.785+0000 7fc2bdae7640 1 --2- 172.21.15.26:0/4279714040 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc2b8101240 0x7fc2b819ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:43.788 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.785+0000 7fc2bdae7640 1 -- 172.21.15.26:0/4279714040 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc2b810e440 msgr2=0x7fc2b819eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:25:43.788 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.785+0000 7fc2bdae7640 1 --2- 172.21.15.26:0/4279714040 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc2b810e440 0x7fc2b819eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:43.789 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.785+0000 7fc2bdae7640 1 -- 172.21.15.26:0/4279714040 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2b819f600 con 0x7fc2b8100890 2024-09-11T23:25:43.789 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.785+0000 7fc2bd2e6640 1 --2- 172.21.15.26:0/4279714040 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc2b8101240 0x7fc2b819ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:25:43.789 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.785+0000 7fc2bdae7640 1 --2- 172.21.15.26:0/4279714040 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc2b8100890 0x7fc2b819a610 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fc2b0004770 tx=0x7fc2b0003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:25:43.789 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.785+0000 7fc2b6ffd640 1 -- 172.21.15.26:0/4279714040 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2b003f070 con 0x7fc2b8100890 2024-09-11T23:25:43.789 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.785+0000 7fc2b6ffd640 1 -- 172.21.15.26:0/4279714040 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc2b0031ca0 con 0x7fc2b8100890 2024-09-11T23:25:43.789 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.785+0000 7fc2bfd4b640 1 -- 172.21.15.26:0/4279714040 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2b819f890 con 0x7fc2b8100890 2024-09-11T23:25:43.789 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.785+0000 7fc2b6ffd640 1 -- 172.21.15.26:0/4279714040 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2b0035030 con 0x7fc2b8100890 2024-09-11T23:25:43.789 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.785+0000 7fc2bfd4b640 1 -- 172.21.15.26:0/4279714040 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2b81a7180 con 0x7fc2b8100890 2024-09-11T23:25:43.792 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.788+0000 7fc2b6ffd640 1 -- 172.21.15.26:0/4279714040 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fc2b0042070 con 0x7fc2b8100890 2024-09-11T23:25:43.793 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.788+0000 7fc2c2722740 1 Processor -- start 2024-09-11T23:25:43.793 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.789+0000 7fc2c2722740 1 -- start start 2024-09-11T23:25:43.793 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.789+0000 7fc2c2722740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fb44d8eaf0 0x55fb44d8eed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:43.793 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.789+0000 7fc2c2722740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fb44d8f410 0x55fb44d99770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:43.793 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.789+0000 7fc2c2722740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fb44d99cb0 0x55fb44d9c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:43.793 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.789+0000 7fc2c2722740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55fb44cf32f0 con 0x55fb44d8eaf0 2024-09-11T23:25:43.793 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.789+0000 7fc2c2722740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55fb44c9b270 con 0x55fb44d8f410 2024-09-11T23:25:43.793 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.789+0000 7fc2c2722740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55fb44cef2b0 con 0x55fb44d99cb0 2024-09-11T23:25:43.793 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.789+0000 7fc2be2e8640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fb44d8eaf0 0x55fb44d8eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:43.793 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.789+0000 7fc2b6ffd640 1 --2- 172.21.15.26:0/4279714040 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fc2a00a2da0 0x7fc2a00a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:43.794 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.789+0000 7fc2be2e8640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fb44d8eaf0 0x55fb44d8eed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34086/0 (socket says 172.21.15.26:34086) 2024-09-11T23:25:43.794 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.789+0000 7fc2be2e8640 1 -- 172.21.15.26:0/1228105061 learned_addr learned my addr 172.21.15.26:0/1228105061 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:25:43.794 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.789+0000 7fc2bdae7640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fb44d8f410 0x55fb44d99770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:43.794 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.789+0000 7fc2be2e8640 1 --2- 172.21.15.26:0/4279714040 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fc2a00a2da0 0x7fc2a00a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:43.794 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.789+0000 7fc2bd2e6640 1 --2- 172.21.15.26:0/1228105061 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fb44d99cb0 0x55fb44d9c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:43.794 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.789+0000 7fc2b6ffd640 1 -- 172.21.15.26:0/4279714040 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8542+0+0 (secure 0 0 0) 0x7fc2b00efd00 con 0x7fc2b8100890 2024-09-11T23:25:43.794 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.790+0000 7fc2be2e8640 1 -- 172.21.15.26:0/1228105061 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fb44d99cb0 msgr2=0x55fb44d9c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:25:43.794 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.790+0000 7fc2be2e8640 1 --2- 172.21.15.26:0/1228105061 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fb44d99cb0 0x55fb44d9c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:43.794 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.790+0000 7fc2be2e8640 1 -- 172.21.15.26:0/1228105061 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fb44d8f410 msgr2=0x55fb44d99770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:25:43.795 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.790+0000 7fc2be2e8640 1 --2- 172.21.15.26:0/1228105061 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fb44d8f410 0x55fb44d99770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:43.795 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.790+0000 7fc2be2e8640 1 -- 172.21.15.26:0/1228105061 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb44c9a340 con 0x55fb44d8eaf0 2024-09-11T23:25:43.795 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.790+0000 7fc2be2e8640 1 --2- 172.21.15.26:0/1228105061 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fb44d8eaf0 0x55fb44d8eed0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fc2a8012d20 tx=0x7fc2a8010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5248c80087636ca5 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:25:43.795 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.790+0000 7fc2be2e8640 1 --2- 172.21.15.26:0/4279714040 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fc2a00a2da0 0x7fc2a00a5260 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fc2a8010c40 tx=0x7fc2a8017000 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:25:43.795 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.790+0000 7fc29ffff640 1 -- 172.21.15.26:0/1228105061 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2a8012800 con 0x55fb44d8eaf0 2024-09-11T23:25:43.795 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.790+0000 7fc29ffff640 1 -- 172.21.15.26:0/1228105061 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc2a8009ce0 con 0x55fb44d8eaf0 2024-09-11T23:25:43.795 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.790+0000 7fc29ffff640 1 -- 172.21.15.26:0/1228105061 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2a801e8d0 con 0x55fb44d8eaf0 2024-09-11T23:25:43.795 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.791+0000 7fc2c2722740 1 -- 172.21.15.26:0/1228105061 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fb44d8eaf0 msgr2=0x55fb44d8eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:25:43.795 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.791+0000 7fc2c2722740 1 --2- 172.21.15.26:0/1228105061 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fb44d8eaf0 0x55fb44d8eed0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fc2a8012d20 tx=0x7fc2a8010740 comp rx=0 tx=0).stop 2024-09-11T23:25:43.795 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.791+0000 7fc2c2722740 1 -- 172.21.15.26:0/1228105061 shutdown_connections 2024-09-11T23:25:43.796 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.791+0000 7fc2c2722740 1 --2- 172.21.15.26:0/1228105061 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fb44d8f410 0x55fb44d99770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:43.796 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.791+0000 7fc2c2722740 1 --2- 172.21.15.26:0/1228105061 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fb44d99cb0 0x55fb44d9c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:43.796 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.791+0000 7fc2c2722740 1 --2- 172.21.15.26:0/1228105061 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fb44d8eaf0 0x55fb44d8eed0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:43.796 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.791+0000 7fc2c2722740 1 -- 172.21.15.26:0/1228105061 >> 172.21.15.26:0/1228105061 conn(0x55fb44d8b060 msgr2=0x55fb44d8b440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:25:43.796 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.791+0000 7fc2c2722740 1 -- 172.21.15.26:0/1228105061 shutdown_connections 2024-09-11T23:25:43.796 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.791+0000 7fc2c2722740 1 -- 172.21.15.26:0/1228105061 wait complete. 2024-09-11T23:25:43.796 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.791+0000 7fc2c2722740 1 -- start start 2024-09-11T23:25:43.796 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.792+0000 7fc2c2722740 1 Processor -- start 2024-09-11T23:25:43.801 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.792+0000 7fc2c2722740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fb44d8eaf0 0x55fb44d96b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:43.801 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.792+0000 7fc2c2722740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fb44d8f410 0x55fb44d97060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:43.801 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.792+0000 7fc2c2722740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fb44d99cb0 0x55fb44d975a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:43.801 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.792+0000 7fc2c2722740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55fb44d9c880 con 0x55fb44d99cb0 2024-09-11T23:25:43.801 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.792+0000 7fc2c2722740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55fb44c7c290 con 0x55fb44d8f410 2024-09-11T23:25:43.801 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.792+0000 7fc2c2722740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55fb44d9ca00 con 0x55fb44d8eaf0 2024-09-11T23:25:43.801 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.792+0000 7fc2c2722740 10 client.0 _ll_register_callbacks cb 0x55fb44cad140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:25:43.801 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.792+0000 7fc2be2e8640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fb44d99cb0 0x55fb44d975a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:43.801 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.792+0000 7fc2be2e8640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fb44d99cb0 0x55fb44d975a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34102/0 (socket says 172.21.15.26:34102) 2024-09-11T23:25:43.801 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.792+0000 7fc2be2e8640 1 -- 172.21.15.26:0/3281847494 learned_addr learned my addr 172.21.15.26:0/3281847494 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:25:43.801 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.792+0000 7fc2bdae7640 1 --2- 172.21.15.26:0/3281847494 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fb44d8eaf0 0x55fb44d96b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:43.801 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.792+0000 7fc2bd2e6640 1 --2- 172.21.15.26:0/3281847494 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fb44d8f410 0x55fb44d97060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:43.801 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.793+0000 7fc2be2e8640 1 -- 172.21.15.26:0/3281847494 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fb44d8eaf0 msgr2=0x55fb44d96b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:25:43.802 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.793+0000 7fc2be2e8640 1 --2- 172.21.15.26:0/3281847494 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fb44d8eaf0 0x55fb44d96b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:43.802 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.793+0000 7fc2be2e8640 1 -- 172.21.15.26:0/3281847494 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fb44d8f410 msgr2=0x55fb44d97060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:25:43.802 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.793+0000 7fc2be2e8640 1 --2- 172.21.15.26:0/3281847494 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fb44d8f410 0x55fb44d97060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:43.802 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.793+0000 7fc2be2e8640 1 -- 172.21.15.26:0/3281847494 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb44a06210 con 0x55fb44d99cb0 2024-09-11T23:25:43.802 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.793+0000 7fc2be2e8640 1 --2- 172.21.15.26:0/3281847494 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fb44d99cb0 0x55fb44d975a0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fc2a8005c00 tx=0x7fc2a8012ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb41b6bbe1846753 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:25:43.802 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.793+0000 7fc29d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:25:43.802 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.793+0000 7fc29d7fa640 1 -- 172.21.15.26:0/3281847494 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2a8033060 con 0x55fb44d99cb0 2024-09-11T23:25:43.802 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.793+0000 7fc29d7fa640 1 -- 172.21.15.26:0/3281847494 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8542+0+0 (secure 0 0 0) 0x7fc2a8004a00 con 0x55fb44d99cb0 2024-09-11T23:25:43.802 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.793+0000 7fc2c2722740 1 -- 172.21.15.26:0/3281847494 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc2a8009ce0 con 0x55fb44d99cb0 2024-09-11T23:25:43.802 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.793+0000 7fc29d7fa640 1 -- 172.21.15.26:0/3281847494 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7fc2a8004c00 con 0x55fb44d99cb0 2024-09-11T23:25:43.802 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.793+0000 7fc2c2722740 10 client.16599 fetch_fsmap learned FSMap version 46 2024-09-11T23:25:43.802 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.793+0000 7fc2c2722740 1 -- 172.21.15.26:0/3281847494 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55fb44ad2620 con 0x55fb44d99cb0 2024-09-11T23:25:43.802 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.793+0000 7fc29d7fa640 1 -- 172.21.15.26:0/3281847494 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc2a8009ce0 con 0x55fb44d99cb0 2024-09-11T23:25:43.802 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.793+0000 7fc29d7fa640 1 -- 172.21.15.26:0/3281847494 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 46) ==== 8418+0+0 (secure 0 0 0) 0x7fc2a800f040 con 0x55fb44d99cb0 2024-09-11T23:25:43.803 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.794+0000 7fc2c2722740 10 client.16599 fetch_fsmap finished waiting for FSMap version 46 2024-09-11T23:25:43.803 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.794+0000 7fc2c2722740 10 client.16599 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:25:43.803 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.794+0000 7fc2c2722740 20 client.16599 populate_metadata read hostname 'smithi026' 2024-09-11T23:25:43.803 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.794+0000 7fc2c2722740 1 --2- 172.21.15.26:0/3281847494 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55fb44da9b20 0x55fb44dabf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:43.803 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.794+0000 7fc2c2722740 4 client.16599 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:25:43.803 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.794+0000 7fc2c2722740 1 -- 172.21.15.26:0/3281847494 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fb44d00a20 con 0x55fb44da9b20 2024-09-11T23:25:43.803 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.794+0000 7fc2bdae7640 1 --2- 172.21.15.26:0/3281847494 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55fb44da9b20 0x55fb44dabf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:43.803 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.794+0000 7fc2bdae7640 1 --2- 172.21.15.26:0/3281847494 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55fb44da9b20 0x55fb44dabf10 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=721de35e4802694 server_cookie=416813a92a26738a in_seq=0 out_seq=0 2024-09-11T23:25:43.803 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.794+0000 7fc29d7fa640 10 client.16599 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:25:43.803 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.795+0000 7fc29d7fa640 1 -- 172.21.15.26:0/3281847494 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fb44d00a20 con 0x55fb44da9b20 2024-09-11T23:25:43.803 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.795+0000 7fc29d7fa640 10 client.16599 handle_command_reply: tid=0 2024-09-11T23:25:43.803 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.795+0000 7fc2c2722740 1 client.16599 shutdown 2024-09-11T23:25:43.803 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.795+0000 7fc2c2722740 1 -- 172.21.15.26:0/3281847494 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fb44d99cb0 msgr2=0x55fb44d975a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:25:43.804 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.795+0000 7fc2c2722740 1 --2- 172.21.15.26:0/3281847494 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fb44d99cb0 0x55fb44d975a0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fc2a8005c00 tx=0x7fc2a8012ed0 comp rx=0 tx=0).stop 2024-09-11T23:25:43.804 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.795+0000 7fc2c2722740 1 -- 172.21.15.26:0/3281847494 shutdown_connections 2024-09-11T23:25:43.804 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.795+0000 7fc2c2722740 1 --2- 172.21.15.26:0/3281847494 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fb44d8f410 0x55fb44d97060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:43.804 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.795+0000 7fc2c2722740 1 --2- 172.21.15.26:0/3281847494 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55fb44da9b20 0x55fb44dabf10 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:43.804 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.795+0000 7fc2c2722740 1 --2- 172.21.15.26:0/3281847494 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fb44d99cb0 0x55fb44d975a0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:43.804 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.795+0000 7fc2c2722740 1 --2- 172.21.15.26:0/3281847494 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fb44d8eaf0 0x55fb44d96b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:43.804 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.795+0000 7fc29d7fa640 0 client.16599 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:25:43.804 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.795+0000 7fc2c2722740 1 -- 172.21.15.26:0/3281847494 >> 172.21.15.26:0/3281847494 conn(0x55fb44d8b060 msgr2=0x55fb44d90dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:25:43.804 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.795+0000 7fc2c2722740 1 -- 172.21.15.26:0/3281847494 shutdown_connections 2024-09-11T23:25:43.805 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.795+0000 7fc2c2722740 1 -- 172.21.15.26:0/3281847494 wait complete. 2024-09-11T23:25:43.805 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.795+0000 7fc2c2722740 20 client.16599 trim_cache size 0 max 16384 2024-09-11T23:25:43.812 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.810+0000 7fc2c2722740 1 Processor -- start 2024-09-11T23:25:43.812 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.810+0000 7fc2c2722740 1 -- start start 2024-09-11T23:25:43.812 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.810+0000 7fc2c2722740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fb44d8eaf0 0x55fb44d973d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:43.812 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.810+0000 7fc2c2722740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fb44d8f410 0x55fb44db15a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:43.812 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.810+0000 7fc2c2722740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fb44d99cb0 0x55fb44db1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:43.812 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.810+0000 7fc2c2722740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc2b81afa40 con 0x55fb44d8f410 2024-09-11T23:25:43.813 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.810+0000 7fc2c2722740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fc2b81b1820 con 0x55fb44d99cb0 2024-09-11T23:25:43.813 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.810+0000 7fc2c2722740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fc2b81b01d0 con 0x55fb44d8eaf0 2024-09-11T23:25:43.813 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.810+0000 7fc2bd2e6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fb44d8f410 0x55fb44db15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:43.813 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.810+0000 7fc2bd2e6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fb44d8f410 0x55fb44db15a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34118/0 (socket says 172.21.15.26:34118) 2024-09-11T23:25:43.813 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.810+0000 7fc2bd2e6640 1 -- 172.21.15.26:0/1690442933 learned_addr learned my addr 172.21.15.26:0/1690442933 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:25:43.813 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.811+0000 7fc2be2e8640 1 --2- 172.21.15.26:0/1690442933 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fb44d99cb0 0x55fb44db1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:43.813 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.811+0000 7fc2bdae7640 1 --2- 172.21.15.26:0/1690442933 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fb44d8eaf0 0x55fb44d973d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:43.813 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.811+0000 7fc2bd2e6640 1 -- 172.21.15.26:0/1690442933 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fb44d8eaf0 msgr2=0x55fb44d973d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:25:43.813 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.811+0000 7fc2bd2e6640 1 --2- 172.21.15.26:0/1690442933 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fb44d8eaf0 0x55fb44d973d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:43.813 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.811+0000 7fc2bd2e6640 1 -- 172.21.15.26:0/1690442933 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fb44d99cb0 msgr2=0x55fb44db1ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:25:43.813 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.811+0000 7fc2bd2e6640 1 --2- 172.21.15.26:0/1690442933 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fb44d99cb0 0x55fb44db1ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:43.813 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.811+0000 7fc2bd2e6640 1 -- 172.21.15.26:0/1690442933 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb44d8c110 con 0x55fb44d8f410 2024-09-11T23:25:43.814 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.811+0000 7fc2bd2e6640 1 --2- 172.21.15.26:0/1690442933 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fb44d8f410 0x55fb44db15a0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fc2b8102760 tx=0x7fc2ac012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db67fa8ea1490be2 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:25:43.814 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.811+0000 7fc29f7fe640 1 -- 172.21.15.26:0/1690442933 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2ac020070 con 0x55fb44d8f410 2024-09-11T23:25:43.814 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.811+0000 7fc29f7fe640 1 -- 172.21.15.26:0/1690442933 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc2ac00bda0 con 0x55fb44d8f410 2024-09-11T23:25:43.814 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.811+0000 7fc29f7fe640 1 -- 172.21.15.26:0/1690442933 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2ac0107f0 con 0x55fb44d8f410 2024-09-11T23:25:43.817 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.812+0000 7fc2c2722740 1 -- 172.21.15.26:0/1690442933 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fb44d8f410 msgr2=0x55fb44db15a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:25:43.817 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.812+0000 7fc2c2722740 1 --2- 172.21.15.26:0/1690442933 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fb44d8f410 0x55fb44db15a0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fc2b8102760 tx=0x7fc2ac012820 comp rx=0 tx=0).stop 2024-09-11T23:25:43.817 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.812+0000 7fc2c2722740 1 -- 172.21.15.26:0/1690442933 shutdown_connections 2024-09-11T23:25:43.817 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.812+0000 7fc2c2722740 1 --2- 172.21.15.26:0/1690442933 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fb44d99cb0 0x55fb44db1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:43.817 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.812+0000 7fc2c2722740 1 --2- 172.21.15.26:0/1690442933 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fb44d8f410 0x55fb44db15a0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:43.817 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.812+0000 7fc2c2722740 1 --2- 172.21.15.26:0/1690442933 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fb44d8eaf0 0x55fb44d973d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:43.817 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.812+0000 7fc2c2722740 1 -- 172.21.15.26:0/1690442933 >> 172.21.15.26:0/1690442933 conn(0x55fb44d8b060 msgr2=0x55fb44d953f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:25:43.817 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.812+0000 7fc2c2722740 1 -- 172.21.15.26:0/1690442933 shutdown_connections 2024-09-11T23:25:43.818 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.812+0000 7fc2c2722740 1 -- 172.21.15.26:0/1690442933 wait complete. 2024-09-11T23:25:43.818 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.812+0000 7fc2c2722740 1 -- start start 2024-09-11T23:25:43.818 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.812+0000 7fc2c2722740 1 Processor -- start 2024-09-11T23:25:43.818 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.813+0000 7fc2c2722740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fb44d8eaf0 0x55fb44da2190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:43.818 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.813+0000 7fc2c2722740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fb44d8f410 0x55fb44d93650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:43.818 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.813+0000 7fc2c2722740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fb44d99cb0 0x55fb44d93b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:43.818 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.813+0000 7fc2c2722740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc2b81b10f0 con 0x55fb44d8f410 2024-09-11T23:25:43.818 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.813+0000 7fc2c2722740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fc2b81b0960 con 0x55fb44d99cb0 2024-09-11T23:25:43.818 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.813+0000 7fc2c2722740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fc2b81ada40 con 0x55fb44d8eaf0 2024-09-11T23:25:43.818 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.813+0000 7fc2c2722740 10 client.0 _ll_register_callbacks cb 0x7fc2a8012af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:25:43.818 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.813+0000 7fc2bd2e6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fb44d8f410 0x55fb44d93650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:43.818 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.813+0000 7fc2bd2e6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fb44d8f410 0x55fb44d93650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34130/0 (socket says 172.21.15.26:34130) 2024-09-11T23:25:43.818 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.813+0000 7fc2be2e8640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fb44d99cb0 0x55fb44d93b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:43.819 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.813+0000 7fc2bd2e6640 1 -- 172.21.15.26:0/755485170 learned_addr learned my addr 172.21.15.26:0/755485170 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:25:43.819 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.813+0000 7fc2bdae7640 1 --2- 172.21.15.26:0/755485170 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fb44d8eaf0 0x55fb44da2190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:43.819 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.813+0000 7fc2bd2e6640 1 -- 172.21.15.26:0/755485170 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fb44d8eaf0 msgr2=0x55fb44da2190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:25:43.819 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.813+0000 7fc2bd2e6640 1 --2- 172.21.15.26:0/755485170 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fb44d8eaf0 0x55fb44da2190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:43.819 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.813+0000 7fc2bd2e6640 1 -- 172.21.15.26:0/755485170 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fb44d99cb0 msgr2=0x55fb44d93b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:25:43.819 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.813+0000 7fc2bd2e6640 1 --2- 172.21.15.26:0/755485170 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fb44d99cb0 0x55fb44d93b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:43.819 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.813+0000 7fc2bd2e6640 1 -- 172.21.15.26:0/755485170 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb44d94150 con 0x55fb44d8f410 2024-09-11T23:25:43.819 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.813+0000 7fc2be2e8640 1 --2- 172.21.15.26:0/755485170 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fb44d99cb0 0x55fb44d93b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:25:43.819 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.813+0000 7fc2bd2e6640 1 --2- 172.21.15.26:0/755485170 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fb44d8f410 0x55fb44d93650 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fc2ac00bb10 tx=0x7fc2ac01b410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe147e0bee706a3a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:25:43.819 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.813+0000 7fc29d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:25:43.819 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.814+0000 7fc29d7fa640 1 -- 172.21.15.26:0/755485170 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2ac020040 con 0x55fb44d8f410 2024-09-11T23:25:43.819 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.814+0000 7fc29d7fa640 1 -- 172.21.15.26:0/755485170 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8542+0+0 (secure 0 0 0) 0x7fc2ac01b6f0 con 0x55fb44d8f410 2024-09-11T23:25:43.819 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.814+0000 7fc2c2722740 1 -- 172.21.15.26:0/755485170 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc2ac00bda0 con 0x55fb44d8f410 2024-09-11T23:25:43.820 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.814+0000 7fc29d7fa640 1 -- 172.21.15.26:0/755485170 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7fc2ac033050 con 0x55fb44d8f410 2024-09-11T23:25:43.820 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.814+0000 7fc2c2722740 10 client.16605 fetch_fsmap learned FSMap version 46 2024-09-11T23:25:43.820 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.814+0000 7fc2c2722740 1 -- 172.21.15.26:0/755485170 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55fb44e14350 con 0x55fb44d8f410 2024-09-11T23:25:43.820 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.814+0000 7fc29d7fa640 1 -- 172.21.15.26:0/755485170 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc2ac00bda0 con 0x55fb44d8f410 2024-09-11T23:25:43.820 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.814+0000 7fc29d7fa640 1 -- 172.21.15.26:0/755485170 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 46) ==== 8418+0+0 (secure 0 0 0) 0x7fc2ac02e030 con 0x55fb44d8f410 2024-09-11T23:25:43.820 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.814+0000 7fc2c2722740 10 client.16605 fetch_fsmap finished waiting for FSMap version 46 2024-09-11T23:25:43.820 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.814+0000 7fc2c2722740 10 client.16605 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:25:43.820 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.814+0000 7fc2c2722740 20 client.16605 populate_metadata read hostname 'smithi026' 2024-09-11T23:25:43.820 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.814+0000 7fc2c2722740 1 --2- 172.21.15.26:0/755485170 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55fb44e14ad0 0x55fb44e16ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:43.820 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.814+0000 7fc2c2722740 4 client.16605 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T23:25:43.820 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.814+0000 7fc2c2722740 1 -- 172.21.15.26:0/755485170 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55fb44cfad70 con 0x55fb44e14ad0 2024-09-11T23:25:43.821 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.815+0000 7fc2bdae7640 1 --2- 172.21.15.26:0/755485170 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55fb44e14ad0 0x55fb44e16ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:43.821 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.815+0000 7fc2bdae7640 1 --2- 172.21.15.26:0/755485170 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55fb44e14ad0 0x55fb44e16ef0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9686cdf27e5d058b server_cookie=7f1879cec369ed73 in_seq=0 out_seq=0 2024-09-11T23:25:43.821 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.815+0000 7fc29d7fa640 10 client.16605 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:25:43.836 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.834+0000 7fc29d7fa640 1 -- 172.21.15.26:0/755485170 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55fb44cfad70 con 0x55fb44e14ad0 2024-09-11T23:25:43.836 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.834+0000 7fc29d7fa640 10 client.16605 handle_command_reply: tid=0 2024-09-11T23:25:43.836 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.834+0000 7fc2c2722740 1 client.16605 shutdown 2024-09-11T23:25:43.837 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.835+0000 7fc2c2722740 1 -- 172.21.15.26:0/755485170 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fb44d8f410 msgr2=0x55fb44d93650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:25:43.837 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.835+0000 7fc2c2722740 1 --2- 172.21.15.26:0/755485170 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fb44d8f410 0x55fb44d93650 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fc2ac00bb10 tx=0x7fc2ac01b410 comp rx=0 tx=0).stop 2024-09-11T23:25:43.837 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.835+0000 7fc2c2722740 1 -- 172.21.15.26:0/755485170 shutdown_connections 2024-09-11T23:25:43.837 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.835+0000 7fc2c2722740 1 --2- 172.21.15.26:0/755485170 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fb44d99cb0 0x55fb44d93b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:43.837 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.835+0000 7fc2c2722740 1 --2- 172.21.15.26:0/755485170 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55fb44e14ad0 0x55fb44e16ef0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:43.837 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.835+0000 7fc2c2722740 1 --2- 172.21.15.26:0/755485170 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fb44d8f410 0x55fb44d93650 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:43.837 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.835+0000 7fc2c2722740 1 --2- 172.21.15.26:0/755485170 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fb44d8eaf0 0x55fb44da2190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:43.837 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.835+0000 7fc29d7fa640 0 client.16605 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:25:43.837 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.835+0000 7fc2c2722740 1 -- 172.21.15.26:0/755485170 >> 172.21.15.26:0/755485170 conn(0x55fb44d8b060 msgr2=0x55fb44d90550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:25:43.837 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.835+0000 7fc2c2722740 1 -- 172.21.15.26:0/755485170 shutdown_connections 2024-09-11T23:25:43.837 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.835+0000 7fc2c2722740 1 -- 172.21.15.26:0/755485170 wait complete. 2024-09-11T23:25:43.837 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.835+0000 7fc2c2722740 20 client.16605 trim_cache size 0 max 16384 2024-09-11T23:25:43.838 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-09-11T23:25:43.838 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "message": "", 2024-09-11T23:25:43.838 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-09-11T23:25:43.838 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-09-11T23:25:43.839 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.835+0000 7fc2bfd4b640 1 -- 172.21.15.26:0/4279714040 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fc2a00a2da0 msgr2=0x7fc2a00a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:25:43.839 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.835+0000 7fc2bfd4b640 1 --2- 172.21.15.26:0/4279714040 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fc2a00a2da0 0x7fc2a00a5260 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fc2a8010c40 tx=0x7fc2a8017000 comp rx=0 tx=0).stop 2024-09-11T23:25:43.839 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.835+0000 7fc2bfd4b640 1 -- 172.21.15.26:0/4279714040 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc2b8100890 msgr2=0x7fc2b819a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:25:43.839 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.835+0000 7fc2bfd4b640 1 --2- 172.21.15.26:0/4279714040 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc2b8100890 0x7fc2b819a610 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fc2b0004770 tx=0x7fc2b0003c10 comp rx=0 tx=0).stop 2024-09-11T23:25:43.839 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.836+0000 7fc2bfd4b640 1 -- 172.21.15.26:0/4279714040 shutdown_connections 2024-09-11T23:25:43.839 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.836+0000 7fc2bfd4b640 1 --2- 172.21.15.26:0/4279714040 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fc2a00a2da0 0x7fc2a00a5260 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:43.839 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.836+0000 7fc2bfd4b640 1 --2- 172.21.15.26:0/4279714040 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc2b810e440 0x7fc2b819eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:43.839 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.836+0000 7fc2bfd4b640 1 --2- 172.21.15.26:0/4279714040 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc2b8101240 0x7fc2b819ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:43.839 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.836+0000 7fc2bfd4b640 1 --2- 172.21.15.26:0/4279714040 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc2b8100890 0x7fc2b819a610 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:43.839 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.836+0000 7fc2bfd4b640 1 -- 172.21.15.26:0/4279714040 >> 172.21.15.26:0/4279714040 conn(0x7fc2b8077990 msgr2=0x7fc2b80ff540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:25:43.839 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.836+0000 7fc2bfd4b640 1 -- 172.21.15.26:0/4279714040 shutdown_connections 2024-09-11T23:25:43.839 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:43.836+0000 7fc2bfd4b640 1 -- 172.21.15.26:0/4279714040 wait complete. 2024-09-11T23:25:43.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:25:43 smithi136 ceph-mon[35121]: pgmap v225: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 243 KiB/s wr, 47 op/s 2024-09-11T23:25:44.000 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:25:43 smithi026 ceph-mon[31172]: pgmap v225: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 243 KiB/s wr, 47 op/s 2024-09-11T23:25:44.052 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:25:43 smithi076 ceph-mon[35296]: pgmap v225: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 243 KiB/s wr, 47 op/s 2024-09-11T23:25:46.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:25:45 smithi026 ceph-mon[31172]: pgmap v226: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 633 KiB/s rd, 657 KiB/s wr, 65 op/s 2024-09-11T23:25:46.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:25:45 smithi076 ceph-mon[35296]: pgmap v226: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 633 KiB/s rd, 657 KiB/s wr, 65 op/s 2024-09-11T23:25:46.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:25:45 smithi136 ceph-mon[35121]: pgmap v226: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 633 KiB/s rd, 657 KiB/s wr, 65 op/s 2024-09-11T23:25:48.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:25:47 smithi076 ceph-mon[35296]: pgmap v227: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 711 KiB/s rd, 868 KiB/s wr, 63 op/s 2024-09-11T23:25:48.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:25:47 smithi076 ceph-mon[35296]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:25:48.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:25:47 smithi136 ceph-mon[35121]: pgmap v227: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 711 KiB/s rd, 868 KiB/s wr, 63 op/s 2024-09-11T23:25:48.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:25:47 smithi136 ceph-mon[35121]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:25:48.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:25:47 smithi026 ceph-mon[31172]: pgmap v227: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 711 KiB/s rd, 868 KiB/s wr, 63 op/s 2024-09-11T23:25:48.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:25:47 smithi026 ceph-mon[31172]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:25:50.000 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:25:49 smithi026 ceph-mon[31172]: pgmap v228: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 708 KiB/s rd, 893 KiB/s wr, 58 op/s 2024-09-11T23:25:50.041 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:25:49 smithi136 ceph-mon[35121]: pgmap v228: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 708 KiB/s rd, 893 KiB/s wr, 58 op/s 2024-09-11T23:25:50.052 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:25:49 smithi076 ceph-mon[35296]: pgmap v228: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 708 KiB/s rd, 893 KiB/s wr, 58 op/s 2024-09-11T23:25:51.052 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:25:50 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:25:51.246 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:25:50 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:25:51.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:25:50 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:25:52.052 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:25:51 smithi076 ceph-mon[35296]: pgmap v229: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 793 KiB/s rd, 1.3 MiB/s wr, 95 op/s 2024-09-11T23:25:52.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:25:51 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:25:52.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:25:51 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:25:52.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:25:51 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:25:52.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:25:51 smithi136 ceph-mon[35121]: pgmap v229: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 793 KiB/s rd, 1.3 MiB/s wr, 95 op/s 2024-09-11T23:25:52.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:25:51 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:25:52.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:25:51 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:25:52.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:25:51 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:25:52.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:25:51 smithi026 ceph-mon[31172]: pgmap v229: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 793 KiB/s rd, 1.3 MiB/s wr, 95 op/s 2024-09-11T23:25:52.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:25:51 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:25:52.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:25:51 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:25:52.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:25:51 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:25:54.052 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:25:53 smithi076 ceph-mon[35296]: pgmap v230: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 609 KiB/s rd, 1.1 MiB/s wr, 76 op/s 2024-09-11T23:25:54.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:25:53 smithi136 ceph-mon[35121]: pgmap v230: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 609 KiB/s rd, 1.1 MiB/s wr, 76 op/s 2024-09-11T23:25:54.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:25:53 smithi026 ceph-mon[31172]: pgmap v230: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 609 KiB/s rd, 1.1 MiB/s wr, 76 op/s 2024-09-11T23:25:56.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:25:55 smithi076 ceph-mon[35296]: pgmap v231: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 618 KiB/s rd, 1.1 MiB/s wr, 82 op/s 2024-09-11T23:25:56.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:25:55 smithi136 ceph-mon[35121]: pgmap v231: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 618 KiB/s rd, 1.1 MiB/s wr, 82 op/s 2024-09-11T23:25:56.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:25:55 smithi026 ceph-mon[31172]: pgmap v231: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 618 KiB/s rd, 1.1 MiB/s wr, 82 op/s 2024-09-11T23:25:56.937 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.936+0000 7f12671c6640 1 -- 172.21.15.26:0/2456140164 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1260102d80 msgr2=0x7f1260103160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:25:56.937 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.936+0000 7f12671c6640 1 --2- 172.21.15.26:0/2456140164 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1260102d80 0x7f1260103160 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f1250009900 tx=0x7f125002a480 comp rx=0 tx=0).stop 2024-09-11T23:25:56.937 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.936+0000 7f12671c6640 1 -- 172.21.15.26:0/2456140164 shutdown_connections 2024-09-11T23:25:56.938 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.936+0000 7f12671c6640 1 --2- 172.21.15.26:0/2456140164 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f12601036a0 0x7f126010bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:56.938 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.936+0000 7f12671c6640 1 --2- 172.21.15.26:0/2456140164 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f126010c150 0x7f126010e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:56.938 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.936+0000 7f12671c6640 1 --2- 172.21.15.26:0/2456140164 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1260102d80 0x7f1260103160 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:56.938 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.936+0000 7f12671c6640 1 -- 172.21.15.26:0/2456140164 >> 172.21.15.26:0/2456140164 conn(0x7f12600fc6b0 msgr2=0x7f12600fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:25:56.938 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.936+0000 7f12671c6640 1 -- 172.21.15.26:0/2456140164 shutdown_connections 2024-09-11T23:25:56.938 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.936+0000 7f12671c6640 1 -- 172.21.15.26:0/2456140164 wait complete. 2024-09-11T23:25:56.938 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.937+0000 7f12671c6640 1 Processor -- start 2024-09-11T23:25:56.938 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.937+0000 7f12671c6640 1 -- start start 2024-09-11T23:25:56.940 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.937+0000 7f12671c6640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1260102d80 0x7f1260198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:56.940 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.937+0000 7f12671c6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12601036a0 0x7f12601988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:56.940 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.937+0000 7f12671c6640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f126010c150 0x7f126019cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:56.940 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.937+0000 7f12671c6640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f1260111ef0 con 0x7f12601036a0 2024-09-11T23:25:56.940 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.937+0000 7f12671c6640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f1260111d70 con 0x7f1260102d80 2024-09-11T23:25:56.940 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.937+0000 7f12671c6640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f1260112070 con 0x7f126010c150 2024-09-11T23:25:56.941 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.937+0000 7f125ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12601036a0 0x7f12601988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:56.941 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.937+0000 7f1264f62640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1260102d80 0x7f1260198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:56.941 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.937+0000 7f1264f62640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1260102d80 0x7f1260198360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:55650/0 (socket says 172.21.15.26:55650) 2024-09-11T23:25:56.941 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.937+0000 7f125ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12601036a0 0x7f12601988a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40312/0 (socket says 172.21.15.26:40312) 2024-09-11T23:25:56.941 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.937+0000 7f1264f62640 1 -- 172.21.15.26:0/2999835468 learned_addr learned my addr 172.21.15.26:0/2999835468 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:25:56.941 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.937+0000 7f1265763640 1 --2- 172.21.15.26:0/2999835468 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f126010c150 0x7f126019cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:56.941 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.937+0000 7f125ffff640 1 -- 172.21.15.26:0/2999835468 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f126010c150 msgr2=0x7f126019cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:25:56.941 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.937+0000 7f125ffff640 1 --2- 172.21.15.26:0/2999835468 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f126010c150 0x7f126019cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:56.941 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.937+0000 7f125ffff640 1 -- 172.21.15.26:0/2999835468 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1260102d80 msgr2=0x7f1260198360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:25:56.941 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.937+0000 7f125ffff640 1 --2- 172.21.15.26:0/2999835468 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1260102d80 0x7f1260198360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:56.941 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.937+0000 7f125ffff640 1 -- 172.21.15.26:0/2999835468 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f126019d350 con 0x7f12601036a0 2024-09-11T23:25:56.941 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.938+0000 7f1264f62640 1 --2- 172.21.15.26:0/2999835468 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1260102d80 0x7f1260198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:25:56.941 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.938+0000 7f1265763640 1 --2- 172.21.15.26:0/2999835468 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f126010c150 0x7f126019cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:25:56.941 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.938+0000 7f125ffff640 1 --2- 172.21.15.26:0/2999835468 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12601036a0 0x7f12601988a0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f1254005fd0 tx=0x7f125400dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:25:56.942 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.938+0000 7f125dffb640 1 -- 172.21.15.26:0/2999835468 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1254009c30 con 0x7f12601036a0 2024-09-11T23:25:56.942 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.938+0000 7f12671c6640 1 -- 172.21.15.26:0/2999835468 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f126019d640 con 0x7f12601036a0 2024-09-11T23:25:56.943 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.938+0000 7f125dffb640 1 -- 172.21.15.26:0/2999835468 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1254009dd0 con 0x7f12601036a0 2024-09-11T23:25:56.943 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.938+0000 7f125dffb640 1 -- 172.21.15.26:0/2999835468 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1254010460 con 0x7f12601036a0 2024-09-11T23:25:56.943 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.938+0000 7f12671c6640 1 -- 172.21.15.26:0/2999835468 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12601a4ed0 con 0x7f12601036a0 2024-09-11T23:25:56.949 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.941+0000 7f125dffb640 1 -- 172.21.15.26:0/2999835468 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f1254003f50 con 0x7f12601036a0 2024-09-11T23:25:56.949 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.941+0000 7f126a036740 1 Processor -- start 2024-09-11T23:25:56.949 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.941+0000 7f126a036740 1 -- start start 2024-09-11T23:25:56.950 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.941+0000 7f125dffb640 1 --2- 172.21.15.26:0/2999835468 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f12480a2da0 0x7f12480a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:56.950 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.941+0000 7f126a036740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55978f71baf0 0x55978f71bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:56.950 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.941+0000 7f126a036740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55978f71c410 0x55978f726770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:56.950 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.941+0000 7f126a036740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55978f726cb0 0x55978f7290a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:56.950 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.941+0000 7f126a036740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55978f6802f0 con 0x55978f71baf0 2024-09-11T23:25:56.950 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.941+0000 7f126a036740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55978f628270 con 0x55978f726cb0 2024-09-11T23:25:56.950 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.941+0000 7f126a036740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55978f67c2b0 con 0x55978f71c410 2024-09-11T23:25:56.950 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.942+0000 7f1265763640 1 --2- 172.21.15.26:0/2999835468 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f12480a2da0 0x7f12480a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:56.950 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.942+0000 7f1264f62640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55978f71baf0 0x55978f71bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:56.950 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.942+0000 7f1264f62640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55978f71baf0 0x55978f71bed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40324/0 (socket says 172.21.15.26:40324) 2024-09-11T23:25:56.950 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.942+0000 7f1264f62640 1 -- 172.21.15.26:0/1404335931 learned_addr learned my addr 172.21.15.26:0/1404335931 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:25:56.950 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.942+0000 7f125dffb640 1 -- 172.21.15.26:0/2999835468 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8542+0+0 (secure 0 0 0) 0x7f12540cf3d0 con 0x7f12601036a0 2024-09-11T23:25:56.951 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.942+0000 7f125ffff640 1 --2- 172.21.15.26:0/1404335931 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55978f71c410 0x55978f726770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:56.951 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.942+0000 7f1265763640 1 --2- 172.21.15.26:0/1404335931 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55978f726cb0 0x55978f7290a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:56.951 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.942+0000 7f1264f62640 1 -- 172.21.15.26:0/1404335931 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55978f71c410 msgr2=0x55978f726770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:25:56.951 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.942+0000 7f1264f62640 1 --2- 172.21.15.26:0/1404335931 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55978f71c410 0x55978f726770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:56.951 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.942+0000 7f1264f62640 1 -- 172.21.15.26:0/1404335931 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55978f726cb0 msgr2=0x55978f7290a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:25:56.951 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.942+0000 7f1264f62640 1 --2- 172.21.15.26:0/1404335931 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55978f726cb0 0x55978f7290a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:56.951 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.942+0000 7f1264f62640 1 -- 172.21.15.26:0/1404335931 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55978f627340 con 0x55978f71baf0 2024-09-11T23:25:56.951 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.942+0000 7f1265763640 1 --2- 172.21.15.26:0/2999835468 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f12480a2da0 0x7f12480a5260 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f1258005fd0 tx=0x7f1258014830 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:25:56.951 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.942+0000 7f1264f62640 1 --2- 172.21.15.26:0/1404335931 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55978f71baf0 0x55978f71bed0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f1250031550 tx=0x7f1250031630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93a08210cef6aee3 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:25:56.951 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.942+0000 7f1246ffd640 1 -- 172.21.15.26:0/1404335931 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1250040070 con 0x55978f71baf0 2024-09-11T23:25:56.951 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.942+0000 7f1246ffd640 1 -- 172.21.15.26:0/1404335931 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1250005040 con 0x55978f71baf0 2024-09-11T23:25:56.951 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.943+0000 7f1246ffd640 1 -- 172.21.15.26:0/1404335931 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1250035040 con 0x55978f71baf0 2024-09-11T23:25:56.951 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.943+0000 7f126a036740 1 -- 172.21.15.26:0/1404335931 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55978f71baf0 msgr2=0x55978f71bed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:25:56.951 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.943+0000 7f126a036740 1 --2- 172.21.15.26:0/1404335931 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55978f71baf0 0x55978f71bed0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f1250031550 tx=0x7f1250031630 comp rx=0 tx=0).stop 2024-09-11T23:25:56.952 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.943+0000 7f126a036740 1 -- 172.21.15.26:0/1404335931 shutdown_connections 2024-09-11T23:25:56.952 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.943+0000 7f126a036740 1 --2- 172.21.15.26:0/1404335931 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55978f726cb0 0x55978f7290a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:56.952 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.943+0000 7f126a036740 1 --2- 172.21.15.26:0/1404335931 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55978f71c410 0x55978f726770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:56.952 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.943+0000 7f126a036740 1 --2- 172.21.15.26:0/1404335931 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55978f71baf0 0x55978f71bed0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:56.952 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.943+0000 7f126a036740 1 -- 172.21.15.26:0/1404335931 >> 172.21.15.26:0/1404335931 conn(0x55978f718060 msgr2=0x55978f718440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:25:56.952 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.943+0000 7f126a036740 1 -- 172.21.15.26:0/1404335931 shutdown_connections 2024-09-11T23:25:56.952 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.943+0000 7f126a036740 1 -- 172.21.15.26:0/1404335931 wait complete. 2024-09-11T23:25:56.952 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.944+0000 7f126a036740 1 -- start start 2024-09-11T23:25:56.952 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.944+0000 7f126a036740 1 Processor -- start 2024-09-11T23:25:56.952 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.944+0000 7f126a036740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55978f71baf0 0x55978f723b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:56.952 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.944+0000 7f126a036740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55978f71c410 0x55978f7240d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:56.953 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.944+0000 7f126a036740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55978f726cb0 0x55978f724610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:56.953 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.944+0000 7f126a036740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55978f729880 con 0x55978f71c410 2024-09-11T23:25:56.953 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.944+0000 7f126a036740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55978f609290 con 0x55978f71baf0 2024-09-11T23:25:56.953 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.944+0000 7f126a036740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55978f729a00 con 0x55978f726cb0 2024-09-11T23:25:56.953 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.944+0000 7f126a036740 10 client.0 _ll_register_callbacks cb 0x55978f63a140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:25:56.953 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.944+0000 7f125ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55978f71c410 0x55978f7240d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:56.953 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.944+0000 7f125ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55978f71c410 0x55978f7240d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40338/0 (socket says 172.21.15.26:40338) 2024-09-11T23:25:56.953 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.944+0000 7f125ffff640 1 -- 172.21.15.26:0/219271637 learned_addr learned my addr 172.21.15.26:0/219271637 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:25:56.953 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.945+0000 7f1265763640 1 --2- 172.21.15.26:0/219271637 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55978f726cb0 0x55978f724610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:56.953 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.945+0000 7f1264f62640 1 --2- 172.21.15.26:0/219271637 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55978f71baf0 0x55978f723b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:56.953 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.945+0000 7f125ffff640 1 -- 172.21.15.26:0/219271637 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55978f726cb0 msgr2=0x55978f724610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:25:56.953 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.945+0000 7f125ffff640 1 --2- 172.21.15.26:0/219271637 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55978f726cb0 0x55978f724610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:56.954 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.945+0000 7f125ffff640 1 -- 172.21.15.26:0/219271637 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55978f71baf0 msgr2=0x55978f723b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:25:56.954 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.945+0000 7f125ffff640 1 --2- 172.21.15.26:0/219271637 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55978f71baf0 0x55978f723b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:56.954 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.945+0000 7f125ffff640 1 -- 172.21.15.26:0/219271637 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55978f393210 con 0x55978f71c410 2024-09-11T23:25:56.954 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.945+0000 7f1264f62640 1 --2- 172.21.15.26:0/219271637 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55978f71baf0 0x55978f723b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:25:56.954 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.945+0000 7f125ffff640 1 --2- 172.21.15.26:0/219271637 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55978f71c410 0x55978f7240d0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f12540ccb70 tx=0x7f125409b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8984f192c852df server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:25:56.954 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.945+0000 7f123ffff640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:25:56.954 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.945+0000 7f123ffff640 1 -- 172.21.15.26:0/219271637 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1254020070 con 0x55978f71c410 2024-09-11T23:25:56.954 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.945+0000 7f123ffff640 1 -- 172.21.15.26:0/219271637 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8542+0+0 (secure 0 0 0) 0x7f12540b71d0 con 0x55978f71c410 2024-09-11T23:25:56.954 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.945+0000 7f126a036740 1 -- 172.21.15.26:0/219271637 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1250005040 con 0x55978f71c410 2024-09-11T23:25:56.954 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.945+0000 7f123ffff640 1 -- 172.21.15.26:0/219271637 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f12540ccd90 con 0x55978f71c410 2024-09-11T23:25:56.954 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.945+0000 7f126a036740 10 client.16617 fetch_fsmap learned FSMap version 47 2024-09-11T23:25:56.954 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.945+0000 7f126a036740 1 -- 172.21.15.26:0/219271637 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55978f45f620 con 0x55978f71c410 2024-09-11T23:25:56.954 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.946+0000 7f123ffff640 1 -- 172.21.15.26:0/219271637 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1250005040 con 0x55978f71c410 2024-09-11T23:25:56.954 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.946+0000 7f123ffff640 1 -- 172.21.15.26:0/219271637 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 47) ==== 8414+0+0 (secure 0 0 0) 0x7f1254097f50 con 0x55978f71c410 2024-09-11T23:25:56.955 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.946+0000 7f126a036740 10 client.16617 fetch_fsmap finished waiting for FSMap version 47 2024-09-11T23:25:56.955 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.946+0000 7f126a036740 10 client.16617 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:25:56.955 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.946+0000 7f126a036740 20 client.16617 populate_metadata read hostname 'smithi026' 2024-09-11T23:25:56.955 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.946+0000 7f126a036740 1 --2- 172.21.15.26:0/219271637 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55978f736c00 0x55978f739050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:56.955 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.946+0000 7f126a036740 4 client.16617 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:25:56.955 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.946+0000 7f126a036740 1 -- 172.21.15.26:0/219271637 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55978f68da20 con 0x55978f736c00 2024-09-11T23:25:56.955 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.946+0000 7f1264f62640 1 --2- 172.21.15.26:0/219271637 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55978f736c00 0x55978f739050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:56.955 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.947+0000 7f123ffff640 10 client.16617 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:25:56.955 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.947+0000 7f1264f62640 1 --2- 172.21.15.26:0/219271637 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55978f736c00 0x55978f739050 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d6a7c1bde4a5a798 server_cookie=f263408a5abb524d in_seq=0 out_seq=0 2024-09-11T23:25:56.955 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.947+0000 7f123ffff640 1 -- 172.21.15.26:0/219271637 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55978f68da20 con 0x55978f736c00 2024-09-11T23:25:56.955 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.947+0000 7f123ffff640 10 client.16617 handle_command_reply: tid=0 2024-09-11T23:25:56.955 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.947+0000 7f126a036740 1 client.16617 shutdown 2024-09-11T23:25:56.955 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.948+0000 7f126a036740 1 -- 172.21.15.26:0/219271637 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55978f71c410 msgr2=0x55978f7240d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:25:56.955 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.948+0000 7f126a036740 1 --2- 172.21.15.26:0/219271637 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55978f71c410 0x55978f7240d0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f12540ccb70 tx=0x7f125409b9a0 comp rx=0 tx=0).stop 2024-09-11T23:25:56.956 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.948+0000 7f126a036740 1 -- 172.21.15.26:0/219271637 shutdown_connections 2024-09-11T23:25:56.956 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.948+0000 7f126a036740 1 --2- 172.21.15.26:0/219271637 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55978f736c00 0x55978f739050 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:56.956 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.948+0000 7f126a036740 1 --2- 172.21.15.26:0/219271637 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55978f726cb0 0x55978f724610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:56.956 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.948+0000 7f126a036740 1 --2- 172.21.15.26:0/219271637 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55978f71c410 0x55978f7240d0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:56.956 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.948+0000 7f123ffff640 0 client.16617 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:25:56.956 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.948+0000 7f126a036740 1 --2- 172.21.15.26:0/219271637 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55978f71baf0 0x55978f723b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:56.956 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.948+0000 7f126a036740 1 -- 172.21.15.26:0/219271637 >> 172.21.15.26:0/219271637 conn(0x55978f718060 msgr2=0x55978f71dd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:25:56.956 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.948+0000 7f126a036740 1 -- 172.21.15.26:0/219271637 shutdown_connections 2024-09-11T23:25:56.956 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.948+0000 7f126a036740 1 -- 172.21.15.26:0/219271637 wait complete. 2024-09-11T23:25:56.956 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.948+0000 7f126a036740 20 client.16617 trim_cache size 0 max 16384 2024-09-11T23:25:56.968 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.963+0000 7f126a036740 1 Processor -- start 2024-09-11T23:25:56.968 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.963+0000 7f126a036740 1 -- start start 2024-09-11T23:25:56.968 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.963+0000 7f126a036740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55978f71baf0 0x55978f724440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:56.968 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.963+0000 7f126a036740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55978f71c410 0x55978f73e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:56.968 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.963+0000 7f126a036740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55978f726cb0 0x55978f73eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:56.968 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.963+0000 7f126a036740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f12601ad870 con 0x55978f726cb0 2024-09-11T23:25:56.968 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.963+0000 7f126a036740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f12601af620 con 0x55978f71baf0 2024-09-11T23:25:56.968 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.963+0000 7f126a036740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f12601ae000 con 0x55978f71c410 2024-09-11T23:25:56.969 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.963+0000 7f1265763640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55978f726cb0 0x55978f73eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:56.969 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.963+0000 7f1264f62640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55978f71baf0 0x55978f724440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:56.969 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.963+0000 7f1265763640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55978f726cb0 0x55978f73eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40348/0 (socket says 172.21.15.26:40348) 2024-09-11T23:25:56.969 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.963+0000 7f1265763640 1 -- 172.21.15.26:0/2888672294 learned_addr learned my addr 172.21.15.26:0/2888672294 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:25:56.969 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.963+0000 7f125ffff640 1 --2- 172.21.15.26:0/2888672294 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55978f71c410 0x55978f73e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:56.969 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.963+0000 7f1265763640 1 -- 172.21.15.26:0/2888672294 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55978f71c410 msgr2=0x55978f73e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:25:56.969 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.963+0000 7f1265763640 1 --2- 172.21.15.26:0/2888672294 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55978f71c410 0x55978f73e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:56.969 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.963+0000 7f1265763640 1 -- 172.21.15.26:0/2888672294 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55978f71baf0 msgr2=0x55978f724440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:25:56.969 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.963+0000 7f1265763640 1 --2- 172.21.15.26:0/2888672294 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55978f71baf0 0x55978f724440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:56.969 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.963+0000 7f1265763640 1 -- 172.21.15.26:0/2888672294 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55978f7396d0 con 0x55978f726cb0 2024-09-11T23:25:56.969 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.963+0000 7f125ffff640 1 --2- 172.21.15.26:0/2888672294 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55978f71c410 0x55978f73e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:25:56.969 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.964+0000 7f1265763640 1 --2- 172.21.15.26:0/2888672294 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55978f726cb0 0x55978f73eef0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f12580124e0 tx=0x7f1258017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5a12ce4fa92f0b1 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:25:56.969 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.964+0000 7f12467fc640 1 -- 172.21.15.26:0/2888672294 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1258020700 con 0x55978f726cb0 2024-09-11T23:25:56.969 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.964+0000 7f12467fc640 1 -- 172.21.15.26:0/2888672294 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1258020d80 con 0x55978f726cb0 2024-09-11T23:25:56.970 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.964+0000 7f12467fc640 1 -- 172.21.15.26:0/2888672294 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f125801f440 con 0x55978f726cb0 2024-09-11T23:25:56.970 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.964+0000 7f126a036740 1 -- 172.21.15.26:0/2888672294 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55978f726cb0 msgr2=0x55978f73eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:25:56.970 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.964+0000 7f126a036740 1 --2- 172.21.15.26:0/2888672294 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55978f726cb0 0x55978f73eef0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f12580124e0 tx=0x7f1258017000 comp rx=0 tx=0).stop 2024-09-11T23:25:56.970 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.964+0000 7f126a036740 1 -- 172.21.15.26:0/2888672294 shutdown_connections 2024-09-11T23:25:56.970 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.964+0000 7f126a036740 1 --2- 172.21.15.26:0/2888672294 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55978f726cb0 0x55978f73eef0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:56.970 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.964+0000 7f126a036740 1 --2- 172.21.15.26:0/2888672294 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55978f71c410 0x55978f73e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:56.970 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.964+0000 7f126a036740 1 --2- 172.21.15.26:0/2888672294 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55978f71baf0 0x55978f724440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:56.970 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.964+0000 7f126a036740 1 -- 172.21.15.26:0/2888672294 >> 172.21.15.26:0/2888672294 conn(0x55978f718060 msgr2=0x55978f725c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:25:56.970 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.964+0000 7f126a036740 1 -- 172.21.15.26:0/2888672294 shutdown_connections 2024-09-11T23:25:56.970 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.964+0000 7f126a036740 1 -- 172.21.15.26:0/2888672294 wait complete. 2024-09-11T23:25:56.970 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.965+0000 7f126a036740 1 -- start start 2024-09-11T23:25:56.970 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.965+0000 7f126a036740 1 Processor -- start 2024-09-11T23:25:56.970 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.965+0000 7f126a036740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55978f71baf0 0x55978f72aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:56.971 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.965+0000 7f126a036740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55978f71c410 0x55978f72b030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:56.971 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.965+0000 7f126a036740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55978f726cb0 0x55978f72b570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:56.971 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.965+0000 7f126a036740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f12601aef20 con 0x55978f726cb0 2024-09-11T23:25:56.971 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.965+0000 7f126a036740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f12601ae790 con 0x55978f71baf0 2024-09-11T23:25:56.971 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.965+0000 7f126a036740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f12601ab870 con 0x55978f71c410 2024-09-11T23:25:56.971 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.965+0000 7f126a036740 10 client.0 _ll_register_callbacks cb 0x7f125409b770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:25:56.971 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.965+0000 7f1265763640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55978f726cb0 0x55978f72b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:56.971 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.965+0000 7f1265763640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55978f726cb0 0x55978f72b570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40362/0 (socket says 172.21.15.26:40362) 2024-09-11T23:25:56.971 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.965+0000 7f1265763640 1 -- 172.21.15.26:0/569409112 learned_addr learned my addr 172.21.15.26:0/569409112 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:25:56.971 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.965+0000 7f1264f62640 1 --2- 172.21.15.26:0/569409112 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55978f71baf0 0x55978f72aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:56.971 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.966+0000 7f125ffff640 1 --2- 172.21.15.26:0/569409112 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55978f71c410 0x55978f72b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:56.971 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.966+0000 7f1265763640 1 -- 172.21.15.26:0/569409112 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55978f71c410 msgr2=0x55978f72b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:25:56.971 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.966+0000 7f1265763640 1 --2- 172.21.15.26:0/569409112 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55978f71c410 0x55978f72b030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:56.971 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.966+0000 7f1265763640 1 -- 172.21.15.26:0/569409112 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55978f71baf0 msgr2=0x55978f72aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:25:56.972 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.966+0000 7f1265763640 1 --2- 172.21.15.26:0/569409112 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55978f71baf0 0x55978f72aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:56.972 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.966+0000 7f1265763640 1 -- 172.21.15.26:0/569409112 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55978f7a0ae0 con 0x55978f726cb0 2024-09-11T23:25:56.972 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.966+0000 7f1265763640 1 --2- 172.21.15.26:0/569409112 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55978f726cb0 0x55978f72b570 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f125800a010 tx=0x7f125802ce50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2436ba71793aa1c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:25:56.972 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.966+0000 7f123ffff640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:25:56.972 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.966+0000 7f123ffff640 1 -- 172.21.15.26:0/569409112 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f125803b060 con 0x55978f726cb0 2024-09-11T23:25:56.972 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.966+0000 7f123ffff640 1 -- 172.21.15.26:0/569409112 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8542+0+0 (secure 0 0 0) 0x7f125802b680 con 0x55978f726cb0 2024-09-11T23:25:56.972 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.966+0000 7f126a036740 1 -- 172.21.15.26:0/569409112 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1258020d80 con 0x55978f726cb0 2024-09-11T23:25:56.972 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.966+0000 7f123ffff640 1 -- 172.21.15.26:0/569409112 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f1258004a40 con 0x55978f726cb0 2024-09-11T23:25:56.972 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.966+0000 7f126a036740 10 client.16623 fetch_fsmap learned FSMap version 47 2024-09-11T23:25:56.972 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.966+0000 7f126a036740 1 -- 172.21.15.26:0/569409112 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55978f7a1d70 con 0x55978f726cb0 2024-09-11T23:25:56.972 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.967+0000 7f123ffff640 1 -- 172.21.15.26:0/569409112 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1258020d80 con 0x55978f726cb0 2024-09-11T23:25:56.972 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.967+0000 7f123ffff640 1 -- 172.21.15.26:0/569409112 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 47) ==== 8414+0+0 (secure 0 0 0) 0x7f125800f040 con 0x55978f726cb0 2024-09-11T23:25:56.972 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.967+0000 7f126a036740 10 client.16623 fetch_fsmap finished waiting for FSMap version 47 2024-09-11T23:25:56.973 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.967+0000 7f126a036740 10 client.16623 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:25:56.973 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.967+0000 7f126a036740 20 client.16623 populate_metadata read hostname 'smithi026' 2024-09-11T23:25:56.973 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.967+0000 7f126a036740 1 --2- 172.21.15.26:0/569409112 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55978f7a2410 0x55978f7a4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:56.973 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.967+0000 7f126a036740 4 client.16623 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T23:25:56.973 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.967+0000 7f126a036740 1 -- 172.21.15.26:0/569409112 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55978f687d70 con 0x55978f7a2410 2024-09-11T23:25:56.973 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.967+0000 7f1264f62640 1 --2- 172.21.15.26:0/569409112 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55978f7a2410 0x55978f7a4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:56.973 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.967+0000 7f123ffff640 10 client.16623 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:25:56.973 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.967+0000 7f1264f62640 1 --2- 172.21.15.26:0/569409112 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55978f7a2410 0x55978f7a4830 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a640af392868414 server_cookie=398bbb3b23361ba in_seq=0 out_seq=0 2024-09-11T23:25:56.988 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.986+0000 7f123ffff640 1 -- 172.21.15.26:0/569409112 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55978f687d70 con 0x55978f7a2410 2024-09-11T23:25:56.988 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.986+0000 7f123ffff640 10 client.16623 handle_command_reply: tid=0 2024-09-11T23:25:56.988 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.986+0000 7f126a036740 1 client.16623 shutdown 2024-09-11T23:25:56.988 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.986+0000 7f126a036740 1 -- 172.21.15.26:0/569409112 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55978f726cb0 msgr2=0x55978f72b570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:25:56.988 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.986+0000 7f126a036740 1 --2- 172.21.15.26:0/569409112 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55978f726cb0 0x55978f72b570 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f125800a010 tx=0x7f125802ce50 comp rx=0 tx=0).stop 2024-09-11T23:25:56.988 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.986+0000 7f126a036740 1 -- 172.21.15.26:0/569409112 shutdown_connections 2024-09-11T23:25:56.988 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.986+0000 7f126a036740 1 --2- 172.21.15.26:0/569409112 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55978f7a2410 0x55978f7a4830 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:56.988 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.986+0000 7f126a036740 1 --2- 172.21.15.26:0/569409112 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55978f726cb0 0x55978f72b570 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:56.988 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.986+0000 7f126a036740 1 --2- 172.21.15.26:0/569409112 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55978f71c410 0x55978f72b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:56.988 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.986+0000 7f126a036740 1 --2- 172.21.15.26:0/569409112 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55978f71baf0 0x55978f72aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:56.988 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.986+0000 7f123ffff640 0 client.16623 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:25:56.989 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.986+0000 7f126a036740 1 -- 172.21.15.26:0/569409112 >> 172.21.15.26:0/569409112 conn(0x55978f718060 msgr2=0x55978f71d550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:25:56.989 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.987+0000 7f126a036740 1 -- 172.21.15.26:0/569409112 shutdown_connections 2024-09-11T23:25:56.989 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.987+0000 7f126a036740 1 -- 172.21.15.26:0/569409112 wait complete. 2024-09-11T23:25:56.989 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.987+0000 7f126a036740 20 client.16623 trim_cache size 0 max 16384 2024-09-11T23:25:56.989 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-09-11T23:25:56.989 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "message": "", 2024-09-11T23:25:56.989 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-09-11T23:25:56.989 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-09-11T23:25:56.990 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.987+0000 7f12671c6640 1 -- 172.21.15.26:0/2999835468 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f12480a2da0 msgr2=0x7f12480a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:25:56.990 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.987+0000 7f12671c6640 1 --2- 172.21.15.26:0/2999835468 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f12480a2da0 0x7f12480a5260 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f1258005fd0 tx=0x7f1258014830 comp rx=0 tx=0).stop 2024-09-11T23:25:56.990 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.987+0000 7f12671c6640 1 -- 172.21.15.26:0/2999835468 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12601036a0 msgr2=0x7f12601988a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:25:56.990 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.987+0000 7f12671c6640 1 --2- 172.21.15.26:0/2999835468 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12601036a0 0x7f12601988a0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f1254005fd0 tx=0x7f125400dde0 comp rx=0 tx=0).stop 2024-09-11T23:25:56.990 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.987+0000 7f12671c6640 1 -- 172.21.15.26:0/2999835468 shutdown_connections 2024-09-11T23:25:56.990 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.987+0000 7f12671c6640 1 --2- 172.21.15.26:0/2999835468 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f12480a2da0 0x7f12480a5260 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:56.990 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.987+0000 7f12671c6640 1 --2- 172.21.15.26:0/2999835468 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f126010c150 0x7f126019cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:56.990 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.987+0000 7f12671c6640 1 --2- 172.21.15.26:0/2999835468 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12601036a0 0x7f12601988a0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:56.991 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.987+0000 7f12671c6640 1 --2- 172.21.15.26:0/2999835468 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1260102d80 0x7f1260198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:56.991 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.987+0000 7f12671c6640 1 -- 172.21.15.26:0/2999835468 >> 172.21.15.26:0/2999835468 conn(0x7f12600fc6b0 msgr2=0x7f12600fd310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:25:56.991 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.988+0000 7f12671c6640 1 -- 172.21.15.26:0/2999835468 shutdown_connections 2024-09-11T23:25:56.991 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:25:56.988+0000 7f12671c6640 1 -- 172.21.15.26:0/2999835468 wait complete. 2024-09-11T23:25:58.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:25:57 smithi076 ceph-mon[35296]: pgmap v232: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 219 KiB/s rd, 729 KiB/s wr, 67 op/s 2024-09-11T23:25:58.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:25:57 smithi136 ceph-mon[35121]: pgmap v232: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 219 KiB/s rd, 729 KiB/s wr, 67 op/s 2024-09-11T23:25:58.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:25:57 smithi026 ceph-mon[31172]: pgmap v232: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 219 KiB/s rd, 729 KiB/s wr, 67 op/s 2024-09-11T23:25:58.285 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 16.6 2024-09-11T23:25:58.286 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:25:58.559 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:25:59.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:59.045+0000 7f78cde85640 1 -- 172.21.15.26:0/3377755424 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f78c8102dd0 msgr2=0x7f78c81031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:25:59.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:59.045+0000 7f78cde85640 1 --2- 172.21.15.26:0/3377755424 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f78c8102dd0 0x7f78c81031b0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f78b8009930 tx=0x7f78b802a420 comp rx=0 tx=0).stop 2024-09-11T23:25:59.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:59.045+0000 7f78cde85640 1 -- 172.21.15.26:0/3377755424 shutdown_connections 2024-09-11T23:25:59.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:59.045+0000 7f78cde85640 1 --2- 172.21.15.26:0/3377755424 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f78c8103780 0x7f78c8107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:59.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:59.045+0000 7f78cde85640 1 --2- 172.21.15.26:0/3377755424 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f78c8108300 0x7f78c810ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:59.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:59.045+0000 7f78cde85640 1 --2- 172.21.15.26:0/3377755424 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f78c8102dd0 0x7f78c81031b0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:59.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:59.045+0000 7f78cde85640 1 -- 172.21.15.26:0/3377755424 >> 172.21.15.26:0/3377755424 conn(0x7f78c80fc7d0 msgr2=0x7f78c80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:25:59.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:59.046+0000 7f78cde85640 1 -- 172.21.15.26:0/3377755424 shutdown_connections 2024-09-11T23:25:59.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:59.046+0000 7f78cde85640 1 -- 172.21.15.26:0/3377755424 wait complete. 2024-09-11T23:25:59.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:59.046+0000 7f78cde85640 1 Processor -- start 2024-09-11T23:25:59.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:59.046+0000 7f78cde85640 1 -- start start 2024-09-11T23:25:59.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:59.047+0000 7f78cde85640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f78c8102dd0 0x7f78c8198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:59.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:59.047+0000 7f78cde85640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f78c8103780 0x7f78c8198bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:59.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:59.047+0000 7f78cde85640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f78c8108300 0x7f78c819cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:59.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:59.047+0000 7f78cde85640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f78c81120b0 con 0x7f78c8103780 2024-09-11T23:25:59.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:59.047+0000 7f78cde85640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f78c8111f30 con 0x7f78c8102dd0 2024-09-11T23:25:59.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:59.047+0000 7f78cde85640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f78c8112230 con 0x7f78c8108300 2024-09-11T23:25:59.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:59.047+0000 7f78c6ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f78c8103780 0x7f78c8198bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:59.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:59.047+0000 7f78c77fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f78c8102dd0 0x7f78c8198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:59.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:59.047+0000 7f78c6ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f78c8103780 0x7f78c8198bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40382/0 (socket says 172.21.15.26:40382) 2024-09-11T23:25:59.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:59.047+0000 7f78c77fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f78c8102dd0 0x7f78c8198670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:55694/0 (socket says 172.21.15.26:55694) 2024-09-11T23:25:59.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:59.047+0000 7f78c6ffd640 1 -- 172.21.15.26:0/2391139811 learned_addr learned my addr 172.21.15.26:0/2391139811 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:25:59.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:59.047+0000 7f78c7fff640 1 --2- 172.21.15.26:0/2391139811 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f78c8108300 0x7f78c819cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:59.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:59.047+0000 7f78c6ffd640 1 -- 172.21.15.26:0/2391139811 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f78c8108300 msgr2=0x7f78c819cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:25:59.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:59.047+0000 7f78c6ffd640 1 --2- 172.21.15.26:0/2391139811 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f78c8108300 0x7f78c819cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:59.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:59.047+0000 7f78c6ffd640 1 -- 172.21.15.26:0/2391139811 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f78c8102dd0 msgr2=0x7f78c8198670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:25:59.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:59.047+0000 7f78c6ffd640 1 --2- 172.21.15.26:0/2391139811 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f78c8102dd0 0x7f78c8198670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:59.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:59.047+0000 7f78c6ffd640 1 -- 172.21.15.26:0/2391139811 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78c819d620 con 0x7f78c8103780 2024-09-11T23:25:59.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:59.047+0000 7f78c7fff640 1 --2- 172.21.15.26:0/2391139811 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f78c8108300 0x7f78c819cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:25:59.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:59.047+0000 7f78c6ffd640 1 --2- 172.21.15.26:0/2391139811 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f78c8103780 0x7f78c8198bb0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f78b000d950 tx=0x7f78b000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:25:59.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:59.047+0000 7f78c4ff9640 1 -- 172.21.15.26:0/2391139811 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78b0017070 con 0x7f78c8103780 2024-09-11T23:25:59.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:59.047+0000 7f78c4ff9640 1 -- 172.21.15.26:0/2391139811 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f78b0009c30 con 0x7f78c8103780 2024-09-11T23:25:59.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:59.047+0000 7f78cde85640 1 -- 172.21.15.26:0/2391139811 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78c819d910 con 0x7f78c8103780 2024-09-11T23:25:59.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:59.047+0000 7f78c4ff9640 1 -- 172.21.15.26:0/2391139811 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78b00047a0 con 0x7f78c8103780 2024-09-11T23:25:59.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:59.048+0000 7f78cde85640 1 -- 172.21.15.26:0/2391139811 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78c81a51f0 con 0x7f78c8103780 2024-09-11T23:25:59.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:59.048+0000 7f78cde85640 1 -- 172.21.15.26:0/2391139811 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78c806a7b0 con 0x7f78c8103780 2024-09-11T23:25:59.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:59.050+0000 7f78c4ff9640 1 -- 172.21.15.26:0/2391139811 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f78b0004020 con 0x7f78c8103780 2024-09-11T23:25:59.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:59.051+0000 7f78c4ff9640 1 --2- 172.21.15.26:0/2391139811 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f78b40a2cf0 0x7f78b40a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:59.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:59.051+0000 7f78c77fe640 1 --2- 172.21.15.26:0/2391139811 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f78b40a2cf0 0x7f78b40a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:59.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:59.051+0000 7f78c4ff9640 1 -- 172.21.15.26:0/2391139811 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8542+0+0 (secure 0 0 0) 0x7f78b00cf0d0 con 0x7f78c8103780 2024-09-11T23:25:59.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:59.051+0000 7f78c77fe640 1 --2- 172.21.15.26:0/2391139811 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f78b40a2cf0 0x7f78b40a51b0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f78b8009900 tx=0x7f78b80058d0 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:25:59.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:59.055+0000 7f78c4ff9640 1 -- 172.21.15.26:0/2391139811 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f78b0020020 con 0x7f78c8103780 2024-09-11T23:25:59.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:59.270+0000 7f78cde85640 1 -- 172.21.15.26:0/2391139811 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f78c810a290 con 0x7f78c8103780 2024-09-11T23:25:59.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:59.270+0000 7f78c4ff9640 1 -- 172.21.15.26:0/2391139811 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) ==== 94+0+11994 (secure 0 0 0) 0x7f78b009bce0 con 0x7f78c8103780 2024-09-11T23:25:59.272 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-11T23:25:59.272 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":47,"btime":"2024-09-11T23:25:46:746180+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14499,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/367659515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":367659515},{"type":"v1","addr":"172.21.15.26:6835","nonce":367659515}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14511,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/2041065299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":2041065299},{"type":"v1","addr":"172.21.15.26:6837","nonce":2041065299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":47,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:16:26.544203+0000","modified":"2024-09-11T23:25:46.511698+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24485,"mds_1":24463,"mds_2":24458,"mds_3":24448,"mds_4":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14550":{"gid":14550,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6839/2233068305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":2233068305},{"type":"v1","addr":"172.21.15.26:6839","nonce":2233068305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14562":{"gid":14562,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6841/2984982336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":2984982336},{"type":"v1","addr":"172.21.15.26:6841","nonce":2984982336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6833/2019841455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":2019841455},{"type":"v1","addr":"172.21.15.136:6833","nonce":2019841455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.76:6833/3438147050","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3438147050},{"type":"v1","addr":"172.21.15.76:6833","nonce":3438147050}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6835/3617665975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3617665975},{"type":"v1","addr":"172.21.15.136:6835","nonce":3617665975}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":12,"addr":"172.21.15.76:6835/393308499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":393308499},{"type":"v1","addr":"172.21.15.76:6835","nonce":393308499}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":10,"addr":"172.21.15.136:6837/1633082387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":1633082387},{"type":"v1","addr":"172.21.15.136:6837","nonce":1633082387}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.76:6837/2078978748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":2078978748},{"type":"v1","addr":"172.21.15.76:6837","nonce":2078978748}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":63,"addr":"172.21.15.136:6839/3211398630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":3211398630},{"type":"v1","addr":"172.21.15.136:6839","nonce":3211398630}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.76:6839/2077637090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6838","nonce":2077637090},{"type":"v1","addr":"172.21.15.76:6839","nonce":2077637090}]},"join_fscid":-1,"export_targets":[3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24485,"qdb_cluster":[24443,24448,24458,24463,24485]},"id":1}]} 2024-09-11T23:25:59.272 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 47 2024-09-11T23:25:59.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:59.273+0000 7f78cde85640 1 -- 172.21.15.26:0/2391139811 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f78b40a2cf0 msgr2=0x7f78b40a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:25:59.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:59.273+0000 7f78cde85640 1 --2- 172.21.15.26:0/2391139811 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f78b40a2cf0 0x7f78b40a51b0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f78b8009900 tx=0x7f78b80058d0 comp rx=0 tx=0).stop 2024-09-11T23:25:59.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:59.273+0000 7f78cde85640 1 -- 172.21.15.26:0/2391139811 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f78c8103780 msgr2=0x7f78c8198bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:25:59.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:59.273+0000 7f78cde85640 1 --2- 172.21.15.26:0/2391139811 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f78c8103780 0x7f78c8198bb0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f78b000d950 tx=0x7f78b000de10 comp rx=0 tx=0).stop 2024-09-11T23:25:59.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:59.273+0000 7f78cde85640 1 -- 172.21.15.26:0/2391139811 shutdown_connections 2024-09-11T23:25:59.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:59.273+0000 7f78cde85640 1 --2- 172.21.15.26:0/2391139811 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f78b40a2cf0 0x7f78b40a51b0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:59.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:59.273+0000 7f78cde85640 1 --2- 172.21.15.26:0/2391139811 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f78c8108300 0x7f78c819cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:59.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:59.273+0000 7f78cde85640 1 --2- 172.21.15.26:0/2391139811 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f78c8103780 0x7f78c8198bb0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:59.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:59.273+0000 7f78cde85640 1 --2- 172.21.15.26:0/2391139811 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f78c8102dd0 0x7f78c8198670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:59.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:59.273+0000 7f78cde85640 1 -- 172.21.15.26:0/2391139811 >> 172.21.15.26:0/2391139811 conn(0x7f78c80fc7d0 msgr2=0x7f78c8105e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:25:59.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:59.273+0000 7f78cde85640 1 -- 172.21.15.26:0/2391139811 shutdown_connections 2024-09-11T23:25:59.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:25:59.273+0000 7f78cde85640 1 -- 172.21.15.26:0/2391139811 wait complete. 2024-09-11T23:26:00.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:25:59 smithi026 ceph-mon[31172]: pgmap v233: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 469 KiB/s wr, 48 op/s 2024-09-11T23:26:00.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:25:59 smithi026 ceph-mon[31172]: from='client.? 172.21.15.26:0/2391139811' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:26:00.052 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:25:59 smithi076 ceph-mon[35296]: pgmap v233: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 469 KiB/s wr, 48 op/s 2024-09-11T23:26:00.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:25:59 smithi076 ceph-mon[35296]: from='client.? 172.21.15.26:0/2391139811' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:26:00.238 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24485 quiesce db / --timeout 90 --expiration 136.6 --await-for 45.0' 2024-09-11T23:26:00.238 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24485 quiesce db / --timeout 90 --expiration 136.6 --await-for 45.0 2024-09-11T23:26:00.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:25:59 smithi136 ceph-mon[35121]: pgmap v233: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 469 KiB/s wr, 48 op/s 2024-09-11T23:26:00.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:25:59 smithi136 ceph-mon[35121]: from='client.? 172.21.15.26:0/2391139811' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:26:00.506 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:26:01.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.005+0000 7f2b438d9640 1 -- 172.21.15.26:0/2394500728 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b3c100a80 msgr2=0x7f2b3c100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:26:01.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.005+0000 7f2b438d9640 1 --2- 172.21.15.26:0/2394500728 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b3c100a80 0x7f2b3c100ee0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f2b2c009930 tx=0x7f2b2c02a440 comp rx=0 tx=0).stop 2024-09-11T23:26:01.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.005+0000 7f2b438d9640 1 -- 172.21.15.26:0/2394500728 shutdown_connections 2024-09-11T23:26:01.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.005+0000 7f2b438d9640 1 --2- 172.21.15.26:0/2394500728 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2b3c101420 0x7f2b3c10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:01.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.005+0000 7f2b438d9640 1 --2- 172.21.15.26:0/2394500728 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b3c100a80 0x7f2b3c100ee0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:01.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.005+0000 7f2b438d9640 1 --2- 172.21.15.26:0/2394500728 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2b3c106a80 0x7f2b3c106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:01.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.005+0000 7f2b438d9640 1 -- 172.21.15.26:0/2394500728 >> 172.21.15.26:0/2394500728 conn(0x7f2b3c0fc7a0 msgr2=0x7f2b3c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:26:01.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.005+0000 7f2b438d9640 1 -- 172.21.15.26:0/2394500728 shutdown_connections 2024-09-11T23:26:01.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.005+0000 7f2b438d9640 1 -- 172.21.15.26:0/2394500728 wait complete. 2024-09-11T23:26:01.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.006+0000 7f2b438d9640 1 Processor -- start 2024-09-11T23:26:01.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.006+0000 7f2b438d9640 1 -- start start 2024-09-11T23:26:01.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.006+0000 7f2b438d9640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2b3c100a80 0x7f2b3c199200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:01.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.006+0000 7f2b438d9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b3c101420 0x7f2b3c199740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:01.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.006+0000 7f2b438d9640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2b3c106a80 0x7f2b3c1a4280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:01.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.006+0000 7f2b438d9640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f2b3c110e40 con 0x7f2b3c101420 2024-09-11T23:26:01.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.006+0000 7f2b438d9640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f2b3c110cc0 con 0x7f2b3c100a80 2024-09-11T23:26:01.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.006+0000 7f2b438d9640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f2b3c110fc0 con 0x7f2b3c106a80 2024-09-11T23:26:01.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.006+0000 7f2b40e74640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b3c101420 0x7f2b3c199740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:01.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.007+0000 7f2b40e74640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b3c101420 0x7f2b3c199740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49618/0 (socket says 172.21.15.26:49618) 2024-09-11T23:26:01.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.007+0000 7f2b40e74640 1 -- 172.21.15.26:0/2662206115 learned_addr learned my addr 172.21.15.26:0/2662206115 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:26:01.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.007+0000 7f2b41e76640 1 --2- 172.21.15.26:0/2662206115 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2b3c106a80 0x7f2b3c1a4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:01.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.007+0000 7f2b41675640 1 --2- 172.21.15.26:0/2662206115 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2b3c100a80 0x7f2b3c199200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:01.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.007+0000 7f2b40e74640 1 -- 172.21.15.26:0/2662206115 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2b3c106a80 msgr2=0x7f2b3c1a4280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:26:01.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.007+0000 7f2b40e74640 1 --2- 172.21.15.26:0/2662206115 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2b3c106a80 0x7f2b3c1a4280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:01.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.007+0000 7f2b40e74640 1 -- 172.21.15.26:0/2662206115 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2b3c100a80 msgr2=0x7f2b3c199200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:26:01.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.007+0000 7f2b40e74640 1 --2- 172.21.15.26:0/2662206115 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2b3c100a80 0x7f2b3c199200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:01.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.007+0000 7f2b40e74640 1 -- 172.21.15.26:0/2662206115 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b3c1a47c0 con 0x7f2b3c101420 2024-09-11T23:26:01.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.007+0000 7f2b41e76640 1 --2- 172.21.15.26:0/2662206115 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2b3c106a80 0x7f2b3c1a4280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:26:01.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.007+0000 7f2b40e74640 1 --2- 172.21.15.26:0/2662206115 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b3c101420 0x7f2b3c199740 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f2b2c009a50 tx=0x7f2b2c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:26:01.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.007+0000 7f2b327fc640 1 -- 172.21.15.26:0/2662206115 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b2c03f030 con 0x7f2b3c101420 2024-09-11T23:26:01.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.007+0000 7f2b438d9640 1 -- 172.21.15.26:0/2662206115 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b3c1a4990 con 0x7f2b3c101420 2024-09-11T23:26:01.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.007+0000 7f2b327fc640 1 -- 172.21.15.26:0/2662206115 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b2c003ce0 con 0x7f2b3c101420 2024-09-11T23:26:01.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.007+0000 7f2b327fc640 1 -- 172.21.15.26:0/2662206115 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b2c035070 con 0x7f2b3c101420 2024-09-11T23:26:01.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.007+0000 7f2b438d9640 1 -- 172.21.15.26:0/2662206115 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b3c1a4e40 con 0x7f2b3c101420 2024-09-11T23:26:01.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.010+0000 7f2b327fc640 1 -- 172.21.15.26:0/2662206115 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f2b2c042030 con 0x7f2b3c101420 2024-09-11T23:26:01.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.010+0000 7f2b45cb2740 1 Processor -- start 2024-09-11T23:26:01.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.010+0000 7f2b45cb2740 1 -- start start 2024-09-11T23:26:01.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.010+0000 7f2b45cb2740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562047349620 0x562047349a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:01.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.010+0000 7f2b45cb2740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562047349f40 0x562047358810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:01.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.010+0000 7f2b45cb2740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562047358d50 0x56204735b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:01.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.010+0000 7f2b45cb2740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5620472af2a0 con 0x562047349620 2024-09-11T23:26:01.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.010+0000 7f2b45cb2740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x562047256b10 con 0x562047349f40 2024-09-11T23:26:01.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.010+0000 7f2b45cb2740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5620472ab260 con 0x562047358d50 2024-09-11T23:26:01.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.010+0000 7f2b41675640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562047349f40 0x562047358810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:01.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.010+0000 7f2b41e76640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562047349620 0x562047349a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:01.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.010+0000 7f2b41675640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562047349f40 0x562047358810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:50618/0 (socket says 172.21.15.26:50618) 2024-09-11T23:26:01.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.010+0000 7f2b41675640 1 -- 172.21.15.26:0/2278336214 learned_addr learned my addr 172.21.15.26:0/2278336214 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:26:01.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.011+0000 7f2b327fc640 1 --2- 172.21.15.26:0/2662206115 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f2b280a2cf0 0x7f2b280a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:01.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.011+0000 7f2b40e74640 1 --2- 172.21.15.26:0/2278336214 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562047358d50 0x56204735b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:01.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.011+0000 7f2b41e76640 1 --2- 172.21.15.26:0/2662206115 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f2b280a2cf0 0x7f2b280a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:01.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.011+0000 7f2b41675640 1 -- 172.21.15.26:0/2278336214 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562047358d50 msgr2=0x56204735b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:26:01.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.011+0000 7f2b41675640 1 --2- 172.21.15.26:0/2278336214 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562047358d50 0x56204735b140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:01.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.011+0000 7f2b41675640 1 -- 172.21.15.26:0/2278336214 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562047349620 msgr2=0x562047349a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:26:01.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.011+0000 7f2b41675640 1 --2- 172.21.15.26:0/2278336214 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562047349620 0x562047349a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:01.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.011+0000 7f2b41675640 1 -- 172.21.15.26:0/2278336214 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562047020b90 con 0x562047349f40 2024-09-11T23:26:01.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.011+0000 7f2b327fc640 1 -- 172.21.15.26:0/2662206115 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8542+0+0 (secure 0 0 0) 0x7f2b2c0efb40 con 0x7f2b3c101420 2024-09-11T23:26:01.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.011+0000 7f2b40e74640 1 --2- 172.21.15.26:0/2278336214 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562047358d50 0x56204735b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:26:01.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.011+0000 7f2b41675640 1 --2- 172.21.15.26:0/2278336214 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562047349f40 0x562047358810 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f2b3800cbe0 tx=0x7f2b3800e720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ae347dcb3794a87 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:26:01.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.011+0000 7f2b41e76640 1 --2- 172.21.15.26:0/2662206115 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f2b280a2cf0 0x7f2b280a51b0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f2b34012830 tx=0x7f2b34004ab0 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:26:01.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.011+0000 7f2b277fe640 1 -- 172.21.15.26:0/2278336214 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b38005220 con 0x562047349f40 2024-09-11T23:26:01.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.011+0000 7f2b277fe640 1 -- 172.21.15.26:0/2278336214 <== mon.2 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b38004c80 con 0x562047349f40 2024-09-11T23:26:01.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.011+0000 7f2b277fe640 1 -- 172.21.15.26:0/2278336214 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b38010870 con 0x562047349f40 2024-09-11T23:26:01.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.012+0000 7f2b45cb2740 1 -- 172.21.15.26:0/2278336214 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562047349f40 msgr2=0x562047358810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:26:01.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.012+0000 7f2b45cb2740 1 --2- 172.21.15.26:0/2278336214 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562047349f40 0x562047358810 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f2b3800cbe0 tx=0x7f2b3800e720 comp rx=0 tx=0).stop 2024-09-11T23:26:01.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.012+0000 7f2b45cb2740 1 -- 172.21.15.26:0/2278336214 shutdown_connections 2024-09-11T23:26:01.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.012+0000 7f2b45cb2740 1 --2- 172.21.15.26:0/2278336214 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562047349f40 0x562047358810 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:01.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.012+0000 7f2b45cb2740 1 --2- 172.21.15.26:0/2278336214 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562047358d50 0x56204735b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:01.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.012+0000 7f2b45cb2740 1 --2- 172.21.15.26:0/2278336214 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562047349620 0x562047349a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:01.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.012+0000 7f2b45cb2740 1 -- 172.21.15.26:0/2278336214 >> 172.21.15.26:0/2278336214 conn(0x562047345cf0 msgr2=0x5620473460d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:26:01.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.012+0000 7f2b45cb2740 1 -- 172.21.15.26:0/2278336214 shutdown_connections 2024-09-11T23:26:01.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.012+0000 7f2b45cb2740 1 -- 172.21.15.26:0/2278336214 wait complete. 2024-09-11T23:26:01.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.012+0000 7f2b45cb2740 1 -- start start 2024-09-11T23:26:01.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.013+0000 7f2b45cb2740 1 Processor -- start 2024-09-11T23:26:01.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.013+0000 7f2b45cb2740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562047349620 0x562047355c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:01.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.013+0000 7f2b45cb2740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562047349f40 0x562047356140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:01.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.013+0000 7f2b45cb2740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562047358d50 0x562047356680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:01.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.013+0000 7f2b45cb2740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x56204735b920 con 0x562047349620 2024-09-11T23:26:01.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.013+0000 7f2b45cb2740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5620472376e0 con 0x562047349f40 2024-09-11T23:26:01.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.013+0000 7f2b45cb2740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x56204735baa0 con 0x562047358d50 2024-09-11T23:26:01.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.013+0000 7f2b45cb2740 10 client.0 _ll_register_callbacks cb 0x56204725ba80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:26:01.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.013+0000 7f2b41675640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562047349620 0x562047355c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:01.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.013+0000 7f2b41675640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562047349620 0x562047355c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49630/0 (socket says 172.21.15.26:49630) 2024-09-11T23:26:01.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.013+0000 7f2b41675640 1 -- 172.21.15.26:0/3300406489 learned_addr learned my addr 172.21.15.26:0/3300406489 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:26:01.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.013+0000 7f2b40e74640 1 --2- 172.21.15.26:0/3300406489 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562047349f40 0x562047356140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:01.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.013+0000 7f2b41675640 1 -- 172.21.15.26:0/3300406489 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562047358d50 msgr2=0x562047356680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:26:01.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.013+0000 7f2b41e76640 1 --2- 172.21.15.26:0/3300406489 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562047358d50 0x562047356680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:01.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.013+0000 7f2b41675640 1 --2- 172.21.15.26:0/3300406489 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562047358d50 0x562047356680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:01.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.013+0000 7f2b41675640 1 -- 172.21.15.26:0/3300406489 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562047349f40 msgr2=0x562047356140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:26:01.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.013+0000 7f2b41675640 1 --2- 172.21.15.26:0/3300406489 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562047349f40 0x562047356140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:01.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.013+0000 7f2b41675640 1 -- 172.21.15.26:0/3300406489 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562047364e50 con 0x562047349620 2024-09-11T23:26:01.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.013+0000 7f2b40e74640 1 --2- 172.21.15.26:0/3300406489 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562047349f40 0x562047356140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:26:01.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.014+0000 7f2b41675640 1 --2- 172.21.15.26:0/3300406489 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562047349620 0x562047355c00 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f2b38010470 tx=0x7f2b3800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff57c7b23a93e927 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:26:01.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.014+0000 7f2b24ff9640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:26:01.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.014+0000 7f2b24ff9640 1 -- 172.21.15.26:0/3300406489 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b38005650 con 0x562047349620 2024-09-11T23:26:01.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.014+0000 7f2b24ff9640 1 -- 172.21.15.26:0/3300406489 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8542+0+0 (secure 0 0 0) 0x7f2b38004240 con 0x562047349620 2024-09-11T23:26:01.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.014+0000 7f2b45cb2740 1 -- 172.21.15.26:0/3300406489 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b38004c80 con 0x562047349620 2024-09-11T23:26:01.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.014+0000 7f2b24ff9640 1 -- 172.21.15.26:0/3300406489 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f2b38004440 con 0x562047349620 2024-09-11T23:26:01.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.014+0000 7f2b45cb2740 10 client.16641 fetch_fsmap learned FSMap version 47 2024-09-11T23:26:01.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.014+0000 7f2b45cb2740 1 -- 172.21.15.26:0/3300406489 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x562047367750 con 0x562047349620 2024-09-11T23:26:01.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.014+0000 7f2b24ff9640 1 -- 172.21.15.26:0/3300406489 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b38004c80 con 0x562047349620 2024-09-11T23:26:01.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.014+0000 7f2b24ff9640 1 -- 172.21.15.26:0/3300406489 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 47) ==== 8414+0+0 (secure 0 0 0) 0x7f2b38010c20 con 0x562047349620 2024-09-11T23:26:01.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.014+0000 7f2b45cb2740 10 client.16641 fetch_fsmap finished waiting for FSMap version 47 2024-09-11T23:26:01.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.014+0000 7f2b45cb2740 10 client.16641 resolve_mds: validated gid 24485 aka daemon mds.b 2024-09-11T23:26:01.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.014+0000 7f2b45cb2740 20 client.16641 populate_metadata read hostname 'smithi026' 2024-09-11T23:26:01.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.015+0000 7f2b45cb2740 1 --2- 172.21.15.26:0/3300406489 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x562047368b50 0x56204736af40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:01.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.015+0000 7f2b45cb2740 4 client.16641 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:26:01.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.015+0000 7f2b45cb2740 1 -- 172.21.15.26:0/3300406489 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5620472bc9d0 con 0x562047368b50 2024-09-11T23:26:01.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.015+0000 7f2b40e74640 1 --2- 172.21.15.26:0/3300406489 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x562047368b50 0x56204736af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:01.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.015+0000 7f2b40e74640 1 --2- 172.21.15.26:0/3300406489 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x562047368b50 0x56204736af40 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3694efe0fdce3553 server_cookie=9af7c18e13ac078c in_seq=0 out_seq=0 2024-09-11T23:26:01.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.015+0000 7f2b24ff9640 10 client.16641 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:26:01.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.016+0000 7f2b24ff9640 1 -- 172.21.15.26:0/3300406489 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5620472bc9d0 con 0x562047368b50 2024-09-11T23:26:01.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.016+0000 7f2b24ff9640 10 client.16641 handle_command_reply: tid=0 2024-09-11T23:26:01.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.016+0000 7f2b45cb2740 1 client.16641 shutdown 2024-09-11T23:26:01.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.016+0000 7f2b45cb2740 1 -- 172.21.15.26:0/3300406489 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562047349620 msgr2=0x562047355c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:26:01.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.016+0000 7f2b45cb2740 1 --2- 172.21.15.26:0/3300406489 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562047349620 0x562047355c00 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f2b38010470 tx=0x7f2b3800c920 comp rx=0 tx=0).stop 2024-09-11T23:26:01.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.016+0000 7f2b45cb2740 1 -- 172.21.15.26:0/3300406489 shutdown_connections 2024-09-11T23:26:01.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.016+0000 7f2b45cb2740 1 --2- 172.21.15.26:0/3300406489 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562047349f40 0x562047356140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:01.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.016+0000 7f2b45cb2740 1 --2- 172.21.15.26:0/3300406489 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x562047368b50 0x56204736af40 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:01.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.016+0000 7f2b45cb2740 1 --2- 172.21.15.26:0/3300406489 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562047358d50 0x562047356680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:01.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.016+0000 7f2b45cb2740 1 --2- 172.21.15.26:0/3300406489 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562047349620 0x562047355c00 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:01.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.016+0000 7f2b24ff9640 0 client.16641 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:26:01.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.016+0000 7f2b45cb2740 1 -- 172.21.15.26:0/3300406489 >> 172.21.15.26:0/3300406489 conn(0x562047345cf0 msgr2=0x56204734ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:26:01.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.016+0000 7f2b45cb2740 1 -- 172.21.15.26:0/3300406489 shutdown_connections 2024-09-11T23:26:01.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.016+0000 7f2b45cb2740 1 -- 172.21.15.26:0/3300406489 wait complete. 2024-09-11T23:26:01.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.016+0000 7f2b45cb2740 20 client.16641 trim_cache size 0 max 16384 2024-09-11T23:26:01.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.034+0000 7f2b45cb2740 1 Processor -- start 2024-09-11T23:26:01.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.034+0000 7f2b45cb2740 1 -- start start 2024-09-11T23:26:01.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.034+0000 7f2b45cb2740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562047349620 0x56204734b860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:01.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.034+0000 7f2b45cb2740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562047349f40 0x562047370770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:01.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.034+0000 7f2b45cb2740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562047358d50 0x562047370cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:01.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.034+0000 7f2b45cb2740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f2b3c1ab640 con 0x562047349620 2024-09-11T23:26:01.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.034+0000 7f2b45cb2740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f2b3c1ad3f0 con 0x562047349f40 2024-09-11T23:26:01.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.034+0000 7f2b45cb2740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f2b3c1abdd0 con 0x562047358d50 2024-09-11T23:26:01.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.035+0000 7f2b41675640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562047349620 0x56204734b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:01.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.035+0000 7f2b41e76640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562047358d50 0x562047370cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:01.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.035+0000 7f2b41675640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562047349620 0x56204734b860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49646/0 (socket says 172.21.15.26:49646) 2024-09-11T23:26:01.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.035+0000 7f2b41675640 1 -- 172.21.15.26:0/2493966306 learned_addr learned my addr 172.21.15.26:0/2493966306 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:26:01.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.035+0000 7f2b40e74640 1 --2- 172.21.15.26:0/2493966306 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562047349f40 0x562047370770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:01.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.035+0000 7f2b41e76640 1 -- 172.21.15.26:0/2493966306 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562047349f40 msgr2=0x562047370770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:26:01.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.035+0000 7f2b41e76640 1 --2- 172.21.15.26:0/2493966306 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562047349f40 0x562047370770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:01.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.035+0000 7f2b41e76640 1 -- 172.21.15.26:0/2493966306 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562047349620 msgr2=0x56204734b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:26:01.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.035+0000 7f2b41e76640 1 --2- 172.21.15.26:0/2493966306 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562047349620 0x56204734b860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:01.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.035+0000 7f2b41e76640 1 -- 172.21.15.26:0/2493966306 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562047341260 con 0x562047358d50 2024-09-11T23:26:01.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.035+0000 7f2b41675640 1 --2- 172.21.15.26:0/2493966306 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562047349620 0x56204734b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:26:01.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.035+0000 7f2b40e74640 1 --2- 172.21.15.26:0/2493966306 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562047349f40 0x562047370770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:26:01.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.035+0000 7f2b41e76640 1 --2- 172.21.15.26:0/2493966306 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562047358d50 0x562047370cb0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f2b340077c0 tx=0x7f2b34007820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2dac378454318bf3 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:26:01.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.035+0000 7f2b26ffd640 1 -- 172.21.15.26:0/2493966306 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b34018070 con 0x562047358d50 2024-09-11T23:26:01.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.036+0000 7f2b26ffd640 1 -- 172.21.15.26:0/2493966306 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b34023070 con 0x562047358d50 2024-09-11T23:26:01.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.036+0000 7f2b26ffd640 1 -- 172.21.15.26:0/2493966306 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b34005280 con 0x562047358d50 2024-09-11T23:26:01.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.036+0000 7f2b45cb2740 1 -- 172.21.15.26:0/2493966306 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562047358d50 msgr2=0x562047370cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:26:01.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.036+0000 7f2b45cb2740 1 --2- 172.21.15.26:0/2493966306 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562047358d50 0x562047370cb0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f2b340077c0 tx=0x7f2b34007820 comp rx=0 tx=0).stop 2024-09-11T23:26:01.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.036+0000 7f2b45cb2740 1 -- 172.21.15.26:0/2493966306 shutdown_connections 2024-09-11T23:26:01.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.036+0000 7f2b45cb2740 1 --2- 172.21.15.26:0/2493966306 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562047349f40 0x562047370770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:01.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.036+0000 7f2b45cb2740 1 --2- 172.21.15.26:0/2493966306 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562047358d50 0x562047370cb0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:01.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.036+0000 7f2b45cb2740 1 --2- 172.21.15.26:0/2493966306 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562047349620 0x56204734b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:01.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.036+0000 7f2b45cb2740 1 -- 172.21.15.26:0/2493966306 >> 172.21.15.26:0/2493966306 conn(0x562047345cf0 msgr2=0x56204734b150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:26:01.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.036+0000 7f2b45cb2740 1 -- 172.21.15.26:0/2493966306 shutdown_connections 2024-09-11T23:26:01.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.036+0000 7f2b45cb2740 1 -- 172.21.15.26:0/2493966306 wait complete. 2024-09-11T23:26:01.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.037+0000 7f2b45cb2740 1 -- start start 2024-09-11T23:26:01.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.037+0000 7f2b45cb2740 1 Processor -- start 2024-09-11T23:26:01.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.037+0000 7f2b45cb2740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562047349620 0x562047352d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:01.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.037+0000 7f2b45cb2740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562047349f40 0x562047353250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:01.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.037+0000 7f2b45cb2740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562047358d50 0x562047353790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:01.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.037+0000 7f2b45cb2740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f2b3c1accf0 con 0x562047349620 2024-09-11T23:26:01.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.037+0000 7f2b45cb2740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f2b3c1ac560 con 0x562047349f40 2024-09-11T23:26:01.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.037+0000 7f2b45cb2740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f2b3c1a9660 con 0x562047358d50 2024-09-11T23:26:01.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.037+0000 7f2b45cb2740 10 client.0 _ll_register_callbacks cb 0x7f2b3800ca30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:26:01.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.037+0000 7f2b41675640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562047349620 0x562047352d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:01.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.038+0000 7f2b41675640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562047349620 0x562047352d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49656/0 (socket says 172.21.15.26:49656) 2024-09-11T23:26:01.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.038+0000 7f2b41675640 1 -- 172.21.15.26:0/212604489 learned_addr learned my addr 172.21.15.26:0/212604489 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:26:01.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.038+0000 7f2b40e74640 1 --2- 172.21.15.26:0/212604489 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562047349f40 0x562047353250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:01.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.038+0000 7f2b41e76640 1 --2- 172.21.15.26:0/212604489 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562047358d50 0x562047353790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:01.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.038+0000 7f2b41675640 1 -- 172.21.15.26:0/212604489 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562047358d50 msgr2=0x562047353790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:26:01.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.038+0000 7f2b41675640 1 --2- 172.21.15.26:0/212604489 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562047358d50 0x562047353790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:01.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.038+0000 7f2b41675640 1 -- 172.21.15.26:0/212604489 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562047349f40 msgr2=0x562047353250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:26:01.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.038+0000 7f2b41675640 1 --2- 172.21.15.26:0/212604489 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562047349f40 0x562047353250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:01.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.038+0000 7f2b41675640 1 -- 172.21.15.26:0/212604489 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5620473509e0 con 0x562047349620 2024-09-11T23:26:01.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.038+0000 7f2b41675640 1 --2- 172.21.15.26:0/212604489 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562047349620 0x562047352d10 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f2b38003f80 tx=0x7f2b38005900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42c026439f20e514 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:26:01.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.038+0000 7f2b24ff9640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:26:01.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.038+0000 7f2b24ff9640 1 -- 172.21.15.26:0/212604489 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b38004240 con 0x562047349620 2024-09-11T23:26:01.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.038+0000 7f2b24ff9640 1 -- 172.21.15.26:0/212604489 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8542+0+0 (secure 0 0 0) 0x7f2b38010680 con 0x562047349620 2024-09-11T23:26:01.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.038+0000 7f2b45cb2740 1 -- 172.21.15.26:0/212604489 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b34023070 con 0x562047349620 2024-09-11T23:26:01.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.038+0000 7f2b24ff9640 1 -- 172.21.15.26:0/212604489 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f2b3803d050 con 0x562047349620 2024-09-11T23:26:01.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.038+0000 7f2b45cb2740 10 client.16647 fetch_fsmap learned FSMap version 47 2024-09-11T23:26:01.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.038+0000 7f2b45cb2740 1 -- 172.21.15.26:0/212604489 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x562047346980 con 0x562047349620 2024-09-11T23:26:01.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.039+0000 7f2b24ff9640 1 -- 172.21.15.26:0/212604489 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b34023070 con 0x562047349620 2024-09-11T23:26:01.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.039+0000 7f2b24ff9640 1 -- 172.21.15.26:0/212604489 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 47) ==== 8414+0+0 (secure 0 0 0) 0x7f2b380258e0 con 0x562047349620 2024-09-11T23:26:01.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.039+0000 7f2b45cb2740 10 client.16647 fetch_fsmap finished waiting for FSMap version 47 2024-09-11T23:26:01.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.039+0000 7f2b45cb2740 10 client.16647 resolve_mds: validated gid 24485 aka daemon mds.b 2024-09-11T23:26:01.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.039+0000 7f2b45cb2740 20 client.16647 populate_metadata read hostname 'smithi026' 2024-09-11T23:26:01.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.039+0000 7f2b45cb2740 1 --2- 172.21.15.26:0/212604489 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5620473d3f50 0x5620473d6370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:01.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.039+0000 7f2b45cb2740 4 client.16647 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 136.6, "await_for": 45.0}] 2024-09-11T23:26:01.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.039+0000 7f2b45cb2740 1 -- 172.21.15.26:0/212604489 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 136.6, "await_for": 45.0}) -- 0x5620472b6d20 con 0x5620473d3f50 2024-09-11T23:26:01.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.039+0000 7f2b40e74640 1 --2- 172.21.15.26:0/212604489 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5620473d3f50 0x5620473d6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:01.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.039+0000 7f2b40e74640 1 --2- 172.21.15.26:0/212604489 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5620473d3f50 0x5620473d6370 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb3744a9061fb4ca server_cookie=362b86fc6ea8f55a in_seq=0 out_seq=0 2024-09-11T23:26:01.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:01.039+0000 7f2b24ff9640 10 client.16647 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:26:02.052 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:26:01 smithi076 ceph-mon[35296]: pgmap v234: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 483 KiB/s wr, 57 op/s 2024-09-11T23:26:02.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:26:01 smithi136 ceph-mon[35121]: pgmap v234: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 483 KiB/s wr, 57 op/s 2024-09-11T23:26:02.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:26:01 smithi026 ceph-mon[31172]: pgmap v234: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 483 KiB/s wr, 57 op/s 2024-09-11T23:26:04.052 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:26:03 smithi076 ceph-mon[35296]: pgmap v235: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 56 KiB/s wr, 21 op/s 2024-09-11T23:26:04.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:26:03 smithi136 ceph-mon[35121]: pgmap v235: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 56 KiB/s wr, 21 op/s 2024-09-11T23:26:04.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:26:03 smithi026 ceph-mon[31172]: pgmap v235: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 56 KiB/s wr, 21 op/s 2024-09-11T23:26:06.052 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:26:05 smithi076 ceph-mon[35296]: pgmap v236: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 118 KiB/s wr, 54 op/s 2024-09-11T23:26:06.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:26:05 smithi136 ceph-mon[35121]: pgmap v236: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 118 KiB/s wr, 54 op/s 2024-09-11T23:26:06.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:26:05 smithi026 ceph-mon[31172]: pgmap v236: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 118 KiB/s wr, 54 op/s 2024-09-11T23:26:08.052 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:26:07 smithi026 ceph-mon[31172]: pgmap v237: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 110 KiB/s wr, 54 op/s 2024-09-11T23:26:08.052 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:26:07 smithi026 ceph-mon[31172]: osdmap e89: 12 total, 12 up, 12 in 2024-09-11T23:26:08.052 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:26:07 smithi076 ceph-mon[35296]: pgmap v237: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 110 KiB/s wr, 54 op/s 2024-09-11T23:26:08.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:26:07 smithi076 ceph-mon[35296]: osdmap e89: 12 total, 12 up, 12 in 2024-09-11T23:26:08.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:26:07 smithi136 ceph-mon[35121]: pgmap v237: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 110 KiB/s wr, 54 op/s 2024-09-11T23:26:08.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:26:07 smithi136 ceph-mon[35121]: osdmap e89: 12 total, 12 up, 12 in 2024-09-11T23:26:10.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:26:09 smithi076 ceph-mon[35296]: pgmap v239: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 129 KiB/s wr, 60 op/s 2024-09-11T23:26:10.116 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.116+0000 7f2b508ba640 1 -- 172.21.15.26:0/1148975969 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b4c101ad0 msgr2=0x7f2b4c101f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:26:10.116 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.116+0000 7f2b508ba640 1 --2- 172.21.15.26:0/1148975969 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b4c101ad0 0x7f2b4c101f30 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f2b40009930 tx=0x7f2b4002a420 comp rx=0 tx=0).stop 2024-09-11T23:26:10.117 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.116+0000 7f2b508ba640 1 -- 172.21.15.26:0/1148975969 shutdown_connections 2024-09-11T23:26:10.117 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.116+0000 7f2b508ba640 1 --2- 172.21.15.26:0/1148975969 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2b4c102470 0x7f2b4c10e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:10.117 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.116+0000 7f2b508ba640 1 --2- 172.21.15.26:0/1148975969 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b4c101ad0 0x7f2b4c101f30 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:10.117 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.116+0000 7f2b508ba640 1 --2- 172.21.15.26:0/1148975969 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2b4c107b70 0x7f2b4c107f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:10.117 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.116+0000 7f2b508ba640 1 -- 172.21.15.26:0/1148975969 >> 172.21.15.26:0/1148975969 conn(0x7f2b4c0fd7b0 msgr2=0x7f2b4c0ffbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:26:10.117 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.117+0000 7f2b508ba640 1 -- 172.21.15.26:0/1148975969 shutdown_connections 2024-09-11T23:26:10.117 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.117+0000 7f2b508ba640 1 -- 172.21.15.26:0/1148975969 wait complete. 2024-09-11T23:26:10.119 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.117+0000 7f2b508ba640 1 Processor -- start 2024-09-11T23:26:10.119 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.118+0000 7f2b508ba640 1 -- start start 2024-09-11T23:26:10.120 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.118+0000 7f2b508ba640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b4c101ad0 0x7f2b4c198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:10.120 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.118+0000 7f2b508ba640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2b4c102470 0x7f2b4c1988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:10.120 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.118+0000 7f2b508ba640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2b4c107b70 0x7f2b4c19cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:10.120 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.118+0000 7f2b508ba640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f2b4c111f20 con 0x7f2b4c101ad0 2024-09-11T23:26:10.120 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.118+0000 7f2b508ba640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f2b4c111da0 con 0x7f2b4c102470 2024-09-11T23:26:10.120 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.118+0000 7f2b508ba640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f2b4c1120a0 con 0x7f2b4c107b70 2024-09-11T23:26:10.120 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.118+0000 7f2b49d9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2b4c102470 0x7f2b4c1988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:10.120 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.118+0000 7f2b49d9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2b4c102470 0x7f2b4c1988d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:50670/0 (socket says 172.21.15.26:50670) 2024-09-11T23:26:10.120 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.118+0000 7f2b49d9b640 1 -- 172.21.15.26:0/3590852507 learned_addr learned my addr 172.21.15.26:0/3590852507 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:26:10.120 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.118+0000 7f2b4ad9d640 1 --2- 172.21.15.26:0/3590852507 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2b4c107b70 0x7f2b4c19cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:10.120 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.118+0000 7f2b4a59c640 1 --2- 172.21.15.26:0/3590852507 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b4c101ad0 0x7f2b4c198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:10.120 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.119+0000 7f2b49d9b640 1 -- 172.21.15.26:0/3590852507 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2b4c107b70 msgr2=0x7f2b4c19cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:26:10.120 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.119+0000 7f2b49d9b640 1 --2- 172.21.15.26:0/3590852507 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2b4c107b70 0x7f2b4c19cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:10.121 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.119+0000 7f2b49d9b640 1 -- 172.21.15.26:0/3590852507 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b4c101ad0 msgr2=0x7f2b4c198390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:26:10.121 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.119+0000 7f2b49d9b640 1 --2- 172.21.15.26:0/3590852507 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b4c101ad0 0x7f2b4c198390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:10.121 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.119+0000 7f2b49d9b640 1 -- 172.21.15.26:0/3590852507 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b4c19d380 con 0x7f2b4c102470 2024-09-11T23:26:10.121 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.119+0000 7f2b49d9b640 1 --2- 172.21.15.26:0/3590852507 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2b4c102470 0x7f2b4c1988d0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f2b40009230 tx=0x7f2b40004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:26:10.121 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.119+0000 7f2b3b7fe640 1 -- 172.21.15.26:0/3590852507 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b4003f030 con 0x7f2b4c102470 2024-09-11T23:26:10.121 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.119+0000 7f2b508ba640 1 -- 172.21.15.26:0/3590852507 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b4c19d610 con 0x7f2b4c102470 2024-09-11T23:26:10.121 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.119+0000 7f2b3b7fe640 1 -- 172.21.15.26:0/3590852507 <== mon.2 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b40004440 con 0x7f2b4c102470 2024-09-11T23:26:10.121 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.119+0000 7f2b3b7fe640 1 -- 172.21.15.26:0/3590852507 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b40035070 con 0x7f2b4c102470 2024-09-11T23:26:10.121 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.119+0000 7f2b508ba640 1 -- 172.21.15.26:0/3590852507 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b4c1a4f00 con 0x7f2b4c102470 2024-09-11T23:26:10.123 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.123+0000 7f2b5365b740 1 Processor -- start 2024-09-11T23:26:10.124 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.123+0000 7f2b5365b740 1 -- start start 2024-09-11T23:26:10.124 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.124+0000 7f2b5365b740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f43a364cc0 0x55f43a3650a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:10.124 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.124+0000 7f2b5365b740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f43a360660 0x55f43a360ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:10.124 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.124+0000 7f2b5365b740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f43a361000 0x55f43a36c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:10.124 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.124+0000 7f2b5365b740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55f43a2bb2f0 con 0x55f43a360660 2024-09-11T23:26:10.124 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.124+0000 7f2b5365b740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55f43a263270 con 0x55f43a364cc0 2024-09-11T23:26:10.124 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.124+0000 7f2b5365b740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55f43a2b72b0 con 0x55f43a361000 2024-09-11T23:26:10.124 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.124+0000 7f2b4ad9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f43a364cc0 0x55f43a3650a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:10.125 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.124+0000 7f2b4ad9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f43a364cc0 0x55f43a3650a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:50678/0 (socket says 172.21.15.26:50678) 2024-09-11T23:26:10.125 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.124+0000 7f2b4ad9d640 1 -- 172.21.15.26:0/1918515117 learned_addr learned my addr 172.21.15.26:0/1918515117 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:26:10.125 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.124+0000 7f2b3b7fe640 1 -- 172.21.15.26:0/3590852507 <== mon.2 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f2b40042030 con 0x7f2b4c102470 2024-09-11T23:26:10.125 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.124+0000 7f2b4a59c640 1 --2- 172.21.15.26:0/1918515117 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f43a360660 0x55f43a360ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:10.125 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.124+0000 7f2b49d9b640 1 --2- 172.21.15.26:0/1918515117 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f43a361000 0x55f43a36c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:10.125 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.124+0000 7f2b4ad9d640 1 -- 172.21.15.26:0/1918515117 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f43a361000 msgr2=0x55f43a36c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:26:10.126 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.124+0000 7f2b4ad9d640 1 --2- 172.21.15.26:0/1918515117 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f43a361000 0x55f43a36c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:10.126 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.124+0000 7f2b4ad9d640 1 -- 172.21.15.26:0/1918515117 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f43a360660 msgr2=0x55f43a360ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:26:10.126 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.124+0000 7f2b4ad9d640 1 --2- 172.21.15.26:0/1918515117 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f43a360660 0x55f43a360ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:10.126 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.124+0000 7f2b4ad9d640 1 -- 172.21.15.26:0/1918515117 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f43a262340 con 0x55f43a364cc0 2024-09-11T23:26:10.126 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.124+0000 7f2b4a59c640 1 --2- 172.21.15.26:0/1918515117 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f43a360660 0x55f43a360ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:26:10.126 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.124+0000 7f2b49d9b640 1 --2- 172.21.15.26:0/1918515117 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f43a361000 0x55f43a36c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:26:10.126 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.125+0000 7f2b4ad9d640 1 --2- 172.21.15.26:0/1918515117 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f43a364cc0 0x55f43a3650a0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f2b44009fd0 tx=0x7f2b440118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=21e33cdf93d69d2a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:26:10.126 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.125+0000 7f2b38ff9640 1 -- 172.21.15.26:0/1918515117 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b44004d20 con 0x55f43a364cc0 2024-09-11T23:26:10.126 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.125+0000 7f2b38ff9640 1 -- 172.21.15.26:0/1918515117 <== mon.2 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b44004050 con 0x55f43a364cc0 2024-09-11T23:26:10.127 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.125+0000 7f2b38ff9640 1 -- 172.21.15.26:0/1918515117 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b44005260 con 0x55f43a364cc0 2024-09-11T23:26:10.127 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.125+0000 7f2b3b7fe640 1 --2- 172.21.15.26:0/3590852507 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f2b340a2e20 0x7f2b340a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:10.127 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.126+0000 7f2b3b7fe640 1 -- 172.21.15.26:0/3590852507 <== mon.2 v2:172.21.15.76:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8586+0+0 (secure 0 0 0) 0x7f2b400ef880 con 0x7f2b4c102470 2024-09-11T23:26:10.127 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.126+0000 7f2b4a59c640 1 --2- 172.21.15.26:0/3590852507 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f2b340a2e20 0x7f2b340a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:10.129 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.126+0000 7f2b4a59c640 1 --2- 172.21.15.26:0/3590852507 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f2b340a2e20 0x7f2b340a52e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x55f43a354260 tx=0x7f2b3c005f90 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:26:10.129 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.127+0000 7f2b5365b740 1 -- 172.21.15.26:0/1918515117 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f43a364cc0 msgr2=0x55f43a3650a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:26:10.129 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.127+0000 7f2b5365b740 1 --2- 172.21.15.26:0/1918515117 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f43a364cc0 0x55f43a3650a0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f2b44009fd0 tx=0x7f2b440118d0 comp rx=0 tx=0).stop 2024-09-11T23:26:10.129 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.127+0000 7f2b5365b740 1 -- 172.21.15.26:0/1918515117 shutdown_connections 2024-09-11T23:26:10.130 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.127+0000 7f2b5365b740 1 --2- 172.21.15.26:0/1918515117 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f43a361000 0x55f43a36c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:10.130 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.127+0000 7f2b5365b740 1 --2- 172.21.15.26:0/1918515117 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f43a360660 0x55f43a360ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:10.130 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.127+0000 7f2b5365b740 1 --2- 172.21.15.26:0/1918515117 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f43a364cc0 0x55f43a3650a0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:10.130 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.127+0000 7f2b5365b740 1 -- 172.21.15.26:0/1918515117 >> 172.21.15.26:0/1918515117 conn(0x55f43a352b10 msgr2=0x55f43a352ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:26:10.130 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.127+0000 7f2b5365b740 1 -- 172.21.15.26:0/1918515117 shutdown_connections 2024-09-11T23:26:10.130 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.127+0000 7f2b5365b740 1 -- 172.21.15.26:0/1918515117 wait complete. 2024-09-11T23:26:10.130 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.128+0000 7f2b5365b740 1 -- start start 2024-09-11T23:26:10.130 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.128+0000 7f2b5365b740 1 Processor -- start 2024-09-11T23:26:10.130 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.129+0000 7f2b5365b740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f43a360660 0x55f43a369ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:10.130 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.129+0000 7f2b5365b740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f43a361000 0x55f43a36a430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:10.130 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.129+0000 7f2b5365b740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f43a364cc0 0x55f43a36a970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:10.130 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.129+0000 7f2b4a59c640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f43a360660 0x55f43a369ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:10.131 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.129+0000 7f2b4a59c640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f43a360660 0x55f43a369ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:57818/0 (socket says 172.21.15.26:57818) 2024-09-11T23:26:10.131 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.129+0000 7f2b4a59c640 1 -- 172.21.15.26:0/1204252247 learned_addr learned my addr 172.21.15.26:0/1204252247 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:26:10.131 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.129+0000 7f2b4ad9d640 1 --2- 172.21.15.26:0/1204252247 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f43a364cc0 0x55f43a36a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:10.131 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.129+0000 7f2b49d9b640 1 --2- 172.21.15.26:0/1204252247 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f43a361000 0x55f43a36a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:10.131 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.129+0000 7f2b5365b740 1 -- 172.21.15.26:0/1204252247 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55f43a36d0b0 con 0x55f43a364cc0 2024-09-11T23:26:10.131 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.129+0000 7f2b5365b740 1 -- 172.21.15.26:0/1204252247 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55f43a244290 con 0x55f43a361000 2024-09-11T23:26:10.131 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.130+0000 7f2b5365b740 1 -- 172.21.15.26:0/1204252247 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55f43a36d230 con 0x55f43a360660 2024-09-11T23:26:10.133 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.130+0000 7f2b5365b740 10 client.0 _ll_register_callbacks cb 0x55f43a275140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:26:10.133 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.130+0000 7f2b4ad9d640 1 -- 172.21.15.26:0/1204252247 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f43a360660 msgr2=0x55f43a369ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:26:10.133 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.130+0000 7f2b4ad9d640 1 --2- 172.21.15.26:0/1204252247 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f43a360660 0x55f43a369ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:10.133 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.130+0000 7f2b4ad9d640 1 -- 172.21.15.26:0/1204252247 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f43a361000 msgr2=0x55f43a36a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:26:10.133 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.130+0000 7f2b4ad9d640 1 --2- 172.21.15.26:0/1204252247 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f43a361000 0x55f43a36a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:10.133 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.130+0000 7f2b4ad9d640 1 -- 172.21.15.26:0/1204252247 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f439fce210 con 0x55f43a364cc0 2024-09-11T23:26:10.134 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.130+0000 7f2b5365b740 1 -- 172.21.15.26:0/1204252247 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b44004050 con 0x55f43a364cc0 2024-09-11T23:26:10.134 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.130+0000 7f2b4ad9d640 1 --2- 172.21.15.26:0/1204252247 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f43a364cc0 0x55f43a36a970 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f2b4401b040 tx=0x7f2b440048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e0d424638bffed4 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:26:10.134 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.130+0000 7f2b31ffb640 10 client.16659 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:26:10.134 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.131+0000 7f2b31ffb640 1 -- 172.21.15.26:0/1204252247 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b44028020 con 0x55f43a364cc0 2024-09-11T23:26:10.134 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.131+0000 7f2b31ffb640 1 -- 172.21.15.26:0/1204252247 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8586+0+0 (secure 0 0 0) 0x7f2b44004410 con 0x55f43a364cc0 2024-09-11T23:26:10.134 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.131+0000 7f2b31ffb640 1 -- 172.21.15.26:0/1204252247 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f2b44004630 con 0x55f43a364cc0 2024-09-11T23:26:10.134 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.131+0000 7f2b5365b740 10 client.16659 fetch_fsmap learned FSMap version 47 2024-09-11T23:26:10.134 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.131+0000 7f2b5365b740 1 -- 172.21.15.26:0/1204252247 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({fsmap=47}) -- 0x55f43a09a620 con 0x55f43a364cc0 2024-09-11T23:26:10.134 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.132+0000 7f2b31ffb640 1 -- 172.21.15.26:0/1204252247 <== mon.0 v2:172.21.15.26:3300/0 4 ==== fsmap(e 47) ==== 8414+0+0 (secure 0 0 0) 0x7f2b44005040 con 0x55f43a364cc0 2024-09-11T23:26:10.134 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.132+0000 7f2b5365b740 10 client.16659 fetch_fsmap finished waiting for FSMap version 47 2024-09-11T23:26:10.134 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.132+0000 7f2b5365b740 10 client.16659 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:26:10.134 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.132+0000 7f2b5365b740 20 client.16659 populate_metadata read hostname 'smithi026' 2024-09-11T23:26:10.134 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.132+0000 7f2b5365b740 1 --2- 172.21.15.26:0/1204252247 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55f43a37a1f0 0x55f43a37c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:10.135 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.132+0000 7f2b5365b740 4 client.16659 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:26:10.135 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.132+0000 7f2b5365b740 1 -- 172.21.15.26:0/1204252247 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f43a2c8a20 con 0x55f43a37a1f0 2024-09-11T23:26:10.135 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.133+0000 7f2b4a59c640 1 --2- 172.21.15.26:0/1204252247 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55f43a37a1f0 0x55f43a37c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:10.135 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.133+0000 7f2b4a59c640 1 --2- 172.21.15.26:0/1204252247 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55f43a37a1f0 0x55f43a37c5e0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=38615af91db2741b server_cookie=61bc9f7e462540eb in_seq=0 out_seq=0 2024-09-11T23:26:10.135 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.133+0000 7f2b31ffb640 10 client.16659 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:26:10.136 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.134+0000 7f2b31ffb640 1 -- 172.21.15.26:0/1204252247 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f43a2c8a20 con 0x55f43a37a1f0 2024-09-11T23:26:10.136 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.134+0000 7f2b31ffb640 10 client.16659 handle_command_reply: tid=0 2024-09-11T23:26:10.136 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.134+0000 7f2b5365b740 1 client.16659 shutdown 2024-09-11T23:26:10.136 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.134+0000 7f2b5365b740 1 -- 172.21.15.26:0/1204252247 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f43a364cc0 msgr2=0x55f43a36a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:26:10.136 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.134+0000 7f2b5365b740 1 --2- 172.21.15.26:0/1204252247 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f43a364cc0 0x55f43a36a970 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f2b4401b040 tx=0x7f2b440048f0 comp rx=0 tx=0).stop 2024-09-11T23:26:10.136 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.134+0000 7f2b5365b740 1 -- 172.21.15.26:0/1204252247 shutdown_connections 2024-09-11T23:26:10.136 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.134+0000 7f2b5365b740 1 --2- 172.21.15.26:0/1204252247 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f43a361000 0x55f43a36a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:10.136 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.134+0000 7f2b5365b740 1 --2- 172.21.15.26:0/1204252247 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55f43a37a1f0 0x55f43a37c5e0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:10.137 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.135+0000 7f2b31ffb640 0 client.16659 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:26:10.137 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.135+0000 7f2b5365b740 1 --2- 172.21.15.26:0/1204252247 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f43a364cc0 0x55f43a36a970 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:10.137 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.135+0000 7f2b5365b740 1 --2- 172.21.15.26:0/1204252247 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f43a360660 0x55f43a369ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:10.137 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.135+0000 7f2b5365b740 1 -- 172.21.15.26:0/1204252247 >> 172.21.15.26:0/1204252247 conn(0x55f43a352b10 msgr2=0x55f43a35fe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:26:10.137 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.135+0000 7f2b5365b740 1 -- 172.21.15.26:0/1204252247 shutdown_connections 2024-09-11T23:26:10.137 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.135+0000 7f2b5365b740 1 -- 172.21.15.26:0/1204252247 wait complete. 2024-09-11T23:26:10.137 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.135+0000 7f2b5365b740 20 client.16659 trim_cache size 0 max 16384 2024-09-11T23:26:10.157 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.157+0000 7f2b5365b740 1 Processor -- start 2024-09-11T23:26:10.157 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.157+0000 7f2b5365b740 1 -- start start 2024-09-11T23:26:10.157 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.157+0000 7f2b5365b740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f43a360660 0x55f43a36be60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:10.157 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.157+0000 7f2b5365b740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f43a361000 0x55f43a381d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:10.157 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.157+0000 7f2b5365b740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f43a364cc0 0x55f43a382260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:10.158 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.157+0000 7f2b5365b740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f2b4c1ab760 con 0x55f43a361000 2024-09-11T23:26:10.158 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.157+0000 7f2b5365b740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f2b4c1ad510 con 0x55f43a360660 2024-09-11T23:26:10.158 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.157+0000 7f2b5365b740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f2b4c1abef0 con 0x55f43a364cc0 2024-09-11T23:26:10.158 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.157+0000 7f2b4a59c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f43a360660 0x55f43a36be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:10.158 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.157+0000 7f2b49d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f43a361000 0x55f43a381d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:10.158 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.157+0000 7f2b4a59c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f43a360660 0x55f43a36be60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:50700/0 (socket says 172.21.15.26:50700) 2024-09-11T23:26:10.158 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.157+0000 7f2b4a59c640 1 -- 172.21.15.26:0/3837272381 learned_addr learned my addr 172.21.15.26:0/3837272381 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:26:10.158 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.158+0000 7f2b4ad9d640 1 --2- 172.21.15.26:0/3837272381 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f43a364cc0 0x55f43a382260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:10.160 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.158+0000 7f2b4a59c640 1 -- 172.21.15.26:0/3837272381 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f43a364cc0 msgr2=0x55f43a382260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:26:10.160 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.158+0000 7f2b4a59c640 1 --2- 172.21.15.26:0/3837272381 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f43a364cc0 0x55f43a382260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:10.160 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.158+0000 7f2b4a59c640 1 -- 172.21.15.26:0/3837272381 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f43a361000 msgr2=0x55f43a381d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:26:10.160 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.158+0000 7f2b4a59c640 1 --2- 172.21.15.26:0/3837272381 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f43a361000 0x55f43a381d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:10.160 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.158+0000 7f2b4a59c640 1 -- 172.21.15.26:0/3837272381 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f43a369cc0 con 0x55f43a360660 2024-09-11T23:26:10.160 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.158+0000 7f2b4a59c640 1 --2- 172.21.15.26:0/3837272381 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f43a360660 0x55f43a36be60 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f2b3c009f20 tx=0x7f2b3c016000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f9bc481d8957c354 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:26:10.160 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.158+0000 7f2b33fff640 1 -- 172.21.15.26:0/3837272381 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b3c01fbd0 con 0x55f43a360660 2024-09-11T23:26:10.160 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.158+0000 7f2b33fff640 1 -- 172.21.15.26:0/3837272381 <== mon.2 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b3c01fd70 con 0x55f43a360660 2024-09-11T23:26:10.160 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.158+0000 7f2b33fff640 1 -- 172.21.15.26:0/3837272381 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b3c0116d0 con 0x55f43a360660 2024-09-11T23:26:10.160 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.159+0000 7f2b5365b740 1 -- 172.21.15.26:0/3837272381 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f43a360660 msgr2=0x55f43a36be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:26:10.160 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.159+0000 7f2b5365b740 1 --2- 172.21.15.26:0/3837272381 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f43a360660 0x55f43a36be60 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f2b3c009f20 tx=0x7f2b3c016000 comp rx=0 tx=0).stop 2024-09-11T23:26:10.161 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.159+0000 7f2b5365b740 1 -- 172.21.15.26:0/3837272381 shutdown_connections 2024-09-11T23:26:10.161 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.159+0000 7f2b5365b740 1 --2- 172.21.15.26:0/3837272381 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f43a364cc0 0x55f43a382260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:10.161 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.159+0000 7f2b5365b740 1 --2- 172.21.15.26:0/3837272381 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f43a361000 0x55f43a381d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:10.161 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.159+0000 7f2b5365b740 1 --2- 172.21.15.26:0/3837272381 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f43a360660 0x55f43a36be60 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:10.161 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.159+0000 7f2b5365b740 1 -- 172.21.15.26:0/3837272381 >> 172.21.15.26:0/3837272381 conn(0x55f43a352b10 msgr2=0x55f43a36b750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:26:10.161 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.159+0000 7f2b5365b740 1 -- 172.21.15.26:0/3837272381 shutdown_connections 2024-09-11T23:26:10.161 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.159+0000 7f2b5365b740 1 -- 172.21.15.26:0/3837272381 wait complete. 2024-09-11T23:26:10.161 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.160+0000 7f2b5365b740 1 -- start start 2024-09-11T23:26:10.163 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.160+0000 7f2b5365b740 1 Processor -- start 2024-09-11T23:26:10.163 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.161+0000 7f2b5365b740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f43a360660 0x55f43a372ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:10.163 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.161+0000 7f2b5365b740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f43a361000 0x55f43a36ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:10.163 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.161+0000 7f2b5365b740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f43a364cc0 0x55f43a36e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:10.163 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.161+0000 7f2b5365b740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f2b4c1ace10 con 0x55f43a364cc0 2024-09-11T23:26:10.163 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.161+0000 7f2b4ad9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f43a364cc0 0x55f43a36e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:10.163 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.161+0000 7f2b4a59c640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f43a360660 0x55f43a372ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:10.163 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.161+0000 7f2b49d9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f43a361000 0x55f43a36ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:10.164 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.161+0000 7f2b4a59c640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f43a360660 0x55f43a372ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:57842/0 (socket says 172.21.15.26:57842) 2024-09-11T23:26:10.164 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.161+0000 7f2b4ad9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f43a364cc0 0x55f43a36e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49714/0 (socket says 172.21.15.26:49714) 2024-09-11T23:26:10.164 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.161+0000 7f2b4a59c640 1 -- 172.21.15.26:0/3711589260 learned_addr learned my addr 172.21.15.26:0/3711589260 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:26:10.164 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.161+0000 7f2b5365b740 1 -- 172.21.15.26:0/3711589260 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f2b4c1ac680 con 0x55f43a361000 2024-09-11T23:26:10.164 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.161+0000 7f2b5365b740 1 -- 172.21.15.26:0/3711589260 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f2b4c1a9780 con 0x55f43a360660 2024-09-11T23:26:10.164 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.161+0000 7f2b5365b740 10 client.0 _ll_register_callbacks cb 0x7f2b44009ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:26:10.164 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.162+0000 7f2b49d9b640 1 -- 172.21.15.26:0/3711589260 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f43a360660 msgr2=0x55f43a372ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:26:10.164 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.162+0000 7f2b49d9b640 1 --2- 172.21.15.26:0/3711589260 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f43a360660 0x55f43a372ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:10.164 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.162+0000 7f2b49d9b640 1 -- 172.21.15.26:0/3711589260 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f43a364cc0 msgr2=0x55f43a36e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:26:10.164 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.162+0000 7f2b49d9b640 1 --2- 172.21.15.26:0/3711589260 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f43a364cc0 0x55f43a36e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:10.164 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.162+0000 7f2b49d9b640 1 -- 172.21.15.26:0/3711589260 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f43a36ea20 con 0x55f43a361000 2024-09-11T23:26:10.164 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.162+0000 7f2b4ad9d640 1 --2- 172.21.15.26:0/3711589260 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f43a364cc0 0x55f43a36e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:26:10.165 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.162+0000 7f2b49d9b640 1 --2- 172.21.15.26:0/3711589260 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f43a361000 0x55f43a36ddf0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f2b400ecf30 tx=0x7f2b400bbb90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=332aa9591efe3682 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:26:10.165 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.162+0000 7f2b31ffb640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-11T23:26:10.166 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.164+0000 7f2b31ffb640 1 -- 172.21.15.26:0/3711589260 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b40049040 con 0x55f43a361000 2024-09-11T23:26:10.166 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.164+0000 7f2b31ffb640 1 -- 172.21.15.26:0/3711589260 <== mon.2 v2:172.21.15.76:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8586+0+0 (secure 0 0 0) 0x7f2b400ed1b0 con 0x55f43a361000 2024-09-11T23:26:10.166 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.164+0000 7f2b5365b740 1 -- 172.21.15.26:0/3711589260 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b3c01fd70 con 0x55f43a361000 2024-09-11T23:26:10.167 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.164+0000 7f2b31ffb640 1 -- 172.21.15.26:0/3711589260 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f2b400bbd00 con 0x55f43a361000 2024-09-11T23:26:10.167 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.164+0000 7f2b5365b740 10 client.26261 fetch_fsmap learned FSMap version 47 2024-09-11T23:26:10.167 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.164+0000 7f2b5365b740 1 -- 172.21.15.26:0/3711589260 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55f43a3e4ba0 con 0x55f43a361000 2024-09-11T23:26:10.167 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.165+0000 7f2b31ffb640 1 -- 172.21.15.26:0/3711589260 <== mon.2 v2:172.21.15.76:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b400c00b0 con 0x55f43a361000 2024-09-11T23:26:10.167 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.165+0000 7f2b31ffb640 1 -- 172.21.15.26:0/3711589260 <== mon.2 v2:172.21.15.76:3300/0 5 ==== fsmap(e 47) ==== 8414+0+0 (secure 0 0 0) 0x7f2b400c44e0 con 0x55f43a361000 2024-09-11T23:26:10.167 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.165+0000 7f2b5365b740 10 client.26261 fetch_fsmap finished waiting for FSMap version 47 2024-09-11T23:26:10.167 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.165+0000 7f2b5365b740 10 client.26261 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:26:10.167 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.165+0000 7f2b5365b740 20 client.26261 populate_metadata read hostname 'smithi026' 2024-09-11T23:26:10.167 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.165+0000 7f2b5365b740 1 --2- 172.21.15.26:0/3711589260 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55f43a37a1f0 0x55f43a3e7360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:10.167 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.165+0000 7f2b5365b740 4 client.26261 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T23:26:10.167 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.165+0000 7f2b5365b740 1 -- 172.21.15.26:0/3711589260 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f2b3c01fd70 con 0x55f43a37a1f0 2024-09-11T23:26:10.167 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.166+0000 7f2b4a59c640 1 --2- 172.21.15.26:0/3711589260 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55f43a37a1f0 0x55f43a3e7360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:10.167 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.166+0000 7f2b4a59c640 1 --2- 172.21.15.26:0/3711589260 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55f43a37a1f0 0x55f43a3e7360 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c06b8cd2a8302b7 server_cookie=d2c2c12e1529c9f9 in_seq=0 out_seq=0 2024-09-11T23:26:10.168 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.166+0000 7f2b31ffb640 10 client.26261 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:26:10.199 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.199+0000 7f2b31ffb640 1 -- 172.21.15.26:0/3711589260 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f2b3c01fd70 con 0x55f43a37a1f0 2024-09-11T23:26:10.199 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.199+0000 7f2b31ffb640 10 client.26261 handle_command_reply: tid=0 2024-09-11T23:26:10.199 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.199+0000 7f2b5365b740 1 client.26261 shutdown 2024-09-11T23:26:10.200 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.199+0000 7f2b5365b740 1 -- 172.21.15.26:0/3711589260 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f43a361000 msgr2=0x55f43a36ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:26:10.200 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.199+0000 7f2b5365b740 1 --2- 172.21.15.26:0/3711589260 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f43a361000 0x55f43a36ddf0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f2b400ecf30 tx=0x7f2b400bbb90 comp rx=0 tx=0).stop 2024-09-11T23:26:10.200 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.200+0000 7f2b5365b740 1 -- 172.21.15.26:0/3711589260 shutdown_connections 2024-09-11T23:26:10.200 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.200+0000 7f2b5365b740 1 --2- 172.21.15.26:0/3711589260 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f43a361000 0x55f43a36ddf0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:10.200 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.200+0000 7f2b5365b740 1 --2- 172.21.15.26:0/3711589260 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55f43a37a1f0 0x55f43a3e7360 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:10.200 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.200+0000 7f2b5365b740 1 --2- 172.21.15.26:0/3711589260 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f43a364cc0 0x55f43a36e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:10.201 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.200+0000 7f2b31ffb640 0 client.26261 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:26:10.201 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.200+0000 7f2b5365b740 1 --2- 172.21.15.26:0/3711589260 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f43a360660 0x55f43a372ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:10.201 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.200+0000 7f2b5365b740 1 -- 172.21.15.26:0/3711589260 >> 172.21.15.26:0/3711589260 conn(0x55f43a352b10 msgr2=0x55f43a35fe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:26:10.201 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.200+0000 7f2b5365b740 1 -- 172.21.15.26:0/3711589260 shutdown_connections 2024-09-11T23:26:10.201 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.200+0000 7f2b5365b740 1 -- 172.21.15.26:0/3711589260 wait complete. 2024-09-11T23:26:10.201 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.200+0000 7f2b5365b740 20 client.26261 trim_cache size 0 max 16384 2024-09-11T23:26:10.201 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-09-11T23:26:10.201 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "message": "", 2024-09-11T23:26:10.201 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-09-11T23:26:10.202 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-09-11T23:26:10.202 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.201+0000 7f2b508ba640 1 -- 172.21.15.26:0/3590852507 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f2b340a2e20 msgr2=0x7f2b340a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:26:10.202 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.201+0000 7f2b508ba640 1 --2- 172.21.15.26:0/3590852507 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f2b340a2e20 0x7f2b340a52e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x55f43a354260 tx=0x7f2b3c005f90 comp rx=0 tx=0).stop 2024-09-11T23:26:10.203 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.201+0000 7f2b508ba640 1 -- 172.21.15.26:0/3590852507 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2b4c102470 msgr2=0x7f2b4c1988d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:26:10.203 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.201+0000 7f2b508ba640 1 --2- 172.21.15.26:0/3590852507 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2b4c102470 0x7f2b4c1988d0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f2b40009230 tx=0x7f2b40004040 comp rx=0 tx=0).stop 2024-09-11T23:26:10.203 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.201+0000 7f2b508ba640 1 -- 172.21.15.26:0/3590852507 shutdown_connections 2024-09-11T23:26:10.203 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.201+0000 7f2b508ba640 1 --2- 172.21.15.26:0/3590852507 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f2b340a2e20 0x7f2b340a52e0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:10.203 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.201+0000 7f2b508ba640 1 --2- 172.21.15.26:0/3590852507 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2b4c102470 0x7f2b4c1988d0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:10.203 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.202+0000 7f2b508ba640 1 --2- 172.21.15.26:0/3590852507 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2b4c107b70 0x7f2b4c19cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:10.203 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.202+0000 7f2b508ba640 1 --2- 172.21.15.26:0/3590852507 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b4c101ad0 0x7f2b4c198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:10.203 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.202+0000 7f2b508ba640 1 -- 172.21.15.26:0/3590852507 >> 172.21.15.26:0/3590852507 conn(0x7f2b4c0fd7b0 msgr2=0x7f2b4c0fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:26:10.203 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.202+0000 7f2b508ba640 1 -- 172.21.15.26:0/3590852507 shutdown_connections 2024-09-11T23:26:10.203 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:10.202+0000 7f2b508ba640 1 -- 172.21.15.26:0/3590852507 wait complete. 2024-09-11T23:26:10.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:26:09 smithi136 ceph-mon[35121]: pgmap v239: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 129 KiB/s wr, 60 op/s 2024-09-11T23:26:10.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:26:09 smithi026 ceph-mon[31172]: pgmap v239: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 129 KiB/s wr, 60 op/s 2024-09-11T23:26:11.656 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-11T23:26:11.656 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 47, 2024-09-11T23:26:11.656 INFO:teuthology.orchestra.run.smithi026.stdout: "leader": 24485, 2024-09-11T23:26:11.656 INFO:teuthology.orchestra.run.smithi026.stdout: "set_version": 26, 2024-09-11T23:26:11.656 INFO:teuthology.orchestra.run.smithi026.stdout: "sets": { 2024-09-11T23:26:11.656 INFO:teuthology.orchestra.run.smithi026.stdout: "a647e4b1": { 2024-09-11T23:26:11.656 INFO:teuthology.orchestra.run.smithi026.stdout: "version": 26, 2024-09-11T23:26:11.656 INFO:teuthology.orchestra.run.smithi026.stdout: "age_ref": 0.0, 2024-09-11T23:26:11.657 INFO:teuthology.orchestra.run.smithi026.stdout: "state": { 2024-09-11T23:26:11.657 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "QUIESCED", 2024-09-11T23:26:11.657 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.0 2024-09-11T23:26:11.657 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:26:11.657 INFO:teuthology.orchestra.run.smithi026.stdout: "timeout": 90.0, 2024-09-11T23:26:11.657 INFO:teuthology.orchestra.run.smithi026.stdout: "expiration": 136.6, 2024-09-11T23:26:11.657 INFO:teuthology.orchestra.run.smithi026.stdout: "members": { 2024-09-11T23:26:11.657 INFO:teuthology.orchestra.run.smithi026.stdout: "file:/": { 2024-09-11T23:26:11.657 INFO:teuthology.orchestra.run.smithi026.stdout: "excluded": false, 2024-09-11T23:26:11.657 INFO:teuthology.orchestra.run.smithi026.stdout: "state": { 2024-09-11T23:26:11.657 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "QUIESCED", 2024-09-11T23:26:11.657 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.0 2024-09-11T23:26:11.657 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:26:11.657 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:26:11.658 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:26:11.658 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:26:11.658 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:26:11.658 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-11T23:26:11.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:11.655+0000 7f2b24ff9640 1 -- 172.21.15.26:0/212604489 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5620472b6d20 con 0x5620473d3f50 2024-09-11T23:26:11.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:11.655+0000 7f2b24ff9640 10 client.16647 handle_command_reply: tid=0 2024-09-11T23:26:11.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:11.655+0000 7f2b45cb2740 1 client.16647 shutdown 2024-09-11T23:26:11.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:11.655+0000 7f2b45cb2740 1 -- 172.21.15.26:0/212604489 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562047349620 msgr2=0x562047352d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:26:11.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:11.655+0000 7f2b45cb2740 1 --2- 172.21.15.26:0/212604489 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562047349620 0x562047352d10 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f2b38003f80 tx=0x7f2b38005900 comp rx=0 tx=0).stop 2024-09-11T23:26:11.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:11.655+0000 7f2b45cb2740 1 -- 172.21.15.26:0/212604489 shutdown_connections 2024-09-11T23:26:11.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:11.655+0000 7f2b45cb2740 1 --2- 172.21.15.26:0/212604489 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562047349f40 0x562047353250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:11.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:11.655+0000 7f2b45cb2740 1 --2- 172.21.15.26:0/212604489 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5620473d3f50 0x5620473d6370 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:11.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:11.655+0000 7f2b45cb2740 1 --2- 172.21.15.26:0/212604489 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562047358d50 0x562047353790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:11.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:11.655+0000 7f2b45cb2740 1 --2- 172.21.15.26:0/212604489 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562047349620 0x562047352d10 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:11.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:11.655+0000 7f2b45cb2740 1 -- 172.21.15.26:0/212604489 >> 172.21.15.26:0/212604489 conn(0x562047345cf0 msgr2=0x562047344e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:26:11.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:11.655+0000 7f2b24ff9640 0 client.16647 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:26:11.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:11.655+0000 7f2b45cb2740 1 -- 172.21.15.26:0/212604489 shutdown_connections 2024-09-11T23:26:11.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:11.655+0000 7f2b45cb2740 1 -- 172.21.15.26:0/212604489 wait complete. 2024-09-11T23:26:11.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:11.655+0000 7f2b45cb2740 20 client.16647 trim_cache size 0 max 16384 2024-09-11T23:26:11.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:11.656+0000 7f2b438d9640 1 -- 172.21.15.26:0/2662206115 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f2b280a2cf0 msgr2=0x7f2b280a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:26:11.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:11.656+0000 7f2b438d9640 1 --2- 172.21.15.26:0/2662206115 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f2b280a2cf0 0x7f2b280a51b0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f2b34012830 tx=0x7f2b34004ab0 comp rx=0 tx=0).stop 2024-09-11T23:26:11.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:11.656+0000 7f2b438d9640 1 -- 172.21.15.26:0/2662206115 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b3c101420 msgr2=0x7f2b3c199740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:26:11.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:11.656+0000 7f2b438d9640 1 --2- 172.21.15.26:0/2662206115 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b3c101420 0x7f2b3c199740 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f2b2c009a50 tx=0x7f2b2c004270 comp rx=0 tx=0).stop 2024-09-11T23:26:11.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:11.656+0000 7f2b438d9640 1 -- 172.21.15.26:0/2662206115 shutdown_connections 2024-09-11T23:26:11.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:11.656+0000 7f2b438d9640 1 --2- 172.21.15.26:0/2662206115 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f2b280a2cf0 0x7f2b280a51b0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:11.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:11.656+0000 7f2b438d9640 1 --2- 172.21.15.26:0/2662206115 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2b3c106a80 0x7f2b3c1a4280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:11.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:11.656+0000 7f2b438d9640 1 --2- 172.21.15.26:0/2662206115 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b3c101420 0x7f2b3c199740 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:11.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:11.656+0000 7f2b438d9640 1 --2- 172.21.15.26:0/2662206115 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2b3c100a80 0x7f2b3c199200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:11.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:11.656+0000 7f2b438d9640 1 -- 172.21.15.26:0/2662206115 >> 172.21.15.26:0/2662206115 conn(0x7f2b3c0fc7a0 msgr2=0x7f2b3c0fce70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:26:11.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:11.657+0000 7f2b438d9640 1 -- 172.21.15.26:0/2662206115 shutdown_connections 2024-09-11T23:26:11.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:11.657+0000 7f2b438d9640 1 -- 172.21.15.26:0/2662206115 wait complete. 2024-09-11T23:26:12.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:26:11 smithi026 ceph-mon[31172]: pgmap v240: 97 pgs: 16 active+clean+snaptrim, 28 active+clean+snaptrim_wait, 53 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 153 KiB/s wr, 74 op/s 2024-09-11T23:26:12.052 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:26:11 smithi076 ceph-mon[35296]: pgmap v240: 97 pgs: 16 active+clean+snaptrim, 28 active+clean+snaptrim_wait, 53 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 153 KiB/s wr, 74 op/s 2024-09-11T23:26:12.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:26:11 smithi136 ceph-mon[35121]: pgmap v240: 97 pgs: 16 active+clean+snaptrim, 28 active+clean+snaptrim_wait, 53 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 153 KiB/s wr, 74 op/s 2024-09-11T23:26:12.852 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'a647e4b1', quiesce took 14.6 seconds. Will release after: 2.0000000000000018 2024-09-11T23:26:14.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:26:13 smithi136 ceph-mon[35121]: pgmap v241: 97 pgs: 16 active+clean+snaptrim, 28 active+clean+snaptrim_wait, 53 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 151 KiB/s wr, 74 op/s 2024-09-11T23:26:14.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:26:13 smithi026 ceph-mon[31172]: pgmap v241: 97 pgs: 16 active+clean+snaptrim, 28 active+clean+snaptrim_wait, 53 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 151 KiB/s wr, 74 op/s 2024-09-11T23:26:14.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:26:13 smithi076 ceph-mon[35296]: pgmap v241: 97 pgs: 16 active+clean+snaptrim, 28 active+clean+snaptrim_wait, 53 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 151 KiB/s wr, 74 op/s 2024-09-11T23:26:14.853 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:26:15.135 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:26:15.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:15.901+0000 7fee09493640 1 -- 172.21.15.26:0/1850053673 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee04106f10 msgr2=0x7fee0410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:26:15.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:15.901+0000 7fee09493640 1 --2- 172.21.15.26:0/1850053673 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee04106f10 0x7fee0410ea90 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fedfc009ca0 tx=0x7fedfc02a8c0 comp rx=0 tx=0).stop 2024-09-11T23:26:15.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:15.901+0000 7fee09493640 1 -- 172.21.15.26:0/1850053673 shutdown_connections 2024-09-11T23:26:15.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:15.901+0000 7fee09493640 1 --2- 172.21.15.26:0/1850053673 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fee04102790 0x7fee041067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:15.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:15.901+0000 7fee09493640 1 --2- 172.21.15.26:0/1850053673 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee04106f10 0x7fee0410ea90 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:15.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:15.901+0000 7fee09493640 1 --2- 172.21.15.26:0/1850053673 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fee04101de0 0x7fee041021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:15.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:15.901+0000 7fee09493640 1 -- 172.21.15.26:0/1850053673 >> 172.21.15.26:0/1850053673 conn(0x7fee040fd8c0 msgr2=0x7fee040ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:26:15.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:15.901+0000 7fee09493640 1 -- 172.21.15.26:0/1850053673 shutdown_connections 2024-09-11T23:26:15.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:15.902+0000 7fee09493640 1 -- 172.21.15.26:0/1850053673 wait complete. 2024-09-11T23:26:15.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:15.902+0000 7fee09493640 1 Processor -- start 2024-09-11T23:26:15.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:15.902+0000 7fee09493640 1 -- start start 2024-09-11T23:26:15.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:15.903+0000 7fee09493640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fee04101de0 0x7fee04198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:15.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:15.903+0000 7fee09493640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee04102790 0x7fee041989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:15.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:15.903+0000 7fee09493640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fee04106f10 0x7fee0419cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:15.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:15.903+0000 7fee09493640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fee04111fc0 con 0x7fee04102790 2024-09-11T23:26:15.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:15.903+0000 7fee09493640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fee04111e40 con 0x7fee04101de0 2024-09-11T23:26:15.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:15.903+0000 7fee09493640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fee04112140 con 0x7fee04106f10 2024-09-11T23:26:15.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:15.903+0000 7fee027fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee04102790 0x7fee041989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:15.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:15.903+0000 7fee027fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee04102790 0x7fee041989c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35468/0 (socket says 172.21.15.26:35468) 2024-09-11T23:26:15.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:15.903+0000 7fee027fc640 1 -- 172.21.15.26:0/2333874780 learned_addr learned my addr 172.21.15.26:0/2333874780 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:26:15.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:15.903+0000 7fee02ffd640 1 --2- 172.21.15.26:0/2333874780 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fee04101de0 0x7fee04198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:15.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:15.903+0000 7fee037fe640 1 --2- 172.21.15.26:0/2333874780 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fee04106f10 0x7fee0419cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:15.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:15.903+0000 7fee027fc640 1 -- 172.21.15.26:0/2333874780 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fee04106f10 msgr2=0x7fee0419cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:26:15.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:15.903+0000 7fee027fc640 1 --2- 172.21.15.26:0/2333874780 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fee04106f10 0x7fee0419cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:15.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:15.903+0000 7fee027fc640 1 -- 172.21.15.26:0/2333874780 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fee04101de0 msgr2=0x7fee04198480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:26:15.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:15.903+0000 7fee027fc640 1 --2- 172.21.15.26:0/2333874780 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fee04101de0 0x7fee04198480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:15.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:15.903+0000 7fee027fc640 1 -- 172.21.15.26:0/2333874780 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee0419d430 con 0x7fee04102790 2024-09-11T23:26:15.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:15.903+0000 7fee027fc640 1 --2- 172.21.15.26:0/2333874780 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee04102790 0x7fee041989c0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fedf800d950 tx=0x7fedf800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:26:15.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:15.903+0000 7fedebfff640 1 -- 172.21.15.26:0/2333874780 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedf8017030 con 0x7fee04102790 2024-09-11T23:26:15.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:15.903+0000 7fee09493640 1 -- 172.21.15.26:0/2333874780 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee0419d720 con 0x7fee04102790 2024-09-11T23:26:15.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:15.903+0000 7fedebfff640 1 -- 172.21.15.26:0/2333874780 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fedf8004020 con 0x7fee04102790 2024-09-11T23:26:15.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:15.903+0000 7fedebfff640 1 -- 172.21.15.26:0/2333874780 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedf80047a0 con 0x7fee04102790 2024-09-11T23:26:15.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:15.904+0000 7fee09493640 1 -- 172.21.15.26:0/2333874780 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee041a5000 con 0x7fee04102790 2024-09-11T23:26:15.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:15.905+0000 7fee09493640 1 -- 172.21.15.26:0/2333874780 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee041021c0 con 0x7fee04102790 2024-09-11T23:26:15.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:15.906+0000 7fedebfff640 1 -- 172.21.15.26:0/2333874780 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fedf80041d0 con 0x7fee04102790 2024-09-11T23:26:15.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:15.907+0000 7fedebfff640 1 --2- 172.21.15.26:0/2333874780 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fedec0a2cf0 0x7fedec0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:15.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:15.907+0000 7fee02ffd640 1 --2- 172.21.15.26:0/2333874780 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fedec0a2cf0 0x7fedec0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:15.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:15.907+0000 7fedebfff640 1 -- 172.21.15.26:0/2333874780 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8586+0+0 (secure 0 0 0) 0x7fedf80cee80 con 0x7fee04102790 2024-09-11T23:26:15.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:15.907+0000 7fee02ffd640 1 --2- 172.21.15.26:0/2333874780 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fedec0a2cf0 0x7fedec0a51b0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fedf4005d00 tx=0x7fedf400b410 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:26:15.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:15.911+0000 7fedebfff640 1 -- 172.21.15.26:0/2333874780 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fedf809ba00 con 0x7fee04102790 2024-09-11T23:26:16.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:16.126+0000 7fee09493640 1 -- 172.21.15.26:0/2333874780 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fee0410a0f0 con 0x7fee04102790 2024-09-11T23:26:16.126 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:26:15 smithi026 ceph-mon[31172]: pgmap v242: 97 pgs: 3 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 88 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 126 KiB/s wr, 43 op/s 2024-09-11T23:26:16.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:16.127+0000 7fedebfff640 1 -- 172.21.15.26:0/2333874780 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) ==== 94+0+11994 (secure 0 0 0) 0x7fedf809b300 con 0x7fee04102790 2024-09-11T23:26:16.127 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-11T23:26:16.128 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":47,"btime":"2024-09-11T23:25:46:746180+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14499,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/367659515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":367659515},{"type":"v1","addr":"172.21.15.26:6835","nonce":367659515}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14511,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/2041065299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":2041065299},{"type":"v1","addr":"172.21.15.26:6837","nonce":2041065299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":47,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:16:26.544203+0000","modified":"2024-09-11T23:25:46.511698+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24485,"mds_1":24463,"mds_2":24458,"mds_3":24448,"mds_4":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14550":{"gid":14550,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6839/2233068305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":2233068305},{"type":"v1","addr":"172.21.15.26:6839","nonce":2233068305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14562":{"gid":14562,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6841/2984982336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":2984982336},{"type":"v1","addr":"172.21.15.26:6841","nonce":2984982336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6833/2019841455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":2019841455},{"type":"v1","addr":"172.21.15.136:6833","nonce":2019841455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.76:6833/3438147050","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3438147050},{"type":"v1","addr":"172.21.15.76:6833","nonce":3438147050}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6835/3617665975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3617665975},{"type":"v1","addr":"172.21.15.136:6835","nonce":3617665975}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":12,"addr":"172.21.15.76:6835/393308499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":393308499},{"type":"v1","addr":"172.21.15.76:6835","nonce":393308499}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":10,"addr":"172.21.15.136:6837/1633082387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":1633082387},{"type":"v1","addr":"172.21.15.136:6837","nonce":1633082387}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.76:6837/2078978748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":2078978748},{"type":"v1","addr":"172.21.15.76:6837","nonce":2078978748}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":63,"addr":"172.21.15.136:6839/3211398630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":3211398630},{"type":"v1","addr":"172.21.15.136:6839","nonce":3211398630}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.76:6839/2077637090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6838","nonce":2077637090},{"type":"v1","addr":"172.21.15.76:6839","nonce":2077637090}]},"join_fscid":-1,"export_targets":[3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24485,"qdb_cluster":[24443,24448,24458,24463,24485]},"id":1}]} 2024-09-11T23:26:16.128 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 47 2024-09-11T23:26:16.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:16.129+0000 7fee09493640 1 -- 172.21.15.26:0/2333874780 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fedec0a2cf0 msgr2=0x7fedec0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:26:16.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:16.129+0000 7fee09493640 1 --2- 172.21.15.26:0/2333874780 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fedec0a2cf0 0x7fedec0a51b0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fedf4005d00 tx=0x7fedf400b410 comp rx=0 tx=0).stop 2024-09-11T23:26:16.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:16.129+0000 7fee09493640 1 -- 172.21.15.26:0/2333874780 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee04102790 msgr2=0x7fee041989c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:26:16.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:16.129+0000 7fee09493640 1 --2- 172.21.15.26:0/2333874780 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee04102790 0x7fee041989c0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fedf800d950 tx=0x7fedf800de10 comp rx=0 tx=0).stop 2024-09-11T23:26:16.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:16.129+0000 7fee09493640 1 -- 172.21.15.26:0/2333874780 shutdown_connections 2024-09-11T23:26:16.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:16.129+0000 7fee09493640 1 --2- 172.21.15.26:0/2333874780 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fedec0a2cf0 0x7fedec0a51b0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:16.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:16.129+0000 7fee09493640 1 --2- 172.21.15.26:0/2333874780 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fee04106f10 0x7fee0419cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:16.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:16.129+0000 7fee09493640 1 --2- 172.21.15.26:0/2333874780 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee04102790 0x7fee041989c0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:16.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:16.129+0000 7fee09493640 1 --2- 172.21.15.26:0/2333874780 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fee04101de0 0x7fee04198480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:16.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:16.129+0000 7fee09493640 1 -- 172.21.15.26:0/2333874780 >> 172.21.15.26:0/2333874780 conn(0x7fee040fd8c0 msgr2=0x7fee040febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:26:16.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:16.130+0000 7fee09493640 1 -- 172.21.15.26:0/2333874780 shutdown_connections 2024-09-11T23:26:16.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:16.130+0000 7fee09493640 1 -- 172.21.15.26:0/2333874780 wait complete. 2024-09-11T23:26:16.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:26:15 smithi136 ceph-mon[35121]: pgmap v242: 97 pgs: 3 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 88 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 126 KiB/s wr, 43 op/s 2024-09-11T23:26:16.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:26:15 smithi076 ceph-mon[35296]: pgmap v242: 97 pgs: 3 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 88 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 126 KiB/s wr, 43 op/s 2024-09-11T23:26:17.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:26:16 smithi136 ceph-mon[35121]: from='client.? 172.21.15.26:0/2333874780' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:26:17.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:26:16 smithi026 ceph-mon[31172]: from='client.? 172.21.15.26:0/2333874780' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:26:17.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:26:16 smithi076 ceph-mon[35296]: from='client.? 172.21.15.26:0/2333874780' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:26:17.381 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24485 quiesce db --set-id a647e4b1 --release --await' 2024-09-11T23:26:17.381 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24485 quiesce db --set-id a647e4b1 --release --await 2024-09-11T23:26:17.652 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:26:18.048 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:26:17 smithi026 ceph-mon[31172]: pgmap v243: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 126 KiB/s wr, 40 op/s 2024-09-11T23:26:18.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.137+0000 7fcb392a2640 1 -- 172.21.15.26:0/4211928862 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcb341082a0 msgr2=0x7fcb3410ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:26:18.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.137+0000 7fcb392a2640 1 --2- 172.21.15.26:0/4211928862 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcb341082a0 0x7fcb3410ec40 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fcb2c009930 tx=0x7fcb2c02a420 comp rx=0 tx=0).stop 2024-09-11T23:26:18.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.138+0000 7fcb392a2640 1 -- 172.21.15.26:0/4211928862 shutdown_connections 2024-09-11T23:26:18.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.138+0000 7fcb392a2640 1 --2- 172.21.15.26:0/4211928862 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcb341082a0 0x7fcb3410ec40 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:18.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.138+0000 7fcb392a2640 1 --2- 172.21.15.26:0/4211928862 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcb341036e0 0x7fcb34107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:18.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.138+0000 7fcb392a2640 1 --2- 172.21.15.26:0/4211928862 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcb34102d30 0x7fcb34103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:18.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.138+0000 7fcb392a2640 1 -- 172.21.15.26:0/4211928862 >> 172.21.15.26:0/4211928862 conn(0x7fcb340fc7d0 msgr2=0x7fcb340febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:26:18.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.138+0000 7fcb392a2640 1 -- 172.21.15.26:0/4211928862 shutdown_connections 2024-09-11T23:26:18.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.138+0000 7fcb392a2640 1 -- 172.21.15.26:0/4211928862 wait complete. 2024-09-11T23:26:18.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.138+0000 7fcb392a2640 1 Processor -- start 2024-09-11T23:26:18.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.138+0000 7fcb392a2640 1 -- start start 2024-09-11T23:26:18.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.139+0000 7fcb392a2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcb34102d30 0x7fcb341984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:18.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.139+0000 7fcb392a2640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcb341036e0 0x7fcb34198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:18.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.139+0000 7fcb392a2640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcb341082a0 0x7fcb3419cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:18.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.139+0000 7fcb392a2640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fcb341120d0 con 0x7fcb34102d30 2024-09-11T23:26:18.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.139+0000 7fcb392a2640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fcb34111f50 con 0x7fcb341082a0 2024-09-11T23:26:18.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.139+0000 7fcb392a2640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fcb34112250 con 0x7fcb341036e0 2024-09-11T23:26:18.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.139+0000 7fcb32ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcb34102d30 0x7fcb341984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:18.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.139+0000 7fcb32ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcb34102d30 0x7fcb341984c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35492/0 (socket says 172.21.15.26:35492) 2024-09-11T23:26:18.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.139+0000 7fcb32ffd640 1 -- 172.21.15.26:0/547503264 learned_addr learned my addr 172.21.15.26:0/547503264 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:26:18.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.139+0000 7fcb327fc640 1 --2- 172.21.15.26:0/547503264 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcb341036e0 0x7fcb34198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:18.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.139+0000 7fcb337fe640 1 --2- 172.21.15.26:0/547503264 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcb341082a0 0x7fcb3419cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:18.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.139+0000 7fcb32ffd640 1 -- 172.21.15.26:0/547503264 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcb341036e0 msgr2=0x7fcb34198a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:26:18.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.139+0000 7fcb32ffd640 1 --2- 172.21.15.26:0/547503264 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcb341036e0 0x7fcb34198a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:18.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.139+0000 7fcb32ffd640 1 -- 172.21.15.26:0/547503264 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcb341082a0 msgr2=0x7fcb3419cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:26:18.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.139+0000 7fcb32ffd640 1 --2- 172.21.15.26:0/547503264 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcb341082a0 0x7fcb3419cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:18.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.139+0000 7fcb32ffd640 1 -- 172.21.15.26:0/547503264 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb3419d470 con 0x7fcb34102d30 2024-09-11T23:26:18.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.139+0000 7fcb337fe640 1 --2- 172.21.15.26:0/547503264 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcb341082a0 0x7fcb3419cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:26:18.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.139+0000 7fcb32ffd640 1 --2- 172.21.15.26:0/547503264 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcb34102d30 0x7fcb341984c0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fcb2400cce0 tx=0x7fcb2400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:26:18.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.139+0000 7fcb1bfff640 1 -- 172.21.15.26:0/547503264 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb2400f650 con 0x7fcb34102d30 2024-09-11T23:26:18.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.139+0000 7fcb1bfff640 1 -- 172.21.15.26:0/547503264 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb2400fcd0 con 0x7fcb34102d30 2024-09-11T23:26:18.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.139+0000 7fcb392a2640 1 -- 172.21.15.26:0/547503264 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb3419d760 con 0x7fcb34102d30 2024-09-11T23:26:18.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.139+0000 7fcb1bfff640 1 -- 172.21.15.26:0/547503264 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb24018c70 con 0x7fcb34102d30 2024-09-11T23:26:18.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.139+0000 7fcb392a2640 1 -- 172.21.15.26:0/547503264 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb341a5040 con 0x7fcb34102d30 2024-09-11T23:26:18.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.142+0000 7fcb1bfff640 1 -- 172.21.15.26:0/547503264 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fcb2400f7f0 con 0x7fcb34102d30 2024-09-11T23:26:18.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.142+0000 7fcb3b67b740 1 Processor -- start 2024-09-11T23:26:18.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.142+0000 7fcb3b67b740 1 -- start start 2024-09-11T23:26:18.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.142+0000 7fcb3b67b740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5624f0426f60 0x5624f0427340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:18.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.142+0000 7fcb3b67b740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5624f0427880 0x5624f0436110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:18.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.142+0000 7fcb3b67b740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5624f0436650 0x5624f0438a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:18.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.142+0000 7fcb3b67b740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5624f038c620 con 0x5624f0436650 2024-09-11T23:26:18.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.142+0000 7fcb3b67b740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5624f0333e50 con 0x5624f0427880 2024-09-11T23:26:18.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.142+0000 7fcb3b67b740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5624f03885e0 con 0x5624f0426f60 2024-09-11T23:26:18.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.142+0000 7fcb327fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5624f0436650 0x5624f0438a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:18.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.142+0000 7fcb327fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5624f0436650 0x5624f0438a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35496/0 (socket says 172.21.15.26:35496) 2024-09-11T23:26:18.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.142+0000 7fcb327fc640 1 -- 172.21.15.26:0/1039446406 learned_addr learned my addr 172.21.15.26:0/1039446406 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:26:18.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.143+0000 7fcb337fe640 1 --2- 172.21.15.26:0/1039446406 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5624f0426f60 0x5624f0427340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:18.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.143+0000 7fcb1bfff640 1 --2- 172.21.15.26:0/547503264 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fcb1c0a2cf0 0x7fcb1c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:18.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.143+0000 7fcb32ffd640 1 --2- 172.21.15.26:0/1039446406 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5624f0427880 0x5624f0436110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:18.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.143+0000 7fcb327fc640 1 -- 172.21.15.26:0/1039446406 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5624f0426f60 msgr2=0x5624f0427340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:26:18.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.143+0000 7fcb327fc640 1 --2- 172.21.15.26:0/1039446406 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5624f0426f60 0x5624f0427340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:18.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.143+0000 7fcb327fc640 1 -- 172.21.15.26:0/1039446406 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5624f0427880 msgr2=0x5624f0436110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:26:18.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.143+0000 7fcb337fe640 1 --2- 172.21.15.26:0/547503264 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fcb1c0a2cf0 0x7fcb1c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:18.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.143+0000 7fcb327fc640 1 --2- 172.21.15.26:0/1039446406 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5624f0427880 0x5624f0436110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:18.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.143+0000 7fcb327fc640 1 -- 172.21.15.26:0/1039446406 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5624f0324d40 con 0x5624f0436650 2024-09-11T23:26:18.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.143+0000 7fcb1bfff640 1 -- 172.21.15.26:0/547503264 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8586+0+0 (secure 0 0 0) 0x7fcb240cf9f0 con 0x7fcb34102d30 2024-09-11T23:26:18.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.143+0000 7fcb327fc640 1 --2- 172.21.15.26:0/1039446406 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5624f0436650 0x5624f0438a40 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fcb2800dba0 tx=0x7fcb2800f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=255edfd659f7df7d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:26:18.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.143+0000 7fcb197fa640 1 -- 172.21.15.26:0/1039446406 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb2800de90 con 0x5624f0436650 2024-09-11T23:26:18.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.143+0000 7fcb197fa640 1 -- 172.21.15.26:0/1039446406 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb28011920 con 0x5624f0436650 2024-09-11T23:26:18.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.143+0000 7fcb197fa640 1 -- 172.21.15.26:0/1039446406 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb2801b9e0 con 0x5624f0436650 2024-09-11T23:26:18.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.143+0000 7fcb337fe640 1 --2- 172.21.15.26:0/547503264 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fcb1c0a2cf0 0x7fcb1c0a51b0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fcb2c009230 tx=0x7fcb2c02a420 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:26:18.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.144+0000 7fcb3b67b740 1 -- 172.21.15.26:0/1039446406 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5624f0436650 msgr2=0x5624f0438a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:26:18.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.144+0000 7fcb3b67b740 1 --2- 172.21.15.26:0/1039446406 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5624f0436650 0x5624f0438a40 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fcb2800dba0 tx=0x7fcb2800f730 comp rx=0 tx=0).stop 2024-09-11T23:26:18.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.144+0000 7fcb3b67b740 1 -- 172.21.15.26:0/1039446406 shutdown_connections 2024-09-11T23:26:18.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.144+0000 7fcb3b67b740 1 --2- 172.21.15.26:0/1039446406 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5624f0427880 0x5624f0436110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:18.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.144+0000 7fcb3b67b740 1 --2- 172.21.15.26:0/1039446406 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5624f0436650 0x5624f0438a40 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:18.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.144+0000 7fcb3b67b740 1 --2- 172.21.15.26:0/1039446406 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5624f0426f60 0x5624f0427340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:18.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.144+0000 7fcb3b67b740 1 -- 172.21.15.26:0/1039446406 >> 172.21.15.26:0/1039446406 conn(0x5624f04235f0 msgr2=0x5624f04239d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:26:18.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.144+0000 7fcb3b67b740 1 -- 172.21.15.26:0/1039446406 shutdown_connections 2024-09-11T23:26:18.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.144+0000 7fcb3b67b740 1 -- 172.21.15.26:0/1039446406 wait complete. 2024-09-11T23:26:18.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.144+0000 7fcb3b67b740 1 -- start start 2024-09-11T23:26:18.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.145+0000 7fcb3b67b740 1 Processor -- start 2024-09-11T23:26:18.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.145+0000 7fcb3b67b740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5624f0426f60 0x5624f0433600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:18.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.145+0000 7fcb3b67b740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5624f0427880 0x5624f0433b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:18.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.145+0000 7fcb3b67b740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5624f0436650 0x5624f0434080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:18.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.145+0000 7fcb3b67b740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5624f0439280 con 0x5624f0426f60 2024-09-11T23:26:18.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.145+0000 7fcb3b67b740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5624f0314f10 con 0x5624f0427880 2024-09-11T23:26:18.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.145+0000 7fcb3b67b740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5624f0439400 con 0x5624f0436650 2024-09-11T23:26:18.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.145+0000 7fcb3b67b740 10 client.0 _ll_register_callbacks cb 0x5624f0338db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:26:18.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.145+0000 7fcb32ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5624f0426f60 0x5624f0433600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:18.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.145+0000 7fcb32ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5624f0426f60 0x5624f0433600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35510/0 (socket says 172.21.15.26:35510) 2024-09-11T23:26:18.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.145+0000 7fcb32ffd640 1 -- 172.21.15.26:0/1475815482 learned_addr learned my addr 172.21.15.26:0/1475815482 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:26:18.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.145+0000 7fcb327fc640 1 --2- 172.21.15.26:0/1475815482 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5624f0427880 0x5624f0433b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:18.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.145+0000 7fcb32ffd640 1 -- 172.21.15.26:0/1475815482 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5624f0436650 msgr2=0x5624f0434080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:26:18.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.145+0000 7fcb337fe640 1 --2- 172.21.15.26:0/1475815482 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5624f0436650 0x5624f0434080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:18.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.145+0000 7fcb32ffd640 1 --2- 172.21.15.26:0/1475815482 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5624f0436650 0x5624f0434080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:18.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.145+0000 7fcb32ffd640 1 -- 172.21.15.26:0/1475815482 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5624f0427880 msgr2=0x5624f0433b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:26:18.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.145+0000 7fcb32ffd640 1 --2- 172.21.15.26:0/1475815482 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5624f0427880 0x5624f0433b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:18.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.145+0000 7fcb32ffd640 1 -- 172.21.15.26:0/1475815482 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5624f009b580 con 0x5624f0426f60 2024-09-11T23:26:18.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.145+0000 7fcb327fc640 1 --2- 172.21.15.26:0/1475815482 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5624f0427880 0x5624f0433b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:26:18.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.145+0000 7fcb337fe640 1 --2- 172.21.15.26:0/1475815482 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5624f0436650 0x5624f0434080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:26:18.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.146+0000 7fcb32ffd640 1 --2- 172.21.15.26:0/1475815482 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5624f0426f60 0x5624f0433600 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fcb240cd1b0 tx=0x7fcb240184d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17a27cecfae5bd67 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:26:18.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.146+0000 7fcb16ff5640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:26:18.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.146+0000 7fcb16ff5640 1 -- 172.21.15.26:0/1475815482 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb2409c5d0 con 0x5624f0426f60 2024-09-11T23:26:18.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.146+0000 7fcb16ff5640 1 -- 172.21.15.26:0/1475815482 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8586+0+0 (secure 0 0 0) 0x7fcb240a46b0 con 0x5624f0426f60 2024-09-11T23:26:18.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.146+0000 7fcb3b67b740 1 -- 172.21.15.26:0/1475815482 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcb28011920 con 0x5624f0426f60 2024-09-11T23:26:18.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.146+0000 7fcb16ff5640 1 -- 172.21.15.26:0/1475815482 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fcb240a4b40 con 0x5624f0426f60 2024-09-11T23:26:18.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.146+0000 7fcb3b67b740 10 client.16683 fetch_fsmap learned FSMap version 47 2024-09-11T23:26:18.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.146+0000 7fcb3b67b740 1 -- 172.21.15.26:0/1475815482 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x5624f0165960 con 0x5624f0426f60 2024-09-11T23:26:18.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.146+0000 7fcb16ff5640 1 -- 172.21.15.26:0/1475815482 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb28011920 con 0x5624f0426f60 2024-09-11T23:26:18.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.146+0000 7fcb16ff5640 1 -- 172.21.15.26:0/1475815482 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 47) ==== 8414+0+0 (secure 0 0 0) 0x7fcb2401fbb0 con 0x5624f0426f60 2024-09-11T23:26:18.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.146+0000 7fcb3b67b740 10 client.16683 fetch_fsmap finished waiting for FSMap version 47 2024-09-11T23:26:18.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.146+0000 7fcb3b67b740 10 client.16683 resolve_mds: validated gid 24485 aka daemon mds.b 2024-09-11T23:26:18.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.146+0000 7fcb3b67b740 20 client.16683 populate_metadata read hostname 'smithi026' 2024-09-11T23:26:18.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.146+0000 7fcb3b67b740 1 --2- 172.21.15.26:0/1475815482 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5624f0446370 0x5624f0448760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:18.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.146+0000 7fcb3b67b740 4 client.16683 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:26:18.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.146+0000 7fcb3b67b740 1 -- 172.21.15.26:0/1475815482 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5624f0399b90 con 0x5624f0446370 2024-09-11T23:26:18.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.147+0000 7fcb327fc640 1 --2- 172.21.15.26:0/1475815482 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5624f0446370 0x5624f0448760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:18.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.147+0000 7fcb327fc640 1 --2- 172.21.15.26:0/1475815482 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5624f0446370 0x5624f0448760 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=944046f8bee85297 server_cookie=5b99882b0b10c1fc in_seq=0 out_seq=0 2024-09-11T23:26:18.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.147+0000 7fcb16ff5640 10 client.16683 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:26:18.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.148+0000 7fcb16ff5640 1 -- 172.21.15.26:0/1475815482 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5624f0399b90 con 0x5624f0446370 2024-09-11T23:26:18.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.148+0000 7fcb16ff5640 10 client.16683 handle_command_reply: tid=0 2024-09-11T23:26:18.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.148+0000 7fcb3b67b740 1 client.16683 shutdown 2024-09-11T23:26:18.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.149+0000 7fcb3b67b740 1 -- 172.21.15.26:0/1475815482 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5624f0426f60 msgr2=0x5624f0433600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:26:18.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.149+0000 7fcb3b67b740 1 --2- 172.21.15.26:0/1475815482 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5624f0426f60 0x5624f0433600 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fcb240cd1b0 tx=0x7fcb240184d0 comp rx=0 tx=0).stop 2024-09-11T23:26:18.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.149+0000 7fcb3b67b740 1 -- 172.21.15.26:0/1475815482 shutdown_connections 2024-09-11T23:26:18.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.149+0000 7fcb3b67b740 1 --2- 172.21.15.26:0/1475815482 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5624f0427880 0x5624f0433b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:18.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.149+0000 7fcb3b67b740 1 --2- 172.21.15.26:0/1475815482 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5624f0446370 0x5624f0448760 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:18.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.149+0000 7fcb3b67b740 1 --2- 172.21.15.26:0/1475815482 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5624f0436650 0x5624f0434080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:18.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.149+0000 7fcb3b67b740 1 --2- 172.21.15.26:0/1475815482 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5624f0426f60 0x5624f0433600 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:18.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.149+0000 7fcb16ff5640 0 client.16683 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:26:18.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.149+0000 7fcb3b67b740 1 -- 172.21.15.26:0/1475815482 >> 172.21.15.26:0/1475815482 conn(0x5624f04235f0 msgr2=0x5624f0429350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:26:18.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.149+0000 7fcb3b67b740 1 -- 172.21.15.26:0/1475815482 shutdown_connections 2024-09-11T23:26:18.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.149+0000 7fcb3b67b740 1 -- 172.21.15.26:0/1475815482 wait complete. 2024-09-11T23:26:18.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.149+0000 7fcb3b67b740 20 client.16683 trim_cache size 0 max 16384 2024-09-11T23:26:18.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.166+0000 7fcb3b67b740 1 Processor -- start 2024-09-11T23:26:18.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.166+0000 7fcb3b67b740 1 -- start start 2024-09-11T23:26:18.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.167+0000 7fcb3b67b740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5624f0426f60 0x5624f0429240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:18.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.167+0000 7fcb3b67b740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5624f0427880 0x5624f044e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:18.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.167+0000 7fcb3b67b740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5624f0436650 0x5624f044e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:18.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.167+0000 7fcb3b67b740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fcb341ab790 con 0x5624f0436650 2024-09-11T23:26:18.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.167+0000 7fcb3b67b740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fcb341ad540 con 0x5624f0426f60 2024-09-11T23:26:18.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.167+0000 7fcb3b67b740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fcb341abf20 con 0x5624f0427880 2024-09-11T23:26:18.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.167+0000 7fcb337fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5624f0436650 0x5624f044e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:18.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.167+0000 7fcb337fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5624f0436650 0x5624f044e540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35526/0 (socket says 172.21.15.26:35526) 2024-09-11T23:26:18.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.167+0000 7fcb337fe640 1 -- 172.21.15.26:0/3098708154 learned_addr learned my addr 172.21.15.26:0/3098708154 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:26:18.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.167+0000 7fcb32ffd640 1 --2- 172.21.15.26:0/3098708154 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5624f0426f60 0x5624f0429240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:18.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.167+0000 7fcb327fc640 1 --2- 172.21.15.26:0/3098708154 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5624f0427880 0x5624f044e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:18.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.167+0000 7fcb337fe640 1 -- 172.21.15.26:0/3098708154 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5624f0427880 msgr2=0x5624f044e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:26:18.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.167+0000 7fcb337fe640 1 --2- 172.21.15.26:0/3098708154 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5624f0427880 0x5624f044e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:18.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.167+0000 7fcb337fe640 1 -- 172.21.15.26:0/3098708154 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5624f0426f60 msgr2=0x5624f0429240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:26:18.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.167+0000 7fcb337fe640 1 --2- 172.21.15.26:0/3098708154 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5624f0426f60 0x5624f0429240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:18.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.167+0000 7fcb337fe640 1 -- 172.21.15.26:0/3098708154 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5624f0423090 con 0x5624f0436650 2024-09-11T23:26:18.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.167+0000 7fcb327fc640 1 --2- 172.21.15.26:0/3098708154 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5624f0427880 0x5624f044e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:26:18.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.167+0000 7fcb32ffd640 1 --2- 172.21.15.26:0/3098708154 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5624f0426f60 0x5624f0429240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:26:18.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.167+0000 7fcb337fe640 1 --2- 172.21.15.26:0/3098708154 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5624f0436650 0x5624f044e540 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fcb2c009930 tx=0x7fcb2c035000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d61ab3dd8de967d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:26:18.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.168+0000 7fcb18ff9640 1 -- 172.21.15.26:0/3098708154 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb2c050070 con 0x5624f0436650 2024-09-11T23:26:18.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.168+0000 7fcb18ff9640 1 -- 172.21.15.26:0/3098708154 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb2c004da0 con 0x5624f0436650 2024-09-11T23:26:18.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.168+0000 7fcb18ff9640 1 -- 172.21.15.26:0/3098708154 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb2c043070 con 0x5624f0436650 2024-09-11T23:26:18.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.168+0000 7fcb3b67b740 1 -- 172.21.15.26:0/3098708154 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5624f0436650 msgr2=0x5624f044e540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:26:18.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.168+0000 7fcb3b67b740 1 --2- 172.21.15.26:0/3098708154 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5624f0436650 0x5624f044e540 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fcb2c009930 tx=0x7fcb2c035000 comp rx=0 tx=0).stop 2024-09-11T23:26:18.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.168+0000 7fcb3b67b740 1 -- 172.21.15.26:0/3098708154 shutdown_connections 2024-09-11T23:26:18.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.168+0000 7fcb3b67b740 1 --2- 172.21.15.26:0/3098708154 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5624f0436650 0x5624f044e540 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:18.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.168+0000 7fcb3b67b740 1 --2- 172.21.15.26:0/3098708154 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5624f0427880 0x5624f044e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:18.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.168+0000 7fcb3b67b740 1 --2- 172.21.15.26:0/3098708154 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5624f0426f60 0x5624f0429240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:18.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.168+0000 7fcb3b67b740 1 -- 172.21.15.26:0/3098708154 >> 172.21.15.26:0/3098708154 conn(0x5624f04235f0 msgr2=0x5624f0428b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:26:18.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.169+0000 7fcb3b67b740 1 -- 172.21.15.26:0/3098708154 shutdown_connections 2024-09-11T23:26:18.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.169+0000 7fcb3b67b740 1 -- 172.21.15.26:0/3098708154 wait complete. 2024-09-11T23:26:18.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.169+0000 7fcb3b67b740 1 -- start start 2024-09-11T23:26:18.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.169+0000 7fcb3b67b740 1 Processor -- start 2024-09-11T23:26:18.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.169+0000 7fcb3b67b740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5624f0426f60 0x5624f043aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:18.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.169+0000 7fcb3b67b740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5624f0427880 0x5624f043b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:18.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.169+0000 7fcb3b67b740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5624f0436650 0x5624f04affd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:18.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.169+0000 7fcb3b67b740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fcb341ace40 con 0x5624f0436650 2024-09-11T23:26:18.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.169+0000 7fcb3b67b740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fcb341ac6b0 con 0x5624f0427880 2024-09-11T23:26:18.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.169+0000 7fcb3b67b740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fcb341a9790 con 0x5624f0426f60 2024-09-11T23:26:18.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.169+0000 7fcb3b67b740 10 client.0 _ll_register_callbacks cb 0x7fcb24007870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:26:18.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.169+0000 7fcb337fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5624f0436650 0x5624f04affd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:18.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.169+0000 7fcb337fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5624f0436650 0x5624f04affd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35538/0 (socket says 172.21.15.26:35538) 2024-09-11T23:26:18.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.169+0000 7fcb337fe640 1 -- 172.21.15.26:0/1771095647 learned_addr learned my addr 172.21.15.26:0/1771095647 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:26:18.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.169+0000 7fcb32ffd640 1 --2- 172.21.15.26:0/1771095647 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5624f0426f60 0x5624f043aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:18.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.170+0000 7fcb327fc640 1 --2- 172.21.15.26:0/1771095647 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5624f0427880 0x5624f043b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:18.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.170+0000 7fcb337fe640 1 -- 172.21.15.26:0/1771095647 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5624f0426f60 msgr2=0x5624f043aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:26:18.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.170+0000 7fcb337fe640 1 --2- 172.21.15.26:0/1771095647 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5624f0426f60 0x5624f043aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:18.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.170+0000 7fcb337fe640 1 -- 172.21.15.26:0/1771095647 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5624f0427880 msgr2=0x5624f043b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:26:18.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.170+0000 7fcb337fe640 1 --2- 172.21.15.26:0/1771095647 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5624f0427880 0x5624f043b0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:18.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.170+0000 7fcb337fe640 1 -- 172.21.15.26:0/1771095647 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5624f04b0540 con 0x5624f0436650 2024-09-11T23:26:18.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.170+0000 7fcb327fc640 1 --2- 172.21.15.26:0/1771095647 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5624f0427880 0x5624f043b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:26:18.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.170+0000 7fcb337fe640 1 --2- 172.21.15.26:0/1771095647 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5624f0436650 0x5624f04affd0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fcb2c02a940 tx=0x7fcb2c0042b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ca181da8c241055 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:26:18.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.170+0000 7fcb16ff5640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:26:18.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.170+0000 7fcb16ff5640 1 -- 172.21.15.26:0/1771095647 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb2c058020 con 0x5624f0436650 2024-09-11T23:26:18.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.170+0000 7fcb16ff5640 1 -- 172.21.15.26:0/1771095647 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8586+0+0 (secure 0 0 0) 0x7fcb2c0048c0 con 0x5624f0436650 2024-09-11T23:26:18.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.170+0000 7fcb3b67b740 1 -- 172.21.15.26:0/1771095647 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcb2c004da0 con 0x5624f0436650 2024-09-11T23:26:18.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.170+0000 7fcb16ff5640 1 -- 172.21.15.26:0/1771095647 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fcb2c063050 con 0x5624f0436650 2024-09-11T23:26:18.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.170+0000 7fcb3b67b740 10 client.16689 fetch_fsmap learned FSMap version 47 2024-09-11T23:26:18.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.170+0000 7fcb3b67b740 1 -- 172.21.15.26:0/1771095647 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x5624f04b1860 con 0x5624f0436650 2024-09-11T23:26:18.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.171+0000 7fcb16ff5640 1 -- 172.21.15.26:0/1771095647 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb2c004da0 con 0x5624f0436650 2024-09-11T23:26:18.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.171+0000 7fcb16ff5640 1 -- 172.21.15.26:0/1771095647 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 47) ==== 8414+0+0 (secure 0 0 0) 0x7fcb2c053030 con 0x5624f0436650 2024-09-11T23:26:18.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.171+0000 7fcb3b67b740 10 client.16689 fetch_fsmap finished waiting for FSMap version 47 2024-09-11T23:26:18.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.171+0000 7fcb3b67b740 10 client.16689 resolve_mds: validated gid 24485 aka daemon mds.b 2024-09-11T23:26:18.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.171+0000 7fcb3b67b740 20 client.16689 populate_metadata read hostname 'smithi026' 2024-09-11T23:26:18.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.171+0000 7fcb3b67b740 1 --2- 172.21.15.26:0/1771095647 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5624f04b1fb0 0x5624f04b43d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:18.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.171+0000 7fcb3b67b740 4 client.16689 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "a647e4b1", "release": true, "await": true}] 2024-09-11T23:26:18.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.171+0000 7fcb3b67b740 1 -- 172.21.15.26:0/1771095647 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "quiesce db", "set_id": "a647e4b1", "release": true, "await": true}) -- 0x5624f0434410 con 0x5624f04b1fb0 2024-09-11T23:26:18.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.171+0000 7fcb32ffd640 1 --2- 172.21.15.26:0/1771095647 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5624f04b1fb0 0x5624f04b43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:18.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.172+0000 7fcb32ffd640 1 --2- 172.21.15.26:0/1771095647 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5624f04b1fb0 0x5624f04b43d0 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fda090e9fa9577ef server_cookie=b9b9764597248c49 in_seq=0 out_seq=0 2024-09-11T23:26:18.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:18.172+0000 7fcb16ff5640 10 client.16689 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:26:18.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:26:17 smithi136 ceph-mon[35121]: pgmap v243: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 126 KiB/s wr, 40 op/s 2024-09-11T23:26:18.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:26:17 smithi076 ceph-mon[35296]: pgmap v243: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 126 KiB/s wr, 40 op/s 2024-09-11T23:26:19.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:19.484+0000 7fcb16ff5640 1 -- 172.21.15.26:0/1771095647 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5624f0434410 con 0x5624f04b1fb0 2024-09-11T23:26:19.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:19.484+0000 7fcb16ff5640 10 client.16689 handle_command_reply: tid=0 2024-09-11T23:26:19.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:19.484+0000 7fcb3b67b740 1 client.16689 shutdown 2024-09-11T23:26:19.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:19.485+0000 7fcb3b67b740 1 -- 172.21.15.26:0/1771095647 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5624f0436650 msgr2=0x5624f04affd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:26:19.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:19.485+0000 7fcb3b67b740 1 --2- 172.21.15.26:0/1771095647 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5624f0436650 0x5624f04affd0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fcb2c02a940 tx=0x7fcb2c0042b0 comp rx=0 tx=0).stop 2024-09-11T23:26:19.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:19.485+0000 7fcb3b67b740 1 -- 172.21.15.26:0/1771095647 shutdown_connections 2024-09-11T23:26:19.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:19.485+0000 7fcb3b67b740 1 --2- 172.21.15.26:0/1771095647 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5624f0427880 0x5624f043b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:19.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:19.485+0000 7fcb3b67b740 1 --2- 172.21.15.26:0/1771095647 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5624f04b1fb0 0x5624f04b43d0 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:19.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:19.485+0000 7fcb3b67b740 1 --2- 172.21.15.26:0/1771095647 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5624f0436650 0x5624f04affd0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:19.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:19.485+0000 7fcb3b67b740 1 --2- 172.21.15.26:0/1771095647 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5624f0426f60 0x5624f043aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:19.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:19.485+0000 7fcb16ff5640 0 client.16689 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:26:19.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:19.485+0000 7fcb3b67b740 1 -- 172.21.15.26:0/1771095647 >> 172.21.15.26:0/1771095647 conn(0x5624f04235f0 msgr2=0x5624f0422750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:26:19.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:19.485+0000 7fcb3b67b740 1 -- 172.21.15.26:0/1771095647 shutdown_connections 2024-09-11T23:26:19.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:19.485+0000 7fcb3b67b740 1 -- 172.21.15.26:0/1771095647 wait complete. 2024-09-11T23:26:19.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:19.485+0000 7fcb3b67b740 20 client.16689 trim_cache size 0 max 16384 2024-09-11T23:26:19.487 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-11T23:26:19.487 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 47, 2024-09-11T23:26:19.487 INFO:teuthology.orchestra.run.smithi026.stdout: "leader": 24485, 2024-09-11T23:26:19.487 INFO:teuthology.orchestra.run.smithi026.stdout: "set_version": 28, 2024-09-11T23:26:19.487 INFO:teuthology.orchestra.run.smithi026.stdout: "sets": { 2024-09-11T23:26:19.487 INFO:teuthology.orchestra.run.smithi026.stdout: "a647e4b1": { 2024-09-11T23:26:19.487 INFO:teuthology.orchestra.run.smithi026.stdout: "version": 28, 2024-09-11T23:26:19.487 INFO:teuthology.orchestra.run.smithi026.stdout: "age_ref": 0.0, 2024-09-11T23:26:19.487 INFO:teuthology.orchestra.run.smithi026.stdout: "state": { 2024-09-11T23:26:19.487 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "RELEASED", 2024-09-11T23:26:19.487 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.0 2024-09-11T23:26:19.487 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:26:19.488 INFO:teuthology.orchestra.run.smithi026.stdout: "timeout": 90.0, 2024-09-11T23:26:19.488 INFO:teuthology.orchestra.run.smithi026.stdout: "expiration": 136.6, 2024-09-11T23:26:19.488 INFO:teuthology.orchestra.run.smithi026.stdout: "members": { 2024-09-11T23:26:19.488 INFO:teuthology.orchestra.run.smithi026.stdout: "file:/": { 2024-09-11T23:26:19.488 INFO:teuthology.orchestra.run.smithi026.stdout: "excluded": false, 2024-09-11T23:26:19.488 INFO:teuthology.orchestra.run.smithi026.stdout: "state": { 2024-09-11T23:26:19.488 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "RELEASED", 2024-09-11T23:26:19.488 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.0 2024-09-11T23:26:19.488 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:26:19.488 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:26:19.488 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:26:19.488 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:26:19.489 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:26:19.489 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-11T23:26:19.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:19.486+0000 7fcb392a2640 1 -- 172.21.15.26:0/547503264 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fcb1c0a2cf0 msgr2=0x7fcb1c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:26:19.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:19.486+0000 7fcb392a2640 1 --2- 172.21.15.26:0/547503264 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fcb1c0a2cf0 0x7fcb1c0a51b0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fcb2c009230 tx=0x7fcb2c02a420 comp rx=0 tx=0).stop 2024-09-11T23:26:19.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:19.486+0000 7fcb392a2640 1 -- 172.21.15.26:0/547503264 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcb34102d30 msgr2=0x7fcb341984c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:26:19.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:19.486+0000 7fcb392a2640 1 --2- 172.21.15.26:0/547503264 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcb34102d30 0x7fcb341984c0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fcb2400cce0 tx=0x7fcb2400a620 comp rx=0 tx=0).stop 2024-09-11T23:26:19.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:19.486+0000 7fcb392a2640 1 -- 172.21.15.26:0/547503264 shutdown_connections 2024-09-11T23:26:19.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:19.486+0000 7fcb392a2640 1 --2- 172.21.15.26:0/547503264 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fcb1c0a2cf0 0x7fcb1c0a51b0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:19.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:19.486+0000 7fcb392a2640 1 --2- 172.21.15.26:0/547503264 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcb341082a0 0x7fcb3419cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:19.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:19.486+0000 7fcb392a2640 1 --2- 172.21.15.26:0/547503264 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcb341036e0 0x7fcb34198a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:19.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:19.486+0000 7fcb392a2640 1 --2- 172.21.15.26:0/547503264 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcb34102d30 0x7fcb341984c0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:19.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:19.486+0000 7fcb392a2640 1 -- 172.21.15.26:0/547503264 >> 172.21.15.26:0/547503264 conn(0x7fcb340fc7d0 msgr2=0x7fcb340fe3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:26:19.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:19.486+0000 7fcb392a2640 1 -- 172.21.15.26:0/547503264 shutdown_connections 2024-09-11T23:26:19.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:19.486+0000 7fcb392a2640 1 -- 172.21.15.26:0/547503264 wait complete. 2024-09-11T23:26:20.103 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:26:19 smithi076 ceph-mon[35296]: pgmap v244: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 109 KiB/s wr, 35 op/s 2024-09-11T23:26:20.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:26:19 smithi136 ceph-mon[35121]: pgmap v244: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 109 KiB/s wr, 35 op/s 2024-09-11T23:26:20.246 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:26:19 smithi026 ceph-mon[31172]: pgmap v244: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 109 KiB/s wr, 35 op/s 2024-09-11T23:26:20.311 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'a647e4b1', total seconds elapsed: 22.0 2024-09-11T23:26:21.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:26:21 smithi136 ceph-mon[35121]: pgmap v245: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 110 KiB/s wr, 39 op/s 2024-09-11T23:26:21.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:26:21 smithi136 ceph-mon[35121]: osdmap e90: 12 total, 12 up, 12 in 2024-09-11T23:26:22.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:26:21 smithi026 ceph-mon[31172]: pgmap v245: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 110 KiB/s wr, 39 op/s 2024-09-11T23:26:22.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:26:21 smithi026 ceph-mon[31172]: osdmap e90: 12 total, 12 up, 12 in 2024-09-11T23:26:22.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:26:21 smithi076 ceph-mon[35296]: pgmap v245: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 110 KiB/s wr, 39 op/s 2024-09-11T23:26:22.077 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:26:21 smithi076 ceph-mon[35296]: osdmap e90: 12 total, 12 up, 12 in 2024-09-11T23:26:23.309 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.308+0000 7fb4fbe5c640 1 -- 172.21.15.26:0/3233507824 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb4f4102470 msgr2=0x7fb4f410e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:26:23.309 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.308+0000 7fb4fbe5c640 1 --2- 172.21.15.26:0/3233507824 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb4f4102470 0x7fb4f410e9a0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fb4ec009cd0 tx=0x7fb4ec02a8e0 comp rx=0 tx=0).stop 2024-09-11T23:26:23.309 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.308+0000 7fb4fbe5c640 1 -- 172.21.15.26:0/3233507824 shutdown_connections 2024-09-11T23:26:23.309 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.308+0000 7fb4fbe5c640 1 --2- 172.21.15.26:0/3233507824 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb4f4101ad0 0x7fb4f4101f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:23.309 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.308+0000 7fb4fbe5c640 1 --2- 172.21.15.26:0/3233507824 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb4f4102470 0x7fb4f410e9a0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:23.309 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.308+0000 7fb4fbe5c640 1 --2- 172.21.15.26:0/3233507824 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb4f4107b70 0x7fb4f4107f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:23.309 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.308+0000 7fb4fbe5c640 1 -- 172.21.15.26:0/3233507824 >> 172.21.15.26:0/3233507824 conn(0x7fb4f40fd7d0 msgr2=0x7fb4f40ffbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:26:23.309 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.309+0000 7fb4fbe5c640 1 -- 172.21.15.26:0/3233507824 shutdown_connections 2024-09-11T23:26:23.310 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.309+0000 7fb4fbe5c640 1 -- 172.21.15.26:0/3233507824 wait complete. 2024-09-11T23:26:23.310 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.309+0000 7fb4fbe5c640 1 Processor -- start 2024-09-11T23:26:23.310 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.309+0000 7fb4fbe5c640 1 -- start start 2024-09-11T23:26:23.311 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.310+0000 7fb4fbe5c640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb4f4101ad0 0x7fb4f4198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:23.312 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.310+0000 7fb4fbe5c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb4f4102470 0x7fb4f4198850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:23.312 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.310+0000 7fb4fbe5c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb4f4107b70 0x7fb4f419cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:23.312 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.310+0000 7fb4fbe5c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb4f4111ee0 con 0x7fb4f4102470 2024-09-11T23:26:23.312 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.310+0000 7fb4fbe5c640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fb4f4111d60 con 0x7fb4f4107b70 2024-09-11T23:26:23.312 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.310+0000 7fb4fbe5c640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb4f4112060 con 0x7fb4f4101ad0 2024-09-11T23:26:23.312 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.310+0000 7fb4f93f7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb4f4102470 0x7fb4f4198850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:23.312 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.310+0000 7fb4f93f7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb4f4102470 0x7fb4f4198850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36392/0 (socket says 172.21.15.26:36392) 2024-09-11T23:26:23.312 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.310+0000 7fb4f93f7640 1 -- 172.21.15.26:0/1422643827 learned_addr learned my addr 172.21.15.26:0/1422643827 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:26:23.312 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.310+0000 7fb4fa3f9640 1 --2- 172.21.15.26:0/1422643827 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb4f4107b70 0x7fb4f419cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:23.313 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.310+0000 7fb4f9bf8640 1 --2- 172.21.15.26:0/1422643827 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb4f4101ad0 0x7fb4f4198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:23.313 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.310+0000 7fb4f93f7640 1 -- 172.21.15.26:0/1422643827 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb4f4101ad0 msgr2=0x7fb4f4198310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:26:23.313 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.310+0000 7fb4f93f7640 1 --2- 172.21.15.26:0/1422643827 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb4f4101ad0 0x7fb4f4198310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:23.313 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.310+0000 7fb4f93f7640 1 -- 172.21.15.26:0/1422643827 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb4f4107b70 msgr2=0x7fb4f419cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:26:23.313 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.310+0000 7fb4f93f7640 1 --2- 172.21.15.26:0/1422643827 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb4f4107b70 0x7fb4f419cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:23.313 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.310+0000 7fb4f93f7640 1 -- 172.21.15.26:0/1422643827 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4f419d300 con 0x7fb4f4102470 2024-09-11T23:26:23.313 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.310+0000 7fb4f93f7640 1 --2- 172.21.15.26:0/1422643827 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb4f4102470 0x7fb4f4198850 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fb4e400d950 tx=0x7fb4e400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:26:23.313 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.310+0000 7fb4eaffd640 1 -- 172.21.15.26:0/1422643827 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4e4017030 con 0x7fb4f4102470 2024-09-11T23:26:23.313 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.311+0000 7fb4eaffd640 1 -- 172.21.15.26:0/1422643827 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4e4004020 con 0x7fb4f4102470 2024-09-11T23:26:23.313 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.311+0000 7fb4fbe5c640 1 -- 172.21.15.26:0/1422643827 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4f419d5f0 con 0x7fb4f4102470 2024-09-11T23:26:23.313 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.311+0000 7fb4eaffd640 1 -- 172.21.15.26:0/1422643827 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4e40047a0 con 0x7fb4f4102470 2024-09-11T23:26:23.313 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.311+0000 7fb4fbe5c640 1 -- 172.21.15.26:0/1422643827 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4f41a4e80 con 0x7fb4f4102470 2024-09-11T23:26:23.316 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.313+0000 7fb4eaffd640 1 -- 172.21.15.26:0/1422643827 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fb4e40041d0 con 0x7fb4f4102470 2024-09-11T23:26:23.316 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.313+0000 7fb4febf4740 1 Processor -- start 2024-09-11T23:26:23.316 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.314+0000 7fb4febf4740 1 -- start start 2024-09-11T23:26:23.316 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.314+0000 7fb4febf4740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55707feda9f0 0x55707fedadd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:23.317 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.314+0000 7fb4febf4740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55707fedb310 0x55707fee9b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:23.317 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.314+0000 7fb4febf4740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55707feea0c0 0x55707feec4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:23.317 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.314+0000 7fb4febf4740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55707fe3f2f0 con 0x55707feda9f0 2024-09-11T23:26:23.317 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.314+0000 7fb4febf4740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55707fde7270 con 0x55707fedb310 2024-09-11T23:26:23.317 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.314+0000 7fb4febf4740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55707fe3b2b0 con 0x55707feea0c0 2024-09-11T23:26:23.317 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.314+0000 7fb4fa3f9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55707feda9f0 0x55707fedadd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:23.317 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.314+0000 7fb4fa3f9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55707feda9f0 0x55707fedadd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36404/0 (socket says 172.21.15.26:36404) 2024-09-11T23:26:23.317 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.314+0000 7fb4fa3f9640 1 -- 172.21.15.26:0/1289043 learned_addr learned my addr 172.21.15.26:0/1289043 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:26:23.317 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.314+0000 7fb4f9bf8640 1 --2- 172.21.15.26:0/1289043 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55707fedb310 0x55707fee9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:23.317 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.314+0000 7fb4f93f7640 1 --2- 172.21.15.26:0/1289043 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55707feea0c0 0x55707feec4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:23.317 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.314+0000 7fb4fa3f9640 1 -- 172.21.15.26:0/1289043 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55707feea0c0 msgr2=0x55707feec4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:26:23.317 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.314+0000 7fb4fa3f9640 1 --2- 172.21.15.26:0/1289043 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55707feea0c0 0x55707feec4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:23.318 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.314+0000 7fb4fa3f9640 1 -- 172.21.15.26:0/1289043 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55707fedb310 msgr2=0x55707fee9b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:26:23.318 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.314+0000 7fb4fa3f9640 1 --2- 172.21.15.26:0/1289043 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55707fedb310 0x55707fee9b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:23.318 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.314+0000 7fb4fa3f9640 1 -- 172.21.15.26:0/1289043 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55707fde6340 con 0x55707feda9f0 2024-09-11T23:26:23.318 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.314+0000 7fb4f9bf8640 1 --2- 172.21.15.26:0/1289043 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55707fedb310 0x55707fee9b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:26:23.318 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.315+0000 7fb4eaffd640 1 --2- 172.21.15.26:0/1422643827 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fb4e00a2da0 0x7fb4e00a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:23.318 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.315+0000 7fb4fa3f9640 1 --2- 172.21.15.26:0/1289043 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55707feda9f0 0x55707fedadd0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fb4ec004ab0 tx=0x7fb4ec009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=342766e03d095731 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:26:23.318 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.315+0000 7fb4f9bf8640 1 --2- 172.21.15.26:0/1422643827 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fb4e00a2da0 0x7fb4e00a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:23.318 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.315+0000 7fb4eaffd640 1 -- 172.21.15.26:0/1422643827 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8542+0+0 (secure 0 0 0) 0x7fb4e40cee20 con 0x7fb4f4102470 2024-09-11T23:26:23.318 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.315+0000 7fb4dffff640 1 -- 172.21.15.26:0/1289043 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4ec048070 con 0x55707feda9f0 2024-09-11T23:26:23.318 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.315+0000 7fb4dffff640 1 -- 172.21.15.26:0/1289043 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4ec036070 con 0x55707feda9f0 2024-09-11T23:26:23.318 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.315+0000 7fb4dffff640 1 -- 172.21.15.26:0/1289043 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4ec004520 con 0x55707feda9f0 2024-09-11T23:26:23.318 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.316+0000 7fb4febf4740 1 -- 172.21.15.26:0/1289043 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55707feda9f0 msgr2=0x55707fedadd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:26:23.319 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.316+0000 7fb4f9bf8640 1 --2- 172.21.15.26:0/1422643827 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fb4e00a2da0 0x7fb4e00a5260 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fb4f0005fd0 tx=0x7fb4f0005d10 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:26:23.319 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.316+0000 7fb4febf4740 1 --2- 172.21.15.26:0/1289043 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55707feda9f0 0x55707fedadd0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fb4ec004ab0 tx=0x7fb4ec009a20 comp rx=0 tx=0).stop 2024-09-11T23:26:23.319 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.316+0000 7fb4febf4740 1 -- 172.21.15.26:0/1289043 shutdown_connections 2024-09-11T23:26:23.319 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.316+0000 7fb4febf4740 1 --2- 172.21.15.26:0/1289043 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55707fedb310 0x55707fee9b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:23.319 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.316+0000 7fb4febf4740 1 --2- 172.21.15.26:0/1289043 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55707feea0c0 0x55707feec4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:23.319 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.316+0000 7fb4febf4740 1 --2- 172.21.15.26:0/1289043 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55707feda9f0 0x55707fedadd0 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:23.319 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.316+0000 7fb4febf4740 1 -- 172.21.15.26:0/1289043 >> 172.21.15.26:0/1289043 conn(0x55707fed7060 msgr2=0x55707fed7440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:26:23.319 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.316+0000 7fb4febf4740 1 -- 172.21.15.26:0/1289043 shutdown_connections 2024-09-11T23:26:23.319 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.316+0000 7fb4febf4740 1 -- 172.21.15.26:0/1289043 wait complete. 2024-09-11T23:26:23.319 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.316+0000 7fb4febf4740 1 -- start start 2024-09-11T23:26:23.319 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.316+0000 7fb4febf4740 1 Processor -- start 2024-09-11T23:26:23.323 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.317+0000 7fb4febf4740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55707feda9f0 0x55707fee6fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:23.323 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.317+0000 7fb4febf4740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55707fedb310 0x55707fee74f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:23.323 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.317+0000 7fb4febf4740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55707feea0c0 0x55707fee7a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:23.323 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.317+0000 7fb4febf4740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55707feeccc0 con 0x55707feda9f0 2024-09-11T23:26:23.323 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.317+0000 7fb4febf4740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55707fdc8290 con 0x55707feea0c0 2024-09-11T23:26:23.323 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.317+0000 7fb4febf4740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55707feece40 con 0x55707fedb310 2024-09-11T23:26:23.323 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.317+0000 7fb4febf4740 10 client.0 _ll_register_callbacks cb 0x55707fdf9140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:26:23.323 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.317+0000 7fb4f9bf8640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55707feda9f0 0x55707fee6fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:23.324 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.317+0000 7fb4f9bf8640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55707feda9f0 0x55707fee6fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36420/0 (socket says 172.21.15.26:36420) 2024-09-11T23:26:23.324 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.317+0000 7fb4f9bf8640 1 -- 172.21.15.26:0/1993872146 learned_addr learned my addr 172.21.15.26:0/1993872146 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:26:23.324 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.317+0000 7fb4f93f7640 1 --2- 172.21.15.26:0/1993872146 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55707fedb310 0x55707fee74f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:23.324 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.317+0000 7fb4fa3f9640 1 --2- 172.21.15.26:0/1993872146 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55707feea0c0 0x55707fee7a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:23.324 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.317+0000 7fb4f9bf8640 1 -- 172.21.15.26:0/1993872146 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55707fedb310 msgr2=0x55707fee74f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:26:23.324 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.317+0000 7fb4f9bf8640 1 --2- 172.21.15.26:0/1993872146 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55707fedb310 0x55707fee74f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:23.324 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.317+0000 7fb4f9bf8640 1 -- 172.21.15.26:0/1993872146 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55707feea0c0 msgr2=0x55707fee7a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:26:23.324 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.317+0000 7fb4f9bf8640 1 --2- 172.21.15.26:0/1993872146 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55707feea0c0 0x55707fee7a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:23.324 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.317+0000 7fb4f9bf8640 1 -- 172.21.15.26:0/1993872146 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55707fb52210 con 0x55707feda9f0 2024-09-11T23:26:23.324 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.318+0000 7fb4f9bf8640 1 --2- 172.21.15.26:0/1993872146 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55707feda9f0 0x55707fee6fb0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fb4f0005f80 tx=0x7fb4f00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c34842086cd3eaa server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:26:23.324 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.318+0000 7fb4dd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:26:23.325 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.318+0000 7fb4dd7fa640 1 -- 172.21.15.26:0/1993872146 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4f001f8f0 con 0x55707feda9f0 2024-09-11T23:26:23.325 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.318+0000 7fb4febf4740 1 -- 172.21.15.26:0/1993872146 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb4ec036070 con 0x55707feda9f0 2024-09-11T23:26:23.325 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.318+0000 7fb4dd7fa640 1 -- 172.21.15.26:0/1993872146 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8542+0+0 (secure 0 0 0) 0x7fb4f001fa90 con 0x55707feda9f0 2024-09-11T23:26:23.325 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.318+0000 7fb4dd7fa640 1 -- 172.21.15.26:0/1993872146 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fb4f001fc90 con 0x55707feda9f0 2024-09-11T23:26:23.325 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.318+0000 7fb4febf4740 10 client.16701 fetch_fsmap learned FSMap version 47 2024-09-11T23:26:23.325 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.318+0000 7fb4febf4740 1 -- 172.21.15.26:0/1993872146 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55707fc1e620 con 0x55707feda9f0 2024-09-11T23:26:23.325 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.318+0000 7fb4dd7fa640 1 -- 172.21.15.26:0/1993872146 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4ec036070 con 0x55707feda9f0 2024-09-11T23:26:23.325 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.318+0000 7fb4dd7fa640 1 -- 172.21.15.26:0/1993872146 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 47) ==== 8414+0+0 (secure 0 0 0) 0x7fb4f001d4b0 con 0x55707feda9f0 2024-09-11T23:26:23.325 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.318+0000 7fb4febf4740 10 client.16701 fetch_fsmap finished waiting for FSMap version 47 2024-09-11T23:26:23.325 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.318+0000 7fb4febf4740 10 client.16701 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:26:23.325 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.318+0000 7fb4febf4740 20 client.16701 populate_metadata read hostname 'smithi026' 2024-09-11T23:26:23.325 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.318+0000 7fb4febf4740 1 --2- 172.21.15.26:0/1993872146 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55707fefa0d0 0x55707fefc4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:23.325 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.318+0000 7fb4febf4740 4 client.16701 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:26:23.326 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.318+0000 7fb4febf4740 1 -- 172.21.15.26:0/1993872146 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55707fe4ca20 con 0x55707fefa0d0 2024-09-11T23:26:23.326 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.319+0000 7fb4f93f7640 1 --2- 172.21.15.26:0/1993872146 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55707fefa0d0 0x55707fefc4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:23.326 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.319+0000 7fb4f93f7640 1 --2- 172.21.15.26:0/1993872146 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55707fefa0d0 0x55707fefc4c0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15c2e229043d11 server_cookie=f48ec0f13e03274d in_seq=0 out_seq=0 2024-09-11T23:26:23.326 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.319+0000 7fb4dd7fa640 10 client.16701 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:26:23.326 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.320+0000 7fb4dd7fa640 1 -- 172.21.15.26:0/1993872146 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55707fe4ca20 con 0x55707fefa0d0 2024-09-11T23:26:23.326 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.320+0000 7fb4dd7fa640 10 client.16701 handle_command_reply: tid=0 2024-09-11T23:26:23.326 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.320+0000 7fb4febf4740 1 client.16701 shutdown 2024-09-11T23:26:23.326 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.320+0000 7fb4febf4740 1 -- 172.21.15.26:0/1993872146 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55707feda9f0 msgr2=0x55707fee6fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:26:23.326 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.320+0000 7fb4febf4740 1 --2- 172.21.15.26:0/1993872146 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55707feda9f0 0x55707fee6fb0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fb4f0005f80 tx=0x7fb4f00046c0 comp rx=0 tx=0).stop 2024-09-11T23:26:23.326 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.320+0000 7fb4febf4740 1 -- 172.21.15.26:0/1993872146 shutdown_connections 2024-09-11T23:26:23.326 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.320+0000 7fb4febf4740 1 --2- 172.21.15.26:0/1993872146 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55707feea0c0 0x55707fee7a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:23.326 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.320+0000 7fb4febf4740 1 --2- 172.21.15.26:0/1993872146 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55707fefa0d0 0x55707fefc4c0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:23.326 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.320+0000 7fb4febf4740 1 --2- 172.21.15.26:0/1993872146 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55707fedb310 0x55707fee74f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:23.327 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.320+0000 7fb4febf4740 1 --2- 172.21.15.26:0/1993872146 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55707feda9f0 0x55707fee6fb0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:23.327 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.320+0000 7fb4dd7fa640 0 client.16701 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:26:23.327 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.320+0000 7fb4febf4740 1 -- 172.21.15.26:0/1993872146 >> 172.21.15.26:0/1993872146 conn(0x55707fed7060 msgr2=0x55707fedcd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:26:23.327 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.320+0000 7fb4febf4740 1 -- 172.21.15.26:0/1993872146 shutdown_connections 2024-09-11T23:26:23.327 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.320+0000 7fb4febf4740 1 -- 172.21.15.26:0/1993872146 wait complete. 2024-09-11T23:26:23.327 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.320+0000 7fb4febf4740 20 client.16701 trim_cache size 0 max 16384 2024-09-11T23:26:23.336 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.335+0000 7fb4febf4740 1 Processor -- start 2024-09-11T23:26:23.336 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.335+0000 7fb4febf4740 1 -- start start 2024-09-11T23:26:23.336 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.335+0000 7fb4febf4740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55707feda9f0 0x55707ff01dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:23.336 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.335+0000 7fb4febf4740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55707fedb310 0x55707ff02310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:23.336 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.335+0000 7fb4febf4740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55707feea0c0 0x55707fee41f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:23.336 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.335+0000 7fb4febf4740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb4f41ab620 con 0x55707feea0c0 2024-09-11T23:26:23.336 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.335+0000 7fb4febf4740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fb4f41ad3d0 con 0x55707feda9f0 2024-09-11T23:26:23.336 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.335+0000 7fb4febf4740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb4f41abdb0 con 0x55707fedb310 2024-09-11T23:26:23.336 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.335+0000 7fb4fa3f9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55707feea0c0 0x55707fee41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:23.337 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.335+0000 7fb4fa3f9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55707feea0c0 0x55707fee41f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36428/0 (socket says 172.21.15.26:36428) 2024-09-11T23:26:23.337 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.335+0000 7fb4fa3f9640 1 -- 172.21.15.26:0/2061482303 learned_addr learned my addr 172.21.15.26:0/2061482303 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:26:23.337 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.335+0000 7fb4f9bf8640 1 --2- 172.21.15.26:0/2061482303 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55707feda9f0 0x55707ff01dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:23.337 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.335+0000 7fb4f93f7640 1 --2- 172.21.15.26:0/2061482303 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55707fedb310 0x55707ff02310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:23.337 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.335+0000 7fb4fa3f9640 1 -- 172.21.15.26:0/2061482303 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55707fedb310 msgr2=0x55707ff02310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:26:23.337 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.335+0000 7fb4fa3f9640 1 --2- 172.21.15.26:0/2061482303 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55707fedb310 0x55707ff02310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:23.337 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.335+0000 7fb4fa3f9640 1 -- 172.21.15.26:0/2061482303 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55707feda9f0 msgr2=0x55707ff01dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:26:23.337 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.335+0000 7fb4fa3f9640 1 --2- 172.21.15.26:0/2061482303 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55707feda9f0 0x55707ff01dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:23.337 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.336+0000 7fb4fa3f9640 1 -- 172.21.15.26:0/2061482303 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55707fed7fc0 con 0x55707feea0c0 2024-09-11T23:26:23.337 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.336+0000 7fb4fa3f9640 1 --2- 172.21.15.26:0/2061482303 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55707feea0c0 0x55707fee41f0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fb4ec004150 tx=0x7fb4ec004180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ead62b037a050a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:26:23.337 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.336+0000 7fb4df7fe640 1 -- 172.21.15.26:0/2061482303 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4ec048040 con 0x55707feea0c0 2024-09-11T23:26:23.337 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.336+0000 7fb4df7fe640 1 -- 172.21.15.26:0/2061482303 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4ec036040 con 0x55707feea0c0 2024-09-11T23:26:23.337 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.336+0000 7fb4df7fe640 1 -- 172.21.15.26:0/2061482303 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4ec004570 con 0x55707feea0c0 2024-09-11T23:26:23.341 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.336+0000 7fb4febf4740 1 -- 172.21.15.26:0/2061482303 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55707feea0c0 msgr2=0x55707fee41f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:26:23.341 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.336+0000 7fb4febf4740 1 --2- 172.21.15.26:0/2061482303 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55707feea0c0 0x55707fee41f0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fb4ec004150 tx=0x7fb4ec004180 comp rx=0 tx=0).stop 2024-09-11T23:26:23.341 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.337+0000 7fb4febf4740 1 -- 172.21.15.26:0/2061482303 shutdown_connections 2024-09-11T23:26:23.341 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.337+0000 7fb4febf4740 1 --2- 172.21.15.26:0/2061482303 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55707feea0c0 0x55707fee41f0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:23.341 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.337+0000 7fb4febf4740 1 --2- 172.21.15.26:0/2061482303 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55707fedb310 0x55707ff02310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:23.341 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.337+0000 7fb4febf4740 1 --2- 172.21.15.26:0/2061482303 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55707feda9f0 0x55707ff01dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:23.342 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.337+0000 7fb4febf4740 1 -- 172.21.15.26:0/2061482303 >> 172.21.15.26:0/2061482303 conn(0x55707fed7060 msgr2=0x55707fedcd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:26:23.342 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.337+0000 7fb4febf4740 1 -- 172.21.15.26:0/2061482303 shutdown_connections 2024-09-11T23:26:23.342 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.337+0000 7fb4febf4740 1 -- 172.21.15.26:0/2061482303 wait complete. 2024-09-11T23:26:23.342 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.337+0000 7fb4febf4740 1 -- start start 2024-09-11T23:26:23.342 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.337+0000 7fb4febf4740 1 Processor -- start 2024-09-11T23:26:23.342 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.337+0000 7fb4febf4740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55707feda9f0 0x55707fef2860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:23.342 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.337+0000 7fb4febf4740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55707fedb310 0x55707feedb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:23.342 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.337+0000 7fb4febf4740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55707feea0c0 0x55707feee0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:23.342 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.337+0000 7fb4febf4740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb4f41accd0 con 0x55707feea0c0 2024-09-11T23:26:23.342 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.337+0000 7fb4febf4740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fb4f41ac540 con 0x55707feda9f0 2024-09-11T23:26:23.342 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.337+0000 7fb4febf4740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb4f41a9620 con 0x55707fedb310 2024-09-11T23:26:23.342 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.337+0000 7fb4febf4740 10 client.0 _ll_register_callbacks cb 0x7fb4f000b970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:26:23.343 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.338+0000 7fb4fa3f9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55707feea0c0 0x55707feee0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:23.343 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.338+0000 7fb4fa3f9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55707feea0c0 0x55707feee0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36434/0 (socket says 172.21.15.26:36434) 2024-09-11T23:26:23.343 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.338+0000 7fb4fa3f9640 1 -- 172.21.15.26:0/1466616999 learned_addr learned my addr 172.21.15.26:0/1466616999 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:26:23.343 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.338+0000 7fb4f9bf8640 1 --2- 172.21.15.26:0/1466616999 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55707feda9f0 0x55707fef2860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:23.343 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.338+0000 7fb4f93f7640 1 --2- 172.21.15.26:0/1466616999 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55707fedb310 0x55707feedb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:23.343 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.338+0000 7fb4fa3f9640 1 -- 172.21.15.26:0/1466616999 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55707fedb310 msgr2=0x55707feedb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:26:23.343 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.338+0000 7fb4fa3f9640 1 --2- 172.21.15.26:0/1466616999 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55707fedb310 0x55707feedb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:23.343 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.338+0000 7fb4fa3f9640 1 -- 172.21.15.26:0/1466616999 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55707feda9f0 msgr2=0x55707fef2860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:26:23.343 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.338+0000 7fb4fa3f9640 1 --2- 172.21.15.26:0/1466616999 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55707feda9f0 0x55707fef2860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:23.343 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.338+0000 7fb4fa3f9640 1 -- 172.21.15.26:0/1466616999 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55707feee8a0 con 0x55707feea0c0 2024-09-11T23:26:23.343 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.338+0000 7fb4fa3f9640 1 --2- 172.21.15.26:0/1466616999 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55707feea0c0 0x55707feee0c0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fb4ec000f80 tx=0x7fb4ec031560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbebf751b1c793b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:26:23.343 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.338+0000 7fb4dd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:26:23.343 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.338+0000 7fb4dd7fa640 1 -- 172.21.15.26:0/1466616999 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4ec05d070 con 0x55707feea0c0 2024-09-11T23:26:23.344 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.338+0000 7fb4dd7fa640 1 -- 172.21.15.26:0/1466616999 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8542+0+0 (secure 0 0 0) 0x7fb4ec003d80 con 0x55707feea0c0 2024-09-11T23:26:23.344 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.338+0000 7fb4febf4740 1 -- 172.21.15.26:0/1466616999 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb4ec036040 con 0x55707feea0c0 2024-09-11T23:26:23.344 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.338+0000 7fb4dd7fa640 1 -- 172.21.15.26:0/1466616999 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fb4ec069050 con 0x55707feea0c0 2024-09-11T23:26:23.344 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.338+0000 7fb4febf4740 10 client.16707 fetch_fsmap learned FSMap version 47 2024-09-11T23:26:23.344 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.338+0000 7fb4febf4740 1 -- 172.21.15.26:0/1466616999 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55707ff64dc0 con 0x55707feea0c0 2024-09-11T23:26:23.344 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.339+0000 7fb4dd7fa640 1 -- 172.21.15.26:0/1466616999 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4ec036040 con 0x55707feea0c0 2024-09-11T23:26:23.344 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.339+0000 7fb4dd7fa640 1 -- 172.21.15.26:0/1466616999 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 47) ==== 8414+0+0 (secure 0 0 0) 0x7fb4ec042030 con 0x55707feea0c0 2024-09-11T23:26:23.344 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.339+0000 7fb4febf4740 10 client.16707 fetch_fsmap finished waiting for FSMap version 47 2024-09-11T23:26:23.344 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.339+0000 7fb4febf4740 10 client.16707 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:26:23.344 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.339+0000 7fb4febf4740 20 client.16707 populate_metadata read hostname 'smithi026' 2024-09-11T23:26:23.344 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.339+0000 7fb4febf4740 1 --2- 172.21.15.26:0/1466616999 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55707ff64ff0 0x55707ff673e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:23.344 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.339+0000 7fb4febf4740 4 client.16707 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T23:26:23.345 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.339+0000 7fb4febf4740 1 -- 172.21.15.26:0/1466616999 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55707fe46d70 con 0x55707ff64ff0 2024-09-11T23:26:23.345 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.339+0000 7fb4f9bf8640 1 --2- 172.21.15.26:0/1466616999 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55707ff64ff0 0x55707ff673e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:23.345 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.340+0000 7fb4dd7fa640 10 client.16707 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:26:23.345 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.340+0000 7fb4f9bf8640 1 --2- 172.21.15.26:0/1466616999 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55707ff64ff0 0x55707ff673e0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c6fb0090f5dbb982 server_cookie=c0028469ca7f9870 in_seq=0 out_seq=0 2024-09-11T23:26:23.358 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.357+0000 7fb4dd7fa640 1 -- 172.21.15.26:0/1466616999 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55707fe46d70 con 0x55707ff64ff0 2024-09-11T23:26:23.358 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.357+0000 7fb4dd7fa640 10 client.16707 handle_command_reply: tid=0 2024-09-11T23:26:23.358 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.357+0000 7fb4febf4740 1 client.16707 shutdown 2024-09-11T23:26:23.358 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.357+0000 7fb4febf4740 1 -- 172.21.15.26:0/1466616999 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55707feea0c0 msgr2=0x55707feee0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:26:23.358 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.357+0000 7fb4febf4740 1 --2- 172.21.15.26:0/1466616999 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55707feea0c0 0x55707feee0c0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fb4ec000f80 tx=0x7fb4ec031560 comp rx=0 tx=0).stop 2024-09-11T23:26:23.358 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.357+0000 7fb4febf4740 1 -- 172.21.15.26:0/1466616999 shutdown_connections 2024-09-11T23:26:23.358 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.357+0000 7fb4febf4740 1 --2- 172.21.15.26:0/1466616999 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55707ff64ff0 0x55707ff673e0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:23.358 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.357+0000 7fb4febf4740 1 --2- 172.21.15.26:0/1466616999 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55707feea0c0 0x55707feee0c0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:23.358 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.357+0000 7fb4febf4740 1 --2- 172.21.15.26:0/1466616999 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55707fedb310 0x55707feedb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:23.358 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.357+0000 7fb4dd7fa640 0 client.16707 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:26:23.358 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.357+0000 7fb4febf4740 1 --2- 172.21.15.26:0/1466616999 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55707feda9f0 0x55707fef2860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:23.359 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.357+0000 7fb4febf4740 1 -- 172.21.15.26:0/1466616999 >> 172.21.15.26:0/1466616999 conn(0x55707fed7060 msgr2=0x55707fedc530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:26:23.359 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.358+0000 7fb4febf4740 1 -- 172.21.15.26:0/1466616999 shutdown_connections 2024-09-11T23:26:23.359 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.358+0000 7fb4febf4740 1 -- 172.21.15.26:0/1466616999 wait complete. 2024-09-11T23:26:23.359 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.358+0000 7fb4febf4740 20 client.16707 trim_cache size 0 max 16384 2024-09-11T23:26:23.359 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-09-11T23:26:23.359 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "message": "", 2024-09-11T23:26:23.359 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-09-11T23:26:23.359 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-09-11T23:26:23.360 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.358+0000 7fb4fbe5c640 1 -- 172.21.15.26:0/1422643827 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fb4e00a2da0 msgr2=0x7fb4e00a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:26:23.360 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.358+0000 7fb4fbe5c640 1 --2- 172.21.15.26:0/1422643827 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fb4e00a2da0 0x7fb4e00a5260 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fb4f0005fd0 tx=0x7fb4f0005d10 comp rx=0 tx=0).stop 2024-09-11T23:26:23.360 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.358+0000 7fb4fbe5c640 1 -- 172.21.15.26:0/1422643827 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb4f4102470 msgr2=0x7fb4f4198850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:26:23.360 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.358+0000 7fb4fbe5c640 1 --2- 172.21.15.26:0/1422643827 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb4f4102470 0x7fb4f4198850 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fb4e400d950 tx=0x7fb4e400de10 comp rx=0 tx=0).stop 2024-09-11T23:26:23.360 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.358+0000 7fb4fbe5c640 1 -- 172.21.15.26:0/1422643827 shutdown_connections 2024-09-11T23:26:23.360 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.358+0000 7fb4fbe5c640 1 --2- 172.21.15.26:0/1422643827 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fb4e00a2da0 0x7fb4e00a5260 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:23.361 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.358+0000 7fb4fbe5c640 1 --2- 172.21.15.26:0/1422643827 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb4f4107b70 0x7fb4f419cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:23.361 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.358+0000 7fb4fbe5c640 1 --2- 172.21.15.26:0/1422643827 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb4f4102470 0x7fb4f4198850 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:23.361 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.358+0000 7fb4fbe5c640 1 --2- 172.21.15.26:0/1422643827 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb4f4101ad0 0x7fb4f4198310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:23.361 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.358+0000 7fb4fbe5c640 1 -- 172.21.15.26:0/1422643827 >> 172.21.15.26:0/1422643827 conn(0x7fb4f40fd7d0 msgr2=0x7fb4f40fde60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:26:23.361 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.358+0000 7fb4fbe5c640 1 -- 172.21.15.26:0/1422643827 shutdown_connections 2024-09-11T23:26:23.361 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:23.358+0000 7fb4fbe5c640 1 -- 172.21.15.26:0/1422643827 wait complete. 2024-09-11T23:26:23.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:26:23 smithi136 ceph-mon[35121]: pgmap v247: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 58 KiB/s wr, 18 op/s 2024-09-11T23:26:24.000 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:26:23 smithi026 ceph-mon[31172]: pgmap v247: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 58 KiB/s wr, 18 op/s 2024-09-11T23:26:24.052 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:26:23 smithi076 ceph-mon[35296]: pgmap v247: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 58 KiB/s wr, 18 op/s 2024-09-11T23:26:25.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:26:25 smithi136 ceph-mon[35121]: pgmap v248: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 47 KiB/s wr, 15 op/s 2024-09-11T23:26:26.000 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:26:25 smithi026 ceph-mon[31172]: pgmap v248: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 47 KiB/s wr, 15 op/s 2024-09-11T23:26:26.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:26:25 smithi076 ceph-mon[35296]: pgmap v248: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 47 KiB/s wr, 15 op/s 2024-09-11T23:26:27.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:26:27 smithi136 ceph-mon[35121]: pgmap v249: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 46 KiB/s wr, 15 op/s 2024-09-11T23:26:28.000 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:26:27 smithi026 ceph-mon[31172]: pgmap v249: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 46 KiB/s wr, 15 op/s 2024-09-11T23:26:28.052 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:26:27 smithi076 ceph-mon[35296]: pgmap v249: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 46 KiB/s wr, 15 op/s 2024-09-11T23:26:29.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:26:29 smithi136 ceph-mon[35121]: pgmap v250: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 47 KiB/s wr, 16 op/s 2024-09-11T23:26:30.000 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:26:29 smithi026 ceph-mon[31172]: pgmap v250: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 47 KiB/s wr, 16 op/s 2024-09-11T23:26:30.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:26:29 smithi076 ceph-mon[35296]: pgmap v250: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 47 KiB/s wr, 16 op/s 2024-09-11T23:26:31.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:26:31 smithi136 ceph-mon[35121]: pgmap v251: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 60 KiB/s wr, 19 op/s 2024-09-11T23:26:32.000 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:26:31 smithi026 ceph-mon[31172]: pgmap v251: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 60 KiB/s wr, 19 op/s 2024-09-11T23:26:32.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:26:31 smithi076 ceph-mon[35296]: pgmap v251: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 60 KiB/s wr, 19 op/s 2024-09-11T23:26:32.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:26:32 smithi136 ceph-mon[35121]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:26:33.000 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:26:32 smithi026 ceph-mon[31172]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:26:33.052 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:26:32 smithi076 ceph-mon[35296]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:26:33.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:26:33 smithi136 ceph-mon[35121]: pgmap v252: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 51 KiB/s wr, 16 op/s 2024-09-11T23:26:34.000 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:26:33 smithi026 ceph-mon[31172]: pgmap v252: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 51 KiB/s wr, 16 op/s 2024-09-11T23:26:34.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:26:33 smithi076 ceph-mon[35296]: pgmap v252: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 51 KiB/s wr, 16 op/s 2024-09-11T23:26:35.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:26:35 smithi136 ceph-mon[35121]: pgmap v253: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 60 KiB/s wr, 21 op/s 2024-09-11T23:26:36.000 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:26:35 smithi026 ceph-mon[31172]: pgmap v253: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 60 KiB/s wr, 21 op/s 2024-09-11T23:26:36.052 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:26:35 smithi076 ceph-mon[35296]: pgmap v253: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 60 KiB/s wr, 21 op/s 2024-09-11T23:26:36.466 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.465+0000 7f4c59827640 1 -- 172.21.15.26:0/14042292 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4c54101370 msgr2=0x7f4c5410d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:26:36.467 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.465+0000 7f4c59827640 1 --2- 172.21.15.26:0/14042292 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4c54101370 0x7f4c5410d8a0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f4c4c009cd0 tx=0x7f4c4c02a8e0 comp rx=0 tx=0).stop 2024-09-11T23:26:36.467 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.465+0000 7f4c59827640 1 -- 172.21.15.26:0/14042292 shutdown_connections 2024-09-11T23:26:36.467 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.465+0000 7f4c59827640 1 --2- 172.21.15.26:0/14042292 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4c54101370 0x7f4c5410d8a0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:36.467 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.465+0000 7f4c59827640 1 --2- 172.21.15.26:0/14042292 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4c541009d0 0x7f4c54100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:36.467 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.465+0000 7f4c59827640 1 --2- 172.21.15.26:0/14042292 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4c541069e0 0x7f4c54106dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:36.467 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.465+0000 7f4c59827640 1 -- 172.21.15.26:0/14042292 >> 172.21.15.26:0/14042292 conn(0x7f4c540fc6b0 msgr2=0x7f4c540fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:26:36.467 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.465+0000 7f4c59827640 1 -- 172.21.15.26:0/14042292 shutdown_connections 2024-09-11T23:26:36.467 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.465+0000 7f4c59827640 1 -- 172.21.15.26:0/14042292 wait complete. 2024-09-11T23:26:36.467 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.465+0000 7f4c59827640 1 Processor -- start 2024-09-11T23:26:36.467 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.466+0000 7f4c59827640 1 -- start start 2024-09-11T23:26:36.467 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.466+0000 7f4c59827640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4c541009d0 0x7f4c54198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:36.467 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.466+0000 7f4c59827640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4c54101370 0x7f4c54198840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:36.468 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.466+0000 7f4c59827640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4c541069e0 0x7f4c5419cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:36.468 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.466+0000 7f4c59827640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f4c54111f10 con 0x7f4c541009d0 2024-09-11T23:26:36.468 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.466+0000 7f4c59827640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f4c54111d90 con 0x7f4c54101370 2024-09-11T23:26:36.468 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.466+0000 7f4c59827640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f4c54112090 con 0x7f4c541069e0 2024-09-11T23:26:36.468 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.466+0000 7f4c52ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4c541009d0 0x7f4c54198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:36.468 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.466+0000 7f4c52ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4c541009d0 0x7f4c54198300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60932/0 (socket says 172.21.15.26:60932) 2024-09-11T23:26:36.468 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.466+0000 7f4c52ffd640 1 -- 172.21.15.26:0/1702933344 learned_addr learned my addr 172.21.15.26:0/1702933344 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:26:36.468 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.466+0000 7f4c527fc640 1 --2- 172.21.15.26:0/1702933344 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4c54101370 0x7f4c54198840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:36.468 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.466+0000 7f4c537fe640 1 --2- 172.21.15.26:0/1702933344 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4c541069e0 0x7f4c5419cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:36.468 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.466+0000 7f4c52ffd640 1 -- 172.21.15.26:0/1702933344 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4c541069e0 msgr2=0x7f4c5419cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:26:36.468 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.466+0000 7f4c52ffd640 1 --2- 172.21.15.26:0/1702933344 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4c541069e0 0x7f4c5419cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:36.468 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.466+0000 7f4c52ffd640 1 -- 172.21.15.26:0/1702933344 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4c54101370 msgr2=0x7f4c54198840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:26:36.468 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.466+0000 7f4c52ffd640 1 --2- 172.21.15.26:0/1702933344 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4c54101370 0x7f4c54198840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:36.469 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.466+0000 7f4c52ffd640 1 -- 172.21.15.26:0/1702933344 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c5419d2f0 con 0x7f4c541009d0 2024-09-11T23:26:36.469 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.466+0000 7f4c52ffd640 1 --2- 172.21.15.26:0/1702933344 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4c541009d0 0x7f4c54198300 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f4c4400c970 tx=0x7f4c4400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:26:36.469 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.467+0000 7f4c58825640 1 -- 172.21.15.26:0/1702933344 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c44007bb0 con 0x7f4c541009d0 2024-09-11T23:26:36.469 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.467+0000 7f4c59827640 1 -- 172.21.15.26:0/1702933344 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c5419d5e0 con 0x7f4c541009d0 2024-09-11T23:26:36.469 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.467+0000 7f4c58825640 1 -- 172.21.15.26:0/1702933344 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c44007d50 con 0x7f4c541009d0 2024-09-11T23:26:36.469 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.467+0000 7f4c58825640 1 -- 172.21.15.26:0/1702933344 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c44018800 con 0x7f4c541009d0 2024-09-11T23:26:36.469 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.467+0000 7f4c59827640 1 -- 172.21.15.26:0/1702933344 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c541a4e70 con 0x7f4c541009d0 2024-09-11T23:26:36.476 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.469+0000 7f4c58825640 1 -- 172.21.15.26:0/1702933344 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f4c440189e0 con 0x7f4c541009d0 2024-09-11T23:26:36.477 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.469+0000 7f4c5c5c3740 1 Processor -- start 2024-09-11T23:26:36.477 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.470+0000 7f4c5c5c3740 1 -- start start 2024-09-11T23:26:36.477 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.470+0000 7f4c5c5c3740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5645aacf5af0 0x5645aacf5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:36.477 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.470+0000 7f4c5c5c3740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5645aacf6410 0x5645aad00770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:36.477 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.470+0000 7f4c5c5c3740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5645aad00cb0 0x5645aad030a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:36.477 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.470+0000 7f4c5c5c3740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5645aac5a2f0 con 0x5645aacf5af0 2024-09-11T23:26:36.477 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.470+0000 7f4c5c5c3740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5645aac02270 con 0x5645aad00cb0 2024-09-11T23:26:36.477 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.470+0000 7f4c5c5c3740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5645aac562b0 con 0x5645aacf6410 2024-09-11T23:26:36.477 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.470+0000 7f4c537fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5645aacf5af0 0x5645aacf5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:36.477 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.470+0000 7f4c537fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5645aacf5af0 0x5645aacf5ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60938/0 (socket says 172.21.15.26:60938) 2024-09-11T23:26:36.477 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.470+0000 7f4c537fe640 1 -- 172.21.15.26:0/3079169985 learned_addr learned my addr 172.21.15.26:0/3079169985 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:26:36.477 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.470+0000 7f4c52ffd640 1 --2- 172.21.15.26:0/3079169985 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5645aacf6410 0x5645aad00770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:36.477 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.470+0000 7f4c527fc640 1 --2- 172.21.15.26:0/3079169985 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5645aad00cb0 0x5645aad030a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:36.478 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.470+0000 7f4c58825640 1 --2- 172.21.15.26:0/1702933344 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f4c3c0a2da0 0x7f4c3c0a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:36.478 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.470+0000 7f4c537fe640 1 -- 172.21.15.26:0/3079169985 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5645aacf6410 msgr2=0x5645aad00770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:26:36.478 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.470+0000 7f4c537fe640 1 --2- 172.21.15.26:0/3079169985 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5645aacf6410 0x5645aad00770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:36.478 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.470+0000 7f4c537fe640 1 -- 172.21.15.26:0/3079169985 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5645aad00cb0 msgr2=0x5645aad030a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:26:36.478 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.470+0000 7f4c537fe640 1 --2- 172.21.15.26:0/3079169985 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5645aad00cb0 0x5645aad030a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:36.478 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.470+0000 7f4c537fe640 1 -- 172.21.15.26:0/3079169985 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5645aac01340 con 0x5645aacf5af0 2024-09-11T23:26:36.478 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.470+0000 7f4c52ffd640 1 --2- 172.21.15.26:0/3079169985 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5645aacf6410 0x5645aad00770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:26:36.478 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.471+0000 7f4c537fe640 1 --2- 172.21.15.26:0/1702933344 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f4c3c0a2da0 0x7f4c3c0a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:36.478 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.471+0000 7f4c58825640 1 -- 172.21.15.26:0/1702933344 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8542+0+0 (secure 0 0 0) 0x7f4c440cf3b0 con 0x7f4c541009d0 2024-09-11T23:26:36.478 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.471+0000 7f4c537fe640 1 --2- 172.21.15.26:0/3079169985 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5645aacf5af0 0x5645aacf5ed0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f4c4c035000 tx=0x7f4c4c031660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c97b28964361dc84 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:26:36.478 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.471+0000 7f4c39ffb640 1 -- 172.21.15.26:0/3079169985 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c4c041040 con 0x5645aacf5af0 2024-09-11T23:26:36.478 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.471+0000 7f4c39ffb640 1 -- 172.21.15.26:0/3079169985 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c4c043070 con 0x5645aacf5af0 2024-09-11T23:26:36.478 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.471+0000 7f4c39ffb640 1 -- 172.21.15.26:0/3079169985 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c4c030d00 con 0x5645aacf5af0 2024-09-11T23:26:36.478 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.472+0000 7f4c5c5c3740 1 -- 172.21.15.26:0/3079169985 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5645aacf5af0 msgr2=0x5645aacf5ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:26:36.479 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.472+0000 7f4c5c5c3740 1 --2- 172.21.15.26:0/3079169985 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5645aacf5af0 0x5645aacf5ed0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f4c4c035000 tx=0x7f4c4c031660 comp rx=0 tx=0).stop 2024-09-11T23:26:36.479 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.472+0000 7f4c537fe640 1 --2- 172.21.15.26:0/1702933344 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f4c3c0a2da0 0x7f4c3c0a5260 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f4c4c04c000 tx=0x7f4c4c04d000 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:26:36.479 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.472+0000 7f4c5c5c3740 1 -- 172.21.15.26:0/3079169985 shutdown_connections 2024-09-11T23:26:36.479 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.472+0000 7f4c5c5c3740 1 --2- 172.21.15.26:0/3079169985 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5645aad00cb0 0x5645aad030a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:36.479 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.472+0000 7f4c5c5c3740 1 --2- 172.21.15.26:0/3079169985 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5645aacf6410 0x5645aad00770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:36.479 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.472+0000 7f4c5c5c3740 1 --2- 172.21.15.26:0/3079169985 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5645aacf5af0 0x5645aacf5ed0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:36.479 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.472+0000 7f4c5c5c3740 1 -- 172.21.15.26:0/3079169985 >> 172.21.15.26:0/3079169985 conn(0x5645aacf2060 msgr2=0x5645aacf2440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:26:36.479 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.472+0000 7f4c5c5c3740 1 -- 172.21.15.26:0/3079169985 shutdown_connections 2024-09-11T23:26:36.479 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.472+0000 7f4c5c5c3740 1 -- 172.21.15.26:0/3079169985 wait complete. 2024-09-11T23:26:36.479 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.472+0000 7f4c5c5c3740 1 -- start start 2024-09-11T23:26:36.479 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.472+0000 7f4c5c5c3740 1 Processor -- start 2024-09-11T23:26:36.479 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.473+0000 7f4c5c5c3740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5645aacf5af0 0x5645aacfdb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:36.479 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.473+0000 7f4c5c5c3740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5645aacf6410 0x5645aacfe0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:36.480 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.473+0000 7f4c5c5c3740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5645aad00cb0 0x5645aacfe600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:36.480 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.473+0000 7f4c5c5c3740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5645aad03880 con 0x5645aacf5af0 2024-09-11T23:26:36.480 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.473+0000 7f4c5c5c3740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5645aabe3290 con 0x5645aacf6410 2024-09-11T23:26:36.480 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.473+0000 7f4c5c5c3740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5645aad03a00 con 0x5645aad00cb0 2024-09-11T23:26:36.480 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.473+0000 7f4c5c5c3740 10 client.0 _ll_register_callbacks cb 0x5645aac14140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:26:36.480 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.473+0000 7f4c52ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5645aacf5af0 0x5645aacfdb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:36.480 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.473+0000 7f4c52ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5645aacf5af0 0x5645aacfdb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60948/0 (socket says 172.21.15.26:60948) 2024-09-11T23:26:36.480 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.473+0000 7f4c52ffd640 1 -- 172.21.15.26:0/4094488402 learned_addr learned my addr 172.21.15.26:0/4094488402 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:26:36.480 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.473+0000 7f4c527fc640 1 --2- 172.21.15.26:0/4094488402 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5645aacf6410 0x5645aacfe0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:36.480 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.473+0000 7f4c537fe640 1 --2- 172.21.15.26:0/4094488402 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5645aad00cb0 0x5645aacfe600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:36.480 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.473+0000 7f4c52ffd640 1 -- 172.21.15.26:0/4094488402 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5645aad00cb0 msgr2=0x5645aacfe600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:26:36.480 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.473+0000 7f4c52ffd640 1 --2- 172.21.15.26:0/4094488402 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5645aad00cb0 0x5645aacfe600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:36.480 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.473+0000 7f4c52ffd640 1 -- 172.21.15.26:0/4094488402 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5645aacf6410 msgr2=0x5645aacfe0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:26:36.481 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.473+0000 7f4c52ffd640 1 --2- 172.21.15.26:0/4094488402 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5645aacf6410 0x5645aacfe0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:36.481 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.473+0000 7f4c52ffd640 1 -- 172.21.15.26:0/4094488402 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5645aa96d210 con 0x5645aacf5af0 2024-09-11T23:26:36.481 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.473+0000 7f4c52ffd640 1 --2- 172.21.15.26:0/4094488402 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5645aacf5af0 0x5645aacfdb80 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f4c4400c940 tx=0x7f4c4409b710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ed16ef1b45059a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:26:36.481 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.473+0000 7f4c32ffd640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:26:36.481 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.474+0000 7f4c32ffd640 1 -- 172.21.15.26:0/4094488402 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c4409bfc0 con 0x5645aacf5af0 2024-09-11T23:26:36.481 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.474+0000 7f4c32ffd640 1 -- 172.21.15.26:0/4094488402 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8542+0+0 (secure 0 0 0) 0x7f4c4400faa0 con 0x5645aacf5af0 2024-09-11T23:26:36.481 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.474+0000 7f4c5c5c3740 1 -- 172.21.15.26:0/4094488402 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4c4c043070 con 0x5645aacf5af0 2024-09-11T23:26:36.481 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.474+0000 7f4c32ffd640 1 -- 172.21.15.26:0/4094488402 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f4c4409b8c0 con 0x5645aacf5af0 2024-09-11T23:26:36.481 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.474+0000 7f4c5c5c3740 10 client.16719 fetch_fsmap learned FSMap version 48 2024-09-11T23:26:36.481 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.474+0000 7f4c5c5c3740 1 -- 172.21.15.26:0/4094488402 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x5645aaa39620 con 0x5645aacf5af0 2024-09-11T23:26:36.481 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.474+0000 7f4c32ffd640 1 -- 172.21.15.26:0/4094488402 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c4c043070 con 0x5645aacf5af0 2024-09-11T23:26:36.481 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.474+0000 7f4c32ffd640 1 -- 172.21.15.26:0/4094488402 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 48) ==== 8406+0+0 (secure 0 0 0) 0x7f4c44020030 con 0x5645aacf5af0 2024-09-11T23:26:36.481 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.474+0000 7f4c5c5c3740 10 client.16719 fetch_fsmap finished waiting for FSMap version 48 2024-09-11T23:26:36.482 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.474+0000 7f4c5c5c3740 10 client.16719 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:26:36.482 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.474+0000 7f4c5c5c3740 20 client.16719 populate_metadata read hostname 'smithi026' 2024-09-11T23:26:36.482 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.474+0000 7f4c5c5c3740 1 --2- 172.21.15.26:0/4094488402 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5645aad10bf0 0x5645aad12fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:36.482 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.474+0000 7f4c5c5c3740 4 client.16719 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:26:36.482 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.474+0000 7f4c5c5c3740 1 -- 172.21.15.26:0/4094488402 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5645aac67a20 con 0x5645aad10bf0 2024-09-11T23:26:36.482 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.475+0000 7f4c527fc640 1 --2- 172.21.15.26:0/4094488402 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5645aad10bf0 0x5645aad12fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:36.482 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.475+0000 7f4c527fc640 1 --2- 172.21.15.26:0/4094488402 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5645aad10bf0 0x5645aad12fe0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=78f3dba9bfe74174 server_cookie=1f2f69dfc6e29c64 in_seq=0 out_seq=0 2024-09-11T23:26:36.482 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.475+0000 7f4c32ffd640 10 client.16719 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:26:36.482 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.476+0000 7f4c32ffd640 1 -- 172.21.15.26:0/4094488402 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5645aac67a20 con 0x5645aad10bf0 2024-09-11T23:26:36.482 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.476+0000 7f4c32ffd640 10 client.16719 handle_command_reply: tid=0 2024-09-11T23:26:36.482 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.476+0000 7f4c5c5c3740 1 client.16719 shutdown 2024-09-11T23:26:36.482 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.476+0000 7f4c5c5c3740 1 -- 172.21.15.26:0/4094488402 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5645aacf5af0 msgr2=0x5645aacfdb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:26:36.482 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.476+0000 7f4c5c5c3740 1 --2- 172.21.15.26:0/4094488402 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5645aacf5af0 0x5645aacfdb80 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f4c4400c940 tx=0x7f4c4409b710 comp rx=0 tx=0).stop 2024-09-11T23:26:36.482 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.476+0000 7f4c5c5c3740 1 -- 172.21.15.26:0/4094488402 shutdown_connections 2024-09-11T23:26:36.482 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.476+0000 7f4c5c5c3740 1 --2- 172.21.15.26:0/4094488402 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5645aacf6410 0x5645aacfe0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:36.483 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.476+0000 7f4c5c5c3740 1 --2- 172.21.15.26:0/4094488402 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5645aad10bf0 0x5645aad12fe0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:36.483 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.476+0000 7f4c5c5c3740 1 --2- 172.21.15.26:0/4094488402 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5645aad00cb0 0x5645aacfe600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:36.483 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.476+0000 7f4c5c5c3740 1 --2- 172.21.15.26:0/4094488402 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5645aacf5af0 0x5645aacfdb80 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:36.483 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.476+0000 7f4c5c5c3740 1 -- 172.21.15.26:0/4094488402 >> 172.21.15.26:0/4094488402 conn(0x5645aacf2060 msgr2=0x5645aacf7d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:26:36.483 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.476+0000 7f4c32ffd640 0 client.16719 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:26:36.483 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.476+0000 7f4c5c5c3740 1 -- 172.21.15.26:0/4094488402 shutdown_connections 2024-09-11T23:26:36.483 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.476+0000 7f4c5c5c3740 1 -- 172.21.15.26:0/4094488402 wait complete. 2024-09-11T23:26:36.483 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.476+0000 7f4c5c5c3740 20 client.16719 trim_cache size 0 max 16384 2024-09-11T23:26:36.495 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.491+0000 7f4c5c5c3740 1 Processor -- start 2024-09-11T23:26:36.496 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.491+0000 7f4c5c5c3740 1 -- start start 2024-09-11T23:26:36.496 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.491+0000 7f4c5c5c3740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5645aacf5af0 0x5645aacfe430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:36.496 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.491+0000 7f4c5c5c3740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5645aacf6410 0x5645aacffaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:36.496 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.491+0000 7f4c5c5c3740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5645aad00cb0 0x5645aad188a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:36.496 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.491+0000 7f4c5c5c3740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f4c541ad790 con 0x5645aacf6410 2024-09-11T23:26:36.496 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.491+0000 7f4c5c5c3740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f4c541af4e0 con 0x5645aacf5af0 2024-09-11T23:26:36.496 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.491+0000 7f4c5c5c3740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f4c541adef0 con 0x5645aad00cb0 2024-09-11T23:26:36.496 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.491+0000 7f4c527fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5645aacf6410 0x5645aacffaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:36.496 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.491+0000 7f4c527fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5645aacf6410 0x5645aacffaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60962/0 (socket says 172.21.15.26:60962) 2024-09-11T23:26:36.496 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.491+0000 7f4c527fc640 1 -- 172.21.15.26:0/2565417129 learned_addr learned my addr 172.21.15.26:0/2565417129 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:26:36.496 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.491+0000 7f4c537fe640 1 --2- 172.21.15.26:0/2565417129 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5645aad00cb0 0x5645aad188a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:36.496 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.491+0000 7f4c52ffd640 1 --2- 172.21.15.26:0/2565417129 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5645aacf5af0 0x5645aacfe430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:36.497 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.491+0000 7f4c527fc640 1 -- 172.21.15.26:0/2565417129 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5645aad00cb0 msgr2=0x5645aad188a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:26:36.497 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.491+0000 7f4c527fc640 1 --2- 172.21.15.26:0/2565417129 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5645aad00cb0 0x5645aad188a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:36.497 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.491+0000 7f4c527fc640 1 -- 172.21.15.26:0/2565417129 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5645aacf5af0 msgr2=0x5645aacfe430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:26:36.497 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.491+0000 7f4c527fc640 1 --2- 172.21.15.26:0/2565417129 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5645aacf5af0 0x5645aacfe430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:36.497 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.491+0000 7f4c527fc640 1 -- 172.21.15.26:0/2565417129 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5645aacf8940 con 0x5645aacf6410 2024-09-11T23:26:36.497 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.491+0000 7f4c537fe640 1 --2- 172.21.15.26:0/2565417129 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5645aad00cb0 0x5645aad188a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:26:36.497 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.492+0000 7f4c527fc640 1 --2- 172.21.15.26:0/2565417129 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5645aacf6410 0x5645aacffaf0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f4c48000c00 tx=0x7f4c4800b410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=861b55f840e716ca server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:26:36.497 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.492+0000 7f4c397fa640 1 -- 172.21.15.26:0/2565417129 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c48005220 con 0x5645aacf6410 2024-09-11T23:26:36.497 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.492+0000 7f4c397fa640 1 -- 172.21.15.26:0/2565417129 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c48004c80 con 0x5645aacf6410 2024-09-11T23:26:36.497 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.492+0000 7f4c397fa640 1 -- 172.21.15.26:0/2565417129 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c4801da10 con 0x5645aacf6410 2024-09-11T23:26:36.497 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.492+0000 7f4c5c5c3740 1 -- 172.21.15.26:0/2565417129 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5645aacf6410 msgr2=0x5645aacffaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:26:36.497 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.492+0000 7f4c5c5c3740 1 --2- 172.21.15.26:0/2565417129 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5645aacf6410 0x5645aacffaf0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f4c48000c00 tx=0x7f4c4800b410 comp rx=0 tx=0).stop 2024-09-11T23:26:36.497 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.493+0000 7f4c5c5c3740 1 -- 172.21.15.26:0/2565417129 shutdown_connections 2024-09-11T23:26:36.497 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.493+0000 7f4c5c5c3740 1 --2- 172.21.15.26:0/2565417129 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5645aad00cb0 0x5645aad188a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:36.498 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.493+0000 7f4c5c5c3740 1 --2- 172.21.15.26:0/2565417129 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5645aacf6410 0x5645aacffaf0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:36.498 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.493+0000 7f4c5c5c3740 1 --2- 172.21.15.26:0/2565417129 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5645aacf5af0 0x5645aacfe430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:36.498 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.493+0000 7f4c5c5c3740 1 -- 172.21.15.26:0/2565417129 >> 172.21.15.26:0/2565417129 conn(0x5645aacf2060 msgr2=0x5645aacff3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:26:36.498 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.493+0000 7f4c5c5c3740 1 -- 172.21.15.26:0/2565417129 shutdown_connections 2024-09-11T23:26:36.498 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.493+0000 7f4c5c5c3740 1 -- 172.21.15.26:0/2565417129 wait complete. 2024-09-11T23:26:36.498 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.493+0000 7f4c5c5c3740 1 -- start start 2024-09-11T23:26:36.498 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.493+0000 7f4c5c5c3740 1 Processor -- start 2024-09-11T23:26:36.498 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.493+0000 7f4c5c5c3740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5645aacf5af0 0x5645aad04420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:36.498 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.493+0000 7f4c5c5c3740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5645aacf6410 0x5645aad04960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:36.498 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.493+0000 7f4c5c5c3740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5645aad00cb0 0x5645aad79860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:36.498 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.494+0000 7f4c5c5c3740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f4c541aee10 con 0x5645aacf6410 2024-09-11T23:26:36.498 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.494+0000 7f4c5c5c3740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f4c541ae680 con 0x5645aacf5af0 2024-09-11T23:26:36.498 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.494+0000 7f4c5c5c3740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f4c541ab790 con 0x5645aad00cb0 2024-09-11T23:26:36.498 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.494+0000 7f4c5c5c3740 10 client.0 _ll_register_callbacks cb 0x7f4c440046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:26:36.499 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.494+0000 7f4c527fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5645aacf6410 0x5645aad04960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:36.499 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.494+0000 7f4c527fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5645aacf6410 0x5645aad04960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60964/0 (socket says 172.21.15.26:60964) 2024-09-11T23:26:36.499 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.494+0000 7f4c527fc640 1 -- 172.21.15.26:0/2267498262 learned_addr learned my addr 172.21.15.26:0/2267498262 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:26:36.499 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.494+0000 7f4c52ffd640 1 --2- 172.21.15.26:0/2267498262 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5645aacf5af0 0x5645aad04420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:36.499 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.494+0000 7f4c537fe640 1 --2- 172.21.15.26:0/2267498262 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5645aad00cb0 0x5645aad79860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:36.499 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.494+0000 7f4c527fc640 1 -- 172.21.15.26:0/2267498262 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5645aad00cb0 msgr2=0x5645aad79860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:26:36.499 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.494+0000 7f4c527fc640 1 --2- 172.21.15.26:0/2267498262 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5645aad00cb0 0x5645aad79860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:36.499 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.494+0000 7f4c527fc640 1 -- 172.21.15.26:0/2267498262 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5645aacf5af0 msgr2=0x5645aad04420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:26:36.499 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.494+0000 7f4c527fc640 1 --2- 172.21.15.26:0/2267498262 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5645aacf5af0 0x5645aad04420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:36.499 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.494+0000 7f4c527fc640 1 -- 172.21.15.26:0/2267498262 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5645aad79ea0 con 0x5645aacf6410 2024-09-11T23:26:36.499 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.494+0000 7f4c527fc640 1 --2- 172.21.15.26:0/2267498262 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5645aacf6410 0x5645aad04960 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f4c48005940 tx=0x7f4c4800bb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efd580c4149f4ccd server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:26:36.499 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.494+0000 7f4c32ffd640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:26:36.499 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.494+0000 7f4c32ffd640 1 -- 172.21.15.26:0/2267498262 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c48004a80 con 0x5645aacf6410 2024-09-11T23:26:36.500 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.494+0000 7f4c32ffd640 1 -- 172.21.15.26:0/2267498262 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8542+0+0 (secure 0 0 0) 0x7f4c48005000 con 0x5645aacf6410 2024-09-11T23:26:36.500 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.494+0000 7f4c5c5c3740 1 -- 172.21.15.26:0/2267498262 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4c48004c80 con 0x5645aacf6410 2024-09-11T23:26:36.500 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.495+0000 7f4c32ffd640 1 -- 172.21.15.26:0/2267498262 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f4c4803c050 con 0x5645aacf6410 2024-09-11T23:26:36.500 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.495+0000 7f4c5c5c3740 10 client.16725 fetch_fsmap learned FSMap version 48 2024-09-11T23:26:36.500 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.495+0000 7f4c5c5c3740 1 -- 172.21.15.26:0/2267498262 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x5645aad7b1b0 con 0x5645aacf6410 2024-09-11T23:26:36.500 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.495+0000 7f4c32ffd640 1 -- 172.21.15.26:0/2267498262 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c48004c80 con 0x5645aacf6410 2024-09-11T23:26:36.500 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.495+0000 7f4c32ffd640 1 -- 172.21.15.26:0/2267498262 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 48) ==== 8406+0+0 (secure 0 0 0) 0x7f4c4802e4b0 con 0x5645aacf6410 2024-09-11T23:26:36.500 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.495+0000 7f4c5c5c3740 10 client.16725 fetch_fsmap finished waiting for FSMap version 48 2024-09-11T23:26:36.500 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.495+0000 7f4c5c5c3740 10 client.16725 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:26:36.500 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.495+0000 7f4c5c5c3740 20 client.16725 populate_metadata read hostname 'smithi026' 2024-09-11T23:26:36.500 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.495+0000 7f4c5c5c3740 1 --2- 172.21.15.26:0/2267498262 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5645aad7b8a0 0x5645aad7dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:36.500 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.495+0000 7f4c5c5c3740 4 client.16725 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T23:26:36.500 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.495+0000 7f4c5c5c3740 1 -- 172.21.15.26:0/2267498262 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5645aac61d70 con 0x5645aad7b8a0 2024-09-11T23:26:36.500 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.495+0000 7f4c52ffd640 1 --2- 172.21.15.26:0/2267498262 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5645aad7b8a0 0x5645aad7dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:36.501 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.496+0000 7f4c32ffd640 10 client.16725 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:26:36.501 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.496+0000 7f4c52ffd640 1 --2- 172.21.15.26:0/2267498262 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5645aad7b8a0 0x5645aad7dcc0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=266797fd1cb910d3 server_cookie=c15f6494b60e6744 in_seq=0 out_seq=0 2024-09-11T23:26:36.514 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.513+0000 7f4c32ffd640 1 -- 172.21.15.26:0/2267498262 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5645aac61d70 con 0x5645aad7b8a0 2024-09-11T23:26:36.514 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.513+0000 7f4c32ffd640 10 client.16725 handle_command_reply: tid=0 2024-09-11T23:26:36.514 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.513+0000 7f4c5c5c3740 1 client.16725 shutdown 2024-09-11T23:26:36.514 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.513+0000 7f4c5c5c3740 1 -- 172.21.15.26:0/2267498262 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5645aacf6410 msgr2=0x5645aad04960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:26:36.514 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.513+0000 7f4c5c5c3740 1 --2- 172.21.15.26:0/2267498262 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5645aacf6410 0x5645aad04960 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f4c48005940 tx=0x7f4c4800bb40 comp rx=0 tx=0).stop 2024-09-11T23:26:36.514 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.513+0000 7f4c5c5c3740 1 -- 172.21.15.26:0/2267498262 shutdown_connections 2024-09-11T23:26:36.514 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.513+0000 7f4c5c5c3740 1 --2- 172.21.15.26:0/2267498262 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5645aad7b8a0 0x5645aad7dcc0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:36.514 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.513+0000 7f4c5c5c3740 1 --2- 172.21.15.26:0/2267498262 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5645aad00cb0 0x5645aad79860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:36.514 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.513+0000 7f4c5c5c3740 1 --2- 172.21.15.26:0/2267498262 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5645aacf6410 0x5645aad04960 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:36.514 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.513+0000 7f4c5c5c3740 1 --2- 172.21.15.26:0/2267498262 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5645aacf5af0 0x5645aad04420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:36.514 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.513+0000 7f4c5c5c3740 1 -- 172.21.15.26:0/2267498262 >> 172.21.15.26:0/2267498262 conn(0x5645aacf2060 msgr2=0x5645aacf11c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:26:36.515 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.513+0000 7f4c32ffd640 0 client.16725 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:26:36.515 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.513+0000 7f4c5c5c3740 1 -- 172.21.15.26:0/2267498262 shutdown_connections 2024-09-11T23:26:36.515 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.513+0000 7f4c5c5c3740 1 -- 172.21.15.26:0/2267498262 wait complete. 2024-09-11T23:26:36.515 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.514+0000 7f4c5c5c3740 20 client.16725 trim_cache size 0 max 16384 2024-09-11T23:26:36.515 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-09-11T23:26:36.515 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "message": "", 2024-09-11T23:26:36.515 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-09-11T23:26:36.515 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-09-11T23:26:36.516 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.514+0000 7f4c59827640 1 -- 172.21.15.26:0/1702933344 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f4c3c0a2da0 msgr2=0x7f4c3c0a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:26:36.516 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.514+0000 7f4c59827640 1 --2- 172.21.15.26:0/1702933344 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f4c3c0a2da0 0x7f4c3c0a5260 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f4c4c04c000 tx=0x7f4c4c04d000 comp rx=0 tx=0).stop 2024-09-11T23:26:36.516 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.514+0000 7f4c59827640 1 -- 172.21.15.26:0/1702933344 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4c541009d0 msgr2=0x7f4c54198300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:26:36.516 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.514+0000 7f4c59827640 1 --2- 172.21.15.26:0/1702933344 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4c541009d0 0x7f4c54198300 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f4c4400c970 tx=0x7f4c4400ce30 comp rx=0 tx=0).stop 2024-09-11T23:26:36.517 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.514+0000 7f4c59827640 1 -- 172.21.15.26:0/1702933344 shutdown_connections 2024-09-11T23:26:36.517 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.514+0000 7f4c59827640 1 --2- 172.21.15.26:0/1702933344 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f4c3c0a2da0 0x7f4c3c0a5260 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:36.517 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.514+0000 7f4c59827640 1 --2- 172.21.15.26:0/1702933344 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4c54101370 0x7f4c54198840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:36.517 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.514+0000 7f4c59827640 1 --2- 172.21.15.26:0/1702933344 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4c541069e0 0x7f4c5419cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:36.517 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.514+0000 7f4c59827640 1 --2- 172.21.15.26:0/1702933344 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4c541009d0 0x7f4c54198300 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:36.517 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.514+0000 7f4c59827640 1 -- 172.21.15.26:0/1702933344 >> 172.21.15.26:0/1702933344 conn(0x7f4c540fc6b0 msgr2=0x7f4c540fddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:26:36.517 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.514+0000 7f4c59827640 1 -- 172.21.15.26:0/1702933344 shutdown_connections 2024-09-11T23:26:36.517 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:36.514+0000 7f4c59827640 1 -- 172.21.15.26:0/1702933344 wait complete. 2024-09-11T23:26:37.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:26:37 smithi136 ceph-mon[35121]: pgmap v254: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 28 KiB/s wr, 18 op/s 2024-09-11T23:26:38.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:26:37 smithi026 ceph-mon[31172]: pgmap v254: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 28 KiB/s wr, 18 op/s 2024-09-11T23:26:38.054 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:26:37 smithi076 ceph-mon[35296]: pgmap v254: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 28 KiB/s wr, 18 op/s 2024-09-11T23:26:40.000 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:26:39 smithi026 ceph-mon[31172]: pgmap v255: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 28 KiB/s wr, 15 op/s 2024-09-11T23:26:40.052 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:26:39 smithi076 ceph-mon[35296]: pgmap v255: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 28 KiB/s wr, 15 op/s 2024-09-11T23:26:40.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:26:39 smithi136 ceph-mon[35121]: pgmap v255: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 28 KiB/s wr, 15 op/s 2024-09-11T23:26:42.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:26:41 smithi026 ceph-mon[31172]: pgmap v256: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 77 KiB/s wr, 25 op/s 2024-09-11T23:26:42.052 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:26:41 smithi076 ceph-mon[35296]: pgmap v256: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 77 KiB/s wr, 25 op/s 2024-09-11T23:26:42.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:26:41 smithi136 ceph-mon[35121]: pgmap v256: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 77 KiB/s wr, 25 op/s 2024-09-11T23:26:43.052 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:26:42 smithi076 ceph-mon[35296]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:26:43.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:26:42 smithi136 ceph-mon[35121]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:26:43.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:26:42 smithi026 ceph-mon[31172]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:26:44.052 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:26:43 smithi076 ceph-mon[35296]: pgmap v257: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 62 KiB/s wr, 17 op/s 2024-09-11T23:26:44.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:26:43 smithi136 ceph-mon[35121]: pgmap v257: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 62 KiB/s wr, 17 op/s 2024-09-11T23:26:44.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:26:43 smithi026 ceph-mon[31172]: pgmap v257: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 62 KiB/s wr, 17 op/s 2024-09-11T23:26:46.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:26:45 smithi076 ceph-mon[35296]: pgmap v258: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 986 KiB/s rd, 1001 KiB/s wr, 25 op/s 2024-09-11T23:26:46.242 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:26:45 smithi136 ceph-mon[35121]: pgmap v258: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 986 KiB/s rd, 1001 KiB/s wr, 25 op/s 2024-09-11T23:26:46.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:26:45 smithi026 ceph-mon[31172]: pgmap v258: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 986 KiB/s rd, 1001 KiB/s wr, 25 op/s 2024-09-11T23:26:48.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:26:47 smithi076 ceph-mon[35296]: pgmap v259: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1003 KiB/s rd, 1013 KiB/s wr, 23 op/s 2024-09-11T23:26:48.241 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:26:47 smithi136 ceph-mon[35121]: pgmap v259: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1003 KiB/s rd, 1013 KiB/s wr, 23 op/s 2024-09-11T23:26:48.258 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:26:47 smithi026 ceph-mon[31172]: pgmap v259: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1003 KiB/s rd, 1013 KiB/s wr, 23 op/s 2024-09-11T23:26:49.615 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.614+0000 7f4f39d5b640 1 -- 172.21.15.26:0/2619331435 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4f34102d20 msgr2=0x7f4f34103100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:26:49.615 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.614+0000 7f4f39d5b640 1 --2- 172.21.15.26:0/2619331435 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4f34102d20 0x7f4f34103100 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f4f24009900 tx=0x7f4f2402a480 comp rx=0 tx=0).stop 2024-09-11T23:26:49.615 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.614+0000 7f4f39d5b640 1 -- 172.21.15.26:0/2619331435 shutdown_connections 2024-09-11T23:26:49.616 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.614+0000 7f4f39d5b640 1 --2- 172.21.15.26:0/2619331435 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4f3410c0f0 0x7f4f3410e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:49.616 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.614+0000 7f4f39d5b640 1 --2- 172.21.15.26:0/2619331435 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4f34103640 0x7f4f3410bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:49.616 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.614+0000 7f4f39d5b640 1 --2- 172.21.15.26:0/2619331435 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4f34102d20 0x7f4f34103100 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:49.616 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.614+0000 7f4f39d5b640 1 -- 172.21.15.26:0/2619331435 >> 172.21.15.26:0/2619331435 conn(0x7f4f340fc6b0 msgr2=0x7f4f340fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:26:49.616 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.614+0000 7f4f39d5b640 1 -- 172.21.15.26:0/2619331435 shutdown_connections 2024-09-11T23:26:49.616 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.614+0000 7f4f39d5b640 1 -- 172.21.15.26:0/2619331435 wait complete. 2024-09-11T23:26:49.616 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.615+0000 7f4f39d5b640 1 Processor -- start 2024-09-11T23:26:49.618 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.615+0000 7f4f39d5b640 1 -- start start 2024-09-11T23:26:49.618 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.615+0000 7f4f39d5b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4f34102d20 0x7f4f34100970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:49.618 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.615+0000 7f4f39d5b640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4f34103640 0x7f4f34100eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:49.618 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.615+0000 7f4f39d5b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4f3410c0f0 0x7f4f341013f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:49.618 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.615+0000 7f4f39d5b640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f4f34111ef0 con 0x7f4f3410c0f0 2024-09-11T23:26:49.618 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.615+0000 7f4f39d5b640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f4f34111d70 con 0x7f4f34102d20 2024-09-11T23:26:49.619 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.615+0000 7f4f39d5b640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f4f34112070 con 0x7f4f34103640 2024-09-11T23:26:49.619 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.615+0000 7f4f33fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4f3410c0f0 0x7f4f341013f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:49.619 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.615+0000 7f4f33fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4f3410c0f0 0x7f4f341013f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55738/0 (socket says 172.21.15.26:55738) 2024-09-11T23:26:49.619 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.615+0000 7f4f33fff640 1 -- 172.21.15.26:0/352467615 learned_addr learned my addr 172.21.15.26:0/352467615 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:26:49.619 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.615+0000 7f4f337fe640 1 --2- 172.21.15.26:0/352467615 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4f34102d20 0x7f4f34100970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:49.619 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.616+0000 7f4f32ffd640 1 --2- 172.21.15.26:0/352467615 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4f34103640 0x7f4f34100eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:49.619 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.616+0000 7f4f33fff640 1 -- 172.21.15.26:0/352467615 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4f34103640 msgr2=0x7f4f34100eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:26:49.619 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.616+0000 7f4f33fff640 1 --2- 172.21.15.26:0/352467615 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4f34103640 0x7f4f34100eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:49.619 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.616+0000 7f4f33fff640 1 -- 172.21.15.26:0/352467615 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4f34102d20 msgr2=0x7f4f34100970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:26:49.619 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.616+0000 7f4f33fff640 1 --2- 172.21.15.26:0/352467615 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4f34102d20 0x7f4f34100970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:49.619 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.616+0000 7f4f33fff640 1 -- 172.21.15.26:0/352467615 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f341a6940 con 0x7f4f3410c0f0 2024-09-11T23:26:49.619 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.616+0000 7f4f32ffd640 1 --2- 172.21.15.26:0/352467615 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4f34103640 0x7f4f34100eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:26:49.619 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.616+0000 7f4f33fff640 1 --2- 172.21.15.26:0/352467615 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4f3410c0f0 0x7f4f341013f0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f4f2c00eaa0 tx=0x7f4f2c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:26:49.619 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.616+0000 7f4f30ff9640 1 -- 172.21.15.26:0/352467615 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f2c00cdb0 con 0x7f4f3410c0f0 2024-09-11T23:26:49.619 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.616+0000 7f4f39d5b640 1 -- 172.21.15.26:0/352467615 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f341a6c30 con 0x7f4f3410c0f0 2024-09-11T23:26:49.620 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.616+0000 7f4f30ff9640 1 -- 172.21.15.26:0/352467615 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f2c005020 con 0x7f4f3410c0f0 2024-09-11T23:26:49.620 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.616+0000 7f4f30ff9640 1 -- 172.21.15.26:0/352467615 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f2c0116b0 con 0x7f4f3410c0f0 2024-09-11T23:26:49.620 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.616+0000 7f4f39d5b640 1 -- 172.21.15.26:0/352467615 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f341a7130 con 0x7f4f3410c0f0 2024-09-11T23:26:49.627 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.618+0000 7f4f30ff9640 1 -- 172.21.15.26:0/352467615 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f4f2c005560 con 0x7f4f3410c0f0 2024-09-11T23:26:49.627 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.619+0000 7f4f3cbcf740 1 Processor -- start 2024-09-11T23:26:49.627 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.619+0000 7f4f3cbcf740 1 -- start start 2024-09-11T23:26:49.627 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.619+0000 7f4f3cbcf740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fce23e3af0 0x55fce23e3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:49.627 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.619+0000 7f4f3cbcf740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fce23e4410 0x55fce23ee770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:49.627 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.619+0000 7f4f3cbcf740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fce23eecb0 0x55fce23f10a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:49.627 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.619+0000 7f4f3cbcf740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55fce23482f0 con 0x55fce23eecb0 2024-09-11T23:26:49.627 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.619+0000 7f4f3cbcf740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55fce22f0270 con 0x55fce23e4410 2024-09-11T23:26:49.627 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.619+0000 7f4f3cbcf740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55fce23442b0 con 0x55fce23e3af0 2024-09-11T23:26:49.628 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.619+0000 7f4f30ff9640 1 --2- 172.21.15.26:0/352467615 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f4f200a2da0 0x7f4f200a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:49.628 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.619+0000 7f4f32ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fce23eecb0 0x55fce23f10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:49.628 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.619+0000 7f4f32ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fce23eecb0 0x55fce23f10a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55744/0 (socket says 172.21.15.26:55744) 2024-09-11T23:26:49.628 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.619+0000 7f4f32ffd640 1 -- 172.21.15.26:0/1955761104 learned_addr learned my addr 172.21.15.26:0/1955761104 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:26:49.628 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.619+0000 7f4f337fe640 1 --2- 172.21.15.26:0/1955761104 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fce23e4410 0x55fce23ee770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:49.628 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.619+0000 7f4f32ffd640 1 --2- 172.21.15.26:0/1955761104 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fce23e3af0 0x55fce23e3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:49.628 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.619+0000 7f4f33fff640 1 --2- 172.21.15.26:0/352467615 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f4f200a2da0 0x7f4f200a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:49.628 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.620+0000 7f4f30ff9640 1 -- 172.21.15.26:0/352467615 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8542+0+0 (secure 0 0 0) 0x7f4f2c0cf670 con 0x7f4f3410c0f0 2024-09-11T23:26:49.628 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.620+0000 7f4f337fe640 1 -- 172.21.15.26:0/1955761104 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fce23e3af0 msgr2=0x55fce23e3ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:26:49.628 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.620+0000 7f4f337fe640 1 --2- 172.21.15.26:0/1955761104 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fce23e3af0 0x55fce23e3ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:49.628 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.620+0000 7f4f337fe640 1 -- 172.21.15.26:0/1955761104 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fce23eecb0 msgr2=0x55fce23f10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:26:49.628 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.620+0000 7f4f337fe640 1 --2- 172.21.15.26:0/1955761104 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fce23eecb0 0x55fce23f10a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:49.629 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.620+0000 7f4f337fe640 1 -- 172.21.15.26:0/1955761104 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fce22ef340 con 0x55fce23e4410 2024-09-11T23:26:49.629 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.620+0000 7f4f337fe640 1 --2- 172.21.15.26:0/1955761104 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fce23e4410 0x55fce23ee770 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f4f24005940 tx=0x7f4f24005e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9236087169f2e6d4 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:26:49.629 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.620+0000 7f4f33fff640 1 --2- 172.21.15.26:0/352467615 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f4f200a2da0 0x7f4f200a5260 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f4f2c0a42c0 tx=0x7f4f2c06db40 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:26:49.629 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.620+0000 7f4f29ffb640 1 -- 172.21.15.26:0/1955761104 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f24040070 con 0x55fce23e4410 2024-09-11T23:26:49.629 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.620+0000 7f4f29ffb640 1 -- 172.21.15.26:0/1955761104 <== mon.2 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f24005020 con 0x55fce23e4410 2024-09-11T23:26:49.629 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.620+0000 7f4f29ffb640 1 -- 172.21.15.26:0/1955761104 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f24035040 con 0x55fce23e4410 2024-09-11T23:26:49.629 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.621+0000 7f4f3cbcf740 1 -- 172.21.15.26:0/1955761104 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fce23e4410 msgr2=0x55fce23ee770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:26:49.629 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.621+0000 7f4f3cbcf740 1 --2- 172.21.15.26:0/1955761104 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fce23e4410 0x55fce23ee770 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f4f24005940 tx=0x7f4f24005e80 comp rx=0 tx=0).stop 2024-09-11T23:26:49.629 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.621+0000 7f4f3cbcf740 1 -- 172.21.15.26:0/1955761104 shutdown_connections 2024-09-11T23:26:49.629 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.621+0000 7f4f3cbcf740 1 --2- 172.21.15.26:0/1955761104 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fce23e4410 0x55fce23ee770 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:49.629 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.621+0000 7f4f3cbcf740 1 --2- 172.21.15.26:0/1955761104 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fce23eecb0 0x55fce23f10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:49.629 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.621+0000 7f4f3cbcf740 1 --2- 172.21.15.26:0/1955761104 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fce23e3af0 0x55fce23e3ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:49.629 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.621+0000 7f4f3cbcf740 1 -- 172.21.15.26:0/1955761104 >> 172.21.15.26:0/1955761104 conn(0x55fce23e0060 msgr2=0x55fce23e0440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:26:49.630 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.621+0000 7f4f3cbcf740 1 -- 172.21.15.26:0/1955761104 shutdown_connections 2024-09-11T23:26:49.630 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.621+0000 7f4f3cbcf740 1 -- 172.21.15.26:0/1955761104 wait complete. 2024-09-11T23:26:49.630 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.621+0000 7f4f3cbcf740 1 -- start start 2024-09-11T23:26:49.630 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.622+0000 7f4f3cbcf740 1 Processor -- start 2024-09-11T23:26:49.630 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.622+0000 7f4f3cbcf740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fce23e3af0 0x55fce23ebc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:49.630 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.622+0000 7f4f3cbcf740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fce23e4410 0x55fce23ec140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:49.630 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.622+0000 7f4f3cbcf740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fce23eecb0 0x55fce23ec680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:49.630 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.622+0000 7f4f3cbcf740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55fce23f1880 con 0x55fce23e3af0 2024-09-11T23:26:49.630 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.622+0000 7f4f3cbcf740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55fce22d1290 con 0x55fce23eecb0 2024-09-11T23:26:49.630 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.622+0000 7f4f3cbcf740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55fce23f1a00 con 0x55fce23e4410 2024-09-11T23:26:49.630 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.622+0000 7f4f3cbcf740 10 client.0 _ll_register_callbacks cb 0x55fce2302140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:26:49.630 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.622+0000 7f4f32ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fce23e3af0 0x55fce23ebc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:49.630 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.622+0000 7f4f32ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fce23e3af0 0x55fce23ebc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55752/0 (socket says 172.21.15.26:55752) 2024-09-11T23:26:49.631 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.622+0000 7f4f32ffd640 1 -- 172.21.15.26:0/1841720463 learned_addr learned my addr 172.21.15.26:0/1841720463 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:26:49.631 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.622+0000 7f4f32ffd640 1 --2- 172.21.15.26:0/1841720463 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fce23eecb0 0x55fce23ec680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:49.631 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.622+0000 7f4f337fe640 1 --2- 172.21.15.26:0/1841720463 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fce23e4410 0x55fce23ec140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:49.631 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.622+0000 7f4f32ffd640 1 -- 172.21.15.26:0/1841720463 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fce23e4410 msgr2=0x55fce23ec140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:26:49.631 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.622+0000 7f4f32ffd640 1 --2- 172.21.15.26:0/1841720463 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fce23e4410 0x55fce23ec140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:49.631 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.622+0000 7f4f32ffd640 1 -- 172.21.15.26:0/1841720463 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fce23eecb0 msgr2=0x55fce23ec680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:26:49.631 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.622+0000 7f4f32ffd640 1 --2- 172.21.15.26:0/1841720463 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fce23eecb0 0x55fce23ec680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:49.631 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.622+0000 7f4f32ffd640 1 -- 172.21.15.26:0/1841720463 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fce205b210 con 0x55fce23e3af0 2024-09-11T23:26:49.631 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.622+0000 7f4f337fe640 1 --2- 172.21.15.26:0/1841720463 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fce23e4410 0x55fce23ec140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:26:49.631 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.623+0000 7f4f32ffd640 1 --2- 172.21.15.26:0/1841720463 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fce23e3af0 0x55fce23ebc00 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f4f1c005fd0 tx=0x7f4f1c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35785c7c0c1ee9fd server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:26:49.631 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.623+0000 7f4f12ffd640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:26:49.631 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.623+0000 7f4f12ffd640 1 -- 172.21.15.26:0/1841720463 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f1c0105e0 con 0x55fce23e3af0 2024-09-11T23:26:49.631 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.623+0000 7f4f12ffd640 1 -- 172.21.15.26:0/1841720463 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8542+0+0 (secure 0 0 0) 0x7f4f1c010780 con 0x55fce23e3af0 2024-09-11T23:26:49.632 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.623+0000 7f4f3cbcf740 1 -- 172.21.15.26:0/1841720463 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4f24005020 con 0x55fce23e3af0 2024-09-11T23:26:49.632 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.623+0000 7f4f12ffd640 1 -- 172.21.15.26:0/1841720463 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f4f1c010980 con 0x55fce23e3af0 2024-09-11T23:26:49.632 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.623+0000 7f4f3cbcf740 10 client.16737 fetch_fsmap learned FSMap version 49 2024-09-11T23:26:49.632 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.623+0000 7f4f3cbcf740 1 -- 172.21.15.26:0/1841720463 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55fce2127620 con 0x55fce23e3af0 2024-09-11T23:26:49.632 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.623+0000 7f4f12ffd640 1 -- 172.21.15.26:0/1841720463 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f24005020 con 0x55fce23e3af0 2024-09-11T23:26:49.632 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.623+0000 7f4f12ffd640 1 -- 172.21.15.26:0/1841720463 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 49) ==== 8402+0+0 (secure 0 0 0) 0x7f4f1c00f040 con 0x55fce23e3af0 2024-09-11T23:26:49.632 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.624+0000 7f4f3cbcf740 10 client.16737 fetch_fsmap finished waiting for FSMap version 49 2024-09-11T23:26:49.632 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.624+0000 7f4f3cbcf740 10 client.16737 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:26:49.632 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.624+0000 7f4f3cbcf740 20 client.16737 populate_metadata read hostname 'smithi026' 2024-09-11T23:26:49.632 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.624+0000 7f4f3cbcf740 1 --2- 172.21.15.26:0/1841720463 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55fce23fec70 0x55fce2401060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:49.632 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.624+0000 7f4f3cbcf740 4 client.16737 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:26:49.632 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.624+0000 7f4f3cbcf740 1 -- 172.21.15.26:0/1841720463 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fce2355a20 con 0x55fce23fec70 2024-09-11T23:26:49.632 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.624+0000 7f4f337fe640 1 --2- 172.21.15.26:0/1841720463 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55fce23fec70 0x55fce2401060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:49.633 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.624+0000 7f4f12ffd640 10 client.16737 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:26:49.633 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.624+0000 7f4f337fe640 1 --2- 172.21.15.26:0/1841720463 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55fce23fec70 0x55fce2401060 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=57991c332bb1e1d7 server_cookie=cd4aef0d300023d3 in_seq=0 out_seq=0 2024-09-11T23:26:49.633 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.625+0000 7f4f12ffd640 1 -- 172.21.15.26:0/1841720463 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fce2355a20 con 0x55fce23fec70 2024-09-11T23:26:49.633 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.625+0000 7f4f12ffd640 10 client.16737 handle_command_reply: tid=0 2024-09-11T23:26:49.633 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.625+0000 7f4f3cbcf740 1 client.16737 shutdown 2024-09-11T23:26:49.633 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.625+0000 7f4f3cbcf740 1 -- 172.21.15.26:0/1841720463 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fce23e3af0 msgr2=0x55fce23ebc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:26:49.633 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.625+0000 7f4f3cbcf740 1 --2- 172.21.15.26:0/1841720463 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fce23e3af0 0x55fce23ebc00 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f4f1c005fd0 tx=0x7f4f1c004730 comp rx=0 tx=0).stop 2024-09-11T23:26:49.633 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.625+0000 7f4f3cbcf740 1 -- 172.21.15.26:0/1841720463 shutdown_connections 2024-09-11T23:26:49.633 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.625+0000 7f4f3cbcf740 1 --2- 172.21.15.26:0/1841720463 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fce23eecb0 0x55fce23ec680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:49.633 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.625+0000 7f4f3cbcf740 1 --2- 172.21.15.26:0/1841720463 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55fce23fec70 0x55fce2401060 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:49.633 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.625+0000 7f4f3cbcf740 1 --2- 172.21.15.26:0/1841720463 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fce23e4410 0x55fce23ec140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:49.633 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.625+0000 7f4f12ffd640 0 client.16737 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:26:49.633 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.625+0000 7f4f3cbcf740 1 --2- 172.21.15.26:0/1841720463 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fce23e3af0 0x55fce23ebc00 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:49.634 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.625+0000 7f4f3cbcf740 1 -- 172.21.15.26:0/1841720463 >> 172.21.15.26:0/1841720463 conn(0x55fce23e0060 msgr2=0x55fce23e5d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:26:49.634 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.625+0000 7f4f3cbcf740 1 -- 172.21.15.26:0/1841720463 shutdown_connections 2024-09-11T23:26:49.634 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.625+0000 7f4f3cbcf740 1 -- 172.21.15.26:0/1841720463 wait complete. 2024-09-11T23:26:49.634 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.625+0000 7f4f3cbcf740 20 client.16737 trim_cache size 0 max 16384 2024-09-11T23:26:49.642 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.640+0000 7f4f3cbcf740 1 Processor -- start 2024-09-11T23:26:49.642 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.640+0000 7f4f3cbcf740 1 -- start start 2024-09-11T23:26:49.642 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.640+0000 7f4f3cbcf740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fce23e3af0 0x55fce23ed460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:49.642 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.640+0000 7f4f3cbcf740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fce23e4410 0x55fce23ec4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:49.642 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.640+0000 7f4f3cbcf740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fce23eecb0 0x55fce2406900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:49.642 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.640+0000 7f4f3cbcf740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f4f341afac0 con 0x55fce23eecb0 2024-09-11T23:26:49.642 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.640+0000 7f4f3cbcf740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f4f341b17e0 con 0x55fce23e4410 2024-09-11T23:26:49.643 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.640+0000 7f4f3cbcf740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f4f341b01f0 con 0x55fce23e3af0 2024-09-11T23:26:49.643 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.641+0000 7f4f32ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fce23eecb0 0x55fce2406900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:49.643 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.641+0000 7f4f32ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fce23eecb0 0x55fce2406900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55766/0 (socket says 172.21.15.26:55766) 2024-09-11T23:26:49.643 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.641+0000 7f4f32ffd640 1 -- 172.21.15.26:0/2849904360 learned_addr learned my addr 172.21.15.26:0/2849904360 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:26:49.643 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.641+0000 7f4f32ffd640 1 --2- 172.21.15.26:0/2849904360 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fce23e3af0 0x55fce23ed460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:49.643 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.641+0000 7f4f337fe640 1 --2- 172.21.15.26:0/2849904360 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fce23e4410 0x55fce23ec4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:49.643 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.641+0000 7f4f32ffd640 1 -- 172.21.15.26:0/2849904360 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fce23e3af0 msgr2=0x55fce23ed460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:26:49.643 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.641+0000 7f4f32ffd640 1 --2- 172.21.15.26:0/2849904360 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fce23e3af0 0x55fce23ed460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:49.643 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.641+0000 7f4f32ffd640 1 -- 172.21.15.26:0/2849904360 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fce23e4410 msgr2=0x55fce23ec4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:26:49.643 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.641+0000 7f4f32ffd640 1 --2- 172.21.15.26:0/2849904360 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fce23e4410 0x55fce23ec4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:49.643 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.641+0000 7f4f32ffd640 1 -- 172.21.15.26:0/2849904360 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fce23e3920 con 0x55fce23eecb0 2024-09-11T23:26:49.643 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.641+0000 7f4f32ffd640 1 --2- 172.21.15.26:0/2849904360 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fce23eecb0 0x55fce2406900 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f4f1c012010 tx=0x7f4f1c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2aa0a60d076955c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:26:49.643 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.641+0000 7f4f297fa640 1 -- 172.21.15.26:0/2849904360 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f1c02e070 con 0x55fce23eecb0 2024-09-11T23:26:49.643 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.641+0000 7f4f297fa640 1 -- 172.21.15.26:0/2849904360 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f1c02f040 con 0x55fce23eecb0 2024-09-11T23:26:49.643 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.641+0000 7f4f297fa640 1 -- 172.21.15.26:0/2849904360 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f1c02a440 con 0x55fce23eecb0 2024-09-11T23:26:49.647 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.642+0000 7f4f3cbcf740 1 -- 172.21.15.26:0/2849904360 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fce23eecb0 msgr2=0x55fce2406900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:26:49.647 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.642+0000 7f4f3cbcf740 1 --2- 172.21.15.26:0/2849904360 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fce23eecb0 0x55fce2406900 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f4f1c012010 tx=0x7f4f1c0046c0 comp rx=0 tx=0).stop 2024-09-11T23:26:49.647 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.642+0000 7f4f3cbcf740 1 -- 172.21.15.26:0/2849904360 shutdown_connections 2024-09-11T23:26:49.647 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.642+0000 7f4f3cbcf740 1 --2- 172.21.15.26:0/2849904360 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fce23e4410 0x55fce23ec4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:49.647 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.642+0000 7f4f3cbcf740 1 --2- 172.21.15.26:0/2849904360 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fce23eecb0 0x55fce2406900 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:49.647 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.642+0000 7f4f3cbcf740 1 --2- 172.21.15.26:0/2849904360 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fce23e3af0 0x55fce23ed460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:49.647 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.642+0000 7f4f3cbcf740 1 -- 172.21.15.26:0/2849904360 >> 172.21.15.26:0/2849904360 conn(0x55fce23e0060 msgr2=0x55fce23df1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:26:49.648 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.642+0000 7f4f3cbcf740 1 -- 172.21.15.26:0/2849904360 shutdown_connections 2024-09-11T23:26:49.648 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.642+0000 7f4f3cbcf740 1 -- 172.21.15.26:0/2849904360 wait complete. 2024-09-11T23:26:49.648 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.642+0000 7f4f3cbcf740 1 -- start start 2024-09-11T23:26:49.648 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.643+0000 7f4f3cbcf740 1 Processor -- start 2024-09-11T23:26:49.648 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.643+0000 7f4f3cbcf740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fce23e3af0 0x55fce23f30b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:49.648 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.643+0000 7f4f3cbcf740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fce23e4410 0x55fce23f35f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:49.648 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.643+0000 7f4f3cbcf740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fce23eecb0 0x55fce24688d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:49.648 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.643+0000 7f4f3cbcf740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f4f341b1110 con 0x55fce23e4410 2024-09-11T23:26:49.648 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.643+0000 7f4f3cbcf740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f4f341b0980 con 0x55fce23e3af0 2024-09-11T23:26:49.648 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.643+0000 7f4f3cbcf740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f4f341adac0 con 0x55fce23eecb0 2024-09-11T23:26:49.648 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.643+0000 7f4f3cbcf740 10 client.0 _ll_register_callbacks cb 0x7f4f1c00bd20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:26:49.648 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.643+0000 7f4f337fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fce23e4410 0x55fce23f35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:49.648 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.643+0000 7f4f337fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fce23e4410 0x55fce23f35f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55770/0 (socket says 172.21.15.26:55770) 2024-09-11T23:26:49.648 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.643+0000 7f4f337fe640 1 -- 172.21.15.26:0/3402377443 learned_addr learned my addr 172.21.15.26:0/3402377443 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:26:49.649 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.643+0000 7f4f32ffd640 1 --2- 172.21.15.26:0/3402377443 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fce23e3af0 0x55fce23f30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:49.649 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.643+0000 7f4f32ffd640 1 --2- 172.21.15.26:0/3402377443 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fce23eecb0 0x55fce24688d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:49.649 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.643+0000 7f4f337fe640 1 -- 172.21.15.26:0/3402377443 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fce23eecb0 msgr2=0x55fce24688d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:26:49.649 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.643+0000 7f4f337fe640 1 --2- 172.21.15.26:0/3402377443 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fce23eecb0 0x55fce24688d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:49.649 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.643+0000 7f4f337fe640 1 -- 172.21.15.26:0/3402377443 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fce23e3af0 msgr2=0x55fce23f30b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:26:49.649 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.643+0000 7f4f337fe640 1 --2- 172.21.15.26:0/3402377443 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fce23e3af0 0x55fce23f30b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:49.649 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.643+0000 7f4f337fe640 1 -- 172.21.15.26:0/3402377443 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fce2468e10 con 0x55fce23e4410 2024-09-11T23:26:49.649 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.644+0000 7f4f337fe640 1 --2- 172.21.15.26:0/3402377443 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fce23e4410 0x55fce23f35f0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f4f240047c0 tx=0x7f4f2402a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc3afc4498280fcf server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:26:49.649 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.644+0000 7f4f12ffd640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:26:49.649 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.644+0000 7f4f12ffd640 1 -- 172.21.15.26:0/3402377443 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f24035040 con 0x55fce23e4410 2024-09-11T23:26:49.649 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.644+0000 7f4f12ffd640 1 -- 172.21.15.26:0/3402377443 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8542+0+0 (secure 0 0 0) 0x7f4f240319e0 con 0x55fce23e4410 2024-09-11T23:26:49.649 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.644+0000 7f4f3cbcf740 1 -- 172.21.15.26:0/3402377443 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4f1c02f040 con 0x55fce23e4410 2024-09-11T23:26:49.649 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.644+0000 7f4f12ffd640 1 -- 172.21.15.26:0/3402377443 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f4f24040040 con 0x55fce23e4410 2024-09-11T23:26:49.649 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.644+0000 7f4f3cbcf740 10 client.16743 fetch_fsmap learned FSMap version 49 2024-09-11T23:26:49.650 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.644+0000 7f4f3cbcf740 1 -- 172.21.15.26:0/3402377443 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55fce246a120 con 0x55fce23e4410 2024-09-11T23:26:49.650 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.644+0000 7f4f12ffd640 1 -- 172.21.15.26:0/3402377443 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f1c02f040 con 0x55fce23e4410 2024-09-11T23:26:49.650 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.644+0000 7f4f12ffd640 1 -- 172.21.15.26:0/3402377443 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 49) ==== 8402+0+0 (secure 0 0 0) 0x7f4f2404d040 con 0x55fce23e4410 2024-09-11T23:26:49.650 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.644+0000 7f4f3cbcf740 10 client.16743 fetch_fsmap finished waiting for FSMap version 49 2024-09-11T23:26:49.650 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.644+0000 7f4f3cbcf740 10 client.16743 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:26:49.650 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.644+0000 7f4f3cbcf740 20 client.16743 populate_metadata read hostname 'smithi026' 2024-09-11T23:26:49.650 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.644+0000 7f4f3cbcf740 1 --2- 172.21.15.26:0/3402377443 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55fce246a460 0x55fce246c850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:49.650 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.644+0000 7f4f3cbcf740 4 client.16743 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T23:26:49.650 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.644+0000 7f4f3cbcf740 1 -- 172.21.15.26:0/3402377443 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55fce234fd70 con 0x55fce246a460 2024-09-11T23:26:49.650 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.645+0000 7f4f32ffd640 1 --2- 172.21.15.26:0/3402377443 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55fce246a460 0x55fce246c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:49.650 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.645+0000 7f4f32ffd640 1 --2- 172.21.15.26:0/3402377443 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55fce246a460 0x55fce246c850 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ccb1762bfac80f2c server_cookie=5e2187e85e3ea9dc in_seq=0 out_seq=0 2024-09-11T23:26:49.650 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.645+0000 7f4f12ffd640 10 client.16743 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:26:49.664 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.662+0000 7f4f12ffd640 1 -- 172.21.15.26:0/3402377443 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55fce234fd70 con 0x55fce246a460 2024-09-11T23:26:49.664 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.662+0000 7f4f12ffd640 10 client.16743 handle_command_reply: tid=0 2024-09-11T23:26:49.664 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.663+0000 7f4f3cbcf740 1 client.16743 shutdown 2024-09-11T23:26:49.664 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.663+0000 7f4f3cbcf740 1 -- 172.21.15.26:0/3402377443 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fce23e4410 msgr2=0x55fce23f35f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:26:49.664 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.663+0000 7f4f3cbcf740 1 --2- 172.21.15.26:0/3402377443 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fce23e4410 0x55fce23f35f0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f4f240047c0 tx=0x7f4f2402a480 comp rx=0 tx=0).stop 2024-09-11T23:26:49.664 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.663+0000 7f4f3cbcf740 1 -- 172.21.15.26:0/3402377443 shutdown_connections 2024-09-11T23:26:49.664 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.663+0000 7f4f3cbcf740 1 --2- 172.21.15.26:0/3402377443 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55fce246a460 0x55fce246c850 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:49.664 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.663+0000 7f4f3cbcf740 1 --2- 172.21.15.26:0/3402377443 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fce23eecb0 0x55fce24688d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:49.664 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.663+0000 7f4f3cbcf740 1 --2- 172.21.15.26:0/3402377443 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fce23e4410 0x55fce23f35f0 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:49.665 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.663+0000 7f4f12ffd640 0 client.16743 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:26:49.665 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.663+0000 7f4f3cbcf740 1 --2- 172.21.15.26:0/3402377443 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fce23e3af0 0x55fce23f30b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:49.665 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.663+0000 7f4f3cbcf740 1 -- 172.21.15.26:0/3402377443 >> 172.21.15.26:0/3402377443 conn(0x55fce23e0060 msgr2=0x55fce23f9b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:26:49.665 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.663+0000 7f4f3cbcf740 1 -- 172.21.15.26:0/3402377443 shutdown_connections 2024-09-11T23:26:49.665 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.663+0000 7f4f3cbcf740 1 -- 172.21.15.26:0/3402377443 wait complete. 2024-09-11T23:26:49.665 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.663+0000 7f4f3cbcf740 20 client.16743 trim_cache size 0 max 16384 2024-09-11T23:26:49.665 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-09-11T23:26:49.665 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "message": "", 2024-09-11T23:26:49.665 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-09-11T23:26:49.665 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-09-11T23:26:49.666 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.664+0000 7f4f39d5b640 1 -- 172.21.15.26:0/352467615 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f4f200a2da0 msgr2=0x7f4f200a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:26:49.666 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.664+0000 7f4f39d5b640 1 --2- 172.21.15.26:0/352467615 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f4f200a2da0 0x7f4f200a5260 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f4f2c0a42c0 tx=0x7f4f2c06db40 comp rx=0 tx=0).stop 2024-09-11T23:26:49.666 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.664+0000 7f4f39d5b640 1 -- 172.21.15.26:0/352467615 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4f3410c0f0 msgr2=0x7f4f341013f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:26:49.666 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.664+0000 7f4f39d5b640 1 --2- 172.21.15.26:0/352467615 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4f3410c0f0 0x7f4f341013f0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f4f2c00eaa0 tx=0x7f4f2c00ef60 comp rx=0 tx=0).stop 2024-09-11T23:26:49.666 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.664+0000 7f4f39d5b640 1 -- 172.21.15.26:0/352467615 shutdown_connections 2024-09-11T23:26:49.666 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.664+0000 7f4f39d5b640 1 --2- 172.21.15.26:0/352467615 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f4f200a2da0 0x7f4f200a5260 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:49.666 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.664+0000 7f4f39d5b640 1 --2- 172.21.15.26:0/352467615 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4f3410c0f0 0x7f4f341013f0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:49.666 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.664+0000 7f4f39d5b640 1 --2- 172.21.15.26:0/352467615 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4f34103640 0x7f4f34100eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:49.667 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.664+0000 7f4f39d5b640 1 --2- 172.21.15.26:0/352467615 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4f34102d20 0x7f4f34100970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:49.667 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.664+0000 7f4f39d5b640 1 -- 172.21.15.26:0/352467615 >> 172.21.15.26:0/352467615 conn(0x7f4f340fc6b0 msgr2=0x7f4f340fd350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:26:49.667 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.664+0000 7f4f39d5b640 1 -- 172.21.15.26:0/352467615 shutdown_connections 2024-09-11T23:26:49.667 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:26:49.664+0000 7f4f39d5b640 1 -- 172.21.15.26:0/352467615 wait complete. 2024-09-11T23:26:50.041 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:26:49 smithi136 ceph-mon[35121]: pgmap v260: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1001 KiB/s rd, 1013 KiB/s wr, 20 op/s 2024-09-11T23:26:50.052 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:26:49 smithi076 ceph-mon[35296]: pgmap v260: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1001 KiB/s rd, 1013 KiB/s wr, 20 op/s 2024-09-11T23:26:50.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:26:49 smithi026 ceph-mon[31172]: pgmap v260: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1001 KiB/s rd, 1013 KiB/s wr, 20 op/s 2024-09-11T23:26:51.141 INFO:tasks.quiescer.fs.[cephfs]:Total duration (38.8) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-11T23:26:51.141 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 19.4 2024-09-11T23:26:51.142 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:26:51.422 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:26:51.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:26:51 smithi026 ceph-mon[31172]: pgmap v261: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1011 KiB/s rd, 1016 KiB/s wr, 26 op/s 2024-09-11T23:26:51.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:26:51 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:26:51.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:26:51 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:26:51.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:26:51 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:26:51.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:26:51 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:26:52.052 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:26:51 smithi076 ceph-mon[35296]: pgmap v261: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1011 KiB/s rd, 1016 KiB/s wr, 26 op/s 2024-09-11T23:26:52.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:26:51 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:26:52.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:26:51 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:26:52.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:26:51 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:26:52.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:26:51 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:26:52.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:52.057+0000 7f13e95f5640 1 -- 172.21.15.26:0/565795804 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f13e4102790 msgr2=0x7f13e41067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:26:52.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:52.057+0000 7f13e95f5640 1 --2- 172.21.15.26:0/565795804 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f13e4102790 0x7f13e41067e0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f13d8009900 tx=0x7f13d802a410 comp rx=0 tx=0).stop 2024-09-11T23:26:52.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:52.057+0000 7f13e95f5640 1 -- 172.21.15.26:0/565795804 shutdown_connections 2024-09-11T23:26:52.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:52.057+0000 7f13e95f5640 1 --2- 172.21.15.26:0/565795804 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f13e4106f10 0x7f13e410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:52.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:52.057+0000 7f13e95f5640 1 --2- 172.21.15.26:0/565795804 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f13e4102790 0x7f13e41067e0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:52.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:52.057+0000 7f13e95f5640 1 --2- 172.21.15.26:0/565795804 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f13e4101de0 0x7f13e41021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:52.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:52.057+0000 7f13e95f5640 1 -- 172.21.15.26:0/565795804 >> 172.21.15.26:0/565795804 conn(0x7f13e40fd8c0 msgr2=0x7f13e40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:26:52.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:52.057+0000 7f13e95f5640 1 -- 172.21.15.26:0/565795804 shutdown_connections 2024-09-11T23:26:52.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:52.057+0000 7f13e95f5640 1 -- 172.21.15.26:0/565795804 wait complete. 2024-09-11T23:26:52.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:52.058+0000 7f13e95f5640 1 Processor -- start 2024-09-11T23:26:52.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:52.058+0000 7f13e95f5640 1 -- start start 2024-09-11T23:26:52.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:52.058+0000 7f13e95f5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f13e4101de0 0x7f13e4198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:52.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:52.058+0000 7f13e95f5640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f13e4102790 0x7f13e4198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:52.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:52.058+0000 7f13e95f5640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f13e4106f10 0x7f13e419cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:52.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:52.058+0000 7f13e95f5640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f13e4111f30 con 0x7f13e4101de0 2024-09-11T23:26:52.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:52.058+0000 7f13e95f5640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f13e4111db0 con 0x7f13e4102790 2024-09-11T23:26:52.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:52.058+0000 7f13e95f5640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f13e41120b0 con 0x7f13e4106f10 2024-09-11T23:26:52.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:52.058+0000 7f13e2ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f13e4101de0 0x7f13e4198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:52.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:52.058+0000 7f13e2ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f13e4101de0 0x7f13e4198430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56312/0 (socket says 172.21.15.26:56312) 2024-09-11T23:26:52.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:52.058+0000 7f13e2ffd640 1 -- 172.21.15.26:0/428516101 learned_addr learned my addr 172.21.15.26:0/428516101 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:26:52.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:52.058+0000 7f13e27fc640 1 --2- 172.21.15.26:0/428516101 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f13e4102790 0x7f13e4198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:52.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:52.058+0000 7f13e37fe640 1 --2- 172.21.15.26:0/428516101 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f13e4106f10 0x7f13e419cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:52.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:52.059+0000 7f13e2ffd640 1 -- 172.21.15.26:0/428516101 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f13e4106f10 msgr2=0x7f13e419cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:26:52.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:52.059+0000 7f13e2ffd640 1 --2- 172.21.15.26:0/428516101 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f13e4106f10 0x7f13e419cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:52.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:52.059+0000 7f13e2ffd640 1 -- 172.21.15.26:0/428516101 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f13e4102790 msgr2=0x7f13e4198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:26:52.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:52.059+0000 7f13e2ffd640 1 --2- 172.21.15.26:0/428516101 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f13e4102790 0x7f13e4198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:52.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:52.059+0000 7f13e2ffd640 1 -- 172.21.15.26:0/428516101 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13e419d3e0 con 0x7f13e4101de0 2024-09-11T23:26:52.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:52.059+0000 7f13e37fe640 1 --2- 172.21.15.26:0/428516101 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f13e4106f10 0x7f13e419cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:26:52.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:52.059+0000 7f13e27fc640 1 --2- 172.21.15.26:0/428516101 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f13e4102790 0x7f13e4198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:26:52.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:52.059+0000 7f13e2ffd640 1 --2- 172.21.15.26:0/428516101 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f13e4101de0 0x7f13e4198430 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f13d400cce0 tx=0x7f13d400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:26:52.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:52.059+0000 7f13cbfff640 1 -- 172.21.15.26:0/428516101 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13d4004b20 con 0x7f13e4101de0 2024-09-11T23:26:52.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:52.059+0000 7f13cbfff640 1 -- 172.21.15.26:0/428516101 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f13d4004cc0 con 0x7f13e4101de0 2024-09-11T23:26:52.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:52.059+0000 7f13e95f5640 1 -- 172.21.15.26:0/428516101 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13e419d6d0 con 0x7f13e4101de0 2024-09-11T23:26:52.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:52.059+0000 7f13cbfff640 1 -- 172.21.15.26:0/428516101 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13d400f7d0 con 0x7f13e4101de0 2024-09-11T23:26:52.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:52.059+0000 7f13e95f5640 1 -- 172.21.15.26:0/428516101 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13e41a4fb0 con 0x7f13e4101de0 2024-09-11T23:26:52.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:52.060+0000 7f13e95f5640 1 -- 172.21.15.26:0/428516101 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13e41021c0 con 0x7f13e4101de0 2024-09-11T23:26:52.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:52.061+0000 7f13cbfff640 1 -- 172.21.15.26:0/428516101 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f13d400f970 con 0x7f13e4101de0 2024-09-11T23:26:52.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:52.062+0000 7f13cbfff640 1 --2- 172.21.15.26:0/428516101 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f13cc0a2cf0 0x7f13cc0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:52.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:52.062+0000 7f13e27fc640 1 --2- 172.21.15.26:0/428516101 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f13cc0a2cf0 0x7f13cc0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:52.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:52.062+0000 7f13cbfff640 1 -- 172.21.15.26:0/428516101 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8542+0+0 (secure 0 0 0) 0x7f13d40cf590 con 0x7f13e4101de0 2024-09-11T23:26:52.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:52.063+0000 7f13e27fc640 1 --2- 172.21.15.26:0/428516101 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f13cc0a2cf0 0x7f13cc0a51b0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f13d8009230 tx=0x7f13d80058d0 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:26:52.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:52.067+0000 7f13cbfff640 1 -- 172.21.15.26:0/428516101 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f13d4014030 con 0x7f13e4101de0 2024-09-11T23:26:52.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:26:51 smithi136 ceph-mon[35121]: pgmap v261: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1011 KiB/s rd, 1016 KiB/s wr, 26 op/s 2024-09-11T23:26:52.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:26:51 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:26:52.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:26:51 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:26:52.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:26:51 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:26:52.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:26:51 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:26:52.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:52.278+0000 7f13e95f5640 1 -- 172.21.15.26:0/428516101 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f13e402d7c0 con 0x7f13e4101de0 2024-09-11T23:26:52.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:52.279+0000 7f13cbfff640 1 -- 172.21.15.26:0/428516101 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) ==== 94+0+11989 (secure 0 0 0) 0x7f13d409c140 con 0x7f13e4101de0 2024-09-11T23:26:52.280 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-11T23:26:52.280 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":49,"btime":"2024-09-11T23:26:41:742392+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14499,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/367659515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":367659515},{"type":"v1","addr":"172.21.15.26:6835","nonce":367659515}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14511,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/2041065299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":2041065299},{"type":"v1","addr":"172.21.15.26:6837","nonce":2041065299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":49,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:16:26.544203+0000","modified":"2024-09-11T23:26:41.604856+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24485,"mds_1":24463,"mds_2":24458,"mds_3":24448,"mds_4":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14550":{"gid":14550,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6839/2233068305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":2233068305},{"type":"v1","addr":"172.21.15.26:6839","nonce":2233068305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14562":{"gid":14562,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6841/2984982336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":2984982336},{"type":"v1","addr":"172.21.15.26:6841","nonce":2984982336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6833/2019841455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":2019841455},{"type":"v1","addr":"172.21.15.136:6833","nonce":2019841455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.76:6833/3438147050","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3438147050},{"type":"v1","addr":"172.21.15.76:6833","nonce":3438147050}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6835/3617665975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3617665975},{"type":"v1","addr":"172.21.15.136:6835","nonce":3617665975}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":12,"addr":"172.21.15.76:6835/393308499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":393308499},{"type":"v1","addr":"172.21.15.76:6835","nonce":393308499}]},"join_fscid":-1,"export_targets":[2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":10,"addr":"172.21.15.136:6837/1633082387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":1633082387},{"type":"v1","addr":"172.21.15.136:6837","nonce":1633082387}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.76:6837/2078978748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":2078978748},{"type":"v1","addr":"172.21.15.76:6837","nonce":2078978748}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":63,"addr":"172.21.15.136:6839/3211398630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":3211398630},{"type":"v1","addr":"172.21.15.136:6839","nonce":3211398630}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.76:6839/2077637090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6838","nonce":2077637090},{"type":"v1","addr":"172.21.15.76:6839","nonce":2077637090}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24485,"qdb_cluster":[24443,24448,24458,24463,24485]},"id":1}]} 2024-09-11T23:26:52.280 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 49 2024-09-11T23:26:52.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:52.281+0000 7f13e95f5640 1 -- 172.21.15.26:0/428516101 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f13cc0a2cf0 msgr2=0x7f13cc0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:26:52.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:52.281+0000 7f13e95f5640 1 --2- 172.21.15.26:0/428516101 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f13cc0a2cf0 0x7f13cc0a51b0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f13d8009230 tx=0x7f13d80058d0 comp rx=0 tx=0).stop 2024-09-11T23:26:52.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:52.281+0000 7f13e95f5640 1 -- 172.21.15.26:0/428516101 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f13e4101de0 msgr2=0x7f13e4198430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:26:52.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:52.281+0000 7f13e95f5640 1 --2- 172.21.15.26:0/428516101 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f13e4101de0 0x7f13e4198430 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f13d400cce0 tx=0x7f13d400a620 comp rx=0 tx=0).stop 2024-09-11T23:26:52.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:52.281+0000 7f13e95f5640 1 -- 172.21.15.26:0/428516101 shutdown_connections 2024-09-11T23:26:52.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:52.281+0000 7f13e95f5640 1 --2- 172.21.15.26:0/428516101 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f13cc0a2cf0 0x7f13cc0a51b0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:52.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:52.281+0000 7f13e95f5640 1 --2- 172.21.15.26:0/428516101 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f13e4102790 0x7f13e4198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:52.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:52.281+0000 7f13e95f5640 1 --2- 172.21.15.26:0/428516101 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f13e4106f10 0x7f13e419cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:52.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:52.281+0000 7f13e95f5640 1 --2- 172.21.15.26:0/428516101 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f13e4101de0 0x7f13e4198430 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:52.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:52.281+0000 7f13e95f5640 1 -- 172.21.15.26:0/428516101 >> 172.21.15.26:0/428516101 conn(0x7f13e40fd8c0 msgr2=0x7f13e40fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:26:52.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:52.281+0000 7f13e95f5640 1 -- 172.21.15.26:0/428516101 shutdown_connections 2024-09-11T23:26:52.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:52.281+0000 7f13e95f5640 1 -- 172.21.15.26:0/428516101 wait complete. 2024-09-11T23:26:53.052 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:26:52 smithi076 ceph-mon[35296]: from='client.? 172.21.15.26:0/428516101' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:26:53.136 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:26:52 smithi026 ceph-mon[31172]: from='client.? 172.21.15.26:0/428516101' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:26:53.172 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24485 quiesce db / --timeout 90 --expiration 139.4 --await-for 45.0' 2024-09-11T23:26:53.172 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24485 quiesce db / --timeout 90 --expiration 139.4 --await-for 45.0 2024-09-11T23:26:53.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:26:52 smithi136 ceph-mon[35121]: from='client.? 172.21.15.26:0/428516101' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:26:53.440 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:26:53.832 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:26:53 smithi026 ceph-mon[31172]: pgmap v262: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 967 KiB/s rd, 967 KiB/s wr, 17 op/s 2024-09-11T23:26:53.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.925+0000 7f0f12963640 1 -- 172.21.15.26:0/3600453037 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f0c102790 msgr2=0x7f0f0c1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:26:53.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.925+0000 7f0f12963640 1 --2- 172.21.15.26:0/3600453037 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f0c102790 0x7f0f0c1067e0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f0ef4009900 tx=0x7f0ef402a410 comp rx=0 tx=0).stop 2024-09-11T23:26:53.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.925+0000 7f0f12963640 1 -- 172.21.15.26:0/3600453037 shutdown_connections 2024-09-11T23:26:53.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.925+0000 7f0f12963640 1 --2- 172.21.15.26:0/3600453037 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0f0c106f10 0x7f0f0c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:53.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.925+0000 7f0f12963640 1 --2- 172.21.15.26:0/3600453037 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f0c102790 0x7f0f0c1067e0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:53.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.925+0000 7f0f12963640 1 --2- 172.21.15.26:0/3600453037 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0f0c101de0 0x7f0f0c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:53.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.925+0000 7f0f12963640 1 -- 172.21.15.26:0/3600453037 >> 172.21.15.26:0/3600453037 conn(0x7f0f0c0fd8c0 msgr2=0x7f0f0c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:26:53.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.925+0000 7f0f12963640 1 -- 172.21.15.26:0/3600453037 shutdown_connections 2024-09-11T23:26:53.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.925+0000 7f0f12963640 1 -- 172.21.15.26:0/3600453037 wait complete. 2024-09-11T23:26:53.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.925+0000 7f0f12963640 1 Processor -- start 2024-09-11T23:26:53.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.925+0000 7f0f12963640 1 -- start start 2024-09-11T23:26:53.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.926+0000 7f0f12963640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0f0c101de0 0x7f0f0c198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:53.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.926+0000 7f0f12963640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f0c102790 0x7f0f0c198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:53.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.926+0000 7f0f12963640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0f0c106f10 0x7f0f0c19cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:53.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.926+0000 7f0f12963640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f0f0c111fc0 con 0x7f0f0c102790 2024-09-11T23:26:53.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.926+0000 7f0f12963640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f0f0c111e40 con 0x7f0f0c101de0 2024-09-11T23:26:53.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.926+0000 7f0f12963640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f0f0c112140 con 0x7f0f0c106f10 2024-09-11T23:26:53.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.926+0000 7f0f03fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f0c102790 0x7f0f0c198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:53.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.926+0000 7f0f03fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f0c102790 0x7f0f0c198970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56318/0 (socket says 172.21.15.26:56318) 2024-09-11T23:26:53.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.926+0000 7f0f03fff640 1 -- 172.21.15.26:0/2202520038 learned_addr learned my addr 172.21.15.26:0/2202520038 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:26:53.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.926+0000 7f0f0bfff640 1 --2- 172.21.15.26:0/2202520038 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0f0c101de0 0x7f0f0c198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:53.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.926+0000 7f0f10f00640 1 --2- 172.21.15.26:0/2202520038 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0f0c106f10 0x7f0f0c19cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:53.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.926+0000 7f0f03fff640 1 -- 172.21.15.26:0/2202520038 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0f0c106f10 msgr2=0x7f0f0c19cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:26:53.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.926+0000 7f0f03fff640 1 --2- 172.21.15.26:0/2202520038 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0f0c106f10 0x7f0f0c19cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:53.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.926+0000 7f0f03fff640 1 -- 172.21.15.26:0/2202520038 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0f0c101de0 msgr2=0x7f0f0c198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:26:53.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.926+0000 7f0f03fff640 1 --2- 172.21.15.26:0/2202520038 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0f0c101de0 0x7f0f0c198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:53.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.926+0000 7f0f03fff640 1 -- 172.21.15.26:0/2202520038 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f0c19d3e0 con 0x7f0f0c102790 2024-09-11T23:26:53.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.926+0000 7f0f03fff640 1 --2- 172.21.15.26:0/2202520038 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f0c102790 0x7f0f0c198970 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f0ef4004770 tx=0x7f0ef4003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:26:53.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.926+0000 7f0f09ffb640 1 -- 172.21.15.26:0/2202520038 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ef403f070 con 0x7f0f0c102790 2024-09-11T23:26:53.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.926+0000 7f0f12963640 1 -- 172.21.15.26:0/2202520038 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f0c19d670 con 0x7f0f0c102790 2024-09-11T23:26:53.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.927+0000 7f0f09ffb640 1 -- 172.21.15.26:0/2202520038 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ef4031ca0 con 0x7f0f0c102790 2024-09-11T23:26:53.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.927+0000 7f0f09ffb640 1 -- 172.21.15.26:0/2202520038 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ef4035030 con 0x7f0f0c102790 2024-09-11T23:26:53.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.927+0000 7f0f12963640 1 -- 172.21.15.26:0/2202520038 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f0c1a4fb0 con 0x7f0f0c102790 2024-09-11T23:26:53.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.929+0000 7f0f09ffb640 1 -- 172.21.15.26:0/2202520038 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f0ef4042070 con 0x7f0f0c102790 2024-09-11T23:26:53.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.929+0000 7f0f14d3c740 1 Processor -- start 2024-09-11T23:26:53.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.929+0000 7f0f14d3c740 1 -- start start 2024-09-11T23:26:53.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.929+0000 7f0f14d3c740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556a7b593760 0x556a7b593b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:53.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.929+0000 7f0f14d3c740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556a7b594080 0x556a7b59e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:53.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.929+0000 7f0f14d3c740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556a7b59e940 0x556a7b5a0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:53.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.929+0000 7f0f14d3c740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x556a7b4f92a0 con 0x556a7b593760 2024-09-11T23:26:53.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.929+0000 7f0f14d3c740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x556a7b4a0b10 con 0x556a7b594080 2024-09-11T23:26:53.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.929+0000 7f0f14d3c740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x556a7b4f5260 con 0x556a7b59e940 2024-09-11T23:26:53.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.929+0000 7f0f10f00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556a7b593760 0x556a7b593b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:53.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.929+0000 7f0f10f00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556a7b593760 0x556a7b593b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56326/0 (socket says 172.21.15.26:56326) 2024-09-11T23:26:53.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.929+0000 7f0f10f00640 1 -- 172.21.15.26:0/2010668825 learned_addr learned my addr 172.21.15.26:0/2010668825 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:26:53.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.930+0000 7f0f03fff640 1 --2- 172.21.15.26:0/2010668825 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556a7b59e940 0x556a7b5a0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:53.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.930+0000 7f0f0bfff640 1 --2- 172.21.15.26:0/2010668825 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556a7b594080 0x556a7b59e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:53.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.930+0000 7f0f09ffb640 1 --2- 172.21.15.26:0/2202520038 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f0ef80a2cf0 0x7f0ef80a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:53.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.930+0000 7f0f10f00640 1 -- 172.21.15.26:0/2010668825 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556a7b59e940 msgr2=0x556a7b5a0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:26:53.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.930+0000 7f0f10f00640 1 --2- 172.21.15.26:0/2010668825 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556a7b59e940 0x556a7b5a0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:53.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.930+0000 7f0f10f00640 1 -- 172.21.15.26:0/2010668825 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556a7b594080 msgr2=0x556a7b59e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:26:53.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.930+0000 7f0f10f00640 1 --2- 172.21.15.26:0/2010668825 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556a7b594080 0x556a7b59e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:53.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.930+0000 7f0f10f00640 1 -- 172.21.15.26:0/2010668825 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a7b26ab90 con 0x556a7b593760 2024-09-11T23:26:53.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.930+0000 7f0f0bfff640 1 --2- 172.21.15.26:0/2010668825 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556a7b594080 0x556a7b59e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:26:53.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.930+0000 7f0f09ffb640 1 -- 172.21.15.26:0/2202520038 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8542+0+0 (secure 0 0 0) 0x7f0ef40efc50 con 0x7f0f0c102790 2024-09-11T23:26:53.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.930+0000 7f0f10f00640 1 --2- 172.21.15.26:0/2202520038 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f0ef80a2cf0 0x7f0ef80a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:53.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.930+0000 7f0f10f00640 1 --2- 172.21.15.26:0/2010668825 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556a7b593760 0x556a7b593b40 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f0f04009fc0 tx=0x7f0f04010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f215fcfdbad4e5e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:26:53.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.930+0000 7f0f027fc640 1 -- 172.21.15.26:0/2010668825 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f04021070 con 0x556a7b593760 2024-09-11T23:26:53.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.930+0000 7f0f027fc640 1 -- 172.21.15.26:0/2010668825 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f04013b60 con 0x556a7b593760 2024-09-11T23:26:53.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.930+0000 7f0f027fc640 1 -- 172.21.15.26:0/2010668825 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f0401b6b0 con 0x556a7b593760 2024-09-11T23:26:53.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.931+0000 7f0f14d3c740 1 -- 172.21.15.26:0/2010668825 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556a7b593760 msgr2=0x556a7b593b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:26:53.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.931+0000 7f0f14d3c740 1 --2- 172.21.15.26:0/2010668825 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556a7b593760 0x556a7b593b40 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f0f04009fc0 tx=0x7f0f04010800 comp rx=0 tx=0).stop 2024-09-11T23:26:53.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.931+0000 7f0f10f00640 1 --2- 172.21.15.26:0/2202520038 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f0ef80a2cf0 0x7f0ef80a51b0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f0f04010fd0 tx=0x7f0f0401bde0 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:26:53.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.931+0000 7f0f14d3c740 1 -- 172.21.15.26:0/2010668825 shutdown_connections 2024-09-11T23:26:53.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.931+0000 7f0f14d3c740 1 --2- 172.21.15.26:0/2010668825 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556a7b594080 0x556a7b59e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:53.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.931+0000 7f0f14d3c740 1 --2- 172.21.15.26:0/2010668825 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556a7b59e940 0x556a7b5a0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:53.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.931+0000 7f0f14d3c740 1 --2- 172.21.15.26:0/2010668825 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556a7b593760 0x556a7b593b40 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:53.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.931+0000 7f0f14d3c740 1 -- 172.21.15.26:0/2010668825 >> 172.21.15.26:0/2010668825 conn(0x556a7b58fcf0 msgr2=0x556a7b5900d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:26:53.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.931+0000 7f0f14d3c740 1 -- 172.21.15.26:0/2010668825 shutdown_connections 2024-09-11T23:26:53.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.931+0000 7f0f14d3c740 1 -- 172.21.15.26:0/2010668825 wait complete. 2024-09-11T23:26:53.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.932+0000 7f0f14d3c740 1 -- start start 2024-09-11T23:26:53.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.932+0000 7f0f14d3c740 1 Processor -- start 2024-09-11T23:26:53.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.932+0000 7f0f14d3c740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556a7b593760 0x556a7b59b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:53.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.932+0000 7f0f14d3c740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556a7b594080 0x556a7b59bd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:53.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.932+0000 7f0f14d3c740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556a7b59e940 0x556a7b59c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:53.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.932+0000 7f0f14d3c740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x556a7b5a1510 con 0x556a7b594080 2024-09-11T23:26:53.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.932+0000 7f0f14d3c740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x556a7b4816e0 con 0x556a7b59e940 2024-09-11T23:26:53.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.932+0000 7f0f14d3c740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x556a7b5a1690 con 0x556a7b593760 2024-09-11T23:26:53.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.932+0000 7f0f14d3c740 10 client.0 _ll_register_callbacks cb 0x556a7b4a5a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:26:53.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.932+0000 7f0f03fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556a7b594080 0x556a7b59bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:53.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.932+0000 7f0f03fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556a7b594080 0x556a7b59bd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56338/0 (socket says 172.21.15.26:56338) 2024-09-11T23:26:53.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.932+0000 7f0f03fff640 1 -- 172.21.15.26:0/2694047805 learned_addr learned my addr 172.21.15.26:0/2694047805 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:26:53.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.932+0000 7f0f0bfff640 1 --2- 172.21.15.26:0/2694047805 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556a7b593760 0x556a7b59b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:53.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.933+0000 7f0f10f00640 1 --2- 172.21.15.26:0/2694047805 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556a7b59e940 0x556a7b59c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:53.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.933+0000 7f0f03fff640 1 -- 172.21.15.26:0/2694047805 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556a7b593760 msgr2=0x556a7b59b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:26:53.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.933+0000 7f0f03fff640 1 --2- 172.21.15.26:0/2694047805 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556a7b593760 0x556a7b59b800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:53.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.933+0000 7f0f03fff640 1 -- 172.21.15.26:0/2694047805 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556a7b59e940 msgr2=0x556a7b59c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:26:53.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.933+0000 7f0f03fff640 1 --2- 172.21.15.26:0/2694047805 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556a7b59e940 0x556a7b59c280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:53.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.933+0000 7f0f03fff640 1 -- 172.21.15.26:0/2694047805 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a7b5aa9d0 con 0x556a7b594080 2024-09-11T23:26:53.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.933+0000 7f0f10f00640 1 --2- 172.21.15.26:0/2694047805 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556a7b59e940 0x556a7b59c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:26:53.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.933+0000 7f0f03fff640 1 --2- 172.21.15.26:0/2694047805 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556a7b594080 0x556a7b59bd40 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f0ef40ed3f0 tx=0x7f0ef4034000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=457c5f183a9e12cf server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:26:53.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.933+0000 7f0eeb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:26:53.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.933+0000 7f0eeb7fe640 1 -- 172.21.15.26:0/2694047805 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ef4049070 con 0x556a7b594080 2024-09-11T23:26:53.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.933+0000 7f0eeb7fe640 1 -- 172.21.15.26:0/2694047805 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8542+0+0 (secure 0 0 0) 0x7f0ef40c01d0 con 0x556a7b594080 2024-09-11T23:26:53.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.933+0000 7f0f14d3c740 1 -- 172.21.15.26:0/2694047805 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0f04013b60 con 0x556a7b594080 2024-09-11T23:26:53.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.933+0000 7f0eeb7fe640 1 -- 172.21.15.26:0/2694047805 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f0ef40d5050 con 0x556a7b594080 2024-09-11T23:26:53.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.933+0000 7f0f14d3c740 10 client.16761 fetch_fsmap learned FSMap version 49 2024-09-11T23:26:53.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.933+0000 7f0f14d3c740 1 -- 172.21.15.26:0/2694047805 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x556a7b5ad360 con 0x556a7b594080 2024-09-11T23:26:53.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.933+0000 7f0eeb7fe640 1 -- 172.21.15.26:0/2694047805 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f04013b60 con 0x556a7b594080 2024-09-11T23:26:53.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.933+0000 7f0eeb7fe640 1 -- 172.21.15.26:0/2694047805 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 49) ==== 8402+0+0 (secure 0 0 0) 0x7f0ef40dc0b0 con 0x556a7b594080 2024-09-11T23:26:53.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.934+0000 7f0f14d3c740 10 client.16761 fetch_fsmap finished waiting for FSMap version 49 2024-09-11T23:26:53.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.934+0000 7f0f14d3c740 10 client.16761 resolve_mds: validated gid 24485 aka daemon mds.b 2024-09-11T23:26:53.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.934+0000 7f0f14d3c740 20 client.16761 populate_metadata read hostname 'smithi026' 2024-09-11T23:26:53.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.934+0000 7f0f14d3c740 1 --2- 172.21.15.26:0/2694047805 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x556a7b5ae7c0 0x556a7b5b0c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:53.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.934+0000 7f0f14d3c740 4 client.16761 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:26:53.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.934+0000 7f0f14d3c740 1 -- 172.21.15.26:0/2694047805 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556a7b5069d0 con 0x556a7b5ae7c0 2024-09-11T23:26:53.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.934+0000 7f0f0bfff640 1 --2- 172.21.15.26:0/2694047805 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x556a7b5ae7c0 0x556a7b5b0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:53.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.934+0000 7f0f0bfff640 1 --2- 172.21.15.26:0/2694047805 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x556a7b5ae7c0 0x556a7b5b0c20 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee5e048703822aaa server_cookie=317f7cc28bb24dcb in_seq=0 out_seq=0 2024-09-11T23:26:53.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.934+0000 7f0eeb7fe640 10 client.16761 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:26:53.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.935+0000 7f0eeb7fe640 1 -- 172.21.15.26:0/2694047805 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556a7b5069d0 con 0x556a7b5ae7c0 2024-09-11T23:26:53.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.935+0000 7f0eeb7fe640 10 client.16761 handle_command_reply: tid=0 2024-09-11T23:26:53.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.935+0000 7f0f14d3c740 1 client.16761 shutdown 2024-09-11T23:26:53.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.935+0000 7f0f14d3c740 1 -- 172.21.15.26:0/2694047805 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556a7b594080 msgr2=0x556a7b59bd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:26:53.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.935+0000 7f0f14d3c740 1 --2- 172.21.15.26:0/2694047805 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556a7b594080 0x556a7b59bd40 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f0ef40ed3f0 tx=0x7f0ef4034000 comp rx=0 tx=0).stop 2024-09-11T23:26:53.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.935+0000 7f0f14d3c740 1 -- 172.21.15.26:0/2694047805 shutdown_connections 2024-09-11T23:26:53.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.935+0000 7f0f14d3c740 1 --2- 172.21.15.26:0/2694047805 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556a7b59e940 0x556a7b59c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:53.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.935+0000 7f0f14d3c740 1 --2- 172.21.15.26:0/2694047805 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x556a7b5ae7c0 0x556a7b5b0c20 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:53.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.935+0000 7f0f14d3c740 1 --2- 172.21.15.26:0/2694047805 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556a7b594080 0x556a7b59bd40 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:53.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.935+0000 7f0eeb7fe640 0 client.16761 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:26:53.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.935+0000 7f0f14d3c740 1 --2- 172.21.15.26:0/2694047805 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556a7b593760 0x556a7b59b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:53.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.935+0000 7f0f14d3c740 1 -- 172.21.15.26:0/2694047805 >> 172.21.15.26:0/2694047805 conn(0x556a7b58fcf0 msgr2=0x556a7b58ee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:26:53.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.936+0000 7f0f14d3c740 1 -- 172.21.15.26:0/2694047805 shutdown_connections 2024-09-11T23:26:53.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.936+0000 7f0f14d3c740 1 -- 172.21.15.26:0/2694047805 wait complete. 2024-09-11T23:26:53.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.936+0000 7f0f14d3c740 20 client.16761 trim_cache size 0 max 16384 2024-09-11T23:26:53.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.953+0000 7f0f14d3c740 1 Processor -- start 2024-09-11T23:26:53.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.953+0000 7f0f14d3c740 1 -- start start 2024-09-11T23:26:53.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.953+0000 7f0f14d3c740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556a7b593760 0x556a7b596210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:53.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.953+0000 7f0f14d3c740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556a7b594080 0x556a7b5b64f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:53.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.953+0000 7f0f14d3c740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556a7b59e940 0x556a7b5b6a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:53.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.953+0000 7f0f14d3c740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f0f0c1ad870 con 0x556a7b59e940 2024-09-11T23:26:53.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.953+0000 7f0f14d3c740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f0f0c1af590 con 0x556a7b593760 2024-09-11T23:26:53.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.953+0000 7f0f14d3c740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f0f0c1adfa0 con 0x556a7b594080 2024-09-11T23:26:53.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.953+0000 7f0f10f00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556a7b59e940 0x556a7b5b6a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:53.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.953+0000 7f0f10f00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556a7b59e940 0x556a7b5b6a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56350/0 (socket says 172.21.15.26:56350) 2024-09-11T23:26:53.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.953+0000 7f0f10f00640 1 -- 172.21.15.26:0/894662718 learned_addr learned my addr 172.21.15.26:0/894662718 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:26:53.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.953+0000 7f0f0bfff640 1 --2- 172.21.15.26:0/894662718 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556a7b593760 0x556a7b596210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:53.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.953+0000 7f0f03fff640 1 --2- 172.21.15.26:0/894662718 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556a7b594080 0x556a7b5b64f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:53.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.953+0000 7f0f10f00640 1 -- 172.21.15.26:0/894662718 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556a7b594080 msgr2=0x556a7b5b64f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:26:53.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.953+0000 7f0f10f00640 1 --2- 172.21.15.26:0/894662718 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556a7b594080 0x556a7b5b64f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:53.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.953+0000 7f0f10f00640 1 -- 172.21.15.26:0/894662718 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556a7b593760 msgr2=0x556a7b596210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:26:53.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.953+0000 7f0f10f00640 1 --2- 172.21.15.26:0/894662718 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556a7b593760 0x556a7b596210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:53.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.953+0000 7f0f10f00640 1 -- 172.21.15.26:0/894662718 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a7b58b260 con 0x556a7b59e940 2024-09-11T23:26:53.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.953+0000 7f0f0bfff640 1 --2- 172.21.15.26:0/894662718 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556a7b593760 0x556a7b596210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:26:53.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.953+0000 7f0f10f00640 1 --2- 172.21.15.26:0/894662718 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556a7b59e940 0x556a7b5b6a30 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f0f0401c010 tx=0x7f0f04025000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d2f9d344d35ae44 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:26:53.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.954+0000 7f0f01ffb640 1 -- 172.21.15.26:0/894662718 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f04021040 con 0x556a7b59e940 2024-09-11T23:26:53.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.954+0000 7f0f01ffb640 1 -- 172.21.15.26:0/894662718 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f04013240 con 0x556a7b59e940 2024-09-11T23:26:53.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.954+0000 7f0f01ffb640 1 -- 172.21.15.26:0/894662718 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f040138a0 con 0x556a7b59e940 2024-09-11T23:26:53.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.954+0000 7f0f14d3c740 1 -- 172.21.15.26:0/894662718 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556a7b59e940 msgr2=0x556a7b5b6a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:26:53.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.954+0000 7f0f14d3c740 1 --2- 172.21.15.26:0/894662718 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556a7b59e940 0x556a7b5b6a30 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f0f0401c010 tx=0x7f0f04025000 comp rx=0 tx=0).stop 2024-09-11T23:26:53.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.954+0000 7f0f14d3c740 1 -- 172.21.15.26:0/894662718 shutdown_connections 2024-09-11T23:26:53.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.954+0000 7f0f14d3c740 1 --2- 172.21.15.26:0/894662718 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556a7b59e940 0x556a7b5b6a30 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:53.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.954+0000 7f0f14d3c740 1 --2- 172.21.15.26:0/894662718 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556a7b594080 0x556a7b5b64f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:53.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.954+0000 7f0f14d3c740 1 --2- 172.21.15.26:0/894662718 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556a7b593760 0x556a7b596210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:53.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.954+0000 7f0f14d3c740 1 -- 172.21.15.26:0/894662718 >> 172.21.15.26:0/894662718 conn(0x556a7b58fcf0 msgr2=0x556a7b59d880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:26:53.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.954+0000 7f0f14d3c740 1 -- 172.21.15.26:0/894662718 shutdown_connections 2024-09-11T23:26:53.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.954+0000 7f0f14d3c740 1 -- 172.21.15.26:0/894662718 wait complete. 2024-09-11T23:26:53.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.955+0000 7f0f14d3c740 1 -- start start 2024-09-11T23:26:53.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.955+0000 7f0f14d3c740 1 Processor -- start 2024-09-11T23:26:53.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.955+0000 7f0f14d3c740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556a7b593760 0x556a7b5a68e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:53.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.955+0000 7f0f14d3c740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556a7b594080 0x556a7b5a6e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:53.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.955+0000 7f0f14d3c740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556a7b59e940 0x556a7b5a38b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:53.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.955+0000 7f0f14d3c740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f0f0c1aeec0 con 0x556a7b593760 2024-09-11T23:26:53.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.955+0000 7f0f14d3c740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f0f0c1ae730 con 0x556a7b59e940 2024-09-11T23:26:53.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.955+0000 7f0f14d3c740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f0f0c1ab870 con 0x556a7b594080 2024-09-11T23:26:53.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.955+0000 7f0f14d3c740 10 client.0 _ll_register_callbacks cb 0x7f0ef4005b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:26:53.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.955+0000 7f0f0bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556a7b593760 0x556a7b5a68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:53.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.955+0000 7f0f0bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556a7b593760 0x556a7b5a68e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56358/0 (socket says 172.21.15.26:56358) 2024-09-11T23:26:53.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.955+0000 7f0f0bfff640 1 -- 172.21.15.26:0/1329266894 learned_addr learned my addr 172.21.15.26:0/1329266894 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:26:53.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.956+0000 7f0f0bfff640 1 -- 172.21.15.26:0/1329266894 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556a7b594080 msgr2=0x556a7b5a6e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:26:53.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.956+0000 7f0f0bfff640 1 --2- 172.21.15.26:0/1329266894 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556a7b594080 0x556a7b5a6e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:53.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.956+0000 7f0f0bfff640 1 -- 172.21.15.26:0/1329266894 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556a7b59e940 msgr2=0x556a7b5a38b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:26:53.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.956+0000 7f0f0bfff640 1 --2- 172.21.15.26:0/1329266894 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556a7b59e940 0x556a7b5a38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:53.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.956+0000 7f0f0bfff640 1 -- 172.21.15.26:0/1329266894 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a7b5a3df0 con 0x556a7b593760 2024-09-11T23:26:53.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.956+0000 7f0f0bfff640 1 --2- 172.21.15.26:0/1329266894 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556a7b593760 0x556a7b5a68e0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f0efc004650 tx=0x7f0efc0039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6cf87908a5ff092 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:26:53.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.956+0000 7f0eeb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:26:53.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.956+0000 7f0eeb7fe640 1 -- 172.21.15.26:0/1329266894 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0efc011640 con 0x556a7b593760 2024-09-11T23:26:53.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.956+0000 7f0f14d3c740 1 -- 172.21.15.26:0/1329266894 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0f04013240 con 0x556a7b593760 2024-09-11T23:26:53.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.956+0000 7f0eeb7fe640 1 -- 172.21.15.26:0/1329266894 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8542+0+0 (secure 0 0 0) 0x7f0efc0117e0 con 0x556a7b593760 2024-09-11T23:26:53.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.956+0000 7f0eeb7fe640 1 -- 172.21.15.26:0/1329266894 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f0efc0119e0 con 0x556a7b593760 2024-09-11T23:26:53.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.956+0000 7f0f14d3c740 10 client.16767 fetch_fsmap learned FSMap version 49 2024-09-11T23:26:53.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.956+0000 7f0f14d3c740 1 -- 172.21.15.26:0/1329266894 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x556a7b619c90 con 0x556a7b593760 2024-09-11T23:26:53.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.956+0000 7f0eeb7fe640 1 -- 172.21.15.26:0/1329266894 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f04013240 con 0x556a7b593760 2024-09-11T23:26:53.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.957+0000 7f0eeb7fe640 1 -- 172.21.15.26:0/1329266894 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 49) ==== 8402+0+0 (secure 0 0 0) 0x7f0efc02ec20 con 0x556a7b593760 2024-09-11T23:26:53.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.957+0000 7f0f14d3c740 10 client.16767 fetch_fsmap finished waiting for FSMap version 49 2024-09-11T23:26:53.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.957+0000 7f0f14d3c740 10 client.16767 resolve_mds: validated gid 24485 aka daemon mds.b 2024-09-11T23:26:53.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.957+0000 7f0f14d3c740 20 client.16767 populate_metadata read hostname 'smithi026' 2024-09-11T23:26:53.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.957+0000 7f0f14d3c740 1 --2- 172.21.15.26:0/1329266894 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x556a7b61a4d0 0x556a7b61c970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:53.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.957+0000 7f0f14d3c740 4 client.16767 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 139.4, "await_for": 45.0}] 2024-09-11T23:26:53.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.957+0000 7f0f14d3c740 1 -- 172.21.15.26:0/1329266894 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 139.4, "await_for": 45.0}) -- 0x556a7b500d20 con 0x556a7b61a4d0 2024-09-11T23:26:53.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.957+0000 7f0f03fff640 1 --2- 172.21.15.26:0/1329266894 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x556a7b61a4d0 0x556a7b61c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:53.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.957+0000 7f0f03fff640 1 --2- 172.21.15.26:0/1329266894 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x556a7b61a4d0 0x556a7b61c970 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=26691201de96a3a1 server_cookie=20b4065d934d196 in_seq=0 out_seq=0 2024-09-11T23:26:53.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:26:53.957+0000 7f0eeb7fe640 10 client.16767 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:26:54.052 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:26:53 smithi076 ceph-mon[35296]: pgmap v262: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 967 KiB/s rd, 967 KiB/s wr, 17 op/s 2024-09-11T23:26:54.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:26:53 smithi136 ceph-mon[35121]: pgmap v262: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 967 KiB/s rd, 967 KiB/s wr, 17 op/s 2024-09-11T23:26:56.052 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:26:55 smithi076 ceph-mon[35296]: pgmap v263: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1008 KiB/s rd, 1010 KiB/s wr, 24 op/s 2024-09-11T23:26:56.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:26:55 smithi136 ceph-mon[35121]: pgmap v263: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1008 KiB/s rd, 1010 KiB/s wr, 24 op/s 2024-09-11T23:26:56.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:26:55 smithi026 ceph-mon[31172]: pgmap v263: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1008 KiB/s rd, 1010 KiB/s wr, 24 op/s 2024-09-11T23:26:58.052 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:26:57 smithi076 ceph-mon[35296]: pgmap v264: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 72 KiB/s wr, 18 op/s 2024-09-11T23:26:58.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:26:57 smithi076 ceph-mon[35296]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:26:58.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:26:57 smithi136 ceph-mon[35121]: pgmap v264: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 72 KiB/s wr, 18 op/s 2024-09-11T23:26:58.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:26:57 smithi136 ceph-mon[35121]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:26:58.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:26:57 smithi026 ceph-mon[31172]: pgmap v264: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 72 KiB/s wr, 18 op/s 2024-09-11T23:26:58.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:26:57 smithi026 ceph-mon[31172]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:27:00.052 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:26:59 smithi076 ceph-mon[35296]: pgmap v265: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 49 KiB/s wr, 16 op/s 2024-09-11T23:27:00.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:26:59 smithi136 ceph-mon[35121]: pgmap v265: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 49 KiB/s wr, 16 op/s 2024-09-11T23:27:00.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:26:59 smithi026 ceph-mon[31172]: pgmap v265: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 49 KiB/s wr, 16 op/s 2024-09-11T23:27:02.052 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:27:01 smithi076 ceph-mon[35296]: pgmap v266: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 56 KiB/s wr, 23 op/s 2024-09-11T23:27:02.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:02.084+0000 7f0eeb7fe640 1 -- 172.21.15.26:0/1329266894 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x556a7b500d20 con 0x556a7b61a4d0 2024-09-11T23:27:02.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:02.084+0000 7f0eeb7fe640 10 client.16767 handle_command_reply: tid=0 2024-09-11T23:27:02.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:02.084+0000 7f0f14d3c740 1 client.16767 shutdown 2024-09-11T23:27:02.085 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:27:01 smithi026 ceph-mon[31172]: pgmap v266: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 56 KiB/s wr, 23 op/s 2024-09-11T23:27:02.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:02.084+0000 7f0f14d3c740 1 -- 172.21.15.26:0/1329266894 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556a7b593760 msgr2=0x556a7b5a68e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:02.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:02.084+0000 7f0f14d3c740 1 --2- 172.21.15.26:0/1329266894 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556a7b593760 0x556a7b5a68e0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f0efc004650 tx=0x7f0efc0039c0 comp rx=0 tx=0).stop 2024-09-11T23:27:02.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:02.084+0000 7f0f14d3c740 1 -- 172.21.15.26:0/1329266894 shutdown_connections 2024-09-11T23:27:02.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:02.084+0000 7f0f14d3c740 1 --2- 172.21.15.26:0/1329266894 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556a7b59e940 0x556a7b5a38b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:02.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:02.084+0000 7f0f14d3c740 1 --2- 172.21.15.26:0/1329266894 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x556a7b61a4d0 0x556a7b61c970 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:02.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:02.084+0000 7f0eeb7fe640 0 client.16767 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:27:02.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:02.084+0000 7f0f14d3c740 1 --2- 172.21.15.26:0/1329266894 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556a7b594080 0x556a7b5a6e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:02.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:02.084+0000 7f0f14d3c740 1 --2- 172.21.15.26:0/1329266894 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556a7b593760 0x556a7b5a68e0 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:02.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:02.084+0000 7f0f14d3c740 1 -- 172.21.15.26:0/1329266894 >> 172.21.15.26:0/1329266894 conn(0x556a7b58fcf0 msgr2=0x556a7b58e650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:27:02.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:02.085+0000 7f0f14d3c740 1 -- 172.21.15.26:0/1329266894 shutdown_connections 2024-09-11T23:27:02.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:02.085+0000 7f0f14d3c740 1 -- 172.21.15.26:0/1329266894 wait complete. 2024-09-11T23:27:02.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:02.085+0000 7f0f14d3c740 20 client.16767 trim_cache size 0 max 16384 2024-09-11T23:27:02.087 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-11T23:27:02.088 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 51, 2024-09-11T23:27:02.088 INFO:teuthology.orchestra.run.smithi026.stdout: "leader": 24485, 2024-09-11T23:27:02.088 INFO:teuthology.orchestra.run.smithi026.stdout: "set_version": 30, 2024-09-11T23:27:02.088 INFO:teuthology.orchestra.run.smithi026.stdout: "sets": { 2024-09-11T23:27:02.088 INFO:teuthology.orchestra.run.smithi026.stdout: "442853f3": { 2024-09-11T23:27:02.088 INFO:teuthology.orchestra.run.smithi026.stdout: "version": 30, 2024-09-11T23:27:02.088 INFO:teuthology.orchestra.run.smithi026.stdout: "age_ref": 0.0, 2024-09-11T23:27:02.088 INFO:teuthology.orchestra.run.smithi026.stdout: "state": { 2024-09-11T23:27:02.088 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "QUIESCED", 2024-09-11T23:27:02.088 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.0 2024-09-11T23:27:02.088 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:27:02.088 INFO:teuthology.orchestra.run.smithi026.stdout: "timeout": 90.0, 2024-09-11T23:27:02.088 INFO:teuthology.orchestra.run.smithi026.stdout: "expiration": 139.4, 2024-09-11T23:27:02.089 INFO:teuthology.orchestra.run.smithi026.stdout: "members": { 2024-09-11T23:27:02.089 INFO:teuthology.orchestra.run.smithi026.stdout: "file:/": { 2024-09-11T23:27:02.089 INFO:teuthology.orchestra.run.smithi026.stdout: "excluded": false, 2024-09-11T23:27:02.089 INFO:teuthology.orchestra.run.smithi026.stdout: "state": { 2024-09-11T23:27:02.089 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "QUIESCED", 2024-09-11T23:27:02.089 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.0 2024-09-11T23:27:02.089 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:27:02.089 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:27:02.089 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:27:02.089 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:27:02.089 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:27:02.089 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-11T23:27:02.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:02.086+0000 7f0f12963640 1 -- 172.21.15.26:0/2202520038 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f0ef80a2cf0 msgr2=0x7f0ef80a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:02.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:02.086+0000 7f0f12963640 1 --2- 172.21.15.26:0/2202520038 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f0ef80a2cf0 0x7f0ef80a51b0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f0f04010fd0 tx=0x7f0f0401bde0 comp rx=0 tx=0).stop 2024-09-11T23:27:02.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:02.086+0000 7f0f12963640 1 -- 172.21.15.26:0/2202520038 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f0c102790 msgr2=0x7f0f0c198970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:02.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:02.086+0000 7f0f12963640 1 --2- 172.21.15.26:0/2202520038 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f0c102790 0x7f0f0c198970 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f0ef4004770 tx=0x7f0ef4003c10 comp rx=0 tx=0).stop 2024-09-11T23:27:02.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:02.086+0000 7f0f12963640 1 -- 172.21.15.26:0/2202520038 shutdown_connections 2024-09-11T23:27:02.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:02.086+0000 7f0f12963640 1 --2- 172.21.15.26:0/2202520038 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f0ef80a2cf0 0x7f0ef80a51b0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:02.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:02.086+0000 7f0f12963640 1 --2- 172.21.15.26:0/2202520038 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0f0c106f10 0x7f0f0c19cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:02.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:02.086+0000 7f0f12963640 1 --2- 172.21.15.26:0/2202520038 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f0c102790 0x7f0f0c198970 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:02.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:02.086+0000 7f0f12963640 1 --2- 172.21.15.26:0/2202520038 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0f0c101de0 0x7f0f0c198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:02.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:02.086+0000 7f0f12963640 1 -- 172.21.15.26:0/2202520038 >> 172.21.15.26:0/2202520038 conn(0x7f0f0c0fd8c0 msgr2=0x7f0f0c0fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:27:02.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:02.086+0000 7f0f12963640 1 -- 172.21.15.26:0/2202520038 shutdown_connections 2024-09-11T23:27:02.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:02.086+0000 7f0f12963640 1 -- 172.21.15.26:0/2202520038 wait complete. 2024-09-11T23:27:02.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:27:01 smithi136 ceph-mon[35121]: pgmap v266: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 56 KiB/s wr, 23 op/s 2024-09-11T23:27:02.738 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '442853f3', quiesce took 11.6 seconds. Will release after: 7.799999999999999 2024-09-11T23:27:02.765 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.761+0000 7ff3fc79f640 1 -- 172.21.15.26:0/2644440165 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff3f4101350 msgr2=0x7ff3f410d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:02.765 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.761+0000 7ff3fc79f640 1 --2- 172.21.15.26:0/2644440165 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff3f4101350 0x7ff3f410d880 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7ff3ec009cd0 tx=0x7ff3ec02a8e0 comp rx=0 tx=0).stop 2024-09-11T23:27:02.765 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.761+0000 7ff3fc79f640 1 -- 172.21.15.26:0/2644440165 shutdown_connections 2024-09-11T23:27:02.765 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.761+0000 7ff3fc79f640 1 --2- 172.21.15.26:0/2644440165 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff3f41009b0 0x7ff3f4100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:02.765 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.761+0000 7ff3fc79f640 1 --2- 172.21.15.26:0/2644440165 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff3f4101350 0x7ff3f410d880 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:02.765 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.761+0000 7ff3fc79f640 1 --2- 172.21.15.26:0/2644440165 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff3f41069c0 0x7ff3f4106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:02.765 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.761+0000 7ff3fc79f640 1 -- 172.21.15.26:0/2644440165 >> 172.21.15.26:0/2644440165 conn(0x7ff3f4077990 msgr2=0x7ff3f40fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:27:02.766 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.761+0000 7ff3fc79f640 1 -- 172.21.15.26:0/2644440165 shutdown_connections 2024-09-11T23:27:02.766 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.761+0000 7ff3fc79f640 1 -- 172.21.15.26:0/2644440165 wait complete. 2024-09-11T23:27:02.766 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.762+0000 7ff3fc79f640 1 Processor -- start 2024-09-11T23:27:02.766 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.762+0000 7ff3fc79f640 1 -- start start 2024-09-11T23:27:02.766 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.762+0000 7ff3fc79f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff3f41009b0 0x7ff3f4198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:02.766 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.762+0000 7ff3fc79f640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff3f4101350 0x7ff3f41988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:02.766 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.762+0000 7ff3fc79f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff3f41069c0 0x7ff3f419cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:02.766 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.762+0000 7ff3fc79f640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff3f4110e00 con 0x7ff3f41009b0 2024-09-11T23:27:02.766 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.762+0000 7ff3fc79f640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7ff3f4110c80 con 0x7ff3f41069c0 2024-09-11T23:27:02.766 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.762+0000 7ff3fc79f640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ff3f4110f80 con 0x7ff3f4101350 2024-09-11T23:27:02.766 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.762+0000 7ff3fa53b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff3f41009b0 0x7ff3f4198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:02.766 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.762+0000 7ff3fa53b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff3f41009b0 0x7ff3f4198370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58244/0 (socket says 172.21.15.26:58244) 2024-09-11T23:27:02.766 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.762+0000 7ff3fa53b640 1 -- 172.21.15.26:0/2934922743 learned_addr learned my addr 172.21.15.26:0/2934922743 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:27:02.767 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.763+0000 7ff3f9d3a640 1 --2- 172.21.15.26:0/2934922743 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff3f4101350 0x7ff3f41988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:02.767 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.763+0000 7ff3fad3c640 1 --2- 172.21.15.26:0/2934922743 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff3f41069c0 0x7ff3f419cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:02.767 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.763+0000 7ff3fa53b640 1 -- 172.21.15.26:0/2934922743 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff3f4101350 msgr2=0x7ff3f41988b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:02.767 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.763+0000 7ff3fa53b640 1 --2- 172.21.15.26:0/2934922743 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff3f4101350 0x7ff3f41988b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:02.767 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.763+0000 7ff3fa53b640 1 -- 172.21.15.26:0/2934922743 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff3f41069c0 msgr2=0x7ff3f419cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:02.767 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.763+0000 7ff3fa53b640 1 --2- 172.21.15.26:0/2934922743 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff3f41069c0 0x7ff3f419cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:02.767 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.763+0000 7ff3fa53b640 1 -- 172.21.15.26:0/2934922743 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3f419d360 con 0x7ff3f41009b0 2024-09-11T23:27:02.767 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.763+0000 7ff3fa53b640 1 --2- 172.21.15.26:0/2934922743 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff3f41009b0 0x7ff3f4198370 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7ff3f000e9f0 tx=0x7ff3f000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:27:02.767 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.763+0000 7ff3eb7fe640 1 -- 172.21.15.26:0/2934922743 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3f000cd80 con 0x7ff3f41009b0 2024-09-11T23:27:02.768 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.763+0000 7ff3eb7fe640 1 -- 172.21.15.26:0/2934922743 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff3f0005500 con 0x7ff3f41009b0 2024-09-11T23:27:02.768 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.763+0000 7ff3fc79f640 1 -- 172.21.15.26:0/2934922743 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3f419d650 con 0x7ff3f41009b0 2024-09-11T23:27:02.768 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.763+0000 7ff3eb7fe640 1 -- 172.21.15.26:0/2934922743 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3f0010690 con 0x7ff3f41009b0 2024-09-11T23:27:02.768 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.763+0000 7ff3fc79f640 1 -- 172.21.15.26:0/2934922743 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3f41a4ee0 con 0x7ff3f41009b0 2024-09-11T23:27:02.774 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.765+0000 7ff3eb7fe640 1 -- 172.21.15.26:0/2934922743 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7ff3f00046c0 con 0x7ff3f41009b0 2024-09-11T23:27:02.774 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.766+0000 7ff3ff617740 1 Processor -- start 2024-09-11T23:27:02.774 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.766+0000 7ff3ff617740 1 -- start start 2024-09-11T23:27:02.774 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.766+0000 7ff3ff617740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56128fa769d0 0x56128fa76db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:02.774 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.766+0000 7ff3ff617740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56128fa772f0 0x56128fa85b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:02.775 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.766+0000 7ff3ff617740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56128fa860c0 0x56128fa884b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:02.775 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.766+0000 7ff3ff617740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x56128f9db2f0 con 0x56128fa772f0 2024-09-11T23:27:02.775 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.766+0000 7ff3ff617740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x56128f983270 con 0x56128fa769d0 2024-09-11T23:27:02.775 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.766+0000 7ff3ff617740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x56128f9d72b0 con 0x56128fa860c0 2024-09-11T23:27:02.775 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.766+0000 7ff3fa53b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56128fa772f0 0x56128fa85b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:02.775 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.766+0000 7ff3fad3c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56128fa769d0 0x56128fa76db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:02.775 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.766+0000 7ff3fa53b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56128fa772f0 0x56128fa85b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58260/0 (socket says 172.21.15.26:58260) 2024-09-11T23:27:02.775 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.766+0000 7ff3fa53b640 1 -- 172.21.15.26:0/3773206249 learned_addr learned my addr 172.21.15.26:0/3773206249 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:27:02.775 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.766+0000 7ff3f9d3a640 1 --2- 172.21.15.26:0/3773206249 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56128fa860c0 0x56128fa884b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:02.775 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.767+0000 7ff3eb7fe640 1 --2- 172.21.15.26:0/2934922743 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7ff3e00a2da0 0x7ff3e00a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:02.775 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.767+0000 7ff3fad3c640 1 -- 172.21.15.26:0/3773206249 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56128fa860c0 msgr2=0x56128fa884b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:27:02.776 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.767+0000 7ff3fad3c640 1 --2- 172.21.15.26:0/3773206249 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56128fa860c0 0x56128fa884b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:02.776 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.767+0000 7ff3fad3c640 1 -- 172.21.15.26:0/3773206249 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56128fa772f0 msgr2=0x56128fa85b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:27:02.776 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.767+0000 7ff3fad3c640 1 --2- 172.21.15.26:0/3773206249 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56128fa772f0 0x56128fa85b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:02.776 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.767+0000 7ff3fad3c640 1 -- 172.21.15.26:0/3773206249 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56128f982340 con 0x56128fa769d0 2024-09-11T23:27:02.776 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.767+0000 7ff3f9d3a640 1 --2- 172.21.15.26:0/3773206249 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56128fa860c0 0x56128fa884b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:27:02.776 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.767+0000 7ff3fa53b640 1 --2- 172.21.15.26:0/3773206249 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56128fa772f0 0x56128fa85b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:27:02.776 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.767+0000 7ff3eb7fe640 1 -- 172.21.15.26:0/2934922743 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8542+0+0 (secure 0 0 0) 0x7ff3f00cf9a0 con 0x7ff3f41009b0 2024-09-11T23:27:02.776 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.767+0000 7ff3fad3c640 1 --2- 172.21.15.26:0/2934922743 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7ff3e00a2da0 0x7ff3e00a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:02.776 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.767+0000 7ff3fad3c640 1 --2- 172.21.15.26:0/3773206249 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56128fa769d0 0x56128fa76db0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ff3ec006010 tx=0x7ff3ec005f70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f28bccb7a646f97 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:27:02.776 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.767+0000 7ff3e8ff9640 1 -- 172.21.15.26:0/3773206249 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3ec0315b0 con 0x56128fa769d0 2024-09-11T23:27:02.776 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.767+0000 7ff3e8ff9640 1 -- 172.21.15.26:0/3773206249 <== mon.2 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff3ec036040 con 0x56128fa769d0 2024-09-11T23:27:02.776 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.767+0000 7ff3e8ff9640 1 -- 172.21.15.26:0/3773206249 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3ec0305a0 con 0x56128fa769d0 2024-09-11T23:27:02.776 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.767+0000 7ff3fad3c640 1 --2- 172.21.15.26:0/2934922743 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7ff3e00a2da0 0x7ff3e00a5260 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x56128fa77910 tx=0x7ff3ec048000 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:27:02.777 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.768+0000 7ff3ff617740 1 -- 172.21.15.26:0/3773206249 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56128fa769d0 msgr2=0x56128fa76db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:02.777 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.768+0000 7ff3ff617740 1 --2- 172.21.15.26:0/3773206249 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56128fa769d0 0x56128fa76db0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ff3ec006010 tx=0x7ff3ec005f70 comp rx=0 tx=0).stop 2024-09-11T23:27:02.777 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.768+0000 7ff3ff617740 1 -- 172.21.15.26:0/3773206249 shutdown_connections 2024-09-11T23:27:02.777 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.768+0000 7ff3ff617740 1 --2- 172.21.15.26:0/3773206249 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56128fa860c0 0x56128fa884b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:02.777 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.768+0000 7ff3ff617740 1 --2- 172.21.15.26:0/3773206249 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56128fa772f0 0x56128fa85b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:02.777 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.768+0000 7ff3ff617740 1 --2- 172.21.15.26:0/3773206249 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56128fa769d0 0x56128fa76db0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:02.777 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.768+0000 7ff3ff617740 1 -- 172.21.15.26:0/3773206249 >> 172.21.15.26:0/3773206249 conn(0x56128fa73060 msgr2=0x56128fa73440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:27:02.777 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.768+0000 7ff3ff617740 1 -- 172.21.15.26:0/3773206249 shutdown_connections 2024-09-11T23:27:02.777 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.768+0000 7ff3ff617740 1 -- 172.21.15.26:0/3773206249 wait complete. 2024-09-11T23:27:02.777 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.768+0000 7ff3ff617740 1 -- start start 2024-09-11T23:27:02.777 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.768+0000 7ff3ff617740 1 Processor -- start 2024-09-11T23:27:02.777 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.769+0000 7ff3ff617740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56128fa769d0 0x56128fa83050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:02.777 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.769+0000 7ff3ff617740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56128fa772f0 0x56128fa83590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:02.778 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.769+0000 7ff3ff617740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56128fa860c0 0x56128fa83ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:02.778 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.769+0000 7ff3ff617740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x56128fa88cf0 con 0x56128fa772f0 2024-09-11T23:27:02.778 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.769+0000 7ff3ff617740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x56128f964290 con 0x56128fa769d0 2024-09-11T23:27:02.778 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.769+0000 7ff3ff617740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x56128fa88e70 con 0x56128fa860c0 2024-09-11T23:27:02.778 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.769+0000 7ff3ff617740 10 client.0 _ll_register_callbacks cb 0x56128f995140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:27:02.778 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.769+0000 7ff3f9d3a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56128fa772f0 0x56128fa83590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:02.778 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.769+0000 7ff3fa53b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56128fa769d0 0x56128fa83050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:02.778 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.769+0000 7ff3f9d3a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56128fa772f0 0x56128fa83590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58262/0 (socket says 172.21.15.26:58262) 2024-09-11T23:27:02.778 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.769+0000 7ff3f9d3a640 1 -- 172.21.15.26:0/2994177646 learned_addr learned my addr 172.21.15.26:0/2994177646 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:27:02.778 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.769+0000 7ff3fad3c640 1 --2- 172.21.15.26:0/2994177646 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56128fa860c0 0x56128fa83ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:02.778 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.769+0000 7ff3f9d3a640 1 -- 172.21.15.26:0/2994177646 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56128fa860c0 msgr2=0x56128fa83ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:27:02.778 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.769+0000 7ff3f9d3a640 1 --2- 172.21.15.26:0/2994177646 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56128fa860c0 0x56128fa83ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:02.778 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.769+0000 7ff3f9d3a640 1 -- 172.21.15.26:0/2994177646 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56128fa769d0 msgr2=0x56128fa83050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:27:02.778 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.769+0000 7ff3f9d3a640 1 --2- 172.21.15.26:0/2994177646 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56128fa769d0 0x56128fa83050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:02.779 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.769+0000 7ff3f9d3a640 1 -- 172.21.15.26:0/2994177646 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56128f6ee210 con 0x56128fa772f0 2024-09-11T23:27:02.779 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.769+0000 7ff3fa53b640 1 --2- 172.21.15.26:0/2994177646 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56128fa769d0 0x56128fa83050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:27:02.779 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.769+0000 7ff3fad3c640 1 --2- 172.21.15.26:0/2994177646 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56128fa860c0 0x56128fa83ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:27:02.779 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.769+0000 7ff3f9d3a640 1 --2- 172.21.15.26:0/2994177646 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56128fa772f0 0x56128fa83590 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7ff3e4013790 tx=0x7ff3e4013c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8659575e8de84b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:27:02.779 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.769+0000 7ff3ddffb640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:27:02.779 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.770+0000 7ff3ddffb640 1 -- 172.21.15.26:0/2994177646 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3e4020060 con 0x56128fa772f0 2024-09-11T23:27:02.779 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.770+0000 7ff3ddffb640 1 -- 172.21.15.26:0/2994177646 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8542+0+0 (secure 0 0 0) 0x7ff3e4011dd0 con 0x56128fa772f0 2024-09-11T23:27:02.779 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.770+0000 7ff3ff617740 1 -- 172.21.15.26:0/2994177646 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff3ec036040 con 0x56128fa772f0 2024-09-11T23:27:02.779 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.770+0000 7ff3ddffb640 1 -- 172.21.15.26:0/2994177646 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7ff3e400f450 con 0x56128fa772f0 2024-09-11T23:27:02.779 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.770+0000 7ff3ff617740 10 client.16779 fetch_fsmap learned FSMap version 51 2024-09-11T23:27:02.779 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.770+0000 7ff3ff617740 1 -- 172.21.15.26:0/2994177646 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x56128f7ba620 con 0x56128fa772f0 2024-09-11T23:27:02.780 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.770+0000 7ff3ddffb640 1 -- 172.21.15.26:0/2994177646 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff3ec036040 con 0x56128fa772f0 2024-09-11T23:27:02.780 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.770+0000 7ff3ddffb640 1 -- 172.21.15.26:0/2994177646 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 51) ==== 8394+0+0 (secure 0 0 0) 0x7ff3e401a030 con 0x56128fa772f0 2024-09-11T23:27:02.780 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.770+0000 7ff3ff617740 10 client.16779 fetch_fsmap finished waiting for FSMap version 51 2024-09-11T23:27:02.780 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.770+0000 7ff3ff617740 10 client.16779 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:27:02.780 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.770+0000 7ff3ff617740 20 client.16779 populate_metadata read hostname 'smithi026' 2024-09-11T23:27:02.780 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.770+0000 7ff3ff617740 1 --2- 172.21.15.26:0/2994177646 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x56128fa96150 0x56128fa98540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:02.780 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.770+0000 7ff3ff617740 4 client.16779 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:27:02.780 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.770+0000 7ff3ff617740 1 -- 172.21.15.26:0/2994177646 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56128f9e8a20 con 0x56128fa96150 2024-09-11T23:27:02.780 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.771+0000 7ff3fa53b640 1 --2- 172.21.15.26:0/2994177646 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x56128fa96150 0x56128fa98540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:02.780 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.771+0000 7ff3fa53b640 1 --2- 172.21.15.26:0/2994177646 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x56128fa96150 0x56128fa98540 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d7518dad79162a01 server_cookie=d5437d942ec8b5ac in_seq=0 out_seq=0 2024-09-11T23:27:02.780 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.771+0000 7ff3ddffb640 10 client.16779 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:27:02.780 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.772+0000 7ff3ddffb640 1 -- 172.21.15.26:0/2994177646 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56128f9e8a20 con 0x56128fa96150 2024-09-11T23:27:02.780 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.772+0000 7ff3ddffb640 10 client.16779 handle_command_reply: tid=0 2024-09-11T23:27:02.781 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.772+0000 7ff3ff617740 1 client.16779 shutdown 2024-09-11T23:27:02.781 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.772+0000 7ff3ff617740 1 -- 172.21.15.26:0/2994177646 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56128fa772f0 msgr2=0x56128fa83590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:02.781 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.772+0000 7ff3ff617740 1 --2- 172.21.15.26:0/2994177646 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56128fa772f0 0x56128fa83590 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7ff3e4013790 tx=0x7ff3e4013c50 comp rx=0 tx=0).stop 2024-09-11T23:27:02.781 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.772+0000 7ff3ff617740 1 -- 172.21.15.26:0/2994177646 shutdown_connections 2024-09-11T23:27:02.781 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.772+0000 7ff3ff617740 1 --2- 172.21.15.26:0/2994177646 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x56128fa96150 0x56128fa98540 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:02.781 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.772+0000 7ff3ff617740 1 --2- 172.21.15.26:0/2994177646 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56128fa860c0 0x56128fa83ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:02.781 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.772+0000 7ff3ff617740 1 --2- 172.21.15.26:0/2994177646 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56128fa772f0 0x56128fa83590 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:02.781 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.772+0000 7ff3ddffb640 0 client.16779 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:27:02.781 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.772+0000 7ff3ff617740 1 --2- 172.21.15.26:0/2994177646 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56128fa769d0 0x56128fa83050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:02.781 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.772+0000 7ff3ff617740 1 -- 172.21.15.26:0/2994177646 >> 172.21.15.26:0/2994177646 conn(0x56128fa73060 msgr2=0x56128fa721c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:27:02.781 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.772+0000 7ff3ff617740 1 -- 172.21.15.26:0/2994177646 shutdown_connections 2024-09-11T23:27:02.781 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.772+0000 7ff3ff617740 1 -- 172.21.15.26:0/2994177646 wait complete. 2024-09-11T23:27:02.781 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.772+0000 7ff3ff617740 20 client.16779 trim_cache size 0 max 16384 2024-09-11T23:27:02.792 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.786+0000 7ff3ff617740 1 Processor -- start 2024-09-11T23:27:02.792 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.787+0000 7ff3ff617740 1 -- start start 2024-09-11T23:27:02.792 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.787+0000 7ff3ff617740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56128fa769d0 0x56128fa84fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:02.792 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.787+0000 7ff3ff617740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56128fa772f0 0x56128fa9de30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:02.792 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.787+0000 7ff3ff617740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56128fa860c0 0x56128fa9e370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:02.793 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.787+0000 7ff3ff617740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff3f41ab680 con 0x56128fa769d0 2024-09-11T23:27:02.793 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.787+0000 7ff3ff617740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7ff3f41ad340 con 0x56128fa772f0 2024-09-11T23:27:02.793 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.787+0000 7ff3ff617740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ff3f41abd50 con 0x56128fa860c0 2024-09-11T23:27:02.793 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.787+0000 7ff3fa53b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56128fa769d0 0x56128fa84fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:02.793 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.787+0000 7ff3fa53b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56128fa769d0 0x56128fa84fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58278/0 (socket says 172.21.15.26:58278) 2024-09-11T23:27:02.793 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.787+0000 7ff3fa53b640 1 -- 172.21.15.26:0/1343806985 learned_addr learned my addr 172.21.15.26:0/1343806985 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:27:02.793 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.787+0000 7ff3f9d3a640 1 --2- 172.21.15.26:0/1343806985 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56128fa772f0 0x56128fa9de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:02.793 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.787+0000 7ff3fa53b640 1 -- 172.21.15.26:0/1343806985 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56128fa860c0 msgr2=0x56128fa9e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:27:02.793 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.787+0000 7ff3fa53b640 1 --2- 172.21.15.26:0/1343806985 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56128fa860c0 0x56128fa9e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:02.793 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.787+0000 7ff3fa53b640 1 -- 172.21.15.26:0/1343806985 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56128fa772f0 msgr2=0x56128fa9de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:27:02.793 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.787+0000 7ff3fa53b640 1 --2- 172.21.15.26:0/1343806985 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56128fa772f0 0x56128fa9de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:02.793 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.787+0000 7ff3fa53b640 1 -- 172.21.15.26:0/1343806985 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56128fa7ebf0 con 0x56128fa769d0 2024-09-11T23:27:02.793 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.787+0000 7ff3f9d3a640 1 --2- 172.21.15.26:0/1343806985 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56128fa772f0 0x56128fa9de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:27:02.794 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.787+0000 7ff3fa53b640 1 --2- 172.21.15.26:0/1343806985 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56128fa769d0 0x56128fa84fc0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7ff3f000c880 tx=0x7ff3f00bc040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1b5134807382e49 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:27:02.794 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.787+0000 7ff3dffff640 1 -- 172.21.15.26:0/1343806985 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3f009b990 con 0x56128fa769d0 2024-09-11T23:27:02.794 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.787+0000 7ff3dffff640 1 -- 172.21.15.26:0/1343806985 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff3f009e070 con 0x56128fa769d0 2024-09-11T23:27:02.794 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.787+0000 7ff3dffff640 1 -- 172.21.15.26:0/1343806985 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3f009b7b0 con 0x56128fa769d0 2024-09-11T23:27:02.794 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.788+0000 7ff3ff617740 1 -- 172.21.15.26:0/1343806985 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56128fa769d0 msgr2=0x56128fa84fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:02.794 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.788+0000 7ff3ff617740 1 --2- 172.21.15.26:0/1343806985 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56128fa769d0 0x56128fa84fc0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7ff3f000c880 tx=0x7ff3f00bc040 comp rx=0 tx=0).stop 2024-09-11T23:27:02.794 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.788+0000 7ff3ff617740 1 -- 172.21.15.26:0/1343806985 shutdown_connections 2024-09-11T23:27:02.794 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.788+0000 7ff3ff617740 1 --2- 172.21.15.26:0/1343806985 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56128fa772f0 0x56128fa9de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:02.794 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.788+0000 7ff3ff617740 1 --2- 172.21.15.26:0/1343806985 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56128fa860c0 0x56128fa9e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:02.794 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.788+0000 7ff3ff617740 1 --2- 172.21.15.26:0/1343806985 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56128fa769d0 0x56128fa84fc0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:02.795 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.788+0000 7ff3ff617740 1 -- 172.21.15.26:0/1343806985 >> 172.21.15.26:0/1343806985 conn(0x56128fa73060 msgr2=0x56128fa848b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:27:02.795 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.788+0000 7ff3ff617740 1 -- 172.21.15.26:0/1343806985 shutdown_connections 2024-09-11T23:27:02.795 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.788+0000 7ff3ff617740 1 -- 172.21.15.26:0/1343806985 wait complete. 2024-09-11T23:27:02.795 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.788+0000 7ff3ff617740 1 -- start start 2024-09-11T23:27:02.795 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.788+0000 7ff3ff617740 1 Processor -- start 2024-09-11T23:27:02.795 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.789+0000 7ff3ff617740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56128fa769d0 0x56128fa8a110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:02.796 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.789+0000 7ff3ff617740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56128fa772f0 0x56128fa8a650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:02.796 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.789+0000 7ff3ff617740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56128fa860c0 0x56128fa8abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:02.796 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.789+0000 7ff3ff617740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff3f41acc70 con 0x56128fa772f0 2024-09-11T23:27:02.796 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.789+0000 7ff3ff617740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7ff3f41ac4e0 con 0x56128fa769d0 2024-09-11T23:27:02.796 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.789+0000 7ff3ff617740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ff3f41a9680 con 0x56128fa860c0 2024-09-11T23:27:02.797 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.789+0000 7ff3ff617740 10 client.0 _ll_register_callbacks cb 0x7ff3e4009690 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:27:02.797 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.789+0000 7ff3f9d3a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56128fa772f0 0x56128fa8a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:02.797 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.789+0000 7ff3f9d3a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56128fa772f0 0x56128fa8a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58290/0 (socket says 172.21.15.26:58290) 2024-09-11T23:27:02.797 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.789+0000 7ff3f9d3a640 1 -- 172.21.15.26:0/1196583566 learned_addr learned my addr 172.21.15.26:0/1196583566 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:27:02.797 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.789+0000 7ff3fa53b640 1 --2- 172.21.15.26:0/1196583566 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56128fa769d0 0x56128fa8a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:02.797 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.789+0000 7ff3fad3c640 1 --2- 172.21.15.26:0/1196583566 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56128fa860c0 0x56128fa8abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:02.797 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.789+0000 7ff3f9d3a640 1 -- 172.21.15.26:0/1196583566 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56128fa860c0 msgr2=0x56128fa8abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:27:02.797 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.789+0000 7ff3f9d3a640 1 --2- 172.21.15.26:0/1196583566 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56128fa860c0 0x56128fa8abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:02.798 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.789+0000 7ff3f9d3a640 1 -- 172.21.15.26:0/1196583566 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56128fa769d0 msgr2=0x56128fa8a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:27:02.798 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.789+0000 7ff3f9d3a640 1 --2- 172.21.15.26:0/1196583566 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56128fa769d0 0x56128fa8a110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:02.798 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.789+0000 7ff3f9d3a640 1 -- 172.21.15.26:0/1196583566 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56128fb000f0 con 0x56128fa772f0 2024-09-11T23:27:02.798 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.789+0000 7ff3fa53b640 1 --2- 172.21.15.26:0/1196583566 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56128fa769d0 0x56128fa8a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:27:02.798 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.789+0000 7ff3f9d3a640 1 --2- 172.21.15.26:0/1196583566 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56128fa772f0 0x56128fa8a650 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7ff3e4000c00 tx=0x7ff3e4011830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebd8cbf89d0ee363 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:27:02.798 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.789+0000 7ff3ddffb640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:27:02.798 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.789+0000 7ff3ddffb640 1 -- 172.21.15.26:0/1196583566 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3e4023380 con 0x56128fa772f0 2024-09-11T23:27:02.798 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.789+0000 7ff3ddffb640 1 -- 172.21.15.26:0/1196583566 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8542+0+0 (secure 0 0 0) 0x7ff3e4023520 con 0x56128fa772f0 2024-09-11T23:27:02.798 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.789+0000 7ff3ff617740 1 -- 172.21.15.26:0/1196583566 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff3f009e070 con 0x56128fa772f0 2024-09-11T23:27:02.798 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.790+0000 7ff3ddffb640 1 -- 172.21.15.26:0/1196583566 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7ff3e4020030 con 0x56128fa772f0 2024-09-11T23:27:02.798 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.790+0000 7ff3ff617740 10 client.16785 fetch_fsmap learned FSMap version 51 2024-09-11T23:27:02.798 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.790+0000 7ff3ff617740 1 -- 172.21.15.26:0/1196583566 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x56128fb01370 con 0x56128fa772f0 2024-09-11T23:27:02.798 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.790+0000 7ff3ddffb640 1 -- 172.21.15.26:0/1196583566 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff3f009e070 con 0x56128fa772f0 2024-09-11T23:27:02.799 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.790+0000 7ff3ddffb640 1 -- 172.21.15.26:0/1196583566 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 51) ==== 8394+0+0 (secure 0 0 0) 0x7ff3e401a030 con 0x56128fa772f0 2024-09-11T23:27:02.799 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.790+0000 7ff3ff617740 10 client.16785 fetch_fsmap finished waiting for FSMap version 51 2024-09-11T23:27:02.799 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.790+0000 7ff3ff617740 10 client.16785 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:27:02.799 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.790+0000 7ff3ff617740 20 client.16785 populate_metadata read hostname 'smithi026' 2024-09-11T23:27:02.799 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.790+0000 7ff3ff617740 1 --2- 172.21.15.26:0/1196583566 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x56128fb01b70 0x56128fb03f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:02.799 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.790+0000 7ff3ff617740 4 client.16785 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T23:27:02.799 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.790+0000 7ff3ff617740 1 -- 172.21.15.26:0/1196583566 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56128f9e2d70 con 0x56128fb01b70 2024-09-11T23:27:02.799 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.790+0000 7ff3fa53b640 1 --2- 172.21.15.26:0/1196583566 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x56128fb01b70 0x56128fb03f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:02.799 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.791+0000 7ff3ddffb640 10 client.16785 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:27:02.799 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:02.791+0000 7ff3fa53b640 1 --2- 172.21.15.26:0/1196583566 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x56128fb01b70 0x56128fb03f90 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f296ebcc5a4d9996 server_cookie=1877bdcad8bc186e in_seq=0 out_seq=0 2024-09-11T23:27:03.000 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:27:02 smithi026 ceph-mon[31172]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:27:03.052 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:27:02 smithi076 ceph-mon[35296]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:27:03.240 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:27:02 smithi136 ceph-mon[35121]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:27:04.052 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:27:03 smithi076 ceph-mon[35296]: pgmap v267: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 52 KiB/s wr, 17 op/s 2024-09-11T23:27:04.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:27:03 smithi136 ceph-mon[35121]: pgmap v267: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 52 KiB/s wr, 17 op/s 2024-09-11T23:27:04.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:27:03 smithi026 ceph-mon[31172]: pgmap v267: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 52 KiB/s wr, 17 op/s 2024-09-11T23:27:06.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:27:05 smithi136 ceph-mon[35121]: pgmap v268: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 131 KiB/s wr, 40 op/s 2024-09-11T23:27:06.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:27:05 smithi026 ceph-mon[31172]: pgmap v268: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 131 KiB/s wr, 40 op/s 2024-09-11T23:27:06.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:27:05 smithi076 ceph-mon[35296]: pgmap v268: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 131 KiB/s wr, 40 op/s 2024-09-11T23:27:06.593 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:06.592+0000 7ff3ddffb640 1 -- 172.21.15.26:0/1196583566 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56128f9e2d70 con 0x56128fb01b70 2024-09-11T23:27:06.594 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:06.592+0000 7ff3ddffb640 10 client.16785 handle_command_reply: tid=0 2024-09-11T23:27:06.594 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:06.592+0000 7ff3ff617740 1 client.16785 shutdown 2024-09-11T23:27:06.594 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:06.592+0000 7ff3ff617740 1 -- 172.21.15.26:0/1196583566 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56128fa772f0 msgr2=0x56128fa8a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:06.594 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:06.592+0000 7ff3ff617740 1 --2- 172.21.15.26:0/1196583566 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56128fa772f0 0x56128fa8a650 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7ff3e4000c00 tx=0x7ff3e4011830 comp rx=0 tx=0).stop 2024-09-11T23:27:06.594 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:06.592+0000 7ff3ff617740 1 -- 172.21.15.26:0/1196583566 shutdown_connections 2024-09-11T23:27:06.594 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:06.592+0000 7ff3ff617740 1 --2- 172.21.15.26:0/1196583566 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x56128fb01b70 0x56128fb03f90 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:06.594 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:06.592+0000 7ff3ff617740 1 --2- 172.21.15.26:0/1196583566 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56128fa860c0 0x56128fa8abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:06.594 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:06.592+0000 7ff3ddffb640 0 client.16785 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:27:06.594 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:06.592+0000 7ff3ff617740 1 --2- 172.21.15.26:0/1196583566 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56128fa772f0 0x56128fa8a650 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:06.594 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:06.592+0000 7ff3ff617740 1 --2- 172.21.15.26:0/1196583566 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56128fa769d0 0x56128fa8a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:06.594 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:06.592+0000 7ff3ff617740 1 -- 172.21.15.26:0/1196583566 >> 172.21.15.26:0/1196583566 conn(0x56128fa73060 msgr2=0x56128fa719a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:27:06.594 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:06.592+0000 7ff3ff617740 1 -- 172.21.15.26:0/1196583566 shutdown_connections 2024-09-11T23:27:06.595 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:06.592+0000 7ff3ff617740 1 -- 172.21.15.26:0/1196583566 wait complete. 2024-09-11T23:27:06.595 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:06.592+0000 7ff3ff617740 20 client.16785 trim_cache size 0 max 16384 2024-09-11T23:27:06.595 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-09-11T23:27:06.595 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "message": "", 2024-09-11T23:27:06.595 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-09-11T23:27:06.595 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-09-11T23:27:06.596 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:06.593+0000 7ff3fc79f640 1 -- 172.21.15.26:0/2934922743 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7ff3e00a2da0 msgr2=0x7ff3e00a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:06.596 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:06.593+0000 7ff3fc79f640 1 --2- 172.21.15.26:0/2934922743 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7ff3e00a2da0 0x7ff3e00a5260 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x56128fa77910 tx=0x7ff3ec048000 comp rx=0 tx=0).stop 2024-09-11T23:27:06.596 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:06.593+0000 7ff3fc79f640 1 -- 172.21.15.26:0/2934922743 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff3f41009b0 msgr2=0x7ff3f4198370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:06.596 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:06.593+0000 7ff3fc79f640 1 --2- 172.21.15.26:0/2934922743 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff3f41009b0 0x7ff3f4198370 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7ff3f000e9f0 tx=0x7ff3f000edf0 comp rx=0 tx=0).stop 2024-09-11T23:27:06.596 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:06.593+0000 7ff3fc79f640 1 -- 172.21.15.26:0/2934922743 shutdown_connections 2024-09-11T23:27:06.596 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:06.593+0000 7ff3fc79f640 1 --2- 172.21.15.26:0/2934922743 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7ff3e00a2da0 0x7ff3e00a5260 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:06.596 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:06.593+0000 7ff3fc79f640 1 --2- 172.21.15.26:0/2934922743 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff3f41069c0 0x7ff3f419cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:06.596 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:06.593+0000 7ff3fc79f640 1 --2- 172.21.15.26:0/2934922743 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff3f4101350 0x7ff3f41988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:06.597 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:06.593+0000 7ff3fc79f640 1 --2- 172.21.15.26:0/2934922743 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff3f41009b0 0x7ff3f4198370 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:06.597 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:06.593+0000 7ff3fc79f640 1 -- 172.21.15.26:0/2934922743 >> 172.21.15.26:0/2934922743 conn(0x7ff3f4077990 msgr2=0x7ff3f4069af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:27:06.597 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:06.593+0000 7ff3fc79f640 1 -- 172.21.15.26:0/2934922743 shutdown_connections 2024-09-11T23:27:06.597 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:06.593+0000 7ff3fc79f640 1 -- 172.21.15.26:0/2934922743 wait complete. 2024-09-11T23:27:08.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:27:07 smithi136 ceph-mon[35121]: pgmap v269: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 87 KiB/s wr, 34 op/s 2024-09-11T23:27:08.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:27:07 smithi136 ceph-mon[35121]: osdmap e91: 12 total, 12 up, 12 in 2024-09-11T23:27:08.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:27:07 smithi026 ceph-mon[31172]: pgmap v269: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 87 KiB/s wr, 34 op/s 2024-09-11T23:27:08.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:27:07 smithi026 ceph-mon[31172]: osdmap e91: 12 total, 12 up, 12 in 2024-09-11T23:27:08.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:27:07 smithi076 ceph-mon[35296]: pgmap v269: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 87 KiB/s wr, 34 op/s 2024-09-11T23:27:08.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:27:07 smithi076 ceph-mon[35296]: osdmap e91: 12 total, 12 up, 12 in 2024-09-11T23:27:10.238 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:27:09 smithi136 ceph-mon[35121]: pgmap v271: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 104 KiB/s wr, 39 op/s 2024-09-11T23:27:10.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:27:09 smithi026 ceph-mon[31172]: pgmap v271: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 104 KiB/s wr, 39 op/s 2024-09-11T23:27:10.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:27:09 smithi076 ceph-mon[35296]: pgmap v271: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 104 KiB/s wr, 39 op/s 2024-09-11T23:27:10.540 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:27:10.805 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:27:11.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:11.309+0000 7fc7df296640 1 -- 172.21.15.26:0/84691035 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc7d81027a0 msgr2=0x7fc7d81067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:11.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:11.309+0000 7fc7df296640 1 --2- 172.21.15.26:0/84691035 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc7d81027a0 0x7fc7d81067f0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fc7d4009900 tx=0x7fc7d402a410 comp rx=0 tx=0).stop 2024-09-11T23:27:11.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:11.309+0000 7fc7df296640 1 -- 172.21.15.26:0/84691035 shutdown_connections 2024-09-11T23:27:11.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:11.309+0000 7fc7df296640 1 --2- 172.21.15.26:0/84691035 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc7d8106f20 0x7fc7d810eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:11.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:11.309+0000 7fc7df296640 1 --2- 172.21.15.26:0/84691035 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc7d81027a0 0x7fc7d81067f0 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:11.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:11.309+0000 7fc7df296640 1 --2- 172.21.15.26:0/84691035 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc7d8101df0 0x7fc7d81021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:11.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:11.309+0000 7fc7df296640 1 -- 172.21.15.26:0/84691035 >> 172.21.15.26:0/84691035 conn(0x7fc7d80fd8f0 msgr2=0x7fc7d80ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:27:11.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:11.309+0000 7fc7df296640 1 -- 172.21.15.26:0/84691035 shutdown_connections 2024-09-11T23:27:11.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:11.309+0000 7fc7df296640 1 -- 172.21.15.26:0/84691035 wait complete. 2024-09-11T23:27:11.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:11.310+0000 7fc7df296640 1 Processor -- start 2024-09-11T23:27:11.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:11.310+0000 7fc7df296640 1 -- start start 2024-09-11T23:27:11.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:11.310+0000 7fc7df296640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc7d8101df0 0x7fc7d8198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:11.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:11.310+0000 7fc7df296640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc7d81027a0 0x7fc7d8198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:11.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:11.310+0000 7fc7df296640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc7d8106f20 0x7fc7d819ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:11.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:11.310+0000 7fc7df296640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc7d8111f40 con 0x7fc7d81027a0 2024-09-11T23:27:11.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:11.310+0000 7fc7df296640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fc7d8111dc0 con 0x7fc7d8101df0 2024-09-11T23:27:11.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:11.310+0000 7fc7df296640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fc7d81120c0 con 0x7fc7d8106f20 2024-09-11T23:27:11.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:11.310+0000 7fc7dc831640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc7d81027a0 0x7fc7d8198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:11.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:11.310+0000 7fc7dc831640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc7d81027a0 0x7fc7d8198960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36354/0 (socket says 172.21.15.26:36354) 2024-09-11T23:27:11.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:11.310+0000 7fc7dc831640 1 -- 172.21.15.26:0/2217492724 learned_addr learned my addr 172.21.15.26:0/2217492724 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:27:11.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:11.311+0000 7fc7dd032640 1 --2- 172.21.15.26:0/2217492724 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc7d8101df0 0x7fc7d8198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:11.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:11.311+0000 7fc7dd833640 1 --2- 172.21.15.26:0/2217492724 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc7d8106f20 0x7fc7d819ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:11.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:11.311+0000 7fc7dc831640 1 -- 172.21.15.26:0/2217492724 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc7d8106f20 msgr2=0x7fc7d819ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:11.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:11.311+0000 7fc7dc831640 1 --2- 172.21.15.26:0/2217492724 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc7d8106f20 0x7fc7d819ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:11.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:11.311+0000 7fc7dc831640 1 -- 172.21.15.26:0/2217492724 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc7d8101df0 msgr2=0x7fc7d8198420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:11.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:11.311+0000 7fc7dc831640 1 --2- 172.21.15.26:0/2217492724 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc7d8101df0 0x7fc7d8198420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:11.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:11.311+0000 7fc7dc831640 1 -- 172.21.15.26:0/2217492724 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7d819d3d0 con 0x7fc7d81027a0 2024-09-11T23:27:11.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:11.311+0000 7fc7dc831640 1 --2- 172.21.15.26:0/2217492724 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc7d81027a0 0x7fc7d8198960 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fc7d4006010 tx=0x7fc7d40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:27:11.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:11.311+0000 7fc7ce7fc640 1 -- 172.21.15.26:0/2217492724 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7d403f030 con 0x7fc7d81027a0 2024-09-11T23:27:11.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:11.311+0000 7fc7df296640 1 -- 172.21.15.26:0/2217492724 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7d819d660 con 0x7fc7d81027a0 2024-09-11T23:27:11.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:11.311+0000 7fc7ce7fc640 1 -- 172.21.15.26:0/2217492724 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc7d4003ce0 con 0x7fc7d81027a0 2024-09-11T23:27:11.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:11.311+0000 7fc7df296640 1 -- 172.21.15.26:0/2217492724 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7d81a4fa0 con 0x7fc7d81027a0 2024-09-11T23:27:11.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:11.312+0000 7fc7ce7fc640 1 -- 172.21.15.26:0/2217492724 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7d4049060 con 0x7fc7d81027a0 2024-09-11T23:27:11.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:11.312+0000 7fc7df296640 1 -- 172.21.15.26:0/2217492724 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7d81021d0 con 0x7fc7d81027a0 2024-09-11T23:27:11.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:11.314+0000 7fc7ce7fc640 1 -- 172.21.15.26:0/2217492724 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fc7d4042030 con 0x7fc7d81027a0 2024-09-11T23:27:11.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:11.314+0000 7fc7ce7fc640 1 --2- 172.21.15.26:0/2217492724 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fc7c40a2cf0 0x7fc7c40a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:11.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:11.315+0000 7fc7ce7fc640 1 -- 172.21.15.26:0/2217492724 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8586+0+0 (secure 0 0 0) 0x7fc7d40f0070 con 0x7fc7d81027a0 2024-09-11T23:27:11.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:11.315+0000 7fc7dd032640 1 --2- 172.21.15.26:0/2217492724 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fc7c40a2cf0 0x7fc7c40a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:11.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:11.315+0000 7fc7dd032640 1 --2- 172.21.15.26:0/2217492724 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fc7c40a2cf0 0x7fc7c40a51b0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fc7c800bbe0 tx=0x7fc7c800b390 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:27:11.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:11.318+0000 7fc7ce7fc640 1 -- 172.21.15.26:0/2217492724 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc7d40bcbf0 con 0x7fc7d81027a0 2024-09-11T23:27:11.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:11.533+0000 7fc7df296640 1 -- 172.21.15.26:0/2217492724 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc7d810a100 con 0x7fc7d81027a0 2024-09-11T23:27:11.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:11.533+0000 7fc7ce7fc640 1 -- 172.21.15.26:0/2217492724 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) ==== 94+0+11986 (secure 0 0 0) 0x7fc7d40bc4f0 con 0x7fc7d81027a0 2024-09-11T23:27:11.534 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-11T23:27:11.534 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":51,"btime":"2024-09-11T23:27:01:787119+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14499,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/367659515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":367659515},{"type":"v1","addr":"172.21.15.26:6835","nonce":367659515}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14511,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/2041065299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":2041065299},{"type":"v1","addr":"172.21.15.26:6837","nonce":2041065299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":51,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:16:26.544203+0000","modified":"2024-09-11T23:27:01.632593+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24485,"mds_1":24463,"mds_2":24458,"mds_3":24448,"mds_4":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14550":{"gid":14550,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6839/2233068305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":2233068305},{"type":"v1","addr":"172.21.15.26:6839","nonce":2233068305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14562":{"gid":14562,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6841/2984982336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":2984982336},{"type":"v1","addr":"172.21.15.26:6841","nonce":2984982336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6833/2019841455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":2019841455},{"type":"v1","addr":"172.21.15.136:6833","nonce":2019841455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.76:6833/3438147050","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3438147050},{"type":"v1","addr":"172.21.15.76:6833","nonce":3438147050}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6835/3617665975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3617665975},{"type":"v1","addr":"172.21.15.136:6835","nonce":3617665975}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":12,"addr":"172.21.15.76:6835/393308499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":393308499},{"type":"v1","addr":"172.21.15.76:6835","nonce":393308499}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":10,"addr":"172.21.15.136:6837/1633082387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":1633082387},{"type":"v1","addr":"172.21.15.136:6837","nonce":1633082387}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.76:6837/2078978748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":2078978748},{"type":"v1","addr":"172.21.15.76:6837","nonce":2078978748}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":63,"addr":"172.21.15.136:6839/3211398630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":3211398630},{"type":"v1","addr":"172.21.15.136:6839","nonce":3211398630}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.76:6839/2077637090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6838","nonce":2077637090},{"type":"v1","addr":"172.21.15.76:6839","nonce":2077637090}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24485,"qdb_cluster":[24443,24448,24458,24463,24485]},"id":1}]} 2024-09-11T23:27:11.534 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 51 2024-09-11T23:27:11.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:11.535+0000 7fc7df296640 1 -- 172.21.15.26:0/2217492724 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fc7c40a2cf0 msgr2=0x7fc7c40a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:11.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:11.535+0000 7fc7df296640 1 --2- 172.21.15.26:0/2217492724 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fc7c40a2cf0 0x7fc7c40a51b0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fc7c800bbe0 tx=0x7fc7c800b390 comp rx=0 tx=0).stop 2024-09-11T23:27:11.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:11.535+0000 7fc7df296640 1 -- 172.21.15.26:0/2217492724 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc7d81027a0 msgr2=0x7fc7d8198960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:11.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:11.535+0000 7fc7df296640 1 --2- 172.21.15.26:0/2217492724 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc7d81027a0 0x7fc7d8198960 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fc7d4006010 tx=0x7fc7d40042f0 comp rx=0 tx=0).stop 2024-09-11T23:27:11.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:11.536+0000 7fc7df296640 1 -- 172.21.15.26:0/2217492724 shutdown_connections 2024-09-11T23:27:11.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:11.536+0000 7fc7df296640 1 --2- 172.21.15.26:0/2217492724 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fc7c40a2cf0 0x7fc7c40a51b0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:11.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:11.536+0000 7fc7df296640 1 --2- 172.21.15.26:0/2217492724 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc7d8106f20 0x7fc7d819ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:11.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:11.536+0000 7fc7df296640 1 --2- 172.21.15.26:0/2217492724 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc7d81027a0 0x7fc7d8198960 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:11.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:11.536+0000 7fc7df296640 1 --2- 172.21.15.26:0/2217492724 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc7d8101df0 0x7fc7d8198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:11.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:11.536+0000 7fc7df296640 1 -- 172.21.15.26:0/2217492724 >> 172.21.15.26:0/2217492724 conn(0x7fc7d80fd8f0 msgr2=0x7fc7d80fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:27:11.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:11.536+0000 7fc7df296640 1 -- 172.21.15.26:0/2217492724 shutdown_connections 2024-09-11T23:27:11.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:11.536+0000 7fc7df296640 1 -- 172.21.15.26:0/2217492724 wait complete. 2024-09-11T23:27:12.137 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:27:11 smithi026 ceph-mon[31172]: pgmap v272: 97 pgs: 32 active+clean+snaptrim_wait, 16 active+clean+snaptrim, 49 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 98 KiB/s wr, 36 op/s 2024-09-11T23:27:12.138 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:27:11 smithi026 ceph-mon[31172]: from='client.? 172.21.15.26:0/2217492724' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:27:12.164 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24485 quiesce db --set-id 442853f3 --release --await' 2024-09-11T23:27:12.164 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24485 quiesce db --set-id 442853f3 --release --await 2024-09-11T23:27:12.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:27:11 smithi136 ceph-mon[35121]: pgmap v272: 97 pgs: 32 active+clean+snaptrim_wait, 16 active+clean+snaptrim, 49 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 98 KiB/s wr, 36 op/s 2024-09-11T23:27:12.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:27:11 smithi136 ceph-mon[35121]: from='client.? 172.21.15.26:0/2217492724' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:27:12.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:27:11 smithi076 ceph-mon[35296]: pgmap v272: 97 pgs: 32 active+clean+snaptrim_wait, 16 active+clean+snaptrim, 49 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 98 KiB/s wr, 36 op/s 2024-09-11T23:27:12.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:27:11 smithi076 ceph-mon[35296]: from='client.? 172.21.15.26:0/2217492724' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:27:12.424 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:27:13.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.048+0000 7f630dabd640 1 -- 172.21.15.26:0/4199900814 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6308102790 msgr2=0x7f63081067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:13.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.048+0000 7f630dabd640 1 --2- 172.21.15.26:0/4199900814 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6308102790 0x7f63081067e0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f62f0009900 tx=0x7f62f002a410 comp rx=0 tx=0).stop 2024-09-11T23:27:13.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.048+0000 7f630dabd640 1 -- 172.21.15.26:0/4199900814 shutdown_connections 2024-09-11T23:27:13.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.048+0000 7f630dabd640 1 --2- 172.21.15.26:0/4199900814 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6308106f10 0x7f630810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:13.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.048+0000 7f630dabd640 1 --2- 172.21.15.26:0/4199900814 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6308102790 0x7f63081067e0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:13.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.048+0000 7f630dabd640 1 --2- 172.21.15.26:0/4199900814 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6308101de0 0x7f63081021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:13.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.048+0000 7f630dabd640 1 -- 172.21.15.26:0/4199900814 >> 172.21.15.26:0/4199900814 conn(0x7f63080fd8c0 msgr2=0x7f63080ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:27:13.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.048+0000 7f630dabd640 1 -- 172.21.15.26:0/4199900814 shutdown_connections 2024-09-11T23:27:13.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.048+0000 7f630dabd640 1 -- 172.21.15.26:0/4199900814 wait complete. 2024-09-11T23:27:13.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.049+0000 7f630dabd640 1 Processor -- start 2024-09-11T23:27:13.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.049+0000 7f630dabd640 1 -- start start 2024-09-11T23:27:13.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.049+0000 7f630dabd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6308101de0 0x7f63081984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:13.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.049+0000 7f630dabd640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6308102790 0x7f6308198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:13.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.049+0000 7f630dabd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6308106f10 0x7f630819cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:13.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.049+0000 7f630dabd640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f6308111f90 con 0x7f6308101de0 2024-09-11T23:27:13.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.049+0000 7f630dabd640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f6308111e10 con 0x7f6308106f10 2024-09-11T23:27:13.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.049+0000 7f630dabd640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f6308112110 con 0x7f6308102790 2024-09-11T23:27:13.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.049+0000 7f63077fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6308101de0 0x7f63081984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:13.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.049+0000 7f63077fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6308101de0 0x7f63081984f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36374/0 (socket says 172.21.15.26:36374) 2024-09-11T23:27:13.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.049+0000 7f63077fe640 1 -- 172.21.15.26:0/940584467 learned_addr learned my addr 172.21.15.26:0/940584467 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:27:13.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.049+0000 7f6306ffd640 1 --2- 172.21.15.26:0/940584467 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6308102790 0x7f6308198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:13.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.049+0000 7f6307fff640 1 --2- 172.21.15.26:0/940584467 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6308106f10 0x7f630819cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:13.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.049+0000 7f63077fe640 1 -- 172.21.15.26:0/940584467 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6308102790 msgr2=0x7f6308198a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:13.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.049+0000 7f63077fe640 1 --2- 172.21.15.26:0/940584467 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6308102790 0x7f6308198a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:13.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.049+0000 7f63077fe640 1 -- 172.21.15.26:0/940584467 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6308106f10 msgr2=0x7f630819cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:13.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.050+0000 7f63077fe640 1 --2- 172.21.15.26:0/940584467 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6308106f10 0x7f630819cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:13.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.050+0000 7f63077fe640 1 -- 172.21.15.26:0/940584467 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f630819d4a0 con 0x7f6308101de0 2024-09-11T23:27:13.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.050+0000 7f6307fff640 1 --2- 172.21.15.26:0/940584467 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6308106f10 0x7f630819cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:27:13.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.050+0000 7f63077fe640 1 --2- 172.21.15.26:0/940584467 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6308101de0 0x7f63081984f0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f62f800c890 tx=0x7f62f800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:27:13.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.050+0000 7f6304ff9640 1 -- 172.21.15.26:0/940584467 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62f8004220 con 0x7f6308101de0 2024-09-11T23:27:13.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.050+0000 7f6304ff9640 1 -- 172.21.15.26:0/940584467 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f62f80043c0 con 0x7f6308101de0 2024-09-11T23:27:13.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.050+0000 7f630dabd640 1 -- 172.21.15.26:0/940584467 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f630819d790 con 0x7f6308101de0 2024-09-11T23:27:13.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.050+0000 7f6304ff9640 1 -- 172.21.15.26:0/940584467 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62f80049e0 con 0x7f6308101de0 2024-09-11T23:27:13.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.050+0000 7f630dabd640 1 -- 172.21.15.26:0/940584467 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63081a5070 con 0x7f6308101de0 2024-09-11T23:27:13.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.052+0000 7f630fe96740 1 Processor -- start 2024-09-11T23:27:13.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.052+0000 7f6304ff9640 1 -- 172.21.15.26:0/940584467 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f62f80079b0 con 0x7f6308101de0 2024-09-11T23:27:13.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.052+0000 7f630fe96740 1 -- start start 2024-09-11T23:27:13.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.053+0000 7f630fe96740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5612752e9060 0x5612752e9440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:13.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.053+0000 7f630fe96740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5612752e9980 0x5612752f3d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:13.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.053+0000 7f630fe96740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5612752f4240 0x5612752f6630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:13.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.053+0000 7f630fe96740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x56127524e620 con 0x5612752f4240 2024-09-11T23:27:13.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.053+0000 7f630fe96740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5612751f5e50 con 0x5612752e9980 2024-09-11T23:27:13.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.053+0000 7f630fe96740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x56127524a5e0 con 0x5612752e9060 2024-09-11T23:27:13.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.053+0000 7f6306ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5612752f4240 0x5612752f6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:13.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.053+0000 7f6306ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5612752f4240 0x5612752f6630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36376/0 (socket says 172.21.15.26:36376) 2024-09-11T23:27:13.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.053+0000 7f6306ffd640 1 -- 172.21.15.26:0/4243560649 learned_addr learned my addr 172.21.15.26:0/4243560649 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:27:13.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.053+0000 7f6307fff640 1 --2- 172.21.15.26:0/4243560649 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5612752e9060 0x5612752e9440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:13.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.053+0000 7f63077fe640 1 --2- 172.21.15.26:0/4243560649 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5612752e9980 0x5612752f3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:13.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.053+0000 7f6306ffd640 1 -- 172.21.15.26:0/4243560649 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5612752e9060 msgr2=0x5612752e9440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:27:13.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.053+0000 7f6306ffd640 1 --2- 172.21.15.26:0/4243560649 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5612752e9060 0x5612752e9440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:13.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.053+0000 7f6306ffd640 1 -- 172.21.15.26:0/4243560649 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5612752e9980 msgr2=0x5612752f3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:27:13.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.053+0000 7f6306ffd640 1 --2- 172.21.15.26:0/4243560649 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5612752e9980 0x5612752f3d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:13.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.053+0000 7f6306ffd640 1 -- 172.21.15.26:0/4243560649 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5612751e6d40 con 0x5612752f4240 2024-09-11T23:27:13.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.053+0000 7f6304ff9640 1 --2- 172.21.15.26:0/940584467 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f62f40a2cf0 0x7f62f40a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:13.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.053+0000 7f6306ffd640 1 --2- 172.21.15.26:0/4243560649 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5612752f4240 0x5612752f6630 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f62f0005ab0 tx=0x7f62f0031630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=517de0c07866d355 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:27:13.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.054+0000 7f6307fff640 1 --2- 172.21.15.26:0/940584467 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f62f40a2cf0 0x7f62f40a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:13.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.054+0000 7f6304ff9640 1 -- 172.21.15.26:0/940584467 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8586+0+0 (secure 0 0 0) 0x7f62f80cf580 con 0x7f6308101de0 2024-09-11T23:27:13.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.054+0000 7f62fdffb640 1 -- 172.21.15.26:0/4243560649 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62f003f070 con 0x5612752f4240 2024-09-11T23:27:13.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.054+0000 7f62fdffb640 1 -- 172.21.15.26:0/4243560649 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f62f0031a20 con 0x5612752f4240 2024-09-11T23:27:13.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.054+0000 7f62fdffb640 1 -- 172.21.15.26:0/4243560649 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62f0035070 con 0x5612752f4240 2024-09-11T23:27:13.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.054+0000 7f630fe96740 1 -- 172.21.15.26:0/4243560649 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5612752f4240 msgr2=0x5612752f6630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:13.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.054+0000 7f6307fff640 1 --2- 172.21.15.26:0/940584467 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f62f40a2cf0 0x7f62f40a51b0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f630000f4c0 tx=0x7f6300007740 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:27:13.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.054+0000 7f630fe96740 1 --2- 172.21.15.26:0/4243560649 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5612752f4240 0x5612752f6630 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f62f0005ab0 tx=0x7f62f0031630 comp rx=0 tx=0).stop 2024-09-11T23:27:13.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.054+0000 7f630fe96740 1 -- 172.21.15.26:0/4243560649 shutdown_connections 2024-09-11T23:27:13.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.054+0000 7f630fe96740 1 --2- 172.21.15.26:0/4243560649 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5612752e9980 0x5612752f3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:13.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.054+0000 7f630fe96740 1 --2- 172.21.15.26:0/4243560649 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5612752f4240 0x5612752f6630 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:13.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.054+0000 7f630fe96740 1 --2- 172.21.15.26:0/4243560649 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5612752e9060 0x5612752e9440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:13.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.054+0000 7f630fe96740 1 -- 172.21.15.26:0/4243560649 >> 172.21.15.26:0/4243560649 conn(0x5612752e55f0 msgr2=0x5612752e59d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:27:13.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.055+0000 7f630fe96740 1 -- 172.21.15.26:0/4243560649 shutdown_connections 2024-09-11T23:27:13.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.055+0000 7f630fe96740 1 -- 172.21.15.26:0/4243560649 wait complete. 2024-09-11T23:27:13.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.055+0000 7f630fe96740 1 -- start start 2024-09-11T23:27:13.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.055+0000 7f630fe96740 1 Processor -- start 2024-09-11T23:27:13.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.056+0000 7f630fe96740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5612752e9060 0x5612752f1120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:13.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.056+0000 7f630fe96740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5612752e9980 0x5612752f1660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:13.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.056+0000 7f630fe96740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5612752f4240 0x5612752f1ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:13.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.056+0000 7f630fe96740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5612752f6e10 con 0x5612752e9980 2024-09-11T23:27:13.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.056+0000 7f630fe96740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5612751d6f10 con 0x5612752f4240 2024-09-11T23:27:13.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.056+0000 7f630fe96740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5612752f6f90 con 0x5612752e9060 2024-09-11T23:27:13.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.056+0000 7f630fe96740 10 client.0 _ll_register_callbacks cb 0x5612751fadb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:27:13.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.056+0000 7f6306ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5612752e9980 0x5612752f1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:13.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.056+0000 7f63077fe640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5612752e9060 0x5612752f1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:13.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.056+0000 7f6306ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5612752e9980 0x5612752f1660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36390/0 (socket says 172.21.15.26:36390) 2024-09-11T23:27:13.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.056+0000 7f6306ffd640 1 -- 172.21.15.26:0/2523278989 learned_addr learned my addr 172.21.15.26:0/2523278989 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:27:13.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.056+0000 7f6307fff640 1 --2- 172.21.15.26:0/2523278989 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5612752f4240 0x5612752f1ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:13.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.056+0000 7f6306ffd640 1 -- 172.21.15.26:0/2523278989 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5612752e9060 msgr2=0x5612752f1120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:27:13.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.056+0000 7f6306ffd640 1 --2- 172.21.15.26:0/2523278989 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5612752e9060 0x5612752f1120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:13.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.056+0000 7f6306ffd640 1 -- 172.21.15.26:0/2523278989 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5612752f4240 msgr2=0x5612752f1ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:27:13.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.056+0000 7f6306ffd640 1 --2- 172.21.15.26:0/2523278989 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5612752f4240 0x5612752f1ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:13.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.056+0000 7f6306ffd640 1 -- 172.21.15.26:0/2523278989 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561274f5d580 con 0x5612752e9980 2024-09-11T23:27:13.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.056+0000 7f63077fe640 1 --2- 172.21.15.26:0/2523278989 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5612752e9060 0x5612752f1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:27:13.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.056+0000 7f6306ffd640 1 --2- 172.21.15.26:0/2523278989 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5612752e9980 0x5612752f1660 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f62f002a930 tx=0x7f62f0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=241abc0d9df5ecbc server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:27:13.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.056+0000 7f62e6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:27:13.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.056+0000 7f62e6ffd640 1 -- 172.21.15.26:0/2523278989 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62f004c060 con 0x5612752e9980 2024-09-11T23:27:13.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.056+0000 7f62e6ffd640 1 -- 172.21.15.26:0/2523278989 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8586+0+0 (secure 0 0 0) 0x7f62f0004030 con 0x5612752e9980 2024-09-11T23:27:13.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.056+0000 7f630fe96740 1 -- 172.21.15.26:0/2523278989 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f62f0031a20 con 0x5612752e9980 2024-09-11T23:27:13.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.057+0000 7f62e6ffd640 1 -- 172.21.15.26:0/2523278989 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f62f0035070 con 0x5612752e9980 2024-09-11T23:27:13.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.057+0000 7f630fe96740 10 client.16803 fetch_fsmap learned FSMap version 51 2024-09-11T23:27:13.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.057+0000 7f630fe96740 1 -- 172.21.15.26:0/2523278989 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x561275027960 con 0x5612752e9980 2024-09-11T23:27:13.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.057+0000 7f62e6ffd640 1 -- 172.21.15.26:0/2523278989 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f62f0031a20 con 0x5612752e9980 2024-09-11T23:27:13.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.057+0000 7f62e6ffd640 1 -- 172.21.15.26:0/2523278989 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 51) ==== 8394+0+0 (secure 0 0 0) 0x7f62f003b430 con 0x5612752e9980 2024-09-11T23:27:13.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.057+0000 7f630fe96740 10 client.16803 fetch_fsmap finished waiting for FSMap version 51 2024-09-11T23:27:13.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.057+0000 7f630fe96740 10 client.16803 resolve_mds: validated gid 24485 aka daemon mds.b 2024-09-11T23:27:13.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.057+0000 7f630fe96740 20 client.16803 populate_metadata read hostname 'smithi026' 2024-09-11T23:27:13.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.057+0000 7f630fe96740 1 --2- 172.21.15.26:0/2523278989 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x561275303c40 0x561275306030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:13.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.057+0000 7f630fe96740 4 client.16803 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:27:13.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.057+0000 7f630fe96740 1 -- 172.21.15.26:0/2523278989 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56127525bb90 con 0x561275303c40 2024-09-11T23:27:13.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.057+0000 7f63077fe640 1 --2- 172.21.15.26:0/2523278989 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x561275303c40 0x561275306030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:13.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.058+0000 7f63077fe640 1 --2- 172.21.15.26:0/2523278989 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x561275303c40 0x561275306030 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa31571571d5e87b server_cookie=7ffa06106f61fdc9 in_seq=0 out_seq=0 2024-09-11T23:27:13.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.058+0000 7f62e6ffd640 10 client.16803 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:27:13.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.059+0000 7f62e6ffd640 1 -- 172.21.15.26:0/2523278989 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56127525bb90 con 0x561275303c40 2024-09-11T23:27:13.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.059+0000 7f62e6ffd640 10 client.16803 handle_command_reply: tid=0 2024-09-11T23:27:13.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.059+0000 7f630fe96740 1 client.16803 shutdown 2024-09-11T23:27:13.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.059+0000 7f630fe96740 1 -- 172.21.15.26:0/2523278989 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5612752e9980 msgr2=0x5612752f1660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:13.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.059+0000 7f630fe96740 1 --2- 172.21.15.26:0/2523278989 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5612752e9980 0x5612752f1660 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f62f002a930 tx=0x7f62f0004a40 comp rx=0 tx=0).stop 2024-09-11T23:27:13.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.059+0000 7f630fe96740 1 -- 172.21.15.26:0/2523278989 shutdown_connections 2024-09-11T23:27:13.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.059+0000 7f630fe96740 1 --2- 172.21.15.26:0/2523278989 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5612752f4240 0x5612752f1ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:13.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.059+0000 7f630fe96740 1 --2- 172.21.15.26:0/2523278989 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x561275303c40 0x561275306030 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:13.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.059+0000 7f630fe96740 1 --2- 172.21.15.26:0/2523278989 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5612752e9980 0x5612752f1660 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:13.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.059+0000 7f630fe96740 1 --2- 172.21.15.26:0/2523278989 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5612752e9060 0x5612752f1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:13.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.059+0000 7f62e6ffd640 0 client.16803 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:27:13.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.059+0000 7f630fe96740 1 -- 172.21.15.26:0/2523278989 >> 172.21.15.26:0/2523278989 conn(0x5612752e55f0 msgr2=0x5612752eb2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:27:13.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.059+0000 7f630fe96740 1 -- 172.21.15.26:0/2523278989 shutdown_connections 2024-09-11T23:27:13.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.059+0000 7f630fe96740 1 -- 172.21.15.26:0/2523278989 wait complete. 2024-09-11T23:27:13.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.059+0000 7f630fe96740 20 client.16803 trim_cache size 0 max 16384 2024-09-11T23:27:13.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.076+0000 7f630fe96740 1 Processor -- start 2024-09-11T23:27:13.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.077+0000 7f630fe96740 1 -- start start 2024-09-11T23:27:13.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.077+0000 7f630fe96740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5612752e9060 0x561275309050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:13.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.077+0000 7f630fe96740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5612752e9980 0x561275309590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:13.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.077+0000 7f630fe96740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5612752f4240 0x5612752ee380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:13.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.077+0000 7f630fe96740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f63081ada30 con 0x5612752f4240 2024-09-11T23:27:13.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.077+0000 7f630fe96740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f63081af6f0 con 0x5612752e9980 2024-09-11T23:27:13.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.077+0000 7f630fe96740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f63081ae100 con 0x5612752e9060 2024-09-11T23:27:13.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.077+0000 7f6307fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5612752f4240 0x5612752ee380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:13.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.077+0000 7f6307fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5612752f4240 0x5612752ee380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36394/0 (socket says 172.21.15.26:36394) 2024-09-11T23:27:13.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.077+0000 7f6307fff640 1 -- 172.21.15.26:0/3136036225 learned_addr learned my addr 172.21.15.26:0/3136036225 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:27:13.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.077+0000 7f63077fe640 1 --2- 172.21.15.26:0/3136036225 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5612752e9060 0x561275309050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:13.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.077+0000 7f6306ffd640 1 --2- 172.21.15.26:0/3136036225 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5612752e9980 0x561275309590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:13.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.077+0000 7f6307fff640 1 -- 172.21.15.26:0/3136036225 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5612752e9060 msgr2=0x561275309050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:27:13.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.077+0000 7f6307fff640 1 --2- 172.21.15.26:0/3136036225 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5612752e9060 0x561275309050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:13.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.077+0000 7f6307fff640 1 -- 172.21.15.26:0/3136036225 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5612752e9980 msgr2=0x561275309590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:27:13.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.077+0000 7f6307fff640 1 --2- 172.21.15.26:0/3136036225 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5612752e9980 0x561275309590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:13.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.077+0000 7f6307fff640 1 -- 172.21.15.26:0/3136036225 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5612752e9eb0 con 0x5612752f4240 2024-09-11T23:27:13.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.077+0000 7f6307fff640 1 --2- 172.21.15.26:0/3136036225 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5612752f4240 0x5612752ee380 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f63000108e0 tx=0x7f6300012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=134bb18fcfbe5912 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:27:13.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.078+0000 7f62fd7fa640 1 -- 172.21.15.26:0/3136036225 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6300010e60 con 0x5612752f4240 2024-09-11T23:27:13.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.078+0000 7f62fd7fa640 1 -- 172.21.15.26:0/3136036225 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f630001e430 con 0x5612752f4240 2024-09-11T23:27:13.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.078+0000 7f62fd7fa640 1 -- 172.21.15.26:0/3136036225 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6300026630 con 0x5612752f4240 2024-09-11T23:27:13.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.078+0000 7f630fe96740 1 -- 172.21.15.26:0/3136036225 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5612752f4240 msgr2=0x5612752ee380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:13.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.078+0000 7f630fe96740 1 --2- 172.21.15.26:0/3136036225 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5612752f4240 0x5612752ee380 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f63000108e0 tx=0x7f6300012000 comp rx=0 tx=0).stop 2024-09-11T23:27:13.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.078+0000 7f630fe96740 1 -- 172.21.15.26:0/3136036225 shutdown_connections 2024-09-11T23:27:13.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.078+0000 7f630fe96740 1 --2- 172.21.15.26:0/3136036225 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5612752e9980 0x561275309590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:13.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.078+0000 7f630fe96740 1 --2- 172.21.15.26:0/3136036225 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5612752f4240 0x5612752ee380 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:13.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.078+0000 7f630fe96740 1 --2- 172.21.15.26:0/3136036225 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5612752e9060 0x561275309050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:13.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.078+0000 7f630fe96740 1 -- 172.21.15.26:0/3136036225 >> 172.21.15.26:0/3136036225 conn(0x5612752e55f0 msgr2=0x5612752ef9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:27:13.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.079+0000 7f630fe96740 1 -- 172.21.15.26:0/3136036225 shutdown_connections 2024-09-11T23:27:13.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.079+0000 7f630fe96740 1 -- 172.21.15.26:0/3136036225 wait complete. 2024-09-11T23:27:13.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.079+0000 7f630fe96740 1 -- start start 2024-09-11T23:27:13.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.079+0000 7f630fe96740 1 Processor -- start 2024-09-11T23:27:13.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.079+0000 7f630fe96740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5612752e9060 0x5612753074f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:13.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.079+0000 7f630fe96740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5612752e9980 0x561275307a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:13.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.079+0000 7f630fe96740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5612752f4240 0x56127536d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:13.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.079+0000 7f630fe96740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f63081af020 con 0x5612752f4240 2024-09-11T23:27:13.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.079+0000 7f630fe96740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f63081ae890 con 0x5612752e9980 2024-09-11T23:27:13.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.079+0000 7f630fe96740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f63081aba30 con 0x5612752e9060 2024-09-11T23:27:13.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.080+0000 7f630fe96740 10 client.0 _ll_register_callbacks cb 0x7f62f0005350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:27:13.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.080+0000 7f6307fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5612752f4240 0x56127536d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:13.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.080+0000 7f63077fe640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5612752e9060 0x5612753074f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:13.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.080+0000 7f6306ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5612752e9980 0x561275307a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:13.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.080+0000 7f6307fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5612752f4240 0x56127536d580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36402/0 (socket says 172.21.15.26:36402) 2024-09-11T23:27:13.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.080+0000 7f6306ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5612752e9980 0x561275307a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:37454/0 (socket says 172.21.15.26:37454) 2024-09-11T23:27:13.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.080+0000 7f6307fff640 1 -- 172.21.15.26:0/2044841355 learned_addr learned my addr 172.21.15.26:0/2044841355 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:27:13.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.080+0000 7f6307fff640 1 -- 172.21.15.26:0/2044841355 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5612752e9060 msgr2=0x5612753074f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:27:13.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.080+0000 7f6307fff640 1 --2- 172.21.15.26:0/2044841355 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5612752e9060 0x5612753074f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:13.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.080+0000 7f6307fff640 1 -- 172.21.15.26:0/2044841355 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5612752e9980 msgr2=0x561275307a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:27:13.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.080+0000 7f6307fff640 1 --2- 172.21.15.26:0/2044841355 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5612752e9980 0x561275307a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:13.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.080+0000 7f6307fff640 1 -- 172.21.15.26:0/2044841355 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56127536dac0 con 0x5612752f4240 2024-09-11T23:27:13.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.080+0000 7f63077fe640 1 --2- 172.21.15.26:0/2044841355 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5612752e9060 0x5612753074f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:27:13.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.080+0000 7f6307fff640 1 --2- 172.21.15.26:0/2044841355 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5612752f4240 0x56127536d580 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f630001d790 tx=0x7f6300010f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43f45d40b05c4272 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:27:13.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.080+0000 7f62e6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:27:13.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.080+0000 7f62e6ffd640 1 -- 172.21.15.26:0/2044841355 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6300035020 con 0x5612752f4240 2024-09-11T23:27:13.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.080+0000 7f62e6ffd640 1 -- 172.21.15.26:0/2044841355 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8586+0+0 (secure 0 0 0) 0x7f630001e7f0 con 0x5612752f4240 2024-09-11T23:27:13.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.080+0000 7f630fe96740 1 -- 172.21.15.26:0/2044841355 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f630001e430 con 0x5612752f4240 2024-09-11T23:27:13.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.081+0000 7f62e6ffd640 1 -- 172.21.15.26:0/2044841355 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f6300040050 con 0x5612752f4240 2024-09-11T23:27:13.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.081+0000 7f630fe96740 10 client.16809 fetch_fsmap learned FSMap version 51 2024-09-11T23:27:13.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.081+0000 7f630fe96740 1 -- 172.21.15.26:0/2044841355 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x56127536edd0 con 0x5612752f4240 2024-09-11T23:27:13.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.081+0000 7f62e6ffd640 1 -- 172.21.15.26:0/2044841355 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f630001e430 con 0x5612752f4240 2024-09-11T23:27:13.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.081+0000 7f62e6ffd640 1 -- 172.21.15.26:0/2044841355 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 51) ==== 8394+0+0 (secure 0 0 0) 0x7f6300031950 con 0x5612752f4240 2024-09-11T23:27:13.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.081+0000 7f630fe96740 10 client.16809 fetch_fsmap finished waiting for FSMap version 51 2024-09-11T23:27:13.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.081+0000 7f630fe96740 10 client.16809 resolve_mds: validated gid 24485 aka daemon mds.b 2024-09-11T23:27:13.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.081+0000 7f630fe96740 20 client.16809 populate_metadata read hostname 'smithi026' 2024-09-11T23:27:13.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.081+0000 7f630fe96740 1 --2- 172.21.15.26:0/2044841355 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x56127536f510 0x561275371930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:13.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.081+0000 7f630fe96740 4 client.16809 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "442853f3", "release": true, "await": true}] 2024-09-11T23:27:13.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.081+0000 7f630fe96740 1 -- 172.21.15.26:0/2044841355 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "quiesce db", "set_id": "442853f3", "release": true, "await": true}) -- 0x5612752560a0 con 0x56127536f510 2024-09-11T23:27:13.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.081+0000 7f63077fe640 1 --2- 172.21.15.26:0/2044841355 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x56127536f510 0x561275371930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:13.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.082+0000 7f63077fe640 1 --2- 172.21.15.26:0/2044841355 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x56127536f510 0x561275371930 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75d80e179faeee4 server_cookie=3120330f2d702506 in_seq=0 out_seq=0 2024-09-11T23:27:13.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:13.082+0000 7f62e6ffd640 10 client.16809 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:27:14.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:27:13 smithi136 ceph-mon[35121]: pgmap v273: 97 pgs: 32 active+clean+snaptrim_wait, 16 active+clean+snaptrim, 49 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 98 KiB/s wr, 36 op/s 2024-09-11T23:27:14.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:27:13 smithi026 ceph-mon[31172]: pgmap v273: 97 pgs: 32 active+clean+snaptrim_wait, 16 active+clean+snaptrim, 49 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 98 KiB/s wr, 36 op/s 2024-09-11T23:27:14.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:27:13 smithi076 ceph-mon[35296]: pgmap v273: 97 pgs: 32 active+clean+snaptrim_wait, 16 active+clean+snaptrim, 49 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 98 KiB/s wr, 36 op/s 2024-09-11T23:27:14.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:14.616+0000 7f62e6ffd640 1 -- 172.21.15.26:0/2044841355 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5612752560a0 con 0x56127536f510 2024-09-11T23:27:14.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:14.616+0000 7f62e6ffd640 10 client.16809 handle_command_reply: tid=0 2024-09-11T23:27:14.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:14.616+0000 7f630fe96740 1 client.16809 shutdown 2024-09-11T23:27:14.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:14.616+0000 7f630fe96740 1 -- 172.21.15.26:0/2044841355 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5612752f4240 msgr2=0x56127536d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:14.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:14.616+0000 7f630fe96740 1 --2- 172.21.15.26:0/2044841355 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5612752f4240 0x56127536d580 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f630001d790 tx=0x7f6300010f60 comp rx=0 tx=0).stop 2024-09-11T23:27:14.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:14.616+0000 7f630fe96740 1 -- 172.21.15.26:0/2044841355 shutdown_connections 2024-09-11T23:27:14.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:14.616+0000 7f630fe96740 1 --2- 172.21.15.26:0/2044841355 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5612752e9980 0x561275307a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:14.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:14.616+0000 7f630fe96740 1 --2- 172.21.15.26:0/2044841355 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x56127536f510 0x561275371930 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:14.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:14.616+0000 7f62e6ffd640 0 client.16809 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:27:14.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:14.616+0000 7f630fe96740 1 --2- 172.21.15.26:0/2044841355 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5612752f4240 0x56127536d580 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:14.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:14.616+0000 7f630fe96740 1 --2- 172.21.15.26:0/2044841355 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5612752e9060 0x5612753074f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:14.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:14.616+0000 7f630fe96740 1 -- 172.21.15.26:0/2044841355 >> 172.21.15.26:0/2044841355 conn(0x5612752e55f0 msgr2=0x5612752f1090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:27:14.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:14.616+0000 7f630fe96740 1 -- 172.21.15.26:0/2044841355 shutdown_connections 2024-09-11T23:27:14.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:14.616+0000 7f630fe96740 1 -- 172.21.15.26:0/2044841355 wait complete. 2024-09-11T23:27:14.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:14.616+0000 7f630fe96740 20 client.16809 trim_cache size 0 max 16384 2024-09-11T23:27:14.618 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-11T23:27:14.618 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 51, 2024-09-11T23:27:14.618 INFO:teuthology.orchestra.run.smithi026.stdout: "leader": 24485, 2024-09-11T23:27:14.618 INFO:teuthology.orchestra.run.smithi026.stdout: "set_version": 32, 2024-09-11T23:27:14.618 INFO:teuthology.orchestra.run.smithi026.stdout: "sets": { 2024-09-11T23:27:14.618 INFO:teuthology.orchestra.run.smithi026.stdout: "442853f3": { 2024-09-11T23:27:14.618 INFO:teuthology.orchestra.run.smithi026.stdout: "version": 32, 2024-09-11T23:27:14.618 INFO:teuthology.orchestra.run.smithi026.stdout: "age_ref": 0.0, 2024-09-11T23:27:14.618 INFO:teuthology.orchestra.run.smithi026.stdout: "state": { 2024-09-11T23:27:14.619 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "RELEASED", 2024-09-11T23:27:14.619 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.0 2024-09-11T23:27:14.619 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:27:14.619 INFO:teuthology.orchestra.run.smithi026.stdout: "timeout": 90.0, 2024-09-11T23:27:14.619 INFO:teuthology.orchestra.run.smithi026.stdout: "expiration": 139.4, 2024-09-11T23:27:14.619 INFO:teuthology.orchestra.run.smithi026.stdout: "members": { 2024-09-11T23:27:14.619 INFO:teuthology.orchestra.run.smithi026.stdout: "file:/": { 2024-09-11T23:27:14.619 INFO:teuthology.orchestra.run.smithi026.stdout: "excluded": false, 2024-09-11T23:27:14.619 INFO:teuthology.orchestra.run.smithi026.stdout: "state": { 2024-09-11T23:27:14.619 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "RELEASED", 2024-09-11T23:27:14.619 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.0 2024-09-11T23:27:14.619 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:27:14.619 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:27:14.619 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:27:14.620 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:27:14.620 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:27:14.620 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-11T23:27:14.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:14.617+0000 7f630dabd640 1 -- 172.21.15.26:0/940584467 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f62f40a2cf0 msgr2=0x7f62f40a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:14.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:14.617+0000 7f630dabd640 1 --2- 172.21.15.26:0/940584467 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f62f40a2cf0 0x7f62f40a51b0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f630000f4c0 tx=0x7f6300007740 comp rx=0 tx=0).stop 2024-09-11T23:27:14.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:14.617+0000 7f630dabd640 1 -- 172.21.15.26:0/940584467 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6308101de0 msgr2=0x7f63081984f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:14.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:14.617+0000 7f630dabd640 1 --2- 172.21.15.26:0/940584467 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6308101de0 0x7f63081984f0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f62f800c890 tx=0x7f62f800cd50 comp rx=0 tx=0).stop 2024-09-11T23:27:14.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:14.617+0000 7f630dabd640 1 -- 172.21.15.26:0/940584467 shutdown_connections 2024-09-11T23:27:14.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:14.617+0000 7f630dabd640 1 --2- 172.21.15.26:0/940584467 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f62f40a2cf0 0x7f62f40a51b0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:14.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:14.617+0000 7f630dabd640 1 --2- 172.21.15.26:0/940584467 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6308106f10 0x7f630819cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:14.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:14.617+0000 7f630dabd640 1 --2- 172.21.15.26:0/940584467 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6308102790 0x7f6308198a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:14.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:14.617+0000 7f630dabd640 1 --2- 172.21.15.26:0/940584467 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6308101de0 0x7f63081984f0 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:14.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:14.617+0000 7f630dabd640 1 -- 172.21.15.26:0/940584467 >> 172.21.15.26:0/940584467 conn(0x7f63080fd8c0 msgr2=0x7f63080fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:27:14.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:14.617+0000 7f630dabd640 1 -- 172.21.15.26:0/940584467 shutdown_connections 2024-09-11T23:27:14.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:14.618+0000 7f630dabd640 1 -- 172.21.15.26:0/940584467 wait complete. 2024-09-11T23:27:15.542 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '442853f3', total seconds elapsed: 24.4 2024-09-11T23:27:15.542 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 19.4 2024-09-11T23:27:15.542 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:27:15.808 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:27:16.087 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:27:15 smithi026 ceph-mon[31172]: pgmap v274: 97 pgs: 10 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 83 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 3.5 KiB/s wr, 12 op/s 2024-09-11T23:27:16.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:27:15 smithi136 ceph-mon[35121]: pgmap v274: 97 pgs: 10 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 83 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 3.5 KiB/s wr, 12 op/s 2024-09-11T23:27:16.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:27:15 smithi076 ceph-mon[35296]: pgmap v274: 97 pgs: 10 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 83 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 3.5 KiB/s wr, 12 op/s 2024-09-11T23:27:16.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:16.408+0000 7f13fb726640 1 -- 172.21.15.26:0/426052105 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f13f4100e20 msgr2=0x7f13f4101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:16.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:16.408+0000 7f13fb726640 1 --2- 172.21.15.26:0/426052105 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f13f4100e20 0x7f13f4101200 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f13f0009ca0 tx=0x7f13f002a950 comp rx=0 tx=0).stop 2024-09-11T23:27:16.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:16.408+0000 7f13fb726640 1 -- 172.21.15.26:0/426052105 shutdown_connections 2024-09-11T23:27:16.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:16.408+0000 7f13fb726640 1 --2- 172.21.15.26:0/426052105 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f13f4102ef0 0x7f13f410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:16.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:16.408+0000 7f13fb726640 1 --2- 172.21.15.26:0/426052105 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f13f4102570 0x7f13f41029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:16.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:16.408+0000 7f13fb726640 1 --2- 172.21.15.26:0/426052105 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f13f4100e20 0x7f13f4101200 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:16.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:16.408+0000 7f13fb726640 1 -- 172.21.15.26:0/426052105 >> 172.21.15.26:0/426052105 conn(0x7f13f40fc7d0 msgr2=0x7f13f40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:27:16.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:16.408+0000 7f13fb726640 1 -- 172.21.15.26:0/426052105 shutdown_connections 2024-09-11T23:27:16.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:16.408+0000 7f13fb726640 1 -- 172.21.15.26:0/426052105 wait complete. 2024-09-11T23:27:16.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:16.408+0000 7f13fb726640 1 Processor -- start 2024-09-11T23:27:16.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:16.408+0000 7f13fb726640 1 -- start start 2024-09-11T23:27:16.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:16.409+0000 7f13fb726640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f13f4100e20 0x7f13f41993a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:16.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:16.409+0000 7f13fb726640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f13f4102570 0x7f13f41998e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:16.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:16.409+0000 7f13fb726640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f13f4102ef0 0x7f13f41a4420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:16.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:16.409+0000 7f13fb726640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f13f4110fc0 con 0x7f13f4102ef0 2024-09-11T23:27:16.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:16.409+0000 7f13fb726640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f13f4110e40 con 0x7f13f4100e20 2024-09-11T23:27:16.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:16.409+0000 7f13fb726640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f13f4111140 con 0x7f13f4102570 2024-09-11T23:27:16.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:16.409+0000 7f13f9cc3640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f13f4102ef0 0x7f13f41a4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:16.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:16.409+0000 7f13f9cc3640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f13f4102ef0 0x7f13f41a4420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36414/0 (socket says 172.21.15.26:36414) 2024-09-11T23:27:16.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:16.409+0000 7f13f9cc3640 1 -- 172.21.15.26:0/531562213 learned_addr learned my addr 172.21.15.26:0/531562213 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:27:16.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:16.409+0000 7f13f94c2640 1 --2- 172.21.15.26:0/531562213 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f13f4100e20 0x7f13f41993a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:16.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:16.409+0000 7f13f8cc1640 1 --2- 172.21.15.26:0/531562213 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f13f4102570 0x7f13f41998e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:16.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:16.409+0000 7f13f9cc3640 1 -- 172.21.15.26:0/531562213 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f13f4102570 msgr2=0x7f13f41998e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:16.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:16.409+0000 7f13f9cc3640 1 --2- 172.21.15.26:0/531562213 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f13f4102570 0x7f13f41998e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:16.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:16.409+0000 7f13f9cc3640 1 -- 172.21.15.26:0/531562213 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f13f4100e20 msgr2=0x7f13f41993a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:16.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:16.409+0000 7f13f9cc3640 1 --2- 172.21.15.26:0/531562213 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f13f4100e20 0x7f13f41993a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:16.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:16.409+0000 7f13f9cc3640 1 -- 172.21.15.26:0/531562213 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13f41a4960 con 0x7f13f4102ef0 2024-09-11T23:27:16.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:16.409+0000 7f13f8cc1640 1 --2- 172.21.15.26:0/531562213 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f13f4102570 0x7f13f41998e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:27:16.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:16.409+0000 7f13f9cc3640 1 --2- 172.21.15.26:0/531562213 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f13f4102ef0 0x7f13f41a4420 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f13ec009fd0 tx=0x7f13ec00b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:27:16.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:16.410+0000 7f13ea7fc640 1 -- 172.21.15.26:0/531562213 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13ec011830 con 0x7f13f4102ef0 2024-09-11T23:27:16.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:16.410+0000 7f13ea7fc640 1 -- 172.21.15.26:0/531562213 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f13ec016070 con 0x7f13f4102ef0 2024-09-11T23:27:16.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:16.410+0000 7f13fb726640 1 -- 172.21.15.26:0/531562213 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13f41a4b30 con 0x7f13f4102ef0 2024-09-11T23:27:16.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:16.410+0000 7f13ea7fc640 1 -- 172.21.15.26:0/531562213 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13ec00f2e0 con 0x7f13f4102ef0 2024-09-11T23:27:16.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:16.410+0000 7f13fb726640 1 -- 172.21.15.26:0/531562213 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13f41a4ff0 con 0x7f13f4102ef0 2024-09-11T23:27:16.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:16.411+0000 7f13fb726640 1 -- 172.21.15.26:0/531562213 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13f4101200 con 0x7f13f4102ef0 2024-09-11T23:27:16.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:16.412+0000 7f13ea7fc640 1 -- 172.21.15.26:0/531562213 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f13ec0119d0 con 0x7f13f4102ef0 2024-09-11T23:27:16.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:16.413+0000 7f13ea7fc640 1 --2- 172.21.15.26:0/531562213 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f13e00a2cf0 0x7f13e00a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:16.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:16.413+0000 7f13f94c2640 1 --2- 172.21.15.26:0/531562213 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f13e00a2cf0 0x7f13e00a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:16.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:16.413+0000 7f13ea7fc640 1 -- 172.21.15.26:0/531562213 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8586+0+0 (secure 0 0 0) 0x7f13ec0cf0e0 con 0x7f13f4102ef0 2024-09-11T23:27:16.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:16.413+0000 7f13f94c2640 1 --2- 172.21.15.26:0/531562213 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f13e00a2cf0 0x7f13e00a51b0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f13f0004ad0 tx=0x7f13f0004a60 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:27:16.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:16.416+0000 7f13ea7fc640 1 -- 172.21.15.26:0/531562213 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f13ec09bdd0 con 0x7f13f4102ef0 2024-09-11T23:27:16.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:16.634+0000 7f13fb726640 1 -- 172.21.15.26:0/531562213 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f13f406a560 con 0x7f13f4102ef0 2024-09-11T23:27:16.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:16.635+0000 7f13ea7fc640 1 -- 172.21.15.26:0/531562213 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) ==== 94+0+11986 (secure 0 0 0) 0x7f13f406a560 con 0x7f13f4102ef0 2024-09-11T23:27:16.635 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-11T23:27:16.635 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":51,"btime":"2024-09-11T23:27:01:787119+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14499,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/367659515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":367659515},{"type":"v1","addr":"172.21.15.26:6835","nonce":367659515}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14511,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/2041065299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":2041065299},{"type":"v1","addr":"172.21.15.26:6837","nonce":2041065299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":51,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:16:26.544203+0000","modified":"2024-09-11T23:27:01.632593+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24485,"mds_1":24463,"mds_2":24458,"mds_3":24448,"mds_4":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14550":{"gid":14550,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6839/2233068305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":2233068305},{"type":"v1","addr":"172.21.15.26:6839","nonce":2233068305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14562":{"gid":14562,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6841/2984982336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":2984982336},{"type":"v1","addr":"172.21.15.26:6841","nonce":2984982336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6833/2019841455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":2019841455},{"type":"v1","addr":"172.21.15.136:6833","nonce":2019841455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.76:6833/3438147050","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3438147050},{"type":"v1","addr":"172.21.15.76:6833","nonce":3438147050}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6835/3617665975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3617665975},{"type":"v1","addr":"172.21.15.136:6835","nonce":3617665975}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":12,"addr":"172.21.15.76:6835/393308499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":393308499},{"type":"v1","addr":"172.21.15.76:6835","nonce":393308499}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":10,"addr":"172.21.15.136:6837/1633082387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":1633082387},{"type":"v1","addr":"172.21.15.136:6837","nonce":1633082387}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.76:6837/2078978748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":2078978748},{"type":"v1","addr":"172.21.15.76:6837","nonce":2078978748}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":63,"addr":"172.21.15.136:6839/3211398630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":3211398630},{"type":"v1","addr":"172.21.15.136:6839","nonce":3211398630}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.76:6839/2077637090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6838","nonce":2077637090},{"type":"v1","addr":"172.21.15.76:6839","nonce":2077637090}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24485,"qdb_cluster":[24443,24448,24458,24463,24485]},"id":1}]} 2024-09-11T23:27:16.635 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 51 2024-09-11T23:27:16.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:16.637+0000 7f13fb726640 1 -- 172.21.15.26:0/531562213 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f13e00a2cf0 msgr2=0x7f13e00a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:16.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:16.637+0000 7f13fb726640 1 --2- 172.21.15.26:0/531562213 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f13e00a2cf0 0x7f13e00a51b0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f13f0004ad0 tx=0x7f13f0004a60 comp rx=0 tx=0).stop 2024-09-11T23:27:16.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:16.637+0000 7f13fb726640 1 -- 172.21.15.26:0/531562213 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f13f4102ef0 msgr2=0x7f13f41a4420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:16.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:16.637+0000 7f13fb726640 1 --2- 172.21.15.26:0/531562213 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f13f4102ef0 0x7f13f41a4420 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f13ec009fd0 tx=0x7f13ec00b8e0 comp rx=0 tx=0).stop 2024-09-11T23:27:16.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:16.637+0000 7f13fb726640 1 -- 172.21.15.26:0/531562213 shutdown_connections 2024-09-11T23:27:16.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:16.637+0000 7f13fb726640 1 --2- 172.21.15.26:0/531562213 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f13e00a2cf0 0x7f13e00a51b0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:16.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:16.637+0000 7f13fb726640 1 --2- 172.21.15.26:0/531562213 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f13f4102ef0 0x7f13f41a4420 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:16.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:16.637+0000 7f13fb726640 1 --2- 172.21.15.26:0/531562213 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f13f4102570 0x7f13f41998e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:16.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:16.637+0000 7f13fb726640 1 --2- 172.21.15.26:0/531562213 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f13f4100e20 0x7f13f41993a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:16.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:16.637+0000 7f13fb726640 1 -- 172.21.15.26:0/531562213 >> 172.21.15.26:0/531562213 conn(0x7f13f40fc7d0 msgr2=0x7f13f40fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:27:16.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:16.637+0000 7f13fb726640 1 -- 172.21.15.26:0/531562213 shutdown_connections 2024-09-11T23:27:16.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:16.637+0000 7f13fb726640 1 -- 172.21.15.26:0/531562213 wait complete. 2024-09-11T23:27:16.995 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:27:16 smithi026 ceph-mon[31172]: from='client.? 172.21.15.26:0/531562213' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:27:17.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:27:16 smithi136 ceph-mon[35121]: from='client.? 172.21.15.26:0/531562213' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:27:17.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:27:16 smithi076 ceph-mon[35296]: from='client.? 172.21.15.26:0/531562213' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:27:17.334 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24485 quiesce db / --timeout 90 --expiration 139.4 --await-for 45.0' 2024-09-11T23:27:17.334 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24485 quiesce db / --timeout 90 --expiration 139.4 --await-for 45.0 2024-09-11T23:27:17.593 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:27:18.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:27:17 smithi026 ceph-mon[31172]: pgmap v275: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 2.6 KiB/s wr, 12 op/s 2024-09-11T23:27:18.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.192+0000 7f29da4e0640 1 -- 172.21.15.26:0/3320835892 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f29d4106f50 msgr2=0x7f29d410eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:18.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.192+0000 7f29da4e0640 1 --2- 172.21.15.26:0/3320835892 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f29d4106f50 0x7f29d410eaf0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f29cc009cd0 tx=0x7f29cc02a8e0 comp rx=0 tx=0).stop 2024-09-11T23:27:18.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.192+0000 7f29da4e0640 1 -- 172.21.15.26:0/3320835892 shutdown_connections 2024-09-11T23:27:18.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.192+0000 7f29da4e0640 1 --2- 172.21.15.26:0/3320835892 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f29d4106f50 0x7f29d410eaf0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:18.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.192+0000 7f29da4e0640 1 --2- 172.21.15.26:0/3320835892 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f29d4100ca0 0x7f29d4106840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:18.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.192+0000 7f29da4e0640 1 --2- 172.21.15.26:0/3320835892 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f29d41002f0 0x7f29d41006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:18.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.192+0000 7f29da4e0640 1 -- 172.21.15.26:0/3320835892 >> 172.21.15.26:0/3320835892 conn(0x7f29d4077210 msgr2=0x7f29d40fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:27:18.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.192+0000 7f29da4e0640 1 -- 172.21.15.26:0/3320835892 shutdown_connections 2024-09-11T23:27:18.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.192+0000 7f29da4e0640 1 -- 172.21.15.26:0/3320835892 wait complete. 2024-09-11T23:27:18.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.193+0000 7f29da4e0640 1 Processor -- start 2024-09-11T23:27:18.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.193+0000 7f29da4e0640 1 -- start start 2024-09-11T23:27:18.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.193+0000 7f29da4e0640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f29d41002f0 0x7f29d4101270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:18.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.193+0000 7f29da4e0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f29d4100ca0 0x7f29d41017b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:18.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.193+0000 7f29da4e0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f29d4106f50 0x7f29d4101cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:18.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.193+0000 7f29da4e0640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f29d4111fa0 con 0x7f29d4106f50 2024-09-11T23:27:18.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.193+0000 7f29da4e0640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f29d4111e20 con 0x7f29d4100ca0 2024-09-11T23:27:18.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.193+0000 7f29da4e0640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f29d4112120 con 0x7f29d41002f0 2024-09-11T23:27:18.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.193+0000 7f29d8a7d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f29d4106f50 0x7f29d4101cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:18.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.193+0000 7f29d8a7d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f29d4106f50 0x7f29d4101cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36422/0 (socket says 172.21.15.26:36422) 2024-09-11T23:27:18.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.193+0000 7f29d8a7d640 1 -- 172.21.15.26:0/3696073587 learned_addr learned my addr 172.21.15.26:0/3696073587 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:27:18.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.194+0000 7f29d37fe640 1 --2- 172.21.15.26:0/3696073587 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f29d4100ca0 0x7f29d41017b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:18.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.194+0000 7f29d3fff640 1 --2- 172.21.15.26:0/3696073587 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f29d41002f0 0x7f29d4101270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:18.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.194+0000 7f29d8a7d640 1 -- 172.21.15.26:0/3696073587 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f29d41002f0 msgr2=0x7f29d4101270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:18.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.194+0000 7f29d8a7d640 1 --2- 172.21.15.26:0/3696073587 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f29d41002f0 0x7f29d4101270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:18.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.194+0000 7f29d8a7d640 1 -- 172.21.15.26:0/3696073587 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f29d4100ca0 msgr2=0x7f29d41017b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:18.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.194+0000 7f29d8a7d640 1 --2- 172.21.15.26:0/3696073587 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f29d4100ca0 0x7f29d41017b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:18.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.194+0000 7f29d8a7d640 1 -- 172.21.15.26:0/3696073587 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29d41a6920 con 0x7f29d4106f50 2024-09-11T23:27:18.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.194+0000 7f29d3fff640 1 --2- 172.21.15.26:0/3696073587 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f29d41002f0 0x7f29d4101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:27:18.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.194+0000 7f29d8a7d640 1 --2- 172.21.15.26:0/3696073587 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f29d4106f50 0x7f29d4101cf0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f29cc006010 tx=0x7f29cc003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:27:18.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.194+0000 7f29d17fa640 1 -- 172.21.15.26:0/3696073587 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29cc004450 con 0x7f29d4106f50 2024-09-11T23:27:18.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.194+0000 7f29d17fa640 1 -- 172.21.15.26:0/3696073587 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f29cc003e10 con 0x7f29d4106f50 2024-09-11T23:27:18.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.194+0000 7f29da4e0640 1 -- 172.21.15.26:0/3696073587 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29d41a6bb0 con 0x7f29d4106f50 2024-09-11T23:27:18.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.194+0000 7f29d17fa640 1 -- 172.21.15.26:0/3696073587 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29cc0466e0 con 0x7f29d4106f50 2024-09-11T23:27:18.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.194+0000 7f29da4e0640 1 -- 172.21.15.26:0/3696073587 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29d41a70f0 con 0x7f29d4106f50 2024-09-11T23:27:18.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.196+0000 7f29d17fa640 1 -- 172.21.15.26:0/3696073587 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f29cc0305a0 con 0x7f29d4106f50 2024-09-11T23:27:18.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.197+0000 7f29dc8b9740 1 Processor -- start 2024-09-11T23:27:18.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.197+0000 7f29dc8b9740 1 -- start start 2024-09-11T23:27:18.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.197+0000 7f29dc8b9740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a70b9c3760 0x55a70b9c3b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:18.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.197+0000 7f29dc8b9740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a70b9c4080 0x55a70b9ce400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:18.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.197+0000 7f29dc8b9740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a70b9ce940 0x55a70b9d0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:18.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.197+0000 7f29dc8b9740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55a70b9292a0 con 0x55a70b9c3760 2024-09-11T23:27:18.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.197+0000 7f29dc8b9740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55a70b8d0b10 con 0x55a70b9ce940 2024-09-11T23:27:18.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.197+0000 7f29dc8b9740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55a70b925260 con 0x55a70b9c4080 2024-09-11T23:27:18.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.197+0000 7f29d37fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a70b9c3760 0x55a70b9c3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:18.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.197+0000 7f29d37fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a70b9c3760 0x55a70b9c3b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36430/0 (socket says 172.21.15.26:36430) 2024-09-11T23:27:18.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.197+0000 7f29d17fa640 1 --2- 172.21.15.26:0/3696073587 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f29bc0a2cf0 0x7f29bc0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:18.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.197+0000 7f29d37fe640 1 -- 172.21.15.26:0/1582847709 learned_addr learned my addr 172.21.15.26:0/1582847709 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:27:18.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.197+0000 7f29d3fff640 1 --2- 172.21.15.26:0/1582847709 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a70b9c4080 0x55a70b9ce400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:18.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.197+0000 7f29d37fe640 1 --2- 172.21.15.26:0/1582847709 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a70b9ce940 0x55a70b9d0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:18.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.197+0000 7f29d8a7d640 1 --2- 172.21.15.26:0/3696073587 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f29bc0a2cf0 0x7f29bc0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:18.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.198+0000 7f29d37fe640 1 -- 172.21.15.26:0/1582847709 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a70b9c4080 msgr2=0x55a70b9ce400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:27:18.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.198+0000 7f29d37fe640 1 --2- 172.21.15.26:0/1582847709 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a70b9c4080 0x55a70b9ce400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:18.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.198+0000 7f29d37fe640 1 -- 172.21.15.26:0/1582847709 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a70b9ce940 msgr2=0x55a70b9d0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:27:18.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.198+0000 7f29d37fe640 1 --2- 172.21.15.26:0/1582847709 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a70b9ce940 0x55a70b9d0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:18.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.198+0000 7f29d37fe640 1 -- 172.21.15.26:0/1582847709 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a70b69ab90 con 0x55a70b9c3760 2024-09-11T23:27:18.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.198+0000 7f29d17fa640 1 -- 172.21.15.26:0/3696073587 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8586+0+0 (secure 0 0 0) 0x7f29cc02e030 con 0x7f29d4106f50 2024-09-11T23:27:18.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.198+0000 7f29d37fe640 1 --2- 172.21.15.26:0/1582847709 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a70b9c3760 0x55a70b9c3b40 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f29c800afa0 tx=0x7f29c800a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9469ec671a996e8d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:27:18.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.198+0000 7f29ba7fc640 1 -- 172.21.15.26:0/1582847709 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29c8019030 con 0x55a70b9c3760 2024-09-11T23:27:18.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.198+0000 7f29ba7fc640 1 -- 172.21.15.26:0/1582847709 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f29c8005500 con 0x55a70b9c3760 2024-09-11T23:27:18.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.198+0000 7f29d8a7d640 1 --2- 172.21.15.26:0/3696073587 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f29bc0a2cf0 0x7f29bc0a51b0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f29cc0b9a70 tx=0x7f29cc0b8c20 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:27:18.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.198+0000 7f29ba7fc640 1 -- 172.21.15.26:0/1582847709 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29c800f890 con 0x55a70b9c3760 2024-09-11T23:27:18.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.199+0000 7f29dc8b9740 1 -- 172.21.15.26:0/1582847709 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a70b9c3760 msgr2=0x55a70b9c3b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:18.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.199+0000 7f29dc8b9740 1 --2- 172.21.15.26:0/1582847709 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a70b9c3760 0x55a70b9c3b40 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f29c800afa0 tx=0x7f29c800a6a0 comp rx=0 tx=0).stop 2024-09-11T23:27:18.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.199+0000 7f29dc8b9740 1 -- 172.21.15.26:0/1582847709 shutdown_connections 2024-09-11T23:27:18.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.199+0000 7f29dc8b9740 1 --2- 172.21.15.26:0/1582847709 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a70b9ce940 0x55a70b9d0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:18.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.199+0000 7f29dc8b9740 1 --2- 172.21.15.26:0/1582847709 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a70b9c4080 0x55a70b9ce400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:18.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.199+0000 7f29dc8b9740 1 --2- 172.21.15.26:0/1582847709 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a70b9c3760 0x55a70b9c3b40 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:18.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.199+0000 7f29dc8b9740 1 -- 172.21.15.26:0/1582847709 >> 172.21.15.26:0/1582847709 conn(0x55a70b9bfcf0 msgr2=0x55a70b9c00d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:27:18.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.199+0000 7f29dc8b9740 1 -- 172.21.15.26:0/1582847709 shutdown_connections 2024-09-11T23:27:18.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.199+0000 7f29dc8b9740 1 -- 172.21.15.26:0/1582847709 wait complete. 2024-09-11T23:27:18.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.199+0000 7f29dc8b9740 1 -- start start 2024-09-11T23:27:18.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.200+0000 7f29dc8b9740 1 Processor -- start 2024-09-11T23:27:18.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.200+0000 7f29dc8b9740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a70b9c3760 0x55a70b9cb7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:18.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.200+0000 7f29dc8b9740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a70b9c4080 0x55a70b9cbcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:18.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.200+0000 7f29dc8b9740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a70b9ce940 0x55a70b9cc230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:18.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.200+0000 7f29dc8b9740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55a70b9d1510 con 0x55a70b9ce940 2024-09-11T23:27:18.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.200+0000 7f29dc8b9740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55a70b8b16e0 con 0x55a70b9c4080 2024-09-11T23:27:18.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.200+0000 7f29dc8b9740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55a70b9d1690 con 0x55a70b9c3760 2024-09-11T23:27:18.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.200+0000 7f29dc8b9740 10 client.0 _ll_register_callbacks cb 0x55a70b8d5a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:27:18.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.200+0000 7f29d37fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a70b9ce940 0x55a70b9cc230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:18.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.200+0000 7f29d37fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a70b9ce940 0x55a70b9cc230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36436/0 (socket says 172.21.15.26:36436) 2024-09-11T23:27:18.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.200+0000 7f29d37fe640 1 -- 172.21.15.26:0/2838427948 learned_addr learned my addr 172.21.15.26:0/2838427948 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:27:18.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.200+0000 7f29d3fff640 1 --2- 172.21.15.26:0/2838427948 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a70b9c4080 0x55a70b9cbcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:18.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.200+0000 7f29d37fe640 1 --2- 172.21.15.26:0/2838427948 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a70b9c3760 0x55a70b9cb7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:18.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.201+0000 7f29d3fff640 1 -- 172.21.15.26:0/2838427948 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a70b9c3760 msgr2=0x55a70b9cb7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:27:18.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.201+0000 7f29d3fff640 1 --2- 172.21.15.26:0/2838427948 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a70b9c3760 0x55a70b9cb7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:18.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.201+0000 7f29d3fff640 1 -- 172.21.15.26:0/2838427948 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a70b9ce940 msgr2=0x55a70b9cc230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:27:18.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.201+0000 7f29d3fff640 1 --2- 172.21.15.26:0/2838427948 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a70b9ce940 0x55a70b9cc230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:18.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.201+0000 7f29d3fff640 1 -- 172.21.15.26:0/2838427948 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a70b9da9b0 con 0x55a70b9c4080 2024-09-11T23:27:18.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.201+0000 7f29d3fff640 1 --2- 172.21.15.26:0/2838427948 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a70b9c4080 0x55a70b9cbcf0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f29c4010790 tx=0x7f29c4010c50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c9b5c5d252c90418 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:27:18.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.201+0000 7f29b37fe640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-11T23:27:18.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.201+0000 7f29b37fe640 1 -- 172.21.15.26:0/2838427948 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29c4010ea0 con 0x55a70b9c4080 2024-09-11T23:27:18.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.201+0000 7f29dc8b9740 1 -- 172.21.15.26:0/2838427948 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f29c8005500 con 0x55a70b9c4080 2024-09-11T23:27:18.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.201+0000 7f29b37fe640 1 -- 172.21.15.26:0/2838427948 <== mon.2 v2:172.21.15.76:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8586+0+0 (secure 0 0 0) 0x7f29c400ed40 con 0x55a70b9c4080 2024-09-11T23:27:18.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.202+0000 7f29b37fe640 1 -- 172.21.15.26:0/2838427948 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f29c4004020 con 0x55a70b9c4080 2024-09-11T23:27:18.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.202+0000 7f29dc8b9740 10 client.26393 fetch_fsmap learned FSMap version 51 2024-09-11T23:27:18.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.202+0000 7f29dc8b9740 1 -- 172.21.15.26:0/2838427948 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x55a70b9dd3a0 con 0x55a70b9c4080 2024-09-11T23:27:18.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.202+0000 7f29b37fe640 1 -- 172.21.15.26:0/2838427948 <== mon.2 v2:172.21.15.76:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f29c8005500 con 0x55a70b9c4080 2024-09-11T23:27:18.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.202+0000 7f29b37fe640 1 -- 172.21.15.26:0/2838427948 <== mon.2 v2:172.21.15.76:3300/0 5 ==== fsmap(e 51) ==== 8394+0+0 (secure 0 0 0) 0x7f29c4011440 con 0x55a70b9c4080 2024-09-11T23:27:18.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.202+0000 7f29dc8b9740 10 client.26393 fetch_fsmap finished waiting for FSMap version 51 2024-09-11T23:27:18.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.202+0000 7f29dc8b9740 10 client.26393 resolve_mds: validated gid 24485 aka daemon mds.b 2024-09-11T23:27:18.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.202+0000 7f29dc8b9740 20 client.26393 populate_metadata read hostname 'smithi026' 2024-09-11T23:27:18.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.202+0000 7f29dc8b9740 1 --2- 172.21.15.26:0/2838427948 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55a70b9de900 0x55a70b9e0d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:18.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.202+0000 7f29dc8b9740 4 client.26393 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:27:18.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.202+0000 7f29dc8b9740 1 -- 172.21.15.26:0/2838427948 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a70b9369d0 con 0x55a70b9de900 2024-09-11T23:27:18.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.202+0000 7f29d37fe640 1 --2- 172.21.15.26:0/2838427948 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55a70b9de900 0x55a70b9e0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:18.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.203+0000 7f29d37fe640 1 --2- 172.21.15.26:0/2838427948 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55a70b9de900 0x55a70b9e0d20 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f0910161bdc98ae server_cookie=2c28ad3a9467b301 in_seq=0 out_seq=0 2024-09-11T23:27:18.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.203+0000 7f29b37fe640 10 client.26393 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:27:18.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.204+0000 7f29b37fe640 1 -- 172.21.15.26:0/2838427948 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a70b9369d0 con 0x55a70b9de900 2024-09-11T23:27:18.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.204+0000 7f29b37fe640 10 client.26393 handle_command_reply: tid=0 2024-09-11T23:27:18.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.204+0000 7f29dc8b9740 1 client.26393 shutdown 2024-09-11T23:27:18.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.204+0000 7f29dc8b9740 1 -- 172.21.15.26:0/2838427948 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a70b9c4080 msgr2=0x55a70b9cbcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:18.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.204+0000 7f29dc8b9740 1 --2- 172.21.15.26:0/2838427948 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a70b9c4080 0x55a70b9cbcf0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f29c4010790 tx=0x7f29c4010c50 comp rx=0 tx=0).stop 2024-09-11T23:27:18.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.204+0000 7f29dc8b9740 1 -- 172.21.15.26:0/2838427948 shutdown_connections 2024-09-11T23:27:18.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.204+0000 7f29dc8b9740 1 --2- 172.21.15.26:0/2838427948 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a70b9c4080 0x55a70b9cbcf0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:18.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.204+0000 7f29dc8b9740 1 --2- 172.21.15.26:0/2838427948 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55a70b9de900 0x55a70b9e0d20 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:18.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.204+0000 7f29dc8b9740 1 --2- 172.21.15.26:0/2838427948 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a70b9ce940 0x55a70b9cc230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:18.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.204+0000 7f29dc8b9740 1 --2- 172.21.15.26:0/2838427948 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a70b9c3760 0x55a70b9cb7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:18.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.204+0000 7f29b37fe640 0 client.26393 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:27:18.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.204+0000 7f29dc8b9740 1 -- 172.21.15.26:0/2838427948 >> 172.21.15.26:0/2838427948 conn(0x55a70b9bfcf0 msgr2=0x55a70b9c59d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:27:18.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.204+0000 7f29dc8b9740 1 -- 172.21.15.26:0/2838427948 shutdown_connections 2024-09-11T23:27:18.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.204+0000 7f29dc8b9740 1 -- 172.21.15.26:0/2838427948 wait complete. 2024-09-11T23:27:18.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.204+0000 7f29dc8b9740 20 client.26393 trim_cache size 0 max 16384 2024-09-11T23:27:18.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.221+0000 7f29dc8b9740 1 Processor -- start 2024-09-11T23:27:18.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.221+0000 7f29dc8b9740 1 -- start start 2024-09-11T23:27:18.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.221+0000 7f29dc8b9740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a70b9c3760 0x55a70b9e65b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:18.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.221+0000 7f29dc8b9740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a70b9c4080 0x55a70b9e6af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:18.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.221+0000 7f29dc8b9740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a70b9ce940 0x55a70b9c8a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:18.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.221+0000 7f29dc8b9740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f29d41afb70 con 0x55a70b9c4080 2024-09-11T23:27:18.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.221+0000 7f29dc8b9740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f29d41b1830 con 0x55a70b9c3760 2024-09-11T23:27:18.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.221+0000 7f29dc8b9740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f29d41b0240 con 0x55a70b9ce940 2024-09-11T23:27:18.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.221+0000 7f29d3fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a70b9c4080 0x55a70b9e6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:18.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.221+0000 7f29d3fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a70b9c4080 0x55a70b9e6af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36440/0 (socket says 172.21.15.26:36440) 2024-09-11T23:27:18.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.221+0000 7f29d3fff640 1 -- 172.21.15.26:0/2127535646 learned_addr learned my addr 172.21.15.26:0/2127535646 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:27:18.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.222+0000 7f29d37fe640 1 --2- 172.21.15.26:0/2127535646 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a70b9ce940 0x55a70b9c8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:18.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.222+0000 7f29d3fff640 1 -- 172.21.15.26:0/2127535646 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a70b9ce940 msgr2=0x55a70b9c8a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:27:18.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.222+0000 7f29d37fe640 1 --2- 172.21.15.26:0/2127535646 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a70b9c3760 0x55a70b9e65b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:18.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.222+0000 7f29d3fff640 1 --2- 172.21.15.26:0/2127535646 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a70b9ce940 0x55a70b9c8a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:18.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.222+0000 7f29d3fff640 1 -- 172.21.15.26:0/2127535646 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a70b9c3760 msgr2=0x55a70b9e65b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:27:18.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.222+0000 7f29d3fff640 1 --2- 172.21.15.26:0/2127535646 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a70b9c3760 0x55a70b9e65b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:18.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.222+0000 7f29d3fff640 1 -- 172.21.15.26:0/2127535646 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a70b9bfb20 con 0x55a70b9c4080 2024-09-11T23:27:18.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.222+0000 7f29d37fe640 1 --2- 172.21.15.26:0/2127535646 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a70b9c3760 0x55a70b9e65b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:27:18.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.222+0000 7f29d3fff640 1 --2- 172.21.15.26:0/2127535646 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a70b9c4080 0x55a70b9e6af0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f29c4011900 tx=0x7f29c400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bea2958bcd88af34 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:27:18.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.222+0000 7f29b9ffb640 1 -- 172.21.15.26:0/2127535646 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29c4030070 con 0x55a70b9c4080 2024-09-11T23:27:18.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.222+0000 7f29b9ffb640 1 -- 172.21.15.26:0/2127535646 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f29c4011cf0 con 0x55a70b9c4080 2024-09-11T23:27:18.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.222+0000 7f29b9ffb640 1 -- 172.21.15.26:0/2127535646 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29c4027040 con 0x55a70b9c4080 2024-09-11T23:27:18.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.223+0000 7f29dc8b9740 1 -- 172.21.15.26:0/2127535646 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a70b9c4080 msgr2=0x55a70b9e6af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:18.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.223+0000 7f29dc8b9740 1 --2- 172.21.15.26:0/2127535646 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a70b9c4080 0x55a70b9e6af0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f29c4011900 tx=0x7f29c400cf60 comp rx=0 tx=0).stop 2024-09-11T23:27:18.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.223+0000 7f29dc8b9740 1 -- 172.21.15.26:0/2127535646 shutdown_connections 2024-09-11T23:27:18.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.223+0000 7f29dc8b9740 1 --2- 172.21.15.26:0/2127535646 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a70b9ce940 0x55a70b9c8a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:18.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.223+0000 7f29dc8b9740 1 --2- 172.21.15.26:0/2127535646 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a70b9c4080 0x55a70b9e6af0 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:18.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.223+0000 7f29dc8b9740 1 --2- 172.21.15.26:0/2127535646 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a70b9c3760 0x55a70b9e65b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:18.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.223+0000 7f29dc8b9740 1 -- 172.21.15.26:0/2127535646 >> 172.21.15.26:0/2127535646 conn(0x55a70b9bfcf0 msgr2=0x55a70b9cd830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:27:18.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.223+0000 7f29dc8b9740 1 -- 172.21.15.26:0/2127535646 shutdown_connections 2024-09-11T23:27:18.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.223+0000 7f29dc8b9740 1 -- 172.21.15.26:0/2127535646 wait complete. 2024-09-11T23:27:18.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.223+0000 7f29dc8b9740 1 -- start start 2024-09-11T23:27:18.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.223+0000 7f29dc8b9740 1 Processor -- start 2024-09-11T23:27:18.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.224+0000 7f29dc8b9740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a70b9c3760 0x55a70b9d6d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:18.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.224+0000 7f29dc8b9740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a70b9c4080 0x55a70b9d30a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:18.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.224+0000 7f29dc8b9740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a70b9ce940 0x55a70b9d35e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:18.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.224+0000 7f29dc8b9740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f29d41b1160 con 0x55a70b9c3760 2024-09-11T23:27:18.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.224+0000 7f29dc8b9740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f29d41b09d0 con 0x55a70b9c4080 2024-09-11T23:27:18.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.224+0000 7f29dc8b9740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f29d41adb70 con 0x55a70b9ce940 2024-09-11T23:27:18.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.224+0000 7f29dc8b9740 10 client.0 _ll_register_callbacks cb 0x7f29c40103d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:27:18.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.224+0000 7f29d37fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a70b9c3760 0x55a70b9d6d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:18.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.224+0000 7f29d37fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a70b9c3760 0x55a70b9d6d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36446/0 (socket says 172.21.15.26:36446) 2024-09-11T23:27:18.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.224+0000 7f29d37fe640 1 -- 172.21.15.26:0/1645053041 learned_addr learned my addr 172.21.15.26:0/1645053041 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:27:18.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.224+0000 7f29d37fe640 1 --2- 172.21.15.26:0/1645053041 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a70b9ce940 0x55a70b9d35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:18.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.224+0000 7f29d3fff640 1 --2- 172.21.15.26:0/1645053041 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a70b9c4080 0x55a70b9d30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:18.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.224+0000 7f29d37fe640 1 -- 172.21.15.26:0/1645053041 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a70b9ce940 msgr2=0x55a70b9d35e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:27:18.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.224+0000 7f29d37fe640 1 --2- 172.21.15.26:0/1645053041 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a70b9ce940 0x55a70b9d35e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:18.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.224+0000 7f29d37fe640 1 -- 172.21.15.26:0/1645053041 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a70b9c4080 msgr2=0x55a70b9d30a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:27:18.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.224+0000 7f29d37fe640 1 --2- 172.21.15.26:0/1645053041 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a70b9c4080 0x55a70b9d30a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:18.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.224+0000 7f29d37fe640 1 -- 172.21.15.26:0/1645053041 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a70b9c65d0 con 0x55a70b9c3760 2024-09-11T23:27:18.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.224+0000 7f29d37fe640 1 --2- 172.21.15.26:0/1645053041 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a70b9c3760 0x55a70b9d6d10 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f29c8014fd0 tx=0x7f29c800a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=523938140c3dccb8 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:27:18.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.224+0000 7f29b37fe640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:27:18.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.225+0000 7f29b37fe640 1 -- 172.21.15.26:0/1645053041 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29c8032030 con 0x55a70b9c3760 2024-09-11T23:27:18.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.225+0000 7f29dc8b9740 1 -- 172.21.15.26:0/1645053041 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f29c4011cf0 con 0x55a70b9c3760 2024-09-11T23:27:18.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.225+0000 7f29b37fe640 1 -- 172.21.15.26:0/1645053041 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8586+0+0 (secure 0 0 0) 0x7f29c8028040 con 0x55a70b9c3760 2024-09-11T23:27:18.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.225+0000 7f29b37fe640 1 -- 172.21.15.26:0/1645053041 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f29c8019040 con 0x55a70b9c3760 2024-09-11T23:27:18.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.225+0000 7f29dc8b9740 10 client.16833 fetch_fsmap learned FSMap version 51 2024-09-11T23:27:18.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.225+0000 7f29dc8b9740 1 -- 172.21.15.26:0/1645053041 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x55a70b9bb260 con 0x55a70b9c3760 2024-09-11T23:27:18.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.225+0000 7f29b37fe640 1 -- 172.21.15.26:0/1645053041 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f29c4011cf0 con 0x55a70b9c3760 2024-09-11T23:27:18.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.225+0000 7f29b37fe640 1 -- 172.21.15.26:0/1645053041 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 51) ==== 8394+0+0 (secure 0 0 0) 0x7f29c8004f10 con 0x55a70b9c3760 2024-09-11T23:27:18.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.225+0000 7f29dc8b9740 10 client.16833 fetch_fsmap finished waiting for FSMap version 51 2024-09-11T23:27:18.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.225+0000 7f29dc8b9740 10 client.16833 resolve_mds: validated gid 24485 aka daemon mds.b 2024-09-11T23:27:18.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.225+0000 7f29dc8b9740 20 client.16833 populate_metadata read hostname 'smithi026' 2024-09-11T23:27:18.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.225+0000 7f29dc8b9740 1 --2- 172.21.15.26:0/1645053041 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55a70ba4a2b0 0x55a70ba4c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:18.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.225+0000 7f29dc8b9740 4 client.16833 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 139.4, "await_for": 45.0}] 2024-09-11T23:27:18.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.225+0000 7f29dc8b9740 1 -- 172.21.15.26:0/1645053041 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 139.4, "await_for": 45.0}) -- 0x55a70b930d20 con 0x55a70ba4a2b0 2024-09-11T23:27:18.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.226+0000 7f29d3fff640 1 --2- 172.21.15.26:0/1645053041 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55a70ba4a2b0 0x55a70ba4c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:18.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.226+0000 7f29d3fff640 1 --2- 172.21.15.26:0/1645053041 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55a70ba4a2b0 0x55a70ba4c6a0 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce490515feb91e72 server_cookie=49c531233b266f62 in_seq=0 out_seq=0 2024-09-11T23:27:18.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:18.226+0000 7f29b37fe640 10 client.16833 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:27:18.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:27:17 smithi136 ceph-mon[35121]: pgmap v275: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 2.6 KiB/s wr, 12 op/s 2024-09-11T23:27:18.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:27:17 smithi076 ceph-mon[35296]: pgmap v275: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 2.6 KiB/s wr, 12 op/s 2024-09-11T23:27:19.699 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.699+0000 7fa97e559640 1 -- 172.21.15.26:0/2785875163 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa97810c110 msgr2=0x7fa97810e590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:19.699 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.699+0000 7fa97e559640 1 --2- 172.21.15.26:0/2785875163 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa97810c110 0x7fa97810e590 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fa970009ca0 tx=0x7fa97002a950 comp rx=0 tx=0).stop 2024-09-11T23:27:19.700 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.699+0000 7fa97e559640 1 -- 172.21.15.26:0/2785875163 shutdown_connections 2024-09-11T23:27:19.700 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.699+0000 7fa97e559640 1 --2- 172.21.15.26:0/2785875163 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa9781035e0 0x7fa97810bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:19.700 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.699+0000 7fa97e559640 1 --2- 172.21.15.26:0/2785875163 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa97810c110 0x7fa97810e590 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:19.700 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.699+0000 7fa97e559640 1 --2- 172.21.15.26:0/2785875163 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa978102cc0 0x7fa9781030a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:19.700 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.699+0000 7fa97e559640 1 -- 172.21.15.26:0/2785875163 >> 172.21.15.26:0/2785875163 conn(0x7fa9780fc6b0 msgr2=0x7fa9780fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:27:19.700 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.699+0000 7fa97e559640 1 -- 172.21.15.26:0/2785875163 shutdown_connections 2024-09-11T23:27:19.700 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.699+0000 7fa97e559640 1 -- 172.21.15.26:0/2785875163 wait complete. 2024-09-11T23:27:19.700 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.700+0000 7fa97e559640 1 Processor -- start 2024-09-11T23:27:19.700 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.700+0000 7fa97e559640 1 -- start start 2024-09-11T23:27:19.702 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.700+0000 7fa97e559640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa978102cc0 0x7fa978100910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:19.702 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.700+0000 7fa97e559640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa9781035e0 0x7fa978100e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:19.702 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.700+0000 7fa97e559640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa97810c110 0x7fa978101390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:19.702 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.700+0000 7fa97e559640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fa978111ef0 con 0x7fa97810c110 2024-09-11T23:27:19.703 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.700+0000 7fa97e559640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fa978111d70 con 0x7fa978102cc0 2024-09-11T23:27:19.703 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.700+0000 7fa97e559640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fa978112070 con 0x7fa9781035e0 2024-09-11T23:27:19.703 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.700+0000 7fa97caf6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa97810c110 0x7fa978101390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:19.703 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.700+0000 7fa977fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa978102cc0 0x7fa978100910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:19.703 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.700+0000 7fa97caf6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa97810c110 0x7fa978101390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36466/0 (socket says 172.21.15.26:36466) 2024-09-11T23:27:19.703 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.700+0000 7fa9777fe640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa9781035e0 0x7fa978100e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:19.703 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.700+0000 7fa97caf6640 1 -- 172.21.15.26:0/2231266129 learned_addr learned my addr 172.21.15.26:0/2231266129 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:27:19.703 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.701+0000 7fa97caf6640 1 -- 172.21.15.26:0/2231266129 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa9781035e0 msgr2=0x7fa978100e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:19.703 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.701+0000 7fa97caf6640 1 --2- 172.21.15.26:0/2231266129 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa9781035e0 0x7fa978100e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:19.703 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.701+0000 7fa97caf6640 1 -- 172.21.15.26:0/2231266129 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa978102cc0 msgr2=0x7fa978100910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:19.703 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.701+0000 7fa97caf6640 1 --2- 172.21.15.26:0/2231266129 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa978102cc0 0x7fa978100910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:19.703 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.701+0000 7fa97caf6640 1 -- 172.21.15.26:0/2231266129 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9781a67f0 con 0x7fa97810c110 2024-09-11T23:27:19.703 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.701+0000 7fa977fff640 1 --2- 172.21.15.26:0/2231266129 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa978102cc0 0x7fa978100910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:27:19.704 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.701+0000 7fa97caf6640 1 --2- 172.21.15.26:0/2231266129 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa97810c110 0x7fa978101390 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fa970004af0 tx=0x7fa97002f3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:27:19.704 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.701+0000 7fa9757fa640 1 -- 172.21.15.26:0/2231266129 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa970034040 con 0x7fa97810c110 2024-09-11T23:27:19.704 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.701+0000 7fa9757fa640 1 -- 172.21.15.26:0/2231266129 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa970030920 con 0x7fa97810c110 2024-09-11T23:27:19.704 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.701+0000 7fa97e559640 1 -- 172.21.15.26:0/2231266129 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9781a6a80 con 0x7fa97810c110 2024-09-11T23:27:19.704 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.701+0000 7fa9757fa640 1 -- 172.21.15.26:0/2231266129 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa970004670 con 0x7fa97810c110 2024-09-11T23:27:19.704 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.701+0000 7fa97e559640 1 -- 172.21.15.26:0/2231266129 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9781a6fc0 con 0x7fa97810c110 2024-09-11T23:27:19.710 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.704+0000 7fa9757fa640 1 -- 172.21.15.26:0/2231266129 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fa970037020 con 0x7fa97810c110 2024-09-11T23:27:19.710 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.704+0000 7fa9813d3740 1 Processor -- start 2024-09-11T23:27:19.711 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.704+0000 7fa9813d3740 1 -- start start 2024-09-11T23:27:19.711 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.704+0000 7fa9813d3740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564556e1baf0 0x564556e1bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:19.711 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.704+0000 7fa9813d3740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564556e1c410 0x564556e26770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:19.711 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.704+0000 7fa9813d3740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564556e26cb0 0x564556e290a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:19.711 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.704+0000 7fa9813d3740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x564556d802f0 con 0x564556e1baf0 2024-09-11T23:27:19.711 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.704+0000 7fa9813d3740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x564556d28270 con 0x564556e1c410 2024-09-11T23:27:19.711 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.704+0000 7fa9813d3740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x564556d7c2b0 con 0x564556e26cb0 2024-09-11T23:27:19.711 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.704+0000 7fa9777fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564556e1baf0 0x564556e1bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:19.711 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.704+0000 7fa977fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564556e1c410 0x564556e26770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:19.711 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.704+0000 7fa9777fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564556e1baf0 0x564556e1bed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36468/0 (socket says 172.21.15.26:36468) 2024-09-11T23:27:19.711 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.704+0000 7fa9777fe640 1 -- 172.21.15.26:0/1687494662 learned_addr learned my addr 172.21.15.26:0/1687494662 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:27:19.712 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.704+0000 7fa9777fe640 1 --2- 172.21.15.26:0/1687494662 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564556e26cb0 0x564556e290a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:19.712 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.704+0000 7fa9757fa640 1 --2- 172.21.15.26:0/2231266129 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fa9600a2e20 0x7fa9600a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:19.712 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.704+0000 7fa977fff640 1 -- 172.21.15.26:0/1687494662 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564556e26cb0 msgr2=0x564556e290a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:27:19.712 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.704+0000 7fa977fff640 1 --2- 172.21.15.26:0/1687494662 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564556e26cb0 0x564556e290a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:19.712 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.704+0000 7fa977fff640 1 -- 172.21.15.26:0/1687494662 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564556e1baf0 msgr2=0x564556e1bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:27:19.712 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.704+0000 7fa977fff640 1 --2- 172.21.15.26:0/1687494662 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564556e1baf0 0x564556e1bed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:19.712 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.705+0000 7fa977fff640 1 -- 172.21.15.26:0/1687494662 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564556d27340 con 0x564556e1c410 2024-09-11T23:27:19.712 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.705+0000 7fa9777fe640 1 --2- 172.21.15.26:0/1687494662 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564556e26cb0 0x564556e290a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:27:19.712 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.705+0000 7fa97caf6640 1 --2- 172.21.15.26:0/2231266129 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fa9600a2e20 0x7fa9600a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:19.712 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.705+0000 7fa9757fa640 1 -- 172.21.15.26:0/2231266129 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8586+0+0 (secure 0 0 0) 0x7fa9700f0000 con 0x7fa97810c110 2024-09-11T23:27:19.712 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.705+0000 7fa977fff640 1 --2- 172.21.15.26:0/1687494662 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564556e1c410 0x564556e26770 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fa968009fd0 tx=0x7fa9680118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=97bb9c805d5365d4 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:27:19.712 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.705+0000 7fa95e7fc640 1 -- 172.21.15.26:0/1687494662 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa968004310 con 0x564556e1c410 2024-09-11T23:27:19.713 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.705+0000 7fa95e7fc640 1 -- 172.21.15.26:0/1687494662 <== mon.2 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9680044b0 con 0x564556e1c410 2024-09-11T23:27:19.713 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.705+0000 7fa95e7fc640 1 -- 172.21.15.26:0/1687494662 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9680035f0 con 0x564556e1c410 2024-09-11T23:27:19.713 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.706+0000 7fa9813d3740 1 -- 172.21.15.26:0/1687494662 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564556e1c410 msgr2=0x564556e26770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:19.713 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.706+0000 7fa97caf6640 1 --2- 172.21.15.26:0/2231266129 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fa9600a2e20 0x7fa9600a52e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fa9700b8fc0 tx=0x7fa9700a7120 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:27:19.713 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.706+0000 7fa9813d3740 1 --2- 172.21.15.26:0/1687494662 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564556e1c410 0x564556e26770 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fa968009fd0 tx=0x7fa9680118d0 comp rx=0 tx=0).stop 2024-09-11T23:27:19.713 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.706+0000 7fa9813d3740 1 -- 172.21.15.26:0/1687494662 shutdown_connections 2024-09-11T23:27:19.713 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.706+0000 7fa9813d3740 1 --2- 172.21.15.26:0/1687494662 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564556e1c410 0x564556e26770 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:19.713 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.706+0000 7fa9813d3740 1 --2- 172.21.15.26:0/1687494662 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564556e26cb0 0x564556e290a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:19.713 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.706+0000 7fa9813d3740 1 --2- 172.21.15.26:0/1687494662 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564556e1baf0 0x564556e1bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:19.713 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.706+0000 7fa9813d3740 1 -- 172.21.15.26:0/1687494662 >> 172.21.15.26:0/1687494662 conn(0x564556e18060 msgr2=0x564556e18440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:27:19.713 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.706+0000 7fa9813d3740 1 -- 172.21.15.26:0/1687494662 shutdown_connections 2024-09-11T23:27:19.713 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.706+0000 7fa9813d3740 1 -- 172.21.15.26:0/1687494662 wait complete. 2024-09-11T23:27:19.714 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.706+0000 7fa9813d3740 1 -- start start 2024-09-11T23:27:19.714 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.706+0000 7fa9813d3740 1 Processor -- start 2024-09-11T23:27:19.714 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.707+0000 7fa9813d3740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564556e1baf0 0x564556e23ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:19.714 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.707+0000 7fa9813d3740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564556e1c410 0x564556e24010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:19.714 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.707+0000 7fa9813d3740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564556e26cb0 0x564556e24550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:19.714 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.707+0000 7fa9813d3740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x564556e29880 con 0x564556e1baf0 2024-09-11T23:27:19.714 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.707+0000 7fa9813d3740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x564556d09290 con 0x564556e1c410 2024-09-11T23:27:19.714 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.707+0000 7fa9813d3740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x564556e29a00 con 0x564556e26cb0 2024-09-11T23:27:19.714 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.707+0000 7fa9813d3740 10 client.0 _ll_register_callbacks cb 0x564556d3a140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:27:19.714 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.707+0000 7fa9777fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564556e1baf0 0x564556e23ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:19.714 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.707+0000 7fa977fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564556e1c410 0x564556e24010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:19.714 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.707+0000 7fa9777fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564556e1baf0 0x564556e23ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36482/0 (socket says 172.21.15.26:36482) 2024-09-11T23:27:19.715 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.707+0000 7fa9777fe640 1 -- 172.21.15.26:0/3207634991 learned_addr learned my addr 172.21.15.26:0/3207634991 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:27:19.715 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.707+0000 7fa9777fe640 1 --2- 172.21.15.26:0/3207634991 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564556e26cb0 0x564556e24550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:19.715 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.707+0000 7fa977fff640 1 -- 172.21.15.26:0/3207634991 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564556e26cb0 msgr2=0x564556e24550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:27:19.715 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.707+0000 7fa977fff640 1 --2- 172.21.15.26:0/3207634991 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564556e26cb0 0x564556e24550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:19.715 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.707+0000 7fa977fff640 1 -- 172.21.15.26:0/3207634991 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564556e1baf0 msgr2=0x564556e23ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:27:19.715 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.707+0000 7fa977fff640 1 --2- 172.21.15.26:0/3207634991 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564556e1baf0 0x564556e23ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:19.715 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.707+0000 7fa977fff640 1 -- 172.21.15.26:0/3207634991 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564556a93210 con 0x564556e1c410 2024-09-11T23:27:19.715 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.707+0000 7fa9777fe640 1 --2- 172.21.15.26:0/3207634991 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564556e1baf0 0x564556e23ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:27:19.715 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.708+0000 7fa977fff640 1 --2- 172.21.15.26:0/3207634991 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564556e1c410 0x564556e24010 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fa96801b040 tx=0x7fa9680049e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a2bd98060f9ecbb server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:27:19.715 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.708+0000 7fa9577fe640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-11T23:27:19.715 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.708+0000 7fa9577fe640 1 -- 172.21.15.26:0/3207634991 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa968029070 con 0x564556e1c410 2024-09-11T23:27:19.715 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.708+0000 7fa9577fe640 1 -- 172.21.15.26:0/3207634991 <== mon.2 v2:172.21.15.76:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8586+0+0 (secure 0 0 0) 0x7fa9680052c0 con 0x564556e1c410 2024-09-11T23:27:19.715 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.708+0000 7fa9813d3740 1 -- 172.21.15.26:0/3207634991 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9680044b0 con 0x564556e1c410 2024-09-11T23:27:19.716 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.708+0000 7fa9577fe640 1 -- 172.21.15.26:0/3207634991 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7fa968035050 con 0x564556e1c410 2024-09-11T23:27:19.716 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.708+0000 7fa9813d3740 10 client.26408 fetch_fsmap learned FSMap version 51 2024-09-11T23:27:19.716 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.708+0000 7fa9813d3740 1 -- 172.21.15.26:0/3207634991 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x564556b5f620 con 0x564556e1c410 2024-09-11T23:27:19.716 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.708+0000 7fa9577fe640 1 -- 172.21.15.26:0/3207634991 <== mon.2 v2:172.21.15.76:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9680044b0 con 0x564556e1c410 2024-09-11T23:27:19.716 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.708+0000 7fa9577fe640 1 -- 172.21.15.26:0/3207634991 <== mon.2 v2:172.21.15.76:3300/0 5 ==== fsmap(e 51) ==== 8394+0+0 (secure 0 0 0) 0x7fa96800f570 con 0x564556e1c410 2024-09-11T23:27:19.716 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.708+0000 7fa9813d3740 10 client.26408 fetch_fsmap finished waiting for FSMap version 51 2024-09-11T23:27:19.716 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.708+0000 7fa9813d3740 10 client.26408 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:27:19.716 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.708+0000 7fa9813d3740 20 client.26408 populate_metadata read hostname 'smithi026' 2024-09-11T23:27:19.716 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.708+0000 7fa9813d3740 1 --2- 172.21.15.26:0/3207634991 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x564556e36a90 0x564556e38e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:19.716 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.708+0000 7fa9813d3740 4 client.26408 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:27:19.716 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.708+0000 7fa9813d3740 1 -- 172.21.15.26:0/3207634991 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564556d8da20 con 0x564556e36a90 2024-09-11T23:27:19.716 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.709+0000 7fa9777fe640 1 --2- 172.21.15.26:0/3207634991 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x564556e36a90 0x564556e38e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:19.716 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.709+0000 7fa9577fe640 10 client.26408 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:27:19.717 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.709+0000 7fa9777fe640 1 --2- 172.21.15.26:0/3207634991 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x564556e36a90 0x564556e38e80 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5cba8dd64482e7a server_cookie=7e5c44bfbddbe95c in_seq=0 out_seq=0 2024-09-11T23:27:19.717 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.710+0000 7fa9577fe640 1 -- 172.21.15.26:0/3207634991 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564556d8da20 con 0x564556e36a90 2024-09-11T23:27:19.717 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.710+0000 7fa9577fe640 10 client.26408 handle_command_reply: tid=0 2024-09-11T23:27:19.717 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.710+0000 7fa9813d3740 1 client.26408 shutdown 2024-09-11T23:27:19.717 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.710+0000 7fa9813d3740 1 -- 172.21.15.26:0/3207634991 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564556e1c410 msgr2=0x564556e24010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:19.717 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.710+0000 7fa9813d3740 1 --2- 172.21.15.26:0/3207634991 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564556e1c410 0x564556e24010 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fa96801b040 tx=0x7fa9680049e0 comp rx=0 tx=0).stop 2024-09-11T23:27:19.717 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.710+0000 7fa9813d3740 1 -- 172.21.15.26:0/3207634991 shutdown_connections 2024-09-11T23:27:19.717 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.710+0000 7fa9813d3740 1 --2- 172.21.15.26:0/3207634991 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564556e1c410 0x564556e24010 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:19.717 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.710+0000 7fa9813d3740 1 --2- 172.21.15.26:0/3207634991 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x564556e36a90 0x564556e38e80 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:19.717 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.710+0000 7fa9813d3740 1 --2- 172.21.15.26:0/3207634991 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564556e26cb0 0x564556e24550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:19.717 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.710+0000 7fa9813d3740 1 --2- 172.21.15.26:0/3207634991 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564556e1baf0 0x564556e23ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:19.717 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.710+0000 7fa9813d3740 1 -- 172.21.15.26:0/3207634991 >> 172.21.15.26:0/3207634991 conn(0x564556e18060 msgr2=0x564556e1ddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:27:19.717 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.710+0000 7fa9577fe640 0 client.26408 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:27:19.718 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.710+0000 7fa9813d3740 1 -- 172.21.15.26:0/3207634991 shutdown_connections 2024-09-11T23:27:19.718 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.710+0000 7fa9813d3740 1 -- 172.21.15.26:0/3207634991 wait complete. 2024-09-11T23:27:19.718 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.710+0000 7fa9813d3740 20 client.26408 trim_cache size 0 max 16384 2024-09-11T23:27:19.726 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.725+0000 7fa9813d3740 1 Processor -- start 2024-09-11T23:27:19.727 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.725+0000 7fa9813d3740 1 -- start start 2024-09-11T23:27:19.727 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.725+0000 7fa9813d3740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564556e1baf0 0x564556e24380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:19.727 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.725+0000 7fa9813d3740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564556e1c410 0x564556e3e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:19.727 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.725+0000 7fa9813d3740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564556e26cb0 0x564556e3eb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:19.727 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.725+0000 7fa9813d3740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fa9781af9e0 con 0x564556e1c410 2024-09-11T23:27:19.727 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.725+0000 7fa9813d3740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fa9781b16a0 con 0x564556e26cb0 2024-09-11T23:27:19.727 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.725+0000 7fa9813d3740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fa9781b00b0 con 0x564556e1baf0 2024-09-11T23:27:19.727 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.725+0000 7fa977fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564556e1c410 0x564556e3e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:19.728 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.725+0000 7fa977fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564556e1c410 0x564556e3e5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36496/0 (socket says 172.21.15.26:36496) 2024-09-11T23:27:19.728 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.725+0000 7fa977fff640 1 -- 172.21.15.26:0/1270884138 learned_addr learned my addr 172.21.15.26:0/1270884138 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:27:19.728 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.725+0000 7fa9777fe640 1 --2- 172.21.15.26:0/1270884138 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564556e1baf0 0x564556e24380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:19.728 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.725+0000 7fa9777fe640 1 --2- 172.21.15.26:0/1270884138 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564556e26cb0 0x564556e3eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:19.728 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.725+0000 7fa977fff640 1 -- 172.21.15.26:0/1270884138 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564556e1baf0 msgr2=0x564556e24380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:27:19.728 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.725+0000 7fa977fff640 1 --2- 172.21.15.26:0/1270884138 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564556e1baf0 0x564556e24380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:19.728 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.725+0000 7fa977fff640 1 -- 172.21.15.26:0/1270884138 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564556e26cb0 msgr2=0x564556e3eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:27:19.728 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.725+0000 7fa977fff640 1 --2- 172.21.15.26:0/1270884138 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564556e26cb0 0x564556e3eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:19.728 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.726+0000 7fa977fff640 1 -- 172.21.15.26:0/1270884138 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564556e17e90 con 0x564556e1c410 2024-09-11T23:27:19.728 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.726+0000 7fa977fff640 1 --2- 172.21.15.26:0/1270884138 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564556e1c410 0x564556e3e5f0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fa968009250 tx=0x7fa968009c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0b9e9c6d8617d0e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:27:19.728 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.726+0000 7fa95dffb640 1 -- 172.21.15.26:0/1270884138 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa968032020 con 0x564556e1c410 2024-09-11T23:27:19.728 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.726+0000 7fa95dffb640 1 -- 172.21.15.26:0/1270884138 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9680055a0 con 0x564556e1c410 2024-09-11T23:27:19.728 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.726+0000 7fa95dffb640 1 -- 172.21.15.26:0/1270884138 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa968029040 con 0x564556e1c410 2024-09-11T23:27:19.729 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.726+0000 7fa9813d3740 1 -- 172.21.15.26:0/1270884138 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564556e1c410 msgr2=0x564556e3e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:19.729 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.726+0000 7fa9813d3740 1 --2- 172.21.15.26:0/1270884138 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564556e1c410 0x564556e3e5f0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fa968009250 tx=0x7fa968009c50 comp rx=0 tx=0).stop 2024-09-11T23:27:19.729 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.726+0000 7fa9813d3740 1 -- 172.21.15.26:0/1270884138 shutdown_connections 2024-09-11T23:27:19.729 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.726+0000 7fa9813d3740 1 --2- 172.21.15.26:0/1270884138 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564556e26cb0 0x564556e3eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:19.729 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.726+0000 7fa9813d3740 1 --2- 172.21.15.26:0/1270884138 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564556e1c410 0x564556e3e5f0 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:19.729 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.726+0000 7fa9813d3740 1 --2- 172.21.15.26:0/1270884138 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564556e1baf0 0x564556e24380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:19.729 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.726+0000 7fa9813d3740 1 -- 172.21.15.26:0/1270884138 >> 172.21.15.26:0/1270884138 conn(0x564556e18060 msgr2=0x564556e25b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:27:19.729 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.726+0000 7fa9813d3740 1 -- 172.21.15.26:0/1270884138 shutdown_connections 2024-09-11T23:27:19.729 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.727+0000 7fa9813d3740 1 -- 172.21.15.26:0/1270884138 wait complete. 2024-09-11T23:27:19.729 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.727+0000 7fa9813d3740 1 -- start start 2024-09-11T23:27:19.729 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.727+0000 7fa9813d3740 1 Processor -- start 2024-09-11T23:27:19.732 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.727+0000 7fa9813d3740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564556e1baf0 0x564556e2ef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:19.732 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.727+0000 7fa9813d3740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564556e1c410 0x564556e2f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:19.732 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.727+0000 7fa9813d3740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564556e26cb0 0x564556e203a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:19.732 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.727+0000 7fa9813d3740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fa9781b0fd0 con 0x564556e26cb0 2024-09-11T23:27:19.732 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.727+0000 7fa9813d3740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fa9781b0840 con 0x564556e1baf0 2024-09-11T23:27:19.732 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.727+0000 7fa9813d3740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fa9781ad9e0 con 0x564556e1c410 2024-09-11T23:27:19.732 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.727+0000 7fa9813d3740 10 client.0 _ll_register_callbacks cb 0x7fa96c009a90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:27:19.732 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.727+0000 7fa9777fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564556e26cb0 0x564556e203a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:19.732 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.727+0000 7fa9777fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564556e26cb0 0x564556e203a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36502/0 (socket says 172.21.15.26:36502) 2024-09-11T23:27:19.732 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.727+0000 7fa9777fe640 1 -- 172.21.15.26:0/2168945284 learned_addr learned my addr 172.21.15.26:0/2168945284 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:27:19.733 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.727+0000 7fa977fff640 1 --2- 172.21.15.26:0/2168945284 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564556e1c410 0x564556e2f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:19.733 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.727+0000 7fa9777fe640 1 --2- 172.21.15.26:0/2168945284 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564556e1baf0 0x564556e2ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:19.733 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.728+0000 7fa9777fe640 1 -- 172.21.15.26:0/2168945284 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564556e1c410 msgr2=0x564556e2f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:27:19.733 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.728+0000 7fa9777fe640 1 --2- 172.21.15.26:0/2168945284 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564556e1c410 0x564556e2f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:19.733 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.728+0000 7fa9777fe640 1 -- 172.21.15.26:0/2168945284 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564556e1baf0 msgr2=0x564556e2ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:27:19.733 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.728+0000 7fa9777fe640 1 --2- 172.21.15.26:0/2168945284 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564556e1baf0 0x564556e2ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:19.733 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.728+0000 7fa9777fe640 1 -- 172.21.15.26:0/2168945284 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564556e20bd0 con 0x564556e26cb0 2024-09-11T23:27:19.733 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.728+0000 7fa977fff640 1 --2- 172.21.15.26:0/2168945284 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564556e1c410 0x564556e2f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:27:19.733 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.728+0000 7fa9777fe640 1 --2- 172.21.15.26:0/2168945284 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564556e26cb0 0x564556e203a0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x564556e39530 tx=0x7fa968009be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38945280411857fb server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:27:19.733 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.728+0000 7fa9577fe640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:27:19.733 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.728+0000 7fa9577fe640 1 -- 172.21.15.26:0/2168945284 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa96c012e60 con 0x564556e26cb0 2024-09-11T23:27:19.733 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.728+0000 7fa9577fe640 1 -- 172.21.15.26:0/2168945284 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8586+0+0 (secure 0 0 0) 0x7fa96c005020 con 0x564556e26cb0 2024-09-11T23:27:19.734 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.728+0000 7fa9813d3740 1 -- 172.21.15.26:0/2168945284 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9680055a0 con 0x564556e26cb0 2024-09-11T23:27:19.734 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.728+0000 7fa9577fe640 1 -- 172.21.15.26:0/2168945284 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7fa96c01c040 con 0x564556e26cb0 2024-09-11T23:27:19.734 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.728+0000 7fa9813d3740 10 client.16851 fetch_fsmap learned FSMap version 51 2024-09-11T23:27:19.734 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.728+0000 7fa9813d3740 1 -- 172.21.15.26:0/2168945284 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x564556e2e880 con 0x564556e26cb0 2024-09-11T23:27:19.734 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.729+0000 7fa9577fe640 1 -- 172.21.15.26:0/2168945284 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9680055a0 con 0x564556e26cb0 2024-09-11T23:27:19.734 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.729+0000 7fa9577fe640 1 -- 172.21.15.26:0/2168945284 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 51) ==== 8394+0+0 (secure 0 0 0) 0x7fa96c010700 con 0x564556e26cb0 2024-09-11T23:27:19.734 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.729+0000 7fa9813d3740 10 client.16851 fetch_fsmap finished waiting for FSMap version 51 2024-09-11T23:27:19.734 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.729+0000 7fa9813d3740 10 client.16851 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:27:19.734 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.729+0000 7fa9813d3740 20 client.16851 populate_metadata read hostname 'smithi026' 2024-09-11T23:27:19.734 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.729+0000 7fa9813d3740 1 --2- 172.21.15.26:0/2168945284 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x564556ea18a0 0x564556ea3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:19.734 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.729+0000 7fa9813d3740 4 client.16851 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T23:27:19.734 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.729+0000 7fa9813d3740 1 -- 172.21.15.26:0/2168945284 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564556d87d70 con 0x564556ea18a0 2024-09-11T23:27:19.734 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.729+0000 7fa977fff640 1 --2- 172.21.15.26:0/2168945284 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x564556ea18a0 0x564556ea3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:19.734 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.729+0000 7fa977fff640 1 --2- 172.21.15.26:0/2168945284 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x564556ea18a0 0x564556ea3cc0 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=848d64222b1eab56 server_cookie=50ecfcfb29f6eb29 in_seq=0 out_seq=0 2024-09-11T23:27:19.735 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.729+0000 7fa9577fe640 10 client.16851 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:27:19.744 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.744+0000 7fa9577fe640 1 -- 172.21.15.26:0/2168945284 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564556d87d70 con 0x564556ea18a0 2024-09-11T23:27:19.744 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.744+0000 7fa9577fe640 10 client.16851 handle_command_reply: tid=0 2024-09-11T23:27:19.744 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.744+0000 7fa9813d3740 1 client.16851 shutdown 2024-09-11T23:27:19.744 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.744+0000 7fa9813d3740 1 -- 172.21.15.26:0/2168945284 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564556e26cb0 msgr2=0x564556e203a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:19.744 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.744+0000 7fa9813d3740 1 --2- 172.21.15.26:0/2168945284 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564556e26cb0 0x564556e203a0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x564556e39530 tx=0x7fa968009be0 comp rx=0 tx=0).stop 2024-09-11T23:27:19.744 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.744+0000 7fa9813d3740 1 -- 172.21.15.26:0/2168945284 shutdown_connections 2024-09-11T23:27:19.744 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.744+0000 7fa9813d3740 1 --2- 172.21.15.26:0/2168945284 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x564556ea18a0 0x564556ea3cc0 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:19.745 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.744+0000 7fa9813d3740 1 --2- 172.21.15.26:0/2168945284 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564556e26cb0 0x564556e203a0 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:19.745 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.744+0000 7fa9813d3740 1 --2- 172.21.15.26:0/2168945284 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564556e1c410 0x564556e2f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:19.745 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.744+0000 7fa9813d3740 1 --2- 172.21.15.26:0/2168945284 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564556e1baf0 0x564556e2ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:19.745 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.744+0000 7fa9577fe640 0 client.16851 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:27:19.745 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.744+0000 7fa9813d3740 1 -- 172.21.15.26:0/2168945284 >> 172.21.15.26:0/2168945284 conn(0x564556e18060 msgr2=0x564556e169a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:27:19.745 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.744+0000 7fa9813d3740 1 -- 172.21.15.26:0/2168945284 shutdown_connections 2024-09-11T23:27:19.745 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.744+0000 7fa9813d3740 1 -- 172.21.15.26:0/2168945284 wait complete. 2024-09-11T23:27:19.745 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.744+0000 7fa9813d3740 20 client.16851 trim_cache size 0 max 16384 2024-09-11T23:27:19.745 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-09-11T23:27:19.746 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "message": "", 2024-09-11T23:27:19.746 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-09-11T23:27:19.746 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-09-11T23:27:19.746 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.745+0000 7fa97e559640 1 -- 172.21.15.26:0/2231266129 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fa9600a2e20 msgr2=0x7fa9600a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:19.747 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.745+0000 7fa97e559640 1 --2- 172.21.15.26:0/2231266129 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fa9600a2e20 0x7fa9600a52e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fa9700b8fc0 tx=0x7fa9700a7120 comp rx=0 tx=0).stop 2024-09-11T23:27:19.747 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.745+0000 7fa97e559640 1 -- 172.21.15.26:0/2231266129 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa97810c110 msgr2=0x7fa978101390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:19.747 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.745+0000 7fa97e559640 1 --2- 172.21.15.26:0/2231266129 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa97810c110 0x7fa978101390 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fa970004af0 tx=0x7fa97002f3a0 comp rx=0 tx=0).stop 2024-09-11T23:27:19.747 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.745+0000 7fa97e559640 1 -- 172.21.15.26:0/2231266129 shutdown_connections 2024-09-11T23:27:19.747 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.745+0000 7fa97e559640 1 --2- 172.21.15.26:0/2231266129 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fa9600a2e20 0x7fa9600a52e0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:19.747 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.745+0000 7fa97e559640 1 --2- 172.21.15.26:0/2231266129 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa97810c110 0x7fa978101390 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:19.747 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.745+0000 7fa97e559640 1 --2- 172.21.15.26:0/2231266129 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa9781035e0 0x7fa978100e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:19.747 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.745+0000 7fa97e559640 1 --2- 172.21.15.26:0/2231266129 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa978102cc0 0x7fa978100910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:19.747 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.745+0000 7fa97e559640 1 -- 172.21.15.26:0/2231266129 >> 172.21.15.26:0/2231266129 conn(0x7fa9780fc6b0 msgr2=0x7fa9780fdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:27:19.747 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.745+0000 7fa97e559640 1 -- 172.21.15.26:0/2231266129 shutdown_connections 2024-09-11T23:27:19.747 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:19.745+0000 7fa97e559640 1 -- 172.21.15.26:0/2231266129 wait complete. 2024-09-11T23:27:20.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:27:19 smithi136 ceph-mon[35121]: pgmap v276: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 2.2 KiB/s wr, 10 op/s 2024-09-11T23:27:20.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:27:19 smithi026 ceph-mon[31172]: pgmap v276: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 2.2 KiB/s wr, 10 op/s 2024-09-11T23:27:20.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:27:19 smithi076 ceph-mon[35296]: pgmap v276: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 2.2 KiB/s wr, 10 op/s 2024-09-11T23:27:21.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:27:21 smithi136 ceph-mon[35121]: pgmap v277: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.8 KiB/s wr, 15 op/s 2024-09-11T23:27:21.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:27:21 smithi136 ceph-mon[35121]: osdmap e92: 12 total, 12 up, 12 in 2024-09-11T23:27:22.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:27:21 smithi026 ceph-mon[31172]: pgmap v277: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.8 KiB/s wr, 15 op/s 2024-09-11T23:27:22.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:27:21 smithi026 ceph-mon[31172]: osdmap e92: 12 total, 12 up, 12 in 2024-09-11T23:27:22.052 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:27:21 smithi076 ceph-mon[35296]: pgmap v277: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.8 KiB/s wr, 15 op/s 2024-09-11T23:27:22.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:27:21 smithi076 ceph-mon[35296]: osdmap e92: 12 total, 12 up, 12 in 2024-09-11T23:27:23.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:27:23 smithi136 ceph-mon[35121]: pgmap v279: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.6 KiB/s wr, 12 op/s 2024-09-11T23:27:24.074 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:27:23 smithi076 ceph-mon[35296]: pgmap v279: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.6 KiB/s wr, 12 op/s 2024-09-11T23:27:24.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:27:23 smithi026 ceph-mon[31172]: pgmap v279: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.6 KiB/s wr, 12 op/s 2024-09-11T23:27:25.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:27:25 smithi136 ceph-mon[35121]: pgmap v280: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 45 KiB/s wr, 15 op/s 2024-09-11T23:27:26.000 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:27:25 smithi026 ceph-mon[31172]: pgmap v280: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 45 KiB/s wr, 15 op/s 2024-09-11T23:27:26.052 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:27:25 smithi076 ceph-mon[35296]: pgmap v280: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 45 KiB/s wr, 15 op/s 2024-09-11T23:27:26.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:26.304+0000 7f29b37fe640 1 -- 172.21.15.26:0/1645053041 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55a70b930d20 con 0x55a70ba4a2b0 2024-09-11T23:27:26.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:26.304+0000 7f29b37fe640 10 client.16833 handle_command_reply: tid=0 2024-09-11T23:27:26.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:26.304+0000 7f29dc8b9740 1 client.16833 shutdown 2024-09-11T23:27:26.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:26.304+0000 7f29dc8b9740 1 -- 172.21.15.26:0/1645053041 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a70b9c3760 msgr2=0x55a70b9d6d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:26.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:26.304+0000 7f29dc8b9740 1 --2- 172.21.15.26:0/1645053041 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a70b9c3760 0x55a70b9d6d10 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f29c8014fd0 tx=0x7f29c800a6a0 comp rx=0 tx=0).stop 2024-09-11T23:27:26.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:26.304+0000 7f29dc8b9740 1 -- 172.21.15.26:0/1645053041 shutdown_connections 2024-09-11T23:27:26.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:26.304+0000 7f29dc8b9740 1 --2- 172.21.15.26:0/1645053041 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a70b9c4080 0x55a70b9d30a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:26.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:26.304+0000 7f29dc8b9740 1 --2- 172.21.15.26:0/1645053041 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55a70ba4a2b0 0x55a70ba4c6a0 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:26.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:26.304+0000 7f29dc8b9740 1 --2- 172.21.15.26:0/1645053041 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a70b9ce940 0x55a70b9d35e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:26.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:26.304+0000 7f29dc8b9740 1 --2- 172.21.15.26:0/1645053041 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a70b9c3760 0x55a70b9d6d10 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:26.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:26.304+0000 7f29b37fe640 0 client.16833 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:27:26.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:26.304+0000 7f29dc8b9740 1 -- 172.21.15.26:0/1645053041 >> 172.21.15.26:0/1645053041 conn(0x55a70b9bfcf0 msgr2=0x55a70b9be650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:27:26.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:26.304+0000 7f29dc8b9740 1 -- 172.21.15.26:0/1645053041 shutdown_connections 2024-09-11T23:27:26.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:26.304+0000 7f29dc8b9740 1 -- 172.21.15.26:0/1645053041 wait complete. 2024-09-11T23:27:26.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:26.304+0000 7f29dc8b9740 20 client.16833 trim_cache size 0 max 16384 2024-09-11T23:27:26.306 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-11T23:27:26.306 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 51, 2024-09-11T23:27:26.306 INFO:teuthology.orchestra.run.smithi026.stdout: "leader": 24485, 2024-09-11T23:27:26.306 INFO:teuthology.orchestra.run.smithi026.stdout: "set_version": 34, 2024-09-11T23:27:26.306 INFO:teuthology.orchestra.run.smithi026.stdout: "sets": { 2024-09-11T23:27:26.306 INFO:teuthology.orchestra.run.smithi026.stdout: "670d17b7": { 2024-09-11T23:27:26.306 INFO:teuthology.orchestra.run.smithi026.stdout: "version": 34, 2024-09-11T23:27:26.306 INFO:teuthology.orchestra.run.smithi026.stdout: "age_ref": 0.0, 2024-09-11T23:27:26.306 INFO:teuthology.orchestra.run.smithi026.stdout: "state": { 2024-09-11T23:27:26.306 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "QUIESCED", 2024-09-11T23:27:26.307 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.0 2024-09-11T23:27:26.307 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:27:26.307 INFO:teuthology.orchestra.run.smithi026.stdout: "timeout": 90.0, 2024-09-11T23:27:26.307 INFO:teuthology.orchestra.run.smithi026.stdout: "expiration": 139.4, 2024-09-11T23:27:26.307 INFO:teuthology.orchestra.run.smithi026.stdout: "members": { 2024-09-11T23:27:26.307 INFO:teuthology.orchestra.run.smithi026.stdout: "file:/": { 2024-09-11T23:27:26.307 INFO:teuthology.orchestra.run.smithi026.stdout: "excluded": false, 2024-09-11T23:27:26.307 INFO:teuthology.orchestra.run.smithi026.stdout: "state": { 2024-09-11T23:27:26.307 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "QUIESCED", 2024-09-11T23:27:26.307 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.0 2024-09-11T23:27:26.307 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:27:26.307 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:27:26.307 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:27:26.307 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:27:26.307 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:27:26.308 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-11T23:27:26.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:26.305+0000 7f29da4e0640 1 -- 172.21.15.26:0/3696073587 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f29bc0a2cf0 msgr2=0x7f29bc0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:26.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:26.305+0000 7f29da4e0640 1 --2- 172.21.15.26:0/3696073587 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f29bc0a2cf0 0x7f29bc0a51b0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f29cc0b9a70 tx=0x7f29cc0b8c20 comp rx=0 tx=0).stop 2024-09-11T23:27:26.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:26.305+0000 7f29da4e0640 1 -- 172.21.15.26:0/3696073587 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f29d4106f50 msgr2=0x7f29d4101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:26.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:26.305+0000 7f29da4e0640 1 --2- 172.21.15.26:0/3696073587 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f29d4106f50 0x7f29d4101cf0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f29cc006010 tx=0x7f29cc003c20 comp rx=0 tx=0).stop 2024-09-11T23:27:26.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:26.305+0000 7f29da4e0640 1 -- 172.21.15.26:0/3696073587 shutdown_connections 2024-09-11T23:27:26.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:26.305+0000 7f29da4e0640 1 --2- 172.21.15.26:0/3696073587 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f29bc0a2cf0 0x7f29bc0a51b0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:26.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:26.305+0000 7f29da4e0640 1 --2- 172.21.15.26:0/3696073587 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f29d4100ca0 0x7f29d41017b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:26.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:26.305+0000 7f29da4e0640 1 --2- 172.21.15.26:0/3696073587 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f29d4106f50 0x7f29d4101cf0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:26.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:26.305+0000 7f29da4e0640 1 --2- 172.21.15.26:0/3696073587 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f29d41002f0 0x7f29d4101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:26.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:26.305+0000 7f29da4e0640 1 -- 172.21.15.26:0/3696073587 >> 172.21.15.26:0/3696073587 conn(0x7f29d4077210 msgr2=0x7f29d4105870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:27:26.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:26.306+0000 7f29da4e0640 1 -- 172.21.15.26:0/3696073587 shutdown_connections 2024-09-11T23:27:26.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:26.306+0000 7f29da4e0640 1 -- 172.21.15.26:0/3696073587 wait complete. 2024-09-11T23:27:27.242 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '670d17b7', quiesce took 11.7 seconds. Will release after: 7.699999999999999 2024-09-11T23:27:27.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:27:27 smithi136 ceph-mon[35121]: pgmap v281: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 46 KiB/s wr, 16 op/s 2024-09-11T23:27:28.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:27:27 smithi026 ceph-mon[31172]: pgmap v281: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 46 KiB/s wr, 16 op/s 2024-09-11T23:27:28.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:27:27 smithi076 ceph-mon[35296]: pgmap v281: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 46 KiB/s wr, 16 op/s 2024-09-11T23:27:29.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:27:29 smithi136 ceph-mon[35121]: pgmap v282: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 46 KiB/s wr, 16 op/s 2024-09-11T23:27:30.000 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:27:29 smithi026 ceph-mon[31172]: pgmap v282: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 46 KiB/s wr, 16 op/s 2024-09-11T23:27:30.052 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:27:29 smithi076 ceph-mon[35296]: pgmap v282: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 46 KiB/s wr, 16 op/s 2024-09-11T23:27:32.000 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:27:31 smithi026 ceph-mon[31172]: pgmap v283: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 46 KiB/s wr, 16 op/s 2024-09-11T23:27:32.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:27:31 smithi076 ceph-mon[35296]: pgmap v283: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 46 KiB/s wr, 16 op/s 2024-09-11T23:27:32.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:27:31 smithi136 ceph-mon[35121]: pgmap v283: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 46 KiB/s wr, 16 op/s 2024-09-11T23:27:32.850 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.846+0000 7f0f79674640 1 -- 172.21.15.26:0/1716800322 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f74101250 msgr2=0x7f0f74107990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:32.850 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.846+0000 7f0f79674640 1 --2- 172.21.15.26:0/1716800322 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f74101250 0x7f0f74107990 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f0f5c009900 tx=0x7f0f5c02a410 comp rx=0 tx=0).stop 2024-09-11T23:27:32.850 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.846+0000 7f0f79674640 1 -- 172.21.15.26:0/1716800322 shutdown_connections 2024-09-11T23:27:32.850 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.846+0000 7f0f79674640 1 --2- 172.21.15.26:0/1716800322 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0f741080f0 0x7f0f7410ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:32.850 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.846+0000 7f0f79674640 1 --2- 172.21.15.26:0/1716800322 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f74101250 0x7f0f74107990 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:32.850 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.846+0000 7f0f79674640 1 --2- 172.21.15.26:0/1716800322 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0f741008a0 0x7f0f74100c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:32.851 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.846+0000 7f0f79674640 1 -- 172.21.15.26:0/1716800322 >> 172.21.15.26:0/1716800322 conn(0x7f0f74075ef0 msgr2=0x7f0f74076300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:27:32.851 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.846+0000 7f0f79674640 1 -- 172.21.15.26:0/1716800322 shutdown_connections 2024-09-11T23:27:32.851 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.846+0000 7f0f79674640 1 -- 172.21.15.26:0/1716800322 wait complete. 2024-09-11T23:27:32.851 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.847+0000 7f0f79674640 1 Processor -- start 2024-09-11T23:27:32.851 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.847+0000 7f0f79674640 1 -- start start 2024-09-11T23:27:32.851 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.847+0000 7f0f79674640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0f741008a0 0x7f0f74198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:32.851 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.848+0000 7f0f79674640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f741080f0 0x7f0f74198860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:32.851 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.848+0000 7f0f79674640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0f7419cbe0 0x7f0f7419d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:32.851 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.848+0000 7f0f79674640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f0f74111f20 con 0x7f0f741080f0 2024-09-11T23:27:32.851 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.848+0000 7f0f79674640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f0f74111da0 con 0x7f0f741008a0 2024-09-11T23:27:32.851 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.848+0000 7f0f79674640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f0f741120a0 con 0x7f0f7419cbe0 2024-09-11T23:27:32.851 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.848+0000 7f0f727fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f741080f0 0x7f0f74198860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:32.851 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.848+0000 7f0f727fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f741080f0 0x7f0f74198860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52576/0 (socket says 172.21.15.26:52576) 2024-09-11T23:27:32.852 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.848+0000 7f0f727fc640 1 -- 172.21.15.26:0/4065816995 learned_addr learned my addr 172.21.15.26:0/4065816995 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:27:32.852 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.848+0000 7f0f72ffd640 1 --2- 172.21.15.26:0/4065816995 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0f741008a0 0x7f0f74198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:32.852 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.848+0000 7f0f737fe640 1 --2- 172.21.15.26:0/4065816995 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0f7419cbe0 0x7f0f7419d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:32.852 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.848+0000 7f0f727fc640 1 -- 172.21.15.26:0/4065816995 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0f7419cbe0 msgr2=0x7f0f7419d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:32.852 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.848+0000 7f0f727fc640 1 --2- 172.21.15.26:0/4065816995 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0f7419cbe0 0x7f0f7419d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:32.852 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.848+0000 7f0f727fc640 1 -- 172.21.15.26:0/4065816995 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0f741008a0 msgr2=0x7f0f74198320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:32.852 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.848+0000 7f0f727fc640 1 --2- 172.21.15.26:0/4065816995 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0f741008a0 0x7f0f74198320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:32.852 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.848+0000 7f0f727fc640 1 -- 172.21.15.26:0/4065816995 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f7419d6d0 con 0x7f0f741080f0 2024-09-11T23:27:32.852 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.848+0000 7f0f727fc640 1 --2- 172.21.15.26:0/4065816995 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f741080f0 0x7f0f74198860 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f0f5c005940 tx=0x7f0f5c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:27:32.852 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.848+0000 7f0f6bfff640 1 -- 172.21.15.26:0/4065816995 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f5c03f030 con 0x7f0f741080f0 2024-09-11T23:27:32.852 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.848+0000 7f0f6bfff640 1 -- 172.21.15.26:0/4065816995 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f5c003ad0 con 0x7f0f741080f0 2024-09-11T23:27:32.852 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.848+0000 7f0f79674640 1 -- 172.21.15.26:0/4065816995 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f7419d960 con 0x7f0f741080f0 2024-09-11T23:27:32.852 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.849+0000 7f0f6bfff640 1 -- 172.21.15.26:0/4065816995 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f5c035070 con 0x7f0f741080f0 2024-09-11T23:27:32.852 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.849+0000 7f0f79674640 1 -- 172.21.15.26:0/4065816995 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f741a5120 con 0x7f0f741080f0 2024-09-11T23:27:32.860 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.851+0000 7f0f6bfff640 1 -- 172.21.15.26:0/4065816995 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f0f5c042030 con 0x7f0f741080f0 2024-09-11T23:27:32.860 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.851+0000 7f0f7c417740 1 Processor -- start 2024-09-11T23:27:32.860 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.851+0000 7f0f7c417740 1 -- start start 2024-09-11T23:27:32.860 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.852+0000 7f0f7c417740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5622fc0f6af0 0x5622fc0f6ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:32.861 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.852+0000 7f0f7c417740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5622fc0f7410 0x5622fc101770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:32.861 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.852+0000 7f0f7c417740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5622fc101cb0 0x5622fc1040a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:32.861 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.852+0000 7f0f7c417740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5622fc05b2f0 con 0x5622fc101cb0 2024-09-11T23:27:32.861 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.852+0000 7f0f7c417740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5622fc003270 con 0x5622fc0f7410 2024-09-11T23:27:32.861 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.852+0000 7f0f7c417740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5622fc0572b0 con 0x5622fc0f6af0 2024-09-11T23:27:32.861 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.852+0000 7f0f6bfff640 1 --2- 172.21.15.26:0/4065816995 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f0f600a2da0 0x7f0f600a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:32.861 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.852+0000 7f0f727fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5622fc101cb0 0x5622fc1040a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:32.861 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.852+0000 7f0f727fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5622fc101cb0 0x5622fc1040a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52584/0 (socket says 172.21.15.26:52584) 2024-09-11T23:27:32.861 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.852+0000 7f0f727fc640 1 -- 172.21.15.26:0/1117469131 learned_addr learned my addr 172.21.15.26:0/1117469131 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:27:32.861 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.852+0000 7f0f72ffd640 1 --2- 172.21.15.26:0/1117469131 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5622fc0f7410 0x5622fc101770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:32.861 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.852+0000 7f0f737fe640 1 --2- 172.21.15.26:0/4065816995 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f0f600a2da0 0x7f0f600a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:32.861 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.852+0000 7f0f737fe640 1 --2- 172.21.15.26:0/1117469131 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5622fc0f6af0 0x5622fc0f6ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:32.861 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.852+0000 7f0f6bfff640 1 -- 172.21.15.26:0/4065816995 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8542+0+0 (secure 0 0 0) 0x7f0f5c0ef7b0 con 0x7f0f741080f0 2024-09-11T23:27:32.862 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.852+0000 7f0f727fc640 1 -- 172.21.15.26:0/1117469131 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5622fc0f6af0 msgr2=0x5622fc0f6ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:27:32.862 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.852+0000 7f0f727fc640 1 --2- 172.21.15.26:0/1117469131 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5622fc0f6af0 0x5622fc0f6ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:32.862 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.852+0000 7f0f727fc640 1 -- 172.21.15.26:0/1117469131 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5622fc0f7410 msgr2=0x5622fc101770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:27:32.862 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.852+0000 7f0f727fc640 1 --2- 172.21.15.26:0/1117469131 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5622fc0f7410 0x5622fc101770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:32.862 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.852+0000 7f0f727fc640 1 -- 172.21.15.26:0/1117469131 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622fc002340 con 0x5622fc101cb0 2024-09-11T23:27:32.862 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.852+0000 7f0f737fe640 1 --2- 172.21.15.26:0/1117469131 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5622fc0f6af0 0x5622fc0f6ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:27:32.862 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.852+0000 7f0f72ffd640 1 --2- 172.21.15.26:0/1117469131 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5622fc0f7410 0x5622fc101770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:27:32.862 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.853+0000 7f0f727fc640 1 --2- 172.21.15.26:0/1117469131 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5622fc101cb0 0x5622fc1040a0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f0f5c0b8740 tx=0x7f0f5c0bbc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f382613a9b74d653 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:27:32.862 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.853+0000 7f0f737fe640 1 --2- 172.21.15.26:0/4065816995 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f0f600a2da0 0x7f0f600a5260 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f0f6c005fd0 tx=0x7f0f6c009be0 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:27:32.862 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.853+0000 7f0f697fa640 1 -- 172.21.15.26:0/1117469131 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f5c049070 con 0x5622fc101cb0 2024-09-11T23:27:32.862 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.853+0000 7f0f697fa640 1 -- 172.21.15.26:0/1117469131 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f5c0c0070 con 0x5622fc101cb0 2024-09-11T23:27:32.863 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.853+0000 7f0f697fa640 1 -- 172.21.15.26:0/1117469131 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f5c0ed2d0 con 0x5622fc101cb0 2024-09-11T23:27:32.863 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.854+0000 7f0f7c417740 1 -- 172.21.15.26:0/1117469131 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5622fc101cb0 msgr2=0x5622fc1040a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:32.863 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.854+0000 7f0f7c417740 1 --2- 172.21.15.26:0/1117469131 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5622fc101cb0 0x5622fc1040a0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f0f5c0b8740 tx=0x7f0f5c0bbc70 comp rx=0 tx=0).stop 2024-09-11T23:27:32.863 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.854+0000 7f0f7c417740 1 -- 172.21.15.26:0/1117469131 shutdown_connections 2024-09-11T23:27:32.863 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.854+0000 7f0f7c417740 1 --2- 172.21.15.26:0/1117469131 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5622fc0f7410 0x5622fc101770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:32.863 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.854+0000 7f0f7c417740 1 --2- 172.21.15.26:0/1117469131 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5622fc101cb0 0x5622fc1040a0 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:32.863 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.854+0000 7f0f7c417740 1 --2- 172.21.15.26:0/1117469131 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5622fc0f6af0 0x5622fc0f6ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:32.863 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.854+0000 7f0f7c417740 1 -- 172.21.15.26:0/1117469131 >> 172.21.15.26:0/1117469131 conn(0x5622fc0f3060 msgr2=0x5622fc0f3440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:27:32.863 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.854+0000 7f0f7c417740 1 -- 172.21.15.26:0/1117469131 shutdown_connections 2024-09-11T23:27:32.863 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.854+0000 7f0f7c417740 1 -- 172.21.15.26:0/1117469131 wait complete. 2024-09-11T23:27:32.863 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.854+0000 7f0f7c417740 1 -- start start 2024-09-11T23:27:32.864 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.854+0000 7f0f7c417740 1 Processor -- start 2024-09-11T23:27:32.864 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.854+0000 7f0f7c417740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5622fc0f6af0 0x5622fc0febf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:32.864 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.854+0000 7f0f7c417740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5622fc0f7410 0x5622fc0ff130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:32.864 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.854+0000 7f0f7c417740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5622fc101cb0 0x5622fc0ff670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:32.864 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.854+0000 7f0f7c417740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5622fc104640 con 0x5622fc0f6af0 2024-09-11T23:27:32.864 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.854+0000 7f0f7c417740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5622fbfe4290 con 0x5622fc0f7410 2024-09-11T23:27:32.864 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.854+0000 7f0f7c417740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5622fc1047c0 con 0x5622fc101cb0 2024-09-11T23:27:32.864 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.854+0000 7f0f7c417740 10 client.0 _ll_register_callbacks cb 0x5622fc015140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:27:32.864 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.855+0000 7f0f72ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5622fc0f6af0 0x5622fc0febf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:32.864 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.855+0000 7f0f72ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5622fc0f6af0 0x5622fc0febf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52598/0 (socket says 172.21.15.26:52598) 2024-09-11T23:27:32.864 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.855+0000 7f0f72ffd640 1 -- 172.21.15.26:0/2086948016 learned_addr learned my addr 172.21.15.26:0/2086948016 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:27:32.864 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.855+0000 7f0f737fe640 1 --2- 172.21.15.26:0/2086948016 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5622fc101cb0 0x5622fc0ff670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:32.864 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.855+0000 7f0f727fc640 1 --2- 172.21.15.26:0/2086948016 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5622fc0f7410 0x5622fc0ff130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:32.865 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.855+0000 7f0f72ffd640 1 -- 172.21.15.26:0/2086948016 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5622fc101cb0 msgr2=0x5622fc0ff670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:27:32.865 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.855+0000 7f0f72ffd640 1 --2- 172.21.15.26:0/2086948016 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5622fc101cb0 0x5622fc0ff670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:32.865 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.855+0000 7f0f72ffd640 1 -- 172.21.15.26:0/2086948016 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5622fc0f7410 msgr2=0x5622fc0ff130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:27:32.865 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.855+0000 7f0f72ffd640 1 --2- 172.21.15.26:0/2086948016 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5622fc0f7410 0x5622fc0ff130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:32.865 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.855+0000 7f0f72ffd640 1 -- 172.21.15.26:0/2086948016 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5622fbd6e210 con 0x5622fc0f6af0 2024-09-11T23:27:32.865 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.855+0000 7f0f72ffd640 1 --2- 172.21.15.26:0/2086948016 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5622fc0f6af0 0x5622fc0febf0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f0f6400ba10 tx=0x7f0f6400bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8e5e3ae15f9abbd server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:27:32.865 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.855+0000 7f0f5a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:27:32.865 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.855+0000 7f0f5a7fc640 1 -- 172.21.15.26:0/2086948016 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f64003e80 con 0x5622fc0f6af0 2024-09-11T23:27:32.865 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.855+0000 7f0f5a7fc640 1 -- 172.21.15.26:0/2086948016 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8542+0+0 (secure 0 0 0) 0x7f0f64011440 con 0x5622fc0f6af0 2024-09-11T23:27:32.865 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.855+0000 7f0f7c417740 1 -- 172.21.15.26:0/2086948016 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0f5c0c0070 con 0x5622fc0f6af0 2024-09-11T23:27:32.865 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.855+0000 7f0f5a7fc640 1 -- 172.21.15.26:0/2086948016 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f0f64011640 con 0x5622fc0f6af0 2024-09-11T23:27:32.866 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.855+0000 7f0f7c417740 10 client.16863 fetch_fsmap learned FSMap version 51 2024-09-11T23:27:32.866 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.856+0000 7f0f7c417740 1 -- 172.21.15.26:0/2086948016 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x5622fbe3a620 con 0x5622fc0f6af0 2024-09-11T23:27:32.866 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.856+0000 7f0f5a7fc640 1 -- 172.21.15.26:0/2086948016 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f5c0c0070 con 0x5622fc0f6af0 2024-09-11T23:27:32.866 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.856+0000 7f0f5a7fc640 1 -- 172.21.15.26:0/2086948016 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 51) ==== 8394+0+0 (secure 0 0 0) 0x7f0f6401bad0 con 0x5622fc0f6af0 2024-09-11T23:27:32.866 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.856+0000 7f0f7c417740 10 client.16863 fetch_fsmap finished waiting for FSMap version 51 2024-09-11T23:27:32.866 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.856+0000 7f0f7c417740 10 client.16863 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:27:32.866 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.856+0000 7f0f7c417740 20 client.16863 populate_metadata read hostname 'smithi026' 2024-09-11T23:27:32.866 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.856+0000 7f0f7c417740 1 --2- 172.21.15.26:0/2086948016 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5622fc111a00 0x5622fc113e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:32.866 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.856+0000 7f0f7c417740 4 client.16863 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:27:32.866 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.856+0000 7f0f7c417740 1 -- 172.21.15.26:0/2086948016 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5622fc068a20 con 0x5622fc111a00 2024-09-11T23:27:32.866 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.856+0000 7f0f727fc640 1 --2- 172.21.15.26:0/2086948016 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5622fc111a00 0x5622fc113e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:32.867 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.857+0000 7f0f727fc640 1 --2- 172.21.15.26:0/2086948016 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5622fc111a00 0x5622fc113e90 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ab5fb2e18242677 server_cookie=8998a2f661859770 in_seq=0 out_seq=0 2024-09-11T23:27:32.867 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.857+0000 7f0f5a7fc640 10 client.16863 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:27:32.867 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.857+0000 7f0f5a7fc640 1 -- 172.21.15.26:0/2086948016 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5622fc068a20 con 0x5622fc111a00 2024-09-11T23:27:32.867 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.857+0000 7f0f5a7fc640 10 client.16863 handle_command_reply: tid=0 2024-09-11T23:27:32.867 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.858+0000 7f0f7c417740 1 client.16863 shutdown 2024-09-11T23:27:32.867 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.858+0000 7f0f7c417740 1 -- 172.21.15.26:0/2086948016 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5622fc0f6af0 msgr2=0x5622fc0febf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:32.867 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.858+0000 7f0f7c417740 1 --2- 172.21.15.26:0/2086948016 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5622fc0f6af0 0x5622fc0febf0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f0f6400ba10 tx=0x7f0f6400bed0 comp rx=0 tx=0).stop 2024-09-11T23:27:32.867 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.858+0000 7f0f7c417740 1 -- 172.21.15.26:0/2086948016 shutdown_connections 2024-09-11T23:27:32.867 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.858+0000 7f0f7c417740 1 --2- 172.21.15.26:0/2086948016 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5622fc0f7410 0x5622fc0ff130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:32.867 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.858+0000 7f0f7c417740 1 --2- 172.21.15.26:0/2086948016 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5622fc111a00 0x5622fc113e90 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:32.867 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.858+0000 7f0f7c417740 1 --2- 172.21.15.26:0/2086948016 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5622fc101cb0 0x5622fc0ff670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:32.867 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.858+0000 7f0f7c417740 1 --2- 172.21.15.26:0/2086948016 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5622fc0f6af0 0x5622fc0febf0 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:32.868 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.858+0000 7f0f7c417740 1 -- 172.21.15.26:0/2086948016 >> 172.21.15.26:0/2086948016 conn(0x5622fc0f3060 msgr2=0x5622fc0f8d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:27:32.868 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.858+0000 7f0f5a7fc640 0 client.16863 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:27:32.868 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.858+0000 7f0f7c417740 1 -- 172.21.15.26:0/2086948016 shutdown_connections 2024-09-11T23:27:32.868 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.858+0000 7f0f7c417740 1 -- 172.21.15.26:0/2086948016 wait complete. 2024-09-11T23:27:32.868 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.858+0000 7f0f7c417740 20 client.16863 trim_cache size 0 max 16384 2024-09-11T23:27:32.874 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.873+0000 7f0f7c417740 1 Processor -- start 2024-09-11T23:27:32.874 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.873+0000 7f0f7c417740 1 -- start start 2024-09-11T23:27:32.874 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.873+0000 7f0f7c417740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5622fc0f6af0 0x5622fc0ff4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:32.874 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.873+0000 7f0f7c417740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5622fc0f7410 0x5622fc1197f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:32.874 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.873+0000 7f0f7c417740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5622fc101cb0 0x5622fc119d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:32.874 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.873+0000 7f0f7c417740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f0f741ad760 con 0x5622fc0f6af0 2024-09-11T23:27:32.874 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.873+0000 7f0f7c417740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f0f741af420 con 0x5622fc0f7410 2024-09-11T23:27:32.874 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.873+0000 7f0f7c417740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f0f741ade30 con 0x5622fc101cb0 2024-09-11T23:27:32.875 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.873+0000 7f0f72ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5622fc0f6af0 0x5622fc0ff4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:32.875 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.873+0000 7f0f72ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5622fc0f6af0 0x5622fc0ff4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52608/0 (socket says 172.21.15.26:52608) 2024-09-11T23:27:32.875 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.873+0000 7f0f72ffd640 1 -- 172.21.15.26:0/248289856 learned_addr learned my addr 172.21.15.26:0/248289856 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:27:32.875 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.873+0000 7f0f737fe640 1 --2- 172.21.15.26:0/248289856 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5622fc101cb0 0x5622fc119d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:32.875 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.873+0000 7f0f727fc640 1 --2- 172.21.15.26:0/248289856 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5622fc0f7410 0x5622fc1197f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:32.875 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.873+0000 7f0f72ffd640 1 -- 172.21.15.26:0/248289856 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5622fc101cb0 msgr2=0x5622fc119d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:27:32.875 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.873+0000 7f0f72ffd640 1 --2- 172.21.15.26:0/248289856 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5622fc101cb0 0x5622fc119d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:32.875 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.873+0000 7f0f72ffd640 1 -- 172.21.15.26:0/248289856 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5622fc0f7410 msgr2=0x5622fc1197f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:27:32.875 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.873+0000 7f0f72ffd640 1 --2- 172.21.15.26:0/248289856 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5622fc0f7410 0x5622fc1197f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:32.875 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.873+0000 7f0f72ffd640 1 -- 172.21.15.26:0/248289856 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622fc1105a0 con 0x5622fc0f6af0 2024-09-11T23:27:32.875 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.873+0000 7f0f737fe640 1 --2- 172.21.15.26:0/248289856 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5622fc101cb0 0x5622fc119d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:27:32.875 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.874+0000 7f0f72ffd640 1 --2- 172.21.15.26:0/248289856 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5622fc0f6af0 0x5622fc0ff4a0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f0f64006010 tx=0x7f0f64003a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26355d3dda2416c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:27:32.875 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.874+0000 7f0f68ff9640 1 -- 172.21.15.26:0/248289856 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f64011440 con 0x5622fc0f6af0 2024-09-11T23:27:32.875 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.874+0000 7f0f68ff9640 1 -- 172.21.15.26:0/248289856 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f64005520 con 0x5622fc0f6af0 2024-09-11T23:27:32.876 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.874+0000 7f0f68ff9640 1 -- 172.21.15.26:0/248289856 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f64004880 con 0x5622fc0f6af0 2024-09-11T23:27:32.876 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.874+0000 7f0f7c417740 1 -- 172.21.15.26:0/248289856 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5622fc0f6af0 msgr2=0x5622fc0ff4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:32.876 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.874+0000 7f0f7c417740 1 --2- 172.21.15.26:0/248289856 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5622fc0f6af0 0x5622fc0ff4a0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f0f64006010 tx=0x7f0f64003a40 comp rx=0 tx=0).stop 2024-09-11T23:27:32.879 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.875+0000 7f0f7c417740 1 -- 172.21.15.26:0/248289856 shutdown_connections 2024-09-11T23:27:32.879 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.875+0000 7f0f7c417740 1 --2- 172.21.15.26:0/248289856 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5622fc0f7410 0x5622fc1197f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:32.879 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.875+0000 7f0f7c417740 1 --2- 172.21.15.26:0/248289856 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5622fc101cb0 0x5622fc119d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:32.879 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.875+0000 7f0f7c417740 1 --2- 172.21.15.26:0/248289856 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5622fc0f6af0 0x5622fc0ff4a0 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:32.879 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.875+0000 7f0f7c417740 1 -- 172.21.15.26:0/248289856 >> 172.21.15.26:0/248289856 conn(0x5622fc0f3060 msgr2=0x5622fc100c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:27:32.879 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.875+0000 7f0f7c417740 1 -- 172.21.15.26:0/248289856 shutdown_connections 2024-09-11T23:27:32.880 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.875+0000 7f0f7c417740 1 -- 172.21.15.26:0/248289856 wait complete. 2024-09-11T23:27:32.880 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.875+0000 7f0f7c417740 1 -- start start 2024-09-11T23:27:32.880 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.875+0000 7f0f7c417740 1 Processor -- start 2024-09-11T23:27:32.880 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.875+0000 7f0f7c417740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5622fc0f6af0 0x5622fc105830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:32.880 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.875+0000 7f0f7c417740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5622fc0f7410 0x5622fc105d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:32.880 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.875+0000 7f0f7c417740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5622fc101cb0 0x5622fc1062b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:32.880 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.875+0000 7f0f7c417740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f0f741aed50 con 0x5622fc0f6af0 2024-09-11T23:27:32.880 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.875+0000 7f0f7c417740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f0f741ae5c0 con 0x5622fc101cb0 2024-09-11T23:27:32.880 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.875+0000 7f0f7c417740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f0f741ab760 con 0x5622fc0f7410 2024-09-11T23:27:32.880 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.875+0000 7f0f7c417740 10 client.0 _ll_register_callbacks cb 0x7f0f640042b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:27:32.880 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.875+0000 7f0f72ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5622fc0f6af0 0x5622fc105830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:32.880 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.875+0000 7f0f72ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5622fc0f6af0 0x5622fc105830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52614/0 (socket says 172.21.15.26:52614) 2024-09-11T23:27:32.881 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.875+0000 7f0f72ffd640 1 -- 172.21.15.26:0/1756146023 learned_addr learned my addr 172.21.15.26:0/1756146023 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:27:32.881 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.875+0000 7f0f727fc640 1 --2- 172.21.15.26:0/1756146023 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5622fc0f7410 0x5622fc105d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:32.881 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.875+0000 7f0f737fe640 1 --2- 172.21.15.26:0/1756146023 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5622fc101cb0 0x5622fc1062b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:32.881 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.876+0000 7f0f72ffd640 1 -- 172.21.15.26:0/1756146023 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5622fc0f7410 msgr2=0x5622fc105d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:27:32.881 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.876+0000 7f0f72ffd640 1 --2- 172.21.15.26:0/1756146023 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5622fc0f7410 0x5622fc105d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:32.881 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.876+0000 7f0f72ffd640 1 -- 172.21.15.26:0/1756146023 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5622fc101cb0 msgr2=0x5622fc1062b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:27:32.881 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.876+0000 7f0f72ffd640 1 --2- 172.21.15.26:0/1756146023 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5622fc101cb0 0x5622fc1062b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:32.881 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.876+0000 7f0f72ffd640 1 -- 172.21.15.26:0/1756146023 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5622fc17b890 con 0x5622fc0f6af0 2024-09-11T23:27:32.881 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.876+0000 7f0f72ffd640 1 --2- 172.21.15.26:0/1756146023 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5622fc0f6af0 0x5622fc105830 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f0f64023430 tx=0x7f0f640056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58ecca05dfd4d98c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:27:32.881 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.876+0000 7f0f5a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:27:32.881 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.876+0000 7f0f5a7fc640 1 -- 172.21.15.26:0/1756146023 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f640119f0 con 0x5622fc0f6af0 2024-09-11T23:27:32.881 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.876+0000 7f0f5a7fc640 1 -- 172.21.15.26:0/1756146023 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8542+0+0 (secure 0 0 0) 0x7f0f64011b90 con 0x5622fc0f6af0 2024-09-11T23:27:32.881 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.876+0000 7f0f7c417740 1 -- 172.21.15.26:0/1756146023 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0f64005520 con 0x5622fc0f6af0 2024-09-11T23:27:32.882 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.876+0000 7f0f5a7fc640 1 -- 172.21.15.26:0/1756146023 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f0f6403d050 con 0x5622fc0f6af0 2024-09-11T23:27:32.882 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.876+0000 7f0f7c417740 10 client.16869 fetch_fsmap learned FSMap version 51 2024-09-11T23:27:32.882 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.876+0000 7f0f7c417740 1 -- 172.21.15.26:0/1756146023 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x5622fc17caf0 con 0x5622fc0f6af0 2024-09-11T23:27:32.882 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.876+0000 7f0f5a7fc640 1 -- 172.21.15.26:0/1756146023 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f64005520 con 0x5622fc0f6af0 2024-09-11T23:27:32.882 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.877+0000 7f0f5a7fc640 1 -- 172.21.15.26:0/1756146023 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 51) ==== 8394+0+0 (secure 0 0 0) 0x7f0f640048e0 con 0x5622fc0f6af0 2024-09-11T23:27:32.882 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.877+0000 7f0f7c417740 10 client.16869 fetch_fsmap finished waiting for FSMap version 51 2024-09-11T23:27:32.882 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.877+0000 7f0f7c417740 10 client.16869 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:27:32.882 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.877+0000 7f0f7c417740 20 client.16869 populate_metadata read hostname 'smithi026' 2024-09-11T23:27:32.882 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.877+0000 7f0f7c417740 1 --2- 172.21.15.26:0/1756146023 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5622fc17d140 0x5622fc17f530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:32.882 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.877+0000 7f0f7c417740 4 client.16869 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T23:27:32.882 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.877+0000 7f0f7c417740 1 -- 172.21.15.26:0/1756146023 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5622fc062d70 con 0x5622fc17d140 2024-09-11T23:27:32.882 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.877+0000 7f0f727fc640 1 --2- 172.21.15.26:0/1756146023 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5622fc17d140 0x5622fc17f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:32.883 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.877+0000 7f0f727fc640 1 --2- 172.21.15.26:0/1756146023 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5622fc17d140 0x5622fc17f530 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=34bcb78c507aad82 server_cookie=18900a2cced852db in_seq=0 out_seq=0 2024-09-11T23:27:32.883 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.877+0000 7f0f5a7fc640 10 client.16869 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:27:32.894 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.894+0000 7f0f5a7fc640 1 -- 172.21.15.26:0/1756146023 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5622fc062d70 con 0x5622fc17d140 2024-09-11T23:27:32.895 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.894+0000 7f0f5a7fc640 10 client.16869 handle_command_reply: tid=0 2024-09-11T23:27:32.895 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.894+0000 7f0f7c417740 1 client.16869 shutdown 2024-09-11T23:27:32.895 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.894+0000 7f0f7c417740 1 -- 172.21.15.26:0/1756146023 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5622fc0f6af0 msgr2=0x5622fc105830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:32.895 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.894+0000 7f0f7c417740 1 --2- 172.21.15.26:0/1756146023 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5622fc0f6af0 0x5622fc105830 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f0f64023430 tx=0x7f0f640056d0 comp rx=0 tx=0).stop 2024-09-11T23:27:32.895 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.894+0000 7f0f7c417740 1 -- 172.21.15.26:0/1756146023 shutdown_connections 2024-09-11T23:27:32.895 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.894+0000 7f0f7c417740 1 --2- 172.21.15.26:0/1756146023 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5622fc101cb0 0x5622fc1062b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:32.895 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.894+0000 7f0f7c417740 1 --2- 172.21.15.26:0/1756146023 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5622fc17d140 0x5622fc17f530 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:32.895 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.894+0000 7f0f7c417740 1 --2- 172.21.15.26:0/1756146023 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5622fc0f7410 0x5622fc105d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:32.895 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.894+0000 7f0f7c417740 1 --2- 172.21.15.26:0/1756146023 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5622fc0f6af0 0x5622fc105830 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:32.895 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.894+0000 7f0f5a7fc640 0 client.16869 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:27:32.895 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.894+0000 7f0f7c417740 1 -- 172.21.15.26:0/1756146023 >> 172.21.15.26:0/1756146023 conn(0x5622fc0f3060 msgr2=0x5622fc0f19a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:27:32.895 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.894+0000 7f0f7c417740 1 -- 172.21.15.26:0/1756146023 shutdown_connections 2024-09-11T23:27:32.896 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.894+0000 7f0f7c417740 1 -- 172.21.15.26:0/1756146023 wait complete. 2024-09-11T23:27:32.896 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.894+0000 7f0f7c417740 20 client.16869 trim_cache size 0 max 16384 2024-09-11T23:27:32.896 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-09-11T23:27:32.896 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "message": "", 2024-09-11T23:27:32.896 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-09-11T23:27:32.896 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-09-11T23:27:32.897 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.895+0000 7f0f79674640 1 -- 172.21.15.26:0/4065816995 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f0f600a2da0 msgr2=0x7f0f600a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:32.897 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.895+0000 7f0f79674640 1 --2- 172.21.15.26:0/4065816995 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f0f600a2da0 0x7f0f600a5260 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f0f6c005fd0 tx=0x7f0f6c009be0 comp rx=0 tx=0).stop 2024-09-11T23:27:32.898 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.895+0000 7f0f79674640 1 -- 172.21.15.26:0/4065816995 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f741080f0 msgr2=0x7f0f74198860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:32.898 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.895+0000 7f0f79674640 1 --2- 172.21.15.26:0/4065816995 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f741080f0 0x7f0f74198860 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f0f5c005940 tx=0x7f0f5c004270 comp rx=0 tx=0).stop 2024-09-11T23:27:32.898 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.895+0000 7f0f79674640 1 -- 172.21.15.26:0/4065816995 shutdown_connections 2024-09-11T23:27:32.898 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.895+0000 7f0f79674640 1 --2- 172.21.15.26:0/4065816995 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f0f600a2da0 0x7f0f600a5260 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:32.898 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.895+0000 7f0f79674640 1 --2- 172.21.15.26:0/4065816995 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0f7419cbe0 0x7f0f7419d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:32.899 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.895+0000 7f0f79674640 1 --2- 172.21.15.26:0/4065816995 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f741080f0 0x7f0f74198860 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:32.899 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.895+0000 7f0f79674640 1 --2- 172.21.15.26:0/4065816995 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0f741008a0 0x7f0f74198320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:32.899 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.895+0000 7f0f79674640 1 -- 172.21.15.26:0/4065816995 >> 172.21.15.26:0/4065816995 conn(0x7f0f74075ef0 msgr2=0x7f0f740ff6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:27:32.899 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.895+0000 7f0f79674640 1 -- 172.21.15.26:0/4065816995 shutdown_connections 2024-09-11T23:27:32.899 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:32.895+0000 7f0f79674640 1 -- 172.21.15.26:0/4065816995 wait complete. 2024-09-11T23:27:34.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:27:33 smithi026 ceph-mon[31172]: pgmap v284: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 KiB/s wr, 13 op/s 2024-09-11T23:27:34.052 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:27:33 smithi076 ceph-mon[35296]: pgmap v284: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 KiB/s wr, 13 op/s 2024-09-11T23:27:34.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:27:33 smithi136 ceph-mon[35121]: pgmap v284: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 KiB/s wr, 13 op/s 2024-09-11T23:27:34.943 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:27:35.216 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:27:35.965 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:27:35 smithi026 ceph-mon[31172]: pgmap v285: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 68 KiB/s wr, 19 op/s 2024-09-11T23:27:35.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:35.965+0000 7f4d86acc640 1 -- 172.21.15.26:0/3586571591 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4d8010c170 msgr2=0x7f4d8010e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:35.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:35.965+0000 7f4d86acc640 1 --2- 172.21.15.26:0/3586571591 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4d8010c170 0x7f4d8010e610 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f4d78009ca0 tx=0x7f4d7802a950 comp rx=0 tx=0).stop 2024-09-11T23:27:35.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:35.965+0000 7f4d86acc640 1 -- 172.21.15.26:0/3586571591 shutdown_connections 2024-09-11T23:27:35.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:35.965+0000 7f4d86acc640 1 --2- 172.21.15.26:0/3586571591 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4d80103660 0x7f4d8010bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:35.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:35.965+0000 7f4d86acc640 1 --2- 172.21.15.26:0/3586571591 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4d8010c170 0x7f4d8010e610 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:35.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:35.965+0000 7f4d86acc640 1 --2- 172.21.15.26:0/3586571591 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4d80102d40 0x7f4d80103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:35.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:35.965+0000 7f4d86acc640 1 -- 172.21.15.26:0/3586571591 >> 172.21.15.26:0/3586571591 conn(0x7f4d800fc7a0 msgr2=0x7f4d800febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:27:35.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:35.965+0000 7f4d86acc640 1 -- 172.21.15.26:0/3586571591 shutdown_connections 2024-09-11T23:27:35.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:35.965+0000 7f4d86acc640 1 -- 172.21.15.26:0/3586571591 wait complete. 2024-09-11T23:27:35.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:35.966+0000 7f4d86acc640 1 Processor -- start 2024-09-11T23:27:35.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:35.966+0000 7f4d86acc640 1 -- start start 2024-09-11T23:27:35.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:35.966+0000 7f4d86acc640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4d80102d40 0x7f4d801009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:35.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:35.966+0000 7f4d86acc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4d80103660 0x7f4d80100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:35.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:35.966+0000 7f4d86acc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4d8010c170 0x7f4d80101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:35.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:35.966+0000 7f4d86acc640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f4d80111f70 con 0x7f4d8010c170 2024-09-11T23:27:35.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:35.966+0000 7f4d86acc640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f4d80111df0 con 0x7f4d80103660 2024-09-11T23:27:35.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:35.966+0000 7f4d86acc640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f4d801120f0 con 0x7f4d80102d40 2024-09-11T23:27:35.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:35.966+0000 7f4d7ffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4d80103660 0x7f4d80100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:35.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:35.966+0000 7f4d7ffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4d80103660 0x7f4d80100ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:59240/0 (socket says 172.21.15.26:59240) 2024-09-11T23:27:35.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:35.966+0000 7f4d84868640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4d80102d40 0x7f4d801009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:35.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:35.966+0000 7f4d85069640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4d8010c170 0x7f4d80101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:35.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:35.967+0000 7f4d7ffff640 1 -- 172.21.15.26:0/1081092186 learned_addr learned my addr 172.21.15.26:0/1081092186 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:27:35.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:35.967+0000 7f4d85069640 1 -- 172.21.15.26:0/1081092186 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4d80102d40 msgr2=0x7f4d801009a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:35.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:35.967+0000 7f4d85069640 1 --2- 172.21.15.26:0/1081092186 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4d80102d40 0x7f4d801009a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:35.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:35.967+0000 7f4d85069640 1 -- 172.21.15.26:0/1081092186 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4d80103660 msgr2=0x7f4d80100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:35.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:35.967+0000 7f4d85069640 1 --2- 172.21.15.26:0/1081092186 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4d80103660 0x7f4d80100ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:35.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:35.967+0000 7f4d85069640 1 -- 172.21.15.26:0/1081092186 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d801a6890 con 0x7f4d8010c170 2024-09-11T23:27:35.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:35.967+0000 7f4d7ffff640 1 --2- 172.21.15.26:0/1081092186 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4d80103660 0x7f4d80100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:27:35.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:35.967+0000 7f4d85069640 1 --2- 172.21.15.26:0/1081092186 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4d8010c170 0x7f4d80101420 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f4d78005cc0 tx=0x7f4d7802f630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:27:35.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:35.967+0000 7f4d7dffb640 1 -- 172.21.15.26:0/1081092186 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d78034040 con 0x7f4d8010c170 2024-09-11T23:27:35.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:35.967+0000 7f4d7dffb640 1 -- 172.21.15.26:0/1081092186 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d78030920 con 0x7f4d8010c170 2024-09-11T23:27:35.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:35.967+0000 7f4d86acc640 1 -- 172.21.15.26:0/1081092186 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d801a6b20 con 0x7f4d8010c170 2024-09-11T23:27:35.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:35.967+0000 7f4d7dffb640 1 -- 172.21.15.26:0/1081092186 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d78004750 con 0x7f4d8010c170 2024-09-11T23:27:35.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:35.968+0000 7f4d86acc640 1 -- 172.21.15.26:0/1081092186 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d801a7020 con 0x7f4d8010c170 2024-09-11T23:27:35.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:35.969+0000 7f4d86acc640 1 -- 172.21.15.26:0/1081092186 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d80103120 con 0x7f4d8010c170 2024-09-11T23:27:35.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:35.970+0000 7f4d7dffb640 1 -- 172.21.15.26:0/1081092186 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f4d78037020 con 0x7f4d8010c170 2024-09-11T23:27:35.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:35.971+0000 7f4d7dffb640 1 --2- 172.21.15.26:0/1081092186 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f4d700a2d70 0x7f4d700a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:35.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:35.971+0000 7f4d84868640 1 --2- 172.21.15.26:0/1081092186 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f4d700a2d70 0x7f4d700a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:35.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:35.971+0000 7f4d7dffb640 1 -- 172.21.15.26:0/1081092186 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8542+0+0 (secure 0 0 0) 0x7f4d780f00e0 con 0x7f4d8010c170 2024-09-11T23:27:35.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:35.971+0000 7f4d84868640 1 --2- 172.21.15.26:0/1081092186 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f4d700a2d70 0x7f4d700a5230 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f4d7400bbe0 tx=0x7f4d7400b390 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:27:35.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:35.975+0000 7f4d7dffb640 1 -- 172.21.15.26:0/1081092186 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4d780bcea0 con 0x7f4d8010c170 2024-09-11T23:27:36.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:27:35 smithi076 ceph-mon[35296]: pgmap v285: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 68 KiB/s wr, 19 op/s 2024-09-11T23:27:36.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:36.188+0000 7f4d86acc640 1 -- 172.21.15.26:0/1081092186 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4d80107210 con 0x7f4d8010c170 2024-09-11T23:27:36.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:36.189+0000 7f4d7dffb640 1 -- 172.21.15.26:0/1081092186 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) ==== 94+0+11986 (secure 0 0 0) 0x7f4d780bc7a0 con 0x7f4d8010c170 2024-09-11T23:27:36.189 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-11T23:27:36.189 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":51,"btime":"2024-09-11T23:27:01:787119+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14499,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/367659515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":367659515},{"type":"v1","addr":"172.21.15.26:6835","nonce":367659515}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14511,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/2041065299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":2041065299},{"type":"v1","addr":"172.21.15.26:6837","nonce":2041065299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":51,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:16:26.544203+0000","modified":"2024-09-11T23:27:01.632593+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24485,"mds_1":24463,"mds_2":24458,"mds_3":24448,"mds_4":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14550":{"gid":14550,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6839/2233068305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":2233068305},{"type":"v1","addr":"172.21.15.26:6839","nonce":2233068305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14562":{"gid":14562,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6841/2984982336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":2984982336},{"type":"v1","addr":"172.21.15.26:6841","nonce":2984982336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6833/2019841455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":2019841455},{"type":"v1","addr":"172.21.15.136:6833","nonce":2019841455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.76:6833/3438147050","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3438147050},{"type":"v1","addr":"172.21.15.76:6833","nonce":3438147050}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6835/3617665975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3617665975},{"type":"v1","addr":"172.21.15.136:6835","nonce":3617665975}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":12,"addr":"172.21.15.76:6835/393308499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":393308499},{"type":"v1","addr":"172.21.15.76:6835","nonce":393308499}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":10,"addr":"172.21.15.136:6837/1633082387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":1633082387},{"type":"v1","addr":"172.21.15.136:6837","nonce":1633082387}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.76:6837/2078978748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":2078978748},{"type":"v1","addr":"172.21.15.76:6837","nonce":2078978748}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":63,"addr":"172.21.15.136:6839/3211398630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":3211398630},{"type":"v1","addr":"172.21.15.136:6839","nonce":3211398630}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.76:6839/2077637090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6838","nonce":2077637090},{"type":"v1","addr":"172.21.15.76:6839","nonce":2077637090}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24485,"qdb_cluster":[24443,24448,24458,24463,24485]},"id":1}]} 2024-09-11T23:27:36.189 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 51 2024-09-11T23:27:36.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:36.191+0000 7f4d86acc640 1 -- 172.21.15.26:0/1081092186 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f4d700a2d70 msgr2=0x7f4d700a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:36.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:36.191+0000 7f4d86acc640 1 --2- 172.21.15.26:0/1081092186 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f4d700a2d70 0x7f4d700a5230 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f4d7400bbe0 tx=0x7f4d7400b390 comp rx=0 tx=0).stop 2024-09-11T23:27:36.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:36.191+0000 7f4d86acc640 1 -- 172.21.15.26:0/1081092186 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4d8010c170 msgr2=0x7f4d80101420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:36.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:36.191+0000 7f4d86acc640 1 --2- 172.21.15.26:0/1081092186 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4d8010c170 0x7f4d80101420 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f4d78005cc0 tx=0x7f4d7802f630 comp rx=0 tx=0).stop 2024-09-11T23:27:36.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:36.191+0000 7f4d86acc640 1 -- 172.21.15.26:0/1081092186 shutdown_connections 2024-09-11T23:27:36.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:36.191+0000 7f4d86acc640 1 --2- 172.21.15.26:0/1081092186 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f4d700a2d70 0x7f4d700a5230 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:36.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:36.191+0000 7f4d86acc640 1 --2- 172.21.15.26:0/1081092186 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4d80103660 0x7f4d80100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:36.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:36.191+0000 7f4d86acc640 1 --2- 172.21.15.26:0/1081092186 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4d8010c170 0x7f4d80101420 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:36.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:36.191+0000 7f4d86acc640 1 --2- 172.21.15.26:0/1081092186 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4d80102d40 0x7f4d801009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:36.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:36.191+0000 7f4d86acc640 1 -- 172.21.15.26:0/1081092186 >> 172.21.15.26:0/1081092186 conn(0x7f4d800fc7a0 msgr2=0x7f4d800fd400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:27:36.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:36.191+0000 7f4d86acc640 1 -- 172.21.15.26:0/1081092186 shutdown_connections 2024-09-11T23:27:36.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:36.191+0000 7f4d86acc640 1 -- 172.21.15.26:0/1081092186 wait complete. 2024-09-11T23:27:36.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:27:35 smithi136 ceph-mon[35121]: pgmap v285: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 68 KiB/s wr, 19 op/s 2024-09-11T23:27:36.857 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:27:36 smithi026 ceph-mon[31172]: from='client.? 172.21.15.26:0/1081092186' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:27:36.885 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24485 quiesce db --set-id 670d17b7 --release --await' 2024-09-11T23:27:36.886 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24485 quiesce db --set-id 670d17b7 --release --await 2024-09-11T23:27:37.052 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:27:36 smithi076 ceph-mon[35296]: from='client.? 172.21.15.26:0/1081092186' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:27:37.147 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:27:37.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:27:36 smithi136 ceph-mon[35121]: from='client.? 172.21.15.26:0/1081092186' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:27:37.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.625+0000 7f798c5f7640 1 -- 172.21.15.26:0/3010738767 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7984106a80 msgr2=0x7f7984106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:37.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.625+0000 7f798c5f7640 1 --2- 172.21.15.26:0/3010738767 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7984106a80 0x7f7984106e60 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f7980009930 tx=0x7f798002a420 comp rx=0 tx=0).stop 2024-09-11T23:27:37.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.626+0000 7f798c5f7640 1 -- 172.21.15.26:0/3010738767 shutdown_connections 2024-09-11T23:27:37.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.626+0000 7f798c5f7640 1 --2- 172.21.15.26:0/3010738767 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7984100a80 0x7f7984100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:37.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.626+0000 7f798c5f7640 1 --2- 172.21.15.26:0/3010738767 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7984101420 0x7f798410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:37.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.626+0000 7f798c5f7640 1 --2- 172.21.15.26:0/3010738767 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7984106a80 0x7f7984106e60 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:37.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.626+0000 7f798c5f7640 1 -- 172.21.15.26:0/3010738767 >> 172.21.15.26:0/3010738767 conn(0x7f79840fc7a0 msgr2=0x7f79840febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:27:37.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.626+0000 7f798c5f7640 1 -- 172.21.15.26:0/3010738767 shutdown_connections 2024-09-11T23:27:37.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.626+0000 7f798c5f7640 1 -- 172.21.15.26:0/3010738767 wait complete. 2024-09-11T23:27:37.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.626+0000 7f798c5f7640 1 Processor -- start 2024-09-11T23:27:37.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.626+0000 7f798c5f7640 1 -- start start 2024-09-11T23:27:37.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.626+0000 7f798c5f7640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7984100a80 0x7f7984197080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:37.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.626+0000 7f798c5f7640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7984101420 0x7f79841975c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:37.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.626+0000 7f798c5f7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7984106a80 0x7f7984197b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:37.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.626+0000 7f798c5f7640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f7984110e40 con 0x7f7984106a80 2024-09-11T23:27:37.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.626+0000 7f798c5f7640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f7984110cc0 con 0x7f7984100a80 2024-09-11T23:27:37.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.626+0000 7f798c5f7640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f7984110fc0 con 0x7f7984101420 2024-09-11T23:27:37.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.627+0000 7f798ab94640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7984106a80 0x7f7984197b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:37.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.627+0000 7f798ab94640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7984106a80 0x7f7984197b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52660/0 (socket says 172.21.15.26:52660) 2024-09-11T23:27:37.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.627+0000 7f798ab94640 1 -- 172.21.15.26:0/2484564579 learned_addr learned my addr 172.21.15.26:0/2484564579 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:27:37.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.627+0000 7f7989b92640 1 --2- 172.21.15.26:0/2484564579 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7984101420 0x7f79841975c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:37.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.627+0000 7f798a393640 1 --2- 172.21.15.26:0/2484564579 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7984100a80 0x7f7984197080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:37.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.627+0000 7f798ab94640 1 -- 172.21.15.26:0/2484564579 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7984101420 msgr2=0x7f79841975c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:37.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.627+0000 7f798ab94640 1 --2- 172.21.15.26:0/2484564579 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7984101420 0x7f79841975c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:37.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.627+0000 7f798ab94640 1 -- 172.21.15.26:0/2484564579 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7984100a80 msgr2=0x7f7984197080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:37.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.627+0000 7f798ab94640 1 --2- 172.21.15.26:0/2484564579 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7984100a80 0x7f7984197080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:37.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.627+0000 7f798ab94640 1 -- 172.21.15.26:0/2484564579 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79841a68f0 con 0x7f7984106a80 2024-09-11T23:27:37.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.627+0000 7f798a393640 1 --2- 172.21.15.26:0/2484564579 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7984100a80 0x7f7984197080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:27:37.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.627+0000 7f798ab94640 1 --2- 172.21.15.26:0/2484564579 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7984106a80 0x7f7984197b00 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f797c00cad0 tx=0x7f797c00ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:27:37.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.627+0000 7f797b7fe640 1 -- 172.21.15.26:0/2484564579 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f797c00ad60 con 0x7f7984106a80 2024-09-11T23:27:37.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.627+0000 7f798c5f7640 1 -- 172.21.15.26:0/2484564579 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79841a6be0 con 0x7f7984106a80 2024-09-11T23:27:37.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.627+0000 7f797b7fe640 1 -- 172.21.15.26:0/2484564579 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f797c005020 con 0x7f7984106a80 2024-09-11T23:27:37.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.627+0000 7f797b7fe640 1 -- 172.21.15.26:0/2484564579 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f797c00f6b0 con 0x7f7984106a80 2024-09-11T23:27:37.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.627+0000 7f798c5f7640 1 -- 172.21.15.26:0/2484564579 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79841a70d0 con 0x7f7984106a80 2024-09-11T23:27:37.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.630+0000 7f798e9d0740 1 Processor -- start 2024-09-11T23:27:37.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.630+0000 7f797b7fe640 1 -- 172.21.15.26:0/2484564579 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f797c005560 con 0x7f7984106a80 2024-09-11T23:27:37.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.631+0000 7f798e9d0740 1 -- start start 2024-09-11T23:27:37.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.631+0000 7f798e9d0740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5560dad8b060 0x5560dad8b440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:37.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.631+0000 7f798e9d0740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5560dad8b980 0x5560dad95d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:37.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.631+0000 7f798e9d0740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5560dad96240 0x5560dad98630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:37.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.631+0000 7f798e9d0740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5560dacf0620 con 0x5560dad96240 2024-09-11T23:27:37.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.631+0000 7f798e9d0740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5560dac97e50 con 0x5560dad8b060 2024-09-11T23:27:37.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.631+0000 7f798e9d0740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5560dacec5e0 con 0x5560dad8b980 2024-09-11T23:27:37.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.631+0000 7f7989b92640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5560dad8b060 0x5560dad8b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:37.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.631+0000 7f7989b92640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5560dad8b060 0x5560dad8b440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:59264/0 (socket says 172.21.15.26:59264) 2024-09-11T23:27:37.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.631+0000 7f7989b92640 1 -- 172.21.15.26:0/2309733000 learned_addr learned my addr 172.21.15.26:0/2309733000 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:27:37.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.631+0000 7f7989b92640 1 --2- 172.21.15.26:0/2309733000 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5560dad96240 0x5560dad98630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:37.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.631+0000 7f798a393640 1 --2- 172.21.15.26:0/2309733000 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5560dad8b980 0x5560dad95d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:37.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.631+0000 7f7989b92640 1 -- 172.21.15.26:0/2309733000 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5560dad8b980 msgr2=0x5560dad95d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:27:37.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.631+0000 7f7989b92640 1 --2- 172.21.15.26:0/2309733000 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5560dad8b980 0x5560dad95d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:37.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.631+0000 7f7989b92640 1 -- 172.21.15.26:0/2309733000 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5560dad8b060 msgr2=0x5560dad8b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:27:37.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.631+0000 7f7989b92640 1 --2- 172.21.15.26:0/2309733000 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5560dad8b060 0x5560dad8b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:37.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.631+0000 7f797b7fe640 1 --2- 172.21.15.26:0/2484564579 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f79700a2cf0 0x7f79700a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:37.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.631+0000 7f7989b92640 1 -- 172.21.15.26:0/2309733000 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5560dac88d40 con 0x5560dad96240 2024-09-11T23:27:37.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.632+0000 7f798ab94640 1 --2- 172.21.15.26:0/2484564579 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f79700a2cf0 0x7f79700a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:37.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.632+0000 7f7989b92640 1 --2- 172.21.15.26:0/2309733000 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5560dad96240 0x5560dad98630 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f79740117e0 tx=0x7f797400a2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e89a58da3818fdbe server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:27:37.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.632+0000 7f797b7fe640 1 -- 172.21.15.26:0/2484564579 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8542+0+0 (secure 0 0 0) 0x7f797c0cf450 con 0x7f7984106a80 2024-09-11T23:27:37.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.632+0000 7f7978ff9640 1 -- 172.21.15.26:0/2309733000 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f797400d950 con 0x5560dad96240 2024-09-11T23:27:37.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.632+0000 7f7978ff9640 1 -- 172.21.15.26:0/2309733000 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7974005020 con 0x5560dad96240 2024-09-11T23:27:37.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.632+0000 7f7978ff9640 1 -- 172.21.15.26:0/2309733000 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7974004b90 con 0x5560dad96240 2024-09-11T23:27:37.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.633+0000 7f798e9d0740 1 -- 172.21.15.26:0/2309733000 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5560dad96240 msgr2=0x5560dad98630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:37.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.633+0000 7f798ab94640 1 --2- 172.21.15.26:0/2484564579 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f79700a2cf0 0x7f79700a51b0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f797c097dc0 tx=0x7f797c090df0 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:27:37.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.633+0000 7f798e9d0740 1 --2- 172.21.15.26:0/2309733000 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5560dad96240 0x5560dad98630 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f79740117e0 tx=0x7f797400a2d0 comp rx=0 tx=0).stop 2024-09-11T23:27:37.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.633+0000 7f798e9d0740 1 -- 172.21.15.26:0/2309733000 shutdown_connections 2024-09-11T23:27:37.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.633+0000 7f798e9d0740 1 --2- 172.21.15.26:0/2309733000 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5560dad96240 0x5560dad98630 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:37.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.633+0000 7f798e9d0740 1 --2- 172.21.15.26:0/2309733000 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5560dad8b980 0x5560dad95d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:37.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.633+0000 7f798e9d0740 1 --2- 172.21.15.26:0/2309733000 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5560dad8b060 0x5560dad8b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:37.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.633+0000 7f798e9d0740 1 -- 172.21.15.26:0/2309733000 >> 172.21.15.26:0/2309733000 conn(0x5560dad875f0 msgr2=0x5560dad879d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:27:37.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.633+0000 7f798e9d0740 1 -- 172.21.15.26:0/2309733000 shutdown_connections 2024-09-11T23:27:37.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.633+0000 7f798e9d0740 1 -- 172.21.15.26:0/2309733000 wait complete. 2024-09-11T23:27:37.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.633+0000 7f798e9d0740 1 -- start start 2024-09-11T23:27:37.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.633+0000 7f798e9d0740 1 Processor -- start 2024-09-11T23:27:37.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.634+0000 7f798e9d0740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5560dad8b060 0x5560dad93230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:37.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.634+0000 7f798e9d0740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5560dad8b980 0x5560dad93770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:37.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.634+0000 7f798e9d0740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5560dad96240 0x5560dad93cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:37.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.634+0000 7f798e9d0740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5560dad98e10 con 0x5560dad96240 2024-09-11T23:27:37.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.634+0000 7f798e9d0740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5560dac78f10 con 0x5560dad8b060 2024-09-11T23:27:37.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.634+0000 7f798e9d0740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5560dad98f90 con 0x5560dad8b980 2024-09-11T23:27:37.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.634+0000 7f798e9d0740 10 client.0 _ll_register_callbacks cb 0x5560dac9cdb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:27:37.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.634+0000 7f7989b92640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5560dad96240 0x5560dad93cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:37.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.634+0000 7f7989b92640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5560dad96240 0x5560dad93cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52686/0 (socket says 172.21.15.26:52686) 2024-09-11T23:27:37.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.634+0000 7f7989b92640 1 -- 172.21.15.26:0/725510477 learned_addr learned my addr 172.21.15.26:0/725510477 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:27:37.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.634+0000 7f798a393640 1 --2- 172.21.15.26:0/725510477 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5560dad8b980 0x5560dad93770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:37.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.634+0000 7f7989b92640 1 --2- 172.21.15.26:0/725510477 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5560dad8b060 0x5560dad93230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:37.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.634+0000 7f7989b92640 1 -- 172.21.15.26:0/725510477 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5560dad8b980 msgr2=0x5560dad93770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:27:37.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.634+0000 7f7989b92640 1 --2- 172.21.15.26:0/725510477 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5560dad8b980 0x5560dad93770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:37.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.634+0000 7f7989b92640 1 -- 172.21.15.26:0/725510477 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5560dad8b060 msgr2=0x5560dad93230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:27:37.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.634+0000 7f7989b92640 1 --2- 172.21.15.26:0/725510477 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5560dad8b060 0x5560dad93230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:37.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.634+0000 7f7989b92640 1 -- 172.21.15.26:0/725510477 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5560da9ff580 con 0x5560dad96240 2024-09-11T23:27:37.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.634+0000 7f7989b92640 1 --2- 172.21.15.26:0/725510477 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5560dad96240 0x5560dad93cb0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f797400d710 tx=0x7f79740115b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=626ef9d8bdefaf4b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:27:37.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.634+0000 7f796dffb640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:27:37.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.634+0000 7f796dffb640 1 -- 172.21.15.26:0/725510477 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f797400de10 con 0x5560dad96240 2024-09-11T23:27:37.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.634+0000 7f796dffb640 1 -- 172.21.15.26:0/725510477 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8542+0+0 (secure 0 0 0) 0x7f7974004910 con 0x5560dad96240 2024-09-11T23:27:37.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.634+0000 7f798e9d0740 1 -- 172.21.15.26:0/725510477 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7974005020 con 0x5560dad96240 2024-09-11T23:27:37.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.635+0000 7f796dffb640 1 -- 172.21.15.26:0/725510477 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f7974032050 con 0x5560dad96240 2024-09-11T23:27:37.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.635+0000 7f798e9d0740 10 client.16887 fetch_fsmap learned FSMap version 51 2024-09-11T23:27:37.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.635+0000 7f798e9d0740 1 -- 172.21.15.26:0/725510477 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x5560daac9960 con 0x5560dad96240 2024-09-11T23:27:37.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.635+0000 7f796dffb640 1 -- 172.21.15.26:0/725510477 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7974005020 con 0x5560dad96240 2024-09-11T23:27:37.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.635+0000 7f796dffb640 1 -- 172.21.15.26:0/725510477 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 51) ==== 8394+0+0 (secure 0 0 0) 0x7f797402d420 con 0x5560dad96240 2024-09-11T23:27:37.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.635+0000 7f798e9d0740 10 client.16887 fetch_fsmap finished waiting for FSMap version 51 2024-09-11T23:27:37.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.635+0000 7f798e9d0740 10 client.16887 resolve_mds: validated gid 24485 aka daemon mds.b 2024-09-11T23:27:37.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.635+0000 7f798e9d0740 20 client.16887 populate_metadata read hostname 'smithi026' 2024-09-11T23:27:37.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.635+0000 7f798e9d0740 1 --2- 172.21.15.26:0/725510477 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5560dada5d40 0x5560dada8130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:37.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.635+0000 7f798e9d0740 4 client.16887 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:27:37.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.635+0000 7f798e9d0740 1 -- 172.21.15.26:0/725510477 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5560dacfdb90 con 0x5560dada5d40 2024-09-11T23:27:37.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.635+0000 7f798a393640 1 --2- 172.21.15.26:0/725510477 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5560dada5d40 0x5560dada8130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:37.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.636+0000 7f796dffb640 10 client.16887 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:27:37.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.636+0000 7f798a393640 1 --2- 172.21.15.26:0/725510477 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5560dada5d40 0x5560dada8130 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=96e1960e0d00f05b server_cookie=8204f9492102bb03 in_seq=0 out_seq=0 2024-09-11T23:27:37.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.636+0000 7f796dffb640 1 -- 172.21.15.26:0/725510477 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5560dacfdb90 con 0x5560dada5d40 2024-09-11T23:27:37.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.636+0000 7f796dffb640 10 client.16887 handle_command_reply: tid=0 2024-09-11T23:27:37.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.636+0000 7f798e9d0740 1 client.16887 shutdown 2024-09-11T23:27:37.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.637+0000 7f798e9d0740 1 -- 172.21.15.26:0/725510477 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5560dad96240 msgr2=0x5560dad93cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:37.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.637+0000 7f798e9d0740 1 --2- 172.21.15.26:0/725510477 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5560dad96240 0x5560dad93cb0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f797400d710 tx=0x7f79740115b0 comp rx=0 tx=0).stop 2024-09-11T23:27:37.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.637+0000 7f798e9d0740 1 -- 172.21.15.26:0/725510477 shutdown_connections 2024-09-11T23:27:37.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.637+0000 7f798e9d0740 1 --2- 172.21.15.26:0/725510477 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5560dada5d40 0x5560dada8130 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:37.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.637+0000 7f798e9d0740 1 --2- 172.21.15.26:0/725510477 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5560dad96240 0x5560dad93cb0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:37.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.637+0000 7f796dffb640 0 client.16887 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:27:37.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.637+0000 7f798e9d0740 1 --2- 172.21.15.26:0/725510477 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5560dad8b980 0x5560dad93770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:37.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.637+0000 7f798e9d0740 1 --2- 172.21.15.26:0/725510477 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5560dad8b060 0x5560dad93230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:37.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.637+0000 7f798e9d0740 1 -- 172.21.15.26:0/725510477 >> 172.21.15.26:0/725510477 conn(0x5560dad875f0 msgr2=0x5560dad8d320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:27:37.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.637+0000 7f798e9d0740 1 -- 172.21.15.26:0/725510477 shutdown_connections 2024-09-11T23:27:37.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.637+0000 7f798e9d0740 1 -- 172.21.15.26:0/725510477 wait complete. 2024-09-11T23:27:37.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.637+0000 7f798e9d0740 20 client.16887 trim_cache size 0 max 16384 2024-09-11T23:27:37.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.654+0000 7f798e9d0740 1 Processor -- start 2024-09-11T23:27:37.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.654+0000 7f798e9d0740 1 -- start start 2024-09-11T23:27:37.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.654+0000 7f798e9d0740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5560dad8b060 0x5560dadad820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:37.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.654+0000 7f798e9d0740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5560dad8b980 0x5560dadadd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:37.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.654+0000 7f798e9d0740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5560dad96240 0x5560dad90490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:37.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.654+0000 7f798e9d0740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f79841afb20 con 0x5560dad8b060 2024-09-11T23:27:37.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.654+0000 7f798e9d0740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f79841b17e0 con 0x5560dad96240 2024-09-11T23:27:37.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.654+0000 7f798e9d0740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f79841b01f0 con 0x5560dad8b980 2024-09-11T23:27:37.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.654+0000 7f7989b92640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5560dad8b060 0x5560dadad820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:37.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.654+0000 7f7989b92640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5560dad8b060 0x5560dadad820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52694/0 (socket says 172.21.15.26:52694) 2024-09-11T23:27:37.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.655+0000 7f7989b92640 1 -- 172.21.15.26:0/1946883026 learned_addr learned my addr 172.21.15.26:0/1946883026 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:27:37.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.655+0000 7f798a393640 1 --2- 172.21.15.26:0/1946883026 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5560dad8b980 0x5560dadadd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:37.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.655+0000 7f7989b92640 1 -- 172.21.15.26:0/1946883026 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5560dad8b980 msgr2=0x5560dadadd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:27:37.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.655+0000 7f7989b92640 1 --2- 172.21.15.26:0/1946883026 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5560dad8b980 0x5560dadadd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:37.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.655+0000 7f7989b92640 1 -- 172.21.15.26:0/1946883026 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5560dad96240 msgr2=0x5560dad90490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:27:37.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.655+0000 7f7989b92640 1 --2- 172.21.15.26:0/1946883026 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5560dad96240 0x5560dad90490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:37.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.655+0000 7f7989b92640 1 -- 172.21.15.26:0/1946883026 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5560dad85bb0 con 0x5560dad8b060 2024-09-11T23:27:37.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.655+0000 7f7989b92640 1 --2- 172.21.15.26:0/1946883026 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5560dad8b060 0x5560dadad820 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f797400dbd0 tx=0x7f7974011780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c273d3217a89492 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:27:37.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.655+0000 7f796ffff640 1 -- 172.21.15.26:0/1946883026 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f797403b070 con 0x5560dad8b060 2024-09-11T23:27:37.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.655+0000 7f796ffff640 1 -- 172.21.15.26:0/1946883026 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7974028040 con 0x5560dad8b060 2024-09-11T23:27:37.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.655+0000 7f796ffff640 1 -- 172.21.15.26:0/1946883026 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7974015070 con 0x5560dad8b060 2024-09-11T23:27:37.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.656+0000 7f798e9d0740 1 -- 172.21.15.26:0/1946883026 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5560dad8b060 msgr2=0x5560dadad820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:37.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.656+0000 7f798e9d0740 1 --2- 172.21.15.26:0/1946883026 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5560dad8b060 0x5560dadad820 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f797400dbd0 tx=0x7f7974011780 comp rx=0 tx=0).stop 2024-09-11T23:27:37.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.656+0000 7f798e9d0740 1 -- 172.21.15.26:0/1946883026 shutdown_connections 2024-09-11T23:27:37.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.656+0000 7f798e9d0740 1 --2- 172.21.15.26:0/1946883026 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5560dad96240 0x5560dad90490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:37.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.656+0000 7f798e9d0740 1 --2- 172.21.15.26:0/1946883026 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5560dad8b980 0x5560dadadd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:37.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.656+0000 7f798e9d0740 1 --2- 172.21.15.26:0/1946883026 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5560dad8b060 0x5560dadad820 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:37.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.656+0000 7f798e9d0740 1 -- 172.21.15.26:0/1946883026 >> 172.21.15.26:0/1946883026 conn(0x5560dad875f0 msgr2=0x5560dad931f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:27:37.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.656+0000 7f798e9d0740 1 -- 172.21.15.26:0/1946883026 shutdown_connections 2024-09-11T23:27:37.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.656+0000 7f798e9d0740 1 -- 172.21.15.26:0/1946883026 wait complete. 2024-09-11T23:27:37.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.656+0000 7f798e9d0740 1 -- start start 2024-09-11T23:27:37.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.656+0000 7f798e9d0740 1 Processor -- start 2024-09-11T23:27:37.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.657+0000 7f798e9d0740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5560dad8b060 0x5560dad9a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:37.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.657+0000 7f798e9d0740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5560dad8b980 0x5560dad9a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:37.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.657+0000 7f798e9d0740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5560dad96240 0x5560dad9abc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:37.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.657+0000 7f798e9d0740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f79841b1110 con 0x5560dad8b980 2024-09-11T23:27:37.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.657+0000 7f798e9d0740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f79841b0980 con 0x5560dad96240 2024-09-11T23:27:37.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.657+0000 7f798e9d0740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f79841adb20 con 0x5560dad8b060 2024-09-11T23:27:37.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.657+0000 7f798e9d0740 10 client.0 _ll_register_callbacks cb 0x7f7974005660 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:27:37.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.657+0000 7f798a393640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5560dad8b980 0x5560dad9a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:37.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.657+0000 7f798a393640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5560dad8b980 0x5560dad9a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52710/0 (socket says 172.21.15.26:52710) 2024-09-11T23:27:37.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.657+0000 7f798a393640 1 -- 172.21.15.26:0/1868214751 learned_addr learned my addr 172.21.15.26:0/1868214751 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:27:37.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.657+0000 7f7989b92640 1 --2- 172.21.15.26:0/1868214751 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5560dad8b060 0x5560dad9a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:37.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.657+0000 7f7989b92640 1 --2- 172.21.15.26:0/1868214751 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5560dad96240 0x5560dad9abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:37.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.657+0000 7f798a393640 1 -- 172.21.15.26:0/1868214751 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5560dad8b060 msgr2=0x5560dad9a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:27:37.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.657+0000 7f798a393640 1 --2- 172.21.15.26:0/1868214751 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5560dad8b060 0x5560dad9a140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:37.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.657+0000 7f798a393640 1 -- 172.21.15.26:0/1868214751 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5560dad96240 msgr2=0x5560dad9abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:27:37.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.657+0000 7f798a393640 1 --2- 172.21.15.26:0/1868214751 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5560dad96240 0x5560dad9abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:37.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.657+0000 7f798a393640 1 -- 172.21.15.26:0/1868214751 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5560dae0f9f0 con 0x5560dad8b980 2024-09-11T23:27:37.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.657+0000 7f798a393640 1 --2- 172.21.15.26:0/1868214751 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5560dad8b980 0x5560dad9a680 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f798002a940 tx=0x7f7980031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=497a91f4761fa025 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:27:37.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.657+0000 7f796dffb640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:27:37.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.657+0000 7f796dffb640 1 -- 172.21.15.26:0/1868214751 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7980035040 con 0x5560dad8b980 2024-09-11T23:27:37.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.657+0000 7f796dffb640 1 -- 172.21.15.26:0/1868214751 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8542+0+0 (secure 0 0 0) 0x7f7980004450 con 0x5560dad8b980 2024-09-11T23:27:37.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.657+0000 7f798e9d0740 1 -- 172.21.15.26:0/1868214751 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7974028040 con 0x5560dad8b980 2024-09-11T23:27:37.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.658+0000 7f796dffb640 1 -- 172.21.15.26:0/1868214751 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f7980005020 con 0x5560dad8b980 2024-09-11T23:27:37.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.658+0000 7f798e9d0740 10 client.16893 fetch_fsmap learned FSMap version 51 2024-09-11T23:27:37.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.658+0000 7f798e9d0740 1 -- 172.21.15.26:0/1868214751 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x5560dae10d70 con 0x5560dad8b980 2024-09-11T23:27:37.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.658+0000 7f796dffb640 1 -- 172.21.15.26:0/1868214751 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7974028040 con 0x5560dad8b980 2024-09-11T23:27:37.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.658+0000 7f796dffb640 1 -- 172.21.15.26:0/1868214751 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 51) ==== 8394+0+0 (secure 0 0 0) 0x7f79800035f0 con 0x5560dad8b980 2024-09-11T23:27:37.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.658+0000 7f798e9d0740 10 client.16893 fetch_fsmap finished waiting for FSMap version 51 2024-09-11T23:27:37.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.658+0000 7f798e9d0740 10 client.16893 resolve_mds: validated gid 24485 aka daemon mds.b 2024-09-11T23:27:37.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.658+0000 7f798e9d0740 20 client.16893 populate_metadata read hostname 'smithi026' 2024-09-11T23:27:37.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.658+0000 7f798e9d0740 1 --2- 172.21.15.26:0/1868214751 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5560dae114d0 0x5560dae138f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:37.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.658+0000 7f798e9d0740 4 client.16893 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "670d17b7", "release": true, "await": true}] 2024-09-11T23:27:37.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.658+0000 7f798e9d0740 1 -- 172.21.15.26:0/1868214751 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "quiesce db", "set_id": "670d17b7", "release": true, "await": true}) -- 0x5560dacf80a0 con 0x5560dae114d0 2024-09-11T23:27:37.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.658+0000 7f7989b92640 1 --2- 172.21.15.26:0/1868214751 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5560dae114d0 0x5560dae138f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:37.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.659+0000 7f7989b92640 1 --2- 172.21.15.26:0/1868214751 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5560dae114d0 0x5560dae138f0 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f8b23dc919278856 server_cookie=15a4858171e033c9 in_seq=0 out_seq=0 2024-09-11T23:27:37.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:37.659+0000 7f796dffb640 10 client.16893 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:27:38.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:27:37 smithi026 ceph-mon[31172]: pgmap v286: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 36 KiB/s wr, 15 op/s 2024-09-11T23:27:38.052 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:27:37 smithi076 ceph-mon[35296]: pgmap v286: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 36 KiB/s wr, 15 op/s 2024-09-11T23:27:38.238 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:27:37 smithi136 ceph-mon[35121]: pgmap v286: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 36 KiB/s wr, 15 op/s 2024-09-11T23:27:39.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:39.129+0000 7f796dffb640 1 -- 172.21.15.26:0/1868214751 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5560dacf80a0 con 0x5560dae114d0 2024-09-11T23:27:39.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:39.129+0000 7f796dffb640 10 client.16893 handle_command_reply: tid=0 2024-09-11T23:27:39.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:39.129+0000 7f798e9d0740 1 client.16893 shutdown 2024-09-11T23:27:39.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:39.129+0000 7f798e9d0740 1 -- 172.21.15.26:0/1868214751 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5560dad8b980 msgr2=0x5560dad9a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:39.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:39.129+0000 7f798e9d0740 1 --2- 172.21.15.26:0/1868214751 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5560dad8b980 0x5560dad9a680 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f798002a940 tx=0x7f7980031a50 comp rx=0 tx=0).stop 2024-09-11T23:27:39.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:39.129+0000 7f798e9d0740 1 -- 172.21.15.26:0/1868214751 shutdown_connections 2024-09-11T23:27:39.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:39.129+0000 7f798e9d0740 1 --2- 172.21.15.26:0/1868214751 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5560dad96240 0x5560dad9abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:39.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:39.129+0000 7f798e9d0740 1 --2- 172.21.15.26:0/1868214751 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5560dae114d0 0x5560dae138f0 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:39.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:39.129+0000 7f798e9d0740 1 --2- 172.21.15.26:0/1868214751 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5560dad8b980 0x5560dad9a680 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:39.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:39.129+0000 7f796dffb640 0 client.16893 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:27:39.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:39.129+0000 7f798e9d0740 1 --2- 172.21.15.26:0/1868214751 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5560dad8b060 0x5560dad9a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:39.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:39.129+0000 7f798e9d0740 1 -- 172.21.15.26:0/1868214751 >> 172.21.15.26:0/1868214751 conn(0x5560dad875f0 msgr2=0x5560dad85f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:27:39.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:39.129+0000 7f798e9d0740 1 -- 172.21.15.26:0/1868214751 shutdown_connections 2024-09-11T23:27:39.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:39.129+0000 7f798e9d0740 1 -- 172.21.15.26:0/1868214751 wait complete. 2024-09-11T23:27:39.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:39.130+0000 7f798e9d0740 20 client.16893 trim_cache size 0 max 16384 2024-09-11T23:27:39.131 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-11T23:27:39.131 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 51, 2024-09-11T23:27:39.131 INFO:teuthology.orchestra.run.smithi026.stdout: "leader": 24485, 2024-09-11T23:27:39.131 INFO:teuthology.orchestra.run.smithi026.stdout: "set_version": 36, 2024-09-11T23:27:39.131 INFO:teuthology.orchestra.run.smithi026.stdout: "sets": { 2024-09-11T23:27:39.131 INFO:teuthology.orchestra.run.smithi026.stdout: "670d17b7": { 2024-09-11T23:27:39.132 INFO:teuthology.orchestra.run.smithi026.stdout: "version": 36, 2024-09-11T23:27:39.132 INFO:teuthology.orchestra.run.smithi026.stdout: "age_ref": 0.0, 2024-09-11T23:27:39.132 INFO:teuthology.orchestra.run.smithi026.stdout: "state": { 2024-09-11T23:27:39.132 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "RELEASED", 2024-09-11T23:27:39.132 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.0 2024-09-11T23:27:39.132 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:27:39.132 INFO:teuthology.orchestra.run.smithi026.stdout: "timeout": 90.0, 2024-09-11T23:27:39.132 INFO:teuthology.orchestra.run.smithi026.stdout: "expiration": 139.4, 2024-09-11T23:27:39.132 INFO:teuthology.orchestra.run.smithi026.stdout: "members": { 2024-09-11T23:27:39.132 INFO:teuthology.orchestra.run.smithi026.stdout: "file:/": { 2024-09-11T23:27:39.132 INFO:teuthology.orchestra.run.smithi026.stdout: "excluded": false, 2024-09-11T23:27:39.132 INFO:teuthology.orchestra.run.smithi026.stdout: "state": { 2024-09-11T23:27:39.132 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "RELEASED", 2024-09-11T23:27:39.133 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.0 2024-09-11T23:27:39.133 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:27:39.133 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:27:39.133 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:27:39.133 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:27:39.133 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:27:39.133 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-11T23:27:39.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:39.130+0000 7f798c5f7640 1 -- 172.21.15.26:0/2484564579 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f79700a2cf0 msgr2=0x7f79700a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:39.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:39.130+0000 7f798c5f7640 1 --2- 172.21.15.26:0/2484564579 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f79700a2cf0 0x7f79700a51b0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f797c097dc0 tx=0x7f797c090df0 comp rx=0 tx=0).stop 2024-09-11T23:27:39.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:39.130+0000 7f798c5f7640 1 -- 172.21.15.26:0/2484564579 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7984106a80 msgr2=0x7f7984197b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:39.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:39.130+0000 7f798c5f7640 1 --2- 172.21.15.26:0/2484564579 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7984106a80 0x7f7984197b00 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f797c00cad0 tx=0x7f797c00ced0 comp rx=0 tx=0).stop 2024-09-11T23:27:39.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:39.130+0000 7f798c5f7640 1 -- 172.21.15.26:0/2484564579 shutdown_connections 2024-09-11T23:27:39.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:39.130+0000 7f798c5f7640 1 --2- 172.21.15.26:0/2484564579 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f79700a2cf0 0x7f79700a51b0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:39.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:39.130+0000 7f798c5f7640 1 --2- 172.21.15.26:0/2484564579 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7984106a80 0x7f7984197b00 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:39.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:39.130+0000 7f798c5f7640 1 --2- 172.21.15.26:0/2484564579 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7984101420 0x7f79841975c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:39.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:39.130+0000 7f798c5f7640 1 --2- 172.21.15.26:0/2484564579 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7984100a80 0x7f7984197080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:39.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:39.131+0000 7f798c5f7640 1 -- 172.21.15.26:0/2484564579 >> 172.21.15.26:0/2484564579 conn(0x7f79840fc7a0 msgr2=0x7f79840fcf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:27:39.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:39.131+0000 7f798c5f7640 1 -- 172.21.15.26:0/2484564579 shutdown_connections 2024-09-11T23:27:39.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:27:39.131+0000 7f798c5f7640 1 -- 172.21.15.26:0/2484564579 wait complete. 2024-09-11T23:27:39.857 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:27:39 smithi026 ceph-mon[31172]: pgmap v287: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 36 KiB/s wr, 13 op/s 2024-09-11T23:27:39.891 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '670d17b7', total seconds elapsed: 24.3 2024-09-11T23:27:40.052 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:27:39 smithi076 ceph-mon[35296]: pgmap v287: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 36 KiB/s wr, 13 op/s 2024-09-11T23:27:40.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:27:39 smithi136 ceph-mon[35121]: pgmap v287: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 36 KiB/s wr, 13 op/s 2024-09-11T23:27:42.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:27:41 smithi076 ceph-mon[35296]: pgmap v288: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 37 KiB/s wr, 17 op/s 2024-09-11T23:27:42.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:27:41 smithi136 ceph-mon[35121]: pgmap v288: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 37 KiB/s wr, 17 op/s 2024-09-11T23:27:42.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:27:41 smithi026 ceph-mon[31172]: pgmap v288: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 37 KiB/s wr, 17 op/s 2024-09-11T23:27:44.052 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:27:43 smithi076 ceph-mon[35296]: pgmap v289: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 32 KiB/s wr, 12 op/s 2024-09-11T23:27:44.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:27:43 smithi136 ceph-mon[35121]: pgmap v289: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 32 KiB/s wr, 12 op/s 2024-09-11T23:27:44.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:27:43 smithi026 ceph-mon[31172]: pgmap v289: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 32 KiB/s wr, 12 op/s 2024-09-11T23:27:46.000 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.000+0000 7ff2549fe640 1 -- 172.21.15.26:0/1009904182 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff250101170 msgr2=0x7ff25010d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:46.000 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.000+0000 7ff2549fe640 1 --2- 172.21.15.26:0/1009904182 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff250101170 0x7ff25010d6a0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7ff248009cd0 tx=0x7ff24802a8e0 comp rx=0 tx=0).stop 2024-09-11T23:27:46.000 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.000+0000 7ff2549fe640 1 -- 172.21.15.26:0/1009904182 shutdown_connections 2024-09-11T23:27:46.001 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.000+0000 7ff2549fe640 1 --2- 172.21.15.26:0/1009904182 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff2501007d0 0x7ff250100c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:46.001 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.000+0000 7ff2549fe640 1 --2- 172.21.15.26:0/1009904182 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff250101170 0x7ff25010d6a0 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:46.001 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.000+0000 7ff2549fe640 1 --2- 172.21.15.26:0/1009904182 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff2501067e0 0x7ff250106bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:46.001 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.000+0000 7ff2549fe640 1 -- 172.21.15.26:0/1009904182 >> 172.21.15.26:0/1009904182 conn(0x7ff2500777c0 msgr2=0x7ff2500feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:27:46.001 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.000+0000 7ff2549fe640 1 -- 172.21.15.26:0/1009904182 shutdown_connections 2024-09-11T23:27:46.001 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.000+0000 7ff2549fe640 1 -- 172.21.15.26:0/1009904182 wait complete. 2024-09-11T23:27:46.003 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.001+0000 7ff2549fe640 1 Processor -- start 2024-09-11T23:27:46.003 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.001+0000 7ff2549fe640 1 -- start start 2024-09-11T23:27:46.003 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.002+0000 7ff2549fe640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff2501007d0 0x7ff2501971f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:46.003 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.002+0000 7ff2549fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff250101170 0x7ff250197730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:46.003 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.002+0000 7ff2549fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2501067e0 0x7ff25019ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:46.003 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.002+0000 7ff2549fe640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff250110c20 con 0x7ff2501067e0 2024-09-11T23:27:46.003 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.002+0000 7ff2549fe640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7ff250110aa0 con 0x7ff250101170 2024-09-11T23:27:46.003 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.002+0000 7ff2549fe640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ff250110da0 con 0x7ff2501007d0 2024-09-11T23:27:46.003 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.002+0000 7ff24f59e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2501067e0 0x7ff25019ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:46.003 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.002+0000 7ff24f59e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2501067e0 0x7ff25019ba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51884/0 (socket says 172.21.15.26:51884) 2024-09-11T23:27:46.004 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.002+0000 7ff24f59e640 1 -- 172.21.15.26:0/2154682632 learned_addr learned my addr 172.21.15.26:0/2154682632 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:27:46.004 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.002+0000 7ff24ed9d640 1 --2- 172.21.15.26:0/2154682632 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff2501007d0 0x7ff2501971f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:46.004 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.002+0000 7ff24e59c640 1 --2- 172.21.15.26:0/2154682632 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff250101170 0x7ff250197730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:46.004 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.002+0000 7ff24f59e640 1 -- 172.21.15.26:0/2154682632 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff2501007d0 msgr2=0x7ff2501971f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:46.004 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.002+0000 7ff24f59e640 1 --2- 172.21.15.26:0/2154682632 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff2501007d0 0x7ff2501971f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:46.004 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.002+0000 7ff24f59e640 1 -- 172.21.15.26:0/2154682632 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff250101170 msgr2=0x7ff250197730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:46.004 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.002+0000 7ff24f59e640 1 --2- 172.21.15.26:0/2154682632 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff250101170 0x7ff250197730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:46.004 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.002+0000 7ff24f59e640 1 -- 172.21.15.26:0/2154682632 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff25019c140 con 0x7ff2501067e0 2024-09-11T23:27:46.004 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.002+0000 7ff24f59e640 1 --2- 172.21.15.26:0/2154682632 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2501067e0 0x7ff25019ba10 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7ff248005cc0 tx=0x7ff248003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:27:46.004 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.003+0000 7ff237fff640 1 -- 172.21.15.26:0/2154682632 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff248004040 con 0x7ff2501067e0 2024-09-11T23:27:46.004 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.003+0000 7ff237fff640 1 -- 172.21.15.26:0/2154682632 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff2480046c0 con 0x7ff2501067e0 2024-09-11T23:27:46.004 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.003+0000 7ff237fff640 1 -- 172.21.15.26:0/2154682632 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff248046480 con 0x7ff2501067e0 2024-09-11T23:27:46.005 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.003+0000 7ff2549fe640 1 -- 172.21.15.26:0/2154682632 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff25019c3d0 con 0x7ff2501067e0 2024-09-11T23:27:46.005 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.003+0000 7ff2549fe640 1 -- 172.21.15.26:0/2154682632 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2501a3cc0 con 0x7ff2501067e0 2024-09-11T23:27:46.011 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.005+0000 7ff237fff640 1 -- 172.21.15.26:0/2154682632 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7ff2480305a0 con 0x7ff2501067e0 2024-09-11T23:27:46.012 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.005+0000 7ff25806e740 1 Processor -- start 2024-09-11T23:27:46.012 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.005+0000 7ff25806e740 1 -- start start 2024-09-11T23:27:46.012 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.006+0000 7ff237fff640 1 --2- 172.21.15.26:0/2154682632 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7ff2380a2cf0 0x7ff2380a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:46.012 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.006+0000 7ff25806e740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5630ef21a360 0x5630ef218b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:46.012 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.006+0000 7ff25806e740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5630ef219110 0x5630ef219570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:46.012 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.006+0000 7ff25806e740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5630ef227f40 0x5630ef22a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:46.012 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.006+0000 7ff25806e740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5630ef17d2f0 con 0x5630ef219110 2024-09-11T23:27:46.012 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.006+0000 7ff25806e740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5630ef125270 con 0x5630ef21a360 2024-09-11T23:27:46.012 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.006+0000 7ff25806e740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5630ef1792b0 con 0x5630ef227f40 2024-09-11T23:27:46.012 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.006+0000 7ff24e59c640 1 --2- 172.21.15.26:0/2154682632 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7ff2380a2cf0 0x7ff2380a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:46.012 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.006+0000 7ff237fff640 1 -- 172.21.15.26:0/2154682632 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8542+0+0 (secure 0 0 0) 0x7ff24802e030 con 0x7ff2501067e0 2024-09-11T23:27:46.012 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.006+0000 7ff24f59e640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5630ef227f40 0x5630ef22a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:46.012 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.006+0000 7ff24e59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5630ef219110 0x5630ef219570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:46.013 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.006+0000 7ff24f59e640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5630ef227f40 0x5630ef22a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:32962/0 (socket says 172.21.15.26:32962) 2024-09-11T23:27:46.013 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.006+0000 7ff24f59e640 1 -- 172.21.15.26:0/997582635 learned_addr learned my addr 172.21.15.26:0/997582635 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:27:46.013 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.006+0000 7ff24ed9d640 1 --2- 172.21.15.26:0/997582635 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5630ef21a360 0x5630ef218b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:46.013 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.006+0000 7ff24f59e640 1 -- 172.21.15.26:0/997582635 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5630ef21a360 msgr2=0x5630ef218b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:27:46.013 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.006+0000 7ff24f59e640 1 --2- 172.21.15.26:0/997582635 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5630ef21a360 0x5630ef218b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:46.013 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.006+0000 7ff24f59e640 1 -- 172.21.15.26:0/997582635 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5630ef219110 msgr2=0x5630ef219570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:27:46.013 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.006+0000 7ff24f59e640 1 --2- 172.21.15.26:0/997582635 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5630ef219110 0x5630ef219570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:46.013 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.006+0000 7ff24f59e640 1 -- 172.21.15.26:0/997582635 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630ef124340 con 0x5630ef227f40 2024-09-11T23:27:46.013 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.007+0000 7ff24e59c640 1 --2- 172.21.15.26:0/2154682632 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7ff2380a2cf0 0x7ff2380a51b0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7ff24400bfd0 tx=0x7ff24400bb40 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:27:46.013 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.007+0000 7ff24f59e640 1 --2- 172.21.15.26:0/997582635 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5630ef227f40 0x5630ef22a350 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ff2480b9880 tx=0x7ff24802a8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=224124303c59ac4f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:27:46.013 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.007+0000 7ff2357fa640 1 -- 172.21.15.26:0/997582635 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff248087070 con 0x5630ef227f40 2024-09-11T23:27:46.013 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.007+0000 7ff2357fa640 1 -- 172.21.15.26:0/997582635 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff248036020 con 0x5630ef227f40 2024-09-11T23:27:46.013 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.007+0000 7ff2357fa640 1 -- 172.21.15.26:0/997582635 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2480bceb0 con 0x5630ef227f40 2024-09-11T23:27:46.014 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.008+0000 7ff25806e740 1 -- 172.21.15.26:0/997582635 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5630ef227f40 msgr2=0x5630ef22a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:46.014 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.008+0000 7ff25806e740 1 --2- 172.21.15.26:0/997582635 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5630ef227f40 0x5630ef22a350 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ff2480b9880 tx=0x7ff24802a8e0 comp rx=0 tx=0).stop 2024-09-11T23:27:46.014 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.008+0000 7ff25806e740 1 -- 172.21.15.26:0/997582635 shutdown_connections 2024-09-11T23:27:46.014 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.008+0000 7ff25806e740 1 --2- 172.21.15.26:0/997582635 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5630ef227f40 0x5630ef22a350 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:46.014 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.008+0000 7ff25806e740 1 --2- 172.21.15.26:0/997582635 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5630ef219110 0x5630ef219570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:46.014 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.008+0000 7ff25806e740 1 --2- 172.21.15.26:0/997582635 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5630ef21a360 0x5630ef218b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:46.014 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.008+0000 7ff25806e740 1 -- 172.21.15.26:0/997582635 >> 172.21.15.26:0/997582635 conn(0x5630ef215230 msgr2=0x5630ef215610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:27:46.014 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.008+0000 7ff25806e740 1 -- 172.21.15.26:0/997582635 shutdown_connections 2024-09-11T23:27:46.014 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.008+0000 7ff25806e740 1 -- 172.21.15.26:0/997582635 wait complete. 2024-09-11T23:27:46.014 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.008+0000 7ff25806e740 1 -- start start 2024-09-11T23:27:46.014 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.008+0000 7ff25806e740 1 Processor -- start 2024-09-11T23:27:46.014 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.008+0000 7ff25806e740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5630ef219110 0x5630ef224f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:46.014 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.008+0000 7ff25806e740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5630ef21a360 0x5630ef225440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:46.014 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.008+0000 7ff25806e740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5630ef227f40 0x5630ef225980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:46.014 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.008+0000 7ff25806e740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5630ef22ac30 con 0x5630ef21a360 2024-09-11T23:27:46.015 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.008+0000 7ff25806e740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5630ef106290 con 0x5630ef227f40 2024-09-11T23:27:46.015 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.008+0000 7ff25806e740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5630ef22adb0 con 0x5630ef219110 2024-09-11T23:27:46.015 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.008+0000 7ff25806e740 10 client.0 _ll_register_callbacks cb 0x5630ef137140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:27:46.015 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.009+0000 7ff24ed9d640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5630ef219110 0x5630ef224f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:46.015 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.009+0000 7ff24e59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5630ef21a360 0x5630ef225440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:46.015 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.009+0000 7ff24ed9d640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5630ef219110 0x5630ef224f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:32974/0 (socket says 172.21.15.26:32974) 2024-09-11T23:27:46.015 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.009+0000 7ff24ed9d640 1 -- 172.21.15.26:0/3821000499 learned_addr learned my addr 172.21.15.26:0/3821000499 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:27:46.015 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.009+0000 7ff24f59e640 1 --2- 172.21.15.26:0/3821000499 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5630ef227f40 0x5630ef225980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:46.015 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.009+0000 7ff24e59c640 1 -- 172.21.15.26:0/3821000499 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5630ef219110 msgr2=0x5630ef224f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:27:46.015 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.009+0000 7ff24e59c640 1 --2- 172.21.15.26:0/3821000499 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5630ef219110 0x5630ef224f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:46.015 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.009+0000 7ff24e59c640 1 -- 172.21.15.26:0/3821000499 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5630ef227f40 msgr2=0x5630ef225980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:27:46.015 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.009+0000 7ff24e59c640 1 --2- 172.21.15.26:0/3821000499 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5630ef227f40 0x5630ef225980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:46.015 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.009+0000 7ff24e59c640 1 -- 172.21.15.26:0/3821000499 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5630eee90210 con 0x5630ef21a360 2024-09-11T23:27:46.015 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.009+0000 7ff24e59c640 1 --2- 172.21.15.26:0/3821000499 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5630ef21a360 0x5630ef225440 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x5630ef21a740 tx=0x7ff24400b410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21472c080a1b77bf server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:27:46.016 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.009+0000 7ff232ff5640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:27:46.016 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.009+0000 7ff232ff5640 1 -- 172.21.15.26:0/3821000499 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff24401ebd0 con 0x5630ef21a360 2024-09-11T23:27:46.016 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.009+0000 7ff232ff5640 1 -- 172.21.15.26:0/3821000499 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8542+0+0 (secure 0 0 0) 0x7ff24401ed70 con 0x5630ef21a360 2024-09-11T23:27:46.016 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.009+0000 7ff25806e740 1 -- 172.21.15.26:0/3821000499 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff248036020 con 0x5630ef21a360 2024-09-11T23:27:46.016 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.009+0000 7ff232ff5640 1 -- 172.21.15.26:0/3821000499 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7ff24402a410 con 0x5630ef21a360 2024-09-11T23:27:46.016 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.010+0000 7ff25806e740 10 client.16905 fetch_fsmap learned FSMap version 51 2024-09-11T23:27:46.016 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.010+0000 7ff25806e740 1 -- 172.21.15.26:0/3821000499 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x5630eef5c620 con 0x5630ef21a360 2024-09-11T23:27:46.016 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.010+0000 7ff232ff5640 1 -- 172.21.15.26:0/3821000499 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff248036020 con 0x5630ef21a360 2024-09-11T23:27:46.016 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.010+0000 7ff232ff5640 1 -- 172.21.15.26:0/3821000499 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 51) ==== 8394+0+0 (secure 0 0 0) 0x7ff24401f070 con 0x5630ef21a360 2024-09-11T23:27:46.016 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.010+0000 7ff25806e740 10 client.16905 fetch_fsmap finished waiting for FSMap version 51 2024-09-11T23:27:46.016 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.010+0000 7ff25806e740 10 client.16905 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:27:46.016 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.010+0000 7ff25806e740 20 client.16905 populate_metadata read hostname 'smithi026' 2024-09-11T23:27:46.016 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.010+0000 7ff25806e740 1 --2- 172.21.15.26:0/3821000499 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5630ef237f60 0x5630ef23a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:46.016 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.010+0000 7ff25806e740 4 client.16905 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:27:46.017 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.010+0000 7ff25806e740 1 -- 172.21.15.26:0/3821000499 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5630ef18aa20 con 0x5630ef237f60 2024-09-11T23:27:46.017 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.010+0000 7ff24ed9d640 1 --2- 172.21.15.26:0/3821000499 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5630ef237f60 0x5630ef23a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:46.017 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.011+0000 7ff24ed9d640 1 --2- 172.21.15.26:0/3821000499 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5630ef237f60 0x5630ef23a350 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c77a6160c937e47f server_cookie=a270f5bf426a7a06 in_seq=0 out_seq=0 2024-09-11T23:27:46.017 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.011+0000 7ff232ff5640 10 client.16905 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:27:46.017 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.011+0000 7ff232ff5640 1 -- 172.21.15.26:0/3821000499 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5630ef18aa20 con 0x5630ef237f60 2024-09-11T23:27:46.017 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.011+0000 7ff232ff5640 10 client.16905 handle_command_reply: tid=0 2024-09-11T23:27:46.017 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.011+0000 7ff25806e740 1 client.16905 shutdown 2024-09-11T23:27:46.017 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.012+0000 7ff25806e740 1 -- 172.21.15.26:0/3821000499 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5630ef21a360 msgr2=0x5630ef225440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:46.017 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.012+0000 7ff25806e740 1 --2- 172.21.15.26:0/3821000499 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5630ef21a360 0x5630ef225440 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x5630ef21a740 tx=0x7ff24400b410 comp rx=0 tx=0).stop 2024-09-11T23:27:46.017 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.012+0000 7ff25806e740 1 -- 172.21.15.26:0/3821000499 shutdown_connections 2024-09-11T23:27:46.017 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.012+0000 7ff25806e740 1 --2- 172.21.15.26:0/3821000499 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5630ef227f40 0x5630ef225980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:46.017 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.012+0000 7ff25806e740 1 --2- 172.21.15.26:0/3821000499 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5630ef237f60 0x5630ef23a350 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:46.018 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.012+0000 7ff25806e740 1 --2- 172.21.15.26:0/3821000499 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5630ef21a360 0x5630ef225440 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:46.018 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.012+0000 7ff25806e740 1 --2- 172.21.15.26:0/3821000499 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5630ef219110 0x5630ef224f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:46.018 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.012+0000 7ff232ff5640 0 client.16905 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:27:46.018 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.012+0000 7ff25806e740 1 -- 172.21.15.26:0/3821000499 >> 172.21.15.26:0/3821000499 conn(0x5630ef215230 msgr2=0x5630ef21af90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:27:46.018 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.012+0000 7ff25806e740 1 -- 172.21.15.26:0/3821000499 shutdown_connections 2024-09-11T23:27:46.018 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.012+0000 7ff25806e740 1 -- 172.21.15.26:0/3821000499 wait complete. 2024-09-11T23:27:46.018 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.012+0000 7ff25806e740 20 client.16905 trim_cache size 0 max 16384 2024-09-11T23:27:46.030 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.026+0000 7ff25806e740 1 Processor -- start 2024-09-11T23:27:46.030 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.026+0000 7ff25806e740 1 -- start start 2024-09-11T23:27:46.030 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.026+0000 7ff25806e740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5630ef219110 0x5630ef21c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:46.030 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.026+0000 7ff25806e740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5630ef21a360 0x5630ef21ce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:46.031 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.026+0000 7ff25806e740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5630ef227f40 0x5630ef229ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:46.031 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.026+0000 7ff25806e740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff2501ab530 con 0x5630ef219110 2024-09-11T23:27:46.031 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.026+0000 7ff25806e740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7ff2501ad1f0 con 0x5630ef227f40 2024-09-11T23:27:46.031 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.026+0000 7ff25806e740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ff2501abc00 con 0x5630ef21a360 2024-09-11T23:27:46.031 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.027+0000 7ff24ed9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5630ef219110 0x5630ef21c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:46.031 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.027+0000 7ff24ed9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5630ef219110 0x5630ef21c8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51912/0 (socket says 172.21.15.26:51912) 2024-09-11T23:27:46.031 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.027+0000 7ff24ed9d640 1 -- 172.21.15.26:0/3653949159 learned_addr learned my addr 172.21.15.26:0/3653949159 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:27:46.031 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.027+0000 7ff24f59e640 1 --2- 172.21.15.26:0/3653949159 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5630ef227f40 0x5630ef229ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:46.031 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.027+0000 7ff24e59c640 1 --2- 172.21.15.26:0/3653949159 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5630ef21a360 0x5630ef21ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:46.031 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.027+0000 7ff24ed9d640 1 -- 172.21.15.26:0/3653949159 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5630ef21a360 msgr2=0x5630ef21ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:27:46.031 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.027+0000 7ff24ed9d640 1 --2- 172.21.15.26:0/3653949159 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5630ef21a360 0x5630ef21ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:46.031 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.027+0000 7ff24ed9d640 1 -- 172.21.15.26:0/3653949159 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5630ef227f40 msgr2=0x5630ef229ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:27:46.031 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.027+0000 7ff24ed9d640 1 --2- 172.21.15.26:0/3653949159 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5630ef227f40 0x5630ef229ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:46.031 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.027+0000 7ff24ed9d640 1 -- 172.21.15.26:0/3653949159 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630ef233e00 con 0x5630ef219110 2024-09-11T23:27:46.031 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.027+0000 7ff24f59e640 1 --2- 172.21.15.26:0/3653949159 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5630ef227f40 0x5630ef229ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:27:46.032 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.027+0000 7ff24ed9d640 1 --2- 172.21.15.26:0/3653949159 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5630ef219110 0x5630ef21c8f0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7ff240005fd0 tx=0x7ff24000b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c9afa4a12e0d808 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:27:46.032 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.027+0000 7ff234ff9640 1 -- 172.21.15.26:0/3653949159 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff240005020 con 0x5630ef219110 2024-09-11T23:27:46.032 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.027+0000 7ff234ff9640 1 -- 172.21.15.26:0/3653949159 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff2400047a0 con 0x5630ef219110 2024-09-11T23:27:46.032 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.027+0000 7ff234ff9640 1 -- 172.21.15.26:0/3653949159 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2400116a0 con 0x5630ef219110 2024-09-11T23:27:46.032 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.028+0000 7ff25806e740 1 -- 172.21.15.26:0/3653949159 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5630ef219110 msgr2=0x5630ef21c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:46.032 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.028+0000 7ff25806e740 1 --2- 172.21.15.26:0/3653949159 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5630ef219110 0x5630ef21c8f0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7ff240005fd0 tx=0x7ff24000b390 comp rx=0 tx=0).stop 2024-09-11T23:27:46.032 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.028+0000 7ff25806e740 1 -- 172.21.15.26:0/3653949159 shutdown_connections 2024-09-11T23:27:46.032 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.028+0000 7ff25806e740 1 --2- 172.21.15.26:0/3653949159 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5630ef227f40 0x5630ef229ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:46.032 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.028+0000 7ff25806e740 1 --2- 172.21.15.26:0/3653949159 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5630ef21a360 0x5630ef21ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:46.032 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.028+0000 7ff25806e740 1 --2- 172.21.15.26:0/3653949159 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5630ef219110 0x5630ef21c8f0 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:46.032 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.028+0000 7ff25806e740 1 -- 172.21.15.26:0/3653949159 >> 172.21.15.26:0/3653949159 conn(0x5630ef215230 msgr2=0x5630ef215610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:27:46.032 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.028+0000 7ff25806e740 1 -- 172.21.15.26:0/3653949159 shutdown_connections 2024-09-11T23:27:46.032 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.028+0000 7ff25806e740 1 -- 172.21.15.26:0/3653949159 wait complete. 2024-09-11T23:27:46.032 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.028+0000 7ff25806e740 1 -- start start 2024-09-11T23:27:46.033 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.028+0000 7ff25806e740 1 Processor -- start 2024-09-11T23:27:46.033 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.029+0000 7ff25806e740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5630ef219110 0x5630ef240430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:46.033 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.029+0000 7ff25806e740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5630ef21a360 0x5630ef22feb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:46.033 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.029+0000 7ff25806e740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5630ef227f40 0x5630ef2303f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:46.033 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.029+0000 7ff25806e740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff2501acb20 con 0x5630ef21a360 2024-09-11T23:27:46.033 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.029+0000 7ff25806e740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7ff2501ac390 con 0x5630ef219110 2024-09-11T23:27:46.033 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.029+0000 7ff25806e740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ff2501a9530 con 0x5630ef227f40 2024-09-11T23:27:46.033 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.029+0000 7ff25806e740 10 client.0 _ll_register_callbacks cb 0x7ff24400bd50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:27:46.033 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.029+0000 7ff24e59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5630ef21a360 0x5630ef22feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:46.033 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.029+0000 7ff24e59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5630ef21a360 0x5630ef22feb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51918/0 (socket says 172.21.15.26:51918) 2024-09-11T23:27:46.033 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.029+0000 7ff24e59c640 1 -- 172.21.15.26:0/2703267780 learned_addr learned my addr 172.21.15.26:0/2703267780 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:27:46.033 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.029+0000 7ff24ed9d640 1 --2- 172.21.15.26:0/2703267780 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5630ef219110 0x5630ef240430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:46.033 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.029+0000 7ff24f59e640 1 --2- 172.21.15.26:0/2703267780 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5630ef227f40 0x5630ef2303f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:46.033 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.029+0000 7ff24e59c640 1 -- 172.21.15.26:0/2703267780 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5630ef227f40 msgr2=0x5630ef2303f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:27:46.034 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.029+0000 7ff24e59c640 1 --2- 172.21.15.26:0/2703267780 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5630ef227f40 0x5630ef2303f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:46.034 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.029+0000 7ff24e59c640 1 -- 172.21.15.26:0/2703267780 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5630ef219110 msgr2=0x5630ef240430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:27:46.034 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.029+0000 7ff24e59c640 1 --2- 172.21.15.26:0/2703267780 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5630ef219110 0x5630ef240430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:46.034 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.029+0000 7ff24e59c640 1 -- 172.21.15.26:0/2703267780 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5630ef230bf0 con 0x5630ef21a360 2024-09-11T23:27:46.034 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.029+0000 7ff24e59c640 1 --2- 172.21.15.26:0/2703267780 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5630ef21a360 0x5630ef22feb0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7ff244006010 tx=0x7ff2440126d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6996ada2a0a2fb0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:27:46.034 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.029+0000 7ff232ff5640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:27:46.034 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.029+0000 7ff232ff5640 1 -- 172.21.15.26:0/2703267780 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff244037070 con 0x5630ef21a360 2024-09-11T23:27:46.034 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.030+0000 7ff232ff5640 1 -- 172.21.15.26:0/2703267780 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8542+0+0 (secure 0 0 0) 0x7ff24401ebd0 con 0x5630ef21a360 2024-09-11T23:27:46.034 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.030+0000 7ff25806e740 1 -- 172.21.15.26:0/2703267780 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2400047a0 con 0x5630ef21a360 2024-09-11T23:27:46.034 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.030+0000 7ff232ff5640 1 -- 172.21.15.26:0/2703267780 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7ff244020b20 con 0x5630ef21a360 2024-09-11T23:27:46.034 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.030+0000 7ff25806e740 10 client.16911 fetch_fsmap learned FSMap version 51 2024-09-11T23:27:46.034 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.030+0000 7ff25806e740 1 -- 172.21.15.26:0/2703267780 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x5630ef2a2cb0 con 0x5630ef21a360 2024-09-11T23:27:46.034 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.030+0000 7ff232ff5640 1 -- 172.21.15.26:0/2703267780 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff2400047a0 con 0x5630ef21a360 2024-09-11T23:27:46.034 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.030+0000 7ff232ff5640 1 -- 172.21.15.26:0/2703267780 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 51) ==== 8394+0+0 (secure 0 0 0) 0x7ff244020ce0 con 0x5630ef21a360 2024-09-11T23:27:46.035 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.030+0000 7ff25806e740 10 client.16911 fetch_fsmap finished waiting for FSMap version 51 2024-09-11T23:27:46.035 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.030+0000 7ff25806e740 10 client.16911 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:27:46.035 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.030+0000 7ff25806e740 20 client.16911 populate_metadata read hostname 'smithi026' 2024-09-11T23:27:46.035 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.030+0000 7ff25806e740 1 --2- 172.21.15.26:0/2703267780 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5630ef2a33a0 0x5630ef2a57c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:46.035 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.030+0000 7ff25806e740 4 client.16911 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T23:27:46.035 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.030+0000 7ff25806e740 1 -- 172.21.15.26:0/2703267780 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5630ef184d70 con 0x5630ef2a33a0 2024-09-11T23:27:46.035 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.030+0000 7ff24ed9d640 1 --2- 172.21.15.26:0/2703267780 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5630ef2a33a0 0x5630ef2a57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:46.035 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.031+0000 7ff232ff5640 10 client.16911 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:27:46.035 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.031+0000 7ff24ed9d640 1 --2- 172.21.15.26:0/2703267780 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5630ef2a33a0 0x5630ef2a57c0 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=164f30c6f8da470f server_cookie=e9e94caf7fb1bb4d in_seq=0 out_seq=0 2024-09-11T23:27:46.043 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.043+0000 7ff232ff5640 1 -- 172.21.15.26:0/2703267780 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5630ef184d70 con 0x5630ef2a33a0 2024-09-11T23:27:46.043 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.043+0000 7ff232ff5640 10 client.16911 handle_command_reply: tid=0 2024-09-11T23:27:46.043 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.043+0000 7ff25806e740 1 client.16911 shutdown 2024-09-11T23:27:46.043 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.043+0000 7ff25806e740 1 -- 172.21.15.26:0/2703267780 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5630ef21a360 msgr2=0x5630ef22feb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:46.043 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.043+0000 7ff25806e740 1 --2- 172.21.15.26:0/2703267780 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5630ef21a360 0x5630ef22feb0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7ff244006010 tx=0x7ff2440126d0 comp rx=0 tx=0).stop 2024-09-11T23:27:46.043 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.043+0000 7ff25806e740 1 -- 172.21.15.26:0/2703267780 shutdown_connections 2024-09-11T23:27:46.043 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.043+0000 7ff25806e740 1 --2- 172.21.15.26:0/2703267780 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5630ef2a33a0 0x5630ef2a57c0 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:46.044 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.043+0000 7ff25806e740 1 --2- 172.21.15.26:0/2703267780 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5630ef227f40 0x5630ef2303f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:46.044 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.043+0000 7ff25806e740 1 --2- 172.21.15.26:0/2703267780 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5630ef21a360 0x5630ef22feb0 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:46.044 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.043+0000 7ff232ff5640 0 client.16911 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:27:46.044 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.043+0000 7ff25806e740 1 --2- 172.21.15.26:0/2703267780 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5630ef219110 0x5630ef240430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:46.044 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.043+0000 7ff25806e740 1 -- 172.21.15.26:0/2703267780 >> 172.21.15.26:0/2703267780 conn(0x5630ef215230 msgr2=0x5630ef229ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:27:46.044 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.043+0000 7ff25806e740 1 -- 172.21.15.26:0/2703267780 shutdown_connections 2024-09-11T23:27:46.044 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.043+0000 7ff25806e740 1 -- 172.21.15.26:0/2703267780 wait complete. 2024-09-11T23:27:46.044 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.043+0000 7ff25806e740 20 client.16911 trim_cache size 0 max 16384 2024-09-11T23:27:46.044 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-09-11T23:27:46.044 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "message": "", 2024-09-11T23:27:46.044 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-09-11T23:27:46.045 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-09-11T23:27:46.045 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.043+0000 7ff2549fe640 1 -- 172.21.15.26:0/2154682632 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7ff2380a2cf0 msgr2=0x7ff2380a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:46.045 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.043+0000 7ff2549fe640 1 --2- 172.21.15.26:0/2154682632 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7ff2380a2cf0 0x7ff2380a51b0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7ff24400bfd0 tx=0x7ff24400bb40 comp rx=0 tx=0).stop 2024-09-11T23:27:46.046 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.043+0000 7ff2549fe640 1 -- 172.21.15.26:0/2154682632 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2501067e0 msgr2=0x7ff25019ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:46.046 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.043+0000 7ff2549fe640 1 --2- 172.21.15.26:0/2154682632 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2501067e0 0x7ff25019ba10 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7ff248005cc0 tx=0x7ff248003970 comp rx=0 tx=0).stop 2024-09-11T23:27:46.046 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.044+0000 7ff2549fe640 1 -- 172.21.15.26:0/2154682632 shutdown_connections 2024-09-11T23:27:46.046 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.044+0000 7ff2549fe640 1 --2- 172.21.15.26:0/2154682632 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7ff2380a2cf0 0x7ff2380a51b0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:46.046 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.044+0000 7ff2549fe640 1 --2- 172.21.15.26:0/2154682632 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff250101170 0x7ff250197730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:46.046 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.044+0000 7ff2549fe640 1 --2- 172.21.15.26:0/2154682632 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2501067e0 0x7ff25019ba10 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:46.046 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.044+0000 7ff2549fe640 1 --2- 172.21.15.26:0/2154682632 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff2501007d0 0x7ff2501971f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:46.046 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.044+0000 7ff2549fe640 1 -- 172.21.15.26:0/2154682632 >> 172.21.15.26:0/2154682632 conn(0x7ff2500777c0 msgr2=0x7ff2501037b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:27:46.046 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.044+0000 7ff2549fe640 1 -- 172.21.15.26:0/2154682632 shutdown_connections 2024-09-11T23:27:46.046 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:46.044+0000 7ff2549fe640 1 -- 172.21.15.26:0/2154682632 wait complete. 2024-09-11T23:27:46.052 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:27:45 smithi076 ceph-mon[35296]: pgmap v290: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 33 KiB/s wr, 15 op/s 2024-09-11T23:27:46.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:27:45 smithi136 ceph-mon[35121]: pgmap v290: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 33 KiB/s wr, 15 op/s 2024-09-11T23:27:46.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:27:45 smithi026 ceph-mon[31172]: pgmap v290: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 33 KiB/s wr, 15 op/s 2024-09-11T23:27:48.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:27:47 smithi076 ceph-mon[35296]: pgmap v291: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 3.3 KiB/s wr, 11 op/s 2024-09-11T23:27:48.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:27:47 smithi136 ceph-mon[35121]: pgmap v291: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 3.3 KiB/s wr, 11 op/s 2024-09-11T23:27:48.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:27:47 smithi026 ceph-mon[31172]: pgmap v291: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 3.3 KiB/s wr, 11 op/s 2024-09-11T23:27:50.041 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:27:49 smithi136 ceph-mon[35121]: pgmap v292: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.4 KiB/s wr, 9 op/s 2024-09-11T23:27:50.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:27:49 smithi076 ceph-mon[35296]: pgmap v292: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.4 KiB/s wr, 9 op/s 2024-09-11T23:27:50.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:27:49 smithi026 ceph-mon[31172]: pgmap v292: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.4 KiB/s wr, 9 op/s 2024-09-11T23:27:52.052 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:27:51 smithi076 ceph-mon[35296]: pgmap v293: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 38 KiB/s wr, 16 op/s 2024-09-11T23:27:52.052 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:27:51 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:27:52.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:27:51 smithi136 ceph-mon[35121]: pgmap v293: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 38 KiB/s wr, 16 op/s 2024-09-11T23:27:52.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:27:51 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:27:52.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:27:51 smithi026 ceph-mon[31172]: pgmap v293: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 38 KiB/s wr, 16 op/s 2024-09-11T23:27:52.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:27:51 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:27:53.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:27:52 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:27:53.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:27:52 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:27:53.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:27:52 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:27:53.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:27:52 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:27:53.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:27:52 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:27:53.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:27:52 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:27:53.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:27:52 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:27:53.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:27:52 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:27:53.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:27:52 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:27:54.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:27:54 smithi076 ceph-mon[35296]: pgmap v294: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 37 KiB/s wr, 11 op/s 2024-09-11T23:27:54.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:27:54 smithi136 ceph-mon[35121]: pgmap v294: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 37 KiB/s wr, 11 op/s 2024-09-11T23:27:54.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:27:54 smithi026 ceph-mon[31172]: pgmap v294: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 37 KiB/s wr, 11 op/s 2024-09-11T23:27:56.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:27:56 smithi076 ceph-mon[35296]: pgmap v295: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 43 KiB/s wr, 16 op/s 2024-09-11T23:27:56.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:27:56 smithi136 ceph-mon[35121]: pgmap v295: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 43 KiB/s wr, 16 op/s 2024-09-11T23:27:56.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:27:56 smithi026 ceph-mon[31172]: pgmap v295: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 43 KiB/s wr, 16 op/s 2024-09-11T23:27:58.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:27:58 smithi076 ceph-mon[35296]: pgmap v296: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 42 KiB/s wr, 14 op/s 2024-09-11T23:27:58.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:27:58 smithi136 ceph-mon[35121]: pgmap v296: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 42 KiB/s wr, 14 op/s 2024-09-11T23:27:58.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:27:58 smithi026 ceph-mon[31172]: pgmap v296: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 42 KiB/s wr, 14 op/s 2024-09-11T23:27:59.148 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.144+0000 7fae3efcf640 1 -- 172.21.15.26:0/2146353274 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae38107b70 msgr2=0x7fae38107f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:59.148 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.144+0000 7fae3efcf640 1 --2- 172.21.15.26:0/2146353274 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae38107b70 0x7fae38107f50 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fae20009930 tx=0x7fae2002a420 comp rx=0 tx=0).stop 2024-09-11T23:27:59.148 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.144+0000 7fae3efcf640 1 -- 172.21.15.26:0/2146353274 shutdown_connections 2024-09-11T23:27:59.148 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.144+0000 7fae3efcf640 1 --2- 172.21.15.26:0/2146353274 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae38102470 0x7fae3810e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:59.148 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.144+0000 7fae3efcf640 1 --2- 172.21.15.26:0/2146353274 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fae38101ad0 0x7fae38101f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:59.148 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.144+0000 7fae3efcf640 1 --2- 172.21.15.26:0/2146353274 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae38107b70 0x7fae38107f50 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:59.148 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.144+0000 7fae3efcf640 1 -- 172.21.15.26:0/2146353274 >> 172.21.15.26:0/2146353274 conn(0x7fae380fd7d0 msgr2=0x7fae380ffbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:27:59.148 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.145+0000 7fae3efcf640 1 -- 172.21.15.26:0/2146353274 shutdown_connections 2024-09-11T23:27:59.149 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.145+0000 7fae3efcf640 1 -- 172.21.15.26:0/2146353274 wait complete. 2024-09-11T23:27:59.149 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.145+0000 7fae3efcf640 1 Processor -- start 2024-09-11T23:27:59.149 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.145+0000 7fae3efcf640 1 -- start start 2024-09-11T23:27:59.149 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.145+0000 7fae3efcf640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fae38101ad0 0x7fae38198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:59.149 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.145+0000 7fae3efcf640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae38102470 0x7fae38198850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:59.149 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.145+0000 7fae3efcf640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae38107b70 0x7fae3819cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:59.149 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.145+0000 7fae3efcf640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fae38111ee0 con 0x7fae38102470 2024-09-11T23:27:59.149 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.145+0000 7fae3efcf640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fae38111d60 con 0x7fae38107b70 2024-09-11T23:27:59.149 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.145+0000 7fae3efcf640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fae38112060 con 0x7fae38101ad0 2024-09-11T23:27:59.149 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.146+0000 7fae37fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae38102470 0x7fae38198850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:59.149 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.146+0000 7fae37fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae38102470 0x7fae38198850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44100/0 (socket says 172.21.15.26:44100) 2024-09-11T23:27:59.149 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.146+0000 7fae37fff640 1 -- 172.21.15.26:0/4033658195 learned_addr learned my addr 172.21.15.26:0/4033658195 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:27:59.150 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.146+0000 7fae3cd6b640 1 --2- 172.21.15.26:0/4033658195 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fae38101ad0 0x7fae38198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:59.150 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.146+0000 7fae3d56c640 1 --2- 172.21.15.26:0/4033658195 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae38107b70 0x7fae3819cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:59.150 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.146+0000 7fae37fff640 1 -- 172.21.15.26:0/4033658195 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fae38101ad0 msgr2=0x7fae38198310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:59.150 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.146+0000 7fae37fff640 1 --2- 172.21.15.26:0/4033658195 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fae38101ad0 0x7fae38198310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:59.150 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.146+0000 7fae37fff640 1 -- 172.21.15.26:0/4033658195 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae38107b70 msgr2=0x7fae3819cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:59.150 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.146+0000 7fae37fff640 1 --2- 172.21.15.26:0/4033658195 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae38107b70 0x7fae3819cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:59.150 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.146+0000 7fae37fff640 1 -- 172.21.15.26:0/4033658195 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae3819d300 con 0x7fae38102470 2024-09-11T23:27:59.150 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.146+0000 7fae37fff640 1 --2- 172.21.15.26:0/4033658195 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae38102470 0x7fae38198850 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fae2800db80 tx=0x7fae2800b630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:27:59.150 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.146+0000 7fae35ffb640 1 -- 172.21.15.26:0/4033658195 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae28017070 con 0x7fae38102470 2024-09-11T23:27:59.150 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.146+0000 7fae3efcf640 1 -- 172.21.15.26:0/4033658195 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae3819d5f0 con 0x7fae38102470 2024-09-11T23:27:59.150 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.146+0000 7fae35ffb640 1 -- 172.21.15.26:0/4033658195 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae28003b70 con 0x7fae38102470 2024-09-11T23:27:59.150 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.146+0000 7fae35ffb640 1 -- 172.21.15.26:0/4033658195 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae280106c0 con 0x7fae38102470 2024-09-11T23:27:59.150 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.147+0000 7fae3efcf640 1 -- 172.21.15.26:0/4033658195 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae381a4e80 con 0x7fae38102470 2024-09-11T23:27:59.156 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.149+0000 7fae35ffb640 1 -- 172.21.15.26:0/4033658195 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fae28004020 con 0x7fae38102470 2024-09-11T23:27:59.156 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.149+0000 7fae41e43740 1 Processor -- start 2024-09-11T23:27:59.156 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.149+0000 7fae41e43740 1 -- start start 2024-09-11T23:27:59.156 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.150+0000 7fae41e43740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5641329c2af0 0x5641329c2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:59.156 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.150+0000 7fae41e43740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5641329c3410 0x5641329cd770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:59.156 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.150+0000 7fae41e43740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5641329cdcb0 0x5641329d00a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:59.156 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.150+0000 7fae41e43740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5641329272f0 con 0x5641329c2af0 2024-09-11T23:27:59.156 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.150+0000 7fae41e43740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5641328cf270 con 0x5641329cdcb0 2024-09-11T23:27:59.156 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.150+0000 7fae41e43740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5641329232b0 con 0x5641329c3410 2024-09-11T23:27:59.156 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.150+0000 7fae3d56c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5641329c2af0 0x5641329c2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:59.156 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.150+0000 7fae3d56c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5641329c2af0 0x5641329c2ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44114/0 (socket says 172.21.15.26:44114) 2024-09-11T23:27:59.157 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.150+0000 7fae3d56c640 1 -- 172.21.15.26:0/1986070077 learned_addr learned my addr 172.21.15.26:0/1986070077 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:27:59.157 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.150+0000 7fae37fff640 1 --2- 172.21.15.26:0/1986070077 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5641329cdcb0 0x5641329d00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:59.157 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.150+0000 7fae3cd6b640 1 --2- 172.21.15.26:0/1986070077 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5641329c3410 0x5641329cd770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:59.157 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.150+0000 7fae35ffb640 1 --2- 172.21.15.26:0/4033658195 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fae240a2da0 0x7fae240a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:59.157 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.150+0000 7fae3d56c640 1 -- 172.21.15.26:0/1986070077 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5641329c3410 msgr2=0x5641329cd770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:27:59.157 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.150+0000 7fae3d56c640 1 --2- 172.21.15.26:0/1986070077 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5641329c3410 0x5641329cd770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:59.157 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.150+0000 7fae3d56c640 1 -- 172.21.15.26:0/1986070077 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5641329cdcb0 msgr2=0x5641329d00a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:27:59.157 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.150+0000 7fae3d56c640 1 --2- 172.21.15.26:0/1986070077 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5641329cdcb0 0x5641329d00a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:59.157 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.150+0000 7fae3d56c640 1 -- 172.21.15.26:0/1986070077 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5641328ce340 con 0x5641329c2af0 2024-09-11T23:27:59.157 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.150+0000 7fae35ffb640 1 -- 172.21.15.26:0/4033658195 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8542+0+0 (secure 0 0 0) 0x7fae280cf210 con 0x7fae38102470 2024-09-11T23:27:59.157 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.150+0000 7fae3d56c640 1 --2- 172.21.15.26:0/4033658195 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fae240a2da0 0x7fae240a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:59.157 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.150+0000 7fae3d56c640 1 --2- 172.21.15.26:0/1986070077 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5641329c2af0 0x5641329c2ed0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fae30009da0 tx=0x7fae30011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67b2cde6f90e6c63 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:27:59.157 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.151+0000 7fae2effd640 1 -- 172.21.15.26:0/1986070077 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae30017050 con 0x5641329c2af0 2024-09-11T23:27:59.157 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.151+0000 7fae2effd640 1 -- 172.21.15.26:0/1986070077 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae3000f8a0 con 0x5641329c2af0 2024-09-11T23:27:59.158 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.151+0000 7fae2effd640 1 -- 172.21.15.26:0/1986070077 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae3000e610 con 0x5641329c2af0 2024-09-11T23:27:59.158 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.151+0000 7fae3d56c640 1 --2- 172.21.15.26:0/4033658195 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fae240a2da0 0x7fae240a5260 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fae30023000 tx=0x7fae30024000 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:27:59.158 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.151+0000 7fae41e43740 1 -- 172.21.15.26:0/1986070077 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5641329c2af0 msgr2=0x5641329c2ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:59.158 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.151+0000 7fae41e43740 1 --2- 172.21.15.26:0/1986070077 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5641329c2af0 0x5641329c2ed0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fae30009da0 tx=0x7fae30011810 comp rx=0 tx=0).stop 2024-09-11T23:27:59.158 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.151+0000 7fae41e43740 1 -- 172.21.15.26:0/1986070077 shutdown_connections 2024-09-11T23:27:59.158 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.151+0000 7fae41e43740 1 --2- 172.21.15.26:0/1986070077 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5641329cdcb0 0x5641329d00a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:59.158 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.151+0000 7fae41e43740 1 --2- 172.21.15.26:0/1986070077 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5641329c3410 0x5641329cd770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:59.158 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.151+0000 7fae41e43740 1 --2- 172.21.15.26:0/1986070077 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5641329c2af0 0x5641329c2ed0 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:59.158 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.151+0000 7fae41e43740 1 -- 172.21.15.26:0/1986070077 >> 172.21.15.26:0/1986070077 conn(0x5641329bf060 msgr2=0x5641329bf440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:27:59.158 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.152+0000 7fae41e43740 1 -- 172.21.15.26:0/1986070077 shutdown_connections 2024-09-11T23:27:59.158 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.152+0000 7fae41e43740 1 -- 172.21.15.26:0/1986070077 wait complete. 2024-09-11T23:27:59.158 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.152+0000 7fae41e43740 1 -- start start 2024-09-11T23:27:59.158 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.152+0000 7fae41e43740 1 Processor -- start 2024-09-11T23:27:59.158 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.152+0000 7fae41e43740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5641329c2af0 0x5641329cac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:59.159 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.152+0000 7fae41e43740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5641329c3410 0x5641329cb170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:59.159 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.152+0000 7fae41e43740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5641329cdcb0 0x5641329cb6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:59.159 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.152+0000 7fae41e43740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5641329d0880 con 0x5641329c2af0 2024-09-11T23:27:59.159 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.152+0000 7fae41e43740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5641328b0290 con 0x5641329cdcb0 2024-09-11T23:27:59.159 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.152+0000 7fae41e43740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5641329d0a00 con 0x5641329c3410 2024-09-11T23:27:59.159 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.153+0000 7fae41e43740 10 client.0 _ll_register_callbacks cb 0x5641328e1140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:27:59.159 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.153+0000 7fae3cd6b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5641329c2af0 0x5641329cac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:59.159 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.153+0000 7fae3cd6b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5641329c2af0 0x5641329cac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44130/0 (socket says 172.21.15.26:44130) 2024-09-11T23:27:59.159 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.153+0000 7fae3cd6b640 1 -- 172.21.15.26:0/213502004 learned_addr learned my addr 172.21.15.26:0/213502004 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:27:59.159 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.153+0000 7fae37fff640 1 --2- 172.21.15.26:0/213502004 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5641329c3410 0x5641329cb170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:59.159 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.153+0000 7fae3d56c640 1 --2- 172.21.15.26:0/213502004 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5641329cdcb0 0x5641329cb6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:59.159 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.153+0000 7fae3cd6b640 1 -- 172.21.15.26:0/213502004 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5641329c3410 msgr2=0x5641329cb170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:27:59.159 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.153+0000 7fae3cd6b640 1 --2- 172.21.15.26:0/213502004 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5641329c3410 0x5641329cb170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:59.159 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.153+0000 7fae3cd6b640 1 -- 172.21.15.26:0/213502004 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5641329cdcb0 msgr2=0x5641329cb6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:27:59.160 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.153+0000 7fae3cd6b640 1 --2- 172.21.15.26:0/213502004 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5641329cdcb0 0x5641329cb6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:59.160 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.153+0000 7fae3cd6b640 1 -- 172.21.15.26:0/213502004 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56413263a210 con 0x5641329c2af0 2024-09-11T23:27:59.160 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.153+0000 7fae3cd6b640 1 --2- 172.21.15.26:0/213502004 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5641329c2af0 0x5641329cac30 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fae20000c00 tx=0x7fae20031920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abf3cbfc75ddea0f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:27:59.160 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.153+0000 7fae17fff640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:27:59.160 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.153+0000 7fae17fff640 1 -- 172.21.15.26:0/213502004 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae20035040 con 0x5641329c2af0 2024-09-11T23:27:59.160 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.153+0000 7fae17fff640 1 -- 172.21.15.26:0/213502004 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8542+0+0 (secure 0 0 0) 0x7fae20004240 con 0x5641329c2af0 2024-09-11T23:27:59.160 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.153+0000 7fae41e43740 1 -- 172.21.15.26:0/213502004 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae3000f8a0 con 0x5641329c2af0 2024-09-11T23:27:59.160 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.153+0000 7fae17fff640 1 -- 172.21.15.26:0/213502004 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7fae20004440 con 0x5641329c2af0 2024-09-11T23:27:59.160 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.153+0000 7fae41e43740 10 client.16923 fetch_fsmap learned FSMap version 51 2024-09-11T23:27:59.160 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.154+0000 7fae41e43740 1 -- 172.21.15.26:0/213502004 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x564132706620 con 0x5641329c2af0 2024-09-11T23:27:59.160 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.154+0000 7fae17fff640 1 -- 172.21.15.26:0/213502004 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae3000f8a0 con 0x5641329c2af0 2024-09-11T23:27:59.160 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.154+0000 7fae17fff640 1 -- 172.21.15.26:0/213502004 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 51) ==== 8394+0+0 (secure 0 0 0) 0x7fae20004c20 con 0x5641329c2af0 2024-09-11T23:27:59.160 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.154+0000 7fae41e43740 10 client.16923 fetch_fsmap finished waiting for FSMap version 51 2024-09-11T23:27:59.161 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.154+0000 7fae41e43740 10 client.16923 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:27:59.161 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.154+0000 7fae41e43740 20 client.16923 populate_metadata read hostname 'smithi026' 2024-09-11T23:27:59.161 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.154+0000 7fae41e43740 1 --2- 172.21.15.26:0/213502004 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5641329ddc90 0x5641329e0080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:59.161 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.154+0000 7fae41e43740 4 client.16923 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:27:59.161 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.154+0000 7fae41e43740 1 -- 172.21.15.26:0/213502004 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564132934a20 con 0x5641329ddc90 2024-09-11T23:27:59.161 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.154+0000 7fae37fff640 1 --2- 172.21.15.26:0/213502004 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5641329ddc90 0x5641329e0080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:59.161 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.155+0000 7fae17fff640 10 client.16923 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:27:59.161 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.155+0000 7fae37fff640 1 --2- 172.21.15.26:0/213502004 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5641329ddc90 0x5641329e0080 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f10c4a05f7724120 server_cookie=ac6d425ac08c3bb7 in_seq=0 out_seq=0 2024-09-11T23:27:59.162 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.155+0000 7fae17fff640 1 -- 172.21.15.26:0/213502004 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564132934a20 con 0x5641329ddc90 2024-09-11T23:27:59.162 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.155+0000 7fae17fff640 10 client.16923 handle_command_reply: tid=0 2024-09-11T23:27:59.162 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.156+0000 7fae41e43740 1 client.16923 shutdown 2024-09-11T23:27:59.162 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.156+0000 7fae41e43740 1 -- 172.21.15.26:0/213502004 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5641329c2af0 msgr2=0x5641329cac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:59.162 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.156+0000 7fae41e43740 1 --2- 172.21.15.26:0/213502004 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5641329c2af0 0x5641329cac30 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fae20000c00 tx=0x7fae20031920 comp rx=0 tx=0).stop 2024-09-11T23:27:59.163 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.156+0000 7fae41e43740 1 -- 172.21.15.26:0/213502004 shutdown_connections 2024-09-11T23:27:59.163 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.156+0000 7fae41e43740 1 --2- 172.21.15.26:0/213502004 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5641329cdcb0 0x5641329cb6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:59.163 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.156+0000 7fae41e43740 1 --2- 172.21.15.26:0/213502004 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5641329ddc90 0x5641329e0080 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:59.163 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.156+0000 7fae41e43740 1 --2- 172.21.15.26:0/213502004 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5641329c3410 0x5641329cb170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:59.163 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.156+0000 7fae41e43740 1 --2- 172.21.15.26:0/213502004 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5641329c2af0 0x5641329cac30 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:59.163 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.156+0000 7fae17fff640 0 client.16923 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:27:59.163 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.156+0000 7fae41e43740 1 -- 172.21.15.26:0/213502004 >> 172.21.15.26:0/213502004 conn(0x5641329bf060 msgr2=0x5641329c4d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:27:59.163 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.156+0000 7fae41e43740 1 -- 172.21.15.26:0/213502004 shutdown_connections 2024-09-11T23:27:59.163 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.156+0000 7fae41e43740 1 -- 172.21.15.26:0/213502004 wait complete. 2024-09-11T23:27:59.163 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.156+0000 7fae41e43740 20 client.16923 trim_cache size 0 max 16384 2024-09-11T23:27:59.172 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.171+0000 7fae41e43740 1 Processor -- start 2024-09-11T23:27:59.172 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.171+0000 7fae41e43740 1 -- start start 2024-09-11T23:27:59.173 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.171+0000 7fae41e43740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5641329c2af0 0x5641329c5560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:59.173 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.171+0000 7fae41e43740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5641329c3410 0x5641329cb4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:59.173 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.171+0000 7fae41e43740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5641329cdcb0 0x5641329d6070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:59.173 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.171+0000 7fae41e43740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fae381ad780 con 0x5641329c2af0 2024-09-11T23:27:59.173 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.171+0000 7fae41e43740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fae381af440 con 0x5641329cdcb0 2024-09-11T23:27:59.173 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.171+0000 7fae41e43740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fae381ade50 con 0x5641329c3410 2024-09-11T23:27:59.173 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.171+0000 7fae3cd6b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5641329c2af0 0x5641329c5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:59.173 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.171+0000 7fae3cd6b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5641329c2af0 0x5641329c5560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44132/0 (socket says 172.21.15.26:44132) 2024-09-11T23:27:59.173 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.171+0000 7fae3cd6b640 1 -- 172.21.15.26:0/3792670774 learned_addr learned my addr 172.21.15.26:0/3792670774 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:27:59.173 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.171+0000 7fae37fff640 1 --2- 172.21.15.26:0/3792670774 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5641329c3410 0x5641329cb4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:59.173 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.171+0000 7fae3d56c640 1 --2- 172.21.15.26:0/3792670774 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5641329cdcb0 0x5641329d6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:59.173 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.171+0000 7fae3cd6b640 1 -- 172.21.15.26:0/3792670774 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5641329c3410 msgr2=0x5641329cb4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:27:59.174 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.171+0000 7fae3cd6b640 1 --2- 172.21.15.26:0/3792670774 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5641329c3410 0x5641329cb4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:59.174 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.171+0000 7fae3cd6b640 1 -- 172.21.15.26:0/3792670774 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5641329cdcb0 msgr2=0x5641329d6070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:27:59.174 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.171+0000 7fae3cd6b640 1 --2- 172.21.15.26:0/3792670774 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5641329cdcb0 0x5641329d6070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:59.174 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.171+0000 7fae3cd6b640 1 -- 172.21.15.26:0/3792670774 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5641329ba2d0 con 0x5641329c2af0 2024-09-11T23:27:59.174 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.171+0000 7fae37fff640 1 --2- 172.21.15.26:0/3792670774 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5641329c3410 0x5641329cb4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:27:59.174 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.171+0000 7fae3cd6b640 1 --2- 172.21.15.26:0/3792670774 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5641329c2af0 0x5641329c5560 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fae200096c0 tx=0x7fae20004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=add9575478863b71 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:27:59.174 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.172+0000 7fae2e7fc640 1 -- 172.21.15.26:0/3792670774 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae20035030 con 0x5641329c2af0 2024-09-11T23:27:59.174 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.172+0000 7fae2e7fc640 1 -- 172.21.15.26:0/3792670774 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae20004190 con 0x5641329c2af0 2024-09-11T23:27:59.174 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.172+0000 7fae2e7fc640 1 -- 172.21.15.26:0/3792670774 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae20053070 con 0x5641329c2af0 2024-09-11T23:27:59.178 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.172+0000 7fae41e43740 1 -- 172.21.15.26:0/3792670774 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5641329c2af0 msgr2=0x5641329c5560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:59.178 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.172+0000 7fae41e43740 1 --2- 172.21.15.26:0/3792670774 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5641329c2af0 0x5641329c5560 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fae200096c0 tx=0x7fae20004600 comp rx=0 tx=0).stop 2024-09-11T23:27:59.178 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.172+0000 7fae41e43740 1 -- 172.21.15.26:0/3792670774 shutdown_connections 2024-09-11T23:27:59.178 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.172+0000 7fae41e43740 1 --2- 172.21.15.26:0/3792670774 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5641329cdcb0 0x5641329d6070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:59.178 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.172+0000 7fae41e43740 1 --2- 172.21.15.26:0/3792670774 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5641329c3410 0x5641329cb4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:59.178 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.172+0000 7fae41e43740 1 --2- 172.21.15.26:0/3792670774 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5641329c2af0 0x5641329c5560 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:59.178 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.172+0000 7fae41e43740 1 -- 172.21.15.26:0/3792670774 >> 172.21.15.26:0/3792670774 conn(0x5641329bf060 msgr2=0x5641329cccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:27:59.178 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.172+0000 7fae41e43740 1 -- 172.21.15.26:0/3792670774 shutdown_connections 2024-09-11T23:27:59.178 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.172+0000 7fae41e43740 1 -- 172.21.15.26:0/3792670774 wait complete. 2024-09-11T23:27:59.178 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.173+0000 7fae41e43740 1 -- start start 2024-09-11T23:27:59.178 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.173+0000 7fae41e43740 1 Processor -- start 2024-09-11T23:27:59.178 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.173+0000 7fae41e43740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5641329c2af0 0x564132a45010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:59.179 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.173+0000 7fae41e43740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5641329c3410 0x564132a45550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:59.179 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.173+0000 7fae41e43740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5641329cdcb0 0x5641329e5380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:59.179 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.173+0000 7fae41e43740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fae381aed70 con 0x5641329c2af0 2024-09-11T23:27:59.179 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.173+0000 7fae41e43740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fae381ae5e0 con 0x5641329cdcb0 2024-09-11T23:27:59.179 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.173+0000 7fae41e43740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fae381ab780 con 0x5641329c3410 2024-09-11T23:27:59.179 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.173+0000 7fae41e43740 10 client.0 _ll_register_callbacks cb 0x7fae20009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:27:59.179 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.173+0000 7fae3cd6b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5641329c2af0 0x564132a45010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:59.179 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.173+0000 7fae3cd6b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5641329c2af0 0x564132a45010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44134/0 (socket says 172.21.15.26:44134) 2024-09-11T23:27:59.179 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.173+0000 7fae3cd6b640 1 -- 172.21.15.26:0/1755383165 learned_addr learned my addr 172.21.15.26:0/1755383165 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:27:59.179 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.173+0000 7fae3d56c640 1 --2- 172.21.15.26:0/1755383165 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5641329cdcb0 0x5641329e5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:59.179 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.173+0000 7fae37fff640 1 --2- 172.21.15.26:0/1755383165 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5641329c3410 0x564132a45550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:59.179 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.173+0000 7fae3cd6b640 1 -- 172.21.15.26:0/1755383165 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5641329c3410 msgr2=0x564132a45550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:27:59.179 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.173+0000 7fae3cd6b640 1 --2- 172.21.15.26:0/1755383165 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5641329c3410 0x564132a45550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:59.179 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.174+0000 7fae3cd6b640 1 -- 172.21.15.26:0/1755383165 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5641329cdcb0 msgr2=0x5641329e5380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:27:59.180 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.174+0000 7fae3cd6b640 1 --2- 172.21.15.26:0/1755383165 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5641329cdcb0 0x5641329e5380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:59.180 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.174+0000 7fae3cd6b640 1 -- 172.21.15.26:0/1755383165 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5641329e5b80 con 0x5641329c2af0 2024-09-11T23:27:59.180 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.174+0000 7fae37fff640 1 --2- 172.21.15.26:0/1755383165 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5641329c3410 0x564132a45550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:27:59.180 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.174+0000 7fae3cd6b640 1 --2- 172.21.15.26:0/1755383165 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5641329c2af0 0x564132a45010 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fae20003640 tx=0x7fae20031620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74e661642e7054b5 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:27:59.180 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.174+0000 7fae17fff640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:27:59.180 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.174+0000 7fae17fff640 1 -- 172.21.15.26:0/1755383165 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae20053070 con 0x5641329c2af0 2024-09-11T23:27:59.180 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.174+0000 7fae17fff640 1 -- 172.21.15.26:0/1755383165 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8542+0+0 (secure 0 0 0) 0x7fae20003a30 con 0x5641329c2af0 2024-09-11T23:27:59.180 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.174+0000 7fae41e43740 1 -- 172.21.15.26:0/1755383165 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae20004190 con 0x5641329c2af0 2024-09-11T23:27:59.180 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.174+0000 7fae17fff640 1 -- 172.21.15.26:0/1755383165 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7fae20066050 con 0x5641329c2af0 2024-09-11T23:27:59.180 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.174+0000 7fae41e43740 10 client.16929 fetch_fsmap learned FSMap version 51 2024-09-11T23:27:59.180 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.174+0000 7fae41e43740 1 -- 172.21.15.26:0/1755383165 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x564132a4c880 con 0x5641329c2af0 2024-09-11T23:27:59.180 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.174+0000 7fae17fff640 1 -- 172.21.15.26:0/1755383165 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae20004190 con 0x5641329c2af0 2024-09-11T23:27:59.180 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.175+0000 7fae17fff640 1 -- 172.21.15.26:0/1755383165 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 51) ==== 8394+0+0 (secure 0 0 0) 0x7fae20046480 con 0x5641329c2af0 2024-09-11T23:27:59.180 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.175+0000 7fae41e43740 10 client.16929 fetch_fsmap finished waiting for FSMap version 51 2024-09-11T23:27:59.181 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.175+0000 7fae41e43740 10 client.16929 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:27:59.181 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.175+0000 7fae41e43740 20 client.16929 populate_metadata read hostname 'smithi026' 2024-09-11T23:27:59.181 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.175+0000 7fae41e43740 1 --2- 172.21.15.26:0/1755383165 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x564132a4ceb0 0x564132a4f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:59.181 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.175+0000 7fae41e43740 4 client.16929 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T23:27:59.181 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.175+0000 7fae41e43740 1 -- 172.21.15.26:0/1755383165 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56413292ed70 con 0x564132a4ceb0 2024-09-11T23:27:59.181 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.175+0000 7fae37fff640 1 --2- 172.21.15.26:0/1755383165 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x564132a4ceb0 0x564132a4f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:59.181 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.175+0000 7fae17fff640 10 client.16929 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:27:59.181 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.175+0000 7fae37fff640 1 --2- 172.21.15.26:0/1755383165 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x564132a4ceb0 0x564132a4f2d0 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9861a55e040fe10 server_cookie=750b9eeac7aff13c in_seq=0 out_seq=0 2024-09-11T23:27:59.193 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.192+0000 7fae17fff640 1 -- 172.21.15.26:0/1755383165 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56413292ed70 con 0x564132a4ceb0 2024-09-11T23:27:59.193 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.192+0000 7fae17fff640 10 client.16929 handle_command_reply: tid=0 2024-09-11T23:27:59.193 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.192+0000 7fae41e43740 1 client.16929 shutdown 2024-09-11T23:27:59.194 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.192+0000 7fae41e43740 1 -- 172.21.15.26:0/1755383165 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5641329c2af0 msgr2=0x564132a45010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:59.194 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.192+0000 7fae41e43740 1 --2- 172.21.15.26:0/1755383165 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5641329c2af0 0x564132a45010 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fae20003640 tx=0x7fae20031620 comp rx=0 tx=0).stop 2024-09-11T23:27:59.194 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.192+0000 7fae41e43740 1 -- 172.21.15.26:0/1755383165 shutdown_connections 2024-09-11T23:27:59.194 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.192+0000 7fae41e43740 1 --2- 172.21.15.26:0/1755383165 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5641329cdcb0 0x5641329e5380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:59.194 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.192+0000 7fae41e43740 1 --2- 172.21.15.26:0/1755383165 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x564132a4ceb0 0x564132a4f2d0 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:59.194 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.192+0000 7fae41e43740 1 --2- 172.21.15.26:0/1755383165 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5641329c3410 0x564132a45550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:59.194 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.192+0000 7fae17fff640 0 client.16929 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:27:59.194 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.192+0000 7fae41e43740 1 --2- 172.21.15.26:0/1755383165 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5641329c2af0 0x564132a45010 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:59.194 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.192+0000 7fae41e43740 1 -- 172.21.15.26:0/1755383165 >> 172.21.15.26:0/1755383165 conn(0x5641329bf060 msgr2=0x5641329bd9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:27:59.194 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.192+0000 7fae41e43740 1 -- 172.21.15.26:0/1755383165 shutdown_connections 2024-09-11T23:27:59.194 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.192+0000 7fae41e43740 1 -- 172.21.15.26:0/1755383165 wait complete. 2024-09-11T23:27:59.194 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.192+0000 7fae41e43740 20 client.16929 trim_cache size 0 max 16384 2024-09-11T23:27:59.195 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-09-11T23:27:59.195 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "message": "", 2024-09-11T23:27:59.195 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-09-11T23:27:59.195 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-09-11T23:27:59.196 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.193+0000 7fae3efcf640 1 -- 172.21.15.26:0/4033658195 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fae240a2da0 msgr2=0x7fae240a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:59.196 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.193+0000 7fae3efcf640 1 --2- 172.21.15.26:0/4033658195 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fae240a2da0 0x7fae240a5260 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fae30023000 tx=0x7fae30024000 comp rx=0 tx=0).stop 2024-09-11T23:27:59.196 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.193+0000 7fae3efcf640 1 -- 172.21.15.26:0/4033658195 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae38102470 msgr2=0x7fae38198850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:59.196 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.193+0000 7fae3efcf640 1 --2- 172.21.15.26:0/4033658195 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae38102470 0x7fae38198850 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fae2800db80 tx=0x7fae2800b630 comp rx=0 tx=0).stop 2024-09-11T23:27:59.196 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.193+0000 7fae3efcf640 1 -- 172.21.15.26:0/4033658195 shutdown_connections 2024-09-11T23:27:59.196 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.193+0000 7fae3efcf640 1 --2- 172.21.15.26:0/4033658195 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fae240a2da0 0x7fae240a5260 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:59.196 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.193+0000 7fae3efcf640 1 --2- 172.21.15.26:0/4033658195 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae38107b70 0x7fae3819cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:59.196 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.193+0000 7fae3efcf640 1 --2- 172.21.15.26:0/4033658195 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae38102470 0x7fae38198850 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:59.196 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.193+0000 7fae3efcf640 1 --2- 172.21.15.26:0/4033658195 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fae38101ad0 0x7fae38198310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:59.196 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.193+0000 7fae3efcf640 1 -- 172.21.15.26:0/4033658195 >> 172.21.15.26:0/4033658195 conn(0x7fae380fd7d0 msgr2=0x7fae380fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:27:59.196 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.193+0000 7fae3efcf640 1 -- 172.21.15.26:0/4033658195 shutdown_connections 2024-09-11T23:27:59.196 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:27:59.193+0000 7fae3efcf640 1 -- 172.21.15.26:0/4033658195 wait complete. 2024-09-11T23:28:00.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:28:00 smithi076 ceph-mon[35296]: pgmap v297: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 42 KiB/s wr, 12 op/s 2024-09-11T23:28:00.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:28:00 smithi136 ceph-mon[35121]: pgmap v297: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 42 KiB/s wr, 12 op/s 2024-09-11T23:28:00.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:28:00 smithi026 ceph-mon[31172]: pgmap v297: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 42 KiB/s wr, 12 op/s 2024-09-11T23:28:02.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:28:02 smithi076 ceph-mon[35296]: pgmap v298: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 53 KiB/s wr, 19 op/s 2024-09-11T23:28:02.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:28:02 smithi136 ceph-mon[35121]: pgmap v298: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 53 KiB/s wr, 19 op/s 2024-09-11T23:28:02.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:28:02 smithi026 ceph-mon[31172]: pgmap v298: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 53 KiB/s wr, 19 op/s 2024-09-11T23:28:04.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:28:04 smithi076 ceph-mon[35296]: pgmap v299: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-09-11T23:28:04.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:28:04 smithi136 ceph-mon[35121]: pgmap v299: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-09-11T23:28:04.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:28:04 smithi026 ceph-mon[31172]: pgmap v299: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-09-11T23:28:06.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:28:06 smithi136 ceph-mon[35121]: pgmap v300: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 94 KiB/s wr, 37 op/s 2024-09-11T23:28:06.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:28:06 smithi026 ceph-mon[31172]: pgmap v300: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 94 KiB/s wr, 37 op/s 2024-09-11T23:28:06.552 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:28:06 smithi076 ceph-mon[35296]: pgmap v300: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 94 KiB/s wr, 37 op/s 2024-09-11T23:28:08.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:28:08 smithi136 ceph-mon[35121]: pgmap v301: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 89 KiB/s wr, 34 op/s 2024-09-11T23:28:08.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:28:08 smithi026 ceph-mon[31172]: pgmap v301: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 89 KiB/s wr, 34 op/s 2024-09-11T23:28:08.552 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:28:08 smithi076 ceph-mon[35296]: pgmap v301: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 89 KiB/s wr, 34 op/s 2024-09-11T23:28:10.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:28:10 smithi136 ceph-mon[35121]: pgmap v302: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 89 KiB/s wr, 32 op/s 2024-09-11T23:28:10.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:28:10 smithi026 ceph-mon[31172]: pgmap v302: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 89 KiB/s wr, 32 op/s 2024-09-11T23:28:10.552 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:28:10 smithi076 ceph-mon[35296]: pgmap v302: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 89 KiB/s wr, 32 op/s 2024-09-11T23:28:12.343 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.341+0000 7fa56f1cf640 1 -- 172.21.15.26:0/4205737596 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa568114ab0 msgr2=0x7fa568116ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:12.343 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.341+0000 7fa56f1cf640 1 --2- 172.21.15.26:0/4205737596 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa568114ab0 0x7fa568116ea0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fa560009ca0 tx=0x7fa56002a8c0 comp rx=0 tx=0).stop 2024-09-11T23:28:12.343 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.341+0000 7fa56f1cf640 1 -- 172.21.15.26:0/4205737596 shutdown_connections 2024-09-11T23:28:12.343 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.341+0000 7fa56f1cf640 1 --2- 172.21.15.26:0/4205737596 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa568114ab0 0x7fa568116ea0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:12.343 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.341+0000 7fa56f1cf640 1 --2- 172.21.15.26:0/4205737596 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa568109b10 0x7fa568114570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:12.343 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.341+0000 7fa56f1cf640 1 --2- 172.21.15.26:0/4205737596 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa5681091f0 0x7fa5681095d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:12.343 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.341+0000 7fa56f1cf640 1 -- 172.21.15.26:0/4205737596 >> 172.21.15.26:0/4205737596 conn(0x7fa56806ce00 msgr2=0x7fa56806d210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:28:12.344 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.342+0000 7fa56f1cf640 1 -- 172.21.15.26:0/4205737596 shutdown_connections 2024-09-11T23:28:12.344 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.342+0000 7fa56f1cf640 1 -- 172.21.15.26:0/4205737596 wait complete. 2024-09-11T23:28:12.344 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.343+0000 7fa56f1cf640 1 Processor -- start 2024-09-11T23:28:12.344 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.343+0000 7fa56f1cf640 1 -- start start 2024-09-11T23:28:12.345 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.343+0000 7fa56f1cf640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa5681091f0 0x7fa56811e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:12.345 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.343+0000 7fa56f1cf640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa568109b10 0x7fa56811ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:12.345 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.343+0000 7fa56f1cf640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa568114ab0 0x7fa5681199b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:12.345 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.343+0000 7fa56f1cf640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fa56810c040 con 0x7fa5681091f0 2024-09-11T23:28:12.345 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.343+0000 7fa56f1cf640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fa56810bec0 con 0x7fa568109b10 2024-09-11T23:28:12.345 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.343+0000 7fa56f1cf640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fa56810c1c0 con 0x7fa568114ab0 2024-09-11T23:28:12.345 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.344+0000 7fa56d76c640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa568114ab0 0x7fa5681199b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:12.346 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.344+0000 7fa56d76c640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa568114ab0 0x7fa5681199b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:55098/0 (socket says 172.21.15.26:55098) 2024-09-11T23:28:12.346 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.344+0000 7fa56d76c640 1 -- 172.21.15.26:0/1668899829 learned_addr learned my addr 172.21.15.26:0/1668899829 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:28:12.346 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.344+0000 7fa56cf6b640 1 --2- 172.21.15.26:0/1668899829 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa5681091f0 0x7fa56811e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:12.346 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.345+0000 7fa567fff640 1 --2- 172.21.15.26:0/1668899829 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa568109b10 0x7fa56811ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:12.347 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.345+0000 7fa56cf6b640 1 -- 172.21.15.26:0/1668899829 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa568114ab0 msgr2=0x7fa5681199b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:12.347 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.345+0000 7fa56cf6b640 1 --2- 172.21.15.26:0/1668899829 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa568114ab0 0x7fa5681199b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:12.347 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.345+0000 7fa56cf6b640 1 -- 172.21.15.26:0/1668899829 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa568109b10 msgr2=0x7fa56811ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:12.348 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.345+0000 7fa56cf6b640 1 --2- 172.21.15.26:0/1668899829 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa568109b10 0x7fa56811ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:12.348 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.345+0000 7fa56cf6b640 1 -- 172.21.15.26:0/1668899829 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa56811a270 con 0x7fa5681091f0 2024-09-11T23:28:12.348 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.346+0000 7fa56cf6b640 1 --2- 172.21.15.26:0/1668899829 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa5681091f0 0x7fa56811e8a0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fa55800b3f0 tx=0x7fa55800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:28:12.348 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.346+0000 7fa565ffb640 1 -- 172.21.15.26:0/1668899829 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa558015020 con 0x7fa5681091f0 2024-09-11T23:28:12.348 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.346+0000 7fa56f1cf640 1 -- 172.21.15.26:0/1668899829 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa56811a560 con 0x7fa5681091f0 2024-09-11T23:28:12.348 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.346+0000 7fa56f1cf640 1 -- 172.21.15.26:0/1668899829 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa568124790 con 0x7fa5681091f0 2024-09-11T23:28:12.348 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.346+0000 7fa565ffb640 1 -- 172.21.15.26:0/1668899829 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa558011d10 con 0x7fa5681091f0 2024-09-11T23:28:12.348 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.347+0000 7fa567fff640 1 --2- 172.21.15.26:0/1668899829 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa568109b10 0x7fa56811ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:28:12.349 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.348+0000 7fa565ffb640 1 -- 172.21.15.26:0/1668899829 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa558015020 con 0x7fa5681091f0 2024-09-11T23:28:12.353 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.351+0000 7fa572046740 1 Processor -- start 2024-09-11T23:28:12.353 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.352+0000 7fa572046740 1 -- start start 2024-09-11T23:28:12.353 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.352+0000 7fa565ffb640 1 -- 172.21.15.26:0/1668899829 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fa558018020 con 0x7fa5681091f0 2024-09-11T23:28:12.353 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.352+0000 7fa572046740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558d355daaf0 0x558d355daed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:12.353 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.352+0000 7fa572046740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558d355db410 0x558d355e5770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:12.353 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.352+0000 7fa572046740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558d355e5cb0 0x558d355e80a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:12.354 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.352+0000 7fa572046740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x558d3553f2f0 con 0x558d355e5cb0 2024-09-11T23:28:12.354 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.352+0000 7fa572046740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x558d354e7270 con 0x558d355db410 2024-09-11T23:28:12.354 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.352+0000 7fa572046740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x558d3553b2b0 con 0x558d355daaf0 2024-09-11T23:28:12.354 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.352+0000 7fa56d76c640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558d355daaf0 0x558d355daed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:12.354 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.352+0000 7fa56d76c640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558d355daaf0 0x558d355daed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:55102/0 (socket says 172.21.15.26:55102) 2024-09-11T23:28:12.354 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.352+0000 7fa56d76c640 1 -- 172.21.15.26:0/3986863557 learned_addr learned my addr 172.21.15.26:0/3986863557 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:28:12.354 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.353+0000 7fa567fff640 1 --2- 172.21.15.26:0/3986863557 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558d355e5cb0 0x558d355e80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:12.354 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.353+0000 7fa56d76c640 1 -- 172.21.15.26:0/3986863557 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558d355db410 msgr2=0x558d355e5770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:28:12.355 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.353+0000 7fa565ffb640 1 --2- 172.21.15.26:0/1668899829 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fa5500a3200 0x7fa5500a56c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:12.355 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.353+0000 7fa56cf6b640 1 --2- 172.21.15.26:0/3986863557 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558d355db410 0x558d355e5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:12.355 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.353+0000 7fa56d76c640 1 --2- 172.21.15.26:0/3986863557 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558d355db410 0x558d355e5770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:12.355 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.353+0000 7fa56d76c640 1 -- 172.21.15.26:0/3986863557 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558d355e5cb0 msgr2=0x558d355e80a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:28:12.355 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.353+0000 7fa56d76c640 1 --2- 172.21.15.26:0/3986863557 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558d355e5cb0 0x558d355e80a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:12.355 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.353+0000 7fa56d76c640 1 -- 172.21.15.26:0/3986863557 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d354e6340 con 0x558d355daaf0 2024-09-11T23:28:12.355 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.353+0000 7fa567fff640 1 --2- 172.21.15.26:0/3986863557 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558d355e5cb0 0x558d355e80a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:28:12.355 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.354+0000 7fa56d76c640 1 --2- 172.21.15.26:0/3986863557 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558d355daaf0 0x558d355daed0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa560031c90 tx=0x7fa560031cc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7f17bc0fe21a573c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:28:12.355 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.354+0000 7fa54effd640 1 -- 172.21.15.26:0/3986863557 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa560003b70 con 0x558d355daaf0 2024-09-11T23:28:12.355 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.354+0000 7fa56d76c640 1 --2- 172.21.15.26:0/1668899829 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fa5500a3200 0x7fa5500a56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:12.356 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.354+0000 7fa54effd640 1 -- 172.21.15.26:0/3986863557 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa560036030 con 0x558d355daaf0 2024-09-11T23:28:12.356 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.354+0000 7fa54effd640 1 -- 172.21.15.26:0/3986863557 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa560004720 con 0x558d355daaf0 2024-09-11T23:28:12.356 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.354+0000 7fa565ffb640 1 -- 172.21.15.26:0/1668899829 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8586+0+0 (secure 0 0 0) 0x7fa558011630 con 0x7fa5681091f0 2024-09-11T23:28:12.357 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.355+0000 7fa572046740 1 -- 172.21.15.26:0/3986863557 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558d355daaf0 msgr2=0x558d355daed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:12.357 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.355+0000 7fa572046740 1 --2- 172.21.15.26:0/3986863557 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558d355daaf0 0x558d355daed0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa560031c90 tx=0x7fa560031cc0 comp rx=0 tx=0).stop 2024-09-11T23:28:12.357 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.357+0000 7fa56d76c640 1 --2- 172.21.15.26:0/1668899829 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fa5500a3200 0x7fa5500a56c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fa560045000 tx=0x7fa56002f610 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:28:12.360 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.357+0000 7fa572046740 1 -- 172.21.15.26:0/3986863557 shutdown_connections 2024-09-11T23:28:12.360 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.357+0000 7fa572046740 1 --2- 172.21.15.26:0/3986863557 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558d355db410 0x558d355e5770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:12.360 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.357+0000 7fa572046740 1 --2- 172.21.15.26:0/3986863557 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558d355e5cb0 0x558d355e80a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:12.360 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.357+0000 7fa572046740 1 --2- 172.21.15.26:0/3986863557 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558d355daaf0 0x558d355daed0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:12.360 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.357+0000 7fa572046740 1 -- 172.21.15.26:0/3986863557 >> 172.21.15.26:0/3986863557 conn(0x558d355d7060 msgr2=0x558d355d7440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:28:12.360 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.359+0000 7fa572046740 1 -- 172.21.15.26:0/3986863557 shutdown_connections 2024-09-11T23:28:12.360 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.359+0000 7fa572046740 1 -- 172.21.15.26:0/3986863557 wait complete. 2024-09-11T23:28:12.361 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.360+0000 7fa572046740 1 -- start start 2024-09-11T23:28:12.363 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.360+0000 7fa572046740 1 Processor -- start 2024-09-11T23:28:12.363 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.360+0000 7fa572046740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558d355daaf0 0x558d355e2b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:12.363 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.360+0000 7fa572046740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558d355db410 0x558d355e3070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:12.363 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.360+0000 7fa572046740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558d355e5cb0 0x558d355e35b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:12.363 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.360+0000 7fa572046740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x558d355e8700 con 0x558d355daaf0 2024-09-11T23:28:12.363 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.360+0000 7fa572046740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x558d354c8290 con 0x558d355db410 2024-09-11T23:28:12.363 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.360+0000 7fa572046740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x558d355e8880 con 0x558d355e5cb0 2024-09-11T23:28:12.363 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.360+0000 7fa572046740 10 client.0 _ll_register_callbacks cb 0x558d354f9140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:28:12.363 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.361+0000 7fa567fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558d355db410 0x558d355e3070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:12.363 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.361+0000 7fa567fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558d355db410 0x558d355e3070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:39010/0 (socket says 172.21.15.26:39010) 2024-09-11T23:28:12.363 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.361+0000 7fa567fff640 1 -- 172.21.15.26:0/478808705 learned_addr learned my addr 172.21.15.26:0/478808705 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:28:12.363 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.361+0000 7fa56d76c640 1 --2- 172.21.15.26:0/478808705 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558d355e5cb0 0x558d355e35b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:12.363 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.361+0000 7fa567fff640 1 -- 172.21.15.26:0/478808705 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558d355e5cb0 msgr2=0x558d355e35b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:28:12.364 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.361+0000 7fa567fff640 1 --2- 172.21.15.26:0/478808705 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558d355e5cb0 0x558d355e35b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:12.364 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.361+0000 7fa567fff640 1 -- 172.21.15.26:0/478808705 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558d355daaf0 msgr2=0x558d355e2b30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T23:28:12.364 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.362+0000 7fa567fff640 1 --2- 172.21.15.26:0/478808705 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558d355daaf0 0x558d355e2b30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:12.364 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.362+0000 7fa567fff640 1 -- 172.21.15.26:0/478808705 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d35252210 con 0x558d355db410 2024-09-11T23:28:12.364 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.362+0000 7fa567fff640 1 --2- 172.21.15.26:0/478808705 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558d355db410 0x558d355e3070 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fa55c00e660 tx=0x7fa55c00eb20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e77677e9879ee58e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:28:12.365 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.362+0000 7fa547fff640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-11T23:28:12.365 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.363+0000 7fa547fff640 1 -- 172.21.15.26:0/478808705 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa55c03e060 con 0x558d355db410 2024-09-11T23:28:12.365 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.363+0000 7fa572046740 1 -- 172.21.15.26:0/478808705 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa560036030 con 0x558d355db410 2024-09-11T23:28:12.365 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.363+0000 7fa547fff640 1 -- 172.21.15.26:0/478808705 <== mon.2 v2:172.21.15.76:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8586+0+0 (secure 0 0 0) 0x7fa55c009a80 con 0x558d355db410 2024-09-11T23:28:12.365 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.363+0000 7fa547fff640 1 -- 172.21.15.26:0/478808705 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7fa55c009c80 con 0x558d355db410 2024-09-11T23:28:12.365 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.363+0000 7fa572046740 10 client.26489 fetch_fsmap learned FSMap version 51 2024-09-11T23:28:12.365 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.363+0000 7fa572046740 1 -- 172.21.15.26:0/478808705 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x558d3531e620 con 0x558d355db410 2024-09-11T23:28:12.365 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.364+0000 7fa547fff640 1 -- 172.21.15.26:0/478808705 <== mon.2 v2:172.21.15.76:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa560036030 con 0x558d355db410 2024-09-11T23:28:12.366 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.364+0000 7fa547fff640 1 -- 172.21.15.26:0/478808705 <== mon.2 v2:172.21.15.76:3300/0 5 ==== fsmap(e 51) ==== 8394+0+0 (secure 0 0 0) 0x7fa55c041590 con 0x558d355db410 2024-09-11T23:28:12.366 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.364+0000 7fa572046740 10 client.26489 fetch_fsmap finished waiting for FSMap version 51 2024-09-11T23:28:12.366 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.364+0000 7fa572046740 10 client.26489 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:28:12.366 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.364+0000 7fa572046740 20 client.26489 populate_metadata read hostname 'smithi026' 2024-09-11T23:28:12.366 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.364+0000 7fa572046740 1 --2- 172.21.15.26:0/478808705 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x558d355f5e60 0x558d355f8300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:12.366 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.364+0000 7fa572046740 4 client.26489 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:28:12.366 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.364+0000 7fa572046740 1 -- 172.21.15.26:0/478808705 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558d3554ca20 con 0x558d355f5e60 2024-09-11T23:28:12.366 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.365+0000 7fa56cf6b640 1 --2- 172.21.15.26:0/478808705 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x558d355f5e60 0x558d355f8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:12.367 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.366+0000 7fa56cf6b640 1 --2- 172.21.15.26:0/478808705 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x558d355f5e60 0x558d355f8300 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20b50e139a8ab0fc server_cookie=49d8752d1c89385a in_seq=0 out_seq=0 2024-09-11T23:28:12.368 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.367+0000 7fa547fff640 10 client.26489 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:28:12.370 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.370+0000 7fa547fff640 1 -- 172.21.15.26:0/478808705 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558d3554ca20 con 0x558d355f5e60 2024-09-11T23:28:12.371 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.370+0000 7fa547fff640 10 client.26489 handle_command_reply: tid=0 2024-09-11T23:28:12.371 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.370+0000 7fa572046740 1 client.26489 shutdown 2024-09-11T23:28:12.375 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.374+0000 7fa572046740 1 -- 172.21.15.26:0/478808705 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558d355db410 msgr2=0x558d355e3070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:12.375 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.374+0000 7fa572046740 1 --2- 172.21.15.26:0/478808705 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558d355db410 0x558d355e3070 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fa55c00e660 tx=0x7fa55c00eb20 comp rx=0 tx=0).stop 2024-09-11T23:28:12.375 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.375+0000 7fa572046740 1 -- 172.21.15.26:0/478808705 shutdown_connections 2024-09-11T23:28:12.376 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.375+0000 7fa572046740 1 --2- 172.21.15.26:0/478808705 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558d355db410 0x558d355e3070 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:12.376 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.375+0000 7fa572046740 1 --2- 172.21.15.26:0/478808705 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x558d355f5e60 0x558d355f8300 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:12.376 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.375+0000 7fa572046740 1 --2- 172.21.15.26:0/478808705 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558d355e5cb0 0x558d355e35b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:12.376 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.375+0000 7fa572046740 1 --2- 172.21.15.26:0/478808705 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558d355daaf0 0x558d355e2b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:12.376 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.375+0000 7fa572046740 1 -- 172.21.15.26:0/478808705 >> 172.21.15.26:0/478808705 conn(0x558d355d7060 msgr2=0x558d355dcdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:28:12.376 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.375+0000 7fa547fff640 0 client.26489 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:28:12.376 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.376+0000 7fa572046740 1 -- 172.21.15.26:0/478808705 shutdown_connections 2024-09-11T23:28:12.377 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.376+0000 7fa572046740 1 -- 172.21.15.26:0/478808705 wait complete. 2024-09-11T23:28:12.377 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.377+0000 7fa572046740 20 client.26489 trim_cache size 0 max 16384 2024-09-11T23:28:12.414 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.413+0000 7fa572046740 1 Processor -- start 2024-09-11T23:28:12.414 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.413+0000 7fa572046740 1 -- start start 2024-09-11T23:28:12.415 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.414+0000 7fa572046740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558d355daaf0 0x558d355dd5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:12.415 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.414+0000 7fa572046740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558d355db410 0x558d355e33e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:12.415 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.414+0000 7fa56cf6b640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558d355daaf0 0x558d355dd5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:12.415 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.414+0000 7fa56cf6b640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558d355daaf0 0x558d355dd5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:55126/0 (socket says 172.21.15.26:55126) 2024-09-11T23:28:12.415 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.414+0000 7fa56cf6b640 1 -- 172.21.15.26:0/2894619214 learned_addr learned my addr 172.21.15.26:0/2894619214 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:28:12.415 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.414+0000 7fa572046740 1 --2- 172.21.15.26:0/2894619214 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558d355e5cb0 0x558d355e4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:12.416 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.415+0000 7fa572046740 1 -- 172.21.15.26:0/2894619214 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fa55400e980 con 0x558d355db410 2024-09-11T23:28:12.416 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.415+0000 7fa572046740 1 -- 172.21.15.26:0/2894619214 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fa554010640 con 0x558d355e5cb0 2024-09-11T23:28:12.416 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.415+0000 7fa56d76c640 1 --2- 172.21.15.26:0/2894619214 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558d355e5cb0 0x558d355e4aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:12.416 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.415+0000 7fa572046740 1 -- 172.21.15.26:0/2894619214 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fa55400f050 con 0x558d355daaf0 2024-09-11T23:28:12.417 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.415+0000 7fa56d76c640 1 -- 172.21.15.26:0/2894619214 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558d355daaf0 msgr2=0x558d355dd5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:28:12.417 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.415+0000 7fa56d76c640 1 --2- 172.21.15.26:0/2894619214 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558d355daaf0 0x558d355dd5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:12.417 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.415+0000 7fa56d76c640 1 -- 172.21.15.26:0/2894619214 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558d355db410 msgr2=0x558d355e33e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T23:28:12.417 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.415+0000 7fa56d76c640 1 --2- 172.21.15.26:0/2894619214 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558d355db410 0x558d355e33e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:12.417 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.415+0000 7fa56d76c640 1 -- 172.21.15.26:0/2894619214 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d355da870 con 0x558d355e5cb0 2024-09-11T23:28:12.417 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.416+0000 7fa56d76c640 1 --2- 172.21.15.26:0/2894619214 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558d355e5cb0 0x558d355e4aa0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fa560031c60 tx=0x7fa56004f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b2972c8d7b820dc8 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:28:12.417 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.416+0000 7fa54e7fc640 1 -- 172.21.15.26:0/2894619214 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa560056040 con 0x558d355e5cb0 2024-09-11T23:28:12.418 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.416+0000 7fa54e7fc640 1 -- 172.21.15.26:0/2894619214 <== mon.2 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa560061070 con 0x558d355e5cb0 2024-09-11T23:28:12.418 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.416+0000 7fa54e7fc640 1 -- 172.21.15.26:0/2894619214 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa560031450 con 0x558d355e5cb0 2024-09-11T23:28:12.419 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.417+0000 7fa572046740 1 -- 172.21.15.26:0/2894619214 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558d355e5cb0 msgr2=0x558d355e4aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:12.419 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.417+0000 7fa572046740 1 --2- 172.21.15.26:0/2894619214 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558d355e5cb0 0x558d355e4aa0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fa560031c60 tx=0x7fa56004f000 comp rx=0 tx=0).stop 2024-09-11T23:28:12.419 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.418+0000 7fa572046740 1 -- 172.21.15.26:0/2894619214 shutdown_connections 2024-09-11T23:28:12.419 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.418+0000 7fa572046740 1 --2- 172.21.15.26:0/2894619214 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558d355e5cb0 0x558d355e4aa0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:12.419 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.418+0000 7fa572046740 1 --2- 172.21.15.26:0/2894619214 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558d355db410 0x558d355e33e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:12.419 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.418+0000 7fa572046740 1 --2- 172.21.15.26:0/2894619214 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558d355daaf0 0x558d355dd5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:12.419 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.418+0000 7fa572046740 1 -- 172.21.15.26:0/2894619214 >> 172.21.15.26:0/2894619214 conn(0x558d355d7060 msgr2=0x558d355e4390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:28:12.419 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.418+0000 7fa572046740 1 -- 172.21.15.26:0/2894619214 shutdown_connections 2024-09-11T23:28:12.420 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.418+0000 7fa572046740 1 -- 172.21.15.26:0/2894619214 wait complete. 2024-09-11T23:28:12.420 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.418+0000 7fa572046740 1 -- start start 2024-09-11T23:28:12.420 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.418+0000 7fa572046740 1 Processor -- start 2024-09-11T23:28:12.421 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.419+0000 7fa572046740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558d355daaf0 0x558d35658f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:12.422 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.419+0000 7fa572046740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558d355db410 0x558d35659490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:12.422 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.419+0000 7fa56cf6b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558d355daaf0 0x558d35658f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:12.422 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.419+0000 7fa56cf6b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558d355daaf0 0x558d35658f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35994/0 (socket says 172.21.15.26:35994) 2024-09-11T23:28:12.422 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.419+0000 7fa56cf6b640 1 -- 172.21.15.26:0/1201705806 learned_addr learned my addr 172.21.15.26:0/1201705806 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:28:12.422 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.419+0000 7fa572046740 1 --2- 172.21.15.26:0/1201705806 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558d355e5cb0 0x558d355e9110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:12.422 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.419+0000 7fa572046740 1 -- 172.21.15.26:0/1201705806 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fa55400ff70 con 0x558d355daaf0 2024-09-11T23:28:12.422 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.419+0000 7fa572046740 1 -- 172.21.15.26:0/1201705806 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fa55400f7e0 con 0x558d355e5cb0 2024-09-11T23:28:12.422 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.419+0000 7fa567fff640 1 --2- 172.21.15.26:0/1201705806 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558d355db410 0x558d35659490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:12.422 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.420+0000 7fa572046740 1 -- 172.21.15.26:0/1201705806 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fa55400c980 con 0x558d355db410 2024-09-11T23:28:12.422 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.420+0000 7fa56d76c640 1 --2- 172.21.15.26:0/1201705806 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558d355e5cb0 0x558d355e9110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:12.422 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.420+0000 7fa572046740 10 client.0 _ll_register_callbacks cb 0x7fa55c00c330 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:28:12.422 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.420+0000 7fa56d76c640 1 -- 172.21.15.26:0/1201705806 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558d355db410 msgr2=0x558d35659490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:28:12.423 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.420+0000 7fa56d76c640 1 --2- 172.21.15.26:0/1201705806 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558d355db410 0x558d35659490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:12.423 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.420+0000 7fa56d76c640 1 -- 172.21.15.26:0/1201705806 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558d355daaf0 msgr2=0x558d35658f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:28:12.423 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.420+0000 7fa56d76c640 1 --2- 172.21.15.26:0/1201705806 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558d355daaf0 0x558d35658f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:12.423 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.420+0000 7fa56d76c640 1 -- 172.21.15.26:0/1201705806 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d355e98c0 con 0x558d355e5cb0 2024-09-11T23:28:12.423 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.420+0000 7fa56d76c640 1 --2- 172.21.15.26:0/1201705806 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558d355e5cb0 0x558d355e9110 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa56005e040 tx=0x7fa56002a8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e925abfbabfe31b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:28:12.423 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.421+0000 7fa547fff640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-11T23:28:12.423 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.421+0000 7fa547fff640 1 -- 172.21.15.26:0/1201705806 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa56002f9e0 con 0x558d355e5cb0 2024-09-11T23:28:12.423 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.421+0000 7fa547fff640 1 -- 172.21.15.26:0/1201705806 <== mon.2 v2:172.21.15.76:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8586+0+0 (secure 0 0 0) 0x7fa560003b80 con 0x558d355e5cb0 2024-09-11T23:28:12.423 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.421+0000 7fa572046740 1 -- 172.21.15.26:0/1201705806 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa560061070 con 0x558d355e5cb0 2024-09-11T23:28:12.423 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.421+0000 7fa567fff640 1 --2- 172.21.15.26:0/1201705806 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558d355db410 0x558d35659490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:28:12.424 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.423+0000 7fa547fff640 1 -- 172.21.15.26:0/1201705806 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7fa560074050 con 0x558d355e5cb0 2024-09-11T23:28:12.425 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.424+0000 7fa572046740 10 client.26495 fetch_fsmap learned FSMap version 51 2024-09-11T23:28:12.425 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.424+0000 7fa572046740 1 -- 172.21.15.26:0/1201705806 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x558d356606d0 con 0x558d355e5cb0 2024-09-11T23:28:12.427 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.426+0000 7fa547fff640 1 -- 172.21.15.26:0/1201705806 <== mon.2 v2:172.21.15.76:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa560061070 con 0x558d355e5cb0 2024-09-11T23:28:12.427 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.426+0000 7fa547fff640 1 -- 172.21.15.26:0/1201705806 <== mon.2 v2:172.21.15.76:3300/0 5 ==== fsmap(e 51) ==== 8394+0+0 (secure 0 0 0) 0x7fa560043070 con 0x558d355e5cb0 2024-09-11T23:28:12.427 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.426+0000 7fa572046740 10 client.26495 fetch_fsmap finished waiting for FSMap version 51 2024-09-11T23:28:12.427 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.426+0000 7fa572046740 10 client.26495 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:28:12.427 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.426+0000 7fa572046740 20 client.26495 populate_metadata read hostname 'smithi026' 2024-09-11T23:28:12.427 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.426+0000 7fa572046740 1 --2- 172.21.15.26:0/1201705806 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x558d35660d30 0x558d35663120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:12.427 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.426+0000 7fa572046740 4 client.26495 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T23:28:12.428 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.426+0000 7fa572046740 1 -- 172.21.15.26:0/1201705806 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558d35546d70 con 0x558d35660d30 2024-09-11T23:28:12.428 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.427+0000 7fa56cf6b640 1 --2- 172.21.15.26:0/1201705806 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x558d35660d30 0x558d35663120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:12.428 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.427+0000 7fa547fff640 10 client.26495 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:28:12.428 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.427+0000 7fa56cf6b640 1 --2- 172.21.15.26:0/1201705806 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x558d35660d30 0x558d35663120 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=22ad5cc4bc96052b server_cookie=8d06c7d63c3aa20f in_seq=0 out_seq=0 2024-09-11T23:28:12.462 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.462+0000 7fa547fff640 1 -- 172.21.15.26:0/1201705806 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558d35546d70 con 0x558d35660d30 2024-09-11T23:28:12.462 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.462+0000 7fa547fff640 10 client.26495 handle_command_reply: tid=0 2024-09-11T23:28:12.463 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.462+0000 7fa572046740 1 client.26495 shutdown 2024-09-11T23:28:12.463 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.463+0000 7fa572046740 1 -- 172.21.15.26:0/1201705806 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558d355e5cb0 msgr2=0x558d355e9110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:12.464 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.463+0000 7fa572046740 1 --2- 172.21.15.26:0/1201705806 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558d355e5cb0 0x558d355e9110 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa56005e040 tx=0x7fa56002a8c0 comp rx=0 tx=0).stop 2024-09-11T23:28:12.464 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.464+0000 7fa572046740 1 -- 172.21.15.26:0/1201705806 shutdown_connections 2024-09-11T23:28:12.464 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.464+0000 7fa572046740 1 --2- 172.21.15.26:0/1201705806 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558d355e5cb0 0x558d355e9110 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:12.464 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.464+0000 7fa572046740 1 --2- 172.21.15.26:0/1201705806 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x558d35660d30 0x558d35663120 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:12.464 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.464+0000 7fa572046740 1 --2- 172.21.15.26:0/1201705806 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558d355db410 0x558d35659490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:12.465 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.464+0000 7fa572046740 1 --2- 172.21.15.26:0/1201705806 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558d355daaf0 0x558d35658f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:12.465 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.464+0000 7fa547fff640 0 client.26495 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:28:12.465 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.464+0000 7fa572046740 1 -- 172.21.15.26:0/1201705806 >> 172.21.15.26:0/1201705806 conn(0x558d355d7060 msgr2=0x558d355d61c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:28:12.468 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.464+0000 7fa572046740 1 -- 172.21.15.26:0/1201705806 shutdown_connections 2024-09-11T23:28:12.468 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.465+0000 7fa572046740 1 -- 172.21.15.26:0/1201705806 wait complete. 2024-09-11T23:28:12.468 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.465+0000 7fa572046740 20 client.26495 trim_cache size 0 max 16384 2024-09-11T23:28:12.469 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-09-11T23:28:12.469 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "message": "", 2024-09-11T23:28:12.469 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-09-11T23:28:12.469 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-09-11T23:28:12.469 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.466+0000 7fa56f1cf640 1 -- 172.21.15.26:0/1668899829 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fa5500a3200 msgr2=0x7fa5500a56c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:12.469 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.466+0000 7fa56f1cf640 1 --2- 172.21.15.26:0/1668899829 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fa5500a3200 0x7fa5500a56c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fa560045000 tx=0x7fa56002f610 comp rx=0 tx=0).stop 2024-09-11T23:28:12.469 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.466+0000 7fa56f1cf640 1 -- 172.21.15.26:0/1668899829 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa5681091f0 msgr2=0x7fa56811e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:12.469 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.466+0000 7fa56f1cf640 1 --2- 172.21.15.26:0/1668899829 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa5681091f0 0x7fa56811e8a0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fa55800b3f0 tx=0x7fa55800b8b0 comp rx=0 tx=0).stop 2024-09-11T23:28:12.470 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.466+0000 7fa56f1cf640 1 -- 172.21.15.26:0/1668899829 shutdown_connections 2024-09-11T23:28:12.470 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.466+0000 7fa56f1cf640 1 --2- 172.21.15.26:0/1668899829 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fa5500a3200 0x7fa5500a56c0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:12.470 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.466+0000 7fa56f1cf640 1 --2- 172.21.15.26:0/1668899829 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa568109b10 0x7fa56811ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:12.470 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.466+0000 7fa56f1cf640 1 --2- 172.21.15.26:0/1668899829 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa568114ab0 0x7fa5681199b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:12.470 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.466+0000 7fa56f1cf640 1 --2- 172.21.15.26:0/1668899829 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa5681091f0 0x7fa56811e8a0 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:12.470 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.466+0000 7fa56f1cf640 1 -- 172.21.15.26:0/1668899829 >> 172.21.15.26:0/1668899829 conn(0x7fa56806ce00 msgr2=0x7fa568070750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:28:12.470 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.466+0000 7fa56f1cf640 1 -- 172.21.15.26:0/1668899829 shutdown_connections 2024-09-11T23:28:12.470 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:12.466+0000 7fa56f1cf640 1 -- 172.21.15.26:0/1668899829 wait complete. 2024-09-11T23:28:12.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:28:12 smithi136 ceph-mon[35121]: pgmap v303: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 98 KiB/s wr, 39 op/s 2024-09-11T23:28:12.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:28:12 smithi026 ceph-mon[31172]: pgmap v303: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 98 KiB/s wr, 39 op/s 2024-09-11T23:28:12.552 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:28:12 smithi076 ceph-mon[35296]: pgmap v303: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 98 KiB/s wr, 39 op/s 2024-09-11T23:28:13.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:28:13 smithi136 ceph-mon[35121]: osdmap e93: 12 total, 12 up, 12 in 2024-09-11T23:28:13.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:28:13 smithi026 ceph-mon[31172]: osdmap e93: 12 total, 12 up, 12 in 2024-09-11T23:28:13.552 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:28:13 smithi076 ceph-mon[35296]: osdmap e93: 12 total, 12 up, 12 in 2024-09-11T23:28:14.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:28:14 smithi136 ceph-mon[35121]: pgmap v305: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 104 KiB/s wr, 39 op/s 2024-09-11T23:28:14.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:28:14 smithi026 ceph-mon[31172]: pgmap v305: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 104 KiB/s wr, 39 op/s 2024-09-11T23:28:14.552 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:28:14 smithi076 ceph-mon[35296]: pgmap v305: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 104 KiB/s wr, 39 op/s 2024-09-11T23:28:16.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:28:16 smithi136 ceph-mon[35121]: pgmap v306: 97 pgs: 24 active+clean+snaptrim_wait, 14 active+clean+snaptrim, 59 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 51 KiB/s wr, 18 op/s 2024-09-11T23:28:16.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:28:16 smithi026 ceph-mon[31172]: pgmap v306: 97 pgs: 24 active+clean+snaptrim_wait, 14 active+clean+snaptrim, 59 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 51 KiB/s wr, 18 op/s 2024-09-11T23:28:16.552 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:28:16 smithi076 ceph-mon[35296]: pgmap v306: 97 pgs: 24 active+clean+snaptrim_wait, 14 active+clean+snaptrim, 59 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 51 KiB/s wr, 18 op/s 2024-09-11T23:28:18.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:28:18 smithi136 ceph-mon[35121]: pgmap v307: 97 pgs: 39 active+clean+snaptrim_wait, 21 active+clean+snaptrim, 37 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 52 KiB/s wr, 18 op/s 2024-09-11T23:28:18.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:28:18 smithi026 ceph-mon[31172]: pgmap v307: 97 pgs: 39 active+clean+snaptrim_wait, 21 active+clean+snaptrim, 37 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 52 KiB/s wr, 18 op/s 2024-09-11T23:28:18.552 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:28:18 smithi076 ceph-mon[35296]: pgmap v307: 97 pgs: 39 active+clean+snaptrim_wait, 21 active+clean+snaptrim, 37 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 52 KiB/s wr, 18 op/s 2024-09-11T23:28:19.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:28:19 smithi136 ceph-mon[35121]: pgmap v308: 97 pgs: 39 active+clean+snaptrim_wait, 21 active+clean+snaptrim, 37 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 52 KiB/s wr, 18 op/s 2024-09-11T23:28:19.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:28:19 smithi026 ceph-mon[31172]: pgmap v308: 97 pgs: 39 active+clean+snaptrim_wait, 21 active+clean+snaptrim, 37 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 52 KiB/s wr, 18 op/s 2024-09-11T23:28:19.552 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:28:19 smithi076 ceph-mon[35296]: pgmap v308: 97 pgs: 39 active+clean+snaptrim_wait, 21 active+clean+snaptrim, 37 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 52 KiB/s wr, 18 op/s 2024-09-11T23:28:21.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:28:21 smithi136 ceph-mon[35121]: pgmap v309: 97 pgs: 5 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 90 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 52 KiB/s wr, 17 op/s 2024-09-11T23:28:22.000 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:28:21 smithi026 ceph-mon[31172]: pgmap v309: 97 pgs: 5 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 90 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 52 KiB/s wr, 17 op/s 2024-09-11T23:28:22.052 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:28:21 smithi076 ceph-mon[35296]: pgmap v309: 97 pgs: 5 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 90 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 52 KiB/s wr, 17 op/s 2024-09-11T23:28:24.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:28:23 smithi026 ceph-mon[31172]: pgmap v310: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 50 KiB/s wr, 17 op/s 2024-09-11T23:28:24.052 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:28:23 smithi076 ceph-mon[35296]: pgmap v310: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 50 KiB/s wr, 17 op/s 2024-09-11T23:28:24.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:28:23 smithi136 ceph-mon[35121]: pgmap v310: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 50 KiB/s wr, 17 op/s 2024-09-11T23:28:25.576 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.575+0000 7fc38ce5e640 1 -- 172.21.15.26:0/3937379764 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc3881020c0 msgr2=0x7fc38810d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:25.576 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.575+0000 7fc38ce5e640 1 --2- 172.21.15.26:0/3937379764 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc3881020c0 0x7fc38810d8f0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fc380009ca0 tx=0x7fc38002a8c0 comp rx=0 tx=0).stop 2024-09-11T23:28:25.576 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.575+0000 7fc38ce5e640 1 -- 172.21.15.26:0/3937379764 shutdown_connections 2024-09-11T23:28:25.576 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.575+0000 7fc38ce5e640 1 --2- 172.21.15.26:0/3937379764 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc388101740 0x7fc388101b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:25.576 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.575+0000 7fc38ce5e640 1 --2- 172.21.15.26:0/3937379764 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc3881020c0 0x7fc38810d8f0 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:25.576 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.575+0000 7fc38ce5e640 1 --2- 172.21.15.26:0/3937379764 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc388106b70 0x7fc388106f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:25.576 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.575+0000 7fc38ce5e640 1 -- 172.21.15.26:0/3937379764 >> 172.21.15.26:0/3937379764 conn(0x7fc388075ef0 msgr2=0x7fc388076300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:28:25.576 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.575+0000 7fc38ce5e640 1 -- 172.21.15.26:0/3937379764 shutdown_connections 2024-09-11T23:28:25.576 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.575+0000 7fc38ce5e640 1 -- 172.21.15.26:0/3937379764 wait complete. 2024-09-11T23:28:25.577 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.576+0000 7fc38ce5e640 1 Processor -- start 2024-09-11T23:28:25.577 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.576+0000 7fc38ce5e640 1 -- start start 2024-09-11T23:28:25.578 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.577+0000 7fc38ce5e640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc388101740 0x7fc3881983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:25.578 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.577+0000 7fc38ce5e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc3881020c0 0x7fc388198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:25.578 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.577+0000 7fc38ce5e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc388106b70 0x7fc38819cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:25.578 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.577+0000 7fc38ce5e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc388110e90 con 0x7fc388106b70 2024-09-11T23:28:25.579 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.577+0000 7fc38ce5e640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fc388110d10 con 0x7fc3881020c0 2024-09-11T23:28:25.579 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.577+0000 7fc38ce5e640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fc388111010 con 0x7fc388101740 2024-09-11T23:28:25.579 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.577+0000 7fc386d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc388106b70 0x7fc38819cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:25.579 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.577+0000 7fc385d9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc3881020c0 0x7fc388198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:25.579 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.577+0000 7fc386d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc388106b70 0x7fc38819cc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59226/0 (socket says 172.21.15.26:59226) 2024-09-11T23:28:25.579 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.577+0000 7fc386d9d640 1 -- 172.21.15.26:0/2440789784 learned_addr learned my addr 172.21.15.26:0/2440789784 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:28:25.579 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.577+0000 7fc38659c640 1 --2- 172.21.15.26:0/2440789784 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc388101740 0x7fc3881983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:25.579 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.577+0000 7fc385d9b640 1 -- 172.21.15.26:0/2440789784 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc388101740 msgr2=0x7fc3881983c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:25.579 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.577+0000 7fc385d9b640 1 --2- 172.21.15.26:0/2440789784 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc388101740 0x7fc3881983c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:25.579 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.577+0000 7fc385d9b640 1 -- 172.21.15.26:0/2440789784 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc388106b70 msgr2=0x7fc38819cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:25.579 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.577+0000 7fc385d9b640 1 --2- 172.21.15.26:0/2440789784 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc388106b70 0x7fc38819cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:25.579 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.577+0000 7fc385d9b640 1 -- 172.21.15.26:0/2440789784 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc38819d3b0 con 0x7fc3881020c0 2024-09-11T23:28:25.579 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.577+0000 7fc386d9d640 1 --2- 172.21.15.26:0/2440789784 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc388106b70 0x7fc38819cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:28:25.580 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.577+0000 7fc385d9b640 1 --2- 172.21.15.26:0/2440789784 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc3881020c0 0x7fc388198900 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc37000e9f0 tx=0x7fc37000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:28:25.580 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.578+0000 7fc37f7fe640 1 -- 172.21.15.26:0/2440789784 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc37000ccc0 con 0x7fc3881020c0 2024-09-11T23:28:25.580 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.578+0000 7fc38ce5e640 1 -- 172.21.15.26:0/2440789784 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc38819d6a0 con 0x7fc3881020c0 2024-09-11T23:28:25.580 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.578+0000 7fc37f7fe640 1 -- 172.21.15.26:0/2440789784 <== mon.2 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc370010910 con 0x7fc3881020c0 2024-09-11T23:28:25.580 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.578+0000 7fc37f7fe640 1 -- 172.21.15.26:0/2440789784 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc370018960 con 0x7fc3881020c0 2024-09-11T23:28:25.580 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.578+0000 7fc38ce5e640 1 -- 172.21.15.26:0/2440789784 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3881a4f30 con 0x7fc3881020c0 2024-09-11T23:28:25.587 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.580+0000 7fc38fbf7740 1 Processor -- start 2024-09-11T23:28:25.587 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.580+0000 7fc38fbf7740 1 -- start start 2024-09-11T23:28:25.587 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.581+0000 7fc37f7fe640 1 -- 172.21.15.26:0/2440789784 <== mon.2 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fc370010ac0 con 0x7fc3881020c0 2024-09-11T23:28:25.587 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.581+0000 7fc38fbf7740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555cd19d1af0 0x555cd19d1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:25.587 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.581+0000 7fc38fbf7740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555cd19d2410 0x555cd19dc770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:25.587 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.581+0000 7fc38fbf7740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555cd19dccb0 0x555cd19df0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:25.587 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.581+0000 7fc38fbf7740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x555cd19362f0 con 0x555cd19d1af0 2024-09-11T23:28:25.587 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.581+0000 7fc38fbf7740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x555cd18de270 con 0x555cd19dccb0 2024-09-11T23:28:25.588 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.581+0000 7fc38fbf7740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x555cd19322b0 con 0x555cd19d2410 2024-09-11T23:28:25.588 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.581+0000 7fc386d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555cd19d1af0 0x555cd19d1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:25.588 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.581+0000 7fc386d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555cd19d1af0 0x555cd19d1ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59232/0 (socket says 172.21.15.26:59232) 2024-09-11T23:28:25.588 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.581+0000 7fc386d9d640 1 -- 172.21.15.26:0/35341031 learned_addr learned my addr 172.21.15.26:0/35341031 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:28:25.588 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.581+0000 7fc38659c640 1 --2- 172.21.15.26:0/35341031 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555cd19d2410 0x555cd19dc770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:25.588 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.581+0000 7fc385d9b640 1 --2- 172.21.15.26:0/35341031 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555cd19dccb0 0x555cd19df0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:25.588 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.581+0000 7fc386d9d640 1 -- 172.21.15.26:0/35341031 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555cd19d2410 msgr2=0x555cd19dc770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:28:25.588 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.581+0000 7fc386d9d640 1 --2- 172.21.15.26:0/35341031 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555cd19d2410 0x555cd19dc770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:25.589 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.581+0000 7fc386d9d640 1 -- 172.21.15.26:0/35341031 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555cd19dccb0 msgr2=0x555cd19df0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:28:25.589 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.581+0000 7fc386d9d640 1 --2- 172.21.15.26:0/35341031 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555cd19dccb0 0x555cd19df0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:25.589 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.581+0000 7fc386d9d640 1 -- 172.21.15.26:0/35341031 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555cd18dd340 con 0x555cd19d1af0 2024-09-11T23:28:25.589 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.581+0000 7fc385d9b640 1 --2- 172.21.15.26:0/35341031 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555cd19dccb0 0x555cd19df0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:28:25.589 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.581+0000 7fc386d9d640 1 --2- 172.21.15.26:0/35341031 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555cd19d1af0 0x555cd19d1ed0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fc380031f80 tx=0x7fc3800098d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2195927e3f33782b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:28:25.589 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.582+0000 7fc37f7fe640 1 --2- 172.21.15.26:0/2440789784 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fc3740a2da0 0x7fc3740a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:25.589 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.582+0000 7fc37cff9640 1 -- 172.21.15.26:0/35341031 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc380003b70 con 0x555cd19d1af0 2024-09-11T23:28:25.589 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.582+0000 7fc37cff9640 1 -- 172.21.15.26:0/35341031 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc380036040 con 0x555cd19d1af0 2024-09-11T23:28:25.589 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.582+0000 7fc38659c640 1 --2- 172.21.15.26:0/2440789784 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fc3740a2da0 0x7fc3740a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:25.589 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.582+0000 7fc37cff9640 1 -- 172.21.15.26:0/35341031 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc380004700 con 0x555cd19d1af0 2024-09-11T23:28:25.589 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.582+0000 7fc37f7fe640 1 -- 172.21.15.26:0/2440789784 <== mon.2 v2:172.21.15.76:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8586+0+0 (secure 0 0 0) 0x7fc3700cf800 con 0x7fc3881020c0 2024-09-11T23:28:25.589 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.583+0000 7fc38fbf7740 1 -- 172.21.15.26:0/35341031 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555cd19d1af0 msgr2=0x555cd19d1ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:25.589 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.583+0000 7fc38659c640 1 --2- 172.21.15.26:0/2440789784 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fc3740a2da0 0x7fc3740a5260 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x555cd19ceec0 tx=0x7fc37800b410 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:28:25.590 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.583+0000 7fc38fbf7740 1 --2- 172.21.15.26:0/35341031 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555cd19d1af0 0x555cd19d1ed0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fc380031f80 tx=0x7fc3800098d0 comp rx=0 tx=0).stop 2024-09-11T23:28:25.590 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.583+0000 7fc38fbf7740 1 -- 172.21.15.26:0/35341031 shutdown_connections 2024-09-11T23:28:25.590 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.583+0000 7fc38fbf7740 1 --2- 172.21.15.26:0/35341031 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555cd19dccb0 0x555cd19df0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:25.590 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.583+0000 7fc38fbf7740 1 --2- 172.21.15.26:0/35341031 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555cd19d2410 0x555cd19dc770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:25.590 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.583+0000 7fc38fbf7740 1 --2- 172.21.15.26:0/35341031 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555cd19d1af0 0x555cd19d1ed0 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:25.590 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.583+0000 7fc38fbf7740 1 -- 172.21.15.26:0/35341031 >> 172.21.15.26:0/35341031 conn(0x555cd19ce060 msgr2=0x555cd19ce440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:28:25.590 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.583+0000 7fc38fbf7740 1 -- 172.21.15.26:0/35341031 shutdown_connections 2024-09-11T23:28:25.590 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.583+0000 7fc38fbf7740 1 -- 172.21.15.26:0/35341031 wait complete. 2024-09-11T23:28:25.590 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.583+0000 7fc38fbf7740 1 -- start start 2024-09-11T23:28:25.590 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.583+0000 7fc38fbf7740 1 Processor -- start 2024-09-11T23:28:25.590 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.583+0000 7fc38fbf7740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555cd19d1af0 0x555cd19d9b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:25.590 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.583+0000 7fc38fbf7740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555cd19d2410 0x555cd19da0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:25.590 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.583+0000 7fc38fbf7740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555cd19dccb0 0x555cd19da5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:25.590 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.583+0000 7fc38fbf7740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x555cd19df700 con 0x555cd19d2410 2024-09-11T23:28:25.590 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.583+0000 7fc38fbf7740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x555cd18bf290 con 0x555cd19dccb0 2024-09-11T23:28:25.591 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.583+0000 7fc38fbf7740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x555cd19df880 con 0x555cd19d1af0 2024-09-11T23:28:25.591 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.583+0000 7fc38fbf7740 10 client.0 _ll_register_callbacks cb 0x555cd18f0140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:28:25.591 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.584+0000 7fc385d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555cd19d2410 0x555cd19da0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:25.591 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.584+0000 7fc385d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555cd19d2410 0x555cd19da0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59246/0 (socket says 172.21.15.26:59246) 2024-09-11T23:28:25.591 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.584+0000 7fc385d9b640 1 -- 172.21.15.26:0/2872115611 learned_addr learned my addr 172.21.15.26:0/2872115611 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:28:25.591 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.584+0000 7fc38659c640 1 --2- 172.21.15.26:0/2872115611 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555cd19d1af0 0x555cd19d9b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:25.591 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.584+0000 7fc386d9d640 1 --2- 172.21.15.26:0/2872115611 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555cd19dccb0 0x555cd19da5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:25.591 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.584+0000 7fc385d9b640 1 -- 172.21.15.26:0/2872115611 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555cd19d1af0 msgr2=0x555cd19d9b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:28:25.591 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.584+0000 7fc385d9b640 1 --2- 172.21.15.26:0/2872115611 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555cd19d1af0 0x555cd19d9b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:25.591 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.584+0000 7fc385d9b640 1 -- 172.21.15.26:0/2872115611 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555cd19dccb0 msgr2=0x555cd19da5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:28:25.591 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.584+0000 7fc385d9b640 1 --2- 172.21.15.26:0/2872115611 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555cd19dccb0 0x555cd19da5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:25.591 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.584+0000 7fc385d9b640 1 -- 172.21.15.26:0/2872115611 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555cd1649210 con 0x555cd19d2410 2024-09-11T23:28:25.592 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.584+0000 7fc385d9b640 1 --2- 172.21.15.26:0/2872115611 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555cd19d2410 0x555cd19da0a0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fc37009bd20 tx=0x7fc3700d5310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9e28f552ada01ac server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:28:25.592 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.584+0000 7fc36dffb640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:28:25.592 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.584+0000 7fc36dffb640 1 -- 172.21.15.26:0/2872115611 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3700d2e60 con 0x555cd19d2410 2024-09-11T23:28:25.592 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.584+0000 7fc36dffb640 1 -- 172.21.15.26:0/2872115611 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8586+0+0 (secure 0 0 0) 0x7fc37009b1e0 con 0x555cd19d2410 2024-09-11T23:28:25.592 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.584+0000 7fc38fbf7740 1 -- 172.21.15.26:0/2872115611 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc380036040 con 0x555cd19d2410 2024-09-11T23:28:25.592 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.584+0000 7fc36dffb640 1 -- 172.21.15.26:0/2872115611 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7fc37009bd90 con 0x555cd19d2410 2024-09-11T23:28:25.592 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.585+0000 7fc38fbf7740 10 client.16947 fetch_fsmap learned FSMap version 51 2024-09-11T23:28:25.592 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.585+0000 7fc38fbf7740 1 -- 172.21.15.26:0/2872115611 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x555cd1715620 con 0x555cd19d2410 2024-09-11T23:28:25.592 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.585+0000 7fc36dffb640 1 -- 172.21.15.26:0/2872115611 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc380036040 con 0x555cd19d2410 2024-09-11T23:28:25.592 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.585+0000 7fc36dffb640 1 -- 172.21.15.26:0/2872115611 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 51) ==== 8394+0+0 (secure 0 0 0) 0x7fc370018c20 con 0x555cd19d2410 2024-09-11T23:28:25.592 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.585+0000 7fc38fbf7740 10 client.16947 fetch_fsmap finished waiting for FSMap version 51 2024-09-11T23:28:25.592 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.585+0000 7fc38fbf7740 10 client.16947 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:28:25.592 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.585+0000 7fc38fbf7740 20 client.16947 populate_metadata read hostname 'smithi026' 2024-09-11T23:28:25.593 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.585+0000 7fc38fbf7740 1 --2- 172.21.15.26:0/2872115611 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x555cd19ecb00 0x555cd19eeef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:25.593 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.585+0000 7fc38fbf7740 4 client.16947 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:28:25.593 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.585+0000 7fc38fbf7740 1 -- 172.21.15.26:0/2872115611 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555cd1943a20 con 0x555cd19ecb00 2024-09-11T23:28:25.593 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.585+0000 7fc38659c640 1 --2- 172.21.15.26:0/2872115611 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x555cd19ecb00 0x555cd19eeef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:25.593 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.586+0000 7fc38659c640 1 --2- 172.21.15.26:0/2872115611 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x555cd19ecb00 0x555cd19eeef0 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9bfa359e1fc606db server_cookie=300f7d61d1a418ed in_seq=0 out_seq=0 2024-09-11T23:28:25.593 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.586+0000 7fc36dffb640 10 client.16947 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:28:25.593 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.586+0000 7fc36dffb640 1 -- 172.21.15.26:0/2872115611 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555cd1943a20 con 0x555cd19ecb00 2024-09-11T23:28:25.593 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.586+0000 7fc36dffb640 10 client.16947 handle_command_reply: tid=0 2024-09-11T23:28:25.593 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.587+0000 7fc38fbf7740 1 client.16947 shutdown 2024-09-11T23:28:25.593 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.587+0000 7fc38fbf7740 1 -- 172.21.15.26:0/2872115611 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555cd19d2410 msgr2=0x555cd19da0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:25.593 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.587+0000 7fc38fbf7740 1 --2- 172.21.15.26:0/2872115611 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555cd19d2410 0x555cd19da0a0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fc37009bd20 tx=0x7fc3700d5310 comp rx=0 tx=0).stop 2024-09-11T23:28:25.593 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.587+0000 7fc38fbf7740 1 -- 172.21.15.26:0/2872115611 shutdown_connections 2024-09-11T23:28:25.593 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.587+0000 7fc38fbf7740 1 --2- 172.21.15.26:0/2872115611 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555cd19dccb0 0x555cd19da5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:25.593 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.587+0000 7fc38fbf7740 1 --2- 172.21.15.26:0/2872115611 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x555cd19ecb00 0x555cd19eeef0 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:25.594 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.587+0000 7fc38fbf7740 1 --2- 172.21.15.26:0/2872115611 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555cd19d2410 0x555cd19da0a0 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:25.594 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.587+0000 7fc38fbf7740 1 --2- 172.21.15.26:0/2872115611 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555cd19d1af0 0x555cd19d9b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:25.594 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.587+0000 7fc36dffb640 0 client.16947 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:28:25.594 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.587+0000 7fc38fbf7740 1 -- 172.21.15.26:0/2872115611 >> 172.21.15.26:0/2872115611 conn(0x555cd19ce060 msgr2=0x555cd19cd1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:28:25.594 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.587+0000 7fc38fbf7740 1 -- 172.21.15.26:0/2872115611 shutdown_connections 2024-09-11T23:28:25.594 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.587+0000 7fc38fbf7740 1 -- 172.21.15.26:0/2872115611 wait complete. 2024-09-11T23:28:25.594 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.587+0000 7fc38fbf7740 20 client.16947 trim_cache size 0 max 16384 2024-09-11T23:28:25.603 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.602+0000 7fc38fbf7740 1 Processor -- start 2024-09-11T23:28:25.604 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.602+0000 7fc38fbf7740 1 -- start start 2024-09-11T23:28:25.604 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.602+0000 7fc38fbf7740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555cd19d1af0 0x555cd19d8410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:25.604 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.602+0000 7fc38fbf7740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555cd19dccb0 0x555cd19da410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:25.604 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.602+0000 7fc38fbf7740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555cd19f4730 0x555cd19f4b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:25.604 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.602+0000 7fc38fbf7740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc3881ad8a0 con 0x555cd19f4730 2024-09-11T23:28:25.604 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.602+0000 7fc38fbf7740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fc3881af560 con 0x555cd19dccb0 2024-09-11T23:28:25.604 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.602+0000 7fc38fbf7740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fc3881adf70 con 0x555cd19d1af0 2024-09-11T23:28:25.604 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.602+0000 7fc386d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555cd19f4730 0x555cd19f4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:25.604 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.602+0000 7fc386d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555cd19f4730 0x555cd19f4b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59256/0 (socket says 172.21.15.26:59256) 2024-09-11T23:28:25.604 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.602+0000 7fc386d9d640 1 -- 172.21.15.26:0/637716500 learned_addr learned my addr 172.21.15.26:0/637716500 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:28:25.604 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.602+0000 7fc385d9b640 1 --2- 172.21.15.26:0/637716500 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555cd19dccb0 0x555cd19da410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:25.604 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.602+0000 7fc38659c640 1 --2- 172.21.15.26:0/637716500 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555cd19d1af0 0x555cd19d8410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:25.604 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.602+0000 7fc386d9d640 1 -- 172.21.15.26:0/637716500 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555cd19d1af0 msgr2=0x555cd19d8410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:28:25.605 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.602+0000 7fc386d9d640 1 --2- 172.21.15.26:0/637716500 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555cd19d1af0 0x555cd19d8410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:25.605 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.602+0000 7fc386d9d640 1 -- 172.21.15.26:0/637716500 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555cd19dccb0 msgr2=0x555cd19da410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:28:25.605 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.602+0000 7fc386d9d640 1 --2- 172.21.15.26:0/637716500 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555cd19dccb0 0x555cd19da410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:25.605 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.602+0000 7fc386d9d640 1 -- 172.21.15.26:0/637716500 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555cd19cecf0 con 0x555cd19f4730 2024-09-11T23:28:25.605 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.603+0000 7fc386d9d640 1 --2- 172.21.15.26:0/637716500 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555cd19f4730 0x555cd19f4b70 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fc380009580 tx=0x7fc3800043a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f67f500a66e7249 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:28:25.605 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.603+0000 7fc36ffff640 1 -- 172.21.15.26:0/637716500 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc380051070 con 0x555cd19f4730 2024-09-11T23:28:25.605 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.603+0000 7fc36ffff640 1 -- 172.21.15.26:0/637716500 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc380036040 con 0x555cd19f4730 2024-09-11T23:28:25.605 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.603+0000 7fc36ffff640 1 -- 172.21.15.26:0/637716500 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc380003c20 con 0x555cd19f4730 2024-09-11T23:28:25.605 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.603+0000 7fc38fbf7740 1 -- 172.21.15.26:0/637716500 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555cd19f4730 msgr2=0x555cd19f4b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:25.605 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.603+0000 7fc38fbf7740 1 --2- 172.21.15.26:0/637716500 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555cd19f4730 0x555cd19f4b70 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fc380009580 tx=0x7fc3800043a0 comp rx=0 tx=0).stop 2024-09-11T23:28:25.605 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.603+0000 7fc38fbf7740 1 -- 172.21.15.26:0/637716500 shutdown_connections 2024-09-11T23:28:25.605 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.603+0000 7fc38fbf7740 1 --2- 172.21.15.26:0/637716500 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555cd19dccb0 0x555cd19da410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:25.605 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.603+0000 7fc38fbf7740 1 --2- 172.21.15.26:0/637716500 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555cd19f4730 0x555cd19f4b70 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:25.606 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.603+0000 7fc38fbf7740 1 --2- 172.21.15.26:0/637716500 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555cd19d1af0 0x555cd19d8410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:25.606 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.603+0000 7fc38fbf7740 1 -- 172.21.15.26:0/637716500 >> 172.21.15.26:0/637716500 conn(0x555cd19ce060 msgr2=0x555cd19dbbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:28:25.606 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.604+0000 7fc38fbf7740 1 -- 172.21.15.26:0/637716500 shutdown_connections 2024-09-11T23:28:25.606 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.604+0000 7fc38fbf7740 1 -- 172.21.15.26:0/637716500 wait complete. 2024-09-11T23:28:25.606 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.604+0000 7fc38fbf7740 1 -- start start 2024-09-11T23:28:25.606 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.604+0000 7fc38fbf7740 1 Processor -- start 2024-09-11T23:28:25.609 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.604+0000 7fc38fbf7740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555cd19d1af0 0x555cd19e0890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:25.609 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.604+0000 7fc38fbf7740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555cd19dccb0 0x555cd19e0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:25.609 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.604+0000 7fc38fbf7740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555cd19f4730 0x555cd19e1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:25.609 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.604+0000 7fc38fbf7740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc3881aee90 con 0x555cd19d1af0 2024-09-11T23:28:25.609 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.604+0000 7fc38fbf7740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fc3881ae700 con 0x555cd19f4730 2024-09-11T23:28:25.609 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.604+0000 7fc38fbf7740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fc3881ab8a0 con 0x555cd19dccb0 2024-09-11T23:28:25.609 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.604+0000 7fc38fbf7740 10 client.0 _ll_register_callbacks cb 0x7fc37000e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:28:25.609 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.604+0000 7fc38659c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555cd19d1af0 0x555cd19e0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:25.609 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.604+0000 7fc38659c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555cd19d1af0 0x555cd19e0890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59268/0 (socket says 172.21.15.26:59268) 2024-09-11T23:28:25.610 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.604+0000 7fc38659c640 1 -- 172.21.15.26:0/4062229770 learned_addr learned my addr 172.21.15.26:0/4062229770 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:28:25.610 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.604+0000 7fc385d9b640 1 --2- 172.21.15.26:0/4062229770 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555cd19dccb0 0x555cd19e0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:25.610 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.605+0000 7fc386d9d640 1 --2- 172.21.15.26:0/4062229770 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555cd19f4730 0x555cd19e1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:25.610 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.605+0000 7fc38659c640 1 -- 172.21.15.26:0/4062229770 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555cd19dccb0 msgr2=0x555cd19e0dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:28:25.610 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.605+0000 7fc38659c640 1 --2- 172.21.15.26:0/4062229770 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555cd19dccb0 0x555cd19e0dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:25.610 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.605+0000 7fc38659c640 1 -- 172.21.15.26:0/4062229770 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555cd19f4730 msgr2=0x555cd19e1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:28:25.610 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.605+0000 7fc38659c640 1 --2- 172.21.15.26:0/4062229770 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555cd19f4730 0x555cd19e1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:25.610 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.605+0000 7fc38659c640 1 -- 172.21.15.26:0/4062229770 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555cd1a56750 con 0x555cd19d1af0 2024-09-11T23:28:25.610 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.605+0000 7fc386d9d640 1 --2- 172.21.15.26:0/4062229770 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555cd19f4730 0x555cd19e1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:28:25.610 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.605+0000 7fc38659c640 1 --2- 172.21.15.26:0/4062229770 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555cd19d1af0 0x555cd19e0890 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fc378005fd0 tx=0x7fc3780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c918f099645777a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:28:25.610 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.605+0000 7fc36dffb640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:28:25.610 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.605+0000 7fc36dffb640 1 -- 172.21.15.26:0/4062229770 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc37802a410 con 0x555cd19d1af0 2024-09-11T23:28:25.610 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.605+0000 7fc36dffb640 1 -- 172.21.15.26:0/4062229770 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8586+0+0 (secure 0 0 0) 0x7fc37802a5b0 con 0x555cd19d1af0 2024-09-11T23:28:25.611 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.605+0000 7fc38fbf7740 1 -- 172.21.15.26:0/4062229770 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc380036040 con 0x555cd19d1af0 2024-09-11T23:28:25.611 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.605+0000 7fc36dffb640 1 -- 172.21.15.26:0/4062229770 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7fc37803a050 con 0x555cd19d1af0 2024-09-11T23:28:25.611 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.605+0000 7fc38fbf7740 10 client.16953 fetch_fsmap learned FSMap version 51 2024-09-11T23:28:25.611 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.605+0000 7fc38fbf7740 1 -- 172.21.15.26:0/4062229770 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x555cd1a57a80 con 0x555cd19d1af0 2024-09-11T23:28:25.611 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.605+0000 7fc36dffb640 1 -- 172.21.15.26:0/4062229770 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc380036040 con 0x555cd19d1af0 2024-09-11T23:28:25.611 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.605+0000 7fc36dffb640 1 -- 172.21.15.26:0/4062229770 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 51) ==== 8394+0+0 (secure 0 0 0) 0x7fc378010b70 con 0x555cd19d1af0 2024-09-11T23:28:25.611 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.606+0000 7fc38fbf7740 10 client.16953 fetch_fsmap finished waiting for FSMap version 51 2024-09-11T23:28:25.611 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.606+0000 7fc38fbf7740 10 client.16953 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:28:25.611 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.606+0000 7fc38fbf7740 20 client.16953 populate_metadata read hostname 'smithi026' 2024-09-11T23:28:25.611 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.606+0000 7fc38fbf7740 1 --2- 172.21.15.26:0/4062229770 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x555cd1a581d0 0x555cd1a5a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:25.611 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.606+0000 7fc38fbf7740 4 client.16953 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T23:28:25.611 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.606+0000 7fc38fbf7740 1 -- 172.21.15.26:0/4062229770 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555cd193dd70 con 0x555cd1a581d0 2024-09-11T23:28:25.611 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.606+0000 7fc385d9b640 1 --2- 172.21.15.26:0/4062229770 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x555cd1a581d0 0x555cd1a5a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:25.611 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.606+0000 7fc36dffb640 10 client.16953 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:28:25.611 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.606+0000 7fc385d9b640 1 --2- 172.21.15.26:0/4062229770 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x555cd1a581d0 0x555cd1a5a5f0 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=403f5e51ff968234 server_cookie=b7562006114e30d in_seq=0 out_seq=0 2024-09-11T23:28:25.624 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.624+0000 7fc36dffb640 1 -- 172.21.15.26:0/4062229770 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555cd193dd70 con 0x555cd1a581d0 2024-09-11T23:28:25.624 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.624+0000 7fc36dffb640 10 client.16953 handle_command_reply: tid=0 2024-09-11T23:28:25.624 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.624+0000 7fc38fbf7740 1 client.16953 shutdown 2024-09-11T23:28:25.624 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.624+0000 7fc38fbf7740 1 -- 172.21.15.26:0/4062229770 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555cd19d1af0 msgr2=0x555cd19e0890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:25.624 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.624+0000 7fc38fbf7740 1 --2- 172.21.15.26:0/4062229770 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555cd19d1af0 0x555cd19e0890 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fc378005fd0 tx=0x7fc3780046c0 comp rx=0 tx=0).stop 2024-09-11T23:28:25.624 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.624+0000 7fc38fbf7740 1 -- 172.21.15.26:0/4062229770 shutdown_connections 2024-09-11T23:28:25.624 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.624+0000 7fc38fbf7740 1 --2- 172.21.15.26:0/4062229770 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555cd19f4730 0x555cd19e1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:25.625 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.624+0000 7fc38fbf7740 1 --2- 172.21.15.26:0/4062229770 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x555cd1a581d0 0x555cd1a5a5f0 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:25.625 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.624+0000 7fc38fbf7740 1 --2- 172.21.15.26:0/4062229770 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555cd19dccb0 0x555cd19e0dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:25.625 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.624+0000 7fc38fbf7740 1 --2- 172.21.15.26:0/4062229770 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555cd19d1af0 0x555cd19e0890 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:25.625 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.624+0000 7fc36dffb640 0 client.16953 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:28:25.625 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.624+0000 7fc38fbf7740 1 -- 172.21.15.26:0/4062229770 >> 172.21.15.26:0/4062229770 conn(0x555cd19ce060 msgr2=0x555cd19f3f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:28:25.625 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.624+0000 7fc38fbf7740 1 -- 172.21.15.26:0/4062229770 shutdown_connections 2024-09-11T23:28:25.625 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.624+0000 7fc38fbf7740 1 -- 172.21.15.26:0/4062229770 wait complete. 2024-09-11T23:28:25.625 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.624+0000 7fc38fbf7740 20 client.16953 trim_cache size 0 max 16384 2024-09-11T23:28:25.625 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-09-11T23:28:25.625 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "message": "", 2024-09-11T23:28:25.626 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-09-11T23:28:25.626 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-09-11T23:28:25.626 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.625+0000 7fc38ce5e640 1 -- 172.21.15.26:0/2440789784 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fc3740a2da0 msgr2=0x7fc3740a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:25.626 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.625+0000 7fc38ce5e640 1 --2- 172.21.15.26:0/2440789784 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fc3740a2da0 0x7fc3740a5260 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x555cd19ceec0 tx=0x7fc37800b410 comp rx=0 tx=0).stop 2024-09-11T23:28:25.627 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.625+0000 7fc38ce5e640 1 -- 172.21.15.26:0/2440789784 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc3881020c0 msgr2=0x7fc388198900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:25.627 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.625+0000 7fc38ce5e640 1 --2- 172.21.15.26:0/2440789784 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc3881020c0 0x7fc388198900 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc37000e9f0 tx=0x7fc37000eeb0 comp rx=0 tx=0).stop 2024-09-11T23:28:25.627 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.625+0000 7fc38ce5e640 1 -- 172.21.15.26:0/2440789784 shutdown_connections 2024-09-11T23:28:25.627 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.625+0000 7fc38ce5e640 1 --2- 172.21.15.26:0/2440789784 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fc3740a2da0 0x7fc3740a5260 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:25.627 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.625+0000 7fc38ce5e640 1 --2- 172.21.15.26:0/2440789784 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc3881020c0 0x7fc388198900 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:25.627 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.625+0000 7fc38ce5e640 1 --2- 172.21.15.26:0/2440789784 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc388106b70 0x7fc38819cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:25.627 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.625+0000 7fc38ce5e640 1 --2- 172.21.15.26:0/2440789784 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc388101740 0x7fc3881983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:25.627 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.625+0000 7fc38ce5e640 1 -- 172.21.15.26:0/2440789784 >> 172.21.15.26:0/2440789784 conn(0x7fc388075ef0 msgr2=0x7fc388105260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:28:25.627 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.625+0000 7fc38ce5e640 1 -- 172.21.15.26:0/2440789784 shutdown_connections 2024-09-11T23:28:25.627 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:25.625+0000 7fc38ce5e640 1 -- 172.21.15.26:0/2440789784 wait complete. 2024-09-11T23:28:26.000 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:28:25 smithi026 ceph-mon[31172]: pgmap v311: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 51 KiB/s wr, 20 op/s 2024-09-11T23:28:26.000 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:28:25 smithi026 ceph-mon[31172]: osdmap e94: 12 total, 12 up, 12 in 2024-09-11T23:28:26.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:28:25 smithi076 ceph-mon[35296]: pgmap v311: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 51 KiB/s wr, 20 op/s 2024-09-11T23:28:26.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:28:25 smithi076 ceph-mon[35296]: osdmap e94: 12 total, 12 up, 12 in 2024-09-11T23:28:26.238 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:28:25 smithi136 ceph-mon[35121]: pgmap v311: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 51 KiB/s wr, 20 op/s 2024-09-11T23:28:26.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:28:25 smithi136 ceph-mon[35121]: osdmap e94: 12 total, 12 up, 12 in 2024-09-11T23:28:28.000 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:28:27 smithi026 ceph-mon[31172]: pgmap v313: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 21 KiB/s wr, 16 op/s 2024-09-11T23:28:28.052 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:28:27 smithi076 ceph-mon[35296]: pgmap v313: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 21 KiB/s wr, 16 op/s 2024-09-11T23:28:28.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:28:27 smithi136 ceph-mon[35121]: pgmap v313: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 21 KiB/s wr, 16 op/s 2024-09-11T23:28:30.000 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:28:29 smithi026 ceph-mon[31172]: pgmap v314: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 21 KiB/s wr, 17 op/s 2024-09-11T23:28:30.052 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:28:29 smithi076 ceph-mon[35296]: pgmap v314: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 21 KiB/s wr, 17 op/s 2024-09-11T23:28:30.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:28:29 smithi136 ceph-mon[35121]: pgmap v314: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 21 KiB/s wr, 17 op/s 2024-09-11T23:28:32.052 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:28:31 smithi076 ceph-mon[35296]: pgmap v315: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 51 KiB/s wr, 18 op/s 2024-09-11T23:28:32.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:28:31 smithi136 ceph-mon[35121]: pgmap v315: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 51 KiB/s wr, 18 op/s 2024-09-11T23:28:32.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:28:31 smithi026 ceph-mon[31172]: pgmap v315: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 51 KiB/s wr, 18 op/s 2024-09-11T23:28:34.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:28:33 smithi076 ceph-mon[35296]: pgmap v316: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 51 KiB/s wr, 18 op/s 2024-09-11T23:28:34.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:28:33 smithi076 ceph-mon[35296]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:28:34.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:28:33 smithi136 ceph-mon[35121]: pgmap v316: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 51 KiB/s wr, 18 op/s 2024-09-11T23:28:34.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:28:33 smithi136 ceph-mon[35121]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:28:34.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:28:33 smithi026 ceph-mon[31172]: pgmap v316: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 51 KiB/s wr, 18 op/s 2024-09-11T23:28:34.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:28:33 smithi026 ceph-mon[31172]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:28:36.052 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:28:35 smithi076 ceph-mon[35296]: pgmap v317: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 49 KiB/s wr, 16 op/s 2024-09-11T23:28:36.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:28:35 smithi136 ceph-mon[35121]: pgmap v317: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 49 KiB/s wr, 16 op/s 2024-09-11T23:28:36.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:28:35 smithi026 ceph-mon[31172]: pgmap v317: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 49 KiB/s wr, 16 op/s 2024-09-11T23:28:38.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:28:37 smithi076 ceph-mon[35296]: pgmap v318: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 46 KiB/s wr, 15 op/s 2024-09-11T23:28:38.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:28:37 smithi136 ceph-mon[35121]: pgmap v318: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 46 KiB/s wr, 15 op/s 2024-09-11T23:28:38.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:28:37 smithi026 ceph-mon[31172]: pgmap v318: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 46 KiB/s wr, 15 op/s 2024-09-11T23:28:38.725 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.725+0000 7f8d8b7d6640 1 -- 172.21.15.26:0/3849948117 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8d84102480 msgr2=0x7f8d8410e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:38.725 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.725+0000 7f8d8b7d6640 1 --2- 172.21.15.26:0/3849948117 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8d84102480 0x7f8d8410e9b0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f8d7c009cd0 tx=0x7f8d7c02a8e0 comp rx=0 tx=0).stop 2024-09-11T23:28:38.725 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.725+0000 7f8d8b7d6640 1 -- 172.21.15.26:0/3849948117 shutdown_connections 2024-09-11T23:28:38.725 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.725+0000 7f8d8b7d6640 1 --2- 172.21.15.26:0/3849948117 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8d84101ae0 0x7f8d84101f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:38.725 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.725+0000 7f8d8b7d6640 1 --2- 172.21.15.26:0/3849948117 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8d84102480 0x7f8d8410e9b0 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:38.726 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.725+0000 7f8d8b7d6640 1 --2- 172.21.15.26:0/3849948117 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8d84107b80 0x7f8d84107f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:38.726 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.725+0000 7f8d8b7d6640 1 -- 172.21.15.26:0/3849948117 >> 172.21.15.26:0/3849948117 conn(0x7f8d840fd7e0 msgr2=0x7f8d840ffc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:28:38.726 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.725+0000 7f8d8b7d6640 1 -- 172.21.15.26:0/3849948117 shutdown_connections 2024-09-11T23:28:38.726 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.725+0000 7f8d8b7d6640 1 -- 172.21.15.26:0/3849948117 wait complete. 2024-09-11T23:28:38.726 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.725+0000 7f8d8b7d6640 1 Processor -- start 2024-09-11T23:28:38.726 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.726+0000 7f8d8b7d6640 1 -- start start 2024-09-11T23:28:38.728 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.726+0000 7f8d8b7d6640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8d84101ae0 0x7f8d8419a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:38.728 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.726+0000 7f8d8b7d6640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8d84102480 0x7f8d8419aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:38.728 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.726+0000 7f8d8b7d6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8d84107b80 0x7f8d8419edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:38.728 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.726+0000 7f8d8b7d6640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f8d84111ef0 con 0x7f8d84107b80 2024-09-11T23:28:38.728 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.726+0000 7f8d8b7d6640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f8d84111d70 con 0x7f8d84102480 2024-09-11T23:28:38.728 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.726+0000 7f8d8b7d6640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f8d84112070 con 0x7f8d84101ae0 2024-09-11T23:28:38.728 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.726+0000 7f8d89d73640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8d84107b80 0x7f8d8419edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:38.728 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.726+0000 7f8d89d73640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8d84107b80 0x7f8d8419edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40060/0 (socket says 172.21.15.26:40060) 2024-09-11T23:28:38.728 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.726+0000 7f8d89d73640 1 -- 172.21.15.26:0/242945941 learned_addr learned my addr 172.21.15.26:0/242945941 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:28:38.728 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.726+0000 7f8d88d71640 1 --2- 172.21.15.26:0/242945941 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8d84102480 0x7f8d8419aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:38.729 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.726+0000 7f8d89572640 1 --2- 172.21.15.26:0/242945941 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8d84101ae0 0x7f8d8419a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:38.729 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.726+0000 7f8d89d73640 1 -- 172.21.15.26:0/242945941 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8d84101ae0 msgr2=0x7f8d8419a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:38.729 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.726+0000 7f8d89d73640 1 --2- 172.21.15.26:0/242945941 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8d84101ae0 0x7f8d8419a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:38.729 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.726+0000 7f8d89d73640 1 -- 172.21.15.26:0/242945941 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8d84102480 msgr2=0x7f8d8419aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:38.729 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.726+0000 7f8d89d73640 1 --2- 172.21.15.26:0/242945941 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8d84102480 0x7f8d8419aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:38.729 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.726+0000 7f8d89d73640 1 -- 172.21.15.26:0/242945941 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d8419f520 con 0x7f8d84107b80 2024-09-11T23:28:38.729 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.727+0000 7f8d89d73640 1 --2- 172.21.15.26:0/242945941 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8d84107b80 0x7f8d8419edf0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f8d7c005cc0 tx=0x7f8d7c003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:28:38.729 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.727+0000 7f8d7a7fc640 1 -- 172.21.15.26:0/242945941 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d7c004040 con 0x7f8d84107b80 2024-09-11T23:28:38.729 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.727+0000 7f8d7a7fc640 1 -- 172.21.15.26:0/242945941 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d7c0046c0 con 0x7f8d84107b80 2024-09-11T23:28:38.729 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.727+0000 7f8d7a7fc640 1 -- 172.21.15.26:0/242945941 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d7c046480 con 0x7f8d84107b80 2024-09-11T23:28:38.729 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.727+0000 7f8d8b7d6640 1 -- 172.21.15.26:0/242945941 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d8419f7b0 con 0x7f8d84107b80 2024-09-11T23:28:38.729 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.727+0000 7f8d8b7d6640 1 -- 172.21.15.26:0/242945941 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d841a70a0 con 0x7f8d84107b80 2024-09-11T23:28:38.732 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.730+0000 7f8d7a7fc640 1 -- 172.21.15.26:0/242945941 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f8d7c0305a0 con 0x7f8d84107b80 2024-09-11T23:28:38.732 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.730+0000 7f8d8e64f740 1 Processor -- start 2024-09-11T23:28:38.732 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.730+0000 7f8d8e64f740 1 -- start start 2024-09-11T23:28:38.732 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.730+0000 7f8d8e64f740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a47b59baf0 0x55a47b59bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:38.733 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.730+0000 7f8d8e64f740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a47b59c410 0x55a47b5a6770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:38.733 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.730+0000 7f8d8e64f740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a47b5a6cb0 0x55a47b5a90a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:38.733 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.730+0000 7f8d8e64f740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55a47b5002f0 con 0x55a47b5a6cb0 2024-09-11T23:28:38.733 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.730+0000 7f8d8e64f740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55a47b4a8270 con 0x55a47b59baf0 2024-09-11T23:28:38.733 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.730+0000 7f8d8e64f740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55a47b4fc2b0 con 0x55a47b59c410 2024-09-11T23:28:38.733 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.731+0000 7f8d89572640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a47b59c410 0x55a47b5a6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:38.733 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.731+0000 7f8d88d71640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a47b59baf0 0x55a47b59bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:38.733 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.731+0000 7f8d88d71640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a47b5a6cb0 0x55a47b5a90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:38.733 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.731+0000 7f8d89572640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a47b59c410 0x55a47b5a6770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:35844/0 (socket says 172.21.15.26:35844) 2024-09-11T23:28:38.733 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.731+0000 7f8d89572640 1 -- 172.21.15.26:0/2894339956 learned_addr learned my addr 172.21.15.26:0/2894339956 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:28:38.733 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.731+0000 7f8d7a7fc640 1 --2- 172.21.15.26:0/242945941 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f8d700a2e20 0x7f8d700a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:38.733 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.731+0000 7f8d89572640 1 -- 172.21.15.26:0/2894339956 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a47b59baf0 msgr2=0x55a47b59bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:28:38.734 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.731+0000 7f8d89572640 1 --2- 172.21.15.26:0/2894339956 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a47b59baf0 0x55a47b59bed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:38.734 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.731+0000 7f8d89572640 1 -- 172.21.15.26:0/2894339956 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a47b5a6cb0 msgr2=0x55a47b5a90a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:28:38.734 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.731+0000 7f8d89572640 1 --2- 172.21.15.26:0/2894339956 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a47b5a6cb0 0x55a47b5a90a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:38.734 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.731+0000 7f8d89572640 1 -- 172.21.15.26:0/2894339956 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a47b4a7340 con 0x55a47b59c410 2024-09-11T23:28:38.734 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.731+0000 7f8d89d73640 1 --2- 172.21.15.26:0/242945941 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f8d700a2e20 0x7f8d700a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:38.734 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.731+0000 7f8d7a7fc640 1 -- 172.21.15.26:0/242945941 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8542+0+0 (secure 0 0 0) 0x7f8d7c02e030 con 0x7f8d84107b80 2024-09-11T23:28:38.734 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.731+0000 7f8d89572640 1 --2- 172.21.15.26:0/2894339956 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a47b59c410 0x55a47b5a6770 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f8d8000aba0 tx=0x7f8d8000f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1b22a9b994df9b41 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:28:38.734 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.731+0000 7f8d6f7fe640 1 -- 172.21.15.26:0/2894339956 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d8001d070 con 0x55a47b59c410 2024-09-11T23:28:38.734 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.731+0000 7f8d6f7fe640 1 -- 172.21.15.26:0/2894339956 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d80018b20 con 0x55a47b59c410 2024-09-11T23:28:38.734 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.731+0000 7f8d89d73640 1 --2- 172.21.15.26:0/242945941 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f8d700a2e20 0x7f8d700a52e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f8d7c0ee780 tx=0x7f8d7c0dd040 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:28:38.734 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.732+0000 7f8d6f7fe640 1 -- 172.21.15.26:0/2894339956 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d80010b30 con 0x55a47b59c410 2024-09-11T23:28:38.734 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.732+0000 7f8d8e64f740 1 -- 172.21.15.26:0/2894339956 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a47b59c410 msgr2=0x55a47b5a6770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:38.734 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.732+0000 7f8d8e64f740 1 --2- 172.21.15.26:0/2894339956 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a47b59c410 0x55a47b5a6770 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f8d8000aba0 tx=0x7f8d8000f790 comp rx=0 tx=0).stop 2024-09-11T23:28:38.734 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.732+0000 7f8d8e64f740 1 -- 172.21.15.26:0/2894339956 shutdown_connections 2024-09-11T23:28:38.735 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.732+0000 7f8d8e64f740 1 --2- 172.21.15.26:0/2894339956 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a47b5a6cb0 0x55a47b5a90a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:38.735 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.732+0000 7f8d8e64f740 1 --2- 172.21.15.26:0/2894339956 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a47b59c410 0x55a47b5a6770 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:38.735 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.732+0000 7f8d8e64f740 1 --2- 172.21.15.26:0/2894339956 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a47b59baf0 0x55a47b59bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:38.735 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.732+0000 7f8d8e64f740 1 -- 172.21.15.26:0/2894339956 >> 172.21.15.26:0/2894339956 conn(0x55a47b598060 msgr2=0x55a47b598440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:28:38.735 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.732+0000 7f8d8e64f740 1 -- 172.21.15.26:0/2894339956 shutdown_connections 2024-09-11T23:28:38.735 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.732+0000 7f8d8e64f740 1 -- 172.21.15.26:0/2894339956 wait complete. 2024-09-11T23:28:38.735 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.733+0000 7f8d8e64f740 1 -- start start 2024-09-11T23:28:38.735 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.733+0000 7f8d8e64f740 1 Processor -- start 2024-09-11T23:28:38.739 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.733+0000 7f8d8e64f740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a47b59baf0 0x55a47b5a3b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:38.739 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.733+0000 7f8d8e64f740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a47b59c410 0x55a47b5a40d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:38.739 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.733+0000 7f8d8e64f740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a47b5a6cb0 0x55a47b5a4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:38.739 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.733+0000 7f8d8e64f740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55a47b5a9880 con 0x55a47b59c410 2024-09-11T23:28:38.739 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.733+0000 7f8d8e64f740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55a47b489290 con 0x55a47b5a6cb0 2024-09-11T23:28:38.739 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.733+0000 7f8d8e64f740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55a47b5a9a00 con 0x55a47b59baf0 2024-09-11T23:28:38.740 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.733+0000 7f8d8e64f740 10 client.0 _ll_register_callbacks cb 0x55a47b4ba140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:28:38.740 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.733+0000 7f8d89572640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a47b59c410 0x55a47b5a40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:38.740 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.733+0000 7f8d89572640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a47b59c410 0x55a47b5a40d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40080/0 (socket says 172.21.15.26:40080) 2024-09-11T23:28:38.740 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.733+0000 7f8d89572640 1 -- 172.21.15.26:0/2777074306 learned_addr learned my addr 172.21.15.26:0/2777074306 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:28:38.740 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.733+0000 7f8d88d71640 1 --2- 172.21.15.26:0/2777074306 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a47b59baf0 0x55a47b5a3b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:38.740 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.734+0000 7f8d88d71640 1 --2- 172.21.15.26:0/2777074306 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a47b5a6cb0 0x55a47b5a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:38.740 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.734+0000 7f8d89572640 1 -- 172.21.15.26:0/2777074306 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a47b59baf0 msgr2=0x55a47b5a3b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:28:38.740 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.734+0000 7f8d89572640 1 --2- 172.21.15.26:0/2777074306 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a47b59baf0 0x55a47b5a3b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:38.740 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.734+0000 7f8d89572640 1 -- 172.21.15.26:0/2777074306 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a47b5a6cb0 msgr2=0x55a47b5a4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:28:38.740 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.734+0000 7f8d89572640 1 --2- 172.21.15.26:0/2777074306 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a47b5a6cb0 0x55a47b5a4610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:38.740 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.734+0000 7f8d89572640 1 -- 172.21.15.26:0/2777074306 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a47b213210 con 0x55a47b59c410 2024-09-11T23:28:38.740 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.734+0000 7f8d88d71640 1 --2- 172.21.15.26:0/2777074306 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a47b5a6cb0 0x55a47b5a4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:28:38.740 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.734+0000 7f8d89572640 1 --2- 172.21.15.26:0/2777074306 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a47b59c410 0x55a47b5a40d0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f8d80000c00 tx=0x7f8d8000a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=719c4db2d2f2cb86 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:28:38.741 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.734+0000 7f8d6cff9640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:28:38.741 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.734+0000 7f8d6cff9640 1 -- 172.21.15.26:0/2777074306 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d8001d050 con 0x55a47b59c410 2024-09-11T23:28:38.741 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.734+0000 7f8d8e64f740 1 -- 172.21.15.26:0/2777074306 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d80018b20 con 0x55a47b59c410 2024-09-11T23:28:38.741 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.734+0000 7f8d6cff9640 1 -- 172.21.15.26:0/2777074306 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8542+0+0 (secure 0 0 0) 0x7f8d80004020 con 0x55a47b59c410 2024-09-11T23:28:38.741 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.734+0000 7f8d6cff9640 1 -- 172.21.15.26:0/2777074306 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f8d80031050 con 0x55a47b59c410 2024-09-11T23:28:38.741 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.734+0000 7f8d8e64f740 10 client.16965 fetch_fsmap learned FSMap version 52 2024-09-11T23:28:38.741 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.734+0000 7f8d8e64f740 1 -- 172.21.15.26:0/2777074306 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x55a47b2df620 con 0x55a47b59c410 2024-09-11T23:28:38.741 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.735+0000 7f8d6cff9640 1 -- 172.21.15.26:0/2777074306 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d80018b20 con 0x55a47b59c410 2024-09-11T23:28:38.741 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.735+0000 7f8d6cff9640 1 -- 172.21.15.26:0/2777074306 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 52) ==== 8390+0+0 (secure 0 0 0) 0x7f8d80005040 con 0x55a47b59c410 2024-09-11T23:28:38.741 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.735+0000 7f8d8e64f740 10 client.16965 fetch_fsmap finished waiting for FSMap version 52 2024-09-11T23:28:38.741 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.735+0000 7f8d8e64f740 10 client.16965 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:28:38.741 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.735+0000 7f8d8e64f740 20 client.16965 populate_metadata read hostname 'smithi026' 2024-09-11T23:28:38.741 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.735+0000 7f8d8e64f740 1 --2- 172.21.15.26:0/2777074306 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55a47b5b6bd0 0x55a47b5b8fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:38.742 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.735+0000 7f8d8e64f740 4 client.16965 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:28:38.742 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.735+0000 7f8d8e64f740 1 -- 172.21.15.26:0/2777074306 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a47b50da20 con 0x55a47b5b6bd0 2024-09-11T23:28:38.742 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.735+0000 7f8d88d71640 1 --2- 172.21.15.26:0/2777074306 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55a47b5b6bd0 0x55a47b5b8fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:38.742 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.736+0000 7f8d88d71640 1 --2- 172.21.15.26:0/2777074306 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55a47b5b6bd0 0x55a47b5b8fc0 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5cbda342595d9e73 server_cookie=8bb839a0a8c024c1 in_seq=0 out_seq=0 2024-09-11T23:28:38.742 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.736+0000 7f8d6cff9640 10 client.16965 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:28:38.742 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.736+0000 7f8d6cff9640 1 -- 172.21.15.26:0/2777074306 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a47b50da20 con 0x55a47b5b6bd0 2024-09-11T23:28:38.742 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.736+0000 7f8d6cff9640 10 client.16965 handle_command_reply: tid=0 2024-09-11T23:28:38.742 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.736+0000 7f8d8e64f740 1 client.16965 shutdown 2024-09-11T23:28:38.742 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.736+0000 7f8d8e64f740 1 -- 172.21.15.26:0/2777074306 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a47b59c410 msgr2=0x55a47b5a40d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:38.742 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.736+0000 7f8d8e64f740 1 --2- 172.21.15.26:0/2777074306 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a47b59c410 0x55a47b5a40d0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f8d80000c00 tx=0x7f8d8000a710 comp rx=0 tx=0).stop 2024-09-11T23:28:38.742 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.736+0000 7f8d8e64f740 1 -- 172.21.15.26:0/2777074306 shutdown_connections 2024-09-11T23:28:38.742 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.736+0000 7f8d8e64f740 1 --2- 172.21.15.26:0/2777074306 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a47b5a6cb0 0x55a47b5a4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:38.742 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.736+0000 7f8d8e64f740 1 --2- 172.21.15.26:0/2777074306 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55a47b5b6bd0 0x55a47b5b8fc0 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:38.743 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.736+0000 7f8d8e64f740 1 --2- 172.21.15.26:0/2777074306 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a47b59c410 0x55a47b5a40d0 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:38.743 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.737+0000 7f8d8e64f740 1 --2- 172.21.15.26:0/2777074306 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a47b59baf0 0x55a47b5a3b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:38.743 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.737+0000 7f8d6cff9640 0 client.16965 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:28:38.743 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.737+0000 7f8d8e64f740 1 -- 172.21.15.26:0/2777074306 >> 172.21.15.26:0/2777074306 conn(0x55a47b598060 msgr2=0x55a47b59dd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:28:38.743 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.737+0000 7f8d8e64f740 1 -- 172.21.15.26:0/2777074306 shutdown_connections 2024-09-11T23:28:38.743 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.737+0000 7f8d8e64f740 1 -- 172.21.15.26:0/2777074306 wait complete. 2024-09-11T23:28:38.743 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.737+0000 7f8d8e64f740 20 client.16965 trim_cache size 0 max 16384 2024-09-11T23:28:38.752 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.751+0000 7f8d8e64f740 1 Processor -- start 2024-09-11T23:28:38.752 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.751+0000 7f8d8e64f740 1 -- start start 2024-09-11T23:28:38.752 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.752+0000 7f8d8e64f740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a47b59baf0 0x55a47b5a4440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:38.752 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.752+0000 7f8d8e64f740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a47b59c410 0x55a47b5be5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:38.753 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.752+0000 7f8d8e64f740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a47b5a6cb0 0x55a47b5beb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:38.753 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.752+0000 7f8d8e64f740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f8d841afa20 con 0x55a47b59baf0 2024-09-11T23:28:38.753 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.752+0000 7f8d8e64f740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f8d841b16b0 con 0x55a47b59c410 2024-09-11T23:28:38.753 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.752+0000 7f8d8e64f740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f8d841b00f0 con 0x55a47b5a6cb0 2024-09-11T23:28:38.753 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.752+0000 7f8d88d71640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a47b59baf0 0x55a47b5a4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:38.753 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.752+0000 7f8d88d71640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a47b59baf0 0x55a47b5a4440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40088/0 (socket says 172.21.15.26:40088) 2024-09-11T23:28:38.753 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.752+0000 7f8d88d71640 1 -- 172.21.15.26:0/1509101674 learned_addr learned my addr 172.21.15.26:0/1509101674 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:28:38.753 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.752+0000 7f8d88d71640 1 --2- 172.21.15.26:0/1509101674 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a47b5a6cb0 0x55a47b5beb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:38.753 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.752+0000 7f8d89572640 1 --2- 172.21.15.26:0/1509101674 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a47b59c410 0x55a47b5be5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:38.753 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.752+0000 7f8d88d71640 1 -- 172.21.15.26:0/1509101674 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a47b5a6cb0 msgr2=0x55a47b5beb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:28:38.753 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.752+0000 7f8d88d71640 1 --2- 172.21.15.26:0/1509101674 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a47b5a6cb0 0x55a47b5beb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:38.753 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.752+0000 7f8d88d71640 1 -- 172.21.15.26:0/1509101674 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a47b59c410 msgr2=0x55a47b5be5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:28:38.753 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.752+0000 7f8d88d71640 1 --2- 172.21.15.26:0/1509101674 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a47b59c410 0x55a47b5be5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:38.754 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.752+0000 7f8d88d71640 1 -- 172.21.15.26:0/1509101674 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a47b5a9410 con 0x55a47b59baf0 2024-09-11T23:28:38.754 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.752+0000 7f8d89572640 1 --2- 172.21.15.26:0/1509101674 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a47b59c410 0x55a47b5be5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:28:38.754 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.752+0000 7f8d88d71640 1 --2- 172.21.15.26:0/1509101674 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a47b59baf0 0x55a47b5a4440 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f8d8410ec10 tx=0x7f8d74004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=171955a1a425002d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:28:38.754 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.752+0000 7f8d6effd640 1 -- 172.21.15.26:0/1509101674 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d74004020 con 0x55a47b59baf0 2024-09-11T23:28:38.754 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.752+0000 7f8d6effd640 1 -- 172.21.15.26:0/1509101674 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d74012e00 con 0x55a47b59baf0 2024-09-11T23:28:38.754 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.752+0000 7f8d6effd640 1 -- 172.21.15.26:0/1509101674 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d740053e0 con 0x55a47b59baf0 2024-09-11T23:28:38.757 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.753+0000 7f8d8e64f740 1 -- 172.21.15.26:0/1509101674 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a47b59baf0 msgr2=0x55a47b5a4440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:38.757 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.753+0000 7f8d8e64f740 1 --2- 172.21.15.26:0/1509101674 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a47b59baf0 0x55a47b5a4440 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f8d8410ec10 tx=0x7f8d74004730 comp rx=0 tx=0).stop 2024-09-11T23:28:38.757 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.753+0000 7f8d8e64f740 1 -- 172.21.15.26:0/1509101674 shutdown_connections 2024-09-11T23:28:38.758 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.753+0000 7f8d8e64f740 1 --2- 172.21.15.26:0/1509101674 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a47b59c410 0x55a47b5be5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:38.758 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.753+0000 7f8d8e64f740 1 --2- 172.21.15.26:0/1509101674 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a47b5a6cb0 0x55a47b5beb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:38.758 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.753+0000 7f8d8e64f740 1 --2- 172.21.15.26:0/1509101674 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a47b59baf0 0x55a47b5a4440 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:38.758 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.753+0000 7f8d8e64f740 1 -- 172.21.15.26:0/1509101674 >> 172.21.15.26:0/1509101674 conn(0x55a47b598060 msgr2=0x55a47b5a5c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:28:38.758 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.753+0000 7f8d8e64f740 1 -- 172.21.15.26:0/1509101674 shutdown_connections 2024-09-11T23:28:38.758 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.753+0000 7f8d8e64f740 1 -- 172.21.15.26:0/1509101674 wait complete. 2024-09-11T23:28:38.758 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.753+0000 7f8d8e64f740 1 -- start start 2024-09-11T23:28:38.758 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.754+0000 7f8d8e64f740 1 Processor -- start 2024-09-11T23:28:38.758 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.754+0000 7f8d8e64f740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a47b59baf0 0x55a47b5aabc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:38.758 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.754+0000 7f8d8e64f740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a47b59c410 0x55a47b5ab100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:38.758 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.754+0000 7f8d8e64f740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a47b5a6cb0 0x55a47b5ab640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:38.758 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.754+0000 7f8d8e64f740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f8d841b0fe0 con 0x55a47b5a6cb0 2024-09-11T23:28:38.759 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.754+0000 7f8d8e64f740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f8d841b0880 con 0x55a47b59baf0 2024-09-11T23:28:38.759 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.754+0000 7f8d8e64f740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f8d841ada20 con 0x55a47b59c410 2024-09-11T23:28:38.759 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.754+0000 7f8d8e64f740 10 client.0 _ll_register_callbacks cb 0x7f8d8001b070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:28:38.759 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.754+0000 7f8d88d71640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a47b5a6cb0 0x55a47b5ab640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:38.759 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.754+0000 7f8d88d71640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a47b5a6cb0 0x55a47b5ab640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40090/0 (socket says 172.21.15.26:40090) 2024-09-11T23:28:38.759 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.754+0000 7f8d88d71640 1 -- 172.21.15.26:0/684181296 learned_addr learned my addr 172.21.15.26:0/684181296 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:28:38.759 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.754+0000 7f8d89572640 1 --2- 172.21.15.26:0/684181296 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a47b59c410 0x55a47b5ab100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:38.759 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.754+0000 7f8d88d71640 1 --2- 172.21.15.26:0/684181296 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a47b59baf0 0x55a47b5aabc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:38.759 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.754+0000 7f8d88d71640 1 -- 172.21.15.26:0/684181296 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a47b59c410 msgr2=0x55a47b5ab100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:28:38.759 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.754+0000 7f8d88d71640 1 --2- 172.21.15.26:0/684181296 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a47b59c410 0x55a47b5ab100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:38.759 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.754+0000 7f8d88d71640 1 -- 172.21.15.26:0/684181296 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a47b59baf0 msgr2=0x55a47b5aabc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:28:38.759 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.754+0000 7f8d88d71640 1 --2- 172.21.15.26:0/684181296 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a47b59baf0 0x55a47b5aabc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:38.759 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.754+0000 7f8d88d71640 1 -- 172.21.15.26:0/684181296 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a47b620790 con 0x55a47b5a6cb0 2024-09-11T23:28:38.760 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.755+0000 7f8d88d71640 1 --2- 172.21.15.26:0/684181296 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a47b5a6cb0 0x55a47b5ab640 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f8d740052c0 tx=0x7f8d74025040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41361b2609899479 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:28:38.760 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.755+0000 7f8d6cff9640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:28:38.760 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.755+0000 7f8d6cff9640 1 -- 172.21.15.26:0/684181296 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d740055e0 con 0x55a47b5a6cb0 2024-09-11T23:28:38.760 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.755+0000 7f8d6cff9640 1 -- 172.21.15.26:0/684181296 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8542+0+0 (secure 0 0 0) 0x7f8d74035040 con 0x55a47b5a6cb0 2024-09-11T23:28:38.760 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.755+0000 7f8d8e64f740 1 -- 172.21.15.26:0/684181296 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d74012e00 con 0x55a47b5a6cb0 2024-09-11T23:28:38.760 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.755+0000 7f8d6cff9640 1 -- 172.21.15.26:0/684181296 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f8d74012920 con 0x55a47b5a6cb0 2024-09-11T23:28:38.760 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.755+0000 7f8d8e64f740 10 client.16971 fetch_fsmap learned FSMap version 52 2024-09-11T23:28:38.760 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.755+0000 7f8d8e64f740 1 -- 172.21.15.26:0/684181296 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x55a47b621a20 con 0x55a47b5a6cb0 2024-09-11T23:28:38.760 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.755+0000 7f8d6cff9640 1 -- 172.21.15.26:0/684181296 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d74012e00 con 0x55a47b5a6cb0 2024-09-11T23:28:38.760 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.755+0000 7f8d6cff9640 1 -- 172.21.15.26:0/684181296 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 52) ==== 8390+0+0 (secure 0 0 0) 0x7f8d80019ce0 con 0x55a47b5a6cb0 2024-09-11T23:28:38.760 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.755+0000 7f8d8e64f740 10 client.16971 fetch_fsmap finished waiting for FSMap version 52 2024-09-11T23:28:38.760 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.755+0000 7f8d8e64f740 10 client.16971 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:28:38.761 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.755+0000 7f8d8e64f740 20 client.16971 populate_metadata read hostname 'smithi026' 2024-09-11T23:28:38.761 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.755+0000 7f8d8e64f740 1 --2- 172.21.15.26:0/684181296 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55a47b6220e0 0x55a47b624500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:38.761 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.755+0000 7f8d8e64f740 4 client.16971 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T23:28:38.761 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.755+0000 7f8d8e64f740 1 -- 172.21.15.26:0/684181296 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a47b507d70 con 0x55a47b6220e0 2024-09-11T23:28:38.761 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.756+0000 7f8d89572640 1 --2- 172.21.15.26:0/684181296 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55a47b6220e0 0x55a47b624500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:38.761 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.756+0000 7f8d89572640 1 --2- 172.21.15.26:0/684181296 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55a47b6220e0 0x55a47b624500 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=850934492a6ccc1b server_cookie=13b1b54e2125a866 in_seq=0 out_seq=0 2024-09-11T23:28:38.761 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.756+0000 7f8d6cff9640 10 client.16971 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:28:38.773 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.773+0000 7f8d6cff9640 1 -- 172.21.15.26:0/684181296 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a47b507d70 con 0x55a47b6220e0 2024-09-11T23:28:38.774 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.773+0000 7f8d6cff9640 10 client.16971 handle_command_reply: tid=0 2024-09-11T23:28:38.774 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.773+0000 7f8d8e64f740 1 client.16971 shutdown 2024-09-11T23:28:38.774 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.773+0000 7f8d8e64f740 1 -- 172.21.15.26:0/684181296 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a47b5a6cb0 msgr2=0x55a47b5ab640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:38.774 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.773+0000 7f8d8e64f740 1 --2- 172.21.15.26:0/684181296 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a47b5a6cb0 0x55a47b5ab640 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f8d740052c0 tx=0x7f8d74025040 comp rx=0 tx=0).stop 2024-09-11T23:28:38.774 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.773+0000 7f8d8e64f740 1 -- 172.21.15.26:0/684181296 shutdown_connections 2024-09-11T23:28:38.774 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.773+0000 7f8d8e64f740 1 --2- 172.21.15.26:0/684181296 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55a47b6220e0 0x55a47b624500 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:38.774 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.773+0000 7f8d8e64f740 1 --2- 172.21.15.26:0/684181296 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a47b5a6cb0 0x55a47b5ab640 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:38.774 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.773+0000 7f8d8e64f740 1 --2- 172.21.15.26:0/684181296 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a47b59c410 0x55a47b5ab100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:38.774 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.773+0000 7f8d8e64f740 1 --2- 172.21.15.26:0/684181296 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a47b59baf0 0x55a47b5aabc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:38.774 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.773+0000 7f8d6cff9640 0 client.16971 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:28:38.774 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.773+0000 7f8d8e64f740 1 -- 172.21.15.26:0/684181296 >> 172.21.15.26:0/684181296 conn(0x55a47b598060 msgr2=0x55a47b5969a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:28:38.774 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.774+0000 7f8d8e64f740 1 -- 172.21.15.26:0/684181296 shutdown_connections 2024-09-11T23:28:38.774 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.774+0000 7f8d8e64f740 1 -- 172.21.15.26:0/684181296 wait complete. 2024-09-11T23:28:38.775 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.774+0000 7f8d8e64f740 20 client.16971 trim_cache size 0 max 16384 2024-09-11T23:28:38.775 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-09-11T23:28:38.775 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "message": "", 2024-09-11T23:28:38.775 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-09-11T23:28:38.775 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-09-11T23:28:38.776 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.774+0000 7f8d8b7d6640 1 -- 172.21.15.26:0/242945941 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f8d700a2e20 msgr2=0x7f8d700a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:38.776 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.774+0000 7f8d8b7d6640 1 --2- 172.21.15.26:0/242945941 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f8d700a2e20 0x7f8d700a52e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f8d7c0ee780 tx=0x7f8d7c0dd040 comp rx=0 tx=0).stop 2024-09-11T23:28:38.776 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.774+0000 7f8d8b7d6640 1 -- 172.21.15.26:0/242945941 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8d84107b80 msgr2=0x7f8d8419edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:38.776 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.774+0000 7f8d8b7d6640 1 --2- 172.21.15.26:0/242945941 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8d84107b80 0x7f8d8419edf0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f8d7c005cc0 tx=0x7f8d7c003970 comp rx=0 tx=0).stop 2024-09-11T23:28:38.776 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.774+0000 7f8d8b7d6640 1 -- 172.21.15.26:0/242945941 shutdown_connections 2024-09-11T23:28:38.776 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.774+0000 7f8d8b7d6640 1 --2- 172.21.15.26:0/242945941 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f8d700a2e20 0x7f8d700a52e0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:38.776 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.774+0000 7f8d8b7d6640 1 --2- 172.21.15.26:0/242945941 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8d84102480 0x7f8d8419aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:38.776 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.774+0000 7f8d8b7d6640 1 --2- 172.21.15.26:0/242945941 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8d84107b80 0x7f8d8419edf0 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:38.776 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.774+0000 7f8d8b7d6640 1 --2- 172.21.15.26:0/242945941 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8d84101ae0 0x7f8d8419a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:38.777 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.774+0000 7f8d8b7d6640 1 -- 172.21.15.26:0/242945941 >> 172.21.15.26:0/242945941 conn(0x7f8d840fd7e0 msgr2=0x7f8d8410d130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:28:38.777 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.774+0000 7f8d8b7d6640 1 -- 172.21.15.26:0/242945941 shutdown_connections 2024-09-11T23:28:38.777 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:38.774+0000 7f8d8b7d6640 1 -- 172.21.15.26:0/242945941 wait complete. 2024-09-11T23:28:40.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:28:39 smithi076 ceph-mon[35296]: pgmap v319: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 13 op/s 2024-09-11T23:28:40.053 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:28:39 smithi026 ceph-mon[31172]: pgmap v319: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 13 op/s 2024-09-11T23:28:40.238 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:28:39 smithi136 ceph-mon[35121]: pgmap v319: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 13 op/s 2024-09-11T23:28:42.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:28:41 smithi076 ceph-mon[35296]: pgmap v320: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 51 KiB/s wr, 20 op/s 2024-09-11T23:28:42.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:28:41 smithi076 ceph-mon[35296]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:28:42.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:28:41 smithi136 ceph-mon[35121]: pgmap v320: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 51 KiB/s wr, 20 op/s 2024-09-11T23:28:42.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:28:41 smithi136 ceph-mon[35121]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:28:42.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:28:41 smithi026 ceph-mon[31172]: pgmap v320: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 51 KiB/s wr, 20 op/s 2024-09-11T23:28:42.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:28:41 smithi026 ceph-mon[31172]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:28:44.052 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:28:43 smithi076 ceph-mon[35296]: pgmap v321: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-09-11T23:28:44.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:28:43 smithi136 ceph-mon[35121]: pgmap v321: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-09-11T23:28:44.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:28:43 smithi026 ceph-mon[31172]: pgmap v321: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-09-11T23:28:46.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:28:45 smithi076 ceph-mon[35296]: pgmap v322: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 47 KiB/s wr, 18 op/s 2024-09-11T23:28:46.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:28:45 smithi136 ceph-mon[35121]: pgmap v322: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 47 KiB/s wr, 18 op/s 2024-09-11T23:28:46.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:28:45 smithi026 ceph-mon[31172]: pgmap v322: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 47 KiB/s wr, 18 op/s 2024-09-11T23:28:48.052 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:28:47 smithi076 ceph-mon[35296]: pgmap v323: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-09-11T23:28:48.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:28:47 smithi136 ceph-mon[35121]: pgmap v323: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-09-11T23:28:48.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:28:47 smithi026 ceph-mon[31172]: pgmap v323: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-09-11T23:28:50.052 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:28:49 smithi076 ceph-mon[35296]: pgmap v324: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 40 KiB/s wr, 14 op/s 2024-09-11T23:28:50.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:28:49 smithi136 ceph-mon[35121]: pgmap v324: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 40 KiB/s wr, 14 op/s 2024-09-11T23:28:50.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:28:49 smithi026 ceph-mon[31172]: pgmap v324: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 40 KiB/s wr, 14 op/s 2024-09-11T23:28:51.877 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.875+0000 7f682408d640 1 -- 172.21.15.26:0/721458127 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f681c102470 msgr2=0x7f681c10e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:51.877 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.875+0000 7f682408d640 1 --2- 172.21.15.26:0/721458127 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f681c102470 0x7f681c10e9a0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f6814009cd0 tx=0x7f681402a8e0 comp rx=0 tx=0).stop 2024-09-11T23:28:51.877 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.875+0000 7f682408d640 1 -- 172.21.15.26:0/721458127 shutdown_connections 2024-09-11T23:28:51.877 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.875+0000 7f682408d640 1 --2- 172.21.15.26:0/721458127 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f681c101ad0 0x7f681c101f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:51.877 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.875+0000 7f682408d640 1 --2- 172.21.15.26:0/721458127 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f681c102470 0x7f681c10e9a0 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:51.877 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.875+0000 7f682408d640 1 --2- 172.21.15.26:0/721458127 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f681c107b70 0x7f681c107f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:51.877 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.875+0000 7f682408d640 1 -- 172.21.15.26:0/721458127 >> 172.21.15.26:0/721458127 conn(0x7f681c0fd7d0 msgr2=0x7f681c0ffbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:28:51.877 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.875+0000 7f682408d640 1 -- 172.21.15.26:0/721458127 shutdown_connections 2024-09-11T23:28:51.877 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.875+0000 7f682408d640 1 -- 172.21.15.26:0/721458127 wait complete. 2024-09-11T23:28:51.878 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.875+0000 7f682408d640 1 Processor -- start 2024-09-11T23:28:51.878 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.876+0000 7f682408d640 1 -- start start 2024-09-11T23:28:51.878 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.876+0000 7f682408d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f681c101ad0 0x7f681c1982d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:51.878 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.876+0000 7f682408d640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f681c102470 0x7f681c198810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:51.878 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.876+0000 7f682408d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f681c107b70 0x7f681c19cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:51.878 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.876+0000 7f682408d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f681c111ee0 con 0x7f681c107b70 2024-09-11T23:28:51.878 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.876+0000 7f682408d640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f681c111d60 con 0x7f681c101ad0 2024-09-11T23:28:51.878 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.876+0000 7f682408d640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f681c112060 con 0x7f681c102470 2024-09-11T23:28:51.878 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.876+0000 7f682262a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f681c107b70 0x7f681c19cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:51.878 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.876+0000 7f682262a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f681c107b70 0x7f681c19cb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36302/0 (socket says 172.21.15.26:36302) 2024-09-11T23:28:51.878 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.876+0000 7f682262a640 1 -- 172.21.15.26:0/4275316869 learned_addr learned my addr 172.21.15.26:0/4275316869 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:28:51.878 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.876+0000 7f6821628640 1 --2- 172.21.15.26:0/4275316869 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f681c102470 0x7f681c198810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:51.878 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.876+0000 7f6821e29640 1 --2- 172.21.15.26:0/4275316869 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f681c101ad0 0x7f681c1982d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:51.879 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.876+0000 7f682262a640 1 -- 172.21.15.26:0/4275316869 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f681c102470 msgr2=0x7f681c198810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:51.879 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.876+0000 7f682262a640 1 --2- 172.21.15.26:0/4275316869 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f681c102470 0x7f681c198810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:51.879 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.876+0000 7f682262a640 1 -- 172.21.15.26:0/4275316869 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f681c101ad0 msgr2=0x7f681c1982d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:51.879 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.876+0000 7f682262a640 1 --2- 172.21.15.26:0/4275316869 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f681c101ad0 0x7f681c1982d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:51.879 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.876+0000 7f682262a640 1 -- 172.21.15.26:0/4275316869 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f681c19d2c0 con 0x7f681c107b70 2024-09-11T23:28:51.879 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.877+0000 7f682262a640 1 --2- 172.21.15.26:0/4275316869 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f681c107b70 0x7f681c19cb90 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f6814005cc0 tx=0x7f68140093d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:28:51.879 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.877+0000 7f6812ffd640 1 -- 172.21.15.26:0/4275316869 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6814003e00 con 0x7f681c107b70 2024-09-11T23:28:51.879 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.877+0000 7f6812ffd640 1 -- 172.21.15.26:0/4275316869 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6814004520 con 0x7f681c107b70 2024-09-11T23:28:51.879 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.877+0000 7f682408d640 1 -- 172.21.15.26:0/4275316869 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f681c19d550 con 0x7f681c107b70 2024-09-11T23:28:51.879 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.877+0000 7f6812ffd640 1 -- 172.21.15.26:0/4275316869 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6814030db0 con 0x7f681c107b70 2024-09-11T23:28:51.879 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.877+0000 7f682408d640 1 -- 172.21.15.26:0/4275316869 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f681c1a4e40 con 0x7f681c107b70 2024-09-11T23:28:51.882 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.879+0000 7f6812ffd640 1 -- 172.21.15.26:0/4275316869 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f68140046d0 con 0x7f681c107b70 2024-09-11T23:28:51.882 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.879+0000 7f6826e26740 1 Processor -- start 2024-09-11T23:28:51.882 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.880+0000 7f6826e26740 1 -- start start 2024-09-11T23:28:51.882 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.880+0000 7f6826e26740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561974801af0 0x561974801ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:51.882 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.880+0000 7f6826e26740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561974802410 0x56197480c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:51.883 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.880+0000 7f6826e26740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56197480ccb0 0x56197480f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:51.883 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.880+0000 7f6826e26740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5619747662f0 con 0x561974801af0 2024-09-11T23:28:51.883 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.880+0000 7f6826e26740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x56197470e270 con 0x561974802410 2024-09-11T23:28:51.883 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.880+0000 7f6826e26740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5619747622b0 con 0x56197480ccb0 2024-09-11T23:28:51.883 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.880+0000 7f6821628640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561974801af0 0x561974801ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:51.883 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.880+0000 7f6821628640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561974801af0 0x561974801ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36312/0 (socket says 172.21.15.26:36312) 2024-09-11T23:28:51.883 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.880+0000 7f6821628640 1 -- 172.21.15.26:0/2670394808 learned_addr learned my addr 172.21.15.26:0/2670394808 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:28:51.883 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.880+0000 7f6821e29640 1 --2- 172.21.15.26:0/2670394808 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561974802410 0x56197480c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:51.883 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.880+0000 7f6812ffd640 1 --2- 172.21.15.26:0/4275316869 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f68080a2e20 0x7f68080a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:51.883 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.881+0000 7f6821628640 1 --2- 172.21.15.26:0/2670394808 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56197480ccb0 0x56197480f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:51.883 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.881+0000 7f6821628640 1 -- 172.21.15.26:0/2670394808 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56197480ccb0 msgr2=0x56197480f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:28:51.883 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.881+0000 7f6821628640 1 --2- 172.21.15.26:0/2670394808 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56197480ccb0 0x56197480f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:51.883 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.881+0000 7f6821628640 1 -- 172.21.15.26:0/2670394808 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561974802410 msgr2=0x56197480c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:28:51.883 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.881+0000 7f6821628640 1 --2- 172.21.15.26:0/2670394808 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561974802410 0x56197480c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:51.884 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.881+0000 7f682262a640 1 --2- 172.21.15.26:0/4275316869 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f68080a2e20 0x7f68080a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:51.884 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.881+0000 7f6821628640 1 -- 172.21.15.26:0/2670394808 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56197470d340 con 0x561974801af0 2024-09-11T23:28:51.884 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.881+0000 7f6812ffd640 1 -- 172.21.15.26:0/4275316869 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8542+0+0 (secure 0 0 0) 0x7f681402e030 con 0x7f681c107b70 2024-09-11T23:28:51.884 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.881+0000 7f6821628640 1 --2- 172.21.15.26:0/2670394808 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561974801af0 0x561974801ed0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f680c0135a0 tx=0x7f680c009850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cba7c1f336c0caa1 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:28:51.884 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.881+0000 7f6807fff640 1 -- 172.21.15.26:0/2670394808 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f680c004050 con 0x561974801af0 2024-09-11T23:28:51.884 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.881+0000 7f6807fff640 1 -- 172.21.15.26:0/2670394808 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f680c0041f0 con 0x561974801af0 2024-09-11T23:28:51.884 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.881+0000 7f6807fff640 1 -- 172.21.15.26:0/2670394808 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f680c00f7e0 con 0x561974801af0 2024-09-11T23:28:51.884 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.882+0000 7f6826e26740 1 -- 172.21.15.26:0/2670394808 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561974801af0 msgr2=0x561974801ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:51.884 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.882+0000 7f6826e26740 1 --2- 172.21.15.26:0/2670394808 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561974801af0 0x561974801ed0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f680c0135a0 tx=0x7f680c009850 comp rx=0 tx=0).stop 2024-09-11T23:28:51.884 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.882+0000 7f682262a640 1 --2- 172.21.15.26:0/4275316869 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f68080a2e20 0x7f68080a52e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f68140bc980 tx=0x7f68140c0040 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:28:51.884 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.882+0000 7f6826e26740 1 -- 172.21.15.26:0/2670394808 shutdown_connections 2024-09-11T23:28:51.884 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.882+0000 7f6826e26740 1 --2- 172.21.15.26:0/2670394808 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561974802410 0x56197480c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:51.884 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.882+0000 7f6826e26740 1 --2- 172.21.15.26:0/2670394808 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56197480ccb0 0x56197480f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:51.885 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.882+0000 7f6826e26740 1 --2- 172.21.15.26:0/2670394808 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561974801af0 0x561974801ed0 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:51.885 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.882+0000 7f6826e26740 1 -- 172.21.15.26:0/2670394808 >> 172.21.15.26:0/2670394808 conn(0x5619747fe060 msgr2=0x5619747fe440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:28:51.885 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.882+0000 7f6826e26740 1 -- 172.21.15.26:0/2670394808 shutdown_connections 2024-09-11T23:28:51.885 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.882+0000 7f6826e26740 1 -- 172.21.15.26:0/2670394808 wait complete. 2024-09-11T23:28:51.885 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.882+0000 7f6826e26740 1 -- start start 2024-09-11T23:28:51.885 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.883+0000 7f6826e26740 1 Processor -- start 2024-09-11T23:28:51.889 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.883+0000 7f6826e26740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561974801af0 0x561974809ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:51.889 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.883+0000 7f6826e26740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561974802410 0x561974809ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:51.889 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.883+0000 7f6826e26740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56197480ccb0 0x56197480a530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:51.889 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.883+0000 7f6826e26740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x56197480f880 con 0x56197480ccb0 2024-09-11T23:28:51.889 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.883+0000 7f6826e26740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5619746ef290 con 0x561974802410 2024-09-11T23:28:51.889 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.883+0000 7f6826e26740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x56197480fa00 con 0x561974801af0 2024-09-11T23:28:51.889 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.883+0000 7f6826e26740 10 client.0 _ll_register_callbacks cb 0x561974720140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:28:51.889 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.883+0000 7f6821628640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56197480ccb0 0x56197480a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:51.889 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.883+0000 7f6821628640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56197480ccb0 0x56197480a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36328/0 (socket says 172.21.15.26:36328) 2024-09-11T23:28:51.889 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.883+0000 7f6821628640 1 -- 172.21.15.26:0/3860145746 learned_addr learned my addr 172.21.15.26:0/3860145746 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:28:51.889 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.883+0000 7f6821628640 1 --2- 172.21.15.26:0/3860145746 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561974801af0 0x561974809ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:51.890 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.883+0000 7f6821e29640 1 --2- 172.21.15.26:0/3860145746 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561974802410 0x561974809ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:51.890 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.883+0000 7f6821628640 1 -- 172.21.15.26:0/3860145746 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561974801af0 msgr2=0x561974809ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:28:51.890 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.883+0000 7f6821628640 1 --2- 172.21.15.26:0/3860145746 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561974801af0 0x561974809ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:51.890 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.884+0000 7f6821628640 1 -- 172.21.15.26:0/3860145746 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561974802410 msgr2=0x561974809ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:28:51.890 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.884+0000 7f6821628640 1 --2- 172.21.15.26:0/3860145746 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561974802410 0x561974809ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:51.890 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.884+0000 7f6821628640 1 -- 172.21.15.26:0/3860145746 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561974479210 con 0x56197480ccb0 2024-09-11T23:28:51.890 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.884+0000 7f6821628640 1 --2- 172.21.15.26:0/3860145746 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56197480ccb0 0x56197480a530 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f680c003e30 tx=0x7f680c003d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=151943b186f1bd1a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:28:51.890 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.884+0000 7f68057fa640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:28:51.890 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.884+0000 7f68057fa640 1 -- 172.21.15.26:0/3860145746 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f680c003800 con 0x56197480ccb0 2024-09-11T23:28:51.890 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.884+0000 7f68057fa640 1 -- 172.21.15.26:0/3860145746 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8542+0+0 (secure 0 0 0) 0x7f680c0039a0 con 0x56197480ccb0 2024-09-11T23:28:51.890 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.884+0000 7f6826e26740 1 -- 172.21.15.26:0/3860145746 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f680c0041f0 con 0x56197480ccb0 2024-09-11T23:28:51.890 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.884+0000 7f68057fa640 1 -- 172.21.15.26:0/3860145746 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f680c034050 con 0x56197480ccb0 2024-09-11T23:28:51.890 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.884+0000 7f6826e26740 10 client.16983 fetch_fsmap learned FSMap version 54 2024-09-11T23:28:51.891 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.884+0000 7f6826e26740 1 -- 172.21.15.26:0/3860145746 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x561974545620 con 0x56197480ccb0 2024-09-11T23:28:51.891 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.884+0000 7f68057fa640 1 -- 172.21.15.26:0/3860145746 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f680c0041f0 con 0x56197480ccb0 2024-09-11T23:28:51.891 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.884+0000 7f68057fa640 1 -- 172.21.15.26:0/3860145746 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 54) ==== 8382+0+0 (secure 0 0 0) 0x7f680c01e070 con 0x56197480ccb0 2024-09-11T23:28:51.891 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.885+0000 7f6826e26740 10 client.16983 fetch_fsmap finished waiting for FSMap version 54 2024-09-11T23:28:51.891 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.885+0000 7f6826e26740 10 client.16983 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:28:51.891 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.885+0000 7f6826e26740 20 client.16983 populate_metadata read hostname 'smithi026' 2024-09-11T23:28:51.891 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.885+0000 7f6826e26740 1 --2- 172.21.15.26:0/3860145746 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x56197481c9e0 0x56197481edd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:51.891 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.885+0000 7f6826e26740 4 client.16983 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:28:51.891 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.885+0000 7f6826e26740 1 -- 172.21.15.26:0/3860145746 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561974773a20 con 0x56197481c9e0 2024-09-11T23:28:51.891 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.885+0000 7f6821e29640 1 --2- 172.21.15.26:0/3860145746 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x56197481c9e0 0x56197481edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:51.891 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.885+0000 7f6821e29640 1 --2- 172.21.15.26:0/3860145746 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x56197481c9e0 0x56197481edd0 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dfbc92362b59ec2 server_cookie=3172782d5e96bbac in_seq=0 out_seq=0 2024-09-11T23:28:51.891 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.885+0000 7f68057fa640 10 client.16983 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:28:51.891 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.886+0000 7f68057fa640 1 -- 172.21.15.26:0/3860145746 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561974773a20 con 0x56197481c9e0 2024-09-11T23:28:51.891 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.886+0000 7f68057fa640 10 client.16983 handle_command_reply: tid=0 2024-09-11T23:28:51.892 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.886+0000 7f6826e26740 1 client.16983 shutdown 2024-09-11T23:28:51.892 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.886+0000 7f6826e26740 1 -- 172.21.15.26:0/3860145746 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56197480ccb0 msgr2=0x56197480a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:51.892 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.886+0000 7f6826e26740 1 --2- 172.21.15.26:0/3860145746 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56197480ccb0 0x56197480a530 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f680c003e30 tx=0x7f680c003d30 comp rx=0 tx=0).stop 2024-09-11T23:28:51.892 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.886+0000 7f6826e26740 1 -- 172.21.15.26:0/3860145746 shutdown_connections 2024-09-11T23:28:51.892 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.886+0000 7f6826e26740 1 --2- 172.21.15.26:0/3860145746 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561974802410 0x561974809ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:51.892 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.886+0000 7f6826e26740 1 --2- 172.21.15.26:0/3860145746 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x56197481c9e0 0x56197481edd0 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:51.892 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.886+0000 7f6826e26740 1 --2- 172.21.15.26:0/3860145746 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56197480ccb0 0x56197480a530 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:51.892 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.886+0000 7f6826e26740 1 --2- 172.21.15.26:0/3860145746 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561974801af0 0x561974809ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:51.892 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.886+0000 7f68057fa640 0 client.16983 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:28:51.892 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.886+0000 7f6826e26740 1 -- 172.21.15.26:0/3860145746 >> 172.21.15.26:0/3860145746 conn(0x5619747fe060 msgr2=0x561974803dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:28:51.892 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.886+0000 7f6826e26740 1 -- 172.21.15.26:0/3860145746 shutdown_connections 2024-09-11T23:28:51.892 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.886+0000 7f6826e26740 1 -- 172.21.15.26:0/3860145746 wait complete. 2024-09-11T23:28:51.892 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.887+0000 7f6826e26740 20 client.16983 trim_cache size 0 max 16384 2024-09-11T23:28:51.902 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.901+0000 7f6826e26740 1 Processor -- start 2024-09-11T23:28:51.902 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.901+0000 7f6826e26740 1 -- start start 2024-09-11T23:28:51.902 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.901+0000 7f6826e26740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561974801af0 0x561974808360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:51.902 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.901+0000 7f6826e26740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561974802410 0x56197480a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:51.902 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.901+0000 7f6826e26740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56197480ccb0 0x5619748150d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:51.902 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.901+0000 7f6826e26740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f681c1ad790 con 0x561974801af0 2024-09-11T23:28:51.902 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.901+0000 7f6826e26740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f681c1af3c0 con 0x56197480ccb0 2024-09-11T23:28:51.902 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.901+0000 7f6826e26740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f681c1ade60 con 0x561974802410 2024-09-11T23:28:51.903 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.902+0000 7f6821628640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561974801af0 0x561974808360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:51.903 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.902+0000 7f6821628640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561974801af0 0x561974808360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36332/0 (socket says 172.21.15.26:36332) 2024-09-11T23:28:51.903 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.902+0000 7f6821628640 1 -- 172.21.15.26:0/3848370295 learned_addr learned my addr 172.21.15.26:0/3848370295 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:28:51.903 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.902+0000 7f6821628640 1 -- 172.21.15.26:0/3848370295 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561974802410 msgr2=0x56197480a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:28:51.903 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.902+0000 7f6821628640 1 --2- 172.21.15.26:0/3848370295 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561974802410 0x56197480a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:51.903 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.902+0000 7f6821628640 1 -- 172.21.15.26:0/3848370295 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56197480ccb0 msgr2=0x5619748150d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:28:51.903 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.902+0000 7f6821628640 1 --2- 172.21.15.26:0/3848370295 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56197480ccb0 0x5619748150d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:51.903 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.902+0000 7f6821628640 1 -- 172.21.15.26:0/3848370295 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5619747fde90 con 0x561974801af0 2024-09-11T23:28:51.903 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.902+0000 7f6821628640 1 --2- 172.21.15.26:0/3848370295 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561974801af0 0x561974808360 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f680c006010 tx=0x7f680c005d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=522d1f1b57f76b57 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:28:51.903 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.902+0000 7f68077fe640 1 -- 172.21.15.26:0/3848370295 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f680c025070 con 0x561974801af0 2024-09-11T23:28:51.903 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.902+0000 7f68077fe640 1 -- 172.21.15.26:0/3848370295 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f680c013270 con 0x561974801af0 2024-09-11T23:28:51.903 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.902+0000 7f68077fe640 1 -- 172.21.15.26:0/3848370295 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f680c00f830 con 0x561974801af0 2024-09-11T23:28:51.907 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.903+0000 7f6826e26740 1 -- 172.21.15.26:0/3848370295 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561974801af0 msgr2=0x561974808360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:51.907 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.903+0000 7f6826e26740 1 --2- 172.21.15.26:0/3848370295 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561974801af0 0x561974808360 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f680c006010 tx=0x7f680c005d30 comp rx=0 tx=0).stop 2024-09-11T23:28:51.907 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.903+0000 7f6826e26740 1 -- 172.21.15.26:0/3848370295 shutdown_connections 2024-09-11T23:28:51.907 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.903+0000 7f6826e26740 1 --2- 172.21.15.26:0/3848370295 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56197480ccb0 0x5619748150d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:51.907 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.903+0000 7f6826e26740 1 --2- 172.21.15.26:0/3848370295 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561974802410 0x56197480a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:51.907 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.903+0000 7f6826e26740 1 --2- 172.21.15.26:0/3848370295 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561974801af0 0x561974808360 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:51.907 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.903+0000 7f6826e26740 1 -- 172.21.15.26:0/3848370295 >> 172.21.15.26:0/3848370295 conn(0x5619747fe060 msgr2=0x56197480bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:28:51.907 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.903+0000 7f6826e26740 1 -- 172.21.15.26:0/3848370295 shutdown_connections 2024-09-11T23:28:51.908 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.903+0000 7f6826e26740 1 -- 172.21.15.26:0/3848370295 wait complete. 2024-09-11T23:28:51.908 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.903+0000 7f6826e26740 1 -- start start 2024-09-11T23:28:51.908 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.903+0000 7f6826e26740 1 Processor -- start 2024-09-11T23:28:51.908 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.904+0000 7f6826e26740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561974801af0 0x5619748074c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:51.908 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.904+0000 7f6826e26740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561974802410 0x561974805f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:51.908 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.904+0000 7f6826e26740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56197480ccb0 0x5619748064d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:51.908 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.904+0000 7f6826e26740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f681c1aecf0 con 0x561974801af0 2024-09-11T23:28:51.908 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.904+0000 7f6826e26740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f681c1ae590 con 0x56197480ccb0 2024-09-11T23:28:51.908 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.904+0000 7f6826e26740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f681c1ab790 con 0x561974802410 2024-09-11T23:28:51.908 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.904+0000 7f6826e26740 10 client.0 _ll_register_callbacks cb 0x7f68180097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:28:51.908 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.904+0000 7f6821628640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561974801af0 0x5619748074c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:51.908 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.904+0000 7f6821628640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561974801af0 0x5619748074c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36346/0 (socket says 172.21.15.26:36346) 2024-09-11T23:28:51.908 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.904+0000 7f6821628640 1 -- 172.21.15.26:0/789439737 learned_addr learned my addr 172.21.15.26:0/789439737 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:28:51.909 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.904+0000 7f6821e29640 1 --2- 172.21.15.26:0/789439737 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561974802410 0x561974805f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:51.909 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.904+0000 7f6821628640 1 --2- 172.21.15.26:0/789439737 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56197480ccb0 0x5619748064d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:51.909 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.904+0000 7f6821628640 1 -- 172.21.15.26:0/789439737 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561974802410 msgr2=0x561974805f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:28:51.909 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.904+0000 7f6821628640 1 --2- 172.21.15.26:0/789439737 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561974802410 0x561974805f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:51.909 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.904+0000 7f6821628640 1 -- 172.21.15.26:0/789439737 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56197480ccb0 msgr2=0x5619748064d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:28:51.909 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.904+0000 7f6821628640 1 --2- 172.21.15.26:0/789439737 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56197480ccb0 0x5619748064d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:51.909 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.904+0000 7f6821628640 1 -- 172.21.15.26:0/789439737 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5619747f92d0 con 0x561974801af0 2024-09-11T23:28:51.909 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.904+0000 7f6821628640 1 --2- 172.21.15.26:0/789439737 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561974801af0 0x5619748074c0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f680c00a010 tx=0x7f680c004330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d509c4ca9489ed0c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:28:51.909 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.904+0000 7f68057fa640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:28:51.909 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.904+0000 7f68057fa640 1 -- 172.21.15.26:0/789439737 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f680c025040 con 0x561974801af0 2024-09-11T23:28:51.909 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.905+0000 7f68057fa640 1 -- 172.21.15.26:0/789439737 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8542+0+0 (secure 0 0 0) 0x7f680c02b070 con 0x561974801af0 2024-09-11T23:28:51.909 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.905+0000 7f6826e26740 1 -- 172.21.15.26:0/789439737 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f680c013270 con 0x561974801af0 2024-09-11T23:28:51.909 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.905+0000 7f68057fa640 1 -- 172.21.15.26:0/789439737 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f680c049050 con 0x561974801af0 2024-09-11T23:28:51.910 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.905+0000 7f6826e26740 10 client.16989 fetch_fsmap learned FSMap version 54 2024-09-11T23:28:51.910 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.905+0000 7f6826e26740 1 -- 172.21.15.26:0/789439737 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x5619747ff110 con 0x561974801af0 2024-09-11T23:28:51.910 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.905+0000 7f68057fa640 1 -- 172.21.15.26:0/789439737 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f680c013270 con 0x561974801af0 2024-09-11T23:28:51.910 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.905+0000 7f68057fa640 1 -- 172.21.15.26:0/789439737 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 54) ==== 8382+0+0 (secure 0 0 0) 0x7f680c01e050 con 0x561974801af0 2024-09-11T23:28:51.910 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.905+0000 7f6826e26740 10 client.16989 fetch_fsmap finished waiting for FSMap version 54 2024-09-11T23:28:51.910 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.905+0000 7f6826e26740 10 client.16989 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:28:51.910 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.905+0000 7f6826e26740 20 client.16989 populate_metadata read hostname 'smithi026' 2024-09-11T23:28:51.910 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.905+0000 7f6826e26740 1 --2- 172.21.15.26:0/789439737 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x56197488b700 0x56197488daf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:51.910 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.905+0000 7f6826e26740 4 client.16989 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T23:28:51.910 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.905+0000 7f6826e26740 1 -- 172.21.15.26:0/789439737 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56197476dd70 con 0x56197488b700 2024-09-11T23:28:51.910 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.905+0000 7f6821e29640 1 --2- 172.21.15.26:0/789439737 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x56197488b700 0x56197488daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:51.910 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.906+0000 7f6821e29640 1 --2- 172.21.15.26:0/789439737 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x56197488b700 0x56197488daf0 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ffe59fcc7929af3c server_cookie=91616bf6606ab96c in_seq=0 out_seq=0 2024-09-11T23:28:51.911 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.906+0000 7f68057fa640 10 client.16989 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:28:51.922 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.922+0000 7f68057fa640 1 -- 172.21.15.26:0/789439737 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56197476dd70 con 0x56197488b700 2024-09-11T23:28:51.922 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.922+0000 7f68057fa640 10 client.16989 handle_command_reply: tid=0 2024-09-11T23:28:51.923 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.922+0000 7f6826e26740 1 client.16989 shutdown 2024-09-11T23:28:51.923 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.922+0000 7f6826e26740 1 -- 172.21.15.26:0/789439737 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561974801af0 msgr2=0x5619748074c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:51.923 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.922+0000 7f6826e26740 1 --2- 172.21.15.26:0/789439737 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561974801af0 0x5619748074c0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f680c00a010 tx=0x7f680c004330 comp rx=0 tx=0).stop 2024-09-11T23:28:51.923 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.922+0000 7f6826e26740 1 -- 172.21.15.26:0/789439737 shutdown_connections 2024-09-11T23:28:51.923 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.922+0000 7f6826e26740 1 --2- 172.21.15.26:0/789439737 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56197480ccb0 0x5619748064d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:51.923 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.922+0000 7f6826e26740 1 --2- 172.21.15.26:0/789439737 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x56197488b700 0x56197488daf0 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:51.923 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.922+0000 7f6826e26740 1 --2- 172.21.15.26:0/789439737 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561974802410 0x561974805f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:51.923 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.922+0000 7f6826e26740 1 --2- 172.21.15.26:0/789439737 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561974801af0 0x5619748074c0 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:51.923 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.922+0000 7f68057fa640 0 client.16989 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:28:51.923 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.922+0000 7f6826e26740 1 -- 172.21.15.26:0/789439737 >> 172.21.15.26:0/789439737 conn(0x5619747fe060 msgr2=0x5619747fc9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:28:51.923 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.922+0000 7f6826e26740 1 -- 172.21.15.26:0/789439737 shutdown_connections 2024-09-11T23:28:51.923 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.922+0000 7f6826e26740 1 -- 172.21.15.26:0/789439737 wait complete. 2024-09-11T23:28:51.924 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.922+0000 7f6826e26740 20 client.16989 trim_cache size 0 max 16384 2024-09-11T23:28:51.924 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-09-11T23:28:51.924 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "message": "", 2024-09-11T23:28:51.924 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-09-11T23:28:51.924 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-09-11T23:28:51.925 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.923+0000 7f682408d640 1 -- 172.21.15.26:0/4275316869 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f68080a2e20 msgr2=0x7f68080a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:51.925 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.923+0000 7f682408d640 1 --2- 172.21.15.26:0/4275316869 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f68080a2e20 0x7f68080a52e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f68140bc980 tx=0x7f68140c0040 comp rx=0 tx=0).stop 2024-09-11T23:28:51.925 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.923+0000 7f682408d640 1 -- 172.21.15.26:0/4275316869 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f681c107b70 msgr2=0x7f681c19cb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:51.926 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.923+0000 7f682408d640 1 --2- 172.21.15.26:0/4275316869 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f681c107b70 0x7f681c19cb90 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f6814005cc0 tx=0x7f68140093d0 comp rx=0 tx=0).stop 2024-09-11T23:28:51.926 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.923+0000 7f682408d640 1 -- 172.21.15.26:0/4275316869 shutdown_connections 2024-09-11T23:28:51.926 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.923+0000 7f682408d640 1 --2- 172.21.15.26:0/4275316869 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f68080a2e20 0x7f68080a52e0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:51.926 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.923+0000 7f682408d640 1 --2- 172.21.15.26:0/4275316869 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f681c107b70 0x7f681c19cb90 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:51.926 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.923+0000 7f682408d640 1 --2- 172.21.15.26:0/4275316869 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f681c102470 0x7f681c198810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:51.926 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.923+0000 7f682408d640 1 --2- 172.21.15.26:0/4275316869 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f681c101ad0 0x7f681c1982d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:51.926 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.923+0000 7f682408d640 1 -- 172.21.15.26:0/4275316869 >> 172.21.15.26:0/4275316869 conn(0x7f681c0fd7d0 msgr2=0x7f681c0fdde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:28:51.926 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.923+0000 7f682408d640 1 -- 172.21.15.26:0/4275316869 shutdown_connections 2024-09-11T23:28:51.926 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:28:51.923+0000 7f682408d640 1 -- 172.21.15.26:0/4275316869 wait complete. 2024-09-11T23:28:51.950 INFO:tasks.quiescer.fs.[cephfs]:Total duration (45.1) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-11T23:28:51.950 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 22.55 2024-09-11T23:28:51.950 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:28:52.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:28:51 smithi076 ceph-mon[35296]: pgmap v325: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 44 KiB/s wr, 19 op/s 2024-09-11T23:28:52.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:28:51 smithi076 ceph-mon[35296]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:28:52.215 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:28:52.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:28:51 smithi136 ceph-mon[35121]: pgmap v325: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 44 KiB/s wr, 19 op/s 2024-09-11T23:28:52.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:28:51 smithi136 ceph-mon[35121]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:28:52.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:28:51 smithi026 ceph-mon[31172]: pgmap v325: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 44 KiB/s wr, 19 op/s 2024-09-11T23:28:52.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:28:51 smithi026 ceph-mon[31172]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:28:52.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:52.706+0000 7f8cff8d3640 1 -- 172.21.15.26:0/2470570330 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8cf8103720 msgr2=0x7f8cf810bc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:52.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:52.706+0000 7f8cff8d3640 1 --2- 172.21.15.26:0/2470570330 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8cf8103720 0x7f8cf810bc10 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f8ce8009900 tx=0x7f8ce802a480 comp rx=0 tx=0).stop 2024-09-11T23:28:52.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:52.706+0000 7f8cff8d3640 1 -- 172.21.15.26:0/2470570330 shutdown_connections 2024-09-11T23:28:52.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:52.706+0000 7f8cff8d3640 1 --2- 172.21.15.26:0/2470570330 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8cf810c1a0 0x7f8cf810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:52.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:52.706+0000 7f8cff8d3640 1 --2- 172.21.15.26:0/2470570330 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8cf8103720 0x7f8cf810bc10 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:52.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:52.706+0000 7f8cff8d3640 1 --2- 172.21.15.26:0/2470570330 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8cf8102e00 0x7f8cf81031e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:52.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:52.706+0000 7f8cff8d3640 1 -- 172.21.15.26:0/2470570330 >> 172.21.15.26:0/2470570330 conn(0x7f8cf80fc7a0 msgr2=0x7f8cf80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:28:52.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:52.707+0000 7f8cff8d3640 1 -- 172.21.15.26:0/2470570330 shutdown_connections 2024-09-11T23:28:52.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:52.707+0000 7f8cff8d3640 1 -- 172.21.15.26:0/2470570330 wait complete. 2024-09-11T23:28:52.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:52.707+0000 7f8cff8d3640 1 Processor -- start 2024-09-11T23:28:52.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:52.707+0000 7f8cff8d3640 1 -- start start 2024-09-11T23:28:52.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:52.707+0000 7f8cff8d3640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8cf8102e00 0x7f8cf8100a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:52.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:52.707+0000 7f8cff8d3640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8cf8103720 0x7f8cf8100fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:52.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:52.707+0000 7f8cff8d3640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8cf810c1a0 0x7f8cf81014e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:52.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:52.707+0000 7f8cff8d3640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f8cf8111f70 con 0x7f8cf810c1a0 2024-09-11T23:28:52.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:52.707+0000 7f8cff8d3640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f8cf8111df0 con 0x7f8cf8102e00 2024-09-11T23:28:52.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:52.707+0000 7f8cff8d3640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f8cf81120f0 con 0x7f8cf8103720 2024-09-11T23:28:52.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:52.708+0000 7f8cfde70640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8cf810c1a0 0x7f8cf81014e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:52.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:52.708+0000 7f8cfce6e640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8cf8103720 0x7f8cf8100fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:52.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:52.708+0000 7f8cfde70640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8cf810c1a0 0x7f8cf81014e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36362/0 (socket says 172.21.15.26:36362) 2024-09-11T23:28:52.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:52.708+0000 7f8cfde70640 1 -- 172.21.15.26:0/3541385956 learned_addr learned my addr 172.21.15.26:0/3541385956 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:28:52.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:52.708+0000 7f8cfd66f640 1 --2- 172.21.15.26:0/3541385956 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8cf8102e00 0x7f8cf8100a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:52.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:52.708+0000 7f8cfde70640 1 -- 172.21.15.26:0/3541385956 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8cf8103720 msgr2=0x7f8cf8100fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:52.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:52.708+0000 7f8cfde70640 1 --2- 172.21.15.26:0/3541385956 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8cf8103720 0x7f8cf8100fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:52.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:52.708+0000 7f8cfde70640 1 -- 172.21.15.26:0/3541385956 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8cf8102e00 msgr2=0x7f8cf8100a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:52.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:52.708+0000 7f8cfde70640 1 --2- 172.21.15.26:0/3541385956 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8cf8102e00 0x7f8cf8100a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:52.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:52.708+0000 7f8cfde70640 1 -- 172.21.15.26:0/3541385956 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cf81a6910 con 0x7f8cf810c1a0 2024-09-11T23:28:52.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:52.708+0000 7f8cfde70640 1 --2- 172.21.15.26:0/3541385956 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8cf810c1a0 0x7f8cf81014e0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f8cf000eaa0 tx=0x7f8cf000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:28:52.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:52.708+0000 7f8cee7fc640 1 -- 172.21.15.26:0/3541385956 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cf000cdb0 con 0x7f8cf810c1a0 2024-09-11T23:28:52.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:52.708+0000 7f8cee7fc640 1 -- 172.21.15.26:0/3541385956 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8cf0005020 con 0x7f8cf810c1a0 2024-09-11T23:28:52.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:52.708+0000 7f8cff8d3640 1 -- 172.21.15.26:0/3541385956 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8cf81a6c00 con 0x7f8cf810c1a0 2024-09-11T23:28:52.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:52.709+0000 7f8cee7fc640 1 -- 172.21.15.26:0/3541385956 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cf00116b0 con 0x7f8cf810c1a0 2024-09-11T23:28:52.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:52.709+0000 7f8cff8d3640 1 -- 172.21.15.26:0/3541385956 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8cf81a7110 con 0x7f8cf810c1a0 2024-09-11T23:28:52.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:52.710+0000 7f8cff8d3640 1 -- 172.21.15.26:0/3541385956 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8cf806afb0 con 0x7f8cf810c1a0 2024-09-11T23:28:52.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:52.711+0000 7f8cee7fc640 1 -- 172.21.15.26:0/3541385956 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f8cf0005560 con 0x7f8cf810c1a0 2024-09-11T23:28:52.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:52.712+0000 7f8cee7fc640 1 --2- 172.21.15.26:0/3541385956 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f8ce40a2d70 0x7f8ce40a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:52.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:52.712+0000 7f8cfd66f640 1 --2- 172.21.15.26:0/3541385956 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f8ce40a2d70 0x7f8ce40a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:52.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:52.712+0000 7f8cee7fc640 1 -- 172.21.15.26:0/3541385956 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8542+0+0 (secure 0 0 0) 0x7f8cf00cf530 con 0x7f8cf810c1a0 2024-09-11T23:28:52.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:52.713+0000 7f8cfd66f640 1 --2- 172.21.15.26:0/3541385956 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f8ce40a2d70 0x7f8ce40a5230 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f8cf400bbe0 tx=0x7f8cf400b390 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:28:52.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:52.717+0000 7f8cee7fc640 1 -- 172.21.15.26:0/3541385956 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8cf0016030 con 0x7f8cf810c1a0 2024-09-11T23:28:52.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:52.938+0000 7f8cff8d3640 1 -- 172.21.15.26:0/3541385956 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8cf8107220 con 0x7f8cf810c1a0 2024-09-11T23:28:52.938 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:28:52 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:28:52.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:52.939+0000 7f8cee7fc640 1 -- 172.21.15.26:0/3541385956 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) ==== 94+0+11976 (secure 0 0 0) 0x7f8cf009baf0 con 0x7f8cf810c1a0 2024-09-11T23:28:52.939 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-11T23:28:52.939 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":55,"btime":"2024-09-11T23:28:52:794802+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14499,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/367659515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":367659515},{"type":"v1","addr":"172.21.15.26:6835","nonce":367659515}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14511,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/2041065299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":2041065299},{"type":"v1","addr":"172.21.15.26:6837","nonce":2041065299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":55,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:16:26.544203+0000","modified":"2024-09-11T23:28:51.905238+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24485,"mds_1":24463,"mds_2":24458,"mds_3":24448,"mds_4":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14550":{"gid":14550,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6839/2233068305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":2233068305},{"type":"v1","addr":"172.21.15.26:6839","nonce":2233068305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14562":{"gid":14562,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6841/2984982336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":2984982336},{"type":"v1","addr":"172.21.15.26:6841","nonce":2984982336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6833/2019841455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":2019841455},{"type":"v1","addr":"172.21.15.136:6833","nonce":2019841455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.76:6833/3438147050","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3438147050},{"type":"v1","addr":"172.21.15.76:6833","nonce":3438147050}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6835/3617665975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3617665975},{"type":"v1","addr":"172.21.15.136:6835","nonce":3617665975}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":12,"addr":"172.21.15.76:6835/393308499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":393308499},{"type":"v1","addr":"172.21.15.76:6835","nonce":393308499}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":10,"addr":"172.21.15.136:6837/1633082387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":1633082387},{"type":"v1","addr":"172.21.15.136:6837","nonce":1633082387}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.76:6837/2078978748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":2078978748},{"type":"v1","addr":"172.21.15.76:6837","nonce":2078978748}]},"join_fscid":-1,"export_targets":[1,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":63,"addr":"172.21.15.136:6839/3211398630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":3211398630},{"type":"v1","addr":"172.21.15.136:6839","nonce":3211398630}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.76:6839/2077637090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6838","nonce":2077637090},{"type":"v1","addr":"172.21.15.76:6839","nonce":2077637090}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24485,"qdb_cluster":[24443,24448,24458,24463,24485]},"id":1}]} 2024-09-11T23:28:52.939 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 55 2024-09-11T23:28:52.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:52.941+0000 7f8cff8d3640 1 -- 172.21.15.26:0/3541385956 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f8ce40a2d70 msgr2=0x7f8ce40a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:52.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:52.941+0000 7f8cff8d3640 1 --2- 172.21.15.26:0/3541385956 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f8ce40a2d70 0x7f8ce40a5230 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f8cf400bbe0 tx=0x7f8cf400b390 comp rx=0 tx=0).stop 2024-09-11T23:28:52.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:52.941+0000 7f8cff8d3640 1 -- 172.21.15.26:0/3541385956 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8cf810c1a0 msgr2=0x7f8cf81014e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:52.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:52.941+0000 7f8cff8d3640 1 --2- 172.21.15.26:0/3541385956 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8cf810c1a0 0x7f8cf81014e0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f8cf000eaa0 tx=0x7f8cf000ef60 comp rx=0 tx=0).stop 2024-09-11T23:28:52.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:52.941+0000 7f8cff8d3640 1 -- 172.21.15.26:0/3541385956 shutdown_connections 2024-09-11T23:28:52.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:52.941+0000 7f8cff8d3640 1 --2- 172.21.15.26:0/3541385956 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f8ce40a2d70 0x7f8ce40a5230 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:52.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:52.941+0000 7f8cff8d3640 1 --2- 172.21.15.26:0/3541385956 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8cf810c1a0 0x7f8cf81014e0 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:52.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:52.941+0000 7f8cff8d3640 1 --2- 172.21.15.26:0/3541385956 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8cf8103720 0x7f8cf8100fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:52.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:52.941+0000 7f8cff8d3640 1 --2- 172.21.15.26:0/3541385956 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8cf8102e00 0x7f8cf8100a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:52.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:52.941+0000 7f8cff8d3640 1 -- 172.21.15.26:0/3541385956 >> 172.21.15.26:0/3541385956 conn(0x7f8cf80fc7a0 msgr2=0x7f8cf80fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:28:52.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:52.941+0000 7f8cff8d3640 1 -- 172.21.15.26:0/3541385956 shutdown_connections 2024-09-11T23:28:52.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:52.941+0000 7f8cff8d3640 1 -- 172.21.15.26:0/3541385956 wait complete. 2024-09-11T23:28:53.052 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:28:52 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:28:53.070 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:28:52 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:28:53.891 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:28:53 smithi026 ceph-mon[31172]: pgmap v326: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 33 KiB/s wr, 11 op/s 2024-09-11T23:28:53.891 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:28:53 smithi026 ceph-mon[31172]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:28:53.891 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:28:53 smithi026 ceph-mon[31172]: from='client.? 172.21.15.26:0/3541385956' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:28:53.891 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:28:53 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:28:53.891 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:28:53 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:28:53.891 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:28:53 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:28:53.891 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:28:53 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:28:54.052 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:28:53 smithi076 ceph-mon[35296]: pgmap v326: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 33 KiB/s wr, 11 op/s 2024-09-11T23:28:54.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:28:53 smithi076 ceph-mon[35296]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:28:54.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:28:53 smithi076 ceph-mon[35296]: from='client.? 172.21.15.26:0/3541385956' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:28:54.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:28:53 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:28:54.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:28:53 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:28:54.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:28:53 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:28:54.054 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:28:53 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:28:54.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:28:53 smithi136 ceph-mon[35121]: pgmap v326: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 33 KiB/s wr, 11 op/s 2024-09-11T23:28:54.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:28:53 smithi136 ceph-mon[35121]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:28:54.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:28:53 smithi136 ceph-mon[35121]: from='client.? 172.21.15.26:0/3541385956' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:28:54.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:28:53 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:28:54.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:28:53 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:28:54.240 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:28:53 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:28:54.240 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:28:53 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:28:54.319 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24485 quiesce db / --timeout 90 --expiration 142.55 --await-for 45.0' 2024-09-11T23:28:54.320 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24485 quiesce db / --timeout 90 --expiration 142.55 --await-for 45.0 2024-09-11T23:28:54.595 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:28:54.995 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:28:54 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:28:54.995 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:28:54 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:28:54.996 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:28:54 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:28:54.996 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:28:54 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:28:54.996 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:28:54 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:28:54.996 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:28:54 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:28:54.996 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:28:54 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:28:54.996 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:28:54 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:28:54.996 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:28:54 smithi026 ceph-mon[31172]: Adjusting osd_memory_target on smithi076 to 176.0M 2024-09-11T23:28:54.996 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:28:54 smithi026 ceph-mon[31172]: Adjusting osd_memory_target on smithi136 to 176.0M 2024-09-11T23:28:54.997 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:28:54 smithi026 ceph-mon[31172]: Unable to set osd_memory_target on smithi076 to 184591052: error parsing value: Value '184591052' is below minimum 939524096 2024-09-11T23:28:54.997 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:28:54 smithi026 ceph-mon[31172]: Unable to set osd_memory_target on smithi136 to 184574566: error parsing value: Value '184574566' is below minimum 939524096 2024-09-11T23:28:55.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.088+0000 7f52df033640 1 -- 172.21.15.26:0/1849872516 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52d8103660 msgr2=0x7f52d810bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:55.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.088+0000 7f52df033640 1 --2- 172.21.15.26:0/1849872516 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52d8103660 0x7f52d810bb50 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f52cc009900 tx=0x7f52cc02a480 comp rx=0 tx=0).stop 2024-09-11T23:28:55.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.088+0000 7f52df033640 1 -- 172.21.15.26:0/1849872516 shutdown_connections 2024-09-11T23:28:55.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.088+0000 7f52df033640 1 --2- 172.21.15.26:0/1849872516 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f52d810c170 0x7f52d810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:55.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.088+0000 7f52df033640 1 --2- 172.21.15.26:0/1849872516 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52d8103660 0x7f52d810bb50 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:55.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.088+0000 7f52df033640 1 --2- 172.21.15.26:0/1849872516 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f52d8102d40 0x7f52d8103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:55.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.088+0000 7f52df033640 1 -- 172.21.15.26:0/1849872516 >> 172.21.15.26:0/1849872516 conn(0x7f52d80fc7a0 msgr2=0x7f52d80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:28:55.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.088+0000 7f52df033640 1 -- 172.21.15.26:0/1849872516 shutdown_connections 2024-09-11T23:28:55.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.088+0000 7f52df033640 1 -- 172.21.15.26:0/1849872516 wait complete. 2024-09-11T23:28:55.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.089+0000 7f52df033640 1 Processor -- start 2024-09-11T23:28:55.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.089+0000 7f52df033640 1 -- start start 2024-09-11T23:28:55.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.089+0000 7f52df033640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f52d8102d40 0x7f52d81009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:55.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.089+0000 7f52df033640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f52d8103660 0x7f52d8100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:55.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.089+0000 7f52df033640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52d810c170 0x7f52d8101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:55.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.089+0000 7f52df033640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f52d8111fb0 con 0x7f52d810c170 2024-09-11T23:28:55.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.089+0000 7f52df033640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f52d8111e30 con 0x7f52d8102d40 2024-09-11T23:28:55.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.089+0000 7f52df033640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f52d8112130 con 0x7f52d8103660 2024-09-11T23:28:55.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.089+0000 7f52dd5d0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52d810c170 0x7f52d8101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:55.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.089+0000 7f52dd5d0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52d810c170 0x7f52d8101420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36382/0 (socket says 172.21.15.26:36382) 2024-09-11T23:28:55.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.089+0000 7f52dd5d0640 1 -- 172.21.15.26:0/4114285627 learned_addr learned my addr 172.21.15.26:0/4114285627 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:28:55.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.089+0000 7f52dcdcf640 1 --2- 172.21.15.26:0/4114285627 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f52d8102d40 0x7f52d81009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:55.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.089+0000 7f52d7fff640 1 --2- 172.21.15.26:0/4114285627 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f52d8103660 0x7f52d8100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:55.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.089+0000 7f52dd5d0640 1 -- 172.21.15.26:0/4114285627 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f52d8103660 msgr2=0x7f52d8100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:55.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.089+0000 7f52dd5d0640 1 --2- 172.21.15.26:0/4114285627 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f52d8103660 0x7f52d8100ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:55.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.089+0000 7f52dd5d0640 1 -- 172.21.15.26:0/4114285627 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f52d8102d40 msgr2=0x7f52d81009a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:55.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.089+0000 7f52dd5d0640 1 --2- 172.21.15.26:0/4114285627 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f52d8102d40 0x7f52d81009a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:55.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.090+0000 7f52dd5d0640 1 -- 172.21.15.26:0/4114285627 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52d81a68d0 con 0x7f52d810c170 2024-09-11T23:28:55.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.090+0000 7f52dd5d0640 1 --2- 172.21.15.26:0/4114285627 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52d810c170 0x7f52d8101420 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f52d0009fd0 tx=0x7f52d000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:28:55.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.090+0000 7f52d5ffb640 1 -- 172.21.15.26:0/4114285627 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52d0011830 con 0x7f52d810c170 2024-09-11T23:28:55.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.090+0000 7f52d5ffb640 1 -- 172.21.15.26:0/4114285627 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f52d0016070 con 0x7f52d810c170 2024-09-11T23:28:55.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.090+0000 7f52d5ffb640 1 -- 172.21.15.26:0/4114285627 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52d000f2e0 con 0x7f52d810c170 2024-09-11T23:28:55.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.090+0000 7f52df033640 1 -- 172.21.15.26:0/4114285627 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52d81a6bc0 con 0x7f52d810c170 2024-09-11T23:28:55.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.090+0000 7f52df033640 1 -- 172.21.15.26:0/4114285627 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52d81a7100 con 0x7f52d810c170 2024-09-11T23:28:55.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.092+0000 7f52d5ffb640 1 -- 172.21.15.26:0/4114285627 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f52d00119d0 con 0x7f52d810c170 2024-09-11T23:28:55.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.093+0000 7f52e140c740 1 Processor -- start 2024-09-11T23:28:55.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.093+0000 7f52e140c740 1 -- start start 2024-09-11T23:28:55.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.093+0000 7f52e140c740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55af2adbe850 0x55af2adbec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:55.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.093+0000 7f52e140c740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55af2adbf170 0x55af2adc94f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:55.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.093+0000 7f52e140c740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55af2adc9a30 0x55af2adcbe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:55.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.093+0000 7f52e140c740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55af2ad23160 con 0x55af2adbe850 2024-09-11T23:28:55.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.093+0000 7f52e140c740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55af2acca720 con 0x55af2adc9a30 2024-09-11T23:28:55.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.093+0000 7f52e140c740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55af2ad1f120 con 0x55af2adbf170 2024-09-11T23:28:55.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.093+0000 7f52d5ffb640 1 --2- 172.21.15.26:0/4114285627 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f52c00a2d70 0x7f52c00a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:55.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.093+0000 7f52d7fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55af2adbe850 0x55af2adbec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:55.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.093+0000 7f52d7fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55af2adbe850 0x55af2adbec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36398/0 (socket says 172.21.15.26:36398) 2024-09-11T23:28:55.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.093+0000 7f52d7fff640 1 -- 172.21.15.26:0/2583392309 learned_addr learned my addr 172.21.15.26:0/2583392309 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:28:55.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.093+0000 7f52dd5d0640 1 --2- 172.21.15.26:0/4114285627 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f52c00a2d70 0x7f52c00a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:55.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.093+0000 7f52d7fff640 1 --2- 172.21.15.26:0/2583392309 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55af2adc9a30 0x55af2adcbe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:55.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.093+0000 7f52dcdcf640 1 --2- 172.21.15.26:0/2583392309 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55af2adbf170 0x55af2adc94f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:55.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.093+0000 7f52d7fff640 1 -- 172.21.15.26:0/2583392309 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55af2adbf170 msgr2=0x55af2adc94f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:28:55.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.093+0000 7f52d5ffb640 1 -- 172.21.15.26:0/4114285627 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8542+0+0 (secure 0 0 0) 0x7f52d00ce9e0 con 0x7f52d810c170 2024-09-11T23:28:55.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.093+0000 7f52d7fff640 1 --2- 172.21.15.26:0/2583392309 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55af2adbf170 0x55af2adc94f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:55.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.093+0000 7f52d7fff640 1 -- 172.21.15.26:0/2583392309 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55af2adc9a30 msgr2=0x55af2adcbe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:28:55.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.093+0000 7f52d7fff640 1 --2- 172.21.15.26:0/2583392309 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55af2adc9a30 0x55af2adcbe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:55.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.093+0000 7f52d7fff640 1 -- 172.21.15.26:0/2583392309 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af2aa94ba0 con 0x55af2adbe850 2024-09-11T23:28:55.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.094+0000 7f52d7fff640 1 --2- 172.21.15.26:0/2583392309 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55af2adbe850 0x55af2adbec30 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f52cc0096c0 tx=0x7f52cc005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45d15a664fc7a1b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:28:55.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.094+0000 7f52dd5d0640 1 --2- 172.21.15.26:0/4114285627 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f52c00a2d70 0x7f52c00a5230 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f52d0097400 tx=0x7f52d0010400 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:28:55.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.094+0000 7f52beffd640 1 -- 172.21.15.26:0/2583392309 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52cc035070 con 0x55af2adbe850 2024-09-11T23:28:55.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.094+0000 7f52beffd640 1 -- 172.21.15.26:0/2583392309 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f52cc031a00 con 0x55af2adbe850 2024-09-11T23:28:55.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.094+0000 7f52beffd640 1 -- 172.21.15.26:0/2583392309 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52cc0049b0 con 0x55af2adbe850 2024-09-11T23:28:55.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.095+0000 7f52e140c740 1 -- 172.21.15.26:0/2583392309 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55af2adbe850 msgr2=0x55af2adbec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:55.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.095+0000 7f52e140c740 1 --2- 172.21.15.26:0/2583392309 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55af2adbe850 0x55af2adbec30 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f52cc0096c0 tx=0x7f52cc005a20 comp rx=0 tx=0).stop 2024-09-11T23:28:55.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.095+0000 7f52e140c740 1 -- 172.21.15.26:0/2583392309 shutdown_connections 2024-09-11T23:28:55.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.095+0000 7f52e140c740 1 --2- 172.21.15.26:0/2583392309 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55af2adc9a30 0x55af2adcbe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:55.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.095+0000 7f52e140c740 1 --2- 172.21.15.26:0/2583392309 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55af2adbf170 0x55af2adc94f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:55.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.095+0000 7f52e140c740 1 --2- 172.21.15.26:0/2583392309 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55af2adbe850 0x55af2adbec30 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:55.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.095+0000 7f52e140c740 1 -- 172.21.15.26:0/2583392309 >> 172.21.15.26:0/2583392309 conn(0x55af2adbade0 msgr2=0x55af2adbb1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:28:55.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.095+0000 7f52e140c740 1 -- 172.21.15.26:0/2583392309 shutdown_connections 2024-09-11T23:28:55.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.095+0000 7f52e140c740 1 -- 172.21.15.26:0/2583392309 wait complete. 2024-09-11T23:28:55.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.095+0000 7f52e140c740 1 -- start start 2024-09-11T23:28:55.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.095+0000 7f52e140c740 1 Processor -- start 2024-09-11T23:28:55.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.096+0000 7f52e140c740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55af2adbe850 0x55af2adc69e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:55.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.096+0000 7f52e140c740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55af2adbf170 0x55af2adc6f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:55.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.096+0000 7f52e140c740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55af2adc9a30 0x55af2adc7460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:55.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.096+0000 7f52e140c740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55af2adcc600 con 0x55af2adbe850 2024-09-11T23:28:55.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.096+0000 7f52e140c740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55af2acab900 con 0x55af2adbf170 2024-09-11T23:28:55.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.096+0000 7f52e140c740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55af2adcc780 con 0x55af2adc9a30 2024-09-11T23:28:55.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.096+0000 7f52e140c740 10 client.0 _ll_register_callbacks cb 0x55af2aca9780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:28:55.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.096+0000 7f52d7fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55af2adbe850 0x55af2adc69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:55.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.096+0000 7f52d7fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55af2adbe850 0x55af2adc69e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36412/0 (socket says 172.21.15.26:36412) 2024-09-11T23:28:55.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.096+0000 7f52d7fff640 1 -- 172.21.15.26:0/3048656536 learned_addr learned my addr 172.21.15.26:0/3048656536 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:28:55.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.096+0000 7f52dcdcf640 1 --2- 172.21.15.26:0/3048656536 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55af2adbf170 0x55af2adc6f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:55.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.096+0000 7f52d7fff640 1 --2- 172.21.15.26:0/3048656536 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55af2adc9a30 0x55af2adc7460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:55.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.096+0000 7f52d7fff640 1 -- 172.21.15.26:0/3048656536 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55af2adc9a30 msgr2=0x55af2adc7460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:28:55.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.096+0000 7f52d7fff640 1 --2- 172.21.15.26:0/3048656536 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55af2adc9a30 0x55af2adc7460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:55.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.096+0000 7f52d7fff640 1 -- 172.21.15.26:0/3048656536 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55af2adbf170 msgr2=0x55af2adc6f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:28:55.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.096+0000 7f52d7fff640 1 --2- 172.21.15.26:0/3048656536 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55af2adbf170 0x55af2adc6f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:55.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.097+0000 7f52d7fff640 1 -- 172.21.15.26:0/3048656536 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af2acebc50 con 0x55af2adbe850 2024-09-11T23:28:55.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.097+0000 7f52dcdcf640 1 --2- 172.21.15.26:0/3048656536 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55af2adbf170 0x55af2adc6f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:28:55.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.097+0000 7f52d7fff640 1 --2- 172.21.15.26:0/3048656536 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55af2adbe850 0x55af2adc69e0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f52cc02a9a0 tx=0x7f52cc005750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e044ba257f22b785 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:28:55.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.097+0000 7f52b7fff640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:28:55.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.097+0000 7f52b7fff640 1 -- 172.21.15.26:0/3048656536 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52cc035070 con 0x55af2adbe850 2024-09-11T23:28:55.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.097+0000 7f52b7fff640 1 -- 172.21.15.26:0/3048656536 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8542+0+0 (secure 0 0 0) 0x7f52cc003a20 con 0x55af2adbe850 2024-09-11T23:28:55.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.097+0000 7f52e140c740 1 -- 172.21.15.26:0/3048656536 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f52cc031a00 con 0x55af2adbe850 2024-09-11T23:28:55.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.097+0000 7f52b7fff640 1 -- 172.21.15.26:0/3048656536 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f52cc054050 con 0x55af2adbe850 2024-09-11T23:28:55.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.097+0000 7f52e140c740 10 client.17007 fetch_fsmap learned FSMap version 55 2024-09-11T23:28:55.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.097+0000 7f52e140c740 1 -- 172.21.15.26:0/3048656536 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x55af2ade0b80 con 0x55af2adbe850 2024-09-11T23:28:55.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.097+0000 7f52b7fff640 1 -- 172.21.15.26:0/3048656536 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f52cc031a00 con 0x55af2adbe850 2024-09-11T23:28:55.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.098+0000 7f52b7fff640 1 -- 172.21.15.26:0/3048656536 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 55) ==== 8374+0+0 (secure 0 0 0) 0x7f52cc04b070 con 0x55af2adbe850 2024-09-11T23:28:55.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.098+0000 7f52e140c740 10 client.17007 fetch_fsmap finished waiting for FSMap version 55 2024-09-11T23:28:55.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.098+0000 7f52e140c740 10 client.17007 resolve_mds: validated gid 24485 aka daemon mds.b 2024-09-11T23:28:55.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.098+0000 7f52e140c740 20 client.17007 populate_metadata read hostname 'smithi026' 2024-09-11T23:28:55.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.098+0000 7f52e140c740 1 --2- 172.21.15.26:0/3048656536 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55af2ade1ed0 0x55af2ade42c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:55.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.098+0000 7f52e140c740 4 client.17007 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:28:55.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.098+0000 7f52e140c740 1 -- 172.21.15.26:0/3048656536 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55af2ad30890 con 0x55af2ade1ed0 2024-09-11T23:28:55.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.098+0000 7f52dcdcf640 1 --2- 172.21.15.26:0/3048656536 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55af2ade1ed0 0x55af2ade42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:55.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.098+0000 7f52b7fff640 10 client.17007 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:28:55.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.098+0000 7f52dcdcf640 1 --2- 172.21.15.26:0/3048656536 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55af2ade1ed0 0x55af2ade42c0 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d43ea2e56779f10a server_cookie=8db02d7b9c35b514 in_seq=0 out_seq=0 2024-09-11T23:28:55.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.099+0000 7f52b7fff640 1 -- 172.21.15.26:0/3048656536 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55af2ad30890 con 0x55af2ade1ed0 2024-09-11T23:28:55.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.099+0000 7f52b7fff640 10 client.17007 handle_command_reply: tid=0 2024-09-11T23:28:55.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.099+0000 7f52e140c740 1 client.17007 shutdown 2024-09-11T23:28:55.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.099+0000 7f52e140c740 1 -- 172.21.15.26:0/3048656536 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55af2adbe850 msgr2=0x55af2adc69e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:55.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.099+0000 7f52e140c740 1 --2- 172.21.15.26:0/3048656536 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55af2adbe850 0x55af2adc69e0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f52cc02a9a0 tx=0x7f52cc005750 comp rx=0 tx=0).stop 2024-09-11T23:28:55.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.099+0000 7f52e140c740 1 -- 172.21.15.26:0/3048656536 shutdown_connections 2024-09-11T23:28:55.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.099+0000 7f52e140c740 1 --2- 172.21.15.26:0/3048656536 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55af2adbf170 0x55af2adc6f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:55.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.099+0000 7f52e140c740 1 --2- 172.21.15.26:0/3048656536 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55af2ade1ed0 0x55af2ade42c0 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:55.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.099+0000 7f52e140c740 1 --2- 172.21.15.26:0/3048656536 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55af2adc9a30 0x55af2adc7460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:55.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.099+0000 7f52e140c740 1 --2- 172.21.15.26:0/3048656536 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55af2adbe850 0x55af2adc69e0 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:55.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.099+0000 7f52e140c740 1 -- 172.21.15.26:0/3048656536 >> 172.21.15.26:0/3048656536 conn(0x55af2adbade0 msgr2=0x55af2adc0b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:28:55.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.099+0000 7f52b7fff640 0 client.17007 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:28:55.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.100+0000 7f52e140c740 1 -- 172.21.15.26:0/3048656536 shutdown_connections 2024-09-11T23:28:55.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.100+0000 7f52e140c740 1 -- 172.21.15.26:0/3048656536 wait complete. 2024-09-11T23:28:55.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.100+0000 7f52e140c740 20 client.17007 trim_cache size 0 max 16384 2024-09-11T23:28:55.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.117+0000 7f52e140c740 1 Processor -- start 2024-09-11T23:28:55.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.117+0000 7f52e140c740 1 -- start start 2024-09-11T23:28:55.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.117+0000 7f52e140c740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55af2adbe850 0x55af2adc5890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:55.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.117+0000 7f52e140c740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55af2adbf170 0x55af2ade72e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:55.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.118+0000 7f52e140c740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55af2adc9a30 0x55af2ade7820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:55.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.118+0000 7f52e140c740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f52d81ab6f0 con 0x55af2adc9a30 2024-09-11T23:28:55.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.118+0000 7f52e140c740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f52d81ad2c0 con 0x55af2adbf170 2024-09-11T23:28:55.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.118+0000 7f52e140c740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f52d81abdc0 con 0x55af2adbe850 2024-09-11T23:28:55.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.118+0000 7f52d7fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55af2adc9a30 0x55af2ade7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:55.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.118+0000 7f52d7fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55af2adc9a30 0x55af2ade7820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36414/0 (socket says 172.21.15.26:36414) 2024-09-11T23:28:55.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.118+0000 7f52d7fff640 1 -- 172.21.15.26:0/1888621577 learned_addr learned my addr 172.21.15.26:0/1888621577 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:28:55.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.118+0000 7f52dcdcf640 1 --2- 172.21.15.26:0/1888621577 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55af2adbf170 0x55af2ade72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:55.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.118+0000 7f52d7fff640 1 --2- 172.21.15.26:0/1888621577 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55af2adbe850 0x55af2adc5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:55.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.118+0000 7f52d7fff640 1 -- 172.21.15.26:0/1888621577 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55af2adbe850 msgr2=0x55af2adc5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:28:55.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.118+0000 7f52d7fff640 1 --2- 172.21.15.26:0/1888621577 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55af2adbe850 0x55af2adc5890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:55.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.118+0000 7f52d7fff640 1 -- 172.21.15.26:0/1888621577 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55af2adbf170 msgr2=0x55af2ade72e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:28:55.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.118+0000 7f52d7fff640 1 --2- 172.21.15.26:0/1888621577 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55af2adbf170 0x55af2ade72e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:55.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.118+0000 7f52d7fff640 1 -- 172.21.15.26:0/1888621577 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af2adc1800 con 0x55af2adc9a30 2024-09-11T23:28:55.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.118+0000 7f52d7fff640 1 --2- 172.21.15.26:0/1888621577 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55af2adc9a30 0x55af2ade7820 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f52cc005ab0 tx=0x7f52cc005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e571adc17ed5c7c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:28:55.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.118+0000 7f52be7fc640 1 -- 172.21.15.26:0/1888621577 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52cc035070 con 0x55af2adc9a30 2024-09-11T23:28:55.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.119+0000 7f52be7fc640 1 -- 172.21.15.26:0/1888621577 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f52cc042040 con 0x55af2adc9a30 2024-09-11T23:28:55.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.119+0000 7f52be7fc640 1 -- 172.21.15.26:0/1888621577 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52cc045040 con 0x55af2adc9a30 2024-09-11T23:28:55.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.119+0000 7f52e140c740 1 -- 172.21.15.26:0/1888621577 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55af2adc9a30 msgr2=0x55af2ade7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:55.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.119+0000 7f52e140c740 1 --2- 172.21.15.26:0/1888621577 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55af2adc9a30 0x55af2ade7820 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f52cc005ab0 tx=0x7f52cc005fd0 comp rx=0 tx=0).stop 2024-09-11T23:28:55.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.119+0000 7f52e140c740 1 -- 172.21.15.26:0/1888621577 shutdown_connections 2024-09-11T23:28:55.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.119+0000 7f52e140c740 1 --2- 172.21.15.26:0/1888621577 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55af2adbf170 0x55af2ade72e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:55.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.119+0000 7f52e140c740 1 --2- 172.21.15.26:0/1888621577 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55af2adc9a30 0x55af2ade7820 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:55.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.119+0000 7f52e140c740 1 --2- 172.21.15.26:0/1888621577 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55af2adbe850 0x55af2adc5890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:55.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.119+0000 7f52e140c740 1 -- 172.21.15.26:0/1888621577 >> 172.21.15.26:0/1888621577 conn(0x55af2adbade0 msgr2=0x55af2adc5180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:28:55.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.119+0000 7f52e140c740 1 -- 172.21.15.26:0/1888621577 shutdown_connections 2024-09-11T23:28:55.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.119+0000 7f52e140c740 1 -- 172.21.15.26:0/1888621577 wait complete. 2024-09-11T23:28:55.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.120+0000 7f52e140c740 1 -- start start 2024-09-11T23:28:55.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.120+0000 7f52e140c740 1 Processor -- start 2024-09-11T23:28:55.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.120+0000 7f52e140c740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55af2adbe850 0x55af2ade7a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:55.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.120+0000 7f52e140c740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55af2adbf170 0x55af2ade4de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:55.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.120+0000 7f52e140c740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55af2adc9a30 0x55af2ade5320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:55.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.120+0000 7f52e140c740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f52d81acbf0 con 0x55af2adbf170 2024-09-11T23:28:55.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.120+0000 7f52e140c740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f52d81ac4f0 con 0x55af2adbe850 2024-09-11T23:28:55.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.120+0000 7f52e140c740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f52d81a9710 con 0x55af2adc9a30 2024-09-11T23:28:55.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.120+0000 7f52e140c740 10 client.0 _ll_register_callbacks cb 0x7f52cc009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:28:55.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.120+0000 7f52dcdcf640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55af2adbf170 0x55af2ade4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:55.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.121+0000 7f52d7fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55af2adbe850 0x55af2ade7a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:55.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.121+0000 7f52d7fff640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55af2adc9a30 0x55af2ade5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:55.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.121+0000 7f52dcdcf640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55af2adbf170 0x55af2ade4de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36426/0 (socket says 172.21.15.26:36426) 2024-09-11T23:28:55.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.121+0000 7f52dcdcf640 1 -- 172.21.15.26:0/1845755596 learned_addr learned my addr 172.21.15.26:0/1845755596 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:28:55.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.121+0000 7f52d7fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55af2adbe850 0x55af2ade7a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:55684/0 (socket says 172.21.15.26:55684) 2024-09-11T23:28:55.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.121+0000 7f52dcdcf640 1 -- 172.21.15.26:0/1845755596 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55af2adc9a30 msgr2=0x55af2ade5320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:28:55.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.121+0000 7f52dcdcf640 1 --2- 172.21.15.26:0/1845755596 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55af2adc9a30 0x55af2ade5320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:55.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.121+0000 7f52dcdcf640 1 -- 172.21.15.26:0/1845755596 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55af2adbe850 msgr2=0x55af2ade7a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:28:55.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.121+0000 7f52dcdcf640 1 --2- 172.21.15.26:0/1845755596 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55af2adbe850 0x55af2ade7a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:55.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.121+0000 7f52dcdcf640 1 -- 172.21.15.26:0/1845755596 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af2ade5b50 con 0x55af2adbf170 2024-09-11T23:28:55.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.121+0000 7f52d7fff640 1 --2- 172.21.15.26:0/1845755596 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55af2adbe850 0x55af2ade7a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:28:55.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.121+0000 7f52dcdcf640 1 --2- 172.21.15.26:0/1845755596 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55af2adbf170 0x55af2ade4de0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f52c8007f20 tx=0x7f52c8012900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4006d7ba0288d9d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:28:55.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.121+0000 7f52b7fff640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:28:55.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.121+0000 7f52b7fff640 1 -- 172.21.15.26:0/1845755596 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52c8022030 con 0x55af2adbf170 2024-09-11T23:28:55.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.121+0000 7f52b7fff640 1 -- 172.21.15.26:0/1845755596 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8542+0+0 (secure 0 0 0) 0x7f52c801e660 con 0x55af2adbf170 2024-09-11T23:28:55.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.122+0000 7f52e140c740 1 -- 172.21.15.26:0/1845755596 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f52cc042040 con 0x55af2adbf170 2024-09-11T23:28:55.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.122+0000 7f52b7fff640 1 -- 172.21.15.26:0/1845755596 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f52c801e860 con 0x55af2adbf170 2024-09-11T23:28:55.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.122+0000 7f52e140c740 10 client.17013 fetch_fsmap learned FSMap version 55 2024-09-11T23:28:55.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.122+0000 7f52e140c740 1 -- 172.21.15.26:0/1845755596 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x55af2ae4c8b0 con 0x55af2adbf170 2024-09-11T23:28:55.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.122+0000 7f52b7fff640 1 -- 172.21.15.26:0/1845755596 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f52cc042040 con 0x55af2adbf170 2024-09-11T23:28:55.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.122+0000 7f52b7fff640 1 -- 172.21.15.26:0/1845755596 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 55) ==== 8374+0+0 (secure 0 0 0) 0x7f52c8030c60 con 0x55af2adbf170 2024-09-11T23:28:55.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.122+0000 7f52e140c740 10 client.17013 fetch_fsmap finished waiting for FSMap version 55 2024-09-11T23:28:55.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.122+0000 7f52e140c740 10 client.17013 resolve_mds: validated gid 24485 aka daemon mds.b 2024-09-11T23:28:55.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.122+0000 7f52e140c740 20 client.17013 populate_metadata read hostname 'smithi026' 2024-09-11T23:28:55.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.122+0000 7f52e140c740 1 --2- 172.21.15.26:0/1845755596 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55af2ae4d0a0 0x55af2ae4f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:55.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.122+0000 7f52e140c740 4 client.17013 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.55, "await_for": 45.0}] 2024-09-11T23:28:55.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.122+0000 7f52e140c740 1 -- 172.21.15.26:0/1845755596 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.55, "await_for": 45.0}) -- 0x55af2ad2abe0 con 0x55af2ae4d0a0 2024-09-11T23:28:55.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.123+0000 7f52d7fff640 1 --2- 172.21.15.26:0/1845755596 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55af2ae4d0a0 0x55af2ae4f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:55.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.123+0000 7f52d7fff640 1 --2- 172.21.15.26:0/1845755596 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55af2ae4d0a0 0x55af2ae4f4c0 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=225428e5fd422aff server_cookie=49c057fabe9ff028 in_seq=0 out_seq=0 2024-09-11T23:28:55.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:28:55.123+0000 7f52b7fff640 10 client.17013 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:28:55.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:28:54 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:28:55.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:28:54 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:28:55.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:28:54 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:28:55.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:28:54 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:28:55.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:28:54 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:28:55.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:28:54 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:28:55.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:28:54 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:28:55.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:28:54 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:28:55.240 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:28:54 smithi136 ceph-mon[35121]: Adjusting osd_memory_target on smithi076 to 176.0M 2024-09-11T23:28:55.240 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:28:54 smithi136 ceph-mon[35121]: Adjusting osd_memory_target on smithi136 to 176.0M 2024-09-11T23:28:55.240 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:28:54 smithi136 ceph-mon[35121]: Unable to set osd_memory_target on smithi076 to 184591052: error parsing value: Value '184591052' is below minimum 939524096 2024-09-11T23:28:55.240 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:28:54 smithi136 ceph-mon[35121]: Unable to set osd_memory_target on smithi136 to 184574566: error parsing value: Value '184574566' is below minimum 939524096 2024-09-11T23:28:55.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:28:54 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:28:55.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:28:54 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:28:55.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:28:54 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:28:55.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:28:54 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:28:55.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:28:54 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:28:55.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:28:54 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:28:55.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:28:54 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:28:55.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:28:54 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:28:55.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:28:54 smithi076 ceph-mon[35296]: Adjusting osd_memory_target on smithi076 to 176.0M 2024-09-11T23:28:55.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:28:54 smithi076 ceph-mon[35296]: Adjusting osd_memory_target on smithi136 to 176.0M 2024-09-11T23:28:55.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:28:54 smithi076 ceph-mon[35296]: Unable to set osd_memory_target on smithi076 to 184591052: error parsing value: Value '184591052' is below minimum 939524096 2024-09-11T23:28:55.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:28:54 smithi076 ceph-mon[35296]: Unable to set osd_memory_target on smithi136 to 184574566: error parsing value: Value '184574566' is below minimum 939524096 2024-09-11T23:28:56.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:28:55 smithi136 ceph-mon[35121]: pgmap v327: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 66 KiB/s wr, 16 op/s 2024-09-11T23:28:56.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:28:55 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:28:56.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:28:55 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:28:56.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:28:55 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:28:56.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:28:55 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:28:56.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:28:55 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:28:56.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:28:55 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:28:56.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:28:55 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:28:56.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:28:55 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:28:56.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:28:55 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:28:56.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:28:55 smithi026 ceph-mon[31172]: pgmap v327: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 66 KiB/s wr, 16 op/s 2024-09-11T23:28:56.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:28:55 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:28:56.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:28:55 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:28:56.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:28:55 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:28:56.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:28:55 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:28:56.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:28:55 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:28:56.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:28:55 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:28:56.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:28:55 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:28:56.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:28:55 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:28:56.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:28:55 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:28:56.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:28:55 smithi076 ceph-mon[35296]: pgmap v327: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 66 KiB/s wr, 16 op/s 2024-09-11T23:28:56.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:28:55 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:28:56.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:28:55 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:28:56.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:28:55 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:28:56.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:28:55 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:28:56.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:28:55 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:28:56.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:28:55 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:28:56.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:28:55 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:28:56.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:28:55 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:28:56.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:28:55 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:28:57.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:28:56 smithi136 ceph-mon[35121]: Adjusting osd_memory_target on smithi026 to 177.0M 2024-09-11T23:28:57.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:28:56 smithi136 ceph-mon[35121]: Unable to set osd_memory_target on smithi026 to 185689190: error parsing value: Value '185689190' is below minimum 939524096 2024-09-11T23:28:57.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:28:56 smithi026 ceph-mon[31172]: Adjusting osd_memory_target on smithi026 to 177.0M 2024-09-11T23:28:57.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:28:56 smithi026 ceph-mon[31172]: Unable to set osd_memory_target on smithi026 to 185689190: error parsing value: Value '185689190' is below minimum 939524096 2024-09-11T23:28:57.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:28:56 smithi076 ceph-mon[35296]: Adjusting osd_memory_target on smithi026 to 177.0M 2024-09-11T23:28:57.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:28:56 smithi076 ceph-mon[35296]: Unable to set osd_memory_target on smithi026 to 185689190: error parsing value: Value '185689190' is below minimum 939524096 2024-09-11T23:28:58.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:28:57 smithi136 ceph-mon[35121]: pgmap v328: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 38 KiB/s wr, 14 op/s 2024-09-11T23:28:58.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:28:57 smithi136 ceph-mon[35121]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:28:58.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:28:57 smithi026 ceph-mon[31172]: pgmap v328: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 38 KiB/s wr, 14 op/s 2024-09-11T23:28:58.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:28:57 smithi026 ceph-mon[31172]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:28:58.305 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:28:57 smithi076 ceph-mon[35296]: pgmap v328: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 38 KiB/s wr, 14 op/s 2024-09-11T23:28:58.305 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:28:57 smithi076 ceph-mon[35296]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:29:00.238 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:28:59 smithi136 ceph-mon[35121]: pgmap v329: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 38 KiB/s wr, 12 op/s 2024-09-11T23:29:00.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:28:59 smithi026 ceph-mon[31172]: pgmap v329: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 38 KiB/s wr, 12 op/s 2024-09-11T23:29:00.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:28:59 smithi076 ceph-mon[35296]: pgmap v329: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 38 KiB/s wr, 12 op/s 2024-09-11T23:29:02.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:29:01 smithi136 ceph-mon[35121]: pgmap v330: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 42 KiB/s wr, 18 op/s 2024-09-11T23:29:02.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:29:01 smithi026 ceph-mon[31172]: pgmap v330: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 42 KiB/s wr, 18 op/s 2024-09-11T23:29:02.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:29:01 smithi076 ceph-mon[35296]: pgmap v330: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 42 KiB/s wr, 18 op/s 2024-09-11T23:29:03.040 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-11T23:29:03.040 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 56, 2024-09-11T23:29:03.040 INFO:teuthology.orchestra.run.smithi026.stdout: "leader": 24485, 2024-09-11T23:29:03.040 INFO:teuthology.orchestra.run.smithi026.stdout: "set_version": 38, 2024-09-11T23:29:03.040 INFO:teuthology.orchestra.run.smithi026.stdout: "sets": { 2024-09-11T23:29:03.040 INFO:teuthology.orchestra.run.smithi026.stdout: "57f3de60": { 2024-09-11T23:29:03.040 INFO:teuthology.orchestra.run.smithi026.stdout: "version": 38, 2024-09-11T23:29:03.041 INFO:teuthology.orchestra.run.smithi026.stdout: "age_ref": 0.0, 2024-09-11T23:29:03.041 INFO:teuthology.orchestra.run.smithi026.stdout: "state": { 2024-09-11T23:29:03.041 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "QUIESCED", 2024-09-11T23:29:03.041 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.0 2024-09-11T23:29:03.041 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:29:03.041 INFO:teuthology.orchestra.run.smithi026.stdout: "timeout": 90.0, 2024-09-11T23:29:03.041 INFO:teuthology.orchestra.run.smithi026.stdout: "expiration": 142.6, 2024-09-11T23:29:03.041 INFO:teuthology.orchestra.run.smithi026.stdout: "members": { 2024-09-11T23:29:03.041 INFO:teuthology.orchestra.run.smithi026.stdout: "file:/": { 2024-09-11T23:29:03.041 INFO:teuthology.orchestra.run.smithi026.stdout: "excluded": false, 2024-09-11T23:29:03.041 INFO:teuthology.orchestra.run.smithi026.stdout: "state": { 2024-09-11T23:29:03.041 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "QUIESCED", 2024-09-11T23:29:03.041 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.0 2024-09-11T23:29:03.042 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:29:03.042 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:29:03.042 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:29:03.042 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:29:03.042 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:29:03.042 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-11T23:29:03.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:03.038+0000 7f52b7fff640 1 -- 172.21.15.26:0/1845755596 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55af2ad2abe0 con 0x55af2ae4d0a0 2024-09-11T23:29:03.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:03.038+0000 7f52b7fff640 10 client.17013 handle_command_reply: tid=0 2024-09-11T23:29:03.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:03.038+0000 7f52e140c740 1 client.17013 shutdown 2024-09-11T23:29:03.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:03.038+0000 7f52e140c740 1 -- 172.21.15.26:0/1845755596 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55af2adbf170 msgr2=0x55af2ade4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:03.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:03.038+0000 7f52e140c740 1 --2- 172.21.15.26:0/1845755596 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55af2adbf170 0x55af2ade4de0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f52c8007f20 tx=0x7f52c8012900 comp rx=0 tx=0).stop 2024-09-11T23:29:03.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:03.038+0000 7f52e140c740 1 -- 172.21.15.26:0/1845755596 shutdown_connections 2024-09-11T23:29:03.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:03.038+0000 7f52e140c740 1 --2- 172.21.15.26:0/1845755596 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55af2ae4d0a0 0x55af2ae4f4c0 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:03.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:03.038+0000 7f52e140c740 1 --2- 172.21.15.26:0/1845755596 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55af2adc9a30 0x55af2ade5320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:03.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:03.038+0000 7f52e140c740 1 --2- 172.21.15.26:0/1845755596 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55af2adbf170 0x55af2ade4de0 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:03.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:03.038+0000 7f52e140c740 1 --2- 172.21.15.26:0/1845755596 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55af2adbe850 0x55af2ade7a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:03.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:03.038+0000 7f52b7fff640 0 client.17013 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:29:03.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:03.038+0000 7f52e140c740 1 -- 172.21.15.26:0/1845755596 >> 172.21.15.26:0/1845755596 conn(0x55af2adbade0 msgr2=0x55af2adb9ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:29:03.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:03.038+0000 7f52e140c740 1 -- 172.21.15.26:0/1845755596 shutdown_connections 2024-09-11T23:29:03.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:03.038+0000 7f52e140c740 1 -- 172.21.15.26:0/1845755596 wait complete. 2024-09-11T23:29:03.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:03.039+0000 7f52e140c740 20 client.17013 trim_cache size 0 max 16384 2024-09-11T23:29:03.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:03.039+0000 7f52df033640 1 -- 172.21.15.26:0/4114285627 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f52c00a2d70 msgr2=0x7f52c00a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:03.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:03.039+0000 7f52df033640 1 --2- 172.21.15.26:0/4114285627 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f52c00a2d70 0x7f52c00a5230 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f52d0097400 tx=0x7f52d0010400 comp rx=0 tx=0).stop 2024-09-11T23:29:03.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:03.039+0000 7f52df033640 1 -- 172.21.15.26:0/4114285627 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52d810c170 msgr2=0x7f52d8101420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:03.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:03.039+0000 7f52df033640 1 --2- 172.21.15.26:0/4114285627 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52d810c170 0x7f52d8101420 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f52d0009fd0 tx=0x7f52d000b8e0 comp rx=0 tx=0).stop 2024-09-11T23:29:03.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:03.039+0000 7f52df033640 1 -- 172.21.15.26:0/4114285627 shutdown_connections 2024-09-11T23:29:03.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:03.039+0000 7f52df033640 1 --2- 172.21.15.26:0/4114285627 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f52c00a2d70 0x7f52c00a5230 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:03.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:03.039+0000 7f52df033640 1 --2- 172.21.15.26:0/4114285627 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52d810c170 0x7f52d8101420 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:03.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:03.039+0000 7f52df033640 1 --2- 172.21.15.26:0/4114285627 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f52d8103660 0x7f52d8100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:03.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:03.039+0000 7f52df033640 1 --2- 172.21.15.26:0/4114285627 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f52d8102d40 0x7f52d81009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:03.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:03.039+0000 7f52df033640 1 -- 172.21.15.26:0/4114285627 >> 172.21.15.26:0/4114285627 conn(0x7f52d80fc7a0 msgr2=0x7f52d80fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:29:03.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:03.040+0000 7f52df033640 1 -- 172.21.15.26:0/4114285627 shutdown_connections 2024-09-11T23:29:03.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:03.040+0000 7f52df033640 1 -- 172.21.15.26:0/4114285627 wait complete. 2024-09-11T23:29:03.766 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '57f3de60', quiesce took 11.8 seconds. Will release after: 10.75 2024-09-11T23:29:04.000 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:29:03 smithi026 ceph-mon[31172]: pgmap v331: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 39 KiB/s wr, 13 op/s 2024-09-11T23:29:04.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:29:03 smithi136 ceph-mon[35121]: pgmap v331: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 39 KiB/s wr, 13 op/s 2024-09-11T23:29:04.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:29:03 smithi076 ceph-mon[35296]: pgmap v331: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 39 KiB/s wr, 13 op/s 2024-09-11T23:29:05.023 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.022+0000 7fe60ed5f640 1 -- 172.21.15.26:0/2751002016 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe608101390 msgr2=0x7fe60810d8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:05.024 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.022+0000 7fe60ed5f640 1 --2- 172.21.15.26:0/2751002016 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe608101390 0x7fe60810d8c0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fe600009cd0 tx=0x7fe60002a8e0 comp rx=0 tx=0).stop 2024-09-11T23:29:05.024 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.022+0000 7fe60ed5f640 1 -- 172.21.15.26:0/2751002016 shutdown_connections 2024-09-11T23:29:05.024 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.022+0000 7fe60ed5f640 1 --2- 172.21.15.26:0/2751002016 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe608101390 0x7fe60810d8c0 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:05.024 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.022+0000 7fe60ed5f640 1 --2- 172.21.15.26:0/2751002016 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe6081009f0 0x7fe608100e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:05.024 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.022+0000 7fe60ed5f640 1 --2- 172.21.15.26:0/2751002016 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe608106a00 0x7fe608106de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:05.024 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.022+0000 7fe60ed5f640 1 -- 172.21.15.26:0/2751002016 >> 172.21.15.26:0/2751002016 conn(0x7fe6080fc6b0 msgr2=0x7fe6080fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:29:05.024 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.022+0000 7fe60ed5f640 1 -- 172.21.15.26:0/2751002016 shutdown_connections 2024-09-11T23:29:05.024 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.022+0000 7fe60ed5f640 1 -- 172.21.15.26:0/2751002016 wait complete. 2024-09-11T23:29:05.026 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.023+0000 7fe60ed5f640 1 Processor -- start 2024-09-11T23:29:05.026 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.023+0000 7fe60ed5f640 1 -- start start 2024-09-11T23:29:05.026 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.023+0000 7fe60ed5f640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe6081009f0 0x7fe608198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:05.027 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.023+0000 7fe60ed5f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe608101390 0x7fe6081988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:05.027 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.023+0000 7fe60ed5f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe608106a00 0x7fe60819cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:05.027 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.023+0000 7fe60ed5f640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fe608110e40 con 0x7fe608106a00 2024-09-11T23:29:05.027 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.023+0000 7fe60ed5f640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fe608110cc0 con 0x7fe608101390 2024-09-11T23:29:05.027 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.023+0000 7fe60ed5f640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe608110fc0 con 0x7fe6081009f0 2024-09-11T23:29:05.027 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.024+0000 7fe60d2fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe608106a00 0x7fe60819cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:05.027 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.024+0000 7fe60cafb640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe6081009f0 0x7fe608198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:05.027 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.024+0000 7fe60d2fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe608106a00 0x7fe60819cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40786/0 (socket says 172.21.15.26:40786) 2024-09-11T23:29:05.027 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.024+0000 7fe60d2fc640 1 -- 172.21.15.26:0/793771599 learned_addr learned my addr 172.21.15.26:0/793771599 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:29:05.027 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.024+0000 7fe5fffff640 1 --2- 172.21.15.26:0/793771599 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe608101390 0x7fe6081988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:05.027 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.024+0000 7fe60d2fc640 1 -- 172.21.15.26:0/793771599 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe6081009f0 msgr2=0x7fe608198360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:05.027 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.024+0000 7fe60d2fc640 1 --2- 172.21.15.26:0/793771599 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe6081009f0 0x7fe608198360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:05.027 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.024+0000 7fe60d2fc640 1 -- 172.21.15.26:0/793771599 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe608101390 msgr2=0x7fe6081988a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:05.028 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.024+0000 7fe60d2fc640 1 --2- 172.21.15.26:0/793771599 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe608101390 0x7fe6081988a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:05.028 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.024+0000 7fe60d2fc640 1 -- 172.21.15.26:0/793771599 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe60819d350 con 0x7fe608106a00 2024-09-11T23:29:05.028 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.024+0000 7fe60d2fc640 1 --2- 172.21.15.26:0/793771599 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe608106a00 0x7fe60819cc20 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fe600005cc0 tx=0x7fe600003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:29:05.028 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.024+0000 7fe5fdffb640 1 -- 172.21.15.26:0/793771599 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe600004040 con 0x7fe608106a00 2024-09-11T23:29:05.028 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.024+0000 7fe5fdffb640 1 -- 172.21.15.26:0/793771599 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe6000046c0 con 0x7fe608106a00 2024-09-11T23:29:05.028 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.024+0000 7fe60ed5f640 1 -- 172.21.15.26:0/793771599 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe60819d5e0 con 0x7fe608106a00 2024-09-11T23:29:05.028 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.024+0000 7fe5fdffb640 1 -- 172.21.15.26:0/793771599 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe600046480 con 0x7fe608106a00 2024-09-11T23:29:05.028 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.025+0000 7fe60ed5f640 1 -- 172.21.15.26:0/793771599 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6081a4ed0 con 0x7fe608106a00 2024-09-11T23:29:05.035 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.027+0000 7fe5fdffb640 1 -- 172.21.15.26:0/793771599 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fe6000305a0 con 0x7fe608106a00 2024-09-11T23:29:05.035 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.027+0000 7fe611bd4740 1 Processor -- start 2024-09-11T23:29:05.035 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.027+0000 7fe611bd4740 1 -- start start 2024-09-11T23:29:05.036 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.028+0000 7fe611bd4740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5638300ed9d0 0x5638300eddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:05.036 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.028+0000 7fe611bd4740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5638300ee2f0 0x5638300fcb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:05.036 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.028+0000 7fe611bd4740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5638300fd0c0 0x5638300ff4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:05.036 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.028+0000 7fe611bd4740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5638300522f0 con 0x5638300ed9d0 2024-09-11T23:29:05.036 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.028+0000 7fe611bd4740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x56382fffa270 con 0x5638300fd0c0 2024-09-11T23:29:05.036 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.028+0000 7fe611bd4740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x56383004e2b0 con 0x5638300ee2f0 2024-09-11T23:29:05.036 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.028+0000 7fe5fffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5638300ed9d0 0x5638300eddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:05.037 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.028+0000 7fe5fffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5638300ed9d0 0x5638300eddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40790/0 (socket says 172.21.15.26:40790) 2024-09-11T23:29:05.037 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.028+0000 7fe5fffff640 1 -- 172.21.15.26:0/2990763605 learned_addr learned my addr 172.21.15.26:0/2990763605 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:29:05.037 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.028+0000 7fe60cafb640 1 --2- 172.21.15.26:0/2990763605 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5638300ee2f0 0x5638300fcb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:05.037 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.028+0000 7fe5fdffb640 1 --2- 172.21.15.26:0/793771599 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fe5f00a2da0 0x7fe5f00a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:05.037 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.028+0000 7fe5fffff640 1 --2- 172.21.15.26:0/2990763605 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5638300fd0c0 0x5638300ff4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:05.037 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.028+0000 7fe60d2fc640 1 --2- 172.21.15.26:0/793771599 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fe5f00a2da0 0x7fe5f00a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:05.037 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.028+0000 7fe5fffff640 1 -- 172.21.15.26:0/2990763605 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5638300ee2f0 msgr2=0x5638300fcb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:29:05.037 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.028+0000 7fe5fffff640 1 --2- 172.21.15.26:0/2990763605 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5638300ee2f0 0x5638300fcb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:05.037 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.028+0000 7fe5fffff640 1 -- 172.21.15.26:0/2990763605 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5638300fd0c0 msgr2=0x5638300ff4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:29:05.037 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.028+0000 7fe5fffff640 1 --2- 172.21.15.26:0/2990763605 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5638300fd0c0 0x5638300ff4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:05.037 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.028+0000 7fe5fffff640 1 -- 172.21.15.26:0/2990763605 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56382fff9340 con 0x5638300ed9d0 2024-09-11T23:29:05.037 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.028+0000 7fe5fdffb640 1 -- 172.21.15.26:0/793771599 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8542+0+0 (secure 0 0 0) 0x7fe60002e030 con 0x7fe608106a00 2024-09-11T23:29:05.037 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.028+0000 7fe5fffff640 1 --2- 172.21.15.26:0/2990763605 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5638300ed9d0 0x5638300eddb0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fe5f800cfc0 tx=0x7fe5f800c5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e9024c4587cf85e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:29:05.038 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.029+0000 7fe5eeffd640 1 -- 172.21.15.26:0/2990763605 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5f8018030 con 0x5638300ed9d0 2024-09-11T23:29:05.038 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.029+0000 7fe5eeffd640 1 -- 172.21.15.26:0/2990763605 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5f8005500 con 0x5638300ed9d0 2024-09-11T23:29:05.038 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.029+0000 7fe5eeffd640 1 -- 172.21.15.26:0/2990763605 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5f800d750 con 0x5638300ed9d0 2024-09-11T23:29:05.038 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.029+0000 7fe611bd4740 1 -- 172.21.15.26:0/2990763605 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5638300ed9d0 msgr2=0x5638300eddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:05.038 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.029+0000 7fe611bd4740 1 --2- 172.21.15.26:0/2990763605 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5638300ed9d0 0x5638300eddb0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fe5f800cfc0 tx=0x7fe5f800c5a0 comp rx=0 tx=0).stop 2024-09-11T23:29:05.038 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.029+0000 7fe60d2fc640 1 --2- 172.21.15.26:0/793771599 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fe5f00a2da0 0x7fe5f00a5260 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe6000bd120 tx=0x7fe6000dd040 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:29:05.038 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.029+0000 7fe611bd4740 1 -- 172.21.15.26:0/2990763605 shutdown_connections 2024-09-11T23:29:05.038 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.029+0000 7fe611bd4740 1 --2- 172.21.15.26:0/2990763605 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5638300fd0c0 0x5638300ff4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:05.038 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.029+0000 7fe611bd4740 1 --2- 172.21.15.26:0/2990763605 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5638300ee2f0 0x5638300fcb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:05.038 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.029+0000 7fe611bd4740 1 --2- 172.21.15.26:0/2990763605 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5638300ed9d0 0x5638300eddb0 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:05.038 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.029+0000 7fe611bd4740 1 -- 172.21.15.26:0/2990763605 >> 172.21.15.26:0/2990763605 conn(0x5638300ea060 msgr2=0x5638300ea440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:29:05.038 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.029+0000 7fe611bd4740 1 -- 172.21.15.26:0/2990763605 shutdown_connections 2024-09-11T23:29:05.039 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.029+0000 7fe611bd4740 1 -- 172.21.15.26:0/2990763605 wait complete. 2024-09-11T23:29:05.039 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.030+0000 7fe611bd4740 1 -- start start 2024-09-11T23:29:05.039 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.030+0000 7fe611bd4740 1 Processor -- start 2024-09-11T23:29:05.039 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.030+0000 7fe611bd4740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5638300ed9d0 0x5638300f9fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:05.039 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.030+0000 7fe611bd4740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5638300ee2f0 0x5638300fa4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:05.039 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.030+0000 7fe611bd4740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5638300fd0c0 0x5638300faa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:05.039 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.030+0000 7fe611bd4740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5638300ffcf0 con 0x5638300ed9d0 2024-09-11T23:29:05.039 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.030+0000 7fe611bd4740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x56382ffdb290 con 0x5638300ee2f0 2024-09-11T23:29:05.039 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.030+0000 7fe611bd4740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5638300ffe70 con 0x5638300fd0c0 2024-09-11T23:29:05.039 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.030+0000 7fe611bd4740 10 client.0 _ll_register_callbacks cb 0x56383000c140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:29:05.039 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.031+0000 7fe5fffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5638300ed9d0 0x5638300f9fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:05.039 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.031+0000 7fe5fffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5638300ed9d0 0x5638300f9fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40792/0 (socket says 172.21.15.26:40792) 2024-09-11T23:29:05.039 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.031+0000 7fe5fffff640 1 -- 172.21.15.26:0/3981420612 learned_addr learned my addr 172.21.15.26:0/3981420612 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:29:05.040 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.031+0000 7fe60cafb640 1 --2- 172.21.15.26:0/3981420612 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5638300ee2f0 0x5638300fa4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:05.040 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.031+0000 7fe5fffff640 1 --2- 172.21.15.26:0/3981420612 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5638300fd0c0 0x5638300faa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:05.040 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.031+0000 7fe5fffff640 1 -- 172.21.15.26:0/3981420612 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5638300fd0c0 msgr2=0x5638300faa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:29:05.040 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.031+0000 7fe5fffff640 1 --2- 172.21.15.26:0/3981420612 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5638300fd0c0 0x5638300faa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:05.040 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.031+0000 7fe5fffff640 1 -- 172.21.15.26:0/3981420612 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5638300ee2f0 msgr2=0x5638300fa4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:29:05.040 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.031+0000 7fe5fffff640 1 --2- 172.21.15.26:0/3981420612 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5638300ee2f0 0x5638300fa4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:05.040 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.031+0000 7fe5fffff640 1 -- 172.21.15.26:0/3981420612 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56382fd65210 con 0x5638300ed9d0 2024-09-11T23:29:05.040 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.031+0000 7fe5fffff640 1 --2- 172.21.15.26:0/3981420612 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5638300ed9d0 0x5638300f9fb0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fe5f8004d90 tx=0x7fe5f8005cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba3dcce5f5b9b8b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:29:05.040 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.031+0000 7fe5e7fff640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:29:05.040 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.031+0000 7fe5e7fff640 1 -- 172.21.15.26:0/3981420612 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5f8018040 con 0x5638300ed9d0 2024-09-11T23:29:05.040 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.031+0000 7fe5e7fff640 1 -- 172.21.15.26:0/3981420612 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8542+0+0 (secure 0 0 0) 0x7fe5f800d550 con 0x5638300ed9d0 2024-09-11T23:29:05.040 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.031+0000 7fe611bd4740 1 -- 172.21.15.26:0/3981420612 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe5f8005500 con 0x5638300ed9d0 2024-09-11T23:29:05.041 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.031+0000 7fe5e7fff640 1 -- 172.21.15.26:0/3981420612 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) ==== 24+0+0 (secure 0 0 0) 0x7fe5f8030050 con 0x5638300ed9d0 2024-09-11T23:29:05.041 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.031+0000 7fe611bd4740 10 client.17025 fetch_fsmap learned FSMap version 56 2024-09-11T23:29:05.041 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.031+0000 7fe611bd4740 1 -- 172.21.15.26:0/3981420612 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) -- 0x56382fe31620 con 0x5638300ed9d0 2024-09-11T23:29:05.041 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.032+0000 7fe5e7fff640 1 -- 172.21.15.26:0/3981420612 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5f8005500 con 0x5638300ed9d0 2024-09-11T23:29:05.041 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.032+0000 7fe5e7fff640 1 -- 172.21.15.26:0/3981420612 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 56) ==== 8370+0+0 (secure 0 0 0) 0x7fe5f8029060 con 0x5638300ed9d0 2024-09-11T23:29:05.041 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.032+0000 7fe611bd4740 10 client.17025 fetch_fsmap finished waiting for FSMap version 56 2024-09-11T23:29:05.041 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.032+0000 7fe611bd4740 10 client.17025 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:29:05.041 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.032+0000 7fe611bd4740 20 client.17025 populate_metadata read hostname 'smithi026' 2024-09-11T23:29:05.041 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.032+0000 7fe611bd4740 1 --2- 172.21.15.26:0/3981420612 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x56383010d040 0x56383010f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:05.041 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.032+0000 7fe611bd4740 4 client.17025 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:29:05.041 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.032+0000 7fe611bd4740 1 -- 172.21.15.26:0/3981420612 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56383005fa20 con 0x56383010d040 2024-09-11T23:29:05.041 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.032+0000 7fe60cafb640 1 --2- 172.21.15.26:0/3981420612 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x56383010d040 0x56383010f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:05.041 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.032+0000 7fe5e7fff640 10 client.17025 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:29:05.041 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.032+0000 7fe60cafb640 1 --2- 172.21.15.26:0/3981420612 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x56383010d040 0x56383010f430 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d6e1b4521c87829b server_cookie=8c61d19722893630 in_seq=0 out_seq=0 2024-09-11T23:29:05.042 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.033+0000 7fe5e7fff640 1 -- 172.21.15.26:0/3981420612 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56383005fa20 con 0x56383010d040 2024-09-11T23:29:05.042 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.033+0000 7fe5e7fff640 10 client.17025 handle_command_reply: tid=0 2024-09-11T23:29:05.042 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.033+0000 7fe611bd4740 1 client.17025 shutdown 2024-09-11T23:29:05.042 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.033+0000 7fe611bd4740 1 -- 172.21.15.26:0/3981420612 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5638300ed9d0 msgr2=0x5638300f9fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:05.042 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.033+0000 7fe611bd4740 1 --2- 172.21.15.26:0/3981420612 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5638300ed9d0 0x5638300f9fb0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fe5f8004d90 tx=0x7fe5f8005cc0 comp rx=0 tx=0).stop 2024-09-11T23:29:05.042 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.033+0000 7fe611bd4740 1 -- 172.21.15.26:0/3981420612 shutdown_connections 2024-09-11T23:29:05.042 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.033+0000 7fe611bd4740 1 --2- 172.21.15.26:0/3981420612 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5638300ee2f0 0x5638300fa4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:05.042 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.033+0000 7fe611bd4740 1 --2- 172.21.15.26:0/3981420612 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x56383010d040 0x56383010f430 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:05.042 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.033+0000 7fe611bd4740 1 --2- 172.21.15.26:0/3981420612 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5638300fd0c0 0x5638300faa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:05.042 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.033+0000 7fe611bd4740 1 --2- 172.21.15.26:0/3981420612 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5638300ed9d0 0x5638300f9fb0 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:05.042 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.033+0000 7fe5e7fff640 0 client.17025 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:29:05.042 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.033+0000 7fe611bd4740 1 -- 172.21.15.26:0/3981420612 >> 172.21.15.26:0/3981420612 conn(0x5638300ea060 msgr2=0x5638300efdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:29:05.042 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.034+0000 7fe611bd4740 1 -- 172.21.15.26:0/3981420612 shutdown_connections 2024-09-11T23:29:05.043 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.034+0000 7fe611bd4740 1 -- 172.21.15.26:0/3981420612 wait complete. 2024-09-11T23:29:05.043 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.034+0000 7fe611bd4740 20 client.17025 trim_cache size 0 max 16384 2024-09-11T23:29:05.051 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.048+0000 7fe611bd4740 1 Processor -- start 2024-09-11T23:29:05.051 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.048+0000 7fe611bd4740 1 -- start start 2024-09-11T23:29:05.051 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.048+0000 7fe611bd4740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5638300ed9d0 0x563830114bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:05.051 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.049+0000 7fe611bd4740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5638300ee2f0 0x563830115100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:05.051 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.049+0000 7fe611bd4740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5638300fd0c0 0x5638300f7210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:05.051 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.049+0000 7fe611bd4740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fe6081ab590 con 0x5638300ed9d0 2024-09-11T23:29:05.052 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.049+0000 7fe611bd4740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fe6081ad130 con 0x5638300ee2f0 2024-09-11T23:29:05.052 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.049+0000 7fe611bd4740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe6081abc60 con 0x5638300fd0c0 2024-09-11T23:29:05.052 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.049+0000 7fe5fffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5638300ed9d0 0x563830114bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:05.052 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.049+0000 7fe5fffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5638300ed9d0 0x563830114bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40798/0 (socket says 172.21.15.26:40798) 2024-09-11T23:29:05.052 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.049+0000 7fe5fffff640 1 -- 172.21.15.26:0/1225501205 learned_addr learned my addr 172.21.15.26:0/1225501205 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:29:05.052 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.049+0000 7fe60cafb640 1 --2- 172.21.15.26:0/1225501205 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5638300ee2f0 0x563830115100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:05.052 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.049+0000 7fe5fffff640 1 --2- 172.21.15.26:0/1225501205 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5638300fd0c0 0x5638300f7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:05.052 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.049+0000 7fe5fffff640 1 -- 172.21.15.26:0/1225501205 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5638300fd0c0 msgr2=0x5638300f7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:29:05.052 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.049+0000 7fe5fffff640 1 --2- 172.21.15.26:0/1225501205 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5638300fd0c0 0x5638300f7210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:05.052 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.049+0000 7fe5fffff640 1 -- 172.21.15.26:0/1225501205 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5638300ee2f0 msgr2=0x563830115100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:29:05.052 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.049+0000 7fe5fffff640 1 --2- 172.21.15.26:0/1225501205 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5638300ee2f0 0x563830115100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:05.052 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.049+0000 7fe5fffff640 1 -- 172.21.15.26:0/1225501205 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5638300e9e90 con 0x5638300ed9d0 2024-09-11T23:29:05.052 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.049+0000 7fe5fffff640 1 --2- 172.21.15.26:0/1225501205 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5638300ed9d0 0x563830114bc0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fe5f800c4a0 tx=0x7fe5f802ba00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f0ad8dc261bf725 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:29:05.053 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.049+0000 7fe5ee7fc640 1 -- 172.21.15.26:0/1225501205 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5f8020040 con 0x5638300ed9d0 2024-09-11T23:29:05.053 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.049+0000 7fe5ee7fc640 1 -- 172.21.15.26:0/1225501205 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5f800c500 con 0x5638300ed9d0 2024-09-11T23:29:05.053 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.049+0000 7fe5ee7fc640 1 -- 172.21.15.26:0/1225501205 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5f8018040 con 0x5638300ed9d0 2024-09-11T23:29:05.056 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.050+0000 7fe611bd4740 1 -- 172.21.15.26:0/1225501205 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5638300ed9d0 msgr2=0x563830114bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:05.056 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.050+0000 7fe611bd4740 1 --2- 172.21.15.26:0/1225501205 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5638300ed9d0 0x563830114bc0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fe5f800c4a0 tx=0x7fe5f802ba00 comp rx=0 tx=0).stop 2024-09-11T23:29:05.056 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.050+0000 7fe611bd4740 1 -- 172.21.15.26:0/1225501205 shutdown_connections 2024-09-11T23:29:05.056 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.050+0000 7fe611bd4740 1 --2- 172.21.15.26:0/1225501205 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5638300ee2f0 0x563830115100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:05.056 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.050+0000 7fe611bd4740 1 --2- 172.21.15.26:0/1225501205 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5638300fd0c0 0x5638300f7210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:05.056 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.050+0000 7fe611bd4740 1 --2- 172.21.15.26:0/1225501205 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5638300ed9d0 0x563830114bc0 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:05.056 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.050+0000 7fe611bd4740 1 -- 172.21.15.26:0/1225501205 >> 172.21.15.26:0/1225501205 conn(0x5638300ea060 msgr2=0x5638300fc030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:29:05.056 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.050+0000 7fe611bd4740 1 -- 172.21.15.26:0/1225501205 shutdown_connections 2024-09-11T23:29:05.057 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.050+0000 7fe611bd4740 1 -- 172.21.15.26:0/1225501205 wait complete. 2024-09-11T23:29:05.057 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.050+0000 7fe611bd4740 1 -- start start 2024-09-11T23:29:05.057 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.050+0000 7fe611bd4740 1 Processor -- start 2024-09-11T23:29:05.060 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.051+0000 7fe611bd4740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5638300ed9d0 0x5638300f7080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:05.060 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.051+0000 7fe611bd4740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5638300ee2f0 0x5638300f75c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:05.060 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.051+0000 7fe611bd4740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5638300fd0c0 0x5638300f7b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:05.060 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.051+0000 7fe611bd4740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fe6081aca60 con 0x5638300fd0c0 2024-09-11T23:29:05.060 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.051+0000 7fe611bd4740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fe6081ac360 con 0x5638300ee2f0 2024-09-11T23:29:05.060 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.051+0000 7fe611bd4740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe6081a9590 con 0x5638300ed9d0 2024-09-11T23:29:05.060 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.051+0000 7fe611bd4740 10 client.0 _ll_register_callbacks cb 0x7fe60400b730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:29:05.060 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.051+0000 7fe5fffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5638300fd0c0 0x5638300f7b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:05.060 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.051+0000 7fe5fffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5638300fd0c0 0x5638300f7b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40800/0 (socket says 172.21.15.26:40800) 2024-09-11T23:29:05.061 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.051+0000 7fe5fffff640 1 -- 172.21.15.26:0/67768741 learned_addr learned my addr 172.21.15.26:0/67768741 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:29:05.061 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.051+0000 7fe60cafb640 1 --2- 172.21.15.26:0/67768741 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5638300ee2f0 0x5638300f75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:05.061 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.051+0000 7fe5fffff640 1 --2- 172.21.15.26:0/67768741 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5638300ed9d0 0x5638300f7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:05.061 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.051+0000 7fe5fffff640 1 -- 172.21.15.26:0/67768741 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5638300ed9d0 msgr2=0x5638300f7080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:29:05.061 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.051+0000 7fe5fffff640 1 --2- 172.21.15.26:0/67768741 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5638300ed9d0 0x5638300f7080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:05.061 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.051+0000 7fe5fffff640 1 -- 172.21.15.26:0/67768741 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5638300ee2f0 msgr2=0x5638300f75c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:29:05.061 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.051+0000 7fe5fffff640 1 --2- 172.21.15.26:0/67768741 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5638300ee2f0 0x5638300f75c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:05.061 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.051+0000 7fe5fffff640 1 -- 172.21.15.26:0/67768741 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5638300eacf0 con 0x5638300fd0c0 2024-09-11T23:29:05.061 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.052+0000 7fe5fffff640 1 --2- 172.21.15.26:0/67768741 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5638300fd0c0 0x5638300f7b00 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fe5f8005d40 tx=0x7fe5f8037040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa03670962b860d4 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:29:05.061 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.052+0000 7fe5e7fff640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:29:05.061 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.052+0000 7fe5e7fff640 1 -- 172.21.15.26:0/67768741 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5f8018030 con 0x5638300fd0c0 2024-09-11T23:29:05.061 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.052+0000 7fe5e7fff640 1 -- 172.21.15.26:0/67768741 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8542+0+0 (secure 0 0 0) 0x7fe5f8004d00 con 0x5638300fd0c0 2024-09-11T23:29:05.062 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.052+0000 7fe611bd4740 1 -- 172.21.15.26:0/67768741 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe5f800c500 con 0x5638300fd0c0 2024-09-11T23:29:05.062 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.052+0000 7fe5e7fff640 1 -- 172.21.15.26:0/67768741 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) ==== 24+0+0 (secure 0 0 0) 0x7fe5f8005230 con 0x5638300fd0c0 2024-09-11T23:29:05.062 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.052+0000 7fe611bd4740 10 client.17031 fetch_fsmap learned FSMap version 56 2024-09-11T23:29:05.062 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.052+0000 7fe611bd4740 1 -- 172.21.15.26:0/67768741 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) -- 0x563830177d60 con 0x5638300fd0c0 2024-09-11T23:29:05.062 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.052+0000 7fe5e7fff640 1 -- 172.21.15.26:0/67768741 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5f800c500 con 0x5638300fd0c0 2024-09-11T23:29:05.062 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.052+0000 7fe5e7fff640 1 -- 172.21.15.26:0/67768741 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 56) ==== 8370+0+0 (secure 0 0 0) 0x7fe5f8040030 con 0x5638300fd0c0 2024-09-11T23:29:05.062 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.052+0000 7fe611bd4740 10 client.17031 fetch_fsmap finished waiting for FSMap version 56 2024-09-11T23:29:05.062 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.052+0000 7fe611bd4740 10 client.17031 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:29:05.062 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.052+0000 7fe611bd4740 20 client.17031 populate_metadata read hostname 'smithi026' 2024-09-11T23:29:05.062 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.052+0000 7fe611bd4740 1 --2- 172.21.15.26:0/67768741 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5638301781a0 0x56383017a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:05.062 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.052+0000 7fe611bd4740 4 client.17031 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T23:29:05.062 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.052+0000 7fe611bd4740 1 -- 172.21.15.26:0/67768741 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563830059d70 con 0x5638301781a0 2024-09-11T23:29:05.062 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.053+0000 7fe60cafb640 1 --2- 172.21.15.26:0/67768741 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5638301781a0 0x56383017a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:05.063 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.053+0000 7fe60cafb640 1 --2- 172.21.15.26:0/67768741 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5638301781a0 0x56383017a5c0 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d1f911a1a608250d server_cookie=2742c89505f2e5a4 in_seq=0 out_seq=0 2024-09-11T23:29:05.063 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:05.053+0000 7fe5e7fff640 10 client.17031 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:29:06.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:29:05 smithi136 ceph-mon[35121]: pgmap v332: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 84 KiB/s wr, 33 op/s 2024-09-11T23:29:06.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:29:05 smithi026 ceph-mon[31172]: pgmap v332: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 84 KiB/s wr, 33 op/s 2024-09-11T23:29:06.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:29:05 smithi076 ceph-mon[35296]: pgmap v332: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 84 KiB/s wr, 33 op/s 2024-09-11T23:29:06.602 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:06.601+0000 7fe5e7fff640 1 -- 172.21.15.26:0/67768741 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563830059d70 con 0x5638301781a0 2024-09-11T23:29:06.602 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:06.601+0000 7fe5e7fff640 10 client.17031 handle_command_reply: tid=0 2024-09-11T23:29:06.602 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:06.601+0000 7fe611bd4740 1 client.17031 shutdown 2024-09-11T23:29:06.602 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:06.601+0000 7fe611bd4740 1 -- 172.21.15.26:0/67768741 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5638300fd0c0 msgr2=0x5638300f7b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:06.602 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:06.601+0000 7fe611bd4740 1 --2- 172.21.15.26:0/67768741 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5638300fd0c0 0x5638300f7b00 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fe5f8005d40 tx=0x7fe5f8037040 comp rx=0 tx=0).stop 2024-09-11T23:29:06.602 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:06.601+0000 7fe611bd4740 1 -- 172.21.15.26:0/67768741 shutdown_connections 2024-09-11T23:29:06.602 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:06.601+0000 7fe611bd4740 1 --2- 172.21.15.26:0/67768741 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5638300ee2f0 0x5638300f75c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:06.603 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:06.601+0000 7fe611bd4740 1 --2- 172.21.15.26:0/67768741 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5638301781a0 0x56383017a5c0 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:06.603 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:06.601+0000 7fe611bd4740 1 --2- 172.21.15.26:0/67768741 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5638300fd0c0 0x5638300f7b00 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:06.603 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:06.601+0000 7fe611bd4740 1 --2- 172.21.15.26:0/67768741 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5638300ed9d0 0x5638300f7080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:06.603 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:06.601+0000 7fe5e7fff640 0 client.17031 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:29:06.603 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:06.601+0000 7fe611bd4740 1 -- 172.21.15.26:0/67768741 >> 172.21.15.26:0/67768741 conn(0x5638300ea060 msgr2=0x5638300e89a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:29:06.603 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:06.601+0000 7fe611bd4740 1 -- 172.21.15.26:0/67768741 shutdown_connections 2024-09-11T23:29:06.603 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:06.601+0000 7fe611bd4740 1 -- 172.21.15.26:0/67768741 wait complete. 2024-09-11T23:29:06.603 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:06.601+0000 7fe611bd4740 20 client.17031 trim_cache size 0 max 16384 2024-09-11T23:29:06.603 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-09-11T23:29:06.603 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "message": "", 2024-09-11T23:29:06.603 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-09-11T23:29:06.604 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-09-11T23:29:06.604 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:06.602+0000 7fe60ed5f640 1 -- 172.21.15.26:0/793771599 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fe5f00a2da0 msgr2=0x7fe5f00a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:06.604 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:06.602+0000 7fe60ed5f640 1 --2- 172.21.15.26:0/793771599 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fe5f00a2da0 0x7fe5f00a5260 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe6000bd120 tx=0x7fe6000dd040 comp rx=0 tx=0).stop 2024-09-11T23:29:06.604 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:06.602+0000 7fe60ed5f640 1 -- 172.21.15.26:0/793771599 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe608106a00 msgr2=0x7fe60819cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:06.604 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:06.602+0000 7fe60ed5f640 1 --2- 172.21.15.26:0/793771599 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe608106a00 0x7fe60819cc20 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fe600005cc0 tx=0x7fe600003970 comp rx=0 tx=0).stop 2024-09-11T23:29:06.604 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:06.602+0000 7fe60ed5f640 1 -- 172.21.15.26:0/793771599 shutdown_connections 2024-09-11T23:29:06.605 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:06.602+0000 7fe60ed5f640 1 --2- 172.21.15.26:0/793771599 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fe5f00a2da0 0x7fe5f00a5260 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:06.605 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:06.602+0000 7fe60ed5f640 1 --2- 172.21.15.26:0/793771599 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe608101390 0x7fe6081988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:06.605 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:06.602+0000 7fe60ed5f640 1 --2- 172.21.15.26:0/793771599 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe608106a00 0x7fe60819cc20 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:06.605 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:06.602+0000 7fe60ed5f640 1 --2- 172.21.15.26:0/793771599 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe6081009f0 0x7fe608198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:06.605 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:06.602+0000 7fe60ed5f640 1 -- 172.21.15.26:0/793771599 >> 172.21.15.26:0/793771599 conn(0x7fe6080fc6b0 msgr2=0x7fe60806a4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:29:06.605 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:06.602+0000 7fe60ed5f640 1 -- 172.21.15.26:0/793771599 shutdown_connections 2024-09-11T23:29:06.605 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:06.602+0000 7fe60ed5f640 1 -- 172.21.15.26:0/793771599 wait complete. 2024-09-11T23:29:08.238 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:29:07 smithi136 ceph-mon[35121]: pgmap v333: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 51 KiB/s wr, 30 op/s 2024-09-11T23:29:08.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:29:07 smithi136 ceph-mon[35121]: osdmap e95: 12 total, 12 up, 12 in 2024-09-11T23:29:08.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:29:07 smithi136 ceph-mon[35121]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:29:08.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:29:07 smithi026 ceph-mon[31172]: pgmap v333: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 51 KiB/s wr, 30 op/s 2024-09-11T23:29:08.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:29:07 smithi026 ceph-mon[31172]: osdmap e95: 12 total, 12 up, 12 in 2024-09-11T23:29:08.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:29:07 smithi026 ceph-mon[31172]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:29:08.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:29:07 smithi076 ceph-mon[35296]: pgmap v333: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 51 KiB/s wr, 30 op/s 2024-09-11T23:29:08.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:29:07 smithi076 ceph-mon[35296]: osdmap e95: 12 total, 12 up, 12 in 2024-09-11T23:29:08.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:29:07 smithi076 ceph-mon[35296]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:29:10.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:29:09 smithi136 ceph-mon[35121]: pgmap v335: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 60 KiB/s wr, 33 op/s 2024-09-11T23:29:10.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:29:09 smithi026 ceph-mon[31172]: pgmap v335: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 60 KiB/s wr, 33 op/s 2024-09-11T23:29:10.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:29:09 smithi076 ceph-mon[35296]: pgmap v335: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 60 KiB/s wr, 33 op/s 2024-09-11T23:29:12.238 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:29:11 smithi136 ceph-mon[35121]: pgmap v336: 97 pgs: 13 active+clean+snaptrim, 22 active+clean+snaptrim_wait, 62 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 93 KiB/s wr, 32 op/s 2024-09-11T23:29:12.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:29:11 smithi026 ceph-mon[31172]: pgmap v336: 97 pgs: 13 active+clean+snaptrim, 22 active+clean+snaptrim_wait, 62 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 93 KiB/s wr, 32 op/s 2024-09-11T23:29:12.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:29:11 smithi076 ceph-mon[35296]: pgmap v336: 97 pgs: 13 active+clean+snaptrim, 22 active+clean+snaptrim_wait, 62 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 93 KiB/s wr, 32 op/s 2024-09-11T23:29:14.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:29:13 smithi136 ceph-mon[35121]: pgmap v337: 97 pgs: 13 active+clean+snaptrim, 22 active+clean+snaptrim_wait, 62 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 93 KiB/s wr, 32 op/s 2024-09-11T23:29:14.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:29:13 smithi026 ceph-mon[31172]: pgmap v337: 97 pgs: 13 active+clean+snaptrim, 22 active+clean+snaptrim_wait, 62 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 93 KiB/s wr, 32 op/s 2024-09-11T23:29:14.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:29:13 smithi076 ceph-mon[35296]: pgmap v337: 97 pgs: 13 active+clean+snaptrim, 22 active+clean+snaptrim_wait, 62 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 93 KiB/s wr, 32 op/s 2024-09-11T23:29:14.517 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:29:14.793 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:29:15.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:15.279+0000 7f70dfba6640 1 -- 172.21.15.26:0/4120977379 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70d8101420 msgr2=0x7f70d810d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:15.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:15.279+0000 7f70dfba6640 1 --2- 172.21.15.26:0/4120977379 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70d8101420 0x7f70d810d950 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f70d0009cd0 tx=0x7f70d002a8e0 comp rx=0 tx=0).stop 2024-09-11T23:29:15.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:15.279+0000 7f70dfba6640 1 -- 172.21.15.26:0/4120977379 shutdown_connections 2024-09-11T23:29:15.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:15.279+0000 7f70dfba6640 1 --2- 172.21.15.26:0/4120977379 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f70d8100a80 0x7f70d8100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:15.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:15.279+0000 7f70dfba6640 1 --2- 172.21.15.26:0/4120977379 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70d8101420 0x7f70d810d950 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:15.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:15.279+0000 7f70dfba6640 1 --2- 172.21.15.26:0/4120977379 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f70d8106a80 0x7f70d8106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:15.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:15.279+0000 7f70dfba6640 1 -- 172.21.15.26:0/4120977379 >> 172.21.15.26:0/4120977379 conn(0x7f70d80fc7a0 msgr2=0x7f70d80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:29:15.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:15.279+0000 7f70dfba6640 1 -- 172.21.15.26:0/4120977379 shutdown_connections 2024-09-11T23:29:15.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:15.279+0000 7f70dfba6640 1 -- 172.21.15.26:0/4120977379 wait complete. 2024-09-11T23:29:15.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:15.280+0000 7f70dfba6640 1 Processor -- start 2024-09-11T23:29:15.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:15.280+0000 7f70dfba6640 1 -- start start 2024-09-11T23:29:15.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:15.280+0000 7f70dfba6640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f70d8100a80 0x7f70d819a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:15.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:15.280+0000 7f70dfba6640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f70d8101420 0x7f70d819abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:15.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:15.280+0000 7f70dfba6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70d8106a80 0x7f70d819ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:15.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:15.280+0000 7f70dfba6640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f70d8110e80 con 0x7f70d8106a80 2024-09-11T23:29:15.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:15.280+0000 7f70dfba6640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f70d8110d00 con 0x7f70d8100a80 2024-09-11T23:29:15.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:15.280+0000 7f70dfba6640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f70d8111000 con 0x7f70d8101420 2024-09-11T23:29:15.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:15.280+0000 7f70de143640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70d8106a80 0x7f70d819ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:15.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:15.280+0000 7f70de143640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70d8106a80 0x7f70d819ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58932/0 (socket says 172.21.15.26:58932) 2024-09-11T23:29:15.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:15.280+0000 7f70de143640 1 -- 172.21.15.26:0/2470625588 learned_addr learned my addr 172.21.15.26:0/2470625588 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:29:15.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:15.280+0000 7f70dd942640 1 --2- 172.21.15.26:0/2470625588 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f70d8100a80 0x7f70d819a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:15.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:15.281+0000 7f70dd141640 1 --2- 172.21.15.26:0/2470625588 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f70d8101420 0x7f70d819abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:15.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:15.281+0000 7f70de143640 1 -- 172.21.15.26:0/2470625588 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f70d8101420 msgr2=0x7f70d819abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:15.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:15.281+0000 7f70de143640 1 --2- 172.21.15.26:0/2470625588 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f70d8101420 0x7f70d819abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:15.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:15.281+0000 7f70de143640 1 -- 172.21.15.26:0/2470625588 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f70d8100a80 msgr2=0x7f70d819a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:15.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:15.281+0000 7f70de143640 1 --2- 172.21.15.26:0/2470625588 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f70d8100a80 0x7f70d819a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:15.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:15.281+0000 7f70de143640 1 -- 172.21.15.26:0/2470625588 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70d819f630 con 0x7f70d8106a80 2024-09-11T23:29:15.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:15.281+0000 7f70dd141640 1 --2- 172.21.15.26:0/2470625588 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f70d8101420 0x7f70d819abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:29:15.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:15.281+0000 7f70de143640 1 --2- 172.21.15.26:0/2470625588 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70d8106a80 0x7f70d819ef50 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f70d0005cc0 tx=0x7f70d0003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:29:15.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:15.281+0000 7f70ceffd640 1 -- 172.21.15.26:0/2470625588 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70d0004040 con 0x7f70d8106a80 2024-09-11T23:29:15.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:15.281+0000 7f70ceffd640 1 -- 172.21.15.26:0/2470625588 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70d00046c0 con 0x7f70d8106a80 2024-09-11T23:29:15.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:15.281+0000 7f70ceffd640 1 -- 172.21.15.26:0/2470625588 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70d0046480 con 0x7f70d8106a80 2024-09-11T23:29:15.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:15.281+0000 7f70dfba6640 1 -- 172.21.15.26:0/2470625588 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70d819f8c0 con 0x7f70d8106a80 2024-09-11T23:29:15.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:15.281+0000 7f70dfba6640 1 -- 172.21.15.26:0/2470625588 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70d81a7200 con 0x7f70d8106a80 2024-09-11T23:29:15.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:15.282+0000 7f70dfba6640 1 -- 172.21.15.26:0/2470625588 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70d81021c0 con 0x7f70d8106a80 2024-09-11T23:29:15.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:15.283+0000 7f70ceffd640 1 -- 172.21.15.26:0/2470625588 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f70d00305a0 con 0x7f70d8106a80 2024-09-11T23:29:15.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:15.284+0000 7f70ceffd640 1 --2- 172.21.15.26:0/2470625588 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f70c40a2cf0 0x7f70c40a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:15.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:15.284+0000 7f70dd942640 1 --2- 172.21.15.26:0/2470625588 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f70c40a2cf0 0x7f70c40a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:15.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:15.285+0000 7f70ceffd640 1 -- 172.21.15.26:0/2470625588 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8586+0+0 (secure 0 0 0) 0x7f70d002e030 con 0x7f70d8106a80 2024-09-11T23:29:15.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:15.285+0000 7f70dd942640 1 --2- 172.21.15.26:0/2470625588 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f70c40a2cf0 0x7f70c40a51b0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f70d40058c0 tx=0x7f70d400a2d0 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:29:15.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:15.288+0000 7f70ceffd640 1 -- 172.21.15.26:0/2470625588 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f70d004b060 con 0x7f70d8106a80 2024-09-11T23:29:15.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:15.497+0000 7f70dfba6640 1 -- 172.21.15.26:0/2470625588 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f70d8100ee0 con 0x7f70d8106a80 2024-09-11T23:29:15.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:15.498+0000 7f70ceffd640 1 -- 172.21.15.26:0/2470625588 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) ==== 94+0+11973 (secure 0 0 0) 0x7f70d8100ee0 con 0x7f70d8106a80 2024-09-11T23:29:15.500 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 57 2024-09-11T23:29:15.500 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-11T23:29:15.500 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":57,"btime":"2024-09-11T23:29:06:826223+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14499,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/367659515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":367659515},{"type":"v1","addr":"172.21.15.26:6835","nonce":367659515}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14511,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/2041065299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":2041065299},{"type":"v1","addr":"172.21.15.26:6837","nonce":2041065299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":57,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:16:26.544203+0000","modified":"2024-09-11T23:29:06.658248+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24485,"mds_1":24463,"mds_2":24458,"mds_3":24448,"mds_4":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14550":{"gid":14550,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6839/2233068305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":2233068305},{"type":"v1","addr":"172.21.15.26:6839","nonce":2233068305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14562":{"gid":14562,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6841/2984982336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":2984982336},{"type":"v1","addr":"172.21.15.26:6841","nonce":2984982336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6833/2019841455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":2019841455},{"type":"v1","addr":"172.21.15.136:6833","nonce":2019841455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.76:6833/3438147050","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3438147050},{"type":"v1","addr":"172.21.15.76:6833","nonce":3438147050}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6835/3617665975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3617665975},{"type":"v1","addr":"172.21.15.136:6835","nonce":3617665975}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":12,"addr":"172.21.15.76:6835/393308499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":393308499},{"type":"v1","addr":"172.21.15.76:6835","nonce":393308499}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":10,"addr":"172.21.15.136:6837/1633082387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":1633082387},{"type":"v1","addr":"172.21.15.136:6837","nonce":1633082387}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.76:6837/2078978748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":2078978748},{"type":"v1","addr":"172.21.15.76:6837","nonce":2078978748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":63,"addr":"172.21.15.136:6839/3211398630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":3211398630},{"type":"v1","addr":"172.21.15.136:6839","nonce":3211398630}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.76:6839/2077637090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6838","nonce":2077637090},{"type":"v1","addr":"172.21.15.76:6839","nonce":2077637090}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24485,"qdb_cluster":[24443,24448,24458,24463,24485]},"id":1}]} 2024-09-11T23:29:15.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:15.500+0000 7f70dfba6640 1 -- 172.21.15.26:0/2470625588 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f70c40a2cf0 msgr2=0x7f70c40a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:15.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:15.500+0000 7f70dfba6640 1 --2- 172.21.15.26:0/2470625588 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f70c40a2cf0 0x7f70c40a51b0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f70d40058c0 tx=0x7f70d400a2d0 comp rx=0 tx=0).stop 2024-09-11T23:29:15.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:15.500+0000 7f70dfba6640 1 -- 172.21.15.26:0/2470625588 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70d8106a80 msgr2=0x7f70d819ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:15.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:15.500+0000 7f70dfba6640 1 --2- 172.21.15.26:0/2470625588 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70d8106a80 0x7f70d819ef50 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f70d0005cc0 tx=0x7f70d0003970 comp rx=0 tx=0).stop 2024-09-11T23:29:15.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:15.501+0000 7f70dfba6640 1 -- 172.21.15.26:0/2470625588 shutdown_connections 2024-09-11T23:29:15.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:15.501+0000 7f70dfba6640 1 --2- 172.21.15.26:0/2470625588 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f70c40a2cf0 0x7f70c40a51b0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:15.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:15.501+0000 7f70dfba6640 1 --2- 172.21.15.26:0/2470625588 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70d8106a80 0x7f70d819ef50 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:15.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:15.501+0000 7f70dfba6640 1 --2- 172.21.15.26:0/2470625588 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f70d8101420 0x7f70d819abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:15.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:15.501+0000 7f70dfba6640 1 --2- 172.21.15.26:0/2470625588 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f70d8100a80 0x7f70d819a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:15.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:15.501+0000 7f70dfba6640 1 -- 172.21.15.26:0/2470625588 >> 172.21.15.26:0/2470625588 conn(0x7f70d80fc7a0 msgr2=0x7f70d806a7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:29:15.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:15.501+0000 7f70dfba6640 1 -- 172.21.15.26:0/2470625588 shutdown_connections 2024-09-11T23:29:15.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:15.501+0000 7f70dfba6640 1 -- 172.21.15.26:0/2470625588 wait complete. 2024-09-11T23:29:16.134 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:29:15 smithi026 ceph-mon[31172]: pgmap v338: 97 pgs: 1 active+clean+snaptrim, 96 active+clean; 2.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 39 KiB/s wr, 12 op/s 2024-09-11T23:29:16.134 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:29:15 smithi026 ceph-mon[31172]: from='client.? 172.21.15.26:0/2470625588' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:29:16.164 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24485 quiesce db --set-id 57f3de60 --release --await' 2024-09-11T23:29:16.165 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24485 quiesce db --set-id 57f3de60 --release --await 2024-09-11T23:29:16.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:29:15 smithi136 ceph-mon[35121]: pgmap v338: 97 pgs: 1 active+clean+snaptrim, 96 active+clean; 2.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 39 KiB/s wr, 12 op/s 2024-09-11T23:29:16.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:29:15 smithi136 ceph-mon[35121]: from='client.? 172.21.15.26:0/2470625588' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:29:16.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:29:15 smithi076 ceph-mon[35296]: pgmap v338: 97 pgs: 1 active+clean+snaptrim, 96 active+clean; 2.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 39 KiB/s wr, 12 op/s 2024-09-11T23:29:16.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:29:15 smithi076 ceph-mon[35296]: from='client.? 172.21.15.26:0/2470625588' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:29:16.425 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:29:16.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.877+0000 7f75895b9640 1 -- 172.21.15.26:0/2292282272 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7584106f10 msgr2=0x7f758410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:16.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.877+0000 7f75895b9640 1 --2- 172.21.15.26:0/2292282272 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7584106f10 0x7f758410ea90 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f757c009ca0 tx=0x7f757c02a8c0 comp rx=0 tx=0).stop 2024-09-11T23:29:16.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.877+0000 7f75895b9640 1 -- 172.21.15.26:0/2292282272 shutdown_connections 2024-09-11T23:29:16.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.877+0000 7f75895b9640 1 --2- 172.21.15.26:0/2292282272 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7584102790 0x7f75841067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:16.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.877+0000 7f75895b9640 1 --2- 172.21.15.26:0/2292282272 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7584106f10 0x7f758410ea90 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:16.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.877+0000 7f75895b9640 1 --2- 172.21.15.26:0/2292282272 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7584101de0 0x7f75841021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:16.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.877+0000 7f75895b9640 1 -- 172.21.15.26:0/2292282272 >> 172.21.15.26:0/2292282272 conn(0x7f75840fd8c0 msgr2=0x7f75840ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:29:16.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.877+0000 7f75895b9640 1 -- 172.21.15.26:0/2292282272 shutdown_connections 2024-09-11T23:29:16.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.877+0000 7f75895b9640 1 -- 172.21.15.26:0/2292282272 wait complete. 2024-09-11T23:29:16.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.877+0000 7f75895b9640 1 Processor -- start 2024-09-11T23:29:16.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.878+0000 7f75895b9640 1 -- start start 2024-09-11T23:29:16.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.878+0000 7f75895b9640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7584101de0 0x7f7584198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:16.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.878+0000 7f75895b9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7584102790 0x7f75841988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:16.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.878+0000 7f75895b9640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7584106f10 0x7f758419cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:16.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.878+0000 7f75895b9640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f7584111f70 con 0x7f7584102790 2024-09-11T23:29:16.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.878+0000 7f75895b9640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f7584111df0 con 0x7f7584101de0 2024-09-11T23:29:16.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.878+0000 7f75895b9640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f75841120f0 con 0x7f7584106f10 2024-09-11T23:29:16.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.878+0000 7f75827fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7584102790 0x7f75841988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:16.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.878+0000 7f75827fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7584102790 0x7f75841988d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58956/0 (socket says 172.21.15.26:58956) 2024-09-11T23:29:16.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.878+0000 7f75827fc640 1 -- 172.21.15.26:0/349805472 learned_addr learned my addr 172.21.15.26:0/349805472 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:29:16.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.878+0000 7f7582ffd640 1 --2- 172.21.15.26:0/349805472 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7584101de0 0x7f7584198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:16.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.878+0000 7f75837fe640 1 --2- 172.21.15.26:0/349805472 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7584106f10 0x7f758419cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:16.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.878+0000 7f75827fc640 1 -- 172.21.15.26:0/349805472 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7584106f10 msgr2=0x7f758419cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:16.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.878+0000 7f75827fc640 1 --2- 172.21.15.26:0/349805472 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7584106f10 0x7f758419cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:16.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.878+0000 7f75827fc640 1 -- 172.21.15.26:0/349805472 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7584101de0 msgr2=0x7f7584198390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:16.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.878+0000 7f75827fc640 1 --2- 172.21.15.26:0/349805472 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7584101de0 0x7f7584198390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:16.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.878+0000 7f75827fc640 1 -- 172.21.15.26:0/349805472 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f758419d340 con 0x7f7584102790 2024-09-11T23:29:16.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.879+0000 7f75827fc640 1 --2- 172.21.15.26:0/349805472 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7584102790 0x7f75841988d0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f757800d950 tx=0x7f757800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:29:16.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.879+0000 7f756bfff640 1 -- 172.21.15.26:0/349805472 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7578017070 con 0x7f7584102790 2024-09-11T23:29:16.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.879+0000 7f75895b9640 1 -- 172.21.15.26:0/349805472 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f758419d630 con 0x7f7584102790 2024-09-11T23:29:16.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.879+0000 7f756bfff640 1 -- 172.21.15.26:0/349805472 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7578009c30 con 0x7f7584102790 2024-09-11T23:29:16.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.879+0000 7f756bfff640 1 -- 172.21.15.26:0/349805472 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f757800f6b0 con 0x7f7584102790 2024-09-11T23:29:16.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.879+0000 7f75895b9640 1 -- 172.21.15.26:0/349805472 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75841a4f10 con 0x7f7584102790 2024-09-11T23:29:16.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.881+0000 7f756bfff640 1 -- 172.21.15.26:0/349805472 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f75780109d0 con 0x7f7584102790 2024-09-11T23:29:16.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.882+0000 7f758b992740 1 Processor -- start 2024-09-11T23:29:16.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.882+0000 7f758b992740 1 -- start start 2024-09-11T23:29:16.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.882+0000 7f758b992740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555f3d4f2060 0x555f3d4f2440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:16.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.882+0000 7f758b992740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555f3d4f2980 0x555f3d4fcd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:16.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.882+0000 7f758b992740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555f3d4fd240 0x555f3d4ff630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:16.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.882+0000 7f758b992740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x555f3d457620 con 0x555f3d4fd240 2024-09-11T23:29:16.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.882+0000 7f758b992740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x555f3d3fee50 con 0x555f3d4f2980 2024-09-11T23:29:16.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.882+0000 7f758b992740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x555f3d4535e0 con 0x555f3d4f2060 2024-09-11T23:29:16.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.882+0000 7f756bfff640 1 --2- 172.21.15.26:0/349805472 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f756c0a2cf0 0x7f756c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:16.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.882+0000 7f75827fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555f3d4fd240 0x555f3d4ff630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:16.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.882+0000 7f75837fe640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555f3d4f2060 0x555f3d4f2440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:16.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.882+0000 7f75837fe640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555f3d4f2060 0x555f3d4f2440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:53610/0 (socket says 172.21.15.26:53610) 2024-09-11T23:29:16.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.882+0000 7f75837fe640 1 -- 172.21.15.26:0/2730389467 learned_addr learned my addr 172.21.15.26:0/2730389467 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:29:16.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.882+0000 7f75837fe640 1 --2- 172.21.15.26:0/349805472 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f756c0a2cf0 0x7f756c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:16.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.882+0000 7f7582ffd640 1 --2- 172.21.15.26:0/2730389467 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555f3d4f2980 0x555f3d4fcd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:16.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.883+0000 7f756bfff640 1 -- 172.21.15.26:0/349805472 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8586+0+0 (secure 0 0 0) 0x7f75780cf0f0 con 0x7f7584102790 2024-09-11T23:29:16.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.883+0000 7f75827fc640 1 -- 172.21.15.26:0/2730389467 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555f3d4f2060 msgr2=0x555f3d4f2440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:29:16.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.883+0000 7f75827fc640 1 --2- 172.21.15.26:0/2730389467 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555f3d4f2060 0x555f3d4f2440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:16.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.883+0000 7f75827fc640 1 -- 172.21.15.26:0/2730389467 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555f3d4f2980 msgr2=0x555f3d4fcd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:29:16.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.883+0000 7f75827fc640 1 --2- 172.21.15.26:0/2730389467 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555f3d4f2980 0x555f3d4fcd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:16.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.883+0000 7f75827fc640 1 -- 172.21.15.26:0/2730389467 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f3d3efd40 con 0x555f3d4fd240 2024-09-11T23:29:16.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.883+0000 7f75827fc640 1 --2- 172.21.15.26:0/2730389467 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555f3d4fd240 0x555f3d4ff630 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f75780cc7a0 tx=0x7f757806e5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=270dd3d3dd6520fe server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:29:16.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.883+0000 7f75837fe640 1 --2- 172.21.15.26:0/349805472 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f756c0a2cf0 0x7f756c0a51b0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f757c031f80 tx=0x7f757c0095c0 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:29:16.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.883+0000 7f75697fa640 1 -- 172.21.15.26:0/2730389467 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7578009de0 con 0x555f3d4fd240 2024-09-11T23:29:16.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.883+0000 7f75697fa640 1 -- 172.21.15.26:0/2730389467 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f757809b570 con 0x555f3d4fd240 2024-09-11T23:29:16.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.883+0000 7f75697fa640 1 -- 172.21.15.26:0/2730389467 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75780cca20 con 0x555f3d4fd240 2024-09-11T23:29:16.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.884+0000 7f758b992740 1 -- 172.21.15.26:0/2730389467 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555f3d4fd240 msgr2=0x555f3d4ff630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:16.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.884+0000 7f758b992740 1 --2- 172.21.15.26:0/2730389467 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555f3d4fd240 0x555f3d4ff630 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f75780cc7a0 tx=0x7f757806e5e0 comp rx=0 tx=0).stop 2024-09-11T23:29:16.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.884+0000 7f758b992740 1 -- 172.21.15.26:0/2730389467 shutdown_connections 2024-09-11T23:29:16.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.884+0000 7f758b992740 1 --2- 172.21.15.26:0/2730389467 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555f3d4f2980 0x555f3d4fcd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:16.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.884+0000 7f758b992740 1 --2- 172.21.15.26:0/2730389467 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555f3d4fd240 0x555f3d4ff630 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:16.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.884+0000 7f758b992740 1 --2- 172.21.15.26:0/2730389467 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555f3d4f2060 0x555f3d4f2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:16.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.884+0000 7f758b992740 1 -- 172.21.15.26:0/2730389467 >> 172.21.15.26:0/2730389467 conn(0x555f3d4ee5f0 msgr2=0x555f3d4ee9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:29:16.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.884+0000 7f758b992740 1 -- 172.21.15.26:0/2730389467 shutdown_connections 2024-09-11T23:29:16.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.884+0000 7f758b992740 1 -- 172.21.15.26:0/2730389467 wait complete. 2024-09-11T23:29:16.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.884+0000 7f758b992740 1 -- start start 2024-09-11T23:29:16.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.884+0000 7f758b992740 1 Processor -- start 2024-09-11T23:29:16.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.885+0000 7f758b992740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555f3d4f2060 0x555f3d4fa1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:16.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.885+0000 7f758b992740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555f3d4f2980 0x555f3d4fa6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:16.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.885+0000 7f758b992740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555f3d4fd240 0x555f3d4fac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:16.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.885+0000 7f758b992740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x555f3d4ffdb0 con 0x555f3d4f2060 2024-09-11T23:29:16.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.885+0000 7f758b992740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x555f3d3dff10 con 0x555f3d4fd240 2024-09-11T23:29:16.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.885+0000 7f758b992740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x555f3d4fff30 con 0x555f3d4f2980 2024-09-11T23:29:16.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.885+0000 7f758b992740 10 client.0 _ll_register_callbacks cb 0x555f3d403db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:29:16.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.885+0000 7f7582ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555f3d4f2060 0x555f3d4fa1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:16.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.885+0000 7f7582ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555f3d4f2060 0x555f3d4fa1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58968/0 (socket says 172.21.15.26:58968) 2024-09-11T23:29:16.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.885+0000 7f7582ffd640 1 -- 172.21.15.26:0/3476349762 learned_addr learned my addr 172.21.15.26:0/3476349762 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:29:16.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.885+0000 7f75837fe640 1 --2- 172.21.15.26:0/3476349762 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555f3d4fd240 0x555f3d4fac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:16.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.885+0000 7f75827fc640 1 --2- 172.21.15.26:0/3476349762 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555f3d4f2980 0x555f3d4fa6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:16.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.885+0000 7f7582ffd640 1 -- 172.21.15.26:0/3476349762 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555f3d4f2980 msgr2=0x555f3d4fa6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:29:16.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.885+0000 7f7582ffd640 1 --2- 172.21.15.26:0/3476349762 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555f3d4f2980 0x555f3d4fa6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:16.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.885+0000 7f7582ffd640 1 -- 172.21.15.26:0/3476349762 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555f3d4fd240 msgr2=0x555f3d4fac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:29:16.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.885+0000 7f7582ffd640 1 --2- 172.21.15.26:0/3476349762 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555f3d4fd240 0x555f3d4fac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:16.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.885+0000 7f7582ffd640 1 -- 172.21.15.26:0/3476349762 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f3d166580 con 0x555f3d4f2060 2024-09-11T23:29:16.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.886+0000 7f7582ffd640 1 --2- 172.21.15.26:0/3476349762 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555f3d4f2060 0x555f3d4fa1b0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f7574010820 tx=0x7f7574010ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72f6b5957a3dbbf server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:29:16.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.886+0000 7f7566ff5640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:29:16.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.886+0000 7f7566ff5640 1 -- 172.21.15.26:0/3476349762 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f757400fb90 con 0x555f3d4f2060 2024-09-11T23:29:16.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.886+0000 7f7566ff5640 1 -- 172.21.15.26:0/3476349762 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8586+0+0 (secure 0 0 0) 0x7f757400fd30 con 0x555f3d4f2060 2024-09-11T23:29:16.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.886+0000 7f758b992740 1 -- 172.21.15.26:0/3476349762 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f757809b570 con 0x555f3d4f2060 2024-09-11T23:29:16.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.886+0000 7f7566ff5640 1 -- 172.21.15.26:0/3476349762 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7f7574005020 con 0x555f3d4f2060 2024-09-11T23:29:16.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.886+0000 7f758b992740 10 client.17049 fetch_fsmap learned FSMap version 57 2024-09-11T23:29:16.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.886+0000 7f758b992740 1 -- 172.21.15.26:0/3476349762 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x555f3d230960 con 0x555f3d4f2060 2024-09-11T23:29:16.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.886+0000 7f7566ff5640 1 -- 172.21.15.26:0/3476349762 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f757809b570 con 0x555f3d4f2060 2024-09-11T23:29:16.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.886+0000 7f7566ff5640 1 -- 172.21.15.26:0/3476349762 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 57) ==== 8366+0+0 (secure 0 0 0) 0x7f7574019410 con 0x555f3d4f2060 2024-09-11T23:29:16.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.886+0000 7f758b992740 10 client.17049 fetch_fsmap finished waiting for FSMap version 57 2024-09-11T23:29:16.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.886+0000 7f758b992740 10 client.17049 resolve_mds: validated gid 24485 aka daemon mds.b 2024-09-11T23:29:16.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.886+0000 7f758b992740 20 client.17049 populate_metadata read hostname 'smithi026' 2024-09-11T23:29:16.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.886+0000 7f758b992740 1 --2- 172.21.15.26:0/3476349762 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x555f3d511240 0x555f3d5136c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:16.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.886+0000 7f758b992740 4 client.17049 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:29:16.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.886+0000 7f758b992740 1 -- 172.21.15.26:0/3476349762 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555f3d464b90 con 0x555f3d511240 2024-09-11T23:29:16.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.887+0000 7f75827fc640 1 --2- 172.21.15.26:0/3476349762 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x555f3d511240 0x555f3d5136c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:16.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.887+0000 7f75827fc640 1 --2- 172.21.15.26:0/3476349762 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x555f3d511240 0x555f3d5136c0 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b890c0e36ca1b0 server_cookie=e7453c0ec7c2870d in_seq=0 out_seq=0 2024-09-11T23:29:16.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.887+0000 7f7566ff5640 10 client.17049 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:29:16.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.888+0000 7f7566ff5640 1 -- 172.21.15.26:0/3476349762 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555f3d464b90 con 0x555f3d511240 2024-09-11T23:29:16.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.888+0000 7f7566ff5640 10 client.17049 handle_command_reply: tid=0 2024-09-11T23:29:16.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.888+0000 7f758b992740 1 client.17049 shutdown 2024-09-11T23:29:16.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.888+0000 7f758b992740 1 -- 172.21.15.26:0/3476349762 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555f3d4f2060 msgr2=0x555f3d4fa1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:16.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.888+0000 7f758b992740 1 --2- 172.21.15.26:0/3476349762 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555f3d4f2060 0x555f3d4fa1b0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f7574010820 tx=0x7f7574010ce0 comp rx=0 tx=0).stop 2024-09-11T23:29:16.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.888+0000 7f758b992740 1 -- 172.21.15.26:0/3476349762 shutdown_connections 2024-09-11T23:29:16.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.888+0000 7f758b992740 1 --2- 172.21.15.26:0/3476349762 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555f3d4fd240 0x555f3d4fac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:16.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.888+0000 7f758b992740 1 --2- 172.21.15.26:0/3476349762 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x555f3d511240 0x555f3d5136c0 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:16.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.888+0000 7f7566ff5640 0 client.17049 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:29:16.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.888+0000 7f758b992740 1 --2- 172.21.15.26:0/3476349762 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555f3d4f2980 0x555f3d4fa6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:16.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.888+0000 7f758b992740 1 --2- 172.21.15.26:0/3476349762 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555f3d4f2060 0x555f3d4fa1b0 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:16.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.888+0000 7f758b992740 1 -- 172.21.15.26:0/3476349762 >> 172.21.15.26:0/3476349762 conn(0x555f3d4ee5f0 msgr2=0x555f3d4f42d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:29:16.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.888+0000 7f758b992740 1 -- 172.21.15.26:0/3476349762 shutdown_connections 2024-09-11T23:29:16.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.888+0000 7f758b992740 1 -- 172.21.15.26:0/3476349762 wait complete. 2024-09-11T23:29:16.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.888+0000 7f758b992740 20 client.17049 trim_cache size 0 max 16384 2024-09-11T23:29:16.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.905+0000 7f758b992740 1 Processor -- start 2024-09-11T23:29:16.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.905+0000 7f758b992740 1 -- start start 2024-09-11T23:29:16.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.905+0000 7f758b992740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555f3d4f2060 0x555f3d518fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:16.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.905+0000 7f758b992740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555f3d4f2980 0x555f3d519520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:16.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.905+0000 7f758b992740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555f3d4fd240 0x555f3d4f7410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:16.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.905+0000 7f758b992740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f75841ab820 con 0x555f3d4f2060 2024-09-11T23:29:16.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.905+0000 7f758b992740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f75841ad390 con 0x555f3d4fd240 2024-09-11T23:29:16.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.905+0000 7f758b992740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f75841abef0 con 0x555f3d4f2980 2024-09-11T23:29:16.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.905+0000 7f7582ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555f3d4f2060 0x555f3d518fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:16.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.905+0000 7f7582ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555f3d4f2060 0x555f3d518fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58980/0 (socket says 172.21.15.26:58980) 2024-09-11T23:29:16.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.905+0000 7f7582ffd640 1 -- 172.21.15.26:0/1384176154 learned_addr learned my addr 172.21.15.26:0/1384176154 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:29:16.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.906+0000 7f75837fe640 1 --2- 172.21.15.26:0/1384176154 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555f3d4fd240 0x555f3d4f7410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:16.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.906+0000 7f7582ffd640 1 -- 172.21.15.26:0/1384176154 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555f3d4f2980 msgr2=0x555f3d519520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:29:16.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.906+0000 7f7582ffd640 1 --2- 172.21.15.26:0/1384176154 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555f3d4f2980 0x555f3d519520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:16.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.906+0000 7f7582ffd640 1 -- 172.21.15.26:0/1384176154 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555f3d4fd240 msgr2=0x555f3d4f7410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:29:16.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.906+0000 7f7582ffd640 1 --2- 172.21.15.26:0/1384176154 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555f3d4fd240 0x555f3d4f7410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:16.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.906+0000 7f7582ffd640 1 -- 172.21.15.26:0/1384176154 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f3d4ecbb0 con 0x555f3d4f2060 2024-09-11T23:29:16.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.906+0000 7f7582ffd640 1 --2- 172.21.15.26:0/1384176154 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555f3d4f2060 0x555f3d518fe0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f7574010540 tx=0x7f757400ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e21d6e3e6c1273e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:29:16.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.906+0000 7f7568ff9640 1 -- 172.21.15.26:0/1384176154 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f757402f070 con 0x555f3d4f2060 2024-09-11T23:29:16.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.906+0000 7f7568ff9640 1 -- 172.21.15.26:0/1384176154 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f75740193e0 con 0x555f3d4f2060 2024-09-11T23:29:16.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.906+0000 7f7568ff9640 1 -- 172.21.15.26:0/1384176154 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75740054b0 con 0x555f3d4f2060 2024-09-11T23:29:16.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.907+0000 7f758b992740 1 -- 172.21.15.26:0/1384176154 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555f3d4f2060 msgr2=0x555f3d518fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:16.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.907+0000 7f758b992740 1 --2- 172.21.15.26:0/1384176154 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555f3d4f2060 0x555f3d518fe0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f7574010540 tx=0x7f757400ad30 comp rx=0 tx=0).stop 2024-09-11T23:29:16.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.907+0000 7f758b992740 1 -- 172.21.15.26:0/1384176154 shutdown_connections 2024-09-11T23:29:16.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.907+0000 7f758b992740 1 --2- 172.21.15.26:0/1384176154 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555f3d4fd240 0x555f3d4f7410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:16.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.907+0000 7f758b992740 1 --2- 172.21.15.26:0/1384176154 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555f3d4f2980 0x555f3d519520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:16.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.907+0000 7f758b992740 1 --2- 172.21.15.26:0/1384176154 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555f3d4f2060 0x555f3d518fe0 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:16.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.907+0000 7f758b992740 1 -- 172.21.15.26:0/1384176154 >> 172.21.15.26:0/1384176154 conn(0x555f3d4ee5f0 msgr2=0x555f3d4f8a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:29:16.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.907+0000 7f758b992740 1 -- 172.21.15.26:0/1384176154 shutdown_connections 2024-09-11T23:29:16.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.907+0000 7f758b992740 1 -- 172.21.15.26:0/1384176154 wait complete. 2024-09-11T23:29:16.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.907+0000 7f758b992740 1 -- start start 2024-09-11T23:29:16.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.907+0000 7f758b992740 1 Processor -- start 2024-09-11T23:29:16.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.908+0000 7f758b992740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555f3d4f2060 0x555f3d505250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:16.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.908+0000 7f758b992740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555f3d4f2980 0x555f3d505790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:16.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.908+0000 7f758b992740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555f3d4fd240 0x555f3d501690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:16.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.908+0000 7f758b992740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f75841accc0 con 0x555f3d4fd240 2024-09-11T23:29:16.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.908+0000 7f758b992740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f75841ac5c0 con 0x555f3d4f2980 2024-09-11T23:29:16.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.908+0000 7f758b992740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f75841a9820 con 0x555f3d4f2060 2024-09-11T23:29:16.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.908+0000 7f758b992740 10 client.0 _ll_register_callbacks cb 0x7f7574010430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:29:16.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.908+0000 7f75837fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555f3d4fd240 0x555f3d501690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:16.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.908+0000 7f75837fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555f3d4fd240 0x555f3d501690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58986/0 (socket says 172.21.15.26:58986) 2024-09-11T23:29:16.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.908+0000 7f7582ffd640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555f3d4f2060 0x555f3d505250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:16.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.908+0000 7f75837fe640 1 -- 172.21.15.26:0/3720316177 learned_addr learned my addr 172.21.15.26:0/3720316177 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:29:16.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.908+0000 7f75827fc640 1 --2- 172.21.15.26:0/3720316177 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555f3d4f2980 0x555f3d505790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:16.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.908+0000 7f75837fe640 1 -- 172.21.15.26:0/3720316177 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555f3d4f2060 msgr2=0x555f3d505250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:29:16.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.908+0000 7f75837fe640 1 --2- 172.21.15.26:0/3720316177 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555f3d4f2060 0x555f3d505250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:16.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.908+0000 7f75837fe640 1 -- 172.21.15.26:0/3720316177 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555f3d4f2980 msgr2=0x555f3d505790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:29:16.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.908+0000 7f75837fe640 1 --2- 172.21.15.26:0/3720316177 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555f3d4f2980 0x555f3d505790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:16.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.908+0000 7f75837fe640 1 -- 172.21.15.26:0/3720316177 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f3d501ec0 con 0x555f3d4fd240 2024-09-11T23:29:16.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.908+0000 7f75837fe640 1 --2- 172.21.15.26:0/3720316177 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555f3d4fd240 0x555f3d501690 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f757c009a60 tx=0x7f757c009360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f5659cdbeecce88 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:29:16.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.908+0000 7f7566ff5640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:29:16.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.908+0000 7f7566ff5640 1 -- 172.21.15.26:0/3720316177 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f757c05a060 con 0x555f3d4fd240 2024-09-11T23:29:16.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.909+0000 7f758b992740 1 -- 172.21.15.26:0/3720316177 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f75740193e0 con 0x555f3d4fd240 2024-09-11T23:29:16.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.909+0000 7f7566ff5640 1 -- 172.21.15.26:0/3720316177 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8586+0+0 (secure 0 0 0) 0x7f757c004250 con 0x555f3d4fd240 2024-09-11T23:29:16.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.909+0000 7f7566ff5640 1 -- 172.21.15.26:0/3720316177 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7f757c004450 con 0x555f3d4fd240 2024-09-11T23:29:16.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.909+0000 7f758b992740 10 client.17055 fetch_fsmap learned FSMap version 57 2024-09-11T23:29:16.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.909+0000 7f758b992740 1 -- 172.21.15.26:0/3720316177 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x555f3d57cae0 con 0x555f3d4fd240 2024-09-11T23:29:16.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.909+0000 7f7566ff5640 1 -- 172.21.15.26:0/3720316177 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f75740193e0 con 0x555f3d4fd240 2024-09-11T23:29:16.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.909+0000 7f7566ff5640 1 -- 172.21.15.26:0/3720316177 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 57) ==== 8366+0+0 (secure 0 0 0) 0x7f757c054070 con 0x555f3d4fd240 2024-09-11T23:29:16.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.909+0000 7f758b992740 10 client.17055 fetch_fsmap finished waiting for FSMap version 57 2024-09-11T23:29:16.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.909+0000 7f758b992740 10 client.17055 resolve_mds: validated gid 24485 aka daemon mds.b 2024-09-11T23:29:16.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.909+0000 7f758b992740 20 client.17055 populate_metadata read hostname 'smithi026' 2024-09-11T23:29:16.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.909+0000 7f758b992740 1 --2- 172.21.15.26:0/3720316177 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x555f3d511240 0x555f3d57f270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:16.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.909+0000 7f758b992740 4 client.17055 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "57f3de60", "release": true, "await": true}] 2024-09-11T23:29:16.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.909+0000 7f758b992740 1 -- 172.21.15.26:0/3720316177 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "quiesce db", "set_id": "57f3de60", "release": true, "await": true}) -- 0x555f3d45f0a0 con 0x555f3d511240 2024-09-11T23:29:16.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.909+0000 7f7582ffd640 1 --2- 172.21.15.26:0/3720316177 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x555f3d511240 0x555f3d57f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:16.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.910+0000 7f7582ffd640 1 --2- 172.21.15.26:0/3720316177 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x555f3d511240 0x555f3d57f270 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15fd0b63d7f9cf1d server_cookie=ba256a1ef4215617 in_seq=0 out_seq=0 2024-09-11T23:29:16.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:16.910+0000 7f7566ff5640 10 client.17055 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:29:18.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:18.029+0000 7f7566ff5640 1 -- 172.21.15.26:0/3720316177 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x555f3d45f0a0 con 0x555f3d511240 2024-09-11T23:29:18.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:18.029+0000 7f7566ff5640 10 client.17055 handle_command_reply: tid=0 2024-09-11T23:29:18.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:18.029+0000 7f758b992740 1 client.17055 shutdown 2024-09-11T23:29:18.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:18.029+0000 7f758b992740 1 -- 172.21.15.26:0/3720316177 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555f3d4fd240 msgr2=0x555f3d501690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:18.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:18.029+0000 7f758b992740 1 --2- 172.21.15.26:0/3720316177 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555f3d4fd240 0x555f3d501690 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f757c009a60 tx=0x7f757c009360 comp rx=0 tx=0).stop 2024-09-11T23:29:18.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:18.029+0000 7f758b992740 1 -- 172.21.15.26:0/3720316177 shutdown_connections 2024-09-11T23:29:18.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:18.029+0000 7f758b992740 1 --2- 172.21.15.26:0/3720316177 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555f3d4f2980 0x555f3d505790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:18.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:18.029+0000 7f758b992740 1 --2- 172.21.15.26:0/3720316177 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x555f3d511240 0x555f3d57f270 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:18.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:18.029+0000 7f758b992740 1 --2- 172.21.15.26:0/3720316177 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555f3d4fd240 0x555f3d501690 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:18.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:18.029+0000 7f758b992740 1 --2- 172.21.15.26:0/3720316177 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555f3d4f2060 0x555f3d505250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:18.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:18.029+0000 7f758b992740 1 -- 172.21.15.26:0/3720316177 >> 172.21.15.26:0/3720316177 conn(0x555f3d4ee5f0 msgr2=0x555f3d4f3ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:29:18.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:18.029+0000 7f7566ff5640 0 client.17055 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:29:18.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:18.029+0000 7f758b992740 1 -- 172.21.15.26:0/3720316177 shutdown_connections 2024-09-11T23:29:18.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:18.029+0000 7f758b992740 1 -- 172.21.15.26:0/3720316177 wait complete. 2024-09-11T23:29:18.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:18.029+0000 7f758b992740 20 client.17055 trim_cache size 0 max 16384 2024-09-11T23:29:18.031 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-11T23:29:18.031 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 57, 2024-09-11T23:29:18.031 INFO:teuthology.orchestra.run.smithi026.stdout: "leader": 24485, 2024-09-11T23:29:18.031 INFO:teuthology.orchestra.run.smithi026.stdout: "set_version": 40, 2024-09-11T23:29:18.031 INFO:teuthology.orchestra.run.smithi026.stdout: "sets": { 2024-09-11T23:29:18.031 INFO:teuthology.orchestra.run.smithi026.stdout: "57f3de60": { 2024-09-11T23:29:18.031 INFO:teuthology.orchestra.run.smithi026.stdout: "version": 40, 2024-09-11T23:29:18.031 INFO:teuthology.orchestra.run.smithi026.stdout: "age_ref": 0.0, 2024-09-11T23:29:18.031 INFO:teuthology.orchestra.run.smithi026.stdout: "state": { 2024-09-11T23:29:18.032 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "RELEASED", 2024-09-11T23:29:18.032 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.0 2024-09-11T23:29:18.032 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:29:18.032 INFO:teuthology.orchestra.run.smithi026.stdout: "timeout": 90.0, 2024-09-11T23:29:18.032 INFO:teuthology.orchestra.run.smithi026.stdout: "expiration": 142.6, 2024-09-11T23:29:18.032 INFO:teuthology.orchestra.run.smithi026.stdout: "members": { 2024-09-11T23:29:18.032 INFO:teuthology.orchestra.run.smithi026.stdout: "file:/": { 2024-09-11T23:29:18.032 INFO:teuthology.orchestra.run.smithi026.stdout: "excluded": false, 2024-09-11T23:29:18.032 INFO:teuthology.orchestra.run.smithi026.stdout: "state": { 2024-09-11T23:29:18.032 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "RELEASED", 2024-09-11T23:29:18.032 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.0 2024-09-11T23:29:18.032 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:29:18.032 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:29:18.033 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:29:18.033 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:29:18.033 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:29:18.033 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-11T23:29:18.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:18.029+0000 7f75895b9640 1 -- 172.21.15.26:0/349805472 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f756c0a2cf0 msgr2=0x7f756c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:18.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:18.029+0000 7f75895b9640 1 --2- 172.21.15.26:0/349805472 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f756c0a2cf0 0x7f756c0a51b0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f757c031f80 tx=0x7f757c0095c0 comp rx=0 tx=0).stop 2024-09-11T23:29:18.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:18.029+0000 7f75895b9640 1 -- 172.21.15.26:0/349805472 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7584102790 msgr2=0x7f75841988d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:18.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:18.029+0000 7f75895b9640 1 --2- 172.21.15.26:0/349805472 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7584102790 0x7f75841988d0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f757800d950 tx=0x7f757800de10 comp rx=0 tx=0).stop 2024-09-11T23:29:18.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:18.029+0000 7f75895b9640 1 -- 172.21.15.26:0/349805472 shutdown_connections 2024-09-11T23:29:18.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:18.030+0000 7f75895b9640 1 --2- 172.21.15.26:0/349805472 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f756c0a2cf0 0x7f756c0a51b0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:18.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:18.030+0000 7f75895b9640 1 --2- 172.21.15.26:0/349805472 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7584106f10 0x7f758419cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:18.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:18.030+0000 7f75895b9640 1 --2- 172.21.15.26:0/349805472 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7584102790 0x7f75841988d0 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:18.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:18.030+0000 7f75895b9640 1 --2- 172.21.15.26:0/349805472 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7584101de0 0x7f7584198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:18.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:18.030+0000 7f75895b9640 1 -- 172.21.15.26:0/349805472 >> 172.21.15.26:0/349805472 conn(0x7f75840fd8c0 msgr2=0x7f75840feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:29:18.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:18.030+0000 7f75895b9640 1 -- 172.21.15.26:0/349805472 shutdown_connections 2024-09-11T23:29:18.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:18.030+0000 7f75895b9640 1 -- 172.21.15.26:0/349805472 wait complete. 2024-09-11T23:29:18.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:29:17 smithi136 ceph-mon[35121]: pgmap v339: 97 pgs: 97 active+clean; 2.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 40 KiB/s wr, 12 op/s 2024-09-11T23:29:18.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:29:17 smithi026 ceph-mon[31172]: pgmap v339: 97 pgs: 97 active+clean; 2.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 40 KiB/s wr, 12 op/s 2024-09-11T23:29:18.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:29:17 smithi076 ceph-mon[35296]: pgmap v339: 97 pgs: 97 active+clean; 2.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 40 KiB/s wr, 12 op/s 2024-09-11T23:29:18.772 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '57f3de60', total seconds elapsed: 26.8 2024-09-11T23:29:18.772 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 22.55 2024-09-11T23:29:18.772 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:29:19.038 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:29:19.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:19.528+0000 7f9070ba5640 1 -- 172.21.15.26:0/4271318973 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f906c10c170 msgr2=0x7f906c10e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:19.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:19.528+0000 7f9070ba5640 1 --2- 172.21.15.26:0/4271318973 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f906c10c170 0x7f906c10e610 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f9064009ca0 tx=0x7f906402a950 comp rx=0 tx=0).stop 2024-09-11T23:29:19.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:19.528+0000 7f9070ba5640 1 -- 172.21.15.26:0/4271318973 shutdown_connections 2024-09-11T23:29:19.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:19.528+0000 7f9070ba5640 1 --2- 172.21.15.26:0/4271318973 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f906c103660 0x7f906c10bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:19.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:19.528+0000 7f9070ba5640 1 --2- 172.21.15.26:0/4271318973 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f906c10c170 0x7f906c10e610 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:19.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:19.528+0000 7f9070ba5640 1 --2- 172.21.15.26:0/4271318973 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f906c102d40 0x7f906c103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:19.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:19.528+0000 7f9070ba5640 1 -- 172.21.15.26:0/4271318973 >> 172.21.15.26:0/4271318973 conn(0x7f906c0fc7a0 msgr2=0x7f906c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:29:19.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:19.528+0000 7f9070ba5640 1 -- 172.21.15.26:0/4271318973 shutdown_connections 2024-09-11T23:29:19.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:19.528+0000 7f9070ba5640 1 -- 172.21.15.26:0/4271318973 wait complete. 2024-09-11T23:29:19.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:19.529+0000 7f9070ba5640 1 Processor -- start 2024-09-11T23:29:19.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:19.529+0000 7f9070ba5640 1 -- start start 2024-09-11T23:29:19.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:19.529+0000 7f9070ba5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f906c102d40 0x7f906c1983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:19.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:19.529+0000 7f9070ba5640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f906c103660 0x7f906c198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:19.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:19.529+0000 7f9070ba5640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f906c10c170 0x7f906c19cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:19.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:19.529+0000 7f9070ba5640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f906c111f30 con 0x7f906c102d40 2024-09-11T23:29:19.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:19.529+0000 7f9070ba5640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f906c111db0 con 0x7f906c103660 2024-09-11T23:29:19.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:19.529+0000 7f9070ba5640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f906c1120b0 con 0x7f906c10c170 2024-09-11T23:29:19.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:19.529+0000 7f906a59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f906c102d40 0x7f906c1983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:19.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:19.529+0000 7f906a59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f906c102d40 0x7f906c1983d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59004/0 (socket says 172.21.15.26:59004) 2024-09-11T23:29:19.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:19.529+0000 7f906a59c640 1 -- 172.21.15.26:0/1603270173 learned_addr learned my addr 172.21.15.26:0/1603270173 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:29:19.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:19.529+0000 7f9069d9b640 1 --2- 172.21.15.26:0/1603270173 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f906c103660 0x7f906c198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:19.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:19.529+0000 7f906ad9d640 1 --2- 172.21.15.26:0/1603270173 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f906c10c170 0x7f906c19cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:19.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:19.530+0000 7f906a59c640 1 -- 172.21.15.26:0/1603270173 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f906c10c170 msgr2=0x7f906c19cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:19.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:19.530+0000 7f906a59c640 1 --2- 172.21.15.26:0/1603270173 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f906c10c170 0x7f906c19cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:19.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:19.530+0000 7f906a59c640 1 -- 172.21.15.26:0/1603270173 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f906c103660 msgr2=0x7f906c198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:19.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:19.530+0000 7f906a59c640 1 --2- 172.21.15.26:0/1603270173 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f906c103660 0x7f906c198910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:19.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:19.530+0000 7f906a59c640 1 -- 172.21.15.26:0/1603270173 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f906c19d380 con 0x7f906c102d40 2024-09-11T23:29:19.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:19.530+0000 7f906a59c640 1 --2- 172.21.15.26:0/1603270173 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f906c102d40 0x7f906c1983d0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f905c00c920 tx=0x7f905c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:29:19.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:19.530+0000 7f905b7fe640 1 -- 172.21.15.26:0/1603270173 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f905c007c70 con 0x7f906c102d40 2024-09-11T23:29:19.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:19.530+0000 7f905b7fe640 1 -- 172.21.15.26:0/1603270173 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f905c007e10 con 0x7f906c102d40 2024-09-11T23:29:19.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:19.530+0000 7f9070ba5640 1 -- 172.21.15.26:0/1603270173 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f906c19d670 con 0x7f906c102d40 2024-09-11T23:29:19.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:19.530+0000 7f905b7fe640 1 -- 172.21.15.26:0/1603270173 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f905c018890 con 0x7f906c102d40 2024-09-11T23:29:19.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:19.530+0000 7f9070ba5640 1 -- 172.21.15.26:0/1603270173 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f906c1a4f50 con 0x7f906c102d40 2024-09-11T23:29:19.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:19.531+0000 7f9070ba5640 1 -- 172.21.15.26:0/1603270173 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f906c103120 con 0x7f906c102d40 2024-09-11T23:29:19.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:19.532+0000 7f905b7fe640 1 -- 172.21.15.26:0/1603270173 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f905c018a50 con 0x7f906c102d40 2024-09-11T23:29:19.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:19.533+0000 7f905b7fe640 1 --2- 172.21.15.26:0/1603270173 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f90540a2cf0 0x7f90540a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:19.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:19.533+0000 7f905b7fe640 1 -- 172.21.15.26:0/1603270173 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8586+0+0 (secure 0 0 0) 0x7f905c0cf570 con 0x7f906c102d40 2024-09-11T23:29:19.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:19.533+0000 7f9069d9b640 1 --2- 172.21.15.26:0/1603270173 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f90540a2cf0 0x7f90540a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:19.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:19.533+0000 7f9069d9b640 1 --2- 172.21.15.26:0/1603270173 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f90540a2cf0 0x7f90540a51b0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f906c10d4b0 tx=0x7f9060005f90 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:29:19.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:19.537+0000 7f905b7fe640 1 -- 172.21.15.26:0/1603270173 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f905c014030 con 0x7f906c102d40 2024-09-11T23:29:19.698 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.695+0000 7f63cd559640 1 -- 172.21.15.26:0/1717876645 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63c81069e0 msgr2=0x7f63c8106dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:19.698 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.695+0000 7f63cd559640 1 --2- 172.21.15.26:0/1717876645 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63c81069e0 0x7f63c8106dc0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f63b8009930 tx=0x7f63b802a420 comp rx=0 tx=0).stop 2024-09-11T23:29:19.698 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.695+0000 7f63cd559640 1 -- 172.21.15.26:0/1717876645 shutdown_connections 2024-09-11T23:29:19.699 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.696+0000 7f63cd559640 1 --2- 172.21.15.26:0/1717876645 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f63c8101370 0x7f63c810d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:19.699 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.696+0000 7f63cd559640 1 --2- 172.21.15.26:0/1717876645 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f63c81009d0 0x7f63c8100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:19.699 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.696+0000 7f63cd559640 1 --2- 172.21.15.26:0/1717876645 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63c81069e0 0x7f63c8106dc0 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:19.699 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.696+0000 7f63cd559640 1 -- 172.21.15.26:0/1717876645 >> 172.21.15.26:0/1717876645 conn(0x7f63c80fc6f0 msgr2=0x7f63c80feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:29:19.699 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.696+0000 7f63cd559640 1 -- 172.21.15.26:0/1717876645 shutdown_connections 2024-09-11T23:29:19.699 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.696+0000 7f63cd559640 1 -- 172.21.15.26:0/1717876645 wait complete. 2024-09-11T23:29:19.699 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.696+0000 7f63cd559640 1 Processor -- start 2024-09-11T23:29:19.699 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.696+0000 7f63cd559640 1 -- start start 2024-09-11T23:29:19.699 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.696+0000 7f63cd559640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f63c81009d0 0x7f63c8197270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:19.699 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.696+0000 7f63cd559640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f63c8101370 0x7f63c81977b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:19.700 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.696+0000 7f63cd559640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63c81069e0 0x7f63c819bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:19.700 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.696+0000 7f63cd559640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f63c8110e20 con 0x7f63c81069e0 2024-09-11T23:29:19.700 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.696+0000 7f63cd559640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f63c8110ca0 con 0x7f63c81009d0 2024-09-11T23:29:19.700 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.696+0000 7f63cd559640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f63c8110fa0 con 0x7f63c8101370 2024-09-11T23:29:19.700 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.697+0000 7f63c77fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63c81069e0 0x7f63c819bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:19.700 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.697+0000 7f63c77fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63c81069e0 0x7f63c819bb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59024/0 (socket says 172.21.15.26:59024) 2024-09-11T23:29:19.700 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.697+0000 7f63c77fe640 1 -- 172.21.15.26:0/1960211098 learned_addr learned my addr 172.21.15.26:0/1960211098 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:29:19.700 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.697+0000 7f63c6ffd640 1 --2- 172.21.15.26:0/1960211098 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f63c81009d0 0x7f63c8197270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:19.700 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.697+0000 7f63bffff640 1 --2- 172.21.15.26:0/1960211098 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f63c8101370 0x7f63c81977b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:19.700 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.697+0000 7f63c77fe640 1 -- 172.21.15.26:0/1960211098 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f63c8101370 msgr2=0x7f63c81977b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:19.700 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.697+0000 7f63c77fe640 1 --2- 172.21.15.26:0/1960211098 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f63c8101370 0x7f63c81977b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:19.700 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.697+0000 7f63c77fe640 1 -- 172.21.15.26:0/1960211098 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f63c81009d0 msgr2=0x7f63c8197270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:19.700 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.697+0000 7f63c77fe640 1 --2- 172.21.15.26:0/1960211098 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f63c81009d0 0x7f63c8197270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:19.700 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.697+0000 7f63c77fe640 1 -- 172.21.15.26:0/1960211098 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63c819c260 con 0x7f63c81069e0 2024-09-11T23:29:19.701 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.697+0000 7f63c6ffd640 1 --2- 172.21.15.26:0/1960211098 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f63c81009d0 0x7f63c8197270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:29:19.701 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.697+0000 7f63c77fe640 1 --2- 172.21.15.26:0/1960211098 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63c81069e0 0x7f63c819bb30 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f63c000eb10 tx=0x7f63c000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:29:19.701 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.697+0000 7f63c4ff9640 1 -- 172.21.15.26:0/1960211098 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63c0004b20 con 0x7f63c81069e0 2024-09-11T23:29:19.701 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.697+0000 7f63c4ff9640 1 -- 172.21.15.26:0/1960211098 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63c0004cc0 con 0x7f63c81069e0 2024-09-11T23:29:19.701 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.697+0000 7f63cd559640 1 -- 172.21.15.26:0/1960211098 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63c819c550 con 0x7f63c81069e0 2024-09-11T23:29:19.701 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.697+0000 7f63c4ff9640 1 -- 172.21.15.26:0/1960211098 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63c0010760 con 0x7f63c81069e0 2024-09-11T23:29:19.702 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.697+0000 7f63cd559640 1 -- 172.21.15.26:0/1960211098 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63c81a3de0 con 0x7f63c81069e0 2024-09-11T23:29:19.708 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.700+0000 7f63c4ff9640 1 -- 172.21.15.26:0/1960211098 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f63c0005020 con 0x7f63c81069e0 2024-09-11T23:29:19.708 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.700+0000 7f63d03d0740 1 Processor -- start 2024-09-11T23:29:19.708 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.701+0000 7f63d03d0740 1 -- start start 2024-09-11T23:29:19.708 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.701+0000 7f63d03d0740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ac386389d0 0x55ac38638db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:19.708 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.701+0000 7f63d03d0740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ac386392f0 0x55ac38645980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:19.709 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.701+0000 7f63d03d0740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ac38645ec0 0x55ac386482b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:19.709 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.701+0000 7f63d03d0740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55ac3859d2f0 con 0x55ac38645ec0 2024-09-11T23:29:19.709 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.701+0000 7f63d03d0740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55ac38545270 con 0x55ac386392f0 2024-09-11T23:29:19.709 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.701+0000 7f63d03d0740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55ac385992b0 con 0x55ac386389d0 2024-09-11T23:29:19.709 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.701+0000 7f63bffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ac38645ec0 0x55ac386482b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:19.709 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.701+0000 7f63c6ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ac386392f0 0x55ac38645980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:19.709 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.701+0000 7f63c6ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ac386392f0 0x55ac38645980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:44252/0 (socket says 172.21.15.26:44252) 2024-09-11T23:29:19.709 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.701+0000 7f63bffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ac38645ec0 0x55ac386482b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59032/0 (socket says 172.21.15.26:59032) 2024-09-11T23:29:19.709 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.701+0000 7f63c6ffd640 1 -- 172.21.15.26:0/3540417081 learned_addr learned my addr 172.21.15.26:0/3540417081 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:29:19.709 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.701+0000 7f63bffff640 1 --2- 172.21.15.26:0/3540417081 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ac386389d0 0x55ac38638db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:19.709 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.701+0000 7f63c6ffd640 1 -- 172.21.15.26:0/3540417081 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ac386389d0 msgr2=0x55ac38638db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:29:19.709 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.701+0000 7f63c6ffd640 1 --2- 172.21.15.26:0/3540417081 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ac386389d0 0x55ac38638db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:19.710 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.701+0000 7f63c6ffd640 1 -- 172.21.15.26:0/3540417081 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ac38645ec0 msgr2=0x55ac386482b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:29:19.710 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.701+0000 7f63c6ffd640 1 --2- 172.21.15.26:0/3540417081 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ac38645ec0 0x55ac386482b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:19.710 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.702+0000 7f63c6ffd640 1 -- 172.21.15.26:0/3540417081 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac38544340 con 0x55ac386392f0 2024-09-11T23:29:19.710 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.702+0000 7f63c4ff9640 1 --2- 172.21.15.26:0/1960211098 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f63b40a2da0 0x7f63b40a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:19.710 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.702+0000 7f63bffff640 1 --2- 172.21.15.26:0/1960211098 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f63b40a2da0 0x7f63b40a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:19.710 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.702+0000 7f63c4ff9640 1 -- 172.21.15.26:0/1960211098 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8586+0+0 (secure 0 0 0) 0x7f63c00cfb70 con 0x7f63c81069e0 2024-09-11T23:29:19.710 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.702+0000 7f63c6ffd640 1 --2- 172.21.15.26:0/3540417081 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ac386392f0 0x55ac38645980 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f63b8033000 tx=0x7f63b8005e60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3586385d27a49c15 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:29:19.710 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.702+0000 7f63bd7fa640 1 -- 172.21.15.26:0/3540417081 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63b8035030 con 0x55ac386392f0 2024-09-11T23:29:19.710 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.702+0000 7f63bd7fa640 1 -- 172.21.15.26:0/3540417081 <== mon.2 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63b8004030 con 0x55ac386392f0 2024-09-11T23:29:19.710 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.702+0000 7f63bd7fa640 1 -- 172.21.15.26:0/3540417081 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63b8005500 con 0x55ac386392f0 2024-09-11T23:29:19.710 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.703+0000 7f63d03d0740 1 -- 172.21.15.26:0/3540417081 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ac386392f0 msgr2=0x55ac38645980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:19.711 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.703+0000 7f63bffff640 1 --2- 172.21.15.26:0/1960211098 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f63b40a2da0 0x7f63b40a5260 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x55ac386363e0 tx=0x7f63b0009640 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:29:19.711 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.703+0000 7f63d03d0740 1 --2- 172.21.15.26:0/3540417081 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ac386392f0 0x55ac38645980 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f63b8033000 tx=0x7f63b8005e60 comp rx=0 tx=0).stop 2024-09-11T23:29:19.711 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.703+0000 7f63d03d0740 1 -- 172.21.15.26:0/3540417081 shutdown_connections 2024-09-11T23:29:19.711 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.703+0000 7f63d03d0740 1 --2- 172.21.15.26:0/3540417081 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ac386392f0 0x55ac38645980 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:19.711 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.703+0000 7f63d03d0740 1 --2- 172.21.15.26:0/3540417081 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ac38645ec0 0x55ac386482b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:19.711 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.703+0000 7f63d03d0740 1 --2- 172.21.15.26:0/3540417081 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ac386389d0 0x55ac38638db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:19.711 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.703+0000 7f63d03d0740 1 -- 172.21.15.26:0/3540417081 >> 172.21.15.26:0/3540417081 conn(0x55ac38635060 msgr2=0x55ac38635440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:29:19.711 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.703+0000 7f63d03d0740 1 -- 172.21.15.26:0/3540417081 shutdown_connections 2024-09-11T23:29:19.711 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.703+0000 7f63d03d0740 1 -- 172.21.15.26:0/3540417081 wait complete. 2024-09-11T23:29:19.711 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.703+0000 7f63d03d0740 1 -- start start 2024-09-11T23:29:19.712 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.703+0000 7f63d03d0740 1 Processor -- start 2024-09-11T23:29:19.712 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.704+0000 7f63d03d0740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ac386389d0 0x55ac3863b810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:19.712 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.704+0000 7f63d03d0740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ac386392f0 0x55ac3863bd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:19.712 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.704+0000 7f63d03d0740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ac38645ec0 0x55ac3863c290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:19.712 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.704+0000 7f63d03d0740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55ac38648a90 con 0x55ac386392f0 2024-09-11T23:29:19.712 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.704+0000 7f63d03d0740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55ac38526290 con 0x55ac386389d0 2024-09-11T23:29:19.712 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.704+0000 7f63d03d0740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55ac38648c10 con 0x55ac38645ec0 2024-09-11T23:29:19.712 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.704+0000 7f63d03d0740 10 client.0 _ll_register_callbacks cb 0x55ac38557140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:29:19.712 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.704+0000 7f63bffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ac386392f0 0x55ac3863bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:19.712 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.704+0000 7f63c6ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ac386389d0 0x55ac3863b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:19.712 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.704+0000 7f63c6ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ac386389d0 0x55ac3863b810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:44262/0 (socket says 172.21.15.26:44262) 2024-09-11T23:29:19.712 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.704+0000 7f63c6ffd640 1 -- 172.21.15.26:0/3613341832 learned_addr learned my addr 172.21.15.26:0/3613341832 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:29:19.713 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.704+0000 7f63bffff640 1 --2- 172.21.15.26:0/3613341832 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ac386392f0 0x55ac3863bd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59036/0 (socket says 172.21.15.26:59036) 2024-09-11T23:29:19.713 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.704+0000 7f63c77fe640 1 --2- 172.21.15.26:0/3613341832 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ac38645ec0 0x55ac3863c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:19.713 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.704+0000 7f63c6ffd640 1 -- 172.21.15.26:0/3613341832 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ac38645ec0 msgr2=0x55ac3863c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:29:19.713 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.704+0000 7f63c6ffd640 1 --2- 172.21.15.26:0/3613341832 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ac38645ec0 0x55ac3863c290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:19.713 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.704+0000 7f63c6ffd640 1 -- 172.21.15.26:0/3613341832 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ac386392f0 msgr2=0x55ac3863bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:29:19.713 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.704+0000 7f63c6ffd640 1 --2- 172.21.15.26:0/3613341832 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ac386392f0 0x55ac3863bd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:19.713 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.704+0000 7f63c6ffd640 1 -- 172.21.15.26:0/3613341832 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac382b0210 con 0x55ac386389d0 2024-09-11T23:29:19.713 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.704+0000 7f63bffff640 1 --2- 172.21.15.26:0/3613341832 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ac386392f0 0x55ac3863bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:29:19.713 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.704+0000 7f63c6ffd640 1 --2- 172.21.15.26:0/3613341832 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ac386389d0 0x55ac3863b810 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f63b802a940 tx=0x7f63b8031b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f7932f6a61d3045f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:29:19.713 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.705+0000 7f63ae7fc640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-11T23:29:19.713 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.705+0000 7f63ae7fc640 1 -- 172.21.15.26:0/3613341832 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63b8051060 con 0x55ac386389d0 2024-09-11T23:29:19.713 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.705+0000 7f63d03d0740 1 -- 172.21.15.26:0/3613341832 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f63b8004030 con 0x55ac386389d0 2024-09-11T23:29:19.713 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.705+0000 7f63ae7fc640 1 -- 172.21.15.26:0/3613341832 <== mon.2 v2:172.21.15.76:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8586+0+0 (secure 0 0 0) 0x7f63b80317c0 con 0x55ac386389d0 2024-09-11T23:29:19.713 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.705+0000 7f63ae7fc640 1 -- 172.21.15.26:0/3613341832 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7f63b8035070 con 0x55ac386389d0 2024-09-11T23:29:19.714 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.705+0000 7f63d03d0740 10 client.26621 fetch_fsmap learned FSMap version 57 2024-09-11T23:29:19.714 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.705+0000 7f63d03d0740 1 -- 172.21.15.26:0/3613341832 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x55ac3837c620 con 0x55ac386389d0 2024-09-11T23:29:19.714 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.705+0000 7f63ae7fc640 1 -- 172.21.15.26:0/3613341832 <== mon.2 v2:172.21.15.76:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63b8004030 con 0x55ac386389d0 2024-09-11T23:29:19.714 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.705+0000 7f63ae7fc640 1 -- 172.21.15.26:0/3613341832 <== mon.2 v2:172.21.15.76:3300/0 5 ==== fsmap(e 57) ==== 8366+0+0 (secure 0 0 0) 0x7f63b8004c80 con 0x55ac386389d0 2024-09-11T23:29:19.714 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.705+0000 7f63d03d0740 10 client.26621 fetch_fsmap finished waiting for FSMap version 57 2024-09-11T23:29:19.714 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.705+0000 7f63d03d0740 10 client.26621 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:29:19.714 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.705+0000 7f63d03d0740 20 client.26621 populate_metadata read hostname 'smithi026' 2024-09-11T23:29:19.714 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.705+0000 7f63d03d0740 1 --2- 172.21.15.26:0/3613341832 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55ac38657f00 0x55ac3865a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:19.714 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.705+0000 7f63d03d0740 4 client.26621 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:29:19.714 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.705+0000 7f63d03d0740 1 -- 172.21.15.26:0/3613341832 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ac385aaa20 con 0x55ac38657f00 2024-09-11T23:29:19.714 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.706+0000 7f63bffff640 1 --2- 172.21.15.26:0/3613341832 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55ac38657f00 0x55ac3865a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:19.714 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.706+0000 7f63bffff640 1 --2- 172.21.15.26:0/3613341832 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55ac38657f00 0x55ac3865a2f0 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a1c951b34902e9e2 server_cookie=45be1857ef695f93 in_seq=0 out_seq=0 2024-09-11T23:29:19.714 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.706+0000 7f63ae7fc640 10 client.26621 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:29:19.714 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.707+0000 7f63ae7fc640 1 -- 172.21.15.26:0/3613341832 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ac385aaa20 con 0x55ac38657f00 2024-09-11T23:29:19.715 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.707+0000 7f63ae7fc640 10 client.26621 handle_command_reply: tid=0 2024-09-11T23:29:19.715 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.707+0000 7f63d03d0740 1 client.26621 shutdown 2024-09-11T23:29:19.715 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.707+0000 7f63d03d0740 1 -- 172.21.15.26:0/3613341832 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ac386389d0 msgr2=0x55ac3863b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:19.715 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.707+0000 7f63d03d0740 1 --2- 172.21.15.26:0/3613341832 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ac386389d0 0x55ac3863b810 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f63b802a940 tx=0x7f63b8031b20 comp rx=0 tx=0).stop 2024-09-11T23:29:19.715 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.707+0000 7f63d03d0740 1 -- 172.21.15.26:0/3613341832 shutdown_connections 2024-09-11T23:29:19.715 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.707+0000 7f63d03d0740 1 --2- 172.21.15.26:0/3613341832 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55ac38657f00 0x55ac3865a2f0 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:19.715 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.707+0000 7f63d03d0740 1 --2- 172.21.15.26:0/3613341832 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ac38645ec0 0x55ac3863c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:19.715 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.707+0000 7f63d03d0740 1 --2- 172.21.15.26:0/3613341832 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ac386392f0 0x55ac3863bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:19.715 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.707+0000 7f63d03d0740 1 --2- 172.21.15.26:0/3613341832 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ac386389d0 0x55ac3863b810 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:19.715 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.707+0000 7f63ae7fc640 0 client.26621 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:29:19.715 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.707+0000 7f63d03d0740 1 -- 172.21.15.26:0/3613341832 >> 172.21.15.26:0/3613341832 conn(0x55ac38635060 msgr2=0x55ac3863adc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:29:19.715 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.707+0000 7f63d03d0740 1 -- 172.21.15.26:0/3613341832 shutdown_connections 2024-09-11T23:29:19.715 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.707+0000 7f63d03d0740 1 -- 172.21.15.26:0/3613341832 wait complete. 2024-09-11T23:29:19.716 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.707+0000 7f63d03d0740 20 client.26621 trim_cache size 0 max 16384 2024-09-11T23:29:19.731 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.721+0000 7f63d03d0740 1 Processor -- start 2024-09-11T23:29:19.731 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.722+0000 7f63d03d0740 1 -- start start 2024-09-11T23:29:19.731 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.722+0000 7f63d03d0740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ac386389d0 0x55ac386339a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:19.731 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.722+0000 7f63d03d0740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ac386392f0 0x55ac38640050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:19.731 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.722+0000 7f63d03d0740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ac38645ec0 0x55ac38640590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:19.732 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.722+0000 7f63d03d0740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f63c81ab690 con 0x55ac386389d0 2024-09-11T23:29:19.732 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.722+0000 7f63d03d0740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f63c81ad200 con 0x55ac386392f0 2024-09-11T23:29:19.732 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.722+0000 7f63d03d0740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f63c81abd60 con 0x55ac38645ec0 2024-09-11T23:29:19.732 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.722+0000 7f63c6ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ac386389d0 0x55ac386339a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:19.732 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.722+0000 7f63c6ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ac386389d0 0x55ac386339a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59050/0 (socket says 172.21.15.26:59050) 2024-09-11T23:29:19.733 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.722+0000 7f63c6ffd640 1 -- 172.21.15.26:0/1889601725 learned_addr learned my addr 172.21.15.26:0/1889601725 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:29:19.733 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.722+0000 7f63bffff640 1 --2- 172.21.15.26:0/1889601725 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ac386392f0 0x55ac38640050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:19.733 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.722+0000 7f63c77fe640 1 --2- 172.21.15.26:0/1889601725 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ac38645ec0 0x55ac38640590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:19.733 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.722+0000 7f63c6ffd640 1 -- 172.21.15.26:0/1889601725 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ac38645ec0 msgr2=0x55ac38640590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:29:19.733 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.722+0000 7f63c6ffd640 1 --2- 172.21.15.26:0/1889601725 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ac38645ec0 0x55ac38640590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:19.733 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.722+0000 7f63c6ffd640 1 -- 172.21.15.26:0/1889601725 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ac386392f0 msgr2=0x55ac38640050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:29:19.734 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.722+0000 7f63c6ffd640 1 --2- 172.21.15.26:0/1889601725 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ac386392f0 0x55ac38640050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:19.734 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.722+0000 7f63c6ffd640 1 -- 172.21.15.26:0/1889601725 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac3863db50 con 0x55ac386389d0 2024-09-11T23:29:19.734 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.723+0000 7f63c6ffd640 1 --2- 172.21.15.26:0/1889601725 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ac386389d0 0x55ac386339a0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f63b8005130 tx=0x7f63b8031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1437e3eff1cc09c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:29:19.734 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.723+0000 7f63bcff9640 1 -- 172.21.15.26:0/1889601725 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63b8051040 con 0x55ac386389d0 2024-09-11T23:29:19.734 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.723+0000 7f63bcff9640 1 -- 172.21.15.26:0/1889601725 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63b80317c0 con 0x55ac386389d0 2024-09-11T23:29:19.734 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.723+0000 7f63bcff9640 1 -- 172.21.15.26:0/1889601725 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63b8003db0 con 0x55ac386389d0 2024-09-11T23:29:19.735 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.724+0000 7f63d03d0740 1 -- 172.21.15.26:0/1889601725 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ac386389d0 msgr2=0x55ac386339a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:19.735 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.724+0000 7f63d03d0740 1 --2- 172.21.15.26:0/1889601725 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ac386389d0 0x55ac386339a0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f63b8005130 tx=0x7f63b8031400 comp rx=0 tx=0).stop 2024-09-11T23:29:19.735 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.724+0000 7f63d03d0740 1 -- 172.21.15.26:0/1889601725 shutdown_connections 2024-09-11T23:29:19.735 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.724+0000 7f63d03d0740 1 --2- 172.21.15.26:0/1889601725 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ac386392f0 0x55ac38640050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:19.735 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.724+0000 7f63d03d0740 1 --2- 172.21.15.26:0/1889601725 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ac38645ec0 0x55ac38640590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:19.735 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.724+0000 7f63d03d0740 1 --2- 172.21.15.26:0/1889601725 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ac386389d0 0x55ac386339a0 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:19.736 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.724+0000 7f63d03d0740 1 -- 172.21.15.26:0/1889601725 >> 172.21.15.26:0/1889601725 conn(0x55ac38635060 msgr2=0x55ac3863ad10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:29:19.736 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.724+0000 7f63d03d0740 1 -- 172.21.15.26:0/1889601725 shutdown_connections 2024-09-11T23:29:19.736 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.724+0000 7f63d03d0740 1 -- 172.21.15.26:0/1889601725 wait complete. 2024-09-11T23:29:19.736 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.724+0000 7f63d03d0740 1 -- start start 2024-09-11T23:29:19.736 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.724+0000 7f63d03d0740 1 Processor -- start 2024-09-11T23:29:19.736 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.725+0000 7f63d03d0740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ac386389d0 0x55ac386513b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:19.737 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.725+0000 7f63d03d0740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ac386392f0 0x55ac386518f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:19.737 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.725+0000 7f63d03d0740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ac38645ec0 0x55ac38651e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:19.737 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.725+0000 7f63d03d0740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f63c81acb30 con 0x55ac386392f0 2024-09-11T23:29:19.737 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.725+0000 7f63d03d0740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f63c81ac430 con 0x55ac386389d0 2024-09-11T23:29:19.737 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.725+0000 7f63d03d0740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f63c81a9690 con 0x55ac38645ec0 2024-09-11T23:29:19.738 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.725+0000 7f63d03d0740 10 client.0 _ll_register_callbacks cb 0x7f63b0009990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:29:19.738 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.725+0000 7f63bffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ac386392f0 0x55ac386518f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:19.738 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.725+0000 7f63bffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ac386392f0 0x55ac386518f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59064/0 (socket says 172.21.15.26:59064) 2024-09-11T23:29:19.738 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.725+0000 7f63bffff640 1 -- 172.21.15.26:0/1991269307 learned_addr learned my addr 172.21.15.26:0/1991269307 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:29:19.738 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.725+0000 7f63c6ffd640 1 --2- 172.21.15.26:0/1991269307 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ac386389d0 0x55ac386513b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:19.738 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.725+0000 7f63c77fe640 1 --2- 172.21.15.26:0/1991269307 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ac38645ec0 0x55ac38651e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:19.739 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.725+0000 7f63bffff640 1 -- 172.21.15.26:0/1991269307 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ac38645ec0 msgr2=0x55ac38651e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:29:19.739 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.725+0000 7f63bffff640 1 --2- 172.21.15.26:0/1991269307 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ac38645ec0 0x55ac38651e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:19.739 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.725+0000 7f63bffff640 1 -- 172.21.15.26:0/1991269307 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ac386389d0 msgr2=0x55ac386513b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:29:19.739 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.725+0000 7f63bffff640 1 --2- 172.21.15.26:0/1991269307 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ac386389d0 0x55ac386513b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:19.739 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.725+0000 7f63bffff640 1 -- 172.21.15.26:0/1991269307 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac386c1b00 con 0x55ac386392f0 2024-09-11T23:29:19.739 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.725+0000 7f63c77fe640 1 --2- 172.21.15.26:0/1991269307 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ac38645ec0 0x55ac38651e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:29:19.739 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.725+0000 7f63bffff640 1 --2- 172.21.15.26:0/1991269307 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ac386392f0 0x55ac386518f0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f63b0009fd0 tx=0x7f63b0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62d2cd66f00c80bd server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:29:19.739 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.725+0000 7f63ae7fc640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:29:19.740 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.725+0000 7f63ae7fc640 1 -- 172.21.15.26:0/1991269307 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63b0035420 con 0x55ac386392f0 2024-09-11T23:29:19.740 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.725+0000 7f63ae7fc640 1 -- 172.21.15.26:0/1991269307 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8586+0+0 (secure 0 0 0) 0x7f63b00355c0 con 0x55ac386392f0 2024-09-11T23:29:19.740 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.725+0000 7f63d03d0740 1 -- 172.21.15.26:0/1991269307 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f63b80317c0 con 0x55ac386392f0 2024-09-11T23:29:19.740 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.726+0000 7f63ae7fc640 1 -- 172.21.15.26:0/1991269307 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7f63b00357c0 con 0x55ac386392f0 2024-09-11T23:29:19.740 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.726+0000 7f63d03d0740 10 client.17079 fetch_fsmap learned FSMap version 57 2024-09-11T23:29:19.740 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.726+0000 7f63d03d0740 1 -- 172.21.15.26:0/1991269307 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x55ac386c2e60 con 0x55ac386392f0 2024-09-11T23:29:19.740 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.726+0000 7f63ae7fc640 1 -- 172.21.15.26:0/1991269307 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63b80317c0 con 0x55ac386392f0 2024-09-11T23:29:19.740 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.726+0000 7f63ae7fc640 1 -- 172.21.15.26:0/1991269307 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 57) ==== 8366+0+0 (secure 0 0 0) 0x7f63b000fba0 con 0x55ac386392f0 2024-09-11T23:29:19.740 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.726+0000 7f63d03d0740 10 client.17079 fetch_fsmap finished waiting for FSMap version 57 2024-09-11T23:29:19.740 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.726+0000 7f63d03d0740 10 client.17079 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:29:19.740 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.726+0000 7f63d03d0740 20 client.17079 populate_metadata read hostname 'smithi026' 2024-09-11T23:29:19.740 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.726+0000 7f63d03d0740 1 --2- 172.21.15.26:0/1991269307 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55ac386c35c0 0x55ac386c59e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:19.741 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.726+0000 7f63d03d0740 4 client.17079 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T23:29:19.741 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.726+0000 7f63d03d0740 1 -- 172.21.15.26:0/1991269307 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ac385a4d70 con 0x55ac386c35c0 2024-09-11T23:29:19.741 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.726+0000 7f63c6ffd640 1 --2- 172.21.15.26:0/1991269307 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55ac386c35c0 0x55ac386c59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:19.741 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.727+0000 7f63c6ffd640 1 --2- 172.21.15.26:0/1991269307 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55ac386c35c0 0x55ac386c59e0 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd52fed9726b2fe server_cookie=4452c7639bf6d9f1 in_seq=0 out_seq=0 2024-09-11T23:29:19.741 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.727+0000 7f63ae7fc640 10 client.17079 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:29:19.745 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.743+0000 7f63ae7fc640 1 -- 172.21.15.26:0/1991269307 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ac385a4d70 con 0x55ac386c35c0 2024-09-11T23:29:19.745 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.744+0000 7f63ae7fc640 10 client.17079 handle_command_reply: tid=0 2024-09-11T23:29:19.745 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.744+0000 7f63d03d0740 1 client.17079 shutdown 2024-09-11T23:29:19.745 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.744+0000 7f63d03d0740 1 -- 172.21.15.26:0/1991269307 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ac386392f0 msgr2=0x55ac386518f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:19.745 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.744+0000 7f63d03d0740 1 --2- 172.21.15.26:0/1991269307 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ac386392f0 0x55ac386518f0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f63b0009fd0 tx=0x7f63b0004730 comp rx=0 tx=0).stop 2024-09-11T23:29:19.746 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.744+0000 7f63d03d0740 1 -- 172.21.15.26:0/1991269307 shutdown_connections 2024-09-11T23:29:19.746 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.744+0000 7f63d03d0740 1 --2- 172.21.15.26:0/1991269307 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55ac386c35c0 0x55ac386c59e0 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:19.746 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.744+0000 7f63d03d0740 1 --2- 172.21.15.26:0/1991269307 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ac38645ec0 0x55ac38651e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:19.746 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.744+0000 7f63d03d0740 1 --2- 172.21.15.26:0/1991269307 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ac386392f0 0x55ac386518f0 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:19.746 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.744+0000 7f63ae7fc640 0 client.17079 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:29:19.746 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.744+0000 7f63d03d0740 1 --2- 172.21.15.26:0/1991269307 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ac386389d0 0x55ac386513b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:19.746 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.744+0000 7f63d03d0740 1 -- 172.21.15.26:0/1991269307 >> 172.21.15.26:0/1991269307 conn(0x55ac38635060 msgr2=0x55ac3865f8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:29:19.746 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.744+0000 7f63d03d0740 1 -- 172.21.15.26:0/1991269307 shutdown_connections 2024-09-11T23:29:19.746 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.744+0000 7f63d03d0740 1 -- 172.21.15.26:0/1991269307 wait complete. 2024-09-11T23:29:19.746 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.744+0000 7f63d03d0740 20 client.17079 trim_cache size 0 max 16384 2024-09-11T23:29:19.746 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-09-11T23:29:19.747 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "message": "", 2024-09-11T23:29:19.747 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-09-11T23:29:19.747 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-09-11T23:29:19.748 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.744+0000 7f63cd559640 1 -- 172.21.15.26:0/1960211098 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f63b40a2da0 msgr2=0x7f63b40a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:19.748 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.744+0000 7f63cd559640 1 --2- 172.21.15.26:0/1960211098 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f63b40a2da0 0x7f63b40a5260 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x55ac386363e0 tx=0x7f63b0009640 comp rx=0 tx=0).stop 2024-09-11T23:29:19.748 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.745+0000 7f63cd559640 1 -- 172.21.15.26:0/1960211098 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63c81069e0 msgr2=0x7f63c819bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:19.748 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.745+0000 7f63cd559640 1 --2- 172.21.15.26:0/1960211098 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63c81069e0 0x7f63c819bb30 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f63c000eb10 tx=0x7f63c000c2d0 comp rx=0 tx=0).stop 2024-09-11T23:29:19.748 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.745+0000 7f63cd559640 1 -- 172.21.15.26:0/1960211098 shutdown_connections 2024-09-11T23:29:19.748 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.745+0000 7f63cd559640 1 --2- 172.21.15.26:0/1960211098 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f63b40a2da0 0x7f63b40a5260 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:19.748 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.745+0000 7f63cd559640 1 --2- 172.21.15.26:0/1960211098 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63c81069e0 0x7f63c819bb30 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:19.748 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.745+0000 7f63cd559640 1 --2- 172.21.15.26:0/1960211098 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f63c8101370 0x7f63c81977b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:19.748 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.745+0000 7f63cd559640 1 --2- 172.21.15.26:0/1960211098 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f63c81009d0 0x7f63c8197270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:19.748 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.745+0000 7f63cd559640 1 -- 172.21.15.26:0/1960211098 >> 172.21.15.26:0/1960211098 conn(0x7f63c80fc6f0 msgr2=0x7f63c80fcdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:29:19.748 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.745+0000 7f63cd559640 1 -- 172.21.15.26:0/1960211098 shutdown_connections 2024-09-11T23:29:19.748 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:19.745+0000 7f63cd559640 1 -- 172.21.15.26:0/1960211098 wait complete. 2024-09-11T23:29:19.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:19.754+0000 7f9070ba5640 1 -- 172.21.15.26:0/1603270173 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f906c02d7c0 con 0x7f906c102d40 2024-09-11T23:29:19.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:19.755+0000 7f905b7fe640 1 -- 172.21.15.26:0/1603270173 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) ==== 94+0+11973 (secure 0 0 0) 0x7f905c09c0f0 con 0x7f906c102d40 2024-09-11T23:29:19.756 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-11T23:29:19.756 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":57,"btime":"2024-09-11T23:29:06:826223+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14499,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/367659515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":367659515},{"type":"v1","addr":"172.21.15.26:6835","nonce":367659515}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14511,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/2041065299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":2041065299},{"type":"v1","addr":"172.21.15.26:6837","nonce":2041065299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":57,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:16:26.544203+0000","modified":"2024-09-11T23:29:06.658248+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24485,"mds_1":24463,"mds_2":24458,"mds_3":24448,"mds_4":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14550":{"gid":14550,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6839/2233068305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":2233068305},{"type":"v1","addr":"172.21.15.26:6839","nonce":2233068305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14562":{"gid":14562,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6841/2984982336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":2984982336},{"type":"v1","addr":"172.21.15.26:6841","nonce":2984982336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6833/2019841455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":2019841455},{"type":"v1","addr":"172.21.15.136:6833","nonce":2019841455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.76:6833/3438147050","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3438147050},{"type":"v1","addr":"172.21.15.76:6833","nonce":3438147050}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6835/3617665975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3617665975},{"type":"v1","addr":"172.21.15.136:6835","nonce":3617665975}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":12,"addr":"172.21.15.76:6835/393308499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":393308499},{"type":"v1","addr":"172.21.15.76:6835","nonce":393308499}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":10,"addr":"172.21.15.136:6837/1633082387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":1633082387},{"type":"v1","addr":"172.21.15.136:6837","nonce":1633082387}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.76:6837/2078978748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":2078978748},{"type":"v1","addr":"172.21.15.76:6837","nonce":2078978748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":63,"addr":"172.21.15.136:6839/3211398630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":3211398630},{"type":"v1","addr":"172.21.15.136:6839","nonce":3211398630}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.76:6839/2077637090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6838","nonce":2077637090},{"type":"v1","addr":"172.21.15.76:6839","nonce":2077637090}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24485,"qdb_cluster":[24443,24448,24458,24463,24485]},"id":1}]} 2024-09-11T23:29:19.756 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 57 2024-09-11T23:29:19.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:19.757+0000 7f9070ba5640 1 -- 172.21.15.26:0/1603270173 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f90540a2cf0 msgr2=0x7f90540a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:19.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:19.757+0000 7f9070ba5640 1 --2- 172.21.15.26:0/1603270173 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f90540a2cf0 0x7f90540a51b0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f906c10d4b0 tx=0x7f9060005f90 comp rx=0 tx=0).stop 2024-09-11T23:29:19.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:19.757+0000 7f9070ba5640 1 -- 172.21.15.26:0/1603270173 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f906c102d40 msgr2=0x7f906c1983d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:19.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:19.757+0000 7f9070ba5640 1 --2- 172.21.15.26:0/1603270173 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f906c102d40 0x7f906c1983d0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f905c00c920 tx=0x7f905c00cde0 comp rx=0 tx=0).stop 2024-09-11T23:29:19.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:19.757+0000 7f9070ba5640 1 -- 172.21.15.26:0/1603270173 shutdown_connections 2024-09-11T23:29:19.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:19.757+0000 7f9070ba5640 1 --2- 172.21.15.26:0/1603270173 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f90540a2cf0 0x7f90540a51b0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:19.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:19.757+0000 7f9070ba5640 1 --2- 172.21.15.26:0/1603270173 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f906c103660 0x7f906c198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:19.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:19.757+0000 7f9070ba5640 1 --2- 172.21.15.26:0/1603270173 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f906c10c170 0x7f906c19cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:19.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:19.757+0000 7f9070ba5640 1 --2- 172.21.15.26:0/1603270173 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f906c102d40 0x7f906c1983d0 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:19.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:19.757+0000 7f9070ba5640 1 -- 172.21.15.26:0/1603270173 >> 172.21.15.26:0/1603270173 conn(0x7f906c0fc7a0 msgr2=0x7f906c0fd400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:29:19.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:19.757+0000 7f9070ba5640 1 -- 172.21.15.26:0/1603270173 shutdown_connections 2024-09-11T23:29:19.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:19.757+0000 7f9070ba5640 1 -- 172.21.15.26:0/1603270173 wait complete. 2024-09-11T23:29:20.058 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:29:19 smithi026 ceph-mon[31172]: pgmap v340: 97 pgs: 97 active+clean; 2.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 34 KiB/s wr, 10 op/s 2024-09-11T23:29:20.059 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:29:19 smithi026 ceph-mon[31172]: from='client.? 172.21.15.26:0/1603270173' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:29:20.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:29:19 smithi136 ceph-mon[35121]: pgmap v340: 97 pgs: 97 active+clean; 2.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 34 KiB/s wr, 10 op/s 2024-09-11T23:29:20.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:29:19 smithi136 ceph-mon[35121]: from='client.? 172.21.15.26:0/1603270173' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:29:20.305 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:29:19 smithi076 ceph-mon[35296]: pgmap v340: 97 pgs: 97 active+clean; 2.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 34 KiB/s wr, 10 op/s 2024-09-11T23:29:20.305 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:29:19 smithi076 ceph-mon[35296]: from='client.? 172.21.15.26:0/1603270173' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:29:20.514 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24485 quiesce db / --timeout 90 --expiration 142.55 --await-for 45.0' 2024-09-11T23:29:20.515 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24485 quiesce db / --timeout 90 --expiration 142.55 --await-for 45.0 2024-09-11T23:29:20.777 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:29:21.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.411+0000 7f25d7ddf640 1 -- 172.21.15.26:0/1624054441 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25d0101380 msgr2=0x7f25d010dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:21.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.411+0000 7f25d7ddf640 1 --2- 172.21.15.26:0/1624054441 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25d0101380 0x7f25d010dfb0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f25c0009900 tx=0x7f25c002a410 comp rx=0 tx=0).stop 2024-09-11T23:29:21.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.411+0000 7f25d7ddf640 1 -- 172.21.15.26:0/1624054441 shutdown_connections 2024-09-11T23:29:21.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.411+0000 7f25d7ddf640 1 --2- 172.21.15.26:0/1624054441 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f25d010e4f0 0x7f25d0110900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:21.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.411+0000 7f25d7ddf640 1 --2- 172.21.15.26:0/1624054441 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25d0101380 0x7f25d010dfb0 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:21.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.411+0000 7f25d7ddf640 1 --2- 172.21.15.26:0/1624054441 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f25d01009d0 0x7f25d0100db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:21.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.411+0000 7f25d7ddf640 1 -- 172.21.15.26:0/1624054441 >> 172.21.15.26:0/1624054441 conn(0x7f25d00fc7d0 msgr2=0x7f25d00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:29:21.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.411+0000 7f25d7ddf640 1 -- 172.21.15.26:0/1624054441 shutdown_connections 2024-09-11T23:29:21.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.411+0000 7f25d7ddf640 1 -- 172.21.15.26:0/1624054441 wait complete. 2024-09-11T23:29:21.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.412+0000 7f25d7ddf640 1 Processor -- start 2024-09-11T23:29:21.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.412+0000 7f25d7ddf640 1 -- start start 2024-09-11T23:29:21.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.412+0000 7f25d7ddf640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f25d01009d0 0x7f25d019a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:21.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.412+0000 7f25d7ddf640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25d0101380 0x7f25d019ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:21.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.412+0000 7f25d7ddf640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f25d010e4f0 0x7f25d019eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:21.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.412+0000 7f25d7ddf640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f25d0114260 con 0x7f25d0101380 2024-09-11T23:29:21.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.412+0000 7f25d7ddf640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f25d01140e0 con 0x7f25d010e4f0 2024-09-11T23:29:21.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.412+0000 7f25d7ddf640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f25d01143e0 con 0x7f25d01009d0 2024-09-11T23:29:21.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.412+0000 7f25d537a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25d0101380 0x7f25d019ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:21.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.412+0000 7f25d537a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25d0101380 0x7f25d019ac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50254/0 (socket says 172.21.15.26:50254) 2024-09-11T23:29:21.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.412+0000 7f25d537a640 1 -- 172.21.15.26:0/3528034187 learned_addr learned my addr 172.21.15.26:0/3528034187 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:29:21.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.412+0000 7f25d5b7b640 1 --2- 172.21.15.26:0/3528034187 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f25d01009d0 0x7f25d019a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:21.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.412+0000 7f25d637c640 1 --2- 172.21.15.26:0/3528034187 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f25d010e4f0 0x7f25d019eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:21.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.412+0000 7f25d537a640 1 -- 172.21.15.26:0/3528034187 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f25d01009d0 msgr2=0x7f25d019a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:21.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.412+0000 7f25d537a640 1 --2- 172.21.15.26:0/3528034187 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f25d01009d0 0x7f25d019a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:21.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.412+0000 7f25d537a640 1 -- 172.21.15.26:0/3528034187 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f25d010e4f0 msgr2=0x7f25d019eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:21.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.412+0000 7f25d537a640 1 --2- 172.21.15.26:0/3528034187 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f25d010e4f0 0x7f25d019eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:21.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.412+0000 7f25d537a640 1 -- 172.21.15.26:0/3528034187 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25d019f6d0 con 0x7f25d0101380 2024-09-11T23:29:21.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.412+0000 7f25d637c640 1 --2- 172.21.15.26:0/3528034187 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f25d010e4f0 0x7f25d019eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:29:21.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.413+0000 7f25d537a640 1 --2- 172.21.15.26:0/3528034187 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25d0101380 0x7f25d019ac60 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f25c0006010 tx=0x7f25c00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:29:21.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.413+0000 7f25c6ffd640 1 -- 172.21.15.26:0/3528034187 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25c003f030 con 0x7f25d0101380 2024-09-11T23:29:21.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.413+0000 7f25c6ffd640 1 -- 172.21.15.26:0/3528034187 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f25c0003ce0 con 0x7f25d0101380 2024-09-11T23:29:21.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.413+0000 7f25d7ddf640 1 -- 172.21.15.26:0/3528034187 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25d019f960 con 0x7f25d0101380 2024-09-11T23:29:21.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.413+0000 7f25c6ffd640 1 -- 172.21.15.26:0/3528034187 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25c0035070 con 0x7f25d0101380 2024-09-11T23:29:21.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.413+0000 7f25d7ddf640 1 -- 172.21.15.26:0/3528034187 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25d01a72a0 con 0x7f25d0101380 2024-09-11T23:29:21.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.415+0000 7f25c6ffd640 1 -- 172.21.15.26:0/3528034187 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f25c0042030 con 0x7f25d0101380 2024-09-11T23:29:21.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.415+0000 7f25da1b8740 1 Processor -- start 2024-09-11T23:29:21.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.416+0000 7f25da1b8740 1 -- start start 2024-09-11T23:29:21.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.416+0000 7f25da1b8740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55db04f91850 0x55db04f91c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:21.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.416+0000 7f25da1b8740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55db04f92170 0x55db04f9c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:21.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.416+0000 7f25da1b8740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55db04f9ca30 0x55db04f9ee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:21.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.416+0000 7f25da1b8740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55db04ef6160 con 0x55db04f9ca30 2024-09-11T23:29:21.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.416+0000 7f25da1b8740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55db04e9d720 con 0x55db04f92170 2024-09-11T23:29:21.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.416+0000 7f25da1b8740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55db04ef2120 con 0x55db04f91850 2024-09-11T23:29:21.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.416+0000 7f25c6ffd640 1 --2- 172.21.15.26:0/3528034187 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f25bc0a2cf0 0x7f25bc0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:21.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.416+0000 7f25d537a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55db04f9ca30 0x55db04f9ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:21.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.416+0000 7f25d5b7b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55db04f92170 0x55db04f9c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:21.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.416+0000 7f25d537a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55db04f9ca30 0x55db04f9ee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50268/0 (socket says 172.21.15.26:50268) 2024-09-11T23:29:21.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.416+0000 7f25d5b7b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55db04f92170 0x55db04f9c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:52648/0 (socket says 172.21.15.26:52648) 2024-09-11T23:29:21.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.416+0000 7f25d537a640 1 -- 172.21.15.26:0/3934736093 learned_addr learned my addr 172.21.15.26:0/3934736093 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:29:21.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.416+0000 7f25d637c640 1 --2- 172.21.15.26:0/3528034187 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f25bc0a2cf0 0x7f25bc0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:21.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.416+0000 7f25d637c640 1 --2- 172.21.15.26:0/3934736093 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55db04f91850 0x55db04f91c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:21.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.416+0000 7f25c6ffd640 1 -- 172.21.15.26:0/3528034187 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8542+0+0 (secure 0 0 0) 0x7f25c00ef640 con 0x7f25d0101380 2024-09-11T23:29:21.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.417+0000 7f25d637c640 1 -- 172.21.15.26:0/3934736093 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55db04f92170 msgr2=0x55db04f9c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:29:21.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.417+0000 7f25d637c640 1 --2- 172.21.15.26:0/3934736093 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55db04f92170 0x55db04f9c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:21.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.417+0000 7f25d637c640 1 -- 172.21.15.26:0/3934736093 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55db04f9ca30 msgr2=0x55db04f9ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:29:21.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.417+0000 7f25d637c640 1 --2- 172.21.15.26:0/3934736093 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55db04f9ca30 0x55db04f9ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:21.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.417+0000 7f25d637c640 1 -- 172.21.15.26:0/3934736093 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db04c67ba0 con 0x55db04f91850 2024-09-11T23:29:21.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.417+0000 7f25d537a640 1 --2- 172.21.15.26:0/3934736093 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55db04f9ca30 0x55db04f9ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:29:21.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.417+0000 7f25d637c640 1 --2- 172.21.15.26:0/3528034187 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f25bc0a2cf0 0x7f25bc0a51b0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f25c8005cc0 tx=0x7f25c8015000 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:29:21.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.417+0000 7f25d637c640 1 --2- 172.21.15.26:0/3934736093 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55db04f91850 0x55db04f91c30 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f25d0069760 tx=0x7f25c80107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fda86e9a834cde7c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:29:21.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.417+0000 7f25bbfff640 1 -- 172.21.15.26:0/3934736093 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25c8021070 con 0x55db04f91850 2024-09-11T23:29:21.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.417+0000 7f25bbfff640 1 -- 172.21.15.26:0/3934736093 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f25c800de10 con 0x55db04f91850 2024-09-11T23:29:21.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.417+0000 7f25bbfff640 1 -- 172.21.15.26:0/3934736093 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25c8005910 con 0x55db04f91850 2024-09-11T23:29:21.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.418+0000 7f25da1b8740 1 -- 172.21.15.26:0/3934736093 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55db04f91850 msgr2=0x55db04f91c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:21.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.418+0000 7f25da1b8740 1 --2- 172.21.15.26:0/3934736093 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55db04f91850 0x55db04f91c30 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f25d0069760 tx=0x7f25c80107a0 comp rx=0 tx=0).stop 2024-09-11T23:29:21.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.418+0000 7f25da1b8740 1 -- 172.21.15.26:0/3934736093 shutdown_connections 2024-09-11T23:29:21.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.418+0000 7f25da1b8740 1 --2- 172.21.15.26:0/3934736093 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55db04f92170 0x55db04f9c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:21.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.418+0000 7f25da1b8740 1 --2- 172.21.15.26:0/3934736093 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55db04f9ca30 0x55db04f9ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:21.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.418+0000 7f25da1b8740 1 --2- 172.21.15.26:0/3934736093 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55db04f91850 0x55db04f91c30 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:21.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.418+0000 7f25da1b8740 1 -- 172.21.15.26:0/3934736093 >> 172.21.15.26:0/3934736093 conn(0x55db04f8dde0 msgr2=0x55db04f8e1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:29:21.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.418+0000 7f25da1b8740 1 -- 172.21.15.26:0/3934736093 shutdown_connections 2024-09-11T23:29:21.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.418+0000 7f25da1b8740 1 -- 172.21.15.26:0/3934736093 wait complete. 2024-09-11T23:29:21.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.418+0000 7f25da1b8740 1 -- start start 2024-09-11T23:29:21.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.419+0000 7f25da1b8740 1 Processor -- start 2024-09-11T23:29:21.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.419+0000 7f25da1b8740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55db04f91850 0x55db04f99930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:21.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.419+0000 7f25da1b8740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55db04f92170 0x55db04f99e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:21.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.419+0000 7f25da1b8740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55db04f9ca30 0x55db04f9a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:21.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.419+0000 7f25da1b8740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55db04f9f600 con 0x55db04f9ca30 2024-09-11T23:29:21.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.419+0000 7f25da1b8740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55db04e7e900 con 0x55db04f92170 2024-09-11T23:29:21.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.419+0000 7f25da1b8740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55db04f9f780 con 0x55db04f91850 2024-09-11T23:29:21.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.419+0000 7f25da1b8740 10 client.0 _ll_register_callbacks cb 0x55db04e7c780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:29:21.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.419+0000 7f25d637c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55db04f9ca30 0x55db04f9a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:21.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.419+0000 7f25d637c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55db04f9ca30 0x55db04f9a3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50280/0 (socket says 172.21.15.26:50280) 2024-09-11T23:29:21.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.419+0000 7f25d637c640 1 -- 172.21.15.26:0/2751506241 learned_addr learned my addr 172.21.15.26:0/2751506241 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:29:21.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.419+0000 7f25d537a640 1 --2- 172.21.15.26:0/2751506241 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55db04f92170 0x55db04f99e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:21.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.419+0000 7f25d5b7b640 1 --2- 172.21.15.26:0/2751506241 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55db04f91850 0x55db04f99930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:21.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.419+0000 7f25d637c640 1 -- 172.21.15.26:0/2751506241 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55db04f91850 msgr2=0x55db04f99930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:29:21.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.419+0000 7f25d637c640 1 --2- 172.21.15.26:0/2751506241 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55db04f91850 0x55db04f99930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:21.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.419+0000 7f25d637c640 1 -- 172.21.15.26:0/2751506241 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55db04f92170 msgr2=0x55db04f99e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:29:21.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.419+0000 7f25d637c640 1 --2- 172.21.15.26:0/2751506241 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55db04f92170 0x55db04f99e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:21.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.420+0000 7f25d637c640 1 -- 172.21.15.26:0/2751506241 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db04ebec50 con 0x55db04f9ca30 2024-09-11T23:29:21.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.420+0000 7f25d537a640 1 --2- 172.21.15.26:0/2751506241 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55db04f92170 0x55db04f99e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:29:21.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.420+0000 7f25d637c640 1 --2- 172.21.15.26:0/2751506241 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55db04f9ca30 0x55db04f9a3b0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f25c8005c90 tx=0x7f25c8004c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9151aa0acbd951d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:29:21.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.420+0000 7f25b97fa640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:29:21.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.420+0000 7f25b97fa640 1 -- 172.21.15.26:0/2751506241 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25c8021040 con 0x55db04f9ca30 2024-09-11T23:29:21.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.420+0000 7f25b97fa640 1 -- 172.21.15.26:0/2751506241 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8542+0+0 (secure 0 0 0) 0x7f25c800dc10 con 0x55db04f9ca30 2024-09-11T23:29:21.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.420+0000 7f25da1b8740 1 -- 172.21.15.26:0/2751506241 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f25c800de10 con 0x55db04f9ca30 2024-09-11T23:29:21.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.420+0000 7f25b97fa640 1 -- 172.21.15.26:0/2751506241 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7f25c8013260 con 0x55db04f9ca30 2024-09-11T23:29:21.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.420+0000 7f25da1b8740 10 client.17091 fetch_fsmap learned FSMap version 57 2024-09-11T23:29:21.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.420+0000 7f25da1b8740 1 -- 172.21.15.26:0/2751506241 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x55db04fab2c0 con 0x55db04f9ca30 2024-09-11T23:29:21.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.420+0000 7f25b97fa640 1 -- 172.21.15.26:0/2751506241 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f25c800de10 con 0x55db04f9ca30 2024-09-11T23:29:21.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.421+0000 7f25b97fa640 1 -- 172.21.15.26:0/2751506241 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 57) ==== 8366+0+0 (secure 0 0 0) 0x7f25c801c070 con 0x55db04f9ca30 2024-09-11T23:29:21.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.421+0000 7f25da1b8740 10 client.17091 fetch_fsmap finished waiting for FSMap version 57 2024-09-11T23:29:21.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.421+0000 7f25da1b8740 10 client.17091 resolve_mds: validated gid 24485 aka daemon mds.b 2024-09-11T23:29:21.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.421+0000 7f25da1b8740 20 client.17091 populate_metadata read hostname 'smithi026' 2024-09-11T23:29:21.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.421+0000 7f25da1b8740 1 --2- 172.21.15.26:0/2751506241 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55db04fac780 0x55db04faeb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:21.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.421+0000 7f25da1b8740 4 client.17091 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:29:21.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.421+0000 7f25da1b8740 1 -- 172.21.15.26:0/2751506241 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55db04f03890 con 0x55db04fac780 2024-09-11T23:29:21.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.421+0000 7f25d5b7b640 1 --2- 172.21.15.26:0/2751506241 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55db04fac780 0x55db04faeb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:21.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.421+0000 7f25d5b7b640 1 --2- 172.21.15.26:0/2751506241 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55db04fac780 0x55db04faeb70 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b51a79c0e61a18b server_cookie=9551d72d8ac6e05c in_seq=0 out_seq=0 2024-09-11T23:29:21.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.421+0000 7f25b97fa640 10 client.17091 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:29:21.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.422+0000 7f25b97fa640 1 -- 172.21.15.26:0/2751506241 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55db04f03890 con 0x55db04fac780 2024-09-11T23:29:21.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.422+0000 7f25b97fa640 10 client.17091 handle_command_reply: tid=0 2024-09-11T23:29:21.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.422+0000 7f25da1b8740 1 client.17091 shutdown 2024-09-11T23:29:21.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.422+0000 7f25da1b8740 1 -- 172.21.15.26:0/2751506241 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55db04f9ca30 msgr2=0x55db04f9a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:21.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.422+0000 7f25da1b8740 1 --2- 172.21.15.26:0/2751506241 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55db04f9ca30 0x55db04f9a3b0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f25c8005c90 tx=0x7f25c8004c30 comp rx=0 tx=0).stop 2024-09-11T23:29:21.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.422+0000 7f25da1b8740 1 -- 172.21.15.26:0/2751506241 shutdown_connections 2024-09-11T23:29:21.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.422+0000 7f25da1b8740 1 --2- 172.21.15.26:0/2751506241 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55db04f92170 0x55db04f99e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:21.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.422+0000 7f25da1b8740 1 --2- 172.21.15.26:0/2751506241 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55db04fac780 0x55db04faeb70 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:21.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.422+0000 7f25da1b8740 1 --2- 172.21.15.26:0/2751506241 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55db04f9ca30 0x55db04f9a3b0 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:21.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.422+0000 7f25da1b8740 1 --2- 172.21.15.26:0/2751506241 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55db04f91850 0x55db04f99930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:21.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.422+0000 7f25b97fa640 0 client.17091 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:29:21.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.422+0000 7f25da1b8740 1 -- 172.21.15.26:0/2751506241 >> 172.21.15.26:0/2751506241 conn(0x55db04f8dde0 msgr2=0x55db04f93b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:29:21.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.422+0000 7f25da1b8740 1 -- 172.21.15.26:0/2751506241 shutdown_connections 2024-09-11T23:29:21.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.423+0000 7f25da1b8740 1 -- 172.21.15.26:0/2751506241 wait complete. 2024-09-11T23:29:21.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.423+0000 7f25da1b8740 20 client.17091 trim_cache size 0 max 16384 2024-09-11T23:29:21.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.440+0000 7f25da1b8740 1 Processor -- start 2024-09-11T23:29:21.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.440+0000 7f25da1b8740 1 -- start start 2024-09-11T23:29:21.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.440+0000 7f25da1b8740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55db04f91850 0x55db04f98870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:21.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.440+0000 7f25da1b8740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55db04f92170 0x55db04f88d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:21.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.440+0000 7f25da1b8740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55db04f9ca30 0x55db04fb4140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:21.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.440+0000 7f25da1b8740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f25d01afac0 con 0x55db04f92170 2024-09-11T23:29:21.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.440+0000 7f25da1b8740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f25d01b1630 con 0x55db04f91850 2024-09-11T23:29:21.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.440+0000 7f25da1b8740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f25d01b0190 con 0x55db04f9ca30 2024-09-11T23:29:21.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.440+0000 7f25d537a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55db04f92170 0x55db04f88d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:21.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.440+0000 7f25d537a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55db04f92170 0x55db04f88d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50296/0 (socket says 172.21.15.26:50296) 2024-09-11T23:29:21.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.440+0000 7f25d537a640 1 -- 172.21.15.26:0/2170121652 learned_addr learned my addr 172.21.15.26:0/2170121652 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:29:21.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.440+0000 7f25d5b7b640 1 --2- 172.21.15.26:0/2170121652 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55db04f91850 0x55db04f98870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:21.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.440+0000 7f25d637c640 1 --2- 172.21.15.26:0/2170121652 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55db04f9ca30 0x55db04fb4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:21.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.440+0000 7f25d537a640 1 -- 172.21.15.26:0/2170121652 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55db04f9ca30 msgr2=0x55db04fb4140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:29:21.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.440+0000 7f25d537a640 1 --2- 172.21.15.26:0/2170121652 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55db04f9ca30 0x55db04fb4140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:21.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.440+0000 7f25d537a640 1 -- 172.21.15.26:0/2170121652 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55db04f91850 msgr2=0x55db04f98870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:29:21.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.440+0000 7f25d537a640 1 --2- 172.21.15.26:0/2170121652 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55db04f91850 0x55db04f98870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:21.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.440+0000 7f25d537a640 1 -- 172.21.15.26:0/2170121652 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db04f91680 con 0x55db04f92170 2024-09-11T23:29:21.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.440+0000 7f25d5b7b640 1 --2- 172.21.15.26:0/2170121652 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55db04f91850 0x55db04f98870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:29:21.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.440+0000 7f25d537a640 1 --2- 172.21.15.26:0/2170121652 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55db04f92170 0x55db04f88d30 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f25c008ed10 tx=0x7f25c008ed40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c95cb4d1a556d818 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:29:21.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.441+0000 7f25bb7fe640 1 -- 172.21.15.26:0/2170121652 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25c0049070 con 0x55db04f92170 2024-09-11T23:29:21.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.441+0000 7f25bb7fe640 1 -- 172.21.15.26:0/2170121652 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f25c003b580 con 0x55db04f92170 2024-09-11T23:29:21.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.441+0000 7f25bb7fe640 1 -- 172.21.15.26:0/2170121652 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25c00b8280 con 0x55db04f92170 2024-09-11T23:29:21.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.441+0000 7f25da1b8740 1 -- 172.21.15.26:0/2170121652 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55db04f92170 msgr2=0x55db04f88d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:21.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.441+0000 7f25da1b8740 1 --2- 172.21.15.26:0/2170121652 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55db04f92170 0x55db04f88d30 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f25c008ed10 tx=0x7f25c008ed40 comp rx=0 tx=0).stop 2024-09-11T23:29:21.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.441+0000 7f25da1b8740 1 -- 172.21.15.26:0/2170121652 shutdown_connections 2024-09-11T23:29:21.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.441+0000 7f25da1b8740 1 --2- 172.21.15.26:0/2170121652 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55db04f9ca30 0x55db04fb4140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:21.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.441+0000 7f25da1b8740 1 --2- 172.21.15.26:0/2170121652 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55db04f92170 0x55db04f88d30 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:21.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.441+0000 7f25da1b8740 1 --2- 172.21.15.26:0/2170121652 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55db04f91850 0x55db04f98870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:21.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.441+0000 7f25da1b8740 1 -- 172.21.15.26:0/2170121652 >> 172.21.15.26:0/2170121652 conn(0x55db04f8dde0 msgr2=0x55db04f9b9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:29:21.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.441+0000 7f25da1b8740 1 -- 172.21.15.26:0/2170121652 shutdown_connections 2024-09-11T23:29:21.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.441+0000 7f25da1b8740 1 -- 172.21.15.26:0/2170121652 wait complete. 2024-09-11T23:29:21.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.442+0000 7f25da1b8740 1 -- start start 2024-09-11T23:29:21.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.442+0000 7f25da1b8740 1 Processor -- start 2024-09-11T23:29:21.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.442+0000 7f25da1b8740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55db04f91850 0x55db04f969a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:21.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.442+0000 7f25da1b8740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55db04f92170 0x55db04f96ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:21.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.442+0000 7f25da1b8740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55db04f9ca30 0x55db04f97420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:21.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.442+0000 7f25da1b8740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f25d01b0f60 con 0x55db04f91850 2024-09-11T23:29:21.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.442+0000 7f25da1b8740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f25d01b0860 con 0x55db04f9ca30 2024-09-11T23:29:21.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.442+0000 7f25da1b8740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f25d01adac0 con 0x55db04f92170 2024-09-11T23:29:21.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.442+0000 7f25da1b8740 10 client.0 _ll_register_callbacks cb 0x7f25c00b7f80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:29:21.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.442+0000 7f25d5b7b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55db04f91850 0x55db04f969a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:21.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.442+0000 7f25d5b7b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55db04f91850 0x55db04f969a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50302/0 (socket says 172.21.15.26:50302) 2024-09-11T23:29:21.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.442+0000 7f25d5b7b640 1 -- 172.21.15.26:0/2269247046 learned_addr learned my addr 172.21.15.26:0/2269247046 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:29:21.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.442+0000 7f25d637c640 1 --2- 172.21.15.26:0/2269247046 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55db04f9ca30 0x55db04f97420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:21.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.442+0000 7f25d537a640 1 --2- 172.21.15.26:0/2269247046 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55db04f92170 0x55db04f96ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:21.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.442+0000 7f25d5b7b640 1 -- 172.21.15.26:0/2269247046 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55db04f92170 msgr2=0x55db04f96ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:29:21.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.442+0000 7f25d5b7b640 1 --2- 172.21.15.26:0/2269247046 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55db04f92170 0x55db04f96ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:21.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.443+0000 7f25d5b7b640 1 -- 172.21.15.26:0/2269247046 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55db04f9ca30 msgr2=0x55db04f97420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:29:21.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.443+0000 7f25d5b7b640 1 --2- 172.21.15.26:0/2269247046 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55db04f9ca30 0x55db04f97420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:21.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.443+0000 7f25d5b7b640 1 -- 172.21.15.26:0/2269247046 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db04f8ee90 con 0x55db04f91850 2024-09-11T23:29:21.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.443+0000 7f25d637c640 1 --2- 172.21.15.26:0/2269247046 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55db04f9ca30 0x55db04f97420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:29:21.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.443+0000 7f25d5b7b640 1 --2- 172.21.15.26:0/2269247046 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55db04f91850 0x55db04f969a0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f25cc005fd0 tx=0x7f25cc00bf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ec71a36c7fa274f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:29:21.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.443+0000 7f25b97fa640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:29:21.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.443+0000 7f25b97fa640 1 -- 172.21.15.26:0/2269247046 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25cc005220 con 0x55db04f91850 2024-09-11T23:29:21.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.443+0000 7f25b97fa640 1 -- 172.21.15.26:0/2269247046 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8542+0+0 (secure 0 0 0) 0x7f25cc0053c0 con 0x55db04f91850 2024-09-11T23:29:21.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.443+0000 7f25da1b8740 1 -- 172.21.15.26:0/2269247046 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55db04efdbe0 con 0x55db04f91850 2024-09-11T23:29:21.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.443+0000 7f25b97fa640 1 -- 172.21.15.26:0/2269247046 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7f25cc0055c0 con 0x55db04f91850 2024-09-11T23:29:21.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.443+0000 7f25da1b8740 10 client.17097 fetch_fsmap learned FSMap version 57 2024-09-11T23:29:21.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.443+0000 7f25da1b8740 1 -- 172.21.15.26:0/2269247046 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x55db050172d0 con 0x55db04f91850 2024-09-11T23:29:21.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.443+0000 7f25b97fa640 1 -- 172.21.15.26:0/2269247046 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55db04efdbe0 con 0x55db04f91850 2024-09-11T23:29:21.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.443+0000 7f25b97fa640 1 -- 172.21.15.26:0/2269247046 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 57) ==== 8366+0+0 (secure 0 0 0) 0x7f25cc0268a0 con 0x55db04f91850 2024-09-11T23:29:21.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.443+0000 7f25da1b8740 10 client.17097 fetch_fsmap finished waiting for FSMap version 57 2024-09-11T23:29:21.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.444+0000 7f25da1b8740 10 client.17097 resolve_mds: validated gid 24485 aka daemon mds.b 2024-09-11T23:29:21.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.444+0000 7f25da1b8740 20 client.17097 populate_metadata read hostname 'smithi026' 2024-09-11T23:29:21.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.444+0000 7f25da1b8740 1 --2- 172.21.15.26:0/2269247046 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55db05017aa0 0x55db05019e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:21.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.444+0000 7f25da1b8740 4 client.17097 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.55, "await_for": 45.0}] 2024-09-11T23:29:21.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.444+0000 7f25da1b8740 1 -- 172.21.15.26:0/2269247046 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.55, "await_for": 45.0}) -- 0x55db04e79f40 con 0x55db05017aa0 2024-09-11T23:29:21.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.444+0000 7f25d537a640 1 --2- 172.21.15.26:0/2269247046 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55db05017aa0 0x55db05019e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:21.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.444+0000 7f25d537a640 1 --2- 172.21.15.26:0/2269247046 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55db05017aa0 0x55db05019e90 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8af20a5a0577eae server_cookie=b3cefa8de40ddcad in_seq=0 out_seq=0 2024-09-11T23:29:21.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:21.444+0000 7f25b97fa640 10 client.17097 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:29:22.041 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:29:21 smithi136 ceph-mon[35121]: pgmap v341: 97 pgs: 97 active+clean; 2.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 36 KiB/s wr, 14 op/s 2024-09-11T23:29:22.041 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:29:21 smithi136 ceph-mon[35121]: osdmap e96: 12 total, 12 up, 12 in 2024-09-11T23:29:22.052 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:29:21 smithi076 ceph-mon[35296]: pgmap v341: 97 pgs: 97 active+clean; 2.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 36 KiB/s wr, 14 op/s 2024-09-11T23:29:22.052 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:29:21 smithi076 ceph-mon[35296]: osdmap e96: 12 total, 12 up, 12 in 2024-09-11T23:29:22.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:29:21 smithi026 ceph-mon[31172]: pgmap v341: 97 pgs: 97 active+clean; 2.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 36 KiB/s wr, 14 op/s 2024-09-11T23:29:22.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:29:21 smithi026 ceph-mon[31172]: osdmap e96: 12 total, 12 up, 12 in 2024-09-11T23:29:24.052 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:29:23 smithi076 ceph-mon[35296]: pgmap v343: 97 pgs: 97 active+clean; 2.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.9 KiB/s wr, 11 op/s 2024-09-11T23:29:24.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:29:23 smithi136 ceph-mon[35121]: pgmap v343: 97 pgs: 97 active+clean; 2.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.9 KiB/s wr, 11 op/s 2024-09-11T23:29:24.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:29:23 smithi026 ceph-mon[31172]: pgmap v343: 97 pgs: 97 active+clean; 2.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.9 KiB/s wr, 11 op/s 2024-09-11T23:29:26.052 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:29:25 smithi076 ceph-mon[35296]: pgmap v344: 97 pgs: 97 active+clean; 2.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 41 KiB/s wr, 14 op/s 2024-09-11T23:29:26.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:29:25 smithi136 ceph-mon[35121]: pgmap v344: 97 pgs: 97 active+clean; 2.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 41 KiB/s wr, 14 op/s 2024-09-11T23:29:26.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:29:25 smithi026 ceph-mon[31172]: pgmap v344: 97 pgs: 97 active+clean; 2.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 41 KiB/s wr, 14 op/s 2024-09-11T23:29:28.052 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:29:27 smithi076 ceph-mon[35296]: pgmap v345: 97 pgs: 97 active+clean; 2.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 42 KiB/s wr, 14 op/s 2024-09-11T23:29:28.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:29:27 smithi136 ceph-mon[35121]: pgmap v345: 97 pgs: 97 active+clean; 2.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 42 KiB/s wr, 14 op/s 2024-09-11T23:29:28.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:29:27 smithi026 ceph-mon[31172]: pgmap v345: 97 pgs: 97 active+clean; 2.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 42 KiB/s wr, 14 op/s 2024-09-11T23:29:29.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:29.617+0000 7f25b97fa640 1 -- 172.21.15.26:0/2269247046 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x55db04e79f40 con 0x55db05017aa0 2024-09-11T23:29:29.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:29.617+0000 7f25b97fa640 10 client.17097 handle_command_reply: tid=0 2024-09-11T23:29:29.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:29.617+0000 7f25da1b8740 1 client.17097 shutdown 2024-09-11T23:29:29.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:29.617+0000 7f25da1b8740 1 -- 172.21.15.26:0/2269247046 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55db04f91850 msgr2=0x55db04f969a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:29.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:29.617+0000 7f25da1b8740 1 --2- 172.21.15.26:0/2269247046 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55db04f91850 0x55db04f969a0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f25cc005fd0 tx=0x7f25cc00bf20 comp rx=0 tx=0).stop 2024-09-11T23:29:29.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:29.618+0000 7f25da1b8740 1 -- 172.21.15.26:0/2269247046 shutdown_connections 2024-09-11T23:29:29.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:29.618+0000 7f25da1b8740 1 --2- 172.21.15.26:0/2269247046 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55db04f9ca30 0x55db04f97420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:29.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:29.618+0000 7f25da1b8740 1 --2- 172.21.15.26:0/2269247046 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55db05017aa0 0x55db05019e90 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:29.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:29.618+0000 7f25da1b8740 1 --2- 172.21.15.26:0/2269247046 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55db04f92170 0x55db04f96ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:29.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:29.618+0000 7f25da1b8740 1 --2- 172.21.15.26:0/2269247046 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55db04f91850 0x55db04f969a0 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:29.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:29.618+0000 7f25b97fa640 0 client.17097 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:29:29.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:29.618+0000 7f25da1b8740 1 -- 172.21.15.26:0/2269247046 >> 172.21.15.26:0/2269247046 conn(0x55db04f8dde0 msgr2=0x55db04f932f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:29:29.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:29.618+0000 7f25da1b8740 1 -- 172.21.15.26:0/2269247046 shutdown_connections 2024-09-11T23:29:29.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:29.618+0000 7f25da1b8740 1 -- 172.21.15.26:0/2269247046 wait complete. 2024-09-11T23:29:29.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:29.618+0000 7f25da1b8740 20 client.17097 trim_cache size 0 max 16384 2024-09-11T23:29:29.619 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-11T23:29:29.619 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 57, 2024-09-11T23:29:29.619 INFO:teuthology.orchestra.run.smithi026.stdout: "leader": 24485, 2024-09-11T23:29:29.619 INFO:teuthology.orchestra.run.smithi026.stdout: "set_version": 42, 2024-09-11T23:29:29.620 INFO:teuthology.orchestra.run.smithi026.stdout: "sets": { 2024-09-11T23:29:29.620 INFO:teuthology.orchestra.run.smithi026.stdout: "a85f23b": { 2024-09-11T23:29:29.620 INFO:teuthology.orchestra.run.smithi026.stdout: "version": 42, 2024-09-11T23:29:29.620 INFO:teuthology.orchestra.run.smithi026.stdout: "age_ref": 0.0, 2024-09-11T23:29:29.620 INFO:teuthology.orchestra.run.smithi026.stdout: "state": { 2024-09-11T23:29:29.620 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "QUIESCED", 2024-09-11T23:29:29.620 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.0 2024-09-11T23:29:29.620 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:29:29.620 INFO:teuthology.orchestra.run.smithi026.stdout: "timeout": 90.0, 2024-09-11T23:29:29.620 INFO:teuthology.orchestra.run.smithi026.stdout: "expiration": 142.6, 2024-09-11T23:29:29.620 INFO:teuthology.orchestra.run.smithi026.stdout: "members": { 2024-09-11T23:29:29.620 INFO:teuthology.orchestra.run.smithi026.stdout: "file:/": { 2024-09-11T23:29:29.621 INFO:teuthology.orchestra.run.smithi026.stdout: "excluded": false, 2024-09-11T23:29:29.621 INFO:teuthology.orchestra.run.smithi026.stdout: "state": { 2024-09-11T23:29:29.621 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "QUIESCED", 2024-09-11T23:29:29.621 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.0 2024-09-11T23:29:29.621 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:29:29.621 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:29:29.621 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:29:29.621 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:29:29.621 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:29:29.621 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-11T23:29:29.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:29.619+0000 7f25d7ddf640 1 -- 172.21.15.26:0/3528034187 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f25bc0a2cf0 msgr2=0x7f25bc0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:29.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:29.619+0000 7f25d7ddf640 1 --2- 172.21.15.26:0/3528034187 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f25bc0a2cf0 0x7f25bc0a51b0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f25c8005cc0 tx=0x7f25c8015000 comp rx=0 tx=0).stop 2024-09-11T23:29:29.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:29.619+0000 7f25d7ddf640 1 -- 172.21.15.26:0/3528034187 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25d0101380 msgr2=0x7f25d019ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:29.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:29.619+0000 7f25d7ddf640 1 --2- 172.21.15.26:0/3528034187 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25d0101380 0x7f25d019ac60 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f25c0006010 tx=0x7f25c00042f0 comp rx=0 tx=0).stop 2024-09-11T23:29:29.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:29.619+0000 7f25d7ddf640 1 -- 172.21.15.26:0/3528034187 shutdown_connections 2024-09-11T23:29:29.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:29.619+0000 7f25d7ddf640 1 --2- 172.21.15.26:0/3528034187 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f25bc0a2cf0 0x7f25bc0a51b0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:29.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:29.619+0000 7f25d7ddf640 1 --2- 172.21.15.26:0/3528034187 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f25d010e4f0 0x7f25d019eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:29.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:29.619+0000 7f25d7ddf640 1 --2- 172.21.15.26:0/3528034187 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25d0101380 0x7f25d019ac60 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:29.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:29.619+0000 7f25d7ddf640 1 --2- 172.21.15.26:0/3528034187 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f25d01009d0 0x7f25d019a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:29.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:29.619+0000 7f25d7ddf640 1 -- 172.21.15.26:0/3528034187 >> 172.21.15.26:0/3528034187 conn(0x7f25d00fc7d0 msgr2=0x7f25d00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:29:29.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:29.619+0000 7f25d7ddf640 1 -- 172.21.15.26:0/3528034187 shutdown_connections 2024-09-11T23:29:29.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:29.619+0000 7f25d7ddf640 1 -- 172.21.15.26:0/3528034187 wait complete. 2024-09-11T23:29:30.000 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:29:29 smithi026 ceph-mon[31172]: pgmap v346: 97 pgs: 97 active+clean; 2.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 42 KiB/s wr, 14 op/s 2024-09-11T23:29:30.052 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:29:29 smithi076 ceph-mon[35296]: pgmap v346: 97 pgs: 97 active+clean; 2.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 42 KiB/s wr, 14 op/s 2024-09-11T23:29:30.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:29:29 smithi136 ceph-mon[35121]: pgmap v346: 97 pgs: 97 active+clean; 2.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 42 KiB/s wr, 14 op/s 2024-09-11T23:29:30.372 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'a85f23b', quiesce took 11.6 seconds. Will release after: 10.950000000000001 2024-09-11T23:29:32.052 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:29:31 smithi076 ceph-mon[35296]: pgmap v347: 97 pgs: 97 active+clean; 2.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-09-11T23:29:32.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:29:31 smithi136 ceph-mon[35121]: pgmap v347: 97 pgs: 97 active+clean; 2.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-09-11T23:29:32.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:29:31 smithi026 ceph-mon[31172]: pgmap v347: 97 pgs: 97 active+clean; 2.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-09-11T23:29:32.851 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.849+0000 7fcca0b59640 1 -- 172.21.15.26:0/2947948269 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc9c101ad0 msgr2=0x7fcc9c101f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:32.851 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.849+0000 7fcca0b59640 1 --2- 172.21.15.26:0/2947948269 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc9c101ad0 0x7fcc9c101f30 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fcc90009930 tx=0x7fcc9002a420 comp rx=0 tx=0).stop 2024-09-11T23:29:32.852 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.849+0000 7fcca0b59640 1 -- 172.21.15.26:0/2947948269 shutdown_connections 2024-09-11T23:29:32.852 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.849+0000 7fcca0b59640 1 --2- 172.21.15.26:0/2947948269 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcc9c102470 0x7fcc9c10e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:32.852 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.849+0000 7fcca0b59640 1 --2- 172.21.15.26:0/2947948269 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc9c101ad0 0x7fcc9c101f30 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:32.852 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.849+0000 7fcca0b59640 1 --2- 172.21.15.26:0/2947948269 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcc9c107b70 0x7fcc9c107f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:32.852 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.849+0000 7fcca0b59640 1 -- 172.21.15.26:0/2947948269 >> 172.21.15.26:0/2947948269 conn(0x7fcc9c0fd7d0 msgr2=0x7fcc9c0ffbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:29:32.852 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.849+0000 7fcca0b59640 1 -- 172.21.15.26:0/2947948269 shutdown_connections 2024-09-11T23:29:32.852 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.849+0000 7fcca0b59640 1 -- 172.21.15.26:0/2947948269 wait complete. 2024-09-11T23:29:32.852 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.850+0000 7fcca0b59640 1 Processor -- start 2024-09-11T23:29:32.852 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.850+0000 7fcca0b59640 1 -- start start 2024-09-11T23:29:32.852 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.850+0000 7fcca0b59640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc9c101ad0 0x7fcc9c198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:32.852 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.850+0000 7fcca0b59640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcc9c102470 0x7fcc9c198890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:32.852 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.850+0000 7fcca0b59640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcc9c107b70 0x7fcc9c19cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:32.852 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.850+0000 7fcca0b59640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fcc9c111ee0 con 0x7fcc9c101ad0 2024-09-11T23:29:32.853 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.850+0000 7fcca0b59640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fcc9c111d60 con 0x7fcc9c107b70 2024-09-11T23:29:32.853 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.850+0000 7fcca0b59640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fcc9c112060 con 0x7fcc9c102470 2024-09-11T23:29:32.853 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.850+0000 7fcc9a59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc9c101ad0 0x7fcc9c198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:32.853 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.851+0000 7fcc9a59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc9c101ad0 0x7fcc9c198350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39200/0 (socket says 172.21.15.26:39200) 2024-09-11T23:29:32.853 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.851+0000 7fcc9a59c640 1 -- 172.21.15.26:0/2888836926 learned_addr learned my addr 172.21.15.26:0/2888836926 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:29:32.853 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.851+0000 7fcc9ad9d640 1 --2- 172.21.15.26:0/2888836926 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcc9c107b70 0x7fcc9c19cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:32.853 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.851+0000 7fcc99d9b640 1 --2- 172.21.15.26:0/2888836926 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcc9c102470 0x7fcc9c198890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:32.853 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.851+0000 7fcc9a59c640 1 -- 172.21.15.26:0/2888836926 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcc9c102470 msgr2=0x7fcc9c198890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:32.853 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.851+0000 7fcc9a59c640 1 --2- 172.21.15.26:0/2888836926 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcc9c102470 0x7fcc9c198890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:32.853 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.851+0000 7fcc9a59c640 1 -- 172.21.15.26:0/2888836926 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcc9c107b70 msgr2=0x7fcc9c19cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:32.853 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.851+0000 7fcc9a59c640 1 --2- 172.21.15.26:0/2888836926 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcc9c107b70 0x7fcc9c19cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:32.853 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.851+0000 7fcc9a59c640 1 -- 172.21.15.26:0/2888836926 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc9c19d340 con 0x7fcc9c101ad0 2024-09-11T23:29:32.853 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.851+0000 7fcc9a59c640 1 --2- 172.21.15.26:0/2888836926 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc9c101ad0 0x7fcc9c198350 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fcc8c00d950 tx=0x7fcc8c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:29:32.854 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.851+0000 7fcc8b7fe640 1 -- 172.21.15.26:0/2888836926 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc8c017070 con 0x7fcc9c101ad0 2024-09-11T23:29:32.854 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.851+0000 7fcca0b59640 1 -- 172.21.15.26:0/2888836926 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc9c19d630 con 0x7fcc9c101ad0 2024-09-11T23:29:32.854 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.851+0000 7fcc8b7fe640 1 -- 172.21.15.26:0/2888836926 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc8c009c30 con 0x7fcc9c101ad0 2024-09-11T23:29:32.854 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.851+0000 7fcc8b7fe640 1 -- 172.21.15.26:0/2888836926 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc8c00f6b0 con 0x7fcc9c101ad0 2024-09-11T23:29:32.854 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.851+0000 7fcca0b59640 1 -- 172.21.15.26:0/2888836926 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc9c1a4ec0 con 0x7fcc9c101ad0 2024-09-11T23:29:32.859 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.854+0000 7fcc8b7fe640 1 -- 172.21.15.26:0/2888836926 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fcc8c0109d0 con 0x7fcc9c101ad0 2024-09-11T23:29:32.859 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.854+0000 7fcca39d1740 1 Processor -- start 2024-09-11T23:29:32.859 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.854+0000 7fcca39d1740 1 -- start start 2024-09-11T23:29:32.860 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.855+0000 7fcca39d1740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5563b4211af0 0x5563b4211ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:32.860 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.855+0000 7fcca39d1740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5563b4212410 0x5563b421c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:32.860 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.855+0000 7fcca39d1740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5563b421ccb0 0x5563b421f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:32.860 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.855+0000 7fcca39d1740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5563b41762f0 con 0x5563b4211af0 2024-09-11T23:29:32.860 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.855+0000 7fcca39d1740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5563b411e270 con 0x5563b4212410 2024-09-11T23:29:32.860 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.855+0000 7fcca39d1740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5563b41722b0 con 0x5563b421ccb0 2024-09-11T23:29:32.860 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.855+0000 7fcc9ad9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5563b4211af0 0x5563b4211ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:32.860 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.855+0000 7fcc8b7fe640 1 --2- 172.21.15.26:0/2888836926 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fcc840a2da0 0x7fcc840a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:32.860 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.855+0000 7fcc9ad9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5563b4211af0 0x5563b4211ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39202/0 (socket says 172.21.15.26:39202) 2024-09-11T23:29:32.860 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.855+0000 7fcc9ad9d640 1 -- 172.21.15.26:0/737350644 learned_addr learned my addr 172.21.15.26:0/737350644 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:29:32.860 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.855+0000 7fcc9a59c640 1 --2- 172.21.15.26:0/737350644 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5563b4212410 0x5563b421c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:32.860 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.855+0000 7fcc99d9b640 1 --2- 172.21.15.26:0/737350644 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5563b421ccb0 0x5563b421f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:32.860 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.855+0000 7fcc9ad9d640 1 --2- 172.21.15.26:0/2888836926 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fcc840a2da0 0x7fcc840a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:32.861 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.855+0000 7fcc9ad9d640 1 -- 172.21.15.26:0/737350644 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5563b421ccb0 msgr2=0x5563b421f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:29:32.861 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.855+0000 7fcc9ad9d640 1 --2- 172.21.15.26:0/737350644 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5563b421ccb0 0x5563b421f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:32.861 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.855+0000 7fcc9ad9d640 1 -- 172.21.15.26:0/737350644 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5563b4212410 msgr2=0x5563b421c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:29:32.861 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.855+0000 7fcc9ad9d640 1 --2- 172.21.15.26:0/737350644 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5563b4212410 0x5563b421c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:32.861 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.855+0000 7fcc8b7fe640 1 -- 172.21.15.26:0/2888836926 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8542+0+0 (secure 0 0 0) 0x7fcc8c0cefb0 con 0x7fcc9c101ad0 2024-09-11T23:29:32.861 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.855+0000 7fcc9ad9d640 1 -- 172.21.15.26:0/737350644 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5563b411d340 con 0x5563b4211af0 2024-09-11T23:29:32.861 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.856+0000 7fcc9ad9d640 1 --2- 172.21.15.26:0/737350644 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5563b4211af0 0x5563b4211ed0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7fcc94005f40 tx=0x7fcc94014eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e25fb22267ea6a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:29:32.861 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.856+0000 7fcc88ff9640 1 -- 172.21.15.26:0/737350644 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc94004b20 con 0x5563b4211af0 2024-09-11T23:29:32.861 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.856+0000 7fcc88ff9640 1 -- 172.21.15.26:0/737350644 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc94023070 con 0x5563b4211af0 2024-09-11T23:29:32.861 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.856+0000 7fcc88ff9640 1 -- 172.21.15.26:0/737350644 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc940055e0 con 0x5563b4211af0 2024-09-11T23:29:32.861 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.856+0000 7fcca39d1740 1 -- 172.21.15.26:0/737350644 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5563b4211af0 msgr2=0x5563b4211ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:32.861 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.856+0000 7fcc9ad9d640 1 --2- 172.21.15.26:0/2888836926 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fcc840a2da0 0x7fcc840a5260 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fcc94022000 tx=0x7fcc94020000 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:29:32.861 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.856+0000 7fcca39d1740 1 --2- 172.21.15.26:0/737350644 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5563b4211af0 0x5563b4211ed0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7fcc94005f40 tx=0x7fcc94014eb0 comp rx=0 tx=0).stop 2024-09-11T23:29:32.861 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.857+0000 7fcca39d1740 1 -- 172.21.15.26:0/737350644 shutdown_connections 2024-09-11T23:29:32.862 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.857+0000 7fcca39d1740 1 --2- 172.21.15.26:0/737350644 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5563b4212410 0x5563b421c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:32.862 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.857+0000 7fcca39d1740 1 --2- 172.21.15.26:0/737350644 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5563b421ccb0 0x5563b421f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:32.862 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.857+0000 7fcca39d1740 1 --2- 172.21.15.26:0/737350644 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5563b4211af0 0x5563b4211ed0 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:32.862 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.857+0000 7fcca39d1740 1 -- 172.21.15.26:0/737350644 >> 172.21.15.26:0/737350644 conn(0x5563b420e060 msgr2=0x5563b420e440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:29:32.862 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.857+0000 7fcca39d1740 1 -- 172.21.15.26:0/737350644 shutdown_connections 2024-09-11T23:29:32.862 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.857+0000 7fcca39d1740 1 -- 172.21.15.26:0/737350644 wait complete. 2024-09-11T23:29:32.862 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.857+0000 7fcca39d1740 1 -- start start 2024-09-11T23:29:32.862 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.857+0000 7fcca39d1740 1 Processor -- start 2024-09-11T23:29:32.862 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.857+0000 7fcca39d1740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5563b4211af0 0x5563b4219b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:32.862 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.857+0000 7fcca39d1740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5563b4212410 0x5563b421a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:32.862 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.857+0000 7fcca39d1740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5563b421ccb0 0x5563b421a600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:32.862 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.857+0000 7fcca39d1740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5563b421f880 con 0x5563b4211af0 2024-09-11T23:29:32.862 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.857+0000 7fcca39d1740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5563b40ff290 con 0x5563b4212410 2024-09-11T23:29:32.863 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.857+0000 7fcca39d1740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5563b421fa00 con 0x5563b421ccb0 2024-09-11T23:29:32.863 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.858+0000 7fcca39d1740 10 client.0 _ll_register_callbacks cb 0x5563b4130140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:29:32.863 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.858+0000 7fcc9a59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5563b4211af0 0x5563b4219b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:32.863 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.858+0000 7fcc9a59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5563b4211af0 0x5563b4219b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39218/0 (socket says 172.21.15.26:39218) 2024-09-11T23:29:32.863 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.858+0000 7fcc9a59c640 1 -- 172.21.15.26:0/2522713302 learned_addr learned my addr 172.21.15.26:0/2522713302 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:29:32.863 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.858+0000 7fcc99d9b640 1 --2- 172.21.15.26:0/2522713302 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5563b4212410 0x5563b421a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:32.863 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.858+0000 7fcc9ad9d640 1 --2- 172.21.15.26:0/2522713302 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5563b421ccb0 0x5563b421a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:32.863 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.858+0000 7fcc9a59c640 1 -- 172.21.15.26:0/2522713302 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5563b421ccb0 msgr2=0x5563b421a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:29:32.863 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.858+0000 7fcc9a59c640 1 --2- 172.21.15.26:0/2522713302 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5563b421ccb0 0x5563b421a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:32.863 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.858+0000 7fcc9a59c640 1 -- 172.21.15.26:0/2522713302 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5563b4212410 msgr2=0x5563b421a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:29:32.863 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.858+0000 7fcc9a59c640 1 --2- 172.21.15.26:0/2522713302 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5563b4212410 0x5563b421a0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:32.863 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.858+0000 7fcc9a59c640 1 -- 172.21.15.26:0/2522713302 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5563b3e89210 con 0x5563b4211af0 2024-09-11T23:29:32.863 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.858+0000 7fcc9a59c640 1 --2- 172.21.15.26:0/2522713302 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5563b4211af0 0x5563b4219b80 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fcc8c097970 tx=0x7fcc8c010d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf686d704433058c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:29:32.864 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.858+0000 7fcc81ffb640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:29:32.864 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.858+0000 7fcc81ffb640 1 -- 172.21.15.26:0/2522713302 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc8c020070 con 0x5563b4211af0 2024-09-11T23:29:32.864 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.858+0000 7fcc81ffb640 1 -- 172.21.15.26:0/2522713302 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8542+0+0 (secure 0 0 0) 0x7fcc8c09a960 con 0x5563b4211af0 2024-09-11T23:29:32.864 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.858+0000 7fcca39d1740 1 -- 172.21.15.26:0/2522713302 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc94023070 con 0x5563b4211af0 2024-09-11T23:29:32.864 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.858+0000 7fcc81ffb640 1 -- 172.21.15.26:0/2522713302 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7fcc8c0ccb10 con 0x5563b4211af0 2024-09-11T23:29:32.864 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.858+0000 7fcca39d1740 10 client.17109 fetch_fsmap learned FSMap version 57 2024-09-11T23:29:32.864 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.858+0000 7fcca39d1740 1 -- 172.21.15.26:0/2522713302 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x5563b3f55620 con 0x5563b4211af0 2024-09-11T23:29:32.864 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.859+0000 7fcc81ffb640 1 -- 172.21.15.26:0/2522713302 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc94023070 con 0x5563b4211af0 2024-09-11T23:29:32.864 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.859+0000 7fcc81ffb640 1 -- 172.21.15.26:0/2522713302 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 57) ==== 8366+0+0 (secure 0 0 0) 0x7fcc8c0104f0 con 0x5563b4211af0 2024-09-11T23:29:32.864 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.859+0000 7fcca39d1740 10 client.17109 fetch_fsmap finished waiting for FSMap version 57 2024-09-11T23:29:32.864 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.859+0000 7fcca39d1740 10 client.17109 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:29:32.864 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.859+0000 7fcca39d1740 20 client.17109 populate_metadata read hostname 'smithi026' 2024-09-11T23:29:32.864 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.859+0000 7fcca39d1740 1 --2- 172.21.15.26:0/2522713302 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5563b422cba0 0x5563b422ef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:32.864 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.859+0000 7fcca39d1740 4 client.17109 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:29:32.865 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.859+0000 7fcca39d1740 1 -- 172.21.15.26:0/2522713302 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5563b4183a20 con 0x5563b422cba0 2024-09-11T23:29:32.865 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.859+0000 7fcc99d9b640 1 --2- 172.21.15.26:0/2522713302 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5563b422cba0 0x5563b422ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:32.865 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.860+0000 7fcc99d9b640 1 --2- 172.21.15.26:0/2522713302 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5563b422cba0 0x5563b422ef90 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=44e76648d43bf7bb server_cookie=6107be6682ad28e7 in_seq=0 out_seq=0 2024-09-11T23:29:32.865 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.860+0000 7fcc81ffb640 10 client.17109 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:29:32.866 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.860+0000 7fcc81ffb640 1 -- 172.21.15.26:0/2522713302 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5563b4183a20 con 0x5563b422cba0 2024-09-11T23:29:32.866 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.860+0000 7fcc81ffb640 10 client.17109 handle_command_reply: tid=0 2024-09-11T23:29:32.866 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.860+0000 7fcca39d1740 1 client.17109 shutdown 2024-09-11T23:29:32.866 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.861+0000 7fcca39d1740 1 -- 172.21.15.26:0/2522713302 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5563b4211af0 msgr2=0x5563b4219b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:32.866 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.861+0000 7fcca39d1740 1 --2- 172.21.15.26:0/2522713302 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5563b4211af0 0x5563b4219b80 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fcc8c097970 tx=0x7fcc8c010d70 comp rx=0 tx=0).stop 2024-09-11T23:29:32.866 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.861+0000 7fcca39d1740 1 -- 172.21.15.26:0/2522713302 shutdown_connections 2024-09-11T23:29:32.866 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.861+0000 7fcca39d1740 1 --2- 172.21.15.26:0/2522713302 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5563b4212410 0x5563b421a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:32.866 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.861+0000 7fcca39d1740 1 --2- 172.21.15.26:0/2522713302 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5563b422cba0 0x5563b422ef90 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:32.867 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.861+0000 7fcca39d1740 1 --2- 172.21.15.26:0/2522713302 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5563b421ccb0 0x5563b421a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:32.867 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.861+0000 7fcca39d1740 1 --2- 172.21.15.26:0/2522713302 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5563b4211af0 0x5563b4219b80 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:32.867 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.861+0000 7fcca39d1740 1 -- 172.21.15.26:0/2522713302 >> 172.21.15.26:0/2522713302 conn(0x5563b420e060 msgr2=0x5563b4213dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:29:32.867 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.861+0000 7fcc81ffb640 0 client.17109 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:29:32.867 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.861+0000 7fcca39d1740 1 -- 172.21.15.26:0/2522713302 shutdown_connections 2024-09-11T23:29:32.867 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.861+0000 7fcca39d1740 1 -- 172.21.15.26:0/2522713302 wait complete. 2024-09-11T23:29:32.867 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.861+0000 7fcca39d1740 20 client.17109 trim_cache size 0 max 16384 2024-09-11T23:29:32.879 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.875+0000 7fcca39d1740 1 Processor -- start 2024-09-11T23:29:32.880 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.875+0000 7fcca39d1740 1 -- start start 2024-09-11T23:29:32.880 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.876+0000 7fcca39d1740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5563b4211af0 0x5563b421a430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:32.880 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.876+0000 7fcca39d1740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5563b4212410 0x5563b4234850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:32.880 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.876+0000 7fcca39d1740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5563b421ccb0 0x5563b4234d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:32.880 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.876+0000 7fcca39d1740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fcc9c1ad860 con 0x5563b4211af0 2024-09-11T23:29:32.880 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.876+0000 7fcca39d1740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fcc9c1af3d0 con 0x5563b4212410 2024-09-11T23:29:32.880 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.876+0000 7fcca39d1740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fcc9c1adf30 con 0x5563b421ccb0 2024-09-11T23:29:32.880 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.876+0000 7fcc9a59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5563b4211af0 0x5563b421a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:32.880 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.876+0000 7fcc9a59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5563b4211af0 0x5563b421a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39220/0 (socket says 172.21.15.26:39220) 2024-09-11T23:29:32.880 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.876+0000 7fcc9a59c640 1 -- 172.21.15.26:0/1781926594 learned_addr learned my addr 172.21.15.26:0/1781926594 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:29:32.880 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.876+0000 7fcc99d9b640 1 --2- 172.21.15.26:0/1781926594 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5563b4212410 0x5563b4234850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:32.881 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.876+0000 7fcc9a59c640 1 -- 172.21.15.26:0/1781926594 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5563b421ccb0 msgr2=0x5563b4234d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:29:32.881 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.876+0000 7fcc9a59c640 1 --2- 172.21.15.26:0/1781926594 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5563b421ccb0 0x5563b4234d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:32.881 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.876+0000 7fcc9a59c640 1 -- 172.21.15.26:0/1781926594 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5563b4212410 msgr2=0x5563b4234850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:29:32.881 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.876+0000 7fcc9a59c640 1 --2- 172.21.15.26:0/1781926594 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5563b4212410 0x5563b4234850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:32.881 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.876+0000 7fcc9a59c640 1 -- 172.21.15.26:0/1781926594 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5563b4212240 con 0x5563b4211af0 2024-09-11T23:29:32.881 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.876+0000 7fcc9a59c640 1 --2- 172.21.15.26:0/1781926594 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5563b4211af0 0x5563b421a430 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fcc8c09b4a0 tx=0x7fcc8c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=639cdb640f9d4554 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:29:32.881 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.876+0000 7fcc83fff640 1 -- 172.21.15.26:0/1781926594 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc8c0a40d0 con 0x5563b4211af0 2024-09-11T23:29:32.881 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.876+0000 7fcc83fff640 1 -- 172.21.15.26:0/1781926594 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc8c015070 con 0x5563b4211af0 2024-09-11T23:29:32.881 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.876+0000 7fcc83fff640 1 -- 172.21.15.26:0/1781926594 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc8c0106c0 con 0x5563b4211af0 2024-09-11T23:29:32.881 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.877+0000 7fcca39d1740 1 -- 172.21.15.26:0/1781926594 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5563b4211af0 msgr2=0x5563b421a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:32.881 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.877+0000 7fcca39d1740 1 --2- 172.21.15.26:0/1781926594 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5563b4211af0 0x5563b421a430 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fcc8c09b4a0 tx=0x7fcc8c000bc0 comp rx=0 tx=0).stop 2024-09-11T23:29:32.881 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.877+0000 7fcca39d1740 1 -- 172.21.15.26:0/1781926594 shutdown_connections 2024-09-11T23:29:32.881 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.877+0000 7fcca39d1740 1 --2- 172.21.15.26:0/1781926594 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5563b4212410 0x5563b4234850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:32.881 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.877+0000 7fcca39d1740 1 --2- 172.21.15.26:0/1781926594 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5563b421ccb0 0x5563b4234d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:32.882 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.877+0000 7fcca39d1740 1 --2- 172.21.15.26:0/1781926594 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5563b4211af0 0x5563b421a430 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:32.882 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.877+0000 7fcca39d1740 1 -- 172.21.15.26:0/1781926594 >> 172.21.15.26:0/1781926594 conn(0x5563b420e060 msgr2=0x5563b421bc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:29:32.882 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.877+0000 7fcca39d1740 1 -- 172.21.15.26:0/1781926594 shutdown_connections 2024-09-11T23:29:32.882 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.877+0000 7fcca39d1740 1 -- 172.21.15.26:0/1781926594 wait complete. 2024-09-11T23:29:32.882 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.877+0000 7fcca39d1740 1 -- start start 2024-09-11T23:29:32.882 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.877+0000 7fcca39d1740 1 Processor -- start 2024-09-11T23:29:32.882 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.878+0000 7fcca39d1740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5563b4211af0 0x5563b42250e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:32.882 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.878+0000 7fcca39d1740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5563b4212410 0x5563b4216550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:32.882 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.878+0000 7fcca39d1740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5563b421ccb0 0x5563b4216a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:32.882 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.878+0000 7fcca39d1740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fcc9c1aed00 con 0x5563b4211af0 2024-09-11T23:29:32.882 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.878+0000 7fcca39d1740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fcc9c1ae600 con 0x5563b421ccb0 2024-09-11T23:29:32.882 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.878+0000 7fcca39d1740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fcc9c1ab860 con 0x5563b4212410 2024-09-11T23:29:32.882 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.878+0000 7fcca39d1740 10 client.0 _ll_register_callbacks cb 0x7fcc8c00df40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:29:32.883 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.878+0000 7fcc9a59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5563b4211af0 0x5563b42250e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:32.883 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.878+0000 7fcc9a59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5563b4211af0 0x5563b42250e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39234/0 (socket says 172.21.15.26:39234) 2024-09-11T23:29:32.883 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.878+0000 7fcc9a59c640 1 -- 172.21.15.26:0/3503007415 learned_addr learned my addr 172.21.15.26:0/3503007415 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:29:32.883 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.878+0000 7fcc9ad9d640 1 --2- 172.21.15.26:0/3503007415 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5563b421ccb0 0x5563b4216a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:32.883 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.878+0000 7fcc99d9b640 1 --2- 172.21.15.26:0/3503007415 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5563b4212410 0x5563b4216550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:32.883 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.878+0000 7fcc9a59c640 1 -- 172.21.15.26:0/3503007415 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5563b4212410 msgr2=0x5563b4216550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:29:32.883 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.878+0000 7fcc9a59c640 1 --2- 172.21.15.26:0/3503007415 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5563b4212410 0x5563b4216550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:32.883 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.878+0000 7fcc9a59c640 1 -- 172.21.15.26:0/3503007415 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5563b421ccb0 msgr2=0x5563b4216a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:29:32.883 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.878+0000 7fcc9a59c640 1 --2- 172.21.15.26:0/3503007415 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5563b421ccb0 0x5563b4216a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:32.883 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.878+0000 7fcc9a59c640 1 -- 172.21.15.26:0/3503007415 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5563b4216fd0 con 0x5563b4211af0 2024-09-11T23:29:32.883 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.878+0000 7fcc9a59c640 1 --2- 172.21.15.26:0/3503007415 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5563b4211af0 0x5563b42250e0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fcc8c0a3ba0 tx=0x7fcc8c005f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=959756aaaf1af36b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:29:32.883 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.878+0000 7fcc81ffb640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:29:32.883 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.878+0000 7fcc81ffb640 1 -- 172.21.15.26:0/3503007415 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc8c010660 con 0x5563b4211af0 2024-09-11T23:29:32.884 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.878+0000 7fcc81ffb640 1 -- 172.21.15.26:0/3503007415 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8542+0+0 (secure 0 0 0) 0x7fcc8c09a960 con 0x5563b4211af0 2024-09-11T23:29:32.884 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.878+0000 7fcca39d1740 1 -- 172.21.15.26:0/3503007415 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc8c015070 con 0x5563b4211af0 2024-09-11T23:29:32.884 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.879+0000 7fcc81ffb640 1 -- 172.21.15.26:0/3503007415 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7fcc8c0a40d0 con 0x5563b4211af0 2024-09-11T23:29:32.884 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.879+0000 7fcca39d1740 10 client.17115 fetch_fsmap learned FSMap version 57 2024-09-11T23:29:32.884 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.879+0000 7fcca39d1740 1 -- 172.21.15.26:0/3503007415 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x5563b4297500 con 0x5563b4211af0 2024-09-11T23:29:32.884 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.879+0000 7fcc81ffb640 1 -- 172.21.15.26:0/3503007415 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc8c015070 con 0x5563b4211af0 2024-09-11T23:29:32.884 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.879+0000 7fcc81ffb640 1 -- 172.21.15.26:0/3503007415 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 57) ==== 8366+0+0 (secure 0 0 0) 0x7fcc8c0a0aa0 con 0x5563b4211af0 2024-09-11T23:29:32.884 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.879+0000 7fcca39d1740 10 client.17115 fetch_fsmap finished waiting for FSMap version 57 2024-09-11T23:29:32.884 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.879+0000 7fcca39d1740 10 client.17115 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:29:32.884 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.879+0000 7fcca39d1740 20 client.17115 populate_metadata read hostname 'smithi026' 2024-09-11T23:29:32.884 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.879+0000 7fcca39d1740 1 --2- 172.21.15.26:0/3503007415 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5563b4297a20 0x5563b4299e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:32.884 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.879+0000 7fcca39d1740 4 client.17115 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T23:29:32.884 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.879+0000 7fcca39d1740 1 -- 172.21.15.26:0/3503007415 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5563b417dd70 con 0x5563b4297a20 2024-09-11T23:29:32.884 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.879+0000 7fcc99d9b640 1 --2- 172.21.15.26:0/3503007415 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5563b4297a20 0x5563b4299e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:32.885 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.880+0000 7fcc81ffb640 10 client.17115 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:29:32.885 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.880+0000 7fcc99d9b640 1 --2- 172.21.15.26:0/3503007415 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5563b4297a20 0x5563b4299e10 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc99de0c811e07e2 server_cookie=b9defde3d663dbc1 in_seq=0 out_seq=0 2024-09-11T23:29:32.897 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.896+0000 7fcc81ffb640 1 -- 172.21.15.26:0/3503007415 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5563b417dd70 con 0x5563b4297a20 2024-09-11T23:29:32.897 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.896+0000 7fcc81ffb640 10 client.17115 handle_command_reply: tid=0 2024-09-11T23:29:32.897 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.896+0000 7fcca39d1740 1 client.17115 shutdown 2024-09-11T23:29:32.897 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.897+0000 7fcca39d1740 1 -- 172.21.15.26:0/3503007415 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5563b4211af0 msgr2=0x5563b42250e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:32.897 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.897+0000 7fcca39d1740 1 --2- 172.21.15.26:0/3503007415 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5563b4211af0 0x5563b42250e0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fcc8c0a3ba0 tx=0x7fcc8c005f70 comp rx=0 tx=0).stop 2024-09-11T23:29:32.897 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.897+0000 7fcca39d1740 1 -- 172.21.15.26:0/3503007415 shutdown_connections 2024-09-11T23:29:32.897 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.897+0000 7fcca39d1740 1 --2- 172.21.15.26:0/3503007415 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5563b421ccb0 0x5563b4216a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:32.897 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.897+0000 7fcca39d1740 1 --2- 172.21.15.26:0/3503007415 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5563b4297a20 0x5563b4299e10 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:32.898 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.897+0000 7fcca39d1740 1 --2- 172.21.15.26:0/3503007415 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5563b4212410 0x5563b4216550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:32.898 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.897+0000 7fcca39d1740 1 --2- 172.21.15.26:0/3503007415 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5563b4211af0 0x5563b42250e0 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:32.898 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.897+0000 7fcc81ffb640 0 client.17115 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:29:32.898 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.897+0000 7fcca39d1740 1 -- 172.21.15.26:0/3503007415 >> 172.21.15.26:0/3503007415 conn(0x5563b420e060 msgr2=0x5563b420c9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:29:32.898 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.897+0000 7fcca39d1740 1 -- 172.21.15.26:0/3503007415 shutdown_connections 2024-09-11T23:29:32.898 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.897+0000 7fcca39d1740 1 -- 172.21.15.26:0/3503007415 wait complete. 2024-09-11T23:29:32.898 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.897+0000 7fcca39d1740 20 client.17115 trim_cache size 0 max 16384 2024-09-11T23:29:32.898 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-09-11T23:29:32.898 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "message": "", 2024-09-11T23:29:32.898 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-09-11T23:29:32.899 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-09-11T23:29:32.899 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.897+0000 7fcca0b59640 1 -- 172.21.15.26:0/2888836926 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fcc840a2da0 msgr2=0x7fcc840a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:32.899 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.897+0000 7fcca0b59640 1 --2- 172.21.15.26:0/2888836926 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fcc840a2da0 0x7fcc840a5260 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fcc94022000 tx=0x7fcc94020000 comp rx=0 tx=0).stop 2024-09-11T23:29:32.900 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.897+0000 7fcca0b59640 1 -- 172.21.15.26:0/2888836926 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc9c101ad0 msgr2=0x7fcc9c198350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:32.900 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.897+0000 7fcca0b59640 1 --2- 172.21.15.26:0/2888836926 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc9c101ad0 0x7fcc9c198350 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fcc8c00d950 tx=0x7fcc8c00de10 comp rx=0 tx=0).stop 2024-09-11T23:29:32.900 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.897+0000 7fcca0b59640 1 -- 172.21.15.26:0/2888836926 shutdown_connections 2024-09-11T23:29:32.900 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.897+0000 7fcca0b59640 1 --2- 172.21.15.26:0/2888836926 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fcc840a2da0 0x7fcc840a5260 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:32.900 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.897+0000 7fcca0b59640 1 --2- 172.21.15.26:0/2888836926 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcc9c107b70 0x7fcc9c19cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:32.900 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.897+0000 7fcca0b59640 1 --2- 172.21.15.26:0/2888836926 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcc9c102470 0x7fcc9c198890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:32.900 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.898+0000 7fcca0b59640 1 --2- 172.21.15.26:0/2888836926 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc9c101ad0 0x7fcc9c198350 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:32.900 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.898+0000 7fcca0b59640 1 -- 172.21.15.26:0/2888836926 >> 172.21.15.26:0/2888836926 conn(0x7fcc9c0fd7d0 msgr2=0x7fcc9c0fdf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:29:32.900 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.898+0000 7fcca0b59640 1 -- 172.21.15.26:0/2888836926 shutdown_connections 2024-09-11T23:29:32.900 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:32.898+0000 7fcca0b59640 1 -- 172.21.15.26:0/2888836926 wait complete. 2024-09-11T23:29:34.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:29:33 smithi076 ceph-mon[35296]: pgmap v348: 97 pgs: 97 active+clean; 2.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 35 KiB/s wr, 13 op/s 2024-09-11T23:29:34.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:29:33 smithi136 ceph-mon[35121]: pgmap v348: 97 pgs: 97 active+clean; 2.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 35 KiB/s wr, 13 op/s 2024-09-11T23:29:34.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:29:33 smithi026 ceph-mon[31172]: pgmap v348: 97 pgs: 97 active+clean; 2.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 35 KiB/s wr, 13 op/s 2024-09-11T23:29:36.052 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:29:35 smithi076 ceph-mon[35296]: pgmap v349: 97 pgs: 97 active+clean; 2.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 67 KiB/s wr, 17 op/s 2024-09-11T23:29:36.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:29:35 smithi136 ceph-mon[35121]: pgmap v349: 97 pgs: 97 active+clean; 2.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 67 KiB/s wr, 17 op/s 2024-09-11T23:29:36.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:29:35 smithi026 ceph-mon[31172]: pgmap v349: 97 pgs: 97 active+clean; 2.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 67 KiB/s wr, 17 op/s 2024-09-11T23:29:38.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:29:37 smithi076 ceph-mon[35296]: pgmap v350: 97 pgs: 97 active+clean; 2.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 37 KiB/s wr, 14 op/s 2024-09-11T23:29:38.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:29:37 smithi136 ceph-mon[35121]: pgmap v350: 97 pgs: 97 active+clean; 2.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 37 KiB/s wr, 14 op/s 2024-09-11T23:29:38.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:29:37 smithi026 ceph-mon[31172]: pgmap v350: 97 pgs: 97 active+clean; 2.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 37 KiB/s wr, 14 op/s 2024-09-11T23:29:40.052 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:29:39 smithi076 ceph-mon[35296]: pgmap v351: 97 pgs: 97 active+clean; 2.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 36 KiB/s wr, 12 op/s 2024-09-11T23:29:40.053 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:29:39 smithi026 ceph-mon[31172]: pgmap v351: 97 pgs: 97 active+clean; 2.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 36 KiB/s wr, 12 op/s 2024-09-11T23:29:40.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:29:39 smithi136 ceph-mon[35121]: pgmap v351: 97 pgs: 97 active+clean; 2.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 36 KiB/s wr, 12 op/s 2024-09-11T23:29:41.323 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:29:41.594 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:29:41.989 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:29:41 smithi026 ceph-mon[31172]: pgmap v352: 97 pgs: 97 active+clean; 2.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 36 KiB/s wr, 17 op/s 2024-09-11T23:29:42.052 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:29:41 smithi076 ceph-mon[35296]: pgmap v352: 97 pgs: 97 active+clean; 2.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 36 KiB/s wr, 17 op/s 2024-09-11T23:29:42.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:42.075+0000 7f989b976640 1 -- 172.21.15.26:0/2572375423 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f98941082a0 msgr2=0x7f989410ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:42.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:42.075+0000 7f989b976640 1 --2- 172.21.15.26:0/2572375423 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f98941082a0 0x7f989410ec40 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f988c009cd0 tx=0x7f988c02a8e0 comp rx=0 tx=0).stop 2024-09-11T23:29:42.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:42.075+0000 7f989b976640 1 -- 172.21.15.26:0/2572375423 shutdown_connections 2024-09-11T23:29:42.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:42.075+0000 7f989b976640 1 --2- 172.21.15.26:0/2572375423 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f98941082a0 0x7f989410ec40 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:42.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:42.075+0000 7f989b976640 1 --2- 172.21.15.26:0/2572375423 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f98941036e0 0x7f9894107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:42.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:42.075+0000 7f989b976640 1 --2- 172.21.15.26:0/2572375423 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9894102d30 0x7f9894103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:42.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:42.075+0000 7f989b976640 1 -- 172.21.15.26:0/2572375423 >> 172.21.15.26:0/2572375423 conn(0x7f98940fc7d0 msgr2=0x7f98940febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:29:42.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:42.075+0000 7f989b976640 1 -- 172.21.15.26:0/2572375423 shutdown_connections 2024-09-11T23:29:42.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:42.075+0000 7f989b976640 1 -- 172.21.15.26:0/2572375423 wait complete. 2024-09-11T23:29:42.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:42.076+0000 7f989b976640 1 Processor -- start 2024-09-11T23:29:42.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:42.076+0000 7f989b976640 1 -- start start 2024-09-11T23:29:42.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:42.076+0000 7f989b976640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9894102d30 0x7f98941a03f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:42.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:42.076+0000 7f989b976640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f98941036e0 0x7f98941a0930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:42.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:42.076+0000 7f989b976640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f98941082a0 0x7f989419a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:42.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:42.076+0000 7f989b976640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f98941120a0 con 0x7f9894102d30 2024-09-11T23:29:42.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:42.076+0000 7f989b976640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f9894111f20 con 0x7f98941082a0 2024-09-11T23:29:42.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:42.076+0000 7f989b976640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f9894112220 con 0x7f98941036e0 2024-09-11T23:29:42.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:42.076+0000 7f9899712640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9894102d30 0x7f98941a03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:42.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:42.076+0000 7f9899712640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9894102d30 0x7f98941a03f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42024/0 (socket says 172.21.15.26:42024) 2024-09-11T23:29:42.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:42.076+0000 7f9899712640 1 -- 172.21.15.26:0/2184249924 learned_addr learned my addr 172.21.15.26:0/2184249924 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:29:42.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:42.076+0000 7f9899f13640 1 --2- 172.21.15.26:0/2184249924 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f98941082a0 0x7f989419a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:42.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:42.076+0000 7f9898f11640 1 --2- 172.21.15.26:0/2184249924 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f98941036e0 0x7f98941a0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:42.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:42.077+0000 7f9899712640 1 -- 172.21.15.26:0/2184249924 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f98941036e0 msgr2=0x7f98941a0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:42.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:42.077+0000 7f9899712640 1 --2- 172.21.15.26:0/2184249924 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f98941036e0 0x7f98941a0930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:42.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:42.077+0000 7f9899712640 1 -- 172.21.15.26:0/2184249924 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f98941082a0 msgr2=0x7f989419a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:42.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:42.077+0000 7f9899712640 1 --2- 172.21.15.26:0/2184249924 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f98941082a0 0x7f989419a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:42.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:42.077+0000 7f9899712640 1 -- 172.21.15.26:0/2184249924 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f989419ace0 con 0x7f9894102d30 2024-09-11T23:29:42.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:42.077+0000 7f9899712640 1 --2- 172.21.15.26:0/2184249924 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9894102d30 0x7f98941a03f0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f989000e9f0 tx=0x7f989000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:29:42.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:42.077+0000 7f988a7fc640 1 -- 172.21.15.26:0/2184249924 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f989000cd80 con 0x7f9894102d30 2024-09-11T23:29:42.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:42.077+0000 7f989b976640 1 -- 172.21.15.26:0/2184249924 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f989419afd0 con 0x7f9894102d30 2024-09-11T23:29:42.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:42.077+0000 7f988a7fc640 1 -- 172.21.15.26:0/2184249924 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9890010910 con 0x7f9894102d30 2024-09-11T23:29:42.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:42.077+0000 7f988a7fc640 1 -- 172.21.15.26:0/2184249924 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98900188f0 con 0x7f9894102d30 2024-09-11T23:29:42.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:42.077+0000 7f989b976640 1 -- 172.21.15.26:0/2184249924 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98941a72c0 con 0x7f9894102d30 2024-09-11T23:29:42.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:42.078+0000 7f989b976640 1 -- 172.21.15.26:0/2184249924 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9894103110 con 0x7f9894102d30 2024-09-11T23:29:42.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:42.079+0000 7f988a7fc640 1 -- 172.21.15.26:0/2184249924 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f9890010ac0 con 0x7f9894102d30 2024-09-11T23:29:42.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:42.080+0000 7f988a7fc640 1 --2- 172.21.15.26:0/2184249924 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f98800a2cf0 0x7f98800a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:42.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:42.080+0000 7f988a7fc640 1 -- 172.21.15.26:0/2184249924 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8542+0+0 (secure 0 0 0) 0x7f98900cf810 con 0x7f9894102d30 2024-09-11T23:29:42.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:42.080+0000 7f9898f11640 1 --2- 172.21.15.26:0/2184249924 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f98800a2cf0 0x7f98800a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:42.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:42.081+0000 7f9898f11640 1 --2- 172.21.15.26:0/2184249924 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f98800a2cf0 0x7f98800a51b0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f989410f010 tx=0x7f988400b410 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:29:42.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:42.084+0000 7f988a7fc640 1 -- 172.21.15.26:0/2184249924 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f989009bdd0 con 0x7f9894102d30 2024-09-11T23:29:42.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:29:41 smithi136 ceph-mon[35121]: pgmap v352: 97 pgs: 97 active+clean; 2.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 36 KiB/s wr, 17 op/s 2024-09-11T23:29:42.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:42.298+0000 7f989b976640 1 -- 172.21.15.26:0/2184249924 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f98941a74f0 con 0x7f9894102d30 2024-09-11T23:29:42.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:42.298+0000 7f988a7fc640 1 -- 172.21.15.26:0/2184249924 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) ==== 94+0+11973 (secure 0 0 0) 0x7f98900a00d0 con 0x7f9894102d30 2024-09-11T23:29:42.299 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-11T23:29:42.299 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":57,"btime":"2024-09-11T23:29:06:826223+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14499,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/367659515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":367659515},{"type":"v1","addr":"172.21.15.26:6835","nonce":367659515}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14511,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/2041065299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":2041065299},{"type":"v1","addr":"172.21.15.26:6837","nonce":2041065299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":57,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:16:26.544203+0000","modified":"2024-09-11T23:29:06.658248+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24485,"mds_1":24463,"mds_2":24458,"mds_3":24448,"mds_4":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14550":{"gid":14550,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6839/2233068305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":2233068305},{"type":"v1","addr":"172.21.15.26:6839","nonce":2233068305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14562":{"gid":14562,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6841/2984982336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":2984982336},{"type":"v1","addr":"172.21.15.26:6841","nonce":2984982336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6833/2019841455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":2019841455},{"type":"v1","addr":"172.21.15.136:6833","nonce":2019841455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.76:6833/3438147050","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3438147050},{"type":"v1","addr":"172.21.15.76:6833","nonce":3438147050}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6835/3617665975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3617665975},{"type":"v1","addr":"172.21.15.136:6835","nonce":3617665975}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":12,"addr":"172.21.15.76:6835/393308499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":393308499},{"type":"v1","addr":"172.21.15.76:6835","nonce":393308499}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":10,"addr":"172.21.15.136:6837/1633082387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":1633082387},{"type":"v1","addr":"172.21.15.136:6837","nonce":1633082387}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.76:6837/2078978748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":2078978748},{"type":"v1","addr":"172.21.15.76:6837","nonce":2078978748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":63,"addr":"172.21.15.136:6839/3211398630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":3211398630},{"type":"v1","addr":"172.21.15.136:6839","nonce":3211398630}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.76:6839/2077637090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6838","nonce":2077637090},{"type":"v1","addr":"172.21.15.76:6839","nonce":2077637090}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24485,"qdb_cluster":[24443,24448,24458,24463,24485]},"id":1}]} 2024-09-11T23:29:42.299 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 57 2024-09-11T23:29:42.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:42.301+0000 7f989b976640 1 -- 172.21.15.26:0/2184249924 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f98800a2cf0 msgr2=0x7f98800a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:42.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:42.301+0000 7f989b976640 1 --2- 172.21.15.26:0/2184249924 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f98800a2cf0 0x7f98800a51b0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f989410f010 tx=0x7f988400b410 comp rx=0 tx=0).stop 2024-09-11T23:29:42.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:42.301+0000 7f989b976640 1 -- 172.21.15.26:0/2184249924 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9894102d30 msgr2=0x7f98941a03f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:42.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:42.301+0000 7f989b976640 1 --2- 172.21.15.26:0/2184249924 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9894102d30 0x7f98941a03f0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f989000e9f0 tx=0x7f989000eeb0 comp rx=0 tx=0).stop 2024-09-11T23:29:42.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:42.301+0000 7f989b976640 1 -- 172.21.15.26:0/2184249924 shutdown_connections 2024-09-11T23:29:42.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:42.301+0000 7f989b976640 1 --2- 172.21.15.26:0/2184249924 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f98800a2cf0 0x7f98800a51b0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:42.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:42.301+0000 7f989b976640 1 --2- 172.21.15.26:0/2184249924 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f98941082a0 0x7f989419a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:42.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:42.301+0000 7f989b976640 1 --2- 172.21.15.26:0/2184249924 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f98941036e0 0x7f98941a0930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:42.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:42.301+0000 7f989b976640 1 --2- 172.21.15.26:0/2184249924 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9894102d30 0x7f98941a03f0 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:42.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:42.301+0000 7f989b976640 1 -- 172.21.15.26:0/2184249924 >> 172.21.15.26:0/2184249924 conn(0x7f98940fc7d0 msgr2=0x7f98941008c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:29:42.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:42.301+0000 7f989b976640 1 -- 172.21.15.26:0/2184249924 shutdown_connections 2024-09-11T23:29:42.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:42.301+0000 7f989b976640 1 -- 172.21.15.26:0/2184249924 wait complete. 2024-09-11T23:29:43.000 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:29:42 smithi026 ceph-mon[31172]: from='client.? 172.21.15.26:0/2184249924' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:29:43.052 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:29:42 smithi076 ceph-mon[35296]: from='client.? 172.21.15.26:0/2184249924' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:29:43.219 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24485 quiesce db --set-id a85f23b --release --await' 2024-09-11T23:29:43.219 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24485 quiesce db --set-id a85f23b --release --await 2024-09-11T23:29:43.238 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:29:42 smithi136 ceph-mon[35121]: from='client.? 172.21.15.26:0/2184249924' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:29:43.481 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:29:43.882 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:29:43 smithi026 ceph-mon[31172]: pgmap v353: 97 pgs: 97 active+clean; 2.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 33 KiB/s wr, 11 op/s 2024-09-11T23:29:43.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:43.974+0000 7fa6fe0e3640 1 -- 172.21.15.26:0/1433334388 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa6f8102790 msgr2=0x7fa6f81067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:43.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:43.974+0000 7fa6fe0e3640 1 --2- 172.21.15.26:0/1433334388 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa6f8102790 0x7fa6f81067e0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fa6ec009900 tx=0x7fa6ec02a410 comp rx=0 tx=0).stop 2024-09-11T23:29:43.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:43.974+0000 7fa6fe0e3640 1 -- 172.21.15.26:0/1433334388 shutdown_connections 2024-09-11T23:29:43.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:43.974+0000 7fa6fe0e3640 1 --2- 172.21.15.26:0/1433334388 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa6f8106f10 0x7fa6f810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:43.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:43.974+0000 7fa6fe0e3640 1 --2- 172.21.15.26:0/1433334388 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa6f8102790 0x7fa6f81067e0 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:43.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:43.974+0000 7fa6fe0e3640 1 --2- 172.21.15.26:0/1433334388 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa6f8101de0 0x7fa6f81021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:43.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:43.974+0000 7fa6fe0e3640 1 -- 172.21.15.26:0/1433334388 >> 172.21.15.26:0/1433334388 conn(0x7fa6f80fd8c0 msgr2=0x7fa6f80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:29:43.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:43.974+0000 7fa6fe0e3640 1 -- 172.21.15.26:0/1433334388 shutdown_connections 2024-09-11T23:29:43.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:43.974+0000 7fa6fe0e3640 1 -- 172.21.15.26:0/1433334388 wait complete. 2024-09-11T23:29:43.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:43.975+0000 7fa6fe0e3640 1 Processor -- start 2024-09-11T23:29:43.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:43.975+0000 7fa6fe0e3640 1 -- start start 2024-09-11T23:29:43.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:43.976+0000 7fa6fe0e3640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa6f8101de0 0x7fa6f8198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:43.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:43.976+0000 7fa6fe0e3640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa6f8102790 0x7fa6f8198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:43.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:43.976+0000 7fa6fe0e3640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa6f8106f10 0x7fa6f819ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:43.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:43.976+0000 7fa6fe0e3640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fa6f8111fc0 con 0x7fa6f8102790 2024-09-11T23:29:43.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:43.976+0000 7fa6fe0e3640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fa6f8111e40 con 0x7fa6f8106f10 2024-09-11T23:29:43.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:43.976+0000 7fa6fe0e3640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fa6f8112140 con 0x7fa6f8101de0 2024-09-11T23:29:43.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:43.976+0000 7fa6f6ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa6f8102790 0x7fa6f8198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:43.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:43.976+0000 7fa6f6ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa6f8102790 0x7fa6f8198960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42048/0 (socket says 172.21.15.26:42048) 2024-09-11T23:29:43.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:43.976+0000 7fa6f6ffd640 1 -- 172.21.15.26:0/1125987602 learned_addr learned my addr 172.21.15.26:0/1125987602 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:29:43.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:43.976+0000 7fa6f77fe640 1 --2- 172.21.15.26:0/1125987602 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa6f8101de0 0x7fa6f8198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:43.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:43.976+0000 7fa6f6ffd640 1 -- 172.21.15.26:0/1125987602 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa6f8101de0 msgr2=0x7fa6f8198420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:43.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:43.976+0000 7fa6f6ffd640 1 --2- 172.21.15.26:0/1125987602 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa6f8101de0 0x7fa6f8198420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:43.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:43.976+0000 7fa6f7fff640 1 --2- 172.21.15.26:0/1125987602 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa6f8106f10 0x7fa6f819ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:43.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:43.976+0000 7fa6f6ffd640 1 -- 172.21.15.26:0/1125987602 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa6f8106f10 msgr2=0x7fa6f819ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:43.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:43.976+0000 7fa6f6ffd640 1 --2- 172.21.15.26:0/1125987602 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa6f8106f10 0x7fa6f819ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:43.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:43.976+0000 7fa6f6ffd640 1 -- 172.21.15.26:0/1125987602 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6f819d3d0 con 0x7fa6f8102790 2024-09-11T23:29:43.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:43.976+0000 7fa6f7fff640 1 --2- 172.21.15.26:0/1125987602 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa6f8106f10 0x7fa6f819ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:29:43.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:43.976+0000 7fa6f6ffd640 1 --2- 172.21.15.26:0/1125987602 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa6f8102790 0x7fa6f8198960 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fa6ec004770 tx=0x7fa6ec003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:29:43.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:43.976+0000 7fa6f4ff9640 1 -- 172.21.15.26:0/1125987602 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6ec03f070 con 0x7fa6f8102790 2024-09-11T23:29:43.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:43.976+0000 7fa6f4ff9640 1 -- 172.21.15.26:0/1125987602 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6ec031ca0 con 0x7fa6f8102790 2024-09-11T23:29:43.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:43.976+0000 7fa6f4ff9640 1 -- 172.21.15.26:0/1125987602 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6ec035030 con 0x7fa6f8102790 2024-09-11T23:29:43.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:43.976+0000 7fa6fe0e3640 1 -- 172.21.15.26:0/1125987602 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6f819d660 con 0x7fa6f8102790 2024-09-11T23:29:43.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:43.977+0000 7fa6fe0e3640 1 -- 172.21.15.26:0/1125987602 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6f819daf0 con 0x7fa6f8102790 2024-09-11T23:29:43.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:43.979+0000 7fa6f4ff9640 1 -- 172.21.15.26:0/1125987602 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fa6ec042070 con 0x7fa6f8102790 2024-09-11T23:29:43.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:43.979+0000 7fa7004bc740 1 Processor -- start 2024-09-11T23:29:43.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:43.979+0000 7fa7004bc740 1 -- start start 2024-09-11T23:29:43.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:43.980+0000 7fa7004bc740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f6b8ec7060 0x55f6b8ec7440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:43.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:43.980+0000 7fa7004bc740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f6b8ec7980 0x55f6b8ed1d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:43.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:43.980+0000 7fa7004bc740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f6b8ed2240 0x55f6b8ed4630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:43.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:43.980+0000 7fa7004bc740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55f6b8e2c620 con 0x55f6b8ed2240 2024-09-11T23:29:43.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:43.980+0000 7fa7004bc740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55f6b8dd3e50 con 0x55f6b8ec7980 2024-09-11T23:29:43.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:43.980+0000 7fa7004bc740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55f6b8e285e0 con 0x55f6b8ec7060 2024-09-11T23:29:43.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:43.980+0000 7fa6f6ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f6b8ed2240 0x55f6b8ed4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:43.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:43.980+0000 7fa6f6ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f6b8ed2240 0x55f6b8ed4630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42056/0 (socket says 172.21.15.26:42056) 2024-09-11T23:29:43.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:43.980+0000 7fa6f6ffd640 1 -- 172.21.15.26:0/282258235 learned_addr learned my addr 172.21.15.26:0/282258235 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:29:43.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:43.980+0000 7fa6f7fff640 1 --2- 172.21.15.26:0/282258235 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f6b8ec7060 0x55f6b8ec7440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:43.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:43.980+0000 7fa6f77fe640 1 --2- 172.21.15.26:0/282258235 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f6b8ec7980 0x55f6b8ed1d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:43.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:43.980+0000 7fa6f4ff9640 1 --2- 172.21.15.26:0/1125987602 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fa6e00a2d70 0x7fa6e00a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:43.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:43.980+0000 7fa6f7fff640 1 --2- 172.21.15.26:0/1125987602 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fa6e00a2d70 0x7fa6e00a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:43.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:43.980+0000 7fa6f4ff9640 1 -- 172.21.15.26:0/1125987602 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8542+0+0 (secure 0 0 0) 0x7fa6ec0efb70 con 0x7fa6f8102790 2024-09-11T23:29:43.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:43.980+0000 7fa6f6ffd640 1 -- 172.21.15.26:0/282258235 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f6b8ec7060 msgr2=0x55f6b8ec7440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:29:43.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:43.980+0000 7fa6f6ffd640 1 --2- 172.21.15.26:0/282258235 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f6b8ec7060 0x55f6b8ec7440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:43.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:43.980+0000 7fa6f6ffd640 1 -- 172.21.15.26:0/282258235 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f6b8ec7980 msgr2=0x55f6b8ed1d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:29:43.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:43.980+0000 7fa6f6ffd640 1 --2- 172.21.15.26:0/282258235 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f6b8ec7980 0x55f6b8ed1d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:43.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:43.980+0000 7fa6f6ffd640 1 -- 172.21.15.26:0/282258235 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f6b8dc4d40 con 0x55f6b8ed2240 2024-09-11T23:29:43.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:43.980+0000 7fa6f77fe640 1 --2- 172.21.15.26:0/282258235 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f6b8ec7980 0x55f6b8ed1d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:29:43.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:43.981+0000 7fa6f6ffd640 1 --2- 172.21.15.26:0/282258235 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f6b8ed2240 0x55f6b8ed4630 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fa6ec0ed1c0 tx=0x7fa6ec009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17928c57ce2df345 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:29:43.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:43.981+0000 7fa6f7fff640 1 --2- 172.21.15.26:0/1125987602 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fa6e00a2d70 0x7fa6e00a5230 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fa6f0005fd0 tx=0x7fa6f0004ab0 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:29:43.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:43.981+0000 7fa6ddffb640 1 -- 172.21.15.26:0/282258235 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6ec049070 con 0x55f6b8ed2240 2024-09-11T23:29:43.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:43.981+0000 7fa6ddffb640 1 -- 172.21.15.26:0/282258235 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6ec0bc720 con 0x55f6b8ed2240 2024-09-11T23:29:43.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:43.981+0000 7fa6ddffb640 1 -- 172.21.15.26:0/282258235 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6ec0b8690 con 0x55f6b8ed2240 2024-09-11T23:29:43.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:43.981+0000 7fa7004bc740 1 -- 172.21.15.26:0/282258235 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f6b8ed2240 msgr2=0x55f6b8ed4630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:43.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:43.981+0000 7fa7004bc740 1 --2- 172.21.15.26:0/282258235 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f6b8ed2240 0x55f6b8ed4630 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fa6ec0ed1c0 tx=0x7fa6ec009680 comp rx=0 tx=0).stop 2024-09-11T23:29:43.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:43.981+0000 7fa7004bc740 1 -- 172.21.15.26:0/282258235 shutdown_connections 2024-09-11T23:29:43.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:43.981+0000 7fa7004bc740 1 --2- 172.21.15.26:0/282258235 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f6b8ec7980 0x55f6b8ed1d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:43.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:43.981+0000 7fa7004bc740 1 --2- 172.21.15.26:0/282258235 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f6b8ed2240 0x55f6b8ed4630 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:43.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:43.982+0000 7fa7004bc740 1 --2- 172.21.15.26:0/282258235 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f6b8ec7060 0x55f6b8ec7440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:43.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:43.982+0000 7fa7004bc740 1 -- 172.21.15.26:0/282258235 >> 172.21.15.26:0/282258235 conn(0x55f6b8ec35f0 msgr2=0x55f6b8ec39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:29:43.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:43.982+0000 7fa7004bc740 1 -- 172.21.15.26:0/282258235 shutdown_connections 2024-09-11T23:29:43.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:43.982+0000 7fa7004bc740 1 -- 172.21.15.26:0/282258235 wait complete. 2024-09-11T23:29:43.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:43.982+0000 7fa7004bc740 1 -- start start 2024-09-11T23:29:43.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:43.982+0000 7fa7004bc740 1 Processor -- start 2024-09-11T23:29:43.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:43.982+0000 7fa7004bc740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f6b8ec7060 0x55f6b8ecf140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:43.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:43.982+0000 7fa7004bc740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f6b8ec7980 0x55f6b8ecf680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:43.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:43.982+0000 7fa7004bc740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f6b8ed2240 0x55f6b8ecfbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:43.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:43.982+0000 7fa7004bc740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55f6b8e2c620 con 0x55f6b8ec7060 2024-09-11T23:29:43.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:43.982+0000 7fa7004bc740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55f6b8db4f10 con 0x55f6b8ed2240 2024-09-11T23:29:43.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:43.982+0000 7fa7004bc740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55f6b8ed8fe0 con 0x55f6b8ec7980 2024-09-11T23:29:43.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:43.982+0000 7fa7004bc740 10 client.0 _ll_register_callbacks cb 0x55f6b8dd8db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:29:43.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:43.982+0000 7fa6f77fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f6b8ec7060 0x55f6b8ecf140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:43.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:43.982+0000 7fa6f77fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f6b8ec7060 0x55f6b8ecf140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42068/0 (socket says 172.21.15.26:42068) 2024-09-11T23:29:43.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:43.983+0000 7fa6f77fe640 1 -- 172.21.15.26:0/141185190 learned_addr learned my addr 172.21.15.26:0/141185190 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:29:43.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:43.983+0000 7fa6f7fff640 1 --2- 172.21.15.26:0/141185190 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f6b8ed2240 0x55f6b8ecfbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:43.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:43.983+0000 7fa6f6ffd640 1 --2- 172.21.15.26:0/141185190 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f6b8ec7980 0x55f6b8ecf680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:43.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:43.983+0000 7fa6f77fe640 1 -- 172.21.15.26:0/141185190 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f6b8ec7980 msgr2=0x55f6b8ecf680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:29:44.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:43.983+0000 7fa6f77fe640 1 --2- 172.21.15.26:0/141185190 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f6b8ec7980 0x55f6b8ecf680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:44.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:43.983+0000 7fa6f77fe640 1 -- 172.21.15.26:0/141185190 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f6b8ed2240 msgr2=0x55f6b8ecfbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:29:44.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:43.983+0000 7fa6f77fe640 1 --2- 172.21.15.26:0/141185190 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f6b8ed2240 0x55f6b8ecfbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:44.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:43.983+0000 7fa6f77fe640 1 -- 172.21.15.26:0/141185190 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f6b8b3b580 con 0x55f6b8ec7060 2024-09-11T23:29:44.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:43.983+0000 7fa6f77fe640 1 --2- 172.21.15.26:0/141185190 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f6b8ec7060 0x55f6b8ecf140 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fa6e800ba10 tx=0x7fa6e800bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=893c9c75af97b2c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:29:44.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:43.983+0000 7fa6d6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:29:44.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:43.983+0000 7fa6d6ffd640 1 -- 172.21.15.26:0/141185190 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6e8003e80 con 0x55f6b8ec7060 2024-09-11T23:29:44.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:43.983+0000 7fa6d6ffd640 1 -- 172.21.15.26:0/141185190 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8542+0+0 (secure 0 0 0) 0x7fa6e8011440 con 0x55f6b8ec7060 2024-09-11T23:29:44.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:43.983+0000 7fa7004bc740 1 -- 172.21.15.26:0/141185190 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa6ec0bc720 con 0x55f6b8ec7060 2024-09-11T23:29:44.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:43.983+0000 7fa6d6ffd640 1 -- 172.21.15.26:0/141185190 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7fa6e8011640 con 0x55f6b8ec7060 2024-09-11T23:29:44.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:43.983+0000 7fa7004bc740 10 client.17133 fetch_fsmap learned FSMap version 57 2024-09-11T23:29:44.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:43.983+0000 7fa7004bc740 1 -- 172.21.15.26:0/141185190 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x55f6b8c05960 con 0x55f6b8ec7060 2024-09-11T23:29:44.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:43.984+0000 7fa6d6ffd640 1 -- 172.21.15.26:0/141185190 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6ec0bc720 con 0x55f6b8ec7060 2024-09-11T23:29:44.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:43.984+0000 7fa6d6ffd640 1 -- 172.21.15.26:0/141185190 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 57) ==== 8366+0+0 (secure 0 0 0) 0x7fa6e801bad0 con 0x55f6b8ec7060 2024-09-11T23:29:44.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:43.984+0000 7fa7004bc740 10 client.17133 fetch_fsmap finished waiting for FSMap version 57 2024-09-11T23:29:44.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:43.984+0000 7fa7004bc740 10 client.17133 resolve_mds: validated gid 24485 aka daemon mds.b 2024-09-11T23:29:44.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:43.984+0000 7fa7004bc740 20 client.17133 populate_metadata read hostname 'smithi026' 2024-09-11T23:29:44.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:43.984+0000 7fa7004bc740 1 --2- 172.21.15.26:0/141185190 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55f6b8ee5e30 0x55f6b8ee82d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:44.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:43.984+0000 7fa7004bc740 4 client.17133 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:29:44.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:43.984+0000 7fa7004bc740 1 -- 172.21.15.26:0/141185190 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f6b8e39b90 con 0x55f6b8ee5e30 2024-09-11T23:29:44.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:43.984+0000 7fa6f6ffd640 1 --2- 172.21.15.26:0/141185190 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55f6b8ee5e30 0x55f6b8ee82d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:44.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:43.984+0000 7fa6d6ffd640 10 client.17133 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:29:44.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:43.984+0000 7fa6f6ffd640 1 --2- 172.21.15.26:0/141185190 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55f6b8ee5e30 0x55f6b8ee82d0 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=697a9a003a430aa server_cookie=6de70fe181ecb4a9 in_seq=0 out_seq=0 2024-09-11T23:29:44.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:43.985+0000 7fa6d6ffd640 1 -- 172.21.15.26:0/141185190 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f6b8e39b90 con 0x55f6b8ee5e30 2024-09-11T23:29:44.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:43.985+0000 7fa6d6ffd640 10 client.17133 handle_command_reply: tid=0 2024-09-11T23:29:44.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:43.985+0000 7fa7004bc740 1 client.17133 shutdown 2024-09-11T23:29:44.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:43.985+0000 7fa7004bc740 1 -- 172.21.15.26:0/141185190 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f6b8ec7060 msgr2=0x55f6b8ecf140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:44.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:43.985+0000 7fa7004bc740 1 --2- 172.21.15.26:0/141185190 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f6b8ec7060 0x55f6b8ecf140 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fa6e800ba10 tx=0x7fa6e800bed0 comp rx=0 tx=0).stop 2024-09-11T23:29:44.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:43.985+0000 7fa7004bc740 1 -- 172.21.15.26:0/141185190 shutdown_connections 2024-09-11T23:29:44.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:43.985+0000 7fa7004bc740 1 --2- 172.21.15.26:0/141185190 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f6b8ed2240 0x55f6b8ecfbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:44.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:43.985+0000 7fa7004bc740 1 --2- 172.21.15.26:0/141185190 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55f6b8ee5e30 0x55f6b8ee82d0 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:44.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:43.985+0000 7fa7004bc740 1 --2- 172.21.15.26:0/141185190 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f6b8ec7980 0x55f6b8ecf680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:44.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:43.985+0000 7fa7004bc740 1 --2- 172.21.15.26:0/141185190 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f6b8ec7060 0x55f6b8ecf140 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:44.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:43.985+0000 7fa7004bc740 1 -- 172.21.15.26:0/141185190 >> 172.21.15.26:0/141185190 conn(0x55f6b8ec35f0 msgr2=0x55f6b8ec92d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:29:44.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:43.985+0000 7fa6d6ffd640 0 client.17133 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:29:44.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:43.986+0000 7fa7004bc740 1 -- 172.21.15.26:0/141185190 shutdown_connections 2024-09-11T23:29:44.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:43.986+0000 7fa7004bc740 1 -- 172.21.15.26:0/141185190 wait complete. 2024-09-11T23:29:44.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:43.986+0000 7fa7004bc740 20 client.17133 trim_cache size 0 max 16384 2024-09-11T23:29:44.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:44.003+0000 7fa7004bc740 1 Processor -- start 2024-09-11T23:29:44.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:44.003+0000 7fa7004bc740 1 -- start start 2024-09-11T23:29:44.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:44.003+0000 7fa7004bc740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f6b8ec7060 0x55f6b8ece010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:44.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:44.003+0000 7fa7004bc740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f6b8ec7980 0x55f6b8eedba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:44.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:44.003+0000 7fa7004bc740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f6b8ed2240 0x55f6b8eee0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:44.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:44.003+0000 7fa7004bc740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fa6f81ab6e0 con 0x55f6b8ec7980 2024-09-11T23:29:44.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:44.003+0000 7fa7004bc740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fa6f81ad250 con 0x55f6b8ed2240 2024-09-11T23:29:44.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:44.003+0000 7fa7004bc740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fa6f81abdb0 con 0x55f6b8ec7060 2024-09-11T23:29:44.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:44.003+0000 7fa6f6ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f6b8ec7980 0x55f6b8eedba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:44.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:44.003+0000 7fa6f6ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f6b8ec7980 0x55f6b8eedba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42072/0 (socket says 172.21.15.26:42072) 2024-09-11T23:29:44.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:44.003+0000 7fa6f6ffd640 1 -- 172.21.15.26:0/3680439380 learned_addr learned my addr 172.21.15.26:0/3680439380 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:29:44.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:44.003+0000 7fa6f77fe640 1 --2- 172.21.15.26:0/3680439380 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f6b8ec7060 0x55f6b8ece010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:44.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:44.004+0000 7fa6f7fff640 1 --2- 172.21.15.26:0/3680439380 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f6b8ed2240 0x55f6b8eee0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:44.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:44.004+0000 7fa6f6ffd640 1 -- 172.21.15.26:0/3680439380 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f6b8ec7060 msgr2=0x55f6b8ece010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:29:44.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:44.004+0000 7fa6f6ffd640 1 --2- 172.21.15.26:0/3680439380 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f6b8ec7060 0x55f6b8ece010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:44.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:44.004+0000 7fa6f6ffd640 1 -- 172.21.15.26:0/3680439380 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f6b8ed2240 msgr2=0x55f6b8eee0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:29:44.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:44.004+0000 7fa6f6ffd640 1 --2- 172.21.15.26:0/3680439380 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f6b8ed2240 0x55f6b8eee0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:44.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:44.004+0000 7fa6f6ffd640 1 -- 172.21.15.26:0/3680439380 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f6b8ed4990 con 0x55f6b8ec7980 2024-09-11T23:29:44.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:44.004+0000 7fa6f7fff640 1 --2- 172.21.15.26:0/3680439380 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f6b8ed2240 0x55f6b8eee0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:29:44.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:44.004+0000 7fa6f6ffd640 1 --2- 172.21.15.26:0/3680439380 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f6b8ec7980 0x55f6b8eedba0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fa6ec0b89f0 tx=0x7fa6ec005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9623b9daae712fdb server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:29:44.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:44.004+0000 7fa6dd7fa640 1 -- 172.21.15.26:0/3680439380 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6ec049040 con 0x55f6b8ec7980 2024-09-11T23:29:44.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:44.004+0000 7fa6dd7fa640 1 -- 172.21.15.26:0/3680439380 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6ec0c4c90 con 0x55f6b8ec7980 2024-09-11T23:29:44.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:44.004+0000 7fa6dd7fa640 1 -- 172.21.15.26:0/3680439380 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6ec03b4f0 con 0x55f6b8ec7980 2024-09-11T23:29:44.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:44.005+0000 7fa7004bc740 1 -- 172.21.15.26:0/3680439380 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f6b8ec7980 msgr2=0x55f6b8eedba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:44.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:44.005+0000 7fa7004bc740 1 --2- 172.21.15.26:0/3680439380 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f6b8ec7980 0x55f6b8eedba0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fa6ec0b89f0 tx=0x7fa6ec005c00 comp rx=0 tx=0).stop 2024-09-11T23:29:44.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:44.005+0000 7fa7004bc740 1 -- 172.21.15.26:0/3680439380 shutdown_connections 2024-09-11T23:29:44.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:44.005+0000 7fa7004bc740 1 --2- 172.21.15.26:0/3680439380 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f6b8ed2240 0x55f6b8eee0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:44.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:44.005+0000 7fa7004bc740 1 --2- 172.21.15.26:0/3680439380 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f6b8ec7980 0x55f6b8eedba0 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:44.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:44.005+0000 7fa7004bc740 1 --2- 172.21.15.26:0/3680439380 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f6b8ec7060 0x55f6b8ece010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:44.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:44.005+0000 7fa7004bc740 1 -- 172.21.15.26:0/3680439380 >> 172.21.15.26:0/3680439380 conn(0x55f6b8ec35f0 msgr2=0x55f6b8ecd900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:29:44.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:44.005+0000 7fa7004bc740 1 -- 172.21.15.26:0/3680439380 shutdown_connections 2024-09-11T23:29:44.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:44.005+0000 7fa7004bc740 1 -- 172.21.15.26:0/3680439380 wait complete. 2024-09-11T23:29:44.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:44.005+0000 7fa7004bc740 1 -- start start 2024-09-11T23:29:44.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:44.005+0000 7fa7004bc740 1 Processor -- start 2024-09-11T23:29:44.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:44.006+0000 7fa7004bc740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f6b8ec7060 0x55f6b8edeac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:44.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:44.006+0000 7fa7004bc740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f6b8ec7980 0x55f6b8ecc040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:44.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:44.006+0000 7fa7004bc740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f6b8ed2240 0x55f6b8ecc780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:44.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:44.006+0000 7fa7004bc740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fa6f81acb80 con 0x55f6b8ec7060 2024-09-11T23:29:44.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:44.006+0000 7fa7004bc740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fa6f81ac480 con 0x55f6b8ed2240 2024-09-11T23:29:44.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:44.006+0000 7fa7004bc740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fa6f81a96e0 con 0x55f6b8ec7980 2024-09-11T23:29:44.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:44.006+0000 7fa7004bc740 10 client.0 _ll_register_callbacks cb 0x7fa6e800b670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:29:44.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:44.006+0000 7fa6f77fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f6b8ec7060 0x55f6b8edeac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:44.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:44.006+0000 7fa6f77fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f6b8ec7060 0x55f6b8edeac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42084/0 (socket says 172.21.15.26:42084) 2024-09-11T23:29:44.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:44.006+0000 7fa6f77fe640 1 -- 172.21.15.26:0/451204563 learned_addr learned my addr 172.21.15.26:0/451204563 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:29:44.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:44.006+0000 7fa6f7fff640 1 --2- 172.21.15.26:0/451204563 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f6b8ed2240 0x55f6b8ecc780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:44.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:44.006+0000 7fa6f6ffd640 1 --2- 172.21.15.26:0/451204563 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f6b8ec7980 0x55f6b8ecc040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:44.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:44.006+0000 7fa6f77fe640 1 -- 172.21.15.26:0/451204563 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f6b8ec7980 msgr2=0x55f6b8ecc040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:29:44.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:44.006+0000 7fa6f77fe640 1 --2- 172.21.15.26:0/451204563 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f6b8ec7980 0x55f6b8ecc040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:44.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:44.006+0000 7fa6f77fe640 1 -- 172.21.15.26:0/451204563 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f6b8ed2240 msgr2=0x55f6b8ecc780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:29:44.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:44.006+0000 7fa6f77fe640 1 --2- 172.21.15.26:0/451204563 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f6b8ed2240 0x55f6b8ecc780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:44.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:44.006+0000 7fa6f77fe640 1 -- 172.21.15.26:0/451204563 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f6b8f4fb70 con 0x55f6b8ec7060 2024-09-11T23:29:44.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:44.006+0000 7fa6f6ffd640 1 --2- 172.21.15.26:0/451204563 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f6b8ec7980 0x55f6b8ecc040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:29:44.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:44.006+0000 7fa6f7fff640 1 --2- 172.21.15.26:0/451204563 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f6b8ed2240 0x55f6b8ecc780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:29:44.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:44.006+0000 7fa6f77fe640 1 --2- 172.21.15.26:0/451204563 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f6b8ec7060 0x55f6b8edeac0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fa6e8000c00 tx=0x7fa6e80039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=997be426c3925141 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:29:44.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:44.006+0000 7fa6d6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:29:44.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:44.006+0000 7fa6d6ffd640 1 -- 172.21.15.26:0/451204563 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6e800b4a0 con 0x55f6b8ec7060 2024-09-11T23:29:44.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:44.006+0000 7fa6d6ffd640 1 -- 172.21.15.26:0/451204563 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8542+0+0 (secure 0 0 0) 0x7fa6e8011440 con 0x55f6b8ec7060 2024-09-11T23:29:44.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:44.006+0000 7fa7004bc740 1 -- 172.21.15.26:0/451204563 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa6ec0c4c90 con 0x55f6b8ec7060 2024-09-11T23:29:44.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:44.007+0000 7fa6d6ffd640 1 -- 172.21.15.26:0/451204563 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7fa6e8003e80 con 0x55f6b8ec7060 2024-09-11T23:29:44.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:44.007+0000 7fa7004bc740 10 client.17139 fetch_fsmap learned FSMap version 57 2024-09-11T23:29:44.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:44.007+0000 7fa7004bc740 1 -- 172.21.15.26:0/451204563 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x55f6b8f50cc0 con 0x55f6b8ec7060 2024-09-11T23:29:44.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:44.007+0000 7fa6d6ffd640 1 -- 172.21.15.26:0/451204563 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6ec0c4c90 con 0x55f6b8ec7060 2024-09-11T23:29:44.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:44.007+0000 7fa6d6ffd640 1 -- 172.21.15.26:0/451204563 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 57) ==== 8366+0+0 (secure 0 0 0) 0x7fa6e802a070 con 0x55f6b8ec7060 2024-09-11T23:29:44.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:44.007+0000 7fa7004bc740 10 client.17139 fetch_fsmap finished waiting for FSMap version 57 2024-09-11T23:29:44.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:44.007+0000 7fa7004bc740 10 client.17139 resolve_mds: validated gid 24485 aka daemon mds.b 2024-09-11T23:29:44.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:44.007+0000 7fa7004bc740 20 client.17139 populate_metadata read hostname 'smithi026' 2024-09-11T23:29:44.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:44.007+0000 7fa7004bc740 1 --2- 172.21.15.26:0/451204563 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55f6b8f513e0 0x55f6b8f537d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:44.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:44.007+0000 7fa7004bc740 4 client.17139 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "a85f23b", "release": true, "await": true}] 2024-09-11T23:29:44.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:44.007+0000 7fa7004bc740 1 -- 172.21.15.26:0/451204563 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "quiesce db", "set_id": "a85f23b", "release": true, "await": true}) -- 0x55f6b8e340a0 con 0x55f6b8f513e0 2024-09-11T23:29:44.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:44.007+0000 7fa6f6ffd640 1 --2- 172.21.15.26:0/451204563 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55f6b8f513e0 0x55f6b8f537d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:44.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:44.008+0000 7fa6f6ffd640 1 --2- 172.21.15.26:0/451204563 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55f6b8f513e0 0x55f6b8f537d0 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a9c8d43ad8cfb7 server_cookie=d690a8ea8c443fdd in_seq=0 out_seq=0 2024-09-11T23:29:44.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:44.008+0000 7fa6d6ffd640 10 client.17139 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:29:44.052 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:29:43 smithi076 ceph-mon[35296]: pgmap v353: 97 pgs: 97 active+clean; 2.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 33 KiB/s wr, 11 op/s 2024-09-11T23:29:44.238 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:29:43 smithi136 ceph-mon[35121]: pgmap v353: 97 pgs: 97 active+clean; 2.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 33 KiB/s wr, 11 op/s 2024-09-11T23:29:45.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:45.212+0000 7fa6d6ffd640 1 -- 172.21.15.26:0/451204563 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x55f6b8e340a0 con 0x55f6b8f513e0 2024-09-11T23:29:45.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:45.212+0000 7fa6d6ffd640 10 client.17139 handle_command_reply: tid=0 2024-09-11T23:29:45.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:45.212+0000 7fa7004bc740 1 client.17139 shutdown 2024-09-11T23:29:45.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:45.213+0000 7fa7004bc740 1 -- 172.21.15.26:0/451204563 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f6b8ec7060 msgr2=0x55f6b8edeac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:45.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:45.213+0000 7fa7004bc740 1 --2- 172.21.15.26:0/451204563 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f6b8ec7060 0x55f6b8edeac0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fa6e8000c00 tx=0x7fa6e80039c0 comp rx=0 tx=0).stop 2024-09-11T23:29:45.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:45.213+0000 7fa7004bc740 1 -- 172.21.15.26:0/451204563 shutdown_connections 2024-09-11T23:29:45.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:45.213+0000 7fa7004bc740 1 --2- 172.21.15.26:0/451204563 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f6b8ed2240 0x55f6b8ecc780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:45.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:45.213+0000 7fa7004bc740 1 --2- 172.21.15.26:0/451204563 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55f6b8f513e0 0x55f6b8f537d0 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:45.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:45.213+0000 7fa7004bc740 1 --2- 172.21.15.26:0/451204563 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f6b8ec7980 0x55f6b8ecc040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:45.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:45.213+0000 7fa7004bc740 1 --2- 172.21.15.26:0/451204563 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f6b8ec7060 0x55f6b8edeac0 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:45.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:45.213+0000 7fa7004bc740 1 -- 172.21.15.26:0/451204563 >> 172.21.15.26:0/451204563 conn(0x55f6b8ec35f0 msgr2=0x55f6b8ec2750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:29:45.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:45.213+0000 7fa6d6ffd640 0 client.17139 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:29:45.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:45.213+0000 7fa7004bc740 1 -- 172.21.15.26:0/451204563 shutdown_connections 2024-09-11T23:29:45.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:45.213+0000 7fa7004bc740 1 -- 172.21.15.26:0/451204563 wait complete. 2024-09-11T23:29:45.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:45.213+0000 7fa7004bc740 20 client.17139 trim_cache size 0 max 16384 2024-09-11T23:29:45.214 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-11T23:29:45.214 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 57, 2024-09-11T23:29:45.214 INFO:teuthology.orchestra.run.smithi026.stdout: "leader": 24485, 2024-09-11T23:29:45.215 INFO:teuthology.orchestra.run.smithi026.stdout: "set_version": 44, 2024-09-11T23:29:45.215 INFO:teuthology.orchestra.run.smithi026.stdout: "sets": { 2024-09-11T23:29:45.215 INFO:teuthology.orchestra.run.smithi026.stdout: "a85f23b": { 2024-09-11T23:29:45.215 INFO:teuthology.orchestra.run.smithi026.stdout: "version": 44, 2024-09-11T23:29:45.215 INFO:teuthology.orchestra.run.smithi026.stdout: "age_ref": 0.0, 2024-09-11T23:29:45.215 INFO:teuthology.orchestra.run.smithi026.stdout: "state": { 2024-09-11T23:29:45.215 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "RELEASED", 2024-09-11T23:29:45.215 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.0 2024-09-11T23:29:45.215 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:29:45.215 INFO:teuthology.orchestra.run.smithi026.stdout: "timeout": 90.0, 2024-09-11T23:29:45.215 INFO:teuthology.orchestra.run.smithi026.stdout: "expiration": 142.6, 2024-09-11T23:29:45.215 INFO:teuthology.orchestra.run.smithi026.stdout: "members": { 2024-09-11T23:29:45.215 INFO:teuthology.orchestra.run.smithi026.stdout: "file:/": { 2024-09-11T23:29:45.216 INFO:teuthology.orchestra.run.smithi026.stdout: "excluded": false, 2024-09-11T23:29:45.216 INFO:teuthology.orchestra.run.smithi026.stdout: "state": { 2024-09-11T23:29:45.216 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "RELEASED", 2024-09-11T23:29:45.216 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.0 2024-09-11T23:29:45.216 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:29:45.216 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:29:45.216 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:29:45.216 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:29:45.216 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:29:45.216 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-11T23:29:45.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:45.214+0000 7fa6fe0e3640 1 -- 172.21.15.26:0/1125987602 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fa6e00a2d70 msgr2=0x7fa6e00a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:45.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:45.214+0000 7fa6fe0e3640 1 --2- 172.21.15.26:0/1125987602 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fa6e00a2d70 0x7fa6e00a5230 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fa6f0005fd0 tx=0x7fa6f0004ab0 comp rx=0 tx=0).stop 2024-09-11T23:29:45.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:45.214+0000 7fa6fe0e3640 1 -- 172.21.15.26:0/1125987602 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa6f8102790 msgr2=0x7fa6f8198960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:45.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:45.214+0000 7fa6fe0e3640 1 --2- 172.21.15.26:0/1125987602 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa6f8102790 0x7fa6f8198960 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fa6ec004770 tx=0x7fa6ec003c10 comp rx=0 tx=0).stop 2024-09-11T23:29:45.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:45.214+0000 7fa6fe0e3640 1 -- 172.21.15.26:0/1125987602 shutdown_connections 2024-09-11T23:29:45.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:45.214+0000 7fa6fe0e3640 1 --2- 172.21.15.26:0/1125987602 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fa6e00a2d70 0x7fa6e00a5230 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:45.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:45.214+0000 7fa6fe0e3640 1 --2- 172.21.15.26:0/1125987602 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa6f8106f10 0x7fa6f819ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:45.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:45.214+0000 7fa6fe0e3640 1 --2- 172.21.15.26:0/1125987602 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa6f8102790 0x7fa6f8198960 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:45.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:45.214+0000 7fa6fe0e3640 1 --2- 172.21.15.26:0/1125987602 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa6f8101de0 0x7fa6f8198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:45.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:45.214+0000 7fa6fe0e3640 1 -- 172.21.15.26:0/1125987602 >> 172.21.15.26:0/1125987602 conn(0x7fa6f80fd8c0 msgr2=0x7fa6f80fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:29:45.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:45.214+0000 7fa6fe0e3640 1 -- 172.21.15.26:0/1125987602 shutdown_connections 2024-09-11T23:29:45.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:29:45.214+0000 7fa6fe0e3640 1 -- 172.21.15.26:0/1125987602 wait complete. 2024-09-11T23:29:46.001 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.000+0000 7f37847f1640 1 -- 172.21.15.26:0/781749579 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f377c101ae0 msgr2=0x7f377c101f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:46.002 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.000+0000 7f37847f1640 1 --2- 172.21.15.26:0/781749579 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f377c101ae0 0x7f377c101f40 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f376c009930 tx=0x7f376c02a420 comp rx=0 tx=0).stop 2024-09-11T23:29:46.002 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.000+0000 7f37847f1640 1 -- 172.21.15.26:0/781749579 shutdown_connections 2024-09-11T23:29:46.002 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.000+0000 7f37847f1640 1 --2- 172.21.15.26:0/781749579 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f377c102480 0x7f377c10e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:46.002 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.000+0000 7f37847f1640 1 --2- 172.21.15.26:0/781749579 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f377c101ae0 0x7f377c101f40 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:46.002 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.000+0000 7f37847f1640 1 --2- 172.21.15.26:0/781749579 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f377c107b80 0x7f377c107f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:46.002 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.000+0000 7f37847f1640 1 -- 172.21.15.26:0/781749579 >> 172.21.15.26:0/781749579 conn(0x7f377c0fd7e0 msgr2=0x7f377c0ffc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:29:46.002 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.000+0000 7f37847f1640 1 -- 172.21.15.26:0/781749579 shutdown_connections 2024-09-11T23:29:46.002 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.000+0000 7f37847f1640 1 -- 172.21.15.26:0/781749579 wait complete. 2024-09-11T23:29:46.002 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.000+0000 7f37847f1640 1 Processor -- start 2024-09-11T23:29:46.002 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.000+0000 7f37847f1640 1 -- start start 2024-09-11T23:29:46.002 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.001+0000 7f37847f1640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f377c101ae0 0x7f377c198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:46.003 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.001+0000 7f37847f1640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f377c102480 0x7f377c198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:46.003 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.001+0000 7f37847f1640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f377c107b80 0x7f377c19ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:46.003 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.001+0000 7f37847f1640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f377c111f80 con 0x7f377c107b80 2024-09-11T23:29:46.003 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.001+0000 7f37847f1640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f377c111e00 con 0x7f377c102480 2024-09-11T23:29:46.003 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.001+0000 7f37847f1640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f377c112100 con 0x7f377c101ae0 2024-09-11T23:29:46.003 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.001+0000 7f3782d8e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f377c107b80 0x7f377c19ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:46.003 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.001+0000 7f3782d8e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f377c107b80 0x7f377c19ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42094/0 (socket says 172.21.15.26:42094) 2024-09-11T23:29:46.003 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.001+0000 7f3782d8e640 1 -- 172.21.15.26:0/10556895 learned_addr learned my addr 172.21.15.26:0/10556895 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:29:46.003 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.001+0000 7f3781d8c640 1 --2- 172.21.15.26:0/10556895 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f377c102480 0x7f377c198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:46.003 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.001+0000 7f378258d640 1 --2- 172.21.15.26:0/10556895 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f377c101ae0 0x7f377c198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:46.003 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.001+0000 7f3782d8e640 1 -- 172.21.15.26:0/10556895 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f377c101ae0 msgr2=0x7f377c198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:46.003 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.001+0000 7f3782d8e640 1 --2- 172.21.15.26:0/10556895 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f377c101ae0 0x7f377c198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:46.003 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.001+0000 7f3782d8e640 1 -- 172.21.15.26:0/10556895 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f377c102480 msgr2=0x7f377c198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:46.004 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.001+0000 7f3782d8e640 1 --2- 172.21.15.26:0/10556895 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f377c102480 0x7f377c198950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:46.004 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.001+0000 7f3782d8e640 1 -- 172.21.15.26:0/10556895 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f377c19d400 con 0x7f377c107b80 2024-09-11T23:29:46.004 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.001+0000 7f378258d640 1 --2- 172.21.15.26:0/10556895 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f377c101ae0 0x7f377c198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:29:46.004 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.001+0000 7f3782d8e640 1 --2- 172.21.15.26:0/10556895 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f377c107b80 0x7f377c19ccd0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f377400eb10 tx=0x7f377400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:29:46.004 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.002+0000 7f37737fe640 1 -- 172.21.15.26:0/10556895 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3774004b20 con 0x7f377c107b80 2024-09-11T23:29:46.004 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.002+0000 7f37737fe640 1 -- 172.21.15.26:0/10556895 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3774004cc0 con 0x7f377c107b80 2024-09-11T23:29:46.004 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.002+0000 7f37847f1640 1 -- 172.21.15.26:0/10556895 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f377c19d6f0 con 0x7f377c107b80 2024-09-11T23:29:46.004 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.002+0000 7f37737fe640 1 -- 172.21.15.26:0/10556895 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3774010760 con 0x7f377c107b80 2024-09-11T23:29:46.004 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.002+0000 7f37847f1640 1 -- 172.21.15.26:0/10556895 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f377c1a4f80 con 0x7f377c107b80 2024-09-11T23:29:46.006 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.005+0000 7f37737fe640 1 -- 172.21.15.26:0/10556895 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f3774005020 con 0x7f377c107b80 2024-09-11T23:29:46.006 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.005+0000 7f3787664740 1 Processor -- start 2024-09-11T23:29:46.006 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.005+0000 7f3787664740 1 -- start start 2024-09-11T23:29:46.007 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.006+0000 7f3787664740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555b1bb259d0 0x555b1bb25db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:46.007 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.006+0000 7f3787664740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555b1bb262f0 0x555b1bb34b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:46.007 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.006+0000 7f3787664740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555b1bb350c0 0x555b1bb374b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:46.007 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.006+0000 7f3787664740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x555b1ba8a2f0 con 0x555b1bb262f0 2024-09-11T23:29:46.007 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.006+0000 7f3787664740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x555b1ba32270 con 0x555b1bb350c0 2024-09-11T23:29:46.007 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.006+0000 7f3787664740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x555b1ba862b0 con 0x555b1bb259d0 2024-09-11T23:29:46.007 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.006+0000 7f378258d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555b1bb262f0 0x555b1bb34b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:46.007 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.006+0000 7f378258d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555b1bb262f0 0x555b1bb34b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42100/0 (socket says 172.21.15.26:42100) 2024-09-11T23:29:46.007 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.006+0000 7f378258d640 1 -- 172.21.15.26:0/1755880558 learned_addr learned my addr 172.21.15.26:0/1755880558 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:29:46.007 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.006+0000 7f3781d8c640 1 --2- 172.21.15.26:0/1755880558 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555b1bb259d0 0x555b1bb25db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:46.007 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.006+0000 7f37737fe640 1 --2- 172.21.15.26:0/10556895 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f37680a2da0 0x7f37680a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:46.007 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.006+0000 7f3781d8c640 1 --2- 172.21.15.26:0/1755880558 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555b1bb350c0 0x555b1bb374b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:46.007 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.006+0000 7f378258d640 1 -- 172.21.15.26:0/1755880558 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555b1bb259d0 msgr2=0x555b1bb25db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:29:46.008 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.006+0000 7f378258d640 1 --2- 172.21.15.26:0/1755880558 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555b1bb259d0 0x555b1bb25db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:46.008 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.006+0000 7f378258d640 1 -- 172.21.15.26:0/1755880558 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555b1bb350c0 msgr2=0x555b1bb374b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:29:46.008 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.006+0000 7f378258d640 1 --2- 172.21.15.26:0/1755880558 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555b1bb350c0 0x555b1bb374b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:46.008 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.006+0000 7f378258d640 1 -- 172.21.15.26:0/1755880558 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b1ba31340 con 0x555b1bb262f0 2024-09-11T23:29:46.008 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.006+0000 7f3782d8e640 1 --2- 172.21.15.26:0/10556895 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f37680a2da0 0x7f37680a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:46.008 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.006+0000 7f3781d8c640 1 --2- 172.21.15.26:0/1755880558 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555b1bb350c0 0x555b1bb374b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:29:46.008 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.006+0000 7f37737fe640 1 -- 172.21.15.26:0/10556895 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8542+0+0 (secure 0 0 0) 0x7f37740cfc10 con 0x7f377c107b80 2024-09-11T23:29:46.008 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.006+0000 7f378258d640 1 --2- 172.21.15.26:0/1755880558 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555b1bb262f0 0x555b1bb34b80 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f3778009d00 tx=0x7f37780117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b099e94a4ee15160 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:29:46.008 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.007+0000 7f3770ff9640 1 -- 172.21.15.26:0/1755880558 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f377801b470 con 0x555b1bb262f0 2024-09-11T23:29:46.008 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.007+0000 7f3770ff9640 1 -- 172.21.15.26:0/1755880558 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3778016070 con 0x555b1bb262f0 2024-09-11T23:29:46.008 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.007+0000 7f3770ff9640 1 -- 172.21.15.26:0/1755880558 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3778012b50 con 0x555b1bb262f0 2024-09-11T23:29:46.013 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.007+0000 7f3787664740 1 -- 172.21.15.26:0/1755880558 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555b1bb262f0 msgr2=0x555b1bb34b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:46.013 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.007+0000 7f3782d8e640 1 --2- 172.21.15.26:0/10556895 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f37680a2da0 0x7f37680a5260 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f3774098a90 tx=0x7f377406bf90 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:29:46.013 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.007+0000 7f3787664740 1 --2- 172.21.15.26:0/1755880558 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555b1bb262f0 0x555b1bb34b80 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f3778009d00 tx=0x7f37780117b0 comp rx=0 tx=0).stop 2024-09-11T23:29:46.013 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.007+0000 7f3787664740 1 -- 172.21.15.26:0/1755880558 shutdown_connections 2024-09-11T23:29:46.013 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.007+0000 7f3787664740 1 --2- 172.21.15.26:0/1755880558 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555b1bb350c0 0x555b1bb374b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:46.013 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.007+0000 7f3787664740 1 --2- 172.21.15.26:0/1755880558 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555b1bb262f0 0x555b1bb34b80 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:46.013 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.007+0000 7f3787664740 1 --2- 172.21.15.26:0/1755880558 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555b1bb259d0 0x555b1bb25db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:46.013 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.007+0000 7f3787664740 1 -- 172.21.15.26:0/1755880558 >> 172.21.15.26:0/1755880558 conn(0x555b1bb22060 msgr2=0x555b1bb22440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:29:46.013 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.008+0000 7f3787664740 1 -- 172.21.15.26:0/1755880558 shutdown_connections 2024-09-11T23:29:46.013 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.008+0000 7f3787664740 1 -- 172.21.15.26:0/1755880558 wait complete. 2024-09-11T23:29:46.014 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.008+0000 7f3787664740 1 -- start start 2024-09-11T23:29:46.014 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.008+0000 7f3787664740 1 Processor -- start 2024-09-11T23:29:46.014 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.008+0000 7f3787664740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555b1bb259d0 0x555b1bb32070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:46.014 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.008+0000 7f3787664740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555b1bb262f0 0x555b1bb325b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:46.014 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.008+0000 7f3787664740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555b1bb350c0 0x555b1bb32af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:46.014 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.008+0000 7f3787664740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x555b1bb37cf0 con 0x555b1bb259d0 2024-09-11T23:29:46.014 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.008+0000 7f3787664740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x555b1ba13290 con 0x555b1bb350c0 2024-09-11T23:29:46.014 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.008+0000 7f3787664740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x555b1bb37e70 con 0x555b1bb262f0 2024-09-11T23:29:46.014 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.008+0000 7f3787664740 10 client.0 _ll_register_callbacks cb 0x555b1ba44140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:29:46.014 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.008+0000 7f3781d8c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555b1bb259d0 0x555b1bb32070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:46.014 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.008+0000 7f3781d8c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555b1bb259d0 0x555b1bb32070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42108/0 (socket says 172.21.15.26:42108) 2024-09-11T23:29:46.014 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.008+0000 7f3781d8c640 1 -- 172.21.15.26:0/541969222 learned_addr learned my addr 172.21.15.26:0/541969222 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:29:46.014 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.009+0000 7f3781d8c640 1 --2- 172.21.15.26:0/541969222 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555b1bb350c0 0x555b1bb32af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:46.015 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.009+0000 7f378258d640 1 --2- 172.21.15.26:0/541969222 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555b1bb262f0 0x555b1bb325b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:46.015 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.009+0000 7f3781d8c640 1 -- 172.21.15.26:0/541969222 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555b1bb262f0 msgr2=0x555b1bb325b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:29:46.015 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.009+0000 7f3781d8c640 1 --2- 172.21.15.26:0/541969222 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555b1bb262f0 0x555b1bb325b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:46.015 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.009+0000 7f3781d8c640 1 -- 172.21.15.26:0/541969222 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555b1bb350c0 msgr2=0x555b1bb32af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:29:46.015 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.009+0000 7f3781d8c640 1 --2- 172.21.15.26:0/541969222 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555b1bb350c0 0x555b1bb32af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:46.015 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.009+0000 7f3781d8c640 1 -- 172.21.15.26:0/541969222 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b1b79d210 con 0x555b1bb259d0 2024-09-11T23:29:46.015 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.009+0000 7f3781d8c640 1 --2- 172.21.15.26:0/541969222 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555b1bb259d0 0x555b1bb32070 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f376c02aa10 tx=0x7f376c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4feb270deef6cc9f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:29:46.015 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.009+0000 7f3765ffb640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:29:46.015 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.009+0000 7f3765ffb640 1 -- 172.21.15.26:0/541969222 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f376c035040 con 0x555b1bb259d0 2024-09-11T23:29:46.015 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.009+0000 7f3765ffb640 1 -- 172.21.15.26:0/541969222 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8542+0+0 (secure 0 0 0) 0x7f376c005230 con 0x555b1bb259d0 2024-09-11T23:29:46.015 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.009+0000 7f3787664740 1 -- 172.21.15.26:0/541969222 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3778016070 con 0x555b1bb259d0 2024-09-11T23:29:46.015 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.009+0000 7f3765ffb640 1 -- 172.21.15.26:0/541969222 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7f376c005430 con 0x555b1bb259d0 2024-09-11T23:29:46.015 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.009+0000 7f3787664740 10 client.17151 fetch_fsmap learned FSMap version 57 2024-09-11T23:29:46.016 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.009+0000 7f3787664740 1 -- 172.21.15.26:0/541969222 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x555b1b869620 con 0x555b1bb259d0 2024-09-11T23:29:46.016 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.009+0000 7f3765ffb640 1 -- 172.21.15.26:0/541969222 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3778016070 con 0x555b1bb259d0 2024-09-11T23:29:46.016 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.010+0000 7f3765ffb640 1 -- 172.21.15.26:0/541969222 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 57) ==== 8366+0+0 (secure 0 0 0) 0x7f376c004c30 con 0x555b1bb259d0 2024-09-11T23:29:46.016 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.010+0000 7f3787664740 10 client.17151 fetch_fsmap finished waiting for FSMap version 57 2024-09-11T23:29:46.016 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.010+0000 7f3787664740 10 client.17151 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:29:46.016 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.010+0000 7f3787664740 20 client.17151 populate_metadata read hostname 'smithi026' 2024-09-11T23:29:46.016 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.010+0000 7f3787664740 1 --2- 172.21.15.26:0/541969222 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x555b1bb45080 0x555b1bb474b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:46.016 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.010+0000 7f3787664740 4 client.17151 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:29:46.016 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.010+0000 7f3787664740 1 -- 172.21.15.26:0/541969222 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555b1ba97a20 con 0x555b1bb45080 2024-09-11T23:29:46.016 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.010+0000 7f378258d640 1 --2- 172.21.15.26:0/541969222 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x555b1bb45080 0x555b1bb474b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:46.016 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.010+0000 7f3765ffb640 10 client.17151 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:29:46.016 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.010+0000 7f378258d640 1 --2- 172.21.15.26:0/541969222 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x555b1bb45080 0x555b1bb474b0 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d7c5db059e488de8 server_cookie=4775004586f6f47c in_seq=0 out_seq=0 2024-09-11T23:29:46.016 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.011+0000 7f3765ffb640 1 -- 172.21.15.26:0/541969222 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555b1ba97a20 con 0x555b1bb45080 2024-09-11T23:29:46.017 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.011+0000 7f3765ffb640 10 client.17151 handle_command_reply: tid=0 2024-09-11T23:29:46.017 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.011+0000 7f3787664740 1 client.17151 shutdown 2024-09-11T23:29:46.017 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.011+0000 7f3787664740 1 -- 172.21.15.26:0/541969222 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555b1bb259d0 msgr2=0x555b1bb32070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:46.017 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.011+0000 7f3787664740 1 --2- 172.21.15.26:0/541969222 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555b1bb259d0 0x555b1bb32070 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f376c02aa10 tx=0x7f376c009260 comp rx=0 tx=0).stop 2024-09-11T23:29:46.017 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.011+0000 7f3787664740 1 -- 172.21.15.26:0/541969222 shutdown_connections 2024-09-11T23:29:46.017 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.011+0000 7f3787664740 1 --2- 172.21.15.26:0/541969222 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555b1bb350c0 0x555b1bb32af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:46.017 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.011+0000 7f3787664740 1 --2- 172.21.15.26:0/541969222 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x555b1bb45080 0x555b1bb474b0 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:46.017 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.011+0000 7f3787664740 1 --2- 172.21.15.26:0/541969222 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555b1bb262f0 0x555b1bb325b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:46.017 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.011+0000 7f3787664740 1 --2- 172.21.15.26:0/541969222 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555b1bb259d0 0x555b1bb32070 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:46.017 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.011+0000 7f3787664740 1 -- 172.21.15.26:0/541969222 >> 172.21.15.26:0/541969222 conn(0x555b1bb22060 msgr2=0x555b1bb27dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:29:46.017 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.011+0000 7f3765ffb640 0 client.17151 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:29:46.017 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.011+0000 7f3787664740 1 -- 172.21.15.26:0/541969222 shutdown_connections 2024-09-11T23:29:46.017 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.011+0000 7f3787664740 1 -- 172.21.15.26:0/541969222 wait complete. 2024-09-11T23:29:46.018 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.011+0000 7f3787664740 20 client.17151 trim_cache size 0 max 16384 2024-09-11T23:29:46.027 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.027+0000 7f3787664740 1 Processor -- start 2024-09-11T23:29:46.027 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.027+0000 7f3787664740 1 -- start start 2024-09-11T23:29:46.028 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.027+0000 7f3787664740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555b1bb259d0 0x555b1bb210b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:46.028 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.027+0000 7f3787664740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555b1bb262f0 0x555b1bb4cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:46.028 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.027+0000 7f3787664740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555b1bb350c0 0x555b1bb4d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:46.028 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.027+0000 7f3787664740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f377c1ab710 con 0x555b1bb259d0 2024-09-11T23:29:46.028 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.027+0000 7f3787664740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f377c1ad280 con 0x555b1bb262f0 2024-09-11T23:29:46.028 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.027+0000 7f3787664740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f377c1abde0 con 0x555b1bb350c0 2024-09-11T23:29:46.028 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.027+0000 7f3781d8c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555b1bb259d0 0x555b1bb210b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:46.028 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.027+0000 7f3781d8c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555b1bb259d0 0x555b1bb210b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42114/0 (socket says 172.21.15.26:42114) 2024-09-11T23:29:46.028 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.027+0000 7f3781d8c640 1 -- 172.21.15.26:0/1081314850 learned_addr learned my addr 172.21.15.26:0/1081314850 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:29:46.028 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.027+0000 7f378258d640 1 --2- 172.21.15.26:0/1081314850 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555b1bb262f0 0x555b1bb4cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:46.028 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.027+0000 7f3781d8c640 1 -- 172.21.15.26:0/1081314850 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555b1bb350c0 msgr2=0x555b1bb4d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:29:46.028 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.027+0000 7f3781d8c640 1 --2- 172.21.15.26:0/1081314850 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555b1bb350c0 0x555b1bb4d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:46.028 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.027+0000 7f3781d8c640 1 -- 172.21.15.26:0/1081314850 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555b1bb262f0 msgr2=0x555b1bb4cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:29:46.029 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.027+0000 7f3781d8c640 1 --2- 172.21.15.26:0/1081314850 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555b1bb262f0 0x555b1bb4cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:46.029 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.027+0000 7f3781d8c640 1 -- 172.21.15.26:0/1081314850 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b1bb31000 con 0x555b1bb259d0 2024-09-11T23:29:46.029 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.027+0000 7f3781d8c640 1 --2- 172.21.15.26:0/1081314850 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555b1bb259d0 0x555b1bb210b0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f376c02a460 tx=0x7f376c004120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1741510f255cb67d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:29:46.029 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.028+0000 7f3767fff640 1 -- 172.21.15.26:0/1081314850 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f376c058020 con 0x555b1bb259d0 2024-09-11T23:29:46.029 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.028+0000 7f3767fff640 1 -- 172.21.15.26:0/1081314850 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f376c047070 con 0x555b1bb259d0 2024-09-11T23:29:46.029 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.028+0000 7f3767fff640 1 -- 172.21.15.26:0/1081314850 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f376c035040 con 0x555b1bb259d0 2024-09-11T23:29:46.032 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.028+0000 7f3787664740 1 -- 172.21.15.26:0/1081314850 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555b1bb259d0 msgr2=0x555b1bb210b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:46.032 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.028+0000 7f3787664740 1 --2- 172.21.15.26:0/1081314850 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555b1bb259d0 0x555b1bb210b0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f376c02a460 tx=0x7f376c004120 comp rx=0 tx=0).stop 2024-09-11T23:29:46.033 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.028+0000 7f3787664740 1 -- 172.21.15.26:0/1081314850 shutdown_connections 2024-09-11T23:29:46.033 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.028+0000 7f3787664740 1 --2- 172.21.15.26:0/1081314850 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555b1bb262f0 0x555b1bb4cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:46.033 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.028+0000 7f3787664740 1 --2- 172.21.15.26:0/1081314850 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555b1bb350c0 0x555b1bb4d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:46.033 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.028+0000 7f3787664740 1 --2- 172.21.15.26:0/1081314850 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555b1bb259d0 0x555b1bb210b0 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:46.033 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.028+0000 7f3787664740 1 -- 172.21.15.26:0/1081314850 >> 172.21.15.26:0/1081314850 conn(0x555b1bb22060 msgr2=0x555b1bb209a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:29:46.033 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.028+0000 7f3787664740 1 -- 172.21.15.26:0/1081314850 shutdown_connections 2024-09-11T23:29:46.033 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.028+0000 7f3787664740 1 -- 172.21.15.26:0/1081314850 wait complete. 2024-09-11T23:29:46.033 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.029+0000 7f3787664740 1 -- start start 2024-09-11T23:29:46.033 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.029+0000 7f3787664740 1 Processor -- start 2024-09-11T23:29:46.033 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.029+0000 7f3787664740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555b1bb259d0 0x555b1bb3d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:46.033 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.029+0000 7f3787664740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555b1bb262f0 0x555b1bb3d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:46.033 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.029+0000 7f3787664740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555b1bb350c0 0x555b1bb38610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:46.034 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.029+0000 7f3787664740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f377c1acbb0 con 0x555b1bb262f0 2024-09-11T23:29:46.034 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.029+0000 7f3787664740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f377c1ac4b0 con 0x555b1bb259d0 2024-09-11T23:29:46.034 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.029+0000 7f3787664740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f377c1a9730 con 0x555b1bb350c0 2024-09-11T23:29:46.034 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.029+0000 7f3787664740 10 client.0 _ll_register_callbacks cb 0x7f376c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:29:46.034 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.029+0000 7f378258d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555b1bb262f0 0x555b1bb3d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:46.034 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.029+0000 7f378258d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555b1bb262f0 0x555b1bb3d6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42122/0 (socket says 172.21.15.26:42122) 2024-09-11T23:29:46.034 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.029+0000 7f378258d640 1 -- 172.21.15.26:0/2254964166 learned_addr learned my addr 172.21.15.26:0/2254964166 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:29:46.034 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.029+0000 7f3781d8c640 1 --2- 172.21.15.26:0/2254964166 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555b1bb259d0 0x555b1bb3d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:46.034 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.029+0000 7f378258d640 1 -- 172.21.15.26:0/2254964166 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555b1bb350c0 msgr2=0x555b1bb38610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:29:46.034 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.029+0000 7f3781d8c640 1 --2- 172.21.15.26:0/2254964166 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555b1bb350c0 0x555b1bb38610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:46.034 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.030+0000 7f378258d640 1 --2- 172.21.15.26:0/2254964166 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555b1bb350c0 0x555b1bb38610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:46.034 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.030+0000 7f378258d640 1 -- 172.21.15.26:0/2254964166 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555b1bb259d0 msgr2=0x555b1bb3d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:29:46.034 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.030+0000 7f378258d640 1 --2- 172.21.15.26:0/2254964166 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555b1bb259d0 0x555b1bb3d1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:46.035 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.030+0000 7f378258d640 1 -- 172.21.15.26:0/2254964166 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b1bb38e10 con 0x555b1bb262f0 2024-09-11T23:29:46.035 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.030+0000 7f3781d8c640 1 --2- 172.21.15.26:0/2254964166 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555b1bb259d0 0x555b1bb3d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:29:46.035 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.030+0000 7f378258d640 1 --2- 172.21.15.26:0/2254964166 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555b1bb262f0 0x555b1bb3d6e0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f3778009bf0 tx=0x7f3778012690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c697e72ca337f48f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:29:46.035 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.030+0000 7f3765ffb640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:29:46.035 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.030+0000 7f3765ffb640 1 -- 172.21.15.26:0/2254964166 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3778005240 con 0x555b1bb262f0 2024-09-11T23:29:46.035 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.030+0000 7f3765ffb640 1 -- 172.21.15.26:0/2254964166 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8542+0+0 (secure 0 0 0) 0x7f37780053e0 con 0x555b1bb262f0 2024-09-11T23:29:46.035 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.030+0000 7f3787664740 1 -- 172.21.15.26:0/2254964166 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f376c047070 con 0x555b1bb262f0 2024-09-11T23:29:46.035 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.030+0000 7f3765ffb640 1 -- 172.21.15.26:0/2254964166 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7f37780124c0 con 0x555b1bb262f0 2024-09-11T23:29:46.035 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.030+0000 7f3787664740 10 client.17157 fetch_fsmap learned FSMap version 57 2024-09-11T23:29:46.035 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.030+0000 7f3787664740 1 -- 172.21.15.26:0/2254964166 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x555b1bbaf820 con 0x555b1bb262f0 2024-09-11T23:29:46.035 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.030+0000 7f3765ffb640 1 -- 172.21.15.26:0/2254964166 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f376c047070 con 0x555b1bb262f0 2024-09-11T23:29:46.035 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.030+0000 7f3765ffb640 1 -- 172.21.15.26:0/2254964166 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 57) ==== 8366+0+0 (secure 0 0 0) 0x7f3778003990 con 0x555b1bb262f0 2024-09-11T23:29:46.035 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.030+0000 7f3787664740 10 client.17157 fetch_fsmap finished waiting for FSMap version 57 2024-09-11T23:29:46.036 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.030+0000 7f3787664740 10 client.17157 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:29:46.036 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.031+0000 7f3787664740 20 client.17157 populate_metadata read hostname 'smithi026' 2024-09-11T23:29:46.036 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.031+0000 7f3787664740 1 --2- 172.21.15.26:0/2254964166 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x555b1bbafdb0 0x555b1bbb21d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:46.036 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.031+0000 7f3787664740 4 client.17157 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T23:29:46.036 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.031+0000 7f3787664740 1 -- 172.21.15.26:0/2254964166 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555b1ba91d70 con 0x555b1bbafdb0 2024-09-11T23:29:46.036 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.031+0000 7f3781d8c640 1 --2- 172.21.15.26:0/2254964166 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x555b1bbafdb0 0x555b1bbb21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:46.036 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.031+0000 7f3781d8c640 1 --2- 172.21.15.26:0/2254964166 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x555b1bbafdb0 0x555b1bbb21d0 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9091558428c7a3fd server_cookie=277810955931b5da in_seq=0 out_seq=0 2024-09-11T23:29:46.036 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.031+0000 7f3765ffb640 10 client.17157 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:29:46.041 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.041+0000 7f3765ffb640 1 -- 172.21.15.26:0/2254964166 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555b1ba91d70 con 0x555b1bbafdb0 2024-09-11T23:29:46.041 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.041+0000 7f3765ffb640 10 client.17157 handle_command_reply: tid=0 2024-09-11T23:29:46.041 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.041+0000 7f3787664740 1 client.17157 shutdown 2024-09-11T23:29:46.041 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.041+0000 7f3787664740 1 -- 172.21.15.26:0/2254964166 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555b1bb262f0 msgr2=0x555b1bb3d6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:46.041 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.041+0000 7f3787664740 1 --2- 172.21.15.26:0/2254964166 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555b1bb262f0 0x555b1bb3d6e0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f3778009bf0 tx=0x7f3778012690 comp rx=0 tx=0).stop 2024-09-11T23:29:46.041 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.041+0000 7f3787664740 1 -- 172.21.15.26:0/2254964166 shutdown_connections 2024-09-11T23:29:46.041 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.041+0000 7f3787664740 1 --2- 172.21.15.26:0/2254964166 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x555b1bbafdb0 0x555b1bbb21d0 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:46.041 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.041+0000 7f3787664740 1 --2- 172.21.15.26:0/2254964166 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555b1bb350c0 0x555b1bb38610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:46.042 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.041+0000 7f3765ffb640 0 client.17157 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:29:46.042 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.041+0000 7f3787664740 1 --2- 172.21.15.26:0/2254964166 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555b1bb262f0 0x555b1bb3d6e0 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:46.042 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.041+0000 7f3787664740 1 --2- 172.21.15.26:0/2254964166 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555b1bb259d0 0x555b1bb3d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:46.042 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.041+0000 7f3787664740 1 -- 172.21.15.26:0/2254964166 >> 172.21.15.26:0/2254964166 conn(0x555b1bb22060 msgr2=0x555b1bb32850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:29:46.042 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.041+0000 7f3787664740 1 -- 172.21.15.26:0/2254964166 shutdown_connections 2024-09-11T23:29:46.042 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.041+0000 7f3787664740 1 -- 172.21.15.26:0/2254964166 wait complete. 2024-09-11T23:29:46.042 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.041+0000 7f3787664740 20 client.17157 trim_cache size 0 max 16384 2024-09-11T23:29:46.042 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-09-11T23:29:46.042 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "message": "", 2024-09-11T23:29:46.042 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-09-11T23:29:46.042 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-09-11T23:29:46.043 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.042+0000 7f37847f1640 1 -- 172.21.15.26:0/10556895 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f37680a2da0 msgr2=0x7f37680a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:46.043 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.042+0000 7f37847f1640 1 --2- 172.21.15.26:0/10556895 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f37680a2da0 0x7f37680a5260 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f3774098a90 tx=0x7f377406bf90 comp rx=0 tx=0).stop 2024-09-11T23:29:46.043 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.042+0000 7f37847f1640 1 -- 172.21.15.26:0/10556895 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f377c107b80 msgr2=0x7f377c19ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:46.044 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.042+0000 7f37847f1640 1 --2- 172.21.15.26:0/10556895 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f377c107b80 0x7f377c19ccd0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f377400eb10 tx=0x7f377400c2d0 comp rx=0 tx=0).stop 2024-09-11T23:29:46.044 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.042+0000 7f37847f1640 1 -- 172.21.15.26:0/10556895 shutdown_connections 2024-09-11T23:29:46.044 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.042+0000 7f37847f1640 1 --2- 172.21.15.26:0/10556895 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f37680a2da0 0x7f37680a5260 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:46.044 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.042+0000 7f37847f1640 1 --2- 172.21.15.26:0/10556895 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f377c102480 0x7f377c198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:46.044 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.042+0000 7f37847f1640 1 --2- 172.21.15.26:0/10556895 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f377c107b80 0x7f377c19ccd0 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:46.044 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.042+0000 7f37847f1640 1 --2- 172.21.15.26:0/10556895 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f377c101ae0 0x7f377c198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:46.044 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.042+0000 7f37847f1640 1 -- 172.21.15.26:0/10556895 >> 172.21.15.26:0/10556895 conn(0x7f377c0fd7e0 msgr2=0x7f377c0fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:29:46.044 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.042+0000 7f37847f1640 1 -- 172.21.15.26:0/10556895 shutdown_connections 2024-09-11T23:29:46.044 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:46.042+0000 7f37847f1640 1 -- 172.21.15.26:0/10556895 wait complete. 2024-09-11T23:29:46.052 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:29:45 smithi076 ceph-mon[35296]: pgmap v354: 97 pgs: 97 active+clean; 2.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 33 KiB/s wr, 14 op/s 2024-09-11T23:29:46.121 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:29:45 smithi026 ceph-mon[31172]: pgmap v354: 97 pgs: 97 active+clean; 2.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 33 KiB/s wr, 14 op/s 2024-09-11T23:29:46.148 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'a85f23b', total seconds elapsed: 27.4 2024-09-11T23:29:46.238 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:29:45 smithi136 ceph-mon[35121]: pgmap v354: 97 pgs: 97 active+clean; 2.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 33 KiB/s wr, 14 op/s 2024-09-11T23:29:48.052 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:29:47 smithi076 ceph-mon[35296]: pgmap v355: 97 pgs: 97 active+clean; 2.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 938 B/s wr, 10 op/s 2024-09-11T23:29:48.238 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:29:47 smithi136 ceph-mon[35121]: pgmap v355: 97 pgs: 97 active+clean; 2.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 938 B/s wr, 10 op/s 2024-09-11T23:29:48.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:29:47 smithi026 ceph-mon[31172]: pgmap v355: 97 pgs: 97 active+clean; 2.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 938 B/s wr, 10 op/s 2024-09-11T23:29:50.052 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:29:49 smithi076 ceph-mon[35296]: pgmap v356: 97 pgs: 97 active+clean; 2.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 85 B/s wr, 8 op/s 2024-09-11T23:29:50.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:29:49 smithi136 ceph-mon[35121]: pgmap v356: 97 pgs: 97 active+clean; 2.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 85 B/s wr, 8 op/s 2024-09-11T23:29:50.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:29:49 smithi026 ceph-mon[31172]: pgmap v356: 97 pgs: 97 active+clean; 2.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 85 B/s wr, 8 op/s 2024-09-11T23:29:52.052 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:29:51 smithi076 ceph-mon[35296]: pgmap v357: 97 pgs: 97 active+clean; 2.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 28 KiB/s wr, 14 op/s 2024-09-11T23:29:52.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:29:51 smithi136 ceph-mon[35121]: pgmap v357: 97 pgs: 97 active+clean; 2.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 28 KiB/s wr, 14 op/s 2024-09-11T23:29:52.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:29:51 smithi026 ceph-mon[31172]: pgmap v357: 97 pgs: 97 active+clean; 2.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 28 KiB/s wr, 14 op/s 2024-09-11T23:29:54.052 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:29:53 smithi076 ceph-mon[35296]: pgmap v358: 97 pgs: 97 active+clean; 2.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 28 KiB/s wr, 10 op/s 2024-09-11T23:29:54.238 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:29:53 smithi136 ceph-mon[35121]: pgmap v358: 97 pgs: 97 active+clean; 2.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 28 KiB/s wr, 10 op/s 2024-09-11T23:29:54.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:29:53 smithi026 ceph-mon[31172]: pgmap v358: 97 pgs: 97 active+clean; 2.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 28 KiB/s wr, 10 op/s 2024-09-11T23:29:56.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:29:55 smithi136 ceph-mon[35121]: pgmap v359: 97 pgs: 97 active+clean; 2.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 32 KiB/s wr, 14 op/s 2024-09-11T23:29:56.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:29:55 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:29:56.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:29:55 smithi026 ceph-mon[31172]: pgmap v359: 97 pgs: 97 active+clean; 2.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 32 KiB/s wr, 14 op/s 2024-09-11T23:29:56.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:29:55 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:29:56.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:29:55 smithi076 ceph-mon[35296]: pgmap v359: 97 pgs: 97 active+clean; 2.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 32 KiB/s wr, 14 op/s 2024-09-11T23:29:56.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:29:55 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:29:57.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:29:57 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:29:57.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:29:57 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:29:57.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:29:57 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:29:57.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:29:57 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:29:57.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:29:57 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:29:57.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:29:57 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:29:57.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:29:57 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:29:57.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:29:57 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:29:57.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:29:57 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:29:57.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:29:57 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:29:57.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:29:57 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:29:57.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:29:57 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:29:57.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:29:57 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:29:57.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:29:57 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:29:57.501 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:29:57 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:29:58.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:29:58 smithi076 ceph-mon[35296]: pgmap v360: 97 pgs: 97 active+clean; 2.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 32 KiB/s wr, 13 op/s 2024-09-11T23:29:58.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:29:58 smithi136 ceph-mon[35121]: pgmap v360: 97 pgs: 97 active+clean; 2.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 32 KiB/s wr, 13 op/s 2024-09-11T23:29:58.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:29:58 smithi026 ceph-mon[31172]: pgmap v360: 97 pgs: 97 active+clean; 2.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 32 KiB/s wr, 13 op/s 2024-09-11T23:29:59.143 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.141+0000 7fd1e2dc4640 1 -- 172.21.15.26:0/128625199 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd1dc106a00 msgr2=0x7fd1dc106de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:59.143 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.141+0000 7fd1e2dc4640 1 --2- 172.21.15.26:0/128625199 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd1dc106a00 0x7fd1dc106de0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fd1d0009930 tx=0x7fd1d002a420 comp rx=0 tx=0).stop 2024-09-11T23:29:59.143 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.141+0000 7fd1e2dc4640 1 -- 172.21.15.26:0/128625199 shutdown_connections 2024-09-11T23:29:59.144 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.141+0000 7fd1e2dc4640 1 --2- 172.21.15.26:0/128625199 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd1dc101390 0x7fd1dc10d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:59.144 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.141+0000 7fd1e2dc4640 1 --2- 172.21.15.26:0/128625199 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd1dc1009f0 0x7fd1dc100e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:59.144 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.141+0000 7fd1e2dc4640 1 --2- 172.21.15.26:0/128625199 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd1dc106a00 0x7fd1dc106de0 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:59.144 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.141+0000 7fd1e2dc4640 1 -- 172.21.15.26:0/128625199 >> 172.21.15.26:0/128625199 conn(0x7fd1dc0fc6b0 msgr2=0x7fd1dc0fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:29:59.144 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.142+0000 7fd1e2dc4640 1 -- 172.21.15.26:0/128625199 shutdown_connections 2024-09-11T23:29:59.144 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.142+0000 7fd1e2dc4640 1 -- 172.21.15.26:0/128625199 wait complete. 2024-09-11T23:29:59.144 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.142+0000 7fd1e2dc4640 1 Processor -- start 2024-09-11T23:29:59.144 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.142+0000 7fd1e2dc4640 1 -- start start 2024-09-11T23:29:59.144 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.142+0000 7fd1e2dc4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd1dc1009f0 0x7fd1dc1972b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:59.144 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.142+0000 7fd1e2dc4640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd1dc101390 0x7fd1dc1977f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:59.144 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.142+0000 7fd1e2dc4640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd1dc106a00 0x7fd1dc19bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:59.144 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.142+0000 7fd1e2dc4640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd1dc110e40 con 0x7fd1dc1009f0 2024-09-11T23:29:59.144 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.142+0000 7fd1e2dc4640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fd1dc110cc0 con 0x7fd1dc106a00 2024-09-11T23:29:59.145 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.142+0000 7fd1e2dc4640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd1dc110fc0 con 0x7fd1dc101390 2024-09-11T23:29:59.145 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.142+0000 7fd1e0b60640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd1dc1009f0 0x7fd1dc1972b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:59.145 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.143+0000 7fd1e0b60640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd1dc1009f0 0x7fd1dc1972b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37714/0 (socket says 172.21.15.26:37714) 2024-09-11T23:29:59.145 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.143+0000 7fd1e0b60640 1 -- 172.21.15.26:0/3004680384 learned_addr learned my addr 172.21.15.26:0/3004680384 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:29:59.145 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.143+0000 7fd1dbfff640 1 --2- 172.21.15.26:0/3004680384 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd1dc101390 0x7fd1dc1977f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:59.145 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.143+0000 7fd1e1361640 1 --2- 172.21.15.26:0/3004680384 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd1dc106a00 0x7fd1dc19bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:59.145 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.143+0000 7fd1e0b60640 1 -- 172.21.15.26:0/3004680384 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd1dc101390 msgr2=0x7fd1dc1977f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:59.145 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.143+0000 7fd1e0b60640 1 --2- 172.21.15.26:0/3004680384 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd1dc101390 0x7fd1dc1977f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:59.145 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.143+0000 7fd1e0b60640 1 -- 172.21.15.26:0/3004680384 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd1dc106a00 msgr2=0x7fd1dc19bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:59.145 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.143+0000 7fd1e0b60640 1 --2- 172.21.15.26:0/3004680384 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd1dc106a00 0x7fd1dc19bb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:59.145 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.143+0000 7fd1e0b60640 1 -- 172.21.15.26:0/3004680384 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1dc19c2a0 con 0x7fd1dc1009f0 2024-09-11T23:29:59.145 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.143+0000 7fd1e1361640 1 --2- 172.21.15.26:0/3004680384 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd1dc106a00 0x7fd1dc19bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:29:59.145 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.143+0000 7fd1e0b60640 1 --2- 172.21.15.26:0/3004680384 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd1dc1009f0 0x7fd1dc1972b0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fd1d0009230 tx=0x7fd1d0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:29:59.145 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.143+0000 7fd1d9ffb640 1 -- 172.21.15.26:0/3004680384 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1d003f030 con 0x7fd1dc1009f0 2024-09-11T23:29:59.146 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.143+0000 7fd1d9ffb640 1 -- 172.21.15.26:0/3004680384 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1d0003ad0 con 0x7fd1dc1009f0 2024-09-11T23:29:59.146 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.143+0000 7fd1e2dc4640 1 -- 172.21.15.26:0/3004680384 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1dc19c530 con 0x7fd1dc1009f0 2024-09-11T23:29:59.146 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.143+0000 7fd1d9ffb640 1 -- 172.21.15.26:0/3004680384 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1d0035070 con 0x7fd1dc1009f0 2024-09-11T23:29:59.146 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.144+0000 7fd1e2dc4640 1 -- 172.21.15.26:0/3004680384 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1dc06a300 con 0x7fd1dc1009f0 2024-09-11T23:29:59.148 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.147+0000 7fd1d9ffb640 1 -- 172.21.15.26:0/3004680384 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fd1d0042030 con 0x7fd1dc1009f0 2024-09-11T23:29:59.148 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.147+0000 7fd1e5c3c740 1 Processor -- start 2024-09-11T23:29:59.148 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.147+0000 7fd1e5c3c740 1 -- start start 2024-09-11T23:29:59.148 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.147+0000 7fd1e5c3c740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5616608d2af0 0x5616608d2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:59.148 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.147+0000 7fd1e5c3c740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5616608d3410 0x5616608dd770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:59.148 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.147+0000 7fd1e5c3c740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5616608ddcb0 0x5616608e00a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:59.148 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.147+0000 7fd1e5c3c740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5616608372f0 con 0x5616608ddcb0 2024-09-11T23:29:59.148 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.147+0000 7fd1e5c3c740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5616607df270 con 0x5616608d2af0 2024-09-11T23:29:59.149 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.147+0000 7fd1e5c3c740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5616608332b0 con 0x5616608d3410 2024-09-11T23:29:59.149 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.147+0000 7fd1dbfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5616608ddcb0 0x5616608e00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:59.149 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.147+0000 7fd1dbfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5616608ddcb0 0x5616608e00a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37722/0 (socket says 172.21.15.26:37722) 2024-09-11T23:29:59.149 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.147+0000 7fd1dbfff640 1 -- 172.21.15.26:0/565311379 learned_addr learned my addr 172.21.15.26:0/565311379 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:29:59.149 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.147+0000 7fd1e1361640 1 --2- 172.21.15.26:0/565311379 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5616608d2af0 0x5616608d2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:59.149 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.147+0000 7fd1e0b60640 1 --2- 172.21.15.26:0/565311379 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5616608d3410 0x5616608dd770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:59.149 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.148+0000 7fd1d9ffb640 1 --2- 172.21.15.26:0/3004680384 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fd1cc0a2e20 0x7fd1cc0a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:59.149 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.148+0000 7fd1dbfff640 1 -- 172.21.15.26:0/565311379 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5616608d3410 msgr2=0x5616608dd770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:29:59.149 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.148+0000 7fd1dbfff640 1 --2- 172.21.15.26:0/565311379 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5616608d3410 0x5616608dd770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:59.149 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.148+0000 7fd1dbfff640 1 -- 172.21.15.26:0/565311379 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5616608d2af0 msgr2=0x5616608d2ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:29:59.149 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.148+0000 7fd1dbfff640 1 --2- 172.21.15.26:0/565311379 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5616608d2af0 0x5616608d2ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:59.149 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.148+0000 7fd1dbfff640 1 -- 172.21.15.26:0/565311379 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5616607de340 con 0x5616608ddcb0 2024-09-11T23:29:59.149 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.148+0000 7fd1e1361640 1 --2- 172.21.15.26:0/3004680384 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fd1cc0a2e20 0x7fd1cc0a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:59.150 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.148+0000 7fd1d9ffb640 1 -- 172.21.15.26:0/3004680384 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8542+0+0 (secure 0 0 0) 0x7fd1d00efa40 con 0x7fd1dc1009f0 2024-09-11T23:29:59.150 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.148+0000 7fd1dbfff640 1 --2- 172.21.15.26:0/565311379 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5616608ddcb0 0x5616608e00a0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fd1c800b740 tx=0x7fd1c800bc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14dd67d2d421530c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:29:59.150 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.148+0000 7fd1c6ffd640 1 -- 172.21.15.26:0/565311379 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1c8003dd0 con 0x5616608ddcb0 2024-09-11T23:29:59.150 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.148+0000 7fd1c6ffd640 1 -- 172.21.15.26:0/565311379 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1c8015070 con 0x5616608ddcb0 2024-09-11T23:29:59.150 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.148+0000 7fd1c6ffd640 1 -- 172.21.15.26:0/565311379 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1c8004e00 con 0x5616608ddcb0 2024-09-11T23:29:59.154 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.149+0000 7fd1e5c3c740 1 -- 172.21.15.26:0/565311379 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5616608ddcb0 msgr2=0x5616608e00a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:59.154 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.149+0000 7fd1e5c3c740 1 --2- 172.21.15.26:0/565311379 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5616608ddcb0 0x5616608e00a0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fd1c800b740 tx=0x7fd1c800bc00 comp rx=0 tx=0).stop 2024-09-11T23:29:59.154 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.149+0000 7fd1e1361640 1 --2- 172.21.15.26:0/3004680384 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fd1cc0a2e20 0x7fd1cc0a52e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fd1d4005fd0 tx=0x7fd1d4004a40 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:29:59.154 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.149+0000 7fd1e5c3c740 1 -- 172.21.15.26:0/565311379 shutdown_connections 2024-09-11T23:29:59.154 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.149+0000 7fd1e5c3c740 1 --2- 172.21.15.26:0/565311379 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5616608ddcb0 0x5616608e00a0 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:59.154 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.149+0000 7fd1e5c3c740 1 --2- 172.21.15.26:0/565311379 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5616608d3410 0x5616608dd770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:59.154 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.149+0000 7fd1e5c3c740 1 --2- 172.21.15.26:0/565311379 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5616608d2af0 0x5616608d2ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:59.154 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.149+0000 7fd1e5c3c740 1 -- 172.21.15.26:0/565311379 >> 172.21.15.26:0/565311379 conn(0x5616608cf060 msgr2=0x5616608cf440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:29:59.154 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.149+0000 7fd1e5c3c740 1 -- 172.21.15.26:0/565311379 shutdown_connections 2024-09-11T23:29:59.155 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.149+0000 7fd1e5c3c740 1 -- 172.21.15.26:0/565311379 wait complete. 2024-09-11T23:29:59.155 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.149+0000 7fd1e5c3c740 1 -- start start 2024-09-11T23:29:59.155 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.149+0000 7fd1e5c3c740 1 Processor -- start 2024-09-11T23:29:59.155 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.150+0000 7fd1e5c3c740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5616608d2af0 0x5616608dab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:59.155 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.150+0000 7fd1e5c3c740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5616608d3410 0x5616608db0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:59.155 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.150+0000 7fd1e5c3c740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5616608ddcb0 0x5616608db600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:59.155 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.150+0000 7fd1e5c3c740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5616608e0880 con 0x5616608ddcb0 2024-09-11T23:29:59.155 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.150+0000 7fd1e5c3c740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5616607c0290 con 0x5616608d3410 2024-09-11T23:29:59.155 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.150+0000 7fd1e5c3c740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5616608e0a00 con 0x5616608d2af0 2024-09-11T23:29:59.155 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.150+0000 7fd1e5c3c740 10 client.0 _ll_register_callbacks cb 0x5616607f1140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:29:59.155 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.150+0000 7fd1e1361640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5616608ddcb0 0x5616608db600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:59.155 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.150+0000 7fd1e1361640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5616608ddcb0 0x5616608db600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37734/0 (socket says 172.21.15.26:37734) 2024-09-11T23:29:59.155 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.150+0000 7fd1e1361640 1 -- 172.21.15.26:0/292447780 learned_addr learned my addr 172.21.15.26:0/292447780 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:29:59.156 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.150+0000 7fd1e0b60640 1 --2- 172.21.15.26:0/292447780 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5616608d2af0 0x5616608dab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:59.156 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.150+0000 7fd1dbfff640 1 --2- 172.21.15.26:0/292447780 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5616608d3410 0x5616608db0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:59.156 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.150+0000 7fd1e1361640 1 -- 172.21.15.26:0/292447780 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5616608d2af0 msgr2=0x5616608dab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:29:59.156 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.150+0000 7fd1e1361640 1 --2- 172.21.15.26:0/292447780 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5616608d2af0 0x5616608dab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:59.156 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.150+0000 7fd1e1361640 1 -- 172.21.15.26:0/292447780 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5616608d3410 msgr2=0x5616608db0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:29:59.156 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.150+0000 7fd1e1361640 1 --2- 172.21.15.26:0/292447780 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5616608d3410 0x5616608db0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:59.156 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.150+0000 7fd1e1361640 1 -- 172.21.15.26:0/292447780 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56166054a210 con 0x5616608ddcb0 2024-09-11T23:29:59.156 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.151+0000 7fd1e1361640 1 --2- 172.21.15.26:0/292447780 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5616608ddcb0 0x5616608db600 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fd1d400f990 tx=0x7fd1d4004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9d54174582654fc server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:29:59.156 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.151+0000 7fd1bbfff640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:29:59.156 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.151+0000 7fd1bbfff640 1 -- 172.21.15.26:0/292447780 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1d401ede0 con 0x5616608ddcb0 2024-09-11T23:29:59.156 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.151+0000 7fd1bbfff640 1 -- 172.21.15.26:0/292447780 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8542+0+0 (secure 0 0 0) 0x7fd1d401b410 con 0x5616608ddcb0 2024-09-11T23:29:59.156 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.151+0000 7fd1e5c3c740 1 -- 172.21.15.26:0/292447780 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd1c8015070 con 0x5616608ddcb0 2024-09-11T23:29:59.156 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.151+0000 7fd1bbfff640 1 -- 172.21.15.26:0/292447780 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7fd1d400fe50 con 0x5616608ddcb0 2024-09-11T23:29:59.157 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.151+0000 7fd1e5c3c740 10 client.17169 fetch_fsmap learned FSMap version 57 2024-09-11T23:29:59.157 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.151+0000 7fd1e5c3c740 1 -- 172.21.15.26:0/292447780 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x561660616620 con 0x5616608ddcb0 2024-09-11T23:29:59.157 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.151+0000 7fd1bbfff640 1 -- 172.21.15.26:0/292447780 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1c8015070 con 0x5616608ddcb0 2024-09-11T23:29:59.157 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.151+0000 7fd1bbfff640 1 -- 172.21.15.26:0/292447780 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 57) ==== 8366+0+0 (secure 0 0 0) 0x7fd1d4025020 con 0x5616608ddcb0 2024-09-11T23:29:59.157 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.152+0000 7fd1e5c3c740 10 client.17169 fetch_fsmap finished waiting for FSMap version 57 2024-09-11T23:29:59.157 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.152+0000 7fd1e5c3c740 10 client.17169 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:29:59.157 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.152+0000 7fd1e5c3c740 20 client.17169 populate_metadata read hostname 'smithi026' 2024-09-11T23:29:59.157 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.152+0000 7fd1e5c3c740 1 --2- 172.21.15.26:0/292447780 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5616608edc50 0x5616608f0080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:59.157 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.152+0000 7fd1e5c3c740 4 client.17169 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:29:59.157 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.152+0000 7fd1e5c3c740 1 -- 172.21.15.26:0/292447780 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561660844a20 con 0x5616608edc50 2024-09-11T23:29:59.157 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.152+0000 7fd1e0b60640 1 --2- 172.21.15.26:0/292447780 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5616608edc50 0x5616608f0080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:59.157 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.152+0000 7fd1bbfff640 10 client.17169 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:29:59.157 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.152+0000 7fd1e0b60640 1 --2- 172.21.15.26:0/292447780 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5616608edc50 0x5616608f0080 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8576e5cc6bf8670f server_cookie=98f78685e165dd2f in_seq=0 out_seq=0 2024-09-11T23:29:59.158 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.153+0000 7fd1bbfff640 1 -- 172.21.15.26:0/292447780 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561660844a20 con 0x5616608edc50 2024-09-11T23:29:59.158 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.153+0000 7fd1bbfff640 10 client.17169 handle_command_reply: tid=0 2024-09-11T23:29:59.158 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.153+0000 7fd1e5c3c740 1 client.17169 shutdown 2024-09-11T23:29:59.158 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.153+0000 7fd1e5c3c740 1 -- 172.21.15.26:0/292447780 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5616608ddcb0 msgr2=0x5616608db600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:59.158 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.153+0000 7fd1e5c3c740 1 --2- 172.21.15.26:0/292447780 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5616608ddcb0 0x5616608db600 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fd1d400f990 tx=0x7fd1d4004ab0 comp rx=0 tx=0).stop 2024-09-11T23:29:59.158 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.153+0000 7fd1e5c3c740 1 -- 172.21.15.26:0/292447780 shutdown_connections 2024-09-11T23:29:59.158 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.153+0000 7fd1e5c3c740 1 --2- 172.21.15.26:0/292447780 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5616608d3410 0x5616608db0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:59.158 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.153+0000 7fd1e5c3c740 1 --2- 172.21.15.26:0/292447780 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5616608edc50 0x5616608f0080 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:59.158 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.153+0000 7fd1e5c3c740 1 --2- 172.21.15.26:0/292447780 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5616608ddcb0 0x5616608db600 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:59.158 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.153+0000 7fd1e5c3c740 1 --2- 172.21.15.26:0/292447780 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5616608d2af0 0x5616608dab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:59.158 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.153+0000 7fd1bbfff640 0 client.17169 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:29:59.159 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.153+0000 7fd1e5c3c740 1 -- 172.21.15.26:0/292447780 >> 172.21.15.26:0/292447780 conn(0x5616608cf060 msgr2=0x5616608d4d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:29:59.159 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.153+0000 7fd1e5c3c740 1 -- 172.21.15.26:0/292447780 shutdown_connections 2024-09-11T23:29:59.159 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.153+0000 7fd1e5c3c740 1 -- 172.21.15.26:0/292447780 wait complete. 2024-09-11T23:29:59.159 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.153+0000 7fd1e5c3c740 20 client.17169 trim_cache size 0 max 16384 2024-09-11T23:29:59.172 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.168+0000 7fd1e5c3c740 1 Processor -- start 2024-09-11T23:29:59.172 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.168+0000 7fd1e5c3c740 1 -- start start 2024-09-11T23:29:59.172 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.168+0000 7fd1e5c3c740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5616608d2af0 0x5616608db430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:59.172 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.168+0000 7fd1e5c3c740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5616608d3410 0x5616608f5990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:59.172 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.168+0000 7fd1e5c3c740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5616608ddcb0 0x5616608f5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:59.173 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.168+0000 7fd1e5c3c740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd1dc1ad8b0 con 0x5616608ddcb0 2024-09-11T23:29:59.173 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.168+0000 7fd1e5c3c740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fd1dc1af420 con 0x5616608d3410 2024-09-11T23:29:59.173 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.168+0000 7fd1e5c3c740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd1dc1adf80 con 0x5616608d2af0 2024-09-11T23:29:59.173 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.168+0000 7fd1e1361640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5616608ddcb0 0x5616608f5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:59.173 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.169+0000 7fd1e1361640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5616608ddcb0 0x5616608f5ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37738/0 (socket says 172.21.15.26:37738) 2024-09-11T23:29:59.173 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.169+0000 7fd1e1361640 1 -- 172.21.15.26:0/545666115 learned_addr learned my addr 172.21.15.26:0/545666115 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:29:59.173 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.169+0000 7fd1e0b60640 1 --2- 172.21.15.26:0/545666115 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5616608d2af0 0x5616608db430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:59.173 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.169+0000 7fd1dbfff640 1 --2- 172.21.15.26:0/545666115 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5616608d3410 0x5616608f5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:59.173 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.169+0000 7fd1e1361640 1 -- 172.21.15.26:0/545666115 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5616608d2af0 msgr2=0x5616608db430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:29:59.173 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.169+0000 7fd1e1361640 1 --2- 172.21.15.26:0/545666115 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5616608d2af0 0x5616608db430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:59.173 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.169+0000 7fd1e1361640 1 -- 172.21.15.26:0/545666115 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5616608d3410 msgr2=0x5616608f5990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:29:59.173 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.169+0000 7fd1e1361640 1 --2- 172.21.15.26:0/545666115 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5616608d3410 0x5616608f5990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:59.173 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.169+0000 7fd1e1361640 1 -- 172.21.15.26:0/545666115 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5616608cee90 con 0x5616608ddcb0 2024-09-11T23:29:59.174 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.169+0000 7fd1dbfff640 1 --2- 172.21.15.26:0/545666115 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5616608d3410 0x5616608f5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:29:59.174 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.169+0000 7fd1e1361640 1 --2- 172.21.15.26:0/545666115 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5616608ddcb0 0x5616608f5ed0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fd1d4000f80 tx=0x7fd1d401e820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc8ef59168ffe349 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:29:59.174 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.169+0000 7fd1c67fc640 1 -- 172.21.15.26:0/545666115 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1d4033030 con 0x5616608ddcb0 2024-09-11T23:29:59.174 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.169+0000 7fd1c67fc640 1 -- 172.21.15.26:0/545666115 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1d4027bc0 con 0x5616608ddcb0 2024-09-11T23:29:59.174 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.169+0000 7fd1c67fc640 1 -- 172.21.15.26:0/545666115 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1d4011910 con 0x5616608ddcb0 2024-09-11T23:29:59.174 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.170+0000 7fd1e5c3c740 1 -- 172.21.15.26:0/545666115 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5616608ddcb0 msgr2=0x5616608f5ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:59.174 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.170+0000 7fd1e5c3c740 1 --2- 172.21.15.26:0/545666115 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5616608ddcb0 0x5616608f5ed0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fd1d4000f80 tx=0x7fd1d401e820 comp rx=0 tx=0).stop 2024-09-11T23:29:59.174 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.170+0000 7fd1e5c3c740 1 -- 172.21.15.26:0/545666115 shutdown_connections 2024-09-11T23:29:59.174 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.170+0000 7fd1e5c3c740 1 --2- 172.21.15.26:0/545666115 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5616608d3410 0x5616608f5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:59.174 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.170+0000 7fd1e5c3c740 1 --2- 172.21.15.26:0/545666115 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5616608ddcb0 0x5616608f5ed0 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:59.174 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.170+0000 7fd1e5c3c740 1 --2- 172.21.15.26:0/545666115 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5616608d2af0 0x5616608db430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:59.174 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.170+0000 7fd1e5c3c740 1 -- 172.21.15.26:0/545666115 >> 172.21.15.26:0/545666115 conn(0x5616608cf060 msgr2=0x5616608dcc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:29:59.174 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.170+0000 7fd1e5c3c740 1 -- 172.21.15.26:0/545666115 shutdown_connections 2024-09-11T23:29:59.174 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.170+0000 7fd1e5c3c740 1 -- 172.21.15.26:0/545666115 wait complete. 2024-09-11T23:29:59.175 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.170+0000 7fd1e5c3c740 1 -- start start 2024-09-11T23:29:59.175 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.170+0000 7fd1e5c3c740 1 Processor -- start 2024-09-11T23:29:59.175 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.171+0000 7fd1e5c3c740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5616608d2af0 0x5616608e5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:59.175 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.171+0000 7fd1e5c3c740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5616608d3410 0x5616608e61d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:59.175 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.171+0000 7fd1e5c3c740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5616608ddcb0 0x5616608e2c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:59.175 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.171+0000 7fd1e5c3c740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd1dc1aed50 con 0x5616608d3410 2024-09-11T23:29:59.175 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.171+0000 7fd1e5c3c740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fd1dc1ae650 con 0x5616608d2af0 2024-09-11T23:29:59.175 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.171+0000 7fd1e5c3c740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd1dc1ab8b0 con 0x5616608ddcb0 2024-09-11T23:29:59.175 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.171+0000 7fd1e5c3c740 10 client.0 _ll_register_callbacks cb 0x7fd1d4009d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:29:59.175 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.171+0000 7fd1dbfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5616608d3410 0x5616608e61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:59.175 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.171+0000 7fd1dbfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5616608d3410 0x5616608e61d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37740/0 (socket says 172.21.15.26:37740) 2024-09-11T23:29:59.175 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.171+0000 7fd1dbfff640 1 -- 172.21.15.26:0/3988865935 learned_addr learned my addr 172.21.15.26:0/3988865935 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:29:59.175 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.171+0000 7fd1e1361640 1 --2- 172.21.15.26:0/3988865935 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5616608ddcb0 0x5616608e2c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:59.175 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.171+0000 7fd1e0b60640 1 --2- 172.21.15.26:0/3988865935 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5616608d2af0 0x5616608e5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:59.176 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.171+0000 7fd1dbfff640 1 -- 172.21.15.26:0/3988865935 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5616608ddcb0 msgr2=0x5616608e2c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:29:59.176 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.171+0000 7fd1dbfff640 1 --2- 172.21.15.26:0/3988865935 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5616608ddcb0 0x5616608e2c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:59.176 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.171+0000 7fd1dbfff640 1 -- 172.21.15.26:0/3988865935 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5616608d2af0 msgr2=0x5616608e5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:29:59.176 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.171+0000 7fd1dbfff640 1 --2- 172.21.15.26:0/3988865935 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5616608d2af0 0x5616608e5c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:59.176 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.171+0000 7fd1dbfff640 1 -- 172.21.15.26:0/3988865935 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5616608e6870 con 0x5616608d3410 2024-09-11T23:29:59.176 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.171+0000 7fd1e1361640 1 --2- 172.21.15.26:0/3988865935 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5616608ddcb0 0x5616608e2c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:29:59.176 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.171+0000 7fd1e0b60640 1 --2- 172.21.15.26:0/3988865935 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5616608d2af0 0x5616608e5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:29:59.176 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.171+0000 7fd1dbfff640 1 --2- 172.21.15.26:0/3988865935 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5616608d3410 0x5616608e61d0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fd1c800b740 tx=0x7fd1c800bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88c8aa3c39204131 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:29:59.176 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.171+0000 7fd1bbfff640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:29:59.176 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.171+0000 7fd1bbfff640 1 -- 172.21.15.26:0/3988865935 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1c80047e0 con 0x5616608d3410 2024-09-11T23:29:59.176 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.172+0000 7fd1bbfff640 1 -- 172.21.15.26:0/3988865935 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8542+0+0 (secure 0 0 0) 0x7fd1c8025480 con 0x5616608d3410 2024-09-11T23:29:59.176 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.172+0000 7fd1e5c3c740 1 -- 172.21.15.26:0/3988865935 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd1d4027bc0 con 0x5616608d3410 2024-09-11T23:29:59.176 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.172+0000 7fd1bbfff640 1 -- 172.21.15.26:0/3988865935 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7fd1c8033050 con 0x5616608d3410 2024-09-11T23:29:59.177 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.172+0000 7fd1e5c3c740 10 client.17175 fetch_fsmap learned FSMap version 57 2024-09-11T23:29:59.177 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.172+0000 7fd1e5c3c740 1 -- 172.21.15.26:0/3988865935 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x561660958eb0 con 0x5616608d3410 2024-09-11T23:29:59.177 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.172+0000 7fd1bbfff640 1 -- 172.21.15.26:0/3988865935 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1d4027bc0 con 0x5616608d3410 2024-09-11T23:29:59.177 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.172+0000 7fd1bbfff640 1 -- 172.21.15.26:0/3988865935 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 57) ==== 8366+0+0 (secure 0 0 0) 0x7fd1c80116c0 con 0x5616608d3410 2024-09-11T23:29:59.177 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.172+0000 7fd1e5c3c740 10 client.17175 fetch_fsmap finished waiting for FSMap version 57 2024-09-11T23:29:59.177 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.172+0000 7fd1e5c3c740 10 client.17175 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:29:59.177 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.172+0000 7fd1e5c3c740 20 client.17175 populate_metadata read hostname 'smithi026' 2024-09-11T23:29:59.177 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.172+0000 7fd1e5c3c740 1 --2- 172.21.15.26:0/3988865935 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5616609593b0 0x56166095b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:59.177 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.172+0000 7fd1e5c3c740 4 client.17175 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T23:29:59.177 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.172+0000 7fd1e5c3c740 1 -- 172.21.15.26:0/3988865935 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56166083ed70 con 0x5616609593b0 2024-09-11T23:29:59.177 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.172+0000 7fd1e0b60640 1 --2- 172.21.15.26:0/3988865935 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5616609593b0 0x56166095b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:59.177 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.173+0000 7fd1e0b60640 1 --2- 172.21.15.26:0/3988865935 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5616609593b0 0x56166095b7d0 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=30b93de1e0373e9b server_cookie=be5b87acdce58f75 in_seq=0 out_seq=0 2024-09-11T23:29:59.177 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.173+0000 7fd1bbfff640 10 client.17175 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:29:59.183 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.182+0000 7fd1bbfff640 1 -- 172.21.15.26:0/3988865935 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56166083ed70 con 0x5616609593b0 2024-09-11T23:29:59.183 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.182+0000 7fd1bbfff640 10 client.17175 handle_command_reply: tid=0 2024-09-11T23:29:59.183 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.182+0000 7fd1e5c3c740 1 client.17175 shutdown 2024-09-11T23:29:59.183 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.183+0000 7fd1e5c3c740 1 -- 172.21.15.26:0/3988865935 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5616608d3410 msgr2=0x5616608e61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:59.183 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.183+0000 7fd1e5c3c740 1 --2- 172.21.15.26:0/3988865935 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5616608d3410 0x5616608e61d0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fd1c800b740 tx=0x7fd1c800bee0 comp rx=0 tx=0).stop 2024-09-11T23:29:59.183 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.183+0000 7fd1e5c3c740 1 -- 172.21.15.26:0/3988865935 shutdown_connections 2024-09-11T23:29:59.183 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.183+0000 7fd1e5c3c740 1 --2- 172.21.15.26:0/3988865935 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5616609593b0 0x56166095b7d0 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:59.183 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.183+0000 7fd1e5c3c740 1 --2- 172.21.15.26:0/3988865935 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5616608ddcb0 0x5616608e2c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:59.183 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.183+0000 7fd1e5c3c740 1 --2- 172.21.15.26:0/3988865935 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5616608d3410 0x5616608e61d0 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:59.183 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.183+0000 7fd1e5c3c740 1 --2- 172.21.15.26:0/3988865935 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5616608d2af0 0x5616608e5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:59.183 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.183+0000 7fd1bbfff640 0 client.17175 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:29:59.183 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.183+0000 7fd1e5c3c740 1 -- 172.21.15.26:0/3988865935 >> 172.21.15.26:0/3988865935 conn(0x5616608cf060 msgr2=0x5616608d4550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:29:59.184 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.183+0000 7fd1e5c3c740 1 -- 172.21.15.26:0/3988865935 shutdown_connections 2024-09-11T23:29:59.184 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.183+0000 7fd1e5c3c740 1 -- 172.21.15.26:0/3988865935 wait complete. 2024-09-11T23:29:59.184 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.183+0000 7fd1e5c3c740 20 client.17175 trim_cache size 0 max 16384 2024-09-11T23:29:59.184 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-09-11T23:29:59.184 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "message": "", 2024-09-11T23:29:59.184 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-09-11T23:29:59.184 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-09-11T23:29:59.185 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.184+0000 7fd1e2dc4640 1 -- 172.21.15.26:0/3004680384 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fd1cc0a2e20 msgr2=0x7fd1cc0a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:59.185 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.184+0000 7fd1e2dc4640 1 --2- 172.21.15.26:0/3004680384 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fd1cc0a2e20 0x7fd1cc0a52e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fd1d4005fd0 tx=0x7fd1d4004a40 comp rx=0 tx=0).stop 2024-09-11T23:29:59.185 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.184+0000 7fd1e2dc4640 1 -- 172.21.15.26:0/3004680384 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd1dc1009f0 msgr2=0x7fd1dc1972b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:59.185 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.184+0000 7fd1e2dc4640 1 --2- 172.21.15.26:0/3004680384 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd1dc1009f0 0x7fd1dc1972b0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fd1d0009230 tx=0x7fd1d0004270 comp rx=0 tx=0).stop 2024-09-11T23:29:59.185 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.184+0000 7fd1e2dc4640 1 -- 172.21.15.26:0/3004680384 shutdown_connections 2024-09-11T23:29:59.185 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.184+0000 7fd1e2dc4640 1 --2- 172.21.15.26:0/3004680384 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fd1cc0a2e20 0x7fd1cc0a52e0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:59.185 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.184+0000 7fd1e2dc4640 1 --2- 172.21.15.26:0/3004680384 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd1dc106a00 0x7fd1dc19bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:59.186 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.184+0000 7fd1e2dc4640 1 --2- 172.21.15.26:0/3004680384 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd1dc101390 0x7fd1dc1977f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:59.186 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.184+0000 7fd1e2dc4640 1 --2- 172.21.15.26:0/3004680384 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd1dc1009f0 0x7fd1dc1972b0 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:59.186 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.184+0000 7fd1e2dc4640 1 -- 172.21.15.26:0/3004680384 >> 172.21.15.26:0/3004680384 conn(0x7fd1dc0fc6b0 msgr2=0x7fd1dc0fce10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:29:59.186 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.184+0000 7fd1e2dc4640 1 -- 172.21.15.26:0/3004680384 shutdown_connections 2024-09-11T23:29:59.186 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:29:59.184+0000 7fd1e2dc4640 1 -- 172.21.15.26:0/3004680384 wait complete. 2024-09-11T23:30:00.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:30:00 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mon-a[31168]: 2024-09-11T23:29:59.999+0000 7f5d93536640 -1 log_channel(cluster) log [ERR] : Health detail: HEALTH_ERR 1 MDSs report damaged metadata 2024-09-11T23:30:00.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:30:00 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mon-a[31168]: 2024-09-11T23:29:59.999+0000 7f5d93536640 -1 log_channel(cluster) log [ERR] : [ERR] MDS_DAMAGE: 1 MDSs report damaged metadata 2024-09-11T23:30:00.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:30:00 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mon-a[31168]: 2024-09-11T23:29:59.999+0000 7f5d93536640 -1 log_channel(cluster) log [ERR] : mds.i(mds.1): Metadata damage detected 2024-09-11T23:30:00.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:30:00 smithi026 ceph-mon[31172]: pgmap v361: 97 pgs: 97 active+clean; 2.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 32 KiB/s wr, 11 op/s 2024-09-11T23:30:00.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:30:00 smithi026 ceph-mon[31172]: Health detail: HEALTH_ERR 1 MDSs report damaged metadata 2024-09-11T23:30:00.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:30:00 smithi026 ceph-mon[31172]: [ERR] MDS_DAMAGE: 1 MDSs report damaged metadata 2024-09-11T23:30:00.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:30:00 smithi026 ceph-mon[31172]: mds.i(mds.1): Metadata damage detected 2024-09-11T23:30:00.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:30:00 smithi076 ceph-mon[35296]: pgmap v361: 97 pgs: 97 active+clean; 2.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 32 KiB/s wr, 11 op/s 2024-09-11T23:30:00.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:30:00 smithi076 ceph-mon[35296]: Health detail: HEALTH_ERR 1 MDSs report damaged metadata 2024-09-11T23:30:00.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:30:00 smithi076 ceph-mon[35296]: [ERR] MDS_DAMAGE: 1 MDSs report damaged metadata 2024-09-11T23:30:00.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:30:00 smithi076 ceph-mon[35296]: mds.i(mds.1): Metadata damage detected 2024-09-11T23:30:00.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:30:00 smithi136 ceph-mon[35121]: pgmap v361: 97 pgs: 97 active+clean; 2.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 32 KiB/s wr, 11 op/s 2024-09-11T23:30:00.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:30:00 smithi136 ceph-mon[35121]: Health detail: HEALTH_ERR 1 MDSs report damaged metadata 2024-09-11T23:30:00.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:30:00 smithi136 ceph-mon[35121]: [ERR] MDS_DAMAGE: 1 MDSs report damaged metadata 2024-09-11T23:30:00.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:30:00 smithi136 ceph-mon[35121]: mds.i(mds.1): Metadata damage detected 2024-09-11T23:30:02.304 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:30:02 smithi076 ceph-mon[35296]: pgmap v362: 97 pgs: 97 active+clean; 2.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 34 KiB/s wr, 16 op/s 2024-09-11T23:30:02.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:30:02 smithi136 ceph-mon[35121]: pgmap v362: 97 pgs: 97 active+clean; 2.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 34 KiB/s wr, 16 op/s 2024-09-11T23:30:02.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:30:02 smithi026 ceph-mon[31172]: pgmap v362: 97 pgs: 97 active+clean; 2.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 34 KiB/s wr, 16 op/s 2024-09-11T23:30:03.266 INFO:tasks.workunit.client.0.smithi026.stdout: 2024-09-11T23:30:03.266 INFO:tasks.workunit.client.0.smithi026.stdout:Final score for writes: 613 2024-09-11T23:30:03.266 INFO:tasks.workunit.client.0.smithi026.stdout:Final score for reads : 616497 2024-09-11T23:30:03.266 INFO:tasks.workunit.client.0.smithi026.stdout: 2024-09-11T23:30:03.267 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-11T23:30:03.267 DEBUG:teuthology.orchestra.run.smithi026:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-11T23:30:04.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:30:04 smithi076 ceph-mon[35296]: pgmap v363: 97 pgs: 97 active+clean; 2.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 5.4 KiB/s wr, 10 op/s 2024-09-11T23:30:04.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:30:04 smithi136 ceph-mon[35121]: pgmap v363: 97 pgs: 97 active+clean; 2.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 5.4 KiB/s wr, 10 op/s 2024-09-11T23:30:04.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:30:04 smithi026 ceph-mon[31172]: pgmap v363: 97 pgs: 97 active+clean; 2.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 5.4 KiB/s wr, 10 op/s 2024-09-11T23:30:06.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:30:06 smithi076 ceph-mon[35296]: pgmap v364: 97 pgs: 97 active+clean; 2.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 122 KiB/s wr, 34 op/s 2024-09-11T23:30:06.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:30:06 smithi136 ceph-mon[35121]: pgmap v364: 97 pgs: 97 active+clean; 2.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 122 KiB/s wr, 34 op/s 2024-09-11T23:30:06.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:30:06 smithi026 ceph-mon[31172]: pgmap v364: 97 pgs: 97 active+clean; 2.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 122 KiB/s wr, 34 op/s 2024-09-11T23:30:08.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:30:08 smithi076 ceph-mon[35296]: pgmap v365: 97 pgs: 97 active+clean; 2.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 170 KiB/s wr, 34 op/s 2024-09-11T23:30:08.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:30:08 smithi136 ceph-mon[35121]: pgmap v365: 97 pgs: 97 active+clean; 2.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 170 KiB/s wr, 34 op/s 2024-09-11T23:30:08.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:30:08 smithi026 ceph-mon[31172]: pgmap v365: 97 pgs: 97 active+clean; 2.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 170 KiB/s wr, 34 op/s 2024-09-11T23:30:09.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:30:09 smithi076 ceph-mon[35296]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:30:09.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:30:09 smithi136 ceph-mon[35121]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:30:09.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:30:09 smithi026 ceph-mon[31172]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:30:10.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:30:10 smithi076 ceph-mon[35296]: pgmap v366: 97 pgs: 97 active+clean; 2.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 220 KiB/s wr, 39 op/s 2024-09-11T23:30:10.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:30:10 smithi076 ceph-mon[35296]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:30:10.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:30:10 smithi076 ceph-mon[35296]: osdmap e97: 12 total, 12 up, 12 in 2024-09-11T23:30:10.401 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:30:10 smithi026 ceph-mon[31172]: pgmap v366: 97 pgs: 97 active+clean; 2.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 220 KiB/s wr, 39 op/s 2024-09-11T23:30:10.402 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:30:10 smithi026 ceph-mon[31172]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:30:10.402 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:30:10 smithi026 ceph-mon[31172]: osdmap e97: 12 total, 12 up, 12 in 2024-09-11T23:30:10.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:30:10 smithi136 ceph-mon[35121]: pgmap v366: 97 pgs: 97 active+clean; 2.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 220 KiB/s wr, 39 op/s 2024-09-11T23:30:10.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:30:10 smithi136 ceph-mon[35121]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:30:10.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:30:10 smithi136 ceph-mon[35121]: osdmap e97: 12 total, 12 up, 12 in 2024-09-11T23:30:12.291 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.288+0000 7f6c8044c640 1 -- 172.21.15.26:0/505785239 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6c781009f0 msgr2=0x7f6c78100e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:30:12.291 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.288+0000 7f6c8044c640 1 --2- 172.21.15.26:0/505785239 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6c781009f0 0x7f6c78100e50 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f6c6c00d940 tx=0x7f6c6c02a1b0 comp rx=0 tx=0).stop 2024-09-11T23:30:12.291 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.288+0000 7f6c8044c640 1 -- 172.21.15.26:0/505785239 shutdown_connections 2024-09-11T23:30:12.291 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.288+0000 7f6c8044c640 1 --2- 172.21.15.26:0/505785239 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6c78101390 0x7f6c7810d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:12.291 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.288+0000 7f6c8044c640 1 --2- 172.21.15.26:0/505785239 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6c781009f0 0x7f6c78100e50 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:12.291 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.288+0000 7f6c8044c640 1 --2- 172.21.15.26:0/505785239 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6c78106a00 0x7f6c78106de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:12.291 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.288+0000 7f6c8044c640 1 -- 172.21.15.26:0/505785239 >> 172.21.15.26:0/505785239 conn(0x7f6c780fc6b0 msgr2=0x7f6c780fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:30:12.291 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.288+0000 7f6c8044c640 1 -- 172.21.15.26:0/505785239 shutdown_connections 2024-09-11T23:30:12.291 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.288+0000 7f6c8044c640 1 -- 172.21.15.26:0/505785239 wait complete. 2024-09-11T23:30:12.291 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.289+0000 7f6c8044c640 1 Processor -- start 2024-09-11T23:30:12.292 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.289+0000 7f6c8044c640 1 -- start start 2024-09-11T23:30:12.292 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.289+0000 7f6c8044c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6c781009f0 0x7f6c78196210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:12.292 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.289+0000 7f6c8044c640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6c78101390 0x7f6c78196750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:12.292 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.289+0000 7f6c8044c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6c78106a00 0x7f6c7819aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:12.292 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.289+0000 7f6c8044c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f6c7810fd80 con 0x7f6c781009f0 2024-09-11T23:30:12.292 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.289+0000 7f6c8044c640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f6c7810fc00 con 0x7f6c78106a00 2024-09-11T23:30:12.292 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.289+0000 7f6c8044c640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f6c7810ff00 con 0x7f6c78101390 2024-09-11T23:30:12.292 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.289+0000 7f6c7e1e8640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6c781009f0 0x7f6c78196210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:12.292 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.289+0000 7f6c7e1e8640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6c781009f0 0x7f6c78196210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59358/0 (socket says 172.21.15.26:59358) 2024-09-11T23:30:12.292 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.289+0000 7f6c7e1e8640 1 -- 172.21.15.26:0/2023906604 learned_addr learned my addr 172.21.15.26:0/2023906604 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:30:12.292 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.289+0000 7f6c7d9e7640 1 --2- 172.21.15.26:0/2023906604 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6c78101390 0x7f6c78196750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:12.293 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.289+0000 7f6c7e9e9640 1 --2- 172.21.15.26:0/2023906604 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6c78106a00 0x7f6c7819aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:12.293 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.290+0000 7f6c7e1e8640 1 -- 172.21.15.26:0/2023906604 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6c78101390 msgr2=0x7f6c78196750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:30:12.293 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.290+0000 7f6c7e1e8640 1 --2- 172.21.15.26:0/2023906604 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6c78101390 0x7f6c78196750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:12.293 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.290+0000 7f6c7e1e8640 1 -- 172.21.15.26:0/2023906604 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6c78106a00 msgr2=0x7f6c7819aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:30:12.293 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.290+0000 7f6c7e1e8640 1 --2- 172.21.15.26:0/2023906604 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6c78106a00 0x7f6c7819aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:12.293 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.290+0000 7f6c7e1e8640 1 -- 172.21.15.26:0/2023906604 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c7819b200 con 0x7f6c781009f0 2024-09-11T23:30:12.293 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.290+0000 7f6c7e1e8640 1 --2- 172.21.15.26:0/2023906604 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6c781009f0 0x7f6c78196210 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f6c680120f0 tx=0x7f6c680125b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:30:12.293 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.290+0000 7f6c777fe640 1 -- 172.21.15.26:0/2023906604 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c6801c030 con 0x7f6c781009f0 2024-09-11T23:30:12.293 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.290+0000 7f6c777fe640 1 -- 172.21.15.26:0/2023906604 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c68003e50 con 0x7f6c781009f0 2024-09-11T23:30:12.293 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.290+0000 7f6c8044c640 1 -- 172.21.15.26:0/2023906604 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c7819b4f0 con 0x7f6c781009f0 2024-09-11T23:30:12.293 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.290+0000 7f6c777fe640 1 -- 172.21.15.26:0/2023906604 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c68018660 con 0x7f6c781009f0 2024-09-11T23:30:12.293 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.290+0000 7f6c8044c640 1 -- 172.21.15.26:0/2023906604 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c781a2d80 con 0x7f6c781009f0 2024-09-11T23:30:12.299 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.293+0000 7f6c777fe640 1 -- 172.21.15.26:0/2023906604 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f6c6801f070 con 0x7f6c781009f0 2024-09-11T23:30:12.299 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.293+0000 7f6c831e6740 1 Processor -- start 2024-09-11T23:30:12.299 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.293+0000 7f6c831e6740 1 -- start start 2024-09-11T23:30:12.299 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.293+0000 7f6c831e6740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ce7457baf0 0x55ce7457bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:12.299 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.293+0000 7f6c831e6740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ce7457c410 0x55ce74586770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:12.299 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.293+0000 7f6c831e6740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ce74586cb0 0x55ce745890a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:12.299 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.293+0000 7f6c831e6740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55ce744e02f0 con 0x55ce74586cb0 2024-09-11T23:30:12.299 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.293+0000 7f6c831e6740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55ce74488270 con 0x55ce7457baf0 2024-09-11T23:30:12.299 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.293+0000 7f6c831e6740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55ce744dc2b0 con 0x55ce7457c410 2024-09-11T23:30:12.299 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.293+0000 7f6c7e1e8640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ce7457c410 0x55ce74586770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:12.299 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.293+0000 7f6c7e9e9640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ce7457baf0 0x55ce7457bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:12.299 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.293+0000 7f6c7d9e7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ce74586cb0 0x55ce745890a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:12.299 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.293+0000 7f6c7e9e9640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ce7457baf0 0x55ce7457bed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:37592/0 (socket says 172.21.15.26:37592) 2024-09-11T23:30:12.300 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.293+0000 7f6c7e1e8640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ce7457c410 0x55ce74586770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:49316/0 (socket says 172.21.15.26:49316) 2024-09-11T23:30:12.300 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.293+0000 7f6c7e9e9640 1 -- 172.21.15.26:0/3783340874 learned_addr learned my addr 172.21.15.26:0/3783340874 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:30:12.300 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.294+0000 7f6c777fe640 1 --2- 172.21.15.26:0/2023906604 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f6c600a2da0 0x7f6c600a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:12.300 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.294+0000 7f6c7e9e9640 1 -- 172.21.15.26:0/3783340874 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ce7457c410 msgr2=0x55ce74586770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:30:12.300 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.294+0000 7f6c7e9e9640 1 --2- 172.21.15.26:0/3783340874 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ce7457c410 0x55ce74586770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:12.300 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.294+0000 7f6c7e9e9640 1 -- 172.21.15.26:0/3783340874 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ce74586cb0 msgr2=0x55ce745890a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:30:12.300 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.294+0000 7f6c7e9e9640 1 --2- 172.21.15.26:0/3783340874 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ce74586cb0 0x55ce745890a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:12.300 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.294+0000 7f6c7e9e9640 1 -- 172.21.15.26:0/3783340874 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ce74487340 con 0x55ce7457baf0 2024-09-11T23:30:12.300 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.294+0000 7f6c7d9e7640 1 --2- 172.21.15.26:0/3783340874 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ce74586cb0 0x55ce745890a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:30:12.300 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.294+0000 7f6c777fe640 1 -- 172.21.15.26:0/2023906604 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8586+0+0 (secure 0 0 0) 0x7f6c680d2f30 con 0x7f6c781009f0 2024-09-11T23:30:12.300 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.294+0000 7f6c7e9e9640 1 --2- 172.21.15.26:0/2023906604 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f6c600a2da0 0x7f6c600a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:12.300 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.294+0000 7f6c7e9e9640 1 --2- 172.21.15.26:0/3783340874 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ce7457baf0 0x55ce7457bed0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f6c70009b60 tx=0x7f6c7000e5a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=65b61727147d56d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:30:12.300 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.294+0000 7f6c74bfe640 1 -- 172.21.15.26:0/3783340874 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c70023070 con 0x55ce7457baf0 2024-09-11T23:30:12.301 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.294+0000 7f6c74bfe640 1 -- 172.21.15.26:0/3783340874 <== mon.2 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c7001b870 con 0x55ce7457baf0 2024-09-11T23:30:12.301 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.294+0000 7f6c74bfe640 1 -- 172.21.15.26:0/3783340874 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c70013660 con 0x55ce7457baf0 2024-09-11T23:30:12.301 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.294+0000 7f6c7e9e9640 1 --2- 172.21.15.26:0/2023906604 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f6c600a2da0 0x7f6c600a5260 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f6c70020000 tx=0x7f6c7001d000 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:30:12.301 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.295+0000 7f6c831e6740 1 -- 172.21.15.26:0/3783340874 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ce7457baf0 msgr2=0x55ce7457bed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:30:12.301 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.295+0000 7f6c831e6740 1 --2- 172.21.15.26:0/3783340874 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ce7457baf0 0x55ce7457bed0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f6c70009b60 tx=0x7f6c7000e5a0 comp rx=0 tx=0).stop 2024-09-11T23:30:12.301 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.295+0000 7f6c831e6740 1 -- 172.21.15.26:0/3783340874 shutdown_connections 2024-09-11T23:30:12.301 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.295+0000 7f6c831e6740 1 --2- 172.21.15.26:0/3783340874 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ce74586cb0 0x55ce745890a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:12.301 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.295+0000 7f6c831e6740 1 --2- 172.21.15.26:0/3783340874 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ce7457c410 0x55ce74586770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:12.301 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.295+0000 7f6c831e6740 1 --2- 172.21.15.26:0/3783340874 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ce7457baf0 0x55ce7457bed0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:12.301 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.295+0000 7f6c831e6740 1 -- 172.21.15.26:0/3783340874 >> 172.21.15.26:0/3783340874 conn(0x55ce74578060 msgr2=0x55ce74578440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:30:12.301 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.295+0000 7f6c831e6740 1 -- 172.21.15.26:0/3783340874 shutdown_connections 2024-09-11T23:30:12.301 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.295+0000 7f6c831e6740 1 -- 172.21.15.26:0/3783340874 wait complete. 2024-09-11T23:30:12.301 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.295+0000 7f6c831e6740 1 -- start start 2024-09-11T23:30:12.302 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.296+0000 7f6c831e6740 1 Processor -- start 2024-09-11T23:30:12.302 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.296+0000 7f6c831e6740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ce7457baf0 0x55ce74583b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:12.302 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.296+0000 7f6c831e6740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ce7457c410 0x55ce745840c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:12.302 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.296+0000 7f6c831e6740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ce74586cb0 0x55ce74584600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:12.302 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.296+0000 7f6c831e6740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55ce74589880 con 0x55ce74586cb0 2024-09-11T23:30:12.302 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.296+0000 7f6c831e6740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55ce74469290 con 0x55ce7457baf0 2024-09-11T23:30:12.302 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.296+0000 7f6c831e6740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55ce74589a00 con 0x55ce7457c410 2024-09-11T23:30:12.302 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.296+0000 7f6c831e6740 10 client.0 _ll_register_callbacks cb 0x55ce7449a140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:30:12.302 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.296+0000 7f6c7e1e8640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ce7457baf0 0x55ce74583b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:12.302 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.296+0000 7f6c7e9e9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ce74586cb0 0x55ce74584600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:12.302 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.296+0000 7f6c7e9e9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ce74586cb0 0x55ce74584600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59386/0 (socket says 172.21.15.26:59386) 2024-09-11T23:30:12.302 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.296+0000 7f6c7e1e8640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ce7457baf0 0x55ce74583b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:37606/0 (socket says 172.21.15.26:37606) 2024-09-11T23:30:12.302 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.296+0000 7f6c7d9e7640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ce7457c410 0x55ce745840c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:12.302 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.296+0000 7f6c7e9e9640 1 -- 172.21.15.26:0/3694185709 learned_addr learned my addr 172.21.15.26:0/3694185709 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:30:12.303 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.296+0000 7f6c7e9e9640 1 -- 172.21.15.26:0/3694185709 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ce7457c410 msgr2=0x55ce745840c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:30:12.303 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.296+0000 7f6c7e9e9640 1 --2- 172.21.15.26:0/3694185709 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ce7457c410 0x55ce745840c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:12.303 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.296+0000 7f6c7e9e9640 1 -- 172.21.15.26:0/3694185709 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ce7457baf0 msgr2=0x55ce74583b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:30:12.303 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.296+0000 7f6c7e9e9640 1 --2- 172.21.15.26:0/3694185709 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ce7457baf0 0x55ce74583b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:12.303 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.296+0000 7f6c7e9e9640 1 -- 172.21.15.26:0/3694185709 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ce741f3210 con 0x55ce74586cb0 2024-09-11T23:30:12.303 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.296+0000 7f6c7e1e8640 1 --2- 172.21.15.26:0/3694185709 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ce7457baf0 0x55ce74583b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:30:12.303 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.297+0000 7f6c7e9e9640 1 --2- 172.21.15.26:0/3694185709 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ce74586cb0 0x55ce74584600 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f6c7000eaa0 tx=0x7f6c700095a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fa747f2002c668b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:30:12.303 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.297+0000 7f6c5dffb640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:30:12.303 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.297+0000 7f6c5dffb640 1 -- 172.21.15.26:0/3694185709 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c7002b070 con 0x55ce74586cb0 2024-09-11T23:30:12.303 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.297+0000 7f6c831e6740 1 -- 172.21.15.26:0/3694185709 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c7001b870 con 0x55ce74586cb0 2024-09-11T23:30:12.303 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.297+0000 7f6c5dffb640 1 -- 172.21.15.26:0/3694185709 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8586+0+0 (secure 0 0 0) 0x7f6c700139f0 con 0x55ce74586cb0 2024-09-11T23:30:12.303 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.297+0000 7f6c5dffb640 1 -- 172.21.15.26:0/3694185709 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7f6c70037050 con 0x55ce74586cb0 2024-09-11T23:30:12.304 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.297+0000 7f6c831e6740 10 client.17187 fetch_fsmap learned FSMap version 59 2024-09-11T23:30:12.304 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.297+0000 7f6c831e6740 1 -- 172.21.15.26:0/3694185709 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x55ce742bf620 con 0x55ce74586cb0 2024-09-11T23:30:12.304 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.297+0000 7f6c5dffb640 1 -- 172.21.15.26:0/3694185709 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c7001b870 con 0x55ce74586cb0 2024-09-11T23:30:12.304 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.297+0000 7f6c5dffb640 1 -- 172.21.15.26:0/3694185709 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 59) ==== 8378+0+0 (secure 0 0 0) 0x7f6c7001b070 con 0x55ce74586cb0 2024-09-11T23:30:12.304 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.297+0000 7f6c831e6740 10 client.17187 fetch_fsmap finished waiting for FSMap version 59 2024-09-11T23:30:12.304 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.298+0000 7f6c831e6740 10 client.17187 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:30:12.304 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.298+0000 7f6c831e6740 20 client.17187 populate_metadata read hostname 'smithi026' 2024-09-11T23:30:12.304 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.298+0000 7f6c831e6740 1 --2- 172.21.15.26:0/3694185709 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55ce74596b90 0x55ce74598f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:12.304 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.298+0000 7f6c831e6740 4 client.17187 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:30:12.304 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.298+0000 7f6c831e6740 1 -- 172.21.15.26:0/3694185709 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ce744eda20 con 0x55ce74596b90 2024-09-11T23:30:12.305 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.298+0000 7f6c7e1e8640 1 --2- 172.21.15.26:0/3694185709 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55ce74596b90 0x55ce74598f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:12.305 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.298+0000 7f6c5dffb640 10 client.17187 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:30:12.305 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.298+0000 7f6c7e1e8640 1 --2- 172.21.15.26:0/3694185709 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55ce74596b90 0x55ce74598f80 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=78711fb833f0589f server_cookie=a1dbeb825319537 in_seq=0 out_seq=0 2024-09-11T23:30:12.305 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:30:12 smithi076 ceph-mon[35296]: pgmap v368: 97 pgs: 14 active+clean+snaptrim_wait, 8 active+clean+snaptrim, 75 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 850 KiB/s rd, 884 KiB/s wr, 185 op/s 2024-09-11T23:30:12.306 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.299+0000 7f6c5dffb640 1 -- 172.21.15.26:0/3694185709 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ce744eda20 con 0x55ce74596b90 2024-09-11T23:30:12.307 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.299+0000 7f6c5dffb640 10 client.17187 handle_command_reply: tid=0 2024-09-11T23:30:12.307 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.299+0000 7f6c831e6740 1 client.17187 shutdown 2024-09-11T23:30:12.307 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.299+0000 7f6c831e6740 1 -- 172.21.15.26:0/3694185709 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ce74586cb0 msgr2=0x55ce74584600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:30:12.307 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.299+0000 7f6c831e6740 1 --2- 172.21.15.26:0/3694185709 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ce74586cb0 0x55ce74584600 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f6c7000eaa0 tx=0x7f6c700095a0 comp rx=0 tx=0).stop 2024-09-11T23:30:12.307 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.299+0000 7f6c831e6740 1 -- 172.21.15.26:0/3694185709 shutdown_connections 2024-09-11T23:30:12.307 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.299+0000 7f6c831e6740 1 --2- 172.21.15.26:0/3694185709 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55ce74596b90 0x55ce74598f80 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:12.307 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.299+0000 7f6c831e6740 1 --2- 172.21.15.26:0/3694185709 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ce74586cb0 0x55ce74584600 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:12.307 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.299+0000 7f6c831e6740 1 --2- 172.21.15.26:0/3694185709 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ce7457c410 0x55ce745840c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:12.307 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.299+0000 7f6c5dffb640 0 client.17187 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:30:12.307 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.299+0000 7f6c831e6740 1 --2- 172.21.15.26:0/3694185709 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ce7457baf0 0x55ce74583b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:12.307 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.299+0000 7f6c831e6740 1 -- 172.21.15.26:0/3694185709 >> 172.21.15.26:0/3694185709 conn(0x55ce74578060 msgr2=0x55ce7457dd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:30:12.307 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.300+0000 7f6c831e6740 1 -- 172.21.15.26:0/3694185709 shutdown_connections 2024-09-11T23:30:12.307 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.300+0000 7f6c831e6740 1 -- 172.21.15.26:0/3694185709 wait complete. 2024-09-11T23:30:12.308 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.300+0000 7f6c831e6740 20 client.17187 trim_cache size 0 max 16384 2024-09-11T23:30:12.317 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.315+0000 7f6c831e6740 1 Processor -- start 2024-09-11T23:30:12.317 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.315+0000 7f6c831e6740 1 -- start start 2024-09-11T23:30:12.317 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.315+0000 7f6c831e6740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ce7457baf0 0x55ce7457e560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:12.317 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.315+0000 7f6c831e6740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ce7457c410 0x55ce74584430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:12.317 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.315+0000 7f6c831e6740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ce74586cb0 0x55ce7459e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:12.317 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.315+0000 7f6c831e6740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f6c781ab880 con 0x55ce7457c410 2024-09-11T23:30:12.317 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.315+0000 7f6c831e6740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f6c781ad450 con 0x55ce74586cb0 2024-09-11T23:30:12.317 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.315+0000 7f6c831e6740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f6c781abf80 con 0x55ce7457baf0 2024-09-11T23:30:12.317 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.315+0000 7f6c7d9e7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ce7457c410 0x55ce74584430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:12.317 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.315+0000 7f6c7d9e7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ce7457c410 0x55ce74584430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59390/0 (socket says 172.21.15.26:59390) 2024-09-11T23:30:12.317 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.315+0000 7f6c7d9e7640 1 -- 172.21.15.26:0/1782343555 learned_addr learned my addr 172.21.15.26:0/1782343555 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:30:12.318 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.315+0000 7f6c7e9e9640 1 --2- 172.21.15.26:0/1782343555 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ce74586cb0 0x55ce7459e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:12.318 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.315+0000 7f6c7e1e8640 1 --2- 172.21.15.26:0/1782343555 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ce7457baf0 0x55ce7457e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:12.318 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.315+0000 7f6c7d9e7640 1 -- 172.21.15.26:0/1782343555 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ce7457baf0 msgr2=0x55ce7457e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:30:12.318 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.315+0000 7f6c7d9e7640 1 --2- 172.21.15.26:0/1782343555 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ce7457baf0 0x55ce7457e560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:12.318 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.315+0000 7f6c7d9e7640 1 -- 172.21.15.26:0/1782343555 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ce74586cb0 msgr2=0x55ce7459e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:30:12.318 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.315+0000 7f6c7d9e7640 1 --2- 172.21.15.26:0/1782343555 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ce74586cb0 0x55ce7459e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:12.318 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.315+0000 7f6c7d9e7640 1 -- 172.21.15.26:0/1782343555 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ce745899f0 con 0x55ce7457c410 2024-09-11T23:30:12.318 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.316+0000 7f6c7d9e7640 1 --2- 172.21.15.26:0/1782343555 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ce7457c410 0x55ce74584430 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f6c6c00d910 tx=0x7f6c6c02ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c33bc5c60598fe2 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:30:12.318 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.316+0000 7f6c5ffff640 1 -- 172.21.15.26:0/1782343555 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c6c00d110 con 0x55ce7457c410 2024-09-11T23:30:12.318 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.316+0000 7f6c5ffff640 1 -- 172.21.15.26:0/1782343555 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c6c02e040 con 0x55ce7457c410 2024-09-11T23:30:12.318 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.316+0000 7f6c5ffff640 1 -- 172.21.15.26:0/1782343555 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c6c0046c0 con 0x55ce7457c410 2024-09-11T23:30:12.322 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.316+0000 7f6c831e6740 1 -- 172.21.15.26:0/1782343555 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ce7457c410 msgr2=0x55ce74584430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:30:12.322 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.316+0000 7f6c831e6740 1 --2- 172.21.15.26:0/1782343555 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ce7457c410 0x55ce74584430 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f6c6c00d910 tx=0x7f6c6c02ad30 comp rx=0 tx=0).stop 2024-09-11T23:30:12.322 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.316+0000 7f6c831e6740 1 -- 172.21.15.26:0/1782343555 shutdown_connections 2024-09-11T23:30:12.322 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.316+0000 7f6c831e6740 1 --2- 172.21.15.26:0/1782343555 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ce74586cb0 0x55ce7459e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:12.322 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.316+0000 7f6c831e6740 1 --2- 172.21.15.26:0/1782343555 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ce7457c410 0x55ce74584430 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:12.322 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.316+0000 7f6c831e6740 1 --2- 172.21.15.26:0/1782343555 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ce7457baf0 0x55ce7457e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:12.322 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.316+0000 7f6c831e6740 1 -- 172.21.15.26:0/1782343555 >> 172.21.15.26:0/1782343555 conn(0x55ce74578060 msgr2=0x55ce74585c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:30:12.322 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.317+0000 7f6c831e6740 1 -- 172.21.15.26:0/1782343555 shutdown_connections 2024-09-11T23:30:12.322 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.317+0000 7f6c831e6740 1 -- 172.21.15.26:0/1782343555 wait complete. 2024-09-11T23:30:12.323 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.317+0000 7f6c831e6740 1 -- start start 2024-09-11T23:30:12.323 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.317+0000 7f6c831e6740 1 Processor -- start 2024-09-11T23:30:12.323 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.317+0000 7f6c831e6740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ce7457baf0 0x55ce7458f010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:12.323 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.317+0000 7f6c831e6740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ce7457c410 0x55ce7458f550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:12.323 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.317+0000 7f6c831e6740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ce74586cb0 0x55ce745806e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:12.323 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.317+0000 7f6c831e6740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f6c781acd50 con 0x55ce74586cb0 2024-09-11T23:30:12.323 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.317+0000 7f6c831e6740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f6c781ac650 con 0x55ce7457c410 2024-09-11T23:30:12.323 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.317+0000 7f6c831e6740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f6c781a9880 con 0x55ce7457baf0 2024-09-11T23:30:12.323 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.317+0000 7f6c831e6740 10 client.0 _ll_register_callbacks cb 0x7f6c70021070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:30:12.323 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.317+0000 7f6c7e9e9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ce74586cb0 0x55ce745806e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:12.323 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.317+0000 7f6c7d9e7640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ce7457c410 0x55ce7458f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:12.323 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.317+0000 7f6c7e9e9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ce74586cb0 0x55ce745806e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59394/0 (socket says 172.21.15.26:59394) 2024-09-11T23:30:12.323 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.317+0000 7f6c7e9e9640 1 -- 172.21.15.26:0/773071168 learned_addr learned my addr 172.21.15.26:0/773071168 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:30:12.324 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.317+0000 7f6c7e1e8640 1 --2- 172.21.15.26:0/773071168 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ce7457baf0 0x55ce7458f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:12.324 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.318+0000 7f6c7e9e9640 1 -- 172.21.15.26:0/773071168 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ce7457baf0 msgr2=0x55ce7458f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:30:12.324 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.318+0000 7f6c7e9e9640 1 --2- 172.21.15.26:0/773071168 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ce7457baf0 0x55ce7458f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:12.324 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.318+0000 7f6c7e9e9640 1 -- 172.21.15.26:0/773071168 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ce7457c410 msgr2=0x55ce7458f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:30:12.324 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.318+0000 7f6c7e9e9640 1 --2- 172.21.15.26:0/773071168 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ce7457c410 0x55ce7458f550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:12.324 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.318+0000 7f6c7e9e9640 1 -- 172.21.15.26:0/773071168 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ce745732d0 con 0x55ce74586cb0 2024-09-11T23:30:12.324 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.318+0000 7f6c7d9e7640 1 --2- 172.21.15.26:0/773071168 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ce7457c410 0x55ce7458f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:30:12.324 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.318+0000 7f6c7e9e9640 1 --2- 172.21.15.26:0/773071168 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ce74586cb0 0x55ce745806e0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f6c7001b360 tx=0x7f6c70013370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7e3c4fc35a7f6e6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:30:12.324 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.318+0000 7f6c5dffb640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:30:12.324 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.318+0000 7f6c5dffb640 1 -- 172.21.15.26:0/773071168 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c7002b040 con 0x55ce74586cb0 2024-09-11T23:30:12.324 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.318+0000 7f6c5dffb640 1 -- 172.21.15.26:0/773071168 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8586+0+0 (secure 0 0 0) 0x7f6c7001be30 con 0x55ce74586cb0 2024-09-11T23:30:12.324 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.318+0000 7f6c831e6740 1 -- 172.21.15.26:0/773071168 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c6c02e040 con 0x55ce74586cb0 2024-09-11T23:30:12.324 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.318+0000 7f6c5dffb640 1 -- 172.21.15.26:0/773071168 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7f6c70049050 con 0x55ce74586cb0 2024-09-11T23:30:12.325 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.318+0000 7f6c831e6740 10 client.17193 fetch_fsmap learned FSMap version 59 2024-09-11T23:30:12.325 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.318+0000 7f6c831e6740 1 -- 172.21.15.26:0/773071168 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x55ce74578cf0 con 0x55ce74586cb0 2024-09-11T23:30:12.325 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.318+0000 7f6c5dffb640 1 -- 172.21.15.26:0/773071168 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c6c02e040 con 0x55ce74586cb0 2024-09-11T23:30:12.325 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.318+0000 7f6c5dffb640 1 -- 172.21.15.26:0/773071168 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 59) ==== 8378+0+0 (secure 0 0 0) 0x7f6c70013f50 con 0x55ce74586cb0 2024-09-11T23:30:12.325 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.318+0000 7f6c831e6740 10 client.17193 fetch_fsmap finished waiting for FSMap version 59 2024-09-11T23:30:12.325 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.318+0000 7f6c831e6740 10 client.17193 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:30:12.325 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.318+0000 7f6c831e6740 20 client.17193 populate_metadata read hostname 'smithi026' 2024-09-11T23:30:12.325 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.318+0000 7f6c831e6740 1 --2- 172.21.15.26:0/773071168 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55ce74601560 0x55ce74603950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:12.325 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.319+0000 7f6c831e6740 4 client.17193 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T23:30:12.325 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.319+0000 7f6c831e6740 1 -- 172.21.15.26:0/773071168 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ce744e7d70 con 0x55ce74601560 2024-09-11T23:30:12.325 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.319+0000 7f6c7e1e8640 1 --2- 172.21.15.26:0/773071168 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55ce74601560 0x55ce74603950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:12.325 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.319+0000 7f6c5dffb640 10 client.17193 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:30:12.326 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.319+0000 7f6c7e1e8640 1 --2- 172.21.15.26:0/773071168 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55ce74601560 0x55ce74603950 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc0777dacc81182 server_cookie=95f7980b8c623b32 in_seq=0 out_seq=0 2024-09-11T23:30:12.340 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.339+0000 7f6c5dffb640 1 -- 172.21.15.26:0/773071168 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ce744e7d70 con 0x55ce74601560 2024-09-11T23:30:12.341 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.339+0000 7f6c5dffb640 10 client.17193 handle_command_reply: tid=0 2024-09-11T23:30:12.341 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.339+0000 7f6c831e6740 1 client.17193 shutdown 2024-09-11T23:30:12.341 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.339+0000 7f6c831e6740 1 -- 172.21.15.26:0/773071168 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ce74586cb0 msgr2=0x55ce745806e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:30:12.341 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.339+0000 7f6c831e6740 1 --2- 172.21.15.26:0/773071168 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ce74586cb0 0x55ce745806e0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f6c7001b360 tx=0x7f6c70013370 comp rx=0 tx=0).stop 2024-09-11T23:30:12.341 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.339+0000 7f6c831e6740 1 -- 172.21.15.26:0/773071168 shutdown_connections 2024-09-11T23:30:12.341 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.339+0000 7f6c831e6740 1 --2- 172.21.15.26:0/773071168 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ce7457c410 0x55ce7458f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:12.341 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.339+0000 7f6c831e6740 1 --2- 172.21.15.26:0/773071168 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55ce74601560 0x55ce74603950 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:12.341 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.339+0000 7f6c831e6740 1 --2- 172.21.15.26:0/773071168 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ce74586cb0 0x55ce745806e0 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:12.341 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.339+0000 7f6c831e6740 1 --2- 172.21.15.26:0/773071168 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ce7457baf0 0x55ce7458f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:12.341 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.340+0000 7f6c831e6740 1 -- 172.21.15.26:0/773071168 >> 172.21.15.26:0/773071168 conn(0x55ce74578060 msgr2=0x55ce74585c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:30:12.341 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.340+0000 7f6c5dffb640 0 client.17193 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:30:12.341 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.340+0000 7f6c831e6740 1 -- 172.21.15.26:0/773071168 shutdown_connections 2024-09-11T23:30:12.341 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.340+0000 7f6c831e6740 1 -- 172.21.15.26:0/773071168 wait complete. 2024-09-11T23:30:12.342 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.340+0000 7f6c831e6740 20 client.17193 trim_cache size 0 max 16384 2024-09-11T23:30:12.342 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-09-11T23:30:12.342 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "message": "", 2024-09-11T23:30:12.342 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-09-11T23:30:12.342 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-09-11T23:30:12.343 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.340+0000 7f6c8044c640 1 -- 172.21.15.26:0/2023906604 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f6c600a2da0 msgr2=0x7f6c600a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:30:12.343 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.340+0000 7f6c8044c640 1 --2- 172.21.15.26:0/2023906604 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f6c600a2da0 0x7f6c600a5260 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f6c70020000 tx=0x7f6c7001d000 comp rx=0 tx=0).stop 2024-09-11T23:30:12.343 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.340+0000 7f6c8044c640 1 -- 172.21.15.26:0/2023906604 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6c781009f0 msgr2=0x7f6c78196210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:30:12.343 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.340+0000 7f6c8044c640 1 --2- 172.21.15.26:0/2023906604 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6c781009f0 0x7f6c78196210 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f6c680120f0 tx=0x7f6c680125b0 comp rx=0 tx=0).stop 2024-09-11T23:30:12.343 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.340+0000 7f6c8044c640 1 -- 172.21.15.26:0/2023906604 shutdown_connections 2024-09-11T23:30:12.343 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.340+0000 7f6c8044c640 1 --2- 172.21.15.26:0/2023906604 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f6c600a2da0 0x7f6c600a5260 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:12.343 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.340+0000 7f6c8044c640 1 --2- 172.21.15.26:0/2023906604 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6c78106a00 0x7f6c7819aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:12.343 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.340+0000 7f6c8044c640 1 --2- 172.21.15.26:0/2023906604 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6c78101390 0x7f6c78196750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:12.344 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.340+0000 7f6c8044c640 1 --2- 172.21.15.26:0/2023906604 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6c781009f0 0x7f6c78196210 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:12.344 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.340+0000 7f6c8044c640 1 -- 172.21.15.26:0/2023906604 >> 172.21.15.26:0/2023906604 conn(0x7f6c780fc6b0 msgr2=0x7f6c780fccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:30:12.344 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.340+0000 7f6c8044c640 1 -- 172.21.15.26:0/2023906604 shutdown_connections 2024-09-11T23:30:12.344 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:12.340+0000 7f6c8044c640 1 -- 172.21.15.26:0/2023906604 wait complete. 2024-09-11T23:30:12.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:30:12 smithi136 ceph-mon[35121]: pgmap v368: 97 pgs: 14 active+clean+snaptrim_wait, 8 active+clean+snaptrim, 75 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 850 KiB/s rd, 884 KiB/s wr, 185 op/s 2024-09-11T23:30:12.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:30:12 smithi026 ceph-mon[31172]: pgmap v368: 97 pgs: 14 active+clean+snaptrim_wait, 8 active+clean+snaptrim, 75 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 850 KiB/s rd, 884 KiB/s wr, 185 op/s 2024-09-11T23:30:14.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:30:14 smithi076 ceph-mon[35296]: pgmap v369: 97 pgs: 19 active+clean+snaptrim_wait, 10 active+clean+snaptrim, 68 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 214 op/s 2024-09-11T23:30:14.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:30:14 smithi076 ceph-mon[35296]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:30:14.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:30:14 smithi136 ceph-mon[35121]: pgmap v369: 97 pgs: 19 active+clean+snaptrim_wait, 10 active+clean+snaptrim, 68 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 214 op/s 2024-09-11T23:30:14.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:30:14 smithi136 ceph-mon[35121]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:30:14.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:30:14 smithi026 ceph-mon[31172]: pgmap v369: 97 pgs: 19 active+clean+snaptrim_wait, 10 active+clean+snaptrim, 68 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 214 op/s 2024-09-11T23:30:14.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:30:14 smithi026 ceph-mon[31172]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:30:15.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:30:15 smithi136 ceph-mon[35121]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:30:15.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:30:15 smithi026 ceph-mon[31172]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:30:15.552 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:30:15 smithi076 ceph-mon[35296]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:30:16.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:30:16 smithi136 ceph-mon[35121]: pgmap v370: 97 pgs: 19 active+clean+snaptrim_wait, 10 active+clean+snaptrim, 68 active+clean; 2.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 305 op/s 2024-09-11T23:30:16.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:30:16 smithi026 ceph-mon[31172]: pgmap v370: 97 pgs: 19 active+clean+snaptrim_wait, 10 active+clean+snaptrim, 68 active+clean; 2.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 305 op/s 2024-09-11T23:30:16.552 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:30:16 smithi076 ceph-mon[35296]: pgmap v370: 97 pgs: 19 active+clean+snaptrim_wait, 10 active+clean+snaptrim, 68 active+clean; 2.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 305 op/s 2024-09-11T23:30:18.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:30:18 smithi136 ceph-mon[35121]: pgmap v371: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.6 MiB/s wr, 499 op/s 2024-09-11T23:30:18.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:30:18 smithi136 ceph-mon[35121]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:30:18.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:30:18 smithi026 ceph-mon[31172]: pgmap v371: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.6 MiB/s wr, 499 op/s 2024-09-11T23:30:18.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:30:18 smithi026 ceph-mon[31172]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:30:18.552 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:30:18 smithi076 ceph-mon[35296]: pgmap v371: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.6 MiB/s wr, 499 op/s 2024-09-11T23:30:18.552 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:30:18 smithi076 ceph-mon[35296]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:30:19.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:30:19 smithi136 ceph-mon[35121]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:30:19.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:30:19 smithi026 ceph-mon[31172]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:30:19.552 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:30:19 smithi076 ceph-mon[35296]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:30:20.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:30:20 smithi136 ceph-mon[35121]: pgmap v372: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.5 MiB/s wr, 491 op/s 2024-09-11T23:30:20.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:30:20 smithi136 ceph-mon[35121]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:30:20.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:30:20 smithi026 ceph-mon[31172]: pgmap v372: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.5 MiB/s wr, 491 op/s 2024-09-11T23:30:20.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:30:20 smithi026 ceph-mon[31172]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:30:20.552 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:30:20 smithi076 ceph-mon[35296]: pgmap v372: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.5 MiB/s wr, 491 op/s 2024-09-11T23:30:20.552 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:30:20 smithi076 ceph-mon[35296]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:30:22.000 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:30:21 smithi026 ceph-mon[31172]: pgmap v373: 97 pgs: 97 active+clean; 2.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 4.0 MiB/s wr, 888 op/s 2024-09-11T23:30:22.000 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:30:21 smithi026 ceph-mon[31172]: osdmap e98: 12 total, 12 up, 12 in 2024-09-11T23:30:22.052 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:30:21 smithi076 ceph-mon[35296]: pgmap v373: 97 pgs: 97 active+clean; 2.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 4.0 MiB/s wr, 888 op/s 2024-09-11T23:30:22.052 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:30:21 smithi076 ceph-mon[35296]: osdmap e98: 12 total, 12 up, 12 in 2024-09-11T23:30:22.238 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:30:21 smithi136 ceph-mon[35121]: pgmap v373: 97 pgs: 97 active+clean; 2.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 4.0 MiB/s wr, 888 op/s 2024-09-11T23:30:22.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:30:21 smithi136 ceph-mon[35121]: osdmap e98: 12 total, 12 up, 12 in 2024-09-11T23:30:24.052 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:30:23 smithi076 ceph-mon[35296]: pgmap v375: 97 pgs: 97 active+clean; 2.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.5 MiB/s wr, 949 op/s 2024-09-11T23:30:24.095 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:30:23 smithi136 ceph-mon[35121]: pgmap v375: 97 pgs: 97 active+clean; 2.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.5 MiB/s wr, 949 op/s 2024-09-11T23:30:24.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:30:23 smithi026 ceph-mon[31172]: pgmap v375: 97 pgs: 97 active+clean; 2.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.5 MiB/s wr, 949 op/s 2024-09-11T23:30:25.052 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:30:24 smithi076 ceph-mon[35296]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:30:25.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:30:24 smithi136 ceph-mon[35121]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:30:25.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:30:24 smithi026 ceph-mon[31172]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:30:25.442 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.441+0000 7fd6d3627640 1 -- 172.21.15.26:0/1507017444 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd6cc1016d0 msgr2=0x7fd6cc101b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:30:25.442 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.441+0000 7fd6d3627640 1 --2- 172.21.15.26:0/1507017444 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd6cc1016d0 0x7fd6cc101b10 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fd6bc009900 tx=0x7fd6bc02a480 comp rx=0 tx=0).stop 2024-09-11T23:30:25.442 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.441+0000 7fd6d3627640 1 -- 172.21.15.26:0/1507017444 shutdown_connections 2024-09-11T23:30:25.442 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.441+0000 7fd6d3627640 1 --2- 172.21.15.26:0/1507017444 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd6cc102050 0x7fd6cc10d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:25.443 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.441+0000 7fd6d3627640 1 --2- 172.21.15.26:0/1507017444 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd6cc1016d0 0x7fd6cc101b10 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:25.443 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.441+0000 7fd6d3627640 1 --2- 172.21.15.26:0/1507017444 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd6cc105af0 0x7fd6cc105ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:25.443 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.441+0000 7fd6d3627640 1 -- 172.21.15.26:0/1507017444 >> 172.21.15.26:0/1507017444 conn(0x7fd6cc075ec0 msgr2=0x7fd6cc0762d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:30:25.443 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.441+0000 7fd6d3627640 1 -- 172.21.15.26:0/1507017444 shutdown_connections 2024-09-11T23:30:25.443 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.441+0000 7fd6d3627640 1 -- 172.21.15.26:0/1507017444 wait complete. 2024-09-11T23:30:25.443 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.442+0000 7fd6d3627640 1 Processor -- start 2024-09-11T23:30:25.443 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.442+0000 7fd6d3627640 1 -- start start 2024-09-11T23:30:25.445 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.442+0000 7fd6d3627640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd6cc1016d0 0x7fd6cc199080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:25.445 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.442+0000 7fd6d3627640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd6cc102050 0x7fd6cc1995c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:25.445 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.442+0000 7fd6d3627640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd6cc105af0 0x7fd6cc1a4100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:25.445 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.442+0000 7fd6d3627640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd6cc110e30 con 0x7fd6cc102050 2024-09-11T23:30:25.445 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.442+0000 7fd6d3627640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fd6cc110cb0 con 0x7fd6cc105af0 2024-09-11T23:30:25.445 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.442+0000 7fd6d3627640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd6cc110fb0 con 0x7fd6cc1016d0 2024-09-11T23:30:25.445 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.442+0000 7fd6d0bc2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd6cc102050 0x7fd6cc1995c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:25.445 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.442+0000 7fd6d0bc2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd6cc102050 0x7fd6cc1995c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41988/0 (socket says 172.21.15.26:41988) 2024-09-11T23:30:25.446 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.442+0000 7fd6d0bc2640 1 -- 172.21.15.26:0/2277061702 learned_addr learned my addr 172.21.15.26:0/2277061702 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:30:25.446 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.442+0000 7fd6d13c3640 1 --2- 172.21.15.26:0/2277061702 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd6cc1016d0 0x7fd6cc199080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:25.446 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.442+0000 7fd6d1bc4640 1 --2- 172.21.15.26:0/2277061702 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd6cc105af0 0x7fd6cc1a4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:25.446 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.443+0000 7fd6d0bc2640 1 -- 172.21.15.26:0/2277061702 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd6cc1016d0 msgr2=0x7fd6cc199080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:30:25.446 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.443+0000 7fd6d0bc2640 1 --2- 172.21.15.26:0/2277061702 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd6cc1016d0 0x7fd6cc199080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:25.446 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.443+0000 7fd6d0bc2640 1 -- 172.21.15.26:0/2277061702 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd6cc105af0 msgr2=0x7fd6cc1a4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:30:25.446 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.443+0000 7fd6d0bc2640 1 --2- 172.21.15.26:0/2277061702 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd6cc105af0 0x7fd6cc1a4100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:25.446 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.443+0000 7fd6d0bc2640 1 -- 172.21.15.26:0/2277061702 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6cc1a4640 con 0x7fd6cc102050 2024-09-11T23:30:25.446 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.443+0000 7fd6d0bc2640 1 --2- 172.21.15.26:0/2277061702 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd6cc102050 0x7fd6cc1995c0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fd6bc004770 tx=0x7fd6bc003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:30:25.446 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.443+0000 7fd6c27fc640 1 -- 172.21.15.26:0/2277061702 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6bc03f070 con 0x7fd6cc102050 2024-09-11T23:30:25.446 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.443+0000 7fd6c27fc640 1 -- 172.21.15.26:0/2277061702 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd6bc031ca0 con 0x7fd6cc102050 2024-09-11T23:30:25.446 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.443+0000 7fd6d3627640 1 -- 172.21.15.26:0/2277061702 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6cc1a4810 con 0x7fd6cc102050 2024-09-11T23:30:25.446 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.443+0000 7fd6c27fc640 1 -- 172.21.15.26:0/2277061702 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6bc035030 con 0x7fd6cc102050 2024-09-11T23:30:25.447 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.443+0000 7fd6d3627640 1 -- 172.21.15.26:0/2277061702 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6cc1a4d00 con 0x7fd6cc102050 2024-09-11T23:30:25.453 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.446+0000 7fd6c27fc640 1 -- 172.21.15.26:0/2277061702 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fd6bc042070 con 0x7fd6cc102050 2024-09-11T23:30:25.453 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.446+0000 7fd6d63c5740 1 Processor -- start 2024-09-11T23:30:25.454 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.446+0000 7fd6d63c5740 1 -- start start 2024-09-11T23:30:25.454 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.446+0000 7fd6d63c5740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a4eda489d0 0x55a4eda48db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:25.454 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.446+0000 7fd6d63c5740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a4eda492f0 0x55a4eda57b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:25.454 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.446+0000 7fd6d63c5740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a4eda580c0 0x55a4eda5a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:25.454 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.446+0000 7fd6d63c5740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55a4ed9ad2f0 con 0x55a4eda492f0 2024-09-11T23:30:25.454 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.446+0000 7fd6d63c5740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55a4ed955270 con 0x55a4eda489d0 2024-09-11T23:30:25.454 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.446+0000 7fd6d63c5740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55a4ed9a92b0 con 0x55a4eda580c0 2024-09-11T23:30:25.454 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.446+0000 7fd6d13c3640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a4eda492f0 0x55a4eda57b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:25.454 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.446+0000 7fd6d0bc2640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a4eda580c0 0x55a4eda5a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:25.454 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.446+0000 7fd6d1bc4640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a4eda489d0 0x55a4eda48db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:25.454 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.446+0000 7fd6d13c3640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a4eda492f0 0x55a4eda57b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42000/0 (socket says 172.21.15.26:42000) 2024-09-11T23:30:25.454 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.446+0000 7fd6d13c3640 1 -- 172.21.15.26:0/1377741912 learned_addr learned my addr 172.21.15.26:0/1377741912 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:30:25.454 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.447+0000 7fd6d13c3640 1 -- 172.21.15.26:0/1377741912 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a4eda580c0 msgr2=0x55a4eda5a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:30:25.455 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.447+0000 7fd6d13c3640 1 --2- 172.21.15.26:0/1377741912 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a4eda580c0 0x55a4eda5a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:25.455 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.447+0000 7fd6d13c3640 1 -- 172.21.15.26:0/1377741912 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a4eda489d0 msgr2=0x55a4eda48db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:30:25.455 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.447+0000 7fd6d13c3640 1 --2- 172.21.15.26:0/1377741912 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a4eda489d0 0x55a4eda48db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:25.455 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.447+0000 7fd6d13c3640 1 -- 172.21.15.26:0/1377741912 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a4ed954340 con 0x55a4eda492f0 2024-09-11T23:30:25.455 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.447+0000 7fd6c27fc640 1 --2- 172.21.15.26:0/2277061702 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fd6b80a2e20 0x7fd6b80a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:25.455 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.447+0000 7fd6d1bc4640 1 --2- 172.21.15.26:0/1377741912 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a4eda489d0 0x55a4eda48db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:30:25.455 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.447+0000 7fd6d1bc4640 1 --2- 172.21.15.26:0/2277061702 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fd6b80a2e20 0x7fd6b80a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:25.455 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.447+0000 7fd6c27fc640 1 -- 172.21.15.26:0/2277061702 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8542+0+0 (secure 0 0 0) 0x7fd6bc0ef9a0 con 0x7fd6cc102050 2024-09-11T23:30:25.455 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.447+0000 7fd6d13c3640 1 --2- 172.21.15.26:0/1377741912 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a4eda492f0 0x55a4eda57b80 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fd6c8009ca0 tx=0x7fd6c8011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=194fd84facc5d2be server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:30:25.455 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.447+0000 7fd6b77fe640 1 -- 172.21.15.26:0/1377741912 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6c8004250 con 0x55a4eda492f0 2024-09-11T23:30:25.455 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.447+0000 7fd6b77fe640 1 -- 172.21.15.26:0/1377741912 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd6c80043f0 con 0x55a4eda492f0 2024-09-11T23:30:25.455 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.447+0000 7fd6b77fe640 1 -- 172.21.15.26:0/1377741912 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6c8019820 con 0x55a4eda492f0 2024-09-11T23:30:25.455 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.448+0000 7fd6d1bc4640 1 --2- 172.21.15.26:0/2277061702 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fd6b80a2e20 0x7fd6b80a52e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fd6c4009fd0 tx=0x7fd6c4009500 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:30:25.456 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.448+0000 7fd6d63c5740 1 -- 172.21.15.26:0/1377741912 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a4eda492f0 msgr2=0x55a4eda57b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:30:25.456 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.448+0000 7fd6d63c5740 1 --2- 172.21.15.26:0/1377741912 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a4eda492f0 0x55a4eda57b80 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fd6c8009ca0 tx=0x7fd6c8011810 comp rx=0 tx=0).stop 2024-09-11T23:30:25.456 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.448+0000 7fd6d63c5740 1 -- 172.21.15.26:0/1377741912 shutdown_connections 2024-09-11T23:30:25.456 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.448+0000 7fd6d63c5740 1 --2- 172.21.15.26:0/1377741912 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a4eda580c0 0x55a4eda5a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:25.456 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.448+0000 7fd6d63c5740 1 --2- 172.21.15.26:0/1377741912 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a4eda492f0 0x55a4eda57b80 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:25.456 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.448+0000 7fd6d63c5740 1 --2- 172.21.15.26:0/1377741912 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a4eda489d0 0x55a4eda48db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:25.456 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.448+0000 7fd6d63c5740 1 -- 172.21.15.26:0/1377741912 >> 172.21.15.26:0/1377741912 conn(0x55a4eda45060 msgr2=0x55a4eda45440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:30:25.456 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.448+0000 7fd6d63c5740 1 -- 172.21.15.26:0/1377741912 shutdown_connections 2024-09-11T23:30:25.456 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.448+0000 7fd6d63c5740 1 -- 172.21.15.26:0/1377741912 wait complete. 2024-09-11T23:30:25.456 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.448+0000 7fd6d63c5740 1 -- start start 2024-09-11T23:30:25.456 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.448+0000 7fd6d63c5740 1 Processor -- start 2024-09-11T23:30:25.456 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.449+0000 7fd6d63c5740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a4eda489d0 0x55a4eda54fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:25.456 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.449+0000 7fd6d63c5740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a4eda492f0 0x55a4eda55520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:25.457 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.449+0000 7fd6d63c5740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a4eda580c0 0x55a4eda55a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:25.457 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.449+0000 7fd6d63c5740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55a4eda5acf0 con 0x55a4eda489d0 2024-09-11T23:30:25.457 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.449+0000 7fd6d63c5740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55a4ed936290 con 0x55a4eda580c0 2024-09-11T23:30:25.457 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.449+0000 7fd6d63c5740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55a4eda5ae70 con 0x55a4eda492f0 2024-09-11T23:30:25.457 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.449+0000 7fd6d63c5740 10 client.0 _ll_register_callbacks cb 0x55a4ed967140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:30:25.457 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.449+0000 7fd6d13c3640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a4eda489d0 0x55a4eda54fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:25.457 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.449+0000 7fd6d13c3640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a4eda489d0 0x55a4eda54fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42004/0 (socket says 172.21.15.26:42004) 2024-09-11T23:30:25.457 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.449+0000 7fd6d13c3640 1 -- 172.21.15.26:0/3736378046 learned_addr learned my addr 172.21.15.26:0/3736378046 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:30:25.457 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.449+0000 7fd6d0bc2640 1 --2- 172.21.15.26:0/3736378046 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a4eda492f0 0x55a4eda55520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:25.457 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.449+0000 7fd6d1bc4640 1 --2- 172.21.15.26:0/3736378046 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a4eda580c0 0x55a4eda55a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:25.457 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.449+0000 7fd6d13c3640 1 -- 172.21.15.26:0/3736378046 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a4eda492f0 msgr2=0x55a4eda55520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:30:25.457 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.449+0000 7fd6d13c3640 1 --2- 172.21.15.26:0/3736378046 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a4eda492f0 0x55a4eda55520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:25.457 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.449+0000 7fd6d13c3640 1 -- 172.21.15.26:0/3736378046 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a4eda580c0 msgr2=0x55a4eda55a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:30:25.458 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.449+0000 7fd6d13c3640 1 --2- 172.21.15.26:0/3736378046 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a4eda580c0 0x55a4eda55a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:25.458 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.449+0000 7fd6d13c3640 1 -- 172.21.15.26:0/3736378046 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a4ed6c0210 con 0x55a4eda489d0 2024-09-11T23:30:25.458 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.449+0000 7fd6d13c3640 1 --2- 172.21.15.26:0/3736378046 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a4eda489d0 0x55a4eda54fe0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fd6c80043a0 tx=0x7fd6c8005d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a035e35de2f8c01 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:30:25.458 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.449+0000 7fd6b4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:30:25.458 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.450+0000 7fd6b4ff9640 1 -- 172.21.15.26:0/3736378046 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6c8027020 con 0x55a4eda489d0 2024-09-11T23:30:25.458 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.450+0000 7fd6b4ff9640 1 -- 172.21.15.26:0/3736378046 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8542+0+0 (secure 0 0 0) 0x7fd6c8003800 con 0x55a4eda489d0 2024-09-11T23:30:25.458 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.450+0000 7fd6d63c5740 1 -- 172.21.15.26:0/3736378046 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd6c80043f0 con 0x55a4eda489d0 2024-09-11T23:30:25.458 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.450+0000 7fd6b4ff9640 1 -- 172.21.15.26:0/3736378046 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7fd6c8032050 con 0x55a4eda489d0 2024-09-11T23:30:25.458 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.450+0000 7fd6d63c5740 10 client.17205 fetch_fsmap learned FSMap version 65 2024-09-11T23:30:25.458 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.450+0000 7fd6d63c5740 1 -- 172.21.15.26:0/3736378046 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) -- 0x55a4ed78c620 con 0x55a4eda489d0 2024-09-11T23:30:25.458 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.450+0000 7fd6b4ff9640 1 -- 172.21.15.26:0/3736378046 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd6c80043f0 con 0x55a4eda489d0 2024-09-11T23:30:25.458 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.450+0000 7fd6b4ff9640 1 -- 172.21.15.26:0/3736378046 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 65) ==== 8402+0+0 (secure 0 0 0) 0x7fd6c8018070 con 0x55a4eda489d0 2024-09-11T23:30:25.458 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.450+0000 7fd6d63c5740 10 client.17205 fetch_fsmap finished waiting for FSMap version 65 2024-09-11T23:30:25.458 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.450+0000 7fd6d63c5740 10 client.17205 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:30:25.459 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.450+0000 7fd6d63c5740 20 client.17205 populate_metadata read hostname 'smithi026' 2024-09-11T23:30:25.459 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.450+0000 7fd6d63c5740 1 --2- 172.21.15.26:0/3736378046 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55a4eda67ff0 0x55a4eda6a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:25.459 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.450+0000 7fd6d63c5740 4 client.17205 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:30:25.459 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.450+0000 7fd6d63c5740 1 -- 172.21.15.26:0/3736378046 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a4ed9baa20 con 0x55a4eda67ff0 2024-09-11T23:30:25.459 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.451+0000 7fd6d0bc2640 1 --2- 172.21.15.26:0/3736378046 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55a4eda67ff0 0x55a4eda6a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:25.459 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.451+0000 7fd6b4ff9640 10 client.17205 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:30:25.459 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.451+0000 7fd6d0bc2640 1 --2- 172.21.15.26:0/3736378046 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55a4eda67ff0 0x55a4eda6a3e0 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a79df3fa479bc51 server_cookie=fd13dd39d1f6767f in_seq=0 out_seq=0 2024-09-11T23:30:25.459 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.452+0000 7fd6b4ff9640 1 -- 172.21.15.26:0/3736378046 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a4ed9baa20 con 0x55a4eda67ff0 2024-09-11T23:30:25.459 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.452+0000 7fd6b4ff9640 10 client.17205 handle_command_reply: tid=0 2024-09-11T23:30:25.459 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.452+0000 7fd6d63c5740 1 client.17205 shutdown 2024-09-11T23:30:25.459 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.452+0000 7fd6d63c5740 1 -- 172.21.15.26:0/3736378046 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a4eda489d0 msgr2=0x55a4eda54fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:30:25.459 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.452+0000 7fd6d63c5740 1 --2- 172.21.15.26:0/3736378046 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a4eda489d0 0x55a4eda54fe0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fd6c80043a0 tx=0x7fd6c8005d30 comp rx=0 tx=0).stop 2024-09-11T23:30:25.459 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.452+0000 7fd6d63c5740 1 -- 172.21.15.26:0/3736378046 shutdown_connections 2024-09-11T23:30:25.460 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.452+0000 7fd6d63c5740 1 --2- 172.21.15.26:0/3736378046 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a4eda580c0 0x55a4eda55a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:25.460 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.452+0000 7fd6d63c5740 1 --2- 172.21.15.26:0/3736378046 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55a4eda67ff0 0x55a4eda6a3e0 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:25.460 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.452+0000 7fd6d63c5740 1 --2- 172.21.15.26:0/3736378046 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a4eda492f0 0x55a4eda55520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:25.460 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.452+0000 7fd6d63c5740 1 --2- 172.21.15.26:0/3736378046 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a4eda489d0 0x55a4eda54fe0 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:25.460 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.452+0000 7fd6b4ff9640 0 client.17205 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:30:25.460 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.452+0000 7fd6d63c5740 1 -- 172.21.15.26:0/3736378046 >> 172.21.15.26:0/3736378046 conn(0x55a4eda45060 msgr2=0x55a4eda4ad60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:30:25.460 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.452+0000 7fd6d63c5740 1 -- 172.21.15.26:0/3736378046 shutdown_connections 2024-09-11T23:30:25.460 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.452+0000 7fd6d63c5740 1 -- 172.21.15.26:0/3736378046 wait complete. 2024-09-11T23:30:25.460 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.452+0000 7fd6d63c5740 20 client.17205 trim_cache size 0 max 16384 2024-09-11T23:30:25.469 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.466+0000 7fd6d63c5740 1 Processor -- start 2024-09-11T23:30:25.469 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.466+0000 7fd6d63c5740 1 -- start start 2024-09-11T23:30:25.469 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.466+0000 7fd6d63c5740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a4eda489d0 0x55a4eda6fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:25.469 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.467+0000 7fd6d63c5740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a4eda492f0 0x55a4eda700f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:25.469 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.467+0000 7fd6d63c5740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a4eda580c0 0x55a4eda52220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:25.469 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.467+0000 7fd6d63c5740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd6cc1ab640 con 0x55a4eda489d0 2024-09-11T23:30:25.469 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.467+0000 7fd6d63c5740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fd6cc1ad300 con 0x55a4eda492f0 2024-09-11T23:30:25.469 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.467+0000 7fd6d63c5740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd6cc1abd70 con 0x55a4eda580c0 2024-09-11T23:30:25.469 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.467+0000 7fd6d13c3640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a4eda489d0 0x55a4eda6fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:25.469 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.467+0000 7fd6d0bc2640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a4eda492f0 0x55a4eda700f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:25.470 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.467+0000 7fd6d13c3640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a4eda489d0 0x55a4eda6fbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42012/0 (socket says 172.21.15.26:42012) 2024-09-11T23:30:25.470 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.467+0000 7fd6d13c3640 1 -- 172.21.15.26:0/4200905038 learned_addr learned my addr 172.21.15.26:0/4200905038 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:30:25.470 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.467+0000 7fd6d1bc4640 1 --2- 172.21.15.26:0/4200905038 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a4eda580c0 0x55a4eda52220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:25.470 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.467+0000 7fd6d0bc2640 1 -- 172.21.15.26:0/4200905038 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a4eda580c0 msgr2=0x55a4eda52220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:30:25.470 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.467+0000 7fd6d0bc2640 1 --2- 172.21.15.26:0/4200905038 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a4eda580c0 0x55a4eda52220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:25.470 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.467+0000 7fd6d0bc2640 1 -- 172.21.15.26:0/4200905038 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a4eda489d0 msgr2=0x55a4eda6fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:30:25.470 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.467+0000 7fd6d0bc2640 1 --2- 172.21.15.26:0/4200905038 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a4eda489d0 0x55a4eda6fbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:25.470 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.467+0000 7fd6d0bc2640 1 -- 172.21.15.26:0/4200905038 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a4eda53f70 con 0x55a4eda492f0 2024-09-11T23:30:25.470 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.467+0000 7fd6d13c3640 1 --2- 172.21.15.26:0/4200905038 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a4eda489d0 0x55a4eda6fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:30:25.470 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.467+0000 7fd6d0bc2640 1 --2- 172.21.15.26:0/4200905038 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a4eda492f0 0x55a4eda700f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fd6bc0ed0e0 tx=0x7fd6bc0f5550 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c230fc14b32ab31b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:30:25.470 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.467+0000 7fd6b6ffd640 1 -- 172.21.15.26:0/4200905038 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6bc049040 con 0x55a4eda492f0 2024-09-11T23:30:25.470 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.467+0000 7fd6b6ffd640 1 -- 172.21.15.26:0/4200905038 <== mon.2 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd6bc0efe50 con 0x55a4eda492f0 2024-09-11T23:30:25.471 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.467+0000 7fd6b6ffd640 1 -- 172.21.15.26:0/4200905038 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6bc0bc020 con 0x55a4eda492f0 2024-09-11T23:30:25.471 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.468+0000 7fd6d63c5740 1 -- 172.21.15.26:0/4200905038 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a4eda492f0 msgr2=0x55a4eda700f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:30:25.471 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.468+0000 7fd6d63c5740 1 --2- 172.21.15.26:0/4200905038 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a4eda492f0 0x55a4eda700f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fd6bc0ed0e0 tx=0x7fd6bc0f5550 comp rx=0 tx=0).stop 2024-09-11T23:30:25.471 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.468+0000 7fd6d63c5740 1 -- 172.21.15.26:0/4200905038 shutdown_connections 2024-09-11T23:30:25.471 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.468+0000 7fd6d63c5740 1 --2- 172.21.15.26:0/4200905038 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a4eda492f0 0x55a4eda700f0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:25.471 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.468+0000 7fd6d63c5740 1 --2- 172.21.15.26:0/4200905038 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a4eda580c0 0x55a4eda52220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:25.471 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.468+0000 7fd6d63c5740 1 --2- 172.21.15.26:0/4200905038 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a4eda489d0 0x55a4eda6fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:25.471 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.468+0000 7fd6d63c5740 1 -- 172.21.15.26:0/4200905038 >> 172.21.15.26:0/4200905038 conn(0x55a4eda45060 msgr2=0x55a4eda57060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:30:25.471 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.468+0000 7fd6d63c5740 1 -- 172.21.15.26:0/4200905038 shutdown_connections 2024-09-11T23:30:25.471 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.468+0000 7fd6d63c5740 1 -- 172.21.15.26:0/4200905038 wait complete. 2024-09-11T23:30:25.471 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.468+0000 7fd6d63c5740 1 -- start start 2024-09-11T23:30:25.471 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.468+0000 7fd6d63c5740 1 Processor -- start 2024-09-11T23:30:25.474 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.469+0000 7fd6d63c5740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a4eda489d0 0x55a4eda5be60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:25.474 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.469+0000 7fd6d63c5740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a4eda492f0 0x55a4eda5c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:25.474 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.469+0000 7fd6d63c5740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a4eda580c0 0x55a4eda5c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:25.474 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.469+0000 7fd6d63c5740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd6cc1acbd0 con 0x55a4eda580c0 2024-09-11T23:30:25.475 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.469+0000 7fd6d63c5740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fd6cc1ac500 con 0x55a4eda489d0 2024-09-11T23:30:25.475 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.469+0000 7fd6d63c5740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd6cc1a9640 con 0x55a4eda492f0 2024-09-11T23:30:25.475 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.469+0000 7fd6d63c5740 10 client.0 _ll_register_callbacks cb 0x7fd6c4009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:30:25.475 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.469+0000 7fd6d1bc4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a4eda580c0 0x55a4eda5c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:25.475 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.469+0000 7fd6d0bc2640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a4eda492f0 0x55a4eda5c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:25.475 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.469+0000 7fd6d13c3640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a4eda489d0 0x55a4eda5be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:25.475 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.469+0000 7fd6d1bc4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a4eda580c0 0x55a4eda5c8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42026/0 (socket says 172.21.15.26:42026) 2024-09-11T23:30:25.475 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.469+0000 7fd6d1bc4640 1 -- 172.21.15.26:0/274495765 learned_addr learned my addr 172.21.15.26:0/274495765 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:30:25.475 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.469+0000 7fd6d0bc2640 1 --2- 172.21.15.26:0/274495765 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a4eda492f0 0x55a4eda5c3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:33630/0 (socket says 172.21.15.26:33630) 2024-09-11T23:30:25.475 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.469+0000 7fd6d1bc4640 1 -- 172.21.15.26:0/274495765 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a4eda492f0 msgr2=0x55a4eda5c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:30:25.475 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.469+0000 7fd6d1bc4640 1 --2- 172.21.15.26:0/274495765 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a4eda492f0 0x55a4eda5c3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:25.475 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.469+0000 7fd6d1bc4640 1 -- 172.21.15.26:0/274495765 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a4eda489d0 msgr2=0x55a4eda5be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:30:25.476 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.469+0000 7fd6d1bc4640 1 --2- 172.21.15.26:0/274495765 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a4eda489d0 0x55a4eda5be60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:25.476 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.469+0000 7fd6d1bc4640 1 -- 172.21.15.26:0/274495765 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a4eda49120 con 0x55a4eda580c0 2024-09-11T23:30:25.476 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.469+0000 7fd6d13c3640 1 --2- 172.21.15.26:0/274495765 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a4eda489d0 0x55a4eda5be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:30:25.476 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.469+0000 7fd6d1bc4640 1 --2- 172.21.15.26:0/274495765 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a4eda580c0 0x55a4eda5c8e0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fd6c4011f20 tx=0x7fd6c4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5bc683cf1d0203d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:30:25.476 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.469+0000 7fd6b4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:30:25.476 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.470+0000 7fd6b4ff9640 1 -- 172.21.15.26:0/274495765 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6c4004b20 con 0x55a4eda580c0 2024-09-11T23:30:25.476 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.470+0000 7fd6d63c5740 1 -- 172.21.15.26:0/274495765 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd6bc0efe50 con 0x55a4eda580c0 2024-09-11T23:30:25.476 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.470+0000 7fd6b4ff9640 1 -- 172.21.15.26:0/274495765 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8542+0+0 (secure 0 0 0) 0x7fd6c4004cc0 con 0x55a4eda580c0 2024-09-11T23:30:25.476 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.470+0000 7fd6b4ff9640 1 -- 172.21.15.26:0/274495765 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7fd6c4010640 con 0x55a4eda580c0 2024-09-11T23:30:25.476 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.470+0000 7fd6d63c5740 10 client.17211 fetch_fsmap learned FSMap version 65 2024-09-11T23:30:25.476 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.470+0000 7fd6d63c5740 1 -- 172.21.15.26:0/274495765 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) -- 0x55a4edad2d50 con 0x55a4eda580c0 2024-09-11T23:30:25.476 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.470+0000 7fd6b4ff9640 1 -- 172.21.15.26:0/274495765 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd6bc0efe50 con 0x55a4eda580c0 2024-09-11T23:30:25.476 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.470+0000 7fd6b4ff9640 1 -- 172.21.15.26:0/274495765 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 65) ==== 8402+0+0 (secure 0 0 0) 0x7fd6c4005540 con 0x55a4eda580c0 2024-09-11T23:30:25.477 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.470+0000 7fd6d63c5740 10 client.17211 fetch_fsmap finished waiting for FSMap version 65 2024-09-11T23:30:25.477 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.470+0000 7fd6d63c5740 10 client.17211 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:30:25.477 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.470+0000 7fd6d63c5740 20 client.17211 populate_metadata read hostname 'smithi026' 2024-09-11T23:30:25.477 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.470+0000 7fd6d63c5740 1 --2- 172.21.15.26:0/274495765 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55a4edad34c0 0x55a4edad58e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:25.477 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.470+0000 7fd6d63c5740 4 client.17211 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T23:30:25.477 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.470+0000 7fd6d63c5740 1 -- 172.21.15.26:0/274495765 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a4ed9b4d70 con 0x55a4edad34c0 2024-09-11T23:30:25.477 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.470+0000 7fd6d13c3640 1 --2- 172.21.15.26:0/274495765 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55a4edad34c0 0x55a4edad58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:25.477 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.471+0000 7fd6d13c3640 1 --2- 172.21.15.26:0/274495765 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55a4edad34c0 0x55a4edad58e0 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=995bba9dc5053486 server_cookie=161bc2ea18c46df8 in_seq=0 out_seq=0 2024-09-11T23:30:25.477 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.471+0000 7fd6b4ff9640 10 client.17211 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:30:25.569 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.568+0000 7fd6b4ff9640 1 -- 172.21.15.26:0/274495765 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a4ed9b4d70 con 0x55a4edad34c0 2024-09-11T23:30:25.569 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.568+0000 7fd6b4ff9640 10 client.17211 handle_command_reply: tid=0 2024-09-11T23:30:25.569 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.568+0000 7fd6d63c5740 1 client.17211 shutdown 2024-09-11T23:30:25.569 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.568+0000 7fd6d63c5740 1 -- 172.21.15.26:0/274495765 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a4eda580c0 msgr2=0x55a4eda5c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:30:25.569 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.568+0000 7fd6d63c5740 1 --2- 172.21.15.26:0/274495765 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a4eda580c0 0x55a4eda5c8e0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fd6c4011f20 tx=0x7fd6c4004a40 comp rx=0 tx=0).stop 2024-09-11T23:30:25.570 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.568+0000 7fd6d63c5740 1 -- 172.21.15.26:0/274495765 shutdown_connections 2024-09-11T23:30:25.570 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.568+0000 7fd6d63c5740 1 --2- 172.21.15.26:0/274495765 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55a4edad34c0 0x55a4edad58e0 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:25.570 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.568+0000 7fd6d63c5740 1 --2- 172.21.15.26:0/274495765 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a4eda580c0 0x55a4eda5c8e0 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:25.570 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.568+0000 7fd6d63c5740 1 --2- 172.21.15.26:0/274495765 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a4eda492f0 0x55a4eda5c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:25.570 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.568+0000 7fd6d63c5740 1 --2- 172.21.15.26:0/274495765 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a4eda489d0 0x55a4eda5be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:25.570 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.568+0000 7fd6b4ff9640 0 client.17211 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:30:25.570 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.568+0000 7fd6d63c5740 1 -- 172.21.15.26:0/274495765 >> 172.21.15.26:0/274495765 conn(0x55a4eda45060 msgr2=0x55a4eda439a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:30:25.570 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.568+0000 7fd6d63c5740 1 -- 172.21.15.26:0/274495765 shutdown_connections 2024-09-11T23:30:25.570 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.568+0000 7fd6d63c5740 1 -- 172.21.15.26:0/274495765 wait complete. 2024-09-11T23:30:25.570 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.568+0000 7fd6d63c5740 20 client.17211 trim_cache size 0 max 16384 2024-09-11T23:30:25.571 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-09-11T23:30:25.571 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "message": "", 2024-09-11T23:30:25.571 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-09-11T23:30:25.571 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-09-11T23:30:25.572 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.569+0000 7fd6d3627640 1 -- 172.21.15.26:0/2277061702 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fd6b80a2e20 msgr2=0x7fd6b80a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:30:25.572 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.569+0000 7fd6d3627640 1 --2- 172.21.15.26:0/2277061702 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fd6b80a2e20 0x7fd6b80a52e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fd6c4009fd0 tx=0x7fd6c4009500 comp rx=0 tx=0).stop 2024-09-11T23:30:25.572 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.569+0000 7fd6d3627640 1 -- 172.21.15.26:0/2277061702 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd6cc102050 msgr2=0x7fd6cc1995c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:30:25.572 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.569+0000 7fd6d3627640 1 --2- 172.21.15.26:0/2277061702 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd6cc102050 0x7fd6cc1995c0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fd6bc004770 tx=0x7fd6bc003c10 comp rx=0 tx=0).stop 2024-09-11T23:30:25.572 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.569+0000 7fd6d3627640 1 -- 172.21.15.26:0/2277061702 shutdown_connections 2024-09-11T23:30:25.572 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.569+0000 7fd6d3627640 1 --2- 172.21.15.26:0/2277061702 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fd6b80a2e20 0x7fd6b80a52e0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:25.572 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.569+0000 7fd6d3627640 1 --2- 172.21.15.26:0/2277061702 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd6cc105af0 0x7fd6cc1a4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:25.572 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.569+0000 7fd6d3627640 1 --2- 172.21.15.26:0/2277061702 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd6cc102050 0x7fd6cc1995c0 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:25.572 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.569+0000 7fd6d3627640 1 --2- 172.21.15.26:0/2277061702 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd6cc1016d0 0x7fd6cc199080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:25.572 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.569+0000 7fd6d3627640 1 -- 172.21.15.26:0/2277061702 >> 172.21.15.26:0/2277061702 conn(0x7fd6cc075ec0 msgr2=0x7fd6cc0ff0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:30:25.572 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.569+0000 7fd6d3627640 1 -- 172.21.15.26:0/2277061702 shutdown_connections 2024-09-11T23:30:25.573 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:25.569+0000 7fd6d3627640 1 -- 172.21.15.26:0/2277061702 wait complete. 2024-09-11T23:30:26.052 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:30:25 smithi076 ceph-mon[35296]: pgmap v376: 97 pgs: 97 active+clean; 2.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.6 MiB/s wr, 904 op/s 2024-09-11T23:30:26.238 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:30:25 smithi136 ceph-mon[35121]: pgmap v376: 97 pgs: 97 active+clean; 2.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.6 MiB/s wr, 904 op/s 2024-09-11T23:30:26.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:30:25 smithi026 ceph-mon[31172]: pgmap v376: 97 pgs: 97 active+clean; 2.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.6 MiB/s wr, 904 op/s 2024-09-11T23:30:28.052 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:30:27 smithi076 ceph-mon[35296]: pgmap v377: 97 pgs: 97 active+clean; 2.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.8 MiB/s wr, 807 op/s 2024-09-11T23:30:28.238 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:30:27 smithi136 ceph-mon[35121]: pgmap v377: 97 pgs: 97 active+clean; 2.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.8 MiB/s wr, 807 op/s 2024-09-11T23:30:28.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:30:27 smithi026 ceph-mon[31172]: pgmap v377: 97 pgs: 97 active+clean; 2.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.8 MiB/s wr, 807 op/s 2024-09-11T23:30:29.052 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:30:28 smithi076 ceph-mon[35296]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:30:29.238 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:30:28 smithi136 ceph-mon[35121]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:30:29.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:30:28 smithi026 ceph-mon[31172]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:30:30.052 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:30:29 smithi076 ceph-mon[35296]: pgmap v378: 97 pgs: 97 active+clean; 2.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.0 MiB/s wr, 828 op/s 2024-09-11T23:30:30.052 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:30:29 smithi076 ceph-mon[35296]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:30:30.238 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:30:29 smithi136 ceph-mon[35121]: pgmap v378: 97 pgs: 97 active+clean; 2.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.0 MiB/s wr, 828 op/s 2024-09-11T23:30:30.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:30:29 smithi136 ceph-mon[35121]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:30:30.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:30:29 smithi026 ceph-mon[31172]: pgmap v378: 97 pgs: 97 active+clean; 2.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.0 MiB/s wr, 828 op/s 2024-09-11T23:30:30.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:30:29 smithi026 ceph-mon[31172]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:30:32.052 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:30:31 smithi076 ceph-mon[35296]: pgmap v379: 97 pgs: 97 active+clean; 2.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.9 MiB/s wr, 441 op/s 2024-09-11T23:30:32.238 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:30:31 smithi136 ceph-mon[35121]: pgmap v379: 97 pgs: 97 active+clean; 2.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.9 MiB/s wr, 441 op/s 2024-09-11T23:30:32.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:30:31 smithi026 ceph-mon[31172]: pgmap v379: 97 pgs: 97 active+clean; 2.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.9 MiB/s wr, 441 op/s 2024-09-11T23:30:34.052 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:30:33 smithi076 ceph-mon[35296]: pgmap v380: 97 pgs: 97 active+clean; 2.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.9 MiB/s wr, 370 op/s 2024-09-11T23:30:34.052 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:30:33 smithi076 ceph-mon[35296]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:30:34.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:30:33 smithi136 ceph-mon[35121]: pgmap v380: 97 pgs: 97 active+clean; 2.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.9 MiB/s wr, 370 op/s 2024-09-11T23:30:34.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:30:33 smithi136 ceph-mon[35121]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:30:34.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:30:33 smithi026 ceph-mon[31172]: pgmap v380: 97 pgs: 97 active+clean; 2.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.9 MiB/s wr, 370 op/s 2024-09-11T23:30:34.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:30:33 smithi026 ceph-mon[31172]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:30:36.052 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:30:35 smithi076 ceph-mon[35296]: pgmap v381: 97 pgs: 97 active+clean; 2.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.4 MiB/s wr, 458 op/s 2024-09-11T23:30:36.238 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:30:35 smithi136 ceph-mon[35121]: pgmap v381: 97 pgs: 97 active+clean; 2.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.4 MiB/s wr, 458 op/s 2024-09-11T23:30:36.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:30:35 smithi026 ceph-mon[31172]: pgmap v381: 97 pgs: 97 active+clean; 2.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.4 MiB/s wr, 458 op/s 2024-09-11T23:30:38.238 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:30:37 smithi136 ceph-mon[35121]: pgmap v382: 97 pgs: 97 active+clean; 2.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.5 MiB/s wr, 479 op/s 2024-09-11T23:30:38.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:30:37 smithi026 ceph-mon[31172]: pgmap v382: 97 pgs: 97 active+clean; 2.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.5 MiB/s wr, 479 op/s 2024-09-11T23:30:38.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:30:37 smithi076 ceph-mon[35296]: pgmap v382: 97 pgs: 97 active+clean; 2.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.5 MiB/s wr, 479 op/s 2024-09-11T23:30:38.663 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.660+0000 7f6df37c4640 1 -- 172.21.15.26:0/3407064828 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6dec102c80 msgr2=0x7f6dec103060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:30:38.663 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.660+0000 7f6df37c4640 1 --2- 172.21.15.26:0/3407064828 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6dec102c80 0x7f6dec103060 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f6de4009900 tx=0x7f6de402a480 comp rx=0 tx=0).stop 2024-09-11T23:30:38.663 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.660+0000 7f6df37c4640 1 -- 172.21.15.26:0/3407064828 shutdown_connections 2024-09-11T23:30:38.663 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.660+0000 7f6df37c4640 1 --2- 172.21.15.26:0/3407064828 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6dec1035a0 0x7f6dec10ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:38.664 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.660+0000 7f6df37c4640 1 --2- 172.21.15.26:0/3407064828 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6dec10c0d0 0x7f6dec10e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:38.664 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.660+0000 7f6df37c4640 1 --2- 172.21.15.26:0/3407064828 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6dec102c80 0x7f6dec103060 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:38.664 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.660+0000 7f6df37c4640 1 -- 172.21.15.26:0/3407064828 >> 172.21.15.26:0/3407064828 conn(0x7f6dec077990 msgr2=0x7f6dec0fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:30:38.664 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.660+0000 7f6df37c4640 1 -- 172.21.15.26:0/3407064828 shutdown_connections 2024-09-11T23:30:38.664 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.660+0000 7f6df37c4640 1 -- 172.21.15.26:0/3407064828 wait complete. 2024-09-11T23:30:38.664 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.660+0000 7f6df37c4640 1 Processor -- start 2024-09-11T23:30:38.664 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.660+0000 7f6df37c4640 1 -- start start 2024-09-11T23:30:38.664 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.661+0000 7f6df37c4640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6dec102c80 0x7f6dec198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:38.664 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.661+0000 7f6df37c4640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6dec1035a0 0x7f6dec198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:38.665 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.661+0000 7f6df37c4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6dec10c0d0 0x7f6dec19cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:38.665 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.661+0000 7f6df37c4640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f6dec111eb0 con 0x7f6dec10c0d0 2024-09-11T23:30:38.665 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.661+0000 7f6df37c4640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f6dec111d30 con 0x7f6dec1035a0 2024-09-11T23:30:38.665 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.661+0000 7f6df37c4640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f6dec112030 con 0x7f6dec102c80 2024-09-11T23:30:38.665 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.661+0000 7f6df1d61640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6dec10c0d0 0x7f6dec19cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:38.665 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.661+0000 7f6df1560640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6dec102c80 0x7f6dec198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:38.665 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.661+0000 7f6df1560640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6dec102c80 0x7f6dec198340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:59476/0 (socket says 172.21.15.26:59476) 2024-09-11T23:30:38.666 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.661+0000 7f6df1d61640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6dec10c0d0 0x7f6dec19cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49656/0 (socket says 172.21.15.26:49656) 2024-09-11T23:30:38.666 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.661+0000 7f6df1560640 1 -- 172.21.15.26:0/228103427 learned_addr learned my addr 172.21.15.26:0/228103427 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:30:38.666 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.661+0000 7f6df0d5f640 1 --2- 172.21.15.26:0/228103427 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6dec1035a0 0x7f6dec198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:38.666 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.661+0000 7f6df1d61640 1 -- 172.21.15.26:0/228103427 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6dec102c80 msgr2=0x7f6dec198340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:30:38.666 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.661+0000 7f6df1d61640 1 --2- 172.21.15.26:0/228103427 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6dec102c80 0x7f6dec198340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:38.666 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.661+0000 7f6df1d61640 1 -- 172.21.15.26:0/228103427 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6dec1035a0 msgr2=0x7f6dec198880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:30:38.666 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.661+0000 7f6df1d61640 1 --2- 172.21.15.26:0/228103427 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6dec1035a0 0x7f6dec198880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:38.666 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.661+0000 7f6df1d61640 1 -- 172.21.15.26:0/228103427 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6dec19d330 con 0x7f6dec10c0d0 2024-09-11T23:30:38.666 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.661+0000 7f6df1d61640 1 --2- 172.21.15.26:0/228103427 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6dec10c0d0 0x7f6dec19cc00 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc009fd0 tx=0x7f6ddc00ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:30:38.667 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.661+0000 7f6dea7fc640 1 -- 172.21.15.26:0/228103427 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ddc00ce20 con 0x7f6dec10c0d0 2024-09-11T23:30:38.667 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.661+0000 7f6dea7fc640 1 -- 172.21.15.26:0/228103427 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ddc005020 con 0x7f6dec10c0d0 2024-09-11T23:30:38.667 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.661+0000 7f6dea7fc640 1 -- 172.21.15.26:0/228103427 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ddc010720 con 0x7f6dec10c0d0 2024-09-11T23:30:38.667 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.661+0000 7f6df37c4640 1 -- 172.21.15.26:0/228103427 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6dec19d620 con 0x7f6dec10c0d0 2024-09-11T23:30:38.667 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.662+0000 7f6df37c4640 1 -- 172.21.15.26:0/228103427 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6dec1a4eb0 con 0x7f6dec10c0d0 2024-09-11T23:30:38.675 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.664+0000 7f6df663f740 1 Processor -- start 2024-09-11T23:30:38.675 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.664+0000 7f6dea7fc640 1 -- 172.21.15.26:0/228103427 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f6ddc005540 con 0x7f6dec10c0d0 2024-09-11T23:30:38.675 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.664+0000 7f6df663f740 1 -- start start 2024-09-11T23:30:38.675 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.664+0000 7f6df663f740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56085b871af0 0x56085b871ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:38.675 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.664+0000 7f6df663f740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56085b872410 0x56085b87c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:38.675 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.664+0000 7f6df663f740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56085b87ccb0 0x56085b87f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:38.675 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.664+0000 7f6df663f740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x56085b7d62f0 con 0x56085b87ccb0 2024-09-11T23:30:38.675 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.664+0000 7f6df663f740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x56085b77e270 con 0x56085b871af0 2024-09-11T23:30:38.676 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.664+0000 7f6df663f740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x56085b7d22b0 con 0x56085b872410 2024-09-11T23:30:38.676 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.664+0000 7f6df0d5f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56085b871af0 0x56085b871ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:38.676 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.664+0000 7f6df0d5f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56085b871af0 0x56085b871ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:55600/0 (socket says 172.21.15.26:55600) 2024-09-11T23:30:38.676 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.665+0000 7f6df0d5f640 1 -- 172.21.15.26:0/1277691740 learned_addr learned my addr 172.21.15.26:0/1277691740 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:30:38.676 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.665+0000 7f6df1560640 1 --2- 172.21.15.26:0/1277691740 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56085b872410 0x56085b87c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:38.676 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.665+0000 7f6df0d5f640 1 --2- 172.21.15.26:0/1277691740 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56085b87ccb0 0x56085b87f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:38.676 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.665+0000 7f6df1560640 1 -- 172.21.15.26:0/1277691740 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56085b871af0 msgr2=0x56085b871ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:30:38.676 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.665+0000 7f6df1560640 1 --2- 172.21.15.26:0/1277691740 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56085b871af0 0x56085b871ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:38.676 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.665+0000 7f6dea7fc640 1 --2- 172.21.15.26:0/228103427 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f6dd40a2e20 0x7f6dd40a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:38.676 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.665+0000 7f6df1560640 1 -- 172.21.15.26:0/1277691740 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56085b87ccb0 msgr2=0x56085b87f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:30:38.676 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.665+0000 7f6df1560640 1 --2- 172.21.15.26:0/1277691740 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56085b87ccb0 0x56085b87f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:38.676 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.665+0000 7f6df1560640 1 -- 172.21.15.26:0/1277691740 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56085b77d340 con 0x56085b872410 2024-09-11T23:30:38.677 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.665+0000 7f6df0d5f640 1 --2- 172.21.15.26:0/1277691740 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56085b87ccb0 0x56085b87f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:30:38.677 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.665+0000 7f6df1d61640 1 --2- 172.21.15.26:0/228103427 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f6dd40a2e20 0x7f6dd40a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:38.677 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.665+0000 7f6df1560640 1 --2- 172.21.15.26:0/1277691740 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56085b872410 0x56085b87c770 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f6de4009230 tx=0x7f6de4005dd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=54a07db916bb5fe1 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:30:38.677 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.665+0000 7f6dea7fc640 1 -- 172.21.15.26:0/228103427 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8542+0+0 (secure 0 0 0) 0x7f6ddc0cfa80 con 0x7f6dec10c0d0 2024-09-11T23:30:38.677 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.665+0000 7f6dd37fe640 1 -- 172.21.15.26:0/1277691740 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6de4040070 con 0x56085b872410 2024-09-11T23:30:38.677 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.665+0000 7f6dd37fe640 1 -- 172.21.15.26:0/1277691740 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6de4004030 con 0x56085b872410 2024-09-11T23:30:38.677 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.665+0000 7f6dd37fe640 1 -- 172.21.15.26:0/1277691740 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6de4035040 con 0x56085b872410 2024-09-11T23:30:38.677 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.666+0000 7f6df663f740 1 -- 172.21.15.26:0/1277691740 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56085b872410 msgr2=0x56085b87c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:30:38.677 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.666+0000 7f6df1d61640 1 --2- 172.21.15.26:0/228103427 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f6dd40a2e20 0x7f6dd40a52e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc098900 tx=0x7f6ddc097bd0 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:30:38.677 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.666+0000 7f6df663f740 1 --2- 172.21.15.26:0/1277691740 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56085b872410 0x56085b87c770 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f6de4009230 tx=0x7f6de4005dd0 comp rx=0 tx=0).stop 2024-09-11T23:30:38.677 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.666+0000 7f6df663f740 1 -- 172.21.15.26:0/1277691740 shutdown_connections 2024-09-11T23:30:38.677 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.666+0000 7f6df663f740 1 --2- 172.21.15.26:0/1277691740 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56085b87ccb0 0x56085b87f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:38.677 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.666+0000 7f6df663f740 1 --2- 172.21.15.26:0/1277691740 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56085b872410 0x56085b87c770 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:38.678 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.666+0000 7f6df663f740 1 --2- 172.21.15.26:0/1277691740 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56085b871af0 0x56085b871ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:38.678 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.666+0000 7f6df663f740 1 -- 172.21.15.26:0/1277691740 >> 172.21.15.26:0/1277691740 conn(0x56085b86e060 msgr2=0x56085b86e440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:30:38.678 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.666+0000 7f6df663f740 1 -- 172.21.15.26:0/1277691740 shutdown_connections 2024-09-11T23:30:38.678 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.666+0000 7f6df663f740 1 -- 172.21.15.26:0/1277691740 wait complete. 2024-09-11T23:30:38.678 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.666+0000 7f6df663f740 1 -- start start 2024-09-11T23:30:38.678 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.666+0000 7f6df663f740 1 Processor -- start 2024-09-11T23:30:38.678 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.667+0000 7f6df663f740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56085b871af0 0x56085b879be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:38.678 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.667+0000 7f6df663f740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56085b872410 0x56085b87a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:38.678 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.667+0000 7f6df663f740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56085b87ccb0 0x56085b87a660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:38.678 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.667+0000 7f6df663f740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x56085b87f880 con 0x56085b871af0 2024-09-11T23:30:38.678 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.667+0000 7f6df663f740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x56085b75f290 con 0x56085b872410 2024-09-11T23:30:38.678 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.667+0000 7f6df663f740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x56085b87fa00 con 0x56085b87ccb0 2024-09-11T23:30:38.678 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.667+0000 7f6df663f740 10 client.0 _ll_register_callbacks cb 0x56085b790140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:30:38.679 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.667+0000 7f6df0d5f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56085b871af0 0x56085b879be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:38.679 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.667+0000 7f6df1560640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56085b872410 0x56085b87a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:38.679 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.667+0000 7f6df0d5f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56085b871af0 0x56085b879be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49676/0 (socket says 172.21.15.26:49676) 2024-09-11T23:30:38.679 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.667+0000 7f6df1560640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56085b872410 0x56085b87a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:55602/0 (socket says 172.21.15.26:55602) 2024-09-11T23:30:38.679 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.667+0000 7f6df0d5f640 1 -- 172.21.15.26:0/2915041509 learned_addr learned my addr 172.21.15.26:0/2915041509 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:30:38.679 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.667+0000 7f6df0d5f640 1 --2- 172.21.15.26:0/2915041509 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56085b87ccb0 0x56085b87a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:38.679 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.667+0000 7f6df0d5f640 1 -- 172.21.15.26:0/2915041509 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56085b87ccb0 msgr2=0x56085b87a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:30:38.679 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.667+0000 7f6df0d5f640 1 --2- 172.21.15.26:0/2915041509 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56085b87ccb0 0x56085b87a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:38.679 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.667+0000 7f6df0d5f640 1 -- 172.21.15.26:0/2915041509 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56085b872410 msgr2=0x56085b87a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:30:38.679 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.667+0000 7f6df0d5f640 1 --2- 172.21.15.26:0/2915041509 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56085b872410 0x56085b87a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:38.679 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.667+0000 7f6df0d5f640 1 -- 172.21.15.26:0/2915041509 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56085b4e9210 con 0x56085b871af0 2024-09-11T23:30:38.679 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.667+0000 7f6df1560640 1 --2- 172.21.15.26:0/2915041509 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56085b872410 0x56085b87a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:30:38.679 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.668+0000 7f6df0d5f640 1 --2- 172.21.15.26:0/2915041509 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56085b871af0 0x56085b879be0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f6de0009f20 tx=0x7f6de0005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b32bf2b0c7e77f24 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:30:38.680 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.668+0000 7f6dd0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:30:38.680 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.668+0000 7f6dd0ff9640 1 -- 172.21.15.26:0/2915041509 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6de0004050 con 0x56085b871af0 2024-09-11T23:30:38.680 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.668+0000 7f6df663f740 1 -- 172.21.15.26:0/2915041509 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6de4004030 con 0x56085b871af0 2024-09-11T23:30:38.680 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.668+0000 7f6dd0ff9640 1 -- 172.21.15.26:0/2915041509 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8542+0+0 (secure 0 0 0) 0x7f6de00041f0 con 0x56085b871af0 2024-09-11T23:30:38.680 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.668+0000 7f6dd0ff9640 1 -- 172.21.15.26:0/2915041509 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) ==== 24+0+0 (secure 0 0 0) 0x7f6de0027050 con 0x56085b871af0 2024-09-11T23:30:38.680 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.668+0000 7f6df663f740 10 client.17220 fetch_fsmap learned FSMap version 68 2024-09-11T23:30:38.680 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.668+0000 7f6df663f740 1 -- 172.21.15.26:0/2915041509 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=4+}) -- 0x56085b5b5620 con 0x56085b871af0 2024-09-11T23:30:38.680 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.668+0000 7f6dd0ff9640 1 -- 172.21.15.26:0/2915041509 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6de4004030 con 0x56085b871af0 2024-09-11T23:30:38.680 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.668+0000 7f6dd0ff9640 1 -- 172.21.15.26:0/2915041509 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 68) ==== 8434+0+0 (secure 0 0 0) 0x7f6de001f4e0 con 0x56085b871af0 2024-09-11T23:30:38.680 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.668+0000 7f6df663f740 10 client.17220 fetch_fsmap finished waiting for FSMap version 68 2024-09-11T23:30:38.680 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.668+0000 7f6df663f740 10 client.17220 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:30:38.680 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.668+0000 7f6df663f740 20 client.17220 populate_metadata read hostname 'smithi026' 2024-09-11T23:30:38.680 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.668+0000 7f6df663f740 1 --2- 172.21.15.26:0/2915041509 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x56085b88cc90 0x56085b88f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:38.680 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.668+0000 7f6df663f740 4 client.17220 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:30:38.680 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.668+0000 7f6df663f740 1 -- 172.21.15.26:0/2915041509 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56085b7e3a20 con 0x56085b88cc90 2024-09-11T23:30:38.681 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.668+0000 7f6df1560640 1 --2- 172.21.15.26:0/2915041509 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x56085b88cc90 0x56085b88f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:38.681 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.669+0000 7f6df1560640 1 --2- 172.21.15.26:0/2915041509 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x56085b88cc90 0x56085b88f0b0 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=815d1bc00a1f95bd server_cookie=f718726cefa2aa1a in_seq=0 out_seq=0 2024-09-11T23:30:38.681 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.669+0000 7f6dd0ff9640 10 client.17220 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:30:38.681 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.670+0000 7f6dd0ff9640 1 -- 172.21.15.26:0/2915041509 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56085b7e3a20 con 0x56085b88cc90 2024-09-11T23:30:38.681 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.670+0000 7f6dd0ff9640 10 client.17220 handle_command_reply: tid=0 2024-09-11T23:30:38.681 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.670+0000 7f6df663f740 1 client.17220 shutdown 2024-09-11T23:30:38.681 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.670+0000 7f6df663f740 1 -- 172.21.15.26:0/2915041509 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56085b871af0 msgr2=0x56085b879be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:30:38.681 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.670+0000 7f6df663f740 1 --2- 172.21.15.26:0/2915041509 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56085b871af0 0x56085b879be0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f6de0009f20 tx=0x7f6de0005f90 comp rx=0 tx=0).stop 2024-09-11T23:30:38.681 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.670+0000 7f6df663f740 1 -- 172.21.15.26:0/2915041509 shutdown_connections 2024-09-11T23:30:38.681 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.670+0000 7f6df663f740 1 --2- 172.21.15.26:0/2915041509 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56085b872410 0x56085b87a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:38.681 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.670+0000 7f6df663f740 1 --2- 172.21.15.26:0/2915041509 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x56085b88cc90 0x56085b88f0b0 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:38.681 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.670+0000 7f6df663f740 1 --2- 172.21.15.26:0/2915041509 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56085b87ccb0 0x56085b87a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:38.681 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.670+0000 7f6df663f740 1 --2- 172.21.15.26:0/2915041509 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56085b871af0 0x56085b879be0 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:38.681 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.670+0000 7f6df663f740 1 -- 172.21.15.26:0/2915041509 >> 172.21.15.26:0/2915041509 conn(0x56085b86e060 msgr2=0x56085b873d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:30:38.682 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.670+0000 7f6dd0ff9640 0 client.17220 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:30:38.682 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.670+0000 7f6df663f740 1 -- 172.21.15.26:0/2915041509 shutdown_connections 2024-09-11T23:30:38.682 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.670+0000 7f6df663f740 1 -- 172.21.15.26:0/2915041509 wait complete. 2024-09-11T23:30:38.682 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.670+0000 7f6df663f740 20 client.17220 trim_cache size 0 max 16384 2024-09-11T23:30:38.688 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.684+0000 7f6df663f740 1 Processor -- start 2024-09-11T23:30:38.688 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.684+0000 7f6df663f740 1 -- start start 2024-09-11T23:30:38.689 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.685+0000 7f6df663f740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56085b871af0 0x56085b87a460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:38.689 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.685+0000 7f6df663f740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56085b872410 0x56085b87bb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:38.689 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.685+0000 7f6df663f740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56085b87ccb0 0x56085b8949c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:38.689 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.685+0000 7f6df663f740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f6dec1ad790 con 0x56085b872410 2024-09-11T23:30:38.689 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.685+0000 7f6df663f740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f6dec1af570 con 0x56085b871af0 2024-09-11T23:30:38.689 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.685+0000 7f6df663f740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f6dec1adec0 con 0x56085b87ccb0 2024-09-11T23:30:38.689 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.685+0000 7f6df1560640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56085b872410 0x56085b87bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:38.689 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.685+0000 7f6df1560640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56085b872410 0x56085b87bb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49686/0 (socket says 172.21.15.26:49686) 2024-09-11T23:30:38.689 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.685+0000 7f6df1560640 1 -- 172.21.15.26:0/709965825 learned_addr learned my addr 172.21.15.26:0/709965825 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:30:38.689 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.685+0000 7f6df0d5f640 1 --2- 172.21.15.26:0/709965825 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56085b871af0 0x56085b87a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:38.690 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.685+0000 7f6df0d5f640 1 --2- 172.21.15.26:0/709965825 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56085b87ccb0 0x56085b8949c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:38.690 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.685+0000 7f6df1560640 1 -- 172.21.15.26:0/709965825 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56085b87ccb0 msgr2=0x56085b8949c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:30:38.690 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.685+0000 7f6df1560640 1 --2- 172.21.15.26:0/709965825 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56085b87ccb0 0x56085b8949c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:38.690 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.685+0000 7f6df1560640 1 -- 172.21.15.26:0/709965825 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56085b871af0 msgr2=0x56085b87a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:30:38.690 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.685+0000 7f6df1560640 1 --2- 172.21.15.26:0/709965825 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56085b871af0 0x56085b87a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:38.690 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.685+0000 7f6df1560640 1 -- 172.21.15.26:0/709965825 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56085b871870 con 0x56085b872410 2024-09-11T23:30:38.690 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.685+0000 7f6df1560640 1 --2- 172.21.15.26:0/709965825 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56085b872410 0x56085b87bb50 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f6de40092a0 tx=0x7f6de4005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f4a77476690d563 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:30:38.690 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.685+0000 7f6dd2ffd640 1 -- 172.21.15.26:0/709965825 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6de4040040 con 0x56085b872410 2024-09-11T23:30:38.690 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.685+0000 7f6dd2ffd640 1 -- 172.21.15.26:0/709965825 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6de40054a0 con 0x56085b872410 2024-09-11T23:30:38.690 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.685+0000 7f6dd2ffd640 1 -- 172.21.15.26:0/709965825 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6de4035040 con 0x56085b872410 2024-09-11T23:30:38.690 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.686+0000 7f6df663f740 1 -- 172.21.15.26:0/709965825 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56085b872410 msgr2=0x56085b87bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:30:38.690 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.686+0000 7f6df663f740 1 --2- 172.21.15.26:0/709965825 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56085b872410 0x56085b87bb50 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f6de40092a0 tx=0x7f6de4005dd0 comp rx=0 tx=0).stop 2024-09-11T23:30:38.690 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.686+0000 7f6df663f740 1 -- 172.21.15.26:0/709965825 shutdown_connections 2024-09-11T23:30:38.691 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.686+0000 7f6df663f740 1 --2- 172.21.15.26:0/709965825 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56085b87ccb0 0x56085b8949c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:38.691 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.686+0000 7f6df663f740 1 --2- 172.21.15.26:0/709965825 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56085b872410 0x56085b87bb50 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:38.691 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.686+0000 7f6df663f740 1 --2- 172.21.15.26:0/709965825 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56085b871af0 0x56085b87a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:38.691 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.686+0000 7f6df663f740 1 -- 172.21.15.26:0/709965825 >> 172.21.15.26:0/709965825 conn(0x56085b86e060 msgr2=0x56085b87b440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:30:38.691 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.686+0000 7f6df663f740 1 -- 172.21.15.26:0/709965825 shutdown_connections 2024-09-11T23:30:38.691 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.686+0000 7f6df663f740 1 -- 172.21.15.26:0/709965825 wait complete. 2024-09-11T23:30:38.691 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.686+0000 7f6df663f740 1 -- start start 2024-09-11T23:30:38.691 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.686+0000 7f6df663f740 1 Processor -- start 2024-09-11T23:30:38.691 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.687+0000 7f6df663f740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56085b871af0 0x56085b884cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:38.691 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.687+0000 7f6df663f740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56085b872410 0x56085b885200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:38.691 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.687+0000 7f6df663f740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56085b87ccb0 0x56085b881c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:38.691 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.687+0000 7f6df663f740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f6dec1aede0 con 0x56085b872410 2024-09-11T23:30:38.691 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.687+0000 7f6df663f740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f6dec1ae650 con 0x56085b871af0 2024-09-11T23:30:38.691 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.687+0000 7f6df663f740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f6dec1ab790 con 0x56085b87ccb0 2024-09-11T23:30:38.691 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.687+0000 7f6df663f740 10 client.0 _ll_register_callbacks cb 0x7f6de0009890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:30:38.692 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.687+0000 7f6df1560640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56085b872410 0x56085b885200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:38.692 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.687+0000 7f6df1560640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56085b872410 0x56085b885200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49692/0 (socket says 172.21.15.26:49692) 2024-09-11T23:30:38.692 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.687+0000 7f6df1560640 1 -- 172.21.15.26:0/2986149798 learned_addr learned my addr 172.21.15.26:0/2986149798 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:30:38.692 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.687+0000 7f6df0d5f640 1 --2- 172.21.15.26:0/2986149798 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56085b871af0 0x56085b884cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:38.692 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.687+0000 7f6df0d5f640 1 --2- 172.21.15.26:0/2986149798 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56085b87ccb0 0x56085b881c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:38.692 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.687+0000 7f6df1560640 1 -- 172.21.15.26:0/2986149798 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56085b87ccb0 msgr2=0x56085b881c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:30:38.692 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.687+0000 7f6df1560640 1 --2- 172.21.15.26:0/2986149798 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56085b87ccb0 0x56085b881c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:38.692 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.687+0000 7f6df1560640 1 -- 172.21.15.26:0/2986149798 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56085b871af0 msgr2=0x56085b884cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:30:38.692 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.687+0000 7f6df1560640 1 --2- 172.21.15.26:0/2986149798 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56085b871af0 0x56085b884cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:38.692 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.687+0000 7f6df1560640 1 -- 172.21.15.26:0/2986149798 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56085b8821d0 con 0x56085b872410 2024-09-11T23:30:38.692 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.687+0000 7f6df0d5f640 1 --2- 172.21.15.26:0/2986149798 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56085b87ccb0 0x56085b881c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:30:38.692 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.687+0000 7f6df1560640 1 --2- 172.21.15.26:0/2986149798 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56085b872410 0x56085b885200 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f6de402a4c0 tx=0x7f6de402ab80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10ac6c8c0d635739 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:30:38.692 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.687+0000 7f6dd0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:30:38.693 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.687+0000 7f6dd0ff9640 1 -- 172.21.15.26:0/2986149798 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6de4040070 con 0x56085b872410 2024-09-11T23:30:38.693 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.687+0000 7f6dd0ff9640 1 -- 172.21.15.26:0/2986149798 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8542+0+0 (secure 0 0 0) 0x7f6de40319c0 con 0x56085b872410 2024-09-11T23:30:38.693 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.687+0000 7f6df663f740 1 -- 172.21.15.26:0/2986149798 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6de40054a0 con 0x56085b872410 2024-09-11T23:30:38.693 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.688+0000 7f6dd0ff9640 1 -- 172.21.15.26:0/2986149798 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) ==== 24+0+0 (secure 0 0 0) 0x7f6de4061050 con 0x56085b872410 2024-09-11T23:30:38.693 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.688+0000 7f6df663f740 10 client.17226 fetch_fsmap learned FSMap version 68 2024-09-11T23:30:38.693 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.688+0000 7f6df663f740 1 -- 172.21.15.26:0/2986149798 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=4+}) -- 0x56085b8f8190 con 0x56085b872410 2024-09-11T23:30:38.693 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.688+0000 7f6dd0ff9640 1 -- 172.21.15.26:0/2986149798 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6de40054a0 con 0x56085b872410 2024-09-11T23:30:38.693 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.688+0000 7f6dd0ff9640 1 -- 172.21.15.26:0/2986149798 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 68) ==== 8434+0+0 (secure 0 0 0) 0x7f6de404f070 con 0x56085b872410 2024-09-11T23:30:38.693 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.688+0000 7f6df663f740 10 client.17226 fetch_fsmap finished waiting for FSMap version 68 2024-09-11T23:30:38.693 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.688+0000 7f6df663f740 10 client.17226 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:30:38.693 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.688+0000 7f6df663f740 20 client.17226 populate_metadata read hostname 'smithi026' 2024-09-11T23:30:38.694 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.688+0000 7f6df663f740 1 --2- 172.21.15.26:0/2986149798 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x56085b8f8520 0x56085b8fa940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:38.694 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.688+0000 7f6df663f740 4 client.17226 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T23:30:38.694 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.688+0000 7f6df663f740 1 -- 172.21.15.26:0/2986149798 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56085b7ddd70 con 0x56085b8f8520 2024-09-11T23:30:38.694 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.688+0000 7f6df0d5f640 1 --2- 172.21.15.26:0/2986149798 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x56085b8f8520 0x56085b8fa940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:38.694 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.689+0000 7f6dd0ff9640 10 client.17226 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:30:38.694 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.689+0000 7f6df0d5f640 1 --2- 172.21.15.26:0/2986149798 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x56085b8f8520 0x56085b8fa940 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf780782c48d0675 server_cookie=9b2c59808ca10b5b in_seq=0 out_seq=0 2024-09-11T23:30:38.949 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.949+0000 7f6dd0ff9640 1 -- 172.21.15.26:0/2986149798 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56085b7ddd70 con 0x56085b8f8520 2024-09-11T23:30:38.949 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.950+0000 7f6dd0ff9640 10 client.17226 handle_command_reply: tid=0 2024-09-11T23:30:38.949 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.950+0000 7f6df663f740 1 client.17226 shutdown 2024-09-11T23:30:38.951 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.950+0000 7f6df663f740 1 -- 172.21.15.26:0/2986149798 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56085b872410 msgr2=0x56085b885200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:30:38.951 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.950+0000 7f6df663f740 1 --2- 172.21.15.26:0/2986149798 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56085b872410 0x56085b885200 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f6de402a4c0 tx=0x7f6de402ab80 comp rx=0 tx=0).stop 2024-09-11T23:30:38.951 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.950+0000 7f6df663f740 1 -- 172.21.15.26:0/2986149798 shutdown_connections 2024-09-11T23:30:38.951 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.950+0000 7f6df663f740 1 --2- 172.21.15.26:0/2986149798 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x56085b8f8520 0x56085b8fa940 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:38.951 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.950+0000 7f6df663f740 1 --2- 172.21.15.26:0/2986149798 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56085b87ccb0 0x56085b881c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:38.951 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.950+0000 7f6df663f740 1 --2- 172.21.15.26:0/2986149798 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56085b872410 0x56085b885200 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:38.951 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.950+0000 7f6df663f740 1 --2- 172.21.15.26:0/2986149798 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56085b871af0 0x56085b884cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:38.951 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.950+0000 7f6df663f740 1 -- 172.21.15.26:0/2986149798 >> 172.21.15.26:0/2986149798 conn(0x56085b86e060 msgr2=0x56085b86d1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:30:38.951 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.950+0000 7f6dd0ff9640 0 client.17226 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:30:38.951 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.950+0000 7f6df663f740 1 -- 172.21.15.26:0/2986149798 shutdown_connections 2024-09-11T23:30:38.951 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.950+0000 7f6df663f740 1 -- 172.21.15.26:0/2986149798 wait complete. 2024-09-11T23:30:38.951 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.950+0000 7f6df663f740 20 client.17226 trim_cache size 0 max 16384 2024-09-11T23:30:38.952 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-09-11T23:30:38.952 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "message": "", 2024-09-11T23:30:38.952 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-09-11T23:30:38.952 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-09-11T23:30:38.953 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.951+0000 7f6df37c4640 1 -- 172.21.15.26:0/228103427 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f6dd40a2e20 msgr2=0x7f6dd40a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:30:38.953 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.951+0000 7f6df37c4640 1 --2- 172.21.15.26:0/228103427 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f6dd40a2e20 0x7f6dd40a52e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc098900 tx=0x7f6ddc097bd0 comp rx=0 tx=0).stop 2024-09-11T23:30:38.953 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.951+0000 7f6df37c4640 1 -- 172.21.15.26:0/228103427 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6dec10c0d0 msgr2=0x7f6dec19cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:30:38.953 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.951+0000 7f6df37c4640 1 --2- 172.21.15.26:0/228103427 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6dec10c0d0 0x7f6dec19cc00 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc009fd0 tx=0x7f6ddc00ef10 comp rx=0 tx=0).stop 2024-09-11T23:30:38.953 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.951+0000 7f6df37c4640 1 -- 172.21.15.26:0/228103427 shutdown_connections 2024-09-11T23:30:38.953 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.951+0000 7f6df37c4640 1 --2- 172.21.15.26:0/228103427 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f6dd40a2e20 0x7f6dd40a52e0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:38.953 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.951+0000 7f6df37c4640 1 --2- 172.21.15.26:0/228103427 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6dec1035a0 0x7f6dec198880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:38.953 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.951+0000 7f6df37c4640 1 --2- 172.21.15.26:0/228103427 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6dec10c0d0 0x7f6dec19cc00 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:38.953 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.951+0000 7f6df37c4640 1 --2- 172.21.15.26:0/228103427 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6dec102c80 0x7f6dec198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:38.953 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.951+0000 7f6df37c4640 1 -- 172.21.15.26:0/228103427 >> 172.21.15.26:0/228103427 conn(0x7f6dec077990 msgr2=0x7f6dec0fd000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:30:38.953 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.951+0000 7f6df37c4640 1 -- 172.21.15.26:0/228103427 shutdown_connections 2024-09-11T23:30:38.954 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:38.951+0000 7f6df37c4640 1 -- 172.21.15.26:0/228103427 wait complete. 2024-09-11T23:30:40.238 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:30:39 smithi136 ceph-mon[35121]: pgmap v383: 97 pgs: 97 active+clean; 2.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.7 MiB/s wr, 407 op/s 2024-09-11T23:30:40.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:30:39 smithi026 ceph-mon[31172]: pgmap v383: 97 pgs: 97 active+clean; 2.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.7 MiB/s wr, 407 op/s 2024-09-11T23:30:40.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:30:39 smithi076 ceph-mon[35296]: pgmap v383: 97 pgs: 97 active+clean; 2.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.7 MiB/s wr, 407 op/s 2024-09-11T23:30:42.238 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:30:41 smithi136 ceph-mon[35121]: pgmap v384: 97 pgs: 97 active+clean; 2.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.4 MiB/s wr, 533 op/s 2024-09-11T23:30:42.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:30:41 smithi026 ceph-mon[31172]: pgmap v384: 97 pgs: 97 active+clean; 2.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.4 MiB/s wr, 533 op/s 2024-09-11T23:30:42.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:30:41 smithi076 ceph-mon[35296]: pgmap v384: 97 pgs: 97 active+clean; 2.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.4 MiB/s wr, 533 op/s 2024-09-11T23:30:44.238 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:30:43 smithi136 ceph-mon[35121]: pgmap v385: 97 pgs: 97 active+clean; 2.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.5 MiB/s wr, 380 op/s 2024-09-11T23:30:44.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:30:43 smithi026 ceph-mon[31172]: pgmap v385: 97 pgs: 97 active+clean; 2.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.5 MiB/s wr, 380 op/s 2024-09-11T23:30:44.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:30:43 smithi076 ceph-mon[35296]: pgmap v385: 97 pgs: 97 active+clean; 2.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.5 MiB/s wr, 380 op/s 2024-09-11T23:30:46.238 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:30:45 smithi136 ceph-mon[35121]: pgmap v386: 97 pgs: 97 active+clean; 2.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.1 MiB/s wr, 470 op/s 2024-09-11T23:30:46.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:30:45 smithi026 ceph-mon[31172]: pgmap v386: 97 pgs: 97 active+clean; 2.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.1 MiB/s wr, 470 op/s 2024-09-11T23:30:46.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:30:45 smithi076 ceph-mon[35296]: pgmap v386: 97 pgs: 97 active+clean; 2.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.1 MiB/s wr, 470 op/s 2024-09-11T23:30:48.238 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:30:47 smithi136 ceph-mon[35121]: pgmap v387: 97 pgs: 97 active+clean; 2.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.8 MiB/s wr, 447 op/s 2024-09-11T23:30:48.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:30:47 smithi026 ceph-mon[31172]: pgmap v387: 97 pgs: 97 active+clean; 2.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.8 MiB/s wr, 447 op/s 2024-09-11T23:30:48.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:30:47 smithi076 ceph-mon[35296]: pgmap v387: 97 pgs: 97 active+clean; 2.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.8 MiB/s wr, 447 op/s 2024-09-11T23:30:50.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:30:49 smithi136 ceph-mon[35121]: pgmap v388: 97 pgs: 97 active+clean; 2.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.4 MiB/s wr, 375 op/s 2024-09-11T23:30:50.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:30:49 smithi136 ceph-mon[35121]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:30:50.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:30:49 smithi026 ceph-mon[31172]: pgmap v388: 97 pgs: 97 active+clean; 2.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.4 MiB/s wr, 375 op/s 2024-09-11T23:30:50.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:30:49 smithi026 ceph-mon[31172]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:30:50.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:30:49 smithi076 ceph-mon[35296]: pgmap v388: 97 pgs: 97 active+clean; 2.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.4 MiB/s wr, 375 op/s 2024-09-11T23:30:50.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:30:49 smithi076 ceph-mon[35296]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:30:52.060 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.060+0000 7fd947250640 1 -- 172.21.15.26:0/593651383 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd940107ad0 msgr2=0x7fd940107eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:30:52.060 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.060+0000 7fd947250640 1 --2- 172.21.15.26:0/593651383 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd940107ad0 0x7fd940107eb0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7fd934009930 tx=0x7fd93402a440 comp rx=0 tx=0).stop 2024-09-11T23:30:52.060 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.060+0000 7fd947250640 1 -- 172.21.15.26:0/593651383 shutdown_connections 2024-09-11T23:30:52.061 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.060+0000 7fd947250640 1 --2- 172.21.15.26:0/593651383 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd940101ac0 0x7fd940101f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:52.061 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.060+0000 7fd947250640 1 --2- 172.21.15.26:0/593651383 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd940102460 0x7fd94010e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:52.061 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.060+0000 7fd947250640 1 --2- 172.21.15.26:0/593651383 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd940107ad0 0x7fd940107eb0 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:52.061 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.061+0000 7fd947250640 1 -- 172.21.15.26:0/593651383 >> 172.21.15.26:0/593651383 conn(0x7fd9400fd7e0 msgr2=0x7fd9400ffc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:30:52.061 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.061+0000 7fd947250640 1 -- 172.21.15.26:0/593651383 shutdown_connections 2024-09-11T23:30:52.061 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.061+0000 7fd947250640 1 -- 172.21.15.26:0/593651383 wait complete. 2024-09-11T23:30:52.062 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.062+0000 7fd947250640 1 Processor -- start 2024-09-11T23:30:52.062 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.062+0000 7fd947250640 1 -- start start 2024-09-11T23:30:52.062 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.062+0000 7fd947250640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd940101ac0 0x7fd940075a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:52.062 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.062+0000 7fd947250640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd940102460 0x7fd940075f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:52.062 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.062+0000 7fd947250640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd940107ad0 0x7fd94006ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:52.062 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.062+0000 7fd947250640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd940111f80 con 0x7fd940102460 2024-09-11T23:30:52.062 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.062+0000 7fd947250640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fd940111e00 con 0x7fd940101ac0 2024-09-11T23:30:52.062 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.062+0000 7fd947250640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd940112100 con 0x7fd940107ad0 2024-09-11T23:30:52.063 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.062+0000 7fd9457ed640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd940107ad0 0x7fd94006ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:52.063 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.062+0000 7fd9457ed640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd940107ad0 0x7fd94006ca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:40390/0 (socket says 172.21.15.26:40390) 2024-09-11T23:30:52.063 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.062+0000 7fd9457ed640 1 -- 172.21.15.26:0/25837294 learned_addr learned my addr 172.21.15.26:0/25837294 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:30:52.063 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.063+0000 7fd944fec640 1 --2- 172.21.15.26:0/25837294 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd940101ac0 0x7fd940075a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:52.063 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.063+0000 7fd9457ed640 1 -- 172.21.15.26:0/25837294 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd940101ac0 msgr2=0x7fd940075a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:30:52.063 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.063+0000 7fd9457ed640 1 --2- 172.21.15.26:0/25837294 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd940101ac0 0x7fd940075a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:52.063 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.063+0000 7fd9457ed640 1 -- 172.21.15.26:0/25837294 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd940102460 msgr2=0x7fd940075f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:30:52.063 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.063+0000 7fd9457ed640 1 --2- 172.21.15.26:0/25837294 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd940102460 0x7fd940075f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:52.064 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.063+0000 7fd9457ed640 1 -- 172.21.15.26:0/25837294 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd94006d360 con 0x7fd940107ad0 2024-09-11T23:30:52.064 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.063+0000 7fd9457ed640 1 --2- 172.21.15.26:0/25837294 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd940107ad0 0x7fd94006ca50 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fd93800b990 tx=0x7fd93800be50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:30:52.064 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.064+0000 7fd93dffb640 1 -- 172.21.15.26:0/25837294 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd938019070 con 0x7fd940107ad0 2024-09-11T23:30:52.064 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.064+0000 7fd93dffb640 1 -- 172.21.15.26:0/25837294 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd93800cba0 con 0x7fd940107ad0 2024-09-11T23:30:52.064 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.064+0000 7fd93dffb640 1 -- 172.21.15.26:0/25837294 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9380076a0 con 0x7fd940107ad0 2024-09-11T23:30:52.065 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.065+0000 7fd947250640 1 -- 172.21.15.26:0/25837294 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd94006d5f0 con 0x7fd940107ad0 2024-09-11T23:30:52.065 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.065+0000 7fd947250640 1 -- 172.21.15.26:0/25837294 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd940071660 con 0x7fd940107ad0 2024-09-11T23:30:52.068 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.068+0000 7fd93dffb640 1 -- 172.21.15.26:0/25837294 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fd93801c070 con 0x7fd940107ad0 2024-09-11T23:30:52.068 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.068+0000 7fd949fe2740 1 Processor -- start 2024-09-11T23:30:52.068 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.068+0000 7fd949fe2740 1 -- start start 2024-09-11T23:30:52.068 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.069+0000 7fd949fe2740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5628eb469af0 0x5628eb469ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:52.069 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.069+0000 7fd949fe2740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5628eb46a410 0x5628eb474770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:52.069 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.069+0000 7fd949fe2740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5628eb474cb0 0x5628eb4770a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:52.069 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.069+0000 7fd949fe2740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5628eb3ce2f0 con 0x5628eb474cb0 2024-09-11T23:30:52.069 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.069+0000 7fd949fe2740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5628eb376270 con 0x5628eb46a410 2024-09-11T23:30:52.069 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.069+0000 7fd949fe2740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5628eb3ca2b0 con 0x5628eb469af0 2024-09-11T23:30:52.070 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.069+0000 7fd944fec640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5628eb46a410 0x5628eb474770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:52.071 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.069+0000 7fd944fec640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5628eb46a410 0x5628eb474770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:45052/0 (socket says 172.21.15.26:45052) 2024-09-11T23:30:52.071 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.069+0000 7fd944fec640 1 -- 172.21.15.26:0/328539782 learned_addr learned my addr 172.21.15.26:0/328539782 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:30:52.071 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.069+0000 7fd93dffb640 1 --2- 172.21.15.26:0/25837294 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fd9300a2f00 0x7fd9300a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:52.071 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.069+0000 7fd93dffb640 1 -- 172.21.15.26:0/25837294 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8542+0+0 (secure 0 0 0) 0x7fd9380ce8e0 con 0x7fd940107ad0 2024-09-11T23:30:52.071 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.070+0000 7fd93ffff640 1 --2- 172.21.15.26:0/328539782 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5628eb469af0 0x5628eb469ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:52.071 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.070+0000 7fd944fec640 1 -- 172.21.15.26:0/328539782 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5628eb469af0 msgr2=0x5628eb469ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:30:52.071 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.070+0000 7fd944fec640 1 --2- 172.21.15.26:0/328539782 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5628eb469af0 0x5628eb469ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:52.071 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.070+0000 7fd944fec640 1 -- 172.21.15.26:0/328539782 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5628eb474cb0 msgr2=0x5628eb4770a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:30:52.071 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.070+0000 7fd944fec640 1 --2- 172.21.15.26:0/328539782 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5628eb474cb0 0x5628eb4770a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:52.071 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.070+0000 7fd944fec640 1 -- 172.21.15.26:0/328539782 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5628eb375340 con 0x5628eb46a410 2024-09-11T23:30:52.071 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.070+0000 7fd93ffff640 1 --2- 172.21.15.26:0/328539782 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5628eb469af0 0x5628eb469ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:30:52.071 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.070+0000 7fd9457ed640 1 --2- 172.21.15.26:0/25837294 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fd9300a2f00 0x7fd9300a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:52.072 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.070+0000 7fd944fec640 1 --2- 172.21.15.26:0/328539782 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5628eb46a410 0x5628eb474770 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fd93402aa30 tx=0x7fd9340040b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=105e10b5ddbe490d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:30:52.072 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.071+0000 7fd9457ed640 1 --2- 172.21.15.26:0/25837294 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fd9300a2f00 0x7fd9300a53c0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fd93809ae90 tx=0x7fd93800c610 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:30:52.072 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.071+0000 7fd92affd640 1 -- 172.21.15.26:0/328539782 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd934035030 con 0x5628eb46a410 2024-09-11T23:30:52.072 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.071+0000 7fd92affd640 1 -- 172.21.15.26:0/328539782 <== mon.2 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd934005520 con 0x5628eb46a410 2024-09-11T23:30:52.072 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.071+0000 7fd92affd640 1 -- 172.21.15.26:0/328539782 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9340035f0 con 0x5628eb46a410 2024-09-11T23:30:52.074 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.072+0000 7fd949fe2740 1 -- 172.21.15.26:0/328539782 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5628eb46a410 msgr2=0x5628eb474770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:30:52.074 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.072+0000 7fd949fe2740 1 --2- 172.21.15.26:0/328539782 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5628eb46a410 0x5628eb474770 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fd93402aa30 tx=0x7fd9340040b0 comp rx=0 tx=0).stop 2024-09-11T23:30:52.074 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.072+0000 7fd949fe2740 1 -- 172.21.15.26:0/328539782 shutdown_connections 2024-09-11T23:30:52.074 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.072+0000 7fd949fe2740 1 --2- 172.21.15.26:0/328539782 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5628eb46a410 0x5628eb474770 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:52.074 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.072+0000 7fd949fe2740 1 --2- 172.21.15.26:0/328539782 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5628eb474cb0 0x5628eb4770a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:52.074 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.072+0000 7fd949fe2740 1 --2- 172.21.15.26:0/328539782 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5628eb469af0 0x5628eb469ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:52.075 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.072+0000 7fd949fe2740 1 -- 172.21.15.26:0/328539782 >> 172.21.15.26:0/328539782 conn(0x5628eb466060 msgr2=0x5628eb466440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:30:52.075 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.072+0000 7fd949fe2740 1 -- 172.21.15.26:0/328539782 shutdown_connections 2024-09-11T23:30:52.075 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.072+0000 7fd949fe2740 1 -- 172.21.15.26:0/328539782 wait complete. 2024-09-11T23:30:52.075 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.072+0000 7fd949fe2740 1 -- start start 2024-09-11T23:30:52.075 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.073+0000 7fd949fe2740 1 Processor -- start 2024-09-11T23:30:52.075 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.073+0000 7fd949fe2740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5628eb469af0 0x5628eb471c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:52.075 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.074+0000 7fd949fe2740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5628eb46a410 0x5628eb4721c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:52.075 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.074+0000 7fd949fe2740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5628eb474cb0 0x5628eb472700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:52.075 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.074+0000 7fd93ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5628eb469af0 0x5628eb471c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:52.075 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.074+0000 7fd949fe2740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5628eb477880 con 0x5628eb469af0 2024-09-11T23:30:52.075 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.074+0000 7fd949fe2740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5628eb357290 con 0x5628eb474cb0 2024-09-11T23:30:52.075 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.074+0000 7fd949fe2740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5628eb477a00 con 0x5628eb46a410 2024-09-11T23:30:52.075 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.074+0000 7fd93ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5628eb469af0 0x5628eb471c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49624/0 (socket says 172.21.15.26:49624) 2024-09-11T23:30:52.076 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.074+0000 7fd93ffff640 1 -- 172.21.15.26:0/3228460535 learned_addr learned my addr 172.21.15.26:0/3228460535 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:30:52.076 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.074+0000 7fd949fe2740 10 client.0 _ll_register_callbacks cb 0x5628eb388140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:30:52.076 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.074+0000 7fd944fec640 1 --2- 172.21.15.26:0/3228460535 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5628eb46a410 0x5628eb4721c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:52.076 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.074+0000 7fd93ffff640 1 -- 172.21.15.26:0/3228460535 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5628eb46a410 msgr2=0x5628eb4721c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:30:52.076 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.074+0000 7fd93ffff640 1 --2- 172.21.15.26:0/3228460535 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5628eb46a410 0x5628eb4721c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:52.076 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.074+0000 7fd93ffff640 1 -- 172.21.15.26:0/3228460535 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5628eb474cb0 msgr2=0x5628eb472700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:30:52.076 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.074+0000 7fd93ffff640 1 --2- 172.21.15.26:0/3228460535 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5628eb474cb0 0x5628eb472700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:52.076 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.074+0000 7fd93ffff640 1 -- 172.21.15.26:0/3228460535 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5628eb0e1210 con 0x5628eb469af0 2024-09-11T23:30:52.076 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.074+0000 7fd93ffff640 1 --2- 172.21.15.26:0/3228460535 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5628eb469af0 0x5628eb471c80 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7fd94006e240 tx=0x7fd92c00fc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1f1af6a9ca3fe98 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:30:52.076 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.074+0000 7fd91ffff640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:30:52.078 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.075+0000 7fd91ffff640 1 -- 172.21.15.26:0/3228460535 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd92c00d9d0 con 0x5628eb469af0 2024-09-11T23:30:52.078 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.075+0000 7fd91ffff640 1 -- 172.21.15.26:0/3228460535 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8542+0+0 (secure 0 0 0) 0x7fd92c00db70 con 0x5628eb469af0 2024-09-11T23:30:52.078 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.075+0000 7fd949fe2740 1 -- 172.21.15.26:0/3228460535 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd934005520 con 0x5628eb469af0 2024-09-11T23:30:52.078 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.075+0000 7fd91ffff640 1 -- 172.21.15.26:0/3228460535 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7fd92c00dd70 con 0x5628eb469af0 2024-09-11T23:30:52.078 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.075+0000 7fd949fe2740 10 client.17232 fetch_fsmap learned FSMap version 69 2024-09-11T23:30:52.078 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.076+0000 7fd949fe2740 1 -- 172.21.15.26:0/3228460535 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) -- 0x5628eb1ad620 con 0x5628eb469af0 2024-09-11T23:30:52.078 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.076+0000 7fd91ffff640 1 -- 172.21.15.26:0/3228460535 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd92c005020 con 0x5628eb469af0 2024-09-11T23:30:52.078 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.077+0000 7fd91ffff640 1 -- 172.21.15.26:0/3228460535 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 69) ==== 8438+0+0 (secure 0 0 0) 0x7fd92c0048c0 con 0x5628eb469af0 2024-09-11T23:30:52.078 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.077+0000 7fd949fe2740 10 client.17232 fetch_fsmap finished waiting for FSMap version 69 2024-09-11T23:30:52.078 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.077+0000 7fd949fe2740 10 client.17232 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:30:52.079 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.077+0000 7fd949fe2740 20 client.17232 populate_metadata read hostname 'smithi026' 2024-09-11T23:30:52.079 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.077+0000 7fd949fe2740 1 --2- 172.21.15.26:0/3228460535 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5628eb484d70 0x5628eb4871b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:52.079 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.077+0000 7fd949fe2740 4 client.17232 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:30:52.079 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.077+0000 7fd949fe2740 1 -- 172.21.15.26:0/3228460535 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fd934005520 con 0x5628eb484d70 2024-09-11T23:30:52.079 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.077+0000 7fd944fec640 1 --2- 172.21.15.26:0/3228460535 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5628eb484d70 0x5628eb4871b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:52.079 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.078+0000 7fd91ffff640 10 client.17232 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:30:52.079 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.078+0000 7fd944fec640 1 --2- 172.21.15.26:0/3228460535 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5628eb484d70 0x5628eb4871b0 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81ca51cd0c59dcef server_cookie=7587ef9e88eeb74 in_seq=0 out_seq=0 2024-09-11T23:30:52.079 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.079+0000 7fd91ffff640 1 -- 172.21.15.26:0/3228460535 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7fd934005520 con 0x5628eb484d70 2024-09-11T23:30:52.080 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.079+0000 7fd91ffff640 10 client.17232 handle_command_reply: tid=0 2024-09-11T23:30:52.080 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.079+0000 7fd949fe2740 1 client.17232 shutdown 2024-09-11T23:30:52.080 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.079+0000 7fd949fe2740 1 -- 172.21.15.26:0/3228460535 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5628eb469af0 msgr2=0x5628eb471c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:30:52.080 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.079+0000 7fd949fe2740 1 --2- 172.21.15.26:0/3228460535 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5628eb469af0 0x5628eb471c80 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7fd94006e240 tx=0x7fd92c00fc10 comp rx=0 tx=0).stop 2024-09-11T23:30:52.080 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.079+0000 7fd949fe2740 1 -- 172.21.15.26:0/3228460535 shutdown_connections 2024-09-11T23:30:52.080 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.079+0000 7fd949fe2740 1 --2- 172.21.15.26:0/3228460535 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5628eb474cb0 0x5628eb472700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:52.080 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.079+0000 7fd949fe2740 1 --2- 172.21.15.26:0/3228460535 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5628eb484d70 0x5628eb4871b0 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:52.080 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.079+0000 7fd949fe2740 1 --2- 172.21.15.26:0/3228460535 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5628eb46a410 0x5628eb4721c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:52.080 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.079+0000 7fd949fe2740 1 --2- 172.21.15.26:0/3228460535 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5628eb469af0 0x5628eb471c80 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:52.080 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.079+0000 7fd91ffff640 0 client.17232 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:30:52.080 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.079+0000 7fd949fe2740 1 -- 172.21.15.26:0/3228460535 >> 172.21.15.26:0/3228460535 conn(0x5628eb466060 msgr2=0x5628eb46bd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:30:52.080 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.080+0000 7fd949fe2740 1 -- 172.21.15.26:0/3228460535 shutdown_connections 2024-09-11T23:30:52.080 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.080+0000 7fd949fe2740 1 -- 172.21.15.26:0/3228460535 wait complete. 2024-09-11T23:30:52.081 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.080+0000 7fd949fe2740 20 client.17232 trim_cache size 0 max 16384 2024-09-11T23:30:52.103 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.103+0000 7fd949fe2740 1 Processor -- start 2024-09-11T23:30:52.103 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.103+0000 7fd949fe2740 1 -- start start 2024-09-11T23:30:52.103 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.103+0000 7fd949fe2740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5628eb469af0 0x5628eb472530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:52.103 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.103+0000 7fd949fe2740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5628eb46a410 0x5628eb48cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:52.104 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.103+0000 7fd949fe2740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5628eb474cb0 0x5628eb48d000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:52.104 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.103+0000 7fd949fe2740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd9401afd90 con 0x5628eb474cb0 2024-09-11T23:30:52.104 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.103+0000 7fd949fe2740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fd9401c6be0 con 0x5628eb46a410 2024-09-11T23:30:52.104 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.103+0000 7fd949fe2740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd9401b04f0 con 0x5628eb469af0 2024-09-11T23:30:52.104 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.104+0000 7fd93ffff640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5628eb469af0 0x5628eb472530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:52.104 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.104+0000 7fd93ffff640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5628eb469af0 0x5628eb472530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:40418/0 (socket says 172.21.15.26:40418) 2024-09-11T23:30:52.104 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.104+0000 7fd93ffff640 1 -- 172.21.15.26:0/877342669 learned_addr learned my addr 172.21.15.26:0/877342669 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:30:52.104 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.104+0000 7fd944fec640 1 --2- 172.21.15.26:0/877342669 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5628eb46a410 0x5628eb48cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:52.104 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.104+0000 7fd93ffff640 1 --2- 172.21.15.26:0/877342669 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5628eb474cb0 0x5628eb48d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:52.105 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.104+0000 7fd944fec640 1 -- 172.21.15.26:0/877342669 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5628eb469af0 msgr2=0x5628eb472530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:30:52.105 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.104+0000 7fd944fec640 1 --2- 172.21.15.26:0/877342669 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5628eb469af0 0x5628eb472530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:52.105 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.104+0000 7fd944fec640 1 -- 172.21.15.26:0/877342669 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5628eb474cb0 msgr2=0x5628eb48d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:30:52.105 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.104+0000 7fd944fec640 1 --2- 172.21.15.26:0/877342669 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5628eb474cb0 0x5628eb48d000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:52.105 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.104+0000 7fd944fec640 1 -- 172.21.15.26:0/877342669 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5628eb477410 con 0x5628eb46a410 2024-09-11T23:30:52.105 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.105+0000 7fd944fec640 1 --2- 172.21.15.26:0/877342669 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5628eb46a410 0x5628eb48cac0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fd93402a960 tx=0x7fd934004b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4509ad1ae861e692 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:30:52.105 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.105+0000 7fd92a7fc640 1 -- 172.21.15.26:0/877342669 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd934035070 con 0x5628eb46a410 2024-09-11T23:30:52.105 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.105+0000 7fd92a7fc640 1 -- 172.21.15.26:0/877342669 <== mon.2 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd934005380 con 0x5628eb46a410 2024-09-11T23:30:52.106 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.105+0000 7fd92a7fc640 1 -- 172.21.15.26:0/877342669 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9340318a0 con 0x5628eb46a410 2024-09-11T23:30:52.107 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.106+0000 7fd949fe2740 1 -- 172.21.15.26:0/877342669 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5628eb46a410 msgr2=0x5628eb48cac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:30:52.107 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.106+0000 7fd949fe2740 1 --2- 172.21.15.26:0/877342669 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5628eb46a410 0x5628eb48cac0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fd93402a960 tx=0x7fd934004b90 comp rx=0 tx=0).stop 2024-09-11T23:30:52.107 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.106+0000 7fd949fe2740 1 -- 172.21.15.26:0/877342669 shutdown_connections 2024-09-11T23:30:52.107 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.106+0000 7fd949fe2740 1 --2- 172.21.15.26:0/877342669 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5628eb46a410 0x5628eb48cac0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:52.107 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.106+0000 7fd949fe2740 1 --2- 172.21.15.26:0/877342669 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5628eb474cb0 0x5628eb48d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:52.107 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.106+0000 7fd949fe2740 1 --2- 172.21.15.26:0/877342669 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5628eb469af0 0x5628eb472530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:52.107 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.106+0000 7fd949fe2740 1 -- 172.21.15.26:0/877342669 >> 172.21.15.26:0/877342669 conn(0x5628eb466060 msgr2=0x5628eb473d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:30:52.107 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.106+0000 7fd949fe2740 1 -- 172.21.15.26:0/877342669 shutdown_connections 2024-09-11T23:30:52.107 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.106+0000 7fd949fe2740 1 -- 172.21.15.26:0/877342669 wait complete. 2024-09-11T23:30:52.107 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.107+0000 7fd949fe2740 1 -- start start 2024-09-11T23:30:52.108 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.107+0000 7fd949fe2740 1 Processor -- start 2024-09-11T23:30:52.108 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.107+0000 7fd949fe2740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5628eb469af0 0x5628eb47cc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:52.109 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.108+0000 7fd949fe2740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5628eb46a410 0x5628eb47d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:52.109 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.108+0000 7fd949fe2740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5628eb474cb0 0x5628eb479c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:52.109 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.108+0000 7fd949fe2740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd9401c6450 con 0x5628eb474cb0 2024-09-11T23:30:52.109 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.108+0000 7fd93ffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5628eb469af0 0x5628eb47cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:52.110 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.108+0000 7fd93ffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5628eb469af0 0x5628eb47cc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:45074/0 (socket says 172.21.15.26:45074) 2024-09-11T23:30:52.110 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.108+0000 7fd93ffff640 1 -- 172.21.15.26:0/3331957162 learned_addr learned my addr 172.21.15.26:0/3331957162 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:30:52.110 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.108+0000 7fd944fec640 1 --2- 172.21.15.26:0/3331957162 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5628eb46a410 0x5628eb47d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:52.110 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.109+0000 7fd949fe2740 1 -- 172.21.15.26:0/3331957162 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fd9401c5cf0 con 0x5628eb469af0 2024-09-11T23:30:52.110 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.109+0000 7fd949fe2740 1 -- 172.21.15.26:0/3331957162 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd9401add90 con 0x5628eb46a410 2024-09-11T23:30:52.110 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.109+0000 7fd944fec640 1 -- 172.21.15.26:0/3331957162 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5628eb469af0 msgr2=0x5628eb47cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:30:52.110 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.109+0000 7fd944fec640 1 --2- 172.21.15.26:0/3331957162 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5628eb469af0 0x5628eb47cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:52.110 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.109+0000 7fd944fec640 1 -- 172.21.15.26:0/3331957162 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5628eb474cb0 msgr2=0x5628eb479c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:30:52.110 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.109+0000 7fd944fec640 1 --2- 172.21.15.26:0/3331957162 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5628eb474cb0 0x5628eb479c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:52.110 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.109+0000 7fd944fec640 1 -- 172.21.15.26:0/3331957162 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5628eb47a1a0 con 0x5628eb46a410 2024-09-11T23:30:52.110 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.110+0000 7fd949fe2740 10 client.0 _ll_register_callbacks cb 0x7fd92c009660 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:30:52.110 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.110+0000 7fd949fe2740 1 -- 172.21.15.26:0/3331957162 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd934005380 con 0x5628eb46a410 2024-09-11T23:30:52.112 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.111+0000 7fd944fec640 1 --2- 172.21.15.26:0/3331957162 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5628eb46a410 0x5628eb47d1d0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fd934009850 tx=0x7fd934004ed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f30c4bc28c56392a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:30:52.112 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.111+0000 7fd91ffff640 10 client.26752 ms_handle_connect on v2:172.21.15.136:3300/0 2024-09-11T23:30:52.112 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.111+0000 7fd91ffff640 1 -- 172.21.15.26:0/3331957162 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd93405c060 con 0x5628eb46a410 2024-09-11T23:30:52.112 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.111+0000 7fd91ffff640 1 -- 172.21.15.26:0/3331957162 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8542+0+0 (secure 0 0 0) 0x7fd934031820 con 0x5628eb46a410 2024-09-11T23:30:52.112 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.111+0000 7fd91ffff640 1 -- 172.21.15.26:0/3331957162 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7fd934035030 con 0x5628eb46a410 2024-09-11T23:30:52.112 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.111+0000 7fd949fe2740 10 client.26752 fetch_fsmap learned FSMap version 69 2024-09-11T23:30:52.112 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.111+0000 7fd949fe2740 1 -- 172.21.15.26:0/3331957162 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({fsmap=69}) -- 0x5628eb4f0020 con 0x5628eb46a410 2024-09-11T23:30:52.113 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.112+0000 7fd91ffff640 1 -- 172.21.15.26:0/3331957162 <== mon.1 v2:172.21.15.136:3300/0 4 ==== fsmap(e 69) ==== 8438+0+0 (secure 0 0 0) 0x7fd934031ac0 con 0x5628eb46a410 2024-09-11T23:30:52.113 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.112+0000 7fd949fe2740 10 client.26752 fetch_fsmap finished waiting for FSMap version 69 2024-09-11T23:30:52.113 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.112+0000 7fd949fe2740 10 client.26752 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:30:52.113 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.112+0000 7fd949fe2740 20 client.26752 populate_metadata read hostname 'smithi026' 2024-09-11T23:30:52.113 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.112+0000 7fd949fe2740 1 --2- 172.21.15.26:0/3331957162 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5628eb4f0250 0x5628eb4f2640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:52.113 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.112+0000 7fd949fe2740 4 client.26752 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T23:30:52.113 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.112+0000 7fd949fe2740 1 -- 172.21.15.26:0/3331957162 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5628eb3dba20 con 0x5628eb4f0250 2024-09-11T23:30:52.113 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.112+0000 7fd93ffff640 1 --2- 172.21.15.26:0/3331957162 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5628eb4f0250 0x5628eb4f2640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:52.114 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.113+0000 7fd93ffff640 1 --2- 172.21.15.26:0/3331957162 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5628eb4f0250 0x5628eb4f2640 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=53911f71eb0bc945 server_cookie=29f25a02241db8e7 in_seq=0 out_seq=0 2024-09-11T23:30:52.114 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.113+0000 7fd91ffff640 10 client.26752 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:30:52.137 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.136+0000 7fd91ffff640 1 -- 172.21.15.26:0/3331957162 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5628eb3dba20 con 0x5628eb4f0250 2024-09-11T23:30:52.137 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.136+0000 7fd91ffff640 10 client.26752 handle_command_reply: tid=0 2024-09-11T23:30:52.137 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.136+0000 7fd949fe2740 1 client.26752 shutdown 2024-09-11T23:30:52.137 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.136+0000 7fd949fe2740 1 -- 172.21.15.26:0/3331957162 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5628eb46a410 msgr2=0x5628eb47d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:30:52.137 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.136+0000 7fd949fe2740 1 --2- 172.21.15.26:0/3331957162 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5628eb46a410 0x5628eb47d1d0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fd934009850 tx=0x7fd934004ed0 comp rx=0 tx=0).stop 2024-09-11T23:30:52.137 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.137+0000 7fd949fe2740 1 -- 172.21.15.26:0/3331957162 shutdown_connections 2024-09-11T23:30:52.137 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.137+0000 7fd949fe2740 1 --2- 172.21.15.26:0/3331957162 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5628eb4f0250 0x5628eb4f2640 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:52.137 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.137+0000 7fd949fe2740 1 --2- 172.21.15.26:0/3331957162 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5628eb474cb0 0x5628eb479c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:52.138 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.137+0000 7fd949fe2740 1 --2- 172.21.15.26:0/3331957162 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5628eb46a410 0x5628eb47d1d0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:52.138 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.137+0000 7fd949fe2740 1 --2- 172.21.15.26:0/3331957162 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5628eb469af0 0x5628eb47cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:52.138 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.137+0000 7fd949fe2740 1 -- 172.21.15.26:0/3331957162 >> 172.21.15.26:0/3331957162 conn(0x5628eb466060 msgr2=0x5628eb4649a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:30:52.138 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.137+0000 7fd91ffff640 0 client.26752 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:30:52.138 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.137+0000 7fd949fe2740 1 -- 172.21.15.26:0/3331957162 shutdown_connections 2024-09-11T23:30:52.138 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.137+0000 7fd949fe2740 1 -- 172.21.15.26:0/3331957162 wait complete. 2024-09-11T23:30:52.138 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.137+0000 7fd949fe2740 20 client.26752 trim_cache size 0 max 16384 2024-09-11T23:30:52.138 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-09-11T23:30:52.138 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "message": "", 2024-09-11T23:30:52.138 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-09-11T23:30:52.138 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-09-11T23:30:52.139 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.138+0000 7fd947250640 1 -- 172.21.15.26:0/25837294 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fd9300a2f00 msgr2=0x7fd9300a53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:30:52.139 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.138+0000 7fd947250640 1 --2- 172.21.15.26:0/25837294 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fd9300a2f00 0x7fd9300a53c0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fd93809ae90 tx=0x7fd93800c610 comp rx=0 tx=0).stop 2024-09-11T23:30:52.139 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.138+0000 7fd947250640 1 -- 172.21.15.26:0/25837294 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd940107ad0 msgr2=0x7fd94006ca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:30:52.139 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.138+0000 7fd947250640 1 --2- 172.21.15.26:0/25837294 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd940107ad0 0x7fd94006ca50 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fd93800b990 tx=0x7fd93800be50 comp rx=0 tx=0).stop 2024-09-11T23:30:52.139 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.138+0000 7fd947250640 1 -- 172.21.15.26:0/25837294 shutdown_connections 2024-09-11T23:30:52.139 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.138+0000 7fd947250640 1 --2- 172.21.15.26:0/25837294 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fd9300a2f00 0x7fd9300a53c0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:52.139 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.138+0000 7fd947250640 1 --2- 172.21.15.26:0/25837294 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd940107ad0 0x7fd94006ca50 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:52.139 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.138+0000 7fd947250640 1 --2- 172.21.15.26:0/25837294 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd940102460 0x7fd940075f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:52.139 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.138+0000 7fd947250640 1 --2- 172.21.15.26:0/25837294 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd940101ac0 0x7fd940075a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:52.140 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.138+0000 7fd947250640 1 -- 172.21.15.26:0/25837294 >> 172.21.15.26:0/25837294 conn(0x7fd9400fd7e0 msgr2=0x7fd9400fdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:30:52.140 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.139+0000 7fd947250640 1 -- 172.21.15.26:0/25837294 shutdown_connections 2024-09-11T23:30:52.140 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:30:52.139+0000 7fd947250640 1 -- 172.21.15.26:0/25837294 wait complete. 2024-09-11T23:30:52.238 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:30:51 smithi136 ceph-mon[35121]: pgmap v389: 97 pgs: 97 active+clean; 2.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.2 MiB/s wr, 506 op/s 2024-09-11T23:30:52.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:30:51 smithi026 ceph-mon[31172]: pgmap v389: 97 pgs: 97 active+clean; 2.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.2 MiB/s wr, 506 op/s 2024-09-11T23:30:52.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:30:51 smithi076 ceph-mon[35296]: pgmap v389: 97 pgs: 97 active+clean; 2.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.2 MiB/s wr, 506 op/s 2024-09-11T23:30:54.238 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:30:53 smithi136 ceph-mon[35121]: pgmap v390: 97 pgs: 97 active+clean; 2.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.4 MiB/s wr, 379 op/s 2024-09-11T23:30:54.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:30:53 smithi026 ceph-mon[31172]: pgmap v390: 97 pgs: 97 active+clean; 2.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.4 MiB/s wr, 379 op/s 2024-09-11T23:30:54.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:30:53 smithi076 ceph-mon[35296]: pgmap v390: 97 pgs: 97 active+clean; 2.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.4 MiB/s wr, 379 op/s 2024-09-11T23:30:56.238 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:30:55 smithi136 ceph-mon[35121]: pgmap v391: 97 pgs: 97 active+clean; 2.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.0 MiB/s wr, 497 op/s 2024-09-11T23:30:56.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:30:55 smithi026 ceph-mon[31172]: pgmap v391: 97 pgs: 97 active+clean; 2.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.0 MiB/s wr, 497 op/s 2024-09-11T23:30:56.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:30:55 smithi076 ceph-mon[35296]: pgmap v391: 97 pgs: 97 active+clean; 2.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.0 MiB/s wr, 497 op/s 2024-09-11T23:30:57.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:30:56 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:30:57.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:30:56 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:30:57.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:30:56 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:30:58.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:30:57 smithi136 ceph-mon[35121]: pgmap v392: 97 pgs: 97 active+clean; 2.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.8 MiB/s wr, 476 op/s 2024-09-11T23:30:58.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:30:57 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:30:58.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:30:57 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:30:58.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:30:57 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:30:58.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:30:57 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:30:58.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:30:57 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:30:58.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:30:57 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:30:58.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:30:57 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:30:58.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:30:57 smithi026 ceph-mon[31172]: pgmap v392: 97 pgs: 97 active+clean; 2.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.8 MiB/s wr, 476 op/s 2024-09-11T23:30:58.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:30:57 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:30:58.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:30:57 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:30:58.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:30:57 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:30:58.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:30:57 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:30:58.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:30:57 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:30:58.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:30:57 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:30:58.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:30:57 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:30:58.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:30:57 smithi076 ceph-mon[35296]: pgmap v392: 97 pgs: 97 active+clean; 2.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.8 MiB/s wr, 476 op/s 2024-09-11T23:30:58.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:30:57 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:30:58.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:30:57 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:30:58.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:30:57 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:30:58.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:30:57 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:30:58.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:30:57 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:30:58.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:30:57 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:30:58.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:30:57 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:31:00.238 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:30:59 smithi136 ceph-mon[35121]: pgmap v393: 97 pgs: 97 active+clean; 2.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.5 MiB/s wr, 413 op/s 2024-09-11T23:31:00.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:30:59 smithi026 ceph-mon[31172]: pgmap v393: 97 pgs: 97 active+clean; 2.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.5 MiB/s wr, 413 op/s 2024-09-11T23:31:00.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:30:59 smithi076 ceph-mon[35296]: pgmap v393: 97 pgs: 97 active+clean; 2.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.5 MiB/s wr, 413 op/s 2024-09-11T23:31:02.238 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:31:01 smithi136 ceph-mon[35121]: pgmap v394: 97 pgs: 97 active+clean; 2.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.6 MiB/s wr, 587 op/s 2024-09-11T23:31:02.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:31:01 smithi026 ceph-mon[31172]: pgmap v394: 97 pgs: 97 active+clean; 2.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.6 MiB/s wr, 587 op/s 2024-09-11T23:31:02.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:31:01 smithi076 ceph-mon[35296]: pgmap v394: 97 pgs: 97 active+clean; 2.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.6 MiB/s wr, 587 op/s 2024-09-11T23:31:04.238 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:31:03 smithi136 ceph-mon[35121]: pgmap v395: 97 pgs: 97 active+clean; 2.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.7 MiB/s wr, 468 op/s 2024-09-11T23:31:04.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:31:03 smithi026 ceph-mon[31172]: pgmap v395: 97 pgs: 97 active+clean; 2.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.7 MiB/s wr, 468 op/s 2024-09-11T23:31:04.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:31:03 smithi076 ceph-mon[35296]: pgmap v395: 97 pgs: 97 active+clean; 2.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.7 MiB/s wr, 468 op/s 2024-09-11T23:31:05.238 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:31:04 smithi136 ceph-mon[35121]: osdmap e99: 12 total, 12 up, 12 in 2024-09-11T23:31:05.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:31:04 smithi026 ceph-mon[31172]: osdmap e99: 12 total, 12 up, 12 in 2024-09-11T23:31:05.298 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.298+0000 7fdb21986640 1 -- 172.21.15.26:0/1863542701 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdb1c10ea10 msgr2=0x7fdb1c115430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:31:05.298 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.298+0000 7fdb21986640 1 --2- 172.21.15.26:0/1863542701 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdb1c10ea10 0x7fdb1c115430 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fdb14009be0 tx=0x7fdb1402a6c0 comp rx=0 tx=0).stop 2024-09-11T23:31:05.299 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.298+0000 7fdb21986640 1 -- 172.21.15.26:0/1863542701 shutdown_connections 2024-09-11T23:31:05.299 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.298+0000 7fdb21986640 1 --2- 172.21.15.26:0/1863542701 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdb1c10ea10 0x7fdb1c115430 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:05.299 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.298+0000 7fdb21986640 1 --2- 172.21.15.26:0/1863542701 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb1c109e10 0x7fdb1c10e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:05.299 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.298+0000 7fdb21986640 1 --2- 172.21.15.26:0/1863542701 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdb1c109460 0x7fdb1c109840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:05.299 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.298+0000 7fdb21986640 1 -- 172.21.15.26:0/1863542701 >> 172.21.15.26:0/1863542701 conn(0x7fdb1c06d210 msgr2=0x7fdb1c06d620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:31:05.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:31:04 smithi076 ceph-mon[35296]: osdmap e99: 12 total, 12 up, 12 in 2024-09-11T23:31:05.303 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.303+0000 7fdb21986640 1 -- 172.21.15.26:0/1863542701 shutdown_connections 2024-09-11T23:31:05.304 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.304+0000 7fdb21986640 1 -- 172.21.15.26:0/1863542701 wait complete. 2024-09-11T23:31:05.304 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.305+0000 7fdb21986640 1 Processor -- start 2024-09-11T23:31:05.307 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.305+0000 7fdb21986640 1 -- start start 2024-09-11T23:31:05.307 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.306+0000 7fdb21986640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb1c109460 0x7fdb1c1a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:05.307 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.306+0000 7fdb21986640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdb1c10ea10 0x7fdb1c1a5c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:05.307 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.306+0000 7fdb21986640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdb1c19e910 0x7fdb1c19ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:05.307 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.306+0000 7fdb21986640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fdb1c118500 con 0x7fdb1c109460 2024-09-11T23:31:05.307 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.306+0000 7fdb21986640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fdb1c118380 con 0x7fdb1c19e910 2024-09-11T23:31:05.308 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.306+0000 7fdb21986640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fdb1c118680 con 0x7fdb1c10ea10 2024-09-11T23:31:05.308 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.306+0000 7fdb13fff640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdb1c10ea10 0x7fdb1c1a5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:05.308 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.306+0000 7fdb13fff640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdb1c10ea10 0x7fdb1c1a5c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:37646/0 (socket says 172.21.15.26:37646) 2024-09-11T23:31:05.308 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.306+0000 7fdb13fff640 1 -- 172.21.15.26:0/4041588747 learned_addr learned my addr 172.21.15.26:0/4041588747 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:31:05.308 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.306+0000 7fdb1b7fe640 1 --2- 172.21.15.26:0/4041588747 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdb1c19e910 0x7fdb1c19ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:05.308 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.307+0000 7fdb1affd640 1 --2- 172.21.15.26:0/4041588747 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb1c109460 0x7fdb1c1a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:05.308 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.307+0000 7fdb13fff640 1 -- 172.21.15.26:0/4041588747 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdb1c19e910 msgr2=0x7fdb1c19ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:31:05.308 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.307+0000 7fdb13fff640 1 --2- 172.21.15.26:0/4041588747 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdb1c19e910 0x7fdb1c19ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:05.308 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.307+0000 7fdb13fff640 1 -- 172.21.15.26:0/4041588747 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb1c109460 msgr2=0x7fdb1c1a5730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:31:05.308 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.307+0000 7fdb13fff640 1 --2- 172.21.15.26:0/4041588747 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb1c109460 0x7fdb1c1a5730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:05.308 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.307+0000 7fdb13fff640 1 -- 172.21.15.26:0/4041588747 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb1c1a3250 con 0x7fdb1c10ea10 2024-09-11T23:31:05.308 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.307+0000 7fdb13fff640 1 --2- 172.21.15.26:0/4041588747 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdb1c10ea10 0x7fdb1c1a5c70 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fdb0c0095f0 tx=0x7fdb0c009ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:31:05.309 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.308+0000 7fdb20984640 1 -- 172.21.15.26:0/4041588747 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb0c018030 con 0x7fdb1c10ea10 2024-09-11T23:31:05.309 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.308+0000 7fdb20984640 1 -- 172.21.15.26:0/4041588747 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb0c005020 con 0x7fdb1c10ea10 2024-09-11T23:31:05.309 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.308+0000 7fdb20984640 1 -- 172.21.15.26:0/4041588747 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb0c004c40 con 0x7fdb1c10ea10 2024-09-11T23:31:05.310 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.308+0000 7fdb21986640 1 -- 172.21.15.26:0/4041588747 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb1c1a3540 con 0x7fdb1c10ea10 2024-09-11T23:31:05.310 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.308+0000 7fdb21986640 1 -- 172.21.15.26:0/4041588747 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb1c1a39e0 con 0x7fdb1c10ea10 2024-09-11T23:31:05.313 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.313+0000 7fdb247fa740 1 Processor -- start 2024-09-11T23:31:05.314 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.314+0000 7fdb20984640 1 -- 172.21.15.26:0/4041588747 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fdb0c0051d0 con 0x7fdb1c10ea10 2024-09-11T23:31:05.314 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.314+0000 7fdb247fa740 1 -- start start 2024-09-11T23:31:05.314 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.315+0000 7fdb247fa740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f715347af0 0x55f715347ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:05.314 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.315+0000 7fdb247fa740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f715348410 0x55f715352770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:05.315 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.315+0000 7fdb247fa740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f715352cb0 0x55f7153550a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:05.319 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.315+0000 7fdb247fa740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55f7152ac2f0 con 0x55f715348410 2024-09-11T23:31:05.322 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.315+0000 7fdb247fa740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55f715254270 con 0x55f715347af0 2024-09-11T23:31:05.322 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.315+0000 7fdb247fa740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55f7152a82b0 con 0x55f715352cb0 2024-09-11T23:31:05.322 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.316+0000 7fdb1b7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f715347af0 0x55f715347ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:05.322 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.317+0000 7fdb1b7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f715347af0 0x55f715347ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:38822/0 (socket says 172.21.15.26:38822) 2024-09-11T23:31:05.322 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.317+0000 7fdb1b7fe640 1 -- 172.21.15.26:0/2209709059 learned_addr learned my addr 172.21.15.26:0/2209709059 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:31:05.322 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.318+0000 7fdb1affd640 1 --2- 172.21.15.26:0/2209709059 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f715348410 0x55f715352770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:05.323 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.323+0000 7fdb1affd640 1 -- 172.21.15.26:0/2209709059 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f715352cb0 msgr2=0x55f7153550a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:31:05.323 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.324+0000 7fdb13fff640 1 --2- 172.21.15.26:0/2209709059 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f715352cb0 0x55f7153550a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:05.324 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.324+0000 7fdb20984640 1 --2- 172.21.15.26:0/4041588747 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fdb080a2df0 0x7fdb080a52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:05.324 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.325+0000 7fdb1affd640 1 --2- 172.21.15.26:0/2209709059 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f715352cb0 0x55f7153550a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:05.324 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.325+0000 7fdb20984640 1 -- 172.21.15.26:0/4041588747 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8586+0+0 (secure 0 0 0) 0x7fdb0c0ceba0 con 0x7fdb1c10ea10 2024-09-11T23:31:05.324 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.325+0000 7fdb1affd640 1 -- 172.21.15.26:0/2209709059 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f715347af0 msgr2=0x55f715347ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:31:05.325 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.325+0000 7fdb1affd640 1 --2- 172.21.15.26:0/2209709059 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f715347af0 0x55f715347ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:05.325 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.325+0000 7fdb1affd640 1 -- 172.21.15.26:0/2209709059 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f715253340 con 0x55f715348410 2024-09-11T23:31:05.325 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.325+0000 7fdb1b7fe640 1 --2- 172.21.15.26:0/2209709059 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f715347af0 0x55f715347ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:31:05.325 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.325+0000 7fdb1b7fe640 1 --2- 172.21.15.26:0/4041588747 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fdb080a2df0 0x7fdb080a52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:05.325 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.326+0000 7fdb1affd640 1 --2- 172.21.15.26:0/2209709059 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f715348410 0x55f715352770 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fdb0400bbe0 tx=0x7fdb0400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c69f246cd9a76916 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:31:05.325 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.326+0000 7fdb13fff640 1 --2- 172.21.15.26:0/2209709059 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f715352cb0 0x55f7153550a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:31:05.326 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.326+0000 7fdb11ffb640 1 -- 172.21.15.26:0/2209709059 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb04015070 con 0x55f715348410 2024-09-11T23:31:05.326 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.326+0000 7fdb11ffb640 1 -- 172.21.15.26:0/2209709059 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb040035f0 con 0x55f715348410 2024-09-11T23:31:05.326 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.326+0000 7fdb1b7fe640 1 --2- 172.21.15.26:0/4041588747 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fdb080a2df0 0x7fdb080a52b0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fdb14009be0 tx=0x7fdb14009500 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:31:05.327 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.327+0000 7fdb11ffb640 1 -- 172.21.15.26:0/2209709059 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb040116a0 con 0x55f715348410 2024-09-11T23:31:05.327 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.328+0000 7fdb247fa740 1 -- 172.21.15.26:0/2209709059 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f715348410 msgr2=0x55f715352770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:31:05.327 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.328+0000 7fdb247fa740 1 --2- 172.21.15.26:0/2209709059 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f715348410 0x55f715352770 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fdb0400bbe0 tx=0x7fdb0400f730 comp rx=0 tx=0).stop 2024-09-11T23:31:05.328 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.328+0000 7fdb247fa740 1 -- 172.21.15.26:0/2209709059 shutdown_connections 2024-09-11T23:31:05.328 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.328+0000 7fdb247fa740 1 --2- 172.21.15.26:0/2209709059 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f715352cb0 0x55f7153550a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:05.328 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.328+0000 7fdb247fa740 1 --2- 172.21.15.26:0/2209709059 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f715348410 0x55f715352770 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:05.328 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.328+0000 7fdb247fa740 1 --2- 172.21.15.26:0/2209709059 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f715347af0 0x55f715347ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:05.328 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.328+0000 7fdb247fa740 1 -- 172.21.15.26:0/2209709059 >> 172.21.15.26:0/2209709059 conn(0x55f715344060 msgr2=0x55f715344440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:31:05.328 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.328+0000 7fdb247fa740 1 -- 172.21.15.26:0/2209709059 shutdown_connections 2024-09-11T23:31:05.329 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.329+0000 7fdb247fa740 1 -- 172.21.15.26:0/2209709059 wait complete. 2024-09-11T23:31:05.329 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.329+0000 7fdb247fa740 1 -- start start 2024-09-11T23:31:05.329 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.329+0000 7fdb247fa740 1 Processor -- start 2024-09-11T23:31:05.329 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.330+0000 7fdb247fa740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f715347af0 0x55f71534fb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:05.330 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.330+0000 7fdb247fa740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f715348410 0x55f715350050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:05.330 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.330+0000 7fdb247fa740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f715352cb0 0x55f715350590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:05.330 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.330+0000 7fdb247fa740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55f7153558d0 con 0x55f715347af0 2024-09-11T23:31:05.330 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.330+0000 7fdb247fa740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55f715235290 con 0x55f715352cb0 2024-09-11T23:31:05.330 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.330+0000 7fdb247fa740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55f715355a50 con 0x55f715348410 2024-09-11T23:31:05.330 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.330+0000 7fdb247fa740 10 client.0 _ll_register_callbacks cb 0x55f715266140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:31:05.330 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.330+0000 7fdb13fff640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f715348410 0x55f715350050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:05.330 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.330+0000 7fdb1affd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f715347af0 0x55f71534fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:05.330 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.330+0000 7fdb13fff640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f715348410 0x55f715350050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:37660/0 (socket says 172.21.15.26:37660) 2024-09-11T23:31:05.330 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.330+0000 7fdb13fff640 1 -- 172.21.15.26:0/367907207 learned_addr learned my addr 172.21.15.26:0/367907207 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:31:05.331 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.331+0000 7fdb13fff640 1 -- 172.21.15.26:0/367907207 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f715352cb0 msgr2=0x55f715350590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:31:05.331 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.331+0000 7fdb13fff640 1 --2- 172.21.15.26:0/367907207 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f715352cb0 0x55f715350590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:05.331 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.331+0000 7fdb13fff640 1 -- 172.21.15.26:0/367907207 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f715347af0 msgr2=0x55f71534fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:31:05.331 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.331+0000 7fdb13fff640 1 --2- 172.21.15.26:0/367907207 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f715347af0 0x55f71534fb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:05.331 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.331+0000 7fdb13fff640 1 -- 172.21.15.26:0/367907207 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f714fbf210 con 0x55f715348410 2024-09-11T23:31:05.331 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.331+0000 7fdb13fff640 1 --2- 172.21.15.26:0/367907207 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f715348410 0x55f715350050 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fdb0c0cc310 tx=0x7fdb0c09ae50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5e1ab70edf66fe76 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:31:05.331 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.331+0000 7fdafaffd640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-09-11T23:31:05.332 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.331+0000 7fdafaffd640 1 -- 172.21.15.26:0/367907207 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb0c09b780 con 0x55f715348410 2024-09-11T23:31:05.332 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.331+0000 7fdafaffd640 1 -- 172.21.15.26:0/367907207 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8586+0+0 (secure 0 0 0) 0x7fdb0c0976f0 con 0x55f715348410 2024-09-11T23:31:05.332 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.332+0000 7fdb247fa740 1 -- 172.21.15.26:0/367907207 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb040035f0 con 0x55f715348410 2024-09-11T23:31:05.332 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.332+0000 7fdafaffd640 1 -- 172.21.15.26:0/367907207 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7fdb0c004e40 con 0x55f715348410 2024-09-11T23:31:05.333 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.333+0000 7fdb247fa740 10 client.26761 fetch_fsmap learned FSMap version 69 2024-09-11T23:31:05.333 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.333+0000 7fdb247fa740 1 -- 172.21.15.26:0/367907207 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) -- 0x55f71508b620 con 0x55f715348410 2024-09-11T23:31:05.333 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.334+0000 7fdafaffd640 1 -- 172.21.15.26:0/367907207 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb040035f0 con 0x55f715348410 2024-09-11T23:31:05.334 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.334+0000 7fdafaffd640 1 -- 172.21.15.26:0/367907207 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 69) ==== 8438+0+0 (secure 0 0 0) 0x7fdb0c09f080 con 0x55f715348410 2024-09-11T23:31:05.334 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.335+0000 7fdb247fa740 10 client.26761 fetch_fsmap finished waiting for FSMap version 69 2024-09-11T23:31:05.334 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.335+0000 7fdb247fa740 10 client.26761 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:31:05.334 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.335+0000 7fdb247fa740 20 client.26761 populate_metadata read hostname 'smithi026' 2024-09-11T23:31:05.334 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.335+0000 7fdb247fa740 1 --2- 172.21.15.26:0/367907207 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55f715362d00 0x55f7153651a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:05.334 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.335+0000 7fdb247fa740 4 client.26761 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:31:05.334 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.335+0000 7fdb247fa740 1 -- 172.21.15.26:0/367907207 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f7152b9a20 con 0x55f715362d00 2024-09-11T23:31:05.337 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.338+0000 7fdb1affd640 1 --2- 172.21.15.26:0/367907207 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55f715362d00 0x55f7153651a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:05.341 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.342+0000 7fdb1affd640 1 --2- 172.21.15.26:0/367907207 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55f715362d00 0x55f7153651a0 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1839291040c4a5f server_cookie=7abf452b5bb1073a in_seq=0 out_seq=0 2024-09-11T23:31:05.341 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.342+0000 7fdafaffd640 10 client.26761 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:31:05.342 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.343+0000 7fdafaffd640 1 -- 172.21.15.26:0/367907207 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f7152b9a20 con 0x55f715362d00 2024-09-11T23:31:05.342 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.343+0000 7fdafaffd640 10 client.26761 handle_command_reply: tid=0 2024-09-11T23:31:05.343 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.344+0000 7fdb247fa740 1 client.26761 shutdown 2024-09-11T23:31:05.351 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.352+0000 7fdb247fa740 1 -- 172.21.15.26:0/367907207 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f715348410 msgr2=0x55f715350050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:31:05.352 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.352+0000 7fdb247fa740 1 --2- 172.21.15.26:0/367907207 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f715348410 0x55f715350050 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fdb0c0cc310 tx=0x7fdb0c09ae50 comp rx=0 tx=0).stop 2024-09-11T23:31:05.355 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.355+0000 7fdb247fa740 1 -- 172.21.15.26:0/367907207 shutdown_connections 2024-09-11T23:31:05.355 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.355+0000 7fdb247fa740 1 --2- 172.21.15.26:0/367907207 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f715352cb0 0x55f715350590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:05.355 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.355+0000 7fdb247fa740 1 --2- 172.21.15.26:0/367907207 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55f715362d00 0x55f7153651a0 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:05.355 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.355+0000 7fdb247fa740 1 --2- 172.21.15.26:0/367907207 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f715348410 0x55f715350050 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:05.355 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.355+0000 7fdb247fa740 1 --2- 172.21.15.26:0/367907207 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f715347af0 0x55f71534fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:05.355 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.355+0000 7fdb247fa740 1 -- 172.21.15.26:0/367907207 >> 172.21.15.26:0/367907207 conn(0x55f715344060 msgr2=0x55f7153431c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:31:05.355 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.356+0000 7fdafaffd640 0 client.26761 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:31:05.358 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.359+0000 7fdb247fa740 1 -- 172.21.15.26:0/367907207 shutdown_connections 2024-09-11T23:31:05.362 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.362+0000 7fdb247fa740 1 -- 172.21.15.26:0/367907207 wait complete. 2024-09-11T23:31:05.363 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.364+0000 7fdb247fa740 20 client.26761 trim_cache size 0 max 16384 2024-09-11T23:31:05.399 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.396+0000 7fdb247fa740 1 Processor -- start 2024-09-11T23:31:05.399 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.398+0000 7fdb247fa740 1 -- start start 2024-09-11T23:31:05.399 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.399+0000 7fdb247fa740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f715347af0 0x55f7153503c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:05.399 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.399+0000 7fdb247fa740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f715348410 0x55f71536aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:05.400 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.399+0000 7fdb247fa740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f715352cb0 0x55f71536afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:05.400 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.399+0000 7fdb1affd640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f715347af0 0x55f7153503c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:05.400 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.399+0000 7fdb1affd640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f715347af0 0x55f7153503c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:37666/0 (socket says 172.21.15.26:37666) 2024-09-11T23:31:05.400 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.399+0000 7fdb1affd640 1 -- 172.21.15.26:0/2516634245 learned_addr learned my addr 172.21.15.26:0/2516634245 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:31:05.400 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.400+0000 7fdb247fa740 1 -- 172.21.15.26:0/2516634245 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fdafc00e870 con 0x55f715348410 2024-09-11T23:31:05.400 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.400+0000 7fdb1b7fe640 1 --2- 172.21.15.26:0/2516634245 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f715352cb0 0x55f71536afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:05.400 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.400+0000 7fdb247fa740 1 -- 172.21.15.26:0/2516634245 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fdafc010680 con 0x55f715352cb0 2024-09-11T23:31:05.400 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.400+0000 7fdb247fa740 1 -- 172.21.15.26:0/2516634245 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fdafc00efd0 con 0x55f715347af0 2024-09-11T23:31:05.400 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.400+0000 7fdb1affd640 1 -- 172.21.15.26:0/2516634245 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f715352cb0 msgr2=0x55f71536afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:31:05.401 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.400+0000 7fdb1affd640 1 --2- 172.21.15.26:0/2516634245 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f715352cb0 0x55f71536afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:05.401 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.400+0000 7fdb1affd640 1 -- 172.21.15.26:0/2516634245 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f715348410 msgr2=0x55f71536aa70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T23:31:05.401 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.400+0000 7fdb1affd640 1 --2- 172.21.15.26:0/2516634245 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f715348410 0x55f71536aa70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:05.401 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.401+0000 7fdb1affd640 1 -- 172.21.15.26:0/2516634245 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f715344ba0 con 0x55f715347af0 2024-09-11T23:31:05.403 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.404+0000 7fdb1affd640 1 --2- 172.21.15.26:0/2516634245 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f715347af0 0x55f7153503c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fdb0400bb00 tx=0x7fdb0400b4d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=50868a421fa51646 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:31:05.404 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.405+0000 7fdb117fa640 1 -- 172.21.15.26:0/2516634245 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb04015040 con 0x55f715347af0 2024-09-11T23:31:05.404 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.405+0000 7fdb117fa640 1 -- 172.21.15.26:0/2516634245 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb0400be80 con 0x55f715347af0 2024-09-11T23:31:05.405 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.405+0000 7fdb117fa640 1 -- 172.21.15.26:0/2516634245 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb040257e0 con 0x55f715347af0 2024-09-11T23:31:05.406 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.406+0000 7fdb247fa740 1 -- 172.21.15.26:0/2516634245 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f715347af0 msgr2=0x55f7153503c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:31:05.406 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.406+0000 7fdb247fa740 1 --2- 172.21.15.26:0/2516634245 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f715347af0 0x55f7153503c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fdb0400bb00 tx=0x7fdb0400b4d0 comp rx=0 tx=0).stop 2024-09-11T23:31:05.406 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.406+0000 7fdb247fa740 1 -- 172.21.15.26:0/2516634245 shutdown_connections 2024-09-11T23:31:05.406 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.406+0000 7fdb247fa740 1 --2- 172.21.15.26:0/2516634245 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f715352cb0 0x55f71536afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:05.406 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.407+0000 7fdb247fa740 1 --2- 172.21.15.26:0/2516634245 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f715348410 0x55f71536aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:05.406 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.407+0000 7fdb247fa740 1 --2- 172.21.15.26:0/2516634245 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f715347af0 0x55f7153503c0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:05.407 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.407+0000 7fdb247fa740 1 -- 172.21.15.26:0/2516634245 >> 172.21.15.26:0/2516634245 conn(0x55f715344060 msgr2=0x55f715351b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:31:05.407 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.407+0000 7fdb247fa740 1 -- 172.21.15.26:0/2516634245 shutdown_connections 2024-09-11T23:31:05.409 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.409+0000 7fdb247fa740 1 -- 172.21.15.26:0/2516634245 wait complete. 2024-09-11T23:31:05.409 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.410+0000 7fdb247fa740 1 -- start start 2024-09-11T23:31:05.409 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.410+0000 7fdb247fa740 1 Processor -- start 2024-09-11T23:31:05.412 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.411+0000 7fdb247fa740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f715347af0 0x55f71535b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:05.412 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.411+0000 7fdb247fa740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f715348410 0x55f71534c940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:05.413 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.411+0000 7fdb247fa740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f715352cb0 0x55f71534cf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:05.413 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.411+0000 7fdb247fa740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fdafc00fef0 con 0x55f715352cb0 2024-09-11T23:31:05.413 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.411+0000 7fdb247fa740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fdafc00f760 con 0x55f715348410 2024-09-11T23:31:05.413 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.411+0000 7fdb247fa740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fdafc00c870 con 0x55f715347af0 2024-09-11T23:31:05.413 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.411+0000 7fdb13fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f715348410 0x55f71534c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:05.413 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.411+0000 7fdb1b7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f715352cb0 0x55f71534cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:05.413 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.411+0000 7fdb1affd640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f715347af0 0x55f71535b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:05.413 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.411+0000 7fdb13fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f715348410 0x55f71534c940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:38856/0 (socket says 172.21.15.26:38856) 2024-09-11T23:31:05.413 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.411+0000 7fdb13fff640 1 -- 172.21.15.26:0/2686592100 learned_addr learned my addr 172.21.15.26:0/2686592100 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:31:05.414 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.412+0000 7fdb247fa740 10 client.0 _ll_register_callbacks cb 0x7fdb0c097320 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:31:05.415 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.416+0000 7fdb1affd640 1 -- 172.21.15.26:0/2686592100 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f715348410 msgr2=0x55f71534c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:31:05.416 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.416+0000 7fdb1affd640 1 --2- 172.21.15.26:0/2686592100 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f715348410 0x55f71534c940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:05.416 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.416+0000 7fdb1affd640 1 -- 172.21.15.26:0/2686592100 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f715352cb0 msgr2=0x55f71534cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:31:05.416 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.416+0000 7fdb1affd640 1 --2- 172.21.15.26:0/2686592100 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f715352cb0 0x55f71534cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:05.416 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.416+0000 7fdb1affd640 1 -- 172.21.15.26:0/2686592100 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f715347920 con 0x55f715347af0 2024-09-11T23:31:05.422 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.419+0000 7fdb1affd640 1 --2- 172.21.15.26:0/2686592100 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f715347af0 0x55f71535b5f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fdb0400bb00 tx=0x7fdb0400f730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3a13ab4eda59b0b5 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:31:05.422 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.419+0000 7fdafaffd640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-09-11T23:31:05.422 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.419+0000 7fdafaffd640 1 -- 172.21.15.26:0/2686592100 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb04003b40 con 0x55f715347af0 2024-09-11T23:31:05.422 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.419+0000 7fdafaffd640 1 -- 172.21.15.26:0/2686592100 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8586+0+0 (secure 0 0 0) 0x7fdb04003f10 con 0x55f715347af0 2024-09-11T23:31:05.422 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.420+0000 7fdb247fa740 1 -- 172.21.15.26:0/2686592100 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55f7152b3d70 con 0x55f715347af0 2024-09-11T23:31:05.422 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.421+0000 7fdafaffd640 1 -- 172.21.15.26:0/2686592100 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7fdb0403e050 con 0x55f715347af0 2024-09-11T23:31:05.422 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.421+0000 7fdb247fa740 10 client.26767 fetch_fsmap learned FSMap version 69 2024-09-11T23:31:05.422 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.421+0000 7fdb247fa740 1 -- 172.21.15.26:0/2686592100 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) -- 0x55f7153cd9c0 con 0x55f715347af0 2024-09-11T23:31:05.422 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.422+0000 7fdafaffd640 1 -- 172.21.15.26:0/2686592100 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb04005030 con 0x55f715347af0 2024-09-11T23:31:05.423 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.422+0000 7fdafaffd640 1 -- 172.21.15.26:0/2686592100 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 69) ==== 8438+0+0 (secure 0 0 0) 0x7fdb04004b60 con 0x55f715347af0 2024-09-11T23:31:05.423 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.422+0000 7fdb247fa740 10 client.26767 fetch_fsmap finished waiting for FSMap version 69 2024-09-11T23:31:05.423 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.422+0000 7fdb247fa740 10 client.26767 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:31:05.423 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.422+0000 7fdb247fa740 20 client.26767 populate_metadata read hostname 'smithi026' 2024-09-11T23:31:05.423 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.422+0000 7fdb247fa740 1 --2- 172.21.15.26:0/2686592100 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55f7153cde70 0x55f7153d0290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:05.423 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.422+0000 7fdb247fa740 4 client.26767 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T23:31:05.423 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.422+0000 7fdb247fa740 1 -- 172.21.15.26:0/2686592100 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f7152b3d70 con 0x55f7153cde70 2024-09-11T23:31:05.423 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.423+0000 7fdb13fff640 1 --2- 172.21.15.26:0/2686592100 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55f7153cde70 0x55f7153d0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:05.424 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.424+0000 7fdafaffd640 10 client.26767 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:31:05.424 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:05.425+0000 7fdb13fff640 1 --2- 172.21.15.26:0/2686592100 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55f7153cde70 0x55f7153d0290 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bfc461676c9ef050 server_cookie=df3e231053eb09c0 in_seq=0 out_seq=0 2024-09-11T23:31:06.238 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:31:05 smithi136 ceph-mon[35121]: pgmap v397: 97 pgs: 97 active+clean; 2.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.4 MiB/s wr, 460 op/s 2024-09-11T23:31:06.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:31:05 smithi026 ceph-mon[31172]: pgmap v397: 97 pgs: 97 active+clean; 2.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.4 MiB/s wr, 460 op/s 2024-09-11T23:31:06.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:31:05 smithi076 ceph-mon[35296]: pgmap v397: 97 pgs: 97 active+clean; 2.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.4 MiB/s wr, 460 op/s 2024-09-11T23:31:08.238 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:31:07 smithi136 ceph-mon[35121]: pgmap v398: 97 pgs: 19 active+clean+snaptrim_wait, 10 active+clean+snaptrim, 68 active+clean; 2.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.2 MiB/s wr, 445 op/s 2024-09-11T23:31:08.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:31:07 smithi026 ceph-mon[31172]: pgmap v398: 97 pgs: 19 active+clean+snaptrim_wait, 10 active+clean+snaptrim, 68 active+clean; 2.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.2 MiB/s wr, 445 op/s 2024-09-11T23:31:08.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:31:07 smithi076 ceph-mon[35296]: pgmap v398: 97 pgs: 19 active+clean+snaptrim_wait, 10 active+clean+snaptrim, 68 active+clean; 2.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.2 MiB/s wr, 445 op/s 2024-09-11T23:31:09.189 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:09.188+0000 7fdafaffd640 1 -- 172.21.15.26:0/2686592100 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f7152b3d70 con 0x55f7153cde70 2024-09-11T23:31:09.189 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:09.188+0000 7fdafaffd640 10 client.26767 handle_command_reply: tid=0 2024-09-11T23:31:09.189 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:09.188+0000 7fdb247fa740 1 client.26767 shutdown 2024-09-11T23:31:09.189 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:09.188+0000 7fdb247fa740 1 -- 172.21.15.26:0/2686592100 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f715347af0 msgr2=0x55f71535b5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:31:09.189 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:09.188+0000 7fdb247fa740 1 --2- 172.21.15.26:0/2686592100 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f715347af0 0x55f71535b5f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fdb0400bb00 tx=0x7fdb0400f730 comp rx=0 tx=0).stop 2024-09-11T23:31:09.189 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:09.188+0000 7fdb247fa740 1 -- 172.21.15.26:0/2686592100 shutdown_connections 2024-09-11T23:31:09.189 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:09.189+0000 7fdb247fa740 1 --2- 172.21.15.26:0/2686592100 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f715348410 0x55f71534c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:09.189 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:09.189+0000 7fdb247fa740 1 --2- 172.21.15.26:0/2686592100 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55f7153cde70 0x55f7153d0290 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:09.190 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:09.189+0000 7fdafaffd640 0 client.26767 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:31:09.190 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:09.189+0000 7fdb247fa740 1 --2- 172.21.15.26:0/2686592100 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f715352cb0 0x55f71534cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:09.190 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:09.189+0000 7fdb247fa740 1 --2- 172.21.15.26:0/2686592100 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f715347af0 0x55f71535b5f0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:09.190 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:09.189+0000 7fdb247fa740 1 -- 172.21.15.26:0/2686592100 >> 172.21.15.26:0/2686592100 conn(0x55f715344060 msgr2=0x55f7153429a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:31:09.190 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:09.189+0000 7fdb247fa740 1 -- 172.21.15.26:0/2686592100 shutdown_connections 2024-09-11T23:31:09.190 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:09.189+0000 7fdb247fa740 1 -- 172.21.15.26:0/2686592100 wait complete. 2024-09-11T23:31:09.190 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:09.189+0000 7fdb247fa740 20 client.26767 trim_cache size 0 max 16384 2024-09-11T23:31:09.191 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-09-11T23:31:09.191 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "message": "", 2024-09-11T23:31:09.191 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-09-11T23:31:09.191 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-09-11T23:31:09.192 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:09.190+0000 7fdb21986640 1 -- 172.21.15.26:0/4041588747 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fdb080a2df0 msgr2=0x7fdb080a52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:31:09.192 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:09.190+0000 7fdb21986640 1 --2- 172.21.15.26:0/4041588747 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fdb080a2df0 0x7fdb080a52b0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fdb14009be0 tx=0x7fdb14009500 comp rx=0 tx=0).stop 2024-09-11T23:31:09.192 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:09.190+0000 7fdb21986640 1 -- 172.21.15.26:0/4041588747 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdb1c10ea10 msgr2=0x7fdb1c1a5c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:31:09.192 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:09.190+0000 7fdb21986640 1 --2- 172.21.15.26:0/4041588747 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdb1c10ea10 0x7fdb1c1a5c70 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fdb0c0095f0 tx=0x7fdb0c009ab0 comp rx=0 tx=0).stop 2024-09-11T23:31:09.192 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:09.192+0000 7fdb21986640 1 -- 172.21.15.26:0/4041588747 shutdown_connections 2024-09-11T23:31:09.192 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:09.192+0000 7fdb21986640 1 --2- 172.21.15.26:0/4041588747 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fdb080a2df0 0x7fdb080a52b0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:09.192 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:09.192+0000 7fdb21986640 1 --2- 172.21.15.26:0/4041588747 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdb1c19e910 0x7fdb1c19ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:09.192 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:09.192+0000 7fdb21986640 1 --2- 172.21.15.26:0/4041588747 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdb1c10ea10 0x7fdb1c1a5c70 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:09.192 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:09.192+0000 7fdb21986640 1 --2- 172.21.15.26:0/4041588747 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb1c109460 0x7fdb1c1a5730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:09.193 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:09.192+0000 7fdb21986640 1 -- 172.21.15.26:0/4041588747 >> 172.21.15.26:0/4041588747 conn(0x7fdb1c06d210 msgr2=0x7fdb1c0728b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:31:09.193 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:09.192+0000 7fdb21986640 1 -- 172.21.15.26:0/4041588747 shutdown_connections 2024-09-11T23:31:09.193 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:09.192+0000 7fdb21986640 1 -- 172.21.15.26:0/4041588747 wait complete. 2024-09-11T23:31:09.907 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 27.2 2024-09-11T23:31:09.908 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:31:10.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:31:09 smithi136 ceph-mon[35121]: pgmap v399: 97 pgs: 19 active+clean+snaptrim_wait, 10 active+clean+snaptrim, 68 active+clean; 2.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.2 MiB/s wr, 430 op/s 2024-09-11T23:31:10.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:31:09 smithi026 ceph-mon[31172]: pgmap v399: 97 pgs: 19 active+clean+snaptrim_wait, 10 active+clean+snaptrim, 68 active+clean; 2.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.2 MiB/s wr, 430 op/s 2024-09-11T23:31:10.304 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:31:09 smithi076 ceph-mon[35296]: pgmap v399: 97 pgs: 19 active+clean+snaptrim_wait, 10 active+clean+snaptrim, 68 active+clean; 2.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.2 MiB/s wr, 430 op/s 2024-09-11T23:31:10.374 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:31:11.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:11.192+0000 7f75749a0640 1 -- 172.21.15.26:0/2331795588 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f75701169f0 msgr2=0x7f7570118e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:31:11.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:11.192+0000 7f75749a0640 1 --2- 172.21.15.26:0/2331795588 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f75701169f0 0x7f7570118e20 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f7568009be0 tx=0x7f756802a6f0 comp rx=0 tx=0).stop 2024-09-11T23:31:11.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:11.192+0000 7f75749a0640 1 -- 172.21.15.26:0/2331795588 shutdown_connections 2024-09-11T23:31:11.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:11.192+0000 7f75749a0640 1 --2- 172.21.15.26:0/2331795588 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f75701169f0 0x7f7570118e20 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:11.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:11.192+0000 7f75749a0640 1 --2- 172.21.15.26:0/2331795588 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f757006ea30 0x7f757006ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:11.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:11.192+0000 7f75749a0640 1 --2- 172.21.15.26:0/2331795588 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f757010c0a0 0x7f757006e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:11.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:11.192+0000 7f75749a0640 1 -- 172.21.15.26:0/2331795588 >> 172.21.15.26:0/2331795588 conn(0x7f7570073180 msgr2=0x7f7570073590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:31:11.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:11.194+0000 7f75749a0640 1 -- 172.21.15.26:0/2331795588 shutdown_connections 2024-09-11T23:31:11.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:11.194+0000 7f75749a0640 1 -- 172.21.15.26:0/2331795588 wait complete. 2024-09-11T23:31:11.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:11.195+0000 7f75749a0640 1 Processor -- start 2024-09-11T23:31:11.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:11.196+0000 7f75749a0640 1 -- start start 2024-09-11T23:31:11.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:11.196+0000 7f75749a0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f757006ea30 0x7f757010c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:11.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:11.196+0000 7f75749a0640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f757010c0a0 0x7f757010cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:11.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:11.196+0000 7f75749a0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f75701169f0 0x7f757010d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:11.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:11.196+0000 7f75749a0640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f757011c7e0 con 0x7f757006ea30 2024-09-11T23:31:11.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:11.196+0000 7f75749a0640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f757011c660 con 0x7f75701169f0 2024-09-11T23:31:11.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:11.196+0000 7f75749a0640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f757011c960 con 0x7f757010c0a0 2024-09-11T23:31:11.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:11.196+0000 7f756ffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f75701169f0 0x7f757010d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:11.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:11.197+0000 7f756ffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f75701169f0 0x7f757010d060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:46964/0 (socket says 172.21.15.26:46964) 2024-09-11T23:31:11.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:11.197+0000 7f756ffff640 1 -- 172.21.15.26:0/64564143 learned_addr learned my addr 172.21.15.26:0/64564143 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:31:11.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:11.197+0000 7f756f7fe640 1 --2- 172.21.15.26:0/64564143 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f757006ea30 0x7f757010c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:11.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:11.197+0000 7f756effd640 1 --2- 172.21.15.26:0/64564143 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f757010c0a0 0x7f757010cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:11.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:11.197+0000 7f756ffff640 1 -- 172.21.15.26:0/64564143 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f757010c0a0 msgr2=0x7f757010cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:31:11.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:11.197+0000 7f756ffff640 1 --2- 172.21.15.26:0/64564143 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f757010c0a0 0x7f757010cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:11.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:11.197+0000 7f756ffff640 1 -- 172.21.15.26:0/64564143 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f757006ea30 msgr2=0x7f757010c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:31:11.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:11.197+0000 7f756ffff640 1 --2- 172.21.15.26:0/64564143 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f757006ea30 0x7f757010c5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:11.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:11.197+0000 7f756ffff640 1 -- 172.21.15.26:0/64564143 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75701b3400 con 0x7f75701169f0 2024-09-11T23:31:11.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:11.197+0000 7f756f7fe640 1 --2- 172.21.15.26:0/64564143 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f757006ea30 0x7f757010c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:31:11.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:11.197+0000 7f756effd640 1 --2- 172.21.15.26:0/64564143 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f757010c0a0 0x7f757010cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:31:11.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:11.198+0000 7f756ffff640 1 --2- 172.21.15.26:0/64564143 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f75701169f0 0x7f757010d060 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f7568030630 tx=0x7f7568030660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:31:11.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:11.198+0000 7f756cff9640 1 -- 172.21.15.26:0/64564143 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7568040030 con 0x7f75701169f0 2024-09-11T23:31:11.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:11.198+0000 7f75749a0640 1 -- 172.21.15.26:0/64564143 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75701b3690 con 0x7f75701169f0 2024-09-11T23:31:11.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:11.199+0000 7f75749a0640 1 -- 172.21.15.26:0/64564143 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75701b3b70 con 0x7f75701169f0 2024-09-11T23:31:11.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:11.199+0000 7f756cff9640 1 -- 172.21.15.26:0/64564143 <== mon.2 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7568003e50 con 0x7f75701169f0 2024-09-11T23:31:11.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:11.199+0000 7f756cff9640 1 -- 172.21.15.26:0/64564143 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7568036040 con 0x7f75701169f0 2024-09-11T23:31:11.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:11.201+0000 7f75749a0640 1 -- 172.21.15.26:0/64564143 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75701096f0 con 0x7f75701169f0 2024-09-11T23:31:11.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:11.203+0000 7f756cff9640 1 -- 172.21.15.26:0/64564143 <== mon.2 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f7568043070 con 0x7f75701169f0 2024-09-11T23:31:11.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:11.204+0000 7f756cff9640 1 --2- 172.21.15.26:0/64564143 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f75580a2f00 0x7f75580a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:11.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:11.205+0000 7f756cff9640 1 -- 172.21.15.26:0/64564143 <== mon.2 v2:172.21.15.76:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8586+0+0 (secure 0 0 0) 0x7f75680efd50 con 0x7f75701169f0 2024-09-11T23:31:11.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:11.205+0000 7f756f7fe640 1 --2- 172.21.15.26:0/64564143 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f75580a2f00 0x7f75580a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:11.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:11.205+0000 7f756f7fe640 1 --2- 172.21.15.26:0/64564143 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f75580a2f00 0x7f75580a53c0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f7560005fd0 tx=0x7f7560009260 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:31:11.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:11.214+0000 7f756cff9640 1 -- 172.21.15.26:0/64564143 <== mon.2 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f75680bc950 con 0x7f75701169f0 2024-09-11T23:31:11.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:11.595+0000 7f75749a0640 1 -- 172.21.15.26:0/64564143 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f757006ee70 con 0x7f75701169f0 2024-09-11T23:31:11.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:11.597+0000 7f756cff9640 1 -- 172.21.15.26:0/64564143 <== mon.2 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) ==== 94+0+12005 (secure 0 0 0) 0x7f757006ee70 con 0x7f75701169f0 2024-09-11T23:31:11.597 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-11T23:31:11.598 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":69,"btime":"2024-09-11T23:30:48:816398+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14499,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/367659515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":367659515},{"type":"v1","addr":"172.21.15.26:6835","nonce":367659515}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14511,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/2041065299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":2041065299},{"type":"v1","addr":"172.21.15.26:6837","nonce":2041065299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":69,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:16:26.544203+0000","modified":"2024-09-11T23:30:47.842277+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24485,"mds_1":24463,"mds_2":24458,"mds_3":24448,"mds_4":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14550":{"gid":14550,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6839/2233068305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":2233068305},{"type":"v1","addr":"172.21.15.26:6839","nonce":2233068305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14562":{"gid":14562,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6841/2984982336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":2984982336},{"type":"v1","addr":"172.21.15.26:6841","nonce":2984982336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6833/2019841455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":2019841455},{"type":"v1","addr":"172.21.15.136:6833","nonce":2019841455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.76:6833/3438147050","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3438147050},{"type":"v1","addr":"172.21.15.76:6833","nonce":3438147050}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6835/3617665975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3617665975},{"type":"v1","addr":"172.21.15.136:6835","nonce":3617665975}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":12,"addr":"172.21.15.76:6835/393308499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":393308499},{"type":"v1","addr":"172.21.15.76:6835","nonce":393308499}]},"join_fscid":-1,"export_targets":[1,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":10,"addr":"172.21.15.136:6837/1633082387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":1633082387},{"type":"v1","addr":"172.21.15.136:6837","nonce":1633082387}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.76:6837/2078978748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":2078978748},{"type":"v1","addr":"172.21.15.76:6837","nonce":2078978748}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":63,"addr":"172.21.15.136:6839/3211398630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":3211398630},{"type":"v1","addr":"172.21.15.136:6839","nonce":3211398630}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.76:6839/2077637090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6838","nonce":2077637090},{"type":"v1","addr":"172.21.15.76:6839","nonce":2077637090}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24485,"qdb_cluster":[24443,24448,24458,24463,24485]},"id":1}]} 2024-09-11T23:31:11.598 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 69 2024-09-11T23:31:11.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:11.601+0000 7f75749a0640 1 -- 172.21.15.26:0/64564143 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f75580a2f00 msgr2=0x7f75580a53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:31:11.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:11.601+0000 7f75749a0640 1 --2- 172.21.15.26:0/64564143 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f75580a2f00 0x7f75580a53c0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f7560005fd0 tx=0x7f7560009260 comp rx=0 tx=0).stop 2024-09-11T23:31:11.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:11.601+0000 7f75749a0640 1 -- 172.21.15.26:0/64564143 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f75701169f0 msgr2=0x7f757010d060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:31:11.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:11.601+0000 7f75749a0640 1 --2- 172.21.15.26:0/64564143 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f75701169f0 0x7f757010d060 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f7568030630 tx=0x7f7568030660 comp rx=0 tx=0).stop 2024-09-11T23:31:11.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:11.602+0000 7f75749a0640 1 -- 172.21.15.26:0/64564143 shutdown_connections 2024-09-11T23:31:11.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:11.602+0000 7f75749a0640 1 --2- 172.21.15.26:0/64564143 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f75580a2f00 0x7f75580a53c0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:11.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:11.602+0000 7f75749a0640 1 --2- 172.21.15.26:0/64564143 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f75701169f0 0x7f757010d060 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:11.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:11.602+0000 7f75749a0640 1 --2- 172.21.15.26:0/64564143 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f757010c0a0 0x7f757010cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:11.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:11.602+0000 7f75749a0640 1 --2- 172.21.15.26:0/64564143 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f757006ea30 0x7f757010c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:11.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:11.602+0000 7f75749a0640 1 -- 172.21.15.26:0/64564143 >> 172.21.15.26:0/64564143 conn(0x7f7570073180 msgr2=0x7f7570116dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:31:11.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:11.604+0000 7f75749a0640 1 -- 172.21.15.26:0/64564143 shutdown_connections 2024-09-11T23:31:11.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:11.604+0000 7f75749a0640 1 -- 172.21.15.26:0/64564143 wait complete. 2024-09-11T23:31:12.238 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:31:11 smithi136 ceph-mon[35121]: pgmap v400: 97 pgs: 41 active+clean+snaptrim_wait, 23 active+clean+snaptrim, 33 active+clean; 2.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.0 MiB/s wr, 1.40k op/s 2024-09-11T23:31:12.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:31:11 smithi136 ceph-mon[35121]: from='client.? 172.21.15.26:0/64564143' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:31:12.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:31:11 smithi026 ceph-mon[31172]: pgmap v400: 97 pgs: 41 active+clean+snaptrim_wait, 23 active+clean+snaptrim, 33 active+clean; 2.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.0 MiB/s wr, 1.40k op/s 2024-09-11T23:31:12.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:31:11 smithi026 ceph-mon[31172]: from='client.? 172.21.15.26:0/64564143' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:31:12.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:31:11 smithi076 ceph-mon[35296]: pgmap v400: 97 pgs: 41 active+clean+snaptrim_wait, 23 active+clean+snaptrim, 33 active+clean; 2.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.0 MiB/s wr, 1.40k op/s 2024-09-11T23:31:12.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:31:11 smithi076 ceph-mon[35296]: from='client.? 172.21.15.26:0/64564143' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:31:12.419 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24485 quiesce db / --timeout 90 --expiration 147.2 --await-for 45.0' 2024-09-11T23:31:12.420 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24485 quiesce db / --timeout 90 --expiration 147.2 --await-for 45.0 2024-09-11T23:31:12.822 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:31:13.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.421+0000 7f0b8aa09640 1 -- 172.21.15.26:0/524774586 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0b84102ef0 msgr2=0x7f0b8410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:31:13.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.421+0000 7f0b8aa09640 1 --2- 172.21.15.26:0/524774586 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0b84102ef0 0x7f0b8410da40 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f0b78009860 tx=0x7f0b7802a3d0 comp rx=0 tx=0).stop 2024-09-11T23:31:13.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.421+0000 7f0b8aa09640 1 -- 172.21.15.26:0/524774586 shutdown_connections 2024-09-11T23:31:13.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.421+0000 7f0b8aa09640 1 --2- 172.21.15.26:0/524774586 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0b84102ef0 0x7f0b8410da40 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:13.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.421+0000 7f0b8aa09640 1 --2- 172.21.15.26:0/524774586 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0b84102570 0x7f0b841029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:13.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.421+0000 7f0b8aa09640 1 --2- 172.21.15.26:0/524774586 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0b84100e20 0x7f0b84101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:13.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.421+0000 7f0b8aa09640 1 -- 172.21.15.26:0/524774586 >> 172.21.15.26:0/524774586 conn(0x7f0b840fc7d0 msgr2=0x7f0b840febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:31:13.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.421+0000 7f0b8aa09640 1 -- 172.21.15.26:0/524774586 shutdown_connections 2024-09-11T23:31:13.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.421+0000 7f0b8aa09640 1 -- 172.21.15.26:0/524774586 wait complete. 2024-09-11T23:31:13.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.422+0000 7f0b8aa09640 1 Processor -- start 2024-09-11T23:31:13.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.422+0000 7f0b8aa09640 1 -- start start 2024-09-11T23:31:13.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.423+0000 7f0b8aa09640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0b84100e20 0x7f0b84198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:13.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.423+0000 7f0b8aa09640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0b84102570 0x7f0b841989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:13.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.423+0000 7f0b8aa09640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0b84102ef0 0x7f0b8419cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:13.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.423+0000 7f0b8aa09640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f0b84110fd0 con 0x7f0b84102570 2024-09-11T23:31:13.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.423+0000 7f0b8aa09640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f0b84110e50 con 0x7f0b84100e20 2024-09-11T23:31:13.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.423+0000 7f0b8aa09640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f0b84111150 con 0x7f0b84102ef0 2024-09-11T23:31:13.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.423+0000 7f0b83fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0b84100e20 0x7f0b84198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:13.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.423+0000 7f0b83fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0b84100e20 0x7f0b84198480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:46982/0 (socket says 172.21.15.26:46982) 2024-09-11T23:31:13.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.423+0000 7f0b83fff640 1 -- 172.21.15.26:0/1241765135 learned_addr learned my addr 172.21.15.26:0/1241765135 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:31:13.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.424+0000 7f0b88fa6640 1 --2- 172.21.15.26:0/1241765135 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0b84102ef0 0x7f0b8419cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:13.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.424+0000 7f0b83fff640 1 -- 172.21.15.26:0/1241765135 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0b84102ef0 msgr2=0x7f0b8419cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:31:13.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.424+0000 7f0b83fff640 1 --2- 172.21.15.26:0/1241765135 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0b84102ef0 0x7f0b8419cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:13.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.424+0000 7f0b83fff640 1 -- 172.21.15.26:0/1241765135 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0b84102570 msgr2=0x7f0b841989c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:31:13.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.424+0000 7f0b83fff640 1 --2- 172.21.15.26:0/1241765135 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0b84102570 0x7f0b841989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:13.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.424+0000 7f0b83fff640 1 -- 172.21.15.26:0/1241765135 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b8419d430 con 0x7f0b84100e20 2024-09-11T23:31:13.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.425+0000 7f0b83fff640 1 --2- 172.21.15.26:0/1241765135 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0b84100e20 0x7f0b84198480 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f0b7c00cce0 tx=0x7f0b7c00a620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:31:13.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.426+0000 7f0b817fa640 1 -- 172.21.15.26:0/1241765135 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b7c004b20 con 0x7f0b84100e20 2024-09-11T23:31:13.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.426+0000 7f0b817fa640 1 -- 172.21.15.26:0/1241765135 <== mon.2 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b7c004cc0 con 0x7f0b84100e20 2024-09-11T23:31:13.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.426+0000 7f0b817fa640 1 -- 172.21.15.26:0/1241765135 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b7c00f7d0 con 0x7f0b84100e20 2024-09-11T23:31:13.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.426+0000 7f0b8aa09640 1 -- 172.21.15.26:0/1241765135 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b8419d720 con 0x7f0b84100e20 2024-09-11T23:31:13.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.426+0000 7f0b8aa09640 1 -- 172.21.15.26:0/1241765135 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b841a5000 con 0x7f0b84100e20 2024-09-11T23:31:13.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.429+0000 7f0b8cde2740 1 Processor -- start 2024-09-11T23:31:13.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.430+0000 7f0b8cde2740 1 -- start start 2024-09-11T23:31:13.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.430+0000 7f0b8cde2740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e5293b50a0 0x55e5293b3a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:13.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.430+0000 7f0b8cde2740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e5293b3fd0 0x55e5293b43f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:13.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.430+0000 7f0b8cde2740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e5293c2fe0 0x55e5293c53d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:13.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.430+0000 7f0b8cde2740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55e5293152a0 con 0x55e5293b50a0 2024-09-11T23:31:13.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.430+0000 7f0b8cde2740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55e5292bcb10 con 0x55e5293b3fd0 2024-09-11T23:31:13.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.430+0000 7f0b8cde2740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55e529311260 con 0x55e5293c2fe0 2024-09-11T23:31:13.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.430+0000 7f0b837fe640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e5293c2fe0 0x55e5293c53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:13.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.430+0000 7f0b837fe640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e5293c2fe0 0x55e5293c53d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:38156/0 (socket says 172.21.15.26:38156) 2024-09-11T23:31:13.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.430+0000 7f0b837fe640 1 -- 172.21.15.26:0/3481538205 learned_addr learned my addr 172.21.15.26:0/3481538205 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:31:13.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.430+0000 7f0b88fa6640 1 --2- 172.21.15.26:0/3481538205 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e5293b50a0 0x55e5293b3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:13.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.431+0000 7f0b837fe640 1 -- 172.21.15.26:0/3481538205 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e5293b3fd0 msgr2=0x55e5293b43f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T23:31:13.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.431+0000 7f0b837fe640 1 --2- 172.21.15.26:0/3481538205 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e5293b3fd0 0x55e5293b43f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:13.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.431+0000 7f0b837fe640 1 -- 172.21.15.26:0/3481538205 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e5293b50a0 msgr2=0x55e5293b3a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:31:13.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.431+0000 7f0b837fe640 1 --2- 172.21.15.26:0/3481538205 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e5293b50a0 0x55e5293b3a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:13.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.431+0000 7f0b837fe640 1 -- 172.21.15.26:0/3481538205 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e529086b90 con 0x55e5293c2fe0 2024-09-11T23:31:13.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.432+0000 7f0b837fe640 1 --2- 172.21.15.26:0/3481538205 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e5293c2fe0 0x55e5293c53d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f0b7400bab0 tx=0x7f0b7400bf70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e6591ab0ae6dc6ef server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:31:13.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.432+0000 7f0b817fa640 1 -- 172.21.15.26:0/1241765135 <== mon.2 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f0b7c005020 con 0x7f0b84100e20 2024-09-11T23:31:13.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.433+0000 7f0b6a7fc640 1 -- 172.21.15.26:0/3481538205 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b74016070 con 0x55e5293c2fe0 2024-09-11T23:31:13.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.433+0000 7f0b6a7fc640 1 -- 172.21.15.26:0/3481538205 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b74005500 con 0x55e5293c2fe0 2024-09-11T23:31:13.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.433+0000 7f0b817fa640 1 --2- 172.21.15.26:0/1241765135 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f0b6c0a2e40 0x7f0b6c0a5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:13.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.433+0000 7f0b817fa640 1 -- 172.21.15.26:0/1241765135 <== mon.2 v2:172.21.15.76:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8586+0+0 (secure 0 0 0) 0x7f0b7c0cf540 con 0x7f0b84100e20 2024-09-11T23:31:13.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.434+0000 7f0b8cde2740 1 -- 172.21.15.26:0/3481538205 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e5293c2fe0 msgr2=0x55e5293c53d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:31:13.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.434+0000 7f0b8cde2740 1 --2- 172.21.15.26:0/3481538205 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e5293c2fe0 0x55e5293c53d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f0b7400bab0 tx=0x7f0b7400bf70 comp rx=0 tx=0).stop 2024-09-11T23:31:13.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.434+0000 7f0b8cde2740 1 -- 172.21.15.26:0/3481538205 shutdown_connections 2024-09-11T23:31:13.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.434+0000 7f0b8cde2740 1 --2- 172.21.15.26:0/3481538205 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e5293b3fd0 0x55e5293b43f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:13.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.434+0000 7f0b8cde2740 1 --2- 172.21.15.26:0/3481538205 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e5293c2fe0 0x55e5293c53d0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:13.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.434+0000 7f0b8cde2740 1 --2- 172.21.15.26:0/3481538205 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e5293b50a0 0x55e5293b3a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:13.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.434+0000 7f0b8cde2740 1 -- 172.21.15.26:0/3481538205 >> 172.21.15.26:0/3481538205 conn(0x55e5293b0100 msgr2=0x55e5293b04e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:31:13.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.434+0000 7f0b88fa6640 1 --2- 172.21.15.26:0/1241765135 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f0b6c0a2e40 0x7f0b6c0a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:13.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.435+0000 7f0b8cde2740 1 -- 172.21.15.26:0/3481538205 shutdown_connections 2024-09-11T23:31:13.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.435+0000 7f0b8cde2740 1 -- 172.21.15.26:0/3481538205 wait complete. 2024-09-11T23:31:13.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.435+0000 7f0b8cde2740 1 -- start start 2024-09-11T23:31:13.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.435+0000 7f0b8cde2740 1 Processor -- start 2024-09-11T23:31:13.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.435+0000 7f0b88fa6640 1 --2- 172.21.15.26:0/1241765135 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f0b6c0a2e40 0x7f0b6c0a5300 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f0b78009830 tx=0x7f0b78005dc0 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:31:13.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.436+0000 7f0b8cde2740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e5293b3fd0 0x55e5293ac790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:13.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.436+0000 7f0b8cde2740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e5293b50a0 0x55e5293accd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:13.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.436+0000 7f0b8cde2740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e5293c2fe0 0x55e5293ad210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:13.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.436+0000 7f0b8cde2740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55e5293c5dc0 con 0x55e5293b3fd0 2024-09-11T23:31:13.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.436+0000 7f0b8cde2740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55e52929d6e0 con 0x55e5293b50a0 2024-09-11T23:31:13.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.436+0000 7f0b8cde2740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55e5293c5f40 con 0x55e5293c2fe0 2024-09-11T23:31:13.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.436+0000 7f0b8cde2740 10 client.0 _ll_register_callbacks cb 0x55e5292c1a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:31:13.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.436+0000 7f0b837fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e5293b50a0 0x55e5293accd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:13.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.436+0000 7f0b837fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e5293b50a0 0x55e5293accd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:46996/0 (socket says 172.21.15.26:46996) 2024-09-11T23:31:13.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.436+0000 7f0b837fe640 1 -- 172.21.15.26:0/1791227683 learned_addr learned my addr 172.21.15.26:0/1791227683 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:31:13.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.437+0000 7f0b88fa6640 1 --2- 172.21.15.26:0/1791227683 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e5293c2fe0 0x55e5293ad210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:13.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.437+0000 7f0b83fff640 1 --2- 172.21.15.26:0/1791227683 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e5293b3fd0 0x55e5293ac790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:13.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.437+0000 7f0b83fff640 1 -- 172.21.15.26:0/1791227683 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e5293c2fe0 msgr2=0x55e5293ad210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:31:13.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.437+0000 7f0b83fff640 1 --2- 172.21.15.26:0/1791227683 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e5293c2fe0 0x55e5293ad210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:13.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.437+0000 7f0b83fff640 1 -- 172.21.15.26:0/1791227683 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e5293b50a0 msgr2=0x55e5293accd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:31:13.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.437+0000 7f0b83fff640 1 --2- 172.21.15.26:0/1791227683 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e5293b50a0 0x55e5293accd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:13.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.437+0000 7f0b83fff640 1 -- 172.21.15.26:0/1791227683 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e5293cf370 con 0x55e5293b3fd0 2024-09-11T23:31:13.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.438+0000 7f0b83fff640 1 --2- 172.21.15.26:0/1791227683 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e5293b3fd0 0x55e5293ac790 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f0b7c0ccbc0 tx=0x7f0b7c09b320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90d5b7883ab4056b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:31:13.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.438+0000 7f0b637fe640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:31:13.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.438+0000 7f0b637fe640 1 -- 172.21.15.26:0/1791227683 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b7c0b4060 con 0x55e5293b3fd0 2024-09-11T23:31:13.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.438+0000 7f0b637fe640 1 -- 172.21.15.26:0/1791227683 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8586+0+0 (secure 0 0 0) 0x7f0b7c09b010 con 0x55e5293b3fd0 2024-09-11T23:31:13.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.439+0000 7f0b8cde2740 1 -- 172.21.15.26:0/1791227683 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0b74005500 con 0x55e5293b3fd0 2024-09-11T23:31:13.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.439+0000 7f0b637fe640 1 -- 172.21.15.26:0/1791227683 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7f0b7c01d040 con 0x55e5293b3fd0 2024-09-11T23:31:13.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.439+0000 7f0b8cde2740 10 client.17265 fetch_fsmap learned FSMap version 69 2024-09-11T23:31:13.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.439+0000 7f0b8cde2740 1 -- 172.21.15.26:0/1791227683 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) -- 0x55e5293d1d50 con 0x55e5293b3fd0 2024-09-11T23:31:13.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.440+0000 7f0b637fe640 1 -- 172.21.15.26:0/1791227683 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b74005500 con 0x55e5293b3fd0 2024-09-11T23:31:13.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.440+0000 7f0b637fe640 1 -- 172.21.15.26:0/1791227683 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 69) ==== 8438+0+0 (secure 0 0 0) 0x7f0b7c097710 con 0x55e5293b3fd0 2024-09-11T23:31:13.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.440+0000 7f0b8cde2740 10 client.17265 fetch_fsmap finished waiting for FSMap version 69 2024-09-11T23:31:13.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.440+0000 7f0b8cde2740 10 client.17265 resolve_mds: validated gid 24485 aka daemon mds.b 2024-09-11T23:31:13.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.440+0000 7f0b8cde2740 20 client.17265 populate_metadata read hostname 'smithi026' 2024-09-11T23:31:13.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.440+0000 7f0b8cde2740 1 --2- 172.21.15.26:0/1791227683 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55e5293d36e0 0x55e5293d5b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:13.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.440+0000 7f0b8cde2740 4 client.17265 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:31:13.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.440+0000 7f0b8cde2740 1 -- 172.21.15.26:0/1791227683 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e5293229d0 con 0x55e5293d36e0 2024-09-11T23:31:13.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.440+0000 7f0b837fe640 1 --2- 172.21.15.26:0/1791227683 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55e5293d36e0 0x55e5293d5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:13.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.441+0000 7f0b637fe640 10 client.17265 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:31:13.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.441+0000 7f0b837fe640 1 --2- 172.21.15.26:0/1791227683 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55e5293d36e0 0x55e5293d5b60 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5a14abce9e108b8 server_cookie=84ba1d699add5aa in_seq=0 out_seq=0 2024-09-11T23:31:13.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.442+0000 7f0b637fe640 1 -- 172.21.15.26:0/1791227683 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e5293229d0 con 0x55e5293d36e0 2024-09-11T23:31:13.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.442+0000 7f0b637fe640 10 client.17265 handle_command_reply: tid=0 2024-09-11T23:31:13.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.442+0000 7f0b8cde2740 1 client.17265 shutdown 2024-09-11T23:31:13.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.442+0000 7f0b8cde2740 1 -- 172.21.15.26:0/1791227683 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e5293b3fd0 msgr2=0x55e5293ac790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:31:13.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.442+0000 7f0b8cde2740 1 --2- 172.21.15.26:0/1791227683 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e5293b3fd0 0x55e5293ac790 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f0b7c0ccbc0 tx=0x7f0b7c09b320 comp rx=0 tx=0).stop 2024-09-11T23:31:13.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.442+0000 7f0b8cde2740 1 -- 172.21.15.26:0/1791227683 shutdown_connections 2024-09-11T23:31:13.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.442+0000 7f0b8cde2740 1 --2- 172.21.15.26:0/1791227683 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e5293b50a0 0x55e5293accd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:13.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.442+0000 7f0b8cde2740 1 --2- 172.21.15.26:0/1791227683 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55e5293d36e0 0x55e5293d5b60 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:13.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.443+0000 7f0b8cde2740 1 --2- 172.21.15.26:0/1791227683 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e5293c2fe0 0x55e5293ad210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:13.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.443+0000 7f0b8cde2740 1 --2- 172.21.15.26:0/1791227683 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e5293b3fd0 0x55e5293ac790 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:13.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.443+0000 7f0b8cde2740 1 -- 172.21.15.26:0/1791227683 >> 172.21.15.26:0/1791227683 conn(0x55e5293b0100 msgr2=0x55e5293b5e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:31:13.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.443+0000 7f0b637fe640 0 client.17265 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:31:13.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.443+0000 7f0b8cde2740 1 -- 172.21.15.26:0/1791227683 shutdown_connections 2024-09-11T23:31:13.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.443+0000 7f0b8cde2740 1 -- 172.21.15.26:0/1791227683 wait complete. 2024-09-11T23:31:13.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.443+0000 7f0b8cde2740 20 client.17265 trim_cache size 0 max 16384 2024-09-11T23:31:13.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.473+0000 7f0b8cde2740 1 Processor -- start 2024-09-11T23:31:13.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.473+0000 7f0b8cde2740 1 -- start start 2024-09-11T23:31:13.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.473+0000 7f0b8cde2740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e5293b3fd0 0x55e5293c23b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:13.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.473+0000 7f0b8cde2740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e5293b50a0 0x55e5293c28f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:13.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.473+0000 7f0b8cde2740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e5293c2fe0 0x55e5293db470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:13.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.473+0000 7f0b8cde2740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f0b841ab770 con 0x55e5293b3fd0 2024-09-11T23:31:13.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.473+0000 7f0b8cde2740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f0b841ad580 con 0x55e5293b50a0 2024-09-11T23:31:13.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.474+0000 7f0b8cde2740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f0b841abed0 con 0x55e5293c2fe0 2024-09-11T23:31:13.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.474+0000 7f0b88fa6640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e5293c2fe0 0x55e5293db470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:13.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.474+0000 7f0b88fa6640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e5293c2fe0 0x55e5293db470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:38174/0 (socket says 172.21.15.26:38174) 2024-09-11T23:31:13.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.474+0000 7f0b88fa6640 1 -- 172.21.15.26:0/2156601672 learned_addr learned my addr 172.21.15.26:0/2156601672 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:31:13.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.474+0000 7f0b837fe640 1 --2- 172.21.15.26:0/2156601672 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e5293b50a0 0x55e5293c28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:13.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.474+0000 7f0b837fe640 1 -- 172.21.15.26:0/2156601672 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e5293c2fe0 msgr2=0x55e5293db470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:31:13.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.474+0000 7f0b837fe640 1 --2- 172.21.15.26:0/2156601672 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e5293c2fe0 0x55e5293db470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:13.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.474+0000 7f0b837fe640 1 -- 172.21.15.26:0/2156601672 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e5293b3fd0 msgr2=0x55e5293c23b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:31:13.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.474+0000 7f0b837fe640 1 --2- 172.21.15.26:0/2156601672 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e5293b3fd0 0x55e5293c23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:13.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.474+0000 7f0b837fe640 1 -- 172.21.15.26:0/2156601672 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e5293a7000 con 0x55e5293b50a0 2024-09-11T23:31:13.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.475+0000 7f0b837fe640 1 --2- 172.21.15.26:0/2156601672 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e5293b50a0 0x55e5293c28f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x55e5293ac040 tx=0x7f0b74005890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4074e7c9746823d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:31:13.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.475+0000 7f0b69ffb640 1 -- 172.21.15.26:0/2156601672 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b74016040 con 0x55e5293b50a0 2024-09-11T23:31:13.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.476+0000 7f0b69ffb640 1 -- 172.21.15.26:0/2156601672 <== mon.2 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b740049a0 con 0x55e5293b50a0 2024-09-11T23:31:13.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.476+0000 7f0b69ffb640 1 -- 172.21.15.26:0/2156601672 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b74004350 con 0x55e5293b50a0 2024-09-11T23:31:13.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.477+0000 7f0b8cde2740 1 -- 172.21.15.26:0/2156601672 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e5293b50a0 msgr2=0x55e5293c28f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:31:13.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.477+0000 7f0b8cde2740 1 --2- 172.21.15.26:0/2156601672 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e5293b50a0 0x55e5293c28f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x55e5293ac040 tx=0x7f0b74005890 comp rx=0 tx=0).stop 2024-09-11T23:31:13.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.477+0000 7f0b8cde2740 1 -- 172.21.15.26:0/2156601672 shutdown_connections 2024-09-11T23:31:13.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.477+0000 7f0b8cde2740 1 --2- 172.21.15.26:0/2156601672 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e5293b50a0 0x55e5293c28f0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:13.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.477+0000 7f0b8cde2740 1 --2- 172.21.15.26:0/2156601672 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e5293c2fe0 0x55e5293db470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:13.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.477+0000 7f0b8cde2740 1 --2- 172.21.15.26:0/2156601672 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e5293b3fd0 0x55e5293c23b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:13.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.477+0000 7f0b8cde2740 1 -- 172.21.15.26:0/2156601672 >> 172.21.15.26:0/2156601672 conn(0x55e5293b0100 msgr2=0x55e5293c03f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:31:13.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.477+0000 7f0b8cde2740 1 -- 172.21.15.26:0/2156601672 shutdown_connections 2024-09-11T23:31:13.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.477+0000 7f0b8cde2740 1 -- 172.21.15.26:0/2156601672 wait complete. 2024-09-11T23:31:13.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.478+0000 7f0b8cde2740 1 -- start start 2024-09-11T23:31:13.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.478+0000 7f0b8cde2740 1 Processor -- start 2024-09-11T23:31:13.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.479+0000 7f0b8cde2740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e5293b3fd0 0x55e5293cb540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:13.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.479+0000 7f0b8cde2740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e5293b50a0 0x55e5293cba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:13.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.479+0000 7f0b83fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e5293b3fd0 0x55e5293cb540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:13.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.479+0000 7f0b8cde2740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e5293c2fe0 0x55e529438ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:13.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.480+0000 7f0b8cde2740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f0b841acdf0 con 0x55e5293b3fd0 2024-09-11T23:31:13.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.480+0000 7f0b8cde2740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f0b841ac660 con 0x55e5293c2fe0 2024-09-11T23:31:13.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.480+0000 7f0b8cde2740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f0b841a9790 con 0x55e5293b50a0 2024-09-11T23:31:13.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.480+0000 7f0b837fe640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e5293b50a0 0x55e5293cba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:13.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.480+0000 7f0b837fe640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e5293b50a0 0x55e5293cba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:38176/0 (socket says 172.21.15.26:38176) 2024-09-11T23:31:13.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.480+0000 7f0b837fe640 1 -- 172.21.15.26:0/1703602378 learned_addr learned my addr 172.21.15.26:0/1703602378 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:31:13.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.480+0000 7f0b8cde2740 10 client.0 _ll_register_callbacks cb 0x7f0b7c09b790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:31:13.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.480+0000 7f0b837fe640 1 -- 172.21.15.26:0/1703602378 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e5293c2fe0 msgr2=0x55e529438ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:31:13.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.480+0000 7f0b837fe640 1 --2- 172.21.15.26:0/1703602378 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e5293c2fe0 0x55e529438ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:13.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.480+0000 7f0b837fe640 1 -- 172.21.15.26:0/1703602378 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e5293b3fd0 msgr2=0x55e5293cb540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:31:13.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.480+0000 7f0b837fe640 1 --2- 172.21.15.26:0/1703602378 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e5293b3fd0 0x55e5293cb540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:13.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.480+0000 7f0b837fe640 1 -- 172.21.15.26:0/1703602378 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e529439510 con 0x55e5293b50a0 2024-09-11T23:31:13.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.481+0000 7f0b837fe640 1 --2- 172.21.15.26:0/1703602378 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e5293b50a0 0x55e5293cba80 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f0b74005920 tx=0x7f0b74005f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=67d0055574ddfe0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:31:13.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.481+0000 7f0b637fe640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-09-11T23:31:13.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.482+0000 7f0b637fe640 1 -- 172.21.15.26:0/1703602378 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b74033020 con 0x55e5293b50a0 2024-09-11T23:31:13.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.482+0000 7f0b637fe640 1 -- 172.21.15.26:0/1703602378 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8586+0+0 (secure 0 0 0) 0x7f0b740044f0 con 0x55e5293b50a0 2024-09-11T23:31:13.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.482+0000 7f0b8cde2740 1 -- 172.21.15.26:0/1703602378 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0b740049a0 con 0x55e5293b50a0 2024-09-11T23:31:13.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.483+0000 7f0b637fe640 1 -- 172.21.15.26:0/1703602378 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7f0b74016040 con 0x55e5293b50a0 2024-09-11T23:31:13.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.483+0000 7f0b8cde2740 10 client.26788 fetch_fsmap learned FSMap version 69 2024-09-11T23:31:13.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.483+0000 7f0b8cde2740 1 -- 172.21.15.26:0/1703602378 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) -- 0x55e5293caea0 con 0x55e5293b50a0 2024-09-11T23:31:13.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.484+0000 7f0b637fe640 1 -- 172.21.15.26:0/1703602378 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b7400daa0 con 0x55e5293b50a0 2024-09-11T23:31:13.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.485+0000 7f0b637fe640 1 -- 172.21.15.26:0/1703602378 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 69) ==== 8438+0+0 (secure 0 0 0) 0x7f0b74003820 con 0x55e5293b50a0 2024-09-11T23:31:13.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.485+0000 7f0b8cde2740 10 client.26788 fetch_fsmap finished waiting for FSMap version 69 2024-09-11T23:31:13.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.485+0000 7f0b8cde2740 10 client.26788 resolve_mds: validated gid 24485 aka daemon mds.b 2024-09-11T23:31:13.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.485+0000 7f0b8cde2740 20 client.26788 populate_metadata read hostname 'smithi026' 2024-09-11T23:31:13.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.486+0000 7f0b8cde2740 1 --2- 172.21.15.26:0/1703602378 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55e5293bb7f0 0x55e5293bdc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:13.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.486+0000 7f0b8cde2740 4 client.26788 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 147.2, "await_for": 45.0}] 2024-09-11T23:31:13.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.486+0000 7f0b8cde2740 1 -- 172.21.15.26:0/1703602378 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 147.2, "await_for": 45.0}) -- 0x7f0b740049a0 con 0x55e5293bb7f0 2024-09-11T23:31:13.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.486+0000 7f0b83fff640 1 --2- 172.21.15.26:0/1703602378 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55e5293bb7f0 0x55e5293bdc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:13.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.487+0000 7f0b83fff640 1 --2- 172.21.15.26:0/1703602378 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55e5293bb7f0 0x55e5293bdc10 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf993847ea1a823f server_cookie=efce43edde6634ea in_seq=0 out_seq=0 2024-09-11T23:31:13.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:13.487+0000 7f0b637fe640 10 client.26788 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:31:14.238 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:31:13 smithi136 ceph-mon[35121]: pgmap v401: 97 pgs: 41 active+clean+snaptrim_wait, 23 active+clean+snaptrim, 33 active+clean; 2.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.9 MiB/s wr, 1.53k op/s 2024-09-11T23:31:14.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:31:13 smithi026 ceph-mon[31172]: pgmap v401: 97 pgs: 41 active+clean+snaptrim_wait, 23 active+clean+snaptrim, 33 active+clean; 2.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.9 MiB/s wr, 1.53k op/s 2024-09-11T23:31:14.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:31:13 smithi076 ceph-mon[35296]: pgmap v401: 97 pgs: 41 active+clean+snaptrim_wait, 23 active+clean+snaptrim, 33 active+clean; 2.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.9 MiB/s wr, 1.53k op/s 2024-09-11T23:31:16.238 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:31:15 smithi136 ceph-mon[35121]: pgmap v402: 97 pgs: 33 active+clean+snaptrim_wait, 18 active+clean+snaptrim, 46 active+clean; 2.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.0 MiB/s wr, 1.46k op/s 2024-09-11T23:31:16.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:31:15 smithi026 ceph-mon[31172]: pgmap v402: 97 pgs: 33 active+clean+snaptrim_wait, 18 active+clean+snaptrim, 46 active+clean; 2.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.0 MiB/s wr, 1.46k op/s 2024-09-11T23:31:16.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:31:15 smithi076 ceph-mon[35296]: pgmap v402: 97 pgs: 33 active+clean+snaptrim_wait, 18 active+clean+snaptrim, 46 active+clean; 2.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.0 MiB/s wr, 1.46k op/s 2024-09-11T23:31:18.238 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:31:17 smithi136 ceph-mon[35121]: pgmap v403: 97 pgs: 23 active+clean+snaptrim_wait, 13 active+clean+snaptrim, 61 active+clean; 2.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.0 MiB/s wr, 1.31k op/s 2024-09-11T23:31:18.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:31:17 smithi026 ceph-mon[31172]: pgmap v403: 97 pgs: 23 active+clean+snaptrim_wait, 13 active+clean+snaptrim, 61 active+clean; 2.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.0 MiB/s wr, 1.31k op/s 2024-09-11T23:31:18.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:31:17 smithi076 ceph-mon[35296]: pgmap v403: 97 pgs: 23 active+clean+snaptrim_wait, 13 active+clean+snaptrim, 61 active+clean; 2.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.0 MiB/s wr, 1.31k op/s 2024-09-11T23:31:20.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:31:19 smithi136 ceph-mon[35121]: pgmap v404: 97 pgs: 23 active+clean+snaptrim_wait, 13 active+clean+snaptrim, 61 active+clean; 2.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 1.24k op/s 2024-09-11T23:31:20.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:31:19 smithi026 ceph-mon[31172]: pgmap v404: 97 pgs: 23 active+clean+snaptrim_wait, 13 active+clean+snaptrim, 61 active+clean; 2.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 1.24k op/s 2024-09-11T23:31:20.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:31:19 smithi076 ceph-mon[35296]: pgmap v404: 97 pgs: 23 active+clean+snaptrim_wait, 13 active+clean+snaptrim, 61 active+clean; 2.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 1.24k op/s 2024-09-11T23:31:22.238 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:31:21 smithi136 ceph-mon[35121]: pgmap v405: 97 pgs: 1 active+clean+snaptrim, 96 active+clean; 2.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 1.25k op/s 2024-09-11T23:31:22.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:31:21 smithi026 ceph-mon[31172]: pgmap v405: 97 pgs: 1 active+clean+snaptrim, 96 active+clean; 2.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 1.25k op/s 2024-09-11T23:31:22.297 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.296+0000 7fc9e8f31640 1 -- 172.21.15.26:0/1199004167 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc9e41069c0 msgr2=0x7fc9e4106da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:31:22.297 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.296+0000 7fc9e8f31640 1 --2- 172.21.15.26:0/1199004167 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc9e41069c0 0x7fc9e4106da0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fc9d4009930 tx=0x7fc9d402a420 comp rx=0 tx=0).stop 2024-09-11T23:31:22.297 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.296+0000 7fc9e8f31640 1 -- 172.21.15.26:0/1199004167 shutdown_connections 2024-09-11T23:31:22.297 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.296+0000 7fc9e8f31640 1 --2- 172.21.15.26:0/1199004167 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc9e41009b0 0x7fc9e4100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:22.298 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.296+0000 7fc9e8f31640 1 --2- 172.21.15.26:0/1199004167 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc9e4101350 0x7fc9e410d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:22.298 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.296+0000 7fc9e8f31640 1 --2- 172.21.15.26:0/1199004167 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc9e41069c0 0x7fc9e4106da0 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:22.298 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.296+0000 7fc9e8f31640 1 -- 172.21.15.26:0/1199004167 >> 172.21.15.26:0/1199004167 conn(0x7fc9e4077990 msgr2=0x7fc9e40fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:31:22.298 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.296+0000 7fc9e8f31640 1 -- 172.21.15.26:0/1199004167 shutdown_connections 2024-09-11T23:31:22.298 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.296+0000 7fc9e8f31640 1 -- 172.21.15.26:0/1199004167 wait complete. 2024-09-11T23:31:22.300 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.297+0000 7fc9e8f31640 1 Processor -- start 2024-09-11T23:31:22.300 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.297+0000 7fc9e8f31640 1 -- start start 2024-09-11T23:31:22.300 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.297+0000 7fc9e8f31640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc9e41009b0 0x7fc9e4198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:22.300 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.297+0000 7fc9e8f31640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc9e4101350 0x7fc9e4198860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:22.300 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.297+0000 7fc9e8f31640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc9e41069c0 0x7fc9e419cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:22.300 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.297+0000 7fc9e8f31640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc9e4110e00 con 0x7fc9e4101350 2024-09-11T23:31:22.300 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.297+0000 7fc9e8f31640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fc9e4110c80 con 0x7fc9e41009b0 2024-09-11T23:31:22.301 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.297+0000 7fc9e8f31640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fc9e4110f80 con 0x7fc9e41069c0 2024-09-11T23:31:22.301 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.298+0000 7fc9dbfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc9e4101350 0x7fc9e4198860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:22.301 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.298+0000 7fc9dbfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc9e4101350 0x7fc9e4198860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33048/0 (socket says 172.21.15.26:33048) 2024-09-11T23:31:22.301 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.298+0000 7fc9dbfff640 1 -- 172.21.15.26:0/3018585629 learned_addr learned my addr 172.21.15.26:0/3018585629 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:31:22.301 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.298+0000 7fc9e259c640 1 --2- 172.21.15.26:0/3018585629 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc9e41009b0 0x7fc9e4198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:22.301 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.298+0000 7fc9e2d9d640 1 --2- 172.21.15.26:0/3018585629 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc9e41069c0 0x7fc9e419cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:22.301 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.298+0000 7fc9dbfff640 1 -- 172.21.15.26:0/3018585629 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc9e41069c0 msgr2=0x7fc9e419cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:31:22.301 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.298+0000 7fc9dbfff640 1 --2- 172.21.15.26:0/3018585629 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc9e41069c0 0x7fc9e419cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:22.301 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.298+0000 7fc9dbfff640 1 -- 172.21.15.26:0/3018585629 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc9e41009b0 msgr2=0x7fc9e4198320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:31:22.301 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.298+0000 7fc9dbfff640 1 --2- 172.21.15.26:0/3018585629 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc9e41009b0 0x7fc9e4198320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:22.301 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.298+0000 7fc9dbfff640 1 -- 172.21.15.26:0/3018585629 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9e419d310 con 0x7fc9e4101350 2024-09-11T23:31:22.301 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.298+0000 7fc9dbfff640 1 --2- 172.21.15.26:0/3018585629 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc9e4101350 0x7fc9e4198860 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fc9cc00e9f0 tx=0x7fc9cc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:31:22.301 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.298+0000 7fc9db7fe640 1 -- 172.21.15.26:0/3018585629 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9cc00cd80 con 0x7fc9e4101350 2024-09-11T23:31:22.302 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.298+0000 7fc9db7fe640 1 -- 172.21.15.26:0/3018585629 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc9cc010910 con 0x7fc9e4101350 2024-09-11T23:31:22.302 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.298+0000 7fc9e8f31640 1 -- 172.21.15.26:0/3018585629 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9e419d600 con 0x7fc9e4101350 2024-09-11T23:31:22.302 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.298+0000 7fc9db7fe640 1 -- 172.21.15.26:0/3018585629 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9cc018990 con 0x7fc9e4101350 2024-09-11T23:31:22.302 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.298+0000 7fc9e8f31640 1 -- 172.21.15.26:0/3018585629 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9e41a4e90 con 0x7fc9e4101350 2024-09-11T23:31:22.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:31:21 smithi076 ceph-mon[35296]: pgmap v405: 97 pgs: 1 active+clean+snaptrim, 96 active+clean; 2.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 1.25k op/s 2024-09-11T23:31:22.308 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.301+0000 7fc9db7fe640 1 -- 172.21.15.26:0/3018585629 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fc9cc010ac0 con 0x7fc9e4101350 2024-09-11T23:31:22.308 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.301+0000 7fc9ebccb740 1 Processor -- start 2024-09-11T23:31:22.308 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.302+0000 7fc9ebccb740 1 -- start start 2024-09-11T23:31:22.308 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.302+0000 7fc9ebccb740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55869f06aaf0 0x55869f06aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:22.308 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.302+0000 7fc9ebccb740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55869f06b410 0x55869f075770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:22.308 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.302+0000 7fc9ebccb740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55869f075cb0 0x55869f0780a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:22.308 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.302+0000 7fc9ebccb740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55869efcf2f0 con 0x55869f075cb0 2024-09-11T23:31:22.308 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.302+0000 7fc9ebccb740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55869ef77270 con 0x55869f06aaf0 2024-09-11T23:31:22.308 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.302+0000 7fc9ebccb740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55869efcb2b0 con 0x55869f06b410 2024-09-11T23:31:22.308 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.302+0000 7fc9e2d9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55869f06aaf0 0x55869f06aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:22.309 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.302+0000 7fc9dbfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55869f075cb0 0x55869f0780a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:22.309 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.302+0000 7fc9e2d9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55869f06aaf0 0x55869f06aed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:36326/0 (socket says 172.21.15.26:36326) 2024-09-11T23:31:22.309 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.302+0000 7fc9e2d9d640 1 -- 172.21.15.26:0/1912368337 learned_addr learned my addr 172.21.15.26:0/1912368337 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:31:22.309 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.302+0000 7fc9db7fe640 1 --2- 172.21.15.26:0/3018585629 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fc9d00a2da0 0x7fc9d00a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:22.309 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.302+0000 7fc9e259c640 1 --2- 172.21.15.26:0/1912368337 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55869f06b410 0x55869f075770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:22.309 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.302+0000 7fc9e2d9d640 1 -- 172.21.15.26:0/1912368337 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55869f06b410 msgr2=0x55869f075770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:31:22.309 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.302+0000 7fc9e2d9d640 1 --2- 172.21.15.26:0/1912368337 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55869f06b410 0x55869f075770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:22.309 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.303+0000 7fc9e2d9d640 1 -- 172.21.15.26:0/1912368337 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55869f075cb0 msgr2=0x55869f0780a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:31:22.309 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.303+0000 7fc9e2d9d640 1 --2- 172.21.15.26:0/1912368337 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55869f075cb0 0x55869f0780a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:22.309 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.303+0000 7fc9e2d9d640 1 -- 172.21.15.26:0/1912368337 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55869ef76340 con 0x55869f06aaf0 2024-09-11T23:31:22.309 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.303+0000 7fc9db7fe640 1 -- 172.21.15.26:0/3018585629 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8586+0+0 (secure 0 0 0) 0x7fc9cc0cf7e0 con 0x7fc9e4101350 2024-09-11T23:31:22.310 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.303+0000 7fc9e2d9d640 1 --2- 172.21.15.26:0/3018585629 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fc9d00a2da0 0x7fc9d00a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:22.310 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.303+0000 7fc9e2d9d640 1 --2- 172.21.15.26:0/1912368337 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55869f06aaf0 0x55869f06aed0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fc9dc010440 tx=0x7fc9dc010900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d2c6be38a0cd4e86 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:31:22.310 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.303+0000 7fc9d8ff9640 1 -- 172.21.15.26:0/1912368337 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9dc01d400 con 0x55869f06aaf0 2024-09-11T23:31:22.310 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.303+0000 7fc9d8ff9640 1 -- 172.21.15.26:0/1912368337 <== mon.2 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc9dc013520 con 0x55869f06aaf0 2024-09-11T23:31:22.310 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.303+0000 7fc9d8ff9640 1 -- 172.21.15.26:0/1912368337 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9dc0118e0 con 0x55869f06aaf0 2024-09-11T23:31:22.310 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.304+0000 7fc9ebccb740 1 -- 172.21.15.26:0/1912368337 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55869f06aaf0 msgr2=0x55869f06aed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:31:22.310 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.304+0000 7fc9ebccb740 1 --2- 172.21.15.26:0/1912368337 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55869f06aaf0 0x55869f06aed0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fc9dc010440 tx=0x7fc9dc010900 comp rx=0 tx=0).stop 2024-09-11T23:31:22.310 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.304+0000 7fc9e2d9d640 1 --2- 172.21.15.26:0/3018585629 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fc9d00a2da0 0x7fc9d00a5260 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fc9dc021000 tx=0x7fc9dc01e000 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:31:22.310 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.304+0000 7fc9ebccb740 1 -- 172.21.15.26:0/1912368337 shutdown_connections 2024-09-11T23:31:22.310 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.304+0000 7fc9ebccb740 1 --2- 172.21.15.26:0/1912368337 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55869f075cb0 0x55869f0780a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:22.310 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.304+0000 7fc9ebccb740 1 --2- 172.21.15.26:0/1912368337 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55869f06b410 0x55869f075770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:22.310 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.304+0000 7fc9ebccb740 1 --2- 172.21.15.26:0/1912368337 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55869f06aaf0 0x55869f06aed0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:22.310 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.304+0000 7fc9ebccb740 1 -- 172.21.15.26:0/1912368337 >> 172.21.15.26:0/1912368337 conn(0x55869f067060 msgr2=0x55869f067440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:31:22.311 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.304+0000 7fc9ebccb740 1 -- 172.21.15.26:0/1912368337 shutdown_connections 2024-09-11T23:31:22.311 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.304+0000 7fc9ebccb740 1 -- 172.21.15.26:0/1912368337 wait complete. 2024-09-11T23:31:22.311 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.304+0000 7fc9ebccb740 1 -- start start 2024-09-11T23:31:22.311 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.304+0000 7fc9ebccb740 1 Processor -- start 2024-09-11T23:31:22.311 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.305+0000 7fc9ebccb740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55869f06aaf0 0x55869f072ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:22.311 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.305+0000 7fc9ebccb740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55869f06b410 0x55869f0730e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:22.311 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.305+0000 7fc9ebccb740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55869f075cb0 0x55869f073620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:22.311 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.305+0000 7fc9ebccb740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55869f078880 con 0x55869f075cb0 2024-09-11T23:31:22.311 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.305+0000 7fc9ebccb740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55869ef58290 con 0x55869f06b410 2024-09-11T23:31:22.311 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.305+0000 7fc9ebccb740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55869f078a00 con 0x55869f06aaf0 2024-09-11T23:31:22.311 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.305+0000 7fc9ebccb740 10 client.0 _ll_register_callbacks cb 0x55869ef89140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:31:22.312 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.305+0000 7fc9e2d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55869f075cb0 0x55869f073620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:22.312 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.305+0000 7fc9dbfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55869f06b410 0x55869f0730e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:22.312 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.305+0000 7fc9e2d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55869f075cb0 0x55869f073620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33064/0 (socket says 172.21.15.26:33064) 2024-09-11T23:31:22.312 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.305+0000 7fc9e2d9d640 1 -- 172.21.15.26:0/360023307 learned_addr learned my addr 172.21.15.26:0/360023307 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:31:22.312 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.305+0000 7fc9e259c640 1 --2- 172.21.15.26:0/360023307 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55869f06aaf0 0x55869f072ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:22.312 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.305+0000 7fc9dbfff640 1 -- 172.21.15.26:0/360023307 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55869f06aaf0 msgr2=0x55869f072ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:31:22.312 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.305+0000 7fc9dbfff640 1 --2- 172.21.15.26:0/360023307 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55869f06aaf0 0x55869f072ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:22.312 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.305+0000 7fc9dbfff640 1 -- 172.21.15.26:0/360023307 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55869f075cb0 msgr2=0x55869f073620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:31:22.312 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.305+0000 7fc9dbfff640 1 --2- 172.21.15.26:0/360023307 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55869f075cb0 0x55869f073620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:22.312 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.305+0000 7fc9dbfff640 1 -- 172.21.15.26:0/360023307 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55869ece2210 con 0x55869f06b410 2024-09-11T23:31:22.313 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.305+0000 7fc9e259c640 1 --2- 172.21.15.26:0/360023307 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55869f06aaf0 0x55869f072ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:31:22.313 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.305+0000 7fc9e2d9d640 1 --2- 172.21.15.26:0/360023307 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55869f075cb0 0x55869f073620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:31:22.313 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.305+0000 7fc9dbfff640 1 --2- 172.21.15.26:0/360023307 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55869f06b410 0x55869f0730e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fc9cc0cce60 tx=0x7fc9cc09bae0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eaa920e360bbaf1a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:31:22.313 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.305+0000 7fc9c9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-11T23:31:22.313 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.305+0000 7fc9c9ffb640 1 -- 172.21.15.26:0/360023307 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9cc0cd320 con 0x55869f06b410 2024-09-11T23:31:22.313 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.306+0000 7fc9c9ffb640 1 -- 172.21.15.26:0/360023307 <== mon.2 v2:172.21.15.76:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8586+0+0 (secure 0 0 0) 0x7fc9cc0cfdd0 con 0x55869f06b410 2024-09-11T23:31:22.313 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.306+0000 7fc9ebccb740 1 -- 172.21.15.26:0/360023307 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc9dc013520 con 0x55869f06b410 2024-09-11T23:31:22.313 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.306+0000 7fc9c9ffb640 1 -- 172.21.15.26:0/360023307 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7fc9cc09bd10 con 0x55869f06b410 2024-09-11T23:31:22.313 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.306+0000 7fc9ebccb740 10 client.26810 fetch_fsmap learned FSMap version 69 2024-09-11T23:31:22.313 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.306+0000 7fc9ebccb740 1 -- 172.21.15.26:0/360023307 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) -- 0x55869edae620 con 0x55869f06b410 2024-09-11T23:31:22.314 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.306+0000 7fc9c9ffb640 1 -- 172.21.15.26:0/360023307 <== mon.2 v2:172.21.15.76:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc9dc013520 con 0x55869f06b410 2024-09-11T23:31:22.314 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.306+0000 7fc9c9ffb640 1 -- 172.21.15.26:0/360023307 <== mon.2 v2:172.21.15.76:3300/0 5 ==== fsmap(e 69) ==== 8438+0+0 (secure 0 0 0) 0x7fc9cc09b280 con 0x55869f06b410 2024-09-11T23:31:22.314 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.306+0000 7fc9ebccb740 10 client.26810 fetch_fsmap finished waiting for FSMap version 69 2024-09-11T23:31:22.314 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.306+0000 7fc9ebccb740 10 client.26810 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:31:22.314 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.306+0000 7fc9ebccb740 20 client.26810 populate_metadata read hostname 'smithi026' 2024-09-11T23:31:22.314 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.306+0000 7fc9ebccb740 1 --2- 172.21.15.26:0/360023307 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55869f085c60 0x55869f088080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:22.314 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.306+0000 7fc9ebccb740 4 client.26810 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:31:22.314 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.306+0000 7fc9ebccb740 1 -- 172.21.15.26:0/360023307 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55869efdca20 con 0x55869f085c60 2024-09-11T23:31:22.314 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.306+0000 7fc9e259c640 1 --2- 172.21.15.26:0/360023307 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55869f085c60 0x55869f088080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:22.314 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.307+0000 7fc9c9ffb640 10 client.26810 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:31:22.314 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.307+0000 7fc9e259c640 1 --2- 172.21.15.26:0/360023307 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55869f085c60 0x55869f088080 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=71cbcd802f13a3f8 server_cookie=b3c28312ff4f6212 in_seq=0 out_seq=0 2024-09-11T23:31:22.329 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.327+0000 7fc9c9ffb640 1 -- 172.21.15.26:0/360023307 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55869efdca20 con 0x55869f085c60 2024-09-11T23:31:22.329 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.328+0000 7fc9c9ffb640 10 client.26810 handle_command_reply: tid=0 2024-09-11T23:31:22.329 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.328+0000 7fc9ebccb740 1 client.26810 shutdown 2024-09-11T23:31:22.329 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.328+0000 7fc9ebccb740 1 -- 172.21.15.26:0/360023307 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55869f06b410 msgr2=0x55869f0730e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:31:22.329 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.328+0000 7fc9ebccb740 1 --2- 172.21.15.26:0/360023307 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55869f06b410 0x55869f0730e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fc9cc0cce60 tx=0x7fc9cc09bae0 comp rx=0 tx=0).stop 2024-09-11T23:31:22.329 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.328+0000 7fc9ebccb740 1 -- 172.21.15.26:0/360023307 shutdown_connections 2024-09-11T23:31:22.330 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.328+0000 7fc9ebccb740 1 --2- 172.21.15.26:0/360023307 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55869f06b410 0x55869f0730e0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:22.330 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.328+0000 7fc9ebccb740 1 --2- 172.21.15.26:0/360023307 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55869f085c60 0x55869f088080 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:22.330 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.328+0000 7fc9ebccb740 1 --2- 172.21.15.26:0/360023307 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55869f075cb0 0x55869f073620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:22.330 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.328+0000 7fc9c9ffb640 0 client.26810 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:31:22.330 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.328+0000 7fc9ebccb740 1 --2- 172.21.15.26:0/360023307 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55869f06aaf0 0x55869f072ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:22.330 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.328+0000 7fc9ebccb740 1 -- 172.21.15.26:0/360023307 >> 172.21.15.26:0/360023307 conn(0x55869f067060 msgr2=0x55869f06cd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:31:22.330 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.328+0000 7fc9ebccb740 1 -- 172.21.15.26:0/360023307 shutdown_connections 2024-09-11T23:31:22.330 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.328+0000 7fc9ebccb740 1 -- 172.21.15.26:0/360023307 wait complete. 2024-09-11T23:31:22.330 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.328+0000 7fc9ebccb740 20 client.26810 trim_cache size 0 max 16384 2024-09-11T23:31:22.345 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.343+0000 7fc9ebccb740 1 Processor -- start 2024-09-11T23:31:22.345 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.343+0000 7fc9ebccb740 1 -- start start 2024-09-11T23:31:22.345 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.343+0000 7fc9ebccb740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55869f06aaf0 0x55869f071450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:22.345 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.343+0000 7fc9ebccb740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55869f06b410 0x55869f073450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:22.345 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.343+0000 7fc9ebccb740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55869f075cb0 0x55869f08d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:22.345 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.343+0000 7fc9ebccb740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc9e41ad950 con 0x55869f075cb0 2024-09-11T23:31:22.345 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.343+0000 7fc9ebccb740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fc9e41af760 con 0x55869f06aaf0 2024-09-11T23:31:22.345 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.343+0000 7fc9ebccb740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fc9e41ae0b0 con 0x55869f06b410 2024-09-11T23:31:22.345 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.343+0000 7fc9e2d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55869f075cb0 0x55869f08d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:22.345 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.343+0000 7fc9e259c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55869f06aaf0 0x55869f071450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:22.345 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.343+0000 7fc9e2d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55869f075cb0 0x55869f08d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33068/0 (socket says 172.21.15.26:33068) 2024-09-11T23:31:22.345 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.343+0000 7fc9e2d9d640 1 -- 172.21.15.26:0/2946875245 learned_addr learned my addr 172.21.15.26:0/2946875245 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:31:22.346 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.343+0000 7fc9dbfff640 1 --2- 172.21.15.26:0/2946875245 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55869f06b410 0x55869f073450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:22.346 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.343+0000 7fc9e2d9d640 1 -- 172.21.15.26:0/2946875245 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55869f06b410 msgr2=0x55869f073450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:31:22.346 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.343+0000 7fc9e2d9d640 1 --2- 172.21.15.26:0/2946875245 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55869f06b410 0x55869f073450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:22.346 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.343+0000 7fc9e2d9d640 1 -- 172.21.15.26:0/2946875245 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55869f06aaf0 msgr2=0x55869f071450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:31:22.346 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.343+0000 7fc9e2d9d640 1 --2- 172.21.15.26:0/2946875245 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55869f06aaf0 0x55869f071450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:22.346 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.343+0000 7fc9e2d9d640 1 -- 172.21.15.26:0/2946875245 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55869f06a870 con 0x55869f075cb0 2024-09-11T23:31:22.346 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.344+0000 7fc9e2d9d640 1 --2- 172.21.15.26:0/2946875245 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55869f075cb0 0x55869f08d930 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fc9dc010370 tx=0x7fc9dc011520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=337f8f8a2953d403 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:31:22.346 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.344+0000 7fc9cbfff640 1 -- 172.21.15.26:0/2946875245 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9dc011630 con 0x55869f075cb0 2024-09-11T23:31:22.346 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.344+0000 7fc9cbfff640 1 -- 172.21.15.26:0/2946875245 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc9dc02a030 con 0x55869f075cb0 2024-09-11T23:31:22.346 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.344+0000 7fc9cbfff640 1 -- 172.21.15.26:0/2946875245 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9dc0042b0 con 0x55869f075cb0 2024-09-11T23:31:22.350 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.345+0000 7fc9ebccb740 1 -- 172.21.15.26:0/2946875245 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55869f075cb0 msgr2=0x55869f08d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:31:22.350 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.345+0000 7fc9ebccb740 1 --2- 172.21.15.26:0/2946875245 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55869f075cb0 0x55869f08d930 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fc9dc010370 tx=0x7fc9dc011520 comp rx=0 tx=0).stop 2024-09-11T23:31:22.350 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.345+0000 7fc9ebccb740 1 -- 172.21.15.26:0/2946875245 shutdown_connections 2024-09-11T23:31:22.350 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.345+0000 7fc9ebccb740 1 --2- 172.21.15.26:0/2946875245 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55869f075cb0 0x55869f08d930 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:22.350 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.345+0000 7fc9ebccb740 1 --2- 172.21.15.26:0/2946875245 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55869f06b410 0x55869f073450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:22.350 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.345+0000 7fc9ebccb740 1 --2- 172.21.15.26:0/2946875245 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55869f06aaf0 0x55869f071450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:22.350 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.345+0000 7fc9ebccb740 1 -- 172.21.15.26:0/2946875245 >> 172.21.15.26:0/2946875245 conn(0x55869f067060 msgr2=0x55869f074c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:31:22.350 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.345+0000 7fc9ebccb740 1 -- 172.21.15.26:0/2946875245 shutdown_connections 2024-09-11T23:31:22.350 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.345+0000 7fc9ebccb740 1 -- 172.21.15.26:0/2946875245 wait complete. 2024-09-11T23:31:22.350 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.345+0000 7fc9ebccb740 1 -- start start 2024-09-11T23:31:22.350 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.345+0000 7fc9ebccb740 1 Processor -- start 2024-09-11T23:31:22.350 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.345+0000 7fc9ebccb740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55869f06aaf0 0x55869f07e070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:22.351 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.345+0000 7fc9ebccb740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55869f06b410 0x55869f07e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:22.351 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.345+0000 7fc9ebccb740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55869f075cb0 0x55869f0795e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:22.351 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.345+0000 7fc9ebccb740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc9e41aefd0 con 0x55869f06b410 2024-09-11T23:31:22.351 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.345+0000 7fc9ebccb740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fc9e41ae840 con 0x55869f06aaf0 2024-09-11T23:31:22.351 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.345+0000 7fc9ebccb740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fc9e41ab950 con 0x55869f075cb0 2024-09-11T23:31:22.351 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.345+0000 7fc9ebccb740 10 client.0 _ll_register_callbacks cb 0x7fc9cc00e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:31:22.351 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.345+0000 7fc9dbfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55869f06b410 0x55869f07e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:22.351 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.346+0000 7fc9e259c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55869f06aaf0 0x55869f07e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:22.351 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.346+0000 7fc9dbfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55869f06b410 0x55869f07e5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33076/0 (socket says 172.21.15.26:33076) 2024-09-11T23:31:22.351 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.346+0000 7fc9dbfff640 1 -- 172.21.15.26:0/31828806 learned_addr learned my addr 172.21.15.26:0/31828806 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:31:22.351 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.346+0000 7fc9e2d9d640 1 --2- 172.21.15.26:0/31828806 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55869f075cb0 0x55869f0795e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:22.351 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.346+0000 7fc9dbfff640 1 -- 172.21.15.26:0/31828806 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55869f075cb0 msgr2=0x55869f0795e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:31:22.351 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.346+0000 7fc9dbfff640 1 --2- 172.21.15.26:0/31828806 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55869f075cb0 0x55869f0795e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:22.352 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.346+0000 7fc9dbfff640 1 -- 172.21.15.26:0/31828806 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55869f06aaf0 msgr2=0x55869f07e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:31:22.352 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.346+0000 7fc9dbfff640 1 --2- 172.21.15.26:0/31828806 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55869f06aaf0 0x55869f07e070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:22.352 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.346+0000 7fc9dbfff640 1 -- 172.21.15.26:0/31828806 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55869f079e10 con 0x55869f06b410 2024-09-11T23:31:22.352 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.346+0000 7fc9dbfff640 1 --2- 172.21.15.26:0/31828806 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55869f06b410 0x55869f07e5b0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7fc9cc0986e0 tx=0x7fc9cc018e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8fc773e2a023e82 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:31:22.352 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.346+0000 7fc9c9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:31:22.352 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.346+0000 7fc9c9ffb640 1 -- 172.21.15.26:0/31828806 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9cc0b4e60 con 0x55869f06b410 2024-09-11T23:31:22.352 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.346+0000 7fc9c9ffb640 1 -- 172.21.15.26:0/31828806 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8586+0+0 (secure 0 0 0) 0x7fc9cc0cfab0 con 0x55869f06b410 2024-09-11T23:31:22.352 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.346+0000 7fc9ebccb740 1 -- 172.21.15.26:0/31828806 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc9dc02a030 con 0x55869f06b410 2024-09-11T23:31:22.352 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.346+0000 7fc9c9ffb640 1 -- 172.21.15.26:0/31828806 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7fc9cc0cd320 con 0x55869f06b410 2024-09-11T23:31:22.352 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.346+0000 7fc9ebccb740 10 client.17283 fetch_fsmap learned FSMap version 69 2024-09-11T23:31:22.352 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.346+0000 7fc9ebccb740 1 -- 172.21.15.26:0/31828806 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) -- 0x55869f0f05c0 con 0x55869f06b410 2024-09-11T23:31:22.352 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.346+0000 7fc9c9ffb640 1 -- 172.21.15.26:0/31828806 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc9dc02a030 con 0x55869f06b410 2024-09-11T23:31:22.352 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.347+0000 7fc9c9ffb640 1 -- 172.21.15.26:0/31828806 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 69) ==== 8438+0+0 (secure 0 0 0) 0x7fc9cc097a60 con 0x55869f06b410 2024-09-11T23:31:22.353 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.347+0000 7fc9ebccb740 10 client.17283 fetch_fsmap finished waiting for FSMap version 69 2024-09-11T23:31:22.353 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.347+0000 7fc9ebccb740 10 client.17283 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:31:22.353 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.347+0000 7fc9ebccb740 20 client.17283 populate_metadata read hostname 'smithi026' 2024-09-11T23:31:22.353 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.347+0000 7fc9ebccb740 1 --2- 172.21.15.26:0/31828806 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55869f0f0e30 0x55869f0f3250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:22.353 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.347+0000 7fc9ebccb740 4 client.17283 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T23:31:22.353 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.347+0000 7fc9ebccb740 1 -- 172.21.15.26:0/31828806 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55869efd6d70 con 0x55869f0f0e30 2024-09-11T23:31:22.353 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.347+0000 7fc9e259c640 1 --2- 172.21.15.26:0/31828806 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55869f0f0e30 0x55869f0f3250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:22.353 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.347+0000 7fc9c9ffb640 10 client.17283 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:31:22.353 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.347+0000 7fc9e259c640 1 --2- 172.21.15.26:0/31828806 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55869f0f0e30 0x55869f0f3250 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a2629cd3e6c44364 server_cookie=ed39679f72a25813 in_seq=0 out_seq=0 2024-09-11T23:31:22.728 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.727+0000 7fc9c9ffb640 1 -- 172.21.15.26:0/31828806 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55869efd6d70 con 0x55869f0f0e30 2024-09-11T23:31:22.728 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.727+0000 7fc9c9ffb640 10 client.17283 handle_command_reply: tid=0 2024-09-11T23:31:22.728 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.727+0000 7fc9ebccb740 1 client.17283 shutdown 2024-09-11T23:31:22.729 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.727+0000 7fc9ebccb740 1 -- 172.21.15.26:0/31828806 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55869f06b410 msgr2=0x55869f07e5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:31:22.729 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.727+0000 7fc9ebccb740 1 --2- 172.21.15.26:0/31828806 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55869f06b410 0x55869f07e5b0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7fc9cc0986e0 tx=0x7fc9cc018e60 comp rx=0 tx=0).stop 2024-09-11T23:31:22.729 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.727+0000 7fc9ebccb740 1 -- 172.21.15.26:0/31828806 shutdown_connections 2024-09-11T23:31:22.729 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.727+0000 7fc9ebccb740 1 --2- 172.21.15.26:0/31828806 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55869f0f0e30 0x55869f0f3250 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:22.729 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.727+0000 7fc9ebccb740 1 --2- 172.21.15.26:0/31828806 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55869f075cb0 0x55869f0795e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:22.729 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.727+0000 7fc9ebccb740 1 --2- 172.21.15.26:0/31828806 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55869f06b410 0x55869f07e5b0 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:22.729 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.727+0000 7fc9c9ffb640 0 client.17283 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:31:22.729 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.727+0000 7fc9ebccb740 1 --2- 172.21.15.26:0/31828806 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55869f06aaf0 0x55869f07e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:22.729 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.727+0000 7fc9ebccb740 1 -- 172.21.15.26:0/31828806 >> 172.21.15.26:0/31828806 conn(0x55869f067060 msgr2=0x55869f0659a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:31:22.729 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.727+0000 7fc9ebccb740 1 -- 172.21.15.26:0/31828806 shutdown_connections 2024-09-11T23:31:22.729 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.727+0000 7fc9ebccb740 1 -- 172.21.15.26:0/31828806 wait complete. 2024-09-11T23:31:22.729 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.727+0000 7fc9ebccb740 20 client.17283 trim_cache size 0 max 16384 2024-09-11T23:31:22.730 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-09-11T23:31:22.730 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "message": "", 2024-09-11T23:31:22.730 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-09-11T23:31:22.730 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-09-11T23:31:22.731 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.728+0000 7fc9e8f31640 1 -- 172.21.15.26:0/3018585629 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fc9d00a2da0 msgr2=0x7fc9d00a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:31:22.731 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.728+0000 7fc9e8f31640 1 --2- 172.21.15.26:0/3018585629 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fc9d00a2da0 0x7fc9d00a5260 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fc9dc021000 tx=0x7fc9dc01e000 comp rx=0 tx=0).stop 2024-09-11T23:31:22.731 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.728+0000 7fc9e8f31640 1 -- 172.21.15.26:0/3018585629 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc9e4101350 msgr2=0x7fc9e4198860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:31:22.731 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.728+0000 7fc9e8f31640 1 --2- 172.21.15.26:0/3018585629 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc9e4101350 0x7fc9e4198860 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fc9cc00e9f0 tx=0x7fc9cc00eeb0 comp rx=0 tx=0).stop 2024-09-11T23:31:22.731 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.728+0000 7fc9e8f31640 1 -- 172.21.15.26:0/3018585629 shutdown_connections 2024-09-11T23:31:22.731 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.728+0000 7fc9e8f31640 1 --2- 172.21.15.26:0/3018585629 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fc9d00a2da0 0x7fc9d00a5260 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:22.731 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.728+0000 7fc9e8f31640 1 --2- 172.21.15.26:0/3018585629 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc9e41069c0 0x7fc9e419cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:22.731 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.728+0000 7fc9e8f31640 1 --2- 172.21.15.26:0/3018585629 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc9e4101350 0x7fc9e4198860 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:22.731 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.728+0000 7fc9e8f31640 1 --2- 172.21.15.26:0/3018585629 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc9e41009b0 0x7fc9e4198320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:22.731 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.728+0000 7fc9e8f31640 1 -- 172.21.15.26:0/3018585629 >> 172.21.15.26:0/3018585629 conn(0x7fc9e4077990 msgr2=0x7fc9e4069b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:31:22.731 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.728+0000 7fc9e8f31640 1 -- 172.21.15.26:0/3018585629 shutdown_connections 2024-09-11T23:31:22.732 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:22.728+0000 7fc9e8f31640 1 -- 172.21.15.26:0/3018585629 wait complete. 2024-09-11T23:31:22.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:22.921+0000 7f0b637fe640 1 -- 172.21.15.26:0/1703602378 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x7f0b740049a0 con 0x55e5293bb7f0 2024-09-11T23:31:22.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:22.921+0000 7f0b637fe640 10 client.26788 handle_command_reply: tid=0 2024-09-11T23:31:22.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:22.921+0000 7f0b8cde2740 1 client.26788 shutdown 2024-09-11T23:31:22.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:22.921+0000 7f0b8cde2740 1 -- 172.21.15.26:0/1703602378 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e5293b50a0 msgr2=0x55e5293cba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:31:22.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:22.921+0000 7f0b8cde2740 1 --2- 172.21.15.26:0/1703602378 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e5293b50a0 0x55e5293cba80 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f0b74005920 tx=0x7f0b74005f10 comp rx=0 tx=0).stop 2024-09-11T23:31:22.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:22.921+0000 7f0b8cde2740 1 -- 172.21.15.26:0/1703602378 shutdown_connections 2024-09-11T23:31:22.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:22.921+0000 7f0b8cde2740 1 --2- 172.21.15.26:0/1703602378 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e5293c2fe0 0x55e529438ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:22.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:22.921+0000 7f0b8cde2740 1 --2- 172.21.15.26:0/1703602378 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55e5293bb7f0 0x55e5293bdc10 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:22.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:22.921+0000 7f0b8cde2740 1 --2- 172.21.15.26:0/1703602378 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e5293b50a0 0x55e5293cba80 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:22.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:22.921+0000 7f0b8cde2740 1 --2- 172.21.15.26:0/1703602378 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e5293b3fd0 0x55e5293cb540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:22.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:22.921+0000 7f0b637fe640 0 client.26788 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:31:22.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:22.921+0000 7f0b8cde2740 1 -- 172.21.15.26:0/1703602378 >> 172.21.15.26:0/1703602378 conn(0x55e5293b0100 msgr2=0x55e5293ac730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:31:22.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:22.921+0000 7f0b8cde2740 1 -- 172.21.15.26:0/1703602378 shutdown_connections 2024-09-11T23:31:22.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:22.921+0000 7f0b8cde2740 1 -- 172.21.15.26:0/1703602378 wait complete. 2024-09-11T23:31:22.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:22.921+0000 7f0b8cde2740 20 client.26788 trim_cache size 0 max 16384 2024-09-11T23:31:22.923 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-11T23:31:22.923 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 69, 2024-09-11T23:31:22.923 INFO:teuthology.orchestra.run.smithi026.stdout: "leader": 24485, 2024-09-11T23:31:22.923 INFO:teuthology.orchestra.run.smithi026.stdout: "set_version": 46, 2024-09-11T23:31:22.923 INFO:teuthology.orchestra.run.smithi026.stdout: "sets": { 2024-09-11T23:31:22.923 INFO:teuthology.orchestra.run.smithi026.stdout: "a269f5e9": { 2024-09-11T23:31:22.923 INFO:teuthology.orchestra.run.smithi026.stdout: "version": 46, 2024-09-11T23:31:22.923 INFO:teuthology.orchestra.run.smithi026.stdout: "age_ref": 0.0, 2024-09-11T23:31:22.923 INFO:teuthology.orchestra.run.smithi026.stdout: "state": { 2024-09-11T23:31:22.923 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "QUIESCED", 2024-09-11T23:31:22.924 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.0 2024-09-11T23:31:22.924 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:31:22.924 INFO:teuthology.orchestra.run.smithi026.stdout: "timeout": 90.0, 2024-09-11T23:31:22.924 INFO:teuthology.orchestra.run.smithi026.stdout: "expiration": 147.2, 2024-09-11T23:31:22.924 INFO:teuthology.orchestra.run.smithi026.stdout: "members": { 2024-09-11T23:31:22.924 INFO:teuthology.orchestra.run.smithi026.stdout: "file:/": { 2024-09-11T23:31:22.924 INFO:teuthology.orchestra.run.smithi026.stdout: "excluded": false, 2024-09-11T23:31:22.924 INFO:teuthology.orchestra.run.smithi026.stdout: "state": { 2024-09-11T23:31:22.924 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "QUIESCED", 2024-09-11T23:31:22.924 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.0 2024-09-11T23:31:22.924 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:31:22.924 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:31:22.924 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:31:22.924 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:31:22.925 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:31:22.925 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-11T23:31:22.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:22.921+0000 7f0b8aa09640 1 -- 172.21.15.26:0/1241765135 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f0b6c0a2e40 msgr2=0x7f0b6c0a5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:31:22.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:22.921+0000 7f0b8aa09640 1 --2- 172.21.15.26:0/1241765135 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f0b6c0a2e40 0x7f0b6c0a5300 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f0b78009830 tx=0x7f0b78005dc0 comp rx=0 tx=0).stop 2024-09-11T23:31:22.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:22.921+0000 7f0b8aa09640 1 -- 172.21.15.26:0/1241765135 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0b84100e20 msgr2=0x7f0b84198480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:31:22.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:22.921+0000 7f0b8aa09640 1 --2- 172.21.15.26:0/1241765135 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0b84100e20 0x7f0b84198480 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f0b7c00cce0 tx=0x7f0b7c00a620 comp rx=0 tx=0).stop 2024-09-11T23:31:22.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:22.921+0000 7f0b8aa09640 1 -- 172.21.15.26:0/1241765135 shutdown_connections 2024-09-11T23:31:22.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:22.921+0000 7f0b8aa09640 1 --2- 172.21.15.26:0/1241765135 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f0b6c0a2e40 0x7f0b6c0a5300 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:22.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:22.921+0000 7f0b8aa09640 1 --2- 172.21.15.26:0/1241765135 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0b84102ef0 0x7f0b8419cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:22.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:22.921+0000 7f0b8aa09640 1 --2- 172.21.15.26:0/1241765135 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0b84102570 0x7f0b841989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:22.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:22.921+0000 7f0b8aa09640 1 --2- 172.21.15.26:0/1241765135 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0b84100e20 0x7f0b84198480 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:22.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:22.921+0000 7f0b8aa09640 1 -- 172.21.15.26:0/1241765135 >> 172.21.15.26:0/1241765135 conn(0x7f0b840fc7d0 msgr2=0x7f0b84103770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:31:22.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:22.921+0000 7f0b8aa09640 1 -- 172.21.15.26:0/1241765135 shutdown_connections 2024-09-11T23:31:22.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:22.921+0000 7f0b8aa09640 1 -- 172.21.15.26:0/1241765135 wait complete. 2024-09-11T23:31:23.647 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'a269f5e9', quiesce took 13.7 seconds. Will release after: 13.5 2024-09-11T23:31:24.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:31:23 smithi136 ceph-mon[35121]: pgmap v406: 97 pgs: 97 active+clean; 2.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 615 KiB/s rd, 633 KiB/s wr, 257 op/s 2024-09-11T23:31:24.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:31:23 smithi026 ceph-mon[31172]: pgmap v406: 97 pgs: 97 active+clean; 2.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 615 KiB/s rd, 633 KiB/s wr, 257 op/s 2024-09-11T23:31:24.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:31:23 smithi076 ceph-mon[35296]: pgmap v406: 97 pgs: 97 active+clean; 2.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 615 KiB/s rd, 633 KiB/s wr, 257 op/s 2024-09-11T23:31:26.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:31:25 smithi136 ceph-mon[35121]: pgmap v407: 97 pgs: 97 active+clean; 2.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 584 KiB/s rd, 594 KiB/s wr, 141 op/s 2024-09-11T23:31:26.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:31:25 smithi136 ceph-mon[35121]: osdmap e100: 12 total, 12 up, 12 in 2024-09-11T23:31:26.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:31:25 smithi026 ceph-mon[31172]: pgmap v407: 97 pgs: 97 active+clean; 2.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 584 KiB/s rd, 594 KiB/s wr, 141 op/s 2024-09-11T23:31:26.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:31:25 smithi026 ceph-mon[31172]: osdmap e100: 12 total, 12 up, 12 in 2024-09-11T23:31:26.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:31:25 smithi076 ceph-mon[35296]: pgmap v407: 97 pgs: 97 active+clean; 2.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 584 KiB/s rd, 594 KiB/s wr, 141 op/s 2024-09-11T23:31:26.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:31:25 smithi076 ceph-mon[35296]: osdmap e100: 12 total, 12 up, 12 in 2024-09-11T23:31:28.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:31:27 smithi136 ceph-mon[35121]: pgmap v409: 97 pgs: 97 active+clean; 2.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 97 KiB/s wr, 54 op/s 2024-09-11T23:31:28.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:31:27 smithi026 ceph-mon[31172]: pgmap v409: 97 pgs: 97 active+clean; 2.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 97 KiB/s wr, 54 op/s 2024-09-11T23:31:28.305 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:31:27 smithi076 ceph-mon[35296]: pgmap v409: 97 pgs: 97 active+clean; 2.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 97 KiB/s wr, 54 op/s 2024-09-11T23:31:30.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:31:29 smithi136 ceph-mon[35121]: pgmap v410: 97 pgs: 97 active+clean; 2.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 97 KiB/s wr, 54 op/s 2024-09-11T23:31:30.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:31:29 smithi026 ceph-mon[31172]: pgmap v410: 97 pgs: 97 active+clean; 2.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 97 KiB/s wr, 54 op/s 2024-09-11T23:31:30.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:31:29 smithi076 ceph-mon[35296]: pgmap v410: 97 pgs: 97 active+clean; 2.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 97 KiB/s wr, 54 op/s 2024-09-11T23:31:32.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:31:31 smithi136 ceph-mon[35121]: pgmap v411: 97 pgs: 97 active+clean; 2.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 44 KiB/s wr, 37 op/s 2024-09-11T23:31:32.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:31:31 smithi026 ceph-mon[31172]: pgmap v411: 97 pgs: 97 active+clean; 2.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 44 KiB/s wr, 37 op/s 2024-09-11T23:31:32.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:31:31 smithi076 ceph-mon[35296]: pgmap v411: 97 pgs: 97 active+clean; 2.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 44 KiB/s wr, 37 op/s 2024-09-11T23:31:34.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:31:33 smithi136 ceph-mon[35121]: pgmap v412: 97 pgs: 97 active+clean; 2.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 44 KiB/s wr, 35 op/s 2024-09-11T23:31:34.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:31:33 smithi026 ceph-mon[31172]: pgmap v412: 97 pgs: 97 active+clean; 2.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 44 KiB/s wr, 35 op/s 2024-09-11T23:31:34.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:31:33 smithi076 ceph-mon[35296]: pgmap v412: 97 pgs: 97 active+clean; 2.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 44 KiB/s wr, 35 op/s 2024-09-11T23:31:35.829 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.828+0000 7f7dfa111640 1 -- 172.21.15.26:0/2278575022 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7df4107b80 msgr2=0x7f7df4107f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:31:35.829 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.828+0000 7f7dfa111640 1 --2- 172.21.15.26:0/2278575022 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7df4107b80 0x7f7df4107f60 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f7de4009930 tx=0x7f7de402a420 comp rx=0 tx=0).stop 2024-09-11T23:31:35.829 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.828+0000 7f7dfa111640 1 -- 172.21.15.26:0/2278575022 shutdown_connections 2024-09-11T23:31:35.830 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.828+0000 7f7dfa111640 1 --2- 172.21.15.26:0/2278575022 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7df4102480 0x7f7df410e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:35.830 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.828+0000 7f7dfa111640 1 --2- 172.21.15.26:0/2278575022 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7df4101ae0 0x7f7df4101f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:35.830 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.828+0000 7f7dfa111640 1 --2- 172.21.15.26:0/2278575022 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7df4107b80 0x7f7df4107f60 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:35.830 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.828+0000 7f7dfa111640 1 -- 172.21.15.26:0/2278575022 >> 172.21.15.26:0/2278575022 conn(0x7f7df40fd7e0 msgr2=0x7f7df40ffc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:31:35.830 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.828+0000 7f7dfa111640 1 -- 172.21.15.26:0/2278575022 shutdown_connections 2024-09-11T23:31:35.830 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.828+0000 7f7dfa111640 1 -- 172.21.15.26:0/2278575022 wait complete. 2024-09-11T23:31:35.830 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.829+0000 7f7dfa111640 1 Processor -- start 2024-09-11T23:31:35.832 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.829+0000 7f7dfa111640 1 -- start start 2024-09-11T23:31:35.832 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.829+0000 7f7dfa111640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7df4101ae0 0x7f7df4198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:35.832 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.829+0000 7f7dfa111640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7df4102480 0x7f7df41988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:35.832 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.829+0000 7f7dfa111640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7df4107b80 0x7f7df419cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:35.832 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.829+0000 7f7dfa111640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f7df4111f30 con 0x7f7df4107b80 2024-09-11T23:31:35.832 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.829+0000 7f7dfa111640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f7df4111db0 con 0x7f7df4102480 2024-09-11T23:31:35.833 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.829+0000 7f7dfa111640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f7df41120b0 con 0x7f7df4101ae0 2024-09-11T23:31:35.833 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.829+0000 7f7df3fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7df4107b80 0x7f7df419cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:35.833 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.829+0000 7f7df3fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7df4107b80 0x7f7df419cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53390/0 (socket says 172.21.15.26:53390) 2024-09-11T23:31:35.833 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.829+0000 7f7df3fff640 1 -- 172.21.15.26:0/2812742067 learned_addr learned my addr 172.21.15.26:0/2812742067 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:31:35.833 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.830+0000 7f7df2ffd640 1 --2- 172.21.15.26:0/2812742067 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7df4102480 0x7f7df41988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:35.833 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.830+0000 7f7df37fe640 1 --2- 172.21.15.26:0/2812742067 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7df4101ae0 0x7f7df4198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:35.833 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.830+0000 7f7df3fff640 1 -- 172.21.15.26:0/2812742067 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7df4101ae0 msgr2=0x7f7df4198380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:31:35.833 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.830+0000 7f7df3fff640 1 --2- 172.21.15.26:0/2812742067 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7df4101ae0 0x7f7df4198380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:35.833 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.830+0000 7f7df3fff640 1 -- 172.21.15.26:0/2812742067 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7df4102480 msgr2=0x7f7df41988c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:31:35.833 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.830+0000 7f7df3fff640 1 --2- 172.21.15.26:0/2812742067 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7df4102480 0x7f7df41988c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:35.833 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.830+0000 7f7df3fff640 1 -- 172.21.15.26:0/2812742067 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7df419d370 con 0x7f7df4107b80 2024-09-11T23:31:35.833 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.830+0000 7f7df3fff640 1 --2- 172.21.15.26:0/2812742067 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7df4107b80 0x7f7df419cc40 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f7dec00eb10 tx=0x7f7dec00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:31:35.834 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.830+0000 7f7df0ff9640 1 -- 172.21.15.26:0/2812742067 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dec004b20 con 0x7f7df4107b80 2024-09-11T23:31:35.834 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.830+0000 7f7df0ff9640 1 -- 172.21.15.26:0/2812742067 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7dec004cc0 con 0x7f7df4107b80 2024-09-11T23:31:35.834 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.830+0000 7f7dfa111640 1 -- 172.21.15.26:0/2812742067 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7df419d660 con 0x7f7df4107b80 2024-09-11T23:31:35.834 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.830+0000 7f7df0ff9640 1 -- 172.21.15.26:0/2812742067 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dec010760 con 0x7f7df4107b80 2024-09-11T23:31:35.834 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.830+0000 7f7dfa111640 1 -- 172.21.15.26:0/2812742067 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7df41a4ef0 con 0x7f7df4107b80 2024-09-11T23:31:35.841 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.834+0000 7f7df0ff9640 1 -- 172.21.15.26:0/2812742067 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f7dec005020 con 0x7f7df4107b80 2024-09-11T23:31:35.842 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.834+0000 7f7dfcea7740 1 Processor -- start 2024-09-11T23:31:35.842 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.834+0000 7f7dfcea7740 1 -- start start 2024-09-11T23:31:35.842 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.834+0000 7f7dfcea7740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563a5dd84af0 0x563a5dd84ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:35.842 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.834+0000 7f7dfcea7740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563a5dd85410 0x563a5dd8f770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:35.842 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.834+0000 7f7dfcea7740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563a5dd8fcb0 0x563a5dd920a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:35.842 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.834+0000 7f7dfcea7740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x563a5dce92f0 con 0x563a5dd8fcb0 2024-09-11T23:31:35.842 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.834+0000 7f7dfcea7740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x563a5dc91270 con 0x563a5dd84af0 2024-09-11T23:31:35.842 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.834+0000 7f7dfcea7740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x563a5dce52b0 con 0x563a5dd85410 2024-09-11T23:31:35.842 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.834+0000 7f7df2ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563a5dd84af0 0x563a5dd84ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:35.842 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.834+0000 7f7df2ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563a5dd84af0 0x563a5dd84ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:49092/0 (socket says 172.21.15.26:49092) 2024-09-11T23:31:35.842 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.834+0000 7f7df2ffd640 1 -- 172.21.15.26:0/2304114735 learned_addr learned my addr 172.21.15.26:0/2304114735 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:31:35.843 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.834+0000 7f7df2ffd640 1 --2- 172.21.15.26:0/2304114735 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563a5dd8fcb0 0x563a5dd920a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:35.843 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.834+0000 7f7df0ff9640 1 --2- 172.21.15.26:0/2812742067 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f7ddc0a2da0 0x7f7ddc0a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:35.843 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.834+0000 7f7df37fe640 1 --2- 172.21.15.26:0/2304114735 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563a5dd85410 0x563a5dd8f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:35.843 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.834+0000 7f7df3fff640 1 --2- 172.21.15.26:0/2812742067 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f7ddc0a2da0 0x7f7ddc0a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:35.843 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.835+0000 7f7df2ffd640 1 -- 172.21.15.26:0/2304114735 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563a5dd85410 msgr2=0x563a5dd8f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:31:35.843 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.835+0000 7f7df2ffd640 1 --2- 172.21.15.26:0/2304114735 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563a5dd85410 0x563a5dd8f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:35.843 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.835+0000 7f7df2ffd640 1 -- 172.21.15.26:0/2304114735 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563a5dd8fcb0 msgr2=0x563a5dd920a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:31:35.843 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.835+0000 7f7df2ffd640 1 --2- 172.21.15.26:0/2304114735 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563a5dd8fcb0 0x563a5dd920a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:35.843 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.835+0000 7f7df2ffd640 1 -- 172.21.15.26:0/2304114735 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a5dc90340 con 0x563a5dd84af0 2024-09-11T23:31:35.843 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.835+0000 7f7df0ff9640 1 -- 172.21.15.26:0/2812742067 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8542+0+0 (secure 0 0 0) 0x7f7dec0cfa00 con 0x7f7df4107b80 2024-09-11T23:31:35.843 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.835+0000 7f7df2ffd640 1 --2- 172.21.15.26:0/2304114735 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563a5dd84af0 0x563a5dd84ed0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f7de8011810 tx=0x7f7de80097d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9d38f1da963b671f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:31:35.843 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.835+0000 7f7dd9ffb640 1 -- 172.21.15.26:0/2304114735 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7de8004030 con 0x563a5dd84af0 2024-09-11T23:31:35.844 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.835+0000 7f7df3fff640 1 --2- 172.21.15.26:0/2812742067 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f7ddc0a2da0 0x7f7ddc0a5260 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f7dec0cd740 tx=0x7f7dec06ded0 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:31:35.844 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.835+0000 7f7dd9ffb640 1 -- 172.21.15.26:0/2304114735 <== mon.2 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7de8011e50 con 0x563a5dd84af0 2024-09-11T23:31:35.844 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.835+0000 7f7dd9ffb640 1 -- 172.21.15.26:0/2304114735 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7de8003850 con 0x563a5dd84af0 2024-09-11T23:31:35.844 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.836+0000 7f7dfcea7740 1 -- 172.21.15.26:0/2304114735 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563a5dd84af0 msgr2=0x563a5dd84ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:31:35.844 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.836+0000 7f7dfcea7740 1 --2- 172.21.15.26:0/2304114735 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563a5dd84af0 0x563a5dd84ed0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f7de8011810 tx=0x7f7de80097d0 comp rx=0 tx=0).stop 2024-09-11T23:31:35.844 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.836+0000 7f7dfcea7740 1 -- 172.21.15.26:0/2304114735 shutdown_connections 2024-09-11T23:31:35.844 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.836+0000 7f7dfcea7740 1 --2- 172.21.15.26:0/2304114735 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563a5dd8fcb0 0x563a5dd920a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:35.844 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.836+0000 7f7dfcea7740 1 --2- 172.21.15.26:0/2304114735 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563a5dd85410 0x563a5dd8f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:35.844 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.836+0000 7f7dfcea7740 1 --2- 172.21.15.26:0/2304114735 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563a5dd84af0 0x563a5dd84ed0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:35.844 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.836+0000 7f7dfcea7740 1 -- 172.21.15.26:0/2304114735 >> 172.21.15.26:0/2304114735 conn(0x563a5dd81060 msgr2=0x563a5dd81440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:31:35.844 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.836+0000 7f7dfcea7740 1 -- 172.21.15.26:0/2304114735 shutdown_connections 2024-09-11T23:31:35.844 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.836+0000 7f7dfcea7740 1 -- 172.21.15.26:0/2304114735 wait complete. 2024-09-11T23:31:35.844 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.836+0000 7f7dfcea7740 1 -- start start 2024-09-11T23:31:35.845 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.837+0000 7f7dfcea7740 1 Processor -- start 2024-09-11T23:31:35.845 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.837+0000 7f7dfcea7740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563a5dd84af0 0x563a5dd8cb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:35.845 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.837+0000 7f7dfcea7740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563a5dd85410 0x563a5dd8d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:35.845 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.837+0000 7f7dfcea7740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563a5dd8fcb0 0x563a5dd8d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:35.845 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.837+0000 7f7dfcea7740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x563a5dd92880 con 0x563a5dd8fcb0 2024-09-11T23:31:35.845 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.837+0000 7f7dfcea7740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x563a5dc72290 con 0x563a5dd85410 2024-09-11T23:31:35.845 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.837+0000 7f7dfcea7740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x563a5dd92a00 con 0x563a5dd84af0 2024-09-11T23:31:35.845 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.837+0000 7f7dfcea7740 10 client.0 _ll_register_callbacks cb 0x563a5dca3140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:31:35.845 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.837+0000 7f7df2ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563a5dd8fcb0 0x563a5dd8d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:35.845 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.837+0000 7f7df2ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563a5dd8fcb0 0x563a5dd8d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53410/0 (socket says 172.21.15.26:53410) 2024-09-11T23:31:35.845 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.837+0000 7f7df2ffd640 1 -- 172.21.15.26:0/435483829 learned_addr learned my addr 172.21.15.26:0/435483829 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:31:35.845 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.837+0000 7f7df37fe640 1 --2- 172.21.15.26:0/435483829 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563a5dd85410 0x563a5dd8d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:35.846 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.837+0000 7f7df2ffd640 1 --2- 172.21.15.26:0/435483829 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563a5dd84af0 0x563a5dd8cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:35.846 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.837+0000 7f7df2ffd640 1 -- 172.21.15.26:0/435483829 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563a5dd84af0 msgr2=0x563a5dd8cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:31:35.846 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.837+0000 7f7df2ffd640 1 --2- 172.21.15.26:0/435483829 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563a5dd84af0 0x563a5dd8cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:35.846 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.837+0000 7f7df2ffd640 1 -- 172.21.15.26:0/435483829 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563a5dd85410 msgr2=0x563a5dd8d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:31:35.846 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.837+0000 7f7df2ffd640 1 --2- 172.21.15.26:0/435483829 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563a5dd85410 0x563a5dd8d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:35.846 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.837+0000 7f7df2ffd640 1 -- 172.21.15.26:0/435483829 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a5d9fc210 con 0x563a5dd8fcb0 2024-09-11T23:31:35.846 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.837+0000 7f7df2ffd640 1 --2- 172.21.15.26:0/435483829 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563a5dd8fcb0 0x563a5dd8d5d0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f7de8004650 tx=0x7f7de8004530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36184f219385bef8 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:31:35.846 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.837+0000 7f7dd2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:31:35.846 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.838+0000 7f7dd2ffd640 1 -- 172.21.15.26:0/435483829 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7de800fd10 con 0x563a5dd8fcb0 2024-09-11T23:31:35.846 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.838+0000 7f7dd2ffd640 1 -- 172.21.15.26:0/435483829 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8542+0+0 (secure 0 0 0) 0x7f7de80046c0 con 0x563a5dd8fcb0 2024-09-11T23:31:35.847 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.838+0000 7f7dfcea7740 1 -- 172.21.15.26:0/435483829 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7de8011e50 con 0x563a5dd8fcb0 2024-09-11T23:31:35.847 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.838+0000 7f7dd2ffd640 1 -- 172.21.15.26:0/435483829 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7f7de80048c0 con 0x563a5dd8fcb0 2024-09-11T23:31:35.847 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.838+0000 7f7dfcea7740 10 client.17295 fetch_fsmap learned FSMap version 69 2024-09-11T23:31:35.847 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.838+0000 7f7dfcea7740 1 -- 172.21.15.26:0/435483829 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) -- 0x563a5dac8620 con 0x563a5dd8fcb0 2024-09-11T23:31:35.847 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.838+0000 7f7dd2ffd640 1 -- 172.21.15.26:0/435483829 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7de8011e50 con 0x563a5dd8fcb0 2024-09-11T23:31:35.847 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.838+0000 7f7dd2ffd640 1 -- 172.21.15.26:0/435483829 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 69) ==== 8438+0+0 (secure 0 0 0) 0x7f7de802c020 con 0x563a5dd8fcb0 2024-09-11T23:31:35.847 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.838+0000 7f7dfcea7740 10 client.17295 fetch_fsmap finished waiting for FSMap version 69 2024-09-11T23:31:35.847 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.838+0000 7f7dfcea7740 10 client.17295 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:31:35.847 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.838+0000 7f7dfcea7740 20 client.17295 populate_metadata read hostname 'smithi026' 2024-09-11T23:31:35.847 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.838+0000 7f7dfcea7740 1 --2- 172.21.15.26:0/435483829 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x563a5dd9fb00 0x563a5dda1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:35.848 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.838+0000 7f7dfcea7740 4 client.17295 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:31:35.848 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.838+0000 7f7dfcea7740 1 -- 172.21.15.26:0/435483829 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563a5dcf6a20 con 0x563a5dd9fb00 2024-09-11T23:31:35.848 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.839+0000 7f7df37fe640 1 --2- 172.21.15.26:0/435483829 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x563a5dd9fb00 0x563a5dda1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:35.848 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.839+0000 7f7dd2ffd640 10 client.17295 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:31:35.848 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.839+0000 7f7df37fe640 1 --2- 172.21.15.26:0/435483829 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x563a5dd9fb00 0x563a5dda1ef0 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5bae4789aefdb088 server_cookie=d17e2cfd1f81df82 in_seq=0 out_seq=0 2024-09-11T23:31:35.848 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.841+0000 7f7dd2ffd640 1 -- 172.21.15.26:0/435483829 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563a5dcf6a20 con 0x563a5dd9fb00 2024-09-11T23:31:35.848 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.841+0000 7f7dd2ffd640 10 client.17295 handle_command_reply: tid=0 2024-09-11T23:31:35.848 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.841+0000 7f7dfcea7740 1 client.17295 shutdown 2024-09-11T23:31:35.848 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.841+0000 7f7dfcea7740 1 -- 172.21.15.26:0/435483829 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563a5dd8fcb0 msgr2=0x563a5dd8d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:31:35.848 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.841+0000 7f7dfcea7740 1 --2- 172.21.15.26:0/435483829 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563a5dd8fcb0 0x563a5dd8d5d0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f7de8004650 tx=0x7f7de8004530 comp rx=0 tx=0).stop 2024-09-11T23:31:35.848 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.841+0000 7f7dfcea7740 1 -- 172.21.15.26:0/435483829 shutdown_connections 2024-09-11T23:31:35.848 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.841+0000 7f7dfcea7740 1 --2- 172.21.15.26:0/435483829 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563a5dd85410 0x563a5dd8d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:35.848 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.841+0000 7f7dfcea7740 1 --2- 172.21.15.26:0/435483829 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x563a5dd9fb00 0x563a5dda1ef0 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:35.849 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.841+0000 7f7dfcea7740 1 --2- 172.21.15.26:0/435483829 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563a5dd8fcb0 0x563a5dd8d5d0 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:35.849 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.841+0000 7f7dfcea7740 1 --2- 172.21.15.26:0/435483829 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563a5dd84af0 0x563a5dd8cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:35.849 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.841+0000 7f7dd2ffd640 0 client.17295 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:31:35.849 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.841+0000 7f7dfcea7740 1 -- 172.21.15.26:0/435483829 >> 172.21.15.26:0/435483829 conn(0x563a5dd81060 msgr2=0x563a5dd86d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:31:35.849 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.841+0000 7f7dfcea7740 1 -- 172.21.15.26:0/435483829 shutdown_connections 2024-09-11T23:31:35.849 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.841+0000 7f7dfcea7740 1 -- 172.21.15.26:0/435483829 wait complete. 2024-09-11T23:31:35.849 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.841+0000 7f7dfcea7740 20 client.17295 trim_cache size 0 max 16384 2024-09-11T23:31:35.861 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.855+0000 7f7dfcea7740 1 Processor -- start 2024-09-11T23:31:35.861 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.855+0000 7f7dfcea7740 1 -- start start 2024-09-11T23:31:35.861 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.855+0000 7f7dfcea7740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563a5dd84af0 0x563a5dd8d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:35.861 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.855+0000 7f7dfcea7740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563a5dd85410 0x563a5dd8eac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:35.861 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.855+0000 7f7dfcea7740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563a5dd8fcb0 0x563a5dd98070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:35.861 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.855+0000 7f7dfcea7740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f7df41ad900 con 0x563a5dd85410 2024-09-11T23:31:35.861 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.855+0000 7f7dfcea7740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f7df41af710 con 0x563a5dd84af0 2024-09-11T23:31:35.861 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.855+0000 7f7dfcea7740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f7df41ae060 con 0x563a5dd8fcb0 2024-09-11T23:31:35.864 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.855+0000 7f7df37fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563a5dd85410 0x563a5dd8eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:35.864 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.855+0000 7f7df37fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563a5dd85410 0x563a5dd8eac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53418/0 (socket says 172.21.15.26:53418) 2024-09-11T23:31:35.864 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.855+0000 7f7df37fe640 1 -- 172.21.15.26:0/3973204059 learned_addr learned my addr 172.21.15.26:0/3973204059 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:31:35.865 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.855+0000 7f7df2ffd640 1 --2- 172.21.15.26:0/3973204059 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563a5dd84af0 0x563a5dd8d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:35.865 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.856+0000 7f7df37fe640 1 -- 172.21.15.26:0/3973204059 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563a5dd8fcb0 msgr2=0x563a5dd98070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:31:35.865 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.856+0000 7f7df37fe640 1 --2- 172.21.15.26:0/3973204059 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563a5dd8fcb0 0x563a5dd98070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:35.865 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.856+0000 7f7df37fe640 1 -- 172.21.15.26:0/3973204059 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563a5dd84af0 msgr2=0x563a5dd8d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:31:35.865 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.856+0000 7f7df37fe640 1 --2- 172.21.15.26:0/3973204059 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563a5dd84af0 0x563a5dd8d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:35.865 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.856+0000 7f7df37fe640 1 -- 172.21.15.26:0/3973204059 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a5dd80e90 con 0x563a5dd85410 2024-09-11T23:31:35.865 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.856+0000 7f7df37fe640 1 --2- 172.21.15.26:0/3973204059 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563a5dd85410 0x563a5dd8eac0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f7de402a940 tx=0x7f7de40058b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbeb35facea02b91 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:31:35.865 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.856+0000 7f7dd97fa640 1 -- 172.21.15.26:0/3973204059 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7de4049070 con 0x563a5dd85410 2024-09-11T23:31:35.865 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.856+0000 7f7dd97fa640 1 -- 172.21.15.26:0/3973204059 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7de4005520 con 0x563a5dd85410 2024-09-11T23:31:35.865 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.856+0000 7f7dd97fa640 1 -- 172.21.15.26:0/3973204059 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7de4035040 con 0x563a5dd85410 2024-09-11T23:31:35.866 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.857+0000 7f7dfcea7740 1 -- 172.21.15.26:0/3973204059 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563a5dd85410 msgr2=0x563a5dd8eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:31:35.866 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.857+0000 7f7dfcea7740 1 --2- 172.21.15.26:0/3973204059 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563a5dd85410 0x563a5dd8eac0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f7de402a940 tx=0x7f7de40058b0 comp rx=0 tx=0).stop 2024-09-11T23:31:35.866 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.857+0000 7f7dfcea7740 1 -- 172.21.15.26:0/3973204059 shutdown_connections 2024-09-11T23:31:35.866 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.857+0000 7f7dfcea7740 1 --2- 172.21.15.26:0/3973204059 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563a5dd8fcb0 0x563a5dd98070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:35.866 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.857+0000 7f7dfcea7740 1 --2- 172.21.15.26:0/3973204059 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563a5dd85410 0x563a5dd8eac0 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:35.866 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.857+0000 7f7dfcea7740 1 --2- 172.21.15.26:0/3973204059 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563a5dd84af0 0x563a5dd8d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:35.866 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.857+0000 7f7dfcea7740 1 -- 172.21.15.26:0/3973204059 >> 172.21.15.26:0/3973204059 conn(0x563a5dd81060 msgr2=0x563a5dd8e3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:31:35.866 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.857+0000 7f7dfcea7740 1 -- 172.21.15.26:0/3973204059 shutdown_connections 2024-09-11T23:31:35.866 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.857+0000 7f7dfcea7740 1 -- 172.21.15.26:0/3973204059 wait complete. 2024-09-11T23:31:35.866 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.857+0000 7f7dfcea7740 1 -- start start 2024-09-11T23:31:35.866 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.857+0000 7f7dfcea7740 1 Processor -- start 2024-09-11T23:31:35.866 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.858+0000 7f7dfcea7740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563a5dd84af0 0x563a5dd8a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:35.866 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.858+0000 7f7dfcea7740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563a5dd85410 0x563a5dd89200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:35.867 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.858+0000 7f7dfcea7740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563a5dd8fcb0 0x563a5dd89740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:35.867 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.858+0000 7f7dfcea7740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f7df41aef80 con 0x563a5dd84af0 2024-09-11T23:31:35.867 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.858+0000 7f7dfcea7740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f7df41ae7f0 con 0x563a5dd85410 2024-09-11T23:31:35.867 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.858+0000 7f7dfcea7740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f7df41ab900 con 0x563a5dd8fcb0 2024-09-11T23:31:35.867 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.858+0000 7f7dfcea7740 10 client.0 _ll_register_callbacks cb 0x7f7de8009e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:31:35.867 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.858+0000 7f7df2ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563a5dd84af0 0x563a5dd8a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:35.867 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.858+0000 7f7df2ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563a5dd84af0 0x563a5dd8a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53432/0 (socket says 172.21.15.26:53432) 2024-09-11T23:31:35.867 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.858+0000 7f7df2ffd640 1 -- 172.21.15.26:0/2053766694 learned_addr learned my addr 172.21.15.26:0/2053766694 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:31:35.867 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.858+0000 7f7df2ffd640 1 --2- 172.21.15.26:0/2053766694 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563a5dd8fcb0 0x563a5dd89740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:35.867 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.858+0000 7f7df37fe640 1 --2- 172.21.15.26:0/2053766694 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563a5dd85410 0x563a5dd89200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:35.867 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.858+0000 7f7df2ffd640 1 -- 172.21.15.26:0/2053766694 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563a5dd8fcb0 msgr2=0x563a5dd89740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:31:35.868 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.858+0000 7f7df2ffd640 1 --2- 172.21.15.26:0/2053766694 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563a5dd8fcb0 0x563a5dd89740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:35.868 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.858+0000 7f7df2ffd640 1 -- 172.21.15.26:0/2053766694 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563a5dd85410 msgr2=0x563a5dd89200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:31:35.868 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.858+0000 7f7df2ffd640 1 --2- 172.21.15.26:0/2053766694 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563a5dd85410 0x563a5dd89200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:35.868 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.858+0000 7f7df2ffd640 1 -- 172.21.15.26:0/2053766694 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a5dd81cf0 con 0x563a5dd84af0 2024-09-11T23:31:35.868 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.858+0000 7f7df37fe640 1 --2- 172.21.15.26:0/2053766694 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563a5dd85410 0x563a5dd89200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:31:35.868 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.858+0000 7f7df2ffd640 1 --2- 172.21.15.26:0/2053766694 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563a5dd84af0 0x563a5dd8a6a0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f7de8004620 tx=0x7f7de800f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23086adc6f1e1ed3 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:31:35.868 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.858+0000 7f7dd2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:31:35.868 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.858+0000 7f7dd2ffd640 1 -- 172.21.15.26:0/2053766694 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7de800fb10 con 0x563a5dd84af0 2024-09-11T23:31:35.868 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.858+0000 7f7dfcea7740 1 -- 172.21.15.26:0/2053766694 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7de4005520 con 0x563a5dd84af0 2024-09-11T23:31:35.868 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.858+0000 7f7dd2ffd640 1 -- 172.21.15.26:0/2053766694 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8542+0+0 (secure 0 0 0) 0x7f7de800fcb0 con 0x563a5dd84af0 2024-09-11T23:31:35.868 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.859+0000 7f7dd2ffd640 1 -- 172.21.15.26:0/2053766694 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7f7de8042050 con 0x563a5dd84af0 2024-09-11T23:31:35.868 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.859+0000 7f7dfcea7740 10 client.17301 fetch_fsmap learned FSMap version 69 2024-09-11T23:31:35.869 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.859+0000 7f7dfcea7740 1 -- 172.21.15.26:0/2053766694 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) -- 0x563a5de0e4d0 con 0x563a5dd84af0 2024-09-11T23:31:35.869 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.859+0000 7f7dd2ffd640 1 -- 172.21.15.26:0/2053766694 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7de4005520 con 0x563a5dd84af0 2024-09-11T23:31:35.869 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.859+0000 7f7dd2ffd640 1 -- 172.21.15.26:0/2053766694 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 69) ==== 8438+0+0 (secure 0 0 0) 0x7f7de802c030 con 0x563a5dd84af0 2024-09-11T23:31:35.869 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.859+0000 7f7dfcea7740 10 client.17301 fetch_fsmap finished waiting for FSMap version 69 2024-09-11T23:31:35.869 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.859+0000 7f7dfcea7740 10 client.17301 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:31:35.869 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.859+0000 7f7dfcea7740 20 client.17301 populate_metadata read hostname 'smithi026' 2024-09-11T23:31:35.869 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.859+0000 7f7dfcea7740 1 --2- 172.21.15.26:0/2053766694 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x563a5de0ebf0 0x563a5de11010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:35.869 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.859+0000 7f7dfcea7740 4 client.17301 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T23:31:35.869 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.859+0000 7f7dfcea7740 1 -- 172.21.15.26:0/2053766694 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563a5dcf0d70 con 0x563a5de0ebf0 2024-09-11T23:31:35.869 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.859+0000 7f7df37fe640 1 --2- 172.21.15.26:0/2053766694 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x563a5de0ebf0 0x563a5de11010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:35.869 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.860+0000 7f7dd2ffd640 10 client.17301 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:31:35.869 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.860+0000 7f7df37fe640 1 --2- 172.21.15.26:0/2053766694 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x563a5de0ebf0 0x563a5de11010 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d196b94d36675ebb server_cookie=e4eabdb6caf2f733 in_seq=0 out_seq=0 2024-09-11T23:31:35.876 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.875+0000 7f7dd2ffd640 1 -- 172.21.15.26:0/2053766694 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563a5dcf0d70 con 0x563a5de0ebf0 2024-09-11T23:31:35.876 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.875+0000 7f7dd2ffd640 10 client.17301 handle_command_reply: tid=0 2024-09-11T23:31:35.876 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.875+0000 7f7dfcea7740 1 client.17301 shutdown 2024-09-11T23:31:35.876 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.875+0000 7f7dfcea7740 1 -- 172.21.15.26:0/2053766694 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563a5dd84af0 msgr2=0x563a5dd8a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:31:35.876 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.875+0000 7f7dfcea7740 1 --2- 172.21.15.26:0/2053766694 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563a5dd84af0 0x563a5dd8a6a0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f7de8004620 tx=0x7f7de800f790 comp rx=0 tx=0).stop 2024-09-11T23:31:35.876 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.875+0000 7f7dfcea7740 1 -- 172.21.15.26:0/2053766694 shutdown_connections 2024-09-11T23:31:35.877 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.875+0000 7f7dfcea7740 1 --2- 172.21.15.26:0/2053766694 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563a5dd85410 0x563a5dd89200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:35.877 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.875+0000 7f7dfcea7740 1 --2- 172.21.15.26:0/2053766694 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x563a5de0ebf0 0x563a5de11010 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:35.877 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.875+0000 7f7dfcea7740 1 --2- 172.21.15.26:0/2053766694 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563a5dd8fcb0 0x563a5dd89740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:35.877 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.875+0000 7f7dfcea7740 1 --2- 172.21.15.26:0/2053766694 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563a5dd84af0 0x563a5dd8a6a0 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:35.877 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.875+0000 7f7dfcea7740 1 -- 172.21.15.26:0/2053766694 >> 172.21.15.26:0/2053766694 conn(0x563a5dd81060 msgr2=0x563a5dd86d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:31:35.877 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.875+0000 7f7dd2ffd640 0 client.17301 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:31:35.877 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.876+0000 7f7dfcea7740 1 -- 172.21.15.26:0/2053766694 shutdown_connections 2024-09-11T23:31:35.877 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.876+0000 7f7dfcea7740 1 -- 172.21.15.26:0/2053766694 wait complete. 2024-09-11T23:31:35.877 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.876+0000 7f7dfcea7740 20 client.17301 trim_cache size 0 max 16384 2024-09-11T23:31:35.877 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-09-11T23:31:35.878 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "message": "", 2024-09-11T23:31:35.878 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-09-11T23:31:35.878 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-09-11T23:31:35.878 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.876+0000 7f7dfa111640 1 -- 172.21.15.26:0/2812742067 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f7ddc0a2da0 msgr2=0x7f7ddc0a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:31:35.879 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.876+0000 7f7dfa111640 1 --2- 172.21.15.26:0/2812742067 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f7ddc0a2da0 0x7f7ddc0a5260 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f7dec0cd740 tx=0x7f7dec06ded0 comp rx=0 tx=0).stop 2024-09-11T23:31:35.879 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.876+0000 7f7dfa111640 1 -- 172.21.15.26:0/2812742067 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7df4107b80 msgr2=0x7f7df419cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:31:35.879 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.876+0000 7f7dfa111640 1 --2- 172.21.15.26:0/2812742067 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7df4107b80 0x7f7df419cc40 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f7dec00eb10 tx=0x7f7dec00c2d0 comp rx=0 tx=0).stop 2024-09-11T23:31:35.879 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.876+0000 7f7dfa111640 1 -- 172.21.15.26:0/2812742067 shutdown_connections 2024-09-11T23:31:35.879 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.876+0000 7f7dfa111640 1 --2- 172.21.15.26:0/2812742067 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f7ddc0a2da0 0x7f7ddc0a5260 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:35.879 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.876+0000 7f7dfa111640 1 --2- 172.21.15.26:0/2812742067 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7df4102480 0x7f7df41988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:35.879 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.876+0000 7f7dfa111640 1 --2- 172.21.15.26:0/2812742067 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7df4107b80 0x7f7df419cc40 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:35.879 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.876+0000 7f7dfa111640 1 --2- 172.21.15.26:0/2812742067 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7df4101ae0 0x7f7df4198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:35.879 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.876+0000 7f7dfa111640 1 -- 172.21.15.26:0/2812742067 >> 172.21.15.26:0/2812742067 conn(0x7f7df40fd7e0 msgr2=0x7f7df40fdf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:31:35.879 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.876+0000 7f7dfa111640 1 -- 172.21.15.26:0/2812742067 shutdown_connections 2024-09-11T23:31:35.879 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:35.876+0000 7f7dfa111640 1 -- 172.21.15.26:0/2812742067 wait complete. 2024-09-11T23:31:36.238 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:31:35 smithi136 ceph-mon[35121]: pgmap v413: 97 pgs: 97 active+clean; 2.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 40 KiB/s wr, 17 op/s 2024-09-11T23:31:36.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:31:35 smithi026 ceph-mon[31172]: pgmap v413: 97 pgs: 97 active+clean; 2.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 40 KiB/s wr, 17 op/s 2024-09-11T23:31:36.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:31:35 smithi076 ceph-mon[35296]: pgmap v413: 97 pgs: 97 active+clean; 2.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 40 KiB/s wr, 17 op/s 2024-09-11T23:31:37.148 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:31:37.420 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:31:38.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:38.042+0000 7f0fcf1b2640 1 -- 172.21.15.26:0/2720923850 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0fc8106f10 msgr2=0x7f0fc810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:31:38.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:38.042+0000 7f0fcf1b2640 1 --2- 172.21.15.26:0/2720923850 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0fc8106f10 0x7f0fc810ea90 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f0fc0009ca0 tx=0x7f0fc002a8c0 comp rx=0 tx=0).stop 2024-09-11T23:31:38.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:38.042+0000 7f0fcf1b2640 1 -- 172.21.15.26:0/2720923850 shutdown_connections 2024-09-11T23:31:38.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:38.042+0000 7f0fcf1b2640 1 --2- 172.21.15.26:0/2720923850 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0fc8106f10 0x7f0fc810ea90 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:38.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:38.042+0000 7f0fcf1b2640 1 --2- 172.21.15.26:0/2720923850 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0fc8102790 0x7f0fc81067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:38.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:38.042+0000 7f0fcf1b2640 1 --2- 172.21.15.26:0/2720923850 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0fc8101de0 0x7f0fc81021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:38.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:38.042+0000 7f0fcf1b2640 1 -- 172.21.15.26:0/2720923850 >> 172.21.15.26:0/2720923850 conn(0x7f0fc80fd8c0 msgr2=0x7f0fc80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:31:38.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:38.042+0000 7f0fcf1b2640 1 -- 172.21.15.26:0/2720923850 shutdown_connections 2024-09-11T23:31:38.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:38.042+0000 7f0fcf1b2640 1 -- 172.21.15.26:0/2720923850 wait complete. 2024-09-11T23:31:38.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:38.043+0000 7f0fcf1b2640 1 Processor -- start 2024-09-11T23:31:38.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:38.043+0000 7f0fcf1b2640 1 -- start start 2024-09-11T23:31:38.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:38.043+0000 7f0fcf1b2640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0fc8101de0 0x7f0fc81983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:38.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:38.043+0000 7f0fcf1b2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0fc8102790 0x7f0fc8198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:38.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:38.043+0000 7f0fcf1b2640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0fc8106f10 0x7f0fc819ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:38.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:38.043+0000 7f0fcf1b2640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f0fc8111f70 con 0x7f0fc8102790 2024-09-11T23:31:38.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:38.043+0000 7f0fcf1b2640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f0fc8111df0 con 0x7f0fc8101de0 2024-09-11T23:31:38.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:38.043+0000 7f0fcf1b2640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f0fc81120f0 con 0x7f0fc8106f10 2024-09-11T23:31:38.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:38.043+0000 7f0fbffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0fc8102790 0x7f0fc8198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:38.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:38.043+0000 7f0fbffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0fc8102790 0x7f0fc8198930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53450/0 (socket says 172.21.15.26:53450) 2024-09-11T23:31:38.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:38.043+0000 7f0fbffff640 1 -- 172.21.15.26:0/3943187182 learned_addr learned my addr 172.21.15.26:0/3943187182 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:31:38.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:38.043+0000 7f0fccf4e640 1 --2- 172.21.15.26:0/3943187182 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0fc8101de0 0x7f0fc81983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:38.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:38.044+0000 7f0fcd74f640 1 --2- 172.21.15.26:0/3943187182 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0fc8106f10 0x7f0fc819ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:38.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:38.044+0000 7f0fbffff640 1 -- 172.21.15.26:0/3943187182 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0fc8106f10 msgr2=0x7f0fc819ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:31:38.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:38.044+0000 7f0fbffff640 1 --2- 172.21.15.26:0/3943187182 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0fc8106f10 0x7f0fc819ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:38.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:38.044+0000 7f0fbffff640 1 -- 172.21.15.26:0/3943187182 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0fc8101de0 msgr2=0x7f0fc81983f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:31:38.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:38.044+0000 7f0fbffff640 1 --2- 172.21.15.26:0/3943187182 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0fc8101de0 0x7f0fc81983f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:38.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:38.044+0000 7f0fbffff640 1 -- 172.21.15.26:0/3943187182 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fc819d3a0 con 0x7f0fc8102790 2024-09-11T23:31:38.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:38.044+0000 7f0fbffff640 1 --2- 172.21.15.26:0/3943187182 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0fc8102790 0x7f0fc8198930 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f0fb800d770 tx=0x7f0fb800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:31:38.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:38.044+0000 7f0fbdffb640 1 -- 172.21.15.26:0/3943187182 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fb8017070 con 0x7f0fc8102790 2024-09-11T23:31:38.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:38.044+0000 7f0fbdffb640 1 -- 172.21.15.26:0/3943187182 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0fb8005500 con 0x7f0fc8102790 2024-09-11T23:31:38.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:38.044+0000 7f0fcf1b2640 1 -- 172.21.15.26:0/3943187182 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0fc819d690 con 0x7f0fc8102790 2024-09-11T23:31:38.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:38.044+0000 7f0fbdffb640 1 -- 172.21.15.26:0/3943187182 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fb8004e00 con 0x7f0fc8102790 2024-09-11T23:31:38.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:38.044+0000 7f0fcf1b2640 1 -- 172.21.15.26:0/3943187182 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0fc81a4f70 con 0x7f0fc8102790 2024-09-11T23:31:38.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:38.045+0000 7f0fcf1b2640 1 -- 172.21.15.26:0/3943187182 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0fc81021c0 con 0x7f0fc8102790 2024-09-11T23:31:38.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:38.046+0000 7f0fbdffb640 1 -- 172.21.15.26:0/3943187182 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f0fb8004fa0 con 0x7f0fc8102790 2024-09-11T23:31:38.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:38.047+0000 7f0fbdffb640 1 --2- 172.21.15.26:0/3943187182 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f0fb00a2cf0 0x7f0fb00a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:38.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:38.047+0000 7f0fccf4e640 1 --2- 172.21.15.26:0/3943187182 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f0fb00a2cf0 0x7f0fb00a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:38.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:38.047+0000 7f0fbdffb640 1 -- 172.21.15.26:0/3943187182 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8542+0+0 (secure 0 0 0) 0x7f0fb80cedd0 con 0x7f0fc8102790 2024-09-11T23:31:38.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:38.048+0000 7f0fccf4e640 1 --2- 172.21.15.26:0/3943187182 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f0fb00a2cf0 0x7f0fb00a51b0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f0fc400bbe0 tx=0x7f0fc400b390 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:31:38.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:38.051+0000 7f0fbdffb640 1 -- 172.21.15.26:0/3943187182 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0fb809b390 con 0x7f0fc8102790 2024-09-11T23:31:38.238 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:31:37 smithi136 ceph-mon[35121]: pgmap v414: 97 pgs: 97 active+clean; 2.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 3.8 KiB/s wr, 9 op/s 2024-09-11T23:31:38.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:31:37 smithi026 ceph-mon[31172]: pgmap v414: 97 pgs: 97 active+clean; 2.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 3.8 KiB/s wr, 9 op/s 2024-09-11T23:31:38.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:38.265+0000 7f0fcf1b2640 1 -- 172.21.15.26:0/3943187182 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0fc810a0f0 con 0x7f0fc8102790 2024-09-11T23:31:38.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:38.265+0000 7f0fbdffb640 1 -- 172.21.15.26:0/3943187182 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) ==== 94+0+12005 (secure 0 0 0) 0x7f0fb809f690 con 0x7f0fc8102790 2024-09-11T23:31:38.265 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-11T23:31:38.265 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":69,"btime":"2024-09-11T23:30:48:816398+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14499,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/367659515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":367659515},{"type":"v1","addr":"172.21.15.26:6835","nonce":367659515}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14511,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/2041065299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":2041065299},{"type":"v1","addr":"172.21.15.26:6837","nonce":2041065299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":69,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:16:26.544203+0000","modified":"2024-09-11T23:30:47.842277+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24485,"mds_1":24463,"mds_2":24458,"mds_3":24448,"mds_4":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14550":{"gid":14550,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6839/2233068305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":2233068305},{"type":"v1","addr":"172.21.15.26:6839","nonce":2233068305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14562":{"gid":14562,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6841/2984982336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":2984982336},{"type":"v1","addr":"172.21.15.26:6841","nonce":2984982336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6833/2019841455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":2019841455},{"type":"v1","addr":"172.21.15.136:6833","nonce":2019841455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.76:6833/3438147050","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3438147050},{"type":"v1","addr":"172.21.15.76:6833","nonce":3438147050}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6835/3617665975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3617665975},{"type":"v1","addr":"172.21.15.136:6835","nonce":3617665975}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":12,"addr":"172.21.15.76:6835/393308499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":393308499},{"type":"v1","addr":"172.21.15.76:6835","nonce":393308499}]},"join_fscid":-1,"export_targets":[1,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":10,"addr":"172.21.15.136:6837/1633082387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":1633082387},{"type":"v1","addr":"172.21.15.136:6837","nonce":1633082387}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.76:6837/2078978748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":2078978748},{"type":"v1","addr":"172.21.15.76:6837","nonce":2078978748}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":63,"addr":"172.21.15.136:6839/3211398630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":3211398630},{"type":"v1","addr":"172.21.15.136:6839","nonce":3211398630}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.76:6839/2077637090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6838","nonce":2077637090},{"type":"v1","addr":"172.21.15.76:6839","nonce":2077637090}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24485,"qdb_cluster":[24443,24448,24458,24463,24485]},"id":1}]} 2024-09-11T23:31:38.266 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 69 2024-09-11T23:31:38.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:38.268+0000 7f0fcf1b2640 1 -- 172.21.15.26:0/3943187182 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f0fb00a2cf0 msgr2=0x7f0fb00a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:31:38.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:38.268+0000 7f0fcf1b2640 1 --2- 172.21.15.26:0/3943187182 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f0fb00a2cf0 0x7f0fb00a51b0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f0fc400bbe0 tx=0x7f0fc400b390 comp rx=0 tx=0).stop 2024-09-11T23:31:38.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:38.268+0000 7f0fcf1b2640 1 -- 172.21.15.26:0/3943187182 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0fc8102790 msgr2=0x7f0fc8198930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:31:38.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:38.268+0000 7f0fcf1b2640 1 --2- 172.21.15.26:0/3943187182 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0fc8102790 0x7f0fc8198930 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f0fb800d770 tx=0x7f0fb800dc30 comp rx=0 tx=0).stop 2024-09-11T23:31:38.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:38.268+0000 7f0fcf1b2640 1 -- 172.21.15.26:0/3943187182 shutdown_connections 2024-09-11T23:31:38.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:38.268+0000 7f0fcf1b2640 1 --2- 172.21.15.26:0/3943187182 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f0fb00a2cf0 0x7f0fb00a51b0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:38.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:38.268+0000 7f0fcf1b2640 1 --2- 172.21.15.26:0/3943187182 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0fc8106f10 0x7f0fc819ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:38.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:38.268+0000 7f0fcf1b2640 1 --2- 172.21.15.26:0/3943187182 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0fc8102790 0x7f0fc8198930 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:38.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:38.268+0000 7f0fcf1b2640 1 --2- 172.21.15.26:0/3943187182 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0fc8101de0 0x7f0fc81983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:38.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:38.268+0000 7f0fcf1b2640 1 -- 172.21.15.26:0/3943187182 >> 172.21.15.26:0/3943187182 conn(0x7f0fc80fd8c0 msgr2=0x7f0fc80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:31:38.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:38.268+0000 7f0fcf1b2640 1 -- 172.21.15.26:0/3943187182 shutdown_connections 2024-09-11T23:31:38.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:38.268+0000 7f0fcf1b2640 1 -- 172.21.15.26:0/3943187182 wait complete. 2024-09-11T23:31:38.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:31:37 smithi076 ceph-mon[35296]: pgmap v414: 97 pgs: 97 active+clean; 2.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 3.8 KiB/s wr, 9 op/s 2024-09-11T23:31:39.147 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:31:38 smithi026 ceph-mon[31172]: from='client.? 172.21.15.26:0/3943187182' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:31:39.179 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24485 quiesce db --set-id a269f5e9 --release --await' 2024-09-11T23:31:39.180 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24485 quiesce db --set-id a269f5e9 --release --await 2024-09-11T23:31:39.238 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:31:38 smithi136 ceph-mon[35121]: from='client.? 172.21.15.26:0/3943187182' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:31:39.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:31:38 smithi076 ceph-mon[35296]: from='client.? 172.21.15.26:0/3943187182' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:31:39.440 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:31:40.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.051+0000 7fa5ff8d9640 1 -- 172.21.15.26:0/3906670371 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa5f8109120 msgr2=0x7fa5f8110ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:31:40.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.051+0000 7fa5ff8d9640 1 --2- 172.21.15.26:0/3906670371 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa5f8109120 0x7fa5f8110ca0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7fa5f0009ca0 tx=0x7fa5f002a8c0 comp rx=0 tx=0).stop 2024-09-11T23:31:40.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.051+0000 7fa5ff8d9640 1 -- 172.21.15.26:0/3906670371 shutdown_connections 2024-09-11T23:31:40.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.051+0000 7fa5ff8d9640 1 --2- 172.21.15.26:0/3906670371 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa5f8109120 0x7fa5f8110ca0 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:40.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.051+0000 7fa5ff8d9640 1 --2- 172.21.15.26:0/3906670371 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa5f81049a0 0x7fa5f81089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:40.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.051+0000 7fa5ff8d9640 1 --2- 172.21.15.26:0/3906670371 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa5f8103ff0 0x7fa5f81043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:40.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.051+0000 7fa5ff8d9640 1 -- 172.21.15.26:0/3906670371 >> 172.21.15.26:0/3906670371 conn(0x7fa5f80ffab0 msgr2=0x7fa5f8101ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:31:40.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.051+0000 7fa5ff8d9640 1 -- 172.21.15.26:0/3906670371 shutdown_connections 2024-09-11T23:31:40.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.051+0000 7fa5ff8d9640 1 -- 172.21.15.26:0/3906670371 wait complete. 2024-09-11T23:31:40.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.051+0000 7fa5ff8d9640 1 Processor -- start 2024-09-11T23:31:40.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.052+0000 7fa5ff8d9640 1 -- start start 2024-09-11T23:31:40.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.052+0000 7fa5ff8d9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa5f8103ff0 0x7fa5f819a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:40.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.052+0000 7fa5ff8d9640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa5f81049a0 0x7fa5f819ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:40.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.052+0000 7fa5ff8d9640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa5f8109120 0x7fa5f819eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:40.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.052+0000 7fa5ff8d9640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fa5f8114150 con 0x7fa5f8103ff0 2024-09-11T23:31:40.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.052+0000 7fa5ff8d9640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fa5f8113fd0 con 0x7fa5f8109120 2024-09-11T23:31:40.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.052+0000 7fa5ff8d9640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fa5f81142d0 con 0x7fa5f81049a0 2024-09-11T23:31:40.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.052+0000 7fa5fd675640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa5f8103ff0 0x7fa5f819a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:40.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.052+0000 7fa5fd675640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa5f8103ff0 0x7fa5f819a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53464/0 (socket says 172.21.15.26:53464) 2024-09-11T23:31:40.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.052+0000 7fa5fd675640 1 -- 172.21.15.26:0/691449203 learned_addr learned my addr 172.21.15.26:0/691449203 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:31:40.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.052+0000 7fa5fce74640 1 --2- 172.21.15.26:0/691449203 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa5f81049a0 0x7fa5f819ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:40.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.052+0000 7fa5fde76640 1 --2- 172.21.15.26:0/691449203 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa5f8109120 0x7fa5f819eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:40.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.052+0000 7fa5fd675640 1 -- 172.21.15.26:0/691449203 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa5f81049a0 msgr2=0x7fa5f819ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:31:40.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.052+0000 7fa5fd675640 1 --2- 172.21.15.26:0/691449203 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa5f81049a0 0x7fa5f819ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:40.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.052+0000 7fa5fd675640 1 -- 172.21.15.26:0/691449203 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa5f8109120 msgr2=0x7fa5f819eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:31:40.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.052+0000 7fa5fd675640 1 --2- 172.21.15.26:0/691449203 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa5f8109120 0x7fa5f819eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:40.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.052+0000 7fa5fd675640 1 -- 172.21.15.26:0/691449203 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5f819f590 con 0x7fa5f8103ff0 2024-09-11T23:31:40.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.052+0000 7fa5fde76640 1 --2- 172.21.15.26:0/691449203 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa5f8109120 0x7fa5f819eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:31:40.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.052+0000 7fa5fd675640 1 --2- 172.21.15.26:0/691449203 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa5f8103ff0 0x7fa5f819a5e0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7fa5ec00d950 tx=0x7fa5ec00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:31:40.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.053+0000 7fa5ea7fc640 1 -- 172.21.15.26:0/691449203 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5ec017070 con 0x7fa5f8103ff0 2024-09-11T23:31:40.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.053+0000 7fa5ff8d9640 1 -- 172.21.15.26:0/691449203 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5f819f880 con 0x7fa5f8103ff0 2024-09-11T23:31:40.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.053+0000 7fa5ea7fc640 1 -- 172.21.15.26:0/691449203 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5ec009c30 con 0x7fa5f8103ff0 2024-09-11T23:31:40.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.053+0000 7fa5ea7fc640 1 -- 172.21.15.26:0/691449203 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5ec00f6b0 con 0x7fa5f8103ff0 2024-09-11T23:31:40.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.053+0000 7fa5ff8d9640 1 -- 172.21.15.26:0/691449203 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5f81a7160 con 0x7fa5f8103ff0 2024-09-11T23:31:40.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.055+0000 7fa5ea7fc640 1 -- 172.21.15.26:0/691449203 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fa5ec0109d0 con 0x7fa5f8103ff0 2024-09-11T23:31:40.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.055+0000 7fa601cb2740 1 Processor -- start 2024-09-11T23:31:40.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.055+0000 7fa601cb2740 1 -- start start 2024-09-11T23:31:40.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.056+0000 7fa601cb2740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557e02291060 0x557e02291440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:40.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.056+0000 7fa601cb2740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557e02291980 0x557e0229bd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:40.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.056+0000 7fa601cb2740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557e0229c240 0x557e0229e630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:40.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.056+0000 7fa601cb2740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x557e021f6620 con 0x557e02291060 2024-09-11T23:31:40.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.056+0000 7fa601cb2740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x557e0219de50 con 0x557e02291980 2024-09-11T23:31:40.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.056+0000 7fa601cb2740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x557e021f25e0 con 0x557e0229c240 2024-09-11T23:31:40.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.056+0000 7fa5fde76640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557e02291060 0x557e02291440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:40.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.056+0000 7fa5fde76640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557e02291060 0x557e02291440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53480/0 (socket says 172.21.15.26:53480) 2024-09-11T23:31:40.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.056+0000 7fa5fde76640 1 -- 172.21.15.26:0/2645889592 learned_addr learned my addr 172.21.15.26:0/2645889592 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:31:40.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.056+0000 7fa5ea7fc640 1 --2- 172.21.15.26:0/691449203 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fa5e40a2cf0 0x7fa5e40a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:40.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.056+0000 7fa5fd675640 1 --2- 172.21.15.26:0/2645889592 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557e02291980 0x557e0229bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:40.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.056+0000 7fa5fce74640 1 --2- 172.21.15.26:0/2645889592 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557e0229c240 0x557e0229e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:40.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.056+0000 7fa5fde76640 1 -- 172.21.15.26:0/2645889592 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557e0229c240 msgr2=0x557e0229e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:31:40.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.056+0000 7fa5fde76640 1 --2- 172.21.15.26:0/2645889592 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557e0229c240 0x557e0229e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:40.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.056+0000 7fa5fde76640 1 -- 172.21.15.26:0/2645889592 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557e02291980 msgr2=0x557e0229bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:31:40.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.056+0000 7fa5fde76640 1 --2- 172.21.15.26:0/2645889592 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557e02291980 0x557e0229bd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:40.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.056+0000 7fa5fde76640 1 -- 172.21.15.26:0/2645889592 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e0218ed40 con 0x557e02291060 2024-09-11T23:31:40.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.056+0000 7fa5fce74640 1 --2- 172.21.15.26:0/2645889592 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557e0229c240 0x557e0229e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:31:40.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.056+0000 7fa5ea7fc640 1 -- 172.21.15.26:0/691449203 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8542+0+0 (secure 0 0 0) 0x7fa5ec0cf0e0 con 0x7fa5f8103ff0 2024-09-11T23:31:40.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.056+0000 7fa5fde76640 1 --2- 172.21.15.26:0/691449203 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fa5e40a2cf0 0x7fa5e40a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:40.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.057+0000 7fa5fde76640 1 --2- 172.21.15.26:0/2645889592 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557e02291060 0x557e02291440 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7fa5f00099d0 tx=0x7fa5f0009a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b868f3ef0335528 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:31:40.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.057+0000 7fa5e37fe640 1 -- 172.21.15.26:0/2645889592 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5f0004040 con 0x557e02291060 2024-09-11T23:31:40.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.057+0000 7fa5e37fe640 1 -- 172.21.15.26:0/2645889592 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5f0036070 con 0x557e02291060 2024-09-11T23:31:40.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.057+0000 7fa5e37fe640 1 -- 172.21.15.26:0/2645889592 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5f002fbb0 con 0x557e02291060 2024-09-11T23:31:40.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.058+0000 7fa601cb2740 1 -- 172.21.15.26:0/2645889592 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557e02291060 msgr2=0x557e02291440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:31:40.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.058+0000 7fa5fde76640 1 --2- 172.21.15.26:0/691449203 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fa5e40a2cf0 0x7fa5e40a51b0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fa5f004b000 tx=0x7fa5f0048000 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:31:40.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.058+0000 7fa601cb2740 1 --2- 172.21.15.26:0/2645889592 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557e02291060 0x557e02291440 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7fa5f00099d0 tx=0x7fa5f0009a00 comp rx=0 tx=0).stop 2024-09-11T23:31:40.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.058+0000 7fa601cb2740 1 -- 172.21.15.26:0/2645889592 shutdown_connections 2024-09-11T23:31:40.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.058+0000 7fa601cb2740 1 --2- 172.21.15.26:0/2645889592 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557e02291980 0x557e0229bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:40.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.058+0000 7fa601cb2740 1 --2- 172.21.15.26:0/2645889592 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557e0229c240 0x557e0229e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:40.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.058+0000 7fa601cb2740 1 --2- 172.21.15.26:0/2645889592 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557e02291060 0x557e02291440 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:40.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.058+0000 7fa601cb2740 1 -- 172.21.15.26:0/2645889592 >> 172.21.15.26:0/2645889592 conn(0x557e0228d5f0 msgr2=0x557e0228d9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:31:40.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.058+0000 7fa601cb2740 1 -- 172.21.15.26:0/2645889592 shutdown_connections 2024-09-11T23:31:40.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.058+0000 7fa601cb2740 1 -- 172.21.15.26:0/2645889592 wait complete. 2024-09-11T23:31:40.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.058+0000 7fa601cb2740 1 -- start start 2024-09-11T23:31:40.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.058+0000 7fa601cb2740 1 Processor -- start 2024-09-11T23:31:40.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.059+0000 7fa601cb2740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557e02291060 0x557e02299130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:40.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.059+0000 7fa601cb2740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557e02291980 0x557e02299670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:40.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.059+0000 7fa601cb2740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557e0229c240 0x557e02299bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:40.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.059+0000 7fa601cb2740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x557e0229ec90 con 0x557e02291980 2024-09-11T23:31:40.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.059+0000 7fa601cb2740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x557e0217ef10 con 0x557e02291060 2024-09-11T23:31:40.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.059+0000 7fa601cb2740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x557e0229ee10 con 0x557e0229c240 2024-09-11T23:31:40.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.059+0000 7fa601cb2740 10 client.0 _ll_register_callbacks cb 0x557e021a2db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:31:40.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.059+0000 7fa5fce74640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557e02291980 0x557e02299670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:40.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.059+0000 7fa5fce74640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557e02291980 0x557e02299670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53496/0 (socket says 172.21.15.26:53496) 2024-09-11T23:31:40.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.059+0000 7fa5fce74640 1 -- 172.21.15.26:0/2361355393 learned_addr learned my addr 172.21.15.26:0/2361355393 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:31:40.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.059+0000 7fa5fde76640 1 --2- 172.21.15.26:0/2361355393 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557e0229c240 0x557e02299bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:40.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.059+0000 7fa5fd675640 1 --2- 172.21.15.26:0/2361355393 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557e02291060 0x557e02299130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:40.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.059+0000 7fa5fce74640 1 -- 172.21.15.26:0/2361355393 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557e0229c240 msgr2=0x557e02299bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:31:40.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.059+0000 7fa5fce74640 1 --2- 172.21.15.26:0/2361355393 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557e0229c240 0x557e02299bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:40.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.059+0000 7fa5fce74640 1 -- 172.21.15.26:0/2361355393 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557e02291060 msgr2=0x557e02299130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:31:40.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.059+0000 7fa5fce74640 1 --2- 172.21.15.26:0/2361355393 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557e02291060 0x557e02299130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:40.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.059+0000 7fa5fce74640 1 -- 172.21.15.26:0/2361355393 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e01f05580 con 0x557e02291980 2024-09-11T23:31:40.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.059+0000 7fa5fce74640 1 --2- 172.21.15.26:0/2361355393 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557e02291980 0x557e02299670 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7fa5f40104c0 tx=0x7fa5f4010980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28efee431c47d5a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:31:40.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.059+0000 7fa5e0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:31:40.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.059+0000 7fa5e0ff9640 1 -- 172.21.15.26:0/2361355393 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5f4012920 con 0x557e02291980 2024-09-11T23:31:40.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.059+0000 7fa5e0ff9640 1 -- 172.21.15.26:0/2361355393 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8542+0+0 (secure 0 0 0) 0x7fa5f4012ac0 con 0x557e02291980 2024-09-11T23:31:40.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.059+0000 7fa601cb2740 1 -- 172.21.15.26:0/2361355393 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa5f0036070 con 0x557e02291980 2024-09-11T23:31:40.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.060+0000 7fa5e0ff9640 1 -- 172.21.15.26:0/2361355393 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7fa5f4012cc0 con 0x557e02291980 2024-09-11T23:31:40.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.060+0000 7fa601cb2740 10 client.17319 fetch_fsmap learned FSMap version 69 2024-09-11T23:31:40.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.060+0000 7fa601cb2740 1 -- 172.21.15.26:0/2361355393 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) -- 0x557e01fcf960 con 0x557e02291980 2024-09-11T23:31:40.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.060+0000 7fa5e0ff9640 1 -- 172.21.15.26:0/2361355393 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5f0036070 con 0x557e02291980 2024-09-11T23:31:40.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.060+0000 7fa5e0ff9640 1 -- 172.21.15.26:0/2361355393 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 69) ==== 8438+0+0 (secure 0 0 0) 0x7fa5f400f040 con 0x557e02291980 2024-09-11T23:31:40.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.060+0000 7fa601cb2740 10 client.17319 fetch_fsmap finished waiting for FSMap version 69 2024-09-11T23:31:40.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.060+0000 7fa601cb2740 10 client.17319 resolve_mds: validated gid 24485 aka daemon mds.b 2024-09-11T23:31:40.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.060+0000 7fa601cb2740 20 client.17319 populate_metadata read hostname 'smithi026' 2024-09-11T23:31:40.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.060+0000 7fa601cb2740 1 --2- 172.21.15.26:0/2361355393 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x557e022abd10 0x557e022ae170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:40.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.060+0000 7fa601cb2740 4 client.17319 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:31:40.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.060+0000 7fa601cb2740 1 -- 172.21.15.26:0/2361355393 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557e02203b90 con 0x557e022abd10 2024-09-11T23:31:40.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.060+0000 7fa5fd675640 1 --2- 172.21.15.26:0/2361355393 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x557e022abd10 0x557e022ae170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:40.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.061+0000 7fa5e0ff9640 10 client.17319 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:31:40.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.061+0000 7fa5fd675640 1 --2- 172.21.15.26:0/2361355393 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x557e022abd10 0x557e022ae170 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=434d92ce268d252e server_cookie=d0d1af5ea95d7380 in_seq=0 out_seq=0 2024-09-11T23:31:40.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.061+0000 7fa5e0ff9640 1 -- 172.21.15.26:0/2361355393 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557e02203b90 con 0x557e022abd10 2024-09-11T23:31:40.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.061+0000 7fa5e0ff9640 10 client.17319 handle_command_reply: tid=0 2024-09-11T23:31:40.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.061+0000 7fa601cb2740 1 client.17319 shutdown 2024-09-11T23:31:40.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.061+0000 7fa601cb2740 1 -- 172.21.15.26:0/2361355393 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557e02291980 msgr2=0x557e02299670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:31:40.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.061+0000 7fa601cb2740 1 --2- 172.21.15.26:0/2361355393 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557e02291980 0x557e02299670 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7fa5f40104c0 tx=0x7fa5f4010980 comp rx=0 tx=0).stop 2024-09-11T23:31:40.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.061+0000 7fa601cb2740 1 -- 172.21.15.26:0/2361355393 shutdown_connections 2024-09-11T23:31:40.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.061+0000 7fa601cb2740 1 --2- 172.21.15.26:0/2361355393 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x557e022abd10 0x557e022ae170 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:40.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.061+0000 7fa601cb2740 1 --2- 172.21.15.26:0/2361355393 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557e0229c240 0x557e02299bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:40.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.061+0000 7fa601cb2740 1 --2- 172.21.15.26:0/2361355393 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557e02291980 0x557e02299670 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:40.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.061+0000 7fa5e0ff9640 0 client.17319 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:31:40.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.061+0000 7fa601cb2740 1 --2- 172.21.15.26:0/2361355393 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557e02291060 0x557e02299130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:40.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.061+0000 7fa601cb2740 1 -- 172.21.15.26:0/2361355393 >> 172.21.15.26:0/2361355393 conn(0x557e0228d5f0 msgr2=0x557e0228c750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:31:40.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.062+0000 7fa601cb2740 1 -- 172.21.15.26:0/2361355393 shutdown_connections 2024-09-11T23:31:40.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.062+0000 7fa601cb2740 1 -- 172.21.15.26:0/2361355393 wait complete. 2024-09-11T23:31:40.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.062+0000 7fa601cb2740 20 client.17319 trim_cache size 0 max 16384 2024-09-11T23:31:40.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.079+0000 7fa601cb2740 1 Processor -- start 2024-09-11T23:31:40.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.079+0000 7fa601cb2740 1 -- start start 2024-09-11T23:31:40.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.079+0000 7fa601cb2740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557e02291060 0x557e02293b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:40.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.079+0000 7fa601cb2740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557e02291980 0x557e0228c640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:40.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.079+0000 7fa601cb2740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557e0229c240 0x557e022b3a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:40.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.079+0000 7fa601cb2740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fa5f81afb60 con 0x557e02291060 2024-09-11T23:31:40.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.079+0000 7fa601cb2740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fa5f81b1970 con 0x557e02291980 2024-09-11T23:31:40.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.079+0000 7fa601cb2740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fa5f81b02c0 con 0x557e0229c240 2024-09-11T23:31:40.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.079+0000 7fa5fd675640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557e02291060 0x557e02293b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:40.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.079+0000 7fa5fd675640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557e02291060 0x557e02293b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53506/0 (socket says 172.21.15.26:53506) 2024-09-11T23:31:40.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.079+0000 7fa5fd675640 1 -- 172.21.15.26:0/4270681591 learned_addr learned my addr 172.21.15.26:0/4270681591 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:31:40.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.079+0000 7fa5fce74640 1 --2- 172.21.15.26:0/4270681591 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557e02291980 0x557e0228c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:40.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.079+0000 7fa5fde76640 1 --2- 172.21.15.26:0/4270681591 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557e0229c240 0x557e022b3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:40.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.079+0000 7fa5fd675640 1 -- 172.21.15.26:0/4270681591 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557e0229c240 msgr2=0x557e022b3a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:31:40.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.079+0000 7fa5fd675640 1 --2- 172.21.15.26:0/4270681591 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557e0229c240 0x557e022b3a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:40.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.079+0000 7fa5fd675640 1 -- 172.21.15.26:0/4270681591 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557e02291980 msgr2=0x557e0228c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:31:40.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.079+0000 7fa5fd675640 1 --2- 172.21.15.26:0/4270681591 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557e02291980 0x557e0228c640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:40.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.079+0000 7fa5fd675640 1 -- 172.21.15.26:0/4270681591 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e022917b0 con 0x557e02291060 2024-09-11T23:31:40.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.079+0000 7fa5fde76640 1 --2- 172.21.15.26:0/4270681591 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557e0229c240 0x557e022b3a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:31:40.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.080+0000 7fa5fd675640 1 --2- 172.21.15.26:0/4270681591 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557e02291060 0x557e02293b10 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7fa5ec009610 tx=0x7fa5ec0d1660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a33c6544974a07ee server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:31:40.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.080+0000 7fa5e2ffd640 1 -- 172.21.15.26:0/4270681591 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5ec020040 con 0x557e02291060 2024-09-11T23:31:40.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.080+0000 7fa5e2ffd640 1 -- 172.21.15.26:0/4270681591 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5ec09fa40 con 0x557e02291060 2024-09-11T23:31:40.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.080+0000 7fa5e2ffd640 1 -- 172.21.15.26:0/4270681591 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5ec0a3e70 con 0x557e02291060 2024-09-11T23:31:40.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.080+0000 7fa601cb2740 1 -- 172.21.15.26:0/4270681591 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557e02291060 msgr2=0x557e02293b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:31:40.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.080+0000 7fa601cb2740 1 --2- 172.21.15.26:0/4270681591 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557e02291060 0x557e02293b10 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7fa5ec009610 tx=0x7fa5ec0d1660 comp rx=0 tx=0).stop 2024-09-11T23:31:40.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.080+0000 7fa601cb2740 1 -- 172.21.15.26:0/4270681591 shutdown_connections 2024-09-11T23:31:40.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.080+0000 7fa601cb2740 1 --2- 172.21.15.26:0/4270681591 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557e02291980 0x557e0228c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:40.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.080+0000 7fa601cb2740 1 --2- 172.21.15.26:0/4270681591 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557e0229c240 0x557e022b3a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:40.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.080+0000 7fa601cb2740 1 --2- 172.21.15.26:0/4270681591 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557e02291060 0x557e02293b10 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:40.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.080+0000 7fa601cb2740 1 -- 172.21.15.26:0/4270681591 >> 172.21.15.26:0/4270681591 conn(0x557e0228d5f0 msgr2=0x557e0228bf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:31:40.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.080+0000 7fa601cb2740 1 -- 172.21.15.26:0/4270681591 shutdown_connections 2024-09-11T23:31:40.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.080+0000 7fa601cb2740 1 -- 172.21.15.26:0/4270681591 wait complete. 2024-09-11T23:31:40.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.081+0000 7fa601cb2740 1 -- start start 2024-09-11T23:31:40.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.081+0000 7fa601cb2740 1 Processor -- start 2024-09-11T23:31:40.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.081+0000 7fa601cb2740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557e02291060 0x557e022a4240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:40.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.081+0000 7fa601cb2740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557e02291980 0x557e022a4780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:40.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.081+0000 7fa601cb2740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557e0229c240 0x557e022a1030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:40.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.081+0000 7fa601cb2740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fa5f81b11e0 con 0x557e02291980 2024-09-11T23:31:40.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.081+0000 7fa601cb2740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fa5f81b0a50 con 0x557e0229c240 2024-09-11T23:31:40.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.081+0000 7fa601cb2740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fa5f81adb60 con 0x557e02291060 2024-09-11T23:31:40.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.081+0000 7fa601cb2740 10 client.0 _ll_register_callbacks cb 0x7fa5f400bcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:31:40.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.081+0000 7fa5fce74640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557e02291980 0x557e022a4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:40.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.081+0000 7fa5fce74640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557e02291980 0x557e022a4780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53522/0 (socket says 172.21.15.26:53522) 2024-09-11T23:31:40.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.081+0000 7fa5fce74640 1 -- 172.21.15.26:0/3233629045 learned_addr learned my addr 172.21.15.26:0/3233629045 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:31:40.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.081+0000 7fa5fde76640 1 --2- 172.21.15.26:0/3233629045 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557e0229c240 0x557e022a1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:40.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.081+0000 7fa5fd675640 1 --2- 172.21.15.26:0/3233629045 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557e02291060 0x557e022a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:40.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.082+0000 7fa5fce74640 1 -- 172.21.15.26:0/3233629045 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557e02291060 msgr2=0x557e022a4240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:31:40.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.082+0000 7fa5fce74640 1 --2- 172.21.15.26:0/3233629045 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557e02291060 0x557e022a4240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:40.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.082+0000 7fa5fce74640 1 -- 172.21.15.26:0/3233629045 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557e0229c240 msgr2=0x557e022a1030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:31:40.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.082+0000 7fa5fce74640 1 --2- 172.21.15.26:0/3233629045 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557e0229c240 0x557e022a1030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:40.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.082+0000 7fa5fce74640 1 -- 172.21.15.26:0/3233629045 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e022a16a0 con 0x557e02291980 2024-09-11T23:31:40.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.082+0000 7fa5fce74640 1 --2- 172.21.15.26:0/3233629045 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557e02291980 0x557e022a4780 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fa5f400bc80 tx=0x7fa5f40119e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1e4d278e3db3395 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:31:40.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.082+0000 7fa5e0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:31:40.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.082+0000 7fa5e0ff9640 1 -- 172.21.15.26:0/3233629045 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5f4012e60 con 0x557e02291980 2024-09-11T23:31:40.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.082+0000 7fa5e0ff9640 1 -- 172.21.15.26:0/3233629045 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8542+0+0 (secure 0 0 0) 0x7fa5f4012930 con 0x557e02291980 2024-09-11T23:31:40.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.082+0000 7fa601cb2740 1 -- 172.21.15.26:0/3233629045 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa5ec09fa40 con 0x557e02291980 2024-09-11T23:31:40.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.082+0000 7fa5e0ff9640 1 -- 172.21.15.26:0/3233629045 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7fa5f4039050 con 0x557e02291980 2024-09-11T23:31:40.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.082+0000 7fa601cb2740 10 client.17325 fetch_fsmap learned FSMap version 69 2024-09-11T23:31:40.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.082+0000 7fa601cb2740 1 -- 172.21.15.26:0/3233629045 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) -- 0x557e02317330 con 0x557e02291980 2024-09-11T23:31:40.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.082+0000 7fa5e0ff9640 1 -- 172.21.15.26:0/3233629045 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5ec09fa40 con 0x557e02291980 2024-09-11T23:31:40.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.083+0000 7fa5e0ff9640 1 -- 172.21.15.26:0/3233629045 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 69) ==== 8438+0+0 (secure 0 0 0) 0x7fa5f4004870 con 0x557e02291980 2024-09-11T23:31:40.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.083+0000 7fa601cb2740 10 client.17325 fetch_fsmap finished waiting for FSMap version 69 2024-09-11T23:31:40.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.083+0000 7fa601cb2740 10 client.17325 resolve_mds: validated gid 24485 aka daemon mds.b 2024-09-11T23:31:40.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.083+0000 7fa601cb2740 20 client.17325 populate_metadata read hostname 'smithi026' 2024-09-11T23:31:40.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.083+0000 7fa601cb2740 1 --2- 172.21.15.26:0/3233629045 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x557e02317bc0 0x557e02319fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:40.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.083+0000 7fa601cb2740 4 client.17325 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "a269f5e9", "release": true, "await": true}] 2024-09-11T23:31:40.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.083+0000 7fa601cb2740 1 -- 172.21.15.26:0/3233629045 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "quiesce db", "set_id": "a269f5e9", "release": true, "await": true}) -- 0x557e021fe0a0 con 0x557e02317bc0 2024-09-11T23:31:40.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.083+0000 7fa5fd675640 1 --2- 172.21.15.26:0/3233629045 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x557e02317bc0 0x557e02319fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:40.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.083+0000 7fa5fd675640 1 --2- 172.21.15.26:0/3233629045 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x557e02317bc0 0x557e02319fe0 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=66dfd19fc4f8e85f server_cookie=317deea63f86e5b2 in_seq=0 out_seq=0 2024-09-11T23:31:40.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:40.083+0000 7fa5e0ff9640 10 client.17325 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:31:40.238 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:31:39 smithi136 ceph-mon[35121]: pgmap v415: 97 pgs: 97 active+clean; 2.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 3.4 KiB/s wr, 8 op/s 2024-09-11T23:31:40.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:31:39 smithi026 ceph-mon[31172]: pgmap v415: 97 pgs: 97 active+clean; 2.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 3.4 KiB/s wr, 8 op/s 2024-09-11T23:31:40.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:31:39 smithi076 ceph-mon[35296]: pgmap v415: 97 pgs: 97 active+clean; 2.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 3.4 KiB/s wr, 8 op/s 2024-09-11T23:31:41.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:41.072+0000 7fa5e0ff9640 1 -- 172.21.15.26:0/3233629045 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x557e021fe0a0 con 0x557e02317bc0 2024-09-11T23:31:41.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:41.072+0000 7fa5e0ff9640 10 client.17325 handle_command_reply: tid=0 2024-09-11T23:31:41.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:41.072+0000 7fa601cb2740 1 client.17325 shutdown 2024-09-11T23:31:41.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:41.072+0000 7fa601cb2740 1 -- 172.21.15.26:0/3233629045 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557e02291980 msgr2=0x557e022a4780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:31:41.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:41.072+0000 7fa601cb2740 1 --2- 172.21.15.26:0/3233629045 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557e02291980 0x557e022a4780 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fa5f400bc80 tx=0x7fa5f40119e0 comp rx=0 tx=0).stop 2024-09-11T23:31:41.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:41.072+0000 7fa601cb2740 1 -- 172.21.15.26:0/3233629045 shutdown_connections 2024-09-11T23:31:41.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:41.072+0000 7fa601cb2740 1 --2- 172.21.15.26:0/3233629045 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557e0229c240 0x557e022a1030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:41.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:41.072+0000 7fa601cb2740 1 --2- 172.21.15.26:0/3233629045 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x557e02317bc0 0x557e02319fe0 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:41.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:41.072+0000 7fa601cb2740 1 --2- 172.21.15.26:0/3233629045 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557e02291980 0x557e022a4780 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:41.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:41.072+0000 7fa601cb2740 1 --2- 172.21.15.26:0/3233629045 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557e02291060 0x557e022a4240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:41.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:41.072+0000 7fa601cb2740 1 -- 172.21.15.26:0/3233629045 >> 172.21.15.26:0/3233629045 conn(0x557e0228d5f0 msgr2=0x557e0228c750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:31:41.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:41.072+0000 7fa5e0ff9640 0 client.17325 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:31:41.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:41.072+0000 7fa601cb2740 1 -- 172.21.15.26:0/3233629045 shutdown_connections 2024-09-11T23:31:41.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:41.073+0000 7fa601cb2740 1 -- 172.21.15.26:0/3233629045 wait complete. 2024-09-11T23:31:41.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:41.073+0000 7fa601cb2740 20 client.17325 trim_cache size 0 max 16384 2024-09-11T23:31:41.074 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-11T23:31:41.074 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 69, 2024-09-11T23:31:41.074 INFO:teuthology.orchestra.run.smithi026.stdout: "leader": 24485, 2024-09-11T23:31:41.074 INFO:teuthology.orchestra.run.smithi026.stdout: "set_version": 48, 2024-09-11T23:31:41.074 INFO:teuthology.orchestra.run.smithi026.stdout: "sets": { 2024-09-11T23:31:41.074 INFO:teuthology.orchestra.run.smithi026.stdout: "a269f5e9": { 2024-09-11T23:31:41.074 INFO:teuthology.orchestra.run.smithi026.stdout: "version": 48, 2024-09-11T23:31:41.074 INFO:teuthology.orchestra.run.smithi026.stdout: "age_ref": 0.0, 2024-09-11T23:31:41.074 INFO:teuthology.orchestra.run.smithi026.stdout: "state": { 2024-09-11T23:31:41.075 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "RELEASED", 2024-09-11T23:31:41.075 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.0 2024-09-11T23:31:41.075 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:31:41.075 INFO:teuthology.orchestra.run.smithi026.stdout: "timeout": 90.0, 2024-09-11T23:31:41.075 INFO:teuthology.orchestra.run.smithi026.stdout: "expiration": 147.2, 2024-09-11T23:31:41.075 INFO:teuthology.orchestra.run.smithi026.stdout: "members": { 2024-09-11T23:31:41.075 INFO:teuthology.orchestra.run.smithi026.stdout: "file:/": { 2024-09-11T23:31:41.075 INFO:teuthology.orchestra.run.smithi026.stdout: "excluded": false, 2024-09-11T23:31:41.075 INFO:teuthology.orchestra.run.smithi026.stdout: "state": { 2024-09-11T23:31:41.075 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "RELEASED", 2024-09-11T23:31:41.075 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.0 2024-09-11T23:31:41.075 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:31:41.075 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:31:41.076 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:31:41.076 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:31:41.076 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:31:41.076 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-11T23:31:41.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:41.074+0000 7fa5ff8d9640 1 -- 172.21.15.26:0/691449203 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fa5e40a2cf0 msgr2=0x7fa5e40a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:31:41.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:41.074+0000 7fa5ff8d9640 1 --2- 172.21.15.26:0/691449203 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fa5e40a2cf0 0x7fa5e40a51b0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fa5f004b000 tx=0x7fa5f0048000 comp rx=0 tx=0).stop 2024-09-11T23:31:41.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:41.074+0000 7fa5ff8d9640 1 -- 172.21.15.26:0/691449203 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa5f8103ff0 msgr2=0x7fa5f819a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:31:41.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:41.074+0000 7fa5ff8d9640 1 --2- 172.21.15.26:0/691449203 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa5f8103ff0 0x7fa5f819a5e0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7fa5ec00d950 tx=0x7fa5ec00de10 comp rx=0 tx=0).stop 2024-09-11T23:31:41.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:41.074+0000 7fa5ff8d9640 1 -- 172.21.15.26:0/691449203 shutdown_connections 2024-09-11T23:31:41.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:41.074+0000 7fa5ff8d9640 1 --2- 172.21.15.26:0/691449203 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fa5e40a2cf0 0x7fa5e40a51b0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:41.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:41.074+0000 7fa5ff8d9640 1 --2- 172.21.15.26:0/691449203 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa5f8109120 0x7fa5f819eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:41.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:41.074+0000 7fa5ff8d9640 1 --2- 172.21.15.26:0/691449203 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa5f81049a0 0x7fa5f819ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:41.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:41.074+0000 7fa5ff8d9640 1 --2- 172.21.15.26:0/691449203 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa5f8103ff0 0x7fa5f819a5e0 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:41.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:41.074+0000 7fa5ff8d9640 1 -- 172.21.15.26:0/691449203 >> 172.21.15.26:0/691449203 conn(0x7fa5f80ffab0 msgr2=0x7fa5f8100d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:31:41.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:41.074+0000 7fa5ff8d9640 1 -- 172.21.15.26:0/691449203 shutdown_connections 2024-09-11T23:31:41.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:31:41.074+0000 7fa5ff8d9640 1 -- 172.21.15.26:0/691449203 wait complete. 2024-09-11T23:31:41.959 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'a269f5e9', total seconds elapsed: 32.1 2024-09-11T23:31:42.238 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:31:41 smithi136 ceph-mon[35121]: pgmap v416: 97 pgs: 97 active+clean; 2.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 6.4 KiB/s wr, 14 op/s 2024-09-11T23:31:42.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:31:41 smithi026 ceph-mon[31172]: pgmap v416: 97 pgs: 97 active+clean; 2.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 6.4 KiB/s wr, 14 op/s 2024-09-11T23:31:42.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:31:41 smithi076 ceph-mon[35296]: pgmap v416: 97 pgs: 97 active+clean; 2.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 6.4 KiB/s wr, 14 op/s 2024-09-11T23:31:44.238 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:31:43 smithi136 ceph-mon[35121]: pgmap v417: 97 pgs: 97 active+clean; 2.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 27 KiB/s wr, 11 op/s 2024-09-11T23:31:44.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:31:43 smithi026 ceph-mon[31172]: pgmap v417: 97 pgs: 97 active+clean; 2.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 27 KiB/s wr, 11 op/s 2024-09-11T23:31:44.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:31:43 smithi076 ceph-mon[35296]: pgmap v417: 97 pgs: 97 active+clean; 2.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 27 KiB/s wr, 11 op/s 2024-09-11T23:31:46.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:31:45 smithi136 ceph-mon[35121]: pgmap v418: 97 pgs: 97 active+clean; 2.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 47 KiB/s wr, 36 op/s 2024-09-11T23:31:46.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:31:45 smithi026 ceph-mon[31172]: pgmap v418: 97 pgs: 97 active+clean; 2.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 47 KiB/s wr, 36 op/s 2024-09-11T23:31:46.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:31:45 smithi076 ceph-mon[35296]: pgmap v418: 97 pgs: 97 active+clean; 2.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 47 KiB/s wr, 36 op/s 2024-09-11T23:31:48.238 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:31:47 smithi136 ceph-mon[35121]: pgmap v419: 97 pgs: 97 active+clean; 2.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 724 KiB/s rd, 746 KiB/s wr, 88 op/s 2024-09-11T23:31:48.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:31:47 smithi026 ceph-mon[31172]: pgmap v419: 97 pgs: 97 active+clean; 2.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 724 KiB/s rd, 746 KiB/s wr, 88 op/s 2024-09-11T23:31:48.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:31:47 smithi076 ceph-mon[35296]: pgmap v419: 97 pgs: 97 active+clean; 2.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 724 KiB/s rd, 746 KiB/s wr, 88 op/s 2024-09-11T23:31:48.997 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:48.997+0000 7ff07e623640 1 -- 172.21.15.26:0/2758764842 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff078101370 msgr2=0x7ff07810d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:31:48.997 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:48.997+0000 7ff07e623640 1 --2- 172.21.15.26:0/2758764842 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff078101370 0x7ff07810d8a0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7ff070009cd0 tx=0x7ff07002a8e0 comp rx=0 tx=0).stop 2024-09-11T23:31:48.998 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:48.997+0000 7ff07e623640 1 -- 172.21.15.26:0/2758764842 shutdown_connections 2024-09-11T23:31:48.998 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:48.997+0000 7ff07e623640 1 --2- 172.21.15.26:0/2758764842 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff078101370 0x7ff07810d8a0 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:48.998 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:48.997+0000 7ff07e623640 1 --2- 172.21.15.26:0/2758764842 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff0781009d0 0x7ff078100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:48.998 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:48.997+0000 7ff07e623640 1 --2- 172.21.15.26:0/2758764842 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff0781069e0 0x7ff078106dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:48.998 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:48.997+0000 7ff07e623640 1 -- 172.21.15.26:0/2758764842 >> 172.21.15.26:0/2758764842 conn(0x7ff0780fc6b0 msgr2=0x7ff0780fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:31:48.998 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:48.998+0000 7ff07e623640 1 -- 172.21.15.26:0/2758764842 shutdown_connections 2024-09-11T23:31:48.998 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:48.998+0000 7ff07e623640 1 -- 172.21.15.26:0/2758764842 wait complete. 2024-09-11T23:31:49.000 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:48.998+0000 7ff07e623640 1 Processor -- start 2024-09-11T23:31:49.000 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:48.998+0000 7ff07e623640 1 -- start start 2024-09-11T23:31:49.000 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:48.999+0000 7ff07e623640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff0781009d0 0x7ff078198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:49.000 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:48.999+0000 7ff07e623640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff078101370 0x7ff078198860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:49.000 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:48.999+0000 7ff07e623640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff0781069e0 0x7ff07819cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:49.000 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:48.999+0000 7ff07e623640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff078111f10 con 0x7ff0781069e0 2024-09-11T23:31:49.000 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:48.999+0000 7ff07e623640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7ff078111d90 con 0x7ff0781009d0 2024-09-11T23:31:49.000 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:48.999+0000 7ff07e623640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ff078112090 con 0x7ff078101370 2024-09-11T23:31:49.001 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:48.999+0000 7ff077fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff0781009d0 0x7ff078198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:49.001 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:48.999+0000 7ff07cbc0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff0781069e0 0x7ff07819cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:49.001 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:48.999+0000 7ff0777fe640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff078101370 0x7ff078198860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:49.001 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:48.999+0000 7ff0777fe640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff078101370 0x7ff078198860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:35898/0 (socket says 172.21.15.26:35898) 2024-09-11T23:31:49.001 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:48.999+0000 7ff0777fe640 1 -- 172.21.15.26:0/3792688058 learned_addr learned my addr 172.21.15.26:0/3792688058 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:31:49.001 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.000+0000 7ff077fff640 1 -- 172.21.15.26:0/3792688058 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff078101370 msgr2=0x7ff078198860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:31:49.001 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.000+0000 7ff077fff640 1 --2- 172.21.15.26:0/3792688058 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff078101370 0x7ff078198860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:49.001 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.000+0000 7ff077fff640 1 -- 172.21.15.26:0/3792688058 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff0781069e0 msgr2=0x7ff07819cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:31:49.001 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.000+0000 7ff077fff640 1 --2- 172.21.15.26:0/3792688058 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff0781069e0 0x7ff07819cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:49.001 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.000+0000 7ff077fff640 1 -- 172.21.15.26:0/3792688058 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff07819d310 con 0x7ff0781009d0 2024-09-11T23:31:49.001 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.000+0000 7ff07cbc0640 1 --2- 172.21.15.26:0/3792688058 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff0781069e0 0x7ff07819cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:31:49.001 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.000+0000 7ff077fff640 1 --2- 172.21.15.26:0/3792688058 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff0781009d0 0x7ff078198320 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7ff06800e9f0 tx=0x7ff06800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:31:49.001 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.000+0000 7ff0757fa640 1 -- 172.21.15.26:0/3792688058 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff06800ccc0 con 0x7ff0781009d0 2024-09-11T23:31:49.002 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.000+0000 7ff0757fa640 1 -- 172.21.15.26:0/3792688058 <== mon.2 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff068010910 con 0x7ff0781009d0 2024-09-11T23:31:49.002 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.000+0000 7ff0757fa640 1 -- 172.21.15.26:0/3792688058 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff068018960 con 0x7ff0781009d0 2024-09-11T23:31:49.002 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.000+0000 7ff07e623640 1 -- 172.21.15.26:0/3792688058 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff07819d600 con 0x7ff0781009d0 2024-09-11T23:31:49.002 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.000+0000 7ff07e623640 1 -- 172.21.15.26:0/3792688058 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0781a4e90 con 0x7ff0781009d0 2024-09-11T23:31:49.003 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.004+0000 7ff0757fa640 1 -- 172.21.15.26:0/3792688058 <== mon.2 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7ff068010ac0 con 0x7ff0781009d0 2024-09-11T23:31:49.003 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.004+0000 7ff0813bc740 1 Processor -- start 2024-09-11T23:31:49.003 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.004+0000 7ff0813bc740 1 -- start start 2024-09-11T23:31:49.005 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.004+0000 7ff0813bc740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cb8d08fbc0 0x55cb8d08ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:49.005 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.004+0000 7ff0813bc740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cb8d08b680 0x55cb8d08bae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:49.005 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.004+0000 7ff0813bc740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cb8d08c020 0x55cb8d097ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:49.005 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.004+0000 7ff0813bc740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55cb8cfe62f0 con 0x55cb8d08c020 2024-09-11T23:31:49.005 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.004+0000 7ff0813bc740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55cb8cf8e270 con 0x55cb8d08fbc0 2024-09-11T23:31:49.005 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.004+0000 7ff0813bc740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55cb8cfe22b0 con 0x55cb8d08b680 2024-09-11T23:31:49.005 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.005+0000 7ff07cbc0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cb8d08fbc0 0x55cb8d08ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:49.005 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.005+0000 7ff07cbc0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cb8d08fbc0 0x55cb8d08ffa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:59768/0 (socket says 172.21.15.26:59768) 2024-09-11T23:31:49.005 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.005+0000 7ff07cbc0640 1 -- 172.21.15.26:0/3944366594 learned_addr learned my addr 172.21.15.26:0/3944366594 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:31:49.006 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.005+0000 7ff077fff640 1 --2- 172.21.15.26:0/3944366594 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cb8d08b680 0x55cb8d08bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:49.006 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.005+0000 7ff0777fe640 1 --2- 172.21.15.26:0/3944366594 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cb8d08c020 0x55cb8d097ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:49.006 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.005+0000 7ff07cbc0640 1 -- 172.21.15.26:0/3944366594 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cb8d08b680 msgr2=0x55cb8d08bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:31:49.006 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.005+0000 7ff07cbc0640 1 --2- 172.21.15.26:0/3944366594 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cb8d08b680 0x55cb8d08bae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:49.006 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.005+0000 7ff07cbc0640 1 -- 172.21.15.26:0/3944366594 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cb8d08c020 msgr2=0x55cb8d097ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:31:49.006 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.005+0000 7ff07cbc0640 1 --2- 172.21.15.26:0/3944366594 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cb8d08c020 0x55cb8d097ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:49.006 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.005+0000 7ff07cbc0640 1 -- 172.21.15.26:0/3944366594 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb8cf8d340 con 0x55cb8d08fbc0 2024-09-11T23:31:49.006 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.005+0000 7ff0757fa640 1 --2- 172.21.15.26:0/3792688058 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7ff0600a2da0 0x7ff0600a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:49.006 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.005+0000 7ff077fff640 1 --2- 172.21.15.26:0/3792688058 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7ff0600a2da0 0x7ff0600a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:49.006 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.005+0000 7ff0757fa640 1 -- 172.21.15.26:0/3792688058 <== mon.2 v2:172.21.15.76:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8542+0+0 (secure 0 0 0) 0x7ff0680cf7d0 con 0x7ff0781009d0 2024-09-11T23:31:49.006 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.006+0000 7ff07cbc0640 1 --2- 172.21.15.26:0/3944366594 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cb8d08fbc0 0x55cb8d08ffa0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ff070035000 tx=0x7ff070009a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bf1105cf70e7ada8 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:31:49.007 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.006+0000 7ff077fff640 1 --2- 172.21.15.26:0/3792688058 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7ff0600a2da0 0x7ff0600a5260 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ff068098190 tx=0x7ff068009700 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:31:49.007 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.006+0000 7ff05e7fc640 1 -- 172.21.15.26:0/3944366594 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff070048070 con 0x55cb8d08fbc0 2024-09-11T23:31:49.007 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.006+0000 7ff05e7fc640 1 -- 172.21.15.26:0/3944366594 <== mon.2 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff070036070 con 0x55cb8d08fbc0 2024-09-11T23:31:49.007 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.006+0000 7ff05e7fc640 1 -- 172.21.15.26:0/3944366594 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff070004810 con 0x55cb8d08fbc0 2024-09-11T23:31:49.007 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.007+0000 7ff0813bc740 1 -- 172.21.15.26:0/3944366594 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cb8d08fbc0 msgr2=0x55cb8d08ffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:31:49.008 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.007+0000 7ff0813bc740 1 --2- 172.21.15.26:0/3944366594 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cb8d08fbc0 0x55cb8d08ffa0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ff070035000 tx=0x7ff070009a20 comp rx=0 tx=0).stop 2024-09-11T23:31:49.008 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.007+0000 7ff0813bc740 1 -- 172.21.15.26:0/3944366594 shutdown_connections 2024-09-11T23:31:49.008 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.007+0000 7ff0813bc740 1 --2- 172.21.15.26:0/3944366594 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cb8d08c020 0x55cb8d097ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:49.008 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.007+0000 7ff0813bc740 1 --2- 172.21.15.26:0/3944366594 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cb8d08b680 0x55cb8d08bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:49.008 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.007+0000 7ff0813bc740 1 --2- 172.21.15.26:0/3944366594 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cb8d08fbc0 0x55cb8d08ffa0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:49.008 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.007+0000 7ff0813bc740 1 -- 172.21.15.26:0/3944366594 >> 172.21.15.26:0/3944366594 conn(0x55cb8d07db10 msgr2=0x55cb8d07def0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:31:49.010 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.008+0000 7ff0813bc740 1 -- 172.21.15.26:0/3944366594 shutdown_connections 2024-09-11T23:31:49.010 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.008+0000 7ff0813bc740 1 -- 172.21.15.26:0/3944366594 wait complete. 2024-09-11T23:31:49.010 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.009+0000 7ff0813bc740 1 -- start start 2024-09-11T23:31:49.011 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.009+0000 7ff0813bc740 1 Processor -- start 2024-09-11T23:31:49.011 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.009+0000 7ff0813bc740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cb8d08b680 0x55cb8d094f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:49.011 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.009+0000 7ff0813bc740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cb8d08c020 0x55cb8d0954c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:49.011 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.009+0000 7ff0813bc740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cb8d08fbc0 0x55cb8d095a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:49.011 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.009+0000 7ff0813bc740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55cb8d098060 con 0x55cb8d08b680 2024-09-11T23:31:49.011 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.009+0000 7ff0813bc740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55cb8cf6f290 con 0x55cb8d08fbc0 2024-09-11T23:31:49.011 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.009+0000 7ff0813bc740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55cb8d0981e0 con 0x55cb8d08c020 2024-09-11T23:31:49.011 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.009+0000 7ff0813bc740 10 client.0 _ll_register_callbacks cb 0x55cb8cfa0140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:31:49.011 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.010+0000 7ff07cbc0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cb8d08b680 0x55cb8d094f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:49.011 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.010+0000 7ff07cbc0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cb8d08b680 0x55cb8d094f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36046/0 (socket says 172.21.15.26:36046) 2024-09-11T23:31:49.011 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.010+0000 7ff07cbc0640 1 -- 172.21.15.26:0/1724509416 learned_addr learned my addr 172.21.15.26:0/1724509416 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:31:49.012 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.010+0000 7ff0777fe640 1 --2- 172.21.15.26:0/1724509416 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cb8d08c020 0x55cb8d0954c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:49.012 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.010+0000 7ff07cbc0640 1 --2- 172.21.15.26:0/1724509416 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cb8d08fbc0 0x55cb8d095a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:49.012 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.010+0000 7ff07cbc0640 1 -- 172.21.15.26:0/1724509416 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cb8d08c020 msgr2=0x55cb8d0954c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:31:49.012 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.010+0000 7ff07cbc0640 1 --2- 172.21.15.26:0/1724509416 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cb8d08c020 0x55cb8d0954c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:49.012 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.010+0000 7ff07cbc0640 1 -- 172.21.15.26:0/1724509416 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cb8d08fbc0 msgr2=0x55cb8d095a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:31:49.012 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.010+0000 7ff07cbc0640 1 --2- 172.21.15.26:0/1724509416 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cb8d08fbc0 0x55cb8d095a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:49.012 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.010+0000 7ff07cbc0640 1 -- 172.21.15.26:0/1724509416 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb8ccf9210 con 0x55cb8d08b680 2024-09-11T23:31:49.012 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.010+0000 7ff0777fe640 1 --2- 172.21.15.26:0/1724509416 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cb8d08c020 0x55cb8d0954c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:31:49.012 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.010+0000 7ff07cbc0640 1 --2- 172.21.15.26:0/1724509416 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cb8d08b680 0x55cb8d094f80 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7ff070046040 tx=0x7ff070004370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8134b9b95483e0c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:31:49.012 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.010+0000 7ff0577fe640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:31:49.012 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.010+0000 7ff0577fe640 1 -- 172.21.15.26:0/1724509416 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff070055020 con 0x55cb8d08b680 2024-09-11T23:31:49.012 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.010+0000 7ff0577fe640 1 -- 172.21.15.26:0/1724509416 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8542+0+0 (secure 0 0 0) 0x7ff070003d80 con 0x55cb8d08b680 2024-09-11T23:31:49.012 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.011+0000 7ff0813bc740 1 -- 172.21.15.26:0/1724509416 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff070036070 con 0x55cb8d08b680 2024-09-11T23:31:49.013 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.011+0000 7ff0577fe640 1 -- 172.21.15.26:0/1724509416 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7ff070048070 con 0x55cb8d08b680 2024-09-11T23:31:49.013 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.011+0000 7ff0813bc740 10 client.17337 fetch_fsmap learned FSMap version 69 2024-09-11T23:31:49.013 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.011+0000 7ff0813bc740 1 -- 172.21.15.26:0/1724509416 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) -- 0x55cb8cdc5620 con 0x55cb8d08b680 2024-09-11T23:31:49.013 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.011+0000 7ff0577fe640 1 -- 172.21.15.26:0/1724509416 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff070036070 con 0x55cb8d08b680 2024-09-11T23:31:49.015 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.011+0000 7ff0577fe640 1 -- 172.21.15.26:0/1724509416 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 69) ==== 8438+0+0 (secure 0 0 0) 0x7ff070042030 con 0x55cb8d08b680 2024-09-11T23:31:49.015 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.011+0000 7ff0813bc740 10 client.17337 fetch_fsmap finished waiting for FSMap version 69 2024-09-11T23:31:49.015 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.011+0000 7ff0813bc740 10 client.17337 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:31:49.015 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.011+0000 7ff0813bc740 20 client.17337 populate_metadata read hostname 'smithi026' 2024-09-11T23:31:49.015 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.011+0000 7ff0813bc740 1 --2- 172.21.15.26:0/1724509416 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55cb8d0a5330 0x55cb8d0a7720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:49.015 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.011+0000 7ff0813bc740 4 client.17337 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:31:49.015 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.012+0000 7ff0813bc740 1 -- 172.21.15.26:0/1724509416 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cb8cff3a20 con 0x55cb8d0a5330 2024-09-11T23:31:49.015 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.012+0000 7ff0777fe640 1 --2- 172.21.15.26:0/1724509416 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55cb8d0a5330 0x55cb8d0a7720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:49.016 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.012+0000 7ff0777fe640 1 --2- 172.21.15.26:0/1724509416 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55cb8d0a5330 0x55cb8d0a7720 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f36c39e6beef893 server_cookie=9f39cdd49b0bed71 in_seq=0 out_seq=0 2024-09-11T23:31:49.016 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.012+0000 7ff0577fe640 10 client.17337 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:31:49.016 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.013+0000 7ff0577fe640 1 -- 172.21.15.26:0/1724509416 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55cb8cff3a20 con 0x55cb8d0a5330 2024-09-11T23:31:49.016 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.013+0000 7ff0577fe640 10 client.17337 handle_command_reply: tid=0 2024-09-11T23:31:49.016 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.014+0000 7ff0813bc740 1 client.17337 shutdown 2024-09-11T23:31:49.016 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.015+0000 7ff0813bc740 1 -- 172.21.15.26:0/1724509416 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cb8d08b680 msgr2=0x55cb8d094f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:31:49.016 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.015+0000 7ff0813bc740 1 --2- 172.21.15.26:0/1724509416 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cb8d08b680 0x55cb8d094f80 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7ff070046040 tx=0x7ff070004370 comp rx=0 tx=0).stop 2024-09-11T23:31:49.016 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.015+0000 7ff0813bc740 1 -- 172.21.15.26:0/1724509416 shutdown_connections 2024-09-11T23:31:49.016 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.015+0000 7ff0813bc740 1 --2- 172.21.15.26:0/1724509416 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cb8d08fbc0 0x55cb8d095a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:49.016 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.015+0000 7ff0813bc740 1 --2- 172.21.15.26:0/1724509416 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55cb8d0a5330 0x55cb8d0a7720 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:49.016 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.015+0000 7ff0813bc740 1 --2- 172.21.15.26:0/1724509416 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cb8d08c020 0x55cb8d0954c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:49.017 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.015+0000 7ff0813bc740 1 --2- 172.21.15.26:0/1724509416 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cb8d08b680 0x55cb8d094f80 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:49.017 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.015+0000 7ff0813bc740 1 -- 172.21.15.26:0/1724509416 >> 172.21.15.26:0/1724509416 conn(0x55cb8d07db10 msgr2=0x55cb8d0911b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:31:49.017 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.015+0000 7ff0577fe640 0 client.17337 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:31:49.017 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.016+0000 7ff0813bc740 1 -- 172.21.15.26:0/1724509416 shutdown_connections 2024-09-11T23:31:49.017 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.016+0000 7ff0813bc740 1 -- 172.21.15.26:0/1724509416 wait complete. 2024-09-11T23:31:49.017 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.016+0000 7ff0813bc740 20 client.17337 trim_cache size 0 max 16384 2024-09-11T23:31:49.038 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.038+0000 7ff0813bc740 1 Processor -- start 2024-09-11T23:31:49.038 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.038+0000 7ff0813bc740 1 -- start start 2024-09-11T23:31:49.038 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.038+0000 7ff0813bc740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cb8d08b680 0x55cb8d08f470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:49.038 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.038+0000 7ff0813bc740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cb8d08c020 0x55cb8d08ad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:49.038 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.038+0000 7ff0813bc740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cb8d08fbc0 0x55cb8d0a04a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:49.038 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.038+0000 7ff0813bc740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff0781ab760 con 0x55cb8d08b680 2024-09-11T23:31:49.038 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.038+0000 7ff0813bc740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7ff0781ad570 con 0x55cb8d08c020 2024-09-11T23:31:49.039 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.038+0000 7ff0813bc740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ff0781abec0 con 0x55cb8d08fbc0 2024-09-11T23:31:49.039 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.038+0000 7ff07cbc0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cb8d08b680 0x55cb8d08f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:49.039 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.038+0000 7ff07cbc0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cb8d08b680 0x55cb8d08f470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36058/0 (socket says 172.21.15.26:36058) 2024-09-11T23:31:49.039 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.038+0000 7ff07cbc0640 1 -- 172.21.15.26:0/3421779076 learned_addr learned my addr 172.21.15.26:0/3421779076 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:31:49.039 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.039+0000 7ff0777fe640 1 --2- 172.21.15.26:0/3421779076 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cb8d08c020 0x55cb8d08ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:49.040 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.039+0000 7ff07cbc0640 1 --2- 172.21.15.26:0/3421779076 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cb8d08fbc0 0x55cb8d0a04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:49.040 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.039+0000 7ff0777fe640 1 -- 172.21.15.26:0/3421779076 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cb8d08fbc0 msgr2=0x55cb8d0a04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:31:49.040 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.039+0000 7ff0777fe640 1 --2- 172.21.15.26:0/3421779076 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cb8d08fbc0 0x55cb8d0a04a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:49.040 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.039+0000 7ff0777fe640 1 -- 172.21.15.26:0/3421779076 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cb8d08b680 msgr2=0x55cb8d08f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:31:49.040 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.039+0000 7ff0777fe640 1 --2- 172.21.15.26:0/3421779076 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cb8d08b680 0x55cb8d08f470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:49.040 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.039+0000 7ff0777fe640 1 -- 172.21.15.26:0/3421779076 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb8d08b2c0 con 0x55cb8d08c020 2024-09-11T23:31:49.040 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.039+0000 7ff07cbc0640 1 --2- 172.21.15.26:0/3421779076 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cb8d08b680 0x55cb8d08f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:31:49.040 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.040+0000 7ff0777fe640 1 --2- 172.21.15.26:0/3421779076 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cb8d08c020 0x55cb8d08ad80 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x55cb8d098290 tx=0x7ff06c00b410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=36c5b3597fed5fda server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:31:49.040 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.040+0000 7ff05dffb640 1 -- 172.21.15.26:0/3421779076 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff06c005220 con 0x55cb8d08c020 2024-09-11T23:31:49.040 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.040+0000 7ff05dffb640 1 -- 172.21.15.26:0/3421779076 <== mon.2 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff06c004c80 con 0x55cb8d08c020 2024-09-11T23:31:49.040 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.040+0000 7ff05dffb640 1 -- 172.21.15.26:0/3421779076 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff06c01d8c0 con 0x55cb8d08c020 2024-09-11T23:31:49.041 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.041+0000 7ff0813bc740 1 -- 172.21.15.26:0/3421779076 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cb8d08c020 msgr2=0x55cb8d08ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:31:49.041 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.041+0000 7ff0813bc740 1 --2- 172.21.15.26:0/3421779076 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cb8d08c020 0x55cb8d08ad80 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x55cb8d098290 tx=0x7ff06c00b410 comp rx=0 tx=0).stop 2024-09-11T23:31:49.041 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.041+0000 7ff0813bc740 1 -- 172.21.15.26:0/3421779076 shutdown_connections 2024-09-11T23:31:49.042 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.041+0000 7ff0813bc740 1 --2- 172.21.15.26:0/3421779076 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cb8d08c020 0x55cb8d08ad80 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:49.042 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.041+0000 7ff0813bc740 1 --2- 172.21.15.26:0/3421779076 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cb8d08fbc0 0x55cb8d0a04a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:49.042 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.041+0000 7ff0813bc740 1 --2- 172.21.15.26:0/3421779076 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cb8d08b680 0x55cb8d08f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:49.042 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.041+0000 7ff0813bc740 1 -- 172.21.15.26:0/3421779076 >> 172.21.15.26:0/3421779076 conn(0x55cb8d07db10 msgr2=0x55cb8d08a670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:31:49.042 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.042+0000 7ff0813bc740 1 -- 172.21.15.26:0/3421779076 shutdown_connections 2024-09-11T23:31:49.042 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.042+0000 7ff0813bc740 1 -- 172.21.15.26:0/3421779076 wait complete. 2024-09-11T23:31:49.042 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.042+0000 7ff0813bc740 1 -- start start 2024-09-11T23:31:49.044 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.042+0000 7ff0813bc740 1 Processor -- start 2024-09-11T23:31:49.045 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.043+0000 7ff0813bc740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cb8d08b680 0x55cb8d0aa610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:49.045 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.043+0000 7ff0813bc740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cb8d08c020 0x55cb8d0aab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:49.045 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.043+0000 7ff0813bc740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cb8d08fbc0 0x55cb8d0919e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:49.045 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.043+0000 7ff07cbc0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cb8d08b680 0x55cb8d0aa610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:49.045 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.043+0000 7ff07cbc0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cb8d08b680 0x55cb8d0aa610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:59794/0 (socket says 172.21.15.26:59794) 2024-09-11T23:31:49.045 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.043+0000 7ff07cbc0640 1 -- 172.21.15.26:0/287632298 learned_addr learned my addr 172.21.15.26:0/287632298 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:31:49.045 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.043+0000 7ff0813bc740 1 -- 172.21.15.26:0/287632298 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff0781acde0 con 0x55cb8d08fbc0 2024-09-11T23:31:49.045 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.043+0000 7ff0813bc740 1 -- 172.21.15.26:0/287632298 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7ff0781ac650 con 0x55cb8d08b680 2024-09-11T23:31:49.045 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.043+0000 7ff0777fe640 1 --2- 172.21.15.26:0/287632298 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cb8d08c020 0x55cb8d0aab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:49.045 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.043+0000 7ff0813bc740 1 -- 172.21.15.26:0/287632298 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ff0781a9780 con 0x55cb8d08c020 2024-09-11T23:31:49.045 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.043+0000 7ff07cbc0640 1 --2- 172.21.15.26:0/287632298 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cb8d08fbc0 0x55cb8d0919e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:49.045 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.043+0000 7ff0813bc740 10 client.0 _ll_register_callbacks cb 0x7ff070046070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:31:49.046 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.043+0000 7ff07cbc0640 1 -- 172.21.15.26:0/287632298 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cb8d08c020 msgr2=0x55cb8d0aab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:31:49.046 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.043+0000 7ff07cbc0640 1 --2- 172.21.15.26:0/287632298 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cb8d08c020 0x55cb8d0aab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:49.046 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.043+0000 7ff07cbc0640 1 -- 172.21.15.26:0/287632298 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cb8d08fbc0 msgr2=0x55cb8d0919e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:31:49.046 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.043+0000 7ff07cbc0640 1 --2- 172.21.15.26:0/287632298 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cb8d08fbc0 0x55cb8d0919e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:49.046 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.043+0000 7ff07cbc0640 1 -- 172.21.15.26:0/287632298 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb8d092210 con 0x55cb8d08b680 2024-09-11T23:31:49.046 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.044+0000 7ff07cbc0640 1 --2- 172.21.15.26:0/287632298 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cb8d08b680 0x55cb8d0aa610 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ff070005c20 tx=0x7ff07002a8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b213b7b390c7637 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:31:49.046 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.044+0000 7ff0577fe640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-11T23:31:49.046 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.044+0000 7ff0577fe640 1 -- 172.21.15.26:0/287632298 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff070055040 con 0x55cb8d08b680 2024-09-11T23:31:49.046 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.044+0000 7ff0813bc740 1 -- 172.21.15.26:0/287632298 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff06c004c80 con 0x55cb8d08b680 2024-09-11T23:31:49.046 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.044+0000 7ff0577fe640 1 -- 172.21.15.26:0/287632298 <== mon.2 v2:172.21.15.76:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8542+0+0 (secure 0 0 0) 0x7ff070003b80 con 0x55cb8d08b680 2024-09-11T23:31:49.046 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.044+0000 7ff0577fe640 1 -- 172.21.15.26:0/287632298 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7ff070003d80 con 0x55cb8d08b680 2024-09-11T23:31:49.046 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.044+0000 7ff0813bc740 10 client.26870 fetch_fsmap learned FSMap version 69 2024-09-11T23:31:49.046 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.044+0000 7ff0813bc740 1 -- 172.21.15.26:0/287632298 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) -- 0x55cb8d10f790 con 0x55cb8d08b680 2024-09-11T23:31:49.046 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.045+0000 7ff0577fe640 1 -- 172.21.15.26:0/287632298 <== mon.2 v2:172.21.15.76:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff070036070 con 0x55cb8d08b680 2024-09-11T23:31:49.047 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.045+0000 7ff0577fe640 1 -- 172.21.15.26:0/287632298 <== mon.2 v2:172.21.15.76:3300/0 5 ==== fsmap(e 69) ==== 8438+0+0 (secure 0 0 0) 0x7ff070042030 con 0x55cb8d08b680 2024-09-11T23:31:49.047 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.045+0000 7ff0813bc740 10 client.26870 fetch_fsmap finished waiting for FSMap version 69 2024-09-11T23:31:49.047 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.045+0000 7ff0813bc740 10 client.26870 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:31:49.047 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.045+0000 7ff0813bc740 20 client.26870 populate_metadata read hostname 'smithi026' 2024-09-11T23:31:49.047 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.045+0000 7ff0813bc740 1 --2- 172.21.15.26:0/287632298 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55cb8d10fe00 0x55cb8d112220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:49.047 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.045+0000 7ff0813bc740 4 client.26870 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T23:31:49.047 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.045+0000 7ff0813bc740 1 -- 172.21.15.26:0/287632298 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7ff06c004c80 con 0x55cb8d10fe00 2024-09-11T23:31:49.047 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.045+0000 7ff0777fe640 1 --2- 172.21.15.26:0/287632298 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55cb8d10fe00 0x55cb8d112220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:49.047 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.046+0000 7ff0777fe640 1 --2- 172.21.15.26:0/287632298 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55cb8d10fe00 0x55cb8d112220 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df1b92eb1a2a6feb server_cookie=dc604888f8f68524 in_seq=0 out_seq=0 2024-09-11T23:31:49.048 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.046+0000 7ff0577fe640 10 client.26870 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:31:49.132 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.133+0000 7ff0577fe640 1 -- 172.21.15.26:0/287632298 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7ff06c004c80 con 0x55cb8d10fe00 2024-09-11T23:31:49.133 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.133+0000 7ff0577fe640 10 client.26870 handle_command_reply: tid=0 2024-09-11T23:31:49.133 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.133+0000 7ff0813bc740 1 client.26870 shutdown 2024-09-11T23:31:49.136 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.136+0000 7ff0813bc740 1 -- 172.21.15.26:0/287632298 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cb8d08b680 msgr2=0x55cb8d0aa610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:31:49.136 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.136+0000 7ff0813bc740 1 --2- 172.21.15.26:0/287632298 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cb8d08b680 0x55cb8d0aa610 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ff070005c20 tx=0x7ff07002a8e0 comp rx=0 tx=0).stop 2024-09-11T23:31:49.136 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.136+0000 7ff0813bc740 1 -- 172.21.15.26:0/287632298 shutdown_connections 2024-09-11T23:31:49.136 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.136+0000 7ff0813bc740 1 --2- 172.21.15.26:0/287632298 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55cb8d10fe00 0x55cb8d112220 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:49.136 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.136+0000 7ff0577fe640 0 client.26870 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:31:49.136 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.136+0000 7ff0813bc740 1 --2- 172.21.15.26:0/287632298 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cb8d08fbc0 0x55cb8d0919e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:49.137 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.136+0000 7ff0813bc740 1 --2- 172.21.15.26:0/287632298 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cb8d08c020 0x55cb8d0aab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:49.137 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.136+0000 7ff0813bc740 1 --2- 172.21.15.26:0/287632298 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cb8d08b680 0x55cb8d0aa610 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:49.137 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.136+0000 7ff0813bc740 1 -- 172.21.15.26:0/287632298 >> 172.21.15.26:0/287632298 conn(0x55cb8d07db10 msgr2=0x55cb8d091050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:31:49.137 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.137+0000 7ff0813bc740 1 -- 172.21.15.26:0/287632298 shutdown_connections 2024-09-11T23:31:49.137 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.137+0000 7ff0813bc740 1 -- 172.21.15.26:0/287632298 wait complete. 2024-09-11T23:31:49.137 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.137+0000 7ff0813bc740 20 client.26870 trim_cache size 0 max 16384 2024-09-11T23:31:49.138 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-09-11T23:31:49.138 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "message": "", 2024-09-11T23:31:49.138 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-09-11T23:31:49.138 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-09-11T23:31:49.139 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.138+0000 7ff07e623640 1 -- 172.21.15.26:0/3792688058 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7ff0600a2da0 msgr2=0x7ff0600a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:31:49.140 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.138+0000 7ff07e623640 1 --2- 172.21.15.26:0/3792688058 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7ff0600a2da0 0x7ff0600a5260 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ff068098190 tx=0x7ff068009700 comp rx=0 tx=0).stop 2024-09-11T23:31:49.140 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.138+0000 7ff07e623640 1 -- 172.21.15.26:0/3792688058 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff0781009d0 msgr2=0x7ff078198320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:31:49.140 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.138+0000 7ff07e623640 1 --2- 172.21.15.26:0/3792688058 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff0781009d0 0x7ff078198320 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7ff06800e9f0 tx=0x7ff06800eeb0 comp rx=0 tx=0).stop 2024-09-11T23:31:49.140 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.139+0000 7ff07e623640 1 -- 172.21.15.26:0/3792688058 shutdown_connections 2024-09-11T23:31:49.140 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.139+0000 7ff07e623640 1 --2- 172.21.15.26:0/3792688058 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7ff0600a2da0 0x7ff0600a5260 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:49.140 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.140+0000 7ff07e623640 1 --2- 172.21.15.26:0/3792688058 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff0781069e0 0x7ff07819cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:49.140 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.140+0000 7ff07e623640 1 --2- 172.21.15.26:0/3792688058 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff078101370 0x7ff078198860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:49.140 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.140+0000 7ff07e623640 1 --2- 172.21.15.26:0/3792688058 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff0781009d0 0x7ff078198320 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:49.140 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.140+0000 7ff07e623640 1 -- 172.21.15.26:0/3792688058 >> 172.21.15.26:0/3792688058 conn(0x7ff0780fc6b0 msgr2=0x7ff0780fcde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:31:49.140 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.140+0000 7ff07e623640 1 -- 172.21.15.26:0/3792688058 shutdown_connections 2024-09-11T23:31:49.140 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:31:49.140+0000 7ff07e623640 1 -- 172.21.15.26:0/3792688058 wait complete. 2024-09-11T23:31:50.238 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:31:49 smithi136 ceph-mon[35121]: pgmap v420: 97 pgs: 97 active+clean; 2.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 719 KiB/s rd, 743 KiB/s wr, 84 op/s 2024-09-11T23:31:50.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:31:49 smithi026 ceph-mon[31172]: pgmap v420: 97 pgs: 97 active+clean; 2.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 719 KiB/s rd, 743 KiB/s wr, 84 op/s 2024-09-11T23:31:50.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:31:49 smithi076 ceph-mon[35296]: pgmap v420: 97 pgs: 97 active+clean; 2.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 719 KiB/s rd, 743 KiB/s wr, 84 op/s 2024-09-11T23:31:52.238 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:31:51 smithi136 ceph-mon[35121]: pgmap v421: 97 pgs: 97 active+clean; 2.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.0 MiB/s wr, 234 op/s 2024-09-11T23:31:52.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:31:51 smithi026 ceph-mon[31172]: pgmap v421: 97 pgs: 97 active+clean; 2.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.0 MiB/s wr, 234 op/s 2024-09-11T23:31:52.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:31:51 smithi076 ceph-mon[35296]: pgmap v421: 97 pgs: 97 active+clean; 2.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.0 MiB/s wr, 234 op/s 2024-09-11T23:31:54.238 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:31:53 smithi136 ceph-mon[35121]: pgmap v422: 97 pgs: 97 active+clean; 2.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.2 MiB/s wr, 247 op/s 2024-09-11T23:31:54.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:31:53 smithi026 ceph-mon[31172]: pgmap v422: 97 pgs: 97 active+clean; 2.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.2 MiB/s wr, 247 op/s 2024-09-11T23:31:54.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:31:53 smithi076 ceph-mon[35296]: pgmap v422: 97 pgs: 97 active+clean; 2.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.2 MiB/s wr, 247 op/s 2024-09-11T23:31:56.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:31:55 smithi026 ceph-mon[31172]: pgmap v423: 97 pgs: 97 active+clean; 2.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.6 MiB/s wr, 346 op/s 2024-09-11T23:31:56.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:31:55 smithi076 ceph-mon[35296]: pgmap v423: 97 pgs: 97 active+clean; 2.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.6 MiB/s wr, 346 op/s 2024-09-11T23:31:56.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:31:55 smithi136 ceph-mon[35121]: pgmap v423: 97 pgs: 97 active+clean; 2.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.6 MiB/s wr, 346 op/s 2024-09-11T23:31:58.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:31:57 smithi026 ceph-mon[31172]: pgmap v424: 97 pgs: 97 active+clean; 2.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.1 MiB/s wr, 378 op/s 2024-09-11T23:31:58.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:31:57 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:31:58.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:31:57 smithi076 ceph-mon[35296]: pgmap v424: 97 pgs: 97 active+clean; 2.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.1 MiB/s wr, 378 op/s 2024-09-11T23:31:58.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:31:57 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:31:58.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:31:57 smithi136 ceph-mon[35121]: pgmap v424: 97 pgs: 97 active+clean; 2.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.1 MiB/s wr, 378 op/s 2024-09-11T23:31:58.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:31:57 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:31:59.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:31:58 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:31:59.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:31:58 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:31:59.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:31:58 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:31:59.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:31:58 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:31:59.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:31:58 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:31:59.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:31:58 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:31:59.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:31:58 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:31:59.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:31:58 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:31:59.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:31:58 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:32:00.253 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:31:59 smithi026 ceph-mon[31172]: pgmap v425: 97 pgs: 97 active+clean; 2.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.4 MiB/s wr, 327 op/s 2024-09-11T23:32:00.443 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:31:59 smithi076 ceph-mon[35296]: pgmap v425: 97 pgs: 97 active+clean; 2.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.4 MiB/s wr, 327 op/s 2024-09-11T23:32:00.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:31:59 smithi136 ceph-mon[35121]: pgmap v425: 97 pgs: 97 active+clean; 2.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.4 MiB/s wr, 327 op/s 2024-09-11T23:32:02.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:32:01 smithi026 ceph-mon[31172]: pgmap v426: 97 pgs: 97 active+clean; 2.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.5 MiB/s wr, 359 op/s 2024-09-11T23:32:02.263 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.260+0000 7f010fbaf640 1 -- 172.21.15.26:0/2084763721 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0108107b80 msgr2=0x7f0108107f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:32:02.263 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.260+0000 7f010fbaf640 1 --2- 172.21.15.26:0/2084763721 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0108107b80 0x7f0108107f60 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f0104009930 tx=0x7f010402a420 comp rx=0 tx=0).stop 2024-09-11T23:32:02.263 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.260+0000 7f010fbaf640 1 -- 172.21.15.26:0/2084763721 shutdown_connections 2024-09-11T23:32:02.263 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.260+0000 7f010fbaf640 1 --2- 172.21.15.26:0/2084763721 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0108101ae0 0x7f0108101f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:02.263 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.260+0000 7f010fbaf640 1 --2- 172.21.15.26:0/2084763721 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0108102480 0x7f010810e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:02.264 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.260+0000 7f010fbaf640 1 --2- 172.21.15.26:0/2084763721 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0108107b80 0x7f0108107f60 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:02.264 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.260+0000 7f010fbaf640 1 -- 172.21.15.26:0/2084763721 >> 172.21.15.26:0/2084763721 conn(0x7f01080fd7e0 msgr2=0x7f01080ffc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:32:02.264 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.261+0000 7f010fbaf640 1 -- 172.21.15.26:0/2084763721 shutdown_connections 2024-09-11T23:32:02.264 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.261+0000 7f010fbaf640 1 -- 172.21.15.26:0/2084763721 wait complete. 2024-09-11T23:32:02.264 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.261+0000 7f010fbaf640 1 Processor -- start 2024-09-11T23:32:02.264 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.261+0000 7f010fbaf640 1 -- start start 2024-09-11T23:32:02.264 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.261+0000 7f010fbaf640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0108101ae0 0x7f0108198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:02.264 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.261+0000 7f010fbaf640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0108102480 0x7f0108198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:02.264 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.261+0000 7f010fbaf640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0108107b80 0x7f010819cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:02.264 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.261+0000 7f010fbaf640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f0108111f30 con 0x7f0108102480 2024-09-11T23:32:02.265 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.261+0000 7f010fbaf640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f0108111db0 con 0x7f0108107b80 2024-09-11T23:32:02.265 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.261+0000 7f010fbaf640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f01081120b0 con 0x7f0108101ae0 2024-09-11T23:32:02.265 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.261+0000 7f010d14a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0108102480 0x7f0108198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:02.265 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.262+0000 7f010d94b640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0108101ae0 0x7f0108198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:02.265 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.262+0000 7f010d14a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0108102480 0x7f0108198880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51652/0 (socket says 172.21.15.26:51652) 2024-09-11T23:32:02.265 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.262+0000 7f010d14a640 1 -- 172.21.15.26:0/3951166749 learned_addr learned my addr 172.21.15.26:0/3951166749 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:32:02.265 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.262+0000 7f010e14c640 1 --2- 172.21.15.26:0/3951166749 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0108107b80 0x7f010819cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:02.265 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.262+0000 7f010d14a640 1 -- 172.21.15.26:0/3951166749 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0108101ae0 msgr2=0x7f0108198340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:32:02.265 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.262+0000 7f010d14a640 1 --2- 172.21.15.26:0/3951166749 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0108101ae0 0x7f0108198340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:02.265 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.262+0000 7f010d14a640 1 -- 172.21.15.26:0/3951166749 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0108107b80 msgr2=0x7f010819cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:32:02.265 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.262+0000 7f010d14a640 1 --2- 172.21.15.26:0/3951166749 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0108107b80 0x7f010819cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:02.266 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.262+0000 7f010d14a640 1 -- 172.21.15.26:0/3951166749 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f010819d330 con 0x7f0108102480 2024-09-11T23:32:02.266 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.262+0000 7f010e14c640 1 --2- 172.21.15.26:0/3951166749 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0108107b80 0x7f010819cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:32:02.266 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.262+0000 7f010d14a640 1 --2- 172.21.15.26:0/3951166749 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0108102480 0x7f0108198880 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f00f800e9f0 tx=0x7f00f800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:32:02.266 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.262+0000 7f00feffd640 1 -- 172.21.15.26:0/3951166749 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00f800cd80 con 0x7f0108102480 2024-09-11T23:32:02.266 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.262+0000 7f010fbaf640 1 -- 172.21.15.26:0/3951166749 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f010819d620 con 0x7f0108102480 2024-09-11T23:32:02.266 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.262+0000 7f00feffd640 1 -- 172.21.15.26:0/3951166749 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00f8010910 con 0x7f0108102480 2024-09-11T23:32:02.266 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.262+0000 7f00feffd640 1 -- 172.21.15.26:0/3951166749 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00f80188f0 con 0x7f0108102480 2024-09-11T23:32:02.266 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.262+0000 7f010fbaf640 1 -- 172.21.15.26:0/3951166749 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01081a4eb0 con 0x7f0108102480 2024-09-11T23:32:02.274 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.264+0000 7f00feffd640 1 -- 172.21.15.26:0/3951166749 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f00f8010ac0 con 0x7f0108102480 2024-09-11T23:32:02.274 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.265+0000 7f0112a24740 1 Processor -- start 2024-09-11T23:32:02.274 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.265+0000 7f0112a24740 1 -- start start 2024-09-11T23:32:02.275 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.265+0000 7f0112a24740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5601dbd829f0 0x5601dbd82dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:02.275 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.265+0000 7f0112a24740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5601dbd83310 0x5601dbd91b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:02.275 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.265+0000 7f0112a24740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5601dbd920c0 0x5601dbd944b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:02.275 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.265+0000 7f0112a24740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5601dbce72f0 con 0x5601dbd829f0 2024-09-11T23:32:02.275 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.265+0000 7f0112a24740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5601dbc8f270 con 0x5601dbd83310 2024-09-11T23:32:02.275 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.266+0000 7f0112a24740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5601dbce32b0 con 0x5601dbd920c0 2024-09-11T23:32:02.275 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.266+0000 7f00feffd640 1 --2- 172.21.15.26:0/3951166749 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f00f40a2da0 0x7f00f40a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:02.275 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.266+0000 7f010d14a640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5601dbd920c0 0x5601dbd944b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:02.275 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.266+0000 7f010d94b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5601dbd83310 0x5601dbd91b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:02.275 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.266+0000 7f010e14c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5601dbd829f0 0x5601dbd82dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:02.275 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.266+0000 7f010d14a640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5601dbd920c0 0x5601dbd944b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:40886/0 (socket says 172.21.15.26:40886) 2024-09-11T23:32:02.276 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.266+0000 7f010d94b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5601dbd83310 0x5601dbd91b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:34008/0 (socket says 172.21.15.26:34008) 2024-09-11T23:32:02.276 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.266+0000 7f010d14a640 1 -- 172.21.15.26:0/3035015632 learned_addr learned my addr 172.21.15.26:0/3035015632 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:32:02.276 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.266+0000 7f010e14c640 1 --2- 172.21.15.26:0/3951166749 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f00f40a2da0 0x7f00f40a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:02.276 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.266+0000 7f00feffd640 1 -- 172.21.15.26:0/3951166749 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8542+0+0 (secure 0 0 0) 0x7f00f80cf870 con 0x7f0108102480 2024-09-11T23:32:02.276 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.266+0000 7f010d94b640 1 -- 172.21.15.26:0/3035015632 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5601dbd920c0 msgr2=0x5601dbd944b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:32:02.276 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.266+0000 7f010d94b640 1 --2- 172.21.15.26:0/3035015632 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5601dbd920c0 0x5601dbd944b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:02.276 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.266+0000 7f010d94b640 1 -- 172.21.15.26:0/3035015632 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5601dbd829f0 msgr2=0x5601dbd82dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:32:02.276 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.266+0000 7f010d94b640 1 --2- 172.21.15.26:0/3035015632 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5601dbd829f0 0x5601dbd82dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:02.276 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.266+0000 7f010d94b640 1 -- 172.21.15.26:0/3035015632 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5601dbc8e340 con 0x5601dbd83310 2024-09-11T23:32:02.276 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.266+0000 7f010e14c640 1 --2- 172.21.15.26:0/3951166749 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f00f40a2da0 0x7f00f40a5260 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f01000099e0 tx=0x7f0100009500 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:32:02.276 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.266+0000 7f010d94b640 1 --2- 172.21.15.26:0/3035015632 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5601dbd83310 0x5601dbd91b80 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f0104005940 tx=0x7f0104005bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ee05e23550788a41 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:32:02.276 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.266+0000 7f00f3fff640 1 -- 172.21.15.26:0/3035015632 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0104040070 con 0x5601dbd83310 2024-09-11T23:32:02.276 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.267+0000 7f00f3fff640 1 -- 172.21.15.26:0/3035015632 <== mon.2 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0104031870 con 0x5601dbd83310 2024-09-11T23:32:02.277 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.267+0000 7f00f3fff640 1 -- 172.21.15.26:0/3035015632 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0104035040 con 0x5601dbd83310 2024-09-11T23:32:02.277 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.267+0000 7f0112a24740 1 -- 172.21.15.26:0/3035015632 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5601dbd83310 msgr2=0x5601dbd91b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:32:02.277 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.267+0000 7f0112a24740 1 --2- 172.21.15.26:0/3035015632 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5601dbd83310 0x5601dbd91b80 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f0104005940 tx=0x7f0104005bd0 comp rx=0 tx=0).stop 2024-09-11T23:32:02.277 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.267+0000 7f0112a24740 1 -- 172.21.15.26:0/3035015632 shutdown_connections 2024-09-11T23:32:02.277 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.267+0000 7f0112a24740 1 --2- 172.21.15.26:0/3035015632 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5601dbd83310 0x5601dbd91b80 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:02.277 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.267+0000 7f0112a24740 1 --2- 172.21.15.26:0/3035015632 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5601dbd920c0 0x5601dbd944b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:02.277 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.267+0000 7f0112a24740 1 --2- 172.21.15.26:0/3035015632 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5601dbd829f0 0x5601dbd82dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:02.277 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.267+0000 7f0112a24740 1 -- 172.21.15.26:0/3035015632 >> 172.21.15.26:0/3035015632 conn(0x5601dbd7f060 msgr2=0x5601dbd7f440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:32:02.277 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.267+0000 7f0112a24740 1 -- 172.21.15.26:0/3035015632 shutdown_connections 2024-09-11T23:32:02.277 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.267+0000 7f0112a24740 1 -- 172.21.15.26:0/3035015632 wait complete. 2024-09-11T23:32:02.277 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.268+0000 7f0112a24740 1 -- start start 2024-09-11T23:32:02.277 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.268+0000 7f0112a24740 1 Processor -- start 2024-09-11T23:32:02.277 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.268+0000 7f0112a24740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5601dbd829f0 0x5601dbd8efe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:02.277 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.268+0000 7f0112a24740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5601dbd83310 0x5601dbd8f520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:02.278 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.268+0000 7f0112a24740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5601dbd920c0 0x5601dbd8fa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:02.278 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.268+0000 7f0112a24740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5601dbd94cc0 con 0x5601dbd83310 2024-09-11T23:32:02.278 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.268+0000 7f0112a24740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5601dbc70290 con 0x5601dbd829f0 2024-09-11T23:32:02.278 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.268+0000 7f0112a24740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5601dbd94e40 con 0x5601dbd920c0 2024-09-11T23:32:02.278 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.268+0000 7f0112a24740 10 client.0 _ll_register_callbacks cb 0x5601dbca1140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:32:02.278 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.268+0000 7f010d14a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5601dbd83310 0x5601dbd8f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:02.278 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.268+0000 7f010d94b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5601dbd829f0 0x5601dbd8efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:02.278 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.268+0000 7f010e14c640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5601dbd920c0 0x5601dbd8fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:02.278 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.268+0000 7f010d14a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5601dbd83310 0x5601dbd8f520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51666/0 (socket says 172.21.15.26:51666) 2024-09-11T23:32:02.278 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.268+0000 7f010d94b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5601dbd829f0 0x5601dbd8efe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:34016/0 (socket says 172.21.15.26:34016) 2024-09-11T23:32:02.278 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.268+0000 7f010d14a640 1 -- 172.21.15.26:0/467091516 learned_addr learned my addr 172.21.15.26:0/467091516 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:32:02.278 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.269+0000 7f010e14c640 1 -- 172.21.15.26:0/467091516 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5601dbd829f0 msgr2=0x5601dbd8efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:32:02.279 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.269+0000 7f010e14c640 1 --2- 172.21.15.26:0/467091516 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5601dbd829f0 0x5601dbd8efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:02.279 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.269+0000 7f010e14c640 1 -- 172.21.15.26:0/467091516 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5601dbd83310 msgr2=0x5601dbd8f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:32:02.279 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.269+0000 7f010e14c640 1 --2- 172.21.15.26:0/467091516 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5601dbd83310 0x5601dbd8f520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:02.279 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.269+0000 7f010e14c640 1 -- 172.21.15.26:0/467091516 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5601db9fa210 con 0x5601dbd920c0 2024-09-11T23:32:02.279 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.269+0000 7f010d14a640 1 --2- 172.21.15.26:0/467091516 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5601dbd83310 0x5601dbd8f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:32:02.279 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.269+0000 7f010e14c640 1 --2- 172.21.15.26:0/467091516 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5601dbd920c0 0x5601dbd8fa60 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f0100005c40 tx=0x7f0100004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=88425f65da02db26 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:32:02.279 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.269+0000 7f00f17fa640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-09-11T23:32:02.279 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.269+0000 7f00f17fa640 1 -- 172.21.15.26:0/467091516 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f010001fe60 con 0x5601dbd920c0 2024-09-11T23:32:02.279 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.269+0000 7f0112a24740 1 -- 172.21.15.26:0/467091516 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0104031870 con 0x5601dbd920c0 2024-09-11T23:32:02.279 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.269+0000 7f00f17fa640 1 -- 172.21.15.26:0/467091516 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8542+0+0 (secure 0 0 0) 0x7f010000fd80 con 0x5601dbd920c0 2024-09-11T23:32:02.279 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.269+0000 7f00f17fa640 1 -- 172.21.15.26:0/467091516 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7f0100004b20 con 0x5601dbd920c0 2024-09-11T23:32:02.279 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.269+0000 7f0112a24740 10 client.26863 fetch_fsmap learned FSMap version 69 2024-09-11T23:32:02.279 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.269+0000 7f0112a24740 1 -- 172.21.15.26:0/467091516 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) -- 0x5601dbac6620 con 0x5601dbd920c0 2024-09-11T23:32:02.280 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.269+0000 7f00f17fa640 1 -- 172.21.15.26:0/467091516 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0104031870 con 0x5601dbd920c0 2024-09-11T23:32:02.280 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.270+0000 7f00f17fa640 1 -- 172.21.15.26:0/467091516 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 69) ==== 8438+0+0 (secure 0 0 0) 0x7f010001d070 con 0x5601dbd920c0 2024-09-11T23:32:02.280 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.270+0000 7f0112a24740 10 client.26863 fetch_fsmap finished waiting for FSMap version 69 2024-09-11T23:32:02.280 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.270+0000 7f0112a24740 10 client.26863 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:32:02.280 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.270+0000 7f0112a24740 20 client.26863 populate_metadata read hostname 'smithi026' 2024-09-11T23:32:02.280 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.270+0000 7f0112a24740 1 --2- 172.21.15.26:0/467091516 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5601dbda2190 0x5601dbda45e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:02.280 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.270+0000 7f0112a24740 4 client.26863 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:32:02.280 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.270+0000 7f0112a24740 1 -- 172.21.15.26:0/467091516 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5601dbcf4a20 con 0x5601dbda2190 2024-09-11T23:32:02.280 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.270+0000 7f010d94b640 1 --2- 172.21.15.26:0/467091516 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5601dbda2190 0x5601dbda45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:02.280 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.270+0000 7f00f17fa640 10 client.26863 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:32:02.280 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.270+0000 7f010d94b640 1 --2- 172.21.15.26:0/467091516 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5601dbda2190 0x5601dbda45e0 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2cbf04b0ae9e5db7 server_cookie=c2635a0207ca6276 in_seq=0 out_seq=0 2024-09-11T23:32:02.280 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.271+0000 7f00f17fa640 1 -- 172.21.15.26:0/467091516 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5601dbcf4a20 con 0x5601dbda2190 2024-09-11T23:32:02.281 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.271+0000 7f00f17fa640 10 client.26863 handle_command_reply: tid=0 2024-09-11T23:32:02.281 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.271+0000 7f0112a24740 1 client.26863 shutdown 2024-09-11T23:32:02.281 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.271+0000 7f0112a24740 1 -- 172.21.15.26:0/467091516 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5601dbd920c0 msgr2=0x5601dbd8fa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:32:02.281 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.271+0000 7f0112a24740 1 --2- 172.21.15.26:0/467091516 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5601dbd920c0 0x5601dbd8fa60 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f0100005c40 tx=0x7f0100004a40 comp rx=0 tx=0).stop 2024-09-11T23:32:02.281 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.271+0000 7f0112a24740 1 -- 172.21.15.26:0/467091516 shutdown_connections 2024-09-11T23:32:02.281 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.271+0000 7f0112a24740 1 --2- 172.21.15.26:0/467091516 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5601dbda2190 0x5601dbda45e0 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:02.281 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.271+0000 7f0112a24740 1 --2- 172.21.15.26:0/467091516 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5601dbd920c0 0x5601dbd8fa60 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:02.281 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.271+0000 7f0112a24740 1 --2- 172.21.15.26:0/467091516 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5601dbd83310 0x5601dbd8f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:02.281 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.271+0000 7f0112a24740 1 --2- 172.21.15.26:0/467091516 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5601dbd829f0 0x5601dbd8efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:02.281 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.271+0000 7f00f17fa640 0 client.26863 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:32:02.281 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.271+0000 7f0112a24740 1 -- 172.21.15.26:0/467091516 >> 172.21.15.26:0/467091516 conn(0x5601dbd7f060 msgr2=0x5601dbd84dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:32:02.281 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.271+0000 7f0112a24740 1 -- 172.21.15.26:0/467091516 shutdown_connections 2024-09-11T23:32:02.281 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.271+0000 7f0112a24740 1 -- 172.21.15.26:0/467091516 wait complete. 2024-09-11T23:32:02.281 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.272+0000 7f0112a24740 20 client.26863 trim_cache size 0 max 16384 2024-09-11T23:32:02.290 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.286+0000 7f0112a24740 1 Processor -- start 2024-09-11T23:32:02.290 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.286+0000 7f0112a24740 1 -- start start 2024-09-11T23:32:02.290 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.286+0000 7f0112a24740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5601dbd829f0 0x5601dbda9ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:02.290 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.286+0000 7f0112a24740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5601dbd83310 0x5601dbdaa430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:02.291 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.286+0000 7f0112a24740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5601dbd920c0 0x5601dbd84530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:02.291 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.286+0000 7f0112a24740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f01081ab5f0 con 0x5601dbd83310 2024-09-11T23:32:02.291 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.286+0000 7f0112a24740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f01081ad400 con 0x5601dbd920c0 2024-09-11T23:32:02.291 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.286+0000 7f0112a24740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f01081abd50 con 0x5601dbd829f0 2024-09-11T23:32:02.291 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.286+0000 7f010d14a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5601dbd83310 0x5601dbdaa430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:02.291 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.286+0000 7f010d14a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5601dbd83310 0x5601dbdaa430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51674/0 (socket says 172.21.15.26:51674) 2024-09-11T23:32:02.291 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.286+0000 7f010d14a640 1 -- 172.21.15.26:0/3476512599 learned_addr learned my addr 172.21.15.26:0/3476512599 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:32:02.291 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.286+0000 7f010d94b640 1 --2- 172.21.15.26:0/3476512599 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5601dbd829f0 0x5601dbda9ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:02.291 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.286+0000 7f010e14c640 1 --2- 172.21.15.26:0/3476512599 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5601dbd920c0 0x5601dbd84530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:02.291 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.287+0000 7f010d94b640 1 -- 172.21.15.26:0/3476512599 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5601dbd920c0 msgr2=0x5601dbd84530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:32:02.291 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.287+0000 7f010d94b640 1 --2- 172.21.15.26:0/3476512599 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5601dbd920c0 0x5601dbd84530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:02.291 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.287+0000 7f010d94b640 1 -- 172.21.15.26:0/3476512599 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5601dbd83310 msgr2=0x5601dbdaa430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:32:02.291 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.287+0000 7f010d94b640 1 --2- 172.21.15.26:0/3476512599 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5601dbd83310 0x5601dbdaa430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:02.291 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.287+0000 7f010d94b640 1 -- 172.21.15.26:0/3476512599 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5601dbd83140 con 0x5601dbd829f0 2024-09-11T23:32:02.292 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.287+0000 7f010d14a640 1 --2- 172.21.15.26:0/3476512599 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5601dbd83310 0x5601dbdaa430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:32:02.292 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.287+0000 7f010d94b640 1 --2- 172.21.15.26:0/3476512599 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5601dbd829f0 0x5601dbda9ef0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f010402a940 tx=0x7f01040058d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ef30001b33f1a049 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:32:02.292 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.287+0000 7f00f37fe640 1 -- 172.21.15.26:0/3476512599 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0104040040 con 0x5601dbd829f0 2024-09-11T23:32:02.292 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.287+0000 7f00f37fe640 1 -- 172.21.15.26:0/3476512599 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0104005500 con 0x5601dbd829f0 2024-09-11T23:32:02.292 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.287+0000 7f00f37fe640 1 -- 172.21.15.26:0/3476512599 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0104035040 con 0x5601dbd829f0 2024-09-11T23:32:02.292 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.288+0000 7f0112a24740 1 -- 172.21.15.26:0/3476512599 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5601dbd829f0 msgr2=0x5601dbda9ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:32:02.292 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.288+0000 7f0112a24740 1 --2- 172.21.15.26:0/3476512599 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5601dbd829f0 0x5601dbda9ef0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f010402a940 tx=0x7f01040058d0 comp rx=0 tx=0).stop 2024-09-11T23:32:02.292 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.288+0000 7f0112a24740 1 -- 172.21.15.26:0/3476512599 shutdown_connections 2024-09-11T23:32:02.292 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.288+0000 7f0112a24740 1 --2- 172.21.15.26:0/3476512599 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5601dbd920c0 0x5601dbd84530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:02.292 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.288+0000 7f0112a24740 1 --2- 172.21.15.26:0/3476512599 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5601dbd83310 0x5601dbdaa430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:02.292 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.288+0000 7f0112a24740 1 --2- 172.21.15.26:0/3476512599 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5601dbd829f0 0x5601dbda9ef0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:02.292 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.288+0000 7f0112a24740 1 -- 172.21.15.26:0/3476512599 >> 172.21.15.26:0/3476512599 conn(0x5601dbd7f060 msgr2=0x5601dbd8e0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:32:02.292 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.288+0000 7f0112a24740 1 -- 172.21.15.26:0/3476512599 shutdown_connections 2024-09-11T23:32:02.293 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.288+0000 7f0112a24740 1 -- 172.21.15.26:0/3476512599 wait complete. 2024-09-11T23:32:02.293 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.288+0000 7f0112a24740 1 -- start start 2024-09-11T23:32:02.293 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.288+0000 7f0112a24740 1 Processor -- start 2024-09-11T23:32:02.293 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.288+0000 7f0112a24740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5601dbd829f0 0x5601dbd9a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:02.293 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.288+0000 7f0112a24740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5601dbd83310 0x5601dbd9a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:02.293 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.288+0000 7f0112a24740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5601dbd920c0 0x5601dbd8b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:02.293 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.288+0000 7f0112a24740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f01081acc70 con 0x5601dbd829f0 2024-09-11T23:32:02.293 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.288+0000 7f0112a24740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f01081ac4e0 con 0x5601dbd920c0 2024-09-11T23:32:02.293 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.288+0000 7f0112a24740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f01081a95f0 con 0x5601dbd83310 2024-09-11T23:32:02.293 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.289+0000 7f0112a24740 10 client.0 _ll_register_callbacks cb 0x7f0100011bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:32:02.293 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.289+0000 7f010d94b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5601dbd829f0 0x5601dbd9a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:02.293 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.289+0000 7f010d14a640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5601dbd83310 0x5601dbd9a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:02.293 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.289+0000 7f010d94b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5601dbd829f0 0x5601dbd9a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51690/0 (socket says 172.21.15.26:51690) 2024-09-11T23:32:02.293 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.289+0000 7f010d94b640 1 -- 172.21.15.26:0/939835001 learned_addr learned my addr 172.21.15.26:0/939835001 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:32:02.294 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.289+0000 7f010e14c640 1 --2- 172.21.15.26:0/939835001 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5601dbd920c0 0x5601dbd8b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:02.294 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.289+0000 7f010d94b640 1 -- 172.21.15.26:0/939835001 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5601dbd83310 msgr2=0x5601dbd9a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:32:02.294 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.289+0000 7f010d94b640 1 --2- 172.21.15.26:0/939835001 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5601dbd83310 0x5601dbd9a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:02.294 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.289+0000 7f010d94b640 1 -- 172.21.15.26:0/939835001 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5601dbd920c0 msgr2=0x5601dbd8b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:32:02.294 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.289+0000 7f010d94b640 1 --2- 172.21.15.26:0/939835001 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5601dbd920c0 0x5601dbd8b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:02.294 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.289+0000 7f010d94b640 1 -- 172.21.15.26:0/939835001 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5601dbd8be90 con 0x5601dbd829f0 2024-09-11T23:32:02.294 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.289+0000 7f010d94b640 1 --2- 172.21.15.26:0/939835001 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5601dbd829f0 0x5601dbd9a2c0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f0104009310 tx=0x7f010402aae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0be1578fbf23486 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:32:02.294 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.289+0000 7f00f17fa640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:32:02.294 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.289+0000 7f00f17fa640 1 -- 172.21.15.26:0/939835001 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0104035070 con 0x5601dbd829f0 2024-09-11T23:32:02.294 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.289+0000 7f0112a24740 1 -- 172.21.15.26:0/939835001 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0104005500 con 0x5601dbd829f0 2024-09-11T23:32:02.294 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.289+0000 7f00f17fa640 1 -- 172.21.15.26:0/939835001 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8542+0+0 (secure 0 0 0) 0x7f0104004030 con 0x5601dbd829f0 2024-09-11T23:32:02.295 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.289+0000 7f00f17fa640 1 -- 172.21.15.26:0/939835001 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7f0104067050 con 0x5601dbd829f0 2024-09-11T23:32:02.295 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.289+0000 7f0112a24740 10 client.17361 fetch_fsmap learned FSMap version 69 2024-09-11T23:32:02.295 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.289+0000 7f0112a24740 1 -- 172.21.15.26:0/939835001 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) -- 0x5601dbe0c870 con 0x5601dbd829f0 2024-09-11T23:32:02.295 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.290+0000 7f00f17fa640 1 -- 172.21.15.26:0/939835001 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0104005500 con 0x5601dbd829f0 2024-09-11T23:32:02.295 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.290+0000 7f00f17fa640 1 -- 172.21.15.26:0/939835001 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 69) ==== 8438+0+0 (secure 0 0 0) 0x7f01040035f0 con 0x5601dbd829f0 2024-09-11T23:32:02.295 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.290+0000 7f0112a24740 10 client.17361 fetch_fsmap finished waiting for FSMap version 69 2024-09-11T23:32:02.295 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.290+0000 7f0112a24740 10 client.17361 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:32:02.295 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.290+0000 7f0112a24740 20 client.17361 populate_metadata read hostname 'smithi026' 2024-09-11T23:32:02.295 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.290+0000 7f0112a24740 1 --2- 172.21.15.26:0/939835001 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5601dbe0cbf0 0x5601dbe0efe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:02.295 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.290+0000 7f0112a24740 4 client.17361 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T23:32:02.296 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.290+0000 7f0112a24740 1 -- 172.21.15.26:0/939835001 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5601dbceed70 con 0x5601dbe0cbf0 2024-09-11T23:32:02.296 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.290+0000 7f010d14a640 1 --2- 172.21.15.26:0/939835001 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5601dbe0cbf0 0x5601dbe0efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:02.296 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.290+0000 7f00f17fa640 10 client.17361 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:32:02.296 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:02.290+0000 7f010d14a640 1 --2- 172.21.15.26:0/939835001 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5601dbe0cbf0 0x5601dbe0efe0 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0827523a7d75f server_cookie=549677fc72e641da in_seq=0 out_seq=0 2024-09-11T23:32:02.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:32:02 smithi076 ceph-mon[35296]: pgmap v426: 97 pgs: 97 active+clean; 2.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.5 MiB/s wr, 359 op/s 2024-09-11T23:32:02.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:32:02 smithi136 ceph-mon[35121]: pgmap v426: 97 pgs: 97 active+clean; 2.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.5 MiB/s wr, 359 op/s 2024-09-11T23:32:03.302 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:03.301+0000 7f00f17fa640 1 -- 172.21.15.26:0/939835001 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5601dbceed70 con 0x5601dbe0cbf0 2024-09-11T23:32:03.302 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:03.301+0000 7f00f17fa640 10 client.17361 handle_command_reply: tid=0 2024-09-11T23:32:03.302 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:03.301+0000 7f0112a24740 1 client.17361 shutdown 2024-09-11T23:32:03.302 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:03.301+0000 7f0112a24740 1 -- 172.21.15.26:0/939835001 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5601dbd829f0 msgr2=0x5601dbd9a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:32:03.302 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:03.301+0000 7f0112a24740 1 --2- 172.21.15.26:0/939835001 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5601dbd829f0 0x5601dbd9a2c0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f0104009310 tx=0x7f010402aae0 comp rx=0 tx=0).stop 2024-09-11T23:32:03.302 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:03.301+0000 7f0112a24740 1 -- 172.21.15.26:0/939835001 shutdown_connections 2024-09-11T23:32:03.302 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:03.301+0000 7f0112a24740 1 --2- 172.21.15.26:0/939835001 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5601dbd920c0 0x5601dbd8b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:03.302 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:03.301+0000 7f0112a24740 1 --2- 172.21.15.26:0/939835001 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5601dbe0cbf0 0x5601dbe0efe0 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:03.302 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:03.301+0000 7f0112a24740 1 --2- 172.21.15.26:0/939835001 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5601dbd83310 0x5601dbd9a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:03.302 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:03.301+0000 7f00f17fa640 0 client.17361 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:32:03.303 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:03.301+0000 7f0112a24740 1 --2- 172.21.15.26:0/939835001 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5601dbd829f0 0x5601dbd9a2c0 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:03.303 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:03.301+0000 7f0112a24740 1 -- 172.21.15.26:0/939835001 >> 172.21.15.26:0/939835001 conn(0x5601dbd7f060 msgr2=0x5601dbd84530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:32:03.303 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:03.301+0000 7f0112a24740 1 -- 172.21.15.26:0/939835001 shutdown_connections 2024-09-11T23:32:03.303 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:03.301+0000 7f0112a24740 1 -- 172.21.15.26:0/939835001 wait complete. 2024-09-11T23:32:03.303 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:03.301+0000 7f0112a24740 20 client.17361 trim_cache size 0 max 16384 2024-09-11T23:32:03.303 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-09-11T23:32:03.303 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "message": "", 2024-09-11T23:32:03.303 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-09-11T23:32:03.303 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-09-11T23:32:03.305 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:03.302+0000 7f010fbaf640 1 -- 172.21.15.26:0/3951166749 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f00f40a2da0 msgr2=0x7f00f40a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:32:03.305 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:03.302+0000 7f010fbaf640 1 --2- 172.21.15.26:0/3951166749 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f00f40a2da0 0x7f00f40a5260 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f01000099e0 tx=0x7f0100009500 comp rx=0 tx=0).stop 2024-09-11T23:32:03.305 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:03.302+0000 7f010fbaf640 1 -- 172.21.15.26:0/3951166749 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0108102480 msgr2=0x7f0108198880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:32:03.305 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:03.302+0000 7f010fbaf640 1 --2- 172.21.15.26:0/3951166749 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0108102480 0x7f0108198880 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f00f800e9f0 tx=0x7f00f800eeb0 comp rx=0 tx=0).stop 2024-09-11T23:32:03.305 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:03.302+0000 7f010fbaf640 1 -- 172.21.15.26:0/3951166749 shutdown_connections 2024-09-11T23:32:03.305 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:03.302+0000 7f010fbaf640 1 --2- 172.21.15.26:0/3951166749 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f00f40a2da0 0x7f00f40a5260 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:03.305 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:03.302+0000 7f010fbaf640 1 --2- 172.21.15.26:0/3951166749 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0108107b80 0x7f010819cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:03.305 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:03.302+0000 7f010fbaf640 1 --2- 172.21.15.26:0/3951166749 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0108102480 0x7f0108198880 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:03.305 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:03.302+0000 7f010fbaf640 1 --2- 172.21.15.26:0/3951166749 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0108101ae0 0x7f0108198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:03.306 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:03.302+0000 7f010fbaf640 1 -- 172.21.15.26:0/3951166749 >> 172.21.15.26:0/3951166749 conn(0x7f01080fd7e0 msgr2=0x7f01080fee60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:32:03.306 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:03.302+0000 7f010fbaf640 1 -- 172.21.15.26:0/3951166749 shutdown_connections 2024-09-11T23:32:03.306 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:03.302+0000 7f010fbaf640 1 -- 172.21.15.26:0/3951166749 wait complete. 2024-09-11T23:32:04.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:32:04 smithi076 ceph-mon[35296]: pgmap v427: 97 pgs: 97 active+clean; 2.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 214 op/s 2024-09-11T23:32:04.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:32:04 smithi136 ceph-mon[35121]: pgmap v427: 97 pgs: 97 active+clean; 2.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 214 op/s 2024-09-11T23:32:04.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:32:04 smithi026 ceph-mon[31172]: pgmap v427: 97 pgs: 97 active+clean; 2.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 214 op/s 2024-09-11T23:32:05.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:32:05 smithi076 ceph-mon[35296]: osdmap e101: 12 total, 12 up, 12 in 2024-09-11T23:32:05.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:32:05 smithi136 ceph-mon[35121]: osdmap e101: 12 total, 12 up, 12 in 2024-09-11T23:32:05.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:32:05 smithi026 ceph-mon[31172]: osdmap e101: 12 total, 12 up, 12 in 2024-09-11T23:32:06.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:32:06 smithi076 ceph-mon[35296]: pgmap v429: 97 pgs: 97 active+clean; 2.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.4 MiB/s wr, 217 op/s 2024-09-11T23:32:06.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:32:06 smithi136 ceph-mon[35121]: pgmap v429: 97 pgs: 97 active+clean; 2.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.4 MiB/s wr, 217 op/s 2024-09-11T23:32:06.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:32:06 smithi026 ceph-mon[31172]: pgmap v429: 97 pgs: 97 active+clean; 2.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.4 MiB/s wr, 217 op/s 2024-09-11T23:32:08.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:32:08 smithi076 ceph-mon[35296]: pgmap v430: 97 pgs: 19 active+clean+snaptrim_wait, 10 active+clean+snaptrim, 68 active+clean; 2.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 240 op/s 2024-09-11T23:32:08.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:32:08 smithi136 ceph-mon[35121]: pgmap v430: 97 pgs: 19 active+clean+snaptrim_wait, 10 active+clean+snaptrim, 68 active+clean; 2.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 240 op/s 2024-09-11T23:32:08.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:32:08 smithi026 ceph-mon[31172]: pgmap v430: 97 pgs: 19 active+clean+snaptrim_wait, 10 active+clean+snaptrim, 68 active+clean; 2.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 240 op/s 2024-09-11T23:32:10.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:32:10 smithi076 ceph-mon[35296]: pgmap v431: 97 pgs: 19 active+clean+snaptrim_wait, 10 active+clean+snaptrim, 68 active+clean; 2.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 240 op/s 2024-09-11T23:32:10.302 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:32:10 smithi026 ceph-mon[31172]: pgmap v431: 97 pgs: 19 active+clean+snaptrim_wait, 10 active+clean+snaptrim, 68 active+clean; 2.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 240 op/s 2024-09-11T23:32:10.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:32:10 smithi136 ceph-mon[35121]: pgmap v431: 97 pgs: 19 active+clean+snaptrim_wait, 10 active+clean+snaptrim, 68 active+clean; 2.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 240 op/s 2024-09-11T23:32:12.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:32:12 smithi076 ceph-mon[35296]: pgmap v432: 97 pgs: 41 active+clean+snaptrim_wait, 23 active+clean+snaptrim, 33 active+clean; 2.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.7 MiB/s wr, 306 op/s 2024-09-11T23:32:12.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:32:12 smithi136 ceph-mon[35121]: pgmap v432: 97 pgs: 41 active+clean+snaptrim_wait, 23 active+clean+snaptrim, 33 active+clean; 2.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.7 MiB/s wr, 306 op/s 2024-09-11T23:32:12.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:32:12 smithi026 ceph-mon[31172]: pgmap v432: 97 pgs: 41 active+clean+snaptrim_wait, 23 active+clean+snaptrim, 33 active+clean; 2.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.7 MiB/s wr, 306 op/s 2024-09-11T23:32:14.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:32:14 smithi076 ceph-mon[35296]: pgmap v433: 97 pgs: 41 active+clean+snaptrim_wait, 23 active+clean+snaptrim, 33 active+clean; 2.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.7 MiB/s wr, 309 op/s 2024-09-11T23:32:14.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:32:14 smithi136 ceph-mon[35121]: pgmap v433: 97 pgs: 41 active+clean+snaptrim_wait, 23 active+clean+snaptrim, 33 active+clean; 2.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.7 MiB/s wr, 309 op/s 2024-09-11T23:32:14.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:32:14 smithi026 ceph-mon[31172]: pgmap v433: 97 pgs: 41 active+clean+snaptrim_wait, 23 active+clean+snaptrim, 33 active+clean; 2.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.7 MiB/s wr, 309 op/s 2024-09-11T23:32:16.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:32:16 smithi076 ceph-mon[35296]: pgmap v434: 97 pgs: 39 active+clean+snaptrim_wait, 21 active+clean+snaptrim, 37 active+clean; 2.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.8 MiB/s wr, 265 op/s 2024-09-11T23:32:16.435 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.435+0000 7f5444453640 1 -- 172.21.15.26:0/2452967828 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f543c1069c0 msgr2=0x7f543c106da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:32:16.435 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.435+0000 7f5444453640 1 --2- 172.21.15.26:0/2452967828 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f543c1069c0 0x7f543c106da0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f5438009930 tx=0x7f543802a420 comp rx=0 tx=0).stop 2024-09-11T23:32:16.435 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.435+0000 7f5444453640 1 -- 172.21.15.26:0/2452967828 shutdown_connections 2024-09-11T23:32:16.435 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.435+0000 7f5444453640 1 --2- 172.21.15.26:0/2452967828 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f543c1009b0 0x7f543c100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:16.436 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.435+0000 7f5444453640 1 --2- 172.21.15.26:0/2452967828 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f543c101350 0x7f543c10d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:16.436 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.435+0000 7f5444453640 1 --2- 172.21.15.26:0/2452967828 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f543c1069c0 0x7f543c106da0 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:16.436 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.436+0000 7f5444453640 1 -- 172.21.15.26:0/2452967828 >> 172.21.15.26:0/2452967828 conn(0x7f543c077990 msgr2=0x7f543c0fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:32:16.436 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.436+0000 7f5444453640 1 -- 172.21.15.26:0/2452967828 shutdown_connections 2024-09-11T23:32:16.436 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.436+0000 7f5444453640 1 -- 172.21.15.26:0/2452967828 wait complete. 2024-09-11T23:32:16.438 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.436+0000 7f5444453640 1 Processor -- start 2024-09-11T23:32:16.438 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.436+0000 7f5444453640 1 -- start start 2024-09-11T23:32:16.438 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.437+0000 7f5444453640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f543c1009b0 0x7f543c198170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:16.438 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.437+0000 7f5444453640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f543c101350 0x7f543c1986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:16.438 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.437+0000 7f5444453640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f543c1069c0 0x7f543c19ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:16.438 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.437+0000 7f54421ef640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f543c1009b0 0x7f543c198170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:16.439 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.437+0000 7f54421ef640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f543c1009b0 0x7f543c198170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60630/0 (socket says 172.21.15.26:60630) 2024-09-11T23:32:16.439 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.437+0000 7f54419ee640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f543c101350 0x7f543c1986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:16.439 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.437+0000 7f54419ee640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f543c101350 0x7f543c1986b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:45276/0 (socket says 172.21.15.26:45276) 2024-09-11T23:32:16.439 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.438+0000 7f54421ef640 1 -- 172.21.15.26:0/3647140861 learned_addr learned my addr 172.21.15.26:0/3647140861 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:32:16.439 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.438+0000 7f5444453640 1 -- 172.21.15.26:0/3647140861 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f543c110e50 con 0x7f543c1009b0 2024-09-11T23:32:16.439 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.438+0000 7f54429f0640 1 --2- 172.21.15.26:0/3647140861 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f543c1069c0 0x7f543c19ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:16.439 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.438+0000 7f5444453640 1 -- 172.21.15.26:0/3647140861 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f543c110cd0 con 0x7f543c1069c0 2024-09-11T23:32:16.439 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.438+0000 7f5444453640 1 -- 172.21.15.26:0/3647140861 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f543c110fd0 con 0x7f543c101350 2024-09-11T23:32:16.439 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.438+0000 7f54421ef640 1 -- 172.21.15.26:0/3647140861 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f543c101350 msgr2=0x7f543c1986b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:32:16.439 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.438+0000 7f54421ef640 1 --2- 172.21.15.26:0/3647140861 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f543c101350 0x7f543c1986b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:16.440 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.438+0000 7f54421ef640 1 -- 172.21.15.26:0/3647140861 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f543c1069c0 msgr2=0x7f543c19ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:32:16.440 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.438+0000 7f54421ef640 1 --2- 172.21.15.26:0/3647140861 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f543c1069c0 0x7f543c19ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:16.440 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.438+0000 7f54421ef640 1 -- 172.21.15.26:0/3647140861 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f543c19d160 con 0x7f543c1009b0 2024-09-11T23:32:16.440 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.439+0000 7f54421ef640 1 --2- 172.21.15.26:0/3647140861 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f543c1009b0 0x7f543c198170 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f5438030940 tx=0x7f5438030970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:32:16.441 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.439+0000 7f54337fe640 1 -- 172.21.15.26:0/3647140861 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f543803f030 con 0x7f543c1009b0 2024-09-11T23:32:16.441 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.439+0000 7f54337fe640 1 -- 172.21.15.26:0/3647140861 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5438030d20 con 0x7f543c1009b0 2024-09-11T23:32:16.441 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.439+0000 7f5444453640 1 -- 172.21.15.26:0/3647140861 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f543c19d3f0 con 0x7f543c1009b0 2024-09-11T23:32:16.441 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.440+0000 7f54337fe640 1 -- 172.21.15.26:0/3647140861 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5438035070 con 0x7f543c1009b0 2024-09-11T23:32:16.441 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.441+0000 7f5444453640 1 -- 172.21.15.26:0/3647140861 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f543c1a4ce0 con 0x7f543c1009b0 2024-09-11T23:32:16.442 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.443+0000 7f54337fe640 1 -- 172.21.15.26:0/3647140861 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f5438042030 con 0x7f543c1009b0 2024-09-11T23:32:16.445 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.445+0000 7f54471e9740 1 Processor -- start 2024-09-11T23:32:16.445 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.445+0000 7f54337fe640 1 --2- 172.21.15.26:0/3647140861 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f54280a2e70 0x7f54280a5330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:16.445 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.446+0000 7f54471e9740 1 -- start start 2024-09-11T23:32:16.445 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.446+0000 7f54337fe640 1 -- 172.21.15.26:0/3647140861 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8586+0+0 (secure 0 0 0) 0x7f54380ef920 con 0x7f543c1009b0 2024-09-11T23:32:16.446 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.446+0000 7f54429f0640 1 --2- 172.21.15.26:0/3647140861 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f54280a2e70 0x7f54280a5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:16.447 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.446+0000 7f54471e9740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b8f7ff6af0 0x55b8f7ff6ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:16.447 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.446+0000 7f54471e9740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b8f7ff7410 0x55b8f8001770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:16.447 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.446+0000 7f54471e9740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b8f8001cb0 0x55b8f80040a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:16.447 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.446+0000 7f54471e9740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55b8f7f5b2f0 con 0x55b8f7ff7410 2024-09-11T23:32:16.447 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.446+0000 7f54471e9740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55b8f7f03270 con 0x55b8f8001cb0 2024-09-11T23:32:16.448 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.446+0000 7f54471e9740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55b8f7f572b0 con 0x55b8f7ff6af0 2024-09-11T23:32:16.448 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.446+0000 7f54419ee640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b8f7ff7410 0x55b8f8001770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:16.448 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.446+0000 7f54421ef640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b8f7ff6af0 0x55b8f7ff6ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:16.448 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.447+0000 7f54421ef640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b8f7ff6af0 0x55b8f7ff6ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:45292/0 (socket says 172.21.15.26:45292) 2024-09-11T23:32:16.448 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.447+0000 7f54421ef640 1 -- 172.21.15.26:0/2961887849 learned_addr learned my addr 172.21.15.26:0/2961887849 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:32:16.448 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.447+0000 7f54429f0640 1 --2- 172.21.15.26:0/2961887849 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b8f8001cb0 0x55b8f80040a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:16.448 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.447+0000 7f54429f0640 1 --2- 172.21.15.26:0/3647140861 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f54280a2e70 0x7f54280a5330 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f5434005fd0 tx=0x7f5434009600 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:32:16.448 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.447+0000 7f54419ee640 1 -- 172.21.15.26:0/2961887849 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b8f7ff6af0 msgr2=0x55b8f7ff6ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:32:16.448 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.447+0000 7f54419ee640 1 --2- 172.21.15.26:0/2961887849 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b8f7ff6af0 0x55b8f7ff6ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:16.448 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.447+0000 7f54419ee640 1 -- 172.21.15.26:0/2961887849 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b8f8001cb0 msgr2=0x55b8f80040a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:32:16.448 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.447+0000 7f54419ee640 1 --2- 172.21.15.26:0/2961887849 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b8f8001cb0 0x55b8f80040a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:16.448 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.447+0000 7f54419ee640 1 -- 172.21.15.26:0/2961887849 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b8f7f02340 con 0x55b8f7ff7410 2024-09-11T23:32:16.448 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.447+0000 7f54421ef640 1 --2- 172.21.15.26:0/2961887849 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b8f7ff6af0 0x55b8f7ff6ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:32:16.449 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.447+0000 7f54419ee640 1 --2- 172.21.15.26:0/2961887849 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b8f7ff7410 0x55b8f8001770 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f542c00bbc0 tx=0x7f542c0107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e6c8bfdf7042457 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:32:16.449 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.447+0000 7f5430bfe640 1 -- 172.21.15.26:0/2961887849 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f542c004030 con 0x55b8f7ff7410 2024-09-11T23:32:16.449 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.448+0000 7f5430bfe640 1 -- 172.21.15.26:0/2961887849 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f542c0041d0 con 0x55b8f7ff7410 2024-09-11T23:32:16.449 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.448+0000 7f5430bfe640 1 -- 172.21.15.26:0/2961887849 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f542c0116a0 con 0x55b8f7ff7410 2024-09-11T23:32:16.452 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.448+0000 7f54471e9740 1 -- 172.21.15.26:0/2961887849 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b8f7ff7410 msgr2=0x55b8f8001770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:32:16.452 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.448+0000 7f54471e9740 1 --2- 172.21.15.26:0/2961887849 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b8f7ff7410 0x55b8f8001770 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f542c00bbc0 tx=0x7f542c0107a0 comp rx=0 tx=0).stop 2024-09-11T23:32:16.452 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.449+0000 7f54471e9740 1 -- 172.21.15.26:0/2961887849 shutdown_connections 2024-09-11T23:32:16.452 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.449+0000 7f54471e9740 1 --2- 172.21.15.26:0/2961887849 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b8f8001cb0 0x55b8f80040a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:16.453 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.449+0000 7f54471e9740 1 --2- 172.21.15.26:0/2961887849 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b8f7ff7410 0x55b8f8001770 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:16.453 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.449+0000 7f54471e9740 1 --2- 172.21.15.26:0/2961887849 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b8f7ff6af0 0x55b8f7ff6ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:16.453 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.449+0000 7f54471e9740 1 -- 172.21.15.26:0/2961887849 >> 172.21.15.26:0/2961887849 conn(0x55b8f7ff3060 msgr2=0x55b8f7ff3440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:32:16.453 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.449+0000 7f54471e9740 1 -- 172.21.15.26:0/2961887849 shutdown_connections 2024-09-11T23:32:16.453 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.449+0000 7f54471e9740 1 -- 172.21.15.26:0/2961887849 wait complete. 2024-09-11T23:32:16.453 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.449+0000 7f54471e9740 1 -- start start 2024-09-11T23:32:16.453 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.450+0000 7f54471e9740 1 Processor -- start 2024-09-11T23:32:16.453 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.450+0000 7f54471e9740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b8f7ff6af0 0x55b8f7ffebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:16.453 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.450+0000 7f54471e9740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b8f7ff7410 0x55b8f7fff100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:16.453 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.450+0000 7f54471e9740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b8f8001cb0 0x55b8f7fff640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:16.453 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.450+0000 7f54471e9740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55b8f8008c90 con 0x55b8f8001cb0 2024-09-11T23:32:16.453 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.450+0000 7f54471e9740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55b8f7ee4290 con 0x55b8f7ff6af0 2024-09-11T23:32:16.454 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.450+0000 7f54471e9740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55b8f8008e10 con 0x55b8f7ff7410 2024-09-11T23:32:16.454 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.450+0000 7f54471e9740 10 client.0 _ll_register_callbacks cb 0x55b8f7f15140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:32:16.454 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.450+0000 7f54421ef640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b8f7ff6af0 0x55b8f7ffebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:16.454 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.450+0000 7f54421ef640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b8f7ff6af0 0x55b8f7ffebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:58420/0 (socket says 172.21.15.26:58420) 2024-09-11T23:32:16.454 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.450+0000 7f54421ef640 1 -- 172.21.15.26:0/247216367 learned_addr learned my addr 172.21.15.26:0/247216367 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:32:16.454 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.450+0000 7f54429f0640 1 --2- 172.21.15.26:0/247216367 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b8f8001cb0 0x55b8f7fff640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:16.454 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.450+0000 7f54419ee640 1 --2- 172.21.15.26:0/247216367 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b8f7ff7410 0x55b8f7fff100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:16.454 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.450+0000 7f54421ef640 1 -- 172.21.15.26:0/247216367 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b8f7ff7410 msgr2=0x55b8f7fff100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:32:16.454 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.450+0000 7f54421ef640 1 --2- 172.21.15.26:0/247216367 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b8f7ff7410 0x55b8f7fff100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:16.454 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.450+0000 7f54421ef640 1 -- 172.21.15.26:0/247216367 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b8f8001cb0 msgr2=0x55b8f7fff640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:32:16.454 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.450+0000 7f54421ef640 1 --2- 172.21.15.26:0/247216367 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b8f8001cb0 0x55b8f7fff640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:16.454 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.450+0000 7f54421ef640 1 -- 172.21.15.26:0/247216367 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b8f7c6e210 con 0x55b8f7ff6af0 2024-09-11T23:32:16.454 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.450+0000 7f54429f0640 1 --2- 172.21.15.26:0/247216367 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b8f8001cb0 0x55b8f7fff640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:32:16.454 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.450+0000 7f54419ee640 1 --2- 172.21.15.26:0/247216367 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b8f7ff7410 0x55b8f7fff100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:32:16.455 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.451+0000 7f54421ef640 1 --2- 172.21.15.26:0/247216367 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b8f7ff6af0 0x55b8f7ffebc0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f54380ed630 tx=0x7f54380ed560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2074d832ac68fab0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:32:16.455 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.451+0000 7f5425ffb640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-11T23:32:16.455 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.451+0000 7f5425ffb640 1 -- 172.21.15.26:0/247216367 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5438049040 con 0x55b8f7ff6af0 2024-09-11T23:32:16.455 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.451+0000 7f54471e9740 1 -- 172.21.15.26:0/247216367 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f542c0041d0 con 0x55b8f7ff6af0 2024-09-11T23:32:16.455 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.451+0000 7f5425ffb640 1 -- 172.21.15.26:0/247216367 <== mon.2 v2:172.21.15.76:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8586+0+0 (secure 0 0 0) 0x7f54380c47e0 con 0x55b8f7ff6af0 2024-09-11T23:32:16.455 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.451+0000 7f5425ffb640 1 -- 172.21.15.26:0/247216367 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7f54380bb810 con 0x55b8f7ff6af0 2024-09-11T23:32:16.455 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.451+0000 7f54471e9740 10 client.26900 fetch_fsmap learned FSMap version 69 2024-09-11T23:32:16.455 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.451+0000 7f54471e9740 1 -- 172.21.15.26:0/247216367 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) -- 0x55b8f7d3a620 con 0x55b8f7ff6af0 2024-09-11T23:32:16.455 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.451+0000 7f5425ffb640 1 -- 172.21.15.26:0/247216367 <== mon.2 v2:172.21.15.76:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f542c0041d0 con 0x55b8f7ff6af0 2024-09-11T23:32:16.455 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.452+0000 7f5425ffb640 1 -- 172.21.15.26:0/247216367 <== mon.2 v2:172.21.15.76:3300/0 5 ==== fsmap(e 69) ==== 8438+0+0 (secure 0 0 0) 0x7f54380da020 con 0x55b8f7ff6af0 2024-09-11T23:32:16.455 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.452+0000 7f54471e9740 10 client.26900 fetch_fsmap finished waiting for FSMap version 69 2024-09-11T23:32:16.455 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.452+0000 7f54471e9740 10 client.26900 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:32:16.455 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.452+0000 7f54471e9740 20 client.26900 populate_metadata read hostname 'smithi026' 2024-09-11T23:32:16.455 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.452+0000 7f54471e9740 1 --2- 172.21.15.26:0/247216367 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55b8f8016080 0x55b8f8018500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:16.456 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.452+0000 7f54471e9740 4 client.26900 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:32:16.456 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.452+0000 7f54471e9740 1 -- 172.21.15.26:0/247216367 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b8f7f68a20 con 0x55b8f8016080 2024-09-11T23:32:16.456 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.452+0000 7f54419ee640 1 --2- 172.21.15.26:0/247216367 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55b8f8016080 0x55b8f8018500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:16.456 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.453+0000 7f5425ffb640 10 client.26900 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:32:16.456 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.453+0000 7f54419ee640 1 --2- 172.21.15.26:0/247216367 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55b8f8016080 0x55b8f8018500 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e6f06061c2d7240 server_cookie=42f7275954953603 in_seq=0 out_seq=0 2024-09-11T23:32:16.457 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.454+0000 7f5425ffb640 1 -- 172.21.15.26:0/247216367 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b8f7f68a20 con 0x55b8f8016080 2024-09-11T23:32:16.457 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.454+0000 7f5425ffb640 10 client.26900 handle_command_reply: tid=0 2024-09-11T23:32:16.457 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.454+0000 7f54471e9740 1 client.26900 shutdown 2024-09-11T23:32:16.457 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.454+0000 7f54471e9740 1 -- 172.21.15.26:0/247216367 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b8f7ff6af0 msgr2=0x55b8f7ffebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:32:16.458 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.454+0000 7f54471e9740 1 --2- 172.21.15.26:0/247216367 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b8f7ff6af0 0x55b8f7ffebc0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f54380ed630 tx=0x7f54380ed560 comp rx=0 tx=0).stop 2024-09-11T23:32:16.458 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.454+0000 7f54471e9740 1 -- 172.21.15.26:0/247216367 shutdown_connections 2024-09-11T23:32:16.458 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.454+0000 7f54471e9740 1 --2- 172.21.15.26:0/247216367 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55b8f8016080 0x55b8f8018500 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:16.458 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.454+0000 7f5425ffb640 0 client.26900 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:32:16.458 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.454+0000 7f54471e9740 1 --2- 172.21.15.26:0/247216367 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b8f8001cb0 0x55b8f7fff640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:16.458 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.454+0000 7f54471e9740 1 --2- 172.21.15.26:0/247216367 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b8f7ff7410 0x55b8f7fff100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:16.458 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.454+0000 7f54471e9740 1 --2- 172.21.15.26:0/247216367 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b8f7ff6af0 0x55b8f7ffebc0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:16.458 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.454+0000 7f54471e9740 1 -- 172.21.15.26:0/247216367 >> 172.21.15.26:0/247216367 conn(0x55b8f7ff3060 msgr2=0x55b8f7ff8d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:32:16.458 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.454+0000 7f54471e9740 1 -- 172.21.15.26:0/247216367 shutdown_connections 2024-09-11T23:32:16.458 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.454+0000 7f54471e9740 1 -- 172.21.15.26:0/247216367 wait complete. 2024-09-11T23:32:16.458 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.455+0000 7f54471e9740 20 client.26900 trim_cache size 0 max 16384 2024-09-11T23:32:16.480 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.480+0000 7f54471e9740 1 Processor -- start 2024-09-11T23:32:16.480 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.480+0000 7f54471e9740 1 -- start start 2024-09-11T23:32:16.480 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.480+0000 7f54471e9740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b8f7ff6af0 0x55b8f7ff20b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:16.480 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.480+0000 7f54471e9740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b8f7ff7410 0x55b8f801de10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:16.480 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.480+0000 7f54471e9740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b8f8001cb0 0x55b8f8008400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:16.480 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.480+0000 7f54471e9740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f543c1ab6d0 con 0x55b8f7ff6af0 2024-09-11T23:32:16.480 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.480+0000 7f54471e9740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f543c1ad4e0 con 0x55b8f7ff7410 2024-09-11T23:32:16.480 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.480+0000 7f54471e9740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f543c1abe30 con 0x55b8f8001cb0 2024-09-11T23:32:16.480 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.480+0000 7f54429f0640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b8f8001cb0 0x55b8f8008400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:16.481 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.480+0000 7f54419ee640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b8f7ff7410 0x55b8f801de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:16.481 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.481+0000 7f54429f0640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b8f8001cb0 0x55b8f8008400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:45310/0 (socket says 172.21.15.26:45310) 2024-09-11T23:32:16.481 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.481+0000 7f54429f0640 1 -- 172.21.15.26:0/2421738659 learned_addr learned my addr 172.21.15.26:0/2421738659 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:32:16.481 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.481+0000 7f54419ee640 1 -- 172.21.15.26:0/2421738659 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b8f8001cb0 msgr2=0x55b8f8008400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:32:16.482 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.481+0000 7f54419ee640 1 --2- 172.21.15.26:0/2421738659 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b8f8001cb0 0x55b8f8008400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:16.482 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.481+0000 7f54419ee640 1 -- 172.21.15.26:0/2421738659 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b8f7ff6af0 msgr2=0x55b8f7ff20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:32:16.482 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.481+0000 7f54421ef640 1 --2- 172.21.15.26:0/2421738659 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b8f7ff6af0 0x55b8f7ff20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:16.482 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.481+0000 7f54419ee640 1 --2- 172.21.15.26:0/2421738659 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b8f7ff6af0 0x55b8f7ff20b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:16.482 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.481+0000 7f54419ee640 1 -- 172.21.15.26:0/2421738659 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b8f7ff41f0 con 0x55b8f7ff7410 2024-09-11T23:32:16.482 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.481+0000 7f54421ef640 1 --2- 172.21.15.26:0/2421738659 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b8f7ff6af0 0x55b8f7ff20b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:32:16.482 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.481+0000 7f54419ee640 1 --2- 172.21.15.26:0/2421738659 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b8f7ff7410 0x55b8f801de10 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f542c005890 tx=0x7f542c00b390 comp rx=0 tx=0).ready entity=mon.2 client_cookie=80b295fe1662a4fa server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:32:16.482 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.481+0000 7f5427fff640 1 -- 172.21.15.26:0/2421738659 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f542c00bd70 con 0x55b8f7ff7410 2024-09-11T23:32:16.482 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.481+0000 7f5427fff640 1 -- 172.21.15.26:0/2421738659 <== mon.2 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f542c005520 con 0x55b8f7ff7410 2024-09-11T23:32:16.482 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.482+0000 7f5427fff640 1 -- 172.21.15.26:0/2421738659 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f542c011440 con 0x55b8f7ff7410 2024-09-11T23:32:16.485 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.482+0000 7f54471e9740 1 -- 172.21.15.26:0/2421738659 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b8f7ff7410 msgr2=0x55b8f801de10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:32:16.485 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.482+0000 7f54471e9740 1 --2- 172.21.15.26:0/2421738659 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b8f7ff7410 0x55b8f801de10 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f542c005890 tx=0x7f542c00b390 comp rx=0 tx=0).stop 2024-09-11T23:32:16.485 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.482+0000 7f54471e9740 1 -- 172.21.15.26:0/2421738659 shutdown_connections 2024-09-11T23:32:16.485 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.482+0000 7f54471e9740 1 --2- 172.21.15.26:0/2421738659 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b8f7ff7410 0x55b8f801de10 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:16.485 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.482+0000 7f54471e9740 1 --2- 172.21.15.26:0/2421738659 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b8f8001cb0 0x55b8f8008400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:16.485 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.482+0000 7f54471e9740 1 --2- 172.21.15.26:0/2421738659 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b8f7ff6af0 0x55b8f7ff20b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:16.485 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.482+0000 7f54471e9740 1 -- 172.21.15.26:0/2421738659 >> 172.21.15.26:0/2421738659 conn(0x55b8f7ff3060 msgr2=0x55b8f7ff19a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:32:16.486 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.482+0000 7f54471e9740 1 -- 172.21.15.26:0/2421738659 shutdown_connections 2024-09-11T23:32:16.486 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.483+0000 7f54471e9740 1 -- 172.21.15.26:0/2421738659 wait complete. 2024-09-11T23:32:16.486 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.483+0000 7f54471e9740 1 -- start start 2024-09-11T23:32:16.486 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.483+0000 7f54471e9740 1 Processor -- start 2024-09-11T23:32:16.486 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.483+0000 7f54471e9740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b8f7ff6af0 0x55b8f8004bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:16.486 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.483+0000 7f54471e9740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b8f7ff7410 0x55b8f8005130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:16.486 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.483+0000 7f54471e9740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b8f8001cb0 0x55b8f800d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:16.486 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.484+0000 7f54421ef640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b8f7ff6af0 0x55b8f8004bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:16.486 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.484+0000 7f54421ef640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b8f7ff6af0 0x55b8f8004bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:58438/0 (socket says 172.21.15.26:58438) 2024-09-11T23:32:16.486 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.484+0000 7f54421ef640 1 -- 172.21.15.26:0/2421047711 learned_addr learned my addr 172.21.15.26:0/2421047711 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:32:16.486 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.484+0000 7f54471e9740 1 -- 172.21.15.26:0/2421047711 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f543c1acd50 con 0x55b8f7ff7410 2024-09-11T23:32:16.486 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.484+0000 7f54471e9740 1 -- 172.21.15.26:0/2421047711 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f543c1ac5c0 con 0x55b8f7ff6af0 2024-09-11T23:32:16.487 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.484+0000 7f54471e9740 1 -- 172.21.15.26:0/2421047711 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f543c1a96d0 con 0x55b8f8001cb0 2024-09-11T23:32:16.487 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.484+0000 7f54471e9740 10 client.0 _ll_register_callbacks cb 0x7f54380ecfd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:32:16.487 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.484+0000 7f54419ee640 1 --2- 172.21.15.26:0/2421047711 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b8f7ff7410 0x55b8f8005130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:16.487 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.484+0000 7f54429f0640 1 --2- 172.21.15.26:0/2421047711 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b8f8001cb0 0x55b8f800d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:16.487 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.484+0000 7f54421ef640 1 -- 172.21.15.26:0/2421047711 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b8f8001cb0 msgr2=0x55b8f800d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:32:16.487 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.484+0000 7f54421ef640 1 --2- 172.21.15.26:0/2421047711 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b8f8001cb0 0x55b8f800d140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:16.487 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.484+0000 7f54421ef640 1 -- 172.21.15.26:0/2421047711 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b8f7ff7410 msgr2=0x55b8f8005130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:32:16.487 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.484+0000 7f54421ef640 1 --2- 172.21.15.26:0/2421047711 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b8f7ff7410 0x55b8f8005130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:16.487 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.484+0000 7f54421ef640 1 -- 172.21.15.26:0/2421047711 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b8f8005a10 con 0x55b8f7ff6af0 2024-09-11T23:32:16.487 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.484+0000 7f54421ef640 1 --2- 172.21.15.26:0/2421047711 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b8f7ff6af0 0x55b8f8004bf0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f5438005c20 tx=0x7f5438032e90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=98a2676a39dfcc65 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:32:16.487 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.484+0000 7f5425ffb640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-11T23:32:16.488 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.484+0000 7f5425ffb640 1 -- 172.21.15.26:0/2421047711 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5438049040 con 0x55b8f7ff6af0 2024-09-11T23:32:16.488 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.485+0000 7f54471e9740 1 -- 172.21.15.26:0/2421047711 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f542c005520 con 0x55b8f7ff6af0 2024-09-11T23:32:16.488 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.485+0000 7f5425ffb640 1 -- 172.21.15.26:0/2421047711 <== mon.2 v2:172.21.15.76:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8586+0+0 (secure 0 0 0) 0x7f54380b7d80 con 0x55b8f7ff6af0 2024-09-11T23:32:16.488 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.485+0000 7f5425ffb640 1 -- 172.21.15.26:0/2421047711 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7f54380c6e60 con 0x55b8f7ff6af0 2024-09-11T23:32:16.488 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.485+0000 7f54471e9740 10 client.26906 fetch_fsmap learned FSMap version 69 2024-09-11T23:32:16.488 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.485+0000 7f54471e9740 1 -- 172.21.15.26:0/2421047711 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) -- 0x55b8f800e6f0 con 0x55b8f7ff6af0 2024-09-11T23:32:16.488 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.485+0000 7f5425ffb640 1 -- 172.21.15.26:0/2421047711 <== mon.2 v2:172.21.15.76:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f542c005520 con 0x55b8f7ff6af0 2024-09-11T23:32:16.488 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.485+0000 7f5425ffb640 1 -- 172.21.15.26:0/2421047711 <== mon.2 v2:172.21.15.76:3300/0 5 ==== fsmap(e 69) ==== 8438+0+0 (secure 0 0 0) 0x7f543804c060 con 0x55b8f7ff6af0 2024-09-11T23:32:16.488 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.485+0000 7f54471e9740 10 client.26906 fetch_fsmap finished waiting for FSMap version 69 2024-09-11T23:32:16.488 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.485+0000 7f54471e9740 10 client.26906 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:32:16.488 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.485+0000 7f54471e9740 20 client.26906 populate_metadata read hostname 'smithi026' 2024-09-11T23:32:16.488 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.485+0000 7f54471e9740 1 --2- 172.21.15.26:0/2421047711 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55b8f80855d0 0x55b8f80879c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:16.489 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.485+0000 7f54471e9740 4 client.26906 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T23:32:16.489 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.485+0000 7f54471e9740 1 -- 172.21.15.26:0/2421047711 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b8f7f62d70 con 0x55b8f80855d0 2024-09-11T23:32:16.489 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.486+0000 7f54419ee640 1 --2- 172.21.15.26:0/2421047711 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55b8f80855d0 0x55b8f80879c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:16.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:32:16 smithi136 ceph-mon[35121]: pgmap v434: 97 pgs: 39 active+clean+snaptrim_wait, 21 active+clean+snaptrim, 37 active+clean; 2.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.8 MiB/s wr, 265 op/s 2024-09-11T23:32:16.490 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.487+0000 7f5425ffb640 10 client.26906 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:32:16.490 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:16.488+0000 7f54419ee640 1 --2- 172.21.15.26:0/2421047711 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55b8f80855d0 0x55b8f80879c0 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a9796d4fa00c0282 server_cookie=c697f5692a85078c in_seq=0 out_seq=0 2024-09-11T23:32:16.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:32:16 smithi026 ceph-mon[31172]: pgmap v434: 97 pgs: 39 active+clean+snaptrim_wait, 21 active+clean+snaptrim, 37 active+clean; 2.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.8 MiB/s wr, 265 op/s 2024-09-11T23:32:17.419 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:17.419+0000 7f5425ffb640 1 -- 172.21.15.26:0/2421047711 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b8f7f62d70 con 0x55b8f80855d0 2024-09-11T23:32:17.419 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:17.419+0000 7f5425ffb640 10 client.26906 handle_command_reply: tid=0 2024-09-11T23:32:17.419 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:17.419+0000 7f54471e9740 1 client.26906 shutdown 2024-09-11T23:32:17.419 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:17.419+0000 7f54471e9740 1 -- 172.21.15.26:0/2421047711 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b8f7ff6af0 msgr2=0x55b8f8004bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:32:17.419 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:17.419+0000 7f54471e9740 1 --2- 172.21.15.26:0/2421047711 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b8f7ff6af0 0x55b8f8004bf0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f5438005c20 tx=0x7f5438032e90 comp rx=0 tx=0).stop 2024-09-11T23:32:17.420 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:17.419+0000 7f54471e9740 1 -- 172.21.15.26:0/2421047711 shutdown_connections 2024-09-11T23:32:17.420 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:17.419+0000 7f54471e9740 1 --2- 172.21.15.26:0/2421047711 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55b8f80855d0 0x55b8f80879c0 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:17.420 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:17.419+0000 7f54471e9740 1 --2- 172.21.15.26:0/2421047711 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b8f8001cb0 0x55b8f800d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:17.420 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:17.419+0000 7f54471e9740 1 --2- 172.21.15.26:0/2421047711 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b8f7ff7410 0x55b8f8005130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:17.420 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:17.419+0000 7f54471e9740 1 --2- 172.21.15.26:0/2421047711 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b8f7ff6af0 0x55b8f8004bf0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:17.420 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:17.419+0000 7f54471e9740 1 -- 172.21.15.26:0/2421047711 >> 172.21.15.26:0/2421047711 conn(0x55b8f7ff3060 msgr2=0x55b8f7ff19a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:32:17.420 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:17.419+0000 7f5425ffb640 0 client.26906 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:32:17.420 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:17.419+0000 7f54471e9740 1 -- 172.21.15.26:0/2421047711 shutdown_connections 2024-09-11T23:32:17.420 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:17.419+0000 7f54471e9740 1 -- 172.21.15.26:0/2421047711 wait complete. 2024-09-11T23:32:17.420 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:17.419+0000 7f54471e9740 20 client.26906 trim_cache size 0 max 16384 2024-09-11T23:32:17.421 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-09-11T23:32:17.421 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "message": "", 2024-09-11T23:32:17.421 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-09-11T23:32:17.421 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-09-11T23:32:17.421 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:17.420+0000 7f5444453640 1 -- 172.21.15.26:0/3647140861 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f54280a2e70 msgr2=0x7f54280a5330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:32:17.421 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:17.420+0000 7f5444453640 1 --2- 172.21.15.26:0/3647140861 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f54280a2e70 0x7f54280a5330 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f5434005fd0 tx=0x7f5434009600 comp rx=0 tx=0).stop 2024-09-11T23:32:17.422 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:17.420+0000 7f5444453640 1 -- 172.21.15.26:0/3647140861 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f543c1009b0 msgr2=0x7f543c198170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:32:17.422 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:17.420+0000 7f5444453640 1 --2- 172.21.15.26:0/3647140861 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f543c1009b0 0x7f543c198170 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f5438030940 tx=0x7f5438030970 comp rx=0 tx=0).stop 2024-09-11T23:32:17.422 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:17.420+0000 7f5444453640 1 -- 172.21.15.26:0/3647140861 shutdown_connections 2024-09-11T23:32:17.422 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:17.420+0000 7f5444453640 1 --2- 172.21.15.26:0/3647140861 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f54280a2e70 0x7f54280a5330 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:17.422 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:17.420+0000 7f5444453640 1 --2- 172.21.15.26:0/3647140861 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f543c1069c0 0x7f543c19ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:17.422 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:17.420+0000 7f5444453640 1 --2- 172.21.15.26:0/3647140861 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f543c101350 0x7f543c1986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:17.422 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:17.420+0000 7f5444453640 1 --2- 172.21.15.26:0/3647140861 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f543c1009b0 0x7f543c198170 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:17.422 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:17.420+0000 7f5444453640 1 -- 172.21.15.26:0/3647140861 >> 172.21.15.26:0/3647140861 conn(0x7f543c077990 msgr2=0x7f543c0698a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:32:17.422 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:17.420+0000 7f5444453640 1 -- 172.21.15.26:0/3647140861 shutdown_connections 2024-09-11T23:32:17.422 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:17.420+0000 7f5444453640 1 -- 172.21.15.26:0/3647140861 wait complete. 2024-09-11T23:32:18.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:32:18 smithi136 ceph-mon[35121]: pgmap v435: 97 pgs: 35 active+clean+snaptrim_wait, 17 active+clean+snaptrim, 45 active+clean; 2.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.6 MiB/s wr, 256 op/s 2024-09-11T23:32:18.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:32:18 smithi026 ceph-mon[31172]: pgmap v435: 97 pgs: 35 active+clean+snaptrim_wait, 17 active+clean+snaptrim, 45 active+clean; 2.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.6 MiB/s wr, 256 op/s 2024-09-11T23:32:18.552 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:32:18 smithi076 ceph-mon[35296]: pgmap v435: 97 pgs: 35 active+clean+snaptrim_wait, 17 active+clean+snaptrim, 45 active+clean; 2.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.6 MiB/s wr, 256 op/s 2024-09-11T23:32:20.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:32:20 smithi136 ceph-mon[35121]: pgmap v436: 97 pgs: 35 active+clean+snaptrim_wait, 17 active+clean+snaptrim, 45 active+clean; 2.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.6 MiB/s wr, 179 op/s 2024-09-11T23:32:20.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:32:20 smithi026 ceph-mon[31172]: pgmap v436: 97 pgs: 35 active+clean+snaptrim_wait, 17 active+clean+snaptrim, 45 active+clean; 2.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.6 MiB/s wr, 179 op/s 2024-09-11T23:32:20.552 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:32:20 smithi076 ceph-mon[35296]: pgmap v436: 97 pgs: 35 active+clean+snaptrim_wait, 17 active+clean+snaptrim, 45 active+clean; 2.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.6 MiB/s wr, 179 op/s 2024-09-11T23:32:22.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:32:22 smithi136 ceph-mon[35121]: pgmap v437: 97 pgs: 10 active+clean+snaptrim_wait, 5 active+clean+snaptrim, 82 active+clean; 2.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.5 MiB/s wr, 476 op/s 2024-09-11T23:32:22.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:32:22 smithi026 ceph-mon[31172]: pgmap v437: 97 pgs: 10 active+clean+snaptrim_wait, 5 active+clean+snaptrim, 82 active+clean; 2.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.5 MiB/s wr, 476 op/s 2024-09-11T23:32:22.552 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:32:22 smithi076 ceph-mon[35296]: pgmap v437: 97 pgs: 10 active+clean+snaptrim_wait, 5 active+clean+snaptrim, 82 active+clean; 2.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.5 MiB/s wr, 476 op/s 2024-09-11T23:32:24.354 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:32:24 smithi076 ceph-mon[35296]: pgmap v438: 97 pgs: 5 active+clean+snaptrim_wait, 3 active+clean+snaptrim, 89 active+clean; 2.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 422 op/s 2024-09-11T23:32:24.354 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:32:24 smithi076 ceph-mon[35296]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:32:24.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:32:24 smithi136 ceph-mon[35121]: pgmap v438: 97 pgs: 5 active+clean+snaptrim_wait, 3 active+clean+snaptrim, 89 active+clean; 2.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 422 op/s 2024-09-11T23:32:24.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:32:24 smithi136 ceph-mon[35121]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:32:24.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:32:24 smithi026 ceph-mon[31172]: pgmap v438: 97 pgs: 5 active+clean+snaptrim_wait, 3 active+clean+snaptrim, 89 active+clean; 2.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 422 op/s 2024-09-11T23:32:24.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:32:24 smithi026 ceph-mon[31172]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:32:26.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:32:26 smithi136 ceph-mon[35121]: pgmap v439: 97 pgs: 97 active+clean; 2.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.0 MiB/s wr, 483 op/s 2024-09-11T23:32:26.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:32:26 smithi136 ceph-mon[35121]: osdmap e102: 12 total, 12 up, 12 in 2024-09-11T23:32:26.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:32:26 smithi026 ceph-mon[31172]: pgmap v439: 97 pgs: 97 active+clean; 2.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.0 MiB/s wr, 483 op/s 2024-09-11T23:32:26.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:32:26 smithi026 ceph-mon[31172]: osdmap e102: 12 total, 12 up, 12 in 2024-09-11T23:32:26.552 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:32:26 smithi076 ceph-mon[35296]: pgmap v439: 97 pgs: 97 active+clean; 2.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.0 MiB/s wr, 483 op/s 2024-09-11T23:32:26.552 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:32:26 smithi076 ceph-mon[35296]: osdmap e102: 12 total, 12 up, 12 in 2024-09-11T23:32:28.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:32:28 smithi136 ceph-mon[35121]: pgmap v441: 97 pgs: 97 active+clean; 2.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.4 MiB/s wr, 551 op/s 2024-09-11T23:32:28.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:32:28 smithi026 ceph-mon[31172]: pgmap v441: 97 pgs: 97 active+clean; 2.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.4 MiB/s wr, 551 op/s 2024-09-11T23:32:28.552 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:32:28 smithi076 ceph-mon[35296]: pgmap v441: 97 pgs: 97 active+clean; 2.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.4 MiB/s wr, 551 op/s 2024-09-11T23:32:30.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:32:30 smithi136 ceph-mon[35121]: pgmap v442: 97 pgs: 97 active+clean; 2.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.4 MiB/s wr, 551 op/s 2024-09-11T23:32:30.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:32:30 smithi026 ceph-mon[31172]: pgmap v442: 97 pgs: 97 active+clean; 2.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.4 MiB/s wr, 551 op/s 2024-09-11T23:32:30.516 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.515+0000 7f8ba65e4640 1 -- 172.21.15.26:0/140040152 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ba0101ad0 msgr2=0x7f8ba0101f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:32:30.516 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.515+0000 7f8ba65e4640 1 --2- 172.21.15.26:0/140040152 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ba0101ad0 0x7f8ba0101f30 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f8b94009930 tx=0x7f8b9402a420 comp rx=0 tx=0).stop 2024-09-11T23:32:30.517 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.515+0000 7f8ba65e4640 1 -- 172.21.15.26:0/140040152 shutdown_connections 2024-09-11T23:32:30.517 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.515+0000 7f8ba65e4640 1 --2- 172.21.15.26:0/140040152 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8ba0102470 0x7f8ba010e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:30.517 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.515+0000 7f8ba65e4640 1 --2- 172.21.15.26:0/140040152 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ba0101ad0 0x7f8ba0101f30 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:30.517 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.515+0000 7f8ba65e4640 1 --2- 172.21.15.26:0/140040152 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8ba0107b70 0x7f8ba0107f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:30.517 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.515+0000 7f8ba65e4640 1 -- 172.21.15.26:0/140040152 >> 172.21.15.26:0/140040152 conn(0x7f8ba00fd7d0 msgr2=0x7f8ba00ffbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:32:30.517 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.515+0000 7f8ba65e4640 1 -- 172.21.15.26:0/140040152 shutdown_connections 2024-09-11T23:32:30.517 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.515+0000 7f8ba65e4640 1 -- 172.21.15.26:0/140040152 wait complete. 2024-09-11T23:32:30.517 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.516+0000 7f8ba65e4640 1 Processor -- start 2024-09-11T23:32:30.517 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.516+0000 7f8ba65e4640 1 -- start start 2024-09-11T23:32:30.519 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.516+0000 7f8ba65e4640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8ba0101ad0 0x7f8ba0198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:30.519 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.516+0000 7f8ba65e4640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8ba0102470 0x7f8ba01988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:30.519 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.516+0000 7f8ba65e4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ba0107b70 0x7f8ba019cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:30.519 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.516+0000 7f8ba65e4640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f8ba0111f20 con 0x7f8ba0107b70 2024-09-11T23:32:30.519 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.516+0000 7f8ba65e4640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f8ba0111da0 con 0x7f8ba0101ad0 2024-09-11T23:32:30.519 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.516+0000 7f8ba65e4640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f8ba01120a0 con 0x7f8ba0102470 2024-09-11T23:32:30.519 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.517+0000 7f8ba4b81640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ba0107b70 0x7f8ba019cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:30.520 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.517+0000 7f8ba4b81640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ba0107b70 0x7f8ba019cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33550/0 (socket says 172.21.15.26:33550) 2024-09-11T23:32:30.520 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.517+0000 7f8ba4b81640 1 -- 172.21.15.26:0/3566950549 learned_addr learned my addr 172.21.15.26:0/3566950549 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:32:30.520 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.517+0000 7f8b9ffff640 1 --2- 172.21.15.26:0/3566950549 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8ba0101ad0 0x7f8ba0198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:30.520 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.517+0000 7f8b9f7fe640 1 --2- 172.21.15.26:0/3566950549 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8ba0102470 0x7f8ba01988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:30.520 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.517+0000 7f8ba4b81640 1 -- 172.21.15.26:0/3566950549 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8ba0102470 msgr2=0x7f8ba01988b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:32:30.520 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.517+0000 7f8ba4b81640 1 --2- 172.21.15.26:0/3566950549 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8ba0102470 0x7f8ba01988b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:30.520 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.517+0000 7f8ba4b81640 1 -- 172.21.15.26:0/3566950549 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8ba0101ad0 msgr2=0x7f8ba0198370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:32:30.520 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.517+0000 7f8ba4b81640 1 --2- 172.21.15.26:0/3566950549 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8ba0101ad0 0x7f8ba0198370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:30.520 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.517+0000 7f8ba4b81640 1 -- 172.21.15.26:0/3566950549 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ba019d360 con 0x7f8ba0107b70 2024-09-11T23:32:30.520 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.517+0000 7f8b9ffff640 1 --2- 172.21.15.26:0/3566950549 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8ba0101ad0 0x7f8ba0198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:32:30.520 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.517+0000 7f8ba4b81640 1 --2- 172.21.15.26:0/3566950549 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ba0107b70 0x7f8ba019cc30 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f8b9800eb10 tx=0x7f8b9800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:32:30.520 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.517+0000 7f8b9d7fa640 1 -- 172.21.15.26:0/3566950549 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b98004b20 con 0x7f8ba0107b70 2024-09-11T23:32:30.521 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.517+0000 7f8b9d7fa640 1 -- 172.21.15.26:0/3566950549 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b98004cc0 con 0x7f8ba0107b70 2024-09-11T23:32:30.521 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.517+0000 7f8b9d7fa640 1 -- 172.21.15.26:0/3566950549 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b98010760 con 0x7f8ba0107b70 2024-09-11T23:32:30.521 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.517+0000 7f8ba65e4640 1 -- 172.21.15.26:0/3566950549 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ba019d650 con 0x7f8ba0107b70 2024-09-11T23:32:30.521 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.517+0000 7f8ba65e4640 1 -- 172.21.15.26:0/3566950549 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ba01a4ee0 con 0x7f8ba0107b70 2024-09-11T23:32:30.528 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.520+0000 7f8b9d7fa640 1 -- 172.21.15.26:0/3566950549 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f8b98005020 con 0x7f8ba0107b70 2024-09-11T23:32:30.528 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.520+0000 7f8ba9456740 1 Processor -- start 2024-09-11T23:32:30.528 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.520+0000 7f8ba9456740 1 -- start start 2024-09-11T23:32:30.528 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.520+0000 7f8ba9456740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a6c796daf0 0x55a6c796ded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:30.528 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.520+0000 7f8ba9456740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6c796e410 0x55a6c7978770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:30.528 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.520+0000 7f8ba9456740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a6c7978cb0 0x55a6c797b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:30.528 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.520+0000 7f8ba9456740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55a6c78d22f0 con 0x55a6c796e410 2024-09-11T23:32:30.528 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.520+0000 7f8ba9456740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55a6c787a270 con 0x55a6c7978cb0 2024-09-11T23:32:30.529 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.520+0000 7f8ba9456740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55a6c78ce2b0 con 0x55a6c796daf0 2024-09-11T23:32:30.529 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.520+0000 7f8b9ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6c796e410 0x55a6c7978770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:30.529 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.520+0000 7f8b9ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6c796e410 0x55a6c7978770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33564/0 (socket says 172.21.15.26:33564) 2024-09-11T23:32:30.529 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.520+0000 7f8b9ffff640 1 -- 172.21.15.26:0/500634602 learned_addr learned my addr 172.21.15.26:0/500634602 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:32:30.529 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.520+0000 7f8b9f7fe640 1 --2- 172.21.15.26:0/500634602 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a6c7978cb0 0x55a6c797b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:30.529 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.521+0000 7f8b9f7fe640 1 --2- 172.21.15.26:0/500634602 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a6c796daf0 0x55a6c796ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:30.529 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.521+0000 7f8b9ffff640 1 -- 172.21.15.26:0/500634602 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a6c796daf0 msgr2=0x55a6c796ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:32:30.529 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.521+0000 7f8b9ffff640 1 --2- 172.21.15.26:0/500634602 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a6c796daf0 0x55a6c796ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:30.529 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.521+0000 7f8b9ffff640 1 -- 172.21.15.26:0/500634602 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a6c7978cb0 msgr2=0x55a6c797b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:32:30.529 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.521+0000 7f8b9ffff640 1 --2- 172.21.15.26:0/500634602 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a6c7978cb0 0x55a6c797b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:30.529 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.521+0000 7f8b9ffff640 1 -- 172.21.15.26:0/500634602 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a6c7879340 con 0x55a6c796e410 2024-09-11T23:32:30.529 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.521+0000 7f8b9d7fa640 1 --2- 172.21.15.26:0/3566950549 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f8b880a2da0 0x7f8b880a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:30.530 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.521+0000 7f8b9f7fe640 1 --2- 172.21.15.26:0/3566950549 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f8b880a2da0 0x7f8b880a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:30.530 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.521+0000 7f8b9ffff640 1 --2- 172.21.15.26:0/500634602 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6c796e410 0x55a6c7978770 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f8b90010480 tx=0x7f8b9000bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee6401e695e546cb server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:32:30.530 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.521+0000 7f8b9d7fa640 1 -- 172.21.15.26:0/3566950549 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8542+0+0 (secure 0 0 0) 0x7f8b980cfa70 con 0x7f8ba0107b70 2024-09-11T23:32:30.530 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.521+0000 7f8b867fc640 1 -- 172.21.15.26:0/500634602 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b90011700 con 0x55a6c796e410 2024-09-11T23:32:30.530 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.521+0000 7f8b867fc640 1 -- 172.21.15.26:0/500634602 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b90011d80 con 0x55a6c796e410 2024-09-11T23:32:30.530 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.521+0000 7f8b867fc640 1 -- 172.21.15.26:0/500634602 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b9001a380 con 0x55a6c796e410 2024-09-11T23:32:30.530 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.522+0000 7f8ba9456740 1 -- 172.21.15.26:0/500634602 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6c796e410 msgr2=0x55a6c7978770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:32:30.530 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.522+0000 7f8ba9456740 1 --2- 172.21.15.26:0/500634602 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6c796e410 0x55a6c7978770 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f8b90010480 tx=0x7f8b9000bf90 comp rx=0 tx=0).stop 2024-09-11T23:32:30.530 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.522+0000 7f8b9f7fe640 1 --2- 172.21.15.26:0/3566950549 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f8b880a2da0 0x7f8b880a5260 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f8b94009900 tx=0x7f8b94041040 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:32:30.530 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.522+0000 7f8ba9456740 1 -- 172.21.15.26:0/500634602 shutdown_connections 2024-09-11T23:32:30.530 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.522+0000 7f8ba9456740 1 --2- 172.21.15.26:0/500634602 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a6c7978cb0 0x55a6c797b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:30.530 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.522+0000 7f8ba9456740 1 --2- 172.21.15.26:0/500634602 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6c796e410 0x55a6c7978770 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:30.530 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.522+0000 7f8ba9456740 1 --2- 172.21.15.26:0/500634602 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a6c796daf0 0x55a6c796ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:30.530 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.522+0000 7f8ba9456740 1 -- 172.21.15.26:0/500634602 >> 172.21.15.26:0/500634602 conn(0x55a6c796a060 msgr2=0x55a6c796a440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:32:30.531 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.522+0000 7f8ba9456740 1 -- 172.21.15.26:0/500634602 shutdown_connections 2024-09-11T23:32:30.531 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.522+0000 7f8ba9456740 1 -- 172.21.15.26:0/500634602 wait complete. 2024-09-11T23:32:30.531 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.522+0000 7f8ba9456740 1 -- start start 2024-09-11T23:32:30.531 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.523+0000 7f8ba9456740 1 Processor -- start 2024-09-11T23:32:30.531 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.523+0000 7f8ba9456740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a6c796daf0 0x55a6c7975be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:30.531 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.523+0000 7f8ba9456740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6c796e410 0x55a6c7976120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:30.531 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.523+0000 7f8ba9456740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a6c7978cb0 0x55a6c7976660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:30.531 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.523+0000 7f8ba9456740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55a6c797b880 con 0x55a6c796e410 2024-09-11T23:32:30.531 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.523+0000 7f8ba9456740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55a6c785b290 con 0x55a6c7978cb0 2024-09-11T23:32:30.531 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.523+0000 7f8ba9456740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55a6c797ba00 con 0x55a6c796daf0 2024-09-11T23:32:30.531 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.523+0000 7f8ba9456740 10 client.0 _ll_register_callbacks cb 0x55a6c788c140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:32:30.532 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.523+0000 7f8b9f7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6c796e410 0x55a6c7976120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:30.532 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.523+0000 7f8b9f7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6c796e410 0x55a6c7976120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33566/0 (socket says 172.21.15.26:33566) 2024-09-11T23:32:30.532 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.523+0000 7f8b9f7fe640 1 -- 172.21.15.26:0/2555804056 learned_addr learned my addr 172.21.15.26:0/2555804056 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:32:30.532 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.523+0000 7f8ba4b81640 1 --2- 172.21.15.26:0/2555804056 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a6c7978cb0 0x55a6c7976660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:30.532 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.523+0000 7f8b9ffff640 1 --2- 172.21.15.26:0/2555804056 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a6c796daf0 0x55a6c7975be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:30.532 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.523+0000 7f8b9f7fe640 1 -- 172.21.15.26:0/2555804056 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a6c796daf0 msgr2=0x55a6c7975be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:32:30.532 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.523+0000 7f8b9f7fe640 1 --2- 172.21.15.26:0/2555804056 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a6c796daf0 0x55a6c7975be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:30.532 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.523+0000 7f8b9f7fe640 1 -- 172.21.15.26:0/2555804056 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a6c7978cb0 msgr2=0x55a6c7976660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:32:30.532 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.523+0000 7f8b9f7fe640 1 --2- 172.21.15.26:0/2555804056 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a6c7978cb0 0x55a6c7976660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:30.532 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.523+0000 7f8b9f7fe640 1 -- 172.21.15.26:0/2555804056 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a6c75e5210 con 0x55a6c796e410 2024-09-11T23:32:30.532 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.523+0000 7f8ba4b81640 1 --2- 172.21.15.26:0/2555804056 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a6c7978cb0 0x55a6c7976660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:32:30.532 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.524+0000 7f8b9f7fe640 1 --2- 172.21.15.26:0/2555804056 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6c796e410 0x55a6c7976120 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f8b94009930 tx=0x7f8b9402a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a76d5c3f5bd7374 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:32:30.532 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.524+0000 7f8b7f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:32:30.533 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.524+0000 7f8b7f7fe640 1 -- 172.21.15.26:0/2555804056 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b94050070 con 0x55a6c796e410 2024-09-11T23:32:30.533 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.524+0000 7f8b7f7fe640 1 -- 172.21.15.26:0/2555804056 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8542+0+0 (secure 0 0 0) 0x7f8b94003a00 con 0x55a6c796e410 2024-09-11T23:32:30.533 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.524+0000 7f8ba9456740 1 -- 172.21.15.26:0/2555804056 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8b90011d80 con 0x55a6c796e410 2024-09-11T23:32:30.533 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.524+0000 7f8b7f7fe640 1 -- 172.21.15.26:0/2555804056 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) ==== 24+0+0 (secure 0 0 0) 0x7f8b94003c00 con 0x55a6c796e410 2024-09-11T23:32:30.533 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.524+0000 7f8ba9456740 10 client.17388 fetch_fsmap learned FSMap version 70 2024-09-11T23:32:30.533 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.524+0000 7f8ba9456740 1 -- 172.21.15.26:0/2555804056 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) -- 0x55a6c76b1620 con 0x55a6c796e410 2024-09-11T23:32:30.533 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.524+0000 7f8b7f7fe640 1 -- 172.21.15.26:0/2555804056 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b90011d80 con 0x55a6c796e410 2024-09-11T23:32:30.533 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.524+0000 7f8b7f7fe640 1 -- 172.21.15.26:0/2555804056 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 70) ==== 8434+0+0 (secure 0 0 0) 0x7f8b9403d450 con 0x55a6c796e410 2024-09-11T23:32:30.533 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.524+0000 7f8ba9456740 10 client.17388 fetch_fsmap finished waiting for FSMap version 70 2024-09-11T23:32:30.533 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.524+0000 7f8ba9456740 10 client.17388 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:32:30.533 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.524+0000 7f8ba9456740 20 client.17388 populate_metadata read hostname 'smithi026' 2024-09-11T23:32:30.533 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.524+0000 7f8ba9456740 1 --2- 172.21.15.26:0/2555804056 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55a6c7988d50 0x55a6c798b170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:30.533 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.524+0000 7f8ba9456740 4 client.17388 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:32:30.534 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.524+0000 7f8ba9456740 1 -- 172.21.15.26:0/2555804056 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a6c78dfa20 con 0x55a6c7988d50 2024-09-11T23:32:30.534 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.525+0000 7f8b9ffff640 1 --2- 172.21.15.26:0/2555804056 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55a6c7988d50 0x55a6c798b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:30.534 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.525+0000 7f8b9ffff640 1 --2- 172.21.15.26:0/2555804056 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55a6c7988d50 0x55a6c798b170 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=777f1cef968eae8c server_cookie=3765e3bda6fecb0c in_seq=0 out_seq=0 2024-09-11T23:32:30.534 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.525+0000 7f8b7f7fe640 10 client.17388 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:32:30.534 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.526+0000 7f8b7f7fe640 1 -- 172.21.15.26:0/2555804056 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a6c78dfa20 con 0x55a6c7988d50 2024-09-11T23:32:30.534 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.526+0000 7f8b7f7fe640 10 client.17388 handle_command_reply: tid=0 2024-09-11T23:32:30.534 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.526+0000 7f8ba9456740 1 client.17388 shutdown 2024-09-11T23:32:30.534 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.526+0000 7f8ba9456740 1 -- 172.21.15.26:0/2555804056 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6c796e410 msgr2=0x55a6c7976120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:32:30.534 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.526+0000 7f8ba9456740 1 --2- 172.21.15.26:0/2555804056 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6c796e410 0x55a6c7976120 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f8b94009930 tx=0x7f8b9402a420 comp rx=0 tx=0).stop 2024-09-11T23:32:30.534 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.526+0000 7f8ba9456740 1 -- 172.21.15.26:0/2555804056 shutdown_connections 2024-09-11T23:32:30.534 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.526+0000 7f8ba9456740 1 --2- 172.21.15.26:0/2555804056 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a6c7978cb0 0x55a6c7976660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:30.534 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.526+0000 7f8ba9456740 1 --2- 172.21.15.26:0/2555804056 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55a6c7988d50 0x55a6c798b170 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:30.534 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.526+0000 7f8ba9456740 1 --2- 172.21.15.26:0/2555804056 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6c796e410 0x55a6c7976120 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:30.535 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.526+0000 7f8ba9456740 1 --2- 172.21.15.26:0/2555804056 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a6c796daf0 0x55a6c7975be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:30.535 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.526+0000 7f8ba9456740 1 -- 172.21.15.26:0/2555804056 >> 172.21.15.26:0/2555804056 conn(0x55a6c796a060 msgr2=0x55a6c796fd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:32:30.535 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.526+0000 7f8b7f7fe640 0 client.17388 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:32:30.535 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.526+0000 7f8ba9456740 1 -- 172.21.15.26:0/2555804056 shutdown_connections 2024-09-11T23:32:30.535 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.526+0000 7f8ba9456740 1 -- 172.21.15.26:0/2555804056 wait complete. 2024-09-11T23:32:30.535 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.526+0000 7f8ba9456740 20 client.17388 trim_cache size 0 max 16384 2024-09-11T23:32:30.543 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.541+0000 7f8ba9456740 1 Processor -- start 2024-09-11T23:32:30.544 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.541+0000 7f8ba9456740 1 -- start start 2024-09-11T23:32:30.544 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.541+0000 7f8ba9456740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6c796daf0 0x55a6c796a440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:30.544 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.541+0000 7f8ba9456740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a6c796e410 0x55a6c7976490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:30.544 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.541+0000 7f8ba9456740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a6c7978cb0 0x55a6c7990a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:30.544 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.541+0000 7f8ba9456740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f8ba01ad7d0 con 0x55a6c796daf0 2024-09-11T23:32:30.544 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.541+0000 7f8ba9456740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f8ba01af5b0 con 0x55a6c796e410 2024-09-11T23:32:30.544 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.541+0000 7f8ba9456740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f8ba01adf00 con 0x55a6c7978cb0 2024-09-11T23:32:30.544 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.541+0000 7f8b9ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6c796daf0 0x55a6c796a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:30.544 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.541+0000 7f8ba4b81640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a6c7978cb0 0x55a6c7990a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:30.544 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.541+0000 7f8b9f7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a6c796e410 0x55a6c7976490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:30.544 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.541+0000 7f8ba4b81640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a6c7978cb0 0x55a6c7990a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:33148/0 (socket says 172.21.15.26:33148) 2024-09-11T23:32:30.544 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.541+0000 7f8ba4b81640 1 -- 172.21.15.26:0/4106022001 learned_addr learned my addr 172.21.15.26:0/4106022001 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:32:30.545 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.541+0000 7f8b9f7fe640 1 --2- 172.21.15.26:0/4106022001 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a6c796e410 0x55a6c7976490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:57590/0 (socket says 172.21.15.26:57590) 2024-09-11T23:32:30.545 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.542+0000 7f8ba4b81640 1 -- 172.21.15.26:0/4106022001 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a6c796e410 msgr2=0x55a6c7976490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:32:30.545 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.542+0000 7f8ba4b81640 1 --2- 172.21.15.26:0/4106022001 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a6c796e410 0x55a6c7976490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:30.545 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.542+0000 7f8ba4b81640 1 -- 172.21.15.26:0/4106022001 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6c796daf0 msgr2=0x55a6c796a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:32:30.545 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.542+0000 7f8ba4b81640 1 --2- 172.21.15.26:0/4106022001 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6c796daf0 0x55a6c796a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:30.545 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.542+0000 7f8ba4b81640 1 -- 172.21.15.26:0/4106022001 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a6c796afc0 con 0x55a6c7978cb0 2024-09-11T23:32:30.545 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.542+0000 7f8ba4b81640 1 --2- 172.21.15.26:0/4106022001 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a6c7978cb0 0x55a6c7990a80 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f8b9809beb0 tx=0x7f8b9809bdb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=22a45f6620556c51 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:32:30.545 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.542+0000 7f8b85ffb640 1 -- 172.21.15.26:0/4106022001 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b9809bf20 con 0x55a6c7978cb0 2024-09-11T23:32:30.545 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.542+0000 7f8b85ffb640 1 -- 172.21.15.26:0/4106022001 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b980a02d0 con 0x55a6c7978cb0 2024-09-11T23:32:30.545 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.542+0000 7f8b85ffb640 1 -- 172.21.15.26:0/4106022001 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b98005310 con 0x55a6c7978cb0 2024-09-11T23:32:30.545 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.543+0000 7f8ba9456740 1 -- 172.21.15.26:0/4106022001 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a6c7978cb0 msgr2=0x55a6c7990a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:32:30.545 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.543+0000 7f8ba9456740 1 --2- 172.21.15.26:0/4106022001 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a6c7978cb0 0x55a6c7990a80 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f8b9809beb0 tx=0x7f8b9809bdb0 comp rx=0 tx=0).stop 2024-09-11T23:32:30.545 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.543+0000 7f8ba9456740 1 -- 172.21.15.26:0/4106022001 shutdown_connections 2024-09-11T23:32:30.546 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.543+0000 7f8ba9456740 1 --2- 172.21.15.26:0/4106022001 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a6c796e410 0x55a6c7976490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:30.546 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.543+0000 7f8ba9456740 1 --2- 172.21.15.26:0/4106022001 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a6c7978cb0 0x55a6c7990a80 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:30.546 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.543+0000 7f8ba9456740 1 --2- 172.21.15.26:0/4106022001 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6c796daf0 0x55a6c796a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:30.546 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.543+0000 7f8ba9456740 1 -- 172.21.15.26:0/4106022001 >> 172.21.15.26:0/4106022001 conn(0x55a6c796a060 msgr2=0x55a6c7974cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:32:30.546 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.543+0000 7f8ba9456740 1 -- 172.21.15.26:0/4106022001 shutdown_connections 2024-09-11T23:32:30.546 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.543+0000 7f8ba9456740 1 -- 172.21.15.26:0/4106022001 wait complete. 2024-09-11T23:32:30.546 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.543+0000 7f8ba9456740 1 -- start start 2024-09-11T23:32:30.546 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.543+0000 7f8ba9456740 1 Processor -- start 2024-09-11T23:32:30.549 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.544+0000 7f8ba9456740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6c796daf0 0x55a6c7980c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:30.549 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.544+0000 7f8ba9456740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a6c796e410 0x55a6c79811d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:30.549 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.544+0000 7f8ba9456740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a6c7978cb0 0x55a6c7971fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:30.549 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.544+0000 7f8ba9456740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f8ba01aee20 con 0x55a6c796daf0 2024-09-11T23:32:30.549 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.544+0000 7f8ba9456740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f8ba01ae690 con 0x55a6c796e410 2024-09-11T23:32:30.549 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.544+0000 7f8ba9456740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f8ba01ab7d0 con 0x55a6c7978cb0 2024-09-11T23:32:30.549 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.544+0000 7f8ba9456740 10 client.0 _ll_register_callbacks cb 0x7f8b94032db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:32:30.549 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.544+0000 7f8b9ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6c796daf0 0x55a6c7980c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:30.549 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.544+0000 7f8b9ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6c796daf0 0x55a6c7980c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41732/0 (socket says 172.21.15.26:41732) 2024-09-11T23:32:30.549 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.544+0000 7f8b9ffff640 1 -- 172.21.15.26:0/2111145565 learned_addr learned my addr 172.21.15.26:0/2111145565 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:32:30.549 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.544+0000 7f8b9f7fe640 1 --2- 172.21.15.26:0/2111145565 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a6c796e410 0x55a6c79811d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:30.550 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.544+0000 7f8ba4b81640 1 --2- 172.21.15.26:0/2111145565 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a6c7978cb0 0x55a6c7971fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:30.550 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.544+0000 7f8b9ffff640 1 -- 172.21.15.26:0/2111145565 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a6c7978cb0 msgr2=0x55a6c7971fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:32:30.550 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.544+0000 7f8b9ffff640 1 --2- 172.21.15.26:0/2111145565 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a6c7978cb0 0x55a6c7971fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:30.550 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.544+0000 7f8b9ffff640 1 -- 172.21.15.26:0/2111145565 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a6c796e410 msgr2=0x55a6c79811d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:32:30.550 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.544+0000 7f8b9ffff640 1 --2- 172.21.15.26:0/2111145565 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a6c796e410 0x55a6c79811d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:30.550 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.544+0000 7f8b9ffff640 1 -- 172.21.15.26:0/2111145565 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a6c7972510 con 0x55a6c796daf0 2024-09-11T23:32:30.550 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.544+0000 7f8b9ffff640 1 --2- 172.21.15.26:0/2111145565 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6c796daf0 0x55a6c7980c90 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f8b9000bdd0 tx=0x7f8b9001cb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33777037fef3d047 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:32:30.550 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.544+0000 7f8b7f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:32:30.550 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.544+0000 7f8b7f7fe640 1 -- 172.21.15.26:0/2111145565 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b9002c030 con 0x55a6c796daf0 2024-09-11T23:32:30.550 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.544+0000 7f8b7f7fe640 1 -- 172.21.15.26:0/2111145565 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8542+0+0 (secure 0 0 0) 0x7f8b9001c810 con 0x55a6c796daf0 2024-09-11T23:32:30.550 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.544+0000 7f8ba9456740 1 -- 172.21.15.26:0/2111145565 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8b980a02d0 con 0x55a6c796daf0 2024-09-11T23:32:30.550 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.544+0000 7f8b7f7fe640 1 -- 172.21.15.26:0/2111145565 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) ==== 24+0+0 (secure 0 0 0) 0x7f8b900050b0 con 0x55a6c796daf0 2024-09-11T23:32:30.550 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.544+0000 7f8ba9456740 10 client.17394 fetch_fsmap learned FSMap version 70 2024-09-11T23:32:30.550 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.544+0000 7f8ba9456740 1 -- 172.21.15.26:0/2111145565 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) -- 0x55a6c79f31a0 con 0x55a6c796daf0 2024-09-11T23:32:30.551 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.545+0000 7f8b7f7fe640 1 -- 172.21.15.26:0/2111145565 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b980a02d0 con 0x55a6c796daf0 2024-09-11T23:32:30.551 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.545+0000 7f8b7f7fe640 1 -- 172.21.15.26:0/2111145565 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 70) ==== 8434+0+0 (secure 0 0 0) 0x7f8b90029430 con 0x55a6c796daf0 2024-09-11T23:32:30.551 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.545+0000 7f8ba9456740 10 client.17394 fetch_fsmap finished waiting for FSMap version 70 2024-09-11T23:32:30.551 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.545+0000 7f8ba9456740 10 client.17394 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:32:30.551 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.545+0000 7f8ba9456740 20 client.17394 populate_metadata read hostname 'smithi026' 2024-09-11T23:32:30.551 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.545+0000 7f8ba9456740 1 --2- 172.21.15.26:0/2111145565 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55a6c79f3670 0x55a6c79f5a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:30.551 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.545+0000 7f8ba9456740 4 client.17394 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T23:32:30.551 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.545+0000 7f8ba9456740 1 -- 172.21.15.26:0/2111145565 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a6c78d9d70 con 0x55a6c79f3670 2024-09-11T23:32:30.551 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.545+0000 7f8b9f7fe640 1 --2- 172.21.15.26:0/2111145565 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55a6c79f3670 0x55a6c79f5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:30.551 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.545+0000 7f8b7f7fe640 10 client.17394 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:32:30.551 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.545+0000 7f8b9f7fe640 1 --2- 172.21.15.26:0/2111145565 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55a6c79f3670 0x55a6c79f5a90 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af64dfa4519c9427 server_cookie=41c74a2e7e928703 in_seq=0 out_seq=0 2024-09-11T23:32:30.552 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:32:30 smithi076 ceph-mon[35296]: pgmap v442: 97 pgs: 97 active+clean; 2.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.4 MiB/s wr, 551 op/s 2024-09-11T23:32:30.563 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.562+0000 7f8b7f7fe640 1 -- 172.21.15.26:0/2111145565 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a6c78d9d70 con 0x55a6c79f3670 2024-09-11T23:32:30.563 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.562+0000 7f8b7f7fe640 10 client.17394 handle_command_reply: tid=0 2024-09-11T23:32:30.563 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.562+0000 7f8ba9456740 1 client.17394 shutdown 2024-09-11T23:32:30.563 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.562+0000 7f8ba9456740 1 -- 172.21.15.26:0/2111145565 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6c796daf0 msgr2=0x55a6c7980c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:32:30.563 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.562+0000 7f8ba9456740 1 --2- 172.21.15.26:0/2111145565 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6c796daf0 0x55a6c7980c90 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f8b9000bdd0 tx=0x7f8b9001cb80 comp rx=0 tx=0).stop 2024-09-11T23:32:30.563 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.562+0000 7f8ba9456740 1 -- 172.21.15.26:0/2111145565 shutdown_connections 2024-09-11T23:32:30.563 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.562+0000 7f8ba9456740 1 --2- 172.21.15.26:0/2111145565 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a6c796e410 0x55a6c79811d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:30.563 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.562+0000 7f8ba9456740 1 --2- 172.21.15.26:0/2111145565 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55a6c79f3670 0x55a6c79f5a90 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:30.563 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.562+0000 7f8ba9456740 1 --2- 172.21.15.26:0/2111145565 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a6c7978cb0 0x55a6c7971fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:30.563 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.562+0000 7f8ba9456740 1 --2- 172.21.15.26:0/2111145565 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6c796daf0 0x55a6c7980c90 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:30.563 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.562+0000 7f8b7f7fe640 0 client.17394 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:32:30.564 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.562+0000 7f8ba9456740 1 -- 172.21.15.26:0/2111145565 >> 172.21.15.26:0/2111145565 conn(0x55a6c796a060 msgr2=0x55a6c79689a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:32:30.564 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.562+0000 7f8ba9456740 1 -- 172.21.15.26:0/2111145565 shutdown_connections 2024-09-11T23:32:30.564 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.562+0000 7f8ba9456740 1 -- 172.21.15.26:0/2111145565 wait complete. 2024-09-11T23:32:30.564 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.562+0000 7f8ba9456740 20 client.17394 trim_cache size 0 max 16384 2024-09-11T23:32:30.564 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-09-11T23:32:30.564 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "message": "", 2024-09-11T23:32:30.564 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-09-11T23:32:30.564 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-09-11T23:32:30.565 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.563+0000 7f8ba65e4640 1 -- 172.21.15.26:0/3566950549 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f8b880a2da0 msgr2=0x7f8b880a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:32:30.566 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.563+0000 7f8ba65e4640 1 --2- 172.21.15.26:0/3566950549 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f8b880a2da0 0x7f8b880a5260 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f8b94009900 tx=0x7f8b94041040 comp rx=0 tx=0).stop 2024-09-11T23:32:30.566 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.563+0000 7f8ba65e4640 1 -- 172.21.15.26:0/3566950549 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ba0107b70 msgr2=0x7f8ba019cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:32:30.566 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.563+0000 7f8ba65e4640 1 --2- 172.21.15.26:0/3566950549 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ba0107b70 0x7f8ba019cc30 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f8b9800eb10 tx=0x7f8b9800c2d0 comp rx=0 tx=0).stop 2024-09-11T23:32:30.566 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.563+0000 7f8ba65e4640 1 -- 172.21.15.26:0/3566950549 shutdown_connections 2024-09-11T23:32:30.566 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.563+0000 7f8ba65e4640 1 --2- 172.21.15.26:0/3566950549 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f8b880a2da0 0x7f8b880a5260 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:30.566 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.563+0000 7f8ba65e4640 1 --2- 172.21.15.26:0/3566950549 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ba0107b70 0x7f8ba019cc30 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:30.566 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.563+0000 7f8ba65e4640 1 --2- 172.21.15.26:0/3566950549 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8ba0102470 0x7f8ba01988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:30.566 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.563+0000 7f8ba65e4640 1 --2- 172.21.15.26:0/3566950549 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8ba0101ad0 0x7f8ba0198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:30.566 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.563+0000 7f8ba65e4640 1 -- 172.21.15.26:0/3566950549 >> 172.21.15.26:0/3566950549 conn(0x7f8ba00fd7d0 msgr2=0x7f8ba00fdf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:32:30.566 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.563+0000 7f8ba65e4640 1 -- 172.21.15.26:0/3566950549 shutdown_connections 2024-09-11T23:32:30.566 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:30.563+0000 7f8ba65e4640 1 -- 172.21.15.26:0/3566950549 wait complete. 2024-09-11T23:32:32.474 INFO:tasks.quiescer.fs.[cephfs]:Total duration (60) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-11T23:32:32.474 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 30.0 2024-09-11T23:32:32.475 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:32:32.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:32:32 smithi136 ceph-mon[35121]: pgmap v443: 97 pgs: 97 active+clean; 2.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.8 MiB/s wr, 367 op/s 2024-09-11T23:32:32.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:32:32 smithi026 ceph-mon[31172]: pgmap v443: 97 pgs: 97 active+clean; 2.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.8 MiB/s wr, 367 op/s 2024-09-11T23:32:32.552 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:32:32 smithi076 ceph-mon[35296]: pgmap v443: 97 pgs: 97 active+clean; 2.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.8 MiB/s wr, 367 op/s 2024-09-11T23:32:32.736 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:32:33.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:33.395+0000 7f85de3a1640 1 -- 172.21.15.26:0/2808510334 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f85d8100b00 msgr2=0x7f85d8106750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:32:33.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:33.395+0000 7f85de3a1640 1 --2- 172.21.15.26:0/2808510334 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f85d8100b00 0x7f85d8106750 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f85cc009900 tx=0x7f85cc02a410 comp rx=0 tx=0).stop 2024-09-11T23:32:33.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:33.395+0000 7f85de3a1640 1 -- 172.21.15.26:0/2808510334 shutdown_connections 2024-09-11T23:32:33.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:33.395+0000 7f85de3a1640 1 --2- 172.21.15.26:0/2808510334 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f85d8106e80 0x7f85d810ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:33.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:33.395+0000 7f85de3a1640 1 --2- 172.21.15.26:0/2808510334 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f85d8100b00 0x7f85d8106750 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:33.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:33.395+0000 7f85de3a1640 1 --2- 172.21.15.26:0/2808510334 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f85d8100150 0x7f85d8100530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:33.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:33.395+0000 7f85de3a1640 1 -- 172.21.15.26:0/2808510334 >> 172.21.15.26:0/2808510334 conn(0x7f85d8077010 msgr2=0x7f85d80fe410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:32:33.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:33.395+0000 7f85de3a1640 1 -- 172.21.15.26:0/2808510334 shutdown_connections 2024-09-11T23:32:33.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:33.396+0000 7f85de3a1640 1 -- 172.21.15.26:0/2808510334 wait complete. 2024-09-11T23:32:33.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:33.396+0000 7f85de3a1640 1 Processor -- start 2024-09-11T23:32:33.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:33.397+0000 7f85de3a1640 1 -- start start 2024-09-11T23:32:33.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:33.397+0000 7f85de3a1640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f85d8100150 0x7f85d819a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:33.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:33.397+0000 7f85de3a1640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f85d8100b00 0x7f85d819aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:33.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:33.397+0000 7f85de3a1640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f85d8106e80 0x7f85d819ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:33.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:33.397+0000 7f85de3a1640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f85d8111ee0 con 0x7f85d8106e80 2024-09-11T23:32:33.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:33.397+0000 7f85de3a1640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f85d8111d60 con 0x7f85d8100150 2024-09-11T23:32:33.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:33.397+0000 7f85de3a1640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f85d8112060 con 0x7f85d8100b00 2024-09-11T23:32:33.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:33.397+0000 7f85ddba0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f85d8106e80 0x7f85d819ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:33.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:33.397+0000 7f85dd39f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f85d8100150 0x7f85d819a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:33.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:33.397+0000 7f85ddba0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f85d8106e80 0x7f85d819ede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41750/0 (socket says 172.21.15.26:41750) 2024-09-11T23:32:33.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:33.397+0000 7f85dd39f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f85d8100150 0x7f85d819a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:51328/0 (socket says 172.21.15.26:51328) 2024-09-11T23:32:33.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:33.397+0000 7f85ddba0640 1 -- 172.21.15.26:0/3217986721 learned_addr learned my addr 172.21.15.26:0/3217986721 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:32:33.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:33.397+0000 7f85dcb9e640 1 --2- 172.21.15.26:0/3217986721 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f85d8100b00 0x7f85d819aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:33.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:33.398+0000 7f85ddba0640 1 -- 172.21.15.26:0/3217986721 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f85d8100b00 msgr2=0x7f85d819aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:32:33.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:33.398+0000 7f85ddba0640 1 --2- 172.21.15.26:0/3217986721 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f85d8100b00 0x7f85d819aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:33.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:33.398+0000 7f85ddba0640 1 -- 172.21.15.26:0/3217986721 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f85d8100150 msgr2=0x7f85d819a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:32:33.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:33.398+0000 7f85ddba0640 1 --2- 172.21.15.26:0/3217986721 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f85d8100150 0x7f85d819a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:33.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:33.398+0000 7f85ddba0640 1 -- 172.21.15.26:0/3217986721 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85d819f4c0 con 0x7f85d8106e80 2024-09-11T23:32:33.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:33.398+0000 7f85dd39f640 1 --2- 172.21.15.26:0/3217986721 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f85d8100150 0x7f85d819a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:32:33.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:33.398+0000 7f85ddba0640 1 --2- 172.21.15.26:0/3217986721 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f85d8106e80 0x7f85d819ede0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f85d000e970 tx=0x7f85d000ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:32:33.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:33.398+0000 7f85d67fc640 1 -- 172.21.15.26:0/3217986721 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85d000ccc0 con 0x7f85d8106e80 2024-09-11T23:32:33.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:33.398+0000 7f85de3a1640 1 -- 172.21.15.26:0/3217986721 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85d819f7b0 con 0x7f85d8106e80 2024-09-11T23:32:33.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:33.398+0000 7f85d67fc640 1 -- 172.21.15.26:0/3217986721 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85d000ce60 con 0x7f85d8106e80 2024-09-11T23:32:33.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:33.398+0000 7f85d67fc640 1 -- 172.21.15.26:0/3217986721 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85d0010450 con 0x7f85d8106e80 2024-09-11T23:32:33.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:33.398+0000 7f85de3a1640 1 -- 172.21.15.26:0/3217986721 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85d81a7090 con 0x7f85d8106e80 2024-09-11T23:32:33.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:33.399+0000 7f85de3a1640 1 -- 172.21.15.26:0/3217986721 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85d8100530 con 0x7f85d8106e80 2024-09-11T23:32:33.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:33.400+0000 7f85d67fc640 1 -- 172.21.15.26:0/3217986721 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f85d0005020 con 0x7f85d8106e80 2024-09-11T23:32:33.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:33.401+0000 7f85d67fc640 1 --2- 172.21.15.26:0/3217986721 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f85c00a2cf0 0x7f85c00a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:33.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:33.401+0000 7f85dd39f640 1 --2- 172.21.15.26:0/3217986721 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f85c00a2cf0 0x7f85c00a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:33.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:33.401+0000 7f85d67fc640 1 -- 172.21.15.26:0/3217986721 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8542+0+0 (secure 0 0 0) 0x7f85d00cf8c0 con 0x7f85d8106e80 2024-09-11T23:32:33.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:33.401+0000 7f85dd39f640 1 --2- 172.21.15.26:0/3217986721 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f85c00a2cf0 0x7f85c00a51b0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f85c8005fd0 tx=0x7f85c8009500 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:32:33.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:33.405+0000 7f85d67fc640 1 -- 172.21.15.26:0/3217986721 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f85d009c4d0 con 0x7f85d8106e80 2024-09-11T23:32:33.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:33.620+0000 7f85de3a1640 1 -- 172.21.15.26:0/3217986721 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f85d81a72c0 con 0x7f85d8106e80 2024-09-11T23:32:33.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:33.621+0000 7f85d67fc640 1 -- 172.21.15.26:0/3217986721 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 70 v70) ==== 94+0+12003 (secure 0 0 0) 0x7f85d009bdd0 con 0x7f85d8106e80 2024-09-11T23:32:33.622 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-11T23:32:33.622 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":70,"btime":"2024-09-11T23:32:23:061031+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14499,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/367659515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":367659515},{"type":"v1","addr":"172.21.15.26:6835","nonce":367659515}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14511,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/2041065299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":2041065299},{"type":"v1","addr":"172.21.15.26:6837","nonce":2041065299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":70,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:16:26.544203+0000","modified":"2024-09-11T23:32:22.843897+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24485,"mds_1":24463,"mds_2":24458,"mds_3":24448,"mds_4":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14550":{"gid":14550,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6839/2233068305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":2233068305},{"type":"v1","addr":"172.21.15.26:6839","nonce":2233068305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14562":{"gid":14562,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6841/2984982336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":2984982336},{"type":"v1","addr":"172.21.15.26:6841","nonce":2984982336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6833/2019841455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":2019841455},{"type":"v1","addr":"172.21.15.136:6833","nonce":2019841455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.76:6833/3438147050","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3438147050},{"type":"v1","addr":"172.21.15.76:6833","nonce":3438147050}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6835/3617665975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3617665975},{"type":"v1","addr":"172.21.15.136:6835","nonce":3617665975}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":12,"addr":"172.21.15.76:6835/393308499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":393308499},{"type":"v1","addr":"172.21.15.76:6835","nonce":393308499}]},"join_fscid":-1,"export_targets":[1,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":10,"addr":"172.21.15.136:6837/1633082387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":1633082387},{"type":"v1","addr":"172.21.15.136:6837","nonce":1633082387}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.76:6837/2078978748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":2078978748},{"type":"v1","addr":"172.21.15.76:6837","nonce":2078978748}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":63,"addr":"172.21.15.136:6839/3211398630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":3211398630},{"type":"v1","addr":"172.21.15.136:6839","nonce":3211398630}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.76:6839/2077637090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6838","nonce":2077637090},{"type":"v1","addr":"172.21.15.76:6839","nonce":2077637090}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24485,"qdb_cluster":[24485,24463,24458,24448,24443]},"id":1}]} 2024-09-11T23:32:33.622 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 70 2024-09-11T23:32:33.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:33.623+0000 7f85de3a1640 1 -- 172.21.15.26:0/3217986721 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f85c00a2cf0 msgr2=0x7f85c00a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:32:33.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:33.623+0000 7f85de3a1640 1 --2- 172.21.15.26:0/3217986721 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f85c00a2cf0 0x7f85c00a51b0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f85c8005fd0 tx=0x7f85c8009500 comp rx=0 tx=0).stop 2024-09-11T23:32:33.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:33.623+0000 7f85de3a1640 1 -- 172.21.15.26:0/3217986721 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f85d8106e80 msgr2=0x7f85d819ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:32:33.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:33.623+0000 7f85de3a1640 1 --2- 172.21.15.26:0/3217986721 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f85d8106e80 0x7f85d819ede0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f85d000e970 tx=0x7f85d000ed70 comp rx=0 tx=0).stop 2024-09-11T23:32:33.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:33.623+0000 7f85de3a1640 1 -- 172.21.15.26:0/3217986721 shutdown_connections 2024-09-11T23:32:33.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:33.624+0000 7f85de3a1640 1 --2- 172.21.15.26:0/3217986721 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f85c00a2cf0 0x7f85c00a51b0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:33.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:33.624+0000 7f85de3a1640 1 --2- 172.21.15.26:0/3217986721 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f85d8106e80 0x7f85d819ede0 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:33.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:33.624+0000 7f85de3a1640 1 --2- 172.21.15.26:0/3217986721 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f85d8100b00 0x7f85d819aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:33.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:33.624+0000 7f85de3a1640 1 --2- 172.21.15.26:0/3217986721 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f85d8100150 0x7f85d819a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:33.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:33.624+0000 7f85de3a1640 1 -- 172.21.15.26:0/3217986721 >> 172.21.15.26:0/3217986721 conn(0x7f85d8077010 msgr2=0x7f85d81055f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:32:33.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:33.624+0000 7f85de3a1640 1 -- 172.21.15.26:0/3217986721 shutdown_connections 2024-09-11T23:32:33.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:33.624+0000 7f85de3a1640 1 -- 172.21.15.26:0/3217986721 wait complete. 2024-09-11T23:32:34.201 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:32:34 smithi026 ceph-mon[31172]: pgmap v444: 97 pgs: 97 active+clean; 2.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.8 MiB/s wr, 345 op/s 2024-09-11T23:32:34.201 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:32:34 smithi026 ceph-mon[31172]: from='client.? 172.21.15.26:0/3217986721' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:32:34.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:32:34 smithi136 ceph-mon[35121]: pgmap v444: 97 pgs: 97 active+clean; 2.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.8 MiB/s wr, 345 op/s 2024-09-11T23:32:34.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:32:34 smithi136 ceph-mon[35121]: from='client.? 172.21.15.26:0/3217986721' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:32:34.508 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24485 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0' 2024-09-11T23:32:34.508 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24485 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0 2024-09-11T23:32:34.552 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:32:34 smithi076 ceph-mon[35296]: pgmap v444: 97 pgs: 97 active+clean; 2.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.8 MiB/s wr, 345 op/s 2024-09-11T23:32:34.552 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:32:34 smithi076 ceph-mon[35296]: from='client.? 172.21.15.26:0/3217986721' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:32:34.824 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:32:35.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.575+0000 7f15c6db4640 1 -- 172.21.15.26:0/4113561676 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f15c0100e20 msgr2=0x7f15c0101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:32:35.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.575+0000 7f15c6db4640 1 --2- 172.21.15.26:0/4113561676 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f15c0100e20 0x7f15c0101200 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f15b4009890 tx=0x7f15b402a410 comp rx=0 tx=0).stop 2024-09-11T23:32:35.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.575+0000 7f15c6db4640 1 -- 172.21.15.26:0/4113561676 shutdown_connections 2024-09-11T23:32:35.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.575+0000 7f15c6db4640 1 --2- 172.21.15.26:0/4113561676 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15c0102ef0 0x7f15c010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:35.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.575+0000 7f15c6db4640 1 --2- 172.21.15.26:0/4113561676 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f15c0102570 0x7f15c01029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:35.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.575+0000 7f15c6db4640 1 --2- 172.21.15.26:0/4113561676 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f15c0100e20 0x7f15c0101200 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:35.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.575+0000 7f15c6db4640 1 -- 172.21.15.26:0/4113561676 >> 172.21.15.26:0/4113561676 conn(0x7f15c00fc7d0 msgr2=0x7f15c00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:32:35.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.575+0000 7f15c6db4640 1 -- 172.21.15.26:0/4113561676 shutdown_connections 2024-09-11T23:32:35.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.575+0000 7f15c6db4640 1 -- 172.21.15.26:0/4113561676 wait complete. 2024-09-11T23:32:35.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.576+0000 7f15c6db4640 1 Processor -- start 2024-09-11T23:32:35.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.576+0000 7f15c6db4640 1 -- start start 2024-09-11T23:32:35.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.576+0000 7f15c6db4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15c0100e20 0x7f15c00fcf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:35.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.576+0000 7f15c6db4640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f15c0102570 0x7f15c00fd440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:35.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.576+0000 7f15c6db4640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f15c0102ef0 0x7f15c00fd980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:35.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.576+0000 7f15c6db4640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f15c0110f80 con 0x7f15c0100e20 2024-09-11T23:32:35.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.576+0000 7f15c6db4640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f15c0110e00 con 0x7f15c0102570 2024-09-11T23:32:35.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.576+0000 7f15c6db4640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f15c0111100 con 0x7f15c0102ef0 2024-09-11T23:32:35.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.576+0000 7f15c4b50640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15c0100e20 0x7f15c00fcf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:35.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.576+0000 7f15c4b50640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15c0100e20 0x7f15c00fcf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41756/0 (socket says 172.21.15.26:41756) 2024-09-11T23:32:35.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.576+0000 7f15c4b50640 1 -- 172.21.15.26:0/3308627191 learned_addr learned my addr 172.21.15.26:0/3308627191 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:32:35.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.576+0000 7f15bffff640 1 --2- 172.21.15.26:0/3308627191 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f15c0102570 0x7f15c00fd440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:35.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.577+0000 7f15c5351640 1 --2- 172.21.15.26:0/3308627191 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f15c0102ef0 0x7f15c00fd980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:35.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.577+0000 7f15c4b50640 1 -- 172.21.15.26:0/3308627191 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f15c0102ef0 msgr2=0x7f15c00fd980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:32:35.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.577+0000 7f15c4b50640 1 --2- 172.21.15.26:0/3308627191 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f15c0102ef0 0x7f15c00fd980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:35.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.577+0000 7f15c4b50640 1 -- 172.21.15.26:0/3308627191 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f15c0102570 msgr2=0x7f15c00fd440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:32:35.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.577+0000 7f15c4b50640 1 --2- 172.21.15.26:0/3308627191 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f15c0102570 0x7f15c00fd440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:35.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.577+0000 7f15c4b50640 1 -- 172.21.15.26:0/3308627191 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15c01a6a30 con 0x7f15c0100e20 2024-09-11T23:32:35.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.577+0000 7f15bffff640 1 --2- 172.21.15.26:0/3308627191 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f15c0102570 0x7f15c00fd440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:32:35.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.577+0000 7f15c4b50640 1 --2- 172.21.15.26:0/3308627191 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15c0100e20 0x7f15c00fcf00 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f15b4005940 tx=0x7f15b4030650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:32:35.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.577+0000 7f15bdffb640 1 -- 172.21.15.26:0/3308627191 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15b4040020 con 0x7f15c0100e20 2024-09-11T23:32:35.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.577+0000 7f15c6db4640 1 -- 172.21.15.26:0/3308627191 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15c01a6cc0 con 0x7f15c0100e20 2024-09-11T23:32:35.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.577+0000 7f15bdffb640 1 -- 172.21.15.26:0/3308627191 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f15b4004440 con 0x7f15c0100e20 2024-09-11T23:32:35.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.577+0000 7f15bdffb640 1 -- 172.21.15.26:0/3308627191 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15b40046c0 con 0x7f15c0100e20 2024-09-11T23:32:35.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.578+0000 7f15c6db4640 1 -- 172.21.15.26:0/3308627191 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15c01a71a0 con 0x7f15c0100e20 2024-09-11T23:32:35.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.580+0000 7f15bdffb640 1 -- 172.21.15.26:0/3308627191 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f15b4043020 con 0x7f15c0100e20 2024-09-11T23:32:35.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.580+0000 7f15c918d740 1 Processor -- start 2024-09-11T23:32:35.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.580+0000 7f15c918d740 1 -- start start 2024-09-11T23:32:35.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.580+0000 7f15c918d740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c939f90760 0x55c939f90b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:35.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.580+0000 7f15c918d740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c939f91080 0x55c939f9b400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:35.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.581+0000 7f15c918d740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c939f9b940 0x55c939f9dd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:35.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.581+0000 7f15c918d740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55c939ef62a0 con 0x55c939f91080 2024-09-11T23:32:35.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.581+0000 7f15c918d740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55c939e9db10 con 0x55c939f90760 2024-09-11T23:32:35.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.581+0000 7f15c918d740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55c939ef2260 con 0x55c939f9b940 2024-09-11T23:32:35.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.581+0000 7f15c5351640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c939f90760 0x55c939f90b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:35.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.581+0000 7f15c4b50640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c939f91080 0x55c939f9b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:35.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.581+0000 7f15bdffb640 1 --2- 172.21.15.26:0/3308627191 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f15b00a2cf0 0x7f15b00a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:35.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.581+0000 7f15c5351640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c939f90760 0x55c939f90b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:51370/0 (socket says 172.21.15.26:51370) 2024-09-11T23:32:35.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.581+0000 7f15c5351640 1 -- 172.21.15.26:0/3113267834 learned_addr learned my addr 172.21.15.26:0/3113267834 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:32:35.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.581+0000 7f15bffff640 1 --2- 172.21.15.26:0/3113267834 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c939f9b940 0x55c939f9dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:35.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.581+0000 7f15c5351640 1 --2- 172.21.15.26:0/3308627191 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f15b00a2cf0 0x7f15b00a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:35.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.581+0000 7f15bdffb640 1 -- 172.21.15.26:0/3308627191 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8542+0+0 (secure 0 0 0) 0x7f15b40f03f0 con 0x7f15c0100e20 2024-09-11T23:32:35.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.581+0000 7f15c5351640 1 -- 172.21.15.26:0/3113267834 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c939f9b940 msgr2=0x55c939f9dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:32:35.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.581+0000 7f15c5351640 1 --2- 172.21.15.26:0/3113267834 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c939f9b940 0x55c939f9dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:35.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.581+0000 7f15c5351640 1 -- 172.21.15.26:0/3113267834 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c939f91080 msgr2=0x55c939f9b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:32:35.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.581+0000 7f15c5351640 1 --2- 172.21.15.26:0/3113267834 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c939f91080 0x55c939f9b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:35.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.581+0000 7f15c5351640 1 -- 172.21.15.26:0/3113267834 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c939c67b90 con 0x55c939f90760 2024-09-11T23:32:35.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.581+0000 7f15c4b50640 1 --2- 172.21.15.26:0/3113267834 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c939f91080 0x55c939f9b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:32:35.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.581+0000 7f15bffff640 1 --2- 172.21.15.26:0/3113267834 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c939f9b940 0x55c939f9dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:32:35.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.581+0000 7f15c5351640 1 --2- 172.21.15.26:0/3113267834 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c939f90760 0x55c939f90b40 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f15b800fa20 tx=0x7f15b800fee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eab7564baa2b5738 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:32:35.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.582+0000 7f15c5351640 1 --2- 172.21.15.26:0/3308627191 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f15b00a2cf0 0x7f15b00a51b0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f15b801d000 tx=0x7f15b801f000 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:32:35.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.582+0000 7f15aaffd640 1 -- 172.21.15.26:0/3113267834 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15b8018050 con 0x55c939f90760 2024-09-11T23:32:35.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.582+0000 7f15aaffd640 1 -- 172.21.15.26:0/3113267834 <== mon.2 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f15b8021070 con 0x55c939f90760 2024-09-11T23:32:35.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.582+0000 7f15aaffd640 1 -- 172.21.15.26:0/3113267834 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15b800e690 con 0x55c939f90760 2024-09-11T23:32:35.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.582+0000 7f15c918d740 1 -- 172.21.15.26:0/3113267834 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c939f90760 msgr2=0x55c939f90b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:32:35.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.582+0000 7f15c918d740 1 --2- 172.21.15.26:0/3113267834 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c939f90760 0x55c939f90b40 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f15b800fa20 tx=0x7f15b800fee0 comp rx=0 tx=0).stop 2024-09-11T23:32:35.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.582+0000 7f15c918d740 1 -- 172.21.15.26:0/3113267834 shutdown_connections 2024-09-11T23:32:35.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.582+0000 7f15c918d740 1 --2- 172.21.15.26:0/3113267834 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c939f9b940 0x55c939f9dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:35.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.582+0000 7f15c918d740 1 --2- 172.21.15.26:0/3113267834 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c939f91080 0x55c939f9b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:35.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.582+0000 7f15c918d740 1 --2- 172.21.15.26:0/3113267834 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c939f90760 0x55c939f90b40 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:35.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.582+0000 7f15c918d740 1 -- 172.21.15.26:0/3113267834 >> 172.21.15.26:0/3113267834 conn(0x55c939f8ccf0 msgr2=0x55c939f8d0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:32:35.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.582+0000 7f15c918d740 1 -- 172.21.15.26:0/3113267834 shutdown_connections 2024-09-11T23:32:35.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.582+0000 7f15c918d740 1 -- 172.21.15.26:0/3113267834 wait complete. 2024-09-11T23:32:35.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.583+0000 7f15c918d740 1 -- start start 2024-09-11T23:32:35.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.583+0000 7f15c918d740 1 Processor -- start 2024-09-11T23:32:35.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.583+0000 7f15c918d740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c939f90760 0x55c939f98860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:35.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.583+0000 7f15c918d740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c939f91080 0x55c939f98da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:35.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.583+0000 7f15c918d740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c939f9b940 0x55c939f992e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:35.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.583+0000 7f15c918d740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55c939f9e510 con 0x55c939f90760 2024-09-11T23:32:35.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.583+0000 7f15c918d740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55c939e7e6e0 con 0x55c939f91080 2024-09-11T23:32:35.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.583+0000 7f15c918d740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55c939f9e690 con 0x55c939f9b940 2024-09-11T23:32:35.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.583+0000 7f15c918d740 10 client.0 _ll_register_callbacks cb 0x55c939ea2a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:32:35.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.583+0000 7f15c4b50640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c939f90760 0x55c939f98860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:35.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.583+0000 7f15bffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c939f91080 0x55c939f98da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:35.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.583+0000 7f15bffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c939f91080 0x55c939f98da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:51382/0 (socket says 172.21.15.26:51382) 2024-09-11T23:32:35.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.583+0000 7f15c4b50640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c939f90760 0x55c939f98860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41764/0 (socket says 172.21.15.26:41764) 2024-09-11T23:32:35.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.583+0000 7f15bffff640 1 -- 172.21.15.26:0/824436644 learned_addr learned my addr 172.21.15.26:0/824436644 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:32:35.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.584+0000 7f15c5351640 1 --2- 172.21.15.26:0/824436644 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c939f9b940 0x55c939f992e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:35.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.584+0000 7f15bffff640 1 -- 172.21.15.26:0/824436644 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c939f9b940 msgr2=0x55c939f992e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:32:35.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.584+0000 7f15bffff640 1 --2- 172.21.15.26:0/824436644 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c939f9b940 0x55c939f992e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:35.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.584+0000 7f15bffff640 1 -- 172.21.15.26:0/824436644 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c939f90760 msgr2=0x55c939f98860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:32:35.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.584+0000 7f15bffff640 1 --2- 172.21.15.26:0/824436644 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c939f90760 0x55c939f98860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:35.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.584+0000 7f15bffff640 1 -- 172.21.15.26:0/824436644 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c939fa7a30 con 0x55c939f91080 2024-09-11T23:32:35.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.584+0000 7f15c4b50640 1 --2- 172.21.15.26:0/824436644 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c939f90760 0x55c939f98860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:32:35.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.584+0000 7f15bffff640 1 --2- 172.21.15.26:0/824436644 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c939f91080 0x55c939f98da0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f15ac012a20 tx=0x7f15ac012ee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=21a80029b2a5ee51 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:32:35.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.584+0000 7f159ffff640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-11T23:32:35.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.584+0000 7f159ffff640 1 -- 172.21.15.26:0/824436644 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15ac00fac0 con 0x55c939f91080 2024-09-11T23:32:35.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.584+0000 7f159ffff640 1 -- 172.21.15.26:0/824436644 <== mon.2 v2:172.21.15.76:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8542+0+0 (secure 0 0 0) 0x7f15ac00fc60 con 0x55c939f91080 2024-09-11T23:32:35.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.584+0000 7f15c918d740 1 -- 172.21.15.26:0/824436644 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f15b8021070 con 0x55c939f91080 2024-09-11T23:32:35.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.584+0000 7f159ffff640 1 -- 172.21.15.26:0/824436644 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) ==== 24+0+0 (secure 0 0 0) 0x7f15ac00fe60 con 0x55c939f91080 2024-09-11T23:32:35.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.584+0000 7f15c918d740 10 client.26942 fetch_fsmap learned FSMap version 70 2024-09-11T23:32:35.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.584+0000 7f15c918d740 1 -- 172.21.15.26:0/824436644 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) -- 0x55c939fb2c30 con 0x55c939f91080 2024-09-11T23:32:35.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.584+0000 7f159ffff640 1 -- 172.21.15.26:0/824436644 <== mon.2 v2:172.21.15.76:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f15b8021070 con 0x55c939f91080 2024-09-11T23:32:35.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.585+0000 7f159ffff640 1 -- 172.21.15.26:0/824436644 <== mon.2 v2:172.21.15.76:3300/0 5 ==== fsmap(e 70) ==== 8434+0+0 (secure 0 0 0) 0x7f15ac010430 con 0x55c939f91080 2024-09-11T23:32:35.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.585+0000 7f15c918d740 10 client.26942 fetch_fsmap finished waiting for FSMap version 70 2024-09-11T23:32:35.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.585+0000 7f15c918d740 10 client.26942 resolve_mds: validated gid 24485 aka daemon mds.b 2024-09-11T23:32:35.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.585+0000 7f15c918d740 20 client.26942 populate_metadata read hostname 'smithi026' 2024-09-11T23:32:35.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.585+0000 7f15c918d740 1 --2- 172.21.15.26:0/824436644 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55c939fb40f0 0x55c939fb6500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:35.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.585+0000 7f15c918d740 4 client.26942 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:32:35.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.585+0000 7f15c918d740 1 -- 172.21.15.26:0/824436644 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c939f039d0 con 0x55c939fb40f0 2024-09-11T23:32:35.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.585+0000 7f15c4b50640 1 --2- 172.21.15.26:0/824436644 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55c939fb40f0 0x55c939fb6500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:35.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.585+0000 7f159ffff640 10 client.26942 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:32:35.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.585+0000 7f15c4b50640 1 --2- 172.21.15.26:0/824436644 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55c939fb40f0 0x55c939fb6500 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9cc5bd6a8118aca3 server_cookie=d225d47a1aad4150 in_seq=0 out_seq=0 2024-09-11T23:32:35.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.586+0000 7f159ffff640 1 -- 172.21.15.26:0/824436644 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c939f039d0 con 0x55c939fb40f0 2024-09-11T23:32:35.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.586+0000 7f159ffff640 10 client.26942 handle_command_reply: tid=0 2024-09-11T23:32:35.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.586+0000 7f15c918d740 1 client.26942 shutdown 2024-09-11T23:32:35.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.586+0000 7f15c918d740 1 -- 172.21.15.26:0/824436644 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c939f91080 msgr2=0x55c939f98da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:32:35.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.586+0000 7f15c918d740 1 --2- 172.21.15.26:0/824436644 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c939f91080 0x55c939f98da0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f15ac012a20 tx=0x7f15ac012ee0 comp rx=0 tx=0).stop 2024-09-11T23:32:35.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.586+0000 7f15c918d740 1 -- 172.21.15.26:0/824436644 shutdown_connections 2024-09-11T23:32:35.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.586+0000 7f15c918d740 1 --2- 172.21.15.26:0/824436644 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c939f91080 0x55c939f98da0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:35.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.586+0000 7f15c918d740 1 --2- 172.21.15.26:0/824436644 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55c939fb40f0 0x55c939fb6500 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:35.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.586+0000 7f15c918d740 1 --2- 172.21.15.26:0/824436644 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c939f9b940 0x55c939f992e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:35.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.586+0000 7f15c918d740 1 --2- 172.21.15.26:0/824436644 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c939f90760 0x55c939f98860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:35.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.586+0000 7f15c918d740 1 -- 172.21.15.26:0/824436644 >> 172.21.15.26:0/824436644 conn(0x55c939f8ccf0 msgr2=0x55c939f92a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:32:35.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.586+0000 7f159ffff640 0 client.26942 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:32:35.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.586+0000 7f15c918d740 1 -- 172.21.15.26:0/824436644 shutdown_connections 2024-09-11T23:32:35.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.586+0000 7f15c918d740 1 -- 172.21.15.26:0/824436644 wait complete. 2024-09-11T23:32:35.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.586+0000 7f15c918d740 20 client.26942 trim_cache size 0 max 16384 2024-09-11T23:32:35.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.603+0000 7f15c918d740 1 Processor -- start 2024-09-11T23:32:35.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.603+0000 7f15c918d740 1 -- start start 2024-09-11T23:32:35.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.604+0000 7f15c918d740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c939f90760 0x55c939f931c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:35.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.604+0000 7f15c918d740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c939f91080 0x55c939f8bd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:35.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.604+0000 7f15c918d740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c939f9b940 0x55c939fbbe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:35.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.604+0000 7f15c918d740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f15c01ab790 con 0x55c939f90760 2024-09-11T23:32:35.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.604+0000 7f15c918d740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f15c01ad570 con 0x55c939f9b940 2024-09-11T23:32:35.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.604+0000 7f15c918d740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f15c01abec0 con 0x55c939f91080 2024-09-11T23:32:35.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.604+0000 7f15c4b50640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c939f90760 0x55c939f931c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:35.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.604+0000 7f15c4b50640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c939f90760 0x55c939f931c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41768/0 (socket says 172.21.15.26:41768) 2024-09-11T23:32:35.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.604+0000 7f15c4b50640 1 -- 172.21.15.26:0/421213287 learned_addr learned my addr 172.21.15.26:0/421213287 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:32:35.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.604+0000 7f15c5351640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c939f9b940 0x55c939fbbe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:35.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.604+0000 7f15bffff640 1 --2- 172.21.15.26:0/421213287 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c939f91080 0x55c939f8bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:35.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.604+0000 7f15c4b50640 1 -- 172.21.15.26:0/421213287 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c939f91080 msgr2=0x55c939f8bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:32:35.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.604+0000 7f15c4b50640 1 --2- 172.21.15.26:0/421213287 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c939f91080 0x55c939f8bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:35.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.604+0000 7f15c4b50640 1 -- 172.21.15.26:0/421213287 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c939f9b940 msgr2=0x55c939fbbe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:32:35.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.604+0000 7f15c4b50640 1 --2- 172.21.15.26:0/421213287 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c939f9b940 0x55c939fbbe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:35.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.604+0000 7f15c4b50640 1 -- 172.21.15.26:0/421213287 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c939f88260 con 0x55c939f90760 2024-09-11T23:32:35.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.604+0000 7f15c5351640 1 --2- 172.21.15.26:0/421213287 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c939f9b940 0x55c939fbbe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:32:35.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.604+0000 7f15c4b50640 1 --2- 172.21.15.26:0/421213287 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c939f90760 0x55c939f931c0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f15b40bc830 tx=0x7f15b40a7040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38b3509926149655 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:32:35.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.604+0000 7f15aa7fc640 1 -- 172.21.15.26:0/421213287 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15b40c0c50 con 0x55c939f90760 2024-09-11T23:32:35.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.605+0000 7f15aa7fc640 1 -- 172.21.15.26:0/421213287 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f15b40c5510 con 0x55c939f90760 2024-09-11T23:32:35.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.605+0000 7f15aa7fc640 1 -- 172.21.15.26:0/421213287 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15b40a70b0 con 0x55c939f90760 2024-09-11T23:32:35.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.605+0000 7f15c918d740 1 -- 172.21.15.26:0/421213287 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c939f90760 msgr2=0x55c939f931c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:32:35.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.605+0000 7f15c918d740 1 --2- 172.21.15.26:0/421213287 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c939f90760 0x55c939f931c0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f15b40bc830 tx=0x7f15b40a7040 comp rx=0 tx=0).stop 2024-09-11T23:32:35.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.605+0000 7f15c918d740 1 -- 172.21.15.26:0/421213287 shutdown_connections 2024-09-11T23:32:35.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.605+0000 7f15c918d740 1 --2- 172.21.15.26:0/421213287 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c939f9b940 0x55c939fbbe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:35.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.605+0000 7f15c918d740 1 --2- 172.21.15.26:0/421213287 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c939f91080 0x55c939f8bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:35.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.605+0000 7f15c918d740 1 --2- 172.21.15.26:0/421213287 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c939f90760 0x55c939f931c0 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:35.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.605+0000 7f15c918d740 1 -- 172.21.15.26:0/421213287 >> 172.21.15.26:0/421213287 conn(0x55c939f8ccf0 msgr2=0x55c939f8b650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:32:35.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.605+0000 7f15c918d740 1 -- 172.21.15.26:0/421213287 shutdown_connections 2024-09-11T23:32:35.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.605+0000 7f15c918d740 1 -- 172.21.15.26:0/421213287 wait complete. 2024-09-11T23:32:35.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.606+0000 7f15c918d740 1 -- start start 2024-09-11T23:32:35.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.606+0000 7f15c918d740 1 Processor -- start 2024-09-11T23:32:35.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.606+0000 7f15c918d740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c939f90760 0x55c939fa38e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:35.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.606+0000 7f15c918d740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c939f91080 0x55c939fa3e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:35.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.606+0000 7f15c918d740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c939f9b940 0x55c939fa08b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:35.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.606+0000 7f15c918d740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f15c01acde0 con 0x55c939f90760 2024-09-11T23:32:35.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.606+0000 7f15c918d740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f15c01ac650 con 0x55c939f91080 2024-09-11T23:32:35.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.606+0000 7f15c918d740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f15c01a9790 con 0x55c939f9b940 2024-09-11T23:32:35.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.606+0000 7f15c918d740 10 client.0 _ll_register_callbacks cb 0x7f15ac012600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:32:35.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.606+0000 7f15c4b50640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c939f90760 0x55c939fa38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:35.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.606+0000 7f15bffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c939f91080 0x55c939fa3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:35.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.606+0000 7f15c4b50640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c939f90760 0x55c939fa38e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41778/0 (socket says 172.21.15.26:41778) 2024-09-11T23:32:35.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.606+0000 7f15bffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c939f91080 0x55c939fa3e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:51400/0 (socket says 172.21.15.26:51400) 2024-09-11T23:32:35.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.606+0000 7f15c4b50640 1 -- 172.21.15.26:0/2318607499 learned_addr learned my addr 172.21.15.26:0/2318607499 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:32:35.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.606+0000 7f15c5351640 1 --2- 172.21.15.26:0/2318607499 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c939f9b940 0x55c939fa08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:35.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.607+0000 7f15c4b50640 1 -- 172.21.15.26:0/2318607499 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c939f9b940 msgr2=0x55c939fa08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:32:35.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.607+0000 7f15c4b50640 1 --2- 172.21.15.26:0/2318607499 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c939f9b940 0x55c939fa08b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:35.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.607+0000 7f15c4b50640 1 -- 172.21.15.26:0/2318607499 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c939f91080 msgr2=0x55c939fa3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:32:35.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.607+0000 7f15c4b50640 1 --2- 172.21.15.26:0/2318607499 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c939f91080 0x55c939fa3e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:35.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.607+0000 7f15c4b50640 1 -- 172.21.15.26:0/2318607499 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c939f8d980 con 0x55c939f90760 2024-09-11T23:32:35.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.607+0000 7f15bffff640 1 --2- 172.21.15.26:0/2318607499 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c939f91080 0x55c939fa3e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:32:35.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.607+0000 7f15c4b50640 1 --2- 172.21.15.26:0/2318607499 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c939f90760 0x55c939fa38e0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f15b40ee0d0 tx=0x7f15b4005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54c3cca4b252f43e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:32:35.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.607+0000 7f159ffff640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:32:35.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.607+0000 7f159ffff640 1 -- 172.21.15.26:0/2318607499 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15b40d4070 con 0x55c939f90760 2024-09-11T23:32:35.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.607+0000 7f159ffff640 1 -- 172.21.15.26:0/2318607499 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8542+0+0 (secure 0 0 0) 0x7f15b4031b30 con 0x55c939f90760 2024-09-11T23:32:35.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.607+0000 7f15c918d740 1 -- 172.21.15.26:0/2318607499 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f15b40c5510 con 0x55c939f90760 2024-09-11T23:32:35.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.607+0000 7f159ffff640 1 -- 172.21.15.26:0/2318607499 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) ==== 24+0+0 (secure 0 0 0) 0x7f15b40c0c50 con 0x55c939f90760 2024-09-11T23:32:35.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.607+0000 7f15c918d740 10 client.17418 fetch_fsmap learned FSMap version 70 2024-09-11T23:32:35.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.607+0000 7f15c918d740 1 -- 172.21.15.26:0/2318607499 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) -- 0x55c93a01f2b0 con 0x55c939f90760 2024-09-11T23:32:35.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.607+0000 7f159ffff640 1 -- 172.21.15.26:0/2318607499 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f15b40c5510 con 0x55c939f90760 2024-09-11T23:32:35.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.608+0000 7f159ffff640 1 -- 172.21.15.26:0/2318607499 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 70) ==== 8434+0+0 (secure 0 0 0) 0x7f15b40c5080 con 0x55c939f90760 2024-09-11T23:32:35.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.608+0000 7f15c918d740 10 client.17418 fetch_fsmap finished waiting for FSMap version 70 2024-09-11T23:32:35.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.608+0000 7f15c918d740 10 client.17418 resolve_mds: validated gid 24485 aka daemon mds.b 2024-09-11T23:32:35.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.608+0000 7f15c918d740 20 client.17418 populate_metadata read hostname 'smithi026' 2024-09-11T23:32:35.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.608+0000 7f15c918d740 1 --2- 172.21.15.26:0/2318607499 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55c93a01f930 0x55c93a021d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:35.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.608+0000 7f15c918d740 4 client.17418 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}] 2024-09-11T23:32:35.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.608+0000 7f15c918d740 1 -- 172.21.15.26:0/2318607499 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}) -- 0x55c939efdd20 con 0x55c93a01f930 2024-09-11T23:32:35.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.608+0000 7f15bffff640 1 --2- 172.21.15.26:0/2318607499 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55c93a01f930 0x55c93a021d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:35.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.608+0000 7f15bffff640 1 --2- 172.21.15.26:0/2318607499 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55c93a01f930 0x55c93a021d50 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d342a16c29f6643 server_cookie=726960e0ff0ae1a4 in_seq=0 out_seq=0 2024-09-11T23:32:35.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:35.608+0000 7f159ffff640 10 client.17418 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:32:36.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:32:36 smithi136 ceph-mon[35121]: pgmap v445: 97 pgs: 97 active+clean; 2.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.4 MiB/s wr, 359 op/s 2024-09-11T23:32:36.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:32:36 smithi026 ceph-mon[31172]: pgmap v445: 97 pgs: 97 active+clean; 2.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.4 MiB/s wr, 359 op/s 2024-09-11T23:32:36.552 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:32:36 smithi076 ceph-mon[35296]: pgmap v445: 97 pgs: 97 active+clean; 2.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.4 MiB/s wr, 359 op/s 2024-09-11T23:32:38.490 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:32:38 smithi136 ceph-mon[35121]: pgmap v446: 97 pgs: 97 active+clean; 2.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.1 MiB/s wr, 347 op/s 2024-09-11T23:32:38.843 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:32:38 smithi076 ceph-mon[35296]: pgmap v446: 97 pgs: 97 active+clean; 2.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.1 MiB/s wr, 347 op/s 2024-09-11T23:32:38.844 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:32:38 smithi026 ceph-mon[31172]: pgmap v446: 97 pgs: 97 active+clean; 2.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.1 MiB/s wr, 347 op/s 2024-09-11T23:32:40.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:32:40 smithi136 ceph-mon[35121]: pgmap v447: 97 pgs: 97 active+clean; 2.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.8 MiB/s wr, 308 op/s 2024-09-11T23:32:40.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:32:40 smithi026 ceph-mon[31172]: pgmap v447: 97 pgs: 97 active+clean; 2.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.8 MiB/s wr, 308 op/s 2024-09-11T23:32:40.552 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:32:40 smithi076 ceph-mon[35296]: pgmap v447: 97 pgs: 97 active+clean; 2.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.8 MiB/s wr, 308 op/s 2024-09-11T23:32:42.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:32:42 smithi136 ceph-mon[35121]: pgmap v448: 97 pgs: 97 active+clean; 2.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.0 MiB/s wr, 378 op/s 2024-09-11T23:32:42.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:32:42 smithi026 ceph-mon[31172]: pgmap v448: 97 pgs: 97 active+clean; 2.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.0 MiB/s wr, 378 op/s 2024-09-11T23:32:42.552 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:32:42 smithi076 ceph-mon[35296]: pgmap v448: 97 pgs: 97 active+clean; 2.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.0 MiB/s wr, 378 op/s 2024-09-11T23:32:43.665 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.665+0000 7fe713778640 1 -- 172.21.15.26:0/1790931842 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe70c1035e0 msgr2=0x7fe70c10bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:32:43.665 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.665+0000 7fe713778640 1 --2- 172.21.15.26:0/1790931842 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe70c1035e0 0x7fe70c10bac0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fe6fc009900 tx=0x7fe6fc02a480 comp rx=0 tx=0).stop 2024-09-11T23:32:43.665 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.665+0000 7fe713778640 1 -- 172.21.15.26:0/1790931842 shutdown_connections 2024-09-11T23:32:43.666 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.665+0000 7fe713778640 1 --2- 172.21.15.26:0/1790931842 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe70c10c110 0x7fe70c10e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:43.666 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.665+0000 7fe713778640 1 --2- 172.21.15.26:0/1790931842 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe70c1035e0 0x7fe70c10bac0 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:43.666 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.665+0000 7fe713778640 1 --2- 172.21.15.26:0/1790931842 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe70c102cc0 0x7fe70c1030a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:43.666 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.665+0000 7fe713778640 1 -- 172.21.15.26:0/1790931842 >> 172.21.15.26:0/1790931842 conn(0x7fe70c0fc6b0 msgr2=0x7fe70c0fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:32:43.666 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.665+0000 7fe713778640 1 -- 172.21.15.26:0/1790931842 shutdown_connections 2024-09-11T23:32:43.666 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.665+0000 7fe713778640 1 -- 172.21.15.26:0/1790931842 wait complete. 2024-09-11T23:32:43.666 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.666+0000 7fe713778640 1 Processor -- start 2024-09-11T23:32:43.666 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.666+0000 7fe713778640 1 -- start start 2024-09-11T23:32:43.668 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.666+0000 7fe713778640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe70c102cc0 0x7fe70c198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:43.668 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.666+0000 7fe713778640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe70c1035e0 0x7fe70c1988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:43.668 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.666+0000 7fe713778640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe70c10c110 0x7fe70c19cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:43.668 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.666+0000 7fe713778640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fe70c111ef0 con 0x7fe70c10c110 2024-09-11T23:32:43.668 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.666+0000 7fe713778640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fe70c111d70 con 0x7fe70c102cc0 2024-09-11T23:32:43.668 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.666+0000 7fe713778640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe70c112070 con 0x7fe70c1035e0 2024-09-11T23:32:43.669 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.666+0000 7fe711d15640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe70c10c110 0x7fe70c19cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:43.669 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.666+0000 7fe711d15640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe70c10c110 0x7fe70c19cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38688/0 (socket says 172.21.15.26:38688) 2024-09-11T23:32:43.669 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.666+0000 7fe711d15640 1 -- 172.21.15.26:0/2608155583 learned_addr learned my addr 172.21.15.26:0/2608155583 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:32:43.669 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.666+0000 7fe711514640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe70c102cc0 0x7fe70c198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:43.669 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.666+0000 7fe710d13640 1 --2- 172.21.15.26:0/2608155583 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe70c1035e0 0x7fe70c1988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:43.669 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.667+0000 7fe711d15640 1 -- 172.21.15.26:0/2608155583 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe70c1035e0 msgr2=0x7fe70c1988a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:32:43.669 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.667+0000 7fe711d15640 1 --2- 172.21.15.26:0/2608155583 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe70c1035e0 0x7fe70c1988a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:43.669 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.667+0000 7fe711d15640 1 -- 172.21.15.26:0/2608155583 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe70c102cc0 msgr2=0x7fe70c198360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:32:43.669 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.667+0000 7fe711d15640 1 --2- 172.21.15.26:0/2608155583 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe70c102cc0 0x7fe70c198360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:43.669 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.667+0000 7fe711d15640 1 -- 172.21.15.26:0/2608155583 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe70c19d350 con 0x7fe70c10c110 2024-09-11T23:32:43.669 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.667+0000 7fe711d15640 1 --2- 172.21.15.26:0/2608155583 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe70c10c110 0x7fe70c19cc20 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7fe70400eb10 tx=0x7fe70400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:32:43.669 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.667+0000 7fe7027fc640 1 -- 172.21.15.26:0/2608155583 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe704004b20 con 0x7fe70c10c110 2024-09-11T23:32:43.669 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.667+0000 7fe7027fc640 1 -- 172.21.15.26:0/2608155583 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe704004cc0 con 0x7fe70c10c110 2024-09-11T23:32:43.670 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.667+0000 7fe713778640 1 -- 172.21.15.26:0/2608155583 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe70c19d640 con 0x7fe70c10c110 2024-09-11T23:32:43.670 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.667+0000 7fe7027fc640 1 -- 172.21.15.26:0/2608155583 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe704010760 con 0x7fe70c10c110 2024-09-11T23:32:43.670 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.667+0000 7fe713778640 1 -- 172.21.15.26:0/2608155583 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe70c1a4ed0 con 0x7fe70c10c110 2024-09-11T23:32:43.678 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.670+0000 7fe7027fc640 1 -- 172.21.15.26:0/2608155583 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fe704005020 con 0x7fe70c10c110 2024-09-11T23:32:43.678 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.670+0000 7fe7165f1740 1 Processor -- start 2024-09-11T23:32:43.678 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.670+0000 7fe7165f1740 1 -- start start 2024-09-11T23:32:43.678 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.670+0000 7fe7165f1740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560d95902af0 0x560d95902ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:43.678 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.670+0000 7fe7165f1740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560d95903410 0x560d9590d770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:43.678 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.670+0000 7fe7165f1740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560d9590dcb0 0x560d959100a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:43.679 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.670+0000 7fe7165f1740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x560d958672f0 con 0x560d9590dcb0 2024-09-11T23:32:43.679 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.670+0000 7fe7165f1740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x560d9580f270 con 0x560d95903410 2024-09-11T23:32:43.679 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.670+0000 7fe7165f1740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x560d958632b0 con 0x560d95902af0 2024-09-11T23:32:43.679 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.671+0000 7fe711514640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560d95903410 0x560d9590d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:43.679 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.671+0000 7fe710d13640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560d95902af0 0x560d95902ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:43.679 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.671+0000 7fe711514640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560d95903410 0x560d9590d770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:55782/0 (socket says 172.21.15.26:55782) 2024-09-11T23:32:43.679 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.671+0000 7fe711514640 1 -- 172.21.15.26:0/3385036597 learned_addr learned my addr 172.21.15.26:0/3385036597 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:32:43.679 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.671+0000 7fe7027fc640 1 --2- 172.21.15.26:0/2608155583 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fe6f80a2da0 0x7fe6f80a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:43.680 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.671+0000 7fe710d13640 1 --2- 172.21.15.26:0/3385036597 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560d9590dcb0 0x560d959100a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:43.680 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.671+0000 7fe711d15640 1 --2- 172.21.15.26:0/2608155583 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fe6f80a2da0 0x7fe6f80a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:43.680 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.671+0000 7fe711514640 1 -- 172.21.15.26:0/3385036597 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560d95902af0 msgr2=0x560d95902ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:32:43.680 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.671+0000 7fe711514640 1 --2- 172.21.15.26:0/3385036597 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560d95902af0 0x560d95902ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:43.680 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.671+0000 7fe711514640 1 -- 172.21.15.26:0/3385036597 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560d9590dcb0 msgr2=0x560d959100a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:32:43.680 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.671+0000 7fe711514640 1 --2- 172.21.15.26:0/3385036597 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560d9590dcb0 0x560d959100a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:43.680 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.671+0000 7fe711514640 1 -- 172.21.15.26:0/3385036597 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d9580e340 con 0x560d95903410 2024-09-11T23:32:43.680 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.671+0000 7fe710d13640 1 --2- 172.21.15.26:0/3385036597 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560d9590dcb0 0x560d959100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:32:43.681 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.671+0000 7fe7027fc640 1 -- 172.21.15.26:0/2608155583 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8542+0+0 (secure 0 0 0) 0x7fe7040cf950 con 0x7fe70c10c110 2024-09-11T23:32:43.681 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.671+0000 7fe711514640 1 --2- 172.21.15.26:0/3385036597 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560d95903410 0x560d9590d770 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fe70800bd10 tx=0x7fe7080107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a3bd3c392a87489f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:32:43.681 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.671+0000 7fe711d15640 1 --2- 172.21.15.26:0/2608155583 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fe6f80a2da0 0x7fe6f80a5260 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fe70409bb60 tx=0x7fe7040bc040 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:32:43.681 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.671+0000 7fe6f77fe640 1 -- 172.21.15.26:0/3385036597 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe708004030 con 0x560d95903410 2024-09-11T23:32:43.681 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.671+0000 7fe6f77fe640 1 -- 172.21.15.26:0/3385036597 <== mon.2 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe7080041d0 con 0x560d95903410 2024-09-11T23:32:43.681 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.671+0000 7fe6f77fe640 1 -- 172.21.15.26:0/3385036597 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7080117e0 con 0x560d95903410 2024-09-11T23:32:43.681 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.672+0000 7fe7165f1740 1 -- 172.21.15.26:0/3385036597 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560d95903410 msgr2=0x560d9590d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:32:43.681 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.672+0000 7fe7165f1740 1 --2- 172.21.15.26:0/3385036597 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560d95903410 0x560d9590d770 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fe70800bd10 tx=0x7fe7080107a0 comp rx=0 tx=0).stop 2024-09-11T23:32:43.682 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.672+0000 7fe7165f1740 1 -- 172.21.15.26:0/3385036597 shutdown_connections 2024-09-11T23:32:43.682 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.672+0000 7fe7165f1740 1 --2- 172.21.15.26:0/3385036597 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560d95903410 0x560d9590d770 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:43.682 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.672+0000 7fe7165f1740 1 --2- 172.21.15.26:0/3385036597 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560d9590dcb0 0x560d959100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:43.682 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.672+0000 7fe7165f1740 1 --2- 172.21.15.26:0/3385036597 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560d95902af0 0x560d95902ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:43.682 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.672+0000 7fe7165f1740 1 -- 172.21.15.26:0/3385036597 >> 172.21.15.26:0/3385036597 conn(0x560d958ff060 msgr2=0x560d958ff440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:32:43.682 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.672+0000 7fe7165f1740 1 -- 172.21.15.26:0/3385036597 shutdown_connections 2024-09-11T23:32:43.682 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.672+0000 7fe7165f1740 1 -- 172.21.15.26:0/3385036597 wait complete. 2024-09-11T23:32:43.682 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.673+0000 7fe7165f1740 1 -- start start 2024-09-11T23:32:43.682 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.673+0000 7fe7165f1740 1 Processor -- start 2024-09-11T23:32:43.683 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.673+0000 7fe7165f1740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560d95902af0 0x560d9590ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:43.683 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.673+0000 7fe7165f1740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560d95903410 0x560d9590b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:43.683 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.673+0000 7fe7165f1740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560d9590dcb0 0x560d9590b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:43.683 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.673+0000 7fe7165f1740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x560d95910880 con 0x560d95903410 2024-09-11T23:32:43.683 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.673+0000 7fe7165f1740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x560d957f0290 con 0x560d9590dcb0 2024-09-11T23:32:43.683 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.673+0000 7fe7165f1740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x560d95910a00 con 0x560d95902af0 2024-09-11T23:32:43.683 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.673+0000 7fe7165f1740 10 client.0 _ll_register_callbacks cb 0x560d95821140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:32:43.683 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.673+0000 7fe711514640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560d95903410 0x560d9590b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:43.684 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.673+0000 7fe711514640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560d95903410 0x560d9590b0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38710/0 (socket says 172.21.15.26:38710) 2024-09-11T23:32:43.684 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.673+0000 7fe711514640 1 -- 172.21.15.26:0/2639099762 learned_addr learned my addr 172.21.15.26:0/2639099762 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:32:43.684 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.673+0000 7fe710d13640 1 --2- 172.21.15.26:0/2639099762 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560d9590dcb0 0x560d9590b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:43.684 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.674+0000 7fe710d13640 1 --2- 172.21.15.26:0/2639099762 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560d95902af0 0x560d9590ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:43.684 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.674+0000 7fe711514640 1 -- 172.21.15.26:0/2639099762 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560d95902af0 msgr2=0x560d9590ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:32:43.684 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.674+0000 7fe711514640 1 --2- 172.21.15.26:0/2639099762 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560d95902af0 0x560d9590ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:43.684 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.674+0000 7fe711514640 1 -- 172.21.15.26:0/2639099762 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560d9590dcb0 msgr2=0x560d9590b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:32:43.685 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.674+0000 7fe711514640 1 --2- 172.21.15.26:0/2639099762 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560d9590dcb0 0x560d9590b5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:43.685 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.674+0000 7fe711514640 1 -- 172.21.15.26:0/2639099762 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d9557a210 con 0x560d95903410 2024-09-11T23:32:43.685 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.674+0000 7fe710d13640 1 --2- 172.21.15.26:0/2639099762 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560d95902af0 0x560d9590ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:32:43.685 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.674+0000 7fe711514640 1 --2- 172.21.15.26:0/2639099762 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560d95903410 0x560d9590b0a0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7fe708010ca0 tx=0x7fe70800b950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c990b54b50a34f8 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:32:43.685 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.674+0000 7fe6f4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:32:43.685 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.674+0000 7fe6f4ff9640 1 -- 172.21.15.26:0/2639099762 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe708029060 con 0x560d95903410 2024-09-11T23:32:43.685 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.674+0000 7fe6f4ff9640 1 -- 172.21.15.26:0/2639099762 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8542+0+0 (secure 0 0 0) 0x7fe7080035f0 con 0x560d95903410 2024-09-11T23:32:43.685 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.674+0000 7fe7165f1740 1 -- 172.21.15.26:0/2639099762 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe7080041d0 con 0x560d95903410 2024-09-11T23:32:43.686 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.674+0000 7fe6f4ff9640 1 -- 172.21.15.26:0/2639099762 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) ==== 24+0+0 (secure 0 0 0) 0x7fe708004010 con 0x560d95903410 2024-09-11T23:32:43.686 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.674+0000 7fe7165f1740 10 client.17430 fetch_fsmap learned FSMap version 71 2024-09-11T23:32:43.686 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.674+0000 7fe7165f1740 1 -- 172.21.15.26:0/2639099762 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=4+}) -- 0x560d95646620 con 0x560d95903410 2024-09-11T23:32:43.686 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.675+0000 7fe6f4ff9640 1 -- 172.21.15.26:0/2639099762 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe7080041d0 con 0x560d95903410 2024-09-11T23:32:43.686 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.675+0000 7fe6f4ff9640 1 -- 172.21.15.26:0/2639099762 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 71) ==== 8430+0+0 (secure 0 0 0) 0x7fe708004880 con 0x560d95903410 2024-09-11T23:32:43.686 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.675+0000 7fe7165f1740 10 client.17430 fetch_fsmap finished waiting for FSMap version 71 2024-09-11T23:32:43.686 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.675+0000 7fe7165f1740 10 client.17430 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:32:43.686 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.675+0000 7fe7165f1740 20 client.17430 populate_metadata read hostname 'smithi026' 2024-09-11T23:32:43.687 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.675+0000 7fe7165f1740 1 --2- 172.21.15.26:0/2639099762 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x560d9591db60 0x560d9591ff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:43.687 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.675+0000 7fe7165f1740 4 client.17430 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:32:43.687 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.675+0000 7fe7165f1740 1 -- 172.21.15.26:0/2639099762 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560d95874a20 con 0x560d9591db60 2024-09-11T23:32:43.687 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.675+0000 7fe710d13640 1 --2- 172.21.15.26:0/2639099762 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x560d9591db60 0x560d9591ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:43.687 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.675+0000 7fe6f4ff9640 10 client.17430 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:32:43.687 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.675+0000 7fe710d13640 1 --2- 172.21.15.26:0/2639099762 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x560d9591db60 0x560d9591ff50 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4deedebfc30d0de server_cookie=b17fb0a5a6508399 in_seq=0 out_seq=0 2024-09-11T23:32:43.687 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.676+0000 7fe6f4ff9640 1 -- 172.21.15.26:0/2639099762 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560d95874a20 con 0x560d9591db60 2024-09-11T23:32:43.687 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.676+0000 7fe6f4ff9640 10 client.17430 handle_command_reply: tid=0 2024-09-11T23:32:43.688 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.676+0000 7fe7165f1740 1 client.17430 shutdown 2024-09-11T23:32:43.688 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.676+0000 7fe7165f1740 1 -- 172.21.15.26:0/2639099762 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560d95903410 msgr2=0x560d9590b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:32:43.688 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.676+0000 7fe7165f1740 1 --2- 172.21.15.26:0/2639099762 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560d95903410 0x560d9590b0a0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7fe708010ca0 tx=0x7fe70800b950 comp rx=0 tx=0).stop 2024-09-11T23:32:43.688 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.676+0000 7fe7165f1740 1 -- 172.21.15.26:0/2639099762 shutdown_connections 2024-09-11T23:32:43.688 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.676+0000 7fe7165f1740 1 --2- 172.21.15.26:0/2639099762 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560d9590dcb0 0x560d9590b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:43.688 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.676+0000 7fe7165f1740 1 --2- 172.21.15.26:0/2639099762 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x560d9591db60 0x560d9591ff50 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:43.688 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.676+0000 7fe7165f1740 1 --2- 172.21.15.26:0/2639099762 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560d95903410 0x560d9590b0a0 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:43.688 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.676+0000 7fe7165f1740 1 --2- 172.21.15.26:0/2639099762 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560d95902af0 0x560d9590ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:43.689 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.676+0000 7fe6f4ff9640 0 client.17430 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:32:43.689 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.676+0000 7fe7165f1740 1 -- 172.21.15.26:0/2639099762 >> 172.21.15.26:0/2639099762 conn(0x560d958ff060 msgr2=0x560d958fe1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:32:43.689 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.676+0000 7fe7165f1740 1 -- 172.21.15.26:0/2639099762 shutdown_connections 2024-09-11T23:32:43.689 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.676+0000 7fe7165f1740 1 -- 172.21.15.26:0/2639099762 wait complete. 2024-09-11T23:32:43.689 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.677+0000 7fe7165f1740 20 client.17430 trim_cache size 0 max 16384 2024-09-11T23:32:43.695 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.691+0000 7fe7165f1740 1 Processor -- start 2024-09-11T23:32:43.695 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.691+0000 7fe7165f1740 1 -- start start 2024-09-11T23:32:43.695 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.691+0000 7fe7165f1740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560d95902af0 0x560d9590b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:43.695 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.692+0000 7fe7165f1740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560d95903410 0x560d95916070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:43.695 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.692+0000 7fe7165f1740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560d9590dcb0 0x560d959165b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:43.695 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.692+0000 7fe7165f1740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fe70c1ad8f0 con 0x560d9590dcb0 2024-09-11T23:32:43.695 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.692+0000 7fe7165f1740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fe70c1af6a0 con 0x560d95903410 2024-09-11T23:32:43.695 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.692+0000 7fe7165f1740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe70c1adff0 con 0x560d95902af0 2024-09-11T23:32:43.695 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.692+0000 7fe710d13640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560d9590dcb0 0x560d959165b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:43.695 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.692+0000 7fe710d13640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560d9590dcb0 0x560d959165b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38720/0 (socket says 172.21.15.26:38720) 2024-09-11T23:32:43.696 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.692+0000 7fe710d13640 1 -- 172.21.15.26:0/3086392659 learned_addr learned my addr 172.21.15.26:0/3086392659 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:32:43.696 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.692+0000 7fe711514640 1 --2- 172.21.15.26:0/3086392659 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560d95903410 0x560d95916070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:43.696 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.692+0000 7fe710d13640 1 --2- 172.21.15.26:0/3086392659 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560d95902af0 0x560d9590b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:43.696 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.692+0000 7fe710d13640 1 -- 172.21.15.26:0/3086392659 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560d95902af0 msgr2=0x560d9590b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:32:43.696 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.692+0000 7fe710d13640 1 --2- 172.21.15.26:0/3086392659 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560d95902af0 0x560d9590b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:43.696 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.692+0000 7fe710d13640 1 -- 172.21.15.26:0/3086392659 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560d95903410 msgr2=0x560d95916070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:32:43.696 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.692+0000 7fe710d13640 1 --2- 172.21.15.26:0/3086392659 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560d95903410 0x560d95916070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:43.696 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.692+0000 7fe710d13640 1 -- 172.21.15.26:0/3086392659 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d95903240 con 0x560d9590dcb0 2024-09-11T23:32:43.696 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.692+0000 7fe710d13640 1 --2- 172.21.15.26:0/3086392659 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560d9590dcb0 0x560d959165b0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7fe6fc009900 tx=0x7fe6fc02a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=769b6607ce235838 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:32:43.696 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.692+0000 7fe6f6ffd640 1 -- 172.21.15.26:0/3086392659 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6fc035040 con 0x560d9590dcb0 2024-09-11T23:32:43.696 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.692+0000 7fe6f6ffd640 1 -- 172.21.15.26:0/3086392659 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe6fc004480 con 0x560d9590dcb0 2024-09-11T23:32:43.696 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.692+0000 7fe6f6ffd640 1 -- 172.21.15.26:0/3086392659 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6fc005680 con 0x560d9590dcb0 2024-09-11T23:32:43.697 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.693+0000 7fe7165f1740 1 -- 172.21.15.26:0/3086392659 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560d9590dcb0 msgr2=0x560d959165b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:32:43.697 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.693+0000 7fe7165f1740 1 --2- 172.21.15.26:0/3086392659 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560d9590dcb0 0x560d959165b0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7fe6fc009900 tx=0x7fe6fc02a480 comp rx=0 tx=0).stop 2024-09-11T23:32:43.697 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.693+0000 7fe7165f1740 1 -- 172.21.15.26:0/3086392659 shutdown_connections 2024-09-11T23:32:43.697 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.693+0000 7fe7165f1740 1 --2- 172.21.15.26:0/3086392659 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560d95903410 0x560d95916070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:43.697 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.693+0000 7fe7165f1740 1 --2- 172.21.15.26:0/3086392659 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560d9590dcb0 0x560d959165b0 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:43.697 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.693+0000 7fe7165f1740 1 --2- 172.21.15.26:0/3086392659 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560d95902af0 0x560d9590b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:43.697 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.693+0000 7fe7165f1740 1 -- 172.21.15.26:0/3086392659 >> 172.21.15.26:0/3086392659 conn(0x560d958ff060 msgr2=0x560d9590cbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:32:43.697 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.693+0000 7fe7165f1740 1 -- 172.21.15.26:0/3086392659 shutdown_connections 2024-09-11T23:32:43.697 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.693+0000 7fe7165f1740 1 -- 172.21.15.26:0/3086392659 wait complete. 2024-09-11T23:32:43.697 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.693+0000 7fe7165f1740 1 -- start start 2024-09-11T23:32:43.697 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.693+0000 7fe7165f1740 1 Processor -- start 2024-09-11T23:32:43.697 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.694+0000 7fe7165f1740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560d95902af0 0x560d95985040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:43.698 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.694+0000 7fe7165f1740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560d95903410 0x560d95985580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:43.698 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.694+0000 7fe7165f1740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560d9590dcb0 0x560d95925370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:43.698 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.694+0000 7fe7165f1740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fe70c1aef10 con 0x560d95903410 2024-09-11T23:32:43.698 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.694+0000 7fe7165f1740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fe70c1ae780 con 0x560d95902af0 2024-09-11T23:32:43.698 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.694+0000 7fe7165f1740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe70c1ab8f0 con 0x560d9590dcb0 2024-09-11T23:32:43.698 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.694+0000 7fe7165f1740 10 client.0 _ll_register_callbacks cb 0x7fe7040093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:32:43.698 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.694+0000 7fe711514640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560d95903410 0x560d95985580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:43.698 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.694+0000 7fe711514640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560d95903410 0x560d95985580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38728/0 (socket says 172.21.15.26:38728) 2024-09-11T23:32:43.698 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.694+0000 7fe711514640 1 -- 172.21.15.26:0/2603655936 learned_addr learned my addr 172.21.15.26:0/2603655936 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:32:43.698 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.694+0000 7fe710d13640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560d95902af0 0x560d95985040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:43.698 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.694+0000 7fe711514640 1 -- 172.21.15.26:0/2603655936 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560d9590dcb0 msgr2=0x560d95925370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:32:43.698 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.694+0000 7fe711514640 1 --2- 172.21.15.26:0/2603655936 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560d9590dcb0 0x560d95925370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:43.698 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.694+0000 7fe711514640 1 -- 172.21.15.26:0/2603655936 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560d95902af0 msgr2=0x560d95985040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:32:43.699 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.694+0000 7fe711514640 1 --2- 172.21.15.26:0/2603655936 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560d95902af0 0x560d95985040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:43.699 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.694+0000 7fe711514640 1 -- 172.21.15.26:0/2603655936 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d95925ba0 con 0x560d95903410 2024-09-11T23:32:43.699 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.694+0000 7fe711514640 1 --2- 172.21.15.26:0/2603655936 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560d95903410 0x560d95985580 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x560d9590c1a0 tx=0x7fe70800b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=509ce0cb146234f7 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:32:43.699 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.694+0000 7fe6f4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:32:43.699 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.695+0000 7fe6f4ff9640 1 -- 172.21.15.26:0/2603655936 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe70803a060 con 0x560d95903410 2024-09-11T23:32:43.699 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.695+0000 7fe6f4ff9640 1 -- 172.21.15.26:0/2603655936 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8542+0+0 (secure 0 0 0) 0x7fe708004590 con 0x560d95903410 2024-09-11T23:32:43.699 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.695+0000 7fe7165f1740 1 -- 172.21.15.26:0/2603655936 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe6fc004480 con 0x560d95903410 2024-09-11T23:32:43.699 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.695+0000 7fe6f4ff9640 1 -- 172.21.15.26:0/2603655936 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) ==== 24+0+0 (secure 0 0 0) 0x7fe708029070 con 0x560d95903410 2024-09-11T23:32:43.699 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.695+0000 7fe7165f1740 10 client.17436 fetch_fsmap learned FSMap version 71 2024-09-11T23:32:43.699 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.695+0000 7fe7165f1740 1 -- 172.21.15.26:0/2603655936 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=4+}) -- 0x560d9598c870 con 0x560d95903410 2024-09-11T23:32:43.699 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.695+0000 7fe6f4ff9640 1 -- 172.21.15.26:0/2603655936 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe6fc004480 con 0x560d95903410 2024-09-11T23:32:43.700 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.695+0000 7fe6f4ff9640 1 -- 172.21.15.26:0/2603655936 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 71) ==== 8430+0+0 (secure 0 0 0) 0x7fe708004790 con 0x560d95903410 2024-09-11T23:32:43.700 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.695+0000 7fe7165f1740 10 client.17436 fetch_fsmap finished waiting for FSMap version 71 2024-09-11T23:32:43.701 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.695+0000 7fe7165f1740 10 client.17436 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:32:43.701 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.695+0000 7fe7165f1740 20 client.17436 populate_metadata read hostname 'smithi026' 2024-09-11T23:32:43.701 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.695+0000 7fe7165f1740 1 --2- 172.21.15.26:0/2603655936 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x560d9598cff0 0x560d9598f410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:43.701 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.695+0000 7fe7165f1740 4 client.17436 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T23:32:43.701 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.695+0000 7fe7165f1740 1 -- 172.21.15.26:0/2603655936 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560d9586ed70 con 0x560d9598cff0 2024-09-11T23:32:43.701 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.695+0000 7fe710d13640 1 --2- 172.21.15.26:0/2603655936 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x560d9598cff0 0x560d9598f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:43.701 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.696+0000 7fe710d13640 1 --2- 172.21.15.26:0/2603655936 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x560d9598cff0 0x560d9598f410 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a284018406f1b00c server_cookie=9f379a67fa1848ec in_seq=0 out_seq=0 2024-09-11T23:32:43.701 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:43.696+0000 7fe6f4ff9640 10 client.17436 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:32:44.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:32:44 smithi136 ceph-mon[35121]: pgmap v449: 97 pgs: 97 active+clean; 2.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 237 op/s 2024-09-11T23:32:44.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:32:44 smithi136 ceph-mon[35121]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:32:44.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:32:44 smithi026 ceph-mon[31172]: pgmap v449: 97 pgs: 97 active+clean; 2.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 237 op/s 2024-09-11T23:32:44.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:32:44 smithi026 ceph-mon[31172]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:32:44.552 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:32:44 smithi076 ceph-mon[35296]: pgmap v449: 97 pgs: 97 active+clean; 2.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 237 op/s 2024-09-11T23:32:44.552 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:32:44 smithi076 ceph-mon[35296]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:32:44.964 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:44.964+0000 7fe6f4ff9640 1 -- 172.21.15.26:0/2603655936 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560d9586ed70 con 0x560d9598cff0 2024-09-11T23:32:44.964 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:44.964+0000 7fe6f4ff9640 10 client.17436 handle_command_reply: tid=0 2024-09-11T23:32:44.964 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:44.964+0000 7fe7165f1740 1 client.17436 shutdown 2024-09-11T23:32:44.965 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:44.964+0000 7fe7165f1740 1 -- 172.21.15.26:0/2603655936 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560d95903410 msgr2=0x560d95985580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:32:44.965 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:44.964+0000 7fe7165f1740 1 --2- 172.21.15.26:0/2603655936 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560d95903410 0x560d95985580 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x560d9590c1a0 tx=0x7fe70800b390 comp rx=0 tx=0).stop 2024-09-11T23:32:44.965 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:44.964+0000 7fe7165f1740 1 -- 172.21.15.26:0/2603655936 shutdown_connections 2024-09-11T23:32:44.965 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:44.964+0000 7fe7165f1740 1 --2- 172.21.15.26:0/2603655936 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x560d9598cff0 0x560d9598f410 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:44.965 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:44.964+0000 7fe7165f1740 1 --2- 172.21.15.26:0/2603655936 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560d9590dcb0 0x560d95925370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:44.965 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:44.964+0000 7fe7165f1740 1 --2- 172.21.15.26:0/2603655936 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560d95903410 0x560d95985580 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:44.965 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:44.964+0000 7fe6f4ff9640 0 client.17436 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:32:44.965 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:44.964+0000 7fe7165f1740 1 --2- 172.21.15.26:0/2603655936 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560d95902af0 0x560d95985040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:44.965 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:44.964+0000 7fe7165f1740 1 -- 172.21.15.26:0/2603655936 >> 172.21.15.26:0/2603655936 conn(0x560d958ff060 msgr2=0x560d958fd9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:32:44.965 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:44.965+0000 7fe7165f1740 1 -- 172.21.15.26:0/2603655936 shutdown_connections 2024-09-11T23:32:44.965 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:44.965+0000 7fe7165f1740 1 -- 172.21.15.26:0/2603655936 wait complete. 2024-09-11T23:32:44.965 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:44.965+0000 7fe7165f1740 20 client.17436 trim_cache size 0 max 16384 2024-09-11T23:32:44.966 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-09-11T23:32:44.966 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "message": "", 2024-09-11T23:32:44.966 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-09-11T23:32:44.966 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-09-11T23:32:44.967 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:44.965+0000 7fe713778640 1 -- 172.21.15.26:0/2608155583 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fe6f80a2da0 msgr2=0x7fe6f80a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:32:44.967 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:44.965+0000 7fe713778640 1 --2- 172.21.15.26:0/2608155583 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fe6f80a2da0 0x7fe6f80a5260 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fe70409bb60 tx=0x7fe7040bc040 comp rx=0 tx=0).stop 2024-09-11T23:32:44.967 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:44.965+0000 7fe713778640 1 -- 172.21.15.26:0/2608155583 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe70c10c110 msgr2=0x7fe70c19cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:32:44.967 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:44.965+0000 7fe713778640 1 --2- 172.21.15.26:0/2608155583 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe70c10c110 0x7fe70c19cc20 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7fe70400eb10 tx=0x7fe70400c2d0 comp rx=0 tx=0).stop 2024-09-11T23:32:44.967 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:44.966+0000 7fe713778640 1 -- 172.21.15.26:0/2608155583 shutdown_connections 2024-09-11T23:32:44.967 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:44.966+0000 7fe713778640 1 --2- 172.21.15.26:0/2608155583 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fe6f80a2da0 0x7fe6f80a5260 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:44.967 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:44.966+0000 7fe713778640 1 --2- 172.21.15.26:0/2608155583 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe70c10c110 0x7fe70c19cc20 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:44.967 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:44.966+0000 7fe713778640 1 --2- 172.21.15.26:0/2608155583 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe70c1035e0 0x7fe70c1988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:44.967 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:44.966+0000 7fe713778640 1 --2- 172.21.15.26:0/2608155583 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe70c102cc0 0x7fe70c198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:44.967 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:44.966+0000 7fe713778640 1 -- 172.21.15.26:0/2608155583 >> 172.21.15.26:0/2608155583 conn(0x7fe70c0fc6b0 msgr2=0x7fe70c0fd310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:32:44.967 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:44.966+0000 7fe713778640 1 -- 172.21.15.26:0/2608155583 shutdown_connections 2024-09-11T23:32:44.967 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:44.966+0000 7fe713778640 1 -- 172.21.15.26:0/2608155583 wait complete. 2024-09-11T23:32:45.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:45.959+0000 7f159ffff640 1 -- 172.21.15.26:0/2318607499 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55c939efdd20 con 0x55c93a01f930 2024-09-11T23:32:45.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:45.959+0000 7f159ffff640 10 client.17418 handle_command_reply: tid=0 2024-09-11T23:32:45.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:45.959+0000 7f15c918d740 1 client.17418 shutdown 2024-09-11T23:32:45.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:45.959+0000 7f15c918d740 1 -- 172.21.15.26:0/2318607499 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c939f90760 msgr2=0x55c939fa38e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:32:45.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:45.959+0000 7f15c918d740 1 --2- 172.21.15.26:0/2318607499 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c939f90760 0x55c939fa38e0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f15b40ee0d0 tx=0x7f15b4005f90 comp rx=0 tx=0).stop 2024-09-11T23:32:45.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:45.959+0000 7f15c918d740 1 -- 172.21.15.26:0/2318607499 shutdown_connections 2024-09-11T23:32:45.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:45.959+0000 7f15c918d740 1 --2- 172.21.15.26:0/2318607499 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c939f91080 0x55c939fa3e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:45.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:45.959+0000 7f15c918d740 1 --2- 172.21.15.26:0/2318607499 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55c93a01f930 0x55c93a021d50 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:45.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:45.959+0000 7f15c918d740 1 --2- 172.21.15.26:0/2318607499 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c939f9b940 0x55c939fa08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:45.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:45.959+0000 7f15c918d740 1 --2- 172.21.15.26:0/2318607499 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c939f90760 0x55c939fa38e0 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:45.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:45.959+0000 7f159ffff640 0 client.17418 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:32:45.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:45.959+0000 7f15c918d740 1 -- 172.21.15.26:0/2318607499 >> 172.21.15.26:0/2318607499 conn(0x55c939f8ccf0 msgr2=0x55c939f99040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:32:45.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:45.960+0000 7f15c918d740 1 -- 172.21.15.26:0/2318607499 shutdown_connections 2024-09-11T23:32:45.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:45.960+0000 7f15c918d740 1 -- 172.21.15.26:0/2318607499 wait complete. 2024-09-11T23:32:45.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:45.960+0000 7f15c918d740 20 client.17418 trim_cache size 0 max 16384 2024-09-11T23:32:45.961 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-11T23:32:45.961 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 71, 2024-09-11T23:32:45.961 INFO:teuthology.orchestra.run.smithi026.stdout: "leader": 24485, 2024-09-11T23:32:45.961 INFO:teuthology.orchestra.run.smithi026.stdout: "set_version": 50, 2024-09-11T23:32:45.961 INFO:teuthology.orchestra.run.smithi026.stdout: "sets": { 2024-09-11T23:32:45.961 INFO:teuthology.orchestra.run.smithi026.stdout: "95c56ce2": { 2024-09-11T23:32:45.961 INFO:teuthology.orchestra.run.smithi026.stdout: "version": 50, 2024-09-11T23:32:45.961 INFO:teuthology.orchestra.run.smithi026.stdout: "age_ref": 0.0, 2024-09-11T23:32:45.961 INFO:teuthology.orchestra.run.smithi026.stdout: "state": { 2024-09-11T23:32:45.962 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "QUIESCED", 2024-09-11T23:32:45.962 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.0 2024-09-11T23:32:45.962 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:32:45.962 INFO:teuthology.orchestra.run.smithi026.stdout: "timeout": 90.0, 2024-09-11T23:32:45.962 INFO:teuthology.orchestra.run.smithi026.stdout: "expiration": 150.0, 2024-09-11T23:32:45.962 INFO:teuthology.orchestra.run.smithi026.stdout: "members": { 2024-09-11T23:32:45.962 INFO:teuthology.orchestra.run.smithi026.stdout: "file:/": { 2024-09-11T23:32:45.962 INFO:teuthology.orchestra.run.smithi026.stdout: "excluded": false, 2024-09-11T23:32:45.962 INFO:teuthology.orchestra.run.smithi026.stdout: "state": { 2024-09-11T23:32:45.962 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "QUIESCED", 2024-09-11T23:32:45.962 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.0 2024-09-11T23:32:45.962 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:32:45.962 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:32:45.963 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:32:45.963 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:32:45.963 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:32:45.963 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-11T23:32:45.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:45.961+0000 7f15c6db4640 1 -- 172.21.15.26:0/3308627191 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f15b00a2cf0 msgr2=0x7f15b00a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:32:45.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:45.961+0000 7f15c6db4640 1 --2- 172.21.15.26:0/3308627191 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f15b00a2cf0 0x7f15b00a51b0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f15b801d000 tx=0x7f15b801f000 comp rx=0 tx=0).stop 2024-09-11T23:32:45.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:45.961+0000 7f15c6db4640 1 -- 172.21.15.26:0/3308627191 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15c0100e20 msgr2=0x7f15c00fcf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:32:45.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:45.961+0000 7f15c6db4640 1 --2- 172.21.15.26:0/3308627191 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15c0100e20 0x7f15c00fcf00 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f15b4005940 tx=0x7f15b4030650 comp rx=0 tx=0).stop 2024-09-11T23:32:45.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:45.961+0000 7f15c6db4640 1 -- 172.21.15.26:0/3308627191 shutdown_connections 2024-09-11T23:32:45.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:45.961+0000 7f15c6db4640 1 --2- 172.21.15.26:0/3308627191 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f15b00a2cf0 0x7f15b00a51b0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:45.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:45.961+0000 7f15c6db4640 1 --2- 172.21.15.26:0/3308627191 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f15c0102570 0x7f15c00fd440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:45.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:45.961+0000 7f15c6db4640 1 --2- 172.21.15.26:0/3308627191 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f15c0102ef0 0x7f15c00fd980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:45.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:45.961+0000 7f15c6db4640 1 --2- 172.21.15.26:0/3308627191 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15c0100e20 0x7f15c00fcf00 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:45.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:45.961+0000 7f15c6db4640 1 -- 172.21.15.26:0/3308627191 >> 172.21.15.26:0/3308627191 conn(0x7f15c00fc7d0 msgr2=0x7f15c006a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:32:45.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:45.961+0000 7f15c6db4640 1 -- 172.21.15.26:0/3308627191 shutdown_connections 2024-09-11T23:32:45.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:32:45.961+0000 7f15c6db4640 1 -- 172.21.15.26:0/3308627191 wait complete. 2024-09-11T23:32:46.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:32:46 smithi026 ceph-mon[31172]: pgmap v450: 97 pgs: 97 active+clean; 2.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.8 MiB/s wr, 230 op/s 2024-09-11T23:32:46.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:32:46 smithi136 ceph-mon[35121]: pgmap v450: 97 pgs: 97 active+clean; 2.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.8 MiB/s wr, 230 op/s 2024-09-11T23:32:46.552 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:32:46 smithi076 ceph-mon[35296]: pgmap v450: 97 pgs: 97 active+clean; 2.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.8 MiB/s wr, 230 op/s 2024-09-11T23:32:46.942 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '95c56ce2', quiesce took 14.5 seconds. Will release after: 15.5 2024-09-11T23:32:48.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:32:48 smithi136 ceph-mon[35121]: pgmap v451: 97 pgs: 97 active+clean; 2.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 972 KiB/s rd, 970 KiB/s wr, 158 op/s 2024-09-11T23:32:48.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:32:48 smithi026 ceph-mon[31172]: pgmap v451: 97 pgs: 97 active+clean; 2.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 972 KiB/s rd, 970 KiB/s wr, 158 op/s 2024-09-11T23:32:48.552 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:32:48 smithi076 ceph-mon[35296]: pgmap v451: 97 pgs: 97 active+clean; 2.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 972 KiB/s rd, 970 KiB/s wr, 158 op/s 2024-09-11T23:32:50.357 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:32:50 smithi026 ceph-mon[31172]: pgmap v452: 97 pgs: 97 active+clean; 2.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 285 KiB/s wr, 87 op/s 2024-09-11T23:32:50.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:32:50 smithi136 ceph-mon[35121]: pgmap v452: 97 pgs: 97 active+clean; 2.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 285 KiB/s wr, 87 op/s 2024-09-11T23:32:50.552 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:32:50 smithi076 ceph-mon[35296]: pgmap v452: 97 pgs: 97 active+clean; 2.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 285 KiB/s wr, 87 op/s 2024-09-11T23:32:52.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:32:52 smithi136 ceph-mon[35121]: pgmap v453: 97 pgs: 97 active+clean; 2.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 306 KiB/s rd, 304 KiB/s wr, 103 op/s 2024-09-11T23:32:52.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:32:52 smithi026 ceph-mon[31172]: pgmap v453: 97 pgs: 97 active+clean; 2.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 306 KiB/s rd, 304 KiB/s wr, 103 op/s 2024-09-11T23:32:52.552 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:32:52 smithi076 ceph-mon[35296]: pgmap v453: 97 pgs: 97 active+clean; 2.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 306 KiB/s rd, 304 KiB/s wr, 103 op/s 2024-09-11T23:32:54.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:32:54 smithi136 ceph-mon[35121]: pgmap v454: 97 pgs: 97 active+clean; 2.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 88 KiB/s wr, 34 op/s 2024-09-11T23:32:54.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:32:54 smithi026 ceph-mon[31172]: pgmap v454: 97 pgs: 97 active+clean; 2.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 88 KiB/s wr, 34 op/s 2024-09-11T23:32:54.552 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:32:54 smithi076 ceph-mon[35296]: pgmap v454: 97 pgs: 97 active+clean; 2.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 88 KiB/s wr, 34 op/s 2024-09-11T23:32:56.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:32:56 smithi136 ceph-mon[35121]: pgmap v455: 97 pgs: 97 active+clean; 2.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 91 KiB/s wr, 34 op/s 2024-09-11T23:32:56.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:32:56 smithi026 ceph-mon[31172]: pgmap v455: 97 pgs: 97 active+clean; 2.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 91 KiB/s wr, 34 op/s 2024-09-11T23:32:56.552 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:32:56 smithi076 ceph-mon[35296]: pgmap v455: 97 pgs: 97 active+clean; 2.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 91 KiB/s wr, 34 op/s 2024-09-11T23:32:58.070 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.071+0000 7efd56e3c640 1 -- 172.21.15.26:0/3274172421 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd5010c110 msgr2=0x7efd5010e590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:32:58.071 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.071+0000 7efd56e3c640 1 --2- 172.21.15.26:0/3274172421 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd5010c110 0x7efd5010e590 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7efd48009ca0 tx=0x7efd4802a950 comp rx=0 tx=0).stop 2024-09-11T23:32:58.071 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.071+0000 7efd56e3c640 1 -- 172.21.15.26:0/3274172421 shutdown_connections 2024-09-11T23:32:58.071 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.071+0000 7efd56e3c640 1 --2- 172.21.15.26:0/3274172421 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd5010c110 0x7efd5010e590 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:58.071 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.071+0000 7efd56e3c640 1 --2- 172.21.15.26:0/3274172421 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efd501035e0 0x7efd5010bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:58.071 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.071+0000 7efd56e3c640 1 --2- 172.21.15.26:0/3274172421 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efd50102cc0 0x7efd501030a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:58.071 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.071+0000 7efd56e3c640 1 -- 172.21.15.26:0/3274172421 >> 172.21.15.26:0/3274172421 conn(0x7efd500fc6b0 msgr2=0x7efd500fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:32:58.071 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.071+0000 7efd56e3c640 1 -- 172.21.15.26:0/3274172421 shutdown_connections 2024-09-11T23:32:58.071 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.071+0000 7efd56e3c640 1 -- 172.21.15.26:0/3274172421 wait complete. 2024-09-11T23:32:58.074 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.071+0000 7efd56e3c640 1 Processor -- start 2024-09-11T23:32:58.074 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.072+0000 7efd56e3c640 1 -- start start 2024-09-11T23:32:58.074 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.072+0000 7efd56e3c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd50102cc0 0x7efd501982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:58.074 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.072+0000 7efd56e3c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efd501035e0 0x7efd50198820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:58.074 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.072+0000 7efd56e3c640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efd5010c110 0x7efd5019cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:58.074 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.072+0000 7efd56e3c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7efd50111ef0 con 0x7efd50102cc0 2024-09-11T23:32:58.074 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.072+0000 7efd56e3c640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7efd50111d70 con 0x7efd501035e0 2024-09-11T23:32:58.074 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.072+0000 7efd56e3c640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7efd50112070 con 0x7efd5010c110 2024-09-11T23:32:58.074 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.072+0000 7efd54bd8640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd50102cc0 0x7efd501982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:58.074 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.072+0000 7efd54bd8640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd50102cc0 0x7efd501982e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36834/0 (socket says 172.21.15.26:36834) 2024-09-11T23:32:58.074 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.072+0000 7efd54bd8640 1 -- 172.21.15.26:0/3774668831 learned_addr learned my addr 172.21.15.26:0/3774668831 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:32:58.075 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.072+0000 7efd47fff640 1 --2- 172.21.15.26:0/3774668831 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efd501035e0 0x7efd50198820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:58.075 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.072+0000 7efd553d9640 1 --2- 172.21.15.26:0/3774668831 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efd5010c110 0x7efd5019cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:58.075 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.072+0000 7efd54bd8640 1 -- 172.21.15.26:0/3774668831 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efd5010c110 msgr2=0x7efd5019cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:32:58.075 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.072+0000 7efd54bd8640 1 --2- 172.21.15.26:0/3774668831 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efd5010c110 0x7efd5019cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:58.075 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.072+0000 7efd54bd8640 1 -- 172.21.15.26:0/3774668831 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efd501035e0 msgr2=0x7efd50198820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:32:58.075 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.072+0000 7efd54bd8640 1 --2- 172.21.15.26:0/3774668831 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efd501035e0 0x7efd50198820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:58.075 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.072+0000 7efd54bd8640 1 -- 172.21.15.26:0/3774668831 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd5019d2d0 con 0x7efd50102cc0 2024-09-11T23:32:58.075 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.073+0000 7efd54bd8640 1 --2- 172.21.15.26:0/3774668831 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd50102cc0 0x7efd501982e0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7efd4c00d950 tx=0x7efd4c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:32:58.075 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.073+0000 7efd45ffb640 1 -- 172.21.15.26:0/3774668831 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd4c017070 con 0x7efd50102cc0 2024-09-11T23:32:58.076 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.073+0000 7efd45ffb640 1 -- 172.21.15.26:0/3774668831 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd4c009c30 con 0x7efd50102cc0 2024-09-11T23:32:58.076 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.073+0000 7efd45ffb640 1 -- 172.21.15.26:0/3774668831 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd4c00f6b0 con 0x7efd50102cc0 2024-09-11T23:32:58.076 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.073+0000 7efd56e3c640 1 -- 172.21.15.26:0/3774668831 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd5019d5c0 con 0x7efd50102cc0 2024-09-11T23:32:58.076 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.073+0000 7efd56e3c640 1 -- 172.21.15.26:0/3774668831 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd501a4e50 con 0x7efd50102cc0 2024-09-11T23:32:58.083 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.075+0000 7efd45ffb640 1 -- 172.21.15.26:0/3774668831 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7efd4c0109d0 con 0x7efd50102cc0 2024-09-11T23:32:58.083 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.076+0000 7efd59bd4740 1 Processor -- start 2024-09-11T23:32:58.083 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.076+0000 7efd59bd4740 1 -- start start 2024-09-11T23:32:58.083 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.076+0000 7efd59bd4740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5630e0c8eaf0 0x5630e0c8eed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:58.083 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.076+0000 7efd59bd4740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5630e0c8f410 0x5630e0c99770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:58.083 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.076+0000 7efd59bd4740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5630e0c99cb0 0x5630e0c9c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:58.083 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.076+0000 7efd59bd4740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5630e0bf32f0 con 0x5630e0c99cb0 2024-09-11T23:32:58.084 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.076+0000 7efd59bd4740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5630e0b9b270 con 0x5630e0c8f410 2024-09-11T23:32:58.084 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.076+0000 7efd59bd4740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5630e0bef2b0 con 0x5630e0c8eaf0 2024-09-11T23:32:58.084 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.076+0000 7efd47fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5630e0c99cb0 0x5630e0c9c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:58.084 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.076+0000 7efd45ffb640 1 --2- 172.21.15.26:0/3774668831 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7efd380a2e20 0x7efd380a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:58.084 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.076+0000 7efd553d9640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5630e0c8eaf0 0x5630e0c8eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:58.084 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.076+0000 7efd47fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5630e0c99cb0 0x5630e0c9c0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36836/0 (socket says 172.21.15.26:36836) 2024-09-11T23:32:58.084 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.076+0000 7efd47fff640 1 -- 172.21.15.26:0/65341344 learned_addr learned my addr 172.21.15.26:0/65341344 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:32:58.084 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.076+0000 7efd54bd8640 1 --2- 172.21.15.26:0/65341344 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5630e0c8f410 0x5630e0c99770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:58.084 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.077+0000 7efd553d9640 1 --2- 172.21.15.26:0/3774668831 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7efd380a2e20 0x7efd380a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:58.084 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.077+0000 7efd47fff640 1 -- 172.21.15.26:0/65341344 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5630e0c8eaf0 msgr2=0x5630e0c8eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:32:58.084 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.077+0000 7efd47fff640 1 --2- 172.21.15.26:0/65341344 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5630e0c8eaf0 0x5630e0c8eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:58.085 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.077+0000 7efd47fff640 1 -- 172.21.15.26:0/65341344 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5630e0c8f410 msgr2=0x5630e0c99770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:32:58.085 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.077+0000 7efd47fff640 1 --2- 172.21.15.26:0/65341344 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5630e0c8f410 0x5630e0c99770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:58.085 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.077+0000 7efd47fff640 1 -- 172.21.15.26:0/65341344 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630e0b9a340 con 0x5630e0c99cb0 2024-09-11T23:32:58.085 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.077+0000 7efd45ffb640 1 -- 172.21.15.26:0/3774668831 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8542+0+0 (secure 0 0 0) 0x7efd4c0cefa0 con 0x7efd50102cc0 2024-09-11T23:32:58.085 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.077+0000 7efd553d9640 1 --2- 172.21.15.26:0/65341344 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5630e0c8eaf0 0x5630e0c8eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:32:58.085 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.077+0000 7efd47fff640 1 --2- 172.21.15.26:0/65341344 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5630e0c99cb0 0x5630e0c9c0a0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7efd4000bc00 tx=0x7efd400107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e79e5e46fe09553e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:32:58.085 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.077+0000 7efd553d9640 1 --2- 172.21.15.26:0/3774668831 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7efd380a2e20 0x7efd380a52e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7efd48004ad0 tx=0x7efd4802a950 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:32:58.085 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.077+0000 7efd36ffd640 1 -- 172.21.15.26:0/65341344 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd40005020 con 0x5630e0c99cb0 2024-09-11T23:32:58.085 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.077+0000 7efd36ffd640 1 -- 172.21.15.26:0/65341344 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd400047a0 con 0x5630e0c99cb0 2024-09-11T23:32:58.085 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.077+0000 7efd36ffd640 1 -- 172.21.15.26:0/65341344 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd400117c0 con 0x5630e0c99cb0 2024-09-11T23:32:58.085 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.078+0000 7efd59bd4740 1 -- 172.21.15.26:0/65341344 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5630e0c99cb0 msgr2=0x5630e0c9c0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:32:58.085 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.078+0000 7efd59bd4740 1 --2- 172.21.15.26:0/65341344 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5630e0c99cb0 0x5630e0c9c0a0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7efd4000bc00 tx=0x7efd400107a0 comp rx=0 tx=0).stop 2024-09-11T23:32:58.085 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.078+0000 7efd59bd4740 1 -- 172.21.15.26:0/65341344 shutdown_connections 2024-09-11T23:32:58.086 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.078+0000 7efd59bd4740 1 --2- 172.21.15.26:0/65341344 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5630e0c8f410 0x5630e0c99770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:58.086 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.078+0000 7efd59bd4740 1 --2- 172.21.15.26:0/65341344 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5630e0c99cb0 0x5630e0c9c0a0 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:58.086 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.078+0000 7efd59bd4740 1 --2- 172.21.15.26:0/65341344 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5630e0c8eaf0 0x5630e0c8eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:58.086 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.078+0000 7efd59bd4740 1 -- 172.21.15.26:0/65341344 >> 172.21.15.26:0/65341344 conn(0x5630e0c8b060 msgr2=0x5630e0c8b440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:32:58.086 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.078+0000 7efd59bd4740 1 -- 172.21.15.26:0/65341344 shutdown_connections 2024-09-11T23:32:58.086 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.078+0000 7efd59bd4740 1 -- 172.21.15.26:0/65341344 wait complete. 2024-09-11T23:32:58.086 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.078+0000 7efd59bd4740 1 -- start start 2024-09-11T23:32:58.086 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.079+0000 7efd59bd4740 1 Processor -- start 2024-09-11T23:32:58.086 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.079+0000 7efd59bd4740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5630e0c8eaf0 0x5630e0c96b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:58.086 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.079+0000 7efd59bd4740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5630e0c8f410 0x5630e0c970a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:58.086 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.079+0000 7efd59bd4740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5630e0c99cb0 0x5630e0c975e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:58.087 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.079+0000 7efd59bd4740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5630e0c9c880 con 0x5630e0c8eaf0 2024-09-11T23:32:58.087 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.079+0000 7efd59bd4740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5630e0b7c290 con 0x5630e0c99cb0 2024-09-11T23:32:58.087 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.079+0000 7efd59bd4740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5630e0c9ca00 con 0x5630e0c8f410 2024-09-11T23:32:58.087 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.079+0000 7efd59bd4740 10 client.0 _ll_register_callbacks cb 0x5630e0bad140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:32:58.087 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.079+0000 7efd54bd8640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5630e0c8eaf0 0x5630e0c96b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:58.087 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.079+0000 7efd54bd8640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5630e0c8eaf0 0x5630e0c96b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36840/0 (socket says 172.21.15.26:36840) 2024-09-11T23:32:58.087 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.079+0000 7efd54bd8640 1 -- 172.21.15.26:0/3286017598 learned_addr learned my addr 172.21.15.26:0/3286017598 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:32:58.087 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.079+0000 7efd47fff640 1 --2- 172.21.15.26:0/3286017598 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5630e0c8f410 0x5630e0c970a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:58.087 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.079+0000 7efd553d9640 1 --2- 172.21.15.26:0/3286017598 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5630e0c99cb0 0x5630e0c975e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:58.087 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.080+0000 7efd54bd8640 1 -- 172.21.15.26:0/3286017598 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5630e0c8f410 msgr2=0x5630e0c970a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:32:58.087 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.080+0000 7efd54bd8640 1 --2- 172.21.15.26:0/3286017598 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5630e0c8f410 0x5630e0c970a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:58.087 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.080+0000 7efd54bd8640 1 -- 172.21.15.26:0/3286017598 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5630e0c99cb0 msgr2=0x5630e0c975e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:32:58.087 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.080+0000 7efd54bd8640 1 --2- 172.21.15.26:0/3286017598 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5630e0c99cb0 0x5630e0c975e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:58.088 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.080+0000 7efd54bd8640 1 -- 172.21.15.26:0/3286017598 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5630e0906210 con 0x5630e0c8eaf0 2024-09-11T23:32:58.088 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.080+0000 7efd47fff640 1 --2- 172.21.15.26:0/3286017598 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5630e0c8f410 0x5630e0c970a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:32:58.088 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.080+0000 7efd54bd8640 1 --2- 172.21.15.26:0/3286017598 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5630e0c8eaf0 0x5630e0c96b60 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7efd4c0cc6e0 tx=0x7efd4c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=938b0a204c61f1c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:32:58.088 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.080+0000 7efd2ffff640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:32:58.088 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.080+0000 7efd2ffff640 1 -- 172.21.15.26:0/3286017598 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd4c09f800 con 0x5630e0c8eaf0 2024-09-11T23:32:58.088 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.080+0000 7efd2ffff640 1 -- 172.21.15.26:0/3286017598 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8542+0+0 (secure 0 0 0) 0x7efd4c0cc900 con 0x5630e0c8eaf0 2024-09-11T23:32:58.088 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.080+0000 7efd59bd4740 1 -- 172.21.15.26:0/3286017598 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd400047a0 con 0x5630e0c8eaf0 2024-09-11T23:32:58.088 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.080+0000 7efd2ffff640 1 -- 172.21.15.26:0/3286017598 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) ==== 24+0+0 (secure 0 0 0) 0x7efd4c0ccb00 con 0x5630e0c8eaf0 2024-09-11T23:32:58.088 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.080+0000 7efd59bd4740 10 client.17448 fetch_fsmap learned FSMap version 71 2024-09-11T23:32:58.088 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.080+0000 7efd59bd4740 1 -- 172.21.15.26:0/3286017598 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=4+}) -- 0x5630e09d2620 con 0x5630e0c8eaf0 2024-09-11T23:32:58.088 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.080+0000 7efd2ffff640 1 -- 172.21.15.26:0/3286017598 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd400047a0 con 0x5630e0c8eaf0 2024-09-11T23:32:58.088 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.081+0000 7efd2ffff640 1 -- 172.21.15.26:0/3286017598 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 71) ==== 8430+0+0 (secure 0 0 0) 0x7efd4c09a9e0 con 0x5630e0c8eaf0 2024-09-11T23:32:58.088 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.081+0000 7efd59bd4740 10 client.17448 fetch_fsmap finished waiting for FSMap version 71 2024-09-11T23:32:58.088 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.081+0000 7efd59bd4740 10 client.17448 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:32:58.089 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.081+0000 7efd59bd4740 20 client.17448 populate_metadata read hostname 'smithi026' 2024-09-11T23:32:58.089 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.081+0000 7efd59bd4740 1 --2- 172.21.15.26:0/3286017598 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5630e0cadfd0 0x5630e0cb0440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:58.089 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.081+0000 7efd59bd4740 4 client.17448 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:32:58.089 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.081+0000 7efd59bd4740 1 -- 172.21.15.26:0/3286017598 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5630e0c00a20 con 0x5630e0cadfd0 2024-09-11T23:32:58.089 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.081+0000 7efd47fff640 1 --2- 172.21.15.26:0/3286017598 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5630e0cadfd0 0x5630e0cb0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:58.089 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.081+0000 7efd2ffff640 10 client.17448 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:32:58.089 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.081+0000 7efd47fff640 1 --2- 172.21.15.26:0/3286017598 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5630e0cadfd0 0x5630e0cb0440 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c44bf8bd1e41f779 server_cookie=bd08cb6aa4a55d06 in_seq=0 out_seq=0 2024-09-11T23:32:58.089 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.082+0000 7efd2ffff640 1 -- 172.21.15.26:0/3286017598 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5630e0c00a20 con 0x5630e0cadfd0 2024-09-11T23:32:58.089 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.082+0000 7efd2ffff640 10 client.17448 handle_command_reply: tid=0 2024-09-11T23:32:58.089 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.082+0000 7efd59bd4740 1 client.17448 shutdown 2024-09-11T23:32:58.089 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.082+0000 7efd59bd4740 1 -- 172.21.15.26:0/3286017598 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5630e0c8eaf0 msgr2=0x5630e0c96b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:32:58.089 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.082+0000 7efd59bd4740 1 --2- 172.21.15.26:0/3286017598 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5630e0c8eaf0 0x5630e0c96b60 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7efd4c0cc6e0 tx=0x7efd4c000bc0 comp rx=0 tx=0).stop 2024-09-11T23:32:58.089 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.082+0000 7efd59bd4740 1 -- 172.21.15.26:0/3286017598 shutdown_connections 2024-09-11T23:32:58.090 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.082+0000 7efd59bd4740 1 --2- 172.21.15.26:0/3286017598 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5630e0c99cb0 0x5630e0c975e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:58.090 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.082+0000 7efd59bd4740 1 --2- 172.21.15.26:0/3286017598 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5630e0cadfd0 0x5630e0cb0440 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:58.090 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.082+0000 7efd59bd4740 1 --2- 172.21.15.26:0/3286017598 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5630e0c8f410 0x5630e0c970a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:58.090 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.082+0000 7efd59bd4740 1 --2- 172.21.15.26:0/3286017598 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5630e0c8eaf0 0x5630e0c96b60 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:58.090 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.082+0000 7efd2ffff640 0 client.17448 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:32:58.090 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.082+0000 7efd59bd4740 1 -- 172.21.15.26:0/3286017598 >> 172.21.15.26:0/3286017598 conn(0x5630e0c8b060 msgr2=0x5630e0c90d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:32:58.090 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.082+0000 7efd59bd4740 1 -- 172.21.15.26:0/3286017598 shutdown_connections 2024-09-11T23:32:58.090 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.082+0000 7efd59bd4740 1 -- 172.21.15.26:0/3286017598 wait complete. 2024-09-11T23:32:58.090 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.083+0000 7efd59bd4740 20 client.17448 trim_cache size 0 max 16384 2024-09-11T23:32:58.098 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.097+0000 7efd59bd4740 1 Processor -- start 2024-09-11T23:32:58.098 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.097+0000 7efd59bd4740 1 -- start start 2024-09-11T23:32:58.098 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.097+0000 7efd59bd4740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5630e0c8eaf0 0x5630e0cb5d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:58.098 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.097+0000 7efd59bd4740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5630e0c8f410 0x5630e0cb6290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:58.098 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.098+0000 7efd59bd4740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5630e0c99cb0 0x5630e0c93da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:58.098 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.098+0000 7efd59bd4740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7efd501ab680 con 0x5630e0c8eaf0 2024-09-11T23:32:58.098 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.098+0000 7efd59bd4740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7efd501ad430 con 0x5630e0c99cb0 2024-09-11T23:32:58.098 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.098+0000 7efd59bd4740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7efd501abd80 con 0x5630e0c8f410 2024-09-11T23:32:58.098 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.098+0000 7efd54bd8640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5630e0c8eaf0 0x5630e0cb5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:58.098 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.098+0000 7efd54bd8640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5630e0c8eaf0 0x5630e0cb5d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36846/0 (socket says 172.21.15.26:36846) 2024-09-11T23:32:58.098 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.098+0000 7efd54bd8640 1 -- 172.21.15.26:0/677511165 learned_addr learned my addr 172.21.15.26:0/677511165 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:32:58.098 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.098+0000 7efd54bd8640 1 -- 172.21.15.26:0/677511165 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5630e0c8f410 msgr2=0x5630e0cb6290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:32:58.099 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.098+0000 7efd54bd8640 1 --2- 172.21.15.26:0/677511165 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5630e0c8f410 0x5630e0cb6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:58.099 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.098+0000 7efd54bd8640 1 -- 172.21.15.26:0/677511165 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5630e0c99cb0 msgr2=0x5630e0c93da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:32:58.099 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.098+0000 7efd54bd8640 1 --2- 172.21.15.26:0/677511165 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5630e0c99cb0 0x5630e0c93da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:58.099 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.098+0000 7efd54bd8640 1 -- 172.21.15.26:0/677511165 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630e0c92700 con 0x5630e0c8eaf0 2024-09-11T23:32:58.099 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.098+0000 7efd54bd8640 1 --2- 172.21.15.26:0/677511165 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5630e0c8eaf0 0x5630e0cb5d50 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7efd4c09b490 tx=0x7efd4c010d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5acc0c6aa98fa31 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:32:58.099 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.098+0000 7efd367fc640 1 -- 172.21.15.26:0/677511165 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd4c010800 con 0x5630e0c8eaf0 2024-09-11T23:32:58.099 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.098+0000 7efd367fc640 1 -- 172.21.15.26:0/677511165 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd4c0a40c0 con 0x5630e0c8eaf0 2024-09-11T23:32:58.099 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.098+0000 7efd367fc640 1 -- 172.21.15.26:0/677511165 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd4c09a950 con 0x5630e0c8eaf0 2024-09-11T23:32:58.103 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.099+0000 7efd59bd4740 1 -- 172.21.15.26:0/677511165 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5630e0c8eaf0 msgr2=0x5630e0cb5d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:32:58.103 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.099+0000 7efd59bd4740 1 --2- 172.21.15.26:0/677511165 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5630e0c8eaf0 0x5630e0cb5d50 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7efd4c09b490 tx=0x7efd4c010d70 comp rx=0 tx=0).stop 2024-09-11T23:32:58.103 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.099+0000 7efd59bd4740 1 -- 172.21.15.26:0/677511165 shutdown_connections 2024-09-11T23:32:58.103 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.099+0000 7efd59bd4740 1 --2- 172.21.15.26:0/677511165 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5630e0c99cb0 0x5630e0c93da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:58.103 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.099+0000 7efd59bd4740 1 --2- 172.21.15.26:0/677511165 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5630e0c8f410 0x5630e0cb6290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:58.103 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.099+0000 7efd59bd4740 1 --2- 172.21.15.26:0/677511165 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5630e0c8eaf0 0x5630e0cb5d50 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:58.103 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.099+0000 7efd59bd4740 1 -- 172.21.15.26:0/677511165 >> 172.21.15.26:0/677511165 conn(0x5630e0c8b060 msgr2=0x5630e0c98be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:32:58.103 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.099+0000 7efd59bd4740 1 -- 172.21.15.26:0/677511165 shutdown_connections 2024-09-11T23:32:58.103 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.099+0000 7efd59bd4740 1 -- 172.21.15.26:0/677511165 wait complete. 2024-09-11T23:32:58.103 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.099+0000 7efd59bd4740 1 -- start start 2024-09-11T23:32:58.104 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.099+0000 7efd59bd4740 1 Processor -- start 2024-09-11T23:32:58.104 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.100+0000 7efd59bd4740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5630e0c8eaf0 0x5630e0ca2740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:58.104 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.100+0000 7efd59bd4740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5630e0c8f410 0x5630e0ca2c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:58.104 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.100+0000 7efd59bd4740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5630e0c99cb0 0x5630e0d11cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:58.104 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.100+0000 7efd59bd4740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7efd501acca0 con 0x5630e0c99cb0 2024-09-11T23:32:58.104 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.100+0000 7efd59bd4740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7efd501ac510 con 0x5630e0c8eaf0 2024-09-11T23:32:58.104 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.100+0000 7efd59bd4740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7efd501a9680 con 0x5630e0c8f410 2024-09-11T23:32:58.104 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.100+0000 7efd59bd4740 10 client.0 _ll_register_callbacks cb 0x7efd4c09b0d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:32:58.104 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.100+0000 7efd553d9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5630e0c99cb0 0x5630e0d11cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:58.104 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.100+0000 7efd553d9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5630e0c99cb0 0x5630e0d11cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36852/0 (socket says 172.21.15.26:36852) 2024-09-11T23:32:58.104 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.100+0000 7efd553d9640 1 -- 172.21.15.26:0/3653818437 learned_addr learned my addr 172.21.15.26:0/3653818437 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:32:58.104 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.100+0000 7efd54bd8640 1 --2- 172.21.15.26:0/3653818437 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5630e0c8eaf0 0x5630e0ca2740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:58.104 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.100+0000 7efd47fff640 1 --2- 172.21.15.26:0/3653818437 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5630e0c8f410 0x5630e0ca2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:58.104 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.100+0000 7efd553d9640 1 -- 172.21.15.26:0/3653818437 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5630e0c8f410 msgr2=0x5630e0ca2c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:32:58.105 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.100+0000 7efd553d9640 1 --2- 172.21.15.26:0/3653818437 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5630e0c8f410 0x5630e0ca2c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:58.105 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.100+0000 7efd553d9640 1 -- 172.21.15.26:0/3653818437 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5630e0c8eaf0 msgr2=0x5630e0ca2740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:32:58.105 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.100+0000 7efd553d9640 1 --2- 172.21.15.26:0/3653818437 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5630e0c8eaf0 0x5630e0ca2740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:58.105 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.100+0000 7efd553d9640 1 -- 172.21.15.26:0/3653818437 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5630e0d12330 con 0x5630e0c99cb0 2024-09-11T23:32:58.105 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.100+0000 7efd54bd8640 1 --2- 172.21.15.26:0/3653818437 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5630e0c8eaf0 0x5630e0ca2740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:32:58.105 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.100+0000 7efd47fff640 1 --2- 172.21.15.26:0/3653818437 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5630e0c8f410 0x5630e0ca2c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:32:58.105 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.100+0000 7efd553d9640 1 --2- 172.21.15.26:0/3653818437 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5630e0c99cb0 0x5630e0d11cf0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7efd48035000 tx=0x7efd4802ff80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d315d8de324db32e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:32:58.105 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.100+0000 7efd2ffff640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:32:58.105 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.101+0000 7efd2ffff640 1 -- 172.21.15.26:0/3653818437 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd48043050 con 0x5630e0c99cb0 2024-09-11T23:32:58.105 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.101+0000 7efd2ffff640 1 -- 172.21.15.26:0/3653818437 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8542+0+0 (secure 0 0 0) 0x7efd48004450 con 0x5630e0c99cb0 2024-09-11T23:32:58.105 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.101+0000 7efd59bd4740 1 -- 172.21.15.26:0/3653818437 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd4c0a40c0 con 0x5630e0c99cb0 2024-09-11T23:32:58.105 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.101+0000 7efd2ffff640 1 -- 172.21.15.26:0/3653818437 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) ==== 24+0+0 (secure 0 0 0) 0x7efd48034040 con 0x5630e0c99cb0 2024-09-11T23:32:58.105 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.101+0000 7efd59bd4740 10 client.17454 fetch_fsmap learned FSMap version 71 2024-09-11T23:32:58.106 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.101+0000 7efd59bd4740 1 -- 172.21.15.26:0/3653818437 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=4+}) -- 0x5630e0d196d0 con 0x5630e0c99cb0 2024-09-11T23:32:58.106 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.101+0000 7efd2ffff640 1 -- 172.21.15.26:0/3653818437 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd4c0a40c0 con 0x5630e0c99cb0 2024-09-11T23:32:58.106 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.101+0000 7efd2ffff640 1 -- 172.21.15.26:0/3653818437 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 71) ==== 8430+0+0 (secure 0 0 0) 0x7efd48057530 con 0x5630e0c99cb0 2024-09-11T23:32:58.106 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.101+0000 7efd59bd4740 10 client.17454 fetch_fsmap finished waiting for FSMap version 71 2024-09-11T23:32:58.106 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.101+0000 7efd59bd4740 10 client.17454 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:32:58.106 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.101+0000 7efd59bd4740 20 client.17454 populate_metadata read hostname 'smithi026' 2024-09-11T23:32:58.106 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.101+0000 7efd59bd4740 1 --2- 172.21.15.26:0/3653818437 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5630e0d19f10 0x5630e0d1c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:58.106 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.101+0000 7efd59bd4740 4 client.17454 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T23:32:58.106 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.101+0000 7efd59bd4740 1 -- 172.21.15.26:0/3653818437 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5630e0bfad70 con 0x5630e0d19f10 2024-09-11T23:32:58.106 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.102+0000 7efd54bd8640 1 --2- 172.21.15.26:0/3653818437 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5630e0d19f10 0x5630e0d1c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:58.106 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.102+0000 7efd54bd8640 1 --2- 172.21.15.26:0/3653818437 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5630e0d19f10 0x5630e0d1c330 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb592ddaaa41d44 server_cookie=51df827d0a638e in_seq=0 out_seq=0 2024-09-11T23:32:58.106 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.102+0000 7efd2ffff640 10 client.17454 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:32:58.118 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.117+0000 7efd2ffff640 1 -- 172.21.15.26:0/3653818437 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5630e0bfad70 con 0x5630e0d19f10 2024-09-11T23:32:58.118 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.117+0000 7efd2ffff640 10 client.17454 handle_command_reply: tid=0 2024-09-11T23:32:58.118 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.117+0000 7efd59bd4740 1 client.17454 shutdown 2024-09-11T23:32:58.118 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.118+0000 7efd59bd4740 1 -- 172.21.15.26:0/3653818437 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5630e0c99cb0 msgr2=0x5630e0d11cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:32:58.118 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.118+0000 7efd59bd4740 1 --2- 172.21.15.26:0/3653818437 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5630e0c99cb0 0x5630e0d11cf0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7efd48035000 tx=0x7efd4802ff80 comp rx=0 tx=0).stop 2024-09-11T23:32:58.118 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.118+0000 7efd59bd4740 1 -- 172.21.15.26:0/3653818437 shutdown_connections 2024-09-11T23:32:58.118 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.118+0000 7efd59bd4740 1 --2- 172.21.15.26:0/3653818437 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5630e0d19f10 0x5630e0d1c330 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:58.118 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.118+0000 7efd59bd4740 1 --2- 172.21.15.26:0/3653818437 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5630e0c99cb0 0x5630e0d11cf0 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:58.118 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.118+0000 7efd59bd4740 1 --2- 172.21.15.26:0/3653818437 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5630e0c8f410 0x5630e0ca2c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:58.118 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.118+0000 7efd2ffff640 0 client.17454 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:32:58.118 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.118+0000 7efd59bd4740 1 --2- 172.21.15.26:0/3653818437 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5630e0c8eaf0 0x5630e0ca2740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:58.119 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.118+0000 7efd59bd4740 1 -- 172.21.15.26:0/3653818437 >> 172.21.15.26:0/3653818437 conn(0x5630e0c8b060 msgr2=0x5630e0c899a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:32:58.119 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.118+0000 7efd59bd4740 1 -- 172.21.15.26:0/3653818437 shutdown_connections 2024-09-11T23:32:58.119 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.118+0000 7efd59bd4740 1 -- 172.21.15.26:0/3653818437 wait complete. 2024-09-11T23:32:58.119 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.118+0000 7efd59bd4740 20 client.17454 trim_cache size 0 max 16384 2024-09-11T23:32:58.119 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-09-11T23:32:58.119 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "message": "", 2024-09-11T23:32:58.119 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-09-11T23:32:58.119 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-09-11T23:32:58.120 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.119+0000 7efd56e3c640 1 -- 172.21.15.26:0/3774668831 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7efd380a2e20 msgr2=0x7efd380a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:32:58.120 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.119+0000 7efd56e3c640 1 --2- 172.21.15.26:0/3774668831 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7efd380a2e20 0x7efd380a52e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7efd48004ad0 tx=0x7efd4802a950 comp rx=0 tx=0).stop 2024-09-11T23:32:58.120 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.119+0000 7efd56e3c640 1 -- 172.21.15.26:0/3774668831 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd50102cc0 msgr2=0x7efd501982e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:32:58.120 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.119+0000 7efd56e3c640 1 --2- 172.21.15.26:0/3774668831 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd50102cc0 0x7efd501982e0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7efd4c00d950 tx=0x7efd4c00de10 comp rx=0 tx=0).stop 2024-09-11T23:32:58.120 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.119+0000 7efd56e3c640 1 -- 172.21.15.26:0/3774668831 shutdown_connections 2024-09-11T23:32:58.120 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.119+0000 7efd56e3c640 1 --2- 172.21.15.26:0/3774668831 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7efd380a2e20 0x7efd380a52e0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:58.121 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.119+0000 7efd56e3c640 1 --2- 172.21.15.26:0/3774668831 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efd501035e0 0x7efd50198820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:58.121 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.119+0000 7efd56e3c640 1 --2- 172.21.15.26:0/3774668831 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efd5010c110 0x7efd5019cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:58.121 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.119+0000 7efd56e3c640 1 --2- 172.21.15.26:0/3774668831 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd50102cc0 0x7efd501982e0 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:58.121 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.119+0000 7efd56e3c640 1 -- 172.21.15.26:0/3774668831 >> 172.21.15.26:0/3774668831 conn(0x7efd500fc6b0 msgr2=0x7efd500fd2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:32:58.121 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.119+0000 7efd56e3c640 1 -- 172.21.15.26:0/3774668831 shutdown_connections 2024-09-11T23:32:58.121 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:32:58.119+0000 7efd56e3c640 1 -- 172.21.15.26:0/3774668831 wait complete. 2024-09-11T23:32:58.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:32:58 smithi136 ceph-mon[35121]: pgmap v456: 97 pgs: 97 active+clean; 2.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 23 KiB/s wr, 29 op/s 2024-09-11T23:32:58.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:32:58 smithi026 ceph-mon[31172]: pgmap v456: 97 pgs: 97 active+clean; 2.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 23 KiB/s wr, 29 op/s 2024-09-11T23:32:58.552 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:32:58 smithi076 ceph-mon[35296]: pgmap v456: 97 pgs: 97 active+clean; 2.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 23 KiB/s wr, 29 op/s 2024-09-11T23:32:59.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:32:59 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:32:59.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:32:59 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:32:59.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:32:59 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:32:59.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:32:59 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:32:59.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:32:59 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:32:59.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:32:59 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:32:59.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:32:59 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:32:59.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:32:59 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:32:59.552 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:32:59 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:32:59.553 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:32:59 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:32:59.553 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:32:59 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:32:59.553 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:32:59 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:33:00.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:33:00 smithi136 ceph-mon[35121]: pgmap v457: 97 pgs: 97 active+clean; 2.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 22 KiB/s wr, 21 op/s 2024-09-11T23:33:00.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:33:00 smithi026 ceph-mon[31172]: pgmap v457: 97 pgs: 97 active+clean; 2.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 22 KiB/s wr, 21 op/s 2024-09-11T23:33:00.552 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:33:00 smithi076 ceph-mon[35296]: pgmap v457: 97 pgs: 97 active+clean; 2.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 22 KiB/s wr, 21 op/s 2024-09-11T23:33:02.444 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:33:02.473 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:33:02 smithi026 ceph-mon[31172]: pgmap v458: 97 pgs: 97 active+clean; 2.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 33 KiB/s wr, 29 op/s 2024-09-11T23:33:02.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:33:02 smithi136 ceph-mon[35121]: pgmap v458: 97 pgs: 97 active+clean; 2.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 33 KiB/s wr, 29 op/s 2024-09-11T23:33:02.552 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:33:02 smithi076 ceph-mon[35296]: pgmap v458: 97 pgs: 97 active+clean; 2.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 33 KiB/s wr, 29 op/s 2024-09-11T23:33:02.717 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:33:03.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:03.224+0000 7f0319562640 1 -- 172.21.15.26:0/683939209 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0314100a80 msgr2=0x7f0314100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:33:03.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:03.224+0000 7f0319562640 1 --2- 172.21.15.26:0/683939209 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0314100a80 0x7f0314100ee0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f0308009930 tx=0x7f030802a420 comp rx=0 tx=0).stop 2024-09-11T23:33:03.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:03.224+0000 7f0319562640 1 -- 172.21.15.26:0/683939209 shutdown_connections 2024-09-11T23:33:03.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:03.224+0000 7f0319562640 1 --2- 172.21.15.26:0/683939209 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0314101420 0x7f031410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:03.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:03.224+0000 7f0319562640 1 --2- 172.21.15.26:0/683939209 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0314100a80 0x7f0314100ee0 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:03.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:03.224+0000 7f0319562640 1 --2- 172.21.15.26:0/683939209 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0314106a80 0x7f0314106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:03.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:03.224+0000 7f0319562640 1 -- 172.21.15.26:0/683939209 >> 172.21.15.26:0/683939209 conn(0x7f03140fc7a0 msgr2=0x7f03140febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:33:03.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:03.224+0000 7f0319562640 1 -- 172.21.15.26:0/683939209 shutdown_connections 2024-09-11T23:33:03.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:03.224+0000 7f0319562640 1 -- 172.21.15.26:0/683939209 wait complete. 2024-09-11T23:33:03.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:03.224+0000 7f0319562640 1 Processor -- start 2024-09-11T23:33:03.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:03.225+0000 7f0319562640 1 -- start start 2024-09-11T23:33:03.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:03.225+0000 7f0319562640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0314100a80 0x7f03141992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:03.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:03.225+0000 7f0319562640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0314101420 0x7f03141997e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:03.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:03.225+0000 7f0319562640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0314106a80 0x7f03141a4320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:03.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:03.225+0000 7f0319562640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f0314110e80 con 0x7f0314100a80 2024-09-11T23:33:03.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:03.225+0000 7f0319562640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f0314110d00 con 0x7f0314106a80 2024-09-11T23:33:03.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:03.225+0000 7f0319562640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f0314111000 con 0x7f0314101420 2024-09-11T23:33:03.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:03.225+0000 7f0312ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0314100a80 0x7f03141992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:33:03.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:03.225+0000 7f0312ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0314100a80 0x7f03141992a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52666/0 (socket says 172.21.15.26:52666) 2024-09-11T23:33:03.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:03.225+0000 7f0312ffd640 1 -- 172.21.15.26:0/1337143264 learned_addr learned my addr 172.21.15.26:0/1337143264 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:33:03.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:03.225+0000 7f03137fe640 1 --2- 172.21.15.26:0/1337143264 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0314106a80 0x7f03141a4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:33:03.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:03.225+0000 7f03127fc640 1 --2- 172.21.15.26:0/1337143264 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0314101420 0x7f03141997e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:33:03.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:03.225+0000 7f0312ffd640 1 -- 172.21.15.26:0/1337143264 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0314101420 msgr2=0x7f03141997e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:33:03.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:03.225+0000 7f0312ffd640 1 --2- 172.21.15.26:0/1337143264 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0314101420 0x7f03141997e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:03.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:03.225+0000 7f0312ffd640 1 -- 172.21.15.26:0/1337143264 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0314106a80 msgr2=0x7f03141a4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:33:03.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:03.225+0000 7f0312ffd640 1 --2- 172.21.15.26:0/1337143264 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0314106a80 0x7f03141a4320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:03.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:03.225+0000 7f0312ffd640 1 -- 172.21.15.26:0/1337143264 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03141a4860 con 0x7f0314100a80 2024-09-11T23:33:03.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:03.225+0000 7f03137fe640 1 --2- 172.21.15.26:0/1337143264 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0314106a80 0x7f03141a4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:33:03.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:03.225+0000 7f0312ffd640 1 --2- 172.21.15.26:0/1337143264 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0314100a80 0x7f03141992a0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f030400e9f0 tx=0x7f030400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:33:03.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:03.226+0000 7f02fbfff640 1 -- 172.21.15.26:0/1337143264 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f030400cd80 con 0x7f0314100a80 2024-09-11T23:33:03.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:03.226+0000 7f02fbfff640 1 -- 172.21.15.26:0/1337143264 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0304010910 con 0x7f0314100a80 2024-09-11T23:33:03.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:03.226+0000 7f0319562640 1 -- 172.21.15.26:0/1337143264 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03141a4a30 con 0x7f0314100a80 2024-09-11T23:33:03.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:03.226+0000 7f02fbfff640 1 -- 172.21.15.26:0/1337143264 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0304018990 con 0x7f0314100a80 2024-09-11T23:33:03.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:03.226+0000 7f0319562640 1 -- 172.21.15.26:0/1337143264 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03141a4ef0 con 0x7f0314100a80 2024-09-11T23:33:03.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:03.227+0000 7f0319562640 1 -- 172.21.15.26:0/1337143264 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0314100ee0 con 0x7f0314100a80 2024-09-11T23:33:03.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:03.229+0000 7f02fbfff640 1 -- 172.21.15.26:0/1337143264 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f0304010ac0 con 0x7f0314100a80 2024-09-11T23:33:03.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:03.230+0000 7f02fbfff640 1 --2- 172.21.15.26:0/1337143264 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f02fc0a2cf0 0x7f02fc0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:03.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:03.230+0000 7f03127fc640 1 --2- 172.21.15.26:0/1337143264 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f02fc0a2cf0 0x7f02fc0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:33:03.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:03.230+0000 7f02fbfff640 1 -- 172.21.15.26:0/1337143264 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8542+0+0 (secure 0 0 0) 0x7f03040cfc10 con 0x7f0314100a80 2024-09-11T23:33:03.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:03.230+0000 7f03127fc640 1 --2- 172.21.15.26:0/1337143264 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f02fc0a2cf0 0x7f02fc0a51b0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f0308000c00 tx=0x7f0308005a20 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:33:03.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:03.233+0000 7f02fbfff640 1 -- 172.21.15.26:0/1337143264 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f030409c820 con 0x7f0314100a80 2024-09-11T23:33:03.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:03.444+0000 7f0319562640 1 -- 172.21.15.26:0/1337143264 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f031406ad80 con 0x7f0314100a80 2024-09-11T23:33:03.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:03.445+0000 7f02fbfff640 1 -- 172.21.15.26:0/1337143264 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) ==== 94+0+12001 (secure 0 0 0) 0x7f030409c120 con 0x7f0314100a80 2024-09-11T23:33:03.444 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-11T23:33:03.445 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":71,"btime":"2024-09-11T23:32:43:090819+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14499,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/367659515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":367659515},{"type":"v1","addr":"172.21.15.26:6835","nonce":367659515}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14511,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/2041065299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":2041065299},{"type":"v1","addr":"172.21.15.26:6837","nonce":2041065299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":71,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:16:26.544203+0000","modified":"2024-09-11T23:32:42.846060+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24485,"mds_1":24463,"mds_2":24458,"mds_3":24448,"mds_4":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14550":{"gid":14550,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6839/2233068305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":2233068305},{"type":"v1","addr":"172.21.15.26:6839","nonce":2233068305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14562":{"gid":14562,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6841/2984982336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":2984982336},{"type":"v1","addr":"172.21.15.26:6841","nonce":2984982336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6833/2019841455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":2019841455},{"type":"v1","addr":"172.21.15.136:6833","nonce":2019841455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.76:6833/3438147050","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3438147050},{"type":"v1","addr":"172.21.15.76:6833","nonce":3438147050}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6835/3617665975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3617665975},{"type":"v1","addr":"172.21.15.136:6835","nonce":3617665975}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":12,"addr":"172.21.15.76:6835/393308499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":393308499},{"type":"v1","addr":"172.21.15.76:6835","nonce":393308499}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":10,"addr":"172.21.15.136:6837/1633082387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":1633082387},{"type":"v1","addr":"172.21.15.136:6837","nonce":1633082387}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.76:6837/2078978748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":2078978748},{"type":"v1","addr":"172.21.15.76:6837","nonce":2078978748}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":63,"addr":"172.21.15.136:6839/3211398630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":3211398630},{"type":"v1","addr":"172.21.15.136:6839","nonce":3211398630}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.76:6839/2077637090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6838","nonce":2077637090},{"type":"v1","addr":"172.21.15.76:6839","nonce":2077637090}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24485,"qdb_cluster":[24443,24448,24458,24463,24485]},"id":1}]} 2024-09-11T23:33:03.445 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 71 2024-09-11T23:33:03.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:03.447+0000 7f0319562640 1 -- 172.21.15.26:0/1337143264 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f02fc0a2cf0 msgr2=0x7f02fc0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:33:03.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:03.447+0000 7f0319562640 1 --2- 172.21.15.26:0/1337143264 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f02fc0a2cf0 0x7f02fc0a51b0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f0308000c00 tx=0x7f0308005a20 comp rx=0 tx=0).stop 2024-09-11T23:33:03.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:03.447+0000 7f0319562640 1 -- 172.21.15.26:0/1337143264 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0314100a80 msgr2=0x7f03141992a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:33:03.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:03.447+0000 7f0319562640 1 --2- 172.21.15.26:0/1337143264 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0314100a80 0x7f03141992a0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f030400e9f0 tx=0x7f030400eeb0 comp rx=0 tx=0).stop 2024-09-11T23:33:03.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:03.447+0000 7f0319562640 1 -- 172.21.15.26:0/1337143264 shutdown_connections 2024-09-11T23:33:03.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:03.447+0000 7f0319562640 1 --2- 172.21.15.26:0/1337143264 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f02fc0a2cf0 0x7f02fc0a51b0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:03.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:03.447+0000 7f0319562640 1 --2- 172.21.15.26:0/1337143264 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0314106a80 0x7f03141a4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:03.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:03.447+0000 7f0319562640 1 --2- 172.21.15.26:0/1337143264 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0314101420 0x7f03141997e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:03.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:03.447+0000 7f0319562640 1 --2- 172.21.15.26:0/1337143264 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0314100a80 0x7f03141992a0 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:03.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:03.447+0000 7f0319562640 1 -- 172.21.15.26:0/1337143264 >> 172.21.15.26:0/1337143264 conn(0x7f03140fc7a0 msgr2=0x7f03140fcfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:33:03.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:03.447+0000 7f0319562640 1 -- 172.21.15.26:0/1337143264 shutdown_connections 2024-09-11T23:33:03.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:03.447+0000 7f0319562640 1 -- 172.21.15.26:0/1337143264 wait complete. 2024-09-11T23:33:04.259 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24485 quiesce db --set-id 95c56ce2 --release --await' 2024-09-11T23:33:04.260 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24485 quiesce db --set-id 95c56ce2 --release --await 2024-09-11T23:33:04.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:33:04 smithi136 ceph-mon[35121]: pgmap v459: 97 pgs: 97 active+clean; 2.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 15 KiB/s wr, 14 op/s 2024-09-11T23:33:04.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:33:04 smithi136 ceph-mon[35121]: from='client.? 172.21.15.26:0/1337143264' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:33:04.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:33:04 smithi026 ceph-mon[31172]: pgmap v459: 97 pgs: 97 active+clean; 2.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 15 KiB/s wr, 14 op/s 2024-09-11T23:33:04.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:33:04 smithi026 ceph-mon[31172]: from='client.? 172.21.15.26:0/1337143264' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:33:04.552 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:33:04 smithi076 ceph-mon[35296]: pgmap v459: 97 pgs: 97 active+clean; 2.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 15 KiB/s wr, 14 op/s 2024-09-11T23:33:04.553 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:33:04 smithi076 ceph-mon[35296]: from='client.? 172.21.15.26:0/1337143264' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:33:04.562 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:33:05.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.021+0000 7efe5c7a0640 1 -- 172.21.15.26:0/2078395463 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efe54100a80 msgr2=0x7efe54100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:33:05.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.021+0000 7efe5c7a0640 1 --2- 172.21.15.26:0/2078395463 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efe54100a80 0x7efe54100ee0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7efe44009930 tx=0x7efe4402a420 comp rx=0 tx=0).stop 2024-09-11T23:33:05.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.021+0000 7efe5c7a0640 1 -- 172.21.15.26:0/2078395463 shutdown_connections 2024-09-11T23:33:05.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.021+0000 7efe5c7a0640 1 --2- 172.21.15.26:0/2078395463 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efe54101420 0x7efe5410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:05.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.021+0000 7efe5c7a0640 1 --2- 172.21.15.26:0/2078395463 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efe54100a80 0x7efe54100ee0 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:05.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.021+0000 7efe5c7a0640 1 --2- 172.21.15.26:0/2078395463 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efe54106a80 0x7efe54106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:05.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.021+0000 7efe5c7a0640 1 -- 172.21.15.26:0/2078395463 >> 172.21.15.26:0/2078395463 conn(0x7efe540fc7a0 msgr2=0x7efe540febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:33:05.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.021+0000 7efe5c7a0640 1 -- 172.21.15.26:0/2078395463 shutdown_connections 2024-09-11T23:33:05.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.021+0000 7efe5c7a0640 1 -- 172.21.15.26:0/2078395463 wait complete. 2024-09-11T23:33:05.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.021+0000 7efe5c7a0640 1 Processor -- start 2024-09-11T23:33:05.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.021+0000 7efe5c7a0640 1 -- start start 2024-09-11T23:33:05.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.022+0000 7efe5c7a0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efe54100a80 0x7efe541992b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:05.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.022+0000 7efe5c7a0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efe54101420 0x7efe541997f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:05.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.022+0000 7efe5c7a0640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efe54106a80 0x7efe541a4330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:05.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.022+0000 7efe5c7a0640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7efe54110e80 con 0x7efe54100a80 2024-09-11T23:33:05.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.022+0000 7efe5c7a0640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7efe54110d00 con 0x7efe54101420 2024-09-11T23:33:05.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.022+0000 7efe5c7a0640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7efe54111000 con 0x7efe54106a80 2024-09-11T23:33:05.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.022+0000 7efe5a53c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efe54100a80 0x7efe541992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:33:05.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.022+0000 7efe5a53c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efe54100a80 0x7efe541992b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52680/0 (socket says 172.21.15.26:52680) 2024-09-11T23:33:05.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.022+0000 7efe5a53c640 1 -- 172.21.15.26:0/337480756 learned_addr learned my addr 172.21.15.26:0/337480756 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:33:05.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.022+0000 7efe5ad3d640 1 --2- 172.21.15.26:0/337480756 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efe54106a80 0x7efe541a4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:33:05.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.022+0000 7efe59d3b640 1 --2- 172.21.15.26:0/337480756 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efe54101420 0x7efe541997f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:33:05.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.022+0000 7efe5a53c640 1 -- 172.21.15.26:0/337480756 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efe54106a80 msgr2=0x7efe541a4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:33:05.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.022+0000 7efe5a53c640 1 --2- 172.21.15.26:0/337480756 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efe54106a80 0x7efe541a4330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:05.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.022+0000 7efe5a53c640 1 -- 172.21.15.26:0/337480756 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efe54101420 msgr2=0x7efe541997f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:33:05.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.022+0000 7efe5a53c640 1 --2- 172.21.15.26:0/337480756 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efe54101420 0x7efe541997f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:05.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.022+0000 7efe5a53c640 1 -- 172.21.15.26:0/337480756 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe541a4870 con 0x7efe54100a80 2024-09-11T23:33:05.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.022+0000 7efe5a53c640 1 --2- 172.21.15.26:0/337480756 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efe54100a80 0x7efe541992b0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7efe5000d950 tx=0x7efe5000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:33:05.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.022+0000 7efe4b7fe640 1 -- 172.21.15.26:0/337480756 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe50017070 con 0x7efe54100a80 2024-09-11T23:33:05.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.022+0000 7efe4b7fe640 1 -- 172.21.15.26:0/337480756 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe50009c30 con 0x7efe54100a80 2024-09-11T23:33:05.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.022+0000 7efe5c7a0640 1 -- 172.21.15.26:0/337480756 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe541a4a40 con 0x7efe54100a80 2024-09-11T23:33:05.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.022+0000 7efe4b7fe640 1 -- 172.21.15.26:0/337480756 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe5000f6b0 con 0x7efe54100a80 2024-09-11T23:33:05.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.022+0000 7efe5c7a0640 1 -- 172.21.15.26:0/337480756 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe541a4f50 con 0x7efe54100a80 2024-09-11T23:33:05.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.025+0000 7efe4b7fe640 1 -- 172.21.15.26:0/337480756 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7efe500109d0 con 0x7efe54100a80 2024-09-11T23:33:05.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.025+0000 7efe5eb79740 1 Processor -- start 2024-09-11T23:33:05.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.025+0000 7efe5eb79740 1 -- start start 2024-09-11T23:33:05.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.025+0000 7efe5eb79740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564d702f9060 0x564d702f9440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:05.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.025+0000 7efe5eb79740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564d702f9980 0x564d70303d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:05.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.025+0000 7efe5eb79740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564d70304240 0x564d70306630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:05.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.025+0000 7efe5eb79740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x564d7025e620 con 0x564d70304240 2024-09-11T23:33:05.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.025+0000 7efe5eb79740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x564d70205e50 con 0x564d702f9060 2024-09-11T23:33:05.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.025+0000 7efe5eb79740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x564d7025a5e0 con 0x564d702f9980 2024-09-11T23:33:05.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.026+0000 7efe59d3b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564d70304240 0x564d70306630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:33:05.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.026+0000 7efe5ad3d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564d702f9060 0x564d702f9440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:33:05.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.026+0000 7efe5a53c640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564d702f9980 0x564d70303d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:33:05.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.026+0000 7efe59d3b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564d70304240 0x564d70306630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52696/0 (socket says 172.21.15.26:52696) 2024-09-11T23:33:05.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.026+0000 7efe5ad3d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564d702f9060 0x564d702f9440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:41442/0 (socket says 172.21.15.26:41442) 2024-09-11T23:33:05.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.026+0000 7efe59d3b640 1 -- 172.21.15.26:0/864248852 learned_addr learned my addr 172.21.15.26:0/864248852 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:33:05.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.026+0000 7efe4b7fe640 1 --2- 172.21.15.26:0/337480756 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7efe400a2d70 0x7efe400a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:05.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.026+0000 7efe5ad3d640 1 --2- 172.21.15.26:0/337480756 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7efe400a2d70 0x7efe400a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:33:05.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.026+0000 7efe59d3b640 1 -- 172.21.15.26:0/864248852 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564d702f9980 msgr2=0x564d70303d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:33:05.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.026+0000 7efe59d3b640 1 --2- 172.21.15.26:0/864248852 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564d702f9980 0x564d70303d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:05.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.026+0000 7efe59d3b640 1 -- 172.21.15.26:0/864248852 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564d702f9060 msgr2=0x564d702f9440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:33:05.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.026+0000 7efe59d3b640 1 --2- 172.21.15.26:0/864248852 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564d702f9060 0x564d702f9440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:05.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.026+0000 7efe59d3b640 1 -- 172.21.15.26:0/864248852 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d701f6d40 con 0x564d70304240 2024-09-11T23:33:05.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.026+0000 7efe4b7fe640 1 -- 172.21.15.26:0/337480756 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8542+0+0 (secure 0 0 0) 0x7efe500cf060 con 0x7efe54100a80 2024-09-11T23:33:05.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.026+0000 7efe59d3b640 1 --2- 172.21.15.26:0/864248852 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564d70304240 0x564d70306630 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7efe4402a940 tx=0x7efe44005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92355c8011c49e84 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:33:05.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.026+0000 7efe5ad3d640 1 --2- 172.21.15.26:0/337480756 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7efe400a2d70 0x7efe400a5230 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7efe4c009fd0 tx=0x7efe4c004ab0 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:33:05.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.026+0000 7efe48ff9640 1 -- 172.21.15.26:0/864248852 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe4403f070 con 0x564d70304240 2024-09-11T23:33:05.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.027+0000 7efe48ff9640 1 -- 172.21.15.26:0/864248852 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe44004030 con 0x564d70304240 2024-09-11T23:33:05.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.027+0000 7efe48ff9640 1 -- 172.21.15.26:0/864248852 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe44035070 con 0x564d70304240 2024-09-11T23:33:05.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.027+0000 7efe5eb79740 1 -- 172.21.15.26:0/864248852 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564d70304240 msgr2=0x564d70306630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:33:05.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.027+0000 7efe5eb79740 1 --2- 172.21.15.26:0/864248852 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564d70304240 0x564d70306630 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7efe4402a940 tx=0x7efe44005dd0 comp rx=0 tx=0).stop 2024-09-11T23:33:05.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.027+0000 7efe5eb79740 1 -- 172.21.15.26:0/864248852 shutdown_connections 2024-09-11T23:33:05.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.027+0000 7efe5eb79740 1 --2- 172.21.15.26:0/864248852 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564d70304240 0x564d70306630 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:05.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.027+0000 7efe5eb79740 1 --2- 172.21.15.26:0/864248852 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564d702f9980 0x564d70303d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:05.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.027+0000 7efe5eb79740 1 --2- 172.21.15.26:0/864248852 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564d702f9060 0x564d702f9440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:05.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.027+0000 7efe5eb79740 1 -- 172.21.15.26:0/864248852 >> 172.21.15.26:0/864248852 conn(0x564d702f55f0 msgr2=0x564d702f59d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:33:05.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.027+0000 7efe5eb79740 1 -- 172.21.15.26:0/864248852 shutdown_connections 2024-09-11T23:33:05.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.027+0000 7efe5eb79740 1 -- 172.21.15.26:0/864248852 wait complete. 2024-09-11T23:33:05.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.028+0000 7efe5eb79740 1 -- start start 2024-09-11T23:33:05.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.028+0000 7efe5eb79740 1 Processor -- start 2024-09-11T23:33:05.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.028+0000 7efe5eb79740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564d702f9060 0x564d70301170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:05.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.028+0000 7efe5eb79740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564d702f9980 0x564d703016b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:05.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.028+0000 7efe5eb79740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564d70304240 0x564d70301bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:05.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.028+0000 7efe5eb79740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x564d70306e10 con 0x564d702f9980 2024-09-11T23:33:05.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.028+0000 7efe5eb79740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x564d701e6f10 con 0x564d702f9060 2024-09-11T23:33:05.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.028+0000 7efe5eb79740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x564d70306f90 con 0x564d70304240 2024-09-11T23:33:05.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.028+0000 7efe5eb79740 10 client.0 _ll_register_callbacks cb 0x564d7020adb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:33:05.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.028+0000 7efe59d3b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564d702f9980 0x564d703016b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:33:05.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.028+0000 7efe59d3b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564d702f9980 0x564d703016b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52704/0 (socket says 172.21.15.26:52704) 2024-09-11T23:33:05.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.028+0000 7efe59d3b640 1 -- 172.21.15.26:0/3733433942 learned_addr learned my addr 172.21.15.26:0/3733433942 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:33:05.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.028+0000 7efe5ad3d640 1 --2- 172.21.15.26:0/3733433942 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564d70304240 0x564d70301bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:33:05.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.028+0000 7efe5a53c640 1 --2- 172.21.15.26:0/3733433942 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564d702f9060 0x564d70301170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:33:05.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.028+0000 7efe59d3b640 1 -- 172.21.15.26:0/3733433942 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564d70304240 msgr2=0x564d70301bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:33:05.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.028+0000 7efe59d3b640 1 --2- 172.21.15.26:0/3733433942 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564d70304240 0x564d70301bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:05.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.028+0000 7efe59d3b640 1 -- 172.21.15.26:0/3733433942 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564d702f9060 msgr2=0x564d70301170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:33:05.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.028+0000 7efe59d3b640 1 --2- 172.21.15.26:0/3733433942 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564d702f9060 0x564d70301170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:05.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.029+0000 7efe59d3b640 1 -- 172.21.15.26:0/3733433942 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d6ff6d580 con 0x564d702f9980 2024-09-11T23:33:05.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.029+0000 7efe5a53c640 1 --2- 172.21.15.26:0/3733433942 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564d702f9060 0x564d70301170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:33:05.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.029+0000 7efe59d3b640 1 --2- 172.21.15.26:0/3733433942 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564d702f9980 0x564d703016b0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7efe44032f90 tx=0x7efe4402aac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5790c25de34fa997 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:33:05.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.029+0000 7efe3dffb640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:33:05.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.029+0000 7efe3dffb640 1 -- 172.21.15.26:0/3733433942 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe44035070 con 0x564d702f9980 2024-09-11T23:33:05.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.029+0000 7efe3dffb640 1 -- 172.21.15.26:0/3733433942 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8542+0+0 (secure 0 0 0) 0x7efe440315b0 con 0x564d702f9980 2024-09-11T23:33:05.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.029+0000 7efe5eb79740 1 -- 172.21.15.26:0/3733433942 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efe44004030 con 0x564d702f9980 2024-09-11T23:33:05.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.029+0000 7efe3dffb640 1 -- 172.21.15.26:0/3733433942 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) ==== 24+0+0 (secure 0 0 0) 0x7efe44055050 con 0x564d702f9980 2024-09-11T23:33:05.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.029+0000 7efe5eb79740 10 client.17472 fetch_fsmap learned FSMap version 71 2024-09-11T23:33:05.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.029+0000 7efe5eb79740 1 -- 172.21.15.26:0/3733433942 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=4+}) -- 0x564d70037960 con 0x564d702f9980 2024-09-11T23:33:05.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.029+0000 7efe3dffb640 1 -- 172.21.15.26:0/3733433942 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe44004030 con 0x564d702f9980 2024-09-11T23:33:05.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.029+0000 7efe3dffb640 1 -- 172.21.15.26:0/3733433942 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 71) ==== 8430+0+0 (secure 0 0 0) 0x7efe440046c0 con 0x564d702f9980 2024-09-11T23:33:05.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.029+0000 7efe5eb79740 10 client.17472 fetch_fsmap finished waiting for FSMap version 71 2024-09-11T23:33:05.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.029+0000 7efe5eb79740 10 client.17472 resolve_mds: validated gid 24485 aka daemon mds.b 2024-09-11T23:33:05.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.030+0000 7efe5eb79740 20 client.17472 populate_metadata read hostname 'smithi026' 2024-09-11T23:33:05.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.030+0000 7efe5eb79740 1 --2- 172.21.15.26:0/3733433942 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x564d70313c90 0x564d70316080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:05.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.030+0000 7efe5eb79740 4 client.17472 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:33:05.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.030+0000 7efe5eb79740 1 -- 172.21.15.26:0/3733433942 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564d7026bb90 con 0x564d70313c90 2024-09-11T23:33:05.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.030+0000 7efe5a53c640 1 --2- 172.21.15.26:0/3733433942 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x564d70313c90 0x564d70316080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:33:05.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.030+0000 7efe3dffb640 10 client.17472 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:33:05.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.030+0000 7efe5a53c640 1 --2- 172.21.15.26:0/3733433942 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x564d70313c90 0x564d70316080 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=53928fd1a7f72361 server_cookie=7e1980b9003faa11 in_seq=0 out_seq=0 2024-09-11T23:33:05.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.031+0000 7efe3dffb640 1 -- 172.21.15.26:0/3733433942 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564d7026bb90 con 0x564d70313c90 2024-09-11T23:33:05.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.031+0000 7efe3dffb640 10 client.17472 handle_command_reply: tid=0 2024-09-11T23:33:05.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.031+0000 7efe5eb79740 1 client.17472 shutdown 2024-09-11T23:33:05.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.031+0000 7efe5eb79740 1 -- 172.21.15.26:0/3733433942 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564d702f9980 msgr2=0x564d703016b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:33:05.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.031+0000 7efe5eb79740 1 --2- 172.21.15.26:0/3733433942 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564d702f9980 0x564d703016b0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7efe44032f90 tx=0x7efe4402aac0 comp rx=0 tx=0).stop 2024-09-11T23:33:05.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.031+0000 7efe5eb79740 1 -- 172.21.15.26:0/3733433942 shutdown_connections 2024-09-11T23:33:05.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.031+0000 7efe5eb79740 1 --2- 172.21.15.26:0/3733433942 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x564d70313c90 0x564d70316080 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:05.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.031+0000 7efe5eb79740 1 --2- 172.21.15.26:0/3733433942 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564d70304240 0x564d70301bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:05.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.031+0000 7efe5eb79740 1 --2- 172.21.15.26:0/3733433942 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564d702f9980 0x564d703016b0 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:05.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.031+0000 7efe5eb79740 1 --2- 172.21.15.26:0/3733433942 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564d702f9060 0x564d70301170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:05.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.031+0000 7efe3dffb640 0 client.17472 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:33:05.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.031+0000 7efe5eb79740 1 -- 172.21.15.26:0/3733433942 >> 172.21.15.26:0/3733433942 conn(0x564d702f55f0 msgr2=0x564d702fb2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:33:05.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.031+0000 7efe5eb79740 1 -- 172.21.15.26:0/3733433942 shutdown_connections 2024-09-11T23:33:05.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.032+0000 7efe5eb79740 1 -- 172.21.15.26:0/3733433942 wait complete. 2024-09-11T23:33:05.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.032+0000 7efe5eb79740 20 client.17472 trim_cache size 0 max 16384 2024-09-11T23:33:05.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.049+0000 7efe5eb79740 1 Processor -- start 2024-09-11T23:33:05.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.049+0000 7efe5eb79740 1 -- start start 2024-09-11T23:33:05.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.049+0000 7efe5eb79740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564d702f9060 0x564d702f5c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:05.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.049+0000 7efe5eb79740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564d702f9980 0x564d702fb1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:05.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.049+0000 7efe5eb79740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564d70304240 0x564d7031b620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:05.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.049+0000 7efe5eb79740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7efe541ad8a0 con 0x564d70304240 2024-09-11T23:33:05.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.049+0000 7efe5eb79740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7efe541af650 con 0x564d702f9060 2024-09-11T23:33:05.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.049+0000 7efe5eb79740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7efe541adfa0 con 0x564d702f9980 2024-09-11T23:33:05.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.049+0000 7efe5ad3d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564d70304240 0x564d7031b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:33:05.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.049+0000 7efe5ad3d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564d70304240 0x564d7031b620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52712/0 (socket says 172.21.15.26:52712) 2024-09-11T23:33:05.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.049+0000 7efe5ad3d640 1 -- 172.21.15.26:0/308295746 learned_addr learned my addr 172.21.15.26:0/308295746 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:33:05.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.049+0000 7efe5a53c640 1 --2- 172.21.15.26:0/308295746 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564d702f9060 0x564d702f5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:33:05.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.049+0000 7efe59d3b640 1 --2- 172.21.15.26:0/308295746 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564d702f9980 0x564d702fb1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:33:05.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.049+0000 7efe5ad3d640 1 -- 172.21.15.26:0/308295746 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564d702f9980 msgr2=0x564d702fb1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:33:05.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.049+0000 7efe5ad3d640 1 --2- 172.21.15.26:0/308295746 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564d702f9980 0x564d702fb1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:05.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.049+0000 7efe5ad3d640 1 -- 172.21.15.26:0/308295746 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564d702f9060 msgr2=0x564d702f5c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:33:05.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.049+0000 7efe5ad3d640 1 --2- 172.21.15.26:0/308295746 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564d702f9060 0x564d702f5c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:05.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.049+0000 7efe5ad3d640 1 -- 172.21.15.26:0/308295746 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d703030b0 con 0x564d70304240 2024-09-11T23:33:05.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.049+0000 7efe5ad3d640 1 --2- 172.21.15.26:0/308295746 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564d70304240 0x564d7031b620 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7efe4c014250 tx=0x7efe4c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb0a93d99aba6a21 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:33:05.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.050+0000 7efe3ffff640 1 -- 172.21.15.26:0/308295746 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe4c01a070 con 0x564d70304240 2024-09-11T23:33:05.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.050+0000 7efe3ffff640 1 -- 172.21.15.26:0/308295746 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe4c02a070 con 0x564d70304240 2024-09-11T23:33:05.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.050+0000 7efe3ffff640 1 -- 172.21.15.26:0/308295746 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe4c01d040 con 0x564d70304240 2024-09-11T23:33:05.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.050+0000 7efe5eb79740 1 -- 172.21.15.26:0/308295746 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564d70304240 msgr2=0x564d7031b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:33:05.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.050+0000 7efe5eb79740 1 --2- 172.21.15.26:0/308295746 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564d70304240 0x564d7031b620 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7efe4c014250 tx=0x7efe4c009c00 comp rx=0 tx=0).stop 2024-09-11T23:33:05.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.050+0000 7efe5eb79740 1 -- 172.21.15.26:0/308295746 shutdown_connections 2024-09-11T23:33:05.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.050+0000 7efe5eb79740 1 --2- 172.21.15.26:0/308295746 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564d70304240 0x564d7031b620 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:05.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.050+0000 7efe5eb79740 1 --2- 172.21.15.26:0/308295746 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564d702f9980 0x564d702fb1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:05.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.050+0000 7efe5eb79740 1 --2- 172.21.15.26:0/308295746 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564d702f9060 0x564d702f5c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:05.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.050+0000 7efe5eb79740 1 -- 172.21.15.26:0/308295746 >> 172.21.15.26:0/308295746 conn(0x564d702f55f0 msgr2=0x564d702faab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:33:05.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.050+0000 7efe5eb79740 1 -- 172.21.15.26:0/308295746 shutdown_connections 2024-09-11T23:33:05.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.050+0000 7efe5eb79740 1 -- 172.21.15.26:0/308295746 wait complete. 2024-09-11T23:33:05.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.051+0000 7efe5eb79740 1 -- start start 2024-09-11T23:33:05.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.051+0000 7efe5eb79740 1 Processor -- start 2024-09-11T23:33:05.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.051+0000 7efe5eb79740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564d702f9060 0x564d70308740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:05.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.051+0000 7efe5eb79740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564d702f9980 0x564d70308c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:05.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.051+0000 7efe5eb79740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564d70304240 0x564d7037d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:05.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.051+0000 7efe5eb79740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7efe541aeec0 con 0x564d70304240 2024-09-11T23:33:05.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.051+0000 7efe5eb79740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7efe541ae730 con 0x564d702f9060 2024-09-11T23:33:05.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.051+0000 7efe5eb79740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7efe541ab8a0 con 0x564d702f9980 2024-09-11T23:33:05.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.051+0000 7efe5eb79740 10 client.0 _ll_register_callbacks cb 0x7efe44009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:33:05.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.051+0000 7efe5ad3d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564d70304240 0x564d7037d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:33:05.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.051+0000 7efe5ad3d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564d70304240 0x564d7037d5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52716/0 (socket says 172.21.15.26:52716) 2024-09-11T23:33:05.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.051+0000 7efe5ad3d640 1 -- 172.21.15.26:0/4100005032 learned_addr learned my addr 172.21.15.26:0/4100005032 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:33:05.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.051+0000 7efe5a53c640 1 --2- 172.21.15.26:0/4100005032 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564d702f9060 0x564d70308740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:33:05.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.051+0000 7efe59d3b640 1 --2- 172.21.15.26:0/4100005032 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564d702f9980 0x564d70308c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:33:05.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.051+0000 7efe5ad3d640 1 -- 172.21.15.26:0/4100005032 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564d702f9980 msgr2=0x564d70308c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:33:05.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.051+0000 7efe5ad3d640 1 --2- 172.21.15.26:0/4100005032 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564d702f9980 0x564d70308c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:05.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.051+0000 7efe5ad3d640 1 -- 172.21.15.26:0/4100005032 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564d702f9060 msgr2=0x564d70308740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:33:05.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.051+0000 7efe5ad3d640 1 --2- 172.21.15.26:0/4100005032 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564d702f9060 0x564d70308740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:05.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.051+0000 7efe5ad3d640 1 -- 172.21.15.26:0/4100005032 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d702f97b0 con 0x564d70304240 2024-09-11T23:33:05.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.051+0000 7efe59d3b640 1 --2- 172.21.15.26:0/4100005032 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564d702f9980 0x564d70308c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:33:05.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.052+0000 7efe5ad3d640 1 --2- 172.21.15.26:0/4100005032 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564d70304240 0x564d7037d5f0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7efe4c0256a0 tx=0x7efe4c025df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62907560e9d3fcbd server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:33:05.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.052+0000 7efe3dffb640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:33:05.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.052+0000 7efe3dffb640 1 -- 172.21.15.26:0/4100005032 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe4c01a040 con 0x564d70304240 2024-09-11T23:33:05.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.052+0000 7efe3dffb640 1 -- 172.21.15.26:0/4100005032 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8542+0+0 (secure 0 0 0) 0x7efe4c004b20 con 0x564d70304240 2024-09-11T23:33:05.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.052+0000 7efe5eb79740 1 -- 172.21.15.26:0/4100005032 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efe4c02a070 con 0x564d70304240 2024-09-11T23:33:05.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.052+0000 7efe3dffb640 1 -- 172.21.15.26:0/4100005032 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) ==== 24+0+0 (secure 0 0 0) 0x7efe4c03e050 con 0x564d70304240 2024-09-11T23:33:05.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.052+0000 7efe5eb79740 10 client.17478 fetch_fsmap learned FSMap version 71 2024-09-11T23:33:05.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.052+0000 7efe5eb79740 1 -- 172.21.15.26:0/4100005032 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=4+}) -- 0x564d7037ed60 con 0x564d70304240 2024-09-11T23:33:05.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.052+0000 7efe3dffb640 1 -- 172.21.15.26:0/4100005032 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe4c02a070 con 0x564d70304240 2024-09-11T23:33:05.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.052+0000 7efe3dffb640 1 -- 172.21.15.26:0/4100005032 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 71) ==== 8430+0+0 (secure 0 0 0) 0x7efe4c023040 con 0x564d70304240 2024-09-11T23:33:05.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.052+0000 7efe5eb79740 10 client.17478 fetch_fsmap finished waiting for FSMap version 71 2024-09-11T23:33:05.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.052+0000 7efe5eb79740 10 client.17478 resolve_mds: validated gid 24485 aka daemon mds.b 2024-09-11T23:33:05.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.052+0000 7efe5eb79740 20 client.17478 populate_metadata read hostname 'smithi026' 2024-09-11T23:33:05.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.052+0000 7efe5eb79740 1 --2- 172.21.15.26:0/4100005032 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x564d7037f4b0 0x564d703818d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:05.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.052+0000 7efe5eb79740 4 client.17478 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "95c56ce2", "release": true, "await": true}] 2024-09-11T23:33:05.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.052+0000 7efe5eb79740 1 -- 172.21.15.26:0/4100005032 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "quiesce db", "set_id": "95c56ce2", "release": true, "await": true}) -- 0x564d702660a0 con 0x564d7037f4b0 2024-09-11T23:33:05.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.053+0000 7efe5a53c640 1 --2- 172.21.15.26:0/4100005032 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x564d7037f4b0 0x564d703818d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:33:05.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.053+0000 7efe3dffb640 10 client.17478 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:33:05.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:05.053+0000 7efe5a53c640 1 --2- 172.21.15.26:0/4100005032 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x564d7037f4b0 0x564d703818d0 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c3c0165fe1150c0 server_cookie=bb6cac35498bf568 in_seq=0 out_seq=0 2024-09-11T23:33:06.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:06.073+0000 7efe3dffb640 1 -- 172.21.15.26:0/4100005032 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x564d702660a0 con 0x564d7037f4b0 2024-09-11T23:33:06.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:06.073+0000 7efe3dffb640 10 client.17478 handle_command_reply: tid=0 2024-09-11T23:33:06.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:06.073+0000 7efe5eb79740 1 client.17478 shutdown 2024-09-11T23:33:06.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:06.073+0000 7efe5eb79740 1 -- 172.21.15.26:0/4100005032 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564d70304240 msgr2=0x564d7037d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:33:06.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:06.073+0000 7efe5eb79740 1 --2- 172.21.15.26:0/4100005032 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564d70304240 0x564d7037d5f0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7efe4c0256a0 tx=0x7efe4c025df0 comp rx=0 tx=0).stop 2024-09-11T23:33:06.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:06.073+0000 7efe5eb79740 1 -- 172.21.15.26:0/4100005032 shutdown_connections 2024-09-11T23:33:06.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:06.073+0000 7efe5eb79740 1 --2- 172.21.15.26:0/4100005032 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x564d7037f4b0 0x564d703818d0 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:06.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:06.073+0000 7efe3dffb640 0 client.17478 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:33:06.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:06.073+0000 7efe5eb79740 1 --2- 172.21.15.26:0/4100005032 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564d70304240 0x564d7037d5f0 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:06.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:06.073+0000 7efe5eb79740 1 --2- 172.21.15.26:0/4100005032 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564d702f9980 0x564d70308c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:06.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:06.073+0000 7efe5eb79740 1 --2- 172.21.15.26:0/4100005032 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564d702f9060 0x564d70308740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:06.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:06.073+0000 7efe5eb79740 1 -- 172.21.15.26:0/4100005032 >> 172.21.15.26:0/4100005032 conn(0x564d702f55f0 msgr2=0x564d703018c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:33:06.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:06.074+0000 7efe5eb79740 1 -- 172.21.15.26:0/4100005032 shutdown_connections 2024-09-11T23:33:06.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:06.074+0000 7efe5eb79740 1 -- 172.21.15.26:0/4100005032 wait complete. 2024-09-11T23:33:06.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:06.074+0000 7efe5eb79740 20 client.17478 trim_cache size 0 max 16384 2024-09-11T23:33:06.075 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-11T23:33:06.075 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 71, 2024-09-11T23:33:06.075 INFO:teuthology.orchestra.run.smithi026.stdout: "leader": 24485, 2024-09-11T23:33:06.075 INFO:teuthology.orchestra.run.smithi026.stdout: "set_version": 52, 2024-09-11T23:33:06.076 INFO:teuthology.orchestra.run.smithi026.stdout: "sets": { 2024-09-11T23:33:06.076 INFO:teuthology.orchestra.run.smithi026.stdout: "95c56ce2": { 2024-09-11T23:33:06.076 INFO:teuthology.orchestra.run.smithi026.stdout: "version": 52, 2024-09-11T23:33:06.076 INFO:teuthology.orchestra.run.smithi026.stdout: "age_ref": 0.0, 2024-09-11T23:33:06.076 INFO:teuthology.orchestra.run.smithi026.stdout: "state": { 2024-09-11T23:33:06.076 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "RELEASED", 2024-09-11T23:33:06.076 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.0 2024-09-11T23:33:06.076 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:33:06.077 INFO:teuthology.orchestra.run.smithi026.stdout: "timeout": 90.0, 2024-09-11T23:33:06.077 INFO:teuthology.orchestra.run.smithi026.stdout: "expiration": 150.0, 2024-09-11T23:33:06.077 INFO:teuthology.orchestra.run.smithi026.stdout: "members": { 2024-09-11T23:33:06.077 INFO:teuthology.orchestra.run.smithi026.stdout: "file:/": { 2024-09-11T23:33:06.077 INFO:teuthology.orchestra.run.smithi026.stdout: "excluded": false, 2024-09-11T23:33:06.077 INFO:teuthology.orchestra.run.smithi026.stdout: "state": { 2024-09-11T23:33:06.077 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "RELEASED", 2024-09-11T23:33:06.077 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.0 2024-09-11T23:33:06.077 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:33:06.077 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:33:06.077 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:33:06.077 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:33:06.078 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:33:06.078 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-11T23:33:06.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:06.075+0000 7efe5c7a0640 1 -- 172.21.15.26:0/337480756 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7efe400a2d70 msgr2=0x7efe400a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:33:06.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:06.075+0000 7efe5c7a0640 1 --2- 172.21.15.26:0/337480756 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7efe400a2d70 0x7efe400a5230 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7efe4c009fd0 tx=0x7efe4c004ab0 comp rx=0 tx=0).stop 2024-09-11T23:33:06.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:06.075+0000 7efe5c7a0640 1 -- 172.21.15.26:0/337480756 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efe54100a80 msgr2=0x7efe541992b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:33:06.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:06.075+0000 7efe5c7a0640 1 --2- 172.21.15.26:0/337480756 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efe54100a80 0x7efe541992b0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7efe5000d950 tx=0x7efe5000de10 comp rx=0 tx=0).stop 2024-09-11T23:33:06.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:06.075+0000 7efe5c7a0640 1 -- 172.21.15.26:0/337480756 shutdown_connections 2024-09-11T23:33:06.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:06.075+0000 7efe5c7a0640 1 --2- 172.21.15.26:0/337480756 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7efe400a2d70 0x7efe400a5230 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:06.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:06.075+0000 7efe5c7a0640 1 --2- 172.21.15.26:0/337480756 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efe54101420 0x7efe541997f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:06.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:06.075+0000 7efe5c7a0640 1 --2- 172.21.15.26:0/337480756 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efe54106a80 0x7efe541a4330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:06.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:06.075+0000 7efe5c7a0640 1 --2- 172.21.15.26:0/337480756 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efe54100a80 0x7efe541992b0 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:06.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:06.075+0000 7efe5c7a0640 1 -- 172.21.15.26:0/337480756 >> 172.21.15.26:0/337480756 conn(0x7efe540fc7a0 msgr2=0x7efe540fcef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:33:06.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:06.075+0000 7efe5c7a0640 1 -- 172.21.15.26:0/337480756 shutdown_connections 2024-09-11T23:33:06.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:06.075+0000 7efe5c7a0640 1 -- 172.21.15.26:0/337480756 wait complete. 2024-09-11T23:33:06.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:33:06 smithi026 ceph-mon[31172]: pgmap v460: 97 pgs: 97 active+clean; 2.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 38 KiB/s wr, 17 op/s 2024-09-11T23:33:06.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:33:06 smithi136 ceph-mon[35121]: pgmap v460: 97 pgs: 97 active+clean; 2.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 38 KiB/s wr, 17 op/s 2024-09-11T23:33:06.552 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:33:06 smithi076 ceph-mon[35296]: pgmap v460: 97 pgs: 97 active+clean; 2.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 38 KiB/s wr, 17 op/s 2024-09-11T23:33:06.982 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '95c56ce2', total seconds elapsed: 34.5 2024-09-11T23:33:06.982 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 30.0 2024-09-11T23:33:06.982 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:33:07.303 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:33:07.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:07.834+0000 7fcaa850d640 1 -- 172.21.15.26:0/596053626 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcaa0074ee0 msgr2=0x7fcaa010dee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:33:07.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:07.834+0000 7fcaa850d640 1 --2- 172.21.15.26:0/596053626 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcaa0074ee0 0x7fcaa010dee0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7fca90009900 tx=0x7fca9002a480 comp rx=0 tx=0).stop 2024-09-11T23:33:07.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:07.834+0000 7fcaa850d640 1 -- 172.21.15.26:0/596053626 shutdown_connections 2024-09-11T23:33:07.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:07.834+0000 7fcaa850d640 1 --2- 172.21.15.26:0/596053626 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcaa010e420 0x7fcaa0110810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:07.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:07.834+0000 7fcaa850d640 1 --2- 172.21.15.26:0/596053626 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcaa0074ee0 0x7fcaa010dee0 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:07.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:07.834+0000 7fcaa850d640 1 --2- 172.21.15.26:0/596053626 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcaa00745c0 0x7fcaa00749a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:07.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:07.834+0000 7fcaa850d640 1 -- 172.21.15.26:0/596053626 >> 172.21.15.26:0/596053626 conn(0x7fcaa00fd8c0 msgr2=0x7fcaa00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:33:07.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:07.834+0000 7fcaa850d640 1 -- 172.21.15.26:0/596053626 shutdown_connections 2024-09-11T23:33:07.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:07.834+0000 7fcaa850d640 1 -- 172.21.15.26:0/596053626 wait complete. 2024-09-11T23:33:07.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:07.835+0000 7fcaa850d640 1 Processor -- start 2024-09-11T23:33:07.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:07.835+0000 7fcaa850d640 1 -- start start 2024-09-11T23:33:07.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:07.835+0000 7fcaa850d640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcaa00745c0 0x7fcaa019a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:07.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:07.835+0000 7fcaa850d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcaa0074ee0 0x7fcaa019aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:07.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:07.835+0000 7fcaa850d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcaa010e420 0x7fcaa019ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:07.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:07.835+0000 7fcaa850d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fcaa01141c0 con 0x7fcaa0074ee0 2024-09-11T23:33:07.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:07.835+0000 7fcaa850d640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fcaa0114040 con 0x7fcaa010e420 2024-09-11T23:33:07.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:07.835+0000 7fcaa850d640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fcaa0114340 con 0x7fcaa00745c0 2024-09-11T23:33:07.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:07.836+0000 7fcaa5aa8640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcaa0074ee0 0x7fcaa019aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:33:07.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:07.836+0000 7fcaa5aa8640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcaa0074ee0 0x7fcaa019aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52724/0 (socket says 172.21.15.26:52724) 2024-09-11T23:33:07.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:07.836+0000 7fcaa5aa8640 1 -- 172.21.15.26:0/3453233572 learned_addr learned my addr 172.21.15.26:0/3453233572 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:33:07.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:07.836+0000 7fcaa6aaa640 1 --2- 172.21.15.26:0/3453233572 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcaa010e420 0x7fcaa019ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:33:07.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:07.836+0000 7fcaa62a9640 1 --2- 172.21.15.26:0/3453233572 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcaa00745c0 0x7fcaa019a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:33:07.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:07.836+0000 7fcaa5aa8640 1 -- 172.21.15.26:0/3453233572 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcaa00745c0 msgr2=0x7fcaa019a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:33:07.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:07.836+0000 7fcaa5aa8640 1 --2- 172.21.15.26:0/3453233572 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcaa00745c0 0x7fcaa019a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:07.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:07.836+0000 7fcaa5aa8640 1 -- 172.21.15.26:0/3453233572 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcaa010e420 msgr2=0x7fcaa019ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:33:07.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:07.836+0000 7fcaa5aa8640 1 --2- 172.21.15.26:0/3453233572 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcaa010e420 0x7fcaa019ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:07.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:07.836+0000 7fcaa5aa8640 1 -- 172.21.15.26:0/3453233572 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcaa019f610 con 0x7fcaa0074ee0 2024-09-11T23:33:07.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:07.836+0000 7fcaa62a9640 1 --2- 172.21.15.26:0/3453233572 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcaa00745c0 0x7fcaa019a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:33:07.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:07.836+0000 7fcaa5aa8640 1 --2- 172.21.15.26:0/3453233572 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcaa0074ee0 0x7fcaa019aba0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7fca90004770 tx=0x7fca90003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:33:07.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:07.836+0000 7fca977fe640 1 -- 172.21.15.26:0/3453233572 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca9003f070 con 0x7fcaa0074ee0 2024-09-11T23:33:07.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:07.836+0000 7fca977fe640 1 -- 172.21.15.26:0/3453233572 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca90031ca0 con 0x7fcaa0074ee0 2024-09-11T23:33:07.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:07.836+0000 7fcaa850d640 1 -- 172.21.15.26:0/3453233572 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcaa019f8a0 con 0x7fcaa0074ee0 2024-09-11T23:33:07.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:07.836+0000 7fca977fe640 1 -- 172.21.15.26:0/3453233572 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca90035030 con 0x7fcaa0074ee0 2024-09-11T23:33:07.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:07.837+0000 7fcaa850d640 1 -- 172.21.15.26:0/3453233572 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcaa01a71e0 con 0x7fcaa0074ee0 2024-09-11T23:33:07.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:07.838+0000 7fcaa850d640 1 -- 172.21.15.26:0/3453233572 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcaa0075c70 con 0x7fcaa0074ee0 2024-09-11T23:33:07.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:07.839+0000 7fca977fe640 1 -- 172.21.15.26:0/3453233572 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fca90042070 con 0x7fcaa0074ee0 2024-09-11T23:33:07.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:07.839+0000 7fca977fe640 1 --2- 172.21.15.26:0/3453233572 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fca8c0a2d70 0x7fca8c0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:07.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:07.840+0000 7fcaa62a9640 1 --2- 172.21.15.26:0/3453233572 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fca8c0a2d70 0x7fca8c0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:33:07.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:07.840+0000 7fca977fe640 1 -- 172.21.15.26:0/3453233572 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8542+0+0 (secure 0 0 0) 0x7fca900efb10 con 0x7fcaa0074ee0 2024-09-11T23:33:07.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:07.840+0000 7fcaa62a9640 1 --2- 172.21.15.26:0/3453233572 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fca8c0a2d70 0x7fca8c0a5230 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fca9c009c30 tx=0x7fca9c005f90 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:33:07.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:07.844+0000 7fca977fe640 1 -- 172.21.15.26:0/3453233572 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fca900bc720 con 0x7fcaa0074ee0 2024-09-11T23:33:08.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:08.068+0000 7fcaa850d640 1 -- 172.21.15.26:0/3453233572 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcaa01095d0 con 0x7fcaa0074ee0 2024-09-11T23:33:08.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:08.068+0000 7fca977fe640 1 -- 172.21.15.26:0/3453233572 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) ==== 94+0+12001 (secure 0 0 0) 0x7fca900dc0e0 con 0x7fcaa0074ee0 2024-09-11T23:33:08.068 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-11T23:33:08.068 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":71,"btime":"2024-09-11T23:32:43:090819+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14499,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/367659515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":367659515},{"type":"v1","addr":"172.21.15.26:6835","nonce":367659515}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14511,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/2041065299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":2041065299},{"type":"v1","addr":"172.21.15.26:6837","nonce":2041065299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":71,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:16:26.544203+0000","modified":"2024-09-11T23:32:42.846060+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24485,"mds_1":24463,"mds_2":24458,"mds_3":24448,"mds_4":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14550":{"gid":14550,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6839/2233068305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":2233068305},{"type":"v1","addr":"172.21.15.26:6839","nonce":2233068305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14562":{"gid":14562,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6841/2984982336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":2984982336},{"type":"v1","addr":"172.21.15.26:6841","nonce":2984982336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6833/2019841455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":2019841455},{"type":"v1","addr":"172.21.15.136:6833","nonce":2019841455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.76:6833/3438147050","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3438147050},{"type":"v1","addr":"172.21.15.76:6833","nonce":3438147050}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6835/3617665975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3617665975},{"type":"v1","addr":"172.21.15.136:6835","nonce":3617665975}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":12,"addr":"172.21.15.76:6835/393308499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":393308499},{"type":"v1","addr":"172.21.15.76:6835","nonce":393308499}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":10,"addr":"172.21.15.136:6837/1633082387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":1633082387},{"type":"v1","addr":"172.21.15.136:6837","nonce":1633082387}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.76:6837/2078978748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":2078978748},{"type":"v1","addr":"172.21.15.76:6837","nonce":2078978748}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":63,"addr":"172.21.15.136:6839/3211398630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":3211398630},{"type":"v1","addr":"172.21.15.136:6839","nonce":3211398630}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.76:6839/2077637090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6838","nonce":2077637090},{"type":"v1","addr":"172.21.15.76:6839","nonce":2077637090}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24485,"qdb_cluster":[24443,24448,24458,24463,24485]},"id":1}]} 2024-09-11T23:33:08.069 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 71 2024-09-11T23:33:08.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:08.070+0000 7fcaa850d640 1 -- 172.21.15.26:0/3453233572 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fca8c0a2d70 msgr2=0x7fca8c0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:33:08.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:08.070+0000 7fcaa850d640 1 --2- 172.21.15.26:0/3453233572 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fca8c0a2d70 0x7fca8c0a5230 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fca9c009c30 tx=0x7fca9c005f90 comp rx=0 tx=0).stop 2024-09-11T23:33:08.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:08.070+0000 7fcaa850d640 1 -- 172.21.15.26:0/3453233572 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcaa0074ee0 msgr2=0x7fcaa019aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:33:08.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:08.070+0000 7fcaa850d640 1 --2- 172.21.15.26:0/3453233572 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcaa0074ee0 0x7fcaa019aba0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7fca90004770 tx=0x7fca90003c10 comp rx=0 tx=0).stop 2024-09-11T23:33:08.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:08.070+0000 7fcaa850d640 1 -- 172.21.15.26:0/3453233572 shutdown_connections 2024-09-11T23:33:08.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:08.070+0000 7fcaa850d640 1 --2- 172.21.15.26:0/3453233572 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fca8c0a2d70 0x7fca8c0a5230 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:08.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:08.070+0000 7fcaa850d640 1 --2- 172.21.15.26:0/3453233572 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcaa010e420 0x7fcaa019ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:08.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:08.070+0000 7fcaa850d640 1 --2- 172.21.15.26:0/3453233572 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcaa0074ee0 0x7fcaa019aba0 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:08.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:08.070+0000 7fcaa850d640 1 --2- 172.21.15.26:0/3453233572 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcaa00745c0 0x7fcaa019a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:08.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:08.070+0000 7fcaa850d640 1 -- 172.21.15.26:0/3453233572 >> 172.21.15.26:0/3453233572 conn(0x7fcaa00fd8c0 msgr2=0x7fcaa00fe520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:33:08.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:08.071+0000 7fcaa850d640 1 -- 172.21.15.26:0/3453233572 shutdown_connections 2024-09-11T23:33:08.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:08.071+0000 7fcaa850d640 1 -- 172.21.15.26:0/3453233572 wait complete. 2024-09-11T23:33:08.474 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:33:08 smithi026 ceph-mon[31172]: pgmap v461: 97 pgs: 97 active+clean; 2.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 57 KiB/s wr, 33 op/s 2024-09-11T23:33:08.474 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:33:08 smithi026 ceph-mon[31172]: from='client.? 172.21.15.26:0/3453233572' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:33:08.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:33:08 smithi136 ceph-mon[35121]: pgmap v461: 97 pgs: 97 active+clean; 2.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 57 KiB/s wr, 33 op/s 2024-09-11T23:33:08.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:33:08 smithi136 ceph-mon[35121]: from='client.? 172.21.15.26:0/3453233572' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:33:08.552 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:33:08 smithi076 ceph-mon[35296]: pgmap v461: 97 pgs: 97 active+clean; 2.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 57 KiB/s wr, 33 op/s 2024-09-11T23:33:08.552 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:33:08 smithi076 ceph-mon[35296]: from='client.? 172.21.15.26:0/3453233572' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:33:08.834 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24485 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0' 2024-09-11T23:33:08.834 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24485 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0 2024-09-11T23:33:09.099 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:33:09.390 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:33:09 smithi026 ceph-mon[31172]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:33:09.390 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:33:09 smithi026 ceph-mon[31172]: pgmap v462: 97 pgs: 97 active+clean; 2.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 57 KiB/s wr, 30 op/s 2024-09-11T23:33:09.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:33:09 smithi136 ceph-mon[35121]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:33:09.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:33:09 smithi136 ceph-mon[35121]: pgmap v462: 97 pgs: 97 active+clean; 2.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 57 KiB/s wr, 30 op/s 2024-09-11T23:33:09.552 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:33:09 smithi076 ceph-mon[35296]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:33:09.552 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:33:09 smithi076 ceph-mon[35296]: pgmap v462: 97 pgs: 97 active+clean; 2.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 57 KiB/s wr, 30 op/s 2024-09-11T23:33:09.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.848+0000 7fe6f9ce1640 1 -- 172.21.15.26:0/2284657633 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe6f4109450 msgr2=0x7fe6f4109830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:33:09.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.848+0000 7fe6f9ce1640 1 --2- 172.21.15.26:0/2284657633 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe6f4109450 0x7fe6f4109830 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fe6e8009860 tx=0x7fe6e802a340 comp rx=0 tx=0).stop 2024-09-11T23:33:09.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.848+0000 7fe6f9ce1640 1 -- 172.21.15.26:0/2284657633 shutdown_connections 2024-09-11T23:33:09.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.848+0000 7fe6f9ce1640 1 --2- 172.21.15.26:0/2284657633 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe6f4109e00 0x7fe6f410e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:09.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.848+0000 7fe6f9ce1640 1 --2- 172.21.15.26:0/2284657633 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe6f410ea10 0x7fe6f4115360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:09.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.848+0000 7fe6f9ce1640 1 --2- 172.21.15.26:0/2284657633 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe6f4109450 0x7fe6f4109830 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:09.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.848+0000 7fe6f9ce1640 1 -- 172.21.15.26:0/2284657633 >> 172.21.15.26:0/2284657633 conn(0x7fe6f4073660 msgr2=0x7fe6f4073a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:33:09.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.848+0000 7fe6f9ce1640 1 -- 172.21.15.26:0/2284657633 shutdown_connections 2024-09-11T23:33:09.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.848+0000 7fe6f9ce1640 1 -- 172.21.15.26:0/2284657633 wait complete. 2024-09-11T23:33:09.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.849+0000 7fe6f9ce1640 1 Processor -- start 2024-09-11T23:33:09.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.849+0000 7fe6f9ce1640 1 -- start start 2024-09-11T23:33:09.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.850+0000 7fe6f9ce1640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe6f4109450 0x7fe6f41a7700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:09.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.850+0000 7fe6f9ce1640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe6f4109e00 0x7fe6f41a7c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:09.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.850+0000 7fe6f9ce1640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe6f410ea10 0x7fe6f4114590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:09.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.850+0000 7fe6f9ce1640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fe6f411a5c0 con 0x7fe6f4109e00 2024-09-11T23:33:09.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.850+0000 7fe6f9ce1640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fe6f411a440 con 0x7fe6f4109450 2024-09-11T23:33:09.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.850+0000 7fe6f9ce1640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe6f411a740 con 0x7fe6f410ea10 2024-09-11T23:33:09.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.850+0000 7fe6f3fff640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe6f410ea10 0x7fe6f4114590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:33:09.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.850+0000 7fe6f3fff640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe6f410ea10 0x7fe6f4114590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:50462/0 (socket says 172.21.15.26:50462) 2024-09-11T23:33:09.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.850+0000 7fe6f3fff640 1 -- 172.21.15.26:0/1763176668 learned_addr learned my addr 172.21.15.26:0/1763176668 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:33:09.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.850+0000 7fe6f37fe640 1 --2- 172.21.15.26:0/1763176668 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe6f4109450 0x7fe6f41a7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:33:09.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.851+0000 7fe6f2ffd640 1 --2- 172.21.15.26:0/1763176668 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe6f4109e00 0x7fe6f41a7c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:33:09.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.851+0000 7fe6f37fe640 1 -- 172.21.15.26:0/1763176668 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe6f410ea10 msgr2=0x7fe6f4114590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:33:09.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.851+0000 7fe6f37fe640 1 --2- 172.21.15.26:0/1763176668 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe6f410ea10 0x7fe6f4114590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:09.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.851+0000 7fe6f37fe640 1 -- 172.21.15.26:0/1763176668 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe6f4109e00 msgr2=0x7fe6f41a7c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:33:09.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.851+0000 7fe6f37fe640 1 --2- 172.21.15.26:0/1763176668 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe6f4109e00 0x7fe6f41a7c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:09.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.851+0000 7fe6f37fe640 1 -- 172.21.15.26:0/1763176668 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6f4114e50 con 0x7fe6f4109450 2024-09-11T23:33:09.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.851+0000 7fe6f2ffd640 1 --2- 172.21.15.26:0/1763176668 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe6f4109e00 0x7fe6f41a7c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:33:09.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.851+0000 7fe6f3fff640 1 --2- 172.21.15.26:0/1763176668 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe6f410ea10 0x7fe6f4114590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:33:09.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.851+0000 7fe6f37fe640 1 --2- 172.21.15.26:0/1763176668 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe6f4109450 0x7fe6f41a7700 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fe6e802a860 tx=0x7fe6e8033b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:33:09.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.853+0000 7fe6f0ff9640 1 -- 172.21.15.26:0/1763176668 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6e8037040 con 0x7fe6f4109450 2024-09-11T23:33:09.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.853+0000 7fe6f9ce1640 1 -- 172.21.15.26:0/1763176668 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6f41150e0 con 0x7fe6f4109450 2024-09-11T23:33:09.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.853+0000 7fe6f9ce1640 1 -- 172.21.15.26:0/1763176668 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6f41a3380 con 0x7fe6f4109450 2024-09-11T23:33:09.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.854+0000 7fe6f0ff9640 1 -- 172.21.15.26:0/1763176668 <== mon.2 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe6e8033d50 con 0x7fe6f4109450 2024-09-11T23:33:09.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.854+0000 7fe6f0ff9640 1 -- 172.21.15.26:0/1763176668 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6e8003960 con 0x7fe6f4109450 2024-09-11T23:33:09.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.856+0000 7fe6fc0ba740 1 Processor -- start 2024-09-11T23:33:09.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.857+0000 7fe6fc0ba740 1 -- start start 2024-09-11T23:33:09.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.857+0000 7fe6fc0ba740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d500bdc900 0x55d500bdcce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:09.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.857+0000 7fe6fc0ba740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d500bdd220 0x55d500be75a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:09.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.857+0000 7fe6fc0ba740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d500be7ae0 0x55d500be9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:09.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.857+0000 7fe6fc0ba740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55d500b422a0 con 0x55d500bdc900 2024-09-11T23:33:09.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.857+0000 7fe6fc0ba740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55d500ae9b10 con 0x55d500bdd220 2024-09-11T23:33:09.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.857+0000 7fe6fc0ba740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55d500b3e260 con 0x55d500be7ae0 2024-09-11T23:33:09.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.860+0000 7fe6f2ffd640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d500be7ae0 0x55d500be9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:33:09.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.861+0000 7fe6f2ffd640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d500be7ae0 0x55d500be9ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:50476/0 (socket says 172.21.15.26:50476) 2024-09-11T23:33:09.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.861+0000 7fe6f2ffd640 1 -- 172.21.15.26:0/647022684 learned_addr learned my addr 172.21.15.26:0/647022684 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:33:09.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.862+0000 7fe6f37fe640 1 --2- 172.21.15.26:0/647022684 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d500bdd220 0x55d500be75a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:33:09.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.862+0000 7fe6f2ffd640 1 -- 172.21.15.26:0/647022684 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d500bdd220 msgr2=0x55d500be75a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:33:09.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.862+0000 7fe6f2ffd640 1 --2- 172.21.15.26:0/647022684 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d500bdd220 0x55d500be75a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:09.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.862+0000 7fe6f2ffd640 1 -- 172.21.15.26:0/647022684 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d500bdc900 msgr2=0x55d500bdcce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:33:09.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.862+0000 7fe6f2ffd640 1 --2- 172.21.15.26:0/647022684 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d500bdc900 0x55d500bdcce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:09.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.862+0000 7fe6f2ffd640 1 -- 172.21.15.26:0/647022684 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5008b3b90 con 0x55d500be7ae0 2024-09-11T23:33:09.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.863+0000 7fe6f0ff9640 1 -- 172.21.15.26:0/1763176668 <== mon.2 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fe6e8042030 con 0x7fe6f4109450 2024-09-11T23:33:09.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.863+0000 7fe6f2ffd640 1 --2- 172.21.15.26:0/647022684 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d500be7ae0 0x55d500be9ed0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fe6ec00af70 tx=0x7fe6ec00e8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=433841e637fd0a08 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:33:09.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.864+0000 7fe6ddffb640 1 -- 172.21.15.26:0/647022684 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6ec00c6b0 con 0x55d500be7ae0 2024-09-11T23:33:09.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.865+0000 7fe6ddffb640 1 -- 172.21.15.26:0/647022684 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe6ec00cd30 con 0x55d500be7ae0 2024-09-11T23:33:09.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.865+0000 7fe6ddffb640 1 -- 172.21.15.26:0/647022684 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6ec0093c0 con 0x55d500be7ae0 2024-09-11T23:33:09.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.865+0000 7fe6f0ff9640 1 --2- 172.21.15.26:0/1763176668 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fe6e40a2ec0 0x7fe6e40a5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:09.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.865+0000 7fe6f0ff9640 1 -- 172.21.15.26:0/1763176668 <== mon.2 v2:172.21.15.76:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8586+0+0 (secure 0 0 0) 0x7fe6e80f0630 con 0x7fe6f4109450 2024-09-11T23:33:09.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.866+0000 7fe6fc0ba740 1 -- 172.21.15.26:0/647022684 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d500be7ae0 msgr2=0x55d500be9ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:33:09.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.866+0000 7fe6fc0ba740 1 --2- 172.21.15.26:0/647022684 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d500be7ae0 0x55d500be9ed0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fe6ec00af70 tx=0x7fe6ec00e8f0 comp rx=0 tx=0).stop 2024-09-11T23:33:09.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.868+0000 7fe6fc0ba740 1 -- 172.21.15.26:0/647022684 shutdown_connections 2024-09-11T23:33:09.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.868+0000 7fe6fc0ba740 1 --2- 172.21.15.26:0/647022684 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d500bdd220 0x55d500be75a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:09.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.868+0000 7fe6fc0ba740 1 --2- 172.21.15.26:0/647022684 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d500be7ae0 0x55d500be9ed0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:09.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.868+0000 7fe6fc0ba740 1 --2- 172.21.15.26:0/647022684 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d500bdc900 0x55d500bdcce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:09.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.868+0000 7fe6fc0ba740 1 -- 172.21.15.26:0/647022684 >> 172.21.15.26:0/647022684 conn(0x55d500bd8e90 msgr2=0x55d500bd9270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:33:09.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.870+0000 7fe6f3fff640 1 --2- 172.21.15.26:0/1763176668 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fe6e40a2ec0 0x7fe6e40a5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:33:09.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.870+0000 7fe6fc0ba740 1 -- 172.21.15.26:0/647022684 shutdown_connections 2024-09-11T23:33:09.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.872+0000 7fe6f3fff640 1 --2- 172.21.15.26:0/1763176668 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fe6e40a2ec0 0x7fe6e40a5380 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fe6e00095f0 tx=0x7fe6e00046c0 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:33:09.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.873+0000 7fe6fc0ba740 1 -- 172.21.15.26:0/647022684 wait complete. 2024-09-11T23:33:09.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.873+0000 7fe6fc0ba740 1 -- start start 2024-09-11T23:33:09.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.874+0000 7fe6fc0ba740 1 Processor -- start 2024-09-11T23:33:09.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.874+0000 7fe6fc0ba740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d500bdc900 0x55d500be4a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:09.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.874+0000 7fe6fc0ba740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d500bdd220 0x55d500be4f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:09.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.874+0000 7fe6fc0ba740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d500be7ae0 0x55d500be5480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:09.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.874+0000 7fe6fc0ba740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55d500bea6b0 con 0x55d500be7ae0 2024-09-11T23:33:09.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.874+0000 7fe6fc0ba740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55d500aca6e0 con 0x55d500bdc900 2024-09-11T23:33:09.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.874+0000 7fe6fc0ba740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55d500bea830 con 0x55d500bdd220 2024-09-11T23:33:09.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.874+0000 7fe6fc0ba740 10 client.0 _ll_register_callbacks cb 0x55d500aeea80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:33:09.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.875+0000 7fe6f2ffd640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d500bdd220 0x55d500be4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:33:09.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.875+0000 7fe6f2ffd640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d500bdd220 0x55d500be4f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:50482/0 (socket says 172.21.15.26:50482) 2024-09-11T23:33:09.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.875+0000 7fe6f2ffd640 1 -- 172.21.15.26:0/2790991359 learned_addr learned my addr 172.21.15.26:0/2790991359 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:33:09.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.876+0000 7fe6f3fff640 1 --2- 172.21.15.26:0/2790991359 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d500be7ae0 0x55d500be5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:33:09.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.876+0000 7fe6f2ffd640 1 -- 172.21.15.26:0/2790991359 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d500bdc900 msgr2=0x55d500be4a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:33:09.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.876+0000 7fe6f2ffd640 1 --2- 172.21.15.26:0/2790991359 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d500bdc900 0x55d500be4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:09.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.876+0000 7fe6f2ffd640 1 -- 172.21.15.26:0/2790991359 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d500be7ae0 msgr2=0x55d500be5480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:33:09.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.876+0000 7fe6f2ffd640 1 --2- 172.21.15.26:0/2790991359 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d500be7ae0 0x55d500be5480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:09.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.876+0000 7fe6f2ffd640 1 -- 172.21.15.26:0/2790991359 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d500bf3c60 con 0x55d500bdd220 2024-09-11T23:33:09.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.880+0000 7fe6f2ffd640 1 --2- 172.21.15.26:0/2790991359 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d500bdd220 0x55d500be4f40 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fe6ec000db0 tx=0x7fe6ec0175f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe4ca055b83631d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:33:09.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.881+0000 7fe6d2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-09-11T23:33:09.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.881+0000 7fe6d2ffd640 1 -- 172.21.15.26:0/2790991359 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6ec00ac40 con 0x55d500bdd220 2024-09-11T23:33:09.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.881+0000 7fe6d2ffd640 1 -- 172.21.15.26:0/2790991359 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8586+0+0 (secure 0 0 0) 0x7fe6ec014040 con 0x55d500bdd220 2024-09-11T23:33:09.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.882+0000 7fe6fc0ba740 1 -- 172.21.15.26:0/2790991359 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe6ec00cd30 con 0x55d500bdd220 2024-09-11T23:33:09.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.883+0000 7fe6d2ffd640 1 -- 172.21.15.26:0/2790991359 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) ==== 24+0+0 (secure 0 0 0) 0x7fe6ec00c8c0 con 0x55d500bdd220 2024-09-11T23:33:09.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.883+0000 7fe6fc0ba740 10 client.27001 fetch_fsmap learned FSMap version 72 2024-09-11T23:33:09.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.883+0000 7fe6fc0ba740 1 -- 172.21.15.26:0/2790991359 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) -- 0x55d500bf65f0 con 0x55d500bdd220 2024-09-11T23:33:09.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.885+0000 7fe6d2ffd640 1 -- 172.21.15.26:0/2790991359 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe6ec00cd30 con 0x55d500bdd220 2024-09-11T23:33:09.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.886+0000 7fe6d2ffd640 1 -- 172.21.15.26:0/2790991359 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 72) ==== 8426+0+0 (secure 0 0 0) 0x7fe6ec020420 con 0x55d500bdd220 2024-09-11T23:33:09.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.886+0000 7fe6fc0ba740 10 client.27001 fetch_fsmap finished waiting for FSMap version 72 2024-09-11T23:33:09.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.886+0000 7fe6fc0ba740 10 client.27001 resolve_mds: validated gid 24485 aka daemon mds.b 2024-09-11T23:33:09.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.886+0000 7fe6fc0ba740 20 client.27001 populate_metadata read hostname 'smithi026' 2024-09-11T23:33:09.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.886+0000 7fe6fc0ba740 1 --2- 172.21.15.26:0/2790991359 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55d500bf7f20 0x55d500bfa3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:09.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.886+0000 7fe6fc0ba740 4 client.27001 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:33:09.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.886+0000 7fe6fc0ba740 1 -- 172.21.15.26:0/2790991359 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d500b4f9d0 con 0x55d500bf7f20 2024-09-11T23:33:09.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.887+0000 7fe6f37fe640 1 --2- 172.21.15.26:0/2790991359 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55d500bf7f20 0x55d500bfa3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:33:09.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.892+0000 7fe6f37fe640 1 --2- 172.21.15.26:0/2790991359 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55d500bf7f20 0x55d500bfa3c0 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c88a88cedc2a7be server_cookie=cbf53ac071f6ded8 in_seq=0 out_seq=0 2024-09-11T23:33:09.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.894+0000 7fe6d2ffd640 10 client.27001 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:33:09.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.894+0000 7fe6d2ffd640 1 -- 172.21.15.26:0/2790991359 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d500b4f9d0 con 0x55d500bf7f20 2024-09-11T23:33:09.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.894+0000 7fe6d2ffd640 10 client.27001 handle_command_reply: tid=0 2024-09-11T23:33:09.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.894+0000 7fe6fc0ba740 1 client.27001 shutdown 2024-09-11T23:33:09.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.898+0000 7fe6fc0ba740 1 -- 172.21.15.26:0/2790991359 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d500bdd220 msgr2=0x55d500be4f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:33:09.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.898+0000 7fe6fc0ba740 1 --2- 172.21.15.26:0/2790991359 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d500bdd220 0x55d500be4f40 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fe6ec000db0 tx=0x7fe6ec0175f0 comp rx=0 tx=0).stop 2024-09-11T23:33:09.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.901+0000 7fe6fc0ba740 1 -- 172.21.15.26:0/2790991359 shutdown_connections 2024-09-11T23:33:09.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.901+0000 7fe6fc0ba740 1 --2- 172.21.15.26:0/2790991359 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55d500bf7f20 0x55d500bfa3c0 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:09.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.901+0000 7fe6fc0ba740 1 --2- 172.21.15.26:0/2790991359 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d500be7ae0 0x55d500be5480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:09.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.901+0000 7fe6fc0ba740 1 --2- 172.21.15.26:0/2790991359 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d500bdd220 0x55d500be4f40 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:09.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.901+0000 7fe6fc0ba740 1 --2- 172.21.15.26:0/2790991359 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d500bdc900 0x55d500be4a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:09.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.901+0000 7fe6fc0ba740 1 -- 172.21.15.26:0/2790991359 >> 172.21.15.26:0/2790991359 conn(0x55d500bd8e90 msgr2=0x55d500bdebc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:33:09.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.901+0000 7fe6d2ffd640 0 client.27001 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:33:09.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.902+0000 7fe6fc0ba740 1 -- 172.21.15.26:0/2790991359 shutdown_connections 2024-09-11T23:33:09.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.903+0000 7fe6fc0ba740 1 -- 172.21.15.26:0/2790991359 wait complete. 2024-09-11T23:33:09.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.904+0000 7fe6fc0ba740 20 client.27001 trim_cache size 0 max 16384 2024-09-11T23:33:09.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.940+0000 7fe6fc0ba740 1 Processor -- start 2024-09-11T23:33:09.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.941+0000 7fe6fc0ba740 1 -- start start 2024-09-11T23:33:09.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.941+0000 7fe6fc0ba740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d500bdc900 0x55d500be6970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:09.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.941+0000 7fe6fc0ba740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d500bdd220 0x55d500bffcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:09.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.941+0000 7fe6fc0ba740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d500be7ae0 0x55d500c00210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:09.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.941+0000 7fe6fc0ba740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fe6d800e810 con 0x55d500bdc900 2024-09-11T23:33:09.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.941+0000 7fe6fc0ba740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fe6d8010590 con 0x55d500bdd220 2024-09-11T23:33:09.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.941+0000 7fe6fc0ba740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe6d800eee0 con 0x55d500be7ae0 2024-09-11T23:33:09.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.941+0000 7fe6f37fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d500bdc900 0x55d500be6970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:33:09.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.942+0000 7fe6f37fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d500bdc900 0x55d500be6970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52760/0 (socket says 172.21.15.26:52760) 2024-09-11T23:33:09.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.942+0000 7fe6f37fe640 1 -- 172.21.15.26:0/34158039 learned_addr learned my addr 172.21.15.26:0/34158039 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:33:09.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.942+0000 7fe6f37fe640 1 -- 172.21.15.26:0/34158039 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d500be7ae0 msgr2=0x55d500c00210 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T23:33:09.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.942+0000 7fe6f37fe640 1 --2- 172.21.15.26:0/34158039 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d500be7ae0 0x55d500c00210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:09.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.942+0000 7fe6f37fe640 1 -- 172.21.15.26:0/34158039 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d500bdd220 msgr2=0x55d500bffcd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T23:33:09.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.942+0000 7fe6f37fe640 1 --2- 172.21.15.26:0/34158039 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d500bdd220 0x55d500bffcd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:09.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.942+0000 7fe6f37fe640 1 -- 172.21.15.26:0/34158039 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d500be2f50 con 0x55d500bdc900 2024-09-11T23:33:09.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.942+0000 7fe6f37fe640 1 --2- 172.21.15.26:0/34158039 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d500bdc900 0x55d500be6970 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7fe6e80b9620 tx=0x7fe6e80095b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6aef733b713077a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:33:09.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.943+0000 7fe6dd7fa640 1 -- 172.21.15.26:0/34158039 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6e80bd350 con 0x55d500bdc900 2024-09-11T23:33:09.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.943+0000 7fe6dd7fa640 1 -- 172.21.15.26:0/34158039 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe6e80bcc50 con 0x55d500bdc900 2024-09-11T23:33:09.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.943+0000 7fe6dd7fa640 1 -- 172.21.15.26:0/34158039 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6e80edf60 con 0x55d500bdc900 2024-09-11T23:33:09.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.944+0000 7fe6fc0ba740 1 -- 172.21.15.26:0/34158039 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d500bdc900 msgr2=0x55d500be6970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:33:09.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.944+0000 7fe6fc0ba740 1 --2- 172.21.15.26:0/34158039 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d500bdc900 0x55d500be6970 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7fe6e80b9620 tx=0x7fe6e80095b0 comp rx=0 tx=0).stop 2024-09-11T23:33:09.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.946+0000 7fe6fc0ba740 1 -- 172.21.15.26:0/34158039 shutdown_connections 2024-09-11T23:33:09.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.946+0000 7fe6fc0ba740 1 --2- 172.21.15.26:0/34158039 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d500bdd220 0x55d500bffcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:09.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.946+0000 7fe6fc0ba740 1 --2- 172.21.15.26:0/34158039 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d500be7ae0 0x55d500c00210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:09.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.946+0000 7fe6fc0ba740 1 --2- 172.21.15.26:0/34158039 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d500bdc900 0x55d500be6970 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:09.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.946+0000 7fe6fc0ba740 1 -- 172.21.15.26:0/34158039 >> 172.21.15.26:0/34158039 conn(0x55d500bd8e90 msgr2=0x55d500be6260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:33:09.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.947+0000 7fe6fc0ba740 1 -- 172.21.15.26:0/34158039 shutdown_connections 2024-09-11T23:33:09.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.947+0000 7fe6fc0ba740 1 -- 172.21.15.26:0/34158039 wait complete. 2024-09-11T23:33:09.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.948+0000 7fe6fc0ba740 1 -- start start 2024-09-11T23:33:09.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.948+0000 7fe6fc0ba740 1 Processor -- start 2024-09-11T23:33:09.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.949+0000 7fe6fc0ba740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d500bdc900 0x55d500beb8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:09.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.949+0000 7fe6fc0ba740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d500bdd220 0x55d500bebe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:09.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.949+0000 7fe6fc0ba740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d500be7ae0 0x55d500bec370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:09.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.950+0000 7fe6fc0ba740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fe6d800fe00 con 0x55d500bdd220 2024-09-11T23:33:09.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.950+0000 7fe6fc0ba740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fe6d800f670 con 0x55d500bdc900 2024-09-11T23:33:09.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.950+0000 7fe6fc0ba740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe6d800c8a0 con 0x55d500be7ae0 2024-09-11T23:33:09.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.950+0000 7fe6fc0ba740 10 client.0 _ll_register_callbacks cb 0x7fe6ec017820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:33:09.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.950+0000 7fe6f2ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d500bdd220 0x55d500bebe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:33:09.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.950+0000 7fe6f2ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d500bdd220 0x55d500bebe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52770/0 (socket says 172.21.15.26:52770) 2024-09-11T23:33:09.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.950+0000 7fe6f2ffd640 1 -- 172.21.15.26:0/566286320 learned_addr learned my addr 172.21.15.26:0/566286320 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:33:09.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.951+0000 7fe6f2ffd640 1 -- 172.21.15.26:0/566286320 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d500be7ae0 msgr2=0x55d500bec370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:33:09.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.951+0000 7fe6f2ffd640 1 --2- 172.21.15.26:0/566286320 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d500be7ae0 0x55d500bec370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:09.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.951+0000 7fe6f2ffd640 1 -- 172.21.15.26:0/566286320 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d500bdc900 msgr2=0x55d500beb8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:33:09.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.951+0000 7fe6f2ffd640 1 --2- 172.21.15.26:0/566286320 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d500bdc900 0x55d500beb8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:09.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.951+0000 7fe6f2ffd640 1 -- 172.21.15.26:0/566286320 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d500c61d20 con 0x55d500bdd220 2024-09-11T23:33:09.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.952+0000 7fe6f2ffd640 1 --2- 172.21.15.26:0/566286320 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d500bdd220 0x55d500bebe30 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fe6ec020990 tx=0x7fe6ec017ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e83e93c8a7ba11d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:33:09.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.953+0000 7fe6d2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:33:09.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.953+0000 7fe6d2ffd640 1 -- 172.21.15.26:0/566286320 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6ec032070 con 0x55d500bdd220 2024-09-11T23:33:09.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.953+0000 7fe6d2ffd640 1 -- 172.21.15.26:0/566286320 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8586+0+0 (secure 0 0 0) 0x7fe6ec014040 con 0x55d500bdd220 2024-09-11T23:33:09.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.953+0000 7fe6fc0ba740 1 -- 172.21.15.26:0/566286320 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe6e80bcc50 con 0x55d500bdd220 2024-09-11T23:33:09.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.953+0000 7fe6d2ffd640 1 -- 172.21.15.26:0/566286320 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) ==== 24+0+0 (secure 0 0 0) 0x7fe6ec020630 con 0x55d500bdd220 2024-09-11T23:33:09.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.954+0000 7fe6fc0ba740 10 client.17496 fetch_fsmap learned FSMap version 72 2024-09-11T23:33:09.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.954+0000 7fe6fc0ba740 1 -- 172.21.15.26:0/566286320 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) -- 0x55d500c62fe0 con 0x55d500bdd220 2024-09-11T23:33:09.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.955+0000 7fe6d2ffd640 1 -- 172.21.15.26:0/566286320 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe6ec025070 con 0x55d500bdd220 2024-09-11T23:33:09.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.956+0000 7fe6d2ffd640 1 -- 172.21.15.26:0/566286320 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 72) ==== 8426+0+0 (secure 0 0 0) 0x7fe6ec02a450 con 0x55d500bdd220 2024-09-11T23:33:09.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.956+0000 7fe6fc0ba740 10 client.17496 fetch_fsmap finished waiting for FSMap version 72 2024-09-11T23:33:09.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.956+0000 7fe6fc0ba740 10 client.17496 resolve_mds: validated gid 24485 aka daemon mds.b 2024-09-11T23:33:09.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.956+0000 7fe6fc0ba740 20 client.17496 populate_metadata read hostname 'smithi026' 2024-09-11T23:33:09.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.956+0000 7fe6fc0ba740 1 --2- 172.21.15.26:0/566286320 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55d500c637b0 0x55d500c65bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:09.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.956+0000 7fe6fc0ba740 4 client.17496 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}] 2024-09-11T23:33:09.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.956+0000 7fe6fc0ba740 1 -- 172.21.15.26:0/566286320 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}) -- 0x7fe6e80bcc50 con 0x55d500c637b0 2024-09-11T23:33:09.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.958+0000 7fe6f37fe640 1 --2- 172.21.15.26:0/566286320 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55d500c637b0 0x55d500c65bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:33:09.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.961+0000 7fe6f37fe640 1 --2- 172.21.15.26:0/566286320 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55d500c637b0 0x55d500c65bd0 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f88560b3970d9f71 server_cookie=93a69b7f6fbd945a in_seq=0 out_seq=0 2024-09-11T23:33:09.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:09.962+0000 7fe6d2ffd640 10 client.17496 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:33:10.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:33:10 smithi026 ceph-mon[31172]: osdmap e103: 12 total, 12 up, 12 in 2024-09-11T23:33:10.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:33:10 smithi136 ceph-mon[35121]: osdmap e103: 12 total, 12 up, 12 in 2024-09-11T23:33:10.552 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:33:10 smithi076 ceph-mon[35296]: osdmap e103: 12 total, 12 up, 12 in 2024-09-11T23:33:11.264 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.264+0000 7ff7167fc640 1 -- 172.21.15.26:0/394212849 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff71002f340 con 0x7ff718114ab0 2024-09-11T23:33:11.264 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.265+0000 7ff71cd5b640 1 -- 172.21.15.26:0/394212849 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff718114ab0 msgr2=0x7ff718116ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:33:11.264 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.265+0000 7ff71cd5b640 1 --2- 172.21.15.26:0/394212849 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff718114ab0 0x7ff718116ea0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ff710009ca0 tx=0x7ff71002a8f0 comp rx=0 tx=0).stop 2024-09-11T23:33:11.268 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.269+0000 7ff71cd5b640 1 -- 172.21.15.26:0/394212849 shutdown_connections 2024-09-11T23:33:11.268 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.269+0000 7ff71cd5b640 1 --2- 172.21.15.26:0/394212849 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff718114ab0 0x7ff718116ea0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:11.268 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.269+0000 7ff71cd5b640 1 --2- 172.21.15.26:0/394212849 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff718109b10 0x7ff718114570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:11.268 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.269+0000 7ff71cd5b640 1 --2- 172.21.15.26:0/394212849 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff7181091f0 0x7ff7181095d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:11.268 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.269+0000 7ff71cd5b640 1 -- 172.21.15.26:0/394212849 >> 172.21.15.26:0/394212849 conn(0x7ff71806cbf0 msgr2=0x7ff71806d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:33:11.269 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.269+0000 7ff71cd5b640 1 -- 172.21.15.26:0/394212849 shutdown_connections 2024-09-11T23:33:11.273 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.273+0000 7ff71cd5b640 1 -- 172.21.15.26:0/394212849 wait complete. 2024-09-11T23:33:11.273 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.273+0000 7ff71cd5b640 1 Processor -- start 2024-09-11T23:33:11.273 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.274+0000 7ff71cd5b640 1 -- start start 2024-09-11T23:33:11.273 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.274+0000 7ff71cd5b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff7181091f0 0x7ff71810a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:11.273 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.274+0000 7ff71cd5b640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff718109b10 0x7ff71810a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:11.274 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.274+0000 7ff71cd5b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff718114ab0 0x7ff71810ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:11.274 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.274+0000 7ff71cd5b640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff71811a560 con 0x7ff7181091f0 2024-09-11T23:33:11.274 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.274+0000 7ff71cd5b640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7ff71811a3e0 con 0x7ff718114ab0 2024-09-11T23:33:11.274 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.274+0000 7ff71cd5b640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ff71811a6e0 con 0x7ff718109b10 2024-09-11T23:33:11.278 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.277+0000 7ff716ffd640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff718109b10 0x7ff71810a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:33:11.278 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.277+0000 7ff716ffd640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff718109b10 0x7ff71810a910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:43748/0 (socket says 172.21.15.26:43748) 2024-09-11T23:33:11.278 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.277+0000 7ff716ffd640 1 -- 172.21.15.26:0/400373080 learned_addr learned my addr 172.21.15.26:0/400373080 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:33:11.278 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.278+0000 7ff717fff640 1 --2- 172.21.15.26:0/400373080 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff718114ab0 0x7ff71810ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:33:11.278 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.278+0000 7ff717fff640 1 -- 172.21.15.26:0/400373080 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff718109b10 msgr2=0x7ff71810a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:33:11.278 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.278+0000 7ff717fff640 1 --2- 172.21.15.26:0/400373080 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff718109b10 0x7ff71810a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:11.278 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.278+0000 7ff717fff640 1 -- 172.21.15.26:0/400373080 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff7181091f0 msgr2=0x7ff71810a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:33:11.278 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.278+0000 7ff717fff640 1 --2- 172.21.15.26:0/400373080 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff7181091f0 0x7ff71810a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:11.279 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.279+0000 7ff717fff640 1 -- 172.21.15.26:0/400373080 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7181a4bf0 con 0x7ff718114ab0 2024-09-11T23:33:11.281 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.281+0000 7ff717fff640 1 --2- 172.21.15.26:0/400373080 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff718114ab0 0x7ff71810ae50 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ff71002aee0 tx=0x7ff710030820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:33:11.281 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.282+0000 7ff714ff9640 1 -- 172.21.15.26:0/400373080 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff710003b70 con 0x7ff718114ab0 2024-09-11T23:33:11.281 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.282+0000 7ff714ff9640 1 -- 172.21.15.26:0/400373080 <== mon.2 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff71002d070 con 0x7ff718114ab0 2024-09-11T23:33:11.281 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.282+0000 7ff714ff9640 1 -- 172.21.15.26:0/400373080 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff710004610 con 0x7ff718114ab0 2024-09-11T23:33:11.285 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.282+0000 7ff71cd5b640 1 -- 172.21.15.26:0/400373080 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7181a4e80 con 0x7ff718114ab0 2024-09-11T23:33:11.285 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.282+0000 7ff71cd5b640 1 -- 172.21.15.26:0/400373080 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7181a5380 con 0x7ff718114ab0 2024-09-11T23:33:11.285 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.286+0000 7ff720e35740 1 Processor -- start 2024-09-11T23:33:11.287 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.287+0000 7ff714ff9640 1 -- 172.21.15.26:0/400373080 <== mon.2 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7ff710030890 con 0x7ff718114ab0 2024-09-11T23:33:11.288 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.288+0000 7ff714ff9640 1 --2- 172.21.15.26:0/400373080 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7ff7000a2f00 0x7ff7000a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:11.288 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.288+0000 7ff714ff9640 1 -- 172.21.15.26:0/400373080 <== mon.2 v2:172.21.15.76:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8586+0+0 (secure 0 0 0) 0x7ff7100f0280 con 0x7ff718114ab0 2024-09-11T23:33:11.289 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.289+0000 7ff720e35740 1 -- start start 2024-09-11T23:33:11.289 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.289+0000 7ff720e35740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e1c08a5cc0 0x55e1c08a60a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:11.289 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.289+0000 7ff720e35740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e1c08a65e0 0x55e1c08b0940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:11.289 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.289+0000 7ff720e35740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e1c08b0e80 0x55e1c08b3270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:11.289 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.289+0000 7ff720e35740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55e1c080a2f0 con 0x55e1c08a65e0 2024-09-11T23:33:11.289 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.289+0000 7ff720e35740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55e1c07b2270 con 0x55e1c08b0e80 2024-09-11T23:33:11.289 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.289+0000 7ff720e35740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55e1c08062b0 con 0x55e1c08a5cc0 2024-09-11T23:33:11.289 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.289+0000 7ff716ffd640 1 --2- 172.21.15.26:0/400373080 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7ff7000a2f00 0x7ff7000a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:33:11.290 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.290+0000 7ff716ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e1c08a65e0 0x55e1c08b0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:33:11.290 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.290+0000 7ff716ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e1c08a65e0 0x55e1c08b0940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54046/0 (socket says 172.21.15.26:54046) 2024-09-11T23:33:11.290 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.290+0000 7ff716ffd640 1 -- 172.21.15.26:0/3996854664 learned_addr learned my addr 172.21.15.26:0/3996854664 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:33:11.290 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.291+0000 7ff716ffd640 1 -- 172.21.15.26:0/3996854664 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e1c08a5cc0 msgr2=0x55e1c08a60a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:33:11.290 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.291+0000 7ff716ffd640 1 --2- 172.21.15.26:0/3996854664 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e1c08a5cc0 0x55e1c08a60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:11.290 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.291+0000 7ff716ffd640 1 -- 172.21.15.26:0/3996854664 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e1c08b0e80 msgr2=0x55e1c08b3270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:33:11.291 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.291+0000 7ff716ffd640 1 --2- 172.21.15.26:0/3996854664 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e1c08b0e80 0x55e1c08b3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:11.291 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.291+0000 7ff716ffd640 1 -- 172.21.15.26:0/3996854664 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e1c07b1340 con 0x55e1c08a65e0 2024-09-11T23:33:11.291 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.291+0000 7ff716ffd640 1 --2- 172.21.15.26:0/400373080 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7ff7000a2f00 0x7ff7000a53c0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ff7080058c0 tx=0x7ff7080129b0 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:33:11.293 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.291+0000 7ff716ffd640 1 --2- 172.21.15.26:0/3996854664 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e1c08a65e0 0x55e1c08b0940 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7ff708012eb0 tx=0x7ff70801a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf28393376e9485e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:33:11.293 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.294+0000 7ff6fdffb640 1 -- 172.21.15.26:0/3996854664 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff708023070 con 0x55e1c08a65e0 2024-09-11T23:33:11.294 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.294+0000 7ff6fdffb640 1 -- 172.21.15.26:0/3996854664 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff70800de20 con 0x55e1c08a65e0 2024-09-11T23:33:11.297 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.295+0000 7ff720e35740 1 -- 172.21.15.26:0/3996854664 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e1c08a65e0 msgr2=0x55e1c08b0940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:33:11.297 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.295+0000 7ff720e35740 1 --2- 172.21.15.26:0/3996854664 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e1c08a65e0 0x55e1c08b0940 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7ff708012eb0 tx=0x7ff70801a000 comp rx=0 tx=0).stop 2024-09-11T23:33:11.299 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.299+0000 7ff720e35740 1 -- 172.21.15.26:0/3996854664 shutdown_connections 2024-09-11T23:33:11.299 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.299+0000 7ff720e35740 1 --2- 172.21.15.26:0/3996854664 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e1c08b0e80 0x55e1c08b3270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:11.299 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.299+0000 7ff720e35740 1 --2- 172.21.15.26:0/3996854664 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e1c08a65e0 0x55e1c08b0940 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:11.299 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.299+0000 7ff720e35740 1 --2- 172.21.15.26:0/3996854664 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e1c08a5cc0 0x55e1c08a60a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:11.300 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.299+0000 7ff720e35740 1 -- 172.21.15.26:0/3996854664 >> 172.21.15.26:0/3996854664 conn(0x55e1c08a2230 msgr2=0x55e1c08a2610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:33:11.300 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.300+0000 7ff720e35740 1 -- 172.21.15.26:0/3996854664 shutdown_connections 2024-09-11T23:33:11.300 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.300+0000 7ff720e35740 1 -- 172.21.15.26:0/3996854664 wait complete. 2024-09-11T23:33:11.300 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.300+0000 7ff720e35740 1 -- start start 2024-09-11T23:33:11.300 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.300+0000 7ff720e35740 1 Processor -- start 2024-09-11T23:33:11.306 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.306+0000 7ff720e35740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e1c08a5cc0 0x55e1c08ade10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:11.306 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.306+0000 7ff720e35740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e1c08a65e0 0x55e1c08ae350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:11.306 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.306+0000 7ff720e35740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e1c08b0e80 0x55e1c08ae890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:11.306 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.306+0000 7ff720e35740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55e1c08b3aa0 con 0x55e1c08a5cc0 2024-09-11T23:33:11.306 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.306+0000 7ff720e35740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55e1c0793290 con 0x55e1c08b0e80 2024-09-11T23:33:11.306 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.306+0000 7ff720e35740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55e1c08b3c20 con 0x55e1c08a65e0 2024-09-11T23:33:11.306 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.306+0000 7ff720e35740 10 client.0 _ll_register_callbacks cb 0x55e1c07c4140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:33:11.307 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.307+0000 7ff716ffd640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e1c08a65e0 0x55e1c08ae350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:33:11.307 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.307+0000 7ff716ffd640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e1c08a65e0 0x55e1c08ae350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:43766/0 (socket says 172.21.15.26:43766) 2024-09-11T23:33:11.307 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.307+0000 7ff716ffd640 1 -- 172.21.15.26:0/292100699 learned_addr learned my addr 172.21.15.26:0/292100699 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:33:11.307 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.308+0000 7ff716ffd640 1 -- 172.21.15.26:0/292100699 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e1c08b0e80 msgr2=0x55e1c08ae890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:33:11.307 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.308+0000 7ff716ffd640 1 --2- 172.21.15.26:0/292100699 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e1c08b0e80 0x55e1c08ae890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:11.308 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.308+0000 7ff716ffd640 1 -- 172.21.15.26:0/292100699 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e1c08a5cc0 msgr2=0x55e1c08ade10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:33:11.308 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.308+0000 7ff716ffd640 1 --2- 172.21.15.26:0/292100699 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e1c08a5cc0 0x55e1c08ade10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:11.308 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.308+0000 7ff716ffd640 1 -- 172.21.15.26:0/292100699 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e1c051d210 con 0x55e1c08a65e0 2024-09-11T23:33:11.308 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.308+0000 7ff716ffd640 1 --2- 172.21.15.26:0/292100699 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e1c08a65e0 0x55e1c08ae350 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff708006010 tx=0x7ff708005f30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4ad5b59cfdb729be server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:33:11.308 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.309+0000 7ff6f6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-09-11T23:33:11.308 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.309+0000 7ff6f6ffd640 1 -- 172.21.15.26:0/292100699 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff708023040 con 0x55e1c08a65e0 2024-09-11T23:33:11.309 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.309+0000 7ff6f6ffd640 1 -- 172.21.15.26:0/292100699 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8586+0+0 (secure 0 0 0) 0x7ff70800db20 con 0x55e1c08a65e0 2024-09-11T23:33:11.312 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.312+0000 7ff720e35740 1 -- 172.21.15.26:0/292100699 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff70800de20 con 0x55e1c08a65e0 2024-09-11T23:33:11.319 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.319+0000 7ff6f6ffd640 1 -- 172.21.15.26:0/292100699 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) ==== 24+0+0 (secure 0 0 0) 0x7ff708036050 con 0x55e1c08a65e0 2024-09-11T23:33:11.319 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.320+0000 7ff720e35740 10 client.27010 fetch_fsmap learned FSMap version 72 2024-09-11T23:33:11.319 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.320+0000 7ff720e35740 1 -- 172.21.15.26:0/292100699 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) -- 0x55e1c05e9620 con 0x55e1c08a65e0 2024-09-11T23:33:11.322 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.322+0000 7ff6f6ffd640 1 -- 172.21.15.26:0/292100699 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff70800de20 con 0x55e1c08a65e0 2024-09-11T23:33:11.322 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.323+0000 7ff6f6ffd640 1 -- 172.21.15.26:0/292100699 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 72) ==== 8426+0+0 (secure 0 0 0) 0x7ff708004a70 con 0x55e1c08a65e0 2024-09-11T23:33:11.322 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.323+0000 7ff720e35740 10 client.27010 fetch_fsmap finished waiting for FSMap version 72 2024-09-11T23:33:11.323 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.323+0000 7ff720e35740 10 client.27010 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:33:11.323 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.323+0000 7ff720e35740 20 client.27010 populate_metadata read hostname 'smithi026' 2024-09-11T23:33:11.323 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.323+0000 7ff720e35740 1 --2- 172.21.15.26:0/292100699 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55e1c08c0d90 0x55e1c08c3180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:11.323 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.323+0000 7ff720e35740 4 client.27010 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:33:11.323 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.323+0000 7ff720e35740 1 -- 172.21.15.26:0/292100699 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e1c0817a20 con 0x55e1c08c0d90 2024-09-11T23:33:11.323 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.324+0000 7ff7177fe640 1 --2- 172.21.15.26:0/292100699 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55e1c08c0d90 0x55e1c08c3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:33:11.332 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.333+0000 7ff7177fe640 1 --2- 172.21.15.26:0/292100699 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55e1c08c0d90 0x55e1c08c3180 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f8891037faef2942 server_cookie=12ebaa42103f5223 in_seq=0 out_seq=0 2024-09-11T23:33:11.335 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.335+0000 7ff6f6ffd640 10 client.27010 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:33:11.335 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.335+0000 7ff6f6ffd640 1 -- 172.21.15.26:0/292100699 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e1c0817a20 con 0x55e1c08c0d90 2024-09-11T23:33:11.335 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.335+0000 7ff6f6ffd640 10 client.27010 handle_command_reply: tid=0 2024-09-11T23:33:11.339 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.339+0000 7ff720e35740 1 client.27010 shutdown 2024-09-11T23:33:11.340 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.341+0000 7ff720e35740 1 -- 172.21.15.26:0/292100699 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e1c08a65e0 msgr2=0x55e1c08ae350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:33:11.340 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.341+0000 7ff720e35740 1 --2- 172.21.15.26:0/292100699 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e1c08a65e0 0x55e1c08ae350 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff708006010 tx=0x7ff708005f30 comp rx=0 tx=0).stop 2024-09-11T23:33:11.343 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.343+0000 7ff720e35740 1 -- 172.21.15.26:0/292100699 shutdown_connections 2024-09-11T23:33:11.343 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.343+0000 7ff720e35740 1 --2- 172.21.15.26:0/292100699 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e1c08b0e80 0x55e1c08ae890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:11.344 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.343+0000 7ff720e35740 1 --2- 172.21.15.26:0/292100699 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55e1c08c0d90 0x55e1c08c3180 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:11.344 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.343+0000 7ff720e35740 1 --2- 172.21.15.26:0/292100699 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e1c08a65e0 0x55e1c08ae350 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:11.344 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.343+0000 7ff720e35740 1 --2- 172.21.15.26:0/292100699 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e1c08a5cc0 0x55e1c08ade10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:11.344 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.343+0000 7ff720e35740 1 -- 172.21.15.26:0/292100699 >> 172.21.15.26:0/292100699 conn(0x55e1c08a2230 msgr2=0x55e1c08a7f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:33:11.344 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.344+0000 7ff6f6ffd640 0 client.27010 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:33:11.344 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.344+0000 7ff720e35740 1 -- 172.21.15.26:0/292100699 shutdown_connections 2024-09-11T23:33:11.344 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.345+0000 7ff720e35740 1 -- 172.21.15.26:0/292100699 wait complete. 2024-09-11T23:33:11.348 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.346+0000 7ff720e35740 20 client.27010 trim_cache size 0 max 16384 2024-09-11T23:33:11.380 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.380+0000 7ff720e35740 1 Processor -- start 2024-09-11T23:33:11.380 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.380+0000 7ff720e35740 1 -- start start 2024-09-11T23:33:11.381 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.381+0000 7ff720e35740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e1c08a5cc0 0x55e1c08ae6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:11.382 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.381+0000 7ff720e35740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e1c08a65e0 0x55e1c08c8a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:11.382 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.381+0000 7ff720e35740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e1c08b0e80 0x55e1c08c8f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:11.382 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.381+0000 7ff720e35740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff7181c4d80 con 0x55e1c08a65e0 2024-09-11T23:33:11.382 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.381+0000 7ff7177fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e1c08a5cc0 0x55e1c08ae6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:33:11.382 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.381+0000 7ff7177fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e1c08a5cc0 0x55e1c08ae6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:40416/0 (socket says 172.21.15.26:40416) 2024-09-11T23:33:11.382 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.381+0000 7ff7177fe640 1 -- 172.21.15.26:0/483691062 learned_addr learned my addr 172.21.15.26:0/483691062 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:33:11.382 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.381+0000 7ff720e35740 1 -- 172.21.15.26:0/483691062 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7ff7181c6b00 con 0x55e1c08a5cc0 2024-09-11T23:33:11.382 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.381+0000 7ff720e35740 1 -- 172.21.15.26:0/483691062 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ff7181c5450 con 0x55e1c08b0e80 2024-09-11T23:33:11.382 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.381+0000 7ff717fff640 1 --2- 172.21.15.26:0/483691062 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e1c08b0e80 0x55e1c08c8f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:33:11.383 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.382+0000 7ff716ffd640 1 --2- 172.21.15.26:0/483691062 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e1c08a65e0 0x55e1c08c8a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:33:11.383 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.383+0000 7ff7177fe640 1 -- 172.21.15.26:0/483691062 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e1c08b0e80 msgr2=0x55e1c08c8f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:33:11.383 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.383+0000 7ff7177fe640 1 --2- 172.21.15.26:0/483691062 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e1c08b0e80 0x55e1c08c8f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:11.383 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.383+0000 7ff7177fe640 1 -- 172.21.15.26:0/483691062 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e1c08a65e0 msgr2=0x55e1c08c8a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:33:11.383 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.383+0000 7ff7177fe640 1 --2- 172.21.15.26:0/483691062 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e1c08a65e0 0x55e1c08c8a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:11.384 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.383+0000 7ff7177fe640 1 -- 172.21.15.26:0/483691062 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e1c08c3760 con 0x55e1c08a5cc0 2024-09-11T23:33:11.384 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.383+0000 7ff717fff640 1 --2- 172.21.15.26:0/483691062 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e1c08b0e80 0x55e1c08c8f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:33:11.384 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.383+0000 7ff7177fe640 1 --2- 172.21.15.26:0/483691062 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e1c08a5cc0 0x55e1c08ae6c0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ff70c0098c0 tx=0x7ff70c009d80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b4c559cebe03a974 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:33:11.384 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.383+0000 7ff716ffd640 1 --2- 172.21.15.26:0/483691062 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e1c08a65e0 0x55e1c08c8a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:33:11.384 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.384+0000 7ff6fd7fa640 1 -- 172.21.15.26:0/483691062 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff70c022070 con 0x55e1c08a5cc0 2024-09-11T23:33:11.384 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.384+0000 7ff6fd7fa640 1 -- 172.21.15.26:0/483691062 <== mon.2 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff70c012d00 con 0x55e1c08a5cc0 2024-09-11T23:33:11.385 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.384+0000 7ff6fd7fa640 1 -- 172.21.15.26:0/483691062 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff70c01de30 con 0x55e1c08a5cc0 2024-09-11T23:33:11.385 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.385+0000 7ff720e35740 1 -- 172.21.15.26:0/483691062 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e1c08a5cc0 msgr2=0x55e1c08ae6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:33:11.386 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.385+0000 7ff720e35740 1 --2- 172.21.15.26:0/483691062 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e1c08a5cc0 0x55e1c08ae6c0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ff70c0098c0 tx=0x7ff70c009d80 comp rx=0 tx=0).stop 2024-09-11T23:33:11.386 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.386+0000 7ff720e35740 1 -- 172.21.15.26:0/483691062 shutdown_connections 2024-09-11T23:33:11.386 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.386+0000 7ff720e35740 1 --2- 172.21.15.26:0/483691062 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e1c08b0e80 0x55e1c08c8f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:11.386 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.386+0000 7ff720e35740 1 --2- 172.21.15.26:0/483691062 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e1c08a65e0 0x55e1c08c8a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:11.386 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.386+0000 7ff720e35740 1 --2- 172.21.15.26:0/483691062 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e1c08a5cc0 0x55e1c08ae6c0 secure :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ff70c0098c0 tx=0x7ff70c009d80 comp rx=0 tx=0).stop 2024-09-11T23:33:11.386 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.386+0000 7ff720e35740 1 -- 172.21.15.26:0/483691062 >> 172.21.15.26:0/483691062 conn(0x55e1c08a2230 msgr2=0x55e1c08a9020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:33:11.389 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.389+0000 7ff720e35740 1 -- 172.21.15.26:0/483691062 shutdown_connections 2024-09-11T23:33:11.390 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.390+0000 7ff720e35740 1 -- 172.21.15.26:0/483691062 wait complete. 2024-09-11T23:33:11.390 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.391+0000 7ff720e35740 1 -- start start 2024-09-11T23:33:11.391 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.391+0000 7ff720e35740 1 Processor -- start 2024-09-11T23:33:11.392 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.392+0000 7ff720e35740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e1c08a65e0 0x55e1c08b8f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:11.392 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.392+0000 7ff720e35740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e1c08b0e80 0x55e1c08b9440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:11.392 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.393+0000 7ff720e35740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e1c08aa430 0x55e1c08aa8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:11.393 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.393+0000 7ff716ffd640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e1c08b0e80 0x55e1c08b9440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:33:11.393 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.393+0000 7ff716ffd640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e1c08b0e80 0x55e1c08b9440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:43778/0 (socket says 172.21.15.26:43778) 2024-09-11T23:33:11.393 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.393+0000 7ff716ffd640 1 -- 172.21.15.26:0/2488782982 learned_addr learned my addr 172.21.15.26:0/2488782982 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:33:11.393 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.393+0000 7ff7177fe640 1 --2- 172.21.15.26:0/2488782982 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e1c08a65e0 0x55e1c08b8f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:33:11.393 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.393+0000 7ff717fff640 1 --2- 172.21.15.26:0/2488782982 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e1c08aa430 0x55e1c08aa8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:33:11.393 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.393+0000 7ff720e35740 1 -- 172.21.15.26:0/2488782982 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff7181c6370 con 0x55e1c08a65e0 2024-09-11T23:33:11.393 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.393+0000 7ff720e35740 1 -- 172.21.15.26:0/2488782982 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7ff7181c5be0 con 0x55e1c08aa430 2024-09-11T23:33:11.393 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.393+0000 7ff720e35740 1 -- 172.21.15.26:0/2488782982 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ff7181abdc0 con 0x55e1c08b0e80 2024-09-11T23:33:11.393 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.393+0000 7ff720e35740 10 client.0 _ll_register_callbacks cb 0x7ff70801ff50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:33:11.395 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.394+0000 7ff716ffd640 1 -- 172.21.15.26:0/2488782982 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e1c08aa430 msgr2=0x55e1c08aa8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:33:11.395 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.394+0000 7ff716ffd640 1 --2- 172.21.15.26:0/2488782982 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e1c08aa430 0x55e1c08aa8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:11.395 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.394+0000 7ff716ffd640 1 -- 172.21.15.26:0/2488782982 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e1c08a65e0 msgr2=0x55e1c08b8f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:33:11.395 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.394+0000 7ff716ffd640 1 --2- 172.21.15.26:0/2488782982 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e1c08a65e0 0x55e1c08b8f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:11.395 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.394+0000 7ff716ffd640 1 -- 172.21.15.26:0/2488782982 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e1c08aadf0 con 0x55e1c08b0e80 2024-09-11T23:33:11.395 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.394+0000 7ff716ffd640 1 --2- 172.21.15.26:0/2488782982 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e1c08b0e80 0x55e1c08b9440 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ff708000c00 tx=0x7ff70800be40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e9bc0bd94a6f399d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:33:11.396 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.394+0000 7ff6f6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-09-11T23:33:11.396 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.395+0000 7ff6f6ffd640 1 -- 172.21.15.26:0/2488782982 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff708023070 con 0x55e1c08b0e80 2024-09-11T23:33:11.396 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.395+0000 7ff6f6ffd640 1 -- 172.21.15.26:0/2488782982 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8586+0+0 (secure 0 0 0) 0x7ff70800d920 con 0x55e1c08b0e80 2024-09-11T23:33:11.396 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.395+0000 7ff720e35740 1 -- 172.21.15.26:0/2488782982 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff70c012d00 con 0x55e1c08b0e80 2024-09-11T23:33:11.396 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.396+0000 7ff6f6ffd640 1 -- 172.21.15.26:0/2488782982 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) ==== 24+0+0 (secure 0 0 0) 0x7ff70800db20 con 0x55e1c08b0e80 2024-09-11T23:33:11.396 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.396+0000 7ff720e35740 10 client.27016 fetch_fsmap learned FSMap version 72 2024-09-11T23:33:11.396 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.396+0000 7ff720e35740 1 -- 172.21.15.26:0/2488782982 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) -- 0x55e1c092b580 con 0x55e1c08b0e80 2024-09-11T23:33:11.399 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.399+0000 7ff6f6ffd640 1 -- 172.21.15.26:0/2488782982 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff70c012d00 con 0x55e1c08b0e80 2024-09-11T23:33:11.399 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.400+0000 7ff6f6ffd640 1 -- 172.21.15.26:0/2488782982 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 72) ==== 8426+0+0 (secure 0 0 0) 0x7ff708030900 con 0x55e1c08b0e80 2024-09-11T23:33:11.399 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.400+0000 7ff720e35740 10 client.27016 fetch_fsmap finished waiting for FSMap version 72 2024-09-11T23:33:11.399 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.400+0000 7ff720e35740 10 client.27016 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:33:11.399 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.400+0000 7ff720e35740 20 client.27016 populate_metadata read hostname 'smithi026' 2024-09-11T23:33:11.399 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.400+0000 7ff720e35740 1 --2- 172.21.15.26:0/2488782982 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55e1c092bc90 0x55e1c092e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:11.400 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.400+0000 7ff720e35740 4 client.27016 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T23:33:11.400 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.400+0000 7ff720e35740 1 -- 172.21.15.26:0/2488782982 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e1c0811d70 con 0x55e1c092bc90 2024-09-11T23:33:11.402 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.402+0000 7ff7177fe640 1 --2- 172.21.15.26:0/2488782982 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55e1c092bc90 0x55e1c092e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:33:11.406 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.404+0000 7ff7177fe640 1 --2- 172.21.15.26:0/2488782982 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55e1c092bc90 0x55e1c092e0b0 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=320f517acf203534 server_cookie=d3fd6683f2b89818 in_seq=0 out_seq=0 2024-09-11T23:33:11.406 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.404+0000 7ff6f6ffd640 10 client.27016 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:33:11.481 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.481+0000 7ff6f6ffd640 1 -- 172.21.15.26:0/2488782982 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e1c0811d70 con 0x55e1c092bc90 2024-09-11T23:33:11.481 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.481+0000 7ff6f6ffd640 10 client.27016 handle_command_reply: tid=0 2024-09-11T23:33:11.481 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.481+0000 7ff720e35740 1 client.27016 shutdown 2024-09-11T23:33:11.482 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.481+0000 7ff720e35740 1 -- 172.21.15.26:0/2488782982 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e1c08b0e80 msgr2=0x55e1c08b9440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:33:11.482 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.481+0000 7ff720e35740 1 --2- 172.21.15.26:0/2488782982 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e1c08b0e80 0x55e1c08b9440 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ff708000c00 tx=0x7ff70800be40 comp rx=0 tx=0).stop 2024-09-11T23:33:11.482 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.481+0000 7ff720e35740 1 -- 172.21.15.26:0/2488782982 shutdown_connections 2024-09-11T23:33:11.482 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.481+0000 7ff720e35740 1 --2- 172.21.15.26:0/2488782982 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e1c08aa430 0x55e1c08aa8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:11.482 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.481+0000 7ff720e35740 1 --2- 172.21.15.26:0/2488782982 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55e1c092bc90 0x55e1c092e0b0 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:11.482 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.481+0000 7ff720e35740 1 --2- 172.21.15.26:0/2488782982 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e1c08b0e80 0x55e1c08b9440 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:11.482 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.481+0000 7ff6f6ffd640 0 client.27016 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:33:11.482 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.481+0000 7ff720e35740 1 --2- 172.21.15.26:0/2488782982 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e1c08a65e0 0x55e1c08b8f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:11.482 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.481+0000 7ff720e35740 1 -- 172.21.15.26:0/2488782982 >> 172.21.15.26:0/2488782982 conn(0x55e1c08a2230 msgr2=0x55e1c08a0a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:33:11.482 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.481+0000 7ff720e35740 1 -- 172.21.15.26:0/2488782982 shutdown_connections 2024-09-11T23:33:11.482 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.481+0000 7ff720e35740 1 -- 172.21.15.26:0/2488782982 wait complete. 2024-09-11T23:33:11.482 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.481+0000 7ff720e35740 20 client.27016 trim_cache size 0 max 16384 2024-09-11T23:33:11.483 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-09-11T23:33:11.483 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "message": "", 2024-09-11T23:33:11.483 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-09-11T23:33:11.483 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-09-11T23:33:11.484 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.482+0000 7ff71cd5b640 1 -- 172.21.15.26:0/400373080 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7ff7000a2f00 msgr2=0x7ff7000a53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:33:11.484 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.482+0000 7ff71cd5b640 1 --2- 172.21.15.26:0/400373080 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7ff7000a2f00 0x7ff7000a53c0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ff7080058c0 tx=0x7ff7080129b0 comp rx=0 tx=0).stop 2024-09-11T23:33:11.484 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.482+0000 7ff71cd5b640 1 -- 172.21.15.26:0/400373080 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff718114ab0 msgr2=0x7ff71810ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:33:11.484 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.482+0000 7ff71cd5b640 1 --2- 172.21.15.26:0/400373080 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff718114ab0 0x7ff71810ae50 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ff71002aee0 tx=0x7ff710030820 comp rx=0 tx=0).stop 2024-09-11T23:33:11.484 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.482+0000 7ff71cd5b640 1 -- 172.21.15.26:0/400373080 shutdown_connections 2024-09-11T23:33:11.484 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.482+0000 7ff71cd5b640 1 --2- 172.21.15.26:0/400373080 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7ff7000a2f00 0x7ff7000a53c0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:11.484 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.482+0000 7ff71cd5b640 1 --2- 172.21.15.26:0/400373080 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff718114ab0 0x7ff71810ae50 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:11.484 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.482+0000 7ff71cd5b640 1 --2- 172.21.15.26:0/400373080 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff718109b10 0x7ff71810a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:11.484 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.482+0000 7ff71cd5b640 1 --2- 172.21.15.26:0/400373080 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff7181091f0 0x7ff71810a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:11.484 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.482+0000 7ff71cd5b640 1 -- 172.21.15.26:0/400373080 >> 172.21.15.26:0/400373080 conn(0x7ff71806cbf0 msgr2=0x7ff718070700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:33:11.484 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.483+0000 7ff71cd5b640 1 -- 172.21.15.26:0/400373080 shutdown_connections 2024-09-11T23:33:11.485 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:11.483+0000 7ff71cd5b640 1 -- 172.21.15.26:0/400373080 wait complete. 2024-09-11T23:33:11.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:33:11 smithi136 ceph-mon[35121]: pgmap v464: 97 pgs: 14 active+clean+snaptrim_wait, 8 active+clean+snaptrim, 75 active+clean; 2.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 948 KiB/s rd, 971 KiB/s wr, 86 op/s 2024-09-11T23:33:11.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:33:11 smithi026 ceph-mon[31172]: pgmap v464: 97 pgs: 14 active+clean+snaptrim_wait, 8 active+clean+snaptrim, 75 active+clean; 2.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 948 KiB/s rd, 971 KiB/s wr, 86 op/s 2024-09-11T23:33:11.552 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:33:11 smithi076 ceph-mon[35296]: pgmap v464: 97 pgs: 14 active+clean+snaptrim_wait, 8 active+clean+snaptrim, 75 active+clean; 2.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 948 KiB/s rd, 971 KiB/s wr, 86 op/s 2024-09-11T23:33:13.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:33:13 smithi026 ceph-mon[31172]: pgmap v465: 97 pgs: 19 active+clean+snaptrim_wait, 10 active+clean+snaptrim, 68 active+clean; 2.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 948 KiB/s rd, 971 KiB/s wr, 95 op/s 2024-09-11T23:33:13.802 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:33:13 smithi076 ceph-mon[35296]: pgmap v465: 97 pgs: 19 active+clean+snaptrim_wait, 10 active+clean+snaptrim, 68 active+clean; 2.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 948 KiB/s rd, 971 KiB/s wr, 95 op/s 2024-09-11T23:33:13.988 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:33:13 smithi136 ceph-mon[35121]: pgmap v465: 97 pgs: 19 active+clean+snaptrim_wait, 10 active+clean+snaptrim, 68 active+clean; 2.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 948 KiB/s rd, 971 KiB/s wr, 95 op/s 2024-09-11T23:33:14.802 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:33:14 smithi076 ceph-mon[35296]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:33:14.988 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:33:14 smithi136 ceph-mon[35121]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:33:15.000 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:33:14 smithi026 ceph-mon[31172]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:33:15.802 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:33:15 smithi076 ceph-mon[35296]: pgmap v466: 97 pgs: 41 active+clean+snaptrim_wait, 23 active+clean+snaptrim, 33 active+clean; 2.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 130 op/s 2024-09-11T23:33:15.988 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:33:15 smithi136 ceph-mon[35121]: pgmap v466: 97 pgs: 41 active+clean+snaptrim_wait, 23 active+clean+snaptrim, 33 active+clean; 2.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 130 op/s 2024-09-11T23:33:16.000 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:33:15 smithi026 ceph-mon[31172]: pgmap v466: 97 pgs: 41 active+clean+snaptrim_wait, 23 active+clean+snaptrim, 33 active+clean; 2.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 130 op/s 2024-09-11T23:33:17.802 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:33:17 smithi076 ceph-mon[35296]: pgmap v467: 97 pgs: 41 active+clean+snaptrim_wait, 23 active+clean+snaptrim, 33 active+clean; 2.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 130 op/s 2024-09-11T23:33:17.988 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:33:17 smithi136 ceph-mon[35121]: pgmap v467: 97 pgs: 41 active+clean+snaptrim_wait, 23 active+clean+snaptrim, 33 active+clean; 2.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 130 op/s 2024-09-11T23:33:18.000 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:33:17 smithi026 ceph-mon[31172]: pgmap v467: 97 pgs: 41 active+clean+snaptrim_wait, 23 active+clean+snaptrim, 33 active+clean; 2.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 130 op/s 2024-09-11T23:33:19.802 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:33:19 smithi076 ceph-mon[35296]: pgmap v468: 97 pgs: 41 active+clean+snaptrim_wait, 23 active+clean+snaptrim, 33 active+clean; 2.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 130 op/s 2024-09-11T23:33:19.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:33:19 smithi136 ceph-mon[35121]: pgmap v468: 97 pgs: 41 active+clean+snaptrim_wait, 23 active+clean+snaptrim, 33 active+clean; 2.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 130 op/s 2024-09-11T23:33:20.000 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:33:19 smithi026 ceph-mon[31172]: pgmap v468: 97 pgs: 41 active+clean+snaptrim_wait, 23 active+clean+snaptrim, 33 active+clean; 2.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 130 op/s 2024-09-11T23:33:21.988 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:33:21 smithi136 ceph-mon[35121]: pgmap v469: 97 pgs: 41 active+clean+snaptrim_wait, 23 active+clean+snaptrim, 33 active+clean; 2.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 932 KiB/s rd, 921 KiB/s wr, 94 op/s 2024-09-11T23:33:22.000 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:33:21 smithi026 ceph-mon[31172]: pgmap v469: 97 pgs: 41 active+clean+snaptrim_wait, 23 active+clean+snaptrim, 33 active+clean; 2.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 932 KiB/s rd, 921 KiB/s wr, 94 op/s 2024-09-11T23:33:22.052 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:33:21 smithi076 ceph-mon[35296]: pgmap v469: 97 pgs: 41 active+clean+snaptrim_wait, 23 active+clean+snaptrim, 33 active+clean; 2.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 932 KiB/s rd, 921 KiB/s wr, 94 op/s 2024-09-11T23:33:22.750 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:33:22 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: 2024-09-11T23:33:22.485+0000 7f7ccf119640 -1 calc_pg_upmaps abort due to max <= 0 2024-09-11T23:33:23.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:23.627+0000 7fe6d2ffd640 1 -- 172.21.15.26:0/566286320 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x7fe6e80bcc50 con 0x55d500c637b0 2024-09-11T23:33:23.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:23.628+0000 7fe6d2ffd640 10 client.17496 handle_command_reply: tid=0 2024-09-11T23:33:23.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:23.628+0000 7fe6fc0ba740 1 client.17496 shutdown 2024-09-11T23:33:23.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:23.629+0000 7fe6fc0ba740 1 -- 172.21.15.26:0/566286320 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d500bdd220 msgr2=0x55d500bebe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:33:23.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:23.629+0000 7fe6fc0ba740 1 --2- 172.21.15.26:0/566286320 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d500bdd220 0x55d500bebe30 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fe6ec020990 tx=0x7fe6ec017ae0 comp rx=0 tx=0).stop 2024-09-11T23:33:23.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:23.629+0000 7fe6fc0ba740 1 -- 172.21.15.26:0/566286320 shutdown_connections 2024-09-11T23:33:23.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:23.629+0000 7fe6fc0ba740 1 --2- 172.21.15.26:0/566286320 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55d500c637b0 0x55d500c65bd0 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:23.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:23.629+0000 7fe6fc0ba740 1 --2- 172.21.15.26:0/566286320 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d500be7ae0 0x55d500bec370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:23.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:23.629+0000 7fe6d2ffd640 0 client.17496 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:33:23.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:23.629+0000 7fe6fc0ba740 1 --2- 172.21.15.26:0/566286320 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d500bdd220 0x55d500bebe30 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:23.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:23.629+0000 7fe6fc0ba740 1 --2- 172.21.15.26:0/566286320 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d500bdc900 0x55d500beb8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:23.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:23.629+0000 7fe6fc0ba740 1 -- 172.21.15.26:0/566286320 >> 172.21.15.26:0/566286320 conn(0x55d500bd8e90 msgr2=0x55d500be6a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:33:23.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:23.630+0000 7fe6fc0ba740 1 -- 172.21.15.26:0/566286320 shutdown_connections 2024-09-11T23:33:23.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:23.630+0000 7fe6fc0ba740 1 -- 172.21.15.26:0/566286320 wait complete. 2024-09-11T23:33:23.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:23.630+0000 7fe6fc0ba740 20 client.17496 trim_cache size 0 max 16384 2024-09-11T23:33:23.632 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-11T23:33:23.633 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 73, 2024-09-11T23:33:23.633 INFO:teuthology.orchestra.run.smithi026.stdout: "leader": 24485, 2024-09-11T23:33:23.633 INFO:teuthology.orchestra.run.smithi026.stdout: "set_version": 54, 2024-09-11T23:33:23.633 INFO:teuthology.orchestra.run.smithi026.stdout: "sets": { 2024-09-11T23:33:23.633 INFO:teuthology.orchestra.run.smithi026.stdout: "c2054eab": { 2024-09-11T23:33:23.633 INFO:teuthology.orchestra.run.smithi026.stdout: "version": 54, 2024-09-11T23:33:23.633 INFO:teuthology.orchestra.run.smithi026.stdout: "age_ref": 0.0, 2024-09-11T23:33:23.633 INFO:teuthology.orchestra.run.smithi026.stdout: "state": { 2024-09-11T23:33:23.633 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "QUIESCED", 2024-09-11T23:33:23.633 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.0 2024-09-11T23:33:23.633 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:33:23.634 INFO:teuthology.orchestra.run.smithi026.stdout: "timeout": 90.0, 2024-09-11T23:33:23.634 INFO:teuthology.orchestra.run.smithi026.stdout: "expiration": 150.0, 2024-09-11T23:33:23.634 INFO:teuthology.orchestra.run.smithi026.stdout: "members": { 2024-09-11T23:33:23.634 INFO:teuthology.orchestra.run.smithi026.stdout: "file:/": { 2024-09-11T23:33:23.634 INFO:teuthology.orchestra.run.smithi026.stdout: "excluded": false, 2024-09-11T23:33:23.634 INFO:teuthology.orchestra.run.smithi026.stdout: "state": { 2024-09-11T23:33:23.634 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "QUIESCED", 2024-09-11T23:33:23.634 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.0 2024-09-11T23:33:23.634 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:33:23.634 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:33:23.634 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:33:23.635 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:33:23.635 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:33:23.635 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-11T23:33:23.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:23.631+0000 7fe6f9ce1640 1 -- 172.21.15.26:0/1763176668 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fe6e40a2ec0 msgr2=0x7fe6e40a5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:33:23.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:23.631+0000 7fe6f9ce1640 1 --2- 172.21.15.26:0/1763176668 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fe6e40a2ec0 0x7fe6e40a5380 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fe6e00095f0 tx=0x7fe6e00046c0 comp rx=0 tx=0).stop 2024-09-11T23:33:23.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:23.631+0000 7fe6f9ce1640 1 -- 172.21.15.26:0/1763176668 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe6f4109450 msgr2=0x7fe6f41a7700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:33:23.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:23.631+0000 7fe6f9ce1640 1 --2- 172.21.15.26:0/1763176668 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe6f4109450 0x7fe6f41a7700 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fe6e802a860 tx=0x7fe6e8033b60 comp rx=0 tx=0).stop 2024-09-11T23:33:23.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:23.631+0000 7fe6f9ce1640 1 -- 172.21.15.26:0/1763176668 shutdown_connections 2024-09-11T23:33:23.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:23.631+0000 7fe6f9ce1640 1 --2- 172.21.15.26:0/1763176668 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fe6e40a2ec0 0x7fe6e40a5380 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:23.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:23.631+0000 7fe6f9ce1640 1 --2- 172.21.15.26:0/1763176668 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe6f410ea10 0x7fe6f4114590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:23.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:23.631+0000 7fe6f9ce1640 1 --2- 172.21.15.26:0/1763176668 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe6f4109e00 0x7fe6f41a7c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:23.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:23.631+0000 7fe6f9ce1640 1 --2- 172.21.15.26:0/1763176668 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe6f4109450 0x7fe6f41a7700 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:23.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:23.631+0000 7fe6f9ce1640 1 -- 172.21.15.26:0/1763176668 >> 172.21.15.26:0/1763176668 conn(0x7fe6f4073660 msgr2=0x7fe6f4117680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:33:23.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:23.632+0000 7fe6f9ce1640 1 -- 172.21.15.26:0/1763176668 shutdown_connections 2024-09-11T23:33:23.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:23.632+0000 7fe6f9ce1640 1 -- 172.21.15.26:0/1763176668 wait complete. 2024-09-11T23:33:23.750 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:33:23 smithi026 ceph-mon[31172]: pgmap v470: 97 pgs: 41 active+clean+snaptrim_wait, 23 active+clean+snaptrim, 33 active+clean; 2.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 186 KiB/s wr, 67 op/s 2024-09-11T23:33:23.988 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:33:23 smithi136 ceph-mon[35121]: pgmap v470: 97 pgs: 41 active+clean+snaptrim_wait, 23 active+clean+snaptrim, 33 active+clean; 2.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 186 KiB/s wr, 67 op/s 2024-09-11T23:33:24.052 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:33:23 smithi076 ceph-mon[35296]: pgmap v470: 97 pgs: 41 active+clean+snaptrim_wait, 23 active+clean+snaptrim, 33 active+clean; 2.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 186 KiB/s wr, 67 op/s 2024-09-11T23:33:24.631 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.630+0000 7f4cede8d640 1 -- 172.21.15.26:0/3886749684 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4ce8114ab0 msgr2=0x7f4ce8116ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:33:24.631 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.630+0000 7f4cede8d640 1 --2- 172.21.15.26:0/3886749684 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4ce8114ab0 0x7f4ce8116ea0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f4cd8009900 tx=0x7f4cd802a420 comp rx=0 tx=0).stop 2024-09-11T23:33:24.631 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.630+0000 7f4cede8d640 1 -- 172.21.15.26:0/3886749684 shutdown_connections 2024-09-11T23:33:24.631 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.630+0000 7f4cede8d640 1 --2- 172.21.15.26:0/3886749684 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4ce8109b10 0x7f4ce8114570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:24.631 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.630+0000 7f4cede8d640 1 --2- 172.21.15.26:0/3886749684 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4ce8114ab0 0x7f4ce8116ea0 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:24.631 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.630+0000 7f4cede8d640 1 --2- 172.21.15.26:0/3886749684 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4ce81091f0 0x7f4ce81095d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:24.632 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.630+0000 7f4cede8d640 1 -- 172.21.15.26:0/3886749684 >> 172.21.15.26:0/3886749684 conn(0x7f4ce806ce00 msgr2=0x7f4ce806d210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:33:24.632 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.631+0000 7f4cede8d640 1 -- 172.21.15.26:0/3886749684 shutdown_connections 2024-09-11T23:33:24.633 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.633+0000 7f4cede8d640 1 -- 172.21.15.26:0/3886749684 wait complete. 2024-09-11T23:33:24.634 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.633+0000 7f4cede8d640 1 Processor -- start 2024-09-11T23:33:24.634 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.634+0000 7f4cede8d640 1 -- start start 2024-09-11T23:33:24.635 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.634+0000 7f4cede8d640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4ce81091f0 0x7f4ce81a8940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:24.635 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.634+0000 7f4cede8d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4ce8109b10 0x7f4ce81a8e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:24.635 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.634+0000 7f4cede8d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4ce8114ab0 0x7f4ce81a2af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:24.635 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.634+0000 7f4cede8d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f4ce811a690 con 0x7f4ce8109b10 2024-09-11T23:33:24.635 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.634+0000 7f4cede8d640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f4ce811a510 con 0x7f4ce8114ab0 2024-09-11T23:33:24.635 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.634+0000 7f4cede8d640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f4ce811a810 con 0x7f4ce81091f0 2024-09-11T23:33:24.636 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.635+0000 7f4ce7fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4ce8114ab0 0x7f4ce81a2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:33:24.636 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.635+0000 7f4ce7fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4ce8114ab0 0x7f4ce81a2af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:51794/0 (socket says 172.21.15.26:51794) 2024-09-11T23:33:24.636 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.635+0000 7f4ce7fff640 1 -- 172.21.15.26:0/2154887355 learned_addr learned my addr 172.21.15.26:0/2154887355 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:33:24.637 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.635+0000 7f4ce6ffd640 1 --2- 172.21.15.26:0/2154887355 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4ce8109b10 0x7f4ce81a8e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:33:24.637 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.635+0000 7f4ce7fff640 1 -- 172.21.15.26:0/2154887355 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4ce81091f0 msgr2=0x7f4ce81a8940 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-11T23:33:24.637 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.635+0000 7f4ce7fff640 1 --2- 172.21.15.26:0/2154887355 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4ce81091f0 0x7f4ce81a8940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:24.637 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.635+0000 7f4ce7fff640 1 -- 172.21.15.26:0/2154887355 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4ce8109b10 msgr2=0x7f4ce81a8e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:33:24.637 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.635+0000 7f4ce7fff640 1 --2- 172.21.15.26:0/2154887355 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4ce8109b10 0x7f4ce81a8e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:24.637 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.635+0000 7f4ce7fff640 1 -- 172.21.15.26:0/2154887355 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ce81a3270 con 0x7f4ce8114ab0 2024-09-11T23:33:24.637 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.636+0000 7f4ce7fff640 1 --2- 172.21.15.26:0/2154887355 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4ce8114ab0 0x7f4ce81a2af0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f4cd8006010 tx=0x7f4cd8005230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:33:24.637 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.636+0000 7f4ce4ff9640 1 -- 172.21.15.26:0/2154887355 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cd8040030 con 0x7f4ce8114ab0 2024-09-11T23:33:24.638 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.636+0000 7f4cede8d640 1 -- 172.21.15.26:0/2154887355 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ce81a3500 con 0x7f4ce8114ab0 2024-09-11T23:33:24.638 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.636+0000 7f4cede8d640 1 -- 172.21.15.26:0/2154887355 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ce81af800 con 0x7f4ce8114ab0 2024-09-11T23:33:24.638 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.637+0000 7f4ce4ff9640 1 -- 172.21.15.26:0/2154887355 <== mon.2 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4cd8005530 con 0x7f4ce8114ab0 2024-09-11T23:33:24.638 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.637+0000 7f4ce4ff9640 1 -- 172.21.15.26:0/2154887355 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cd80316a0 con 0x7f4ce8114ab0 2024-09-11T23:33:24.642 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.641+0000 7f4ce4ff9640 1 -- 172.21.15.26:0/2154887355 <== mon.2 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f4cd8043030 con 0x7f4ce8114ab0 2024-09-11T23:33:24.642 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.641+0000 7f4cf0c27740 1 Processor -- start 2024-09-11T23:33:24.643 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.642+0000 7f4ce4ff9640 1 --2- 172.21.15.26:0/2154887355 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f4cd00a2ef0 0x7f4cd00a53b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:24.643 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.642+0000 7f4ce4ff9640 1 -- 172.21.15.26:0/2154887355 <== mon.2 v2:172.21.15.76:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8586+0+0 (secure 0 0 0) 0x7f4cd80efb60 con 0x7f4ce8114ab0 2024-09-11T23:33:24.643 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.642+0000 7f4cf0c27740 1 -- start start 2024-09-11T23:33:24.643 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.643+0000 7f4cf0c27740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558b449ed300 0x558b449ed6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:24.644 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.643+0000 7f4cf0c27740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558b449edc20 0x558b449f7f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:24.644 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.643+0000 7f4cf0c27740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558b449f84c0 0x558b449fa8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:24.644 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.643+0000 7f4cf0c27740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x558b449492f0 con 0x558b449edc20 2024-09-11T23:33:24.644 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.643+0000 7f4cf0c27740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x558b448f1270 con 0x558b449f84c0 2024-09-11T23:33:24.644 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.643+0000 7f4cf0c27740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x558b449452b0 con 0x558b449ed300 2024-09-11T23:33:24.645 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.643+0000 7f4ce6ffd640 1 --2- 172.21.15.26:0/2154887355 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f4cd00a2ef0 0x7f4cd00a53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:33:24.645 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.643+0000 7f4ce6ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558b449edc20 0x558b449f7f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:33:24.645 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.643+0000 7f4ce6ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558b449edc20 0x558b449f7f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49888/0 (socket says 172.21.15.26:49888) 2024-09-11T23:33:24.645 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.643+0000 7f4ce6ffd640 1 -- 172.21.15.26:0/466026556 learned_addr learned my addr 172.21.15.26:0/466026556 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:33:24.645 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.643+0000 7f4ce77fe640 1 --2- 172.21.15.26:0/466026556 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558b449ed300 0x558b449ed6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:33:24.645 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.644+0000 7f4ce6ffd640 1 -- 172.21.15.26:0/466026556 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558b449ed300 msgr2=0x558b449ed6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:33:24.645 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.644+0000 7f4ce6ffd640 1 --2- 172.21.15.26:0/466026556 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558b449ed300 0x558b449ed6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:24.646 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.644+0000 7f4ce6ffd640 1 -- 172.21.15.26:0/466026556 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558b449f84c0 msgr2=0x558b449fa8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:33:24.646 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.644+0000 7f4ce6ffd640 1 --2- 172.21.15.26:0/466026556 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558b449f84c0 0x558b449fa8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:24.646 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.644+0000 7f4ce6ffd640 1 -- 172.21.15.26:0/466026556 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b448f0340 con 0x558b449edc20 2024-09-11T23:33:24.646 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.644+0000 7f4ce6ffd640 1 --2- 172.21.15.26:0/2154887355 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f4cd00a2ef0 0x7f4cd00a53b0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f4cdc005ce0 tx=0x7f4cdc00bb50 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:33:24.646 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.644+0000 7f4ce6ffd640 1 --2- 172.21.15.26:0/466026556 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558b449edc20 0x558b449f7f80 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f4cdc010740 tx=0x7f4cdc01c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85cae8f501df0a9a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:33:24.646 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.644+0000 7f4ccdffb640 1 -- 172.21.15.26:0/466026556 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cdc024070 con 0x558b449edc20 2024-09-11T23:33:24.646 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.645+0000 7f4ccdffb640 1 -- 172.21.15.26:0/466026556 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4cdc010e00 con 0x558b449edc20 2024-09-11T23:33:24.646 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.645+0000 7f4ccdffb640 1 -- 172.21.15.26:0/466026556 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cdc020a60 con 0x558b449edc20 2024-09-11T23:33:24.649 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.646+0000 7f4cf0c27740 1 -- 172.21.15.26:0/466026556 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558b449edc20 msgr2=0x558b449f7f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:33:24.650 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.646+0000 7f4cf0c27740 1 --2- 172.21.15.26:0/466026556 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558b449edc20 0x558b449f7f80 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f4cdc010740 tx=0x7f4cdc01c000 comp rx=0 tx=0).stop 2024-09-11T23:33:24.650 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.647+0000 7f4cf0c27740 1 -- 172.21.15.26:0/466026556 shutdown_connections 2024-09-11T23:33:24.650 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.647+0000 7f4cf0c27740 1 --2- 172.21.15.26:0/466026556 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558b449f84c0 0x558b449fa8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:24.650 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.647+0000 7f4cf0c27740 1 --2- 172.21.15.26:0/466026556 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558b449edc20 0x558b449f7f80 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:24.650 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.647+0000 7f4cf0c27740 1 --2- 172.21.15.26:0/466026556 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558b449ed300 0x558b449ed6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:24.650 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.647+0000 7f4cf0c27740 1 -- 172.21.15.26:0/466026556 >> 172.21.15.26:0/466026556 conn(0x558b449e9870 msgr2=0x558b449e9c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:33:24.650 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.648+0000 7f4cf0c27740 1 -- 172.21.15.26:0/466026556 shutdown_connections 2024-09-11T23:33:24.650 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.648+0000 7f4cf0c27740 1 -- 172.21.15.26:0/466026556 wait complete. 2024-09-11T23:33:24.650 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.649+0000 7f4cf0c27740 1 -- start start 2024-09-11T23:33:24.650 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.649+0000 7f4cf0c27740 1 Processor -- start 2024-09-11T23:33:24.651 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.650+0000 7f4cf0c27740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558b449ed300 0x558b449f5380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:24.651 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.650+0000 7f4cf0c27740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558b449edc20 0x558b449f58c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:24.651 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.650+0000 7f4cf0c27740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558b449f84c0 0x558b449f5e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:24.651 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.650+0000 7f4cf0c27740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x558b449fb090 con 0x558b449edc20 2024-09-11T23:33:24.652 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.650+0000 7f4cf0c27740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x558b448d2290 con 0x558b449ed300 2024-09-11T23:33:24.652 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.650+0000 7f4cf0c27740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x558b449fb210 con 0x558b449f84c0 2024-09-11T23:33:24.652 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.650+0000 7f4cf0c27740 10 client.0 _ll_register_callbacks cb 0x558b44903140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:33:24.652 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.650+0000 7f4ce7fff640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558b449f84c0 0x558b449f5e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:33:24.652 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.650+0000 7f4ce7fff640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558b449f84c0 0x558b449f5e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:41978/0 (socket says 172.21.15.26:41978) 2024-09-11T23:33:24.652 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.650+0000 7f4ce7fff640 1 -- 172.21.15.26:0/2523191296 learned_addr learned my addr 172.21.15.26:0/2523191296 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:33:24.652 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.650+0000 7f4ce77fe640 1 --2- 172.21.15.26:0/2523191296 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558b449ed300 0x558b449f5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:33:24.652 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.651+0000 7f4ce6ffd640 1 --2- 172.21.15.26:0/2523191296 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558b449edc20 0x558b449f58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:33:24.652 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.651+0000 7f4ce7fff640 1 -- 172.21.15.26:0/2523191296 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558b449ed300 msgr2=0x558b449f5380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:33:24.652 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.651+0000 7f4ce7fff640 1 --2- 172.21.15.26:0/2523191296 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558b449ed300 0x558b449f5380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:24.652 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.651+0000 7f4ce7fff640 1 -- 172.21.15.26:0/2523191296 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558b449edc20 msgr2=0x558b449f58c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:33:24.652 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.651+0000 7f4ce7fff640 1 --2- 172.21.15.26:0/2523191296 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558b449edc20 0x558b449f58c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:24.652 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.651+0000 7f4ce7fff640 1 -- 172.21.15.26:0/2523191296 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b4465c210 con 0x558b449f84c0 2024-09-11T23:33:24.653 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.652+0000 7f4ce7fff640 1 --2- 172.21.15.26:0/2523191296 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558b449f84c0 0x558b449f5e00 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f4cd80ed270 tx=0x7f4cd80bbdb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3b5a13203fd73e9 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:33:24.653 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.652+0000 7f4cc6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-09-11T23:33:24.653 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.652+0000 7f4cc6ffd640 1 -- 172.21.15.26:0/2523191296 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cd804a070 con 0x558b449f84c0 2024-09-11T23:33:24.653 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.652+0000 7f4cc6ffd640 1 -- 172.21.15.26:0/2523191296 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8586+0+0 (secure 0 0 0) 0x7f4cd80bc240 con 0x558b449f84c0 2024-09-11T23:33:24.654 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.654+0000 7f4cf0c27740 1 -- 172.21.15.26:0/2523191296 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4cdc010e00 con 0x558b449f84c0 2024-09-11T23:33:24.656 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.655+0000 7f4cc6ffd640 1 -- 172.21.15.26:0/2523191296 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) ==== 24+0+0 (secure 0 0 0) 0x7f4cd80ed690 con 0x558b449f84c0 2024-09-11T23:33:24.656 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.656+0000 7f4cf0c27740 10 client.27025 fetch_fsmap learned FSMap version 73 2024-09-11T23:33:24.656 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.656+0000 7f4cf0c27740 1 -- 172.21.15.26:0/2523191296 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=4+}) -- 0x558b44728620 con 0x558b449f84c0 2024-09-11T23:33:24.657 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.656+0000 7f4cc6ffd640 1 -- 172.21.15.26:0/2523191296 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4cd80bc6e0 con 0x558b449f84c0 2024-09-11T23:33:24.658 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.656+0000 7f4cc6ffd640 1 -- 172.21.15.26:0/2523191296 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 73) ==== 8430+0+0 (secure 0 0 0) 0x7f4cd80b6b50 con 0x558b449f84c0 2024-09-11T23:33:24.658 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.656+0000 7f4cf0c27740 10 client.27025 fetch_fsmap finished waiting for FSMap version 73 2024-09-11T23:33:24.658 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.656+0000 7f4cf0c27740 10 client.27025 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:33:24.658 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.657+0000 7f4cf0c27740 20 client.27025 populate_metadata read hostname 'smithi026' 2024-09-11T23:33:24.658 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.657+0000 7f4cf0c27740 1 --2- 172.21.15.26:0/2523191296 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x558b44a08490 0x558b44a0a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:24.658 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.657+0000 7f4cf0c27740 4 client.27025 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:33:24.658 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.657+0000 7f4cf0c27740 1 -- 172.21.15.26:0/2523191296 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f4cdc010e00 con 0x558b44a08490 2024-09-11T23:33:24.658 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.657+0000 7f4ce77fe640 1 --2- 172.21.15.26:0/2523191296 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x558b44a08490 0x558b44a0a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:33:24.659 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.658+0000 7f4cc6ffd640 10 client.27025 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:33:24.659 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.658+0000 7f4ce77fe640 1 --2- 172.21.15.26:0/2523191296 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x558b44a08490 0x558b44a0a8b0 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cbf3e2e4c19de53e server_cookie=becd358fa9bcfc88 in_seq=0 out_seq=0 2024-09-11T23:33:24.660 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.659+0000 7f4cc6ffd640 1 -- 172.21.15.26:0/2523191296 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f4cdc010e00 con 0x558b44a08490 2024-09-11T23:33:24.660 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.659+0000 7f4cc6ffd640 10 client.27025 handle_command_reply: tid=0 2024-09-11T23:33:24.660 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.659+0000 7f4cf0c27740 1 client.27025 shutdown 2024-09-11T23:33:24.660 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.660+0000 7f4cf0c27740 1 -- 172.21.15.26:0/2523191296 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558b449f84c0 msgr2=0x558b449f5e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:33:24.660 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.660+0000 7f4cf0c27740 1 --2- 172.21.15.26:0/2523191296 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558b449f84c0 0x558b449f5e00 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f4cd80ed270 tx=0x7f4cd80bbdb0 comp rx=0 tx=0).stop 2024-09-11T23:33:24.661 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.660+0000 7f4cf0c27740 1 -- 172.21.15.26:0/2523191296 shutdown_connections 2024-09-11T23:33:24.661 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.660+0000 7f4cf0c27740 1 --2- 172.21.15.26:0/2523191296 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x558b44a08490 0x558b44a0a8b0 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:24.661 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.660+0000 7f4cf0c27740 1 --2- 172.21.15.26:0/2523191296 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558b449f84c0 0x558b449f5e00 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:24.661 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.660+0000 7f4cf0c27740 1 --2- 172.21.15.26:0/2523191296 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558b449edc20 0x558b449f58c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:24.661 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.660+0000 7f4cf0c27740 1 --2- 172.21.15.26:0/2523191296 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558b449ed300 0x558b449f5380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:24.661 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.660+0000 7f4cf0c27740 1 -- 172.21.15.26:0/2523191296 >> 172.21.15.26:0/2523191296 conn(0x558b449e9870 msgr2=0x558b449e01c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:33:24.661 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.660+0000 7f4cc6ffd640 0 client.27025 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:33:24.662 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.661+0000 7f4cf0c27740 1 -- 172.21.15.26:0/2523191296 shutdown_connections 2024-09-11T23:33:24.662 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.661+0000 7f4cf0c27740 1 -- 172.21.15.26:0/2523191296 wait complete. 2024-09-11T23:33:24.662 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.662+0000 7f4cf0c27740 20 client.27025 trim_cache size 0 max 16384 2024-09-11T23:33:24.690 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.690+0000 7f4cf0c27740 1 Processor -- start 2024-09-11T23:33:24.690 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.690+0000 7f4cf0c27740 1 -- start start 2024-09-11T23:33:24.691 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.690+0000 7f4cf0c27740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558b449ed300 0x558b449efdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:24.691 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.690+0000 7f4cf0c27740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558b449edc20 0x558b449f72f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:24.691 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.690+0000 7f4cf0c27740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558b449f84c0 0x558b44a10180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:24.691 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.690+0000 7f4ce77fe640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558b449ed300 0x558b449efdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:33:24.691 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.690+0000 7f4ce77fe640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558b449ed300 0x558b449efdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:41980/0 (socket says 172.21.15.26:41980) 2024-09-11T23:33:24.692 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.690+0000 7f4ce77fe640 1 -- 172.21.15.26:0/2505719288 learned_addr learned my addr 172.21.15.26:0/2505719288 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:33:24.692 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.690+0000 7f4cf0c27740 1 -- 172.21.15.26:0/2505719288 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f4cd400e8e0 con 0x558b449f84c0 2024-09-11T23:33:24.692 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.690+0000 7f4cf0c27740 1 -- 172.21.15.26:0/2505719288 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f4cd4010690 con 0x558b449edc20 2024-09-11T23:33:24.692 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.690+0000 7f4cf0c27740 1 -- 172.21.15.26:0/2505719288 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f4cd400efe0 con 0x558b449ed300 2024-09-11T23:33:24.692 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.691+0000 7f4ce7fff640 1 --2- 172.21.15.26:0/2505719288 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558b449f84c0 0x558b44a10180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:33:24.692 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.691+0000 7f4ce77fe640 1 -- 172.21.15.26:0/2505719288 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558b449edc20 msgr2=0x558b449f72f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:33:24.692 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.691+0000 7f4ce77fe640 1 --2- 172.21.15.26:0/2505719288 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558b449edc20 0x558b449f72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:24.692 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.691+0000 7f4ce77fe640 1 -- 172.21.15.26:0/2505719288 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558b449f84c0 msgr2=0x558b44a10180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:33:24.692 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.691+0000 7f4ce77fe640 1 --2- 172.21.15.26:0/2505719288 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558b449f84c0 0x558b44a10180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:24.692 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.691+0000 7f4ce77fe640 1 -- 172.21.15.26:0/2505719288 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b449ea9e0 con 0x558b449ed300 2024-09-11T23:33:24.692 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.691+0000 7f4ce7fff640 1 --2- 172.21.15.26:0/2505719288 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558b449f84c0 0x558b44a10180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:33:24.693 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.691+0000 7f4ce77fe640 1 --2- 172.21.15.26:0/2505719288 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558b449ed300 0x558b449efdc0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f4ce0003aa0 tx=0x7f4ce0003f60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=de563176603fb33 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:33:24.693 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.692+0000 7f4ccd7fa640 1 -- 172.21.15.26:0/2505719288 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ce001f030 con 0x558b449ed300 2024-09-11T23:33:24.694 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.692+0000 7f4ccd7fa640 1 -- 172.21.15.26:0/2505719288 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ce0004610 con 0x558b449ed300 2024-09-11T23:33:24.694 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.692+0000 7f4ccd7fa640 1 -- 172.21.15.26:0/2505719288 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ce0005020 con 0x558b449ed300 2024-09-11T23:33:24.694 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.693+0000 7f4cf0c27740 1 -- 172.21.15.26:0/2505719288 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558b449ed300 msgr2=0x558b449efdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:33:24.694 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.693+0000 7f4cf0c27740 1 --2- 172.21.15.26:0/2505719288 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558b449ed300 0x558b449efdc0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f4ce0003aa0 tx=0x7f4ce0003f60 comp rx=0 tx=0).stop 2024-09-11T23:33:24.696 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.694+0000 7f4cf0c27740 1 -- 172.21.15.26:0/2505719288 shutdown_connections 2024-09-11T23:33:24.696 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.694+0000 7f4cf0c27740 1 --2- 172.21.15.26:0/2505719288 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558b449edc20 0x558b449f72f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:24.696 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.695+0000 7f4cf0c27740 1 --2- 172.21.15.26:0/2505719288 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558b449f84c0 0x558b44a10180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:24.696 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.695+0000 7f4cf0c27740 1 --2- 172.21.15.26:0/2505719288 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558b449ed300 0x558b449efdc0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:24.696 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.695+0000 7f4cf0c27740 1 -- 172.21.15.26:0/2505719288 >> 172.21.15.26:0/2505719288 conn(0x558b449e9870 msgr2=0x558b449f6be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:33:24.696 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.695+0000 7f4cf0c27740 1 -- 172.21.15.26:0/2505719288 shutdown_connections 2024-09-11T23:33:24.696 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'c2054eab', quiesce took 17.7 seconds. Will release after: 12.3 2024-09-11T23:33:24.698 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.695+0000 7f4cf0c27740 1 -- 172.21.15.26:0/2505719288 wait complete. 2024-09-11T23:33:24.698 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.696+0000 7f4cf0c27740 1 -- start start 2024-09-11T23:33:24.699 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.696+0000 7f4cf0c27740 1 Processor -- start 2024-09-11T23:33:24.699 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.696+0000 7f4cf0c27740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558b449ed300 0x558b449fc6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:24.699 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.696+0000 7f4cf0c27740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558b449edc20 0x558b449fcc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:24.699 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.696+0000 7f4cf0c27740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558b449f84c0 0x558b44a72150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:24.699 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.696+0000 7f4cf0c27740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f4cd400ff00 con 0x558b449ed300 2024-09-11T23:33:24.699 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.696+0000 7f4cf0c27740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f4cd400f770 con 0x558b449edc20 2024-09-11T23:33:24.699 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.696+0000 7f4cf0c27740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f4cd400c8e0 con 0x558b449f84c0 2024-09-11T23:33:24.699 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.696+0000 7f4cf0c27740 10 client.0 _ll_register_callbacks cb 0x7f4cd803cb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:33:24.699 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.697+0000 7f4ce7fff640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558b449f84c0 0x558b44a72150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:33:24.699 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.697+0000 7f4ce7fff640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558b449f84c0 0x558b44a72150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:41984/0 (socket says 172.21.15.26:41984) 2024-09-11T23:33:24.699 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.697+0000 7f4ce7fff640 1 -- 172.21.15.26:0/871700817 learned_addr learned my addr 172.21.15.26:0/871700817 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:33:24.699 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.697+0000 7f4ce6ffd640 1 --2- 172.21.15.26:0/871700817 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558b449edc20 0x558b449fcc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:33:24.699 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.697+0000 7f4ce77fe640 1 --2- 172.21.15.26:0/871700817 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558b449ed300 0x558b449fc6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:33:24.700 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.697+0000 7f4ce6ffd640 1 -- 172.21.15.26:0/871700817 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558b449f84c0 msgr2=0x558b44a72150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:33:24.700 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.697+0000 7f4ce6ffd640 1 --2- 172.21.15.26:0/871700817 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558b449f84c0 0x558b44a72150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:24.700 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.697+0000 7f4ce6ffd640 1 -- 172.21.15.26:0/871700817 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558b449ed300 msgr2=0x558b449fc6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:33:24.700 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.697+0000 7f4ce6ffd640 1 --2- 172.21.15.26:0/871700817 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558b449ed300 0x558b449fc6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:24.700 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.697+0000 7f4ce6ffd640 1 -- 172.21.15.26:0/871700817 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b449e0a10 con 0x558b449edc20 2024-09-11T23:33:24.700 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.698+0000 7f4ce6ffd640 1 --2- 172.21.15.26:0/871700817 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558b449edc20 0x558b449fcc10 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f4cdc010fd0 tx=0x7f4cdc010890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=92082c476507c481 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:33:24.700 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.698+0000 7f4cc6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-11T23:33:24.704 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.700+0000 7f4cc6ffd640 1 -- 172.21.15.26:0/871700817 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cdc024040 con 0x558b449edc20 2024-09-11T23:33:24.704 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.700+0000 7f4cf0c27740 1 -- 172.21.15.26:0/871700817 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ce0004610 con 0x558b449edc20 2024-09-11T23:33:24.704 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.700+0000 7f4cc6ffd640 1 -- 172.21.15.26:0/871700817 <== mon.2 v2:172.21.15.76:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8586+0+0 (secure 0 0 0) 0x7f4cdc005250 con 0x558b449edc20 2024-09-11T23:33:24.704 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.701+0000 7f4cc6ffd640 1 -- 172.21.15.26:0/871700817 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) ==== 24+0+0 (secure 0 0 0) 0x7f4cdc010e00 con 0x558b449edc20 2024-09-11T23:33:24.704 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.701+0000 7f4cf0c27740 10 client.27044 fetch_fsmap learned FSMap version 73 2024-09-11T23:33:24.704 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.701+0000 7f4cf0c27740 1 -- 172.21.15.26:0/871700817 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=4+}) -- 0x558b449e1cc0 con 0x558b449edc20 2024-09-11T23:33:24.704 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.702+0000 7f4cc6ffd640 1 -- 172.21.15.26:0/871700817 <== mon.2 v2:172.21.15.76:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ce0004610 con 0x558b449edc20 2024-09-11T23:33:24.704 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.702+0000 7f4cc6ffd640 1 -- 172.21.15.26:0/871700817 <== mon.2 v2:172.21.15.76:3300/0 5 ==== fsmap(e 73) ==== 8430+0+0 (secure 0 0 0) 0x7f4cdc01f070 con 0x558b449edc20 2024-09-11T23:33:24.705 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.702+0000 7f4cf0c27740 10 client.27044 fetch_fsmap finished waiting for FSMap version 73 2024-09-11T23:33:24.705 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.702+0000 7f4cf0c27740 10 client.27044 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:33:24.705 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.702+0000 7f4cf0c27740 20 client.27044 populate_metadata read hostname 'smithi026' 2024-09-11T23:33:24.705 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.702+0000 7f4cf0c27740 1 --2- 172.21.15.26:0/871700817 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x558b449e2490 0x558b449e48b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:24.705 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.702+0000 7f4cf0c27740 4 client.27044 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T23:33:24.705 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.702+0000 7f4cf0c27740 1 -- 172.21.15.26:0/871700817 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558b44956a20 con 0x558b449e2490 2024-09-11T23:33:24.705 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.703+0000 7f4ce77fe640 1 --2- 172.21.15.26:0/871700817 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x558b449e2490 0x558b449e48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:33:24.708 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.707+0000 7f4ce77fe640 1 --2- 172.21.15.26:0/871700817 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x558b449e2490 0x558b449e48b0 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=86ad1a5d02898b92 server_cookie=38e98dac7702f70a in_seq=0 out_seq=0 2024-09-11T23:33:24.708 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.707+0000 7f4cc6ffd640 10 client.27044 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:33:24.760 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.758+0000 7f4cc6ffd640 1 -- 172.21.15.26:0/871700817 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558b44956a20 con 0x558b449e2490 2024-09-11T23:33:24.760 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.758+0000 7f4cc6ffd640 10 client.27044 handle_command_reply: tid=0 2024-09-11T23:33:24.760 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.758+0000 7f4cf0c27740 1 client.27044 shutdown 2024-09-11T23:33:24.761 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.761+0000 7f4cf0c27740 1 -- 172.21.15.26:0/871700817 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558b449edc20 msgr2=0x558b449fcc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:33:24.761 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.761+0000 7f4cf0c27740 1 --2- 172.21.15.26:0/871700817 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558b449edc20 0x558b449fcc10 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f4cdc010fd0 tx=0x7f4cdc010890 comp rx=0 tx=0).stop 2024-09-11T23:33:24.762 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.762+0000 7f4cf0c27740 1 -- 172.21.15.26:0/871700817 shutdown_connections 2024-09-11T23:33:24.763 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.762+0000 7f4cf0c27740 1 --2- 172.21.15.26:0/871700817 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558b449edc20 0x558b449fcc10 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:24.763 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.762+0000 7f4cf0c27740 1 --2- 172.21.15.26:0/871700817 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x558b449e2490 0x558b449e48b0 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:24.763 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.762+0000 7f4cf0c27740 1 --2- 172.21.15.26:0/871700817 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558b449f84c0 0x558b44a72150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:24.763 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.762+0000 7f4cf0c27740 1 --2- 172.21.15.26:0/871700817 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558b449ed300 0x558b449fc6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:24.763 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.762+0000 7f4cf0c27740 1 -- 172.21.15.26:0/871700817 >> 172.21.15.26:0/871700817 conn(0x558b449e9870 msgr2=0x558b449f5b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:33:24.763 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.762+0000 7f4cc6ffd640 0 client.27044 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:33:24.764 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.764+0000 7f4cf0c27740 1 -- 172.21.15.26:0/871700817 shutdown_connections 2024-09-11T23:33:24.764 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.764+0000 7f4cf0c27740 1 -- 172.21.15.26:0/871700817 wait complete. 2024-09-11T23:33:24.764 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.764+0000 7f4cf0c27740 20 client.27044 trim_cache size 0 max 16384 2024-09-11T23:33:24.766 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-09-11T23:33:24.766 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "message": "", 2024-09-11T23:33:24.766 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-09-11T23:33:24.766 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-09-11T23:33:24.767 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.766+0000 7f4cede8d640 1 -- 172.21.15.26:0/2154887355 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f4cd00a2ef0 msgr2=0x7f4cd00a53b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:33:24.767 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.766+0000 7f4cede8d640 1 --2- 172.21.15.26:0/2154887355 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f4cd00a2ef0 0x7f4cd00a53b0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f4cdc005ce0 tx=0x7f4cdc00bb50 comp rx=0 tx=0).stop 2024-09-11T23:33:24.767 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.766+0000 7f4cede8d640 1 -- 172.21.15.26:0/2154887355 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4ce8114ab0 msgr2=0x7f4ce81a2af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:33:24.767 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.766+0000 7f4cede8d640 1 --2- 172.21.15.26:0/2154887355 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4ce8114ab0 0x7f4ce81a2af0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f4cd8006010 tx=0x7f4cd8005230 comp rx=0 tx=0).stop 2024-09-11T23:33:24.767 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.766+0000 7f4cede8d640 1 -- 172.21.15.26:0/2154887355 shutdown_connections 2024-09-11T23:33:24.767 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.766+0000 7f4cede8d640 1 --2- 172.21.15.26:0/2154887355 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f4cd00a2ef0 0x7f4cd00a53b0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:24.767 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.766+0000 7f4cede8d640 1 --2- 172.21.15.26:0/2154887355 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4ce8114ab0 0x7f4ce81a2af0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:24.767 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.766+0000 7f4cede8d640 1 --2- 172.21.15.26:0/2154887355 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4ce8109b10 0x7f4ce81a8e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:24.767 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.766+0000 7f4cede8d640 1 --2- 172.21.15.26:0/2154887355 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4ce81091f0 0x7f4ce81a8940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:24.767 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.766+0000 7f4cede8d640 1 -- 172.21.15.26:0/2154887355 >> 172.21.15.26:0/2154887355 conn(0x7f4ce806ce00 msgr2=0x7f4ce8070750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:33:24.767 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.766+0000 7f4cede8d640 1 -- 172.21.15.26:0/2154887355 shutdown_connections 2024-09-11T23:33:24.767 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:24.766+0000 7f4cede8d640 1 -- 172.21.15.26:0/2154887355 wait complete. 2024-09-11T23:33:25.854 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:33:25 smithi076 ceph-mon[35296]: pgmap v471: 97 pgs: 34 active+clean+snaptrim_wait, 18 active+clean+snaptrim, 45 active+clean; 2.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 190 KiB/s wr, 61 op/s 2024-09-11T23:33:25.988 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:33:25 smithi136 ceph-mon[35121]: pgmap v471: 97 pgs: 34 active+clean+snaptrim_wait, 18 active+clean+snaptrim, 45 active+clean; 2.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 190 KiB/s wr, 61 op/s 2024-09-11T23:33:26.000 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:33:25 smithi026 ceph-mon[31172]: pgmap v471: 97 pgs: 34 active+clean+snaptrim_wait, 18 active+clean+snaptrim, 45 active+clean; 2.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 190 KiB/s wr, 61 op/s 2024-09-11T23:33:27.988 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:33:27 smithi136 ceph-mon[35121]: pgmap v472: 97 pgs: 32 active+clean+snaptrim_wait, 16 active+clean+snaptrim, 49 active+clean; 2.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 8.2 KiB/s wr, 30 op/s 2024-09-11T23:33:28.000 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:33:27 smithi026 ceph-mon[31172]: pgmap v472: 97 pgs: 32 active+clean+snaptrim_wait, 16 active+clean+snaptrim, 49 active+clean; 2.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 8.2 KiB/s wr, 30 op/s 2024-09-11T23:33:28.052 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:33:27 smithi076 ceph-mon[35296]: pgmap v472: 97 pgs: 32 active+clean+snaptrim_wait, 16 active+clean+snaptrim, 49 active+clean; 2.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 8.2 KiB/s wr, 30 op/s 2024-09-11T23:33:29.988 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:33:29 smithi136 ceph-mon[35121]: pgmap v473: 97 pgs: 32 active+clean+snaptrim_wait, 15 active+clean+snaptrim, 50 active+clean; 2.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 8.0 KiB/s wr, 13 op/s 2024-09-11T23:33:30.000 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:33:29 smithi026 ceph-mon[31172]: pgmap v473: 97 pgs: 32 active+clean+snaptrim_wait, 15 active+clean+snaptrim, 50 active+clean; 2.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 8.0 KiB/s wr, 13 op/s 2024-09-11T23:33:30.052 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:33:29 smithi076 ceph-mon[35296]: pgmap v473: 97 pgs: 32 active+clean+snaptrim_wait, 15 active+clean+snaptrim, 50 active+clean; 2.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 8.0 KiB/s wr, 13 op/s 2024-09-11T23:33:31.988 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:33:31 smithi136 ceph-mon[35121]: pgmap v474: 97 pgs: 20 active+clean+snaptrim_wait, 9 active+clean+snaptrim, 68 active+clean; 2.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 19 KiB/s wr, 19 op/s 2024-09-11T23:33:32.000 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:33:31 smithi026 ceph-mon[31172]: pgmap v474: 97 pgs: 20 active+clean+snaptrim_wait, 9 active+clean+snaptrim, 68 active+clean; 2.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 19 KiB/s wr, 19 op/s 2024-09-11T23:33:32.052 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:33:31 smithi076 ceph-mon[35296]: pgmap v474: 97 pgs: 20 active+clean+snaptrim_wait, 9 active+clean+snaptrim, 68 active+clean; 2.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 19 KiB/s wr, 19 op/s 2024-09-11T23:33:33.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:33:33 smithi136 ceph-mon[35121]: pgmap v475: 97 pgs: 15 active+clean+snaptrim_wait, 8 active+clean+snaptrim, 74 active+clean; 2.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 KiB/s wr, 12 op/s 2024-09-11T23:33:34.000 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:33:33 smithi026 ceph-mon[31172]: pgmap v475: 97 pgs: 15 active+clean+snaptrim_wait, 8 active+clean+snaptrim, 74 active+clean; 2.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 KiB/s wr, 12 op/s 2024-09-11T23:33:34.052 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:33:33 smithi076 ceph-mon[35296]: pgmap v475: 97 pgs: 15 active+clean+snaptrim_wait, 8 active+clean+snaptrim, 74 active+clean; 2.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 KiB/s wr, 12 op/s 2024-09-11T23:33:35.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:33:35 smithi136 ceph-mon[35121]: pgmap v476: 97 pgs: 2 active+clean+snaptrim, 95 active+clean; 2.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 KiB/s wr, 13 op/s 2024-09-11T23:33:36.000 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:33:35 smithi026 ceph-mon[31172]: pgmap v476: 97 pgs: 2 active+clean+snaptrim, 95 active+clean; 2.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 KiB/s wr, 13 op/s 2024-09-11T23:33:36.052 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:33:35 smithi076 ceph-mon[35296]: pgmap v476: 97 pgs: 2 active+clean+snaptrim, 95 active+clean; 2.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 KiB/s wr, 13 op/s 2024-09-11T23:33:36.997 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:33:37.271 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:33:37.671 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:33:37 smithi026 ceph-mon[31172]: pgmap v477: 97 pgs: 97 active+clean; 2.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-11T23:33:37.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:37.753+0000 7fb12a7d6640 1 -- 172.21.15.26:0/1251831681 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb124106a80 msgr2=0x7fb124106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:33:37.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:37.753+0000 7fb12a7d6640 1 --2- 172.21.15.26:0/1251831681 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb124106a80 0x7fb124106e60 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7fb114009930 tx=0x7fb11402a420 comp rx=0 tx=0).stop 2024-09-11T23:33:37.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:37.753+0000 7fb12a7d6640 1 -- 172.21.15.26:0/1251831681 shutdown_connections 2024-09-11T23:33:37.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:37.753+0000 7fb12a7d6640 1 --2- 172.21.15.26:0/1251831681 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb124101420 0x7fb12410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:37.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:37.753+0000 7fb12a7d6640 1 --2- 172.21.15.26:0/1251831681 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb124100a80 0x7fb124100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:37.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:37.753+0000 7fb12a7d6640 1 --2- 172.21.15.26:0/1251831681 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb124106a80 0x7fb124106e60 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:37.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:37.753+0000 7fb12a7d6640 1 -- 172.21.15.26:0/1251831681 >> 172.21.15.26:0/1251831681 conn(0x7fb1240fc7a0 msgr2=0x7fb1240febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:33:37.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:37.753+0000 7fb12a7d6640 1 -- 172.21.15.26:0/1251831681 shutdown_connections 2024-09-11T23:33:37.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:37.754+0000 7fb12a7d6640 1 -- 172.21.15.26:0/1251831681 wait complete. 2024-09-11T23:33:37.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:37.754+0000 7fb12a7d6640 1 Processor -- start 2024-09-11T23:33:37.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:37.754+0000 7fb12a7d6640 1 -- start start 2024-09-11T23:33:37.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:37.755+0000 7fb12a7d6640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb124100a80 0x7fb1241984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:37.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:37.755+0000 7fb12a7d6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb124101420 0x7fb124198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:37.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:37.755+0000 7fb12a7d6640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb124106a80 0x7fb12419cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:37.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:37.755+0000 7fb12a7d6640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb124110e40 con 0x7fb124101420 2024-09-11T23:33:37.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:37.755+0000 7fb12a7d6640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fb124110cc0 con 0x7fb124106a80 2024-09-11T23:33:37.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:37.755+0000 7fb12a7d6640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb124110fc0 con 0x7fb124100a80 2024-09-11T23:33:37.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:37.755+0000 7fb1237fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb124101420 0x7fb124198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:33:37.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:37.755+0000 7fb1237fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb124101420 0x7fb124198a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47310/0 (socket says 172.21.15.26:47310) 2024-09-11T23:33:37.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:37.755+0000 7fb1237fe640 1 -- 172.21.15.26:0/2410170561 learned_addr learned my addr 172.21.15.26:0/2410170561 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:33:37.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:37.755+0000 7fb123fff640 1 --2- 172.21.15.26:0/2410170561 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb124100a80 0x7fb1241984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:33:37.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:37.755+0000 7fb128d73640 1 --2- 172.21.15.26:0/2410170561 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb124106a80 0x7fb12419cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:33:37.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:37.755+0000 7fb1237fe640 1 -- 172.21.15.26:0/2410170561 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb124100a80 msgr2=0x7fb1241984d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:33:37.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:37.755+0000 7fb1237fe640 1 --2- 172.21.15.26:0/2410170561 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb124100a80 0x7fb1241984d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:37.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:37.755+0000 7fb1237fe640 1 -- 172.21.15.26:0/2410170561 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb124106a80 msgr2=0x7fb12419cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:33:37.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:37.755+0000 7fb1237fe640 1 --2- 172.21.15.26:0/2410170561 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb124106a80 0x7fb12419cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:37.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:37.755+0000 7fb1237fe640 1 -- 172.21.15.26:0/2410170561 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb12419d480 con 0x7fb124101420 2024-09-11T23:33:37.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:37.755+0000 7fb123fff640 1 --2- 172.21.15.26:0/2410170561 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb124100a80 0x7fb1241984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:33:37.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:37.755+0000 7fb128d73640 1 --2- 172.21.15.26:0/2410170561 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb124106a80 0x7fb12419cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:33:37.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:37.755+0000 7fb1237fe640 1 --2- 172.21.15.26:0/2410170561 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb124101420 0x7fb124198a10 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7fb11800bf20 tx=0x7fb11800dd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:33:37.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:37.755+0000 7fb1217fa640 1 -- 172.21.15.26:0/2410170561 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb118018070 con 0x7fb124101420 2024-09-11T23:33:37.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:37.755+0000 7fb1217fa640 1 -- 172.21.15.26:0/2410170561 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb118012c40 con 0x7fb124101420 2024-09-11T23:33:37.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:37.755+0000 7fb1217fa640 1 -- 172.21.15.26:0/2410170561 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb11801b9a0 con 0x7fb124101420 2024-09-11T23:33:37.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:37.755+0000 7fb12a7d6640 1 -- 172.21.15.26:0/2410170561 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb12419d770 con 0x7fb124101420 2024-09-11T23:33:37.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:37.756+0000 7fb12a7d6640 1 -- 172.21.15.26:0/2410170561 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1240fcd00 con 0x7fb124101420 2024-09-11T23:33:37.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:37.756+0000 7fb12a7d6640 1 -- 172.21.15.26:0/2410170561 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb124100ee0 con 0x7fb124101420 2024-09-11T23:33:37.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:37.758+0000 7fb1217fa640 1 -- 172.21.15.26:0/2410170561 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fb11801bb40 con 0x7fb124101420 2024-09-11T23:33:37.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:37.759+0000 7fb1217fa640 1 --2- 172.21.15.26:0/2410170561 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fb10c0a2d70 0x7fb10c0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:37.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:37.759+0000 7fb123fff640 1 --2- 172.21.15.26:0/2410170561 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fb10c0a2d70 0x7fb10c0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:33:37.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:37.759+0000 7fb1217fa640 1 -- 172.21.15.26:0/2410170561 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8586+0+0 (secure 0 0 0) 0x7fb1180d3330 con 0x7fb124101420 2024-09-11T23:33:37.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:37.759+0000 7fb123fff640 1 --2- 172.21.15.26:0/2410170561 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fb10c0a2d70 0x7fb10c0a5230 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fb114000c00 tx=0x7fb114005a20 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:33:37.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:37.762+0000 7fb1217fa640 1 -- 172.21.15.26:0/2410170561 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb1180222b0 con 0x7fb124101420 2024-09-11T23:33:37.864 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.863+0000 7fb7e33f6640 1 -- 172.21.15.26:0/629702128 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb7dc102470 msgr2=0x7fb7dc10e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:33:37.864 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.863+0000 7fb7e33f6640 1 --2- 172.21.15.26:0/629702128 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb7dc102470 0x7fb7dc10e9a0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7fb7d4009cd0 tx=0x7fb7d402a8e0 comp rx=0 tx=0).stop 2024-09-11T23:33:37.864 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.863+0000 7fb7e33f6640 1 -- 172.21.15.26:0/629702128 shutdown_connections 2024-09-11T23:33:37.864 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.863+0000 7fb7e33f6640 1 --2- 172.21.15.26:0/629702128 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb7dc101ad0 0x7fb7dc101f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:37.864 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.863+0000 7fb7e33f6640 1 --2- 172.21.15.26:0/629702128 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb7dc102470 0x7fb7dc10e9a0 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:37.864 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.863+0000 7fb7e33f6640 1 --2- 172.21.15.26:0/629702128 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb7dc107b70 0x7fb7dc107f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:37.864 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.863+0000 7fb7e33f6640 1 -- 172.21.15.26:0/629702128 >> 172.21.15.26:0/629702128 conn(0x7fb7dc0fd7d0 msgr2=0x7fb7dc0ffbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:33:37.864 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.863+0000 7fb7e33f6640 1 -- 172.21.15.26:0/629702128 shutdown_connections 2024-09-11T23:33:37.864 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.863+0000 7fb7e33f6640 1 -- 172.21.15.26:0/629702128 wait complete. 2024-09-11T23:33:37.867 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.864+0000 7fb7e33f6640 1 Processor -- start 2024-09-11T23:33:37.867 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.864+0000 7fb7e33f6640 1 -- start start 2024-09-11T23:33:37.867 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.864+0000 7fb7e33f6640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb7dc101ad0 0x7fb7dc1982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:37.867 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.864+0000 7fb7e33f6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb7dc102470 0x7fb7dc198830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:37.867 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.864+0000 7fb7e33f6640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb7dc107b70 0x7fb7dc19cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:37.867 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.864+0000 7fb7e33f6640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb7dc111f20 con 0x7fb7dc102470 2024-09-11T23:33:37.867 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.864+0000 7fb7e33f6640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fb7dc111da0 con 0x7fb7dc101ad0 2024-09-11T23:33:37.867 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.864+0000 7fb7e33f6640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb7dc1120a0 con 0x7fb7dc107b70 2024-09-11T23:33:37.867 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.864+0000 7fb7e0991640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb7dc102470 0x7fb7dc198830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:33:37.867 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.864+0000 7fb7e0991640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb7dc102470 0x7fb7dc198830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47340/0 (socket says 172.21.15.26:47340) 2024-09-11T23:33:37.868 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.864+0000 7fb7e0991640 1 -- 172.21.15.26:0/3347353067 learned_addr learned my addr 172.21.15.26:0/3347353067 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:33:37.868 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.865+0000 7fb7e1192640 1 --2- 172.21.15.26:0/3347353067 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb7dc101ad0 0x7fb7dc1982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:33:37.868 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.865+0000 7fb7e1993640 1 --2- 172.21.15.26:0/3347353067 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb7dc107b70 0x7fb7dc19cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:33:37.868 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.865+0000 7fb7e0991640 1 -- 172.21.15.26:0/3347353067 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb7dc107b70 msgr2=0x7fb7dc19cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:33:37.868 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.865+0000 7fb7e0991640 1 --2- 172.21.15.26:0/3347353067 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb7dc107b70 0x7fb7dc19cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:37.868 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.865+0000 7fb7e0991640 1 -- 172.21.15.26:0/3347353067 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb7dc101ad0 msgr2=0x7fb7dc1982f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:33:37.868 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.865+0000 7fb7e0991640 1 --2- 172.21.15.26:0/3347353067 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb7dc101ad0 0x7fb7dc1982f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:37.868 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.865+0000 7fb7e0991640 1 -- 172.21.15.26:0/3347353067 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7dc19d2e0 con 0x7fb7dc102470 2024-09-11T23:33:37.868 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.865+0000 7fb7e1192640 1 --2- 172.21.15.26:0/3347353067 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb7dc101ad0 0x7fb7dc1982f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:33:37.868 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.865+0000 7fb7e1993640 1 --2- 172.21.15.26:0/3347353067 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb7dc107b70 0x7fb7dc19cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:33:37.868 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.865+0000 7fb7e0991640 1 --2- 172.21.15.26:0/3347353067 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb7dc102470 0x7fb7dc198830 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7fb7cc00d950 tx=0x7fb7cc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:33:37.868 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.865+0000 7fb7d27fc640 1 -- 172.21.15.26:0/3347353067 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7cc017030 con 0x7fb7dc102470 2024-09-11T23:33:37.868 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.865+0000 7fb7d27fc640 1 -- 172.21.15.26:0/3347353067 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb7cc004020 con 0x7fb7dc102470 2024-09-11T23:33:37.869 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.865+0000 7fb7e33f6640 1 -- 172.21.15.26:0/3347353067 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7dc19d5d0 con 0x7fb7dc102470 2024-09-11T23:33:37.869 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.865+0000 7fb7d27fc640 1 -- 172.21.15.26:0/3347353067 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7cc0047a0 con 0x7fb7dc102470 2024-09-11T23:33:37.869 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.865+0000 7fb7e33f6640 1 -- 172.21.15.26:0/3347353067 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7dc1a4e60 con 0x7fb7dc102470 2024-09-11T23:33:37.875 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.868+0000 7fb7d27fc640 1 -- 172.21.15.26:0/3347353067 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fb7cc0041d0 con 0x7fb7dc102470 2024-09-11T23:33:37.875 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.868+0000 7fb7e626b740 1 Processor -- start 2024-09-11T23:33:37.875 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.868+0000 7fb7e626b740 1 -- start start 2024-09-11T23:33:37.876 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.868+0000 7fb7e626b740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55deedcebaf0 0x55deedcebed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:37.876 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.868+0000 7fb7e626b740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55deedcec410 0x55deedcf6770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:37.876 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.868+0000 7fb7e626b740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55deedcf6cb0 0x55deedcf90a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:37.876 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.868+0000 7fb7e626b740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55deedc502f0 con 0x55deedcf6cb0 2024-09-11T23:33:37.876 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.868+0000 7fb7e626b740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55deedbf8270 con 0x55deedcebaf0 2024-09-11T23:33:37.876 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.868+0000 7fb7e626b740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55deedc4c2b0 con 0x55deedcec410 2024-09-11T23:33:37.876 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.868+0000 7fb7e0991640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55deedcf6cb0 0x55deedcf90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:33:37.876 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.869+0000 7fb7e0991640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55deedcf6cb0 0x55deedcf90a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47354/0 (socket says 172.21.15.26:47354) 2024-09-11T23:33:37.876 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.869+0000 7fb7e0991640 1 -- 172.21.15.26:0/2203796451 learned_addr learned my addr 172.21.15.26:0/2203796451 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:33:37.876 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.869+0000 7fb7e1192640 1 --2- 172.21.15.26:0/2203796451 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55deedcec410 0x55deedcf6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:33:37.876 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.869+0000 7fb7e1993640 1 --2- 172.21.15.26:0/2203796451 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55deedcebaf0 0x55deedcebed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:33:37.876 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.869+0000 7fb7d27fc640 1 --2- 172.21.15.26:0/3347353067 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fb7c80a2da0 0x7fb7c80a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:37.876 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.869+0000 7fb7e0991640 1 -- 172.21.15.26:0/2203796451 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55deedcec410 msgr2=0x55deedcf6770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:33:37.877 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.869+0000 7fb7e0991640 1 --2- 172.21.15.26:0/2203796451 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55deedcec410 0x55deedcf6770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:37.877 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.869+0000 7fb7e0991640 1 -- 172.21.15.26:0/2203796451 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55deedcebaf0 msgr2=0x55deedcebed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:33:37.877 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.869+0000 7fb7e0991640 1 --2- 172.21.15.26:0/2203796451 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55deedcebaf0 0x55deedcebed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:37.877 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.869+0000 7fb7e0991640 1 -- 172.21.15.26:0/2203796451 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55deedbf7340 con 0x55deedcf6cb0 2024-09-11T23:33:37.877 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.869+0000 7fb7e1993640 1 --2- 172.21.15.26:0/3347353067 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fb7c80a2da0 0x7fb7c80a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:33:37.877 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.869+0000 7fb7d27fc640 1 -- 172.21.15.26:0/3347353067 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8586+0+0 (secure 0 0 0) 0x7fb7cc0cef30 con 0x7fb7dc102470 2024-09-11T23:33:37.877 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.869+0000 7fb7e0991640 1 --2- 172.21.15.26:0/2203796451 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55deedcf6cb0 0x55deedcf90a0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7fb7cc0cc5b0 tx=0x7fb7cc065f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9eecd356a09d0ab6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:33:37.877 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.869+0000 7fb7c77fe640 1 -- 172.21.15.26:0/2203796451 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7cc09b1e0 con 0x55deedcf6cb0 2024-09-11T23:33:37.877 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.869+0000 7fb7c77fe640 1 -- 172.21.15.26:0/2203796451 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb7cc0664a0 con 0x55deedcf6cb0 2024-09-11T23:33:37.877 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.869+0000 7fb7c77fe640 1 -- 172.21.15.26:0/2203796451 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7cc09ad70 con 0x55deedcf6cb0 2024-09-11T23:33:37.877 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.870+0000 7fb7e626b740 1 -- 172.21.15.26:0/2203796451 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55deedcf6cb0 msgr2=0x55deedcf90a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:33:37.877 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.870+0000 7fb7e1993640 1 --2- 172.21.15.26:0/3347353067 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fb7c80a2da0 0x7fb7c80a5260 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fb7d4004af0 tx=0x7fb7d40095c0 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:33:37.877 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.870+0000 7fb7e626b740 1 --2- 172.21.15.26:0/2203796451 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55deedcf6cb0 0x55deedcf90a0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7fb7cc0cc5b0 tx=0x7fb7cc065f90 comp rx=0 tx=0).stop 2024-09-11T23:33:37.878 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.870+0000 7fb7e626b740 1 -- 172.21.15.26:0/2203796451 shutdown_connections 2024-09-11T23:33:37.878 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.870+0000 7fb7e626b740 1 --2- 172.21.15.26:0/2203796451 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55deedcf6cb0 0x55deedcf90a0 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:37.878 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.870+0000 7fb7e626b740 1 --2- 172.21.15.26:0/2203796451 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55deedcec410 0x55deedcf6770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:37.878 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.870+0000 7fb7e626b740 1 --2- 172.21.15.26:0/2203796451 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55deedcebaf0 0x55deedcebed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:37.878 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.870+0000 7fb7e626b740 1 -- 172.21.15.26:0/2203796451 >> 172.21.15.26:0/2203796451 conn(0x55deedce8060 msgr2=0x55deedce8440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:33:37.878 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.870+0000 7fb7e626b740 1 -- 172.21.15.26:0/2203796451 shutdown_connections 2024-09-11T23:33:37.878 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.870+0000 7fb7e626b740 1 -- 172.21.15.26:0/2203796451 wait complete. 2024-09-11T23:33:37.878 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.871+0000 7fb7e626b740 1 -- start start 2024-09-11T23:33:37.878 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.871+0000 7fb7e626b740 1 Processor -- start 2024-09-11T23:33:37.878 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.871+0000 7fb7e626b740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55deedcebaf0 0x55deedcf3b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:37.878 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.871+0000 7fb7e626b740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55deedcec410 0x55deedcf4080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:37.878 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.871+0000 7fb7e626b740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55deedcf6cb0 0x55deedcf45c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:37.878 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.871+0000 7fb7e626b740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55deedcf9880 con 0x55deedcebaf0 2024-09-11T23:33:37.879 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.871+0000 7fb7e626b740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55deedbd9290 con 0x55deedcec410 2024-09-11T23:33:37.879 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.871+0000 7fb7e626b740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55deedcf9a00 con 0x55deedcf6cb0 2024-09-11T23:33:37.879 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.871+0000 7fb7e626b740 10 client.0 _ll_register_callbacks cb 0x55deedc0a140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:33:37.879 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.871+0000 7fb7e1192640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55deedcebaf0 0x55deedcf3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:33:37.879 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.871+0000 7fb7e1192640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55deedcebaf0 0x55deedcf3b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47364/0 (socket says 172.21.15.26:47364) 2024-09-11T23:33:37.879 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.871+0000 7fb7e1192640 1 -- 172.21.15.26:0/3981590455 learned_addr learned my addr 172.21.15.26:0/3981590455 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:33:37.879 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.871+0000 7fb7e1993640 1 --2- 172.21.15.26:0/3981590455 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55deedcf6cb0 0x55deedcf45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:33:37.879 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.871+0000 7fb7e0991640 1 --2- 172.21.15.26:0/3981590455 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55deedcec410 0x55deedcf4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:33:37.879 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.871+0000 7fb7e1192640 1 -- 172.21.15.26:0/3981590455 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55deedcf6cb0 msgr2=0x55deedcf45c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:33:37.879 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.871+0000 7fb7e1192640 1 --2- 172.21.15.26:0/3981590455 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55deedcf6cb0 0x55deedcf45c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:37.879 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.871+0000 7fb7e1192640 1 -- 172.21.15.26:0/3981590455 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55deedcec410 msgr2=0x55deedcf4080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:33:37.879 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.871+0000 7fb7e1192640 1 --2- 172.21.15.26:0/3981590455 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55deedcec410 0x55deedcf4080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:37.879 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.872+0000 7fb7e1192640 1 -- 172.21.15.26:0/3981590455 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55deed963210 con 0x55deedcebaf0 2024-09-11T23:33:37.879 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.872+0000 7fb7e0991640 1 --2- 172.21.15.26:0/3981590455 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55deedcec410 0x55deedcf4080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:33:37.880 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.872+0000 7fb7e1192640 1 --2- 172.21.15.26:0/3981590455 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55deedcebaf0 0x55deedcf3b40 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7fb7d8013740 tx=0x7fb7d8013c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eeb8c1da0c9e5e05 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:33:37.880 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.872+0000 7fb7c4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:33:37.880 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.872+0000 7fb7c4ff9640 1 -- 172.21.15.26:0/3981590455 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7d8011dd0 con 0x55deedcebaf0 2024-09-11T23:33:37.880 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.872+0000 7fb7c4ff9640 1 -- 172.21.15.26:0/3981590455 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8586+0+0 (secure 0 0 0) 0x7fb7d8004020 con 0x55deedcebaf0 2024-09-11T23:33:37.880 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.872+0000 7fb7e626b740 1 -- 172.21.15.26:0/3981590455 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb7cc0664a0 con 0x55deedcebaf0 2024-09-11T23:33:37.880 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.872+0000 7fb7c4ff9640 1 -- 172.21.15.26:0/3981590455 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) ==== 24+0+0 (secure 0 0 0) 0x7fb7d8004260 con 0x55deedcebaf0 2024-09-11T23:33:37.880 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.872+0000 7fb7e626b740 10 client.17538 fetch_fsmap learned FSMap version 73 2024-09-11T23:33:37.880 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.872+0000 7fb7e626b740 1 -- 172.21.15.26:0/3981590455 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=4+}) -- 0x55deeda2f620 con 0x55deedcebaf0 2024-09-11T23:33:37.880 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.872+0000 7fb7c4ff9640 1 -- 172.21.15.26:0/3981590455 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb7cc0664a0 con 0x55deedcebaf0 2024-09-11T23:33:37.880 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.872+0000 7fb7c4ff9640 1 -- 172.21.15.26:0/3981590455 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 73) ==== 8430+0+0 (secure 0 0 0) 0x7fb7d8021030 con 0x55deedcebaf0 2024-09-11T23:33:37.880 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.873+0000 7fb7e626b740 10 client.17538 fetch_fsmap finished waiting for FSMap version 73 2024-09-11T23:33:37.880 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.873+0000 7fb7e626b740 10 client.17538 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:33:37.880 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.873+0000 7fb7e626b740 20 client.17538 populate_metadata read hostname 'smithi026' 2024-09-11T23:33:37.881 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.873+0000 7fb7e626b740 1 --2- 172.21.15.26:0/3981590455 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55deedd06d20 0x55deedd091e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:37.881 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.873+0000 7fb7e626b740 4 client.17538 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:33:37.881 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.873+0000 7fb7e626b740 1 -- 172.21.15.26:0/3981590455 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55deedc5da20 con 0x55deedd06d20 2024-09-11T23:33:37.881 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.873+0000 7fb7e0991640 1 --2- 172.21.15.26:0/3981590455 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55deedd06d20 0x55deedd091e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:33:37.881 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.873+0000 7fb7c4ff9640 10 client.17538 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:33:37.881 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.873+0000 7fb7e0991640 1 --2- 172.21.15.26:0/3981590455 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55deedd06d20 0x55deedd091e0 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d2872166812a8391 server_cookie=2b30ff158eb26c51 in_seq=0 out_seq=0 2024-09-11T23:33:37.881 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.874+0000 7fb7c4ff9640 1 -- 172.21.15.26:0/3981590455 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55deedc5da20 con 0x55deedd06d20 2024-09-11T23:33:37.881 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.874+0000 7fb7c4ff9640 10 client.17538 handle_command_reply: tid=0 2024-09-11T23:33:37.881 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.874+0000 7fb7e626b740 1 client.17538 shutdown 2024-09-11T23:33:37.881 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.874+0000 7fb7e626b740 1 -- 172.21.15.26:0/3981590455 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55deedcebaf0 msgr2=0x55deedcf3b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:33:37.881 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.874+0000 7fb7e626b740 1 --2- 172.21.15.26:0/3981590455 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55deedcebaf0 0x55deedcf3b40 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7fb7d8013740 tx=0x7fb7d8013c00 comp rx=0 tx=0).stop 2024-09-11T23:33:37.881 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.874+0000 7fb7e626b740 1 -- 172.21.15.26:0/3981590455 shutdown_connections 2024-09-11T23:33:37.881 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.874+0000 7fb7e626b740 1 --2- 172.21.15.26:0/3981590455 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55deedcec410 0x55deedcf4080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:37.881 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.874+0000 7fb7e626b740 1 --2- 172.21.15.26:0/3981590455 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55deedd06d20 0x55deedd091e0 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:37.882 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.874+0000 7fb7e626b740 1 --2- 172.21.15.26:0/3981590455 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55deedcf6cb0 0x55deedcf45c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:37.882 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.874+0000 7fb7c4ff9640 0 client.17538 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:33:37.882 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.874+0000 7fb7e626b740 1 --2- 172.21.15.26:0/3981590455 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55deedcebaf0 0x55deedcf3b40 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:37.882 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.874+0000 7fb7e626b740 1 -- 172.21.15.26:0/3981590455 >> 172.21.15.26:0/3981590455 conn(0x55deedce8060 msgr2=0x55deedceddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:33:37.882 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.874+0000 7fb7e626b740 1 -- 172.21.15.26:0/3981590455 shutdown_connections 2024-09-11T23:33:37.882 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.874+0000 7fb7e626b740 1 -- 172.21.15.26:0/3981590455 wait complete. 2024-09-11T23:33:37.882 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.874+0000 7fb7e626b740 20 client.17538 trim_cache size 0 max 16384 2024-09-11T23:33:37.894 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.889+0000 7fb7e626b740 1 Processor -- start 2024-09-11T23:33:37.894 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.889+0000 7fb7e626b740 1 -- start start 2024-09-11T23:33:37.894 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.889+0000 7fb7e626b740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55deedcebaf0 0x55deedcf43f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:37.894 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.889+0000 7fb7e626b740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55deedcec410 0x55deedd0eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:37.894 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.889+0000 7fb7e626b740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55deedcf6cb0 0x55deedd0eff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:37.894 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.889+0000 7fb7e626b740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb7dc1ad960 con 0x55deedcf6cb0 2024-09-11T23:33:37.894 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.889+0000 7fb7e626b740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fb7dc1af710 con 0x55deedcebaf0 2024-09-11T23:33:37.894 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.889+0000 7fb7e626b740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb7dc1ae060 con 0x55deedcec410 2024-09-11T23:33:37.894 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.889+0000 7fb7e1993640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55deedcf6cb0 0x55deedd0eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:33:37.894 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.889+0000 7fb7e1993640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55deedcf6cb0 0x55deedd0eff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47378/0 (socket says 172.21.15.26:47378) 2024-09-11T23:33:37.894 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.889+0000 7fb7e1993640 1 -- 172.21.15.26:0/3337407276 learned_addr learned my addr 172.21.15.26:0/3337407276 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:33:37.894 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.889+0000 7fb7e1192640 1 --2- 172.21.15.26:0/3337407276 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55deedcebaf0 0x55deedcf43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:33:37.894 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.889+0000 7fb7e0991640 1 --2- 172.21.15.26:0/3337407276 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55deedcec410 0x55deedd0eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:33:37.895 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.889+0000 7fb7e1993640 1 -- 172.21.15.26:0/3337407276 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55deedcec410 msgr2=0x55deedd0eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:33:37.895 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.889+0000 7fb7e1993640 1 --2- 172.21.15.26:0/3337407276 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55deedcec410 0x55deedd0eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:37.895 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.889+0000 7fb7e1993640 1 -- 172.21.15.26:0/3337407276 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55deedcebaf0 msgr2=0x55deedcf43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:33:37.895 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.889+0000 7fb7e1993640 1 --2- 172.21.15.26:0/3337407276 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55deedcebaf0 0x55deedcf43f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:37.895 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.889+0000 7fb7e1993640 1 -- 172.21.15.26:0/3337407276 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55deedce7e90 con 0x55deedcf6cb0 2024-09-11T23:33:37.895 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.890+0000 7fb7e1993640 1 --2- 172.21.15.26:0/3337407276 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55deedcf6cb0 0x55deedd0eff0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7fb7d4031810 tx=0x7fb7d4040000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c9010d89a445917 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:33:37.895 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.890+0000 7fb7c6ffd640 1 -- 172.21.15.26:0/3337407276 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7d4003d70 con 0x55deedcf6cb0 2024-09-11T23:33:37.895 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.890+0000 7fb7c6ffd640 1 -- 172.21.15.26:0/3337407276 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb7d4036040 con 0x55deedcf6cb0 2024-09-11T23:33:37.895 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.890+0000 7fb7c6ffd640 1 -- 172.21.15.26:0/3337407276 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7d4030aa0 con 0x55deedcf6cb0 2024-09-11T23:33:37.895 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.890+0000 7fb7e626b740 1 -- 172.21.15.26:0/3337407276 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55deedcf6cb0 msgr2=0x55deedd0eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:33:37.895 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.890+0000 7fb7e626b740 1 --2- 172.21.15.26:0/3337407276 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55deedcf6cb0 0x55deedd0eff0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7fb7d4031810 tx=0x7fb7d4040000 comp rx=0 tx=0).stop 2024-09-11T23:33:37.895 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.890+0000 7fb7e626b740 1 -- 172.21.15.26:0/3337407276 shutdown_connections 2024-09-11T23:33:37.895 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.890+0000 7fb7e626b740 1 --2- 172.21.15.26:0/3337407276 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55deedcf6cb0 0x55deedd0eff0 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:37.896 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.890+0000 7fb7e626b740 1 --2- 172.21.15.26:0/3337407276 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55deedcec410 0x55deedd0eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:37.896 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.890+0000 7fb7e626b740 1 --2- 172.21.15.26:0/3337407276 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55deedcebaf0 0x55deedcf43f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:37.896 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.890+0000 7fb7e626b740 1 -- 172.21.15.26:0/3337407276 >> 172.21.15.26:0/3337407276 conn(0x55deedce8060 msgr2=0x55deedcf5bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:33:37.896 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.890+0000 7fb7e626b740 1 -- 172.21.15.26:0/3337407276 shutdown_connections 2024-09-11T23:33:37.896 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.891+0000 7fb7e626b740 1 -- 172.21.15.26:0/3337407276 wait complete. 2024-09-11T23:33:37.896 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.891+0000 7fb7e626b740 1 -- start start 2024-09-11T23:33:37.896 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.891+0000 7fb7e626b740 1 Processor -- start 2024-09-11T23:33:37.896 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.891+0000 7fb7e626b740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55deedcebaf0 0x55deedcff2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:37.896 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.891+0000 7fb7e626b740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55deedcec410 0x55deedcf0740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:37.896 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.891+0000 7fb7e626b740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55deedcf6cb0 0x55deedcf0c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:37.896 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.891+0000 7fb7e626b740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb7dc1aef80 con 0x55deedcebaf0 2024-09-11T23:33:37.896 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.891+0000 7fb7e626b740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fb7dc1ae7f0 con 0x55deedcec410 2024-09-11T23:33:37.896 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.891+0000 7fb7e626b740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb7dc1ab960 con 0x55deedcf6cb0 2024-09-11T23:33:37.897 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.891+0000 7fb7e626b740 10 client.0 _ll_register_callbacks cb 0x7fb7d8009d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:33:37.897 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.891+0000 7fb7e1192640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55deedcebaf0 0x55deedcff2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:33:37.897 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.891+0000 7fb7e1192640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55deedcebaf0 0x55deedcff2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47382/0 (socket says 172.21.15.26:47382) 2024-09-11T23:33:37.897 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.891+0000 7fb7e1192640 1 -- 172.21.15.26:0/3364715050 learned_addr learned my addr 172.21.15.26:0/3364715050 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:33:37.897 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.892+0000 7fb7e0991640 1 --2- 172.21.15.26:0/3364715050 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55deedcec410 0x55deedcf0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:33:37.897 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.892+0000 7fb7e1993640 1 --2- 172.21.15.26:0/3364715050 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55deedcf6cb0 0x55deedcf0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:33:37.897 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.892+0000 7fb7e1192640 1 -- 172.21.15.26:0/3364715050 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55deedcf6cb0 msgr2=0x55deedcf0c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:33:37.897 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.892+0000 7fb7e1192640 1 --2- 172.21.15.26:0/3364715050 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55deedcf6cb0 0x55deedcf0c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:37.897 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.892+0000 7fb7e1192640 1 -- 172.21.15.26:0/3364715050 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55deedcec410 msgr2=0x55deedcf0740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:33:37.897 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.892+0000 7fb7e1192640 1 --2- 172.21.15.26:0/3364715050 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55deedcec410 0x55deedcf0740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:37.897 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.892+0000 7fb7e1192640 1 -- 172.21.15.26:0/3364715050 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55deedcf1320 con 0x55deedcebaf0 2024-09-11T23:33:37.897 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.892+0000 7fb7e1993640 1 --2- 172.21.15.26:0/3364715050 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55deedcf6cb0 0x55deedcf0c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:33:37.897 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.892+0000 7fb7e1192640 1 --2- 172.21.15.26:0/3364715050 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55deedcebaf0 0x55deedcff2c0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7fb7d8005d00 tx=0x7fb7d8005d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f1fcb5d64435bd server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:33:37.898 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.892+0000 7fb7c4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:33:37.898 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.892+0000 7fb7c4ff9640 1 -- 172.21.15.26:0/3364715050 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7d8005040 con 0x55deedcebaf0 2024-09-11T23:33:37.898 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.892+0000 7fb7c4ff9640 1 -- 172.21.15.26:0/3364715050 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8586+0+0 (secure 0 0 0) 0x7fb7d80051e0 con 0x55deedcebaf0 2024-09-11T23:33:37.898 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.892+0000 7fb7e626b740 1 -- 172.21.15.26:0/3364715050 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb7d4036040 con 0x55deedcebaf0 2024-09-11T23:33:37.898 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.892+0000 7fb7c4ff9640 1 -- 172.21.15.26:0/3364715050 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) ==== 24+0+0 (secure 0 0 0) 0x7fb7d80053e0 con 0x55deedcebaf0 2024-09-11T23:33:37.898 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.892+0000 7fb7e626b740 10 client.17544 fetch_fsmap learned FSMap version 73 2024-09-11T23:33:37.898 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.892+0000 7fb7e626b740 1 -- 172.21.15.26:0/3364715050 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=4+}) -- 0x55deedd75d10 con 0x55deedcebaf0 2024-09-11T23:33:37.898 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.892+0000 7fb7c4ff9640 1 -- 172.21.15.26:0/3364715050 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb7d4036040 con 0x55deedcebaf0 2024-09-11T23:33:37.898 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.893+0000 7fb7c4ff9640 1 -- 172.21.15.26:0/3364715050 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 73) ==== 8430+0+0 (secure 0 0 0) 0x7fb7d8031030 con 0x55deedcebaf0 2024-09-11T23:33:37.898 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.893+0000 7fb7e626b740 10 client.17544 fetch_fsmap finished waiting for FSMap version 73 2024-09-11T23:33:37.898 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.893+0000 7fb7e626b740 10 client.17544 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:33:37.898 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.893+0000 7fb7e626b740 20 client.17544 populate_metadata read hostname 'smithi026' 2024-09-11T23:33:37.898 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.893+0000 7fb7e626b740 1 --2- 172.21.15.26:0/3364715050 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55deedd76570 0x55deedd78990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:37.899 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.893+0000 7fb7e626b740 4 client.17544 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T23:33:37.899 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.893+0000 7fb7e626b740 1 -- 172.21.15.26:0/3364715050 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55deedc57d70 con 0x55deedd76570 2024-09-11T23:33:37.899 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.893+0000 7fb7e0991640 1 --2- 172.21.15.26:0/3364715050 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55deedd76570 0x55deedd78990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:33:37.899 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.893+0000 7fb7e0991640 1 --2- 172.21.15.26:0/3364715050 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55deedd76570 0x55deedd78990 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51eb80ab30ec0d0e server_cookie=f1a391bc0cdf1e57 in_seq=0 out_seq=0 2024-09-11T23:33:37.899 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.893+0000 7fb7c4ff9640 10 client.17544 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:33:37.901 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.900+0000 7fb7c4ff9640 1 -- 172.21.15.26:0/3364715050 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55deedc57d70 con 0x55deedd76570 2024-09-11T23:33:37.901 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.900+0000 7fb7c4ff9640 10 client.17544 handle_command_reply: tid=0 2024-09-11T23:33:37.902 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.900+0000 7fb7e626b740 1 client.17544 shutdown 2024-09-11T23:33:37.902 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.901+0000 7fb7e626b740 1 -- 172.21.15.26:0/3364715050 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55deedcebaf0 msgr2=0x55deedcff2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:33:37.902 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.901+0000 7fb7e626b740 1 --2- 172.21.15.26:0/3364715050 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55deedcebaf0 0x55deedcff2c0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7fb7d8005d00 tx=0x7fb7d8005d30 comp rx=0 tx=0).stop 2024-09-11T23:33:37.902 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.901+0000 7fb7e626b740 1 -- 172.21.15.26:0/3364715050 shutdown_connections 2024-09-11T23:33:37.902 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.901+0000 7fb7e626b740 1 --2- 172.21.15.26:0/3364715050 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55deedcec410 0x55deedcf0740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:37.902 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.901+0000 7fb7e626b740 1 --2- 172.21.15.26:0/3364715050 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55deedd76570 0x55deedd78990 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:37.902 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.901+0000 7fb7e626b740 1 --2- 172.21.15.26:0/3364715050 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55deedcf6cb0 0x55deedcf0c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:37.902 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.901+0000 7fb7e626b740 1 --2- 172.21.15.26:0/3364715050 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55deedcebaf0 0x55deedcff2c0 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:37.902 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.901+0000 7fb7c4ff9640 0 client.17544 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:33:37.902 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.901+0000 7fb7e626b740 1 -- 172.21.15.26:0/3364715050 >> 172.21.15.26:0/3364715050 conn(0x55deedce8060 msgr2=0x55deedced550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:33:37.902 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.901+0000 7fb7e626b740 1 -- 172.21.15.26:0/3364715050 shutdown_connections 2024-09-11T23:33:37.902 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.901+0000 7fb7e626b740 1 -- 172.21.15.26:0/3364715050 wait complete. 2024-09-11T23:33:37.902 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.901+0000 7fb7e626b740 20 client.17544 trim_cache size 0 max 16384 2024-09-11T23:33:37.903 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-09-11T23:33:37.903 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "message": "", 2024-09-11T23:33:37.903 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-09-11T23:33:37.903 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-09-11T23:33:37.904 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.901+0000 7fb7e33f6640 1 -- 172.21.15.26:0/3347353067 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fb7c80a2da0 msgr2=0x7fb7c80a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:33:37.904 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.901+0000 7fb7e33f6640 1 --2- 172.21.15.26:0/3347353067 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fb7c80a2da0 0x7fb7c80a5260 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fb7d4004af0 tx=0x7fb7d40095c0 comp rx=0 tx=0).stop 2024-09-11T23:33:37.904 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.902+0000 7fb7e33f6640 1 -- 172.21.15.26:0/3347353067 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb7dc102470 msgr2=0x7fb7dc198830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:33:37.904 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.902+0000 7fb7e33f6640 1 --2- 172.21.15.26:0/3347353067 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb7dc102470 0x7fb7dc198830 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7fb7cc00d950 tx=0x7fb7cc00de10 comp rx=0 tx=0).stop 2024-09-11T23:33:37.904 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.902+0000 7fb7e33f6640 1 -- 172.21.15.26:0/3347353067 shutdown_connections 2024-09-11T23:33:37.904 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.902+0000 7fb7e33f6640 1 --2- 172.21.15.26:0/3347353067 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fb7c80a2da0 0x7fb7c80a5260 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:37.904 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.902+0000 7fb7e33f6640 1 --2- 172.21.15.26:0/3347353067 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb7dc107b70 0x7fb7dc19cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:37.904 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.902+0000 7fb7e33f6640 1 --2- 172.21.15.26:0/3347353067 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb7dc102470 0x7fb7dc198830 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:37.904 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.902+0000 7fb7e33f6640 1 --2- 172.21.15.26:0/3347353067 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb7dc101ad0 0x7fb7dc1982f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:37.904 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.902+0000 7fb7e33f6640 1 -- 172.21.15.26:0/3347353067 >> 172.21.15.26:0/3347353067 conn(0x7fb7dc0fd7d0 msgr2=0x7fb7dc0fdde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:33:37.905 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.902+0000 7fb7e33f6640 1 -- 172.21.15.26:0/3347353067 shutdown_connections 2024-09-11T23:33:37.905 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:37.902+0000 7fb7e33f6640 1 -- 172.21.15.26:0/3347353067 wait complete. 2024-09-11T23:33:37.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:37.980+0000 7fb12a7d6640 1 -- 172.21.15.26:0/2410170561 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb124199830 con 0x7fb124101420 2024-09-11T23:33:37.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:37.980+0000 7fb1217fa640 1 -- 172.21.15.26:0/2410170561 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) ==== 94+0+12001 (secure 0 0 0) 0x7fb1180a0020 con 0x7fb124101420 2024-09-11T23:33:37.981 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-11T23:33:37.981 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":73,"btime":"2024-09-11T23:33:13:521783+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14499,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/367659515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":367659515},{"type":"v1","addr":"172.21.15.26:6835","nonce":367659515}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14511,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/2041065299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":2041065299},{"type":"v1","addr":"172.21.15.26:6837","nonce":2041065299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":73,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:16:26.544203+0000","modified":"2024-09-11T23:33:12.865616+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24485,"mds_1":24463,"mds_2":24458,"mds_3":24448,"mds_4":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14550":{"gid":14550,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6839/2233068305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":2233068305},{"type":"v1","addr":"172.21.15.26:6839","nonce":2233068305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14562":{"gid":14562,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6841/2984982336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":2984982336},{"type":"v1","addr":"172.21.15.26:6841","nonce":2984982336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6833/2019841455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":2019841455},{"type":"v1","addr":"172.21.15.136:6833","nonce":2019841455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.76:6833/3438147050","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3438147050},{"type":"v1","addr":"172.21.15.76:6833","nonce":3438147050}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6835/3617665975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3617665975},{"type":"v1","addr":"172.21.15.136:6835","nonce":3617665975}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":12,"addr":"172.21.15.76:6835/393308499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":393308499},{"type":"v1","addr":"172.21.15.76:6835","nonce":393308499}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":10,"addr":"172.21.15.136:6837/1633082387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":1633082387},{"type":"v1","addr":"172.21.15.136:6837","nonce":1633082387}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.76:6837/2078978748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":2078978748},{"type":"v1","addr":"172.21.15.76:6837","nonce":2078978748}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":63,"addr":"172.21.15.136:6839/3211398630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":3211398630},{"type":"v1","addr":"172.21.15.136:6839","nonce":3211398630}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.76:6839/2077637090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6838","nonce":2077637090},{"type":"v1","addr":"172.21.15.76:6839","nonce":2077637090}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24485,"qdb_cluster":[24443,24448,24458,24463,24485]},"id":1}]} 2024-09-11T23:33:37.982 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 73 2024-09-11T23:33:37.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:37.983+0000 7fb12a7d6640 1 -- 172.21.15.26:0/2410170561 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fb10c0a2d70 msgr2=0x7fb10c0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:33:37.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:37.983+0000 7fb12a7d6640 1 --2- 172.21.15.26:0/2410170561 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fb10c0a2d70 0x7fb10c0a5230 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fb114000c00 tx=0x7fb114005a20 comp rx=0 tx=0).stop 2024-09-11T23:33:37.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:37.983+0000 7fb12a7d6640 1 -- 172.21.15.26:0/2410170561 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb124101420 msgr2=0x7fb124198a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:33:37.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:37.983+0000 7fb12a7d6640 1 --2- 172.21.15.26:0/2410170561 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb124101420 0x7fb124198a10 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7fb11800bf20 tx=0x7fb11800dd60 comp rx=0 tx=0).stop 2024-09-11T23:33:37.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:37.984+0000 7fb12a7d6640 1 -- 172.21.15.26:0/2410170561 shutdown_connections 2024-09-11T23:33:37.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:37.984+0000 7fb12a7d6640 1 --2- 172.21.15.26:0/2410170561 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fb10c0a2d70 0x7fb10c0a5230 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:37.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:37.984+0000 7fb12a7d6640 1 --2- 172.21.15.26:0/2410170561 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb124106a80 0x7fb12419cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:37.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:37.984+0000 7fb12a7d6640 1 --2- 172.21.15.26:0/2410170561 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb124101420 0x7fb124198a10 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:37.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:37.984+0000 7fb12a7d6640 1 --2- 172.21.15.26:0/2410170561 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb124100a80 0x7fb1241984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:37.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:37.984+0000 7fb12a7d6640 1 -- 172.21.15.26:0/2410170561 >> 172.21.15.26:0/2410170561 conn(0x7fb1240fc7a0 msgr2=0x7fb12406aad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:33:37.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:37.984+0000 7fb12a7d6640 1 -- 172.21.15.26:0/2410170561 shutdown_connections 2024-09-11T23:33:37.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:37.984+0000 7fb12a7d6640 1 -- 172.21.15.26:0/2410170561 wait complete. 2024-09-11T23:33:37.988 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:33:37 smithi136 ceph-mon[35121]: pgmap v477: 97 pgs: 97 active+clean; 2.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-11T23:33:38.052 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:33:37 smithi076 ceph-mon[35296]: pgmap v477: 97 pgs: 97 active+clean; 2.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-11T23:33:38.707 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24485 quiesce db --set-id c2054eab --release --await' 2024-09-11T23:33:38.707 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24485 quiesce db --set-id c2054eab --release --await 2024-09-11T23:33:38.988 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:33:38 smithi136 ceph-mon[35121]: from='client.? 172.21.15.26:0/2410170561' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:33:39.000 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:33:38 smithi026 ceph-mon[31172]: from='client.? 172.21.15.26:0/2410170561' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:33:39.008 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:33:39.052 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:33:38 smithi076 ceph-mon[35296]: from='client.? 172.21.15.26:0/2410170561' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:33:39.654 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:33:39 smithi026 ceph-mon[31172]: pgmap v478: 97 pgs: 97 active+clean; 2.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-11T23:33:39.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.653+0000 7f07d46eb640 1 -- 172.21.15.26:0/2900054904 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07cc102d40 msgr2=0x7f07cc103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:33:39.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.653+0000 7f07d46eb640 1 --2- 172.21.15.26:0/2900054904 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07cc102d40 0x7f07cc103120 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f07bc009900 tx=0x7f07bc02a480 comp rx=0 tx=0).stop 2024-09-11T23:33:39.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.653+0000 7f07d46eb640 1 -- 172.21.15.26:0/2900054904 shutdown_connections 2024-09-11T23:33:39.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.653+0000 7f07d46eb640 1 --2- 172.21.15.26:0/2900054904 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f07cc10c170 0x7f07cc10e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:39.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.653+0000 7f07d46eb640 1 --2- 172.21.15.26:0/2900054904 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f07cc103660 0x7f07cc10bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:39.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.653+0000 7f07d46eb640 1 --2- 172.21.15.26:0/2900054904 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07cc102d40 0x7f07cc103120 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:39.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.653+0000 7f07d46eb640 1 -- 172.21.15.26:0/2900054904 >> 172.21.15.26:0/2900054904 conn(0x7f07cc0fc7a0 msgr2=0x7f07cc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:33:39.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.654+0000 7f07d46eb640 1 -- 172.21.15.26:0/2900054904 shutdown_connections 2024-09-11T23:33:39.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.654+0000 7f07d46eb640 1 -- 172.21.15.26:0/2900054904 wait complete. 2024-09-11T23:33:39.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.654+0000 7f07d46eb640 1 Processor -- start 2024-09-11T23:33:39.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.655+0000 7f07d46eb640 1 -- start start 2024-09-11T23:33:39.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.655+0000 7f07d46eb640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f07cc102d40 0x7f07cc1984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:39.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.655+0000 7f07d46eb640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07cc103660 0x7f07cc198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:39.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.655+0000 7f07d46eb640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f07cc10c170 0x7f07cc19cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:39.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.655+0000 7f07d46eb640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f07cc111f70 con 0x7f07cc103660 2024-09-11T23:33:39.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.655+0000 7f07d46eb640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f07cc111df0 con 0x7f07cc10c170 2024-09-11T23:33:39.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.655+0000 7f07d46eb640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f07cc1120f0 con 0x7f07cc102d40 2024-09-11T23:33:39.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.655+0000 7f07d1c86640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07cc103660 0x7f07cc198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:33:39.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.655+0000 7f07d1c86640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07cc103660 0x7f07cc198a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47404/0 (socket says 172.21.15.26:47404) 2024-09-11T23:33:39.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.655+0000 7f07d1c86640 1 -- 172.21.15.26:0/113692708 learned_addr learned my addr 172.21.15.26:0/113692708 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:33:39.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.655+0000 7f07d2c88640 1 --2- 172.21.15.26:0/113692708 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f07cc10c170 0x7f07cc19cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:33:39.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.655+0000 7f07d2487640 1 --2- 172.21.15.26:0/113692708 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f07cc102d40 0x7f07cc1984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:33:39.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.655+0000 7f07d1c86640 1 -- 172.21.15.26:0/113692708 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f07cc102d40 msgr2=0x7f07cc1984d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:33:39.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.655+0000 7f07d1c86640 1 --2- 172.21.15.26:0/113692708 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f07cc102d40 0x7f07cc1984d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:39.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.655+0000 7f07d1c86640 1 -- 172.21.15.26:0/113692708 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f07cc10c170 msgr2=0x7f07cc19cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:33:39.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.655+0000 7f07d1c86640 1 --2- 172.21.15.26:0/113692708 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f07cc10c170 0x7f07cc19cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:39.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.655+0000 7f07d1c86640 1 -- 172.21.15.26:0/113692708 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07cc19d480 con 0x7f07cc103660 2024-09-11T23:33:39.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.655+0000 7f07d2487640 1 --2- 172.21.15.26:0/113692708 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f07cc102d40 0x7f07cc1984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:33:39.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.656+0000 7f07d1c86640 1 --2- 172.21.15.26:0/113692708 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07cc103660 0x7f07cc198a10 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f07c000ea00 tx=0x7f07c000eec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:33:39.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.656+0000 7f07cb7fe640 1 -- 172.21.15.26:0/113692708 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07c000cd80 con 0x7f07cc103660 2024-09-11T23:33:39.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.656+0000 7f07cb7fe640 1 -- 172.21.15.26:0/113692708 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f07c0011950 con 0x7f07cc103660 2024-09-11T23:33:39.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.656+0000 7f07d46eb640 1 -- 172.21.15.26:0/113692708 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07cc19d770 con 0x7f07cc103660 2024-09-11T23:33:39.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.656+0000 7f07cb7fe640 1 -- 172.21.15.26:0/113692708 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07c001aa30 con 0x7f07cc103660 2024-09-11T23:33:39.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.656+0000 7f07d46eb640 1 -- 172.21.15.26:0/113692708 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07cc1a5050 con 0x7f07cc103660 2024-09-11T23:33:39.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.658+0000 7f07cb7fe640 1 -- 172.21.15.26:0/113692708 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f07c0011b00 con 0x7f07cc103660 2024-09-11T23:33:39.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.658+0000 7f07d6ac4740 1 Processor -- start 2024-09-11T23:33:39.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.659+0000 7f07d6ac4740 1 -- start start 2024-09-11T23:33:39.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.659+0000 7f07d6ac4740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561906958060 0x561906958440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:39.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.659+0000 7f07d6ac4740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561906958980 0x561906962d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:39.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.659+0000 7f07d6ac4740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561906963240 0x561906965630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:39.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.659+0000 7f07d6ac4740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5619068bd620 con 0x561906958980 2024-09-11T23:33:39.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.659+0000 7f07d6ac4740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x561906864e50 con 0x561906963240 2024-09-11T23:33:39.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.659+0000 7f07d6ac4740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5619068b95e0 con 0x561906958060 2024-09-11T23:33:39.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.659+0000 7f07d2487640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561906958980 0x561906962d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:33:39.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.659+0000 7f07d2487640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561906958980 0x561906962d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47420/0 (socket says 172.21.15.26:47420) 2024-09-11T23:33:39.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.659+0000 7f07d2487640 1 -- 172.21.15.26:0/1255199752 learned_addr learned my addr 172.21.15.26:0/1255199752 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:33:39.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.659+0000 7f07cb7fe640 1 --2- 172.21.15.26:0/113692708 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f07b40a2cf0 0x7f07b40a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:39.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.659+0000 7f07d1c86640 1 --2- 172.21.15.26:0/1255199752 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561906963240 0x561906965630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:33:39.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.659+0000 7f07d2c88640 1 --2- 172.21.15.26:0/1255199752 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561906958060 0x561906958440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:33:39.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.659+0000 7f07d2c88640 1 --2- 172.21.15.26:0/113692708 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f07b40a2cf0 0x7f07b40a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:33:39.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.659+0000 7f07d2487640 1 -- 172.21.15.26:0/1255199752 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561906958060 msgr2=0x561906958440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:33:39.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.659+0000 7f07d2487640 1 --2- 172.21.15.26:0/1255199752 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561906958060 0x561906958440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:39.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.659+0000 7f07cb7fe640 1 -- 172.21.15.26:0/113692708 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8586+0+0 (secure 0 0 0) 0x7f07c00cf970 con 0x7f07cc103660 2024-09-11T23:33:39.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.659+0000 7f07d2487640 1 -- 172.21.15.26:0/1255199752 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561906963240 msgr2=0x561906965630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:33:39.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.659+0000 7f07d2487640 1 --2- 172.21.15.26:0/1255199752 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561906963240 0x561906965630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:39.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.659+0000 7f07d2487640 1 -- 172.21.15.26:0/1255199752 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561906855d40 con 0x561906958980 2024-09-11T23:33:39.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.660+0000 7f07d2487640 1 --2- 172.21.15.26:0/1255199752 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561906958980 0x561906962d00 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f07bc033000 tx=0x7f07bc005e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17950c24964c89e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:33:39.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.660+0000 7f07c8ff9640 1 -- 172.21.15.26:0/1255199752 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07bc03f070 con 0x561906958980 2024-09-11T23:33:39.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.660+0000 7f07d2c88640 1 --2- 172.21.15.26:0/113692708 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f07b40a2cf0 0x7f07b40a51b0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f07c4005f60 tx=0x7f07c4005f90 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:33:39.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.660+0000 7f07c8ff9640 1 -- 172.21.15.26:0/1255199752 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f07bc004030 con 0x561906958980 2024-09-11T23:33:39.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.660+0000 7f07c8ff9640 1 -- 172.21.15.26:0/1255199752 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07bc035070 con 0x561906958980 2024-09-11T23:33:39.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.661+0000 7f07d6ac4740 1 -- 172.21.15.26:0/1255199752 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561906958980 msgr2=0x561906962d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:33:39.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.661+0000 7f07d6ac4740 1 --2- 172.21.15.26:0/1255199752 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561906958980 0x561906962d00 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f07bc033000 tx=0x7f07bc005e60 comp rx=0 tx=0).stop 2024-09-11T23:33:39.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.661+0000 7f07d6ac4740 1 -- 172.21.15.26:0/1255199752 shutdown_connections 2024-09-11T23:33:39.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.661+0000 7f07d6ac4740 1 --2- 172.21.15.26:0/1255199752 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561906963240 0x561906965630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:39.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.661+0000 7f07d6ac4740 1 --2- 172.21.15.26:0/1255199752 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561906958980 0x561906962d00 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:39.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.661+0000 7f07d6ac4740 1 --2- 172.21.15.26:0/1255199752 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561906958060 0x561906958440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:39.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.661+0000 7f07d6ac4740 1 -- 172.21.15.26:0/1255199752 >> 172.21.15.26:0/1255199752 conn(0x5619069545f0 msgr2=0x5619069549d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:33:39.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.661+0000 7f07d6ac4740 1 -- 172.21.15.26:0/1255199752 shutdown_connections 2024-09-11T23:33:39.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.661+0000 7f07d6ac4740 1 -- 172.21.15.26:0/1255199752 wait complete. 2024-09-11T23:33:39.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.661+0000 7f07d6ac4740 1 -- start start 2024-09-11T23:33:39.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.661+0000 7f07d6ac4740 1 Processor -- start 2024-09-11T23:33:39.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.661+0000 7f07d6ac4740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561906958060 0x561906960100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:39.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.661+0000 7f07d6ac4740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561906958980 0x561906960640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:39.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.661+0000 7f07d6ac4740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561906963240 0x561906960b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:39.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.661+0000 7f07d6ac4740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x561906965e10 con 0x561906963240 2024-09-11T23:33:39.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.661+0000 7f07d6ac4740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x561906845f10 con 0x561906958980 2024-09-11T23:33:39.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.661+0000 7f07d6ac4740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x561906965f90 con 0x561906958060 2024-09-11T23:33:39.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.661+0000 7f07d6ac4740 10 client.0 _ll_register_callbacks cb 0x561906869db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:33:39.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.662+0000 7f07d2c88640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561906963240 0x561906960b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:33:39.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.662+0000 7f07d2c88640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561906963240 0x561906960b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47436/0 (socket says 172.21.15.26:47436) 2024-09-11T23:33:39.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.662+0000 7f07d1c86640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561906958980 0x561906960640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:33:39.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.662+0000 7f07d2c88640 1 -- 172.21.15.26:0/2753012555 learned_addr learned my addr 172.21.15.26:0/2753012555 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:33:39.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.662+0000 7f07d2487640 1 --2- 172.21.15.26:0/2753012555 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561906958060 0x561906960100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:33:39.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.662+0000 7f07d2c88640 1 -- 172.21.15.26:0/2753012555 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561906958060 msgr2=0x561906960100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:33:39.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.662+0000 7f07d2c88640 1 --2- 172.21.15.26:0/2753012555 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561906958060 0x561906960100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:39.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.662+0000 7f07d2c88640 1 -- 172.21.15.26:0/2753012555 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561906958980 msgr2=0x561906960640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:33:39.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.662+0000 7f07d2c88640 1 --2- 172.21.15.26:0/2753012555 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561906958980 0x561906960640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:39.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.662+0000 7f07d2c88640 1 -- 172.21.15.26:0/2753012555 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5619065cc580 con 0x561906963240 2024-09-11T23:33:39.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.662+0000 7f07d2c88640 1 --2- 172.21.15.26:0/2753012555 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561906963240 0x561906960b80 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f07c40045b0 tx=0x7f07c40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b5e970370a027e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:33:39.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.662+0000 7f07b1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:33:39.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.662+0000 7f07b1ffb640 1 -- 172.21.15.26:0/2753012555 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07c4026020 con 0x561906963240 2024-09-11T23:33:39.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.662+0000 7f07b1ffb640 1 -- 172.21.15.26:0/2753012555 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8586+0+0 (secure 0 0 0) 0x7f07c400f690 con 0x561906963240 2024-09-11T23:33:39.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.662+0000 7f07d6ac4740 1 -- 172.21.15.26:0/2753012555 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f07bc004030 con 0x561906963240 2024-09-11T23:33:39.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.662+0000 7f07b1ffb640 1 -- 172.21.15.26:0/2753012555 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) ==== 24+0+0 (secure 0 0 0) 0x7f07c400f890 con 0x561906963240 2024-09-11T23:33:39.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.662+0000 7f07d6ac4740 10 client.17556 fetch_fsmap learned FSMap version 73 2024-09-11T23:33:39.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.662+0000 7f07d6ac4740 1 -- 172.21.15.26:0/2753012555 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=4+}) -- 0x561906696960 con 0x561906963240 2024-09-11T23:33:39.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.663+0000 7f07b1ffb640 1 -- 172.21.15.26:0/2753012555 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f07bc004030 con 0x561906963240 2024-09-11T23:33:39.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.663+0000 7f07b1ffb640 1 -- 172.21.15.26:0/2753012555 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 73) ==== 8430+0+0 (secure 0 0 0) 0x7f07c4020040 con 0x561906963240 2024-09-11T23:33:39.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.663+0000 7f07d6ac4740 10 client.17556 fetch_fsmap finished waiting for FSMap version 73 2024-09-11T23:33:39.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.663+0000 7f07d6ac4740 10 client.17556 resolve_mds: validated gid 24485 aka daemon mds.b 2024-09-11T23:33:39.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.663+0000 7f07d6ac4740 20 client.17556 populate_metadata read hostname 'smithi026' 2024-09-11T23:33:39.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.663+0000 7f07d6ac4740 1 --2- 172.21.15.26:0/2753012555 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x561906972eb0 0x561906975350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:39.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.663+0000 7f07d6ac4740 4 client.17556 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:33:39.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.663+0000 7f07d6ac4740 1 -- 172.21.15.26:0/2753012555 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5619068cab90 con 0x561906972eb0 2024-09-11T23:33:39.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.663+0000 7f07d2487640 1 --2- 172.21.15.26:0/2753012555 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x561906972eb0 0x561906975350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:33:39.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.664+0000 7f07d2487640 1 --2- 172.21.15.26:0/2753012555 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x561906972eb0 0x561906975350 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe6ab0562317139c server_cookie=717c7a7c80aada24 in_seq=0 out_seq=0 2024-09-11T23:33:39.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.664+0000 7f07b1ffb640 10 client.17556 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:33:39.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.664+0000 7f07b1ffb640 1 -- 172.21.15.26:0/2753012555 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5619068cab90 con 0x561906972eb0 2024-09-11T23:33:39.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.664+0000 7f07b1ffb640 10 client.17556 handle_command_reply: tid=0 2024-09-11T23:33:39.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.664+0000 7f07d6ac4740 1 client.17556 shutdown 2024-09-11T23:33:39.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.664+0000 7f07d6ac4740 1 -- 172.21.15.26:0/2753012555 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561906963240 msgr2=0x561906960b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:33:39.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.664+0000 7f07d6ac4740 1 --2- 172.21.15.26:0/2753012555 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561906963240 0x561906960b80 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f07c40045b0 tx=0x7f07c40046c0 comp rx=0 tx=0).stop 2024-09-11T23:33:39.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.664+0000 7f07d6ac4740 1 -- 172.21.15.26:0/2753012555 shutdown_connections 2024-09-11T23:33:39.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.664+0000 7f07d6ac4740 1 --2- 172.21.15.26:0/2753012555 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561906958980 0x561906960640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:39.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.664+0000 7f07d6ac4740 1 --2- 172.21.15.26:0/2753012555 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x561906972eb0 0x561906975350 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:39.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.664+0000 7f07b1ffb640 0 client.17556 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:33:39.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.664+0000 7f07d6ac4740 1 --2- 172.21.15.26:0/2753012555 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561906963240 0x561906960b80 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:39.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.664+0000 7f07d6ac4740 1 --2- 172.21.15.26:0/2753012555 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561906958060 0x561906960100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:39.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.665+0000 7f07d6ac4740 1 -- 172.21.15.26:0/2753012555 >> 172.21.15.26:0/2753012555 conn(0x5619069545f0 msgr2=0x56190695a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:33:39.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.665+0000 7f07d6ac4740 1 -- 172.21.15.26:0/2753012555 shutdown_connections 2024-09-11T23:33:39.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.665+0000 7f07d6ac4740 1 -- 172.21.15.26:0/2753012555 wait complete. 2024-09-11T23:33:39.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.665+0000 7f07d6ac4740 20 client.17556 trim_cache size 0 max 16384 2024-09-11T23:33:39.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.681+0000 7f07d6ac4740 1 Processor -- start 2024-09-11T23:33:39.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.682+0000 7f07d6ac4740 1 -- start start 2024-09-11T23:33:39.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.682+0000 7f07d6ac4740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561906958060 0x56190696b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:39.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.682+0000 7f07d6ac4740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561906958980 0x56190696bb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:39.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.682+0000 7f07d6ac4740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561906963240 0x56190697ac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:39.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.682+0000 7f07d6ac4740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f07cc1ad990 con 0x561906963240 2024-09-11T23:33:39.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.682+0000 7f07d6ac4740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f07cc1af740 con 0x561906958980 2024-09-11T23:33:39.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.682+0000 7f07d6ac4740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f07cc1ae090 con 0x561906958060 2024-09-11T23:33:39.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.682+0000 7f07d2c88640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561906963240 0x56190697ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:33:39.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.682+0000 7f07d2c88640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561906963240 0x56190697ac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47442/0 (socket says 172.21.15.26:47442) 2024-09-11T23:33:39.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.682+0000 7f07d2c88640 1 -- 172.21.15.26:0/153960441 learned_addr learned my addr 172.21.15.26:0/153960441 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:33:39.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.682+0000 7f07d2487640 1 --2- 172.21.15.26:0/153960441 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561906958060 0x56190696b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:33:39.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.682+0000 7f07d1c86640 1 --2- 172.21.15.26:0/153960441 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561906958980 0x56190696bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:33:39.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.682+0000 7f07d2c88640 1 -- 172.21.15.26:0/153960441 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561906958060 msgr2=0x56190696b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:33:39.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.682+0000 7f07d2c88640 1 --2- 172.21.15.26:0/153960441 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561906958060 0x56190696b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:39.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.682+0000 7f07d2c88640 1 -- 172.21.15.26:0/153960441 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561906958980 msgr2=0x56190696bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:33:39.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.682+0000 7f07d2c88640 1 --2- 172.21.15.26:0/153960441 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561906958980 0x56190696bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:39.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.682+0000 7f07d2c88640 1 -- 172.21.15.26:0/153960441 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561906954090 con 0x561906963240 2024-09-11T23:33:39.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.682+0000 7f07d2c88640 1 --2- 172.21.15.26:0/153960441 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561906963240 0x56190697ac60 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f07c4000c00 tx=0x7f07c4009830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b82789a7f6e18ca6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:33:39.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.683+0000 7f07b3fff640 1 -- 172.21.15.26:0/153960441 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07c4026070 con 0x561906963240 2024-09-11T23:33:39.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.683+0000 7f07b3fff640 1 -- 172.21.15.26:0/153960441 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f07c4009a20 con 0x561906963240 2024-09-11T23:33:39.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.683+0000 7f07b3fff640 1 -- 172.21.15.26:0/153960441 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07c4035690 con 0x561906963240 2024-09-11T23:33:39.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.683+0000 7f07d6ac4740 1 -- 172.21.15.26:0/153960441 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561906963240 msgr2=0x56190697ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:33:39.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.683+0000 7f07d6ac4740 1 --2- 172.21.15.26:0/153960441 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561906963240 0x56190697ac60 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f07c4000c00 tx=0x7f07c4009830 comp rx=0 tx=0).stop 2024-09-11T23:33:39.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.683+0000 7f07d6ac4740 1 -- 172.21.15.26:0/153960441 shutdown_connections 2024-09-11T23:33:39.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.683+0000 7f07d6ac4740 1 --2- 172.21.15.26:0/153960441 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561906958980 0x56190696bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:39.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.683+0000 7f07d6ac4740 1 --2- 172.21.15.26:0/153960441 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561906963240 0x56190697ac60 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:39.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.683+0000 7f07d6ac4740 1 --2- 172.21.15.26:0/153960441 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561906958060 0x56190696b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:39.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.683+0000 7f07d6ac4740 1 -- 172.21.15.26:0/153960441 >> 172.21.15.26:0/153960441 conn(0x5619069545f0 msgr2=0x561906962180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:33:39.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.683+0000 7f07d6ac4740 1 -- 172.21.15.26:0/153960441 shutdown_connections 2024-09-11T23:33:39.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.683+0000 7f07d6ac4740 1 -- 172.21.15.26:0/153960441 wait complete. 2024-09-11T23:33:39.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.684+0000 7f07d6ac4740 1 -- start start 2024-09-11T23:33:39.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.684+0000 7f07d6ac4740 1 Processor -- start 2024-09-11T23:33:39.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.684+0000 7f07d6ac4740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561906958060 0x56190695dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:39.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.684+0000 7f07d6ac4740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561906958980 0x56190695c370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:39.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.684+0000 7f07d6ac4740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561906963240 0x56190695c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:39.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.684+0000 7f07d6ac4740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f07cc1aefb0 con 0x561906958980 2024-09-11T23:33:39.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.684+0000 7f07d6ac4740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f07cc1ae820 con 0x561906958060 2024-09-11T23:33:39.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.684+0000 7f07d6ac4740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f07cc1ab990 con 0x561906963240 2024-09-11T23:33:39.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.684+0000 7f07d6ac4740 10 client.0 _ll_register_callbacks cb 0x7f07c4004410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:33:39.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.684+0000 7f07d1c86640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561906958980 0x56190695c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:33:39.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.684+0000 7f07d1c86640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561906958980 0x56190695c370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47458/0 (socket says 172.21.15.26:47458) 2024-09-11T23:33:39.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.684+0000 7f07d1c86640 1 -- 172.21.15.26:0/3874544156 learned_addr learned my addr 172.21.15.26:0/3874544156 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:33:39.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.684+0000 7f07d2487640 1 --2- 172.21.15.26:0/3874544156 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561906958060 0x56190695dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:33:39.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.684+0000 7f07d2c88640 1 --2- 172.21.15.26:0/3874544156 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561906963240 0x56190695c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:33:39.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.685+0000 7f07d1c86640 1 -- 172.21.15.26:0/3874544156 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561906963240 msgr2=0x56190695c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:33:39.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.685+0000 7f07d1c86640 1 --2- 172.21.15.26:0/3874544156 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561906963240 0x56190695c8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:39.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.685+0000 7f07d1c86640 1 -- 172.21.15.26:0/3874544156 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561906958060 msgr2=0x56190695dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:33:39.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.685+0000 7f07d1c86640 1 --2- 172.21.15.26:0/3874544156 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561906958060 0x56190695dbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:39.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.685+0000 7f07d1c86640 1 -- 172.21.15.26:0/3874544156 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56190695d0e0 con 0x561906958980 2024-09-11T23:33:39.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.685+0000 7f07d1c86640 1 --2- 172.21.15.26:0/3874544156 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561906958980 0x56190695c370 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f07c00cd680 tx=0x7f07c00094a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5587058f50e1684f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:33:39.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.685+0000 7f07b1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:33:39.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.685+0000 7f07b1ffb640 1 -- 172.21.15.26:0/3874544156 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07c00ca070 con 0x561906958980 2024-09-11T23:33:39.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.685+0000 7f07b1ffb640 1 -- 172.21.15.26:0/3874544156 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8586+0+0 (secure 0 0 0) 0x7f07c00a4630 con 0x561906958980 2024-09-11T23:33:39.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.685+0000 7f07d6ac4740 1 -- 172.21.15.26:0/3874544156 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f07c4009a20 con 0x561906958980 2024-09-11T23:33:39.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.685+0000 7f07b1ffb640 1 -- 172.21.15.26:0/3874544156 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) ==== 24+0+0 (secure 0 0 0) 0x7f07c009be50 con 0x561906958980 2024-09-11T23:33:39.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.685+0000 7f07d6ac4740 10 client.17562 fetch_fsmap learned FSMap version 73 2024-09-11T23:33:39.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.685+0000 7f07d6ac4740 1 -- 172.21.15.26:0/3874544156 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=4+}) -- 0x5619069e2d50 con 0x561906958980 2024-09-11T23:33:39.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.685+0000 7f07b1ffb640 1 -- 172.21.15.26:0/3874544156 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f07c4009a20 con 0x561906958980 2024-09-11T23:33:39.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.685+0000 7f07b1ffb640 1 -- 172.21.15.26:0/3874544156 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 73) ==== 8430+0+0 (secure 0 0 0) 0x7f07c0022030 con 0x561906958980 2024-09-11T23:33:39.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.685+0000 7f07d6ac4740 10 client.17562 fetch_fsmap finished waiting for FSMap version 73 2024-09-11T23:33:39.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.685+0000 7f07d6ac4740 10 client.17562 resolve_mds: validated gid 24485 aka daemon mds.b 2024-09-11T23:33:39.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.685+0000 7f07d6ac4740 20 client.17562 populate_metadata read hostname 'smithi026' 2024-09-11T23:33:39.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.685+0000 7f07d6ac4740 1 --2- 172.21.15.26:0/3874544156 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5619069e3510 0x5619069e5930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:39.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.685+0000 7f07d6ac4740 4 client.17562 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "c2054eab", "release": true, "await": true}] 2024-09-11T23:33:39.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.685+0000 7f07d6ac4740 1 -- 172.21.15.26:0/3874544156 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "quiesce db", "set_id": "c2054eab", "release": true, "await": true}) -- 0x5619068c50a0 con 0x5619069e3510 2024-09-11T23:33:39.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.686+0000 7f07d2487640 1 --2- 172.21.15.26:0/3874544156 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5619069e3510 0x5619069e5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:33:39.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.686+0000 7f07d2487640 1 --2- 172.21.15.26:0/3874544156 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5619069e3510 0x5619069e5930 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f9442ba7f0328a4 server_cookie=db5a5eb6a69a7764 in_seq=0 out_seq=0 2024-09-11T23:33:39.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:39.686+0000 7f07b1ffb640 10 client.17562 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:33:39.988 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:33:39 smithi136 ceph-mon[35121]: pgmap v478: 97 pgs: 97 active+clean; 2.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-11T23:33:40.052 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:33:39 smithi076 ceph-mon[35296]: pgmap v478: 97 pgs: 97 active+clean; 2.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-11T23:33:40.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:40.653+0000 7f07b1ffb640 1 -- 172.21.15.26:0/3874544156 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5619068c50a0 con 0x5619069e3510 2024-09-11T23:33:40.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:40.653+0000 7f07b1ffb640 10 client.17562 handle_command_reply: tid=0 2024-09-11T23:33:40.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:40.653+0000 7f07d6ac4740 1 client.17562 shutdown 2024-09-11T23:33:40.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:40.653+0000 7f07d6ac4740 1 -- 172.21.15.26:0/3874544156 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561906958980 msgr2=0x56190695c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:33:40.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:40.653+0000 7f07d6ac4740 1 --2- 172.21.15.26:0/3874544156 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561906958980 0x56190695c370 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f07c00cd680 tx=0x7f07c00094a0 comp rx=0 tx=0).stop 2024-09-11T23:33:40.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:40.653+0000 7f07d6ac4740 1 -- 172.21.15.26:0/3874544156 shutdown_connections 2024-09-11T23:33:40.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:40.653+0000 7f07d6ac4740 1 --2- 172.21.15.26:0/3874544156 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5619069e3510 0x5619069e5930 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:40.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:40.653+0000 7f07d6ac4740 1 --2- 172.21.15.26:0/3874544156 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561906963240 0x56190695c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:40.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:40.653+0000 7f07d6ac4740 1 --2- 172.21.15.26:0/3874544156 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561906958980 0x56190695c370 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:40.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:40.653+0000 7f07d6ac4740 1 --2- 172.21.15.26:0/3874544156 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561906958060 0x56190695dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:40.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:40.653+0000 7f07b1ffb640 0 client.17562 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:33:40.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:40.653+0000 7f07d6ac4740 1 -- 172.21.15.26:0/3874544156 >> 172.21.15.26:0/3874544156 conn(0x5619069545f0 msgr2=0x561906959ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:33:40.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:40.653+0000 7f07d6ac4740 1 -- 172.21.15.26:0/3874544156 shutdown_connections 2024-09-11T23:33:40.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:40.653+0000 7f07d6ac4740 1 -- 172.21.15.26:0/3874544156 wait complete. 2024-09-11T23:33:40.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:40.653+0000 7f07d6ac4740 20 client.17562 trim_cache size 0 max 16384 2024-09-11T23:33:40.655 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-11T23:33:40.656 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 73, 2024-09-11T23:33:40.656 INFO:teuthology.orchestra.run.smithi026.stdout: "leader": 24485, 2024-09-11T23:33:40.656 INFO:teuthology.orchestra.run.smithi026.stdout: "set_version": 56, 2024-09-11T23:33:40.656 INFO:teuthology.orchestra.run.smithi026.stdout: "sets": { 2024-09-11T23:33:40.656 INFO:teuthology.orchestra.run.smithi026.stdout: "c2054eab": { 2024-09-11T23:33:40.656 INFO:teuthology.orchestra.run.smithi026.stdout: "version": 56, 2024-09-11T23:33:40.656 INFO:teuthology.orchestra.run.smithi026.stdout: "age_ref": 0.0, 2024-09-11T23:33:40.656 INFO:teuthology.orchestra.run.smithi026.stdout: "state": { 2024-09-11T23:33:40.656 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "RELEASED", 2024-09-11T23:33:40.656 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.0 2024-09-11T23:33:40.656 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:33:40.656 INFO:teuthology.orchestra.run.smithi026.stdout: "timeout": 90.0, 2024-09-11T23:33:40.657 INFO:teuthology.orchestra.run.smithi026.stdout: "expiration": 150.0, 2024-09-11T23:33:40.657 INFO:teuthology.orchestra.run.smithi026.stdout: "members": { 2024-09-11T23:33:40.657 INFO:teuthology.orchestra.run.smithi026.stdout: "file:/": { 2024-09-11T23:33:40.657 INFO:teuthology.orchestra.run.smithi026.stdout: "excluded": false, 2024-09-11T23:33:40.657 INFO:teuthology.orchestra.run.smithi026.stdout: "state": { 2024-09-11T23:33:40.657 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "RELEASED", 2024-09-11T23:33:40.657 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.0 2024-09-11T23:33:40.657 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:33:40.657 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:33:40.657 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:33:40.657 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:33:40.658 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:33:40.658 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-11T23:33:40.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:40.654+0000 7f07d46eb640 1 -- 172.21.15.26:0/113692708 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f07b40a2cf0 msgr2=0x7f07b40a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:33:40.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:40.654+0000 7f07d46eb640 1 --2- 172.21.15.26:0/113692708 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f07b40a2cf0 0x7f07b40a51b0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f07c4005f60 tx=0x7f07c4005f90 comp rx=0 tx=0).stop 2024-09-11T23:33:40.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:40.654+0000 7f07d46eb640 1 -- 172.21.15.26:0/113692708 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07cc103660 msgr2=0x7f07cc198a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:33:40.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:40.654+0000 7f07d46eb640 1 --2- 172.21.15.26:0/113692708 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07cc103660 0x7f07cc198a10 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f07c000ea00 tx=0x7f07c000eec0 comp rx=0 tx=0).stop 2024-09-11T23:33:40.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:40.654+0000 7f07d46eb640 1 -- 172.21.15.26:0/113692708 shutdown_connections 2024-09-11T23:33:40.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:40.654+0000 7f07d46eb640 1 --2- 172.21.15.26:0/113692708 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f07b40a2cf0 0x7f07b40a51b0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:40.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:40.654+0000 7f07d46eb640 1 --2- 172.21.15.26:0/113692708 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f07cc10c170 0x7f07cc19cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:40.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:40.654+0000 7f07d46eb640 1 --2- 172.21.15.26:0/113692708 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07cc103660 0x7f07cc198a10 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:40.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:40.654+0000 7f07d46eb640 1 --2- 172.21.15.26:0/113692708 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f07cc102d40 0x7f07cc1984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:40.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:40.654+0000 7f07d46eb640 1 -- 172.21.15.26:0/113692708 >> 172.21.15.26:0/113692708 conn(0x7f07cc0fc7a0 msgr2=0x7f07cc0fd4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:33:40.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:40.654+0000 7f07d46eb640 1 -- 172.21.15.26:0/113692708 shutdown_connections 2024-09-11T23:33:40.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:33:40.654+0000 7f07d46eb640 1 -- 172.21.15.26:0/113692708 wait complete. 2024-09-11T23:33:41.379 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'c2054eab', total seconds elapsed: 34.4 2024-09-11T23:33:42.238 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:33:41 smithi136 ceph-mon[35121]: pgmap v479: 97 pgs: 97 active+clean; 2.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 KiB/s wr, 14 op/s 2024-09-11T23:33:42.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:33:41 smithi136 ceph-mon[35121]: osdmap e104: 12 total, 12 up, 12 in 2024-09-11T23:33:42.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:33:41 smithi026 ceph-mon[31172]: pgmap v479: 97 pgs: 97 active+clean; 2.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 KiB/s wr, 14 op/s 2024-09-11T23:33:42.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:33:41 smithi026 ceph-mon[31172]: osdmap e104: 12 total, 12 up, 12 in 2024-09-11T23:33:42.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:33:41 smithi076 ceph-mon[35296]: pgmap v479: 97 pgs: 97 active+clean; 2.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 KiB/s wr, 14 op/s 2024-09-11T23:33:42.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:33:41 smithi076 ceph-mon[35296]: osdmap e104: 12 total, 12 up, 12 in 2024-09-11T23:33:44.238 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:33:43 smithi136 ceph-mon[35121]: pgmap v481: 97 pgs: 97 active+clean; 2.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-09-11T23:33:44.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:33:43 smithi026 ceph-mon[31172]: pgmap v481: 97 pgs: 97 active+clean; 2.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-09-11T23:33:44.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:33:43 smithi076 ceph-mon[35296]: pgmap v481: 97 pgs: 97 active+clean; 2.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-09-11T23:33:46.238 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:33:45 smithi136 ceph-mon[35121]: pgmap v482: 97 pgs: 97 active+clean; 2.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 745 KiB/s rd, 1.0 MiB/s wr, 86 op/s 2024-09-11T23:33:46.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:33:45 smithi026 ceph-mon[31172]: pgmap v482: 97 pgs: 97 active+clean; 2.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 745 KiB/s rd, 1.0 MiB/s wr, 86 op/s 2024-09-11T23:33:46.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:33:45 smithi076 ceph-mon[35296]: pgmap v482: 97 pgs: 97 active+clean; 2.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 745 KiB/s rd, 1.0 MiB/s wr, 86 op/s 2024-09-11T23:33:48.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:33:47 smithi136 ceph-mon[35121]: pgmap v483: 97 pgs: 97 active+clean; 2.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 791 KiB/s rd, 1.3 MiB/s wr, 287 op/s 2024-09-11T23:33:48.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:33:47 smithi026 ceph-mon[31172]: pgmap v483: 97 pgs: 97 active+clean; 2.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 791 KiB/s rd, 1.3 MiB/s wr, 287 op/s 2024-09-11T23:33:48.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:33:47 smithi076 ceph-mon[35296]: pgmap v483: 97 pgs: 97 active+clean; 2.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 791 KiB/s rd, 1.3 MiB/s wr, 287 op/s 2024-09-11T23:33:50.076 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:33:49 smithi026 ceph-mon[31172]: pgmap v484: 97 pgs: 97 active+clean; 2.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 791 KiB/s rd, 1.3 MiB/s wr, 320 op/s 2024-09-11T23:33:50.076 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:33:49 smithi026 ceph-mon[31172]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:33:50.238 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:33:49 smithi136 ceph-mon[35121]: pgmap v484: 97 pgs: 97 active+clean; 2.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 791 KiB/s rd, 1.3 MiB/s wr, 320 op/s 2024-09-11T23:33:50.238 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:33:49 smithi136 ceph-mon[35121]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:33:50.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:33:49 smithi076 ceph-mon[35296]: pgmap v484: 97 pgs: 97 active+clean; 2.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 791 KiB/s rd, 1.3 MiB/s wr, 320 op/s 2024-09-11T23:33:50.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:33:49 smithi076 ceph-mon[35296]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:33:51.005 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.005+0000 7f796eb2c640 1 -- 172.21.15.26:0/2038373899 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7968107ad0 msgr2=0x7f7968107eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:33:51.005 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.005+0000 7f796eb2c640 1 --2- 172.21.15.26:0/2038373899 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7968107ad0 0x7f7968107eb0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f795c009930 tx=0x7f795c02a420 comp rx=0 tx=0).stop 2024-09-11T23:33:51.007 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.007+0000 7f796eb2c640 1 -- 172.21.15.26:0/2038373899 shutdown_connections 2024-09-11T23:33:51.007 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.007+0000 7f796eb2c640 1 --2- 172.21.15.26:0/2038373899 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7968101ac0 0x7f7968101f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:51.007 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.007+0000 7f796eb2c640 1 --2- 172.21.15.26:0/2038373899 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7968102460 0x7f796810e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:51.007 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.007+0000 7f796eb2c640 1 --2- 172.21.15.26:0/2038373899 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7968107ad0 0x7f7968107eb0 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:51.007 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.007+0000 7f796eb2c640 1 -- 172.21.15.26:0/2038373899 >> 172.21.15.26:0/2038373899 conn(0x7f79680fd820 msgr2=0x7f79680ffc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:33:51.008 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.008+0000 7f796eb2c640 1 -- 172.21.15.26:0/2038373899 shutdown_connections 2024-09-11T23:33:51.008 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.008+0000 7f796eb2c640 1 -- 172.21.15.26:0/2038373899 wait complete. 2024-09-11T23:33:51.011 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.009+0000 7f796eb2c640 1 Processor -- start 2024-09-11T23:33:51.011 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.009+0000 7f796eb2c640 1 -- start start 2024-09-11T23:33:51.011 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.010+0000 7f796eb2c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7968101ac0 0x7f7968198180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:51.012 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.010+0000 7f796eb2c640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7968102460 0x7f79681986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:51.012 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.010+0000 7f796eb2c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7968107ad0 0x7f796819ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:51.012 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.010+0000 7f796eb2c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f7968111f10 con 0x7f7968107ad0 2024-09-11T23:33:51.012 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.010+0000 7f796eb2c640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f7968111d90 con 0x7f7968101ac0 2024-09-11T23:33:51.012 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.010+0000 7f796eb2c640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f7968112090 con 0x7f7968102460 2024-09-11T23:33:51.012 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.010+0000 7f796c8c8640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7968101ac0 0x7f7968198180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:33:51.012 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.010+0000 7f796d0c9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7968107ad0 0x7f796819ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:33:51.013 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.010+0000 7f796c8c8640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7968101ac0 0x7f7968198180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:51514/0 (socket says 172.21.15.26:51514) 2024-09-11T23:33:51.013 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.010+0000 7f796c8c8640 1 -- 172.21.15.26:0/422260731 learned_addr learned my addr 172.21.15.26:0/422260731 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:33:51.013 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.010+0000 7f7967fff640 1 --2- 172.21.15.26:0/422260731 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7968102460 0x7f79681986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:33:51.013 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.010+0000 7f796d0c9640 1 -- 172.21.15.26:0/422260731 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7968102460 msgr2=0x7f79681986c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:33:51.013 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.010+0000 7f796d0c9640 1 --2- 172.21.15.26:0/422260731 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7968102460 0x7f79681986c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:51.013 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.010+0000 7f796d0c9640 1 -- 172.21.15.26:0/422260731 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7968101ac0 msgr2=0x7f7968198180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:33:51.013 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.010+0000 7f796d0c9640 1 --2- 172.21.15.26:0/422260731 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7968101ac0 0x7f7968198180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:51.013 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.010+0000 7f796d0c9640 1 -- 172.21.15.26:0/422260731 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f796819d170 con 0x7f7968107ad0 2024-09-11T23:33:51.014 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.011+0000 7f796d0c9640 1 --2- 172.21.15.26:0/422260731 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7968107ad0 0x7f796819ca40 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f7960009fd0 tx=0x7f796000ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:33:51.014 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.011+0000 7f7965ffb640 1 -- 172.21.15.26:0/422260731 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f796000ce20 con 0x7f7968107ad0 2024-09-11T23:33:51.014 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.011+0000 7f796eb2c640 1 -- 172.21.15.26:0/422260731 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f796819d460 con 0x7f7968107ad0 2024-09-11T23:33:51.014 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.011+0000 7f7965ffb640 1 -- 172.21.15.26:0/422260731 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f79600137d0 con 0x7f7968107ad0 2024-09-11T23:33:51.014 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.011+0000 7f796eb2c640 1 -- 172.21.15.26:0/422260731 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7968074420 con 0x7f7968107ad0 2024-09-11T23:33:51.014 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.013+0000 7f7965ffb640 1 -- 172.21.15.26:0/422260731 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79600055e0 con 0x7f7968107ad0 2024-09-11T23:33:51.017 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.015+0000 7f7965ffb640 1 -- 172.21.15.26:0/422260731 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f7960005880 con 0x7f7968107ad0 2024-09-11T23:33:51.017 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.016+0000 7f79718c2740 1 Processor -- start 2024-09-11T23:33:51.019 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.019+0000 7f79718c2740 1 -- start start 2024-09-11T23:33:51.019 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.020+0000 7f79718c2740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f568a4baf0 0x55f568a4bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:51.020 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.020+0000 7f7965ffb640 1 --2- 172.21.15.26:0/422260731 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f79580a2e20 0x7f79580a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:51.020 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.020+0000 7f7965ffb640 1 -- 172.21.15.26:0/422260731 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 8542+0+0 (secure 0 0 0) 0x7f7960004b20 con 0x7f7968107ad0 2024-09-11T23:33:51.020 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.020+0000 7f796c8c8640 1 --2- 172.21.15.26:0/422260731 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f79580a2e20 0x7f79580a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:33:51.020 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.020+0000 7f79718c2740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f568a4c410 0x55f568a56770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:51.020 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.020+0000 7f79718c2740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f568a56cb0 0x55f568a590a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:51.020 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.020+0000 7f79718c2740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55f5689b02f0 con 0x55f568a4baf0 2024-09-11T23:33:51.020 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.020+0000 7f79718c2740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55f568958270 con 0x55f568a4c410 2024-09-11T23:33:51.020 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.020+0000 7f79718c2740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55f5689ac2b0 con 0x55f568a56cb0 2024-09-11T23:33:51.021 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.021+0000 7f7967fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f568a4baf0 0x55f568a4bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:33:51.021 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.021+0000 7f7967fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f568a4baf0 0x55f568a4bed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44146/0 (socket says 172.21.15.26:44146) 2024-09-11T23:33:51.022 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.021+0000 7f7967fff640 1 -- 172.21.15.26:0/3037293497 learned_addr learned my addr 172.21.15.26:0/3037293497 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:33:51.022 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.021+0000 7f7967fff640 1 --2- 172.21.15.26:0/3037293497 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f568a4c410 0x55f568a56770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:33:51.022 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.021+0000 7f796d0c9640 1 --2- 172.21.15.26:0/3037293497 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f568a56cb0 0x55f568a590a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:33:51.022 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.021+0000 7f796c8c8640 1 --2- 172.21.15.26:0/422260731 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f79580a2e20 0x7f79580a52e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f795c004770 tx=0x7f795c00b040 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:33:51.022 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.021+0000 7f7967fff640 1 -- 172.21.15.26:0/3037293497 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f568a56cb0 msgr2=0x55f568a590a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:33:51.022 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.021+0000 7f7967fff640 1 --2- 172.21.15.26:0/3037293497 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f568a56cb0 0x55f568a590a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:51.022 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.021+0000 7f7967fff640 1 -- 172.21.15.26:0/3037293497 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f568a4c410 msgr2=0x55f568a56770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:33:51.022 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.021+0000 7f7967fff640 1 --2- 172.21.15.26:0/3037293497 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f568a4c410 0x55f568a56770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:51.022 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.021+0000 7f7967fff640 1 -- 172.21.15.26:0/3037293497 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f568957340 con 0x55f568a4baf0 2024-09-11T23:33:51.022 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.021+0000 7f7967fff640 1 --2- 172.21.15.26:0/3037293497 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f568a4baf0 0x55f568a4bed0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f7954004650 tx=0x7f79540096a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96f5193d8ec62aaa server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:33:51.022 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.022+0000 7f7952ffd640 1 -- 172.21.15.26:0/3037293497 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79540130d0 con 0x55f568a4baf0 2024-09-11T23:33:51.022 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.022+0000 7f7952ffd640 1 -- 172.21.15.26:0/3037293497 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7954013c90 con 0x55f568a4baf0 2024-09-11T23:33:51.022 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.022+0000 7f7952ffd640 1 -- 172.21.15.26:0/3037293497 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7954011830 con 0x55f568a4baf0 2024-09-11T23:33:51.023 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.024+0000 7f79718c2740 1 -- 172.21.15.26:0/3037293497 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f568a4baf0 msgr2=0x55f568a4bed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:33:51.023 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.024+0000 7f79718c2740 1 --2- 172.21.15.26:0/3037293497 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f568a4baf0 0x55f568a4bed0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f7954004650 tx=0x7f79540096a0 comp rx=0 tx=0).stop 2024-09-11T23:33:51.023 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.024+0000 7f79718c2740 1 -- 172.21.15.26:0/3037293497 shutdown_connections 2024-09-11T23:33:51.023 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.024+0000 7f79718c2740 1 --2- 172.21.15.26:0/3037293497 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f568a4c410 0x55f568a56770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:51.024 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.024+0000 7f79718c2740 1 --2- 172.21.15.26:0/3037293497 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f568a56cb0 0x55f568a590a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:51.024 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.024+0000 7f79718c2740 1 --2- 172.21.15.26:0/3037293497 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f568a4baf0 0x55f568a4bed0 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:51.024 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.024+0000 7f79718c2740 1 -- 172.21.15.26:0/3037293497 >> 172.21.15.26:0/3037293497 conn(0x55f568a48060 msgr2=0x55f568a48440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:33:51.024 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.024+0000 7f79718c2740 1 -- 172.21.15.26:0/3037293497 shutdown_connections 2024-09-11T23:33:51.024 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.024+0000 7f79718c2740 1 -- 172.21.15.26:0/3037293497 wait complete. 2024-09-11T23:33:51.024 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.025+0000 7f79718c2740 1 -- start start 2024-09-11T23:33:51.025 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.025+0000 7f79718c2740 1 Processor -- start 2024-09-11T23:33:51.026 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.026+0000 7f79718c2740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f568a4baf0 0x55f568a53bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:51.027 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.026+0000 7f79718c2740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f568a4c410 0x55f568a54110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:51.027 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.026+0000 7f79718c2740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f568a56cb0 0x55f568a54650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:51.027 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.026+0000 7f79718c2740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55f568a59880 con 0x55f568a4baf0 2024-09-11T23:33:51.027 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.026+0000 7f79718c2740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55f568939290 con 0x55f568a56cb0 2024-09-11T23:33:51.027 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.026+0000 7f796c8c8640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f568a4baf0 0x55f568a53bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:33:51.028 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.026+0000 7f79718c2740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55f568a59a00 con 0x55f568a4c410 2024-09-11T23:33:51.028 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.026+0000 7f796c8c8640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f568a4baf0 0x55f568a53bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44162/0 (socket says 172.21.15.26:44162) 2024-09-11T23:33:51.028 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.026+0000 7f796c8c8640 1 -- 172.21.15.26:0/2340554558 learned_addr learned my addr 172.21.15.26:0/2340554558 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:33:51.028 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.026+0000 7f79718c2740 10 client.0 _ll_register_callbacks cb 0x55f56896a140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:33:51.028 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.026+0000 7f7967fff640 1 --2- 172.21.15.26:0/2340554558 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f568a4c410 0x55f568a54110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:33:51.028 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.026+0000 7f796d0c9640 1 --2- 172.21.15.26:0/2340554558 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f568a56cb0 0x55f568a54650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:33:51.029 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.026+0000 7f796c8c8640 1 -- 172.21.15.26:0/2340554558 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f568a4c410 msgr2=0x55f568a54110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:33:51.029 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.026+0000 7f796c8c8640 1 --2- 172.21.15.26:0/2340554558 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f568a4c410 0x55f568a54110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:51.029 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.026+0000 7f796c8c8640 1 -- 172.21.15.26:0/2340554558 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f568a56cb0 msgr2=0x55f568a54650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:33:51.029 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.026+0000 7f796c8c8640 1 --2- 172.21.15.26:0/2340554558 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f568a56cb0 0x55f568a54650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:51.029 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.026+0000 7f796c8c8640 1 -- 172.21.15.26:0/2340554558 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f5686c3210 con 0x55f568a4baf0 2024-09-11T23:33:51.029 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.027+0000 7f796c8c8640 1 --2- 172.21.15.26:0/2340554558 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f568a4baf0 0x55f568a53bd0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f795c04cd40 tx=0x7f795c005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35109b36563a7afc server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:33:51.030 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.027+0000 7f7947fff640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:33:51.030 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.027+0000 7f7947fff640 1 -- 172.21.15.26:0/2340554558 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f795c052060 con 0x55f568a4baf0 2024-09-11T23:33:51.030 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.027+0000 7f7947fff640 1 -- 172.21.15.26:0/2340554558 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8542+0+0 (secure 0 0 0) 0x7f795c04c370 con 0x55f568a4baf0 2024-09-11T23:33:51.030 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.027+0000 7f79718c2740 1 -- 172.21.15.26:0/2340554558 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7954013c90 con 0x55f568a4baf0 2024-09-11T23:33:51.030 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.027+0000 7f7947fff640 1 -- 172.21.15.26:0/2340554558 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) ==== 24+0+0 (secure 0 0 0) 0x7f795c035040 con 0x55f568a4baf0 2024-09-11T23:33:51.031 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.027+0000 7f79718c2740 10 client.17574 fetch_fsmap learned FSMap version 74 2024-09-11T23:33:51.031 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.027+0000 7f79718c2740 1 -- 172.21.15.26:0/2340554558 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=4+}) -- 0x55f56878f620 con 0x55f568a4baf0 2024-09-11T23:33:51.034 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.028+0000 7f7947fff640 1 -- 172.21.15.26:0/2340554558 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7954013c90 con 0x55f568a4baf0 2024-09-11T23:33:51.034 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.028+0000 7f7947fff640 1 -- 172.21.15.26:0/2340554558 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 74) ==== 8438+0+0 (secure 0 0 0) 0x7f795c049c50 con 0x55f568a4baf0 2024-09-11T23:33:51.035 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.028+0000 7f79718c2740 10 client.17574 fetch_fsmap finished waiting for FSMap version 74 2024-09-11T23:33:51.035 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.028+0000 7f79718c2740 10 client.17574 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:33:51.035 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.028+0000 7f79718c2740 20 client.17574 populate_metadata read hostname 'smithi026' 2024-09-11T23:33:51.035 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.028+0000 7f79718c2740 1 --2- 172.21.15.26:0/2340554558 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55f568a66ca0 0x55f568a69120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:51.035 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.028+0000 7f79718c2740 4 client.17574 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:33:51.035 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.028+0000 7f79718c2740 1 -- 172.21.15.26:0/2340554558 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f5689bda20 con 0x55f568a66ca0 2024-09-11T23:33:51.035 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.030+0000 7f7967fff640 1 --2- 172.21.15.26:0/2340554558 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55f568a66ca0 0x55f568a69120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:33:51.035 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.031+0000 7f7967fff640 1 --2- 172.21.15.26:0/2340554558 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55f568a66ca0 0x55f568a69120 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b90c7cf4e19b0e1 server_cookie=56327d2cc0a8186 in_seq=0 out_seq=0 2024-09-11T23:33:51.036 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.031+0000 7f7947fff640 10 client.17574 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:33:51.036 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.032+0000 7f7947fff640 1 -- 172.21.15.26:0/2340554558 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f5689bda20 con 0x55f568a66ca0 2024-09-11T23:33:51.036 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.032+0000 7f7947fff640 10 client.17574 handle_command_reply: tid=0 2024-09-11T23:33:51.036 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.032+0000 7f79718c2740 1 client.17574 shutdown 2024-09-11T23:33:51.036 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.032+0000 7f79718c2740 1 -- 172.21.15.26:0/2340554558 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f568a4baf0 msgr2=0x55f568a53bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:33:51.036 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.032+0000 7f79718c2740 1 --2- 172.21.15.26:0/2340554558 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f568a4baf0 0x55f568a53bd0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f795c04cd40 tx=0x7f795c005fd0 comp rx=0 tx=0).stop 2024-09-11T23:33:51.036 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.033+0000 7f79718c2740 1 -- 172.21.15.26:0/2340554558 shutdown_connections 2024-09-11T23:33:51.037 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.033+0000 7f79718c2740 1 --2- 172.21.15.26:0/2340554558 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f568a56cb0 0x55f568a54650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:51.037 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.033+0000 7f79718c2740 1 --2- 172.21.15.26:0/2340554558 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55f568a66ca0 0x55f568a69120 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:51.037 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.033+0000 7f79718c2740 1 --2- 172.21.15.26:0/2340554558 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f568a4c410 0x55f568a54110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:51.037 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.033+0000 7f79718c2740 1 --2- 172.21.15.26:0/2340554558 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f568a4baf0 0x55f568a53bd0 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:51.037 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.033+0000 7f79718c2740 1 -- 172.21.15.26:0/2340554558 >> 172.21.15.26:0/2340554558 conn(0x55f568a48060 msgr2=0x55f568a4ddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:33:51.038 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.033+0000 7f7947fff640 0 client.17574 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:33:51.038 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.033+0000 7f79718c2740 1 -- 172.21.15.26:0/2340554558 shutdown_connections 2024-09-11T23:33:51.038 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.033+0000 7f79718c2740 1 -- 172.21.15.26:0/2340554558 wait complete. 2024-09-11T23:33:51.038 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.033+0000 7f79718c2740 20 client.17574 trim_cache size 0 max 16384 2024-09-11T23:33:51.059 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.056+0000 7f79718c2740 1 Processor -- start 2024-09-11T23:33:51.059 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.056+0000 7f79718c2740 1 -- start start 2024-09-11T23:33:51.059 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.056+0000 7f79718c2740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f568a4baf0 0x55f568a4e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:51.059 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.056+0000 7f79718c2740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f568a4c410 0x55f568a6ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:51.059 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.057+0000 7f79718c2740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f568a56cb0 0x55f568a6ef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:51.059 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.057+0000 7f7967fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f568a4c410 0x55f568a6ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:33:51.059 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.057+0000 7f7967fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f568a4c410 0x55f568a6ea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44166/0 (socket says 172.21.15.26:44166) 2024-09-11T23:33:51.060 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.057+0000 7f7967fff640 1 -- 172.21.15.26:0/3048094216 learned_addr learned my addr 172.21.15.26:0/3048094216 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:33:51.060 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.057+0000 7f79718c2740 1 -- 172.21.15.26:0/3048094216 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f794c00e8d0 con 0x55f568a4c410 2024-09-11T23:33:51.060 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.057+0000 7f79718c2740 1 -- 172.21.15.26:0/3048094216 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f794c0106e0 con 0x55f568a56cb0 2024-09-11T23:33:51.060 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.057+0000 7f79718c2740 1 -- 172.21.15.26:0/3048094216 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f794c00f060 con 0x55f568a4baf0 2024-09-11T23:33:51.060 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.057+0000 7f796c8c8640 1 --2- 172.21.15.26:0/3048094216 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f568a4baf0 0x55f568a4e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:33:51.060 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.057+0000 7f796d0c9640 1 --2- 172.21.15.26:0/3048094216 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f568a56cb0 0x55f568a6ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:33:51.060 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.057+0000 7f7967fff640 1 -- 172.21.15.26:0/3048094216 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f568a4baf0 msgr2=0x55f568a4e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:33:51.060 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.057+0000 7f7967fff640 1 --2- 172.21.15.26:0/3048094216 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f568a4baf0 0x55f568a4e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:51.060 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.057+0000 7f7967fff640 1 -- 172.21.15.26:0/3048094216 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f568a56cb0 msgr2=0x55f568a6ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:33:51.060 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.057+0000 7f7967fff640 1 --2- 172.21.15.26:0/3048094216 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f568a56cb0 0x55f568a6ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:51.060 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.057+0000 7f7967fff640 1 -- 172.21.15.26:0/3048094216 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f568a47e90 con 0x55f568a4c410 2024-09-11T23:33:51.061 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.058+0000 7f7967fff640 1 --2- 172.21.15.26:0/3048094216 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f568a4c410 0x55f568a6ea30 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f795400ff60 tx=0x7f795400ff90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a19ef1460d6b4335 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:33:51.061 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.058+0000 7f79527fc640 1 -- 172.21.15.26:0/3048094216 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f795402b070 con 0x55f568a4c410 2024-09-11T23:33:51.061 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.058+0000 7f79527fc640 1 -- 172.21.15.26:0/3048094216 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7954014e50 con 0x55f568a4c410 2024-09-11T23:33:51.061 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.058+0000 7f79527fc640 1 -- 172.21.15.26:0/3048094216 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f795400f8b0 con 0x55f568a4c410 2024-09-11T23:33:51.061 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.059+0000 7f79718c2740 1 -- 172.21.15.26:0/3048094216 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f568a4c410 msgr2=0x55f568a6ea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:33:51.061 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.059+0000 7f79718c2740 1 --2- 172.21.15.26:0/3048094216 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f568a4c410 0x55f568a6ea30 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f795400ff60 tx=0x7f795400ff90 comp rx=0 tx=0).stop 2024-09-11T23:33:51.061 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.059+0000 7f79718c2740 1 -- 172.21.15.26:0/3048094216 shutdown_connections 2024-09-11T23:33:51.061 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.059+0000 7f79718c2740 1 --2- 172.21.15.26:0/3048094216 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f568a56cb0 0x55f568a6ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:51.061 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.059+0000 7f79718c2740 1 --2- 172.21.15.26:0/3048094216 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f568a4c410 0x55f568a6ea30 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:51.061 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.059+0000 7f79718c2740 1 --2- 172.21.15.26:0/3048094216 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f568a4baf0 0x55f568a4e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:51.062 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.059+0000 7f79718c2740 1 -- 172.21.15.26:0/3048094216 >> 172.21.15.26:0/3048094216 conn(0x55f568a48060 msgr2=0x55f568a55c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:33:51.062 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.059+0000 7f79718c2740 1 -- 172.21.15.26:0/3048094216 shutdown_connections 2024-09-11T23:33:51.062 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.059+0000 7f79718c2740 1 -- 172.21.15.26:0/3048094216 wait complete. 2024-09-11T23:33:51.062 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.059+0000 7f79718c2740 1 -- start start 2024-09-11T23:33:51.062 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.059+0000 7f79718c2740 1 Processor -- start 2024-09-11T23:33:51.064 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.060+0000 7f79718c2740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f568a4baf0 0x55f568a5c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:51.065 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.060+0000 7f79718c2740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f568a4c410 0x55f568a5c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:51.065 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.060+0000 7f79718c2740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f568a56cb0 0x55f568acfa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:51.065 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.060+0000 7f79718c2740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f794c00ff50 con 0x55f568a4c410 2024-09-11T23:33:51.065 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.060+0000 7f79718c2740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f794c00f7f0 con 0x55f568a4baf0 2024-09-11T23:33:51.065 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.060+0000 7f79718c2740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f794c00c8d0 con 0x55f568a56cb0 2024-09-11T23:33:51.065 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.060+0000 7f796c8c8640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f568a4baf0 0x55f568a5c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:33:51.066 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.060+0000 7f796c8c8640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f568a4baf0 0x55f568a5c1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:51548/0 (socket says 172.21.15.26:51548) 2024-09-11T23:33:51.066 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.060+0000 7f796c8c8640 1 -- 172.21.15.26:0/3924348928 learned_addr learned my addr 172.21.15.26:0/3924348928 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:33:51.066 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.060+0000 7f7967fff640 1 --2- 172.21.15.26:0/3924348928 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f568a4c410 0x55f568a5c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:33:51.066 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.060+0000 7f79718c2740 10 client.0 _ll_register_callbacks cb 0x7f795c040070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:33:51.066 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.060+0000 7f796d0c9640 1 --2- 172.21.15.26:0/3924348928 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f568a56cb0 0x55f568acfa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:33:51.066 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.060+0000 7f796c8c8640 1 -- 172.21.15.26:0/3924348928 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f568a56cb0 msgr2=0x55f568acfa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:33:51.066 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.060+0000 7f796c8c8640 1 --2- 172.21.15.26:0/3924348928 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f568a56cb0 0x55f568acfa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:51.067 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.060+0000 7f796c8c8640 1 -- 172.21.15.26:0/3924348928 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f568a4c410 msgr2=0x55f568a5c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:33:51.067 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.060+0000 7f796c8c8640 1 --2- 172.21.15.26:0/3924348928 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f568a4c410 0x55f568a5c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:51.067 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.060+0000 7f796c8c8640 1 -- 172.21.15.26:0/3924348928 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f568a432d0 con 0x55f568a4baf0 2024-09-11T23:33:51.067 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.061+0000 7f796c8c8640 1 --2- 172.21.15.26:0/3924348928 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f568a4baf0 0x55f568a5c1a0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f795c04c6f0 tx=0x7f795c04c720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a8a3d677961ddf37 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:33:51.067 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.061+0000 7f7947fff640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-11T23:33:51.067 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.061+0000 7f7947fff640 1 -- 172.21.15.26:0/3924348928 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f795c063060 con 0x55f568a4baf0 2024-09-11T23:33:51.068 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.061+0000 7f7947fff640 1 -- 172.21.15.26:0/3924348928 <== mon.2 v2:172.21.15.76:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8542+0+0 (secure 0 0 0) 0x7f795c03cae0 con 0x55f568a4baf0 2024-09-11T23:33:51.068 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.061+0000 7f79718c2740 1 -- 172.21.15.26:0/3924348928 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7954014e50 con 0x55f568a4baf0 2024-09-11T23:33:51.068 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.061+0000 7f7947fff640 1 -- 172.21.15.26:0/3924348928 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) ==== 24+0+0 (secure 0 0 0) 0x7f795c052070 con 0x55f568a4baf0 2024-09-11T23:33:51.068 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.061+0000 7f79718c2740 10 client.27098 fetch_fsmap learned FSMap version 74 2024-09-11T23:33:51.068 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.061+0000 7f79718c2740 1 -- 172.21.15.26:0/3924348928 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=4+}) -- 0x55f568ad1070 con 0x55f568a4baf0 2024-09-11T23:33:51.068 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.061+0000 7f7947fff640 1 -- 172.21.15.26:0/3924348928 <== mon.2 v2:172.21.15.76:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7954014e50 con 0x55f568a4baf0 2024-09-11T23:33:51.068 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.062+0000 7f7947fff640 1 -- 172.21.15.26:0/3924348928 <== mon.2 v2:172.21.15.76:3300/0 5 ==== fsmap(e 74) ==== 8438+0+0 (secure 0 0 0) 0x7f795c049bf0 con 0x55f568a4baf0 2024-09-11T23:33:51.069 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.062+0000 7f79718c2740 10 client.27098 fetch_fsmap finished waiting for FSMap version 74 2024-09-11T23:33:51.069 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.062+0000 7f79718c2740 10 client.27098 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:33:51.069 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.062+0000 7f79718c2740 20 client.27098 populate_metadata read hostname 'smithi026' 2024-09-11T23:33:51.069 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.062+0000 7f79718c2740 1 --2- 172.21.15.26:0/3924348928 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55f568ad1850 0x55f568ad3cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:51.069 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.062+0000 7f79718c2740 4 client.27098 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T23:33:51.069 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.062+0000 7f79718c2740 1 -- 172.21.15.26:0/3924348928 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f5689b7d70 con 0x55f568ad1850 2024-09-11T23:33:51.070 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.062+0000 7f7967fff640 1 --2- 172.21.15.26:0/3924348928 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55f568ad1850 0x55f568ad3cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:33:51.070 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.062+0000 7f7947fff640 10 client.27098 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:33:51.070 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.063+0000 7f7967fff640 1 --2- 172.21.15.26:0/3924348928 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55f568ad1850 0x55f568ad3cb0 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80c4f24680ea6b82 server_cookie=bb59ec1d2bd77be in_seq=0 out_seq=0 2024-09-11T23:33:51.404 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.404+0000 7f7947fff640 1 -- 172.21.15.26:0/3924348928 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f5689b7d70 con 0x55f568ad1850 2024-09-11T23:33:51.405 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.404+0000 7f7947fff640 10 client.27098 handle_command_reply: tid=0 2024-09-11T23:33:51.405 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.404+0000 7f79718c2740 1 client.27098 shutdown 2024-09-11T23:33:51.405 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.404+0000 7f79718c2740 1 -- 172.21.15.26:0/3924348928 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f568a4baf0 msgr2=0x55f568a5c1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:33:51.405 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.404+0000 7f79718c2740 1 --2- 172.21.15.26:0/3924348928 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f568a4baf0 0x55f568a5c1a0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f795c04c6f0 tx=0x7f795c04c720 comp rx=0 tx=0).stop 2024-09-11T23:33:51.405 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.404+0000 7f79718c2740 1 -- 172.21.15.26:0/3924348928 shutdown_connections 2024-09-11T23:33:51.405 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.404+0000 7f79718c2740 1 --2- 172.21.15.26:0/3924348928 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55f568ad1850 0x55f568ad3cb0 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:51.405 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.404+0000 7f79718c2740 1 --2- 172.21.15.26:0/3924348928 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f568a56cb0 0x55f568acfa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:51.405 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.404+0000 7f7947fff640 0 client.27098 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:33:51.405 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.404+0000 7f79718c2740 1 --2- 172.21.15.26:0/3924348928 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f568a4c410 0x55f568a5c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:51.406 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.404+0000 7f79718c2740 1 --2- 172.21.15.26:0/3924348928 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f568a4baf0 0x55f568a5c1a0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:51.406 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.404+0000 7f79718c2740 1 -- 172.21.15.26:0/3924348928 >> 172.21.15.26:0/3924348928 conn(0x55f568a48060 msgr2=0x55f568a469a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:33:51.406 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.404+0000 7f79718c2740 1 -- 172.21.15.26:0/3924348928 shutdown_connections 2024-09-11T23:33:51.406 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.404+0000 7f79718c2740 1 -- 172.21.15.26:0/3924348928 wait complete. 2024-09-11T23:33:51.406 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.404+0000 7f79718c2740 20 client.27098 trim_cache size 0 max 16384 2024-09-11T23:33:51.406 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-09-11T23:33:51.406 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "message": "", 2024-09-11T23:33:51.406 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-09-11T23:33:51.406 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-09-11T23:33:51.407 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.405+0000 7f796eb2c640 1 -- 172.21.15.26:0/422260731 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f79580a2e20 msgr2=0x7f79580a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:33:51.407 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.405+0000 7f796eb2c640 1 --2- 172.21.15.26:0/422260731 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f79580a2e20 0x7f79580a52e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f795c004770 tx=0x7f795c00b040 comp rx=0 tx=0).stop 2024-09-11T23:33:51.407 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.405+0000 7f796eb2c640 1 -- 172.21.15.26:0/422260731 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7968107ad0 msgr2=0x7f796819ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:33:51.407 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.405+0000 7f796eb2c640 1 --2- 172.21.15.26:0/422260731 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7968107ad0 0x7f796819ca40 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f7960009fd0 tx=0x7f796000ef10 comp rx=0 tx=0).stop 2024-09-11T23:33:51.407 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.405+0000 7f796eb2c640 1 -- 172.21.15.26:0/422260731 shutdown_connections 2024-09-11T23:33:51.408 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.405+0000 7f796eb2c640 1 --2- 172.21.15.26:0/422260731 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f79580a2e20 0x7f79580a52e0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:51.408 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.405+0000 7f796eb2c640 1 --2- 172.21.15.26:0/422260731 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7968107ad0 0x7f796819ca40 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:51.408 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.405+0000 7f796eb2c640 1 --2- 172.21.15.26:0/422260731 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7968102460 0x7f79681986c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:51.408 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.405+0000 7f796eb2c640 1 --2- 172.21.15.26:0/422260731 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7968101ac0 0x7f7968198180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:51.408 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.405+0000 7f796eb2c640 1 -- 172.21.15.26:0/422260731 >> 172.21.15.26:0/422260731 conn(0x7f79680fd820 msgr2=0x7f79680fdc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:33:51.408 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.405+0000 7f796eb2c640 1 -- 172.21.15.26:0/422260731 shutdown_connections 2024-09-11T23:33:51.408 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:33:51.405+0000 7f796eb2c640 1 -- 172.21.15.26:0/422260731 wait complete. 2024-09-11T23:33:52.238 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:33:51 smithi136 ceph-mon[35121]: pgmap v485: 97 pgs: 97 active+clean; 2.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.9 MiB/s wr, 664 op/s 2024-09-11T23:33:52.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:33:51 smithi026 ceph-mon[31172]: pgmap v485: 97 pgs: 97 active+clean; 2.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.9 MiB/s wr, 664 op/s 2024-09-11T23:33:52.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:33:51 smithi076 ceph-mon[35296]: pgmap v485: 97 pgs: 97 active+clean; 2.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.9 MiB/s wr, 664 op/s 2024-09-11T23:33:54.238 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:33:53 smithi136 ceph-mon[35121]: pgmap v486: 97 pgs: 97 active+clean; 2.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.5 MiB/s wr, 584 op/s 2024-09-11T23:33:54.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:33:53 smithi026 ceph-mon[31172]: pgmap v486: 97 pgs: 97 active+clean; 2.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.5 MiB/s wr, 584 op/s 2024-09-11T23:33:54.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:33:53 smithi076 ceph-mon[35296]: pgmap v486: 97 pgs: 97 active+clean; 2.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.5 MiB/s wr, 584 op/s 2024-09-11T23:33:56.104 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:33:55 smithi076 ceph-mon[35296]: pgmap v487: 97 pgs: 97 active+clean; 2.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.0 MiB/s wr, 658 op/s 2024-09-11T23:33:56.238 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:33:55 smithi136 ceph-mon[35121]: pgmap v487: 97 pgs: 97 active+clean; 2.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.0 MiB/s wr, 658 op/s 2024-09-11T23:33:56.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:33:55 smithi026 ceph-mon[31172]: pgmap v487: 97 pgs: 97 active+clean; 2.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.0 MiB/s wr, 658 op/s 2024-09-11T23:33:58.238 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:33:57 smithi136 ceph-mon[35121]: pgmap v488: 97 pgs: 97 active+clean; 2.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.2 MiB/s wr, 685 op/s 2024-09-11T23:33:58.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:33:57 smithi026 ceph-mon[31172]: pgmap v488: 97 pgs: 97 active+clean; 2.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.2 MiB/s wr, 685 op/s 2024-09-11T23:33:58.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:33:57 smithi076 ceph-mon[35296]: pgmap v488: 97 pgs: 97 active+clean; 2.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.2 MiB/s wr, 685 op/s 2024-09-11T23:33:59.238 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:33:58 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:33:59.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:33:58 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:33:59.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:33:58 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:34:00.238 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:33:59 smithi136 ceph-mon[35121]: pgmap v489: 97 pgs: 97 active+clean; 2.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.0 MiB/s wr, 524 op/s 2024-09-11T23:34:00.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:33:59 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:34:00.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:33:59 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:34:00.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:33:59 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:34:00.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:33:59 smithi026 ceph-mon[31172]: pgmap v489: 97 pgs: 97 active+clean; 2.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.0 MiB/s wr, 524 op/s 2024-09-11T23:34:00.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:33:59 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:34:00.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:33:59 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:34:00.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:33:59 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:34:00.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:33:59 smithi076 ceph-mon[35296]: pgmap v489: 97 pgs: 97 active+clean; 2.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.0 MiB/s wr, 524 op/s 2024-09-11T23:34:00.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:33:59 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:34:00.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:33:59 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:34:00.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:33:59 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:34:02.238 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:34:01 smithi136 ceph-mon[35121]: pgmap v490: 97 pgs: 97 active+clean; 2.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.0 MiB/s wr, 617 op/s 2024-09-11T23:34:02.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:34:01 smithi026 ceph-mon[31172]: pgmap v490: 97 pgs: 97 active+clean; 2.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.0 MiB/s wr, 617 op/s 2024-09-11T23:34:02.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:34:01 smithi076 ceph-mon[35296]: pgmap v490: 97 pgs: 97 active+clean; 2.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.0 MiB/s wr, 617 op/s 2024-09-11T23:34:04.238 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:34:03 smithi136 ceph-mon[35121]: pgmap v491: 97 pgs: 97 active+clean; 2.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.6 MiB/s wr, 340 op/s 2024-09-11T23:34:04.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:34:03 smithi026 ceph-mon[31172]: pgmap v491: 97 pgs: 97 active+clean; 2.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.6 MiB/s wr, 340 op/s 2024-09-11T23:34:04.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:34:03 smithi076 ceph-mon[35296]: pgmap v491: 97 pgs: 97 active+clean; 2.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.6 MiB/s wr, 340 op/s 2024-09-11T23:34:04.561 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.560+0000 7f53cb5d8640 1 -- 172.21.15.26:0/2320613242 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f53c4100a90 msgr2=0x7f53c4100e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:34:04.561 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.560+0000 7f53cb5d8640 1 --2- 172.21.15.26:0/2320613242 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f53c4100a90 0x7f53c4100e70 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f53b4009930 tx=0x7f53b402a480 comp rx=0 tx=0).stop 2024-09-11T23:34:04.564 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.564+0000 7f53cb5d8640 1 -- 172.21.15.26:0/2320613242 shutdown_connections 2024-09-11T23:34:04.564 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.564+0000 7f53cb5d8640 1 --2- 172.21.15.26:0/2320613242 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f53c41013b0 0x7f53c4105870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:04.565 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.564+0000 7f53cb5d8640 1 --2- 172.21.15.26:0/2320613242 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f53c4105f10 0x7f53c41063c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:04.565 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.564+0000 7f53cb5d8640 1 --2- 172.21.15.26:0/2320613242 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f53c4100a90 0x7f53c4100e70 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:04.565 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.564+0000 7f53cb5d8640 1 -- 172.21.15.26:0/2320613242 >> 172.21.15.26:0/2320613242 conn(0x7f53c40fc6b0 msgr2=0x7f53c40fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:34:04.567 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.568+0000 7f53cb5d8640 1 -- 172.21.15.26:0/2320613242 shutdown_connections 2024-09-11T23:34:04.570 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.569+0000 7f53cb5d8640 1 -- 172.21.15.26:0/2320613242 wait complete. 2024-09-11T23:34:04.571 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.570+0000 7f53cb5d8640 1 Processor -- start 2024-09-11T23:34:04.571 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.570+0000 7f53cb5d8640 1 -- start start 2024-09-11T23:34:04.571 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.570+0000 7f53cb5d8640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f53c4100a90 0x7f53c4114ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:04.571 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.570+0000 7f53cb5d8640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f53c41013b0 0x7f53c410dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:04.571 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.570+0000 7f53cb5d8640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f53c4105f10 0x7f53c410e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:04.571 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.570+0000 7f53cb5d8640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f53c4118610 con 0x7f53c4105f10 2024-09-11T23:34:04.571 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.571+0000 7f53cb5d8640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f53c4118490 con 0x7f53c4100a90 2024-09-11T23:34:04.571 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.571+0000 7f53cb5d8640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f53c4118790 con 0x7f53c41013b0 2024-09-11T23:34:04.571 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.571+0000 7f53c9b75640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f53c4105f10 0x7f53c410e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:34:04.571 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.571+0000 7f53c9b75640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f53c4105f10 0x7f53c410e260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38028/0 (socket says 172.21.15.26:38028) 2024-09-11T23:34:04.571 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.571+0000 7f53c9b75640 1 -- 172.21.15.26:0/2313679085 learned_addr learned my addr 172.21.15.26:0/2313679085 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:34:04.571 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.571+0000 7f53c9374640 1 --2- 172.21.15.26:0/2313679085 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f53c4100a90 0x7f53c4114ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:34:04.572 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.572+0000 7f53c9b75640 1 -- 172.21.15.26:0/2313679085 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f53c41013b0 msgr2=0x7f53c410dd20 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-11T23:34:04.572 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.572+0000 7f53c9b75640 1 --2- 172.21.15.26:0/2313679085 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f53c41013b0 0x7f53c410dd20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:04.572 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.572+0000 7f53c9b75640 1 -- 172.21.15.26:0/2313679085 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f53c4100a90 msgr2=0x7f53c4114ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:34:04.572 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.572+0000 7f53c9b75640 1 --2- 172.21.15.26:0/2313679085 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f53c4100a90 0x7f53c4114ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:04.573 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.572+0000 7f53c9b75640 1 -- 172.21.15.26:0/2313679085 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53c410e8a0 con 0x7f53c4105f10 2024-09-11T23:34:04.573 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.572+0000 7f53c9b75640 1 --2- 172.21.15.26:0/2313679085 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f53c4105f10 0x7f53c410e260 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f53ac00b970 tx=0x7f53ac00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:34:04.573 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.573+0000 7f53c27fc640 1 -- 172.21.15.26:0/2313679085 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53ac019070 con 0x7f53c4105f10 2024-09-11T23:34:04.573 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.573+0000 7f53c27fc640 1 -- 172.21.15.26:0/2313679085 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f53ac00cb40 con 0x7f53c4105f10 2024-09-11T23:34:04.573 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.573+0000 7f53c27fc640 1 -- 172.21.15.26:0/2313679085 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53ac0076a0 con 0x7f53c4105f10 2024-09-11T23:34:04.573 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.573+0000 7f53cb5d8640 1 -- 172.21.15.26:0/2313679085 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53c41127c0 con 0x7f53c4105f10 2024-09-11T23:34:04.573 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.573+0000 7f53cb5d8640 1 -- 172.21.15.26:0/2313679085 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53c4112d10 con 0x7f53c4105f10 2024-09-11T23:34:04.579 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.579+0000 7f53ce448740 1 Processor -- start 2024-09-11T23:34:04.580 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.580+0000 7f53c27fc640 1 -- 172.21.15.26:0/2313679085 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f53ac00ccf0 con 0x7f53c4105f10 2024-09-11T23:34:04.580 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.580+0000 7f53ce448740 1 -- start start 2024-09-11T23:34:04.581 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.580+0000 7f53ce448740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5638dd50aaf0 0x5638dd50aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:04.582 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.580+0000 7f53ce448740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5638dd50b410 0x5638dd515770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:04.582 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.581+0000 7f53ce448740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5638dd515cb0 0x5638dd5180a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:04.582 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.581+0000 7f53ce448740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5638dd46f2f0 con 0x5638dd50b410 2024-09-11T23:34:04.582 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.581+0000 7f53ce448740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5638dd417270 con 0x5638dd515cb0 2024-09-11T23:34:04.582 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.581+0000 7f53ce448740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5638dd46b2b0 con 0x5638dd50aaf0 2024-09-11T23:34:04.582 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.581+0000 7f53c27fc640 1 --2- 172.21.15.26:0/2313679085 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f53b00a2fd0 0x7f53b00a5490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:04.582 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.582+0000 7f53c27fc640 1 -- 172.21.15.26:0/2313679085 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 8586+0+0 (secure 0 0 0) 0x7f53ac0ce800 con 0x7f53c4105f10 2024-09-11T23:34:04.583 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.582+0000 7f53c8b73640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5638dd50aaf0 0x5638dd50aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:34:04.583 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.582+0000 7f53c9b75640 1 --2- 172.21.15.26:0/2313679085 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f53b00a2fd0 0x7f53b00a5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:34:04.583 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.582+0000 7f53c8b73640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5638dd50aaf0 0x5638dd50aed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:44368/0 (socket says 172.21.15.26:44368) 2024-09-11T23:34:04.583 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.582+0000 7f53c8b73640 1 -- 172.21.15.26:0/1157991161 learned_addr learned my addr 172.21.15.26:0/1157991161 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:34:04.583 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.582+0000 7f53c8b73640 1 --2- 172.21.15.26:0/1157991161 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5638dd515cb0 0x5638dd5180a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:34:04.583 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.583+0000 7f53c9b75640 1 --2- 172.21.15.26:0/2313679085 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f53b00a2fd0 0x7f53b00a5490 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f53ac0cc020 tx=0x7f53ac00e040 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:34:04.584 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.583+0000 7f53c9374640 1 --2- 172.21.15.26:0/1157991161 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5638dd50b410 0x5638dd515770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:34:04.584 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.584+0000 7f53c9374640 1 -- 172.21.15.26:0/1157991161 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5638dd50aaf0 msgr2=0x5638dd50aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:34:04.584 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.584+0000 7f53c9374640 1 --2- 172.21.15.26:0/1157991161 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5638dd50aaf0 0x5638dd50aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:04.584 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.584+0000 7f53c9374640 1 -- 172.21.15.26:0/1157991161 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5638dd515cb0 msgr2=0x5638dd5180a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:34:04.584 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.584+0000 7f53c9374640 1 --2- 172.21.15.26:0/1157991161 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5638dd515cb0 0x5638dd5180a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:04.584 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.584+0000 7f53c9374640 1 -- 172.21.15.26:0/1157991161 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5638dd416340 con 0x5638dd50b410 2024-09-11T23:34:04.585 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.585+0000 7f53c9374640 1 --2- 172.21.15.26:0/1157991161 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5638dd50b410 0x5638dd515770 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f53b4009850 tx=0x7f53b4004150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff1b892ad52a7f3e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:34:04.586 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.587+0000 7f53bb7fe640 1 -- 172.21.15.26:0/1157991161 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53b4040070 con 0x5638dd50b410 2024-09-11T23:34:04.586 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.587+0000 7f53bb7fe640 1 -- 172.21.15.26:0/1157991161 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f53b4005500 con 0x5638dd50b410 2024-09-11T23:34:04.586 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.587+0000 7f53bb7fe640 1 -- 172.21.15.26:0/1157991161 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53b4035040 con 0x5638dd50b410 2024-09-11T23:34:04.591 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.589+0000 7f53ce448740 1 -- 172.21.15.26:0/1157991161 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5638dd50b410 msgr2=0x5638dd515770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:34:04.591 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.589+0000 7f53ce448740 1 --2- 172.21.15.26:0/1157991161 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5638dd50b410 0x5638dd515770 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f53b4009850 tx=0x7f53b4004150 comp rx=0 tx=0).stop 2024-09-11T23:34:04.597 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.597+0000 7f53ce448740 1 -- 172.21.15.26:0/1157991161 shutdown_connections 2024-09-11T23:34:04.597 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.597+0000 7f53ce448740 1 --2- 172.21.15.26:0/1157991161 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5638dd515cb0 0x5638dd5180a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:04.597 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.597+0000 7f53ce448740 1 --2- 172.21.15.26:0/1157991161 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5638dd50b410 0x5638dd515770 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:04.597 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.597+0000 7f53ce448740 1 --2- 172.21.15.26:0/1157991161 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5638dd50aaf0 0x5638dd50aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:04.597 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.597+0000 7f53ce448740 1 -- 172.21.15.26:0/1157991161 >> 172.21.15.26:0/1157991161 conn(0x5638dd507060 msgr2=0x5638dd507440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:34:04.598 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.599+0000 7f53ce448740 1 -- 172.21.15.26:0/1157991161 shutdown_connections 2024-09-11T23:34:04.599 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.600+0000 7f53ce448740 1 -- 172.21.15.26:0/1157991161 wait complete. 2024-09-11T23:34:04.600 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.600+0000 7f53ce448740 1 -- start start 2024-09-11T23:34:04.600 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.601+0000 7f53ce448740 1 Processor -- start 2024-09-11T23:34:04.602 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.602+0000 7f53ce448740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5638dd50aaf0 0x5638dd512ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:04.602 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.602+0000 7f53ce448740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5638dd50b410 0x5638dd5131e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:04.602 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.602+0000 7f53ce448740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5638dd515cb0 0x5638dd513720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:04.603 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.602+0000 7f53ce448740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5638dd518880 con 0x5638dd515cb0 2024-09-11T23:34:04.603 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.602+0000 7f53ce448740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5638dd3f8290 con 0x5638dd50b410 2024-09-11T23:34:04.603 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.602+0000 7f53ce448740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5638dd518a00 con 0x5638dd50aaf0 2024-09-11T23:34:04.603 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.602+0000 7f53ce448740 10 client.0 _ll_register_callbacks cb 0x5638dd429140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:34:04.603 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.602+0000 7f53c9374640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5638dd50b410 0x5638dd5131e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:34:04.603 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.602+0000 7f53c9374640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5638dd50b410 0x5638dd5131e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:48356/0 (socket says 172.21.15.26:48356) 2024-09-11T23:34:04.603 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.602+0000 7f53c9374640 1 -- 172.21.15.26:0/4235537329 learned_addr learned my addr 172.21.15.26:0/4235537329 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:34:04.604 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.603+0000 7f53c9374640 1 -- 172.21.15.26:0/4235537329 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5638dd50aaf0 msgr2=0x5638dd512ca0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T23:34:04.604 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.603+0000 7f53c9374640 1 --2- 172.21.15.26:0/4235537329 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5638dd50aaf0 0x5638dd512ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:04.604 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.603+0000 7f53c9374640 1 -- 172.21.15.26:0/4235537329 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5638dd515cb0 msgr2=0x5638dd513720 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T23:34:04.604 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.603+0000 7f53c9374640 1 --2- 172.21.15.26:0/4235537329 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5638dd515cb0 0x5638dd513720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:04.604 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.603+0000 7f53c9374640 1 -- 172.21.15.26:0/4235537329 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5638dd182210 con 0x5638dd50b410 2024-09-11T23:34:04.607 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.607+0000 7f53c9374640 1 --2- 172.21.15.26:0/4235537329 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5638dd50b410 0x5638dd5131e0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f53b4004fd0 tx=0x7f53b4005c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=85b64df5f099bfcf server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:34:04.607 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.607+0000 7f53b8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-11T23:34:04.609 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.609+0000 7f53b8ff9640 1 -- 172.21.15.26:0/4235537329 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53b4040030 con 0x5638dd50b410 2024-09-11T23:34:04.609 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.609+0000 7f53b8ff9640 1 -- 172.21.15.26:0/4235537329 <== mon.2 v2:172.21.15.76:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 8586+0+0 (secure 0 0 0) 0x7f53b40049b0 con 0x5638dd50b410 2024-09-11T23:34:04.609 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.610+0000 7f53ce448740 1 -- 172.21.15.26:0/4235537329 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f53b4005500 con 0x5638dd50b410 2024-09-11T23:34:04.611 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.610+0000 7f53b8ff9640 1 -- 172.21.15.26:0/4235537329 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) ==== 24+0+0 (secure 0 0 0) 0x7f53b4035040 con 0x5638dd50b410 2024-09-11T23:34:04.611 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.611+0000 7f53ce448740 10 client.27107 fetch_fsmap learned FSMap version 74 2024-09-11T23:34:04.611 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.611+0000 7f53ce448740 1 -- 172.21.15.26:0/4235537329 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=4+}) -- 0x5638dd24e620 con 0x5638dd50b410 2024-09-11T23:34:04.612 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.612+0000 7f53b8ff9640 1 -- 172.21.15.26:0/4235537329 <== mon.2 v2:172.21.15.76:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f53b4009520 con 0x5638dd50b410 2024-09-11T23:34:04.612 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.612+0000 7f53b8ff9640 1 -- 172.21.15.26:0/4235537329 <== mon.2 v2:172.21.15.76:3300/0 5 ==== fsmap(e 74) ==== 8438+0+0 (secure 0 0 0) 0x7f53b4003790 con 0x5638dd50b410 2024-09-11T23:34:04.612 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.612+0000 7f53ce448740 10 client.27107 fetch_fsmap finished waiting for FSMap version 74 2024-09-11T23:34:04.612 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.612+0000 7f53ce448740 10 client.27107 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:34:04.612 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.612+0000 7f53ce448740 20 client.27107 populate_metadata read hostname 'smithi026' 2024-09-11T23:34:04.613 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.612+0000 7f53ce448740 1 --2- 172.21.15.26:0/4235537329 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5638dd525ba0 0x5638dd527f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:04.613 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.612+0000 7f53ce448740 4 client.27107 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:34:04.613 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.612+0000 7f53ce448740 1 -- 172.21.15.26:0/4235537329 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f53b4005500 con 0x5638dd525ba0 2024-09-11T23:34:04.613 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.612+0000 7f53c8b73640 1 --2- 172.21.15.26:0/4235537329 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5638dd525ba0 0x5638dd527f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:34:04.614 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.613+0000 7f53c8b73640 1 --2- 172.21.15.26:0/4235537329 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5638dd525ba0 0x5638dd527f90 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db107d96c30c03d8 server_cookie=2980c567e43df5ef in_seq=0 out_seq=0 2024-09-11T23:34:04.614 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.614+0000 7f53b8ff9640 10 client.27107 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:34:04.614 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.614+0000 7f53b8ff9640 1 -- 172.21.15.26:0/4235537329 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f53b4005500 con 0x5638dd525ba0 2024-09-11T23:34:04.614 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.614+0000 7f53b8ff9640 10 client.27107 handle_command_reply: tid=0 2024-09-11T23:34:04.614 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.615+0000 7f53ce448740 1 client.27107 shutdown 2024-09-11T23:34:04.615 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.615+0000 7f53ce448740 1 -- 172.21.15.26:0/4235537329 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5638dd50b410 msgr2=0x5638dd5131e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:34:04.615 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.615+0000 7f53ce448740 1 --2- 172.21.15.26:0/4235537329 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5638dd50b410 0x5638dd5131e0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f53b4004fd0 tx=0x7f53b4005c00 comp rx=0 tx=0).stop 2024-09-11T23:34:04.615 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.615+0000 7f53ce448740 1 -- 172.21.15.26:0/4235537329 shutdown_connections 2024-09-11T23:34:04.615 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.615+0000 7f53ce448740 1 --2- 172.21.15.26:0/4235537329 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5638dd50b410 0x5638dd5131e0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:04.615 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.615+0000 7f53ce448740 1 --2- 172.21.15.26:0/4235537329 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5638dd525ba0 0x5638dd527f90 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:04.615 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.615+0000 7f53b8ff9640 0 client.27107 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:34:04.615 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.615+0000 7f53ce448740 1 --2- 172.21.15.26:0/4235537329 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5638dd515cb0 0x5638dd513720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:04.616 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.615+0000 7f53ce448740 1 --2- 172.21.15.26:0/4235537329 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5638dd50aaf0 0x5638dd512ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:04.616 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.615+0000 7f53ce448740 1 -- 172.21.15.26:0/4235537329 >> 172.21.15.26:0/4235537329 conn(0x5638dd507060 msgr2=0x5638dd50cdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:34:04.621 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.622+0000 7f53ce448740 1 -- 172.21.15.26:0/4235537329 shutdown_connections 2024-09-11T23:34:04.624 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.625+0000 7f53ce448740 1 -- 172.21.15.26:0/4235537329 wait complete. 2024-09-11T23:34:04.625 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.626+0000 7f53ce448740 20 client.27107 trim_cache size 0 max 16384 2024-09-11T23:34:04.656 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.656+0000 7f53ce448740 1 Processor -- start 2024-09-11T23:34:04.656 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.656+0000 7f53ce448740 1 -- start start 2024-09-11T23:34:04.657 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.657+0000 7f53ce448740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5638dd50aaf0 0x5638dd513550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:04.657 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.657+0000 7f53ce448740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5638dd50b410 0x5638dd52afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:04.657 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.657+0000 7f53ce448740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5638dd515cb0 0x5638dd52b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:04.658 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.657+0000 7f53c9374640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5638dd50b410 0x5638dd52afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:34:04.658 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.657+0000 7f53c9374640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5638dd50b410 0x5638dd52afb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:44378/0 (socket says 172.21.15.26:44378) 2024-09-11T23:34:04.658 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.657+0000 7f53c9374640 1 -- 172.21.15.26:0/221819772 learned_addr learned my addr 172.21.15.26:0/221819772 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:34:04.658 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.657+0000 7f53ce448740 1 -- 172.21.15.26:0/221819772 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f53c40739d0 con 0x5638dd50aaf0 2024-09-11T23:34:04.658 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.657+0000 7f53ce448740 1 -- 172.21.15.26:0/221819772 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f53c41c6ef0 con 0x5638dd515cb0 2024-09-11T23:34:04.658 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.658+0000 7f53ce448740 1 -- 172.21.15.26:0/221819772 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f53c4074160 con 0x5638dd50b410 2024-09-11T23:34:04.661 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.662+0000 7f53c8b73640 1 --2- 172.21.15.26:0/221819772 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5638dd50aaf0 0x5638dd513550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:34:04.661 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.662+0000 7f53c8b73640 1 --2- 172.21.15.26:0/221819772 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5638dd515cb0 0x5638dd52b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:34:04.661 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.662+0000 7f53c9374640 1 -- 172.21.15.26:0/221819772 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5638dd515cb0 msgr2=0x5638dd52b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:34:04.661 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.662+0000 7f53c9374640 1 --2- 172.21.15.26:0/221819772 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5638dd515cb0 0x5638dd52b4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:04.662 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.662+0000 7f53c9374640 1 -- 172.21.15.26:0/221819772 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5638dd50aaf0 msgr2=0x5638dd513550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:34:04.662 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.662+0000 7f53c9374640 1 --2- 172.21.15.26:0/221819772 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5638dd50aaf0 0x5638dd513550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:04.662 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.662+0000 7f53c9374640 1 -- 172.21.15.26:0/221819772 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5638dd511c30 con 0x5638dd50b410 2024-09-11T23:34:04.662 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.662+0000 7f53c9374640 1 --2- 172.21.15.26:0/221819772 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5638dd50b410 0x5638dd52afb0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f53b4053040 tx=0x7f53b4005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eff36c0a6caa62cd server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:34:04.663 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.664+0000 7f53baffd640 1 -- 172.21.15.26:0/221819772 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53b405f070 con 0x5638dd50b410 2024-09-11T23:34:04.663 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.664+0000 7f53baffd640 1 -- 172.21.15.26:0/221819772 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f53b4005410 con 0x5638dd50b410 2024-09-11T23:34:04.663 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.664+0000 7f53baffd640 1 -- 172.21.15.26:0/221819772 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53b4040070 con 0x5638dd50b410 2024-09-11T23:34:04.664 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.665+0000 7f53ce448740 1 -- 172.21.15.26:0/221819772 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5638dd50b410 msgr2=0x5638dd52afb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:34:04.664 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.665+0000 7f53ce448740 1 --2- 172.21.15.26:0/221819772 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5638dd50b410 0x5638dd52afb0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f53b4053040 tx=0x7f53b4005fd0 comp rx=0 tx=0).stop 2024-09-11T23:34:04.667 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.668+0000 7f53ce448740 1 -- 172.21.15.26:0/221819772 shutdown_connections 2024-09-11T23:34:04.667 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.668+0000 7f53ce448740 1 --2- 172.21.15.26:0/221819772 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5638dd515cb0 0x5638dd52b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:04.668 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.668+0000 7f53ce448740 1 --2- 172.21.15.26:0/221819772 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5638dd50b410 0x5638dd52afb0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:04.668 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.668+0000 7f53ce448740 1 --2- 172.21.15.26:0/221819772 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5638dd50aaf0 0x5638dd513550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:04.668 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.668+0000 7f53ce448740 1 -- 172.21.15.26:0/221819772 >> 172.21.15.26:0/221819772 conn(0x5638dd507060 msgr2=0x5638dd514d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:34:04.674 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.674+0000 7f53ce448740 1 -- 172.21.15.26:0/221819772 shutdown_connections 2024-09-11T23:34:04.674 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.675+0000 7f53ce448740 1 -- 172.21.15.26:0/221819772 wait complete. 2024-09-11T23:34:04.675 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.675+0000 7f53ce448740 1 -- start start 2024-09-11T23:34:04.675 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.675+0000 7f53ce448740 1 Processor -- start 2024-09-11T23:34:04.676 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.676+0000 7f53ce448740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5638dd50aaf0 0x5638dd529420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:04.676 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.676+0000 7f53ce448740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5638dd50b410 0x5638dd529960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:04.677 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.676+0000 7f53ce448740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5638dd515cb0 0x5638dd58f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:04.677 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.677+0000 7f53ce448740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f53c41c6760 con 0x5638dd515cb0 2024-09-11T23:34:04.677 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.677+0000 7f53ce448740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f53c41c6000 con 0x5638dd50b410 2024-09-11T23:34:04.677 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.677+0000 7f53ce448740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f53c41058f0 con 0x5638dd50aaf0 2024-09-11T23:34:04.677 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.677+0000 7f53ce448740 10 client.0 _ll_register_callbacks cb 0x7f53bc00d2a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:34:04.678 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.679+0000 7f53c9374640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5638dd50b410 0x5638dd529960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:34:04.678 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.679+0000 7f53c8b73640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5638dd50aaf0 0x5638dd529420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:34:04.679 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.679+0000 7f53c8b73640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5638dd50aaf0 0x5638dd529420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:44394/0 (socket says 172.21.15.26:44394) 2024-09-11T23:34:04.679 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.679+0000 7f53c8b73640 1 -- 172.21.15.26:0/3153407130 learned_addr learned my addr 172.21.15.26:0/3153407130 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:34:04.682 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.682+0000 7f53c8b73640 1 --2- 172.21.15.26:0/3153407130 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5638dd515cb0 0x5638dd58f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:34:04.683 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.683+0000 7f53c9374640 1 -- 172.21.15.26:0/3153407130 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5638dd50aaf0 msgr2=0x5638dd529420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:34:04.683 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.683+0000 7f53c9374640 1 --2- 172.21.15.26:0/3153407130 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5638dd50aaf0 0x5638dd529420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:04.683 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.683+0000 7f53c9374640 1 -- 172.21.15.26:0/3153407130 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5638dd515cb0 msgr2=0x5638dd58f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:34:04.683 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.683+0000 7f53c9374640 1 --2- 172.21.15.26:0/3153407130 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5638dd515cb0 0x5638dd58f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:04.684 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.683+0000 7f53c9374640 1 -- 172.21.15.26:0/3153407130 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5638dd58fa20 con 0x5638dd50b410 2024-09-11T23:34:04.684 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.683+0000 7f53c8b73640 1 --2- 172.21.15.26:0/3153407130 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5638dd50aaf0 0x5638dd529420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:34:04.688 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.689+0000 7f53c9374640 1 --2- 172.21.15.26:0/3153407130 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5638dd50b410 0x5638dd529960 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f53b4053040 tx=0x7f53b4005230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=10a4867b806064ec server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:34:04.695 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.692+0000 7f53b8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-11T23:34:04.695 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.692+0000 7f53b8ff9640 1 -- 172.21.15.26:0/3153407130 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53b4040070 con 0x5638dd50b410 2024-09-11T23:34:04.695 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.692+0000 7f53ce448740 1 -- 172.21.15.26:0/3153407130 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f53b4005410 con 0x5638dd50b410 2024-09-11T23:34:04.695 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.692+0000 7f53b8ff9640 1 -- 172.21.15.26:0/3153407130 <== mon.2 v2:172.21.15.76:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 8586+0+0 (secure 0 0 0) 0x7f53b4003d00 con 0x5638dd50b410 2024-09-11T23:34:04.695 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.692+0000 7f53b8ff9640 1 -- 172.21.15.26:0/3153407130 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) ==== 24+0+0 (secure 0 0 0) 0x7f53b405f040 con 0x5638dd50b410 2024-09-11T23:34:04.695 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.693+0000 7f53ce448740 10 client.27110 fetch_fsmap learned FSMap version 74 2024-09-11T23:34:04.695 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.693+0000 7f53ce448740 1 -- 172.21.15.26:0/3153407130 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=4+}) -- 0x5638dd590cd0 con 0x5638dd50b410 2024-09-11T23:34:04.695 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.693+0000 7f53b8ff9640 1 -- 172.21.15.26:0/3153407130 <== mon.2 v2:172.21.15.76:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f53b4005410 con 0x5638dd50b410 2024-09-11T23:34:04.696 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.694+0000 7f53b8ff9640 1 -- 172.21.15.26:0/3153407130 <== mon.2 v2:172.21.15.76:3300/0 5 ==== fsmap(e 74) ==== 8438+0+0 (secure 0 0 0) 0x7f53b406a030 con 0x5638dd50b410 2024-09-11T23:34:04.696 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.694+0000 7f53ce448740 10 client.27110 fetch_fsmap finished waiting for FSMap version 74 2024-09-11T23:34:04.696 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.694+0000 7f53ce448740 10 client.27110 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:34:04.696 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.694+0000 7f53ce448740 20 client.27110 populate_metadata read hostname 'smithi026' 2024-09-11T23:34:04.696 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.694+0000 7f53ce448740 1 --2- 172.21.15.26:0/3153407130 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5638dd591280 0x5638dd5936a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:04.696 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.694+0000 7f53ce448740 4 client.27110 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T23:34:04.696 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.694+0000 7f53ce448740 1 -- 172.21.15.26:0/3153407130 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5638dd47ca20 con 0x5638dd591280 2024-09-11T23:34:04.697 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.696+0000 7f53c8b73640 1 --2- 172.21.15.26:0/3153407130 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5638dd591280 0x5638dd5936a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:34:04.697 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.697+0000 7f53c8b73640 1 --2- 172.21.15.26:0/3153407130 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5638dd591280 0x5638dd5936a0 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=880a2dc4085eddbe server_cookie=90350430d07aa8a1 in_seq=0 out_seq=0 2024-09-11T23:34:04.697 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.697+0000 7f53b8ff9640 10 client.27110 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:34:04.811 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.810+0000 7f53b8ff9640 1 -- 172.21.15.26:0/3153407130 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5638dd47ca20 con 0x5638dd591280 2024-09-11T23:34:04.812 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.810+0000 7f53b8ff9640 10 client.27110 handle_command_reply: tid=0 2024-09-11T23:34:04.812 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.810+0000 7f53ce448740 1 client.27110 shutdown 2024-09-11T23:34:04.812 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.810+0000 7f53ce448740 1 -- 172.21.15.26:0/3153407130 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5638dd50b410 msgr2=0x5638dd529960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:34:04.812 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.810+0000 7f53ce448740 1 --2- 172.21.15.26:0/3153407130 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5638dd50b410 0x5638dd529960 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f53b4053040 tx=0x7f53b4005230 comp rx=0 tx=0).stop 2024-09-11T23:34:04.812 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.810+0000 7f53ce448740 1 -- 172.21.15.26:0/3153407130 shutdown_connections 2024-09-11T23:34:04.812 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.811+0000 7f53ce448740 1 --2- 172.21.15.26:0/3153407130 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5638dd50b410 0x5638dd529960 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:04.812 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.811+0000 7f53ce448740 1 --2- 172.21.15.26:0/3153407130 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5638dd591280 0x5638dd5936a0 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:04.812 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.811+0000 7f53ce448740 1 --2- 172.21.15.26:0/3153407130 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5638dd515cb0 0x5638dd58f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:04.812 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.811+0000 7f53ce448740 1 --2- 172.21.15.26:0/3153407130 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5638dd50aaf0 0x5638dd529420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:04.812 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.811+0000 7f53ce448740 1 -- 172.21.15.26:0/3153407130 >> 172.21.15.26:0/3153407130 conn(0x5638dd507060 msgr2=0x5638dd520c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:34:04.813 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.811+0000 7f53b8ff9640 0 client.27110 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:34:04.813 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.811+0000 7f53ce448740 1 -- 172.21.15.26:0/3153407130 shutdown_connections 2024-09-11T23:34:04.813 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.811+0000 7f53ce448740 1 -- 172.21.15.26:0/3153407130 wait complete. 2024-09-11T23:34:04.813 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.811+0000 7f53ce448740 20 client.27110 trim_cache size 0 max 16384 2024-09-11T23:34:04.813 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-09-11T23:34:04.814 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "message": "", 2024-09-11T23:34:04.814 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-09-11T23:34:04.814 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-09-11T23:34:04.815 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.812+0000 7f53cb5d8640 1 -- 172.21.15.26:0/2313679085 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f53b00a2fd0 msgr2=0x7f53b00a5490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:34:04.815 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.812+0000 7f53cb5d8640 1 --2- 172.21.15.26:0/2313679085 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f53b00a2fd0 0x7f53b00a5490 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f53ac0cc020 tx=0x7f53ac00e040 comp rx=0 tx=0).stop 2024-09-11T23:34:04.815 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.812+0000 7f53cb5d8640 1 -- 172.21.15.26:0/2313679085 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f53c4105f10 msgr2=0x7f53c410e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:34:04.815 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.812+0000 7f53cb5d8640 1 --2- 172.21.15.26:0/2313679085 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f53c4105f10 0x7f53c410e260 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f53ac00b970 tx=0x7f53ac00be30 comp rx=0 tx=0).stop 2024-09-11T23:34:04.815 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.812+0000 7f53cb5d8640 1 -- 172.21.15.26:0/2313679085 shutdown_connections 2024-09-11T23:34:04.815 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.812+0000 7f53cb5d8640 1 --2- 172.21.15.26:0/2313679085 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f53b00a2fd0 0x7f53b00a5490 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:04.816 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.812+0000 7f53cb5d8640 1 --2- 172.21.15.26:0/2313679085 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f53c4105f10 0x7f53c410e260 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:04.816 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.812+0000 7f53cb5d8640 1 --2- 172.21.15.26:0/2313679085 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f53c41013b0 0x7f53c410dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:04.816 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.812+0000 7f53cb5d8640 1 --2- 172.21.15.26:0/2313679085 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f53c4100a90 0x7f53c4114ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:04.816 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.812+0000 7f53cb5d8640 1 -- 172.21.15.26:0/2313679085 >> 172.21.15.26:0/2313679085 conn(0x7f53c40fc6b0 msgr2=0x7f53c4103840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:34:04.816 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.812+0000 7f53cb5d8640 1 -- 172.21.15.26:0/2313679085 shutdown_connections 2024-09-11T23:34:04.816 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:04.812+0000 7f53cb5d8640 1 -- 172.21.15.26:0/2313679085 wait complete. 2024-09-11T23:34:05.238 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:34:04 smithi136 ceph-mon[35121]: osdmap e105: 12 total, 12 up, 12 in 2024-09-11T23:34:05.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:34:04 smithi026 ceph-mon[31172]: osdmap e105: 12 total, 12 up, 12 in 2024-09-11T23:34:05.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:34:04 smithi076 ceph-mon[35296]: osdmap e105: 12 total, 12 up, 12 in 2024-09-11T23:34:06.238 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:34:05 smithi136 ceph-mon[35121]: pgmap v493: 97 pgs: 4 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 86 active+clean; 2.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.3 MiB/s wr, 391 op/s 2024-09-11T23:34:06.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:34:05 smithi026 ceph-mon[31172]: pgmap v493: 97 pgs: 4 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 86 active+clean; 2.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.3 MiB/s wr, 391 op/s 2024-09-11T23:34:06.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:34:05 smithi076 ceph-mon[35296]: pgmap v493: 97 pgs: 4 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 86 active+clean; 2.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.3 MiB/s wr, 391 op/s 2024-09-11T23:34:08.238 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:34:07 smithi136 ceph-mon[35121]: pgmap v494: 97 pgs: 14 active+clean+snaptrim, 26 active+clean+snaptrim_wait, 57 active+clean; 2.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.0 MiB/s wr, 366 op/s 2024-09-11T23:34:08.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:34:07 smithi026 ceph-mon[31172]: pgmap v494: 97 pgs: 14 active+clean+snaptrim, 26 active+clean+snaptrim_wait, 57 active+clean; 2.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.0 MiB/s wr, 366 op/s 2024-09-11T23:34:08.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:34:07 smithi076 ceph-mon[35296]: pgmap v494: 97 pgs: 14 active+clean+snaptrim, 26 active+clean+snaptrim_wait, 57 active+clean; 2.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.0 MiB/s wr, 366 op/s 2024-09-11T23:34:10.238 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:34:09 smithi136 ceph-mon[35121]: pgmap v495: 97 pgs: 16 active+clean+snaptrim, 28 active+clean+snaptrim_wait, 53 active+clean; 2.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.9 MiB/s wr, 360 op/s 2024-09-11T23:34:10.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:34:09 smithi026 ceph-mon[31172]: pgmap v495: 97 pgs: 16 active+clean+snaptrim, 28 active+clean+snaptrim_wait, 53 active+clean; 2.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.9 MiB/s wr, 360 op/s 2024-09-11T23:34:10.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:34:09 smithi076 ceph-mon[35296]: pgmap v495: 97 pgs: 16 active+clean+snaptrim, 28 active+clean+snaptrim_wait, 53 active+clean; 2.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.9 MiB/s wr, 360 op/s 2024-09-11T23:34:12.238 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:34:11 smithi136 ceph-mon[35121]: pgmap v496: 97 pgs: 23 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 33 active+clean; 2.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.6 MiB/s wr, 286 op/s 2024-09-11T23:34:12.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:34:11 smithi026 ceph-mon[31172]: pgmap v496: 97 pgs: 23 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 33 active+clean; 2.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.6 MiB/s wr, 286 op/s 2024-09-11T23:34:12.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:34:11 smithi076 ceph-mon[35296]: pgmap v496: 97 pgs: 23 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 33 active+clean; 2.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.6 MiB/s wr, 286 op/s 2024-09-11T23:34:14.238 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:34:13 smithi136 ceph-mon[35121]: pgmap v497: 97 pgs: 23 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 33 active+clean; 2.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.6 MiB/s wr, 276 op/s 2024-09-11T23:34:14.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:34:13 smithi026 ceph-mon[31172]: pgmap v497: 97 pgs: 23 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 33 active+clean; 2.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.6 MiB/s wr, 276 op/s 2024-09-11T23:34:14.304 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:34:13 smithi076 ceph-mon[35296]: pgmap v497: 97 pgs: 23 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 33 active+clean; 2.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.6 MiB/s wr, 276 op/s 2024-09-11T23:34:16.238 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:34:15 smithi136 ceph-mon[35121]: pgmap v498: 97 pgs: 23 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 33 active+clean; 2.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 232 op/s 2024-09-11T23:34:16.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:34:15 smithi026 ceph-mon[31172]: pgmap v498: 97 pgs: 23 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 33 active+clean; 2.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 232 op/s 2024-09-11T23:34:16.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:34:15 smithi076 ceph-mon[35296]: pgmap v498: 97 pgs: 23 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 33 active+clean; 2.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 232 op/s 2024-09-11T23:34:17.970 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:17.969+0000 7fadf6653640 1 -- 172.21.15.26:0/876509154 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fadf01009e0 msgr2=0x7fadf01046a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:34:17.970 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:17.969+0000 7fadf6653640 1 --2- 172.21.15.26:0/876509154 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fadf01009e0 0x7fadf01046a0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fade8009ca0 tx=0x7fade802a8f0 comp rx=0 tx=0).stop 2024-09-11T23:34:17.973 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:17.973+0000 7fadf6653640 1 -- 172.21.15.26:0/876509154 shutdown_connections 2024-09-11T23:34:17.973 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:17.973+0000 7fadf6653640 1 --2- 172.21.15.26:0/876509154 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fadf01009e0 0x7fadf01046a0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:17.973 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:17.973+0000 7fadf6653640 1 --2- 172.21.15.26:0/876509154 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fadf00fff60 0x7fadf01003c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:17.973 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:17.973+0000 7fadf6653640 1 --2- 172.21.15.26:0/876509154 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fadf0106be0 0x7fadf0106fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:17.973 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:17.973+0000 7fadf6653640 1 -- 172.21.15.26:0/876509154 >> 172.21.15.26:0/876509154 conn(0x7fadf00770f0 msgr2=0x7fadf00fe480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:34:17.977 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:17.978+0000 7fadf6653640 1 -- 172.21.15.26:0/876509154 shutdown_connections 2024-09-11T23:34:17.980 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:17.980+0000 7fadf6653640 1 -- 172.21.15.26:0/876509154 wait complete. 2024-09-11T23:34:17.980 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:17.980+0000 7fadf6653640 1 Processor -- start 2024-09-11T23:34:17.985 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:17.984+0000 7fadf6653640 1 -- start start 2024-09-11T23:34:17.985 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:17.985+0000 7fadf6653640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fadf00fff60 0x7fadf01153a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:17.985 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:17.985+0000 7fadf6653640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fadf01009e0 0x7fadf01158e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:17.985 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:17.985+0000 7fadf6653640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fadf0106be0 0x7fadf010e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:17.985 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:17.985+0000 7fadf6653640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fadf0118600 con 0x7fadf01009e0 2024-09-11T23:34:17.985 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:17.985+0000 7fadf6653640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fadf0118480 con 0x7fadf00fff60 2024-09-11T23:34:17.986 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:17.985+0000 7fadf6653640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fadf0118780 con 0x7fadf0106be0 2024-09-11T23:34:17.986 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:17.987+0000 7fadf4bf0640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fadf0106be0 0x7fadf010e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:34:17.987 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:17.987+0000 7fadf4bf0640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fadf0106be0 0x7fadf010e560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:36390/0 (socket says 172.21.15.26:36390) 2024-09-11T23:34:17.987 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:17.987+0000 7fadf4bf0640 1 -- 172.21.15.26:0/866310909 learned_addr learned my addr 172.21.15.26:0/866310909 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:34:17.988 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:17.988+0000 7fadef7fe640 1 --2- 172.21.15.26:0/866310909 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fadf01009e0 0x7fadf01158e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:34:17.988 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:17.988+0000 7fadf4bf0640 1 -- 172.21.15.26:0/866310909 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fadf00fff60 msgr2=0x7fadf01153a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-11T23:34:17.988 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:17.988+0000 7fadf4bf0640 1 --2- 172.21.15.26:0/866310909 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fadf00fff60 0x7fadf01153a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:17.988 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:17.988+0000 7fadf4bf0640 1 -- 172.21.15.26:0/866310909 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fadf01009e0 msgr2=0x7fadf01158e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:34:17.988 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:17.988+0000 7fadf4bf0640 1 --2- 172.21.15.26:0/866310909 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fadf01009e0 0x7fadf01158e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:17.989 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:17.988+0000 7fadf4bf0640 1 -- 172.21.15.26:0/866310909 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadf010ee20 con 0x7fadf0106be0 2024-09-11T23:34:17.990 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:17.990+0000 7fadf4bf0640 1 --2- 172.21.15.26:0/866310909 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fadf0106be0 0x7fadf010e560 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fade802ae60 tx=0x7fade8003f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:34:17.990 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:17.990+0000 7faded7fa640 1 -- 172.21.15.26:0/866310909 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fade8031700 con 0x7fadf0106be0 2024-09-11T23:34:17.991 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:17.990+0000 7fadf6653640 1 -- 172.21.15.26:0/866310909 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadf010f0b0 con 0x7fadf0106be0 2024-09-11T23:34:17.991 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:17.990+0000 7fadf6653640 1 -- 172.21.15.26:0/866310909 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadf01130f0 con 0x7fadf0106be0 2024-09-11T23:34:17.991 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:17.991+0000 7faded7fa640 1 -- 172.21.15.26:0/866310909 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fade802d070 con 0x7fadf0106be0 2024-09-11T23:34:17.991 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:17.991+0000 7faded7fa640 1 -- 172.21.15.26:0/866310909 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fade802f8a0 con 0x7fadf0106be0 2024-09-11T23:34:18.004 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:17.997+0000 7fadf93f0740 1 Processor -- start 2024-09-11T23:34:18.004 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:17.997+0000 7fadf93f0740 1 -- start start 2024-09-11T23:34:18.004 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:17.997+0000 7fadf93f0740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562f56f34af0 0x562f56f34ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:18.006 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:17.997+0000 7fadf93f0740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562f56f35410 0x562f56f3f770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:18.006 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:17.997+0000 7fadf93f0740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562f56f3fcb0 0x562f56f420a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:18.006 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:17.997+0000 7fadf93f0740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x562f56e992f0 con 0x562f56f35410 2024-09-11T23:34:18.006 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:17.997+0000 7fadf93f0740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x562f56e41270 con 0x562f56f34af0 2024-09-11T23:34:18.006 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:17.997+0000 7fadf93f0740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x562f56e952b0 con 0x562f56f3fcb0 2024-09-11T23:34:18.006 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:17.997+0000 7fadef7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562f56f34af0 0x562f56f34ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:34:18.006 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:17.998+0000 7fadef7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562f56f34af0 0x562f56f34ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:53830/0 (socket says 172.21.15.26:53830) 2024-09-11T23:34:18.006 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:17.998+0000 7fadef7fe640 1 -- 172.21.15.26:0/1735620739 learned_addr learned my addr 172.21.15.26:0/1735620739 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:34:18.006 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:17.998+0000 7fadef7fe640 1 --2- 172.21.15.26:0/1735620739 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562f56f3fcb0 0x562f56f420a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:34:18.007 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:17.998+0000 7fadef7fe640 1 -- 172.21.15.26:0/1735620739 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562f56f3fcb0 msgr2=0x562f56f420a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:34:18.007 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:17.998+0000 7fadef7fe640 1 --2- 172.21.15.26:0/1735620739 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562f56f3fcb0 0x562f56f420a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:18.007 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:17.998+0000 7fadef7fe640 1 -- 172.21.15.26:0/1735620739 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562f56f35410 msgr2=0x562f56f3f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:34:18.007 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:17.998+0000 7fadef7fe640 1 --2- 172.21.15.26:0/1735620739 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562f56f35410 0x562f56f3f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:18.007 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:17.998+0000 7fadef7fe640 1 -- 172.21.15.26:0/1735620739 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f56e40340 con 0x562f56f34af0 2024-09-11T23:34:18.007 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:17.998+0000 7fadef7fe640 1 --2- 172.21.15.26:0/1735620739 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562f56f34af0 0x562f56f34ed0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fade4034790 tx=0x7fade4004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=31c0661655dbbc56 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:34:18.007 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:17.998+0000 7faded7fa640 1 -- 172.21.15.26:0/866310909 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fade802fa40 con 0x7fadf0106be0 2024-09-11T23:34:18.007 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:17.999+0000 7fadd67fc640 1 -- 172.21.15.26:0/1735620739 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fade4034d10 con 0x562f56f34af0 2024-09-11T23:34:18.007 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:17.999+0000 7fadd67fc640 1 -- 172.21.15.26:0/1735620739 <== mon.2 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fade4032780 con 0x562f56f34af0 2024-09-11T23:34:18.007 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:18.000+0000 7faded7fa640 1 --2- 172.21.15.26:0/866310909 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fadd80a2ef0 0x7fadd80a53b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:18.007 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:18.000+0000 7fadf93f0740 1 -- 172.21.15.26:0/1735620739 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562f56f34af0 msgr2=0x562f56f34ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:34:18.007 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:18.000+0000 7fadf93f0740 1 --2- 172.21.15.26:0/1735620739 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562f56f34af0 0x562f56f34ed0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fade4034790 tx=0x7fade4004730 comp rx=0 tx=0).stop 2024-09-11T23:34:18.008 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:18.000+0000 7faded7fa640 1 -- 172.21.15.26:0/866310909 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 8586+0+0 (secure 0 0 0) 0x7fade80f1990 con 0x7fadf0106be0 2024-09-11T23:34:18.008 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:18.000+0000 7fadf93f0740 1 -- 172.21.15.26:0/1735620739 shutdown_connections 2024-09-11T23:34:18.008 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:18.000+0000 7fadf93f0740 1 --2- 172.21.15.26:0/1735620739 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562f56f3fcb0 0x562f56f420a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:18.008 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:18.000+0000 7fadf93f0740 1 --2- 172.21.15.26:0/1735620739 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562f56f35410 0x562f56f3f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:18.008 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:18.000+0000 7fadf93f0740 1 --2- 172.21.15.26:0/1735620739 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562f56f34af0 0x562f56f34ed0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:18.008 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:18.000+0000 7fadf93f0740 1 -- 172.21.15.26:0/1735620739 >> 172.21.15.26:0/1735620739 conn(0x562f56f31060 msgr2=0x562f56f31440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:34:18.008 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:18.002+0000 7fadeffff640 1 --2- 172.21.15.26:0/866310909 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fadd80a2ef0 0x7fadd80a53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:34:18.008 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:18.002+0000 7fadf93f0740 1 -- 172.21.15.26:0/1735620739 shutdown_connections 2024-09-11T23:34:18.008 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:18.002+0000 7fadf93f0740 1 -- 172.21.15.26:0/1735620739 wait complete. 2024-09-11T23:34:18.008 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:18.002+0000 7fadf93f0740 1 -- start start 2024-09-11T23:34:18.008 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:18.003+0000 7fadf93f0740 1 Processor -- start 2024-09-11T23:34:18.008 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:18.003+0000 7fadeffff640 1 --2- 172.21.15.26:0/866310909 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fadd80a2ef0 0x7fadd80a53b0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fade0006fd0 tx=0x7fade0006bc0 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:34:18.008 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:18.003+0000 7fadf93f0740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562f56f34af0 0x562f56f3f750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:18.009 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:18.003+0000 7fadf93f0740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562f56f35410 0x562f56f3c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:18.009 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:18.003+0000 7fadf93f0740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562f56f3fcb0 0x562f56f3d180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:18.009 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:18.003+0000 7fadf93f0740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x562f56f428d0 con 0x562f56f34af0 2024-09-11T23:34:18.009 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:18.003+0000 7fadf93f0740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x562f56e22290 con 0x562f56f3fcb0 2024-09-11T23:34:18.009 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:18.003+0000 7fadf93f0740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x562f56f42a50 con 0x562f56f35410 2024-09-11T23:34:18.009 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:18.003+0000 7fadf93f0740 10 client.0 _ll_register_callbacks cb 0x562f56e53140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:34:18.009 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:18.004+0000 7fadef7fe640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562f56f35410 0x562f56f3c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:34:18.009 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:18.004+0000 7fadef7fe640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562f56f35410 0x562f56f3c9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:36406/0 (socket says 172.21.15.26:36406) 2024-09-11T23:34:18.009 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:18.004+0000 7fadef7fe640 1 -- 172.21.15.26:0/2457630655 learned_addr learned my addr 172.21.15.26:0/2457630655 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:34:18.009 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:18.004+0000 7fadeffff640 1 --2- 172.21.15.26:0/2457630655 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562f56f34af0 0x562f56f3f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:34:18.009 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:18.004+0000 7fadef7fe640 1 -- 172.21.15.26:0/2457630655 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562f56f3fcb0 msgr2=0x562f56f3d180 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T23:34:18.009 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:18.004+0000 7fadef7fe640 1 --2- 172.21.15.26:0/2457630655 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562f56f3fcb0 0x562f56f3d180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:18.009 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:18.004+0000 7fadef7fe640 1 -- 172.21.15.26:0/2457630655 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562f56f34af0 msgr2=0x562f56f3f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:34:18.009 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:18.004+0000 7fadef7fe640 1 --2- 172.21.15.26:0/2457630655 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562f56f34af0 0x562f56f3f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:18.010 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:18.004+0000 7fadef7fe640 1 -- 172.21.15.26:0/2457630655 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562f56bac210 con 0x562f56f35410 2024-09-11T23:34:18.010 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:18.004+0000 7fadeffff640 1 --2- 172.21.15.26:0/2457630655 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562f56f34af0 0x562f56f3f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:34:18.010 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:18.004+0000 7fadef7fe640 1 --2- 172.21.15.26:0/2457630655 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562f56f35410 0x562f56f3c9f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fade400bac0 tx=0x7fade4032be0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f6b185b7a6c19e5f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:34:18.010 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:18.004+0000 7fadcf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-09-11T23:34:18.011 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:18.008+0000 7fadcf7fe640 1 -- 172.21.15.26:0/2457630655 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fade4041830 con 0x562f56f35410 2024-09-11T23:34:18.011 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:18.008+0000 7fadf93f0740 1 -- 172.21.15.26:0/2457630655 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fade4032780 con 0x562f56f35410 2024-09-11T23:34:18.011 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:18.008+0000 7fadcf7fe640 1 -- 172.21.15.26:0/2457630655 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 8586+0+0 (secure 0 0 0) 0x7fade40419d0 con 0x562f56f35410 2024-09-11T23:34:18.011 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:18.009+0000 7fadcf7fe640 1 -- 172.21.15.26:0/2457630655 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) ==== 24+0+0 (secure 0 0 0) 0x7fade4032dd0 con 0x562f56f35410 2024-09-11T23:34:18.011 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:18.010+0000 7fadf93f0740 10 client.27100 fetch_fsmap learned FSMap version 74 2024-09-11T23:34:18.011 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:18.010+0000 7fadf93f0740 1 -- 172.21.15.26:0/2457630655 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=4+}) -- 0x562f56c78620 con 0x562f56f35410 2024-09-11T23:34:18.011 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:18.012+0000 7fadcf7fe640 1 -- 172.21.15.26:0/2457630655 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fade4038040 con 0x562f56f35410 2024-09-11T23:34:18.011 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:18.012+0000 7fadcf7fe640 1 -- 172.21.15.26:0/2457630655 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 74) ==== 8438+0+0 (secure 0 0 0) 0x7fade4051030 con 0x562f56f35410 2024-09-11T23:34:18.013 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:18.013+0000 7fadf93f0740 10 client.27100 fetch_fsmap finished waiting for FSMap version 74 2024-09-11T23:34:18.013 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:18.013+0000 7fadf93f0740 10 client.27100 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:34:18.013 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:18.013+0000 7fadf93f0740 20 client.27100 populate_metadata read hostname 'smithi026' 2024-09-11T23:34:18.013 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:18.013+0000 7fadf93f0740 1 --2- 172.21.15.26:0/2457630655 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x562f56f4fac0 0x562f56f51eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:18.013 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:18.013+0000 7fadf93f0740 4 client.27100 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:34:18.013 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:18.013+0000 7fadf93f0740 1 -- 172.21.15.26:0/2457630655 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fade4032780 con 0x562f56f4fac0 2024-09-11T23:34:18.014 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:18.014+0000 7fadeffff640 1 --2- 172.21.15.26:0/2457630655 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x562f56f4fac0 0x562f56f51eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:34:18.018 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:18.019+0000 7fadeffff640 1 --2- 172.21.15.26:0/2457630655 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x562f56f4fac0 0x562f56f51eb0 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b79d57b2976fa92 server_cookie=ba42082aa14729ad in_seq=0 out_seq=0 2024-09-11T23:34:18.018 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:18.019+0000 7fadcf7fe640 10 client.27100 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:34:18.020 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:18.020+0000 7fadcf7fe640 1 -- 172.21.15.26:0/2457630655 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7fade4032780 con 0x562f56f4fac0 2024-09-11T23:34:18.020 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:18.021+0000 7fadcf7fe640 10 client.27100 handle_command_reply: tid=0 2024-09-11T23:34:18.020 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:18.021+0000 7fadf93f0740 1 client.27100 shutdown 2024-09-11T23:34:18.023 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:18.022+0000 7fadf93f0740 1 -- 172.21.15.26:0/2457630655 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562f56f35410 msgr2=0x562f56f3c9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:34:18.023 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:18.022+0000 7fadf93f0740 1 --2- 172.21.15.26:0/2457630655 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562f56f35410 0x562f56f3c9f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fade400bac0 tx=0x7fade4032be0 comp rx=0 tx=0).stop 2024-09-11T23:34:18.024 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:18.024+0000 7fadf93f0740 1 -- 172.21.15.26:0/2457630655 shutdown_connections 2024-09-11T23:34:18.024 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:18.024+0000 7fadf93f0740 1 --2- 172.21.15.26:0/2457630655 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562f56f3fcb0 0x562f56f3d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:18.024 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:18.024+0000 7fadf93f0740 1 --2- 172.21.15.26:0/2457630655 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x562f56f4fac0 0x562f56f51eb0 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:18.024 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:18.024+0000 7fadf93f0740 1 --2- 172.21.15.26:0/2457630655 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562f56f35410 0x562f56f3c9f0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:18.024 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:18.024+0000 7fadf93f0740 1 --2- 172.21.15.26:0/2457630655 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562f56f34af0 0x562f56f3f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:18.024 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:18.024+0000 7fadf93f0740 1 -- 172.21.15.26:0/2457630655 >> 172.21.15.26:0/2457630655 conn(0x562f56f31060 msgr2=0x562f56f36de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:34:18.024 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:18.024+0000 7fadcf7fe640 0 client.27100 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:34:18.025 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:18.025+0000 7fadf93f0740 1 -- 172.21.15.26:0/2457630655 shutdown_connections 2024-09-11T23:34:18.026 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:18.027+0000 7fadf93f0740 1 -- 172.21.15.26:0/2457630655 wait complete. 2024-09-11T23:34:18.026 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:18.027+0000 7fadf93f0740 20 client.27100 trim_cache size 0 max 16384 2024-09-11T23:34:18.063 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:18.062+0000 7fadf93f0740 1 Processor -- start 2024-09-11T23:34:18.063 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:18.062+0000 7fadf93f0740 1 -- start start 2024-09-11T23:34:18.063 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:18.062+0000 7fadf93f0740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562f56f34af0 0x562f56f300b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:18.063 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:18.062+0000 7fadf93f0740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562f56f35410 0x562f56f305f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:18.063 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:18.062+0000 7fadf93f0740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562f56f3fcb0 0x562f56f576c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:18.064 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:18.062+0000 7fadf93f0740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7faddc00e9e0 con 0x562f56f34af0 2024-09-11T23:34:18.064 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:18.062+0000 7fadf93f0740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7faddc0107f0 con 0x562f56f3fcb0 2024-09-11T23:34:18.064 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:18.062+0000 7fadf93f0740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7faddc00f170 con 0x562f56f35410 2024-09-11T23:34:18.064 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:18.063+0000 7fadef7fe640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562f56f35410 0x562f56f305f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:34:18.064 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:18.063+0000 7fadef7fe640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562f56f35410 0x562f56f305f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:36410/0 (socket says 172.21.15.26:36410) 2024-09-11T23:34:18.064 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:18.063+0000 7fadef7fe640 1 -- 172.21.15.26:0/3427398948 learned_addr learned my addr 172.21.15.26:0/3427398948 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:34:18.064 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:18.063+0000 7fadf4bf0640 1 --2- 172.21.15.26:0/3427398948 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562f56f3fcb0 0x562f56f576c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:34:18.064 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:18.063+0000 7fadef7fe640 1 -- 172.21.15.26:0/3427398948 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562f56f3fcb0 msgr2=0x562f56f576c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:34:18.064 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:18.063+0000 7fadef7fe640 1 --2- 172.21.15.26:0/3427398948 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562f56f3fcb0 0x562f56f576c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:18.064 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:18.063+0000 7fadef7fe640 1 -- 172.21.15.26:0/3427398948 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562f56f34af0 msgr2=0x562f56f300b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T23:34:18.064 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:18.063+0000 7fadef7fe640 1 --2- 172.21.15.26:0/3427398948 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562f56f34af0 0x562f56f300b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:18.064 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:18.063+0000 7fadef7fe640 1 -- 172.21.15.26:0/3427398948 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f56f30e90 con 0x562f56f35410 2024-09-11T23:34:18.065 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:18.064+0000 7fadef7fe640 1 --2- 172.21.15.26:0/3427398948 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562f56f35410 0x562f56f305f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fade4052f70 tx=0x7fade400bed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=42e3bce7df805ca0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:34:18.065 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:18.064+0000 7fadd5ffb640 1 -- 172.21.15.26:0/3427398948 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fade4032750 con 0x562f56f35410 2024-09-11T23:34:18.066 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:18.067+0000 7fadd5ffb640 1 -- 172.21.15.26:0/3427398948 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fade400b390 con 0x562f56f35410 2024-09-11T23:34:18.070 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:18.068+0000 7fadf93f0740 1 -- 172.21.15.26:0/3427398948 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562f56f35410 msgr2=0x562f56f305f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:34:18.070 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:18.068+0000 7fadf93f0740 1 --2- 172.21.15.26:0/3427398948 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562f56f35410 0x562f56f305f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fade4052f70 tx=0x7fade400bed0 comp rx=0 tx=0).stop 2024-09-11T23:34:18.070 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:18.069+0000 7fadf93f0740 1 -- 172.21.15.26:0/3427398948 shutdown_connections 2024-09-11T23:34:18.070 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:18.069+0000 7fadf93f0740 1 --2- 172.21.15.26:0/3427398948 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562f56f3fcb0 0x562f56f576c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:18.070 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:18.070+0000 7fadf93f0740 1 --2- 172.21.15.26:0/3427398948 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562f56f35410 0x562f56f305f0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:18.070 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:18.070+0000 7fadf93f0740 1 --2- 172.21.15.26:0/3427398948 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562f56f34af0 0x562f56f300b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:18.070 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:18.070+0000 7fadf93f0740 1 -- 172.21.15.26:0/3427398948 >> 172.21.15.26:0/3427398948 conn(0x562f56f31060 msgr2=0x562f56f2f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:34:18.072 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:18.071+0000 7fadf93f0740 1 -- 172.21.15.26:0/3427398948 shutdown_connections 2024-09-11T23:34:18.076 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:18.074+0000 7fadf93f0740 1 -- 172.21.15.26:0/3427398948 wait complete. 2024-09-11T23:34:18.076 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:18.074+0000 7fadf93f0740 1 -- start start 2024-09-11T23:34:18.076 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:18.074+0000 7fadf93f0740 1 Processor -- start 2024-09-11T23:34:18.076 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:18.076+0000 7fadf93f0740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562f56f34af0 0x562f56f48680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:18.076 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:18.076+0000 7fadf93f0740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562f56f35410 0x562f56f39850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:18.076 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:18.076+0000 7fadf93f0740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562f56f3fcb0 0x562f56f39fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:18.076 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:18.076+0000 7fadf93f0740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7faddc010060 con 0x562f56f35410 2024-09-11T23:34:18.076 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:18.076+0000 7fadf93f0740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7faddc00f900 con 0x562f56f3fcb0 2024-09-11T23:34:18.076 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:18.076+0000 7fadf93f0740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7faddc00c9e0 con 0x562f56f34af0 2024-09-11T23:34:18.077 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:18.076+0000 7fadf93f0740 10 client.0 _ll_register_callbacks cb 0x7fade400bdb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:34:18.077 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:18.077+0000 7fadef7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562f56f35410 0x562f56f39850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:34:18.077 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:18.077+0000 7fadf4bf0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562f56f3fcb0 0x562f56f39fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:34:18.077 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:18.077+0000 7fadf4bf0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562f56f3fcb0 0x562f56f39fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:53862/0 (socket says 172.21.15.26:53862) 2024-09-11T23:34:18.077 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:18.077+0000 7fadf4bf0640 1 -- 172.21.15.26:0/225959687 learned_addr learned my addr 172.21.15.26:0/225959687 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:34:18.077 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:18.077+0000 7fadf4bf0640 1 -- 172.21.15.26:0/225959687 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562f56f34af0 msgr2=0x562f56f48680 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T23:34:18.077 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:18.077+0000 7fadf4bf0640 1 --2- 172.21.15.26:0/225959687 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562f56f34af0 0x562f56f48680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:18.077 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:18.077+0000 7fadf4bf0640 1 -- 172.21.15.26:0/225959687 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562f56f35410 msgr2=0x562f56f39850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:34:18.077 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:18.077+0000 7fadf4bf0640 1 --2- 172.21.15.26:0/225959687 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562f56f35410 0x562f56f39850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:18.078 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:18.077+0000 7fadf4bf0640 1 -- 172.21.15.26:0/225959687 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562f56f2c2d0 con 0x562f56f3fcb0 2024-09-11T23:34:18.078 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:18.078+0000 7fadf4bf0640 1 --2- 172.21.15.26:0/225959687 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562f56f3fcb0 0x562f56f39fe0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fade8006010 tx=0x7fade80be820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3aa340807d332617 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:34:18.078 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:18.078+0000 7fadcf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-11T23:34:18.078 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:18.078+0000 7fadcf7fe640 1 -- 172.21.15.26:0/225959687 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fade80ba620 con 0x562f56f3fcb0 2024-09-11T23:34:18.079 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:18.078+0000 7fadf93f0740 1 -- 172.21.15.26:0/225959687 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fade400b390 con 0x562f56f3fcb0 2024-09-11T23:34:18.079 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:18.078+0000 7fadcf7fe640 1 -- 172.21.15.26:0/225959687 <== mon.2 v2:172.21.15.76:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 8586+0+0 (secure 0 0 0) 0x7fade802fd30 con 0x562f56f3fcb0 2024-09-11T23:34:18.079 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:18.079+0000 7fadcf7fe640 1 -- 172.21.15.26:0/225959687 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) ==== 24+0+0 (secure 0 0 0) 0x7fade80ba7c0 con 0x562f56f3fcb0 2024-09-11T23:34:18.081 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:18.082+0000 7fadf93f0740 10 client.27122 fetch_fsmap learned FSMap version 74 2024-09-11T23:34:18.081 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:18.082+0000 7fadf93f0740 1 -- 172.21.15.26:0/225959687 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=4+}) -- 0x562f56fba740 con 0x562f56f3fcb0 2024-09-11T23:34:18.085 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:18.086+0000 7fadcf7fe640 1 -- 172.21.15.26:0/225959687 <== mon.2 v2:172.21.15.76:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fade400b390 con 0x562f56f3fcb0 2024-09-11T23:34:18.085 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:18.086+0000 7fadcf7fe640 1 -- 172.21.15.26:0/225959687 <== mon.2 v2:172.21.15.76:3300/0 5 ==== fsmap(e 74) ==== 8438+0+0 (secure 0 0 0) 0x7fade80dcb60 con 0x562f56f3fcb0 2024-09-11T23:34:18.085 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:18.086+0000 7fadf93f0740 10 client.27122 fetch_fsmap finished waiting for FSMap version 74 2024-09-11T23:34:18.085 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:18.086+0000 7fadf93f0740 10 client.27122 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:34:18.085 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:18.086+0000 7fadf93f0740 20 client.27122 populate_metadata read hostname 'smithi026' 2024-09-11T23:34:18.085 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:18.086+0000 7fadf93f0740 1 --2- 172.21.15.26:0/225959687 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x562f56fbaea0 0x562f56fbd2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:18.085 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:18.086+0000 7fadf93f0740 4 client.27122 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T23:34:18.086 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:18.086+0000 7fadf93f0740 1 -- 172.21.15.26:0/225959687 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562f56ea6a20 con 0x562f56fbaea0 2024-09-11T23:34:18.093 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:18.094+0000 7fadeffff640 1 --2- 172.21.15.26:0/225959687 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x562f56fbaea0 0x562f56fbd2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:34:18.101 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:18.102+0000 7fadeffff640 1 --2- 172.21.15.26:0/225959687 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x562f56fbaea0 0x562f56fbd2c0 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7601134d2931fe31 server_cookie=c5ce4fe14086956d in_seq=0 out_seq=0 2024-09-11T23:34:18.101 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:18.102+0000 7fadcf7fe640 10 client.27122 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:34:18.175 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:18.175+0000 7fadcf7fe640 1 -- 172.21.15.26:0/225959687 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562f56ea6a20 con 0x562f56fbaea0 2024-09-11T23:34:18.175 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:18.175+0000 7fadcf7fe640 10 client.27122 handle_command_reply: tid=0 2024-09-11T23:34:18.175 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:18.175+0000 7fadf93f0740 1 client.27122 shutdown 2024-09-11T23:34:18.175 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:18.175+0000 7fadf93f0740 1 -- 172.21.15.26:0/225959687 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562f56f3fcb0 msgr2=0x562f56f39fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:34:18.175 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:18.175+0000 7fadf93f0740 1 --2- 172.21.15.26:0/225959687 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562f56f3fcb0 0x562f56f39fe0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fade8006010 tx=0x7fade80be820 comp rx=0 tx=0).stop 2024-09-11T23:34:18.175 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:18.175+0000 7fadf93f0740 1 -- 172.21.15.26:0/225959687 shutdown_connections 2024-09-11T23:34:18.175 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:18.175+0000 7fadf93f0740 1 --2- 172.21.15.26:0/225959687 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562f56f3fcb0 0x562f56f39fe0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:18.175 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:18.175+0000 7fadf93f0740 1 --2- 172.21.15.26:0/225959687 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x562f56fbaea0 0x562f56fbd2c0 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:18.175 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:18.175+0000 7fadf93f0740 1 --2- 172.21.15.26:0/225959687 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562f56f35410 0x562f56f39850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:18.176 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:18.175+0000 7fadf93f0740 1 --2- 172.21.15.26:0/225959687 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562f56f34af0 0x562f56f48680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:18.176 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:18.175+0000 7fadcf7fe640 0 client.27122 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:34:18.176 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:18.175+0000 7fadf93f0740 1 -- 172.21.15.26:0/225959687 >> 172.21.15.26:0/225959687 conn(0x562f56f31060 msgr2=0x562f56f3b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:34:18.176 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:18.175+0000 7fadf93f0740 1 -- 172.21.15.26:0/225959687 shutdown_connections 2024-09-11T23:34:18.176 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:18.175+0000 7fadf93f0740 1 -- 172.21.15.26:0/225959687 wait complete. 2024-09-11T23:34:18.176 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:18.175+0000 7fadf93f0740 20 client.27122 trim_cache size 0 max 16384 2024-09-11T23:34:18.176 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-09-11T23:34:18.176 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "message": "", 2024-09-11T23:34:18.176 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-09-11T23:34:18.176 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-09-11T23:34:18.179 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:18.179+0000 7fadf6653640 1 -- 172.21.15.26:0/866310909 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fadd80a2ef0 msgr2=0x7fadd80a53b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:34:18.179 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:18.179+0000 7fadf6653640 1 --2- 172.21.15.26:0/866310909 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fadd80a2ef0 0x7fadd80a53b0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fade0006fd0 tx=0x7fade0006bc0 comp rx=0 tx=0).stop 2024-09-11T23:34:18.180 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:18.179+0000 7fadf6653640 1 -- 172.21.15.26:0/866310909 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fadf0106be0 msgr2=0x7fadf010e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:34:18.180 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:18.179+0000 7fadf6653640 1 --2- 172.21.15.26:0/866310909 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fadf0106be0 0x7fadf010e560 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fade802ae60 tx=0x7fade8003f10 comp rx=0 tx=0).stop 2024-09-11T23:34:18.180 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:18.179+0000 7fadf6653640 1 -- 172.21.15.26:0/866310909 shutdown_connections 2024-09-11T23:34:18.180 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:18.179+0000 7fadf6653640 1 --2- 172.21.15.26:0/866310909 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fadd80a2ef0 0x7fadd80a53b0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:18.180 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:18.179+0000 7fadf6653640 1 --2- 172.21.15.26:0/866310909 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fadf0106be0 0x7fadf010e560 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:18.180 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:18.180+0000 7fadf6653640 1 --2- 172.21.15.26:0/866310909 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fadf01009e0 0x7fadf01158e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:18.180 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:18.180+0000 7fadf6653640 1 --2- 172.21.15.26:0/866310909 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fadf00fff60 0x7fadf01153a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:18.180 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:18.180+0000 7fadf6653640 1 -- 172.21.15.26:0/866310909 >> 172.21.15.26:0/866310909 conn(0x7fadf00770f0 msgr2=0x7fadf0102500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:34:18.180 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:18.180+0000 7fadf6653640 1 -- 172.21.15.26:0/866310909 shutdown_connections 2024-09-11T23:34:18.180 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:18.180+0000 7fadf6653640 1 -- 172.21.15.26:0/866310909 wait complete. 2024-09-11T23:34:18.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:34:17 smithi136 ceph-mon[35121]: pgmap v499: 97 pgs: 23 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 33 active+clean; 2.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.1 MiB/s wr, 216 op/s 2024-09-11T23:34:18.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:34:17 smithi026 ceph-mon[31172]: pgmap v499: 97 pgs: 23 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 33 active+clean; 2.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.1 MiB/s wr, 216 op/s 2024-09-11T23:34:18.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:34:17 smithi076 ceph-mon[35296]: pgmap v499: 97 pgs: 23 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 33 active+clean; 2.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.1 MiB/s wr, 216 op/s 2024-09-11T23:34:20.238 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:34:19 smithi136 ceph-mon[35121]: pgmap v500: 97 pgs: 23 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 33 active+clean; 2.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 149 op/s 2024-09-11T23:34:20.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:34:19 smithi136 ceph-mon[35121]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:34:20.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:34:19 smithi026 ceph-mon[31172]: pgmap v500: 97 pgs: 23 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 33 active+clean; 2.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 149 op/s 2024-09-11T23:34:20.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:34:19 smithi026 ceph-mon[31172]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:34:20.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:34:19 smithi076 ceph-mon[35296]: pgmap v500: 97 pgs: 23 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 33 active+clean; 2.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 149 op/s 2024-09-11T23:34:20.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:34:19 smithi076 ceph-mon[35296]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:34:22.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:34:21 smithi136 ceph-mon[35121]: pgmap v501: 97 pgs: 21 active+clean+snaptrim, 39 active+clean+snaptrim_wait, 37 active+clean; 2.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 195 op/s 2024-09-11T23:34:22.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:34:21 smithi026 ceph-mon[31172]: pgmap v501: 97 pgs: 21 active+clean+snaptrim, 39 active+clean+snaptrim_wait, 37 active+clean; 2.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 195 op/s 2024-09-11T23:34:22.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:34:21 smithi076 ceph-mon[35296]: pgmap v501: 97 pgs: 21 active+clean+snaptrim, 39 active+clean+snaptrim_wait, 37 active+clean; 2.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 195 op/s 2024-09-11T23:34:24.238 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:34:23 smithi136 ceph-mon[35121]: pgmap v502: 97 pgs: 21 active+clean+snaptrim, 39 active+clean+snaptrim_wait, 37 active+clean; 2.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 150 op/s 2024-09-11T23:34:24.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:34:23 smithi136 ceph-mon[35121]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:34:24.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:34:23 smithi026 ceph-mon[31172]: pgmap v502: 97 pgs: 21 active+clean+snaptrim, 39 active+clean+snaptrim_wait, 37 active+clean; 2.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 150 op/s 2024-09-11T23:34:24.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:34:23 smithi026 ceph-mon[31172]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:34:24.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:34:23 smithi076 ceph-mon[35296]: pgmap v502: 97 pgs: 21 active+clean+snaptrim, 39 active+clean+snaptrim_wait, 37 active+clean; 2.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 150 op/s 2024-09-11T23:34:24.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:34:23 smithi076 ceph-mon[35296]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:34:26.238 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:34:25 smithi136 ceph-mon[35121]: pgmap v503: 97 pgs: 20 active+clean+snaptrim, 37 active+clean+snaptrim_wait, 40 active+clean; 2.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 179 op/s 2024-09-11T23:34:26.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:34:25 smithi026 ceph-mon[31172]: pgmap v503: 97 pgs: 20 active+clean+snaptrim, 37 active+clean+snaptrim_wait, 40 active+clean; 2.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 179 op/s 2024-09-11T23:34:26.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:34:25 smithi076 ceph-mon[35296]: pgmap v503: 97 pgs: 20 active+clean+snaptrim, 37 active+clean+snaptrim_wait, 40 active+clean; 2.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 179 op/s 2024-09-11T23:34:28.238 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:34:27 smithi136 ceph-mon[35121]: pgmap v504: 97 pgs: 19 active+clean+snaptrim, 35 active+clean+snaptrim_wait, 43 active+clean; 2.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 171 op/s 2024-09-11T23:34:28.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:34:27 smithi026 ceph-mon[31172]: pgmap v504: 97 pgs: 19 active+clean+snaptrim, 35 active+clean+snaptrim_wait, 43 active+clean; 2.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 171 op/s 2024-09-11T23:34:28.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:34:27 smithi076 ceph-mon[35296]: pgmap v504: 97 pgs: 19 active+clean+snaptrim, 35 active+clean+snaptrim_wait, 43 active+clean; 2.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 171 op/s 2024-09-11T23:34:30.238 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:34:29 smithi136 ceph-mon[35121]: pgmap v505: 97 pgs: 19 active+clean+snaptrim, 35 active+clean+snaptrim_wait, 43 active+clean; 2.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 141 op/s 2024-09-11T23:34:30.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:34:29 smithi026 ceph-mon[31172]: pgmap v505: 97 pgs: 19 active+clean+snaptrim, 35 active+clean+snaptrim_wait, 43 active+clean; 2.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 141 op/s 2024-09-11T23:34:30.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:34:29 smithi076 ceph-mon[35296]: pgmap v505: 97 pgs: 19 active+clean+snaptrim, 35 active+clean+snaptrim_wait, 43 active+clean; 2.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 141 op/s 2024-09-11T23:34:31.303 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.303+0000 7f9370166640 1 -- 172.21.15.26:0/1856606812 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9368102470 msgr2=0x7f936810e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:34:31.304 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.303+0000 7f9370166640 1 --2- 172.21.15.26:0/1856606812 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9368102470 0x7f936810e9a0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f9360009cd0 tx=0x7f936002a8e0 comp rx=0 tx=0).stop 2024-09-11T23:34:31.304 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.303+0000 7f9370166640 1 -- 172.21.15.26:0/1856606812 shutdown_connections 2024-09-11T23:34:31.304 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.303+0000 7f9370166640 1 --2- 172.21.15.26:0/1856606812 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9368101ad0 0x7f9368101f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:31.304 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.303+0000 7f9370166640 1 --2- 172.21.15.26:0/1856606812 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9368102470 0x7f936810e9a0 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:31.304 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.303+0000 7f9370166640 1 --2- 172.21.15.26:0/1856606812 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9368107b70 0x7f9368107f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:31.304 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.303+0000 7f9370166640 1 -- 172.21.15.26:0/1856606812 >> 172.21.15.26:0/1856606812 conn(0x7f93680fd7d0 msgr2=0x7f93680ffbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:34:31.304 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.303+0000 7f9370166640 1 -- 172.21.15.26:0/1856606812 shutdown_connections 2024-09-11T23:34:31.304 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.303+0000 7f9370166640 1 -- 172.21.15.26:0/1856606812 wait complete. 2024-09-11T23:34:31.306 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.304+0000 7f9370166640 1 Processor -- start 2024-09-11T23:34:31.306 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.304+0000 7f9370166640 1 -- start start 2024-09-11T23:34:31.306 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.304+0000 7f9370166640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9368101ad0 0x7f9368198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:31.306 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.304+0000 7f9370166640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9368102470 0x7f9368198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:31.306 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.304+0000 7f9370166640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9368107b70 0x7f936819ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:31.307 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.304+0000 7f9370166640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f9368111f70 con 0x7f9368102470 2024-09-11T23:34:31.307 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.304+0000 7f9370166640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f9368111df0 con 0x7f9368101ad0 2024-09-11T23:34:31.307 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.304+0000 7f9370166640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f93681120f0 con 0x7f9368107b70 2024-09-11T23:34:31.307 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.304+0000 7f936d701640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9368102470 0x7f9368198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:34:31.307 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.304+0000 7f936d701640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9368102470 0x7f9368198940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40934/0 (socket says 172.21.15.26:40934) 2024-09-11T23:34:31.307 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.304+0000 7f936d701640 1 -- 172.21.15.26:0/3945188498 learned_addr learned my addr 172.21.15.26:0/3945188498 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:34:31.307 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.305+0000 7f936e703640 1 --2- 172.21.15.26:0/3945188498 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9368107b70 0x7f936819ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:34:31.307 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.305+0000 7f936df02640 1 --2- 172.21.15.26:0/3945188498 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9368101ad0 0x7f9368198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:34:31.307 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.305+0000 7f936d701640 1 -- 172.21.15.26:0/3945188498 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9368107b70 msgr2=0x7f936819ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:34:31.307 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.305+0000 7f936d701640 1 --2- 172.21.15.26:0/3945188498 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9368107b70 0x7f936819ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:31.308 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.305+0000 7f936d701640 1 -- 172.21.15.26:0/3945188498 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9368101ad0 msgr2=0x7f9368198400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:34:31.308 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.305+0000 7f936d701640 1 --2- 172.21.15.26:0/3945188498 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9368101ad0 0x7f9368198400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:31.308 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.305+0000 7f936d701640 1 -- 172.21.15.26:0/3945188498 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f936819d3f0 con 0x7f9368102470 2024-09-11T23:34:31.308 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.305+0000 7f936d701640 1 --2- 172.21.15.26:0/3945188498 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9368102470 0x7f9368198940 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f935800e9f0 tx=0x7f935800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:34:31.308 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.306+0000 7f935effd640 1 -- 172.21.15.26:0/3945188498 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f935800cd80 con 0x7f9368102470 2024-09-11T23:34:31.308 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.306+0000 7f9370166640 1 -- 172.21.15.26:0/3945188498 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f936819d6e0 con 0x7f9368102470 2024-09-11T23:34:31.308 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.306+0000 7f935effd640 1 -- 172.21.15.26:0/3945188498 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9358010910 con 0x7f9368102470 2024-09-11T23:34:31.308 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.306+0000 7f9370166640 1 -- 172.21.15.26:0/3945188498 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93681a4f70 con 0x7f9368102470 2024-09-11T23:34:31.309 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.306+0000 7f935effd640 1 -- 172.21.15.26:0/3945188498 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93580188f0 con 0x7f9368102470 2024-09-11T23:34:31.310 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.309+0000 7f935effd640 1 -- 172.21.15.26:0/3945188498 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f9358018bd0 con 0x7f9368102470 2024-09-11T23:34:31.310 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.309+0000 7f9372fdf740 1 Processor -- start 2024-09-11T23:34:31.310 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.310+0000 7f9372fdf740 1 -- start start 2024-09-11T23:34:31.322 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.310+0000 7f9372fdf740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55da29894990 0x55da29894d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:31.322 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.310+0000 7f9372fdf740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55da298952b0 0x55da298a3b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:31.323 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.310+0000 7f9372fdf740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55da298a40c0 0x55da298a64b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:31.323 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.310+0000 7f9372fdf740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55da297f92f0 con 0x55da298a40c0 2024-09-11T23:34:31.323 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.310+0000 7f9372fdf740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55da297a1270 con 0x55da298952b0 2024-09-11T23:34:31.323 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.310+0000 7f9372fdf740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55da297f52b0 con 0x55da29894990 2024-09-11T23:34:31.323 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.310+0000 7f936e703640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55da29894990 0x55da29894d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:34:31.323 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.310+0000 7f936e703640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55da29894990 0x55da29894d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:33850/0 (socket says 172.21.15.26:33850) 2024-09-11T23:34:31.323 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.310+0000 7f936e703640 1 -- 172.21.15.26:0/1996405512 learned_addr learned my addr 172.21.15.26:0/1996405512 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:34:31.324 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.311+0000 7f936df02640 1 --2- 172.21.15.26:0/1996405512 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55da298952b0 0x55da298a3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:34:31.324 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.311+0000 7f936d701640 1 --2- 172.21.15.26:0/1996405512 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55da298a40c0 0x55da298a64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:34:31.324 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.311+0000 7f936e703640 1 -- 172.21.15.26:0/1996405512 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55da298952b0 msgr2=0x55da298a3b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:34:31.324 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.311+0000 7f936e703640 1 --2- 172.21.15.26:0/1996405512 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55da298952b0 0x55da298a3b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:31.324 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.311+0000 7f936e703640 1 -- 172.21.15.26:0/1996405512 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55da298a40c0 msgr2=0x55da298a64b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:34:31.324 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.311+0000 7f936e703640 1 --2- 172.21.15.26:0/1996405512 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55da298a40c0 0x55da298a64b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:31.324 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.311+0000 7f936e703640 1 -- 172.21.15.26:0/1996405512 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da297a0340 con 0x55da29894990 2024-09-11T23:34:31.325 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.311+0000 7f936df02640 1 --2- 172.21.15.26:0/1996405512 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55da298952b0 0x55da298a3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:34:31.325 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.311+0000 7f936d701640 1 --2- 172.21.15.26:0/1996405512 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55da298a40c0 0x55da298a64b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:34:31.325 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.311+0000 7f935effd640 1 --2- 172.21.15.26:0/3945188498 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f93540a2ea0 0x7f93540a5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:31.325 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.311+0000 7f936e703640 1 --2- 172.21.15.26:0/1996405512 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55da29894990 0x55da29894d70 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f9360009cd0 tx=0x7f9360009a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6866a97c3669252d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:34:31.325 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.311+0000 7f935effd640 1 -- 172.21.15.26:0/3945188498 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 8586+0+0 (secure 0 0 0) 0x7f93580cf710 con 0x7f9368102470 2024-09-11T23:34:31.325 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.311+0000 7f936df02640 1 --2- 172.21.15.26:0/3945188498 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f93540a2ea0 0x7f93540a5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:34:31.326 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.312+0000 7f9353fff640 1 -- 172.21.15.26:0/1996405512 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9360049070 con 0x55da29894990 2024-09-11T23:34:31.326 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.312+0000 7f9353fff640 1 -- 172.21.15.26:0/1996405512 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9360036040 con 0x55da29894990 2024-09-11T23:34:31.326 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.312+0000 7f9353fff640 1 -- 172.21.15.26:0/1996405512 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93600042a0 con 0x55da29894990 2024-09-11T23:34:31.326 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.313+0000 7f9372fdf740 1 -- 172.21.15.26:0/1996405512 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55da29894990 msgr2=0x55da29894d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:34:31.326 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.313+0000 7f936df02640 1 --2- 172.21.15.26:0/3945188498 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f93540a2ea0 0x7f93540a5360 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x55da29891ec0 tx=0x7f9364008040 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:34:31.326 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.313+0000 7f9372fdf740 1 --2- 172.21.15.26:0/1996405512 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55da29894990 0x55da29894d70 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f9360009cd0 tx=0x7f9360009a20 comp rx=0 tx=0).stop 2024-09-11T23:34:31.326 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.313+0000 7f9372fdf740 1 -- 172.21.15.26:0/1996405512 shutdown_connections 2024-09-11T23:34:31.327 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.313+0000 7f9372fdf740 1 --2- 172.21.15.26:0/1996405512 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55da298952b0 0x55da298a3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:31.327 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.313+0000 7f9372fdf740 1 --2- 172.21.15.26:0/1996405512 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55da298a40c0 0x55da298a64b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:31.327 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.313+0000 7f9372fdf740 1 --2- 172.21.15.26:0/1996405512 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55da29894990 0x55da29894d70 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:31.327 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.313+0000 7f9372fdf740 1 -- 172.21.15.26:0/1996405512 >> 172.21.15.26:0/1996405512 conn(0x55da29891060 msgr2=0x55da29891440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:34:31.327 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.313+0000 7f9372fdf740 1 -- 172.21.15.26:0/1996405512 shutdown_connections 2024-09-11T23:34:31.327 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.313+0000 7f9372fdf740 1 -- 172.21.15.26:0/1996405512 wait complete. 2024-09-11T23:34:31.328 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.313+0000 7f9372fdf740 1 -- start start 2024-09-11T23:34:31.328 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.313+0000 7f9372fdf740 1 Processor -- start 2024-09-11T23:34:31.328 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.314+0000 7f9372fdf740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55da29894990 0x55da2989d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:31.328 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.314+0000 7f9372fdf740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55da298952b0 0x55da2989d820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:31.328 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.314+0000 7f9372fdf740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55da298a40c0 0x55da298a1b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:31.329 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.314+0000 7f9372fdf740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55da298978f0 con 0x55da298952b0 2024-09-11T23:34:31.329 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.314+0000 7f9372fdf740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55da29782290 con 0x55da29894990 2024-09-11T23:34:31.329 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.314+0000 7f9372fdf740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55da29897a70 con 0x55da298a40c0 2024-09-11T23:34:31.329 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.314+0000 7f9372fdf740 10 client.0 _ll_register_callbacks cb 0x55da297b3140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:34:31.329 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.314+0000 7f936df02640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55da29894990 0x55da2989d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:34:31.329 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.314+0000 7f936df02640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55da29894990 0x55da2989d2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:51340/0 (socket says 172.21.15.26:51340) 2024-09-11T23:34:31.330 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.314+0000 7f936df02640 1 -- 172.21.15.26:0/1321753915 learned_addr learned my addr 172.21.15.26:0/1321753915 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:34:31.330 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.314+0000 7f936d701640 1 --2- 172.21.15.26:0/1321753915 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55da298952b0 0x55da2989d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:34:31.330 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.314+0000 7f936e703640 1 --2- 172.21.15.26:0/1321753915 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55da298a40c0 0x55da298a1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:34:31.330 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.315+0000 7f936d701640 1 -- 172.21.15.26:0/1321753915 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55da298a40c0 msgr2=0x55da298a1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:34:31.330 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.315+0000 7f936d701640 1 --2- 172.21.15.26:0/1321753915 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55da298a40c0 0x55da298a1b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:31.330 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.315+0000 7f936d701640 1 -- 172.21.15.26:0/1321753915 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55da29894990 msgr2=0x55da2989d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:34:31.330 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.315+0000 7f936d701640 1 --2- 172.21.15.26:0/1321753915 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55da29894990 0x55da2989d2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:31.330 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.315+0000 7f936d701640 1 -- 172.21.15.26:0/1321753915 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da2950c210 con 0x55da298952b0 2024-09-11T23:34:31.331 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.315+0000 7f936d701640 1 --2- 172.21.15.26:0/1321753915 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55da298952b0 0x55da2989d820 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f93580ccdc0 tx=0x7f9358097ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6c1b38092dc0102 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:34:31.331 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.315+0000 7f93517fa640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:34:31.331 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.315+0000 7f93517fa640 1 -- 172.21.15.26:0/1321753915 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f935809b1e0 con 0x55da298952b0 2024-09-11T23:34:31.331 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.315+0000 7f93517fa640 1 -- 172.21.15.26:0/1321753915 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 8586+0+0 (secure 0 0 0) 0x7f9358010430 con 0x55da298952b0 2024-09-11T23:34:31.331 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.315+0000 7f9372fdf740 1 -- 172.21.15.26:0/1321753915 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9360036040 con 0x55da298952b0 2024-09-11T23:34:31.331 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.315+0000 7f93517fa640 1 -- 172.21.15.26:0/1321753915 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7f935809c290 con 0x55da298952b0 2024-09-11T23:34:31.331 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.315+0000 7f9372fdf740 10 client.17613 fetch_fsmap learned FSMap version 76 2024-09-11T23:34:31.331 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.315+0000 7f9372fdf740 1 -- 172.21.15.26:0/1321753915 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) -- 0x55da295d8620 con 0x55da298952b0 2024-09-11T23:34:31.331 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.316+0000 7f93517fa640 1 -- 172.21.15.26:0/1321753915 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f935809bb90 con 0x55da298952b0 2024-09-11T23:34:31.331 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.316+0000 7f93517fa640 1 -- 172.21.15.26:0/1321753915 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 76) ==== 8438+0+0 (secure 0 0 0) 0x7f93580cfac0 con 0x55da298952b0 2024-09-11T23:34:31.331 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.316+0000 7f9372fdf740 10 client.17613 fetch_fsmap finished waiting for FSMap version 76 2024-09-11T23:34:31.331 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.316+0000 7f9372fdf740 10 client.17613 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:34:31.331 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.316+0000 7f9372fdf740 20 client.17613 populate_metadata read hostname 'smithi026' 2024-09-11T23:34:31.332 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.316+0000 7f9372fdf740 1 --2- 172.21.15.26:0/1321753915 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55da298b8660 0x55da298baa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:31.332 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.316+0000 7f9372fdf740 4 client.17613 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:34:31.332 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.316+0000 7f9372fdf740 1 -- 172.21.15.26:0/1321753915 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f9360036040 con 0x55da298b8660 2024-09-11T23:34:31.332 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.316+0000 7f936df02640 1 --2- 172.21.15.26:0/1321753915 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55da298b8660 0x55da298baa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:34:31.332 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.317+0000 7f93517fa640 10 client.17613 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:34:31.332 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.317+0000 7f936df02640 1 --2- 172.21.15.26:0/1321753915 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55da298b8660 0x55da298baa80 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=10804d226c8f460d server_cookie=7a4a6bcbeeae52a7 in_seq=0 out_seq=0 2024-09-11T23:34:31.332 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.317+0000 7f93517fa640 1 -- 172.21.15.26:0/1321753915 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f9360036040 con 0x55da298b8660 2024-09-11T23:34:31.332 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.317+0000 7f93517fa640 10 client.17613 handle_command_reply: tid=0 2024-09-11T23:34:31.332 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.318+0000 7f9372fdf740 1 client.17613 shutdown 2024-09-11T23:34:31.333 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.318+0000 7f9372fdf740 1 -- 172.21.15.26:0/1321753915 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55da298952b0 msgr2=0x55da2989d820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:34:31.333 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.318+0000 7f9372fdf740 1 --2- 172.21.15.26:0/1321753915 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55da298952b0 0x55da2989d820 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f93580ccdc0 tx=0x7f9358097ef0 comp rx=0 tx=0).stop 2024-09-11T23:34:31.333 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.318+0000 7f9372fdf740 1 -- 172.21.15.26:0/1321753915 shutdown_connections 2024-09-11T23:34:31.333 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.318+0000 7f9372fdf740 1 --2- 172.21.15.26:0/1321753915 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55da298b8660 0x55da298baa80 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:31.333 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.318+0000 7f9372fdf740 1 --2- 172.21.15.26:0/1321753915 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55da298a40c0 0x55da298a1b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:31.333 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.318+0000 7f9372fdf740 1 --2- 172.21.15.26:0/1321753915 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55da298952b0 0x55da2989d820 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:31.333 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.318+0000 7f9372fdf740 1 --2- 172.21.15.26:0/1321753915 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55da29894990 0x55da2989d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:31.333 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.318+0000 7f9372fdf740 1 -- 172.21.15.26:0/1321753915 >> 172.21.15.26:0/1321753915 conn(0x55da29891060 msgr2=0x55da29896d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:34:31.333 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.318+0000 7f93517fa640 0 client.17613 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:34:31.333 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.318+0000 7f9372fdf740 1 -- 172.21.15.26:0/1321753915 shutdown_connections 2024-09-11T23:34:31.333 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.318+0000 7f9372fdf740 1 -- 172.21.15.26:0/1321753915 wait complete. 2024-09-11T23:34:31.333 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.318+0000 7f9372fdf740 20 client.17613 trim_cache size 0 max 16384 2024-09-11T23:34:31.344 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.342+0000 7f9372fdf740 1 Processor -- start 2024-09-11T23:34:31.344 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.342+0000 7f9372fdf740 1 -- start start 2024-09-11T23:34:31.344 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.343+0000 7f9372fdf740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55da29894990 0x55da298a3b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:31.344 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.343+0000 7f9372fdf740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55da298952b0 0x55da298a1b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:31.344 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.343+0000 7f9372fdf740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55da298a40c0 0x55da298c0310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:31.344 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.343+0000 7f9372fdf740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f934c00e870 con 0x55da298a40c0 2024-09-11T23:34:31.345 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.343+0000 7f9372fdf740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f934c010680 con 0x55da29894990 2024-09-11T23:34:31.345 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.343+0000 7f9372fdf740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f934c00f000 con 0x55da298952b0 2024-09-11T23:34:31.345 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.343+0000 7f936df02640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55da29894990 0x55da298a3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:34:31.345 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.343+0000 7f936e703640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55da298a40c0 0x55da298c0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:34:31.345 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.343+0000 7f936df02640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55da29894990 0x55da298a3b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:51342/0 (socket says 172.21.15.26:51342) 2024-09-11T23:34:31.345 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.343+0000 7f936df02640 1 -- 172.21.15.26:0/2792806567 learned_addr learned my addr 172.21.15.26:0/2792806567 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:34:31.345 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.343+0000 7f936d701640 1 --2- 172.21.15.26:0/2792806567 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55da298952b0 0x55da298a1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:34:31.345 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.343+0000 7f936df02640 1 -- 172.21.15.26:0/2792806567 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55da298952b0 msgr2=0x55da298a1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:34:31.345 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.343+0000 7f936df02640 1 --2- 172.21.15.26:0/2792806567 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55da298952b0 0x55da298a1b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:31.345 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.343+0000 7f936df02640 1 -- 172.21.15.26:0/2792806567 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55da298a40c0 msgr2=0x55da298c0310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:34:31.345 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.343+0000 7f936df02640 1 --2- 172.21.15.26:0/2792806567 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55da298a40c0 0x55da298c0310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:31.345 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.343+0000 7f936df02640 1 -- 172.21.15.26:0/2792806567 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da298a6820 con 0x55da29894990 2024-09-11T23:34:31.345 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.343+0000 7f936df02640 1 --2- 172.21.15.26:0/2792806567 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55da29894990 0x55da298a3b20 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f936400f5b0 tx=0x7f9364012000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cc5b046925946375 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:34:31.346 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.344+0000 7f93537fe640 1 -- 172.21.15.26:0/2792806567 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f936401b040 con 0x55da29894990 2024-09-11T23:34:31.346 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.344+0000 7f93537fe640 1 -- 172.21.15.26:0/2792806567 <== mon.2 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f936401c040 con 0x55da29894990 2024-09-11T23:34:31.346 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.344+0000 7f93537fe640 1 -- 172.21.15.26:0/2792806567 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9364015670 con 0x55da29894990 2024-09-11T23:34:31.348 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.345+0000 7f9372fdf740 1 -- 172.21.15.26:0/2792806567 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55da29894990 msgr2=0x55da298a3b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:34:31.348 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.345+0000 7f9372fdf740 1 --2- 172.21.15.26:0/2792806567 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55da29894990 0x55da298a3b20 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f936400f5b0 tx=0x7f9364012000 comp rx=0 tx=0).stop 2024-09-11T23:34:31.348 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.345+0000 7f9372fdf740 1 -- 172.21.15.26:0/2792806567 shutdown_connections 2024-09-11T23:34:31.348 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.345+0000 7f9372fdf740 1 --2- 172.21.15.26:0/2792806567 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55da298a40c0 0x55da298c0310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:31.348 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.345+0000 7f9372fdf740 1 --2- 172.21.15.26:0/2792806567 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55da298952b0 0x55da298a1b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:31.348 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.345+0000 7f9372fdf740 1 --2- 172.21.15.26:0/2792806567 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55da29894990 0x55da298a3b20 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:31.348 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.345+0000 7f9372fdf740 1 -- 172.21.15.26:0/2792806567 >> 172.21.15.26:0/2792806567 conn(0x55da29891060 msgr2=0x55da298901c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:34:31.348 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.345+0000 7f9372fdf740 1 -- 172.21.15.26:0/2792806567 shutdown_connections 2024-09-11T23:34:31.348 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.345+0000 7f9372fdf740 1 -- 172.21.15.26:0/2792806567 wait complete. 2024-09-11T23:34:31.348 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.345+0000 7f9372fdf740 1 -- start start 2024-09-11T23:34:31.349 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.345+0000 7f9372fdf740 1 Processor -- start 2024-09-11T23:34:31.349 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.346+0000 7f9372fdf740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55da29894990 0x55da298b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:31.349 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.346+0000 7f9372fdf740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55da298952b0 0x55da298b1810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:31.349 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.346+0000 7f9372fdf740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55da298a40c0 0x55da298b1d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:31.349 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.346+0000 7f9372fdf740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f934c00fef0 con 0x55da298a40c0 2024-09-11T23:34:31.349 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.346+0000 7f9372fdf740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f934c00f760 con 0x55da298952b0 2024-09-11T23:34:31.349 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.346+0000 7f9372fdf740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f934c00c870 con 0x55da29894990 2024-09-11T23:34:31.349 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.346+0000 7f9372fdf740 10 client.0 _ll_register_callbacks cb 0x7f9358097710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:34:31.349 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.346+0000 7f936d701640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55da298952b0 0x55da298b1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:34:31.349 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.346+0000 7f936e703640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55da298a40c0 0x55da298b1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:34:31.349 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.346+0000 7f936d701640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55da298952b0 0x55da298b1810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:51358/0 (socket says 172.21.15.26:51358) 2024-09-11T23:34:31.349 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.346+0000 7f936d701640 1 -- 172.21.15.26:0/1282482386 learned_addr learned my addr 172.21.15.26:0/1282482386 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:34:31.349 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.346+0000 7f936e703640 1 --2- 172.21.15.26:0/1282482386 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55da298a40c0 0x55da298b1d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40990/0 (socket says 172.21.15.26:40990) 2024-09-11T23:34:31.350 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.346+0000 7f936df02640 1 --2- 172.21.15.26:0/1282482386 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55da29894990 0x55da298b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:34:31.350 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.347+0000 7f936e703640 1 -- 172.21.15.26:0/1282482386 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55da29894990 msgr2=0x55da298b12d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:34:31.350 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.347+0000 7f936e703640 1 --2- 172.21.15.26:0/1282482386 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55da29894990 0x55da298b12d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:31.350 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.347+0000 7f936e703640 1 -- 172.21.15.26:0/1282482386 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55da298952b0 msgr2=0x55da298b1810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:34:31.350 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.347+0000 7f936e703640 1 --2- 172.21.15.26:0/1282482386 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55da298952b0 0x55da298b1810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:31.350 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.347+0000 7f936e703640 1 -- 172.21.15.26:0/1282482386 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da299222e0 con 0x55da298a40c0 2024-09-11T23:34:31.350 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.347+0000 7f936e703640 1 --2- 172.21.15.26:0/1282482386 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55da298a40c0 0x55da298b1d50 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f936002ae00 tx=0x7f9360003cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a650310a9c494452 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:34:31.350 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.347+0000 7f93517fa640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:34:31.350 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.347+0000 7f93517fa640 1 -- 172.21.15.26:0/1282482386 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9360054070 con 0x55da298a40c0 2024-09-11T23:34:31.350 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.347+0000 7f9372fdf740 1 -- 172.21.15.26:0/1282482386 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f936401c040 con 0x55da298a40c0 2024-09-11T23:34:31.350 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.347+0000 7f93517fa640 1 -- 172.21.15.26:0/1282482386 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 8586+0+0 (secure 0 0 0) 0x7f93600318c0 con 0x55da298a40c0 2024-09-11T23:34:31.351 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.347+0000 7f93517fa640 1 -- 172.21.15.26:0/1282482386 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7f9360060050 con 0x55da298a40c0 2024-09-11T23:34:31.351 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.347+0000 7f9372fdf740 10 client.17619 fetch_fsmap learned FSMap version 76 2024-09-11T23:34:31.351 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.347+0000 7f9372fdf740 1 -- 172.21.15.26:0/1282482386 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) -- 0x55da299233f0 con 0x55da298a40c0 2024-09-11T23:34:31.352 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.348+0000 7f93517fa640 1 -- 172.21.15.26:0/1282482386 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f936401c040 con 0x55da298a40c0 2024-09-11T23:34:31.352 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.348+0000 7f93517fa640 1 -- 172.21.15.26:0/1282482386 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 76) ==== 8438+0+0 (secure 0 0 0) 0x7f9360030070 con 0x55da298a40c0 2024-09-11T23:34:31.352 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.348+0000 7f9372fdf740 10 client.17619 fetch_fsmap finished waiting for FSMap version 76 2024-09-11T23:34:31.352 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.348+0000 7f9372fdf740 10 client.17619 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:34:31.352 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.348+0000 7f9372fdf740 20 client.17619 populate_metadata read hostname 'smithi026' 2024-09-11T23:34:31.352 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.348+0000 7f9372fdf740 1 --2- 172.21.15.26:0/1282482386 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55da29923720 0x55da29925b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:31.352 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.348+0000 7f9372fdf740 4 client.17619 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T23:34:31.352 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.348+0000 7f9372fdf740 1 -- 172.21.15.26:0/1282482386 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55da29890930 con 0x55da29923720 2024-09-11T23:34:31.352 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.348+0000 7f936df02640 1 --2- 172.21.15.26:0/1282482386 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55da29923720 0x55da29925b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:34:31.353 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.349+0000 7f93517fa640 10 client.17619 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:34:31.353 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.349+0000 7f936df02640 1 --2- 172.21.15.26:0/1282482386 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55da29923720 0x55da29925b10 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e52c2a9fd897add0 server_cookie=4688ec6cf4916d75 in_seq=0 out_seq=0 2024-09-11T23:34:31.407 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.407+0000 7f93517fa640 1 -- 172.21.15.26:0/1282482386 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55da29890930 con 0x55da29923720 2024-09-11T23:34:31.408 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.407+0000 7f93517fa640 10 client.17619 handle_command_reply: tid=0 2024-09-11T23:34:31.412 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.409+0000 7f9372fdf740 1 client.17619 shutdown 2024-09-11T23:34:31.412 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.409+0000 7f9372fdf740 1 -- 172.21.15.26:0/1282482386 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55da298a40c0 msgr2=0x55da298b1d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:34:31.413 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.409+0000 7f9372fdf740 1 --2- 172.21.15.26:0/1282482386 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55da298a40c0 0x55da298b1d50 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f936002ae00 tx=0x7f9360003cb0 comp rx=0 tx=0).stop 2024-09-11T23:34:31.413 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.410+0000 7f9372fdf740 1 -- 172.21.15.26:0/1282482386 shutdown_connections 2024-09-11T23:34:31.413 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.410+0000 7f9372fdf740 1 --2- 172.21.15.26:0/1282482386 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55da298952b0 0x55da298b1810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:31.413 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.410+0000 7f9372fdf740 1 --2- 172.21.15.26:0/1282482386 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55da29923720 0x55da29925b10 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:31.413 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.410+0000 7f9372fdf740 1 --2- 172.21.15.26:0/1282482386 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55da298a40c0 0x55da298b1d50 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:31.414 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.410+0000 7f93517fa640 0 client.17619 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:34:31.414 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.410+0000 7f9372fdf740 1 --2- 172.21.15.26:0/1282482386 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55da29894990 0x55da298b12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:31.414 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.410+0000 7f9372fdf740 1 -- 172.21.15.26:0/1282482386 >> 172.21.15.26:0/1282482386 conn(0x55da29891060 msgr2=0x55da2989cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:34:31.414 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.411+0000 7f9372fdf740 1 -- 172.21.15.26:0/1282482386 shutdown_connections 2024-09-11T23:34:31.414 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.411+0000 7f9372fdf740 1 -- 172.21.15.26:0/1282482386 wait complete. 2024-09-11T23:34:31.414 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.412+0000 7f9372fdf740 20 client.17619 trim_cache size 0 max 16384 2024-09-11T23:34:31.415 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-09-11T23:34:31.415 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "message": "", 2024-09-11T23:34:31.415 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-09-11T23:34:31.415 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-09-11T23:34:31.416 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.412+0000 7f9370166640 1 -- 172.21.15.26:0/3945188498 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f93540a2ea0 msgr2=0x7f93540a5360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:34:31.416 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.412+0000 7f9370166640 1 --2- 172.21.15.26:0/3945188498 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f93540a2ea0 0x7f93540a5360 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x55da29891ec0 tx=0x7f9364008040 comp rx=0 tx=0).stop 2024-09-11T23:34:31.416 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.412+0000 7f9370166640 1 -- 172.21.15.26:0/3945188498 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9368102470 msgr2=0x7f9368198940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:34:31.416 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.412+0000 7f9370166640 1 --2- 172.21.15.26:0/3945188498 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9368102470 0x7f9368198940 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f935800e9f0 tx=0x7f935800eeb0 comp rx=0 tx=0).stop 2024-09-11T23:34:31.417 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.413+0000 7f9370166640 1 -- 172.21.15.26:0/3945188498 shutdown_connections 2024-09-11T23:34:31.417 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.413+0000 7f9370166640 1 --2- 172.21.15.26:0/3945188498 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f93540a2ea0 0x7f93540a5360 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:31.417 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.413+0000 7f9370166640 1 --2- 172.21.15.26:0/3945188498 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9368107b70 0x7f936819ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:31.417 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.413+0000 7f9370166640 1 --2- 172.21.15.26:0/3945188498 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9368102470 0x7f9368198940 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:31.417 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.413+0000 7f9370166640 1 --2- 172.21.15.26:0/3945188498 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9368101ad0 0x7f9368198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:31.417 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.413+0000 7f9370166640 1 -- 172.21.15.26:0/3945188498 >> 172.21.15.26:0/3945188498 conn(0x7f93680fd7d0 msgr2=0x7f93680fdc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:34:31.417 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.413+0000 7f9370166640 1 -- 172.21.15.26:0/3945188498 shutdown_connections 2024-09-11T23:34:31.417 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:31.413+0000 7f9370166640 1 -- 172.21.15.26:0/3945188498 wait complete. 2024-09-11T23:34:32.238 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:34:31 smithi136 ceph-mon[35121]: pgmap v506: 97 pgs: 12 active+clean+snaptrim, 25 active+clean+snaptrim_wait, 60 active+clean; 2.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.0 MiB/s wr, 198 op/s 2024-09-11T23:34:32.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:34:31 smithi026 ceph-mon[31172]: pgmap v506: 97 pgs: 12 active+clean+snaptrim, 25 active+clean+snaptrim_wait, 60 active+clean; 2.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.0 MiB/s wr, 198 op/s 2024-09-11T23:34:32.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:34:31 smithi076 ceph-mon[35296]: pgmap v506: 97 pgs: 12 active+clean+snaptrim, 25 active+clean+snaptrim_wait, 60 active+clean; 2.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.0 MiB/s wr, 198 op/s 2024-09-11T23:34:34.238 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:34:33 smithi136 ceph-mon[35121]: pgmap v507: 97 pgs: 12 active+clean+snaptrim, 25 active+clean+snaptrim_wait, 60 active+clean; 2.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 162 op/s 2024-09-11T23:34:34.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:34:33 smithi026 ceph-mon[31172]: pgmap v507: 97 pgs: 12 active+clean+snaptrim, 25 active+clean+snaptrim_wait, 60 active+clean; 2.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 162 op/s 2024-09-11T23:34:34.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:34:33 smithi076 ceph-mon[35296]: pgmap v507: 97 pgs: 12 active+clean+snaptrim, 25 active+clean+snaptrim_wait, 60 active+clean; 2.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 162 op/s 2024-09-11T23:34:36.238 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:34:35 smithi136 ceph-mon[35121]: pgmap v508: 97 pgs: 8 active+clean+snaptrim, 22 active+clean+snaptrim_wait, 67 active+clean; 2.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 192 op/s 2024-09-11T23:34:36.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:34:35 smithi026 ceph-mon[31172]: pgmap v508: 97 pgs: 8 active+clean+snaptrim, 22 active+clean+snaptrim_wait, 67 active+clean; 2.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 192 op/s 2024-09-11T23:34:36.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:34:35 smithi076 ceph-mon[35296]: pgmap v508: 97 pgs: 8 active+clean+snaptrim, 22 active+clean+snaptrim_wait, 67 active+clean; 2.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 192 op/s 2024-09-11T23:34:38.238 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:34:37 smithi136 ceph-mon[35121]: pgmap v509: 97 pgs: 7 active+clean+snaptrim, 16 active+clean+snaptrim_wait, 74 active+clean; 2.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.8 MiB/s wr, 197 op/s 2024-09-11T23:34:38.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:34:37 smithi026 ceph-mon[31172]: pgmap v509: 97 pgs: 7 active+clean+snaptrim, 16 active+clean+snaptrim_wait, 74 active+clean; 2.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.8 MiB/s wr, 197 op/s 2024-09-11T23:34:38.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:34:37 smithi076 ceph-mon[35296]: pgmap v509: 97 pgs: 7 active+clean+snaptrim, 16 active+clean+snaptrim_wait, 74 active+clean; 2.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.8 MiB/s wr, 197 op/s 2024-09-11T23:34:40.238 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:34:39 smithi136 ceph-mon[35121]: pgmap v510: 97 pgs: 7 active+clean+snaptrim, 16 active+clean+snaptrim_wait, 74 active+clean; 2.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 168 op/s 2024-09-11T23:34:40.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:34:39 smithi026 ceph-mon[31172]: pgmap v510: 97 pgs: 7 active+clean+snaptrim, 16 active+clean+snaptrim_wait, 74 active+clean; 2.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 168 op/s 2024-09-11T23:34:40.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:34:39 smithi076 ceph-mon[35296]: pgmap v510: 97 pgs: 7 active+clean+snaptrim, 16 active+clean+snaptrim_wait, 74 active+clean; 2.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 168 op/s 2024-09-11T23:34:42.238 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:34:41 smithi136 ceph-mon[35121]: pgmap v511: 97 pgs: 4 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 88 active+clean; 2.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.6 MiB/s wr, 244 op/s 2024-09-11T23:34:42.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:34:41 smithi026 ceph-mon[31172]: pgmap v511: 97 pgs: 4 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 88 active+clean; 2.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.6 MiB/s wr, 244 op/s 2024-09-11T23:34:42.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:34:41 smithi076 ceph-mon[35296]: pgmap v511: 97 pgs: 4 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 88 active+clean; 2.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.6 MiB/s wr, 244 op/s 2024-09-11T23:34:44.238 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:34:43 smithi136 ceph-mon[35121]: pgmap v512: 97 pgs: 1 active+clean+snaptrim, 96 active+clean; 2.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 205 op/s 2024-09-11T23:34:44.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:34:43 smithi026 ceph-mon[31172]: pgmap v512: 97 pgs: 1 active+clean+snaptrim, 96 active+clean; 2.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 205 op/s 2024-09-11T23:34:44.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:34:43 smithi076 ceph-mon[35296]: pgmap v512: 97 pgs: 1 active+clean+snaptrim, 96 active+clean; 2.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 205 op/s 2024-09-11T23:34:44.508 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.507+0000 7fda4f4d2640 1 -- 172.21.15.26:0/2532296569 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fda48106a00 msgr2=0x7fda48106de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:34:44.509 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.508+0000 7fda4f4d2640 1 --2- 172.21.15.26:0/2532296569 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fda48106a00 0x7fda48106de0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7fda44009930 tx=0x7fda4402a420 comp rx=0 tx=0).stop 2024-09-11T23:34:44.509 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.508+0000 7fda4f4d2640 1 -- 172.21.15.26:0/2532296569 shutdown_connections 2024-09-11T23:34:44.509 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.508+0000 7fda4f4d2640 1 --2- 172.21.15.26:0/2532296569 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fda481009f0 0x7fda48100e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:44.509 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.508+0000 7fda4f4d2640 1 --2- 172.21.15.26:0/2532296569 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fda48101390 0x7fda4810d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:44.509 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.508+0000 7fda4f4d2640 1 --2- 172.21.15.26:0/2532296569 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fda48106a00 0x7fda48106de0 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:44.509 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.508+0000 7fda4f4d2640 1 -- 172.21.15.26:0/2532296569 >> 172.21.15.26:0/2532296569 conn(0x7fda480fc6b0 msgr2=0x7fda480fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:34:44.509 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.508+0000 7fda4f4d2640 1 -- 172.21.15.26:0/2532296569 shutdown_connections 2024-09-11T23:34:44.509 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.508+0000 7fda4f4d2640 1 -- 172.21.15.26:0/2532296569 wait complete. 2024-09-11T23:34:44.511 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.508+0000 7fda4f4d2640 1 Processor -- start 2024-09-11T23:34:44.511 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.508+0000 7fda4f4d2640 1 -- start start 2024-09-11T23:34:44.511 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.509+0000 7fda4f4d2640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fda481009f0 0x7fda480fcca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:44.511 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.509+0000 7fda4f4d2640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fda48101390 0x7fda480fd1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:44.511 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.509+0000 7fda4f4d2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fda48106a00 0x7fda480fd720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:44.511 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.509+0000 7fda4f4d2640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fda48110e00 con 0x7fda48106a00 2024-09-11T23:34:44.512 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.509+0000 7fda4f4d2640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fda48110c80 con 0x7fda48101390 2024-09-11T23:34:44.512 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.509+0000 7fda4f4d2640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fda48110f80 con 0x7fda481009f0 2024-09-11T23:34:44.512 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.509+0000 7fda4da6f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fda48106a00 0x7fda480fd720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:34:44.512 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.509+0000 7fda4da6f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fda48106a00 0x7fda480fd720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58514/0 (socket says 172.21.15.26:58514) 2024-09-11T23:34:44.512 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.509+0000 7fda4da6f640 1 -- 172.21.15.26:0/3486192611 learned_addr learned my addr 172.21.15.26:0/3486192611 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:34:44.512 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.509+0000 7fda4d26e640 1 --2- 172.21.15.26:0/3486192611 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fda481009f0 0x7fda480fcca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:34:44.512 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.509+0000 7fda4ca6d640 1 --2- 172.21.15.26:0/3486192611 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fda48101390 0x7fda480fd1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:34:44.512 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.509+0000 7fda4da6f640 1 -- 172.21.15.26:0/3486192611 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fda481009f0 msgr2=0x7fda480fcca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:34:44.512 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.509+0000 7fda4da6f640 1 --2- 172.21.15.26:0/3486192611 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fda481009f0 0x7fda480fcca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:44.512 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.509+0000 7fda4da6f640 1 -- 172.21.15.26:0/3486192611 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fda48101390 msgr2=0x7fda480fd1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:34:44.512 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.509+0000 7fda4da6f640 1 --2- 172.21.15.26:0/3486192611 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fda48101390 0x7fda480fd1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:44.512 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.509+0000 7fda4da6f640 1 -- 172.21.15.26:0/3486192611 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda481a6910 con 0x7fda48106a00 2024-09-11T23:34:44.513 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.509+0000 7fda4da6f640 1 --2- 172.21.15.26:0/3486192611 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fda48106a00 0x7fda480fd720 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7fda40009fd0 tx=0x7fda4000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:34:44.513 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.510+0000 7fda3e7fc640 1 -- 172.21.15.26:0/3486192611 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda40017070 con 0x7fda48106a00 2024-09-11T23:34:44.513 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.510+0000 7fda4f4d2640 1 -- 172.21.15.26:0/3486192611 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda481a6c00 con 0x7fda48106a00 2024-09-11T23:34:44.513 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.510+0000 7fda3e7fc640 1 -- 172.21.15.26:0/3486192611 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda40003e50 con 0x7fda48106a00 2024-09-11T23:34:44.513 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.510+0000 7fda3e7fc640 1 -- 172.21.15.26:0/3486192611 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda400106c0 con 0x7fda48106a00 2024-09-11T23:34:44.513 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.510+0000 7fda4f4d2640 1 -- 172.21.15.26:0/3486192611 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda481a7150 con 0x7fda48106a00 2024-09-11T23:34:44.518 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.512+0000 7fda3e7fc640 1 -- 172.21.15.26:0/3486192611 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fda40010860 con 0x7fda48106a00 2024-09-11T23:34:44.518 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.512+0000 7fda52264740 1 Processor -- start 2024-09-11T23:34:44.518 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.513+0000 7fda52264740 1 -- start start 2024-09-11T23:34:44.519 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.513+0000 7fda52264740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e80ed5aaf0 0x55e80ed5aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:44.519 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.513+0000 7fda52264740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e80ed5b410 0x55e80ed65770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:44.519 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.513+0000 7fda52264740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e80ed65cb0 0x55e80ed680a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:44.519 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.513+0000 7fda52264740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55e80ecbf2f0 con 0x55e80ed5b410 2024-09-11T23:34:44.519 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.513+0000 7fda52264740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55e80ec67270 con 0x55e80ed65cb0 2024-09-11T23:34:44.519 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.513+0000 7fda52264740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55e80ecbb2b0 con 0x55e80ed5aaf0 2024-09-11T23:34:44.519 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.513+0000 7fda4ca6d640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e80ed5aaf0 0x55e80ed5aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:34:44.519 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.513+0000 7fda4d26e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e80ed5b410 0x55e80ed65770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:34:44.519 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.513+0000 7fda4ca6d640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e80ed5aaf0 0x55e80ed5aed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:47856/0 (socket says 172.21.15.26:47856) 2024-09-11T23:34:44.519 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.513+0000 7fda4ca6d640 1 -- 172.21.15.26:0/2919995774 learned_addr learned my addr 172.21.15.26:0/2919995774 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:34:44.519 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.513+0000 7fda4d26e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e80ed5b410 0x55e80ed65770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58520/0 (socket says 172.21.15.26:58520) 2024-09-11T23:34:44.519 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.513+0000 7fda4ca6d640 1 --2- 172.21.15.26:0/2919995774 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e80ed65cb0 0x55e80ed680a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:34:44.520 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.513+0000 7fda4ca6d640 1 -- 172.21.15.26:0/2919995774 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e80ed65cb0 msgr2=0x55e80ed680a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:34:44.520 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.513+0000 7fda4ca6d640 1 --2- 172.21.15.26:0/2919995774 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e80ed65cb0 0x55e80ed680a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:44.520 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.513+0000 7fda4ca6d640 1 -- 172.21.15.26:0/2919995774 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e80ed5b410 msgr2=0x55e80ed65770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:34:44.520 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.513+0000 7fda4ca6d640 1 --2- 172.21.15.26:0/2919995774 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e80ed5b410 0x55e80ed65770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:44.520 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.513+0000 7fda3e7fc640 1 --2- 172.21.15.26:0/3486192611 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fda340a2da0 0x7fda340a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:44.520 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.513+0000 7fda4ca6d640 1 -- 172.21.15.26:0/2919995774 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e80ec66340 con 0x55e80ed5aaf0 2024-09-11T23:34:44.520 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.513+0000 7fda4d26e640 1 --2- 172.21.15.26:0/2919995774 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e80ed5b410 0x55e80ed65770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:34:44.520 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.513+0000 7fda4d26e640 1 --2- 172.21.15.26:0/3486192611 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fda340a2da0 0x7fda340a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:34:44.520 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.514+0000 7fda4ca6d640 1 --2- 172.21.15.26:0/2919995774 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e80ed5aaf0 0x55e80ed5aed0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fda380125a0 tx=0x7fda38005c90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1fc8194714c7476c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:34:44.520 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.514+0000 7fda3e7fc640 1 -- 172.21.15.26:0/3486192611 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 8586+0+0 (secure 0 0 0) 0x7fda400cf180 con 0x7fda48106a00 2024-09-11T23:34:44.520 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.514+0000 7fda337fe640 1 -- 172.21.15.26:0/2919995774 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda38010420 con 0x55e80ed5aaf0 2024-09-11T23:34:44.520 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.514+0000 7fda337fe640 1 -- 172.21.15.26:0/2919995774 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda38010aa0 con 0x55e80ed5aaf0 2024-09-11T23:34:44.520 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.514+0000 7fda337fe640 1 -- 172.21.15.26:0/2919995774 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda3800e940 con 0x55e80ed5aaf0 2024-09-11T23:34:44.521 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.514+0000 7fda52264740 1 -- 172.21.15.26:0/2919995774 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e80ed5aaf0 msgr2=0x55e80ed5aed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:34:44.521 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.514+0000 7fda4d26e640 1 --2- 172.21.15.26:0/3486192611 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fda340a2da0 0x7fda340a5260 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fda44009900 tx=0x7fda440058d0 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:34:44.521 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.514+0000 7fda52264740 1 --2- 172.21.15.26:0/2919995774 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e80ed5aaf0 0x55e80ed5aed0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fda380125a0 tx=0x7fda38005c90 comp rx=0 tx=0).stop 2024-09-11T23:34:44.521 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.515+0000 7fda52264740 1 -- 172.21.15.26:0/2919995774 shutdown_connections 2024-09-11T23:34:44.521 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.515+0000 7fda52264740 1 --2- 172.21.15.26:0/2919995774 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e80ed65cb0 0x55e80ed680a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:44.521 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.515+0000 7fda52264740 1 --2- 172.21.15.26:0/2919995774 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e80ed5b410 0x55e80ed65770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:44.521 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.515+0000 7fda52264740 1 --2- 172.21.15.26:0/2919995774 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e80ed5aaf0 0x55e80ed5aed0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:44.521 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.515+0000 7fda52264740 1 -- 172.21.15.26:0/2919995774 >> 172.21.15.26:0/2919995774 conn(0x55e80ed57060 msgr2=0x55e80ed57440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:34:44.521 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.515+0000 7fda52264740 1 -- 172.21.15.26:0/2919995774 shutdown_connections 2024-09-11T23:34:44.521 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.515+0000 7fda52264740 1 -- 172.21.15.26:0/2919995774 wait complete. 2024-09-11T23:34:44.521 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.515+0000 7fda52264740 1 -- start start 2024-09-11T23:34:44.521 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.515+0000 7fda52264740 1 Processor -- start 2024-09-11T23:34:44.522 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.515+0000 7fda52264740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e80ed5aaf0 0x55e80ed62b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:44.522 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.515+0000 7fda52264740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e80ed5b410 0x55e80ed63040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:44.522 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.515+0000 7fda52264740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e80ed65cb0 0x55e80ed63580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:44.522 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.515+0000 7fda52264740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55e80ed68880 con 0x55e80ed5b410 2024-09-11T23:34:44.522 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.515+0000 7fda52264740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55e80ec48290 con 0x55e80ed5aaf0 2024-09-11T23:34:44.522 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.515+0000 7fda52264740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55e80ed68a00 con 0x55e80ed65cb0 2024-09-11T23:34:44.522 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.515+0000 7fda52264740 10 client.0 _ll_register_callbacks cb 0x55e80ec79140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:34:44.522 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.516+0000 7fda4ca6d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e80ed5b410 0x55e80ed63040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:34:44.522 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.516+0000 7fda4da6f640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e80ed65cb0 0x55e80ed63580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:34:44.522 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.516+0000 7fda4ca6d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e80ed5b410 0x55e80ed63040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58524/0 (socket says 172.21.15.26:58524) 2024-09-11T23:34:44.522 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.516+0000 7fda4ca6d640 1 -- 172.21.15.26:0/3095633010 learned_addr learned my addr 172.21.15.26:0/3095633010 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:34:44.522 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.516+0000 7fda4d26e640 1 --2- 172.21.15.26:0/3095633010 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e80ed5aaf0 0x55e80ed62b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:34:44.522 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.516+0000 7fda4da6f640 1 -- 172.21.15.26:0/3095633010 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e80ed5aaf0 msgr2=0x55e80ed62b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:34:44.523 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.516+0000 7fda4da6f640 1 --2- 172.21.15.26:0/3095633010 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e80ed5aaf0 0x55e80ed62b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:44.523 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.516+0000 7fda4da6f640 1 -- 172.21.15.26:0/3095633010 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e80ed5b410 msgr2=0x55e80ed63040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:34:44.523 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.516+0000 7fda4da6f640 1 --2- 172.21.15.26:0/3095633010 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e80ed5b410 0x55e80ed63040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:44.523 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.516+0000 7fda4da6f640 1 -- 172.21.15.26:0/3095633010 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e80e9d2210 con 0x55e80ed65cb0 2024-09-11T23:34:44.523 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.516+0000 7fda4d26e640 1 --2- 172.21.15.26:0/3095633010 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e80ed5aaf0 0x55e80ed62b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:34:44.523 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.516+0000 7fda4ca6d640 1 --2- 172.21.15.26:0/3095633010 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e80ed5b410 0x55e80ed63040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:34:44.523 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.516+0000 7fda4da6f640 1 --2- 172.21.15.26:0/3095633010 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e80ed65cb0 0x55e80ed63580 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fda400cc830 tx=0x7fda4009b570 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c83a7bb04e362ebf server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:34:44.523 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.516+0000 7fda30ff9640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-09-11T23:34:44.523 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.516+0000 7fda30ff9640 1 -- 172.21.15.26:0/3095633010 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda40020070 con 0x55e80ed65cb0 2024-09-11T23:34:44.523 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.516+0000 7fda52264740 1 -- 172.21.15.26:0/3095633010 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fda38010aa0 con 0x55e80ed65cb0 2024-09-11T23:34:44.523 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.516+0000 7fda30ff9640 1 -- 172.21.15.26:0/3095633010 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 8586+0+0 (secure 0 0 0) 0x7fda40097d20 con 0x55e80ed65cb0 2024-09-11T23:34:44.523 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.516+0000 7fda30ff9640 1 -- 172.21.15.26:0/3095633010 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7fda4009b6b0 con 0x55e80ed65cb0 2024-09-11T23:34:44.523 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.516+0000 7fda52264740 10 client.27130 fetch_fsmap learned FSMap version 76 2024-09-11T23:34:44.524 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.517+0000 7fda52264740 1 -- 172.21.15.26:0/3095633010 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) -- 0x55e80ea9e620 con 0x55e80ed65cb0 2024-09-11T23:34:44.524 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.517+0000 7fda30ff9640 1 -- 172.21.15.26:0/3095633010 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda38010aa0 con 0x55e80ed65cb0 2024-09-11T23:34:44.524 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.517+0000 7fda30ff9640 1 -- 172.21.15.26:0/3095633010 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 76) ==== 8438+0+0 (secure 0 0 0) 0x7fda4009ab00 con 0x55e80ed65cb0 2024-09-11T23:34:44.524 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.517+0000 7fda52264740 10 client.27130 fetch_fsmap finished waiting for FSMap version 76 2024-09-11T23:34:44.524 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.517+0000 7fda52264740 10 client.27130 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:34:44.524 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.517+0000 7fda52264740 20 client.27130 populate_metadata read hostname 'smithi026' 2024-09-11T23:34:44.524 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.517+0000 7fda52264740 1 --2- 172.21.15.26:0/3095633010 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55e80ed75c50 0x55e80ed780b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:44.524 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.517+0000 7fda52264740 4 client.27130 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:34:44.524 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.517+0000 7fda52264740 1 -- 172.21.15.26:0/3095633010 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e80eccca20 con 0x55e80ed75c50 2024-09-11T23:34:44.524 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.517+0000 7fda4d26e640 1 --2- 172.21.15.26:0/3095633010 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55e80ed75c50 0x55e80ed780b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:34:44.524 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.518+0000 7fda4d26e640 1 --2- 172.21.15.26:0/3095633010 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55e80ed75c50 0x55e80ed780b0 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1bc5d43f42cc36ff server_cookie=218fe66b568201a7 in_seq=0 out_seq=0 2024-09-11T23:34:44.524 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.518+0000 7fda30ff9640 10 client.27130 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:34:44.526 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.518+0000 7fda30ff9640 1 -- 172.21.15.26:0/3095633010 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e80eccca20 con 0x55e80ed75c50 2024-09-11T23:34:44.526 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.518+0000 7fda30ff9640 10 client.27130 handle_command_reply: tid=0 2024-09-11T23:34:44.526 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.518+0000 7fda52264740 1 client.27130 shutdown 2024-09-11T23:34:44.526 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.519+0000 7fda52264740 1 -- 172.21.15.26:0/3095633010 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e80ed65cb0 msgr2=0x55e80ed63580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:34:44.526 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.519+0000 7fda52264740 1 --2- 172.21.15.26:0/3095633010 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e80ed65cb0 0x55e80ed63580 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fda400cc830 tx=0x7fda4009b570 comp rx=0 tx=0).stop 2024-09-11T23:34:44.526 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.519+0000 7fda52264740 1 -- 172.21.15.26:0/3095633010 shutdown_connections 2024-09-11T23:34:44.526 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.519+0000 7fda52264740 1 --2- 172.21.15.26:0/3095633010 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55e80ed75c50 0x55e80ed780b0 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:44.526 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.519+0000 7fda52264740 1 --2- 172.21.15.26:0/3095633010 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e80ed65cb0 0x55e80ed63580 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:44.526 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.519+0000 7fda52264740 1 --2- 172.21.15.26:0/3095633010 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e80ed5b410 0x55e80ed63040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:44.526 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.519+0000 7fda52264740 1 --2- 172.21.15.26:0/3095633010 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e80ed5aaf0 0x55e80ed62b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:44.526 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.519+0000 7fda30ff9640 0 client.27130 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:34:44.526 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.519+0000 7fda52264740 1 -- 172.21.15.26:0/3095633010 >> 172.21.15.26:0/3095633010 conn(0x55e80ed57060 msgr2=0x55e80ed5cdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:34:44.527 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.519+0000 7fda52264740 1 -- 172.21.15.26:0/3095633010 shutdown_connections 2024-09-11T23:34:44.527 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.519+0000 7fda52264740 1 -- 172.21.15.26:0/3095633010 wait complete. 2024-09-11T23:34:44.527 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.519+0000 7fda52264740 20 client.27130 trim_cache size 0 max 16384 2024-09-11T23:34:44.535 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.534+0000 7fda52264740 1 Processor -- start 2024-09-11T23:34:44.535 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.534+0000 7fda52264740 1 -- start start 2024-09-11T23:34:44.535 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.534+0000 7fda52264740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e80ed5aaf0 0x55e80ed5d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:44.536 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.534+0000 7fda52264740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e80ed5b410 0x55e80ed633b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:44.536 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.534+0000 7fda52264740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e80ed65cb0 0x55e80ed7d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:44.536 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.534+0000 7fda52264740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fda481afae0 con 0x55e80ed5b410 2024-09-11T23:34:44.536 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.534+0000 7fda52264740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fda481b18f0 con 0x55e80ed5aaf0 2024-09-11T23:34:44.536 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.534+0000 7fda52264740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fda481b0270 con 0x55e80ed65cb0 2024-09-11T23:34:44.536 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.534+0000 7fda4ca6d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e80ed5b410 0x55e80ed633b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:34:44.536 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.534+0000 7fda4ca6d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e80ed5b410 0x55e80ed633b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58532/0 (socket says 172.21.15.26:58532) 2024-09-11T23:34:44.536 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.534+0000 7fda4ca6d640 1 -- 172.21.15.26:0/3402796688 learned_addr learned my addr 172.21.15.26:0/3402796688 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:34:44.536 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.534+0000 7fda4da6f640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e80ed65cb0 0x55e80ed7d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:34:44.536 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.534+0000 7fda4d26e640 1 --2- 172.21.15.26:0/3402796688 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e80ed5aaf0 0x55e80ed5d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:34:44.536 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.534+0000 7fda4ca6d640 1 -- 172.21.15.26:0/3402796688 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e80ed65cb0 msgr2=0x55e80ed7d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:34:44.536 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.534+0000 7fda4ca6d640 1 --2- 172.21.15.26:0/3402796688 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e80ed65cb0 0x55e80ed7d9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:44.536 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.534+0000 7fda4ca6d640 1 -- 172.21.15.26:0/3402796688 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e80ed5aaf0 msgr2=0x55e80ed5d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:34:44.537 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.534+0000 7fda4ca6d640 1 --2- 172.21.15.26:0/3402796688 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e80ed5aaf0 0x55e80ed5d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:44.537 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.534+0000 7fda4ca6d640 1 -- 172.21.15.26:0/3402796688 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e80ed5b240 con 0x55e80ed5b410 2024-09-11T23:34:44.537 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.534+0000 7fda4d26e640 1 --2- 172.21.15.26:0/3402796688 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e80ed5aaf0 0x55e80ed5d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:34:44.537 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.534+0000 7fda4ca6d640 1 --2- 172.21.15.26:0/3402796688 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e80ed5b410 0x55e80ed633b0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7fda380058c0 tx=0x7fda38012530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cc5b44a5c699262 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:34:44.537 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.535+0000 7fda32ffd640 1 -- 172.21.15.26:0/3402796688 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda38005040 con 0x55e80ed5b410 2024-09-11T23:34:44.537 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.535+0000 7fda32ffd640 1 -- 172.21.15.26:0/3402796688 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda38004680 con 0x55e80ed5b410 2024-09-11T23:34:44.537 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.535+0000 7fda32ffd640 1 -- 172.21.15.26:0/3402796688 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda3800e8b0 con 0x55e80ed5b410 2024-09-11T23:34:44.541 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.535+0000 7fda52264740 1 -- 172.21.15.26:0/3402796688 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e80ed5b410 msgr2=0x55e80ed633b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:34:44.541 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.535+0000 7fda52264740 1 --2- 172.21.15.26:0/3402796688 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e80ed5b410 0x55e80ed633b0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7fda380058c0 tx=0x7fda38012530 comp rx=0 tx=0).stop 2024-09-11T23:34:44.541 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.535+0000 7fda52264740 1 -- 172.21.15.26:0/3402796688 shutdown_connections 2024-09-11T23:34:44.541 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.535+0000 7fda52264740 1 --2- 172.21.15.26:0/3402796688 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e80ed65cb0 0x55e80ed7d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:44.541 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.535+0000 7fda52264740 1 --2- 172.21.15.26:0/3402796688 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e80ed5b410 0x55e80ed633b0 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:44.541 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.535+0000 7fda52264740 1 --2- 172.21.15.26:0/3402796688 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e80ed5aaf0 0x55e80ed5d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:44.541 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.535+0000 7fda52264740 1 -- 172.21.15.26:0/3402796688 >> 172.21.15.26:0/3402796688 conn(0x55e80ed57060 msgr2=0x55e80ed64b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:34:44.541 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.535+0000 7fda52264740 1 -- 172.21.15.26:0/3402796688 shutdown_connections 2024-09-11T23:34:44.541 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.535+0000 7fda52264740 1 -- 172.21.15.26:0/3402796688 wait complete. 2024-09-11T23:34:44.541 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.536+0000 7fda52264740 1 -- start start 2024-09-11T23:34:44.541 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.536+0000 7fda52264740 1 Processor -- start 2024-09-11T23:34:44.541 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.536+0000 7fda52264740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e80ed5aaf0 0x55e80ed6e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:44.542 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.536+0000 7fda52264740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e80ed5b410 0x55e80ed69420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:44.542 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.536+0000 7fda52264740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e80ed65cb0 0x55e80ed69960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:44.542 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.536+0000 7fda52264740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fda481b1160 con 0x55e80ed5aaf0 2024-09-11T23:34:44.542 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.536+0000 7fda52264740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fda481b09d0 con 0x55e80ed5b410 2024-09-11T23:34:44.542 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.536+0000 7fda52264740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fda481adae0 con 0x55e80ed65cb0 2024-09-11T23:34:44.542 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.536+0000 7fda52264740 10 client.0 _ll_register_callbacks cb 0x7fda400092a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:34:44.542 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.536+0000 7fda4d26e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e80ed5aaf0 0x55e80ed6e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:34:44.542 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.536+0000 7fda4d26e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e80ed5aaf0 0x55e80ed6e100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58542/0 (socket says 172.21.15.26:58542) 2024-09-11T23:34:44.542 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.536+0000 7fda4d26e640 1 -- 172.21.15.26:0/1622615847 learned_addr learned my addr 172.21.15.26:0/1622615847 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:34:44.542 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.536+0000 7fda4da6f640 1 --2- 172.21.15.26:0/1622615847 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e80ed65cb0 0x55e80ed69960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:34:44.542 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.537+0000 7fda4ca6d640 1 --2- 172.21.15.26:0/1622615847 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e80ed5b410 0x55e80ed69420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:34:44.542 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.537+0000 7fda4d26e640 1 -- 172.21.15.26:0/1622615847 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e80ed65cb0 msgr2=0x55e80ed69960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:34:44.542 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.537+0000 7fda4d26e640 1 --2- 172.21.15.26:0/1622615847 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e80ed65cb0 0x55e80ed69960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:44.542 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.537+0000 7fda4d26e640 1 -- 172.21.15.26:0/1622615847 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e80ed5b410 msgr2=0x55e80ed69420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:34:44.543 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.537+0000 7fda4d26e640 1 --2- 172.21.15.26:0/1622615847 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e80ed5b410 0x55e80ed69420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:44.543 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.537+0000 7fda4d26e640 1 -- 172.21.15.26:0/1622615847 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e80ed69ea0 con 0x55e80ed5aaf0 2024-09-11T23:34:44.543 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.537+0000 7fda4d26e640 1 --2- 172.21.15.26:0/1622615847 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e80ed5aaf0 0x55e80ed6e100 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7fda44009930 tx=0x7fda44009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72d9743079522592 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:34:44.543 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.537+0000 7fda30ff9640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:34:44.543 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.537+0000 7fda30ff9640 1 -- 172.21.15.26:0/1622615847 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda44035040 con 0x55e80ed5aaf0 2024-09-11T23:34:44.543 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.537+0000 7fda30ff9640 1 -- 172.21.15.26:0/1622615847 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 8586+0+0 (secure 0 0 0) 0x7fda44003dd0 con 0x55e80ed5aaf0 2024-09-11T23:34:44.543 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.537+0000 7fda52264740 1 -- 172.21.15.26:0/1622615847 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fda38004680 con 0x55e80ed5aaf0 2024-09-11T23:34:44.543 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.537+0000 7fda30ff9640 1 -- 172.21.15.26:0/1622615847 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7fda4405d470 con 0x55e80ed5aaf0 2024-09-11T23:34:44.543 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.537+0000 7fda52264740 10 client.17637 fetch_fsmap learned FSMap version 76 2024-09-11T23:34:44.543 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.537+0000 7fda52264740 1 -- 172.21.15.26:0/1622615847 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) -- 0x55e80ede06f0 con 0x55e80ed5aaf0 2024-09-11T23:34:44.543 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.537+0000 7fda30ff9640 1 -- 172.21.15.26:0/1622615847 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda38004680 con 0x55e80ed5aaf0 2024-09-11T23:34:44.543 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.538+0000 7fda30ff9640 1 -- 172.21.15.26:0/1622615847 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 76) ==== 8438+0+0 (secure 0 0 0) 0x7fda4403d8c0 con 0x55e80ed5aaf0 2024-09-11T23:34:44.543 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.538+0000 7fda52264740 10 client.17637 fetch_fsmap finished waiting for FSMap version 76 2024-09-11T23:34:44.544 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.538+0000 7fda52264740 10 client.17637 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:34:44.544 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.538+0000 7fda52264740 20 client.17637 populate_metadata read hostname 'smithi026' 2024-09-11T23:34:44.544 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.538+0000 7fda52264740 1 --2- 172.21.15.26:0/1622615847 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55e80ede0eb0 0x55e80ede32d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:44.544 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.538+0000 7fda52264740 4 client.17637 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T23:34:44.544 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.538+0000 7fda52264740 1 -- 172.21.15.26:0/1622615847 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e80ecc6d70 con 0x55e80ede0eb0 2024-09-11T23:34:44.544 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.538+0000 7fda4ca6d640 1 --2- 172.21.15.26:0/1622615847 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55e80ede0eb0 0x55e80ede32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:34:44.544 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.538+0000 7fda30ff9640 10 client.17637 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:34:44.544 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.538+0000 7fda4ca6d640 1 --2- 172.21.15.26:0/1622615847 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55e80ede0eb0 0x55e80ede32d0 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e985cae0f79e00b server_cookie=725af7679ae737f6 in_seq=0 out_seq=0 2024-09-11T23:34:44.550 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.549+0000 7fda30ff9640 1 -- 172.21.15.26:0/1622615847 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e80ecc6d70 con 0x55e80ede0eb0 2024-09-11T23:34:44.551 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.549+0000 7fda30ff9640 10 client.17637 handle_command_reply: tid=0 2024-09-11T23:34:44.551 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.549+0000 7fda52264740 1 client.17637 shutdown 2024-09-11T23:34:44.551 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.550+0000 7fda52264740 1 -- 172.21.15.26:0/1622615847 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e80ed5aaf0 msgr2=0x55e80ed6e100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:34:44.551 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.550+0000 7fda52264740 1 --2- 172.21.15.26:0/1622615847 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e80ed5aaf0 0x55e80ed6e100 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7fda44009930 tx=0x7fda44009260 comp rx=0 tx=0).stop 2024-09-11T23:34:44.551 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.550+0000 7fda52264740 1 -- 172.21.15.26:0/1622615847 shutdown_connections 2024-09-11T23:34:44.551 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.550+0000 7fda52264740 1 --2- 172.21.15.26:0/1622615847 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e80ed5b410 0x55e80ed69420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:44.551 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.550+0000 7fda52264740 1 --2- 172.21.15.26:0/1622615847 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55e80ede0eb0 0x55e80ede32d0 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:44.551 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.550+0000 7fda52264740 1 --2- 172.21.15.26:0/1622615847 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e80ed65cb0 0x55e80ed69960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:44.551 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.550+0000 7fda52264740 1 --2- 172.21.15.26:0/1622615847 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e80ed5aaf0 0x55e80ed6e100 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:44.551 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.550+0000 7fda30ff9640 0 client.17637 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:34:44.551 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.550+0000 7fda52264740 1 -- 172.21.15.26:0/1622615847 >> 172.21.15.26:0/1622615847 conn(0x55e80ed57060 msgr2=0x55e80ed559a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:34:44.551 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.550+0000 7fda52264740 1 -- 172.21.15.26:0/1622615847 shutdown_connections 2024-09-11T23:34:44.552 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.550+0000 7fda52264740 1 -- 172.21.15.26:0/1622615847 wait complete. 2024-09-11T23:34:44.552 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.550+0000 7fda52264740 20 client.17637 trim_cache size 0 max 16384 2024-09-11T23:34:44.552 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-09-11T23:34:44.552 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "message": "", 2024-09-11T23:34:44.552 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-09-11T23:34:44.552 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-09-11T23:34:44.553 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.550+0000 7fda4f4d2640 1 -- 172.21.15.26:0/3486192611 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fda340a2da0 msgr2=0x7fda340a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:34:44.553 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.550+0000 7fda4f4d2640 1 --2- 172.21.15.26:0/3486192611 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fda340a2da0 0x7fda340a5260 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fda44009900 tx=0x7fda440058d0 comp rx=0 tx=0).stop 2024-09-11T23:34:44.553 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.550+0000 7fda4f4d2640 1 -- 172.21.15.26:0/3486192611 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fda48106a00 msgr2=0x7fda480fd720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:34:44.553 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.550+0000 7fda4f4d2640 1 --2- 172.21.15.26:0/3486192611 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fda48106a00 0x7fda480fd720 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7fda40009fd0 tx=0x7fda4000b260 comp rx=0 tx=0).stop 2024-09-11T23:34:44.553 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.551+0000 7fda4f4d2640 1 -- 172.21.15.26:0/3486192611 shutdown_connections 2024-09-11T23:34:44.553 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.551+0000 7fda4f4d2640 1 --2- 172.21.15.26:0/3486192611 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fda340a2da0 0x7fda340a5260 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:44.553 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.551+0000 7fda4f4d2640 1 --2- 172.21.15.26:0/3486192611 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fda48101390 0x7fda480fd1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:44.553 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.551+0000 7fda4f4d2640 1 --2- 172.21.15.26:0/3486192611 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fda48106a00 0x7fda480fd720 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:44.554 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.551+0000 7fda4f4d2640 1 --2- 172.21.15.26:0/3486192611 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fda481009f0 0x7fda480fcca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:44.554 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.551+0000 7fda4f4d2640 1 -- 172.21.15.26:0/3486192611 >> 172.21.15.26:0/3486192611 conn(0x7fda480fc6b0 msgr2=0x7fda4806a580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:34:44.554 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.551+0000 7fda4f4d2640 1 -- 172.21.15.26:0/3486192611 shutdown_connections 2024-09-11T23:34:44.554 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:44.551+0000 7fda4f4d2640 1 -- 172.21.15.26:0/3486192611 wait complete. 2024-09-11T23:34:46.238 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:34:45 smithi136 ceph-mon[35121]: pgmap v513: 97 pgs: 97 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.2 MiB/s wr, 269 op/s 2024-09-11T23:34:46.238 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:34:45 smithi136 ceph-mon[35121]: osdmap e106: 12 total, 12 up, 12 in 2024-09-11T23:34:46.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:34:45 smithi026 ceph-mon[31172]: pgmap v513: 97 pgs: 97 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.2 MiB/s wr, 269 op/s 2024-09-11T23:34:46.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:34:45 smithi026 ceph-mon[31172]: osdmap e106: 12 total, 12 up, 12 in 2024-09-11T23:34:46.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:34:45 smithi076 ceph-mon[35296]: pgmap v513: 97 pgs: 97 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.2 MiB/s wr, 269 op/s 2024-09-11T23:34:46.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:34:45 smithi076 ceph-mon[35296]: osdmap e106: 12 total, 12 up, 12 in 2024-09-11T23:34:48.238 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:34:47 smithi136 ceph-mon[35121]: pgmap v515: 97 pgs: 97 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.1 MiB/s wr, 304 op/s 2024-09-11T23:34:48.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:34:47 smithi026 ceph-mon[31172]: pgmap v515: 97 pgs: 97 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.1 MiB/s wr, 304 op/s 2024-09-11T23:34:48.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:34:47 smithi076 ceph-mon[35296]: pgmap v515: 97 pgs: 97 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.1 MiB/s wr, 304 op/s 2024-09-11T23:34:50.238 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:34:49 smithi136 ceph-mon[35121]: pgmap v516: 97 pgs: 97 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.5 MiB/s wr, 310 op/s 2024-09-11T23:34:50.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:34:49 smithi026 ceph-mon[31172]: pgmap v516: 97 pgs: 97 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.5 MiB/s wr, 310 op/s 2024-09-11T23:34:50.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:34:49 smithi076 ceph-mon[35296]: pgmap v516: 97 pgs: 97 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.5 MiB/s wr, 310 op/s 2024-09-11T23:34:52.238 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:34:51 smithi136 ceph-mon[35121]: pgmap v517: 97 pgs: 97 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.2 MiB/s wr, 323 op/s 2024-09-11T23:34:52.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:34:51 smithi026 ceph-mon[31172]: pgmap v517: 97 pgs: 97 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.2 MiB/s wr, 323 op/s 2024-09-11T23:34:52.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:34:51 smithi076 ceph-mon[35296]: pgmap v517: 97 pgs: 97 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.2 MiB/s wr, 323 op/s 2024-09-11T23:34:54.238 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:34:53 smithi136 ceph-mon[35121]: pgmap v518: 97 pgs: 97 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.2 MiB/s wr, 332 op/s 2024-09-11T23:34:54.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:34:53 smithi026 ceph-mon[31172]: pgmap v518: 97 pgs: 97 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.2 MiB/s wr, 332 op/s 2024-09-11T23:34:54.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:34:53 smithi076 ceph-mon[35296]: pgmap v518: 97 pgs: 97 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.2 MiB/s wr, 332 op/s 2024-09-11T23:34:56.238 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:34:55 smithi136 ceph-mon[35121]: pgmap v519: 97 pgs: 97 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.2 MiB/s wr, 322 op/s 2024-09-11T23:34:56.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:34:55 smithi026 ceph-mon[31172]: pgmap v519: 97 pgs: 97 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.2 MiB/s wr, 322 op/s 2024-09-11T23:34:56.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:34:55 smithi076 ceph-mon[35296]: pgmap v519: 97 pgs: 97 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.2 MiB/s wr, 322 op/s 2024-09-11T23:34:57.642 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.640+0000 7f3a19430640 1 -- 172.21.15.26:0/4065261931 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a14101350 msgr2=0x7f3a1410d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:34:57.643 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.640+0000 7f3a19430640 1 --2- 172.21.15.26:0/4065261931 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a14101350 0x7f3a1410d880 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f3a0c009cd0 tx=0x7f3a0c02a8e0 comp rx=0 tx=0).stop 2024-09-11T23:34:57.643 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.640+0000 7f3a19430640 1 -- 172.21.15.26:0/4065261931 shutdown_connections 2024-09-11T23:34:57.643 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.640+0000 7f3a19430640 1 --2- 172.21.15.26:0/4065261931 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a14101350 0x7f3a1410d880 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:57.643 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.640+0000 7f3a19430640 1 --2- 172.21.15.26:0/4065261931 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3a141009b0 0x7f3a14100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:57.643 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.640+0000 7f3a19430640 1 --2- 172.21.15.26:0/4065261931 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3a141069c0 0x7f3a14106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:57.643 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.640+0000 7f3a19430640 1 -- 172.21.15.26:0/4065261931 >> 172.21.15.26:0/4065261931 conn(0x7f3a14077990 msgr2=0x7f3a140fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:34:57.643 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.640+0000 7f3a19430640 1 -- 172.21.15.26:0/4065261931 shutdown_connections 2024-09-11T23:34:57.643 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.640+0000 7f3a19430640 1 -- 172.21.15.26:0/4065261931 wait complete. 2024-09-11T23:34:57.643 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.641+0000 7f3a19430640 1 Processor -- start 2024-09-11T23:34:57.643 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.641+0000 7f3a19430640 1 -- start start 2024-09-11T23:34:57.643 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.641+0000 7f3a19430640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a141009b0 0x7f3a141982d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:57.644 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.641+0000 7f3a19430640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3a14101350 0x7f3a14198810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:57.644 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.641+0000 7f3a19430640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3a141069c0 0x7f3a1419cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:57.644 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.641+0000 7f3a19430640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f3a14110e00 con 0x7f3a141009b0 2024-09-11T23:34:57.644 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.641+0000 7f3a19430640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f3a14110c80 con 0x7f3a14101350 2024-09-11T23:34:57.644 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.641+0000 7f3a19430640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f3a14110f80 con 0x7f3a141069c0 2024-09-11T23:34:57.644 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.641+0000 7f3a12ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a141009b0 0x7f3a141982d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:34:57.644 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.642+0000 7f3a12ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a141009b0 0x7f3a141982d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42164/0 (socket says 172.21.15.26:42164) 2024-09-11T23:34:57.644 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.642+0000 7f3a12ffd640 1 -- 172.21.15.26:0/339381116 learned_addr learned my addr 172.21.15.26:0/339381116 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:34:57.644 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.642+0000 7f3a0bfff640 1 --2- 172.21.15.26:0/339381116 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3a14101350 0x7f3a14198810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:34:57.644 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.642+0000 7f3a137fe640 1 --2- 172.21.15.26:0/339381116 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3a141069c0 0x7f3a1419cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:34:57.644 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.642+0000 7f3a12ffd640 1 -- 172.21.15.26:0/339381116 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3a141069c0 msgr2=0x7f3a1419cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:34:57.644 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.642+0000 7f3a12ffd640 1 --2- 172.21.15.26:0/339381116 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3a141069c0 0x7f3a1419cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:57.644 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.642+0000 7f3a12ffd640 1 -- 172.21.15.26:0/339381116 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3a14101350 msgr2=0x7f3a14198810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:34:57.645 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.642+0000 7f3a12ffd640 1 --2- 172.21.15.26:0/339381116 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3a14101350 0x7f3a14198810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:57.645 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.642+0000 7f3a12ffd640 1 -- 172.21.15.26:0/339381116 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a1419d2c0 con 0x7f3a141009b0 2024-09-11T23:34:57.646 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.642+0000 7f3a12ffd640 1 --2- 172.21.15.26:0/339381116 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a141009b0 0x7f3a141982d0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f3a0400e9f0 tx=0x7f3a0400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:34:57.646 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.642+0000 7f3a10ff9640 1 -- 172.21.15.26:0/339381116 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a0400cd80 con 0x7f3a141009b0 2024-09-11T23:34:57.646 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.642+0000 7f3a10ff9640 1 -- 172.21.15.26:0/339381116 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a04010910 con 0x7f3a141009b0 2024-09-11T23:34:57.646 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.642+0000 7f3a19430640 1 -- 172.21.15.26:0/339381116 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a1419d5b0 con 0x7f3a141009b0 2024-09-11T23:34:57.646 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.642+0000 7f3a10ff9640 1 -- 172.21.15.26:0/339381116 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a04018990 con 0x7f3a141009b0 2024-09-11T23:34:57.646 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.642+0000 7f3a19430640 1 -- 172.21.15.26:0/339381116 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a141a4e40 con 0x7f3a141009b0 2024-09-11T23:34:57.653 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.645+0000 7f3a10ff9640 1 -- 172.21.15.26:0/339381116 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f3a04010ac0 con 0x7f3a141009b0 2024-09-11T23:34:57.653 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.645+0000 7f3a1c1c8740 1 Processor -- start 2024-09-11T23:34:57.653 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.645+0000 7f3a1c1c8740 1 -- start start 2024-09-11T23:34:57.653 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.645+0000 7f3a1c1c8740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5588a340baf0 0x5588a340bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:57.653 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.645+0000 7f3a1c1c8740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5588a340c410 0x5588a3416770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:57.653 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.645+0000 7f3a1c1c8740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5588a3416cb0 0x5588a34190a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:57.654 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.645+0000 7f3a1c1c8740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5588a33702f0 con 0x5588a3416cb0 2024-09-11T23:34:57.654 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.645+0000 7f3a1c1c8740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5588a3318270 con 0x5588a340baf0 2024-09-11T23:34:57.654 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.645+0000 7f3a1c1c8740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5588a336c2b0 con 0x5588a340c410 2024-09-11T23:34:57.654 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.646+0000 7f3a137fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5588a340baf0 0x5588a340bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:34:57.654 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.646+0000 7f3a0bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5588a3416cb0 0x5588a34190a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:34:57.654 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.646+0000 7f3a137fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5588a340baf0 0x5588a340bed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:54354/0 (socket says 172.21.15.26:54354) 2024-09-11T23:34:57.654 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.646+0000 7f3a137fe640 1 -- 172.21.15.26:0/2537709914 learned_addr learned my addr 172.21.15.26:0/2537709914 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:34:57.654 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.646+0000 7f3a0bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5588a3416cb0 0x5588a34190a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42172/0 (socket says 172.21.15.26:42172) 2024-09-11T23:34:57.654 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.646+0000 7f3a12ffd640 1 --2- 172.21.15.26:0/2537709914 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5588a340c410 0x5588a3416770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:34:57.654 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.646+0000 7f3a10ff9640 1 --2- 172.21.15.26:0/339381116 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f3a000a2e20 0x7f3a000a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:57.654 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.646+0000 7f3a137fe640 1 -- 172.21.15.26:0/2537709914 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5588a340c410 msgr2=0x5588a3416770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:34:57.654 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.646+0000 7f3a137fe640 1 --2- 172.21.15.26:0/2537709914 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5588a340c410 0x5588a3416770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:57.654 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.646+0000 7f3a137fe640 1 -- 172.21.15.26:0/2537709914 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5588a3416cb0 msgr2=0x5588a34190a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:34:57.655 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.646+0000 7f3a137fe640 1 --2- 172.21.15.26:0/2537709914 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5588a3416cb0 0x5588a34190a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:57.655 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.646+0000 7f3a137fe640 1 -- 172.21.15.26:0/2537709914 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5588a3317340 con 0x5588a340baf0 2024-09-11T23:34:57.655 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.646+0000 7f3a0bfff640 1 --2- 172.21.15.26:0/2537709914 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5588a3416cb0 0x5588a34190a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:34:57.655 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.646+0000 7f3a10ff9640 1 -- 172.21.15.26:0/339381116 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 8542+0+0 (secure 0 0 0) 0x7f3a040cf630 con 0x7f3a141009b0 2024-09-11T23:34:57.655 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.646+0000 7f3a137fe640 1 --2- 172.21.15.26:0/339381116 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f3a000a2e20 0x7f3a000a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:34:57.655 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.646+0000 7f3a137fe640 1 --2- 172.21.15.26:0/2537709914 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5588a340baf0 0x5588a340bed0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f3a0c009cd0 tx=0x7f3a0c031500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6d225f4a22b42194 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:34:57.655 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.646+0000 7f3a097fa640 1 -- 172.21.15.26:0/2537709914 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a0c041040 con 0x5588a340baf0 2024-09-11T23:34:57.655 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.646+0000 7f3a137fe640 1 --2- 172.21.15.26:0/339381116 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f3a000a2e20 0x7f3a000a52e0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f3a0c04c000 tx=0x7f3a0c04d000 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:34:57.655 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.647+0000 7f3a097fa640 1 -- 172.21.15.26:0/2537709914 <== mon.2 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a0c043070 con 0x5588a340baf0 2024-09-11T23:34:57.655 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.647+0000 7f3a097fa640 1 -- 172.21.15.26:0/2537709914 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a0c030d30 con 0x5588a340baf0 2024-09-11T23:34:57.655 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.647+0000 7f3a1c1c8740 1 -- 172.21.15.26:0/2537709914 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5588a340baf0 msgr2=0x5588a340bed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:34:57.655 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.647+0000 7f3a1c1c8740 1 --2- 172.21.15.26:0/2537709914 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5588a340baf0 0x5588a340bed0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f3a0c009cd0 tx=0x7f3a0c031500 comp rx=0 tx=0).stop 2024-09-11T23:34:57.655 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.647+0000 7f3a1c1c8740 1 -- 172.21.15.26:0/2537709914 shutdown_connections 2024-09-11T23:34:57.656 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.647+0000 7f3a1c1c8740 1 --2- 172.21.15.26:0/2537709914 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5588a3416cb0 0x5588a34190a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:57.656 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.647+0000 7f3a1c1c8740 1 --2- 172.21.15.26:0/2537709914 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5588a340c410 0x5588a3416770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:57.656 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.647+0000 7f3a1c1c8740 1 --2- 172.21.15.26:0/2537709914 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5588a340baf0 0x5588a340bed0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:57.656 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.647+0000 7f3a1c1c8740 1 -- 172.21.15.26:0/2537709914 >> 172.21.15.26:0/2537709914 conn(0x5588a3408060 msgr2=0x5588a3408440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:34:57.656 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.647+0000 7f3a1c1c8740 1 -- 172.21.15.26:0/2537709914 shutdown_connections 2024-09-11T23:34:57.656 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.647+0000 7f3a1c1c8740 1 -- 172.21.15.26:0/2537709914 wait complete. 2024-09-11T23:34:57.656 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.648+0000 7f3a1c1c8740 1 -- start start 2024-09-11T23:34:57.656 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.648+0000 7f3a1c1c8740 1 Processor -- start 2024-09-11T23:34:57.656 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.648+0000 7f3a1c1c8740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5588a340baf0 0x5588a3413bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:57.656 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.648+0000 7f3a1c1c8740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5588a340c410 0x5588a3414130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:57.656 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.648+0000 7f3a1c1c8740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5588a3416cb0 0x5588a3414670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:57.656 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.648+0000 7f3a1c1c8740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5588a3419880 con 0x5588a340baf0 2024-09-11T23:34:57.656 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.648+0000 7f3a1c1c8740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5588a32f9290 con 0x5588a340c410 2024-09-11T23:34:57.657 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.648+0000 7f3a1c1c8740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5588a3419a00 con 0x5588a3416cb0 2024-09-11T23:34:57.657 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.648+0000 7f3a1c1c8740 10 client.0 _ll_register_callbacks cb 0x5588a332a140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:34:57.657 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.648+0000 7f3a12ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5588a340baf0 0x5588a3413bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:34:57.657 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.648+0000 7f3a12ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5588a340baf0 0x5588a3413bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42182/0 (socket says 172.21.15.26:42182) 2024-09-11T23:34:57.657 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.648+0000 7f3a12ffd640 1 -- 172.21.15.26:0/3638458832 learned_addr learned my addr 172.21.15.26:0/3638458832 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:34:57.657 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.648+0000 7f3a0bfff640 1 --2- 172.21.15.26:0/3638458832 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5588a340c410 0x5588a3414130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:34:57.657 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.648+0000 7f3a137fe640 1 --2- 172.21.15.26:0/3638458832 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5588a3416cb0 0x5588a3414670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:34:57.657 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.648+0000 7f3a12ffd640 1 -- 172.21.15.26:0/3638458832 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5588a3416cb0 msgr2=0x5588a3414670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:34:57.657 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.648+0000 7f3a12ffd640 1 --2- 172.21.15.26:0/3638458832 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5588a3416cb0 0x5588a3414670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:57.657 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.648+0000 7f3a12ffd640 1 -- 172.21.15.26:0/3638458832 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5588a340c410 msgr2=0x5588a3414130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:34:57.657 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.648+0000 7f3a12ffd640 1 --2- 172.21.15.26:0/3638458832 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5588a340c410 0x5588a3414130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:57.657 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.648+0000 7f3a12ffd640 1 -- 172.21.15.26:0/3638458832 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5588a3083210 con 0x5588a340baf0 2024-09-11T23:34:57.658 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.648+0000 7f3a0bfff640 1 --2- 172.21.15.26:0/3638458832 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5588a340c410 0x5588a3414130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:34:57.658 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.648+0000 7f3a137fe640 1 --2- 172.21.15.26:0/3638458832 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5588a3416cb0 0x5588a3414670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:34:57.658 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.649+0000 7f3a12ffd640 1 --2- 172.21.15.26:0/3638458832 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5588a340baf0 0x5588a3413bf0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f3a0409b8f0 tx=0x7f3a040d5240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcb924d34b5fae69 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:34:57.658 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.649+0000 7f39fa7fc640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:34:57.658 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.649+0000 7f39fa7fc640 1 -- 172.21.15.26:0/3638458832 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a04018d30 con 0x5588a340baf0 2024-09-11T23:34:57.658 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.649+0000 7f39fa7fc640 1 -- 172.21.15.26:0/3638458832 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 8542+0+0 (secure 0 0 0) 0x7f3a0409b040 con 0x5588a340baf0 2024-09-11T23:34:57.658 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.649+0000 7f3a1c1c8740 1 -- 172.21.15.26:0/3638458832 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3a0c043070 con 0x5588a340baf0 2024-09-11T23:34:57.658 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.649+0000 7f39fa7fc640 1 -- 172.21.15.26:0/3638458832 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7f3a0409fef0 con 0x5588a340baf0 2024-09-11T23:34:57.658 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.649+0000 7f3a1c1c8740 10 client.17649 fetch_fsmap learned FSMap version 76 2024-09-11T23:34:57.658 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.649+0000 7f3a1c1c8740 1 -- 172.21.15.26:0/3638458832 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) -- 0x5588a314f620 con 0x5588a340baf0 2024-09-11T23:34:57.658 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.649+0000 7f39fa7fc640 1 -- 172.21.15.26:0/3638458832 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a0c043070 con 0x5588a340baf0 2024-09-11T23:34:57.658 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.649+0000 7f39fa7fc640 1 -- 172.21.15.26:0/3638458832 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 76) ==== 8438+0+0 (secure 0 0 0) 0x7f3a040cfce0 con 0x5588a340baf0 2024-09-11T23:34:57.658 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.649+0000 7f3a1c1c8740 10 client.17649 fetch_fsmap finished waiting for FSMap version 76 2024-09-11T23:34:57.659 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.649+0000 7f3a1c1c8740 10 client.17649 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:34:57.659 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.649+0000 7f3a1c1c8740 20 client.17649 populate_metadata read hostname 'smithi026' 2024-09-11T23:34:57.659 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.649+0000 7f3a1c1c8740 1 --2- 172.21.15.26:0/3638458832 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5588a3426cd0 0x5588a3429120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:57.659 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.649+0000 7f3a1c1c8740 4 client.17649 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:34:57.659 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.649+0000 7f3a1c1c8740 1 -- 172.21.15.26:0/3638458832 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5588a337da20 con 0x5588a3426cd0 2024-09-11T23:34:57.659 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.650+0000 7f3a0bfff640 1 --2- 172.21.15.26:0/3638458832 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5588a3426cd0 0x5588a3429120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:34:57.659 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.650+0000 7f39fa7fc640 10 client.17649 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:34:57.659 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.650+0000 7f3a0bfff640 1 --2- 172.21.15.26:0/3638458832 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5588a3426cd0 0x5588a3429120 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=71a75543fb022e7b server_cookie=8542e3d7c00272 in_seq=0 out_seq=0 2024-09-11T23:34:57.659 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.651+0000 7f39fa7fc640 1 -- 172.21.15.26:0/3638458832 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5588a337da20 con 0x5588a3426cd0 2024-09-11T23:34:57.659 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.651+0000 7f39fa7fc640 10 client.17649 handle_command_reply: tid=0 2024-09-11T23:34:57.659 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.651+0000 7f3a1c1c8740 1 client.17649 shutdown 2024-09-11T23:34:57.659 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.651+0000 7f3a1c1c8740 1 -- 172.21.15.26:0/3638458832 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5588a340baf0 msgr2=0x5588a3413bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:34:57.659 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.651+0000 7f3a1c1c8740 1 --2- 172.21.15.26:0/3638458832 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5588a340baf0 0x5588a3413bf0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f3a0409b8f0 tx=0x7f3a040d5240 comp rx=0 tx=0).stop 2024-09-11T23:34:57.659 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.651+0000 7f3a1c1c8740 1 -- 172.21.15.26:0/3638458832 shutdown_connections 2024-09-11T23:34:57.660 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.651+0000 7f3a1c1c8740 1 --2- 172.21.15.26:0/3638458832 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5588a340c410 0x5588a3414130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:57.660 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.651+0000 7f3a1c1c8740 1 --2- 172.21.15.26:0/3638458832 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5588a3426cd0 0x5588a3429120 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:57.660 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.651+0000 7f3a1c1c8740 1 --2- 172.21.15.26:0/3638458832 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5588a3416cb0 0x5588a3414670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:57.660 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.651+0000 7f3a1c1c8740 1 --2- 172.21.15.26:0/3638458832 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5588a340baf0 0x5588a3413bf0 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:57.660 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.651+0000 7f39fa7fc640 0 client.17649 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:34:57.660 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.651+0000 7f3a1c1c8740 1 -- 172.21.15.26:0/3638458832 >> 172.21.15.26:0/3638458832 conn(0x5588a3408060 msgr2=0x5588a340dd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:34:57.660 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.651+0000 7f3a1c1c8740 1 -- 172.21.15.26:0/3638458832 shutdown_connections 2024-09-11T23:34:57.660 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.651+0000 7f3a1c1c8740 1 -- 172.21.15.26:0/3638458832 wait complete. 2024-09-11T23:34:57.660 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.651+0000 7f3a1c1c8740 20 client.17649 trim_cache size 0 max 16384 2024-09-11T23:34:57.667 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.667+0000 7f3a1c1c8740 1 Processor -- start 2024-09-11T23:34:57.668 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.667+0000 7f3a1c1c8740 1 -- start start 2024-09-11T23:34:57.668 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.667+0000 7f3a1c1c8740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5588a340baf0 0x5588a34144c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:57.668 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.667+0000 7f3a1c1c8740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5588a340c410 0x5588a342ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:57.668 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.667+0000 7f3a1c1c8740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5588a3416cb0 0x5588a342ef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:57.668 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.667+0000 7f3a1c1c8740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f3a141ad770 con 0x5588a340c410 2024-09-11T23:34:57.668 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.667+0000 7f3a1c1c8740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f3a141af580 con 0x5588a3416cb0 2024-09-11T23:34:57.668 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.667+0000 7f3a1c1c8740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f3a141adf00 con 0x5588a340baf0 2024-09-11T23:34:57.668 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.667+0000 7f3a0bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5588a340c410 0x5588a342ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:34:57.668 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.667+0000 7f3a0bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5588a340c410 0x5588a342ea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42194/0 (socket says 172.21.15.26:42194) 2024-09-11T23:34:57.668 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.667+0000 7f3a0bfff640 1 -- 172.21.15.26:0/3153042362 learned_addr learned my addr 172.21.15.26:0/3153042362 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:34:57.668 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.667+0000 7f3a137fe640 1 --2- 172.21.15.26:0/3153042362 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5588a3416cb0 0x5588a342ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:34:57.668 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.667+0000 7f3a12ffd640 1 --2- 172.21.15.26:0/3153042362 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5588a340baf0 0x5588a34144c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:34:57.668 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.667+0000 7f3a0bfff640 1 -- 172.21.15.26:0/3153042362 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5588a340baf0 msgr2=0x5588a34144c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:34:57.669 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.667+0000 7f3a0bfff640 1 --2- 172.21.15.26:0/3153042362 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5588a340baf0 0x5588a34144c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:57.669 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.667+0000 7f3a0bfff640 1 -- 172.21.15.26:0/3153042362 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5588a3416cb0 msgr2=0x5588a342ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:34:57.669 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.667+0000 7f3a0bfff640 1 --2- 172.21.15.26:0/3153042362 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5588a3416cb0 0x5588a342ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:57.669 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.667+0000 7f3a0bfff640 1 -- 172.21.15.26:0/3153042362 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5588a34091d0 con 0x5588a340c410 2024-09-11T23:34:57.669 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.667+0000 7f3a137fe640 1 --2- 172.21.15.26:0/3153042362 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5588a3416cb0 0x5588a342ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:34:57.669 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.668+0000 7f3a0bfff640 1 --2- 172.21.15.26:0/3153042362 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5588a340c410 0x5588a342ea30 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f3a141053a0 tx=0x7f39fc011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97e258d13ae34866 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:34:57.669 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.668+0000 7f3a08ff9640 1 -- 172.21.15.26:0/3153042362 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39fc0108e0 con 0x5588a340c410 2024-09-11T23:34:57.669 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.668+0000 7f3a08ff9640 1 -- 172.21.15.26:0/3153042362 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f39fc005020 con 0x5588a340c410 2024-09-11T23:34:57.669 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.668+0000 7f3a08ff9640 1 -- 172.21.15.26:0/3153042362 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39fc004ae0 con 0x5588a340c410 2024-09-11T23:34:57.673 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.668+0000 7f3a1c1c8740 1 -- 172.21.15.26:0/3153042362 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5588a340c410 msgr2=0x5588a342ea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:34:57.673 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.668+0000 7f3a1c1c8740 1 --2- 172.21.15.26:0/3153042362 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5588a340c410 0x5588a342ea30 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f3a141053a0 tx=0x7f39fc011ba0 comp rx=0 tx=0).stop 2024-09-11T23:34:57.673 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.669+0000 7f3a1c1c8740 1 -- 172.21.15.26:0/3153042362 shutdown_connections 2024-09-11T23:34:57.673 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.669+0000 7f3a1c1c8740 1 --2- 172.21.15.26:0/3153042362 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5588a3416cb0 0x5588a342ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:57.673 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.669+0000 7f3a1c1c8740 1 --2- 172.21.15.26:0/3153042362 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5588a340c410 0x5588a342ea30 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:57.673 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.669+0000 7f3a1c1c8740 1 --2- 172.21.15.26:0/3153042362 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5588a340baf0 0x5588a34144c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:57.673 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.669+0000 7f3a1c1c8740 1 -- 172.21.15.26:0/3153042362 >> 172.21.15.26:0/3153042362 conn(0x5588a3408060 msgr2=0x5588a3415c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:34:57.673 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.669+0000 7f3a1c1c8740 1 -- 172.21.15.26:0/3153042362 shutdown_connections 2024-09-11T23:34:57.673 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.669+0000 7f3a1c1c8740 1 -- 172.21.15.26:0/3153042362 wait complete. 2024-09-11T23:34:57.673 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.669+0000 7f3a1c1c8740 1 -- start start 2024-09-11T23:34:57.674 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.669+0000 7f3a1c1c8740 1 Processor -- start 2024-09-11T23:34:57.674 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.669+0000 7f3a1c1c8740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5588a340baf0 0x5588a3410b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:57.674 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.669+0000 7f3a1c1c8740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5588a340c410 0x5588a34110b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:57.674 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.669+0000 7f3a1c1c8740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5588a3416cb0 0x5588a34115f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:57.674 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.669+0000 7f3a1c1c8740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f3a141aedf0 con 0x5588a340baf0 2024-09-11T23:34:57.674 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.669+0000 7f3a1c1c8740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f3a141ae660 con 0x5588a3416cb0 2024-09-11T23:34:57.674 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.669+0000 7f3a1c1c8740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f3a141ab770 con 0x5588a340c410 2024-09-11T23:34:57.674 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.669+0000 7f3a1c1c8740 10 client.0 _ll_register_callbacks cb 0x7f3a0400cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:34:57.674 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.669+0000 7f3a12ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5588a340baf0 0x5588a3410b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:34:57.674 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.669+0000 7f3a137fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5588a3416cb0 0x5588a34115f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:34:57.674 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.669+0000 7f3a12ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5588a340baf0 0x5588a3410b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42200/0 (socket says 172.21.15.26:42200) 2024-09-11T23:34:57.674 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.669+0000 7f3a12ffd640 1 -- 172.21.15.26:0/2743507799 learned_addr learned my addr 172.21.15.26:0/2743507799 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:34:57.674 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.669+0000 7f3a0bfff640 1 --2- 172.21.15.26:0/2743507799 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5588a340c410 0x5588a34110b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:34:57.674 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.670+0000 7f3a12ffd640 1 -- 172.21.15.26:0/2743507799 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5588a340c410 msgr2=0x5588a34110b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:34:57.675 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.670+0000 7f3a12ffd640 1 --2- 172.21.15.26:0/2743507799 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5588a340c410 0x5588a34110b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:57.675 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.670+0000 7f3a12ffd640 1 -- 172.21.15.26:0/2743507799 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5588a3416cb0 msgr2=0x5588a34115f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:34:57.675 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.670+0000 7f3a12ffd640 1 --2- 172.21.15.26:0/2743507799 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5588a3416cb0 0x5588a34115f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:57.675 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.670+0000 7f3a12ffd640 1 -- 172.21.15.26:0/2743507799 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5588a3494e10 con 0x5588a340baf0 2024-09-11T23:34:57.675 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.670+0000 7f3a12ffd640 1 --2- 172.21.15.26:0/2743507799 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5588a340baf0 0x5588a3410b70 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f3a0400c6a0 tx=0x7f3a0409b270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d78460d689d52f0f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:34:57.675 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.670+0000 7f39fa7fc640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:34:57.675 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.670+0000 7f39fa7fc640 1 -- 172.21.15.26:0/2743507799 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a040b8020 con 0x5588a340baf0 2024-09-11T23:34:57.675 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.670+0000 7f39fa7fc640 1 -- 172.21.15.26:0/2743507799 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 8542+0+0 (secure 0 0 0) 0x7f3a04010640 con 0x5588a340baf0 2024-09-11T23:34:57.675 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.670+0000 7f3a1c1c8740 1 -- 172.21.15.26:0/2743507799 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f39fc005020 con 0x5588a340baf0 2024-09-11T23:34:57.675 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.670+0000 7f39fa7fc640 1 -- 172.21.15.26:0/2743507799 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7f3a0409b550 con 0x5588a340baf0 2024-09-11T23:34:57.675 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.670+0000 7f3a1c1c8740 10 client.17655 fetch_fsmap learned FSMap version 76 2024-09-11T23:34:57.675 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.670+0000 7f3a1c1c8740 1 -- 172.21.15.26:0/2743507799 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) -- 0x5588a3496020 con 0x5588a340baf0 2024-09-11T23:34:57.675 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.670+0000 7f39fa7fc640 1 -- 172.21.15.26:0/2743507799 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f39fc005020 con 0x5588a340baf0 2024-09-11T23:34:57.676 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.670+0000 7f39fa7fc640 1 -- 172.21.15.26:0/2743507799 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 76) ==== 8438+0+0 (secure 0 0 0) 0x7f3a040ba340 con 0x5588a340baf0 2024-09-11T23:34:57.676 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.670+0000 7f3a1c1c8740 10 client.17655 fetch_fsmap finished waiting for FSMap version 76 2024-09-11T23:34:57.676 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.670+0000 7f3a1c1c8740 10 client.17655 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:34:57.676 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.671+0000 7f3a1c1c8740 20 client.17655 populate_metadata read hostname 'smithi026' 2024-09-11T23:34:57.676 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.671+0000 7f3a1c1c8740 1 --2- 172.21.15.26:0/2743507799 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5588a3496830 0x5588a3498c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:57.676 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.671+0000 7f3a1c1c8740 4 client.17655 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T23:34:57.676 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.671+0000 7f3a1c1c8740 1 -- 172.21.15.26:0/2743507799 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5588a3377d70 con 0x5588a3496830 2024-09-11T23:34:57.676 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.671+0000 7f3a0bfff640 1 --2- 172.21.15.26:0/2743507799 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5588a3496830 0x5588a3498c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:34:57.676 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.671+0000 7f3a0bfff640 1 --2- 172.21.15.26:0/2743507799 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5588a3496830 0x5588a3498c50 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6aba6e0ac034e441 server_cookie=4b4794a5e3c33dcd in_seq=0 out_seq=0 2024-09-11T23:34:57.676 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:57.671+0000 7f39fa7fc640 10 client.17655 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:34:58.238 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:34:57 smithi136 ceph-mon[35121]: pgmap v520: 97 pgs: 97 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.1 MiB/s wr, 321 op/s 2024-09-11T23:34:58.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:34:57 smithi026 ceph-mon[31172]: pgmap v520: 97 pgs: 97 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.1 MiB/s wr, 321 op/s 2024-09-11T23:34:58.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:34:57 smithi076 ceph-mon[35296]: pgmap v520: 97 pgs: 97 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.1 MiB/s wr, 321 op/s 2024-09-11T23:34:58.552 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:58.551+0000 7f39fa7fc640 1 -- 172.21.15.26:0/2743507799 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5588a3377d70 con 0x5588a3496830 2024-09-11T23:34:58.552 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:58.551+0000 7f39fa7fc640 10 client.17655 handle_command_reply: tid=0 2024-09-11T23:34:58.552 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:58.551+0000 7f3a1c1c8740 1 client.17655 shutdown 2024-09-11T23:34:58.552 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:58.551+0000 7f3a1c1c8740 1 -- 172.21.15.26:0/2743507799 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5588a340baf0 msgr2=0x5588a3410b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:34:58.552 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:58.551+0000 7f3a1c1c8740 1 --2- 172.21.15.26:0/2743507799 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5588a340baf0 0x5588a3410b70 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f3a0400c6a0 tx=0x7f3a0409b270 comp rx=0 tx=0).stop 2024-09-11T23:34:58.552 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:58.552+0000 7f3a1c1c8740 1 -- 172.21.15.26:0/2743507799 shutdown_connections 2024-09-11T23:34:58.552 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:58.552+0000 7f3a1c1c8740 1 --2- 172.21.15.26:0/2743507799 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5588a3416cb0 0x5588a34115f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:58.552 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:58.552+0000 7f3a1c1c8740 1 --2- 172.21.15.26:0/2743507799 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5588a3496830 0x5588a3498c50 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:58.553 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:58.552+0000 7f3a1c1c8740 1 --2- 172.21.15.26:0/2743507799 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5588a340c410 0x5588a34110b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:58.553 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:58.552+0000 7f39fa7fc640 0 client.17655 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:34:58.553 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:58.552+0000 7f3a1c1c8740 1 --2- 172.21.15.26:0/2743507799 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5588a340baf0 0x5588a3410b70 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:58.553 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:58.552+0000 7f3a1c1c8740 1 -- 172.21.15.26:0/2743507799 >> 172.21.15.26:0/2743507799 conn(0x5588a3408060 msgr2=0x5588a340d550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:34:58.553 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:58.552+0000 7f3a1c1c8740 1 -- 172.21.15.26:0/2743507799 shutdown_connections 2024-09-11T23:34:58.553 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:58.552+0000 7f3a1c1c8740 1 -- 172.21.15.26:0/2743507799 wait complete. 2024-09-11T23:34:58.553 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:58.552+0000 7f3a1c1c8740 20 client.17655 trim_cache size 0 max 16384 2024-09-11T23:34:58.554 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-09-11T23:34:58.554 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "message": "", 2024-09-11T23:34:58.554 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-09-11T23:34:58.554 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-09-11T23:34:58.555 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:58.553+0000 7f3a19430640 1 -- 172.21.15.26:0/339381116 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f3a000a2e20 msgr2=0x7f3a000a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:34:58.555 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:58.553+0000 7f3a19430640 1 --2- 172.21.15.26:0/339381116 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f3a000a2e20 0x7f3a000a52e0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f3a0c04c000 tx=0x7f3a0c04d000 comp rx=0 tx=0).stop 2024-09-11T23:34:58.555 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:58.553+0000 7f3a19430640 1 -- 172.21.15.26:0/339381116 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a141009b0 msgr2=0x7f3a141982d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:34:58.555 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:58.553+0000 7f3a19430640 1 --2- 172.21.15.26:0/339381116 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a141009b0 0x7f3a141982d0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f3a0400e9f0 tx=0x7f3a0400eeb0 comp rx=0 tx=0).stop 2024-09-11T23:34:58.555 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:58.553+0000 7f3a19430640 1 -- 172.21.15.26:0/339381116 shutdown_connections 2024-09-11T23:34:58.555 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:58.553+0000 7f3a19430640 1 --2- 172.21.15.26:0/339381116 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f3a000a2e20 0x7f3a000a52e0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:58.555 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:58.553+0000 7f3a19430640 1 --2- 172.21.15.26:0/339381116 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3a14101350 0x7f3a14198810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:58.555 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:58.553+0000 7f3a19430640 1 --2- 172.21.15.26:0/339381116 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3a141069c0 0x7f3a1419cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:58.555 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:58.553+0000 7f3a19430640 1 --2- 172.21.15.26:0/339381116 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a141009b0 0x7f3a141982d0 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:58.555 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:58.553+0000 7f3a19430640 1 -- 172.21.15.26:0/339381116 >> 172.21.15.26:0/339381116 conn(0x7f3a14077990 msgr2=0x7f3a14069a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:34:58.555 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:58.553+0000 7f3a19430640 1 -- 172.21.15.26:0/339381116 shutdown_connections 2024-09-11T23:34:58.555 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:34:58.553+0000 7f3a19430640 1 -- 172.21.15.26:0/339381116 wait complete. 2024-09-11T23:35:00.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:34:59 smithi026 ceph-mon[31172]: pgmap v521: 97 pgs: 97 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.9 MiB/s wr, 288 op/s 2024-09-11T23:35:00.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:34:59 smithi026 ceph-mon[31172]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:35:00.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:34:59 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:35:00.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:34:59 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:35:00.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:34:59 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:35:00.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:34:59 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:35:00.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:34:59 smithi076 ceph-mon[35296]: pgmap v521: 97 pgs: 97 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.9 MiB/s wr, 288 op/s 2024-09-11T23:35:00.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:34:59 smithi076 ceph-mon[35296]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:35:00.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:34:59 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:35:00.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:34:59 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:35:00.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:34:59 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:35:00.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:34:59 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:35:00.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:34:59 smithi136 ceph-mon[35121]: pgmap v521: 97 pgs: 97 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.9 MiB/s wr, 288 op/s 2024-09-11T23:35:00.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:34:59 smithi136 ceph-mon[35121]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:35:00.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:34:59 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:35:00.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:34:59 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:35:00.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:34:59 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:35:00.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:34:59 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:35:02.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:35:01 smithi026 ceph-mon[31172]: pgmap v522: 97 pgs: 97 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.2 MiB/s wr, 458 op/s 2024-09-11T23:35:02.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:35:01 smithi076 ceph-mon[35296]: pgmap v522: 97 pgs: 97 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.2 MiB/s wr, 458 op/s 2024-09-11T23:35:02.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:35:01 smithi136 ceph-mon[35121]: pgmap v522: 97 pgs: 97 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.2 MiB/s wr, 458 op/s 2024-09-11T23:35:04.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:35:04 smithi076 ceph-mon[35296]: pgmap v523: 97 pgs: 97 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 407 op/s 2024-09-11T23:35:04.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:35:04 smithi136 ceph-mon[35121]: pgmap v523: 97 pgs: 97 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 407 op/s 2024-09-11T23:35:04.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:35:04 smithi026 ceph-mon[31172]: pgmap v523: 97 pgs: 97 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 407 op/s 2024-09-11T23:35:05.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:35:05 smithi076 ceph-mon[35296]: osdmap e107: 12 total, 12 up, 12 in 2024-09-11T23:35:05.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:35:05 smithi136 ceph-mon[35121]: osdmap e107: 12 total, 12 up, 12 in 2024-09-11T23:35:05.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:35:05 smithi026 ceph-mon[31172]: osdmap e107: 12 total, 12 up, 12 in 2024-09-11T23:35:06.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:35:06 smithi076 ceph-mon[35296]: pgmap v525: 97 pgs: 4 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 86 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 452 op/s 2024-09-11T23:35:06.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:35:06 smithi136 ceph-mon[35121]: pgmap v525: 97 pgs: 4 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 86 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 452 op/s 2024-09-11T23:35:06.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:35:06 smithi026 ceph-mon[31172]: pgmap v525: 97 pgs: 4 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 86 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 452 op/s 2024-09-11T23:35:08.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:35:08 smithi076 ceph-mon[35296]: pgmap v526: 97 pgs: 14 active+clean+snaptrim, 26 active+clean+snaptrim_wait, 57 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.0 MiB/s wr, 440 op/s 2024-09-11T23:35:08.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:35:08 smithi136 ceph-mon[35121]: pgmap v526: 97 pgs: 14 active+clean+snaptrim, 26 active+clean+snaptrim_wait, 57 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.0 MiB/s wr, 440 op/s 2024-09-11T23:35:08.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:35:08 smithi026 ceph-mon[31172]: pgmap v526: 97 pgs: 14 active+clean+snaptrim, 26 active+clean+snaptrim_wait, 57 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.0 MiB/s wr, 440 op/s 2024-09-11T23:35:10.301 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:35:10 smithi026 ceph-mon[31172]: pgmap v527: 97 pgs: 16 active+clean+snaptrim, 28 active+clean+snaptrim_wait, 53 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.0 MiB/s wr, 426 op/s 2024-09-11T23:35:10.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:35:10 smithi076 ceph-mon[35296]: pgmap v527: 97 pgs: 16 active+clean+snaptrim, 28 active+clean+snaptrim_wait, 53 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.0 MiB/s wr, 426 op/s 2024-09-11T23:35:10.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:35:10 smithi136 ceph-mon[35121]: pgmap v527: 97 pgs: 16 active+clean+snaptrim, 28 active+clean+snaptrim_wait, 53 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.0 MiB/s wr, 426 op/s 2024-09-11T23:35:11.700 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:11.699+0000 7f0b3555f640 1 -- 172.21.15.26:0/2833693639 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0b301069e0 msgr2=0x7f0b30106dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:11.701 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:11.699+0000 7f0b3555f640 1 --2- 172.21.15.26:0/2833693639 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0b301069e0 0x7f0b30106dc0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f0b20009ca0 tx=0x7f0b2002a8c0 comp rx=0 tx=0).stop 2024-09-11T23:35:11.701 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:11.700+0000 7f0b3555f640 1 -- 172.21.15.26:0/2833693639 shutdown_connections 2024-09-11T23:35:11.701 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:11.700+0000 7f0b3555f640 1 --2- 172.21.15.26:0/2833693639 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0b30101370 0x7f0b3010d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:11.701 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:11.700+0000 7f0b3555f640 1 --2- 172.21.15.26:0/2833693639 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0b301009d0 0x7f0b30100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:11.701 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:11.700+0000 7f0b3555f640 1 --2- 172.21.15.26:0/2833693639 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0b301069e0 0x7f0b30106dc0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:11.701 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:11.700+0000 7f0b3555f640 1 -- 172.21.15.26:0/2833693639 >> 172.21.15.26:0/2833693639 conn(0x7f0b300fc6b0 msgr2=0x7f0b300fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:35:11.701 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:11.700+0000 7f0b3555f640 1 -- 172.21.15.26:0/2833693639 shutdown_connections 2024-09-11T23:35:11.701 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:11.700+0000 7f0b3555f640 1 -- 172.21.15.26:0/2833693639 wait complete. 2024-09-11T23:35:11.701 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:11.701+0000 7f0b3555f640 1 Processor -- start 2024-09-11T23:35:11.704 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:11.701+0000 7f0b3555f640 1 -- start start 2024-09-11T23:35:11.704 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:11.701+0000 7f0b3555f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0b301009d0 0x7f0b301a34e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:11.704 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:11.701+0000 7f0b3555f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0b30101370 0x7f0b301a3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:11.704 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:11.701+0000 7f0b3555f640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0b301069e0 0x7f0b3010c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:11.705 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:11.701+0000 7f0b3555f640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f0b30111ef0 con 0x7f0b301009d0 2024-09-11T23:35:11.705 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:11.701+0000 7f0b3555f640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f0b30111d70 con 0x7f0b30101370 2024-09-11T23:35:11.705 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:11.701+0000 7f0b3555f640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f0b30112070 con 0x7f0b301069e0 2024-09-11T23:35:11.705 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:11.701+0000 7f0b27fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0b30101370 0x7f0b301a3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:11.705 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:11.702+0000 7f0b2effd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0b301009d0 0x7f0b301a34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:11.705 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:11.702+0000 7f0b2effd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0b301009d0 0x7f0b301a34e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42744/0 (socket says 172.21.15.26:42744) 2024-09-11T23:35:11.705 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:11.702+0000 7f0b2effd640 1 -- 172.21.15.26:0/2311100193 learned_addr learned my addr 172.21.15.26:0/2311100193 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:35:11.705 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:11.702+0000 7f0b2effd640 1 -- 172.21.15.26:0/2311100193 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0b301069e0 msgr2=0x7f0b3010c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:11.705 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:11.702+0000 7f0b2effd640 1 --2- 172.21.15.26:0/2311100193 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0b301069e0 0x7f0b3010c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:11.705 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:11.702+0000 7f0b2effd640 1 -- 172.21.15.26:0/2311100193 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0b30101370 msgr2=0x7f0b301a3a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:11.705 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:11.702+0000 7f0b2effd640 1 --2- 172.21.15.26:0/2311100193 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0b30101370 0x7f0b301a3a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:11.705 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:11.702+0000 7f0b2effd640 1 -- 172.21.15.26:0/2311100193 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b3010d110 con 0x7f0b301009d0 2024-09-11T23:35:11.705 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:11.703+0000 7f0b2effd640 1 --2- 172.21.15.26:0/2311100193 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0b301009d0 0x7f0b301a34e0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f0b20030f80 tx=0x7f0b20030fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:35:11.706 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:11.703+0000 7f0b2cff9640 1 -- 172.21.15.26:0/2311100193 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b200319e0 con 0x7f0b301009d0 2024-09-11T23:35:11.706 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:11.703+0000 7f0b3555f640 1 -- 172.21.15.26:0/2311100193 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b3010d340 con 0x7f0b301009d0 2024-09-11T23:35:11.706 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:11.703+0000 7f0b3555f640 1 -- 172.21.15.26:0/2311100193 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b30197360 con 0x7f0b301009d0 2024-09-11T23:35:11.706 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:11.703+0000 7f0b2cff9640 1 -- 172.21.15.26:0/2311100193 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b2003b8e0 con 0x7f0b301009d0 2024-09-11T23:35:11.706 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:11.704+0000 7f0b2cff9640 1 -- 172.21.15.26:0/2311100193 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b2003aad0 con 0x7f0b301009d0 2024-09-11T23:35:11.706 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:11.706+0000 7f0b383d6740 1 Processor -- start 2024-09-11T23:35:11.707 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:11.707+0000 7f0b383d6740 1 -- start start 2024-09-11T23:35:11.707 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:11.707+0000 7f0b383d6740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fa03cb3af0 0x55fa03cb3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:11.707 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:11.707+0000 7f0b383d6740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fa03cb4410 0x55fa03cbe770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:11.707 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:11.707+0000 7f0b383d6740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fa03cbecb0 0x55fa03cc10a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:11.707 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:11.707+0000 7f0b383d6740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55fa03c182f0 con 0x55fa03cbecb0 2024-09-11T23:35:11.707 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:11.707+0000 7f0b383d6740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55fa03bc0270 con 0x55fa03cb3af0 2024-09-11T23:35:11.707 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:11.707+0000 7f0b383d6740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55fa03c142b0 con 0x55fa03cb4410 2024-09-11T23:35:11.707 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:11.707+0000 7f0b2f7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fa03cb3af0 0x55fa03cb3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:11.707 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:11.707+0000 7f0b2f7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fa03cb3af0 0x55fa03cb3ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:60500/0 (socket says 172.21.15.26:60500) 2024-09-11T23:35:11.707 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:11.707+0000 7f0b2f7fe640 1 -- 172.21.15.26:0/4123564415 learned_addr learned my addr 172.21.15.26:0/4123564415 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:35:11.708 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:11.708+0000 7f0b27fff640 1 --2- 172.21.15.26:0/4123564415 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fa03cbecb0 0x55fa03cc10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:11.708 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:11.708+0000 7f0b27fff640 1 -- 172.21.15.26:0/4123564415 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fa03cb4410 msgr2=0x55fa03cbe770 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T23:35:11.708 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:11.708+0000 7f0b27fff640 1 --2- 172.21.15.26:0/4123564415 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fa03cb4410 0x55fa03cbe770 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:11.708 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:11.708+0000 7f0b27fff640 1 -- 172.21.15.26:0/4123564415 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fa03cb3af0 msgr2=0x55fa03cb3ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:35:11.708 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:11.708+0000 7f0b27fff640 1 --2- 172.21.15.26:0/4123564415 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fa03cb3af0 0x55fa03cb3ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:11.708 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:11.708+0000 7f0b27fff640 1 -- 172.21.15.26:0/4123564415 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa03bbf340 con 0x55fa03cbecb0 2024-09-11T23:35:11.709 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:11.708+0000 7f0b2cff9640 1 -- 172.21.15.26:0/2311100193 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f0b2003ac70 con 0x7f0b301009d0 2024-09-11T23:35:11.709 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:11.709+0000 7f0b27fff640 1 --2- 172.21.15.26:0/4123564415 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fa03cbecb0 0x55fa03cc10a0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f0b3019ff00 tx=0x7f0b1800f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e3caa04c88a151d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:35:11.709 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:11.710+0000 7f0b2cff9640 1 --2- 172.21.15.26:0/2311100193 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f0b1c0a2ea0 0x7f0b1c0a5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:11.709 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:11.710+0000 7f0b2cff9640 1 -- 172.21.15.26:0/2311100193 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 8586+0+0 (secure 0 0 0) 0x7f0b200f0be0 con 0x7f0b301009d0 2024-09-11T23:35:11.710 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:11.710+0000 7f0b2f7fe640 1 --2- 172.21.15.26:0/2311100193 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f0b1c0a2ea0 0x7f0b1c0a5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:11.710 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:11.711+0000 7f0b257fa640 1 -- 172.21.15.26:0/4123564415 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b18005220 con 0x55fa03cbecb0 2024-09-11T23:35:11.710 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:11.711+0000 7f0b257fa640 1 -- 172.21.15.26:0/4123564415 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b18004c80 con 0x55fa03cbecb0 2024-09-11T23:35:11.710 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:11.711+0000 7f0b257fa640 1 -- 172.21.15.26:0/4123564415 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b18018880 con 0x55fa03cbecb0 2024-09-11T23:35:11.711 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:11.712+0000 7f0b383d6740 1 -- 172.21.15.26:0/4123564415 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fa03cbecb0 msgr2=0x55fa03cc10a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:11.711 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:11.712+0000 7f0b383d6740 1 --2- 172.21.15.26:0/4123564415 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fa03cbecb0 0x55fa03cc10a0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f0b3019ff00 tx=0x7f0b1800f790 comp rx=0 tx=0).stop 2024-09-11T23:35:11.712 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:11.712+0000 7f0b2f7fe640 1 --2- 172.21.15.26:0/2311100193 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f0b1c0a2ea0 0x7f0b1c0a5360 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f0b28005fd0 tx=0x7f0b28005ea0 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:35:11.712 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:11.713+0000 7f0b383d6740 1 -- 172.21.15.26:0/4123564415 shutdown_connections 2024-09-11T23:35:11.712 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:11.713+0000 7f0b383d6740 1 --2- 172.21.15.26:0/4123564415 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fa03cbecb0 0x55fa03cc10a0 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:11.712 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:11.713+0000 7f0b383d6740 1 --2- 172.21.15.26:0/4123564415 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fa03cb4410 0x55fa03cbe770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:11.712 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:11.713+0000 7f0b383d6740 1 --2- 172.21.15.26:0/4123564415 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fa03cb3af0 0x55fa03cb3ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:11.712 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:11.713+0000 7f0b383d6740 1 -- 172.21.15.26:0/4123564415 >> 172.21.15.26:0/4123564415 conn(0x55fa03cb0060 msgr2=0x55fa03cb0440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:35:11.713 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:11.713+0000 7f0b383d6740 1 -- 172.21.15.26:0/4123564415 shutdown_connections 2024-09-11T23:35:11.713 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:11.713+0000 7f0b383d6740 1 -- 172.21.15.26:0/4123564415 wait complete. 2024-09-11T23:35:11.713 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:11.713+0000 7f0b383d6740 1 -- start start 2024-09-11T23:35:11.713 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:11.714+0000 7f0b383d6740 1 Processor -- start 2024-09-11T23:35:11.714 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:11.714+0000 7f0b383d6740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fa03cb3af0 0x55fa03cbbbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:11.714 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:11.714+0000 7f0b383d6740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fa03cb4410 0x55fa03cbc120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:11.714 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:11.714+0000 7f0b383d6740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fa03cbecb0 0x55fa03cbc660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:11.714 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:11.714+0000 7f0b383d6740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55fa03cc1880 con 0x55fa03cb4410 2024-09-11T23:35:11.714 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:11.714+0000 7f0b383d6740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55fa03ba1290 con 0x55fa03cb3af0 2024-09-11T23:35:11.714 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:11.714+0000 7f0b383d6740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55fa03cc1a00 con 0x55fa03cbecb0 2024-09-11T23:35:11.714 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:11.714+0000 7f0b383d6740 10 client.0 _ll_register_callbacks cb 0x55fa03bd2140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:35:11.715 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:11.715+0000 7f0b27fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fa03cb4410 0x55fa03cbc120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:11.715 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:11.715+0000 7f0b2f7fe640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fa03cbecb0 0x55fa03cbc660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:11.715 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:11.715+0000 7f0b2f7fe640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fa03cbecb0 0x55fa03cbc660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:56006/0 (socket says 172.21.15.26:56006) 2024-09-11T23:35:11.715 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:11.715+0000 7f0b2f7fe640 1 -- 172.21.15.26:0/595864092 learned_addr learned my addr 172.21.15.26:0/595864092 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:35:11.715 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:11.715+0000 7f0b2effd640 1 --2- 172.21.15.26:0/595864092 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fa03cb3af0 0x55fa03cbbbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:11.716 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:11.715+0000 7f0b2f7fe640 1 -- 172.21.15.26:0/595864092 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fa03cb3af0 msgr2=0x55fa03cbbbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:35:11.716 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:11.715+0000 7f0b2f7fe640 1 --2- 172.21.15.26:0/595864092 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fa03cb3af0 0x55fa03cbbbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:11.716 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:11.715+0000 7f0b2f7fe640 1 -- 172.21.15.26:0/595864092 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fa03cb4410 msgr2=0x55fa03cbc120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:35:11.716 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:11.715+0000 7f0b2f7fe640 1 --2- 172.21.15.26:0/595864092 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fa03cb4410 0x55fa03cbc120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:11.716 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:11.715+0000 7f0b2f7fe640 1 -- 172.21.15.26:0/595864092 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa0392b210 con 0x55fa03cbecb0 2024-09-11T23:35:11.716 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:11.716+0000 7f0b2f7fe640 1 --2- 172.21.15.26:0/595864092 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fa03cbecb0 0x55fa03cbc660 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f0b280109b0 tx=0x7f0b28005890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aa5143cdd1327dcb server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:35:11.717 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:11.717+0000 7f0b167fc640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-09-11T23:35:11.717 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:11.718+0000 7f0b167fc640 1 -- 172.21.15.26:0/595864092 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b28021070 con 0x55fa03cbecb0 2024-09-11T23:35:11.717 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:11.718+0000 7f0b383d6740 1 -- 172.21.15.26:0/595864092 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0b18004c80 con 0x55fa03cbecb0 2024-09-11T23:35:11.717 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:11.718+0000 7f0b167fc640 1 -- 172.21.15.26:0/595864092 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 8586+0+0 (secure 0 0 0) 0x7f0b2800fb40 con 0x55fa03cbecb0 2024-09-11T23:35:11.718 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:11.719+0000 7f0b167fc640 1 -- 172.21.15.26:0/595864092 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7f0b2802d050 con 0x55fa03cbecb0 2024-09-11T23:35:11.718 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:11.719+0000 7f0b383d6740 10 client.27154 fetch_fsmap learned FSMap version 77 2024-09-11T23:35:11.718 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:11.719+0000 7f0b383d6740 1 -- 172.21.15.26:0/595864092 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=77,monmap=4+}) -- 0x55fa039f7620 con 0x55fa03cbecb0 2024-09-11T23:35:11.721 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:11.721+0000 7f0b167fc640 1 -- 172.21.15.26:0/595864092 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b18004c80 con 0x55fa03cbecb0 2024-09-11T23:35:11.721 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:11.722+0000 7f0b167fc640 1 -- 172.21.15.26:0/595864092 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 77) ==== 8434+0+0 (secure 0 0 0) 0x7f0b280253b0 con 0x55fa03cbecb0 2024-09-11T23:35:11.722 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:11.722+0000 7f0b383d6740 10 client.27154 fetch_fsmap finished waiting for FSMap version 77 2024-09-11T23:35:11.722 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:11.722+0000 7f0b383d6740 10 client.27154 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:35:11.722 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:11.722+0000 7f0b383d6740 20 client.27154 populate_metadata read hostname 'smithi026' 2024-09-11T23:35:11.722 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:11.722+0000 7f0b383d6740 1 --2- 172.21.15.26:0/595864092 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55fa03ccf260 0x55fa03cd16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:11.722 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:11.722+0000 7f0b383d6740 4 client.27154 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:35:11.722 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:11.722+0000 7f0b383d6740 1 -- 172.21.15.26:0/595864092 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fa03c25a20 con 0x55fa03ccf260 2024-09-11T23:35:11.722 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:11.723+0000 7f0b2effd640 1 --2- 172.21.15.26:0/595864092 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55fa03ccf260 0x55fa03cd16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:11.730 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:11.731+0000 7f0b2effd640 1 --2- 172.21.15.26:0/595864092 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55fa03ccf260 0x55fa03cd16e0 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5954b468aacf4395 server_cookie=a48797df71ebfb10 in_seq=0 out_seq=0 2024-09-11T23:35:11.730 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:11.731+0000 7f0b167fc640 10 client.27154 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:35:12.461 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:12.461+0000 7f0b167fc640 1 -- 172.21.15.26:0/595864092 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fa03c25a20 con 0x55fa03ccf260 2024-09-11T23:35:12.461 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:12.461+0000 7f0b167fc640 10 client.27154 handle_command_reply: tid=0 2024-09-11T23:35:12.461 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:12.461+0000 7f0b383d6740 1 client.27154 shutdown 2024-09-11T23:35:12.461 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:12.461+0000 7f0b383d6740 1 -- 172.21.15.26:0/595864092 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fa03cbecb0 msgr2=0x55fa03cbc660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:12.461 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:12.461+0000 7f0b383d6740 1 --2- 172.21.15.26:0/595864092 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fa03cbecb0 0x55fa03cbc660 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f0b280109b0 tx=0x7f0b28005890 comp rx=0 tx=0).stop 2024-09-11T23:35:12.462 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:12.461+0000 7f0b383d6740 1 -- 172.21.15.26:0/595864092 shutdown_connections 2024-09-11T23:35:12.462 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:12.461+0000 7f0b383d6740 1 --2- 172.21.15.26:0/595864092 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55fa03ccf260 0x55fa03cd16e0 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:12.462 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:12.461+0000 7f0b383d6740 1 --2- 172.21.15.26:0/595864092 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fa03cbecb0 0x55fa03cbc660 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:12.462 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:12.461+0000 7f0b383d6740 1 --2- 172.21.15.26:0/595864092 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fa03cb4410 0x55fa03cbc120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:12.462 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:12.461+0000 7f0b383d6740 1 --2- 172.21.15.26:0/595864092 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fa03cb3af0 0x55fa03cbbbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:12.462 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:12.461+0000 7f0b383d6740 1 -- 172.21.15.26:0/595864092 >> 172.21.15.26:0/595864092 conn(0x55fa03cb0060 msgr2=0x55fa03cb5d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:35:12.462 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:12.462+0000 7f0b167fc640 0 client.27154 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:35:12.462 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:12.462+0000 7f0b383d6740 1 -- 172.21.15.26:0/595864092 shutdown_connections 2024-09-11T23:35:12.462 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:12.462+0000 7f0b383d6740 1 -- 172.21.15.26:0/595864092 wait complete. 2024-09-11T23:35:12.462 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:12.462+0000 7f0b383d6740 20 client.27154 trim_cache size 0 max 16384 2024-09-11T23:35:12.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:35:12 smithi136 ceph-mon[35121]: pgmap v528: 97 pgs: 23 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 33 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.6 MiB/s wr, 277 op/s 2024-09-11T23:35:12.489 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:12.490+0000 7f0b383d6740 1 Processor -- start 2024-09-11T23:35:12.489 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:12.490+0000 7f0b383d6740 1 -- start start 2024-09-11T23:35:12.490 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:12.491+0000 7f0b383d6740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fa03cb3af0 0x55fa03cbc490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:12.491 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:12.491+0000 7f0b383d6740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fa03cb4410 0x55fa03cbdb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:12.491 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:12.491+0000 7f0b383d6740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fa03cbecb0 0x55fa03cd6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:12.491 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:12.491+0000 7f0b383d6740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f0b1000e930 con 0x55fa03cbecb0 2024-09-11T23:35:12.491 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:12.491+0000 7f0b383d6740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f0b10010740 con 0x55fa03cb4410 2024-09-11T23:35:12.491 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:12.491+0000 7f0b383d6740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f0b1000f0c0 con 0x55fa03cb3af0 2024-09-11T23:35:12.491 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:12.491+0000 7f0b27fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fa03cb4410 0x55fa03cbdb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:12.491 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:12.491+0000 7f0b27fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fa03cb4410 0x55fa03cbdb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:60528/0 (socket says 172.21.15.26:60528) 2024-09-11T23:35:12.491 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:12.491+0000 7f0b27fff640 1 -- 172.21.15.26:0/1999035985 learned_addr learned my addr 172.21.15.26:0/1999035985 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:35:12.491 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:12.491+0000 7f0b2effd640 1 --2- 172.21.15.26:0/1999035985 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fa03cb3af0 0x55fa03cbc490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:12.492 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:12.492+0000 7f0b2f7fe640 1 --2- 172.21.15.26:0/1999035985 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fa03cbecb0 0x55fa03cd6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:12.492 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:12.492+0000 7f0b27fff640 1 -- 172.21.15.26:0/1999035985 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fa03cb3af0 msgr2=0x55fa03cbc490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:35:12.493 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:12.492+0000 7f0b27fff640 1 --2- 172.21.15.26:0/1999035985 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fa03cb3af0 0x55fa03cbc490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:12.493 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:12.492+0000 7f0b27fff640 1 -- 172.21.15.26:0/1999035985 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fa03cbecb0 msgr2=0x55fa03cd6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:35:12.493 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:12.492+0000 7f0b27fff640 1 --2- 172.21.15.26:0/1999035985 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fa03cbecb0 0x55fa03cd6ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:12.493 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:12.492+0000 7f0b27fff640 1 -- 172.21.15.26:0/1999035985 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa03cb11d0 con 0x55fa03cb4410 2024-09-11T23:35:12.493 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:12.493+0000 7f0b27fff640 1 --2- 172.21.15.26:0/1999035985 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fa03cb4410 0x55fa03cbdb50 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f0b1800a6b0 tx=0x7f0b1800a910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=24ed6702a856b766 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:35:12.493 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:12.493+0000 7f0b24ff9640 1 -- 172.21.15.26:0/1999035985 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b18027070 con 0x55fa03cb4410 2024-09-11T23:35:12.493 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:12.493+0000 7f0b24ff9640 1 -- 172.21.15.26:0/1999035985 <== mon.2 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b180055e0 con 0x55fa03cb4410 2024-09-11T23:35:12.493 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:12.493+0000 7f0b24ff9640 1 -- 172.21.15.26:0/1999035985 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b18004e60 con 0x55fa03cb4410 2024-09-11T23:35:12.494 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:12.494+0000 7f0b383d6740 1 -- 172.21.15.26:0/1999035985 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fa03cb4410 msgr2=0x55fa03cbdb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:12.494 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:12.494+0000 7f0b383d6740 1 --2- 172.21.15.26:0/1999035985 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fa03cb4410 0x55fa03cbdb50 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f0b1800a6b0 tx=0x7f0b1800a910 comp rx=0 tx=0).stop 2024-09-11T23:35:12.494 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:12.495+0000 7f0b383d6740 1 -- 172.21.15.26:0/1999035985 shutdown_connections 2024-09-11T23:35:12.494 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:12.495+0000 7f0b383d6740 1 --2- 172.21.15.26:0/1999035985 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fa03cb4410 0x55fa03cbdb50 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:12.495 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:12.495+0000 7f0b383d6740 1 --2- 172.21.15.26:0/1999035985 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fa03cbecb0 0x55fa03cd6ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:12.495 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:12.495+0000 7f0b383d6740 1 --2- 172.21.15.26:0/1999035985 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fa03cb3af0 0x55fa03cbc490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:12.495 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:12.495+0000 7f0b383d6740 1 -- 172.21.15.26:0/1999035985 >> 172.21.15.26:0/1999035985 conn(0x55fa03cb0060 msgr2=0x55fa03cbd440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:35:12.498 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:12.498+0000 7f0b2effd640 1 --2- 172.21.15.26:0/1999035985 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fa03cb3af0 0x55fa03cbc490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:35:12.498 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:12.499+0000 7f0b383d6740 1 -- 172.21.15.26:0/1999035985 shutdown_connections 2024-09-11T23:35:12.498 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:12.499+0000 7f0b383d6740 1 -- 172.21.15.26:0/1999035985 wait complete. 2024-09-11T23:35:12.499 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:12.499+0000 7f0b383d6740 1 -- start start 2024-09-11T23:35:12.499 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:12.500+0000 7f0b383d6740 1 Processor -- start 2024-09-11T23:35:12.500 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:12.500+0000 7f0b383d6740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fa03cb4410 0x55fa03cc72e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:12.500 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:12.500+0000 7f0b383d6740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fa03cbecb0 0x55fa03cc2560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:12.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:35:12 smithi026 ceph-mon[31172]: pgmap v528: 97 pgs: 23 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 33 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.6 MiB/s wr, 277 op/s 2024-09-11T23:35:12.501 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:12.501+0000 7f0b383d6740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fa03cc2aa0 0x55fa03cc2f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:12.501 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:12.501+0000 7f0b383d6740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f0b1000ffb0 con 0x55fa03cbecb0 2024-09-11T23:35:12.502 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:12.501+0000 7f0b383d6740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f0b1000f820 con 0x55fa03cb4410 2024-09-11T23:35:12.502 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:12.501+0000 7f0b2f7fe640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fa03cc2aa0 0x55fa03cc2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:12.502 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:12.501+0000 7f0b2f7fe640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fa03cc2aa0 0x55fa03cc2f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:56018/0 (socket says 172.21.15.26:56018) 2024-09-11T23:35:12.502 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:12.501+0000 7f0b2f7fe640 1 -- 172.21.15.26:0/458986117 learned_addr learned my addr 172.21.15.26:0/458986117 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:35:12.502 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:12.501+0000 7f0b383d6740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f0b1000c930 con 0x55fa03cc2aa0 2024-09-11T23:35:12.502 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:12.501+0000 7f0b383d6740 10 client.0 _ll_register_callbacks cb 0x7f0b28014070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:35:12.502 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:12.502+0000 7f0b2f7fe640 1 -- 172.21.15.26:0/458986117 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fa03cb4410 msgr2=0x55fa03cc72e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T23:35:12.502 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:12.502+0000 7f0b2f7fe640 1 --2- 172.21.15.26:0/458986117 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fa03cb4410 0x55fa03cc72e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:12.502 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:12.502+0000 7f0b2f7fe640 1 -- 172.21.15.26:0/458986117 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fa03cbecb0 msgr2=0x55fa03cc2560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:35:12.502 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:12.502+0000 7f0b2f7fe640 1 --2- 172.21.15.26:0/458986117 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fa03cbecb0 0x55fa03cc2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:12.502 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:12.502+0000 7f0b2f7fe640 1 -- 172.21.15.26:0/458986117 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa03cab2d0 con 0x55fa03cc2aa0 2024-09-11T23:35:12.502 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:12.502+0000 7f0b2f7fe640 1 --2- 172.21.15.26:0/458986117 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fa03cc2aa0 0x55fa03cc2f20 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f0b28000c00 tx=0x7f0b280255e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7346e50c99b5599a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:35:12.503 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:12.502+0000 7f0b167fc640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-09-11T23:35:12.503 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:12.502+0000 7f0b167fc640 1 -- 172.21.15.26:0/458986117 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b28021050 con 0x55fa03cc2aa0 2024-09-11T23:35:12.503 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:12.503+0000 7f0b383d6740 1 -- 172.21.15.26:0/458986117 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0b180055e0 con 0x55fa03cc2aa0 2024-09-11T23:35:12.503 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:12.503+0000 7f0b167fc640 1 -- 172.21.15.26:0/458986117 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 8586+0+0 (secure 0 0 0) 0x7f0b28025ba0 con 0x55fa03cc2aa0 2024-09-11T23:35:12.505 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:12.504+0000 7f0b167fc640 1 -- 172.21.15.26:0/458986117 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7f0b28025e20 con 0x55fa03cc2aa0 2024-09-11T23:35:12.507 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:12.508+0000 7f0b383d6740 10 client.27157 fetch_fsmap learned FSMap version 77 2024-09-11T23:35:12.507 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:12.508+0000 7f0b383d6740 1 -- 172.21.15.26:0/458986117 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=77,monmap=4+}) -- 0x55fa03cbe770 con 0x55fa03cc2aa0 2024-09-11T23:35:12.510 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:12.510+0000 7f0b167fc640 1 -- 172.21.15.26:0/458986117 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b180055e0 con 0x55fa03cc2aa0 2024-09-11T23:35:12.510 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:12.510+0000 7f0b167fc640 1 -- 172.21.15.26:0/458986117 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 77) ==== 8434+0+0 (secure 0 0 0) 0x7f0b280047a0 con 0x55fa03cc2aa0 2024-09-11T23:35:12.510 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:12.510+0000 7f0b383d6740 10 client.27157 fetch_fsmap finished waiting for FSMap version 77 2024-09-11T23:35:12.510 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:12.510+0000 7f0b383d6740 10 client.27157 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:35:12.510 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:12.510+0000 7f0b383d6740 20 client.27157 populate_metadata read hostname 'smithi026' 2024-09-11T23:35:12.510 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:12.510+0000 7f0b383d6740 1 --2- 172.21.15.26:0/458986117 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55fa03d3a5b0 0x55fa03d3ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:12.510 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:12.510+0000 7f0b383d6740 4 client.27157 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T23:35:12.510 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:12.510+0000 7f0b383d6740 1 -- 172.21.15.26:0/458986117 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55fa03c1fd70 con 0x55fa03d3a5b0 2024-09-11T23:35:12.511 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:12.511+0000 7f0b2effd640 1 --2- 172.21.15.26:0/458986117 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55fa03d3a5b0 0x55fa03d3ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:12.511 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:12.512+0000 7f0b2effd640 1 --2- 172.21.15.26:0/458986117 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55fa03d3a5b0 0x55fa03d3ca70 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e0189bb35ee8c32 server_cookie=b0eee3c5e39eebea in_seq=0 out_seq=0 2024-09-11T23:35:12.511 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:12.512+0000 7f0b167fc640 10 client.27157 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:35:12.552 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:35:12 smithi076 ceph-mon[35296]: pgmap v528: 97 pgs: 23 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 33 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.6 MiB/s wr, 277 op/s 2024-09-11T23:35:12.983 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:12.981+0000 7f0b167fc640 1 -- 172.21.15.26:0/458986117 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55fa03c1fd70 con 0x55fa03d3a5b0 2024-09-11T23:35:12.983 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:12.981+0000 7f0b167fc640 10 client.27157 handle_command_reply: tid=0 2024-09-11T23:35:12.983 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:12.981+0000 7f0b383d6740 1 client.27157 shutdown 2024-09-11T23:35:12.983 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:12.982+0000 7f0b383d6740 1 -- 172.21.15.26:0/458986117 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fa03cc2aa0 msgr2=0x55fa03cc2f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:12.983 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:12.982+0000 7f0b383d6740 1 --2- 172.21.15.26:0/458986117 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fa03cc2aa0 0x55fa03cc2f20 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f0b28000c00 tx=0x7f0b280255e0 comp rx=0 tx=0).stop 2024-09-11T23:35:12.983 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:12.982+0000 7f0b383d6740 1 -- 172.21.15.26:0/458986117 shutdown_connections 2024-09-11T23:35:12.983 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:12.982+0000 7f0b383d6740 1 --2- 172.21.15.26:0/458986117 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55fa03d3a5b0 0x55fa03d3ca70 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:12.983 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:12.982+0000 7f0b167fc640 0 client.27157 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:35:12.983 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:12.982+0000 7f0b383d6740 1 --2- 172.21.15.26:0/458986117 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fa03cc2aa0 0x55fa03cc2f20 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:12.984 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:12.982+0000 7f0b383d6740 1 --2- 172.21.15.26:0/458986117 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fa03cbecb0 0x55fa03cc2560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:12.984 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:12.982+0000 7f0b383d6740 1 --2- 172.21.15.26:0/458986117 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fa03cb4410 0x55fa03cc72e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:12.984 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:12.982+0000 7f0b383d6740 1 -- 172.21.15.26:0/458986117 >> 172.21.15.26:0/458986117 conn(0x55fa03cb0060 msgr2=0x55fa03cb5550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:35:12.984 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:12.983+0000 7f0b383d6740 1 -- 172.21.15.26:0/458986117 shutdown_connections 2024-09-11T23:35:12.984 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:12.983+0000 7f0b383d6740 1 -- 172.21.15.26:0/458986117 wait complete. 2024-09-11T23:35:12.984 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:12.983+0000 7f0b383d6740 20 client.27157 trim_cache size 0 max 16384 2024-09-11T23:35:12.984 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-09-11T23:35:12.984 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "message": "", 2024-09-11T23:35:12.985 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-09-11T23:35:12.985 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-09-11T23:35:12.985 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:12.984+0000 7f0b3555f640 1 -- 172.21.15.26:0/2311100193 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f0b1c0a2ea0 msgr2=0x7f0b1c0a5360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:12.985 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:12.984+0000 7f0b3555f640 1 --2- 172.21.15.26:0/2311100193 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f0b1c0a2ea0 0x7f0b1c0a5360 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f0b28005fd0 tx=0x7f0b28005ea0 comp rx=0 tx=0).stop 2024-09-11T23:35:12.985 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:12.984+0000 7f0b3555f640 1 -- 172.21.15.26:0/2311100193 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0b301009d0 msgr2=0x7f0b301a34e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:12.985 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:12.984+0000 7f0b3555f640 1 --2- 172.21.15.26:0/2311100193 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0b301009d0 0x7f0b301a34e0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f0b20030f80 tx=0x7f0b20030fb0 comp rx=0 tx=0).stop 2024-09-11T23:35:12.986 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:12.984+0000 7f0b3555f640 1 -- 172.21.15.26:0/2311100193 shutdown_connections 2024-09-11T23:35:12.986 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:12.984+0000 7f0b3555f640 1 --2- 172.21.15.26:0/2311100193 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f0b1c0a2ea0 0x7f0b1c0a5360 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:12.986 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:12.984+0000 7f0b3555f640 1 --2- 172.21.15.26:0/2311100193 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0b30101370 0x7f0b301a3a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:12.986 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:12.984+0000 7f0b3555f640 1 --2- 172.21.15.26:0/2311100193 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0b301069e0 0x7f0b3010c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:12.986 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:12.984+0000 7f0b3555f640 1 --2- 172.21.15.26:0/2311100193 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0b301009d0 0x7f0b301a34e0 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:12.986 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:12.984+0000 7f0b3555f640 1 -- 172.21.15.26:0/2311100193 >> 172.21.15.26:0/2311100193 conn(0x7f0b300fc6b0 msgr2=0x7f0b300fdb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:35:12.986 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:12.984+0000 7f0b3555f640 1 -- 172.21.15.26:0/2311100193 shutdown_connections 2024-09-11T23:35:12.986 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:12.984+0000 7f0b3555f640 1 -- 172.21.15.26:0/2311100193 wait complete. 2024-09-11T23:35:14.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:35:14 smithi136 ceph-mon[35121]: pgmap v529: 97 pgs: 23 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 33 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 246 op/s 2024-09-11T23:35:14.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:35:14 smithi026 ceph-mon[31172]: pgmap v529: 97 pgs: 23 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 33 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 246 op/s 2024-09-11T23:35:14.552 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:35:14 smithi076 ceph-mon[35296]: pgmap v529: 97 pgs: 23 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 33 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 246 op/s 2024-09-11T23:35:16.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:35:16 smithi136 ceph-mon[35121]: pgmap v530: 97 pgs: 23 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 33 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.4 MiB/s wr, 241 op/s 2024-09-11T23:35:16.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:35:16 smithi026 ceph-mon[31172]: pgmap v530: 97 pgs: 23 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 33 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.4 MiB/s wr, 241 op/s 2024-09-11T23:35:16.552 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:35:16 smithi076 ceph-mon[35296]: pgmap v530: 97 pgs: 23 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 33 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.4 MiB/s wr, 241 op/s 2024-09-11T23:35:18.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:35:18 smithi136 ceph-mon[35121]: pgmap v531: 97 pgs: 23 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 33 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.6 MiB/s wr, 242 op/s 2024-09-11T23:35:18.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:35:18 smithi026 ceph-mon[31172]: pgmap v531: 97 pgs: 23 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 33 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.6 MiB/s wr, 242 op/s 2024-09-11T23:35:18.552 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:35:18 smithi076 ceph-mon[35296]: pgmap v531: 97 pgs: 23 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 33 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.6 MiB/s wr, 242 op/s 2024-09-11T23:35:20.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:35:20 smithi136 ceph-mon[35121]: pgmap v532: 97 pgs: 23 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 33 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 180 op/s 2024-09-11T23:35:20.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:35:20 smithi136 ceph-mon[35121]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:35:20.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:35:20 smithi026 ceph-mon[31172]: pgmap v532: 97 pgs: 23 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 33 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 180 op/s 2024-09-11T23:35:20.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:35:20 smithi026 ceph-mon[31172]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:35:20.552 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:35:20 smithi076 ceph-mon[35296]: pgmap v532: 97 pgs: 23 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 33 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 180 op/s 2024-09-11T23:35:20.552 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:35:20 smithi076 ceph-mon[35296]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:35:22.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:35:22 smithi136 ceph-mon[35121]: pgmap v533: 97 pgs: 23 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 33 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.8 MiB/s wr, 225 op/s 2024-09-11T23:35:22.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:35:22 smithi026 ceph-mon[31172]: pgmap v533: 97 pgs: 23 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 33 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.8 MiB/s wr, 225 op/s 2024-09-11T23:35:22.552 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:35:22 smithi076 ceph-mon[35296]: pgmap v533: 97 pgs: 23 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 33 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.8 MiB/s wr, 225 op/s 2024-09-11T23:35:23.000 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:35:22 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: 2024-09-11T23:35:22.526+0000 7f7ccf119640 -1 calc_pg_upmaps abort due to max <= 0 2024-09-11T23:35:24.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:35:24 smithi136 ceph-mon[35121]: pgmap v534: 97 pgs: 23 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 33 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 178 op/s 2024-09-11T23:35:24.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:35:24 smithi136 ceph-mon[35121]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:35:24.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:35:24 smithi026 ceph-mon[31172]: pgmap v534: 97 pgs: 23 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 33 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 178 op/s 2024-09-11T23:35:24.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:35:24 smithi026 ceph-mon[31172]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:35:24.552 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:35:24 smithi076 ceph-mon[35296]: pgmap v534: 97 pgs: 23 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 33 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 178 op/s 2024-09-11T23:35:24.552 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:35:24 smithi076 ceph-mon[35296]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:35:25.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:35:25 smithi136 ceph-mon[35121]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:35:25.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:35:25 smithi026 ceph-mon[31172]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:35:25.552 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:35:25 smithi076 ceph-mon[35296]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:35:26.126 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.125+0000 7f314e3fd640 1 -- 172.21.15.26:0/3889054799 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f31481095d0 msgr2=0x7f3148109a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:26.126 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.125+0000 7f314e3fd640 1 --2- 172.21.15.26:0/3889054799 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f31481095d0 0x7f3148109a10 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f313c009900 tx=0x7f313c02a420 comp rx=0 tx=0).stop 2024-09-11T23:35:26.126 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.125+0000 7f314e3fd640 1 -- 172.21.15.26:0/3889054799 shutdown_connections 2024-09-11T23:35:26.126 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.125+0000 7f314e3fd640 1 --2- 172.21.15.26:0/3889054799 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f314811b140 0x7f314811d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:26.126 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.125+0000 7f314e3fd640 1 --2- 172.21.15.26:0/3889054799 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f31481095d0 0x7f3148109a10 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:26.126 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.125+0000 7f314e3fd640 1 --2- 172.21.15.26:0/3889054799 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f31481091f0 0x7f3148110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:26.126 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.126+0000 7f314e3fd640 1 -- 172.21.15.26:0/3889054799 >> 172.21.15.26:0/3889054799 conn(0x7f314806ce00 msgr2=0x7f314806d210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:35:26.126 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.126+0000 7f314e3fd640 1 -- 172.21.15.26:0/3889054799 shutdown_connections 2024-09-11T23:35:26.126 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.126+0000 7f314e3fd640 1 -- 172.21.15.26:0/3889054799 wait complete. 2024-09-11T23:35:26.128 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.127+0000 7f314e3fd640 1 Processor -- start 2024-09-11T23:35:26.128 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.127+0000 7f314e3fd640 1 -- start start 2024-09-11T23:35:26.129 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.127+0000 7f314e3fd640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f31481091f0 0x7f314811a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:26.129 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.127+0000 7f314e3fd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f31481095d0 0x7f3148114890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:26.129 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.127+0000 7f314e3fd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f314811b140 0x7f3148114dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:26.129 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.127+0000 7f314e3fd640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f3148111370 con 0x7f314811b140 2024-09-11T23:35:26.129 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.127+0000 7f314e3fd640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f31481111f0 con 0x7f31481095d0 2024-09-11T23:35:26.129 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.127+0000 7f314e3fd640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f31481114f0 con 0x7f31481091f0 2024-09-11T23:35:26.129 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.128+0000 7f314c99a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f314811b140 0x7f3148114dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:26.129 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.128+0000 7f3147fff640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f31481091f0 0x7f314811a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:26.129 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.128+0000 7f314c99a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f314811b140 0x7f3148114dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47882/0 (socket says 172.21.15.26:47882) 2024-09-11T23:35:26.129 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.128+0000 7f314c99a640 1 -- 172.21.15.26:0/2358914923 learned_addr learned my addr 172.21.15.26:0/2358914923 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:35:26.129 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.128+0000 7f31477fe640 1 --2- 172.21.15.26:0/2358914923 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f31481095d0 0x7f3148114890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:26.129 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.128+0000 7f314c99a640 1 -- 172.21.15.26:0/2358914923 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f31481091f0 msgr2=0x7f314811a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:26.130 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.128+0000 7f314c99a640 1 --2- 172.21.15.26:0/2358914923 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f31481091f0 0x7f314811a820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:26.130 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.128+0000 7f314c99a640 1 -- 172.21.15.26:0/2358914923 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f31481095d0 msgr2=0x7f3148114890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:26.130 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.128+0000 7f314c99a640 1 --2- 172.21.15.26:0/2358914923 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f31481095d0 0x7f3148114890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:26.130 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.128+0000 7f314c99a640 1 -- 172.21.15.26:0/2358914923 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3148115500 con 0x7f314811b140 2024-09-11T23:35:26.130 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.128+0000 7f3147fff640 1 --2- 172.21.15.26:0/2358914923 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f31481091f0 0x7f314811a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:35:26.131 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.129+0000 7f314c99a640 1 --2- 172.21.15.26:0/2358914923 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f314811b140 0x7f3148114dd0 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f313800d750 tx=0x7f313800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:35:26.131 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.130+0000 7f31457fa640 1 -- 172.21.15.26:0/2358914923 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3138017070 con 0x7f314811b140 2024-09-11T23:35:26.131 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.130+0000 7f31457fa640 1 -- 172.21.15.26:0/2358914923 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3138005500 con 0x7f314811b140 2024-09-11T23:35:26.131 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.130+0000 7f314e3fd640 1 -- 172.21.15.26:0/2358914923 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31481bc050 con 0x7f314811b140 2024-09-11T23:35:26.131 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.131+0000 7f314e3fd640 1 -- 172.21.15.26:0/2358914923 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31481bc460 con 0x7f314811b140 2024-09-11T23:35:26.131 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.131+0000 7f31457fa640 1 -- 172.21.15.26:0/2358914923 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3138004d90 con 0x7f314811b140 2024-09-11T23:35:26.133 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.134+0000 7f315127f740 1 Processor -- start 2024-09-11T23:35:26.138 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.135+0000 7f315127f740 1 -- start start 2024-09-11T23:35:26.138 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.135+0000 7f31457fa640 1 -- 172.21.15.26:0/2358914923 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f3138010460 con 0x7f314811b140 2024-09-11T23:35:26.138 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.135+0000 7f315127f740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cd96725af0 0x55cd96725ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:26.138 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.135+0000 7f315127f740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cd96726410 0x55cd96730770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:26.138 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.135+0000 7f315127f740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cd96730cb0 0x55cd967330a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:26.138 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.135+0000 7f315127f740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55cd9668a2f0 con 0x55cd96726410 2024-09-11T23:35:26.138 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.135+0000 7f315127f740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55cd96632270 con 0x55cd96725af0 2024-09-11T23:35:26.139 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.135+0000 7f315127f740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55cd966862b0 con 0x55cd96730cb0 2024-09-11T23:35:26.139 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.136+0000 7f3147fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cd96726410 0x55cd96730770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:26.139 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.136+0000 7f3147fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cd96726410 0x55cd96730770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47898/0 (socket says 172.21.15.26:47898) 2024-09-11T23:35:26.139 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.136+0000 7f3147fff640 1 -- 172.21.15.26:0/1683342094 learned_addr learned my addr 172.21.15.26:0/1683342094 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:35:26.139 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.136+0000 7f3147fff640 1 -- 172.21.15.26:0/1683342094 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cd96730cb0 msgr2=0x55cd967330a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:35:26.139 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.136+0000 7f3147fff640 1 --2- 172.21.15.26:0/1683342094 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cd96730cb0 0x55cd967330a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:26.139 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.136+0000 7f3147fff640 1 -- 172.21.15.26:0/1683342094 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cd96725af0 msgr2=0x55cd96725ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:35:26.139 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.136+0000 7f3147fff640 1 --2- 172.21.15.26:0/1683342094 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cd96725af0 0x55cd96725ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:26.139 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.136+0000 7f3147fff640 1 -- 172.21.15.26:0/1683342094 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd96631340 con 0x55cd96726410 2024-09-11T23:35:26.139 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.136+0000 7f3147fff640 1 --2- 172.21.15.26:0/1683342094 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cd96726410 0x55cd96730770 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f314000a8e0 tx=0x7f31400096d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f2eba9a6d016efc server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:35:26.140 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.137+0000 7f312e7fc640 1 -- 172.21.15.26:0/1683342094 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3140018020 con 0x55cd96726410 2024-09-11T23:35:26.140 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.137+0000 7f312e7fc640 1 -- 172.21.15.26:0/1683342094 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3140014400 con 0x55cd96726410 2024-09-11T23:35:26.140 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.137+0000 7f312e7fc640 1 -- 172.21.15.26:0/1683342094 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3140013580 con 0x55cd96726410 2024-09-11T23:35:26.140 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.137+0000 7f31457fa640 1 --2- 172.21.15.26:0/2358914923 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f31300a2fd0 0x7f31300a5490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:26.140 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.137+0000 7f31457fa640 1 -- 172.21.15.26:0/2358914923 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 8586+0+0 (secure 0 0 0) 0x7f31380cec60 con 0x7f314811b140 2024-09-11T23:35:26.140 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.137+0000 7f31477fe640 1 --2- 172.21.15.26:0/2358914923 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f31300a2fd0 0x7f31300a5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:26.140 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.138+0000 7f315127f740 1 -- 172.21.15.26:0/1683342094 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cd96726410 msgr2=0x55cd96730770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:26.140 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.138+0000 7f315127f740 1 --2- 172.21.15.26:0/1683342094 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cd96726410 0x55cd96730770 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f314000a8e0 tx=0x7f31400096d0 comp rx=0 tx=0).stop 2024-09-11T23:35:26.140 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.138+0000 7f315127f740 1 -- 172.21.15.26:0/1683342094 shutdown_connections 2024-09-11T23:35:26.140 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.138+0000 7f315127f740 1 --2- 172.21.15.26:0/1683342094 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cd96730cb0 0x55cd967330a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:26.140 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.138+0000 7f315127f740 1 --2- 172.21.15.26:0/1683342094 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cd96726410 0x55cd96730770 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:26.140 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.138+0000 7f315127f740 1 --2- 172.21.15.26:0/1683342094 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cd96725af0 0x55cd96725ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:26.141 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.138+0000 7f315127f740 1 -- 172.21.15.26:0/1683342094 >> 172.21.15.26:0/1683342094 conn(0x55cd96722060 msgr2=0x55cd96722440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:35:26.141 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.138+0000 7f315127f740 1 -- 172.21.15.26:0/1683342094 shutdown_connections 2024-09-11T23:35:26.141 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.138+0000 7f315127f740 1 -- 172.21.15.26:0/1683342094 wait complete. 2024-09-11T23:35:26.142 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.139+0000 7f315127f740 1 -- start start 2024-09-11T23:35:26.143 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.139+0000 7f315127f740 1 Processor -- start 2024-09-11T23:35:26.143 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.139+0000 7f31477fe640 1 --2- 172.21.15.26:0/2358914923 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f31300a2fd0 0x7f31300a5490 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f313c005ab0 tx=0x7f313c005fd0 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:35:26.143 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.140+0000 7f315127f740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cd96725af0 0x55cd9672dad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:26.143 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.140+0000 7f315127f740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cd96726410 0x55cd9672e010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:26.143 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.140+0000 7f315127f740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cd96730cb0 0x55cd9672e550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:26.143 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.140+0000 7f315127f740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55cd967338d0 con 0x55cd96725af0 2024-09-11T23:35:26.143 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.140+0000 7f315127f740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55cd96613290 con 0x55cd96726410 2024-09-11T23:35:26.143 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.140+0000 7f315127f740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55cd96733a50 con 0x55cd96730cb0 2024-09-11T23:35:26.143 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.140+0000 7f315127f740 10 client.0 _ll_register_callbacks cb 0x55cd96644140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:35:26.143 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.140+0000 7f3147fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cd96725af0 0x55cd9672dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:26.144 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.140+0000 7f3147fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cd96725af0 0x55cd9672dad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47910/0 (socket says 172.21.15.26:47910) 2024-09-11T23:35:26.144 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.140+0000 7f3147fff640 1 -- 172.21.15.26:0/1552324448 learned_addr learned my addr 172.21.15.26:0/1552324448 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:35:26.144 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.140+0000 7f31477fe640 1 --2- 172.21.15.26:0/1552324448 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cd96726410 0x55cd9672e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:26.144 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.141+0000 7f3147fff640 1 -- 172.21.15.26:0/1552324448 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cd96730cb0 msgr2=0x55cd9672e550 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T23:35:26.144 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.141+0000 7f3147fff640 1 --2- 172.21.15.26:0/1552324448 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cd96730cb0 0x55cd9672e550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:26.144 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.141+0000 7f3147fff640 1 -- 172.21.15.26:0/1552324448 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cd96726410 msgr2=0x55cd9672e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:35:26.144 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.141+0000 7f3147fff640 1 --2- 172.21.15.26:0/1552324448 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cd96726410 0x55cd9672e010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:26.144 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.141+0000 7f3147fff640 1 -- 172.21.15.26:0/1552324448 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd9639d210 con 0x55cd96725af0 2024-09-11T23:35:26.144 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.141+0000 7f31477fe640 1 --2- 172.21.15.26:0/1552324448 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cd96726410 0x55cd9672e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:35:26.144 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.141+0000 7f3147fff640 1 --2- 172.21.15.26:0/1552324448 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cd96725af0 0x55cd9672dad0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f314000eae0 tx=0x7f314000eb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31fb16866f1c0cf2 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:35:26.144 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.141+0000 7f31277fe640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:35:26.145 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.142+0000 7f31277fe640 1 -- 172.21.15.26:0/1552324448 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3140023070 con 0x55cd96725af0 2024-09-11T23:35:26.145 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.142+0000 7f31277fe640 1 -- 172.21.15.26:0/1552324448 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 8586+0+0 (secure 0 0 0) 0x7f314001d420 con 0x55cd96725af0 2024-09-11T23:35:26.145 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.142+0000 7f315127f740 1 -- 172.21.15.26:0/1552324448 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3140014400 con 0x55cd96725af0 2024-09-11T23:35:26.145 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.142+0000 7f31277fe640 1 -- 172.21.15.26:0/1552324448 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7f3140018040 con 0x55cd96725af0 2024-09-11T23:35:26.146 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.144+0000 7f315127f740 10 client.17679 fetch_fsmap learned FSMap version 80 2024-09-11T23:35:26.146 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.144+0000 7f315127f740 1 -- 172.21.15.26:0/1552324448 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=80,monmap=4+}) -- 0x55cd96469620 con 0x55cd96725af0 2024-09-11T23:35:26.146 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.145+0000 7f31277fe640 1 -- 172.21.15.26:0/1552324448 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3140014400 con 0x55cd96725af0 2024-09-11T23:35:26.146 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.145+0000 7f31277fe640 1 -- 172.21.15.26:0/1552324448 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 80) ==== 8426+0+0 (secure 0 0 0) 0x7f3140031600 con 0x55cd96725af0 2024-09-11T23:35:26.146 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.145+0000 7f315127f740 10 client.17679 fetch_fsmap finished waiting for FSMap version 80 2024-09-11T23:35:26.146 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.145+0000 7f315127f740 10 client.17679 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:35:26.146 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.145+0000 7f315127f740 20 client.17679 populate_metadata read hostname 'smithi026' 2024-09-11T23:35:26.146 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.145+0000 7f315127f740 1 --2- 172.21.15.26:0/1552324448 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55cd96740b10 0x55cd96742f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:26.146 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.145+0000 7f315127f740 4 client.17679 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:35:26.147 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.145+0000 7f315127f740 1 -- 172.21.15.26:0/1552324448 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cd96697a20 con 0x55cd96740b10 2024-09-11T23:35:26.148 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.149+0000 7f31477fe640 1 --2- 172.21.15.26:0/1552324448 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55cd96740b10 0x55cd96742f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:26.149 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.150+0000 7f31477fe640 1 --2- 172.21.15.26:0/1552324448 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55cd96740b10 0x55cd96742f00 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f29e820d3e64dbf server_cookie=56a1dd865f273bd7 in_seq=0 out_seq=0 2024-09-11T23:35:26.149 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.150+0000 7f31277fe640 10 client.17679 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:35:26.153 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.153+0000 7f31277fe640 1 -- 172.21.15.26:0/1552324448 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55cd96697a20 con 0x55cd96740b10 2024-09-11T23:35:26.153 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.153+0000 7f31277fe640 10 client.17679 handle_command_reply: tid=0 2024-09-11T23:35:26.153 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.153+0000 7f315127f740 1 client.17679 shutdown 2024-09-11T23:35:26.153 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.153+0000 7f315127f740 1 -- 172.21.15.26:0/1552324448 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cd96725af0 msgr2=0x55cd9672dad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:26.153 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.153+0000 7f315127f740 1 --2- 172.21.15.26:0/1552324448 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cd96725af0 0x55cd9672dad0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f314000eae0 tx=0x7f314000eb10 comp rx=0 tx=0).stop 2024-09-11T23:35:26.153 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.153+0000 7f315127f740 1 -- 172.21.15.26:0/1552324448 shutdown_connections 2024-09-11T23:35:26.154 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.153+0000 7f315127f740 1 --2- 172.21.15.26:0/1552324448 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cd96726410 0x55cd9672e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:26.154 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.153+0000 7f315127f740 1 --2- 172.21.15.26:0/1552324448 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55cd96740b10 0x55cd96742f00 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:26.154 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.153+0000 7f315127f740 1 --2- 172.21.15.26:0/1552324448 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cd96730cb0 0x55cd9672e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:26.154 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.153+0000 7f315127f740 1 --2- 172.21.15.26:0/1552324448 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cd96725af0 0x55cd9672dad0 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:26.154 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.153+0000 7f315127f740 1 -- 172.21.15.26:0/1552324448 >> 172.21.15.26:0/1552324448 conn(0x55cd96722060 msgr2=0x55cd967211c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:35:26.154 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.153+0000 7f31277fe640 0 client.17679 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:35:26.154 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.154+0000 7f315127f740 1 -- 172.21.15.26:0/1552324448 shutdown_connections 2024-09-11T23:35:26.154 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.154+0000 7f315127f740 1 -- 172.21.15.26:0/1552324448 wait complete. 2024-09-11T23:35:26.154 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.154+0000 7f315127f740 20 client.17679 trim_cache size 0 max 16384 2024-09-11T23:35:26.179 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.180+0000 7f315127f740 1 Processor -- start 2024-09-11T23:35:26.180 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.180+0000 7f315127f740 1 -- start start 2024-09-11T23:35:26.180 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.180+0000 7f315127f740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cd96725af0 0x55cd9672f330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:26.180 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.180+0000 7f315127f740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cd96726410 0x55cd9672e380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:26.180 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.180+0000 7f315127f740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cd96730cb0 0x55cd96748640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:26.181 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.180+0000 7f3147fff640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cd96725af0 0x55cd9672f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:26.181 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.180+0000 7f3147fff640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cd96725af0 0x55cd9672f330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:49744/0 (socket says 172.21.15.26:49744) 2024-09-11T23:35:26.181 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.181+0000 7f3147fff640 1 -- 172.21.15.26:0/2466675249 learned_addr learned my addr 172.21.15.26:0/2466675249 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:35:26.181 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.181+0000 7f315127f740 1 -- 172.21.15.26:0/2466675249 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f313400e930 con 0x55cd96730cb0 2024-09-11T23:35:26.181 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.181+0000 7f315127f740 1 -- 172.21.15.26:0/2466675249 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f3134010770 con 0x55cd96726410 2024-09-11T23:35:26.181 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.181+0000 7f315127f740 1 -- 172.21.15.26:0/2466675249 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f313400f0c0 con 0x55cd96725af0 2024-09-11T23:35:26.181 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.181+0000 7f314c99a640 1 --2- 172.21.15.26:0/2466675249 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cd96730cb0 0x55cd96748640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:26.181 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.181+0000 7f31477fe640 1 --2- 172.21.15.26:0/2466675249 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cd96726410 0x55cd9672e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:26.181 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.181+0000 7f314c99a640 1 -- 172.21.15.26:0/2466675249 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cd96725af0 msgr2=0x55cd9672f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:35:26.181 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.181+0000 7f314c99a640 1 --2- 172.21.15.26:0/2466675249 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cd96725af0 0x55cd9672f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:26.182 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.181+0000 7f314c99a640 1 -- 172.21.15.26:0/2466675249 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cd96726410 msgr2=0x55cd9672e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:35:26.182 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.181+0000 7f314c99a640 1 --2- 172.21.15.26:0/2466675249 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cd96726410 0x55cd9672e380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:26.182 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.181+0000 7f314c99a640 1 -- 172.21.15.26:0/2466675249 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd96721e90 con 0x55cd96730cb0 2024-09-11T23:35:26.182 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.181+0000 7f314c99a640 1 --2- 172.21.15.26:0/2466675249 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cd96730cb0 0x55cd96748640 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f31380cc2e0 tx=0x7f3138097560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c11a95426c294f7 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:35:26.183 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.182+0000 7f312dffb640 1 -- 172.21.15.26:0/2466675249 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3138020040 con 0x55cd96730cb0 2024-09-11T23:35:26.183 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.182+0000 7f312dffb640 1 -- 172.21.15.26:0/2466675249 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f313809f490 con 0x55cd96730cb0 2024-09-11T23:35:26.183 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.182+0000 7f312dffb640 1 -- 172.21.15.26:0/2466675249 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31380cc590 con 0x55cd96730cb0 2024-09-11T23:35:26.183 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.183+0000 7f315127f740 1 -- 172.21.15.26:0/2466675249 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cd96730cb0 msgr2=0x55cd96748640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:26.183 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.183+0000 7f315127f740 1 --2- 172.21.15.26:0/2466675249 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cd96730cb0 0x55cd96748640 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f31380cc2e0 tx=0x7f3138097560 comp rx=0 tx=0).stop 2024-09-11T23:35:26.183 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.183+0000 7f315127f740 1 -- 172.21.15.26:0/2466675249 shutdown_connections 2024-09-11T23:35:26.183 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.183+0000 7f315127f740 1 --2- 172.21.15.26:0/2466675249 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cd96726410 0x55cd9672e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:26.183 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.183+0000 7f315127f740 1 --2- 172.21.15.26:0/2466675249 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cd96730cb0 0x55cd96748640 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:26.183 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.183+0000 7f315127f740 1 --2- 172.21.15.26:0/2466675249 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cd96725af0 0x55cd9672f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:26.184 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.183+0000 7f315127f740 1 -- 172.21.15.26:0/2466675249 >> 172.21.15.26:0/2466675249 conn(0x55cd96722060 msgr2=0x55cd9672ca70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:35:26.184 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.183+0000 7f315127f740 1 -- 172.21.15.26:0/2466675249 shutdown_connections 2024-09-11T23:35:26.184 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.183+0000 7f315127f740 1 -- 172.21.15.26:0/2466675249 wait complete. 2024-09-11T23:35:26.184 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.183+0000 7f315127f740 1 -- start start 2024-09-11T23:35:26.184 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.184+0000 7f315127f740 1 Processor -- start 2024-09-11T23:35:26.185 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.184+0000 7f315127f740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cd96725af0 0x55cd967391c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:26.185 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.184+0000 7f315127f740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cd96726410 0x55cd9672d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:26.185 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.184+0000 7f315127f740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cd96730cb0 0x55cd9672da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:26.185 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.184+0000 7f315127f740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f313400ffe0 con 0x55cd96730cb0 2024-09-11T23:35:26.185 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.184+0000 7f315127f740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f313400f850 con 0x55cd96725af0 2024-09-11T23:35:26.186 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.184+0000 7f315127f740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f313400c930 con 0x55cd96726410 2024-09-11T23:35:26.186 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.184+0000 7f315127f740 10 client.0 _ll_register_callbacks cb 0x7f314000a760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:35:26.186 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.186+0000 7f3147fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cd96725af0 0x55cd967391c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:26.186 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.186+0000 7f3147fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cd96725af0 0x55cd967391c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:48234/0 (socket says 172.21.15.26:48234) 2024-09-11T23:35:26.186 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.186+0000 7f3147fff640 1 -- 172.21.15.26:0/1583826927 learned_addr learned my addr 172.21.15.26:0/1583826927 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:35:26.186 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.187+0000 7f31477fe640 1 --2- 172.21.15.26:0/1583826927 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cd96726410 0x55cd9672d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:26.187 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.187+0000 7f3147fff640 1 -- 172.21.15.26:0/1583826927 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cd96726410 msgr2=0x55cd9672d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:35:26.187 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.187+0000 7f3147fff640 1 --2- 172.21.15.26:0/1583826927 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cd96726410 0x55cd9672d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:26.187 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.187+0000 7f314c99a640 1 --2- 172.21.15.26:0/1583826927 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cd96730cb0 0x55cd9672da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:26.187 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.187+0000 7f3147fff640 1 -- 172.21.15.26:0/1583826927 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cd96730cb0 msgr2=0x55cd9672da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:35:26.187 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.188+0000 7f3147fff640 1 --2- 172.21.15.26:0/1583826927 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cd96730cb0 0x55cd9672da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:26.187 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.188+0000 7f3147fff640 1 -- 172.21.15.26:0/1583826927 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd9672dfb0 con 0x55cd96725af0 2024-09-11T23:35:26.187 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.188+0000 7f314c99a640 1 --2- 172.21.15.26:0/1583826927 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cd96730cb0 0x55cd9672da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:35:26.188 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.189+0000 7f3147fff640 1 --2- 172.21.15.26:0/1583826927 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cd96725af0 0x55cd967391c0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x55cd966f7860 tx=0x7f3140031730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=17796fc1ed4dc00 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:35:26.188 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.189+0000 7f31277fe640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-11T23:35:26.189 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.189+0000 7f31277fe640 1 -- 172.21.15.26:0/1583826927 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3140023040 con 0x55cd96725af0 2024-09-11T23:35:26.189 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.189+0000 7f31277fe640 1 -- 172.21.15.26:0/1583826927 <== mon.2 v2:172.21.15.76:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 8586+0+0 (secure 0 0 0) 0x7f3140014bf0 con 0x55cd96725af0 2024-09-11T23:35:26.189 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.189+0000 7f315127f740 1 -- 172.21.15.26:0/1583826927 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f313809f490 con 0x55cd96725af0 2024-09-11T23:35:26.189 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.190+0000 7f31277fe640 1 -- 172.21.15.26:0/1583826927 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7f3140009ea0 con 0x55cd96725af0 2024-09-11T23:35:26.189 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.190+0000 7f315127f740 10 client.27200 fetch_fsmap learned FSMap version 80 2024-09-11T23:35:26.190 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.190+0000 7f315127f740 1 -- 172.21.15.26:0/1583826927 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=80,monmap=4+}) -- 0x55cd96737a40 con 0x55cd96725af0 2024-09-11T23:35:26.190 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.190+0000 7f31277fe640 1 -- 172.21.15.26:0/1583826927 <== mon.2 v2:172.21.15.76:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3140014400 con 0x55cd96725af0 2024-09-11T23:35:26.191 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.190+0000 7f31277fe640 1 -- 172.21.15.26:0/1583826927 <== mon.2 v2:172.21.15.76:3300/0 5 ==== fsmap(e 80) ==== 8426+0+0 (secure 0 0 0) 0x7f3140004470 con 0x55cd96725af0 2024-09-11T23:35:26.191 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.190+0000 7f315127f740 10 client.27200 fetch_fsmap finished waiting for FSMap version 80 2024-09-11T23:35:26.191 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.190+0000 7f315127f740 10 client.27200 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:35:26.191 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.191+0000 7f315127f740 20 client.27200 populate_metadata read hostname 'smithi026' 2024-09-11T23:35:26.191 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.191+0000 7f315127f740 1 --2- 172.21.15.26:0/1583826927 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55cd967abb40 0x55cd967adf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:26.191 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.191+0000 7f315127f740 4 client.27200 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T23:35:26.191 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.191+0000 7f315127f740 1 -- 172.21.15.26:0/1583826927 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f313809f490 con 0x55cd967abb40 2024-09-11T23:35:26.191 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.191+0000 7f31477fe640 1 --2- 172.21.15.26:0/1583826927 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55cd967abb40 0x55cd967adf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:26.196 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.196+0000 7f31477fe640 1 --2- 172.21.15.26:0/1583826927 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55cd967abb40 0x55cd967adf60 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cde3f6dc2b3b5346 server_cookie=ad736abc32fdcd8 in_seq=0 out_seq=0 2024-09-11T23:35:26.196 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:26.197+0000 7f31277fe640 10 client.27200 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:35:26.453 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:35:26 smithi076 ceph-mon[35296]: pgmap v535: 97 pgs: 23 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 33 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 191 op/s 2024-09-11T23:35:26.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:35:26 smithi136 ceph-mon[35121]: pgmap v535: 97 pgs: 23 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 33 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 191 op/s 2024-09-11T23:35:26.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:35:26 smithi026 ceph-mon[31172]: pgmap v535: 97 pgs: 23 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 33 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 191 op/s 2024-09-11T23:35:28.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:35:28 smithi136 ceph-mon[35121]: pgmap v536: 97 pgs: 23 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 33 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 173 op/s 2024-09-11T23:35:28.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:35:28 smithi026 ceph-mon[31172]: pgmap v536: 97 pgs: 23 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 33 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 173 op/s 2024-09-11T23:35:28.552 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:35:28 smithi076 ceph-mon[35296]: pgmap v536: 97 pgs: 23 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 33 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 173 op/s 2024-09-11T23:35:28.559 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:28.558+0000 7f31277fe640 1 -- 172.21.15.26:0/1583826927 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f313809f490 con 0x55cd967abb40 2024-09-11T23:35:28.559 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:28.559+0000 7f31277fe640 10 client.27200 handle_command_reply: tid=0 2024-09-11T23:35:28.559 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:28.559+0000 7f315127f740 1 client.27200 shutdown 2024-09-11T23:35:28.560 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:28.559+0000 7f315127f740 1 -- 172.21.15.26:0/1583826927 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cd96725af0 msgr2=0x55cd967391c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:28.561 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:28.559+0000 7f315127f740 1 --2- 172.21.15.26:0/1583826927 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cd96725af0 0x55cd967391c0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x55cd966f7860 tx=0x7f3140031730 comp rx=0 tx=0).stop 2024-09-11T23:35:28.561 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:28.559+0000 7f315127f740 1 -- 172.21.15.26:0/1583826927 shutdown_connections 2024-09-11T23:35:28.561 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:28.559+0000 7f315127f740 1 --2- 172.21.15.26:0/1583826927 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55cd967abb40 0x55cd967adf60 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:28.561 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:28.559+0000 7f315127f740 1 --2- 172.21.15.26:0/1583826927 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cd96730cb0 0x55cd9672da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:28.561 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:28.559+0000 7f31277fe640 0 client.27200 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:35:28.561 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:28.560+0000 7f315127f740 1 --2- 172.21.15.26:0/1583826927 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cd96726410 0x55cd9672d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:28.561 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:28.560+0000 7f315127f740 1 --2- 172.21.15.26:0/1583826927 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cd96725af0 0x55cd967391c0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:28.561 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:28.560+0000 7f315127f740 1 -- 172.21.15.26:0/1583826927 >> 172.21.15.26:0/1583826927 conn(0x55cd96722060 msgr2=0x55cd9673bb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:35:28.561 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:28.560+0000 7f315127f740 1 -- 172.21.15.26:0/1583826927 shutdown_connections 2024-09-11T23:35:28.561 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:28.560+0000 7f315127f740 1 -- 172.21.15.26:0/1583826927 wait complete. 2024-09-11T23:35:28.561 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:28.560+0000 7f315127f740 20 client.27200 trim_cache size 0 max 16384 2024-09-11T23:35:28.562 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-09-11T23:35:28.562 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "message": "", 2024-09-11T23:35:28.562 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-09-11T23:35:28.562 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-09-11T23:35:28.563 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:28.561+0000 7f314e3fd640 1 -- 172.21.15.26:0/2358914923 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f31300a2fd0 msgr2=0x7f31300a5490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:28.563 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:28.561+0000 7f314e3fd640 1 --2- 172.21.15.26:0/2358914923 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f31300a2fd0 0x7f31300a5490 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f313c005ab0 tx=0x7f313c005fd0 comp rx=0 tx=0).stop 2024-09-11T23:35:28.564 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:28.561+0000 7f314e3fd640 1 -- 172.21.15.26:0/2358914923 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f314811b140 msgr2=0x7f3148114dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:28.564 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:28.561+0000 7f314e3fd640 1 --2- 172.21.15.26:0/2358914923 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f314811b140 0x7f3148114dd0 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f313800d750 tx=0x7f313800dc10 comp rx=0 tx=0).stop 2024-09-11T23:35:28.564 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:28.562+0000 7f314e3fd640 1 -- 172.21.15.26:0/2358914923 shutdown_connections 2024-09-11T23:35:28.564 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:28.562+0000 7f314e3fd640 1 --2- 172.21.15.26:0/2358914923 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f31300a2fd0 0x7f31300a5490 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:28.564 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:28.562+0000 7f314e3fd640 1 --2- 172.21.15.26:0/2358914923 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f31481095d0 0x7f3148114890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:28.564 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:28.562+0000 7f314e3fd640 1 --2- 172.21.15.26:0/2358914923 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f314811b140 0x7f3148114dd0 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:28.564 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:28.562+0000 7f314e3fd640 1 --2- 172.21.15.26:0/2358914923 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f31481091f0 0x7f314811a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:28.564 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:28.562+0000 7f314e3fd640 1 -- 172.21.15.26:0/2358914923 >> 172.21.15.26:0/2358914923 conn(0x7f314806ce00 msgr2=0x7f31480708f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:35:28.564 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:28.562+0000 7f314e3fd640 1 -- 172.21.15.26:0/2358914923 shutdown_connections 2024-09-11T23:35:28.564 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:28.562+0000 7f314e3fd640 1 -- 172.21.15.26:0/2358914923 wait complete. 2024-09-11T23:35:30.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:35:30 smithi136 ceph-mon[35121]: pgmap v537: 97 pgs: 23 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 33 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 746 KiB/s rd, 572 KiB/s wr, 119 op/s 2024-09-11T23:35:30.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:35:30 smithi026 ceph-mon[31172]: pgmap v537: 97 pgs: 23 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 33 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 746 KiB/s rd, 572 KiB/s wr, 119 op/s 2024-09-11T23:35:30.552 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:35:30 smithi076 ceph-mon[35296]: pgmap v537: 97 pgs: 23 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 33 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 746 KiB/s rd, 572 KiB/s wr, 119 op/s 2024-09-11T23:35:32.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:35:32 smithi136 ceph-mon[35121]: pgmap v538: 97 pgs: 20 active+clean+snaptrim, 35 active+clean+snaptrim_wait, 42 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 962 KiB/s rd, 901 KiB/s wr, 227 op/s 2024-09-11T23:35:32.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:35:32 smithi026 ceph-mon[31172]: pgmap v538: 97 pgs: 20 active+clean+snaptrim, 35 active+clean+snaptrim_wait, 42 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 962 KiB/s rd, 901 KiB/s wr, 227 op/s 2024-09-11T23:35:32.552 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:35:32 smithi076 ceph-mon[35296]: pgmap v538: 97 pgs: 20 active+clean+snaptrim, 35 active+clean+snaptrim_wait, 42 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 962 KiB/s rd, 901 KiB/s wr, 227 op/s 2024-09-11T23:35:32.809 INFO:tasks.quiescer.fs.[cephfs]:Total duration (48.5) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-11T23:35:32.809 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 24.25 2024-09-11T23:35:32.810 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:35:33.173 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:35:33.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:33.725+0000 7faa913e8640 1 -- 172.21.15.26:0/1909163890 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa8c100e20 msgr2=0x7faa8c101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:33.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:33.725+0000 7faa913e8640 1 --2- 172.21.15.26:0/1909163890 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa8c100e20 0x7faa8c101200 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7faa7c009900 tx=0x7faa7c02a480 comp rx=0 tx=0).stop 2024-09-11T23:35:33.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:33.725+0000 7faa913e8640 1 -- 172.21.15.26:0/1909163890 shutdown_connections 2024-09-11T23:35:33.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:33.725+0000 7faa913e8640 1 --2- 172.21.15.26:0/1909163890 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faa8c102ef0 0x7faa8c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:33.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:33.725+0000 7faa913e8640 1 --2- 172.21.15.26:0/1909163890 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faa8c102570 0x7faa8c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:33.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:33.725+0000 7faa913e8640 1 --2- 172.21.15.26:0/1909163890 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa8c100e20 0x7faa8c101200 secure :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7faa7c009900 tx=0x7faa7c02a480 comp rx=0 tx=0).stop 2024-09-11T23:35:33.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:33.725+0000 7faa913e8640 1 -- 172.21.15.26:0/1909163890 >> 172.21.15.26:0/1909163890 conn(0x7faa8c0fc7d0 msgr2=0x7faa8c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:35:33.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:33.725+0000 7faa913e8640 1 -- 172.21.15.26:0/1909163890 shutdown_connections 2024-09-11T23:35:33.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:33.725+0000 7faa913e8640 1 -- 172.21.15.26:0/1909163890 wait complete. 2024-09-11T23:35:33.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:33.728+0000 7faa913e8640 1 Processor -- start 2024-09-11T23:35:33.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:33.728+0000 7faa913e8640 1 -- start start 2024-09-11T23:35:33.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:33.728+0000 7faa913e8640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faa8c102570 0x7faa8c10ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:33.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:33.728+0000 7faa913e8640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faa8c102ef0 0x7faa8c10b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:33.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:33.728+0000 7faa913e8640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa8c1a35f0 0x7faa8c1a3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:33.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:33.728+0000 7faa8affd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faa8c102570 0x7faa8c10ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:33.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:33.729+0000 7faa8b7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa8c1a35f0 0x7faa8c1a3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:33.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:33.728+0000 7faa8a7fc640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faa8c102ef0 0x7faa8c10b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:33.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:33.729+0000 7faa8a7fc640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faa8c102ef0 0x7faa8c10b290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:43090/0 (socket says 172.21.15.26:43090) 2024-09-11T23:35:33.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:33.729+0000 7faa8b7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa8c1a35f0 0x7faa8c1a3aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42778/0 (socket says 172.21.15.26:42778) 2024-09-11T23:35:33.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:33.729+0000 7faa8affd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faa8c102570 0x7faa8c10ad50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:58534/0 (socket says 172.21.15.26:58534) 2024-09-11T23:35:33.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:33.729+0000 7faa8a7fc640 1 -- 172.21.15.26:0/3587413866 learned_addr learned my addr 172.21.15.26:0/3587413866 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:35:33.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:33.729+0000 7faa913e8640 1 -- 172.21.15.26:0/3587413866 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7faa8c112080 con 0x7faa8c1a35f0 2024-09-11T23:35:33.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:33.729+0000 7faa913e8640 1 -- 172.21.15.26:0/3587413866 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7faa8c111f00 con 0x7faa8c102570 2024-09-11T23:35:33.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:33.729+0000 7faa913e8640 1 -- 172.21.15.26:0/3587413866 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7faa8c112200 con 0x7faa8c102ef0 2024-09-11T23:35:33.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:33.730+0000 7faa8b7fe640 1 -- 172.21.15.26:0/3587413866 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faa8c102ef0 msgr2=0x7faa8c10b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:33.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:33.730+0000 7faa8b7fe640 1 --2- 172.21.15.26:0/3587413866 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faa8c102ef0 0x7faa8c10b290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:33.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:33.730+0000 7faa8b7fe640 1 -- 172.21.15.26:0/3587413866 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faa8c102570 msgr2=0x7faa8c10ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:33.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:33.730+0000 7faa8b7fe640 1 --2- 172.21.15.26:0/3587413866 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faa8c102570 0x7faa8c10ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:33.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:33.730+0000 7faa8b7fe640 1 -- 172.21.15.26:0/3587413866 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa8c1a4110 con 0x7faa8c1a35f0 2024-09-11T23:35:33.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:33.730+0000 7faa8b7fe640 1 --2- 172.21.15.26:0/3587413866 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa8c1a35f0 0x7faa8c1a3aa0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7faa84009fd0 tx=0x7faa84030f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:35:33.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:33.731+0000 7faa73fff640 1 -- 172.21.15.26:0/3587413866 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa8400ce40 con 0x7faa8c1a35f0 2024-09-11T23:35:33.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:33.731+0000 7faa913e8640 1 -- 172.21.15.26:0/3587413866 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa8c1a4400 con 0x7faa8c1a35f0 2024-09-11T23:35:33.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:33.731+0000 7faa913e8640 1 -- 172.21.15.26:0/3587413866 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa8c06df60 con 0x7faa8c1a35f0 2024-09-11T23:35:33.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:33.731+0000 7faa73fff640 1 -- 172.21.15.26:0/3587413866 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa84005020 con 0x7faa8c1a35f0 2024-09-11T23:35:33.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:33.731+0000 7faa73fff640 1 -- 172.21.15.26:0/3587413866 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa84032730 con 0x7faa8c1a35f0 2024-09-11T23:35:33.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:33.735+0000 7faa73fff640 1 -- 172.21.15.26:0/3587413866 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7faa84004b20 con 0x7faa8c1a35f0 2024-09-11T23:35:33.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:33.736+0000 7faa73fff640 1 --2- 172.21.15.26:0/3587413866 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7faa740a2ef0 0x7faa740a53b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:33.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:33.734+0000 7faa913e8640 1 -- 172.21.15.26:0/3587413866 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa78005290 con 0x7faa8c1a35f0 2024-09-11T23:35:33.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:33.737+0000 7faa73fff640 1 -- 172.21.15.26:0/3587413866 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 8586+0+0 (secure 0 0 0) 0x7faa840f1940 con 0x7faa8c1a35f0 2024-09-11T23:35:33.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:33.737+0000 7faa8affd640 1 --2- 172.21.15.26:0/3587413866 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7faa740a2ef0 0x7faa740a53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:33.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:33.738+0000 7faa8affd640 1 --2- 172.21.15.26:0/3587413866 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7faa740a2ef0 0x7faa740a53b0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7faa7c006010 tx=0x7faa7c005a20 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:35:33.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:33.744+0000 7faa73fff640 1 -- 172.21.15.26:0/3587413866 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7faa840be4c0 con 0x7faa8c1a35f0 2024-09-11T23:35:34.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:34.071+0000 7faa913e8640 1 -- 172.21.15.26:0/3587413866 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faa78005850 con 0x7faa8c1a35f0 2024-09-11T23:35:34.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:34.072+0000 7faa73fff640 1 -- 172.21.15.26:0/3587413866 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) ==== 94+0+12000 (secure 0 0 0) 0x7faa840bddc0 con 0x7faa8c1a35f0 2024-09-11T23:35:34.073 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-11T23:35:34.073 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":80,"btime":"2024-09-11T23:35:24:094757+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14499,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/367659515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":367659515},{"type":"v1","addr":"172.21.15.26:6835","nonce":367659515}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14511,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/2041065299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":2041065299},{"type":"v1","addr":"172.21.15.26:6837","nonce":2041065299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":80,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:16:26.544203+0000","modified":"2024-09-11T23:35:23.267711+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24485,"mds_1":24463,"mds_2":24458,"mds_3":24448,"mds_4":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14550":{"gid":14550,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6839/2233068305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":2233068305},{"type":"v1","addr":"172.21.15.26:6839","nonce":2233068305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14562":{"gid":14562,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6841/2984982336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":2984982336},{"type":"v1","addr":"172.21.15.26:6841","nonce":2984982336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6833/2019841455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":2019841455},{"type":"v1","addr":"172.21.15.136:6833","nonce":2019841455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.76:6833/3438147050","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3438147050},{"type":"v1","addr":"172.21.15.76:6833","nonce":3438147050}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6835/3617665975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3617665975},{"type":"v1","addr":"172.21.15.136:6835","nonce":3617665975}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":12,"addr":"172.21.15.76:6835/393308499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":393308499},{"type":"v1","addr":"172.21.15.76:6835","nonce":393308499}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":10,"addr":"172.21.15.136:6837/1633082387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":1633082387},{"type":"v1","addr":"172.21.15.136:6837","nonce":1633082387}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.76:6837/2078978748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":2078978748},{"type":"v1","addr":"172.21.15.76:6837","nonce":2078978748}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":63,"addr":"172.21.15.136:6839/3211398630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":3211398630},{"type":"v1","addr":"172.21.15.136:6839","nonce":3211398630}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.76:6839/2077637090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6838","nonce":2077637090},{"type":"v1","addr":"172.21.15.76:6839","nonce":2077637090}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24485,"qdb_cluster":[24485,24463,24458,24448,24443]},"id":1}]} 2024-09-11T23:35:34.073 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 80 2024-09-11T23:35:34.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:34.074+0000 7faa913e8640 1 -- 172.21.15.26:0/3587413866 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7faa740a2ef0 msgr2=0x7faa740a53b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:34.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:34.074+0000 7faa913e8640 1 --2- 172.21.15.26:0/3587413866 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7faa740a2ef0 0x7faa740a53b0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7faa7c006010 tx=0x7faa7c005a20 comp rx=0 tx=0).stop 2024-09-11T23:35:34.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:34.074+0000 7faa913e8640 1 -- 172.21.15.26:0/3587413866 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa8c1a35f0 msgr2=0x7faa8c1a3aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:34.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:34.074+0000 7faa913e8640 1 --2- 172.21.15.26:0/3587413866 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa8c1a35f0 0x7faa8c1a3aa0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7faa84009fd0 tx=0x7faa84030f20 comp rx=0 tx=0).stop 2024-09-11T23:35:34.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:34.074+0000 7faa913e8640 1 -- 172.21.15.26:0/3587413866 shutdown_connections 2024-09-11T23:35:34.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:34.074+0000 7faa913e8640 1 --2- 172.21.15.26:0/3587413866 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7faa740a2ef0 0x7faa740a53b0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:34.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:34.074+0000 7faa913e8640 1 --2- 172.21.15.26:0/3587413866 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa8c1a35f0 0x7faa8c1a3aa0 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:34.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:34.074+0000 7faa913e8640 1 --2- 172.21.15.26:0/3587413866 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faa8c102ef0 0x7faa8c10b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:34.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:34.074+0000 7faa913e8640 1 --2- 172.21.15.26:0/3587413866 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faa8c102570 0x7faa8c10ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:34.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:34.074+0000 7faa913e8640 1 -- 172.21.15.26:0/3587413866 >> 172.21.15.26:0/3587413866 conn(0x7faa8c0fc7d0 msgr2=0x7faa8c0fdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:35:34.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:34.075+0000 7faa913e8640 1 -- 172.21.15.26:0/3587413866 shutdown_connections 2024-09-11T23:35:34.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:34.075+0000 7faa913e8640 1 -- 172.21.15.26:0/3587413866 wait complete. 2024-09-11T23:35:34.346 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:35:34 smithi026 ceph-mon[31172]: pgmap v539: 97 pgs: 20 active+clean+snaptrim, 35 active+clean+snaptrim_wait, 42 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 581 KiB/s rd, 628 KiB/s wr, 199 op/s 2024-09-11T23:35:34.346 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:35:34 smithi026 ceph-mon[31172]: from='client.? 172.21.15.26:0/3587413866' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:35:34.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:35:34 smithi136 ceph-mon[35121]: pgmap v539: 97 pgs: 20 active+clean+snaptrim, 35 active+clean+snaptrim_wait, 42 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 581 KiB/s rd, 628 KiB/s wr, 199 op/s 2024-09-11T23:35:34.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:35:34 smithi136 ceph-mon[35121]: from='client.? 172.21.15.26:0/3587413866' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:35:34.552 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:35:34 smithi076 ceph-mon[35296]: pgmap v539: 97 pgs: 20 active+clean+snaptrim, 35 active+clean+snaptrim_wait, 42 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 581 KiB/s rd, 628 KiB/s wr, 199 op/s 2024-09-11T23:35:34.552 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:35:34 smithi076 ceph-mon[35296]: from='client.? 172.21.15.26:0/3587413866' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:35:34.908 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24485 quiesce db / --timeout 90 --expiration 144.25 --await-for 45.0' 2024-09-11T23:35:34.908 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24485 quiesce db / --timeout 90 --expiration 144.25 --await-for 45.0 2024-09-11T23:35:35.352 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:35:36.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:35:36 smithi026 ceph-mon[31172]: pgmap v540: 97 pgs: 19 active+clean+snaptrim, 35 active+clean+snaptrim_wait, 43 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 593 KiB/s rd, 628 KiB/s wr, 205 op/s 2024-09-11T23:35:36.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:35:36 smithi136 ceph-mon[35121]: pgmap v540: 97 pgs: 19 active+clean+snaptrim, 35 active+clean+snaptrim_wait, 43 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 593 KiB/s rd, 628 KiB/s wr, 205 op/s 2024-09-11T23:35:36.552 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:35:36 smithi076 ceph-mon[35296]: pgmap v540: 97 pgs: 19 active+clean+snaptrim, 35 active+clean+snaptrim_wait, 43 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 593 KiB/s rd, 628 KiB/s wr, 205 op/s 2024-09-11T23:35:36.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.251+0000 7f0cb732e640 1 -- 172.21.15.26:0/3119377338 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0cb01012a0 msgr2=0x7f0cb010d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:36.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.251+0000 7f0cb732e640 1 --2- 172.21.15.26:0/3119377338 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0cb01012a0 0x7f0cb010d7d0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f0ca8009c10 tx=0x7f0ca802a720 comp rx=0 tx=0).stop 2024-09-11T23:35:36.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.251+0000 7f0cb732e640 1 -- 172.21.15.26:0/3119377338 shutdown_connections 2024-09-11T23:35:36.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.251+0000 7f0cb732e640 1 --2- 172.21.15.26:0/3119377338 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0cb0100900 0x7f0cb0100d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:36.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.251+0000 7f0cb732e640 1 --2- 172.21.15.26:0/3119377338 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0cb01012a0 0x7f0cb010d7d0 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:36.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.251+0000 7f0cb732e640 1 --2- 172.21.15.26:0/3119377338 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0cb0106900 0x7f0cb0106ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:36.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.251+0000 7f0cb732e640 1 -- 172.21.15.26:0/3119377338 >> 172.21.15.26:0/3119377338 conn(0x7f0cb00fc600 msgr2=0x7f0cb00fea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:35:36.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.252+0000 7f0cb732e640 1 -- 172.21.15.26:0/3119377338 shutdown_connections 2024-09-11T23:35:36.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.252+0000 7f0cb732e640 1 -- 172.21.15.26:0/3119377338 wait complete. 2024-09-11T23:35:36.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.253+0000 7f0cb732e640 1 Processor -- start 2024-09-11T23:35:36.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.253+0000 7f0cb732e640 1 -- start start 2024-09-11T23:35:36.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.253+0000 7f0cb732e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0cb0100900 0x7f0cb0195c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:36.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.253+0000 7f0cb732e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0cb01012a0 0x7f0cb01961c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:36.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.253+0000 7f0cb732e640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0cb0106900 0x7f0cb0196700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:36.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.253+0000 7f0cb732e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f0cb010fbf0 con 0x7f0cb0100900 2024-09-11T23:35:36.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.253+0000 7f0cb732e640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f0cb010fa70 con 0x7f0cb01012a0 2024-09-11T23:35:36.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.253+0000 7f0cb732e640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f0cb010fd70 con 0x7f0cb0106900 2024-09-11T23:35:36.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.254+0000 7f0cb50ca640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0cb0100900 0x7f0cb0195c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:36.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.254+0000 7f0cb50ca640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0cb0100900 0x7f0cb0195c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42794/0 (socket says 172.21.15.26:42794) 2024-09-11T23:35:36.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.254+0000 7f0cb50ca640 1 -- 172.21.15.26:0/2325758671 learned_addr learned my addr 172.21.15.26:0/2325758671 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:35:36.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.254+0000 7f0cb48c9640 1 --2- 172.21.15.26:0/2325758671 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0cb01012a0 0x7f0cb01961c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:36.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.254+0000 7f0cb58cb640 1 --2- 172.21.15.26:0/2325758671 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0cb0106900 0x7f0cb0196700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:36.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.254+0000 7f0cb50ca640 1 -- 172.21.15.26:0/2325758671 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0cb0106900 msgr2=0x7f0cb0196700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:36.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.254+0000 7f0cb50ca640 1 --2- 172.21.15.26:0/2325758671 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0cb0106900 0x7f0cb0196700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:36.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.254+0000 7f0cb50ca640 1 -- 172.21.15.26:0/2325758671 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0cb01012a0 msgr2=0x7f0cb01961c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:36.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.254+0000 7f0cb50ca640 1 --2- 172.21.15.26:0/2325758671 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0cb01012a0 0x7f0cb01961c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:36.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.254+0000 7f0cb50ca640 1 -- 172.21.15.26:0/2325758671 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cb01aa900 con 0x7f0cb0100900 2024-09-11T23:35:36.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.254+0000 7f0cb48c9640 1 --2- 172.21.15.26:0/2325758671 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0cb01012a0 0x7f0cb01961c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:35:36.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.255+0000 7f0cb58cb640 1 --2- 172.21.15.26:0/2325758671 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0cb0106900 0x7f0cb0196700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:35:36.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.255+0000 7f0cb50ca640 1 --2- 172.21.15.26:0/2325758671 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0cb0100900 0x7f0cb0195c80 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f0cac00d980 tx=0x7f0cac00de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:35:36.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.255+0000 7f0ca67fc640 1 -- 172.21.15.26:0/2325758671 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cac017070 con 0x7f0cb0100900 2024-09-11T23:35:36.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.255+0000 7f0ca67fc640 1 -- 172.21.15.26:0/2325758671 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0cac009c30 con 0x7f0cb0100900 2024-09-11T23:35:36.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.255+0000 7f0ca67fc640 1 -- 172.21.15.26:0/2325758671 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cac00f6b0 con 0x7f0cb0100900 2024-09-11T23:35:36.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.258+0000 7f0cb732e640 1 -- 172.21.15.26:0/2325758671 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cb01aabf0 con 0x7f0cb0100900 2024-09-11T23:35:36.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.258+0000 7f0cb732e640 1 -- 172.21.15.26:0/2325758671 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cb01ab0e0 con 0x7f0cb0100900 2024-09-11T23:35:36.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.262+0000 7f0cb9707740 1 Processor -- start 2024-09-11T23:35:36.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.262+0000 7f0cb9707740 1 -- start start 2024-09-11T23:35:36.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.262+0000 7f0cb9707740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5581fce6ea20 0x5581fce6ee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:36.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.262+0000 7f0cb9707740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5581fce6f340 0x5581fce796c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:36.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.263+0000 7f0cb9707740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5581fce79c00 0x5581fce7bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:36.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.263+0000 7f0cb9707740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5581fcdd3160 con 0x5581fce6f340 2024-09-11T23:35:36.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.263+0000 7f0cb9707740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5581fcd7a720 con 0x5581fce79c00 2024-09-11T23:35:36.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.263+0000 7f0cb9707740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5581fcdcf120 con 0x5581fce6ea20 2024-09-11T23:35:36.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.263+0000 7f0cb48c9640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5581fce79c00 0x5581fce7bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:36.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.263+0000 7f0cb48c9640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5581fce79c00 0x5581fce7bff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:58562/0 (socket says 172.21.15.26:58562) 2024-09-11T23:35:36.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.263+0000 7f0cb48c9640 1 -- 172.21.15.26:0/512473796 learned_addr learned my addr 172.21.15.26:0/512473796 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:35:36.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.264+0000 7f0cb58cb640 1 --2- 172.21.15.26:0/512473796 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5581fce6ea20 0x5581fce6ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:36.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.264+0000 7f0cb48c9640 1 -- 172.21.15.26:0/512473796 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5581fce6ea20 msgr2=0x5581fce6ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:35:36.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.264+0000 7f0cb48c9640 1 --2- 172.21.15.26:0/512473796 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5581fce6ea20 0x5581fce6ee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:36.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.264+0000 7f0cb48c9640 1 -- 172.21.15.26:0/512473796 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5581fce6f340 msgr2=0x5581fce796c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:35:36.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.264+0000 7f0cb48c9640 1 --2- 172.21.15.26:0/512473796 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5581fce6f340 0x5581fce796c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:36.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.264+0000 7f0cb48c9640 1 -- 172.21.15.26:0/512473796 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5581fcb44ba0 con 0x5581fce79c00 2024-09-11T23:35:36.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.264+0000 7f0ca67fc640 1 -- 172.21.15.26:0/2325758671 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f0cac010460 con 0x7f0cb0100900 2024-09-11T23:35:36.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.266+0000 7f0ca67fc640 1 --2- 172.21.15.26:0/2325758671 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f0c9c0a2df0 0x7f0c9c0a52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:36.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.266+0000 7f0cb48c9640 1 --2- 172.21.15.26:0/512473796 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5581fce79c00 0x5581fce7bff0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f0ca0009cb0 tx=0x7f0ca00117b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2e105c88301509dc server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:35:36.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.266+0000 7f0ca67fc640 1 -- 172.21.15.26:0/2325758671 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 8586+0+0 (secure 0 0 0) 0x7f0cac0cee10 con 0x7f0cb0100900 2024-09-11T23:35:36.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.266+0000 7f0cb58cb640 1 --2- 172.21.15.26:0/2325758671 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f0c9c0a2df0 0x7f0c9c0a52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:36.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.267+0000 7f0c9b7fe640 1 -- 172.21.15.26:0/512473796 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ca0004020 con 0x5581fce79c00 2024-09-11T23:35:36.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.268+0000 7f0cb58cb640 1 --2- 172.21.15.26:0/2325758671 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f0c9c0a2df0 0x7f0c9c0a52b0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f0ca8009b30 tx=0x7f0ca800b040 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:35:36.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.268+0000 7f0c9b7fe640 1 -- 172.21.15.26:0/512473796 <== mon.2 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ca00041c0 con 0x5581fce79c00 2024-09-11T23:35:36.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.268+0000 7f0c9b7fe640 1 -- 172.21.15.26:0/512473796 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ca0012810 con 0x5581fce79c00 2024-09-11T23:35:36.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.269+0000 7f0cb9707740 1 -- 172.21.15.26:0/512473796 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5581fce79c00 msgr2=0x5581fce7bff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:36.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.269+0000 7f0cb9707740 1 --2- 172.21.15.26:0/512473796 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5581fce79c00 0x5581fce7bff0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f0ca0009cb0 tx=0x7f0ca00117b0 comp rx=0 tx=0).stop 2024-09-11T23:35:36.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.270+0000 7f0cb9707740 1 -- 172.21.15.26:0/512473796 shutdown_connections 2024-09-11T23:35:36.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.270+0000 7f0cb9707740 1 --2- 172.21.15.26:0/512473796 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5581fce79c00 0x5581fce7bff0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:36.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.270+0000 7f0cb9707740 1 --2- 172.21.15.26:0/512473796 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5581fce6f340 0x5581fce796c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:36.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.270+0000 7f0cb9707740 1 --2- 172.21.15.26:0/512473796 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5581fce6ea20 0x5581fce6ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:36.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.270+0000 7f0cb9707740 1 -- 172.21.15.26:0/512473796 >> 172.21.15.26:0/512473796 conn(0x5581fce6afb0 msgr2=0x5581fce6b390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:35:36.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.270+0000 7f0cb9707740 1 -- 172.21.15.26:0/512473796 shutdown_connections 2024-09-11T23:35:36.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.275+0000 7f0cb9707740 1 -- 172.21.15.26:0/512473796 wait complete. 2024-09-11T23:35:36.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.275+0000 7f0cb9707740 1 -- start start 2024-09-11T23:35:36.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.275+0000 7f0cb9707740 1 Processor -- start 2024-09-11T23:35:36.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.277+0000 7f0cb9707740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5581fce6ea20 0x5581fce76b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:36.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.277+0000 7f0cb9707740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5581fce6f340 0x5581fce770a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:36.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.277+0000 7f0cb9707740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5581fce79c00 0x5581fce775e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:36.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.277+0000 7f0cb9707740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5581fce7c7d0 con 0x5581fce79c00 2024-09-11T23:35:36.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.277+0000 7f0cb9707740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5581fcd5b900 con 0x5581fce6ea20 2024-09-11T23:35:36.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.277+0000 7f0cb9707740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5581fce7c950 con 0x5581fce6f340 2024-09-11T23:35:36.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.277+0000 7f0cb9707740 10 client.0 _ll_register_callbacks cb 0x5581fcd59780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:35:36.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.278+0000 7f0cb48c9640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5581fce6f340 0x5581fce770a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:36.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.279+0000 7f0cb48c9640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5581fce6f340 0x5581fce770a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:43116/0 (socket says 172.21.15.26:43116) 2024-09-11T23:35:36.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.279+0000 7f0cb48c9640 1 -- 172.21.15.26:0/974645066 learned_addr learned my addr 172.21.15.26:0/974645066 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:35:36.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.279+0000 7f0cb58cb640 1 --2- 172.21.15.26:0/974645066 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5581fce79c00 0x5581fce775e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:36.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.280+0000 7f0cb58cb640 1 -- 172.21.15.26:0/974645066 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5581fce6f340 msgr2=0x5581fce770a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:35:36.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.280+0000 7f0cb58cb640 1 --2- 172.21.15.26:0/974645066 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5581fce6f340 0x5581fce770a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:36.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.280+0000 7f0cb58cb640 1 -- 172.21.15.26:0/974645066 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5581fce6ea20 msgr2=0x5581fce76b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:35:36.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.280+0000 7f0cb58cb640 1 --2- 172.21.15.26:0/974645066 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5581fce6ea20 0x5581fce76b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:36.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.280+0000 7f0cb58cb640 1 -- 172.21.15.26:0/974645066 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5581fcd9bc50 con 0x5581fce79c00 2024-09-11T23:35:36.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.281+0000 7f0cb58cb640 1 --2- 172.21.15.26:0/974645066 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5581fce79c00 0x5581fce775e0 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f0ca8009be0 tx=0x7f0ca802a720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3456b01bfd2df514 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:35:36.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.281+0000 7f0c98ff9640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:35:36.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.281+0000 7f0c98ff9640 1 -- 172.21.15.26:0/974645066 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ca8043070 con 0x5581fce79c00 2024-09-11T23:35:36.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.281+0000 7f0cb9707740 1 -- 172.21.15.26:0/974645066 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ca00041c0 con 0x5581fce79c00 2024-09-11T23:35:36.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.281+0000 7f0c98ff9640 1 -- 172.21.15.26:0/974645066 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 8586+0+0 (secure 0 0 0) 0x7f0ca8031850 con 0x5581fce79c00 2024-09-11T23:35:36.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.281+0000 7f0c98ff9640 1 -- 172.21.15.26:0/974645066 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7f0ca8058050 con 0x5581fce79c00 2024-09-11T23:35:36.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.285+0000 7f0cb9707740 10 client.17697 fetch_fsmap learned FSMap version 80 2024-09-11T23:35:36.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.285+0000 7f0cb9707740 1 -- 172.21.15.26:0/974645066 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=80,monmap=4+}) -- 0x5581fce99590 con 0x5581fce79c00 2024-09-11T23:35:36.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.287+0000 7f0c98ff9640 1 -- 172.21.15.26:0/974645066 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ca00041c0 con 0x5581fce79c00 2024-09-11T23:35:36.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.287+0000 7f0c98ff9640 1 -- 172.21.15.26:0/974645066 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 80) ==== 8426+0+0 (secure 0 0 0) 0x7f0ca803d460 con 0x5581fce79c00 2024-09-11T23:35:36.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.287+0000 7f0cb9707740 10 client.17697 fetch_fsmap finished waiting for FSMap version 80 2024-09-11T23:35:36.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.287+0000 7f0cb9707740 10 client.17697 resolve_mds: validated gid 24485 aka daemon mds.b 2024-09-11T23:35:36.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.287+0000 7f0cb9707740 20 client.17697 populate_metadata read hostname 'smithi026' 2024-09-11T23:35:36.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.287+0000 7f0cb9707740 1 --2- 172.21.15.26:0/974645066 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5581fce9aef0 0x5581fce9d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:36.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.287+0000 7f0cb9707740 4 client.17697 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:35:36.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.287+0000 7f0cb9707740 1 -- 172.21.15.26:0/974645066 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5581fcde0890 con 0x5581fce9aef0 2024-09-11T23:35:36.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.288+0000 7f0cb50ca640 1 --2- 172.21.15.26:0/974645066 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5581fce9aef0 0x5581fce9d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:36.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.300+0000 7f0cb50ca640 1 --2- 172.21.15.26:0/974645066 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5581fce9aef0 0x5581fce9d3b0 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c2e5e3d3fa2cff7 server_cookie=e64ddf196bb0483e in_seq=0 out_seq=0 2024-09-11T23:35:36.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.302+0000 7f0c98ff9640 10 client.17697 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:35:36.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.302+0000 7f0c98ff9640 1 -- 172.21.15.26:0/974645066 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5581fcde0890 con 0x5581fce9aef0 2024-09-11T23:35:36.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.302+0000 7f0c98ff9640 10 client.17697 handle_command_reply: tid=0 2024-09-11T23:35:36.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.302+0000 7f0cb9707740 1 client.17697 shutdown 2024-09-11T23:35:36.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.303+0000 7f0cb9707740 1 -- 172.21.15.26:0/974645066 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5581fce79c00 msgr2=0x5581fce775e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:36.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.303+0000 7f0cb9707740 1 --2- 172.21.15.26:0/974645066 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5581fce79c00 0x5581fce775e0 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f0ca8009be0 tx=0x7f0ca802a720 comp rx=0 tx=0).stop 2024-09-11T23:35:36.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.303+0000 7f0cb9707740 1 -- 172.21.15.26:0/974645066 shutdown_connections 2024-09-11T23:35:36.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.303+0000 7f0cb9707740 1 --2- 172.21.15.26:0/974645066 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5581fce9aef0 0x5581fce9d3b0 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:36.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.303+0000 7f0cb9707740 1 --2- 172.21.15.26:0/974645066 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5581fce79c00 0x5581fce775e0 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:36.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.303+0000 7f0cb9707740 1 --2- 172.21.15.26:0/974645066 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5581fce6f340 0x5581fce770a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:36.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.303+0000 7f0cb9707740 1 --2- 172.21.15.26:0/974645066 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5581fce6ea20 0x5581fce76b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:36.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.303+0000 7f0cb9707740 1 -- 172.21.15.26:0/974645066 >> 172.21.15.26:0/974645066 conn(0x5581fce6afb0 msgr2=0x5581fce70ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:35:36.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.307+0000 7f0c98ff9640 0 client.17697 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:35:36.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.309+0000 7f0cb9707740 1 -- 172.21.15.26:0/974645066 shutdown_connections 2024-09-11T23:35:36.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.310+0000 7f0cb9707740 1 -- 172.21.15.26:0/974645066 wait complete. 2024-09-11T23:35:36.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.310+0000 7f0cb9707740 20 client.17697 trim_cache size 0 max 16384 2024-09-11T23:35:36.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.343+0000 7f0cb9707740 1 Processor -- start 2024-09-11T23:35:36.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.343+0000 7f0cb9707740 1 -- start start 2024-09-11T23:35:36.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.343+0000 7f0cb9707740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5581fce6ea20 0x5581fce75430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:36.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.343+0000 7f0cb9707740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5581fce6f340 0x5581fce75970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:36.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.343+0000 7f0cb9707740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5581fce79c00 0x5581fcea2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:36.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.344+0000 7f0cb9707740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f0cb01ab6c0 con 0x5581fce79c00 2024-09-11T23:35:36.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.344+0000 7f0cb58cb640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5581fce79c00 0x5581fcea2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:36.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.344+0000 7f0cb50ca640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5581fce6ea20 0x5581fce75430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:36.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.344+0000 7f0cb58cb640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5581fce79c00 0x5581fcea2cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42824/0 (socket says 172.21.15.26:42824) 2024-09-11T23:35:36.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.344+0000 7f0cb58cb640 1 -- 172.21.15.26:0/1833765201 learned_addr learned my addr 172.21.15.26:0/1833765201 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:35:36.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.344+0000 7f0cb9707740 1 -- 172.21.15.26:0/1833765201 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f0cb01ad500 con 0x5581fce6ea20 2024-09-11T23:35:36.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.344+0000 7f0cb9707740 1 -- 172.21.15.26:0/1833765201 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f0cb01abe50 con 0x5581fce6f340 2024-09-11T23:35:36.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.345+0000 7f0cb58cb640 1 -- 172.21.15.26:0/1833765201 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5581fce6f340 msgr2=0x5581fce75970 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T23:35:36.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.345+0000 7f0cb58cb640 1 --2- 172.21.15.26:0/1833765201 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5581fce6f340 0x5581fce75970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:36.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.345+0000 7f0cb58cb640 1 -- 172.21.15.26:0/1833765201 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5581fce6ea20 msgr2=0x5581fce75430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:35:36.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.345+0000 7f0cb58cb640 1 --2- 172.21.15.26:0/1833765201 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5581fce6ea20 0x5581fce75430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:36.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.345+0000 7f0cb58cb640 1 -- 172.21.15.26:0/1833765201 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5581fce750b0 con 0x5581fce79c00 2024-09-11T23:35:36.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.346+0000 7f0cb58cb640 1 --2- 172.21.15.26:0/1833765201 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5581fce79c00 0x5581fcea2cc0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f0ca802ac40 tx=0x7f0ca800b160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5aca741bd819490 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:35:36.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.346+0000 7f0c9affd640 1 -- 172.21.15.26:0/1833765201 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ca8043040 con 0x5581fce79c00 2024-09-11T23:35:36.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.346+0000 7f0c9affd640 1 -- 172.21.15.26:0/1833765201 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ca8063070 con 0x5581fce79c00 2024-09-11T23:35:36.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.346+0000 7f0c9affd640 1 -- 172.21.15.26:0/1833765201 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ca8036040 con 0x5581fce79c00 2024-09-11T23:35:36.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.348+0000 7f0cb9707740 1 -- 172.21.15.26:0/1833765201 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5581fce79c00 msgr2=0x5581fcea2cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:36.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.348+0000 7f0cb9707740 1 --2- 172.21.15.26:0/1833765201 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5581fce79c00 0x5581fcea2cc0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f0ca802ac40 tx=0x7f0ca800b160 comp rx=0 tx=0).stop 2024-09-11T23:35:36.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.348+0000 7f0cb9707740 1 -- 172.21.15.26:0/1833765201 shutdown_connections 2024-09-11T23:35:36.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.348+0000 7f0cb9707740 1 --2- 172.21.15.26:0/1833765201 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5581fce79c00 0x5581fcea2cc0 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:36.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.348+0000 7f0cb9707740 1 --2- 172.21.15.26:0/1833765201 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5581fce6f340 0x5581fce75970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:36.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.348+0000 7f0cb9707740 1 --2- 172.21.15.26:0/1833765201 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5581fce6ea20 0x5581fce75430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:36.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.348+0000 7f0cb9707740 1 -- 172.21.15.26:0/1833765201 >> 172.21.15.26:0/1833765201 conn(0x5581fce6afb0 msgr2=0x5581fce71de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:35:36.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.348+0000 7f0cb9707740 1 -- 172.21.15.26:0/1833765201 shutdown_connections 2024-09-11T23:35:36.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.348+0000 7f0cb9707740 1 -- 172.21.15.26:0/1833765201 wait complete. 2024-09-11T23:35:36.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.349+0000 7f0cb9707740 1 -- start start 2024-09-11T23:35:36.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.349+0000 7f0cb9707740 1 Processor -- start 2024-09-11T23:35:36.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.350+0000 7f0cb9707740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5581fce6ea20 0x5581fce7d180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:36.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.350+0000 7f0cb9707740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5581fce6f340 0x5581fce7d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:36.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.350+0000 7f0cb50ca640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5581fce6ea20 0x5581fce7d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:36.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.350+0000 7f0cb50ca640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5581fce6ea20 0x5581fce7d180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:43126/0 (socket says 172.21.15.26:43126) 2024-09-11T23:35:36.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.350+0000 7f0cb50ca640 1 -- 172.21.15.26:0/2046917756 learned_addr learned my addr 172.21.15.26:0/2046917756 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:35:36.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.350+0000 7f0cb9707740 1 --2- 172.21.15.26:0/2046917756 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5581fce79c00 0x5581fcf03c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:36.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.351+0000 7f0cb9707740 1 -- 172.21.15.26:0/2046917756 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f0cb01acd70 con 0x5581fce6f340 2024-09-11T23:35:36.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.351+0000 7f0cb9707740 1 -- 172.21.15.26:0/2046917756 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f0cb01ac5e0 con 0x5581fce79c00 2024-09-11T23:35:36.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.351+0000 7f0cb9707740 1 -- 172.21.15.26:0/2046917756 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f0cb01971b0 con 0x5581fce6ea20 2024-09-11T23:35:36.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.351+0000 7f0cb9707740 10 client.0 _ll_register_callbacks cb 0x7f0ca8034070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:35:36.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.355+0000 7f0cb48c9640 1 --2- 172.21.15.26:0/2046917756 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5581fce6f340 0x5581fce7d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:36.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.356+0000 7f0cb58cb640 1 --2- 172.21.15.26:0/2046917756 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5581fce79c00 0x5581fcf03c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:36.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.357+0000 7f0cb48c9640 1 -- 172.21.15.26:0/2046917756 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5581fce6ea20 msgr2=0x5581fce7d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:35:36.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.357+0000 7f0cb48c9640 1 --2- 172.21.15.26:0/2046917756 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5581fce6ea20 0x5581fce7d180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:36.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.357+0000 7f0cb48c9640 1 -- 172.21.15.26:0/2046917756 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5581fce79c00 msgr2=0x5581fcf03c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:35:36.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.357+0000 7f0cb48c9640 1 --2- 172.21.15.26:0/2046917756 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5581fce79c00 0x5581fcf03c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:36.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.357+0000 7f0cb48c9640 1 -- 172.21.15.26:0/2046917756 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5581fcf041c0 con 0x5581fce6f340 2024-09-11T23:35:36.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.358+0000 7f0cb48c9640 1 --2- 172.21.15.26:0/2046917756 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5581fce6f340 0x5581fce7d6c0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x5581fce781a0 tx=0x7f0ca0009fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ce100c57c6fa8dd server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:35:36.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.358+0000 7f0c98ff9640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:35:36.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.359+0000 7f0c98ff9640 1 -- 172.21.15.26:0/2046917756 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ca0003880 con 0x5581fce6f340 2024-09-11T23:35:36.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.359+0000 7f0c98ff9640 1 -- 172.21.15.26:0/2046917756 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 8586+0+0 (secure 0 0 0) 0x7f0ca00050a0 con 0x5581fce6f340 2024-09-11T23:35:36.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.361+0000 7f0cb9707740 1 -- 172.21.15.26:0/2046917756 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ca8063070 con 0x5581fce6f340 2024-09-11T23:35:36.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.362+0000 7f0c98ff9640 1 -- 172.21.15.26:0/2046917756 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7f0ca0003a20 con 0x5581fce6f340 2024-09-11T23:35:36.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.362+0000 7f0cb9707740 10 client.17703 fetch_fsmap learned FSMap version 80 2024-09-11T23:35:36.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.362+0000 7f0cb9707740 1 -- 172.21.15.26:0/2046917756 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=80,monmap=4+}) -- 0x5581fcf05410 con 0x5581fce6f340 2024-09-11T23:35:36.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.363+0000 7f0c98ff9640 1 -- 172.21.15.26:0/2046917756 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ca8063070 con 0x5581fce6f340 2024-09-11T23:35:36.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.363+0000 7f0c98ff9640 1 -- 172.21.15.26:0/2046917756 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 80) ==== 8426+0+0 (secure 0 0 0) 0x7f0ca001a400 con 0x5581fce6f340 2024-09-11T23:35:36.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.363+0000 7f0cb9707740 10 client.17703 fetch_fsmap finished waiting for FSMap version 80 2024-09-11T23:35:36.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.363+0000 7f0cb9707740 10 client.17703 resolve_mds: validated gid 24485 aka daemon mds.b 2024-09-11T23:35:36.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.363+0000 7f0cb9707740 20 client.17703 populate_metadata read hostname 'smithi026' 2024-09-11T23:35:36.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.363+0000 7f0cb9707740 1 --2- 172.21.15.26:0/2046917756 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5581fcf05770 0x5581fcf07b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:36.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.363+0000 7f0cb9707740 4 client.17703 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 144.25, "await_for": 45.0}] 2024-09-11T23:35:36.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.363+0000 7f0cb9707740 1 -- 172.21.15.26:0/2046917756 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 144.25, "await_for": 45.0}) -- 0x5581fcddabe0 con 0x5581fcf05770 2024-09-11T23:35:36.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.364+0000 7f0cb50ca640 1 --2- 172.21.15.26:0/2046917756 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5581fcf05770 0x5581fcf07b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:36.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.365+0000 7f0c98ff9640 10 client.17703 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:35:36.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:36.365+0000 7f0cb50ca640 1 --2- 172.21.15.26:0/2046917756 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5581fcf05770 0x5581fcf07b60 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74e37d7b32fa3858 server_cookie=1718d6f2e32cd496 in_seq=0 out_seq=0 2024-09-11T23:35:38.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:35:38 smithi136 ceph-mon[35121]: pgmap v541: 97 pgs: 19 active+clean+snaptrim, 35 active+clean+snaptrim_wait, 43 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 838 KiB/s rd, 826 KiB/s wr, 213 op/s 2024-09-11T23:35:38.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:35:38 smithi026 ceph-mon[31172]: pgmap v541: 97 pgs: 19 active+clean+snaptrim, 35 active+clean+snaptrim_wait, 43 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 838 KiB/s rd, 826 KiB/s wr, 213 op/s 2024-09-11T23:35:38.552 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:35:38 smithi076 ceph-mon[35296]: pgmap v541: 97 pgs: 19 active+clean+snaptrim, 35 active+clean+snaptrim_wait, 43 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 838 KiB/s rd, 826 KiB/s wr, 213 op/s 2024-09-11T23:35:40.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:35:40 smithi136 ceph-mon[35121]: pgmap v542: 97 pgs: 19 active+clean+snaptrim, 35 active+clean+snaptrim_wait, 43 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 508 KiB/s rd, 538 KiB/s wr, 177 op/s 2024-09-11T23:35:40.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:35:40 smithi026 ceph-mon[31172]: pgmap v542: 97 pgs: 19 active+clean+snaptrim, 35 active+clean+snaptrim_wait, 43 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 508 KiB/s rd, 538 KiB/s wr, 177 op/s 2024-09-11T23:35:40.552 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:35:40 smithi076 ceph-mon[35296]: pgmap v542: 97 pgs: 19 active+clean+snaptrim, 35 active+clean+snaptrim_wait, 43 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 508 KiB/s rd, 538 KiB/s wr, 177 op/s 2024-09-11T23:35:41.709 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.708+0000 7f3c375f1640 1 -- 172.21.15.26:0/3280008884 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3c301095d0 msgr2=0x7f3c30109a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:41.710 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.708+0000 7f3c375f1640 1 --2- 172.21.15.26:0/3280008884 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3c301095d0 0x7f3c30109a10 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f3c20009890 tx=0x7f3c2002a390 comp rx=0 tx=0).stop 2024-09-11T23:35:41.714 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.712+0000 7f3c375f1640 1 -- 172.21.15.26:0/3280008884 shutdown_connections 2024-09-11T23:35:41.714 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.712+0000 7f3c375f1640 1 --2- 172.21.15.26:0/3280008884 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3c301095d0 0x7f3c30109a10 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:41.714 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.712+0000 7f3c375f1640 1 --2- 172.21.15.26:0/3280008884 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3c3011b140 0x7f3c3011d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:41.714 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.712+0000 7f3c375f1640 1 --2- 172.21.15.26:0/3280008884 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3c301091f0 0x7f3c30110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:41.714 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.712+0000 7f3c375f1640 1 -- 172.21.15.26:0/3280008884 >> 172.21.15.26:0/3280008884 conn(0x7f3c3006ce00 msgr2=0x7f3c3006d210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:35:41.718 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.716+0000 7f3c375f1640 1 -- 172.21.15.26:0/3280008884 shutdown_connections 2024-09-11T23:35:41.720 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.718+0000 7f3c375f1640 1 -- 172.21.15.26:0/3280008884 wait complete. 2024-09-11T23:35:41.721 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.718+0000 7f3c375f1640 1 Processor -- start 2024-09-11T23:35:41.721 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.718+0000 7f3c375f1640 1 -- start start 2024-09-11T23:35:41.721 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.719+0000 7f3c375f1640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3c301091f0 0x7f3c301185a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:41.721 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.719+0000 7f3c375f1640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3c301095d0 0x7f3c30118ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:41.721 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.719+0000 7f3c375f1640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3c3011b140 0x7f3c301116d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:41.721 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.719+0000 7f3c375f1640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f3c30120d50 con 0x7f3c301091f0 2024-09-11T23:35:41.721 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.719+0000 7f3c375f1640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f3c30120bd0 con 0x7f3c301095d0 2024-09-11T23:35:41.721 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.719+0000 7f3c375f1640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f3c30120ed0 con 0x7f3c3011b140 2024-09-11T23:35:41.721 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.719+0000 7f3c35b8e640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3c3011b140 0x7f3c301116d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:41.721 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.719+0000 7f3c35b8e640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3c3011b140 0x7f3c301116d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:50442/0 (socket says 172.21.15.26:50442) 2024-09-11T23:35:41.721 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.719+0000 7f3c35b8e640 1 -- 172.21.15.26:0/613606576 learned_addr learned my addr 172.21.15.26:0/613606576 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:35:41.721 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.719+0000 7f3c3538d640 1 --2- 172.21.15.26:0/613606576 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3c301091f0 0x7f3c301185a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:41.721 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.719+0000 7f3c34b8c640 1 --2- 172.21.15.26:0/613606576 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3c301095d0 0x7f3c30118ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:41.722 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.720+0000 7f3c35b8e640 1 -- 172.21.15.26:0/613606576 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3c301095d0 msgr2=0x7f3c30118ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:41.722 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.720+0000 7f3c35b8e640 1 --2- 172.21.15.26:0/613606576 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3c301095d0 0x7f3c30118ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:41.722 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.720+0000 7f3c35b8e640 1 -- 172.21.15.26:0/613606576 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3c301091f0 msgr2=0x7f3c301185a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:41.722 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.720+0000 7f3c35b8e640 1 --2- 172.21.15.26:0/613606576 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3c301091f0 0x7f3c301185a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:41.722 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.720+0000 7f3c35b8e640 1 -- 172.21.15.26:0/613606576 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c30111f90 con 0x7f3c3011b140 2024-09-11T23:35:41.722 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.720+0000 7f3c35b8e640 1 --2- 172.21.15.26:0/613606576 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3c3011b140 0x7f3c301116d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f3c2800efc0 tx=0x7f3c2800c800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:35:41.722 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.720+0000 7f3c267fc640 1 -- 172.21.15.26:0/613606576 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c28003970 con 0x7f3c3011b140 2024-09-11T23:35:41.723 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.720+0000 7f3c375f1640 1 -- 172.21.15.26:0/613606576 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c30112280 con 0x7f3c3011b140 2024-09-11T23:35:41.723 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.720+0000 7f3c375f1640 1 -- 172.21.15.26:0/613606576 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c301162a0 con 0x7f3c3011b140 2024-09-11T23:35:41.723 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.721+0000 7f3c267fc640 1 -- 172.21.15.26:0/613606576 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c28004020 con 0x7f3c3011b140 2024-09-11T23:35:41.723 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.721+0000 7f3c267fc640 1 -- 172.21.15.26:0/613606576 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c2800ba60 con 0x7f3c3011b140 2024-09-11T23:35:41.728 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.726+0000 7f3c3a463740 1 Processor -- start 2024-09-11T23:35:41.731 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.728+0000 7f3c267fc640 1 -- 172.21.15.26:0/613606576 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f3c2801d020 con 0x7f3c3011b140 2024-09-11T23:35:41.731 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.729+0000 7f3c267fc640 1 --2- 172.21.15.26:0/613606576 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f3c1c0a2fc0 0x7f3c1c0a5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:41.731 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.730+0000 7f3c267fc640 1 -- 172.21.15.26:0/613606576 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 8586+0+0 (secure 0 0 0) 0x7f3c280cf760 con 0x7f3c3011b140 2024-09-11T23:35:41.731 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.730+0000 7f3c3a463740 1 -- start start 2024-09-11T23:35:41.731 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.730+0000 7f3c3a463740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564fb2fc3af0 0x564fb2fc3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:41.731 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.730+0000 7f3c3a463740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564fb2fc4410 0x564fb2fce770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:41.731 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.730+0000 7f3c3a463740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564fb2fcecb0 0x564fb2fd10a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:41.732 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.730+0000 7f3c3a463740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x564fb2f282f0 con 0x564fb2fcecb0 2024-09-11T23:35:41.732 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.730+0000 7f3c3a463740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x564fb2ed0270 con 0x564fb2fc3af0 2024-09-11T23:35:41.732 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.730+0000 7f3c3a463740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x564fb2f242b0 con 0x564fb2fc4410 2024-09-11T23:35:41.733 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.731+0000 7f3c34b8c640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564fb2fc4410 0x564fb2fce770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:41.733 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.731+0000 7f3c34b8c640 1 --2- 172.21.15.26:0/613606576 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f3c1c0a2fc0 0x7f3c1c0a5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:41.733 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.731+0000 7f3c34b8c640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564fb2fc4410 0x564fb2fce770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:50456/0 (socket says 172.21.15.26:50456) 2024-09-11T23:35:41.733 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.731+0000 7f3c34b8c640 1 -- 172.21.15.26:0/1156662466 learned_addr learned my addr 172.21.15.26:0/1156662466 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:35:41.734 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.731+0000 7f3c35b8e640 1 --2- 172.21.15.26:0/1156662466 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564fb2fcecb0 0x564fb2fd10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:41.734 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.732+0000 7f3c35b8e640 1 -- 172.21.15.26:0/1156662466 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564fb2fc4410 msgr2=0x564fb2fce770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:35:41.734 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.732+0000 7f3c35b8e640 1 --2- 172.21.15.26:0/1156662466 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564fb2fc4410 0x564fb2fce770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:41.734 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.732+0000 7f3c35b8e640 1 -- 172.21.15.26:0/1156662466 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564fb2fc3af0 msgr2=0x564fb2fc3ed0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T23:35:41.734 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.732+0000 7f3c35b8e640 1 --2- 172.21.15.26:0/1156662466 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564fb2fc3af0 0x564fb2fc3ed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:41.734 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.732+0000 7f3c35b8e640 1 -- 172.21.15.26:0/1156662466 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564fb2ecf340 con 0x564fb2fcecb0 2024-09-11T23:35:41.734 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.733+0000 7f3c35b8e640 1 --2- 172.21.15.26:0/1156662466 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564fb2fcecb0 0x564fb2fd10a0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f3c280cce10 tx=0x7f3c2809c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74d0d837a304a69c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:35:41.734 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.733+0000 7f3c34b8c640 1 --2- 172.21.15.26:0/613606576 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f3c1c0a2fc0 0x7f3c1c0a5480 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x564fb2fc0ec0 tx=0x7f3c2002a390 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:35:41.735 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.733+0000 7f3c1b7fe640 1 -- 172.21.15.26:0/1156662466 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c28014df0 con 0x564fb2fcecb0 2024-09-11T23:35:41.735 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.733+0000 7f3c1b7fe640 1 -- 172.21.15.26:0/1156662466 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c28098250 con 0x564fb2fcecb0 2024-09-11T23:35:41.736 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.734+0000 7f3c3a463740 1 -- 172.21.15.26:0/1156662466 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564fb2fcecb0 msgr2=0x564fb2fd10a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:41.736 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.734+0000 7f3c3a463740 1 --2- 172.21.15.26:0/1156662466 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564fb2fcecb0 0x564fb2fd10a0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f3c280cce10 tx=0x7f3c2809c3a0 comp rx=0 tx=0).stop 2024-09-11T23:35:41.736 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.735+0000 7f3c3a463740 1 -- 172.21.15.26:0/1156662466 shutdown_connections 2024-09-11T23:35:41.736 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.735+0000 7f3c3a463740 1 --2- 172.21.15.26:0/1156662466 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564fb2fcecb0 0x564fb2fd10a0 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:41.736 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.735+0000 7f3c3a463740 1 --2- 172.21.15.26:0/1156662466 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564fb2fc4410 0x564fb2fce770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:41.736 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.735+0000 7f3c3a463740 1 --2- 172.21.15.26:0/1156662466 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564fb2fc3af0 0x564fb2fc3ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:41.736 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.735+0000 7f3c3a463740 1 -- 172.21.15.26:0/1156662466 >> 172.21.15.26:0/1156662466 conn(0x564fb2fc0060 msgr2=0x564fb2fc0440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:35:41.736 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.735+0000 7f3c3a463740 1 -- 172.21.15.26:0/1156662466 shutdown_connections 2024-09-11T23:35:41.738 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.737+0000 7f3c3a463740 1 -- 172.21.15.26:0/1156662466 wait complete. 2024-09-11T23:35:41.738 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.737+0000 7f3c3a463740 1 -- start start 2024-09-11T23:35:41.738 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.737+0000 7f3c3a463740 1 Processor -- start 2024-09-11T23:35:41.743 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.742+0000 7f3c3a463740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564fb2fc3af0 0x564fb2fcba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:41.743 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.742+0000 7f3c3a463740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564fb2fc4410 0x564fb2fcbfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:41.743 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.742+0000 7f3c3a463740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564fb2fcecb0 0x564fb2fcc4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:41.743 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.742+0000 7f3c3a463740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x564fb2fd18d0 con 0x564fb2fcecb0 2024-09-11T23:35:41.743 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.742+0000 7f3c3a463740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x564fb2eb1290 con 0x564fb2fc3af0 2024-09-11T23:35:41.743 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.742+0000 7f3c3a463740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x564fb2fd1a50 con 0x564fb2fc4410 2024-09-11T23:35:41.743 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.742+0000 7f3c3a463740 10 client.0 _ll_register_callbacks cb 0x564fb2ee2140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:35:41.744 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.743+0000 7f3c3538d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564fb2fc3af0 0x564fb2fcba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:41.744 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.743+0000 7f3c3538d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564fb2fc3af0 0x564fb2fcba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:39164/0 (socket says 172.21.15.26:39164) 2024-09-11T23:35:41.744 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.743+0000 7f3c3538d640 1 -- 172.21.15.26:0/1057386087 learned_addr learned my addr 172.21.15.26:0/1057386087 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:35:41.745 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.744+0000 7f3c35b8e640 1 --2- 172.21.15.26:0/1057386087 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564fb2fcecb0 0x564fb2fcc4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:41.745 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.744+0000 7f3c3538d640 1 -- 172.21.15.26:0/1057386087 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564fb2fc4410 msgr2=0x564fb2fcbfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:35:41.745 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.744+0000 7f3c3538d640 1 --2- 172.21.15.26:0/1057386087 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564fb2fc4410 0x564fb2fcbfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:41.745 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.744+0000 7f3c3538d640 1 -- 172.21.15.26:0/1057386087 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564fb2fcecb0 msgr2=0x564fb2fcc4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:35:41.745 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.744+0000 7f3c3538d640 1 --2- 172.21.15.26:0/1057386087 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564fb2fcecb0 0x564fb2fcc4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:41.746 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.744+0000 7f3c3538d640 1 -- 172.21.15.26:0/1057386087 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564fb2c3b210 con 0x564fb2fc3af0 2024-09-11T23:35:41.746 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.745+0000 7f3c3538d640 1 --2- 172.21.15.26:0/1057386087 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564fb2fc3af0 0x564fb2fcba70 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f3c2c00d210 tx=0x7f3c2c00bf90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=32191fa15676521 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:35:41.746 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.745+0000 7f3c18ff9640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-11T23:35:41.746 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.745+0000 7f3c18ff9640 1 -- 172.21.15.26:0/1057386087 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c2c0125f0 con 0x564fb2fc3af0 2024-09-11T23:35:41.746 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.745+0000 7f3c3a463740 1 -- 172.21.15.26:0/1057386087 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3c28098250 con 0x564fb2fc3af0 2024-09-11T23:35:41.748 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.745+0000 7f3c18ff9640 1 -- 172.21.15.26:0/1057386087 <== mon.2 v2:172.21.15.76:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 8586+0+0 (secure 0 0 0) 0x7f3c2c012790 con 0x564fb2fc3af0 2024-09-11T23:35:41.748 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.745+0000 7f3c18ff9640 1 -- 172.21.15.26:0/1057386087 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7f3c2c0129d0 con 0x564fb2fc3af0 2024-09-11T23:35:41.748 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.745+0000 7f3c3a463740 10 client.27230 fetch_fsmap learned FSMap version 80 2024-09-11T23:35:41.748 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.745+0000 7f3c3a463740 1 -- 172.21.15.26:0/1057386087 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=80,monmap=4+}) -- 0x564fb2d07620 con 0x564fb2fc3af0 2024-09-11T23:35:41.748 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.746+0000 7f3c18ff9640 1 -- 172.21.15.26:0/1057386087 <== mon.2 v2:172.21.15.76:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c28098250 con 0x564fb2fc3af0 2024-09-11T23:35:41.748 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.747+0000 7f3c18ff9640 1 -- 172.21.15.26:0/1057386087 <== mon.2 v2:172.21.15.76:3300/0 5 ==== fsmap(e 80) ==== 8426+0+0 (secure 0 0 0) 0x7f3c2c013070 con 0x564fb2fc3af0 2024-09-11T23:35:41.749 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.748+0000 7f3c3a463740 10 client.27230 fetch_fsmap finished waiting for FSMap version 80 2024-09-11T23:35:41.749 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.748+0000 7f3c3a463740 10 client.27230 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:35:41.749 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.748+0000 7f3c3a463740 20 client.27230 populate_metadata read hostname 'smithi026' 2024-09-11T23:35:41.749 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.748+0000 7f3c3a463740 1 --2- 172.21.15.26:0/1057386087 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x564fb2fdecd0 0x564fb2fe1150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:41.749 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.748+0000 7f3c3a463740 4 client.27230 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:35:41.749 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.748+0000 7f3c3a463740 1 -- 172.21.15.26:0/1057386087 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564fb2f35a20 con 0x564fb2fdecd0 2024-09-11T23:35:41.749 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.748+0000 7f3c34b8c640 1 --2- 172.21.15.26:0/1057386087 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x564fb2fdecd0 0x564fb2fe1150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:41.751 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.751+0000 7f3c34b8c640 1 --2- 172.21.15.26:0/1057386087 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x564fb2fdecd0 0x564fb2fe1150 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5685d435fe4858f5 server_cookie=dfe14518fd7e410 in_seq=0 out_seq=0 2024-09-11T23:35:41.751 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.751+0000 7f3c18ff9640 10 client.27230 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:35:41.772 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.772+0000 7f3c18ff9640 1 -- 172.21.15.26:0/1057386087 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564fb2f35a20 con 0x564fb2fdecd0 2024-09-11T23:35:41.772 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.772+0000 7f3c18ff9640 10 client.27230 handle_command_reply: tid=0 2024-09-11T23:35:41.773 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.772+0000 7f3c3a463740 1 client.27230 shutdown 2024-09-11T23:35:41.774 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.774+0000 7f3c3a463740 1 -- 172.21.15.26:0/1057386087 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564fb2fc3af0 msgr2=0x564fb2fcba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:41.774 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.774+0000 7f3c3a463740 1 --2- 172.21.15.26:0/1057386087 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564fb2fc3af0 0x564fb2fcba70 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f3c2c00d210 tx=0x7f3c2c00bf90 comp rx=0 tx=0).stop 2024-09-11T23:35:41.778 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.778+0000 7f3c3a463740 1 -- 172.21.15.26:0/1057386087 shutdown_connections 2024-09-11T23:35:41.778 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.778+0000 7f3c3a463740 1 --2- 172.21.15.26:0/1057386087 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x564fb2fdecd0 0x564fb2fe1150 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:41.778 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.778+0000 7f3c3a463740 1 --2- 172.21.15.26:0/1057386087 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564fb2fcecb0 0x564fb2fcc4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:41.778 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.778+0000 7f3c3a463740 1 --2- 172.21.15.26:0/1057386087 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564fb2fc4410 0x564fb2fcbfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:41.778 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.778+0000 7f3c3a463740 1 --2- 172.21.15.26:0/1057386087 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564fb2fc3af0 0x564fb2fcba70 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:41.778 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.778+0000 7f3c3a463740 1 -- 172.21.15.26:0/1057386087 >> 172.21.15.26:0/1057386087 conn(0x564fb2fc0060 msgr2=0x564fb2fbf1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:35:41.779 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.779+0000 7f3c18ff9640 0 client.27230 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:35:41.785 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.779+0000 7f3c3a463740 1 -- 172.21.15.26:0/1057386087 shutdown_connections 2024-09-11T23:35:41.785 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.782+0000 7f3c3a463740 1 -- 172.21.15.26:0/1057386087 wait complete. 2024-09-11T23:35:41.785 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.783+0000 7f3c3a463740 20 client.27230 trim_cache size 0 max 16384 2024-09-11T23:35:41.814 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.812+0000 7f3c3a463740 1 Processor -- start 2024-09-11T23:35:41.814 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.812+0000 7f3c3a463740 1 -- start start 2024-09-11T23:35:41.814 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.813+0000 7f3c3a463740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564fb2fc3af0 0x564fb2fe6a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:41.814 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.813+0000 7f3c3a463740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564fb2fc4410 0x564fb2fe6fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:41.814 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.813+0000 7f3c3a463740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564fb2fcecb0 0x564fb2fd00a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:41.814 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.813+0000 7f3c3a463740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f3c1400e8e0 con 0x564fb2fc3af0 2024-09-11T23:35:41.814 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.813+0000 7f3c3a463740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f3c14010720 con 0x564fb2fcecb0 2024-09-11T23:35:41.814 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.813+0000 7f3c3a463740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f3c1400f070 con 0x564fb2fc4410 2024-09-11T23:35:41.814 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.813+0000 7f3c35b8e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564fb2fcecb0 0x564fb2fd00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:41.815 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.813+0000 7f3c35b8e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564fb2fcecb0 0x564fb2fd00a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:39176/0 (socket says 172.21.15.26:39176) 2024-09-11T23:35:41.815 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.813+0000 7f3c35b8e640 1 -- 172.21.15.26:0/946975423 learned_addr learned my addr 172.21.15.26:0/946975423 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:35:41.816 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.814+0000 7f3c3538d640 1 --2- 172.21.15.26:0/946975423 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564fb2fc3af0 0x564fb2fe6a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:41.816 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.814+0000 7f3c3538d640 1 -- 172.21.15.26:0/946975423 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564fb2fc4410 msgr2=0x564fb2fe6fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:35:41.816 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.814+0000 7f3c3538d640 1 --2- 172.21.15.26:0/946975423 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564fb2fc4410 0x564fb2fe6fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:41.816 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.814+0000 7f3c3538d640 1 -- 172.21.15.26:0/946975423 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564fb2fcecb0 msgr2=0x564fb2fd00a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:35:41.816 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.814+0000 7f3c3538d640 1 --2- 172.21.15.26:0/946975423 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564fb2fcecb0 0x564fb2fd00a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:41.816 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.815+0000 7f3c3538d640 1 -- 172.21.15.26:0/946975423 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564fb2fc4240 con 0x564fb2fc3af0 2024-09-11T23:35:41.816 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.815+0000 7f3c3538d640 1 --2- 172.21.15.26:0/946975423 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564fb2fc3af0 0x564fb2fe6a60 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f3c2c00d210 tx=0x7f3c2c00b360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3452876f5b29edb3 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:35:41.817 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.816+0000 7f3c1affd640 1 -- 172.21.15.26:0/946975423 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c2c00db60 con 0x564fb2fc3af0 2024-09-11T23:35:41.817 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.816+0000 7f3c1affd640 1 -- 172.21.15.26:0/946975423 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c2c019070 con 0x564fb2fc3af0 2024-09-11T23:35:41.817 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.816+0000 7f3c1affd640 1 -- 172.21.15.26:0/946975423 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c2c012ca0 con 0x564fb2fc3af0 2024-09-11T23:35:41.817 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.817+0000 7f3c3a463740 1 -- 172.21.15.26:0/946975423 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564fb2fc3af0 msgr2=0x564fb2fe6a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:41.817 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.817+0000 7f3c3a463740 1 --2- 172.21.15.26:0/946975423 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564fb2fc3af0 0x564fb2fe6a60 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f3c2c00d210 tx=0x7f3c2c00b360 comp rx=0 tx=0).stop 2024-09-11T23:35:41.817 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.817+0000 7f3c3a463740 1 -- 172.21.15.26:0/946975423 shutdown_connections 2024-09-11T23:35:41.817 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.817+0000 7f3c3a463740 1 --2- 172.21.15.26:0/946975423 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564fb2fcecb0 0x564fb2fd00a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:41.817 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.817+0000 7f3c3a463740 1 --2- 172.21.15.26:0/946975423 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564fb2fc4410 0x564fb2fe6fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:41.818 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.817+0000 7f3c3a463740 1 --2- 172.21.15.26:0/946975423 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564fb2fc3af0 0x564fb2fe6a60 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:41.818 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.817+0000 7f3c3a463740 1 -- 172.21.15.26:0/946975423 >> 172.21.15.26:0/946975423 conn(0x564fb2fc0060 msgr2=0x564fb2fcdaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:35:41.818 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.818+0000 7f3c3a463740 1 -- 172.21.15.26:0/946975423 shutdown_connections 2024-09-11T23:35:41.821 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.820+0000 7f3c3a463740 1 -- 172.21.15.26:0/946975423 wait complete. 2024-09-11T23:35:41.821 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.821+0000 7f3c3a463740 1 -- start start 2024-09-11T23:35:41.822 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.821+0000 7f3c3a463740 1 Processor -- start 2024-09-11T23:35:41.822 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.822+0000 7f3c3a463740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564fb2fc3af0 0x564fb2fd71a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:41.822 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.822+0000 7f3c3a463740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564fb2fc4410 0x564fb2fd24c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:41.822 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.822+0000 7f3c3a463740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564fb2fcecb0 0x564fb2fd2a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:41.822 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.822+0000 7f3c3a463740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f3c1400ff90 con 0x564fb2fc4410 2024-09-11T23:35:41.822 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.822+0000 7f3c3a463740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f3c1400f800 con 0x564fb2fcecb0 2024-09-11T23:35:41.822 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.822+0000 7f3c3a463740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f3c1400c8e0 con 0x564fb2fc3af0 2024-09-11T23:35:41.823 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.822+0000 7f3c3a463740 10 client.0 _ll_register_callbacks cb 0x7f3c2c00b1c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:35:41.823 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.822+0000 7f3c3538d640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564fb2fc3af0 0x564fb2fd71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:41.823 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.822+0000 7f3c3538d640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564fb2fc3af0 0x564fb2fd71a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:50484/0 (socket says 172.21.15.26:50484) 2024-09-11T23:35:41.823 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.822+0000 7f3c3538d640 1 -- 172.21.15.26:0/3878881561 learned_addr learned my addr 172.21.15.26:0/3878881561 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:35:41.823 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.822+0000 7f3c34b8c640 1 --2- 172.21.15.26:0/3878881561 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564fb2fc4410 0x564fb2fd24c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:41.824 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.823+0000 7f3c35b8e640 1 --2- 172.21.15.26:0/3878881561 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564fb2fcecb0 0x564fb2fd2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:41.824 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.823+0000 7f3c3538d640 1 -- 172.21.15.26:0/3878881561 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564fb2fcecb0 msgr2=0x564fb2fd2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:35:41.824 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.823+0000 7f3c3538d640 1 --2- 172.21.15.26:0/3878881561 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564fb2fcecb0 0x564fb2fd2a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:41.824 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.823+0000 7f3c3538d640 1 -- 172.21.15.26:0/3878881561 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564fb2fc4410 msgr2=0x564fb2fd24c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:35:41.824 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.823+0000 7f3c3538d640 1 --2- 172.21.15.26:0/3878881561 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564fb2fc4410 0x564fb2fd24c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:41.824 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.823+0000 7f3c3538d640 1 -- 172.21.15.26:0/3878881561 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564fb2fd2f40 con 0x564fb2fc3af0 2024-09-11T23:35:41.825 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.823+0000 7f3c3538d640 1 --2- 172.21.15.26:0/3878881561 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564fb2fc3af0 0x564fb2fd71a0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f3c2c014660 tx=0x7f3c2c014690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=476a443e85a5056b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:35:41.825 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.823+0000 7f3c18ff9640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-09-11T23:35:41.825 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.823+0000 7f3c18ff9640 1 -- 172.21.15.26:0/3878881561 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c2c012d80 con 0x564fb2fc3af0 2024-09-11T23:35:41.825 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.824+0000 7f3c3a463740 1 -- 172.21.15.26:0/3878881561 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3c2c019070 con 0x564fb2fc3af0 2024-09-11T23:35:41.825 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.824+0000 7f3c18ff9640 1 -- 172.21.15.26:0/3878881561 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 8586+0+0 (secure 0 0 0) 0x7f3c2c005990 con 0x564fb2fc3af0 2024-09-11T23:35:41.827 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.827+0000 7f3c18ff9640 1 -- 172.21.15.26:0/3878881561 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7f3c2c005c10 con 0x564fb2fc3af0 2024-09-11T23:35:41.834 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.834+0000 7f3c3a463740 10 client.27199 fetch_fsmap learned FSMap version 80 2024-09-11T23:35:41.834 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.834+0000 7f3c3a463740 1 -- 172.21.15.26:0/3878881561 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=80,monmap=4+}) -- 0x564fb3049730 con 0x564fb2fc3af0 2024-09-11T23:35:41.838 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.837+0000 7f3c18ff9640 1 -- 172.21.15.26:0/3878881561 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c2c019070 con 0x564fb2fc3af0 2024-09-11T23:35:41.838 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.837+0000 7f3c18ff9640 1 -- 172.21.15.26:0/3878881561 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 80) ==== 8426+0+0 (secure 0 0 0) 0x7f3c2c026070 con 0x564fb2fc3af0 2024-09-11T23:35:41.838 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.838+0000 7f3c3a463740 10 client.27199 fetch_fsmap finished waiting for FSMap version 80 2024-09-11T23:35:41.838 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.838+0000 7f3c3a463740 10 client.27199 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:35:41.838 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.838+0000 7f3c3a463740 20 client.27199 populate_metadata read hostname 'smithi026' 2024-09-11T23:35:41.838 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.838+0000 7f3c3a463740 1 --2- 172.21.15.26:0/3878881561 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x564fb3049d70 0x564fb304c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:41.838 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.838+0000 7f3c3a463740 4 client.27199 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T23:35:41.838 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.838+0000 7f3c3a463740 1 -- 172.21.15.26:0/3878881561 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564fb2f2fd70 con 0x564fb3049d70 2024-09-11T23:35:41.839 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.838+0000 7f3c34b8c640 1 --2- 172.21.15.26:0/3878881561 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x564fb3049d70 0x564fb304c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:41.839 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.839+0000 7f3c34b8c640 1 --2- 172.21.15.26:0/3878881561 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x564fb3049d70 0x564fb304c190 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=877a80705924c6e2 server_cookie=a336f4e0cd345f00 in_seq=0 out_seq=0 2024-09-11T23:35:41.840 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:41.839+0000 7f3c18ff9640 10 client.27199 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:35:42.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:35:42 smithi136 ceph-mon[35121]: pgmap v543: 97 pgs: 15 active+clean+snaptrim, 30 active+clean+snaptrim_wait, 52 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 668 KiB/s rd, 680 KiB/s wr, 216 op/s 2024-09-11T23:35:42.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:35:42 smithi026 ceph-mon[31172]: pgmap v543: 97 pgs: 15 active+clean+snaptrim, 30 active+clean+snaptrim_wait, 52 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 668 KiB/s rd, 680 KiB/s wr, 216 op/s 2024-09-11T23:35:42.552 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:35:42 smithi076 ceph-mon[35296]: pgmap v543: 97 pgs: 15 active+clean+snaptrim, 30 active+clean+snaptrim_wait, 52 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 668 KiB/s rd, 680 KiB/s wr, 216 op/s 2024-09-11T23:35:43.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:43.031+0000 7f0c98ff9640 1 -- 172.21.15.26:0/2046917756 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5581fcddabe0 con 0x5581fcf05770 2024-09-11T23:35:43.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:43.031+0000 7f0c98ff9640 10 client.17703 handle_command_reply: tid=0 2024-09-11T23:35:43.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:43.031+0000 7f0cb9707740 1 client.17703 shutdown 2024-09-11T23:35:43.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:43.031+0000 7f0cb9707740 1 -- 172.21.15.26:0/2046917756 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5581fce6f340 msgr2=0x5581fce7d6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:43.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:43.031+0000 7f0cb9707740 1 --2- 172.21.15.26:0/2046917756 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5581fce6f340 0x5581fce7d6c0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x5581fce781a0 tx=0x7f0ca0009fb0 comp rx=0 tx=0).stop 2024-09-11T23:35:43.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:43.031+0000 7f0cb9707740 1 -- 172.21.15.26:0/2046917756 shutdown_connections 2024-09-11T23:35:43.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:43.031+0000 7f0cb9707740 1 --2- 172.21.15.26:0/2046917756 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5581fce79c00 0x5581fcf03c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:43.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:43.031+0000 7f0cb9707740 1 --2- 172.21.15.26:0/2046917756 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5581fcf05770 0x5581fcf07b60 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:43.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:43.031+0000 7f0c98ff9640 0 client.17703 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:35:43.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:43.031+0000 7f0cb9707740 1 --2- 172.21.15.26:0/2046917756 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5581fce6f340 0x5581fce7d6c0 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:43.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:43.031+0000 7f0cb9707740 1 --2- 172.21.15.26:0/2046917756 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5581fce6ea20 0x5581fce7d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:43.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:43.031+0000 7f0cb9707740 1 -- 172.21.15.26:0/2046917756 >> 172.21.15.26:0/2046917756 conn(0x5581fce6afb0 msgr2=0x5581fce753f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:35:43.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:43.032+0000 7f0cb9707740 1 -- 172.21.15.26:0/2046917756 shutdown_connections 2024-09-11T23:35:43.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:43.032+0000 7f0cb9707740 1 -- 172.21.15.26:0/2046917756 wait complete. 2024-09-11T23:35:43.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:43.032+0000 7f0cb9707740 20 client.17703 trim_cache size 0 max 16384 2024-09-11T23:35:43.034 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-11T23:35:43.034 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 80, 2024-09-11T23:35:43.034 INFO:teuthology.orchestra.run.smithi026.stdout: "leader": 24485, 2024-09-11T23:35:43.034 INFO:teuthology.orchestra.run.smithi026.stdout: "set_version": 58, 2024-09-11T23:35:43.034 INFO:teuthology.orchestra.run.smithi026.stdout: "sets": { 2024-09-11T23:35:43.034 INFO:teuthology.orchestra.run.smithi026.stdout: "89adda10": { 2024-09-11T23:35:43.035 INFO:teuthology.orchestra.run.smithi026.stdout: "version": 58, 2024-09-11T23:35:43.035 INFO:teuthology.orchestra.run.smithi026.stdout: "age_ref": 0.0, 2024-09-11T23:35:43.035 INFO:teuthology.orchestra.run.smithi026.stdout: "state": { 2024-09-11T23:35:43.035 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "QUIESCED", 2024-09-11T23:35:43.035 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.0 2024-09-11T23:35:43.035 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:35:43.036 INFO:teuthology.orchestra.run.smithi026.stdout: "timeout": 90.0, 2024-09-11T23:35:43.036 INFO:teuthology.orchestra.run.smithi026.stdout: "expiration": 144.2, 2024-09-11T23:35:43.036 INFO:teuthology.orchestra.run.smithi026.stdout: "members": { 2024-09-11T23:35:43.036 INFO:teuthology.orchestra.run.smithi026.stdout: "file:/": { 2024-09-11T23:35:43.036 INFO:teuthology.orchestra.run.smithi026.stdout: "excluded": false, 2024-09-11T23:35:43.037 INFO:teuthology.orchestra.run.smithi026.stdout: "state": { 2024-09-11T23:35:43.037 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "QUIESCED", 2024-09-11T23:35:43.037 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.0 2024-09-11T23:35:43.037 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:35:43.037 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:35:43.037 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:35:43.037 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:35:43.038 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:35:43.038 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-11T23:35:43.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:43.033+0000 7f0cb732e640 1 -- 172.21.15.26:0/2325758671 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f0c9c0a2df0 msgr2=0x7f0c9c0a52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:43.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:43.033+0000 7f0cb732e640 1 --2- 172.21.15.26:0/2325758671 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f0c9c0a2df0 0x7f0c9c0a52b0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f0ca8009b30 tx=0x7f0ca800b040 comp rx=0 tx=0).stop 2024-09-11T23:35:43.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:43.033+0000 7f0cb732e640 1 -- 172.21.15.26:0/2325758671 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0cb0100900 msgr2=0x7f0cb0195c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:43.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:43.033+0000 7f0cb732e640 1 --2- 172.21.15.26:0/2325758671 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0cb0100900 0x7f0cb0195c80 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f0cac00d980 tx=0x7f0cac00de40 comp rx=0 tx=0).stop 2024-09-11T23:35:43.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:43.033+0000 7f0cb732e640 1 -- 172.21.15.26:0/2325758671 shutdown_connections 2024-09-11T23:35:43.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:43.033+0000 7f0cb732e640 1 --2- 172.21.15.26:0/2325758671 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f0c9c0a2df0 0x7f0c9c0a52b0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:43.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:43.033+0000 7f0cb732e640 1 --2- 172.21.15.26:0/2325758671 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0cb01012a0 0x7f0cb01961c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:43.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:43.033+0000 7f0cb732e640 1 --2- 172.21.15.26:0/2325758671 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0cb0106900 0x7f0cb0196700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:43.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:43.033+0000 7f0cb732e640 1 --2- 172.21.15.26:0/2325758671 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0cb0100900 0x7f0cb0195c80 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:43.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:43.033+0000 7f0cb732e640 1 -- 172.21.15.26:0/2325758671 >> 172.21.15.26:0/2325758671 conn(0x7f0cb00fc600 msgr2=0x7f0cb00fdaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:35:43.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:43.033+0000 7f0cb732e640 1 -- 172.21.15.26:0/2325758671 shutdown_connections 2024-09-11T23:35:43.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:43.033+0000 7f0cb732e640 1 -- 172.21.15.26:0/2325758671 wait complete. 2024-09-11T23:35:43.060 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:43.059+0000 7f3c18ff9640 1 -- 172.21.15.26:0/3878881561 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564fb2f2fd70 con 0x564fb3049d70 2024-09-11T23:35:43.060 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:43.059+0000 7f3c18ff9640 10 client.27199 handle_command_reply: tid=0 2024-09-11T23:35:43.060 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:43.059+0000 7f3c3a463740 1 client.27199 shutdown 2024-09-11T23:35:43.060 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:43.059+0000 7f3c3a463740 1 -- 172.21.15.26:0/3878881561 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564fb2fc3af0 msgr2=0x564fb2fd71a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:43.060 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:43.059+0000 7f3c3a463740 1 --2- 172.21.15.26:0/3878881561 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564fb2fc3af0 0x564fb2fd71a0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f3c2c014660 tx=0x7f3c2c014690 comp rx=0 tx=0).stop 2024-09-11T23:35:43.060 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:43.059+0000 7f3c3a463740 1 -- 172.21.15.26:0/3878881561 shutdown_connections 2024-09-11T23:35:43.060 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:43.059+0000 7f3c3a463740 1 --2- 172.21.15.26:0/3878881561 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564fb2fcecb0 0x564fb2fd2a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:43.060 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:43.059+0000 7f3c3a463740 1 --2- 172.21.15.26:0/3878881561 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x564fb3049d70 0x564fb304c190 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:43.060 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:43.059+0000 7f3c3a463740 1 --2- 172.21.15.26:0/3878881561 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564fb2fc4410 0x564fb2fd24c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:43.060 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:43.059+0000 7f3c3a463740 1 --2- 172.21.15.26:0/3878881561 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564fb2fc3af0 0x564fb2fd71a0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:43.060 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:43.059+0000 7f3c3a463740 1 -- 172.21.15.26:0/3878881561 >> 172.21.15.26:0/3878881561 conn(0x564fb2fc0060 msgr2=0x564fb2fbe9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:35:43.061 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:43.059+0000 7f3c18ff9640 0 client.27199 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:35:43.061 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:43.059+0000 7f3c3a463740 1 -- 172.21.15.26:0/3878881561 shutdown_connections 2024-09-11T23:35:43.061 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:43.059+0000 7f3c3a463740 1 -- 172.21.15.26:0/3878881561 wait complete. 2024-09-11T23:35:43.061 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:43.060+0000 7f3c3a463740 20 client.27199 trim_cache size 0 max 16384 2024-09-11T23:35:43.061 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-09-11T23:35:43.061 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "message": "", 2024-09-11T23:35:43.061 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-09-11T23:35:43.061 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-09-11T23:35:43.062 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:43.060+0000 7f3c375f1640 1 -- 172.21.15.26:0/613606576 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f3c1c0a2fc0 msgr2=0x7f3c1c0a5480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:43.062 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:43.060+0000 7f3c375f1640 1 --2- 172.21.15.26:0/613606576 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f3c1c0a2fc0 0x7f3c1c0a5480 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x564fb2fc0ec0 tx=0x7f3c2002a390 comp rx=0 tx=0).stop 2024-09-11T23:35:43.062 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:43.060+0000 7f3c375f1640 1 -- 172.21.15.26:0/613606576 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3c3011b140 msgr2=0x7f3c301116d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:43.062 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:43.060+0000 7f3c375f1640 1 --2- 172.21.15.26:0/613606576 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3c3011b140 0x7f3c301116d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f3c2800efc0 tx=0x7f3c2800c800 comp rx=0 tx=0).stop 2024-09-11T23:35:43.062 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:43.061+0000 7f3c375f1640 1 -- 172.21.15.26:0/613606576 shutdown_connections 2024-09-11T23:35:43.062 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:43.061+0000 7f3c375f1640 1 --2- 172.21.15.26:0/613606576 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f3c1c0a2fc0 0x7f3c1c0a5480 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:43.062 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:43.061+0000 7f3c375f1640 1 --2- 172.21.15.26:0/613606576 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3c301095d0 0x7f3c30118ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:43.062 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:43.061+0000 7f3c375f1640 1 --2- 172.21.15.26:0/613606576 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3c3011b140 0x7f3c301116d0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:43.062 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:43.061+0000 7f3c375f1640 1 --2- 172.21.15.26:0/613606576 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3c301091f0 0x7f3c301185a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:43.063 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:43.061+0000 7f3c375f1640 1 -- 172.21.15.26:0/613606576 >> 172.21.15.26:0/613606576 conn(0x7f3c3006ce00 msgr2=0x7f3c3011b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:35:43.063 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:43.063+0000 7f3c375f1640 1 -- 172.21.15.26:0/613606576 shutdown_connections 2024-09-11T23:35:43.063 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:43.063+0000 7f3c375f1640 1 -- 172.21.15.26:0/613606576 wait complete. 2024-09-11T23:35:43.912 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '89adda10', quiesce took 11.1 seconds. Will release after: 13.15 2024-09-11T23:35:44.157 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:35:44 smithi026 ceph-mon[31172]: pgmap v544: 97 pgs: 15 active+clean+snaptrim, 30 active+clean+snaptrim_wait, 52 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 452 KiB/s rd, 351 KiB/s wr, 116 op/s 2024-09-11T23:35:44.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:35:44 smithi136 ceph-mon[35121]: pgmap v544: 97 pgs: 15 active+clean+snaptrim, 30 active+clean+snaptrim_wait, 52 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 452 KiB/s rd, 351 KiB/s wr, 116 op/s 2024-09-11T23:35:44.552 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:35:44 smithi076 ceph-mon[35296]: pgmap v544: 97 pgs: 15 active+clean+snaptrim, 30 active+clean+snaptrim_wait, 52 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 452 KiB/s rd, 351 KiB/s wr, 116 op/s 2024-09-11T23:35:46.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:35:46 smithi136 ceph-mon[35121]: pgmap v545: 97 pgs: 14 active+clean+snaptrim, 30 active+clean+snaptrim_wait, 53 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 450 KiB/s rd, 364 KiB/s wr, 106 op/s 2024-09-11T23:35:46.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:35:46 smithi026 ceph-mon[31172]: pgmap v545: 97 pgs: 14 active+clean+snaptrim, 30 active+clean+snaptrim_wait, 53 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 450 KiB/s rd, 364 KiB/s wr, 106 op/s 2024-09-11T23:35:46.552 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:35:46 smithi076 ceph-mon[35296]: pgmap v545: 97 pgs: 14 active+clean+snaptrim, 30 active+clean+snaptrim_wait, 53 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 450 KiB/s rd, 364 KiB/s wr, 106 op/s 2024-09-11T23:35:47.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:35:47 smithi136 ceph-mon[35121]: pgmap v546: 97 pgs: 11 active+clean+snaptrim, 26 active+clean+snaptrim_wait, 60 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 446 KiB/s rd, 403 KiB/s wr, 108 op/s 2024-09-11T23:35:47.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:35:47 smithi026 ceph-mon[31172]: pgmap v546: 97 pgs: 11 active+clean+snaptrim, 26 active+clean+snaptrim_wait, 60 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 446 KiB/s rd, 403 KiB/s wr, 108 op/s 2024-09-11T23:35:47.552 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:35:47 smithi076 ceph-mon[35296]: pgmap v546: 97 pgs: 11 active+clean+snaptrim, 26 active+clean+snaptrim_wait, 60 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 446 KiB/s rd, 403 KiB/s wr, 108 op/s 2024-09-11T23:35:49.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:35:49 smithi136 ceph-mon[35121]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:35:49.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:35:49 smithi136 ceph-mon[35121]: pgmap v547: 97 pgs: 11 active+clean+snaptrim, 26 active+clean+snaptrim_wait, 60 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 202 KiB/s wr, 73 op/s 2024-09-11T23:35:49.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:35:49 smithi026 ceph-mon[31172]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:35:49.501 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:35:49 smithi026 ceph-mon[31172]: pgmap v547: 97 pgs: 11 active+clean+snaptrim, 26 active+clean+snaptrim_wait, 60 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 202 KiB/s wr, 73 op/s 2024-09-11T23:35:49.552 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:35:49 smithi076 ceph-mon[35296]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:35:49.552 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:35:49 smithi076 ceph-mon[35296]: pgmap v547: 97 pgs: 11 active+clean+snaptrim, 26 active+clean+snaptrim_wait, 60 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 202 KiB/s wr, 73 op/s 2024-09-11T23:35:51.750 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:35:51 smithi026 ceph-mon[31172]: pgmap v548: 97 pgs: 9 active+clean+snaptrim, 22 active+clean+snaptrim_wait, 66 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 203 KiB/s wr, 78 op/s 2024-09-11T23:35:51.988 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:35:51 smithi136 ceph-mon[35121]: pgmap v548: 97 pgs: 9 active+clean+snaptrim, 22 active+clean+snaptrim_wait, 66 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 203 KiB/s wr, 78 op/s 2024-09-11T23:35:52.052 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:35:51 smithi076 ceph-mon[35296]: pgmap v548: 97 pgs: 9 active+clean+snaptrim, 22 active+clean+snaptrim_wait, 66 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 203 KiB/s wr, 78 op/s 2024-09-11T23:35:53.988 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:35:53 smithi136 ceph-mon[35121]: pgmap v549: 97 pgs: 7 active+clean+snaptrim, 17 active+clean+snaptrim_wait, 73 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 61 KiB/s wr, 38 op/s 2024-09-11T23:35:54.000 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:35:53 smithi026 ceph-mon[31172]: pgmap v549: 97 pgs: 7 active+clean+snaptrim, 17 active+clean+snaptrim_wait, 73 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 61 KiB/s wr, 38 op/s 2024-09-11T23:35:54.052 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:35:53 smithi076 ceph-mon[35296]: pgmap v549: 97 pgs: 7 active+clean+snaptrim, 17 active+clean+snaptrim_wait, 73 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 61 KiB/s wr, 38 op/s 2024-09-11T23:35:55.988 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:35:55 smithi136 ceph-mon[35121]: pgmap v550: 97 pgs: 3 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 88 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 61 KiB/s wr, 33 op/s 2024-09-11T23:35:56.000 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:35:55 smithi026 ceph-mon[31172]: pgmap v550: 97 pgs: 3 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 88 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 61 KiB/s wr, 33 op/s 2024-09-11T23:35:56.052 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:35:55 smithi076 ceph-mon[35296]: pgmap v550: 97 pgs: 3 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 88 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 61 KiB/s wr, 33 op/s 2024-09-11T23:35:56.176 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.174+0000 7f1313cf1640 1 -- 172.21.15.26:0/1962318766 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f130c106a00 msgr2=0x7f130c106de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:56.176 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.174+0000 7f1313cf1640 1 --2- 172.21.15.26:0/1962318766 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f130c106a00 0x7f130c106de0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f1308009930 tx=0x7f130802a420 comp rx=0 tx=0).stop 2024-09-11T23:35:56.176 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.174+0000 7f1313cf1640 1 -- 172.21.15.26:0/1962318766 shutdown_connections 2024-09-11T23:35:56.176 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.174+0000 7f1313cf1640 1 --2- 172.21.15.26:0/1962318766 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f130c101390 0x7f130c10d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:56.177 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.174+0000 7f1313cf1640 1 --2- 172.21.15.26:0/1962318766 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f130c1009f0 0x7f130c100e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:56.177 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.174+0000 7f1313cf1640 1 --2- 172.21.15.26:0/1962318766 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f130c106a00 0x7f130c106de0 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:56.177 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.174+0000 7f1313cf1640 1 -- 172.21.15.26:0/1962318766 >> 172.21.15.26:0/1962318766 conn(0x7f130c0fc6b0 msgr2=0x7f130c0fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:35:56.177 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.174+0000 7f1313cf1640 1 -- 172.21.15.26:0/1962318766 shutdown_connections 2024-09-11T23:35:56.177 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.174+0000 7f1313cf1640 1 -- 172.21.15.26:0/1962318766 wait complete. 2024-09-11T23:35:56.177 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.174+0000 7f1313cf1640 1 Processor -- start 2024-09-11T23:35:56.177 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.175+0000 7f1313cf1640 1 -- start start 2024-09-11T23:35:56.177 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.175+0000 7f1313cf1640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f130c1009f0 0x7f130c0fcc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:56.177 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.175+0000 7f1313cf1640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f130c101390 0x7f130c0fd190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:56.177 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.175+0000 7f1313cf1640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f130c106a00 0x7f130c0fd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:56.178 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.175+0000 7f1313cf1640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f130c110e50 con 0x7f130c1009f0 2024-09-11T23:35:56.178 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.175+0000 7f1313cf1640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f130c110cd0 con 0x7f130c101390 2024-09-11T23:35:56.178 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.175+0000 7f1313cf1640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f130c110fd0 con 0x7f130c106a00 2024-09-11T23:35:56.178 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.175+0000 7f1311a8d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f130c1009f0 0x7f130c0fcc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:56.178 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.175+0000 7f1311a8d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f130c1009f0 0x7f130c0fcc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46728/0 (socket says 172.21.15.26:46728) 2024-09-11T23:35:56.178 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.175+0000 7f1311a8d640 1 -- 172.21.15.26:0/308277214 learned_addr learned my addr 172.21.15.26:0/308277214 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:35:56.178 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.175+0000 7f131128c640 1 --2- 172.21.15.26:0/308277214 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f130c101390 0x7f130c0fd190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:56.178 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.175+0000 7f131228e640 1 --2- 172.21.15.26:0/308277214 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f130c106a00 0x7f130c0fd6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:56.178 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.175+0000 7f1311a8d640 1 -- 172.21.15.26:0/308277214 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f130c106a00 msgr2=0x7f130c0fd6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:56.178 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.175+0000 7f1311a8d640 1 --2- 172.21.15.26:0/308277214 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f130c106a00 0x7f130c0fd6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:56.178 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.175+0000 7f1311a8d640 1 -- 172.21.15.26:0/308277214 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f130c101390 msgr2=0x7f130c0fd190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:56.179 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.175+0000 7f1311a8d640 1 --2- 172.21.15.26:0/308277214 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f130c101390 0x7f130c0fd190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:56.179 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.175+0000 7f1311a8d640 1 -- 172.21.15.26:0/308277214 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f130c1a67d0 con 0x7f130c1009f0 2024-09-11T23:35:56.179 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.175+0000 7f131128c640 1 --2- 172.21.15.26:0/308277214 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f130c101390 0x7f130c0fd190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:35:56.179 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.176+0000 7f1311a8d640 1 --2- 172.21.15.26:0/308277214 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f130c1009f0 0x7f130c0fcc50 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f130802a940 tx=0x7f1308003c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:35:56.179 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.176+0000 7f1302ffd640 1 -- 172.21.15.26:0/308277214 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f130803f070 con 0x7f130c1009f0 2024-09-11T23:35:56.179 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.176+0000 7f1302ffd640 1 -- 172.21.15.26:0/308277214 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f130803b460 con 0x7f130c1009f0 2024-09-11T23:35:56.179 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.176+0000 7f1313cf1640 1 -- 172.21.15.26:0/308277214 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f130c1a6a60 con 0x7f130c1009f0 2024-09-11T23:35:56.179 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.176+0000 7f1302ffd640 1 -- 172.21.15.26:0/308277214 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1308035030 con 0x7f130c1009f0 2024-09-11T23:35:56.179 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.176+0000 7f1313cf1640 1 -- 172.21.15.26:0/308277214 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f130c1a6f50 con 0x7f130c1009f0 2024-09-11T23:35:56.186 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.179+0000 7f1316a90740 1 Processor -- start 2024-09-11T23:35:56.186 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.179+0000 7f1302ffd640 1 -- 172.21.15.26:0/308277214 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f1308042070 con 0x7f130c1009f0 2024-09-11T23:35:56.186 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.180+0000 7f1316a90740 1 -- start start 2024-09-11T23:35:56.186 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.180+0000 7f1316a90740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55915b351af0 0x55915b351ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:56.187 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.180+0000 7f1316a90740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55915b352410 0x55915b35c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:56.187 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.180+0000 7f1316a90740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55915b35ccb0 0x55915b35f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:56.187 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.180+0000 7f1316a90740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55915b2b62f0 con 0x55915b352410 2024-09-11T23:35:56.187 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.180+0000 7f1316a90740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55915b25e270 con 0x55915b35ccb0 2024-09-11T23:35:56.187 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.180+0000 7f1316a90740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55915b2b22b0 con 0x55915b351af0 2024-09-11T23:35:56.187 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.180+0000 7f1311a8d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55915b352410 0x55915b35c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:56.187 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.180+0000 7f1311a8d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55915b352410 0x55915b35c770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46740/0 (socket says 172.21.15.26:46740) 2024-09-11T23:35:56.187 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.180+0000 7f1311a8d640 1 -- 172.21.15.26:0/1168113664 learned_addr learned my addr 172.21.15.26:0/1168113664 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:35:56.187 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.180+0000 7f131128c640 1 --2- 172.21.15.26:0/1168113664 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55915b35ccb0 0x55915b35f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:56.187 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.180+0000 7f131228e640 1 --2- 172.21.15.26:0/1168113664 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55915b351af0 0x55915b351ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:56.187 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.180+0000 7f1311a8d640 1 -- 172.21.15.26:0/1168113664 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55915b351af0 msgr2=0x55915b351ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:35:56.187 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.180+0000 7f1311a8d640 1 --2- 172.21.15.26:0/1168113664 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55915b351af0 0x55915b351ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:56.187 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.180+0000 7f1311a8d640 1 -- 172.21.15.26:0/1168113664 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55915b35ccb0 msgr2=0x55915b35f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:35:56.188 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.180+0000 7f1311a8d640 1 --2- 172.21.15.26:0/1168113664 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55915b35ccb0 0x55915b35f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:56.188 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.180+0000 7f1302ffd640 1 --2- 172.21.15.26:0/308277214 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f12f80a2da0 0x7f12f80a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:56.188 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.180+0000 7f1311a8d640 1 -- 172.21.15.26:0/1168113664 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55915b25d340 con 0x55915b352410 2024-09-11T23:35:56.188 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.180+0000 7f131128c640 1 --2- 172.21.15.26:0/1168113664 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55915b35ccb0 0x55915b35f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:35:56.188 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.181+0000 7f131228e640 1 --2- 172.21.15.26:0/308277214 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f12f80a2da0 0x7f12f80a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:56.188 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.181+0000 7f1302ffd640 1 -- 172.21.15.26:0/308277214 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 8586+0+0 (secure 0 0 0) 0x7f13080efa40 con 0x7f130c1009f0 2024-09-11T23:35:56.188 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.181+0000 7f1311a8d640 1 --2- 172.21.15.26:0/1168113664 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55915b352410 0x55915b35c770 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f13080ed0c0 tx=0x7f13080b8280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5666ac9d04039b9b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:35:56.188 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.181+0000 7f12f7fff640 1 -- 172.21.15.26:0/1168113664 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1308049070 con 0x55915b352410 2024-09-11T23:35:56.189 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.181+0000 7f12f7fff640 1 -- 172.21.15.26:0/1168113664 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f13080bc620 con 0x55915b352410 2024-09-11T23:35:56.189 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.181+0000 7f12f7fff640 1 -- 172.21.15.26:0/1168113664 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13080bbcf0 con 0x55915b352410 2024-09-11T23:35:56.189 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.182+0000 7f1316a90740 1 -- 172.21.15.26:0/1168113664 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55915b352410 msgr2=0x55915b35c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:56.189 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.182+0000 7f131228e640 1 --2- 172.21.15.26:0/308277214 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f12f80a2da0 0x7f12f80a5260 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f1304008dc0 tx=0x7f1304008570 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:35:56.189 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.182+0000 7f1316a90740 1 --2- 172.21.15.26:0/1168113664 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55915b352410 0x55915b35c770 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f13080ed0c0 tx=0x7f13080b8280 comp rx=0 tx=0).stop 2024-09-11T23:35:56.189 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.182+0000 7f1316a90740 1 -- 172.21.15.26:0/1168113664 shutdown_connections 2024-09-11T23:35:56.189 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.182+0000 7f1316a90740 1 --2- 172.21.15.26:0/1168113664 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55915b35ccb0 0x55915b35f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:56.189 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.182+0000 7f1316a90740 1 --2- 172.21.15.26:0/1168113664 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55915b352410 0x55915b35c770 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:56.189 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.182+0000 7f1316a90740 1 --2- 172.21.15.26:0/1168113664 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55915b351af0 0x55915b351ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:56.189 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.182+0000 7f1316a90740 1 -- 172.21.15.26:0/1168113664 >> 172.21.15.26:0/1168113664 conn(0x55915b34e060 msgr2=0x55915b34e440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:35:56.189 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.182+0000 7f1316a90740 1 -- 172.21.15.26:0/1168113664 shutdown_connections 2024-09-11T23:35:56.189 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.182+0000 7f1316a90740 1 -- 172.21.15.26:0/1168113664 wait complete. 2024-09-11T23:35:56.189 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.182+0000 7f1316a90740 1 -- start start 2024-09-11T23:35:56.190 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.182+0000 7f1316a90740 1 Processor -- start 2024-09-11T23:35:56.190 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.183+0000 7f1316a90740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55915b351af0 0x55915b359c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:56.190 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.183+0000 7f1316a90740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55915b352410 0x55915b35a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:56.190 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.183+0000 7f1316a90740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55915b35ccb0 0x55915b35a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:56.190 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.183+0000 7f1316a90740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55915b35f640 con 0x55915b351af0 2024-09-11T23:35:56.190 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.183+0000 7f1316a90740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55915b23f290 con 0x55915b352410 2024-09-11T23:35:56.190 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.183+0000 7f1316a90740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55915b35f7c0 con 0x55915b35ccb0 2024-09-11T23:35:56.190 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.183+0000 7f1316a90740 10 client.0 _ll_register_callbacks cb 0x55915b270140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:35:56.190 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.183+0000 7f1311a8d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55915b351af0 0x55915b359c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:56.190 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.183+0000 7f1311a8d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55915b351af0 0x55915b359c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46752/0 (socket says 172.21.15.26:46752) 2024-09-11T23:35:56.190 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.183+0000 7f1311a8d640 1 -- 172.21.15.26:0/2438429555 learned_addr learned my addr 172.21.15.26:0/2438429555 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:35:56.190 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.183+0000 7f131128c640 1 --2- 172.21.15.26:0/2438429555 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55915b352410 0x55915b35a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:56.190 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.183+0000 7f131228e640 1 --2- 172.21.15.26:0/2438429555 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55915b35ccb0 0x55915b35a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:56.191 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.183+0000 7f1311a8d640 1 -- 172.21.15.26:0/2438429555 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55915b35ccb0 msgr2=0x55915b35a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:35:56.191 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.183+0000 7f1311a8d640 1 --2- 172.21.15.26:0/2438429555 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55915b35ccb0 0x55915b35a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:56.191 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.183+0000 7f1311a8d640 1 -- 172.21.15.26:0/2438429555 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55915b352410 msgr2=0x55915b35a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:35:56.191 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.183+0000 7f1311a8d640 1 --2- 172.21.15.26:0/2438429555 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55915b352410 0x55915b35a140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:56.191 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.183+0000 7f1311a8d640 1 -- 172.21.15.26:0/2438429555 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55915afc9210 con 0x55915b351af0 2024-09-11T23:35:56.191 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.184+0000 7f1311a8d640 1 --2- 172.21.15.26:0/2438429555 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55915b351af0 0x55915b359c00 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f13080b8380 tx=0x7f130803bd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c87161f775eb5623 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:35:56.191 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.184+0000 7f12f57fa640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:35:56.191 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.184+0000 7f12f57fa640 1 -- 172.21.15.26:0/2438429555 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1308049040 con 0x55915b351af0 2024-09-11T23:35:56.191 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.184+0000 7f12f57fa640 1 -- 172.21.15.26:0/2438429555 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 8586+0+0 (secure 0 0 0) 0x7f130803b610 con 0x55915b351af0 2024-09-11T23:35:56.191 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.184+0000 7f1316a90740 1 -- 172.21.15.26:0/2438429555 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f13080bc620 con 0x55915b351af0 2024-09-11T23:35:56.191 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.184+0000 7f12f57fa640 1 -- 172.21.15.26:0/2438429555 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7f13080d7050 con 0x55915b351af0 2024-09-11T23:35:56.191 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.184+0000 7f1316a90740 10 client.17730 fetch_fsmap learned FSMap version 81 2024-09-11T23:35:56.191 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.184+0000 7f1316a90740 1 -- 172.21.15.26:0/2438429555 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=81,monmap=4+}) -- 0x55915b095620 con 0x55915b351af0 2024-09-11T23:35:56.191 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.184+0000 7f12f57fa640 1 -- 172.21.15.26:0/2438429555 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f13080bc620 con 0x55915b351af0 2024-09-11T23:35:56.192 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.184+0000 7f12f57fa640 1 -- 172.21.15.26:0/2438429555 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 81) ==== 8418+0+0 (secure 0 0 0) 0x7f13080bf070 con 0x55915b351af0 2024-09-11T23:35:56.192 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.184+0000 7f1316a90740 10 client.17730 fetch_fsmap finished waiting for FSMap version 81 2024-09-11T23:35:56.192 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.184+0000 7f1316a90740 10 client.17730 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:35:56.192 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.184+0000 7f1316a90740 20 client.17730 populate_metadata read hostname 'smithi026' 2024-09-11T23:35:56.192 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.184+0000 7f1316a90740 1 --2- 172.21.15.26:0/2438429555 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55915b36c7f0 0x55915b36ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:56.192 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.184+0000 7f1316a90740 4 client.17730 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:35:56.192 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.184+0000 7f1316a90740 1 -- 172.21.15.26:0/2438429555 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55915b2c3a20 con 0x55915b36c7f0 2024-09-11T23:35:56.192 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.185+0000 7f131128c640 1 --2- 172.21.15.26:0/2438429555 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55915b36c7f0 0x55915b36ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:56.192 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.185+0000 7f12f57fa640 10 client.17730 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:35:56.192 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.185+0000 7f131128c640 1 --2- 172.21.15.26:0/2438429555 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55915b36c7f0 0x55915b36ec10 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ff66ee6d7fdacce server_cookie=c0ca084f1caa2a14 in_seq=0 out_seq=0 2024-09-11T23:35:56.192 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.186+0000 7f12f57fa640 1 -- 172.21.15.26:0/2438429555 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55915b2c3a20 con 0x55915b36c7f0 2024-09-11T23:35:56.192 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.186+0000 7f12f57fa640 10 client.17730 handle_command_reply: tid=0 2024-09-11T23:35:56.192 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.186+0000 7f1316a90740 1 client.17730 shutdown 2024-09-11T23:35:56.193 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.186+0000 7f1316a90740 1 -- 172.21.15.26:0/2438429555 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55915b351af0 msgr2=0x55915b359c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:56.193 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.186+0000 7f1316a90740 1 --2- 172.21.15.26:0/2438429555 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55915b351af0 0x55915b359c00 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f13080b8380 tx=0x7f130803bd40 comp rx=0 tx=0).stop 2024-09-11T23:35:56.193 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.187+0000 7f1316a90740 1 -- 172.21.15.26:0/2438429555 shutdown_connections 2024-09-11T23:35:56.193 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.187+0000 7f1316a90740 1 --2- 172.21.15.26:0/2438429555 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55915b352410 0x55915b35a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:56.193 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.187+0000 7f1316a90740 1 --2- 172.21.15.26:0/2438429555 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55915b36c7f0 0x55915b36ec10 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:56.193 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.187+0000 7f1316a90740 1 --2- 172.21.15.26:0/2438429555 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55915b35ccb0 0x55915b35a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:56.193 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.187+0000 7f1316a90740 1 --2- 172.21.15.26:0/2438429555 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55915b351af0 0x55915b359c00 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:56.193 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.187+0000 7f12f57fa640 0 client.17730 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:35:56.193 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.187+0000 7f1316a90740 1 -- 172.21.15.26:0/2438429555 >> 172.21.15.26:0/2438429555 conn(0x55915b34e060 msgr2=0x55915b353d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:35:56.193 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.187+0000 7f1316a90740 1 -- 172.21.15.26:0/2438429555 shutdown_connections 2024-09-11T23:35:56.193 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.187+0000 7f1316a90740 1 -- 172.21.15.26:0/2438429555 wait complete. 2024-09-11T23:35:56.193 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.187+0000 7f1316a90740 20 client.17730 trim_cache size 0 max 16384 2024-09-11T23:35:56.205 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.201+0000 7f1316a90740 1 Processor -- start 2024-09-11T23:35:56.206 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.202+0000 7f1316a90740 1 -- start start 2024-09-11T23:35:56.206 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.202+0000 7f1316a90740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55915b351af0 0x55915b35a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:56.206 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.202+0000 7f1316a90740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55915b352410 0x55915b3744e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:56.206 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.202+0000 7f1316a90740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55915b35ccb0 0x55915b374a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:56.206 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.202+0000 7f1316a90740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f130c1afa10 con 0x55915b35ccb0 2024-09-11T23:35:56.206 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.202+0000 7f1316a90740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f130c1b17f0 con 0x55915b352410 2024-09-11T23:35:56.206 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.202+0000 7f1316a90740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f130c1b01a0 con 0x55915b351af0 2024-09-11T23:35:56.206 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.202+0000 7f131228e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55915b35ccb0 0x55915b374a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:56.206 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.202+0000 7f131228e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55915b35ccb0 0x55915b374a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46754/0 (socket says 172.21.15.26:46754) 2024-09-11T23:35:56.206 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.202+0000 7f131228e640 1 -- 172.21.15.26:0/674015168 learned_addr learned my addr 172.21.15.26:0/674015168 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:35:56.206 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.202+0000 7f1311a8d640 1 --2- 172.21.15.26:0/674015168 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55915b351af0 0x55915b35a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:56.206 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.202+0000 7f131128c640 1 --2- 172.21.15.26:0/674015168 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55915b352410 0x55915b3744e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:56.207 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.202+0000 7f131228e640 1 -- 172.21.15.26:0/674015168 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55915b351af0 msgr2=0x55915b35a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:35:56.207 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.202+0000 7f131228e640 1 --2- 172.21.15.26:0/674015168 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55915b351af0 0x55915b35a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:56.207 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.202+0000 7f131228e640 1 -- 172.21.15.26:0/674015168 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55915b352410 msgr2=0x55915b3744e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:35:56.207 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.202+0000 7f131228e640 1 --2- 172.21.15.26:0/674015168 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55915b352410 0x55915b3744e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:56.207 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.202+0000 7f131228e640 1 -- 172.21.15.26:0/674015168 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55915b351870 con 0x55915b35ccb0 2024-09-11T23:35:56.207 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.202+0000 7f131128c640 1 --2- 172.21.15.26:0/674015168 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55915b352410 0x55915b3744e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:35:56.207 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.202+0000 7f131228e640 1 --2- 172.21.15.26:0/674015168 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55915b35ccb0 0x55915b374a20 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f1304008ba0 tx=0x7f1304014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e8dc1521ee8e368 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:35:56.207 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.203+0000 7f12f77fe640 1 -- 172.21.15.26:0/674015168 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f130402a070 con 0x55915b35ccb0 2024-09-11T23:35:56.207 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.203+0000 7f12f77fe640 1 -- 172.21.15.26:0/674015168 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f130401bd10 con 0x55915b35ccb0 2024-09-11T23:35:56.207 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.203+0000 7f12f77fe640 1 -- 172.21.15.26:0/674015168 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f130401de60 con 0x55915b35ccb0 2024-09-11T23:35:56.207 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.203+0000 7f1316a90740 1 -- 172.21.15.26:0/674015168 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55915b35ccb0 msgr2=0x55915b374a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:56.207 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.203+0000 7f1316a90740 1 --2- 172.21.15.26:0/674015168 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55915b35ccb0 0x55915b374a20 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f1304008ba0 tx=0x7f1304014000 comp rx=0 tx=0).stop 2024-09-11T23:35:56.208 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.203+0000 7f1316a90740 1 -- 172.21.15.26:0/674015168 shutdown_connections 2024-09-11T23:35:56.208 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.203+0000 7f1316a90740 1 --2- 172.21.15.26:0/674015168 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55915b352410 0x55915b3744e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:56.208 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.203+0000 7f1316a90740 1 --2- 172.21.15.26:0/674015168 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55915b35ccb0 0x55915b374a20 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:56.208 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.203+0000 7f1316a90740 1 --2- 172.21.15.26:0/674015168 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55915b351af0 0x55915b35a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:56.208 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.203+0000 7f1316a90740 1 -- 172.21.15.26:0/674015168 >> 172.21.15.26:0/674015168 conn(0x55915b34e060 msgr2=0x55915b35bc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:35:56.208 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.203+0000 7f1316a90740 1 -- 172.21.15.26:0/674015168 shutdown_connections 2024-09-11T23:35:56.208 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.203+0000 7f1316a90740 1 -- 172.21.15.26:0/674015168 wait complete. 2024-09-11T23:35:56.208 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.203+0000 7f1316a90740 1 -- start start 2024-09-11T23:35:56.208 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.204+0000 7f1316a90740 1 Processor -- start 2024-09-11T23:35:56.208 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.204+0000 7f1316a90740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55915b351af0 0x55915b364960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:56.208 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.204+0000 7f1316a90740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55915b352410 0x55915b364ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:56.208 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.204+0000 7f1316a90740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55915b35ccb0 0x55915b361930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:56.209 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.204+0000 7f1316a90740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f130c1b1060 con 0x55915b352410 2024-09-11T23:35:56.209 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.204+0000 7f1316a90740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f130c1b08d0 con 0x55915b351af0 2024-09-11T23:35:56.209 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.204+0000 7f1316a90740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f130c1ada10 con 0x55915b35ccb0 2024-09-11T23:35:56.209 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.204+0000 7f1316a90740 10 client.0 _ll_register_callbacks cb 0x7f12fc008390 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:35:56.209 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.204+0000 7f131128c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55915b352410 0x55915b364ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:56.209 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.204+0000 7f131128c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55915b352410 0x55915b364ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46758/0 (socket says 172.21.15.26:46758) 2024-09-11T23:35:56.209 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.204+0000 7f131128c640 1 -- 172.21.15.26:0/4217066575 learned_addr learned my addr 172.21.15.26:0/4217066575 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:35:56.209 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.204+0000 7f131228e640 1 --2- 172.21.15.26:0/4217066575 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55915b35ccb0 0x55915b361930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:56.209 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.204+0000 7f1311a8d640 1 --2- 172.21.15.26:0/4217066575 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55915b351af0 0x55915b364960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:56.209 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.204+0000 7f131128c640 1 -- 172.21.15.26:0/4217066575 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55915b35ccb0 msgr2=0x55915b361930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:35:56.209 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.204+0000 7f131128c640 1 --2- 172.21.15.26:0/4217066575 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55915b35ccb0 0x55915b361930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:56.210 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.204+0000 7f131128c640 1 -- 172.21.15.26:0/4217066575 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55915b351af0 msgr2=0x55915b364960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:35:56.210 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.204+0000 7f131128c640 1 --2- 172.21.15.26:0/4217066575 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55915b351af0 0x55915b364960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:56.210 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.204+0000 7f131128c640 1 -- 172.21.15.26:0/4217066575 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55915b357ac0 con 0x55915b352410 2024-09-11T23:35:56.210 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.204+0000 7f1311a8d640 1 --2- 172.21.15.26:0/4217066575 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55915b351af0 0x55915b364960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:35:56.210 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.205+0000 7f131128c640 1 --2- 172.21.15.26:0/4217066575 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55915b352410 0x55915b364ea0 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x55915b354690 tx=0x7f12fc010af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dfa85105c834f44 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:35:56.210 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.205+0000 7f12f57fa640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:35:56.210 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.205+0000 7f12f57fa640 1 -- 172.21.15.26:0/4217066575 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12fc01f030 con 0x55915b352410 2024-09-11T23:35:56.210 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.205+0000 7f1316a90740 1 -- 172.21.15.26:0/4217066575 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f130401bd10 con 0x55915b352410 2024-09-11T23:35:56.210 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.205+0000 7f12f57fa640 1 -- 172.21.15.26:0/4217066575 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 8586+0+0 (secure 0 0 0) 0x7f12fc005020 con 0x55915b352410 2024-09-11T23:35:56.210 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.205+0000 7f12f57fa640 1 -- 172.21.15.26:0/4217066575 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7f12fc029050 con 0x55915b352410 2024-09-11T23:35:56.210 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.205+0000 7f1316a90740 10 client.17736 fetch_fsmap learned FSMap version 81 2024-09-11T23:35:56.210 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.205+0000 7f1316a90740 1 -- 172.21.15.26:0/4217066575 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=81,monmap=4+}) -- 0x55915b3d7950 con 0x55915b352410 2024-09-11T23:35:56.210 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.205+0000 7f12f57fa640 1 -- 172.21.15.26:0/4217066575 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f130401bd10 con 0x55915b352410 2024-09-11T23:35:56.210 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.205+0000 7f12f57fa640 1 -- 172.21.15.26:0/4217066575 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 81) ==== 8418+0+0 (secure 0 0 0) 0x7f12fc0048a0 con 0x55915b352410 2024-09-11T23:35:56.211 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.205+0000 7f1316a90740 10 client.17736 fetch_fsmap finished waiting for FSMap version 81 2024-09-11T23:35:56.211 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.205+0000 7f1316a90740 10 client.17736 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:35:56.211 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.205+0000 7f1316a90740 20 client.17736 populate_metadata read hostname 'smithi026' 2024-09-11T23:35:56.211 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.205+0000 7f1316a90740 1 --2- 172.21.15.26:0/4217066575 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55915b3d8210 0x55915b3da630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:56.211 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.205+0000 7f1316a90740 4 client.17736 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T23:35:56.211 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.205+0000 7f1316a90740 1 -- 172.21.15.26:0/4217066575 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55915b2bdd70 con 0x55915b3d8210 2024-09-11T23:35:56.211 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.206+0000 7f1311a8d640 1 --2- 172.21.15.26:0/4217066575 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55915b3d8210 0x55915b3da630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:56.211 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.206+0000 7f1311a8d640 1 --2- 172.21.15.26:0/4217066575 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55915b3d8210 0x55915b3da630 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7bd41b9ad5721311 server_cookie=6e91aa9f006c689f in_seq=0 out_seq=0 2024-09-11T23:35:56.211 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.206+0000 7f12f57fa640 10 client.17736 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:35:56.220 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.220+0000 7f12f57fa640 1 -- 172.21.15.26:0/4217066575 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55915b2bdd70 con 0x55915b3d8210 2024-09-11T23:35:56.220 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.220+0000 7f12f57fa640 10 client.17736 handle_command_reply: tid=0 2024-09-11T23:35:56.221 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.220+0000 7f1316a90740 1 client.17736 shutdown 2024-09-11T23:35:56.221 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.220+0000 7f1316a90740 1 -- 172.21.15.26:0/4217066575 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55915b352410 msgr2=0x55915b364ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:56.221 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.220+0000 7f1316a90740 1 --2- 172.21.15.26:0/4217066575 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55915b352410 0x55915b364ea0 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x55915b354690 tx=0x7f12fc010af0 comp rx=0 tx=0).stop 2024-09-11T23:35:56.221 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.220+0000 7f1316a90740 1 -- 172.21.15.26:0/4217066575 shutdown_connections 2024-09-11T23:35:56.221 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.220+0000 7f1316a90740 1 --2- 172.21.15.26:0/4217066575 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55915b3d8210 0x55915b3da630 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:56.221 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.220+0000 7f1316a90740 1 --2- 172.21.15.26:0/4217066575 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55915b35ccb0 0x55915b361930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:56.221 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.220+0000 7f1316a90740 1 --2- 172.21.15.26:0/4217066575 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55915b352410 0x55915b364ea0 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:56.221 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.220+0000 7f1316a90740 1 --2- 172.21.15.26:0/4217066575 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55915b351af0 0x55915b364960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:56.221 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.220+0000 7f1316a90740 1 -- 172.21.15.26:0/4217066575 >> 172.21.15.26:0/4217066575 conn(0x55915b34e060 msgr2=0x55915b3534c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:35:56.221 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.220+0000 7f12f57fa640 0 client.17736 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:35:56.221 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.220+0000 7f1316a90740 1 -- 172.21.15.26:0/4217066575 shutdown_connections 2024-09-11T23:35:56.221 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.220+0000 7f1316a90740 1 -- 172.21.15.26:0/4217066575 wait complete. 2024-09-11T23:35:56.221 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.220+0000 7f1316a90740 20 client.17736 trim_cache size 0 max 16384 2024-09-11T23:35:56.222 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-09-11T23:35:56.222 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "message": "", 2024-09-11T23:35:56.222 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-09-11T23:35:56.222 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-09-11T23:35:56.223 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.221+0000 7f1313cf1640 1 -- 172.21.15.26:0/308277214 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f12f80a2da0 msgr2=0x7f12f80a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:56.223 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.221+0000 7f1313cf1640 1 --2- 172.21.15.26:0/308277214 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f12f80a2da0 0x7f12f80a5260 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f1304008dc0 tx=0x7f1304008570 comp rx=0 tx=0).stop 2024-09-11T23:35:56.223 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.221+0000 7f1313cf1640 1 -- 172.21.15.26:0/308277214 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f130c1009f0 msgr2=0x7f130c0fcc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:56.223 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.221+0000 7f1313cf1640 1 --2- 172.21.15.26:0/308277214 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f130c1009f0 0x7f130c0fcc50 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f130802a940 tx=0x7f1308003c00 comp rx=0 tx=0).stop 2024-09-11T23:35:56.223 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.221+0000 7f1313cf1640 1 -- 172.21.15.26:0/308277214 shutdown_connections 2024-09-11T23:35:56.223 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.221+0000 7f1313cf1640 1 --2- 172.21.15.26:0/308277214 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f12f80a2da0 0x7f12f80a5260 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:56.223 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.221+0000 7f1313cf1640 1 --2- 172.21.15.26:0/308277214 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f130c101390 0x7f130c0fd190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:56.223 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.221+0000 7f1313cf1640 1 --2- 172.21.15.26:0/308277214 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f130c106a00 0x7f130c0fd6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:56.223 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.221+0000 7f1313cf1640 1 --2- 172.21.15.26:0/308277214 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f130c1009f0 0x7f130c0fcc50 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:56.223 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.221+0000 7f1313cf1640 1 -- 172.21.15.26:0/308277214 >> 172.21.15.26:0/308277214 conn(0x7f130c0fc6b0 msgr2=0x7f130c06a460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:35:56.223 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.221+0000 7f1313cf1640 1 -- 172.21.15.26:0/308277214 shutdown_connections 2024-09-11T23:35:56.223 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:35:56.221+0000 7f1313cf1640 1 -- 172.21.15.26:0/308277214 wait complete. 2024-09-11T23:35:57.063 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:35:57.336 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:35:57.855 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:35:57 smithi076 ceph-mon[35296]: pgmap v551: 97 pgs: 97 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 43 KiB/s wr, 26 op/s 2024-09-11T23:35:58.238 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:35:57 smithi136 ceph-mon[35121]: pgmap v551: 97 pgs: 97 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 43 KiB/s wr, 26 op/s 2024-09-11T23:35:58.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:35:57 smithi026 ceph-mon[31172]: pgmap v551: 97 pgs: 97 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 43 KiB/s wr, 26 op/s 2024-09-11T23:35:58.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:58.710+0000 7fd743910640 1 -- 172.21.15.26:0/1378673237 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd73c1055b0 msgr2=0x7fd73c10d130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:58.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:58.710+0000 7fd743910640 1 --2- 172.21.15.26:0/1378673237 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd73c1055b0 0x7fd73c10d130 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7fd734009ca0 tx=0x7fd73402a8c0 comp rx=0 tx=0).stop 2024-09-11T23:35:58.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:58.710+0000 7fd743910640 1 -- 172.21.15.26:0/1378673237 shutdown_connections 2024-09-11T23:35:58.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:58.710+0000 7fd743910640 1 --2- 172.21.15.26:0/1378673237 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd73c1055b0 0x7fd73c10d130 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:58.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:58.710+0000 7fd743910640 1 --2- 172.21.15.26:0/1378673237 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd73c100e30 0x7fd73c104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:58.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:58.710+0000 7fd743910640 1 --2- 172.21.15.26:0/1378673237 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd73c100480 0x7fd73c100860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:58.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:58.710+0000 7fd743910640 1 -- 172.21.15.26:0/1378673237 >> 172.21.15.26:0/1378673237 conn(0x7fd73c077210 msgr2=0x7fd73c0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:35:58.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:58.711+0000 7fd743910640 1 -- 172.21.15.26:0/1378673237 shutdown_connections 2024-09-11T23:35:58.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:58.711+0000 7fd743910640 1 -- 172.21.15.26:0/1378673237 wait complete. 2024-09-11T23:35:58.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:58.711+0000 7fd743910640 1 Processor -- start 2024-09-11T23:35:58.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:58.711+0000 7fd743910640 1 -- start start 2024-09-11T23:35:58.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:58.712+0000 7fd743910640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd73c100480 0x7fd73c198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:58.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:58.712+0000 7fd743910640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd73c100e30 0x7fd73c198860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:58.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:58.712+0000 7fd743910640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd73c1055b0 0x7fd73c19cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:58.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:58.712+0000 7fd743910640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd73c110e90 con 0x7fd73c100e30 2024-09-11T23:35:58.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:58.712+0000 7fd743910640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fd73c110d10 con 0x7fd73c100480 2024-09-11T23:35:58.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:58.712+0000 7fd743910640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd73c111010 con 0x7fd73c1055b0 2024-09-11T23:35:58.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:58.712+0000 7fd740eab640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd73c100e30 0x7fd73c198860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:58.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:58.712+0000 7fd740eab640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd73c100e30 0x7fd73c198860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46778/0 (socket says 172.21.15.26:46778) 2024-09-11T23:35:58.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:58.712+0000 7fd740eab640 1 -- 172.21.15.26:0/4102001359 learned_addr learned my addr 172.21.15.26:0/4102001359 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:35:58.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:58.712+0000 7fd7416ac640 1 --2- 172.21.15.26:0/4102001359 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd73c100480 0x7fd73c198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:58.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:58.712+0000 7fd741ead640 1 --2- 172.21.15.26:0/4102001359 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd73c1055b0 0x7fd73c19cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:58.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:58.712+0000 7fd740eab640 1 -- 172.21.15.26:0/4102001359 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd73c1055b0 msgr2=0x7fd73c19cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:58.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:58.712+0000 7fd740eab640 1 --2- 172.21.15.26:0/4102001359 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd73c1055b0 0x7fd73c19cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:58.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:58.712+0000 7fd740eab640 1 -- 172.21.15.26:0/4102001359 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd73c100480 msgr2=0x7fd73c198320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:58.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:58.712+0000 7fd740eab640 1 --2- 172.21.15.26:0/4102001359 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd73c100480 0x7fd73c198320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:58.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:58.712+0000 7fd740eab640 1 -- 172.21.15.26:0/4102001359 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd73c19d2d0 con 0x7fd73c100e30 2024-09-11T23:35:58.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:58.712+0000 7fd741ead640 1 --2- 172.21.15.26:0/4102001359 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd73c1055b0 0x7fd73c19cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:35:58.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:58.713+0000 7fd740eab640 1 --2- 172.21.15.26:0/4102001359 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd73c100e30 0x7fd73c198860 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7fd72c00c920 tx=0x7fd72c00cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:35:58.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:58.713+0000 7fd7327fc640 1 -- 172.21.15.26:0/4102001359 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd72c007bb0 con 0x7fd73c100e30 2024-09-11T23:35:58.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:58.713+0000 7fd7327fc640 1 -- 172.21.15.26:0/4102001359 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd72c007d50 con 0x7fd73c100e30 2024-09-11T23:35:58.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:58.713+0000 7fd743910640 1 -- 172.21.15.26:0/4102001359 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd73c19d5c0 con 0x7fd73c100e30 2024-09-11T23:35:58.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:58.713+0000 7fd7327fc640 1 -- 172.21.15.26:0/4102001359 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd72c018780 con 0x7fd73c100e30 2024-09-11T23:35:58.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:58.713+0000 7fd743910640 1 -- 172.21.15.26:0/4102001359 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd73c1a4ea0 con 0x7fd73c100e30 2024-09-11T23:35:58.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:58.714+0000 7fd743910640 1 -- 172.21.15.26:0/4102001359 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd73c100860 con 0x7fd73c100e30 2024-09-11T23:35:58.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:58.715+0000 7fd7327fc640 1 -- 172.21.15.26:0/4102001359 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fd72c00f450 con 0x7fd73c100e30 2024-09-11T23:35:58.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:58.716+0000 7fd7327fc640 1 --2- 172.21.15.26:0/4102001359 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fd7280a2cf0 0x7fd7280a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:58.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:58.716+0000 7fd7416ac640 1 --2- 172.21.15.26:0/4102001359 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fd7280a2cf0 0x7fd7280a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:58.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:58.716+0000 7fd7327fc640 1 -- 172.21.15.26:0/4102001359 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 8586+0+0 (secure 0 0 0) 0x7fd72c0cf540 con 0x7fd73c100e30 2024-09-11T23:35:58.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:58.716+0000 7fd7416ac640 1 --2- 172.21.15.26:0/4102001359 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fd7280a2cf0 0x7fd7280a51b0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fd73800abe0 tx=0x7fd73800a2d0 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:35:58.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:58.720+0000 7fd7327fc640 1 -- 172.21.15.26:0/4102001359 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd72c014030 con 0x7fd73c100e30 2024-09-11T23:35:58.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:58.931+0000 7fd743910640 1 -- 172.21.15.26:0/4102001359 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd73c10a7d0 con 0x7fd73c100e30 2024-09-11T23:35:58.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:58.932+0000 7fd7327fc640 1 -- 172.21.15.26:0/4102001359 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 81 v81) ==== 94+0+11996 (secure 0 0 0) 0x7fd72c09c120 con 0x7fd73c100e30 2024-09-11T23:35:58.932 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-11T23:35:58.932 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":81,"btime":"2024-09-11T23:35:48:177981+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14499,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/367659515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":367659515},{"type":"v1","addr":"172.21.15.26:6835","nonce":367659515}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14511,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/2041065299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":2041065299},{"type":"v1","addr":"172.21.15.26:6837","nonce":2041065299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":81,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:16:26.544203+0000","modified":"2024-09-11T23:35:47.915600+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24485,"mds_1":24463,"mds_2":24458,"mds_3":24448,"mds_4":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14550":{"gid":14550,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6839/2233068305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":2233068305},{"type":"v1","addr":"172.21.15.26:6839","nonce":2233068305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14562":{"gid":14562,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6841/2984982336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":2984982336},{"type":"v1","addr":"172.21.15.26:6841","nonce":2984982336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6833/2019841455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":2019841455},{"type":"v1","addr":"172.21.15.136:6833","nonce":2019841455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.76:6833/3438147050","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3438147050},{"type":"v1","addr":"172.21.15.76:6833","nonce":3438147050}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6835/3617665975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3617665975},{"type":"v1","addr":"172.21.15.136:6835","nonce":3617665975}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":12,"addr":"172.21.15.76:6835/393308499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":393308499},{"type":"v1","addr":"172.21.15.76:6835","nonce":393308499}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":10,"addr":"172.21.15.136:6837/1633082387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":1633082387},{"type":"v1","addr":"172.21.15.136:6837","nonce":1633082387}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.76:6837/2078978748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":2078978748},{"type":"v1","addr":"172.21.15.76:6837","nonce":2078978748}]},"join_fscid":-1,"export_targets":[0,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":63,"addr":"172.21.15.136:6839/3211398630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":3211398630},{"type":"v1","addr":"172.21.15.136:6839","nonce":3211398630}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.76:6839/2077637090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6838","nonce":2077637090},{"type":"v1","addr":"172.21.15.76:6839","nonce":2077637090}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24485,"qdb_cluster":[24443,24448,24458,24463,24485]},"id":1}]} 2024-09-11T23:35:58.932 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 81 2024-09-11T23:35:58.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:58.934+0000 7fd743910640 1 -- 172.21.15.26:0/4102001359 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fd7280a2cf0 msgr2=0x7fd7280a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:58.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:58.934+0000 7fd743910640 1 --2- 172.21.15.26:0/4102001359 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fd7280a2cf0 0x7fd7280a51b0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fd73800abe0 tx=0x7fd73800a2d0 comp rx=0 tx=0).stop 2024-09-11T23:35:58.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:58.934+0000 7fd743910640 1 -- 172.21.15.26:0/4102001359 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd73c100e30 msgr2=0x7fd73c198860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:58.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:58.934+0000 7fd743910640 1 --2- 172.21.15.26:0/4102001359 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd73c100e30 0x7fd73c198860 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7fd72c00c920 tx=0x7fd72c00cd20 comp rx=0 tx=0).stop 2024-09-11T23:35:58.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:58.934+0000 7fd743910640 1 -- 172.21.15.26:0/4102001359 shutdown_connections 2024-09-11T23:35:58.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:58.934+0000 7fd743910640 1 --2- 172.21.15.26:0/4102001359 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fd7280a2cf0 0x7fd7280a51b0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:58.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:58.934+0000 7fd743910640 1 --2- 172.21.15.26:0/4102001359 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd73c1055b0 0x7fd73c19cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:58.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:58.934+0000 7fd743910640 1 --2- 172.21.15.26:0/4102001359 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd73c100e30 0x7fd73c198860 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:58.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:58.934+0000 7fd743910640 1 --2- 172.21.15.26:0/4102001359 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd73c100480 0x7fd73c198320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:58.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:58.934+0000 7fd743910640 1 -- 172.21.15.26:0/4102001359 >> 172.21.15.26:0/4102001359 conn(0x7fd73c077210 msgr2=0x7fd73c1039a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:35:58.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:58.934+0000 7fd743910640 1 -- 172.21.15.26:0/4102001359 shutdown_connections 2024-09-11T23:35:58.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:35:58.934+0000 7fd743910640 1 -- 172.21.15.26:0/4102001359 wait complete. 2024-09-11T23:35:59.600 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24485 quiesce db --set-id 89adda10 --release --await' 2024-09-11T23:35:59.600 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24485 quiesce db --set-id 89adda10 --release --await 2024-09-11T23:35:59.866 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:36:00.146 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:35:59 smithi026 ceph-mon[31172]: pgmap v552: 97 pgs: 97 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 4.4 KiB/s wr, 9 op/s 2024-09-11T23:36:00.146 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:35:59 smithi026 ceph-mon[31172]: from='client.? 172.21.15.26:0/4102001359' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:36:00.238 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:35:59 smithi136 ceph-mon[35121]: pgmap v552: 97 pgs: 97 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 4.4 KiB/s wr, 9 op/s 2024-09-11T23:36:00.238 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:35:59 smithi136 ceph-mon[35121]: from='client.? 172.21.15.26:0/4102001359' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:36:00.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:35:59 smithi076 ceph-mon[35296]: pgmap v552: 97 pgs: 97 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 4.4 KiB/s wr, 9 op/s 2024-09-11T23:36:00.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:35:59 smithi076 ceph-mon[35296]: from='client.? 172.21.15.26:0/4102001359' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:36:00.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.377+0000 7fa14249e640 1 -- 172.21.15.26:0/475406616 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa13c101440 msgr2=0x7fa13c10df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:00.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.377+0000 7fa14249e640 1 --2- 172.21.15.26:0/475406616 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa13c101440 0x7fa13c10df30 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fa130009900 tx=0x7fa13002a410 comp rx=0 tx=0).stop 2024-09-11T23:36:00.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.377+0000 7fa14249e640 1 -- 172.21.15.26:0/475406616 shutdown_connections 2024-09-11T23:36:00.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.377+0000 7fa14249e640 1 --2- 172.21.15.26:0/475406616 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa13c10e500 0x7fa13c110910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:00.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.377+0000 7fa14249e640 1 --2- 172.21.15.26:0/475406616 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa13c101440 0x7fa13c10df30 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:00.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.377+0000 7fa14249e640 1 --2- 172.21.15.26:0/475406616 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa13c100b20 0x7fa13c100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:00.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.377+0000 7fa14249e640 1 -- 172.21.15.26:0/475406616 >> 172.21.15.26:0/475406616 conn(0x7fa13c0fc7d0 msgr2=0x7fa13c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:00.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.377+0000 7fa14249e640 1 -- 172.21.15.26:0/475406616 shutdown_connections 2024-09-11T23:36:00.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.377+0000 7fa14249e640 1 -- 172.21.15.26:0/475406616 wait complete. 2024-09-11T23:36:00.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.377+0000 7fa14249e640 1 Processor -- start 2024-09-11T23:36:00.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.378+0000 7fa14249e640 1 -- start start 2024-09-11T23:36:00.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.378+0000 7fa14249e640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa13c100b20 0x7fa13c1a0460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:00.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.378+0000 7fa14249e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa13c101440 0x7fa13c1a09a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:00.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.378+0000 7fa14249e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa13c10e500 0x7fa13c19a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:00.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.378+0000 7fa14249e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fa13c114250 con 0x7fa13c10e500 2024-09-11T23:36:00.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.378+0000 7fa14249e640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fa13c1140d0 con 0x7fa13c101440 2024-09-11T23:36:00.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.378+0000 7fa14249e640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fa13c1143d0 con 0x7fa13c100b20 2024-09-11T23:36:00.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.378+0000 7fa140a3b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa13c10e500 0x7fa13c19a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:00.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.378+0000 7fa13b7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa13c101440 0x7fa13c1a09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:00.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.378+0000 7fa13bfff640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa13c100b20 0x7fa13c1a0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:00.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.378+0000 7fa140a3b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa13c10e500 0x7fa13c19a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46806/0 (socket says 172.21.15.26:46806) 2024-09-11T23:36:00.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.378+0000 7fa140a3b640 1 -- 172.21.15.26:0/3078334605 learned_addr learned my addr 172.21.15.26:0/3078334605 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:36:00.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.378+0000 7fa140a3b640 1 -- 172.21.15.26:0/3078334605 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa13c100b20 msgr2=0x7fa13c1a0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:00.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.378+0000 7fa140a3b640 1 --2- 172.21.15.26:0/3078334605 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa13c100b20 0x7fa13c1a0460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:00.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.378+0000 7fa140a3b640 1 -- 172.21.15.26:0/3078334605 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa13c101440 msgr2=0x7fa13c1a09a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:00.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.378+0000 7fa140a3b640 1 --2- 172.21.15.26:0/3078334605 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa13c101440 0x7fa13c1a09a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:00.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.378+0000 7fa140a3b640 1 -- 172.21.15.26:0/3078334605 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa13c19ad50 con 0x7fa13c10e500 2024-09-11T23:36:00.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.379+0000 7fa13b7fe640 1 --2- 172.21.15.26:0/3078334605 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa13c101440 0x7fa13c1a09a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:36:00.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.379+0000 7fa140a3b640 1 --2- 172.21.15.26:0/3078334605 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa13c10e500 0x7fa13c19a620 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7fa13400dac0 tx=0x7fa13400df80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:00.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.379+0000 7fa1397fa640 1 -- 172.21.15.26:0/3078334605 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa134017070 con 0x7fa13c10e500 2024-09-11T23:36:00.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.379+0000 7fa14249e640 1 -- 172.21.15.26:0/3078334605 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa13c19b040 con 0x7fa13c10e500 2024-09-11T23:36:00.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.379+0000 7fa1397fa640 1 -- 172.21.15.26:0/3078334605 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa134003e50 con 0x7fa13c10e500 2024-09-11T23:36:00.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.379+0000 7fa1397fa640 1 -- 172.21.15.26:0/3078334605 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1340106c0 con 0x7fa13c10e500 2024-09-11T23:36:00.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.379+0000 7fa14249e640 1 -- 172.21.15.26:0/3078334605 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa13c1a7330 con 0x7fa13c10e500 2024-09-11T23:36:00.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.381+0000 7fa1397fa640 1 -- 172.21.15.26:0/3078334605 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fa134003970 con 0x7fa13c10e500 2024-09-11T23:36:00.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.382+0000 7fa144877740 1 Processor -- start 2024-09-11T23:36:00.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.382+0000 7fa144877740 1 -- start start 2024-09-11T23:36:00.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.382+0000 7fa144877740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b478dbb060 0x55b478dbb440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:00.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.382+0000 7fa144877740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b478dbb980 0x55b478dc5d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:00.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.382+0000 7fa144877740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b478dc6240 0x55b478dc8630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:00.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.382+0000 7fa144877740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55b478d20620 con 0x55b478dbb980 2024-09-11T23:36:00.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.382+0000 7fa144877740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55b478cc7e50 con 0x55b478dbb060 2024-09-11T23:36:00.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.382+0000 7fa144877740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55b478d1c5e0 con 0x55b478dc6240 2024-09-11T23:36:00.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.382+0000 7fa13bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b478dbb980 0x55b478dc5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:00.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.382+0000 7fa13b7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b478dbb060 0x55b478dbb440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:00.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.382+0000 7fa13bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b478dbb980 0x55b478dc5d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46820/0 (socket says 172.21.15.26:46820) 2024-09-11T23:36:00.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.382+0000 7fa13bfff640 1 -- 172.21.15.26:0/2554793335 learned_addr learned my addr 172.21.15.26:0/2554793335 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:36:00.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.382+0000 7fa1397fa640 1 --2- 172.21.15.26:0/3078334605 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fa1240a2cf0 0x7fa1240a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:00.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.382+0000 7fa13b7fe640 1 --2- 172.21.15.26:0/2554793335 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b478dc6240 0x55b478dc8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:00.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.383+0000 7fa140a3b640 1 --2- 172.21.15.26:0/3078334605 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fa1240a2cf0 0x7fa1240a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:00.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.383+0000 7fa13b7fe640 1 -- 172.21.15.26:0/2554793335 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b478dc6240 msgr2=0x55b478dc8630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:36:00.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.383+0000 7fa13b7fe640 1 --2- 172.21.15.26:0/2554793335 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b478dc6240 0x55b478dc8630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:00.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.383+0000 7fa1397fa640 1 -- 172.21.15.26:0/3078334605 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 8586+0+0 (secure 0 0 0) 0x7fa1340ceda0 con 0x7fa13c10e500 2024-09-11T23:36:00.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.383+0000 7fa13b7fe640 1 -- 172.21.15.26:0/2554793335 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b478dbb980 msgr2=0x55b478dc5d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:36:00.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.383+0000 7fa13b7fe640 1 --2- 172.21.15.26:0/2554793335 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b478dbb980 0x55b478dc5d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:00.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.383+0000 7fa13b7fe640 1 -- 172.21.15.26:0/2554793335 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b478cb8d40 con 0x55b478dbb060 2024-09-11T23:36:00.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.383+0000 7fa13b7fe640 1 --2- 172.21.15.26:0/2554793335 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b478dbb060 0x55b478dbb440 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fa130006010 tx=0x7fa1300314e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e8da97ed2af1f27e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:00.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.383+0000 7fa140a3b640 1 --2- 172.21.15.26:0/3078334605 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fa1240a2cf0 0x7fa1240a51b0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fa134097730 tx=0x7fa134021000 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:00.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.383+0000 7fa1227fc640 1 -- 172.21.15.26:0/2554793335 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa130035070 con 0x55b478dbb060 2024-09-11T23:36:00.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.383+0000 7fa1227fc640 1 -- 172.21.15.26:0/2554793335 <== mon.2 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa130005500 con 0x55b478dbb060 2024-09-11T23:36:00.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.383+0000 7fa1227fc640 1 -- 172.21.15.26:0/2554793335 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa130004c00 con 0x55b478dbb060 2024-09-11T23:36:00.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.384+0000 7fa144877740 1 -- 172.21.15.26:0/2554793335 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b478dbb060 msgr2=0x55b478dbb440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:00.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.384+0000 7fa144877740 1 --2- 172.21.15.26:0/2554793335 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b478dbb060 0x55b478dbb440 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fa130006010 tx=0x7fa1300314e0 comp rx=0 tx=0).stop 2024-09-11T23:36:00.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.384+0000 7fa144877740 1 -- 172.21.15.26:0/2554793335 shutdown_connections 2024-09-11T23:36:00.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.384+0000 7fa144877740 1 --2- 172.21.15.26:0/2554793335 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b478dc6240 0x55b478dc8630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:00.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.384+0000 7fa144877740 1 --2- 172.21.15.26:0/2554793335 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b478dbb980 0x55b478dc5d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:00.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.384+0000 7fa144877740 1 --2- 172.21.15.26:0/2554793335 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b478dbb060 0x55b478dbb440 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:00.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.384+0000 7fa144877740 1 -- 172.21.15.26:0/2554793335 >> 172.21.15.26:0/2554793335 conn(0x55b478db75f0 msgr2=0x55b478db79d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:00.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.384+0000 7fa144877740 1 -- 172.21.15.26:0/2554793335 shutdown_connections 2024-09-11T23:36:00.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.384+0000 7fa144877740 1 -- 172.21.15.26:0/2554793335 wait complete. 2024-09-11T23:36:00.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.384+0000 7fa144877740 1 -- start start 2024-09-11T23:36:00.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.384+0000 7fa144877740 1 Processor -- start 2024-09-11T23:36:00.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.385+0000 7fa144877740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b478dbb060 0x55b478dc31b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:00.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.385+0000 7fa144877740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b478dbb980 0x55b478dc36f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:00.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.385+0000 7fa144877740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b478dc6240 0x55b478dc3c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:00.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.385+0000 7fa144877740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55b478dc8e10 con 0x55b478dbb060 2024-09-11T23:36:00.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.385+0000 7fa144877740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55b478ca8f10 con 0x55b478dc6240 2024-09-11T23:36:00.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.385+0000 7fa144877740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55b478dc8f90 con 0x55b478dbb980 2024-09-11T23:36:00.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.385+0000 7fa144877740 10 client.0 _ll_register_callbacks cb 0x55b478cccdb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:36:00.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.385+0000 7fa13b7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b478dbb060 0x55b478dc31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:00.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.385+0000 7fa13b7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b478dbb060 0x55b478dc31b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46830/0 (socket says 172.21.15.26:46830) 2024-09-11T23:36:00.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.385+0000 7fa13b7fe640 1 -- 172.21.15.26:0/1636895954 learned_addr learned my addr 172.21.15.26:0/1636895954 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:36:00.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.385+0000 7fa13bfff640 1 --2- 172.21.15.26:0/1636895954 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b478dbb980 0x55b478dc36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:00.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.385+0000 7fa13b7fe640 1 --2- 172.21.15.26:0/1636895954 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b478dc6240 0x55b478dc3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:00.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.385+0000 7fa13bfff640 1 -- 172.21.15.26:0/1636895954 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b478dc6240 msgr2=0x55b478dc3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:36:00.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.385+0000 7fa13bfff640 1 --2- 172.21.15.26:0/1636895954 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b478dc6240 0x55b478dc3c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:00.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.385+0000 7fa13bfff640 1 -- 172.21.15.26:0/1636895954 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b478dbb060 msgr2=0x55b478dc31b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:36:00.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.385+0000 7fa13bfff640 1 --2- 172.21.15.26:0/1636895954 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b478dbb060 0x55b478dc31b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:00.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.385+0000 7fa13bfff640 1 -- 172.21.15.26:0/1636895954 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b478a2f580 con 0x55b478dbb980 2024-09-11T23:36:00.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.385+0000 7fa13b7fe640 1 --2- 172.21.15.26:0/1636895954 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b478dbb060 0x55b478dc31b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:36:00.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.385+0000 7fa13bfff640 1 --2- 172.21.15.26:0/1636895954 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b478dbb980 0x55b478dc36f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fa12c00bb10 tx=0x7fa12c004500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a969f327300ff93d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:00.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.385+0000 7fa11b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-09-11T23:36:00.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.386+0000 7fa11b7fe640 1 -- 172.21.15.26:0/1636895954 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa12c003e50 con 0x55b478dbb980 2024-09-11T23:36:00.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.386+0000 7fa11b7fe640 1 -- 172.21.15.26:0/1636895954 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 8586+0+0 (secure 0 0 0) 0x7fa12c011440 con 0x55b478dbb980 2024-09-11T23:36:00.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.386+0000 7fa144877740 1 -- 172.21.15.26:0/1636895954 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa130005500 con 0x55b478dbb980 2024-09-11T23:36:00.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.386+0000 7fa11b7fe640 1 -- 172.21.15.26:0/1636895954 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7fa12c011640 con 0x55b478dbb980 2024-09-11T23:36:00.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.386+0000 7fa144877740 10 client.27226 fetch_fsmap learned FSMap version 81 2024-09-11T23:36:00.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.386+0000 7fa144877740 1 -- 172.21.15.26:0/1636895954 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=81,monmap=4+}) -- 0x55b478af9960 con 0x55b478dbb980 2024-09-11T23:36:00.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.386+0000 7fa11b7fe640 1 -- 172.21.15.26:0/1636895954 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa130005500 con 0x55b478dbb980 2024-09-11T23:36:00.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.386+0000 7fa11b7fe640 1 -- 172.21.15.26:0/1636895954 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 81) ==== 8418+0+0 (secure 0 0 0) 0x7fa12c021030 con 0x55b478dbb980 2024-09-11T23:36:00.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.386+0000 7fa144877740 10 client.27226 fetch_fsmap finished waiting for FSMap version 81 2024-09-11T23:36:00.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.386+0000 7fa144877740 10 client.27226 resolve_mds: validated gid 24485 aka daemon mds.b 2024-09-11T23:36:00.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.386+0000 7fa144877740 20 client.27226 populate_metadata read hostname 'smithi026' 2024-09-11T23:36:00.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.386+0000 7fa144877740 1 --2- 172.21.15.26:0/1636895954 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55b478dd5f30 0x55b478dd83b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:00.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.386+0000 7fa144877740 4 client.27226 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:36:00.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.386+0000 7fa144877740 1 -- 172.21.15.26:0/1636895954 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b478d2db90 con 0x55b478dd5f30 2024-09-11T23:36:00.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.387+0000 7fa13b7fe640 1 --2- 172.21.15.26:0/1636895954 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55b478dd5f30 0x55b478dd83b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:00.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.387+0000 7fa13b7fe640 1 --2- 172.21.15.26:0/1636895954 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55b478dd5f30 0x55b478dd83b0 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ea5fd4ccc9c0d89 server_cookie=b51c8e6af724b2e5 in_seq=0 out_seq=0 2024-09-11T23:36:00.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.387+0000 7fa11b7fe640 10 client.27226 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:36:00.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.388+0000 7fa11b7fe640 1 -- 172.21.15.26:0/1636895954 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b478d2db90 con 0x55b478dd5f30 2024-09-11T23:36:00.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.388+0000 7fa11b7fe640 10 client.27226 handle_command_reply: tid=0 2024-09-11T23:36:00.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.388+0000 7fa144877740 1 client.27226 shutdown 2024-09-11T23:36:00.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.388+0000 7fa144877740 1 -- 172.21.15.26:0/1636895954 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b478dbb980 msgr2=0x55b478dc36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:00.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.388+0000 7fa144877740 1 --2- 172.21.15.26:0/1636895954 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b478dbb980 0x55b478dc36f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fa12c00bb10 tx=0x7fa12c004500 comp rx=0 tx=0).stop 2024-09-11T23:36:00.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.388+0000 7fa144877740 1 -- 172.21.15.26:0/1636895954 shutdown_connections 2024-09-11T23:36:00.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.388+0000 7fa144877740 1 --2- 172.21.15.26:0/1636895954 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b478dc6240 0x55b478dc3c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:00.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.388+0000 7fa144877740 1 --2- 172.21.15.26:0/1636895954 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55b478dd5f30 0x55b478dd83b0 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:00.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.388+0000 7fa144877740 1 --2- 172.21.15.26:0/1636895954 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b478dbb980 0x55b478dc36f0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:00.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.388+0000 7fa144877740 1 --2- 172.21.15.26:0/1636895954 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b478dbb060 0x55b478dc31b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:00.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.388+0000 7fa11b7fe640 0 client.27226 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:36:00.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.388+0000 7fa144877740 1 -- 172.21.15.26:0/1636895954 >> 172.21.15.26:0/1636895954 conn(0x55b478db75f0 msgr2=0x55b478dbd320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:00.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.388+0000 7fa144877740 1 -- 172.21.15.26:0/1636895954 shutdown_connections 2024-09-11T23:36:00.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.388+0000 7fa144877740 1 -- 172.21.15.26:0/1636895954 wait complete. 2024-09-11T23:36:00.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.388+0000 7fa144877740 20 client.27226 trim_cache size 0 max 16384 2024-09-11T23:36:00.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.406+0000 7fa144877740 1 Processor -- start 2024-09-11T23:36:00.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.406+0000 7fa144877740 1 -- start start 2024-09-11T23:36:00.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.406+0000 7fa144877740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b478dbb060 0x55b478dbdac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:00.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.406+0000 7fa144877740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b478dbb980 0x55b478db6640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:00.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.406+0000 7fa144877740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b478dc6240 0x55b478dddc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:00.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.406+0000 7fa144877740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fa13c1afb80 con 0x55b478dbb980 2024-09-11T23:36:00.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.406+0000 7fa144877740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fa13c1b1960 con 0x55b478dbb060 2024-09-11T23:36:00.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.406+0000 7fa144877740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fa13c1b0310 con 0x55b478dc6240 2024-09-11T23:36:00.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.406+0000 7fa13bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b478dbb980 0x55b478db6640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:00.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.406+0000 7fa13bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b478dbb980 0x55b478db6640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46836/0 (socket says 172.21.15.26:46836) 2024-09-11T23:36:00.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.406+0000 7fa13b7fe640 1 --2- 172.21.15.26:0/3028505458 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b478dbb060 0x55b478dbdac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:00.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.406+0000 7fa13bfff640 1 -- 172.21.15.26:0/3028505458 learned_addr learned my addr 172.21.15.26:0/3028505458 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:36:00.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.406+0000 7fa13b7fe640 1 --2- 172.21.15.26:0/3028505458 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b478dc6240 0x55b478dddc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:00.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.406+0000 7fa13bfff640 1 -- 172.21.15.26:0/3028505458 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b478dc6240 msgr2=0x55b478dddc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:36:00.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.406+0000 7fa13bfff640 1 --2- 172.21.15.26:0/3028505458 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b478dc6240 0x55b478dddc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:00.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.406+0000 7fa13bfff640 1 -- 172.21.15.26:0/3028505458 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b478dbb060 msgr2=0x55b478dbdac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:36:00.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.406+0000 7fa13bfff640 1 --2- 172.21.15.26:0/3028505458 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b478dbb060 0x55b478dbdac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:00.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.406+0000 7fa13bfff640 1 -- 172.21.15.26:0/3028505458 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b478db5bb0 con 0x55b478dbb980 2024-09-11T23:36:00.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.406+0000 7fa13b7fe640 1 --2- 172.21.15.26:0/3028505458 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b478dbb060 0x55b478dbdac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:36:00.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.407+0000 7fa13bfff640 1 --2- 172.21.15.26:0/3028505458 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b478dbb980 0x55b478db6640 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7fa12c01bc20 tx=0x7fa12c0039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ff2d74f00615480 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:00.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.407+0000 7fa121ffb640 1 -- 172.21.15.26:0/3028505458 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa12c003e50 con 0x55b478dbb980 2024-09-11T23:36:00.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.407+0000 7fa121ffb640 1 -- 172.21.15.26:0/3028505458 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa12c011440 con 0x55b478dbb980 2024-09-11T23:36:00.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.407+0000 7fa121ffb640 1 -- 172.21.15.26:0/3028505458 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa12c0104b0 con 0x55b478dbb980 2024-09-11T23:36:00.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.407+0000 7fa144877740 1 -- 172.21.15.26:0/3028505458 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b478dbb980 msgr2=0x55b478db6640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:00.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.407+0000 7fa144877740 1 --2- 172.21.15.26:0/3028505458 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b478dbb980 0x55b478db6640 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7fa12c01bc20 tx=0x7fa12c0039c0 comp rx=0 tx=0).stop 2024-09-11T23:36:00.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.407+0000 7fa144877740 1 -- 172.21.15.26:0/3028505458 shutdown_connections 2024-09-11T23:36:00.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.407+0000 7fa144877740 1 --2- 172.21.15.26:0/3028505458 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b478dc6240 0x55b478dddc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:00.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.407+0000 7fa144877740 1 --2- 172.21.15.26:0/3028505458 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b478dbb980 0x55b478db6640 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:00.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.407+0000 7fa144877740 1 --2- 172.21.15.26:0/3028505458 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b478dbb060 0x55b478dbdac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:00.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.407+0000 7fa144877740 1 -- 172.21.15.26:0/3028505458 >> 172.21.15.26:0/3028505458 conn(0x55b478db75f0 msgr2=0x55b478db5f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:00.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.408+0000 7fa144877740 1 -- 172.21.15.26:0/3028505458 shutdown_connections 2024-09-11T23:36:00.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.408+0000 7fa144877740 1 -- 172.21.15.26:0/3028505458 wait complete. 2024-09-11T23:36:00.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.408+0000 7fa144877740 1 -- start start 2024-09-11T23:36:00.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.408+0000 7fa144877740 1 Processor -- start 2024-09-11T23:36:00.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.408+0000 7fa144877740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b478dbb060 0x55b478dca680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:00.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.408+0000 7fa144877740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b478dbb980 0x55b478dcabc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:00.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.408+0000 7fa144877740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b478dc6240 0x55b478e3fc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:00.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.408+0000 7fa144877740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fa13c1b11d0 con 0x55b478dc6240 2024-09-11T23:36:00.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.408+0000 7fa144877740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fa13c1b0a40 con 0x55b478dbb980 2024-09-11T23:36:00.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.408+0000 7fa144877740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fa13c1adb80 con 0x55b478dbb060 2024-09-11T23:36:00.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.408+0000 7fa144877740 10 client.0 _ll_register_callbacks cb 0x7fa12c0043c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:36:00.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.409+0000 7fa13b7fe640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b478dbb060 0x55b478dca680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:00.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.409+0000 7fa13b7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b478dc6240 0x55b478e3fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:00.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.409+0000 7fa13bfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b478dbb980 0x55b478dcabc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:00.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.409+0000 7fa13b7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b478dc6240 0x55b478e3fc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46844/0 (socket says 172.21.15.26:46844) 2024-09-11T23:36:00.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.409+0000 7fa13b7fe640 1 -- 172.21.15.26:0/1681548758 learned_addr learned my addr 172.21.15.26:0/1681548758 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:36:00.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.409+0000 7fa13b7fe640 1 -- 172.21.15.26:0/1681548758 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b478dbb060 msgr2=0x55b478dca680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:36:00.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.409+0000 7fa13b7fe640 1 --2- 172.21.15.26:0/1681548758 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b478dbb060 0x55b478dca680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:00.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.409+0000 7fa13b7fe640 1 -- 172.21.15.26:0/1681548758 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b478dbb980 msgr2=0x55b478dcabc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:36:00.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.409+0000 7fa13b7fe640 1 --2- 172.21.15.26:0/1681548758 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b478dbb980 0x55b478dcabc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:00.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.409+0000 7fa13b7fe640 1 -- 172.21.15.26:0/1681548758 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b478e40190 con 0x55b478dc6240 2024-09-11T23:36:00.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.409+0000 7fa13b7fe640 1 --2- 172.21.15.26:0/1681548758 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b478dc6240 0x55b478e3fc50 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7fa130031550 tx=0x7fa130050040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec3ebc89e4a0e25b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:00.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.409+0000 7fa11b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:36:00.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.409+0000 7fa11b7fe640 1 -- 172.21.15.26:0/1681548758 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa130035040 con 0x55b478dc6240 2024-09-11T23:36:00.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.409+0000 7fa11b7fe640 1 -- 172.21.15.26:0/1681548758 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 8586+0+0 (secure 0 0 0) 0x7fa130059e80 con 0x55b478dc6240 2024-09-11T23:36:00.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.409+0000 7fa144877740 1 -- 172.21.15.26:0/1681548758 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa12c011440 con 0x55b478dc6240 2024-09-11T23:36:00.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.410+0000 7fa11b7fe640 1 -- 172.21.15.26:0/1681548758 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7fa130003800 con 0x55b478dc6240 2024-09-11T23:36:00.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.410+0000 7fa144877740 10 client.17760 fetch_fsmap learned FSMap version 81 2024-09-11T23:36:00.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.410+0000 7fa144877740 1 -- 172.21.15.26:0/1681548758 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=81,monmap=4+}) -- 0x55b478e414d0 con 0x55b478dc6240 2024-09-11T23:36:00.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.410+0000 7fa11b7fe640 1 -- 172.21.15.26:0/1681548758 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa12c011440 con 0x55b478dc6240 2024-09-11T23:36:00.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.410+0000 7fa11b7fe640 1 -- 172.21.15.26:0/1681548758 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 81) ==== 8418+0+0 (secure 0 0 0) 0x7fa130064060 con 0x55b478dc6240 2024-09-11T23:36:00.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.410+0000 7fa144877740 10 client.17760 fetch_fsmap finished waiting for FSMap version 81 2024-09-11T23:36:00.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.410+0000 7fa144877740 10 client.17760 resolve_mds: validated gid 24485 aka daemon mds.b 2024-09-11T23:36:00.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.410+0000 7fa144877740 20 client.17760 populate_metadata read hostname 'smithi026' 2024-09-11T23:36:00.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.410+0000 7fa144877740 1 --2- 172.21.15.26:0/1681548758 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55b478e41750 0x55b478e43b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:00.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.410+0000 7fa144877740 4 client.17760 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "89adda10", "release": true, "await": true}] 2024-09-11T23:36:00.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.410+0000 7fa144877740 1 -- 172.21.15.26:0/1681548758 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "quiesce db", "set_id": "89adda10", "release": true, "await": true}) -- 0x55b478d280a0 con 0x55b478e41750 2024-09-11T23:36:00.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.410+0000 7fa13bfff640 1 --2- 172.21.15.26:0/1681548758 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55b478e41750 0x55b478e43b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:00.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.411+0000 7fa11b7fe640 10 client.17760 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:36:00.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:00.411+0000 7fa13bfff640 1 --2- 172.21.15.26:0/1681548758 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55b478e41750 0x55b478e43b40 crc :-1 s=READY pgs=271 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c668ef898cebf641 server_cookie=6f199e4d396ae376 in_seq=0 out_seq=0 2024-09-11T23:36:01.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:01.166+0000 7fa11b7fe640 1 -- 172.21.15.26:0/1681548758 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x7fa12c031260 con 0x55b478e41750 2024-09-11T23:36:01.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:01.166+0000 7fa11b7fe640 10 client.17760 handle_command_reply: tid=0 2024-09-11T23:36:01.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:01.166+0000 7fa144877740 1 client.17760 shutdown 2024-09-11T23:36:01.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:01.166+0000 7fa144877740 1 -- 172.21.15.26:0/1681548758 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b478dc6240 msgr2=0x55b478e3fc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:01.166 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:36:00 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:36:01.166 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:36:00 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:36:01.166 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:36:00 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:36:01.167 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:36:00 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:36:01.167 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:36:00 smithi026 ceph-mon[31172]: osdmap e108: 12 total, 12 up, 12 in 2024-09-11T23:36:01.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:01.166+0000 7fa144877740 1 --2- 172.21.15.26:0/1681548758 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b478dc6240 0x55b478e3fc50 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7fa130031550 tx=0x7fa130050040 comp rx=0 tx=0).stop 2024-09-11T23:36:01.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:01.166+0000 7fa144877740 1 -- 172.21.15.26:0/1681548758 shutdown_connections 2024-09-11T23:36:01.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:01.166+0000 7fa144877740 1 --2- 172.21.15.26:0/1681548758 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b478dbb980 0x55b478dcabc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:01.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:01.166+0000 7fa144877740 1 --2- 172.21.15.26:0/1681548758 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55b478e41750 0x55b478e43b40 crc :-1 s=READY pgs=271 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:01.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:01.166+0000 7fa144877740 1 --2- 172.21.15.26:0/1681548758 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b478dc6240 0x55b478e3fc50 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:01.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:01.166+0000 7fa144877740 1 --2- 172.21.15.26:0/1681548758 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b478dbb060 0x55b478dca680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:01.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:01.166+0000 7fa11b7fe640 0 client.17760 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:36:01.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:01.166+0000 7fa144877740 1 -- 172.21.15.26:0/1681548758 >> 172.21.15.26:0/1681548758 conn(0x55b478db75f0 msgr2=0x55b478dc3990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:01.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:01.167+0000 7fa144877740 1 -- 172.21.15.26:0/1681548758 shutdown_connections 2024-09-11T23:36:01.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:01.167+0000 7fa144877740 1 -- 172.21.15.26:0/1681548758 wait complete. 2024-09-11T23:36:01.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:01.167+0000 7fa144877740 20 client.17760 trim_cache size 0 max 16384 2024-09-11T23:36:01.168 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-11T23:36:01.169 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 81, 2024-09-11T23:36:01.169 INFO:teuthology.orchestra.run.smithi026.stdout: "leader": 24485, 2024-09-11T23:36:01.169 INFO:teuthology.orchestra.run.smithi026.stdout: "set_version": 60, 2024-09-11T23:36:01.169 INFO:teuthology.orchestra.run.smithi026.stdout: "sets": { 2024-09-11T23:36:01.169 INFO:teuthology.orchestra.run.smithi026.stdout: "89adda10": { 2024-09-11T23:36:01.169 INFO:teuthology.orchestra.run.smithi026.stdout: "version": 60, 2024-09-11T23:36:01.169 INFO:teuthology.orchestra.run.smithi026.stdout: "age_ref": 0.0, 2024-09-11T23:36:01.169 INFO:teuthology.orchestra.run.smithi026.stdout: "state": { 2024-09-11T23:36:01.169 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "RELEASED", 2024-09-11T23:36:01.169 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.0 2024-09-11T23:36:01.169 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:36:01.169 INFO:teuthology.orchestra.run.smithi026.stdout: "timeout": 90.0, 2024-09-11T23:36:01.169 INFO:teuthology.orchestra.run.smithi026.stdout: "expiration": 144.2, 2024-09-11T23:36:01.169 INFO:teuthology.orchestra.run.smithi026.stdout: "members": { 2024-09-11T23:36:01.170 INFO:teuthology.orchestra.run.smithi026.stdout: "file:/": { 2024-09-11T23:36:01.170 INFO:teuthology.orchestra.run.smithi026.stdout: "excluded": false, 2024-09-11T23:36:01.170 INFO:teuthology.orchestra.run.smithi026.stdout: "state": { 2024-09-11T23:36:01.170 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "RELEASED", 2024-09-11T23:36:01.170 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.0 2024-09-11T23:36:01.170 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:36:01.170 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:36:01.170 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:36:01.170 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:36:01.170 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:36:01.170 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-11T23:36:01.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:01.168+0000 7fa14249e640 1 -- 172.21.15.26:0/3078334605 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fa1240a2cf0 msgr2=0x7fa1240a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:01.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:01.168+0000 7fa14249e640 1 --2- 172.21.15.26:0/3078334605 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fa1240a2cf0 0x7fa1240a51b0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fa134097730 tx=0x7fa134021000 comp rx=0 tx=0).stop 2024-09-11T23:36:01.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:01.168+0000 7fa14249e640 1 -- 172.21.15.26:0/3078334605 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa13c10e500 msgr2=0x7fa13c19a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:01.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:01.168+0000 7fa14249e640 1 --2- 172.21.15.26:0/3078334605 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa13c10e500 0x7fa13c19a620 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7fa13400dac0 tx=0x7fa13400df80 comp rx=0 tx=0).stop 2024-09-11T23:36:01.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:01.168+0000 7fa14249e640 1 -- 172.21.15.26:0/3078334605 shutdown_connections 2024-09-11T23:36:01.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:01.168+0000 7fa14249e640 1 --2- 172.21.15.26:0/3078334605 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fa1240a2cf0 0x7fa1240a51b0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:01.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:01.168+0000 7fa14249e640 1 --2- 172.21.15.26:0/3078334605 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa13c101440 0x7fa13c1a09a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:01.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:01.168+0000 7fa14249e640 1 --2- 172.21.15.26:0/3078334605 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa13c10e500 0x7fa13c19a620 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:01.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:01.168+0000 7fa14249e640 1 --2- 172.21.15.26:0/3078334605 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa13c100b20 0x7fa13c1a0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:01.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:01.168+0000 7fa14249e640 1 -- 172.21.15.26:0/3078334605 >> 172.21.15.26:0/3078334605 conn(0x7fa13c0fc7d0 msgr2=0x7fa13c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:01.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:01.168+0000 7fa14249e640 1 -- 172.21.15.26:0/3078334605 shutdown_connections 2024-09-11T23:36:01.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:01.168+0000 7fa14249e640 1 -- 172.21.15.26:0/3078334605 wait complete. 2024-09-11T23:36:01.238 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:36:00 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:36:01.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:36:00 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:36:01.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:36:00 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:36:01.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:36:00 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:36:01.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:36:00 smithi136 ceph-mon[35121]: osdmap e108: 12 total, 12 up, 12 in 2024-09-11T23:36:01.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:36:00 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:36:01.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:36:00 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:36:01.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:36:00 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:36:01.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:36:00 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:36:01.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:36:00 smithi076 ceph-mon[35296]: osdmap e108: 12 total, 12 up, 12 in 2024-09-11T23:36:01.882 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '89adda10', total seconds elapsed: 29.1 2024-09-11T23:36:01.882 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 24.25 2024-09-11T23:36:01.882 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:36:02.146 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:36:02.238 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:36:01 smithi136 ceph-mon[35121]: pgmap v553: 97 pgs: 97 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 16 op/s 2024-09-11T23:36:02.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:36:01 smithi026 ceph-mon[31172]: pgmap v553: 97 pgs: 97 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 16 op/s 2024-09-11T23:36:02.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:36:01 smithi076 ceph-mon[35296]: pgmap v553: 97 pgs: 97 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 16 op/s 2024-09-11T23:36:02.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:02.681+0000 7f82098eb640 1 -- 172.21.15.26:0/487434903 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8204106f70 msgr2=0x7f820410eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:02.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:02.681+0000 7f82098eb640 1 --2- 172.21.15.26:0/487434903 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8204106f70 0x7f820410eab0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f81fc009ca0 tx=0x7f81fc02a8c0 comp rx=0 tx=0).stop 2024-09-11T23:36:02.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:02.682+0000 7f82098eb640 1 -- 172.21.15.26:0/487434903 shutdown_connections 2024-09-11T23:36:02.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:02.682+0000 7f82098eb640 1 --2- 172.21.15.26:0/487434903 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8204106f70 0x7f820410eab0 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:02.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:02.682+0000 7f82098eb640 1 --2- 172.21.15.26:0/487434903 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8204100eb0 0x7f8204106840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:02.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:02.682+0000 7f82098eb640 1 --2- 172.21.15.26:0/487434903 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8204100500 0x7f82041008e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:02.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:02.682+0000 7f82098eb640 1 -- 172.21.15.26:0/487434903 >> 172.21.15.26:0/487434903 conn(0x7f82040fc5c0 msgr2=0x7f82040fe9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:02.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:02.682+0000 7f82098eb640 1 -- 172.21.15.26:0/487434903 shutdown_connections 2024-09-11T23:36:02.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:02.682+0000 7f82098eb640 1 -- 172.21.15.26:0/487434903 wait complete. 2024-09-11T23:36:02.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:02.682+0000 7f82098eb640 1 Processor -- start 2024-09-11T23:36:02.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:02.682+0000 7f82098eb640 1 -- start start 2024-09-11T23:36:02.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:02.683+0000 7f82098eb640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8204100500 0x7f820419a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:02.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:02.683+0000 7f82098eb640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8204100eb0 0x7f820419abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:02.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:02.683+0000 7f82098eb640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8204106f70 0x7f820419ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:02.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:02.683+0000 7f82098eb640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f8204111f90 con 0x7f8204100eb0 2024-09-11T23:36:02.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:02.683+0000 7f82098eb640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f8204111e10 con 0x7f8204100500 2024-09-11T23:36:02.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:02.683+0000 7f82098eb640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f8204112110 con 0x7f8204106f70 2024-09-11T23:36:02.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:02.683+0000 7f82027fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8204100eb0 0x7f820419abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:02.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:02.683+0000 7f82027fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8204100eb0 0x7f820419abf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53760/0 (socket says 172.21.15.26:53760) 2024-09-11T23:36:02.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:02.683+0000 7f82027fc640 1 -- 172.21.15.26:0/755796691 learned_addr learned my addr 172.21.15.26:0/755796691 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:36:02.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:02.683+0000 7f8202ffd640 1 --2- 172.21.15.26:0/755796691 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8204100500 0x7f820419a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:02.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:02.683+0000 7f82037fe640 1 --2- 172.21.15.26:0/755796691 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8204106f70 0x7f820419ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:02.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:02.683+0000 7f82027fc640 1 -- 172.21.15.26:0/755796691 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8204106f70 msgr2=0x7f820419ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:02.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:02.683+0000 7f82027fc640 1 --2- 172.21.15.26:0/755796691 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8204106f70 0x7f820419ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:02.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:02.683+0000 7f82027fc640 1 -- 172.21.15.26:0/755796691 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8204100500 msgr2=0x7f820419a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:02.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:02.683+0000 7f82027fc640 1 --2- 172.21.15.26:0/755796691 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8204100500 0x7f820419a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:02.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:02.683+0000 7f82027fc640 1 -- 172.21.15.26:0/755796691 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f820419f660 con 0x7f8204100eb0 2024-09-11T23:36:02.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:02.683+0000 7f82027fc640 1 --2- 172.21.15.26:0/755796691 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8204100eb0 0x7f820419abf0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f81f800d950 tx=0x7f81f800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:02.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:02.683+0000 7f82088e9640 1 -- 172.21.15.26:0/755796691 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81f8017070 con 0x7f8204100eb0 2024-09-11T23:36:02.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:02.684+0000 7f82088e9640 1 -- 172.21.15.26:0/755796691 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f81f8009c30 con 0x7f8204100eb0 2024-09-11T23:36:02.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:02.684+0000 7f82098eb640 1 -- 172.21.15.26:0/755796691 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f820419f950 con 0x7f8204100eb0 2024-09-11T23:36:02.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:02.684+0000 7f82088e9640 1 -- 172.21.15.26:0/755796691 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81f80047a0 con 0x7f8204100eb0 2024-09-11T23:36:02.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:02.684+0000 7f82098eb640 1 -- 172.21.15.26:0/755796691 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82041a7230 con 0x7f8204100eb0 2024-09-11T23:36:02.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:02.685+0000 7f82098eb640 1 -- 172.21.15.26:0/755796691 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82041008e0 con 0x7f8204100eb0 2024-09-11T23:36:02.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:02.686+0000 7f82088e9640 1 -- 172.21.15.26:0/755796691 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f81f8004020 con 0x7f8204100eb0 2024-09-11T23:36:02.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:02.686+0000 7f82088e9640 1 --2- 172.21.15.26:0/755796691 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f81ec0a2cf0 0x7f81ec0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:02.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:02.687+0000 7f8202ffd640 1 --2- 172.21.15.26:0/755796691 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f81ec0a2cf0 0x7f81ec0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:02.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:02.687+0000 7f82088e9640 1 -- 172.21.15.26:0/755796691 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 8542+0+0 (secure 0 0 0) 0x7f81f80cef90 con 0x7f8204100eb0 2024-09-11T23:36:02.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:02.687+0000 7f8202ffd640 1 --2- 172.21.15.26:0/755796691 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f81ec0a2cf0 0x7f81ec0a51b0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f8204069fc0 tx=0x7f81f400a2d0 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:02.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:02.691+0000 7f82088e9640 1 -- 172.21.15.26:0/755796691 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f81f8020020 con 0x7f8204100eb0 2024-09-11T23:36:02.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:02.904+0000 7f82098eb640 1 -- 172.21.15.26:0/755796691 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f82041a74d0 con 0x7f8204100eb0 2024-09-11T23:36:02.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:02.905+0000 7f82088e9640 1 -- 172.21.15.26:0/755796691 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 81 v81) ==== 94+0+11996 (secure 0 0 0) 0x7f81f809bba0 con 0x7f8204100eb0 2024-09-11T23:36:02.905 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-11T23:36:02.905 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":81,"btime":"2024-09-11T23:35:48:177981+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14499,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/367659515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":367659515},{"type":"v1","addr":"172.21.15.26:6835","nonce":367659515}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14511,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/2041065299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":2041065299},{"type":"v1","addr":"172.21.15.26:6837","nonce":2041065299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":81,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:16:26.544203+0000","modified":"2024-09-11T23:35:47.915600+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24485,"mds_1":24463,"mds_2":24458,"mds_3":24448,"mds_4":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14550":{"gid":14550,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6839/2233068305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":2233068305},{"type":"v1","addr":"172.21.15.26:6839","nonce":2233068305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14562":{"gid":14562,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6841/2984982336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":2984982336},{"type":"v1","addr":"172.21.15.26:6841","nonce":2984982336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6833/2019841455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":2019841455},{"type":"v1","addr":"172.21.15.136:6833","nonce":2019841455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.76:6833/3438147050","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3438147050},{"type":"v1","addr":"172.21.15.76:6833","nonce":3438147050}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6835/3617665975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3617665975},{"type":"v1","addr":"172.21.15.136:6835","nonce":3617665975}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":12,"addr":"172.21.15.76:6835/393308499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":393308499},{"type":"v1","addr":"172.21.15.76:6835","nonce":393308499}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":10,"addr":"172.21.15.136:6837/1633082387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":1633082387},{"type":"v1","addr":"172.21.15.136:6837","nonce":1633082387}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.76:6837/2078978748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":2078978748},{"type":"v1","addr":"172.21.15.76:6837","nonce":2078978748}]},"join_fscid":-1,"export_targets":[0,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":63,"addr":"172.21.15.136:6839/3211398630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":3211398630},{"type":"v1","addr":"172.21.15.136:6839","nonce":3211398630}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.76:6839/2077637090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6838","nonce":2077637090},{"type":"v1","addr":"172.21.15.76:6839","nonce":2077637090}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24485,"qdb_cluster":[24443,24448,24458,24463,24485]},"id":1}]} 2024-09-11T23:36:02.905 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 81 2024-09-11T23:36:02.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:02.908+0000 7f82098eb640 1 -- 172.21.15.26:0/755796691 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f81ec0a2cf0 msgr2=0x7f81ec0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:02.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:02.908+0000 7f82098eb640 1 --2- 172.21.15.26:0/755796691 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f81ec0a2cf0 0x7f81ec0a51b0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f8204069fc0 tx=0x7f81f400a2d0 comp rx=0 tx=0).stop 2024-09-11T23:36:02.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:02.908+0000 7f82098eb640 1 -- 172.21.15.26:0/755796691 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8204100eb0 msgr2=0x7f820419abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:02.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:02.908+0000 7f82098eb640 1 --2- 172.21.15.26:0/755796691 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8204100eb0 0x7f820419abf0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f81f800d950 tx=0x7f81f800de10 comp rx=0 tx=0).stop 2024-09-11T23:36:02.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:02.908+0000 7f82098eb640 1 -- 172.21.15.26:0/755796691 shutdown_connections 2024-09-11T23:36:02.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:02.908+0000 7f82098eb640 1 --2- 172.21.15.26:0/755796691 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f81ec0a2cf0 0x7f81ec0a51b0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:02.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:02.908+0000 7f82098eb640 1 --2- 172.21.15.26:0/755796691 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8204106f70 0x7f820419ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:02.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:02.908+0000 7f82098eb640 1 --2- 172.21.15.26:0/755796691 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8204100eb0 0x7f820419abf0 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:02.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:02.908+0000 7f82098eb640 1 --2- 172.21.15.26:0/755796691 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8204100500 0x7f820419a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:02.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:02.908+0000 7f82098eb640 1 -- 172.21.15.26:0/755796691 >> 172.21.15.26:0/755796691 conn(0x7f82040fc5c0 msgr2=0x7f82041056f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:02.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:02.909+0000 7f82098eb640 1 -- 172.21.15.26:0/755796691 shutdown_connections 2024-09-11T23:36:02.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:02.909+0000 7f82098eb640 1 -- 172.21.15.26:0/755796691 wait complete. 2024-09-11T23:36:03.765 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24485 quiesce db / --timeout 90 --expiration 144.25 --await-for 45.0' 2024-09-11T23:36:03.766 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24485 quiesce db / --timeout 90 --expiration 144.25 --await-for 45.0 2024-09-11T23:36:04.000 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:36:03 smithi026 ceph-mon[31172]: pgmap v555: 97 pgs: 97 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 13 op/s 2024-09-11T23:36:04.000 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:36:03 smithi026 ceph-mon[31172]: from='client.? 172.21.15.26:0/755796691' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:36:04.027 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:36:04.238 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:36:03 smithi136 ceph-mon[35121]: pgmap v555: 97 pgs: 97 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 13 op/s 2024-09-11T23:36:04.238 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:36:03 smithi136 ceph-mon[35121]: from='client.? 172.21.15.26:0/755796691' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:36:04.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:36:03 smithi076 ceph-mon[35296]: pgmap v555: 97 pgs: 97 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 13 op/s 2024-09-11T23:36:04.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:36:03 smithi076 ceph-mon[35296]: from='client.? 172.21.15.26:0/755796691' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:36:04.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.602+0000 7f3474c44640 1 -- 172.21.15.26:0/2575347685 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3470102790 msgr2=0x7f34701067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:04.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.602+0000 7f3474c44640 1 --2- 172.21.15.26:0/2575347685 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3470102790 0x7f34701067e0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f3464009900 tx=0x7f346402a410 comp rx=0 tx=0).stop 2024-09-11T23:36:04.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.602+0000 7f3474c44640 1 -- 172.21.15.26:0/2575347685 shutdown_connections 2024-09-11T23:36:04.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.602+0000 7f3474c44640 1 --2- 172.21.15.26:0/2575347685 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3470106f10 0x7f347010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:04.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.602+0000 7f3474c44640 1 --2- 172.21.15.26:0/2575347685 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3470102790 0x7f34701067e0 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:04.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.602+0000 7f3474c44640 1 --2- 172.21.15.26:0/2575347685 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3470101de0 0x7f34701021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:04.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.602+0000 7f3474c44640 1 -- 172.21.15.26:0/2575347685 >> 172.21.15.26:0/2575347685 conn(0x7f34700fd8c0 msgr2=0x7f34700ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:04.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.602+0000 7f3474c44640 1 -- 172.21.15.26:0/2575347685 shutdown_connections 2024-09-11T23:36:04.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.602+0000 7f3474c44640 1 -- 172.21.15.26:0/2575347685 wait complete. 2024-09-11T23:36:04.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.603+0000 7f3474c44640 1 Processor -- start 2024-09-11T23:36:04.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.603+0000 7f3474c44640 1 -- start start 2024-09-11T23:36:04.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.603+0000 7f3474c44640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3470101de0 0x7f3470198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:04.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.603+0000 7f3474c44640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3470102790 0x7f3470198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:04.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.603+0000 7f3474c44640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3470106f10 0x7f347019cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:04.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.603+0000 7f3474c44640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f3470111f70 con 0x7f3470102790 2024-09-11T23:36:04.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.603+0000 7f3474c44640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f3470111df0 con 0x7f3470106f10 2024-09-11T23:36:04.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.603+0000 7f3474c44640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f34701120f0 con 0x7f3470101de0 2024-09-11T23:36:04.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.603+0000 7f346dd9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3470102790 0x7f3470198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:04.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.603+0000 7f346e59c640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3470101de0 0x7f3470198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:04.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.603+0000 7f346dd9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3470102790 0x7f3470198990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53782/0 (socket says 172.21.15.26:53782) 2024-09-11T23:36:04.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.603+0000 7f346e59c640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3470101de0 0x7f3470198450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:42300/0 (socket says 172.21.15.26:42300) 2024-09-11T23:36:04.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.603+0000 7f346dd9b640 1 -- 172.21.15.26:0/3720740307 learned_addr learned my addr 172.21.15.26:0/3720740307 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:36:04.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.603+0000 7f346ed9d640 1 --2- 172.21.15.26:0/3720740307 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3470106f10 0x7f347019cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:04.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.604+0000 7f346dd9b640 1 -- 172.21.15.26:0/3720740307 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3470101de0 msgr2=0x7f3470198450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:04.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.604+0000 7f346dd9b640 1 --2- 172.21.15.26:0/3720740307 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3470101de0 0x7f3470198450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:04.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.604+0000 7f346dd9b640 1 -- 172.21.15.26:0/3720740307 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3470106f10 msgr2=0x7f347019cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:04.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.604+0000 7f346dd9b640 1 --2- 172.21.15.26:0/3720740307 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3470106f10 0x7f347019cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:04.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.604+0000 7f346dd9b640 1 -- 172.21.15.26:0/3720740307 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f347019d400 con 0x7f3470102790 2024-09-11T23:36:04.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.604+0000 7f346dd9b640 1 --2- 172.21.15.26:0/3720740307 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3470102790 0x7f3470198990 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f3464006010 tx=0x7f34640042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:04.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.604+0000 7f345f7fe640 1 -- 172.21.15.26:0/3720740307 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f346403f030 con 0x7f3470102790 2024-09-11T23:36:04.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.604+0000 7f345f7fe640 1 -- 172.21.15.26:0/3720740307 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3464003ce0 con 0x7f3470102790 2024-09-11T23:36:04.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.604+0000 7f3474c44640 1 -- 172.21.15.26:0/3720740307 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f347019d690 con 0x7f3470102790 2024-09-11T23:36:04.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.604+0000 7f345f7fe640 1 -- 172.21.15.26:0/3720740307 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3464035070 con 0x7f3470102790 2024-09-11T23:36:04.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.604+0000 7f3474c44640 1 -- 172.21.15.26:0/3720740307 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34701a4fd0 con 0x7f3470102790 2024-09-11T23:36:04.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.607+0000 7f345f7fe640 1 -- 172.21.15.26:0/3720740307 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f3464042030 con 0x7f3470102790 2024-09-11T23:36:04.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.607+0000 7f347701d740 1 Processor -- start 2024-09-11T23:36:04.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.608+0000 7f347701d740 1 -- start start 2024-09-11T23:36:04.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.608+0000 7f347701d740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560ceb964730 0x560ceb964b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:04.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.608+0000 7f347701d740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560ceb965050 0x560ceb973900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:04.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.608+0000 7f347701d740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560ceb973e40 0x560ceb976230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:04.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.608+0000 7f347701d740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x560ceb8c9160 con 0x560ceb965050 2024-09-11T23:36:04.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.608+0000 7f347701d740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x560ceb870720 con 0x560ceb973e40 2024-09-11T23:36:04.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.608+0000 7f347701d740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x560ceb8c5120 con 0x560ceb964730 2024-09-11T23:36:04.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.608+0000 7f346e59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560ceb965050 0x560ceb973900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:04.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.608+0000 7f346dd9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560ceb973e40 0x560ceb976230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:04.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.608+0000 7f346e59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560ceb965050 0x560ceb973900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53796/0 (socket says 172.21.15.26:53796) 2024-09-11T23:36:04.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.608+0000 7f346e59c640 1 -- 172.21.15.26:0/3052131019 learned_addr learned my addr 172.21.15.26:0/3052131019 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:36:04.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.608+0000 7f346ed9d640 1 --2- 172.21.15.26:0/3052131019 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560ceb964730 0x560ceb964b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:04.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.608+0000 7f345f7fe640 1 --2- 172.21.15.26:0/3720740307 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f34580a2cf0 0x7f34580a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:04.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.608+0000 7f346ed9d640 1 --2- 172.21.15.26:0/3720740307 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f34580a2cf0 0x7f34580a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:04.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.608+0000 7f346e59c640 1 -- 172.21.15.26:0/3052131019 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560ceb964730 msgr2=0x560ceb964b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:36:04.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.608+0000 7f346e59c640 1 --2- 172.21.15.26:0/3052131019 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560ceb964730 0x560ceb964b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:04.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.608+0000 7f346e59c640 1 -- 172.21.15.26:0/3052131019 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560ceb973e40 msgr2=0x560ceb976230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:36:04.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.608+0000 7f346e59c640 1 --2- 172.21.15.26:0/3052131019 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560ceb973e40 0x560ceb976230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:04.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.608+0000 7f346e59c640 1 -- 172.21.15.26:0/3052131019 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ceb63aba0 con 0x560ceb965050 2024-09-11T23:36:04.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.608+0000 7f346dd9b640 1 --2- 172.21.15.26:0/3052131019 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560ceb973e40 0x560ceb976230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:36:04.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.609+0000 7f345f7fe640 1 -- 172.21.15.26:0/3720740307 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 8542+0+0 (secure 0 0 0) 0x7f34640ef990 con 0x7f3470102790 2024-09-11T23:36:04.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.609+0000 7f346e59c640 1 --2- 172.21.15.26:0/3052131019 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560ceb965050 0x560ceb973900 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f346000b760 tx=0x7f346000bc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35d5d9aefe36341b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:04.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.609+0000 7f345cff9640 1 -- 172.21.15.26:0/3052131019 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3460003dd0 con 0x560ceb965050 2024-09-11T23:36:04.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.609+0000 7f345cff9640 1 -- 172.21.15.26:0/3052131019 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3460015070 con 0x560ceb965050 2024-09-11T23:36:04.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.609+0000 7f345cff9640 1 -- 172.21.15.26:0/3052131019 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3460011440 con 0x560ceb965050 2024-09-11T23:36:04.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.609+0000 7f346ed9d640 1 --2- 172.21.15.26:0/3720740307 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f34580a2cf0 0x7f34580a51b0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f3468009fd0 tx=0x7f3468004ab0 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:04.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.610+0000 7f347701d740 1 -- 172.21.15.26:0/3052131019 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560ceb965050 msgr2=0x560ceb973900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:04.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.610+0000 7f347701d740 1 --2- 172.21.15.26:0/3052131019 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560ceb965050 0x560ceb973900 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f346000b760 tx=0x7f346000bc20 comp rx=0 tx=0).stop 2024-09-11T23:36:04.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.610+0000 7f347701d740 1 -- 172.21.15.26:0/3052131019 shutdown_connections 2024-09-11T23:36:04.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.610+0000 7f347701d740 1 --2- 172.21.15.26:0/3052131019 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560ceb973e40 0x560ceb976230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:04.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.610+0000 7f347701d740 1 --2- 172.21.15.26:0/3052131019 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560ceb965050 0x560ceb973900 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:04.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.610+0000 7f347701d740 1 --2- 172.21.15.26:0/3052131019 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560ceb964730 0x560ceb964b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:04.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.610+0000 7f347701d740 1 -- 172.21.15.26:0/3052131019 >> 172.21.15.26:0/3052131019 conn(0x560ceb960de0 msgr2=0x560ceb9611c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:04.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.610+0000 7f347701d740 1 -- 172.21.15.26:0/3052131019 shutdown_connections 2024-09-11T23:36:04.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.610+0000 7f347701d740 1 -- 172.21.15.26:0/3052131019 wait complete. 2024-09-11T23:36:04.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.610+0000 7f347701d740 1 -- start start 2024-09-11T23:36:04.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.610+0000 7f347701d740 1 Processor -- start 2024-09-11T23:36:04.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.611+0000 7f347701d740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560ceb964730 0x560ceb970d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:04.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.611+0000 7f347701d740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560ceb965050 0x560ceb9712b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:04.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.611+0000 7f347701d740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560ceb973e40 0x560ceb9717f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:04.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.611+0000 7f347701d740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x560ceb976a10 con 0x560ceb964730 2024-09-11T23:36:04.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.611+0000 7f347701d740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x560ceb851900 con 0x560ceb965050 2024-09-11T23:36:04.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.611+0000 7f347701d740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x560ceb976b90 con 0x560ceb973e40 2024-09-11T23:36:04.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.611+0000 7f347701d740 10 client.0 _ll_register_callbacks cb 0x560ceb84f780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:36:04.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.611+0000 7f346e59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560ceb964730 0x560ceb970d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:04.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.611+0000 7f346e59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560ceb964730 0x560ceb970d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53800/0 (socket says 172.21.15.26:53800) 2024-09-11T23:36:04.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.611+0000 7f346e59c640 1 -- 172.21.15.26:0/1559361146 learned_addr learned my addr 172.21.15.26:0/1559361146 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:36:04.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.611+0000 7f346dd9b640 1 --2- 172.21.15.26:0/1559361146 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560ceb965050 0x560ceb9712b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:04.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.611+0000 7f346ed9d640 1 --2- 172.21.15.26:0/1559361146 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560ceb973e40 0x560ceb9717f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:04.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.611+0000 7f346e59c640 1 -- 172.21.15.26:0/1559361146 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560ceb973e40 msgr2=0x560ceb9717f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:36:04.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.611+0000 7f346e59c640 1 --2- 172.21.15.26:0/1559361146 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560ceb973e40 0x560ceb9717f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:04.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.611+0000 7f346e59c640 1 -- 172.21.15.26:0/1559361146 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560ceb965050 msgr2=0x560ceb9712b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:36:04.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.611+0000 7f346e59c640 1 --2- 172.21.15.26:0/1559361146 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560ceb965050 0x560ceb9712b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:04.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.611+0000 7f346e59c640 1 -- 172.21.15.26:0/1559361146 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ceb891c50 con 0x560ceb964730 2024-09-11T23:36:04.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.611+0000 7f346dd9b640 1 --2- 172.21.15.26:0/1559361146 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560ceb965050 0x560ceb9712b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:36:04.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.611+0000 7f346ed9d640 1 --2- 172.21.15.26:0/1559361146 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560ceb973e40 0x560ceb9717f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:36:04.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.611+0000 7f346e59c640 1 --2- 172.21.15.26:0/1559361146 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560ceb964730 0x560ceb970d70 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f3460006010 tx=0x7f34600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edd12fa649ddbfb5 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:04.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.611+0000 7f3455ffb640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:36:04.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.612+0000 7f3455ffb640 1 -- 172.21.15.26:0/1559361146 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3460028060 con 0x560ceb964730 2024-09-11T23:36:04.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.612+0000 7f3455ffb640 1 -- 172.21.15.26:0/1559361146 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 8542+0+0 (secure 0 0 0) 0x7f3460011440 con 0x560ceb964730 2024-09-11T23:36:04.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.612+0000 7f347701d740 1 -- 172.21.15.26:0/1559361146 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3460015070 con 0x560ceb964730 2024-09-11T23:36:04.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.612+0000 7f3455ffb640 1 -- 172.21.15.26:0/1559361146 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7f3460011640 con 0x560ceb964730 2024-09-11T23:36:04.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.612+0000 7f347701d740 10 client.17778 fetch_fsmap learned FSMap version 81 2024-09-11T23:36:04.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.612+0000 7f347701d740 1 -- 172.21.15.26:0/1559361146 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=81,monmap=4+}) -- 0x560ceb982750 con 0x560ceb964730 2024-09-11T23:36:04.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.612+0000 7f3455ffb640 1 -- 172.21.15.26:0/1559361146 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3460015070 con 0x560ceb964730 2024-09-11T23:36:04.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.612+0000 7f3455ffb640 1 -- 172.21.15.26:0/1559361146 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 81) ==== 8418+0+0 (secure 0 0 0) 0x7f3460022070 con 0x560ceb964730 2024-09-11T23:36:04.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.612+0000 7f347701d740 10 client.17778 fetch_fsmap finished waiting for FSMap version 81 2024-09-11T23:36:04.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.612+0000 7f347701d740 10 client.17778 resolve_mds: validated gid 24485 aka daemon mds.b 2024-09-11T23:36:04.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.612+0000 7f347701d740 20 client.17778 populate_metadata read hostname 'smithi026' 2024-09-11T23:36:04.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.612+0000 7f347701d740 1 --2- 172.21.15.26:0/1559361146 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x560ceb983c10 0x560ceb986000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:04.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.612+0000 7f347701d740 4 client.17778 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:36:04.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.612+0000 7f347701d740 1 -- 172.21.15.26:0/1559361146 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560ceb8d6890 con 0x560ceb983c10 2024-09-11T23:36:04.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.630+0000 7f346dd9b640 1 --2- 172.21.15.26:0/1559361146 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x560ceb983c10 0x560ceb986000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:04.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.630+0000 7f346dd9b640 1 --2- 172.21.15.26:0/1559361146 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x560ceb983c10 0x560ceb986000 crc :-1 s=READY pgs=272 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5117dbb1a652af96 server_cookie=31dbbec55c24ec08 in_seq=0 out_seq=0 2024-09-11T23:36:04.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.630+0000 7f3455ffb640 10 client.17778 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:36:04.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.631+0000 7f3455ffb640 1 -- 172.21.15.26:0/1559361146 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560ceb8d6890 con 0x560ceb983c10 2024-09-11T23:36:04.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.631+0000 7f3455ffb640 10 client.17778 handle_command_reply: tid=0 2024-09-11T23:36:04.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.631+0000 7f347701d740 1 client.17778 shutdown 2024-09-11T23:36:04.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.632+0000 7f347701d740 1 -- 172.21.15.26:0/1559361146 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560ceb964730 msgr2=0x560ceb970d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:04.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.632+0000 7f347701d740 1 --2- 172.21.15.26:0/1559361146 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560ceb964730 0x560ceb970d70 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f3460006010 tx=0x7f34600046c0 comp rx=0 tx=0).stop 2024-09-11T23:36:04.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.632+0000 7f347701d740 1 -- 172.21.15.26:0/1559361146 shutdown_connections 2024-09-11T23:36:04.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.632+0000 7f347701d740 1 --2- 172.21.15.26:0/1559361146 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560ceb965050 0x560ceb9712b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:04.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.632+0000 7f347701d740 1 --2- 172.21.15.26:0/1559361146 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x560ceb983c10 0x560ceb986000 crc :-1 s=READY pgs=272 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:04.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.632+0000 7f3455ffb640 0 client.17778 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:36:04.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.632+0000 7f347701d740 1 --2- 172.21.15.26:0/1559361146 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560ceb973e40 0x560ceb9717f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:04.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.632+0000 7f347701d740 1 --2- 172.21.15.26:0/1559361146 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560ceb964730 0x560ceb970d70 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:04.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.632+0000 7f347701d740 1 -- 172.21.15.26:0/1559361146 >> 172.21.15.26:0/1559361146 conn(0x560ceb960de0 msgr2=0x560ceb966b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:04.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.632+0000 7f347701d740 1 -- 172.21.15.26:0/1559361146 shutdown_connections 2024-09-11T23:36:04.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.632+0000 7f347701d740 1 -- 172.21.15.26:0/1559361146 wait complete. 2024-09-11T23:36:04.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.632+0000 7f347701d740 20 client.17778 trim_cache size 0 max 16384 2024-09-11T23:36:04.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.649+0000 7f347701d740 1 Processor -- start 2024-09-11T23:36:04.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.649+0000 7f347701d740 1 -- start start 2024-09-11T23:36:04.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.650+0000 7f347701d740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560ceb964730 0x560ceb95bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:04.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.650+0000 7f347701d740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560ceb965050 0x560ceb98b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:04.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.650+0000 7f347701d740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560ceb973e40 0x560ceb98bba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:04.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.650+0000 7f347701d740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f34701ab710 con 0x560ceb965050 2024-09-11T23:36:04.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.650+0000 7f347701d740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f34701ad4f0 con 0x560ceb973e40 2024-09-11T23:36:04.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.650+0000 7f347701d740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f34701abea0 con 0x560ceb964730 2024-09-11T23:36:04.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.650+0000 7f346dd9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560ceb965050 0x560ceb98b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:04.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.650+0000 7f346dd9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560ceb965050 0x560ceb98b660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53816/0 (socket says 172.21.15.26:53816) 2024-09-11T23:36:04.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.650+0000 7f346dd9b640 1 -- 172.21.15.26:0/1782599774 learned_addr learned my addr 172.21.15.26:0/1782599774 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:36:04.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.650+0000 7f346ed9d640 1 --2- 172.21.15.26:0/1782599774 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560ceb973e40 0x560ceb98bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:04.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.650+0000 7f346e59c640 1 --2- 172.21.15.26:0/1782599774 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560ceb964730 0x560ceb95bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:04.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.650+0000 7f346dd9b640 1 -- 172.21.15.26:0/1782599774 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560ceb964730 msgr2=0x560ceb95bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:36:04.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.650+0000 7f346dd9b640 1 --2- 172.21.15.26:0/1782599774 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560ceb964730 0x560ceb95bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:04.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.650+0000 7f346dd9b640 1 -- 172.21.15.26:0/1782599774 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560ceb973e40 msgr2=0x560ceb98bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:36:04.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.650+0000 7f346dd9b640 1 --2- 172.21.15.26:0/1782599774 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560ceb973e40 0x560ceb98bba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:04.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.650+0000 7f346dd9b640 1 -- 172.21.15.26:0/1782599774 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ceb96fd20 con 0x560ceb965050 2024-09-11T23:36:04.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.650+0000 7f346e59c640 1 --2- 172.21.15.26:0/1782599774 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560ceb964730 0x560ceb95bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:36:04.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.650+0000 7f346dd9b640 1 --2- 172.21.15.26:0/1782599774 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560ceb965050 0x560ceb98b660 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f34640bbd10 tx=0x7f34640f3430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85b10f00bfddec52 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:04.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.651+0000 7f3457fff640 1 -- 172.21.15.26:0/1782599774 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3464049040 con 0x560ceb965050 2024-09-11T23:36:04.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.651+0000 7f3457fff640 1 -- 172.21.15.26:0/1782599774 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f34640c4bd0 con 0x560ceb965050 2024-09-11T23:36:04.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.651+0000 7f3457fff640 1 -- 172.21.15.26:0/1782599774 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34640c0310 con 0x560ceb965050 2024-09-11T23:36:04.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.651+0000 7f347701d740 1 -- 172.21.15.26:0/1782599774 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560ceb965050 msgr2=0x560ceb98b660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:04.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.651+0000 7f347701d740 1 --2- 172.21.15.26:0/1782599774 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560ceb965050 0x560ceb98b660 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f34640bbd10 tx=0x7f34640f3430 comp rx=0 tx=0).stop 2024-09-11T23:36:04.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.651+0000 7f347701d740 1 -- 172.21.15.26:0/1782599774 shutdown_connections 2024-09-11T23:36:04.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.651+0000 7f347701d740 1 --2- 172.21.15.26:0/1782599774 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560ceb973e40 0x560ceb98bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:04.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.651+0000 7f347701d740 1 --2- 172.21.15.26:0/1782599774 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560ceb965050 0x560ceb98b660 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:04.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.651+0000 7f347701d740 1 --2- 172.21.15.26:0/1782599774 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560ceb964730 0x560ceb95bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:04.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.651+0000 7f347701d740 1 -- 172.21.15.26:0/1782599774 >> 172.21.15.26:0/1782599774 conn(0x560ceb960de0 msgr2=0x560ceb95fec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:04.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.652+0000 7f347701d740 1 -- 172.21.15.26:0/1782599774 shutdown_connections 2024-09-11T23:36:04.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.652+0000 7f347701d740 1 -- 172.21.15.26:0/1782599774 wait complete. 2024-09-11T23:36:04.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.652+0000 7f347701d740 1 -- start start 2024-09-11T23:36:04.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.652+0000 7f347701d740 1 Processor -- start 2024-09-11T23:36:04.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.652+0000 7f347701d740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560ceb964730 0x560ceb97c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:04.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.652+0000 7f347701d740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560ceb965050 0x560ceb9777e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:04.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.652+0000 7f347701d740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560ceb973e40 0x560ceb977d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:04.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.652+0000 7f347701d740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f34701acd60 con 0x560ceb973e40 2024-09-11T23:36:04.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.652+0000 7f347701d740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f34701ac5d0 con 0x560ceb965050 2024-09-11T23:36:04.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.652+0000 7f347701d740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f34701a9710 con 0x560ceb964730 2024-09-11T23:36:04.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.652+0000 7f347701d740 10 client.0 _ll_register_callbacks cb 0x7f34640f7040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:36:04.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.652+0000 7f346ed9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560ceb973e40 0x560ceb977d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:04.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.653+0000 7f346dd9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560ceb965050 0x560ceb9777e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:04.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.653+0000 7f346ed9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560ceb973e40 0x560ceb977d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53828/0 (socket says 172.21.15.26:53828) 2024-09-11T23:36:04.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.653+0000 7f346ed9d640 1 -- 172.21.15.26:0/668398811 learned_addr learned my addr 172.21.15.26:0/668398811 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:36:04.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.653+0000 7f346e59c640 1 --2- 172.21.15.26:0/668398811 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560ceb964730 0x560ceb97c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:04.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.653+0000 7f346ed9d640 1 -- 172.21.15.26:0/668398811 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560ceb964730 msgr2=0x560ceb97c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:36:04.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.653+0000 7f346ed9d640 1 --2- 172.21.15.26:0/668398811 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560ceb964730 0x560ceb97c4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:04.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.653+0000 7f346ed9d640 1 -- 172.21.15.26:0/668398811 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560ceb965050 msgr2=0x560ceb9777e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:36:04.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.653+0000 7f346ed9d640 1 --2- 172.21.15.26:0/668398811 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560ceb965050 0x560ceb9777e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:04.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.653+0000 7f346ed9d640 1 -- 172.21.15.26:0/668398811 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ceb964e80 con 0x560ceb973e40 2024-09-11T23:36:04.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.653+0000 7f346e59c640 1 --2- 172.21.15.26:0/668398811 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560ceb964730 0x560ceb97c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:36:04.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.653+0000 7f346ed9d640 1 --2- 172.21.15.26:0/668398811 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560ceb973e40 0x560ceb977d20 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f3468003db0 tx=0x7f3468004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=564d2721ed5514f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:04.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.653+0000 7f3455ffb640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:36:04.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.653+0000 7f3455ffb640 1 -- 172.21.15.26:0/668398811 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3468004540 con 0x560ceb973e40 2024-09-11T23:36:04.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.653+0000 7f3455ffb640 1 -- 172.21.15.26:0/668398811 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 8542+0+0 (secure 0 0 0) 0x7f34680046e0 con 0x560ceb973e40 2024-09-11T23:36:04.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.653+0000 7f347701d740 1 -- 172.21.15.26:0/668398811 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f34640c4bd0 con 0x560ceb973e40 2024-09-11T23:36:04.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.653+0000 7f3455ffb640 1 -- 172.21.15.26:0/668398811 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7f3468004b20 con 0x560ceb973e40 2024-09-11T23:36:04.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.654+0000 7f347701d740 10 client.17784 fetch_fsmap learned FSMap version 81 2024-09-11T23:36:04.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.654+0000 7f347701d740 1 -- 172.21.15.26:0/668398811 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=81,monmap=4+}) -- 0x560ceb9ee3a0 con 0x560ceb973e40 2024-09-11T23:36:04.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.654+0000 7f3455ffb640 1 -- 172.21.15.26:0/668398811 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f34640c4bd0 con 0x560ceb973e40 2024-09-11T23:36:04.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.654+0000 7f3455ffb640 1 -- 172.21.15.26:0/668398811 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 81) ==== 8418+0+0 (secure 0 0 0) 0x7f346802c470 con 0x560ceb973e40 2024-09-11T23:36:04.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.654+0000 7f347701d740 10 client.17784 fetch_fsmap finished waiting for FSMap version 81 2024-09-11T23:36:04.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.654+0000 7f347701d740 10 client.17784 resolve_mds: validated gid 24485 aka daemon mds.b 2024-09-11T23:36:04.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.654+0000 7f347701d740 20 client.17784 populate_metadata read hostname 'smithi026' 2024-09-11T23:36:04.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.654+0000 7f347701d740 1 --2- 172.21.15.26:0/668398811 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x560ceb9eec50 0x560ceb9f1070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:04.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.654+0000 7f347701d740 4 client.17784 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 144.25, "await_for": 45.0}] 2024-09-11T23:36:04.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.654+0000 7f347701d740 1 -- 172.21.15.26:0/668398811 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 144.25, "await_for": 45.0}) -- 0x560ceb8d0be0 con 0x560ceb9eec50 2024-09-11T23:36:04.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.654+0000 7f346e59c640 1 --2- 172.21.15.26:0/668398811 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x560ceb9eec50 0x560ceb9f1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:04.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.655+0000 7f346e59c640 1 --2- 172.21.15.26:0/668398811 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x560ceb9eec50 0x560ceb9f1070 crc :-1 s=READY pgs=273 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dbeda9ac3089c372 server_cookie=9d03eed1665aa9a9 in_seq=0 out_seq=0 2024-09-11T23:36:04.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:04.655+0000 7f3455ffb640 10 client.17784 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:36:06.000 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:36:05 smithi026 ceph-mon[31172]: pgmap v556: 97 pgs: 97 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-09-11T23:36:06.238 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:36:05 smithi136 ceph-mon[35121]: pgmap v556: 97 pgs: 97 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-09-11T23:36:06.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:36:05 smithi076 ceph-mon[35296]: pgmap v556: 97 pgs: 97 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-09-11T23:36:08.238 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:36:07 smithi136 ceph-mon[35121]: pgmap v557: 97 pgs: 97 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 213 KiB/s wr, 50 op/s 2024-09-11T23:36:08.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:36:07 smithi026 ceph-mon[31172]: pgmap v557: 97 pgs: 97 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 213 KiB/s wr, 50 op/s 2024-09-11T23:36:08.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:36:07 smithi076 ceph-mon[35296]: pgmap v557: 97 pgs: 97 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 213 KiB/s wr, 50 op/s 2024-09-11T23:36:09.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:09.094+0000 7f3455ffb640 1 -- 172.21.15.26:0/668398811 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x560ceb8d0be0 con 0x560ceb9eec50 2024-09-11T23:36:09.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:09.094+0000 7f3455ffb640 10 client.17784 handle_command_reply: tid=0 2024-09-11T23:36:09.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:09.095+0000 7f347701d740 1 client.17784 shutdown 2024-09-11T23:36:09.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:09.095+0000 7f347701d740 1 -- 172.21.15.26:0/668398811 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560ceb973e40 msgr2=0x560ceb977d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:09.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:09.095+0000 7f347701d740 1 --2- 172.21.15.26:0/668398811 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560ceb973e40 0x560ceb977d20 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f3468003db0 tx=0x7f3468004a40 comp rx=0 tx=0).stop 2024-09-11T23:36:09.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:09.095+0000 7f347701d740 1 -- 172.21.15.26:0/668398811 shutdown_connections 2024-09-11T23:36:09.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:09.095+0000 7f347701d740 1 --2- 172.21.15.26:0/668398811 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560ceb965050 0x560ceb9777e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:09.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:09.096+0000 7f347701d740 1 --2- 172.21.15.26:0/668398811 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x560ceb9eec50 0x560ceb9f1070 crc :-1 s=READY pgs=273 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:09.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:09.096+0000 7f3455ffb640 0 client.17784 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:36:09.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:09.096+0000 7f347701d740 1 --2- 172.21.15.26:0/668398811 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560ceb973e40 0x560ceb977d20 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:09.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:09.096+0000 7f347701d740 1 --2- 172.21.15.26:0/668398811 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560ceb964730 0x560ceb97c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:09.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:09.096+0000 7f347701d740 1 -- 172.21.15.26:0/668398811 >> 172.21.15.26:0/668398811 conn(0x560ceb960de0 msgr2=0x560ceb966240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:09.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:09.096+0000 7f347701d740 1 -- 172.21.15.26:0/668398811 shutdown_connections 2024-09-11T23:36:09.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:09.096+0000 7f347701d740 1 -- 172.21.15.26:0/668398811 wait complete. 2024-09-11T23:36:09.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:09.096+0000 7f347701d740 20 client.17784 trim_cache size 0 max 16384 2024-09-11T23:36:09.099 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-11T23:36:09.099 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 81, 2024-09-11T23:36:09.099 INFO:teuthology.orchestra.run.smithi026.stdout: "leader": 24485, 2024-09-11T23:36:09.099 INFO:teuthology.orchestra.run.smithi026.stdout: "set_version": 62, 2024-09-11T23:36:09.099 INFO:teuthology.orchestra.run.smithi026.stdout: "sets": { 2024-09-11T23:36:09.099 INFO:teuthology.orchestra.run.smithi026.stdout: "cb143a15": { 2024-09-11T23:36:09.099 INFO:teuthology.orchestra.run.smithi026.stdout: "version": 62, 2024-09-11T23:36:09.099 INFO:teuthology.orchestra.run.smithi026.stdout: "age_ref": 0.0, 2024-09-11T23:36:09.099 INFO:teuthology.orchestra.run.smithi026.stdout: "state": { 2024-09-11T23:36:09.099 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "QUIESCED", 2024-09-11T23:36:09.099 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.0 2024-09-11T23:36:09.099 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:36:09.099 INFO:teuthology.orchestra.run.smithi026.stdout: "timeout": 90.0, 2024-09-11T23:36:09.100 INFO:teuthology.orchestra.run.smithi026.stdout: "expiration": 144.2, 2024-09-11T23:36:09.100 INFO:teuthology.orchestra.run.smithi026.stdout: "members": { 2024-09-11T23:36:09.100 INFO:teuthology.orchestra.run.smithi026.stdout: "file:/": { 2024-09-11T23:36:09.100 INFO:teuthology.orchestra.run.smithi026.stdout: "excluded": false, 2024-09-11T23:36:09.100 INFO:teuthology.orchestra.run.smithi026.stdout: "state": { 2024-09-11T23:36:09.100 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "QUIESCED", 2024-09-11T23:36:09.100 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.0 2024-09-11T23:36:09.100 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:36:09.100 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:36:09.100 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:36:09.101 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:36:09.101 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:36:09.101 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-11T23:36:09.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:09.098+0000 7f3474c44640 1 -- 172.21.15.26:0/3720740307 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f34580a2cf0 msgr2=0x7f34580a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:09.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:09.098+0000 7f3474c44640 1 --2- 172.21.15.26:0/3720740307 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f34580a2cf0 0x7f34580a51b0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f3468009fd0 tx=0x7f3468004ab0 comp rx=0 tx=0).stop 2024-09-11T23:36:09.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:09.098+0000 7f3474c44640 1 -- 172.21.15.26:0/3720740307 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3470102790 msgr2=0x7f3470198990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:09.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:09.098+0000 7f3474c44640 1 --2- 172.21.15.26:0/3720740307 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3470102790 0x7f3470198990 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f3464006010 tx=0x7f34640042f0 comp rx=0 tx=0).stop 2024-09-11T23:36:09.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:09.098+0000 7f3474c44640 1 -- 172.21.15.26:0/3720740307 shutdown_connections 2024-09-11T23:36:09.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:09.098+0000 7f3474c44640 1 --2- 172.21.15.26:0/3720740307 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f34580a2cf0 0x7f34580a51b0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:09.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:09.098+0000 7f3474c44640 1 --2- 172.21.15.26:0/3720740307 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3470106f10 0x7f347019cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:09.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:09.098+0000 7f3474c44640 1 --2- 172.21.15.26:0/3720740307 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3470102790 0x7f3470198990 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:09.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:09.098+0000 7f3474c44640 1 --2- 172.21.15.26:0/3720740307 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3470101de0 0x7f3470198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:09.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:09.098+0000 7f3474c44640 1 -- 172.21.15.26:0/3720740307 >> 172.21.15.26:0/3720740307 conn(0x7f34700fd8c0 msgr2=0x7f34700fec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:09.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:09.099+0000 7f3474c44640 1 -- 172.21.15.26:0/3720740307 shutdown_connections 2024-09-11T23:36:09.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:09.099+0000 7f3474c44640 1 -- 172.21.15.26:0/3720740307 wait complete. 2024-09-11T23:36:09.375 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.376+0000 7fc15de82640 1 -- 172.21.15.26:0/857511639 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc1581091f0 msgr2=0x7fc158110250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:09.376 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.376+0000 7fc15de82640 1 --2- 172.21.15.26:0/857511639 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc1581091f0 0x7fc158110250 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7fc148009860 tx=0x7fc14802a380 comp rx=0 tx=0).stop 2024-09-11T23:36:09.379 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.378+0000 7fc15de82640 1 -- 172.21.15.26:0/857511639 shutdown_connections 2024-09-11T23:36:09.379 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.378+0000 7fc15de82640 1 --2- 172.21.15.26:0/857511639 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc1581095d0 0x7fc158109a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:09.379 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.378+0000 7fc15de82640 1 --2- 172.21.15.26:0/857511639 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc15811b140 0x7fc15811d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:09.379 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.378+0000 7fc15de82640 1 --2- 172.21.15.26:0/857511639 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc1581091f0 0x7fc158110250 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:09.379 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.378+0000 7fc15de82640 1 -- 172.21.15.26:0/857511639 >> 172.21.15.26:0/857511639 conn(0x7fc15806ce00 msgr2=0x7fc15806d210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:09.379 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.379+0000 7fc15de82640 1 -- 172.21.15.26:0/857511639 shutdown_connections 2024-09-11T23:36:09.380 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.380+0000 7fc15de82640 1 -- 172.21.15.26:0/857511639 wait complete. 2024-09-11T23:36:09.380 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.380+0000 7fc15de82640 1 Processor -- start 2024-09-11T23:36:09.384 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.384+0000 7fc15de82640 1 -- start start 2024-09-11T23:36:09.384 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.384+0000 7fc15de82640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc1581091f0 0x7fc1581a70f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:09.384 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.384+0000 7fc15de82640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc1581095d0 0x7fc1581a7630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:09.384 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.384+0000 7fc15de82640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc15811b140 0x7fc1581ab9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:09.384 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.384+0000 7fc15de82640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc158120cf0 con 0x7fc1581095d0 2024-09-11T23:36:09.384 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.384+0000 7fc15de82640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fc158120b70 con 0x7fc1581091f0 2024-09-11T23:36:09.384 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.384+0000 7fc15de82640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fc158120e70 con 0x7fc15811b140 2024-09-11T23:36:09.384 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.384+0000 7fc156ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc1581095d0 0x7fc1581a7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:09.384 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.385+0000 7fc157fff640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc15811b140 0x7fc1581ab9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:09.385 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.385+0000 7fc1577fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc1581091f0 0x7fc1581a70f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:09.385 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.385+0000 7fc157fff640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc15811b140 0x7fc1581ab9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:42354/0 (socket says 172.21.15.26:42354) 2024-09-11T23:36:09.385 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.385+0000 7fc156ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc1581095d0 0x7fc1581a7630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53852/0 (socket says 172.21.15.26:53852) 2024-09-11T23:36:09.385 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.385+0000 7fc157fff640 1 -- 172.21.15.26:0/633454341 learned_addr learned my addr 172.21.15.26:0/633454341 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:36:09.385 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.385+0000 7fc157fff640 1 -- 172.21.15.26:0/633454341 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc1581091f0 msgr2=0x7fc1581a70f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:09.386 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.385+0000 7fc157fff640 1 --2- 172.21.15.26:0/633454341 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc1581091f0 0x7fc1581a70f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:09.386 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.385+0000 7fc157fff640 1 -- 172.21.15.26:0/633454341 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc1581095d0 msgr2=0x7fc1581a7630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:09.386 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.385+0000 7fc157fff640 1 --2- 172.21.15.26:0/633454341 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc1581095d0 0x7fc1581a7630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:09.386 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.385+0000 7fc157fff640 1 -- 172.21.15.26:0/633454341 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1581ac0b0 con 0x7fc15811b140 2024-09-11T23:36:09.386 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.385+0000 7fc157fff640 1 --2- 172.21.15.26:0/633454341 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc15811b140 0x7fc1581ab9b0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fc14c00d980 tx=0x7fc14c00de40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:09.386 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.386+0000 7fc154ff9640 1 -- 172.21.15.26:0/633454341 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc14c017030 con 0x7fc15811b140 2024-09-11T23:36:09.386 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.386+0000 7fc15de82640 1 -- 172.21.15.26:0/633454341 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1581ac3a0 con 0x7fc15811b140 2024-09-11T23:36:09.386 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.386+0000 7fc15de82640 1 -- 172.21.15.26:0/633454341 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc158110920 con 0x7fc15811b140 2024-09-11T23:36:09.391 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.389+0000 7fc160c20740 1 Processor -- start 2024-09-11T23:36:09.391 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.389+0000 7fc160c20740 1 -- start start 2024-09-11T23:36:09.391 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.390+0000 7fc160c20740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555c6cc38af0 0x555c6cc38ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:09.391 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.390+0000 7fc160c20740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555c6cc39410 0x555c6cc43770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:09.391 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.390+0000 7fc160c20740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555c6cc43cb0 0x555c6cc460a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:09.391 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.390+0000 7fc160c20740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x555c6cb9d2f0 con 0x555c6cc43cb0 2024-09-11T23:36:09.391 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.390+0000 7fc160c20740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x555c6cb45270 con 0x555c6cc39410 2024-09-11T23:36:09.391 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.390+0000 7fc160c20740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x555c6cb992b0 con 0x555c6cc38af0 2024-09-11T23:36:09.391 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.390+0000 7fc154ff9640 1 -- 172.21.15.26:0/633454341 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc14c004020 con 0x7fc15811b140 2024-09-11T23:36:09.391 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.390+0000 7fc154ff9640 1 -- 172.21.15.26:0/633454341 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc14c003c70 con 0x7fc15811b140 2024-09-11T23:36:09.391 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.391+0000 7fc1577fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555c6cc39410 0x555c6cc43770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:09.392 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.391+0000 7fc1577fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555c6cc39410 0x555c6cc43770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:49876/0 (socket says 172.21.15.26:49876) 2024-09-11T23:36:09.392 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.391+0000 7fc1577fe640 1 -- 172.21.15.26:0/2807830704 learned_addr learned my addr 172.21.15.26:0/2807830704 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:36:09.392 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.391+0000 7fc156ffd640 1 --2- 172.21.15.26:0/2807830704 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555c6cc43cb0 0x555c6cc460a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:09.407 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.403+0000 7fc156ffd640 1 --2- 172.21.15.26:0/2807830704 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555c6cc38af0 0x555c6cc38ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:09.407 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.405+0000 7fc154ff9640 1 -- 172.21.15.26:0/633454341 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fc14c01a070 con 0x7fc15811b140 2024-09-11T23:36:09.407 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.405+0000 7fc1577fe640 1 -- 172.21.15.26:0/2807830704 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555c6cc38af0 msgr2=0x555c6cc38ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:36:09.407 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.405+0000 7fc1577fe640 1 --2- 172.21.15.26:0/2807830704 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555c6cc38af0 0x555c6cc38ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:09.407 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.405+0000 7fc1577fe640 1 -- 172.21.15.26:0/2807830704 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555c6cc43cb0 msgr2=0x555c6cc460a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:36:09.407 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.405+0000 7fc1577fe640 1 --2- 172.21.15.26:0/2807830704 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555c6cc43cb0 0x555c6cc460a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:09.407 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.405+0000 7fc1577fe640 1 -- 172.21.15.26:0/2807830704 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c6cb44340 con 0x555c6cc39410 2024-09-11T23:36:09.407 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.405+0000 7fc1577fe640 1 --2- 172.21.15.26:0/2807830704 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555c6cc39410 0x555c6cc43770 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fc148009830 tx=0x7fc148009540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eaaaafd0ad52469d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:09.408 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.406+0000 7fc154ff9640 1 --2- 172.21.15.26:0/633454341 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fc1400a2ef0 0x7fc1400a53b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:09.410 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.407+0000 7fc154ff9640 1 -- 172.21.15.26:0/633454341 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 8586+0+0 (secure 0 0 0) 0x7fc14c0ceda0 con 0x7fc15811b140 2024-09-11T23:36:09.410 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.407+0000 7fc13dffb640 1 -- 172.21.15.26:0/2807830704 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc148037040 con 0x555c6cc39410 2024-09-11T23:36:09.410 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.407+0000 7fc13dffb640 1 -- 172.21.15.26:0/2807830704 <== mon.2 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc148004040 con 0x555c6cc39410 2024-09-11T23:36:09.410 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.408+0000 7fc160c20740 1 -- 172.21.15.26:0/2807830704 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555c6cc39410 msgr2=0x555c6cc43770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:09.410 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.408+0000 7fc160c20740 1 --2- 172.21.15.26:0/2807830704 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555c6cc39410 0x555c6cc43770 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fc148009830 tx=0x7fc148009540 comp rx=0 tx=0).stop 2024-09-11T23:36:09.411 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.408+0000 7fc160c20740 1 -- 172.21.15.26:0/2807830704 shutdown_connections 2024-09-11T23:36:09.411 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.408+0000 7fc160c20740 1 --2- 172.21.15.26:0/2807830704 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555c6cc39410 0x555c6cc43770 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:09.411 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.408+0000 7fc160c20740 1 --2- 172.21.15.26:0/2807830704 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555c6cc43cb0 0x555c6cc460a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:09.411 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.408+0000 7fc160c20740 1 --2- 172.21.15.26:0/2807830704 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555c6cc38af0 0x555c6cc38ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:09.411 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.408+0000 7fc160c20740 1 -- 172.21.15.26:0/2807830704 >> 172.21.15.26:0/2807830704 conn(0x555c6cc35060 msgr2=0x555c6cc35440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:09.411 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.409+0000 7fc160c20740 1 -- 172.21.15.26:0/2807830704 shutdown_connections 2024-09-11T23:36:09.411 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.411+0000 7fc156ffd640 1 --2- 172.21.15.26:0/633454341 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fc1400a2ef0 0x7fc1400a53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:09.412 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.411+0000 7fc160c20740 1 -- 172.21.15.26:0/2807830704 wait complete. 2024-09-11T23:36:09.412 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.411+0000 7fc160c20740 1 -- start start 2024-09-11T23:36:09.412 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.412+0000 7fc160c20740 1 Processor -- start 2024-09-11T23:36:09.412 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.412+0000 7fc160c20740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555c6cc38af0 0x555c6cc43670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:09.413 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.412+0000 7fc160c20740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555c6cc43cb0 0x555c6cc40910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:09.413 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.412+0000 7fc160c20740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555c6cc40ea0 0x555c6cc412e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:09.413 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.412+0000 7fc160c20740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x555c6cc468d0 con 0x555c6cc40ea0 2024-09-11T23:36:09.413 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.412+0000 7fc160c20740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x555c6cb26290 con 0x555c6cc43cb0 2024-09-11T23:36:09.413 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.412+0000 7fc160c20740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x555c6cc46a50 con 0x555c6cc38af0 2024-09-11T23:36:09.413 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.412+0000 7fc160c20740 10 client.0 _ll_register_callbacks cb 0x555c6cb57140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:36:09.414 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.413+0000 7fc1577fe640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555c6cc38af0 0x555c6cc43670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:09.414 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.413+0000 7fc1577fe640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555c6cc38af0 0x555c6cc43670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:42378/0 (socket says 172.21.15.26:42378) 2024-09-11T23:36:09.414 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.413+0000 7fc1577fe640 1 -- 172.21.15.26:0/1206403879 learned_addr learned my addr 172.21.15.26:0/1206403879 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:36:09.414 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.413+0000 7fc157fff640 1 --2- 172.21.15.26:0/1206403879 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555c6cc40ea0 0x555c6cc412e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:09.414 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.413+0000 7fc1577fe640 1 -- 172.21.15.26:0/1206403879 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555c6cc43cb0 msgr2=0x555c6cc40910 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T23:36:09.414 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.413+0000 7fc1577fe640 1 --2- 172.21.15.26:0/1206403879 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555c6cc43cb0 0x555c6cc40910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:09.414 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.413+0000 7fc1577fe640 1 -- 172.21.15.26:0/1206403879 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555c6cc40ea0 msgr2=0x555c6cc412e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:36:09.414 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.413+0000 7fc1577fe640 1 --2- 172.21.15.26:0/1206403879 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555c6cc40ea0 0x555c6cc412e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:09.414 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.413+0000 7fc1577fe640 1 -- 172.21.15.26:0/1206403879 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c6c8b0210 con 0x555c6cc38af0 2024-09-11T23:36:09.415 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.414+0000 7fc1577fe640 1 --2- 172.21.15.26:0/1206403879 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555c6cc38af0 0x555c6cc43670 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc148009860 tx=0x7fc1480304e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e97b82b9b19c77c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:09.415 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.414+0000 7fc136ffd640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-09-11T23:36:09.415 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.415+0000 7fc136ffd640 1 -- 172.21.15.26:0/1206403879 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc148037040 con 0x555c6cc38af0 2024-09-11T23:36:09.415 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.415+0000 7fc160c20740 1 -- 172.21.15.26:0/1206403879 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x555c6cbaaa20 con 0x555c6cc38af0 2024-09-11T23:36:09.415 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.415+0000 7fc136ffd640 1 -- 172.21.15.26:0/1206403879 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 8586+0+0 (secure 0 0 0) 0x7fc148004030 con 0x555c6cc38af0 2024-09-11T23:36:09.415 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.415+0000 7fc136ffd640 1 -- 172.21.15.26:0/1206403879 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7fc148053050 con 0x555c6cc38af0 2024-09-11T23:36:09.415 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.416+0000 7fc160c20740 10 client.27268 fetch_fsmap learned FSMap version 81 2024-09-11T23:36:09.416 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.416+0000 7fc160c20740 1 -- 172.21.15.26:0/1206403879 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=81,monmap=4+}) -- 0x555c6c97c620 con 0x555c6cc38af0 2024-09-11T23:36:09.416 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.416+0000 7fc156ffd640 1 --2- 172.21.15.26:0/633454341 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fc1400a2ef0 0x7fc1400a53b0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x555c6cc41f50 tx=0x7fc150005bd0 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:09.416 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.416+0000 7fc136ffd640 1 -- 172.21.15.26:0/1206403879 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x555c6cbaaa20 con 0x555c6cc38af0 2024-09-11T23:36:09.416 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.417+0000 7fc136ffd640 1 -- 172.21.15.26:0/1206403879 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 81) ==== 8418+0+0 (secure 0 0 0) 0x7fc14804e030 con 0x555c6cc38af0 2024-09-11T23:36:09.417 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.418+0000 7fc160c20740 10 client.27268 fetch_fsmap finished waiting for FSMap version 81 2024-09-11T23:36:09.417 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.418+0000 7fc160c20740 10 client.27268 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:36:09.417 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.418+0000 7fc160c20740 20 client.27268 populate_metadata read hostname 'smithi026' 2024-09-11T23:36:09.417 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.418+0000 7fc160c20740 1 --2- 172.21.15.26:0/1206403879 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x555c6cc54090 0x555c6cc56510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:09.417 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.418+0000 7fc160c20740 4 client.27268 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:36:09.417 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.418+0000 7fc160c20740 1 -- 172.21.15.26:0/1206403879 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555c6cba4d70 con 0x555c6cc54090 2024-09-11T23:36:09.418 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.419+0000 7fc156ffd640 1 --2- 172.21.15.26:0/1206403879 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x555c6cc54090 0x555c6cc56510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:09.419 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.419+0000 7fc156ffd640 1 --2- 172.21.15.26:0/1206403879 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x555c6cc54090 0x555c6cc56510 crc :-1 s=READY pgs=274 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a45dc11b7ced9a3d server_cookie=6a259ea4ef0fffd2 in_seq=0 out_seq=0 2024-09-11T23:36:09.419 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.419+0000 7fc136ffd640 10 client.27268 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:36:09.419 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.420+0000 7fc136ffd640 1 -- 172.21.15.26:0/1206403879 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555c6cba4d70 con 0x555c6cc54090 2024-09-11T23:36:09.420 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.420+0000 7fc136ffd640 10 client.27268 handle_command_reply: tid=0 2024-09-11T23:36:09.420 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.420+0000 7fc160c20740 1 client.27268 shutdown 2024-09-11T23:36:09.422 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.423+0000 7fc160c20740 1 -- 172.21.15.26:0/1206403879 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555c6cc38af0 msgr2=0x555c6cc43670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:09.422 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.423+0000 7fc160c20740 1 --2- 172.21.15.26:0/1206403879 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555c6cc38af0 0x555c6cc43670 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc148009860 tx=0x7fc1480304e0 comp rx=0 tx=0).stop 2024-09-11T23:36:09.424 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.424+0000 7fc160c20740 1 -- 172.21.15.26:0/1206403879 shutdown_connections 2024-09-11T23:36:09.424 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.424+0000 7fc160c20740 1 --2- 172.21.15.26:0/1206403879 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555c6cc43cb0 0x555c6cc40910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:09.424 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.424+0000 7fc160c20740 1 --2- 172.21.15.26:0/1206403879 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x555c6cc54090 0x555c6cc56510 crc :-1 s=READY pgs=274 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:09.424 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.424+0000 7fc160c20740 1 --2- 172.21.15.26:0/1206403879 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555c6cc40ea0 0x555c6cc412e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:09.424 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.424+0000 7fc160c20740 1 --2- 172.21.15.26:0/1206403879 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555c6cc38af0 0x555c6cc43670 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:09.424 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.424+0000 7fc160c20740 1 -- 172.21.15.26:0/1206403879 >> 172.21.15.26:0/1206403879 conn(0x555c6cc35060 msgr2=0x555c6cc339a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:09.424 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.424+0000 7fc136ffd640 0 client.27268 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:36:09.429 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.429+0000 7fc160c20740 1 -- 172.21.15.26:0/1206403879 shutdown_connections 2024-09-11T23:36:09.429 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.429+0000 7fc160c20740 1 -- 172.21.15.26:0/1206403879 wait complete. 2024-09-11T23:36:09.429 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.430+0000 7fc160c20740 20 client.27268 trim_cache size 0 max 16384 2024-09-11T23:36:09.463 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.463+0000 7fc160c20740 1 Processor -- start 2024-09-11T23:36:09.463 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.464+0000 7fc160c20740 1 -- start start 2024-09-11T23:36:09.463 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.464+0000 7fc160c20740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555c6cc35060 0x555c6cc3f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:09.465 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.464+0000 7fc160c20740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555c6cc38af0 0x555c6cc3f720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:09.465 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.464+0000 7fc1577fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555c6cc35060 0x555c6cc3f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:09.465 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.465+0000 7fc160c20740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555c6cc40ea0 0x555c6cc339a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:09.465 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.465+0000 7fc156ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555c6cc38af0 0x555c6cc3f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:09.465 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.465+0000 7fc156ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555c6cc38af0 0x555c6cc3f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:49882/0 (socket says 172.21.15.26:49882) 2024-09-11T23:36:09.465 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.465+0000 7fc156ffd640 1 -- 172.21.15.26:0/2433270039 learned_addr learned my addr 172.21.15.26:0/2433270039 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:36:09.465 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.465+0000 7fc160c20740 1 -- 172.21.15.26:0/2433270039 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc14400e9c0 con 0x555c6cc35060 2024-09-11T23:36:09.466 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.465+0000 7fc157fff640 1 --2- 172.21.15.26:0/2433270039 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555c6cc40ea0 0x555c6cc339a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:09.466 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.465+0000 7fc160c20740 1 -- 172.21.15.26:0/2433270039 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fc1440107a0 con 0x555c6cc38af0 2024-09-11T23:36:09.466 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.465+0000 7fc160c20740 1 -- 172.21.15.26:0/2433270039 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fc14400f150 con 0x555c6cc40ea0 2024-09-11T23:36:09.466 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.465+0000 7fc156ffd640 1 -- 172.21.15.26:0/2433270039 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555c6cc40ea0 msgr2=0x555c6cc339a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:36:09.466 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.465+0000 7fc156ffd640 1 --2- 172.21.15.26:0/2433270039 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555c6cc40ea0 0x555c6cc339a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:09.466 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.465+0000 7fc156ffd640 1 -- 172.21.15.26:0/2433270039 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555c6cc35060 msgr2=0x555c6cc3f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:36:09.466 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.465+0000 7fc156ffd640 1 --2- 172.21.15.26:0/2433270039 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555c6cc35060 0x555c6cc3f1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:09.466 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.466+0000 7fc156ffd640 1 -- 172.21.15.26:0/2433270039 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c6cc41650 con 0x555c6cc38af0 2024-09-11T23:36:09.466 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.466+0000 7fc156ffd640 1 --2- 172.21.15.26:0/2433270039 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555c6cc38af0 0x555c6cc3f720 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fc15811ca30 tx=0x7fc150004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5d875ba84d091ebe server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:09.467 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.466+0000 7fc13d7fa640 1 -- 172.21.15.26:0/2433270039 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc15002a330 con 0x555c6cc38af0 2024-09-11T23:36:09.467 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.466+0000 7fc13d7fa640 1 -- 172.21.15.26:0/2433270039 <== mon.2 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc15002a9b0 con 0x555c6cc38af0 2024-09-11T23:36:09.467 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.467+0000 7fc160c20740 1 -- 172.21.15.26:0/2433270039 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555c6cc38af0 msgr2=0x555c6cc3f720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:09.467 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.467+0000 7fc160c20740 1 --2- 172.21.15.26:0/2433270039 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555c6cc38af0 0x555c6cc3f720 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fc15811ca30 tx=0x7fc150004a40 comp rx=0 tx=0).stop 2024-09-11T23:36:09.467 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.468+0000 7fc160c20740 1 -- 172.21.15.26:0/2433270039 shutdown_connections 2024-09-11T23:36:09.468 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.468+0000 7fc160c20740 1 --2- 172.21.15.26:0/2433270039 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555c6cc38af0 0x555c6cc3f720 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:09.468 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.468+0000 7fc160c20740 1 --2- 172.21.15.26:0/2433270039 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555c6cc40ea0 0x555c6cc339a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:09.468 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.468+0000 7fc160c20740 1 --2- 172.21.15.26:0/2433270039 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555c6cc35060 0x555c6cc3f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:09.468 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.468+0000 7fc160c20740 1 -- 172.21.15.26:0/2433270039 >> 172.21.15.26:0/2433270039 conn(0x555c6cc437d0 msgr2=0x555c6cc38ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:09.472 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.473+0000 7fc160c20740 1 -- 172.21.15.26:0/2433270039 shutdown_connections 2024-09-11T23:36:09.472 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.473+0000 7fc160c20740 1 -- 172.21.15.26:0/2433270039 wait complete. 2024-09-11T23:36:09.472 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.473+0000 7fc160c20740 1 -- start start 2024-09-11T23:36:09.473 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.474+0000 7fc160c20740 1 Processor -- start 2024-09-11T23:36:09.474 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.474+0000 7fc160c20740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555c6cc35060 0x555c6cc4c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:09.475 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.474+0000 7fc160c20740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555c6cc40ea0 0x555c6cc3db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:09.475 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.475+0000 7fc160c20740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555c6cc43cb0 0x555c6cc3e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:09.475 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.475+0000 7fc1577fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555c6cc35060 0x555c6cc4c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:09.475 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.475+0000 7fc1577fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555c6cc35060 0x555c6cc4c1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:49894/0 (socket says 172.21.15.26:49894) 2024-09-11T23:36:09.475 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.475+0000 7fc1577fe640 1 -- 172.21.15.26:0/3305621800 learned_addr learned my addr 172.21.15.26:0/3305621800 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:36:09.475 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.475+0000 7fc160c20740 1 -- 172.21.15.26:0/3305621800 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc144010010 con 0x555c6cc43cb0 2024-09-11T23:36:09.475 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.475+0000 7fc156ffd640 1 --2- 172.21.15.26:0/3305621800 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555c6cc40ea0 0x555c6cc3db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:09.475 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.475+0000 7fc160c20740 1 -- 172.21.15.26:0/3305621800 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fc14400f880 con 0x555c6cc35060 2024-09-11T23:36:09.475 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.475+0000 7fc160c20740 1 -- 172.21.15.26:0/3305621800 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fc14400c9c0 con 0x555c6cc40ea0 2024-09-11T23:36:09.475 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.475+0000 7fc160c20740 10 client.0 _ll_register_callbacks cb 0x7fc150009db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:36:09.480 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.481+0000 7fc157fff640 1 --2- 172.21.15.26:0/3305621800 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555c6cc43cb0 0x555c6cc3e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:09.481 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.481+0000 7fc156ffd640 1 -- 172.21.15.26:0/3305621800 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555c6cc35060 msgr2=0x555c6cc4c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:36:09.481 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.481+0000 7fc156ffd640 1 --2- 172.21.15.26:0/3305621800 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555c6cc35060 0x555c6cc4c1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:09.481 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.481+0000 7fc156ffd640 1 -- 172.21.15.26:0/3305621800 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555c6cc43cb0 msgr2=0x555c6cc3e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:36:09.481 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.481+0000 7fc156ffd640 1 --2- 172.21.15.26:0/3305621800 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555c6cc43cb0 0x555c6cc3e0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:09.481 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.481+0000 7fc156ffd640 1 -- 172.21.15.26:0/3305621800 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c6cc3e610 con 0x555c6cc40ea0 2024-09-11T23:36:09.482 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.482+0000 7fc156ffd640 1 --2- 172.21.15.26:0/3305621800 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555c6cc40ea0 0x555c6cc3db90 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc150009ac0 tx=0x7fc150022650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=36486c74c06ff3cd server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:09.482 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.482+0000 7fc136ffd640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-09-11T23:36:09.482 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.482+0000 7fc136ffd640 1 -- 172.21.15.26:0/3305621800 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc150038020 con 0x555c6cc40ea0 2024-09-11T23:36:09.482 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.482+0000 7fc160c20740 1 -- 172.21.15.26:0/3305621800 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x555c6cb21900 con 0x555c6cc40ea0 2024-09-11T23:36:09.482 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.482+0000 7fc136ffd640 1 -- 172.21.15.26:0/3305621800 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 8586+0+0 (secure 0 0 0) 0x7fc150034650 con 0x555c6cc40ea0 2024-09-11T23:36:09.482 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.483+0000 7fc136ffd640 1 -- 172.21.15.26:0/3305621800 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7fc150043050 con 0x555c6cc40ea0 2024-09-11T23:36:09.483 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.484+0000 7fc160c20740 10 client.27274 fetch_fsmap learned FSMap version 81 2024-09-11T23:36:09.483 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.484+0000 7fc160c20740 1 -- 172.21.15.26:0/3305621800 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=81,monmap=4+}) -- 0x555c6cc44e80 con 0x555c6cc40ea0 2024-09-11T23:36:09.486 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.486+0000 7fc136ffd640 1 -- 172.21.15.26:0/3305621800 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x555c6cb21900 con 0x555c6cc40ea0 2024-09-11T23:36:09.486 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.486+0000 7fc136ffd640 1 -- 172.21.15.26:0/3305621800 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 81) ==== 8418+0+0 (secure 0 0 0) 0x7fc15000f040 con 0x555c6cc40ea0 2024-09-11T23:36:09.486 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.487+0000 7fc160c20740 10 client.27274 fetch_fsmap finished waiting for FSMap version 81 2024-09-11T23:36:09.486 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.487+0000 7fc160c20740 10 client.27274 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:36:09.486 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.487+0000 7fc160c20740 20 client.27274 populate_metadata read hostname 'smithi026' 2024-09-11T23:36:09.486 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.487+0000 7fc160c20740 1 --2- 172.21.15.26:0/3305621800 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x555c6cc45650 0x555c6ccc0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:09.486 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.487+0000 7fc160c20740 4 client.27274 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T23:36:09.487 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.487+0000 7fc160c20740 1 -- 172.21.15.26:0/3305621800 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555c6cb6e820 con 0x555c6cc45650 2024-09-11T23:36:09.489 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.489+0000 7fc1577fe640 1 --2- 172.21.15.26:0/3305621800 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x555c6cc45650 0x555c6ccc0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:09.491 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.492+0000 7fc1577fe640 1 --2- 172.21.15.26:0/3305621800 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x555c6cc45650 0x555c6ccc0dc0 crc :-1 s=READY pgs=275 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd6e50a9b52d1ea4 server_cookie=4a6d0e16d1bce8f1 in_seq=0 out_seq=0 2024-09-11T23:36:09.492 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.493+0000 7fc136ffd640 10 client.27274 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:36:09.768 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.768+0000 7fc136ffd640 1 -- 172.21.15.26:0/3305621800 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555c6cb6e820 con 0x555c6cc45650 2024-09-11T23:36:09.768 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.768+0000 7fc136ffd640 10 client.27274 handle_command_reply: tid=0 2024-09-11T23:36:09.768 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.768+0000 7fc160c20740 1 client.27274 shutdown 2024-09-11T23:36:09.768 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.768+0000 7fc160c20740 1 -- 172.21.15.26:0/3305621800 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555c6cc40ea0 msgr2=0x555c6cc3db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:09.768 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.768+0000 7fc160c20740 1 --2- 172.21.15.26:0/3305621800 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555c6cc40ea0 0x555c6cc3db90 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc150009ac0 tx=0x7fc150022650 comp rx=0 tx=0).stop 2024-09-11T23:36:09.768 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.768+0000 7fc160c20740 1 -- 172.21.15.26:0/3305621800 shutdown_connections 2024-09-11T23:36:09.768 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.768+0000 7fc160c20740 1 --2- 172.21.15.26:0/3305621800 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x555c6cc45650 0x555c6ccc0dc0 crc :-1 s=READY pgs=275 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:09.769 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.768+0000 7fc160c20740 1 --2- 172.21.15.26:0/3305621800 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555c6cc43cb0 0x555c6cc3e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:09.769 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.768+0000 7fc160c20740 1 --2- 172.21.15.26:0/3305621800 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555c6cc40ea0 0x555c6cc3db90 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:09.769 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.768+0000 7fc160c20740 1 --2- 172.21.15.26:0/3305621800 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555c6cc35060 0x555c6cc4c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:09.769 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.768+0000 7fc160c20740 1 -- 172.21.15.26:0/3305621800 >> 172.21.15.26:0/3305621800 conn(0x555c6cc437d0 msgr2=0x555c6cc3f110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:09.769 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.768+0000 7fc136ffd640 0 client.27274 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:36:09.770 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.769+0000 7fc160c20740 1 -- 172.21.15.26:0/3305621800 shutdown_connections 2024-09-11T23:36:09.770 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.769+0000 7fc160c20740 1 -- 172.21.15.26:0/3305621800 wait complete. 2024-09-11T23:36:09.770 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.769+0000 7fc160c20740 20 client.27274 trim_cache size 0 max 16384 2024-09-11T23:36:09.770 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-09-11T23:36:09.770 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "message": "", 2024-09-11T23:36:09.770 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-09-11T23:36:09.770 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-09-11T23:36:09.772 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.770+0000 7fc15de82640 1 -- 172.21.15.26:0/633454341 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fc1400a2ef0 msgr2=0x7fc1400a53b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:09.772 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.770+0000 7fc15de82640 1 --2- 172.21.15.26:0/633454341 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fc1400a2ef0 0x7fc1400a53b0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x555c6cc41f50 tx=0x7fc150005bd0 comp rx=0 tx=0).stop 2024-09-11T23:36:09.772 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.770+0000 7fc15de82640 1 -- 172.21.15.26:0/633454341 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc15811b140 msgr2=0x7fc1581ab9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:09.772 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.770+0000 7fc15de82640 1 --2- 172.21.15.26:0/633454341 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc15811b140 0x7fc1581ab9b0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fc14c00d980 tx=0x7fc14c00de40 comp rx=0 tx=0).stop 2024-09-11T23:36:09.772 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.770+0000 7fc15de82640 1 -- 172.21.15.26:0/633454341 shutdown_connections 2024-09-11T23:36:09.772 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.770+0000 7fc15de82640 1 --2- 172.21.15.26:0/633454341 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fc1400a2ef0 0x7fc1400a53b0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:09.772 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.770+0000 7fc15de82640 1 --2- 172.21.15.26:0/633454341 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc15811b140 0x7fc1581ab9b0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:09.772 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.770+0000 7fc15de82640 1 --2- 172.21.15.26:0/633454341 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc1581095d0 0x7fc1581a7630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:09.773 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.770+0000 7fc15de82640 1 --2- 172.21.15.26:0/633454341 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc1581091f0 0x7fc1581a70f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:09.773 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.770+0000 7fc15de82640 1 -- 172.21.15.26:0/633454341 >> 172.21.15.26:0/633454341 conn(0x7fc15806ce00 msgr2=0x7fc15811b880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:09.773 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.771+0000 7fc15de82640 1 -- 172.21.15.26:0/633454341 shutdown_connections 2024-09-11T23:36:09.773 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:09.771+0000 7fc15de82640 1 -- 172.21.15.26:0/633454341 wait complete. 2024-09-11T23:36:09.927 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'cb143a15', quiesce took 8.0 seconds. Will release after: 16.25 2024-09-11T23:36:10.238 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:36:09 smithi136 ceph-mon[35121]: pgmap v558: 97 pgs: 97 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 213 KiB/s wr, 50 op/s 2024-09-11T23:36:10.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:36:09 smithi136 ceph-mon[35121]: osdmap e109: 12 total, 12 up, 12 in 2024-09-11T23:36:10.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:36:09 smithi026 ceph-mon[31172]: pgmap v558: 97 pgs: 97 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 213 KiB/s wr, 50 op/s 2024-09-11T23:36:10.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:36:09 smithi026 ceph-mon[31172]: osdmap e109: 12 total, 12 up, 12 in 2024-09-11T23:36:10.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:36:09 smithi076 ceph-mon[35296]: pgmap v558: 97 pgs: 97 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 213 KiB/s wr, 50 op/s 2024-09-11T23:36:10.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:36:09 smithi076 ceph-mon[35296]: osdmap e109: 12 total, 12 up, 12 in 2024-09-11T23:36:12.238 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:36:11 smithi136 ceph-mon[35121]: pgmap v560: 97 pgs: 17 active+clean+snaptrim, 28 active+clean+snaptrim_wait, 52 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 383 KiB/s rd, 566 KiB/s wr, 110 op/s 2024-09-11T23:36:12.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:36:11 smithi026 ceph-mon[31172]: pgmap v560: 97 pgs: 17 active+clean+snaptrim, 28 active+clean+snaptrim_wait, 52 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 383 KiB/s rd, 566 KiB/s wr, 110 op/s 2024-09-11T23:36:12.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:36:11 smithi076 ceph-mon[35296]: pgmap v560: 97 pgs: 17 active+clean+snaptrim, 28 active+clean+snaptrim_wait, 52 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 383 KiB/s rd, 566 KiB/s wr, 110 op/s 2024-09-11T23:36:14.238 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:36:13 smithi136 ceph-mon[35121]: pgmap v561: 97 pgs: 21 active+clean+snaptrim, 39 active+clean+snaptrim_wait, 37 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 370 KiB/s rd, 548 KiB/s wr, 118 op/s 2024-09-11T23:36:14.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:36:13 smithi026 ceph-mon[31172]: pgmap v561: 97 pgs: 21 active+clean+snaptrim, 39 active+clean+snaptrim_wait, 37 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 370 KiB/s rd, 548 KiB/s wr, 118 op/s 2024-09-11T23:36:14.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:36:13 smithi076 ceph-mon[35296]: pgmap v561: 97 pgs: 21 active+clean+snaptrim, 39 active+clean+snaptrim_wait, 37 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 370 KiB/s rd, 548 KiB/s wr, 118 op/s 2024-09-11T23:36:16.241 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:36:15 smithi136 ceph-mon[35121]: pgmap v562: 97 pgs: 23 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 33 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 540 KiB/s rd, 584 KiB/s wr, 142 op/s 2024-09-11T23:36:16.254 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:36:15 smithi026 ceph-mon[31172]: pgmap v562: 97 pgs: 23 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 33 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 540 KiB/s rd, 584 KiB/s wr, 142 op/s 2024-09-11T23:36:16.305 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:36:15 smithi076 ceph-mon[35296]: pgmap v562: 97 pgs: 23 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 33 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 540 KiB/s rd, 584 KiB/s wr, 142 op/s 2024-09-11T23:36:18.238 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:36:17 smithi136 ceph-mon[35121]: pgmap v563: 97 pgs: 23 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 33 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 442 KiB/s rd, 379 KiB/s wr, 110 op/s 2024-09-11T23:36:18.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:36:17 smithi026 ceph-mon[31172]: pgmap v563: 97 pgs: 23 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 33 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 442 KiB/s rd, 379 KiB/s wr, 110 op/s 2024-09-11T23:36:18.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:36:17 smithi076 ceph-mon[35296]: pgmap v563: 97 pgs: 23 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 33 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 442 KiB/s rd, 379 KiB/s wr, 110 op/s 2024-09-11T23:36:20.238 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:36:19 smithi136 ceph-mon[35121]: pgmap v564: 97 pgs: 23 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 33 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 442 KiB/s rd, 379 KiB/s wr, 110 op/s 2024-09-11T23:36:20.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:36:19 smithi136 ceph-mon[35121]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:36:20.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:36:19 smithi026 ceph-mon[31172]: pgmap v564: 97 pgs: 23 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 33 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 442 KiB/s rd, 379 KiB/s wr, 110 op/s 2024-09-11T23:36:20.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:36:19 smithi026 ceph-mon[31172]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:36:20.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:36:19 smithi076 ceph-mon[35296]: pgmap v564: 97 pgs: 23 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 33 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 442 KiB/s rd, 379 KiB/s wr, 110 op/s 2024-09-11T23:36:20.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:36:19 smithi076 ceph-mon[35296]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:36:22.238 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:36:21 smithi136 ceph-mon[35121]: pgmap v565: 97 pgs: 23 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 33 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 382 KiB/s rd, 325 KiB/s wr, 96 op/s 2024-09-11T23:36:22.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:36:21 smithi026 ceph-mon[31172]: pgmap v565: 97 pgs: 23 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 33 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 382 KiB/s rd, 325 KiB/s wr, 96 op/s 2024-09-11T23:36:22.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:36:21 smithi076 ceph-mon[35296]: pgmap v565: 97 pgs: 23 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 33 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 382 KiB/s rd, 325 KiB/s wr, 96 op/s 2024-09-11T23:36:22.929 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:22.930+0000 7f292259f640 1 -- 172.21.15.26:0/1501921727 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f291c1091f0 msgr2=0x7f291c110250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:22.929 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:22.930+0000 7f292259f640 1 --2- 172.21.15.26:0/1501921727 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f291c1091f0 0x7f291c110250 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f291401acb0 tx=0x7f291403b920 comp rx=0 tx=0).stop 2024-09-11T23:36:22.931 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:22.932+0000 7f292259f640 1 -- 172.21.15.26:0/1501921727 shutdown_connections 2024-09-11T23:36:22.932 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:22.932+0000 7f292259f640 1 --2- 172.21.15.26:0/1501921727 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f291c1095d0 0x7f291c109a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:22.932 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:22.932+0000 7f292259f640 1 --2- 172.21.15.26:0/1501921727 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f291c11b140 0x7f291c11d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:22.932 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:22.932+0000 7f292259f640 1 --2- 172.21.15.26:0/1501921727 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f291c1091f0 0x7f291c110250 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:22.932 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:22.932+0000 7f292259f640 1 -- 172.21.15.26:0/1501921727 >> 172.21.15.26:0/1501921727 conn(0x7f291c06cbf0 msgr2=0x7f291c06d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:22.935 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:22.936+0000 7f292259f640 1 -- 172.21.15.26:0/1501921727 shutdown_connections 2024-09-11T23:36:22.939 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:22.939+0000 7f292259f640 1 -- 172.21.15.26:0/1501921727 wait complete. 2024-09-11T23:36:22.939 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:22.939+0000 7f292259f640 1 Processor -- start 2024-09-11T23:36:22.940 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:22.940+0000 7f292259f640 1 -- start start 2024-09-11T23:36:22.940 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:22.940+0000 7f292259f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f291c1091f0 0x7f291c11aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:22.940 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:22.940+0000 7f292259f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f291c1095d0 0x7f291c116b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:22.940 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:22.940+0000 7f292259f640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f291c11b140 0x7f291c117070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:22.940 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:22.940+0000 7f292259f640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f291c113400 con 0x7f291c1095d0 2024-09-11T23:36:22.940 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:22.940+0000 7f292259f640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f291c113280 con 0x7f291c1091f0 2024-09-11T23:36:22.941 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:22.940+0000 7f292259f640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f291c113580 con 0x7f291c11b140 2024-09-11T23:36:22.941 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:22.940+0000 7f292159d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f291c1091f0 0x7f291c11aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:22.941 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:22.940+0000 7f292159d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f291c1091f0 0x7f291c11aa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:41262/0 (socket says 172.21.15.26:41262) 2024-09-11T23:36:22.941 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:22.940+0000 7f292159d640 1 -- 172.21.15.26:0/950614986 learned_addr learned my addr 172.21.15.26:0/950614986 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:36:22.942 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:22.943+0000 7f2920d9c640 1 --2- 172.21.15.26:0/950614986 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f291c1095d0 0x7f291c116b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:22.942 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:22.943+0000 7f292159d640 1 -- 172.21.15.26:0/950614986 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f291c11b140 msgr2=0x7f291c117070 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-11T23:36:22.943 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:22.943+0000 7f292159d640 1 --2- 172.21.15.26:0/950614986 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f291c11b140 0x7f291c117070 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:22.943 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:22.943+0000 7f292159d640 1 -- 172.21.15.26:0/950614986 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f291c1095d0 msgr2=0x7f291c116b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:22.943 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:22.943+0000 7f292159d640 1 --2- 172.21.15.26:0/950614986 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f291c1095d0 0x7f291c116b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:22.943 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:22.943+0000 7f292159d640 1 -- 172.21.15.26:0/950614986 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f291c117930 con 0x7f291c1091f0 2024-09-11T23:36:22.946 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:22.946+0000 7f2920d9c640 1 --2- 172.21.15.26:0/950614986 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f291c1095d0 0x7f291c116b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:36:22.949 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:22.950+0000 7f292159d640 1 --2- 172.21.15.26:0/950614986 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f291c1091f0 0x7f291c11aa50 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f291403bf10 tx=0x7f2914041a60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:22.952 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:22.952+0000 7f29127fc640 1 -- 172.21.15.26:0/950614986 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2914045030 con 0x7f291c1091f0 2024-09-11T23:36:22.952 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:22.953+0000 7f292259f640 1 -- 172.21.15.26:0/950614986 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f291c1aaf50 con 0x7f291c1091f0 2024-09-11T23:36:22.952 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:22.953+0000 7f292259f640 1 -- 172.21.15.26:0/950614986 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f291c1ab410 con 0x7f291c1091f0 2024-09-11T23:36:22.953 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:22.953+0000 7f29127fc640 1 -- 172.21.15.26:0/950614986 <== mon.2 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f291404f070 con 0x7f291c1091f0 2024-09-11T23:36:22.953 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:22.953+0000 7f29127fc640 1 -- 172.21.15.26:0/950614986 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29140047a0 con 0x7f291c1091f0 2024-09-11T23:36:22.959 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:22.959+0000 7f29127fc640 1 -- 172.21.15.26:0/950614986 <== mon.2 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f2914048020 con 0x7f291c1091f0 2024-09-11T23:36:22.960 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:22.961+0000 7f29127fc640 1 --2- 172.21.15.26:0/950614986 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f29080a2f00 0x7f29080a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:22.960 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:22.961+0000 7f29127fc640 1 -- 172.21.15.26:0/950614986 <== mon.2 v2:172.21.15.76:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 8586+0+0 (secure 0 0 0) 0x7f2914101320 con 0x7f291c1091f0 2024-09-11T23:36:22.961 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:22.962+0000 7f2926675740 1 Processor -- start 2024-09-11T23:36:22.961 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:22.962+0000 7f2926675740 1 -- start start 2024-09-11T23:36:22.962 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:22.962+0000 7f2926675740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555847e6bcc0 0x555847e6c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:22.962 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:22.962+0000 7f2926675740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555847e6c5e0 0x555847e76940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:22.962 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:22.962+0000 7f2926675740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555847e76e80 0x555847e79270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:22.962 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:22.962+0000 7f2926675740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x555847dd02f0 con 0x555847e6bcc0 2024-09-11T23:36:22.962 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:22.963+0000 7f2926675740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x555847d78270 con 0x555847e76e80 2024-09-11T23:36:22.962 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:22.963+0000 7f2926675740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x555847dcc2b0 con 0x555847e6c5e0 2024-09-11T23:36:22.965 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:22.963+0000 7f2921d9e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555847e76e80 0x555847e79270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:22.965 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:22.963+0000 7f2921d9e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555847e76e80 0x555847e79270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:41278/0 (socket says 172.21.15.26:41278) 2024-09-11T23:36:22.965 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:22.963+0000 7f2921d9e640 1 -- 172.21.15.26:0/4105190555 learned_addr learned my addr 172.21.15.26:0/4105190555 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:36:22.965 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:22.964+0000 7f292159d640 1 --2- 172.21.15.26:0/4105190555 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555847e6bcc0 0x555847e6c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:22.965 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:22.964+0000 7f2921d9e640 1 -- 172.21.15.26:0/4105190555 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555847e6c5e0 msgr2=0x555847e76940 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T23:36:22.965 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:22.964+0000 7f2921d9e640 1 --2- 172.21.15.26:0/4105190555 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555847e6c5e0 0x555847e76940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:22.966 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:22.964+0000 7f2921d9e640 1 -- 172.21.15.26:0/4105190555 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555847e6bcc0 msgr2=0x555847e6c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:36:22.966 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:22.964+0000 7f2921d9e640 1 --2- 172.21.15.26:0/4105190555 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555847e6bcc0 0x555847e6c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:22.966 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:22.964+0000 7f2921d9e640 1 -- 172.21.15.26:0/4105190555 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555847d77340 con 0x555847e76e80 2024-09-11T23:36:22.966 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:22.965+0000 7f2921d9e640 1 --2- 172.21.15.26:0/4105190555 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555847e76e80 0x555847e79270 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f290c009670 tx=0x7f290c009b30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dbf7c4ecdf67a5a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:22.966 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:22.965+0000 7f2920d9c640 1 --2- 172.21.15.26:0/950614986 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f29080a2f00 0x7f29080a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:22.967 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:22.967+0000 7f29077fe640 1 -- 172.21.15.26:0/4105190555 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f290c015020 con 0x555847e76e80 2024-09-11T23:36:22.967 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:22.967+0000 7f29077fe640 1 -- 172.21.15.26:0/4105190555 <== mon.2 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f290c00bda0 con 0x555847e76e80 2024-09-11T23:36:22.967 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:22.967+0000 7f29077fe640 1 -- 172.21.15.26:0/4105190555 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f290c010380 con 0x555847e76e80 2024-09-11T23:36:22.967 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:22.968+0000 7f2926675740 1 -- 172.21.15.26:0/4105190555 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555847e76e80 msgr2=0x555847e79270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:22.968 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:22.968+0000 7f2926675740 1 --2- 172.21.15.26:0/4105190555 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555847e76e80 0x555847e79270 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f290c009670 tx=0x7f290c009b30 comp rx=0 tx=0).stop 2024-09-11T23:36:22.968 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:22.969+0000 7f2926675740 1 -- 172.21.15.26:0/4105190555 shutdown_connections 2024-09-11T23:36:22.968 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:22.969+0000 7f2926675740 1 --2- 172.21.15.26:0/4105190555 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555847e76e80 0x555847e79270 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:22.969 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:22.969+0000 7f2926675740 1 --2- 172.21.15.26:0/4105190555 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555847e6c5e0 0x555847e76940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:22.969 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:22.969+0000 7f2926675740 1 --2- 172.21.15.26:0/4105190555 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555847e6bcc0 0x555847e6c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:22.969 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:22.969+0000 7f2926675740 1 -- 172.21.15.26:0/4105190555 >> 172.21.15.26:0/4105190555 conn(0x555847e68230 msgr2=0x555847e68610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:22.969 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:22.969+0000 7f2920d9c640 1 --2- 172.21.15.26:0/950614986 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f29080a2f00 0x7f29080a53c0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x555847e69090 tx=0x7f2918008040 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:22.971 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:22.971+0000 7f2926675740 1 -- 172.21.15.26:0/4105190555 shutdown_connections 2024-09-11T23:36:22.972 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:22.973+0000 7f2926675740 1 -- 172.21.15.26:0/4105190555 wait complete. 2024-09-11T23:36:22.973 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:22.974+0000 7f2926675740 1 -- start start 2024-09-11T23:36:22.973 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:22.974+0000 7f2926675740 1 Processor -- start 2024-09-11T23:36:22.976 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:22.976+0000 7f2926675740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555847e6bcc0 0x555847e73d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:22.976 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:22.976+0000 7f2926675740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555847e6c5e0 0x555847e742a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:22.976 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:22.976+0000 7f2926675740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555847e76e80 0x555847e747e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:22.976 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:22.976+0000 7f2926675740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x555847e79a50 con 0x555847e6bcc0 2024-09-11T23:36:22.976 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:22.976+0000 7f2926675740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x555847d59290 con 0x555847e76e80 2024-09-11T23:36:22.976 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:22.976+0000 7f2926675740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x555847e79bd0 con 0x555847e6c5e0 2024-09-11T23:36:22.976 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:22.976+0000 7f2926675740 10 client.0 _ll_register_callbacks cb 0x555847d8a140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:36:22.977 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:22.977+0000 7f2920d9c640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555847e6c5e0 0x555847e742a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:22.977 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:22.977+0000 7f2920d9c640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555847e6c5e0 0x555847e742a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:45326/0 (socket says 172.21.15.26:45326) 2024-09-11T23:36:22.978 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:22.977+0000 7f2920d9c640 1 -- 172.21.15.26:0/1393077886 learned_addr learned my addr 172.21.15.26:0/1393077886 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:36:22.978 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:22.977+0000 7f2921d9e640 1 --2- 172.21.15.26:0/1393077886 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555847e76e80 0x555847e747e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:22.978 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:22.977+0000 7f292159d640 1 --2- 172.21.15.26:0/1393077886 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555847e6bcc0 0x555847e73d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:22.978 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:22.977+0000 7f2920d9c640 1 -- 172.21.15.26:0/1393077886 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555847e76e80 msgr2=0x555847e747e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:36:22.978 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:22.977+0000 7f2920d9c640 1 --2- 172.21.15.26:0/1393077886 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555847e76e80 0x555847e747e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:22.978 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:22.977+0000 7f2920d9c640 1 -- 172.21.15.26:0/1393077886 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555847e6bcc0 msgr2=0x555847e73d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:36:22.978 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:22.977+0000 7f2920d9c640 1 --2- 172.21.15.26:0/1393077886 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555847e6bcc0 0x555847e73d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:22.978 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:22.977+0000 7f2920d9c640 1 -- 172.21.15.26:0/1393077886 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555847ae3210 con 0x555847e6c5e0 2024-09-11T23:36:22.978 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:22.978+0000 7f2920d9c640 1 --2- 172.21.15.26:0/1393077886 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555847e6c5e0 0x555847e742a0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f2918006f30 tx=0x7f2918004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c551abcdd4892cca server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:22.978 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:22.978+0000 7f2904ff9640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-09-11T23:36:22.978 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:22.978+0000 7f2904ff9640 1 -- 172.21.15.26:0/1393077886 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2918023060 con 0x555847e6c5e0 2024-09-11T23:36:22.979 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:22.978+0000 7f2926675740 1 -- 172.21.15.26:0/1393077886 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f290c00bda0 con 0x555847e6c5e0 2024-09-11T23:36:22.979 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:22.979+0000 7f2904ff9640 1 -- 172.21.15.26:0/1393077886 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 8586+0+0 (secure 0 0 0) 0x7f29180117e0 con 0x555847e6c5e0 2024-09-11T23:36:22.979 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:22.979+0000 7f2904ff9640 1 -- 172.21.15.26:0/1393077886 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f2918013e30 con 0x555847e6c5e0 2024-09-11T23:36:22.980 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:22.981+0000 7f2926675740 10 client.27277 fetch_fsmap learned FSMap version 82 2024-09-11T23:36:22.980 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:22.981+0000 7f2926675740 1 -- 172.21.15.26:0/1393077886 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=82,monmap=4+}) -- 0x555847baf620 con 0x555847e6c5e0 2024-09-11T23:36:22.982 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:22.983+0000 7f2904ff9640 1 -- 172.21.15.26:0/1393077886 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f290c00bda0 con 0x555847e6c5e0 2024-09-11T23:36:22.983 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:22.983+0000 7f2904ff9640 1 -- 172.21.15.26:0/1393077886 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 82) ==== 8414+0+0 (secure 0 0 0) 0x7f2918026cd0 con 0x555847e6c5e0 2024-09-11T23:36:22.984 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:22.984+0000 7f2926675740 10 client.27277 fetch_fsmap finished waiting for FSMap version 82 2024-09-11T23:36:22.984 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:22.984+0000 7f2926675740 10 client.27277 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:36:22.984 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:22.984+0000 7f2926675740 20 client.27277 populate_metadata read hostname 'smithi026' 2024-09-11T23:36:22.984 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:22.984+0000 7f2926675740 1 --2- 172.21.15.26:0/1393077886 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x555847e87430 0x555847e89870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:22.984 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:22.984+0000 7f2926675740 4 client.27277 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:36:22.984 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:22.984+0000 7f2926675740 1 -- 172.21.15.26:0/1393077886 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555847ddda20 con 0x555847e87430 2024-09-11T23:36:22.984 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:22.985+0000 7f292159d640 1 --2- 172.21.15.26:0/1393077886 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x555847e87430 0x555847e89870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:22.985 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:22.985+0000 7f292159d640 1 --2- 172.21.15.26:0/1393077886 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x555847e87430 0x555847e89870 crc :-1 s=READY pgs=276 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d13cb3d48bb00e server_cookie=97a97fbd935f4c63 in_seq=0 out_seq=0 2024-09-11T23:36:22.985 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:22.985+0000 7f2904ff9640 10 client.27277 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:36:22.987 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:22.986+0000 7f2904ff9640 1 -- 172.21.15.26:0/1393077886 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555847ddda20 con 0x555847e87430 2024-09-11T23:36:22.987 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:22.987+0000 7f2904ff9640 10 client.27277 handle_command_reply: tid=0 2024-09-11T23:36:22.987 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:22.987+0000 7f2926675740 1 client.27277 shutdown 2024-09-11T23:36:22.988 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:22.988+0000 7f2926675740 1 -- 172.21.15.26:0/1393077886 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555847e6c5e0 msgr2=0x555847e742a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:22.988 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:22.988+0000 7f2926675740 1 --2- 172.21.15.26:0/1393077886 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555847e6c5e0 0x555847e742a0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f2918006f30 tx=0x7f2918004750 comp rx=0 tx=0).stop 2024-09-11T23:36:22.989 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:22.988+0000 7f2926675740 1 -- 172.21.15.26:0/1393077886 shutdown_connections 2024-09-11T23:36:22.989 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:22.988+0000 7f2926675740 1 --2- 172.21.15.26:0/1393077886 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555847e76e80 0x555847e747e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:22.989 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:22.988+0000 7f2926675740 1 --2- 172.21.15.26:0/1393077886 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x555847e87430 0x555847e89870 crc :-1 s=READY pgs=276 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:22.989 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:22.988+0000 7f2926675740 1 --2- 172.21.15.26:0/1393077886 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555847e6c5e0 0x555847e742a0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:22.989 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:22.988+0000 7f2926675740 1 --2- 172.21.15.26:0/1393077886 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555847e6bcc0 0x555847e73d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:22.989 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:22.988+0000 7f2926675740 1 -- 172.21.15.26:0/1393077886 >> 172.21.15.26:0/1393077886 conn(0x555847e68230 msgr2=0x555847e6df90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:22.992 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:22.989+0000 7f2904ff9640 0 client.27277 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:36:22.993 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:22.989+0000 7f2926675740 1 -- 172.21.15.26:0/1393077886 shutdown_connections 2024-09-11T23:36:22.993 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:22.989+0000 7f2926675740 1 -- 172.21.15.26:0/1393077886 wait complete. 2024-09-11T23:36:22.993 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:22.989+0000 7f2926675740 20 client.27277 trim_cache size 0 max 16384 2024-09-11T23:36:23.003 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:36:22 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: 2024-09-11T23:36:22.551+0000 7f7ccf119640 -1 calc_pg_upmaps abort due to max <= 0 2024-09-11T23:36:23.019 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:23.019+0000 7f2926675740 1 Processor -- start 2024-09-11T23:36:23.019 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:23.020+0000 7f2926675740 1 -- start start 2024-09-11T23:36:23.020 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:23.020+0000 7f2926675740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555847e6bcc0 0x555847e6de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:23.021 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:23.020+0000 7f2926675740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555847e6c5e0 0x555847e74610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:23.021 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:23.020+0000 7f292159d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555847e6bcc0 0x555847e6de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:23.021 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:23.020+0000 7f292159d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555847e6bcc0 0x555847e6de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:41300/0 (socket says 172.21.15.26:41300) 2024-09-11T23:36:23.021 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:23.021+0000 7f292159d640 1 -- 172.21.15.26:0/3495247359 learned_addr learned my addr 172.21.15.26:0/3495247359 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:36:23.021 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:23.021+0000 7f2926675740 1 --2- 172.21.15.26:0/3495247359 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555847e76e80 0x555847e8f140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:23.021 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:23.021+0000 7f2920d9c640 1 --2- 172.21.15.26:0/3495247359 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555847e6c5e0 0x555847e74610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:23.021 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:23.021+0000 7f2926675740 1 -- 172.21.15.26:0/3495247359 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f290000e870 con 0x555847e76e80 2024-09-11T23:36:23.021 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:23.021+0000 7f2921d9e640 1 --2- 172.21.15.26:0/3495247359 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555847e76e80 0x555847e8f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:23.021 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:23.021+0000 7f2926675740 1 -- 172.21.15.26:0/3495247359 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f2900010620 con 0x555847e6bcc0 2024-09-11T23:36:23.021 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:23.021+0000 7f2926675740 1 -- 172.21.15.26:0/3495247359 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f290000f000 con 0x555847e6c5e0 2024-09-11T23:36:23.022 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:23.022+0000 7f292159d640 1 -- 172.21.15.26:0/3495247359 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555847e6c5e0 msgr2=0x555847e74610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:36:23.022 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:23.022+0000 7f292159d640 1 --2- 172.21.15.26:0/3495247359 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555847e6c5e0 0x555847e74610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:23.022 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:23.022+0000 7f292159d640 1 -- 172.21.15.26:0/3495247359 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555847e76e80 msgr2=0x555847e8f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:36:23.022 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:23.022+0000 7f292159d640 1 --2- 172.21.15.26:0/3495247359 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555847e76e80 0x555847e8f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:23.022 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:23.022+0000 7f292159d640 1 -- 172.21.15.26:0/3495247359 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555847e6c410 con 0x555847e6bcc0 2024-09-11T23:36:23.022 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:23.022+0000 7f292159d640 1 --2- 172.21.15.26:0/3495247359 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555847e6bcc0 0x555847e6de80 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f29140ca280 tx=0x7f291401a5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=670f205087ea3aba server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:23.022 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:23.022+0000 7f2906ffd640 1 -- 172.21.15.26:0/3495247359 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29140d1c60 con 0x555847e6bcc0 2024-09-11T23:36:23.023 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:23.023+0000 7f2906ffd640 1 -- 172.21.15.26:0/3495247359 <== mon.2 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f29140d6520 con 0x555847e6bcc0 2024-09-11T23:36:23.024 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:23.023+0000 7f2906ffd640 1 -- 172.21.15.26:0/3495247359 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29140fec50 con 0x555847e6bcc0 2024-09-11T23:36:23.024 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:23.024+0000 7f2926675740 1 -- 172.21.15.26:0/3495247359 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555847e6bcc0 msgr2=0x555847e6de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:23.024 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:23.024+0000 7f2926675740 1 --2- 172.21.15.26:0/3495247359 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555847e6bcc0 0x555847e6de80 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f29140ca280 tx=0x7f291401a5d0 comp rx=0 tx=0).stop 2024-09-11T23:36:23.024 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:23.024+0000 7f2926675740 1 -- 172.21.15.26:0/3495247359 shutdown_connections 2024-09-11T23:36:23.024 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:23.024+0000 7f2926675740 1 --2- 172.21.15.26:0/3495247359 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555847e76e80 0x555847e8f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:23.024 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:23.024+0000 7f2926675740 1 --2- 172.21.15.26:0/3495247359 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555847e6c5e0 0x555847e74610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:23.024 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:23.024+0000 7f2926675740 1 --2- 172.21.15.26:0/3495247359 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555847e6bcc0 0x555847e6de80 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:23.024 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:23.024+0000 7f2926675740 1 -- 172.21.15.26:0/3495247359 >> 172.21.15.26:0/3495247359 conn(0x555847e68230 msgr2=0x555847e6d770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:23.025 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:23.025+0000 7f2926675740 1 -- 172.21.15.26:0/3495247359 shutdown_connections 2024-09-11T23:36:23.025 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:23.025+0000 7f2926675740 1 -- 172.21.15.26:0/3495247359 wait complete. 2024-09-11T23:36:23.025 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:23.025+0000 7f2926675740 1 -- start start 2024-09-11T23:36:23.025 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:23.025+0000 7f2926675740 1 Processor -- start 2024-09-11T23:36:23.026 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:23.026+0000 7f2926675740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555847e6bcc0 0x555847eea480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:23.026 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:23.026+0000 7f2926675740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555847e6c5e0 0x555847eea9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:23.026 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:23.026+0000 7f2926675740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555847e76e80 0x555847e7a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:23.026 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:23.026+0000 7f2926675740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f290000fe90 con 0x555847e6bcc0 2024-09-11T23:36:23.026 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:23.026+0000 7f2926675740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f290000f700 con 0x555847e6c5e0 2024-09-11T23:36:23.026 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:23.026+0000 7f2926675740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f290000c870 con 0x555847e76e80 2024-09-11T23:36:23.026 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:23.026+0000 7f2926675740 10 client.0 _ll_register_callbacks cb 0x7f291800a4e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:36:23.026 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:23.026+0000 7f2920d9c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555847e6c5e0 0x555847eea9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:23.026 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:23.026+0000 7f2920d9c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555847e6c5e0 0x555847eea9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:41306/0 (socket says 172.21.15.26:41306) 2024-09-11T23:36:23.026 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:23.026+0000 7f2920d9c640 1 -- 172.21.15.26:0/3087474528 learned_addr learned my addr 172.21.15.26:0/3087474528 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:36:23.027 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:23.027+0000 7f2921d9e640 1 --2- 172.21.15.26:0/3087474528 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555847e76e80 0x555847e7a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:23.029 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:23.029+0000 7f2920d9c640 1 -- 172.21.15.26:0/3087474528 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555847e76e80 msgr2=0x555847e7a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:36:23.029 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:23.029+0000 7f2920d9c640 1 --2- 172.21.15.26:0/3087474528 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555847e76e80 0x555847e7a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:23.029 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:23.029+0000 7f2920d9c640 1 -- 172.21.15.26:0/3087474528 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555847e6bcc0 msgr2=0x555847eea480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:36:23.029 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:23.029+0000 7f2920d9c640 1 --2- 172.21.15.26:0/3087474528 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555847e6bcc0 0x555847eea480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:23.029 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:23.029+0000 7f2920d9c640 1 -- 172.21.15.26:0/3087474528 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555847e7ac10 con 0x555847e6c5e0 2024-09-11T23:36:23.030 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:23.030+0000 7f2920d9c640 1 --2- 172.21.15.26:0/3087474528 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555847e6c5e0 0x555847eea9c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f2918013c80 tx=0x7f2918013db0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=78c4deeda6fdfeec server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:23.030 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:23.030+0000 7f2904ff9640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-11T23:36:23.030 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:23.030+0000 7f2904ff9640 1 -- 172.21.15.26:0/3087474528 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2918034030 con 0x555847e6c5e0 2024-09-11T23:36:23.030 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:23.030+0000 7f2904ff9640 1 -- 172.21.15.26:0/3087474528 <== mon.2 v2:172.21.15.76:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 8586+0+0 (secure 0 0 0) 0x7f29180115e0 con 0x555847e6c5e0 2024-09-11T23:36:23.030 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:23.030+0000 7f2926675740 1 -- 172.21.15.26:0/3087474528 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f29140d6520 con 0x555847e6c5e0 2024-09-11T23:36:23.031 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:23.031+0000 7f2904ff9640 1 -- 172.21.15.26:0/3087474528 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f291801de70 con 0x555847e6c5e0 2024-09-11T23:36:23.034 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:23.035+0000 7f2926675740 10 client.27320 fetch_fsmap learned FSMap version 82 2024-09-11T23:36:23.034 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:23.035+0000 7f2926675740 1 -- 172.21.15.26:0/3087474528 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=82,monmap=4+}) -- 0x555847ef1cd0 con 0x555847e6c5e0 2024-09-11T23:36:23.039 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:23.039+0000 7f2904ff9640 1 -- 172.21.15.26:0/3087474528 <== mon.2 v2:172.21.15.76:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f29140d6520 con 0x555847e6c5e0 2024-09-11T23:36:23.039 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:23.039+0000 7f2904ff9640 1 -- 172.21.15.26:0/3087474528 <== mon.2 v2:172.21.15.76:3300/0 5 ==== fsmap(e 82) ==== 8414+0+0 (secure 0 0 0) 0x7f291802fa40 con 0x555847e6c5e0 2024-09-11T23:36:23.040 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:23.039+0000 7f2926675740 10 client.27320 fetch_fsmap finished waiting for FSMap version 82 2024-09-11T23:36:23.040 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:23.039+0000 7f2926675740 10 client.27320 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:36:23.040 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:23.039+0000 7f2926675740 20 client.27320 populate_metadata read hostname 'smithi026' 2024-09-11T23:36:23.040 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:23.039+0000 7f2926675740 1 --2- 172.21.15.26:0/3087474528 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x555847ef2540 0x555847ef4960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:23.040 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:23.039+0000 7f2926675740 4 client.27320 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T23:36:23.040 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:23.039+0000 7f2926675740 1 -- 172.21.15.26:0/3087474528 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555847dd7d70 con 0x555847ef2540 2024-09-11T23:36:23.043 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:23.044+0000 7f292159d640 1 --2- 172.21.15.26:0/3087474528 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x555847ef2540 0x555847ef4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:23.052 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:23.052+0000 7f292159d640 1 --2- 172.21.15.26:0/3087474528 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x555847ef2540 0x555847ef4960 crc :-1 s=READY pgs=277 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=71297f22720f23cb server_cookie=9612b7877555c926 in_seq=0 out_seq=0 2024-09-11T23:36:23.052 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:23.053+0000 7f2904ff9640 10 client.27320 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:36:23.116 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:23.117+0000 7f2904ff9640 1 -- 172.21.15.26:0/3087474528 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555847dd7d70 con 0x555847ef2540 2024-09-11T23:36:23.116 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:23.117+0000 7f2904ff9640 10 client.27320 handle_command_reply: tid=0 2024-09-11T23:36:23.116 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:23.117+0000 7f2926675740 1 client.27320 shutdown 2024-09-11T23:36:23.118 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:23.119+0000 7f2926675740 1 -- 172.21.15.26:0/3087474528 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555847e6c5e0 msgr2=0x555847eea9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:23.118 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:23.119+0000 7f2926675740 1 --2- 172.21.15.26:0/3087474528 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555847e6c5e0 0x555847eea9c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f2918013c80 tx=0x7f2918013db0 comp rx=0 tx=0).stop 2024-09-11T23:36:23.119 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:23.119+0000 7f2926675740 1 -- 172.21.15.26:0/3087474528 shutdown_connections 2024-09-11T23:36:23.119 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:23.119+0000 7f2926675740 1 --2- 172.21.15.26:0/3087474528 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555847e6c5e0 0x555847eea9c0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:23.119 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:23.119+0000 7f2926675740 1 --2- 172.21.15.26:0/3087474528 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x555847ef2540 0x555847ef4960 crc :-1 s=READY pgs=277 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:23.119 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:23.119+0000 7f2926675740 1 --2- 172.21.15.26:0/3087474528 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555847e76e80 0x555847e7a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:23.119 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:23.119+0000 7f2926675740 1 --2- 172.21.15.26:0/3087474528 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555847e6bcc0 0x555847eea480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:23.120 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:23.119+0000 7f2926675740 1 -- 172.21.15.26:0/3087474528 >> 172.21.15.26:0/3087474528 conn(0x555847e68230 msgr2=0x555847e6d770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:23.120 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:23.119+0000 7f2904ff9640 0 client.27320 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:36:23.120 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:23.119+0000 7f2926675740 1 -- 172.21.15.26:0/3087474528 shutdown_connections 2024-09-11T23:36:23.120 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:23.119+0000 7f2926675740 1 -- 172.21.15.26:0/3087474528 wait complete. 2024-09-11T23:36:23.121 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:23.120+0000 7f2926675740 20 client.27320 trim_cache size 0 max 16384 2024-09-11T23:36:23.121 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-09-11T23:36:23.121 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "message": "", 2024-09-11T23:36:23.121 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-09-11T23:36:23.121 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-09-11T23:36:23.122 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:23.120+0000 7f292259f640 1 -- 172.21.15.26:0/950614986 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f29080a2f00 msgr2=0x7f29080a53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:23.122 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:23.120+0000 7f292259f640 1 --2- 172.21.15.26:0/950614986 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f29080a2f00 0x7f29080a53c0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x555847e69090 tx=0x7f2918008040 comp rx=0 tx=0).stop 2024-09-11T23:36:23.122 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:23.120+0000 7f292259f640 1 -- 172.21.15.26:0/950614986 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f291c1091f0 msgr2=0x7f291c11aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:23.122 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:23.120+0000 7f292259f640 1 --2- 172.21.15.26:0/950614986 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f291c1091f0 0x7f291c11aa50 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f291403bf10 tx=0x7f2914041a60 comp rx=0 tx=0).stop 2024-09-11T23:36:23.122 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:23.121+0000 7f292259f640 1 -- 172.21.15.26:0/950614986 shutdown_connections 2024-09-11T23:36:23.122 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:23.121+0000 7f292259f640 1 --2- 172.21.15.26:0/950614986 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f29080a2f00 0x7f29080a53c0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:23.122 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:23.121+0000 7f292259f640 1 --2- 172.21.15.26:0/950614986 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f291c11b140 0x7f291c117070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:23.123 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:23.121+0000 7f292259f640 1 --2- 172.21.15.26:0/950614986 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f291c1095d0 0x7f291c116b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:23.123 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:23.121+0000 7f292259f640 1 --2- 172.21.15.26:0/950614986 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f291c1091f0 0x7f291c11aa50 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:23.123 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:23.121+0000 7f292259f640 1 -- 172.21.15.26:0/950614986 >> 172.21.15.26:0/950614986 conn(0x7f291c06cbf0 msgr2=0x7f291c0707e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:23.123 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:23.121+0000 7f292259f640 1 -- 172.21.15.26:0/950614986 shutdown_connections 2024-09-11T23:36:23.123 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:23.121+0000 7f292259f640 1 -- 172.21.15.26:0/950614986 wait complete. 2024-09-11T23:36:24.238 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:36:23 smithi136 ceph-mon[35121]: pgmap v566: 97 pgs: 23 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 33 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 32 KiB/s wr, 43 op/s 2024-09-11T23:36:24.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:36:23 smithi026 ceph-mon[31172]: pgmap v566: 97 pgs: 23 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 33 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 32 KiB/s wr, 43 op/s 2024-09-11T23:36:24.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:36:23 smithi076 ceph-mon[35296]: pgmap v566: 97 pgs: 23 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 33 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 32 KiB/s wr, 43 op/s 2024-09-11T23:36:26.177 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:36:26.221 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:36:25 smithi026 ceph-mon[31172]: pgmap v567: 97 pgs: 22 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 34 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 36 KiB/s wr, 36 op/s 2024-09-11T23:36:26.238 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:36:25 smithi136 ceph-mon[35121]: pgmap v567: 97 pgs: 22 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 34 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 36 KiB/s wr, 36 op/s 2024-09-11T23:36:26.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:36:25 smithi076 ceph-mon[35296]: pgmap v567: 97 pgs: 22 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 34 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 36 KiB/s wr, 36 op/s 2024-09-11T23:36:26.681 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:36:27.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:27.336+0000 7f7507585640 1 -- 172.21.15.26:0/3866028437 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7500100aa0 msgr2=0x7f750010ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:27.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:27.336+0000 7f7507585640 1 --2- 172.21.15.26:0/3866028437 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7500100aa0 0x7f750010ea20 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f74f8009ca0 tx=0x7f74f802a8f0 comp rx=0 tx=0).stop 2024-09-11T23:36:27.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:27.337+0000 7f7507585640 1 -- 172.21.15.26:0/3866028437 shutdown_connections 2024-09-11T23:36:27.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:27.337+0000 7f7507585640 1 --2- 172.21.15.26:0/3866028437 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7500100100 0x7f7500100560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:27.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:27.337+0000 7f7507585640 1 --2- 172.21.15.26:0/3866028437 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7500100aa0 0x7f750010ea20 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:27.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:27.337+0000 7f7507585640 1 --2- 172.21.15.26:0/3866028437 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7500106b80 0x7f7500106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:27.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:27.337+0000 7f7507585640 1 -- 172.21.15.26:0/3866028437 >> 172.21.15.26:0/3866028437 conn(0x7f7500077230 msgr2=0x7f75000fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:27.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:27.339+0000 7f7507585640 1 -- 172.21.15.26:0/3866028437 shutdown_connections 2024-09-11T23:36:27.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:27.339+0000 7f7507585640 1 -- 172.21.15.26:0/3866028437 wait complete. 2024-09-11T23:36:27.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:27.340+0000 7f7507585640 1 Processor -- start 2024-09-11T23:36:27.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:27.340+0000 7f7507585640 1 -- start start 2024-09-11T23:36:27.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:27.341+0000 7f7507585640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7500100100 0x7f75001985d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:27.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:27.341+0000 7f7507585640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7500100aa0 0x7f7500198b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:27.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:27.341+0000 7f7505321640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7500100100 0x7f75001985d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:27.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:27.341+0000 7f7505321640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7500100100 0x7f75001985d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:45358/0 (socket says 172.21.15.26:45358) 2024-09-11T23:36:27.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:27.341+0000 7f7505321640 1 -- 172.21.15.26:0/1475926548 learned_addr learned my addr 172.21.15.26:0/1475926548 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:36:27.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:27.341+0000 7f7507585640 1 --2- 172.21.15.26:0/1475926548 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7500106b80 0x7f750019cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:27.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:27.342+0000 7f7504b20640 1 --2- 172.21.15.26:0/1475926548 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7500100aa0 0x7f7500198b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:27.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:27.342+0000 7f7507585640 1 -- 172.21.15.26:0/1475926548 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f7500112130 con 0x7f7500106b80 2024-09-11T23:36:27.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:27.342+0000 7f7507585640 1 -- 172.21.15.26:0/1475926548 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f7500111fb0 con 0x7f7500100aa0 2024-09-11T23:36:27.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:27.342+0000 7f7507585640 1 -- 172.21.15.26:0/1475926548 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f75001122b0 con 0x7f7500100100 2024-09-11T23:36:27.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:27.343+0000 7f7505b22640 1 --2- 172.21.15.26:0/1475926548 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7500106b80 0x7f750019cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:27.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:27.343+0000 7f7505321640 1 -- 172.21.15.26:0/1475926548 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7500100aa0 msgr2=0x7f7500198b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:27.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:27.343+0000 7f7505321640 1 --2- 172.21.15.26:0/1475926548 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7500100aa0 0x7f7500198b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:27.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:27.343+0000 7f7505321640 1 -- 172.21.15.26:0/1475926548 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7500106b80 msgr2=0x7f750019cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:27.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:27.343+0000 7f7505321640 1 --2- 172.21.15.26:0/1475926548 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7500106b80 0x7f750019cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:27.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:27.343+0000 7f7505321640 1 -- 172.21.15.26:0/1475926548 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f750019d580 con 0x7f7500100100 2024-09-11T23:36:27.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:27.344+0000 7f7505321640 1 --2- 172.21.15.26:0/1475926548 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7500100100 0x7f75001985d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f74f000b450 tx=0x7f74f000b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:27.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:27.348+0000 7f74f67fc640 1 -- 172.21.15.26:0/1475926548 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74f001d070 con 0x7f7500100100 2024-09-11T23:36:27.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:27.348+0000 7f74f67fc640 1 -- 172.21.15.26:0/1475926548 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f74f0018440 con 0x7f7500100100 2024-09-11T23:36:27.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:27.348+0000 7f74f67fc640 1 -- 172.21.15.26:0/1475926548 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74f00165a0 con 0x7f7500100100 2024-09-11T23:36:27.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:27.349+0000 7f7507585640 1 -- 172.21.15.26:0/1475926548 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f750019d870 con 0x7f7500100100 2024-09-11T23:36:27.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:27.349+0000 7f7507585640 1 -- 172.21.15.26:0/1475926548 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7500101210 con 0x7f7500100100 2024-09-11T23:36:27.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:27.352+0000 7f74ebfff640 1 -- 172.21.15.26:0/1475926548 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7500100560 con 0x7f7500100100 2024-09-11T23:36:27.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:27.357+0000 7f74f67fc640 1 -- 172.21.15.26:0/1475926548 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f74f0020070 con 0x7f7500100100 2024-09-11T23:36:27.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:27.358+0000 7f74f67fc640 1 --2- 172.21.15.26:0/1475926548 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f74ec0a2e80 0x7f74ec0a5340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:27.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:27.358+0000 7f74f67fc640 1 -- 172.21.15.26:0/1475926548 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 8586+0+0 (secure 0 0 0) 0x7f74f00d2520 con 0x7f7500100100 2024-09-11T23:36:27.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:27.358+0000 7f7504b20640 1 --2- 172.21.15.26:0/1475926548 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f74ec0a2e80 0x7f74ec0a5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:27.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:27.359+0000 7f7504b20640 1 --2- 172.21.15.26:0/1475926548 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f74ec0a2e80 0x7f74ec0a5340 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f74fc006fd0 tx=0x7f74fc008040 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:27.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:27.366+0000 7f74f67fc640 1 -- 172.21.15.26:0/1475926548 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f74f009f210 con 0x7f7500100100 2024-09-11T23:36:27.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:27.783+0000 7f74ebfff640 1 -- 172.21.15.26:0/1475926548 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7500062630 con 0x7f7500100100 2024-09-11T23:36:27.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:27.786+0000 7f74f67fc640 1 -- 172.21.15.26:0/1475926548 <== mon.1 v2:172.21.15.136:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) ==== 94+0+11994 (secure 0 0 0) 0x7f74f009eb10 con 0x7f7500100100 2024-09-11T23:36:27.786 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-11T23:36:27.786 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":82,"btime":"2024-09-11T23:36:18:900758+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14499,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/367659515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":367659515},{"type":"v1","addr":"172.21.15.26:6835","nonce":367659515}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14511,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/2041065299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":2041065299},{"type":"v1","addr":"172.21.15.26:6837","nonce":2041065299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":82,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:16:26.544203+0000","modified":"2024-09-11T23:36:17.914679+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24485,"mds_1":24463,"mds_2":24458,"mds_3":24448,"mds_4":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14550":{"gid":14550,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6839/2233068305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":2233068305},{"type":"v1","addr":"172.21.15.26:6839","nonce":2233068305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14562":{"gid":14562,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6841/2984982336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":2984982336},{"type":"v1","addr":"172.21.15.26:6841","nonce":2984982336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6833/2019841455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":2019841455},{"type":"v1","addr":"172.21.15.136:6833","nonce":2019841455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.76:6833/3438147050","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3438147050},{"type":"v1","addr":"172.21.15.76:6833","nonce":3438147050}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6835/3617665975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3617665975},{"type":"v1","addr":"172.21.15.136:6835","nonce":3617665975}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":12,"addr":"172.21.15.76:6835/393308499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":393308499},{"type":"v1","addr":"172.21.15.76:6835","nonce":393308499}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":10,"addr":"172.21.15.136:6837/1633082387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":1633082387},{"type":"v1","addr":"172.21.15.136:6837","nonce":1633082387}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.76:6837/2078978748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":2078978748},{"type":"v1","addr":"172.21.15.76:6837","nonce":2078978748}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":63,"addr":"172.21.15.136:6839/3211398630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":3211398630},{"type":"v1","addr":"172.21.15.136:6839","nonce":3211398630}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.76:6839/2077637090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6838","nonce":2077637090},{"type":"v1","addr":"172.21.15.76:6839","nonce":2077637090}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24485,"qdb_cluster":[24485,24463,24458,24448,24443]},"id":1}]} 2024-09-11T23:36:27.786 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 82 2024-09-11T23:36:27.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:27.791+0000 7f74ebfff640 1 -- 172.21.15.26:0/1475926548 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f74ec0a2e80 msgr2=0x7f74ec0a5340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:27.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:27.791+0000 7f74ebfff640 1 --2- 172.21.15.26:0/1475926548 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f74ec0a2e80 0x7f74ec0a5340 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f74fc006fd0 tx=0x7f74fc008040 comp rx=0 tx=0).stop 2024-09-11T23:36:27.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:27.791+0000 7f74ebfff640 1 -- 172.21.15.26:0/1475926548 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7500100100 msgr2=0x7f75001985d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:27.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:27.791+0000 7f74ebfff640 1 --2- 172.21.15.26:0/1475926548 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7500100100 0x7f75001985d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f74f000b450 tx=0x7f74f000b910 comp rx=0 tx=0).stop 2024-09-11T23:36:27.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:27.792+0000 7f74ebfff640 1 -- 172.21.15.26:0/1475926548 shutdown_connections 2024-09-11T23:36:27.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:27.792+0000 7f74ebfff640 1 --2- 172.21.15.26:0/1475926548 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f74ec0a2e80 0x7f74ec0a5340 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:27.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:27.792+0000 7f74ebfff640 1 --2- 172.21.15.26:0/1475926548 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7500100aa0 0x7f7500198b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:27.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:27.792+0000 7f74ebfff640 1 --2- 172.21.15.26:0/1475926548 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7500106b80 0x7f750019cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:27.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:27.792+0000 7f74ebfff640 1 --2- 172.21.15.26:0/1475926548 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7500100100 0x7f75001985d0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:27.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:27.792+0000 7f74ebfff640 1 -- 172.21.15.26:0/1475926548 >> 172.21.15.26:0/1475926548 conn(0x7f7500077230 msgr2=0x7f7500104ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:27.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:27.795+0000 7f74ebfff640 1 -- 172.21.15.26:0/1475926548 shutdown_connections 2024-09-11T23:36:27.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:27.796+0000 7f74ebfff640 1 -- 172.21.15.26:0/1475926548 wait complete. 2024-09-11T23:36:28.141 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:36:27 smithi026 ceph-mon[31172]: pgmap v568: 97 pgs: 22 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 34 active+clean; 2.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 6.9 KiB/s wr, 18 op/s 2024-09-11T23:36:28.141 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:36:27 smithi026 ceph-mon[31172]: from='client.? 172.21.15.26:0/1475926548' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:36:28.238 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:36:27 smithi136 ceph-mon[35121]: pgmap v568: 97 pgs: 22 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 34 active+clean; 2.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 6.9 KiB/s wr, 18 op/s 2024-09-11T23:36:28.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:36:27 smithi136 ceph-mon[35121]: from='client.? 172.21.15.26:0/1475926548' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:36:28.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:36:27 smithi076 ceph-mon[35296]: pgmap v568: 97 pgs: 22 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 34 active+clean; 2.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 6.9 KiB/s wr, 18 op/s 2024-09-11T23:36:28.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:36:27 smithi076 ceph-mon[35296]: from='client.? 172.21.15.26:0/1475926548' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:36:29.004 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24485 quiesce db --set-id cb143a15 --release --await' 2024-09-11T23:36:29.004 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24485 quiesce db --set-id cb143a15 --release --await 2024-09-11T23:36:29.550 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:36:30.198 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:36:29 smithi026 ceph-mon[31172]: pgmap v569: 97 pgs: 22 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 34 active+clean; 2.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 7.2 KiB/s wr, 11 op/s 2024-09-11T23:36:30.238 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:36:29 smithi136 ceph-mon[35121]: pgmap v569: 97 pgs: 22 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 34 active+clean; 2.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 7.2 KiB/s wr, 11 op/s 2024-09-11T23:36:30.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.251+0000 7f12509d8640 1 -- 172.21.15.26:0/1446282227 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f124c074eb0 msgr2=0x7f124c112490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:30.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.251+0000 7f12509d8640 1 --2- 172.21.15.26:0/1446282227 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f124c074eb0 0x7f124c112490 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f1240009900 tx=0x7f124002a4d0 comp rx=0 tx=0).stop 2024-09-11T23:36:30.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.251+0000 7f12509d8640 1 -- 172.21.15.26:0/1446282227 shutdown_connections 2024-09-11T23:36:30.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.251+0000 7f12509d8640 1 --2- 172.21.15.26:0/1446282227 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f124c1129d0 0x7f124c114dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:30.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.251+0000 7f12509d8640 1 --2- 172.21.15.26:0/1446282227 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f124c074eb0 0x7f124c112490 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:30.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.251+0000 7f12509d8640 1 --2- 172.21.15.26:0/1446282227 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f124c074590 0x7f124c074970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:30.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.251+0000 7f12509d8640 1 -- 172.21.15.26:0/1446282227 >> 172.21.15.26:0/1446282227 conn(0x7f124c0fd740 msgr2=0x7f124c0ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:30.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.252+0000 7f12509d8640 1 -- 172.21.15.26:0/1446282227 shutdown_connections 2024-09-11T23:36:30.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.252+0000 7f12509d8640 1 -- 172.21.15.26:0/1446282227 wait complete. 2024-09-11T23:36:30.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.253+0000 7f12509d8640 1 Processor -- start 2024-09-11T23:36:30.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.253+0000 7f12509d8640 1 -- start start 2024-09-11T23:36:30.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.253+0000 7f12509d8640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f124c074590 0x7f124c111fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:30.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.253+0000 7f12509d8640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f124c074eb0 0x7f124c10b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:30.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.253+0000 7f12509d8640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f124c1129d0 0x7f124c10b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:30.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.253+0000 7f12509d8640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f124c118420 con 0x7f124c074eb0 2024-09-11T23:36:30.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.254+0000 7f12509d8640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f124c1182a0 con 0x7f124c1129d0 2024-09-11T23:36:30.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.254+0000 7f12509d8640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f124c1185a0 con 0x7f124c074590 2024-09-11T23:36:30.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.254+0000 7f124b59e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f124c1129d0 0x7f124c10b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:30.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.254+0000 7f124b59e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f124c1129d0 0x7f124c10b5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:41352/0 (socket says 172.21.15.26:41352) 2024-09-11T23:36:30.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.254+0000 7f124b59e640 1 -- 172.21.15.26:0/2604807633 learned_addr learned my addr 172.21.15.26:0/2604807633 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:36:30.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.254+0000 7f124a59c640 1 --2- 172.21.15.26:0/2604807633 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f124c074eb0 0x7f124c10b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:30.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.254+0000 7f124ad9d640 1 --2- 172.21.15.26:0/2604807633 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f124c074590 0x7f124c111fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:30.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.255+0000 7f124b59e640 1 -- 172.21.15.26:0/2604807633 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f124c074590 msgr2=0x7f124c111fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:30.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.255+0000 7f124b59e640 1 --2- 172.21.15.26:0/2604807633 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f124c074590 0x7f124c111fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:30.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.255+0000 7f124b59e640 1 -- 172.21.15.26:0/2604807633 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f124c074eb0 msgr2=0x7f124c10b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:30.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.255+0000 7f124b59e640 1 --2- 172.21.15.26:0/2604807633 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f124c074eb0 0x7f124c10b0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:30.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.255+0000 7f124b59e640 1 -- 172.21.15.26:0/2604807633 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f124c10bbe0 con 0x7f124c1129d0 2024-09-11T23:36:30.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.255+0000 7f124b59e640 1 --2- 172.21.15.26:0/2604807633 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f124c1129d0 0x7f124c10b5f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f123c00d720 tx=0x7f123c00dbe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:30.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.256+0000 7f1233fff640 1 -- 172.21.15.26:0/2604807633 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f123c017070 con 0x7f124c1129d0 2024-09-11T23:36:30.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.256+0000 7f1233fff640 1 -- 172.21.15.26:0/2604807633 <== mon.2 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f123c00de30 con 0x7f124c1129d0 2024-09-11T23:36:30.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.256+0000 7f12509d8640 1 -- 172.21.15.26:0/2604807633 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f124c10be70 con 0x7f124c1129d0 2024-09-11T23:36:30.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.256+0000 7f12509d8640 1 -- 172.21.15.26:0/2604807633 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f124c10fd80 con 0x7f124c1129d0 2024-09-11T23:36:30.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.256+0000 7f1233fff640 1 -- 172.21.15.26:0/2604807633 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f123c004d90 con 0x7f124c1129d0 2024-09-11T23:36:30.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.260+0000 7f12535b2740 1 Processor -- start 2024-09-11T23:36:30.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.261+0000 7f12535b2740 1 -- start start 2024-09-11T23:36:30.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.261+0000 7f12535b2740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555a9df5c210 0x555a9df5c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:30.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.262+0000 7f12535b2740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555a9df5cb30 0x555a9df66eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:30.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.262+0000 7f12535b2740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555a9df673f0 0x555a9df697e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:30.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.262+0000 7f12535b2740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x555a9dec1620 con 0x555a9df5cb30 2024-09-11T23:36:30.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.262+0000 7f12535b2740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x555a9de68e50 con 0x555a9df5c210 2024-09-11T23:36:30.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.262+0000 7f12535b2740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x555a9debd5e0 con 0x555a9df673f0 2024-09-11T23:36:30.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.264+0000 7f1233fff640 1 -- 172.21.15.26:0/2604807633 <== mon.2 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f123c009bd0 con 0x7f124c1129d0 2024-09-11T23:36:30.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.265+0000 7f1233fff640 1 --2- 172.21.15.26:0/2604807633 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f12340a2df0 0x7f12340a52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:30.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.265+0000 7f1233fff640 1 -- 172.21.15.26:0/2604807633 <== mon.2 v2:172.21.15.76:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 8586+0+0 (secure 0 0 0) 0x7f123c0ced30 con 0x7f124c1129d0 2024-09-11T23:36:30.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.265+0000 7f124a59c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555a9df5c210 0x555a9df5c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:30.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.265+0000 7f124a59c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555a9df5c210 0x555a9df5c5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:41362/0 (socket says 172.21.15.26:41362) 2024-09-11T23:36:30.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.265+0000 7f124a59c640 1 -- 172.21.15.26:0/1767358305 learned_addr learned my addr 172.21.15.26:0/1767358305 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:36:30.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.265+0000 7f124a59c640 1 --2- 172.21.15.26:0/1767358305 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555a9df673f0 0x555a9df697e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:30.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.269+0000 7f124ad9d640 1 --2- 172.21.15.26:0/1767358305 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555a9df5cb30 0x555a9df66eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:30.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.270+0000 7f124b59e640 1 --2- 172.21.15.26:0/2604807633 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f12340a2df0 0x7f12340a52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:30.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.270+0000 7f124a59c640 1 -- 172.21.15.26:0/1767358305 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555a9df673f0 msgr2=0x555a9df697e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:36:30.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.270+0000 7f124a59c640 1 --2- 172.21.15.26:0/1767358305 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555a9df673f0 0x555a9df697e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:30.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.270+0000 7f124a59c640 1 -- 172.21.15.26:0/1767358305 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555a9df5cb30 msgr2=0x555a9df66eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:36:30.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.270+0000 7f124a59c640 1 --2- 172.21.15.26:0/1767358305 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555a9df5cb30 0x555a9df66eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:30.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.270+0000 7f124a59c640 1 -- 172.21.15.26:0/1767358305 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a9de59d40 con 0x555a9df5c210 2024-09-11T23:36:30.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.271+0000 7f124a59c640 1 --2- 172.21.15.26:0/1767358305 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555a9df5c210 0x555a9df5c5f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f12400096c0 tx=0x7f124002a4d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1942f5c6911ffa71 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:30.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.271+0000 7f12317fa640 1 -- 172.21.15.26:0/1767358305 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1240005000 con 0x555a9df5c210 2024-09-11T23:36:30.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.271+0000 7f12317fa640 1 -- 172.21.15.26:0/1767358305 <== mon.2 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1240005680 con 0x555a9df5c210 2024-09-11T23:36:30.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.271+0000 7f12317fa640 1 -- 172.21.15.26:0/1767358305 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f124003fcf0 con 0x555a9df5c210 2024-09-11T23:36:30.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.272+0000 7f12535b2740 1 -- 172.21.15.26:0/1767358305 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555a9df5c210 msgr2=0x555a9df5c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:30.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.272+0000 7f12535b2740 1 --2- 172.21.15.26:0/1767358305 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555a9df5c210 0x555a9df5c5f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f12400096c0 tx=0x7f124002a4d0 comp rx=0 tx=0).stop 2024-09-11T23:36:30.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.272+0000 7f124b59e640 1 --2- 172.21.15.26:0/2604807633 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f12340a2df0 0x7f12340a52b0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f123c097610 tx=0x7f123c097450 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:30.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.272+0000 7f12535b2740 1 -- 172.21.15.26:0/1767358305 shutdown_connections 2024-09-11T23:36:30.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.272+0000 7f12535b2740 1 --2- 172.21.15.26:0/1767358305 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555a9df673f0 0x555a9df697e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:30.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.272+0000 7f12535b2740 1 --2- 172.21.15.26:0/1767358305 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555a9df5cb30 0x555a9df66eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:30.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.272+0000 7f12535b2740 1 --2- 172.21.15.26:0/1767358305 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555a9df5c210 0x555a9df5c5f0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:30.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.272+0000 7f12535b2740 1 -- 172.21.15.26:0/1767358305 >> 172.21.15.26:0/1767358305 conn(0x555a9df587a0 msgr2=0x555a9df58b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:30.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.277+0000 7f12535b2740 1 -- 172.21.15.26:0/1767358305 shutdown_connections 2024-09-11T23:36:30.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.277+0000 7f12535b2740 1 -- 172.21.15.26:0/1767358305 wait complete. 2024-09-11T23:36:30.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.277+0000 7f12535b2740 1 -- start start 2024-09-11T23:36:30.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.277+0000 7f12535b2740 1 Processor -- start 2024-09-11T23:36:30.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.278+0000 7f12535b2740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555a9df5c210 0x555a9df642e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:30.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.278+0000 7f12535b2740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555a9df5cb30 0x555a9df64820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:30.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.278+0000 7f12535b2740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555a9df673f0 0x555a9df64d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:30.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.278+0000 7f12535b2740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x555a9df69fc0 con 0x555a9df673f0 2024-09-11T23:36:30.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.278+0000 7f12535b2740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x555a9de49f10 con 0x555a9df5cb30 2024-09-11T23:36:30.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.278+0000 7f12535b2740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x555a9df6a140 con 0x555a9df5c210 2024-09-11T23:36:30.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.278+0000 7f12535b2740 10 client.0 _ll_register_callbacks cb 0x555a9de6ddb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:36:30.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.278+0000 7f124a59c640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555a9df5c210 0x555a9df642e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:30.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.278+0000 7f124a59c640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555a9df5c210 0x555a9df642e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:45392/0 (socket says 172.21.15.26:45392) 2024-09-11T23:36:30.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.278+0000 7f124a59c640 1 -- 172.21.15.26:0/4139359746 learned_addr learned my addr 172.21.15.26:0/4139359746 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:36:30.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.278+0000 7f124ad9d640 1 --2- 172.21.15.26:0/4139359746 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555a9df5cb30 0x555a9df64820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:30.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.282+0000 7f124a59c640 1 -- 172.21.15.26:0/4139359746 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555a9df5cb30 msgr2=0x555a9df64820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:36:30.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.282+0000 7f124a59c640 1 --2- 172.21.15.26:0/4139359746 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555a9df5cb30 0x555a9df64820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:30.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.282+0000 7f124a59c640 1 -- 172.21.15.26:0/4139359746 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555a9df673f0 msgr2=0x555a9df64d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:36:30.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.282+0000 7f124a59c640 1 --2- 172.21.15.26:0/4139359746 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555a9df673f0 0x555a9df64d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:30.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.282+0000 7f124a59c640 1 -- 172.21.15.26:0/4139359746 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a9dbd0580 con 0x555a9df5c210 2024-09-11T23:36:30.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.283+0000 7f124a59c640 1 --2- 172.21.15.26:0/4139359746 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555a9df5c210 0x555a9df642e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f1240004bd0 tx=0x7f1240000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8dcd79882298164f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:30.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.283+0000 7f122eff5640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-09-11T23:36:30.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.283+0000 7f122eff5640 1 -- 172.21.15.26:0/4139359746 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f124003fa90 con 0x555a9df5c210 2024-09-11T23:36:30.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.284+0000 7f12535b2740 1 -- 172.21.15.26:0/4139359746 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1240005680 con 0x555a9df5c210 2024-09-11T23:36:30.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.284+0000 7f122eff5640 1 -- 172.21.15.26:0/4139359746 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 8586+0+0 (secure 0 0 0) 0x7f124003fc30 con 0x555a9df5c210 2024-09-11T23:36:30.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.291+0000 7f122eff5640 1 -- 172.21.15.26:0/4139359746 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f124005f050 con 0x555a9df5c210 2024-09-11T23:36:30.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.291+0000 7f12535b2740 10 client.27298 fetch_fsmap learned FSMap version 82 2024-09-11T23:36:30.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.291+0000 7f12535b2740 1 -- 172.21.15.26:0/4139359746 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=82,monmap=4+}) -- 0x555a9dc9a960 con 0x555a9df5c210 2024-09-11T23:36:30.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.295+0000 7f122eff5640 1 -- 172.21.15.26:0/4139359746 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1240005680 con 0x555a9df5c210 2024-09-11T23:36:30.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.296+0000 7f122eff5640 1 -- 172.21.15.26:0/4139359746 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 82) ==== 8414+0+0 (secure 0 0 0) 0x7f12400318f0 con 0x555a9df5c210 2024-09-11T23:36:30.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.296+0000 7f12535b2740 10 client.27298 fetch_fsmap finished waiting for FSMap version 82 2024-09-11T23:36:30.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.296+0000 7f12535b2740 10 client.27298 resolve_mds: validated gid 24485 aka daemon mds.b 2024-09-11T23:36:30.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.296+0000 7f12535b2740 20 client.27298 populate_metadata read hostname 'smithi026' 2024-09-11T23:36:30.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.296+0000 7f12535b2740 1 --2- 172.21.15.26:0/4139359746 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x555a9df76ff0 0x555a9df79450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:30.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.296+0000 7f12535b2740 4 client.27298 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:36:30.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.296+0000 7f12535b2740 1 -- 172.21.15.26:0/4139359746 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555a9deceb90 con 0x555a9df76ff0 2024-09-11T23:36:30.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.297+0000 7f124ad9d640 1 --2- 172.21.15.26:0/4139359746 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x555a9df76ff0 0x555a9df79450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:30.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.300+0000 7f124ad9d640 1 --2- 172.21.15.26:0/4139359746 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x555a9df76ff0 0x555a9df79450 crc :-1 s=READY pgs=278 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=54dfad24fe43595a server_cookie=3444cd685100929f in_seq=0 out_seq=0 2024-09-11T23:36:30.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.301+0000 7f122eff5640 10 client.27298 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:36:30.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.302+0000 7f122eff5640 1 -- 172.21.15.26:0/4139359746 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555a9deceb90 con 0x555a9df76ff0 2024-09-11T23:36:30.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.302+0000 7f122eff5640 10 client.27298 handle_command_reply: tid=0 2024-09-11T23:36:30.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.302+0000 7f12535b2740 1 client.27298 shutdown 2024-09-11T23:36:30.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:36:29 smithi076 ceph-mon[35296]: pgmap v569: 97 pgs: 22 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 34 active+clean; 2.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 7.2 KiB/s wr, 11 op/s 2024-09-11T23:36:30.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.304+0000 7f12535b2740 1 -- 172.21.15.26:0/4139359746 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555a9df5c210 msgr2=0x555a9df642e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:30.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.304+0000 7f12535b2740 1 --2- 172.21.15.26:0/4139359746 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555a9df5c210 0x555a9df642e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f1240004bd0 tx=0x7f1240000bc0 comp rx=0 tx=0).stop 2024-09-11T23:36:30.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.305+0000 7f12535b2740 1 -- 172.21.15.26:0/4139359746 shutdown_connections 2024-09-11T23:36:30.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.305+0000 7f12535b2740 1 --2- 172.21.15.26:0/4139359746 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555a9df5cb30 0x555a9df64820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:30.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.305+0000 7f12535b2740 1 --2- 172.21.15.26:0/4139359746 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x555a9df76ff0 0x555a9df79450 crc :-1 s=READY pgs=278 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:30.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.305+0000 7f12535b2740 1 --2- 172.21.15.26:0/4139359746 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555a9df673f0 0x555a9df64d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:30.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.305+0000 7f12535b2740 1 --2- 172.21.15.26:0/4139359746 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555a9df5c210 0x555a9df642e0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:30.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.305+0000 7f12535b2740 1 -- 172.21.15.26:0/4139359746 >> 172.21.15.26:0/4139359746 conn(0x555a9df587a0 msgr2=0x555a9df57900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:30.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.306+0000 7f122eff5640 0 client.27298 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:36:30.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.306+0000 7f12535b2740 1 -- 172.21.15.26:0/4139359746 shutdown_connections 2024-09-11T23:36:30.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.306+0000 7f12535b2740 1 -- 172.21.15.26:0/4139359746 wait complete. 2024-09-11T23:36:30.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.306+0000 7f12535b2740 20 client.27298 trim_cache size 0 max 16384 2024-09-11T23:36:30.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.338+0000 7f12535b2740 1 Processor -- start 2024-09-11T23:36:30.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.338+0000 7f12535b2740 1 -- start start 2024-09-11T23:36:30.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.339+0000 7f12535b2740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555a9df5c210 0x555a9df5e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:30.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.339+0000 7f12535b2740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555a9df5cb30 0x555a9df7ed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:30.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.339+0000 7f124a59c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555a9df5c210 0x555a9df5e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:30.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.339+0000 7f124a59c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555a9df5c210 0x555a9df5e3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:41392/0 (socket says 172.21.15.26:41392) 2024-09-11T23:36:30.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.339+0000 7f124a59c640 1 -- 172.21.15.26:0/3307232028 learned_addr learned my addr 172.21.15.26:0/3307232028 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:36:30.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.339+0000 7f12535b2740 1 --2- 172.21.15.26:0/3307232028 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555a9df673f0 0x555a9df7f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:30.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.339+0000 7f12535b2740 1 -- 172.21.15.26:0/3307232028 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f123800e8c0 con 0x555a9df5cb30 2024-09-11T23:36:30.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.340+0000 7f12535b2740 1 -- 172.21.15.26:0/3307232028 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f1238010670 con 0x555a9df5c210 2024-09-11T23:36:30.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.340+0000 7f12535b2740 1 -- 172.21.15.26:0/3307232028 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f123800f050 con 0x555a9df673f0 2024-09-11T23:36:30.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.341+0000 7f124a59c640 1 --2- 172.21.15.26:0/3307232028 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555a9df673f0 0x555a9df7f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:30.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.341+0000 7f124ad9d640 1 --2- 172.21.15.26:0/3307232028 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555a9df5cb30 0x555a9df7ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:30.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.342+0000 7f124a59c640 1 -- 172.21.15.26:0/3307232028 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555a9df673f0 msgr2=0x555a9df7f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:36:30.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.342+0000 7f124a59c640 1 --2- 172.21.15.26:0/3307232028 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555a9df673f0 0x555a9df7f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:30.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.342+0000 7f124a59c640 1 -- 172.21.15.26:0/3307232028 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555a9df5cb30 msgr2=0x555a9df7ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:36:30.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.342+0000 7f124a59c640 1 --2- 172.21.15.26:0/3307232028 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555a9df5cb30 0x555a9df7ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:30.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.342+0000 7f124a59c640 1 -- 172.21.15.26:0/3307232028 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a9df63290 con 0x555a9df5c210 2024-09-11T23:36:30.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.342+0000 7f124ad9d640 1 --2- 172.21.15.26:0/3307232028 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555a9df5cb30 0x555a9df7ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:36:30.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.344+0000 7f124a59c640 1 --2- 172.21.15.26:0/3307232028 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555a9df5c210 0x555a9df5e3c0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f124002a9f0 tx=0x7f1240001e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3c594a9222f9470d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:30.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.346+0000 7f1230ff9640 1 -- 172.21.15.26:0/3307232028 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f124003fa20 con 0x555a9df5c210 2024-09-11T23:36:30.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.346+0000 7f1230ff9640 1 -- 172.21.15.26:0/3307232028 <== mon.2 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1240068070 con 0x555a9df5c210 2024-09-11T23:36:30.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.346+0000 7f1230ff9640 1 -- 172.21.15.26:0/3307232028 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f124003e980 con 0x555a9df5c210 2024-09-11T23:36:30.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.347+0000 7f12535b2740 1 -- 172.21.15.26:0/3307232028 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555a9df5c210 msgr2=0x555a9df5e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:30.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.347+0000 7f12535b2740 1 --2- 172.21.15.26:0/3307232028 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555a9df5c210 0x555a9df5e3c0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f124002a9f0 tx=0x7f1240001e70 comp rx=0 tx=0).stop 2024-09-11T23:36:30.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.347+0000 7f12535b2740 1 -- 172.21.15.26:0/3307232028 shutdown_connections 2024-09-11T23:36:30.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.347+0000 7f12535b2740 1 --2- 172.21.15.26:0/3307232028 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555a9df673f0 0x555a9df7f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:30.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.347+0000 7f12535b2740 1 --2- 172.21.15.26:0/3307232028 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555a9df5cb30 0x555a9df7ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:30.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.347+0000 7f12535b2740 1 --2- 172.21.15.26:0/3307232028 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555a9df5c210 0x555a9df5e3c0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:30.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.347+0000 7f12535b2740 1 -- 172.21.15.26:0/3307232028 >> 172.21.15.26:0/3307232028 conn(0x555a9df587a0 msgr2=0x555a9df5dcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:30.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.348+0000 7f12535b2740 1 -- 172.21.15.26:0/3307232028 shutdown_connections 2024-09-11T23:36:30.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.352+0000 7f12535b2740 1 -- 172.21.15.26:0/3307232028 wait complete. 2024-09-11T23:36:30.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.352+0000 7f12535b2740 1 -- start start 2024-09-11T23:36:30.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.352+0000 7f12535b2740 1 Processor -- start 2024-09-11T23:36:30.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.353+0000 7f12535b2740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555a9df5c210 0x555a9df6f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:30.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.353+0000 7f12535b2740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555a9df5cb30 0x555a9df6fb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:30.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.353+0000 7f12535b2740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555a9df673f0 0x555a9df60af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:30.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.353+0000 7f12535b2740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f123800fee0 con 0x555a9df5c210 2024-09-11T23:36:30.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.353+0000 7f12535b2740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f123800f750 con 0x555a9df5cb30 2024-09-11T23:36:30.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.353+0000 7f12535b2740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f123800c8c0 con 0x555a9df673f0 2024-09-11T23:36:30.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.353+0000 7f12535b2740 10 client.0 _ll_register_callbacks cb 0x7f12400048e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:36:30.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.354+0000 7f124ad9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555a9df5cb30 0x555a9df6fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:30.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.354+0000 7f124ad9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555a9df5cb30 0x555a9df6fb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:41402/0 (socket says 172.21.15.26:41402) 2024-09-11T23:36:30.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.354+0000 7f124ad9d640 1 -- 172.21.15.26:0/3868554143 learned_addr learned my addr 172.21.15.26:0/3868554143 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:36:30.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.354+0000 7f124a59c640 1 --2- 172.21.15.26:0/3868554143 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555a9df673f0 0x555a9df60af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:30.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.354+0000 7f124a59c640 1 --2- 172.21.15.26:0/3868554143 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555a9df5c210 0x555a9df6f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:30.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.355+0000 7f124ad9d640 1 -- 172.21.15.26:0/3868554143 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555a9df673f0 msgr2=0x555a9df60af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:36:30.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.355+0000 7f124ad9d640 1 --2- 172.21.15.26:0/3868554143 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555a9df673f0 0x555a9df60af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:30.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.355+0000 7f124ad9d640 1 -- 172.21.15.26:0/3868554143 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555a9df5c210 msgr2=0x555a9df6f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:36:30.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.355+0000 7f124ad9d640 1 --2- 172.21.15.26:0/3868554143 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555a9df5c210 0x555a9df6f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:30.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.355+0000 7f124ad9d640 1 -- 172.21.15.26:0/3868554143 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a9df61320 con 0x555a9df5cb30 2024-09-11T23:36:30.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.355+0000 7f124ad9d640 1 --2- 172.21.15.26:0/3868554143 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555a9df5cb30 0x555a9df6fb10 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x555a9df63b70 tx=0x7f124400ae10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=edb1f5df4645f569 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:30.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.355+0000 7f122eff5640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-11T23:36:30.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.355+0000 7f122eff5640 1 -- 172.21.15.26:0/3868554143 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f124401e030 con 0x555a9df5cb30 2024-09-11T23:36:30.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.355+0000 7f122eff5640 1 -- 172.21.15.26:0/3868554143 <== mon.2 v2:172.21.15.76:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 8586+0+0 (secure 0 0 0) 0x7f1244019b20 con 0x555a9df5cb30 2024-09-11T23:36:30.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.355+0000 7f12535b2740 1 -- 172.21.15.26:0/3868554143 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1240068070 con 0x555a9df5cb30 2024-09-11T23:36:30.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.356+0000 7f122eff5640 1 -- 172.21.15.26:0/3868554143 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f1244019d20 con 0x555a9df5cb30 2024-09-11T23:36:30.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.356+0000 7f12535b2740 10 client.27338 fetch_fsmap learned FSMap version 82 2024-09-11T23:36:30.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.356+0000 7f12535b2740 1 -- 172.21.15.26:0/3868554143 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=82,monmap=4+}) -- 0x555a9dfe1900 con 0x555a9df5cb30 2024-09-11T23:36:30.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.357+0000 7f122eff5640 1 -- 172.21.15.26:0/3868554143 <== mon.2 v2:172.21.15.76:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1240068070 con 0x555a9df5cb30 2024-09-11T23:36:30.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.357+0000 7f122eff5640 1 -- 172.21.15.26:0/3868554143 <== mon.2 v2:172.21.15.76:3300/0 5 ==== fsmap(e 82) ==== 8414+0+0 (secure 0 0 0) 0x7f1244024610 con 0x555a9df5cb30 2024-09-11T23:36:30.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.357+0000 7f12535b2740 10 client.27338 fetch_fsmap finished waiting for FSMap version 82 2024-09-11T23:36:30.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.357+0000 7f12535b2740 10 client.27338 resolve_mds: validated gid 24485 aka daemon mds.b 2024-09-11T23:36:30.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.357+0000 7f12535b2740 20 client.27338 populate_metadata read hostname 'smithi026' 2024-09-11T23:36:30.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.357+0000 7f12535b2740 1 --2- 172.21.15.26:0/3868554143 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x555a9dfe2170 0x555a9dfe4590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:30.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.357+0000 7f12535b2740 4 client.27338 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "cb143a15", "release": true, "await": true}] 2024-09-11T23:36:30.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.357+0000 7f12535b2740 1 -- 172.21.15.26:0/3868554143 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "quiesce db", "set_id": "cb143a15", "release": true, "await": true}) -- 0x555a9dec90a0 con 0x555a9dfe2170 2024-09-11T23:36:30.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.362+0000 7f124a59c640 1 --2- 172.21.15.26:0/3868554143 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x555a9dfe2170 0x555a9dfe4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:30.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.364+0000 7f124a59c640 1 --2- 172.21.15.26:0/3868554143 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x555a9dfe2170 0x555a9dfe4590 crc :-1 s=READY pgs=279 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b2d56217e2e36791 server_cookie=963e1b6ab5fe1396 in_seq=0 out_seq=0 2024-09-11T23:36:30.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:30.364+0000 7f122eff5640 10 client.27338 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:36:32.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:32.082+0000 7f122eff5640 1 -- 172.21.15.26:0/3868554143 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x555a9dec90a0 con 0x555a9dfe2170 2024-09-11T23:36:32.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:32.082+0000 7f122eff5640 10 client.27338 handle_command_reply: tid=0 2024-09-11T23:36:32.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:32.082+0000 7f12535b2740 1 client.27338 shutdown 2024-09-11T23:36:32.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:32.082+0000 7f12535b2740 1 -- 172.21.15.26:0/3868554143 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555a9df5cb30 msgr2=0x555a9df6fb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:32.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:32.082+0000 7f12535b2740 1 --2- 172.21.15.26:0/3868554143 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555a9df5cb30 0x555a9df6fb10 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x555a9df63b70 tx=0x7f124400ae10 comp rx=0 tx=0).stop 2024-09-11T23:36:32.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:32.082+0000 7f12535b2740 1 -- 172.21.15.26:0/3868554143 shutdown_connections 2024-09-11T23:36:32.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:32.082+0000 7f12535b2740 1 --2- 172.21.15.26:0/3868554143 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555a9df5cb30 0x555a9df6fb10 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:32.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:32.082+0000 7f12535b2740 1 --2- 172.21.15.26:0/3868554143 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x555a9dfe2170 0x555a9dfe4590 crc :-1 s=READY pgs=279 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:32.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:32.082+0000 7f12535b2740 1 --2- 172.21.15.26:0/3868554143 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555a9df673f0 0x555a9df60af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:32.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:32.082+0000 7f12535b2740 1 --2- 172.21.15.26:0/3868554143 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555a9df5c210 0x555a9df6f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:32.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:32.082+0000 7f12535b2740 1 -- 172.21.15.26:0/3868554143 >> 172.21.15.26:0/3868554143 conn(0x555a9df587a0 msgr2=0x555a9df57900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:32.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:32.082+0000 7f122eff5640 0 client.27338 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:36:32.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:32.082+0000 7f12535b2740 1 -- 172.21.15.26:0/3868554143 shutdown_connections 2024-09-11T23:36:32.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:32.082+0000 7f12535b2740 1 -- 172.21.15.26:0/3868554143 wait complete. 2024-09-11T23:36:32.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:32.083+0000 7f12535b2740 20 client.27338 trim_cache size 0 max 16384 2024-09-11T23:36:32.084 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-11T23:36:32.085 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 82, 2024-09-11T23:36:32.085 INFO:teuthology.orchestra.run.smithi026.stdout: "leader": 24485, 2024-09-11T23:36:32.085 INFO:teuthology.orchestra.run.smithi026.stdout: "set_version": 64, 2024-09-11T23:36:32.085 INFO:teuthology.orchestra.run.smithi026.stdout: "sets": { 2024-09-11T23:36:32.085 INFO:teuthology.orchestra.run.smithi026.stdout: "cb143a15": { 2024-09-11T23:36:32.085 INFO:teuthology.orchestra.run.smithi026.stdout: "version": 64, 2024-09-11T23:36:32.085 INFO:teuthology.orchestra.run.smithi026.stdout: "age_ref": 0.0, 2024-09-11T23:36:32.085 INFO:teuthology.orchestra.run.smithi026.stdout: "state": { 2024-09-11T23:36:32.085 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "RELEASED", 2024-09-11T23:36:32.085 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.0 2024-09-11T23:36:32.085 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:36:32.085 INFO:teuthology.orchestra.run.smithi026.stdout: "timeout": 90.0, 2024-09-11T23:36:32.086 INFO:teuthology.orchestra.run.smithi026.stdout: "expiration": 144.2, 2024-09-11T23:36:32.086 INFO:teuthology.orchestra.run.smithi026.stdout: "members": { 2024-09-11T23:36:32.086 INFO:teuthology.orchestra.run.smithi026.stdout: "file:/": { 2024-09-11T23:36:32.086 INFO:teuthology.orchestra.run.smithi026.stdout: "excluded": false, 2024-09-11T23:36:32.086 INFO:teuthology.orchestra.run.smithi026.stdout: "state": { 2024-09-11T23:36:32.086 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "RELEASED", 2024-09-11T23:36:32.087 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.0 2024-09-11T23:36:32.087 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:36:32.087 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:36:32.087 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:36:32.087 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:36:32.087 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:36:32.088 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-11T23:36:32.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:32.083+0000 7f12509d8640 1 -- 172.21.15.26:0/2604807633 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f12340a2df0 msgr2=0x7f12340a52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:32.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:32.083+0000 7f12509d8640 1 --2- 172.21.15.26:0/2604807633 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f12340a2df0 0x7f12340a52b0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f123c097610 tx=0x7f123c097450 comp rx=0 tx=0).stop 2024-09-11T23:36:32.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:32.083+0000 7f12509d8640 1 -- 172.21.15.26:0/2604807633 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f124c1129d0 msgr2=0x7f124c10b5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:32.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:32.083+0000 7f12509d8640 1 --2- 172.21.15.26:0/2604807633 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f124c1129d0 0x7f124c10b5f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f123c00d720 tx=0x7f123c00dbe0 comp rx=0 tx=0).stop 2024-09-11T23:36:32.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:32.085+0000 7f12509d8640 1 -- 172.21.15.26:0/2604807633 shutdown_connections 2024-09-11T23:36:32.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:32.085+0000 7f12509d8640 1 --2- 172.21.15.26:0/2604807633 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f12340a2df0 0x7f12340a52b0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:32.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:32.085+0000 7f12509d8640 1 --2- 172.21.15.26:0/2604807633 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f124c1129d0 0x7f124c10b5f0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:32.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:32.085+0000 7f12509d8640 1 --2- 172.21.15.26:0/2604807633 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f124c074eb0 0x7f124c10b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:32.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:32.085+0000 7f12509d8640 1 --2- 172.21.15.26:0/2604807633 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f124c074590 0x7f124c111fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:32.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:32.085+0000 7f12509d8640 1 -- 172.21.15.26:0/2604807633 >> 172.21.15.26:0/2604807633 conn(0x7f124c0fd740 msgr2=0x7f124c0fefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:32.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:32.087+0000 7f12509d8640 1 -- 172.21.15.26:0/2604807633 shutdown_connections 2024-09-11T23:36:32.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:36:32.087+0000 7f12509d8640 1 -- 172.21.15.26:0/2604807633 wait complete. 2024-09-11T23:36:32.238 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:36:31 smithi136 ceph-mon[35121]: pgmap v570: 97 pgs: 22 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 34 active+clean; 2.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 7.2 KiB/s wr, 15 op/s 2024-09-11T23:36:32.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:36:31 smithi026 ceph-mon[31172]: pgmap v570: 97 pgs: 22 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 34 active+clean; 2.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 7.2 KiB/s wr, 15 op/s 2024-09-11T23:36:32.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:36:31 smithi076 ceph-mon[35296]: pgmap v570: 97 pgs: 22 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 34 active+clean; 2.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 7.2 KiB/s wr, 15 op/s 2024-09-11T23:36:32.983 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'cb143a15', total seconds elapsed: 31.1 2024-09-11T23:36:34.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:36:33 smithi136 ceph-mon[35121]: pgmap v571: 97 pgs: 22 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 34 active+clean; 2.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 7.0 KiB/s wr, 11 op/s 2024-09-11T23:36:34.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:36:33 smithi136 ceph-mon[35121]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:36:34.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:36:33 smithi026 ceph-mon[31172]: pgmap v571: 97 pgs: 22 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 34 active+clean; 2.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 7.0 KiB/s wr, 11 op/s 2024-09-11T23:36:34.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:36:33 smithi026 ceph-mon[31172]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:36:34.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:36:33 smithi076 ceph-mon[35296]: pgmap v571: 97 pgs: 22 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 34 active+clean; 2.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 7.0 KiB/s wr, 11 op/s 2024-09-11T23:36:34.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:36:33 smithi076 ceph-mon[35296]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:36:35.238 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:36:34 smithi136 ceph-mon[35121]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:36:35.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:36:34 smithi026 ceph-mon[31172]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:36:35.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:36:34 smithi076 ceph-mon[35296]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:36:36.238 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:36:35 smithi136 ceph-mon[35121]: pgmap v572: 97 pgs: 22 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 34 active+clean; 2.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 94 KiB/s wr, 19 op/s 2024-09-11T23:36:36.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:36:35 smithi026 ceph-mon[31172]: pgmap v572: 97 pgs: 22 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 34 active+clean; 2.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 94 KiB/s wr, 19 op/s 2024-09-11T23:36:36.282 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.279+0000 7f1d91e53640 1 -- 172.21.15.26:0/811222012 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1d8c106be0 msgr2=0x7f1d8c106fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:36.282 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.279+0000 7f1d91e53640 1 --2- 172.21.15.26:0/811222012 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1d8c106be0 0x7f1d8c106fc0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f1d84009ca0 tx=0x7f1d8402a8d0 comp rx=0 tx=0).stop 2024-09-11T23:36:36.284 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.282+0000 7f1d91e53640 1 -- 172.21.15.26:0/811222012 shutdown_connections 2024-09-11T23:36:36.284 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.282+0000 7f1d91e53640 1 --2- 172.21.15.26:0/811222012 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1d8c0fff60 0x7f1d8c1003c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:36.284 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.282+0000 7f1d91e53640 1 --2- 172.21.15.26:0/811222012 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1d8c1009e0 0x7f1d8c1046a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:36.285 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.282+0000 7f1d91e53640 1 --2- 172.21.15.26:0/811222012 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1d8c106be0 0x7f1d8c106fc0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:36.285 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.282+0000 7f1d91e53640 1 -- 172.21.15.26:0/811222012 >> 172.21.15.26:0/811222012 conn(0x7f1d8c0770f0 msgr2=0x7f1d8c0fe480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:36.287 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.287+0000 7f1d91e53640 1 -- 172.21.15.26:0/811222012 shutdown_connections 2024-09-11T23:36:36.291 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.291+0000 7f1d91e53640 1 -- 172.21.15.26:0/811222012 wait complete. 2024-09-11T23:36:36.292 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.292+0000 7f1d91e53640 1 Processor -- start 2024-09-11T23:36:36.292 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.292+0000 7f1d91e53640 1 -- start start 2024-09-11T23:36:36.297 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.292+0000 7f1d91e53640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1d8c0fff60 0x7f1d8c1a46d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:36.297 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.292+0000 7f1d91e53640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1d8c1009e0 0x7f1d8c1a4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:36.298 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.292+0000 7f1d91e53640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1d8c106be0 0x7f1d8c19e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:36.298 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.292+0000 7f1d91e53640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f1d8c1185f0 con 0x7f1d8c106be0 2024-09-11T23:36:36.298 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.292+0000 7f1d91e53640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f1d8c118470 con 0x7f1d8c1009e0 2024-09-11T23:36:36.298 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.292+0000 7f1d91e53640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f1d8c118770 con 0x7f1d8c0fff60 2024-09-11T23:36:36.298 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.293+0000 7f1d8affd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1d8c1009e0 0x7f1d8c1a4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:36.298 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.294+0000 7f1d8affd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1d8c1009e0 0x7f1d8c1a4c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:56572/0 (socket says 172.21.15.26:56572) 2024-09-11T23:36:36.298 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.294+0000 7f1d8affd640 1 -- 172.21.15.26:0/2658053188 learned_addr learned my addr 172.21.15.26:0/2658053188 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:36:36.298 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.294+0000 7f1d8b7fe640 1 --2- 172.21.15.26:0/2658053188 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1d8c0fff60 0x7f1d8c1a46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:36.298 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.294+0000 7f1d8affd640 1 -- 172.21.15.26:0/2658053188 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1d8c0fff60 msgr2=0x7f1d8c1a46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:36.298 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.294+0000 7f1d8affd640 1 --2- 172.21.15.26:0/2658053188 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1d8c0fff60 0x7f1d8c1a46d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:36.298 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.294+0000 7f1d8affd640 1 -- 172.21.15.26:0/2658053188 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1d8c106be0 msgr2=0x7f1d8c19e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:36.298 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.294+0000 7f1d8affd640 1 --2- 172.21.15.26:0/2658053188 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1d8c106be0 0x7f1d8c19e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:36.299 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.294+0000 7f1d8affd640 1 -- 172.21.15.26:0/2658053188 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d8c19f010 con 0x7f1d8c1009e0 2024-09-11T23:36:36.299 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.295+0000 7f1d8affd640 1 --2- 172.21.15.26:0/2658053188 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1d8c1009e0 0x7f1d8c1a4c10 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f1d7c00d770 tx=0x7f1d7c00dc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:36.301 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.301+0000 7f1d88ff9640 1 -- 172.21.15.26:0/2658053188 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d7c017070 con 0x7f1d8c1009e0 2024-09-11T23:36:36.302 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.301+0000 7f1d88ff9640 1 -- 172.21.15.26:0/2658053188 <== mon.2 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d7c005500 con 0x7f1d8c1009e0 2024-09-11T23:36:36.302 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.301+0000 7f1d88ff9640 1 -- 172.21.15.26:0/2658053188 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d7c004e00 con 0x7f1d8c1009e0 2024-09-11T23:36:36.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:36:35 smithi076 ceph-mon[35296]: pgmap v572: 97 pgs: 22 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 34 active+clean; 2.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 94 KiB/s wr, 19 op/s 2024-09-11T23:36:36.303 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.301+0000 7f1d91e53640 1 -- 172.21.15.26:0/2658053188 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d8c19f300 con 0x7f1d8c1009e0 2024-09-11T23:36:36.303 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.301+0000 7f1d91e53640 1 -- 172.21.15.26:0/2658053188 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d8c1ab520 con 0x7f1d8c1009e0 2024-09-11T23:36:36.307 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.306+0000 7f1d94be9740 1 Processor -- start 2024-09-11T23:36:36.308 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.308+0000 7f1d94be9740 1 -- start start 2024-09-11T23:36:36.317 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.315+0000 7f1d94be9740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c028bc2af0 0x55c028bc2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:36.317 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.315+0000 7f1d94be9740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c028bc3410 0x55c028bcd770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:36.317 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.315+0000 7f1d94be9740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c028bcdcb0 0x55c028bd00a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:36.317 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.315+0000 7f1d94be9740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55c028b272f0 con 0x55c028bcdcb0 2024-09-11T23:36:36.317 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.315+0000 7f1d94be9740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55c028acf270 con 0x55c028bc3410 2024-09-11T23:36:36.317 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.315+0000 7f1d94be9740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55c028b232b0 con 0x55c028bc2af0 2024-09-11T23:36:36.317 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.316+0000 7f1d8bfff640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c028bc2af0 0x55c028bc2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:36.317 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.316+0000 7f1d8bfff640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c028bc2af0 0x55c028bc2ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:45672/0 (socket says 172.21.15.26:45672) 2024-09-11T23:36:36.317 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.316+0000 7f1d8bfff640 1 -- 172.21.15.26:0/2564440132 learned_addr learned my addr 172.21.15.26:0/2564440132 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:36:36.317 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.316+0000 7f1d8affd640 1 --2- 172.21.15.26:0/2564440132 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c028bcdcb0 0x55c028bd00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:36.318 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.317+0000 7f1d8bfff640 1 -- 172.21.15.26:0/2564440132 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c028bc3410 msgr2=0x55c028bcd770 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T23:36:36.318 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.317+0000 7f1d8bfff640 1 --2- 172.21.15.26:0/2564440132 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c028bc3410 0x55c028bcd770 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:36.318 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.317+0000 7f1d8bfff640 1 -- 172.21.15.26:0/2564440132 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c028bcdcb0 msgr2=0x55c028bd00a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:36:36.318 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.317+0000 7f1d8bfff640 1 --2- 172.21.15.26:0/2564440132 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c028bcdcb0 0x55c028bd00a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:36.318 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.317+0000 7f1d8bfff640 1 -- 172.21.15.26:0/2564440132 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c028ace340 con 0x55c028bc2af0 2024-09-11T23:36:36.318 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.317+0000 7f1d8bfff640 1 --2- 172.21.15.26:0/2564440132 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c028bc2af0 0x55c028bc2ed0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f1d80009bc0 tx=0x7f1d8000e720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dcfbea7849d0e37f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:36.319 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.318+0000 7f1d88ff9640 1 -- 172.21.15.26:0/2658053188 <== mon.2 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f1d7c005020 con 0x7f1d8c1009e0 2024-09-11T23:36:36.319 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.319+0000 7f1d88ff9640 1 --2- 172.21.15.26:0/2658053188 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f1d740a2fd0 0x7f1d740a5490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:36.319 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.319+0000 7f1d88ff9640 1 -- 172.21.15.26:0/2658053188 <== mon.2 v2:172.21.15.76:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 8586+0+0 (secure 0 0 0) 0x7f1d7c0cef20 con 0x7f1d8c1009e0 2024-09-11T23:36:36.320 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.320+0000 7f1d71ffb640 1 -- 172.21.15.26:0/2564440132 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d80005020 con 0x55c028bc2af0 2024-09-11T23:36:36.321 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.320+0000 7f1d71ffb640 1 -- 172.21.15.26:0/2564440132 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d800047a0 con 0x55c028bc2af0 2024-09-11T23:36:36.321 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.320+0000 7f1d71ffb640 1 -- 172.21.15.26:0/2564440132 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d80010690 con 0x55c028bc2af0 2024-09-11T23:36:36.330 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.328+0000 7f1d94be9740 1 -- 172.21.15.26:0/2564440132 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c028bc2af0 msgr2=0x55c028bc2ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:36.330 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.329+0000 7f1d94be9740 1 --2- 172.21.15.26:0/2564440132 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c028bc2af0 0x55c028bc2ed0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f1d80009bc0 tx=0x7f1d8000e720 comp rx=0 tx=0).stop 2024-09-11T23:36:36.330 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.329+0000 7f1d94be9740 1 -- 172.21.15.26:0/2564440132 shutdown_connections 2024-09-11T23:36:36.330 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.329+0000 7f1d94be9740 1 --2- 172.21.15.26:0/2564440132 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c028bc3410 0x55c028bcd770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:36.330 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.329+0000 7f1d94be9740 1 --2- 172.21.15.26:0/2564440132 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c028bcdcb0 0x55c028bd00a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:36.330 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.329+0000 7f1d94be9740 1 --2- 172.21.15.26:0/2564440132 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c028bc2af0 0x55c028bc2ed0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:36.330 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.329+0000 7f1d94be9740 1 -- 172.21.15.26:0/2564440132 >> 172.21.15.26:0/2564440132 conn(0x55c028bbf060 msgr2=0x55c028bbf440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:36.330 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.329+0000 7f1d8b7fe640 1 --2- 172.21.15.26:0/2658053188 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f1d740a2fd0 0x7f1d740a5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:36.330 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.330+0000 7f1d94be9740 1 -- 172.21.15.26:0/2564440132 shutdown_connections 2024-09-11T23:36:36.331 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.330+0000 7f1d8b7fe640 1 --2- 172.21.15.26:0/2658053188 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f1d740a2fd0 0x7f1d740a5490 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f1d84004af0 tx=0x7f1d84000f40 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:36.334 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.333+0000 7f1d94be9740 1 -- 172.21.15.26:0/2564440132 wait complete. 2024-09-11T23:36:36.334 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.333+0000 7f1d94be9740 1 -- start start 2024-09-11T23:36:36.334 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.334+0000 7f1d94be9740 1 Processor -- start 2024-09-11T23:36:36.338 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.337+0000 7f1d94be9740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c028bc2af0 0x55c028bcac20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:36.338 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.337+0000 7f1d94be9740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c028bc3410 0x55c028bcb160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:36.338 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.337+0000 7f1d94be9740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c028bcdcb0 0x55c028bcb6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:36.338 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.337+0000 7f1d94be9740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55c028bd0880 con 0x55c028bc3410 2024-09-11T23:36:36.338 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.338+0000 7f1d94be9740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55c028ab0290 con 0x55c028bcdcb0 2024-09-11T23:36:36.338 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.338+0000 7f1d94be9740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55c028bd0a00 con 0x55c028bc2af0 2024-09-11T23:36:36.338 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.338+0000 7f1d94be9740 10 client.0 _ll_register_callbacks cb 0x55c028ae1140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:36:36.339 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.338+0000 7f1d8bfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c028bcdcb0 0x55c028bcb6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:36.339 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.338+0000 7f1d8bfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c028bcdcb0 0x55c028bcb6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:56598/0 (socket says 172.21.15.26:56598) 2024-09-11T23:36:36.339 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.338+0000 7f1d8bfff640 1 -- 172.21.15.26:0/359300327 learned_addr learned my addr 172.21.15.26:0/359300327 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:36:36.340 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.340+0000 7f1d8bfff640 1 -- 172.21.15.26:0/359300327 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c028bc2af0 msgr2=0x55c028bcac20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T23:36:36.340 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.340+0000 7f1d8bfff640 1 --2- 172.21.15.26:0/359300327 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c028bc2af0 0x55c028bcac20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:36.340 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.340+0000 7f1d8bfff640 1 -- 172.21.15.26:0/359300327 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c028bc3410 msgr2=0x55c028bcb160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:36:36.340 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.340+0000 7f1d8bfff640 1 --2- 172.21.15.26:0/359300327 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c028bc3410 0x55c028bcb160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:36.340 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.340+0000 7f1d8bfff640 1 -- 172.21.15.26:0/359300327 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c02883a210 con 0x55c028bcdcb0 2024-09-11T23:36:36.343 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.342+0000 7f1d8bfff640 1 --2- 172.21.15.26:0/359300327 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c028bcdcb0 0x55c028bcb6a0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f1d80005250 tx=0x7f1d80005280 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e793f835a8d076c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:36.343 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.342+0000 7f1d6affd640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-11T23:36:36.344 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.343+0000 7f1d6affd640 1 -- 172.21.15.26:0/359300327 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d80028060 con 0x55c028bcdcb0 2024-09-11T23:36:36.344 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.343+0000 7f1d6affd640 1 -- 172.21.15.26:0/359300327 <== mon.2 v2:172.21.15.76:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 8586+0+0 (secure 0 0 0) 0x7f1d80004030 con 0x55c028bcdcb0 2024-09-11T23:36:36.344 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.344+0000 7f1d94be9740 1 -- 172.21.15.26:0/359300327 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1d800047a0 con 0x55c028bcdcb0 2024-09-11T23:36:36.344 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.344+0000 7f1d6affd640 1 -- 172.21.15.26:0/359300327 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=84) ==== 24+0+0 (secure 0 0 0) 0x7f1d8001d040 con 0x55c028bcdcb0 2024-09-11T23:36:36.345 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.344+0000 7f1d94be9740 10 client.27347 fetch_fsmap learned FSMap version 84 2024-09-11T23:36:36.345 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.344+0000 7f1d94be9740 1 -- 172.21.15.26:0/359300327 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=84,monmap=4+}) -- 0x55c028906620 con 0x55c028bcdcb0 2024-09-11T23:36:36.346 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.345+0000 7f1d6affd640 1 -- 172.21.15.26:0/359300327 <== mon.2 v2:172.21.15.76:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d800047a0 con 0x55c028bcdcb0 2024-09-11T23:36:36.346 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.345+0000 7f1d6affd640 1 -- 172.21.15.26:0/359300327 <== mon.2 v2:172.21.15.76:3300/0 5 ==== fsmap(e 84) ==== 8418+0+0 (secure 0 0 0) 0x7f1d80003ad0 con 0x55c028bcdcb0 2024-09-11T23:36:36.346 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.345+0000 7f1d94be9740 10 client.27347 fetch_fsmap finished waiting for FSMap version 84 2024-09-11T23:36:36.346 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.345+0000 7f1d94be9740 10 client.27347 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:36:36.346 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.345+0000 7f1d94be9740 20 client.27347 populate_metadata read hostname 'smithi026' 2024-09-11T23:36:36.346 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.345+0000 7f1d94be9740 1 --2- 172.21.15.26:0/359300327 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55c028bde0e0 0x55c028be0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:36.346 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.345+0000 7f1d94be9740 4 client.27347 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:36:36.346 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.345+0000 7f1d94be9740 1 -- 172.21.15.26:0/359300327 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c028b34a20 con 0x55c028bde0e0 2024-09-11T23:36:36.347 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.347+0000 7f1d8b7fe640 1 --2- 172.21.15.26:0/359300327 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55c028bde0e0 0x55c028be0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:36.348 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.348+0000 7f1d8b7fe640 1 --2- 172.21.15.26:0/359300327 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55c028bde0e0 0x55c028be0580 crc :-1 s=READY pgs=280 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=52f79d293100c48d server_cookie=d2972ae4baa8fdb3 in_seq=0 out_seq=0 2024-09-11T23:36:36.348 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.348+0000 7f1d6affd640 10 client.27347 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:36:36.351 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.350+0000 7f1d6affd640 1 -- 172.21.15.26:0/359300327 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c028b34a20 con 0x55c028bde0e0 2024-09-11T23:36:36.351 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.350+0000 7f1d6affd640 10 client.27347 handle_command_reply: tid=0 2024-09-11T23:36:36.351 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.350+0000 7f1d94be9740 1 client.27347 shutdown 2024-09-11T23:36:36.352 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.352+0000 7f1d94be9740 1 -- 172.21.15.26:0/359300327 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c028bcdcb0 msgr2=0x55c028bcb6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:36.352 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.352+0000 7f1d94be9740 1 --2- 172.21.15.26:0/359300327 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c028bcdcb0 0x55c028bcb6a0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f1d80005250 tx=0x7f1d80005280 comp rx=0 tx=0).stop 2024-09-11T23:36:36.354 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.353+0000 7f1d94be9740 1 -- 172.21.15.26:0/359300327 shutdown_connections 2024-09-11T23:36:36.354 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.353+0000 7f1d94be9740 1 --2- 172.21.15.26:0/359300327 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c028bcdcb0 0x55c028bcb6a0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:36.354 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.353+0000 7f1d94be9740 1 --2- 172.21.15.26:0/359300327 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55c028bde0e0 0x55c028be0580 crc :-1 s=READY pgs=280 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:36.354 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.353+0000 7f1d94be9740 1 --2- 172.21.15.26:0/359300327 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c028bc3410 0x55c028bcb160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:36.355 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.354+0000 7f1d94be9740 1 --2- 172.21.15.26:0/359300327 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c028bc2af0 0x55c028bcac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:36.355 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.354+0000 7f1d94be9740 1 -- 172.21.15.26:0/359300327 >> 172.21.15.26:0/359300327 conn(0x55c028bbf060 msgr2=0x55c028bc4d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:36.357 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.354+0000 7f1d6affd640 0 client.27347 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:36:36.357 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.356+0000 7f1d94be9740 1 -- 172.21.15.26:0/359300327 shutdown_connections 2024-09-11T23:36:36.357 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.357+0000 7f1d94be9740 1 -- 172.21.15.26:0/359300327 wait complete. 2024-09-11T23:36:36.359 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.359+0000 7f1d94be9740 20 client.27347 trim_cache size 0 max 16384 2024-09-11T23:36:36.390 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.389+0000 7f1d94be9740 1 Processor -- start 2024-09-11T23:36:36.390 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.389+0000 7f1d94be9740 1 -- start start 2024-09-11T23:36:36.390 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.390+0000 7f1d94be9740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c028bc2af0 0x55c028bc9b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:36.390 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.390+0000 7f1d94be9740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c028bc3410 0x55c028bcb4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:36.390 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.390+0000 7f1d94be9740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c028bcdcb0 0x55c028be5e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:36.390 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.390+0000 7f1d94be9740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f1d8c1b4030 con 0x55c028bcdcb0 2024-09-11T23:36:36.390 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.390+0000 7f1d94be9740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f1d8c1b5de0 con 0x55c028bc2af0 2024-09-11T23:36:36.391 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.390+0000 7f1d94be9740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f1d8c1b4760 con 0x55c028bc3410 2024-09-11T23:36:36.391 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.390+0000 7f1d8bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c028bcdcb0 0x55c028be5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:36.391 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.390+0000 7f1d8b7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c028bc2af0 0x55c028bc9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:36.391 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.390+0000 7f1d8bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c028bcdcb0 0x55c028be5e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48948/0 (socket says 172.21.15.26:48948) 2024-09-11T23:36:36.391 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.390+0000 7f1d8b7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c028bc2af0 0x55c028bc9b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:56614/0 (socket says 172.21.15.26:56614) 2024-09-11T23:36:36.392 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.390+0000 7f1d8bfff640 1 -- 172.21.15.26:0/1748017050 learned_addr learned my addr 172.21.15.26:0/1748017050 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:36:36.392 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.390+0000 7f1d8affd640 1 --2- 172.21.15.26:0/1748017050 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c028bc3410 0x55c028bcb4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:36.392 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.390+0000 7f1d8affd640 1 -- 172.21.15.26:0/1748017050 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c028bc2af0 msgr2=0x55c028bc9b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:36:36.392 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.390+0000 7f1d8affd640 1 --2- 172.21.15.26:0/1748017050 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c028bc2af0 0x55c028bc9b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:36.392 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.390+0000 7f1d8affd640 1 -- 172.21.15.26:0/1748017050 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c028bcdcb0 msgr2=0x55c028be5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:36:36.392 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.390+0000 7f1d8affd640 1 --2- 172.21.15.26:0/1748017050 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c028bcdcb0 0x55c028be5e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:36.392 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.390+0000 7f1d8affd640 1 -- 172.21.15.26:0/1748017050 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c028bc9190 con 0x55c028bc3410 2024-09-11T23:36:36.392 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.390+0000 7f1d8bfff640 1 --2- 172.21.15.26:0/1748017050 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c028bcdcb0 0x55c028be5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:36:36.392 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.391+0000 7f1d8affd640 1 --2- 172.21.15.26:0/1748017050 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c028bc3410 0x55c028bcb4d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f1d7c0cc5d0 tx=0x7f1d7c0048a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=953080bc71e81090 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:36.392 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.391+0000 7f1d717fa640 1 -- 172.21.15.26:0/1748017050 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d7c020070 con 0x55c028bc3410 2024-09-11T23:36:36.393 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.391+0000 7f1d717fa640 1 -- 172.21.15.26:0/1748017050 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d7c09b3a0 con 0x55c028bc3410 2024-09-11T23:36:36.393 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.391+0000 7f1d717fa640 1 -- 172.21.15.26:0/1748017050 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d7c0cc850 con 0x55c028bc3410 2024-09-11T23:36:36.393 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.392+0000 7f1d94be9740 1 -- 172.21.15.26:0/1748017050 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c028bc3410 msgr2=0x55c028bcb4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:36.393 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.392+0000 7f1d94be9740 1 --2- 172.21.15.26:0/1748017050 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c028bc3410 0x55c028bcb4d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f1d7c0cc5d0 tx=0x7f1d7c0048a0 comp rx=0 tx=0).stop 2024-09-11T23:36:36.393 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.392+0000 7f1d94be9740 1 -- 172.21.15.26:0/1748017050 shutdown_connections 2024-09-11T23:36:36.394 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.392+0000 7f1d94be9740 1 --2- 172.21.15.26:0/1748017050 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c028bcdcb0 0x55c028be5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:36.394 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.393+0000 7f1d94be9740 1 --2- 172.21.15.26:0/1748017050 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c028bc3410 0x55c028bcb4d0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:36.394 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.393+0000 7f1d94be9740 1 --2- 172.21.15.26:0/1748017050 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c028bc2af0 0x55c028bc9b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:36.394 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.393+0000 7f1d94be9740 1 -- 172.21.15.26:0/1748017050 >> 172.21.15.26:0/1748017050 conn(0x55c028bbf060 msgr2=0x55c028bc4d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:36.394 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.394+0000 7f1d94be9740 1 -- 172.21.15.26:0/1748017050 shutdown_connections 2024-09-11T23:36:36.394 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.394+0000 7f1d94be9740 1 -- 172.21.15.26:0/1748017050 wait complete. 2024-09-11T23:36:36.395 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.395+0000 7f1d94be9740 1 -- start start 2024-09-11T23:36:36.395 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.395+0000 7f1d94be9740 1 Processor -- start 2024-09-11T23:36:36.396 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.395+0000 7f1d94be9740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c028bc2af0 0x55c028bd5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:36.396 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.396+0000 7f1d94be9740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c028bc3410 0x55c028bd61d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:36.396 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.396+0000 7f1d8b7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c028bc2af0 0x55c028bd5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:36.396 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.396+0000 7f1d8b7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c028bc2af0 0x55c028bd5c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:56622/0 (socket says 172.21.15.26:56622) 2024-09-11T23:36:36.397 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.396+0000 7f1d94be9740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c028bcdcb0 0x55c028bc6fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:36.397 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.396+0000 7f1d94be9740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f1d8c1b5650 con 0x55c028bcdcb0 2024-09-11T23:36:36.397 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.396+0000 7f1d94be9740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f1d8c1b4ec0 con 0x55c028bc2af0 2024-09-11T23:36:36.397 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.396+0000 7f1d94be9740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f1d8c1b2030 con 0x55c028bc3410 2024-09-11T23:36:36.397 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.396+0000 7f1d94be9740 10 client.0 _ll_register_callbacks cb 0x7f1d80009a90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:36:36.397 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.396+0000 7f1d8affd640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c028bc3410 0x55c028bd61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:36.398 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.396+0000 7f1d8b7fe640 1 -- 172.21.15.26:0/4152335027 learned_addr learned my addr 172.21.15.26:0/4152335027 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:36:36.398 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.396+0000 7f1d8bfff640 1 --2- 172.21.15.26:0/4152335027 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c028bcdcb0 0x55c028bc6fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:36.398 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.397+0000 7f1d8b7fe640 1 -- 172.21.15.26:0/4152335027 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c028bc3410 msgr2=0x55c028bd61d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:36:36.398 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.397+0000 7f1d8b7fe640 1 --2- 172.21.15.26:0/4152335027 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c028bc3410 0x55c028bd61d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:36.398 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.397+0000 7f1d8b7fe640 1 -- 172.21.15.26:0/4152335027 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c028bcdcb0 msgr2=0x55c028bc6fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:36:36.398 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.397+0000 7f1d8b7fe640 1 --2- 172.21.15.26:0/4152335027 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c028bcdcb0 0x55c028bc6fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:36.398 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.397+0000 7f1d8b7fe640 1 -- 172.21.15.26:0/4152335027 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c028bc7510 con 0x55c028bc2af0 2024-09-11T23:36:36.398 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.397+0000 7f1d8b7fe640 1 --2- 172.21.15.26:0/4152335027 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c028bc2af0 0x55c028bd5c90 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f1d84009ca0 tx=0x7f1d8402a8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6d605907eb6d5ef server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:36.398 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.397+0000 7f1d6affd640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-11T23:36:36.399 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.398+0000 7f1d6affd640 1 -- 172.21.15.26:0/4152335027 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d8405e060 con 0x55c028bc2af0 2024-09-11T23:36:36.399 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.398+0000 7f1d94be9740 1 -- 172.21.15.26:0/4152335027 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1d7c09b3a0 con 0x55c028bc2af0 2024-09-11T23:36:36.399 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.399+0000 7f1d6affd640 1 -- 172.21.15.26:0/4152335027 <== mon.2 v2:172.21.15.76:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 8586+0+0 (secure 0 0 0) 0x7f1d840307d0 con 0x55c028bc2af0 2024-09-11T23:36:36.399 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.399+0000 7f1d6affd640 1 -- 172.21.15.26:0/4152335027 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=84) ==== 24+0+0 (secure 0 0 0) 0x7f1d84041070 con 0x55c028bc2af0 2024-09-11T23:36:36.399 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.399+0000 7f1d94be9740 10 client.27353 fetch_fsmap learned FSMap version 84 2024-09-11T23:36:36.400 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.399+0000 7f1d94be9740 1 -- 172.21.15.26:0/4152335027 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=84,monmap=4+}) -- 0x55c028c48520 con 0x55c028bc2af0 2024-09-11T23:36:36.400 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.400+0000 7f1d6affd640 1 -- 172.21.15.26:0/4152335027 <== mon.2 v2:172.21.15.76:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d7c09b3a0 con 0x55c028bc2af0 2024-09-11T23:36:36.400 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.400+0000 7f1d6affd640 1 -- 172.21.15.26:0/4152335027 <== mon.2 v2:172.21.15.76:3300/0 5 ==== fsmap(e 84) ==== 8418+0+0 (secure 0 0 0) 0x7f1d84050840 con 0x55c028bc2af0 2024-09-11T23:36:36.402 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.401+0000 7f1d94be9740 10 client.27353 fetch_fsmap finished waiting for FSMap version 84 2024-09-11T23:36:36.402 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.401+0000 7f1d94be9740 10 client.27353 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:36:36.402 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.401+0000 7f1d94be9740 20 client.27353 populate_metadata read hostname 'smithi026' 2024-09-11T23:36:36.402 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.401+0000 7f1d94be9740 1 --2- 172.21.15.26:0/4152335027 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55c028c48cb0 0x55c028c4b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:36.402 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.401+0000 7f1d94be9740 4 client.27353 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T23:36:36.402 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.401+0000 7f1d94be9740 1 -- 172.21.15.26:0/4152335027 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c028b2ed70 con 0x55c028c48cb0 2024-09-11T23:36:36.402 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.401+0000 7f1d8affd640 1 --2- 172.21.15.26:0/4152335027 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55c028c48cb0 0x55c028c4b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:36.408 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.407+0000 7f1d8affd640 1 --2- 172.21.15.26:0/4152335027 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55c028c48cb0 0x55c028c4b0d0 crc :-1 s=READY pgs=281 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80b2af6ba36d0721 server_cookie=9f5e290540381b1b in_seq=0 out_seq=0 2024-09-11T23:36:36.408 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:36.407+0000 7f1d6affd640 10 client.27353 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:36:38.238 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:36:37 smithi136 ceph-mon[35121]: pgmap v573: 97 pgs: 21 active+clean+snaptrim, 39 active+clean+snaptrim_wait, 37 active+clean; 2.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 168 KiB/s wr, 31 op/s 2024-09-11T23:36:38.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:36:37 smithi026 ceph-mon[31172]: pgmap v573: 97 pgs: 21 active+clean+snaptrim, 39 active+clean+snaptrim_wait, 37 active+clean; 2.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 168 KiB/s wr, 31 op/s 2024-09-11T23:36:38.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:36:37 smithi076 ceph-mon[35296]: pgmap v573: 97 pgs: 21 active+clean+snaptrim, 39 active+clean+snaptrim_wait, 37 active+clean; 2.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 168 KiB/s wr, 31 op/s 2024-09-11T23:36:38.568 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:38.567+0000 7f1d6affd640 1 -- 172.21.15.26:0/4152335027 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c028b2ed70 con 0x55c028c48cb0 2024-09-11T23:36:38.569 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:38.567+0000 7f1d6affd640 10 client.27353 handle_command_reply: tid=0 2024-09-11T23:36:38.569 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:38.567+0000 7f1d94be9740 1 client.27353 shutdown 2024-09-11T23:36:38.569 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:38.567+0000 7f1d94be9740 1 -- 172.21.15.26:0/4152335027 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c028bc2af0 msgr2=0x55c028bd5c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:38.569 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:38.567+0000 7f1d94be9740 1 --2- 172.21.15.26:0/4152335027 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c028bc2af0 0x55c028bd5c90 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f1d84009ca0 tx=0x7f1d8402a8d0 comp rx=0 tx=0).stop 2024-09-11T23:36:38.569 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:38.567+0000 7f1d94be9740 1 -- 172.21.15.26:0/4152335027 shutdown_connections 2024-09-11T23:36:38.569 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:38.567+0000 7f1d94be9740 1 --2- 172.21.15.26:0/4152335027 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55c028c48cb0 0x55c028c4b0d0 crc :-1 s=READY pgs=281 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:38.569 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:38.568+0000 7f1d6affd640 0 client.27353 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:36:38.569 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:38.568+0000 7f1d94be9740 1 --2- 172.21.15.26:0/4152335027 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c028bcdcb0 0x55c028bc6fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:38.569 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:38.568+0000 7f1d94be9740 1 --2- 172.21.15.26:0/4152335027 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c028bc3410 0x55c028bd61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:38.569 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:38.568+0000 7f1d94be9740 1 --2- 172.21.15.26:0/4152335027 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c028bc2af0 0x55c028bd5c90 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:38.569 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:38.568+0000 7f1d94be9740 1 -- 172.21.15.26:0/4152335027 >> 172.21.15.26:0/4152335027 conn(0x55c028bbf060 msgr2=0x55c028bbd9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:38.570 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:38.568+0000 7f1d94be9740 1 -- 172.21.15.26:0/4152335027 shutdown_connections 2024-09-11T23:36:38.570 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:38.568+0000 7f1d94be9740 1 -- 172.21.15.26:0/4152335027 wait complete. 2024-09-11T23:36:38.570 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:38.568+0000 7f1d94be9740 20 client.27353 trim_cache size 0 max 16384 2024-09-11T23:36:38.570 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-09-11T23:36:38.570 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "message": "", 2024-09-11T23:36:38.570 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-09-11T23:36:38.570 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-09-11T23:36:38.571 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:38.569+0000 7f1d91e53640 1 -- 172.21.15.26:0/2658053188 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f1d740a2fd0 msgr2=0x7f1d740a5490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:38.571 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:38.569+0000 7f1d91e53640 1 --2- 172.21.15.26:0/2658053188 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f1d740a2fd0 0x7f1d740a5490 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f1d84004af0 tx=0x7f1d84000f40 comp rx=0 tx=0).stop 2024-09-11T23:36:38.571 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:38.569+0000 7f1d91e53640 1 -- 172.21.15.26:0/2658053188 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1d8c1009e0 msgr2=0x7f1d8c1a4c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:38.571 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:38.569+0000 7f1d91e53640 1 --2- 172.21.15.26:0/2658053188 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1d8c1009e0 0x7f1d8c1a4c10 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f1d7c00d770 tx=0x7f1d7c00dc30 comp rx=0 tx=0).stop 2024-09-11T23:36:38.572 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:38.569+0000 7f1d91e53640 1 -- 172.21.15.26:0/2658053188 shutdown_connections 2024-09-11T23:36:38.572 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:38.569+0000 7f1d91e53640 1 --2- 172.21.15.26:0/2658053188 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f1d740a2fd0 0x7f1d740a5490 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:38.572 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:38.569+0000 7f1d91e53640 1 --2- 172.21.15.26:0/2658053188 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1d8c1009e0 0x7f1d8c1a4c10 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:38.572 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:38.569+0000 7f1d91e53640 1 --2- 172.21.15.26:0/2658053188 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1d8c106be0 0x7f1d8c19e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:38.572 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:38.569+0000 7f1d91e53640 1 --2- 172.21.15.26:0/2658053188 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1d8c0fff60 0x7f1d8c1a46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:38.572 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:38.569+0000 7f1d91e53640 1 -- 172.21.15.26:0/2658053188 >> 172.21.15.26:0/2658053188 conn(0x7f1d8c0770f0 msgr2=0x7f1d8c102500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:38.572 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:38.569+0000 7f1d91e53640 1 -- 172.21.15.26:0/2658053188 shutdown_connections 2024-09-11T23:36:38.572 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:38.569+0000 7f1d91e53640 1 -- 172.21.15.26:0/2658053188 wait complete. 2024-09-11T23:36:39.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:36:39 smithi076 ceph-mon[35296]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:36:39.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:36:39 smithi136 ceph-mon[35121]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:36:39.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:36:39 smithi026 ceph-mon[31172]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:36:40.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:36:40 smithi076 ceph-mon[35296]: pgmap v574: 97 pgs: 21 active+clean+snaptrim, 39 active+clean+snaptrim_wait, 37 active+clean; 2.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 166 KiB/s wr, 30 op/s 2024-09-11T23:36:40.401 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:36:40 smithi026 ceph-mon[31172]: pgmap v574: 97 pgs: 21 active+clean+snaptrim, 39 active+clean+snaptrim_wait, 37 active+clean; 2.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 166 KiB/s wr, 30 op/s 2024-09-11T23:36:40.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:36:40 smithi136 ceph-mon[35121]: pgmap v574: 97 pgs: 21 active+clean+snaptrim, 39 active+clean+snaptrim_wait, 37 active+clean; 2.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 166 KiB/s wr, 30 op/s 2024-09-11T23:36:42.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:36:42 smithi076 ceph-mon[35296]: pgmap v575: 97 pgs: 20 active+clean+snaptrim, 37 active+clean+snaptrim_wait, 40 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 584 KiB/s rd, 559 KiB/s wr, 102 op/s 2024-09-11T23:36:42.441 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:36:42 smithi026 ceph-mon[31172]: pgmap v575: 97 pgs: 20 active+clean+snaptrim, 37 active+clean+snaptrim_wait, 40 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 584 KiB/s rd, 559 KiB/s wr, 102 op/s 2024-09-11T23:36:42.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:36:42 smithi136 ceph-mon[35121]: pgmap v575: 97 pgs: 20 active+clean+snaptrim, 37 active+clean+snaptrim_wait, 40 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 584 KiB/s rd, 559 KiB/s wr, 102 op/s 2024-09-11T23:36:44.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:36:44 smithi076 ceph-mon[35296]: pgmap v576: 97 pgs: 20 active+clean+snaptrim, 37 active+clean+snaptrim_wait, 40 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 583 KiB/s rd, 561 KiB/s wr, 111 op/s 2024-09-11T23:36:44.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:36:44 smithi136 ceph-mon[35121]: pgmap v576: 97 pgs: 20 active+clean+snaptrim, 37 active+clean+snaptrim_wait, 40 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 583 KiB/s rd, 561 KiB/s wr, 111 op/s 2024-09-11T23:36:44.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:36:44 smithi026 ceph-mon[31172]: pgmap v576: 97 pgs: 20 active+clean+snaptrim, 37 active+clean+snaptrim_wait, 40 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 583 KiB/s rd, 561 KiB/s wr, 111 op/s 2024-09-11T23:36:46.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:36:46 smithi076 ceph-mon[35296]: pgmap v577: 97 pgs: 18 active+clean+snaptrim, 35 active+clean+snaptrim_wait, 44 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 606 KiB/s rd, 567 KiB/s wr, 135 op/s 2024-09-11T23:36:46.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:36:46 smithi136 ceph-mon[35121]: pgmap v577: 97 pgs: 18 active+clean+snaptrim, 35 active+clean+snaptrim_wait, 44 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 606 KiB/s rd, 567 KiB/s wr, 135 op/s 2024-09-11T23:36:46.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:36:46 smithi026 ceph-mon[31172]: pgmap v577: 97 pgs: 18 active+clean+snaptrim, 35 active+clean+snaptrim_wait, 44 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 606 KiB/s rd, 567 KiB/s wr, 135 op/s 2024-09-11T23:36:48.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:36:48 smithi076 ceph-mon[35296]: pgmap v578: 97 pgs: 16 active+clean+snaptrim, 32 active+clean+snaptrim_wait, 49 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 878 KiB/s rd, 804 KiB/s wr, 166 op/s 2024-09-11T23:36:48.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:36:48 smithi136 ceph-mon[35121]: pgmap v578: 97 pgs: 16 active+clean+snaptrim, 32 active+clean+snaptrim_wait, 49 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 878 KiB/s rd, 804 KiB/s wr, 166 op/s 2024-09-11T23:36:48.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:36:48 smithi026 ceph-mon[31172]: pgmap v578: 97 pgs: 16 active+clean+snaptrim, 32 active+clean+snaptrim_wait, 49 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 878 KiB/s rd, 804 KiB/s wr, 166 op/s 2024-09-11T23:36:50.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:36:50 smithi076 ceph-mon[35296]: pgmap v579: 97 pgs: 16 active+clean+snaptrim, 32 active+clean+snaptrim_wait, 49 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 841 KiB/s rd, 726 KiB/s wr, 156 op/s 2024-09-11T23:36:50.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:36:50 smithi136 ceph-mon[35121]: pgmap v579: 97 pgs: 16 active+clean+snaptrim, 32 active+clean+snaptrim_wait, 49 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 841 KiB/s rd, 726 KiB/s wr, 156 op/s 2024-09-11T23:36:50.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:36:50 smithi026 ceph-mon[31172]: pgmap v579: 97 pgs: 16 active+clean+snaptrim, 32 active+clean+snaptrim_wait, 49 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 841 KiB/s rd, 726 KiB/s wr, 156 op/s 2024-09-11T23:36:51.715 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.714+0000 7f4e6c59a640 1 -- 172.21.15.26:0/3095036693 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4e641008e0 msgr2=0x7f4e6410ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:51.715 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.714+0000 7f4e6c59a640 1 --2- 172.21.15.26:0/3095036693 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4e641008e0 0x7f4e6410ea40 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c009ca0 tx=0x7f4e5c02a8c0 comp rx=0 tx=0).stop 2024-09-11T23:36:51.715 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.714+0000 7f4e6c59a640 1 -- 172.21.15.26:0/3095036693 shutdown_connections 2024-09-11T23:36:51.715 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.714+0000 7f4e6c59a640 1 --2- 172.21.15.26:0/3095036693 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4e641008e0 0x7f4e6410ea40 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:51.715 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.714+0000 7f4e6c59a640 1 --2- 172.21.15.26:0/3095036693 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4e640fff60 0x7f4e641003a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:51.715 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.714+0000 7f4e6c59a640 1 --2- 172.21.15.26:0/3095036693 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4e64103aa0 0x7f4e64103e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:51.716 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.714+0000 7f4e6c59a640 1 -- 172.21.15.26:0/3095036693 >> 172.21.15.26:0/3095036693 conn(0x7f4e640770f0 msgr2=0x7f4e640fe480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:51.716 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.714+0000 7f4e6c59a640 1 -- 172.21.15.26:0/3095036693 shutdown_connections 2024-09-11T23:36:51.716 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.714+0000 7f4e6c59a640 1 -- 172.21.15.26:0/3095036693 wait complete. 2024-09-11T23:36:51.716 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.715+0000 7f4e6c59a640 1 Processor -- start 2024-09-11T23:36:51.716 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.716+0000 7f4e6c59a640 1 -- start start 2024-09-11T23:36:51.717 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.716+0000 7f4e6c59a640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4e640fff60 0x7f4e64198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:51.717 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.716+0000 7f4e6c59a640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4e641008e0 0x7f4e64198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:51.717 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.716+0000 7f4e6c59a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4e64103aa0 0x7f4e6419cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:51.717 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.716+0000 7f4e6a336640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4e640fff60 0x7f4e64198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:51.717 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.716+0000 7f4e6a336640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4e640fff60 0x7f4e64198500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:37796/0 (socket says 172.21.15.26:37796) 2024-09-11T23:36:51.717 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.716+0000 7f4e6a336640 1 -- 172.21.15.26:0/2350700692 learned_addr learned my addr 172.21.15.26:0/2350700692 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:36:51.717 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.716+0000 7f4e6c59a640 1 -- 172.21.15.26:0/2350700692 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f4e64112050 con 0x7f4e64103aa0 2024-09-11T23:36:51.717 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.716+0000 7f4e6c59a640 1 -- 172.21.15.26:0/2350700692 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f4e64111ed0 con 0x7f4e640fff60 2024-09-11T23:36:51.718 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.716+0000 7f4e6c59a640 1 -- 172.21.15.26:0/2350700692 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f4e641121d0 con 0x7f4e641008e0 2024-09-11T23:36:51.718 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.717+0000 7f4e6ab37640 1 --2- 172.21.15.26:0/2350700692 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4e64103aa0 0x7f4e6419cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:51.718 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.717+0000 7f4e69b35640 1 --2- 172.21.15.26:0/2350700692 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4e641008e0 0x7f4e64198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:51.718 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.717+0000 7f4e6ab37640 1 -- 172.21.15.26:0/2350700692 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4e641008e0 msgr2=0x7f4e64198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:51.718 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.717+0000 7f4e6ab37640 1 --2- 172.21.15.26:0/2350700692 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4e641008e0 0x7f4e64198a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:51.719 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.717+0000 7f4e6ab37640 1 -- 172.21.15.26:0/2350700692 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4e640fff60 msgr2=0x7f4e64198500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:51.719 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.717+0000 7f4e6ab37640 1 --2- 172.21.15.26:0/2350700692 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4e640fff60 0x7f4e64198500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:51.719 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.717+0000 7f4e6ab37640 1 -- 172.21.15.26:0/2350700692 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e6419d4f0 con 0x7f4e64103aa0 2024-09-11T23:36:51.719 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.718+0000 7f4e6ab37640 1 --2- 172.21.15.26:0/2350700692 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4e64103aa0 0x7f4e6419cdc0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c005e30 tx=0x7f4e5c030c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:51.721 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.720+0000 7f4e5b7fe640 1 -- 172.21.15.26:0/2350700692 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e5c02f380 con 0x7f4e64103aa0 2024-09-11T23:36:51.721 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.720+0000 7f4e5b7fe640 1 -- 172.21.15.26:0/2350700692 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e5c003e50 con 0x7f4e64103aa0 2024-09-11T23:36:51.721 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.720+0000 7f4e5b7fe640 1 -- 172.21.15.26:0/2350700692 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e5c004420 con 0x7f4e64103aa0 2024-09-11T23:36:51.721 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.720+0000 7f4e6c59a640 1 -- 172.21.15.26:0/2350700692 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e6419d780 con 0x7f4e64103aa0 2024-09-11T23:36:51.721 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.720+0000 7f4e6c59a640 1 -- 172.21.15.26:0/2350700692 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e641a5070 con 0x7f4e64103aa0 2024-09-11T23:36:51.724 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.723+0000 7f4e6f40e740 1 Processor -- start 2024-09-11T23:36:51.726 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.726+0000 7f4e5b7fe640 1 -- 172.21.15.26:0/2350700692 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f4e5c0316e0 con 0x7f4e64103aa0 2024-09-11T23:36:51.727 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.726+0000 7f4e6f40e740 1 -- start start 2024-09-11T23:36:51.727 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.727+0000 7f4e6f40e740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559a00656440 0x559a00654c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:51.727 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.727+0000 7f4e6f40e740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559a00655140 0x559a00655580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:51.727 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.727+0000 7f4e6f40e740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559a0065fa90 0x559a00661e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:51.727 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.727+0000 7f4e6f40e740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x559a005b92f0 con 0x559a00655140 2024-09-11T23:36:51.728 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.727+0000 7f4e6f40e740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x559a00561270 con 0x559a0065fa90 2024-09-11T23:36:51.728 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.727+0000 7f4e6f40e740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x559a005b52b0 con 0x559a00656440 2024-09-11T23:36:51.728 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.728+0000 7f4e5b7fe640 1 --2- 172.21.15.26:0/2350700692 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f4e500a2f80 0x7f4e500a5440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:51.728 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.728+0000 7f4e5b7fe640 1 -- 172.21.15.26:0/2350700692 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 8586+0+0 (secure 0 0 0) 0x7f4e5c0f05f0 con 0x7f4e64103aa0 2024-09-11T23:36:51.730 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.730+0000 7f4e6a336640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559a00655140 0x559a00655580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:51.730 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.730+0000 7f4e6a336640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559a00655140 0x559a00655580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44048/0 (socket says 172.21.15.26:44048) 2024-09-11T23:36:51.730 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.730+0000 7f4e6a336640 1 -- 172.21.15.26:0/2997927152 learned_addr learned my addr 172.21.15.26:0/2997927152 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:36:51.731 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.730+0000 7f4e69b35640 1 --2- 172.21.15.26:0/2997927152 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559a0065fa90 0x559a00661e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:51.731 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.730+0000 7f4e69b35640 1 --2- 172.21.15.26:0/2997927152 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559a00656440 0x559a00654c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:51.731 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.730+0000 7f4e6a336640 1 -- 172.21.15.26:0/2997927152 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559a00656440 msgr2=0x559a00654c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:36:51.732 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.730+0000 7f4e6a336640 1 --2- 172.21.15.26:0/2997927152 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559a00656440 0x559a00654c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:51.732 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.730+0000 7f4e6a336640 1 -- 172.21.15.26:0/2997927152 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559a0065fa90 msgr2=0x559a00661e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:36:51.732 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.730+0000 7f4e6a336640 1 --2- 172.21.15.26:0/2997927152 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559a0065fa90 0x559a00661e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:51.732 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.730+0000 7f4e6a336640 1 -- 172.21.15.26:0/2997927152 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a00560340 con 0x559a00655140 2024-09-11T23:36:51.732 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.731+0000 7f4e6a336640 1 --2- 172.21.15.26:0/2997927152 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559a00655140 0x559a00655580 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f4e6000bd40 tx=0x7f4e600107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58350985a335b3d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:51.733 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.732+0000 7f4e6ab37640 1 --2- 172.21.15.26:0/2350700692 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f4e500a2f80 0x7f4e500a5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:51.733 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.732+0000 7f4e6c59a640 1 -- 172.21.15.26:0/2997927152 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e60015a50 con 0x559a00655140 2024-09-11T23:36:51.733 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.733+0000 7f4e6c59a640 1 -- 172.21.15.26:0/2997927152 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e60015bf0 con 0x559a00655140 2024-09-11T23:36:51.734 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.733+0000 7f4e6c59a640 1 -- 172.21.15.26:0/2997927152 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e6001f3e0 con 0x559a00655140 2024-09-11T23:36:51.735 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.734+0000 7f4e6f40e740 1 -- 172.21.15.26:0/2997927152 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559a00655140 msgr2=0x559a00655580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:51.735 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.734+0000 7f4e6f40e740 1 --2- 172.21.15.26:0/2997927152 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559a00655140 0x559a00655580 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f4e6000bd40 tx=0x7f4e600107a0 comp rx=0 tx=0).stop 2024-09-11T23:36:51.735 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.734+0000 7f4e6f40e740 1 -- 172.21.15.26:0/2997927152 shutdown_connections 2024-09-11T23:36:51.735 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.734+0000 7f4e6f40e740 1 --2- 172.21.15.26:0/2997927152 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559a0065fa90 0x559a00661e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:51.735 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.734+0000 7f4e6f40e740 1 --2- 172.21.15.26:0/2997927152 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559a00655140 0x559a00655580 secure :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f4e6000bd40 tx=0x7f4e600107a0 comp rx=0 tx=0).stop 2024-09-11T23:36:51.735 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.734+0000 7f4e6f40e740 1 --2- 172.21.15.26:0/2997927152 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559a00656440 0x559a00654c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:51.735 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.734+0000 7f4e6f40e740 1 -- 172.21.15.26:0/2997927152 >> 172.21.15.26:0/2997927152 conn(0x559a00651380 msgr2=0x559a00651760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:51.735 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.734+0000 7f4e6ab37640 1 --2- 172.21.15.26:0/2350700692 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f4e500a2f80 0x7f4e500a5440 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c0ede10 tx=0x7f4e5c0095c0 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:51.735 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.734+0000 7f4e6f40e740 1 -- 172.21.15.26:0/2997927152 shutdown_connections 2024-09-11T23:36:51.736 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.736+0000 7f4e6f40e740 1 -- 172.21.15.26:0/2997927152 wait complete. 2024-09-11T23:36:51.736 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.736+0000 7f4e6f40e740 1 -- start start 2024-09-11T23:36:51.737 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.736+0000 7f4e6f40e740 1 Processor -- start 2024-09-11T23:36:51.738 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.736+0000 7f4e6f40e740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559a00656440 0x559a0065cc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:51.738 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.737+0000 7f4e6f40e740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559a0065fa90 0x559a0065d190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:51.738 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.737+0000 7f4e6f40e740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559a0065d720 0x559a0067ce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:51.738 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.737+0000 7f4e6f40e740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x559a00673980 con 0x559a00656440 2024-09-11T23:36:51.738 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.737+0000 7f4e6f40e740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x559a00542290 con 0x559a0065d720 2024-09-11T23:36:51.738 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.737+0000 7f4e6f40e740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x559a00673b00 con 0x559a0065fa90 2024-09-11T23:36:51.739 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.737+0000 7f4e6f40e740 10 client.0 _ll_register_callbacks cb 0x559a00573140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:36:51.739 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.737+0000 7f4e69b35640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559a00656440 0x559a0065cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:51.739 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.737+0000 7f4e69b35640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559a00656440 0x559a0065cc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44062/0 (socket says 172.21.15.26:44062) 2024-09-11T23:36:51.739 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.737+0000 7f4e69b35640 1 -- 172.21.15.26:0/2872006808 learned_addr learned my addr 172.21.15.26:0/2872006808 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:36:51.739 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.737+0000 7f4e69b35640 1 --2- 172.21.15.26:0/2872006808 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559a0065d720 0x559a0067ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:51.739 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.737+0000 7f4e69b35640 1 -- 172.21.15.26:0/2872006808 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559a0065fa90 msgr2=0x559a0065d190 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T23:36:51.739 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.737+0000 7f4e69b35640 1 --2- 172.21.15.26:0/2872006808 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559a0065fa90 0x559a0065d190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:51.739 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.737+0000 7f4e69b35640 1 -- 172.21.15.26:0/2872006808 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559a00656440 msgr2=0x559a0065cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:36:51.739 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.737+0000 7f4e69b35640 1 --2- 172.21.15.26:0/2872006808 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559a00656440 0x559a0065cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:51.739 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.737+0000 7f4e69b35640 1 -- 172.21.15.26:0/2872006808 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a002cc210 con 0x559a0065d720 2024-09-11T23:36:51.739 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.738+0000 7f4e69b35640 1 --2- 172.21.15.26:0/2872006808 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559a0065d720 0x559a0067ce80 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f4e54013c60 tx=0x7f4e54005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bf87754c2e30df44 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:51.740 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.738+0000 7f4e4dffb640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-11T23:36:51.741 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.738+0000 7f4e4dffb640 1 -- 172.21.15.26:0/2872006808 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e5401c040 con 0x559a0065d720 2024-09-11T23:36:51.741 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.738+0000 7f4e6f40e740 1 -- 172.21.15.26:0/2872006808 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4e60015bf0 con 0x559a0065d720 2024-09-11T23:36:51.741 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.738+0000 7f4e4dffb640 1 -- 172.21.15.26:0/2872006808 <== mon.2 v2:172.21.15.76:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 8586+0+0 (secure 0 0 0) 0x7f4e54010e00 con 0x559a0065d720 2024-09-11T23:36:51.741 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.739+0000 7f4e4dffb640 1 -- 172.21.15.26:0/2872006808 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7f4e54013e10 con 0x559a0065d720 2024-09-11T23:36:51.741 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.739+0000 7f4e6f40e740 10 client.27365 fetch_fsmap learned FSMap version 85 2024-09-11T23:36:51.741 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.739+0000 7f4e6f40e740 1 -- 172.21.15.26:0/2872006808 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=85,monmap=4+}) -- 0x559a00398620 con 0x559a0065d720 2024-09-11T23:36:51.741 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.739+0000 7f4e4dffb640 1 -- 172.21.15.26:0/2872006808 <== mon.2 v2:172.21.15.76:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e60015bf0 con 0x559a0065d720 2024-09-11T23:36:51.741 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.739+0000 7f4e4dffb640 1 -- 172.21.15.26:0/2872006808 <== mon.2 v2:172.21.15.76:3300/0 5 ==== fsmap(e 85) ==== 8422+0+0 (secure 0 0 0) 0x7f4e5400f040 con 0x559a0065d720 2024-09-11T23:36:51.742 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.739+0000 7f4e6f40e740 10 client.27365 fetch_fsmap finished waiting for FSMap version 85 2024-09-11T23:36:51.742 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.739+0000 7f4e6f40e740 10 client.27365 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:36:51.742 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.739+0000 7f4e6f40e740 20 client.27365 populate_metadata read hostname 'smithi026' 2024-09-11T23:36:51.742 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.739+0000 7f4e6f40e740 1 --2- 172.21.15.26:0/2872006808 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x559a00681520 0x559a00683980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:51.742 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.739+0000 7f4e6f40e740 4 client.27365 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:36:51.742 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.739+0000 7f4e6f40e740 1 -- 172.21.15.26:0/2872006808 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559a005c6a20 con 0x559a00681520 2024-09-11T23:36:51.742 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.740+0000 7f4e6a336640 1 --2- 172.21.15.26:0/2872006808 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x559a00681520 0x559a00683980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:51.745 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.740+0000 7f4e4dffb640 10 client.27365 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:36:51.745 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.741+0000 7f4e6a336640 1 --2- 172.21.15.26:0/2872006808 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x559a00681520 0x559a00683980 crc :-1 s=READY pgs=282 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a3ace5b6c9a3e92 server_cookie=483d2339aab3ebd1 in_seq=0 out_seq=0 2024-09-11T23:36:51.745 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.742+0000 7f4e4dffb640 1 -- 172.21.15.26:0/2872006808 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559a005c6a20 con 0x559a00681520 2024-09-11T23:36:51.745 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.742+0000 7f4e4dffb640 10 client.27365 handle_command_reply: tid=0 2024-09-11T23:36:51.745 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.742+0000 7f4e6f40e740 1 client.27365 shutdown 2024-09-11T23:36:51.745 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.742+0000 7f4e6f40e740 1 -- 172.21.15.26:0/2872006808 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559a0065d720 msgr2=0x559a0067ce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:51.746 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.742+0000 7f4e6f40e740 1 --2- 172.21.15.26:0/2872006808 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559a0065d720 0x559a0067ce80 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f4e54013c60 tx=0x7f4e54005f90 comp rx=0 tx=0).stop 2024-09-11T23:36:51.746 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.743+0000 7f4e6f40e740 1 -- 172.21.15.26:0/2872006808 shutdown_connections 2024-09-11T23:36:51.746 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.743+0000 7f4e6f40e740 1 --2- 172.21.15.26:0/2872006808 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559a0065d720 0x559a0067ce80 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:51.746 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.743+0000 7f4e6f40e740 1 --2- 172.21.15.26:0/2872006808 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x559a00681520 0x559a00683980 crc :-1 s=READY pgs=282 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:51.746 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.743+0000 7f4e6f40e740 1 --2- 172.21.15.26:0/2872006808 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559a0065fa90 0x559a0065d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:51.746 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.743+0000 7f4e6f40e740 1 --2- 172.21.15.26:0/2872006808 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559a00656440 0x559a0065cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:51.747 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.743+0000 7f4e6f40e740 1 -- 172.21.15.26:0/2872006808 >> 172.21.15.26:0/2872006808 conn(0x559a00651380 msgr2=0x559a006570e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:51.747 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.743+0000 7f4e4dffb640 0 client.27365 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:36:51.747 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.746+0000 7f4e6f40e740 1 -- 172.21.15.26:0/2872006808 shutdown_connections 2024-09-11T23:36:51.748 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.748+0000 7f4e6f40e740 1 -- 172.21.15.26:0/2872006808 wait complete. 2024-09-11T23:36:51.749 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.749+0000 7f4e6f40e740 20 client.27365 trim_cache size 0 max 16384 2024-09-11T23:36:51.776 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.774+0000 7f4e6f40e740 1 Processor -- start 2024-09-11T23:36:51.776 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.774+0000 7f4e6f40e740 1 -- start start 2024-09-11T23:36:51.777 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.774+0000 7f4e6f40e740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559a00656440 0x559a0065d180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:51.777 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.775+0000 7f4e6f40e740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559a0065d720 0x559a0067cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:51.777 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.775+0000 7f4e6f40e740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559a0065fa90 0x559a00689290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:51.777 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.775+0000 7f4e6f40e740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f4e640720d0 con 0x559a0065fa90 2024-09-11T23:36:51.777 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.775+0000 7f4e6f40e740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f4e64073eb0 con 0x559a00656440 2024-09-11T23:36:51.777 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.775+0000 7f4e6f40e740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f4e64072830 con 0x559a0065d720 2024-09-11T23:36:51.777 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.775+0000 7f4e69b35640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559a00656440 0x559a0065d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:51.777 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.775+0000 7f4e69b35640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559a00656440 0x559a0065d180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:37822/0 (socket says 172.21.15.26:37822) 2024-09-11T23:36:51.778 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.775+0000 7f4e6a336640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559a0065d720 0x559a0067cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:51.778 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.775+0000 7f4e69b35640 1 -- 172.21.15.26:0/1542836067 learned_addr learned my addr 172.21.15.26:0/1542836067 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:36:51.778 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.775+0000 7f4e69b35640 1 --2- 172.21.15.26:0/1542836067 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559a0065fa90 0x559a00689290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:51.778 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.775+0000 7f4e6a336640 1 -- 172.21.15.26:0/1542836067 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559a00656440 msgr2=0x559a0065d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:36:51.778 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.775+0000 7f4e6a336640 1 --2- 172.21.15.26:0/1542836067 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559a00656440 0x559a0065d180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:51.778 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.775+0000 7f4e6a336640 1 -- 172.21.15.26:0/1542836067 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559a0065fa90 msgr2=0x559a00689290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:36:51.778 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.775+0000 7f4e6a336640 1 --2- 172.21.15.26:0/1542836067 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559a0065fa90 0x559a00689290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:51.778 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.775+0000 7f4e6a336640 1 -- 172.21.15.26:0/1542836067 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a0064c210 con 0x559a0065d720 2024-09-11T23:36:51.778 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.775+0000 7f4e6a336640 1 --2- 172.21.15.26:0/1542836067 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559a0065d720 0x559a0067cc70 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x559a006522e0 tx=0x7f4e6001eb10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4aa4ae3ed8c5ee60 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:51.779 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.776+0000 7f4e4ffff640 1 -- 172.21.15.26:0/1542836067 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e6001ec10 con 0x559a0065d720 2024-09-11T23:36:51.779 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.776+0000 7f4e4ffff640 1 -- 172.21.15.26:0/1542836067 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e6001edb0 con 0x559a0065d720 2024-09-11T23:36:51.779 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.776+0000 7f4e4ffff640 1 -- 172.21.15.26:0/1542836067 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e60015ab0 con 0x559a0065d720 2024-09-11T23:36:51.782 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.777+0000 7f4e6f40e740 1 -- 172.21.15.26:0/1542836067 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559a0065d720 msgr2=0x559a0067cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:51.782 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.777+0000 7f4e6f40e740 1 --2- 172.21.15.26:0/1542836067 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559a0065d720 0x559a0067cc70 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x559a006522e0 tx=0x7f4e6001eb10 comp rx=0 tx=0).stop 2024-09-11T23:36:51.783 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.777+0000 7f4e6f40e740 1 -- 172.21.15.26:0/1542836067 shutdown_connections 2024-09-11T23:36:51.783 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.777+0000 7f4e6f40e740 1 --2- 172.21.15.26:0/1542836067 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559a0065fa90 0x559a00689290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:51.783 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.777+0000 7f4e6f40e740 1 --2- 172.21.15.26:0/1542836067 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559a0065d720 0x559a0067cc70 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:51.783 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.777+0000 7f4e6f40e740 1 --2- 172.21.15.26:0/1542836067 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559a00656440 0x559a0065d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:51.783 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.777+0000 7f4e6f40e740 1 -- 172.21.15.26:0/1542836067 >> 172.21.15.26:0/1542836067 conn(0x559a00651380 msgr2=0x559a0065ecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:51.783 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.777+0000 7f4e6f40e740 1 -- 172.21.15.26:0/1542836067 shutdown_connections 2024-09-11T23:36:51.783 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.777+0000 7f4e6f40e740 1 -- 172.21.15.26:0/1542836067 wait complete. 2024-09-11T23:36:51.783 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.777+0000 7f4e6f40e740 1 -- start start 2024-09-11T23:36:51.783 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.778+0000 7f4e6f40e740 1 Processor -- start 2024-09-11T23:36:51.783 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.778+0000 7f4e6f40e740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559a00656440 0x559a0065a870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:51.783 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.778+0000 7f4e6f40e740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559a0065d720 0x559a00678de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:51.783 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.778+0000 7f4e6f40e740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559a0065fa90 0x559a00679360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:51.784 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.778+0000 7f4e6a336640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559a0065d720 0x559a00678de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:51.784 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.778+0000 7f4e6a336640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559a0065d720 0x559a00678de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:49072/0 (socket says 172.21.15.26:49072) 2024-09-11T23:36:51.784 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.779+0000 7f4e69b35640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559a00656440 0x559a0065a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:51.784 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.779+0000 7f4e69b35640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559a00656440 0x559a0065a870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44078/0 (socket says 172.21.15.26:44078) 2024-09-11T23:36:51.784 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.779+0000 7f4e6a336640 1 -- 172.21.15.26:0/4270147172 learned_addr learned my addr 172.21.15.26:0/4270147172 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:36:51.784 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.779+0000 7f4e6f40e740 1 -- 172.21.15.26:0/4270147172 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f4e64073720 con 0x559a00656440 2024-09-11T23:36:51.784 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.779+0000 7f4e6f40e740 1 -- 172.21.15.26:0/4270147172 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f4e64072f90 con 0x559a0065fa90 2024-09-11T23:36:51.784 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.779+0000 7f4e6f40e740 1 -- 172.21.15.26:0/4270147172 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f4e640700d0 con 0x559a0065d720 2024-09-11T23:36:51.784 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.779+0000 7f4e6f40e740 10 client.0 _ll_register_callbacks cb 0x7f4e54009c60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:36:51.784 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.779+0000 7f4e6a336640 1 -- 172.21.15.26:0/4270147172 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559a0065fa90 msgr2=0x559a00679360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:36:51.784 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.779+0000 7f4e6a336640 1 --2- 172.21.15.26:0/4270147172 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559a0065fa90 0x559a00679360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:51.784 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.779+0000 7f4e6a336640 1 -- 172.21.15.26:0/4270147172 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559a00656440 msgr2=0x559a0065a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:36:51.785 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.779+0000 7f4e6a336640 1 --2- 172.21.15.26:0/4270147172 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559a00656440 0x559a0065a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:51.785 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.779+0000 7f4e6a336640 1 -- 172.21.15.26:0/4270147172 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a00660a20 con 0x559a0065d720 2024-09-11T23:36:51.785 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.780+0000 7f4e6a336640 1 --2- 172.21.15.26:0/4270147172 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559a0065d720 0x559a00678de0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f4e60011010 tx=0x7f4e6000bc40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cee9822ccd200cbd server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:51.785 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.780+0000 7f4e6f40e740 1 -- 172.21.15.26:0/4270147172 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4e6001edb0 con 0x559a0065d720 2024-09-11T23:36:51.785 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.780+0000 7f4e4dffb640 10 client.27331 ms_handle_connect on v2:172.21.15.136:3300/0 2024-09-11T23:36:51.785 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.780+0000 7f4e4dffb640 1 -- 172.21.15.26:0/4270147172 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e6001ec10 con 0x559a0065d720 2024-09-11T23:36:51.785 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.781+0000 7f4e4dffb640 1 -- 172.21.15.26:0/4270147172 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 8586+0+0 (secure 0 0 0) 0x7f4e6001e790 con 0x559a0065d720 2024-09-11T23:36:51.785 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.781+0000 7f4e4dffb640 1 -- 172.21.15.26:0/4270147172 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7f4e6003a050 con 0x559a0065d720 2024-09-11T23:36:51.785 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.781+0000 7f4e6f40e740 10 client.27331 fetch_fsmap learned FSMap version 85 2024-09-11T23:36:51.785 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.781+0000 7f4e6f40e740 1 -- 172.21.15.26:0/4270147172 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({fsmap=85}) -- 0x559a006ec380 con 0x559a0065d720 2024-09-11T23:36:51.785 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.781+0000 7f4e4dffb640 1 -- 172.21.15.26:0/4270147172 <== mon.1 v2:172.21.15.136:3300/0 4 ==== fsmap(e 85) ==== 8422+0+0 (secure 0 0 0) 0x7f4e600050d0 con 0x559a0065d720 2024-09-11T23:36:51.786 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.781+0000 7f4e6f40e740 10 client.27331 fetch_fsmap finished waiting for FSMap version 85 2024-09-11T23:36:51.786 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.781+0000 7f4e6f40e740 10 client.27331 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:36:51.786 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.781+0000 7f4e6f40e740 20 client.27331 populate_metadata read hostname 'smithi026' 2024-09-11T23:36:51.786 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.781+0000 7f4e6f40e740 1 --2- 172.21.15.26:0/4270147172 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x559a006ec880 0x559a006eed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:51.786 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.781+0000 7f4e6f40e740 4 client.27331 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T23:36:51.786 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.782+0000 7f4e6f40e740 1 -- 172.21.15.26:0/4270147172 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f4e6001edb0 con 0x559a006ec880 2024-09-11T23:36:51.786 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.785+0000 7f4e69b35640 1 --2- 172.21.15.26:0/4270147172 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x559a006ec880 0x559a006eed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:51.788 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.787+0000 7f4e69b35640 1 --2- 172.21.15.26:0/4270147172 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x559a006ec880 0x559a006eed00 crc :-1 s=READY pgs=283 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2781e044614e4b7a server_cookie=459cdacf63507ad6 in_seq=0 out_seq=0 2024-09-11T23:36:51.788 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:51.787+0000 7f4e4dffb640 10 client.27331 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:36:52.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:36:52 smithi076 ceph-mon[35296]: pgmap v580: 97 pgs: 14 active+clean+snaptrim, 30 active+clean+snaptrim_wait, 53 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 953 KiB/s wr, 209 op/s 2024-09-11T23:36:52.482 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:52.481+0000 7f4e4dffb640 1 -- 172.21.15.26:0/4270147172 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f4e6001edb0 con 0x559a006ec880 2024-09-11T23:36:52.483 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:52.481+0000 7f4e4dffb640 10 client.27331 handle_command_reply: tid=0 2024-09-11T23:36:52.483 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:52.481+0000 7f4e6f40e740 1 client.27331 shutdown 2024-09-11T23:36:52.483 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:52.482+0000 7f4e6f40e740 1 -- 172.21.15.26:0/4270147172 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559a0065d720 msgr2=0x559a00678de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:52.483 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:52.482+0000 7f4e6f40e740 1 --2- 172.21.15.26:0/4270147172 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559a0065d720 0x559a00678de0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f4e60011010 tx=0x7f4e6000bc40 comp rx=0 tx=0).stop 2024-09-11T23:36:52.483 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:52.482+0000 7f4e6f40e740 1 -- 172.21.15.26:0/4270147172 shutdown_connections 2024-09-11T23:36:52.483 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:52.482+0000 7f4e6f40e740 1 --2- 172.21.15.26:0/4270147172 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559a0065fa90 0x559a00679360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:52.483 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:52.482+0000 7f4e6f40e740 1 --2- 172.21.15.26:0/4270147172 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x559a006ec880 0x559a006eed00 crc :-1 s=READY pgs=283 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:52.483 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:52.482+0000 7f4e6f40e740 1 --2- 172.21.15.26:0/4270147172 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559a0065d720 0x559a00678de0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:52.483 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:52.482+0000 7f4e6f40e740 1 --2- 172.21.15.26:0/4270147172 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559a00656440 0x559a0065a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:52.483 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:52.482+0000 7f4e6f40e740 1 -- 172.21.15.26:0/4270147172 >> 172.21.15.26:0/4270147172 conn(0x559a00651380 msgr2=0x559a006568c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:52.483 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:52.482+0000 7f4e4dffb640 0 client.27331 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:36:52.483 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:52.482+0000 7f4e6f40e740 1 -- 172.21.15.26:0/4270147172 shutdown_connections 2024-09-11T23:36:52.483 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:52.482+0000 7f4e6f40e740 1 -- 172.21.15.26:0/4270147172 wait complete. 2024-09-11T23:36:52.484 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:52.482+0000 7f4e6f40e740 20 client.27331 trim_cache size 0 max 16384 2024-09-11T23:36:52.484 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-09-11T23:36:52.484 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "message": "", 2024-09-11T23:36:52.484 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-09-11T23:36:52.484 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-09-11T23:36:52.485 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:52.483+0000 7f4e593ff640 1 -- 172.21.15.26:0/2350700692 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f4e500a2f80 msgr2=0x7f4e500a5440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:52.485 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:52.483+0000 7f4e593ff640 1 --2- 172.21.15.26:0/2350700692 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f4e500a2f80 0x7f4e500a5440 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c0ede10 tx=0x7f4e5c0095c0 comp rx=0 tx=0).stop 2024-09-11T23:36:52.485 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:52.483+0000 7f4e593ff640 1 -- 172.21.15.26:0/2350700692 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4e64103aa0 msgr2=0x7f4e6419cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:52.485 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:52.483+0000 7f4e593ff640 1 --2- 172.21.15.26:0/2350700692 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4e64103aa0 0x7f4e6419cdc0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c005e30 tx=0x7f4e5c030c30 comp rx=0 tx=0).stop 2024-09-11T23:36:52.485 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:52.483+0000 7f4e593ff640 1 -- 172.21.15.26:0/2350700692 shutdown_connections 2024-09-11T23:36:52.485 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:52.483+0000 7f4e593ff640 1 --2- 172.21.15.26:0/2350700692 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f4e500a2f80 0x7f4e500a5440 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:52.485 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:52.483+0000 7f4e593ff640 1 --2- 172.21.15.26:0/2350700692 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4e64103aa0 0x7f4e6419cdc0 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:52.485 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:52.483+0000 7f4e593ff640 1 --2- 172.21.15.26:0/2350700692 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4e641008e0 0x7f4e64198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:52.485 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:52.483+0000 7f4e593ff640 1 --2- 172.21.15.26:0/2350700692 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4e640fff60 0x7f4e64198500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:52.486 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:52.483+0000 7f4e593ff640 1 -- 172.21.15.26:0/2350700692 >> 172.21.15.26:0/2350700692 conn(0x7f4e640770f0 msgr2=0x7f4e64105a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:52.486 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:52.484+0000 7f4e593ff640 1 -- 172.21.15.26:0/2350700692 shutdown_connections 2024-09-11T23:36:52.486 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:36:52.484+0000 7f4e593ff640 1 -- 172.21.15.26:0/2350700692 wait complete. 2024-09-11T23:36:52.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:36:52 smithi136 ceph-mon[35121]: pgmap v580: 97 pgs: 14 active+clean+snaptrim, 30 active+clean+snaptrim_wait, 53 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 953 KiB/s wr, 209 op/s 2024-09-11T23:36:52.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:36:52 smithi026 ceph-mon[31172]: pgmap v580: 97 pgs: 14 active+clean+snaptrim, 30 active+clean+snaptrim_wait, 53 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 953 KiB/s wr, 209 op/s 2024-09-11T23:36:54.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:36:54 smithi076 ceph-mon[35296]: pgmap v581: 97 pgs: 14 active+clean+snaptrim, 30 active+clean+snaptrim_wait, 53 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 639 KiB/s rd, 560 KiB/s wr, 149 op/s 2024-09-11T23:36:54.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:36:54 smithi136 ceph-mon[35121]: pgmap v581: 97 pgs: 14 active+clean+snaptrim, 30 active+clean+snaptrim_wait, 53 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 639 KiB/s rd, 560 KiB/s wr, 149 op/s 2024-09-11T23:36:54.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:36:54 smithi026 ceph-mon[31172]: pgmap v581: 97 pgs: 14 active+clean+snaptrim, 30 active+clean+snaptrim_wait, 53 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 639 KiB/s rd, 560 KiB/s wr, 149 op/s 2024-09-11T23:36:56.290 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:36:56 smithi136 ceph-mon[35121]: pgmap v582: 97 pgs: 13 active+clean+snaptrim, 30 active+clean+snaptrim_wait, 54 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 731 KiB/s rd, 896 KiB/s wr, 187 op/s 2024-09-11T23:36:56.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:36:56 smithi076 ceph-mon[35296]: pgmap v582: 97 pgs: 13 active+clean+snaptrim, 30 active+clean+snaptrim_wait, 54 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 731 KiB/s rd, 896 KiB/s wr, 187 op/s 2024-09-11T23:36:56.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:36:56 smithi026 ceph-mon[31172]: pgmap v582: 97 pgs: 13 active+clean+snaptrim, 30 active+clean+snaptrim_wait, 54 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 731 KiB/s rd, 896 KiB/s wr, 187 op/s 2024-09-11T23:36:58.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:36:58 smithi076 ceph-mon[35296]: pgmap v583: 97 pgs: 11 active+clean+snaptrim, 22 active+clean+snaptrim_wait, 64 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 867 KiB/s rd, 1.4 MiB/s wr, 219 op/s 2024-09-11T23:36:58.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:36:58 smithi136 ceph-mon[35121]: pgmap v583: 97 pgs: 11 active+clean+snaptrim, 22 active+clean+snaptrim_wait, 64 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 867 KiB/s rd, 1.4 MiB/s wr, 219 op/s 2024-09-11T23:36:58.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:36:58 smithi026 ceph-mon[31172]: pgmap v583: 97 pgs: 11 active+clean+snaptrim, 22 active+clean+snaptrim_wait, 64 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 867 KiB/s rd, 1.4 MiB/s wr, 219 op/s 2024-09-11T23:37:00.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:37:00 smithi076 ceph-mon[35296]: pgmap v584: 97 pgs: 10 active+clean+snaptrim, 17 active+clean+snaptrim_wait, 70 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 537 KiB/s rd, 1.1 MiB/s wr, 193 op/s 2024-09-11T23:37:00.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:37:00 smithi136 ceph-mon[35121]: pgmap v584: 97 pgs: 10 active+clean+snaptrim, 17 active+clean+snaptrim_wait, 70 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 537 KiB/s rd, 1.1 MiB/s wr, 193 op/s 2024-09-11T23:37:00.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:37:00 smithi026 ceph-mon[31172]: pgmap v584: 97 pgs: 10 active+clean+snaptrim, 17 active+clean+snaptrim_wait, 70 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 537 KiB/s rd, 1.1 MiB/s wr, 193 op/s 2024-09-11T23:37:01.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:37:01 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:37:01.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:37:01 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:37:01.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:37:01 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:37:02.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:37:02 smithi076 ceph-mon[35296]: pgmap v585: 97 pgs: 6 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 79 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 263 op/s 2024-09-11T23:37:02.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:37:02 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:37:02.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:37:02 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:37:02.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:37:02 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:37:02.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:37:02 smithi136 ceph-mon[35121]: pgmap v585: 97 pgs: 6 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 79 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 263 op/s 2024-09-11T23:37:02.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:37:02 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:37:02.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:37:02 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:37:02.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:37:02 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:37:02.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:37:02 smithi026 ceph-mon[31172]: pgmap v585: 97 pgs: 6 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 79 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 263 op/s 2024-09-11T23:37:02.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:37:02 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:37:02.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:37:02 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:37:02.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:37:02 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:37:04.305 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:37:04 smithi076 ceph-mon[35296]: pgmap v586: 97 pgs: 4 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 87 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 240 op/s 2024-09-11T23:37:04.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:37:04 smithi136 ceph-mon[35121]: pgmap v586: 97 pgs: 4 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 87 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 240 op/s 2024-09-11T23:37:04.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:37:04 smithi026 ceph-mon[31172]: pgmap v586: 97 pgs: 4 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 87 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 240 op/s 2024-09-11T23:37:05.614 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.614+0000 7f20252a5640 1 -- 172.21.15.26:0/2458491144 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20201069c0 msgr2=0x7f2020106da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:05.614 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.614+0000 7f20252a5640 1 --2- 172.21.15.26:0/2458491144 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20201069c0 0x7f2020106da0 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f2010009890 tx=0x7f201002a380 comp rx=0 tx=0).stop 2024-09-11T23:37:05.614 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.614+0000 7f20252a5640 1 -- 172.21.15.26:0/2458491144 shutdown_connections 2024-09-11T23:37:05.614 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.614+0000 7f20252a5640 1 --2- 172.21.15.26:0/2458491144 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2020101350 0x7f202010d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:05.614 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.614+0000 7f20252a5640 1 --2- 172.21.15.26:0/2458491144 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f20201009b0 0x7f2020100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:05.614 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.614+0000 7f20252a5640 1 --2- 172.21.15.26:0/2458491144 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20201069c0 0x7f2020106da0 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:05.614 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.614+0000 7f20252a5640 1 -- 172.21.15.26:0/2458491144 >> 172.21.15.26:0/2458491144 conn(0x7f2020077990 msgr2=0x7f20200fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:05.615 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.615+0000 7f20252a5640 1 -- 172.21.15.26:0/2458491144 shutdown_connections 2024-09-11T23:37:05.615 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.615+0000 7f20252a5640 1 -- 172.21.15.26:0/2458491144 wait complete. 2024-09-11T23:37:05.615 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.615+0000 7f20252a5640 1 Processor -- start 2024-09-11T23:37:05.615 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.616+0000 7f20252a5640 1 -- start start 2024-09-11T23:37:05.616 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.616+0000 7f20252a5640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f20201009b0 0x7f202010d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:05.616 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.616+0000 7f20252a5640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2020101350 0x7f20201086d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:05.616 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.616+0000 7f20252a5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20201069c0 0x7f2020108c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:05.617 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.616+0000 7f201effd640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f20201009b0 0x7f202010d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:05.617 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.617+0000 7f201effd640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f20201009b0 0x7f202010d650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:35990/0 (socket says 172.21.15.26:35990) 2024-09-11T23:37:05.617 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.617+0000 7f201e7fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2020101350 0x7f20201086d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:05.617 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.617+0000 7f201e7fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2020101350 0x7f20201086d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:54908/0 (socket says 172.21.15.26:54908) 2024-09-11T23:37:05.617 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.617+0000 7f201effd640 1 -- 172.21.15.26:0/3145086846 learned_addr learned my addr 172.21.15.26:0/3145086846 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:37:05.617 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.617+0000 7f20252a5640 1 -- 172.21.15.26:0/3145086846 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f2020069c30 con 0x7f20201069c0 2024-09-11T23:37:05.617 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.617+0000 7f20252a5640 1 -- 172.21.15.26:0/3145086846 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f2020069ab0 con 0x7f2020101350 2024-09-11T23:37:05.617 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.617+0000 7f20252a5640 1 -- 172.21.15.26:0/3145086846 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f2020069db0 con 0x7f20201009b0 2024-09-11T23:37:05.617 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.617+0000 7f201f7fe640 1 --2- 172.21.15.26:0/3145086846 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20201069c0 0x7f2020108c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:05.618 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.618+0000 7f201e7fc640 1 -- 172.21.15.26:0/3145086846 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f20201009b0 msgr2=0x7f202010d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:05.618 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.618+0000 7f201e7fc640 1 --2- 172.21.15.26:0/3145086846 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f20201009b0 0x7f202010d650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:05.618 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.618+0000 7f201e7fc640 1 -- 172.21.15.26:0/3145086846 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20201069c0 msgr2=0x7f2020108c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:05.618 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.618+0000 7f201e7fc640 1 --2- 172.21.15.26:0/3145086846 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20201069c0 0x7f2020108c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:05.618 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.618+0000 7f201e7fc640 1 -- 172.21.15.26:0/3145086846 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20201094d0 con 0x7f2020101350 2024-09-11T23:37:05.618 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.618+0000 7f201e7fc640 1 --2- 172.21.15.26:0/3145086846 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2020101350 0x7f20201086d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f2014005fd0 tx=0x7f201400ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:05.618 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.618+0000 7f2007fff640 1 -- 172.21.15.26:0/3145086846 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f201400cd00 con 0x7f2020101350 2024-09-11T23:37:05.618 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.619+0000 7f2007fff640 1 -- 172.21.15.26:0/3145086846 <== mon.2 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20140035f0 con 0x7f2020101350 2024-09-11T23:37:05.619 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.619+0000 7f20252a5640 1 -- 172.21.15.26:0/3145086846 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20201ad170 con 0x7f2020101350 2024-09-11T23:37:05.619 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.619+0000 7f20252a5640 1 -- 172.21.15.26:0/3145086846 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20201ad6e0 con 0x7f2020101350 2024-09-11T23:37:05.619 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.619+0000 7f2007fff640 1 -- 172.21.15.26:0/3145086846 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2014010670 con 0x7f2020101350 2024-09-11T23:37:05.622 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.622+0000 7f2028045740 1 Processor -- start 2024-09-11T23:37:05.622 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.623+0000 7f2028045740 1 -- start start 2024-09-11T23:37:05.622 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.623+0000 7f2028045740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cf9149caf0 0x55cf9149ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:05.623 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.623+0000 7f2028045740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cf9149d410 0x55cf914a7770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:05.623 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.623+0000 7f2028045740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cf914a7cb0 0x55cf914aa0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:05.623 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.623+0000 7f2028045740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55cf914012f0 con 0x55cf914a7cb0 2024-09-11T23:37:05.623 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.623+0000 7f2028045740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55cf913a9270 con 0x55cf9149d410 2024-09-11T23:37:05.623 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.623+0000 7f2028045740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55cf913fd2b0 con 0x55cf9149caf0 2024-09-11T23:37:05.623 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.623+0000 7f201f7fe640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cf9149caf0 0x55cf9149ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:05.623 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.623+0000 7f201f7fe640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cf9149caf0 0x55cf9149ced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:36000/0 (socket says 172.21.15.26:36000) 2024-09-11T23:37:05.623 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.623+0000 7f201f7fe640 1 -- 172.21.15.26:0/1110913205 learned_addr learned my addr 172.21.15.26:0/1110913205 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:37:05.624 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.623+0000 7f2007fff640 1 -- 172.21.15.26:0/3145086846 <== mon.2 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f2014010810 con 0x7f2020101350 2024-09-11T23:37:05.624 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.624+0000 7f201e7fc640 1 --2- 172.21.15.26:0/1110913205 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cf914a7cb0 0x55cf914aa0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:05.624 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.624+0000 7f201f7fe640 1 -- 172.21.15.26:0/1110913205 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cf9149d410 msgr2=0x55cf914a7770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:05.624 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.624+0000 7f201f7fe640 1 --2- 172.21.15.26:0/1110913205 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cf9149d410 0x55cf914a7770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:05.624 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.624+0000 7f201f7fe640 1 -- 172.21.15.26:0/1110913205 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cf914a7cb0 msgr2=0x55cf914aa0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:05.625 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.625+0000 7f201f7fe640 1 --2- 172.21.15.26:0/1110913205 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cf914a7cb0 0x55cf914aa0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:05.625 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.625+0000 7f201f7fe640 1 -- 172.21.15.26:0/1110913205 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf913a8340 con 0x55cf9149caf0 2024-09-11T23:37:05.625 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.625+0000 7f2007fff640 1 --2- 172.21.15.26:0/3145086846 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f20080a2ea0 0x7f20080a5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:05.625 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.625+0000 7f201effd640 1 --2- 172.21.15.26:0/3145086846 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f20080a2ea0 0x7f20080a5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:05.625 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.625+0000 7f2007fff640 1 -- 172.21.15.26:0/3145086846 <== mon.2 v2:172.21.15.76:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 8586+0+0 (secure 0 0 0) 0x7f20140cf4d0 con 0x7f2020101350 2024-09-11T23:37:05.625 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.625+0000 7f201f7fe640 1 --2- 172.21.15.26:0/1110913205 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cf9149caf0 0x55cf9149ced0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f2018009f50 tx=0x7f20180118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c74df09ffeaccb2d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:05.625 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.625+0000 7f20057fa640 1 -- 172.21.15.26:0/1110913205 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2018004d20 con 0x55cf9149caf0 2024-09-11T23:37:05.625 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.625+0000 7f20057fa640 1 -- 172.21.15.26:0/1110913205 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2018004050 con 0x55cf9149caf0 2024-09-11T23:37:05.627 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.626+0000 7f20057fa640 1 -- 172.21.15.26:0/1110913205 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2018005280 con 0x55cf9149caf0 2024-09-11T23:37:05.627 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.626+0000 7f2028045740 1 -- 172.21.15.26:0/1110913205 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cf9149caf0 msgr2=0x55cf9149ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:05.627 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.626+0000 7f2028045740 1 --2- 172.21.15.26:0/1110913205 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cf9149caf0 0x55cf9149ced0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f2018009f50 tx=0x7f20180118d0 comp rx=0 tx=0).stop 2024-09-11T23:37:05.627 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.626+0000 7f201effd640 1 --2- 172.21.15.26:0/3145086846 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f20080a2ea0 0x7f20080a5360 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f20100095f0 tx=0x7f2010005da0 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:05.627 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.627+0000 7f2028045740 1 -- 172.21.15.26:0/1110913205 shutdown_connections 2024-09-11T23:37:05.627 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.627+0000 7f2028045740 1 --2- 172.21.15.26:0/1110913205 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cf9149d410 0x55cf914a7770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:05.627 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.627+0000 7f2028045740 1 --2- 172.21.15.26:0/1110913205 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cf914a7cb0 0x55cf914aa0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:05.627 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.627+0000 7f2028045740 1 --2- 172.21.15.26:0/1110913205 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cf9149caf0 0x55cf9149ced0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:05.627 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.627+0000 7f2028045740 1 -- 172.21.15.26:0/1110913205 >> 172.21.15.26:0/1110913205 conn(0x55cf91499060 msgr2=0x55cf91499440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:05.627 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.627+0000 7f2028045740 1 -- 172.21.15.26:0/1110913205 shutdown_connections 2024-09-11T23:37:05.628 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.627+0000 7f2028045740 1 -- 172.21.15.26:0/1110913205 wait complete. 2024-09-11T23:37:05.628 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.627+0000 7f2028045740 1 -- start start 2024-09-11T23:37:05.628 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.627+0000 7f2028045740 1 Processor -- start 2024-09-11T23:37:05.628 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.628+0000 7f2028045740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cf9149caf0 0x55cf914a4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:05.628 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.628+0000 7f2028045740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cf9149d410 0x55cf914a4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:05.629 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.628+0000 7f2028045740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cf914a7cb0 0x55cf914a5530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:05.629 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.628+0000 7f2028045740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55cf914aed30 con 0x55cf9149caf0 2024-09-11T23:37:05.629 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.628+0000 7f2028045740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55cf9138a290 con 0x55cf9149d410 2024-09-11T23:37:05.629 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.628+0000 7f2028045740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55cf914aeeb0 con 0x55cf914a7cb0 2024-09-11T23:37:05.629 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.628+0000 7f201e7fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cf9149d410 0x55cf914a4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:05.629 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.628+0000 7f201effd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cf9149caf0 0x55cf914a4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:05.629 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.628+0000 7f2028045740 10 client.0 _ll_register_callbacks cb 0x55cf913bb140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:37:05.629 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.628+0000 7f201f7fe640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cf914a7cb0 0x55cf914a5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:05.629 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.628+0000 7f201f7fe640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cf914a7cb0 0x55cf914a5530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:36012/0 (socket says 172.21.15.26:36012) 2024-09-11T23:37:05.629 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.628+0000 7f201f7fe640 1 -- 172.21.15.26:0/1080566866 learned_addr learned my addr 172.21.15.26:0/1080566866 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:37:05.629 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.628+0000 7f201e7fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cf9149d410 0x55cf914a4ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:54928/0 (socket says 172.21.15.26:54928) 2024-09-11T23:37:05.629 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.628+0000 7f201effd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cf9149caf0 0x55cf914a4ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48080/0 (socket says 172.21.15.26:48080) 2024-09-11T23:37:05.630 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.628+0000 7f201f7fe640 1 -- 172.21.15.26:0/1080566866 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cf9149d410 msgr2=0x55cf914a4ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:05.630 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.628+0000 7f201f7fe640 1 --2- 172.21.15.26:0/1080566866 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cf9149d410 0x55cf914a4ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:05.630 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.628+0000 7f201f7fe640 1 -- 172.21.15.26:0/1080566866 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cf9149caf0 msgr2=0x55cf914a4ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:05.630 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.628+0000 7f201f7fe640 1 --2- 172.21.15.26:0/1080566866 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cf9149caf0 0x55cf914a4ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:05.630 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.628+0000 7f201f7fe640 1 -- 172.21.15.26:0/1080566866 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf91114210 con 0x55cf914a7cb0 2024-09-11T23:37:05.630 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.628+0000 7f201effd640 1 --2- 172.21.15.26:0/1080566866 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cf9149caf0 0x55cf914a4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:37:05.630 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.629+0000 7f201f7fe640 1 --2- 172.21.15.26:0/1080566866 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cf914a7cb0 0x55cf914a5530 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f2018012fd0 tx=0x7f2018009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8131cde654c4868c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:05.630 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.629+0000 7f2002ff5640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-09-11T23:37:05.632 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.630+0000 7f2002ff5640 1 -- 172.21.15.26:0/1080566866 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2018028020 con 0x55cf914a7cb0 2024-09-11T23:37:05.632 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.630+0000 7f2028045740 1 -- 172.21.15.26:0/1080566866 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2018004050 con 0x55cf914a7cb0 2024-09-11T23:37:05.632 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.630+0000 7f2002ff5640 1 -- 172.21.15.26:0/1080566866 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 8586+0+0 (secure 0 0 0) 0x7f2018005050 con 0x55cf914a7cb0 2024-09-11T23:37:05.632 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.630+0000 7f2002ff5640 1 -- 172.21.15.26:0/1080566866 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7f2018005270 con 0x55cf914a7cb0 2024-09-11T23:37:05.632 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.630+0000 7f2028045740 10 client.27343 fetch_fsmap learned FSMap version 85 2024-09-11T23:37:05.632 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.630+0000 7f2028045740 1 -- 172.21.15.26:0/1080566866 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=85,monmap=4+}) -- 0x55cf911e0620 con 0x55cf914a7cb0 2024-09-11T23:37:05.632 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.631+0000 7f2002ff5640 1 -- 172.21.15.26:0/1080566866 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2018004050 con 0x55cf914a7cb0 2024-09-11T23:37:05.632 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.631+0000 7f2002ff5640 1 -- 172.21.15.26:0/1080566866 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 85) ==== 8422+0+0 (secure 0 0 0) 0x7f2018024970 con 0x55cf914a7cb0 2024-09-11T23:37:05.632 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.631+0000 7f2028045740 10 client.27343 fetch_fsmap finished waiting for FSMap version 85 2024-09-11T23:37:05.632 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.631+0000 7f2028045740 10 client.27343 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:37:05.633 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.631+0000 7f2028045740 20 client.27343 populate_metadata read hostname 'smithi026' 2024-09-11T23:37:05.633 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.631+0000 7f2028045740 1 --2- 172.21.15.26:0/1080566866 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55cf914ab9d0 0x55cf914addf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:05.633 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.632+0000 7f2028045740 4 client.27343 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:37:05.633 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.632+0000 7f2028045740 1 -- 172.21.15.26:0/1080566866 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cf9140ea20 con 0x55cf914ab9d0 2024-09-11T23:37:05.633 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.633+0000 7f201effd640 1 --2- 172.21.15.26:0/1080566866 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55cf914ab9d0 0x55cf914addf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:05.633 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.634+0000 7f2002ff5640 10 client.27343 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:37:05.633 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.634+0000 7f201effd640 1 --2- 172.21.15.26:0/1080566866 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55cf914ab9d0 0x55cf914addf0 crc :-1 s=READY pgs=284 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b31b6406ca1d43ef server_cookie=64fa6f84d776b99f in_seq=0 out_seq=0 2024-09-11T23:37:05.635 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.635+0000 7f2002ff5640 1 -- 172.21.15.26:0/1080566866 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55cf9140ea20 con 0x55cf914ab9d0 2024-09-11T23:37:05.635 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.635+0000 7f2002ff5640 10 client.27343 handle_command_reply: tid=0 2024-09-11T23:37:05.635 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.635+0000 7f2028045740 1 client.27343 shutdown 2024-09-11T23:37:05.635 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.635+0000 7f2028045740 1 -- 172.21.15.26:0/1080566866 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cf914a7cb0 msgr2=0x55cf914a5530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:05.635 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.635+0000 7f2028045740 1 --2- 172.21.15.26:0/1080566866 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cf914a7cb0 0x55cf914a5530 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f2018012fd0 tx=0x7f2018009ca0 comp rx=0 tx=0).stop 2024-09-11T23:37:05.635 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.635+0000 7f2028045740 1 -- 172.21.15.26:0/1080566866 shutdown_connections 2024-09-11T23:37:05.635 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.635+0000 7f2028045740 1 --2- 172.21.15.26:0/1080566866 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cf9149d410 0x55cf914a4ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:05.635 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.635+0000 7f2028045740 1 --2- 172.21.15.26:0/1080566866 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55cf914ab9d0 0x55cf914addf0 crc :-1 s=READY pgs=284 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:05.636 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.635+0000 7f2002ff5640 0 client.27343 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:37:05.636 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.635+0000 7f2028045740 1 --2- 172.21.15.26:0/1080566866 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cf914a7cb0 0x55cf914a5530 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:05.636 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.635+0000 7f2028045740 1 --2- 172.21.15.26:0/1080566866 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cf9149caf0 0x55cf914a4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:05.636 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.635+0000 7f2028045740 1 -- 172.21.15.26:0/1080566866 >> 172.21.15.26:0/1080566866 conn(0x55cf91499060 msgr2=0x55cf9149ed70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:05.636 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.635+0000 7f2028045740 1 -- 172.21.15.26:0/1080566866 shutdown_connections 2024-09-11T23:37:05.636 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.635+0000 7f2028045740 1 -- 172.21.15.26:0/1080566866 wait complete. 2024-09-11T23:37:05.636 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.636+0000 7f2028045740 20 client.27343 trim_cache size 0 max 16384 2024-09-11T23:37:05.660 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.661+0000 7f2028045740 1 Processor -- start 2024-09-11T23:37:05.660 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.661+0000 7f2028045740 1 -- start start 2024-09-11T23:37:05.662 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.661+0000 7f2028045740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cf9149caf0 0x55cf914bfc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:05.662 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.661+0000 7f2028045740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cf9149d410 0x55cf914c0190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:05.662 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.662+0000 7f2028045740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cf914a7cb0 0x55cf914979a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:05.662 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.662+0000 7f201effd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cf9149caf0 0x55cf914bfc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:05.662 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.662+0000 7f201effd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cf9149caf0 0x55cf914bfc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48086/0 (socket says 172.21.15.26:48086) 2024-09-11T23:37:05.662 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.662+0000 7f201e7fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cf9149d410 0x55cf914c0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:05.662 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.662+0000 7f201e7fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cf9149d410 0x55cf914c0190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:54938/0 (socket says 172.21.15.26:54938) 2024-09-11T23:37:05.662 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.662+0000 7f201effd640 1 -- 172.21.15.26:0/1343355727 learned_addr learned my addr 172.21.15.26:0/1343355727 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:37:05.662 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.662+0000 7f2028045740 1 -- 172.21.15.26:0/1343355727 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f200c00e870 con 0x55cf9149caf0 2024-09-11T23:37:05.662 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.662+0000 7f201f7fe640 1 --2- 172.21.15.26:0/1343355727 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cf914a7cb0 0x55cf914979a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:05.663 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.662+0000 7f2028045740 1 -- 172.21.15.26:0/1343355727 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f200c010650 con 0x55cf9149d410 2024-09-11T23:37:05.663 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.662+0000 7f2028045740 1 -- 172.21.15.26:0/1343355727 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f200c00efd0 con 0x55cf914a7cb0 2024-09-11T23:37:05.663 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.663+0000 7f201effd640 1 -- 172.21.15.26:0/1343355727 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cf914a7cb0 msgr2=0x55cf914979a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:05.663 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.663+0000 7f201effd640 1 --2- 172.21.15.26:0/1343355727 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cf914a7cb0 0x55cf914979a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:05.664 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.663+0000 7f201effd640 1 -- 172.21.15.26:0/1343355727 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cf9149d410 msgr2=0x55cf914c0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:05.664 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.663+0000 7f201effd640 1 --2- 172.21.15.26:0/1343355727 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cf9149d410 0x55cf914c0190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:05.664 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.663+0000 7f201effd640 1 -- 172.21.15.26:0/1343355727 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf9149d240 con 0x55cf9149caf0 2024-09-11T23:37:05.664 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.663+0000 7f201effd640 1 --2- 172.21.15.26:0/1343355727 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cf9149caf0 0x55cf914bfc50 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f2010035040 tx=0x7f201003c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4be6c22f8613c4af server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:05.664 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.664+0000 7f2004ff9640 1 -- 172.21.15.26:0/1343355727 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2010037040 con 0x55cf9149caf0 2024-09-11T23:37:05.664 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.664+0000 7f2004ff9640 1 -- 172.21.15.26:0/1343355727 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f201005e070 con 0x55cf9149caf0 2024-09-11T23:37:05.664 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.664+0000 7f2004ff9640 1 -- 172.21.15.26:0/1343355727 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20100329a0 con 0x55cf9149caf0 2024-09-11T23:37:05.665 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.665+0000 7f2028045740 1 -- 172.21.15.26:0/1343355727 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cf9149caf0 msgr2=0x55cf914bfc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:05.665 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.665+0000 7f2028045740 1 --2- 172.21.15.26:0/1343355727 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cf9149caf0 0x55cf914bfc50 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f2010035040 tx=0x7f201003c000 comp rx=0 tx=0).stop 2024-09-11T23:37:05.665 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.665+0000 7f2028045740 1 -- 172.21.15.26:0/1343355727 shutdown_connections 2024-09-11T23:37:05.666 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.665+0000 7f2028045740 1 --2- 172.21.15.26:0/1343355727 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cf9149d410 0x55cf914c0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:05.669 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.665+0000 7f2028045740 1 --2- 172.21.15.26:0/1343355727 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cf914a7cb0 0x55cf914979a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:05.669 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.665+0000 7f2028045740 1 --2- 172.21.15.26:0/1343355727 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cf9149caf0 0x55cf914bfc50 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:05.669 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.665+0000 7f2028045740 1 -- 172.21.15.26:0/1343355727 >> 172.21.15.26:0/1343355727 conn(0x55cf91499060 msgr2=0x55cf914a3b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:05.670 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.668+0000 7f2028045740 1 -- 172.21.15.26:0/1343355727 shutdown_connections 2024-09-11T23:37:05.670 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.668+0000 7f2028045740 1 -- 172.21.15.26:0/1343355727 wait complete. 2024-09-11T23:37:05.670 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.668+0000 7f2028045740 1 -- start start 2024-09-11T23:37:05.671 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.669+0000 7f2028045740 1 Processor -- start 2024-09-11T23:37:05.671 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.669+0000 7f2028045740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cf9149caf0 0x55cf914a8480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:05.671 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.669+0000 7f2028045740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cf9149d410 0x55cf914a89c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:05.671 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.669+0000 7f2028045740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cf914a7cb0 0x55cf914a8f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:05.671 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.669+0000 7f2028045740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f200c00fec0 con 0x55cf914a7cb0 2024-09-11T23:37:05.671 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.669+0000 7f2028045740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f200c00f730 con 0x55cf9149caf0 2024-09-11T23:37:05.671 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.669+0000 7f2028045740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f200c00c870 con 0x55cf9149d410 2024-09-11T23:37:05.671 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.669+0000 7f2028045740 10 client.0 _ll_register_callbacks cb 0x7f2018009e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:37:05.671 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.669+0000 7f201effd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cf9149caf0 0x55cf914a8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:05.671 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.669+0000 7f201f7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cf914a7cb0 0x55cf914a8f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:05.671 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.669+0000 7f201effd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cf9149caf0 0x55cf914a8480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:54946/0 (socket says 172.21.15.26:54946) 2024-09-11T23:37:05.671 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.669+0000 7f201effd640 1 -- 172.21.15.26:0/738527333 learned_addr learned my addr 172.21.15.26:0/738527333 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:37:05.672 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.669+0000 7f201e7fc640 1 --2- 172.21.15.26:0/738527333 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cf9149d410 0x55cf914a89c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:05.672 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.670+0000 7f201f7fe640 1 -- 172.21.15.26:0/738527333 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cf9149d410 msgr2=0x55cf914a89c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:05.672 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.670+0000 7f201f7fe640 1 --2- 172.21.15.26:0/738527333 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cf9149d410 0x55cf914a89c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:05.672 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.670+0000 7f201f7fe640 1 -- 172.21.15.26:0/738527333 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cf9149caf0 msgr2=0x55cf914a8480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:05.672 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.670+0000 7f201f7fe640 1 --2- 172.21.15.26:0/738527333 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cf9149caf0 0x55cf914a8480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:05.672 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.670+0000 7f201f7fe640 1 -- 172.21.15.26:0/738527333 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf914b3470 con 0x55cf914a7cb0 2024-09-11T23:37:05.672 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.670+0000 7f201e7fc640 1 --2- 172.21.15.26:0/738527333 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cf9149d410 0x55cf914a89c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:37:05.672 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.670+0000 7f201f7fe640 1 --2- 172.21.15.26:0/738527333 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cf914a7cb0 0x55cf914a8f00 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f2018024f80 tx=0x7f2018004720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f23e2d2f571c28d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:05.672 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.670+0000 7f2002ff5640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:37:05.672 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.670+0000 7f2002ff5640 1 -- 172.21.15.26:0/738527333 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2018028040 con 0x55cf914a7cb0 2024-09-11T23:37:05.672 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.671+0000 7f2002ff5640 1 -- 172.21.15.26:0/738527333 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 8586+0+0 (secure 0 0 0) 0x7f201800f920 con 0x55cf914a7cb0 2024-09-11T23:37:05.672 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.671+0000 7f2028045740 1 -- 172.21.15.26:0/738527333 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f201005e070 con 0x55cf914a7cb0 2024-09-11T23:37:05.672 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.671+0000 7f2002ff5640 1 -- 172.21.15.26:0/738527333 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7f20180050a0 con 0x55cf914a7cb0 2024-09-11T23:37:05.675 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.671+0000 7f2028045740 10 client.17874 fetch_fsmap learned FSMap version 85 2024-09-11T23:37:05.675 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.671+0000 7f2028045740 1 -- 172.21.15.26:0/738527333 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=85,monmap=4+}) -- 0x55cf914b4780 con 0x55cf914a7cb0 2024-09-11T23:37:05.675 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.672+0000 7f2002ff5640 1 -- 172.21.15.26:0/738527333 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f201005e070 con 0x55cf914a7cb0 2024-09-11T23:37:05.675 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.672+0000 7f2002ff5640 1 -- 172.21.15.26:0/738527333 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 85) ==== 8422+0+0 (secure 0 0 0) 0x7f2018005950 con 0x55cf914a7cb0 2024-09-11T23:37:05.675 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.672+0000 7f2028045740 10 client.17874 fetch_fsmap finished waiting for FSMap version 85 2024-09-11T23:37:05.675 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.672+0000 7f2028045740 10 client.17874 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:37:05.675 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.672+0000 7f2028045740 20 client.17874 populate_metadata read hostname 'smithi026' 2024-09-11T23:37:05.676 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.672+0000 7f2028045740 1 --2- 172.21.15.26:0/738527333 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55cf91522c20 0x55cf91525010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:05.676 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.672+0000 7f2028045740 4 client.17874 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T23:37:05.676 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.672+0000 7f2028045740 1 -- 172.21.15.26:0/738527333 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55cf91408d70 con 0x55cf91522c20 2024-09-11T23:37:05.676 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.672+0000 7f201effd640 1 --2- 172.21.15.26:0/738527333 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55cf91522c20 0x55cf91525010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:05.676 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.673+0000 7f2002ff5640 10 client.17874 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:37:05.676 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.673+0000 7f201effd640 1 --2- 172.21.15.26:0/738527333 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55cf91522c20 0x55cf91525010 crc :-1 s=READY pgs=285 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3696c5d96ed438aa server_cookie=104f77bd613fee5 in_seq=0 out_seq=0 2024-09-11T23:37:05.732 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.732+0000 7f2002ff5640 1 -- 172.21.15.26:0/738527333 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55cf91408d70 con 0x55cf91522c20 2024-09-11T23:37:05.732 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.732+0000 7f2002ff5640 10 client.17874 handle_command_reply: tid=0 2024-09-11T23:37:05.732 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.732+0000 7f2028045740 1 client.17874 shutdown 2024-09-11T23:37:05.732 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.732+0000 7f2028045740 1 -- 172.21.15.26:0/738527333 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cf914a7cb0 msgr2=0x55cf914a8f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:05.732 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.732+0000 7f2028045740 1 --2- 172.21.15.26:0/738527333 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cf914a7cb0 0x55cf914a8f00 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f2018024f80 tx=0x7f2018004720 comp rx=0 tx=0).stop 2024-09-11T23:37:05.732 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.732+0000 7f2028045740 1 -- 172.21.15.26:0/738527333 shutdown_connections 2024-09-11T23:37:05.733 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.732+0000 7f2028045740 1 --2- 172.21.15.26:0/738527333 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55cf91522c20 0x55cf91525010 crc :-1 s=READY pgs=285 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:05.733 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.732+0000 7f2028045740 1 --2- 172.21.15.26:0/738527333 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cf914a7cb0 0x55cf914a8f00 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:05.733 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.732+0000 7f2028045740 1 --2- 172.21.15.26:0/738527333 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cf9149d410 0x55cf914a89c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:05.733 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.732+0000 7f2028045740 1 --2- 172.21.15.26:0/738527333 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cf9149caf0 0x55cf914a8480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:05.733 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.732+0000 7f2028045740 1 -- 172.21.15.26:0/738527333 >> 172.21.15.26:0/738527333 conn(0x55cf91499060 msgr2=0x55cf914a6310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:05.733 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.732+0000 7f2002ff5640 0 client.17874 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:37:05.733 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.732+0000 7f2028045740 1 -- 172.21.15.26:0/738527333 shutdown_connections 2024-09-11T23:37:05.733 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.732+0000 7f2028045740 1 -- 172.21.15.26:0/738527333 wait complete. 2024-09-11T23:37:05.733 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.732+0000 7f2028045740 20 client.17874 trim_cache size 0 max 16384 2024-09-11T23:37:05.733 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-09-11T23:37:05.734 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "message": "", 2024-09-11T23:37:05.734 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-09-11T23:37:05.734 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-09-11T23:37:05.735 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.733+0000 7f20252a5640 1 -- 172.21.15.26:0/3145086846 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f20080a2ea0 msgr2=0x7f20080a5360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:05.735 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.733+0000 7f20252a5640 1 --2- 172.21.15.26:0/3145086846 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f20080a2ea0 0x7f20080a5360 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f20100095f0 tx=0x7f2010005da0 comp rx=0 tx=0).stop 2024-09-11T23:37:05.735 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.733+0000 7f20252a5640 1 -- 172.21.15.26:0/3145086846 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2020101350 msgr2=0x7f20201086d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:05.735 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.733+0000 7f20252a5640 1 --2- 172.21.15.26:0/3145086846 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2020101350 0x7f20201086d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f2014005fd0 tx=0x7f201400ee50 comp rx=0 tx=0).stop 2024-09-11T23:37:05.735 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.733+0000 7f20252a5640 1 -- 172.21.15.26:0/3145086846 shutdown_connections 2024-09-11T23:37:05.735 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.733+0000 7f20252a5640 1 --2- 172.21.15.26:0/3145086846 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f20080a2ea0 0x7f20080a5360 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:05.735 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.733+0000 7f20252a5640 1 --2- 172.21.15.26:0/3145086846 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2020101350 0x7f20201086d0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:05.735 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.733+0000 7f20252a5640 1 --2- 172.21.15.26:0/3145086846 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20201069c0 0x7f2020108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:05.735 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.733+0000 7f20252a5640 1 --2- 172.21.15.26:0/3145086846 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f20201009b0 0x7f202010d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:05.735 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.733+0000 7f20252a5640 1 -- 172.21.15.26:0/3145086846 >> 172.21.15.26:0/3145086846 conn(0x7f2020077990 msgr2=0x7f2020103340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:05.735 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.733+0000 7f20252a5640 1 -- 172.21.15.26:0/3145086846 shutdown_connections 2024-09-11T23:37:05.735 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:05.733+0000 7f20252a5640 1 -- 172.21.15.26:0/3145086846 wait complete. 2024-09-11T23:37:06.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:37:06 smithi076 ceph-mon[35296]: pgmap v587: 97 pgs: 3 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 88 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.5 MiB/s wr, 279 op/s 2024-09-11T23:37:06.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:37:06 smithi136 ceph-mon[35121]: pgmap v587: 97 pgs: 3 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 88 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.5 MiB/s wr, 279 op/s 2024-09-11T23:37:06.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:37:06 smithi026 ceph-mon[31172]: pgmap v587: 97 pgs: 3 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 88 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.5 MiB/s wr, 279 op/s 2024-09-11T23:37:08.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:37:08 smithi136 ceph-mon[35121]: pgmap v588: 97 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 89 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.6 MiB/s wr, 316 op/s 2024-09-11T23:37:08.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:37:08 smithi026 ceph-mon[31172]: pgmap v588: 97 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 89 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.6 MiB/s wr, 316 op/s 2024-09-11T23:37:08.552 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:37:08 smithi076 ceph-mon[35296]: pgmap v588: 97 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 89 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.6 MiB/s wr, 316 op/s 2024-09-11T23:37:10.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:37:10 smithi136 ceph-mon[35121]: pgmap v589: 97 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 89 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.1 MiB/s wr, 273 op/s 2024-09-11T23:37:10.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:37:10 smithi136 ceph-mon[35121]: osdmap e110: 12 total, 12 up, 12 in 2024-09-11T23:37:10.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:37:10 smithi026 ceph-mon[31172]: pgmap v589: 97 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 89 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.1 MiB/s wr, 273 op/s 2024-09-11T23:37:10.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:37:10 smithi026 ceph-mon[31172]: osdmap e110: 12 total, 12 up, 12 in 2024-09-11T23:37:10.552 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:37:10 smithi076 ceph-mon[35296]: pgmap v589: 97 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 89 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.1 MiB/s wr, 273 op/s 2024-09-11T23:37:10.552 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:37:10 smithi076 ceph-mon[35296]: osdmap e110: 12 total, 12 up, 12 in 2024-09-11T23:37:12.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:37:12 smithi136 ceph-mon[35121]: pgmap v591: 97 pgs: 15 active+clean+snaptrim, 26 active+clean+snaptrim_wait, 56 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.6 MiB/s wr, 363 op/s 2024-09-11T23:37:12.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:37:12 smithi026 ceph-mon[31172]: pgmap v591: 97 pgs: 15 active+clean+snaptrim, 26 active+clean+snaptrim_wait, 56 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.6 MiB/s wr, 363 op/s 2024-09-11T23:37:12.552 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:37:12 smithi076 ceph-mon[35296]: pgmap v591: 97 pgs: 15 active+clean+snaptrim, 26 active+clean+snaptrim_wait, 56 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.6 MiB/s wr, 363 op/s 2024-09-11T23:37:14.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:37:14 smithi136 ceph-mon[35121]: pgmap v592: 97 pgs: 21 active+clean+snaptrim, 39 active+clean+snaptrim_wait, 37 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.6 MiB/s wr, 365 op/s 2024-09-11T23:37:14.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:37:14 smithi136 ceph-mon[35121]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:37:14.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:37:14 smithi026 ceph-mon[31172]: pgmap v592: 97 pgs: 21 active+clean+snaptrim, 39 active+clean+snaptrim_wait, 37 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.6 MiB/s wr, 365 op/s 2024-09-11T23:37:14.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:37:14 smithi026 ceph-mon[31172]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:37:14.552 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:37:14 smithi076 ceph-mon[35296]: pgmap v592: 97 pgs: 21 active+clean+snaptrim, 39 active+clean+snaptrim_wait, 37 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.6 MiB/s wr, 365 op/s 2024-09-11T23:37:14.552 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:37:14 smithi076 ceph-mon[35296]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:37:16.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:37:16 smithi136 ceph-mon[35121]: pgmap v593: 97 pgs: 23 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 33 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.5 MiB/s wr, 328 op/s 2024-09-11T23:37:16.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:37:16 smithi136 ceph-mon[35121]: osdmap e111: 12 total, 12 up, 12 in 2024-09-11T23:37:16.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:37:16 smithi026 ceph-mon[31172]: pgmap v593: 97 pgs: 23 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 33 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.5 MiB/s wr, 328 op/s 2024-09-11T23:37:16.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:37:16 smithi026 ceph-mon[31172]: osdmap e111: 12 total, 12 up, 12 in 2024-09-11T23:37:16.552 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:37:16 smithi076 ceph-mon[35296]: pgmap v593: 97 pgs: 23 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 33 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.5 MiB/s wr, 328 op/s 2024-09-11T23:37:16.552 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:37:16 smithi076 ceph-mon[35296]: osdmap e111: 12 total, 12 up, 12 in 2024-09-11T23:37:18.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:37:18 smithi136 ceph-mon[35121]: pgmap v595: 97 pgs: 23 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 33 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.3 MiB/s wr, 303 op/s 2024-09-11T23:37:18.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:37:18 smithi026 ceph-mon[31172]: pgmap v595: 97 pgs: 23 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 33 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.3 MiB/s wr, 303 op/s 2024-09-11T23:37:18.552 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:37:18 smithi076 ceph-mon[35296]: pgmap v595: 97 pgs: 23 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 33 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.3 MiB/s wr, 303 op/s 2024-09-11T23:37:18.898 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:18.898+0000 7f4ebad34640 1 -- 172.21.15.26:0/1664748993 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4eac0959f0 msgr2=0x7f4eac095e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:18.898 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:18.898+0000 7f4ebad34640 1 --2- 172.21.15.26:0/1664748993 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4eac0959f0 0x7f4eac095e50 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f4eb0009930 tx=0x7f4eb002a480 comp rx=0 tx=0).stop 2024-09-11T23:37:18.904 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:18.904+0000 7f4ebad34640 1 -- 172.21.15.26:0/1664748993 shutdown_connections 2024-09-11T23:37:18.904 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:18.904+0000 7f4ebad34640 1 --2- 172.21.15.26:0/1664748993 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4eac0959f0 0x7f4eac095e50 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:18.904 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:18.904+0000 7f4ebad34640 1 --2- 172.21.15.26:0/1664748993 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4eac096390 0x7f4eac0a2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:18.905 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:18.905+0000 7f4ebad34640 1 --2- 172.21.15.26:0/1664748993 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4eac09b740 0x7f4eac09bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:18.905 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:18.905+0000 7f4ebad34640 1 -- 172.21.15.26:0/1664748993 >> 172.21.15.26:0/1664748993 conn(0x7f4eac00c6c0 msgr2=0x7f4eac00cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:18.910 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:18.910+0000 7f4ebad34640 1 -- 172.21.15.26:0/1664748993 shutdown_connections 2024-09-11T23:37:18.912 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:18.912+0000 7f4ebad34640 1 -- 172.21.15.26:0/1664748993 wait complete. 2024-09-11T23:37:18.912 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:18.912+0000 7f4ebad34640 1 Processor -- start 2024-09-11T23:37:18.913 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:18.913+0000 7f4ebad34640 1 -- start start 2024-09-11T23:37:18.913 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:18.913+0000 7f4ebad34640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4eac0959f0 0x7f4eac141b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:18.913 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:18.913+0000 7f4ebad34640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4eac096390 0x7f4eac13abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:18.913 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:18.913+0000 7f4ebad34640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4eac09b740 0x7f4eac13b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:18.913 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:18.913+0000 7f4ebad34640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f4eac0a3890 con 0x7f4eac0959f0 2024-09-11T23:37:18.913 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:18.913+0000 7f4ebad34640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f4eac0a3710 con 0x7f4eac09b740 2024-09-11T23:37:18.914 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:18.913+0000 7f4ebad34640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f4eac0a3a10 con 0x7f4eac096390 2024-09-11T23:37:18.915 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:18.915+0000 7f4eb9d32640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4eac0959f0 0x7f4eac141b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:18.915 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:18.916+0000 7f4eb9d32640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4eac0959f0 0x7f4eac141b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51636/0 (socket says 172.21.15.26:51636) 2024-09-11T23:37:18.915 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:18.916+0000 7f4eb9d32640 1 -- 172.21.15.26:0/2245700335 learned_addr learned my addr 172.21.15.26:0/2245700335 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:37:18.916 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:18.916+0000 7f4eb9d32640 1 -- 172.21.15.26:0/2245700335 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4eac096390 msgr2=0x7f4eac13abc0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-11T23:37:18.916 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:18.916+0000 7f4eb9d32640 1 --2- 172.21.15.26:0/2245700335 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4eac096390 0x7f4eac13abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:18.917 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:18.916+0000 7f4eb9d32640 1 -- 172.21.15.26:0/2245700335 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4eac09b740 msgr2=0x7f4eac13b100 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-11T23:37:18.917 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:18.916+0000 7f4eb9d32640 1 --2- 172.21.15.26:0/2245700335 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4eac09b740 0x7f4eac13b100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:18.917 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:18.916+0000 7f4eb9d32640 1 -- 172.21.15.26:0/2245700335 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4eac13b6f0 con 0x7f4eac0959f0 2024-09-11T23:37:18.917 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:18.917+0000 7f4eb9d32640 1 --2- 172.21.15.26:0/2245700335 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4eac0959f0 0x7f4eac141b40 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f4eb4065a40 tx=0x7f4eb406e430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:18.918 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:18.918+0000 7f4eaaffd640 1 -- 172.21.15.26:0/2245700335 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eb406eab0 con 0x7f4eac0959f0 2024-09-11T23:37:18.918 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:18.918+0000 7f4ebad34640 1 -- 172.21.15.26:0/2245700335 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4eac13b9e0 con 0x7f4eac0959f0 2024-09-11T23:37:18.918 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:18.918+0000 7f4ebad34640 1 -- 172.21.15.26:0/2245700335 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4eac13f9e0 con 0x7f4eac0959f0 2024-09-11T23:37:18.919 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:18.918+0000 7f4eaaffd640 1 -- 172.21.15.26:0/2245700335 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4eb4078070 con 0x7f4eac0959f0 2024-09-11T23:37:18.919 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:18.918+0000 7f4eaaffd640 1 -- 172.21.15.26:0/2245700335 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eb407c630 con 0x7f4eac0959f0 2024-09-11T23:37:18.921 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:18.921+0000 7f4ebee08740 1 Processor -- start 2024-09-11T23:37:18.921 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:18.922+0000 7f4ebee08740 1 -- start start 2024-09-11T23:37:18.922 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:18.922+0000 7f4ebee08740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555a9062bcc0 0x555a9062c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:18.922 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:18.922+0000 7f4ebee08740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555a9062c5e0 0x555a90636940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:18.922 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:18.922+0000 7f4ebee08740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555a90636e80 0x555a90639270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:18.922 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:18.922+0000 7f4ebee08740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x555a905902f0 con 0x555a90636e80 2024-09-11T23:37:18.922 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:18.922+0000 7f4ebee08740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x555a90538270 con 0x555a9062bcc0 2024-09-11T23:37:18.922 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:18.922+0000 7f4ebee08740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x555a9058c2b0 con 0x555a9062c5e0 2024-09-11T23:37:18.922 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:18.923+0000 7f4eba533640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555a9062bcc0 0x555a9062c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:18.923 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:18.923+0000 7f4eba533640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555a9062bcc0 0x555a9062c0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:34122/0 (socket says 172.21.15.26:34122) 2024-09-11T23:37:18.923 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:18.923+0000 7f4eba533640 1 -- 172.21.15.26:0/1802464783 learned_addr learned my addr 172.21.15.26:0/1802464783 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:37:18.923 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:18.924+0000 7f4eaaffd640 1 -- 172.21.15.26:0/2245700335 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f4eb406ec50 con 0x7f4eac0959f0 2024-09-11T23:37:18.923 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:18.924+0000 7f4eba533640 1 -- 172.21.15.26:0/1802464783 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555a9062c5e0 msgr2=0x555a90636940 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T23:37:18.924 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:18.924+0000 7f4eba533640 1 --2- 172.21.15.26:0/1802464783 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555a9062c5e0 0x555a90636940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:18.924 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:18.924+0000 7f4eba533640 1 -- 172.21.15.26:0/1802464783 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555a90636e80 msgr2=0x555a90639270 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T23:37:18.924 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:18.924+0000 7f4eba533640 1 --2- 172.21.15.26:0/1802464783 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555a90636e80 0x555a90639270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:18.924 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:18.924+0000 7f4eba533640 1 -- 172.21.15.26:0/1802464783 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a90537340 con 0x555a9062bcc0 2024-09-11T23:37:18.924 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:18.925+0000 7f4eba533640 1 --2- 172.21.15.26:0/1802464783 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555a9062bcc0 0x555a9062c0a0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f4ea4009720 tx=0x7f4ea4009be0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f6b3c354cceec4c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:18.924 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:18.925+0000 7f4eaaffd640 1 --2- 172.21.15.26:0/2245700335 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f4ea00a2dd0 0x7f4ea00a5290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:18.925 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:18.925+0000 7f4eaaffd640 1 -- 172.21.15.26:0/2245700335 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 8586+0+0 (secure 0 0 0) 0x7f4eb4132a90 con 0x7f4eac0959f0 2024-09-11T23:37:18.925 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:18.925+0000 7f4e9ffff640 1 -- 172.21.15.26:0/1802464783 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ea4015020 con 0x555a9062bcc0 2024-09-11T23:37:18.925 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:18.926+0000 7f4eb9d32640 1 --2- 172.21.15.26:0/2245700335 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f4ea00a2dd0 0x7f4ea00a5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:18.926 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:18.926+0000 7f4e9ffff640 1 -- 172.21.15.26:0/1802464783 <== mon.2 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ea4009e30 con 0x555a9062bcc0 2024-09-11T23:37:18.926 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:18.926+0000 7f4e9ffff640 1 -- 172.21.15.26:0/1802464783 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ea40105e0 con 0x555a9062bcc0 2024-09-11T23:37:18.928 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:18.927+0000 7f4ebee08740 1 -- 172.21.15.26:0/1802464783 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555a9062bcc0 msgr2=0x555a9062c0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:18.928 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:18.927+0000 7f4ebee08740 1 --2- 172.21.15.26:0/1802464783 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555a9062bcc0 0x555a9062c0a0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f4ea4009720 tx=0x7f4ea4009be0 comp rx=0 tx=0).stop 2024-09-11T23:37:18.928 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:18.927+0000 7f4eb9d32640 1 --2- 172.21.15.26:0/2245700335 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f4ea00a2dd0 0x7f4ea00a5290 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x555a90629090 tx=0x7f4eb411f040 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:18.928 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:18.928+0000 7f4ebee08740 1 -- 172.21.15.26:0/1802464783 shutdown_connections 2024-09-11T23:37:18.929 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:18.928+0000 7f4ebee08740 1 --2- 172.21.15.26:0/1802464783 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555a90636e80 0x555a90639270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:18.929 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:18.928+0000 7f4ebee08740 1 --2- 172.21.15.26:0/1802464783 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555a9062c5e0 0x555a90636940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:18.929 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:18.928+0000 7f4ebee08740 1 --2- 172.21.15.26:0/1802464783 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555a9062bcc0 0x555a9062c0a0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:18.929 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:18.928+0000 7f4ebee08740 1 -- 172.21.15.26:0/1802464783 >> 172.21.15.26:0/1802464783 conn(0x555a90628230 msgr2=0x555a90628610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:18.932 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:18.932+0000 7f4ebee08740 1 -- 172.21.15.26:0/1802464783 shutdown_connections 2024-09-11T23:37:18.934 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:18.935+0000 7f4ebee08740 1 -- 172.21.15.26:0/1802464783 wait complete. 2024-09-11T23:37:18.935 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:18.936+0000 7f4ebee08740 1 -- start start 2024-09-11T23:37:18.936 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:18.936+0000 7f4ebee08740 1 Processor -- start 2024-09-11T23:37:18.937 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:18.937+0000 7f4ebee08740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555a9062bcc0 0x555a90633d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:18.937 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:18.938+0000 7f4ebee08740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555a9062c5e0 0x555a906342a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:18.938 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:18.938+0000 7f4ebee08740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555a906347e0 0x555a90642fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:18.938 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:18.938+0000 7f4ebee08740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x555a90639a50 con 0x555a906347e0 2024-09-11T23:37:18.938 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:18.938+0000 7f4ebee08740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x555a90519290 con 0x555a9062bcc0 2024-09-11T23:37:18.938 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:18.938+0000 7f4ebee08740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x555a90639bd0 con 0x555a9062c5e0 2024-09-11T23:37:18.938 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:18.938+0000 7f4ebee08740 10 client.0 _ll_register_callbacks cb 0x555a9054a140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:37:18.938 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:18.938+0000 7f4eba533640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555a9062bcc0 0x555a90633d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:18.938 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:18.938+0000 7f4eba533640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555a9062bcc0 0x555a90633d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:34124/0 (socket says 172.21.15.26:34124) 2024-09-11T23:37:18.938 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:18.938+0000 7f4eba533640 1 -- 172.21.15.26:0/3800693481 learned_addr learned my addr 172.21.15.26:0/3800693481 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:37:18.938 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:18.938+0000 7f4eba533640 1 --2- 172.21.15.26:0/3800693481 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555a906347e0 0x555a90642fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:18.939 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:18.938+0000 7f4eba533640 1 -- 172.21.15.26:0/3800693481 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555a9062c5e0 msgr2=0x555a906342a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T23:37:18.939 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:18.938+0000 7f4eba533640 1 --2- 172.21.15.26:0/3800693481 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555a9062c5e0 0x555a906342a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:18.939 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:18.938+0000 7f4eba533640 1 -- 172.21.15.26:0/3800693481 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555a906347e0 msgr2=0x555a90642fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:18.939 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:18.938+0000 7f4eba533640 1 --2- 172.21.15.26:0/3800693481 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555a906347e0 0x555a90642fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:18.939 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:18.939+0000 7f4eba533640 1 -- 172.21.15.26:0/3800693481 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a902a3210 con 0x555a9062bcc0 2024-09-11T23:37:18.940 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:18.941+0000 7f4e9d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-11T23:37:18.941 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:18.941+0000 7f4eba533640 1 --2- 172.21.15.26:0/3800693481 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555a9062bcc0 0x555a90633d60 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f4ea4011cf0 tx=0x7f4ea4006b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=66a8ba67842a89f7 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:18.942 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:18.943+0000 7f4e9d7fa640 1 -- 172.21.15.26:0/3800693481 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ea4015070 con 0x555a9062bcc0 2024-09-11T23:37:18.942 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:18.943+0000 7f4e9d7fa640 1 -- 172.21.15.26:0/3800693481 <== mon.2 v2:172.21.15.76:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 8586+0+0 (secure 0 0 0) 0x7f4ea400bd90 con 0x555a9062bcc0 2024-09-11T23:37:18.942 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:18.943+0000 7f4ebee08740 1 -- 172.21.15.26:0/3800693481 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ea4009e30 con 0x555a9062bcc0 2024-09-11T23:37:18.944 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:18.945+0000 7f4e9d7fa640 1 -- 172.21.15.26:0/3800693481 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f4ea4015070 con 0x555a9062bcc0 2024-09-11T23:37:18.946 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:18.946+0000 7f4ebee08740 10 client.27392 fetch_fsmap learned FSMap version 86 2024-09-11T23:37:18.946 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:18.947+0000 7f4ebee08740 1 -- 172.21.15.26:0/3800693481 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=86,monmap=4+}) -- 0x555a9036f620 con 0x555a9062bcc0 2024-09-11T23:37:18.947 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:18.947+0000 7f4e9d7fa640 1 -- 172.21.15.26:0/3800693481 <== mon.2 v2:172.21.15.76:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ea4009e30 con 0x555a9062bcc0 2024-09-11T23:37:18.948 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:18.948+0000 7f4e9d7fa640 1 -- 172.21.15.26:0/3800693481 <== mon.2 v2:172.21.15.76:3300/0 5 ==== fsmap(e 86) ==== 8426+0+0 (secure 0 0 0) 0x7f4ea401d450 con 0x555a9062bcc0 2024-09-11T23:37:18.948 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:18.948+0000 7f4ebee08740 10 client.27392 fetch_fsmap finished waiting for FSMap version 86 2024-09-11T23:37:18.948 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:18.948+0000 7f4ebee08740 10 client.27392 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:37:18.948 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:18.948+0000 7f4ebee08740 20 client.27392 populate_metadata read hostname 'smithi026' 2024-09-11T23:37:18.948 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:18.948+0000 7f4ebee08740 1 --2- 172.21.15.26:0/3800693481 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x555a906476c0 0x555a90649b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:18.948 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:18.948+0000 7f4ebee08740 4 client.27392 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:37:18.948 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:18.948+0000 7f4ebee08740 1 -- 172.21.15.26:0/3800693481 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555a9059da20 con 0x555a906476c0 2024-09-11T23:37:18.951 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:18.952+0000 7f4eb9531640 1 --2- 172.21.15.26:0/3800693481 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x555a906476c0 0x555a90649b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:18.966 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:18.966+0000 7f4eb9531640 1 --2- 172.21.15.26:0/3800693481 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x555a906476c0 0x555a90649b20 crc :-1 s=READY pgs=286 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5fde33c56aba7086 server_cookie=9aefdf8a08bca65c in_seq=0 out_seq=0 2024-09-11T23:37:18.967 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:18.967+0000 7f4e9d7fa640 10 client.27392 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:37:18.967 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:18.967+0000 7f4e9d7fa640 1 -- 172.21.15.26:0/3800693481 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555a9059da20 con 0x555a906476c0 2024-09-11T23:37:18.967 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:18.967+0000 7f4e9d7fa640 10 client.27392 handle_command_reply: tid=0 2024-09-11T23:37:18.967 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:18.968+0000 7f4ebee08740 1 client.27392 shutdown 2024-09-11T23:37:18.969 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:18.969+0000 7f4ebee08740 1 -- 172.21.15.26:0/3800693481 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555a9062bcc0 msgr2=0x555a90633d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:18.969 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:18.969+0000 7f4ebee08740 1 --2- 172.21.15.26:0/3800693481 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555a9062bcc0 0x555a90633d60 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f4ea4011cf0 tx=0x7f4ea4006b60 comp rx=0 tx=0).stop 2024-09-11T23:37:18.969 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:18.969+0000 7f4ebee08740 1 -- 172.21.15.26:0/3800693481 shutdown_connections 2024-09-11T23:37:18.969 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:18.969+0000 7f4ebee08740 1 --2- 172.21.15.26:0/3800693481 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x555a906476c0 0x555a90649b20 crc :-1 s=READY pgs=286 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:18.969 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:18.969+0000 7f4ebee08740 1 --2- 172.21.15.26:0/3800693481 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555a906347e0 0x555a90642fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:18.969 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:18.969+0000 7f4ebee08740 1 --2- 172.21.15.26:0/3800693481 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555a9062c5e0 0x555a906342a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:18.969 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:18.969+0000 7f4ebee08740 1 --2- 172.21.15.26:0/3800693481 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555a9062bcc0 0x555a90633d60 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:18.969 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:18.969+0000 7f4ebee08740 1 -- 172.21.15.26:0/3800693481 >> 172.21.15.26:0/3800693481 conn(0x555a90628230 msgr2=0x555a9062df40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:18.972 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:18.972+0000 7f4e9d7fa640 0 client.27392 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:37:18.973 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:18.974+0000 7f4ebee08740 1 -- 172.21.15.26:0/3800693481 shutdown_connections 2024-09-11T23:37:18.976 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:18.976+0000 7f4ebee08740 1 -- 172.21.15.26:0/3800693481 wait complete. 2024-09-11T23:37:18.980 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:18.981+0000 7f4ebee08740 20 client.27392 trim_cache size 0 max 16384 2024-09-11T23:37:19.018 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:19.017+0000 7f4ebee08740 1 Processor -- start 2024-09-11T23:37:19.018 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:19.018+0000 7f4ebee08740 1 -- start start 2024-09-11T23:37:19.018 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:19.018+0000 7f4ebee08740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555a9062bcc0 0x555a90622ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:19.018 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:19.018+0000 7f4ebee08740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555a9062c5e0 0x555a90642da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:19.018 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:19.018+0000 7f4ebee08740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555a906347e0 0x555a9064f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:19.019 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:19.018+0000 7f4ebee08740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f4e9800e990 con 0x555a906347e0 2024-09-11T23:37:19.019 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:19.018+0000 7f4ebee08740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f4e980107a0 con 0x555a9062c5e0 2024-09-11T23:37:19.019 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:19.018+0000 7f4ebee08740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f4e9800f0f0 con 0x555a9062bcc0 2024-09-11T23:37:19.019 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:19.019+0000 7f4eb9531640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555a9062c5e0 0x555a90642da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:19.019 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:19.019+0000 7f4eb9531640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555a9062c5e0 0x555a90642da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:34130/0 (socket says 172.21.15.26:34130) 2024-09-11T23:37:19.019 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:19.019+0000 7f4eb9531640 1 -- 172.21.15.26:0/2992390063 learned_addr learned my addr 172.21.15.26:0/2992390063 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:37:19.020 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:19.021+0000 7f4eba533640 1 --2- 172.21.15.26:0/2992390063 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555a906347e0 0x555a9064f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:19.020 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:19.021+0000 7f4eba533640 1 --2- 172.21.15.26:0/2992390063 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555a9062bcc0 0x555a90622ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:19.025 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:19.024+0000 7f4eba533640 1 -- 172.21.15.26:0/2992390063 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555a9062c5e0 msgr2=0x555a90642da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:19.025 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:19.024+0000 7f4eba533640 1 --2- 172.21.15.26:0/2992390063 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555a9062c5e0 0x555a90642da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:19.025 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:19.024+0000 7f4eba533640 1 -- 172.21.15.26:0/2992390063 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555a906347e0 msgr2=0x555a9064f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:19.025 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:19.024+0000 7f4eba533640 1 --2- 172.21.15.26:0/2992390063 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555a906347e0 0x555a9064f1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:19.025 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:19.024+0000 7f4eba533640 1 -- 172.21.15.26:0/2992390063 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a90632d10 con 0x555a9062bcc0 2024-09-11T23:37:19.028 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:19.026+0000 7f4eba533640 1 --2- 172.21.15.26:0/2992390063 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555a9062bcc0 0x555a90622ed0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f4ea4018730 tx=0x7f4ea4017340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ecbca72ff819a8d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:19.030 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:19.030+0000 7f4e9f7fe640 1 -- 172.21.15.26:0/2992390063 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ea4006d60 con 0x555a9062bcc0 2024-09-11T23:37:19.030 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:19.030+0000 7f4e9f7fe640 1 -- 172.21.15.26:0/2992390063 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ea4025070 con 0x555a9062bcc0 2024-09-11T23:37:19.030 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:19.030+0000 7f4e9f7fe640 1 -- 172.21.15.26:0/2992390063 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ea4028c80 con 0x555a9062bcc0 2024-09-11T23:37:19.030 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:19.031+0000 7f4ebee08740 1 -- 172.21.15.26:0/2992390063 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555a9062bcc0 msgr2=0x555a90622ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:19.030 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:19.031+0000 7f4ebee08740 1 --2- 172.21.15.26:0/2992390063 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555a9062bcc0 0x555a90622ed0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f4ea4018730 tx=0x7f4ea4017340 comp rx=0 tx=0).stop 2024-09-11T23:37:19.035 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:19.034+0000 7f4ebee08740 1 -- 172.21.15.26:0/2992390063 shutdown_connections 2024-09-11T23:37:19.035 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:19.034+0000 7f4ebee08740 1 --2- 172.21.15.26:0/2992390063 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555a9062c5e0 0x555a90642da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:19.035 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:19.034+0000 7f4ebee08740 1 --2- 172.21.15.26:0/2992390063 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555a906347e0 0x555a9064f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:19.035 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:19.034+0000 7f4ebee08740 1 --2- 172.21.15.26:0/2992390063 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555a9062bcc0 0x555a90622ed0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:19.035 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:19.034+0000 7f4ebee08740 1 -- 172.21.15.26:0/2992390063 >> 172.21.15.26:0/2992390063 conn(0x555a90628230 msgr2=0x555a90635de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:19.039 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:19.039+0000 7f4ebee08740 1 -- 172.21.15.26:0/2992390063 shutdown_connections 2024-09-11T23:37:19.039 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:19.039+0000 7f4ebee08740 1 -- 172.21.15.26:0/2992390063 wait complete. 2024-09-11T23:37:19.039 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:19.040+0000 7f4ebee08740 1 -- start start 2024-09-11T23:37:19.040 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:19.040+0000 7f4ebee08740 1 Processor -- start 2024-09-11T23:37:19.040 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:19.041+0000 7f4ebee08740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555a9062bcc0 0x555a90630e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:19.040 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:19.041+0000 7f4ebee08740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555a9062c5e0 0x555a90631370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:19.040 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:19.041+0000 7f4ebee08740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555a906347e0 0x555a906318b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:19.040 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:19.041+0000 7f4ebee08740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f4e98010010 con 0x555a9062c5e0 2024-09-11T23:37:19.041 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:19.041+0000 7f4ebee08740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f4e9800f880 con 0x555a906347e0 2024-09-11T23:37:19.041 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:19.041+0000 7f4ebee08740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f4e9800c930 con 0x555a9062bcc0 2024-09-11T23:37:19.041 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:19.041+0000 7f4ebee08740 10 client.0 _ll_register_callbacks cb 0x555a90636dd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:37:19.044 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:19.041+0000 7f4eba533640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555a906347e0 0x555a906318b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:19.044 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:19.041+0000 7f4eba533640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555a9062bcc0 0x555a90630e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:19.045 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:19.041+0000 7f4eba533640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555a9062bcc0 0x555a90630e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:33004/0 (socket says 172.21.15.26:33004) 2024-09-11T23:37:19.045 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:19.041+0000 7f4eba533640 1 -- 172.21.15.26:0/1710394788 learned_addr learned my addr 172.21.15.26:0/1710394788 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:37:19.045 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:19.044+0000 7f4eb9531640 1 --2- 172.21.15.26:0/1710394788 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555a9062c5e0 0x555a90631370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:19.045 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:19.045+0000 7f4eb9531640 1 -- 172.21.15.26:0/1710394788 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555a9062bcc0 msgr2=0x555a90630e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:19.045 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:19.045+0000 7f4eb9531640 1 --2- 172.21.15.26:0/1710394788 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555a9062bcc0 0x555a90630e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:19.045 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:19.045+0000 7f4eb9531640 1 -- 172.21.15.26:0/1710394788 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555a906347e0 msgr2=0x555a906318b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:19.045 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:19.045+0000 7f4eb9531640 1 --2- 172.21.15.26:0/1710394788 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555a906347e0 0x555a906318b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:19.045 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:19.045+0000 7f4eb9531640 1 -- 172.21.15.26:0/1710394788 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a9062c410 con 0x555a9062c5e0 2024-09-11T23:37:19.046 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:19.046+0000 7f4eb9531640 1 --2- 172.21.15.26:0/1710394788 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555a9062c5e0 0x555a90631370 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f4eb002aa70 tx=0x7f4eb0005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f271bbc647fcb2f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:19.046 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:19.046+0000 7f4e9d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:37:19.046 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:19.047+0000 7f4e9d7fa640 1 -- 172.21.15.26:0/1710394788 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eb0050060 con 0x555a9062c5e0 2024-09-11T23:37:19.046 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:19.047+0000 7f4e9d7fa640 1 -- 172.21.15.26:0/1710394788 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 8586+0+0 (secure 0 0 0) 0x7f4eb0031440 con 0x555a9062c5e0 2024-09-11T23:37:19.049 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:19.048+0000 7f4ebee08740 1 -- 172.21.15.26:0/1710394788 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ea4025070 con 0x555a9062c5e0 2024-09-11T23:37:19.053 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:19.054+0000 7f4e9d7fa640 1 -- 172.21.15.26:0/1710394788 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f4eb0031640 con 0x555a9062c5e0 2024-09-11T23:37:19.055 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:19.054+0000 7f4ebee08740 10 client.17886 fetch_fsmap learned FSMap version 86 2024-09-11T23:37:19.055 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:19.055+0000 7f4ebee08740 1 -- 172.21.15.26:0/1710394788 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=86,monmap=4+}) -- 0x555a906b2370 con 0x555a9062c5e0 2024-09-11T23:37:19.056 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:19.057+0000 7f4e9d7fa640 1 -- 172.21.15.26:0/1710394788 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ea4025070 con 0x555a9062c5e0 2024-09-11T23:37:19.056 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:19.057+0000 7f4e9d7fa640 1 -- 172.21.15.26:0/1710394788 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 86) ==== 8426+0+0 (secure 0 0 0) 0x7f4eb004a030 con 0x555a9062c5e0 2024-09-11T23:37:19.057 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:19.058+0000 7f4ebee08740 10 client.17886 fetch_fsmap finished waiting for FSMap version 86 2024-09-11T23:37:19.057 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:19.058+0000 7f4ebee08740 10 client.17886 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:37:19.058 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:19.058+0000 7f4ebee08740 20 client.17886 populate_metadata read hostname 'smithi026' 2024-09-11T23:37:19.058 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:19.058+0000 7f4ebee08740 1 --2- 172.21.15.26:0/1710394788 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x555a906b2b10 0x555a906b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:19.058 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:19.058+0000 7f4ebee08740 4 client.17886 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T23:37:19.058 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:19.058+0000 7f4ebee08740 1 -- 172.21.15.26:0/1710394788 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555a90597d70 con 0x555a906b2b10 2024-09-11T23:37:19.060 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:19.060+0000 7f4eba533640 1 --2- 172.21.15.26:0/1710394788 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x555a906b2b10 0x555a906b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:19.061 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:19.061+0000 7f4eba533640 1 --2- 172.21.15.26:0/1710394788 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x555a906b2b10 0x555a906b4f30 crc :-1 s=READY pgs=287 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e9ab9b60dc214edd server_cookie=12c04cc652e019c4 in_seq=0 out_seq=0 2024-09-11T23:37:19.061 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:19.062+0000 7f4e9d7fa640 10 client.17886 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:37:19.171 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:19.170+0000 7f4e9d7fa640 1 -- 172.21.15.26:0/1710394788 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555a90597d70 con 0x555a906b2b10 2024-09-11T23:37:19.171 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:19.170+0000 7f4e9d7fa640 10 client.17886 handle_command_reply: tid=0 2024-09-11T23:37:19.171 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:19.171+0000 7f4ebee08740 1 client.17886 shutdown 2024-09-11T23:37:19.171 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:19.171+0000 7f4ebee08740 1 -- 172.21.15.26:0/1710394788 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555a9062c5e0 msgr2=0x555a90631370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:19.171 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:19.171+0000 7f4ebee08740 1 --2- 172.21.15.26:0/1710394788 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555a9062c5e0 0x555a90631370 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f4eb002aa70 tx=0x7f4eb0005fd0 comp rx=0 tx=0).stop 2024-09-11T23:37:19.171 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:19.171+0000 7f4ebee08740 1 -- 172.21.15.26:0/1710394788 shutdown_connections 2024-09-11T23:37:19.171 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:19.171+0000 7f4ebee08740 1 --2- 172.21.15.26:0/1710394788 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555a906347e0 0x555a906318b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:19.172 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:19.171+0000 7f4ebee08740 1 --2- 172.21.15.26:0/1710394788 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x555a906b2b10 0x555a906b4f30 crc :-1 s=READY pgs=287 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:19.172 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:19.171+0000 7f4ebee08740 1 --2- 172.21.15.26:0/1710394788 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555a9062c5e0 0x555a90631370 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:19.172 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:19.171+0000 7f4e9d7fa640 0 client.17886 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:37:19.172 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:19.171+0000 7f4ebee08740 1 --2- 172.21.15.26:0/1710394788 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555a9062bcc0 0x555a90630e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:19.172 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:19.171+0000 7f4ebee08740 1 -- 172.21.15.26:0/1710394788 >> 172.21.15.26:0/1710394788 conn(0x555a90628230 msgr2=0x555a90626a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:19.172 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:19.171+0000 7f4ebee08740 1 -- 172.21.15.26:0/1710394788 shutdown_connections 2024-09-11T23:37:19.172 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:19.171+0000 7f4ebee08740 1 -- 172.21.15.26:0/1710394788 wait complete. 2024-09-11T23:37:19.172 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:19.172+0000 7f4ebee08740 20 client.17886 trim_cache size 0 max 16384 2024-09-11T23:37:19.172 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-09-11T23:37:19.173 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "message": "", 2024-09-11T23:37:19.173 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-09-11T23:37:19.173 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-09-11T23:37:19.173 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:19.172+0000 7f4ebad34640 1 -- 172.21.15.26:0/2245700335 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f4ea00a2dd0 msgr2=0x7f4ea00a5290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:19.174 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:19.172+0000 7f4ebad34640 1 --2- 172.21.15.26:0/2245700335 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f4ea00a2dd0 0x7f4ea00a5290 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x555a90629090 tx=0x7f4eb411f040 comp rx=0 tx=0).stop 2024-09-11T23:37:19.174 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:19.173+0000 7f4ebad34640 1 -- 172.21.15.26:0/2245700335 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4eac0959f0 msgr2=0x7f4eac141b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:19.174 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:19.173+0000 7f4ebad34640 1 --2- 172.21.15.26:0/2245700335 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4eac0959f0 0x7f4eac141b40 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f4eb4065a40 tx=0x7f4eb406e430 comp rx=0 tx=0).stop 2024-09-11T23:37:19.174 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:19.173+0000 7f4ebad34640 1 -- 172.21.15.26:0/2245700335 shutdown_connections 2024-09-11T23:37:19.174 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:19.173+0000 7f4ebad34640 1 --2- 172.21.15.26:0/2245700335 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f4ea00a2dd0 0x7f4ea00a5290 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:19.174 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:19.173+0000 7f4ebad34640 1 --2- 172.21.15.26:0/2245700335 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4eac09b740 0x7f4eac13b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:19.174 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:19.173+0000 7f4ebad34640 1 --2- 172.21.15.26:0/2245700335 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4eac096390 0x7f4eac13abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:19.174 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:19.173+0000 7f4ebad34640 1 --2- 172.21.15.26:0/2245700335 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4eac0959f0 0x7f4eac141b40 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:19.174 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:19.173+0000 7f4ebad34640 1 -- 172.21.15.26:0/2245700335 >> 172.21.15.26:0/2245700335 conn(0x7f4eac00c6c0 msgr2=0x7f4eac0946c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:19.174 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:19.173+0000 7f4ebad34640 1 -- 172.21.15.26:0/2245700335 shutdown_connections 2024-09-11T23:37:19.174 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:19.173+0000 7f4ebad34640 1 -- 172.21.15.26:0/2245700335 wait complete. 2024-09-11T23:37:20.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:37:20 smithi136 ceph-mon[35121]: pgmap v596: 97 pgs: 23 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 33 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.4 MiB/s wr, 248 op/s 2024-09-11T23:37:20.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:37:20 smithi026 ceph-mon[31172]: pgmap v596: 97 pgs: 23 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 33 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.4 MiB/s wr, 248 op/s 2024-09-11T23:37:20.552 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:37:20 smithi076 ceph-mon[35296]: pgmap v596: 97 pgs: 23 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 33 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.4 MiB/s wr, 248 op/s 2024-09-11T23:37:22.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:37:22 smithi136 ceph-mon[35121]: pgmap v597: 97 pgs: 23 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 33 active+clean; 2.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.7 MiB/s wr, 160 op/s 2024-09-11T23:37:22.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:37:22 smithi026 ceph-mon[31172]: pgmap v597: 97 pgs: 23 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 33 active+clean; 2.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.7 MiB/s wr, 160 op/s 2024-09-11T23:37:22.552 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:37:22 smithi076 ceph-mon[35296]: pgmap v597: 97 pgs: 23 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 33 active+clean; 2.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.7 MiB/s wr, 160 op/s 2024-09-11T23:37:23.000 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:37:22 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: 2024-09-11T23:37:22.585+0000 7f7ccf119640 -1 calc_pg_upmaps abort due to max <= 0 2024-09-11T23:37:24.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:37:24 smithi136 ceph-mon[35121]: pgmap v598: 97 pgs: 23 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 33 active+clean; 2.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.7 MiB/s wr, 147 op/s 2024-09-11T23:37:24.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:37:24 smithi026 ceph-mon[31172]: pgmap v598: 97 pgs: 23 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 33 active+clean; 2.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.7 MiB/s wr, 147 op/s 2024-09-11T23:37:24.552 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:37:24 smithi076 ceph-mon[35296]: pgmap v598: 97 pgs: 23 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 33 active+clean; 2.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.7 MiB/s wr, 147 op/s 2024-09-11T23:37:26.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:37:26 smithi136 ceph-mon[35121]: pgmap v599: 97 pgs: 23 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 33 active+clean; 2.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.1 MiB/s wr, 152 op/s 2024-09-11T23:37:26.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:37:26 smithi026 ceph-mon[31172]: pgmap v599: 97 pgs: 23 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 33 active+clean; 2.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.1 MiB/s wr, 152 op/s 2024-09-11T23:37:26.552 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:37:26 smithi076 ceph-mon[35296]: pgmap v599: 97 pgs: 23 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 33 active+clean; 2.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.1 MiB/s wr, 152 op/s 2024-09-11T23:37:28.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:37:28 smithi136 ceph-mon[35121]: pgmap v600: 97 pgs: 23 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 33 active+clean; 2.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 151 op/s 2024-09-11T23:37:28.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:37:28 smithi026 ceph-mon[31172]: pgmap v600: 97 pgs: 23 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 33 active+clean; 2.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 151 op/s 2024-09-11T23:37:28.552 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:37:28 smithi076 ceph-mon[35296]: pgmap v600: 97 pgs: 23 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 33 active+clean; 2.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 151 op/s 2024-09-11T23:37:30.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:37:30 smithi136 ceph-mon[35121]: pgmap v601: 97 pgs: 23 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 33 active+clean; 2.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 135 op/s 2024-09-11T23:37:30.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:37:30 smithi026 ceph-mon[31172]: pgmap v601: 97 pgs: 23 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 33 active+clean; 2.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 135 op/s 2024-09-11T23:37:30.552 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:37:30 smithi076 ceph-mon[35296]: pgmap v601: 97 pgs: 23 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 33 active+clean; 2.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 135 op/s 2024-09-11T23:37:32.337 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.336+0000 7f75677fe640 1 -- 172.21.15.26:0/4208176333 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f756002f320 con 0x7f75681067f0 2024-09-11T23:37:32.337 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.337+0000 7f756dd9f640 1 -- 172.21.15.26:0/4208176333 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f75681067f0 msgr2=0x7f7568106bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:32.337 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.337+0000 7f756dd9f640 1 --2- 172.21.15.26:0/4208176333 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f75681067f0 0x7f7568106bd0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f7560009cd0 tx=0x7f756002a910 comp rx=0 tx=0).stop 2024-09-11T23:37:32.337 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.337+0000 7f756dd9f640 1 -- 172.21.15.26:0/4208176333 shutdown_connections 2024-09-11T23:37:32.337 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.337+0000 7f756dd9f640 1 --2- 172.21.15.26:0/4208176333 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7568101180 0x7f756810d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:32.337 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.337+0000 7f756dd9f640 1 --2- 172.21.15.26:0/4208176333 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f75681007e0 0x7f7568100c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:32.338 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.337+0000 7f756dd9f640 1 --2- 172.21.15.26:0/4208176333 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f75681067f0 0x7f7568106bd0 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:32.338 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.337+0000 7f756dd9f640 1 -- 172.21.15.26:0/4208176333 >> 172.21.15.26:0/4208176333 conn(0x7f75680fc4e0 msgr2=0x7f75680fe900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:32.340 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.341+0000 7f756dd9f640 1 -- 172.21.15.26:0/4208176333 shutdown_connections 2024-09-11T23:37:32.341 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.341+0000 7f756dd9f640 1 -- 172.21.15.26:0/4208176333 wait complete. 2024-09-11T23:37:32.342 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.341+0000 7f756dd9f640 1 Processor -- start 2024-09-11T23:37:32.343 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.342+0000 7f756dd9f640 1 -- start start 2024-09-11T23:37:32.343 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.342+0000 7f756dd9f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f75681007e0 0x7f75681160a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:32.343 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.342+0000 7f756dd9f640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7568101180 0x7f75681165e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:32.343 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.342+0000 7f756dd9f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f75681067f0 0x7f75681111b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:32.343 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.342+0000 7f756dd9f640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f756810fb90 con 0x7f75681067f0 2024-09-11T23:37:32.343 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.342+0000 7f756dd9f640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f756810fa10 con 0x7f75681007e0 2024-09-11T23:37:32.343 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.342+0000 7f756dd9f640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f756810fd10 con 0x7f7568101180 2024-09-11T23:37:32.343 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.342+0000 7f756cd9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f75681007e0 0x7f75681160a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:32.343 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.342+0000 7f756cd9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f75681007e0 0x7f75681160a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:51780/0 (socket says 172.21.15.26:51780) 2024-09-11T23:37:32.343 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.342+0000 7f756cd9d640 1 -- 172.21.15.26:0/1943764467 learned_addr learned my addr 172.21.15.26:0/1943764467 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:37:32.343 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.342+0000 7f7567fff640 1 --2- 172.21.15.26:0/1943764467 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7568101180 0x7f75681165e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:32.343 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.342+0000 7f756d59e640 1 --2- 172.21.15.26:0/1943764467 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f75681067f0 0x7f75681111b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:32.344 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.343+0000 7f756cd9d640 1 -- 172.21.15.26:0/1943764467 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7568101180 msgr2=0x7f75681165e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:32.344 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.343+0000 7f756cd9d640 1 --2- 172.21.15.26:0/1943764467 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7568101180 0x7f75681165e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:32.344 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.343+0000 7f756cd9d640 1 -- 172.21.15.26:0/1943764467 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f75681067f0 msgr2=0x7f75681111b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:32.344 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.343+0000 7f756cd9d640 1 --2- 172.21.15.26:0/1943764467 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f75681067f0 0x7f75681111b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:32.344 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.343+0000 7f756cd9d640 1 -- 172.21.15.26:0/1943764467 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7568111a70 con 0x7f75681007e0 2024-09-11T23:37:32.344 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.343+0000 7f756cd9d640 1 --2- 172.21.15.26:0/1943764467 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f75681007e0 0x7f75681160a0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f7560009ca0 tx=0x7f7560031690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:32.345 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.343+0000 7f7565ffb640 1 -- 172.21.15.26:0/1943764467 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7560031980 con 0x7f75681007e0 2024-09-11T23:37:32.345 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.344+0000 7f756dd9f640 1 -- 172.21.15.26:0/1943764467 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7568111d00 con 0x7f75681007e0 2024-09-11T23:37:32.345 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.344+0000 7f7565ffb640 1 -- 172.21.15.26:0/1943764467 <== mon.2 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7560004520 con 0x7f75681007e0 2024-09-11T23:37:32.345 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.345+0000 7f7565ffb640 1 -- 172.21.15.26:0/1943764467 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7560031980 con 0x7f75681007e0 2024-09-11T23:37:32.345 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.345+0000 7f756dd9f640 1 -- 172.21.15.26:0/1943764467 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f756811bf90 con 0x7f75681007e0 2024-09-11T23:37:32.350 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.349+0000 7f7571e74740 1 Processor -- start 2024-09-11T23:37:32.350 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.349+0000 7f7565ffb640 1 -- 172.21.15.26:0/1943764467 <== mon.2 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f75600046d0 con 0x7f75681007e0 2024-09-11T23:37:32.350 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.349+0000 7f7571e74740 1 -- start start 2024-09-11T23:37:32.350 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.350+0000 7f7571e74740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a26dc73cc0 0x55a26dc740a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:32.350 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.350+0000 7f7571e74740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a26dc745e0 0x55a26dc7e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:32.350 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.350+0000 7f7571e74740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a26dc7ee80 0x55a26dc81270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:32.350 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.350+0000 7f7571e74740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55a26dbd82f0 con 0x55a26dc7ee80 2024-09-11T23:37:32.350 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.350+0000 7f7571e74740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55a26db80270 con 0x55a26dc745e0 2024-09-11T23:37:32.350 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.350+0000 7f7571e74740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55a26dbd42b0 con 0x55a26dc73cc0 2024-09-11T23:37:32.350 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.350+0000 7f756cd9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a26dc745e0 0x55a26dc7e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:32.350 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.350+0000 7f756cd9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a26dc745e0 0x55a26dc7e940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:51796/0 (socket says 172.21.15.26:51796) 2024-09-11T23:37:32.351 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.350+0000 7f756cd9d640 1 -- 172.21.15.26:0/2186623584 learned_addr learned my addr 172.21.15.26:0/2186623584 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:37:32.351 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.350+0000 7f756d59e640 1 --2- 172.21.15.26:0/2186623584 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a26dc73cc0 0x55a26dc740a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:32.351 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.350+0000 7f7567fff640 1 --2- 172.21.15.26:0/2186623584 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a26dc7ee80 0x55a26dc81270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:32.351 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.350+0000 7f756d59e640 1 -- 172.21.15.26:0/2186623584 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a26dc745e0 msgr2=0x55a26dc7e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:32.351 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.350+0000 7f756d59e640 1 --2- 172.21.15.26:0/2186623584 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a26dc745e0 0x55a26dc7e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:32.351 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.350+0000 7f756d59e640 1 -- 172.21.15.26:0/2186623584 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a26dc7ee80 msgr2=0x55a26dc81270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:32.351 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.350+0000 7f756d59e640 1 --2- 172.21.15.26:0/2186623584 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a26dc7ee80 0x55a26dc81270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:32.351 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.350+0000 7f756d59e640 1 -- 172.21.15.26:0/2186623584 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a26db7f340 con 0x55a26dc73cc0 2024-09-11T23:37:32.352 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.351+0000 7f756d59e640 1 --2- 172.21.15.26:0/2186623584 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a26dc73cc0 0x55a26dc740a0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f7558005fd0 tx=0x7f755800af40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b7fb8537e3e2f419 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:32.352 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.351+0000 7f7565ffb640 1 --2- 172.21.15.26:0/1943764467 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f75500a31b0 0x7f75500a5670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:32.352 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.351+0000 7f756cd9d640 1 --2- 172.21.15.26:0/2186623584 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a26dc745e0 0x55a26dc7e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:37:32.352 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.351+0000 7f754effd640 1 -- 172.21.15.26:0/2186623584 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f755801d070 con 0x55a26dc73cc0 2024-09-11T23:37:32.352 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.352+0000 7f754effd640 1 -- 172.21.15.26:0/2186623584 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7558018920 con 0x55a26dc73cc0 2024-09-11T23:37:32.352 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.352+0000 7f7565ffb640 1 -- 172.21.15.26:0/1943764467 <== mon.2 v2:172.21.15.76:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 8586+0+0 (secure 0 0 0) 0x7f7560031780 con 0x7f75681007e0 2024-09-11T23:37:32.353 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.353+0000 7f7571e74740 1 -- 172.21.15.26:0/2186623584 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a26dc73cc0 msgr2=0x55a26dc740a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:32.353 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.353+0000 7f7571e74740 1 --2- 172.21.15.26:0/2186623584 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a26dc73cc0 0x55a26dc740a0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f7558005fd0 tx=0x7f755800af40 comp rx=0 tx=0).stop 2024-09-11T23:37:32.353 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.353+0000 7f756cd9d640 1 --2- 172.21.15.26:0/1943764467 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f75500a31b0 0x7f75500a5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:32.353 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.353+0000 7f7571e74740 1 -- 172.21.15.26:0/2186623584 shutdown_connections 2024-09-11T23:37:32.353 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.353+0000 7f7571e74740 1 --2- 172.21.15.26:0/2186623584 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a26dc745e0 0x55a26dc7e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:32.353 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.353+0000 7f7571e74740 1 --2- 172.21.15.26:0/2186623584 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a26dc7ee80 0x55a26dc81270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:32.353 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.353+0000 7f7571e74740 1 --2- 172.21.15.26:0/2186623584 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a26dc73cc0 0x55a26dc740a0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:32.353 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.353+0000 7f7571e74740 1 -- 172.21.15.26:0/2186623584 >> 172.21.15.26:0/2186623584 conn(0x55a26dc70230 msgr2=0x55a26dc70610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:32.354 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.353+0000 7f7571e74740 1 -- 172.21.15.26:0/2186623584 shutdown_connections 2024-09-11T23:37:32.354 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.353+0000 7f7571e74740 1 -- 172.21.15.26:0/2186623584 wait complete. 2024-09-11T23:37:32.354 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.354+0000 7f7571e74740 1 -- start start 2024-09-11T23:37:32.354 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.354+0000 7f7571e74740 1 Processor -- start 2024-09-11T23:37:32.354 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.355+0000 7f756cd9d640 1 --2- 172.21.15.26:0/1943764467 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f75500a31b0 0x7f75500a5670 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f75600f0150 tx=0x7f756002a910 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:32.355 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.355+0000 7f7571e74740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a26dc73cc0 0x55a26dc7bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:32.355 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.355+0000 7f7571e74740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a26dc745e0 0x55a26dc7c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:32.355 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.355+0000 7f7571e74740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a26dc7ee80 0x55a26dc7c850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:32.355 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.355+0000 7f7571e74740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55a26dc81aa0 con 0x55a26dc73cc0 2024-09-11T23:37:32.355 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.355+0000 7f7571e74740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55a26db61290 con 0x55a26dc7ee80 2024-09-11T23:37:32.355 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.355+0000 7f7571e74740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55a26dc81c20 con 0x55a26dc745e0 2024-09-11T23:37:32.356 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.356+0000 7f7571e74740 10 client.0 _ll_register_callbacks cb 0x55a26db92140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:37:32.356 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.356+0000 7f756d59e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a26dc73cc0 0x55a26dc7bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:32.356 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.356+0000 7f756d59e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a26dc73cc0 0x55a26dc7bdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59702/0 (socket says 172.21.15.26:59702) 2024-09-11T23:37:32.356 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.356+0000 7f756d59e640 1 -- 172.21.15.26:0/372997124 learned_addr learned my addr 172.21.15.26:0/372997124 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:37:32.356 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.356+0000 7f7567fff640 1 --2- 172.21.15.26:0/372997124 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a26dc745e0 0x55a26dc7c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:32.356 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.356+0000 7f756d59e640 1 --2- 172.21.15.26:0/372997124 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a26dc7ee80 0x55a26dc7c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:32.356 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.356+0000 7f756d59e640 1 -- 172.21.15.26:0/372997124 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a26dc745e0 msgr2=0x55a26dc7c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:32.357 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.356+0000 7f756d59e640 1 --2- 172.21.15.26:0/372997124 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a26dc745e0 0x55a26dc7c310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:32.357 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.356+0000 7f756d59e640 1 -- 172.21.15.26:0/372997124 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a26dc7ee80 msgr2=0x55a26dc7c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:32.357 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.356+0000 7f756d59e640 1 --2- 172.21.15.26:0/372997124 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a26dc7ee80 0x55a26dc7c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:32.357 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.356+0000 7f756d59e640 1 -- 172.21.15.26:0/372997124 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a26d8eb210 con 0x55a26dc73cc0 2024-09-11T23:37:32.357 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.356+0000 7f756d59e640 1 --2- 172.21.15.26:0/372997124 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a26dc73cc0 0x55a26dc7bdd0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f755800b010 tx=0x7f7558010670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca297f2b2c152349 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:32.357 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.357+0000 7f7547fff640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:37:32.357 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.357+0000 7f7547fff640 1 -- 172.21.15.26:0/372997124 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f755802d060 con 0x55a26dc73cc0 2024-09-11T23:37:32.357 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.357+0000 7f7571e74740 1 -- 172.21.15.26:0/372997124 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7558018920 con 0x55a26dc73cc0 2024-09-11T23:37:32.357 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.357+0000 7f7547fff640 1 -- 172.21.15.26:0/372997124 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 8586+0+0 (secure 0 0 0) 0x7f7558005250 con 0x55a26dc73cc0 2024-09-11T23:37:32.358 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.357+0000 7f7547fff640 1 -- 172.21.15.26:0/372997124 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f755801d030 con 0x55a26dc73cc0 2024-09-11T23:37:32.358 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.358+0000 7f7571e74740 10 client.17898 fetch_fsmap learned FSMap version 86 2024-09-11T23:37:32.358 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.358+0000 7f7571e74740 1 -- 172.21.15.26:0/372997124 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=86,monmap=4+}) -- 0x55a26d9b7620 con 0x55a26dc73cc0 2024-09-11T23:37:32.359 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.358+0000 7f7547fff640 1 -- 172.21.15.26:0/372997124 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7558018920 con 0x55a26dc73cc0 2024-09-11T23:37:32.359 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.359+0000 7f7547fff640 1 -- 172.21.15.26:0/372997124 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 86) ==== 8426+0+0 (secure 0 0 0) 0x7f75580046c0 con 0x55a26dc73cc0 2024-09-11T23:37:32.359 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.359+0000 7f7571e74740 10 client.17898 fetch_fsmap finished waiting for FSMap version 86 2024-09-11T23:37:32.359 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.359+0000 7f7571e74740 10 client.17898 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:37:32.359 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.359+0000 7f7571e74740 20 client.17898 populate_metadata read hostname 'smithi026' 2024-09-11T23:37:32.359 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.359+0000 7f7571e74740 1 --2- 172.21.15.26:0/372997124 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55a26dc8eec0 0x55a26dc912b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:32.359 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.359+0000 7f7571e74740 4 client.17898 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:37:32.359 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.359+0000 7f7571e74740 1 -- 172.21.15.26:0/372997124 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a26dbe5a20 con 0x55a26dc8eec0 2024-09-11T23:37:32.359 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.359+0000 7f7567fff640 1 --2- 172.21.15.26:0/372997124 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55a26dc8eec0 0x55a26dc912b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:32.360 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.361+0000 7f7567fff640 1 --2- 172.21.15.26:0/372997124 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55a26dc8eec0 0x55a26dc912b0 crc :-1 s=READY pgs=288 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=31b8def8798a53a2 server_cookie=6902ea8158d9803f in_seq=0 out_seq=0 2024-09-11T23:37:32.360 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.361+0000 7f7547fff640 10 client.17898 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:37:32.362 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.362+0000 7f7547fff640 1 -- 172.21.15.26:0/372997124 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a26dbe5a20 con 0x55a26dc8eec0 2024-09-11T23:37:32.362 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.362+0000 7f7547fff640 10 client.17898 handle_command_reply: tid=0 2024-09-11T23:37:32.362 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.362+0000 7f7571e74740 1 client.17898 shutdown 2024-09-11T23:37:32.362 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.362+0000 7f7571e74740 1 -- 172.21.15.26:0/372997124 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a26dc73cc0 msgr2=0x55a26dc7bdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:32.362 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.362+0000 7f7571e74740 1 --2- 172.21.15.26:0/372997124 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a26dc73cc0 0x55a26dc7bdd0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f755800b010 tx=0x7f7558010670 comp rx=0 tx=0).stop 2024-09-11T23:37:32.362 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.362+0000 7f7571e74740 1 -- 172.21.15.26:0/372997124 shutdown_connections 2024-09-11T23:37:32.362 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.362+0000 7f7571e74740 1 --2- 172.21.15.26:0/372997124 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a26dc7ee80 0x55a26dc7c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:32.362 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.362+0000 7f7571e74740 1 --2- 172.21.15.26:0/372997124 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55a26dc8eec0 0x55a26dc912b0 crc :-1 s=READY pgs=288 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:32.362 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.362+0000 7f7571e74740 1 --2- 172.21.15.26:0/372997124 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a26dc745e0 0x55a26dc7c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:32.362 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.362+0000 7f7571e74740 1 --2- 172.21.15.26:0/372997124 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a26dc73cc0 0x55a26dc7bdd0 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:32.362 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.362+0000 7f7571e74740 1 -- 172.21.15.26:0/372997124 >> 172.21.15.26:0/372997124 conn(0x55a26dc70230 msgr2=0x55a26dc75f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:32.363 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.362+0000 7f7547fff640 0 client.17898 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:37:32.363 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.362+0000 7f7571e74740 1 -- 172.21.15.26:0/372997124 shutdown_connections 2024-09-11T23:37:32.363 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.362+0000 7f7571e74740 1 -- 172.21.15.26:0/372997124 wait complete. 2024-09-11T23:37:32.363 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.362+0000 7f7571e74740 20 client.17898 trim_cache size 0 max 16384 2024-09-11T23:37:32.386 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.386+0000 7f7571e74740 1 Processor -- start 2024-09-11T23:37:32.386 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.386+0000 7f7571e74740 1 -- start start 2024-09-11T23:37:32.388 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.387+0000 7f7571e74740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a26dc73cc0 0x55a26dc7c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:32.388 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.387+0000 7f7571e74740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a26dc745e0 0x55a26dc96960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:32.388 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.387+0000 7f7571e74740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a26dc7ee80 0x55a26dc96ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:32.388 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.387+0000 7f7571e74740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f7568072840 con 0x55a26dc7ee80 2024-09-11T23:37:32.388 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.387+0000 7f7571e74740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f75681c6b90 con 0x55a26dc73cc0 2024-09-11T23:37:32.388 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.387+0000 7f7571e74740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f7568072fa0 con 0x55a26dc745e0 2024-09-11T23:37:32.388 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.387+0000 7f7567fff640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a26dc745e0 0x55a26dc96960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:32.388 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.387+0000 7f7567fff640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a26dc745e0 0x55a26dc96960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:49500/0 (socket says 172.21.15.26:49500) 2024-09-11T23:37:32.389 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.387+0000 7f7567fff640 1 -- 172.21.15.26:0/552746965 learned_addr learned my addr 172.21.15.26:0/552746965 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:37:32.389 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.389+0000 7f756d59e640 1 --2- 172.21.15.26:0/552746965 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a26dc73cc0 0x55a26dc7c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:32.389 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.389+0000 7f756d59e640 1 --2- 172.21.15.26:0/552746965 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a26dc7ee80 0x55a26dc96ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:32.389 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.389+0000 7f756d59e640 1 -- 172.21.15.26:0/552746965 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a26dc745e0 msgr2=0x55a26dc96960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:32.389 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.389+0000 7f756d59e640 1 --2- 172.21.15.26:0/552746965 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a26dc745e0 0x55a26dc96960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:32.389 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.389+0000 7f756d59e640 1 -- 172.21.15.26:0/552746965 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a26dc7ee80 msgr2=0x55a26dc96ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:32.389 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.389+0000 7f756d59e640 1 --2- 172.21.15.26:0/552746965 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a26dc7ee80 0x55a26dc96ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:32.390 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.389+0000 7f756d59e640 1 -- 172.21.15.26:0/552746965 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a26dc81c80 con 0x55a26dc73cc0 2024-09-11T23:37:32.390 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.390+0000 7f756d59e640 1 --2- 172.21.15.26:0/552746965 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a26dc73cc0 0x55a26dc7c680 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f755800a760 tx=0x7f755800a6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6e1e24cc418cc02a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:32.390 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.391+0000 7f754e7fc640 1 -- 172.21.15.26:0/552746965 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7558037080 con 0x55a26dc73cc0 2024-09-11T23:37:32.390 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.391+0000 7f754e7fc640 1 -- 172.21.15.26:0/552746965 <== mon.2 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7558005790 con 0x55a26dc73cc0 2024-09-11T23:37:32.391 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.391+0000 7f754e7fc640 1 -- 172.21.15.26:0/552746965 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7558018870 con 0x55a26dc73cc0 2024-09-11T23:37:32.391 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.392+0000 7f7571e74740 1 -- 172.21.15.26:0/552746965 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a26dc73cc0 msgr2=0x55a26dc7c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:32.391 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.392+0000 7f7571e74740 1 --2- 172.21.15.26:0/552746965 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a26dc73cc0 0x55a26dc7c680 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f755800a760 tx=0x7f755800a6d0 comp rx=0 tx=0).stop 2024-09-11T23:37:32.393 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.394+0000 7f7571e74740 1 -- 172.21.15.26:0/552746965 shutdown_connections 2024-09-11T23:37:32.394 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.394+0000 7f7571e74740 1 --2- 172.21.15.26:0/552746965 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a26dc7ee80 0x55a26dc96ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:32.394 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.394+0000 7f7571e74740 1 --2- 172.21.15.26:0/552746965 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a26dc745e0 0x55a26dc96960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:32.394 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.394+0000 7f7571e74740 1 --2- 172.21.15.26:0/552746965 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a26dc73cc0 0x55a26dc7c680 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:32.394 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.394+0000 7f7571e74740 1 -- 172.21.15.26:0/552746965 >> 172.21.15.26:0/552746965 conn(0x55a26dc70230 msgr2=0x55a26dc7de50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:32.394 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.395+0000 7f7571e74740 1 -- 172.21.15.26:0/552746965 shutdown_connections 2024-09-11T23:37:32.394 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.395+0000 7f7571e74740 1 -- 172.21.15.26:0/552746965 wait complete. 2024-09-11T23:37:32.395 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.396+0000 7f7571e74740 1 -- start start 2024-09-11T23:37:32.395 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.396+0000 7f7571e74740 1 Processor -- start 2024-09-11T23:37:32.396 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.397+0000 7f7571e74740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a26dc73cc0 0x55a26dc875b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:32.397 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.397+0000 7f7571e74740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a26dc745e0 0x55a26dc828d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:32.397 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.397+0000 7f756d59e640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a26dc73cc0 0x55a26dc875b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:32.397 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.397+0000 7f756d59e640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a26dc73cc0 0x55a26dc875b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:49504/0 (socket says 172.21.15.26:49504) 2024-09-11T23:37:32.397 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.397+0000 7f756d59e640 1 -- 172.21.15.26:0/1724698123 learned_addr learned my addr 172.21.15.26:0/1724698123 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:37:32.397 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.397+0000 7f7571e74740 1 --2- 172.21.15.26:0/1724698123 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a26dc7ee80 0x55a26dc82e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:32.398 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.398+0000 7f7567fff640 1 --2- 172.21.15.26:0/1724698123 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a26dc745e0 0x55a26dc828d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:32.398 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.398+0000 7f7571e74740 1 -- 172.21.15.26:0/1724698123 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f7568073ec0 con 0x55a26dc7ee80 2024-09-11T23:37:32.398 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.398+0000 7f7571e74740 1 -- 172.21.15.26:0/1724698123 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f7568073730 con 0x55a26dc745e0 2024-09-11T23:37:32.398 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.398+0000 7f7571e74740 1 -- 172.21.15.26:0/1724698123 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f7568070840 con 0x55a26dc73cc0 2024-09-11T23:37:32.399 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.398+0000 7f756d59e640 1 --2- 172.21.15.26:0/1724698123 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a26dc7ee80 0x55a26dc82e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:32.399 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.398+0000 7f756d59e640 1 -- 172.21.15.26:0/1724698123 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a26dc745e0 msgr2=0x55a26dc828d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:32.399 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.398+0000 7f756d59e640 1 --2- 172.21.15.26:0/1724698123 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a26dc745e0 0x55a26dc828d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:32.399 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.398+0000 7f756d59e640 1 -- 172.21.15.26:0/1724698123 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a26dc7ee80 msgr2=0x55a26dc82e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:32.399 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.398+0000 7f756d59e640 1 --2- 172.21.15.26:0/1724698123 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a26dc7ee80 0x55a26dc82e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:32.399 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.398+0000 7f756d59e640 1 -- 172.21.15.26:0/1724698123 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a26dc83640 con 0x55a26dc73cc0 2024-09-11T23:37:32.399 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.399+0000 7f756d59e640 1 --2- 172.21.15.26:0/1724698123 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a26dc73cc0 0x55a26dc875b0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f7558010550 tx=0x7f755800f910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3bdf55b9176d74d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:32.399 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.399+0000 7f7547fff640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-09-11T23:37:32.399 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.399+0000 7f7547fff640 1 -- 172.21.15.26:0/1724698123 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7558041040 con 0x55a26dc73cc0 2024-09-11T23:37:32.399 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.399+0000 7f7571e74740 10 client.0 _ll_register_callbacks cb 0x7f755801b070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:37:32.400 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.399+0000 7f7547fff640 1 -- 172.21.15.26:0/1724698123 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 8586+0+0 (secure 0 0 0) 0x7f7558018a70 con 0x55a26dc73cc0 2024-09-11T23:37:32.400 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.399+0000 7f7571e74740 1 -- 172.21.15.26:0/1724698123 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7558005790 con 0x55a26dc73cc0 2024-09-11T23:37:32.401 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.400+0000 7f7547fff640 1 -- 172.21.15.26:0/1724698123 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f7558018c70 con 0x55a26dc73cc0 2024-09-11T23:37:32.401 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.400+0000 7f7571e74740 10 client.27370 fetch_fsmap learned FSMap version 86 2024-09-11T23:37:32.401 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.400+0000 7f7571e74740 1 -- 172.21.15.26:0/1724698123 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({fsmap=86}) -- 0x55a26dcf9400 con 0x55a26dc73cc0 2024-09-11T23:37:32.401 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.401+0000 7f7547fff640 1 -- 172.21.15.26:0/1724698123 <== mon.1 v2:172.21.15.136:3300/0 4 ==== fsmap(e 86) ==== 8426+0+0 (secure 0 0 0) 0x7f7558010640 con 0x55a26dc73cc0 2024-09-11T23:37:32.401 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.401+0000 7f7571e74740 10 client.27370 fetch_fsmap finished waiting for FSMap version 86 2024-09-11T23:37:32.401 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.401+0000 7f7571e74740 10 client.27370 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:37:32.401 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.401+0000 7f7571e74740 20 client.27370 populate_metadata read hostname 'smithi026' 2024-09-11T23:37:32.402 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.401+0000 7f7571e74740 1 --2- 172.21.15.26:0/1724698123 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55a26dcf9be0 0x55a26dcfc0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:32.402 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.401+0000 7f7571e74740 4 client.27370 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T23:37:32.402 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.401+0000 7f7571e74740 1 -- 172.21.15.26:0/1724698123 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f7558005790 con 0x55a26dcf9be0 2024-09-11T23:37:32.402 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.402+0000 7f7567fff640 1 --2- 172.21.15.26:0/1724698123 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55a26dcf9be0 0x55a26dcfc0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:32.402 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.402+0000 7f7567fff640 1 --2- 172.21.15.26:0/1724698123 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55a26dcf9be0 0x55a26dcfc0a0 crc :-1 s=READY pgs=289 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=58e68cde2cbbf59b server_cookie=7c6283505dadee9a in_seq=0 out_seq=0 2024-09-11T23:37:32.402 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.402+0000 7f7547fff640 10 client.27370 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:37:32.462 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.462+0000 7f7547fff640 1 -- 172.21.15.26:0/1724698123 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f7558005790 con 0x55a26dcf9be0 2024-09-11T23:37:32.462 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.462+0000 7f7547fff640 10 client.27370 handle_command_reply: tid=0 2024-09-11T23:37:32.462 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.463+0000 7f7571e74740 1 client.27370 shutdown 2024-09-11T23:37:32.464 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.463+0000 7f7571e74740 1 -- 172.21.15.26:0/1724698123 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a26dc73cc0 msgr2=0x55a26dc875b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:32.464 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.463+0000 7f7571e74740 1 --2- 172.21.15.26:0/1724698123 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a26dc73cc0 0x55a26dc875b0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f7558010550 tx=0x7f755800f910 comp rx=0 tx=0).stop 2024-09-11T23:37:32.464 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.463+0000 7f7571e74740 1 -- 172.21.15.26:0/1724698123 shutdown_connections 2024-09-11T23:37:32.464 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.463+0000 7f7571e74740 1 --2- 172.21.15.26:0/1724698123 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a26dc745e0 0x55a26dc828d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:32.464 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.463+0000 7f7571e74740 1 --2- 172.21.15.26:0/1724698123 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55a26dcf9be0 0x55a26dcfc0a0 crc :-1 s=READY pgs=289 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:32.464 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.464+0000 7f7571e74740 1 --2- 172.21.15.26:0/1724698123 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a26dc7ee80 0x55a26dc82e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:32.464 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.464+0000 7f7547fff640 0 client.27370 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:37:32.464 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.464+0000 7f7571e74740 1 --2- 172.21.15.26:0/1724698123 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a26dc73cc0 0x55a26dc875b0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:32.464 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.464+0000 7f7571e74740 1 -- 172.21.15.26:0/1724698123 >> 172.21.15.26:0/1724698123 conn(0x55a26dc70230 msgr2=0x55a26dc6ea40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:32.464 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.464+0000 7f7571e74740 1 -- 172.21.15.26:0/1724698123 shutdown_connections 2024-09-11T23:37:32.464 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.464+0000 7f7571e74740 1 -- 172.21.15.26:0/1724698123 wait complete. 2024-09-11T23:37:32.474 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.465+0000 7f7571e74740 20 client.27370 trim_cache size 0 max 16384 2024-09-11T23:37:32.474 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-09-11T23:37:32.475 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "message": "", 2024-09-11T23:37:32.475 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-09-11T23:37:32.475 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-09-11T23:37:32.476 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.469+0000 7f756dd9f640 1 -- 172.21.15.26:0/1943764467 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f75500a31b0 msgr2=0x7f75500a5670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:32.476 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.469+0000 7f756dd9f640 1 --2- 172.21.15.26:0/1943764467 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f75500a31b0 0x7f75500a5670 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f75600f0150 tx=0x7f756002a910 comp rx=0 tx=0).stop 2024-09-11T23:37:32.477 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.469+0000 7f756dd9f640 1 -- 172.21.15.26:0/1943764467 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f75681007e0 msgr2=0x7f75681160a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:32.477 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.469+0000 7f756dd9f640 1 --2- 172.21.15.26:0/1943764467 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f75681007e0 0x7f75681160a0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f7560009ca0 tx=0x7f7560031690 comp rx=0 tx=0).stop 2024-09-11T23:37:32.477 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.469+0000 7f756dd9f640 1 -- 172.21.15.26:0/1943764467 shutdown_connections 2024-09-11T23:37:32.477 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.469+0000 7f756dd9f640 1 --2- 172.21.15.26:0/1943764467 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f75500a31b0 0x7f75500a5670 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:32.477 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.469+0000 7f756dd9f640 1 --2- 172.21.15.26:0/1943764467 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f75681067f0 0x7f75681111b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:32.477 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.469+0000 7f756dd9f640 1 --2- 172.21.15.26:0/1943764467 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7568101180 0x7f75681165e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:32.478 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.469+0000 7f756dd9f640 1 --2- 172.21.15.26:0/1943764467 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f75681007e0 0x7f75681160a0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:32.478 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.469+0000 7f756dd9f640 1 -- 172.21.15.26:0/1943764467 >> 172.21.15.26:0/1943764467 conn(0x7f75680fc4e0 msgr2=0x7f75680fcd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:32.478 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.469+0000 7f756dd9f640 1 -- 172.21.15.26:0/1943764467 shutdown_connections 2024-09-11T23:37:32.478 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:32.470+0000 7f756dd9f640 1 -- 172.21.15.26:0/1943764467 wait complete. 2024-09-11T23:37:32.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:37:32 smithi136 ceph-mon[35121]: pgmap v602: 97 pgs: 23 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 33 active+clean; 2.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.1 MiB/s wr, 169 op/s 2024-09-11T23:37:32.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:37:32 smithi026 ceph-mon[31172]: pgmap v602: 97 pgs: 23 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 33 active+clean; 2.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.1 MiB/s wr, 169 op/s 2024-09-11T23:37:32.552 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:37:32 smithi076 ceph-mon[35296]: pgmap v602: 97 pgs: 23 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 33 active+clean; 2.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.1 MiB/s wr, 169 op/s 2024-09-11T23:37:34.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:37:34 smithi136 ceph-mon[35121]: pgmap v603: 97 pgs: 23 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 33 active+clean; 2.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 150 op/s 2024-09-11T23:37:34.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:37:34 smithi026 ceph-mon[31172]: pgmap v603: 97 pgs: 23 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 33 active+clean; 2.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 150 op/s 2024-09-11T23:37:34.552 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:37:34 smithi076 ceph-mon[35296]: pgmap v603: 97 pgs: 23 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 33 active+clean; 2.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 150 op/s 2024-09-11T23:37:36.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:37:36 smithi136 ceph-mon[35121]: pgmap v604: 97 pgs: 22 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 34 active+clean; 2.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.8 MiB/s wr, 142 op/s 2024-09-11T23:37:36.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:37:36 smithi026 ceph-mon[31172]: pgmap v604: 97 pgs: 22 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 34 active+clean; 2.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.8 MiB/s wr, 142 op/s 2024-09-11T23:37:36.552 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:37:36 smithi076 ceph-mon[35296]: pgmap v604: 97 pgs: 22 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 34 active+clean; 2.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.8 MiB/s wr, 142 op/s 2024-09-11T23:37:38.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:37:38 smithi136 ceph-mon[35121]: pgmap v605: 97 pgs: 20 active+clean+snaptrim, 37 active+clean+snaptrim_wait, 40 active+clean; 2.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 164 op/s 2024-09-11T23:37:38.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:37:38 smithi026 ceph-mon[31172]: pgmap v605: 97 pgs: 20 active+clean+snaptrim, 37 active+clean+snaptrim_wait, 40 active+clean; 2.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 164 op/s 2024-09-11T23:37:38.552 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:37:38 smithi076 ceph-mon[35296]: pgmap v605: 97 pgs: 20 active+clean+snaptrim, 37 active+clean+snaptrim_wait, 40 active+clean; 2.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 164 op/s 2024-09-11T23:37:40.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:37:40 smithi136 ceph-mon[35121]: pgmap v606: 97 pgs: 20 active+clean+snaptrim, 37 active+clean+snaptrim_wait, 40 active+clean; 2.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 139 op/s 2024-09-11T23:37:40.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:37:40 smithi026 ceph-mon[31172]: pgmap v606: 97 pgs: 20 active+clean+snaptrim, 37 active+clean+snaptrim_wait, 40 active+clean; 2.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 139 op/s 2024-09-11T23:37:40.552 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:37:40 smithi076 ceph-mon[35296]: pgmap v606: 97 pgs: 20 active+clean+snaptrim, 37 active+clean+snaptrim_wait, 40 active+clean; 2.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 139 op/s 2024-09-11T23:37:42.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:37:42 smithi136 ceph-mon[35121]: pgmap v607: 97 pgs: 20 active+clean+snaptrim, 37 active+clean+snaptrim_wait, 40 active+clean; 2.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.0 MiB/s wr, 176 op/s 2024-09-11T23:37:42.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:37:42 smithi026 ceph-mon[31172]: pgmap v607: 97 pgs: 20 active+clean+snaptrim, 37 active+clean+snaptrim_wait, 40 active+clean; 2.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.0 MiB/s wr, 176 op/s 2024-09-11T23:37:42.552 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:37:42 smithi076 ceph-mon[35296]: pgmap v607: 97 pgs: 20 active+clean+snaptrim, 37 active+clean+snaptrim_wait, 40 active+clean; 2.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.0 MiB/s wr, 176 op/s 2024-09-11T23:37:44.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:37:44 smithi136 ceph-mon[35121]: pgmap v608: 97 pgs: 19 active+clean+snaptrim, 37 active+clean+snaptrim_wait, 41 active+clean; 2.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 166 op/s 2024-09-11T23:37:44.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:37:44 smithi026 ceph-mon[31172]: pgmap v608: 97 pgs: 19 active+clean+snaptrim, 37 active+clean+snaptrim_wait, 41 active+clean; 2.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 166 op/s 2024-09-11T23:37:44.552 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:37:44 smithi076 ceph-mon[35296]: pgmap v608: 97 pgs: 19 active+clean+snaptrim, 37 active+clean+snaptrim_wait, 41 active+clean; 2.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 166 op/s 2024-09-11T23:37:45.613 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.612+0000 7f33c8401640 1 -- 172.21.15.26:0/4240839948 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f33c01029a0 msgr2=0x7f33c0102d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:45.614 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.612+0000 7f33c8401640 1 --2- 172.21.15.26:0/4240839948 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f33c01029a0 0x7f33c0102d80 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f33b0009900 tx=0x7f33b002a400 comp rx=0 tx=0).stop 2024-09-11T23:37:45.614 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.612+0000 7f33c8401640 1 -- 172.21.15.26:0/4240839948 shutdown_connections 2024-09-11T23:37:45.614 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.612+0000 7f33c8401640 1 --2- 172.21.15.26:0/4240839948 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f33c0101b10 0x7f33c010d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:45.614 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.612+0000 7f33c8401640 1 --2- 172.21.15.26:0/4240839948 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f33c0101170 0x7f33c01015d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:45.614 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.612+0000 7f33c8401640 1 --2- 172.21.15.26:0/4240839948 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f33c01029a0 0x7f33c0102d80 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:45.614 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.612+0000 7f33c8401640 1 -- 172.21.15.26:0/4240839948 >> 172.21.15.26:0/4240839948 conn(0x7f33c00770f0 msgr2=0x7f33c00fe480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:45.614 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.613+0000 7f33c8401640 1 -- 172.21.15.26:0/4240839948 shutdown_connections 2024-09-11T23:37:45.614 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.613+0000 7f33c8401640 1 -- 172.21.15.26:0/4240839948 wait complete. 2024-09-11T23:37:45.616 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.613+0000 7f33c8401640 1 Processor -- start 2024-09-11T23:37:45.616 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.615+0000 7f33c8401640 1 -- start start 2024-09-11T23:37:45.616 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.615+0000 7f33c8401640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f33c0101170 0x7f33c01161c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:45.616 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.615+0000 7f33c8401640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f33c0101b10 0x7f33c0116700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:45.616 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.615+0000 7f33c8401640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f33c01029a0 0x7f33c0111370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:45.617 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.615+0000 7f33c8401640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f33c010fdc0 con 0x7f33c01029a0 2024-09-11T23:37:45.617 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.615+0000 7f33c8401640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f33c010fc40 con 0x7f33c0101b10 2024-09-11T23:37:45.617 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.615+0000 7f33c8401640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f33c010ff40 con 0x7f33c0101170 2024-09-11T23:37:45.617 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.616+0000 7f33c699e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f33c01029a0 0x7f33c0111370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:45.617 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.616+0000 7f33c619d640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f33c0101170 0x7f33c01161c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:45.617 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.616+0000 7f33c699e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f33c01029a0 0x7f33c0111370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57506/0 (socket says 172.21.15.26:57506) 2024-09-11T23:37:45.617 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.616+0000 7f33c699e640 1 -- 172.21.15.26:0/1165413106 learned_addr learned my addr 172.21.15.26:0/1165413106 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:37:45.617 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.616+0000 7f33c619d640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f33c0101170 0x7f33c01161c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:59408/0 (socket says 172.21.15.26:59408) 2024-09-11T23:37:45.617 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.616+0000 7f33c599c640 1 --2- 172.21.15.26:0/1165413106 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f33c0101b10 0x7f33c0116700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:45.618 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.616+0000 7f33c599c640 1 -- 172.21.15.26:0/1165413106 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f33c0101170 msgr2=0x7f33c01161c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:45.618 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.616+0000 7f33c599c640 1 --2- 172.21.15.26:0/1165413106 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f33c0101170 0x7f33c01161c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:45.618 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.616+0000 7f33c599c640 1 -- 172.21.15.26:0/1165413106 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f33c01029a0 msgr2=0x7f33c0111370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:45.618 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.616+0000 7f33c599c640 1 --2- 172.21.15.26:0/1165413106 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f33c01029a0 0x7f33c0111370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:45.619 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.616+0000 7f33c599c640 1 -- 172.21.15.26:0/1165413106 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33c0111980 con 0x7f33c0101b10 2024-09-11T23:37:45.619 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.617+0000 7f33c599c640 1 --2- 172.21.15.26:0/1165413106 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f33c0101b10 0x7f33c0116700 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f33bc00e990 tx=0x7f33bc00ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:45.619 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.617+0000 7f33b77fe640 1 -- 172.21.15.26:0/1165413106 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33bc00cbe0 con 0x7f33c0101b10 2024-09-11T23:37:45.619 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.617+0000 7f33c8401640 1 -- 172.21.15.26:0/1165413106 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33c0111c70 con 0x7f33c0101b10 2024-09-11T23:37:45.619 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.618+0000 7f33b77fe640 1 -- 172.21.15.26:0/1165413106 <== mon.2 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f33bc00cd80 con 0x7f33c0101b10 2024-09-11T23:37:45.619 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.618+0000 7f33c8401640 1 -- 172.21.15.26:0/1165413106 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33c006bc10 con 0x7f33c0101b10 2024-09-11T23:37:45.619 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.618+0000 7f33b77fe640 1 -- 172.21.15.26:0/1165413106 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33bc005220 con 0x7f33c0101b10 2024-09-11T23:37:45.623 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.623+0000 7f33b77fe640 1 -- 172.21.15.26:0/1165413106 <== mon.2 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f33bc0053c0 con 0x7f33c0101b10 2024-09-11T23:37:45.624 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.624+0000 7f33b77fe640 1 --2- 172.21.15.26:0/1165413106 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f33ac0a2e20 0x7f33ac0a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:45.624 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.624+0000 7f33cb271740 1 Processor -- start 2024-09-11T23:37:45.625 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.624+0000 7f33b77fe640 1 -- 172.21.15.26:0/1165413106 <== mon.2 v2:172.21.15.76:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 8586+0+0 (secure 0 0 0) 0x7f33bc0cf580 con 0x7f33c0101b10 2024-09-11T23:37:45.625 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.624+0000 7f33c619d640 1 --2- 172.21.15.26:0/1165413106 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f33ac0a2e20 0x7f33ac0a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:45.625 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.625+0000 7f33cb271740 1 -- start start 2024-09-11T23:37:45.626 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.625+0000 7f33cb271740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555a69e9caf0 0x555a69e9ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:45.626 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.625+0000 7f33cb271740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555a69e9d410 0x555a69ea7770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:45.626 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.625+0000 7f33cb271740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555a69ea7cb0 0x555a69eaa0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:45.626 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.625+0000 7f33cb271740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x555a69e012f0 con 0x555a69e9d410 2024-09-11T23:37:45.626 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.625+0000 7f33cb271740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x555a69da9270 con 0x555a69e9caf0 2024-09-11T23:37:45.626 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.625+0000 7f33cb271740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x555a69dfd2b0 con 0x555a69ea7cb0 2024-09-11T23:37:45.627 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.625+0000 7f33c619d640 1 --2- 172.21.15.26:0/1165413106 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f33ac0a2e20 0x7f33ac0a52e0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f33b0009230 tx=0x7f33b00058d0 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:45.627 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.625+0000 7f33c619d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555a69e9caf0 0x555a69e9ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:45.627 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.625+0000 7f33c699e640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555a69ea7cb0 0x555a69eaa0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:45.627 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.625+0000 7f33c619d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555a69e9caf0 0x555a69e9ced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:47820/0 (socket says 172.21.15.26:47820) 2024-09-11T23:37:45.627 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.625+0000 7f33c619d640 1 -- 172.21.15.26:0/1082877378 learned_addr learned my addr 172.21.15.26:0/1082877378 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:37:45.628 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.626+0000 7f33c619d640 1 -- 172.21.15.26:0/1082877378 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555a69ea7cb0 msgr2=0x555a69eaa0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:45.628 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.626+0000 7f33c619d640 1 --2- 172.21.15.26:0/1082877378 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555a69ea7cb0 0x555a69eaa0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:45.628 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.626+0000 7f33c619d640 1 -- 172.21.15.26:0/1082877378 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555a69e9d410 msgr2=0x555a69ea7770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:45.628 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.626+0000 7f33c619d640 1 --2- 172.21.15.26:0/1082877378 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555a69e9d410 0x555a69ea7770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:45.628 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.626+0000 7f33c619d640 1 -- 172.21.15.26:0/1082877378 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a69da8340 con 0x555a69e9caf0 2024-09-11T23:37:45.628 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.627+0000 7f33c699e640 1 --2- 172.21.15.26:0/1082877378 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555a69ea7cb0 0x555a69eaa0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:37:45.628 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.627+0000 7f33c619d640 1 --2- 172.21.15.26:0/1082877378 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555a69e9caf0 0x555a69e9ced0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f33b0033000 tx=0x7f33b003d000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9bc1b1a7a3e8bde0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:45.628 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.627+0000 7f33b4bfe640 1 -- 172.21.15.26:0/1082877378 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33b004d070 con 0x555a69e9caf0 2024-09-11T23:37:45.628 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.627+0000 7f33b4bfe640 1 -- 172.21.15.26:0/1082877378 <== mon.2 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f33b0003ad0 con 0x555a69e9caf0 2024-09-11T23:37:45.628 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.627+0000 7f33b4bfe640 1 -- 172.21.15.26:0/1082877378 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33b0035040 con 0x555a69e9caf0 2024-09-11T23:37:45.629 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.628+0000 7f33cb271740 1 -- 172.21.15.26:0/1082877378 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555a69e9caf0 msgr2=0x555a69e9ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:45.629 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.628+0000 7f33cb271740 1 --2- 172.21.15.26:0/1082877378 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555a69e9caf0 0x555a69e9ced0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f33b0033000 tx=0x7f33b003d000 comp rx=0 tx=0).stop 2024-09-11T23:37:45.629 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.629+0000 7f33cb271740 1 -- 172.21.15.26:0/1082877378 shutdown_connections 2024-09-11T23:37:45.629 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.629+0000 7f33cb271740 1 --2- 172.21.15.26:0/1082877378 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555a69ea7cb0 0x555a69eaa0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:45.629 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.629+0000 7f33cb271740 1 --2- 172.21.15.26:0/1082877378 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555a69e9d410 0x555a69ea7770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:45.629 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.629+0000 7f33cb271740 1 --2- 172.21.15.26:0/1082877378 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555a69e9caf0 0x555a69e9ced0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:45.629 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.629+0000 7f33cb271740 1 -- 172.21.15.26:0/1082877378 >> 172.21.15.26:0/1082877378 conn(0x555a69e99060 msgr2=0x555a69e99440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:45.630 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.630+0000 7f33cb271740 1 -- 172.21.15.26:0/1082877378 shutdown_connections 2024-09-11T23:37:45.630 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.630+0000 7f33cb271740 1 -- 172.21.15.26:0/1082877378 wait complete. 2024-09-11T23:37:45.631 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.630+0000 7f33cb271740 1 -- start start 2024-09-11T23:37:45.631 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.630+0000 7f33cb271740 1 Processor -- start 2024-09-11T23:37:45.631 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.631+0000 7f33cb271740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555a69e9caf0 0x555a69ea4b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:45.631 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.631+0000 7f33cb271740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555a69e9d410 0x555a69ea5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:45.631 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.631+0000 7f33cb271740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555a69ea7cb0 0x555a69ea5590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:45.631 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.631+0000 7f33cb271740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x555a69eaa880 con 0x555a69e9d410 2024-09-11T23:37:45.631 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.631+0000 7f33cb271740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x555a69d8a290 con 0x555a69ea7cb0 2024-09-11T23:37:45.631 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.631+0000 7f33cb271740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x555a69eaaa00 con 0x555a69e9caf0 2024-09-11T23:37:45.632 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.631+0000 7f33cb271740 10 client.0 _ll_register_callbacks cb 0x555a69dbb140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:37:45.633 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.631+0000 7f33c619d640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555a69e9caf0 0x555a69ea4b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:45.633 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.631+0000 7f33c619d640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555a69e9caf0 0x555a69ea4b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:59426/0 (socket says 172.21.15.26:59426) 2024-09-11T23:37:45.633 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.631+0000 7f33c619d640 1 -- 172.21.15.26:0/1171129276 learned_addr learned my addr 172.21.15.26:0/1171129276 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:37:45.633 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.631+0000 7f33c599c640 1 --2- 172.21.15.26:0/1171129276 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555a69e9d410 0x555a69ea5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:45.633 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.631+0000 7f33c699e640 1 --2- 172.21.15.26:0/1171129276 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555a69ea7cb0 0x555a69ea5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:45.633 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.632+0000 7f33c619d640 1 -- 172.21.15.26:0/1171129276 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555a69ea7cb0 msgr2=0x555a69ea5590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:45.634 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.632+0000 7f33c619d640 1 --2- 172.21.15.26:0/1171129276 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555a69ea7cb0 0x555a69ea5590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:45.634 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.632+0000 7f33c619d640 1 -- 172.21.15.26:0/1171129276 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555a69e9d410 msgr2=0x555a69ea5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:45.634 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.632+0000 7f33c619d640 1 --2- 172.21.15.26:0/1171129276 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555a69e9d410 0x555a69ea5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:45.634 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.632+0000 7f33c619d640 1 -- 172.21.15.26:0/1171129276 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a69b14210 con 0x555a69e9caf0 2024-09-11T23:37:45.634 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.632+0000 7f33c699e640 1 --2- 172.21.15.26:0/1171129276 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555a69ea7cb0 0x555a69ea5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:37:45.634 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.632+0000 7f33c619d640 1 --2- 172.21.15.26:0/1171129276 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555a69e9caf0 0x555a69ea4b10 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f33b0004770 tx=0x7f33b0031400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eebe8f2006568647 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:45.634 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.632+0000 7f33a9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-09-11T23:37:45.634 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.632+0000 7f33a9ffb640 1 -- 172.21.15.26:0/1171129276 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33b005a070 con 0x555a69e9caf0 2024-09-11T23:37:45.634 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.632+0000 7f33cb271740 1 -- 172.21.15.26:0/1171129276 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f33b0003ad0 con 0x555a69e9caf0 2024-09-11T23:37:45.634 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.632+0000 7f33a9ffb640 1 -- 172.21.15.26:0/1171129276 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 8586+0+0 (secure 0 0 0) 0x7f33b00037f0 con 0x555a69e9caf0 2024-09-11T23:37:45.634 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.633+0000 7f33a9ffb640 1 -- 172.21.15.26:0/1171129276 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f33b0066050 con 0x555a69e9caf0 2024-09-11T23:37:45.634 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.633+0000 7f33cb271740 10 client.27382 fetch_fsmap learned FSMap version 86 2024-09-11T23:37:45.634 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.633+0000 7f33cb271740 1 -- 172.21.15.26:0/1171129276 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=86,monmap=4+}) -- 0x555a69be0620 con 0x555a69e9caf0 2024-09-11T23:37:45.638 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.637+0000 7f33a9ffb640 1 -- 172.21.15.26:0/1171129276 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f33b0003ad0 con 0x555a69e9caf0 2024-09-11T23:37:45.638 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.637+0000 7f33a9ffb640 1 -- 172.21.15.26:0/1171129276 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 86) ==== 8426+0+0 (secure 0 0 0) 0x7f33b0052a30 con 0x555a69e9caf0 2024-09-11T23:37:45.638 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.638+0000 7f33cb271740 10 client.27382 fetch_fsmap finished waiting for FSMap version 86 2024-09-11T23:37:45.638 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.638+0000 7f33cb271740 10 client.27382 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:37:45.638 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.638+0000 7f33cb271740 20 client.27382 populate_metadata read hostname 'smithi026' 2024-09-11T23:37:45.638 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.638+0000 7f33cb271740 1 --2- 172.21.15.26:0/1171129276 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x555a69eb7a90 0x555a69eb9e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:45.638 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.638+0000 7f33cb271740 4 client.27382 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:37:45.638 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.638+0000 7f33cb271740 1 -- 172.21.15.26:0/1171129276 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555a69e0ea20 con 0x555a69eb7a90 2024-09-11T23:37:45.639 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.638+0000 7f33c599c640 1 --2- 172.21.15.26:0/1171129276 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x555a69eb7a90 0x555a69eb9e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:45.639 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.638+0000 7f33a9ffb640 10 client.27382 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:37:45.639 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.639+0000 7f33c599c640 1 --2- 172.21.15.26:0/1171129276 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x555a69eb7a90 0x555a69eb9e80 crc :-1 s=READY pgs=290 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c307463d19225a8 server_cookie=fe0e74a155e031eb in_seq=0 out_seq=0 2024-09-11T23:37:45.640 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.640+0000 7f33a9ffb640 1 -- 172.21.15.26:0/1171129276 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555a69e0ea20 con 0x555a69eb7a90 2024-09-11T23:37:45.641 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.640+0000 7f33a9ffb640 10 client.27382 handle_command_reply: tid=0 2024-09-11T23:37:45.641 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.640+0000 7f33cb271740 1 client.27382 shutdown 2024-09-11T23:37:45.644 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.643+0000 7f33cb271740 1 -- 172.21.15.26:0/1171129276 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555a69e9caf0 msgr2=0x555a69ea4b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:45.644 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.643+0000 7f33cb271740 1 --2- 172.21.15.26:0/1171129276 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555a69e9caf0 0x555a69ea4b10 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f33b0004770 tx=0x7f33b0031400 comp rx=0 tx=0).stop 2024-09-11T23:37:45.644 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.644+0000 7f33cb271740 1 -- 172.21.15.26:0/1171129276 shutdown_connections 2024-09-11T23:37:45.644 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.644+0000 7f33cb271740 1 --2- 172.21.15.26:0/1171129276 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555a69ea7cb0 0x555a69ea5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:45.644 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.644+0000 7f33cb271740 1 --2- 172.21.15.26:0/1171129276 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x555a69eb7a90 0x555a69eb9e80 crc :-1 s=READY pgs=290 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:45.644 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.644+0000 7f33cb271740 1 --2- 172.21.15.26:0/1171129276 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555a69e9d410 0x555a69ea5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:45.644 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.644+0000 7f33cb271740 1 --2- 172.21.15.26:0/1171129276 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555a69e9caf0 0x555a69ea4b10 secure :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f33b0004770 tx=0x7f33b0031400 comp rx=0 tx=0).stop 2024-09-11T23:37:45.645 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.644+0000 7f33cb271740 1 -- 172.21.15.26:0/1171129276 >> 172.21.15.26:0/1171129276 conn(0x555a69e99060 msgr2=0x555a69e9edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:45.645 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.644+0000 7f33a9ffb640 0 client.27382 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:37:45.645 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.644+0000 7f33cb271740 1 -- 172.21.15.26:0/1171129276 shutdown_connections 2024-09-11T23:37:45.645 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.644+0000 7f33cb271740 1 -- 172.21.15.26:0/1171129276 wait complete. 2024-09-11T23:37:45.645 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.644+0000 7f33cb271740 20 client.27382 trim_cache size 0 max 16384 2024-09-11T23:37:45.674 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.673+0000 7f33cb271740 1 Processor -- start 2024-09-11T23:37:45.674 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.673+0000 7f33cb271740 1 -- start start 2024-09-11T23:37:45.674 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.674+0000 7f33cb271740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555a69e9d410 0x555a69ea53c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:45.674 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.674+0000 7f33cb271740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555a69ea7cb0 0x555a69ea6a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:45.675 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.674+0000 7f33cb271740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555a69ea3be0 0x555a69ebf4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:45.675 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.674+0000 7f33cb271740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f33a400e8c0 con 0x555a69ea3be0 2024-09-11T23:37:45.675 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.674+0000 7f33cb271740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f33a40106d0 con 0x555a69ea7cb0 2024-09-11T23:37:45.675 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.674+0000 7f33cb271740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f33a400f020 con 0x555a69e9d410 2024-09-11T23:37:45.675 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.674+0000 7f33c619d640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555a69e9d410 0x555a69ea53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:45.675 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.674+0000 7f33c619d640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555a69e9d410 0x555a69ea53c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:59430/0 (socket says 172.21.15.26:59430) 2024-09-11T23:37:45.676 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.674+0000 7f33c619d640 1 -- 172.21.15.26:0/838343397 learned_addr learned my addr 172.21.15.26:0/838343397 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:37:45.676 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.674+0000 7f33c599c640 1 --2- 172.21.15.26:0/838343397 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555a69ea7cb0 0x555a69ea6a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:45.676 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.675+0000 7f33c699e640 1 --2- 172.21.15.26:0/838343397 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555a69ea3be0 0x555a69ebf4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:45.676 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.675+0000 7f33c619d640 1 -- 172.21.15.26:0/838343397 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555a69ea7cb0 msgr2=0x555a69ea6a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:45.676 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.675+0000 7f33c619d640 1 --2- 172.21.15.26:0/838343397 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555a69ea7cb0 0x555a69ea6a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:45.676 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.675+0000 7f33c619d640 1 -- 172.21.15.26:0/838343397 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555a69ea3be0 msgr2=0x555a69ebf4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:45.676 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.675+0000 7f33c619d640 1 --2- 172.21.15.26:0/838343397 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555a69ea3be0 0x555a69ebf4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:45.676 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.675+0000 7f33c619d640 1 -- 172.21.15.26:0/838343397 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a69e98e90 con 0x555a69e9d410 2024-09-11T23:37:45.676 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.675+0000 7f33c699e640 1 --2- 172.21.15.26:0/838343397 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555a69ea3be0 0x555a69ebf4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:37:45.677 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.676+0000 7f33c619d640 1 --2- 172.21.15.26:0/838343397 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555a69e9d410 0x555a69ea53c0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f33b0009900 tx=0x7f33b0005940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=73ed3c996897d007 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:45.677 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.677+0000 7f33abfff640 1 -- 172.21.15.26:0/838343397 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33b0040070 con 0x555a69e9d410 2024-09-11T23:37:45.677 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.677+0000 7f33abfff640 1 -- 172.21.15.26:0/838343397 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f33b0003ad0 con 0x555a69e9d410 2024-09-11T23:37:45.677 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.677+0000 7f33abfff640 1 -- 172.21.15.26:0/838343397 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33b005a040 con 0x555a69e9d410 2024-09-11T23:37:45.679 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.678+0000 7f33cb271740 1 -- 172.21.15.26:0/838343397 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555a69e9d410 msgr2=0x555a69ea53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:45.679 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.678+0000 7f33cb271740 1 --2- 172.21.15.26:0/838343397 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555a69e9d410 0x555a69ea53c0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f33b0009900 tx=0x7f33b0005940 comp rx=0 tx=0).stop 2024-09-11T23:37:45.679 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.678+0000 7f33cb271740 1 -- 172.21.15.26:0/838343397 shutdown_connections 2024-09-11T23:37:45.679 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.678+0000 7f33cb271740 1 --2- 172.21.15.26:0/838343397 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555a69ea7cb0 0x555a69ea6a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:45.679 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.678+0000 7f33cb271740 1 --2- 172.21.15.26:0/838343397 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555a69ea3be0 0x555a69ebf4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:45.679 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.678+0000 7f33cb271740 1 --2- 172.21.15.26:0/838343397 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555a69e9d410 0x555a69ea53c0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:45.679 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.679+0000 7f33cb271740 1 -- 172.21.15.26:0/838343397 >> 172.21.15.26:0/838343397 conn(0x555a69e99060 msgr2=0x555a69ea6370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:45.679 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.679+0000 7f33cb271740 1 -- 172.21.15.26:0/838343397 shutdown_connections 2024-09-11T23:37:45.680 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.679+0000 7f33cb271740 1 -- 172.21.15.26:0/838343397 wait complete. 2024-09-11T23:37:45.680 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.679+0000 7f33cb271740 1 -- start start 2024-09-11T23:37:45.680 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.680+0000 7f33cb271740 1 Processor -- start 2024-09-11T23:37:45.680 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.680+0000 7f33cb271740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555a69e9d410 0x555a69eb0570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:45.682 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.681+0000 7f33cb271740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555a69ea3be0 0x555a69ea1910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:45.682 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.681+0000 7f33cb271740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555a69ea7cb0 0x555a69ea1f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:45.682 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.681+0000 7f33cb271740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f33a400ff40 con 0x555a69e9d410 2024-09-11T23:37:45.682 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.681+0000 7f33c619d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555a69e9d410 0x555a69eb0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:45.682 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.681+0000 7f33c619d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555a69e9d410 0x555a69eb0570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57552/0 (socket says 172.21.15.26:57552) 2024-09-11T23:37:45.682 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.681+0000 7f33cb271740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f33a400f7b0 con 0x555a69ea3be0 2024-09-11T23:37:45.682 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.681+0000 7f33c619d640 1 -- 172.21.15.26:0/2780456737 learned_addr learned my addr 172.21.15.26:0/2780456737 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:37:45.682 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.681+0000 7f33cb271740 1 -- 172.21.15.26:0/2780456737 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f33a400c8c0 con 0x555a69ea7cb0 2024-09-11T23:37:45.682 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.681+0000 7f33c599c640 1 --2- 172.21.15.26:0/2780456737 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555a69ea3be0 0x555a69ea1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:45.682 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.681+0000 7f33cb271740 10 client.0 _ll_register_callbacks cb 0x7f33bc005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:37:45.682 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.681+0000 7f33c699e640 1 --2- 172.21.15.26:0/2780456737 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555a69ea7cb0 0x555a69ea1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:45.682 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.681+0000 7f33c619d640 1 -- 172.21.15.26:0/2780456737 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555a69ea7cb0 msgr2=0x555a69ea1f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:45.683 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.681+0000 7f33c619d640 1 --2- 172.21.15.26:0/2780456737 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555a69ea7cb0 0x555a69ea1f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:45.683 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.681+0000 7f33c619d640 1 -- 172.21.15.26:0/2780456737 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555a69ea3be0 msgr2=0x555a69ea1910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:45.683 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.681+0000 7f33c619d640 1 --2- 172.21.15.26:0/2780456737 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555a69ea3be0 0x555a69ea1910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:45.683 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.681+0000 7f33c619d640 1 -- 172.21.15.26:0/2780456737 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a69ea2720 con 0x555a69e9d410 2024-09-11T23:37:45.684 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.682+0000 7f33c619d640 1 --2- 172.21.15.26:0/2780456737 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555a69e9d410 0x555a69eb0570 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f33b0006010 tx=0x7f33b0009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac272fc1d0c7d88 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:45.684 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.682+0000 7f33a9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:37:45.684 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.682+0000 7f33a9ffb640 1 -- 172.21.15.26:0/2780456737 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33b0040040 con 0x555a69e9d410 2024-09-11T23:37:45.684 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.682+0000 7f33a9ffb640 1 -- 172.21.15.26:0/2780456737 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 8586+0+0 (secure 0 0 0) 0x7f33b00037f0 con 0x555a69e9d410 2024-09-11T23:37:45.684 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.682+0000 7f33cb271740 1 -- 172.21.15.26:0/2780456737 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f33b0003ad0 con 0x555a69e9d410 2024-09-11T23:37:45.684 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.683+0000 7f33a9ffb640 1 -- 172.21.15.26:0/2780456737 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f33b005a040 con 0x555a69e9d410 2024-09-11T23:37:45.684 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.683+0000 7f33cb271740 10 client.17919 fetch_fsmap learned FSMap version 86 2024-09-11T23:37:45.684 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.683+0000 7f33cb271740 1 -- 172.21.15.26:0/2780456737 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=86,monmap=4+}) -- 0x555a69f225a0 con 0x555a69e9d410 2024-09-11T23:37:45.685 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.683+0000 7f33a9ffb640 1 -- 172.21.15.26:0/2780456737 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f33b0049d30 con 0x555a69e9d410 2024-09-11T23:37:45.685 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.683+0000 7f33a9ffb640 1 -- 172.21.15.26:0/2780456737 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 86) ==== 8426+0+0 (secure 0 0 0) 0x7f33b00529a0 con 0x555a69e9d410 2024-09-11T23:37:45.685 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.683+0000 7f33cb271740 10 client.17919 fetch_fsmap finished waiting for FSMap version 86 2024-09-11T23:37:45.685 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.683+0000 7f33cb271740 10 client.17919 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:37:45.685 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.683+0000 7f33cb271740 20 client.17919 populate_metadata read hostname 'smithi026' 2024-09-11T23:37:45.685 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.684+0000 7f33cb271740 1 --2- 172.21.15.26:0/2780456737 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x555a69f22ad0 0x555a69f24ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:45.685 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.684+0000 7f33cb271740 4 client.17919 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T23:37:45.685 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.684+0000 7f33cb271740 1 -- 172.21.15.26:0/2780456737 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f33b0003ad0 con 0x555a69f22ad0 2024-09-11T23:37:45.686 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.685+0000 7f33c599c640 1 --2- 172.21.15.26:0/2780456737 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x555a69f22ad0 0x555a69f24ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:45.686 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.685+0000 7f33c599c640 1 --2- 172.21.15.26:0/2780456737 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x555a69f22ad0 0x555a69f24ef0 crc :-1 s=READY pgs=291 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8aa07f0133649307 server_cookie=13f97cc9f0f9c688 in_seq=0 out_seq=0 2024-09-11T23:37:45.686 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.685+0000 7f33a9ffb640 10 client.17919 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:37:45.770 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.769+0000 7f33a9ffb640 1 -- 172.21.15.26:0/2780456737 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f33b0003ad0 con 0x555a69f22ad0 2024-09-11T23:37:45.770 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.769+0000 7f33a9ffb640 10 client.17919 handle_command_reply: tid=0 2024-09-11T23:37:45.770 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.769+0000 7f33cb271740 1 client.17919 shutdown 2024-09-11T23:37:45.771 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.769+0000 7f33cb271740 1 -- 172.21.15.26:0/2780456737 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555a69e9d410 msgr2=0x555a69eb0570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:45.771 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.769+0000 7f33cb271740 1 --2- 172.21.15.26:0/2780456737 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555a69e9d410 0x555a69eb0570 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f33b0006010 tx=0x7f33b0009260 comp rx=0 tx=0).stop 2024-09-11T23:37:45.771 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.769+0000 7f33cb271740 1 -- 172.21.15.26:0/2780456737 shutdown_connections 2024-09-11T23:37:45.771 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.769+0000 7f33cb271740 1 --2- 172.21.15.26:0/2780456737 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555a69ea3be0 0x555a69ea1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:45.771 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.769+0000 7f33cb271740 1 --2- 172.21.15.26:0/2780456737 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x555a69f22ad0 0x555a69f24ef0 crc :-1 s=READY pgs=291 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:45.771 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.769+0000 7f33cb271740 1 --2- 172.21.15.26:0/2780456737 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555a69ea7cb0 0x555a69ea1f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:45.771 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.769+0000 7f33cb271740 1 --2- 172.21.15.26:0/2780456737 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555a69e9d410 0x555a69eb0570 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:45.771 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.769+0000 7f33a9ffb640 0 client.17919 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:37:45.771 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.770+0000 7f33cb271740 1 -- 172.21.15.26:0/2780456737 >> 172.21.15.26:0/2780456737 conn(0x555a69e99060 msgr2=0x555a69e9ece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:45.771 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.770+0000 7f33cb271740 1 -- 172.21.15.26:0/2780456737 shutdown_connections 2024-09-11T23:37:45.771 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.770+0000 7f33cb271740 1 -- 172.21.15.26:0/2780456737 wait complete. 2024-09-11T23:37:45.771 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.770+0000 7f33cb271740 20 client.17919 trim_cache size 0 max 16384 2024-09-11T23:37:45.772 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-09-11T23:37:45.772 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "message": "", 2024-09-11T23:37:45.772 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-09-11T23:37:45.772 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-09-11T23:37:45.773 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.770+0000 7f33c8401640 1 -- 172.21.15.26:0/1165413106 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f33ac0a2e20 msgr2=0x7f33ac0a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:45.773 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.770+0000 7f33c8401640 1 --2- 172.21.15.26:0/1165413106 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f33ac0a2e20 0x7f33ac0a52e0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f33b0009230 tx=0x7f33b00058d0 comp rx=0 tx=0).stop 2024-09-11T23:37:45.773 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.770+0000 7f33c8401640 1 -- 172.21.15.26:0/1165413106 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f33c0101b10 msgr2=0x7f33c0116700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:45.773 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.770+0000 7f33c8401640 1 --2- 172.21.15.26:0/1165413106 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f33c0101b10 0x7f33c0116700 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f33bc00e990 tx=0x7f33bc00ee50 comp rx=0 tx=0).stop 2024-09-11T23:37:45.773 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.770+0000 7f33c8401640 1 -- 172.21.15.26:0/1165413106 shutdown_connections 2024-09-11T23:37:45.773 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.770+0000 7f33c8401640 1 --2- 172.21.15.26:0/1165413106 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f33ac0a2e20 0x7f33ac0a52e0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:45.773 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.770+0000 7f33c8401640 1 --2- 172.21.15.26:0/1165413106 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f33c0101b10 0x7f33c0116700 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:45.773 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.770+0000 7f33c8401640 1 --2- 172.21.15.26:0/1165413106 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f33c01029a0 0x7f33c0111370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:45.773 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.770+0000 7f33c8401640 1 --2- 172.21.15.26:0/1165413106 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f33c0101170 0x7f33c01161c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:45.774 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.770+0000 7f33c8401640 1 -- 172.21.15.26:0/1165413106 >> 172.21.15.26:0/1165413106 conn(0x7f33c00770f0 msgr2=0x7f33c0104a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:45.774 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.770+0000 7f33c8401640 1 -- 172.21.15.26:0/1165413106 shutdown_connections 2024-09-11T23:37:45.774 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:45.770+0000 7f33c8401640 1 -- 172.21.15.26:0/1165413106 wait complete. 2024-09-11T23:37:46.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:37:46 smithi136 ceph-mon[35121]: pgmap v609: 97 pgs: 16 active+clean+snaptrim, 33 active+clean+snaptrim_wait, 48 active+clean; 2.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 161 op/s 2024-09-11T23:37:46.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:37:46 smithi026 ceph-mon[31172]: pgmap v609: 97 pgs: 16 active+clean+snaptrim, 33 active+clean+snaptrim_wait, 48 active+clean; 2.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 161 op/s 2024-09-11T23:37:46.552 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:37:46 smithi076 ceph-mon[35296]: pgmap v609: 97 pgs: 16 active+clean+snaptrim, 33 active+clean+snaptrim_wait, 48 active+clean; 2.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 161 op/s 2024-09-11T23:37:48.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:37:48 smithi136 ceph-mon[35121]: pgmap v610: 97 pgs: 14 active+clean+snaptrim, 29 active+clean+snaptrim_wait, 54 active+clean; 2.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 204 op/s 2024-09-11T23:37:48.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:37:48 smithi026 ceph-mon[31172]: pgmap v610: 97 pgs: 14 active+clean+snaptrim, 29 active+clean+snaptrim_wait, 54 active+clean; 2.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 204 op/s 2024-09-11T23:37:48.552 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:37:48 smithi076 ceph-mon[35296]: pgmap v610: 97 pgs: 14 active+clean+snaptrim, 29 active+clean+snaptrim_wait, 54 active+clean; 2.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 204 op/s 2024-09-11T23:37:50.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:37:50 smithi136 ceph-mon[35121]: pgmap v611: 97 pgs: 14 active+clean+snaptrim, 29 active+clean+snaptrim_wait, 54 active+clean; 2.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 167 op/s 2024-09-11T23:37:50.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:37:50 smithi026 ceph-mon[31172]: pgmap v611: 97 pgs: 14 active+clean+snaptrim, 29 active+clean+snaptrim_wait, 54 active+clean; 2.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 167 op/s 2024-09-11T23:37:50.552 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:37:50 smithi076 ceph-mon[35296]: pgmap v611: 97 pgs: 14 active+clean+snaptrim, 29 active+clean+snaptrim_wait, 54 active+clean; 2.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 167 op/s 2024-09-11T23:37:52.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:37:52 smithi136 ceph-mon[35121]: pgmap v612: 97 pgs: 11 active+clean+snaptrim, 22 active+clean+snaptrim_wait, 64 active+clean; 2.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 221 op/s 2024-09-11T23:37:52.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:37:52 smithi026 ceph-mon[31172]: pgmap v612: 97 pgs: 11 active+clean+snaptrim, 22 active+clean+snaptrim_wait, 64 active+clean; 2.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 221 op/s 2024-09-11T23:37:52.552 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:37:52 smithi076 ceph-mon[35296]: pgmap v612: 97 pgs: 11 active+clean+snaptrim, 22 active+clean+snaptrim_wait, 64 active+clean; 2.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 221 op/s 2024-09-11T23:37:54.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:37:54 smithi136 ceph-mon[35121]: pgmap v613: 97 pgs: 11 active+clean+snaptrim, 22 active+clean+snaptrim_wait, 64 active+clean; 2.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 218 op/s 2024-09-11T23:37:54.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:37:54 smithi026 ceph-mon[31172]: pgmap v613: 97 pgs: 11 active+clean+snaptrim, 22 active+clean+snaptrim_wait, 64 active+clean; 2.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 218 op/s 2024-09-11T23:37:54.552 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:37:54 smithi076 ceph-mon[35296]: pgmap v613: 97 pgs: 11 active+clean+snaptrim, 22 active+clean+snaptrim_wait, 64 active+clean; 2.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 218 op/s 2024-09-11T23:37:56.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:37:56 smithi136 ceph-mon[35121]: pgmap v614: 97 pgs: 11 active+clean+snaptrim, 22 active+clean+snaptrim_wait, 64 active+clean; 2.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.0 MiB/s wr, 214 op/s 2024-09-11T23:37:56.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:37:56 smithi026 ceph-mon[31172]: pgmap v614: 97 pgs: 11 active+clean+snaptrim, 22 active+clean+snaptrim_wait, 64 active+clean; 2.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.0 MiB/s wr, 214 op/s 2024-09-11T23:37:56.552 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:37:56 smithi076 ceph-mon[35296]: pgmap v614: 97 pgs: 11 active+clean+snaptrim, 22 active+clean+snaptrim_wait, 64 active+clean; 2.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.0 MiB/s wr, 214 op/s 2024-09-11T23:37:58.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:37:58 smithi136 ceph-mon[35121]: pgmap v615: 97 pgs: 6 active+clean+snaptrim, 16 active+clean+snaptrim_wait, 75 active+clean; 2.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.4 MiB/s wr, 256 op/s 2024-09-11T23:37:58.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:37:58 smithi026 ceph-mon[31172]: pgmap v615: 97 pgs: 6 active+clean+snaptrim, 16 active+clean+snaptrim_wait, 75 active+clean; 2.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.4 MiB/s wr, 256 op/s 2024-09-11T23:37:58.552 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:37:58 smithi076 ceph-mon[35296]: pgmap v615: 97 pgs: 6 active+clean+snaptrim, 16 active+clean+snaptrim_wait, 75 active+clean; 2.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.4 MiB/s wr, 256 op/s 2024-09-11T23:37:58.895 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.893+0000 7f9301a5e640 1 -- 172.21.15.26:0/4234804472 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f92fc101ad0 msgr2=0x7f92fc101f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:58.896 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.893+0000 7f9301a5e640 1 --2- 172.21.15.26:0/4234804472 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f92fc101ad0 0x7f92fc101f30 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f92f0009930 tx=0x7f92f002a420 comp rx=0 tx=0).stop 2024-09-11T23:37:58.896 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.893+0000 7f9301a5e640 1 -- 172.21.15.26:0/4234804472 shutdown_connections 2024-09-11T23:37:58.896 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.893+0000 7f9301a5e640 1 --2- 172.21.15.26:0/4234804472 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f92fc102470 0x7f92fc10e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:58.896 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.893+0000 7f9301a5e640 1 --2- 172.21.15.26:0/4234804472 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f92fc101ad0 0x7f92fc101f30 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:58.896 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.893+0000 7f9301a5e640 1 --2- 172.21.15.26:0/4234804472 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f92fc107b70 0x7f92fc107f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:58.896 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.893+0000 7f9301a5e640 1 -- 172.21.15.26:0/4234804472 >> 172.21.15.26:0/4234804472 conn(0x7f92fc0fd7d0 msgr2=0x7f92fc0ffbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:58.896 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.893+0000 7f9301a5e640 1 -- 172.21.15.26:0/4234804472 shutdown_connections 2024-09-11T23:37:58.896 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.894+0000 7f9301a5e640 1 -- 172.21.15.26:0/4234804472 wait complete. 2024-09-11T23:37:58.896 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.894+0000 7f9301a5e640 1 Processor -- start 2024-09-11T23:37:58.896 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.894+0000 7f9301a5e640 1 -- start start 2024-09-11T23:37:58.896 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.894+0000 7f9301a5e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f92fc101ad0 0x7f92fc198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:58.897 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.894+0000 7f9301a5e640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f92fc102470 0x7f92fc198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:58.897 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.894+0000 7f9301a5e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f92fc107b70 0x7f92fc19cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:58.897 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.894+0000 7f9301a5e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f92fc111f20 con 0x7f92fc101ad0 2024-09-11T23:37:58.897 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.894+0000 7f9301a5e640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f92fc111da0 con 0x7f92fc107b70 2024-09-11T23:37:58.897 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.894+0000 7f9301a5e640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f92fc1120a0 con 0x7f92fc102470 2024-09-11T23:37:58.897 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.895+0000 7f92faffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f92fc101ad0 0x7f92fc198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:58.897 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.895+0000 7f92faffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f92fc101ad0 0x7f92fc198330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43158/0 (socket says 172.21.15.26:43158) 2024-09-11T23:37:58.897 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.895+0000 7f92faffd640 1 -- 172.21.15.26:0/3169914100 learned_addr learned my addr 172.21.15.26:0/3169914100 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:37:58.897 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.895+0000 7f92fa7fc640 1 --2- 172.21.15.26:0/3169914100 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f92fc102470 0x7f92fc198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:58.897 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.895+0000 7f92fb7fe640 1 --2- 172.21.15.26:0/3169914100 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f92fc107b70 0x7f92fc19cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:58.897 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.895+0000 7f92faffd640 1 -- 172.21.15.26:0/3169914100 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f92fc102470 msgr2=0x7f92fc198870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:58.897 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.895+0000 7f92faffd640 1 --2- 172.21.15.26:0/3169914100 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f92fc102470 0x7f92fc198870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:58.897 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.895+0000 7f92faffd640 1 -- 172.21.15.26:0/3169914100 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f92fc107b70 msgr2=0x7f92fc19cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:58.898 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.895+0000 7f92faffd640 1 --2- 172.21.15.26:0/3169914100 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f92fc107b70 0x7f92fc19cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:58.898 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.895+0000 7f92faffd640 1 -- 172.21.15.26:0/3169914100 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92fc19d320 con 0x7f92fc101ad0 2024-09-11T23:37:58.898 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.895+0000 7f92faffd640 1 --2- 172.21.15.26:0/3169914100 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f92fc101ad0 0x7f92fc198330 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f92ec00d950 tx=0x7f92ec00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:58.898 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.896+0000 7f9300a5c640 1 -- 172.21.15.26:0/3169914100 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92ec017030 con 0x7f92fc101ad0 2024-09-11T23:37:58.899 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.896+0000 7f9300a5c640 1 -- 172.21.15.26:0/3169914100 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f92ec004020 con 0x7f92fc101ad0 2024-09-11T23:37:58.899 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.896+0000 7f9301a5e640 1 -- 172.21.15.26:0/3169914100 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92fc19d610 con 0x7f92fc101ad0 2024-09-11T23:37:58.899 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.896+0000 7f9300a5c640 1 -- 172.21.15.26:0/3169914100 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92ec0047a0 con 0x7f92fc101ad0 2024-09-11T23:37:58.899 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.896+0000 7f9301a5e640 1 -- 172.21.15.26:0/3169914100 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92fc1a4ea0 con 0x7f92fc101ad0 2024-09-11T23:37:58.900 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.899+0000 7f93047f9740 1 Processor -- start 2024-09-11T23:37:58.901 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.900+0000 7f93047f9740 1 -- start start 2024-09-11T23:37:58.901 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.900+0000 7f93047f9740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56134e7b5af0 0x56134e7b5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:58.901 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.900+0000 7f93047f9740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56134e7b6410 0x56134e7c0770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:58.901 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.900+0000 7f93047f9740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56134e7c0cb0 0x56134e7c30a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:58.901 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.900+0000 7f93047f9740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x56134e71a2f0 con 0x56134e7b6410 2024-09-11T23:37:58.901 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.900+0000 7f93047f9740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x56134e6c2270 con 0x56134e7c0cb0 2024-09-11T23:37:58.901 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.900+0000 7f93047f9740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x56134e7162b0 con 0x56134e7b5af0 2024-09-11T23:37:58.901 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.900+0000 7f92fb7fe640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56134e7b5af0 0x56134e7b5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:58.901 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.900+0000 7f92fb7fe640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56134e7b5af0 0x56134e7b5ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:44598/0 (socket says 172.21.15.26:44598) 2024-09-11T23:37:58.901 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.900+0000 7f92fb7fe640 1 -- 172.21.15.26:0/2342244484 learned_addr learned my addr 172.21.15.26:0/2342244484 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:37:58.902 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.901+0000 7f92fb7fe640 1 -- 172.21.15.26:0/2342244484 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56134e7c0cb0 msgr2=0x56134e7c30a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T23:37:58.902 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.901+0000 7f92fb7fe640 1 --2- 172.21.15.26:0/2342244484 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56134e7c0cb0 0x56134e7c30a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:58.902 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.901+0000 7f92fb7fe640 1 -- 172.21.15.26:0/2342244484 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56134e7b6410 msgr2=0x56134e7c0770 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T23:37:58.902 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.901+0000 7f92fb7fe640 1 --2- 172.21.15.26:0/2342244484 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56134e7b6410 0x56134e7c0770 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:58.902 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.901+0000 7f92fb7fe640 1 -- 172.21.15.26:0/2342244484 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56134e6c1340 con 0x56134e7b5af0 2024-09-11T23:37:58.902 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.901+0000 7f9300a5c640 1 -- 172.21.15.26:0/3169914100 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f92ec0041d0 con 0x7f92fc101ad0 2024-09-11T23:37:58.903 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.902+0000 7f9300a5c640 1 --2- 172.21.15.26:0/3169914100 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f92e40a2e20 0x7f92e40a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:58.903 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.902+0000 7f9300a5c640 1 -- 172.21.15.26:0/3169914100 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 8586+0+0 (secure 0 0 0) 0x7f92ec0cede0 con 0x7f92fc101ad0 2024-09-11T23:37:58.903 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.903+0000 7f92faffd640 1 --2- 172.21.15.26:0/3169914100 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f92e40a2e20 0x7f92e40a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:58.904 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.903+0000 7f92fb7fe640 1 --2- 172.21.15.26:0/2342244484 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56134e7b5af0 0x56134e7b5ed0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f92f4009fd0 tx=0x7f92f40118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6cd8a870d4d43b8f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:58.904 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.903+0000 7f92e1ffb640 1 -- 172.21.15.26:0/2342244484 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92f4004d20 con 0x56134e7b5af0 2024-09-11T23:37:58.904 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.904+0000 7f92e1ffb640 1 -- 172.21.15.26:0/2342244484 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f92f401a070 con 0x56134e7b5af0 2024-09-11T23:37:58.905 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.905+0000 7f93047f9740 1 -- 172.21.15.26:0/2342244484 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56134e7b5af0 msgr2=0x56134e7b5ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:58.905 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.905+0000 7f93047f9740 1 --2- 172.21.15.26:0/2342244484 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56134e7b5af0 0x56134e7b5ed0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f92f4009fd0 tx=0x7f92f40118d0 comp rx=0 tx=0).stop 2024-09-11T23:37:58.905 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.905+0000 7f92e1ffb640 1 -- 172.21.15.26:0/2342244484 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92f4016570 con 0x56134e7b5af0 2024-09-11T23:37:58.906 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.905+0000 7f92faffd640 1 --2- 172.21.15.26:0/3169914100 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f92e40a2e20 0x7f92e40a52e0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x56134e7b2ec0 tx=0x7f92ec0051c0 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:58.906 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.906+0000 7f93047f9740 1 -- 172.21.15.26:0/2342244484 shutdown_connections 2024-09-11T23:37:58.906 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.906+0000 7f93047f9740 1 --2- 172.21.15.26:0/2342244484 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56134e7c0cb0 0x56134e7c30a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:58.906 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.906+0000 7f93047f9740 1 --2- 172.21.15.26:0/2342244484 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56134e7b6410 0x56134e7c0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:58.906 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.906+0000 7f93047f9740 1 --2- 172.21.15.26:0/2342244484 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56134e7b5af0 0x56134e7b5ed0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:58.906 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.906+0000 7f93047f9740 1 -- 172.21.15.26:0/2342244484 >> 172.21.15.26:0/2342244484 conn(0x56134e7b2060 msgr2=0x56134e7b2440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:58.906 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.906+0000 7f93047f9740 1 -- 172.21.15.26:0/2342244484 shutdown_connections 2024-09-11T23:37:58.907 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.907+0000 7f93047f9740 1 -- 172.21.15.26:0/2342244484 wait complete. 2024-09-11T23:37:58.908 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.908+0000 7f93047f9740 1 -- start start 2024-09-11T23:37:58.909 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.908+0000 7f93047f9740 1 Processor -- start 2024-09-11T23:37:58.910 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.909+0000 7f93047f9740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56134e7b5af0 0x56134e7bdaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:58.910 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.909+0000 7f93047f9740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56134e7b6410 0x56134e7bdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:58.911 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.909+0000 7f93047f9740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56134e7c0cb0 0x56134e7be520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:58.911 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.909+0000 7f92fb7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56134e7b5af0 0x56134e7bdaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:58.911 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.909+0000 7f92fb7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56134e7b5af0 0x56134e7bdaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:46720/0 (socket says 172.21.15.26:46720) 2024-09-11T23:37:58.911 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.910+0000 7f92fb7fe640 1 -- 172.21.15.26:0/3084443935 learned_addr learned my addr 172.21.15.26:0/3084443935 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:37:58.911 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.910+0000 7f92fa7fc640 1 --2- 172.21.15.26:0/3084443935 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56134e7b6410 0x56134e7bdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:58.911 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.910+0000 7f93047f9740 1 -- 172.21.15.26:0/3084443935 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x56134e7d4930 con 0x56134e7b6410 2024-09-11T23:37:58.911 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.910+0000 7f93047f9740 1 -- 172.21.15.26:0/3084443935 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x56134e6a3290 con 0x56134e7b5af0 2024-09-11T23:37:58.911 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.910+0000 7f93047f9740 1 -- 172.21.15.26:0/3084443935 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x56134e7d4ab0 con 0x56134e7c0cb0 2024-09-11T23:37:58.911 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.910+0000 7f93047f9740 10 client.0 _ll_register_callbacks cb 0x56134e6d4140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:37:58.911 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.910+0000 7f92fb7fe640 1 --2- 172.21.15.26:0/3084443935 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56134e7c0cb0 0x56134e7be520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:58.913 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.910+0000 7f92fb7fe640 1 -- 172.21.15.26:0/3084443935 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56134e7c0cb0 msgr2=0x56134e7be520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:58.913 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.910+0000 7f92fb7fe640 1 --2- 172.21.15.26:0/3084443935 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56134e7c0cb0 0x56134e7be520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:58.913 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.910+0000 7f92fb7fe640 1 -- 172.21.15.26:0/3084443935 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56134e7b6410 msgr2=0x56134e7bdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:58.913 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.910+0000 7f92fb7fe640 1 --2- 172.21.15.26:0/3084443935 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56134e7b6410 0x56134e7bdfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:58.913 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.910+0000 7f92fb7fe640 1 -- 172.21.15.26:0/3084443935 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56134e42d210 con 0x56134e7b5af0 2024-09-11T23:37:58.913 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.911+0000 7f92fb7fe640 1 --2- 172.21.15.26:0/3084443935 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56134e7b5af0 0x56134e7bdaa0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f92f4004630 tx=0x7f92f40043b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=35402e46bc3c6f72 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:58.913 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.911+0000 7f92daffd640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-11T23:37:58.913 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.911+0000 7f93047f9740 1 -- 172.21.15.26:0/3084443935 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f92f401a070 con 0x56134e7b5af0 2024-09-11T23:37:58.913 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.911+0000 7f92daffd640 1 -- 172.21.15.26:0/3084443935 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92f4032060 con 0x56134e7b5af0 2024-09-11T23:37:58.913 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.911+0000 7f92daffd640 1 -- 172.21.15.26:0/3084443935 <== mon.2 v2:172.21.15.76:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 8586+0+0 (secure 0 0 0) 0x7f92f4016570 con 0x56134e7b5af0 2024-09-11T23:37:58.913 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.911+0000 7f92daffd640 1 -- 172.21.15.26:0/3084443935 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f92f4016770 con 0x56134e7b5af0 2024-09-11T23:37:58.914 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.911+0000 7f93047f9740 10 client.27440 fetch_fsmap learned FSMap version 86 2024-09-11T23:37:58.914 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.911+0000 7f93047f9740 1 -- 172.21.15.26:0/3084443935 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({fsmap=86}) -- 0x56134e4f9620 con 0x56134e7b5af0 2024-09-11T23:37:58.915 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.913+0000 7f92daffd640 1 -- 172.21.15.26:0/3084443935 <== mon.2 v2:172.21.15.76:3300/0 4 ==== fsmap(e 86) ==== 8426+0+0 (secure 0 0 0) 0x7f92f4026070 con 0x56134e7b5af0 2024-09-11T23:37:58.915 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.913+0000 7f93047f9740 10 client.27440 fetch_fsmap finished waiting for FSMap version 86 2024-09-11T23:37:58.915 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.913+0000 7f93047f9740 10 client.27440 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:37:58.916 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.913+0000 7f93047f9740 20 client.27440 populate_metadata read hostname 'smithi026' 2024-09-11T23:37:58.916 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.913+0000 7f93047f9740 1 --2- 172.21.15.26:0/3084443935 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x56134e7e1a60 0x56134e7e3e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:58.916 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.913+0000 7f93047f9740 4 client.27440 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:37:58.916 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.913+0000 7f93047f9740 1 -- 172.21.15.26:0/3084443935 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56134e727a20 con 0x56134e7e1a60 2024-09-11T23:37:58.916 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.913+0000 7f92fa7fc640 1 --2- 172.21.15.26:0/3084443935 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x56134e7e1a60 0x56134e7e3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:58.916 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.914+0000 7f92daffd640 10 client.27440 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:37:58.916 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.914+0000 7f92fa7fc640 1 --2- 172.21.15.26:0/3084443935 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x56134e7e1a60 0x56134e7e3e50 crc :-1 s=READY pgs=292 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8cb0c924a20a2de8 server_cookie=422c1ce5b38b1550 in_seq=0 out_seq=0 2024-09-11T23:37:58.916 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.914+0000 7f92daffd640 1 -- 172.21.15.26:0/3084443935 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56134e727a20 con 0x56134e7e1a60 2024-09-11T23:37:58.916 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.914+0000 7f92daffd640 10 client.27440 handle_command_reply: tid=0 2024-09-11T23:37:58.916 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.915+0000 7f93047f9740 1 client.27440 shutdown 2024-09-11T23:37:58.916 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.915+0000 7f93047f9740 1 -- 172.21.15.26:0/3084443935 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56134e7b5af0 msgr2=0x56134e7bdaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:58.917 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.915+0000 7f93047f9740 1 --2- 172.21.15.26:0/3084443935 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56134e7b5af0 0x56134e7bdaa0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f92f4004630 tx=0x7f92f40043b0 comp rx=0 tx=0).stop 2024-09-11T23:37:58.917 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.915+0000 7f93047f9740 1 -- 172.21.15.26:0/3084443935 shutdown_connections 2024-09-11T23:37:58.917 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.915+0000 7f93047f9740 1 --2- 172.21.15.26:0/3084443935 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x56134e7e1a60 0x56134e7e3e50 crc :-1 s=READY pgs=292 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:58.917 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.915+0000 7f92daffd640 0 client.27440 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:37:58.917 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.915+0000 7f93047f9740 1 --2- 172.21.15.26:0/3084443935 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56134e7c0cb0 0x56134e7be520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:58.917 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.915+0000 7f93047f9740 1 --2- 172.21.15.26:0/3084443935 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56134e7b6410 0x56134e7bdfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:58.917 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.915+0000 7f93047f9740 1 --2- 172.21.15.26:0/3084443935 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56134e7b5af0 0x56134e7bdaa0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:58.917 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.915+0000 7f93047f9740 1 -- 172.21.15.26:0/3084443935 >> 172.21.15.26:0/3084443935 conn(0x56134e7b2060 msgr2=0x56134e7b7d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:58.917 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.915+0000 7f93047f9740 1 -- 172.21.15.26:0/3084443935 shutdown_connections 2024-09-11T23:37:58.917 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.915+0000 7f93047f9740 1 -- 172.21.15.26:0/3084443935 wait complete. 2024-09-11T23:37:58.917 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.915+0000 7f93047f9740 20 client.27440 trim_cache size 0 max 16384 2024-09-11T23:37:58.941 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.939+0000 7f93047f9740 1 Processor -- start 2024-09-11T23:37:58.941 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.939+0000 7f93047f9740 1 -- start start 2024-09-11T23:37:58.941 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.939+0000 7f93047f9740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56134e7b5af0 0x56134e7bace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:58.941 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.939+0000 7f93047f9740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56134e7b6410 0x56134e7bb220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:58.941 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.939+0000 7f93047f9740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56134e7c0cb0 0x56134e7c20a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:58.941 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.939+0000 7f93047f9740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f92fc0744e0 con 0x56134e7c0cb0 2024-09-11T23:37:58.942 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.939+0000 7f93047f9740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f92fc0762f0 con 0x56134e7b6410 2024-09-11T23:37:58.942 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.939+0000 7f93047f9740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f92fc074c40 con 0x56134e7b5af0 2024-09-11T23:37:58.942 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.939+0000 7f92fb7fe640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56134e7b5af0 0x56134e7bace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:58.942 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.939+0000 7f92fb7fe640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56134e7b5af0 0x56134e7bace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:44606/0 (socket says 172.21.15.26:44606) 2024-09-11T23:37:58.942 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.939+0000 7f92fb7fe640 1 -- 172.21.15.26:0/447763170 learned_addr learned my addr 172.21.15.26:0/447763170 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:37:58.942 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.939+0000 7f92fb7fe640 1 --2- 172.21.15.26:0/447763170 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56134e7c0cb0 0x56134e7c20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:58.942 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.940+0000 7f92fa7fc640 1 --2- 172.21.15.26:0/447763170 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56134e7b6410 0x56134e7bb220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:58.942 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.940+0000 7f92fb7fe640 1 -- 172.21.15.26:0/447763170 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56134e7b6410 msgr2=0x56134e7bb220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:58.942 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.940+0000 7f92fb7fe640 1 --2- 172.21.15.26:0/447763170 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56134e7b6410 0x56134e7bb220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:58.942 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.940+0000 7f92fb7fe640 1 -- 172.21.15.26:0/447763170 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56134e7c0cb0 msgr2=0x56134e7c20a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:58.942 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.940+0000 7f92fb7fe640 1 --2- 172.21.15.26:0/447763170 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56134e7c0cb0 0x56134e7c20a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:58.942 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.940+0000 7f92fb7fe640 1 -- 172.21.15.26:0/447763170 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56134e7b1e90 con 0x56134e7b5af0 2024-09-11T23:37:58.943 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.940+0000 7f92fb7fe640 1 --2- 172.21.15.26:0/447763170 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56134e7b5af0 0x56134e7bace0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f92f402bc70 tx=0x7f92f4020040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7acf9f6e3e9b67a9 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:58.943 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.940+0000 7f92e17fa640 1 -- 172.21.15.26:0/447763170 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92f4016570 con 0x56134e7b5af0 2024-09-11T23:37:58.943 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.941+0000 7f92e17fa640 1 -- 172.21.15.26:0/447763170 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f92f401a070 con 0x56134e7b5af0 2024-09-11T23:37:58.943 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.941+0000 7f92e17fa640 1 -- 172.21.15.26:0/447763170 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92f402b8c0 con 0x56134e7b5af0 2024-09-11T23:37:58.945 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.941+0000 7f93047f9740 1 -- 172.21.15.26:0/447763170 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56134e7b5af0 msgr2=0x56134e7bace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:58.946 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.941+0000 7f93047f9740 1 --2- 172.21.15.26:0/447763170 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56134e7b5af0 0x56134e7bace0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f92f402bc70 tx=0x7f92f4020040 comp rx=0 tx=0).stop 2024-09-11T23:37:58.946 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.941+0000 7f93047f9740 1 -- 172.21.15.26:0/447763170 shutdown_connections 2024-09-11T23:37:58.946 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.941+0000 7f93047f9740 1 --2- 172.21.15.26:0/447763170 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56134e7b6410 0x56134e7bb220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:58.946 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.941+0000 7f93047f9740 1 --2- 172.21.15.26:0/447763170 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56134e7c0cb0 0x56134e7c20a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:58.946 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.941+0000 7f93047f9740 1 --2- 172.21.15.26:0/447763170 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56134e7b5af0 0x56134e7bace0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:58.946 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.941+0000 7f93047f9740 1 -- 172.21.15.26:0/447763170 >> 172.21.15.26:0/447763170 conn(0x56134e7b2060 msgr2=0x56134e7bfb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:58.946 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.942+0000 7f93047f9740 1 -- 172.21.15.26:0/447763170 shutdown_connections 2024-09-11T23:37:58.946 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.942+0000 7f93047f9740 1 -- 172.21.15.26:0/447763170 wait complete. 2024-09-11T23:37:58.946 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.942+0000 7f93047f9740 1 -- start start 2024-09-11T23:37:58.946 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.942+0000 7f93047f9740 1 Processor -- start 2024-09-11T23:37:58.946 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.943+0000 7f93047f9740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56134e7b5af0 0x56134e7e7710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:58.947 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.943+0000 7f93047f9740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56134e7b6410 0x56134e7e2a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:58.947 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.943+0000 7f93047f9740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56134e7c0cb0 0x56134e7e3040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:58.947 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.943+0000 7f93047f9740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f92fc075b60 con 0x56134e7b5af0 2024-09-11T23:37:58.947 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.943+0000 7f92fb7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56134e7b5af0 0x56134e7e7710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:58.947 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.943+0000 7f93047f9740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f92fc0753d0 con 0x56134e7b6410 2024-09-11T23:37:58.947 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.943+0000 7f93047f9740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f92fc101f30 con 0x56134e7c0cb0 2024-09-11T23:37:58.947 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.943+0000 7f92fb7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56134e7b5af0 0x56134e7e7710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43174/0 (socket says 172.21.15.26:43174) 2024-09-11T23:37:58.947 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.943+0000 7f92fb7fe640 1 -- 172.21.15.26:0/3820086120 learned_addr learned my addr 172.21.15.26:0/3820086120 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:37:58.947 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.943+0000 7f93047f9740 10 client.0 _ll_register_callbacks cb 0x7f92f4009ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:37:58.947 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.943+0000 7f92fa7fc640 1 --2- 172.21.15.26:0/3820086120 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56134e7b6410 0x56134e7e2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:58.947 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.943+0000 7f92fb7fe640 1 --2- 172.21.15.26:0/3820086120 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56134e7c0cb0 0x56134e7e3040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:58.947 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.943+0000 7f92fb7fe640 1 -- 172.21.15.26:0/3820086120 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56134e7c0cb0 msgr2=0x56134e7e3040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:58.947 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.943+0000 7f92fb7fe640 1 --2- 172.21.15.26:0/3820086120 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56134e7c0cb0 0x56134e7e3040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:58.948 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.943+0000 7f92fb7fe640 1 -- 172.21.15.26:0/3820086120 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56134e7b6410 msgr2=0x56134e7e2a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:58.948 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.943+0000 7f92fb7fe640 1 --2- 172.21.15.26:0/3820086120 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56134e7b6410 0x56134e7e2a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:58.948 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.943+0000 7f92fb7fe640 1 -- 172.21.15.26:0/3820086120 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56134e7e3870 con 0x56134e7b5af0 2024-09-11T23:37:58.948 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.944+0000 7f92fb7fe640 1 --2- 172.21.15.26:0/3820086120 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56134e7b5af0 0x56134e7e7710 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f92f4005d70 tx=0x7f92f402b420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7384c8ca572368c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:58.948 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.944+0000 7f92daffd640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:37:58.948 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.944+0000 7f92daffd640 1 -- 172.21.15.26:0/3820086120 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92f4032050 con 0x56134e7b5af0 2024-09-11T23:37:58.948 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.944+0000 7f92daffd640 1 -- 172.21.15.26:0/3820086120 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 8586+0+0 (secure 0 0 0) 0x7f92f402b560 con 0x56134e7b5af0 2024-09-11T23:37:58.948 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.944+0000 7f93047f9740 1 -- 172.21.15.26:0/3820086120 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f92f401a070 con 0x56134e7b5af0 2024-09-11T23:37:58.948 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.944+0000 7f92daffd640 1 -- 172.21.15.26:0/3820086120 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f92f4051050 con 0x56134e7b5af0 2024-09-11T23:37:58.948 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.944+0000 7f93047f9740 10 client.17934 fetch_fsmap learned FSMap version 86 2024-09-11T23:37:58.948 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.944+0000 7f93047f9740 1 -- 172.21.15.26:0/3820086120 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=86,monmap=4+}) -- 0x56134e84c4d0 con 0x56134e7b5af0 2024-09-11T23:37:58.948 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.945+0000 7f92daffd640 1 -- 172.21.15.26:0/3820086120 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f92f401a070 con 0x56134e7b5af0 2024-09-11T23:37:58.949 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.946+0000 7f92daffd640 1 -- 172.21.15.26:0/3820086120 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 86) ==== 8426+0+0 (secure 0 0 0) 0x7f92f4026020 con 0x56134e7b5af0 2024-09-11T23:37:58.949 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.946+0000 7f93047f9740 10 client.17934 fetch_fsmap finished waiting for FSMap version 86 2024-09-11T23:37:58.949 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.946+0000 7f93047f9740 10 client.17934 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:37:58.950 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.946+0000 7f93047f9740 20 client.17934 populate_metadata read hostname 'smithi026' 2024-09-11T23:37:58.950 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.946+0000 7f93047f9740 1 --2- 172.21.15.26:0/3820086120 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x56134e84cbf0 0x56134e84f010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:58.950 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.946+0000 7f93047f9740 4 client.17934 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T23:37:58.950 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.946+0000 7f93047f9740 1 -- 172.21.15.26:0/3820086120 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56134e721d70 con 0x56134e84cbf0 2024-09-11T23:37:58.950 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.946+0000 7f92fa7fc640 1 --2- 172.21.15.26:0/3820086120 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x56134e84cbf0 0x56134e84f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:58.950 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.946+0000 7f92fa7fc640 1 --2- 172.21.15.26:0/3820086120 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x56134e84cbf0 0x56134e84f010 crc :-1 s=READY pgs=293 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67cbb2da3d68f3ba server_cookie=cca0cd24ef789e15 in_seq=0 out_seq=0 2024-09-11T23:37:58.950 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.946+0000 7f92daffd640 10 client.17934 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:37:58.963 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.963+0000 7f92daffd640 1 -- 172.21.15.26:0/3820086120 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56134e721d70 con 0x56134e84cbf0 2024-09-11T23:37:58.964 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.963+0000 7f92daffd640 10 client.17934 handle_command_reply: tid=0 2024-09-11T23:37:58.964 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.963+0000 7f93047f9740 1 client.17934 shutdown 2024-09-11T23:37:58.966 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.964+0000 7f93047f9740 1 -- 172.21.15.26:0/3820086120 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56134e7b5af0 msgr2=0x56134e7e7710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:58.966 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.964+0000 7f93047f9740 1 --2- 172.21.15.26:0/3820086120 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56134e7b5af0 0x56134e7e7710 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f92f4005d70 tx=0x7f92f402b420 comp rx=0 tx=0).stop 2024-09-11T23:37:58.966 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.964+0000 7f93047f9740 1 -- 172.21.15.26:0/3820086120 shutdown_connections 2024-09-11T23:37:58.967 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.964+0000 7f93047f9740 1 --2- 172.21.15.26:0/3820086120 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56134e7b6410 0x56134e7e2a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:58.967 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.964+0000 7f93047f9740 1 --2- 172.21.15.26:0/3820086120 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x56134e84cbf0 0x56134e84f010 crc :-1 s=READY pgs=293 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:58.967 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.964+0000 7f93047f9740 1 --2- 172.21.15.26:0/3820086120 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56134e7c0cb0 0x56134e7e3040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:58.967 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.964+0000 7f93047f9740 1 --2- 172.21.15.26:0/3820086120 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56134e7b5af0 0x56134e7e7710 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:58.967 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.964+0000 7f93047f9740 1 -- 172.21.15.26:0/3820086120 >> 172.21.15.26:0/3820086120 conn(0x56134e7b2060 msgr2=0x56134e7b09a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:58.967 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.964+0000 7f92daffd640 0 client.17934 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:37:58.968 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.964+0000 7f93047f9740 1 -- 172.21.15.26:0/3820086120 shutdown_connections 2024-09-11T23:37:58.968 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.964+0000 7f93047f9740 1 -- 172.21.15.26:0/3820086120 wait complete. 2024-09-11T23:37:58.968 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.964+0000 7f93047f9740 20 client.17934 trim_cache size 0 max 16384 2024-09-11T23:37:58.969 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-09-11T23:37:58.969 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "message": "", 2024-09-11T23:37:58.969 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-09-11T23:37:58.969 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-09-11T23:37:58.970 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.968+0000 7f9301a5e640 1 -- 172.21.15.26:0/3169914100 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f92e40a2e20 msgr2=0x7f92e40a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:58.970 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.968+0000 7f9301a5e640 1 --2- 172.21.15.26:0/3169914100 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f92e40a2e20 0x7f92e40a52e0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x56134e7b2ec0 tx=0x7f92ec0051c0 comp rx=0 tx=0).stop 2024-09-11T23:37:58.970 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.968+0000 7f9301a5e640 1 -- 172.21.15.26:0/3169914100 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f92fc101ad0 msgr2=0x7f92fc198330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:58.970 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.968+0000 7f9301a5e640 1 --2- 172.21.15.26:0/3169914100 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f92fc101ad0 0x7f92fc198330 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f92ec00d950 tx=0x7f92ec00de10 comp rx=0 tx=0).stop 2024-09-11T23:37:58.970 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.968+0000 7f9301a5e640 1 -- 172.21.15.26:0/3169914100 shutdown_connections 2024-09-11T23:37:58.971 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.968+0000 7f9301a5e640 1 --2- 172.21.15.26:0/3169914100 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f92e40a2e20 0x7f92e40a52e0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:58.971 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.968+0000 7f9301a5e640 1 --2- 172.21.15.26:0/3169914100 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f92fc107b70 0x7f92fc19cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:58.971 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.968+0000 7f9301a5e640 1 --2- 172.21.15.26:0/3169914100 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f92fc102470 0x7f92fc198870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:58.971 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.968+0000 7f9301a5e640 1 --2- 172.21.15.26:0/3169914100 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f92fc101ad0 0x7f92fc198330 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:58.971 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.968+0000 7f9301a5e640 1 -- 172.21.15.26:0/3169914100 >> 172.21.15.26:0/3169914100 conn(0x7f92fc0fd7d0 msgr2=0x7f92fc0fde10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:58.971 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.968+0000 7f9301a5e640 1 -- 172.21.15.26:0/3169914100 shutdown_connections 2024-09-11T23:37:58.971 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:37:58.968+0000 7f9301a5e640 1 -- 172.21.15.26:0/3169914100 wait complete. 2024-09-11T23:38:00.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:38:00 smithi136 ceph-mon[35121]: pgmap v616: 97 pgs: 4 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 82 active+clean; 2.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 205 op/s 2024-09-11T23:38:00.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:38:00 smithi026 ceph-mon[31172]: pgmap v616: 97 pgs: 4 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 82 active+clean; 2.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 205 op/s 2024-09-11T23:38:00.552 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:38:00 smithi076 ceph-mon[35296]: pgmap v616: 97 pgs: 4 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 82 active+clean; 2.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 205 op/s 2024-09-11T23:38:02.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:38:02 smithi136 ceph-mon[35121]: pgmap v617: 97 pgs: 97 active+clean; 2.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.4 MiB/s wr, 262 op/s 2024-09-11T23:38:02.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:38:02 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:38:02.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:38:02 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:38:02.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:38:02 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:38:02.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:38:02 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:38:02.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:38:02 smithi026 ceph-mon[31172]: pgmap v617: 97 pgs: 97 active+clean; 2.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.4 MiB/s wr, 262 op/s 2024-09-11T23:38:02.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:38:02 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:38:02.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:38:02 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:38:02.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:38:02 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:38:02.501 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:38:02 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:38:02.552 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:38:02 smithi076 ceph-mon[35296]: pgmap v617: 97 pgs: 97 active+clean; 2.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.4 MiB/s wr, 262 op/s 2024-09-11T23:38:02.552 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:38:02 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:38:02.552 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:38:02 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:38:02.553 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:38:02 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:38:02.553 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:38:02 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:38:03.056 INFO:tasks.quiescer.fs.[cephfs]:Total duration (43.6) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-11T23:38:03.056 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 21.8 2024-09-11T23:38:03.057 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:38:03.320 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:38:03.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:03.843+0000 7f8b8d6a2640 1 -- 172.21.15.26:0/998367348 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b88101420 msgr2=0x7f8b8810d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:03.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:03.843+0000 7f8b8d6a2640 1 --2- 172.21.15.26:0/998367348 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b88101420 0x7f8b8810d950 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f8b80009cd0 tx=0x7f8b8002a8e0 comp rx=0 tx=0).stop 2024-09-11T23:38:03.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:03.843+0000 7f8b8d6a2640 1 -- 172.21.15.26:0/998367348 shutdown_connections 2024-09-11T23:38:03.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:03.843+0000 7f8b8d6a2640 1 --2- 172.21.15.26:0/998367348 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8b88100a80 0x7f8b88100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:03.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:03.843+0000 7f8b8d6a2640 1 --2- 172.21.15.26:0/998367348 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b88101420 0x7f8b8810d950 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:03.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:03.843+0000 7f8b8d6a2640 1 --2- 172.21.15.26:0/998367348 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8b88106a80 0x7f8b88106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:03.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:03.844+0000 7f8b8d6a2640 1 -- 172.21.15.26:0/998367348 >> 172.21.15.26:0/998367348 conn(0x7f8b880fc7a0 msgr2=0x7f8b880febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:38:03.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:03.844+0000 7f8b8d6a2640 1 -- 172.21.15.26:0/998367348 shutdown_connections 2024-09-11T23:38:03.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:03.844+0000 7f8b8d6a2640 1 -- 172.21.15.26:0/998367348 wait complete. 2024-09-11T23:38:03.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:03.844+0000 7f8b8d6a2640 1 Processor -- start 2024-09-11T23:38:03.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:03.844+0000 7f8b8d6a2640 1 -- start start 2024-09-11T23:38:03.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:03.845+0000 7f8b8d6a2640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8b88100a80 0x7f8b880fce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:03.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:03.845+0000 7f8b8d6a2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b88101420 0x7f8b880fd340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:03.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:03.845+0000 7f8b8d6a2640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8b88106a80 0x7f8b880fd880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:03.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:03.845+0000 7f8b8d6a2640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f8b88110e80 con 0x7f8b88101420 2024-09-11T23:38:03.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:03.845+0000 7f8b8d6a2640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f8b88110d00 con 0x7f8b88106a80 2024-09-11T23:38:03.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:03.845+0000 7f8b8d6a2640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f8b88111000 con 0x7f8b88100a80 2024-09-11T23:38:03.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:03.845+0000 7f8b86ffd640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8b88100a80 0x7f8b880fce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:03.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:03.845+0000 7f8b867fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b88101420 0x7f8b880fd340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:03.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:03.845+0000 7f8b86ffd640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8b88100a80 0x7f8b880fce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:49430/0 (socket says 172.21.15.26:49430) 2024-09-11T23:38:03.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:03.845+0000 7f8b86ffd640 1 -- 172.21.15.26:0/1882300030 learned_addr learned my addr 172.21.15.26:0/1882300030 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:38:03.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:03.845+0000 7f8b877fe640 1 --2- 172.21.15.26:0/1882300030 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8b88106a80 0x7f8b880fd880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:03.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:03.845+0000 7f8b86ffd640 1 -- 172.21.15.26:0/1882300030 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8b88106a80 msgr2=0x7f8b880fd880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:03.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:03.845+0000 7f8b86ffd640 1 --2- 172.21.15.26:0/1882300030 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8b88106a80 0x7f8b880fd880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:03.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:03.845+0000 7f8b86ffd640 1 -- 172.21.15.26:0/1882300030 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b88101420 msgr2=0x7f8b880fd340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:03.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:03.845+0000 7f8b86ffd640 1 --2- 172.21.15.26:0/1882300030 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b88101420 0x7f8b880fd340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:03.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:03.845+0000 7f8b86ffd640 1 -- 172.21.15.26:0/1882300030 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b881a68f0 con 0x7f8b88100a80 2024-09-11T23:38:03.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:03.846+0000 7f8b86ffd640 1 --2- 172.21.15.26:0/1882300030 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8b88100a80 0x7f8b880fce00 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f8b7800e9f0 tx=0x7f8b7800edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:38:03.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:03.846+0000 7f8b6ffff640 1 -- 172.21.15.26:0/1882300030 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b7800cd80 con 0x7f8b88100a80 2024-09-11T23:38:03.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:03.846+0000 7f8b8d6a2640 1 -- 172.21.15.26:0/1882300030 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b881a6be0 con 0x7f8b88100a80 2024-09-11T23:38:03.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:03.846+0000 7f8b6ffff640 1 -- 172.21.15.26:0/1882300030 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b78005500 con 0x7f8b88100a80 2024-09-11T23:38:03.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:03.846+0000 7f8b6ffff640 1 -- 172.21.15.26:0/1882300030 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b78010690 con 0x7f8b88100a80 2024-09-11T23:38:03.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:03.847+0000 7f8b8d6a2640 1 -- 172.21.15.26:0/1882300030 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b881a70d0 con 0x7f8b88100a80 2024-09-11T23:38:03.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:03.848+0000 7f8b8d6a2640 1 -- 172.21.15.26:0/1882300030 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b88100ee0 con 0x7f8b88100a80 2024-09-11T23:38:03.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:03.848+0000 7f8b6ffff640 1 -- 172.21.15.26:0/1882300030 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f8b780046c0 con 0x7f8b88100a80 2024-09-11T23:38:03.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:03.849+0000 7f8b6ffff640 1 --2- 172.21.15.26:0/1882300030 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f8b700a2cf0 0x7f8b700a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:03.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:03.849+0000 7f8b867fc640 1 --2- 172.21.15.26:0/1882300030 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f8b700a2cf0 0x7f8b700a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:03.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:03.849+0000 7f8b6ffff640 1 -- 172.21.15.26:0/1882300030 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 8586+0+0 (secure 0 0 0) 0x7f8b780cf8f0 con 0x7f8b88100a80 2024-09-11T23:38:03.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:03.849+0000 7f8b867fc640 1 --2- 172.21.15.26:0/1882300030 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f8b700a2cf0 0x7f8b700a51b0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f8b880fe890 tx=0x7f8b7c00b410 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:38:03.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:03.853+0000 7f8b6ffff640 1 -- 172.21.15.26:0/1882300030 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8b7809c4d0 con 0x7f8b88100a80 2024-09-11T23:38:04.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:04.157+0000 7f8b8d6a2640 1 -- 172.21.15.26:0/1882300030 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8b8810a030 con 0x7f8b88100a80 2024-09-11T23:38:04.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:04.158+0000 7f8b6ffff640 1 -- 172.21.15.26:0/1882300030 <== mon.1 v2:172.21.15.136:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 86 v86) ==== 94+0+11999 (secure 0 0 0) 0x7f8b7809bdd0 con 0x7f8b88100a80 2024-09-11T23:38:04.158 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-11T23:38:04.158 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":86,"btime":"2024-09-11T23:37:13:064993+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14499,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/367659515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":367659515},{"type":"v1","addr":"172.21.15.26:6835","nonce":367659515}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14511,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/2041065299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":2041065299},{"type":"v1","addr":"172.21.15.26:6837","nonce":2041065299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":86,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:16:26.544203+0000","modified":"2024-09-11T23:37:12.918780+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24485,"mds_1":24463,"mds_2":24458,"mds_3":24448,"mds_4":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14550":{"gid":14550,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6839/2233068305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":2233068305},{"type":"v1","addr":"172.21.15.26:6839","nonce":2233068305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14562":{"gid":14562,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6841/2984982336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":2984982336},{"type":"v1","addr":"172.21.15.26:6841","nonce":2984982336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6833/2019841455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":2019841455},{"type":"v1","addr":"172.21.15.136:6833","nonce":2019841455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.76:6833/3438147050","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3438147050},{"type":"v1","addr":"172.21.15.76:6833","nonce":3438147050}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6835/3617665975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3617665975},{"type":"v1","addr":"172.21.15.136:6835","nonce":3617665975}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":12,"addr":"172.21.15.76:6835/393308499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":393308499},{"type":"v1","addr":"172.21.15.76:6835","nonce":393308499}]},"join_fscid":-1,"export_targets":[0,1,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":10,"addr":"172.21.15.136:6837/1633082387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":1633082387},{"type":"v1","addr":"172.21.15.136:6837","nonce":1633082387}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.76:6837/2078978748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":2078978748},{"type":"v1","addr":"172.21.15.76:6837","nonce":2078978748}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":63,"addr":"172.21.15.136:6839/3211398630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":3211398630},{"type":"v1","addr":"172.21.15.136:6839","nonce":3211398630}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.76:6839/2077637090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6838","nonce":2077637090},{"type":"v1","addr":"172.21.15.76:6839","nonce":2077637090}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24485,"qdb_cluster":[24485,24463,24458,24448,24443]},"id":1}]} 2024-09-11T23:38:04.159 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 86 2024-09-11T23:38:04.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:04.161+0000 7f8b8d6a2640 1 -- 172.21.15.26:0/1882300030 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f8b700a2cf0 msgr2=0x7f8b700a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:04.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:04.162+0000 7f8b8d6a2640 1 --2- 172.21.15.26:0/1882300030 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f8b700a2cf0 0x7f8b700a51b0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f8b880fe890 tx=0x7f8b7c00b410 comp rx=0 tx=0).stop 2024-09-11T23:38:04.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:04.162+0000 7f8b8d6a2640 1 -- 172.21.15.26:0/1882300030 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8b88100a80 msgr2=0x7f8b880fce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:04.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:04.162+0000 7f8b8d6a2640 1 --2- 172.21.15.26:0/1882300030 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8b88100a80 0x7f8b880fce00 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f8b7800e9f0 tx=0x7f8b7800edf0 comp rx=0 tx=0).stop 2024-09-11T23:38:04.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:04.162+0000 7f8b8d6a2640 1 -- 172.21.15.26:0/1882300030 shutdown_connections 2024-09-11T23:38:04.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:04.162+0000 7f8b8d6a2640 1 --2- 172.21.15.26:0/1882300030 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f8b700a2cf0 0x7f8b700a51b0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:04.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:04.162+0000 7f8b8d6a2640 1 --2- 172.21.15.26:0/1882300030 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8b88106a80 0x7f8b880fd880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:04.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:04.162+0000 7f8b8d6a2640 1 --2- 172.21.15.26:0/1882300030 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b88101420 0x7f8b880fd340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:04.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:04.162+0000 7f8b8d6a2640 1 --2- 172.21.15.26:0/1882300030 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8b88100a80 0x7f8b880fce00 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:04.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:04.162+0000 7f8b8d6a2640 1 -- 172.21.15.26:0/1882300030 >> 172.21.15.26:0/1882300030 conn(0x7f8b880fc7a0 msgr2=0x7f8b8806aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:38:04.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:04.162+0000 7f8b8d6a2640 1 -- 172.21.15.26:0/1882300030 shutdown_connections 2024-09-11T23:38:04.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:04.162+0000 7f8b8d6a2640 1 -- 172.21.15.26:0/1882300030 wait complete. 2024-09-11T23:38:04.347 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:38:04 smithi026 ceph-mon[31172]: pgmap v618: 97 pgs: 97 active+clean; 2.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.4 MiB/s wr, 247 op/s 2024-09-11T23:38:04.347 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:38:04 smithi026 ceph-mon[31172]: from='client.? 172.21.15.26:0/1882300030' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:38:04.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:38:04 smithi136 ceph-mon[35121]: pgmap v618: 97 pgs: 97 active+clean; 2.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.4 MiB/s wr, 247 op/s 2024-09-11T23:38:04.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:38:04 smithi136 ceph-mon[35121]: from='client.? 172.21.15.26:0/1882300030' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:38:04.552 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:38:04 smithi076 ceph-mon[35296]: pgmap v618: 97 pgs: 97 active+clean; 2.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.4 MiB/s wr, 247 op/s 2024-09-11T23:38:04.552 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:38:04 smithi076 ceph-mon[35296]: from='client.? 172.21.15.26:0/1882300030' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:38:04.923 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24485 quiesce db / --timeout 90 --expiration 141.8 --await-for 45.0' 2024-09-11T23:38:04.924 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24485 quiesce db / --timeout 90 --expiration 141.8 --await-for 45.0 2024-09-11T23:38:05.216 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:38:05 smithi026 ceph-mon[31172]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:38:05.216 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:38:05 smithi026 ceph-mon[31172]: pgmap v619: 97 pgs: 97 active+clean; 2.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.4 MiB/s wr, 229 op/s 2024-09-11T23:38:05.266 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:38:05.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:38:05 smithi136 ceph-mon[35121]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:38:05.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:38:05 smithi136 ceph-mon[35121]: pgmap v619: 97 pgs: 97 active+clean; 2.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.4 MiB/s wr, 229 op/s 2024-09-11T23:38:05.552 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:38:05 smithi076 ceph-mon[35296]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:38:05.552 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:38:05 smithi076 ceph-mon[35296]: pgmap v619: 97 pgs: 97 active+clean; 2.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.4 MiB/s wr, 229 op/s 2024-09-11T23:38:05.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.917+0000 7f1bd233a640 1 -- 172.21.15.26:0/2506363059 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1bcc102570 msgr2=0x7f1bcc1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:05.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.917+0000 7f1bd233a640 1 --2- 172.21.15.26:0/2506363059 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1bcc102570 0x7f1bcc1029b0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f1bb4009900 tx=0x7f1bb402a480 comp rx=0 tx=0).stop 2024-09-11T23:38:05.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.917+0000 7f1bd233a640 1 -- 172.21.15.26:0/2506363059 shutdown_connections 2024-09-11T23:38:05.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.917+0000 7f1bd233a640 1 --2- 172.21.15.26:0/2506363059 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1bcc102ef0 0x7f1bcc10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:05.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.917+0000 7f1bd233a640 1 --2- 172.21.15.26:0/2506363059 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1bcc102570 0x7f1bcc1029b0 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:05.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.917+0000 7f1bd233a640 1 --2- 172.21.15.26:0/2506363059 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1bcc100e20 0x7f1bcc101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:05.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.917+0000 7f1bd233a640 1 -- 172.21.15.26:0/2506363059 >> 172.21.15.26:0/2506363059 conn(0x7f1bcc0fc7d0 msgr2=0x7f1bcc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:38:05.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.917+0000 7f1bd233a640 1 -- 172.21.15.26:0/2506363059 shutdown_connections 2024-09-11T23:38:05.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.917+0000 7f1bd233a640 1 -- 172.21.15.26:0/2506363059 wait complete. 2024-09-11T23:38:05.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.918+0000 7f1bd233a640 1 Processor -- start 2024-09-11T23:38:05.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.918+0000 7f1bd233a640 1 -- start start 2024-09-11T23:38:05.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.918+0000 7f1bd233a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1bcc100e20 0x7f1bcc1a03d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:05.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.918+0000 7f1bd233a640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1bcc102570 0x7f1bcc1a0910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:05.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.918+0000 7f1bd233a640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1bcc102ef0 0x7f1bcc19a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:05.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.918+0000 7f1bd233a640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f1bcc110f80 con 0x7f1bcc100e20 2024-09-11T23:38:05.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.918+0000 7f1bd233a640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f1bcc110e00 con 0x7f1bcc102ef0 2024-09-11T23:38:05.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.918+0000 7f1bd233a640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f1bcc111100 con 0x7f1bcc102570 2024-09-11T23:38:05.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.918+0000 7f1bcbfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1bcc100e20 0x7f1bcc1a03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:05.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.918+0000 7f1bc3fff640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1bcc102570 0x7f1bcc1a0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:05.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.918+0000 7f1bcbfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1bcc100e20 0x7f1bcc1a03d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37318/0 (socket says 172.21.15.26:37318) 2024-09-11T23:38:05.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.918+0000 7f1bcbfff640 1 -- 172.21.15.26:0/905734588 learned_addr learned my addr 172.21.15.26:0/905734588 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:38:05.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.918+0000 7f1bd08d7640 1 --2- 172.21.15.26:0/905734588 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1bcc102ef0 0x7f1bcc19a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:05.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.918+0000 7f1bcbfff640 1 -- 172.21.15.26:0/905734588 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1bcc102570 msgr2=0x7f1bcc1a0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:05.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.918+0000 7f1bcbfff640 1 --2- 172.21.15.26:0/905734588 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1bcc102570 0x7f1bcc1a0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:05.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.919+0000 7f1bcbfff640 1 -- 172.21.15.26:0/905734588 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1bcc102ef0 msgr2=0x7f1bcc19a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:05.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.919+0000 7f1bcbfff640 1 --2- 172.21.15.26:0/905734588 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1bcc102ef0 0x7f1bcc19a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:05.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.919+0000 7f1bcbfff640 1 -- 172.21.15.26:0/905734588 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bcc19acc0 con 0x7f1bcc100e20 2024-09-11T23:38:05.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.919+0000 7f1bcbfff640 1 --2- 172.21.15.26:0/905734588 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1bcc100e20 0x7f1bcc1a03d0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f1bbc00e9f0 tx=0x7f1bbc00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:38:05.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.919+0000 7f1bc9ffb640 1 -- 172.21.15.26:0/905734588 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bbc00cd80 con 0x7f1bcc100e20 2024-09-11T23:38:05.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.919+0000 7f1bc9ffb640 1 -- 172.21.15.26:0/905734588 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1bbc005500 con 0x7f1bcc100e20 2024-09-11T23:38:05.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.919+0000 7f1bd233a640 1 -- 172.21.15.26:0/905734588 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1bcc19afb0 con 0x7f1bcc100e20 2024-09-11T23:38:05.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.919+0000 7f1bc9ffb640 1 -- 172.21.15.26:0/905734588 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bbc010690 con 0x7f1bcc100e20 2024-09-11T23:38:05.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.919+0000 7f1bd233a640 1 -- 172.21.15.26:0/905734588 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1bcc1a72a0 con 0x7f1bcc100e20 2024-09-11T23:38:05.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.921+0000 7f1bc9ffb640 1 -- 172.21.15.26:0/905734588 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f1bbc0046c0 con 0x7f1bcc100e20 2024-09-11T23:38:05.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.921+0000 7f1bd4713740 1 Processor -- start 2024-09-11T23:38:05.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.921+0000 7f1bd4713740 1 -- start start 2024-09-11T23:38:05.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.922+0000 7f1bd4713740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5645148d0760 0x5645148d0b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:05.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.922+0000 7f1bd4713740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5645148d1080 0x5645148db400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:05.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.922+0000 7f1bd4713740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5645148db940 0x5645148ddd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:05.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.922+0000 7f1bd4713740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5645148362a0 con 0x5645148d0760 2024-09-11T23:38:05.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.922+0000 7f1bd4713740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5645147ddb10 con 0x5645148db940 2024-09-11T23:38:05.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.922+0000 7f1bd4713740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x564514832260 con 0x5645148d1080 2024-09-11T23:38:05.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.922+0000 7f1bd08d7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5645148d0760 0x5645148d0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:05.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.922+0000 7f1bcbfff640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5645148d1080 0x5645148db400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:05.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.922+0000 7f1bd08d7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5645148d0760 0x5645148d0b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37332/0 (socket says 172.21.15.26:37332) 2024-09-11T23:38:05.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.922+0000 7f1bd08d7640 1 -- 172.21.15.26:0/1519565333 learned_addr learned my addr 172.21.15.26:0/1519565333 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:38:05.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.922+0000 7f1bc3fff640 1 --2- 172.21.15.26:0/1519565333 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5645148db940 0x5645148ddd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:05.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.922+0000 7f1bd08d7640 1 -- 172.21.15.26:0/1519565333 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5645148d1080 msgr2=0x5645148db400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:38:05.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.922+0000 7f1bd08d7640 1 --2- 172.21.15.26:0/1519565333 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5645148d1080 0x5645148db400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:05.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.922+0000 7f1bd08d7640 1 -- 172.21.15.26:0/1519565333 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5645148db940 msgr2=0x5645148ddd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:38:05.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.922+0000 7f1bd08d7640 1 --2- 172.21.15.26:0/1519565333 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5645148db940 0x5645148ddd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:05.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.922+0000 7f1bd08d7640 1 -- 172.21.15.26:0/1519565333 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5645145a7b90 con 0x5645148d0760 2024-09-11T23:38:05.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.922+0000 7f1bcbfff640 1 --2- 172.21.15.26:0/1519565333 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5645148d1080 0x5645148db400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:38:05.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.922+0000 7f1bc9ffb640 1 --2- 172.21.15.26:0/905734588 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f1bb80a2d70 0x7f1bb80a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:05.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.922+0000 7f1bcbfff640 1 --2- 172.21.15.26:0/905734588 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f1bb80a2d70 0x7f1bb80a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:05.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.922+0000 7f1bd08d7640 1 --2- 172.21.15.26:0/1519565333 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5645148d0760 0x5645148d0b40 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f1bc4009f50 tx=0x7f1bc40118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0a5c54443e08a39 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:38:05.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.923+0000 7f1bc9ffb640 1 -- 172.21.15.26:0/905734588 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 8542+0+0 (secure 0 0 0) 0x7f1bbc0cf7e0 con 0x7f1bcc100e20 2024-09-11T23:38:05.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.923+0000 7f1bc27fc640 1 -- 172.21.15.26:0/1519565333 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bc4004b20 con 0x5645148d0760 2024-09-11T23:38:05.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.923+0000 7f1bc27fc640 1 -- 172.21.15.26:0/1519565333 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1bc4004cc0 con 0x5645148d0760 2024-09-11T23:38:05.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.923+0000 7f1bc27fc640 1 -- 172.21.15.26:0/1519565333 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bc4005020 con 0x5645148d0760 2024-09-11T23:38:05.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.923+0000 7f1bd4713740 1 -- 172.21.15.26:0/1519565333 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5645148d0760 msgr2=0x5645148d0b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:05.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.923+0000 7f1bcbfff640 1 --2- 172.21.15.26:0/905734588 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f1bb80a2d70 0x7f1bb80a5230 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f1bbc0ccfd0 tx=0x7f1bbc0d5550 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:38:05.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.923+0000 7f1bd4713740 1 --2- 172.21.15.26:0/1519565333 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5645148d0760 0x5645148d0b40 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f1bc4009f50 tx=0x7f1bc40118d0 comp rx=0 tx=0).stop 2024-09-11T23:38:05.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.923+0000 7f1bd4713740 1 -- 172.21.15.26:0/1519565333 shutdown_connections 2024-09-11T23:38:05.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.923+0000 7f1bd4713740 1 --2- 172.21.15.26:0/1519565333 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5645148db940 0x5645148ddd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:05.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.923+0000 7f1bd4713740 1 --2- 172.21.15.26:0/1519565333 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5645148d1080 0x5645148db400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:05.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.923+0000 7f1bd4713740 1 --2- 172.21.15.26:0/1519565333 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5645148d0760 0x5645148d0b40 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:05.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.923+0000 7f1bd4713740 1 -- 172.21.15.26:0/1519565333 >> 172.21.15.26:0/1519565333 conn(0x5645148cccf0 msgr2=0x5645148cd0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:38:05.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.924+0000 7f1bd4713740 1 -- 172.21.15.26:0/1519565333 shutdown_connections 2024-09-11T23:38:05.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.924+0000 7f1bd4713740 1 -- 172.21.15.26:0/1519565333 wait complete. 2024-09-11T23:38:05.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.924+0000 7f1bd4713740 1 -- start start 2024-09-11T23:38:05.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.924+0000 7f1bd4713740 1 Processor -- start 2024-09-11T23:38:05.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.924+0000 7f1bd4713740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5645148d0760 0x5645148d87b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:05.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.924+0000 7f1bd4713740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5645148d1080 0x5645148d8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:05.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.924+0000 7f1bd4713740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5645148db940 0x5645148d9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:05.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.924+0000 7f1bd4713740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5645148de510 con 0x5645148db940 2024-09-11T23:38:05.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.924+0000 7f1bd4713740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5645147be6e0 con 0x5645148d0760 2024-09-11T23:38:05.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.924+0000 7f1bd4713740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5645148de690 con 0x5645148d1080 2024-09-11T23:38:05.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.924+0000 7f1bd4713740 10 client.0 _ll_register_callbacks cb 0x5645147e2a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:38:05.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.925+0000 7f1bc3fff640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5645148d1080 0x5645148d8cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:05.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.925+0000 7f1bd08d7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5645148db940 0x5645148d9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:05.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.925+0000 7f1bd08d7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5645148db940 0x5645148d9230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37348/0 (socket says 172.21.15.26:37348) 2024-09-11T23:38:05.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.925+0000 7f1bd08d7640 1 -- 172.21.15.26:0/2753638050 learned_addr learned my addr 172.21.15.26:0/2753638050 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:38:05.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.925+0000 7f1bc3fff640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5645148d1080 0x5645148d8cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:49470/0 (socket says 172.21.15.26:49470) 2024-09-11T23:38:05.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.925+0000 7f1bd08d7640 1 --2- 172.21.15.26:0/2753638050 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5645148d0760 0x5645148d87b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:05.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.925+0000 7f1bd08d7640 1 -- 172.21.15.26:0/2753638050 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5645148d1080 msgr2=0x5645148d8cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:38:05.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.925+0000 7f1bd08d7640 1 --2- 172.21.15.26:0/2753638050 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5645148d1080 0x5645148d8cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:05.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.925+0000 7f1bd08d7640 1 -- 172.21.15.26:0/2753638050 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5645148d0760 msgr2=0x5645148d87b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:38:05.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.925+0000 7f1bd08d7640 1 --2- 172.21.15.26:0/2753638050 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5645148d0760 0x5645148d87b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:05.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.925+0000 7f1bd08d7640 1 -- 172.21.15.26:0/2753638050 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5645148e7980 con 0x5645148db940 2024-09-11T23:38:05.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.925+0000 7f1bd08d7640 1 --2- 172.21.15.26:0/2753638050 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5645148db940 0x5645148d9230 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f1bc40044c0 tx=0x7f1bc40045c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=466c0350a0ec759a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:38:05.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.925+0000 7f1bab7fe640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:38:05.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.925+0000 7f1bab7fe640 1 -- 172.21.15.26:0/2753638050 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bc4003cc0 con 0x5645148db940 2024-09-11T23:38:05.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.925+0000 7f1bab7fe640 1 -- 172.21.15.26:0/2753638050 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 8542+0+0 (secure 0 0 0) 0x7f1bc4016040 con 0x5645148db940 2024-09-11T23:38:05.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.925+0000 7f1bd4713740 1 -- 172.21.15.26:0/2753638050 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1bc4004cc0 con 0x5645148db940 2024-09-11T23:38:05.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.925+0000 7f1bab7fe640 1 -- 172.21.15.26:0/2753638050 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7f1bc4034050 con 0x5645148db940 2024-09-11T23:38:05.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.925+0000 7f1bd4713740 10 client.17952 fetch_fsmap learned FSMap version 87 2024-09-11T23:38:05.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.925+0000 7f1bd4713740 1 -- 172.21.15.26:0/2753638050 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=87,monmap=4+}) -- 0x5645148ea310 con 0x5645148db940 2024-09-11T23:38:05.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.926+0000 7f1bab7fe640 1 -- 172.21.15.26:0/2753638050 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1bc4004cc0 con 0x5645148db940 2024-09-11T23:38:05.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.926+0000 7f1bab7fe640 1 -- 172.21.15.26:0/2753638050 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 87) ==== 8430+0+0 (secure 0 0 0) 0x7f1bc4027500 con 0x5645148db940 2024-09-11T23:38:05.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.926+0000 7f1bd4713740 10 client.17952 fetch_fsmap finished waiting for FSMap version 87 2024-09-11T23:38:05.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.926+0000 7f1bd4713740 10 client.17952 resolve_mds: validated gid 24485 aka daemon mds.b 2024-09-11T23:38:05.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.926+0000 7f1bd4713740 20 client.17952 populate_metadata read hostname 'smithi026' 2024-09-11T23:38:05.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.926+0000 7f1bd4713740 1 --2- 172.21.15.26:0/2753638050 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5645148eb770 0x5645148edb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:05.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.926+0000 7f1bd4713740 4 client.17952 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:38:05.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.926+0000 7f1bd4713740 1 -- 172.21.15.26:0/2753638050 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5645148439d0 con 0x5645148eb770 2024-09-11T23:38:05.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.926+0000 7f1bc3fff640 1 --2- 172.21.15.26:0/2753638050 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5645148eb770 0x5645148edb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:05.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.926+0000 7f1bc3fff640 1 --2- 172.21.15.26:0/2753638050 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5645148eb770 0x5645148edb60 crc :-1 s=READY pgs=294 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9da157d5a0985318 server_cookie=41ca5cb091ee669a in_seq=0 out_seq=0 2024-09-11T23:38:05.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.926+0000 7f1bab7fe640 10 client.17952 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:38:05.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.927+0000 7f1bab7fe640 1 -- 172.21.15.26:0/2753638050 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5645148439d0 con 0x5645148eb770 2024-09-11T23:38:05.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.927+0000 7f1bab7fe640 10 client.17952 handle_command_reply: tid=0 2024-09-11T23:38:05.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.927+0000 7f1bd4713740 1 client.17952 shutdown 2024-09-11T23:38:05.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.927+0000 7f1bd4713740 1 -- 172.21.15.26:0/2753638050 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5645148db940 msgr2=0x5645148d9230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:05.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.927+0000 7f1bd4713740 1 --2- 172.21.15.26:0/2753638050 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5645148db940 0x5645148d9230 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f1bc40044c0 tx=0x7f1bc40045c0 comp rx=0 tx=0).stop 2024-09-11T23:38:05.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.927+0000 7f1bd4713740 1 -- 172.21.15.26:0/2753638050 shutdown_connections 2024-09-11T23:38:05.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.927+0000 7f1bd4713740 1 --2- 172.21.15.26:0/2753638050 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5645148eb770 0x5645148edb60 crc :-1 s=READY pgs=294 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:05.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.927+0000 7f1bd4713740 1 --2- 172.21.15.26:0/2753638050 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5645148db940 0x5645148d9230 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:05.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.927+0000 7f1bd4713740 1 --2- 172.21.15.26:0/2753638050 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5645148d1080 0x5645148d8cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:05.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.927+0000 7f1bd4713740 1 --2- 172.21.15.26:0/2753638050 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5645148d0760 0x5645148d87b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:05.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.927+0000 7f1bd4713740 1 -- 172.21.15.26:0/2753638050 >> 172.21.15.26:0/2753638050 conn(0x5645148cccf0 msgr2=0x5645148cbe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:38:05.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.927+0000 7f1bab7fe640 0 client.17952 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:38:05.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.927+0000 7f1bd4713740 1 -- 172.21.15.26:0/2753638050 shutdown_connections 2024-09-11T23:38:05.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.927+0000 7f1bd4713740 1 -- 172.21.15.26:0/2753638050 wait complete. 2024-09-11T23:38:05.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.928+0000 7f1bd4713740 20 client.17952 trim_cache size 0 max 16384 2024-09-11T23:38:05.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.945+0000 7f1bd4713740 1 Processor -- start 2024-09-11T23:38:05.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.945+0000 7f1bd4713740 1 -- start start 2024-09-11T23:38:05.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.945+0000 7f1bd4713740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5645148d0760 0x5645148d7080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:05.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.945+0000 7f1bd4713740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5645148d1080 0x5645148f32f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:05.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.945+0000 7f1bd4713740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5645148db940 0x5645148f3830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:05.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.945+0000 7f1bd4713740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f1bcc1afbc0 con 0x5645148db940 2024-09-11T23:38:05.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.945+0000 7f1bd4713740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f1bcc1b19d0 con 0x5645148d1080 2024-09-11T23:38:05.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.945+0000 7f1bd4713740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f1bcc1b0320 con 0x5645148d0760 2024-09-11T23:38:05.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.945+0000 7f1bd08d7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5645148db940 0x5645148f3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:05.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.945+0000 7f1bd08d7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5645148db940 0x5645148f3830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37362/0 (socket says 172.21.15.26:37362) 2024-09-11T23:38:05.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.946+0000 7f1bd08d7640 1 -- 172.21.15.26:0/3580058117 learned_addr learned my addr 172.21.15.26:0/3580058117 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:38:05.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.946+0000 7f1bd08d7640 1 --2- 172.21.15.26:0/3580058117 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5645148d0760 0x5645148d7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:05.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.946+0000 7f1bc3fff640 1 --2- 172.21.15.26:0/3580058117 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5645148d1080 0x5645148f32f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:05.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.946+0000 7f1bd08d7640 1 -- 172.21.15.26:0/3580058117 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5645148d0760 msgr2=0x5645148d7080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:38:05.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.946+0000 7f1bd08d7640 1 --2- 172.21.15.26:0/3580058117 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5645148d0760 0x5645148d7080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:05.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.946+0000 7f1bd08d7640 1 -- 172.21.15.26:0/3580058117 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5645148d1080 msgr2=0x5645148f32f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:38:05.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.946+0000 7f1bd08d7640 1 --2- 172.21.15.26:0/3580058117 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5645148d1080 0x5645148f32f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:05.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.946+0000 7f1bd08d7640 1 -- 172.21.15.26:0/3580058117 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5645148ede60 con 0x5645148db940 2024-09-11T23:38:05.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.946+0000 7f1bd08d7640 1 --2- 172.21.15.26:0/3580058117 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5645148db940 0x5645148f3830 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f1bc4005d70 tx=0x7f1bc40118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0b3bcc16d8e04d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:38:05.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.946+0000 7f1bc1ffb640 1 -- 172.21.15.26:0/3580058117 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bc4004b80 con 0x5645148db940 2024-09-11T23:38:05.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.946+0000 7f1bc1ffb640 1 -- 172.21.15.26:0/3580058117 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1bc4027470 con 0x5645148db940 2024-09-11T23:38:05.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.946+0000 7f1bc1ffb640 1 -- 172.21.15.26:0/3580058117 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bc400fbc0 con 0x5645148db940 2024-09-11T23:38:05.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.947+0000 7f1bd4713740 1 -- 172.21.15.26:0/3580058117 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5645148db940 msgr2=0x5645148f3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:05.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.947+0000 7f1bd4713740 1 --2- 172.21.15.26:0/3580058117 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5645148db940 0x5645148f3830 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f1bc4005d70 tx=0x7f1bc40118d0 comp rx=0 tx=0).stop 2024-09-11T23:38:05.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.947+0000 7f1bd4713740 1 -- 172.21.15.26:0/3580058117 shutdown_connections 2024-09-11T23:38:05.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.947+0000 7f1bd4713740 1 --2- 172.21.15.26:0/3580058117 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5645148d1080 0x5645148f32f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:05.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.947+0000 7f1bd4713740 1 --2- 172.21.15.26:0/3580058117 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5645148db940 0x5645148f3830 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:05.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.947+0000 7f1bd4713740 1 --2- 172.21.15.26:0/3580058117 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5645148d0760 0x5645148d7080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:05.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.947+0000 7f1bd4713740 1 -- 172.21.15.26:0/3580058117 >> 172.21.15.26:0/3580058117 conn(0x5645148cccf0 msgr2=0x5645148da830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:38:05.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.947+0000 7f1bd4713740 1 -- 172.21.15.26:0/3580058117 shutdown_connections 2024-09-11T23:38:05.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.947+0000 7f1bd4713740 1 -- 172.21.15.26:0/3580058117 wait complete. 2024-09-11T23:38:05.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.947+0000 7f1bd4713740 1 -- start start 2024-09-11T23:38:05.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.948+0000 7f1bd4713740 1 Processor -- start 2024-09-11T23:38:05.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.948+0000 7f1bd4713740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5645148d0760 0x5645148e3da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:05.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.948+0000 7f1bd4713740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5645148d1080 0x5645148df0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:05.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.948+0000 7f1bd4713740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5645148db940 0x5645148df610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:05.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.948+0000 7f1bd4713740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f1bcc1b1240 con 0x5645148d1080 2024-09-11T23:38:05.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.948+0000 7f1bd4713740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f1bcc1b0ab0 con 0x5645148db940 2024-09-11T23:38:05.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.948+0000 7f1bd4713740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f1bcc1adbc0 con 0x5645148d0760 2024-09-11T23:38:05.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.948+0000 7f1bd4713740 10 client.0 _ll_register_callbacks cb 0x7f1bc4004ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:38:05.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.948+0000 7f1bd08d7640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5645148d0760 0x5645148e3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:05.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.948+0000 7f1bc3fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5645148d1080 0x5645148df0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:05.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.948+0000 7f1bd08d7640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5645148d0760 0x5645148e3da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:49488/0 (socket says 172.21.15.26:49488) 2024-09-11T23:38:05.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.948+0000 7f1bd08d7640 1 -- 172.21.15.26:0/1510900824 learned_addr learned my addr 172.21.15.26:0/1510900824 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:38:05.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.948+0000 7f1bc3fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5645148d1080 0x5645148df0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37370/0 (socket says 172.21.15.26:37370) 2024-09-11T23:38:05.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.948+0000 7f1bd08d7640 1 --2- 172.21.15.26:0/1510900824 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5645148db940 0x5645148df610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:05.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.948+0000 7f1bc3fff640 1 -- 172.21.15.26:0/1510900824 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5645148d0760 msgr2=0x5645148e3da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:38:05.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.948+0000 7f1bc3fff640 1 --2- 172.21.15.26:0/1510900824 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5645148d0760 0x5645148e3da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:05.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.948+0000 7f1bc3fff640 1 -- 172.21.15.26:0/1510900824 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5645148db940 msgr2=0x5645148df610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:38:05.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.948+0000 7f1bc3fff640 1 --2- 172.21.15.26:0/1510900824 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5645148db940 0x5645148df610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:05.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.948+0000 7f1bc3fff640 1 -- 172.21.15.26:0/1510900824 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5645148d7760 con 0x5645148d1080 2024-09-11T23:38:05.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.948+0000 7f1bd08d7640 1 --2- 172.21.15.26:0/1510900824 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5645148d0760 0x5645148e3da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:38:05.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.949+0000 7f1bc3fff640 1 --2- 172.21.15.26:0/1510900824 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5645148d1080 0x5645148df0d0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f1bb4005940 tx=0x7f1bb4004520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfe34dee48986476 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:38:05.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.949+0000 7f1bab7fe640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:38:05.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.949+0000 7f1bab7fe640 1 -- 172.21.15.26:0/1510900824 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bb4035040 con 0x5645148d1080 2024-09-11T23:38:05.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.949+0000 7f1bab7fe640 1 -- 172.21.15.26:0/1510900824 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 8542+0+0 (secure 0 0 0) 0x7f1bb4005220 con 0x5645148d1080 2024-09-11T23:38:05.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.949+0000 7f1bd4713740 1 -- 172.21.15.26:0/1510900824 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1bc4027470 con 0x5645148d1080 2024-09-11T23:38:05.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.949+0000 7f1bab7fe640 1 -- 172.21.15.26:0/1510900824 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7f1bb4053050 con 0x5645148d1080 2024-09-11T23:38:05.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.949+0000 7f1bd4713740 10 client.17958 fetch_fsmap learned FSMap version 87 2024-09-11T23:38:05.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.949+0000 7f1bd4713740 1 -- 172.21.15.26:0/1510900824 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=87,monmap=4+}) -- 0x564514955e10 con 0x5645148d1080 2024-09-11T23:38:05.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.949+0000 7f1bab7fe640 1 -- 172.21.15.26:0/1510900824 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1bc4027470 con 0x5645148d1080 2024-09-11T23:38:05.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.949+0000 7f1bab7fe640 1 -- 172.21.15.26:0/1510900824 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 87) ==== 8430+0+0 (secure 0 0 0) 0x7f1bb40049a0 con 0x5645148d1080 2024-09-11T23:38:05.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.949+0000 7f1bd4713740 10 client.17958 fetch_fsmap finished waiting for FSMap version 87 2024-09-11T23:38:05.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.950+0000 7f1bd4713740 10 client.17958 resolve_mds: validated gid 24485 aka daemon mds.b 2024-09-11T23:38:05.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.950+0000 7f1bd4713740 20 client.17958 populate_metadata read hostname 'smithi026' 2024-09-11T23:38:05.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.950+0000 7f1bd4713740 1 --2- 172.21.15.26:0/1510900824 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5645149566f0 0x564514958b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:05.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.950+0000 7f1bd4713740 4 client.17958 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 141.8, "await_for": 45.0}] 2024-09-11T23:38:05.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.950+0000 7f1bd4713740 1 -- 172.21.15.26:0/1510900824 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 141.8, "await_for": 45.0}) -- 0x56451483dd20 con 0x5645149566f0 2024-09-11T23:38:05.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.950+0000 7f1bd08d7640 1 --2- 172.21.15.26:0/1510900824 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5645149566f0 0x564514958b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:05.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.950+0000 7f1bd08d7640 1 --2- 172.21.15.26:0/1510900824 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5645149566f0 0x564514958b10 crc :-1 s=READY pgs=295 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=30abc7eef8f7cd4f server_cookie=9c3e3b29024580a9 in_seq=0 out_seq=0 2024-09-11T23:38:05.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:05.950+0000 7f1bab7fe640 10 client.17958 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:38:07.238 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:38:06 smithi136 ceph-mon[35121]: osdmap e112: 12 total, 12 up, 12 in 2024-09-11T23:38:07.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:38:06 smithi026 ceph-mon[31172]: osdmap e112: 12 total, 12 up, 12 in 2024-09-11T23:38:07.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:38:06 smithi076 ceph-mon[35296]: osdmap e112: 12 total, 12 up, 12 in 2024-09-11T23:38:08.238 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:38:07 smithi136 ceph-mon[35121]: pgmap v621: 97 pgs: 97 active+clean; 2.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.5 MiB/s wr, 250 op/s 2024-09-11T23:38:08.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:38:07 smithi136 ceph-mon[35121]: osdmap e113: 12 total, 12 up, 12 in 2024-09-11T23:38:08.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:38:07 smithi026 ceph-mon[31172]: pgmap v621: 97 pgs: 97 active+clean; 2.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.5 MiB/s wr, 250 op/s 2024-09-11T23:38:08.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:38:07 smithi026 ceph-mon[31172]: osdmap e113: 12 total, 12 up, 12 in 2024-09-11T23:38:08.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:38:07 smithi076 ceph-mon[35296]: pgmap v621: 97 pgs: 97 active+clean; 2.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.5 MiB/s wr, 250 op/s 2024-09-11T23:38:08.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:38:07 smithi076 ceph-mon[35296]: osdmap e113: 12 total, 12 up, 12 in 2024-09-11T23:38:09.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:09.055+0000 7f1bab7fe640 1 -- 172.21.15.26:0/1510900824 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x56451483dd20 con 0x5645149566f0 2024-09-11T23:38:09.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:09.056+0000 7f1bab7fe640 10 client.17958 handle_command_reply: tid=0 2024-09-11T23:38:09.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:09.056+0000 7f1bd4713740 1 client.17958 shutdown 2024-09-11T23:38:09.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:09.056+0000 7f1bd4713740 1 -- 172.21.15.26:0/1510900824 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5645148d1080 msgr2=0x5645148df0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:09.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:09.056+0000 7f1bd4713740 1 --2- 172.21.15.26:0/1510900824 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5645148d1080 0x5645148df0d0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f1bb4005940 tx=0x7f1bb4004520 comp rx=0 tx=0).stop 2024-09-11T23:38:09.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:09.056+0000 7f1bd4713740 1 -- 172.21.15.26:0/1510900824 shutdown_connections 2024-09-11T23:38:09.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:09.056+0000 7f1bd4713740 1 --2- 172.21.15.26:0/1510900824 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5645148db940 0x5645148df610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:09.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:09.056+0000 7f1bd4713740 1 --2- 172.21.15.26:0/1510900824 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5645149566f0 0x564514958b10 crc :-1 s=READY pgs=295 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:09.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:09.056+0000 7f1bd4713740 1 --2- 172.21.15.26:0/1510900824 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5645148d1080 0x5645148df0d0 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:09.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:09.057+0000 7f1bab7fe640 0 client.17958 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:38:09.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:09.057+0000 7f1bd4713740 1 --2- 172.21.15.26:0/1510900824 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5645148d0760 0x5645148e3da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:09.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:09.057+0000 7f1bd4713740 1 -- 172.21.15.26:0/1510900824 >> 172.21.15.26:0/1510900824 conn(0x5645148cccf0 msgr2=0x5645148da830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:38:09.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:09.057+0000 7f1bd4713740 1 -- 172.21.15.26:0/1510900824 shutdown_connections 2024-09-11T23:38:09.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:09.057+0000 7f1bd4713740 1 -- 172.21.15.26:0/1510900824 wait complete. 2024-09-11T23:38:09.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:09.057+0000 7f1bd4713740 20 client.17958 trim_cache size 0 max 16384 2024-09-11T23:38:09.058 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-11T23:38:09.059 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 87, 2024-09-11T23:38:09.059 INFO:teuthology.orchestra.run.smithi026.stdout: "leader": 24485, 2024-09-11T23:38:09.059 INFO:teuthology.orchestra.run.smithi026.stdout: "set_version": 66, 2024-09-11T23:38:09.059 INFO:teuthology.orchestra.run.smithi026.stdout: "sets": { 2024-09-11T23:38:09.059 INFO:teuthology.orchestra.run.smithi026.stdout: "58f7281f": { 2024-09-11T23:38:09.059 INFO:teuthology.orchestra.run.smithi026.stdout: "version": 66, 2024-09-11T23:38:09.059 INFO:teuthology.orchestra.run.smithi026.stdout: "age_ref": 0.0, 2024-09-11T23:38:09.059 INFO:teuthology.orchestra.run.smithi026.stdout: "state": { 2024-09-11T23:38:09.059 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "QUIESCED", 2024-09-11T23:38:09.059 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.0 2024-09-11T23:38:09.059 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:38:09.059 INFO:teuthology.orchestra.run.smithi026.stdout: "timeout": 90.0, 2024-09-11T23:38:09.059 INFO:teuthology.orchestra.run.smithi026.stdout: "expiration": 141.8, 2024-09-11T23:38:09.060 INFO:teuthology.orchestra.run.smithi026.stdout: "members": { 2024-09-11T23:38:09.060 INFO:teuthology.orchestra.run.smithi026.stdout: "file:/": { 2024-09-11T23:38:09.060 INFO:teuthology.orchestra.run.smithi026.stdout: "excluded": false, 2024-09-11T23:38:09.060 INFO:teuthology.orchestra.run.smithi026.stdout: "state": { 2024-09-11T23:38:09.060 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "QUIESCED", 2024-09-11T23:38:09.060 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.0 2024-09-11T23:38:09.060 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:38:09.060 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:38:09.060 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:38:09.060 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:38:09.060 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:38:09.060 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-11T23:38:09.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:09.058+0000 7f1bd233a640 1 -- 172.21.15.26:0/905734588 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f1bb80a2d70 msgr2=0x7f1bb80a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:09.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:09.058+0000 7f1bd233a640 1 --2- 172.21.15.26:0/905734588 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f1bb80a2d70 0x7f1bb80a5230 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f1bbc0ccfd0 tx=0x7f1bbc0d5550 comp rx=0 tx=0).stop 2024-09-11T23:38:09.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:09.058+0000 7f1bd233a640 1 -- 172.21.15.26:0/905734588 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1bcc100e20 msgr2=0x7f1bcc1a03d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:09.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:09.058+0000 7f1bd233a640 1 --2- 172.21.15.26:0/905734588 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1bcc100e20 0x7f1bcc1a03d0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f1bbc00e9f0 tx=0x7f1bbc00edf0 comp rx=0 tx=0).stop 2024-09-11T23:38:09.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:09.058+0000 7f1bd233a640 1 -- 172.21.15.26:0/905734588 shutdown_connections 2024-09-11T23:38:09.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:09.058+0000 7f1bd233a640 1 --2- 172.21.15.26:0/905734588 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f1bb80a2d70 0x7f1bb80a5230 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:09.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:09.058+0000 7f1bd233a640 1 --2- 172.21.15.26:0/905734588 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1bcc102ef0 0x7f1bcc19a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:09.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:09.058+0000 7f1bd233a640 1 --2- 172.21.15.26:0/905734588 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1bcc102570 0x7f1bcc1a0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:09.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:09.058+0000 7f1bd233a640 1 --2- 172.21.15.26:0/905734588 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1bcc100e20 0x7f1bcc1a03d0 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:09.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:09.058+0000 7f1bd233a640 1 -- 172.21.15.26:0/905734588 >> 172.21.15.26:0/905734588 conn(0x7f1bcc0fc7d0 msgr2=0x7f1bcc06a790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:38:09.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:09.059+0000 7f1bd233a640 1 -- 172.21.15.26:0/905734588 shutdown_connections 2024-09-11T23:38:09.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:09.059+0000 7f1bd233a640 1 -- 172.21.15.26:0/905734588 wait complete. 2024-09-11T23:38:09.933 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '58f7281f', quiesce took 6.9 seconds. Will release after: 14.9 2024-09-11T23:38:10.238 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:38:09 smithi136 ceph-mon[35121]: pgmap v623: 97 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 90 active+clean; 2.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 215 op/s 2024-09-11T23:38:10.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:38:09 smithi026 ceph-mon[31172]: pgmap v623: 97 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 90 active+clean; 2.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 215 op/s 2024-09-11T23:38:10.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:38:09 smithi076 ceph-mon[35296]: pgmap v623: 97 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 90 active+clean; 2.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 215 op/s 2024-09-11T23:38:12.128 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.127+0000 7f70f0e5e640 1 -- 172.21.15.26:0/1379277742 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f70ec100a90 msgr2=0x7f70ec100e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:12.128 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.127+0000 7f70f0e5e640 1 --2- 172.21.15.26:0/1379277742 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f70ec100a90 0x7f70ec100e70 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f70e4009ca0 tx=0x7f70e402a8f0 comp rx=0 tx=0).stop 2024-09-11T23:38:12.128 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.128+0000 7f70f0e5e640 1 -- 172.21.15.26:0/1379277742 shutdown_connections 2024-09-11T23:38:12.128 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.128+0000 7f70f0e5e640 1 --2- 172.21.15.26:0/1379277742 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f70ec105f10 0x7f70ec1063c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:12.128 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.128+0000 7f70f0e5e640 1 --2- 172.21.15.26:0/1379277742 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70ec1013b0 0x7f70ec105870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:12.128 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.128+0000 7f70f0e5e640 1 --2- 172.21.15.26:0/1379277742 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f70ec100a90 0x7f70ec100e70 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:12.128 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.128+0000 7f70f0e5e640 1 -- 172.21.15.26:0/1379277742 >> 172.21.15.26:0/1379277742 conn(0x7f70ec0fc6b0 msgr2=0x7f70ec0fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:38:12.133 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.133+0000 7f70f0e5e640 1 -- 172.21.15.26:0/1379277742 shutdown_connections 2024-09-11T23:38:12.137 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.137+0000 7f70f0e5e640 1 -- 172.21.15.26:0/1379277742 wait complete. 2024-09-11T23:38:12.137 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.138+0000 7f70f0e5e640 1 Processor -- start 2024-09-11T23:38:12.143 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.139+0000 7f70f0e5e640 1 -- start start 2024-09-11T23:38:12.143 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.139+0000 7f70f0e5e640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f70ec100a90 0x7f70ec114eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:12.143 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.139+0000 7f70f0e5e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f70ec1013b0 0x7f70ec10ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:12.143 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.140+0000 7f70f0e5e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70ec105f10 0x7f70ec10f460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:12.143 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.140+0000 7f70f0e5e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f70ec107890 con 0x7f70ec105f10 2024-09-11T23:38:12.143 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.140+0000 7f70f0e5e640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f70ec107710 con 0x7f70ec1013b0 2024-09-11T23:38:12.144 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.140+0000 7f70f0e5e640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f70ec107a10 con 0x7f70ec100a90 2024-09-11T23:38:12.144 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.140+0000 7f70ead9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70ec105f10 0x7f70ec10f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:12.144 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.140+0000 7f70ead9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70ec105f10 0x7f70ec10f460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58160/0 (socket says 172.21.15.26:58160) 2024-09-11T23:38:12.144 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.140+0000 7f70ead9d640 1 -- 172.21.15.26:0/3221362971 learned_addr learned my addr 172.21.15.26:0/3221362971 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:38:12.144 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.141+0000 7f70ead9d640 1 -- 172.21.15.26:0/3221362971 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f70ec100a90 msgr2=0x7f70ec114eb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-11T23:38:12.144 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.141+0000 7f70ead9d640 1 --2- 172.21.15.26:0/3221362971 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f70ec100a90 0x7f70ec114eb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:12.144 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.141+0000 7f70ead9d640 1 -- 172.21.15.26:0/3221362971 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f70ec1013b0 msgr2=0x7f70ec10ef20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-11T23:38:12.144 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.141+0000 7f70ead9d640 1 --2- 172.21.15.26:0/3221362971 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f70ec1013b0 0x7f70ec10ef20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:12.144 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.141+0000 7f70ead9d640 1 -- 172.21.15.26:0/3221362971 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70ec10fd20 con 0x7f70ec105f10 2024-09-11T23:38:12.144 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.141+0000 7f70ead9d640 1 --2- 172.21.15.26:0/3221362971 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70ec105f10 0x7f70ec10f460 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f70d400b3f0 tx=0x7f70d400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:38:12.144 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.142+0000 7f70e37fe640 1 -- 172.21.15.26:0/3221362971 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70d4015020 con 0x7f70ec105f10 2024-09-11T23:38:12.144 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.143+0000 7f70e37fe640 1 -- 172.21.15.26:0/3221362971 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70d4011d10 con 0x7f70ec105f10 2024-09-11T23:38:12.145 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.143+0000 7f70e37fe640 1 -- 172.21.15.26:0/3221362971 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70d400f2e0 con 0x7f70ec105f10 2024-09-11T23:38:12.145 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.146+0000 7f70f0e5e640 1 -- 172.21.15.26:0/3221362971 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70ec1b3920 con 0x7f70ec105f10 2024-09-11T23:38:12.145 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.146+0000 7f70f0e5e640 1 -- 172.21.15.26:0/3221362971 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70ec1b3df0 con 0x7f70ec105f10 2024-09-11T23:38:12.151 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.150+0000 7f70f3bf8740 1 Processor -- start 2024-09-11T23:38:12.154 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.154+0000 7f70e37fe640 1 -- 172.21.15.26:0/3221362971 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f70d4018020 con 0x7f70ec105f10 2024-09-11T23:38:12.160 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.154+0000 7f70f3bf8740 1 -- start start 2024-09-11T23:38:12.160 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.155+0000 7f70f3bf8740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dac4f77af0 0x55dac4f77ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:12.160 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.155+0000 7f70f3bf8740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dac4f78410 0x55dac4f82770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:12.160 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.155+0000 7f70f3bf8740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dac4f82cb0 0x55dac4f850a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:12.160 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.155+0000 7f70f3bf8740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55dac4edc2f0 con 0x55dac4f78410 2024-09-11T23:38:12.160 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.155+0000 7f70f3bf8740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55dac4e84270 con 0x55dac4f82cb0 2024-09-11T23:38:12.160 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.155+0000 7f70f3bf8740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55dac4ed82b0 con 0x55dac4f77af0 2024-09-11T23:38:12.160 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.159+0000 7f70ea59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dac4f78410 0x55dac4f82770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:12.161 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.159+0000 7f70ea59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dac4f78410 0x55dac4f82770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58168/0 (socket says 172.21.15.26:58168) 2024-09-11T23:38:12.161 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.159+0000 7f70ea59c640 1 -- 172.21.15.26:0/50961930 learned_addr learned my addr 172.21.15.26:0/50961930 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:38:12.161 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.159+0000 7f70e9d9b640 1 --2- 172.21.15.26:0/50961930 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dac4f77af0 0x55dac4f77ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:12.161 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.159+0000 7f70e9d9b640 1 --2- 172.21.15.26:0/50961930 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dac4f82cb0 0x55dac4f850a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:12.162 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.161+0000 7f70e9d9b640 1 -- 172.21.15.26:0/50961930 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dac4f77af0 msgr2=0x55dac4f77ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:38:12.162 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.161+0000 7f70e9d9b640 1 --2- 172.21.15.26:0/50961930 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dac4f77af0 0x55dac4f77ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:12.162 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.161+0000 7f70e9d9b640 1 -- 172.21.15.26:0/50961930 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dac4f78410 msgr2=0x55dac4f82770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:38:12.162 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.161+0000 7f70e9d9b640 1 --2- 172.21.15.26:0/50961930 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dac4f78410 0x55dac4f82770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:12.162 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.161+0000 7f70e9d9b640 1 -- 172.21.15.26:0/50961930 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dac4e83340 con 0x55dac4f82cb0 2024-09-11T23:38:12.162 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.161+0000 7f70e9d9b640 1 --2- 172.21.15.26:0/50961930 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dac4f82cb0 0x55dac4f850a0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f70ec1106e0 tx=0x7f70dc0068d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9d74bd2cc08f2310 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:38:12.162 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.162+0000 7f70e37fe640 1 --2- 172.21.15.26:0/3221362971 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f70d80a2f80 0x7f70d80a5440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:12.162 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.162+0000 7f70e0ff9640 1 -- 172.21.15.26:0/50961930 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70dc011040 con 0x55dac4f82cb0 2024-09-11T23:38:12.162 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.162+0000 7f70e0ff9640 1 -- 172.21.15.26:0/50961930 <== mon.2 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70dc00bba0 con 0x55dac4f82cb0 2024-09-11T23:38:12.163 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.162+0000 7f70e37fe640 1 -- 172.21.15.26:0/3221362971 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 8586+0+0 (secure 0 0 0) 0x7f70d40cec60 con 0x7f70ec105f10 2024-09-11T23:38:12.163 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.163+0000 7f70f3bf8740 1 -- 172.21.15.26:0/50961930 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dac4f82cb0 msgr2=0x55dac4f850a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:12.163 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.163+0000 7f70f3bf8740 1 --2- 172.21.15.26:0/50961930 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dac4f82cb0 0x55dac4f850a0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f70ec1106e0 tx=0x7f70dc0068d0 comp rx=0 tx=0).stop 2024-09-11T23:38:12.165 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.166+0000 7f70ea59c640 1 --2- 172.21.15.26:0/3221362971 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f70d80a2f80 0x7f70d80a5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:12.166 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.166+0000 7f70f3bf8740 1 -- 172.21.15.26:0/50961930 shutdown_connections 2024-09-11T23:38:12.166 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.166+0000 7f70f3bf8740 1 --2- 172.21.15.26:0/50961930 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dac4f82cb0 0x55dac4f850a0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:12.166 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.166+0000 7f70f3bf8740 1 --2- 172.21.15.26:0/50961930 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dac4f78410 0x55dac4f82770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:12.166 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.166+0000 7f70f3bf8740 1 --2- 172.21.15.26:0/50961930 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dac4f77af0 0x55dac4f77ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:12.166 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.166+0000 7f70f3bf8740 1 -- 172.21.15.26:0/50961930 >> 172.21.15.26:0/50961930 conn(0x55dac4f74060 msgr2=0x55dac4f74440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:38:12.167 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.168+0000 7f70f3bf8740 1 -- 172.21.15.26:0/50961930 shutdown_connections 2024-09-11T23:38:12.168 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.168+0000 7f70f3bf8740 1 -- 172.21.15.26:0/50961930 wait complete. 2024-09-11T23:38:12.168 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.169+0000 7f70f3bf8740 1 -- start start 2024-09-11T23:38:12.168 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.169+0000 7f70f3bf8740 1 Processor -- start 2024-09-11T23:38:12.172 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.172+0000 7f70ea59c640 1 --2- 172.21.15.26:0/3221362971 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f70d80a2f80 0x7f70d80a5440 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f70e402f570 tx=0x7f70e4005c30 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:38:12.173 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.173+0000 7f70f3bf8740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dac4f77af0 0x55dac4f7fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:12.173 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.173+0000 7f70f3bf8740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dac4f78410 0x55dac4f80120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:12.173 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.173+0000 7f70f3bf8740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dac4f82cb0 0x55dac4f80660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:12.173 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.173+0000 7f70f3bf8740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55dac4f968e0 con 0x55dac4f77af0 2024-09-11T23:38:12.173 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.173+0000 7f70f3bf8740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55dac4e65290 con 0x55dac4f82cb0 2024-09-11T23:38:12.173 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.173+0000 7f70f3bf8740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55dac4f96a60 con 0x55dac4f78410 2024-09-11T23:38:12.173 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.173+0000 7f70f3bf8740 10 client.0 _ll_register_callbacks cb 0x55dac4e96140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:38:12.174 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.174+0000 7f70e9d9b640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dac4f78410 0x55dac4f80120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:12.174 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.174+0000 7f70e9d9b640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dac4f78410 0x55dac4f80120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:58636/0 (socket says 172.21.15.26:58636) 2024-09-11T23:38:12.174 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.174+0000 7f70e9d9b640 1 -- 172.21.15.26:0/2155426763 learned_addr learned my addr 172.21.15.26:0/2155426763 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:38:12.174 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.174+0000 7f70e9d9b640 1 -- 172.21.15.26:0/2155426763 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dac4f82cb0 msgr2=0x55dac4f80660 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T23:38:12.174 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.174+0000 7f70e9d9b640 1 --2- 172.21.15.26:0/2155426763 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dac4f82cb0 0x55dac4f80660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:12.174 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.174+0000 7f70e9d9b640 1 -- 172.21.15.26:0/2155426763 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dac4f77af0 msgr2=0x55dac4f7fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:38:12.174 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.174+0000 7f70e9d9b640 1 --2- 172.21.15.26:0/2155426763 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dac4f77af0 0x55dac4f7fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:12.174 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.174+0000 7f70e9d9b640 1 -- 172.21.15.26:0/2155426763 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dac4bef210 con 0x55dac4f78410 2024-09-11T23:38:12.175 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.175+0000 7f70e9d9b640 1 --2- 172.21.15.26:0/2155426763 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dac4f78410 0x55dac4f80120 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f70dc004ae0 tx=0x7f70dc014f80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4bec12de3fe662a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:38:12.175 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.176+0000 7f70d1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-09-11T23:38:12.175 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.176+0000 7f70d1ffb640 1 -- 172.21.15.26:0/2155426763 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70dc0065e0 con 0x55dac4f78410 2024-09-11T23:38:12.175 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.176+0000 7f70f3bf8740 1 -- 172.21.15.26:0/2155426763 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f70dc00bba0 con 0x55dac4f78410 2024-09-11T23:38:12.175 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.176+0000 7f70d1ffb640 1 -- 172.21.15.26:0/2155426763 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 8586+0+0 (secure 0 0 0) 0x7f70dc003780 con 0x55dac4f78410 2024-09-11T23:38:12.180 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.181+0000 7f70d1ffb640 1 -- 172.21.15.26:0/2155426763 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7f70dc003980 con 0x55dac4f78410 2024-09-11T23:38:12.182 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.183+0000 7f70f3bf8740 10 client.27436 fetch_fsmap learned FSMap version 87 2024-09-11T23:38:12.182 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.183+0000 7f70f3bf8740 1 -- 172.21.15.26:0/2155426763 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=87,monmap=4+}) -- 0x55dac4cbb620 con 0x55dac4f78410 2024-09-11T23:38:12.187 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.188+0000 7f70d1ffb640 1 -- 172.21.15.26:0/2155426763 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70dc00bba0 con 0x55dac4f78410 2024-09-11T23:38:12.187 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.188+0000 7f70d1ffb640 1 -- 172.21.15.26:0/2155426763 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 87) ==== 8430+0+0 (secure 0 0 0) 0x7f70dc004030 con 0x55dac4f78410 2024-09-11T23:38:12.188 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.188+0000 7f70f3bf8740 10 client.27436 fetch_fsmap finished waiting for FSMap version 87 2024-09-11T23:38:12.188 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.188+0000 7f70f3bf8740 10 client.27436 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:38:12.188 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.188+0000 7f70f3bf8740 20 client.27436 populate_metadata read hostname 'smithi026' 2024-09-11T23:38:12.188 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.188+0000 7f70f3bf8740 1 --2- 172.21.15.26:0/2155426763 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55dac4fa4080 0x55dac4fa6520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:12.188 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.188+0000 7f70f3bf8740 4 client.27436 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:38:12.188 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.188+0000 7f70f3bf8740 1 -- 172.21.15.26:0/2155426763 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dac4ee9a20 con 0x55dac4fa4080 2024-09-11T23:38:12.192 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.191+0000 7f70ea59c640 1 --2- 172.21.15.26:0/2155426763 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55dac4fa4080 0x55dac4fa6520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:12.196 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.197+0000 7f70ea59c640 1 --2- 172.21.15.26:0/2155426763 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55dac4fa4080 0x55dac4fa6520 crc :-1 s=READY pgs=296 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75e20355bb0bba06 server_cookie=33606f0e32b53db1 in_seq=0 out_seq=0 2024-09-11T23:38:12.197 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.198+0000 7f70d1ffb640 10 client.27436 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:38:12.198 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.198+0000 7f70d1ffb640 1 -- 172.21.15.26:0/2155426763 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55dac4ee9a20 con 0x55dac4fa4080 2024-09-11T23:38:12.198 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.199+0000 7f70d1ffb640 10 client.27436 handle_command_reply: tid=0 2024-09-11T23:38:12.198 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.199+0000 7f70f3bf8740 1 client.27436 shutdown 2024-09-11T23:38:12.202 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.202+0000 7f70f3bf8740 1 -- 172.21.15.26:0/2155426763 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dac4f78410 msgr2=0x55dac4f80120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:12.202 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.202+0000 7f70f3bf8740 1 --2- 172.21.15.26:0/2155426763 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dac4f78410 0x55dac4f80120 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f70dc004ae0 tx=0x7f70dc014f80 comp rx=0 tx=0).stop 2024-09-11T23:38:12.204 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.204+0000 7f70f3bf8740 1 -- 172.21.15.26:0/2155426763 shutdown_connections 2024-09-11T23:38:12.204 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.204+0000 7f70f3bf8740 1 --2- 172.21.15.26:0/2155426763 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dac4f82cb0 0x55dac4f80660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:12.204 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.204+0000 7f70f3bf8740 1 --2- 172.21.15.26:0/2155426763 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55dac4fa4080 0x55dac4fa6520 crc :-1 s=READY pgs=296 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:12.205 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.204+0000 7f70f3bf8740 1 --2- 172.21.15.26:0/2155426763 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dac4f78410 0x55dac4f80120 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:12.205 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.204+0000 7f70f3bf8740 1 --2- 172.21.15.26:0/2155426763 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dac4f77af0 0x55dac4f7fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:12.205 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.204+0000 7f70f3bf8740 1 -- 172.21.15.26:0/2155426763 >> 172.21.15.26:0/2155426763 conn(0x55dac4f74060 msgr2=0x55dac4f79dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:38:12.205 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.205+0000 7f70d1ffb640 0 client.27436 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:38:12.209 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.210+0000 7f70f3bf8740 1 -- 172.21.15.26:0/2155426763 shutdown_connections 2024-09-11T23:38:12.213 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.211+0000 7f70f3bf8740 1 -- 172.21.15.26:0/2155426763 wait complete. 2024-09-11T23:38:12.213 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.211+0000 7f70f3bf8740 20 client.27436 trim_cache size 0 max 16384 2024-09-11T23:38:12.238 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:38:11 smithi136 ceph-mon[35121]: pgmap v624: 97 pgs: 17 active+clean+snaptrim, 29 active+clean+snaptrim_wait, 51 active+clean; 2.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 209 op/s 2024-09-11T23:38:12.241 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.242+0000 7f70f3bf8740 1 Processor -- start 2024-09-11T23:38:12.241 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.242+0000 7f70f3bf8740 1 -- start start 2024-09-11T23:38:12.242 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.242+0000 7f70f3bf8740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dac4f77af0 0x55dac4f7a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:12.242 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.242+0000 7f70f3bf8740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dac4f78410 0x55dac4fabe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:12.242 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.242+0000 7f70f3bf8740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dac4f82cb0 0x55dac4fac370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:12.242 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.242+0000 7f70f3bf8740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f70ec1b40f0 con 0x55dac4f77af0 2024-09-11T23:38:12.242 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.242+0000 7f70f3bf8740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f70ec1b5f00 con 0x55dac4f82cb0 2024-09-11T23:38:12.242 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.242+0000 7f70f3bf8740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f70ec1b4850 con 0x55dac4f78410 2024-09-11T23:38:12.243 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.243+0000 7f70ea59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dac4f77af0 0x55dac4f7a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:12.243 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.243+0000 7f70ea59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dac4f77af0 0x55dac4f7a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58184/0 (socket says 172.21.15.26:58184) 2024-09-11T23:38:12.243 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.243+0000 7f70ea59c640 1 -- 172.21.15.26:0/2377961877 learned_addr learned my addr 172.21.15.26:0/2377961877 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:38:12.243 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.243+0000 7f70ead9d640 1 --2- 172.21.15.26:0/2377961877 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dac4f82cb0 0x55dac4fac370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:12.244 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.243+0000 7f70ea59c640 1 -- 172.21.15.26:0/2377961877 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dac4f78410 msgr2=0x55dac4fabe30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T23:38:12.244 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.243+0000 7f70ea59c640 1 --2- 172.21.15.26:0/2377961877 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dac4f78410 0x55dac4fabe30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:12.244 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.243+0000 7f70ea59c640 1 -- 172.21.15.26:0/2377961877 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dac4f82cb0 msgr2=0x55dac4fac370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:38:12.244 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.243+0000 7f70ea59c640 1 --2- 172.21.15.26:0/2377961877 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dac4f82cb0 0x55dac4fac370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:12.244 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.243+0000 7f70ea59c640 1 -- 172.21.15.26:0/2377961877 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dac4f78240 con 0x55dac4f77af0 2024-09-11T23:38:12.244 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.244+0000 7f70ea59c640 1 --2- 172.21.15.26:0/2377961877 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dac4f77af0 0x55dac4f7a5b0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f70e4005cc0 tx=0x7f70e403e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b4ebaf1ce6d549f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:38:12.248 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.249+0000 7f70d3fff640 1 -- 172.21.15.26:0/2377961877 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70e4056070 con 0x55dac4f77af0 2024-09-11T23:38:12.248 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.249+0000 7f70d3fff640 1 -- 172.21.15.26:0/2377961877 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70e4036040 con 0x55dac4f77af0 2024-09-11T23:38:12.248 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.249+0000 7f70d3fff640 1 -- 172.21.15.26:0/2377961877 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70e40047f0 con 0x55dac4f77af0 2024-09-11T23:38:12.249 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.250+0000 7f70f3bf8740 1 -- 172.21.15.26:0/2377961877 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dac4f77af0 msgr2=0x55dac4f7a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:12.249 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.250+0000 7f70f3bf8740 1 --2- 172.21.15.26:0/2377961877 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dac4f77af0 0x55dac4f7a5b0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f70e4005cc0 tx=0x7f70e403e000 comp rx=0 tx=0).stop 2024-09-11T23:38:12.251 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.252+0000 7f70f3bf8740 1 -- 172.21.15.26:0/2377961877 shutdown_connections 2024-09-11T23:38:12.251 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.252+0000 7f70f3bf8740 1 --2- 172.21.15.26:0/2377961877 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dac4f82cb0 0x55dac4fac370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:12.251 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.252+0000 7f70f3bf8740 1 --2- 172.21.15.26:0/2377961877 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dac4f78410 0x55dac4fabe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:12.251 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.252+0000 7f70f3bf8740 1 --2- 172.21.15.26:0/2377961877 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dac4f77af0 0x55dac4f7a5b0 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:12.251 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.252+0000 7f70f3bf8740 1 -- 172.21.15.26:0/2377961877 >> 172.21.15.26:0/2377961877 conn(0x55dac4f74060 msgr2=0x55dac4f7af10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:38:12.253 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:38:11 smithi026 ceph-mon[31172]: pgmap v624: 97 pgs: 17 active+clean+snaptrim, 29 active+clean+snaptrim_wait, 51 active+clean; 2.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 209 op/s 2024-09-11T23:38:12.254 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.255+0000 7f70f3bf8740 1 -- 172.21.15.26:0/2377961877 shutdown_connections 2024-09-11T23:38:12.255 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.256+0000 7f70f3bf8740 1 -- 172.21.15.26:0/2377961877 wait complete. 2024-09-11T23:38:12.255 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.256+0000 7f70f3bf8740 1 -- start start 2024-09-11T23:38:12.255 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.256+0000 7f70f3bf8740 1 Processor -- start 2024-09-11T23:38:12.256 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.257+0000 7f70f3bf8740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dac4f77af0 0x55dac4f96d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:12.256 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.257+0000 7f70f3bf8740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dac4f78410 0x55dac4f97240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:12.256 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.257+0000 7f70f3bf8740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dac4f82cb0 0x55dac4f97780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:12.256 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.257+0000 7f70f3bf8740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f70ec1b5770 con 0x55dac4f82cb0 2024-09-11T23:38:12.256 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.257+0000 7f70f3bf8740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f70ec1b4fe0 con 0x55dac4f78410 2024-09-11T23:38:12.256 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.257+0000 7f70f3bf8740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f70ec1058f0 con 0x55dac4f77af0 2024-09-11T23:38:12.257 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.257+0000 7f70f3bf8740 10 client.0 _ll_register_callbacks cb 0x7f70dc00b530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:38:12.257 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.258+0000 7f70ead9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dac4f82cb0 0x55dac4f97780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:12.257 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.258+0000 7f70ead9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dac4f82cb0 0x55dac4f97780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58196/0 (socket says 172.21.15.26:58196) 2024-09-11T23:38:12.257 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.258+0000 7f70ead9d640 1 -- 172.21.15.26:0/1560963634 learned_addr learned my addr 172.21.15.26:0/1560963634 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:38:12.257 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.258+0000 7f70e9d9b640 1 --2- 172.21.15.26:0/1560963634 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dac4f78410 0x55dac4f97240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:12.261 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.261+0000 7f70ea59c640 1 --2- 172.21.15.26:0/1560963634 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dac4f77af0 0x55dac4f96d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:12.265 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.262+0000 7f70ea59c640 1 -- 172.21.15.26:0/1560963634 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dac4f78410 msgr2=0x55dac4f97240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:38:12.265 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.262+0000 7f70ea59c640 1 --2- 172.21.15.26:0/1560963634 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dac4f78410 0x55dac4f97240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:12.265 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.262+0000 7f70ea59c640 1 -- 172.21.15.26:0/1560963634 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dac4f82cb0 msgr2=0x55dac4f97780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:38:12.265 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.263+0000 7f70ea59c640 1 --2- 172.21.15.26:0/1560963634 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dac4f82cb0 0x55dac4f97780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:12.265 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.263+0000 7f70ea59c640 1 -- 172.21.15.26:0/1560963634 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dac500d010 con 0x55dac4f77af0 2024-09-11T23:38:12.265 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.263+0000 7f70ea59c640 1 --2- 172.21.15.26:0/1560963634 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dac4f77af0 0x55dac4f96d00 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f70e4009ca0 tx=0x7f70e402a8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=49ecce9d8c2fa474 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:38:12.266 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.264+0000 7f70d1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-09-11T23:38:12.266 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.264+0000 7f70d1ffb640 1 -- 172.21.15.26:0/1560963634 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70e4056040 con 0x55dac4f77af0 2024-09-11T23:38:12.266 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.264+0000 7f70d1ffb640 1 -- 172.21.15.26:0/1560963634 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 8586+0+0 (secure 0 0 0) 0x7f70e4004430 con 0x55dac4f77af0 2024-09-11T23:38:12.266 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.264+0000 7f70f3bf8740 1 -- 172.21.15.26:0/1560963634 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f70e4036040 con 0x55dac4f77af0 2024-09-11T23:38:12.271 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.268+0000 7f70d1ffb640 1 -- 172.21.15.26:0/1560963634 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7f70e4004630 con 0x55dac4f77af0 2024-09-11T23:38:12.271 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.272+0000 7f70f3bf8740 10 client.27439 fetch_fsmap learned FSMap version 87 2024-09-11T23:38:12.271 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.272+0000 7f70f3bf8740 1 -- 172.21.15.26:0/1560963634 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=87,monmap=4+}) -- 0x55dac500e300 con 0x55dac4f77af0 2024-09-11T23:38:12.275 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.276+0000 7f70d1ffb640 1 -- 172.21.15.26:0/1560963634 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70e4036040 con 0x55dac4f77af0 2024-09-11T23:38:12.277 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.277+0000 7f70d1ffb640 1 -- 172.21.15.26:0/1560963634 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 87) ==== 8430+0+0 (secure 0 0 0) 0x7f70e4059070 con 0x55dac4f77af0 2024-09-11T23:38:12.277 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.277+0000 7f70f3bf8740 10 client.27439 fetch_fsmap finished waiting for FSMap version 87 2024-09-11T23:38:12.277 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.277+0000 7f70f3bf8740 10 client.27439 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:38:12.277 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.277+0000 7f70f3bf8740 20 client.27439 populate_metadata read hostname 'smithi026' 2024-09-11T23:38:12.277 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.277+0000 7f70f3bf8740 1 --2- 172.21.15.26:0/1560963634 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55dac500e9e0 0x55dac5010e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:12.277 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.277+0000 7f70f3bf8740 4 client.27439 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T23:38:12.277 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.277+0000 7f70f3bf8740 1 -- 172.21.15.26:0/1560963634 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55dac4ee3d70 con 0x55dac500e9e0 2024-09-11T23:38:12.284 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.284+0000 7f70e9d9b640 1 --2- 172.21.15.26:0/1560963634 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55dac500e9e0 0x55dac5010e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:12.294 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.293+0000 7f70e9d9b640 1 --2- 172.21.15.26:0/1560963634 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55dac500e9e0 0x55dac5010e00 crc :-1 s=READY pgs=297 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a6133f3ddb043d6 server_cookie=32431aad5e0d3582 in_seq=0 out_seq=0 2024-09-11T23:38:12.294 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.293+0000 7f70d1ffb640 10 client.27439 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:38:12.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:38:11 smithi076 ceph-mon[35296]: pgmap v624: 97 pgs: 17 active+clean+snaptrim, 29 active+clean+snaptrim_wait, 51 active+clean; 2.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 209 op/s 2024-09-11T23:38:12.389 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.389+0000 7f70d1ffb640 1 -- 172.21.15.26:0/1560963634 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55dac4ee3d70 con 0x55dac500e9e0 2024-09-11T23:38:12.389 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.389+0000 7f70d1ffb640 10 client.27439 handle_command_reply: tid=0 2024-09-11T23:38:12.390 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.389+0000 7f70f3bf8740 1 client.27439 shutdown 2024-09-11T23:38:12.390 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.389+0000 7f70f3bf8740 1 -- 172.21.15.26:0/1560963634 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dac4f77af0 msgr2=0x55dac4f96d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:12.390 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.389+0000 7f70f3bf8740 1 --2- 172.21.15.26:0/1560963634 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dac4f77af0 0x55dac4f96d00 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f70e4009ca0 tx=0x7f70e402a8f0 comp rx=0 tx=0).stop 2024-09-11T23:38:12.390 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.389+0000 7f70f3bf8740 1 -- 172.21.15.26:0/1560963634 shutdown_connections 2024-09-11T23:38:12.390 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.389+0000 7f70f3bf8740 1 --2- 172.21.15.26:0/1560963634 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dac4f78410 0x55dac4f97240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:12.390 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.389+0000 7f70f3bf8740 1 --2- 172.21.15.26:0/1560963634 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55dac500e9e0 0x55dac5010e00 crc :-1 s=READY pgs=297 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:12.390 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.389+0000 7f70f3bf8740 1 --2- 172.21.15.26:0/1560963634 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dac4f82cb0 0x55dac4f97780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:12.390 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.389+0000 7f70f3bf8740 1 --2- 172.21.15.26:0/1560963634 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dac4f77af0 0x55dac4f96d00 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:12.390 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.389+0000 7f70f3bf8740 1 -- 172.21.15.26:0/1560963634 >> 172.21.15.26:0/1560963634 conn(0x55dac4f74060 msgr2=0x55dac4f729a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:38:12.390 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.389+0000 7f70d1ffb640 0 client.27439 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:38:12.390 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.390+0000 7f70f3bf8740 1 -- 172.21.15.26:0/1560963634 shutdown_connections 2024-09-11T23:38:12.390 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.390+0000 7f70f3bf8740 1 -- 172.21.15.26:0/1560963634 wait complete. 2024-09-11T23:38:12.391 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.390+0000 7f70f3bf8740 20 client.27439 trim_cache size 0 max 16384 2024-09-11T23:38:12.391 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-09-11T23:38:12.391 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "message": "", 2024-09-11T23:38:12.391 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-09-11T23:38:12.391 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-09-11T23:38:12.392 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.391+0000 7f70f0e5e640 1 -- 172.21.15.26:0/3221362971 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f70d80a2f80 msgr2=0x7f70d80a5440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:12.393 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.391+0000 7f70f0e5e640 1 --2- 172.21.15.26:0/3221362971 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f70d80a2f80 0x7f70d80a5440 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f70e402f570 tx=0x7f70e4005c30 comp rx=0 tx=0).stop 2024-09-11T23:38:12.393 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.391+0000 7f70f0e5e640 1 -- 172.21.15.26:0/3221362971 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70ec105f10 msgr2=0x7f70ec10f460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:12.393 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.391+0000 7f70f0e5e640 1 --2- 172.21.15.26:0/3221362971 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70ec105f10 0x7f70ec10f460 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f70d400b3f0 tx=0x7f70d400b8b0 comp rx=0 tx=0).stop 2024-09-11T23:38:12.393 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.391+0000 7f70f0e5e640 1 -- 172.21.15.26:0/3221362971 shutdown_connections 2024-09-11T23:38:12.393 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.391+0000 7f70f0e5e640 1 --2- 172.21.15.26:0/3221362971 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f70d80a2f80 0x7f70d80a5440 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:12.393 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.391+0000 7f70f0e5e640 1 --2- 172.21.15.26:0/3221362971 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f70ec1013b0 0x7f70ec10ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:12.393 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.391+0000 7f70f0e5e640 1 --2- 172.21.15.26:0/3221362971 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70ec105f10 0x7f70ec10f460 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:12.393 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.391+0000 7f70f0e5e640 1 --2- 172.21.15.26:0/3221362971 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f70ec100a90 0x7f70ec114eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:12.393 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.391+0000 7f70f0e5e640 1 -- 172.21.15.26:0/3221362971 >> 172.21.15.26:0/3221362971 conn(0x7f70ec0fc6b0 msgr2=0x7f70ec108050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:38:12.393 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.391+0000 7f70f0e5e640 1 -- 172.21.15.26:0/3221362971 shutdown_connections 2024-09-11T23:38:12.393 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:12.391+0000 7f70f0e5e640 1 -- 172.21.15.26:0/3221362971 wait complete. 2024-09-11T23:38:14.238 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:38:13 smithi136 ceph-mon[35121]: pgmap v625: 97 pgs: 23 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 33 active+clean; 2.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 193 op/s 2024-09-11T23:38:14.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:38:13 smithi026 ceph-mon[31172]: pgmap v625: 97 pgs: 23 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 33 active+clean; 2.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 193 op/s 2024-09-11T23:38:14.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:38:13 smithi076 ceph-mon[35296]: pgmap v625: 97 pgs: 23 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 33 active+clean; 2.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 193 op/s 2024-09-11T23:38:16.238 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:38:15 smithi136 ceph-mon[35121]: pgmap v626: 97 pgs: 23 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 33 active+clean; 2.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 966 KiB/s rd, 972 KiB/s wr, 105 op/s 2024-09-11T23:38:16.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:38:15 smithi026 ceph-mon[31172]: pgmap v626: 97 pgs: 23 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 33 active+clean; 2.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 966 KiB/s rd, 972 KiB/s wr, 105 op/s 2024-09-11T23:38:16.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:38:15 smithi076 ceph-mon[35296]: pgmap v626: 97 pgs: 23 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 33 active+clean; 2.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 966 KiB/s rd, 972 KiB/s wr, 105 op/s 2024-09-11T23:38:18.241 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:38:17 smithi136 ceph-mon[35121]: pgmap v627: 97 pgs: 23 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 33 active+clean; 2.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 863 KiB/s rd, 860 KiB/s wr, 99 op/s 2024-09-11T23:38:18.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:38:17 smithi026 ceph-mon[31172]: pgmap v627: 97 pgs: 23 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 33 active+clean; 2.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 863 KiB/s rd, 860 KiB/s wr, 99 op/s 2024-09-11T23:38:18.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:38:17 smithi076 ceph-mon[35296]: pgmap v627: 97 pgs: 23 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 33 active+clean; 2.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 863 KiB/s rd, 860 KiB/s wr, 99 op/s 2024-09-11T23:38:20.238 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:38:19 smithi136 ceph-mon[35121]: pgmap v628: 97 pgs: 23 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 33 active+clean; 2.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 740 KiB/s rd, 737 KiB/s wr, 87 op/s 2024-09-11T23:38:20.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:38:19 smithi026 ceph-mon[31172]: pgmap v628: 97 pgs: 23 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 33 active+clean; 2.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 740 KiB/s rd, 737 KiB/s wr, 87 op/s 2024-09-11T23:38:20.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:38:19 smithi076 ceph-mon[35296]: pgmap v628: 97 pgs: 23 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 33 active+clean; 2.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 740 KiB/s rd, 737 KiB/s wr, 87 op/s 2024-09-11T23:38:22.238 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:38:21 smithi136 ceph-mon[35121]: pgmap v629: 97 pgs: 23 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 33 active+clean; 2.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 382 KiB/s rd, 379 KiB/s wr, 78 op/s 2024-09-11T23:38:22.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:38:21 smithi026 ceph-mon[31172]: pgmap v629: 97 pgs: 23 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 33 active+clean; 2.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 382 KiB/s rd, 379 KiB/s wr, 78 op/s 2024-09-11T23:38:22.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:38:21 smithi076 ceph-mon[35296]: pgmap v629: 97 pgs: 23 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 33 active+clean; 2.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 382 KiB/s rd, 379 KiB/s wr, 78 op/s 2024-09-11T23:38:23.000 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:38:22 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: 2024-09-11T23:38:22.635+0000 7f7ccf119640 -1 calc_pg_upmaps abort due to max <= 0 2024-09-11T23:38:24.238 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:38:23 smithi136 ceph-mon[35121]: pgmap v630: 97 pgs: 23 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 33 active+clean; 2.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 24 KiB/s wr, 38 op/s 2024-09-11T23:38:24.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:38:23 smithi026 ceph-mon[31172]: pgmap v630: 97 pgs: 23 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 33 active+clean; 2.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 24 KiB/s wr, 38 op/s 2024-09-11T23:38:24.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:38:23 smithi076 ceph-mon[35296]: pgmap v630: 97 pgs: 23 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 33 active+clean; 2.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 24 KiB/s wr, 38 op/s 2024-09-11T23:38:24.834 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:38:25.341 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:38:25.543 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.543+0000 7fb046e1e640 1 -- 172.21.15.26:0/768332985 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb0401091f0 msgr2=0x7fb040110250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:25.543 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.543+0000 7fb046e1e640 1 --2- 172.21.15.26:0/768332985 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb0401091f0 0x7fb040110250 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fb03801ace0 tx=0x7fb03803b970 comp rx=0 tx=0).stop 2024-09-11T23:38:25.547 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.547+0000 7fb046e1e640 1 -- 172.21.15.26:0/768332985 shutdown_connections 2024-09-11T23:38:25.547 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.547+0000 7fb046e1e640 1 --2- 172.21.15.26:0/768332985 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb04011b140 0x7fb04011d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:25.547 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.547+0000 7fb046e1e640 1 --2- 172.21.15.26:0/768332985 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb0401095d0 0x7fb040109a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:25.547 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.547+0000 7fb046e1e640 1 --2- 172.21.15.26:0/768332985 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb0401091f0 0x7fb040110250 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:25.547 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.547+0000 7fb046e1e640 1 -- 172.21.15.26:0/768332985 >> 172.21.15.26:0/768332985 conn(0x7fb04006ce00 msgr2=0x7fb04006d210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:38:25.549 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.549+0000 7fb046e1e640 1 -- 172.21.15.26:0/768332985 shutdown_connections 2024-09-11T23:38:25.550 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.550+0000 7fb046e1e640 1 -- 172.21.15.26:0/768332985 wait complete. 2024-09-11T23:38:25.550 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.551+0000 7fb046e1e640 1 Processor -- start 2024-09-11T23:38:25.550 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.551+0000 7fb046e1e640 1 -- start start 2024-09-11T23:38:25.554 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.553+0000 7fb046e1e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb0401091f0 0x7fb04011a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:25.554 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.553+0000 7fb046e1e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb0401095d0 0x7fb0401168b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:25.554 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.553+0000 7fb046e1e640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb04011b140 0x7fb040116df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:25.554 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.553+0000 7fb046e1e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb040113360 con 0x7fb0401095d0 2024-09-11T23:38:25.554 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.553+0000 7fb046e1e640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fb0401131e0 con 0x7fb0401091f0 2024-09-11T23:38:25.554 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.553+0000 7fb046e1e640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb0401134e0 con 0x7fb04011b140 2024-09-11T23:38:25.554 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.553+0000 7fb044bba640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb0401091f0 0x7fb04011a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:25.554 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.553+0000 7fb044bba640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb0401091f0 0x7fb04011a820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:33390/0 (socket says 172.21.15.26:33390) 2024-09-11T23:38:25.555 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.553+0000 7fb044bba640 1 -- 172.21.15.26:0/1993705165 learned_addr learned my addr 172.21.15.26:0/1993705165 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:38:25.555 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.553+0000 7fb03ffff640 1 --2- 172.21.15.26:0/1993705165 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb0401095d0 0x7fb0401168b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:25.555 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.553+0000 7fb044bba640 1 -- 172.21.15.26:0/1993705165 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb04011b140 msgr2=0x7fb040116df0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-11T23:38:25.555 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.553+0000 7fb044bba640 1 --2- 172.21.15.26:0/1993705165 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb04011b140 0x7fb040116df0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:25.555 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.553+0000 7fb044bba640 1 -- 172.21.15.26:0/1993705165 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb0401095d0 msgr2=0x7fb0401168b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:25.555 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.553+0000 7fb044bba640 1 --2- 172.21.15.26:0/1993705165 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb0401095d0 0x7fb0401168b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:25.555 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.553+0000 7fb044bba640 1 -- 172.21.15.26:0/1993705165 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb040117520 con 0x7fb0401091f0 2024-09-11T23:38:25.555 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.554+0000 7fb044bba640 1 --2- 172.21.15.26:0/1993705165 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb0401091f0 0x7fb04011a820 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fb038000f80 tx=0x7fb038041a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:38:25.555 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.554+0000 7fb03dffb640 1 -- 172.21.15.26:0/1993705165 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb038045070 con 0x7fb0401091f0 2024-09-11T23:38:25.555 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.554+0000 7fb046e1e640 1 -- 172.21.15.26:0/1993705165 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0401ab070 con 0x7fb0401091f0 2024-09-11T23:38:25.555 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.554+0000 7fb046e1e640 1 -- 172.21.15.26:0/1993705165 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0401ab420 con 0x7fb0401091f0 2024-09-11T23:38:25.555 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.554+0000 7fb03dffb640 1 -- 172.21.15.26:0/1993705165 <== mon.2 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb038003e50 con 0x7fb0401091f0 2024-09-11T23:38:25.555 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.554+0000 7fb03dffb640 1 -- 172.21.15.26:0/1993705165 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb038004040 con 0x7fb0401091f0 2024-09-11T23:38:25.562 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.563+0000 7fb03dffb640 1 -- 172.21.15.26:0/1993705165 <== mon.2 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fb038048020 con 0x7fb0401091f0 2024-09-11T23:38:25.565 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.566+0000 7fb049c8f740 1 Processor -- start 2024-09-11T23:38:25.568 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.567+0000 7fb03dffb640 1 --2- 172.21.15.26:0/1993705165 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fb0280a2ef0 0x7fb0280a53b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:25.568 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.567+0000 7fb03dffb640 1 -- 172.21.15.26:0/1993705165 <== mon.2 v2:172.21.15.76:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 8586+0+0 (secure 0 0 0) 0x7fb038100bc0 con 0x7fb0401091f0 2024-09-11T23:38:25.568 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.567+0000 7fb049c8f740 1 -- start start 2024-09-11T23:38:25.568 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.567+0000 7fb049c8f740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562c32c27af0 0x562c32c27ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:25.568 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.567+0000 7fb0453bb640 1 --2- 172.21.15.26:0/1993705165 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fb0280a2ef0 0x7fb0280a53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:25.569 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.567+0000 7fb049c8f740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562c32c28410 0x562c32c32770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:25.569 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.568+0000 7fb049c8f740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562c32c32cb0 0x562c32c350a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:25.569 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.568+0000 7fb049c8f740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x562c32b8c2f0 con 0x562c32c28410 2024-09-11T23:38:25.569 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.568+0000 7fb049c8f740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x562c32b34270 con 0x562c32c27af0 2024-09-11T23:38:25.569 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.568+0000 7fb049c8f740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x562c32b882b0 con 0x562c32c32cb0 2024-09-11T23:38:25.569 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.568+0000 7fb044bba640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562c32c27af0 0x562c32c27ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:25.569 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.568+0000 7fb044bba640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562c32c27af0 0x562c32c27ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:33402/0 (socket says 172.21.15.26:33402) 2024-09-11T23:38:25.569 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.568+0000 7fb044bba640 1 -- 172.21.15.26:0/4207531821 learned_addr learned my addr 172.21.15.26:0/4207531821 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:38:25.569 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.568+0000 7fb0453bb640 1 --2- 172.21.15.26:0/4207531821 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562c32c32cb0 0x562c32c350a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:25.570 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.569+0000 7fb044bba640 1 -- 172.21.15.26:0/4207531821 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562c32c32cb0 msgr2=0x562c32c350a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:38:25.570 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.569+0000 7fb044bba640 1 --2- 172.21.15.26:0/4207531821 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562c32c32cb0 0x562c32c350a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:25.570 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.569+0000 7fb044bba640 1 -- 172.21.15.26:0/4207531821 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562c32c28410 msgr2=0x562c32c32770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:38:25.570 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.569+0000 7fb044bba640 1 --2- 172.21.15.26:0/4207531821 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562c32c28410 0x562c32c32770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:25.570 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.569+0000 7fb044bba640 1 -- 172.21.15.26:0/4207531821 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c32b33340 con 0x562c32c27af0 2024-09-11T23:38:25.570 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.569+0000 7fb0453bb640 1 --2- 172.21.15.26:0/1993705165 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fb0280a2ef0 0x7fb0280a53b0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fb030009ba0 tx=0x7fb030012210 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:38:25.575 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.576+0000 7fb044bba640 1 --2- 172.21.15.26:0/4207531821 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562c32c27af0 0x562c32c27ed0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fb0380c9b20 tx=0x7fb0380ccd80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=44a4f226bf98ffa3 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:38:25.576 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.577+0000 7fb026ffd640 1 -- 172.21.15.26:0/4207531821 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0380e6070 con 0x562c32c27af0 2024-09-11T23:38:25.576 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.577+0000 7fb026ffd640 1 -- 172.21.15.26:0/4207531821 <== mon.2 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0380cd150 con 0x562c32c27af0 2024-09-11T23:38:25.577 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.578+0000 7fb049c8f740 1 -- 172.21.15.26:0/4207531821 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562c32c27af0 msgr2=0x562c32c27ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:25.577 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.578+0000 7fb049c8f740 1 --2- 172.21.15.26:0/4207531821 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562c32c27af0 0x562c32c27ed0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fb0380c9b20 tx=0x7fb0380ccd80 comp rx=0 tx=0).stop 2024-09-11T23:38:25.581 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.582+0000 7fb049c8f740 1 -- 172.21.15.26:0/4207531821 shutdown_connections 2024-09-11T23:38:25.581 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.582+0000 7fb049c8f740 1 --2- 172.21.15.26:0/4207531821 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562c32c32cb0 0x562c32c350a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:25.581 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.582+0000 7fb049c8f740 1 --2- 172.21.15.26:0/4207531821 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562c32c28410 0x562c32c32770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:25.581 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.582+0000 7fb049c8f740 1 --2- 172.21.15.26:0/4207531821 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562c32c27af0 0x562c32c27ed0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:25.582 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.582+0000 7fb049c8f740 1 -- 172.21.15.26:0/4207531821 >> 172.21.15.26:0/4207531821 conn(0x562c32c24060 msgr2=0x562c32c24440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:38:25.589 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.589+0000 7fb049c8f740 1 -- 172.21.15.26:0/4207531821 shutdown_connections 2024-09-11T23:38:25.589 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.589+0000 7fb049c8f740 1 -- 172.21.15.26:0/4207531821 wait complete. 2024-09-11T23:38:25.589 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.590+0000 7fb049c8f740 1 -- start start 2024-09-11T23:38:25.590 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.590+0000 7fb049c8f740 1 Processor -- start 2024-09-11T23:38:25.590 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.591+0000 7fb049c8f740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562c32c27af0 0x562c32c2fb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:25.591 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.591+0000 7fb049c8f740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562c32c28410 0x562c32c30090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:25.591 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.591+0000 7fb049c8f740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562c32c32cb0 0x562c32c305d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:25.591 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.591+0000 7fb049c8f740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x562c32b34270 con 0x562c32c27af0 2024-09-11T23:38:25.591 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.591+0000 7fb049c8f740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x562c32b15290 con 0x562c32c28410 2024-09-11T23:38:25.591 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.591+0000 7fb049c8f740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x562c32c35640 con 0x562c32c32cb0 2024-09-11T23:38:25.591 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.591+0000 7fb049c8f740 10 client.0 _ll_register_callbacks cb 0x562c32b46140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:38:25.591 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.592+0000 7fb03ffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562c32c28410 0x562c32c30090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:25.592 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.592+0000 7fb03ffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562c32c28410 0x562c32c30090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:33412/0 (socket says 172.21.15.26:33412) 2024-09-11T23:38:25.592 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.592+0000 7fb03ffff640 1 -- 172.21.15.26:0/446069499 learned_addr learned my addr 172.21.15.26:0/446069499 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:38:25.592 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.592+0000 7fb0453bb640 1 --2- 172.21.15.26:0/446069499 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562c32c32cb0 0x562c32c305d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:25.592 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.592+0000 7fb044bba640 1 --2- 172.21.15.26:0/446069499 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562c32c27af0 0x562c32c2fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:25.593 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.593+0000 7fb03ffff640 1 -- 172.21.15.26:0/446069499 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562c32c32cb0 msgr2=0x562c32c305d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:38:25.593 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.593+0000 7fb03ffff640 1 --2- 172.21.15.26:0/446069499 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562c32c32cb0 0x562c32c305d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:25.593 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.593+0000 7fb03ffff640 1 -- 172.21.15.26:0/446069499 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562c32c27af0 msgr2=0x562c32c2fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:38:25.593 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.593+0000 7fb03ffff640 1 --2- 172.21.15.26:0/446069499 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562c32c27af0 0x562c32c2fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:25.593 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.593+0000 7fb03ffff640 1 -- 172.21.15.26:0/446069499 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c3289f210 con 0x562c32c28410 2024-09-11T23:38:25.593 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.594+0000 7fb03ffff640 1 --2- 172.21.15.26:0/446069499 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562c32c28410 0x562c32c30090 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fb034009c90 tx=0x7fb03400f730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=80536be6485631a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:38:25.597 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.597+0000 7fb01ffff640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-11T23:38:25.597 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.597+0000 7fb01ffff640 1 -- 172.21.15.26:0/446069499 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb03401a680 con 0x562c32c28410 2024-09-11T23:38:25.597 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.597+0000 7fb01ffff640 1 -- 172.21.15.26:0/446069499 <== mon.2 v2:172.21.15.76:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 8586+0+0 (secure 0 0 0) 0x7fb034011440 con 0x562c32c28410 2024-09-11T23:38:25.597 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.597+0000 7fb049c8f740 1 -- 172.21.15.26:0/446069499 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb0380cd150 con 0x562c32c28410 2024-09-11T23:38:25.605 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.605+0000 7fb01ffff640 1 -- 172.21.15.26:0/446069499 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7fb03401a820 con 0x562c32c28410 2024-09-11T23:38:25.605 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.605+0000 7fb049c8f740 10 client.27491 fetch_fsmap learned FSMap version 87 2024-09-11T23:38:25.605 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.605+0000 7fb049c8f740 1 -- 172.21.15.26:0/446069499 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=87,monmap=4+}) -- 0x562c3296b620 con 0x562c32c28410 2024-09-11T23:38:25.608 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.608+0000 7fb01ffff640 1 -- 172.21.15.26:0/446069499 <== mon.2 v2:172.21.15.76:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0380cd150 con 0x562c32c28410 2024-09-11T23:38:25.608 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.608+0000 7fb01ffff640 1 -- 172.21.15.26:0/446069499 <== mon.2 v2:172.21.15.76:3300/0 5 ==== fsmap(e 87) ==== 8430+0+0 (secure 0 0 0) 0x7fb034024420 con 0x562c32c28410 2024-09-11T23:38:25.608 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.609+0000 7fb049c8f740 10 client.27491 fetch_fsmap finished waiting for FSMap version 87 2024-09-11T23:38:25.608 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.609+0000 7fb049c8f740 10 client.27491 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:38:25.608 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.609+0000 7fb049c8f740 20 client.27491 populate_metadata read hostname 'smithi026' 2024-09-11T23:38:25.608 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.609+0000 7fb049c8f740 1 --2- 172.21.15.26:0/446069499 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x562c32c42990 0x562c32c44dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:25.608 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.609+0000 7fb049c8f740 4 client.27491 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:38:25.609 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.609+0000 7fb049c8f740 1 -- 172.21.15.26:0/446069499 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562c32b99a20 con 0x562c32c42990 2024-09-11T23:38:25.610 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.611+0000 7fb044bba640 1 --2- 172.21.15.26:0/446069499 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x562c32c42990 0x562c32c44dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:25.613 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.613+0000 7fb044bba640 1 --2- 172.21.15.26:0/446069499 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x562c32c42990 0x562c32c44dc0 crc :-1 s=READY pgs=298 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=530b6446d5a0447 server_cookie=1986cfbc5d45ddd3 in_seq=0 out_seq=0 2024-09-11T23:38:25.615 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.616+0000 7fb01ffff640 10 client.27491 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:38:25.615 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.616+0000 7fb01ffff640 1 -- 172.21.15.26:0/446069499 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562c32b99a20 con 0x562c32c42990 2024-09-11T23:38:25.615 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.616+0000 7fb01ffff640 10 client.27491 handle_command_reply: tid=0 2024-09-11T23:38:25.616 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.616+0000 7fb049c8f740 1 client.27491 shutdown 2024-09-11T23:38:25.617 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.618+0000 7fb049c8f740 1 -- 172.21.15.26:0/446069499 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562c32c28410 msgr2=0x562c32c30090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:25.617 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.618+0000 7fb049c8f740 1 --2- 172.21.15.26:0/446069499 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562c32c28410 0x562c32c30090 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fb034009c90 tx=0x7fb03400f730 comp rx=0 tx=0).stop 2024-09-11T23:38:25.618 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.618+0000 7fb049c8f740 1 -- 172.21.15.26:0/446069499 shutdown_connections 2024-09-11T23:38:25.618 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.618+0000 7fb049c8f740 1 --2- 172.21.15.26:0/446069499 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562c32c28410 0x562c32c30090 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:25.618 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.618+0000 7fb049c8f740 1 --2- 172.21.15.26:0/446069499 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x562c32c42990 0x562c32c44dc0 crc :-1 s=READY pgs=298 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:25.618 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.618+0000 7fb049c8f740 1 --2- 172.21.15.26:0/446069499 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562c32c32cb0 0x562c32c305d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:25.618 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.618+0000 7fb049c8f740 1 --2- 172.21.15.26:0/446069499 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562c32c27af0 0x562c32c2fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:25.618 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.618+0000 7fb049c8f740 1 -- 172.21.15.26:0/446069499 >> 172.21.15.26:0/446069499 conn(0x562c32c24060 msgr2=0x562c32c29dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:38:25.618 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.618+0000 7fb01ffff640 0 client.27491 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:38:25.618 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.618+0000 7fb049c8f740 1 -- 172.21.15.26:0/446069499 shutdown_connections 2024-09-11T23:38:25.618 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.618+0000 7fb049c8f740 1 -- 172.21.15.26:0/446069499 wait complete. 2024-09-11T23:38:25.618 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.619+0000 7fb049c8f740 20 client.27491 trim_cache size 0 max 16384 2024-09-11T23:38:25.648 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.649+0000 7fb049c8f740 1 Processor -- start 2024-09-11T23:38:25.650 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.650+0000 7fb049c8f740 1 -- start start 2024-09-11T23:38:25.650 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.650+0000 7fb049c8f740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562c32c27af0 0x562c32c30400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:25.650 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.650+0000 7fb049c8f740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562c32c28410 0x562c32c4a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:25.650 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.650+0000 7fb049c8f740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562c32c32cb0 0x562c32c4ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:25.650 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.650+0000 7fb049c8f740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb02c00e9e0 con 0x562c32c32cb0 2024-09-11T23:38:25.650 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.650+0000 7fb049c8f740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fb02c0107f0 con 0x562c32c27af0 2024-09-11T23:38:25.650 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.650+0000 7fb049c8f740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb02c00f140 con 0x562c32c28410 2024-09-11T23:38:25.651 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.651+0000 7fb044bba640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562c32c27af0 0x562c32c30400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:25.651 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.651+0000 7fb044bba640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562c32c27af0 0x562c32c30400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:33424/0 (socket says 172.21.15.26:33424) 2024-09-11T23:38:25.651 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.651+0000 7fb044bba640 1 -- 172.21.15.26:0/481111071 learned_addr learned my addr 172.21.15.26:0/481111071 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:38:25.651 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.651+0000 7fb044bba640 1 -- 172.21.15.26:0/481111071 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562c32c28410 msgr2=0x562c32c4a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:38:25.651 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.651+0000 7fb044bba640 1 --2- 172.21.15.26:0/481111071 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562c32c28410 0x562c32c4a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:25.651 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.651+0000 7fb044bba640 1 -- 172.21.15.26:0/481111071 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562c32c32cb0 msgr2=0x562c32c4ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:38:25.651 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.651+0000 7fb044bba640 1 --2- 172.21.15.26:0/481111071 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562c32c32cb0 0x562c32c4ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:25.651 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.651+0000 7fb044bba640 1 -- 172.21.15.26:0/481111071 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c32c23e90 con 0x562c32c27af0 2024-09-11T23:38:25.652 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.651+0000 7fb044bba640 1 --2- 172.21.15.26:0/481111071 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562c32c27af0 0x562c32c30400 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fb038005e30 tx=0x7fb0380c94e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc0616e7b77ec93e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:38:25.652 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.652+0000 7fb0267fc640 1 -- 172.21.15.26:0/481111071 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0380e6040 con 0x562c32c27af0 2024-09-11T23:38:25.652 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.652+0000 7fb0267fc640 1 -- 172.21.15.26:0/481111071 <== mon.2 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0380cd150 con 0x562c32c27af0 2024-09-11T23:38:25.652 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.653+0000 7fb0267fc640 1 -- 172.21.15.26:0/481111071 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0380ed0c0 con 0x562c32c27af0 2024-09-11T23:38:25.652 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.653+0000 7fb049c8f740 1 -- 172.21.15.26:0/481111071 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562c32c27af0 msgr2=0x562c32c30400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:25.652 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.653+0000 7fb049c8f740 1 --2- 172.21.15.26:0/481111071 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562c32c27af0 0x562c32c30400 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fb038005e30 tx=0x7fb0380c94e0 comp rx=0 tx=0).stop 2024-09-11T23:38:25.658 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.659+0000 7fb049c8f740 1 -- 172.21.15.26:0/481111071 shutdown_connections 2024-09-11T23:38:25.658 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.659+0000 7fb049c8f740 1 --2- 172.21.15.26:0/481111071 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562c32c32cb0 0x562c32c4ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:25.658 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.659+0000 7fb049c8f740 1 --2- 172.21.15.26:0/481111071 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562c32c28410 0x562c32c4a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:25.658 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.659+0000 7fb049c8f740 1 --2- 172.21.15.26:0/481111071 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562c32c27af0 0x562c32c30400 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:25.658 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.659+0000 7fb049c8f740 1 -- 172.21.15.26:0/481111071 >> 172.21.15.26:0/481111071 conn(0x562c32c24060 msgr2=0x562c32c31bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:38:25.659 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.659+0000 7fb049c8f740 1 -- 172.21.15.26:0/481111071 shutdown_connections 2024-09-11T23:38:25.659 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.659+0000 7fb049c8f740 1 -- 172.21.15.26:0/481111071 wait complete. 2024-09-11T23:38:25.659 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.660+0000 7fb049c8f740 1 -- start start 2024-09-11T23:38:25.659 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.660+0000 7fb049c8f740 1 Processor -- start 2024-09-11T23:38:25.660 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.661+0000 7fb049c8f740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562c32c27af0 0x562c32c3abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:25.660 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.661+0000 7fb049c8f740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562c32c28410 0x562c32c3b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:25.661 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.661+0000 7fb049c8f740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562c32c32cb0 0x562c32c2c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:25.661 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.661+0000 7fb049c8f740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb02c010060 con 0x562c32c28410 2024-09-11T23:38:25.661 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.661+0000 7fb049c8f740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fb02c00f8d0 con 0x562c32c32cb0 2024-09-11T23:38:25.661 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.661+0000 7fb049c8f740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb02c00c9e0 con 0x562c32c27af0 2024-09-11T23:38:25.661 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.661+0000 7fb049c8f740 10 client.0 _ll_register_callbacks cb 0x7fb0340093b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:38:25.664 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.662+0000 7fb03ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562c32c28410 0x562c32c3b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:25.664 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.662+0000 7fb03ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562c32c28410 0x562c32c3b110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50024/0 (socket says 172.21.15.26:50024) 2024-09-11T23:38:25.664 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.662+0000 7fb03ffff640 1 -- 172.21.15.26:0/2177773074 learned_addr learned my addr 172.21.15.26:0/2177773074 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:38:25.664 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.663+0000 7fb03ffff640 1 -- 172.21.15.26:0/2177773074 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562c32c27af0 msgr2=0x562c32c3abd0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T23:38:25.664 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.663+0000 7fb03ffff640 1 --2- 172.21.15.26:0/2177773074 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562c32c27af0 0x562c32c3abd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:25.664 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.663+0000 7fb03ffff640 1 -- 172.21.15.26:0/2177773074 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562c32c32cb0 msgr2=0x562c32c2c4b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T23:38:25.664 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.663+0000 7fb03ffff640 1 --2- 172.21.15.26:0/2177773074 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562c32c32cb0 0x562c32c2c4b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:25.665 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.663+0000 7fb03ffff640 1 -- 172.21.15.26:0/2177773074 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c32c2ccb0 con 0x562c32c28410 2024-09-11T23:38:25.665 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.663+0000 7fb03ffff640 1 --2- 172.21.15.26:0/2177773074 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562c32c28410 0x562c32c3b110 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7fb034001eb0 tx=0x7fb034009910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d18d72eb74b52203 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:38:25.665 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.663+0000 7fb01ffff640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:38:25.665 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.664+0000 7fb01ffff640 1 -- 172.21.15.26:0/2177773074 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb034029070 con 0x562c32c28410 2024-09-11T23:38:25.665 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.664+0000 7fb01ffff640 1 -- 172.21.15.26:0/2177773074 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 8586+0+0 (secure 0 0 0) 0x7fb03401a9c0 con 0x562c32c28410 2024-09-11T23:38:25.665 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.665+0000 7fb049c8f740 1 -- 172.21.15.26:0/2177773074 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb0380cd150 con 0x562c32c28410 2024-09-11T23:38:25.665 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.665+0000 7fb01ffff640 1 -- 172.21.15.26:0/2177773074 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7fb034024c30 con 0x562c32c28410 2024-09-11T23:38:25.671 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.669+0000 7fb049c8f740 10 client.17979 fetch_fsmap learned FSMap version 87 2024-09-11T23:38:25.671 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.669+0000 7fb049c8f740 1 -- 172.21.15.26:0/2177773074 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=87,monmap=4+}) -- 0x562c32cad290 con 0x562c32c28410 2024-09-11T23:38:25.671 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.671+0000 7fb01ffff640 1 -- 172.21.15.26:0/2177773074 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0380cd150 con 0x562c32c28410 2024-09-11T23:38:25.671 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.671+0000 7fb01ffff640 1 -- 172.21.15.26:0/2177773074 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 87) ==== 8430+0+0 (secure 0 0 0) 0x7fb0340037b0 con 0x562c32c28410 2024-09-11T23:38:25.671 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.671+0000 7fb049c8f740 10 client.17979 fetch_fsmap finished waiting for FSMap version 87 2024-09-11T23:38:25.671 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.671+0000 7fb049c8f740 10 client.17979 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:38:25.671 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.671+0000 7fb049c8f740 20 client.17979 populate_metadata read hostname 'smithi026' 2024-09-11T23:38:25.672 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.671+0000 7fb049c8f740 1 --2- 172.21.15.26:0/2177773074 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x562c32cada80 0x562c32cafea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:25.672 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.671+0000 7fb049c8f740 4 client.17979 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T23:38:25.672 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.671+0000 7fb049c8f740 1 -- 172.21.15.26:0/2177773074 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562c32b93d70 con 0x562c32cada80 2024-09-11T23:38:25.672 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.672+0000 7fb044bba640 1 --2- 172.21.15.26:0/2177773074 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x562c32cada80 0x562c32cafea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:25.673 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.673+0000 7fb01ffff640 10 client.17979 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:38:25.673 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.674+0000 7fb044bba640 1 --2- 172.21.15.26:0/2177773074 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x562c32cada80 0x562c32cafea0 crc :-1 s=READY pgs=299 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=823bf5e5d42bd5f8 server_cookie=6a3dd034d76b038b in_seq=0 out_seq=0 2024-09-11T23:38:25.735 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.735+0000 7fb01ffff640 1 -- 172.21.15.26:0/2177773074 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562c32b93d70 con 0x562c32cada80 2024-09-11T23:38:25.735 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.735+0000 7fb01ffff640 10 client.17979 handle_command_reply: tid=0 2024-09-11T23:38:25.735 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.735+0000 7fb049c8f740 1 client.17979 shutdown 2024-09-11T23:38:25.739 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.740+0000 7fb049c8f740 1 -- 172.21.15.26:0/2177773074 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562c32c28410 msgr2=0x562c32c3b110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:25.739 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.740+0000 7fb049c8f740 1 --2- 172.21.15.26:0/2177773074 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562c32c28410 0x562c32c3b110 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7fb034001eb0 tx=0x7fb034009910 comp rx=0 tx=0).stop 2024-09-11T23:38:25.741 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.742+0000 7fb049c8f740 1 -- 172.21.15.26:0/2177773074 shutdown_connections 2024-09-11T23:38:25.741 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.742+0000 7fb049c8f740 1 --2- 172.21.15.26:0/2177773074 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562c32c32cb0 0x562c32c2c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:25.741 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.742+0000 7fb049c8f740 1 --2- 172.21.15.26:0/2177773074 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x562c32cada80 0x562c32cafea0 crc :-1 s=READY pgs=299 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:25.741 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.742+0000 7fb049c8f740 1 --2- 172.21.15.26:0/2177773074 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562c32c28410 0x562c32c3b110 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:25.741 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.742+0000 7fb049c8f740 1 --2- 172.21.15.26:0/2177773074 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562c32c27af0 0x562c32c3abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:25.741 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.742+0000 7fb049c8f740 1 -- 172.21.15.26:0/2177773074 >> 172.21.15.26:0/2177773074 conn(0x562c32c24060 msgr2=0x562c32c29550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:38:25.741 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.742+0000 7fb01ffff640 0 client.17979 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:38:25.742 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.742+0000 7fb049c8f740 1 -- 172.21.15.26:0/2177773074 shutdown_connections 2024-09-11T23:38:25.742 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.742+0000 7fb049c8f740 1 -- 172.21.15.26:0/2177773074 wait complete. 2024-09-11T23:38:25.742 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.742+0000 7fb049c8f740 20 client.17979 trim_cache size 0 max 16384 2024-09-11T23:38:25.744 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-09-11T23:38:25.744 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "message": "", 2024-09-11T23:38:25.744 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-09-11T23:38:25.744 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-09-11T23:38:25.745 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.746+0000 7fb046e1e640 1 -- 172.21.15.26:0/1993705165 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fb0280a2ef0 msgr2=0x7fb0280a53b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:25.745 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.746+0000 7fb046e1e640 1 --2- 172.21.15.26:0/1993705165 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fb0280a2ef0 0x7fb0280a53b0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fb030009ba0 tx=0x7fb030012210 comp rx=0 tx=0).stop 2024-09-11T23:38:25.745 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.746+0000 7fb046e1e640 1 -- 172.21.15.26:0/1993705165 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb0401091f0 msgr2=0x7fb04011a820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:25.745 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.746+0000 7fb046e1e640 1 --2- 172.21.15.26:0/1993705165 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb0401091f0 0x7fb04011a820 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fb038000f80 tx=0x7fb038041a20 comp rx=0 tx=0).stop 2024-09-11T23:38:25.746 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.747+0000 7fb046e1e640 1 -- 172.21.15.26:0/1993705165 shutdown_connections 2024-09-11T23:38:25.746 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.747+0000 7fb046e1e640 1 --2- 172.21.15.26:0/1993705165 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fb0280a2ef0 0x7fb0280a53b0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:25.746 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.747+0000 7fb046e1e640 1 --2- 172.21.15.26:0/1993705165 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb04011b140 0x7fb040116df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:25.746 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.747+0000 7fb046e1e640 1 --2- 172.21.15.26:0/1993705165 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb0401095d0 0x7fb0401168b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:25.746 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.747+0000 7fb046e1e640 1 --2- 172.21.15.26:0/1993705165 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb0401091f0 0x7fb04011a820 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:25.746 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.747+0000 7fb046e1e640 1 -- 172.21.15.26:0/1993705165 >> 172.21.15.26:0/1993705165 conn(0x7fb04006ce00 msgr2=0x7fb0400707f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:38:25.750 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.751+0000 7fb046e1e640 1 -- 172.21.15.26:0/1993705165 shutdown_connections 2024-09-11T23:38:25.750 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:25.751+0000 7fb046e1e640 1 -- 172.21.15.26:0/1993705165 wait complete. 2024-09-11T23:38:26.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:26.156+0000 7f3ed3fff640 1 -- 172.21.15.26:0/3935545831 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3ed41019d0 msgr2=0x7f3ed4101e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:26.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:26.156+0000 7f3ed3fff640 1 --2- 172.21.15.26:0/3935545831 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3ed41019d0 0x7f3ed4101e30 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f3ec8009930 tx=0x7f3ec802a480 comp rx=0 tx=0).stop 2024-09-11T23:38:26.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:26.157+0000 7f3ed3fff640 1 -- 172.21.15.26:0/3935545831 shutdown_connections 2024-09-11T23:38:26.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:26.157+0000 7f3ed3fff640 1 --2- 172.21.15.26:0/3935545831 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3ed4102370 0x7f3ed410e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:26.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:26.157+0000 7f3ed3fff640 1 --2- 172.21.15.26:0/3935545831 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3ed41019d0 0x7f3ed4101e30 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:26.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:26.157+0000 7f3ed3fff640 1 --2- 172.21.15.26:0/3935545831 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3ed41079d0 0x7f3ed4107db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:26.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:26.157+0000 7f3ed3fff640 1 -- 172.21.15.26:0/3935545831 >> 172.21.15.26:0/3935545831 conn(0x7f3ed40fd710 msgr2=0x7f3ed40ffb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:38:26.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:26.163+0000 7f3ed3fff640 1 -- 172.21.15.26:0/3935545831 shutdown_connections 2024-09-11T23:38:26.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:26.163+0000 7f3ed3fff640 1 -- 172.21.15.26:0/3935545831 wait complete. 2024-09-11T23:38:26.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:26.164+0000 7f3ed3fff640 1 Processor -- start 2024-09-11T23:38:26.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:26.168+0000 7f3ed3fff640 1 -- start start 2024-09-11T23:38:26.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:26.168+0000 7f3ed3fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3ed41019d0 0x7f3ed419d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:26.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:26.168+0000 7f3ed3fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3ed4102370 0x7f3ed41960d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:26.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:26.168+0000 7f3ed2ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3ed41019d0 0x7f3ed419d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:26.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:26.169+0000 7f3ed2ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3ed41019d0 0x7f3ed419d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:33452/0 (socket says 172.21.15.26:33452) 2024-09-11T23:38:26.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:26.168+0000 7f3ed3fff640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3ed41079d0 0x7f3ed4196610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:26.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:26.169+0000 7f3ed2ffd640 1 -- 172.21.15.26:0/1339285689 learned_addr learned my addr 172.21.15.26:0/1339285689 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:38:26.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:26.169+0000 7f3ed3fff640 1 -- 172.21.15.26:0/1339285689 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f3ed410fcf0 con 0x7f3ed4102370 2024-09-11T23:38:26.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:26.169+0000 7f3ed3fff640 1 -- 172.21.15.26:0/1339285689 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f3ed410fb70 con 0x7f3ed41019d0 2024-09-11T23:38:26.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:26.169+0000 7f3ed3fff640 1 -- 172.21.15.26:0/1339285689 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f3ed410fe70 con 0x7f3ed41079d0 2024-09-11T23:38:26.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:26.169+0000 7f3ed27fc640 1 --2- 172.21.15.26:0/1339285689 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3ed4102370 0x7f3ed41960d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:26.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:26.172+0000 7f3ed37fe640 1 --2- 172.21.15.26:0/1339285689 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3ed41079d0 0x7f3ed4196610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:26.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:26.173+0000 7f3ed2ffd640 1 -- 172.21.15.26:0/1339285689 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3ed41079d0 msgr2=0x7f3ed4196610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:26.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:26.173+0000 7f3ed2ffd640 1 --2- 172.21.15.26:0/1339285689 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3ed41079d0 0x7f3ed4196610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:26.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:26.173+0000 7f3ed2ffd640 1 -- 172.21.15.26:0/1339285689 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3ed4102370 msgr2=0x7f3ed41960d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:26.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:26.173+0000 7f3ed2ffd640 1 --2- 172.21.15.26:0/1339285689 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3ed4102370 0x7f3ed41960d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:26.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:26.173+0000 7f3ed2ffd640 1 -- 172.21.15.26:0/1339285689 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ed4196c80 con 0x7f3ed41019d0 2024-09-11T23:38:26.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:26.173+0000 7f3ed2ffd640 1 --2- 172.21.15.26:0/1339285689 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3ed41019d0 0x7f3ed419d040 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f3ec401c560 tx=0x7f3ec401ca20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:38:26.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:26.176+0000 7f3ebbfff640 1 -- 172.21.15.26:0/1339285689 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ec4020860 con 0x7f3ed41019d0 2024-09-11T23:38:26.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:26.176+0000 7f3ebbfff640 1 -- 172.21.15.26:0/1339285689 <== mon.2 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ec4028400 con 0x7f3ed41019d0 2024-09-11T23:38:26.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:26.176+0000 7f3ed3fff640 1 -- 172.21.15.26:0/1339285689 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ed419ab70 con 0x7f3ed41019d0 2024-09-11T23:38:26.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:26.176+0000 7f3ed3fff640 1 -- 172.21.15.26:0/1339285689 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ed419b080 con 0x7f3ed41019d0 2024-09-11T23:38:26.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:26.178+0000 7f3ebbfff640 1 -- 172.21.15.26:0/1339285689 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ec4020c00 con 0x7f3ed41019d0 2024-09-11T23:38:26.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:26.180+0000 7f3ed3fff640 1 -- 172.21.15.26:0/1339285689 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ed4196f10 con 0x7f3ed41019d0 2024-09-11T23:38:26.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:26.182+0000 7f3ebbfff640 1 -- 172.21.15.26:0/1339285689 <== mon.2 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f3ec401e040 con 0x7f3ed41019d0 2024-09-11T23:38:26.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:26.183+0000 7f3ebbfff640 1 --2- 172.21.15.26:0/1339285689 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f3ebc0a2ea0 0x7f3ebc0a5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:26.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:26.183+0000 7f3ebbfff640 1 -- 172.21.15.26:0/1339285689 <== mon.2 v2:172.21.15.76:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 8586+0+0 (secure 0 0 0) 0x7f3ec40dff20 con 0x7f3ed41019d0 2024-09-11T23:38:26.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:26.183+0000 7f3ed27fc640 1 --2- 172.21.15.26:0/1339285689 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f3ebc0a2ea0 0x7f3ebc0a5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:26.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:26.184+0000 7f3ed27fc640 1 --2- 172.21.15.26:0/1339285689 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f3ebc0a2ea0 0x7f3ebc0a5360 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f3ec8005ab0 tx=0x7f3ec802a980 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:38:26.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:26.191+0000 7f3ebbfff640 1 -- 172.21.15.26:0/1339285689 <== mon.2 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3ec402d060 con 0x7f3ed41019d0 2024-09-11T23:38:26.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:38:25 smithi136 ceph-mon[35121]: pgmap v631: 97 pgs: 23 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 33 active+clean; 2.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 19 KiB/s wr, 27 op/s 2024-09-11T23:38:26.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:38:25 smithi026 ceph-mon[31172]: pgmap v631: 97 pgs: 23 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 33 active+clean; 2.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 19 KiB/s wr, 27 op/s 2024-09-11T23:38:26.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:38:25 smithi076 ceph-mon[35296]: pgmap v631: 97 pgs: 23 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 33 active+clean; 2.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 19 KiB/s wr, 27 op/s 2024-09-11T23:38:26.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:26.600+0000 7f3ed3fff640 1 -- 172.21.15.26:0/1339285689 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3ed4197600 con 0x7f3ed41019d0 2024-09-11T23:38:26.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:26.601+0000 7f3ebbfff640 1 -- 172.21.15.26:0/1339285689 <== mon.2 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 87 v87) ==== 94+0+12001 (secure 0 0 0) 0x7f3ec4024030 con 0x7f3ed41019d0 2024-09-11T23:38:26.602 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-11T23:38:26.604 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":87,"btime":"2024-09-11T23:38:04:200954+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14499,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/367659515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":367659515},{"type":"v1","addr":"172.21.15.26:6835","nonce":367659515}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14511,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/2041065299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":2041065299},{"type":"v1","addr":"172.21.15.26:6837","nonce":2041065299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":87,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:16:26.544203+0000","modified":"2024-09-11T23:38:03.269999+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24485,"mds_1":24463,"mds_2":24458,"mds_3":24448,"mds_4":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14550":{"gid":14550,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6839/2233068305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":2233068305},{"type":"v1","addr":"172.21.15.26:6839","nonce":2233068305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14562":{"gid":14562,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6841/2984982336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":2984982336},{"type":"v1","addr":"172.21.15.26:6841","nonce":2984982336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6833/2019841455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":2019841455},{"type":"v1","addr":"172.21.15.136:6833","nonce":2019841455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.76:6833/3438147050","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3438147050},{"type":"v1","addr":"172.21.15.76:6833","nonce":3438147050}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6835/3617665975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3617665975},{"type":"v1","addr":"172.21.15.136:6835","nonce":3617665975}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":12,"addr":"172.21.15.76:6835/393308499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":393308499},{"type":"v1","addr":"172.21.15.76:6835","nonce":393308499}]},"join_fscid":-1,"export_targets":[0,1,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":10,"addr":"172.21.15.136:6837/1633082387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":1633082387},{"type":"v1","addr":"172.21.15.136:6837","nonce":1633082387}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.76:6837/2078978748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":2078978748},{"type":"v1","addr":"172.21.15.76:6837","nonce":2078978748}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":63,"addr":"172.21.15.136:6839/3211398630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":3211398630},{"type":"v1","addr":"172.21.15.136:6839","nonce":3211398630}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.76:6839/2077637090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6838","nonce":2077637090},{"type":"v1","addr":"172.21.15.76:6839","nonce":2077637090}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24485,"qdb_cluster":[24443,24448,24458,24463,24485]},"id":1}]} 2024-09-11T23:38:26.605 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 87 2024-09-11T23:38:26.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:26.604+0000 7f3ed3fff640 1 -- 172.21.15.26:0/1339285689 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f3ebc0a2ea0 msgr2=0x7f3ebc0a5360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:26.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:26.604+0000 7f3ed3fff640 1 --2- 172.21.15.26:0/1339285689 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f3ebc0a2ea0 0x7f3ebc0a5360 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f3ec8005ab0 tx=0x7f3ec802a980 comp rx=0 tx=0).stop 2024-09-11T23:38:26.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:26.604+0000 7f3ed3fff640 1 -- 172.21.15.26:0/1339285689 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3ed41019d0 msgr2=0x7f3ed419d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:26.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:26.604+0000 7f3ed3fff640 1 --2- 172.21.15.26:0/1339285689 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3ed41019d0 0x7f3ed419d040 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f3ec401c560 tx=0x7f3ec401ca20 comp rx=0 tx=0).stop 2024-09-11T23:38:26.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:26.605+0000 7f3ed3fff640 1 -- 172.21.15.26:0/1339285689 shutdown_connections 2024-09-11T23:38:26.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:26.605+0000 7f3ed3fff640 1 --2- 172.21.15.26:0/1339285689 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f3ebc0a2ea0 0x7f3ebc0a5360 secure :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f3ec8005ab0 tx=0x7f3ec802a980 comp rx=0 tx=0).stop 2024-09-11T23:38:26.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:26.605+0000 7f3ed3fff640 1 --2- 172.21.15.26:0/1339285689 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3ed41079d0 0x7f3ed4196610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:26.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:26.605+0000 7f3ed3fff640 1 --2- 172.21.15.26:0/1339285689 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3ed4102370 0x7f3ed41960d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:26.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:26.605+0000 7f3ed3fff640 1 --2- 172.21.15.26:0/1339285689 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3ed41019d0 0x7f3ed419d040 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:26.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:26.605+0000 7f3ed3fff640 1 -- 172.21.15.26:0/1339285689 >> 172.21.15.26:0/1339285689 conn(0x7f3ed40fd710 msgr2=0x7f3ed40fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:38:26.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:26.612+0000 7f3ed3fff640 1 -- 172.21.15.26:0/1339285689 shutdown_connections 2024-09-11T23:38:26.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:26.614+0000 7f3ed3fff640 1 -- 172.21.15.26:0/1339285689 wait complete. 2024-09-11T23:38:27.238 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:38:26 smithi136 ceph-mon[35121]: from='client.? 172.21.15.26:0/1339285689' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:38:27.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:38:26 smithi026 ceph-mon[31172]: from='client.? 172.21.15.26:0/1339285689' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:38:27.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:38:26 smithi076 ceph-mon[35296]: from='client.? 172.21.15.26:0/1339285689' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:38:27.468 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24485 quiesce db --set-id 58f7281f --release --await' 2024-09-11T23:38:27.468 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24485 quiesce db --set-id 58f7281f --release --await 2024-09-11T23:38:27.984 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:38:28.238 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:38:27 smithi136 ceph-mon[35121]: pgmap v632: 97 pgs: 22 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 34 active+clean; 2.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 19 KiB/s wr, 26 op/s 2024-09-11T23:38:28.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:38:27 smithi026 ceph-mon[31172]: pgmap v632: 97 pgs: 22 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 34 active+clean; 2.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 19 KiB/s wr, 26 op/s 2024-09-11T23:38:28.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:38:27 smithi076 ceph-mon[35296]: pgmap v632: 97 pgs: 22 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 34 active+clean; 2.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 19 KiB/s wr, 26 op/s 2024-09-11T23:38:28.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:28.901+0000 7f5511b16640 1 -- 172.21.15.26:0/673345378 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f550c101070 msgr2=0x7f550c1014b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:28.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:28.901+0000 7f550b7fe640 1 -- 172.21.15.26:0/673345378 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5500035020 con 0x7f550c101070 2024-09-11T23:38:28.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:28.901+0000 7f5511b16640 1 --2- 172.21.15.26:0/673345378 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f550c101070 0x7f550c1014b0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f5500009930 tx=0x7f550002a440 comp rx=0 tx=0).stop 2024-09-11T23:38:28.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:28.901+0000 7f5511b16640 1 -- 172.21.15.26:0/673345378 shutdown_connections 2024-09-11T23:38:28.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:28.902+0000 7f5511b16640 1 --2- 172.21.15.26:0/673345378 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f550c1019f0 0x7f550c10d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:28.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:28.902+0000 7f5511b16640 1 --2- 172.21.15.26:0/673345378 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f550c101070 0x7f550c1014b0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:28.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:28.902+0000 7f5511b16640 1 --2- 172.21.15.26:0/673345378 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f550c1058c0 0x7f550c105ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:28.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:28.902+0000 7f5511b16640 1 -- 172.21.15.26:0/673345378 >> 172.21.15.26:0/673345378 conn(0x7f550c077070 msgr2=0x7f550c0fe480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:38:28.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:28.904+0000 7f5511b16640 1 -- 172.21.15.26:0/673345378 shutdown_connections 2024-09-11T23:38:28.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:28.904+0000 7f5511b16640 1 -- 172.21.15.26:0/673345378 wait complete. 2024-09-11T23:38:28.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:28.905+0000 7f5511b16640 1 Processor -- start 2024-09-11T23:38:28.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:28.905+0000 7f5511b16640 1 -- start start 2024-09-11T23:38:28.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:28.905+0000 7f5511b16640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f550c101070 0x7f550c19a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:28.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:28.905+0000 7f5511b16640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f550c1019f0 0x7f550c19ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:28.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:28.905+0000 7f5510b14640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f550c101070 0x7f550c19a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:28.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:28.905+0000 7f5510b14640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f550c101070 0x7f550c19a6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50038/0 (socket says 172.21.15.26:50038) 2024-09-11T23:38:28.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:28.905+0000 7f5511b16640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f550c1058c0 0x7f550c19ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:28.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:28.905+0000 7f5510b14640 1 -- 172.21.15.26:0/1677218652 learned_addr learned my addr 172.21.15.26:0/1677218652 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:38:28.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:28.906+0000 7f5511b16640 1 -- 172.21.15.26:0/1677218652 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f550c10fdd0 con 0x7f550c101070 2024-09-11T23:38:28.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:28.906+0000 7f5511b16640 1 -- 172.21.15.26:0/1677218652 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f550c10fc50 con 0x7f550c1058c0 2024-09-11T23:38:28.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:28.906+0000 7f5511b16640 1 -- 172.21.15.26:0/1677218652 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f550c10ff50 con 0x7f550c1019f0 2024-09-11T23:38:28.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:28.906+0000 7f550bfff640 1 --2- 172.21.15.26:0/1677218652 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f550c1019f0 0x7f550c19ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:28.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:28.910+0000 7f5511315640 1 --2- 172.21.15.26:0/1677218652 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f550c1058c0 0x7f550c19ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:28.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:28.910+0000 7f5510b14640 1 -- 172.21.15.26:0/1677218652 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f550c1019f0 msgr2=0x7f550c19ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:28.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:28.910+0000 7f5510b14640 1 --2- 172.21.15.26:0/1677218652 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f550c1019f0 0x7f550c19ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:28.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:28.910+0000 7f5510b14640 1 -- 172.21.15.26:0/1677218652 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f550c1058c0 msgr2=0x7f550c19ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:28.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:28.910+0000 7f5510b14640 1 --2- 172.21.15.26:0/1677218652 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f550c1058c0 0x7f550c19ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:28.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:28.910+0000 7f5510b14640 1 -- 172.21.15.26:0/1677218652 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f550c19f670 con 0x7f550c101070 2024-09-11T23:38:28.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:28.913+0000 7f5510b14640 1 --2- 172.21.15.26:0/1677218652 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f550c101070 0x7f550c19a6c0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f550401ca70 tx=0x7f550401cf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:38:28.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:28.913+0000 7f5509ffb640 1 -- 172.21.15.26:0/1677218652 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f550402a070 con 0x7f550c101070 2024-09-11T23:38:28.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:28.914+0000 7f5509ffb640 1 -- 172.21.15.26:0/1677218652 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f550401dc10 con 0x7f550c101070 2024-09-11T23:38:28.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:28.914+0000 7f5511b16640 1 -- 172.21.15.26:0/1677218652 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f550c19f960 con 0x7f550c101070 2024-09-11T23:38:28.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:28.914+0000 7f5511b16640 1 -- 172.21.15.26:0/1677218652 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f550c06bc10 con 0x7f550c101070 2024-09-11T23:38:28.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:28.914+0000 7f5509ffb640 1 -- 172.21.15.26:0/1677218652 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55040076e0 con 0x7f550c101070 2024-09-11T23:38:28.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:28.920+0000 7f5509ffb640 1 -- 172.21.15.26:0/1677218652 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f550402d070 con 0x7f550c101070 2024-09-11T23:38:28.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:28.925+0000 7f5515151740 1 Processor -- start 2024-09-11T23:38:28.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:28.925+0000 7f5509ffb640 1 --2- 172.21.15.26:0/1677218652 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f54f40a2f00 0x7f54f40a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:28.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:28.926+0000 7f5515151740 1 -- start start 2024-09-11T23:38:28.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:28.926+0000 7f5515151740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562cfe2b8a20 0x562cfe2b8e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:28.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:28.926+0000 7f5515151740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562cfe2b9340 0x562cfe2c36c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:28.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:28.926+0000 7f5515151740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562cfe2c3c00 0x562cfe2c5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:28.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:28.926+0000 7f5515151740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x562cfe215620 con 0x562cfe2b9340 2024-09-11T23:38:28.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:28.926+0000 7f5515151740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x562cfe1bce50 con 0x562cfe2c3c00 2024-09-11T23:38:28.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:28.926+0000 7f5515151740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x562cfe2115e0 con 0x562cfe2b8a20 2024-09-11T23:38:28.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:28.926+0000 7f5509ffb640 1 -- 172.21.15.26:0/1677218652 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 8586+0+0 (secure 0 0 0) 0x7f55040df8e0 con 0x7f550c101070 2024-09-11T23:38:28.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:28.929+0000 7f5511315640 1 --2- 172.21.15.26:0/1677218652 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f54f40a2f00 0x7f54f40a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:28.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:28.931+0000 7f5511315640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562cfe2c3c00 0x562cfe2c5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:28.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:28.931+0000 7f5510b14640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562cfe2b8a20 0x562cfe2b8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:28.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:28.931+0000 7f5510b14640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562cfe2b8a20 0x562cfe2b8e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:33276/0 (socket says 172.21.15.26:33276) 2024-09-11T23:38:28.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:28.931+0000 7f5510b14640 1 -- 172.21.15.26:0/4266144262 learned_addr learned my addr 172.21.15.26:0/4266144262 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:38:28.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:28.932+0000 7f5510b14640 1 -- 172.21.15.26:0/4266144262 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562cfe2c3c00 msgr2=0x562cfe2c5ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:38:28.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:28.932+0000 7f5510b14640 1 --2- 172.21.15.26:0/4266144262 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562cfe2c3c00 0x562cfe2c5ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:28.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:28.932+0000 7f5510b14640 1 -- 172.21.15.26:0/4266144262 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562cfe2b9340 msgr2=0x562cfe2c36c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:38:28.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:28.932+0000 7f5510b14640 1 --2- 172.21.15.26:0/4266144262 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562cfe2b9340 0x562cfe2c36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:28.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:28.932+0000 7f5510b14640 1 -- 172.21.15.26:0/4266144262 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562cfe1add40 con 0x562cfe2b8a20 2024-09-11T23:38:28.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:28.933+0000 7f5510b14640 1 --2- 172.21.15.26:0/4266144262 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562cfe2b8a20 0x562cfe2b8e00 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f5504027000 tx=0x7f55040a8240 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8f0198a12792dfc5 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:38:28.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:28.934+0000 7f54f2ffd640 1 -- 172.21.15.26:0/4266144262 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55040b4ae0 con 0x562cfe2b8a20 2024-09-11T23:38:28.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:28.934+0000 7f54f2ffd640 1 -- 172.21.15.26:0/4266144262 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55040aa170 con 0x562cfe2b8a20 2024-09-11T23:38:28.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:28.934+0000 7f54f2ffd640 1 -- 172.21.15.26:0/4266144262 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55040b4650 con 0x562cfe2b8a20 2024-09-11T23:38:28.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:28.936+0000 7f5515151740 1 -- 172.21.15.26:0/4266144262 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562cfe2b8a20 msgr2=0x562cfe2b8e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:28.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:28.936+0000 7f5515151740 1 --2- 172.21.15.26:0/4266144262 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562cfe2b8a20 0x562cfe2b8e00 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f5504027000 tx=0x7f55040a8240 comp rx=0 tx=0).stop 2024-09-11T23:38:28.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:28.936+0000 7f5511315640 1 --2- 172.21.15.26:0/1677218652 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f54f40a2f00 0x7f54f40a53c0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x562cfe2b6330 tx=0x7f54fc00b250 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:38:28.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:28.938+0000 7f5515151740 1 -- 172.21.15.26:0/4266144262 shutdown_connections 2024-09-11T23:38:28.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:28.938+0000 7f5515151740 1 --2- 172.21.15.26:0/4266144262 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562cfe2c3c00 0x562cfe2c5ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:28.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:28.938+0000 7f5515151740 1 --2- 172.21.15.26:0/4266144262 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562cfe2b9340 0x562cfe2c36c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:28.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:28.938+0000 7f5515151740 1 --2- 172.21.15.26:0/4266144262 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562cfe2b8a20 0x562cfe2b8e00 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:28.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:28.938+0000 7f5515151740 1 -- 172.21.15.26:0/4266144262 >> 172.21.15.26:0/4266144262 conn(0x562cfe2b4fb0 msgr2=0x562cfe2b5390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:38:28.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:28.946+0000 7f5515151740 1 -- 172.21.15.26:0/4266144262 shutdown_connections 2024-09-11T23:38:28.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:28.946+0000 7f5515151740 1 -- 172.21.15.26:0/4266144262 wait complete. 2024-09-11T23:38:28.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:28.946+0000 7f5515151740 1 -- start start 2024-09-11T23:38:28.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:28.947+0000 7f5515151740 1 Processor -- start 2024-09-11T23:38:28.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:28.948+0000 7f5515151740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562cfe2b8a20 0x562cfe2c0b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:28.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:28.948+0000 7f5515151740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562cfe2b9340 0x562cfe2c1040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:28.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:28.948+0000 7f5515151740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562cfe2c3c00 0x562cfe2c1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:28.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:28.949+0000 7f550bfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562cfe2b9340 0x562cfe2c1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:28.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:28.949+0000 7f550bfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562cfe2b9340 0x562cfe2c1040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:33502/0 (socket says 172.21.15.26:33502) 2024-09-11T23:38:28.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:28.949+0000 7f550bfff640 1 -- 172.21.15.26:0/2903797936 learned_addr learned my addr 172.21.15.26:0/2903797936 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:38:28.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:28.949+0000 7f5515151740 1 -- 172.21.15.26:0/2903797936 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x562cfe2c67d0 con 0x562cfe2c3c00 2024-09-11T23:38:28.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:28.949+0000 7f5515151740 1 -- 172.21.15.26:0/2903797936 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x562cfe19df10 con 0x562cfe2b9340 2024-09-11T23:38:28.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:28.949+0000 7f5515151740 1 -- 172.21.15.26:0/2903797936 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x562cfe2c6950 con 0x562cfe2b8a20 2024-09-11T23:38:28.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:28.949+0000 7f5515151740 10 client.0 _ll_register_callbacks cb 0x562cfe1c1db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:38:28.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:28.949+0000 7f5511315640 1 --2- 172.21.15.26:0/2903797936 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562cfe2c3c00 0x562cfe2c1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:28.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:28.950+0000 7f5510b14640 1 --2- 172.21.15.26:0/2903797936 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562cfe2b8a20 0x562cfe2c0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:28.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:28.950+0000 7f5511315640 1 -- 172.21.15.26:0/2903797936 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562cfe2b8a20 msgr2=0x562cfe2c0b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:38:28.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:28.950+0000 7f5511315640 1 --2- 172.21.15.26:0/2903797936 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562cfe2b8a20 0x562cfe2c0b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:28.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:28.950+0000 7f5511315640 1 -- 172.21.15.26:0/2903797936 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562cfe2b9340 msgr2=0x562cfe2c1040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:38:28.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:28.950+0000 7f5511315640 1 --2- 172.21.15.26:0/2903797936 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562cfe2b9340 0x562cfe2c1040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:28.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:28.950+0000 7f5511315640 1 -- 172.21.15.26:0/2903797936 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562cfdf24580 con 0x562cfe2c3c00 2024-09-11T23:38:28.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:28.954+0000 7f5511315640 1 --2- 172.21.15.26:0/2903797936 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562cfe2c3c00 0x562cfe2c1580 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f54fc015690 tx=0x7f54fc008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea4377503d922f8e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:38:28.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:28.954+0000 7f54ebfff640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:38:28.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:28.960+0000 7f54ebfff640 1 -- 172.21.15.26:0/2903797936 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54fc025060 con 0x562cfe2c3c00 2024-09-11T23:38:28.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:28.960+0000 7f5515151740 1 -- 172.21.15.26:0/2903797936 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f55040aa170 con 0x562cfe2c3c00 2024-09-11T23:38:28.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:28.960+0000 7f54ebfff640 1 -- 172.21.15.26:0/2903797936 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 8586+0+0 (secure 0 0 0) 0x7f54fc0148c0 con 0x562cfe2c3c00 2024-09-11T23:38:28.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:28.964+0000 7f54ebfff640 1 -- 172.21.15.26:0/2903797936 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f54fc014ac0 con 0x562cfe2c3c00 2024-09-11T23:38:28.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:28.964+0000 7f5515151740 10 client.17988 fetch_fsmap learned FSMap version 88 2024-09-11T23:38:28.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:28.964+0000 7f5515151740 1 -- 172.21.15.26:0/2903797936 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=88,monmap=4+}) -- 0x562cfdfee960 con 0x562cfe2c3c00 2024-09-11T23:38:28.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:28.965+0000 7f54ebfff640 1 -- 172.21.15.26:0/2903797936 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55040aa170 con 0x562cfe2c3c00 2024-09-11T23:38:28.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:28.965+0000 7f54ebfff640 1 -- 172.21.15.26:0/2903797936 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 88) ==== 8426+0+0 (secure 0 0 0) 0x7f54fc0293d0 con 0x562cfe2c3c00 2024-09-11T23:38:28.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:28.965+0000 7f5515151740 10 client.17988 fetch_fsmap finished waiting for FSMap version 88 2024-09-11T23:38:28.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:28.965+0000 7f5515151740 10 client.17988 resolve_mds: validated gid 24485 aka daemon mds.b 2024-09-11T23:38:28.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:28.966+0000 7f5515151740 20 client.17988 populate_metadata read hostname 'smithi026' 2024-09-11T23:38:28.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:28.966+0000 7f5515151740 1 --2- 172.21.15.26:0/2903797936 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x562cfe2d3cc0 0x562cfe2d6140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:28.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:28.966+0000 7f5515151740 4 client.17988 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:38:28.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:28.966+0000 7f5515151740 1 -- 172.21.15.26:0/2903797936 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562cfe222b90 con 0x562cfe2d3cc0 2024-09-11T23:38:28.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:28.967+0000 7f5510b14640 1 --2- 172.21.15.26:0/2903797936 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x562cfe2d3cc0 0x562cfe2d6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:28.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:28.973+0000 7f54ebfff640 10 client.17988 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:38:28.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:28.973+0000 7f5510b14640 1 --2- 172.21.15.26:0/2903797936 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x562cfe2d3cc0 0x562cfe2d6140 crc :-1 s=READY pgs=300 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c48ed14b66653ac9 server_cookie=8e3e1907e4c4b1c3 in_seq=0 out_seq=0 2024-09-11T23:38:28.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:28.974+0000 7f54ebfff640 1 -- 172.21.15.26:0/2903797936 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562cfe222b90 con 0x562cfe2d3cc0 2024-09-11T23:38:28.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:28.974+0000 7f54ebfff640 10 client.17988 handle_command_reply: tid=0 2024-09-11T23:38:28.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:28.974+0000 7f5515151740 1 client.17988 shutdown 2024-09-11T23:38:28.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:28.974+0000 7f5515151740 1 -- 172.21.15.26:0/2903797936 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562cfe2c3c00 msgr2=0x562cfe2c1580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:28.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:28.974+0000 7f5515151740 1 --2- 172.21.15.26:0/2903797936 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562cfe2c3c00 0x562cfe2c1580 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f54fc015690 tx=0x7f54fc008040 comp rx=0 tx=0).stop 2024-09-11T23:38:28.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:28.974+0000 7f5515151740 1 -- 172.21.15.26:0/2903797936 shutdown_connections 2024-09-11T23:38:28.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:28.974+0000 7f5515151740 1 --2- 172.21.15.26:0/2903797936 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562cfe2b9340 0x562cfe2c1040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:28.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:28.974+0000 7f5515151740 1 --2- 172.21.15.26:0/2903797936 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x562cfe2d3cc0 0x562cfe2d6140 crc :-1 s=READY pgs=300 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:28.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:28.974+0000 7f5515151740 1 --2- 172.21.15.26:0/2903797936 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562cfe2c3c00 0x562cfe2c1580 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:28.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:28.974+0000 7f5515151740 1 --2- 172.21.15.26:0/2903797936 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562cfe2b8a20 0x562cfe2c0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:28.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:28.974+0000 7f5515151740 1 -- 172.21.15.26:0/2903797936 >> 172.21.15.26:0/2903797936 conn(0x562cfe2b4fb0 msgr2=0x562cfe2ab900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:38:28.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:28.974+0000 7f54ebfff640 0 client.17988 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:38:28.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:28.975+0000 7f5515151740 1 -- 172.21.15.26:0/2903797936 shutdown_connections 2024-09-11T23:38:28.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:28.975+0000 7f5515151740 1 -- 172.21.15.26:0/2903797936 wait complete. 2024-09-11T23:38:28.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:28.977+0000 7f5515151740 20 client.17988 trim_cache size 0 max 16384 2024-09-11T23:38:29.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:29.011+0000 7f5515151740 1 Processor -- start 2024-09-11T23:38:29.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:29.012+0000 7f5515151740 1 -- start start 2024-09-11T23:38:29.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:29.012+0000 7f5515151740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562cfe2b8a20 0x562cfe2bb4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:29.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:29.012+0000 7f5515151740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562cfe2b9340 0x562cfe2dba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:29.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:29.012+0000 7f5515151740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562cfe2c3c00 0x562cfe2dbf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:29.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:29.012+0000 7f5510b14640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562cfe2b8a20 0x562cfe2bb4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:29.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:29.012+0000 7f5510b14640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562cfe2b8a20 0x562cfe2bb4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50064/0 (socket says 172.21.15.26:50064) 2024-09-11T23:38:29.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:29.012+0000 7f5515151740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f54f800e930 con 0x562cfe2b8a20 2024-09-11T23:38:29.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:29.012+0000 7f5515151740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f54f8010710 con 0x562cfe2c3c00 2024-09-11T23:38:29.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:29.012+0000 7f5515151740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f54f800f060 con 0x562cfe2b9340 2024-09-11T23:38:29.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:29.012+0000 7f550bfff640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562cfe2b9340 0x562cfe2dba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:29.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:29.013+0000 7f550bfff640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562cfe2b9340 0x562cfe2dba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:33308/0 (socket says 172.21.15.26:33308) 2024-09-11T23:38:29.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:29.013+0000 7f550bfff640 1 -- 172.21.15.26:0/3232906407 learned_addr learned my addr 172.21.15.26:0/3232906407 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:38:29.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:29.013+0000 7f5511315640 1 --2- 172.21.15.26:0/3232906407 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562cfe2c3c00 0x562cfe2dbf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:29.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:29.013+0000 7f550bfff640 1 -- 172.21.15.26:0/3232906407 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562cfe2c3c00 msgr2=0x562cfe2dbf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:38:29.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:29.013+0000 7f550bfff640 1 --2- 172.21.15.26:0/3232906407 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562cfe2c3c00 0x562cfe2dbf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:29.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:29.013+0000 7f550bfff640 1 -- 172.21.15.26:0/3232906407 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562cfe2b8a20 msgr2=0x562cfe2bb4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:38:29.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:29.013+0000 7f550bfff640 1 --2- 172.21.15.26:0/3232906407 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562cfe2b8a20 0x562cfe2bb4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:29.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:29.013+0000 7f550bfff640 1 -- 172.21.15.26:0/3232906407 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562cfe2b4de0 con 0x562cfe2b9340 2024-09-11T23:38:29.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:29.013+0000 7f550bfff640 1 --2- 172.21.15.26:0/3232906407 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562cfe2b9340 0x562cfe2dba50 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f5500009900 tx=0x7f5500004180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eefe5b7abe59e831 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:38:29.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:29.014+0000 7f5510b14640 1 --2- 172.21.15.26:0/3232906407 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562cfe2b8a20 0x562cfe2bb4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2024-09-11T23:38:29.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:29.014+0000 7f54f27fc640 1 -- 172.21.15.26:0/3232906407 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5500035030 con 0x562cfe2b9340 2024-09-11T23:38:29.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:29.019+0000 7f54f27fc640 1 -- 172.21.15.26:0/3232906407 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5500003ad0 con 0x562cfe2b9340 2024-09-11T23:38:29.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:29.019+0000 7f54f27fc640 1 -- 172.21.15.26:0/3232906407 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5500031b60 con 0x562cfe2b9340 2024-09-11T23:38:29.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:29.021+0000 7f5515151740 1 -- 172.21.15.26:0/3232906407 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562cfe2b9340 msgr2=0x562cfe2dba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:29.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:29.021+0000 7f5515151740 1 --2- 172.21.15.26:0/3232906407 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562cfe2b9340 0x562cfe2dba50 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f5500009900 tx=0x7f5500004180 comp rx=0 tx=0).stop 2024-09-11T23:38:29.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:29.026+0000 7f5515151740 1 -- 172.21.15.26:0/3232906407 shutdown_connections 2024-09-11T23:38:29.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:29.026+0000 7f5515151740 1 --2- 172.21.15.26:0/3232906407 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562cfe2c3c00 0x562cfe2dbf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:29.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:29.026+0000 7f5515151740 1 --2- 172.21.15.26:0/3232906407 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562cfe2b9340 0x562cfe2dba50 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:29.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:29.026+0000 7f5515151740 1 --2- 172.21.15.26:0/3232906407 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562cfe2b8a20 0x562cfe2bb4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:29.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:29.026+0000 7f5515151740 1 -- 172.21.15.26:0/3232906407 >> 172.21.15.26:0/3232906407 conn(0x562cfe2b4fb0 msgr2=0x562cfe2bf3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:38:29.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:29.029+0000 7f5515151740 1 -- 172.21.15.26:0/3232906407 shutdown_connections 2024-09-11T23:38:29.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:29.029+0000 7f5515151740 1 -- 172.21.15.26:0/3232906407 wait complete. 2024-09-11T23:38:29.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:29.030+0000 7f5515151740 1 -- start start 2024-09-11T23:38:29.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:29.030+0000 7f5515151740 1 Processor -- start 2024-09-11T23:38:29.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:29.031+0000 7f5515151740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562cfe2b8a20 0x562cfe2cbbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:29.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:29.031+0000 7f5515151740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562cfe2b9340 0x562cfe2cc110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:29.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:29.032+0000 7f5510b14640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562cfe2b8a20 0x562cfe2cbbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:29.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:29.032+0000 7f5510b14640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562cfe2b8a20 0x562cfe2cbbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:33320/0 (socket says 172.21.15.26:33320) 2024-09-11T23:38:29.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:29.032+0000 7f5510b14640 1 -- 172.21.15.26:0/2572550276 learned_addr learned my addr 172.21.15.26:0/2572550276 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:38:29.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:29.032+0000 7f5515151740 1 --2- 172.21.15.26:0/2572550276 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562cfe2c3c00 0x562cfe2b41a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:29.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:29.032+0000 7f5515151740 1 -- 172.21.15.26:0/2572550276 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f54f800ff80 con 0x562cfe2b9340 2024-09-11T23:38:29.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:29.032+0000 7f5511315640 1 --2- 172.21.15.26:0/2572550276 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562cfe2c3c00 0x562cfe2b41a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:29.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:29.032+0000 7f5515151740 1 -- 172.21.15.26:0/2572550276 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f54f800f7f0 con 0x562cfe2c3c00 2024-09-11T23:38:29.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:29.032+0000 7f5515151740 1 -- 172.21.15.26:0/2572550276 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f54f800c930 con 0x562cfe2b8a20 2024-09-11T23:38:29.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:29.033+0000 7f5515151740 10 client.0 _ll_register_callbacks cb 0x7f54fc011f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:38:29.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:29.033+0000 7f5510b14640 1 -- 172.21.15.26:0/2572550276 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562cfe2c3c00 msgr2=0x562cfe2b41a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:38:29.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:29.033+0000 7f5510b14640 1 --2- 172.21.15.26:0/2572550276 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562cfe2c3c00 0x562cfe2b41a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:29.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:29.033+0000 7f5510b14640 1 -- 172.21.15.26:0/2572550276 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562cfe2b9340 msgr2=0x562cfe2cc110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:38:29.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:29.033+0000 7f5510b14640 1 --2- 172.21.15.26:0/2572550276 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562cfe2b9340 0x562cfe2cc110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:29.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:29.033+0000 7f5510b14640 1 -- 172.21.15.26:0/2572550276 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562cfe2d2380 con 0x562cfe2b8a20 2024-09-11T23:38:29.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:29.033+0000 7f5510b14640 1 --2- 172.21.15.26:0/2572550276 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562cfe2b8a20 0x562cfe2cbbd0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f55040b45b0 tx=0x7f55040abd70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6d0f28da484b1520 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:38:29.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:29.033+0000 7f54ebfff640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-09-11T23:38:29.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:29.034+0000 7f54ebfff640 1 -- 172.21.15.26:0/2572550276 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55040b4ae0 con 0x562cfe2b8a20 2024-09-11T23:38:29.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:29.034+0000 7f54ebfff640 1 -- 172.21.15.26:0/2572550276 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 8586+0+0 (secure 0 0 0) 0x7f55040a84e0 con 0x562cfe2b8a20 2024-09-11T23:38:29.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:29.034+0000 7f5515151740 1 -- 172.21.15.26:0/2572550276 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5500003ad0 con 0x562cfe2b8a20 2024-09-11T23:38:29.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:29.034+0000 7f54ebfff640 1 -- 172.21.15.26:0/2572550276 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f5504025440 con 0x562cfe2b8a20 2024-09-11T23:38:29.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:29.035+0000 7f5515151740 10 client.27469 fetch_fsmap learned FSMap version 88 2024-09-11T23:38:29.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:29.035+0000 7f5515151740 1 -- 172.21.15.26:0/2572550276 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=88,monmap=4+}) -- 0x562cfe3369e0 con 0x562cfe2b8a20 2024-09-11T23:38:29.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:29.035+0000 7f54ebfff640 1 -- 172.21.15.26:0/2572550276 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55040255e0 con 0x562cfe2b8a20 2024-09-11T23:38:29.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:29.039+0000 7f54ebfff640 1 -- 172.21.15.26:0/2572550276 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 88) ==== 8426+0+0 (secure 0 0 0) 0x7f55040ab2c0 con 0x562cfe2b8a20 2024-09-11T23:38:29.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:29.039+0000 7f5515151740 10 client.27469 fetch_fsmap finished waiting for FSMap version 88 2024-09-11T23:38:29.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:29.039+0000 7f5515151740 10 client.27469 resolve_mds: validated gid 24485 aka daemon mds.b 2024-09-11T23:38:29.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:29.039+0000 7f5515151740 20 client.27469 populate_metadata read hostname 'smithi026' 2024-09-11T23:38:29.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:29.039+0000 7f5515151740 1 --2- 172.21.15.26:0/2572550276 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x562cfe3369e0 0x562cfe338dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:29.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:29.039+0000 7f5515151740 4 client.27469 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "58f7281f", "release": true, "await": true}] 2024-09-11T23:38:29.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:29.039+0000 7f5515151740 1 -- 172.21.15.26:0/2572550276 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "quiesce db", "set_id": "58f7281f", "release": true, "await": true}) -- 0x7f5500003ad0 con 0x562cfe3369e0 2024-09-11T23:38:29.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:29.040+0000 7f550bfff640 1 --2- 172.21.15.26:0/2572550276 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x562cfe3369e0 0x562cfe338dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:29.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:29.048+0000 7f550bfff640 1 --2- 172.21.15.26:0/2572550276 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x562cfe3369e0 0x562cfe338dd0 crc :-1 s=READY pgs=301 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3366eb9d06522a68 server_cookie=446bad5660d469a3 in_seq=0 out_seq=0 2024-09-11T23:38:29.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:29.048+0000 7f54ebfff640 10 client.27469 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:38:29.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:38:28 smithi026 ceph-mon[31172]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:38:29.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:38:28 smithi076 ceph-mon[35296]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:38:29.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:38:28 smithi136 ceph-mon[35121]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:38:29.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:29.749+0000 7f54ebfff640 1 -- 172.21.15.26:0/2572550276 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x7f5500003ad0 con 0x562cfe3369e0 2024-09-11T23:38:29.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:29.749+0000 7f54ebfff640 10 client.27469 handle_command_reply: tid=0 2024-09-11T23:38:29.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:29.749+0000 7f5515151740 1 client.27469 shutdown 2024-09-11T23:38:29.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:29.750+0000 7f5515151740 1 -- 172.21.15.26:0/2572550276 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562cfe2b8a20 msgr2=0x562cfe2cbbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:29.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:29.750+0000 7f5515151740 1 --2- 172.21.15.26:0/2572550276 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562cfe2b8a20 0x562cfe2cbbd0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f55040b45b0 tx=0x7f55040abd70 comp rx=0 tx=0).stop 2024-09-11T23:38:29.751 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-11T23:38:29.751 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 88, 2024-09-11T23:38:29.751 INFO:teuthology.orchestra.run.smithi026.stdout: "leader": 24485, 2024-09-11T23:38:29.751 INFO:teuthology.orchestra.run.smithi026.stdout: "set_version": 68, 2024-09-11T23:38:29.751 INFO:teuthology.orchestra.run.smithi026.stdout: "sets": { 2024-09-11T23:38:29.751 INFO:teuthology.orchestra.run.smithi026.stdout: "58f7281f": { 2024-09-11T23:38:29.751 INFO:teuthology.orchestra.run.smithi026.stdout: "version": 68, 2024-09-11T23:38:29.751 INFO:teuthology.orchestra.run.smithi026.stdout: "age_ref": 0.0, 2024-09-11T23:38:29.751 INFO:teuthology.orchestra.run.smithi026.stdout: "state": { 2024-09-11T23:38:29.752 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "RELEASED", 2024-09-11T23:38:29.752 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.0 2024-09-11T23:38:29.752 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:38:29.752 INFO:teuthology.orchestra.run.smithi026.stdout: "timeout": 90.0, 2024-09-11T23:38:29.752 INFO:teuthology.orchestra.run.smithi026.stdout: "expiration": 141.8, 2024-09-11T23:38:29.752 INFO:teuthology.orchestra.run.smithi026.stdout: "members": { 2024-09-11T23:38:29.752 INFO:teuthology.orchestra.run.smithi026.stdout: "file:/": { 2024-09-11T23:38:29.752 INFO:teuthology.orchestra.run.smithi026.stdout: "excluded": false, 2024-09-11T23:38:29.752 INFO:teuthology.orchestra.run.smithi026.stdout: "state": { 2024-09-11T23:38:29.752 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "RELEASED", 2024-09-11T23:38:29.752 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.0 2024-09-11T23:38:29.752 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:38:29.752 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:38:29.753 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:38:29.753 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:38:29.753 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:38:29.753 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-11T23:38:29.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:29.750+0000 7f5515151740 1 -- 172.21.15.26:0/2572550276 shutdown_connections 2024-09-11T23:38:29.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:29.750+0000 7f5515151740 1 --2- 172.21.15.26:0/2572550276 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562cfe2c3c00 0x562cfe2b41a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:29.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:29.750+0000 7f5515151740 1 --2- 172.21.15.26:0/2572550276 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x562cfe3369e0 0x562cfe338dd0 crc :-1 s=READY pgs=301 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:29.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:29.750+0000 7f5515151740 1 --2- 172.21.15.26:0/2572550276 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562cfe2b9340 0x562cfe2cc110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:29.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:29.750+0000 7f54ebfff640 0 client.27469 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:38:29.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:29.750+0000 7f5515151740 1 --2- 172.21.15.26:0/2572550276 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562cfe2b8a20 0x562cfe2cbbd0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:29.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:29.750+0000 7f5515151740 1 -- 172.21.15.26:0/2572550276 >> 172.21.15.26:0/2572550276 conn(0x562cfe2b4fb0 msgr2=0x562cfe2ab0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:38:29.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:29.750+0000 7f5515151740 1 -- 172.21.15.26:0/2572550276 shutdown_connections 2024-09-11T23:38:29.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:29.750+0000 7f5515151740 1 -- 172.21.15.26:0/2572550276 wait complete. 2024-09-11T23:38:29.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:29.751+0000 7f5515151740 20 client.27469 trim_cache size 0 max 16384 2024-09-11T23:38:29.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:29.752+0000 7f5511b16640 1 -- 172.21.15.26:0/1677218652 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f54f40a2f00 msgr2=0x7f54f40a53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:29.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:29.752+0000 7f5511b16640 1 --2- 172.21.15.26:0/1677218652 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f54f40a2f00 0x7f54f40a53c0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x562cfe2b6330 tx=0x7f54fc00b250 comp rx=0 tx=0).stop 2024-09-11T23:38:29.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:29.752+0000 7f5511b16640 1 -- 172.21.15.26:0/1677218652 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f550c101070 msgr2=0x7f550c19a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:29.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:29.752+0000 7f5511b16640 1 --2- 172.21.15.26:0/1677218652 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f550c101070 0x7f550c19a6c0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f550401ca70 tx=0x7f550401cf30 comp rx=0 tx=0).stop 2024-09-11T23:38:29.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:29.753+0000 7f5511b16640 1 -- 172.21.15.26:0/1677218652 shutdown_connections 2024-09-11T23:38:29.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:29.753+0000 7f5511b16640 1 --2- 172.21.15.26:0/1677218652 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f54f40a2f00 0x7f54f40a53c0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:29.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:29.753+0000 7f5511b16640 1 --2- 172.21.15.26:0/1677218652 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f550c1058c0 0x7f550c19ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:29.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:29.754+0000 7f5511b16640 1 --2- 172.21.15.26:0/1677218652 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f550c1019f0 0x7f550c19ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:29.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:29.754+0000 7f5511b16640 1 --2- 172.21.15.26:0/1677218652 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f550c101070 0x7f550c19a6c0 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:29.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:29.754+0000 7f5511b16640 1 -- 172.21.15.26:0/1677218652 >> 172.21.15.26:0/1677218652 conn(0x7f550c077070 msgr2=0x7f550c103820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:38:29.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:29.754+0000 7f5511b16640 1 -- 172.21.15.26:0/1677218652 shutdown_connections 2024-09-11T23:38:29.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:29.754+0000 7f5511b16640 1 -- 172.21.15.26:0/1677218652 wait complete. 2024-09-11T23:38:30.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:38:30 smithi076 ceph-mon[35296]: pgmap v633: 97 pgs: 22 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 34 active+clean; 2.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 4.7 KiB/s wr, 11 op/s 2024-09-11T23:38:30.470 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:38:30 smithi026 ceph-mon[31172]: pgmap v633: 97 pgs: 22 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 34 active+clean; 2.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 4.7 KiB/s wr, 11 op/s 2024-09-11T23:38:30.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:38:30 smithi136 ceph-mon[35121]: pgmap v633: 97 pgs: 22 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 34 active+clean; 2.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 4.7 KiB/s wr, 11 op/s 2024-09-11T23:38:30.521 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '58f7281f', total seconds elapsed: 27.5 2024-09-11T23:38:30.521 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 21.8 2024-09-11T23:38:30.522 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:38:31.032 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:38:31.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:31.662+0000 7f4fd759e640 1 -- 172.21.15.26:0/252463197 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4fd8110190 msgr2=0x7f4fd806e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:31.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:31.662+0000 7f4fd759e640 1 --2- 172.21.15.26:0/252463197 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4fd8110190 0x7f4fd806e4f0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f4fcc009860 tx=0x7f4fcc02a380 comp rx=0 tx=0).stop 2024-09-11T23:38:31.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:31.667+0000 7f4fd759e640 1 -- 172.21.15.26:0/252463197 shutdown_connections 2024-09-11T23:38:31.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:31.667+0000 7f4fd759e640 1 --2- 172.21.15.26:0/252463197 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4fd806ea30 0x7f4fd806ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:31.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:31.667+0000 7f4fd759e640 1 --2- 172.21.15.26:0/252463197 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4fd811b140 0x7f4fd811d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:31.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:31.667+0000 7f4fd759e640 1 --2- 172.21.15.26:0/252463197 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4fd8110190 0x7f4fd806e4f0 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:31.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:31.667+0000 7f4fd759e640 1 -- 172.21.15.26:0/252463197 >> 172.21.15.26:0/252463197 conn(0x7f4fd8073180 msgr2=0x7f4fd8073590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:38:31.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:31.672+0000 7f4fd759e640 1 -- 172.21.15.26:0/252463197 shutdown_connections 2024-09-11T23:38:31.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:31.672+0000 7f4fd759e640 1 -- 172.21.15.26:0/252463197 wait complete. 2024-09-11T23:38:31.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:31.673+0000 7f4fd759e640 1 Processor -- start 2024-09-11T23:38:31.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:31.673+0000 7f4fd759e640 1 -- start start 2024-09-11T23:38:31.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:31.674+0000 7f4fd759e640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4fd8110190 0x7f4fd81187a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:31.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:31.674+0000 7f4fd759e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4fd811b140 0x7f4fd8111830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:31.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:31.674+0000 7f4fd759e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4fd8111d70 0x7f4fd8112220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:31.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:31.674+0000 7f4fd759e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f4fd8120ce0 con 0x7f4fd811b140 2024-09-11T23:38:31.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:31.674+0000 7f4fd759e640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f4fd8120b60 con 0x7f4fd8111d70 2024-09-11T23:38:31.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:31.674+0000 7f4fd759e640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f4fd8120e60 con 0x7f4fd8110190 2024-09-11T23:38:31.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:31.675+0000 7f4fd659c640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4fd8110190 0x7f4fd81187a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:31.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:31.675+0000 7f4fd659c640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4fd8110190 0x7f4fd81187a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:36472/0 (socket says 172.21.15.26:36472) 2024-09-11T23:38:31.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:31.675+0000 7f4fd659c640 1 -- 172.21.15.26:0/1049483758 learned_addr learned my addr 172.21.15.26:0/1049483758 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:38:31.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:31.675+0000 7f4fd5d9b640 1 --2- 172.21.15.26:0/1049483758 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4fd811b140 0x7f4fd8111830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:31.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:31.675+0000 7f4fd659c640 1 -- 172.21.15.26:0/1049483758 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4fd8111d70 msgr2=0x7f4fd8112220 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-11T23:38:31.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:31.675+0000 7f4fd659c640 1 --2- 172.21.15.26:0/1049483758 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4fd8111d70 0x7f4fd8112220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:31.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:31.675+0000 7f4fd659c640 1 -- 172.21.15.26:0/1049483758 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4fd811b140 msgr2=0x7f4fd8111830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:31.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:31.675+0000 7f4fd659c640 1 --2- 172.21.15.26:0/1049483758 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4fd811b140 0x7f4fd8111830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:31.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:31.675+0000 7f4fd659c640 1 -- 172.21.15.26:0/1049483758 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fd81162d0 con 0x7f4fd8110190 2024-09-11T23:38:31.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:31.676+0000 7f4fd659c640 1 --2- 172.21.15.26:0/1049483758 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4fd8110190 0x7f4fd81187a0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f4fcc02a8f0 tx=0x7f4fcc033d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:38:31.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:31.677+0000 7f4fd5d9b640 1 --2- 172.21.15.26:0/1049483758 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4fd811b140 0x7f4fd8111830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:38:31.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:31.685+0000 7f4fc77fe640 1 -- 172.21.15.26:0/1049483758 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fcc037040 con 0x7f4fd8110190 2024-09-11T23:38:31.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:31.685+0000 7f4fc77fe640 1 -- 172.21.15.26:0/1049483758 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4fcc005030 con 0x7f4fd8110190 2024-09-11T23:38:31.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:31.686+0000 7f4fc77fe640 1 -- 172.21.15.26:0/1049483758 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fcc004b80 con 0x7f4fd8110190 2024-09-11T23:38:31.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:31.686+0000 7f4fd759e640 1 -- 172.21.15.26:0/1049483758 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fd8116560 con 0x7f4fd8110190 2024-09-11T23:38:31.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:31.686+0000 7f4fd759e640 1 -- 172.21.15.26:0/1049483758 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fd8116900 con 0x7f4fd8110190 2024-09-11T23:38:31.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:31.688+0000 7f4fd759e640 1 -- 172.21.15.26:0/1049483758 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4fd81096f0 con 0x7f4fd8110190 2024-09-11T23:38:31.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:31.691+0000 7f4fc77fe640 1 -- 172.21.15.26:0/1049483758 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f4fcc004050 con 0x7f4fd8110190 2024-09-11T23:38:31.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:31.692+0000 7f4fc77fe640 1 --2- 172.21.15.26:0/1049483758 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f4fc00a2d60 0x7f4fc00a5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:31.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:31.692+0000 7f4fd5d9b640 1 --2- 172.21.15.26:0/1049483758 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f4fc00a2d60 0x7f4fc00a5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:31.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:31.693+0000 7f4fc77fe640 1 -- 172.21.15.26:0/1049483758 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 8586+0+0 (secure 0 0 0) 0x7f4fcc0eed70 con 0x7f4fd8110190 2024-09-11T23:38:31.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:31.693+0000 7f4fd5d9b640 1 --2- 172.21.15.26:0/1049483758 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f4fc00a2d60 0x7f4fc00a5220 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f4fc8006fd0 tx=0x7f4fc8008040 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:38:31.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:31.699+0000 7f4fc77fe640 1 -- 172.21.15.26:0/1049483758 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4fcc0f3070 con 0x7f4fd8110190 2024-09-11T23:38:32.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:32.115+0000 7f4fd759e640 1 -- 172.21.15.26:0/1049483758 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4fd8112d60 con 0x7f4fd8110190 2024-09-11T23:38:32.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:32.116+0000 7f4fc77fe640 1 -- 172.21.15.26:0/1049483758 <== mon.1 v2:172.21.15.136:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 88 v88) ==== 94+0+11999 (secure 0 0 0) 0x7f4fcc042350 con 0x7f4fd8110190 2024-09-11T23:38:32.117 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-11T23:38:32.117 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":88,"btime":"2024-09-11T23:38:27:980678+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14499,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/367659515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":367659515},{"type":"v1","addr":"172.21.15.26:6835","nonce":367659515}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14511,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/2041065299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":2041065299},{"type":"v1","addr":"172.21.15.26:6837","nonce":2041065299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":88,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:16:26.544203+0000","modified":"2024-09-11T23:38:27.872779+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24485,"mds_1":24463,"mds_2":24458,"mds_3":24448,"mds_4":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14550":{"gid":14550,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6839/2233068305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":2233068305},{"type":"v1","addr":"172.21.15.26:6839","nonce":2233068305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14562":{"gid":14562,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6841/2984982336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":2984982336},{"type":"v1","addr":"172.21.15.26:6841","nonce":2984982336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6833/2019841455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":2019841455},{"type":"v1","addr":"172.21.15.136:6833","nonce":2019841455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.76:6833/3438147050","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3438147050},{"type":"v1","addr":"172.21.15.76:6833","nonce":3438147050}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6835/3617665975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3617665975},{"type":"v1","addr":"172.21.15.136:6835","nonce":3617665975}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":12,"addr":"172.21.15.76:6835/393308499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":393308499},{"type":"v1","addr":"172.21.15.76:6835","nonce":393308499}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":10,"addr":"172.21.15.136:6837/1633082387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":1633082387},{"type":"v1","addr":"172.21.15.136:6837","nonce":1633082387}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.76:6837/2078978748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":2078978748},{"type":"v1","addr":"172.21.15.76:6837","nonce":2078978748}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":63,"addr":"172.21.15.136:6839/3211398630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":3211398630},{"type":"v1","addr":"172.21.15.136:6839","nonce":3211398630}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.76:6839/2077637090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6838","nonce":2077637090},{"type":"v1","addr":"172.21.15.76:6839","nonce":2077637090}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24485,"qdb_cluster":[24485,24463,24458,24448,24443]},"id":1}]} 2024-09-11T23:38:32.117 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 88 2024-09-11T23:38:32.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:32.121+0000 7f4fd759e640 1 -- 172.21.15.26:0/1049483758 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f4fc00a2d60 msgr2=0x7f4fc00a5220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:32.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:32.121+0000 7f4fd759e640 1 --2- 172.21.15.26:0/1049483758 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f4fc00a2d60 0x7f4fc00a5220 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f4fc8006fd0 tx=0x7f4fc8008040 comp rx=0 tx=0).stop 2024-09-11T23:38:32.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:32.121+0000 7f4fd759e640 1 -- 172.21.15.26:0/1049483758 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4fd8110190 msgr2=0x7f4fd81187a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:32.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:32.121+0000 7f4fd759e640 1 --2- 172.21.15.26:0/1049483758 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4fd8110190 0x7f4fd81187a0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f4fcc02a8f0 tx=0x7f4fcc033d10 comp rx=0 tx=0).stop 2024-09-11T23:38:32.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:32.122+0000 7f4fd759e640 1 -- 172.21.15.26:0/1049483758 shutdown_connections 2024-09-11T23:38:32.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:32.122+0000 7f4fd759e640 1 --2- 172.21.15.26:0/1049483758 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f4fc00a2d60 0x7f4fc00a5220 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:32.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:32.122+0000 7f4fd759e640 1 --2- 172.21.15.26:0/1049483758 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4fd8111d70 0x7f4fd8112220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:32.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:32.122+0000 7f4fd759e640 1 --2- 172.21.15.26:0/1049483758 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4fd811b140 0x7f4fd8111830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:32.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:32.122+0000 7f4fd759e640 1 --2- 172.21.15.26:0/1049483758 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4fd8110190 0x7f4fd81187a0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:32.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:32.122+0000 7f4fd759e640 1 -- 172.21.15.26:0/1049483758 >> 172.21.15.26:0/1049483758 conn(0x7f4fd8073180 msgr2=0x7f4fd811a9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:38:32.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:32.123+0000 7f4fd759e640 1 -- 172.21.15.26:0/1049483758 shutdown_connections 2024-09-11T23:38:32.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:32.123+0000 7f4fd759e640 1 -- 172.21.15.26:0/1049483758 wait complete. 2024-09-11T23:38:32.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:38:32 smithi076 ceph-mon[35296]: pgmap v634: 97 pgs: 22 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 34 active+clean; 2.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.2 KiB/s wr, 11 op/s 2024-09-11T23:38:32.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:38:32 smithi136 ceph-mon[35121]: pgmap v634: 97 pgs: 22 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 34 active+clean; 2.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.2 KiB/s wr, 11 op/s 2024-09-11T23:38:32.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:38:32 smithi026 ceph-mon[31172]: pgmap v634: 97 pgs: 22 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 34 active+clean; 2.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.2 KiB/s wr, 11 op/s 2024-09-11T23:38:32.924 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24485 quiesce db / --timeout 90 --expiration 141.8 --await-for 45.0' 2024-09-11T23:38:32.925 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24485 quiesce db / --timeout 90 --expiration 141.8 --await-for 45.0 2024-09-11T23:38:33.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:38:33 smithi026 ceph-mon[31172]: from='client.? 172.21.15.26:0/1049483758' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:38:33.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:38:33 smithi076 ceph-mon[35296]: from='client.? 172.21.15.26:0/1049483758' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:38:33.401 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:38:33.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:38:33 smithi136 ceph-mon[35121]: from='client.? 172.21.15.26:0/1049483758' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:38:34.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.045+0000 7f011d59a640 1 -- 172.21.15.26:0/1306898473 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f011802e320 con 0x7f0120100e20 2024-09-11T23:38:34.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.046+0000 7f01248c3640 1 -- 172.21.15.26:0/1306898473 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0120100e20 msgr2=0x7f0120101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:34.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.046+0000 7f01248c3640 1 --2- 172.21.15.26:0/1306898473 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0120100e20 0x7f0120101200 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f0118009ca0 tx=0x7f011802a970 comp rx=0 tx=0).stop 2024-09-11T23:38:34.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.046+0000 7f01248c3640 1 -- 172.21.15.26:0/1306898473 shutdown_connections 2024-09-11T23:38:34.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.046+0000 7f01248c3640 1 --2- 172.21.15.26:0/1306898473 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0120102ef0 0x7f0120069f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:34.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.046+0000 7f01248c3640 1 --2- 172.21.15.26:0/1306898473 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0120102570 0x7f01201029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:34.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.046+0000 7f01248c3640 1 --2- 172.21.15.26:0/1306898473 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0120100e20 0x7f0120101200 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:34.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.046+0000 7f01248c3640 1 -- 172.21.15.26:0/1306898473 >> 172.21.15.26:0/1306898473 conn(0x7f01200fc7d0 msgr2=0x7f01200febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:38:34.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.047+0000 7f01248c3640 1 -- 172.21.15.26:0/1306898473 shutdown_connections 2024-09-11T23:38:34.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.047+0000 7f01248c3640 1 -- 172.21.15.26:0/1306898473 wait complete. 2024-09-11T23:38:34.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.048+0000 7f01248c3640 1 Processor -- start 2024-09-11T23:38:34.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.048+0000 7f01248c3640 1 -- start start 2024-09-11T23:38:34.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.048+0000 7f01248c3640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0120100e20 0x7f01200738d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:34.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.048+0000 7f01248c3640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0120102570 0x7f0120073e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:34.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.048+0000 7f01248c3640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0120102ef0 0x7f012006ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:34.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.048+0000 7f01248c3640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f0120114330 con 0x7f0120100e20 2024-09-11T23:38:34.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.048+0000 7f01248c3640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f01201141b0 con 0x7f0120102ef0 2024-09-11T23:38:34.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.048+0000 7f01248c3640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f01201144b0 con 0x7f0120102570 2024-09-11T23:38:34.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.049+0000 7f011e59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0120100e20 0x7f01200738d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:34.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.049+0000 7f011e59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0120100e20 0x7f01200738d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60350/0 (socket says 172.21.15.26:60350) 2024-09-11T23:38:34.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.049+0000 7f011e59c640 1 -- 172.21.15.26:0/308515937 learned_addr learned my addr 172.21.15.26:0/308515937 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:38:34.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.049+0000 7f011dd9b640 1 --2- 172.21.15.26:0/308515937 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0120102570 0x7f0120073e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:34.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.049+0000 7f011ed9d640 1 --2- 172.21.15.26:0/308515937 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0120102ef0 0x7f012006ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:34.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.050+0000 7f011e59c640 1 -- 172.21.15.26:0/308515937 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0120102570 msgr2=0x7f0120073e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:34.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.050+0000 7f011e59c640 1 --2- 172.21.15.26:0/308515937 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0120102570 0x7f0120073e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:34.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.050+0000 7f011e59c640 1 -- 172.21.15.26:0/308515937 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0120102ef0 msgr2=0x7f012006ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:34.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.050+0000 7f011e59c640 1 --2- 172.21.15.26:0/308515937 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0120102ef0 0x7f012006ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:34.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.050+0000 7f011e59c640 1 -- 172.21.15.26:0/308515937 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f012006cfc0 con 0x7f0120100e20 2024-09-11T23:38:34.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.050+0000 7f011e59c640 1 --2- 172.21.15.26:0/308515937 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0120100e20 0x7f01200738d0 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f01180098d0 tx=0x7f01180308a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:38:34.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.050+0000 7f010f7fe640 1 -- 172.21.15.26:0/308515937 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0118034030 con 0x7f0120100e20 2024-09-11T23:38:34.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.051+0000 7f01248c3640 1 -- 172.21.15.26:0/308515937 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f012006d250 con 0x7f0120100e20 2024-09-11T23:38:34.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.051+0000 7f01248c3640 1 -- 172.21.15.26:0/308515937 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f012006d780 con 0x7f0120100e20 2024-09-11T23:38:34.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.051+0000 7f010f7fe640 1 -- 172.21.15.26:0/308515937 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0118004040 con 0x7f0120100e20 2024-09-11T23:38:34.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.053+0000 7f010f7fe640 1 -- 172.21.15.26:0/308515937 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0118039710 con 0x7f0120100e20 2024-09-11T23:38:34.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.054+0000 7f0126c9c740 1 Processor -- start 2024-09-11T23:38:34.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.055+0000 7f0126c9c740 1 -- start start 2024-09-11T23:38:34.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.055+0000 7f0126c9c740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f488318760 0x55f488318b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:34.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.055+0000 7f0126c9c740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f488319080 0x55f488323400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:34.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.055+0000 7f0126c9c740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f488323940 0x55f488325d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:34.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.055+0000 7f0126c9c740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55f48827e2a0 con 0x55f488323940 2024-09-11T23:38:34.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.055+0000 7f0126c9c740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55f488225b10 con 0x55f488318760 2024-09-11T23:38:34.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.055+0000 7f0126c9c740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55f48827a260 con 0x55f488319080 2024-09-11T23:38:34.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.055+0000 7f011dd9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f488323940 0x55f488325d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:34.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.055+0000 7f011dd9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f488323940 0x55f488325d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60364/0 (socket says 172.21.15.26:60364) 2024-09-11T23:38:34.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.055+0000 7f011dd9b640 1 -- 172.21.15.26:0/3959233608 learned_addr learned my addr 172.21.15.26:0/3959233608 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:38:34.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.056+0000 7f011ed9d640 1 --2- 172.21.15.26:0/3959233608 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f488318760 0x55f488318b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:34.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.056+0000 7f011dd9b640 1 -- 172.21.15.26:0/3959233608 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f488319080 msgr2=0x55f488323400 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T23:38:34.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.056+0000 7f011dd9b640 1 --2- 172.21.15.26:0/3959233608 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f488319080 0x55f488323400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:34.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.056+0000 7f011dd9b640 1 -- 172.21.15.26:0/3959233608 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f488318760 msgr2=0x55f488318b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:38:34.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.056+0000 7f011dd9b640 1 --2- 172.21.15.26:0/3959233608 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f488318760 0x55f488318b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:34.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.056+0000 7f011dd9b640 1 -- 172.21.15.26:0/3959233608 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f487fefb90 con 0x55f488323940 2024-09-11T23:38:34.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.056+0000 7f011dd9b640 1 --2- 172.21.15.26:0/3959233608 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f488323940 0x55f488325d30 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f01140099b0 tx=0x7f0114009e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59c55750eebb1318 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:38:34.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.058+0000 7f010cff9640 1 -- 172.21.15.26:0/3959233608 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0114017070 con 0x55f488323940 2024-09-11T23:38:34.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.058+0000 7f010cff9640 1 -- 172.21.15.26:0/3959233608 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0114013470 con 0x55f488323940 2024-09-11T23:38:34.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.058+0000 7f010cff9640 1 -- 172.21.15.26:0/3959233608 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0114012630 con 0x55f488323940 2024-09-11T23:38:34.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.059+0000 7f010f7fe640 1 -- 172.21.15.26:0/308515937 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f0118037020 con 0x7f0120100e20 2024-09-11T23:38:34.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.060+0000 7f010f7fe640 1 --2- 172.21.15.26:0/308515937 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f01080a2d70 0x7f01080a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:34.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.060+0000 7f0126c9c740 1 -- 172.21.15.26:0/3959233608 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f488323940 msgr2=0x55f488325d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:34.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.060+0000 7f0126c9c740 1 --2- 172.21.15.26:0/3959233608 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f488323940 0x55f488325d30 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f01140099b0 tx=0x7f0114009e70 comp rx=0 tx=0).stop 2024-09-11T23:38:34.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.061+0000 7f010f7fe640 1 -- 172.21.15.26:0/308515937 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 8586+0+0 (secure 0 0 0) 0x7f01180f0400 con 0x7f0120100e20 2024-09-11T23:38:34.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.061+0000 7f0126c9c740 1 -- 172.21.15.26:0/3959233608 shutdown_connections 2024-09-11T23:38:34.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.061+0000 7f0126c9c740 1 --2- 172.21.15.26:0/3959233608 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f488323940 0x55f488325d30 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:34.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.061+0000 7f0126c9c740 1 --2- 172.21.15.26:0/3959233608 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f488319080 0x55f488323400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:34.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.061+0000 7f0126c9c740 1 --2- 172.21.15.26:0/3959233608 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f488318760 0x55f488318b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:34.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.061+0000 7f0126c9c740 1 -- 172.21.15.26:0/3959233608 >> 172.21.15.26:0/3959233608 conn(0x55f488314cf0 msgr2=0x55f4883150d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:38:34.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.061+0000 7f011ed9d640 1 --2- 172.21.15.26:0/308515937 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f01080a2d70 0x7f01080a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:34.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.062+0000 7f0126c9c740 1 -- 172.21.15.26:0/3959233608 shutdown_connections 2024-09-11T23:38:34.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.063+0000 7f011ed9d640 1 --2- 172.21.15.26:0/308515937 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f01080a2d70 0x7f01080a5230 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f011000aba0 tx=0x7f011000a250 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:38:34.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.064+0000 7f0126c9c740 1 -- 172.21.15.26:0/3959233608 wait complete. 2024-09-11T23:38:34.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.064+0000 7f0126c9c740 1 -- start start 2024-09-11T23:38:34.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.064+0000 7f0126c9c740 1 Processor -- start 2024-09-11T23:38:34.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.065+0000 7f0126c9c740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f488318760 0x55f4883208b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:34.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.065+0000 7f0126c9c740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f488319080 0x55f488320df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:34.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.065+0000 7f0126c9c740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f488323940 0x55f488321330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:34.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.065+0000 7f0126c9c740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55f488326510 con 0x55f488318760 2024-09-11T23:38:34.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.065+0000 7f0126c9c740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55f4882066e0 con 0x55f488323940 2024-09-11T23:38:34.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.065+0000 7f0126c9c740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55f488326690 con 0x55f488319080 2024-09-11T23:38:34.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.065+0000 7f0126c9c740 10 client.0 _ll_register_callbacks cb 0x55f48822aa80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:38:34.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.066+0000 7f011e59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f488318760 0x55f4883208b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:34.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.066+0000 7f011e59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f488318760 0x55f4883208b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60366/0 (socket says 172.21.15.26:60366) 2024-09-11T23:38:34.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.066+0000 7f011e59c640 1 -- 172.21.15.26:0/3761162819 learned_addr learned my addr 172.21.15.26:0/3761162819 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:38:34.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.066+0000 7f011dd9b640 1 --2- 172.21.15.26:0/3761162819 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f488319080 0x55f488320df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:34.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.066+0000 7f011ed9d640 1 --2- 172.21.15.26:0/3761162819 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f488323940 0x55f488321330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:34.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.066+0000 7f011e59c640 1 -- 172.21.15.26:0/3761162819 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f488319080 msgr2=0x55f488320df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:38:34.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.066+0000 7f011e59c640 1 --2- 172.21.15.26:0/3761162819 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f488319080 0x55f488320df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:34.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.066+0000 7f011e59c640 1 -- 172.21.15.26:0/3761162819 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f488323940 msgr2=0x55f488321330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:38:34.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.066+0000 7f011e59c640 1 --2- 172.21.15.26:0/3761162819 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f488323940 0x55f488321330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:34.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.066+0000 7f011e59c640 1 -- 172.21.15.26:0/3761162819 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f48832fb10 con 0x55f488318760 2024-09-11T23:38:34.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.067+0000 7f011dd9b640 1 --2- 172.21.15.26:0/3761162819 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f488319080 0x55f488320df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:38:34.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.067+0000 7f011ed9d640 1 --2- 172.21.15.26:0/3761162819 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f488323940 0x55f488321330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:38:34.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.067+0000 7f011e59c640 1 --2- 172.21.15.26:0/3761162819 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f488318760 0x55f4883208b0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f01180ee110 tx=0x7f01180bcfe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d93dba6f6242620e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:38:34.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.070+0000 7f0105ffb640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:38:34.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.070+0000 7f0105ffb640 1 -- 172.21.15.26:0/3761162819 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01180c5550 con 0x55f488318760 2024-09-11T23:38:34.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.070+0000 7f0105ffb640 1 -- 172.21.15.26:0/3761162819 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 8586+0+0 (secure 0 0 0) 0x7f01180b8f50 con 0x55f488318760 2024-09-11T23:38:34.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.070+0000 7f0126c9c740 1 -- 172.21.15.26:0/3761162819 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0114013470 con 0x55f488318760 2024-09-11T23:38:34.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.071+0000 7f0105ffb640 1 -- 172.21.15.26:0/3761162819 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f01180db280 con 0x55f488318760 2024-09-11T23:38:34.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.071+0000 7f0126c9c740 10 client.18006 fetch_fsmap learned FSMap version 88 2024-09-11T23:38:34.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.071+0000 7f0126c9c740 1 -- 172.21.15.26:0/3761162819 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=88,monmap=4+}) -- 0x55f488326510 con 0x55f488318760 2024-09-11T23:38:34.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.071+0000 7f0105ffb640 1 -- 172.21.15.26:0/3761162819 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0114013470 con 0x55f488318760 2024-09-11T23:38:34.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.072+0000 7f0105ffb640 1 -- 172.21.15.26:0/3761162819 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 88) ==== 8426+0+0 (secure 0 0 0) 0x7f0118041070 con 0x55f488318760 2024-09-11T23:38:34.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.072+0000 7f0126c9c740 10 client.18006 fetch_fsmap finished waiting for FSMap version 88 2024-09-11T23:38:34.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.072+0000 7f0126c9c740 10 client.18006 resolve_mds: validated gid 24485 aka daemon mds.b 2024-09-11T23:38:34.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.072+0000 7f0126c9c740 20 client.18006 populate_metadata read hostname 'smithi026' 2024-09-11T23:38:34.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.072+0000 7f0126c9c740 1 --2- 172.21.15.26:0/3761162819 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55f4883339d0 0x55f488335e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:34.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.072+0000 7f0126c9c740 4 client.18006 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:38:34.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.072+0000 7f0126c9c740 1 -- 172.21.15.26:0/3761162819 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f48828b9d0 con 0x55f4883339d0 2024-09-11T23:38:34.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.072+0000 7f011dd9b640 1 --2- 172.21.15.26:0/3761162819 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55f4883339d0 0x55f488335e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:34.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.074+0000 7f011dd9b640 1 --2- 172.21.15.26:0/3761162819 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55f4883339d0 0x55f488335e10 crc :-1 s=READY pgs=302 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be0ec81100a9580e server_cookie=a9e3b611a2005103 in_seq=0 out_seq=0 2024-09-11T23:38:34.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.074+0000 7f0105ffb640 10 client.18006 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:38:34.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.076+0000 7f0105ffb640 1 -- 172.21.15.26:0/3761162819 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f48828b9d0 con 0x55f4883339d0 2024-09-11T23:38:34.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.076+0000 7f0105ffb640 10 client.18006 handle_command_reply: tid=0 2024-09-11T23:38:34.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.076+0000 7f0126c9c740 1 client.18006 shutdown 2024-09-11T23:38:34.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.077+0000 7f0126c9c740 1 -- 172.21.15.26:0/3761162819 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f488318760 msgr2=0x55f4883208b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:34.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.077+0000 7f0126c9c740 1 --2- 172.21.15.26:0/3761162819 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f488318760 0x55f4883208b0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f01180ee110 tx=0x7f01180bcfe0 comp rx=0 tx=0).stop 2024-09-11T23:38:34.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.077+0000 7f0126c9c740 1 -- 172.21.15.26:0/3761162819 shutdown_connections 2024-09-11T23:38:34.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.077+0000 7f0126c9c740 1 --2- 172.21.15.26:0/3761162819 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f488323940 0x55f488321330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:34.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.078+0000 7f0126c9c740 1 --2- 172.21.15.26:0/3761162819 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55f4883339d0 0x55f488335e10 crc :-1 s=READY pgs=302 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:34.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.078+0000 7f0126c9c740 1 --2- 172.21.15.26:0/3761162819 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f488319080 0x55f488320df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:34.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.078+0000 7f0126c9c740 1 --2- 172.21.15.26:0/3761162819 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f488318760 0x55f4883208b0 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:34.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.078+0000 7f0126c9c740 1 -- 172.21.15.26:0/3761162819 >> 172.21.15.26:0/3761162819 conn(0x55f488314cf0 msgr2=0x55f488313e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:38:34.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.078+0000 7f0105ffb640 0 client.18006 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:38:34.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.079+0000 7f0126c9c740 1 -- 172.21.15.26:0/3761162819 shutdown_connections 2024-09-11T23:38:34.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.079+0000 7f0126c9c740 1 -- 172.21.15.26:0/3761162819 wait complete. 2024-09-11T23:38:34.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.079+0000 7f0126c9c740 20 client.18006 trim_cache size 0 max 16384 2024-09-11T23:38:34.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.111+0000 7f0126c9c740 1 Processor -- start 2024-09-11T23:38:34.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.112+0000 7f0126c9c740 1 -- start start 2024-09-11T23:38:34.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.112+0000 7f0126c9c740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f488318760 0x55f488320e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:34.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.112+0000 7f0126c9c740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f488319080 0x55f488321340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:34.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.112+0000 7f011e59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f488318760 0x55f488320e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:34.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.112+0000 7f011e59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f488318760 0x55f488320e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60380/0 (socket says 172.21.15.26:60380) 2024-09-11T23:38:34.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.112+0000 7f011e59c640 1 -- 172.21.15.26:0/1360714070 learned_addr learned my addr 172.21.15.26:0/1360714070 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:38:34.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.112+0000 7f0126c9c740 1 --2- 172.21.15.26:0/1360714070 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f488323940 0x55f48833b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:34.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.112+0000 7f0126c9c740 1 -- 172.21.15.26:0/1360714070 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f010000e970 con 0x55f488318760 2024-09-11T23:38:34.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.112+0000 7f0126c9c740 1 -- 172.21.15.26:0/1360714070 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f0100010750 con 0x55f488319080 2024-09-11T23:38:34.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.112+0000 7f0126c9c740 1 -- 172.21.15.26:0/1360714070 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f010000f0a0 con 0x55f488323940 2024-09-11T23:38:34.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.113+0000 7f011e59c640 1 -- 172.21.15.26:0/1360714070 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f488323940 msgr2=0x55f48833b6e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T23:38:34.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.113+0000 7f011e59c640 1 --2- 172.21.15.26:0/1360714070 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f488323940 0x55f48833b6e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:34.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.113+0000 7f011e59c640 1 -- 172.21.15.26:0/1360714070 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f488319080 msgr2=0x55f488321340 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T23:38:34.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.113+0000 7f011e59c640 1 --2- 172.21.15.26:0/1360714070 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f488319080 0x55f488321340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:34.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.113+0000 7f011e59c640 1 -- 172.21.15.26:0/1360714070 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f488310260 con 0x55f488318760 2024-09-11T23:38:34.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.114+0000 7f011e59c640 1 --2- 172.21.15.26:0/1360714070 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f488318760 0x55f488320e00 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f0118009dc0 tx=0x7f01180badf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=154d030290cd7db9 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:38:34.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.114+0000 7f0107fff640 1 -- 172.21.15.26:0/1360714070 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01180a0080 con 0x55f488318760 2024-09-11T23:38:34.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.114+0000 7f0107fff640 1 -- 172.21.15.26:0/1360714070 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f01180c50c0 con 0x55f488318760 2024-09-11T23:38:34.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.114+0000 7f0107fff640 1 -- 172.21.15.26:0/1360714070 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01180bbd30 con 0x55f488318760 2024-09-11T23:38:34.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.115+0000 7f0126c9c740 1 -- 172.21.15.26:0/1360714070 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f488318760 msgr2=0x55f488320e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:34.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.115+0000 7f0126c9c740 1 --2- 172.21.15.26:0/1360714070 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f488318760 0x55f488320e00 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f0118009dc0 tx=0x7f01180badf0 comp rx=0 tx=0).stop 2024-09-11T23:38:34.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.116+0000 7f0126c9c740 1 -- 172.21.15.26:0/1360714070 shutdown_connections 2024-09-11T23:38:34.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.116+0000 7f0126c9c740 1 --2- 172.21.15.26:0/1360714070 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f488319080 0x55f488321340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:34.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.116+0000 7f0126c9c740 1 --2- 172.21.15.26:0/1360714070 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f488323940 0x55f48833b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:34.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.116+0000 7f0126c9c740 1 --2- 172.21.15.26:0/1360714070 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f488318760 0x55f488320e00 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:34.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.116+0000 7f0126c9c740 1 -- 172.21.15.26:0/1360714070 >> 172.21.15.26:0/1360714070 conn(0x55f488314cf0 msgr2=0x55f488322930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:38:34.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.117+0000 7f0126c9c740 1 -- 172.21.15.26:0/1360714070 shutdown_connections 2024-09-11T23:38:34.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.117+0000 7f0126c9c740 1 -- 172.21.15.26:0/1360714070 wait complete. 2024-09-11T23:38:34.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.118+0000 7f0126c9c740 1 -- start start 2024-09-11T23:38:34.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.118+0000 7f0126c9c740 1 Processor -- start 2024-09-11T23:38:34.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.118+0000 7f0126c9c740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f488318760 0x55f4883276d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:34.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.119+0000 7f0126c9c740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f488319080 0x55f488327c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:34.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.119+0000 7f011dd9b640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f488319080 0x55f488327c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:34.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.119+0000 7f0126c9c740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f488323940 0x55f488328150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:34.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.119+0000 7f011dd9b640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f488319080 0x55f488327c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:36536/0 (socket says 172.21.15.26:36536) 2024-09-11T23:38:34.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.119+0000 7f011dd9b640 1 -- 172.21.15.26:0/1597798838 learned_addr learned my addr 172.21.15.26:0/1597798838 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:38:34.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.119+0000 7f011e59c640 1 --2- 172.21.15.26:0/1597798838 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f488318760 0x55f4883276d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:34.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.119+0000 7f0126c9c740 1 -- 172.21.15.26:0/1597798838 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f010000ffc0 con 0x55f488318760 2024-09-11T23:38:34.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.120+0000 7f0126c9c740 1 -- 172.21.15.26:0/1597798838 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f010000f830 con 0x55f488323940 2024-09-11T23:38:34.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.120+0000 7f0126c9c740 1 -- 172.21.15.26:0/1597798838 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f010000c970 con 0x55f488319080 2024-09-11T23:38:34.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.120+0000 7f0126c9c740 10 client.0 _ll_register_callbacks cb 0x7f01180bc420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:38:34.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.123+0000 7f011ed9d640 1 --2- 172.21.15.26:0/1597798838 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f488323940 0x55f488328150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:34.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.123+0000 7f011dd9b640 1 -- 172.21.15.26:0/1597798838 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f488323940 msgr2=0x55f488328150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:38:34.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.123+0000 7f011dd9b640 1 --2- 172.21.15.26:0/1597798838 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f488323940 0x55f488328150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:34.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.123+0000 7f011dd9b640 1 -- 172.21.15.26:0/1597798838 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f488318760 msgr2=0x55f4883276d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:38:34.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.123+0000 7f011dd9b640 1 --2- 172.21.15.26:0/1597798838 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f488318760 0x55f4883276d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:34.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.123+0000 7f011dd9b640 1 -- 172.21.15.26:0/1597798838 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f48839d6b0 con 0x55f488319080 2024-09-11T23:38:34.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.124+0000 7f011dd9b640 1 --2- 172.21.15.26:0/1597798838 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f488319080 0x55f488327c10 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f01140098d0 tx=0x7f01140093c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b8099bb31d7664b5 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:38:34.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.124+0000 7f0105ffb640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-09-11T23:38:34.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.125+0000 7f0105ffb640 1 -- 172.21.15.26:0/1597798838 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0114017040 con 0x55f488319080 2024-09-11T23:38:34.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.125+0000 7f0105ffb640 1 -- 172.21.15.26:0/1597798838 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 8586+0+0 (secure 0 0 0) 0x7f0114005d80 con 0x55f488319080 2024-09-11T23:38:34.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.125+0000 7f0126c9c740 1 -- 172.21.15.26:0/1597798838 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f01180c50c0 con 0x55f488319080 2024-09-11T23:38:34.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.125+0000 7f0105ffb640 1 -- 172.21.15.26:0/1597798838 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f0114004020 con 0x55f488319080 2024-09-11T23:38:34.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.131+0000 7f0126c9c740 10 client.27484 fetch_fsmap learned FSMap version 88 2024-09-11T23:38:34.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.131+0000 7f0126c9c740 1 -- 172.21.15.26:0/1597798838 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=88,monmap=4+}) -- 0x55f48839e920 con 0x55f488319080 2024-09-11T23:38:34.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.131+0000 7f0105ffb640 1 -- 172.21.15.26:0/1597798838 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0114013470 con 0x55f488319080 2024-09-11T23:38:34.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.132+0000 7f0105ffb640 1 -- 172.21.15.26:0/1597798838 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 88) ==== 8426+0+0 (secure 0 0 0) 0x7f01140039c0 con 0x55f488319080 2024-09-11T23:38:34.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.132+0000 7f0126c9c740 10 client.27484 fetch_fsmap finished waiting for FSMap version 88 2024-09-11T23:38:34.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.132+0000 7f0126c9c740 10 client.27484 resolve_mds: validated gid 24485 aka daemon mds.b 2024-09-11T23:38:34.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.132+0000 7f0126c9c740 20 client.27484 populate_metadata read hostname 'smithi026' 2024-09-11T23:38:34.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.132+0000 7f0126c9c740 1 --2- 172.21.15.26:0/1597798838 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55f48839ef30 0x55f4883a1350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:34.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.132+0000 7f0126c9c740 4 client.27484 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 141.8, "await_for": 45.0}] 2024-09-11T23:38:34.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.132+0000 7f0126c9c740 1 -- 172.21.15.26:0/1597798838 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 141.8, "await_for": 45.0}) -- 0x7f01180c50c0 con 0x55f48839ef30 2024-09-11T23:38:34.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.133+0000 7f011e59c640 1 --2- 172.21.15.26:0/1597798838 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55f48839ef30 0x55f4883a1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:34.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.142+0000 7f011e59c640 1 --2- 172.21.15.26:0/1597798838 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55f48839ef30 0x55f4883a1350 crc :-1 s=READY pgs=303 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92b5bd6ee87af11e server_cookie=fa18dbbb83a4495f in_seq=0 out_seq=0 2024-09-11T23:38:34.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:34.142+0000 7f0105ffb640 10 client.27484 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:38:34.143 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:38:34 smithi026 ceph-mon[31172]: pgmap v635: 97 pgs: 22 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 34 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 5.0 KiB/s wr, 12 op/s 2024-09-11T23:38:34.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:38:34 smithi076 ceph-mon[35296]: pgmap v635: 97 pgs: 22 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 34 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 5.0 KiB/s wr, 12 op/s 2024-09-11T23:38:34.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:38:34 smithi136 ceph-mon[35121]: pgmap v635: 97 pgs: 22 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 34 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 5.0 KiB/s wr, 12 op/s 2024-09-11T23:38:36.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:38:36 smithi076 ceph-mon[35296]: pgmap v636: 97 pgs: 22 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 34 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 5.0 KiB/s wr, 14 op/s 2024-09-11T23:38:36.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:38:36 smithi136 ceph-mon[35121]: pgmap v636: 97 pgs: 22 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 34 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 5.0 KiB/s wr, 14 op/s 2024-09-11T23:38:36.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:38:36 smithi026 ceph-mon[31172]: pgmap v636: 97 pgs: 22 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 34 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 5.0 KiB/s wr, 14 op/s 2024-09-11T23:38:37.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:37.594+0000 7f0105ffb640 1 -- 172.21.15.26:0/1597798838 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x7f01180c50c0 con 0x55f48839ef30 2024-09-11T23:38:37.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:37.594+0000 7f0105ffb640 10 client.27484 handle_command_reply: tid=0 2024-09-11T23:38:37.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:37.594+0000 7f0126c9c740 1 client.27484 shutdown 2024-09-11T23:38:37.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:37.595+0000 7f0126c9c740 1 -- 172.21.15.26:0/1597798838 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f488319080 msgr2=0x55f488327c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:37.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:37.595+0000 7f0126c9c740 1 --2- 172.21.15.26:0/1597798838 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f488319080 0x55f488327c10 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f01140098d0 tx=0x7f01140093c0 comp rx=0 tx=0).stop 2024-09-11T23:38:37.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:37.595+0000 7f0126c9c740 1 -- 172.21.15.26:0/1597798838 shutdown_connections 2024-09-11T23:38:37.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:37.595+0000 7f0126c9c740 1 --2- 172.21.15.26:0/1597798838 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f488323940 0x55f488328150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:37.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:37.595+0000 7f0126c9c740 1 --2- 172.21.15.26:0/1597798838 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55f48839ef30 0x55f4883a1350 crc :-1 s=READY pgs=303 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:37.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:37.595+0000 7f0126c9c740 1 --2- 172.21.15.26:0/1597798838 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f488319080 0x55f488327c10 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:37.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:37.595+0000 7f0105ffb640 0 client.27484 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:38:37.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:37.595+0000 7f0126c9c740 1 --2- 172.21.15.26:0/1597798838 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f488318760 0x55f4883276d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:37.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:37.595+0000 7f0126c9c740 1 -- 172.21.15.26:0/1597798838 >> 172.21.15.26:0/1597798838 conn(0x55f488314cf0 msgr2=0x55f48831a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:38:37.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:37.595+0000 7f0126c9c740 1 -- 172.21.15.26:0/1597798838 shutdown_connections 2024-09-11T23:38:37.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:37.595+0000 7f0126c9c740 1 -- 172.21.15.26:0/1597798838 wait complete. 2024-09-11T23:38:37.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:37.595+0000 7f0126c9c740 20 client.27484 trim_cache size 0 max 16384 2024-09-11T23:38:37.596 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-11T23:38:37.597 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 88, 2024-09-11T23:38:37.597 INFO:teuthology.orchestra.run.smithi026.stdout: "leader": 24485, 2024-09-11T23:38:37.597 INFO:teuthology.orchestra.run.smithi026.stdout: "set_version": 70, 2024-09-11T23:38:37.597 INFO:teuthology.orchestra.run.smithi026.stdout: "sets": { 2024-09-11T23:38:37.597 INFO:teuthology.orchestra.run.smithi026.stdout: "ad57dcac": { 2024-09-11T23:38:37.597 INFO:teuthology.orchestra.run.smithi026.stdout: "version": 70, 2024-09-11T23:38:37.597 INFO:teuthology.orchestra.run.smithi026.stdout: "age_ref": 0.0, 2024-09-11T23:38:37.597 INFO:teuthology.orchestra.run.smithi026.stdout: "state": { 2024-09-11T23:38:37.597 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "QUIESCED", 2024-09-11T23:38:37.597 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.0 2024-09-11T23:38:37.597 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:38:37.597 INFO:teuthology.orchestra.run.smithi026.stdout: "timeout": 90.0, 2024-09-11T23:38:37.598 INFO:teuthology.orchestra.run.smithi026.stdout: "expiration": 141.8, 2024-09-11T23:38:37.598 INFO:teuthology.orchestra.run.smithi026.stdout: "members": { 2024-09-11T23:38:37.598 INFO:teuthology.orchestra.run.smithi026.stdout: "file:/": { 2024-09-11T23:38:37.598 INFO:teuthology.orchestra.run.smithi026.stdout: "excluded": false, 2024-09-11T23:38:37.598 INFO:teuthology.orchestra.run.smithi026.stdout: "state": { 2024-09-11T23:38:37.598 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "QUIESCED", 2024-09-11T23:38:37.598 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.0 2024-09-11T23:38:37.598 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:38:37.598 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:38:37.598 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:38:37.598 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:38:37.598 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:38:37.598 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-11T23:38:37.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:37.596+0000 7f01248c3640 1 -- 172.21.15.26:0/308515937 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f01080a2d70 msgr2=0x7f01080a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:37.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:37.596+0000 7f01248c3640 1 --2- 172.21.15.26:0/308515937 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f01080a2d70 0x7f01080a5230 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f011000aba0 tx=0x7f011000a250 comp rx=0 tx=0).stop 2024-09-11T23:38:37.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:37.596+0000 7f01248c3640 1 -- 172.21.15.26:0/308515937 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0120100e20 msgr2=0x7f01200738d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:37.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:37.596+0000 7f01248c3640 1 --2- 172.21.15.26:0/308515937 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0120100e20 0x7f01200738d0 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f01180098d0 tx=0x7f01180308a0 comp rx=0 tx=0).stop 2024-09-11T23:38:37.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:37.596+0000 7f01248c3640 1 -- 172.21.15.26:0/308515937 shutdown_connections 2024-09-11T23:38:37.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:37.596+0000 7f01248c3640 1 --2- 172.21.15.26:0/308515937 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f01080a2d70 0x7f01080a5230 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:37.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:37.596+0000 7f01248c3640 1 --2- 172.21.15.26:0/308515937 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0120102ef0 0x7f012006ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:37.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:37.596+0000 7f01248c3640 1 --2- 172.21.15.26:0/308515937 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0120102570 0x7f0120073e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:37.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:37.596+0000 7f01248c3640 1 --2- 172.21.15.26:0/308515937 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0120100e20 0x7f01200738d0 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:37.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:37.596+0000 7f01248c3640 1 -- 172.21.15.26:0/308515937 >> 172.21.15.26:0/308515937 conn(0x7f01200fc7d0 msgr2=0x7f0120103d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:38:37.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:37.596+0000 7f01248c3640 1 -- 172.21.15.26:0/308515937 shutdown_connections 2024-09-11T23:38:37.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:37.596+0000 7f01248c3640 1 -- 172.21.15.26:0/308515937 wait complete. 2024-09-11T23:38:38.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:38:38 smithi026 ceph-mon[31172]: pgmap v637: 97 pgs: 20 active+clean+snaptrim, 37 active+clean+snaptrim_wait, 40 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 577 KiB/s rd, 572 KiB/s wr, 44 op/s 2024-09-11T23:38:38.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:38:38 smithi076 ceph-mon[35296]: pgmap v637: 97 pgs: 20 active+clean+snaptrim, 37 active+clean+snaptrim_wait, 40 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 577 KiB/s rd, 572 KiB/s wr, 44 op/s 2024-09-11T23:38:38.461 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'ad57dcac', quiesce took 7.9 seconds. Will release after: 13.9 2024-09-11T23:38:38.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:38:38 smithi136 ceph-mon[35121]: pgmap v637: 97 pgs: 20 active+clean+snaptrim, 37 active+clean+snaptrim_wait, 40 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 577 KiB/s rd, 572 KiB/s wr, 44 op/s 2024-09-11T23:38:38.897 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.895+0000 7f23cd191640 1 -- 172.21.15.26:0/3692083453 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f23c81095d0 msgr2=0x7f23c8109a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:38.897 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.895+0000 7f23cd191640 1 --2- 172.21.15.26:0/3692083453 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f23c81095d0 0x7f23c8109a10 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f23bc009930 tx=0x7f23bc02a420 comp rx=0 tx=0).stop 2024-09-11T23:38:38.897 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.896+0000 7f23cd191640 1 -- 172.21.15.26:0/3692083453 shutdown_connections 2024-09-11T23:38:38.897 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.896+0000 7f23cd191640 1 --2- 172.21.15.26:0/3692083453 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f23c811b140 0x7f23c811d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:38.897 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.896+0000 7f23cd191640 1 --2- 172.21.15.26:0/3692083453 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f23c81095d0 0x7f23c8109a10 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:38.897 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.896+0000 7f23cd191640 1 --2- 172.21.15.26:0/3692083453 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23c81091f0 0x7f23c8110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:38.897 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.896+0000 7f23cd191640 1 -- 172.21.15.26:0/3692083453 >> 172.21.15.26:0/3692083453 conn(0x7f23c806ce00 msgr2=0x7f23c806d210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:38:38.898 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.898+0000 7f23cd191640 1 -- 172.21.15.26:0/3692083453 shutdown_connections 2024-09-11T23:38:38.899 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.898+0000 7f23cd191640 1 -- 172.21.15.26:0/3692083453 wait complete. 2024-09-11T23:38:38.899 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.899+0000 7f23cd191640 1 Processor -- start 2024-09-11T23:38:38.899 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.899+0000 7f23cd191640 1 -- start start 2024-09-11T23:38:38.900 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.899+0000 7f23cd191640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f23c81091f0 0x7f23c81a7090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:38.900 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.899+0000 7f23cd191640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23c81095d0 0x7f23c81a75d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:38.902 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.899+0000 7f23cd191640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f23c811b140 0x7f23c81ab950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:38.902 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.900+0000 7f23c6d9d640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f23c81091f0 0x7f23c81a7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:38.902 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.900+0000 7f23c6d9d640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f23c81091f0 0x7f23c81a7090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:36564/0 (socket says 172.21.15.26:36564) 2024-09-11T23:38:38.902 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.900+0000 7f23c659c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23c81095d0 0x7f23c81a75d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:38.902 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.900+0000 7f23c6d9d640 1 -- 172.21.15.26:0/1639297266 learned_addr learned my addr 172.21.15.26:0/1639297266 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:38:38.902 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.900+0000 7f23cd191640 1 -- 172.21.15.26:0/1639297266 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f23c8120cf0 con 0x7f23c81095d0 2024-09-11T23:38:38.902 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.900+0000 7f23cd191640 1 -- 172.21.15.26:0/1639297266 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f23c8120b70 con 0x7f23c811b140 2024-09-11T23:38:38.902 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.900+0000 7f23cd191640 1 -- 172.21.15.26:0/1639297266 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f23c8120e70 con 0x7f23c81091f0 2024-09-11T23:38:38.903 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.900+0000 7f23c759e640 1 --2- 172.21.15.26:0/1639297266 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f23c811b140 0x7f23c81ab950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:38.903 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.900+0000 7f23c6d9d640 1 -- 172.21.15.26:0/1639297266 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f23c811b140 msgr2=0x7f23c81ab950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:38.903 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.900+0000 7f23c6d9d640 1 --2- 172.21.15.26:0/1639297266 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f23c811b140 0x7f23c81ab950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:38.903 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.900+0000 7f23c6d9d640 1 -- 172.21.15.26:0/1639297266 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23c81095d0 msgr2=0x7f23c81a75d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:38.903 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.900+0000 7f23c6d9d640 1 --2- 172.21.15.26:0/1639297266 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23c81095d0 0x7f23c81a75d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:38.903 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.900+0000 7f23c6d9d640 1 -- 172.21.15.26:0/1639297266 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23c81ac080 con 0x7f23c81091f0 2024-09-11T23:38:38.903 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.901+0000 7f23c6d9d640 1 --2- 172.21.15.26:0/1639297266 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f23c81091f0 0x7f23c81a7090 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f23b800c920 tx=0x7f23b800cde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:38:38.903 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.901+0000 7f23affff640 1 -- 172.21.15.26:0/1639297266 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23b8007c70 con 0x7f23c81091f0 2024-09-11T23:38:38.903 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.901+0000 7f23affff640 1 -- 172.21.15.26:0/1639297266 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f23b8007e10 con 0x7f23c81091f0 2024-09-11T23:38:38.903 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.901+0000 7f23affff640 1 -- 172.21.15.26:0/1639297266 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23b800e860 con 0x7f23c81091f0 2024-09-11T23:38:38.903 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.901+0000 7f23cd191640 1 -- 172.21.15.26:0/1639297266 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23c81ac370 con 0x7f23c81091f0 2024-09-11T23:38:38.903 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.901+0000 7f23cd191640 1 -- 172.21.15.26:0/1639297266 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23c807a450 con 0x7f23c81091f0 2024-09-11T23:38:38.905 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.905+0000 7f23affff640 1 -- 172.21.15.26:0/1639297266 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f23b800ea60 con 0x7f23c81091f0 2024-09-11T23:38:38.908 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.906+0000 7f23affff640 1 --2- 172.21.15.26:0/1639297266 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f23b00a2f80 0x7f23b00a5440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:38.909 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.906+0000 7f23affff640 1 -- 172.21.15.26:0/1639297266 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 8586+0+0 (secure 0 0 0) 0x7f23b80cf570 con 0x7f23c81091f0 2024-09-11T23:38:38.909 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.907+0000 7f23c659c640 1 --2- 172.21.15.26:0/1639297266 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f23b00a2f80 0x7f23b00a5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:38.909 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.908+0000 7f23c659c640 1 --2- 172.21.15.26:0/1639297266 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f23b00a2f80 0x7f23b00a5440 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f23bc0096c0 tx=0x7f23bc005fd0 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:38:38.909 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.909+0000 7f23d000b740 1 Processor -- start 2024-09-11T23:38:38.909 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.909+0000 7f23d000b740 1 -- start start 2024-09-11T23:38:38.909 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.909+0000 7f23d000b740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5563243deaf0 0x5563243deed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:38.909 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.909+0000 7f23d000b740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5563243df410 0x5563243e9770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:38.912 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.909+0000 7f23d000b740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5563243e9cb0 0x5563243ec0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:38.912 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.910+0000 7f23c6d9d640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5563243deaf0 0x5563243deed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:38.912 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.910+0000 7f23c6d9d640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5563243deaf0 0x5563243deed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:36566/0 (socket says 172.21.15.26:36566) 2024-09-11T23:38:38.912 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.910+0000 7f23c659c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5563243df410 0x5563243e9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:38.912 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.910+0000 7f23c6d9d640 1 -- 172.21.15.26:0/1558568076 learned_addr learned my addr 172.21.15.26:0/1558568076 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:38:38.912 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.910+0000 7f23d000b740 1 -- 172.21.15.26:0/1558568076 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5563243432f0 con 0x5563243df410 2024-09-11T23:38:38.912 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.910+0000 7f23d000b740 1 -- 172.21.15.26:0/1558568076 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5563242eb270 con 0x5563243e9cb0 2024-09-11T23:38:38.912 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.910+0000 7f23d000b740 1 -- 172.21.15.26:0/1558568076 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55632433f2b0 con 0x5563243deaf0 2024-09-11T23:38:38.913 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.910+0000 7f23c759e640 1 --2- 172.21.15.26:0/1558568076 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5563243e9cb0 0x5563243ec0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:38.913 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.910+0000 7f23c6d9d640 1 -- 172.21.15.26:0/1558568076 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5563243e9cb0 msgr2=0x5563243ec0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:38:38.913 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.910+0000 7f23c6d9d640 1 --2- 172.21.15.26:0/1558568076 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5563243e9cb0 0x5563243ec0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:38.913 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.910+0000 7f23c6d9d640 1 -- 172.21.15.26:0/1558568076 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5563243df410 msgr2=0x5563243e9770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:38:38.913 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.910+0000 7f23c6d9d640 1 --2- 172.21.15.26:0/1558568076 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5563243df410 0x5563243e9770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:38.913 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.910+0000 7f23c6d9d640 1 -- 172.21.15.26:0/1558568076 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5563242ea340 con 0x5563243deaf0 2024-09-11T23:38:38.913 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.910+0000 7f23c6d9d640 1 --2- 172.21.15.26:0/1558568076 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5563243deaf0 0x5563243deed0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f23b80a4130 tx=0x7f23b809b450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bf2cedab409e475d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:38:38.913 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.911+0000 7f23ad7fa640 1 -- 172.21.15.26:0/1558568076 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23b80b3020 con 0x5563243deaf0 2024-09-11T23:38:38.913 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.911+0000 7f23ad7fa640 1 -- 172.21.15.26:0/1558568076 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f23b809fda0 con 0x5563243deaf0 2024-09-11T23:38:38.913 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.911+0000 7f23ad7fa640 1 -- 172.21.15.26:0/1558568076 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23b80ccea0 con 0x5563243deaf0 2024-09-11T23:38:38.913 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.912+0000 7f23d000b740 1 -- 172.21.15.26:0/1558568076 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5563243deaf0 msgr2=0x5563243deed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:38.913 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.912+0000 7f23d000b740 1 --2- 172.21.15.26:0/1558568076 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5563243deaf0 0x5563243deed0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f23b80a4130 tx=0x7f23b809b450 comp rx=0 tx=0).stop 2024-09-11T23:38:38.913 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.912+0000 7f23d000b740 1 -- 172.21.15.26:0/1558568076 shutdown_connections 2024-09-11T23:38:38.914 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.912+0000 7f23d000b740 1 --2- 172.21.15.26:0/1558568076 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5563243e9cb0 0x5563243ec0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:38.914 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.912+0000 7f23d000b740 1 --2- 172.21.15.26:0/1558568076 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5563243df410 0x5563243e9770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:38.914 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.912+0000 7f23d000b740 1 --2- 172.21.15.26:0/1558568076 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5563243deaf0 0x5563243deed0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:38.914 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.912+0000 7f23d000b740 1 -- 172.21.15.26:0/1558568076 >> 172.21.15.26:0/1558568076 conn(0x5563243db060 msgr2=0x5563243db440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:38:38.914 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.913+0000 7f23d000b740 1 -- 172.21.15.26:0/1558568076 shutdown_connections 2024-09-11T23:38:38.916 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.913+0000 7f23d000b740 1 -- 172.21.15.26:0/1558568076 wait complete. 2024-09-11T23:38:38.916 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.913+0000 7f23d000b740 1 -- start start 2024-09-11T23:38:38.916 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.913+0000 7f23d000b740 1 Processor -- start 2024-09-11T23:38:38.916 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.914+0000 7f23d000b740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5563243deaf0 0x5563243e6c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:38.916 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.914+0000 7f23d000b740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5563243df410 0x5563243e71b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:38.916 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.914+0000 7f23d000b740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5563243e9cb0 0x5563243e76f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:38.916 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.914+0000 7f23c6d9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5563243deaf0 0x5563243e6c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:38.916 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.914+0000 7f23d000b740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5563243ec880 con 0x5563243df410 2024-09-11T23:38:38.917 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.914+0000 7f23d000b740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5563242cc290 con 0x5563243deaf0 2024-09-11T23:38:38.917 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.914+0000 7f23d000b740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5563243eca00 con 0x5563243e9cb0 2024-09-11T23:38:38.917 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.914+0000 7f23c6d9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5563243deaf0 0x5563243e6c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:43556/0 (socket says 172.21.15.26:43556) 2024-09-11T23:38:38.917 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.914+0000 7f23c6d9d640 1 -- 172.21.15.26:0/1921553768 learned_addr learned my addr 172.21.15.26:0/1921553768 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:38:38.917 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.914+0000 7f23d000b740 10 client.0 _ll_register_callbacks cb 0x5563242fd140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:38:38.917 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.914+0000 7f23c759e640 1 --2- 172.21.15.26:0/1921553768 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5563243e9cb0 0x5563243e76f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:38.917 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.915+0000 7f23c659c640 1 --2- 172.21.15.26:0/1921553768 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5563243df410 0x5563243e71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:38.917 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.915+0000 7f23c6d9d640 1 -- 172.21.15.26:0/1921553768 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5563243e9cb0 msgr2=0x5563243e76f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:38:38.917 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.915+0000 7f23c6d9d640 1 --2- 172.21.15.26:0/1921553768 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5563243e9cb0 0x5563243e76f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:38.917 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.915+0000 7f23c6d9d640 1 -- 172.21.15.26:0/1921553768 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5563243df410 msgr2=0x5563243e71b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:38:38.917 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.915+0000 7f23c6d9d640 1 --2- 172.21.15.26:0/1921553768 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5563243df410 0x5563243e71b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:38.917 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.915+0000 7f23c6d9d640 1 -- 172.21.15.26:0/1921553768 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556324056210 con 0x5563243deaf0 2024-09-11T23:38:38.918 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.916+0000 7f23c6d9d640 1 --2- 172.21.15.26:0/1921553768 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5563243deaf0 0x5563243e6c70 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f23b80983c0 tx=0x7f23b809b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=17b4c4a0dadf9382 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:38:38.918 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.916+0000 7f23aaff5640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-11T23:38:38.918 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.917+0000 7f23aaff5640 1 -- 172.21.15.26:0/1921553768 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23b80a8020 con 0x5563243deaf0 2024-09-11T23:38:38.918 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.917+0000 7f23d000b740 1 -- 172.21.15.26:0/1921553768 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f23b809fda0 con 0x5563243deaf0 2024-09-11T23:38:38.918 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.917+0000 7f23aaff5640 1 -- 172.21.15.26:0/1921553768 <== mon.2 v2:172.21.15.76:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 8586+0+0 (secure 0 0 0) 0x7f23b80980d0 con 0x5563243deaf0 2024-09-11T23:38:38.919 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.917+0000 7f23aaff5640 1 -- 172.21.15.26:0/1921553768 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f23b80b3040 con 0x5563243deaf0 2024-09-11T23:38:38.919 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.917+0000 7f23d000b740 10 client.27530 fetch_fsmap learned FSMap version 88 2024-09-11T23:38:38.919 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.917+0000 7f23d000b740 1 -- 172.21.15.26:0/1921553768 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=88,monmap=4+}) -- 0x556324122620 con 0x5563243deaf0 2024-09-11T23:38:38.919 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.918+0000 7f23aaff5640 1 -- 172.21.15.26:0/1921553768 <== mon.2 v2:172.21.15.76:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f23b809fda0 con 0x5563243deaf0 2024-09-11T23:38:38.919 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.918+0000 7f23aaff5640 1 -- 172.21.15.26:0/1921553768 <== mon.2 v2:172.21.15.76:3300/0 5 ==== fsmap(e 88) ==== 8426+0+0 (secure 0 0 0) 0x7f23b809a8b0 con 0x5563243deaf0 2024-09-11T23:38:38.919 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.918+0000 7f23d000b740 10 client.27530 fetch_fsmap finished waiting for FSMap version 88 2024-09-11T23:38:38.919 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.918+0000 7f23d000b740 10 client.27530 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:38:38.919 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.918+0000 7f23d000b740 20 client.27530 populate_metadata read hostname 'smithi026' 2024-09-11T23:38:38.919 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.918+0000 7f23d000b740 1 --2- 172.21.15.26:0/1921553768 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5563243f9b50 0x5563243fbf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:38.919 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.919+0000 7f23d000b740 4 client.27530 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:38:38.919 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.919+0000 7f23d000b740 1 -- 172.21.15.26:0/1921553768 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556324350a20 con 0x5563243f9b50 2024-09-11T23:38:38.920 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.920+0000 7f23c659c640 1 --2- 172.21.15.26:0/1921553768 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5563243f9b50 0x5563243fbf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:38.920 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.920+0000 7f23aaff5640 10 client.27530 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:38:38.920 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.920+0000 7f23c659c640 1 --2- 172.21.15.26:0/1921553768 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5563243f9b50 0x5563243fbf40 crc :-1 s=READY pgs=304 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d282aa82b709bb4a server_cookie=2b677f360204176e in_seq=0 out_seq=0 2024-09-11T23:38:38.921 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.921+0000 7f23aaff5640 1 -- 172.21.15.26:0/1921553768 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556324350a20 con 0x5563243f9b50 2024-09-11T23:38:38.921 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.921+0000 7f23aaff5640 10 client.27530 handle_command_reply: tid=0 2024-09-11T23:38:38.921 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.922+0000 7f23d000b740 1 client.27530 shutdown 2024-09-11T23:38:38.921 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.922+0000 7f23d000b740 1 -- 172.21.15.26:0/1921553768 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5563243deaf0 msgr2=0x5563243e6c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:38.921 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.922+0000 7f23d000b740 1 --2- 172.21.15.26:0/1921553768 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5563243deaf0 0x5563243e6c70 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f23b80983c0 tx=0x7f23b809b8f0 comp rx=0 tx=0).stop 2024-09-11T23:38:38.922 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.922+0000 7f23d000b740 1 -- 172.21.15.26:0/1921553768 shutdown_connections 2024-09-11T23:38:38.922 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.922+0000 7f23d000b740 1 --2- 172.21.15.26:0/1921553768 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5563243f9b50 0x5563243fbf40 crc :-1 s=READY pgs=304 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:38.922 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.922+0000 7f23d000b740 1 --2- 172.21.15.26:0/1921553768 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5563243e9cb0 0x5563243e76f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:38.922 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.922+0000 7f23d000b740 1 --2- 172.21.15.26:0/1921553768 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5563243df410 0x5563243e71b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:38.923 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.922+0000 7f23d000b740 1 --2- 172.21.15.26:0/1921553768 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5563243deaf0 0x5563243e6c70 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:38.923 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.922+0000 7f23d000b740 1 -- 172.21.15.26:0/1921553768 >> 172.21.15.26:0/1921553768 conn(0x5563243db060 msgr2=0x5563243e0dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:38:38.923 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.922+0000 7f23aaff5640 0 client.27530 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:38:38.923 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.922+0000 7f23d000b740 1 -- 172.21.15.26:0/1921553768 shutdown_connections 2024-09-11T23:38:38.923 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.922+0000 7f23d000b740 1 -- 172.21.15.26:0/1921553768 wait complete. 2024-09-11T23:38:38.923 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.923+0000 7f23d000b740 20 client.27530 trim_cache size 0 max 16384 2024-09-11T23:38:38.952 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.953+0000 7f23d000b740 1 Processor -- start 2024-09-11T23:38:38.952 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.953+0000 7f23d000b740 1 -- start start 2024-09-11T23:38:38.953 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.953+0000 7f23d000b740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5563243deaf0 0x5563243e7520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:38.954 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.953+0000 7f23d000b740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5563243df410 0x5563243e8be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:38.954 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.954+0000 7f23d000b740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5563243e9cb0 0x5563244014e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:38.954 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.954+0000 7f23d000b740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f23c8082fe0 con 0x5563243df410 2024-09-11T23:38:38.954 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.954+0000 7f23d000b740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f23c8084dc0 con 0x5563243e9cb0 2024-09-11T23:38:38.954 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.954+0000 7f23d000b740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f23c8083710 con 0x5563243deaf0 2024-09-11T23:38:38.954 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.954+0000 7f23c759e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5563243e9cb0 0x5563244014e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:38.954 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.954+0000 7f23c759e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5563243e9cb0 0x5563244014e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:43570/0 (socket says 172.21.15.26:43570) 2024-09-11T23:38:38.954 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.954+0000 7f23c759e640 1 -- 172.21.15.26:0/3164337532 learned_addr learned my addr 172.21.15.26:0/3164337532 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:38:38.954 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.954+0000 7f23c659c640 1 --2- 172.21.15.26:0/3164337532 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5563243df410 0x5563243e8be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:38.955 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.955+0000 7f23c659c640 1 -- 172.21.15.26:0/3164337532 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5563243deaf0 msgr2=0x5563243e7520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:38:38.955 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.955+0000 7f23c659c640 1 --2- 172.21.15.26:0/3164337532 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5563243deaf0 0x5563243e7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:38.955 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.955+0000 7f23c659c640 1 -- 172.21.15.26:0/3164337532 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5563243e9cb0 msgr2=0x5563244014e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:38:38.955 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.955+0000 7f23c659c640 1 --2- 172.21.15.26:0/3164337532 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5563243e9cb0 0x5563244014e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:38.955 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.955+0000 7f23c659c640 1 -- 172.21.15.26:0/3164337532 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5563243ec940 con 0x5563243df410 2024-09-11T23:38:38.955 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.955+0000 7f23c759e640 1 --2- 172.21.15.26:0/3164337532 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5563243e9cb0 0x5563244014e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:38:38.956 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.955+0000 7f23c659c640 1 --2- 172.21.15.26:0/3164337532 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5563243df410 0x5563243e8be0 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f23bc02aa10 tx=0x7f23bc03b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=894e57d2c799796 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:38:38.956 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.955+0000 7f23acff9640 1 -- 172.21.15.26:0/3164337532 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23bc035050 con 0x5563243df410 2024-09-11T23:38:38.956 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.955+0000 7f23acff9640 1 -- 172.21.15.26:0/3164337532 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f23bc057070 con 0x5563243df410 2024-09-11T23:38:38.956 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.955+0000 7f23acff9640 1 -- 172.21.15.26:0/3164337532 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23bc032b00 con 0x5563243df410 2024-09-11T23:38:38.956 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.957+0000 7f23d000b740 1 -- 172.21.15.26:0/3164337532 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5563243df410 msgr2=0x5563243e8be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:38.956 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.957+0000 7f23d000b740 1 --2- 172.21.15.26:0/3164337532 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5563243df410 0x5563243e8be0 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f23bc02aa10 tx=0x7f23bc03b000 comp rx=0 tx=0).stop 2024-09-11T23:38:38.959 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.959+0000 7f23d000b740 1 -- 172.21.15.26:0/3164337532 shutdown_connections 2024-09-11T23:38:38.959 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.959+0000 7f23d000b740 1 --2- 172.21.15.26:0/3164337532 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5563243e9cb0 0x5563244014e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:38.959 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.959+0000 7f23d000b740 1 --2- 172.21.15.26:0/3164337532 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5563243df410 0x5563243e8be0 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:38.959 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.959+0000 7f23d000b740 1 --2- 172.21.15.26:0/3164337532 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5563243deaf0 0x5563243e7520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:38.959 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.959+0000 7f23d000b740 1 -- 172.21.15.26:0/3164337532 >> 172.21.15.26:0/3164337532 conn(0x5563243db060 msgr2=0x5563243e84d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:38:38.960 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.959+0000 7f23d000b740 1 -- 172.21.15.26:0/3164337532 shutdown_connections 2024-09-11T23:38:38.960 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.959+0000 7f23d000b740 1 -- 172.21.15.26:0/3164337532 wait complete. 2024-09-11T23:38:38.960 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.959+0000 7f23d000b740 1 -- start start 2024-09-11T23:38:38.960 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.959+0000 7f23d000b740 1 Processor -- start 2024-09-11T23:38:38.960 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.960+0000 7f23d000b740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5563243deaf0 0x5563243ecd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:38.960 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.960+0000 7f23d000b740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5563243df410 0x5563243ed250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:38.960 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.960+0000 7f23d000b740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5563243e9cb0 0x5563244624a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:38.960 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.960+0000 7f23d000b740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f23c8084630 con 0x5563243deaf0 2024-09-11T23:38:38.960 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.960+0000 7f23d000b740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f23c8083ea0 con 0x5563243df410 2024-09-11T23:38:38.960 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.960+0000 7f23d000b740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f23c8109a50 con 0x5563243e9cb0 2024-09-11T23:38:38.960 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.960+0000 7f23d000b740 10 client.0 _ll_register_callbacks cb 0x7f23c000a8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:38:38.960 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.960+0000 7f23c6d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5563243deaf0 0x5563243ecd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:38.962 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.960+0000 7f23c6d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5563243deaf0 0x5563243ecd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60444/0 (socket says 172.21.15.26:60444) 2024-09-11T23:38:38.962 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.960+0000 7f23c6d9d640 1 -- 172.21.15.26:0/191207063 learned_addr learned my addr 172.21.15.26:0/191207063 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:38:38.962 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.960+0000 7f23c759e640 1 --2- 172.21.15.26:0/191207063 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5563243e9cb0 0x5563244624a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:38.962 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.960+0000 7f23c6d9d640 1 -- 172.21.15.26:0/191207063 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5563243e9cb0 msgr2=0x5563244624a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:38:38.963 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.960+0000 7f23c6d9d640 1 --2- 172.21.15.26:0/191207063 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5563243e9cb0 0x5563244624a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:38.963 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.960+0000 7f23c6d9d640 1 -- 172.21.15.26:0/191207063 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5563243df410 msgr2=0x5563243ed250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:38:38.963 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.960+0000 7f23c6d9d640 1 --2- 172.21.15.26:0/191207063 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5563243df410 0x5563243ed250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:38.963 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.960+0000 7f23c6d9d640 1 -- 172.21.15.26:0/191207063 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5563244629e0 con 0x5563243deaf0 2024-09-11T23:38:38.963 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.961+0000 7f23c6d9d640 1 --2- 172.21.15.26:0/191207063 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5563243deaf0 0x5563243ecd10 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f23b800c8f0 tx=0x7f23b800fdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5d425145e4fa140 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:38:38.963 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.961+0000 7f23aaff5640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:38:38.963 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.961+0000 7f23aaff5640 1 -- 172.21.15.26:0/191207063 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23b80a8070 con 0x5563243deaf0 2024-09-11T23:38:38.963 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.961+0000 7f23d000b740 1 -- 172.21.15.26:0/191207063 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f23bc057070 con 0x5563243deaf0 2024-09-11T23:38:38.963 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.961+0000 7f23aaff5640 1 -- 172.21.15.26:0/191207063 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 8586+0+0 (secure 0 0 0) 0x7f23b809a8b0 con 0x5563243deaf0 2024-09-11T23:38:38.963 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.961+0000 7f23aaff5640 1 -- 172.21.15.26:0/191207063 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f23b809b350 con 0x5563243deaf0 2024-09-11T23:38:38.963 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.961+0000 7f23d000b740 10 client.18024 fetch_fsmap learned FSMap version 88 2024-09-11T23:38:38.963 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.961+0000 7f23d000b740 1 -- 172.21.15.26:0/191207063 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=88,monmap=4+}) -- 0x5563244629e0 con 0x5563243deaf0 2024-09-11T23:38:38.963 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.961+0000 7f23aaff5640 1 -- 172.21.15.26:0/191207063 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f23bc057070 con 0x5563243deaf0 2024-09-11T23:38:38.963 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.962+0000 7f23aaff5640 1 -- 172.21.15.26:0/191207063 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 88) ==== 8426+0+0 (secure 0 0 0) 0x7f23b80ccea0 con 0x5563243deaf0 2024-09-11T23:38:38.964 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.962+0000 7f23d000b740 10 client.18024 fetch_fsmap finished waiting for FSMap version 88 2024-09-11T23:38:38.964 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.962+0000 7f23d000b740 10 client.18024 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:38:38.964 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.962+0000 7f23d000b740 20 client.18024 populate_metadata read hostname 'smithi026' 2024-09-11T23:38:38.964 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.962+0000 7f23d000b740 1 --2- 172.21.15.26:0/191207063 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x556324463f70 0x556324466390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:38.964 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.962+0000 7f23d000b740 4 client.18024 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T23:38:38.964 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.962+0000 7f23d000b740 1 -- 172.21.15.26:0/191207063 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55632434ad70 con 0x556324463f70 2024-09-11T23:38:38.964 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.962+0000 7f23c659c640 1 --2- 172.21.15.26:0/191207063 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x556324463f70 0x556324466390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:38.964 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.963+0000 7f23c659c640 1 --2- 172.21.15.26:0/191207063 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x556324463f70 0x556324466390 crc :-1 s=READY pgs=305 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=339955f4e14ff00c server_cookie=8550b198d436b3df in_seq=0 out_seq=0 2024-09-11T23:38:38.964 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:38.963+0000 7f23aaff5640 10 client.18024 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:38:39.050 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:39.050+0000 7f23aaff5640 1 -- 172.21.15.26:0/191207063 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55632434ad70 con 0x556324463f70 2024-09-11T23:38:39.050 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:39.050+0000 7f23aaff5640 10 client.18024 handle_command_reply: tid=0 2024-09-11T23:38:39.050 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:39.050+0000 7f23d000b740 1 client.18024 shutdown 2024-09-11T23:38:39.050 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:39.050+0000 7f23d000b740 1 -- 172.21.15.26:0/191207063 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5563243deaf0 msgr2=0x5563243ecd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:39.051 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:39.050+0000 7f23d000b740 1 --2- 172.21.15.26:0/191207063 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5563243deaf0 0x5563243ecd10 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f23b800c8f0 tx=0x7f23b800fdb0 comp rx=0 tx=0).stop 2024-09-11T23:38:39.052 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:39.051+0000 7f23d000b740 1 -- 172.21.15.26:0/191207063 shutdown_connections 2024-09-11T23:38:39.052 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:39.051+0000 7f23d000b740 1 --2- 172.21.15.26:0/191207063 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5563243df410 0x5563243ed250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:39.052 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:39.051+0000 7f23d000b740 1 --2- 172.21.15.26:0/191207063 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x556324463f70 0x556324466390 crc :-1 s=READY pgs=305 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:39.052 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:39.051+0000 7f23d000b740 1 --2- 172.21.15.26:0/191207063 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5563243e9cb0 0x5563244624a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:39.052 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:39.051+0000 7f23d000b740 1 --2- 172.21.15.26:0/191207063 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5563243deaf0 0x5563243ecd10 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:39.053 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:39.051+0000 7f23d000b740 1 -- 172.21.15.26:0/191207063 >> 172.21.15.26:0/191207063 conn(0x5563243db060 msgr2=0x5563243e50f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:38:39.053 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:39.051+0000 7f23aaff5640 0 client.18024 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:38:39.053 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:39.051+0000 7f23d000b740 1 -- 172.21.15.26:0/191207063 shutdown_connections 2024-09-11T23:38:39.053 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:39.051+0000 7f23d000b740 1 -- 172.21.15.26:0/191207063 wait complete. 2024-09-11T23:38:39.053 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:39.051+0000 7f23d000b740 20 client.18024 trim_cache size 0 max 16384 2024-09-11T23:38:39.054 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-09-11T23:38:39.054 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "message": "", 2024-09-11T23:38:39.054 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-09-11T23:38:39.054 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-09-11T23:38:39.055 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:39.053+0000 7f23cd191640 1 -- 172.21.15.26:0/1639297266 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f23b00a2f80 msgr2=0x7f23b00a5440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:39.055 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:39.053+0000 7f23cd191640 1 --2- 172.21.15.26:0/1639297266 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f23b00a2f80 0x7f23b00a5440 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f23bc0096c0 tx=0x7f23bc005fd0 comp rx=0 tx=0).stop 2024-09-11T23:38:39.055 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:39.053+0000 7f23cd191640 1 -- 172.21.15.26:0/1639297266 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f23c81091f0 msgr2=0x7f23c81a7090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:39.055 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:39.053+0000 7f23cd191640 1 --2- 172.21.15.26:0/1639297266 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f23c81091f0 0x7f23c81a7090 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f23b800c920 tx=0x7f23b800cde0 comp rx=0 tx=0).stop 2024-09-11T23:38:39.056 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:39.053+0000 7f23cd191640 1 -- 172.21.15.26:0/1639297266 shutdown_connections 2024-09-11T23:38:39.056 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:39.053+0000 7f23cd191640 1 --2- 172.21.15.26:0/1639297266 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f23b00a2f80 0x7f23b00a5440 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:39.056 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:39.053+0000 7f23cd191640 1 --2- 172.21.15.26:0/1639297266 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f23c811b140 0x7f23c81ab950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:39.056 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:39.053+0000 7f23cd191640 1 --2- 172.21.15.26:0/1639297266 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23c81095d0 0x7f23c81a75d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:39.056 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:39.053+0000 7f23cd191640 1 --2- 172.21.15.26:0/1639297266 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f23c81091f0 0x7f23c81a7090 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:39.056 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:39.053+0000 7f23cd191640 1 -- 172.21.15.26:0/1639297266 >> 172.21.15.26:0/1639297266 conn(0x7f23c806ce00 msgr2=0x7f23c811b5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:38:39.056 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:39.053+0000 7f23cd191640 1 -- 172.21.15.26:0/1639297266 shutdown_connections 2024-09-11T23:38:39.057 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:39.053+0000 7f23cd191640 1 -- 172.21.15.26:0/1639297266 wait complete. 2024-09-11T23:38:40.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:38:40 smithi076 ceph-mon[35296]: pgmap v638: 97 pgs: 20 active+clean+snaptrim, 37 active+clean+snaptrim_wait, 40 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 574 KiB/s rd, 571 KiB/s wr, 44 op/s 2024-09-11T23:38:40.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:38:40 smithi136 ceph-mon[35121]: pgmap v638: 97 pgs: 20 active+clean+snaptrim, 37 active+clean+snaptrim_wait, 40 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 574 KiB/s rd, 571 KiB/s wr, 44 op/s 2024-09-11T23:38:40.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:38:40 smithi026 ceph-mon[31172]: pgmap v638: 97 pgs: 20 active+clean+snaptrim, 37 active+clean+snaptrim_wait, 40 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 574 KiB/s rd, 571 KiB/s wr, 44 op/s 2024-09-11T23:38:42.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:38:42 smithi076 ceph-mon[35296]: pgmap v639: 97 pgs: 19 active+clean+snaptrim, 37 active+clean+snaptrim_wait, 41 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 686 KiB/s rd, 683 KiB/s wr, 62 op/s 2024-09-11T23:38:42.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:38:42 smithi136 ceph-mon[35121]: pgmap v639: 97 pgs: 19 active+clean+snaptrim, 37 active+clean+snaptrim_wait, 41 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 686 KiB/s rd, 683 KiB/s wr, 62 op/s 2024-09-11T23:38:42.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:38:42 smithi026 ceph-mon[31172]: pgmap v639: 97 pgs: 19 active+clean+snaptrim, 37 active+clean+snaptrim_wait, 41 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 686 KiB/s rd, 683 KiB/s wr, 62 op/s 2024-09-11T23:38:44.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:38:44 smithi076 ceph-mon[35296]: pgmap v640: 97 pgs: 15 active+clean+snaptrim, 32 active+clean+snaptrim_wait, 50 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 687 KiB/s rd, 683 KiB/s wr, 73 op/s 2024-09-11T23:38:44.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:38:44 smithi136 ceph-mon[35121]: pgmap v640: 97 pgs: 15 active+clean+snaptrim, 32 active+clean+snaptrim_wait, 50 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 687 KiB/s rd, 683 KiB/s wr, 73 op/s 2024-09-11T23:38:44.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:38:44 smithi026 ceph-mon[31172]: pgmap v640: 97 pgs: 15 active+clean+snaptrim, 32 active+clean+snaptrim_wait, 50 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 687 KiB/s rd, 683 KiB/s wr, 73 op/s 2024-09-11T23:38:46.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:38:46 smithi076 ceph-mon[35296]: pgmap v641: 97 pgs: 13 active+clean+snaptrim, 30 active+clean+snaptrim_wait, 54 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 685 KiB/s rd, 683 KiB/s wr, 69 op/s 2024-09-11T23:38:46.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:38:46 smithi136 ceph-mon[35121]: pgmap v641: 97 pgs: 13 active+clean+snaptrim, 30 active+clean+snaptrim_wait, 54 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 685 KiB/s rd, 683 KiB/s wr, 69 op/s 2024-09-11T23:38:46.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:38:46 smithi026 ceph-mon[31172]: pgmap v641: 97 pgs: 13 active+clean+snaptrim, 30 active+clean+snaptrim_wait, 54 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 685 KiB/s rd, 683 KiB/s wr, 69 op/s 2024-09-11T23:38:48.305 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:38:48 smithi076 ceph-mon[35296]: pgmap v642: 97 pgs: 11 active+clean+snaptrim, 26 active+clean+snaptrim_wait, 60 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 688 KiB/s rd, 684 KiB/s wr, 70 op/s 2024-09-11T23:38:48.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:38:48 smithi136 ceph-mon[35121]: pgmap v642: 97 pgs: 11 active+clean+snaptrim, 26 active+clean+snaptrim_wait, 60 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 688 KiB/s rd, 684 KiB/s wr, 70 op/s 2024-09-11T23:38:48.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:38:48 smithi026 ceph-mon[31172]: pgmap v642: 97 pgs: 11 active+clean+snaptrim, 26 active+clean+snaptrim_wait, 60 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 688 KiB/s rd, 684 KiB/s wr, 70 op/s 2024-09-11T23:38:49.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:38:49 smithi136 ceph-mon[35121]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:38:49.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:38:49 smithi026 ceph-mon[31172]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:38:49.552 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:38:49 smithi076 ceph-mon[35296]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:38:50.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:38:50 smithi136 ceph-mon[35121]: pgmap v643: 97 pgs: 11 active+clean+snaptrim, 26 active+clean+snaptrim_wait, 60 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 117 KiB/s wr, 39 op/s 2024-09-11T23:38:50.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:38:50 smithi026 ceph-mon[31172]: pgmap v643: 97 pgs: 11 active+clean+snaptrim, 26 active+clean+snaptrim_wait, 60 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 117 KiB/s wr, 39 op/s 2024-09-11T23:38:50.552 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:38:50 smithi076 ceph-mon[35296]: pgmap v643: 97 pgs: 11 active+clean+snaptrim, 26 active+clean+snaptrim_wait, 60 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 117 KiB/s wr, 39 op/s 2024-09-11T23:38:52.156 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.153+0000 7f112f41e640 1 -- 172.21.15.26:0/1191472572 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1128102480 msgr2=0x7f112810e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:52.156 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.153+0000 7f112f41e640 1 --2- 172.21.15.26:0/1191472572 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1128102480 0x7f112810e9b0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f1120009cd0 tx=0x7f112002a8e0 comp rx=0 tx=0).stop 2024-09-11T23:38:52.156 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.153+0000 7f112f41e640 1 -- 172.21.15.26:0/1191472572 shutdown_connections 2024-09-11T23:38:52.156 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.153+0000 7f112f41e640 1 --2- 172.21.15.26:0/1191472572 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1128102480 0x7f112810e9b0 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:52.156 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.153+0000 7f112f41e640 1 --2- 172.21.15.26:0/1191472572 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1128101ae0 0x7f1128101f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:52.156 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.153+0000 7f112f41e640 1 --2- 172.21.15.26:0/1191472572 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1128107b80 0x7f1128107f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:52.157 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.153+0000 7f112f41e640 1 -- 172.21.15.26:0/1191472572 >> 172.21.15.26:0/1191472572 conn(0x7f11280fd7e0 msgr2=0x7f11280ffc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:38:52.157 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.153+0000 7f112f41e640 1 -- 172.21.15.26:0/1191472572 shutdown_connections 2024-09-11T23:38:52.157 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.153+0000 7f112f41e640 1 -- 172.21.15.26:0/1191472572 wait complete. 2024-09-11T23:38:52.157 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.154+0000 7f112f41e640 1 Processor -- start 2024-09-11T23:38:52.157 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.154+0000 7f112f41e640 1 -- start start 2024-09-11T23:38:52.157 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.154+0000 7f112f41e640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1128101ae0 0x7f1128198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:52.157 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.154+0000 7f112f41e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1128102480 0x7f1128198840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:52.157 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.154+0000 7f112f41e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1128107b80 0x7f112819cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:52.157 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.154+0000 7f112f41e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f1128111f30 con 0x7f1128102480 2024-09-11T23:38:52.157 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.154+0000 7f112f41e640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f1128111db0 con 0x7f1128107b80 2024-09-11T23:38:52.157 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.154+0000 7f112f41e640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f11281120b0 con 0x7f1128101ae0 2024-09-11T23:38:52.157 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.154+0000 7f112c9b9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1128102480 0x7f1128198840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:52.157 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.154+0000 7f112c9b9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1128102480 0x7f1128198840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50940/0 (socket says 172.21.15.26:50940) 2024-09-11T23:38:52.158 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.154+0000 7f112c9b9640 1 -- 172.21.15.26:0/2306677510 learned_addr learned my addr 172.21.15.26:0/2306677510 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:38:52.158 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.154+0000 7f112d9bb640 1 --2- 172.21.15.26:0/2306677510 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1128107b80 0x7f112819cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:52.158 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.154+0000 7f112d1ba640 1 --2- 172.21.15.26:0/2306677510 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1128101ae0 0x7f1128198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:52.158 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.155+0000 7f112c9b9640 1 -- 172.21.15.26:0/2306677510 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1128101ae0 msgr2=0x7f1128198300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:52.158 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.155+0000 7f112c9b9640 1 --2- 172.21.15.26:0/2306677510 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1128101ae0 0x7f1128198300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:52.158 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.155+0000 7f112c9b9640 1 -- 172.21.15.26:0/2306677510 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1128107b80 msgr2=0x7f112819cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:52.158 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.155+0000 7f112c9b9640 1 --2- 172.21.15.26:0/2306677510 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1128107b80 0x7f112819cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:52.158 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.155+0000 7f112c9b9640 1 -- 172.21.15.26:0/2306677510 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f112819d2f0 con 0x7f1128102480 2024-09-11T23:38:52.158 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.155+0000 7f112d9bb640 1 --2- 172.21.15.26:0/2306677510 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1128107b80 0x7f112819cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:38:52.158 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.155+0000 7f112d1ba640 1 --2- 172.21.15.26:0/2306677510 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1128101ae0 0x7f1128198300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:38:52.158 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.155+0000 7f112c9b9640 1 --2- 172.21.15.26:0/2306677510 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1128102480 0x7f1128198840 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f111800e9f0 tx=0x7f111800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:38:52.158 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.155+0000 7f111e7fc640 1 -- 172.21.15.26:0/2306677510 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f111800cd80 con 0x7f1128102480 2024-09-11T23:38:52.158 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.155+0000 7f112f41e640 1 -- 172.21.15.26:0/2306677510 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f112819d5e0 con 0x7f1128102480 2024-09-11T23:38:52.159 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.155+0000 7f111e7fc640 1 -- 172.21.15.26:0/2306677510 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1118005500 con 0x7f1128102480 2024-09-11T23:38:52.159 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.155+0000 7f111e7fc640 1 -- 172.21.15.26:0/2306677510 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1118010690 con 0x7f1128102480 2024-09-11T23:38:52.159 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.155+0000 7f112f41e640 1 -- 172.21.15.26:0/2306677510 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11281a4e70 con 0x7f1128102480 2024-09-11T23:38:52.166 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.157+0000 7f111e7fc640 1 -- 172.21.15.26:0/2306677510 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f11180046c0 con 0x7f1128102480 2024-09-11T23:38:52.166 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.158+0000 7f1132290740 1 Processor -- start 2024-09-11T23:38:52.167 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.158+0000 7f1132290740 1 -- start start 2024-09-11T23:38:52.167 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.158+0000 7f1132290740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55abb5cc09d0 0x55abb5cc0db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:52.167 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.158+0000 7f1132290740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55abb5cc12f0 0x55abb5ccfb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:52.167 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.158+0000 7f1132290740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55abb5cd00c0 0x55abb5cd24b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:52.167 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.158+0000 7f1132290740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55abb5c252f0 con 0x55abb5cc09d0 2024-09-11T23:38:52.167 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.158+0000 7f1132290740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55abb5bcd270 con 0x55abb5cd00c0 2024-09-11T23:38:52.167 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.158+0000 7f1132290740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55abb5c212b0 con 0x55abb5cc12f0 2024-09-11T23:38:52.167 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.158+0000 7f112d9bb640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55abb5cc09d0 0x55abb5cc0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:52.167 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.158+0000 7f111e7fc640 1 --2- 172.21.15.26:0/2306677510 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f11140a2da0 0x7f11140a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:52.167 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.158+0000 7f112c9b9640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55abb5cd00c0 0x55abb5cd24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:52.167 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.158+0000 7f112d9bb640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55abb5cc09d0 0x55abb5cc0db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50942/0 (socket says 172.21.15.26:50942) 2024-09-11T23:38:52.167 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.158+0000 7f112d9bb640 1 -- 172.21.15.26:0/2426173107 learned_addr learned my addr 172.21.15.26:0/2426173107 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:38:52.168 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.158+0000 7f112d1ba640 1 --2- 172.21.15.26:0/2426173107 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55abb5cc12f0 0x55abb5ccfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:52.168 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.158+0000 7f112d9bb640 1 --2- 172.21.15.26:0/2306677510 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f11140a2da0 0x7f11140a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:52.168 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.159+0000 7f111e7fc640 1 -- 172.21.15.26:0/2306677510 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 8586+0+0 (secure 0 0 0) 0x7f11180cfa30 con 0x7f1128102480 2024-09-11T23:38:52.168 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.159+0000 7f112c9b9640 1 -- 172.21.15.26:0/2426173107 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55abb5cc12f0 msgr2=0x55abb5ccfb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:38:52.168 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.159+0000 7f112c9b9640 1 --2- 172.21.15.26:0/2426173107 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55abb5cc12f0 0x55abb5ccfb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:52.168 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.159+0000 7f112c9b9640 1 -- 172.21.15.26:0/2426173107 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55abb5cc09d0 msgr2=0x55abb5cc0db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:38:52.168 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.159+0000 7f112c9b9640 1 --2- 172.21.15.26:0/2426173107 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55abb5cc09d0 0x55abb5cc0db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:52.168 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.159+0000 7f112c9b9640 1 -- 172.21.15.26:0/2426173107 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55abb5bcc340 con 0x55abb5cd00c0 2024-09-11T23:38:52.168 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.159+0000 7f112d1ba640 1 --2- 172.21.15.26:0/2426173107 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55abb5cc12f0 0x55abb5ccfb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:38:52.168 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.159+0000 7f112c9b9640 1 --2- 172.21.15.26:0/2426173107 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55abb5cd00c0 0x55abb5cd24b0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f11180cd0e0 tx=0x7f1118098280 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9af00fa321679a3c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:38:52.168 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.159+0000 7f112d9bb640 1 --2- 172.21.15.26:0/2306677510 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f11140a2da0 0x7f11140a5260 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f1120031860 tx=0x7f112002a8e0 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:38:52.168 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.159+0000 7f11137fe640 1 -- 172.21.15.26:0/2426173107 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f111809b810 con 0x55abb5cd00c0 2024-09-11T23:38:52.169 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.159+0000 7f11137fe640 1 -- 172.21.15.26:0/2426173107 <== mon.2 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f111809c5b0 con 0x55abb5cd00c0 2024-09-11T23:38:52.169 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.159+0000 7f11137fe640 1 -- 172.21.15.26:0/2426173107 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11180cd360 con 0x55abb5cd00c0 2024-09-11T23:38:52.169 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.160+0000 7f1132290740 1 -- 172.21.15.26:0/2426173107 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55abb5cd00c0 msgr2=0x55abb5cd24b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:52.169 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.160+0000 7f1132290740 1 --2- 172.21.15.26:0/2426173107 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55abb5cd00c0 0x55abb5cd24b0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f11180cd0e0 tx=0x7f1118098280 comp rx=0 tx=0).stop 2024-09-11T23:38:52.169 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.160+0000 7f1132290740 1 -- 172.21.15.26:0/2426173107 shutdown_connections 2024-09-11T23:38:52.169 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.160+0000 7f1132290740 1 --2- 172.21.15.26:0/2426173107 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55abb5cd00c0 0x55abb5cd24b0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:52.169 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.160+0000 7f1132290740 1 --2- 172.21.15.26:0/2426173107 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55abb5cc12f0 0x55abb5ccfb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:52.169 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.160+0000 7f1132290740 1 --2- 172.21.15.26:0/2426173107 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55abb5cc09d0 0x55abb5cc0db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:52.170 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.160+0000 7f1132290740 1 -- 172.21.15.26:0/2426173107 >> 172.21.15.26:0/2426173107 conn(0x55abb5cbd060 msgr2=0x55abb5cbd440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:38:52.170 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.160+0000 7f1132290740 1 -- 172.21.15.26:0/2426173107 shutdown_connections 2024-09-11T23:38:52.170 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.160+0000 7f1132290740 1 -- 172.21.15.26:0/2426173107 wait complete. 2024-09-11T23:38:52.170 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.160+0000 7f1132290740 1 -- start start 2024-09-11T23:38:52.170 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.160+0000 7f1132290740 1 Processor -- start 2024-09-11T23:38:52.170 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.161+0000 7f1132290740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55abb5cc09d0 0x55abb5ccd000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:52.170 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.161+0000 7f1132290740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55abb5cc12f0 0x55abb5ccd540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:52.170 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.161+0000 7f1132290740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55abb5cd00c0 0x55abb5ccda80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:52.170 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.161+0000 7f1132290740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55abb5cd2cf0 con 0x55abb5cc12f0 2024-09-11T23:38:52.170 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.161+0000 7f1132290740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55abb5bae290 con 0x55abb5cd00c0 2024-09-11T23:38:52.170 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.161+0000 7f1132290740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55abb5cd2e70 con 0x55abb5cc09d0 2024-09-11T23:38:52.171 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.161+0000 7f1132290740 10 client.0 _ll_register_callbacks cb 0x55abb5bdf140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:38:52.171 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.161+0000 7f112c9b9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55abb5cc12f0 0x55abb5ccd540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:52.171 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.161+0000 7f112c9b9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55abb5cc12f0 0x55abb5ccd540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50948/0 (socket says 172.21.15.26:50948) 2024-09-11T23:38:52.171 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.161+0000 7f112c9b9640 1 -- 172.21.15.26:0/1368345056 learned_addr learned my addr 172.21.15.26:0/1368345056 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:38:52.171 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.161+0000 7f112d9bb640 1 --2- 172.21.15.26:0/1368345056 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55abb5cd00c0 0x55abb5ccda80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:52.171 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.161+0000 7f112d1ba640 1 --2- 172.21.15.26:0/1368345056 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55abb5cc09d0 0x55abb5ccd000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:52.171 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.161+0000 7f112c9b9640 1 -- 172.21.15.26:0/1368345056 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55abb5cc09d0 msgr2=0x55abb5ccd000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:38:52.171 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.161+0000 7f112c9b9640 1 --2- 172.21.15.26:0/1368345056 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55abb5cc09d0 0x55abb5ccd000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:52.171 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.161+0000 7f112c9b9640 1 -- 172.21.15.26:0/1368345056 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55abb5cd00c0 msgr2=0x55abb5ccda80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:38:52.171 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.161+0000 7f112c9b9640 1 --2- 172.21.15.26:0/1368345056 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55abb5cd00c0 0x55abb5ccda80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:52.171 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.161+0000 7f112c9b9640 1 -- 172.21.15.26:0/1368345056 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55abb5938210 con 0x55abb5cc12f0 2024-09-11T23:38:52.171 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.161+0000 7f112c9b9640 1 --2- 172.21.15.26:0/1368345056 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55abb5cc12f0 0x55abb5ccd540 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f1118098310 tx=0x7f111809beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae4deec643b91737 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:38:52.171 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.161+0000 7f1110ff9640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:38:52.172 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.161+0000 7f1110ff9640 1 -- 172.21.15.26:0/1368345056 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11180a4b20 con 0x55abb5cc12f0 2024-09-11T23:38:52.172 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.161+0000 7f1132290740 1 -- 172.21.15.26:0/1368345056 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f111809c5b0 con 0x55abb5cc12f0 2024-09-11T23:38:52.172 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.161+0000 7f1110ff9640 1 -- 172.21.15.26:0/1368345056 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 8586+0+0 (secure 0 0 0) 0x7f11180cd360 con 0x55abb5cc12f0 2024-09-11T23:38:52.172 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.162+0000 7f1110ff9640 1 -- 172.21.15.26:0/1368345056 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f111809c110 con 0x55abb5cc12f0 2024-09-11T23:38:52.172 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.162+0000 7f1132290740 10 client.18036 fetch_fsmap learned FSMap version 89 2024-09-11T23:38:52.172 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.162+0000 7f1132290740 1 -- 172.21.15.26:0/1368345056 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=89,monmap=4+}) -- 0x55abb5a04620 con 0x55abb5cc12f0 2024-09-11T23:38:52.172 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.162+0000 7f1110ff9640 1 -- 172.21.15.26:0/1368345056 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f111809c5b0 con 0x55abb5cc12f0 2024-09-11T23:38:52.172 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.162+0000 7f1110ff9640 1 -- 172.21.15.26:0/1368345056 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 89) ==== 8422+0+0 (secure 0 0 0) 0x7f111809b420 con 0x55abb5cc12f0 2024-09-11T23:38:52.172 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.162+0000 7f1132290740 10 client.18036 fetch_fsmap finished waiting for FSMap version 89 2024-09-11T23:38:52.172 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.162+0000 7f1132290740 10 client.18036 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:38:52.172 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.162+0000 7f1132290740 20 client.18036 populate_metadata read hostname 'smithi026' 2024-09-11T23:38:52.172 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.162+0000 7f1132290740 1 --2- 172.21.15.26:0/1368345056 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55abb5cdff50 0x55abb5ce2340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:52.172 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.162+0000 7f1132290740 4 client.18036 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:38:52.173 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.162+0000 7f1132290740 1 -- 172.21.15.26:0/1368345056 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55abb5c32a20 con 0x55abb5cdff50 2024-09-11T23:38:52.173 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.162+0000 7f112d1ba640 1 --2- 172.21.15.26:0/1368345056 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55abb5cdff50 0x55abb5ce2340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:52.173 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.162+0000 7f112d1ba640 1 --2- 172.21.15.26:0/1368345056 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55abb5cdff50 0x55abb5ce2340 crc :-1 s=READY pgs=306 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cde33af1cadc43a2 server_cookie=53fb2fd350166a61 in_seq=0 out_seq=0 2024-09-11T23:38:52.173 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.162+0000 7f1110ff9640 10 client.18036 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:38:52.173 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.163+0000 7f1110ff9640 1 -- 172.21.15.26:0/1368345056 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55abb5c32a20 con 0x55abb5cdff50 2024-09-11T23:38:52.173 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.163+0000 7f1110ff9640 10 client.18036 handle_command_reply: tid=0 2024-09-11T23:38:52.173 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.163+0000 7f1132290740 1 client.18036 shutdown 2024-09-11T23:38:52.173 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.163+0000 7f1132290740 1 -- 172.21.15.26:0/1368345056 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55abb5cc12f0 msgr2=0x55abb5ccd540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:52.173 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.163+0000 7f1132290740 1 --2- 172.21.15.26:0/1368345056 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55abb5cc12f0 0x55abb5ccd540 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f1118098310 tx=0x7f111809beb0 comp rx=0 tx=0).stop 2024-09-11T23:38:52.173 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.163+0000 7f1132290740 1 -- 172.21.15.26:0/1368345056 shutdown_connections 2024-09-11T23:38:52.173 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.163+0000 7f1132290740 1 --2- 172.21.15.26:0/1368345056 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55abb5cd00c0 0x55abb5ccda80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:52.173 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.163+0000 7f1132290740 1 --2- 172.21.15.26:0/1368345056 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55abb5cdff50 0x55abb5ce2340 crc :-1 s=READY pgs=306 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:52.173 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.163+0000 7f1132290740 1 --2- 172.21.15.26:0/1368345056 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55abb5cc12f0 0x55abb5ccd540 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:52.173 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.163+0000 7f1132290740 1 --2- 172.21.15.26:0/1368345056 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55abb5cc09d0 0x55abb5ccd000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:52.174 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.163+0000 7f1110ff9640 0 client.18036 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:38:52.174 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.163+0000 7f1132290740 1 -- 172.21.15.26:0/1368345056 >> 172.21.15.26:0/1368345056 conn(0x55abb5cbd060 msgr2=0x55abb5cc2dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:38:52.174 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.163+0000 7f1132290740 1 -- 172.21.15.26:0/1368345056 shutdown_connections 2024-09-11T23:38:52.174 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.163+0000 7f1132290740 1 -- 172.21.15.26:0/1368345056 wait complete. 2024-09-11T23:38:52.174 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.164+0000 7f1132290740 20 client.18036 trim_cache size 0 max 16384 2024-09-11T23:38:52.183 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.178+0000 7f1132290740 1 Processor -- start 2024-09-11T23:38:52.183 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.178+0000 7f1132290740 1 -- start start 2024-09-11T23:38:52.183 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.178+0000 7f1132290740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55abb5cc09d0 0x55abb5ccd8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:52.183 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.178+0000 7f1132290740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55abb5cc12f0 0x55abb5ce5360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:52.183 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.178+0000 7f1132290740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55abb5cd00c0 0x55abb5ce58a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:52.183 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.178+0000 7f1132290740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f11281ab760 con 0x55abb5cc12f0 2024-09-11T23:38:52.184 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.178+0000 7f1132290740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f11281ad510 con 0x55abb5cd00c0 2024-09-11T23:38:52.184 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.178+0000 7f1132290740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f11281abe60 con 0x55abb5cc09d0 2024-09-11T23:38:52.184 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.178+0000 7f112c9b9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55abb5cc12f0 0x55abb5ce5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:52.184 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.178+0000 7f112c9b9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55abb5cc12f0 0x55abb5ce5360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50962/0 (socket says 172.21.15.26:50962) 2024-09-11T23:38:52.184 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.179+0000 7f112c9b9640 1 -- 172.21.15.26:0/718089063 learned_addr learned my addr 172.21.15.26:0/718089063 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:38:52.184 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.179+0000 7f112d1ba640 1 --2- 172.21.15.26:0/718089063 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55abb5cc09d0 0x55abb5ccd8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:52.184 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.179+0000 7f112c9b9640 1 -- 172.21.15.26:0/718089063 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55abb5cc09d0 msgr2=0x55abb5ccd8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:38:52.184 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.179+0000 7f112c9b9640 1 --2- 172.21.15.26:0/718089063 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55abb5cc09d0 0x55abb5ccd8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:52.184 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.179+0000 7f112c9b9640 1 -- 172.21.15.26:0/718089063 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55abb5cd00c0 msgr2=0x55abb5ce58a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:38:52.184 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.179+0000 7f112d9bb640 1 --2- 172.21.15.26:0/718089063 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55abb5cd00c0 0x55abb5ce58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:52.184 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.179+0000 7f112c9b9640 1 --2- 172.21.15.26:0/718089063 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55abb5cd00c0 0x55abb5ce58a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:52.185 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.179+0000 7f112c9b9640 1 -- 172.21.15.26:0/718089063 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55abb5ccbf90 con 0x55abb5cc12f0 2024-09-11T23:38:52.185 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.179+0000 7f112d9bb640 1 --2- 172.21.15.26:0/718089063 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55abb5cd00c0 0x55abb5ce58a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:38:52.185 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.179+0000 7f112c9b9640 1 --2- 172.21.15.26:0/718089063 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55abb5cc12f0 0x55abb5ce5360 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f111809bdf0 tx=0x7f1118017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e120f399f031afee server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:38:52.185 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.179+0000 7f1112ffd640 1 -- 172.21.15.26:0/718089063 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11180a4b20 con 0x55abb5cc12f0 2024-09-11T23:38:52.185 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.179+0000 7f1112ffd640 1 -- 172.21.15.26:0/718089063 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f111809c5b0 con 0x55abb5cc12f0 2024-09-11T23:38:52.185 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.179+0000 7f1112ffd640 1 -- 172.21.15.26:0/718089063 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11180a4690 con 0x55abb5cc12f0 2024-09-11T23:38:52.186 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.180+0000 7f1132290740 1 -- 172.21.15.26:0/718089063 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55abb5cc12f0 msgr2=0x55abb5ce5360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:52.186 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.180+0000 7f1132290740 1 --2- 172.21.15.26:0/718089063 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55abb5cc12f0 0x55abb5ce5360 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f111809bdf0 tx=0x7f1118017000 comp rx=0 tx=0).stop 2024-09-11T23:38:52.186 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.180+0000 7f1132290740 1 -- 172.21.15.26:0/718089063 shutdown_connections 2024-09-11T23:38:52.186 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.180+0000 7f1132290740 1 --2- 172.21.15.26:0/718089063 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55abb5cd00c0 0x55abb5ce58a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:52.186 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.180+0000 7f1132290740 1 --2- 172.21.15.26:0/718089063 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55abb5cc12f0 0x55abb5ce5360 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:52.186 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.180+0000 7f1132290740 1 --2- 172.21.15.26:0/718089063 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55abb5cc09d0 0x55abb5ccd8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:52.186 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.180+0000 7f1132290740 1 -- 172.21.15.26:0/718089063 >> 172.21.15.26:0/718089063 conn(0x55abb5cbd060 msgr2=0x55abb5ccf080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:38:52.186 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.180+0000 7f1132290740 1 -- 172.21.15.26:0/718089063 shutdown_connections 2024-09-11T23:38:52.186 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.180+0000 7f1132290740 1 -- 172.21.15.26:0/718089063 wait complete. 2024-09-11T23:38:52.186 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.180+0000 7f1132290740 1 -- start start 2024-09-11T23:38:52.187 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.180+0000 7f1132290740 1 Processor -- start 2024-09-11T23:38:52.187 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.181+0000 7f1132290740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55abb5cc09d0 0x55abb5ce5c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:52.187 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.181+0000 7f1132290740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55abb5cc12f0 0x55abb5ce2f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:52.187 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.181+0000 7f1132290740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55abb5cd00c0 0x55abb5ce3710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:52.187 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.181+0000 7f1132290740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f11281acd80 con 0x55abb5cc09d0 2024-09-11T23:38:52.187 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.181+0000 7f1132290740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f11281ac5f0 con 0x55abb5cc12f0 2024-09-11T23:38:52.187 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.181+0000 7f1132290740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f11281a9760 con 0x55abb5cd00c0 2024-09-11T23:38:52.187 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.181+0000 7f1132290740 10 client.0 _ll_register_callbacks cb 0x7f1124005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:38:52.187 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.181+0000 7f112d1ba640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55abb5cc09d0 0x55abb5ce5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:52.187 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.181+0000 7f112d1ba640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55abb5cc09d0 0x55abb5ce5c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50974/0 (socket says 172.21.15.26:50974) 2024-09-11T23:38:52.187 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.181+0000 7f112d1ba640 1 -- 172.21.15.26:0/26621538 learned_addr learned my addr 172.21.15.26:0/26621538 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:38:52.187 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.181+0000 7f112c9b9640 1 --2- 172.21.15.26:0/26621538 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55abb5cc12f0 0x55abb5ce2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:52.187 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.181+0000 7f112d9bb640 1 --2- 172.21.15.26:0/26621538 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55abb5cd00c0 0x55abb5ce3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:52.187 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.181+0000 7f112d1ba640 1 -- 172.21.15.26:0/26621538 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55abb5cd00c0 msgr2=0x55abb5ce3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:38:52.188 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.181+0000 7f112d1ba640 1 --2- 172.21.15.26:0/26621538 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55abb5cd00c0 0x55abb5ce3710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:52.188 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.181+0000 7f112d1ba640 1 -- 172.21.15.26:0/26621538 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55abb5cc12f0 msgr2=0x55abb5ce2f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:38:52.188 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.181+0000 7f112d1ba640 1 --2- 172.21.15.26:0/26621538 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55abb5cc12f0 0x55abb5ce2f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:52.188 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.181+0000 7f112d1ba640 1 -- 172.21.15.26:0/26621538 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55abb5d49890 con 0x55abb5cc09d0 2024-09-11T23:38:52.188 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.181+0000 7f112c9b9640 1 --2- 172.21.15.26:0/26621538 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55abb5cc12f0 0x55abb5ce2f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:38:52.188 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.181+0000 7f112d1ba640 1 --2- 172.21.15.26:0/26621538 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55abb5cc09d0 0x55abb5ce5c40 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x55abb5bcd380 tx=0x7f1124004410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c9b226ac77629ee server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:38:52.188 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.181+0000 7f1110ff9640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:38:52.188 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.181+0000 7f1110ff9640 1 -- 172.21.15.26:0/26621538 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f112401e380 con 0x55abb5cc09d0 2024-09-11T23:38:52.188 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.181+0000 7f1110ff9640 1 -- 172.21.15.26:0/26621538 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 8586+0+0 (secure 0 0 0) 0x7f112401e520 con 0x55abb5cc09d0 2024-09-11T23:38:52.188 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.181+0000 7f1132290740 1 -- 172.21.15.26:0/26621538 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f111809c5b0 con 0x55abb5cc09d0 2024-09-11T23:38:52.188 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.182+0000 7f1110ff9640 1 -- 172.21.15.26:0/26621538 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f112402e050 con 0x55abb5cc09d0 2024-09-11T23:38:52.188 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.182+0000 7f1132290740 10 client.18042 fetch_fsmap learned FSMap version 89 2024-09-11T23:38:52.188 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.182+0000 7f1132290740 1 -- 172.21.15.26:0/26621538 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=89,monmap=4+}) -- 0x55abb5d4aa00 con 0x55abb5cc09d0 2024-09-11T23:38:52.189 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.182+0000 7f1110ff9640 1 -- 172.21.15.26:0/26621538 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f111809c5b0 con 0x55abb5cc09d0 2024-09-11T23:38:52.189 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.182+0000 7f1110ff9640 1 -- 172.21.15.26:0/26621538 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 89) ==== 8422+0+0 (secure 0 0 0) 0x7f112400f040 con 0x55abb5cc09d0 2024-09-11T23:38:52.189 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.182+0000 7f1132290740 10 client.18042 fetch_fsmap finished waiting for FSMap version 89 2024-09-11T23:38:52.189 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.182+0000 7f1132290740 10 client.18042 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:38:52.189 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.182+0000 7f1132290740 20 client.18042 populate_metadata read hostname 'smithi026' 2024-09-11T23:38:52.189 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.182+0000 7f1132290740 1 --2- 172.21.15.26:0/26621538 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55abb5d4b1f0 0x55abb5d4d610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:52.189 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.182+0000 7f1132290740 4 client.18042 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T23:38:52.189 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.182+0000 7f1132290740 1 -- 172.21.15.26:0/26621538 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55abb5c2cd70 con 0x55abb5d4b1f0 2024-09-11T23:38:52.189 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.182+0000 7f112c9b9640 1 --2- 172.21.15.26:0/26621538 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55abb5d4b1f0 0x55abb5d4d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:52.189 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.183+0000 7f1110ff9640 10 client.18042 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:38:52.189 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.183+0000 7f112c9b9640 1 --2- 172.21.15.26:0/26621538 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55abb5d4b1f0 0x55abb5d4d610 crc :-1 s=READY pgs=307 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=375248f799207a16 server_cookie=7522645cf54e5a47 in_seq=0 out_seq=0 2024-09-11T23:38:52.192 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.190+0000 7f1110ff9640 1 -- 172.21.15.26:0/26621538 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55abb5c2cd70 con 0x55abb5d4b1f0 2024-09-11T23:38:52.192 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.190+0000 7f1110ff9640 10 client.18042 handle_command_reply: tid=0 2024-09-11T23:38:52.192 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.190+0000 7f1132290740 1 client.18042 shutdown 2024-09-11T23:38:52.192 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.191+0000 7f1132290740 1 -- 172.21.15.26:0/26621538 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55abb5cc09d0 msgr2=0x55abb5ce5c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:52.192 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.191+0000 7f1132290740 1 --2- 172.21.15.26:0/26621538 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55abb5cc09d0 0x55abb5ce5c40 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x55abb5bcd380 tx=0x7f1124004410 comp rx=0 tx=0).stop 2024-09-11T23:38:52.192 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.191+0000 7f1132290740 1 -- 172.21.15.26:0/26621538 shutdown_connections 2024-09-11T23:38:52.192 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.191+0000 7f1132290740 1 --2- 172.21.15.26:0/26621538 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55abb5cc12f0 0x55abb5ce2f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:52.192 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.191+0000 7f1132290740 1 --2- 172.21.15.26:0/26621538 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55abb5d4b1f0 0x55abb5d4d610 crc :-1 s=READY pgs=307 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:52.192 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.191+0000 7f1132290740 1 --2- 172.21.15.26:0/26621538 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55abb5cd00c0 0x55abb5ce3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:52.192 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.191+0000 7f1132290740 1 --2- 172.21.15.26:0/26621538 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55abb5cc09d0 0x55abb5ce5c40 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:52.193 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.191+0000 7f1110ff9640 0 client.18042 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:38:52.193 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.191+0000 7f1132290740 1 -- 172.21.15.26:0/26621538 >> 172.21.15.26:0/26621538 conn(0x55abb5cbd060 msgr2=0x55abb5cc25a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:38:52.193 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.191+0000 7f1132290740 1 -- 172.21.15.26:0/26621538 shutdown_connections 2024-09-11T23:38:52.193 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.191+0000 7f1132290740 1 -- 172.21.15.26:0/26621538 wait complete. 2024-09-11T23:38:52.193 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.191+0000 7f1132290740 20 client.18042 trim_cache size 0 max 16384 2024-09-11T23:38:52.193 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-09-11T23:38:52.193 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "message": "", 2024-09-11T23:38:52.194 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-09-11T23:38:52.194 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-09-11T23:38:52.194 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.191+0000 7f112f41e640 1 -- 172.21.15.26:0/2306677510 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f11140a2da0 msgr2=0x7f11140a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:52.195 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.191+0000 7f112f41e640 1 --2- 172.21.15.26:0/2306677510 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f11140a2da0 0x7f11140a5260 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f1120031860 tx=0x7f112002a8e0 comp rx=0 tx=0).stop 2024-09-11T23:38:52.195 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.191+0000 7f112f41e640 1 -- 172.21.15.26:0/2306677510 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1128102480 msgr2=0x7f1128198840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:52.195 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.191+0000 7f112f41e640 1 --2- 172.21.15.26:0/2306677510 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1128102480 0x7f1128198840 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f111800e9f0 tx=0x7f111800edf0 comp rx=0 tx=0).stop 2024-09-11T23:38:52.195 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.192+0000 7f112f41e640 1 -- 172.21.15.26:0/2306677510 shutdown_connections 2024-09-11T23:38:52.195 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.192+0000 7f112f41e640 1 --2- 172.21.15.26:0/2306677510 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f11140a2da0 0x7f11140a5260 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:52.195 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.192+0000 7f112f41e640 1 --2- 172.21.15.26:0/2306677510 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1128107b80 0x7f112819cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:52.195 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.192+0000 7f112f41e640 1 --2- 172.21.15.26:0/2306677510 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1128102480 0x7f1128198840 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:52.195 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.192+0000 7f112f41e640 1 --2- 172.21.15.26:0/2306677510 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1128101ae0 0x7f1128198300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:52.195 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.192+0000 7f112f41e640 1 -- 172.21.15.26:0/2306677510 >> 172.21.15.26:0/2306677510 conn(0x7f11280fd7e0 msgr2=0x7f11280fdd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:38:52.195 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.192+0000 7f112f41e640 1 -- 172.21.15.26:0/2306677510 shutdown_connections 2024-09-11T23:38:52.195 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:38:52.192+0000 7f112f41e640 1 -- 172.21.15.26:0/2306677510 wait complete. 2024-09-11T23:38:52.363 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:38:52.388 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:38:52 smithi026 ceph-mon[31172]: pgmap v644: 97 pgs: 7 active+clean+snaptrim, 17 active+clean+snaptrim_wait, 73 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 117 KiB/s wr, 39 op/s 2024-09-11T23:38:52.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:38:52 smithi136 ceph-mon[35121]: pgmap v644: 97 pgs: 7 active+clean+snaptrim, 17 active+clean+snaptrim_wait, 73 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 117 KiB/s wr, 39 op/s 2024-09-11T23:38:52.552 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:38:52 smithi076 ceph-mon[35296]: pgmap v644: 97 pgs: 7 active+clean+snaptrim, 17 active+clean+snaptrim_wait, 73 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 117 KiB/s wr, 39 op/s 2024-09-11T23:38:52.626 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:38:53.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:53.117+0000 7f00732f5640 1 -- 172.21.15.26:0/2994690781 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f006c10c170 msgr2=0x7f006c10e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:53.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:53.117+0000 7f00732f5640 1 --2- 172.21.15.26:0/2994690781 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f006c10c170 0x7f006c10e610 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f0064009ca0 tx=0x7f006402a950 comp rx=0 tx=0).stop 2024-09-11T23:38:53.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:53.117+0000 7f00732f5640 1 -- 172.21.15.26:0/2994690781 shutdown_connections 2024-09-11T23:38:53.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:53.117+0000 7f00732f5640 1 --2- 172.21.15.26:0/2994690781 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f006c103660 0x7f006c10bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:53.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:53.117+0000 7f00732f5640 1 --2- 172.21.15.26:0/2994690781 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f006c10c170 0x7f006c10e610 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:53.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:53.117+0000 7f00732f5640 1 --2- 172.21.15.26:0/2994690781 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f006c102d40 0x7f006c103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:53.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:53.117+0000 7f00732f5640 1 -- 172.21.15.26:0/2994690781 >> 172.21.15.26:0/2994690781 conn(0x7f006c0fc7a0 msgr2=0x7f006c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:38:53.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:53.117+0000 7f00732f5640 1 -- 172.21.15.26:0/2994690781 shutdown_connections 2024-09-11T23:38:53.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:53.117+0000 7f00732f5640 1 -- 172.21.15.26:0/2994690781 wait complete. 2024-09-11T23:38:53.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:53.117+0000 7f00732f5640 1 Processor -- start 2024-09-11T23:38:53.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:53.118+0000 7f00732f5640 1 -- start start 2024-09-11T23:38:53.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:53.118+0000 7f00732f5640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f006c102d40 0x7f006c198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:53.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:53.118+0000 7f00732f5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f006c103660 0x7f006c1988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:53.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:53.118+0000 7f00732f5640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f006c10c170 0x7f006c19cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:53.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:53.118+0000 7f00732f5640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f006c111f70 con 0x7f006c103660 2024-09-11T23:38:53.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:53.118+0000 7f00732f5640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f006c111df0 con 0x7f006c102d40 2024-09-11T23:38:53.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:53.118+0000 7f00732f5640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f006c1120f0 con 0x7f006c10c170 2024-09-11T23:38:53.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:53.118+0000 7f0070890640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f006c103660 0x7f006c1988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:53.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:53.118+0000 7f0070890640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f006c103660 0x7f006c1988b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50998/0 (socket says 172.21.15.26:50998) 2024-09-11T23:38:53.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:53.118+0000 7f0070890640 1 -- 172.21.15.26:0/2973699690 learned_addr learned my addr 172.21.15.26:0/2973699690 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:38:53.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:53.118+0000 7f0071091640 1 --2- 172.21.15.26:0/2973699690 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f006c102d40 0x7f006c198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:53.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:53.118+0000 7f0071892640 1 --2- 172.21.15.26:0/2973699690 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f006c10c170 0x7f006c19cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:53.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:53.118+0000 7f0070890640 1 -- 172.21.15.26:0/2973699690 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f006c10c170 msgr2=0x7f006c19cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:53.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:53.118+0000 7f0070890640 1 --2- 172.21.15.26:0/2973699690 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f006c10c170 0x7f006c19cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:53.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:53.118+0000 7f0070890640 1 -- 172.21.15.26:0/2973699690 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f006c102d40 msgr2=0x7f006c198370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:53.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:53.118+0000 7f0070890640 1 --2- 172.21.15.26:0/2973699690 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f006c102d40 0x7f006c198370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:53.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:53.118+0000 7f0070890640 1 -- 172.21.15.26:0/2973699690 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f006c19d320 con 0x7f006c103660 2024-09-11T23:38:53.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:53.118+0000 7f0070890640 1 --2- 172.21.15.26:0/2973699690 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f006c103660 0x7f006c1988b0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f005c00c970 tx=0x7f005c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:38:53.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:53.119+0000 7f00627fc640 1 -- 172.21.15.26:0/2973699690 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f005c007bb0 con 0x7f006c103660 2024-09-11T23:38:53.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:53.119+0000 7f00627fc640 1 -- 172.21.15.26:0/2973699690 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f005c007d50 con 0x7f006c103660 2024-09-11T23:38:53.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:53.119+0000 7f00627fc640 1 -- 172.21.15.26:0/2973699690 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f005c018820 con 0x7f006c103660 2024-09-11T23:38:53.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:53.119+0000 7f00732f5640 1 -- 172.21.15.26:0/2973699690 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f006c19d610 con 0x7f006c103660 2024-09-11T23:38:53.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:53.119+0000 7f00732f5640 1 -- 172.21.15.26:0/2973699690 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f006c1a4ef0 con 0x7f006c103660 2024-09-11T23:38:53.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:53.120+0000 7f00732f5640 1 -- 172.21.15.26:0/2973699690 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f006c103120 con 0x7f006c103660 2024-09-11T23:38:53.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:53.122+0000 7f00627fc640 1 -- 172.21.15.26:0/2973699690 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f005c0189c0 con 0x7f006c103660 2024-09-11T23:38:53.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:53.122+0000 7f00627fc640 1 --2- 172.21.15.26:0/2973699690 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f00580a2cf0 0x7f00580a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:53.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:53.122+0000 7f0071091640 1 --2- 172.21.15.26:0/2973699690 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f00580a2cf0 0x7f00580a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:53.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:53.122+0000 7f00627fc640 1 -- 172.21.15.26:0/2973699690 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 8586+0+0 (secure 0 0 0) 0x7f005c0cf780 con 0x7f006c103660 2024-09-11T23:38:53.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:53.123+0000 7f0071091640 1 --2- 172.21.15.26:0/2973699690 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f00580a2cf0 0x7f00580a51b0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f00680058c0 tx=0x7f006800a2d0 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:38:53.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:53.126+0000 7f00627fc640 1 -- 172.21.15.26:0/2973699690 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f005c014030 con 0x7f006c103660 2024-09-11T23:38:53.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:53.339+0000 7f00732f5640 1 -- 172.21.15.26:0/2973699690 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f006c1a51e0 con 0x7f006c103660 2024-09-11T23:38:53.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:53.340+0000 7f00627fc640 1 -- 172.21.15.26:0/2973699690 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 89 v89) ==== 94+0+11997 (secure 0 0 0) 0x7f005c09c3f0 con 0x7f006c103660 2024-09-11T23:38:53.340 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-11T23:38:53.340 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":89,"btime":"2024-09-11T23:38:48:048219+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14499,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/367659515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":367659515},{"type":"v1","addr":"172.21.15.26:6835","nonce":367659515}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14511,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/2041065299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":2041065299},{"type":"v1","addr":"172.21.15.26:6837","nonce":2041065299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":89,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:16:26.544203+0000","modified":"2024-09-11T23:38:47.872939+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24485,"mds_1":24463,"mds_2":24458,"mds_3":24448,"mds_4":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14550":{"gid":14550,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6839/2233068305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":2233068305},{"type":"v1","addr":"172.21.15.26:6839","nonce":2233068305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14562":{"gid":14562,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6841/2984982336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":2984982336},{"type":"v1","addr":"172.21.15.26:6841","nonce":2984982336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6833/2019841455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":2019841455},{"type":"v1","addr":"172.21.15.136:6833","nonce":2019841455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.76:6833/3438147050","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3438147050},{"type":"v1","addr":"172.21.15.76:6833","nonce":3438147050}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6835/3617665975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3617665975},{"type":"v1","addr":"172.21.15.136:6835","nonce":3617665975}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":12,"addr":"172.21.15.76:6835/393308499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":393308499},{"type":"v1","addr":"172.21.15.76:6835","nonce":393308499}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":10,"addr":"172.21.15.136:6837/1633082387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":1633082387},{"type":"v1","addr":"172.21.15.136:6837","nonce":1633082387}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.76:6837/2078978748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":2078978748},{"type":"v1","addr":"172.21.15.76:6837","nonce":2078978748}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":63,"addr":"172.21.15.136:6839/3211398630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":3211398630},{"type":"v1","addr":"172.21.15.136:6839","nonce":3211398630}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.76:6839/2077637090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6838","nonce":2077637090},{"type":"v1","addr":"172.21.15.76:6839","nonce":2077637090}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24485,"qdb_cluster":[24443,24448,24458,24463,24485]},"id":1}]} 2024-09-11T23:38:53.341 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 89 2024-09-11T23:38:53.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:53.342+0000 7f00732f5640 1 -- 172.21.15.26:0/2973699690 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f00580a2cf0 msgr2=0x7f00580a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:53.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:53.342+0000 7f00732f5640 1 --2- 172.21.15.26:0/2973699690 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f00580a2cf0 0x7f00580a51b0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f00680058c0 tx=0x7f006800a2d0 comp rx=0 tx=0).stop 2024-09-11T23:38:53.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:53.342+0000 7f00732f5640 1 -- 172.21.15.26:0/2973699690 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f006c103660 msgr2=0x7f006c1988b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:53.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:53.342+0000 7f00732f5640 1 --2- 172.21.15.26:0/2973699690 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f006c103660 0x7f006c1988b0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f005c00c970 tx=0x7f005c00ce30 comp rx=0 tx=0).stop 2024-09-11T23:38:53.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:53.342+0000 7f00732f5640 1 -- 172.21.15.26:0/2973699690 shutdown_connections 2024-09-11T23:38:53.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:53.342+0000 7f00732f5640 1 --2- 172.21.15.26:0/2973699690 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f00580a2cf0 0x7f00580a51b0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:53.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:53.342+0000 7f00732f5640 1 --2- 172.21.15.26:0/2973699690 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f006c10c170 0x7f006c19cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:53.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:53.342+0000 7f00732f5640 1 --2- 172.21.15.26:0/2973699690 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f006c103660 0x7f006c1988b0 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:53.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:53.342+0000 7f00732f5640 1 --2- 172.21.15.26:0/2973699690 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f006c102d40 0x7f006c198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:53.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:53.342+0000 7f00732f5640 1 -- 172.21.15.26:0/2973699690 >> 172.21.15.26:0/2973699690 conn(0x7f006c0fc7a0 msgr2=0x7f006c0fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:38:53.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:53.342+0000 7f00732f5640 1 -- 172.21.15.26:0/2973699690 shutdown_connections 2024-09-11T23:38:53.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:53.342+0000 7f00732f5640 1 -- 172.21.15.26:0/2973699690 wait complete. 2024-09-11T23:38:54.342 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:38:54 smithi026 ceph-mon[31172]: pgmap v645: 97 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 89 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.1 KiB/s wr, 22 op/s 2024-09-11T23:38:54.342 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:38:54 smithi026 ceph-mon[31172]: from='client.? 172.21.15.26:0/2973699690' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:38:54.344 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24485 quiesce db --set-id ad57dcac --release --await' 2024-09-11T23:38:54.344 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24485 quiesce db --set-id ad57dcac --release --await 2024-09-11T23:38:54.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:38:54 smithi136 ceph-mon[35121]: pgmap v645: 97 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 89 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.1 KiB/s wr, 22 op/s 2024-09-11T23:38:54.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:38:54 smithi136 ceph-mon[35121]: from='client.? 172.21.15.26:0/2973699690' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:38:54.552 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:38:54 smithi076 ceph-mon[35296]: pgmap v645: 97 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 89 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.1 KiB/s wr, 22 op/s 2024-09-11T23:38:54.552 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:38:54 smithi076 ceph-mon[35296]: from='client.? 172.21.15.26:0/2973699690' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:38:54.650 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:38:55.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.286+0000 7f5e0cfb6640 1 -- 172.21.15.26:0/127478724 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5e0810c170 msgr2=0x7f5e0810e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:55.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.286+0000 7f5e0cfb6640 1 --2- 172.21.15.26:0/127478724 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5e0810c170 0x7f5e0810e610 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f5e00009ca0 tx=0x7f5e0002a950 comp rx=0 tx=0).stop 2024-09-11T23:38:55.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.286+0000 7f5e0cfb6640 1 -- 172.21.15.26:0/127478724 shutdown_connections 2024-09-11T23:38:55.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.286+0000 7f5e0cfb6640 1 --2- 172.21.15.26:0/127478724 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5e0810c170 0x7f5e0810e610 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:55.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.286+0000 7f5e0cfb6640 1 --2- 172.21.15.26:0/127478724 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5e08103660 0x7f5e0810bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:55.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.286+0000 7f5e0cfb6640 1 --2- 172.21.15.26:0/127478724 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5e08102d40 0x7f5e08103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:55.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.286+0000 7f5e0cfb6640 1 -- 172.21.15.26:0/127478724 >> 172.21.15.26:0/127478724 conn(0x7f5e080fc7a0 msgr2=0x7f5e080febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:38:55.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.286+0000 7f5e0cfb6640 1 -- 172.21.15.26:0/127478724 shutdown_connections 2024-09-11T23:38:55.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.286+0000 7f5e0cfb6640 1 -- 172.21.15.26:0/127478724 wait complete. 2024-09-11T23:38:55.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.286+0000 7f5e0cfb6640 1 Processor -- start 2024-09-11T23:38:55.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.287+0000 7f5e0cfb6640 1 -- start start 2024-09-11T23:38:55.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.287+0000 7f5e0cfb6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5e08102d40 0x7f5e081009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:55.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.287+0000 7f5e0cfb6640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5e08103660 0x7f5e08100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:55.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.287+0000 7f5e0cfb6640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5e0810c170 0x7f5e08101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:55.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.287+0000 7f5e0cfb6640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f5e08111f70 con 0x7f5e08102d40 2024-09-11T23:38:55.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.287+0000 7f5e0cfb6640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f5e08111df0 con 0x7f5e08103660 2024-09-11T23:38:55.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.287+0000 7f5e0cfb6640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f5e081120f0 con 0x7f5e0810c170 2024-09-11T23:38:55.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.287+0000 7f5e0659c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5e08102d40 0x7f5e081009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:55.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.287+0000 7f5e0659c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5e08102d40 0x7f5e081009a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51012/0 (socket says 172.21.15.26:51012) 2024-09-11T23:38:55.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.287+0000 7f5e0659c640 1 -- 172.21.15.26:0/1594629463 learned_addr learned my addr 172.21.15.26:0/1594629463 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:38:55.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.287+0000 7f5dfffff640 1 --2- 172.21.15.26:0/1594629463 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5e08103660 0x7f5e08100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:55.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.287+0000 7f5e06d9d640 1 --2- 172.21.15.26:0/1594629463 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5e0810c170 0x7f5e08101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:55.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.287+0000 7f5e0659c640 1 -- 172.21.15.26:0/1594629463 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5e0810c170 msgr2=0x7f5e08101420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:55.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.287+0000 7f5e0659c640 1 --2- 172.21.15.26:0/1594629463 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5e0810c170 0x7f5e08101420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:55.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.287+0000 7f5e0659c640 1 -- 172.21.15.26:0/1594629463 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5e08103660 msgr2=0x7f5e08100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:55.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.287+0000 7f5e0659c640 1 --2- 172.21.15.26:0/1594629463 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5e08103660 0x7f5e08100ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:55.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.287+0000 7f5e0659c640 1 -- 172.21.15.26:0/1594629463 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e081a68b0 con 0x7f5e08102d40 2024-09-11T23:38:55.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.287+0000 7f5e0659c640 1 --2- 172.21.15.26:0/1594629463 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5e08102d40 0x7f5e081009a0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f5df800c890 tx=0x7f5df800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:38:55.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.288+0000 7f5dff7fe640 1 -- 172.21.15.26:0/1594629463 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5df8004220 con 0x7f5e08102d40 2024-09-11T23:38:55.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.288+0000 7f5e0cfb6640 1 -- 172.21.15.26:0/1594629463 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e081a6ba0 con 0x7f5e08102d40 2024-09-11T23:38:55.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.288+0000 7f5dff7fe640 1 -- 172.21.15.26:0/1594629463 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5df80043c0 con 0x7f5e08102d40 2024-09-11T23:38:55.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.288+0000 7f5dff7fe640 1 -- 172.21.15.26:0/1594629463 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5df8004a00 con 0x7f5e08102d40 2024-09-11T23:38:55.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.288+0000 7f5e0cfb6640 1 -- 172.21.15.26:0/1594629463 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e081a70e0 con 0x7f5e08102d40 2024-09-11T23:38:55.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.290+0000 7f5dff7fe640 1 -- 172.21.15.26:0/1594629463 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f5df80079b0 con 0x7f5e08102d40 2024-09-11T23:38:55.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.290+0000 7f5e0f38f740 1 Processor -- start 2024-09-11T23:38:55.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.291+0000 7f5e0f38f740 1 -- start start 2024-09-11T23:38:55.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.291+0000 7f5e0f38f740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559d0302c060 0x559d0302c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:55.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.291+0000 7f5e0f38f740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559d0302c980 0x559d03036d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:55.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.291+0000 7f5e0f38f740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559d03037240 0x559d03039630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:55.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.291+0000 7f5e0f38f740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x559d02f91620 con 0x559d0302c060 2024-09-11T23:38:55.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.291+0000 7f5e0f38f740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x559d02f38e50 con 0x559d03037240 2024-09-11T23:38:55.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.291+0000 7f5e0f38f740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x559d02f8d5e0 con 0x559d0302c980 2024-09-11T23:38:55.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.291+0000 7f5e06d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559d0302c060 0x559d0302c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:55.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.291+0000 7f5e06d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559d0302c060 0x559d0302c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51028/0 (socket says 172.21.15.26:51028) 2024-09-11T23:38:55.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.291+0000 7f5e06d9d640 1 -- 172.21.15.26:0/3771040758 learned_addr learned my addr 172.21.15.26:0/3771040758 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:38:55.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.291+0000 7f5dfffff640 1 --2- 172.21.15.26:0/3771040758 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559d03037240 0x559d03039630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:55.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.291+0000 7f5dff7fe640 1 --2- 172.21.15.26:0/1594629463 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f5df40a2cf0 0x7f5df40a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:55.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.291+0000 7f5e0659c640 1 --2- 172.21.15.26:0/3771040758 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559d0302c980 0x559d03036d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:55.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.291+0000 7f5e06d9d640 1 -- 172.21.15.26:0/3771040758 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559d0302c980 msgr2=0x559d03036d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:38:55.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.291+0000 7f5e06d9d640 1 --2- 172.21.15.26:0/3771040758 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559d0302c980 0x559d03036d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:55.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.291+0000 7f5e06d9d640 1 -- 172.21.15.26:0/3771040758 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559d03037240 msgr2=0x559d03039630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:38:55.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.291+0000 7f5e06d9d640 1 --2- 172.21.15.26:0/3771040758 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559d03037240 0x559d03039630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:55.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.291+0000 7f5e06d9d640 1 -- 172.21.15.26:0/3771040758 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d02f29d40 con 0x559d0302c060 2024-09-11T23:38:55.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.291+0000 7f5dff7fe640 1 -- 172.21.15.26:0/1594629463 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 8586+0+0 (secure 0 0 0) 0x7f5df80cf0e0 con 0x7f5e08102d40 2024-09-11T23:38:55.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.291+0000 7f5e06d9d640 1 --2- 172.21.15.26:0/1594629463 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f5df40a2cf0 0x7f5df40a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:55.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.292+0000 7f5e06d9d640 1 --2- 172.21.15.26:0/3771040758 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559d0302c060 0x559d0302c440 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f5e00035000 tx=0x7f5e000098f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f068e15729ac640 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:38:55.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.292+0000 7f5dfcff9640 1 -- 172.21.15.26:0/3771040758 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e00034040 con 0x559d0302c060 2024-09-11T23:38:55.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.292+0000 7f5dfcff9640 1 -- 172.21.15.26:0/3771040758 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e00003c30 con 0x559d0302c060 2024-09-11T23:38:55.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.292+0000 7f5dfcff9640 1 -- 172.21.15.26:0/3771040758 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e000467e0 con 0x559d0302c060 2024-09-11T23:38:55.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.292+0000 7f5e06d9d640 1 --2- 172.21.15.26:0/1594629463 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f5df40a2cf0 0x7f5df40a51b0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f5e0004d000 tx=0x7f5e0004a000 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:38:55.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.293+0000 7f5e0f38f740 1 -- 172.21.15.26:0/3771040758 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559d0302c060 msgr2=0x559d0302c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:55.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.293+0000 7f5e0f38f740 1 --2- 172.21.15.26:0/3771040758 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559d0302c060 0x559d0302c440 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f5e00035000 tx=0x7f5e000098f0 comp rx=0 tx=0).stop 2024-09-11T23:38:55.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.293+0000 7f5e0f38f740 1 -- 172.21.15.26:0/3771040758 shutdown_connections 2024-09-11T23:38:55.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.293+0000 7f5e0f38f740 1 --2- 172.21.15.26:0/3771040758 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559d03037240 0x559d03039630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:55.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.293+0000 7f5e0f38f740 1 --2- 172.21.15.26:0/3771040758 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559d0302c980 0x559d03036d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:55.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.293+0000 7f5e0f38f740 1 --2- 172.21.15.26:0/3771040758 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559d0302c060 0x559d0302c440 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:55.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.293+0000 7f5e0f38f740 1 -- 172.21.15.26:0/3771040758 >> 172.21.15.26:0/3771040758 conn(0x559d030285f0 msgr2=0x559d030289d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:38:55.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.293+0000 7f5e0f38f740 1 -- 172.21.15.26:0/3771040758 shutdown_connections 2024-09-11T23:38:55.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.293+0000 7f5e0f38f740 1 -- 172.21.15.26:0/3771040758 wait complete. 2024-09-11T23:38:55.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.293+0000 7f5e0f38f740 1 -- start start 2024-09-11T23:38:55.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.293+0000 7f5e0f38f740 1 Processor -- start 2024-09-11T23:38:55.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.293+0000 7f5e0f38f740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559d0302c060 0x559d03034170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:55.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.293+0000 7f5e0f38f740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559d0302c980 0x559d030346b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:55.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.293+0000 7f5e0f38f740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559d03037240 0x559d03034bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:55.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.293+0000 7f5e0f38f740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x559d03039e10 con 0x559d0302c060 2024-09-11T23:38:55.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.293+0000 7f5e0f38f740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x559d02f19f10 con 0x559d0302c980 2024-09-11T23:38:55.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.293+0000 7f5e0f38f740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x559d03039f90 con 0x559d03037240 2024-09-11T23:38:55.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.293+0000 7f5e0f38f740 10 client.0 _ll_register_callbacks cb 0x559d02f3ddb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:38:55.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.293+0000 7f5e0659c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559d0302c060 0x559d03034170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:55.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.294+0000 7f5e0659c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559d0302c060 0x559d03034170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51040/0 (socket says 172.21.15.26:51040) 2024-09-11T23:38:55.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.294+0000 7f5e0659c640 1 -- 172.21.15.26:0/3586201214 learned_addr learned my addr 172.21.15.26:0/3586201214 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:38:55.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.294+0000 7f5e06d9d640 1 --2- 172.21.15.26:0/3586201214 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559d03037240 0x559d03034bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:55.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.294+0000 7f5dfffff640 1 --2- 172.21.15.26:0/3586201214 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559d0302c980 0x559d030346b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:55.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.294+0000 7f5e0659c640 1 -- 172.21.15.26:0/3586201214 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559d03037240 msgr2=0x559d03034bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:38:55.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.294+0000 7f5e0659c640 1 --2- 172.21.15.26:0/3586201214 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559d03037240 0x559d03034bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:55.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.294+0000 7f5e0659c640 1 -- 172.21.15.26:0/3586201214 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559d0302c980 msgr2=0x559d030346b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:38:55.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.294+0000 7f5e0659c640 1 --2- 172.21.15.26:0/3586201214 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559d0302c980 0x559d030346b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:55.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.294+0000 7f5e0659c640 1 -- 172.21.15.26:0/3586201214 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d02ca0580 con 0x559d0302c060 2024-09-11T23:38:55.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.294+0000 7f5e06d9d640 1 --2- 172.21.15.26:0/3586201214 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559d03037240 0x559d03034bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:38:55.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.294+0000 7f5dfffff640 1 --2- 172.21.15.26:0/3586201214 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559d0302c980 0x559d030346b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:38:55.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.294+0000 7f5e0659c640 1 --2- 172.21.15.26:0/3586201214 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559d0302c060 0x559d03034170 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f5df8000c00 tx=0x7f5df8097310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=575cffbbd7c0808e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:38:55.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.294+0000 7f5dedffb640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:38:55.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.294+0000 7f5dedffb640 1 -- 172.21.15.26:0/3586201214 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5df801d070 con 0x559d0302c060 2024-09-11T23:38:55.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.294+0000 7f5dedffb640 1 -- 172.21.15.26:0/3586201214 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 8586+0+0 (secure 0 0 0) 0x7f5df8084bb0 con 0x559d0302c060 2024-09-11T23:38:55.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.294+0000 7f5e0f38f740 1 -- 172.21.15.26:0/3586201214 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5e00003c30 con 0x559d0302c060 2024-09-11T23:38:55.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.294+0000 7f5dedffb640 1 -- 172.21.15.26:0/3586201214 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f5df809b1b0 con 0x559d0302c060 2024-09-11T23:38:55.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.294+0000 7f5e0f38f740 10 client.18060 fetch_fsmap learned FSMap version 90 2024-09-11T23:38:55.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.294+0000 7f5e0f38f740 1 -- 172.21.15.26:0/3586201214 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=90,monmap=4+}) -- 0x559d02d6a960 con 0x559d0302c060 2024-09-11T23:38:55.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.294+0000 7f5dedffb640 1 -- 172.21.15.26:0/3586201214 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e00003c30 con 0x559d0302c060 2024-09-11T23:38:55.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.295+0000 7f5dedffb640 1 -- 172.21.15.26:0/3586201214 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 90) ==== 8418+0+0 (secure 0 0 0) 0x7f5df80b4420 con 0x559d0302c060 2024-09-11T23:38:55.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.295+0000 7f5e0f38f740 10 client.18060 fetch_fsmap finished waiting for FSMap version 90 2024-09-11T23:38:55.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.295+0000 7f5e0f38f740 10 client.18060 resolve_mds: validated gid 24485 aka daemon mds.b 2024-09-11T23:38:55.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.295+0000 7f5e0f38f740 20 client.18060 populate_metadata read hostname 'smithi026' 2024-09-11T23:38:55.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.295+0000 7f5e0f38f740 1 --2- 172.21.15.26:0/3586201214 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x559d03046f00 0x559d03049370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:55.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.295+0000 7f5e0f38f740 4 client.18060 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:38:55.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.295+0000 7f5e0f38f740 1 -- 172.21.15.26:0/3586201214 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559d02f9eb90 con 0x559d03046f00 2024-09-11T23:38:55.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.295+0000 7f5dfffff640 1 --2- 172.21.15.26:0/3586201214 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x559d03046f00 0x559d03049370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:55.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.295+0000 7f5dfffff640 1 --2- 172.21.15.26:0/3586201214 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x559d03046f00 0x559d03049370 crc :-1 s=READY pgs=308 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=99a2d4a7bbe5d9b3 server_cookie=a3e68165d4db0d26 in_seq=0 out_seq=0 2024-09-11T23:38:55.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.295+0000 7f5dedffb640 10 client.18060 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:38:55.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.296+0000 7f5dedffb640 1 -- 172.21.15.26:0/3586201214 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559d02f9eb90 con 0x559d03046f00 2024-09-11T23:38:55.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.296+0000 7f5dedffb640 10 client.18060 handle_command_reply: tid=0 2024-09-11T23:38:55.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.296+0000 7f5e0f38f740 1 client.18060 shutdown 2024-09-11T23:38:55.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.296+0000 7f5e0f38f740 1 -- 172.21.15.26:0/3586201214 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559d0302c060 msgr2=0x559d03034170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:55.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.296+0000 7f5e0f38f740 1 --2- 172.21.15.26:0/3586201214 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559d0302c060 0x559d03034170 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f5df8000c00 tx=0x7f5df8097310 comp rx=0 tx=0).stop 2024-09-11T23:38:55.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.296+0000 7f5e0f38f740 1 -- 172.21.15.26:0/3586201214 shutdown_connections 2024-09-11T23:38:55.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.296+0000 7f5e0f38f740 1 --2- 172.21.15.26:0/3586201214 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559d0302c980 0x559d030346b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:55.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.296+0000 7f5e0f38f740 1 --2- 172.21.15.26:0/3586201214 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x559d03046f00 0x559d03049370 crc :-1 s=READY pgs=308 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:55.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.296+0000 7f5dedffb640 0 client.18060 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:38:55.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.296+0000 7f5e0f38f740 1 --2- 172.21.15.26:0/3586201214 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559d03037240 0x559d03034bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:55.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.296+0000 7f5e0f38f740 1 --2- 172.21.15.26:0/3586201214 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559d0302c060 0x559d03034170 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:55.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.296+0000 7f5e0f38f740 1 -- 172.21.15.26:0/3586201214 >> 172.21.15.26:0/3586201214 conn(0x559d030285f0 msgr2=0x559d0302e2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:38:55.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.296+0000 7f5e0f38f740 1 -- 172.21.15.26:0/3586201214 shutdown_connections 2024-09-11T23:38:55.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.296+0000 7f5e0f38f740 1 -- 172.21.15.26:0/3586201214 wait complete. 2024-09-11T23:38:55.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.297+0000 7f5e0f38f740 20 client.18060 trim_cache size 0 max 16384 2024-09-11T23:38:55.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.313+0000 7f5e0f38f740 1 Processor -- start 2024-09-11T23:38:55.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.313+0000 7f5e0f38f740 1 -- start start 2024-09-11T23:38:55.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.314+0000 7f5e0f38f740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559d0302c060 0x559d03027640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:55.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.314+0000 7f5e0f38f740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559d0302c980 0x559d0304ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:55.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.314+0000 7f5e0f38f740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559d03037240 0x559d0304f180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:55.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.314+0000 7f5e0f38f740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f5e081afc20 con 0x559d03037240 2024-09-11T23:38:55.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.314+0000 7f5e0f38f740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f5e081b19d0 con 0x559d0302c060 2024-09-11T23:38:55.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.314+0000 7f5e0f38f740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f5e081b0320 con 0x559d0302c980 2024-09-11T23:38:55.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.314+0000 7f5e06d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559d03037240 0x559d0304f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:55.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.314+0000 7f5e06d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559d03037240 0x559d0304f180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51042/0 (socket says 172.21.15.26:51042) 2024-09-11T23:38:55.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.314+0000 7f5e06d9d640 1 -- 172.21.15.26:0/964286228 learned_addr learned my addr 172.21.15.26:0/964286228 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:38:55.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.314+0000 7f5e0659c640 1 --2- 172.21.15.26:0/964286228 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559d0302c060 0x559d03027640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:55.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.314+0000 7f5dfffff640 1 --2- 172.21.15.26:0/964286228 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559d0302c980 0x559d0304ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:55.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.314+0000 7f5e06d9d640 1 -- 172.21.15.26:0/964286228 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559d0302c980 msgr2=0x559d0304ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:38:55.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.314+0000 7f5e06d9d640 1 --2- 172.21.15.26:0/964286228 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559d0302c980 0x559d0304ec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:55.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.314+0000 7f5e06d9d640 1 -- 172.21.15.26:0/964286228 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559d0302c060 msgr2=0x559d03027640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:38:55.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.314+0000 7f5e06d9d640 1 --2- 172.21.15.26:0/964286228 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559d0302c060 0x559d03027640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:55.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.314+0000 7f5e06d9d640 1 -- 172.21.15.26:0/964286228 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d0302c7b0 con 0x559d03037240 2024-09-11T23:38:55.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.314+0000 7f5e06d9d640 1 --2- 172.21.15.26:0/964286228 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559d03037240 0x559d0304f180 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f5e00004a40 tx=0x7f5e00005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac8cd1aaf001da9d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:38:55.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.314+0000 7f5deffff640 1 -- 172.21.15.26:0/964286228 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e0002fda0 con 0x559d03037240 2024-09-11T23:38:55.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.314+0000 7f5deffff640 1 -- 172.21.15.26:0/964286228 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e00046b30 con 0x559d03037240 2024-09-11T23:38:55.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.314+0000 7f5deffff640 1 -- 172.21.15.26:0/964286228 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e000042a0 con 0x559d03037240 2024-09-11T23:38:55.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.315+0000 7f5e0f38f740 1 -- 172.21.15.26:0/964286228 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559d03037240 msgr2=0x559d0304f180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:55.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.315+0000 7f5e0f38f740 1 --2- 172.21.15.26:0/964286228 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559d03037240 0x559d0304f180 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f5e00004a40 tx=0x7f5e00005c50 comp rx=0 tx=0).stop 2024-09-11T23:38:55.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.315+0000 7f5e0f38f740 1 -- 172.21.15.26:0/964286228 shutdown_connections 2024-09-11T23:38:55.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.315+0000 7f5e0f38f740 1 --2- 172.21.15.26:0/964286228 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559d03037240 0x559d0304f180 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:55.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.315+0000 7f5e0f38f740 1 --2- 172.21.15.26:0/964286228 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559d0302c980 0x559d0304ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:55.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.315+0000 7f5e0f38f740 1 --2- 172.21.15.26:0/964286228 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559d0302c060 0x559d03027640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:55.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.315+0000 7f5e0f38f740 1 -- 172.21.15.26:0/964286228 >> 172.21.15.26:0/964286228 conn(0x559d030285f0 msgr2=0x559d03026f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:38:55.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.315+0000 7f5e0f38f740 1 -- 172.21.15.26:0/964286228 shutdown_connections 2024-09-11T23:38:55.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.315+0000 7f5e0f38f740 1 -- 172.21.15.26:0/964286228 wait complete. 2024-09-11T23:38:55.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.315+0000 7f5e0f38f740 1 -- start start 2024-09-11T23:38:55.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.316+0000 7f5e0f38f740 1 Processor -- start 2024-09-11T23:38:55.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.316+0000 7f5e0f38f740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559d0302c060 0x559d0303f590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:55.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.316+0000 7f5e0f38f740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559d0302c980 0x559d0303fad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:55.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.316+0000 7f5e0f38f740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559d03037240 0x559d0303bb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:55.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.316+0000 7f5e0f38f740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f5e081b1240 con 0x559d0302c060 2024-09-11T23:38:55.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.316+0000 7f5e0f38f740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f5e081b0ab0 con 0x559d03037240 2024-09-11T23:38:55.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.316+0000 7f5e0f38f740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f5e081adc20 con 0x559d0302c980 2024-09-11T23:38:55.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.316+0000 7f5e0f38f740 10 client.0 _ll_register_callbacks cb 0x7f5df80053a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:38:55.320 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:38:55 smithi026 ceph-mon[31172]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:38:55.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.316+0000 7f5e0659c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559d0302c060 0x559d0303f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:55.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.316+0000 7f5e0659c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559d0302c060 0x559d0303f590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51058/0 (socket says 172.21.15.26:51058) 2024-09-11T23:38:55.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.316+0000 7f5e0659c640 1 -- 172.21.15.26:0/3044484906 learned_addr learned my addr 172.21.15.26:0/3044484906 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:38:55.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.316+0000 7f5dfffff640 1 --2- 172.21.15.26:0/3044484906 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559d0302c980 0x559d0303fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:55.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.316+0000 7f5e06d9d640 1 --2- 172.21.15.26:0/3044484906 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559d03037240 0x559d0303bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:55.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.316+0000 7f5e0659c640 1 -- 172.21.15.26:0/3044484906 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559d0302c980 msgr2=0x559d0303fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:38:55.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.316+0000 7f5e0659c640 1 --2- 172.21.15.26:0/3044484906 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559d0302c980 0x559d0303fad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:55.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.316+0000 7f5e0659c640 1 -- 172.21.15.26:0/3044484906 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559d03037240 msgr2=0x559d0303bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:38:55.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.316+0000 7f5e0659c640 1 --2- 172.21.15.26:0/3044484906 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559d03037240 0x559d0303bb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:55.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.316+0000 7f5e0659c640 1 -- 172.21.15.26:0/3044484906 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d0303c350 con 0x559d0302c060 2024-09-11T23:38:55.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.317+0000 7f5e0659c640 1 --2- 172.21.15.26:0/3044484906 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559d0302c060 0x559d0303f590 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f5df809b600 tx=0x7f5df8084de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd475d64bb7a8c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:38:55.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.317+0000 7f5dedffb640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:38:55.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.317+0000 7f5dedffb640 1 -- 172.21.15.26:0/3044484906 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5df801d040 con 0x559d0302c060 2024-09-11T23:38:55.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.317+0000 7f5dedffb640 1 -- 172.21.15.26:0/3044484906 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 8586+0+0 (secure 0 0 0) 0x7f5df80973c0 con 0x559d0302c060 2024-09-11T23:38:55.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.317+0000 7f5e0f38f740 1 -- 172.21.15.26:0/3044484906 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5e00046b30 con 0x559d0302c060 2024-09-11T23:38:55.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.317+0000 7f5dedffb640 1 -- 172.21.15.26:0/3044484906 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f5df8086070 con 0x559d0302c060 2024-09-11T23:38:55.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.317+0000 7f5e0f38f740 10 client.18066 fetch_fsmap learned FSMap version 90 2024-09-11T23:38:55.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.317+0000 7f5e0f38f740 1 -- 172.21.15.26:0/3044484906 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=90,monmap=4+}) -- 0x559d030b28d0 con 0x559d0302c060 2024-09-11T23:38:55.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.317+0000 7f5dedffb640 1 -- 172.21.15.26:0/3044484906 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e00046b30 con 0x559d0302c060 2024-09-11T23:38:55.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.317+0000 7f5dedffb640 1 -- 172.21.15.26:0/3044484906 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 90) ==== 8418+0+0 (secure 0 0 0) 0x7f5df809a530 con 0x559d0302c060 2024-09-11T23:38:55.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.318+0000 7f5e0f38f740 10 client.18066 fetch_fsmap finished waiting for FSMap version 90 2024-09-11T23:38:55.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.318+0000 7f5e0f38f740 10 client.18066 resolve_mds: validated gid 24485 aka daemon mds.b 2024-09-11T23:38:55.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.318+0000 7f5e0f38f740 20 client.18066 populate_metadata read hostname 'smithi026' 2024-09-11T23:38:55.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.318+0000 7f5e0f38f740 1 --2- 172.21.15.26:0/3044484906 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x559d03046f00 0x559d030b50c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:55.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.318+0000 7f5e0f38f740 4 client.18066 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "ad57dcac", "release": true, "await": true}] 2024-09-11T23:38:55.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.318+0000 7f5e0f38f740 1 -- 172.21.15.26:0/3044484906 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "quiesce db", "set_id": "ad57dcac", "release": true, "await": true}) -- 0x559d02f990a0 con 0x559d03046f00 2024-09-11T23:38:55.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.318+0000 7f5dfffff640 1 --2- 172.21.15.26:0/3044484906 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x559d03046f00 0x559d030b50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:55.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.318+0000 7f5dedffb640 10 client.18066 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:38:55.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.318+0000 7f5dfffff640 1 --2- 172.21.15.26:0/3044484906 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x559d03046f00 0x559d030b50c0 crc :-1 s=READY pgs=309 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ebbed9d4cea678e0 server_cookie=9b99265958e74d85 in_seq=0 out_seq=0 2024-09-11T23:38:55.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:38:55 smithi136 ceph-mon[35121]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:38:55.552 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:38:55 smithi076 ceph-mon[35296]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:38:55.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.623+0000 7f5dedffb640 1 -- 172.21.15.26:0/3044484906 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x559d02f990a0 con 0x559d03046f00 2024-09-11T23:38:55.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.623+0000 7f5dedffb640 10 client.18066 handle_command_reply: tid=0 2024-09-11T23:38:55.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.623+0000 7f5e0f38f740 1 client.18066 shutdown 2024-09-11T23:38:55.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.623+0000 7f5e0f38f740 1 -- 172.21.15.26:0/3044484906 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559d0302c060 msgr2=0x559d0303f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:55.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.623+0000 7f5e0f38f740 1 --2- 172.21.15.26:0/3044484906 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559d0302c060 0x559d0303f590 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f5df809b600 tx=0x7f5df8084de0 comp rx=0 tx=0).stop 2024-09-11T23:38:55.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.623+0000 7f5e0f38f740 1 -- 172.21.15.26:0/3044484906 shutdown_connections 2024-09-11T23:38:55.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.623+0000 7f5e0f38f740 1 --2- 172.21.15.26:0/3044484906 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559d03037240 0x559d0303bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:55.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.623+0000 7f5e0f38f740 1 --2- 172.21.15.26:0/3044484906 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x559d03046f00 0x559d030b50c0 crc :-1 s=READY pgs=309 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:55.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.623+0000 7f5e0f38f740 1 --2- 172.21.15.26:0/3044484906 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559d0302c980 0x559d0303fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:55.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.623+0000 7f5e0f38f740 1 --2- 172.21.15.26:0/3044484906 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559d0302c060 0x559d0303f590 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:55.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.623+0000 7f5e0f38f740 1 -- 172.21.15.26:0/3044484906 >> 172.21.15.26:0/3044484906 conn(0x559d030285f0 msgr2=0x559d03027750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:38:55.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.623+0000 7f5dedffb640 0 client.18066 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:38:55.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.623+0000 7f5e0f38f740 1 -- 172.21.15.26:0/3044484906 shutdown_connections 2024-09-11T23:38:55.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.623+0000 7f5e0f38f740 1 -- 172.21.15.26:0/3044484906 wait complete. 2024-09-11T23:38:55.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.623+0000 7f5e0f38f740 20 client.18066 trim_cache size 0 max 16384 2024-09-11T23:38:55.625 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-11T23:38:55.625 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 90, 2024-09-11T23:38:55.625 INFO:teuthology.orchestra.run.smithi026.stdout: "leader": 24485, 2024-09-11T23:38:55.625 INFO:teuthology.orchestra.run.smithi026.stdout: "set_version": 72, 2024-09-11T23:38:55.625 INFO:teuthology.orchestra.run.smithi026.stdout: "sets": { 2024-09-11T23:38:55.625 INFO:teuthology.orchestra.run.smithi026.stdout: "ad57dcac": { 2024-09-11T23:38:55.625 INFO:teuthology.orchestra.run.smithi026.stdout: "version": 72, 2024-09-11T23:38:55.625 INFO:teuthology.orchestra.run.smithi026.stdout: "age_ref": 0.0, 2024-09-11T23:38:55.625 INFO:teuthology.orchestra.run.smithi026.stdout: "state": { 2024-09-11T23:38:55.625 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "RELEASED", 2024-09-11T23:38:55.625 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.0 2024-09-11T23:38:55.626 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:38:55.626 INFO:teuthology.orchestra.run.smithi026.stdout: "timeout": 90.0, 2024-09-11T23:38:55.626 INFO:teuthology.orchestra.run.smithi026.stdout: "expiration": 141.8, 2024-09-11T23:38:55.626 INFO:teuthology.orchestra.run.smithi026.stdout: "members": { 2024-09-11T23:38:55.626 INFO:teuthology.orchestra.run.smithi026.stdout: "file:/": { 2024-09-11T23:38:55.626 INFO:teuthology.orchestra.run.smithi026.stdout: "excluded": false, 2024-09-11T23:38:55.626 INFO:teuthology.orchestra.run.smithi026.stdout: "state": { 2024-09-11T23:38:55.626 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "RELEASED", 2024-09-11T23:38:55.626 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.0 2024-09-11T23:38:55.626 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:38:55.626 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:38:55.626 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:38:55.626 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:38:55.626 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:38:55.627 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-11T23:38:55.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.623+0000 7f5e0cfb6640 1 -- 172.21.15.26:0/1594629463 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f5df40a2cf0 msgr2=0x7f5df40a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:55.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.623+0000 7f5e0cfb6640 1 --2- 172.21.15.26:0/1594629463 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f5df40a2cf0 0x7f5df40a51b0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f5e0004d000 tx=0x7f5e0004a000 comp rx=0 tx=0).stop 2024-09-11T23:38:55.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.623+0000 7f5e0cfb6640 1 -- 172.21.15.26:0/1594629463 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5e08102d40 msgr2=0x7f5e081009a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:55.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.623+0000 7f5e0cfb6640 1 --2- 172.21.15.26:0/1594629463 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5e08102d40 0x7f5e081009a0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f5df800c890 tx=0x7f5df800cd50 comp rx=0 tx=0).stop 2024-09-11T23:38:55.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.623+0000 7f5e0cfb6640 1 -- 172.21.15.26:0/1594629463 shutdown_connections 2024-09-11T23:38:55.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.623+0000 7f5e0cfb6640 1 --2- 172.21.15.26:0/1594629463 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f5df40a2cf0 0x7f5df40a51b0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:55.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.623+0000 7f5e0cfb6640 1 --2- 172.21.15.26:0/1594629463 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5e08103660 0x7f5e08100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:55.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.623+0000 7f5e0cfb6640 1 --2- 172.21.15.26:0/1594629463 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5e0810c170 0x7f5e08101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:55.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.623+0000 7f5e0cfb6640 1 --2- 172.21.15.26:0/1594629463 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5e08102d40 0x7f5e081009a0 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:55.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.623+0000 7f5e0cfb6640 1 -- 172.21.15.26:0/1594629463 >> 172.21.15.26:0/1594629463 conn(0x7f5e080fc7a0 msgr2=0x7f5e080fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:38:55.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.623+0000 7f5e0cfb6640 1 -- 172.21.15.26:0/1594629463 shutdown_connections 2024-09-11T23:38:55.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:38:55.623+0000 7f5e0cfb6640 1 -- 172.21.15.26:0/1594629463 wait complete. 2024-09-11T23:38:56.389 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:38:56 smithi136 ceph-mon[35121]: pgmap v646: 97 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 89 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.6 KiB/s wr, 10 op/s 2024-09-11T23:38:56.461 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:38:56 smithi026 ceph-mon[31172]: pgmap v646: 97 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 89 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.6 KiB/s wr, 10 op/s 2024-09-11T23:38:56.487 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'ad57dcac', total seconds elapsed: 26.0 2024-09-11T23:38:56.552 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:38:56 smithi076 ceph-mon[35296]: pgmap v646: 97 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 89 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.6 KiB/s wr, 10 op/s 2024-09-11T23:38:58.354 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:38:58 smithi076 ceph-mon[35296]: pgmap v647: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 3.0 KiB/s wr, 15 op/s 2024-09-11T23:38:58.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:38:58 smithi136 ceph-mon[35121]: pgmap v647: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 3.0 KiB/s wr, 15 op/s 2024-09-11T23:38:58.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:38:58 smithi026 ceph-mon[31172]: pgmap v647: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 3.0 KiB/s wr, 15 op/s 2024-09-11T23:39:00.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:39:00 smithi136 ceph-mon[35121]: pgmap v648: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 2.3 KiB/s wr, 12 op/s 2024-09-11T23:39:00.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:39:00 smithi026 ceph-mon[31172]: pgmap v648: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 2.3 KiB/s wr, 12 op/s 2024-09-11T23:39:00.552 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:39:00 smithi076 ceph-mon[35296]: pgmap v648: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 2.3 KiB/s wr, 12 op/s 2024-09-11T23:39:02.238 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:39:01 smithi136 ceph-mon[35121]: pgmap v649: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 330 KiB/s rd, 668 KiB/s wr, 53 op/s 2024-09-11T23:39:02.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:39:01 smithi136 ceph-mon[35121]: osdmap e114: 12 total, 12 up, 12 in 2024-09-11T23:39:02.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:39:01 smithi026 ceph-mon[31172]: pgmap v649: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 330 KiB/s rd, 668 KiB/s wr, 53 op/s 2024-09-11T23:39:02.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:39:01 smithi026 ceph-mon[31172]: osdmap e114: 12 total, 12 up, 12 in 2024-09-11T23:39:02.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:39:01 smithi076 ceph-mon[35296]: pgmap v649: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 330 KiB/s rd, 668 KiB/s wr, 53 op/s 2024-09-11T23:39:02.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:39:01 smithi076 ceph-mon[35296]: osdmap e114: 12 total, 12 up, 12 in 2024-09-11T23:39:03.238 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:39:02 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:39:03.254 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:39:02 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:39:03.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:39:02 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:39:04.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:39:03 smithi136 ceph-mon[35121]: pgmap v651: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 141 op/s 2024-09-11T23:39:04.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:39:03 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:39:04.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:39:03 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:39:04.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:39:03 smithi026 ceph-mon[31172]: pgmap v651: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 141 op/s 2024-09-11T23:39:04.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:39:03 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:39:04.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:39:03 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:39:04.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:39:03 smithi076 ceph-mon[35296]: pgmap v651: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 141 op/s 2024-09-11T23:39:04.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:39:03 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:39:04.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:39:03 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:39:05.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:39:04 smithi136 ceph-mon[35121]: osdmap e115: 12 total, 12 up, 12 in 2024-09-11T23:39:05.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:39:04 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:39:05.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:39:04 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:39:05.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:39:04 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:39:05.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:39:04 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:39:05.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:39:04 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:39:05.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:39:04 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:39:05.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:39:04 smithi136 ceph-mon[35121]: Adjusting osd_memory_target on smithi026 to 177.0M 2024-09-11T23:39:05.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:39:04 smithi136 ceph-mon[35121]: Unable to set osd_memory_target on smithi026 to 185689190: error parsing value: Value '185689190' is below minimum 939524096 2024-09-11T23:39:05.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:39:04 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:39:05.240 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:39:04 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:39:05.240 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:39:04 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:39:05.240 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:39:04 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:39:05.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:39:04 smithi026 ceph-mon[31172]: osdmap e115: 12 total, 12 up, 12 in 2024-09-11T23:39:05.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:39:04 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:39:05.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:39:04 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:39:05.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:39:04 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:39:05.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:39:04 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:39:05.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:39:04 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:39:05.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:39:04 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:39:05.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:39:04 smithi026 ceph-mon[31172]: Adjusting osd_memory_target on smithi026 to 177.0M 2024-09-11T23:39:05.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:39:04 smithi026 ceph-mon[31172]: Unable to set osd_memory_target on smithi026 to 185689190: error parsing value: Value '185689190' is below minimum 939524096 2024-09-11T23:39:05.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:39:04 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:39:05.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:39:04 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:39:05.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:39:04 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:39:05.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:39:04 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:39:05.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:39:04 smithi076 ceph-mon[35296]: osdmap e115: 12 total, 12 up, 12 in 2024-09-11T23:39:05.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:39:04 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:39:05.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:39:04 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:39:05.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:39:04 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:39:05.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:39:04 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:39:05.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:39:04 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:39:05.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:39:04 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:39:05.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:39:04 smithi076 ceph-mon[35296]: Adjusting osd_memory_target on smithi026 to 177.0M 2024-09-11T23:39:05.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:39:04 smithi076 ceph-mon[35296]: Unable to set osd_memory_target on smithi026 to 185689190: error parsing value: Value '185689190' is below minimum 939524096 2024-09-11T23:39:05.304 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:39:04 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:39:05.304 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:39:04 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:39:05.304 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:39:04 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:39:05.304 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:39:04 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:39:05.345 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.345+0000 7f7155c4c640 1 -- 172.21.15.26:0/271154670 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f715011b140 msgr2=0x7f715011d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:05.345 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.345+0000 7f7155c4c640 1 --2- 172.21.15.26:0/271154670 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f715011b140 0x7f715011d530 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f714801ace0 tx=0x7f714803b990 comp rx=0 tx=0).stop 2024-09-11T23:39:05.352 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.353+0000 7f7155c4c640 1 -- 172.21.15.26:0/271154670 shutdown_connections 2024-09-11T23:39:05.352 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.353+0000 7f7155c4c640 1 --2- 172.21.15.26:0/271154670 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f715011b140 0x7f715011d530 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:05.352 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.353+0000 7f7155c4c640 1 --2- 172.21.15.26:0/271154670 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f71501095d0 0x7f7150109a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:05.352 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.353+0000 7f7155c4c640 1 --2- 172.21.15.26:0/271154670 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f71501091f0 0x7f7150110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:05.352 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.353+0000 7f7155c4c640 1 -- 172.21.15.26:0/271154670 >> 172.21.15.26:0/271154670 conn(0x7f715006ce00 msgr2=0x7f715006d210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:39:05.354 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.355+0000 7f7155c4c640 1 -- 172.21.15.26:0/271154670 shutdown_connections 2024-09-11T23:39:05.355 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.355+0000 7f7155c4c640 1 -- 172.21.15.26:0/271154670 wait complete. 2024-09-11T23:39:05.356 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.357+0000 7f7155c4c640 1 Processor -- start 2024-09-11T23:39:05.357 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.358+0000 7f7155c4c640 1 -- start start 2024-09-11T23:39:05.357 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.358+0000 7f7155c4c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f71501091f0 0x7f715011a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:05.358 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.358+0000 7f7155c4c640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f71501095d0 0x7f7150116840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:05.358 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.358+0000 7f7155c4c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f715011b140 0x7f7150116d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:05.358 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.358+0000 7f7155c4c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f7150113310 con 0x7f71501091f0 2024-09-11T23:39:05.358 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.358+0000 7f7155c4c640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f7150113190 con 0x7f715011b140 2024-09-11T23:39:05.358 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.358+0000 7f7155c4c640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f7150113490 con 0x7f71501095d0 2024-09-11T23:39:05.358 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.359+0000 7f714f7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f71501091f0 0x7f715011a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:05.358 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.359+0000 7f714f7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f71501091f0 0x7f715011a7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40292/0 (socket says 172.21.15.26:40292) 2024-09-11T23:39:05.359 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.359+0000 7f714f7fe640 1 -- 172.21.15.26:0/73943422 learned_addr learned my addr 172.21.15.26:0/73943422 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:39:05.359 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.359+0000 7f714ffff640 1 --2- 172.21.15.26:0/73943422 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f715011b140 0x7f7150116d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:05.359 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.359+0000 7f7147fff640 1 --2- 172.21.15.26:0/73943422 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f71501095d0 0x7f7150116840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:05.359 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.360+0000 7f714ffff640 1 -- 172.21.15.26:0/73943422 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f71501095d0 msgr2=0x7f7150116840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:05.359 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.360+0000 7f714ffff640 1 --2- 172.21.15.26:0/73943422 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f71501095d0 0x7f7150116840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:05.359 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.360+0000 7f714ffff640 1 -- 172.21.15.26:0/73943422 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f71501091f0 msgr2=0x7f715011a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:05.360 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.360+0000 7f714ffff640 1 --2- 172.21.15.26:0/73943422 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f71501091f0 0x7f715011a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:05.360 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.360+0000 7f714ffff640 1 -- 172.21.15.26:0/73943422 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7150117370 con 0x7f715011b140 2024-09-11T23:39:05.360 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.360+0000 7f7147fff640 1 --2- 172.21.15.26:0/73943422 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f71501095d0 0x7f7150116840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:39:05.360 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.360+0000 7f714ffff640 1 --2- 172.21.15.26:0/73943422 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f715011b140 0x7f7150116d80 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f714801ace0 tx=0x7f7148003ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:39:05.361 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.362+0000 7f714d7fa640 1 -- 172.21.15.26:0/73943422 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7148057070 con 0x7f715011b140 2024-09-11T23:39:05.365 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.362+0000 7f7155c4c640 1 -- 172.21.15.26:0/73943422 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7150117600 con 0x7f715011b140 2024-09-11T23:39:05.365 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.362+0000 7f7155c4c640 1 -- 172.21.15.26:0/73943422 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71501ab3e0 con 0x7f715011b140 2024-09-11T23:39:05.365 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.365+0000 7f71589e3740 1 Processor -- start 2024-09-11T23:39:05.365 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.366+0000 7f71589e3740 1 -- start start 2024-09-11T23:39:05.365 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.366+0000 7f71589e3740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558556dc0af0 0x558556dc0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:05.365 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.366+0000 7f71589e3740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558556dc1410 0x558556dcb770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:05.366 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.366+0000 7f71589e3740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558556dcbcb0 0x558556dce0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:05.366 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.366+0000 7f71589e3740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x558556d252f0 con 0x558556dc1410 2024-09-11T23:39:05.366 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.366+0000 7f71589e3740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x558556ccd270 con 0x558556dcbcb0 2024-09-11T23:39:05.366 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.366+0000 7f71589e3740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x558556d212b0 con 0x558556dc0af0 2024-09-11T23:39:05.366 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.367+0000 7f714d7fa640 1 -- 172.21.15.26:0/73943422 <== mon.2 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7148042e50 con 0x7f715011b140 2024-09-11T23:39:05.367 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.367+0000 7f714d7fa640 1 -- 172.21.15.26:0/73943422 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7148045040 con 0x7f715011b140 2024-09-11T23:39:05.372 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.371+0000 7f714f7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558556dc1410 0x558556dcb770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:05.372 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.371+0000 7f714f7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558556dc1410 0x558556dcb770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40304/0 (socket says 172.21.15.26:40304) 2024-09-11T23:39:05.372 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.371+0000 7f714f7fe640 1 -- 172.21.15.26:0/1632478897 learned_addr learned my addr 172.21.15.26:0/1632478897 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:39:05.372 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.372+0000 7f7147fff640 1 --2- 172.21.15.26:0/1632478897 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558556dc0af0 0x558556dc0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:05.376 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.374+0000 7f714f7fe640 1 -- 172.21.15.26:0/1632478897 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558556dc0af0 msgr2=0x558556dc0ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:39:05.376 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.374+0000 7f714f7fe640 1 --2- 172.21.15.26:0/1632478897 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558556dc0af0 0x558556dc0ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:05.376 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.374+0000 7f714f7fe640 1 -- 172.21.15.26:0/1632478897 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558556dcbcb0 msgr2=0x558556dce0a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T23:39:05.376 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.374+0000 7f714f7fe640 1 --2- 172.21.15.26:0/1632478897 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558556dcbcb0 0x558556dce0a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:05.376 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.374+0000 7f714f7fe640 1 -- 172.21.15.26:0/1632478897 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558556ccc340 con 0x558556dc1410 2024-09-11T23:39:05.376 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.375+0000 7f714f7fe640 1 --2- 172.21.15.26:0/1632478897 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558556dc1410 0x558556dcb770 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f7140009b00 tx=0x7f714000e1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=514bffbfe3d8877d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:39:05.377 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.378+0000 7f714d7fa640 1 -- 172.21.15.26:0/73943422 <== mon.2 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f7148048020 con 0x7f715011b140 2024-09-11T23:39:05.377 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.377+0000 7f7145ffb640 1 -- 172.21.15.26:0/1632478897 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7140017070 con 0x558556dc1410 2024-09-11T23:39:05.378 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.379+0000 7f7145ffb640 1 -- 172.21.15.26:0/1632478897 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f714000ec10 con 0x558556dc1410 2024-09-11T23:39:05.378 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.379+0000 7f7145ffb640 1 -- 172.21.15.26:0/1632478897 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71400126c0 con 0x558556dc1410 2024-09-11T23:39:05.379 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.379+0000 7f714d7fa640 1 --2- 172.21.15.26:0/73943422 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f713c0a2ef0 0x7f713c0a53b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:05.379 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.380+0000 7f714d7fa640 1 -- 172.21.15.26:0/73943422 <== mon.2 v2:172.21.15.76:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 8586+0+0 (secure 0 0 0) 0x7f7148100370 con 0x7f715011b140 2024-09-11T23:39:05.383 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.384+0000 7f71589e3740 1 -- 172.21.15.26:0/1632478897 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558556dc1410 msgr2=0x558556dcb770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:05.383 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.384+0000 7f71589e3740 1 --2- 172.21.15.26:0/1632478897 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558556dc1410 0x558556dcb770 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f7140009b00 tx=0x7f714000e1d0 comp rx=0 tx=0).stop 2024-09-11T23:39:05.391 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.386+0000 7f7147fff640 1 --2- 172.21.15.26:0/73943422 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f713c0a2ef0 0x7f713c0a53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:05.391 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.386+0000 7f71589e3740 1 -- 172.21.15.26:0/1632478897 shutdown_connections 2024-09-11T23:39:05.391 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.386+0000 7f71589e3740 1 --2- 172.21.15.26:0/1632478897 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558556dcbcb0 0x558556dce0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:05.391 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.386+0000 7f71589e3740 1 --2- 172.21.15.26:0/1632478897 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558556dc1410 0x558556dcb770 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:05.392 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.386+0000 7f71589e3740 1 --2- 172.21.15.26:0/1632478897 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558556dc0af0 0x558556dc0ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:05.392 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.386+0000 7f71589e3740 1 -- 172.21.15.26:0/1632478897 >> 172.21.15.26:0/1632478897 conn(0x558556dbd060 msgr2=0x558556dbd440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:39:05.392 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.386+0000 7f71589e3740 1 -- 172.21.15.26:0/1632478897 shutdown_connections 2024-09-11T23:39:05.392 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.386+0000 7f71589e3740 1 -- 172.21.15.26:0/1632478897 wait complete. 2024-09-11T23:39:05.392 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.387+0000 7f7147fff640 1 --2- 172.21.15.26:0/73943422 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f713c0a2ef0 0x7f713c0a53b0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x558556dbe3e0 tx=0x7f713800b3a0 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:39:05.392 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.387+0000 7f71589e3740 1 -- start start 2024-09-11T23:39:05.392 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.387+0000 7f71589e3740 1 Processor -- start 2024-09-11T23:39:05.392 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.387+0000 7f71589e3740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558556dc0af0 0x558556dc8b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:05.392 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.387+0000 7f71589e3740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558556dc1410 0x558556dc90d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:05.392 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.387+0000 7f71589e3740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558556dcbcb0 0x558556dc9610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:05.392 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.387+0000 7f71589e3740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x558556dce880 con 0x558556dc1410 2024-09-11T23:39:05.392 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.388+0000 7f71589e3740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x558556cae290 con 0x558556dcbcb0 2024-09-11T23:39:05.393 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.388+0000 7f71589e3740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x558556dcea00 con 0x558556dc0af0 2024-09-11T23:39:05.393 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.388+0000 7f71589e3740 10 client.0 _ll_register_callbacks cb 0x558556cdf140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:39:05.393 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.388+0000 7f7147fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558556dc1410 0x558556dc90d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:05.393 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.388+0000 7f7147fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558556dc1410 0x558556dc90d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40320/0 (socket says 172.21.15.26:40320) 2024-09-11T23:39:05.393 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.388+0000 7f7147fff640 1 -- 172.21.15.26:0/2933039517 learned_addr learned my addr 172.21.15.26:0/2933039517 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:39:05.393 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.388+0000 7f714ffff640 1 --2- 172.21.15.26:0/2933039517 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558556dcbcb0 0x558556dc9610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:05.393 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.388+0000 7f7147fff640 1 -- 172.21.15.26:0/2933039517 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558556dc0af0 msgr2=0x558556dc8b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:39:05.393 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.388+0000 7f7147fff640 1 --2- 172.21.15.26:0/2933039517 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558556dc0af0 0x558556dc8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:05.393 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.388+0000 7f7147fff640 1 -- 172.21.15.26:0/2933039517 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558556dcbcb0 msgr2=0x558556dc9610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:39:05.394 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.388+0000 7f7147fff640 1 --2- 172.21.15.26:0/2933039517 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558556dcbcb0 0x558556dc9610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:05.394 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.388+0000 7f7147fff640 1 -- 172.21.15.26:0/2933039517 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558556a38210 con 0x558556dc1410 2024-09-11T23:39:05.394 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.389+0000 7f714ffff640 1 --2- 172.21.15.26:0/2933039517 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558556dcbcb0 0x558556dc9610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:39:05.395 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.394+0000 7f7147fff640 1 --2- 172.21.15.26:0/2933039517 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558556dc1410 0x558556dc90d0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f71380126d0 tx=0x7f71380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc5b47e3c82f945e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:39:05.395 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.394+0000 7f712effd640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:39:05.404 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.399+0000 7f712effd640 1 -- 172.21.15.26:0/2933039517 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f713801f9b0 con 0x558556dc1410 2024-09-11T23:39:05.404 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.399+0000 7f71589e3740 1 -- 172.21.15.26:0/2933039517 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f714000ec10 con 0x558556dc1410 2024-09-11T23:39:05.405 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.399+0000 7f712effd640 1 -- 172.21.15.26:0/2933039517 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 8586+0+0 (secure 0 0 0) 0x7f713801fb50 con 0x558556dc1410 2024-09-11T23:39:05.405 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.399+0000 7f712effd640 1 -- 172.21.15.26:0/2933039517 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f713801fd70 con 0x558556dc1410 2024-09-11T23:39:05.405 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.399+0000 7f71589e3740 10 client.18078 fetch_fsmap learned FSMap version 90 2024-09-11T23:39:05.405 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.399+0000 7f71589e3740 1 -- 172.21.15.26:0/2933039517 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=90,monmap=4+}) -- 0x558556b04620 con 0x558556dc1410 2024-09-11T23:39:05.405 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.400+0000 7f712effd640 1 -- 172.21.15.26:0/2933039517 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f714000ec10 con 0x558556dc1410 2024-09-11T23:39:05.405 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.400+0000 7f712effd640 1 -- 172.21.15.26:0/2933039517 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 90) ==== 8418+0+0 (secure 0 0 0) 0x7f713800f550 con 0x558556dc1410 2024-09-11T23:39:05.405 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.401+0000 7f71589e3740 10 client.18078 fetch_fsmap finished waiting for FSMap version 90 2024-09-11T23:39:05.405 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.401+0000 7f71589e3740 10 client.18078 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:39:05.405 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.401+0000 7f71589e3740 20 client.18078 populate_metadata read hostname 'smithi026' 2024-09-11T23:39:05.405 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.401+0000 7f71589e3740 1 --2- 172.21.15.26:0/2933039517 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x558556dfe230 0x558556e006d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:05.405 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.401+0000 7f71589e3740 4 client.18078 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:39:05.405 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.401+0000 7f71589e3740 1 -- 172.21.15.26:0/2933039517 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558556d32a20 con 0x558556dfe230 2024-09-11T23:39:05.406 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.401+0000 7f714f7fe640 1 --2- 172.21.15.26:0/2933039517 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x558556dfe230 0x558556e006d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:05.406 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.401+0000 7f714f7fe640 1 --2- 172.21.15.26:0/2933039517 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x558556dfe230 0x558556e006d0 crc :-1 s=READY pgs=310 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=43dd8205f791f3f2 server_cookie=8ca524a9a981b14e in_seq=0 out_seq=0 2024-09-11T23:39:05.406 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.401+0000 7f712effd640 10 client.18078 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:39:05.406 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.402+0000 7f712effd640 1 -- 172.21.15.26:0/2933039517 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558556d32a20 con 0x558556dfe230 2024-09-11T23:39:05.406 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.402+0000 7f712effd640 10 client.18078 handle_command_reply: tid=0 2024-09-11T23:39:05.406 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.402+0000 7f71589e3740 1 client.18078 shutdown 2024-09-11T23:39:05.406 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.404+0000 7f71589e3740 1 -- 172.21.15.26:0/2933039517 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558556dc1410 msgr2=0x558556dc90d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:05.406 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.404+0000 7f71589e3740 1 --2- 172.21.15.26:0/2933039517 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558556dc1410 0x558556dc90d0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f71380126d0 tx=0x7f71380046c0 comp rx=0 tx=0).stop 2024-09-11T23:39:05.406 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.405+0000 7f71589e3740 1 -- 172.21.15.26:0/2933039517 shutdown_connections 2024-09-11T23:39:05.406 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.405+0000 7f71589e3740 1 --2- 172.21.15.26:0/2933039517 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558556dcbcb0 0x558556dc9610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:05.406 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.405+0000 7f71589e3740 1 --2- 172.21.15.26:0/2933039517 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x558556dfe230 0x558556e006d0 crc :-1 s=READY pgs=310 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:05.406 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.405+0000 7f71589e3740 1 --2- 172.21.15.26:0/2933039517 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558556dc1410 0x558556dc90d0 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:05.406 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.405+0000 7f71589e3740 1 --2- 172.21.15.26:0/2933039517 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558556dc0af0 0x558556dc8b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:05.406 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.405+0000 7f71589e3740 1 -- 172.21.15.26:0/2933039517 >> 172.21.15.26:0/2933039517 conn(0x558556dbd060 msgr2=0x558556dc2dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:39:05.407 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.405+0000 7f712effd640 0 client.18078 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:39:05.407 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.408+0000 7f71589e3740 1 -- 172.21.15.26:0/2933039517 shutdown_connections 2024-09-11T23:39:05.407 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.408+0000 7f71589e3740 1 -- 172.21.15.26:0/2933039517 wait complete. 2024-09-11T23:39:05.408 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.408+0000 7f71589e3740 20 client.18078 trim_cache size 0 max 16384 2024-09-11T23:39:05.435 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.436+0000 7f71589e3740 1 Processor -- start 2024-09-11T23:39:05.436 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.436+0000 7f71589e3740 1 -- start start 2024-09-11T23:39:05.436 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.437+0000 7f71589e3740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558556dc0af0 0x558556dc3560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:05.437 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.437+0000 7f71589e3740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558556dc1410 0x558556dc9440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:05.437 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.437+0000 7f71589e3740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558556dcbcb0 0x558556dcab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:05.437 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.437+0000 7f71589e3740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f713000e930 con 0x558556dcbcb0 2024-09-11T23:39:05.437 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.437+0000 7f71589e3740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f71300106e0 con 0x558556dc1410 2024-09-11T23:39:05.437 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.437+0000 7f71589e3740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f713000f030 con 0x558556dc0af0 2024-09-11T23:39:05.438 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.438+0000 7f7147fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558556dc1410 0x558556dc9440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:05.438 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.438+0000 7f714ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558556dcbcb0 0x558556dcab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:05.438 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.438+0000 7f714ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558556dcbcb0 0x558556dcab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40324/0 (socket says 172.21.15.26:40324) 2024-09-11T23:39:05.438 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.438+0000 7f714ffff640 1 -- 172.21.15.26:0/1974546382 learned_addr learned my addr 172.21.15.26:0/1974546382 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:39:05.438 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.438+0000 7f714f7fe640 1 --2- 172.21.15.26:0/1974546382 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558556dc0af0 0x558556dc3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:05.439 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.438+0000 7f714ffff640 1 -- 172.21.15.26:0/1974546382 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558556dc0af0 msgr2=0x558556dc3560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:39:05.439 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.438+0000 7f714ffff640 1 --2- 172.21.15.26:0/1974546382 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558556dc0af0 0x558556dc3560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:05.439 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.438+0000 7f714ffff640 1 -- 172.21.15.26:0/1974546382 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558556dc1410 msgr2=0x558556dc9440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:39:05.439 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.438+0000 7f714ffff640 1 --2- 172.21.15.26:0/1974546382 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558556dc1410 0x558556dc9440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:05.439 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.438+0000 7f714ffff640 1 -- 172.21.15.26:0/1974546382 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558556dc0920 con 0x558556dcbcb0 2024-09-11T23:39:05.439 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.438+0000 7f714f7fe640 1 --2- 172.21.15.26:0/1974546382 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558556dc0af0 0x558556dc3560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:39:05.439 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.439+0000 7f714ffff640 1 --2- 172.21.15.26:0/1974546382 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558556dcbcb0 0x558556dcab00 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f7148043000 tx=0x7f71480cc440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c506e04bf8c7148f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:39:05.439 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.439+0000 7f71457fa640 1 -- 172.21.15.26:0/1974546382 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71480ff030 con 0x558556dcbcb0 2024-09-11T23:39:05.440 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.439+0000 7f71457fa640 1 -- 172.21.15.26:0/1974546382 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71480cc900 con 0x558556dcbcb0 2024-09-11T23:39:05.440 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.439+0000 7f71457fa640 1 -- 172.21.15.26:0/1974546382 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f714803d040 con 0x558556dcbcb0 2024-09-11T23:39:05.441 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.441+0000 7f71589e3740 1 -- 172.21.15.26:0/1974546382 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558556dcbcb0 msgr2=0x558556dcab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:05.441 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.441+0000 7f71589e3740 1 --2- 172.21.15.26:0/1974546382 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558556dcbcb0 0x558556dcab00 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f7148043000 tx=0x7f71480cc440 comp rx=0 tx=0).stop 2024-09-11T23:39:05.441 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.441+0000 7f71589e3740 1 -- 172.21.15.26:0/1974546382 shutdown_connections 2024-09-11T23:39:05.441 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.441+0000 7f71589e3740 1 --2- 172.21.15.26:0/1974546382 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558556dc1410 0x558556dc9440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:05.441 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.441+0000 7f71589e3740 1 --2- 172.21.15.26:0/1974546382 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558556dcbcb0 0x558556dcab00 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:05.441 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.441+0000 7f71589e3740 1 --2- 172.21.15.26:0/1974546382 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558556dc0af0 0x558556dc3560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:05.441 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.441+0000 7f71589e3740 1 -- 172.21.15.26:0/1974546382 >> 172.21.15.26:0/1974546382 conn(0x558556dbd060 msgr2=0x558556dca3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:39:05.441 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.442+0000 7f71589e3740 1 -- 172.21.15.26:0/1974546382 shutdown_connections 2024-09-11T23:39:05.442 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.443+0000 7f71589e3740 1 -- 172.21.15.26:0/1974546382 wait complete. 2024-09-11T23:39:05.446 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.443+0000 7f71589e3740 1 -- start start 2024-09-11T23:39:05.446 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.443+0000 7f71589e3740 1 Processor -- start 2024-09-11T23:39:05.446 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.444+0000 7f71589e3740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558556dc0af0 0x558556dd4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:05.446 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.444+0000 7f71589e3740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558556dcbcb0 0x558556dcf8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:05.446 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.444+0000 7f71589e3740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558556dcff80 0x558556e67fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:05.446 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.444+0000 7f71589e3740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f713000ff50 con 0x558556dcff80 2024-09-11T23:39:05.446 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.444+0000 7f71589e3740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f713000f7c0 con 0x558556dcbcb0 2024-09-11T23:39:05.446 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.444+0000 7f71589e3740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f713000c930 con 0x558556dc0af0 2024-09-11T23:39:05.447 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.444+0000 7f71589e3740 10 client.0 _ll_register_callbacks cb 0x7f713800b940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:39:05.447 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.444+0000 7f714ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558556dcff80 0x558556e67fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:05.447 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.444+0000 7f714ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558556dcff80 0x558556e67fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40332/0 (socket says 172.21.15.26:40332) 2024-09-11T23:39:05.447 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.444+0000 7f714ffff640 1 -- 172.21.15.26:0/190750709 learned_addr learned my addr 172.21.15.26:0/190750709 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:39:05.447 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.445+0000 7f714f7fe640 1 --2- 172.21.15.26:0/190750709 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558556dc0af0 0x558556dd4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:05.447 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.445+0000 7f714ffff640 1 -- 172.21.15.26:0/190750709 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558556dc0af0 msgr2=0x558556dd4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:39:05.447 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.445+0000 7f714ffff640 1 --2- 172.21.15.26:0/190750709 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558556dc0af0 0x558556dd4610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:05.447 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.445+0000 7f714ffff640 1 -- 172.21.15.26:0/190750709 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558556dcbcb0 msgr2=0x558556dcf8e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T23:39:05.447 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.445+0000 7f714ffff640 1 --2- 172.21.15.26:0/190750709 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558556dcbcb0 0x558556dcf8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:05.447 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.445+0000 7f714ffff640 1 -- 172.21.15.26:0/190750709 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558556dd06f0 con 0x558556dcff80 2024-09-11T23:39:05.447 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.445+0000 7f714ffff640 1 --2- 172.21.15.26:0/190750709 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558556dcff80 0x558556e67fb0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f71480c8d60 tx=0x7f71480fdeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b76e3e116fa3aee7 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:39:05.447 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.446+0000 7f712effd640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:39:05.447 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.446+0000 7f712effd640 1 -- 172.21.15.26:0/190750709 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71480ff070 con 0x558556dcff80 2024-09-11T23:39:05.448 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.446+0000 7f71589e3740 1 -- 172.21.15.26:0/190750709 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f71480cc900 con 0x558556dcff80 2024-09-11T23:39:05.448 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.446+0000 7f712effd640 1 -- 172.21.15.26:0/190750709 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 8586+0+0 (secure 0 0 0) 0x7f71480d50e0 con 0x558556dcff80 2024-09-11T23:39:05.448 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.446+0000 7f712effd640 1 -- 172.21.15.26:0/190750709 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f714803d040 con 0x558556dcff80 2024-09-11T23:39:05.448 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.446+0000 7f71589e3740 10 client.18084 fetch_fsmap learned FSMap version 90 2024-09-11T23:39:05.448 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.446+0000 7f71589e3740 1 -- 172.21.15.26:0/190750709 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=90,monmap=4+}) -- 0x558556e69530 con 0x558556dcff80 2024-09-11T23:39:05.450 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.447+0000 7f712effd640 1 -- 172.21.15.26:0/190750709 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71480cc900 con 0x558556dcff80 2024-09-11T23:39:05.450 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.447+0000 7f712effd640 1 -- 172.21.15.26:0/190750709 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 90) ==== 8418+0+0 (secure 0 0 0) 0x7f71480eb020 con 0x558556dcff80 2024-09-11T23:39:05.450 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.447+0000 7f71589e3740 10 client.18084 fetch_fsmap finished waiting for FSMap version 90 2024-09-11T23:39:05.450 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.447+0000 7f71589e3740 10 client.18084 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:39:05.450 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.447+0000 7f71589e3740 20 client.18084 populate_metadata read hostname 'smithi026' 2024-09-11T23:39:05.450 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.447+0000 7f71589e3740 1 --2- 172.21.15.26:0/190750709 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x558556e69bb0 0x558556e6bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:05.450 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.447+0000 7f71589e3740 4 client.18084 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T23:39:05.450 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.447+0000 7f71589e3740 1 -- 172.21.15.26:0/190750709 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558556d2cd70 con 0x558556e69bb0 2024-09-11T23:39:05.450 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.450+0000 7f714f7fe640 1 --2- 172.21.15.26:0/190750709 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x558556e69bb0 0x558556e6bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:05.456 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.455+0000 7f714f7fe640 1 --2- 172.21.15.26:0/190750709 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x558556e69bb0 0x558556e6bfd0 crc :-1 s=READY pgs=311 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e938ee038c64a8a server_cookie=ae9100292901c71a in_seq=0 out_seq=0 2024-09-11T23:39:05.456 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.455+0000 7f712effd640 10 client.18084 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:39:05.590 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.590+0000 7f712effd640 1 -- 172.21.15.26:0/190750709 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558556d2cd70 con 0x558556e69bb0 2024-09-11T23:39:05.590 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.590+0000 7f712effd640 10 client.18084 handle_command_reply: tid=0 2024-09-11T23:39:05.590 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.590+0000 7f71589e3740 1 client.18084 shutdown 2024-09-11T23:39:05.590 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.590+0000 7f71589e3740 1 -- 172.21.15.26:0/190750709 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558556dcff80 msgr2=0x558556e67fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:05.590 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.590+0000 7f71589e3740 1 --2- 172.21.15.26:0/190750709 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558556dcff80 0x558556e67fb0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f71480c8d60 tx=0x7f71480fdeb0 comp rx=0 tx=0).stop 2024-09-11T23:39:05.590 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.590+0000 7f71589e3740 1 -- 172.21.15.26:0/190750709 shutdown_connections 2024-09-11T23:39:05.591 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.590+0000 7f71589e3740 1 --2- 172.21.15.26:0/190750709 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558556dcbcb0 0x558556dcf8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:05.591 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.590+0000 7f71589e3740 1 --2- 172.21.15.26:0/190750709 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x558556e69bb0 0x558556e6bfd0 crc :-1 s=READY pgs=311 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:05.591 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.590+0000 7f71589e3740 1 --2- 172.21.15.26:0/190750709 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558556dcff80 0x558556e67fb0 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:05.591 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.590+0000 7f71589e3740 1 --2- 172.21.15.26:0/190750709 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558556dc0af0 0x558556dd4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:05.591 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.590+0000 7f712effd640 0 client.18084 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:39:05.591 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.590+0000 7f71589e3740 1 -- 172.21.15.26:0/190750709 >> 172.21.15.26:0/190750709 conn(0x558556dbd060 msgr2=0x558556dbc1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:39:05.591 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.590+0000 7f71589e3740 1 -- 172.21.15.26:0/190750709 shutdown_connections 2024-09-11T23:39:05.591 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.590+0000 7f71589e3740 1 -- 172.21.15.26:0/190750709 wait complete. 2024-09-11T23:39:05.591 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.591+0000 7f71589e3740 20 client.18084 trim_cache size 0 max 16384 2024-09-11T23:39:05.592 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-09-11T23:39:05.592 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "message": "", 2024-09-11T23:39:05.592 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-09-11T23:39:05.592 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-09-11T23:39:05.593 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.592+0000 7f7155c4c640 1 -- 172.21.15.26:0/73943422 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f713c0a2ef0 msgr2=0x7f713c0a53b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:05.593 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.592+0000 7f7155c4c640 1 --2- 172.21.15.26:0/73943422 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f713c0a2ef0 0x7f713c0a53b0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x558556dbe3e0 tx=0x7f713800b3a0 comp rx=0 tx=0).stop 2024-09-11T23:39:05.593 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.592+0000 7f7155c4c640 1 -- 172.21.15.26:0/73943422 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f715011b140 msgr2=0x7f7150116d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:05.593 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.592+0000 7f7155c4c640 1 --2- 172.21.15.26:0/73943422 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f715011b140 0x7f7150116d80 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f714801ace0 tx=0x7f7148003ba0 comp rx=0 tx=0).stop 2024-09-11T23:39:05.593 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.592+0000 7f7155c4c640 1 -- 172.21.15.26:0/73943422 shutdown_connections 2024-09-11T23:39:05.593 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.592+0000 7f7155c4c640 1 --2- 172.21.15.26:0/73943422 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f713c0a2ef0 0x7f713c0a53b0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:05.593 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.592+0000 7f7155c4c640 1 --2- 172.21.15.26:0/73943422 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f715011b140 0x7f7150116d80 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:05.593 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.592+0000 7f7155c4c640 1 --2- 172.21.15.26:0/73943422 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f71501095d0 0x7f7150116840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:05.593 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.592+0000 7f7155c4c640 1 --2- 172.21.15.26:0/73943422 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f71501091f0 0x7f715011a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:05.593 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.592+0000 7f7155c4c640 1 -- 172.21.15.26:0/73943422 >> 172.21.15.26:0/73943422 conn(0x7f715006ce00 msgr2=0x7f71500707f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:39:05.593 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.592+0000 7f7155c4c640 1 -- 172.21.15.26:0/73943422 shutdown_connections 2024-09-11T23:39:05.594 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:05.592+0000 7f7155c4c640 1 -- 172.21.15.26:0/73943422 wait complete. 2024-09-11T23:39:06.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:39:05 smithi136 ceph-mon[35121]: pgmap v653: 97 pgs: 2 active+clean+snaptrim, 2 active+clean+snaptrim_wait, 93 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.5 MiB/s wr, 182 op/s 2024-09-11T23:39:06.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:39:05 smithi136 ceph-mon[35121]: Adjusting osd_memory_target on smithi076 to 176.0M 2024-09-11T23:39:06.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:39:05 smithi136 ceph-mon[35121]: Unable to set osd_memory_target on smithi076 to 184591052: error parsing value: Value '184591052' is below minimum 939524096 2024-09-11T23:39:06.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:39:05 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:39:06.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:39:05 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:39:06.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:39:05 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:39:06.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:39:05 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:39:06.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:39:05 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:39:06.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:39:05 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:39:06.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:39:05 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:39:06.240 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:39:05 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:39:06.240 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:39:05 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:39:06.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:39:05 smithi026 ceph-mon[31172]: pgmap v653: 97 pgs: 2 active+clean+snaptrim, 2 active+clean+snaptrim_wait, 93 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.5 MiB/s wr, 182 op/s 2024-09-11T23:39:06.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:39:05 smithi026 ceph-mon[31172]: Adjusting osd_memory_target on smithi076 to 176.0M 2024-09-11T23:39:06.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:39:05 smithi026 ceph-mon[31172]: Unable to set osd_memory_target on smithi076 to 184591052: error parsing value: Value '184591052' is below minimum 939524096 2024-09-11T23:39:06.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:39:05 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:39:06.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:39:05 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:39:06.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:39:05 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:39:06.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:39:05 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:39:06.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:39:05 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:39:06.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:39:05 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:39:06.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:39:05 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:39:06.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:39:05 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:39:06.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:39:05 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:39:06.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:39:05 smithi076 ceph-mon[35296]: pgmap v653: 97 pgs: 2 active+clean+snaptrim, 2 active+clean+snaptrim_wait, 93 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.5 MiB/s wr, 182 op/s 2024-09-11T23:39:06.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:39:05 smithi076 ceph-mon[35296]: Adjusting osd_memory_target on smithi076 to 176.0M 2024-09-11T23:39:06.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:39:05 smithi076 ceph-mon[35296]: Unable to set osd_memory_target on smithi076 to 184591052: error parsing value: Value '184591052' is below minimum 939524096 2024-09-11T23:39:06.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:39:05 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:39:06.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:39:05 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:39:06.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:39:05 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:39:06.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:39:05 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:39:06.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:39:05 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:39:06.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:39:05 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:39:06.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:39:05 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:39:06.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:39:05 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:39:06.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:39:05 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:39:07.238 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:39:06 smithi136 ceph-mon[35121]: Adjusting osd_memory_target on smithi136 to 176.0M 2024-09-11T23:39:07.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:39:06 smithi136 ceph-mon[35121]: Unable to set osd_memory_target on smithi136 to 184574566: error parsing value: Value '184574566' is below minimum 939524096 2024-09-11T23:39:07.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:39:06 smithi026 ceph-mon[31172]: Adjusting osd_memory_target on smithi136 to 176.0M 2024-09-11T23:39:07.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:39:06 smithi026 ceph-mon[31172]: Unable to set osd_memory_target on smithi136 to 184574566: error parsing value: Value '184574566' is below minimum 939524096 2024-09-11T23:39:07.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:39:06 smithi076 ceph-mon[35296]: Adjusting osd_memory_target on smithi136 to 176.0M 2024-09-11T23:39:07.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:39:06 smithi076 ceph-mon[35296]: Unable to set osd_memory_target on smithi136 to 184574566: error parsing value: Value '184574566' is below minimum 939524096 2024-09-11T23:39:08.238 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:39:07 smithi136 ceph-mon[35121]: pgmap v654: 97 pgs: 21 active+clean+snaptrim, 36 active+clean+snaptrim_wait, 40 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.1 MiB/s wr, 354 op/s 2024-09-11T23:39:08.254 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:39:07 smithi026 ceph-mon[31172]: pgmap v654: 97 pgs: 21 active+clean+snaptrim, 36 active+clean+snaptrim_wait, 40 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.1 MiB/s wr, 354 op/s 2024-09-11T23:39:08.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:39:07 smithi076 ceph-mon[35296]: pgmap v654: 97 pgs: 21 active+clean+snaptrim, 36 active+clean+snaptrim_wait, 40 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.1 MiB/s wr, 354 op/s 2024-09-11T23:39:09.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:39:08 smithi026 ceph-mon[31172]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:39:09.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:39:08 smithi076 ceph-mon[35296]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:39:09.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:39:08 smithi136 ceph-mon[35121]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:39:10.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:39:10 smithi076 ceph-mon[35296]: pgmap v655: 97 pgs: 23 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 33 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.1 MiB/s wr, 305 op/s 2024-09-11T23:39:10.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:39:10 smithi136 ceph-mon[35121]: pgmap v655: 97 pgs: 23 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 33 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.1 MiB/s wr, 305 op/s 2024-09-11T23:39:10.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:39:10 smithi026 ceph-mon[31172]: pgmap v655: 97 pgs: 23 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 33 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.1 MiB/s wr, 305 op/s 2024-09-11T23:39:12.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:39:12 smithi026 ceph-mon[31172]: pgmap v656: 97 pgs: 23 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 33 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.6 MiB/s wr, 260 op/s 2024-09-11T23:39:12.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:39:12 smithi076 ceph-mon[35296]: pgmap v656: 97 pgs: 23 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 33 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.6 MiB/s wr, 260 op/s 2024-09-11T23:39:12.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:39:12 smithi136 ceph-mon[35121]: pgmap v656: 97 pgs: 23 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 33 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.6 MiB/s wr, 260 op/s 2024-09-11T23:39:14.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:39:14 smithi076 ceph-mon[35296]: pgmap v657: 97 pgs: 23 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 33 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.8 MiB/s wr, 239 op/s 2024-09-11T23:39:14.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:39:14 smithi136 ceph-mon[35121]: pgmap v657: 97 pgs: 23 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 33 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.8 MiB/s wr, 239 op/s 2024-09-11T23:39:14.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:39:14 smithi026 ceph-mon[31172]: pgmap v657: 97 pgs: 23 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 33 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.8 MiB/s wr, 239 op/s 2024-09-11T23:39:16.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:39:16 smithi076 ceph-mon[35296]: pgmap v658: 97 pgs: 23 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 33 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.6 MiB/s wr, 224 op/s 2024-09-11T23:39:16.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:39:16 smithi136 ceph-mon[35121]: pgmap v658: 97 pgs: 23 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 33 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.6 MiB/s wr, 224 op/s 2024-09-11T23:39:16.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:39:16 smithi026 ceph-mon[31172]: pgmap v658: 97 pgs: 23 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 33 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.6 MiB/s wr, 224 op/s 2024-09-11T23:39:18.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:39:18 smithi076 ceph-mon[35296]: pgmap v659: 97 pgs: 23 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 33 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.6 MiB/s wr, 257 op/s 2024-09-11T23:39:18.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:39:18 smithi136 ceph-mon[35121]: pgmap v659: 97 pgs: 23 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 33 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.6 MiB/s wr, 257 op/s 2024-09-11T23:39:18.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:39:18 smithi026 ceph-mon[31172]: pgmap v659: 97 pgs: 23 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 33 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.6 MiB/s wr, 257 op/s 2024-09-11T23:39:18.724 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.724+0000 7f17eabe4640 1 -- 172.21.15.26:0/3168228890 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f17e410c0b0 msgr2=0x7f17e410e550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:18.724 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.724+0000 7f17eabe4640 1 --2- 172.21.15.26:0/3168228890 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f17e410c0b0 0x7f17e410e550 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f17dc009ca0 tx=0x7f17dc02a950 comp rx=0 tx=0).stop 2024-09-11T23:39:18.725 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.724+0000 7f17eabe4640 1 -- 172.21.15.26:0/3168228890 shutdown_connections 2024-09-11T23:39:18.725 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.724+0000 7f17eabe4640 1 --2- 172.21.15.26:0/3168228890 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f17e410c0b0 0x7f17e410e550 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:18.725 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.724+0000 7f17eabe4640 1 --2- 172.21.15.26:0/3168228890 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f17e4103600 0x7f17e410bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:18.725 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.725+0000 7f17eabe4640 1 --2- 172.21.15.26:0/3168228890 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f17e4102ce0 0x7f17e41030c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:18.725 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.725+0000 7f17eabe4640 1 -- 172.21.15.26:0/3168228890 >> 172.21.15.26:0/3168228890 conn(0x7f17e4077990 msgr2=0x7f17e40fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:39:18.725 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.725+0000 7f17eabe4640 1 -- 172.21.15.26:0/3168228890 shutdown_connections 2024-09-11T23:39:18.725 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.725+0000 7f17eabe4640 1 -- 172.21.15.26:0/3168228890 wait complete. 2024-09-11T23:39:18.727 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.726+0000 7f17eabe4640 1 Processor -- start 2024-09-11T23:39:18.727 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.726+0000 7f17eabe4640 1 -- start start 2024-09-11T23:39:18.727 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.727+0000 7f17eabe4640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f17e4102ce0 0x7f17e41982c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:18.728 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.727+0000 7f17eabe4640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f17e4103600 0x7f17e4198800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:18.728 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.727+0000 7f17eabe4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f17e410c0b0 0x7f17e419cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:18.728 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.727+0000 7f17eabe4640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f17e4111e90 con 0x7f17e410c0b0 2024-09-11T23:39:18.728 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.727+0000 7f17eabe4640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f17e4111d10 con 0x7f17e4102ce0 2024-09-11T23:39:18.728 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.727+0000 7f17e8980640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f17e4102ce0 0x7f17e41982c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:18.728 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.727+0000 7f17e8980640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f17e4102ce0 0x7f17e41982c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:46304/0 (socket says 172.21.15.26:46304) 2024-09-11T23:39:18.728 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.727+0000 7f17e3fff640 1 --2- 172.21.15.26:0/4137825675 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f17e4103600 0x7f17e4198800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:18.728 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.727+0000 7f17e8980640 1 -- 172.21.15.26:0/4137825675 learned_addr learned my addr 172.21.15.26:0/4137825675 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:39:18.728 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.727+0000 7f17e9181640 1 --2- 172.21.15.26:0/4137825675 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f17e410c0b0 0x7f17e419cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:18.728 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.727+0000 7f17eabe4640 1 -- 172.21.15.26:0/4137825675 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f17e4112010 con 0x7f17e4103600 2024-09-11T23:39:18.729 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.728+0000 7f17e9181640 1 -- 172.21.15.26:0/4137825675 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f17e4103600 msgr2=0x7f17e4198800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:18.729 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.728+0000 7f17e9181640 1 --2- 172.21.15.26:0/4137825675 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f17e4103600 0x7f17e4198800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:18.729 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.728+0000 7f17e9181640 1 -- 172.21.15.26:0/4137825675 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f17e4102ce0 msgr2=0x7f17e41982c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:18.729 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.728+0000 7f17e9181640 1 --2- 172.21.15.26:0/4137825675 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f17e4102ce0 0x7f17e41982c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:18.729 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.728+0000 7f17e9181640 1 -- 172.21.15.26:0/4137825675 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17e419d2b0 con 0x7f17e410c0b0 2024-09-11T23:39:18.729 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.728+0000 7f17e9181640 1 --2- 172.21.15.26:0/4137825675 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f17e410c0b0 0x7f17e419cb80 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f17dc005cc0 tx=0x7f17dc02f3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:39:18.730 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.728+0000 7f17e1ffb640 1 -- 172.21.15.26:0/4137825675 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17dc034040 con 0x7f17e410c0b0 2024-09-11T23:39:18.730 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.728+0000 7f17e1ffb640 1 -- 172.21.15.26:0/4137825675 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17dc030920 con 0x7f17e410c0b0 2024-09-11T23:39:18.730 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.728+0000 7f17e1ffb640 1 -- 172.21.15.26:0/4137825675 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17dc004580 con 0x7f17e410c0b0 2024-09-11T23:39:18.730 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.728+0000 7f17eabe4640 1 -- 172.21.15.26:0/4137825675 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17e419d540 con 0x7f17e410c0b0 2024-09-11T23:39:18.730 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.729+0000 7f17eabe4640 1 -- 172.21.15.26:0/4137825675 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17e419d9b0 con 0x7f17e410c0b0 2024-09-11T23:39:18.736 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.735+0000 7f17eda5b740 1 Processor -- start 2024-09-11T23:39:18.736 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.736+0000 7f17eda5b740 1 -- start start 2024-09-11T23:39:18.737 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.736+0000 7f17e1ffb640 1 -- 172.21.15.26:0/4137825675 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f17dc037020 con 0x7f17e410c0b0 2024-09-11T23:39:18.738 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.737+0000 7f17eda5b740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5636f6c13cc0 0x5636f6c140a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:18.738 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.737+0000 7f17eda5b740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5636f6c0f660 0x5636f6c0fac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:18.738 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.737+0000 7f17eda5b740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5636f6c10000 0x5636f6c1b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:18.738 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.737+0000 7f17eda5b740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5636f6b6a2f0 con 0x5636f6c13cc0 2024-09-11T23:39:18.738 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.737+0000 7f17eda5b740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5636f6b12270 con 0x5636f6c10000 2024-09-11T23:39:18.738 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.737+0000 7f17eda5b740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5636f6b662b0 con 0x5636f6c0f660 2024-09-11T23:39:18.738 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.737+0000 7f17e3fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5636f6c13cc0 0x5636f6c140a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:18.738 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.738+0000 7f17e3fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5636f6c13cc0 0x5636f6c140a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53726/0 (socket says 172.21.15.26:53726) 2024-09-11T23:39:18.738 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.738+0000 7f17e3fff640 1 -- 172.21.15.26:0/918173315 learned_addr learned my addr 172.21.15.26:0/918173315 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:39:18.738 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.738+0000 7f17e8980640 1 --2- 172.21.15.26:0/918173315 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5636f6c0f660 0x5636f6c0fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:18.738 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.738+0000 7f17e3fff640 1 --2- 172.21.15.26:0/918173315 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5636f6c10000 0x5636f6c1b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:18.740 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.739+0000 7f17e3fff640 1 -- 172.21.15.26:0/918173315 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5636f6c0f660 msgr2=0x5636f6c0fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:39:18.740 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.739+0000 7f17e3fff640 1 --2- 172.21.15.26:0/918173315 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5636f6c0f660 0x5636f6c0fac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:18.740 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.739+0000 7f17e3fff640 1 -- 172.21.15.26:0/918173315 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5636f6c10000 msgr2=0x5636f6c1b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:39:18.740 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.739+0000 7f17e3fff640 1 --2- 172.21.15.26:0/918173315 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5636f6c10000 0x5636f6c1b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:18.740 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.739+0000 7f17e3fff640 1 -- 172.21.15.26:0/918173315 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636f6b11340 con 0x5636f6c13cc0 2024-09-11T23:39:18.740 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.739+0000 7f17e1ffb640 1 --2- 172.21.15.26:0/4137825675 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f17d40a2ea0 0x7f17d40a5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:18.740 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.739+0000 7f17e3fff640 1 --2- 172.21.15.26:0/918173315 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5636f6c13cc0 0x5636f6c140a0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f17d0013790 tx=0x7f17d0005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1942d6fc01557468 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:39:18.740 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.739+0000 7f17e1ffb640 1 -- 172.21.15.26:0/4137825675 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 8586+0+0 (secure 0 0 0) 0x7f17dc0f00a0 con 0x7f17e410c0b0 2024-09-11T23:39:18.740 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.740+0000 7f17e8980640 1 --2- 172.21.15.26:0/4137825675 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f17d40a2ea0 0x7f17d40a5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:18.740 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.740+0000 7f17ceffd640 1 -- 172.21.15.26:0/918173315 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17d0013d10 con 0x5636f6c13cc0 2024-09-11T23:39:18.740 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.740+0000 7f17ceffd640 1 -- 172.21.15.26:0/918173315 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17d0005020 con 0x5636f6c13cc0 2024-09-11T23:39:18.741 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.740+0000 7f17ceffd640 1 -- 172.21.15.26:0/918173315 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17d0004a90 con 0x5636f6c13cc0 2024-09-11T23:39:18.741 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.741+0000 7f17eda5b740 1 -- 172.21.15.26:0/918173315 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5636f6c13cc0 msgr2=0x5636f6c140a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:18.741 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.741+0000 7f17eda5b740 1 --2- 172.21.15.26:0/918173315 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5636f6c13cc0 0x5636f6c140a0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f17d0013790 tx=0x7f17d0005f90 comp rx=0 tx=0).stop 2024-09-11T23:39:18.742 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.741+0000 7f17e8980640 1 --2- 172.21.15.26:0/4137825675 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f17d40a2ea0 0x7f17d40a5360 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x5636f6c03260 tx=0x7f17d8005f90 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:39:18.742 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.741+0000 7f17eda5b740 1 -- 172.21.15.26:0/918173315 shutdown_connections 2024-09-11T23:39:18.742 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.741+0000 7f17eda5b740 1 --2- 172.21.15.26:0/918173315 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5636f6c10000 0x5636f6c1b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:18.742 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.741+0000 7f17eda5b740 1 --2- 172.21.15.26:0/918173315 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5636f6c0f660 0x5636f6c0fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:18.742 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.741+0000 7f17eda5b740 1 --2- 172.21.15.26:0/918173315 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5636f6c13cc0 0x5636f6c140a0 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:18.742 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.741+0000 7f17eda5b740 1 -- 172.21.15.26:0/918173315 >> 172.21.15.26:0/918173315 conn(0x5636f6c01b10 msgr2=0x5636f6c01ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:39:18.742 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.741+0000 7f17eda5b740 1 -- 172.21.15.26:0/918173315 shutdown_connections 2024-09-11T23:39:18.742 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.741+0000 7f17eda5b740 1 -- 172.21.15.26:0/918173315 wait complete. 2024-09-11T23:39:18.742 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.742+0000 7f17eda5b740 1 -- start start 2024-09-11T23:39:18.742 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.742+0000 7f17eda5b740 1 Processor -- start 2024-09-11T23:39:18.745 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.742+0000 7f17eda5b740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5636f6c0f660 0x5636f6c18e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:18.745 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.742+0000 7f17eda5b740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5636f6c10000 0x5636f6c193d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:18.745 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.742+0000 7f17eda5b740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5636f6c13cc0 0x5636f6c19910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:18.746 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.742+0000 7f17eda5b740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5636f6c1c150 con 0x5636f6c13cc0 2024-09-11T23:39:18.746 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.742+0000 7f17eda5b740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5636f6af3290 con 0x5636f6c10000 2024-09-11T23:39:18.746 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.742+0000 7f17eda5b740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5636f6c1c2d0 con 0x5636f6c0f660 2024-09-11T23:39:18.746 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.742+0000 7f17eda5b740 10 client.0 _ll_register_callbacks cb 0x5636f6b24140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:39:18.746 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.743+0000 7f17e9181640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5636f6c13cc0 0x5636f6c19910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:18.746 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.743+0000 7f17e8980640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5636f6c0f660 0x5636f6c18e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:18.746 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.743+0000 7f17e9181640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5636f6c13cc0 0x5636f6c19910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53728/0 (socket says 172.21.15.26:53728) 2024-09-11T23:39:18.746 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.743+0000 7f17e9181640 1 -- 172.21.15.26:0/387533960 learned_addr learned my addr 172.21.15.26:0/387533960 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:39:18.746 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.743+0000 7f17e3fff640 1 --2- 172.21.15.26:0/387533960 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5636f6c10000 0x5636f6c193d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:18.746 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.743+0000 7f17e9181640 1 -- 172.21.15.26:0/387533960 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5636f6c0f660 msgr2=0x5636f6c18e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:39:18.746 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.743+0000 7f17e9181640 1 --2- 172.21.15.26:0/387533960 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5636f6c0f660 0x5636f6c18e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:18.746 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.743+0000 7f17e9181640 1 -- 172.21.15.26:0/387533960 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5636f6c10000 msgr2=0x5636f6c193d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:39:18.747 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.743+0000 7f17e9181640 1 --2- 172.21.15.26:0/387533960 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5636f6c10000 0x5636f6c193d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:18.747 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.743+0000 7f17e9181640 1 -- 172.21.15.26:0/387533960 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636f687d210 con 0x5636f6c13cc0 2024-09-11T23:39:18.747 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.743+0000 7f17e8980640 1 --2- 172.21.15.26:0/387533960 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5636f6c0f660 0x5636f6c18e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:39:18.747 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.743+0000 7f17e9181640 1 --2- 172.21.15.26:0/387533960 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5636f6c13cc0 0x5636f6c19910 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f17dc0b9000 tx=0x7f17dc0bc050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0d3929ae1317af8 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:39:18.747 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.743+0000 7f17c3fff640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:39:18.747 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.744+0000 7f17c3fff640 1 -- 172.21.15.26:0/387533960 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17dc0edbd0 con 0x5636f6c13cc0 2024-09-11T23:39:18.747 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.744+0000 7f17c3fff640 1 -- 172.21.15.26:0/387533960 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 8586+0+0 (secure 0 0 0) 0x7f17dc0c4e10 con 0x5636f6c13cc0 2024-09-11T23:39:18.747 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.744+0000 7f17eda5b740 1 -- 172.21.15.26:0/387533960 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f17d0005020 con 0x5636f6c13cc0 2024-09-11T23:39:18.747 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.744+0000 7f17c3fff640 1 -- 172.21.15.26:0/387533960 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f17dc0bc630 con 0x5636f6c13cc0 2024-09-11T23:39:18.747 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.744+0000 7f17eda5b740 10 client.18096 fetch_fsmap learned FSMap version 91 2024-09-11T23:39:18.747 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.744+0000 7f17eda5b740 1 -- 172.21.15.26:0/387533960 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=91,monmap=4+}) -- 0x5636f6949620 con 0x5636f6c13cc0 2024-09-11T23:39:18.748 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.744+0000 7f17c3fff640 1 -- 172.21.15.26:0/387533960 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17d0005020 con 0x5636f6c13cc0 2024-09-11T23:39:18.748 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.744+0000 7f17c3fff640 1 -- 172.21.15.26:0/387533960 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 91) ==== 8418+0+0 (secure 0 0 0) 0x7f17dc0a5b70 con 0x5636f6c13cc0 2024-09-11T23:39:18.748 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.744+0000 7f17eda5b740 10 client.18096 fetch_fsmap finished waiting for FSMap version 91 2024-09-11T23:39:18.748 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.744+0000 7f17eda5b740 10 client.18096 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:39:18.748 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.744+0000 7f17eda5b740 20 client.18096 populate_metadata read hostname 'smithi026' 2024-09-11T23:39:18.748 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.744+0000 7f17eda5b740 1 --2- 172.21.15.26:0/387533960 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5636f6c295e0 0x5636f6c2ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:18.748 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.744+0000 7f17eda5b740 4 client.18096 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:39:18.748 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.744+0000 7f17eda5b740 1 -- 172.21.15.26:0/387533960 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5636f6b77a20 con 0x5636f6c295e0 2024-09-11T23:39:18.748 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.745+0000 7f17e8980640 1 --2- 172.21.15.26:0/387533960 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5636f6c295e0 0x5636f6c2ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:18.748 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.745+0000 7f17e8980640 1 --2- 172.21.15.26:0/387533960 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5636f6c295e0 0x5636f6c2ba20 crc :-1 s=READY pgs=312 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f1862b93f9191ea9 server_cookie=666017b643eaaddd in_seq=0 out_seq=0 2024-09-11T23:39:18.748 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.745+0000 7f17c3fff640 10 client.18096 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:39:18.749 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.746+0000 7f17c3fff640 1 -- 172.21.15.26:0/387533960 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5636f6b77a20 con 0x5636f6c295e0 2024-09-11T23:39:18.749 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.746+0000 7f17c3fff640 10 client.18096 handle_command_reply: tid=0 2024-09-11T23:39:18.750 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.746+0000 7f17eda5b740 1 client.18096 shutdown 2024-09-11T23:39:18.750 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.746+0000 7f17eda5b740 1 -- 172.21.15.26:0/387533960 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5636f6c13cc0 msgr2=0x5636f6c19910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:18.750 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.746+0000 7f17eda5b740 1 --2- 172.21.15.26:0/387533960 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5636f6c13cc0 0x5636f6c19910 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f17dc0b9000 tx=0x7f17dc0bc050 comp rx=0 tx=0).stop 2024-09-11T23:39:18.750 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.747+0000 7f17eda5b740 1 -- 172.21.15.26:0/387533960 shutdown_connections 2024-09-11T23:39:18.750 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.747+0000 7f17eda5b740 1 --2- 172.21.15.26:0/387533960 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5636f6c10000 0x5636f6c193d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:18.750 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.747+0000 7f17eda5b740 1 --2- 172.21.15.26:0/387533960 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5636f6c295e0 0x5636f6c2ba20 crc :-1 s=READY pgs=312 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:18.751 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.747+0000 7f17eda5b740 1 --2- 172.21.15.26:0/387533960 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5636f6c13cc0 0x5636f6c19910 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:18.751 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.747+0000 7f17eda5b740 1 --2- 172.21.15.26:0/387533960 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5636f6c0f660 0x5636f6c18e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:18.751 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.747+0000 7f17eda5b740 1 -- 172.21.15.26:0/387533960 >> 172.21.15.26:0/387533960 conn(0x5636f6c01b10 msgr2=0x5636f6c0ee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:39:18.751 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.747+0000 7f17c3fff640 0 client.18096 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:39:18.751 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.747+0000 7f17eda5b740 1 -- 172.21.15.26:0/387533960 shutdown_connections 2024-09-11T23:39:18.751 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.747+0000 7f17eda5b740 1 -- 172.21.15.26:0/387533960 wait complete. 2024-09-11T23:39:18.751 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.747+0000 7f17eda5b740 20 client.18096 trim_cache size 0 max 16384 2024-09-11T23:39:18.770 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.769+0000 7f17eda5b740 1 Processor -- start 2024-09-11T23:39:18.770 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.769+0000 7f17eda5b740 1 -- start start 2024-09-11T23:39:18.770 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.769+0000 7f17eda5b740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5636f6c0f660 0x5636f6c31330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:18.771 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.769+0000 7f17eda5b740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5636f6c10000 0x5636f6c31870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:18.771 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.769+0000 7f17eda5b740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5636f6c13cc0 0x5636f6c0e650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:18.771 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.769+0000 7f17e8980640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5636f6c0f660 0x5636f6c31330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:18.771 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.769+0000 7f17e8980640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5636f6c0f660 0x5636f6c31330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:46318/0 (socket says 172.21.15.26:46318) 2024-09-11T23:39:18.771 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.769+0000 7f17e8980640 1 -- 172.21.15.26:0/2520469750 learned_addr learned my addr 172.21.15.26:0/2520469750 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:39:18.771 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.769+0000 7f17e9181640 1 --2- 172.21.15.26:0/2520469750 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5636f6c13cc0 0x5636f6c0e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:18.771 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.769+0000 7f17eda5b740 1 -- 172.21.15.26:0/2520469750 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f17e41ab550 con 0x5636f6c13cc0 2024-09-11T23:39:18.771 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.769+0000 7f17eda5b740 1 -- 172.21.15.26:0/2520469750 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f17e41ad300 con 0x5636f6c0f660 2024-09-11T23:39:18.771 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.769+0000 7f17e3fff640 1 --2- 172.21.15.26:0/2520469750 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5636f6c10000 0x5636f6c31870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:18.771 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.769+0000 7f17eda5b740 1 -- 172.21.15.26:0/2520469750 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f17e41abc50 con 0x5636f6c10000 2024-09-11T23:39:18.771 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.770+0000 7f17e8980640 1 -- 172.21.15.26:0/2520469750 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5636f6c10000 msgr2=0x5636f6c31870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:39:18.771 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.770+0000 7f17e8980640 1 --2- 172.21.15.26:0/2520469750 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5636f6c10000 0x5636f6c31870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:18.771 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.770+0000 7f17e8980640 1 -- 172.21.15.26:0/2520469750 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5636f6c13cc0 msgr2=0x5636f6c0e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:39:18.772 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.770+0000 7f17e8980640 1 --2- 172.21.15.26:0/2520469750 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5636f6c13cc0 0x5636f6c0e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:18.772 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.770+0000 7f17e8980640 1 -- 172.21.15.26:0/2520469750 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636f6c173c0 con 0x5636f6c0f660 2024-09-11T23:39:18.772 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.770+0000 7f17e8980640 1 --2- 172.21.15.26:0/2520469750 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5636f6c0f660 0x5636f6c31330 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f17d8012cb0 tx=0x7f17d801b000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f66e0f8a8dc82c72 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:39:18.772 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.770+0000 7f17ce7fc640 1 -- 172.21.15.26:0/2520469750 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17d803a070 con 0x5636f6c0f660 2024-09-11T23:39:18.772 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.771+0000 7f17ce7fc640 1 -- 172.21.15.26:0/2520469750 <== mon.2 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17d802c9a0 con 0x5636f6c0f660 2024-09-11T23:39:18.772 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.771+0000 7f17ce7fc640 1 -- 172.21.15.26:0/2520469750 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17d8035bb0 con 0x5636f6c0f660 2024-09-11T23:39:18.775 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.772+0000 7f17eda5b740 1 -- 172.21.15.26:0/2520469750 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5636f6c0f660 msgr2=0x5636f6c31330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:18.776 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.772+0000 7f17eda5b740 1 --2- 172.21.15.26:0/2520469750 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5636f6c0f660 0x5636f6c31330 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f17d8012cb0 tx=0x7f17d801b000 comp rx=0 tx=0).stop 2024-09-11T23:39:18.776 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.772+0000 7f17eda5b740 1 -- 172.21.15.26:0/2520469750 shutdown_connections 2024-09-11T23:39:18.776 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.772+0000 7f17eda5b740 1 --2- 172.21.15.26:0/2520469750 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5636f6c13cc0 0x5636f6c0e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:18.776 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.772+0000 7f17eda5b740 1 --2- 172.21.15.26:0/2520469750 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5636f6c10000 0x5636f6c31870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:18.776 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.772+0000 7f17eda5b740 1 --2- 172.21.15.26:0/2520469750 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5636f6c0f660 0x5636f6c31330 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:18.776 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.772+0000 7f17eda5b740 1 -- 172.21.15.26:0/2520469750 >> 172.21.15.26:0/2520469750 conn(0x5636f6c01b10 msgr2=0x5636f6c17740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:39:18.776 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.772+0000 7f17eda5b740 1 -- 172.21.15.26:0/2520469750 shutdown_connections 2024-09-11T23:39:18.776 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.772+0000 7f17eda5b740 1 -- 172.21.15.26:0/2520469750 wait complete. 2024-09-11T23:39:18.776 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.772+0000 7f17eda5b740 1 -- start start 2024-09-11T23:39:18.776 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.773+0000 7f17eda5b740 1 Processor -- start 2024-09-11T23:39:18.776 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.773+0000 7f17eda5b740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5636f6c0f660 0x5636f6c15f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:18.777 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.773+0000 7f17eda5b740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5636f6c10000 0x5636f6c164c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:18.777 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.773+0000 7f17eda5b740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5636f6c13cc0 0x5636f6c16a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:18.777 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.773+0000 7f17e8980640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5636f6c0f660 0x5636f6c15f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:18.777 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.773+0000 7f17e8980640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5636f6c0f660 0x5636f6c15f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:35542/0 (socket says 172.21.15.26:35542) 2024-09-11T23:39:18.777 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.773+0000 7f17e8980640 1 -- 172.21.15.26:0/2667921144 learned_addr learned my addr 172.21.15.26:0/2667921144 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:39:18.777 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.773+0000 7f17eda5b740 1 -- 172.21.15.26:0/2667921144 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f17e41acb70 con 0x5636f6c13cc0 2024-09-11T23:39:18.777 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.773+0000 7f17eda5b740 1 -- 172.21.15.26:0/2667921144 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f17e41ac3e0 con 0x5636f6c10000 2024-09-11T23:39:18.777 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.773+0000 7f17eda5b740 1 -- 172.21.15.26:0/2667921144 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f17e41a9550 con 0x5636f6c0f660 2024-09-11T23:39:18.777 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.773+0000 7f17e3fff640 1 --2- 172.21.15.26:0/2667921144 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5636f6c10000 0x5636f6c164c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:18.777 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.773+0000 7f17eda5b740 10 client.0 _ll_register_callbacks cb 0x7f17dc0b8210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:39:18.777 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.773+0000 7f17e9181640 1 --2- 172.21.15.26:0/2667921144 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5636f6c13cc0 0x5636f6c16a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:18.777 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.774+0000 7f17e3fff640 1 -- 172.21.15.26:0/2667921144 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5636f6c0f660 msgr2=0x5636f6c15f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:39:18.778 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.774+0000 7f17e3fff640 1 --2- 172.21.15.26:0/2667921144 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5636f6c0f660 0x5636f6c15f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:18.778 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.774+0000 7f17e3fff640 1 -- 172.21.15.26:0/2667921144 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5636f6c13cc0 msgr2=0x5636f6c16a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:39:18.778 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.774+0000 7f17e3fff640 1 --2- 172.21.15.26:0/2667921144 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5636f6c13cc0 0x5636f6c16a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:18.778 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.774+0000 7f17e3fff640 1 -- 172.21.15.26:0/2667921144 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636f6c93460 con 0x5636f6c10000 2024-09-11T23:39:18.778 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.774+0000 7f17e3fff640 1 --2- 172.21.15.26:0/2667921144 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5636f6c10000 0x5636f6c164c0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x5636f6c1a4d0 tx=0x7f17d0009c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=16018b58a6c0e94e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:39:18.778 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.774+0000 7f17c3fff640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-11T23:39:18.779 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.774+0000 7f17c3fff640 1 -- 172.21.15.26:0/2667921144 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17d0011bc0 con 0x5636f6c10000 2024-09-11T23:39:18.779 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.774+0000 7f17eda5b740 1 -- 172.21.15.26:0/2667921144 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f17d802c9a0 con 0x5636f6c10000 2024-09-11T23:39:18.779 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.775+0000 7f17c3fff640 1 -- 172.21.15.26:0/2667921144 <== mon.2 v2:172.21.15.76:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 8586+0+0 (secure 0 0 0) 0x7f17d0011d60 con 0x5636f6c10000 2024-09-11T23:39:18.779 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.775+0000 7f17c3fff640 1 -- 172.21.15.26:0/2667921144 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f17d0005580 con 0x5636f6c10000 2024-09-11T23:39:18.779 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.775+0000 7f17eda5b740 10 client.27590 fetch_fsmap learned FSMap version 91 2024-09-11T23:39:18.779 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.775+0000 7f17eda5b740 1 -- 172.21.15.26:0/2667921144 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=91,monmap=4+}) -- 0x5636f6c946f0 con 0x5636f6c10000 2024-09-11T23:39:18.779 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.775+0000 7f17c3fff640 1 -- 172.21.15.26:0/2667921144 <== mon.2 v2:172.21.15.76:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17d802c9a0 con 0x5636f6c10000 2024-09-11T23:39:18.779 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.775+0000 7f17c3fff640 1 -- 172.21.15.26:0/2667921144 <== mon.2 v2:172.21.15.76:3300/0 5 ==== fsmap(e 91) ==== 8418+0+0 (secure 0 0 0) 0x7f17d0020680 con 0x5636f6c10000 2024-09-11T23:39:18.779 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.775+0000 7f17eda5b740 10 client.27590 fetch_fsmap finished waiting for FSMap version 91 2024-09-11T23:39:18.779 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.776+0000 7f17eda5b740 10 client.27590 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:39:18.780 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.776+0000 7f17eda5b740 20 client.27590 populate_metadata read hostname 'smithi026' 2024-09-11T23:39:18.780 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.776+0000 7f17eda5b740 1 --2- 172.21.15.26:0/2667921144 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5636f6c94ca0 0x5636f6c970c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:18.780 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.776+0000 7f17eda5b740 4 client.27590 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T23:39:18.780 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.776+0000 7f17eda5b740 1 -- 172.21.15.26:0/2667921144 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5636f6b71d70 con 0x5636f6c94ca0 2024-09-11T23:39:18.780 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.778+0000 7f17e8980640 1 --2- 172.21.15.26:0/2667921144 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5636f6c94ca0 0x5636f6c970c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:18.780 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.779+0000 7f17e8980640 1 --2- 172.21.15.26:0/2667921144 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5636f6c94ca0 0x5636f6c970c0 crc :-1 s=READY pgs=313 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=52920e429b858204 server_cookie=49a28acd76049a08 in_seq=0 out_seq=0 2024-09-11T23:39:18.781 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.779+0000 7f17c3fff640 10 client.27590 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:39:18.947 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.946+0000 7f17c3fff640 1 -- 172.21.15.26:0/2667921144 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5636f6b71d70 con 0x5636f6c94ca0 2024-09-11T23:39:18.947 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.946+0000 7f17c3fff640 10 client.27590 handle_command_reply: tid=0 2024-09-11T23:39:18.947 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.946+0000 7f17eda5b740 1 client.27590 shutdown 2024-09-11T23:39:18.947 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.947+0000 7f17eda5b740 1 -- 172.21.15.26:0/2667921144 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5636f6c10000 msgr2=0x5636f6c164c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:18.947 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.947+0000 7f17eda5b740 1 --2- 172.21.15.26:0/2667921144 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5636f6c10000 0x5636f6c164c0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x5636f6c1a4d0 tx=0x7f17d0009c80 comp rx=0 tx=0).stop 2024-09-11T23:39:18.947 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.947+0000 7f17eda5b740 1 -- 172.21.15.26:0/2667921144 shutdown_connections 2024-09-11T23:39:18.947 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.947+0000 7f17eda5b740 1 --2- 172.21.15.26:0/2667921144 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5636f6c10000 0x5636f6c164c0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:18.947 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.947+0000 7f17eda5b740 1 --2- 172.21.15.26:0/2667921144 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5636f6c94ca0 0x5636f6c970c0 crc :-1 s=READY pgs=313 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:18.947 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.947+0000 7f17eda5b740 1 --2- 172.21.15.26:0/2667921144 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5636f6c13cc0 0x5636f6c16a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:18.947 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.947+0000 7f17eda5b740 1 --2- 172.21.15.26:0/2667921144 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5636f6c0f660 0x5636f6c15f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:18.947 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.947+0000 7f17eda5b740 1 -- 172.21.15.26:0/2667921144 >> 172.21.15.26:0/2667921144 conn(0x5636f6c01b10 msgr2=0x5636f6c0e650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:39:18.948 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.947+0000 7f17c3fff640 0 client.27590 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:39:18.948 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.947+0000 7f17eda5b740 1 -- 172.21.15.26:0/2667921144 shutdown_connections 2024-09-11T23:39:18.948 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.947+0000 7f17eda5b740 1 -- 172.21.15.26:0/2667921144 wait complete. 2024-09-11T23:39:18.948 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.947+0000 7f17eda5b740 20 client.27590 trim_cache size 0 max 16384 2024-09-11T23:39:18.948 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-09-11T23:39:18.948 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "message": "", 2024-09-11T23:39:18.949 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-09-11T23:39:18.949 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-09-11T23:39:18.949 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.948+0000 7f17eabe4640 1 -- 172.21.15.26:0/4137825675 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f17d40a2ea0 msgr2=0x7f17d40a5360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:18.950 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.948+0000 7f17eabe4640 1 --2- 172.21.15.26:0/4137825675 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f17d40a2ea0 0x7f17d40a5360 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x5636f6c03260 tx=0x7f17d8005f90 comp rx=0 tx=0).stop 2024-09-11T23:39:18.950 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.948+0000 7f17eabe4640 1 -- 172.21.15.26:0/4137825675 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f17e410c0b0 msgr2=0x7f17e419cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:18.950 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.948+0000 7f17eabe4640 1 --2- 172.21.15.26:0/4137825675 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f17e410c0b0 0x7f17e419cb80 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f17dc005cc0 tx=0x7f17dc02f3a0 comp rx=0 tx=0).stop 2024-09-11T23:39:18.950 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.948+0000 7f17eabe4640 1 -- 172.21.15.26:0/4137825675 shutdown_connections 2024-09-11T23:39:18.950 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.948+0000 7f17eabe4640 1 --2- 172.21.15.26:0/4137825675 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f17d40a2ea0 0x7f17d40a5360 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:18.950 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.948+0000 7f17eabe4640 1 --2- 172.21.15.26:0/4137825675 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f17e410c0b0 0x7f17e419cb80 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:18.950 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.948+0000 7f17eabe4640 1 --2- 172.21.15.26:0/4137825675 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f17e4103600 0x7f17e4198800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:18.950 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.948+0000 7f17eabe4640 1 --2- 172.21.15.26:0/4137825675 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f17e4102ce0 0x7f17e41982c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:18.950 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.948+0000 7f17eabe4640 1 -- 172.21.15.26:0/4137825675 >> 172.21.15.26:0/4137825675 conn(0x7f17e4077990 msgr2=0x7f17e40fcf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:39:18.950 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.948+0000 7f17eabe4640 1 -- 172.21.15.26:0/4137825675 shutdown_connections 2024-09-11T23:39:18.950 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:18.948+0000 7f17eabe4640 1 -- 172.21.15.26:0/4137825675 wait complete. 2024-09-11T23:39:20.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:39:20 smithi136 ceph-mon[35121]: pgmap v660: 97 pgs: 23 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 33 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 147 op/s 2024-09-11T23:39:20.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:39:20 smithi136 ceph-mon[35121]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:39:20.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:39:20 smithi026 ceph-mon[31172]: pgmap v660: 97 pgs: 23 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 33 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 147 op/s 2024-09-11T23:39:20.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:39:20 smithi026 ceph-mon[31172]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:39:20.552 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:39:20 smithi076 ceph-mon[35296]: pgmap v660: 97 pgs: 23 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 33 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 147 op/s 2024-09-11T23:39:20.552 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:39:20 smithi076 ceph-mon[35296]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:39:22.406 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:39:22 smithi026 ceph-mon[31172]: pgmap v661: 97 pgs: 23 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 33 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.0 MiB/s wr, 168 op/s 2024-09-11T23:39:22.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:39:22 smithi136 ceph-mon[35121]: pgmap v661: 97 pgs: 23 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 33 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.0 MiB/s wr, 168 op/s 2024-09-11T23:39:22.552 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:39:22 smithi076 ceph-mon[35296]: pgmap v661: 97 pgs: 23 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 33 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.0 MiB/s wr, 168 op/s 2024-09-11T23:39:23.000 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:39:22 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: 2024-09-11T23:39:22.657+0000 7f7ccf119640 -1 calc_pg_upmaps abort due to max <= 0 2024-09-11T23:39:24.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:39:24 smithi136 ceph-mon[35121]: pgmap v662: 97 pgs: 23 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 33 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.0 MiB/s wr, 169 op/s 2024-09-11T23:39:24.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:39:24 smithi026 ceph-mon[31172]: pgmap v662: 97 pgs: 23 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 33 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.0 MiB/s wr, 169 op/s 2024-09-11T23:39:24.552 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:39:24 smithi076 ceph-mon[35296]: pgmap v662: 97 pgs: 23 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 33 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.0 MiB/s wr, 169 op/s 2024-09-11T23:39:26.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:39:26 smithi136 ceph-mon[35121]: pgmap v663: 97 pgs: 23 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 33 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 141 op/s 2024-09-11T23:39:26.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:39:26 smithi026 ceph-mon[31172]: pgmap v663: 97 pgs: 23 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 33 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 141 op/s 2024-09-11T23:39:26.552 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:39:26 smithi076 ceph-mon[35296]: pgmap v663: 97 pgs: 23 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 33 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 141 op/s 2024-09-11T23:39:28.353 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:39:28 smithi076 ceph-mon[35296]: pgmap v664: 97 pgs: 22 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 34 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.2 MiB/s wr, 199 op/s 2024-09-11T23:39:28.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:39:28 smithi136 ceph-mon[35121]: pgmap v664: 97 pgs: 22 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 34 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.2 MiB/s wr, 199 op/s 2024-09-11T23:39:28.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:39:28 smithi026 ceph-mon[31172]: pgmap v664: 97 pgs: 22 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 34 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.2 MiB/s wr, 199 op/s 2024-09-11T23:39:30.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:39:30 smithi136 ceph-mon[35121]: pgmap v665: 97 pgs: 22 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 34 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 140 op/s 2024-09-11T23:39:30.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:39:30 smithi026 ceph-mon[31172]: pgmap v665: 97 pgs: 22 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 34 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 140 op/s 2024-09-11T23:39:30.552 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:39:30 smithi076 ceph-mon[35296]: pgmap v665: 97 pgs: 22 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 34 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 140 op/s 2024-09-11T23:39:32.097 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.097+0000 7fa131f4b640 1 -- 172.21.15.26:0/990789867 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa12c1013d0 msgr2=0x7fa12c105890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:32.098 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.097+0000 7fa131f4b640 1 --2- 172.21.15.26:0/990789867 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa12c1013d0 0x7fa12c105890 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7fa11c009900 tx=0x7fa11c02a480 comp rx=0 tx=0).stop 2024-09-11T23:39:32.098 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.097+0000 7fa131f4b640 1 -- 172.21.15.26:0/990789867 shutdown_connections 2024-09-11T23:39:32.098 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.097+0000 7fa131f4b640 1 --2- 172.21.15.26:0/990789867 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa12c105ee0 0x7fa12c106350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:32.098 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.097+0000 7fa131f4b640 1 --2- 172.21.15.26:0/990789867 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa12c1013d0 0x7fa12c105890 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:32.098 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.097+0000 7fa131f4b640 1 --2- 172.21.15.26:0/990789867 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa12c100ab0 0x7fa12c100e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:32.098 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.097+0000 7fa131f4b640 1 -- 172.21.15.26:0/990789867 >> 172.21.15.26:0/990789867 conn(0x7fa12c0fc6b0 msgr2=0x7fa12c0fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:39:32.098 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.097+0000 7fa131f4b640 1 -- 172.21.15.26:0/990789867 shutdown_connections 2024-09-11T23:39:32.098 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.097+0000 7fa131f4b640 1 -- 172.21.15.26:0/990789867 wait complete. 2024-09-11T23:39:32.098 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.098+0000 7fa131f4b640 1 Processor -- start 2024-09-11T23:39:32.098 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.098+0000 7fa131f4b640 1 -- start start 2024-09-11T23:39:32.100 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.099+0000 7fa131f4b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa12c100ab0 0x7fa12c1a4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:32.101 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.099+0000 7fa131f4b640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa12c1013d0 0x7fa12c1a4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:32.101 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.099+0000 7fa131f4b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa12c105ee0 0x7fa12c19e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:32.101 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.099+0000 7fa131f4b640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fa12c118530 con 0x7fa12c105ee0 2024-09-11T23:39:32.101 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.099+0000 7fa131f4b640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fa12c1183b0 con 0x7fa12c100ab0 2024-09-11T23:39:32.101 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.099+0000 7fa131f4b640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fa12c1186b0 con 0x7fa12c1013d0 2024-09-11T23:39:32.101 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.099+0000 7fa12b7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa12c100ab0 0x7fa12c1a4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:32.101 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.099+0000 7fa12affd640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa12c1013d0 0x7fa12c1a4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:32.101 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.099+0000 7fa12b7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa12c100ab0 0x7fa12c1a4690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:37716/0 (socket says 172.21.15.26:37716) 2024-09-11T23:39:32.101 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.099+0000 7fa12affd640 1 --2- 172.21.15.26:0/216803813 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa12c1013d0 0x7fa12c1a4bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:38282/0 (socket says 172.21.15.26:38282) 2024-09-11T23:39:32.101 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.099+0000 7fa12b7fe640 1 -- 172.21.15.26:0/216803813 learned_addr learned my addr 172.21.15.26:0/216803813 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:39:32.101 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.099+0000 7fa12bfff640 1 --2- 172.21.15.26:0/216803813 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa12c105ee0 0x7fa12c19e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:32.101 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.099+0000 7fa12affd640 1 -- 172.21.15.26:0/216803813 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa12c100ab0 msgr2=0x7fa12c1a4690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:32.101 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.099+0000 7fa12affd640 1 --2- 172.21.15.26:0/216803813 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa12c100ab0 0x7fa12c1a4690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:32.102 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.099+0000 7fa12affd640 1 -- 172.21.15.26:0/216803813 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa12c105ee0 msgr2=0x7fa12c19e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:32.102 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.099+0000 7fa12affd640 1 --2- 172.21.15.26:0/216803813 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa12c105ee0 0x7fa12c19e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:32.102 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.099+0000 7fa12affd640 1 -- 172.21.15.26:0/216803813 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa12c19efd0 con 0x7fa12c1013d0 2024-09-11T23:39:32.102 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.100+0000 7fa12bfff640 1 --2- 172.21.15.26:0/216803813 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa12c105ee0 0x7fa12c19e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:39:32.102 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.100+0000 7fa12affd640 1 --2- 172.21.15.26:0/216803813 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa12c1013d0 0x7fa12c1a4bd0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fa11c006010 tx=0x7fa11c0042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:39:32.102 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.100+0000 7fa128ff9640 1 -- 172.21.15.26:0/216803813 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa11c03f030 con 0x7fa12c1013d0 2024-09-11T23:39:32.102 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.100+0000 7fa131f4b640 1 -- 172.21.15.26:0/216803813 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa12c19f260 con 0x7fa12c1013d0 2024-09-11T23:39:32.102 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.100+0000 7fa128ff9640 1 -- 172.21.15.26:0/216803813 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa11c003ce0 con 0x7fa12c1013d0 2024-09-11T23:39:32.102 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.100+0000 7fa131f4b640 1 -- 172.21.15.26:0/216803813 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa12c1ab4e0 con 0x7fa12c1013d0 2024-09-11T23:39:32.102 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.100+0000 7fa128ff9640 1 -- 172.21.15.26:0/216803813 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa11c049060 con 0x7fa12c1013d0 2024-09-11T23:39:32.104 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.103+0000 7fa134dc1740 1 Processor -- start 2024-09-11T23:39:32.104 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.103+0000 7fa134dc1740 1 -- start start 2024-09-11T23:39:32.104 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.103+0000 7fa134dc1740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5653ff641af0 0x5653ff641ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:32.104 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.103+0000 7fa134dc1740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5653ff642410 0x5653ff64c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:32.104 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.103+0000 7fa128ff9640 1 -- 172.21.15.26:0/216803813 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fa11c042030 con 0x7fa12c1013d0 2024-09-11T23:39:32.104 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.103+0000 7fa134dc1740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5653ff64ccb0 0x5653ff64f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:32.104 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.103+0000 7fa134dc1740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5653ff5a62f0 con 0x5653ff64ccb0 2024-09-11T23:39:32.104 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.103+0000 7fa134dc1740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5653ff54e270 con 0x5653ff642410 2024-09-11T23:39:32.105 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.103+0000 7fa134dc1740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5653ff5a22b0 con 0x5653ff641af0 2024-09-11T23:39:32.105 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.104+0000 7fa12bfff640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5653ff641af0 0x5653ff641ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:32.105 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.104+0000 7fa12bfff640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5653ff641af0 0x5653ff641ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:38288/0 (socket says 172.21.15.26:38288) 2024-09-11T23:39:32.105 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.104+0000 7fa12bfff640 1 -- 172.21.15.26:0/1569037197 learned_addr learned my addr 172.21.15.26:0/1569037197 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:39:32.105 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.104+0000 7fa12affd640 1 --2- 172.21.15.26:0/1569037197 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5653ff64ccb0 0x5653ff64f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:32.105 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.104+0000 7fa12b7fe640 1 --2- 172.21.15.26:0/1569037197 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5653ff642410 0x5653ff64c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:32.105 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.104+0000 7fa12bfff640 1 -- 172.21.15.26:0/1569037197 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5653ff642410 msgr2=0x5653ff64c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:39:32.105 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.104+0000 7fa12bfff640 1 --2- 172.21.15.26:0/1569037197 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5653ff642410 0x5653ff64c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:32.105 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.104+0000 7fa12bfff640 1 -- 172.21.15.26:0/1569037197 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5653ff64ccb0 msgr2=0x5653ff64f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:39:32.105 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.104+0000 7fa12bfff640 1 --2- 172.21.15.26:0/1569037197 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5653ff64ccb0 0x5653ff64f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:32.105 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.104+0000 7fa12bfff640 1 -- 172.21.15.26:0/1569037197 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5653ff54d340 con 0x5653ff641af0 2024-09-11T23:39:32.106 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.104+0000 7fa12bfff640 1 --2- 172.21.15.26:0/1569037197 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5653ff641af0 0x5653ff641ed0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fa12400c450 tx=0x7fa12400c910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=964b53267d552256 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:39:32.107 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.104+0000 7fa128ff9640 1 --2- 172.21.15.26:0/216803813 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fa1140a2da0 0x7fa1140a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:32.107 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.105+0000 7fa128ff9640 1 -- 172.21.15.26:0/216803813 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 8586+0+0 (secure 0 0 0) 0x7fa11c0ef6d0 con 0x7fa12c1013d0 2024-09-11T23:39:32.107 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.105+0000 7fa12b7fe640 1 --2- 172.21.15.26:0/216803813 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fa1140a2da0 0x7fa1140a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:32.107 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.105+0000 7fa12b7fe640 1 --2- 172.21.15.26:0/216803813 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fa1140a2da0 0x7fa1140a5260 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x5653ff63eec0 tx=0x7fa1200084c0 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:39:32.107 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.105+0000 7fa111ffb640 1 -- 172.21.15.26:0/1569037197 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa124007430 con 0x5653ff641af0 2024-09-11T23:39:32.107 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.105+0000 7fa111ffb640 1 -- 172.21.15.26:0/1569037197 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa124007ab0 con 0x5653ff641af0 2024-09-11T23:39:32.107 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.106+0000 7fa111ffb640 1 -- 172.21.15.26:0/1569037197 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa12400ab40 con 0x5653ff641af0 2024-09-11T23:39:32.107 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.106+0000 7fa134dc1740 1 -- 172.21.15.26:0/1569037197 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5653ff641af0 msgr2=0x5653ff641ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:32.107 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.106+0000 7fa134dc1740 1 --2- 172.21.15.26:0/1569037197 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5653ff641af0 0x5653ff641ed0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fa12400c450 tx=0x7fa12400c910 comp rx=0 tx=0).stop 2024-09-11T23:39:32.107 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.107+0000 7fa134dc1740 1 -- 172.21.15.26:0/1569037197 shutdown_connections 2024-09-11T23:39:32.107 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.107+0000 7fa134dc1740 1 --2- 172.21.15.26:0/1569037197 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5653ff642410 0x5653ff64c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:32.107 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.107+0000 7fa134dc1740 1 --2- 172.21.15.26:0/1569037197 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5653ff64ccb0 0x5653ff64f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:32.108 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.107+0000 7fa134dc1740 1 --2- 172.21.15.26:0/1569037197 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5653ff641af0 0x5653ff641ed0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:32.108 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.107+0000 7fa134dc1740 1 -- 172.21.15.26:0/1569037197 >> 172.21.15.26:0/1569037197 conn(0x5653ff63e060 msgr2=0x5653ff63e440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:39:32.108 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.107+0000 7fa134dc1740 1 -- 172.21.15.26:0/1569037197 shutdown_connections 2024-09-11T23:39:32.108 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.107+0000 7fa134dc1740 1 -- 172.21.15.26:0/1569037197 wait complete. 2024-09-11T23:39:32.108 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.107+0000 7fa134dc1740 1 -- start start 2024-09-11T23:39:32.110 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.108+0000 7fa134dc1740 1 Processor -- start 2024-09-11T23:39:32.110 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.108+0000 7fa134dc1740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5653ff641af0 0x5653ff649ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:32.110 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.108+0000 7fa134dc1740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5653ff642410 0x5653ff649ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:32.110 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.109+0000 7fa134dc1740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5653ff64ccb0 0x5653ff64a530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:32.110 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.109+0000 7fa134dc1740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5653ff64f920 con 0x5653ff641af0 2024-09-11T23:39:32.110 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.109+0000 7fa12b7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5653ff641af0 0x5653ff649ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:32.110 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.109+0000 7fa12b7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5653ff641af0 0x5653ff649ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60260/0 (socket says 172.21.15.26:60260) 2024-09-11T23:39:32.110 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.109+0000 7fa12b7fe640 1 -- 172.21.15.26:0/4192548724 learned_addr learned my addr 172.21.15.26:0/4192548724 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:39:32.110 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.109+0000 7fa12affd640 1 --2- 172.21.15.26:0/4192548724 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5653ff642410 0x5653ff649ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:32.111 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.109+0000 7fa12bfff640 1 --2- 172.21.15.26:0/4192548724 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5653ff64ccb0 0x5653ff64a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:32.111 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.109+0000 7fa134dc1740 1 -- 172.21.15.26:0/4192548724 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5653ff52f290 con 0x5653ff64ccb0 2024-09-11T23:39:32.111 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.109+0000 7fa134dc1740 1 -- 172.21.15.26:0/4192548724 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5653ff64faa0 con 0x5653ff642410 2024-09-11T23:39:32.111 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.109+0000 7fa134dc1740 10 client.0 _ll_register_callbacks cb 0x5653ff560140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:39:32.111 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.109+0000 7fa12b7fe640 1 -- 172.21.15.26:0/4192548724 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5653ff642410 msgr2=0x5653ff649ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:39:32.111 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.109+0000 7fa12b7fe640 1 --2- 172.21.15.26:0/4192548724 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5653ff642410 0x5653ff649ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:32.111 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.109+0000 7fa12b7fe640 1 -- 172.21.15.26:0/4192548724 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5653ff64ccb0 msgr2=0x5653ff64a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:39:32.111 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.109+0000 7fa12b7fe640 1 --2- 172.21.15.26:0/4192548724 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5653ff64ccb0 0x5653ff64a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:32.111 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.109+0000 7fa12b7fe640 1 -- 172.21.15.26:0/4192548724 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5653ff2b9210 con 0x5653ff641af0 2024-09-11T23:39:32.111 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.109+0000 7fa12affd640 1 --2- 172.21.15.26:0/4192548724 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5653ff642410 0x5653ff649ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:39:32.111 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.109+0000 7fa12bfff640 1 --2- 172.21.15.26:0/4192548724 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5653ff64ccb0 0x5653ff64a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:39:32.111 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.110+0000 7fa12b7fe640 1 --2- 172.21.15.26:0/4192548724 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5653ff641af0 0x5653ff649ab0 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7fa120003b70 tx=0x7fa1200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62910b95b6dc8710 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:39:32.111 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.110+0000 7fa10affd640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:39:32.112 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.110+0000 7fa10affd640 1 -- 172.21.15.26:0/4192548724 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa12000ecc0 con 0x5653ff641af0 2024-09-11T23:39:32.112 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.110+0000 7fa10affd640 1 -- 172.21.15.26:0/4192548724 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 8586+0+0 (secure 0 0 0) 0x7fa120005020 con 0x5653ff641af0 2024-09-11T23:39:32.112 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.110+0000 7fa134dc1740 1 -- 172.21.15.26:0/4192548724 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa124007ab0 con 0x5653ff641af0 2024-09-11T23:39:32.113 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.111+0000 7fa10affd640 1 -- 172.21.15.26:0/4192548724 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7fa12000ee60 con 0x5653ff641af0 2024-09-11T23:39:32.113 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.111+0000 7fa134dc1740 10 client.18114 fetch_fsmap learned FSMap version 92 2024-09-11T23:39:32.113 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.111+0000 7fa134dc1740 1 -- 172.21.15.26:0/4192548724 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=92,monmap=4+}) -- 0x5653ff385620 con 0x5653ff641af0 2024-09-11T23:39:32.113 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.111+0000 7fa10affd640 1 -- 172.21.15.26:0/4192548724 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa124007ab0 con 0x5653ff641af0 2024-09-11T23:39:32.113 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.112+0000 7fa10affd640 1 -- 172.21.15.26:0/4192548724 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 92) ==== 8430+0+0 (secure 0 0 0) 0x7fa12001e420 con 0x5653ff641af0 2024-09-11T23:39:32.113 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.112+0000 7fa134dc1740 10 client.18114 fetch_fsmap finished waiting for FSMap version 92 2024-09-11T23:39:32.113 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.112+0000 7fa134dc1740 10 client.18114 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:39:32.113 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.112+0000 7fa134dc1740 20 client.18114 populate_metadata read hostname 'smithi026' 2024-09-11T23:39:32.113 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.112+0000 7fa134dc1740 1 --2- 172.21.15.26:0/4192548724 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5653ff65cc70 0x5653ff65f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:32.114 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.112+0000 7fa134dc1740 4 client.18114 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:39:32.114 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.112+0000 7fa134dc1740 1 -- 172.21.15.26:0/4192548724 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5653ff5b3a20 con 0x5653ff65cc70 2024-09-11T23:39:32.114 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.114+0000 7fa12affd640 1 --2- 172.21.15.26:0/4192548724 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5653ff65cc70 0x5653ff65f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:32.114 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.114+0000 7fa12affd640 1 --2- 172.21.15.26:0/4192548724 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5653ff65cc70 0x5653ff65f100 crc :-1 s=READY pgs=314 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a69960a224cb8ce8 server_cookie=e20325ca7143045b in_seq=0 out_seq=0 2024-09-11T23:39:32.114 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.115+0000 7fa10affd640 10 client.18114 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:39:32.116 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.115+0000 7fa10affd640 1 -- 172.21.15.26:0/4192548724 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5653ff5b3a20 con 0x5653ff65cc70 2024-09-11T23:39:32.116 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.115+0000 7fa10affd640 10 client.18114 handle_command_reply: tid=0 2024-09-11T23:39:32.116 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.115+0000 7fa134dc1740 1 client.18114 shutdown 2024-09-11T23:39:32.116 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.116+0000 7fa134dc1740 1 -- 172.21.15.26:0/4192548724 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5653ff641af0 msgr2=0x5653ff649ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:32.116 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.116+0000 7fa134dc1740 1 --2- 172.21.15.26:0/4192548724 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5653ff641af0 0x5653ff649ab0 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7fa120003b70 tx=0x7fa1200046c0 comp rx=0 tx=0).stop 2024-09-11T23:39:32.116 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.116+0000 7fa134dc1740 1 -- 172.21.15.26:0/4192548724 shutdown_connections 2024-09-11T23:39:32.116 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.116+0000 7fa134dc1740 1 --2- 172.21.15.26:0/4192548724 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5653ff64ccb0 0x5653ff64a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:32.116 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.116+0000 7fa134dc1740 1 --2- 172.21.15.26:0/4192548724 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5653ff65cc70 0x5653ff65f100 crc :-1 s=READY pgs=314 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:32.117 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.116+0000 7fa134dc1740 1 --2- 172.21.15.26:0/4192548724 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5653ff642410 0x5653ff649ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:32.117 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.116+0000 7fa134dc1740 1 --2- 172.21.15.26:0/4192548724 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5653ff641af0 0x5653ff649ab0 secure :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7fa120003b70 tx=0x7fa1200046c0 comp rx=0 tx=0).stop 2024-09-11T23:39:32.117 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.116+0000 7fa10affd640 0 client.18114 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:39:32.117 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.116+0000 7fa134dc1740 1 -- 172.21.15.26:0/4192548724 >> 172.21.15.26:0/4192548724 conn(0x5653ff63e060 msgr2=0x5653ff643d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:39:32.117 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.116+0000 7fa134dc1740 1 -- 172.21.15.26:0/4192548724 shutdown_connections 2024-09-11T23:39:32.117 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.117+0000 7fa134dc1740 1 -- 172.21.15.26:0/4192548724 wait complete. 2024-09-11T23:39:32.117 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.117+0000 7fa134dc1740 20 client.18114 trim_cache size 0 max 16384 2024-09-11T23:39:32.141 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.142+0000 7fa134dc1740 1 Processor -- start 2024-09-11T23:39:32.141 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.142+0000 7fa134dc1740 1 -- start start 2024-09-11T23:39:32.143 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.142+0000 7fa134dc1740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5653ff642410 0x5653ff644560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:32.143 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.142+0000 7fa134dc1740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5653ff64ccb0 0x5653ff64b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:32.143 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.143+0000 7fa134dc1740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5653ff65cc70 0x5653ff64a330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:32.143 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.143+0000 7fa12b7fe640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5653ff642410 0x5653ff644560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:32.143 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.143+0000 7fa12affd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5653ff64ccb0 0x5653ff64b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:32.143 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.143+0000 7fa12affd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5653ff64ccb0 0x5653ff64b310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:37744/0 (socket says 172.21.15.26:37744) 2024-09-11T23:39:32.143 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.143+0000 7fa12affd640 1 -- 172.21.15.26:0/273070119 learned_addr learned my addr 172.21.15.26:0/273070119 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:39:32.143 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.143+0000 7fa134dc1740 1 -- 172.21.15.26:0/273070119 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fa12c1b3ff0 con 0x5653ff65cc70 2024-09-11T23:39:32.143 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.143+0000 7fa134dc1740 1 -- 172.21.15.26:0/273070119 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fa12c1b5da0 con 0x5653ff64ccb0 2024-09-11T23:39:32.143 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.143+0000 7fa134dc1740 1 -- 172.21.15.26:0/273070119 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fa12c1b46f0 con 0x5653ff642410 2024-09-11T23:39:32.143 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.143+0000 7fa12bfff640 1 --2- 172.21.15.26:0/273070119 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5653ff65cc70 0x5653ff64a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:32.144 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.143+0000 7fa12bfff640 1 -- 172.21.15.26:0/273070119 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5653ff642410 msgr2=0x5653ff644560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:39:32.144 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.143+0000 7fa12bfff640 1 --2- 172.21.15.26:0/273070119 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5653ff642410 0x5653ff644560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:32.144 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.143+0000 7fa12bfff640 1 -- 172.21.15.26:0/273070119 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5653ff64ccb0 msgr2=0x5653ff64b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:39:32.144 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.143+0000 7fa12bfff640 1 --2- 172.21.15.26:0/273070119 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5653ff64ccb0 0x5653ff64b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:32.144 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.143+0000 7fa12bfff640 1 -- 172.21.15.26:0/273070119 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5653ff63de90 con 0x5653ff65cc70 2024-09-11T23:39:32.146 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.144+0000 7fa12bfff640 1 --2- 172.21.15.26:0/273070119 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5653ff65cc70 0x5653ff64a330 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7fa124004b90 tx=0x7fa12400a720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcf1a05682e80fa0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:39:32.146 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.144+0000 7fa1117fa640 1 -- 172.21.15.26:0/273070119 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa124022020 con 0x5653ff65cc70 2024-09-11T23:39:32.146 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.144+0000 7fa1117fa640 1 -- 172.21.15.26:0/273070119 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa12400a400 con 0x5653ff65cc70 2024-09-11T23:39:32.146 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.144+0000 7fa1117fa640 1 -- 172.21.15.26:0/273070119 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa12400a790 con 0x5653ff65cc70 2024-09-11T23:39:32.146 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.145+0000 7fa134dc1740 1 -- 172.21.15.26:0/273070119 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5653ff65cc70 msgr2=0x5653ff64a330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:32.146 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.145+0000 7fa134dc1740 1 --2- 172.21.15.26:0/273070119 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5653ff65cc70 0x5653ff64a330 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7fa124004b90 tx=0x7fa12400a720 comp rx=0 tx=0).stop 2024-09-11T23:39:32.146 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.145+0000 7fa134dc1740 1 -- 172.21.15.26:0/273070119 shutdown_connections 2024-09-11T23:39:32.147 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.145+0000 7fa134dc1740 1 --2- 172.21.15.26:0/273070119 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5653ff64ccb0 0x5653ff64b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:32.147 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.145+0000 7fa134dc1740 1 --2- 172.21.15.26:0/273070119 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5653ff65cc70 0x5653ff64a330 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:32.147 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.145+0000 7fa134dc1740 1 --2- 172.21.15.26:0/273070119 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5653ff642410 0x5653ff644560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:32.147 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.146+0000 7fa134dc1740 1 -- 172.21.15.26:0/273070119 >> 172.21.15.26:0/273070119 conn(0x5653ff63e060 msgr2=0x5653ff648a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:39:32.147 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.146+0000 7fa134dc1740 1 -- 172.21.15.26:0/273070119 shutdown_connections 2024-09-11T23:39:32.147 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.146+0000 7fa134dc1740 1 -- 172.21.15.26:0/273070119 wait complete. 2024-09-11T23:39:32.147 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.146+0000 7fa134dc1740 1 -- start start 2024-09-11T23:39:32.147 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.147+0000 7fa134dc1740 1 Processor -- start 2024-09-11T23:39:32.149 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.147+0000 7fa134dc1740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5653ff642410 0x5653ff654f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:32.149 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.147+0000 7fa134dc1740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5653ff64ccb0 0x5653ff6554b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:32.149 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.147+0000 7fa134dc1740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5653ff65cc70 0x5653ff646310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:32.149 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.147+0000 7fa134dc1740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fa12c1b5610 con 0x5653ff64ccb0 2024-09-11T23:39:32.149 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.147+0000 7fa134dc1740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fa12c1b4e80 con 0x5653ff65cc70 2024-09-11T23:39:32.149 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.147+0000 7fa134dc1740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fa12c1b1ff0 con 0x5653ff642410 2024-09-11T23:39:32.150 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.147+0000 7fa134dc1740 10 client.0 _ll_register_callbacks cb 0x7fa120008a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:39:32.150 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.148+0000 7fa12b7fe640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5653ff642410 0x5653ff654f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:32.150 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.148+0000 7fa12b7fe640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5653ff642410 0x5653ff654f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:38312/0 (socket says 172.21.15.26:38312) 2024-09-11T23:39:32.150 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.148+0000 7fa12b7fe640 1 -- 172.21.15.26:0/3683362098 learned_addr learned my addr 172.21.15.26:0/3683362098 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:39:32.150 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.148+0000 7fa12b7fe640 1 -- 172.21.15.26:0/3683362098 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5653ff65cc70 msgr2=0x5653ff646310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:39:32.150 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.148+0000 7fa12b7fe640 1 --2- 172.21.15.26:0/3683362098 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5653ff65cc70 0x5653ff646310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:32.150 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.148+0000 7fa12b7fe640 1 -- 172.21.15.26:0/3683362098 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5653ff64ccb0 msgr2=0x5653ff6554b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:39:32.150 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.148+0000 7fa12b7fe640 1 --2- 172.21.15.26:0/3683362098 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5653ff64ccb0 0x5653ff6554b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:32.150 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.148+0000 7fa12b7fe640 1 -- 172.21.15.26:0/3683362098 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5653ff646b40 con 0x5653ff642410 2024-09-11T23:39:32.150 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.149+0000 7fa12b7fe640 1 --2- 172.21.15.26:0/3683362098 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5653ff642410 0x5653ff654f70 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fa120016040 tx=0x7fa12001e630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4153a85131516405 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:39:32.150 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.149+0000 7fa10affd640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-09-11T23:39:32.150 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.149+0000 7fa10affd640 1 -- 172.21.15.26:0/3683362098 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa12003c030 con 0x5653ff642410 2024-09-11T23:39:32.150 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.149+0000 7fa134dc1740 1 -- 172.21.15.26:0/3683362098 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa12400a400 con 0x5653ff642410 2024-09-11T23:39:32.151 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.149+0000 7fa10affd640 1 -- 172.21.15.26:0/3683362098 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 8586+0+0 (secure 0 0 0) 0x7fa12001ecc0 con 0x5653ff642410 2024-09-11T23:39:32.151 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.150+0000 7fa10affd640 1 -- 172.21.15.26:0/3683362098 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7fa120032040 con 0x5653ff642410 2024-09-11T23:39:32.151 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.150+0000 7fa134dc1740 10 client.27577 fetch_fsmap learned FSMap version 92 2024-09-11T23:39:32.151 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.150+0000 7fa134dc1740 1 -- 172.21.15.26:0/3683362098 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=92,monmap=4+}) -- 0x5653ff6c7430 con 0x5653ff642410 2024-09-11T23:39:32.151 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.150+0000 7fa10affd640 1 -- 172.21.15.26:0/3683362098 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa120033040 con 0x5653ff642410 2024-09-11T23:39:32.152 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.151+0000 7fa10affd640 1 -- 172.21.15.26:0/3683362098 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 92) ==== 8430+0+0 (secure 0 0 0) 0x7fa120004730 con 0x5653ff642410 2024-09-11T23:39:32.152 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.151+0000 7fa134dc1740 10 client.27577 fetch_fsmap finished waiting for FSMap version 92 2024-09-11T23:39:32.152 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.151+0000 7fa134dc1740 10 client.27577 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:39:32.152 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.151+0000 7fa134dc1740 20 client.27577 populate_metadata read hostname 'smithi026' 2024-09-11T23:39:32.152 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.151+0000 7fa134dc1740 1 --2- 172.21.15.26:0/3683362098 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5653ff6c7bb0 0x5653ff6ca020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:32.152 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.151+0000 7fa134dc1740 4 client.27577 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T23:39:32.152 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.151+0000 7fa134dc1740 1 -- 172.21.15.26:0/3683362098 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7fa12400a400 con 0x5653ff6c7bb0 2024-09-11T23:39:32.153 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.151+0000 7fa12affd640 1 --2- 172.21.15.26:0/3683362098 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5653ff6c7bb0 0x5653ff6ca020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:32.153 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.152+0000 7fa12affd640 1 --2- 172.21.15.26:0/3683362098 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5653ff6c7bb0 0x5653ff6ca020 crc :-1 s=READY pgs=315 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d47f56c01636bdf5 server_cookie=f42da4ec709a7e77 in_seq=0 out_seq=0 2024-09-11T23:39:32.153 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.152+0000 7fa10affd640 10 client.27577 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:39:32.231 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.230+0000 7fa10affd640 1 -- 172.21.15.26:0/3683362098 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7fa12400a400 con 0x5653ff6c7bb0 2024-09-11T23:39:32.231 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.230+0000 7fa10affd640 10 client.27577 handle_command_reply: tid=0 2024-09-11T23:39:32.231 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.230+0000 7fa134dc1740 1 client.27577 shutdown 2024-09-11T23:39:32.231 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.231+0000 7fa134dc1740 1 -- 172.21.15.26:0/3683362098 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5653ff642410 msgr2=0x5653ff654f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:32.232 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.231+0000 7fa134dc1740 1 --2- 172.21.15.26:0/3683362098 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5653ff642410 0x5653ff654f70 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fa120016040 tx=0x7fa12001e630 comp rx=0 tx=0).stop 2024-09-11T23:39:32.232 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.231+0000 7fa134dc1740 1 -- 172.21.15.26:0/3683362098 shutdown_connections 2024-09-11T23:39:32.232 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.231+0000 7fa134dc1740 1 --2- 172.21.15.26:0/3683362098 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5653ff65cc70 0x5653ff646310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:32.232 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.231+0000 7fa134dc1740 1 --2- 172.21.15.26:0/3683362098 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x5653ff6c7bb0 0x5653ff6ca020 crc :-1 s=READY pgs=315 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:32.232 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.231+0000 7fa10affd640 0 client.27577 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:39:32.232 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.231+0000 7fa134dc1740 1 --2- 172.21.15.26:0/3683362098 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5653ff64ccb0 0x5653ff6554b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:32.232 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.231+0000 7fa134dc1740 1 --2- 172.21.15.26:0/3683362098 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5653ff642410 0x5653ff654f70 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:32.232 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.231+0000 7fa134dc1740 1 -- 172.21.15.26:0/3683362098 >> 172.21.15.26:0/3683362098 conn(0x5653ff63e060 msgr2=0x5653ff63c9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:39:32.232 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.231+0000 7fa134dc1740 1 -- 172.21.15.26:0/3683362098 shutdown_connections 2024-09-11T23:39:32.233 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.232+0000 7fa134dc1740 1 -- 172.21.15.26:0/3683362098 wait complete. 2024-09-11T23:39:32.233 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.232+0000 7fa134dc1740 20 client.27577 trim_cache size 0 max 16384 2024-09-11T23:39:32.233 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-09-11T23:39:32.233 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "message": "", 2024-09-11T23:39:32.233 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-09-11T23:39:32.233 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-09-11T23:39:32.234 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.233+0000 7fa131f4b640 1 -- 172.21.15.26:0/216803813 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fa1140a2da0 msgr2=0x7fa1140a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:32.234 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.233+0000 7fa131f4b640 1 --2- 172.21.15.26:0/216803813 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fa1140a2da0 0x7fa1140a5260 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x5653ff63eec0 tx=0x7fa1200084c0 comp rx=0 tx=0).stop 2024-09-11T23:39:32.234 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.233+0000 7fa131f4b640 1 -- 172.21.15.26:0/216803813 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa12c1013d0 msgr2=0x7fa12c1a4bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:32.234 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.233+0000 7fa131f4b640 1 --2- 172.21.15.26:0/216803813 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa12c1013d0 0x7fa12c1a4bd0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fa11c006010 tx=0x7fa11c0042f0 comp rx=0 tx=0).stop 2024-09-11T23:39:32.234 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.233+0000 7fa131f4b640 1 -- 172.21.15.26:0/216803813 shutdown_connections 2024-09-11T23:39:32.234 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.233+0000 7fa131f4b640 1 --2- 172.21.15.26:0/216803813 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fa1140a2da0 0x7fa1140a5260 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:32.235 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.233+0000 7fa131f4b640 1 --2- 172.21.15.26:0/216803813 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa12c105ee0 0x7fa12c19e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:32.235 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.233+0000 7fa131f4b640 1 --2- 172.21.15.26:0/216803813 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa12c1013d0 0x7fa12c1a4bd0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:32.235 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.233+0000 7fa131f4b640 1 --2- 172.21.15.26:0/216803813 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa12c100ab0 0x7fa12c1a4690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:32.235 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.233+0000 7fa131f4b640 1 -- 172.21.15.26:0/216803813 >> 172.21.15.26:0/216803813 conn(0x7fa12c0fc6b0 msgr2=0x7fa12c103860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:39:32.235 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.233+0000 7fa131f4b640 1 -- 172.21.15.26:0/216803813 shutdown_connections 2024-09-11T23:39:32.235 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:32.233+0000 7fa131f4b640 1 -- 172.21.15.26:0/216803813 wait complete. 2024-09-11T23:39:32.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:39:32 smithi136 ceph-mon[35121]: pgmap v666: 97 pgs: 22 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 34 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.3 MiB/s wr, 162 op/s 2024-09-11T23:39:32.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:39:32 smithi026 ceph-mon[31172]: pgmap v666: 97 pgs: 22 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 34 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.3 MiB/s wr, 162 op/s 2024-09-11T23:39:32.552 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:39:32 smithi076 ceph-mon[35296]: pgmap v666: 97 pgs: 22 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 34 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.3 MiB/s wr, 162 op/s 2024-09-11T23:39:34.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:39:34 smithi136 ceph-mon[35121]: pgmap v667: 97 pgs: 20 active+clean+snaptrim, 37 active+clean+snaptrim_wait, 40 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.0 MiB/s wr, 171 op/s 2024-09-11T23:39:34.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:39:34 smithi026 ceph-mon[31172]: pgmap v667: 97 pgs: 20 active+clean+snaptrim, 37 active+clean+snaptrim_wait, 40 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.0 MiB/s wr, 171 op/s 2024-09-11T23:39:34.552 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:39:34 smithi076 ceph-mon[35296]: pgmap v667: 97 pgs: 20 active+clean+snaptrim, 37 active+clean+snaptrim_wait, 40 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.0 MiB/s wr, 171 op/s 2024-09-11T23:39:36.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:39:36 smithi136 ceph-mon[35121]: pgmap v668: 97 pgs: 20 active+clean+snaptrim, 37 active+clean+snaptrim_wait, 40 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 146 op/s 2024-09-11T23:39:36.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:39:36 smithi026 ceph-mon[31172]: pgmap v668: 97 pgs: 20 active+clean+snaptrim, 37 active+clean+snaptrim_wait, 40 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 146 op/s 2024-09-11T23:39:36.552 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:39:36 smithi076 ceph-mon[35296]: pgmap v668: 97 pgs: 20 active+clean+snaptrim, 37 active+clean+snaptrim_wait, 40 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 146 op/s 2024-09-11T23:39:38.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:39:38 smithi136 ceph-mon[35121]: pgmap v669: 97 pgs: 15 active+clean+snaptrim, 33 active+clean+snaptrim_wait, 49 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.2 MiB/s wr, 211 op/s 2024-09-11T23:39:38.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:39:38 smithi026 ceph-mon[31172]: pgmap v669: 97 pgs: 15 active+clean+snaptrim, 33 active+clean+snaptrim_wait, 49 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.2 MiB/s wr, 211 op/s 2024-09-11T23:39:38.552 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:39:38 smithi076 ceph-mon[35296]: pgmap v669: 97 pgs: 15 active+clean+snaptrim, 33 active+clean+snaptrim_wait, 49 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.2 MiB/s wr, 211 op/s 2024-09-11T23:39:40.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:39:40 smithi136 ceph-mon[35121]: pgmap v670: 97 pgs: 15 active+clean+snaptrim, 33 active+clean+snaptrim_wait, 49 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 162 op/s 2024-09-11T23:39:40.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:39:40 smithi026 ceph-mon[31172]: pgmap v670: 97 pgs: 15 active+clean+snaptrim, 33 active+clean+snaptrim_wait, 49 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 162 op/s 2024-09-11T23:39:40.552 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:39:40 smithi076 ceph-mon[35296]: pgmap v670: 97 pgs: 15 active+clean+snaptrim, 33 active+clean+snaptrim_wait, 49 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 162 op/s 2024-09-11T23:39:42.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:39:42 smithi136 ceph-mon[35121]: pgmap v671: 97 pgs: 14 active+clean+snaptrim, 33 active+clean+snaptrim_wait, 50 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 189 op/s 2024-09-11T23:39:42.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:39:42 smithi026 ceph-mon[31172]: pgmap v671: 97 pgs: 14 active+clean+snaptrim, 33 active+clean+snaptrim_wait, 50 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 189 op/s 2024-09-11T23:39:42.552 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:39:42 smithi076 ceph-mon[35296]: pgmap v671: 97 pgs: 14 active+clean+snaptrim, 33 active+clean+snaptrim_wait, 50 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 189 op/s 2024-09-11T23:39:44.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:39:44 smithi136 ceph-mon[35121]: pgmap v672: 97 pgs: 13 active+clean+snaptrim, 30 active+clean+snaptrim_wait, 54 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 203 op/s 2024-09-11T23:39:44.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:39:44 smithi026 ceph-mon[31172]: pgmap v672: 97 pgs: 13 active+clean+snaptrim, 30 active+clean+snaptrim_wait, 54 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 203 op/s 2024-09-11T23:39:44.552 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:39:44 smithi076 ceph-mon[35296]: pgmap v672: 97 pgs: 13 active+clean+snaptrim, 30 active+clean+snaptrim_wait, 54 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 203 op/s 2024-09-11T23:39:45.376 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.376+0000 7f0d974e0640 1 -- 172.21.15.26:0/2532261363 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0d880942f0 msgr2=0x7f0d880946d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:45.376 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.376+0000 7f0d974e0640 1 --2- 172.21.15.26:0/2532261363 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0d880942f0 0x7f0d880946d0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f0d8c009900 tx=0x7f0d8c03b400 comp rx=0 tx=0).stop 2024-09-11T23:39:45.376 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.376+0000 7f0d974e0640 1 -- 172.21.15.26:0/2532261363 shutdown_connections 2024-09-11T23:39:45.376 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.376+0000 7f0d974e0640 1 --2- 172.21.15.26:0/2532261363 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0d8809bbb0 0x7f0d880a2500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:45.376 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.376+0000 7f0d974e0640 1 --2- 172.21.15.26:0/2532261363 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0d88094ca0 0x7f0d8809b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:45.376 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.376+0000 7f0d974e0640 1 --2- 172.21.15.26:0/2532261363 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0d880942f0 0x7f0d880946d0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:45.376 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.376+0000 7f0d974e0640 1 -- 172.21.15.26:0/2532261363 >> 172.21.15.26:0/2532261363 conn(0x7f0d88009960 msgr2=0x7f0d88009d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:39:45.376 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.376+0000 7f0d974e0640 1 -- 172.21.15.26:0/2532261363 shutdown_connections 2024-09-11T23:39:45.376 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.376+0000 7f0d974e0640 1 -- 172.21.15.26:0/2532261363 wait complete. 2024-09-11T23:39:45.378 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.377+0000 7f0d974e0640 1 Processor -- start 2024-09-11T23:39:45.378 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.377+0000 7f0d974e0640 1 -- start start 2024-09-11T23:39:45.378 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.377+0000 7f0d974e0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0d880942f0 0x7f0d88129d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:45.378 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.377+0000 7f0d974e0640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0d8809bbb0 0x7f0d8812a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:45.379 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.377+0000 7f0d974e0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0d8812e5e0 0x7f0d8812ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:45.379 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.377+0000 7f0d974e0640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f0d880a3850 con 0x7f0d880942f0 2024-09-11T23:39:45.379 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.377+0000 7f0d974e0640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f0d880a36d0 con 0x7f0d8812e5e0 2024-09-11T23:39:45.379 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.378+0000 7f0d974e0640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f0d880a39d0 con 0x7f0d8809bbb0 2024-09-11T23:39:45.379 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.378+0000 7f0d964de640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0d880942f0 0x7f0d88129d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:45.379 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.378+0000 7f0d96cdf640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0d8812e5e0 0x7f0d8812ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:45.379 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.378+0000 7f0d96cdf640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0d8812e5e0 0x7f0d8812ea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:34858/0 (socket says 172.21.15.26:34858) 2024-09-11T23:39:45.379 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.378+0000 7f0d96cdf640 1 -- 172.21.15.26:0/1674125989 learned_addr learned my addr 172.21.15.26:0/1674125989 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:39:45.379 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.378+0000 7f0d964de640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0d880942f0 0x7f0d88129d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59714/0 (socket says 172.21.15.26:59714) 2024-09-11T23:39:45.379 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.378+0000 7f0d95cdd640 1 --2- 172.21.15.26:0/1674125989 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0d8809bbb0 0x7f0d8812a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:45.379 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.378+0000 7f0d96cdf640 1 -- 172.21.15.26:0/1674125989 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0d8809bbb0 msgr2=0x7f0d8812a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:45.379 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.378+0000 7f0d96cdf640 1 --2- 172.21.15.26:0/1674125989 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0d8809bbb0 0x7f0d8812a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:45.379 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.378+0000 7f0d96cdf640 1 -- 172.21.15.26:0/1674125989 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0d880942f0 msgr2=0x7f0d88129d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:45.380 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.378+0000 7f0d96cdf640 1 --2- 172.21.15.26:0/1674125989 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0d880942f0 0x7f0d88129d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:45.380 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.379+0000 7f0d96cdf640 1 -- 172.21.15.26:0/1674125989 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d8812f0d0 con 0x7f0d8812e5e0 2024-09-11T23:39:45.380 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.379+0000 7f0d96cdf640 1 --2- 172.21.15.26:0/1674125989 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0d8812e5e0 0x7f0d8812ea90 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f0d90067f00 tx=0x7f0d90074420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:39:45.380 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.379+0000 7f0d877fe640 1 -- 172.21.15.26:0/1674125989 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d9007d470 con 0x7f0d8812e5e0 2024-09-11T23:39:45.380 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.379+0000 7f0d877fe640 1 -- 172.21.15.26:0/1674125989 <== mon.2 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d9007daf0 con 0x7f0d8812e5e0 2024-09-11T23:39:45.380 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.380+0000 7f0d877fe640 1 -- 172.21.15.26:0/1674125989 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d9007fac0 con 0x7f0d8812e5e0 2024-09-11T23:39:45.380 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.380+0000 7f0d974e0640 1 -- 172.21.15.26:0/1674125989 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d8812f3c0 con 0x7f0d8812e5e0 2024-09-11T23:39:45.380 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.380+0000 7f0d974e0640 1 -- 172.21.15.26:0/1674125989 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d88136b30 con 0x7f0d8812e5e0 2024-09-11T23:39:45.383 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.383+0000 7f0d9b4dc740 1 Processor -- start 2024-09-11T23:39:45.389 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.388+0000 7f0d9b4dc740 1 -- start start 2024-09-11T23:39:45.389 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.389+0000 7f0d9b4dc740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b72974bcc0 0x55b72974c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:45.389 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.389+0000 7f0d9b4dc740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b72974c5e0 0x55b729756940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:45.389 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.389+0000 7f0d9b4dc740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b729756e80 0x55b729759270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:45.389 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.389+0000 7f0d9b4dc740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55b7296b02f0 con 0x55b72974c5e0 2024-09-11T23:39:45.390 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.389+0000 7f0d9b4dc740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55b729658270 con 0x55b72974bcc0 2024-09-11T23:39:45.390 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.389+0000 7f0d9b4dc740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55b7296ac2b0 con 0x55b729756e80 2024-09-11T23:39:45.390 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.389+0000 7f0d964de640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b72974c5e0 0x55b729756940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:45.390 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.389+0000 7f0d964de640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b72974c5e0 0x55b729756940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59730/0 (socket says 172.21.15.26:59730) 2024-09-11T23:39:45.390 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.389+0000 7f0d964de640 1 -- 172.21.15.26:0/394544742 learned_addr learned my addr 172.21.15.26:0/394544742 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:39:45.390 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.389+0000 7f0d95cdd640 1 --2- 172.21.15.26:0/394544742 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b72974bcc0 0x55b72974c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:45.390 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.389+0000 7f0d95cdd640 1 --2- 172.21.15.26:0/394544742 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b729756e80 0x55b729759270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:45.390 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.389+0000 7f0d964de640 1 -- 172.21.15.26:0/394544742 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b729756e80 msgr2=0x55b729759270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:39:45.390 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.389+0000 7f0d964de640 1 --2- 172.21.15.26:0/394544742 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b729756e80 0x55b729759270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:45.390 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.389+0000 7f0d964de640 1 -- 172.21.15.26:0/394544742 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b72974bcc0 msgr2=0x55b72974c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:39:45.390 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.389+0000 7f0d964de640 1 --2- 172.21.15.26:0/394544742 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b72974bcc0 0x55b72974c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:45.390 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.389+0000 7f0d964de640 1 -- 172.21.15.26:0/394544742 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b729657340 con 0x55b72974c5e0 2024-09-11T23:39:45.391 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.390+0000 7f0d95cdd640 1 --2- 172.21.15.26:0/394544742 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b729756e80 0x55b729759270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:39:45.391 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.390+0000 7f0d964de640 1 --2- 172.21.15.26:0/394544742 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b72974c5e0 0x55b729756940 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f0d8c009230 tx=0x7f0d8c004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7021e807c3b5b8e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:39:45.393 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.391+0000 7f0d84ff9640 1 -- 172.21.15.26:0/394544742 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d8c046040 con 0x55b72974c5e0 2024-09-11T23:39:45.393 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.391+0000 7f0d877fe640 1 -- 172.21.15.26:0/1674125989 <== mon.2 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f0d9007b040 con 0x7f0d8812e5e0 2024-09-11T23:39:45.393 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.391+0000 7f0d84ff9640 1 -- 172.21.15.26:0/394544742 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d8c005520 con 0x55b72974c5e0 2024-09-11T23:39:45.393 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.391+0000 7f0d84ff9640 1 -- 172.21.15.26:0/394544742 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d8c003850 con 0x55b72974c5e0 2024-09-11T23:39:45.393 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.392+0000 7f0d9b4dc740 1 -- 172.21.15.26:0/394544742 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b72974c5e0 msgr2=0x55b729756940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:45.393 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.392+0000 7f0d9b4dc740 1 --2- 172.21.15.26:0/394544742 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b72974c5e0 0x55b729756940 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f0d8c009230 tx=0x7f0d8c004040 comp rx=0 tx=0).stop 2024-09-11T23:39:45.393 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.392+0000 7f0d9b4dc740 1 -- 172.21.15.26:0/394544742 shutdown_connections 2024-09-11T23:39:45.393 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.392+0000 7f0d9b4dc740 1 --2- 172.21.15.26:0/394544742 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b729756e80 0x55b729759270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:45.393 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.392+0000 7f0d9b4dc740 1 --2- 172.21.15.26:0/394544742 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b72974c5e0 0x55b729756940 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:45.393 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.392+0000 7f0d9b4dc740 1 --2- 172.21.15.26:0/394544742 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b72974bcc0 0x55b72974c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:45.394 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.392+0000 7f0d9b4dc740 1 -- 172.21.15.26:0/394544742 >> 172.21.15.26:0/394544742 conn(0x55b729748230 msgr2=0x55b729748610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:39:45.394 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.392+0000 7f0d877fe640 1 --2- 172.21.15.26:0/1674125989 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f0d7c0a2d10 0x7f0d7c0a51d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:45.394 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.392+0000 7f0d9b4dc740 1 -- 172.21.15.26:0/394544742 shutdown_connections 2024-09-11T23:39:45.394 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.393+0000 7f0d9b4dc740 1 -- 172.21.15.26:0/394544742 wait complete. 2024-09-11T23:39:45.394 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.393+0000 7f0d877fe640 1 -- 172.21.15.26:0/1674125989 <== mon.2 v2:172.21.15.76:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 8586+0+0 (secure 0 0 0) 0x7f0d9013b7b0 con 0x7f0d8812e5e0 2024-09-11T23:39:45.394 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.393+0000 7f0d964de640 1 --2- 172.21.15.26:0/1674125989 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f0d7c0a2d10 0x7f0d7c0a51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:45.394 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.393+0000 7f0d9b4dc740 1 -- start start 2024-09-11T23:39:45.394 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.393+0000 7f0d9b4dc740 1 Processor -- start 2024-09-11T23:39:45.396 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.394+0000 7f0d9b4dc740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b72974bcc0 0x55b729753d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:45.396 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.394+0000 7f0d9b4dc740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b72974c5e0 0x55b7297542b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:45.397 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.394+0000 7f0d964de640 1 --2- 172.21.15.26:0/1674125989 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f0d7c0a2d10 0x7f0d7c0a51d0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f0d8c009900 tx=0x7f0d8c03b400 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:39:45.397 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.394+0000 7f0d9b4dc740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b729756e80 0x55b7297547f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:45.397 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.394+0000 7f0d9b4dc740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55b729759a50 con 0x55b729756e80 2024-09-11T23:39:45.397 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.394+0000 7f0d9b4dc740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55b729639290 con 0x55b72974bcc0 2024-09-11T23:39:45.397 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.394+0000 7f0d9b4dc740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55b729759bd0 con 0x55b72974c5e0 2024-09-11T23:39:45.397 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.394+0000 7f0d9b4dc740 10 client.0 _ll_register_callbacks cb 0x55b72966a140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:39:45.397 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.394+0000 7f0d95cdd640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b72974c5e0 0x55b7297542b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:45.397 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.394+0000 7f0d964de640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b72974bcc0 0x55b729753d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:45.397 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.394+0000 7f0d95cdd640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b72974c5e0 0x55b7297542b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:53780/0 (socket says 172.21.15.26:53780) 2024-09-11T23:39:45.397 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.394+0000 7f0d95cdd640 1 -- 172.21.15.26:0/2932677393 learned_addr learned my addr 172.21.15.26:0/2932677393 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:39:45.397 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.394+0000 7f0d96cdf640 1 --2- 172.21.15.26:0/2932677393 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b729756e80 0x55b7297547f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:45.398 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.394+0000 7f0d964de640 1 -- 172.21.15.26:0/2932677393 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b72974c5e0 msgr2=0x55b7297542b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:39:45.398 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.394+0000 7f0d964de640 1 --2- 172.21.15.26:0/2932677393 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b72974c5e0 0x55b7297542b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:45.398 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.394+0000 7f0d964de640 1 -- 172.21.15.26:0/2932677393 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b729756e80 msgr2=0x55b7297547f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:39:45.398 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.394+0000 7f0d964de640 1 --2- 172.21.15.26:0/2932677393 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b729756e80 0x55b7297547f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:45.398 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.394+0000 7f0d964de640 1 -- 172.21.15.26:0/2932677393 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b7293c3210 con 0x55b72974bcc0 2024-09-11T23:39:45.398 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.394+0000 7f0d95cdd640 1 --2- 172.21.15.26:0/2932677393 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b72974c5e0 0x55b7297542b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:39:45.398 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.394+0000 7f0d96cdf640 1 --2- 172.21.15.26:0/2932677393 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b729756e80 0x55b7297547f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:39:45.398 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.395+0000 7f0d964de640 1 --2- 172.21.15.26:0/2932677393 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b72974bcc0 0x55b729753d70 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f0d8c000c00 tx=0x7f0d8c004c50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a4d1f013128bd4bf server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:39:45.398 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.395+0000 7f0d79ffb640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-11T23:39:45.398 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.395+0000 7f0d79ffb640 1 -- 172.21.15.26:0/2932677393 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d8c06a070 con 0x55b72974bcc0 2024-09-11T23:39:45.398 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.395+0000 7f0d79ffb640 1 -- 172.21.15.26:0/2932677393 <== mon.2 v2:172.21.15.76:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 8586+0+0 (secure 0 0 0) 0x7f0d8c0035f0 con 0x55b72974bcc0 2024-09-11T23:39:45.399 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.395+0000 7f0d9b4dc740 1 -- 172.21.15.26:0/2932677393 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d8c005520 con 0x55b72974bcc0 2024-09-11T23:39:45.399 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.395+0000 7f0d79ffb640 1 -- 172.21.15.26:0/2932677393 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f0d8c076050 con 0x55b72974bcc0 2024-09-11T23:39:45.399 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.395+0000 7f0d9b4dc740 10 client.27614 fetch_fsmap learned FSMap version 92 2024-09-11T23:39:45.399 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.395+0000 7f0d9b4dc740 1 -- 172.21.15.26:0/2932677393 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=92,monmap=4+}) -- 0x55b72948f620 con 0x55b72974bcc0 2024-09-11T23:39:45.399 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.396+0000 7f0d79ffb640 1 -- 172.21.15.26:0/2932677393 <== mon.2 v2:172.21.15.76:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d8c0048d0 con 0x55b72974bcc0 2024-09-11T23:39:45.399 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.396+0000 7f0d79ffb640 1 -- 172.21.15.26:0/2932677393 <== mon.2 v2:172.21.15.76:3300/0 5 ==== fsmap(e 92) ==== 8430+0+0 (secure 0 0 0) 0x7f0d8c042980 con 0x55b72974bcc0 2024-09-11T23:39:45.399 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.396+0000 7f0d9b4dc740 10 client.27614 fetch_fsmap finished waiting for FSMap version 92 2024-09-11T23:39:45.399 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.396+0000 7f0d9b4dc740 10 client.27614 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:39:45.399 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.396+0000 7f0d9b4dc740 20 client.27614 populate_metadata read hostname 'smithi026' 2024-09-11T23:39:45.399 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.396+0000 7f0d9b4dc740 1 --2- 172.21.15.26:0/2932677393 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55b729766ce0 0x55b7297690d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:45.399 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.396+0000 7f0d9b4dc740 4 client.27614 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:39:45.399 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.396+0000 7f0d9b4dc740 1 -- 172.21.15.26:0/2932677393 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f0d8c005520 con 0x55b729766ce0 2024-09-11T23:39:45.399 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.397+0000 7f0d95cdd640 1 --2- 172.21.15.26:0/2932677393 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55b729766ce0 0x55b7297690d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:45.402 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.398+0000 7f0d95cdd640 1 --2- 172.21.15.26:0/2932677393 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55b729766ce0 0x55b7297690d0 crc :-1 s=READY pgs=316 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c60dd3d0536ebdd server_cookie=470207d39098cd12 in_seq=0 out_seq=0 2024-09-11T23:39:45.402 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.398+0000 7f0d79ffb640 10 client.27614 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:39:45.402 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.399+0000 7f0d79ffb640 1 -- 172.21.15.26:0/2932677393 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f0d8c005520 con 0x55b729766ce0 2024-09-11T23:39:45.402 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.399+0000 7f0d79ffb640 10 client.27614 handle_command_reply: tid=0 2024-09-11T23:39:45.402 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.399+0000 7f0d9b4dc740 1 client.27614 shutdown 2024-09-11T23:39:45.402 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.401+0000 7f0d9b4dc740 1 -- 172.21.15.26:0/2932677393 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b72974bcc0 msgr2=0x55b729753d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:45.402 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.401+0000 7f0d9b4dc740 1 --2- 172.21.15.26:0/2932677393 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b72974bcc0 0x55b729753d70 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f0d8c000c00 tx=0x7f0d8c004c50 comp rx=0 tx=0).stop 2024-09-11T23:39:45.402 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.401+0000 7f0d9b4dc740 1 -- 172.21.15.26:0/2932677393 shutdown_connections 2024-09-11T23:39:45.402 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.402+0000 7f0d9b4dc740 1 --2- 172.21.15.26:0/2932677393 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55b729766ce0 0x55b7297690d0 crc :-1 s=READY pgs=316 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:45.402 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.402+0000 7f0d79ffb640 0 client.27614 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:39:45.402 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.402+0000 7f0d9b4dc740 1 --2- 172.21.15.26:0/2932677393 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b729756e80 0x55b7297547f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:45.403 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.402+0000 7f0d9b4dc740 1 --2- 172.21.15.26:0/2932677393 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b72974c5e0 0x55b7297542b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:45.403 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.402+0000 7f0d9b4dc740 1 --2- 172.21.15.26:0/2932677393 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b72974bcc0 0x55b729753d70 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:45.403 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.402+0000 7f0d9b4dc740 1 -- 172.21.15.26:0/2932677393 >> 172.21.15.26:0/2932677393 conn(0x55b729748230 msgr2=0x55b72974df90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:39:45.403 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.402+0000 7f0d9b4dc740 1 -- 172.21.15.26:0/2932677393 shutdown_connections 2024-09-11T23:39:45.403 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.402+0000 7f0d9b4dc740 1 -- 172.21.15.26:0/2932677393 wait complete. 2024-09-11T23:39:45.403 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.402+0000 7f0d9b4dc740 20 client.27614 trim_cache size 0 max 16384 2024-09-11T23:39:45.427 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.427+0000 7f0d9b4dc740 1 Processor -- start 2024-09-11T23:39:45.428 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.427+0000 7f0d9b4dc740 1 -- start start 2024-09-11T23:39:45.428 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.428+0000 7f0d9b4dc740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b72974bcc0 0x55b72974e780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:45.430 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.428+0000 7f0d9b4dc740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b72974c5e0 0x55b729754620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:45.430 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.428+0000 7f0d9b4dc740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b729756e80 0x55b72976e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:45.430 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.428+0000 7f0d95cdd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b72974c5e0 0x55b729754620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:45.430 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.428+0000 7f0d95cdd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b72974c5e0 0x55b729754620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:34884/0 (socket says 172.21.15.26:34884) 2024-09-11T23:39:45.430 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.429+0000 7f0d95cdd640 1 -- 172.21.15.26:0/782953685 learned_addr learned my addr 172.21.15.26:0/782953685 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:39:45.430 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.429+0000 7f0d96cdf640 1 --2- 172.21.15.26:0/782953685 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b729756e80 0x55b72976e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:45.430 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.429+0000 7f0d9b4dc740 1 -- 172.21.15.26:0/782953685 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f0d8813f2a0 con 0x55b729756e80 2024-09-11T23:39:45.430 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.429+0000 7f0d9b4dc740 1 -- 172.21.15.26:0/782953685 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f0d88141050 con 0x55b72974c5e0 2024-09-11T23:39:45.431 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.429+0000 7f0d9b4dc740 1 -- 172.21.15.26:0/782953685 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f0d8813f9a0 con 0x55b72974bcc0 2024-09-11T23:39:45.431 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.430+0000 7f0d95cdd640 1 -- 172.21.15.26:0/782953685 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b72974bcc0 msgr2=0x55b72974e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:39:45.431 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.430+0000 7f0d95cdd640 1 --2- 172.21.15.26:0/782953685 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b72974bcc0 0x55b72974e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:45.431 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.430+0000 7f0d95cdd640 1 -- 172.21.15.26:0/782953685 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b729756e80 msgr2=0x55b72976e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:39:45.431 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.430+0000 7f0d95cdd640 1 --2- 172.21.15.26:0/782953685 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b729756e80 0x55b72976e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:45.431 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.430+0000 7f0d95cdd640 1 -- 172.21.15.26:0/782953685 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b729759930 con 0x55b72974c5e0 2024-09-11T23:39:45.431 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.430+0000 7f0d95cdd640 1 --2- 172.21.15.26:0/782953685 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b72974c5e0 0x55b729754620 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f0d80009fd0 tx=0x7f0d80004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6cc010903800316b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:39:45.431 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.431+0000 7f0d7bfff640 1 -- 172.21.15.26:0/782953685 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d80004030 con 0x55b72974c5e0 2024-09-11T23:39:45.431 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.431+0000 7f0d7bfff640 1 -- 172.21.15.26:0/782953685 <== mon.2 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d800041d0 con 0x55b72974c5e0 2024-09-11T23:39:45.431 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.431+0000 7f0d7bfff640 1 -- 172.21.15.26:0/782953685 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d80010520 con 0x55b72974c5e0 2024-09-11T23:39:45.436 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.432+0000 7f0d9b4dc740 1 -- 172.21.15.26:0/782953685 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b72974c5e0 msgr2=0x55b729754620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:45.436 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.432+0000 7f0d9b4dc740 1 --2- 172.21.15.26:0/782953685 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b72974c5e0 0x55b729754620 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f0d80009fd0 tx=0x7f0d80004730 comp rx=0 tx=0).stop 2024-09-11T23:39:45.436 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.432+0000 7f0d9b4dc740 1 -- 172.21.15.26:0/782953685 shutdown_connections 2024-09-11T23:39:45.436 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.432+0000 7f0d9b4dc740 1 --2- 172.21.15.26:0/782953685 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b72974c5e0 0x55b729754620 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:45.436 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.432+0000 7f0d9b4dc740 1 --2- 172.21.15.26:0/782953685 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b729756e80 0x55b72976e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:45.436 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.432+0000 7f0d9b4dc740 1 --2- 172.21.15.26:0/782953685 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b72974bcc0 0x55b72974e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:45.436 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.432+0000 7f0d9b4dc740 1 -- 172.21.15.26:0/782953685 >> 172.21.15.26:0/782953685 conn(0x55b729748230 msgr2=0x55b729752e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:39:45.436 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.432+0000 7f0d9b4dc740 1 -- 172.21.15.26:0/782953685 shutdown_connections 2024-09-11T23:39:45.436 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.432+0000 7f0d9b4dc740 1 -- 172.21.15.26:0/782953685 wait complete. 2024-09-11T23:39:45.436 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.432+0000 7f0d9b4dc740 1 -- start start 2024-09-11T23:39:45.436 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.432+0000 7f0d9b4dc740 1 Processor -- start 2024-09-11T23:39:45.436 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.433+0000 7f0d9b4dc740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b72974bcc0 0x55b729759eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:45.437 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.433+0000 7f0d9b4dc740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b72974c5e0 0x55b72975a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:45.437 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.433+0000 7f0d9b4dc740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b729756e80 0x55b72975a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:45.437 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.433+0000 7f0d9b4dc740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f0d881408c0 con 0x55b729756e80 2024-09-11T23:39:45.437 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.433+0000 7f0d9b4dc740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f0d88140130 con 0x55b72974bcc0 2024-09-11T23:39:45.437 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.433+0000 7f0d9b4dc740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f0d8813d2a0 con 0x55b72974c5e0 2024-09-11T23:39:45.437 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.433+0000 7f0d9b4dc740 10 client.0 _ll_register_callbacks cb 0x7f0d80009a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:39:45.437 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.433+0000 7f0d964de640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b72974bcc0 0x55b729759eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:45.437 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.433+0000 7f0d95cdd640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b72974c5e0 0x55b72975a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:45.437 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.433+0000 7f0d964de640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b72974bcc0 0x55b729759eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:34900/0 (socket says 172.21.15.26:34900) 2024-09-11T23:39:45.437 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.433+0000 7f0d964de640 1 -- 172.21.15.26:0/4120351420 learned_addr learned my addr 172.21.15.26:0/4120351420 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:39:45.437 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.433+0000 7f0d96cdf640 1 --2- 172.21.15.26:0/4120351420 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b729756e80 0x55b72975a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:45.437 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.433+0000 7f0d964de640 1 -- 172.21.15.26:0/4120351420 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b72974c5e0 msgr2=0x55b72975a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:39:45.437 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.433+0000 7f0d964de640 1 --2- 172.21.15.26:0/4120351420 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b72974c5e0 0x55b72975a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:45.438 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.433+0000 7f0d964de640 1 -- 172.21.15.26:0/4120351420 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b729756e80 msgr2=0x55b72975a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:39:45.438 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.433+0000 7f0d964de640 1 --2- 172.21.15.26:0/4120351420 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b729756e80 0x55b72975a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:45.438 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.433+0000 7f0d964de640 1 -- 172.21.15.26:0/4120351420 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b7297cf8a0 con 0x55b72974bcc0 2024-09-11T23:39:45.438 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.433+0000 7f0d964de640 1 --2- 172.21.15.26:0/4120351420 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b72974bcc0 0x55b729759eb0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f0d8c042550 tx=0x7f0d8c042480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d317e804c0a70977 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:39:45.438 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.433+0000 7f0d79ffb640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-11T23:39:45.438 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.434+0000 7f0d79ffb640 1 -- 172.21.15.26:0/4120351420 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d8c06a070 con 0x55b72974bcc0 2024-09-11T23:39:45.438 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.434+0000 7f0d79ffb640 1 -- 172.21.15.26:0/4120351420 <== mon.2 v2:172.21.15.76:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 8586+0+0 (secure 0 0 0) 0x7f0d8c062420 con 0x55b72974bcc0 2024-09-11T23:39:45.438 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.434+0000 7f0d9b4dc740 1 -- 172.21.15.26:0/4120351420 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d800041d0 con 0x55b72974bcc0 2024-09-11T23:39:45.438 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.434+0000 7f0d79ffb640 1 -- 172.21.15.26:0/4120351420 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f0d8c046050 con 0x55b72974bcc0 2024-09-11T23:39:45.438 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.434+0000 7f0d9b4dc740 10 client.27620 fetch_fsmap learned FSMap version 92 2024-09-11T23:39:45.438 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.434+0000 7f0d9b4dc740 1 -- 172.21.15.26:0/4120351420 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=92,monmap=4+}) -- 0x55b7297d0af0 con 0x55b72974bcc0 2024-09-11T23:39:45.438 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.434+0000 7f0d79ffb640 1 -- 172.21.15.26:0/4120351420 <== mon.2 v2:172.21.15.76:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d800041d0 con 0x55b72974bcc0 2024-09-11T23:39:45.438 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.435+0000 7f0d79ffb640 1 -- 172.21.15.26:0/4120351420 <== mon.2 v2:172.21.15.76:3300/0 5 ==== fsmap(e 92) ==== 8430+0+0 (secure 0 0 0) 0x7f0d8c062940 con 0x55b72974bcc0 2024-09-11T23:39:45.439 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.435+0000 7f0d9b4dc740 10 client.27620 fetch_fsmap finished waiting for FSMap version 92 2024-09-11T23:39:45.439 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.435+0000 7f0d9b4dc740 10 client.27620 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:39:45.439 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.435+0000 7f0d9b4dc740 20 client.27620 populate_metadata read hostname 'smithi026' 2024-09-11T23:39:45.439 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.435+0000 7f0d9b4dc740 1 --2- 172.21.15.26:0/4120351420 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55b7297d1270 0x55b7297d3690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:45.439 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.435+0000 7f0d9b4dc740 4 client.27620 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T23:39:45.439 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.435+0000 7f0d9b4dc740 1 -- 172.21.15.26:0/4120351420 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b7296bda20 con 0x55b7297d1270 2024-09-11T23:39:45.439 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.436+0000 7f0d95cdd640 1 --2- 172.21.15.26:0/4120351420 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55b7297d1270 0x55b7297d3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:45.439 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.436+0000 7f0d95cdd640 1 --2- 172.21.15.26:0/4120351420 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55b7297d1270 0x55b7297d3690 crc :-1 s=READY pgs=317 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b05bb157e65db26 server_cookie=1e53c1a7a04f5353 in_seq=0 out_seq=0 2024-09-11T23:39:45.439 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.436+0000 7f0d79ffb640 10 client.27620 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:39:45.472 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.473+0000 7f0d79ffb640 1 -- 172.21.15.26:0/4120351420 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b7296bda20 con 0x55b7297d1270 2024-09-11T23:39:45.472 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.473+0000 7f0d79ffb640 10 client.27620 handle_command_reply: tid=0 2024-09-11T23:39:45.473 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.474+0000 7f0d9b4dc740 1 client.27620 shutdown 2024-09-11T23:39:45.473 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.474+0000 7f0d9b4dc740 1 -- 172.21.15.26:0/4120351420 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b72974bcc0 msgr2=0x55b729759eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:45.474 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.474+0000 7f0d9b4dc740 1 --2- 172.21.15.26:0/4120351420 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b72974bcc0 0x55b729759eb0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f0d8c042550 tx=0x7f0d8c042480 comp rx=0 tx=0).stop 2024-09-11T23:39:45.474 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.474+0000 7f0d9b4dc740 1 -- 172.21.15.26:0/4120351420 shutdown_connections 2024-09-11T23:39:45.474 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.474+0000 7f0d9b4dc740 1 --2- 172.21.15.26:0/4120351420 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55b7297d1270 0x55b7297d3690 crc :-1 s=READY pgs=317 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:45.474 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.474+0000 7f0d9b4dc740 1 --2- 172.21.15.26:0/4120351420 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b729756e80 0x55b72975a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:45.474 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.474+0000 7f0d79ffb640 0 client.27620 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:39:45.474 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.474+0000 7f0d9b4dc740 1 --2- 172.21.15.26:0/4120351420 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b72974c5e0 0x55b72975a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:45.474 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.474+0000 7f0d9b4dc740 1 --2- 172.21.15.26:0/4120351420 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b72974bcc0 0x55b729759eb0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:45.474 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.474+0000 7f0d9b4dc740 1 -- 172.21.15.26:0/4120351420 >> 172.21.15.26:0/4120351420 conn(0x55b729748230 msgr2=0x55b729754630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:39:45.475 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.475+0000 7f0d9b4dc740 1 -- 172.21.15.26:0/4120351420 shutdown_connections 2024-09-11T23:39:45.475 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.475+0000 7f0d9b4dc740 1 -- 172.21.15.26:0/4120351420 wait complete. 2024-09-11T23:39:45.475 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.476+0000 7f0d9b4dc740 20 client.27620 trim_cache size 0 max 16384 2024-09-11T23:39:45.477 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-09-11T23:39:45.478 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "message": "", 2024-09-11T23:39:45.478 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-09-11T23:39:45.478 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-09-11T23:39:45.478 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.479+0000 7f0d974e0640 1 -- 172.21.15.26:0/1674125989 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f0d7c0a2d10 msgr2=0x7f0d7c0a51d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:45.479 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.479+0000 7f0d974e0640 1 --2- 172.21.15.26:0/1674125989 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f0d7c0a2d10 0x7f0d7c0a51d0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f0d8c009900 tx=0x7f0d8c03b400 comp rx=0 tx=0).stop 2024-09-11T23:39:45.479 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.479+0000 7f0d974e0640 1 -- 172.21.15.26:0/1674125989 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0d8812e5e0 msgr2=0x7f0d8812ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:45.479 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.479+0000 7f0d974e0640 1 --2- 172.21.15.26:0/1674125989 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0d8812e5e0 0x7f0d8812ea90 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f0d90067f00 tx=0x7f0d90074420 comp rx=0 tx=0).stop 2024-09-11T23:39:45.480 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.480+0000 7f0d974e0640 1 -- 172.21.15.26:0/1674125989 shutdown_connections 2024-09-11T23:39:45.480 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.480+0000 7f0d974e0640 1 --2- 172.21.15.26:0/1674125989 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f0d7c0a2d10 0x7f0d7c0a51d0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:45.480 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.480+0000 7f0d974e0640 1 --2- 172.21.15.26:0/1674125989 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0d8812e5e0 0x7f0d8812ea90 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:45.480 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.480+0000 7f0d974e0640 1 --2- 172.21.15.26:0/1674125989 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0d8809bbb0 0x7f0d8812a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:45.480 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.480+0000 7f0d974e0640 1 --2- 172.21.15.26:0/1674125989 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0d880942f0 0x7f0d88129d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:45.480 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.480+0000 7f0d974e0640 1 -- 172.21.15.26:0/1674125989 >> 172.21.15.26:0/1674125989 conn(0x7f0d88009960 msgr2=0x7f0d88099600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:39:45.480 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.480+0000 7f0d974e0640 1 -- 172.21.15.26:0/1674125989 shutdown_connections 2024-09-11T23:39:45.480 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-09-11T23:39:45.480+0000 7f0d974e0640 1 -- 172.21.15.26:0/1674125989 wait complete. 2024-09-11T23:39:45.809 INFO:tasks.workunit:Stopping ['suites/blogbench.sh'] on client.0... 2024-09-11T23:39:45.809 DEBUG:teuthology.orchestra.run.smithi026:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-09-11T23:39:46.313 DEBUG:teuthology.parallel:result is None 2024-09-11T23:39:46.314 DEBUG:teuthology.orchestra.run.smithi026:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-09-11T23:39:46.391 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:39:46 smithi026 ceph-mon[31172]: pgmap v673: 97 pgs: 13 active+clean+snaptrim, 30 active+clean+snaptrim_wait, 54 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.6 MiB/s wr, 181 op/s 2024-09-11T23:39:46.392 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-09-11T23:39:46.393 DEBUG:teuthology.run_tasks:Unwinding manager quiescer 2024-09-11T23:39:46.409 INFO:tasks.quiescer:joining Quiescers 2024-09-11T23:39:46.409 WARNING:tasks.quiescer:The quiescer is requested to stop, running cancel all 2024-09-11T23:39:46.410 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:39:46.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:39:46 smithi136 ceph-mon[35121]: pgmap v673: 97 pgs: 13 active+clean+snaptrim, 30 active+clean+snaptrim_wait, 54 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.6 MiB/s wr, 181 op/s 2024-09-11T23:39:46.552 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:39:46 smithi076 ceph-mon[35296]: pgmap v673: 97 pgs: 13 active+clean+snaptrim, 30 active+clean+snaptrim_wait, 54 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.6 MiB/s wr, 181 op/s 2024-09-11T23:39:46.771 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:39:47.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:47.498+0000 7fcc4e058640 1 -- 172.21.15.26:0/4052199318 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc48101df0 msgr2=0x7fcc481021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:47.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:47.498+0000 7fcc4e058640 1 --2- 172.21.15.26:0/4052199318 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc48101df0 0x7fcc481021d0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7fcc38009860 tx=0x7fcc3802a340 comp rx=0 tx=0).stop 2024-09-11T23:39:47.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:47.498+0000 7fcc4e058640 1 -- 172.21.15.26:0/4052199318 shutdown_connections 2024-09-11T23:39:47.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:47.498+0000 7fcc4e058640 1 --2- 172.21.15.26:0/4052199318 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcc481027a0 0x7fcc481067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:47.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:47.498+0000 7fcc4e058640 1 --2- 172.21.15.26:0/4052199318 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcc48106f20 0x7fcc4810eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:47.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:47.498+0000 7fcc4e058640 1 --2- 172.21.15.26:0/4052199318 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc48101df0 0x7fcc481021d0 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:47.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:47.498+0000 7fcc4e058640 1 -- 172.21.15.26:0/4052199318 >> 172.21.15.26:0/4052199318 conn(0x7fcc480fd8f0 msgr2=0x7fcc480ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:39:47.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:47.498+0000 7fcc4e058640 1 -- 172.21.15.26:0/4052199318 shutdown_connections 2024-09-11T23:39:47.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:47.498+0000 7fcc4e058640 1 -- 172.21.15.26:0/4052199318 wait complete. 2024-09-11T23:39:47.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:47.499+0000 7fcc4e058640 1 Processor -- start 2024-09-11T23:39:47.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:47.499+0000 7fcc4e058640 1 -- start start 2024-09-11T23:39:47.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:47.499+0000 7fcc4e058640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcc48101df0 0x7fcc48198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:47.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:47.499+0000 7fcc4e058640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcc481027a0 0x7fcc48198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:47.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:47.500+0000 7fcc4e058640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc48106f20 0x7fcc4819ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:47.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:47.500+0000 7fcc477fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcc48101df0 0x7fcc48198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:47.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:47.500+0000 7fcc477fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcc48101df0 0x7fcc48198420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:34916/0 (socket says 172.21.15.26:34916) 2024-09-11T23:39:47.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:47.500+0000 7fcc46ffd640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcc481027a0 0x7fcc48198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:47.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:47.500+0000 7fcc46ffd640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcc481027a0 0x7fcc48198960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:53818/0 (socket says 172.21.15.26:53818) 2024-09-11T23:39:47.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:47.500+0000 7fcc477fe640 1 -- 172.21.15.26:0/448008134 learned_addr learned my addr 172.21.15.26:0/448008134 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:39:47.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:47.500+0000 7fcc4e058640 1 -- 172.21.15.26:0/448008134 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fcc48111f80 con 0x7fcc48106f20 2024-09-11T23:39:47.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:47.500+0000 7fcc4e058640 1 -- 172.21.15.26:0/448008134 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fcc48111e00 con 0x7fcc48101df0 2024-09-11T23:39:47.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:47.500+0000 7fcc4e058640 1 -- 172.21.15.26:0/448008134 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fcc48112100 con 0x7fcc481027a0 2024-09-11T23:39:47.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:47.500+0000 7fcc47fff640 1 --2- 172.21.15.26:0/448008134 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc48106f20 0x7fcc4819ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:47.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:47.501+0000 7fcc46ffd640 1 -- 172.21.15.26:0/448008134 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcc48101df0 msgr2=0x7fcc48198420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:47.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:47.501+0000 7fcc46ffd640 1 --2- 172.21.15.26:0/448008134 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcc48101df0 0x7fcc48198420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:47.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:47.501+0000 7fcc46ffd640 1 -- 172.21.15.26:0/448008134 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc48106f20 msgr2=0x7fcc4819ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:47.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:47.501+0000 7fcc46ffd640 1 --2- 172.21.15.26:0/448008134 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc48106f20 0x7fcc4819ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:47.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:47.501+0000 7fcc46ffd640 1 -- 172.21.15.26:0/448008134 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc4819d3d0 con 0x7fcc481027a0 2024-09-11T23:39:47.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:47.501+0000 7fcc46ffd640 1 --2- 172.21.15.26:0/448008134 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcc481027a0 0x7fcc48198960 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fcc3c00d6d0 tx=0x7fcc3c00db90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:39:47.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:47.502+0000 7fcc44ff9640 1 -- 172.21.15.26:0/448008134 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc3c018070 con 0x7fcc481027a0 2024-09-11T23:39:47.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:47.502+0000 7fcc4e058640 1 -- 172.21.15.26:0/448008134 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc4819d6c0 con 0x7fcc481027a0 2024-09-11T23:39:47.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:47.502+0000 7fcc4e058640 1 -- 172.21.15.26:0/448008134 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc48074420 con 0x7fcc481027a0 2024-09-11T23:39:47.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:47.502+0000 7fcc44ff9640 1 -- 172.21.15.26:0/448008134 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc3c009820 con 0x7fcc481027a0 2024-09-11T23:39:47.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:47.502+0000 7fcc44ff9640 1 -- 172.21.15.26:0/448008134 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc3c004cd0 con 0x7fcc481027a0 2024-09-11T23:39:47.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:47.504+0000 7fcc4e058640 1 -- 172.21.15.26:0/448008134 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc34005290 con 0x7fcc481027a0 2024-09-11T23:39:47.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:47.505+0000 7fcc44ff9640 1 -- 172.21.15.26:0/448008134 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fcc3c005220 con 0x7fcc481027a0 2024-09-11T23:39:47.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:47.506+0000 7fcc44ff9640 1 --2- 172.21.15.26:0/448008134 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fcc300a2d70 0x7fcc300a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:47.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:47.506+0000 7fcc44ff9640 1 -- 172.21.15.26:0/448008134 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 8586+0+0 (secure 0 0 0) 0x7fcc3c0d2d00 con 0x7fcc481027a0 2024-09-11T23:39:47.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:47.506+0000 7fcc477fe640 1 --2- 172.21.15.26:0/448008134 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fcc300a2d70 0x7fcc300a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:47.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:47.507+0000 7fcc477fe640 1 --2- 172.21.15.26:0/448008134 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fcc300a2d70 0x7fcc300a5230 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fcc380057c0 tx=0x7fcc380056d0 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:39:47.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:47.514+0000 7fcc44ff9640 1 -- 172.21.15.26:0/448008134 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fcc3c09f9f0 con 0x7fcc481027a0 2024-09-11T23:39:47.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:47.859+0000 7fcc4e058640 1 -- 172.21.15.26:0/448008134 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcc34005850 con 0x7fcc481027a0 2024-09-11T23:39:47.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:47.860+0000 7fcc44ff9640 1 -- 172.21.15.26:0/448008134 <== mon.1 v2:172.21.15.136:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 92 v92) ==== 94+0+12001 (secure 0 0 0) 0x7fcc3c09f2f0 con 0x7fcc481027a0 2024-09-11T23:39:47.861 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-11T23:39:47.861 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":92,"btime":"2024-09-11T23:39:19:039942+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14499,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/367659515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":367659515},{"type":"v1","addr":"172.21.15.26:6835","nonce":367659515}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14511,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/2041065299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":2041065299},{"type":"v1","addr":"172.21.15.26:6837","nonce":2041065299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":92,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:16:26.544203+0000","modified":"2024-09-11T23:39:18.275139+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24485,"mds_1":24463,"mds_2":24458,"mds_3":24448,"mds_4":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14550":{"gid":14550,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6839/2233068305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":2233068305},{"type":"v1","addr":"172.21.15.26:6839","nonce":2233068305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14562":{"gid":14562,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6841/2984982336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":2984982336},{"type":"v1","addr":"172.21.15.26:6841","nonce":2984982336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6833/2019841455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":2019841455},{"type":"v1","addr":"172.21.15.136:6833","nonce":2019841455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.76:6833/3438147050","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3438147050},{"type":"v1","addr":"172.21.15.76:6833","nonce":3438147050}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6835/3617665975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3617665975},{"type":"v1","addr":"172.21.15.136:6835","nonce":3617665975}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":12,"addr":"172.21.15.76:6835/393308499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":393308499},{"type":"v1","addr":"172.21.15.76:6835","nonce":393308499}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":10,"addr":"172.21.15.136:6837/1633082387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":1633082387},{"type":"v1","addr":"172.21.15.136:6837","nonce":1633082387}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.76:6837/2078978748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":2078978748},{"type":"v1","addr":"172.21.15.76:6837","nonce":2078978748}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":63,"addr":"172.21.15.136:6839/3211398630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":3211398630},{"type":"v1","addr":"172.21.15.136:6839","nonce":3211398630}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.76:6839/2077637090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6838","nonce":2077637090},{"type":"v1","addr":"172.21.15.76:6839","nonce":2077637090}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24485,"qdb_cluster":[24485,24463,24458,24448,24443]},"id":1}]} 2024-09-11T23:39:47.862 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 92 2024-09-11T23:39:47.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:47.863+0000 7fcc4e058640 1 -- 172.21.15.26:0/448008134 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fcc300a2d70 msgr2=0x7fcc300a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:47.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:47.863+0000 7fcc4e058640 1 --2- 172.21.15.26:0/448008134 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fcc300a2d70 0x7fcc300a5230 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fcc380057c0 tx=0x7fcc380056d0 comp rx=0 tx=0).stop 2024-09-11T23:39:47.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:47.863+0000 7fcc4e058640 1 -- 172.21.15.26:0/448008134 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcc481027a0 msgr2=0x7fcc48198960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:47.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:47.863+0000 7fcc4e058640 1 --2- 172.21.15.26:0/448008134 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcc481027a0 0x7fcc48198960 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fcc3c00d6d0 tx=0x7fcc3c00db90 comp rx=0 tx=0).stop 2024-09-11T23:39:47.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:47.863+0000 7fcc4e058640 1 -- 172.21.15.26:0/448008134 shutdown_connections 2024-09-11T23:39:47.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:47.863+0000 7fcc4e058640 1 --2- 172.21.15.26:0/448008134 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fcc300a2d70 0x7fcc300a5230 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:47.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:47.863+0000 7fcc4e058640 1 --2- 172.21.15.26:0/448008134 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc48106f20 0x7fcc4819ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:47.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:47.863+0000 7fcc4e058640 1 --2- 172.21.15.26:0/448008134 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcc481027a0 0x7fcc48198960 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:47.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:47.863+0000 7fcc4e058640 1 --2- 172.21.15.26:0/448008134 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcc48101df0 0x7fcc48198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:47.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:47.863+0000 7fcc4e058640 1 -- 172.21.15.26:0/448008134 >> 172.21.15.26:0/448008134 conn(0x7fcc480fd8f0 msgr2=0x7fcc480fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:39:47.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:47.863+0000 7fcc4e058640 1 -- 172.21.15.26:0/448008134 shutdown_connections 2024-09-11T23:39:47.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:47.864+0000 7fcc4e058640 1 -- 172.21.15.26:0/448008134 wait complete. 2024-09-11T23:39:48.196 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:39:48 smithi026 ceph-mon[31172]: pgmap v674: 97 pgs: 10 active+clean+snaptrim, 22 active+clean+snaptrim_wait, 65 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.2 MiB/s wr, 451 op/s 2024-09-11T23:39:48.196 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:39:48 smithi026 ceph-mon[31172]: from='client.? 172.21.15.26:0/448008134' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:39:48.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:39:48 smithi136 ceph-mon[35121]: pgmap v674: 97 pgs: 10 active+clean+snaptrim, 22 active+clean+snaptrim_wait, 65 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.2 MiB/s wr, 451 op/s 2024-09-11T23:39:48.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:39:48 smithi136 ceph-mon[35121]: from='client.? 172.21.15.26:0/448008134' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:39:48.552 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:39:48 smithi076 ceph-mon[35296]: pgmap v674: 97 pgs: 10 active+clean+snaptrim, 22 active+clean+snaptrim_wait, 65 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.2 MiB/s wr, 451 op/s 2024-09-11T23:39:48.552 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:39:48 smithi076 ceph-mon[35296]: from='client.? 172.21.15.26:0/448008134' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:39:48.656 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24485 quiesce db --cancel --all' 2024-09-11T23:39:48.657 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24485 quiesce db --cancel --all 2024-09-11T23:39:48.987 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:39:49.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.508+0000 7f75f0699640 1 -- 172.21.15.26:0/157711052 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f75e81036e0 msgr2=0x7f75e8107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:49.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.508+0000 7f75f0699640 1 --2- 172.21.15.26:0/157711052 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f75e81036e0 0x7f75e8107b90 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f75d8009930 tx=0x7f75d802a420 comp rx=0 tx=0).stop 2024-09-11T23:39:49.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.508+0000 7f75f0699640 1 -- 172.21.15.26:0/157711052 shutdown_connections 2024-09-11T23:39:49.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.508+0000 7f75f0699640 1 --2- 172.21.15.26:0/157711052 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f75e81082a0 0x7f75e810ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:49.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.508+0000 7f75f0699640 1 --2- 172.21.15.26:0/157711052 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f75e81036e0 0x7f75e8107b90 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:49.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.508+0000 7f75f0699640 1 --2- 172.21.15.26:0/157711052 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f75e8102d30 0x7f75e8103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:49.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.508+0000 7f75f0699640 1 -- 172.21.15.26:0/157711052 >> 172.21.15.26:0/157711052 conn(0x7f75e80fc7d0 msgr2=0x7f75e80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:39:49.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.508+0000 7f75f0699640 1 -- 172.21.15.26:0/157711052 shutdown_connections 2024-09-11T23:39:49.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.508+0000 7f75f0699640 1 -- 172.21.15.26:0/157711052 wait complete. 2024-09-11T23:39:49.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.508+0000 7f75f0699640 1 Processor -- start 2024-09-11T23:39:49.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.508+0000 7f75f0699640 1 -- start start 2024-09-11T23:39:49.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.509+0000 7f75f0699640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f75e8102d30 0x7f75e8198640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:49.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.509+0000 7f75f0699640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f75e81036e0 0x7f75e8198b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:49.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.509+0000 7f75f0699640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f75e81082a0 0x7f75e819cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:49.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.509+0000 7f75f0699640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f75e81120a0 con 0x7f75e81082a0 2024-09-11T23:39:49.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.509+0000 7f75f0699640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f75e8111f20 con 0x7f75e81036e0 2024-09-11T23:39:49.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.509+0000 7f75f0699640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f75e8112220 con 0x7f75e8102d30 2024-09-11T23:39:49.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.509+0000 7f75eec36640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f75e81082a0 0x7f75e819cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:49.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.509+0000 7f75edc34640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f75e81036e0 0x7f75e8198b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:49.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.509+0000 7f75eec36640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f75e81082a0 0x7f75e819cf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59810/0 (socket says 172.21.15.26:59810) 2024-09-11T23:39:49.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.509+0000 7f75eec36640 1 -- 172.21.15.26:0/3659579331 learned_addr learned my addr 172.21.15.26:0/3659579331 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:39:49.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.509+0000 7f75ee435640 1 --2- 172.21.15.26:0/3659579331 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f75e8102d30 0x7f75e8198640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:49.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.509+0000 7f75edc34640 1 -- 172.21.15.26:0/3659579331 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f75e8102d30 msgr2=0x7f75e8198640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:49.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.509+0000 7f75edc34640 1 --2- 172.21.15.26:0/3659579331 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f75e8102d30 0x7f75e8198640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:49.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.509+0000 7f75edc34640 1 -- 172.21.15.26:0/3659579331 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f75e81082a0 msgr2=0x7f75e819cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:49.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.509+0000 7f75edc34640 1 --2- 172.21.15.26:0/3659579331 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f75e81082a0 0x7f75e819cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:49.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.509+0000 7f75edc34640 1 -- 172.21.15.26:0/3659579331 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75e819d5f0 con 0x7f75e81036e0 2024-09-11T23:39:49.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.509+0000 7f75eec36640 1 --2- 172.21.15.26:0/3659579331 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f75e81082a0 0x7f75e819cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:39:49.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.510+0000 7f75edc34640 1 --2- 172.21.15.26:0/3659579331 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f75e81036e0 0x7f75e8198b80 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f75d802a940 tx=0x7f75d8031b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:39:49.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.510+0000 7f75df7fe640 1 -- 172.21.15.26:0/3659579331 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75d803f070 con 0x7f75e81036e0 2024-09-11T23:39:49.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.510+0000 7f75f0699640 1 -- 172.21.15.26:0/3659579331 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75e819d880 con 0x7f75e81036e0 2024-09-11T23:39:49.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.510+0000 7f75df7fe640 1 -- 172.21.15.26:0/3659579331 <== mon.2 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f75d8003ad0 con 0x7f75e81036e0 2024-09-11T23:39:49.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.510+0000 7f75df7fe640 1 -- 172.21.15.26:0/3659579331 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75d8035070 con 0x7f75e81036e0 2024-09-11T23:39:49.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.510+0000 7f75f0699640 1 -- 172.21.15.26:0/3659579331 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75e81a51c0 con 0x7f75e81036e0 2024-09-11T23:39:49.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.512+0000 7f75df7fe640 1 -- 172.21.15.26:0/3659579331 <== mon.2 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f75d80035f0 con 0x7f75e81036e0 2024-09-11T23:39:49.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.513+0000 7f75df7fe640 1 --2- 172.21.15.26:0/3659579331 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f75d40a2cf0 0x7f75d40a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:49.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.513+0000 7f75ee435640 1 --2- 172.21.15.26:0/3659579331 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f75d40a2cf0 0x7f75d40a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:49.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.513+0000 7f75df7fe640 1 -- 172.21.15.26:0/3659579331 <== mon.2 v2:172.21.15.76:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 8586+0+0 (secure 0 0 0) 0x7f75d80ef9a0 con 0x7f75e81036e0 2024-09-11T23:39:49.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.513+0000 7f75f2a72740 1 Processor -- start 2024-09-11T23:39:49.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.514+0000 7f75ee435640 1 --2- 172.21.15.26:0/3659579331 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f75d40a2cf0 0x7f75d40a51b0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f75e4009bd0 tx=0x7f75e4005f90 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:39:49.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.514+0000 7f75f2a72740 1 -- start start 2024-09-11T23:39:49.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.514+0000 7f75f2a72740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a6da5e4a00 0x55a6da5e4de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:49.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.514+0000 7f75f2a72740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a6da5e5320 0x55a6da5ef6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:49.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.514+0000 7f75f2a72740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6da5efbe0 0x55a6da5f1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:49.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.514+0000 7f75f2a72740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55a6da549200 con 0x55a6da5efbe0 2024-09-11T23:39:49.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.514+0000 7f75f2a72740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55a6da4efdc0 con 0x55a6da5e4a00 2024-09-11T23:39:49.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.514+0000 7f75f2a72740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55a6da5451c0 con 0x55a6da5e5320 2024-09-11T23:39:49.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.514+0000 7f75eec36640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6da5efbe0 0x55a6da5f1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:49.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.514+0000 7f75eec36640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6da5efbe0 0x55a6da5f1fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59824/0 (socket says 172.21.15.26:59824) 2024-09-11T23:39:49.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.514+0000 7f75eec36640 1 -- 172.21.15.26:0/778850705 learned_addr learned my addr 172.21.15.26:0/778850705 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:39:49.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.514+0000 7f75ee435640 1 --2- 172.21.15.26:0/778850705 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a6da5e4a00 0x55a6da5e4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:49.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.514+0000 7f75edc34640 1 --2- 172.21.15.26:0/778850705 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a6da5e5320 0x55a6da5ef6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:49.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.515+0000 7f75eec36640 1 -- 172.21.15.26:0/778850705 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a6da5e5320 msgr2=0x55a6da5ef6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:39:49.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.515+0000 7f75eec36640 1 --2- 172.21.15.26:0/778850705 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a6da5e5320 0x55a6da5ef6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:49.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.515+0000 7f75eec36640 1 -- 172.21.15.26:0/778850705 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a6da5e4a00 msgr2=0x55a6da5e4de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:39:49.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.515+0000 7f75eec36640 1 --2- 172.21.15.26:0/778850705 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a6da5e4a00 0x55a6da5e4de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:49.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.515+0000 7f75eec36640 1 -- 172.21.15.26:0/778850705 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a6da2ba620 con 0x55a6da5efbe0 2024-09-11T23:39:49.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.515+0000 7f75eec36640 1 --2- 172.21.15.26:0/778850705 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6da5efbe0 0x55a6da5f1fd0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f75e0011850 tx=0x7f75e0011d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=823219dd45b9e1f9 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:39:49.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.515+0000 7f75dcff9640 1 -- 172.21.15.26:0/778850705 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75e0011330 con 0x55a6da5efbe0 2024-09-11T23:39:49.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.515+0000 7f75dcff9640 1 -- 172.21.15.26:0/778850705 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f75e000fe80 con 0x55a6da5efbe0 2024-09-11T23:39:49.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.515+0000 7f75dcff9640 1 -- 172.21.15.26:0/778850705 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75e00193e0 con 0x55a6da5efbe0 2024-09-11T23:39:49.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.516+0000 7f75f2a72740 1 -- 172.21.15.26:0/778850705 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6da5efbe0 msgr2=0x55a6da5f1fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:49.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.516+0000 7f75f2a72740 1 --2- 172.21.15.26:0/778850705 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6da5efbe0 0x55a6da5f1fd0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f75e0011850 tx=0x7f75e0011d10 comp rx=0 tx=0).stop 2024-09-11T23:39:49.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.516+0000 7f75f2a72740 1 -- 172.21.15.26:0/778850705 shutdown_connections 2024-09-11T23:39:49.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.516+0000 7f75f2a72740 1 --2- 172.21.15.26:0/778850705 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6da5efbe0 0x55a6da5f1fd0 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:49.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.516+0000 7f75f2a72740 1 --2- 172.21.15.26:0/778850705 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a6da5e5320 0x55a6da5ef6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:49.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.516+0000 7f75f2a72740 1 --2- 172.21.15.26:0/778850705 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a6da5e4a00 0x55a6da5e4de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:49.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.516+0000 7f75f2a72740 1 -- 172.21.15.26:0/778850705 >> 172.21.15.26:0/778850705 conn(0x55a6da5e0f90 msgr2=0x55a6da5e1370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:39:49.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.516+0000 7f75f2a72740 1 -- 172.21.15.26:0/778850705 shutdown_connections 2024-09-11T23:39:49.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.516+0000 7f75f2a72740 1 -- 172.21.15.26:0/778850705 wait complete. 2024-09-11T23:39:49.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.517+0000 7f75f2a72740 1 -- start start 2024-09-11T23:39:49.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.517+0000 7f75f2a72740 1 Processor -- start 2024-09-11T23:39:49.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.517+0000 7f75f2a72740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6da5e4a00 0x55a6da5ef3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:49.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.517+0000 7f75f2a72740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a6da5e5320 0x55a6da5ec6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:49.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.517+0000 7f75f2a72740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a6da5efbe0 0x55a6da5ecbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:49.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.517+0000 7f75f2a72740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55a6da5f27b0 con 0x55a6da5e4a00 2024-09-11T23:39:49.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.517+0000 7f75f2a72740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55a6da4e3c50 con 0x55a6da5e5320 2024-09-11T23:39:49.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.517+0000 7f75f2a72740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55a6da5f2930 con 0x55a6da5efbe0 2024-09-11T23:39:49.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.517+0000 7f75f2a72740 10 client.0 _ll_register_callbacks cb 0x55a6da4f5ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:39:49.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.517+0000 7f75ee435640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6da5e4a00 0x55a6da5ef3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:49.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.517+0000 7f75eec36640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a6da5efbe0 0x55a6da5ecbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:49.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.517+0000 7f75ee435640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6da5e4a00 0x55a6da5ef3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59836/0 (socket says 172.21.15.26:59836) 2024-09-11T23:39:49.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.517+0000 7f75ee435640 1 -- 172.21.15.26:0/2871831086 learned_addr learned my addr 172.21.15.26:0/2871831086 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:39:49.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.517+0000 7f75edc34640 1 --2- 172.21.15.26:0/2871831086 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a6da5e5320 0x55a6da5ec6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:49.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.518+0000 7f75ee435640 1 -- 172.21.15.26:0/2871831086 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a6da5efbe0 msgr2=0x55a6da5ecbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:39:49.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.518+0000 7f75ee435640 1 --2- 172.21.15.26:0/2871831086 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a6da5efbe0 0x55a6da5ecbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:49.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.518+0000 7f75ee435640 1 -- 172.21.15.26:0/2871831086 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a6da5e5320 msgr2=0x55a6da5ec6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:39:49.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.518+0000 7f75ee435640 1 --2- 172.21.15.26:0/2871831086 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a6da5e5320 0x55a6da5ec6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:49.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.518+0000 7f75ee435640 1 -- 172.21.15.26:0/2871831086 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a6da5ed410 con 0x55a6da5e4a00 2024-09-11T23:39:49.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.518+0000 7f75ee435640 1 --2- 172.21.15.26:0/2871831086 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6da5e4a00 0x55a6da5ef3f0 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f75e4003970 tx=0x7f75e4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=286ca8efc9dc107e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:39:49.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.518+0000 7f75d1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-11T23:39:49.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.518+0000 7f75d1ffb640 1 -- 172.21.15.26:0/2871831086 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75e400e820 con 0x55a6da5e4a00 2024-09-11T23:39:49.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.518+0000 7f75d1ffb640 1 -- 172.21.15.26:0/2871831086 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 8586+0+0 (secure 0 0 0) 0x7f75e400e9c0 con 0x55a6da5e4a00 2024-09-11T23:39:49.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.518+0000 7f75f2a72740 1 -- 172.21.15.26:0/2871831086 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a6da556770 con 0x55a6da5e4a00 2024-09-11T23:39:49.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.518+0000 7f75d1ffb640 1 -- 172.21.15.26:0/2871831086 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f75e400ebc0 con 0x55a6da5e4a00 2024-09-11T23:39:49.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.518+0000 7f75f2a72740 10 client.18153 fetch_fsmap learned FSMap version 92 2024-09-11T23:39:49.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.518+0000 7f75f2a72740 1 -- 172.21.15.26:0/2871831086 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=92,monmap=4+}) -- 0x55a6da606a80 con 0x55a6da5e4a00 2024-09-11T23:39:49.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.519+0000 7f75d1ffb640 1 -- 172.21.15.26:0/2871831086 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55a6da556770 con 0x55a6da5e4a00 2024-09-11T23:39:49.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.519+0000 7f75d1ffb640 1 -- 172.21.15.26:0/2871831086 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 92) ==== 8430+0+0 (secure 0 0 0) 0x7f75e402d030 con 0x55a6da5e4a00 2024-09-11T23:39:49.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.519+0000 7f75f2a72740 10 client.18153 fetch_fsmap finished waiting for FSMap version 92 2024-09-11T23:39:49.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.519+0000 7f75f2a72740 10 client.18153 resolve_mds: validated gid 24485 aka daemon mds.b 2024-09-11T23:39:49.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.519+0000 7f75f2a72740 20 client.18153 populate_metadata read hostname 'smithi026' 2024-09-11T23:39:49.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.519+0000 7f75f2a72740 1 --2- 172.21.15.26:0/2871831086 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55a6da608010 0x55a6da60a470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:49.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.519+0000 7f75f2a72740 4 client.18153 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:39:49.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.519+0000 7f75f2a72740 1 -- 172.21.15.26:0/2871831086 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a6da550c80 con 0x55a6da608010 2024-09-11T23:39:49.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.519+0000 7f75edc34640 1 --2- 172.21.15.26:0/2871831086 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55a6da608010 0x55a6da60a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:49.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.519+0000 7f75d1ffb640 10 client.18153 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:39:49.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.519+0000 7f75edc34640 1 --2- 172.21.15.26:0/2871831086 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55a6da608010 0x55a6da60a470 crc :-1 s=READY pgs=318 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e5fe0df57c986784 server_cookie=69609179e05d2f9a in_seq=0 out_seq=0 2024-09-11T23:39:49.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.520+0000 7f75d1ffb640 1 -- 172.21.15.26:0/2871831086 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a6da550c80 con 0x55a6da608010 2024-09-11T23:39:49.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.520+0000 7f75d1ffb640 10 client.18153 handle_command_reply: tid=0 2024-09-11T23:39:49.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.520+0000 7f75f2a72740 1 client.18153 shutdown 2024-09-11T23:39:49.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.520+0000 7f75f2a72740 1 -- 172.21.15.26:0/2871831086 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6da5e4a00 msgr2=0x55a6da5ef3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:49.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.520+0000 7f75f2a72740 1 --2- 172.21.15.26:0/2871831086 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6da5e4a00 0x55a6da5ef3f0 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f75e4003970 tx=0x7f75e4004730 comp rx=0 tx=0).stop 2024-09-11T23:39:49.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.520+0000 7f75f2a72740 1 -- 172.21.15.26:0/2871831086 shutdown_connections 2024-09-11T23:39:49.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.520+0000 7f75f2a72740 1 --2- 172.21.15.26:0/2871831086 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a6da5e5320 0x55a6da5ec6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:49.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.520+0000 7f75f2a72740 1 --2- 172.21.15.26:0/2871831086 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55a6da608010 0x55a6da60a470 crc :-1 s=READY pgs=318 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:49.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.520+0000 7f75f2a72740 1 --2- 172.21.15.26:0/2871831086 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a6da5efbe0 0x55a6da5ecbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:49.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.520+0000 7f75d1ffb640 0 client.18153 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:39:49.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.520+0000 7f75f2a72740 1 --2- 172.21.15.26:0/2871831086 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6da5e4a00 0x55a6da5ef3f0 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:49.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.520+0000 7f75f2a72740 1 -- 172.21.15.26:0/2871831086 >> 172.21.15.26:0/2871831086 conn(0x55a6da5e0f90 msgr2=0x55a6da5e01d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:39:49.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.521+0000 7f75f2a72740 1 -- 172.21.15.26:0/2871831086 shutdown_connections 2024-09-11T23:39:49.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.521+0000 7f75f2a72740 1 -- 172.21.15.26:0/2871831086 wait complete. 2024-09-11T23:39:49.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.521+0000 7f75f2a72740 20 client.18153 trim_cache size 0 max 16384 2024-09-11T23:39:49.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.540+0000 7f75f2a72740 1 Processor -- start 2024-09-11T23:39:49.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.540+0000 7f75f2a72740 1 -- start start 2024-09-11T23:39:49.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.540+0000 7f75f2a72740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a6da5e4a00 0x55a6da6042a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:49.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.540+0000 7f75f2a72740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6da5e5320 0x55a6da5e7be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:49.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.540+0000 7f75f2a72740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a6da5efbe0 0x55a6da60fd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:49.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.540+0000 7f75f2a72740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f75e81ab710 con 0x55a6da5e5320 2024-09-11T23:39:49.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.540+0000 7f75f2a72740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f75e81ad4c0 con 0x55a6da5efbe0 2024-09-11T23:39:49.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.540+0000 7f75f2a72740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f75e81abe10 con 0x55a6da5e4a00 2024-09-11T23:39:49.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.540+0000 7f75edc34640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6da5e5320 0x55a6da5e7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:49.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.540+0000 7f75edc34640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6da5e5320 0x55a6da5e7be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59848/0 (socket says 172.21.15.26:59848) 2024-09-11T23:39:49.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.540+0000 7f75edc34640 1 -- 172.21.15.26:0/1167948584 learned_addr learned my addr 172.21.15.26:0/1167948584 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:39:49.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.540+0000 7f75ee435640 1 --2- 172.21.15.26:0/1167948584 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a6da5e4a00 0x55a6da6042a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:49.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.540+0000 7f75eec36640 1 --2- 172.21.15.26:0/1167948584 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a6da5efbe0 0x55a6da60fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:49.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.541+0000 7f75edc34640 1 -- 172.21.15.26:0/1167948584 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a6da5e4a00 msgr2=0x55a6da6042a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:39:49.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.541+0000 7f75edc34640 1 --2- 172.21.15.26:0/1167948584 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a6da5e4a00 0x55a6da6042a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:49.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.541+0000 7f75edc34640 1 -- 172.21.15.26:0/1167948584 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a6da5efbe0 msgr2=0x55a6da60fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:39:49.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.541+0000 7f75edc34640 1 --2- 172.21.15.26:0/1167948584 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a6da5efbe0 0x55a6da60fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:49.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.541+0000 7f75edc34640 1 -- 172.21.15.26:0/1167948584 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a6da5e1a50 con 0x55a6da5e5320 2024-09-11T23:39:49.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.541+0000 7f75edc34640 1 --2- 172.21.15.26:0/1167948584 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6da5e5320 0x55a6da5e7be0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f75d8032c00 tx=0x7f75d803d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d038ede25233110c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:39:49.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.541+0000 7f75d3fff640 1 -- 172.21.15.26:0/1167948584 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75d8048040 con 0x55a6da5e5320 2024-09-11T23:39:49.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.541+0000 7f75d3fff640 1 -- 172.21.15.26:0/1167948584 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f75d80ba1d0 con 0x55a6da5e5320 2024-09-11T23:39:49.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.541+0000 7f75d3fff640 1 -- 172.21.15.26:0/1167948584 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75d80b86b0 con 0x55a6da5e5320 2024-09-11T23:39:49.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.542+0000 7f75f2a72740 1 -- 172.21.15.26:0/1167948584 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6da5e5320 msgr2=0x55a6da5e7be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:49.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.542+0000 7f75f2a72740 1 --2- 172.21.15.26:0/1167948584 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6da5e5320 0x55a6da5e7be0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f75d8032c00 tx=0x7f75d803d040 comp rx=0 tx=0).stop 2024-09-11T23:39:49.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.542+0000 7f75f2a72740 1 -- 172.21.15.26:0/1167948584 shutdown_connections 2024-09-11T23:39:49.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.542+0000 7f75f2a72740 1 --2- 172.21.15.26:0/1167948584 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a6da5efbe0 0x55a6da60fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:49.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.542+0000 7f75f2a72740 1 --2- 172.21.15.26:0/1167948584 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6da5e5320 0x55a6da5e7be0 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:49.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.542+0000 7f75f2a72740 1 --2- 172.21.15.26:0/1167948584 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a6da5e4a00 0x55a6da6042a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:49.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.542+0000 7f75f2a72740 1 -- 172.21.15.26:0/1167948584 >> 172.21.15.26:0/1167948584 conn(0x55a6da5e0f90 msgr2=0x55a6da5e74d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:39:49.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.542+0000 7f75f2a72740 1 -- 172.21.15.26:0/1167948584 shutdown_connections 2024-09-11T23:39:49.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.542+0000 7f75f2a72740 1 -- 172.21.15.26:0/1167948584 wait complete. 2024-09-11T23:39:49.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.543+0000 7f75f2a72740 1 -- start start 2024-09-11T23:39:49.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.543+0000 7f75f2a72740 1 Processor -- start 2024-09-11T23:39:49.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.543+0000 7f75f2a72740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6da5e4a00 0x55a6da5e9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:49.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.543+0000 7f75f2a72740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a6da5e5320 0x55a6da600410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:49.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.543+0000 7f75f2a72740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a6da5efbe0 0x55a6da600950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:49.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.543+0000 7f75edc34640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a6da5e5320 0x55a6da600410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:49.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.543+0000 7f75ee435640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6da5e4a00 0x55a6da5e9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:49.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.543+0000 7f75ee435640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6da5e4a00 0x55a6da5e9ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59852/0 (socket says 172.21.15.26:59852) 2024-09-11T23:39:49.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.543+0000 7f75edc34640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a6da5e5320 0x55a6da600410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:53876/0 (socket says 172.21.15.26:53876) 2024-09-11T23:39:49.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.544+0000 7f75eec36640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a6da5efbe0 0x55a6da600950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:49.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.544+0000 7f75f2a72740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f75e81acd30 con 0x55a6da5e4a00 2024-09-11T23:39:49.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.544+0000 7f75f2a72740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f75e81ac5a0 con 0x55a6da5efbe0 2024-09-11T23:39:49.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.544+0000 7f75f2a72740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f75e81a9710 con 0x55a6da5e5320 2024-09-11T23:39:49.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.544+0000 7f75f2a72740 10 client.0 _ll_register_callbacks cb 0x7f75e4004510 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:39:49.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.544+0000 7f75ee435640 1 -- 172.21.15.26:0/3407886688 learned_addr learned my addr 172.21.15.26:0/3407886688 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:39:49.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.544+0000 7f75eec36640 1 -- 172.21.15.26:0/3407886688 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a6da5e5320 msgr2=0x55a6da600410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:39:49.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.544+0000 7f75eec36640 1 --2- 172.21.15.26:0/3407886688 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a6da5e5320 0x55a6da600410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:49.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.544+0000 7f75eec36640 1 -- 172.21.15.26:0/3407886688 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6da5e4a00 msgr2=0x55a6da5e9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:39:49.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.544+0000 7f75eec36640 1 --2- 172.21.15.26:0/3407886688 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6da5e4a00 0x55a6da5e9ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:49.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.544+0000 7f75eec36640 1 -- 172.21.15.26:0/3407886688 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a6da600f40 con 0x55a6da5efbe0 2024-09-11T23:39:49.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.544+0000 7f75eec36640 1 --2- 172.21.15.26:0/3407886688 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a6da5efbe0 0x55a6da600950 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f75e00049d0 tx=0x7f75e0005aa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e61168c269a78dda server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:39:49.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.544+0000 7f75d1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-11T23:39:49.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.545+0000 7f75d1ffb640 1 -- 172.21.15.26:0/3407886688 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75e001f040 con 0x55a6da5efbe0 2024-09-11T23:39:49.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.545+0000 7f75d1ffb640 1 -- 172.21.15.26:0/3407886688 <== mon.2 v2:172.21.15.76:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 8586+0+0 (secure 0 0 0) 0x7f75e000fbb0 con 0x55a6da5efbe0 2024-09-11T23:39:49.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.545+0000 7f75f2a72740 1 -- 172.21.15.26:0/3407886688 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f75d80ba1d0 con 0x55a6da5efbe0 2024-09-11T23:39:49.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.545+0000 7f75d1ffb640 1 -- 172.21.15.26:0/3407886688 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f75e0011130 con 0x55a6da5efbe0 2024-09-11T23:39:49.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.545+0000 7f75f2a72740 10 client.27644 fetch_fsmap learned FSMap version 92 2024-09-11T23:39:49.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.545+0000 7f75f2a72740 1 -- 172.21.15.26:0/3407886688 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=92,monmap=4+}) -- 0x55a6da672d10 con 0x55a6da5efbe0 2024-09-11T23:39:49.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.545+0000 7f75d1ffb640 1 -- 172.21.15.26:0/3407886688 <== mon.2 v2:172.21.15.76:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f75e0010910 con 0x55a6da5efbe0 2024-09-11T23:39:49.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.545+0000 7f75d1ffb640 1 -- 172.21.15.26:0/3407886688 <== mon.2 v2:172.21.15.76:3300/0 5 ==== fsmap(e 92) ==== 8430+0+0 (secure 0 0 0) 0x7f75e0018070 con 0x55a6da5efbe0 2024-09-11T23:39:49.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.545+0000 7f75f2a72740 10 client.27644 fetch_fsmap finished waiting for FSMap version 92 2024-09-11T23:39:49.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.545+0000 7f75f2a72740 10 client.27644 resolve_mds: validated gid 24485 aka daemon mds.b 2024-09-11T23:39:49.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.545+0000 7f75f2a72740 20 client.27644 populate_metadata read hostname 'smithi026' 2024-09-11T23:39:49.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.545+0000 7f75f2a72740 1 --2- 172.21.15.26:0/3407886688 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55a6da608010 0x55a6da675350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:49.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.545+0000 7f75f2a72740 4 client.27644 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "quiesce db", "cancel": true, "all": true}] 2024-09-11T23:39:49.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.545+0000 7f75f2a72740 1 -- 172.21.15.26:0/3407886688 --> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] -- command(tid 0: {"prefix": "quiesce db", "cancel": true, "all": true}) -- 0x7f75d80ba1d0 con 0x55a6da608010 2024-09-11T23:39:49.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.546+0000 7f75ee435640 1 --2- 172.21.15.26:0/3407886688 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55a6da608010 0x55a6da675350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:49.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.546+0000 7f75d1ffb640 10 client.27644 ms_handle_connect on v2:172.21.15.76:6838/2077637090 2024-09-11T23:39:49.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.546+0000 7f75ee435640 1 --2- 172.21.15.26:0/3407886688 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55a6da608010 0x55a6da675350 crc :-1 s=READY pgs=319 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=217ed6551658193a server_cookie=343a1c01552195ce in_seq=0 out_seq=0 2024-09-11T23:39:49.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.546+0000 7f75d1ffb640 1 -- 172.21.15.26:0/3407886688 <== mds.0 v2:172.21.15.76:6838/2077637090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+80 (crc 0 0 0) 0x7f75d80ba1d0 con 0x55a6da608010 2024-09-11T23:39:49.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.546+0000 7f75d1ffb640 10 client.27644 handle_command_reply: tid=0 2024-09-11T23:39:49.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.547+0000 7f75f2a72740 1 client.27644 shutdown 2024-09-11T23:39:49.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.547+0000 7f75f2a72740 1 -- 172.21.15.26:0/3407886688 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a6da5efbe0 msgr2=0x55a6da600950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:49.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.547+0000 7f75f2a72740 1 --2- 172.21.15.26:0/3407886688 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a6da5efbe0 0x55a6da600950 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f75e00049d0 tx=0x7f75e0005aa0 comp rx=0 tx=0).stop 2024-09-11T23:39:49.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.547+0000 7f75f2a72740 1 -- 172.21.15.26:0/3407886688 shutdown_connections 2024-09-11T23:39:49.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.547+0000 7f75f2a72740 1 --2- 172.21.15.26:0/3407886688 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a6da5efbe0 0x55a6da600950 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:49.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.547+0000 7f75f2a72740 1 --2- 172.21.15.26:0/3407886688 >> [v2:172.21.15.76:6838/2077637090,v1:172.21.15.76:6839/2077637090] conn(0x55a6da608010 0x55a6da675350 crc :-1 s=READY pgs=319 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:49.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.547+0000 7f75f2a72740 1 --2- 172.21.15.26:0/3407886688 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a6da5e5320 0x55a6da600410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:49.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.547+0000 7f75f2a72740 1 --2- 172.21.15.26:0/3407886688 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6da5e4a00 0x55a6da5e9ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:49.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.547+0000 7f75f2a72740 1 -- 172.21.15.26:0/3407886688 >> 172.21.15.26:0/3407886688 conn(0x55a6da5e0f90 msgr2=0x55a6da5edf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:39:49.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.547+0000 7f75d1ffb640 0 client.27644 ms_handle_reset on v2:172.21.15.76:6838/2077637090 2024-09-11T23:39:49.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.547+0000 7f75f2a72740 1 -- 172.21.15.26:0/3407886688 shutdown_connections 2024-09-11T23:39:49.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.547+0000 7f75f2a72740 1 -- 172.21.15.26:0/3407886688 wait complete. 2024-09-11T23:39:49.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.548+0000 7f75f2a72740 20 client.27644 trim_cache size 0 max 16384 2024-09-11T23:39:49.553 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-11T23:39:49.553 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 92, 2024-09-11T23:39:49.553 INFO:teuthology.orchestra.run.smithi026.stdout: "leader": 24485, 2024-09-11T23:39:49.554 INFO:teuthology.orchestra.run.smithi026.stdout: "set_version": 72, 2024-09-11T23:39:49.554 INFO:teuthology.orchestra.run.smithi026.stdout: "sets": {} 2024-09-11T23:39:49.554 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-11T23:39:49.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.548+0000 7f75f0699640 1 -- 172.21.15.26:0/3659579331 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f75d40a2cf0 msgr2=0x7f75d40a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:49.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.548+0000 7f75f0699640 1 --2- 172.21.15.26:0/3659579331 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f75d40a2cf0 0x7f75d40a51b0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f75e4009bd0 tx=0x7f75e4005f90 comp rx=0 tx=0).stop 2024-09-11T23:39:49.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.548+0000 7f75f0699640 1 -- 172.21.15.26:0/3659579331 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f75e81036e0 msgr2=0x7f75e8198b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:49.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.548+0000 7f75f0699640 1 --2- 172.21.15.26:0/3659579331 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f75e81036e0 0x7f75e8198b80 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f75d802a940 tx=0x7f75d8031b50 comp rx=0 tx=0).stop 2024-09-11T23:39:49.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.549+0000 7f75f0699640 1 -- 172.21.15.26:0/3659579331 shutdown_connections 2024-09-11T23:39:49.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.549+0000 7f75f0699640 1 --2- 172.21.15.26:0/3659579331 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f75d40a2cf0 0x7f75d40a51b0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:49.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.549+0000 7f75f0699640 1 --2- 172.21.15.26:0/3659579331 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f75e81036e0 0x7f75e8198b80 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:49.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.549+0000 7f75f0699640 1 --2- 172.21.15.26:0/3659579331 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f75e81082a0 0x7f75e819cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:49.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.549+0000 7f75f0699640 1 --2- 172.21.15.26:0/3659579331 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f75e8102d30 0x7f75e8198640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:49.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.549+0000 7f75f0699640 1 -- 172.21.15.26:0/3659579331 >> 172.21.15.26:0/3659579331 conn(0x7f75e80fc7d0 msgr2=0x7f75e8106380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:39:49.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.549+0000 7f75f0699640 1 -- 172.21.15.26:0/3659579331 shutdown_connections 2024-09-11T23:39:49.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:49.549+0000 7f75f0699640 1 -- 172.21.15.26:0/3659579331 wait complete. 2024-09-11T23:39:50.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:39:50 smithi136 ceph-mon[35121]: pgmap v675: 97 pgs: 10 active+clean+snaptrim, 22 active+clean+snaptrim_wait, 65 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.7 MiB/s wr, 399 op/s 2024-09-11T23:39:50.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:39:50 smithi026 ceph-mon[31172]: pgmap v675: 97 pgs: 10 active+clean+snaptrim, 22 active+clean+snaptrim_wait, 65 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.7 MiB/s wr, 399 op/s 2024-09-11T23:39:50.552 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:39:50 smithi076 ceph-mon[35296]: pgmap v675: 97 pgs: 10 active+clean+snaptrim, 22 active+clean+snaptrim_wait, 65 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.7 MiB/s wr, 399 op/s 2024-09-11T23:39:50.941 INFO:tasks.quiescer:done joining Quiescers 2024-09-11T23:39:50.941 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-09-11T23:39:50.951 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi026.front.sepia.ceph.com 2024-09-11T23:39:50.961 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-09-11T23:39:50.998 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-09-11T23:39:50.998 INFO:teuthology.task.exec:Executing custom commands... 2024-09-11T23:39:50.998 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi026.front.sepia.ceph.com 2024-09-11T23:39:50.998 DEBUG:teuthology.orchestra.run.smithi026:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-09-11T23:39:51.033 DEBUG:teuthology.orchestra.run.smithi026:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-09-11T23:39:51.105 DEBUG:teuthology.orchestra.run.smithi026:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-09-11T23:39:51.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:51.255+0000 7f7b04b31640 1 -- 172.21.15.26:0/394234722 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b001035e0 msgr2=0x7f7b0010bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:51.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:51.255+0000 7f7b04b31640 1 --2- 172.21.15.26:0/394234722 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b001035e0 0x7f7b0010bac0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f7af4009900 tx=0x7f7af402a480 comp rx=0 tx=0).stop 2024-09-11T23:39:51.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:51.256+0000 7f7b04b31640 1 -- 172.21.15.26:0/394234722 shutdown_connections 2024-09-11T23:39:51.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:51.256+0000 7f7b04b31640 1 --2- 172.21.15.26:0/394234722 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7b0010c110 0x7f7b0010e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:51.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:51.256+0000 7f7b04b31640 1 --2- 172.21.15.26:0/394234722 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b001035e0 0x7f7b0010bac0 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:51.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:51.256+0000 7f7b04b31640 1 --2- 172.21.15.26:0/394234722 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7b00102cc0 0x7f7b001030a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:51.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:51.256+0000 7f7b04b31640 1 -- 172.21.15.26:0/394234722 >> 172.21.15.26:0/394234722 conn(0x7f7b000fc6b0 msgr2=0x7f7b000fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:39:51.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:51.256+0000 7f7b04b31640 1 -- 172.21.15.26:0/394234722 shutdown_connections 2024-09-11T23:39:51.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:51.256+0000 7f7b04b31640 1 -- 172.21.15.26:0/394234722 wait complete. 2024-09-11T23:39:51.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:51.257+0000 7f7b04b31640 1 Processor -- start 2024-09-11T23:39:51.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:51.257+0000 7f7b04b31640 1 -- start start 2024-09-11T23:39:51.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:51.257+0000 7f7b04b31640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7b00102cc0 0x7f7b001983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:51.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:51.257+0000 7f7b04b31640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b001035e0 0x7f7b00198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:51.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:51.257+0000 7f7b04b31640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7b0010c110 0x7f7b0019cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:51.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:51.257+0000 7f7b04b31640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f7b00111ef0 con 0x7f7b001035e0 2024-09-11T23:39:51.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:51.257+0000 7f7b04b31640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f7b00111d70 con 0x7f7b00102cc0 2024-09-11T23:39:51.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:51.257+0000 7f7b04b31640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f7b00112070 con 0x7f7b0010c110 2024-09-11T23:39:51.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:51.257+0000 7f7afdd9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b001035e0 0x7f7b00198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:51.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:51.257+0000 7f7afdd9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b001035e0 0x7f7b00198920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39474/0 (socket says 172.21.15.26:39474) 2024-09-11T23:39:51.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:51.257+0000 7f7afdd9b640 1 -- 172.21.15.26:0/2918973041 learned_addr learned my addr 172.21.15.26:0/2918973041 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:39:51.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:51.257+0000 7f7afed9d640 1 --2- 172.21.15.26:0/2918973041 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7b0010c110 0x7f7b0019cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:51.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:51.257+0000 7f7afe59c640 1 --2- 172.21.15.26:0/2918973041 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7b00102cc0 0x7f7b001983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:51.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:51.257+0000 7f7afdd9b640 1 -- 172.21.15.26:0/2918973041 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7b0010c110 msgr2=0x7f7b0019cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:51.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:51.257+0000 7f7afdd9b640 1 --2- 172.21.15.26:0/2918973041 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7b0010c110 0x7f7b0019cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:51.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:51.257+0000 7f7afdd9b640 1 -- 172.21.15.26:0/2918973041 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7b00102cc0 msgr2=0x7f7b001983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:51.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:51.257+0000 7f7afdd9b640 1 --2- 172.21.15.26:0/2918973041 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7b00102cc0 0x7f7b001983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:51.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:51.257+0000 7f7afdd9b640 1 -- 172.21.15.26:0/2918973041 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b0019d3d0 con 0x7f7b001035e0 2024-09-11T23:39:51.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:51.257+0000 7f7afdd9b640 1 --2- 172.21.15.26:0/2918973041 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b001035e0 0x7f7b00198920 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f7af4006010 tx=0x7f7af4004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:39:51.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:51.258+0000 7f7aef7fe640 1 -- 172.21.15.26:0/2918973041 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7af403f030 con 0x7f7b001035e0 2024-09-11T23:39:51.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:51.258+0000 7f7aef7fe640 1 -- 172.21.15.26:0/2918973041 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7af4004440 con 0x7f7b001035e0 2024-09-11T23:39:51.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:51.258+0000 7f7b04b31640 1 -- 172.21.15.26:0/2918973041 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b0019d660 con 0x7f7b001035e0 2024-09-11T23:39:51.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:51.258+0000 7f7aef7fe640 1 -- 172.21.15.26:0/2918973041 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7af4035070 con 0x7f7b001035e0 2024-09-11T23:39:51.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:51.258+0000 7f7b04b31640 1 -- 172.21.15.26:0/2918973041 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b001a4f50 con 0x7f7b001035e0 2024-09-11T23:39:51.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:51.259+0000 7f7b04b31640 1 -- 172.21.15.26:0/2918973041 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b001030a0 con 0x7f7b001035e0 2024-09-11T23:39:51.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:51.260+0000 7f7aef7fe640 1 -- 172.21.15.26:0/2918973041 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f7af4042030 con 0x7f7b001035e0 2024-09-11T23:39:51.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:51.261+0000 7f7aef7fe640 1 --2- 172.21.15.26:0/2918973041 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f7ae80a2e20 0x7f7ae80a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:51.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:51.261+0000 7f7afe59c640 1 --2- 172.21.15.26:0/2918973041 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f7ae80a2e20 0x7f7ae80a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:51.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:51.261+0000 7f7aef7fe640 1 -- 172.21.15.26:0/2918973041 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 8586+0+0 (secure 0 0 0) 0x7f7af40ef950 con 0x7f7b001035e0 2024-09-11T23:39:51.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:51.261+0000 7f7afe59c640 1 --2- 172.21.15.26:0/2918973041 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f7ae80a2e20 0x7f7ae80a52e0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f7af000bbe0 tx=0x7f7af000b390 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:39:51.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:51.264+0000 7f7aef7fe640 1 -- 172.21.15.26:0/2918973041 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7af40bbfa0 con 0x7f7b001035e0 2024-09-11T23:39:51.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:51.460+0000 7f7b04b31640 1 -- 172.21.15.26:0/2918973041 --> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f7b00062630 con 0x7f7ae80a2e20 2024-09-11T23:39:51.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:51.469+0000 7f7aef7fe640 1 -- 172.21.15.26:0/2918973041 <== mgr.15171 v2:172.21.15.26:6800/1989555790 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+351 (secure 0 0 0) 0x7f7b00062630 con 0x7f7ae80a2e20 2024-09-11T23:39:51.470 INFO:teuthology.orchestra.run.smithi026.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-09-11T00:00:00", "created": "2024-09-11T23:18:30", "first": "2024-09-11T23:19:00", "last": "2024-09-11T23:39:01", "last_pruned": "2024-09-11T23:39:01", "created_count": 21, "pruned_count": 15, "active": true} 2024-09-11T23:39:51.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:51.471+0000 7f7b04b31640 1 -- 172.21.15.26:0/2918973041 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f7ae80a2e20 msgr2=0x7f7ae80a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:51.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:51.471+0000 7f7b04b31640 1 --2- 172.21.15.26:0/2918973041 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f7ae80a2e20 0x7f7ae80a52e0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f7af000bbe0 tx=0x7f7af000b390 comp rx=0 tx=0).stop 2024-09-11T23:39:51.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:51.471+0000 7f7b04b31640 1 -- 172.21.15.26:0/2918973041 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b001035e0 msgr2=0x7f7b00198920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:51.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:51.471+0000 7f7b04b31640 1 --2- 172.21.15.26:0/2918973041 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b001035e0 0x7f7b00198920 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f7af4006010 tx=0x7f7af4004040 comp rx=0 tx=0).stop 2024-09-11T23:39:51.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:51.471+0000 7f7b04b31640 1 -- 172.21.15.26:0/2918973041 shutdown_connections 2024-09-11T23:39:51.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:51.471+0000 7f7b04b31640 1 --2- 172.21.15.26:0/2918973041 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f7ae80a2e20 0x7f7ae80a52e0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:51.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:51.471+0000 7f7b04b31640 1 --2- 172.21.15.26:0/2918973041 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7b0010c110 0x7f7b0019cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:51.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:51.471+0000 7f7b04b31640 1 --2- 172.21.15.26:0/2918973041 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b001035e0 0x7f7b00198920 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:51.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:51.471+0000 7f7b04b31640 1 --2- 172.21.15.26:0/2918973041 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7b00102cc0 0x7f7b001983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:51.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:51.471+0000 7f7b04b31640 1 -- 172.21.15.26:0/2918973041 >> 172.21.15.26:0/2918973041 conn(0x7f7b000fc6b0 msgr2=0x7f7b000fd310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:39:51.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:51.472+0000 7f7b04b31640 1 -- 172.21.15.26:0/2918973041 shutdown_connections 2024-09-11T23:39:51.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:51.472+0000 7f7b04b31640 1 -- 172.21.15.26:0/2918973041 wait complete. 2024-09-11T23:39:51.483 DEBUG:teuthology.orchestra.run.smithi026:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-09-11T23:39:51.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:51.587+0000 7fe0b6311640 1 -- 172.21.15.26:0/2019925315 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe0b0107b80 msgr2=0x7fe0b0107f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:51.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:51.587+0000 7fe0b6311640 1 --2- 172.21.15.26:0/2019925315 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe0b0107b80 0x7fe0b0107f60 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7fe0a0009930 tx=0x7fe0a002a420 comp rx=0 tx=0).stop 2024-09-11T23:39:51.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:51.587+0000 7fe0b6311640 1 -- 172.21.15.26:0/2019925315 shutdown_connections 2024-09-11T23:39:51.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:51.587+0000 7fe0b6311640 1 --2- 172.21.15.26:0/2019925315 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe0b0101ae0 0x7fe0b0101f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:51.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:51.587+0000 7fe0b6311640 1 --2- 172.21.15.26:0/2019925315 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe0b0102480 0x7fe0b010e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:51.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:51.587+0000 7fe0b6311640 1 --2- 172.21.15.26:0/2019925315 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe0b0107b80 0x7fe0b0107f60 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:51.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:51.587+0000 7fe0b6311640 1 -- 172.21.15.26:0/2019925315 >> 172.21.15.26:0/2019925315 conn(0x7fe0b00fd7e0 msgr2=0x7fe0b00ffc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:39:51.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:51.588+0000 7fe0b6311640 1 -- 172.21.15.26:0/2019925315 shutdown_connections 2024-09-11T23:39:51.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:51.588+0000 7fe0b6311640 1 -- 172.21.15.26:0/2019925315 wait complete. 2024-09-11T23:39:51.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:51.588+0000 7fe0b6311640 1 Processor -- start 2024-09-11T23:39:51.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:51.588+0000 7fe0b6311640 1 -- start start 2024-09-11T23:39:51.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:51.589+0000 7fe0b6311640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe0b0101ae0 0x7fe0b0198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:51.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:51.589+0000 7fe0b6311640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe0b0102480 0x7fe0b01988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:51.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:51.589+0000 7fe0b6311640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe0b0107b80 0x7fe0b019cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:51.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:51.589+0000 7fe0b6311640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fe0b0111f30 con 0x7fe0b0107b80 2024-09-11T23:39:51.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:51.589+0000 7fe0b6311640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fe0b0111db0 con 0x7fe0b0102480 2024-09-11T23:39:51.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:51.589+0000 7fe0b6311640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe0b01120b0 con 0x7fe0b0101ae0 2024-09-11T23:39:51.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:51.589+0000 7fe0b48ae640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe0b0107b80 0x7fe0b019cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:51.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:51.589+0000 7fe0b48ae640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe0b0107b80 0x7fe0b019cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39492/0 (socket says 172.21.15.26:39492) 2024-09-11T23:39:51.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:51.589+0000 7fe0b48ae640 1 -- 172.21.15.26:0/1872615347 learned_addr learned my addr 172.21.15.26:0/1872615347 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:39:51.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:51.589+0000 7fe0affff640 1 --2- 172.21.15.26:0/1872615347 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe0b0101ae0 0x7fe0b0198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:51.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:51.589+0000 7fe0af7fe640 1 --2- 172.21.15.26:0/1872615347 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe0b0102480 0x7fe0b01988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:51.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:51.589+0000 7fe0b48ae640 1 -- 172.21.15.26:0/1872615347 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe0b0101ae0 msgr2=0x7fe0b0198380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:51.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:51.589+0000 7fe0b48ae640 1 --2- 172.21.15.26:0/1872615347 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe0b0101ae0 0x7fe0b0198380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:51.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:51.589+0000 7fe0b48ae640 1 -- 172.21.15.26:0/1872615347 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe0b0102480 msgr2=0x7fe0b01988c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:51.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:51.589+0000 7fe0b48ae640 1 --2- 172.21.15.26:0/1872615347 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe0b0102480 0x7fe0b01988c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:51.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:51.589+0000 7fe0b48ae640 1 -- 172.21.15.26:0/1872615347 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0b019d370 con 0x7fe0b0107b80 2024-09-11T23:39:51.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:51.589+0000 7fe0b48ae640 1 --2- 172.21.15.26:0/1872615347 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe0b0107b80 0x7fe0b019cc40 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7fe0a800eb10 tx=0x7fe0a800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:39:51.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:51.589+0000 7fe0ad7fa640 1 -- 172.21.15.26:0/1872615347 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0a8004b20 con 0x7fe0b0107b80 2024-09-11T23:39:51.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:51.589+0000 7fe0ad7fa640 1 -- 172.21.15.26:0/1872615347 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe0a8004cc0 con 0x7fe0b0107b80 2024-09-11T23:39:51.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:51.589+0000 7fe0b6311640 1 -- 172.21.15.26:0/1872615347 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0b019d660 con 0x7fe0b0107b80 2024-09-11T23:39:51.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:51.589+0000 7fe0ad7fa640 1 -- 172.21.15.26:0/1872615347 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0a8010760 con 0x7fe0b0107b80 2024-09-11T23:39:51.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:51.589+0000 7fe0b6311640 1 -- 172.21.15.26:0/1872615347 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0b01a4ef0 con 0x7fe0b0107b80 2024-09-11T23:39:51.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:51.591+0000 7fe0b6311640 1 -- 172.21.15.26:0/1872615347 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0b0101f40 con 0x7fe0b0107b80 2024-09-11T23:39:51.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:51.591+0000 7fe0ad7fa640 1 -- 172.21.15.26:0/1872615347 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fe0a8005020 con 0x7fe0b0107b80 2024-09-11T23:39:51.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:51.592+0000 7fe0ad7fa640 1 --2- 172.21.15.26:0/1872615347 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fe0980a2da0 0x7fe0980a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:51.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:51.592+0000 7fe0affff640 1 --2- 172.21.15.26:0/1872615347 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fe0980a2da0 0x7fe0980a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:51.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:51.592+0000 7fe0ad7fa640 1 -- 172.21.15.26:0/1872615347 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 8586+0+0 (secure 0 0 0) 0x7fe0a80cfac0 con 0x7fe0b0107b80 2024-09-11T23:39:51.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:51.593+0000 7fe0affff640 1 --2- 172.21.15.26:0/1872615347 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fe0980a2da0 0x7fe0980a5260 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fe0a0009900 tx=0x7fe0a00058d0 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:39:51.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:51.596+0000 7fe0ad7fa640 1 -- 172.21.15.26:0/1872615347 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe0a809c6a0 con 0x7fe0b0107b80 2024-09-11T23:39:51.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:51.796+0000 7fe0b6311640 1 -- 172.21.15.26:0/1872615347 --> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7fe0b00fe5e0 con 0x7fe0980a2da0 2024-09-11T23:39:51.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:51.804+0000 7fe0ad7fa640 1 -- 172.21.15.26:0/1872615347 <== mgr.15171 v2:172.21.15.26:6800/1989555790 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7fe0b00fe5e0 con 0x7fe0980a2da0 2024-09-11T23:39:51.805 INFO:teuthology.orchestra.run.smithi026.stdout:/ 1m 6m3h 2024-09-11T23:39:51.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:51.806+0000 7fe0b6311640 1 -- 172.21.15.26:0/1872615347 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fe0980a2da0 msgr2=0x7fe0980a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:51.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:51.806+0000 7fe0b6311640 1 --2- 172.21.15.26:0/1872615347 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fe0980a2da0 0x7fe0980a5260 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fe0a0009900 tx=0x7fe0a00058d0 comp rx=0 tx=0).stop 2024-09-11T23:39:51.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:51.806+0000 7fe0b6311640 1 -- 172.21.15.26:0/1872615347 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe0b0107b80 msgr2=0x7fe0b019cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:51.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:51.806+0000 7fe0b6311640 1 --2- 172.21.15.26:0/1872615347 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe0b0107b80 0x7fe0b019cc40 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7fe0a800eb10 tx=0x7fe0a800c2d0 comp rx=0 tx=0).stop 2024-09-11T23:39:51.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:51.806+0000 7fe0b6311640 1 -- 172.21.15.26:0/1872615347 shutdown_connections 2024-09-11T23:39:51.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:51.806+0000 7fe0b6311640 1 --2- 172.21.15.26:0/1872615347 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fe0980a2da0 0x7fe0980a5260 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:51.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:51.806+0000 7fe0b6311640 1 --2- 172.21.15.26:0/1872615347 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe0b0102480 0x7fe0b01988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:51.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:51.806+0000 7fe0b6311640 1 --2- 172.21.15.26:0/1872615347 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe0b0107b80 0x7fe0b019cc40 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:51.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:51.806+0000 7fe0b6311640 1 --2- 172.21.15.26:0/1872615347 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe0b0101ae0 0x7fe0b0198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:51.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:51.806+0000 7fe0b6311640 1 -- 172.21.15.26:0/1872615347 >> 172.21.15.26:0/1872615347 conn(0x7fe0b00fd7e0 msgr2=0x7fe0b00fded0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:39:51.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:51.806+0000 7fe0b6311640 1 -- 172.21.15.26:0/1872615347 shutdown_connections 2024-09-11T23:39:51.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:51.806+0000 7fe0b6311640 1 -- 172.21.15.26:0/1872615347 wait complete. 2024-09-11T23:39:51.817 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-09-11T23:39:51.828 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-09-11T23:39:51.828 ERROR:teuthology.run_tasks:Manager failed: fwd_scrub Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/run_tasks.py", line 160, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.10/contextlib.py", line 142, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph_086c737fa75c22d521a3932c59b31c5e1d1c9ef4/qa/tasks/fwd_scrub.py", line 155, in task stop_all_fwd_scrubbers(ctx.ceph[config['cluster']].thrashers) File "/home/teuthworker/src/git.ceph.com_ceph_086c737fa75c22d521a3932c59b31c5e1d1c9ef4/qa/tasks/fwd_scrub.py", line 90, in stop_all_fwd_scrubbers raise RuntimeError(f"error during scrub thrashing: {thrasher.exception}") RuntimeError: error during scrub thrashing: rank damage found: {'backtrace'} 2024-09-11T23:39:51.829 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-09-11T23:39:51.839 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-11T23:39:51.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:51.943+0000 7fc5c5fe4640 1 -- 172.21.15.26:0/1223767574 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc5c0102430 msgr2=0x7fc5c010e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:51.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:51.943+0000 7fc5c5fe4640 1 --2- 172.21.15.26:0/1223767574 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc5c0102430 0x7fc5c010e960 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7fc5b8009cd0 tx=0x7fc5b802a8e0 comp rx=0 tx=0).stop 2024-09-11T23:39:51.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:51.943+0000 7fc5c5fe4640 1 -- 172.21.15.26:0/1223767574 shutdown_connections 2024-09-11T23:39:51.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:51.943+0000 7fc5c5fe4640 1 --2- 172.21.15.26:0/1223767574 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc5c0102430 0x7fc5c010e960 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:51.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:51.943+0000 7fc5c5fe4640 1 --2- 172.21.15.26:0/1223767574 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc5c0101a90 0x7fc5c0101ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:51.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:51.943+0000 7fc5c5fe4640 1 --2- 172.21.15.26:0/1223767574 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc5c0107aa0 0x7fc5c0107e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:51.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:51.943+0000 7fc5c5fe4640 1 -- 172.21.15.26:0/1223767574 >> 172.21.15.26:0/1223767574 conn(0x7fc5c00fd7d0 msgr2=0x7fc5c00ffbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:39:51.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:51.943+0000 7fc5c5fe4640 1 -- 172.21.15.26:0/1223767574 shutdown_connections 2024-09-11T23:39:51.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:51.943+0000 7fc5c5fe4640 1 -- 172.21.15.26:0/1223767574 wait complete. 2024-09-11T23:39:51.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:51.943+0000 7fc5c5fe4640 1 Processor -- start 2024-09-11T23:39:51.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:51.944+0000 7fc5c5fe4640 1 -- start start 2024-09-11T23:39:51.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:51.944+0000 7fc5c5fe4640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc5c0101a90 0x7fc5c01982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:51.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:51.944+0000 7fc5c5fe4640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc5c0102430 0x7fc5c0198830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:51.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:51.944+0000 7fc5c5fe4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc5c0107aa0 0x7fc5c019cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:51.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:51.944+0000 7fc5c5fe4640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc5c0111ee0 con 0x7fc5c0107aa0 2024-09-11T23:39:51.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:51.944+0000 7fc5c5fe4640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fc5c0111d60 con 0x7fc5c0102430 2024-09-11T23:39:51.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:51.944+0000 7fc5c5fe4640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fc5c0112060 con 0x7fc5c0101a90 2024-09-11T23:39:51.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:51.944+0000 7fc5bffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc5c0107aa0 0x7fc5c019cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:51.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:51.944+0000 7fc5bffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc5c0107aa0 0x7fc5c019cbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39520/0 (socket says 172.21.15.26:39520) 2024-09-11T23:39:51.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:51.944+0000 7fc5bffff640 1 -- 172.21.15.26:0/723856560 learned_addr learned my addr 172.21.15.26:0/723856560 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:39:51.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:51.944+0000 7fc5beffd640 1 --2- 172.21.15.26:0/723856560 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc5c0102430 0x7fc5c0198830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:51.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:51.944+0000 7fc5bf7fe640 1 --2- 172.21.15.26:0/723856560 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc5c0101a90 0x7fc5c01982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:51.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:51.944+0000 7fc5bffff640 1 -- 172.21.15.26:0/723856560 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc5c0101a90 msgr2=0x7fc5c01982f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:51.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:51.944+0000 7fc5bffff640 1 --2- 172.21.15.26:0/723856560 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc5c0101a90 0x7fc5c01982f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:51.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:51.944+0000 7fc5bffff640 1 -- 172.21.15.26:0/723856560 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc5c0102430 msgr2=0x7fc5c0198830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:51.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:51.944+0000 7fc5bffff640 1 --2- 172.21.15.26:0/723856560 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc5c0102430 0x7fc5c0198830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:51.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:51.944+0000 7fc5bffff640 1 -- 172.21.15.26:0/723856560 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5c019d2e0 con 0x7fc5c0107aa0 2024-09-11T23:39:51.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:51.945+0000 7fc5bffff640 1 --2- 172.21.15.26:0/723856560 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc5c0107aa0 0x7fc5c019cbb0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7fc5b8005cc0 tx=0x7fc5b8003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:39:51.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:51.945+0000 7fc5bcff9640 1 -- 172.21.15.26:0/723856560 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5b8004040 con 0x7fc5c0107aa0 2024-09-11T23:39:51.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:51.945+0000 7fc5bcff9640 1 -- 172.21.15.26:0/723856560 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5b80046c0 con 0x7fc5c0107aa0 2024-09-11T23:39:51.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:51.945+0000 7fc5c5fe4640 1 -- 172.21.15.26:0/723856560 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5c019d570 con 0x7fc5c0107aa0 2024-09-11T23:39:51.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:51.945+0000 7fc5bcff9640 1 -- 172.21.15.26:0/723856560 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5b8030e70 con 0x7fc5c0107aa0 2024-09-11T23:39:51.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:51.945+0000 7fc5c5fe4640 1 -- 172.21.15.26:0/723856560 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5c01a4e60 con 0x7fc5c0107aa0 2024-09-11T23:39:51.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:51.946+0000 7fc5c5fe4640 1 -- 172.21.15.26:0/723856560 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5c0101ef0 con 0x7fc5c0107aa0 2024-09-11T23:39:51.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:51.947+0000 7fc5bcff9640 1 -- 172.21.15.26:0/723856560 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fc5b80305a0 con 0x7fc5c0107aa0 2024-09-11T23:39:51.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:51.948+0000 7fc5bcff9640 1 --2- 172.21.15.26:0/723856560 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fc5a80a2d00 0x7fc5a80a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:51.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:51.948+0000 7fc5bf7fe640 1 --2- 172.21.15.26:0/723856560 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fc5a80a2d00 0x7fc5a80a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:51.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:51.948+0000 7fc5bcff9640 1 -- 172.21.15.26:0/723856560 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 8586+0+0 (secure 0 0 0) 0x7fc5b802e030 con 0x7fc5c0107aa0 2024-09-11T23:39:51.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:51.948+0000 7fc5bf7fe640 1 --2- 172.21.15.26:0/723856560 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fc5a80a2d00 0x7fc5a80a51c0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fc5b000ab60 tx=0x7fc5b000a2d0 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:39:51.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:51.951+0000 7fc5bcff9640 1 -- 172.21.15.26:0/723856560 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc5b80bd640 con 0x7fc5c0107aa0 2024-09-11T23:39:52.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:52.132+0000 7fc5c5fe4640 1 -- 172.21.15.26:0/723856560 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc5c01a5090 con 0x7fc5c0107aa0 2024-09-11T23:39:52.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:52.133+0000 7fc5bcff9640 1 -- 172.21.15.26:0/723856560 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v115) ==== 74+0+23271 (secure 0 0 0) 0x7fc5b80098b0 con 0x7fc5c0107aa0 2024-09-11T23:39:52.134 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-11T23:39:52.134 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":115,"fsid":"af7e6fc8-7092-11ef-bcea-c7b262605968","created":"2024-09-11T23:08:40.486438+0000","modified":"2024-09-11T23:39:03.903267+0000","last_up_change":"2024-09-11T23:15:35.179446+0000","last_in_change":"2024-09-11T23:15:11.064627+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T23:11:50.494160+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-11T23:16:22.503928+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-11T23:16:24.876554+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"115","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":37,"snap_epoch":115,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a4d62fcf-c31e-41b1-932c-3d87ff640531","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":2014857905},{"type":"v1","addr":"172.21.15.26:6803","nonce":2014857905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":2014857905},{"type":"v1","addr":"172.21.15.26:6805","nonce":2014857905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":2014857905},{"type":"v1","addr":"172.21.15.26:6809","nonce":2014857905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":2014857905},{"type":"v1","addr":"172.21.15.26:6807","nonce":2014857905}]},"public_addr":"172.21.15.26:6803/2014857905","cluster_addr":"172.21.15.26:6805/2014857905","heartbeat_back_addr":"172.21.15.26:6809/2014857905","heartbeat_front_addr":"172.21.15.26:6807/2014857905","state":["exists","up"]},{"osd":1,"uuid":"e0d69112-3c0a-4fc5-8783-e41b4081e0c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":333383087},{"type":"v1","addr":"172.21.15.76:6801","nonce":333383087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":333383087},{"type":"v1","addr":"172.21.15.76:6803","nonce":333383087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":333383087},{"type":"v1","addr":"172.21.15.76:6807","nonce":333383087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":333383087},{"type":"v1","addr":"172.21.15.76:6805","nonce":333383087}]},"public_addr":"172.21.15.76:6801/333383087","cluster_addr":"172.21.15.76:6803/333383087","heartbeat_back_addr":"172.21.15.76:6807/333383087","heartbeat_front_addr":"172.21.15.76:6805/333383087","state":["exists","up"]},{"osd":2,"uuid":"d578429e-bf23-4a1e-890f-7172955c0eb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":1271538911},{"type":"v1","addr":"172.21.15.136:6801","nonce":1271538911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":1271538911},{"type":"v1","addr":"172.21.15.136:6803","nonce":1271538911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":1271538911},{"type":"v1","addr":"172.21.15.136:6807","nonce":1271538911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":1271538911},{"type":"v1","addr":"172.21.15.136:6805","nonce":1271538911}]},"public_addr":"172.21.15.136:6801/1271538911","cluster_addr":"172.21.15.136:6803/1271538911","heartbeat_back_addr":"172.21.15.136:6807/1271538911","heartbeat_front_addr":"172.21.15.136:6805/1271538911","state":["exists","up"]},{"osd":3,"uuid":"3e34623f-0ea5-4b10-b8a9-f8756c35d512","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":1740304151},{"type":"v1","addr":"172.21.15.26:6811","nonce":1740304151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":1740304151},{"type":"v1","addr":"172.21.15.26:6813","nonce":1740304151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1740304151},{"type":"v1","addr":"172.21.15.26:6817","nonce":1740304151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":1740304151},{"type":"v1","addr":"172.21.15.26:6815","nonce":1740304151}]},"public_addr":"172.21.15.26:6811/1740304151","cluster_addr":"172.21.15.26:6813/1740304151","heartbeat_back_addr":"172.21.15.26:6817/1740304151","heartbeat_front_addr":"172.21.15.26:6815/1740304151","state":["exists","up"]},{"osd":4,"uuid":"77607929-f9ed-49a9-abb5-48cd88eec4ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":2702262182},{"type":"v1","addr":"172.21.15.76:6809","nonce":2702262182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":2702262182},{"type":"v1","addr":"172.21.15.76:6811","nonce":2702262182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":2702262182},{"type":"v1","addr":"172.21.15.76:6815","nonce":2702262182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":2702262182},{"type":"v1","addr":"172.21.15.76:6813","nonce":2702262182}]},"public_addr":"172.21.15.76:6809/2702262182","cluster_addr":"172.21.15.76:6811/2702262182","heartbeat_back_addr":"172.21.15.76:6815/2702262182","heartbeat_front_addr":"172.21.15.76:6813/2702262182","state":["exists","up"]},{"osd":5,"uuid":"29e20e37-09a0-4924-8aaa-17939763a7c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":1749229277},{"type":"v1","addr":"172.21.15.136:6809","nonce":1749229277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":1749229277},{"type":"v1","addr":"172.21.15.136:6811","nonce":1749229277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":1749229277},{"type":"v1","addr":"172.21.15.136:6815","nonce":1749229277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":1749229277},{"type":"v1","addr":"172.21.15.136:6813","nonce":1749229277}]},"public_addr":"172.21.15.136:6809/1749229277","cluster_addr":"172.21.15.136:6811/1749229277","heartbeat_back_addr":"172.21.15.136:6815/1749229277","heartbeat_front_addr":"172.21.15.136:6813/1749229277","state":["exists","up"]},{"osd":6,"uuid":"e0ddd1c0-1c46-41e6-b2f4-396bdcd2bf8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1928245624},{"type":"v1","addr":"172.21.15.26:6819","nonce":1928245624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1928245624},{"type":"v1","addr":"172.21.15.26:6821","nonce":1928245624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":1928245624},{"type":"v1","addr":"172.21.15.26:6825","nonce":1928245624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1928245624},{"type":"v1","addr":"172.21.15.26:6823","nonce":1928245624}]},"public_addr":"172.21.15.26:6819/1928245624","cluster_addr":"172.21.15.26:6821/1928245624","heartbeat_back_addr":"172.21.15.26:6825/1928245624","heartbeat_front_addr":"172.21.15.26:6823/1928245624","state":["exists","up"]},{"osd":7,"uuid":"fe031f5c-d915-48fb-91a6-c76f252467fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":1409789455},{"type":"v1","addr":"172.21.15.76:6817","nonce":1409789455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":1409789455},{"type":"v1","addr":"172.21.15.76:6819","nonce":1409789455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":1409789455},{"type":"v1","addr":"172.21.15.76:6823","nonce":1409789455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":1409789455},{"type":"v1","addr":"172.21.15.76:6821","nonce":1409789455}]},"public_addr":"172.21.15.76:6817/1409789455","cluster_addr":"172.21.15.76:6819/1409789455","heartbeat_back_addr":"172.21.15.76:6823/1409789455","heartbeat_front_addr":"172.21.15.76:6821/1409789455","state":["exists","up"]},{"osd":8,"uuid":"18851bde-9e6c-4e05-b4bf-ac5c3073eee7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":543619079},{"type":"v1","addr":"172.21.15.136:6817","nonce":543619079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":543619079},{"type":"v1","addr":"172.21.15.136:6819","nonce":543619079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":543619079},{"type":"v1","addr":"172.21.15.136:6823","nonce":543619079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":543619079},{"type":"v1","addr":"172.21.15.136:6821","nonce":543619079}]},"public_addr":"172.21.15.136:6817/543619079","cluster_addr":"172.21.15.136:6819/543619079","heartbeat_back_addr":"172.21.15.136:6823/543619079","heartbeat_front_addr":"172.21.15.136:6821/543619079","state":["exists","up"]},{"osd":9,"uuid":"52a65eab-8446-47ad-bfe7-5133f7e3eb98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":2201297064},{"type":"v1","addr":"172.21.15.26:6827","nonce":2201297064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":2201297064},{"type":"v1","addr":"172.21.15.26:6829","nonce":2201297064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":2201297064},{"type":"v1","addr":"172.21.15.26:6833","nonce":2201297064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":2201297064},{"type":"v1","addr":"172.21.15.26:6831","nonce":2201297064}]},"public_addr":"172.21.15.26:6827/2201297064","cluster_addr":"172.21.15.26:6829/2201297064","heartbeat_back_addr":"172.21.15.26:6833/2201297064","heartbeat_front_addr":"172.21.15.26:6831/2201297064","state":["exists","up"]},{"osd":10,"uuid":"b758d4bb-da62-4c5f-a992-bb6a145d35d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":3184128096},{"type":"v1","addr":"172.21.15.76:6825","nonce":3184128096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":3184128096},{"type":"v1","addr":"172.21.15.76:6827","nonce":3184128096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":3184128096},{"type":"v1","addr":"172.21.15.76:6831","nonce":3184128096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":3184128096},{"type":"v1","addr":"172.21.15.76:6829","nonce":3184128096}]},"public_addr":"172.21.15.76:6825/3184128096","cluster_addr":"172.21.15.76:6827/3184128096","heartbeat_back_addr":"172.21.15.76:6831/3184128096","heartbeat_front_addr":"172.21.15.76:6829/3184128096","state":["exists","up"]},{"osd":11,"uuid":"8fddda46-e22b-4d0c-ac48-e038fb29dfec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":1960689685},{"type":"v1","addr":"172.21.15.136:6825","nonce":1960689685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":1960689685},{"type":"v1","addr":"172.21.15.136:6827","nonce":1960689685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":1960689685},{"type":"v1","addr":"172.21.15.136:6831","nonce":1960689685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":1960689685},{"type":"v1","addr":"172.21.15.136:6829","nonce":1960689685}]},"public_addr":"172.21.15.136:6825/1960689685","cluster_addr":"172.21.15.136:6827/1960689685","heartbeat_back_addr":"172.21.15.136:6831/1960689685","heartbeat_front_addr":"172.21.15.136:6829/1960689685","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:10:56.656177+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:11:22.234031+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:11:46.464102+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:12:12.353967+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:12:37.836200+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:13:01.728839+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:13:27.527230+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:13:51.981393+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:14:17.035464+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:14:42.805029+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:15:08.208999+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:15:32.800844+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.18","mappings":[{"from":3,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":6}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.26:0/2626831531":"2024-09-12T23:18:22.177934+0000","172.21.15.26:0/3861103130":"2024-09-12T23:09:09.781665+0000","172.21.15.26:0/17142658":"2024-09-12T23:09:09.781665+0000","172.21.15.26:0/2817092653":"2024-09-12T23:09:09.781665+0000","172.21.15.26:6801/110833859":"2024-09-12T23:18:22.177934+0000","172.21.15.26:0/2107281413":"2024-09-12T23:09:33.983310+0000","172.21.15.26:0/883212708":"2024-09-12T23:09:33.983310+0000","172.21.15.26:6800/2673575521":"2024-09-12T23:09:09.781665+0000","172.21.15.26:0/2559066894":"2024-09-12T23:18:22.177934+0000","172.21.15.26:6801/2673575521":"2024-09-12T23:09:09.781665+0000","172.21.15.26:0/3597996003":"2024-09-12T23:09:33.983310+0000","172.21.15.26:6800/677148151":"2024-09-12T23:09:33.983310+0000","172.21.15.26:6801/677148151":"2024-09-12T23:09:33.983310+0000","172.21.15.26:0/1584355185":"2024-09-12T23:18:22.177934+0000","172.21.15.26:6800/110833859":"2024-09-12T23:18:22.177934+0000","172.21.15.26:0/1256290054":"2024-09-12T23:18:22.177934+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":3,"snaps":[{"begin":24,"length":1},{"begin":37,"length":1}]}],"new_removed_snaps":[{"pool":3,"snaps":[{"begin":24,"length":1},{"begin":37,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T23:39:52.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:52.135+0000 7fc5c5fe4640 1 -- 172.21.15.26:0/723856560 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fc5a80a2d00 msgr2=0x7fc5a80a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:52.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:52.135+0000 7fc5c5fe4640 1 --2- 172.21.15.26:0/723856560 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fc5a80a2d00 0x7fc5a80a51c0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fc5b000ab60 tx=0x7fc5b000a2d0 comp rx=0 tx=0).stop 2024-09-11T23:39:52.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:52.135+0000 7fc5c5fe4640 1 -- 172.21.15.26:0/723856560 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc5c0107aa0 msgr2=0x7fc5c019cbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:52.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:52.135+0000 7fc5c5fe4640 1 --2- 172.21.15.26:0/723856560 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc5c0107aa0 0x7fc5c019cbb0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7fc5b8005cc0 tx=0x7fc5b8003970 comp rx=0 tx=0).stop 2024-09-11T23:39:52.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:52.135+0000 7fc5c5fe4640 1 -- 172.21.15.26:0/723856560 shutdown_connections 2024-09-11T23:39:52.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:52.135+0000 7fc5c5fe4640 1 --2- 172.21.15.26:0/723856560 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fc5a80a2d00 0x7fc5a80a51c0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:52.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:52.135+0000 7fc5c5fe4640 1 --2- 172.21.15.26:0/723856560 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc5c0102430 0x7fc5c0198830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:52.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:52.135+0000 7fc5c5fe4640 1 --2- 172.21.15.26:0/723856560 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc5c0107aa0 0x7fc5c019cbb0 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:52.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:52.135+0000 7fc5c5fe4640 1 --2- 172.21.15.26:0/723856560 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc5c0101a90 0x7fc5c01982f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:52.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:52.135+0000 7fc5c5fe4640 1 -- 172.21.15.26:0/723856560 >> 172.21.15.26:0/723856560 conn(0x7fc5c00fd7d0 msgr2=0x7fc5c00fde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:39:52.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:52.136+0000 7fc5c5fe4640 1 -- 172.21.15.26:0/723856560 shutdown_connections 2024-09-11T23:39:52.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:52.136+0000 7fc5c5fe4640 1 -- 172.21.15.26:0/723856560 wait complete. 2024-09-11T23:39:52.148 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-11T23:11:50.494160+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-11T23:16:22.503928+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-11T23:16:24.876554+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '115', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 37, 'snap_epoch': 115, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-11T23:39:52.148 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-11T23:39:52.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:52.255+0000 7fe8856f1640 1 -- 172.21.15.26:0/2165718324 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe88010c0d0 msgr2=0x7fe88010e550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:52.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:52.255+0000 7fe8856f1640 1 --2- 172.21.15.26:0/2165718324 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe88010c0d0 0x7fe88010e550 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7fe878009ca0 tx=0x7fe87802a950 comp rx=0 tx=0).stop 2024-09-11T23:39:52.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:52.255+0000 7fe8856f1640 1 -- 172.21.15.26:0/2165718324 shutdown_connections 2024-09-11T23:39:52.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:52.255+0000 7fe8856f1640 1 --2- 172.21.15.26:0/2165718324 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe88010c0d0 0x7fe88010e550 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:52.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:52.255+0000 7fe8856f1640 1 --2- 172.21.15.26:0/2165718324 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe8801035a0 0x7fe88010ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:52.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:52.255+0000 7fe8856f1640 1 --2- 172.21.15.26:0/2165718324 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe880102c80 0x7fe880103060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:52.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:52.255+0000 7fe8856f1640 1 -- 172.21.15.26:0/2165718324 >> 172.21.15.26:0/2165718324 conn(0x7fe8800fc690 msgr2=0x7fe8800feab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:39:52.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:52.255+0000 7fe8856f1640 1 -- 172.21.15.26:0/2165718324 shutdown_connections 2024-09-11T23:39:52.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:52.255+0000 7fe8856f1640 1 -- 172.21.15.26:0/2165718324 wait complete. 2024-09-11T23:39:52.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:52.255+0000 7fe8856f1640 1 Processor -- start 2024-09-11T23:39:52.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:52.256+0000 7fe8856f1640 1 -- start start 2024-09-11T23:39:52.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:52.256+0000 7fe8856f1640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe880102c80 0x7fe8801983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:52.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:52.256+0000 7fe8856f1640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe8801035a0 0x7fe880198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:52.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:52.256+0000 7fe8856f1640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe88010c0d0 0x7fe88019cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:52.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:52.256+0000 7fe8856f1640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fe880111e70 con 0x7fe88010c0d0 2024-09-11T23:39:52.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:52.256+0000 7fe8856f1640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fe880111cf0 con 0x7fe8801035a0 2024-09-11T23:39:52.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:52.256+0000 7fe8856f1640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe880111ff0 con 0x7fe880102c80 2024-09-11T23:39:52.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:52.256+0000 7fe87f7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe88010c0d0 0x7fe88019cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:52.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:52.256+0000 7fe87f7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe88010c0d0 0x7fe88019cc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39544/0 (socket says 172.21.15.26:39544) 2024-09-11T23:39:52.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:52.256+0000 7fe87f7fe640 1 -- 172.21.15.26:0/3175047535 learned_addr learned my addr 172.21.15.26:0/3175047535 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:39:52.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:52.256+0000 7fe87effd640 1 --2- 172.21.15.26:0/3175047535 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe880102c80 0x7fe8801983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:52.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:52.256+0000 7fe87e7fc640 1 --2- 172.21.15.26:0/3175047535 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe8801035a0 0x7fe880198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:52.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:52.256+0000 7fe87f7fe640 1 -- 172.21.15.26:0/3175047535 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe880102c80 msgr2=0x7fe8801983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:52.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:52.256+0000 7fe87f7fe640 1 --2- 172.21.15.26:0/3175047535 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe880102c80 0x7fe8801983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:52.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:52.256+0000 7fe87f7fe640 1 -- 172.21.15.26:0/3175047535 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe8801035a0 msgr2=0x7fe880198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:52.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:52.256+0000 7fe87f7fe640 1 --2- 172.21.15.26:0/3175047535 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe8801035a0 0x7fe880198910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:52.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:52.256+0000 7fe87f7fe640 1 -- 172.21.15.26:0/3175047535 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe88019d3c0 con 0x7fe88010c0d0 2024-09-11T23:39:52.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:52.256+0000 7fe87f7fe640 1 --2- 172.21.15.26:0/3175047535 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe88010c0d0 0x7fe88019cc90 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7fe87802ae70 tx=0x7fe87802f5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:39:52.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:52.257+0000 7fe877fff640 1 -- 172.21.15.26:0/3175047535 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe878034040 con 0x7fe88010c0d0 2024-09-11T23:39:52.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:52.257+0000 7fe8856f1640 1 -- 172.21.15.26:0/3175047535 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe88019d650 con 0x7fe88010c0d0 2024-09-11T23:39:52.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:52.257+0000 7fe877fff640 1 -- 172.21.15.26:0/3175047535 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe878030920 con 0x7fe88010c0d0 2024-09-11T23:39:52.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:52.257+0000 7fe877fff640 1 -- 172.21.15.26:0/3175047535 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8780047b0 con 0x7fe88010c0d0 2024-09-11T23:39:52.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:52.257+0000 7fe8856f1640 1 -- 172.21.15.26:0/3175047535 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8801a4f40 con 0x7fe88010c0d0 2024-09-11T23:39:52.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:52.258+0000 7fe8856f1640 1 -- 172.21.15.26:0/3175047535 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe880103060 con 0x7fe88010c0d0 2024-09-11T23:39:52.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:52.259+0000 7fe877fff640 1 -- 172.21.15.26:0/3175047535 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fe878037020 con 0x7fe88010c0d0 2024-09-11T23:39:52.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:52.260+0000 7fe877fff640 1 --2- 172.21.15.26:0/3175047535 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fe86c0a2d00 0x7fe86c0a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:52.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:52.260+0000 7fe87effd640 1 --2- 172.21.15.26:0/3175047535 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fe86c0a2d00 0x7fe86c0a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:52.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:52.260+0000 7fe877fff640 1 -- 172.21.15.26:0/3175047535 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 8586+0+0 (secure 0 0 0) 0x7fe8780f0260 con 0x7fe88010c0d0 2024-09-11T23:39:52.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:52.260+0000 7fe87effd640 1 --2- 172.21.15.26:0/3175047535 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fe86c0a2d00 0x7fe86c0a51c0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fe87000abe0 tx=0x7fe87000a2d0 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:39:52.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:52.264+0000 7fe877fff640 1 -- 172.21.15.26:0/3175047535 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe8780bcef0 con 0x7fe88010c0d0 2024-09-11T23:39:52.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:52.444+0000 7fe8856f1640 1 -- 172.21.15.26:0/3175047535 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fe880107120 con 0x7fe88010c0d0 2024-09-11T23:39:52.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:52.445+0000 7fe877fff640 1 -- 172.21.15.26:0/3175047535 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v115) ==== 93+0+10 (secure 0 0 0) 0x7fe8780bc7f0 con 0x7fe88010c0d0 2024-09-11T23:39:52.445 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-09-11T23:39:52.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:52.447+0000 7fe8856f1640 1 -- 172.21.15.26:0/3175047535 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fe86c0a2d00 msgr2=0x7fe86c0a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:52.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:52.447+0000 7fe8856f1640 1 --2- 172.21.15.26:0/3175047535 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fe86c0a2d00 0x7fe86c0a51c0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fe87000abe0 tx=0x7fe87000a2d0 comp rx=0 tx=0).stop 2024-09-11T23:39:52.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:52.447+0000 7fe8856f1640 1 -- 172.21.15.26:0/3175047535 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe88010c0d0 msgr2=0x7fe88019cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:52.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:52.447+0000 7fe8856f1640 1 --2- 172.21.15.26:0/3175047535 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe88010c0d0 0x7fe88019cc90 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7fe87802ae70 tx=0x7fe87802f5b0 comp rx=0 tx=0).stop 2024-09-11T23:39:52.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:52.447+0000 7fe8856f1640 1 -- 172.21.15.26:0/3175047535 shutdown_connections 2024-09-11T23:39:52.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:52.447+0000 7fe8856f1640 1 --2- 172.21.15.26:0/3175047535 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fe86c0a2d00 0x7fe86c0a51c0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:52.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:52.447+0000 7fe8856f1640 1 --2- 172.21.15.26:0/3175047535 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe8801035a0 0x7fe880198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:52.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:52.447+0000 7fe8856f1640 1 --2- 172.21.15.26:0/3175047535 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe88010c0d0 0x7fe88019cc90 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:52.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:52.447+0000 7fe8856f1640 1 --2- 172.21.15.26:0/3175047535 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe880102c80 0x7fe8801983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:52.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:52.447+0000 7fe8856f1640 1 -- 172.21.15.26:0/3175047535 >> 172.21.15.26:0/3175047535 conn(0x7fe8800fc690 msgr2=0x7fe8800fddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:39:52.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:52.447+0000 7fe8856f1640 1 -- 172.21.15.26:0/3175047535 shutdown_connections 2024-09-11T23:39:52.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:52.447+0000 7fe8856f1640 1 -- 172.21.15.26:0/3175047535 wait complete. 2024-09-11T23:39:52.457 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:39:52 smithi026 ceph-mon[31172]: pgmap v676: 97 pgs: 8 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 78 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 407 op/s 2024-09-11T23:39:52.458 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-11T23:39:52.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:39:52 smithi136 ceph-mon[35121]: pgmap v676: 97 pgs: 8 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 78 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 407 op/s 2024-09-11T23:39:52.552 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:39:52 smithi076 ceph-mon[35296]: pgmap v676: 97 pgs: 8 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 78 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 407 op/s 2024-09-11T23:39:52.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:52.567+0000 7f1b97386640 1 -- 172.21.15.26:0/367647540 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1b90101350 msgr2=0x7f1b9010d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:52.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:52.567+0000 7f1b97386640 1 --2- 172.21.15.26:0/367647540 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1b90101350 0x7f1b9010d880 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f1b88009cd0 tx=0x7f1b8802a8e0 comp rx=0 tx=0).stop 2024-09-11T23:39:52.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:52.567+0000 7f1b97386640 1 -- 172.21.15.26:0/367647540 shutdown_connections 2024-09-11T23:39:52.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:52.567+0000 7f1b97386640 1 --2- 172.21.15.26:0/367647540 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1b901009b0 0x7f1b90100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:52.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:52.567+0000 7f1b97386640 1 --2- 172.21.15.26:0/367647540 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1b90101350 0x7f1b9010d880 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:52.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:52.567+0000 7f1b97386640 1 --2- 172.21.15.26:0/367647540 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1b901069c0 0x7f1b90106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:52.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:52.567+0000 7f1b97386640 1 -- 172.21.15.26:0/367647540 >> 172.21.15.26:0/367647540 conn(0x7f1b900fc690 msgr2=0x7f1b900feab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:39:52.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:52.568+0000 7f1b97386640 1 -- 172.21.15.26:0/367647540 shutdown_connections 2024-09-11T23:39:52.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:52.568+0000 7f1b97386640 1 -- 172.21.15.26:0/367647540 wait complete. 2024-09-11T23:39:52.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:52.568+0000 7f1b97386640 1 Processor -- start 2024-09-11T23:39:52.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:52.568+0000 7f1b97386640 1 -- start start 2024-09-11T23:39:52.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:52.568+0000 7f1b97386640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1b901009b0 0x7f1b901991a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:52.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:52.568+0000 7f1b97386640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1b90101350 0x7f1b901996e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:52.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:52.568+0000 7f1b97386640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1b901069c0 0x7f1b901a4220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:52.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:52.568+0000 7f1b97386640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f1b90110e50 con 0x7f1b90101350 2024-09-11T23:39:52.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:52.568+0000 7f1b97386640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f1b90110cd0 con 0x7f1b901009b0 2024-09-11T23:39:52.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:52.568+0000 7f1b97386640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f1b90110fd0 con 0x7f1b901069c0 2024-09-11T23:39:52.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:52.568+0000 7f1b94921640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1b90101350 0x7f1b901996e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:52.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:52.568+0000 7f1b94921640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1b90101350 0x7f1b901996e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39556/0 (socket says 172.21.15.26:39556) 2024-09-11T23:39:52.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:52.568+0000 7f1b95122640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1b901009b0 0x7f1b901991a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:52.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:52.568+0000 7f1b94921640 1 -- 172.21.15.26:0/2213146306 learned_addr learned my addr 172.21.15.26:0/2213146306 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:39:52.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:52.569+0000 7f1b95923640 1 --2- 172.21.15.26:0/2213146306 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1b901069c0 0x7f1b901a4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:52.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:52.569+0000 7f1b94921640 1 -- 172.21.15.26:0/2213146306 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1b901069c0 msgr2=0x7f1b901a4220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:52.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:52.569+0000 7f1b94921640 1 --2- 172.21.15.26:0/2213146306 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1b901069c0 0x7f1b901a4220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:52.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:52.569+0000 7f1b94921640 1 -- 172.21.15.26:0/2213146306 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1b901009b0 msgr2=0x7f1b901991a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:52.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:52.569+0000 7f1b94921640 1 --2- 172.21.15.26:0/2213146306 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1b901009b0 0x7f1b901991a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:52.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:52.569+0000 7f1b94921640 1 -- 172.21.15.26:0/2213146306 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b901a4760 con 0x7f1b90101350 2024-09-11T23:39:52.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:52.569+0000 7f1b94921640 1 --2- 172.21.15.26:0/2213146306 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1b90101350 0x7f1b901996e0 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f1b8000d950 tx=0x7f1b8000dd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:39:52.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:52.569+0000 7f1b867fc640 1 -- 172.21.15.26:0/2213146306 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b80017070 con 0x7f1b90101350 2024-09-11T23:39:52.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:52.569+0000 7f1b867fc640 1 -- 172.21.15.26:0/2213146306 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b8000de80 con 0x7f1b90101350 2024-09-11T23:39:52.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:52.569+0000 7f1b97386640 1 -- 172.21.15.26:0/2213146306 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b901a4930 con 0x7f1b90101350 2024-09-11T23:39:52.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:52.569+0000 7f1b867fc640 1 -- 172.21.15.26:0/2213146306 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b8000f620 con 0x7f1b90101350 2024-09-11T23:39:52.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:52.569+0000 7f1b97386640 1 -- 172.21.15.26:0/2213146306 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b901a4e50 con 0x7f1b90101350 2024-09-11T23:39:52.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:52.570+0000 7f1b97386640 1 -- 172.21.15.26:0/2213146306 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b90100e10 con 0x7f1b90101350 2024-09-11T23:39:52.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:52.572+0000 7f1b867fc640 1 -- 172.21.15.26:0/2213146306 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f1b80009c30 con 0x7f1b90101350 2024-09-11T23:39:52.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:52.572+0000 7f1b867fc640 1 --2- 172.21.15.26:0/2213146306 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f1b7c0a2d00 0x7f1b7c0a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:52.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:52.572+0000 7f1b95122640 1 --2- 172.21.15.26:0/2213146306 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f1b7c0a2d00 0x7f1b7c0a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:52.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:52.573+0000 7f1b867fc640 1 -- 172.21.15.26:0/2213146306 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 8586+0+0 (secure 0 0 0) 0x7f1b800cf040 con 0x7f1b90101350 2024-09-11T23:39:52.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:52.573+0000 7f1b95122640 1 --2- 172.21.15.26:0/2213146306 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f1b7c0a2d00 0x7f1b7c0a51c0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f1b8c00ab60 tx=0x7f1b8c00a2d0 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:39:52.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:52.575+0000 7f1b867fc640 1 -- 172.21.15.26:0/2213146306 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1b8009bc20 con 0x7f1b90101350 2024-09-11T23:39:52.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:52.751+0000 7f1b97386640 1 -- 172.21.15.26:0/2213146306 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f1b901a50c0 con 0x7f1b90101350 2024-09-11T23:39:52.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:52.751+0000 7f1b867fc640 1 -- 172.21.15.26:0/2213146306 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v115) ==== 104+0+11 (secure 0 0 0) 0x7f1b8009b520 con 0x7f1b90101350 2024-09-11T23:39:52.752 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2024-09-11T23:39:52.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:52.753+0000 7f1b97386640 1 -- 172.21.15.26:0/2213146306 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f1b7c0a2d00 msgr2=0x7f1b7c0a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:52.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:52.753+0000 7f1b97386640 1 --2- 172.21.15.26:0/2213146306 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f1b7c0a2d00 0x7f1b7c0a51c0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f1b8c00ab60 tx=0x7f1b8c00a2d0 comp rx=0 tx=0).stop 2024-09-11T23:39:52.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:52.753+0000 7f1b97386640 1 -- 172.21.15.26:0/2213146306 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1b90101350 msgr2=0x7f1b901996e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:52.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:52.753+0000 7f1b97386640 1 --2- 172.21.15.26:0/2213146306 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1b90101350 0x7f1b901996e0 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f1b8000d950 tx=0x7f1b8000dd50 comp rx=0 tx=0).stop 2024-09-11T23:39:52.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:52.753+0000 7f1b97386640 1 -- 172.21.15.26:0/2213146306 shutdown_connections 2024-09-11T23:39:52.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:52.753+0000 7f1b97386640 1 --2- 172.21.15.26:0/2213146306 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f1b7c0a2d00 0x7f1b7c0a51c0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:52.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:52.753+0000 7f1b97386640 1 --2- 172.21.15.26:0/2213146306 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1b901069c0 0x7f1b901a4220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:52.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:52.753+0000 7f1b97386640 1 --2- 172.21.15.26:0/2213146306 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1b90101350 0x7f1b901996e0 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:52.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:52.753+0000 7f1b97386640 1 --2- 172.21.15.26:0/2213146306 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1b901009b0 0x7f1b901991a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:52.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:52.753+0000 7f1b97386640 1 -- 172.21.15.26:0/2213146306 >> 172.21.15.26:0/2213146306 conn(0x7f1b900fc690 msgr2=0x7f1b900fdc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:39:52.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:52.753+0000 7f1b97386640 1 -- 172.21.15.26:0/2213146306 shutdown_connections 2024-09-11T23:39:52.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:52.753+0000 7f1b97386640 1 -- 172.21.15.26:0/2213146306 wait complete. 2024-09-11T23:39:52.765 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-11T23:39:52.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:52.871+0000 7f34098e4640 1 -- 172.21.15.26:0/1365767306 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3404101a80 msgr2=0x7f3404101ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:52.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:52.871+0000 7f34098e4640 1 --2- 172.21.15.26:0/1365767306 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3404101a80 0x7f3404101ee0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f33f8009930 tx=0x7f33f802a420 comp rx=0 tx=0).stop 2024-09-11T23:39:52.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:52.871+0000 7f34098e4640 1 -- 172.21.15.26:0/1365767306 shutdown_connections 2024-09-11T23:39:52.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:52.871+0000 7f34098e4640 1 --2- 172.21.15.26:0/1365767306 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3404102420 0x7f340410e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:52.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:52.871+0000 7f34098e4640 1 --2- 172.21.15.26:0/1365767306 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3404101a80 0x7f3404101ee0 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:52.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:52.871+0000 7f34098e4640 1 --2- 172.21.15.26:0/1365767306 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3404107a90 0x7f3404107e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:52.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:52.871+0000 7f34098e4640 1 -- 172.21.15.26:0/1365767306 >> 172.21.15.26:0/1365767306 conn(0x7f34040fd7c0 msgr2=0x7f34040ffbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:39:52.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:52.871+0000 7f34098e4640 1 -- 172.21.15.26:0/1365767306 shutdown_connections 2024-09-11T23:39:52.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:52.871+0000 7f34098e4640 1 -- 172.21.15.26:0/1365767306 wait complete. 2024-09-11T23:39:52.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:52.872+0000 7f34098e4640 1 Processor -- start 2024-09-11T23:39:52.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:52.872+0000 7f34098e4640 1 -- start start 2024-09-11T23:39:52.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:52.872+0000 7f34098e4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3404101a80 0x7f34040744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:52.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:52.872+0000 7f34098e4640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3404102420 0x7f3404074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:52.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:52.872+0000 7f34098e4640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3404107a90 0x7f3404074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:52.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:52.872+0000 7f34098e4640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f3404111ed0 con 0x7f3404101a80 2024-09-11T23:39:52.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:52.872+0000 7f34098e4640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f3404111d50 con 0x7f3404107a90 2024-09-11T23:39:52.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:52.872+0000 7f34098e4640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f3404112050 con 0x7f3404102420 2024-09-11T23:39:52.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:52.872+0000 7f3402ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3404101a80 0x7f34040744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:52.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:52.872+0000 7f3402ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3404101a80 0x7f34040744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39572/0 (socket says 172.21.15.26:39572) 2024-09-11T23:39:52.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:52.872+0000 7f3402ffd640 1 -- 172.21.15.26:0/3827593029 learned_addr learned my addr 172.21.15.26:0/3827593029 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:39:52.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:52.872+0000 7f34037fe640 1 --2- 172.21.15.26:0/3827593029 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3404107a90 0x7f3404074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:52.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:52.872+0000 7f34027fc640 1 --2- 172.21.15.26:0/3827593029 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3404102420 0x7f3404074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:52.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:52.872+0000 7f3402ffd640 1 -- 172.21.15.26:0/3827593029 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3404102420 msgr2=0x7f3404074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:52.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:52.872+0000 7f3402ffd640 1 --2- 172.21.15.26:0/3827593029 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3404102420 0x7f3404074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:52.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:52.873+0000 7f3402ffd640 1 -- 172.21.15.26:0/3827593029 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3404107a90 msgr2=0x7f3404074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:52.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:52.873+0000 7f3402ffd640 1 --2- 172.21.15.26:0/3827593029 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3404107a90 0x7f3404074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:52.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:52.873+0000 7f3402ffd640 1 -- 172.21.15.26:0/3827593029 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34041a6940 con 0x7f3404101a80 2024-09-11T23:39:52.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:52.873+0000 7f3402ffd640 1 --2- 172.21.15.26:0/3827593029 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3404101a80 0x7f34040744f0 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f33f400e9f0 tx=0x7f33f400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:39:52.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:52.873+0000 7f34088e2640 1 -- 172.21.15.26:0/3827593029 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33f400cd80 con 0x7f3404101a80 2024-09-11T23:39:52.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:52.873+0000 7f34088e2640 1 -- 172.21.15.26:0/3827593029 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f33f4010910 con 0x7f3404101a80 2024-09-11T23:39:52.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:52.873+0000 7f34088e2640 1 -- 172.21.15.26:0/3827593029 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33f4018990 con 0x7f3404101a80 2024-09-11T23:39:52.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:52.873+0000 7f34098e4640 1 -- 172.21.15.26:0/3827593029 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34041a6c30 con 0x7f3404101a80 2024-09-11T23:39:52.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:52.873+0000 7f34098e4640 1 -- 172.21.15.26:0/3827593029 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34041a7150 con 0x7f3404101a80 2024-09-11T23:39:52.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:52.874+0000 7f34098e4640 1 -- 172.21.15.26:0/3827593029 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3404101ee0 con 0x7f3404101a80 2024-09-11T23:39:52.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:52.875+0000 7f34088e2640 1 -- 172.21.15.26:0/3827593029 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f33f4010ac0 con 0x7f3404101a80 2024-09-11T23:39:52.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:52.876+0000 7f34088e2640 1 --2- 172.21.15.26:0/3827593029 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f33ec0a2d00 0x7f33ec0a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:52.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:52.876+0000 7f34088e2640 1 -- 172.21.15.26:0/3827593029 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 8586+0+0 (secure 0 0 0) 0x7f33f40cfae0 con 0x7f3404101a80 2024-09-11T23:39:52.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:52.876+0000 7f34027fc640 1 --2- 172.21.15.26:0/3827593029 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f33ec0a2d00 0x7f33ec0a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:52.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:52.876+0000 7f34027fc640 1 --2- 172.21.15.26:0/3827593029 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f33ec0a2d00 0x7f33ec0a51c0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f33f8009900 tx=0x7f33f80058d0 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:39:52.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:52.879+0000 7f34088e2640 1 -- 172.21.15.26:0/3827593029 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f33f409c070 con 0x7f3404101a80 2024-09-11T23:39:53.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:53.055+0000 7f34098e4640 1 -- 172.21.15.26:0/3827593029 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f3404109ff0 con 0x7f3404101a80 2024-09-11T23:39:53.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:53.056+0000 7f34088e2640 1 -- 172.21.15.26:0/3827593029 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v115) ==== 100+0+11 (secure 0 0 0) 0x7f33f40a0370 con 0x7f3404101a80 2024-09-11T23:39:53.056 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2024-09-11T23:39:53.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:53.057+0000 7f34098e4640 1 -- 172.21.15.26:0/3827593029 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f33ec0a2d00 msgr2=0x7f33ec0a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:53.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:53.057+0000 7f34098e4640 1 --2- 172.21.15.26:0/3827593029 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f33ec0a2d00 0x7f33ec0a51c0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f33f8009900 tx=0x7f33f80058d0 comp rx=0 tx=0).stop 2024-09-11T23:39:53.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:53.057+0000 7f34098e4640 1 -- 172.21.15.26:0/3827593029 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3404101a80 msgr2=0x7f34040744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:53.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:53.057+0000 7f34098e4640 1 --2- 172.21.15.26:0/3827593029 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3404101a80 0x7f34040744f0 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f33f400e9f0 tx=0x7f33f400eeb0 comp rx=0 tx=0).stop 2024-09-11T23:39:53.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:53.058+0000 7f34098e4640 1 -- 172.21.15.26:0/3827593029 shutdown_connections 2024-09-11T23:39:53.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:53.058+0000 7f34098e4640 1 --2- 172.21.15.26:0/3827593029 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f33ec0a2d00 0x7f33ec0a51c0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:53.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:53.058+0000 7f34098e4640 1 --2- 172.21.15.26:0/3827593029 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3404107a90 0x7f3404074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:53.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:53.058+0000 7f34098e4640 1 --2- 172.21.15.26:0/3827593029 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3404102420 0x7f3404074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:53.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:53.058+0000 7f34098e4640 1 --2- 172.21.15.26:0/3827593029 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3404101a80 0x7f34040744f0 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:53.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:53.058+0000 7f34098e4640 1 -- 172.21.15.26:0/3827593029 >> 172.21.15.26:0/3827593029 conn(0x7f34040fd7c0 msgr2=0x7f34040fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:39:53.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:53.058+0000 7f34098e4640 1 -- 172.21.15.26:0/3827593029 shutdown_connections 2024-09-11T23:39:53.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:53.058+0000 7f34098e4640 1 -- 172.21.15.26:0/3827593029 wait complete. 2024-09-11T23:39:53.069 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-09-11T23:39:53.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:53.173+0000 7f551a6b7640 1 -- 172.21.15.26:0/2501372299 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5514106990 msgr2=0x7f5514106d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:53.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:53.173+0000 7f551a6b7640 1 --2- 172.21.15.26:0/2501372299 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5514106990 0x7f5514106d70 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f5504009930 tx=0x7f550402a420 comp rx=0 tx=0).stop 2024-09-11T23:39:53.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:53.173+0000 7f551a6b7640 1 -- 172.21.15.26:0/2501372299 shutdown_connections 2024-09-11T23:39:53.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:53.173+0000 7f551a6b7640 1 --2- 172.21.15.26:0/2501372299 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5514100980 0x7f5514100de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:53.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:53.173+0000 7f551a6b7640 1 --2- 172.21.15.26:0/2501372299 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5514101320 0x7f551410d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:53.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:53.173+0000 7f551a6b7640 1 --2- 172.21.15.26:0/2501372299 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5514106990 0x7f5514106d70 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:53.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:53.173+0000 7f551a6b7640 1 -- 172.21.15.26:0/2501372299 >> 172.21.15.26:0/2501372299 conn(0x7f5514077a10 msgr2=0x7f55140fed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:39:53.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:53.173+0000 7f551a6b7640 1 -- 172.21.15.26:0/2501372299 shutdown_connections 2024-09-11T23:39:53.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:53.173+0000 7f551a6b7640 1 -- 172.21.15.26:0/2501372299 wait complete. 2024-09-11T23:39:53.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:53.173+0000 7f551a6b7640 1 Processor -- start 2024-09-11T23:39:53.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:53.173+0000 7f551a6b7640 1 -- start start 2024-09-11T23:39:53.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:53.174+0000 7f551a6b7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5514100980 0x7f55141992e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:53.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:53.174+0000 7f551a6b7640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5514101320 0x7f5514199820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:53.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:53.174+0000 7f551a6b7640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5514106990 0x7f55141a4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:53.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:53.174+0000 7f551a6b7640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f5514110dd0 con 0x7f5514100980 2024-09-11T23:39:53.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:53.174+0000 7f551a6b7640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f5514110c50 con 0x7f5514101320 2024-09-11T23:39:53.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:53.174+0000 7f551a6b7640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f5514110f50 con 0x7f5514106990 2024-09-11T23:39:53.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:53.174+0000 7f5513fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5514100980 0x7f55141992e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:53.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:53.174+0000 7f5513fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5514100980 0x7f55141992e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39598/0 (socket says 172.21.15.26:39598) 2024-09-11T23:39:53.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:53.174+0000 7f5513fff640 1 -- 172.21.15.26:0/1393308929 learned_addr learned my addr 172.21.15.26:0/1393308929 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:39:53.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:53.174+0000 7f55137fe640 1 --2- 172.21.15.26:0/1393308929 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5514101320 0x7f5514199820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:53.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:53.174+0000 7f5518c54640 1 --2- 172.21.15.26:0/1393308929 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5514106990 0x7f55141a4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:53.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:53.174+0000 7f5513fff640 1 -- 172.21.15.26:0/1393308929 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5514106990 msgr2=0x7f55141a4360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:53.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:53.174+0000 7f5513fff640 1 --2- 172.21.15.26:0/1393308929 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5514106990 0x7f55141a4360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:53.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:53.174+0000 7f5513fff640 1 -- 172.21.15.26:0/1393308929 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5514101320 msgr2=0x7f5514199820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:53.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:53.174+0000 7f5513fff640 1 --2- 172.21.15.26:0/1393308929 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5514101320 0x7f5514199820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:53.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:53.174+0000 7f5513fff640 1 -- 172.21.15.26:0/1393308929 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55141a48a0 con 0x7f5514100980 2024-09-11T23:39:53.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:53.174+0000 7f5513fff640 1 --2- 172.21.15.26:0/1393308929 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5514100980 0x7f55141992e0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f5504009230 tx=0x7f5504004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:39:53.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:53.175+0000 7f55117fa640 1 -- 172.21.15.26:0/1393308929 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f550403f030 con 0x7f5514100980 2024-09-11T23:39:53.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:53.175+0000 7f551a6b7640 1 -- 172.21.15.26:0/1393308929 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55141a4a70 con 0x7f5514100980 2024-09-11T23:39:53.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:53.175+0000 7f55117fa640 1 -- 172.21.15.26:0/1393308929 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5504003ad0 con 0x7f5514100980 2024-09-11T23:39:53.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:53.175+0000 7f55117fa640 1 -- 172.21.15.26:0/1393308929 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5504035070 con 0x7f5514100980 2024-09-11T23:39:53.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:53.175+0000 7f551a6b7640 1 -- 172.21.15.26:0/1393308929 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55141a4ee0 con 0x7f5514100980 2024-09-11T23:39:53.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:53.176+0000 7f551a6b7640 1 -- 172.21.15.26:0/1393308929 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5514100de0 con 0x7f5514100980 2024-09-11T23:39:53.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:53.177+0000 7f55117fa640 1 -- 172.21.15.26:0/1393308929 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f5504042030 con 0x7f5514100980 2024-09-11T23:39:53.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:53.178+0000 7f55117fa640 1 --2- 172.21.15.26:0/1393308929 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f54fc0a2d00 0x7f54fc0a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:53.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:53.178+0000 7f55117fa640 1 -- 172.21.15.26:0/1393308929 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 8586+0+0 (secure 0 0 0) 0x7f55040ef960 con 0x7f5514100980 2024-09-11T23:39:53.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:53.178+0000 7f55137fe640 1 --2- 172.21.15.26:0/1393308929 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f54fc0a2d00 0x7f54fc0a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:53.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:53.178+0000 7f55137fe640 1 --2- 172.21.15.26:0/1393308929 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f54fc0a2d00 0x7f54fc0a51c0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f5514069ed0 tx=0x7f550800b390 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:39:53.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:53.182+0000 7f55117fa640 1 -- 172.21.15.26:0/1393308929 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f55040bbe90 con 0x7f5514100980 2024-09-11T23:39:53.370 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:39:53 smithi026 ceph-mon[31172]: from='client.18165 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:39:53.370 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:39:53 smithi026 ceph-mon[31172]: from='client.18171 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:39:53.370 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:39:53 smithi026 ceph-mon[31172]: from='client.? 172.21.15.26:0/723856560' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T23:39:53.370 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:39:53 smithi026 ceph-mon[31172]: from='client.? 172.21.15.26:0/3175047535' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-11T23:39:53.370 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:39:53 smithi026 ceph-mon[31172]: from='client.? 172.21.15.26:0/2213146306' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-11T23:39:53.370 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:39:53 smithi026 ceph-mon[31172]: from='client.? 172.21.15.26:0/3827593029' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-11T23:39:53.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:53.408+0000 7f551a6b7640 1 -- 172.21.15.26:0/1393308929 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f5514108ef0 con 0x7f5514100980 2024-09-11T23:39:53.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:53.416+0000 7f55117fa640 1 -- 172.21.15.26:0/1393308929 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v26) ==== 81+0+740903 (secure 0 0 0) 0x7f55040c0190 con 0x7f5514100980 2024-09-11T23:39:53.427 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-11T23:39:53.427 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-11T23:39:53.427 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 26, 2024-09-11T23:39:53.427 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.427 INFO:teuthology.orchestra.run.smithi026.stdout: "active_gid": 15171, 2024-09-11T23:39:53.427 INFO:teuthology.orchestra.run.smithi026.stdout: "active_name": "x", 2024-09-11T23:39:53.428 INFO:teuthology.orchestra.run.smithi026.stdout: "active_addrs": { 2024-09-11T23:39:53.428 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-11T23:39:53.428 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-11T23:39:53.428 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v2", 2024-09-11T23:39:53.428 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6800", 2024-09-11T23:39:53.428 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 1989555790 2024-09-11T23:39:53.428 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.428 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-11T23:39:53.428 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-11T23:39:53.428 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6801", 2024-09-11T23:39:53.428 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 1989555790 2024-09-11T23:39:53.428 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:53.428 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-11T23:39:53.429 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.429 INFO:teuthology.orchestra.run.smithi026.stdout: "active_addr": "172.21.15.26:6801/1989555790", 2024-09-11T23:39:53.429 INFO:teuthology.orchestra.run.smithi026.stdout: "active_change": "2024-09-11T23:18:22.178145+0000", 2024-09-11T23:39:53.429 INFO:teuthology.orchestra.run.smithi026.stdout: "active_mgr_features": 4540701547738038271, 2024-09-11T23:39:53.429 INFO:teuthology.orchestra.run.smithi026.stdout: "available": true, 2024-09-11T23:39:53.429 INFO:teuthology.orchestra.run.smithi026.stdout: "standbys": [ 2024-09-11T23:39:53.429 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-11T23:39:53.429 INFO:teuthology.orchestra.run.smithi026.stdout: "gid": 24851, 2024-09-11T23:39:53.429 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "y", 2024-09-11T23:39:53.429 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr_features": 4540701547738038271, 2024-09-11T23:39:53.429 INFO:teuthology.orchestra.run.smithi026.stdout: "available_modules": [ 2024-09-11T23:39:53.430 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-11T23:39:53.430 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "alerts", 2024-09-11T23:39:53.430 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-09-11T23:39:53.430 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-09-11T23:39:53.430 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-09-11T23:39:53.430 INFO:teuthology.orchestra.run.smithi026.stdout: "interval": { 2024-09-11T23:39:53.430 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "interval", 2024-09-11T23:39:53.430 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-09-11T23:39:53.430 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.430 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.430 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "60", 2024-09-11T23:39:53.430 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.430 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.430 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.431 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "How frequently to reexamine health status", 2024-09-11T23:39:53.431 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.431 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.431 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.431 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.431 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-09-11T23:39:53.431 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-09-11T23:39:53.431 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.431 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.431 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.431 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:53.431 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.431 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.431 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:53.432 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:53.432 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:53.432 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:53.432 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:53.432 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:53.432 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:53.432 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:53.432 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.432 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.432 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.432 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.432 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.432 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-09-11T23:39:53.433 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-09-11T23:39:53.433 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.433 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.433 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.433 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:53.433 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.433 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.433 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.433 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.433 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.433 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.433 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.433 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.434 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-09-11T23:39:53.434 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-09-11T23:39:53.434 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.434 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.434 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.434 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-09-11T23:39:53.434 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.434 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.434 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:53.434 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:53.434 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:53.434 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:53.434 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:53.434 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:53.434 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:53.435 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:53.435 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.435 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.435 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.435 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.435 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.435 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-09-11T23:39:53.435 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-09-11T23:39:53.435 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.435 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.435 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.435 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:53.435 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.435 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.436 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.436 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.436 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.436 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.436 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.436 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.436 INFO:teuthology.orchestra.run.smithi026.stdout: "smtp_destination": { 2024-09-11T23:39:53.436 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "smtp_destination", 2024-09-11T23:39:53.436 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.436 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.436 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.437 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:53.437 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.437 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.437 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.437 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Email address to send alerts to", 2024-09-11T23:39:53.437 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.437 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.437 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.437 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.438 INFO:teuthology.orchestra.run.smithi026.stdout: "smtp_from_name": { 2024-09-11T23:39:53.438 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "smtp_from_name", 2024-09-11T23:39:53.438 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.438 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.438 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.438 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "Ceph", 2024-09-11T23:39:53.438 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.438 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.438 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.438 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Email From: name", 2024-09-11T23:39:53.438 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.439 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.439 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.439 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.439 INFO:teuthology.orchestra.run.smithi026.stdout: "smtp_host": { 2024-09-11T23:39:53.439 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "smtp_host", 2024-09-11T23:39:53.439 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.439 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.439 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.439 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:53.439 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.439 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.439 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.440 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "SMTP server", 2024-09-11T23:39:53.440 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.440 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.440 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.440 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.440 INFO:teuthology.orchestra.run.smithi026.stdout: "smtp_password": { 2024-09-11T23:39:53.440 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "smtp_password", 2024-09-11T23:39:53.440 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.440 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.440 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.440 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:53.440 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.440 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.440 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.441 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Password to authenticate with", 2024-09-11T23:39:53.441 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.441 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.441 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.441 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.441 INFO:teuthology.orchestra.run.smithi026.stdout: "smtp_port": { 2024-09-11T23:39:53.441 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "smtp_port", 2024-09-11T23:39:53.441 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-09-11T23:39:53.441 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.441 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.441 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "465", 2024-09-11T23:39:53.441 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.441 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.441 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.442 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "SMTP port", 2024-09-11T23:39:53.442 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.442 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.442 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.442 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.442 INFO:teuthology.orchestra.run.smithi026.stdout: "smtp_sender": { 2024-09-11T23:39:53.442 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "smtp_sender", 2024-09-11T23:39:53.442 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.442 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.442 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.443 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:53.443 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.443 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.443 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.443 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "SMTP envelope sender", 2024-09-11T23:39:53.443 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.443 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.443 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.444 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.444 INFO:teuthology.orchestra.run.smithi026.stdout: "smtp_ssl": { 2024-09-11T23:39:53.444 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "smtp_ssl", 2024-09-11T23:39:53.444 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.444 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.444 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.444 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-09-11T23:39:53.444 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.444 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.444 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.444 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-11T23:39:53.444 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.444 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.445 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.445 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.445 INFO:teuthology.orchestra.run.smithi026.stdout: "smtp_user": { 2024-09-11T23:39:53.445 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "smtp_user", 2024-09-11T23:39:53.445 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.445 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.445 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.445 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:53.445 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.445 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.445 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.445 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "User to authenticate as", 2024-09-11T23:39:53.445 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.445 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.446 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.446 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:53.446 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:53.446 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.446 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-11T23:39:53.446 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "balancer", 2024-09-11T23:39:53.446 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-09-11T23:39:53.446 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-09-11T23:39:53.446 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-09-11T23:39:53.446 INFO:teuthology.orchestra.run.smithi026.stdout: "active": { 2024-09-11T23:39:53.446 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "active", 2024-09-11T23:39:53.446 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.446 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.446 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.447 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-09-11T23:39:53.447 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.447 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.447 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.447 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "automatically balance PGs across cluster", 2024-09-11T23:39:53.447 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.447 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.447 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.447 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.447 INFO:teuthology.orchestra.run.smithi026.stdout: "begin_time": { 2024-09-11T23:39:53.448 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "begin_time", 2024-09-11T23:39:53.448 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.448 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.448 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.448 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0000", 2024-09-11T23:39:53.448 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.448 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.448 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.448 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "beginning time of day to automatically balance", 2024-09-11T23:39:53.448 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-11T23:39:53.448 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.448 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.449 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.449 INFO:teuthology.orchestra.run.smithi026.stdout: "begin_weekday": { 2024-09-11T23:39:53.449 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "begin_weekday", 2024-09-11T23:39:53.449 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "uint", 2024-09-11T23:39:53.449 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.449 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.449 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2024-09-11T23:39:53.449 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "0", 2024-09-11T23:39:53.449 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "6", 2024-09-11T23:39:53.449 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.449 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-11T23:39:53.449 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-11T23:39:53.449 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.449 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.450 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.450 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_compat_max_iterations": { 2024-09-11T23:39:53.450 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "crush_compat_max_iterations", 2024-09-11T23:39:53.450 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "uint", 2024-09-11T23:39:53.450 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.450 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.450 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "25", 2024-09-11T23:39:53.450 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "1", 2024-09-11T23:39:53.450 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "250", 2024-09-11T23:39:53.450 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.450 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-11T23:39:53.450 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.450 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.450 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.451 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.451 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_compat_metrics": { 2024-09-11T23:39:53.451 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "crush_compat_metrics", 2024-09-11T23:39:53.451 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.451 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.451 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.451 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "pgs,objects,bytes", 2024-09-11T23:39:53.451 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.451 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.451 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.451 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-11T23:39:53.451 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-11T23:39:53.451 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.451 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.451 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.452 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_compat_step": { 2024-09-11T23:39:53.452 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "crush_compat_step", 2024-09-11T23:39:53.452 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "float", 2024-09-11T23:39:53.452 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.452 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.452 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0.5", 2024-09-11T23:39:53.452 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "0.001", 2024-09-11T23:39:53.452 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "0.999", 2024-09-11T23:39:53.452 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.452 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "aggressiveness of optimization", 2024-09-11T23:39:53.452 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-11T23:39:53.452 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.452 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.452 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.453 INFO:teuthology.orchestra.run.smithi026.stdout: "end_time": { 2024-09-11T23:39:53.453 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "end_time", 2024-09-11T23:39:53.453 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.453 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.453 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.453 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "2359", 2024-09-11T23:39:53.453 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.453 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.453 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.453 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "ending time of day to automatically balance", 2024-09-11T23:39:53.453 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-11T23:39:53.453 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.453 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.453 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.454 INFO:teuthology.orchestra.run.smithi026.stdout: "end_weekday": { 2024-09-11T23:39:53.454 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "end_weekday", 2024-09-11T23:39:53.454 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "uint", 2024-09-11T23:39:53.454 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.454 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.454 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2024-09-11T23:39:53.454 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "0", 2024-09-11T23:39:53.454 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "6", 2024-09-11T23:39:53.454 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.454 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-11T23:39:53.454 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-11T23:39:53.454 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.454 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.455 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.455 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-09-11T23:39:53.455 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-09-11T23:39:53.455 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.455 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.455 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.455 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:53.455 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.455 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.456 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:53.456 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:53.456 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:53.456 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:53.456 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:53.456 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:53.456 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:53.456 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:53.457 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.457 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.457 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.457 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.457 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.457 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-09-11T23:39:53.457 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-09-11T23:39:53.457 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.457 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.457 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.457 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:53.457 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.458 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.458 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.458 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.458 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.458 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.458 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.458 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.458 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-09-11T23:39:53.458 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-09-11T23:39:53.458 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.458 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.458 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.458 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-09-11T23:39:53.458 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.459 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.459 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:53.459 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:53.459 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:53.459 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:53.459 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:53.459 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:53.459 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:53.459 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:53.459 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.459 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.459 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.459 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.459 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.460 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-09-11T23:39:53.460 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-09-11T23:39:53.460 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.460 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.460 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.460 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:53.460 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.460 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.460 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.460 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.460 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.460 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.461 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.461 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.461 INFO:teuthology.orchestra.run.smithi026.stdout: "min_score": { 2024-09-11T23:39:53.461 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "min_score", 2024-09-11T23:39:53.461 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "float", 2024-09-11T23:39:53.461 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.461 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.461 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2024-09-11T23:39:53.461 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.461 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.461 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.461 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-11T23:39:53.461 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.461 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.462 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.462 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.462 INFO:teuthology.orchestra.run.smithi026.stdout: "mode": { 2024-09-11T23:39:53.462 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "mode", 2024-09-11T23:39:53.462 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.462 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.462 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.462 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "upmap", 2024-09-11T23:39:53.462 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.462 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.462 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:53.462 INFO:teuthology.orchestra.run.smithi026.stdout: "crush-compat", 2024-09-11T23:39:53.462 INFO:teuthology.orchestra.run.smithi026.stdout: "none", 2024-09-11T23:39:53.462 INFO:teuthology.orchestra.run.smithi026.stdout: "read", 2024-09-11T23:39:53.462 INFO:teuthology.orchestra.run.smithi026.stdout: "upmap", 2024-09-11T23:39:53.463 INFO:teuthology.orchestra.run.smithi026.stdout: "upmap-read" 2024-09-11T23:39:53.463 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:53.463 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Balancer mode", 2024-09-11T23:39:53.463 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.463 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.463 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.463 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.463 INFO:teuthology.orchestra.run.smithi026.stdout: "pool_ids": { 2024-09-11T23:39:53.463 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "pool_ids", 2024-09-11T23:39:53.463 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.463 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.463 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.463 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:53.463 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.464 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.464 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.464 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-11T23:39:53.464 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.464 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.464 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.464 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.464 INFO:teuthology.orchestra.run.smithi026.stdout: "sleep_interval": { 2024-09-11T23:39:53.464 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sleep_interval", 2024-09-11T23:39:53.464 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-09-11T23:39:53.464 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.464 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.464 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "60", 2024-09-11T23:39:53.464 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.465 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.465 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.465 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-11T23:39:53.465 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.465 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.465 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.465 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.465 INFO:teuthology.orchestra.run.smithi026.stdout: "upmap_max_deviation": { 2024-09-11T23:39:53.465 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "upmap_max_deviation", 2024-09-11T23:39:53.465 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-09-11T23:39:53.465 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.465 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.465 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "5", 2024-09-11T23:39:53.466 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "1", 2024-09-11T23:39:53.466 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.466 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.466 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-11T23:39:53.466 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-11T23:39:53.466 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.466 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.466 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.466 INFO:teuthology.orchestra.run.smithi026.stdout: "upmap_max_optimizations": { 2024-09-11T23:39:53.466 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "upmap_max_optimizations", 2024-09-11T23:39:53.466 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "uint", 2024-09-11T23:39:53.466 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.466 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.466 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "10", 2024-09-11T23:39:53.467 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.467 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.467 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.467 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-11T23:39:53.467 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.467 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.467 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.467 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:53.467 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:53.467 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.467 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-11T23:39:53.467 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "cephadm", 2024-09-11T23:39:53.467 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-09-11T23:39:53.467 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-09-11T23:39:53.468 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-09-11T23:39:53.468 INFO:teuthology.orchestra.run.smithi026.stdout: "agent_down_multiplier": { 2024-09-11T23:39:53.468 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "agent_down_multiplier", 2024-09-11T23:39:53.468 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "float", 2024-09-11T23:39:53.468 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.468 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.468 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "3.0", 2024-09-11T23:39:53.468 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.468 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.468 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.468 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-11T23:39:53.468 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.468 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.468 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.469 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.469 INFO:teuthology.orchestra.run.smithi026.stdout: "agent_refresh_rate": { 2024-09-11T23:39:53.469 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "agent_refresh_rate", 2024-09-11T23:39:53.469 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-09-11T23:39:53.469 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.469 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.469 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "20", 2024-09-11T23:39:53.469 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.469 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.469 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.469 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-11T23:39:53.469 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.469 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.470 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.470 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.470 INFO:teuthology.orchestra.run.smithi026.stdout: "agent_starting_port": { 2024-09-11T23:39:53.470 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "agent_starting_port", 2024-09-11T23:39:53.470 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-09-11T23:39:53.470 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.470 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.470 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "4721", 2024-09-11T23:39:53.470 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.470 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.470 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.470 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-11T23:39:53.470 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.471 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.471 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.471 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.471 INFO:teuthology.orchestra.run.smithi026.stdout: "allow_ptrace": { 2024-09-11T23:39:53.471 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "allow_ptrace", 2024-09-11T23:39:53.471 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.471 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.471 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.471 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:53.471 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.471 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.471 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.471 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-11T23:39:53.472 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-11T23:39:53.472 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.472 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.472 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.472 INFO:teuthology.orchestra.run.smithi026.stdout: "autotune_interval": { 2024-09-11T23:39:53.472 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "autotune_interval", 2024-09-11T23:39:53.472 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-09-11T23:39:53.472 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.472 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.473 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "600", 2024-09-11T23:39:53.473 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.473 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.473 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.473 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-11T23:39:53.473 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.473 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.473 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.473 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.473 INFO:teuthology.orchestra.run.smithi026.stdout: "autotune_memory_target_ratio": { 2024-09-11T23:39:53.473 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "autotune_memory_target_ratio", 2024-09-11T23:39:53.473 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "float", 2024-09-11T23:39:53.473 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.473 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.474 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0.7", 2024-09-11T23:39:53.474 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.474 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.474 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.474 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-11T23:39:53.474 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.474 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.474 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.474 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.474 INFO:teuthology.orchestra.run.smithi026.stdout: "cephadm_log_destination": { 2024-09-11T23:39:53.474 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "cephadm_log_destination", 2024-09-11T23:39:53.475 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.475 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.475 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.475 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:53.475 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.475 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.475 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:53.475 INFO:teuthology.orchestra.run.smithi026.stdout: "file", 2024-09-11T23:39:53.475 INFO:teuthology.orchestra.run.smithi026.stdout: "file,syslog", 2024-09-11T23:39:53.476 INFO:teuthology.orchestra.run.smithi026.stdout: "syslog" 2024-09-11T23:39:53.476 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:53.476 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-11T23:39:53.476 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.476 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.476 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.476 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.476 INFO:teuthology.orchestra.run.smithi026.stdout: "cgroups_split": { 2024-09-11T23:39:53.476 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "cgroups_split", 2024-09-11T23:39:53.476 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.476 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.476 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.476 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-09-11T23:39:53.477 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.477 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.477 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.477 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-11T23:39:53.477 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.477 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.477 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.477 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.477 INFO:teuthology.orchestra.run.smithi026.stdout: "config_checks_enabled": { 2024-09-11T23:39:53.477 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "config_checks_enabled", 2024-09-11T23:39:53.477 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.477 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.477 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.477 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:53.478 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.478 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.478 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.478 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-11T23:39:53.478 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.478 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.478 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.478 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.478 INFO:teuthology.orchestra.run.smithi026.stdout: "config_dashboard": { 2024-09-11T23:39:53.478 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "config_dashboard", 2024-09-11T23:39:53.479 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.479 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.479 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.479 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-09-11T23:39:53.479 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.479 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.479 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.479 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-11T23:39:53.479 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.479 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.479 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.479 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.479 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_alertmanager": { 2024-09-11T23:39:53.479 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_alertmanager", 2024-09-11T23:39:53.479 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.480 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.480 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.480 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-11T23:39:53.480 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.480 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.480 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.480 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Prometheus container image", 2024-09-11T23:39:53.480 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.480 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.480 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.480 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.480 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_base": { 2024-09-11T23:39:53.480 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_base", 2024-09-11T23:39:53.481 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.481 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.481 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.481 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-11T23:39:53.481 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.481 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.481 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.481 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Container image name, without the tag", 2024-09-11T23:39:53.481 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.481 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.481 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.481 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.481 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_elasticsearch": { 2024-09-11T23:39:53.481 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_elasticsearch", 2024-09-11T23:39:53.482 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.482 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.482 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.482 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-11T23:39:53.482 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.482 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.482 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.482 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "elasticsearch container image", 2024-09-11T23:39:53.482 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.482 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.482 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.482 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.482 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_grafana": { 2024-09-11T23:39:53.482 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_grafana", 2024-09-11T23:39:53.483 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.483 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.483 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.483 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-11T23:39:53.483 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.483 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.483 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.483 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Prometheus container image", 2024-09-11T23:39:53.483 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.483 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.483 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.483 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.483 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_haproxy": { 2024-09-11T23:39:53.483 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_haproxy", 2024-09-11T23:39:53.483 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.484 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.484 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.484 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-11T23:39:53.484 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.484 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.484 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.484 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "HAproxy container image", 2024-09-11T23:39:53.484 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.484 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.484 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.484 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.484 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_jaeger_agent": { 2024-09-11T23:39:53.485 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_jaeger_agent", 2024-09-11T23:39:53.485 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.485 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.485 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.485 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-11T23:39:53.485 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.485 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.485 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.485 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Jaeger agent container image", 2024-09-11T23:39:53.485 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.485 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.485 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.485 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.485 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_jaeger_collector": { 2024-09-11T23:39:53.486 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_jaeger_collector", 2024-09-11T23:39:53.486 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.486 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.486 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.486 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-11T23:39:53.486 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.486 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.486 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.486 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Jaeger collector container image", 2024-09-11T23:39:53.486 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.486 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.486 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.486 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.486 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_jaeger_query": { 2024-09-11T23:39:53.487 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_jaeger_query", 2024-09-11T23:39:53.487 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.487 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.487 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.487 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-11T23:39:53.487 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.487 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.487 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.487 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Jaeger query container image", 2024-09-11T23:39:53.487 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.487 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.487 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.487 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.487 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_keepalived": { 2024-09-11T23:39:53.487 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_keepalived", 2024-09-11T23:39:53.488 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.488 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.488 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.488 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-11T23:39:53.488 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.488 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.488 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.488 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Keepalived container image", 2024-09-11T23:39:53.488 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.488 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.488 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.488 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.488 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_loki": { 2024-09-11T23:39:53.489 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_loki", 2024-09-11T23:39:53.489 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.489 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.489 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.489 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-11T23:39:53.489 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.489 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.489 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.489 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Loki container image", 2024-09-11T23:39:53.489 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.490 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.490 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.490 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.490 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_node_exporter": { 2024-09-11T23:39:53.490 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_node_exporter", 2024-09-11T23:39:53.490 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.490 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.490 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.490 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-11T23:39:53.490 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.490 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.490 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.490 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Prometheus container image", 2024-09-11T23:39:53.490 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.491 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.491 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.491 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.491 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_nvmeof": { 2024-09-11T23:39:53.491 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_nvmeof", 2024-09-11T23:39:53.491 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.491 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.491 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.491 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-11T23:39:53.491 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.491 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.491 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.491 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Nvme-of container image", 2024-09-11T23:39:53.492 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.492 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.492 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.492 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.492 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_prometheus": { 2024-09-11T23:39:53.492 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_prometheus", 2024-09-11T23:39:53.492 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.492 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.492 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.492 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-11T23:39:53.492 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.492 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.492 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.492 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Prometheus container image", 2024-09-11T23:39:53.493 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.493 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.493 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.493 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.493 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_promtail": { 2024-09-11T23:39:53.493 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_promtail", 2024-09-11T23:39:53.493 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.493 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.493 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.493 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-11T23:39:53.493 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.493 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.493 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.494 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Promtail container image", 2024-09-11T23:39:53.494 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.494 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.494 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.494 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.494 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_samba": { 2024-09-11T23:39:53.494 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_samba", 2024-09-11T23:39:53.494 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.494 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.494 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.494 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-11T23:39:53.494 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.494 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.495 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.495 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Samba/SMB container image", 2024-09-11T23:39:53.495 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.495 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.495 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.495 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.495 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_snmp_gateway": { 2024-09-11T23:39:53.495 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_snmp_gateway", 2024-09-11T23:39:53.495 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.495 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.495 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.495 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-11T23:39:53.495 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.496 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.496 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.496 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "SNMP Gateway container image", 2024-09-11T23:39:53.496 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.496 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.496 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.496 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.496 INFO:teuthology.orchestra.run.smithi026.stdout: "container_init": { 2024-09-11T23:39:53.496 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_init", 2024-09-11T23:39:53.496 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.497 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.497 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.497 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-09-11T23:39:53.497 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.497 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.497 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.497 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Run podman/docker with `--init`", 2024-09-11T23:39:53.497 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.497 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.497 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.497 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.497 INFO:teuthology.orchestra.run.smithi026.stdout: "daemon_cache_timeout": { 2024-09-11T23:39:53.498 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "daemon_cache_timeout", 2024-09-11T23:39:53.498 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-09-11T23:39:53.498 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.498 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.498 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "600", 2024-09-11T23:39:53.498 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.498 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.498 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.498 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-11T23:39:53.498 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.498 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.498 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.498 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.499 INFO:teuthology.orchestra.run.smithi026.stdout: "default_cephadm_command_timeout": { 2024-09-11T23:39:53.499 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "default_cephadm_command_timeout", 2024-09-11T23:39:53.499 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-09-11T23:39:53.499 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.499 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.499 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "900", 2024-09-11T23:39:53.499 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.499 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.499 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.499 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-11T23:39:53.499 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.499 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.499 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.499 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.499 INFO:teuthology.orchestra.run.smithi026.stdout: "default_registry": { 2024-09-11T23:39:53.500 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "default_registry", 2024-09-11T23:39:53.500 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.500 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.500 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.500 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "docker.io", 2024-09-11T23:39:53.500 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.500 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.500 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.500 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-11T23:39:53.500 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.500 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.500 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.500 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.500 INFO:teuthology.orchestra.run.smithi026.stdout: "device_cache_timeout": { 2024-09-11T23:39:53.500 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "device_cache_timeout", 2024-09-11T23:39:53.501 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-09-11T23:39:53.501 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.501 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.501 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "1800", 2024-09-11T23:39:53.501 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.501 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.501 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.501 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "seconds to cache device inventory", 2024-09-11T23:39:53.501 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.501 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.501 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.501 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.501 INFO:teuthology.orchestra.run.smithi026.stdout: "device_enhanced_scan": { 2024-09-11T23:39:53.502 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "device_enhanced_scan", 2024-09-11T23:39:53.502 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.502 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.502 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.502 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:53.502 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.502 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.502 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.502 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-11T23:39:53.502 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.502 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.502 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.502 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.503 INFO:teuthology.orchestra.run.smithi026.stdout: "facts_cache_timeout": { 2024-09-11T23:39:53.503 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "facts_cache_timeout", 2024-09-11T23:39:53.503 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-09-11T23:39:53.503 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.503 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.503 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "60", 2024-09-11T23:39:53.503 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.503 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.503 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.503 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "seconds to cache host facts data", 2024-09-11T23:39:53.503 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.503 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.503 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.503 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.504 INFO:teuthology.orchestra.run.smithi026.stdout: "grafana_dashboards_path": { 2024-09-11T23:39:53.504 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "grafana_dashboards_path", 2024-09-11T23:39:53.504 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.504 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.504 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.504 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-11T23:39:53.504 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.504 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.504 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.504 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-11T23:39:53.504 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.504 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.504 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.505 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.505 INFO:teuthology.orchestra.run.smithi026.stdout: "host_check_interval": { 2024-09-11T23:39:53.505 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "host_check_interval", 2024-09-11T23:39:53.505 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-09-11T23:39:53.505 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.505 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.505 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "600", 2024-09-11T23:39:53.505 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.505 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.505 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.505 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "how frequently to perform a host check", 2024-09-11T23:39:53.505 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.505 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.505 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.506 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.506 INFO:teuthology.orchestra.run.smithi026.stdout: "hw_monitoring": { 2024-09-11T23:39:53.506 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "hw_monitoring", 2024-09-11T23:39:53.506 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.506 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.506 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.506 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:53.506 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.506 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.506 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.506 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-11T23:39:53.506 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.506 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.506 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.507 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.507 INFO:teuthology.orchestra.run.smithi026.stdout: "inventory_list_all": { 2024-09-11T23:39:53.507 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "inventory_list_all", 2024-09-11T23:39:53.507 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.507 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.507 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.507 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:53.507 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.507 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.507 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.507 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-11T23:39:53.507 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.507 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.507 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.507 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.508 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-09-11T23:39:53.508 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-09-11T23:39:53.508 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.508 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.508 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.508 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:53.508 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.508 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.508 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:53.508 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:53.508 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:53.508 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:53.508 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:53.509 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:53.509 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:53.509 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:53.509 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.509 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.509 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.509 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.509 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.509 INFO:teuthology.orchestra.run.smithi026.stdout: "log_refresh_metadata": { 2024-09-11T23:39:53.509 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_refresh_metadata", 2024-09-11T23:39:53.509 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.509 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.509 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.509 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:53.510 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.510 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.510 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.510 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-11T23:39:53.510 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.510 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.510 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.510 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.510 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-09-11T23:39:53.510 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-09-11T23:39:53.510 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.510 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.511 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.511 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-09-11T23:39:53.511 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.511 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.511 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.511 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-11T23:39:53.511 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.511 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.511 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.511 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.511 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-09-11T23:39:53.511 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-09-11T23:39:53.511 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.511 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.512 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.512 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-09-11T23:39:53.512 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.512 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.512 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:53.512 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:53.512 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:53.512 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:53.512 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:53.512 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:53.512 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:53.512 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:53.512 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.512 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.513 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.513 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.513 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.513 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-09-11T23:39:53.513 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-09-11T23:39:53.513 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.513 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.513 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.513 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:53.513 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.513 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.513 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.513 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.513 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.514 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.514 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.514 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.514 INFO:teuthology.orchestra.run.smithi026.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-11T23:39:53.514 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-11T23:39:53.514 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.514 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.514 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.514 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:53.514 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.514 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.514 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.514 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-11T23:39:53.514 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.514 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.515 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.515 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.515 INFO:teuthology.orchestra.run.smithi026.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-11T23:39:53.515 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-11T23:39:53.515 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.515 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.515 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.515 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "*", 2024-09-11T23:39:53.515 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.515 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.515 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.515 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-11T23:39:53.515 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.515 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.516 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.516 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.516 INFO:teuthology.orchestra.run.smithi026.stdout: "max_count_per_host": { 2024-09-11T23:39:53.516 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "max_count_per_host", 2024-09-11T23:39:53.516 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-09-11T23:39:53.516 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.516 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.516 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "10", 2024-09-11T23:39:53.516 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.516 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.516 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.516 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "max number of daemons per service per host", 2024-09-11T23:39:53.516 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.516 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.517 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.517 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.517 INFO:teuthology.orchestra.run.smithi026.stdout: "max_osd_draining_count": { 2024-09-11T23:39:53.517 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "max_osd_draining_count", 2024-09-11T23:39:53.517 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-09-11T23:39:53.517 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.517 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.517 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "10", 2024-09-11T23:39:53.517 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.517 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.517 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.517 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-11T23:39:53.517 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.517 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.518 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.518 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.518 INFO:teuthology.orchestra.run.smithi026.stdout: "migration_current": { 2024-09-11T23:39:53.518 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "migration_current", 2024-09-11T23:39:53.518 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-09-11T23:39:53.518 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.518 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.518 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:53.518 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.518 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.518 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.518 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "internal - do not modify", 2024-09-11T23:39:53.518 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.518 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.518 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.519 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.519 INFO:teuthology.orchestra.run.smithi026.stdout: "mode": { 2024-09-11T23:39:53.519 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "mode", 2024-09-11T23:39:53.519 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.519 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.519 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.519 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "root", 2024-09-11T23:39:53.519 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.519 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.519 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:53.519 INFO:teuthology.orchestra.run.smithi026.stdout: "cephadm-package", 2024-09-11T23:39:53.519 INFO:teuthology.orchestra.run.smithi026.stdout: "root" 2024-09-11T23:39:53.519 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:53.519 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "mode for remote execution of cephadm", 2024-09-11T23:39:53.520 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.520 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.520 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.520 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.520 INFO:teuthology.orchestra.run.smithi026.stdout: "oob_default_addr": { 2024-09-11T23:39:53.520 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "oob_default_addr", 2024-09-11T23:39:53.520 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.520 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.520 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.520 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "169.254.1.1", 2024-09-11T23:39:53.520 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.520 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.520 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.520 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-11T23:39:53.521 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.521 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.521 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.521 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.521 INFO:teuthology.orchestra.run.smithi026.stdout: "prometheus_alerts_path": { 2024-09-11T23:39:53.521 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "prometheus_alerts_path", 2024-09-11T23:39:53.521 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.521 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.521 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.521 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-11T23:39:53.521 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.521 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.521 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.521 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-11T23:39:53.522 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.522 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.522 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.522 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.522 INFO:teuthology.orchestra.run.smithi026.stdout: "registry_insecure": { 2024-09-11T23:39:53.522 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "registry_insecure", 2024-09-11T23:39:53.522 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.522 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.522 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.522 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:53.522 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.522 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.522 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.523 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-11T23:39:53.523 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.523 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.523 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.523 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.523 INFO:teuthology.orchestra.run.smithi026.stdout: "registry_password": { 2024-09-11T23:39:53.523 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "registry_password", 2024-09-11T23:39:53.523 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.523 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.523 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.523 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:53.523 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.523 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.523 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.524 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-11T23:39:53.524 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.524 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.524 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.524 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.524 INFO:teuthology.orchestra.run.smithi026.stdout: "registry_url": { 2024-09-11T23:39:53.524 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "registry_url", 2024-09-11T23:39:53.524 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.524 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.524 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.524 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:53.524 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.524 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.524 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.525 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-11T23:39:53.525 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.525 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.525 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.525 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.525 INFO:teuthology.orchestra.run.smithi026.stdout: "registry_username": { 2024-09-11T23:39:53.525 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "registry_username", 2024-09-11T23:39:53.525 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.525 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.525 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.525 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:53.525 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.525 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.525 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.526 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-11T23:39:53.526 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.526 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.526 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.526 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.526 INFO:teuthology.orchestra.run.smithi026.stdout: "secure_monitoring_stack": { 2024-09-11T23:39:53.526 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "secure_monitoring_stack", 2024-09-11T23:39:53.526 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.526 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.526 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.526 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:53.526 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.526 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.526 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.527 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-11T23:39:53.527 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.527 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.527 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.527 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.527 INFO:teuthology.orchestra.run.smithi026.stdout: "service_discovery_port": { 2024-09-11T23:39:53.527 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "service_discovery_port", 2024-09-11T23:39:53.527 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-09-11T23:39:53.527 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.527 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.527 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "8765", 2024-09-11T23:39:53.527 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.527 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.527 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.528 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "cephadm service discovery port", 2024-09-11T23:39:53.528 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.528 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.528 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.528 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.528 INFO:teuthology.orchestra.run.smithi026.stdout: "ssh_config_file": { 2024-09-11T23:39:53.528 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ssh_config_file", 2024-09-11T23:39:53.528 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.528 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.528 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.528 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:53.528 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.528 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.528 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.529 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-11T23:39:53.529 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.529 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.529 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.529 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.529 INFO:teuthology.orchestra.run.smithi026.stdout: "use_agent": { 2024-09-11T23:39:53.529 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "use_agent", 2024-09-11T23:39:53.529 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.529 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.529 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.529 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:53.529 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.529 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.529 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.530 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-11T23:39:53.530 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.530 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.530 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.530 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.530 INFO:teuthology.orchestra.run.smithi026.stdout: "use_repo_digest": { 2024-09-11T23:39:53.530 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "use_repo_digest", 2024-09-11T23:39:53.530 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.530 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.530 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.530 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-09-11T23:39:53.530 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.530 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.530 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.531 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-11T23:39:53.531 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.531 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.531 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.531 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.531 INFO:teuthology.orchestra.run.smithi026.stdout: "warn_on_failed_host_check": { 2024-09-11T23:39:53.531 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "warn_on_failed_host_check", 2024-09-11T23:39:53.531 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.531 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.531 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.531 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-09-11T23:39:53.531 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.531 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.532 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.532 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "raise a health warning if the host check fails", 2024-09-11T23:39:53.532 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.532 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.532 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.532 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.532 INFO:teuthology.orchestra.run.smithi026.stdout: "warn_on_stray_daemons": { 2024-09-11T23:39:53.532 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "warn_on_stray_daemons", 2024-09-11T23:39:53.532 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.532 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.532 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.532 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-09-11T23:39:53.532 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.533 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.533 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.533 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-11T23:39:53.533 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.533 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.533 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.533 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.533 INFO:teuthology.orchestra.run.smithi026.stdout: "warn_on_stray_hosts": { 2024-09-11T23:39:53.533 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "warn_on_stray_hosts", 2024-09-11T23:39:53.533 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.533 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.533 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.533 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-09-11T23:39:53.533 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.534 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.534 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.534 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-11T23:39:53.534 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.534 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.534 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.534 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:53.534 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:53.534 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.534 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-11T23:39:53.534 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "crash", 2024-09-11T23:39:53.534 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-09-11T23:39:53.535 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-09-11T23:39:53.535 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-09-11T23:39:53.535 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-09-11T23:39:53.535 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-09-11T23:39:53.535 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.535 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.535 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.535 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:53.535 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.535 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.535 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:53.535 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:53.535 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:53.535 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:53.536 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:53.536 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:53.536 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:53.536 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:53.536 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.536 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.536 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.536 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.536 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.536 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-09-11T23:39:53.536 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-09-11T23:39:53.536 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.537 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.537 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.537 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:53.537 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.537 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.537 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.537 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.537 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.537 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.537 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.537 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.537 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-09-11T23:39:53.537 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-09-11T23:39:53.538 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.538 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.538 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.538 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-09-11T23:39:53.538 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.538 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.538 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:53.538 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:53.538 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:53.538 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:53.538 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:53.538 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:53.538 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:53.538 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:53.539 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.539 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.539 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.539 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.539 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.539 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-09-11T23:39:53.539 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-09-11T23:39:53.539 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.539 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.539 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.539 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:53.539 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.539 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.539 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.540 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.540 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.540 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.540 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.540 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.540 INFO:teuthology.orchestra.run.smithi026.stdout: "retain_interval": { 2024-09-11T23:39:53.540 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "retain_interval", 2024-09-11T23:39:53.540 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-09-11T23:39:53.540 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.540 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.540 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "31536000", 2024-09-11T23:39:53.540 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.540 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.540 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.541 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-11T23:39:53.541 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.541 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.541 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.541 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.541 INFO:teuthology.orchestra.run.smithi026.stdout: "warn_recent_interval": { 2024-09-11T23:39:53.541 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "warn_recent_interval", 2024-09-11T23:39:53.541 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-09-11T23:39:53.541 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.541 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.541 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "1209600", 2024-09-11T23:39:53.541 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.541 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.541 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.542 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-11T23:39:53.542 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.542 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.542 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.542 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:53.542 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:53.542 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.542 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-11T23:39:53.542 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "dashboard", 2024-09-11T23:39:53.542 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-09-11T23:39:53.542 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-09-11T23:39:53.542 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-09-11T23:39:53.542 INFO:teuthology.orchestra.run.smithi026.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-11T23:39:53.542 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-11T23:39:53.543 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-09-11T23:39:53.543 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.543 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.543 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "10", 2024-09-11T23:39:53.543 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.543 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.543 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.543 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.543 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.543 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.543 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.543 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.543 INFO:teuthology.orchestra.run.smithi026.stdout: "ALERTMANAGER_API_HOST": { 2024-09-11T23:39:53.543 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-11T23:39:53.543 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.544 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.544 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.544 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:53.544 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.544 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.544 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.544 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.544 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.544 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.544 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.544 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.544 INFO:teuthology.orchestra.run.smithi026.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-11T23:39:53.544 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-11T23:39:53.544 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.545 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.545 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.545 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-09-11T23:39:53.545 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.545 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.545 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.545 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.545 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.545 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.545 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.545 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.545 INFO:teuthology.orchestra.run.smithi026.stdout: "AUDIT_API_ENABLED": { 2024-09-11T23:39:53.545 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "AUDIT_API_ENABLED", 2024-09-11T23:39:53.545 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.546 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.546 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.546 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:53.546 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.546 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.546 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.546 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.546 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.546 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.546 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.546 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.546 INFO:teuthology.orchestra.run.smithi026.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-11T23:39:53.546 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-11T23:39:53.546 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.546 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.547 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.547 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-09-11T23:39:53.547 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.547 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.547 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.547 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.547 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.547 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.547 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.547 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.547 INFO:teuthology.orchestra.run.smithi026.stdout: "ENABLE_BROWSABLE_API": { 2024-09-11T23:39:53.547 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-11T23:39:53.547 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.547 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.548 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.548 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-09-11T23:39:53.548 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.548 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.548 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.548 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.548 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.548 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.548 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.548 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.548 INFO:teuthology.orchestra.run.smithi026.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-11T23:39:53.548 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-11T23:39:53.548 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.549 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.549 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.549 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-09-11T23:39:53.549 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.549 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.549 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.549 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.549 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.552 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.552 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.552 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.552 INFO:teuthology.orchestra.run.smithi026.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-11T23:39:53.553 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-11T23:39:53.553 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.553 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.553 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.553 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-09-11T23:39:53.553 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.553 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.553 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.553 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.553 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.553 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.553 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.553 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.553 INFO:teuthology.orchestra.run.smithi026.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-11T23:39:53.553 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-11T23:39:53.554 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.554 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.554 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.554 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-09-11T23:39:53.554 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.554 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.554 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.554 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.554 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.554 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.554 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.554 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.554 INFO:teuthology.orchestra.run.smithi026.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-11T23:39:53.555 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-11T23:39:53.555 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.555 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.555 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.555 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-09-11T23:39:53.555 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.555 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.555 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.555 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.555 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.555 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.555 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.555 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.555 INFO:teuthology.orchestra.run.smithi026.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-11T23:39:53.556 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-11T23:39:53.556 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.556 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.556 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.556 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-09-11T23:39:53.556 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.556 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.556 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.556 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.556 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.556 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.556 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.556 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.557 INFO:teuthology.orchestra.run.smithi026.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-11T23:39:53.557 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-11T23:39:53.557 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.557 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.557 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.557 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-09-11T23:39:53.557 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.557 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.557 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.557 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.557 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.557 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.557 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.557 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.558 INFO:teuthology.orchestra.run.smithi026.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-11T23:39:53.558 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-11T23:39:53.558 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.558 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.558 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.558 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-09-11T23:39:53.558 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.558 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.558 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.558 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.558 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.558 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.558 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.559 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.559 INFO:teuthology.orchestra.run.smithi026.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-11T23:39:53.559 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-11T23:39:53.559 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.559 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.559 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.559 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:53.559 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.559 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.559 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.559 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.559 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.559 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.559 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.560 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.560 INFO:teuthology.orchestra.run.smithi026.stdout: "GRAFANA_API_PASSWORD": { 2024-09-11T23:39:53.560 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-11T23:39:53.560 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.560 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.560 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.560 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "admin", 2024-09-11T23:39:53.560 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.560 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.560 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.560 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.560 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.560 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.560 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.561 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.561 INFO:teuthology.orchestra.run.smithi026.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-11T23:39:53.561 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-11T23:39:53.561 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.561 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.561 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.561 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-09-11T23:39:53.561 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.561 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.561 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.561 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.561 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.561 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.561 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.562 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.562 INFO:teuthology.orchestra.run.smithi026.stdout: "GRAFANA_API_URL": { 2024-09-11T23:39:53.562 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "GRAFANA_API_URL", 2024-09-11T23:39:53.562 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.562 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.562 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.562 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:53.562 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.562 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.562 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.562 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.562 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.562 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.563 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.563 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.563 INFO:teuthology.orchestra.run.smithi026.stdout: "GRAFANA_API_USERNAME": { 2024-09-11T23:39:53.563 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-11T23:39:53.563 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.563 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.563 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.563 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "admin", 2024-09-11T23:39:53.563 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.563 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.563 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.563 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.563 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.563 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.564 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.564 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.564 INFO:teuthology.orchestra.run.smithi026.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-11T23:39:53.564 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-11T23:39:53.564 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.564 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.564 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.564 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:53.564 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.564 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.564 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.564 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.564 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.564 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.565 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.565 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.565 INFO:teuthology.orchestra.run.smithi026.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-11T23:39:53.565 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-11T23:39:53.565 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.565 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.565 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.565 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:53.565 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.565 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.565 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.565 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.565 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.565 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.566 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.566 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.566 INFO:teuthology.orchestra.run.smithi026.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-11T23:39:53.566 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-11T23:39:53.566 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.566 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.566 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.566 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-09-11T23:39:53.566 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.566 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.566 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.566 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.566 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.566 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.566 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.567 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.567 INFO:teuthology.orchestra.run.smithi026.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-11T23:39:53.567 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-11T23:39:53.567 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.567 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.567 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.567 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:53.567 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.567 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.567 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.567 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.567 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.567 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.567 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.568 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.568 INFO:teuthology.orchestra.run.smithi026.stdout: "PROMETHEUS_API_HOST": { 2024-09-11T23:39:53.568 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-11T23:39:53.568 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.568 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.568 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.568 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:53.568 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.568 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.568 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.568 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.568 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.569 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.569 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.569 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.569 INFO:teuthology.orchestra.run.smithi026.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-11T23:39:53.569 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-11T23:39:53.569 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.569 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.569 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.569 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-09-11T23:39:53.569 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.569 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.569 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.569 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.570 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.570 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.570 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.570 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.570 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-11T23:39:53.570 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-11T23:39:53.570 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.570 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.570 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.570 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:53.570 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.570 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.570 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.570 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.571 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.571 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.571 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.571 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.571 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-11T23:39:53.571 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-11T23:39:53.571 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.571 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.571 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.571 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:53.571 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.571 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.571 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.571 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.571 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.572 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.572 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.572 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.572 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-11T23:39:53.572 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-11T23:39:53.572 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.572 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.572 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.572 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-09-11T23:39:53.572 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.572 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.572 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.572 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.572 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.572 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.573 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.573 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.573 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-11T23:39:53.573 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-11T23:39:53.573 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.573 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.573 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.573 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-09-11T23:39:53.573 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.573 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.573 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.573 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.573 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.574 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.574 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.574 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.574 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-11T23:39:53.574 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-11T23:39:53.574 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.574 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.574 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.574 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:53.574 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.574 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.574 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.574 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.574 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.575 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.575 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.575 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.575 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-11T23:39:53.575 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-11T23:39:53.575 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.575 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.575 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.575 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:53.575 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.575 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.575 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.575 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.575 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.576 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.576 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.576 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.576 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-11T23:39:53.576 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-11T23:39:53.576 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.576 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.576 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.576 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:53.576 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.576 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.576 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.576 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.576 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.577 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.577 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.577 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.577 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_ENABLED": { 2024-09-11T23:39:53.577 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-11T23:39:53.577 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.577 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.577 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.577 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-09-11T23:39:53.577 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.577 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.577 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.577 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.578 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.578 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.578 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.578 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.578 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-11T23:39:53.578 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-11T23:39:53.578 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.578 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.578 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.578 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-11T23:39:53.578 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.578 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.578 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.579 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.579 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.579 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.579 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.579 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.579 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-11T23:39:53.579 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-11T23:39:53.579 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-09-11T23:39:53.579 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.579 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.579 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "10", 2024-09-11T23:39:53.579 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.579 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.579 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.579 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.580 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.580 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.580 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.580 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.580 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-11T23:39:53.580 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-11T23:39:53.580 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-09-11T23:39:53.580 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.580 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.580 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "8", 2024-09-11T23:39:53.580 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.580 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.580 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.580 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.581 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.581 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.581 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.581 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.581 INFO:teuthology.orchestra.run.smithi026.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-11T23:39:53.581 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-11T23:39:53.581 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-09-11T23:39:53.581 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.581 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.581 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "45", 2024-09-11T23:39:53.581 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.581 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.581 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.581 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.582 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.582 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.582 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.582 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.582 INFO:teuthology.orchestra.run.smithi026.stdout: "RGW_API_ACCESS_KEY": { 2024-09-11T23:39:53.582 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-11T23:39:53.582 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.582 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.582 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.582 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:53.582 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.582 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.582 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.582 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.583 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.583 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.583 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.583 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.583 INFO:teuthology.orchestra.run.smithi026.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-11T23:39:53.583 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-11T23:39:53.583 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.583 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.583 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.583 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "admin", 2024-09-11T23:39:53.583 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.583 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.583 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.583 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.583 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.584 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.584 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.584 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.584 INFO:teuthology.orchestra.run.smithi026.stdout: "RGW_API_SECRET_KEY": { 2024-09-11T23:39:53.584 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-11T23:39:53.584 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.584 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.584 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.584 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:53.584 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.584 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.584 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.584 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.584 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.585 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.585 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.585 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.585 INFO:teuthology.orchestra.run.smithi026.stdout: "RGW_API_SSL_VERIFY": { 2024-09-11T23:39:53.585 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-11T23:39:53.585 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.585 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.585 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.585 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-09-11T23:39:53.585 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.585 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.585 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.585 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.585 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.586 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.586 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.586 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.586 INFO:teuthology.orchestra.run.smithi026.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-11T23:39:53.586 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-11T23:39:53.586 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-09-11T23:39:53.586 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.586 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.586 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2024-09-11T23:39:53.586 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.586 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.586 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.586 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.586 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.586 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.587 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.587 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.587 INFO:teuthology.orchestra.run.smithi026.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-11T23:39:53.587 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-11T23:39:53.587 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-09-11T23:39:53.587 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.587 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.587 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "10", 2024-09-11T23:39:53.587 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.587 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.587 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.587 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.587 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.587 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.587 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.588 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.588 INFO:teuthology.orchestra.run.smithi026.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-11T23:39:53.588 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-11T23:39:53.588 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-09-11T23:39:53.588 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.588 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.588 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "5", 2024-09-11T23:39:53.588 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.588 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.588 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.588 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.588 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.588 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.588 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.589 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.589 INFO:teuthology.orchestra.run.smithi026.stdout: "cross_origin_url": { 2024-09-11T23:39:53.589 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "cross_origin_url", 2024-09-11T23:39:53.589 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.589 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.589 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.589 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:53.589 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.589 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.589 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.589 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.589 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.589 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.589 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.590 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.590 INFO:teuthology.orchestra.run.smithi026.stdout: "crt_file": { 2024-09-11T23:39:53.590 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "crt_file", 2024-09-11T23:39:53.590 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.590 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.590 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.590 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:53.590 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.590 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.590 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.590 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.590 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.590 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.590 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.591 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.591 INFO:teuthology.orchestra.run.smithi026.stdout: "debug": { 2024-09-11T23:39:53.591 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "debug", 2024-09-11T23:39:53.591 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.591 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.591 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.591 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:53.591 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.591 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.591 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.591 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Enable/disable debug options", 2024-09-11T23:39:53.591 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.591 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.591 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.592 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.592 INFO:teuthology.orchestra.run.smithi026.stdout: "jwt_token_ttl": { 2024-09-11T23:39:53.592 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "jwt_token_ttl", 2024-09-11T23:39:53.592 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-09-11T23:39:53.592 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.592 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.592 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "28800", 2024-09-11T23:39:53.592 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.592 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.592 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.592 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.592 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.592 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.592 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.593 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.593 INFO:teuthology.orchestra.run.smithi026.stdout: "key_file": { 2024-09-11T23:39:53.593 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "key_file", 2024-09-11T23:39:53.593 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.593 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.593 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.593 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:53.593 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.593 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.593 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.593 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.593 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.593 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.593 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.594 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.594 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-09-11T23:39:53.594 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-09-11T23:39:53.594 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.594 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.594 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.594 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:53.594 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.594 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.594 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:53.594 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:53.594 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:53.594 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:53.594 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:53.594 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:53.595 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:53.595 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:53.595 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.595 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.595 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.595 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.595 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.595 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-09-11T23:39:53.595 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-09-11T23:39:53.595 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.595 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.595 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.595 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:53.595 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.596 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.596 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.596 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.596 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.596 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.596 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.596 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.596 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-09-11T23:39:53.596 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-09-11T23:39:53.596 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.596 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.596 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.596 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-09-11T23:39:53.596 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.597 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.597 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:53.597 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:53.597 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:53.597 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:53.597 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:53.597 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:53.597 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:53.597 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:53.597 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.597 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.597 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.597 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.597 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.597 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-09-11T23:39:53.598 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-09-11T23:39:53.598 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.598 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.598 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.598 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:53.598 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.598 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.598 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.598 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.598 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.598 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.598 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.598 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.598 INFO:teuthology.orchestra.run.smithi026.stdout: "motd": { 2024-09-11T23:39:53.598 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "motd", 2024-09-11T23:39:53.599 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.599 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.599 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.599 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:53.599 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.599 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.599 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.599 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "The message of the day", 2024-09-11T23:39:53.599 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.599 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.599 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.599 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.599 INFO:teuthology.orchestra.run.smithi026.stdout: "redirect_resolve_ip_addr": { 2024-09-11T23:39:53.599 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "redirect_resolve_ip_addr", 2024-09-11T23:39:53.600 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.600 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.600 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.600 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:53.600 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.600 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.600 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.600 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.600 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.600 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.600 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.600 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.600 INFO:teuthology.orchestra.run.smithi026.stdout: "server_addr": { 2024-09-11T23:39:53.600 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "server_addr", 2024-09-11T23:39:53.601 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.601 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.601 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.601 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "::", 2024-09-11T23:39:53.601 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.601 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.601 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.601 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.601 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.601 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.601 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.601 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.601 INFO:teuthology.orchestra.run.smithi026.stdout: "server_port": { 2024-09-11T23:39:53.601 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "server_port", 2024-09-11T23:39:53.602 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-09-11T23:39:53.602 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.602 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.602 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "8080", 2024-09-11T23:39:53.602 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.602 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.602 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.602 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.602 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.602 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.602 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.602 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.602 INFO:teuthology.orchestra.run.smithi026.stdout: "ssl": { 2024-09-11T23:39:53.602 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ssl", 2024-09-11T23:39:53.603 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.603 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.603 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.603 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-09-11T23:39:53.603 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.603 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.603 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.603 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.603 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.603 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.603 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.603 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.603 INFO:teuthology.orchestra.run.smithi026.stdout: "ssl_server_port": { 2024-09-11T23:39:53.603 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ssl_server_port", 2024-09-11T23:39:53.603 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-09-11T23:39:53.604 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.604 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.604 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "8443", 2024-09-11T23:39:53.604 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.604 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.604 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.604 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.604 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.604 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.604 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.604 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.604 INFO:teuthology.orchestra.run.smithi026.stdout: "standby_behaviour": { 2024-09-11T23:39:53.605 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "standby_behaviour", 2024-09-11T23:39:53.605 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.605 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.605 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.605 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "redirect", 2024-09-11T23:39:53.605 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.605 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.605 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:53.605 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:53.605 INFO:teuthology.orchestra.run.smithi026.stdout: "redirect" 2024-09-11T23:39:53.605 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:53.605 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.605 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.605 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.606 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.606 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.606 INFO:teuthology.orchestra.run.smithi026.stdout: "standby_error_status_code": { 2024-09-11T23:39:53.606 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "standby_error_status_code", 2024-09-11T23:39:53.606 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-09-11T23:39:53.606 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.606 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.606 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "500", 2024-09-11T23:39:53.606 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "400", 2024-09-11T23:39:53.606 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "599", 2024-09-11T23:39:53.606 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.606 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.606 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.607 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.607 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.607 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.607 INFO:teuthology.orchestra.run.smithi026.stdout: "url_prefix": { 2024-09-11T23:39:53.607 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "url_prefix", 2024-09-11T23:39:53.607 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.607 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.607 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.607 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:53.607 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.607 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.607 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.607 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.607 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.607 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.608 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.608 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:53.608 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:53.608 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.608 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-11T23:39:53.608 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "devicehealth", 2024-09-11T23:39:53.608 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-09-11T23:39:53.608 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-09-11T23:39:53.608 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-09-11T23:39:53.608 INFO:teuthology.orchestra.run.smithi026.stdout: "enable_monitoring": { 2024-09-11T23:39:53.608 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "enable_monitoring", 2024-09-11T23:39:53.608 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.608 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.608 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.609 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-09-11T23:39:53.609 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.609 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.609 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.609 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "monitor device health metrics", 2024-09-11T23:39:53.609 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.609 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.609 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.609 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.609 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-09-11T23:39:53.609 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-09-11T23:39:53.609 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.609 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.609 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.610 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:53.610 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.610 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.610 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:53.610 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:53.610 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:53.610 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:53.610 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:53.610 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:53.610 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:53.610 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:53.610 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.610 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.611 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.611 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.611 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.611 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-09-11T23:39:53.611 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-09-11T23:39:53.611 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.611 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.611 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.611 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:53.611 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.611 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.611 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.611 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.611 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.612 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.612 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.612 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.612 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-09-11T23:39:53.612 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-09-11T23:39:53.612 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.612 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.612 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.612 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-09-11T23:39:53.612 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.612 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.612 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:53.612 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:53.612 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:53.613 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:53.613 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:53.613 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:53.613 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:53.613 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:53.613 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.613 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.613 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.613 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.613 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.613 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-09-11T23:39:53.613 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-09-11T23:39:53.613 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.613 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.613 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.614 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:53.614 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.614 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.614 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.614 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.614 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.614 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.614 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.614 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.614 INFO:teuthology.orchestra.run.smithi026.stdout: "mark_out_threshold": { 2024-09-11T23:39:53.614 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "mark_out_threshold", 2024-09-11T23:39:53.614 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-09-11T23:39:53.614 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.614 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.615 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "2419200", 2024-09-11T23:39:53.615 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.615 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.615 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.615 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-11T23:39:53.615 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.615 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.615 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.615 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.615 INFO:teuthology.orchestra.run.smithi026.stdout: "pool_name": { 2024-09-11T23:39:53.615 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "pool_name", 2024-09-11T23:39:53.615 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.615 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.615 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.615 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "device_health_metrics", 2024-09-11T23:39:53.616 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.616 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.616 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.616 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-11T23:39:53.616 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.616 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.616 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.616 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.616 INFO:teuthology.orchestra.run.smithi026.stdout: "retention_period": { 2024-09-11T23:39:53.616 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "retention_period", 2024-09-11T23:39:53.616 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-09-11T23:39:53.616 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.616 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.616 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "15552000", 2024-09-11T23:39:53.617 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.617 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.617 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.617 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "how long to retain device health metrics", 2024-09-11T23:39:53.617 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.617 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.617 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.617 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.617 INFO:teuthology.orchestra.run.smithi026.stdout: "scrape_frequency": { 2024-09-11T23:39:53.617 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "scrape_frequency", 2024-09-11T23:39:53.617 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-09-11T23:39:53.617 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.617 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.617 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "86400", 2024-09-11T23:39:53.617 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.618 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.618 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.618 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-11T23:39:53.618 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.618 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.618 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.618 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.618 INFO:teuthology.orchestra.run.smithi026.stdout: "self_heal": { 2024-09-11T23:39:53.618 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "self_heal", 2024-09-11T23:39:53.618 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.618 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.618 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.618 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-09-11T23:39:53.619 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.619 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.619 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.619 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-11T23:39:53.619 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.619 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.619 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.619 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.619 INFO:teuthology.orchestra.run.smithi026.stdout: "sleep_interval": { 2024-09-11T23:39:53.619 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sleep_interval", 2024-09-11T23:39:53.619 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-09-11T23:39:53.619 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.619 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.619 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "600", 2024-09-11T23:39:53.620 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.620 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.620 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.620 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "how frequently to wake up and check device health", 2024-09-11T23:39:53.620 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.620 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.620 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.620 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.620 INFO:teuthology.orchestra.run.smithi026.stdout: "warn_threshold": { 2024-09-11T23:39:53.620 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "warn_threshold", 2024-09-11T23:39:53.620 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-09-11T23:39:53.620 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.620 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.620 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "7257600", 2024-09-11T23:39:53.621 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.621 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.621 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.621 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-11T23:39:53.621 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.621 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.621 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.621 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:53.621 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:53.621 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.621 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-11T23:39:53.621 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "diskprediction_local", 2024-09-11T23:39:53.621 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-09-11T23:39:53.621 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-09-11T23:39:53.621 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-09-11T23:39:53.622 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-09-11T23:39:53.622 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-09-11T23:39:53.622 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.622 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.622 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.622 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:53.622 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.622 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.622 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:53.622 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:53.622 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:53.622 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:53.622 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:53.622 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:53.623 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:53.623 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:53.623 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.623 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.623 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.623 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.623 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.623 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-09-11T23:39:53.623 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-09-11T23:39:53.623 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.623 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.623 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.623 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:53.623 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.624 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.624 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.624 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.624 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.624 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.624 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.624 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.624 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-09-11T23:39:53.624 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-09-11T23:39:53.624 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.624 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.624 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.624 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-09-11T23:39:53.624 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.624 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.625 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:53.625 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:53.625 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:53.625 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:53.625 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:53.625 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:53.625 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:53.625 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:53.625 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.625 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.625 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.625 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.625 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.625 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-09-11T23:39:53.626 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-09-11T23:39:53.626 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.626 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.626 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.626 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:53.626 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.626 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.626 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.626 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.626 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.626 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.626 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.626 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.626 INFO:teuthology.orchestra.run.smithi026.stdout: "predict_interval": { 2024-09-11T23:39:53.627 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "predict_interval", 2024-09-11T23:39:53.627 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.627 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.627 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.627 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "86400", 2024-09-11T23:39:53.627 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.627 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.627 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.627 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.627 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.627 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.627 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.627 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.627 INFO:teuthology.orchestra.run.smithi026.stdout: "predictor_model": { 2024-09-11T23:39:53.628 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "predictor_model", 2024-09-11T23:39:53.628 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.628 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.628 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.628 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "prophetstor", 2024-09-11T23:39:53.628 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.628 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.628 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.628 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.628 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.628 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.628 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.628 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.628 INFO:teuthology.orchestra.run.smithi026.stdout: "sleep_interval": { 2024-09-11T23:39:53.628 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sleep_interval", 2024-09-11T23:39:53.629 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.629 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.629 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.629 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "600", 2024-09-11T23:39:53.629 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.629 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.629 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.629 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.629 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.629 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.629 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.629 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:53.629 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:53.629 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.630 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-11T23:39:53.630 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "influx", 2024-09-11T23:39:53.630 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": false, 2024-09-11T23:39:53.630 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "influxdb python module not found", 2024-09-11T23:39:53.630 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-09-11T23:39:53.630 INFO:teuthology.orchestra.run.smithi026.stdout: "batch_size": { 2024-09-11T23:39:53.630 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "batch_size", 2024-09-11T23:39:53.630 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-09-11T23:39:53.630 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.630 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.630 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "5000", 2024-09-11T23:39:53.630 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.630 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.630 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.630 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-11T23:39:53.631 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.631 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.631 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.631 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.631 INFO:teuthology.orchestra.run.smithi026.stdout: "database": { 2024-09-11T23:39:53.631 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "database", 2024-09-11T23:39:53.631 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.631 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.631 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.631 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "ceph", 2024-09-11T23:39:53.631 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.631 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.631 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.632 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-11T23:39:53.632 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.632 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.632 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.632 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.632 INFO:teuthology.orchestra.run.smithi026.stdout: "hostname": { 2024-09-11T23:39:53.632 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "hostname", 2024-09-11T23:39:53.632 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.632 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.632 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.632 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:53.632 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.632 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.633 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.633 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "InfluxDB server hostname", 2024-09-11T23:39:53.633 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.633 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.633 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.633 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.633 INFO:teuthology.orchestra.run.smithi026.stdout: "interval": { 2024-09-11T23:39:53.633 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "interval", 2024-09-11T23:39:53.633 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-09-11T23:39:53.633 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.633 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.633 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "30", 2024-09-11T23:39:53.633 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "5", 2024-09-11T23:39:53.633 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.634 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.634 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-11T23:39:53.634 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.634 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.634 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.634 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.634 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-09-11T23:39:53.634 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-09-11T23:39:53.634 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.634 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.634 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.634 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:53.634 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.634 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.635 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:53.635 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:53.635 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:53.635 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:53.635 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:53.635 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:53.635 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:53.635 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:53.635 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.635 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.635 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.635 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.635 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.636 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-09-11T23:39:53.636 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-09-11T23:39:53.636 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.636 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.636 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.636 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:53.636 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.636 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.636 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.636 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.636 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.636 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.636 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.636 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.637 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-09-11T23:39:53.637 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-09-11T23:39:53.637 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.637 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.637 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.637 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-09-11T23:39:53.637 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.637 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.637 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:53.637 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:53.637 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:53.637 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:53.637 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:53.637 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:53.638 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:53.638 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:53.638 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.638 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.638 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.638 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.638 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.638 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-09-11T23:39:53.638 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-09-11T23:39:53.638 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.638 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.638 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.638 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:53.638 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.639 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.639 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.639 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.639 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.639 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.639 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.639 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.639 INFO:teuthology.orchestra.run.smithi026.stdout: "password": { 2024-09-11T23:39:53.639 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "password", 2024-09-11T23:39:53.639 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.639 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.639 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.639 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:53.639 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.639 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.640 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.640 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "password of InfluxDB server user", 2024-09-11T23:39:53.640 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.640 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.640 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.640 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.640 INFO:teuthology.orchestra.run.smithi026.stdout: "port": { 2024-09-11T23:39:53.640 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "port", 2024-09-11T23:39:53.640 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-09-11T23:39:53.640 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.640 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.640 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "8086", 2024-09-11T23:39:53.640 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.640 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.641 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.641 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "InfluxDB server port", 2024-09-11T23:39:53.641 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.641 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.641 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.641 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.641 INFO:teuthology.orchestra.run.smithi026.stdout: "ssl": { 2024-09-11T23:39:53.641 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ssl", 2024-09-11T23:39:53.641 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.641 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.641 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.641 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "false", 2024-09-11T23:39:53.641 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.641 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.642 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.642 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-11T23:39:53.642 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.642 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.642 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.642 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.642 INFO:teuthology.orchestra.run.smithi026.stdout: "threads": { 2024-09-11T23:39:53.642 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "threads", 2024-09-11T23:39:53.642 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-09-11T23:39:53.642 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.642 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.642 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "5", 2024-09-11T23:39:53.642 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "1", 2024-09-11T23:39:53.642 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "32", 2024-09-11T23:39:53.643 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.643 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-11T23:39:53.643 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.643 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.643 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.643 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.643 INFO:teuthology.orchestra.run.smithi026.stdout: "username": { 2024-09-11T23:39:53.643 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "username", 2024-09-11T23:39:53.643 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.643 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.643 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.643 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:53.643 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.643 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.643 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.644 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "username of InfluxDB server user", 2024-09-11T23:39:53.644 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.644 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.644 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.644 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.644 INFO:teuthology.orchestra.run.smithi026.stdout: "verify_ssl": { 2024-09-11T23:39:53.644 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "verify_ssl", 2024-09-11T23:39:53.644 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.644 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.644 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.644 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "true", 2024-09-11T23:39:53.644 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.644 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.645 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.645 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-11T23:39:53.645 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.645 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.645 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.645 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:53.645 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:53.645 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.645 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-11T23:39:53.645 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "insights", 2024-09-11T23:39:53.645 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-09-11T23:39:53.645 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-09-11T23:39:53.645 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-09-11T23:39:53.645 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-09-11T23:39:53.646 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-09-11T23:39:53.646 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.646 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.646 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.646 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:53.646 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.646 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.646 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:53.646 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:53.646 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:53.646 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:53.646 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:53.646 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:53.646 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:53.647 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:53.647 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.647 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.647 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.647 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.647 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.647 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-09-11T23:39:53.647 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-09-11T23:39:53.647 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.647 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.647 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.647 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:53.647 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.647 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.647 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.648 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.648 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.648 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.648 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.648 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.648 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-09-11T23:39:53.648 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-09-11T23:39:53.648 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.648 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.648 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.648 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-09-11T23:39:53.648 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.648 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.648 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:53.649 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:53.649 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:53.649 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:53.649 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:53.649 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:53.649 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:53.649 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:53.649 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.649 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.649 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.649 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.649 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.649 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-09-11T23:39:53.649 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-09-11T23:39:53.650 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.650 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.650 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.650 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:53.650 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.650 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.650 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.650 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.650 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.650 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.650 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.650 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:53.650 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:53.650 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.651 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-11T23:39:53.651 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "iostat", 2024-09-11T23:39:53.651 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-09-11T23:39:53.651 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-09-11T23:39:53.651 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-09-11T23:39:53.651 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-09-11T23:39:53.651 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-09-11T23:39:53.651 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.651 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.651 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.651 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:53.651 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.651 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.651 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:53.652 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:53.652 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:53.652 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:53.652 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:53.652 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:53.652 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:53.652 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:53.652 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.652 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.652 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.652 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.652 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.652 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-09-11T23:39:53.652 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-09-11T23:39:53.653 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.653 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.653 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.653 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:53.653 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.653 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.653 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.653 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.653 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.653 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.653 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.653 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.653 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-09-11T23:39:53.653 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-09-11T23:39:53.653 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.654 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.654 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.654 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-09-11T23:39:53.654 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.654 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.654 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:53.654 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:53.654 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:53.654 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:53.654 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:53.654 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:53.654 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:53.654 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:53.654 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.655 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.655 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.655 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.655 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.655 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-09-11T23:39:53.655 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-09-11T23:39:53.655 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.655 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.655 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.655 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:53.655 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.655 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.655 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.655 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.656 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.656 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.656 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.656 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:53.656 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:53.656 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.656 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-11T23:39:53.656 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "k8sevents", 2024-09-11T23:39:53.656 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-09-11T23:39:53.656 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-09-11T23:39:53.656 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-09-11T23:39:53.656 INFO:teuthology.orchestra.run.smithi026.stdout: "ceph_event_retention_days": { 2024-09-11T23:39:53.656 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ceph_event_retention_days", 2024-09-11T23:39:53.656 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-09-11T23:39:53.657 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.657 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.657 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "7", 2024-09-11T23:39:53.657 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.657 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.657 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.657 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-11T23:39:53.657 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.657 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.657 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.657 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.657 INFO:teuthology.orchestra.run.smithi026.stdout: "config_check_secs": { 2024-09-11T23:39:53.657 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "config_check_secs", 2024-09-11T23:39:53.657 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-09-11T23:39:53.658 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.658 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.658 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "10", 2024-09-11T23:39:53.658 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "10", 2024-09-11T23:39:53.658 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.658 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.658 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-11T23:39:53.658 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.658 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.658 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.658 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.658 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-09-11T23:39:53.658 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-09-11T23:39:53.659 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.659 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.659 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.659 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:53.659 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.659 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.659 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:53.659 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:53.659 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:53.659 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:53.659 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:53.659 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:53.659 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:53.659 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:53.660 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.660 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.660 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.660 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.660 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.660 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-09-11T23:39:53.660 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-09-11T23:39:53.660 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.660 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.660 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.660 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:53.660 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.660 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.660 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.661 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.661 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.661 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.661 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.661 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.661 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-09-11T23:39:53.661 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-09-11T23:39:53.661 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.661 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.661 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.661 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-09-11T23:39:53.661 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.661 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.661 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:53.662 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:53.662 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:53.662 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:53.662 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:53.662 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:53.662 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:53.662 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:53.662 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.662 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.662 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.662 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.662 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.662 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-09-11T23:39:53.662 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-09-11T23:39:53.663 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.663 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.663 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.663 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:53.663 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.663 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.663 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.663 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.663 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.663 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.663 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.663 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:53.663 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:53.663 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.664 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-11T23:39:53.664 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "localpool", 2024-09-11T23:39:53.664 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-09-11T23:39:53.664 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-09-11T23:39:53.664 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-09-11T23:39:53.664 INFO:teuthology.orchestra.run.smithi026.stdout: "failure_domain": { 2024-09-11T23:39:53.664 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "failure_domain", 2024-09-11T23:39:53.664 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.664 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.664 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.664 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "host", 2024-09-11T23:39:53.664 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.664 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.664 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.664 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "failure domain for any created local pool", 2024-09-11T23:39:53.665 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-11T23:39:53.665 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.665 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.665 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.665 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-09-11T23:39:53.665 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-09-11T23:39:53.665 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.665 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.665 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.665 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:53.665 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.665 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.665 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:53.665 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:53.666 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:53.666 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:53.666 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:53.666 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:53.666 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:53.666 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:53.666 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.666 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.666 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.666 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.666 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.666 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-09-11T23:39:53.666 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-09-11T23:39:53.666 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.666 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.667 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.667 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:53.667 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.667 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.667 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.667 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.667 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.667 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.667 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.667 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.667 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-09-11T23:39:53.667 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-09-11T23:39:53.667 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.667 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.668 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.668 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-09-11T23:39:53.668 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.668 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.668 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:53.668 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:53.668 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:53.668 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:53.668 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:53.668 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:53.668 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:53.668 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:53.668 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.668 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.669 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.669 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.669 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.669 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-09-11T23:39:53.669 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-09-11T23:39:53.669 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.669 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.669 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.669 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:53.669 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.669 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.669 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.669 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.669 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.670 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.670 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.670 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.670 INFO:teuthology.orchestra.run.smithi026.stdout: "min_size": { 2024-09-11T23:39:53.670 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "min_size", 2024-09-11T23:39:53.670 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-09-11T23:39:53.670 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.670 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.670 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:53.670 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.670 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.670 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.670 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "default min_size for any created local pool", 2024-09-11T23:39:53.670 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-11T23:39:53.671 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.671 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.671 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.671 INFO:teuthology.orchestra.run.smithi026.stdout: "num_rep": { 2024-09-11T23:39:53.671 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "num_rep", 2024-09-11T23:39:53.671 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-09-11T23:39:53.671 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.671 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.671 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "3", 2024-09-11T23:39:53.671 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.671 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.671 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.672 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "default replica count for any created local pool", 2024-09-11T23:39:53.672 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.672 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.672 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.672 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.672 INFO:teuthology.orchestra.run.smithi026.stdout: "pg_num": { 2024-09-11T23:39:53.672 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "pg_num", 2024-09-11T23:39:53.672 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-09-11T23:39:53.672 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.672 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.672 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "128", 2024-09-11T23:39:53.672 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.672 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.672 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.673 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "default pg_num for any created local pool", 2024-09-11T23:39:53.673 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.673 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.673 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.673 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.673 INFO:teuthology.orchestra.run.smithi026.stdout: "prefix": { 2024-09-11T23:39:53.673 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "prefix", 2024-09-11T23:39:53.673 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.673 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.673 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.673 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:53.673 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.673 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.673 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.674 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "name prefix for any created local pool", 2024-09-11T23:39:53.674 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.674 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.674 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.674 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.674 INFO:teuthology.orchestra.run.smithi026.stdout: "subtree": { 2024-09-11T23:39:53.674 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "subtree", 2024-09-11T23:39:53.674 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.674 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.674 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.674 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "rack", 2024-09-11T23:39:53.674 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.674 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.674 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.674 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-11T23:39:53.675 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-11T23:39:53.675 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.675 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.675 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:53.675 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:53.675 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.675 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-11T23:39:53.675 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "mds_autoscaler", 2024-09-11T23:39:53.675 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-09-11T23:39:53.675 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-09-11T23:39:53.675 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-09-11T23:39:53.675 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-09-11T23:39:53.675 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-09-11T23:39:53.675 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.676 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.676 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.676 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:53.676 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.676 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.676 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:53.676 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:53.676 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:53.676 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:53.676 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:53.676 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:53.676 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:53.676 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:53.677 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.677 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.677 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.677 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.677 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.677 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-09-11T23:39:53.677 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-09-11T23:39:53.677 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.677 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.677 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.677 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:53.677 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.677 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.677 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.678 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.678 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.678 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.678 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.678 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.678 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-09-11T23:39:53.678 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-09-11T23:39:53.678 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.678 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.678 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.678 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-09-11T23:39:53.678 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.678 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.679 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:53.679 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:53.679 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:53.679 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:53.679 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:53.679 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:53.679 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:53.679 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:53.679 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.679 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.679 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.679 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.679 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.679 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-09-11T23:39:53.679 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-09-11T23:39:53.680 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.680 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.680 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.680 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:53.680 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.680 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.680 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.680 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.680 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.680 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.680 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.680 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:53.680 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:53.680 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.681 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-11T23:39:53.681 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "mirroring", 2024-09-11T23:39:53.681 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-09-11T23:39:53.681 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-09-11T23:39:53.681 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-09-11T23:39:53.681 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-09-11T23:39:53.681 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-09-11T23:39:53.681 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.681 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.681 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.681 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:53.681 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.681 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.681 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:53.682 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:53.682 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:53.682 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:53.682 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:53.682 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:53.682 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:53.682 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:53.682 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.682 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.682 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.682 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.682 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.682 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-09-11T23:39:53.682 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-09-11T23:39:53.682 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.683 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.683 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.683 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:53.683 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.683 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.683 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.683 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.683 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.683 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.683 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.683 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.683 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-09-11T23:39:53.683 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-09-11T23:39:53.683 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.684 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.684 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.684 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-09-11T23:39:53.684 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.684 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.684 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:53.684 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:53.684 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:53.684 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:53.684 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:53.684 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:53.684 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:53.684 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:53.684 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.685 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.685 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.685 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.685 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.685 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-09-11T23:39:53.685 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-09-11T23:39:53.685 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.685 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.685 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.685 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:53.685 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.685 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.685 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.686 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.686 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.686 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.686 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.686 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:53.686 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:53.686 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.686 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-11T23:39:53.686 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "nfs", 2024-09-11T23:39:53.686 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-09-11T23:39:53.686 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-09-11T23:39:53.686 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-09-11T23:39:53.686 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-09-11T23:39:53.686 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-09-11T23:39:53.687 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.687 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.687 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.687 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:53.687 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.687 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.687 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:53.687 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:53.687 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:53.687 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:53.687 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:53.687 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:53.687 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:53.687 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:53.688 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.688 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.688 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.688 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.688 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.688 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-09-11T23:39:53.688 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-09-11T23:39:53.688 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.688 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.688 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.688 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:53.688 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.688 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.688 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.689 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.689 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.689 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.689 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.689 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.689 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-09-11T23:39:53.689 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-09-11T23:39:53.689 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.689 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.689 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.689 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-09-11T23:39:53.689 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.689 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.689 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:53.690 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:53.690 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:53.690 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:53.690 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:53.690 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:53.690 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:53.690 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:53.690 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.690 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.690 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.690 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.690 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.690 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-09-11T23:39:53.690 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-09-11T23:39:53.691 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.691 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.691 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.691 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:53.691 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.691 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.691 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.691 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.691 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.691 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.691 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.691 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:53.691 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:53.691 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.691 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-11T23:39:53.692 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "orchestrator", 2024-09-11T23:39:53.692 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-09-11T23:39:53.692 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-09-11T23:39:53.692 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-09-11T23:39:53.692 INFO:teuthology.orchestra.run.smithi026.stdout: "fail_fs": { 2024-09-11T23:39:53.692 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "fail_fs", 2024-09-11T23:39:53.692 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.692 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.692 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.692 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:53.692 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.692 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.692 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.693 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-11T23:39:53.693 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.693 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.693 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.693 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.693 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-09-11T23:39:53.693 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-09-11T23:39:53.693 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.693 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.693 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.693 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:53.693 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.693 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.694 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:53.694 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:53.694 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:53.694 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:53.694 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:53.694 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:53.694 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:53.694 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:53.694 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.694 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.694 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.694 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.694 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.694 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-09-11T23:39:53.695 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-09-11T23:39:53.695 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.695 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.695 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.695 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:53.695 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.695 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.695 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.695 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.695 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.695 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.695 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.695 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.695 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-09-11T23:39:53.695 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-09-11T23:39:53.696 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.696 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.696 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.696 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-09-11T23:39:53.696 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.696 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.696 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:53.696 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:53.696 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:53.696 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:53.696 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:53.696 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:53.696 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:53.696 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:53.697 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.697 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.697 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.697 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.697 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.697 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-09-11T23:39:53.697 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-09-11T23:39:53.697 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.697 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.697 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.697 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:53.697 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.697 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.697 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.698 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.698 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.698 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.698 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.698 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.698 INFO:teuthology.orchestra.run.smithi026.stdout: "orchestrator": { 2024-09-11T23:39:53.698 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "orchestrator", 2024-09-11T23:39:53.698 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.698 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.698 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.698 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:53.698 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.698 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.698 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:53.699 INFO:teuthology.orchestra.run.smithi026.stdout: "cephadm", 2024-09-11T23:39:53.699 INFO:teuthology.orchestra.run.smithi026.stdout: "rook", 2024-09-11T23:39:53.699 INFO:teuthology.orchestra.run.smithi026.stdout: "test_orchestrator" 2024-09-11T23:39:53.699 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:53.699 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Orchestrator backend", 2024-09-11T23:39:53.699 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.699 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.699 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.699 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:53.699 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:53.699 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.699 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-11T23:39:53.699 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "osd_perf_query", 2024-09-11T23:39:53.700 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-09-11T23:39:53.700 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-09-11T23:39:53.700 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-09-11T23:39:53.700 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-09-11T23:39:53.700 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-09-11T23:39:53.700 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.700 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.700 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.700 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:53.700 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.700 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.700 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:53.700 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:53.700 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:53.701 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:53.701 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:53.701 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:53.701 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:53.701 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:53.701 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.701 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.701 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.701 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.701 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.701 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-09-11T23:39:53.701 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-09-11T23:39:53.701 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.701 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.702 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.702 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:53.702 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.702 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.702 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.702 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.702 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.702 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.702 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.702 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.702 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-09-11T23:39:53.702 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-09-11T23:39:53.702 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.702 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.703 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.703 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-09-11T23:39:53.703 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.703 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.703 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:53.703 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:53.703 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:53.703 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:53.703 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:53.703 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:53.703 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:53.703 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:53.703 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.703 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.704 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.704 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.704 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.704 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-09-11T23:39:53.704 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-09-11T23:39:53.704 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.704 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.704 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.704 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:53.704 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.704 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.704 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.704 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.704 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.705 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.705 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.705 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:53.705 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:53.705 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.705 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-11T23:39:53.705 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "osd_support", 2024-09-11T23:39:53.705 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-09-11T23:39:53.705 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-09-11T23:39:53.705 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-09-11T23:39:53.705 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-09-11T23:39:53.705 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-09-11T23:39:53.705 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.706 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.706 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.706 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:53.706 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.706 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.706 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:53.706 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:53.706 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:53.706 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:53.706 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:53.706 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:53.706 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:53.706 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:53.707 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.707 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.707 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.707 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.707 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.707 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-09-11T23:39:53.707 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-09-11T23:39:53.707 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.707 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.707 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.707 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:53.707 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.707 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.707 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.708 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.708 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.708 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.708 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.708 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.708 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-09-11T23:39:53.708 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-09-11T23:39:53.708 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.708 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.708 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.708 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-09-11T23:39:53.708 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.709 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.709 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:53.709 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:53.709 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:53.709 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:53.709 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:53.709 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:53.709 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:53.709 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:53.709 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.709 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.709 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.709 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.710 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.710 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-09-11T23:39:53.710 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-09-11T23:39:53.710 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.710 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.710 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.710 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:53.710 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.710 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.710 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.710 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.710 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.710 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.711 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.711 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:53.711 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:53.711 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.711 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-11T23:39:53.711 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "pg_autoscaler", 2024-09-11T23:39:53.711 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-09-11T23:39:53.711 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-09-11T23:39:53.711 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-09-11T23:39:53.711 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-09-11T23:39:53.711 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-09-11T23:39:53.711 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.711 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.711 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.712 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:53.712 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.712 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.712 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:53.712 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:53.712 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:53.712 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:53.712 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:53.712 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:53.712 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:53.712 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:53.712 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.712 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.712 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.712 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.713 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.713 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-09-11T23:39:53.713 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-09-11T23:39:53.713 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.713 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.713 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.713 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:53.713 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.713 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.713 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.713 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.713 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.713 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.713 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.714 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.714 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-09-11T23:39:53.714 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-09-11T23:39:53.714 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.714 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.714 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.714 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-09-11T23:39:53.714 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.714 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.714 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:53.714 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:53.714 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:53.714 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:53.715 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:53.715 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:53.715 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:53.715 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:53.715 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.715 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.715 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.715 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.715 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.715 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-09-11T23:39:53.715 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-09-11T23:39:53.715 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.715 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.715 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.716 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:53.716 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.716 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.716 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.716 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.716 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.716 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.716 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.716 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.716 INFO:teuthology.orchestra.run.smithi026.stdout: "sleep_interval": { 2024-09-11T23:39:53.716 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sleep_interval", 2024-09-11T23:39:53.716 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-09-11T23:39:53.716 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.716 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.717 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "60", 2024-09-11T23:39:53.717 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.717 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.717 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.717 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.717 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.717 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.717 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.717 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.717 INFO:teuthology.orchestra.run.smithi026.stdout: "threshold": { 2024-09-11T23:39:53.717 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "threshold", 2024-09-11T23:39:53.717 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "float", 2024-09-11T23:39:53.717 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.717 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.718 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "3.0", 2024-09-11T23:39:53.718 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "1.0", 2024-09-11T23:39:53.718 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.718 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.718 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "scaling threshold", 2024-09-11T23:39:53.718 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-11T23:39:53.718 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.718 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.718 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:53.718 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:53.718 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.718 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-11T23:39:53.718 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "progress", 2024-09-11T23:39:53.718 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-09-11T23:39:53.719 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-09-11T23:39:53.719 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-09-11T23:39:53.719 INFO:teuthology.orchestra.run.smithi026.stdout: "allow_pg_recovery_event": { 2024-09-11T23:39:53.719 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "allow_pg_recovery_event", 2024-09-11T23:39:53.719 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.719 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.719 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.719 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:53.719 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.719 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.719 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.719 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-11T23:39:53.719 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.720 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.720 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.720 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.720 INFO:teuthology.orchestra.run.smithi026.stdout: "enabled": { 2024-09-11T23:39:53.720 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "enabled", 2024-09-11T23:39:53.720 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.720 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.720 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.720 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-09-11T23:39:53.720 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.720 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.720 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.720 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.720 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.721 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.721 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.721 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.721 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-09-11T23:39:53.721 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-09-11T23:39:53.721 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.721 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.721 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.721 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:53.721 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.721 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.721 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:53.721 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:53.722 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:53.722 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:53.722 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:53.722 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:53.722 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:53.722 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:53.722 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.722 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.722 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.722 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.722 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.722 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-09-11T23:39:53.722 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-09-11T23:39:53.722 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.723 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.723 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.723 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:53.723 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.723 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.723 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.723 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.723 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.723 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.723 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.723 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.723 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-09-11T23:39:53.723 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-09-11T23:39:53.723 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.724 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.724 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.724 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-09-11T23:39:53.724 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.724 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.724 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:53.724 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:53.724 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:53.724 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:53.724 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:53.724 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:53.724 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:53.724 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:53.724 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.725 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.725 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.725 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.725 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.725 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-09-11T23:39:53.725 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-09-11T23:39:53.725 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.725 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.725 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.725 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:53.725 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.725 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.725 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.726 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.726 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.726 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.726 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.726 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.726 INFO:teuthology.orchestra.run.smithi026.stdout: "max_completed_events": { 2024-09-11T23:39:53.726 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "max_completed_events", 2024-09-11T23:39:53.726 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-09-11T23:39:53.726 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.726 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.726 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "50", 2024-09-11T23:39:53.726 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.726 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.726 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.726 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "number of past completed events to remember", 2024-09-11T23:39:53.727 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.727 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.727 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.727 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.727 INFO:teuthology.orchestra.run.smithi026.stdout: "sleep_interval": { 2024-09-11T23:39:53.727 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sleep_interval", 2024-09-11T23:39:53.727 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-09-11T23:39:53.727 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.727 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.727 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "5", 2024-09-11T23:39:53.727 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.727 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.727 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.728 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "how long the module is going to sleep", 2024-09-11T23:39:53.728 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.728 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.728 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.728 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:53.728 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:53.728 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.728 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-11T23:39:53.728 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "prometheus", 2024-09-11T23:39:53.728 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-09-11T23:39:53.728 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-09-11T23:39:53.728 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-09-11T23:39:53.728 INFO:teuthology.orchestra.run.smithi026.stdout: "cache": { 2024-09-11T23:39:53.729 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "cache", 2024-09-11T23:39:53.729 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.729 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.729 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.729 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-09-11T23:39:53.729 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.729 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.729 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.729 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.729 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.729 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.729 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.729 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.730 INFO:teuthology.orchestra.run.smithi026.stdout: "exclude_perf_counters": { 2024-09-11T23:39:53.730 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "exclude_perf_counters", 2024-09-11T23:39:53.730 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.730 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.730 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.730 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-09-11T23:39:53.730 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.730 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.730 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.730 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-11T23:39:53.730 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-11T23:39:53.730 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.730 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.730 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.731 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-09-11T23:39:53.731 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-09-11T23:39:53.731 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.731 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.731 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.731 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:53.731 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.731 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.731 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:53.731 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:53.731 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:53.731 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:53.731 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:53.731 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:53.731 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:53.732 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:53.732 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.732 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.732 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.732 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.732 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.732 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-09-11T23:39:53.732 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-09-11T23:39:53.732 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.732 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.732 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.732 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:53.733 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.733 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.733 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.733 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.733 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.733 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.733 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.733 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.733 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-09-11T23:39:53.733 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-09-11T23:39:53.733 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.733 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.733 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.734 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-09-11T23:39:53.734 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.734 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.734 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:53.734 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:53.734 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:53.734 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:53.734 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:53.734 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:53.734 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:53.734 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:53.734 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.734 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.734 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.735 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.735 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.735 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-09-11T23:39:53.735 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-09-11T23:39:53.735 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.735 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.735 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.735 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:53.735 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.735 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.735 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.735 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.735 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.735 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.735 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.736 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.736 INFO:teuthology.orchestra.run.smithi026.stdout: "rbd_stats_pools": { 2024-09-11T23:39:53.736 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rbd_stats_pools", 2024-09-11T23:39:53.736 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.736 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.736 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.736 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:53.736 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.736 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.736 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.736 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.736 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.736 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.736 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.737 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.737 INFO:teuthology.orchestra.run.smithi026.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-11T23:39:53.737 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-11T23:39:53.737 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-09-11T23:39:53.737 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.737 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.737 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "300", 2024-09-11T23:39:53.737 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.737 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.737 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.737 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.737 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.737 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.738 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.738 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.738 INFO:teuthology.orchestra.run.smithi026.stdout: "scrape_interval": { 2024-09-11T23:39:53.738 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "scrape_interval", 2024-09-11T23:39:53.738 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "float", 2024-09-11T23:39:53.738 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.738 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.738 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "15.0", 2024-09-11T23:39:53.738 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.738 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.738 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.738 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.738 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.738 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.739 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.739 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.739 INFO:teuthology.orchestra.run.smithi026.stdout: "server_addr": { 2024-09-11T23:39:53.739 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "server_addr", 2024-09-11T23:39:53.739 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.739 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.739 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.739 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "::", 2024-09-11T23:39:53.739 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.739 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.739 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.739 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-11T23:39:53.739 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.739 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.740 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.740 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.740 INFO:teuthology.orchestra.run.smithi026.stdout: "server_port": { 2024-09-11T23:39:53.740 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "server_port", 2024-09-11T23:39:53.740 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-09-11T23:39:53.740 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.740 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.740 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "9283", 2024-09-11T23:39:53.740 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.740 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.740 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.740 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-11T23:39:53.740 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.740 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.741 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.741 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.741 INFO:teuthology.orchestra.run.smithi026.stdout: "stale_cache_strategy": { 2024-09-11T23:39:53.741 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "stale_cache_strategy", 2024-09-11T23:39:53.741 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.741 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.741 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.741 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "log", 2024-09-11T23:39:53.741 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.741 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.741 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.741 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.741 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.741 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.742 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.742 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.742 INFO:teuthology.orchestra.run.smithi026.stdout: "standby_behaviour": { 2024-09-11T23:39:53.742 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "standby_behaviour", 2024-09-11T23:39:53.742 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.742 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.742 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.742 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "default", 2024-09-11T23:39:53.742 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.742 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.742 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:53.742 INFO:teuthology.orchestra.run.smithi026.stdout: "default", 2024-09-11T23:39:53.742 INFO:teuthology.orchestra.run.smithi026.stdout: "error" 2024-09-11T23:39:53.742 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:53.743 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.743 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.743 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.743 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.743 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.743 INFO:teuthology.orchestra.run.smithi026.stdout: "standby_error_status_code": { 2024-09-11T23:39:53.743 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "standby_error_status_code", 2024-09-11T23:39:53.743 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-09-11T23:39:53.743 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.743 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.743 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "500", 2024-09-11T23:39:53.743 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "400", 2024-09-11T23:39:53.743 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "599", 2024-09-11T23:39:53.743 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.743 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.744 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.744 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.744 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.744 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:53.744 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:53.744 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.744 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-11T23:39:53.744 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rbd_support", 2024-09-11T23:39:53.744 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-09-11T23:39:53.744 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-09-11T23:39:53.744 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-09-11T23:39:53.744 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-09-11T23:39:53.744 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-09-11T23:39:53.744 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.745 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.745 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.745 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:53.745 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.745 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.745 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:53.745 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:53.745 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:53.745 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:53.745 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:53.745 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:53.745 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:53.745 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:53.746 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.746 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.746 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.746 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.746 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.746 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-09-11T23:39:53.746 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-09-11T23:39:53.746 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.746 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.746 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.746 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:53.746 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.746 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.746 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.747 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.747 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.747 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.747 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.747 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.747 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-09-11T23:39:53.747 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-09-11T23:39:53.747 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.747 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.747 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.747 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-09-11T23:39:53.747 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.747 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.747 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:53.748 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:53.748 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:53.748 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:53.748 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:53.748 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:53.748 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:53.748 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:53.748 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.748 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.748 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.748 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.748 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.748 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-09-11T23:39:53.748 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-09-11T23:39:53.749 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.749 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.749 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.749 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:53.749 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.749 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.749 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.749 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.749 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.749 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.749 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.749 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.749 INFO:teuthology.orchestra.run.smithi026.stdout: "max_concurrent_snap_create": { 2024-09-11T23:39:53.749 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "max_concurrent_snap_create", 2024-09-11T23:39:53.749 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-09-11T23:39:53.750 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.750 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.750 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "10", 2024-09-11T23:39:53.750 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.750 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.750 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.750 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.750 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.750 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.750 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.750 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.750 INFO:teuthology.orchestra.run.smithi026.stdout: "mirror_snapshot_schedule": { 2024-09-11T23:39:53.750 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "mirror_snapshot_schedule", 2024-09-11T23:39:53.750 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.751 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.751 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.751 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:53.751 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.751 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.751 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.751 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.751 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.751 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.751 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.751 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.751 INFO:teuthology.orchestra.run.smithi026.stdout: "trash_purge_schedule": { 2024-09-11T23:39:53.751 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "trash_purge_schedule", 2024-09-11T23:39:53.752 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.752 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.752 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.752 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:53.752 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.752 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.752 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.752 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.752 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.752 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.752 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.752 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:53.752 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:53.752 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.753 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-11T23:39:53.753 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "restful", 2024-09-11T23:39:53.753 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-09-11T23:39:53.753 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-09-11T23:39:53.753 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-09-11T23:39:53.753 INFO:teuthology.orchestra.run.smithi026.stdout: "enable_auth": { 2024-09-11T23:39:53.753 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "enable_auth", 2024-09-11T23:39:53.753 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.753 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.753 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.753 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-09-11T23:39:53.753 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.753 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.753 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.754 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.754 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.754 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.754 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.754 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.754 INFO:teuthology.orchestra.run.smithi026.stdout: "key_file": { 2024-09-11T23:39:53.754 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "key_file", 2024-09-11T23:39:53.754 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.754 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.754 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.754 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:53.754 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.754 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.754 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.755 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.755 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.755 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.755 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.755 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.755 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-09-11T23:39:53.755 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-09-11T23:39:53.755 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.755 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.756 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.756 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:53.756 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.756 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.756 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:53.756 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:53.756 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:53.756 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:53.756 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:53.756 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:53.756 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:53.756 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:53.756 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.756 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.757 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.757 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.757 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.757 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-09-11T23:39:53.757 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-09-11T23:39:53.757 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.757 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.757 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.757 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:53.757 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.757 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.757 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.757 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.757 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.758 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.758 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.758 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.758 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-09-11T23:39:53.758 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-09-11T23:39:53.758 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.758 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.758 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.758 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-09-11T23:39:53.758 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.758 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.758 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:53.758 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:53.758 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:53.759 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:53.759 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:53.759 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:53.759 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:53.759 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:53.759 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.759 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.759 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.759 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.759 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.759 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-09-11T23:39:53.759 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-09-11T23:39:53.759 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.759 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.760 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.760 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:53.760 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.760 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.760 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.760 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.760 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.760 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.760 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.760 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.760 INFO:teuthology.orchestra.run.smithi026.stdout: "server_addr": { 2024-09-11T23:39:53.760 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "server_addr", 2024-09-11T23:39:53.760 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.761 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.761 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.761 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:53.761 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.761 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.761 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.761 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.761 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.761 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.761 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.761 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.761 INFO:teuthology.orchestra.run.smithi026.stdout: "server_port": { 2024-09-11T23:39:53.761 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "server_port", 2024-09-11T23:39:53.762 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.762 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.762 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.762 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:53.762 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.762 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.762 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.762 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.762 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.762 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.762 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.762 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:53.762 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:53.762 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.762 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-11T23:39:53.763 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rgw", 2024-09-11T23:39:53.763 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-09-11T23:39:53.763 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-09-11T23:39:53.763 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-09-11T23:39:53.763 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-09-11T23:39:53.763 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-09-11T23:39:53.763 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.763 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.763 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.763 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:53.763 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.763 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.763 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:53.763 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:53.764 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:53.764 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:53.764 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:53.764 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:53.764 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:53.764 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:53.764 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.764 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.764 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.764 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.764 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.764 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-09-11T23:39:53.764 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-09-11T23:39:53.764 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.765 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.765 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.765 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:53.765 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.765 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.765 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.765 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.765 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.765 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.765 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.765 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.765 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-09-11T23:39:53.765 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-09-11T23:39:53.765 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.766 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.766 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.766 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-09-11T23:39:53.766 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.766 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.766 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:53.766 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:53.766 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:53.766 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:53.766 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:53.766 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:53.766 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:53.766 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:53.766 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.767 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.767 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.767 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.767 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.767 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-09-11T23:39:53.767 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-09-11T23:39:53.767 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.767 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.767 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.767 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:53.767 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.767 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.767 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.767 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.768 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.768 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.768 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.768 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:53.768 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:53.768 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.768 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-11T23:39:53.768 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rook", 2024-09-11T23:39:53.768 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-09-11T23:39:53.768 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-09-11T23:39:53.768 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-09-11T23:39:53.768 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-09-11T23:39:53.768 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-09-11T23:39:53.768 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.769 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.769 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.769 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:53.769 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.769 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.769 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:53.769 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:53.769 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:53.769 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:53.769 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:53.769 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:53.769 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:53.769 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:53.770 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.770 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.770 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.770 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.770 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.770 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-09-11T23:39:53.770 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-09-11T23:39:53.770 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.770 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.770 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.770 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:53.770 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.770 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.771 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.771 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.771 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.771 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.771 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.771 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.771 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-09-11T23:39:53.771 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-09-11T23:39:53.771 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.771 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.771 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.771 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-09-11T23:39:53.771 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.771 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.772 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:53.772 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:53.772 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:53.772 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:53.772 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:53.772 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:53.772 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:53.772 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:53.772 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.772 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.772 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.772 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.772 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.772 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-09-11T23:39:53.773 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-09-11T23:39:53.773 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.773 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.773 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.773 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:53.773 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.773 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.773 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.773 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.773 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.773 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.773 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.773 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.773 INFO:teuthology.orchestra.run.smithi026.stdout: "storage_class": { 2024-09-11T23:39:53.773 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "storage_class", 2024-09-11T23:39:53.774 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.774 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.774 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.774 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "local", 2024-09-11T23:39:53.774 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.774 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.774 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.774 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-11T23:39:53.774 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.774 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.774 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.774 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:53.774 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:53.775 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.775 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-11T23:39:53.775 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "selftest", 2024-09-11T23:39:53.775 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-09-11T23:39:53.775 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-09-11T23:39:53.775 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-09-11T23:39:53.775 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-09-11T23:39:53.775 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-09-11T23:39:53.775 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.775 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.775 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.775 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:53.775 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.775 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.776 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:53.776 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:53.776 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:53.776 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:53.776 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:53.776 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:53.776 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:53.776 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:53.776 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.776 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.776 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.776 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.776 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.776 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-09-11T23:39:53.777 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-09-11T23:39:53.777 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.777 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.777 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.777 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:53.777 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.777 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.777 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.777 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.777 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.777 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.777 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.777 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.777 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-09-11T23:39:53.778 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-09-11T23:39:53.778 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.778 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.778 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.778 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-09-11T23:39:53.778 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.778 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.778 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:53.778 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:53.778 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:53.778 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:53.778 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:53.778 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:53.778 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:53.779 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:53.779 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.779 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.779 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.779 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.779 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.779 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-09-11T23:39:53.779 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-09-11T23:39:53.779 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.779 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.779 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.779 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:53.779 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.779 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.780 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.780 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.780 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.780 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.780 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.780 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.780 INFO:teuthology.orchestra.run.smithi026.stdout: "roption1": { 2024-09-11T23:39:53.780 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "roption1", 2024-09-11T23:39:53.780 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.780 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.780 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.780 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:53.780 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.780 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.781 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.781 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.781 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.781 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.781 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.781 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.781 INFO:teuthology.orchestra.run.smithi026.stdout: "roption2": { 2024-09-11T23:39:53.781 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "roption2", 2024-09-11T23:39:53.781 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.781 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.781 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.781 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "xyz", 2024-09-11T23:39:53.781 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.781 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.782 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.782 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.782 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.782 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.782 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.782 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.782 INFO:teuthology.orchestra.run.smithi026.stdout: "rwoption1": { 2024-09-11T23:39:53.782 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rwoption1", 2024-09-11T23:39:53.782 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.782 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.782 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.782 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:53.782 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.782 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.783 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.783 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.783 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.783 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.783 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.783 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.783 INFO:teuthology.orchestra.run.smithi026.stdout: "rwoption2": { 2024-09-11T23:39:53.783 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rwoption2", 2024-09-11T23:39:53.783 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-09-11T23:39:53.783 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.783 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.783 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:53.784 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.784 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.784 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.784 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.784 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.784 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.784 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.784 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.784 INFO:teuthology.orchestra.run.smithi026.stdout: "rwoption3": { 2024-09-11T23:39:53.784 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rwoption3", 2024-09-11T23:39:53.784 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "float", 2024-09-11T23:39:53.784 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.784 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.785 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:53.785 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.785 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.785 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.785 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.785 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.785 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.785 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.785 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.785 INFO:teuthology.orchestra.run.smithi026.stdout: "rwoption4": { 2024-09-11T23:39:53.785 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rwoption4", 2024-09-11T23:39:53.785 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.785 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.786 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.786 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:53.786 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.786 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.786 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.786 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.786 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.786 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.786 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.786 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.786 INFO:teuthology.orchestra.run.smithi026.stdout: "rwoption5": { 2024-09-11T23:39:53.786 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rwoption5", 2024-09-11T23:39:53.786 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.786 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.786 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.787 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:53.787 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.787 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.787 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.787 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.787 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.787 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.787 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.787 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.787 INFO:teuthology.orchestra.run.smithi026.stdout: "rwoption6": { 2024-09-11T23:39:53.787 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rwoption6", 2024-09-11T23:39:53.787 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.787 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.787 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.788 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-09-11T23:39:53.788 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.788 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.788 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.788 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.788 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.788 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.788 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.788 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.788 INFO:teuthology.orchestra.run.smithi026.stdout: "rwoption7": { 2024-09-11T23:39:53.788 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rwoption7", 2024-09-11T23:39:53.788 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-09-11T23:39:53.788 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.789 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.789 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:53.789 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "1", 2024-09-11T23:39:53.789 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "42", 2024-09-11T23:39:53.789 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.789 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.789 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.789 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.789 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.789 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.789 INFO:teuthology.orchestra.run.smithi026.stdout: "testkey": { 2024-09-11T23:39:53.789 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "testkey", 2024-09-11T23:39:53.789 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.790 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.790 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.790 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:53.790 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.790 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.790 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.790 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.790 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.790 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.790 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.790 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.790 INFO:teuthology.orchestra.run.smithi026.stdout: "testlkey": { 2024-09-11T23:39:53.790 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "testlkey", 2024-09-11T23:39:53.790 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.790 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.791 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.791 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:53.791 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.791 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.791 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.791 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.791 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.791 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.791 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.791 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.791 INFO:teuthology.orchestra.run.smithi026.stdout: "testnewline": { 2024-09-11T23:39:53.791 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "testnewline", 2024-09-11T23:39:53.791 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.791 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.791 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.792 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:53.792 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.792 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.792 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.792 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.792 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.792 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.792 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.792 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:53.792 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:53.792 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.792 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-11T23:39:53.792 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "snap_schedule", 2024-09-11T23:39:53.793 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-09-11T23:39:53.793 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-09-11T23:39:53.793 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-09-11T23:39:53.793 INFO:teuthology.orchestra.run.smithi026.stdout: "allow_m_granularity": { 2024-09-11T23:39:53.793 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "allow_m_granularity", 2024-09-11T23:39:53.793 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.793 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.793 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.793 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:53.793 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.793 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.793 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.793 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "allow minute scheduled snapshots", 2024-09-11T23:39:53.793 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.794 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.794 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.794 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.794 INFO:teuthology.orchestra.run.smithi026.stdout: "dump_on_update": { 2024-09-11T23:39:53.794 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "dump_on_update", 2024-09-11T23:39:53.794 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.794 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.794 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.794 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:53.794 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.794 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.794 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.794 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "dump database to debug log on update", 2024-09-11T23:39:53.794 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.795 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.795 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.795 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.795 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-09-11T23:39:53.795 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-09-11T23:39:53.795 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.795 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.795 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.795 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:53.795 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.795 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.795 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:53.795 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:53.796 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:53.796 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:53.796 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:53.796 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:53.796 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:53.796 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:53.796 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.796 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.796 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.796 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.796 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.796 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-09-11T23:39:53.796 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-09-11T23:39:53.796 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.796 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.797 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.797 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:53.797 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.797 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.797 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.797 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.797 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.797 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.797 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.797 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.797 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-09-11T23:39:53.797 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-09-11T23:39:53.797 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.798 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.798 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.798 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-09-11T23:39:53.798 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.798 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.798 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:53.798 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:53.798 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:53.798 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:53.798 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:53.798 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:53.798 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:53.798 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:53.799 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.799 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.799 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.799 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.799 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.799 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-09-11T23:39:53.799 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-09-11T23:39:53.799 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.799 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.799 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.799 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:53.799 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.799 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.799 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.800 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.800 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.800 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.800 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.800 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:53.800 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:53.800 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.800 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-11T23:39:53.800 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "stats", 2024-09-11T23:39:53.800 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-09-11T23:39:53.800 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-09-11T23:39:53.800 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-09-11T23:39:53.800 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-09-11T23:39:53.800 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-09-11T23:39:53.801 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.801 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.801 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.801 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:53.801 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.801 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.801 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:53.801 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:53.801 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:53.801 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:53.801 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:53.801 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:53.801 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:53.801 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:53.802 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.802 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.802 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.802 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.802 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.802 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-09-11T23:39:53.802 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-09-11T23:39:53.802 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.802 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.802 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.802 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:53.802 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.802 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.802 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.803 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.803 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.803 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.803 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.803 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.803 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-09-11T23:39:53.803 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-09-11T23:39:53.803 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.803 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.803 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.803 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-09-11T23:39:53.803 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.803 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.804 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:53.804 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:53.804 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:53.804 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:53.804 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:53.804 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:53.804 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:53.804 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:53.804 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.804 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.804 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.804 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.804 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.804 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-09-11T23:39:53.804 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-09-11T23:39:53.805 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.805 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.805 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.805 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:53.805 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.805 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.805 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.805 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.805 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.805 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.805 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.805 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:53.805 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:53.805 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.806 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-11T23:39:53.806 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "status", 2024-09-11T23:39:53.806 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-09-11T23:39:53.806 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-09-11T23:39:53.806 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-09-11T23:39:53.806 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-09-11T23:39:53.806 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-09-11T23:39:53.806 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.806 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.806 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.806 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:53.806 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.806 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.806 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:53.807 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:53.807 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:53.807 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:53.807 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:53.807 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:53.807 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:53.807 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:53.807 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.807 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.807 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.807 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.807 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.807 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-09-11T23:39:53.807 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-09-11T23:39:53.808 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.808 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.808 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.808 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:53.808 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.808 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.808 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.808 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.808 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.808 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.808 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.808 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.808 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-09-11T23:39:53.808 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-09-11T23:39:53.809 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.809 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.809 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.809 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-09-11T23:39:53.809 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.809 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.809 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:53.809 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:53.809 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:53.809 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:53.809 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:53.809 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:53.809 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:53.809 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:53.810 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.810 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.810 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.810 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.810 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.810 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-09-11T23:39:53.810 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-09-11T23:39:53.810 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.810 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.810 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.810 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:53.810 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.810 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.810 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.810 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.811 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.811 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.811 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.811 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:53.811 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:53.811 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.811 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-11T23:39:53.811 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "telegraf", 2024-09-11T23:39:53.811 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-09-11T23:39:53.811 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-09-11T23:39:53.811 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-09-11T23:39:53.811 INFO:teuthology.orchestra.run.smithi026.stdout: "address": { 2024-09-11T23:39:53.811 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "address", 2024-09-11T23:39:53.811 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.812 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.812 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.812 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-11T23:39:53.812 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.812 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.812 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.812 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.812 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.812 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.812 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.812 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.812 INFO:teuthology.orchestra.run.smithi026.stdout: "interval": { 2024-09-11T23:39:53.813 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "interval", 2024-09-11T23:39:53.813 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-09-11T23:39:53.813 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.813 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.813 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "15", 2024-09-11T23:39:53.813 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.813 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.813 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.813 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.813 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.813 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.813 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.813 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.813 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-09-11T23:39:53.813 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-09-11T23:39:53.814 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.814 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.814 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.814 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:53.814 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.814 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.814 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:53.814 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:53.814 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:53.814 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:53.814 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:53.814 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:53.814 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:53.814 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:53.814 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.815 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.815 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.815 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.815 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.815 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-09-11T23:39:53.815 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-09-11T23:39:53.815 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.815 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.815 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.815 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:53.815 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.815 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.815 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.816 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.816 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.816 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.816 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.816 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.816 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-09-11T23:39:53.816 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-09-11T23:39:53.816 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.816 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.816 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.816 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-09-11T23:39:53.816 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.816 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.817 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:53.817 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:53.817 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:53.817 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:53.817 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:53.817 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:53.817 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:53.817 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:53.817 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.817 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.817 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.817 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.817 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.817 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-09-11T23:39:53.818 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-09-11T23:39:53.818 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.818 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.818 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.818 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:53.818 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.818 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.818 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.818 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.818 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.818 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.818 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.818 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:53.818 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:53.819 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.819 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-11T23:39:53.819 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "telemetry", 2024-09-11T23:39:53.819 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-09-11T23:39:53.819 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-09-11T23:39:53.819 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-09-11T23:39:53.819 INFO:teuthology.orchestra.run.smithi026.stdout: "channel_basic": { 2024-09-11T23:39:53.819 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "channel_basic", 2024-09-11T23:39:53.819 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.819 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.819 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.819 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-09-11T23:39:53.819 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.819 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.819 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.820 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-11T23:39:53.820 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.820 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.820 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.820 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.820 INFO:teuthology.orchestra.run.smithi026.stdout: "channel_crash": { 2024-09-11T23:39:53.820 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "channel_crash", 2024-09-11T23:39:53.820 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.820 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.820 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.820 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-09-11T23:39:53.820 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.820 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.821 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.821 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-11T23:39:53.821 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.821 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.821 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.821 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.821 INFO:teuthology.orchestra.run.smithi026.stdout: "channel_device": { 2024-09-11T23:39:53.821 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "channel_device", 2024-09-11T23:39:53.821 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.821 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.821 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.821 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-09-11T23:39:53.821 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.822 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.822 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.822 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-11T23:39:53.822 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.822 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.822 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.822 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.822 INFO:teuthology.orchestra.run.smithi026.stdout: "channel_ident": { 2024-09-11T23:39:53.822 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "channel_ident", 2024-09-11T23:39:53.822 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.822 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.822 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.822 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:53.822 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.823 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.823 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.823 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-11T23:39:53.823 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.823 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.823 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.823 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.823 INFO:teuthology.orchestra.run.smithi026.stdout: "channel_perf": { 2024-09-11T23:39:53.823 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "channel_perf", 2024-09-11T23:39:53.823 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.823 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.823 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.823 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:53.823 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.824 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.824 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.824 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-11T23:39:53.824 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.824 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.824 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.824 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.824 INFO:teuthology.orchestra.run.smithi026.stdout: "contact": { 2024-09-11T23:39:53.824 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "contact", 2024-09-11T23:39:53.824 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.824 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.824 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.824 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:53.824 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.825 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.825 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.825 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.825 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.825 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.825 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.825 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.825 INFO:teuthology.orchestra.run.smithi026.stdout: "description": { 2024-09-11T23:39:53.825 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "description", 2024-09-11T23:39:53.825 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.825 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.825 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.825 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:53.825 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.826 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.826 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.826 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.826 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.826 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.826 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.826 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.826 INFO:teuthology.orchestra.run.smithi026.stdout: "device_url": { 2024-09-11T23:39:53.826 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "device_url", 2024-09-11T23:39:53.826 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.826 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.826 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.826 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-11T23:39:53.826 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.827 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.827 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.827 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.827 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.827 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.827 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.827 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.827 INFO:teuthology.orchestra.run.smithi026.stdout: "enabled": { 2024-09-11T23:39:53.827 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "enabled", 2024-09-11T23:39:53.827 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.827 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.827 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.827 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:53.827 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.828 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.828 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.828 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.828 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.828 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.828 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.828 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.828 INFO:teuthology.orchestra.run.smithi026.stdout: "interval": { 2024-09-11T23:39:53.828 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "interval", 2024-09-11T23:39:53.828 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-09-11T23:39:53.828 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.828 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.828 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "24", 2024-09-11T23:39:53.828 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "8", 2024-09-11T23:39:53.829 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.829 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.829 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.829 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.829 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.829 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.829 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.829 INFO:teuthology.orchestra.run.smithi026.stdout: "last_opt_revision": { 2024-09-11T23:39:53.829 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "last_opt_revision", 2024-09-11T23:39:53.829 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-09-11T23:39:53.829 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.829 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.829 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "1", 2024-09-11T23:39:53.829 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.830 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.830 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.830 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.830 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.830 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.830 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.830 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.830 INFO:teuthology.orchestra.run.smithi026.stdout: "leaderboard": { 2024-09-11T23:39:53.830 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "leaderboard", 2024-09-11T23:39:53.830 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.830 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.830 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.830 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:53.831 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.831 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.831 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.831 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.831 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.831 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.831 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.831 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.831 INFO:teuthology.orchestra.run.smithi026.stdout: "leaderboard_description": { 2024-09-11T23:39:53.831 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "leaderboard_description", 2024-09-11T23:39:53.831 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.831 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.831 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.831 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:53.832 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.832 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.832 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.832 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.832 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.832 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.832 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.832 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.832 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-09-11T23:39:53.832 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-09-11T23:39:53.832 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.832 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.832 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.832 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:53.833 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.833 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.833 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:53.833 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:53.833 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:53.833 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:53.833 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:53.833 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:53.833 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:53.833 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:53.833 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.833 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.833 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.833 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.834 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.834 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-09-11T23:39:53.834 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-09-11T23:39:53.834 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.834 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.834 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.834 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:53.834 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.834 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.834 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.834 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.834 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.834 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.834 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.835 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.835 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-09-11T23:39:53.835 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-09-11T23:39:53.835 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.835 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.835 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.835 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-09-11T23:39:53.835 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.835 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.835 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:53.835 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:53.835 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:53.835 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:53.835 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:53.836 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:53.836 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:53.836 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:53.836 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.836 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.836 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.836 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.836 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.836 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-09-11T23:39:53.836 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-09-11T23:39:53.836 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.836 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.836 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.836 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:53.837 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.837 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.837 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.837 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.837 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.837 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.837 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.837 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.837 INFO:teuthology.orchestra.run.smithi026.stdout: "organization": { 2024-09-11T23:39:53.837 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "organization", 2024-09-11T23:39:53.837 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.837 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.837 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.837 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:53.838 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.838 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.838 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.838 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.838 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.838 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.838 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.838 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.838 INFO:teuthology.orchestra.run.smithi026.stdout: "proxy": { 2024-09-11T23:39:53.838 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "proxy", 2024-09-11T23:39:53.838 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.838 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.838 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.838 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:53.839 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.839 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.839 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.839 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.839 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.839 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.839 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.839 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.839 INFO:teuthology.orchestra.run.smithi026.stdout: "url": { 2024-09-11T23:39:53.839 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "url", 2024-09-11T23:39:53.839 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.839 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.839 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.839 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-11T23:39:53.840 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.840 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.840 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.840 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.840 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.840 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.840 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.840 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:53.840 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:53.840 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.840 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-11T23:39:53.840 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "test_orchestrator", 2024-09-11T23:39:53.840 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-09-11T23:39:53.840 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-09-11T23:39:53.841 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-09-11T23:39:53.841 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-09-11T23:39:53.841 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-09-11T23:39:53.841 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.841 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.841 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.841 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:53.841 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.841 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.841 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:53.841 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:53.841 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:53.841 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:53.841 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:53.841 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:53.842 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:53.842 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:53.842 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.842 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.842 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.842 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.842 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.842 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-09-11T23:39:53.842 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-09-11T23:39:53.842 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.842 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.842 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.842 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:53.842 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.843 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.843 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.843 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.843 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.843 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.843 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.843 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.843 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-09-11T23:39:53.843 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-09-11T23:39:53.843 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.843 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.843 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.843 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-09-11T23:39:53.843 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.844 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.844 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:53.844 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:53.844 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:53.844 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:53.844 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:53.844 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:53.844 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:53.844 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:53.844 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.844 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.844 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.844 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.844 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.845 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-09-11T23:39:53.845 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-09-11T23:39:53.845 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.845 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.845 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.845 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:53.845 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.845 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.845 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.845 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.845 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.845 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.845 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.845 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:53.846 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:53.846 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.846 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-11T23:39:53.846 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "volumes", 2024-09-11T23:39:53.846 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-09-11T23:39:53.846 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-09-11T23:39:53.846 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-09-11T23:39:53.846 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-09-11T23:39:53.846 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-09-11T23:39:53.846 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.846 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.846 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.846 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:53.846 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.847 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.847 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:53.847 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:53.847 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:53.847 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:53.847 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:53.847 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:53.847 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:53.847 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:53.847 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.847 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.847 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.847 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.847 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.847 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-09-11T23:39:53.848 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-09-11T23:39:53.848 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.848 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.848 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.848 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:53.848 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.848 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.848 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.848 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.848 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.848 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.848 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.848 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.848 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-09-11T23:39:53.849 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-09-11T23:39:53.849 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.849 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.849 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.849 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-09-11T23:39:53.849 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.849 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.849 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:53.849 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:53.849 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:53.849 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:53.849 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:53.849 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:53.849 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:53.850 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:53.850 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.850 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.850 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.850 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.850 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.850 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-09-11T23:39:53.850 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-09-11T23:39:53.850 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.850 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.850 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.850 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:53.850 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.850 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.851 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.851 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.851 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.851 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.851 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.851 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.851 INFO:teuthology.orchestra.run.smithi026.stdout: "max_concurrent_clones": { 2024-09-11T23:39:53.851 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "max_concurrent_clones", 2024-09-11T23:39:53.851 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-09-11T23:39:53.851 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.851 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.851 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "4", 2024-09-11T23:39:53.851 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.851 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.852 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.852 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-11T23:39:53.852 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.852 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.852 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.852 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.852 INFO:teuthology.orchestra.run.smithi026.stdout: "periodic_async_work": { 2024-09-11T23:39:53.852 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "periodic_async_work", 2024-09-11T23:39:53.852 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.852 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.852 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.852 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:53.852 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.852 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.853 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.853 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Periodically check for async work", 2024-09-11T23:39:53.853 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.853 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.853 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.853 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.853 INFO:teuthology.orchestra.run.smithi026.stdout: "snapshot_clone_delay": { 2024-09-11T23:39:53.853 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "snapshot_clone_delay", 2024-09-11T23:39:53.853 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-09-11T23:39:53.853 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.853 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.853 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2024-09-11T23:39:53.853 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.854 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.854 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.854 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-11T23:39:53.854 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.854 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.854 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.854 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.854 INFO:teuthology.orchestra.run.smithi026.stdout: "snapshot_clone_no_wait": { 2024-09-11T23:39:53.854 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "snapshot_clone_no_wait", 2024-09-11T23:39:53.854 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.854 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.854 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.854 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-09-11T23:39:53.854 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.855 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.855 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.855 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-11T23:39:53.855 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.855 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.855 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.855 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:53.855 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:53.855 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.855 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-11T23:39:53.855 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "zabbix", 2024-09-11T23:39:53.855 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-09-11T23:39:53.855 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-09-11T23:39:53.855 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-09-11T23:39:53.855 INFO:teuthology.orchestra.run.smithi026.stdout: "discovery_interval": { 2024-09-11T23:39:53.856 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "discovery_interval", 2024-09-11T23:39:53.856 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "uint", 2024-09-11T23:39:53.856 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.856 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.856 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "100", 2024-09-11T23:39:53.856 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.856 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.856 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.856 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.856 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.856 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.856 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.856 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.856 INFO:teuthology.orchestra.run.smithi026.stdout: "identifier": { 2024-09-11T23:39:53.857 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "identifier", 2024-09-11T23:39:53.857 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.857 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.857 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.857 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:53.857 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.857 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.857 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.857 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.857 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.857 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.857 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.857 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.857 INFO:teuthology.orchestra.run.smithi026.stdout: "interval": { 2024-09-11T23:39:53.858 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "interval", 2024-09-11T23:39:53.858 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-09-11T23:39:53.858 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.858 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.858 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "60", 2024-09-11T23:39:53.858 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.858 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.858 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.858 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.858 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.858 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.858 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.858 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.859 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-09-11T23:39:53.859 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-09-11T23:39:53.859 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.859 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.859 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.859 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:53.859 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.859 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.859 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:53.859 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:53.859 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:53.859 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:53.859 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:53.860 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:53.860 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:53.860 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:53.860 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.860 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.860 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.860 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.860 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.860 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-09-11T23:39:53.860 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-09-11T23:39:53.860 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.860 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.860 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.860 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:53.861 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.861 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.861 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.861 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.861 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.861 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.861 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.861 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.861 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-09-11T23:39:53.861 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-09-11T23:39:53.861 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.861 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.861 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.861 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-09-11T23:39:53.862 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.862 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.862 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:53.862 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:53.862 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:53.862 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:53.862 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:53.862 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:53.862 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:53.862 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:53.862 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.862 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.862 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.862 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.863 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.863 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-09-11T23:39:53.863 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-09-11T23:39:53.863 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.863 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.863 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.863 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:53.863 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.863 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.863 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.863 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.863 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.863 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.863 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.863 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.864 INFO:teuthology.orchestra.run.smithi026.stdout: "zabbix_host": { 2024-09-11T23:39:53.864 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "zabbix_host", 2024-09-11T23:39:53.864 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.864 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.864 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.864 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:53.864 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.864 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.864 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.864 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.864 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.864 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.864 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.864 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.865 INFO:teuthology.orchestra.run.smithi026.stdout: "zabbix_port": { 2024-09-11T23:39:53.865 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "zabbix_port", 2024-09-11T23:39:53.865 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-09-11T23:39:53.865 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.865 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.865 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "10051", 2024-09-11T23:39:53.865 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.865 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.865 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.865 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.865 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.865 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.865 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.865 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.865 INFO:teuthology.orchestra.run.smithi026.stdout: "zabbix_sender": { 2024-09-11T23:39:53.866 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "zabbix_sender", 2024-09-11T23:39:53.866 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.866 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.866 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.866 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-11T23:39:53.866 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.866 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.866 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.866 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.866 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.866 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.866 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.866 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:53.867 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:53.867 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:53.867 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-11T23:39:53.867 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.867 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-11T23:39:53.867 INFO:teuthology.orchestra.run.smithi026.stdout: "gid": 24856, 2024-09-11T23:39:53.867 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "z", 2024-09-11T23:39:53.867 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr_features": 4540701547738038271, 2024-09-11T23:39:53.867 INFO:teuthology.orchestra.run.smithi026.stdout: "available_modules": [ 2024-09-11T23:39:53.867 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-11T23:39:53.867 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "alerts", 2024-09-11T23:39:53.867 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-09-11T23:39:53.867 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-09-11T23:39:53.867 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-09-11T23:39:53.868 INFO:teuthology.orchestra.run.smithi026.stdout: "interval": { 2024-09-11T23:39:53.868 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "interval", 2024-09-11T23:39:53.868 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-09-11T23:39:53.868 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.868 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.868 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "60", 2024-09-11T23:39:53.868 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.868 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.868 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.868 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "How frequently to reexamine health status", 2024-09-11T23:39:53.868 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.868 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.868 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.868 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.869 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-09-11T23:39:53.869 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-09-11T23:39:53.869 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.869 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.869 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.869 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:53.869 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.869 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.869 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:53.869 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:53.869 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:53.869 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:53.869 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:53.869 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:53.869 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:53.870 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:53.870 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.870 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.870 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.870 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.870 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.870 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-09-11T23:39:53.870 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-09-11T23:39:53.870 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.870 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.870 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.870 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:53.870 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.870 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.871 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.871 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.871 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.871 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.871 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.871 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.871 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-09-11T23:39:53.871 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-09-11T23:39:53.871 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.871 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.871 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.871 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-09-11T23:39:53.871 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.871 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.872 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:53.872 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:53.872 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:53.872 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:53.872 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:53.872 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:53.872 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:53.872 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:53.872 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.872 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.872 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.872 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.872 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.872 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-09-11T23:39:53.873 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-09-11T23:39:53.873 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.873 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.873 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.873 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:53.873 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.873 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.873 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.873 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.873 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.873 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.873 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.873 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.873 INFO:teuthology.orchestra.run.smithi026.stdout: "smtp_destination": { 2024-09-11T23:39:53.874 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "smtp_destination", 2024-09-11T23:39:53.874 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.874 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.874 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.874 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:53.874 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.874 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.874 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.874 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Email address to send alerts to", 2024-09-11T23:39:53.874 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.874 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.874 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.874 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.875 INFO:teuthology.orchestra.run.smithi026.stdout: "smtp_from_name": { 2024-09-11T23:39:53.875 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "smtp_from_name", 2024-09-11T23:39:53.875 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.875 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.875 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.875 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "Ceph", 2024-09-11T23:39:53.875 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.875 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.875 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.875 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Email From: name", 2024-09-11T23:39:53.875 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.875 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.875 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.875 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.876 INFO:teuthology.orchestra.run.smithi026.stdout: "smtp_host": { 2024-09-11T23:39:53.876 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "smtp_host", 2024-09-11T23:39:53.876 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.876 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.876 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.876 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:53.876 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.876 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.876 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.876 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "SMTP server", 2024-09-11T23:39:53.876 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.876 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.876 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.876 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.877 INFO:teuthology.orchestra.run.smithi026.stdout: "smtp_password": { 2024-09-11T23:39:53.877 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "smtp_password", 2024-09-11T23:39:53.877 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.877 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.877 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.877 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:53.877 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.877 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.877 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.877 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Password to authenticate with", 2024-09-11T23:39:53.877 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.877 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.877 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.877 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.878 INFO:teuthology.orchestra.run.smithi026.stdout: "smtp_port": { 2024-09-11T23:39:53.878 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "smtp_port", 2024-09-11T23:39:53.878 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-09-11T23:39:53.878 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.878 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.878 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "465", 2024-09-11T23:39:53.878 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.878 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.878 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.878 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "SMTP port", 2024-09-11T23:39:53.878 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.878 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.878 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.878 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.879 INFO:teuthology.orchestra.run.smithi026.stdout: "smtp_sender": { 2024-09-11T23:39:53.879 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "smtp_sender", 2024-09-11T23:39:53.879 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.879 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.879 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.879 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:53.879 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.879 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.879 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.879 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "SMTP envelope sender", 2024-09-11T23:39:53.879 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.879 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.879 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.879 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.879 INFO:teuthology.orchestra.run.smithi026.stdout: "smtp_ssl": { 2024-09-11T23:39:53.880 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "smtp_ssl", 2024-09-11T23:39:53.880 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.880 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.880 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.880 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-09-11T23:39:53.880 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.880 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.880 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.880 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-11T23:39:53.880 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.880 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.880 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.880 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.881 INFO:teuthology.orchestra.run.smithi026.stdout: "smtp_user": { 2024-09-11T23:39:53.881 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "smtp_user", 2024-09-11T23:39:53.881 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.881 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.881 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.881 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:53.881 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.881 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.881 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.881 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "User to authenticate as", 2024-09-11T23:39:53.881 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.881 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.881 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.881 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:53.882 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:53.882 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.882 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-11T23:39:53.882 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "balancer", 2024-09-11T23:39:53.882 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-09-11T23:39:53.882 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-09-11T23:39:53.882 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-09-11T23:39:53.882 INFO:teuthology.orchestra.run.smithi026.stdout: "active": { 2024-09-11T23:39:53.882 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "active", 2024-09-11T23:39:53.882 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.882 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.882 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.882 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-09-11T23:39:53.882 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.883 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.883 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.883 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "automatically balance PGs across cluster", 2024-09-11T23:39:53.883 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.883 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.883 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.883 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.883 INFO:teuthology.orchestra.run.smithi026.stdout: "begin_time": { 2024-09-11T23:39:53.883 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "begin_time", 2024-09-11T23:39:53.883 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.883 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.883 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.883 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0000", 2024-09-11T23:39:53.883 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.884 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.884 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.884 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "beginning time of day to automatically balance", 2024-09-11T23:39:53.884 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-11T23:39:53.884 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.884 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.884 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.884 INFO:teuthology.orchestra.run.smithi026.stdout: "begin_weekday": { 2024-09-11T23:39:53.884 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "begin_weekday", 2024-09-11T23:39:53.884 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "uint", 2024-09-11T23:39:53.884 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.884 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.884 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2024-09-11T23:39:53.885 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "0", 2024-09-11T23:39:53.885 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "6", 2024-09-11T23:39:53.885 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.885 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-11T23:39:53.885 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-11T23:39:53.885 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.885 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.885 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.885 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_compat_max_iterations": { 2024-09-11T23:39:53.885 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "crush_compat_max_iterations", 2024-09-11T23:39:53.885 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "uint", 2024-09-11T23:39:53.885 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.885 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.885 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "25", 2024-09-11T23:39:53.886 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "1", 2024-09-11T23:39:53.886 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "250", 2024-09-11T23:39:53.886 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.886 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-11T23:39:53.886 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.886 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.886 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.886 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.886 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_compat_metrics": { 2024-09-11T23:39:53.886 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "crush_compat_metrics", 2024-09-11T23:39:53.886 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.886 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.886 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.886 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "pgs,objects,bytes", 2024-09-11T23:39:53.887 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.887 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.887 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.887 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-11T23:39:53.887 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-11T23:39:53.887 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.887 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.887 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.887 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_compat_step": { 2024-09-11T23:39:53.887 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "crush_compat_step", 2024-09-11T23:39:53.887 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "float", 2024-09-11T23:39:53.887 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.887 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.887 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0.5", 2024-09-11T23:39:53.887 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "0.001", 2024-09-11T23:39:53.888 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "0.999", 2024-09-11T23:39:53.888 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.888 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "aggressiveness of optimization", 2024-09-11T23:39:53.888 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-11T23:39:53.888 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.888 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.888 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.888 INFO:teuthology.orchestra.run.smithi026.stdout: "end_time": { 2024-09-11T23:39:53.888 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "end_time", 2024-09-11T23:39:53.888 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.888 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.888 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.888 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "2359", 2024-09-11T23:39:53.888 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.889 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.889 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.889 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "ending time of day to automatically balance", 2024-09-11T23:39:53.889 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-11T23:39:53.889 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.889 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.889 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.889 INFO:teuthology.orchestra.run.smithi026.stdout: "end_weekday": { 2024-09-11T23:39:53.889 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "end_weekday", 2024-09-11T23:39:53.889 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "uint", 2024-09-11T23:39:53.889 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.889 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.889 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2024-09-11T23:39:53.889 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "0", 2024-09-11T23:39:53.890 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "6", 2024-09-11T23:39:53.890 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.890 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-11T23:39:53.890 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-11T23:39:53.890 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.890 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.890 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.890 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-09-11T23:39:53.890 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-09-11T23:39:53.890 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.890 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.890 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.890 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:53.890 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.890 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.891 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:53.891 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:53.891 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:53.891 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:53.891 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:53.891 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:53.891 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:53.891 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:53.891 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.891 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.891 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.891 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.891 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.891 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-09-11T23:39:53.891 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-09-11T23:39:53.892 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.892 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.892 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.892 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:53.892 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.892 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.892 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.892 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.892 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.892 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.892 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.892 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.892 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-09-11T23:39:53.892 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-09-11T23:39:53.893 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.893 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.893 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.893 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-09-11T23:39:53.893 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.893 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.893 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:53.893 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:53.893 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:53.893 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:53.893 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:53.893 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:53.894 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:53.894 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:53.894 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.894 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.894 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.894 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.894 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.894 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-09-11T23:39:53.894 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-09-11T23:39:53.894 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.894 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.894 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.894 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:53.894 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.895 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.895 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.895 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.895 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.895 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.895 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.895 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.895 INFO:teuthology.orchestra.run.smithi026.stdout: "min_score": { 2024-09-11T23:39:53.895 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "min_score", 2024-09-11T23:39:53.895 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "float", 2024-09-11T23:39:53.895 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.895 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.895 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2024-09-11T23:39:53.895 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.896 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.896 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.896 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-11T23:39:53.896 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.896 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.896 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.896 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.896 INFO:teuthology.orchestra.run.smithi026.stdout: "mode": { 2024-09-11T23:39:53.896 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "mode", 2024-09-11T23:39:53.896 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.896 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.896 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.896 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "upmap", 2024-09-11T23:39:53.896 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.897 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.897 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:53.897 INFO:teuthology.orchestra.run.smithi026.stdout: "crush-compat", 2024-09-11T23:39:53.897 INFO:teuthology.orchestra.run.smithi026.stdout: "none", 2024-09-11T23:39:53.897 INFO:teuthology.orchestra.run.smithi026.stdout: "read", 2024-09-11T23:39:53.897 INFO:teuthology.orchestra.run.smithi026.stdout: "upmap", 2024-09-11T23:39:53.897 INFO:teuthology.orchestra.run.smithi026.stdout: "upmap-read" 2024-09-11T23:39:53.897 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:53.897 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Balancer mode", 2024-09-11T23:39:53.897 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.897 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.897 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.897 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.897 INFO:teuthology.orchestra.run.smithi026.stdout: "pool_ids": { 2024-09-11T23:39:53.897 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "pool_ids", 2024-09-11T23:39:53.898 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.898 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.898 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.898 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:53.898 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.898 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.898 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.898 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-11T23:39:53.898 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.898 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.898 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.898 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.898 INFO:teuthology.orchestra.run.smithi026.stdout: "sleep_interval": { 2024-09-11T23:39:53.898 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sleep_interval", 2024-09-11T23:39:53.899 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-09-11T23:39:53.899 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.899 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.899 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "60", 2024-09-11T23:39:53.899 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.899 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.899 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.899 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-11T23:39:53.899 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.899 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.899 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.899 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.899 INFO:teuthology.orchestra.run.smithi026.stdout: "upmap_max_deviation": { 2024-09-11T23:39:53.899 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "upmap_max_deviation", 2024-09-11T23:39:53.900 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-09-11T23:39:53.900 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.900 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.900 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "5", 2024-09-11T23:39:53.900 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "1", 2024-09-11T23:39:53.900 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.900 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.900 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-11T23:39:53.900 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-11T23:39:53.900 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.900 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.900 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.900 INFO:teuthology.orchestra.run.smithi026.stdout: "upmap_max_optimizations": { 2024-09-11T23:39:53.900 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "upmap_max_optimizations", 2024-09-11T23:39:53.900 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "uint", 2024-09-11T23:39:53.901 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.901 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.901 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "10", 2024-09-11T23:39:53.901 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.901 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.901 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.901 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-11T23:39:53.901 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.901 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.901 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.901 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:53.901 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:53.901 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.901 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-11T23:39:53.902 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "cephadm", 2024-09-11T23:39:53.902 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-09-11T23:39:53.902 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-09-11T23:39:53.902 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-09-11T23:39:53.902 INFO:teuthology.orchestra.run.smithi026.stdout: "agent_down_multiplier": { 2024-09-11T23:39:53.902 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "agent_down_multiplier", 2024-09-11T23:39:53.902 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "float", 2024-09-11T23:39:53.902 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.902 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.902 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "3.0", 2024-09-11T23:39:53.902 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.902 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.902 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.902 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-11T23:39:53.902 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.903 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.903 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.903 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.903 INFO:teuthology.orchestra.run.smithi026.stdout: "agent_refresh_rate": { 2024-09-11T23:39:53.903 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "agent_refresh_rate", 2024-09-11T23:39:53.903 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-09-11T23:39:53.903 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.903 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.903 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "20", 2024-09-11T23:39:53.903 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.903 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.903 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.903 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-11T23:39:53.903 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.904 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.904 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.904 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.904 INFO:teuthology.orchestra.run.smithi026.stdout: "agent_starting_port": { 2024-09-11T23:39:53.904 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "agent_starting_port", 2024-09-11T23:39:53.904 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-09-11T23:39:53.904 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.904 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.904 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "4721", 2024-09-11T23:39:53.904 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.904 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.904 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.904 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-11T23:39:53.904 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.905 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.905 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.905 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.905 INFO:teuthology.orchestra.run.smithi026.stdout: "allow_ptrace": { 2024-09-11T23:39:53.905 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "allow_ptrace", 2024-09-11T23:39:53.905 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.905 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.905 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.905 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:53.905 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.905 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.905 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.905 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-11T23:39:53.905 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-11T23:39:53.906 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.906 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.906 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.906 INFO:teuthology.orchestra.run.smithi026.stdout: "autotune_interval": { 2024-09-11T23:39:53.906 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "autotune_interval", 2024-09-11T23:39:53.906 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-09-11T23:39:53.906 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.906 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.906 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "600", 2024-09-11T23:39:53.906 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.906 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.906 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.906 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-11T23:39:53.906 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.907 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.907 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.907 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.907 INFO:teuthology.orchestra.run.smithi026.stdout: "autotune_memory_target_ratio": { 2024-09-11T23:39:53.907 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "autotune_memory_target_ratio", 2024-09-11T23:39:53.907 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "float", 2024-09-11T23:39:53.907 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.907 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.907 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0.7", 2024-09-11T23:39:53.907 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.907 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.907 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.907 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-11T23:39:53.907 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.908 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.908 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.908 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.908 INFO:teuthology.orchestra.run.smithi026.stdout: "cephadm_log_destination": { 2024-09-11T23:39:53.908 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "cephadm_log_destination", 2024-09-11T23:39:53.908 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.908 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.908 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.908 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:53.908 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.908 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.908 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:53.908 INFO:teuthology.orchestra.run.smithi026.stdout: "file", 2024-09-11T23:39:53.908 INFO:teuthology.orchestra.run.smithi026.stdout: "file,syslog", 2024-09-11T23:39:53.909 INFO:teuthology.orchestra.run.smithi026.stdout: "syslog" 2024-09-11T23:39:53.909 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:53.909 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-11T23:39:53.909 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.909 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.909 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.909 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.909 INFO:teuthology.orchestra.run.smithi026.stdout: "cgroups_split": { 2024-09-11T23:39:53.909 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "cgroups_split", 2024-09-11T23:39:53.909 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.909 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.909 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.909 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-09-11T23:39:53.909 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.910 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.910 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.910 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-11T23:39:53.910 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.910 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.910 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.910 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.910 INFO:teuthology.orchestra.run.smithi026.stdout: "config_checks_enabled": { 2024-09-11T23:39:53.910 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "config_checks_enabled", 2024-09-11T23:39:53.910 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.910 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.910 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.910 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:53.910 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.911 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.911 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.911 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-11T23:39:53.911 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.911 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.911 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.911 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.911 INFO:teuthology.orchestra.run.smithi026.stdout: "config_dashboard": { 2024-09-11T23:39:53.911 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "config_dashboard", 2024-09-11T23:39:53.911 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.911 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.911 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.911 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-09-11T23:39:53.911 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.912 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.912 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.912 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-11T23:39:53.912 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.912 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.912 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.912 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.912 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_alertmanager": { 2024-09-11T23:39:53.912 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_alertmanager", 2024-09-11T23:39:53.912 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.912 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.912 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.912 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-11T23:39:53.913 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.913 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.913 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.913 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Prometheus container image", 2024-09-11T23:39:53.913 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.913 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.913 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.913 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.913 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_base": { 2024-09-11T23:39:53.913 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_base", 2024-09-11T23:39:53.913 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.913 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.913 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.913 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-11T23:39:53.913 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.914 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.914 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.914 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Container image name, without the tag", 2024-09-11T23:39:53.914 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.914 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.914 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.914 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.914 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_elasticsearch": { 2024-09-11T23:39:53.914 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_elasticsearch", 2024-09-11T23:39:53.914 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.914 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.914 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.914 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-11T23:39:53.914 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.915 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.915 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.915 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "elasticsearch container image", 2024-09-11T23:39:53.915 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.915 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.915 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.915 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.915 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_grafana": { 2024-09-11T23:39:53.915 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_grafana", 2024-09-11T23:39:53.915 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.915 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.915 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.915 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-11T23:39:53.915 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.916 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.916 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.916 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Prometheus container image", 2024-09-11T23:39:53.916 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.916 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.916 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.916 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.916 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_haproxy": { 2024-09-11T23:39:53.916 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_haproxy", 2024-09-11T23:39:53.916 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.916 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.916 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.916 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-11T23:39:53.916 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.917 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.917 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.917 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "HAproxy container image", 2024-09-11T23:39:53.917 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.917 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.917 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.917 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.917 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_jaeger_agent": { 2024-09-11T23:39:53.917 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_jaeger_agent", 2024-09-11T23:39:53.917 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.917 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.917 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.917 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-11T23:39:53.917 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.918 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.918 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.918 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Jaeger agent container image", 2024-09-11T23:39:53.918 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.918 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.918 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.918 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.918 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_jaeger_collector": { 2024-09-11T23:39:53.918 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_jaeger_collector", 2024-09-11T23:39:53.918 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.918 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.918 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.918 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-11T23:39:53.918 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.919 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.919 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.919 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Jaeger collector container image", 2024-09-11T23:39:53.919 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.919 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.919 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.919 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.919 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_jaeger_query": { 2024-09-11T23:39:53.919 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_jaeger_query", 2024-09-11T23:39:53.919 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.919 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.919 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.919 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-11T23:39:53.919 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.919 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.920 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.920 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Jaeger query container image", 2024-09-11T23:39:53.920 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.920 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.920 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.920 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.920 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_keepalived": { 2024-09-11T23:39:53.920 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_keepalived", 2024-09-11T23:39:53.920 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.920 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.920 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.920 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-11T23:39:53.920 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.920 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.921 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.921 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Keepalived container image", 2024-09-11T23:39:53.921 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.921 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.921 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.921 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.921 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_loki": { 2024-09-11T23:39:53.921 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_loki", 2024-09-11T23:39:53.921 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.921 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.921 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.921 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-11T23:39:53.921 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.921 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.922 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.922 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Loki container image", 2024-09-11T23:39:53.922 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.922 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.922 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.922 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.922 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_node_exporter": { 2024-09-11T23:39:53.922 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_node_exporter", 2024-09-11T23:39:53.922 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.922 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.922 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.922 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-11T23:39:53.922 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.922 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.923 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.923 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Prometheus container image", 2024-09-11T23:39:53.923 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.923 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.923 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.923 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.923 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_nvmeof": { 2024-09-11T23:39:53.923 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_nvmeof", 2024-09-11T23:39:53.923 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.923 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.923 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.923 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-11T23:39:53.923 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.923 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.924 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.924 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Nvme-of container image", 2024-09-11T23:39:53.924 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.924 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.924 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.924 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.924 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_prometheus": { 2024-09-11T23:39:53.924 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_prometheus", 2024-09-11T23:39:53.924 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.924 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.924 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.924 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-11T23:39:53.924 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.924 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.925 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.925 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Prometheus container image", 2024-09-11T23:39:53.925 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.925 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.925 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.925 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.925 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_promtail": { 2024-09-11T23:39:53.925 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_promtail", 2024-09-11T23:39:53.925 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.925 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.925 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.925 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-11T23:39:53.925 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.925 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.926 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.926 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Promtail container image", 2024-09-11T23:39:53.926 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.926 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.926 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.926 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.926 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_samba": { 2024-09-11T23:39:53.926 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_samba", 2024-09-11T23:39:53.926 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.926 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.926 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.926 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-11T23:39:53.926 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.926 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.926 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.927 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Samba/SMB container image", 2024-09-11T23:39:53.927 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.927 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.927 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.927 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.927 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_snmp_gateway": { 2024-09-11T23:39:53.927 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_snmp_gateway", 2024-09-11T23:39:53.927 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.927 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.927 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.927 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-11T23:39:53.927 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.927 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.927 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.928 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "SNMP Gateway container image", 2024-09-11T23:39:53.928 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.928 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.928 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.928 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.928 INFO:teuthology.orchestra.run.smithi026.stdout: "container_init": { 2024-09-11T23:39:53.928 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_init", 2024-09-11T23:39:53.928 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.928 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.928 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.928 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-09-11T23:39:53.928 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.928 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.928 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.928 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Run podman/docker with `--init`", 2024-09-11T23:39:53.929 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.929 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.929 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.929 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.929 INFO:teuthology.orchestra.run.smithi026.stdout: "daemon_cache_timeout": { 2024-09-11T23:39:53.929 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "daemon_cache_timeout", 2024-09-11T23:39:53.929 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-09-11T23:39:53.929 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.929 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.929 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "600", 2024-09-11T23:39:53.929 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.929 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.929 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.930 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-11T23:39:53.930 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.930 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.930 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.930 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.930 INFO:teuthology.orchestra.run.smithi026.stdout: "default_cephadm_command_timeout": { 2024-09-11T23:39:53.930 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "default_cephadm_command_timeout", 2024-09-11T23:39:53.930 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-09-11T23:39:53.930 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.930 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.930 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "900", 2024-09-11T23:39:53.930 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.930 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.930 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.931 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-11T23:39:53.931 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.931 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.931 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.931 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.931 INFO:teuthology.orchestra.run.smithi026.stdout: "default_registry": { 2024-09-11T23:39:53.931 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "default_registry", 2024-09-11T23:39:53.931 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.931 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.931 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.931 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "docker.io", 2024-09-11T23:39:53.931 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.931 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.931 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.931 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-11T23:39:53.932 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.932 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.932 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.932 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.932 INFO:teuthology.orchestra.run.smithi026.stdout: "device_cache_timeout": { 2024-09-11T23:39:53.932 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "device_cache_timeout", 2024-09-11T23:39:53.932 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-09-11T23:39:53.932 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.932 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.932 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "1800", 2024-09-11T23:39:53.932 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.932 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.932 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.932 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "seconds to cache device inventory", 2024-09-11T23:39:53.933 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.933 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.933 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.933 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.933 INFO:teuthology.orchestra.run.smithi026.stdout: "device_enhanced_scan": { 2024-09-11T23:39:53.933 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "device_enhanced_scan", 2024-09-11T23:39:53.933 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.933 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.933 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.933 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:53.933 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.933 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.933 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.933 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-11T23:39:53.934 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.934 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.934 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.934 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.934 INFO:teuthology.orchestra.run.smithi026.stdout: "facts_cache_timeout": { 2024-09-11T23:39:53.934 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "facts_cache_timeout", 2024-09-11T23:39:53.934 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-09-11T23:39:53.934 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.934 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.934 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "60", 2024-09-11T23:39:53.934 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.934 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.934 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.934 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "seconds to cache host facts data", 2024-09-11T23:39:53.935 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.935 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.935 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.935 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.935 INFO:teuthology.orchestra.run.smithi026.stdout: "grafana_dashboards_path": { 2024-09-11T23:39:53.935 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "grafana_dashboards_path", 2024-09-11T23:39:53.935 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.935 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.935 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.935 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-11T23:39:53.935 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.935 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.935 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.935 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-11T23:39:53.935 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.936 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.936 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.936 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.936 INFO:teuthology.orchestra.run.smithi026.stdout: "host_check_interval": { 2024-09-11T23:39:53.936 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "host_check_interval", 2024-09-11T23:39:53.936 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-09-11T23:39:53.936 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.936 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.936 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "600", 2024-09-11T23:39:53.936 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.936 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.936 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.936 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "how frequently to perform a host check", 2024-09-11T23:39:53.936 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.937 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.937 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.937 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.937 INFO:teuthology.orchestra.run.smithi026.stdout: "hw_monitoring": { 2024-09-11T23:39:53.937 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "hw_monitoring", 2024-09-11T23:39:53.937 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.937 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.937 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.937 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:53.937 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.937 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.937 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.937 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-11T23:39:53.937 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.938 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.938 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.938 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.938 INFO:teuthology.orchestra.run.smithi026.stdout: "inventory_list_all": { 2024-09-11T23:39:53.938 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "inventory_list_all", 2024-09-11T23:39:53.938 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.938 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.938 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.938 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:53.938 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.938 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.938 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.938 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-11T23:39:53.938 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.939 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.939 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.939 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.939 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-09-11T23:39:53.939 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-09-11T23:39:53.939 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.939 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.939 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.939 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:53.939 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.939 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.939 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:53.939 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:53.939 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:53.940 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:53.940 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:53.940 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:53.940 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:53.940 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:53.940 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.940 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.940 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.940 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.940 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.940 INFO:teuthology.orchestra.run.smithi026.stdout: "log_refresh_metadata": { 2024-09-11T23:39:53.940 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_refresh_metadata", 2024-09-11T23:39:53.940 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.940 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.941 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.941 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:53.941 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.941 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.941 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.941 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-11T23:39:53.941 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.941 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.941 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.941 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.941 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-09-11T23:39:53.941 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-09-11T23:39:53.941 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.941 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.942 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.942 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-09-11T23:39:53.942 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.942 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.942 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.942 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-11T23:39:53.942 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.942 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.942 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.942 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.942 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-09-11T23:39:53.942 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-09-11T23:39:53.942 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.942 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.943 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.943 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-09-11T23:39:53.943 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.943 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.943 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:53.943 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:53.943 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:53.943 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:53.943 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:53.943 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:53.943 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:53.943 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:53.943 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.943 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.944 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.944 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.944 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.944 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-09-11T23:39:53.944 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-09-11T23:39:53.944 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.944 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.944 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.944 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:53.944 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.944 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.944 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.944 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.944 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.944 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.945 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.945 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.945 INFO:teuthology.orchestra.run.smithi026.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-11T23:39:53.945 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-11T23:39:53.945 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.945 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.945 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.945 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:53.945 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.945 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.945 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.945 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-11T23:39:53.945 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.945 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.946 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.946 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.946 INFO:teuthology.orchestra.run.smithi026.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-11T23:39:53.946 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-11T23:39:53.946 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.946 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.946 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.946 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "*", 2024-09-11T23:39:53.946 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.946 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.946 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.946 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-11T23:39:53.946 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.946 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.947 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.947 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.947 INFO:teuthology.orchestra.run.smithi026.stdout: "max_count_per_host": { 2024-09-11T23:39:53.947 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "max_count_per_host", 2024-09-11T23:39:53.947 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-09-11T23:39:53.947 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.947 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.947 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "10", 2024-09-11T23:39:53.947 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.947 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.947 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.947 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "max number of daemons per service per host", 2024-09-11T23:39:53.947 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.947 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.947 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.948 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.948 INFO:teuthology.orchestra.run.smithi026.stdout: "max_osd_draining_count": { 2024-09-11T23:39:53.948 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "max_osd_draining_count", 2024-09-11T23:39:53.948 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-09-11T23:39:53.948 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.948 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.948 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "10", 2024-09-11T23:39:53.948 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.948 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.948 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.948 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-11T23:39:53.948 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.948 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.948 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.949 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.949 INFO:teuthology.orchestra.run.smithi026.stdout: "migration_current": { 2024-09-11T23:39:53.949 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "migration_current", 2024-09-11T23:39:53.949 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-09-11T23:39:53.949 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.949 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.949 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:53.949 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.949 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.949 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.949 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "internal - do not modify", 2024-09-11T23:39:53.949 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.949 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.949 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.950 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.950 INFO:teuthology.orchestra.run.smithi026.stdout: "mode": { 2024-09-11T23:39:53.950 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "mode", 2024-09-11T23:39:53.950 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.950 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.950 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.950 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "root", 2024-09-11T23:39:53.950 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.950 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.950 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:53.950 INFO:teuthology.orchestra.run.smithi026.stdout: "cephadm-package", 2024-09-11T23:39:53.950 INFO:teuthology.orchestra.run.smithi026.stdout: "root" 2024-09-11T23:39:53.950 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:53.950 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "mode for remote execution of cephadm", 2024-09-11T23:39:53.950 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.951 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.951 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.951 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.951 INFO:teuthology.orchestra.run.smithi026.stdout: "oob_default_addr": { 2024-09-11T23:39:53.951 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "oob_default_addr", 2024-09-11T23:39:53.951 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.951 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.951 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.951 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "169.254.1.1", 2024-09-11T23:39:53.951 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.951 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.951 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.951 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-11T23:39:53.951 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.952 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.952 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.952 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.952 INFO:teuthology.orchestra.run.smithi026.stdout: "prometheus_alerts_path": { 2024-09-11T23:39:53.952 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "prometheus_alerts_path", 2024-09-11T23:39:53.952 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.952 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.952 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.952 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-11T23:39:53.952 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.952 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.952 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.952 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-11T23:39:53.952 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.952 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.953 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.953 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.953 INFO:teuthology.orchestra.run.smithi026.stdout: "registry_insecure": { 2024-09-11T23:39:53.953 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "registry_insecure", 2024-09-11T23:39:53.953 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.953 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.953 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.953 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:53.953 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.953 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.953 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.953 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-11T23:39:53.953 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.954 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.954 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.954 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.954 INFO:teuthology.orchestra.run.smithi026.stdout: "registry_password": { 2024-09-11T23:39:53.954 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "registry_password", 2024-09-11T23:39:53.954 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.954 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.954 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.954 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:53.954 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.954 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.954 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.954 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-11T23:39:53.954 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.955 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.955 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.955 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.955 INFO:teuthology.orchestra.run.smithi026.stdout: "registry_url": { 2024-09-11T23:39:53.955 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "registry_url", 2024-09-11T23:39:53.955 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.955 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.955 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.955 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:53.955 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.955 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.955 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.955 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-11T23:39:53.955 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.955 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.956 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.956 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.956 INFO:teuthology.orchestra.run.smithi026.stdout: "registry_username": { 2024-09-11T23:39:53.956 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "registry_username", 2024-09-11T23:39:53.956 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.956 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.956 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.956 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:53.956 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.956 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.956 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.956 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-11T23:39:53.956 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.956 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.957 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.957 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.957 INFO:teuthology.orchestra.run.smithi026.stdout: "secure_monitoring_stack": { 2024-09-11T23:39:53.957 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "secure_monitoring_stack", 2024-09-11T23:39:53.957 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.957 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.957 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.957 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:53.957 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.957 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.957 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.957 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-11T23:39:53.957 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.957 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.958 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.958 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.958 INFO:teuthology.orchestra.run.smithi026.stdout: "service_discovery_port": { 2024-09-11T23:39:53.958 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "service_discovery_port", 2024-09-11T23:39:53.958 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-09-11T23:39:53.958 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.958 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.958 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "8765", 2024-09-11T23:39:53.958 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.958 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.958 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.958 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "cephadm service discovery port", 2024-09-11T23:39:53.958 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.958 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.959 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.959 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.959 INFO:teuthology.orchestra.run.smithi026.stdout: "ssh_config_file": { 2024-09-11T23:39:53.959 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ssh_config_file", 2024-09-11T23:39:53.959 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.959 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.959 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.959 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:53.959 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.959 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.959 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.959 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-11T23:39:53.959 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.959 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.959 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.960 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.960 INFO:teuthology.orchestra.run.smithi026.stdout: "use_agent": { 2024-09-11T23:39:53.960 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "use_agent", 2024-09-11T23:39:53.960 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.960 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.960 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.960 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:53.960 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.960 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.960 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.960 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-11T23:39:53.960 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.960 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.960 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.961 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.961 INFO:teuthology.orchestra.run.smithi026.stdout: "use_repo_digest": { 2024-09-11T23:39:53.961 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "use_repo_digest", 2024-09-11T23:39:53.961 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.961 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.961 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.961 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-09-11T23:39:53.961 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.961 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.961 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.961 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-11T23:39:53.961 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.961 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.961 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.962 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.962 INFO:teuthology.orchestra.run.smithi026.stdout: "warn_on_failed_host_check": { 2024-09-11T23:39:53.962 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "warn_on_failed_host_check", 2024-09-11T23:39:53.962 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.962 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.962 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.962 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-09-11T23:39:53.962 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.962 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.962 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.962 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "raise a health warning if the host check fails", 2024-09-11T23:39:53.962 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.962 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.962 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.962 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.963 INFO:teuthology.orchestra.run.smithi026.stdout: "warn_on_stray_daemons": { 2024-09-11T23:39:53.963 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "warn_on_stray_daemons", 2024-09-11T23:39:53.963 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.963 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.963 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.963 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-09-11T23:39:53.963 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.963 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.963 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.963 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-11T23:39:53.963 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.963 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.963 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.963 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.964 INFO:teuthology.orchestra.run.smithi026.stdout: "warn_on_stray_hosts": { 2024-09-11T23:39:53.964 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "warn_on_stray_hosts", 2024-09-11T23:39:53.964 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.964 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.964 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.964 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-09-11T23:39:53.964 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.964 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.964 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.964 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-11T23:39:53.964 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.964 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.964 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.964 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:53.965 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:53.965 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.965 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-11T23:39:53.965 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "crash", 2024-09-11T23:39:53.965 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-09-11T23:39:53.965 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-09-11T23:39:53.965 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-09-11T23:39:53.965 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-09-11T23:39:53.965 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-09-11T23:39:53.965 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.965 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.965 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.965 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:53.965 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.965 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.966 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:53.966 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:53.966 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:53.966 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:53.966 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:53.966 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:53.966 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:53.966 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:53.966 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.966 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.966 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.966 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.966 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.967 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-09-11T23:39:53.967 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-09-11T23:39:53.967 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.967 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.967 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.967 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:53.967 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.967 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.967 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.967 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.967 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.967 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.967 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.967 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.967 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-09-11T23:39:53.968 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-09-11T23:39:53.968 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.968 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.968 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.968 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-09-11T23:39:53.968 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.968 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.968 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:53.968 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:53.968 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:53.968 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:53.968 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:53.968 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:53.968 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:53.969 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:53.969 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.969 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.969 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.969 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.969 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.969 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-09-11T23:39:53.969 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-09-11T23:39:53.969 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.969 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.969 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.969 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:53.969 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.969 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.970 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.970 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.970 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.970 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.970 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.970 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.970 INFO:teuthology.orchestra.run.smithi026.stdout: "retain_interval": { 2024-09-11T23:39:53.970 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "retain_interval", 2024-09-11T23:39:53.970 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-09-11T23:39:53.970 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.970 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.970 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "31536000", 2024-09-11T23:39:53.970 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.970 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.971 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.971 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-11T23:39:53.971 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.971 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.971 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.971 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.971 INFO:teuthology.orchestra.run.smithi026.stdout: "warn_recent_interval": { 2024-09-11T23:39:53.971 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "warn_recent_interval", 2024-09-11T23:39:53.971 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-09-11T23:39:53.971 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.971 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:53.971 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "1209600", 2024-09-11T23:39:53.971 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.972 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.972 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.972 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-11T23:39:53.972 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.972 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.972 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.972 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:53.972 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:53.972 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.972 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-11T23:39:53.972 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "dashboard", 2024-09-11T23:39:53.972 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-09-11T23:39:53.972 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-09-11T23:39:53.972 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-09-11T23:39:53.972 INFO:teuthology.orchestra.run.smithi026.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-11T23:39:53.973 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-11T23:39:53.973 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-09-11T23:39:53.973 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.973 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.973 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "10", 2024-09-11T23:39:53.973 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.973 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.973 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.973 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.973 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.973 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.973 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.973 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.973 INFO:teuthology.orchestra.run.smithi026.stdout: "ALERTMANAGER_API_HOST": { 2024-09-11T23:39:53.974 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-11T23:39:53.974 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.974 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.974 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.974 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:53.974 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.974 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.974 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.974 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.974 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.974 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.974 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.974 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.974 INFO:teuthology.orchestra.run.smithi026.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-11T23:39:53.974 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-11T23:39:53.975 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.975 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.975 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.975 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-09-11T23:39:53.975 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.975 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.975 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.975 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.975 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.975 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.975 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.975 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.975 INFO:teuthology.orchestra.run.smithi026.stdout: "AUDIT_API_ENABLED": { 2024-09-11T23:39:53.975 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "AUDIT_API_ENABLED", 2024-09-11T23:39:53.976 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.976 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.976 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.976 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:53.976 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.976 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.976 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.976 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.976 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.976 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.976 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.976 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.976 INFO:teuthology.orchestra.run.smithi026.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-11T23:39:53.976 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-11T23:39:53.977 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.977 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.977 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.977 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-09-11T23:39:53.977 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.977 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.977 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.977 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.977 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.977 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.977 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.977 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.977 INFO:teuthology.orchestra.run.smithi026.stdout: "ENABLE_BROWSABLE_API": { 2024-09-11T23:39:53.977 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-11T23:39:53.977 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.978 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.978 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.978 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-09-11T23:39:53.978 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.978 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.978 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.978 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.978 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.978 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.978 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.978 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.978 INFO:teuthology.orchestra.run.smithi026.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-11T23:39:53.978 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-11T23:39:53.978 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.978 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.979 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.979 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-09-11T23:39:53.979 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.979 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.979 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.979 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.979 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.979 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.979 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.979 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.979 INFO:teuthology.orchestra.run.smithi026.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-11T23:39:53.979 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-11T23:39:53.979 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.980 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.980 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.980 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-09-11T23:39:53.980 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.980 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.980 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.980 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.980 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.980 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.980 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.980 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.980 INFO:teuthology.orchestra.run.smithi026.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-11T23:39:53.980 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-11T23:39:53.980 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.981 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.981 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.981 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-09-11T23:39:53.981 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.981 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.981 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.981 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.981 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.981 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.981 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.981 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.981 INFO:teuthology.orchestra.run.smithi026.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-11T23:39:53.981 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-11T23:39:53.981 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.981 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.982 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.982 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-09-11T23:39:53.982 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.982 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.982 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.982 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.982 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.982 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.982 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.982 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.982 INFO:teuthology.orchestra.run.smithi026.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-11T23:39:53.982 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-11T23:39:53.982 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.982 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.983 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.983 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-09-11T23:39:53.983 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.983 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.983 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.983 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.983 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.983 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.983 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.983 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.983 INFO:teuthology.orchestra.run.smithi026.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-11T23:39:53.983 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-11T23:39:53.983 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.983 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.984 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.984 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-09-11T23:39:53.984 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.984 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.984 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.984 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.984 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.984 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.984 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.984 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.984 INFO:teuthology.orchestra.run.smithi026.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-11T23:39:53.984 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-11T23:39:53.984 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.984 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.985 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.985 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-09-11T23:39:53.985 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.985 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.985 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.985 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.985 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.985 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.985 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.985 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.985 INFO:teuthology.orchestra.run.smithi026.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-11T23:39:53.985 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-11T23:39:53.985 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.985 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.986 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.986 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:53.986 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.986 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.986 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.986 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.986 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.986 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.986 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.986 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.986 INFO:teuthology.orchestra.run.smithi026.stdout: "GRAFANA_API_PASSWORD": { 2024-09-11T23:39:53.986 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-11T23:39:53.986 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.986 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.986 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.987 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "admin", 2024-09-11T23:39:53.987 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.987 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.987 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.987 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.987 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.987 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.987 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.987 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.987 INFO:teuthology.orchestra.run.smithi026.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-11T23:39:53.987 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-11T23:39:53.987 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.987 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.987 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.988 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-09-11T23:39:53.988 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.988 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.988 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.988 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.988 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.988 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.988 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.988 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.988 INFO:teuthology.orchestra.run.smithi026.stdout: "GRAFANA_API_URL": { 2024-09-11T23:39:53.988 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "GRAFANA_API_URL", 2024-09-11T23:39:53.988 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.988 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.988 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.988 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:53.989 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.989 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.989 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.989 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.989 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.989 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.989 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.989 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.989 INFO:teuthology.orchestra.run.smithi026.stdout: "GRAFANA_API_USERNAME": { 2024-09-11T23:39:53.989 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-11T23:39:53.989 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.989 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.989 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.989 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "admin", 2024-09-11T23:39:53.990 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.990 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.990 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.990 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.990 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.990 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.990 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.990 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.990 INFO:teuthology.orchestra.run.smithi026.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-11T23:39:53.990 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-11T23:39:53.990 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.990 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.990 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.990 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:53.990 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.991 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.991 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.991 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.991 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.991 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.991 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.991 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.991 INFO:teuthology.orchestra.run.smithi026.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-11T23:39:53.991 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-11T23:39:53.991 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.991 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.991 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.991 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:53.991 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.992 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.992 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.992 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.992 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.992 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.992 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.992 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.992 INFO:teuthology.orchestra.run.smithi026.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-11T23:39:53.992 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-11T23:39:53.992 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.992 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.992 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.992 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-09-11T23:39:53.993 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.993 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.993 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.993 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.993 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.993 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.993 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.993 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.993 INFO:teuthology.orchestra.run.smithi026.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-11T23:39:53.993 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-11T23:39:53.993 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.993 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.993 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.993 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:53.993 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.994 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.994 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.994 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.994 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.994 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.994 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.994 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.994 INFO:teuthology.orchestra.run.smithi026.stdout: "PROMETHEUS_API_HOST": { 2024-09-11T23:39:53.994 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-11T23:39:53.994 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:53.994 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.994 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.994 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:53.994 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.995 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.995 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.995 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.995 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.995 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.995 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.995 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.995 INFO:teuthology.orchestra.run.smithi026.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-11T23:39:53.995 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-11T23:39:53.995 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.995 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.995 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.995 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-09-11T23:39:53.995 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.996 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.996 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.996 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.996 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.996 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.996 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.996 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.996 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-11T23:39:53.996 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-11T23:39:53.996 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.996 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.996 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.996 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:53.996 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.997 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.997 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.997 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.997 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.997 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.997 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.997 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.997 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-11T23:39:53.997 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-11T23:39:53.997 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.997 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.997 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.997 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:53.997 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.998 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.998 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.998 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.998 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.998 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.998 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.998 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.998 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-11T23:39:53.998 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-11T23:39:53.998 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.998 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.998 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.998 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-09-11T23:39:53.998 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.999 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:53.999 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:53.999 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:53.999 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:53.999 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:53.999 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:53.999 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:53.999 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-11T23:39:53.999 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-11T23:39:53.999 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:53.999 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:53.999 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:53.999 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-09-11T23:39:53.999 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:53.999 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.000 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.000 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.000 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.000 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.000 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.000 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.000 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-11T23:39:54.000 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-11T23:39:54.000 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.000 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.000 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.000 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:54.000 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.000 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.001 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.001 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.001 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.001 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.001 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.001 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.001 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-11T23:39:54.001 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-11T23:39:54.001 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.001 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.001 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.001 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:54.001 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.001 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.002 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.002 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.002 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.002 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.002 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.002 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.002 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-11T23:39:54.002 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-11T23:39:54.002 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.002 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.002 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.002 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:54.002 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.002 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.002 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.003 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.003 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.003 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.003 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.003 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.003 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_ENABLED": { 2024-09-11T23:39:54.003 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-11T23:39:54.003 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.003 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.003 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.003 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-09-11T23:39:54.003 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.003 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.003 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.004 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.004 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.004 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.004 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.004 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.004 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-11T23:39:54.004 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-11T23:39:54.004 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.004 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.004 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.004 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-11T23:39:54.004 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.004 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.004 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.005 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.005 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.005 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.005 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.005 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.005 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-11T23:39:54.005 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-11T23:39:54.005 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-09-11T23:39:54.005 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.005 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.005 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "10", 2024-09-11T23:39:54.005 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.005 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.006 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.006 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.006 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.006 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.006 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.006 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.006 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-11T23:39:54.006 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-11T23:39:54.006 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-09-11T23:39:54.006 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.006 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.006 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "8", 2024-09-11T23:39:54.006 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.006 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.007 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.007 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.007 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.007 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.007 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.007 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.007 INFO:teuthology.orchestra.run.smithi026.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-11T23:39:54.007 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-11T23:39:54.007 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-09-11T23:39:54.007 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.007 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.007 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "45", 2024-09-11T23:39:54.007 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.007 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.007 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.008 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.008 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.008 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.008 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.008 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.008 INFO:teuthology.orchestra.run.smithi026.stdout: "RGW_API_ACCESS_KEY": { 2024-09-11T23:39:54.008 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-11T23:39:54.008 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.008 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.008 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.008 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:54.008 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.008 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.009 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.009 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.009 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.009 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.009 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.009 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.009 INFO:teuthology.orchestra.run.smithi026.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-11T23:39:54.009 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-11T23:39:54.009 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.009 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.009 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.009 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "admin", 2024-09-11T23:39:54.009 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.009 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.010 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.010 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.010 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.010 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.010 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.010 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.010 INFO:teuthology.orchestra.run.smithi026.stdout: "RGW_API_SECRET_KEY": { 2024-09-11T23:39:54.010 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-11T23:39:54.010 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.010 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.010 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.010 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:54.010 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.010 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.010 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.011 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.011 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.011 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.011 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.011 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.011 INFO:teuthology.orchestra.run.smithi026.stdout: "RGW_API_SSL_VERIFY": { 2024-09-11T23:39:54.011 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-11T23:39:54.011 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.011 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.011 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.011 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-09-11T23:39:54.011 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.011 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.011 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.012 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.012 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.012 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.012 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.012 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.012 INFO:teuthology.orchestra.run.smithi026.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-11T23:39:54.012 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-11T23:39:54.012 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-09-11T23:39:54.012 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.012 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.012 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2024-09-11T23:39:54.012 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.012 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.012 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.013 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.013 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.013 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.013 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.013 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.013 INFO:teuthology.orchestra.run.smithi026.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-11T23:39:54.013 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-11T23:39:54.013 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-09-11T23:39:54.013 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.013 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.013 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "10", 2024-09-11T23:39:54.013 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.013 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.013 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.014 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.014 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.014 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.014 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.014 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.014 INFO:teuthology.orchestra.run.smithi026.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-11T23:39:54.014 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-11T23:39:54.014 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-09-11T23:39:54.014 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.014 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.014 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "5", 2024-09-11T23:39:54.014 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.014 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.014 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.014 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.015 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.015 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.015 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.015 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.015 INFO:teuthology.orchestra.run.smithi026.stdout: "cross_origin_url": { 2024-09-11T23:39:54.015 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "cross_origin_url", 2024-09-11T23:39:54.015 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.015 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.015 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.015 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:54.015 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.015 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.015 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.015 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.016 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.016 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.016 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.016 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.016 INFO:teuthology.orchestra.run.smithi026.stdout: "crt_file": { 2024-09-11T23:39:54.016 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "crt_file", 2024-09-11T23:39:54.016 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.016 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.016 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.016 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:54.016 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.016 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.016 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.016 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.017 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.017 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.017 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.017 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.017 INFO:teuthology.orchestra.run.smithi026.stdout: "debug": { 2024-09-11T23:39:54.017 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "debug", 2024-09-11T23:39:54.017 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.017 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.017 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.017 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:54.017 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.017 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.017 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.017 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Enable/disable debug options", 2024-09-11T23:39:54.018 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.018 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.018 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.018 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.018 INFO:teuthology.orchestra.run.smithi026.stdout: "jwt_token_ttl": { 2024-09-11T23:39:54.018 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "jwt_token_ttl", 2024-09-11T23:39:54.018 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-09-11T23:39:54.018 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.018 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.018 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "28800", 2024-09-11T23:39:54.018 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.018 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.018 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.019 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.019 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.019 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.019 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.019 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.019 INFO:teuthology.orchestra.run.smithi026.stdout: "key_file": { 2024-09-11T23:39:54.019 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "key_file", 2024-09-11T23:39:54.019 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.019 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.019 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.019 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:54.019 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.019 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.019 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.020 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.020 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.020 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.020 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.020 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.020 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-09-11T23:39:54.020 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-09-11T23:39:54.020 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.020 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.020 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.020 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:54.020 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.020 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.020 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:54.020 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:54.021 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:54.021 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:54.021 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:54.021 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:54.021 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:54.021 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:54.021 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.021 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.021 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.021 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.021 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.021 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-09-11T23:39:54.021 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-09-11T23:39:54.021 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.022 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.022 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.022 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:54.022 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.022 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.022 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.022 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.022 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.022 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.022 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.022 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.022 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-09-11T23:39:54.022 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-09-11T23:39:54.023 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.023 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.023 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.023 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-09-11T23:39:54.023 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.023 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.023 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:54.023 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:54.023 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:54.023 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:54.023 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:54.023 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:54.023 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:54.023 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:54.024 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.024 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.024 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.024 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.024 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.024 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-09-11T23:39:54.024 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-09-11T23:39:54.024 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.024 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.024 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.024 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:54.024 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.024 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.024 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.025 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.025 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.025 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.025 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.025 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.025 INFO:teuthology.orchestra.run.smithi026.stdout: "motd": { 2024-09-11T23:39:54.025 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "motd", 2024-09-11T23:39:54.025 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.025 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.025 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.025 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:54.025 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.025 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.025 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.025 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "The message of the day", 2024-09-11T23:39:54.026 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.026 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.026 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.026 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.026 INFO:teuthology.orchestra.run.smithi026.stdout: "redirect_resolve_ip_addr": { 2024-09-11T23:39:54.026 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "redirect_resolve_ip_addr", 2024-09-11T23:39:54.026 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.026 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.026 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.026 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:54.026 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.026 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.026 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.026 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.027 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.027 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.027 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.027 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.027 INFO:teuthology.orchestra.run.smithi026.stdout: "server_addr": { 2024-09-11T23:39:54.027 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "server_addr", 2024-09-11T23:39:54.027 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.027 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.027 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.027 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "::", 2024-09-11T23:39:54.027 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.027 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.027 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.027 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.028 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.028 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.028 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.028 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.028 INFO:teuthology.orchestra.run.smithi026.stdout: "server_port": { 2024-09-11T23:39:54.028 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "server_port", 2024-09-11T23:39:54.028 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-09-11T23:39:54.028 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.028 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.028 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "8080", 2024-09-11T23:39:54.028 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.028 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.028 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.028 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.029 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.029 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.029 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.029 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.029 INFO:teuthology.orchestra.run.smithi026.stdout: "ssl": { 2024-09-11T23:39:54.029 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ssl", 2024-09-11T23:39:54.029 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.029 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.029 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.029 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-09-11T23:39:54.029 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.029 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.029 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.029 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.029 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.030 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.030 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.030 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.030 INFO:teuthology.orchestra.run.smithi026.stdout: "ssl_server_port": { 2024-09-11T23:39:54.030 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ssl_server_port", 2024-09-11T23:39:54.030 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-09-11T23:39:54.030 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.030 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.030 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "8443", 2024-09-11T23:39:54.030 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.030 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.030 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.030 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.030 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.031 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.031 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.031 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.031 INFO:teuthology.orchestra.run.smithi026.stdout: "standby_behaviour": { 2024-09-11T23:39:54.031 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "standby_behaviour", 2024-09-11T23:39:54.031 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.031 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.031 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.031 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "redirect", 2024-09-11T23:39:54.031 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.031 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.031 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:54.031 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:54.032 INFO:teuthology.orchestra.run.smithi026.stdout: "redirect" 2024-09-11T23:39:54.032 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:54.032 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.032 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.032 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.032 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.032 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.032 INFO:teuthology.orchestra.run.smithi026.stdout: "standby_error_status_code": { 2024-09-11T23:39:54.032 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "standby_error_status_code", 2024-09-11T23:39:54.032 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-09-11T23:39:54.032 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.032 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.032 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "500", 2024-09-11T23:39:54.032 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "400", 2024-09-11T23:39:54.033 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "599", 2024-09-11T23:39:54.033 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.033 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.033 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.033 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.033 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.033 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.033 INFO:teuthology.orchestra.run.smithi026.stdout: "url_prefix": { 2024-09-11T23:39:54.033 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "url_prefix", 2024-09-11T23:39:54.033 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.033 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.033 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.033 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:54.033 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.034 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.034 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.034 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.034 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.034 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.034 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.034 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:54.034 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:54.034 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.034 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-11T23:39:54.034 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "devicehealth", 2024-09-11T23:39:54.034 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-09-11T23:39:54.034 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-09-11T23:39:54.034 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-09-11T23:39:54.035 INFO:teuthology.orchestra.run.smithi026.stdout: "enable_monitoring": { 2024-09-11T23:39:54.035 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "enable_monitoring", 2024-09-11T23:39:54.035 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.035 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.035 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.035 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-09-11T23:39:54.035 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.035 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.035 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.035 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "monitor device health metrics", 2024-09-11T23:39:54.035 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.035 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.035 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.035 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.035 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-09-11T23:39:54.036 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-09-11T23:39:54.036 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.036 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.036 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.036 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:54.036 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.036 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.036 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:54.036 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:54.036 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:54.036 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:54.036 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:54.036 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:54.037 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:54.037 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:54.037 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.037 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.037 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.037 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.037 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.037 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-09-11T23:39:54.037 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-09-11T23:39:54.037 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.037 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.037 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.037 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:54.038 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.038 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.038 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.038 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.038 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.038 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.038 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.038 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.038 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-09-11T23:39:54.038 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-09-11T23:39:54.038 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.038 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.038 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.039 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-09-11T23:39:54.039 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.039 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.039 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:54.039 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:54.039 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:54.039 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:54.039 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:54.039 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:54.039 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:54.039 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:54.039 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.039 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.039 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.040 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.040 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.040 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-09-11T23:39:54.040 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-09-11T23:39:54.040 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.040 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.040 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.040 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:54.040 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.040 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.040 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.040 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.040 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.040 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.040 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.041 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.041 INFO:teuthology.orchestra.run.smithi026.stdout: "mark_out_threshold": { 2024-09-11T23:39:54.041 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "mark_out_threshold", 2024-09-11T23:39:54.041 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-09-11T23:39:54.041 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.041 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.041 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "2419200", 2024-09-11T23:39:54.041 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.041 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.041 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.041 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-11T23:39:54.041 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.041 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.041 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.041 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.042 INFO:teuthology.orchestra.run.smithi026.stdout: "pool_name": { 2024-09-11T23:39:54.042 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "pool_name", 2024-09-11T23:39:54.042 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.042 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.042 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.042 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "device_health_metrics", 2024-09-11T23:39:54.042 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.042 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.042 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.042 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-11T23:39:54.042 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.042 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.042 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.042 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.042 INFO:teuthology.orchestra.run.smithi026.stdout: "retention_period": { 2024-09-11T23:39:54.043 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "retention_period", 2024-09-11T23:39:54.043 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-09-11T23:39:54.043 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.043 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.043 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "15552000", 2024-09-11T23:39:54.043 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.043 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.043 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.043 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "how long to retain device health metrics", 2024-09-11T23:39:54.043 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.043 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.043 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.043 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.043 INFO:teuthology.orchestra.run.smithi026.stdout: "scrape_frequency": { 2024-09-11T23:39:54.043 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "scrape_frequency", 2024-09-11T23:39:54.044 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-09-11T23:39:54.044 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.044 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.044 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "86400", 2024-09-11T23:39:54.044 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.044 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.044 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.044 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-11T23:39:54.044 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.044 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.044 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.044 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.044 INFO:teuthology.orchestra.run.smithi026.stdout: "self_heal": { 2024-09-11T23:39:54.044 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "self_heal", 2024-09-11T23:39:54.045 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.045 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.045 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.045 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-09-11T23:39:54.045 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.045 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.045 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.045 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-11T23:39:54.045 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.045 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.045 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.045 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.045 INFO:teuthology.orchestra.run.smithi026.stdout: "sleep_interval": { 2024-09-11T23:39:54.046 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sleep_interval", 2024-09-11T23:39:54.046 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-09-11T23:39:54.046 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.046 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.046 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "600", 2024-09-11T23:39:54.046 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.046 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.046 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.046 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "how frequently to wake up and check device health", 2024-09-11T23:39:54.046 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.046 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.046 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.046 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.047 INFO:teuthology.orchestra.run.smithi026.stdout: "warn_threshold": { 2024-09-11T23:39:54.047 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "warn_threshold", 2024-09-11T23:39:54.047 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-09-11T23:39:54.047 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.047 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.047 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "7257600", 2024-09-11T23:39:54.047 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.047 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.047 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.047 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-11T23:39:54.047 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.047 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.047 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.047 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:54.047 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:54.048 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.048 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-11T23:39:54.048 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "diskprediction_local", 2024-09-11T23:39:54.048 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-09-11T23:39:54.048 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-09-11T23:39:54.048 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-09-11T23:39:54.048 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-09-11T23:39:54.048 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-09-11T23:39:54.048 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.048 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.048 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.048 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:54.048 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.049 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.049 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:54.049 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:54.049 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:54.049 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:54.049 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:54.049 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:54.049 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:54.049 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:54.049 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.049 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.049 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.049 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.049 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.050 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-09-11T23:39:54.050 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-09-11T23:39:54.050 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.050 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.050 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.050 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:54.050 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.050 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.050 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.050 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.050 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.050 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.050 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.050 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.051 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-09-11T23:39:54.051 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-09-11T23:39:54.051 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.051 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.051 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.051 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-09-11T23:39:54.051 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.051 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.051 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:54.051 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:54.051 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:54.051 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:54.051 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:54.051 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:54.051 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:54.052 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:54.052 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.052 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.052 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.052 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.052 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.052 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-09-11T23:39:54.052 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-09-11T23:39:54.052 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.052 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.052 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.052 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:54.052 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.053 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.053 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.053 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.053 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.053 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.053 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.053 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.053 INFO:teuthology.orchestra.run.smithi026.stdout: "predict_interval": { 2024-09-11T23:39:54.053 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "predict_interval", 2024-09-11T23:39:54.053 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.053 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.053 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.053 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "86400", 2024-09-11T23:39:54.053 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.054 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.054 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.054 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.054 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.054 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.054 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.054 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.054 INFO:teuthology.orchestra.run.smithi026.stdout: "predictor_model": { 2024-09-11T23:39:54.054 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "predictor_model", 2024-09-11T23:39:54.054 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.054 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.054 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.054 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "prophetstor", 2024-09-11T23:39:54.054 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.055 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.055 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.055 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.055 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.055 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.055 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.055 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.055 INFO:teuthology.orchestra.run.smithi026.stdout: "sleep_interval": { 2024-09-11T23:39:54.055 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sleep_interval", 2024-09-11T23:39:54.055 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.055 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.055 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.055 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "600", 2024-09-11T23:39:54.055 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.056 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.056 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.056 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.056 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.056 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.056 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.056 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:54.056 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:54.056 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.056 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-11T23:39:54.056 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "influx", 2024-09-11T23:39:54.056 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": false, 2024-09-11T23:39:54.056 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "influxdb python module not found", 2024-09-11T23:39:54.056 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-09-11T23:39:54.057 INFO:teuthology.orchestra.run.smithi026.stdout: "batch_size": { 2024-09-11T23:39:54.057 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "batch_size", 2024-09-11T23:39:54.057 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-09-11T23:39:54.057 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.057 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.057 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "5000", 2024-09-11T23:39:54.057 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.057 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.057 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.057 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-11T23:39:54.057 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.057 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.057 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.057 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.058 INFO:teuthology.orchestra.run.smithi026.stdout: "database": { 2024-09-11T23:39:54.058 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "database", 2024-09-11T23:39:54.058 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.058 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.058 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.058 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "ceph", 2024-09-11T23:39:54.058 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.058 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.058 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.058 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-11T23:39:54.058 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.058 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.058 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.059 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.059 INFO:teuthology.orchestra.run.smithi026.stdout: "hostname": { 2024-09-11T23:39:54.059 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "hostname", 2024-09-11T23:39:54.059 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.059 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.059 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.059 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:54.059 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.059 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.059 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.059 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "InfluxDB server hostname", 2024-09-11T23:39:54.059 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.059 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.059 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.060 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.060 INFO:teuthology.orchestra.run.smithi026.stdout: "interval": { 2024-09-11T23:39:54.060 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "interval", 2024-09-11T23:39:54.060 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-09-11T23:39:54.060 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.060 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.060 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "30", 2024-09-11T23:39:54.060 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "5", 2024-09-11T23:39:54.060 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.060 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.060 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-11T23:39:54.060 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.060 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.060 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.061 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.061 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-09-11T23:39:54.061 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-09-11T23:39:54.061 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.061 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.061 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.061 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:54.061 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.061 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.061 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:54.061 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:54.061 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:54.061 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:54.061 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:54.062 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:54.062 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:54.062 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:54.062 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.062 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.062 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.062 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.062 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.062 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-09-11T23:39:54.062 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-09-11T23:39:54.062 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.062 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.063 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.063 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:54.063 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.063 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.063 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.063 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.063 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.063 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.063 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.063 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.063 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-09-11T23:39:54.063 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-09-11T23:39:54.063 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.063 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.063 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.064 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-09-11T23:39:54.064 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.064 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.064 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:54.064 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:54.064 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:54.064 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:54.064 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:54.064 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:54.064 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:54.064 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:54.064 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.064 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.065 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.065 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.065 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.065 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-09-11T23:39:54.065 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-09-11T23:39:54.065 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.065 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.065 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.065 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:54.065 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.065 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.065 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.065 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.065 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.066 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.066 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.066 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.066 INFO:teuthology.orchestra.run.smithi026.stdout: "password": { 2024-09-11T23:39:54.066 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "password", 2024-09-11T23:39:54.066 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.066 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.066 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.066 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:54.066 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.066 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.066 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.066 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "password of InfluxDB server user", 2024-09-11T23:39:54.066 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.067 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.067 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.067 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.067 INFO:teuthology.orchestra.run.smithi026.stdout: "port": { 2024-09-11T23:39:54.067 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "port", 2024-09-11T23:39:54.067 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-09-11T23:39:54.067 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.067 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.067 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "8086", 2024-09-11T23:39:54.067 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.067 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.067 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.067 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "InfluxDB server port", 2024-09-11T23:39:54.067 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.067 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.068 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.068 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.068 INFO:teuthology.orchestra.run.smithi026.stdout: "ssl": { 2024-09-11T23:39:54.068 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ssl", 2024-09-11T23:39:54.068 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.068 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.068 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.068 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "false", 2024-09-11T23:39:54.068 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.068 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.068 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.068 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-11T23:39:54.068 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.069 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.069 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.069 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.069 INFO:teuthology.orchestra.run.smithi026.stdout: "threads": { 2024-09-11T23:39:54.069 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "threads", 2024-09-11T23:39:54.069 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-09-11T23:39:54.069 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.069 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.069 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "5", 2024-09-11T23:39:54.069 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "1", 2024-09-11T23:39:54.069 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "32", 2024-09-11T23:39:54.069 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.069 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-11T23:39:54.069 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.069 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.070 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.070 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.070 INFO:teuthology.orchestra.run.smithi026.stdout: "username": { 2024-09-11T23:39:54.070 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "username", 2024-09-11T23:39:54.070 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.070 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.070 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.070 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:54.070 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.070 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.070 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.070 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "username of InfluxDB server user", 2024-09-11T23:39:54.070 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.070 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.071 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.071 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.071 INFO:teuthology.orchestra.run.smithi026.stdout: "verify_ssl": { 2024-09-11T23:39:54.071 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "verify_ssl", 2024-09-11T23:39:54.071 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.071 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.071 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.071 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "true", 2024-09-11T23:39:54.071 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.071 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.071 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.071 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-11T23:39:54.071 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.071 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.072 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.072 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:54.072 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:54.072 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.072 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-11T23:39:54.072 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "insights", 2024-09-11T23:39:54.072 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-09-11T23:39:54.072 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-09-11T23:39:54.072 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-09-11T23:39:54.072 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-09-11T23:39:54.072 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-09-11T23:39:54.072 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.072 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.073 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.073 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:54.073 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.073 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.073 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:54.073 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:54.073 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:54.073 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:54.073 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:54.073 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:54.073 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:54.073 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:54.073 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.073 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.073 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.074 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.074 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.074 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-09-11T23:39:54.074 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-09-11T23:39:54.074 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.074 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.074 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.074 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:54.074 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.074 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.074 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.074 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.074 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.074 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.075 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.075 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.075 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-09-11T23:39:54.075 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-09-11T23:39:54.075 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.075 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.075 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.075 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-09-11T23:39:54.075 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.075 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.075 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:54.075 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:54.075 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:54.075 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:54.075 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:54.076 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:54.076 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:54.076 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:54.076 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.076 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.076 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.076 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.076 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.076 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-09-11T23:39:54.076 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-09-11T23:39:54.076 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.076 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.076 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.076 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:54.077 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.077 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.077 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.077 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.077 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.077 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.077 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.077 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:54.077 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:54.077 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.077 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-11T23:39:54.077 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "iostat", 2024-09-11T23:39:54.077 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-09-11T23:39:54.077 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-09-11T23:39:54.078 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-09-11T23:39:54.078 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-09-11T23:39:54.078 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-09-11T23:39:54.078 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.078 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.078 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.078 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:54.078 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.078 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.078 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:54.078 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:54.078 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:54.078 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:54.079 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:54.079 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:54.079 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:54.079 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:54.079 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.079 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.079 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.079 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.079 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.079 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-09-11T23:39:54.079 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-09-11T23:39:54.079 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.079 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.079 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.079 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:54.080 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.080 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.080 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.080 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.080 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.080 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.080 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.080 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.080 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-09-11T23:39:54.080 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-09-11T23:39:54.080 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.080 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.080 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.080 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-09-11T23:39:54.080 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.081 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.081 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:54.081 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:54.081 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:54.081 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:54.081 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:54.081 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:54.081 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:54.081 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:54.081 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.081 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.081 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.081 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.081 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.081 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-09-11T23:39:54.082 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-09-11T23:39:54.082 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.082 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.082 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.082 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:54.082 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.082 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.082 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.082 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.082 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.082 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.082 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.082 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:54.083 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:54.083 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.083 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-11T23:39:54.083 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "k8sevents", 2024-09-11T23:39:54.083 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-09-11T23:39:54.083 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-09-11T23:39:54.083 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-09-11T23:39:54.083 INFO:teuthology.orchestra.run.smithi026.stdout: "ceph_event_retention_days": { 2024-09-11T23:39:54.083 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ceph_event_retention_days", 2024-09-11T23:39:54.083 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-09-11T23:39:54.083 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.083 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.083 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "7", 2024-09-11T23:39:54.083 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.084 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.084 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.084 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-11T23:39:54.084 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.084 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.084 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.084 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.084 INFO:teuthology.orchestra.run.smithi026.stdout: "config_check_secs": { 2024-09-11T23:39:54.084 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "config_check_secs", 2024-09-11T23:39:54.084 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-09-11T23:39:54.084 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.084 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.084 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "10", 2024-09-11T23:39:54.084 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "10", 2024-09-11T23:39:54.085 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.085 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.085 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-11T23:39:54.085 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.085 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.085 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.085 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.085 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-09-11T23:39:54.085 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-09-11T23:39:54.085 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.085 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.085 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.085 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:54.086 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.086 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.086 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:54.086 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:54.086 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:54.086 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:54.086 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:54.086 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:54.086 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:54.086 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:54.086 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.086 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.086 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.086 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.087 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.087 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-09-11T23:39:54.087 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-09-11T23:39:54.087 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.087 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.087 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.087 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:54.087 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.087 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.087 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.087 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.087 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.087 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.087 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.088 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.088 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-09-11T23:39:54.088 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-09-11T23:39:54.088 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.088 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.088 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.088 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-09-11T23:39:54.088 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.088 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.088 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:54.088 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:54.088 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:54.088 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:54.088 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:54.089 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:54.089 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:54.089 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:54.089 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.089 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.089 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.089 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.089 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.089 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-09-11T23:39:54.089 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-09-11T23:39:54.089 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.089 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.089 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.089 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:54.090 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.090 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.090 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.090 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.090 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.090 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.090 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.090 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:54.090 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:54.090 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.090 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-11T23:39:54.090 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "localpool", 2024-09-11T23:39:54.090 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-09-11T23:39:54.090 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-09-11T23:39:54.091 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-09-11T23:39:54.091 INFO:teuthology.orchestra.run.smithi026.stdout: "failure_domain": { 2024-09-11T23:39:54.091 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "failure_domain", 2024-09-11T23:39:54.091 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.091 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.091 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.091 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "host", 2024-09-11T23:39:54.091 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.091 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.091 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.091 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "failure domain for any created local pool", 2024-09-11T23:39:54.091 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-11T23:39:54.091 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.091 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.092 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.092 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-09-11T23:39:54.092 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-09-11T23:39:54.092 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.092 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.092 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.092 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:54.092 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.092 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.092 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:54.092 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:54.092 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:54.092 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:54.092 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:54.093 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:54.093 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:54.093 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:54.093 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.093 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.093 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.093 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.093 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.093 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-09-11T23:39:54.093 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-09-11T23:39:54.093 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.093 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.093 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.093 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:54.094 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.094 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.094 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.094 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.094 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.094 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.094 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.094 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.094 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-09-11T23:39:54.094 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-09-11T23:39:54.094 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.094 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.094 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.094 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-09-11T23:39:54.095 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.095 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.095 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:54.095 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:54.095 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:54.095 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:54.095 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:54.095 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:54.095 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:54.095 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:54.095 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.095 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.095 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.095 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.096 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.096 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-09-11T23:39:54.096 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-09-11T23:39:54.096 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.096 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.096 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.096 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:54.096 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.096 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.096 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.096 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.096 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.096 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.096 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.097 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.097 INFO:teuthology.orchestra.run.smithi026.stdout: "min_size": { 2024-09-11T23:39:54.097 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "min_size", 2024-09-11T23:39:54.097 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-09-11T23:39:54.097 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.097 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.097 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:54.097 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.097 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.097 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.097 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "default min_size for any created local pool", 2024-09-11T23:39:54.097 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-11T23:39:54.097 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.097 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.097 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.098 INFO:teuthology.orchestra.run.smithi026.stdout: "num_rep": { 2024-09-11T23:39:54.098 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "num_rep", 2024-09-11T23:39:54.098 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-09-11T23:39:54.098 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.098 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.098 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "3", 2024-09-11T23:39:54.098 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.098 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.098 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.098 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "default replica count for any created local pool", 2024-09-11T23:39:54.098 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.098 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.098 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.098 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.099 INFO:teuthology.orchestra.run.smithi026.stdout: "pg_num": { 2024-09-11T23:39:54.099 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "pg_num", 2024-09-11T23:39:54.099 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-09-11T23:39:54.099 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.099 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.099 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "128", 2024-09-11T23:39:54.099 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.099 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.099 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.099 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "default pg_num for any created local pool", 2024-09-11T23:39:54.100 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.100 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.100 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.100 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.100 INFO:teuthology.orchestra.run.smithi026.stdout: "prefix": { 2024-09-11T23:39:54.100 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "prefix", 2024-09-11T23:39:54.100 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.100 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.100 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.100 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:54.100 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.100 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.100 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.101 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "name prefix for any created local pool", 2024-09-11T23:39:54.101 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.101 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.101 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.101 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.101 INFO:teuthology.orchestra.run.smithi026.stdout: "subtree": { 2024-09-11T23:39:54.101 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "subtree", 2024-09-11T23:39:54.101 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.101 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.101 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.101 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "rack", 2024-09-11T23:39:54.101 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.102 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.102 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.102 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-11T23:39:54.102 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-11T23:39:54.102 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.102 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.102 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:54.102 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:54.102 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.102 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-11T23:39:54.102 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "mds_autoscaler", 2024-09-11T23:39:54.102 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-09-11T23:39:54.102 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-09-11T23:39:54.102 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-09-11T23:39:54.102 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-09-11T23:39:54.102 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-09-11T23:39:54.103 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.103 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.103 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.103 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:54.103 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.103 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.103 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:54.103 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:54.103 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:54.103 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:54.103 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:54.103 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:54.103 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:54.104 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:54.104 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.104 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.104 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.104 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.104 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.104 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-09-11T23:39:54.104 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-09-11T23:39:54.104 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.104 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.104 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.104 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:54.104 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.104 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.104 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.105 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.105 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.105 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.105 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.105 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.105 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-09-11T23:39:54.105 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-09-11T23:39:54.105 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.105 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.105 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.105 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-09-11T23:39:54.105 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.105 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.105 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:54.106 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:54.106 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:54.106 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:54.106 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:54.106 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:54.106 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:54.106 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:54.106 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.106 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.106 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.106 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.106 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.106 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-09-11T23:39:54.106 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-09-11T23:39:54.107 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.107 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.107 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.107 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:54.107 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.107 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.107 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.107 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.107 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.107 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.107 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.107 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:54.107 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:54.107 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.108 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-11T23:39:54.108 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "mirroring", 2024-09-11T23:39:54.108 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-09-11T23:39:54.108 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-09-11T23:39:54.108 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-09-11T23:39:54.108 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-09-11T23:39:54.108 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-09-11T23:39:54.108 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.108 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.108 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.108 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:54.108 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.108 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.109 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:54.109 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:54.109 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:54.109 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:54.109 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:54.109 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:54.109 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:54.109 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:54.109 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.109 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.109 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.109 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.109 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.109 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-09-11T23:39:54.110 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-09-11T23:39:54.110 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.110 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.110 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.110 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:54.110 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.110 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.110 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.110 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.110 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.110 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.110 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.110 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.110 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-09-11T23:39:54.111 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-09-11T23:39:54.111 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.111 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.111 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.111 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-09-11T23:39:54.111 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.111 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.111 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:54.111 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:54.111 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:54.111 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:54.111 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:54.111 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:54.111 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:54.112 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:54.112 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.112 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.112 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.112 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.112 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.112 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-09-11T23:39:54.112 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-09-11T23:39:54.112 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.112 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.112 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.112 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:54.112 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.112 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.112 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.113 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.113 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.113 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.113 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.113 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:54.113 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:54.113 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.113 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-11T23:39:54.113 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "nfs", 2024-09-11T23:39:54.113 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-09-11T23:39:54.113 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-09-11T23:39:54.113 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-09-11T23:39:54.114 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-09-11T23:39:54.114 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-09-11T23:39:54.114 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.114 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.114 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.114 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:54.114 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.114 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.114 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:54.114 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:54.114 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:54.114 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:54.114 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:54.114 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:54.114 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:54.115 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:54.115 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.115 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.115 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.115 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.115 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.115 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-09-11T23:39:54.115 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-09-11T23:39:54.115 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.115 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.115 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.115 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:54.115 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.115 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.116 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.116 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.116 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.116 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.116 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.116 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.116 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-09-11T23:39:54.116 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-09-11T23:39:54.116 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.116 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.116 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.116 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-09-11T23:39:54.116 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.117 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.117 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:54.117 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:54.117 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:54.117 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:54.117 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:54.117 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:54.117 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:54.117 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:54.117 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.117 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.117 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.117 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.117 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.118 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-09-11T23:39:54.118 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-09-11T23:39:54.118 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.118 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.118 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.118 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:54.118 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.118 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.118 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.118 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.118 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.118 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.118 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.118 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:54.118 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:54.119 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.119 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-11T23:39:54.119 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "orchestrator", 2024-09-11T23:39:54.119 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-09-11T23:39:54.119 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-09-11T23:39:54.119 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-09-11T23:39:54.119 INFO:teuthology.orchestra.run.smithi026.stdout: "fail_fs": { 2024-09-11T23:39:54.119 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "fail_fs", 2024-09-11T23:39:54.119 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.119 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.119 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.119 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:54.119 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.119 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.120 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.120 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-11T23:39:54.120 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.120 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.120 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.120 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.120 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-09-11T23:39:54.120 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-09-11T23:39:54.120 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.120 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.120 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.120 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:54.120 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.120 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.121 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:54.121 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:54.121 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:54.121 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:54.121 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:54.121 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:54.121 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:54.121 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:54.121 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.121 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.121 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.121 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.121 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.121 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-09-11T23:39:54.122 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-09-11T23:39:54.122 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.122 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.122 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.122 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:54.122 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.122 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.122 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.122 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.122 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.122 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.122 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.122 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.122 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-09-11T23:39:54.122 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-09-11T23:39:54.123 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.123 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.123 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.123 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-09-11T23:39:54.123 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.123 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.123 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:54.123 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:54.123 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:54.123 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:54.123 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:54.123 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:54.123 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:54.123 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:54.124 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.124 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.124 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.124 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.124 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.124 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-09-11T23:39:54.124 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-09-11T23:39:54.124 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.124 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.124 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.124 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:54.124 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.124 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.124 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.124 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.125 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.125 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.125 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.125 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.125 INFO:teuthology.orchestra.run.smithi026.stdout: "orchestrator": { 2024-09-11T23:39:54.125 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "orchestrator", 2024-09-11T23:39:54.125 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.125 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.125 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.125 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:54.125 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.125 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.125 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:54.125 INFO:teuthology.orchestra.run.smithi026.stdout: "cephadm", 2024-09-11T23:39:54.126 INFO:teuthology.orchestra.run.smithi026.stdout: "rook", 2024-09-11T23:39:54.126 INFO:teuthology.orchestra.run.smithi026.stdout: "test_orchestrator" 2024-09-11T23:39:54.126 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:54.126 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Orchestrator backend", 2024-09-11T23:39:54.126 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.126 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.126 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.126 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:54.126 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:54.126 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.126 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-11T23:39:54.126 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "osd_perf_query", 2024-09-11T23:39:54.126 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-09-11T23:39:54.126 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-09-11T23:39:54.126 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-09-11T23:39:54.127 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-09-11T23:39:54.127 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-09-11T23:39:54.127 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.127 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.127 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.127 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:54.127 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.127 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.127 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:54.127 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:54.127 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:54.127 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:54.127 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:54.128 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:54.128 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:54.128 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:54.128 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.128 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.128 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.128 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.128 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.128 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-09-11T23:39:54.128 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-09-11T23:39:54.128 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.128 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.128 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.128 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:54.129 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.129 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.129 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.129 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.129 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.129 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.129 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.129 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.129 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-09-11T23:39:54.129 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-09-11T23:39:54.129 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.129 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.129 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.129 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-09-11T23:39:54.130 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.130 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.130 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:54.130 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:54.130 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:54.130 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:54.130 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:54.130 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:54.130 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:54.130 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:54.130 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.130 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.130 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.130 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.130 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.131 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-09-11T23:39:54.131 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-09-11T23:39:54.131 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.131 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.131 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.131 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:54.131 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.131 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.131 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.131 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.131 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.131 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.131 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.131 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:54.132 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:54.132 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.132 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-11T23:39:54.132 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "osd_support", 2024-09-11T23:39:54.132 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-09-11T23:39:54.132 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-09-11T23:39:54.132 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-09-11T23:39:54.132 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-09-11T23:39:54.132 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-09-11T23:39:54.132 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.132 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.132 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.132 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:54.133 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.133 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.133 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:54.133 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:54.133 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:54.133 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:54.133 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:54.133 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:54.133 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:54.133 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:54.133 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.133 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.133 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.134 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.134 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.134 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-09-11T23:39:54.134 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-09-11T23:39:54.134 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.134 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.134 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.134 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:54.134 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.134 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.134 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.134 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.134 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.134 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.135 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.135 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.135 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-09-11T23:39:54.135 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-09-11T23:39:54.135 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.135 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.135 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.135 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-09-11T23:39:54.135 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.135 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.135 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:54.135 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:54.135 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:54.135 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:54.135 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:54.136 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:54.136 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:54.136 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:54.136 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.136 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.136 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.136 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.136 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.136 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-09-11T23:39:54.136 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-09-11T23:39:54.136 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.136 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.136 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.136 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:54.137 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.137 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.137 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.137 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.137 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.137 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.137 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.137 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:54.137 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:54.137 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.137 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-11T23:39:54.137 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "pg_autoscaler", 2024-09-11T23:39:54.137 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-09-11T23:39:54.137 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-09-11T23:39:54.138 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-09-11T23:39:54.138 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-09-11T23:39:54.138 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-09-11T23:39:54.138 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.138 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.138 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.138 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:54.138 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.138 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.138 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:54.138 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:54.138 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:54.138 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:54.138 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:54.138 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:54.139 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:54.139 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:54.139 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.139 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.139 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.139 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.139 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.139 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-09-11T23:39:54.139 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-09-11T23:39:54.139 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.139 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.139 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.139 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:54.139 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.140 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.140 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.140 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.140 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.140 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.140 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.140 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.140 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-09-11T23:39:54.140 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-09-11T23:39:54.140 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.140 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.140 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.140 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-09-11T23:39:54.140 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.141 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.141 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:54.141 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:54.141 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:54.141 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:54.141 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:54.141 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:54.141 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:54.141 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:54.141 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.141 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.141 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.141 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.141 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.142 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-09-11T23:39:54.142 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-09-11T23:39:54.142 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.142 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.142 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.142 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:54.142 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.142 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.142 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.142 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.142 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.142 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.142 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.143 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.143 INFO:teuthology.orchestra.run.smithi026.stdout: "sleep_interval": { 2024-09-11T23:39:54.143 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sleep_interval", 2024-09-11T23:39:54.143 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-09-11T23:39:54.143 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.143 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.143 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "60", 2024-09-11T23:39:54.143 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.143 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.143 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.143 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.143 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.143 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.143 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.143 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.144 INFO:teuthology.orchestra.run.smithi026.stdout: "threshold": { 2024-09-11T23:39:54.144 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "threshold", 2024-09-11T23:39:54.144 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "float", 2024-09-11T23:39:54.144 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.144 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.144 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "3.0", 2024-09-11T23:39:54.144 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "1.0", 2024-09-11T23:39:54.144 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.144 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.144 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "scaling threshold", 2024-09-11T23:39:54.144 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-11T23:39:54.144 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.144 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.145 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:54.145 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:54.145 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.145 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-11T23:39:54.145 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "progress", 2024-09-11T23:39:54.145 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-09-11T23:39:54.145 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-09-11T23:39:54.145 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-09-11T23:39:54.145 INFO:teuthology.orchestra.run.smithi026.stdout: "allow_pg_recovery_event": { 2024-09-11T23:39:54.145 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "allow_pg_recovery_event", 2024-09-11T23:39:54.145 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.145 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.145 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.145 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:54.146 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.146 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.146 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.146 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-11T23:39:54.146 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.146 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.146 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.146 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.146 INFO:teuthology.orchestra.run.smithi026.stdout: "enabled": { 2024-09-11T23:39:54.146 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "enabled", 2024-09-11T23:39:54.146 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.146 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.146 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.147 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-09-11T23:39:54.147 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.147 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.147 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.147 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.147 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.147 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.147 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.147 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.147 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-09-11T23:39:54.147 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-09-11T23:39:54.147 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.147 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.147 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.147 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:54.148 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.148 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.148 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:54.148 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:54.148 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:54.148 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:54.148 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:54.148 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:54.148 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:54.148 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:54.148 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.148 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.148 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.148 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.149 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.149 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-09-11T23:39:54.149 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-09-11T23:39:54.149 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.149 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.149 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.149 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:54.149 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.149 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.149 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.149 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.149 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.149 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.149 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.150 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.150 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-09-11T23:39:54.150 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-09-11T23:39:54.150 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.150 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.150 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.150 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-09-11T23:39:54.150 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.150 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.150 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:54.150 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:54.150 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:54.150 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:54.150 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:54.150 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:54.151 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:54.151 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:54.151 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.151 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.151 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.151 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.151 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.151 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-09-11T23:39:54.151 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-09-11T23:39:54.151 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.151 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.151 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.151 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:54.151 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.152 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.152 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.152 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.152 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.152 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.152 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.152 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.152 INFO:teuthology.orchestra.run.smithi026.stdout: "max_completed_events": { 2024-09-11T23:39:54.152 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "max_completed_events", 2024-09-11T23:39:54.152 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-09-11T23:39:54.152 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.152 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.152 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "50", 2024-09-11T23:39:54.152 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.153 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.153 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.153 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "number of past completed events to remember", 2024-09-11T23:39:54.153 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.153 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.153 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.153 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.153 INFO:teuthology.orchestra.run.smithi026.stdout: "sleep_interval": { 2024-09-11T23:39:54.153 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sleep_interval", 2024-09-11T23:39:54.153 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-09-11T23:39:54.153 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.153 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.153 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "5", 2024-09-11T23:39:54.153 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.153 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.154 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.154 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "how long the module is going to sleep", 2024-09-11T23:39:54.154 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.154 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.154 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.154 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:54.154 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:54.154 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.154 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-11T23:39:54.154 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "prometheus", 2024-09-11T23:39:54.154 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-09-11T23:39:54.154 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-09-11T23:39:54.154 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-09-11T23:39:54.154 INFO:teuthology.orchestra.run.smithi026.stdout: "cache": { 2024-09-11T23:39:54.155 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "cache", 2024-09-11T23:39:54.155 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.155 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.155 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.155 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-09-11T23:39:54.155 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.155 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.155 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.155 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.155 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.155 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.155 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.155 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.156 INFO:teuthology.orchestra.run.smithi026.stdout: "exclude_perf_counters": { 2024-09-11T23:39:54.156 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "exclude_perf_counters", 2024-09-11T23:39:54.156 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.156 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.156 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.156 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-09-11T23:39:54.156 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.156 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.156 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.156 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-11T23:39:54.156 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-11T23:39:54.156 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.156 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.156 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.157 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-09-11T23:39:54.157 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-09-11T23:39:54.157 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.157 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.157 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.157 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:54.157 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.157 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.157 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:54.157 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:54.157 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:54.157 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:54.157 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:54.157 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:54.158 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:54.158 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:54.158 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.158 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.158 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.158 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.158 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.158 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-09-11T23:39:54.158 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-09-11T23:39:54.158 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.158 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.158 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.158 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:54.158 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.159 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.159 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.159 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.159 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.159 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.159 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.159 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.159 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-09-11T23:39:54.159 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-09-11T23:39:54.159 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.159 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.159 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.159 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-09-11T23:39:54.159 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.159 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.160 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:54.160 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:54.160 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:54.160 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:54.160 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:54.160 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:54.160 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:54.160 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:54.160 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.160 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.160 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.160 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.160 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.161 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-09-11T23:39:54.161 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-09-11T23:39:54.161 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.161 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.161 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.161 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:54.161 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.161 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.161 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.161 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.161 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.161 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.161 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.161 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.161 INFO:teuthology.orchestra.run.smithi026.stdout: "rbd_stats_pools": { 2024-09-11T23:39:54.162 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rbd_stats_pools", 2024-09-11T23:39:54.162 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.162 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.162 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.162 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:54.162 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.162 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.162 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.162 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.162 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.162 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.162 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.162 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.162 INFO:teuthology.orchestra.run.smithi026.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-11T23:39:54.163 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-11T23:39:54.163 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-09-11T23:39:54.163 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.163 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.163 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "300", 2024-09-11T23:39:54.163 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.163 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.163 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.163 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.163 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.163 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.163 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.163 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.163 INFO:teuthology.orchestra.run.smithi026.stdout: "scrape_interval": { 2024-09-11T23:39:54.163 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "scrape_interval", 2024-09-11T23:39:54.164 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "float", 2024-09-11T23:39:54.164 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.164 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.164 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "15.0", 2024-09-11T23:39:54.164 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.164 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.164 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.164 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.164 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.164 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.164 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.164 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.164 INFO:teuthology.orchestra.run.smithi026.stdout: "server_addr": { 2024-09-11T23:39:54.164 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "server_addr", 2024-09-11T23:39:54.165 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.165 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.165 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.165 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "::", 2024-09-11T23:39:54.165 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.165 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.165 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.165 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-11T23:39:54.165 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.165 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.165 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.165 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.165 INFO:teuthology.orchestra.run.smithi026.stdout: "server_port": { 2024-09-11T23:39:54.165 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "server_port", 2024-09-11T23:39:54.166 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-09-11T23:39:54.166 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.166 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.166 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "9283", 2024-09-11T23:39:54.166 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.166 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.166 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.166 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-11T23:39:54.168 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.168 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.168 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.168 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.168 INFO:teuthology.orchestra.run.smithi026.stdout: "stale_cache_strategy": { 2024-09-11T23:39:54.169 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "stale_cache_strategy", 2024-09-11T23:39:54.169 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.169 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.169 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.169 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "log", 2024-09-11T23:39:54.169 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.169 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.169 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.169 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.169 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.169 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.169 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.169 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.170 INFO:teuthology.orchestra.run.smithi026.stdout: "standby_behaviour": { 2024-09-11T23:39:54.170 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "standby_behaviour", 2024-09-11T23:39:54.170 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.170 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.170 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.170 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "default", 2024-09-11T23:39:54.170 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.170 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.170 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:54.170 INFO:teuthology.orchestra.run.smithi026.stdout: "default", 2024-09-11T23:39:54.170 INFO:teuthology.orchestra.run.smithi026.stdout: "error" 2024-09-11T23:39:54.170 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:54.170 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.170 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.171 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.171 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.171 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.171 INFO:teuthology.orchestra.run.smithi026.stdout: "standby_error_status_code": { 2024-09-11T23:39:54.171 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "standby_error_status_code", 2024-09-11T23:39:54.171 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-09-11T23:39:54.171 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.171 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.171 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "500", 2024-09-11T23:39:54.171 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "400", 2024-09-11T23:39:54.171 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "599", 2024-09-11T23:39:54.171 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.171 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.171 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.172 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.172 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.172 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:54.172 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:54.172 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.172 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-11T23:39:54.172 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rbd_support", 2024-09-11T23:39:54.172 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-09-11T23:39:54.172 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-09-11T23:39:54.172 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-09-11T23:39:54.172 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-09-11T23:39:54.172 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-09-11T23:39:54.172 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.173 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.173 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.173 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:54.173 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.173 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.173 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:54.173 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:54.173 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:54.173 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:54.173 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:54.173 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:54.173 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:54.173 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:54.173 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.173 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.174 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.174 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.174 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.174 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-09-11T23:39:54.174 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-09-11T23:39:54.174 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.174 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.174 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.174 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:54.174 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.174 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.174 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.174 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.174 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.175 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.175 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.175 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.175 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-09-11T23:39:54.175 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-09-11T23:39:54.175 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.175 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.175 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.175 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-09-11T23:39:54.175 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.175 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.175 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:54.175 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:54.175 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:54.175 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:54.176 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:54.176 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:54.176 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:54.176 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:54.176 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.176 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.176 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.176 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.176 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.176 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-09-11T23:39:54.176 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-09-11T23:39:54.176 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.176 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.176 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.177 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:54.177 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.177 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.177 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.177 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.177 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.177 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.177 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.177 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.177 INFO:teuthology.orchestra.run.smithi026.stdout: "max_concurrent_snap_create": { 2024-09-11T23:39:54.177 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "max_concurrent_snap_create", 2024-09-11T23:39:54.177 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-09-11T23:39:54.177 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.177 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.177 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "10", 2024-09-11T23:39:54.178 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.178 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.178 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.178 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.178 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.178 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.178 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.178 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.178 INFO:teuthology.orchestra.run.smithi026.stdout: "mirror_snapshot_schedule": { 2024-09-11T23:39:54.178 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "mirror_snapshot_schedule", 2024-09-11T23:39:54.178 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.178 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.178 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.178 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:54.179 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.179 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.179 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.179 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.179 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.179 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.179 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.179 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.179 INFO:teuthology.orchestra.run.smithi026.stdout: "trash_purge_schedule": { 2024-09-11T23:39:54.179 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "trash_purge_schedule", 2024-09-11T23:39:54.179 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.179 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.179 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.179 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:54.179 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.180 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.180 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.180 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.180 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.180 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.180 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.180 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:54.180 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:54.180 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.180 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-11T23:39:54.180 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "restful", 2024-09-11T23:39:54.180 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-09-11T23:39:54.180 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-09-11T23:39:54.180 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-09-11T23:39:54.181 INFO:teuthology.orchestra.run.smithi026.stdout: "enable_auth": { 2024-09-11T23:39:54.181 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "enable_auth", 2024-09-11T23:39:54.181 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.181 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.181 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.181 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-09-11T23:39:54.181 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.181 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.181 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.181 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.181 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.181 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.181 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.181 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.181 INFO:teuthology.orchestra.run.smithi026.stdout: "key_file": { 2024-09-11T23:39:54.182 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "key_file", 2024-09-11T23:39:54.182 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.182 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.182 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.182 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:54.182 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.182 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.182 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.182 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.182 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.182 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.182 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.182 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.182 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-09-11T23:39:54.182 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-09-11T23:39:54.183 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.183 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.183 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.183 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:54.183 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.183 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.183 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:54.183 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:54.183 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:54.183 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:54.183 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:54.183 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:54.183 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:54.183 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:54.184 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.184 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.184 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.184 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.184 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.184 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-09-11T23:39:54.184 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-09-11T23:39:54.184 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.184 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.184 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.184 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:54.184 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.184 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.184 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.185 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.185 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.185 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.185 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.185 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.185 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-09-11T23:39:54.185 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-09-11T23:39:54.185 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.185 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.185 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.185 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-09-11T23:39:54.185 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.185 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.185 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:54.186 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:54.186 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:54.186 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:54.186 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:54.186 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:54.186 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:54.186 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:54.186 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.186 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.186 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.186 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.186 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.186 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-09-11T23:39:54.186 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-09-11T23:39:54.187 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.187 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.187 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.187 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:54.187 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.187 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.187 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.187 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.187 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.187 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.187 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.187 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.187 INFO:teuthology.orchestra.run.smithi026.stdout: "server_addr": { 2024-09-11T23:39:54.187 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "server_addr", 2024-09-11T23:39:54.188 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.188 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.188 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.188 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:54.188 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.188 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.188 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.188 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.188 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.188 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.188 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.188 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.188 INFO:teuthology.orchestra.run.smithi026.stdout: "server_port": { 2024-09-11T23:39:54.188 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "server_port", 2024-09-11T23:39:54.188 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.189 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.189 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.189 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:54.189 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.189 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.189 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.189 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.189 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.189 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.189 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.189 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:54.189 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:54.189 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.189 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-11T23:39:54.190 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rgw", 2024-09-11T23:39:54.190 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-09-11T23:39:54.190 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-09-11T23:39:54.190 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-09-11T23:39:54.190 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-09-11T23:39:54.190 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-09-11T23:39:54.190 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.190 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.190 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.190 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:54.190 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.190 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.190 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:54.190 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:54.191 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:54.191 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:54.191 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:54.191 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:54.191 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:54.191 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:54.191 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.191 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.191 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.191 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.191 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.191 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-09-11T23:39:54.191 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-09-11T23:39:54.191 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.192 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.192 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.192 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:54.192 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.192 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.192 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.192 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.192 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.192 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.192 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.192 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.192 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-09-11T23:39:54.192 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-09-11T23:39:54.192 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.193 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.193 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.193 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-09-11T23:39:54.193 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.193 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.193 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:54.193 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:54.193 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:54.193 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:54.193 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:54.193 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:54.193 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:54.193 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:54.193 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.193 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.194 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.194 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.194 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.194 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-09-11T23:39:54.194 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-09-11T23:39:54.194 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.194 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.194 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.194 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:54.194 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.194 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.194 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.194 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.194 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.194 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.195 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.195 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:54.195 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:54.195 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.195 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-11T23:39:54.195 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rook", 2024-09-11T23:39:54.195 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-09-11T23:39:54.195 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-09-11T23:39:54.195 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-09-11T23:39:54.195 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-09-11T23:39:54.195 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-09-11T23:39:54.195 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.195 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.195 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.196 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:54.196 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.196 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.196 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:54.196 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:54.196 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:54.196 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:54.196 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:54.196 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:54.196 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:54.196 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:54.196 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.196 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.196 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.196 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.197 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.197 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-09-11T23:39:54.197 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-09-11T23:39:54.197 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.197 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.197 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.197 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:54.197 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.197 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.197 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.197 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.197 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.197 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.198 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.198 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.198 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-09-11T23:39:54.198 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-09-11T23:39:54.198 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.198 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.198 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.198 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-09-11T23:39:54.198 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.198 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.198 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:54.198 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:54.198 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:54.198 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:54.199 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:54.199 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:54.199 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:54.199 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:54.199 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.199 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.199 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.199 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.199 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.199 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-09-11T23:39:54.199 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-09-11T23:39:54.199 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.199 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.200 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.200 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:54.200 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.200 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.200 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.200 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.200 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.200 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.200 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.200 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.200 INFO:teuthology.orchestra.run.smithi026.stdout: "storage_class": { 2024-09-11T23:39:54.200 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "storage_class", 2024-09-11T23:39:54.200 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.200 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.201 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.201 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "local", 2024-09-11T23:39:54.201 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.201 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.201 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.201 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-11T23:39:54.201 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.201 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.201 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.201 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:54.201 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:54.201 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.201 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-11T23:39:54.201 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "selftest", 2024-09-11T23:39:54.202 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-09-11T23:39:54.202 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-09-11T23:39:54.202 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-09-11T23:39:54.202 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-09-11T23:39:54.202 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-09-11T23:39:54.202 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.202 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.202 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.202 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:54.202 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.202 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.202 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:54.202 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:54.202 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:54.203 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:54.203 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:54.203 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:54.203 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:54.203 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:54.203 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.203 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.203 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.203 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.203 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.203 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-09-11T23:39:54.203 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-09-11T23:39:54.203 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.203 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.204 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.204 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:54.204 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.204 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.204 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.204 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.204 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.204 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.204 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.204 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.204 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-09-11T23:39:54.204 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-09-11T23:39:54.204 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.204 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.205 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.205 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-09-11T23:39:54.205 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.205 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.205 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:54.205 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:54.205 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:54.205 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:54.205 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:54.205 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:54.205 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:54.205 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:54.205 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.205 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.205 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.206 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.206 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.206 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-09-11T23:39:54.206 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-09-11T23:39:54.206 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.206 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.206 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.206 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:54.206 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.206 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.206 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.206 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.206 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.206 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.207 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.207 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.207 INFO:teuthology.orchestra.run.smithi026.stdout: "roption1": { 2024-09-11T23:39:54.207 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "roption1", 2024-09-11T23:39:54.207 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.207 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.207 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.207 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:54.207 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.207 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.207 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.207 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.207 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.207 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.208 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.208 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.208 INFO:teuthology.orchestra.run.smithi026.stdout: "roption2": { 2024-09-11T23:39:54.208 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "roption2", 2024-09-11T23:39:54.208 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.208 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.208 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.208 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "xyz", 2024-09-11T23:39:54.208 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.208 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.208 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.208 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.208 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.208 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.209 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.209 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.209 INFO:teuthology.orchestra.run.smithi026.stdout: "rwoption1": { 2024-09-11T23:39:54.209 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rwoption1", 2024-09-11T23:39:54.209 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.209 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.209 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.209 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:54.209 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.209 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.209 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.209 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.209 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.209 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.210 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.210 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.210 INFO:teuthology.orchestra.run.smithi026.stdout: "rwoption2": { 2024-09-11T23:39:54.210 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rwoption2", 2024-09-11T23:39:54.210 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-09-11T23:39:54.210 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.210 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.210 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:54.210 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.210 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.210 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.210 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.210 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.210 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.211 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.211 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.211 INFO:teuthology.orchestra.run.smithi026.stdout: "rwoption3": { 2024-09-11T23:39:54.211 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rwoption3", 2024-09-11T23:39:54.211 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "float", 2024-09-11T23:39:54.211 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.211 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.211 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:54.211 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.211 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.211 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.211 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.211 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.211 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.212 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.212 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.212 INFO:teuthology.orchestra.run.smithi026.stdout: "rwoption4": { 2024-09-11T23:39:54.212 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rwoption4", 2024-09-11T23:39:54.212 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.212 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.212 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.212 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:54.212 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.212 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.212 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.212 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.212 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.212 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.213 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.213 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.213 INFO:teuthology.orchestra.run.smithi026.stdout: "rwoption5": { 2024-09-11T23:39:54.213 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rwoption5", 2024-09-11T23:39:54.213 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.213 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.213 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.213 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:54.213 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.213 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.213 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.213 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.213 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.214 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.214 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.214 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.214 INFO:teuthology.orchestra.run.smithi026.stdout: "rwoption6": { 2024-09-11T23:39:54.214 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rwoption6", 2024-09-11T23:39:54.214 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.214 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.214 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.214 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-09-11T23:39:54.214 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.214 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.214 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.214 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.214 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.215 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.215 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.215 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.215 INFO:teuthology.orchestra.run.smithi026.stdout: "rwoption7": { 2024-09-11T23:39:54.215 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rwoption7", 2024-09-11T23:39:54.215 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-09-11T23:39:54.215 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.215 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.215 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:54.215 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "1", 2024-09-11T23:39:54.215 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "42", 2024-09-11T23:39:54.215 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.215 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.215 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.216 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.216 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.216 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.216 INFO:teuthology.orchestra.run.smithi026.stdout: "testkey": { 2024-09-11T23:39:54.216 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "testkey", 2024-09-11T23:39:54.216 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.216 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.216 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.216 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:54.216 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.216 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.216 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.216 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.216 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.216 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.217 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.217 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.217 INFO:teuthology.orchestra.run.smithi026.stdout: "testlkey": { 2024-09-11T23:39:54.217 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "testlkey", 2024-09-11T23:39:54.217 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.217 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.217 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.217 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:54.217 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.217 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.217 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.217 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.217 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.217 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.218 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.218 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.218 INFO:teuthology.orchestra.run.smithi026.stdout: "testnewline": { 2024-09-11T23:39:54.218 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "testnewline", 2024-09-11T23:39:54.218 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.218 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.218 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.218 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:54.218 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.218 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.218 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.218 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.218 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.218 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.219 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.219 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:54.219 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:54.219 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.219 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-11T23:39:54.219 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "snap_schedule", 2024-09-11T23:39:54.219 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-09-11T23:39:54.219 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-09-11T23:39:54.219 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-09-11T23:39:54.219 INFO:teuthology.orchestra.run.smithi026.stdout: "allow_m_granularity": { 2024-09-11T23:39:54.219 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "allow_m_granularity", 2024-09-11T23:39:54.219 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.219 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.219 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.219 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:54.220 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.220 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.220 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.220 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "allow minute scheduled snapshots", 2024-09-11T23:39:54.220 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.220 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.220 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.220 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.220 INFO:teuthology.orchestra.run.smithi026.stdout: "dump_on_update": { 2024-09-11T23:39:54.220 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "dump_on_update", 2024-09-11T23:39:54.220 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.220 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.220 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.220 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:54.221 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.221 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.221 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.221 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "dump database to debug log on update", 2024-09-11T23:39:54.221 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.221 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.221 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.221 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.221 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-09-11T23:39:54.221 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-09-11T23:39:54.221 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.221 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.221 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.221 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:54.222 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.222 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.222 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:54.222 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:54.222 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:54.222 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:54.222 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:54.222 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:54.222 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:54.222 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:54.222 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.222 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.222 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.223 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.223 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.223 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-09-11T23:39:54.223 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-09-11T23:39:54.223 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.223 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.223 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.223 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:54.223 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.223 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.223 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.223 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.223 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.223 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.223 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.224 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.224 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-09-11T23:39:54.224 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-09-11T23:39:54.224 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.224 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.224 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.224 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-09-11T23:39:54.224 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.224 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.224 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:54.224 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:54.224 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:54.224 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:54.224 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:54.225 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:54.225 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:54.225 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:54.225 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.225 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.225 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.225 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.225 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.225 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-09-11T23:39:54.225 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-09-11T23:39:54.225 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.225 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.225 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.225 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:54.226 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.226 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.226 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.226 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.226 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.226 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.226 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.226 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:54.226 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:54.226 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.226 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-11T23:39:54.226 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "stats", 2024-09-11T23:39:54.226 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-09-11T23:39:54.226 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-09-11T23:39:54.227 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-09-11T23:39:54.227 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-09-11T23:39:54.227 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-09-11T23:39:54.227 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.227 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.227 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.227 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:54.227 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.227 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.227 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:54.227 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:54.227 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:54.227 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:54.227 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:54.228 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:54.228 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:54.228 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:54.228 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.228 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.228 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.228 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.228 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.228 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-09-11T23:39:54.228 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-09-11T23:39:54.228 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.228 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.228 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.228 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:54.229 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.229 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.229 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.229 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.229 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.229 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.229 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.229 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.229 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-09-11T23:39:54.229 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-09-11T23:39:54.229 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.229 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.229 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.229 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-09-11T23:39:54.230 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.230 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.230 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:54.230 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:54.230 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:54.230 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:54.230 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:54.230 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:54.230 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:54.230 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:54.230 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.230 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.230 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.230 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.230 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.231 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-09-11T23:39:54.231 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-09-11T23:39:54.231 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.231 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.231 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.231 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:54.231 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.231 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.231 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.231 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.231 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.231 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.231 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.231 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:54.232 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:54.232 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.232 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-11T23:39:54.232 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "status", 2024-09-11T23:39:54.232 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-09-11T23:39:54.232 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-09-11T23:39:54.232 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-09-11T23:39:54.232 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-09-11T23:39:54.232 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-09-11T23:39:54.232 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.232 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.232 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.232 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:54.232 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.232 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.233 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:54.233 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:54.233 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:54.233 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:54.233 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:54.233 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:54.233 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:54.233 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:54.233 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.233 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.233 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.233 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.233 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.233 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-09-11T23:39:54.234 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-09-11T23:39:54.234 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.234 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.234 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.234 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:54.234 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.234 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.234 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.234 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.234 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.234 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.234 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.234 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.234 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-09-11T23:39:54.234 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-09-11T23:39:54.235 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.235 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.235 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.235 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-09-11T23:39:54.235 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.235 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.235 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:54.235 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:54.235 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:54.235 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:54.235 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:54.235 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:54.235 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:54.235 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:54.236 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.236 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.236 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.236 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.236 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.236 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-09-11T23:39:54.236 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-09-11T23:39:54.236 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.236 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.236 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.236 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:54.236 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.236 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.236 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.237 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.237 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.237 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.237 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.237 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:54.237 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:54.237 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.237 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-11T23:39:54.237 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "telegraf", 2024-09-11T23:39:54.237 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-09-11T23:39:54.237 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-09-11T23:39:54.237 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-09-11T23:39:54.237 INFO:teuthology.orchestra.run.smithi026.stdout: "address": { 2024-09-11T23:39:54.237 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "address", 2024-09-11T23:39:54.237 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.238 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.238 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.238 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-11T23:39:54.238 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.238 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.238 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.238 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.238 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.238 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.238 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.238 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.238 INFO:teuthology.orchestra.run.smithi026.stdout: "interval": { 2024-09-11T23:39:54.238 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "interval", 2024-09-11T23:39:54.239 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-09-11T23:39:54.239 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.239 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.239 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "15", 2024-09-11T23:39:54.239 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.239 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.239 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.239 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.239 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.239 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.239 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.239 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.239 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-09-11T23:39:54.239 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-09-11T23:39:54.240 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.240 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.240 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.240 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:54.240 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.240 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.240 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:54.240 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:54.240 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:54.240 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:54.240 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:54.240 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:54.240 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:54.240 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:54.241 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.241 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.241 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.241 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.241 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.241 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-09-11T23:39:54.241 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-09-11T23:39:54.241 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.241 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.241 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.241 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:54.241 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.241 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.241 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.241 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.242 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.242 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.242 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.242 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.242 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-09-11T23:39:54.242 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-09-11T23:39:54.242 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.242 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.242 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.242 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-09-11T23:39:54.242 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.242 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.242 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:54.242 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:54.243 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:54.243 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:54.243 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:54.243 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:54.243 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:54.243 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:54.243 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.243 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.243 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.243 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.243 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.243 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-09-11T23:39:54.243 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-09-11T23:39:54.243 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.244 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.244 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.244 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:54.244 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.244 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.244 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.244 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.244 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.244 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.244 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.244 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:54.244 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:54.244 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.244 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-11T23:39:54.244 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "telemetry", 2024-09-11T23:39:54.245 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-09-11T23:39:54.245 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-09-11T23:39:54.245 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-09-11T23:39:54.245 INFO:teuthology.orchestra.run.smithi026.stdout: "channel_basic": { 2024-09-11T23:39:54.245 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "channel_basic", 2024-09-11T23:39:54.245 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.245 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.245 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.245 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-09-11T23:39:54.245 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.245 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.245 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.245 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-11T23:39:54.245 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.246 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.246 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.246 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.246 INFO:teuthology.orchestra.run.smithi026.stdout: "channel_crash": { 2024-09-11T23:39:54.246 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "channel_crash", 2024-09-11T23:39:54.246 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.246 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.246 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.246 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-09-11T23:39:54.246 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.246 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.246 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.246 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-11T23:39:54.246 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.246 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.247 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.247 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.247 INFO:teuthology.orchestra.run.smithi026.stdout: "channel_device": { 2024-09-11T23:39:54.247 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "channel_device", 2024-09-11T23:39:54.247 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.247 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.247 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.247 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-09-11T23:39:54.247 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.247 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.247 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.247 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-11T23:39:54.247 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.247 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.248 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.248 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.248 INFO:teuthology.orchestra.run.smithi026.stdout: "channel_ident": { 2024-09-11T23:39:54.248 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "channel_ident", 2024-09-11T23:39:54.248 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.248 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.248 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.248 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:54.248 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.248 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.248 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.248 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-11T23:39:54.248 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.248 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.249 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.249 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.249 INFO:teuthology.orchestra.run.smithi026.stdout: "channel_perf": { 2024-09-11T23:39:54.249 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "channel_perf", 2024-09-11T23:39:54.249 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.249 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.249 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.249 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:54.249 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.249 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.249 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.249 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-11T23:39:54.249 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.249 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.249 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.250 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.250 INFO:teuthology.orchestra.run.smithi026.stdout: "contact": { 2024-09-11T23:39:54.250 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "contact", 2024-09-11T23:39:54.250 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.250 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.250 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.250 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:54.250 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.250 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.250 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.250 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.250 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.250 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.250 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.251 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.251 INFO:teuthology.orchestra.run.smithi026.stdout: "description": { 2024-09-11T23:39:54.251 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "description", 2024-09-11T23:39:54.251 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.251 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.251 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.251 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:54.251 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.251 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.251 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.251 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.251 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.251 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.251 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.251 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.252 INFO:teuthology.orchestra.run.smithi026.stdout: "device_url": { 2024-09-11T23:39:54.252 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "device_url", 2024-09-11T23:39:54.252 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.252 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.252 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.252 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-11T23:39:54.252 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.252 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.252 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.252 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.252 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.252 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.252 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.253 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.253 INFO:teuthology.orchestra.run.smithi026.stdout: "enabled": { 2024-09-11T23:39:54.253 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "enabled", 2024-09-11T23:39:54.253 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.253 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.253 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.253 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:54.253 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.253 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.253 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.253 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.253 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.253 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.253 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.253 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.254 INFO:teuthology.orchestra.run.smithi026.stdout: "interval": { 2024-09-11T23:39:54.254 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "interval", 2024-09-11T23:39:54.254 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-09-11T23:39:54.254 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.254 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.254 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "24", 2024-09-11T23:39:54.254 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "8", 2024-09-11T23:39:54.254 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.254 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.254 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.254 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.254 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.254 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.254 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.255 INFO:teuthology.orchestra.run.smithi026.stdout: "last_opt_revision": { 2024-09-11T23:39:54.255 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "last_opt_revision", 2024-09-11T23:39:54.255 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-09-11T23:39:54.255 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.255 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.255 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "1", 2024-09-11T23:39:54.255 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.255 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.255 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.255 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.255 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.255 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.255 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.255 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.255 INFO:teuthology.orchestra.run.smithi026.stdout: "leaderboard": { 2024-09-11T23:39:54.256 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "leaderboard", 2024-09-11T23:39:54.256 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.256 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.256 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.256 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:54.256 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.256 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.256 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.256 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.256 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.256 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.256 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.256 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.257 INFO:teuthology.orchestra.run.smithi026.stdout: "leaderboard_description": { 2024-09-11T23:39:54.257 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "leaderboard_description", 2024-09-11T23:39:54.257 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.257 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.257 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.257 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:54.257 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.257 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.257 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.257 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.257 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.257 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.257 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.257 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.258 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-09-11T23:39:54.258 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-09-11T23:39:54.258 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.258 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.258 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.258 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:54.258 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.258 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.258 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:54.258 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:54.258 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:54.258 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:54.258 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:54.259 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:54.259 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:54.259 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:54.259 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.259 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.259 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.259 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.259 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.259 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-09-11T23:39:54.259 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-09-11T23:39:54.259 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.259 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.259 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.259 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:54.260 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.260 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.260 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.260 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.260 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.260 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.260 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.260 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.260 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-09-11T23:39:54.260 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-09-11T23:39:54.260 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.260 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.260 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.260 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-09-11T23:39:54.261 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.261 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.261 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:54.261 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:54.261 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:54.261 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:54.261 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:54.261 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:54.261 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:54.261 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:54.261 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.261 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.261 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.261 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.261 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.262 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-09-11T23:39:54.262 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-09-11T23:39:54.262 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.262 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.262 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.262 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:54.262 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.262 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.262 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.262 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.262 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.262 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.262 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.262 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.263 INFO:teuthology.orchestra.run.smithi026.stdout: "organization": { 2024-09-11T23:39:54.263 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "organization", 2024-09-11T23:39:54.263 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.263 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.263 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.263 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:54.263 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.263 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.263 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.263 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.263 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.263 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.263 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.263 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.263 INFO:teuthology.orchestra.run.smithi026.stdout: "proxy": { 2024-09-11T23:39:54.264 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "proxy", 2024-09-11T23:39:54.264 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.264 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.264 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.264 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:54.264 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.264 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.264 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.264 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.264 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.264 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.264 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.264 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.264 INFO:teuthology.orchestra.run.smithi026.stdout: "url": { 2024-09-11T23:39:54.264 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "url", 2024-09-11T23:39:54.265 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.265 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.265 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.265 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-11T23:39:54.265 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.265 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.265 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.265 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.265 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.265 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.265 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.265 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:54.265 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:54.266 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.266 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-11T23:39:54.266 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "test_orchestrator", 2024-09-11T23:39:54.266 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-09-11T23:39:54.266 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-09-11T23:39:54.266 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-09-11T23:39:54.266 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-09-11T23:39:54.266 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-09-11T23:39:54.266 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.266 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.266 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.266 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:54.266 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.266 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.266 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:54.267 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:54.267 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:54.267 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:54.267 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:54.267 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:54.267 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:54.267 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:54.267 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.267 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.267 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.267 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.267 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.267 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-09-11T23:39:54.267 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-09-11T23:39:54.268 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.268 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.268 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.268 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:54.268 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.268 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.268 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.268 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.268 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.268 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.268 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.268 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.268 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-09-11T23:39:54.268 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-09-11T23:39:54.269 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.269 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.269 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.269 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-09-11T23:39:54.269 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.269 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.269 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:54.269 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:54.269 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:54.269 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:54.269 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:54.269 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:54.269 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:54.269 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:54.270 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.270 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.270 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.270 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.270 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.270 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-09-11T23:39:54.270 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-09-11T23:39:54.270 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.270 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.270 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.270 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:54.270 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.270 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.270 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.271 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.271 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.271 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.271 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.271 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:54.271 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:54.271 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.271 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-11T23:39:54.271 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "volumes", 2024-09-11T23:39:54.271 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-09-11T23:39:54.271 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-09-11T23:39:54.271 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-09-11T23:39:54.271 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-09-11T23:39:54.271 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-09-11T23:39:54.272 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.272 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.272 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.272 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:54.272 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.272 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.272 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:54.272 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:54.272 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:54.272 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:54.272 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:54.272 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:54.272 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:54.272 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:54.273 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.273 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.273 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.273 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.273 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.273 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-09-11T23:39:54.273 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-09-11T23:39:54.273 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.273 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.273 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.273 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:54.273 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.273 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.273 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.273 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.274 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.274 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.274 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.274 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.274 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-09-11T23:39:54.274 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-09-11T23:39:54.274 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.274 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.274 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.274 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-09-11T23:39:54.274 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.274 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.274 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:54.274 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:54.275 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:54.275 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:54.275 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:54.275 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:54.275 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:54.275 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:54.275 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.275 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.275 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.275 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.275 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.275 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-09-11T23:39:54.275 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-09-11T23:39:54.275 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.275 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.276 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.276 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:54.276 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.276 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.276 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.276 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.276 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.276 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.276 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.276 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.276 INFO:teuthology.orchestra.run.smithi026.stdout: "max_concurrent_clones": { 2024-09-11T23:39:54.276 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "max_concurrent_clones", 2024-09-11T23:39:54.276 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-09-11T23:39:54.276 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.277 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.277 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "4", 2024-09-11T23:39:54.277 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.277 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.277 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.277 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-11T23:39:54.277 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.277 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.277 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.277 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.277 INFO:teuthology.orchestra.run.smithi026.stdout: "periodic_async_work": { 2024-09-11T23:39:54.277 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "periodic_async_work", 2024-09-11T23:39:54.277 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.277 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.277 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.278 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:54.278 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.278 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.278 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.278 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Periodically check for async work", 2024-09-11T23:39:54.278 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.278 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.278 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.278 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.278 INFO:teuthology.orchestra.run.smithi026.stdout: "snapshot_clone_delay": { 2024-09-11T23:39:54.278 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "snapshot_clone_delay", 2024-09-11T23:39:54.278 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-09-11T23:39:54.278 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.278 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.278 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2024-09-11T23:39:54.279 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.279 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.279 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.279 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-11T23:39:54.279 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.279 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.279 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.279 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.279 INFO:teuthology.orchestra.run.smithi026.stdout: "snapshot_clone_no_wait": { 2024-09-11T23:39:54.279 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "snapshot_clone_no_wait", 2024-09-11T23:39:54.279 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.279 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.279 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.280 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-09-11T23:39:54.280 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.280 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.280 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.280 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-11T23:39:54.280 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.280 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.280 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.280 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:54.280 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:54.280 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.280 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-11T23:39:54.280 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "zabbix", 2024-09-11T23:39:54.280 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-09-11T23:39:54.281 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-09-11T23:39:54.281 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-09-11T23:39:54.281 INFO:teuthology.orchestra.run.smithi026.stdout: "discovery_interval": { 2024-09-11T23:39:54.281 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "discovery_interval", 2024-09-11T23:39:54.281 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "uint", 2024-09-11T23:39:54.281 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.281 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.281 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "100", 2024-09-11T23:39:54.281 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.281 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.281 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.281 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.281 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.281 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.282 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.282 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.282 INFO:teuthology.orchestra.run.smithi026.stdout: "identifier": { 2024-09-11T23:39:54.282 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "identifier", 2024-09-11T23:39:54.282 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.282 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.282 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.282 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:54.282 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.282 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.282 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.282 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.282 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.282 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.282 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.283 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.283 INFO:teuthology.orchestra.run.smithi026.stdout: "interval": { 2024-09-11T23:39:54.283 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "interval", 2024-09-11T23:39:54.283 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-09-11T23:39:54.283 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.283 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.283 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "60", 2024-09-11T23:39:54.283 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.283 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.283 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.283 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.283 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.283 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.283 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.284 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.284 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-09-11T23:39:54.284 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-09-11T23:39:54.284 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.284 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.284 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.284 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:54.284 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.284 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.284 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:54.284 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:54.284 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:54.284 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:54.284 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:54.285 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:54.285 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:54.285 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:54.285 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.285 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.285 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.285 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.285 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.285 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-09-11T23:39:54.285 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-09-11T23:39:54.285 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.285 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.285 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.285 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:54.286 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.286 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.286 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.286 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.286 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.286 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.286 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.286 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.286 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-09-11T23:39:54.286 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-09-11T23:39:54.286 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.286 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.286 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.286 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-09-11T23:39:54.287 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.287 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.287 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:54.287 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:54.287 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:54.287 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:54.287 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:54.287 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:54.287 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:54.287 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:54.287 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.287 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.287 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.287 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.287 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.287 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-09-11T23:39:54.288 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-09-11T23:39:54.288 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.288 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.288 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.288 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:54.288 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.288 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.288 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.288 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.288 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.288 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.288 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.288 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.288 INFO:teuthology.orchestra.run.smithi026.stdout: "zabbix_host": { 2024-09-11T23:39:54.289 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "zabbix_host", 2024-09-11T23:39:54.289 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.289 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.289 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.289 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:54.289 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.289 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.289 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.289 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.289 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.289 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.289 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.289 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.289 INFO:teuthology.orchestra.run.smithi026.stdout: "zabbix_port": { 2024-09-11T23:39:54.290 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "zabbix_port", 2024-09-11T23:39:54.290 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-09-11T23:39:54.290 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.290 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.290 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "10051", 2024-09-11T23:39:54.290 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.290 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.290 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.290 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.290 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.290 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.290 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.290 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.290 INFO:teuthology.orchestra.run.smithi026.stdout: "zabbix_sender": { 2024-09-11T23:39:54.291 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "zabbix_sender", 2024-09-11T23:39:54.291 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.291 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.291 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.291 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-11T23:39:54.291 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.291 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.291 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.291 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.291 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.291 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.291 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.291 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:54.291 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:54.291 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:54.292 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-11T23:39:54.292 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:54.292 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:54.292 INFO:teuthology.orchestra.run.smithi026.stdout: "modules": [ 2024-09-11T23:39:54.292 INFO:teuthology.orchestra.run.smithi026.stdout: "cephadm", 2024-09-11T23:39:54.292 INFO:teuthology.orchestra.run.smithi026.stdout: "dashboard", 2024-09-11T23:39:54.292 INFO:teuthology.orchestra.run.smithi026.stdout: "iostat", 2024-09-11T23:39:54.292 INFO:teuthology.orchestra.run.smithi026.stdout: "nfs", 2024-09-11T23:39:54.292 INFO:teuthology.orchestra.run.smithi026.stdout: "restful", 2024-09-11T23:39:54.292 INFO:teuthology.orchestra.run.smithi026.stdout: "snap_schedule" 2024-09-11T23:39:54.292 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:54.292 INFO:teuthology.orchestra.run.smithi026.stdout: "available_modules": [ 2024-09-11T23:39:54.292 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-11T23:39:54.292 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "alerts", 2024-09-11T23:39:54.293 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-09-11T23:39:54.293 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-09-11T23:39:54.293 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-09-11T23:39:54.293 INFO:teuthology.orchestra.run.smithi026.stdout: "interval": { 2024-09-11T23:39:54.293 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "interval", 2024-09-11T23:39:54.293 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-09-11T23:39:54.293 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.293 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.293 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "60", 2024-09-11T23:39:54.293 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.293 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.293 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.293 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "How frequently to reexamine health status", 2024-09-11T23:39:54.294 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.294 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.294 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.294 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.294 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-09-11T23:39:54.294 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-09-11T23:39:54.294 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.294 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.294 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.294 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:54.294 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.294 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.294 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:54.294 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:54.294 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:54.295 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:54.295 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:54.295 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:54.295 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:54.295 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:54.295 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.295 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.295 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.295 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.295 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.295 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-09-11T23:39:54.295 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-09-11T23:39:54.295 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.295 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.296 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.296 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:54.296 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.296 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.296 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.296 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.296 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.296 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.296 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.296 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.296 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-09-11T23:39:54.296 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-09-11T23:39:54.296 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.296 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.296 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.297 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-09-11T23:39:54.297 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.297 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.297 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:54.297 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:54.297 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:54.297 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:54.297 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:54.297 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:54.297 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:54.297 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:54.297 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.297 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.297 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.298 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.298 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.298 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-09-11T23:39:54.298 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-09-11T23:39:54.298 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.298 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.298 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.298 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:54.298 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.298 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.298 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.298 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.298 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.298 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.299 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.299 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.299 INFO:teuthology.orchestra.run.smithi026.stdout: "smtp_destination": { 2024-09-11T23:39:54.299 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "smtp_destination", 2024-09-11T23:39:54.299 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.299 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.299 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.299 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:54.299 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.299 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.299 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.299 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Email address to send alerts to", 2024-09-11T23:39:54.299 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.299 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.300 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.300 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.300 INFO:teuthology.orchestra.run.smithi026.stdout: "smtp_from_name": { 2024-09-11T23:39:54.300 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "smtp_from_name", 2024-09-11T23:39:54.300 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.300 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.300 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.300 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "Ceph", 2024-09-11T23:39:54.300 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.300 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.300 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.300 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Email From: name", 2024-09-11T23:39:54.300 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.300 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.301 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.301 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.301 INFO:teuthology.orchestra.run.smithi026.stdout: "smtp_host": { 2024-09-11T23:39:54.301 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "smtp_host", 2024-09-11T23:39:54.301 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.301 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.301 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.301 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:54.301 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.301 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.301 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.301 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "SMTP server", 2024-09-11T23:39:54.301 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.301 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.302 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.302 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.302 INFO:teuthology.orchestra.run.smithi026.stdout: "smtp_password": { 2024-09-11T23:39:54.302 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "smtp_password", 2024-09-11T23:39:54.302 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.302 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.302 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.302 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:54.302 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.302 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.302 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.302 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Password to authenticate with", 2024-09-11T23:39:54.302 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.302 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.303 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.303 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.303 INFO:teuthology.orchestra.run.smithi026.stdout: "smtp_port": { 2024-09-11T23:39:54.303 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "smtp_port", 2024-09-11T23:39:54.303 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-09-11T23:39:54.303 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.303 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.303 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "465", 2024-09-11T23:39:54.303 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.303 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.303 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.303 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "SMTP port", 2024-09-11T23:39:54.303 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.303 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.303 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.304 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.304 INFO:teuthology.orchestra.run.smithi026.stdout: "smtp_sender": { 2024-09-11T23:39:54.304 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "smtp_sender", 2024-09-11T23:39:54.304 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.304 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.304 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.304 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:54.304 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.304 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.304 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.304 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "SMTP envelope sender", 2024-09-11T23:39:54.304 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.304 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.304 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.305 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.305 INFO:teuthology.orchestra.run.smithi026.stdout: "smtp_ssl": { 2024-09-11T23:39:54.305 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "smtp_ssl", 2024-09-11T23:39:54.305 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.305 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.305 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.305 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-09-11T23:39:54.305 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.305 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.305 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.305 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-11T23:39:54.305 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.305 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.305 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.306 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.306 INFO:teuthology.orchestra.run.smithi026.stdout: "smtp_user": { 2024-09-11T23:39:54.306 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "smtp_user", 2024-09-11T23:39:54.306 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.306 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.306 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.306 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:54.306 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.306 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.306 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.306 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "User to authenticate as", 2024-09-11T23:39:54.306 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.306 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.306 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.306 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:54.307 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:54.307 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.307 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-11T23:39:54.307 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "balancer", 2024-09-11T23:39:54.307 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-09-11T23:39:54.307 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-09-11T23:39:54.307 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-09-11T23:39:54.307 INFO:teuthology.orchestra.run.smithi026.stdout: "active": { 2024-09-11T23:39:54.307 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "active", 2024-09-11T23:39:54.307 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.307 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.307 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.307 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-09-11T23:39:54.307 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.307 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.308 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.308 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "automatically balance PGs across cluster", 2024-09-11T23:39:54.308 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.308 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.308 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.308 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.308 INFO:teuthology.orchestra.run.smithi026.stdout: "begin_time": { 2024-09-11T23:39:54.308 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "begin_time", 2024-09-11T23:39:54.308 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.308 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.308 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.308 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0000", 2024-09-11T23:39:54.308 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.309 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.309 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.309 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "beginning time of day to automatically balance", 2024-09-11T23:39:54.309 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-11T23:39:54.309 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.309 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.309 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.309 INFO:teuthology.orchestra.run.smithi026.stdout: "begin_weekday": { 2024-09-11T23:39:54.309 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "begin_weekday", 2024-09-11T23:39:54.309 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "uint", 2024-09-11T23:39:54.309 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.309 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.309 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2024-09-11T23:39:54.309 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "0", 2024-09-11T23:39:54.310 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "6", 2024-09-11T23:39:54.310 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.310 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-11T23:39:54.310 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-11T23:39:54.310 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.310 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.310 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.310 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_compat_max_iterations": { 2024-09-11T23:39:54.310 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "crush_compat_max_iterations", 2024-09-11T23:39:54.310 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "uint", 2024-09-11T23:39:54.310 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.310 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.310 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "25", 2024-09-11T23:39:54.310 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "1", 2024-09-11T23:39:54.311 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "250", 2024-09-11T23:39:54.311 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.311 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-11T23:39:54.311 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.311 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.311 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.311 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.311 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_compat_metrics": { 2024-09-11T23:39:54.311 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "crush_compat_metrics", 2024-09-11T23:39:54.311 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.311 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.311 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.311 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "pgs,objects,bytes", 2024-09-11T23:39:54.311 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.312 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.312 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.312 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-11T23:39:54.312 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-11T23:39:54.312 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.312 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.312 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.312 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_compat_step": { 2024-09-11T23:39:54.312 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "crush_compat_step", 2024-09-11T23:39:54.312 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "float", 2024-09-11T23:39:54.312 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.312 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.312 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0.5", 2024-09-11T23:39:54.312 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "0.001", 2024-09-11T23:39:54.313 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "0.999", 2024-09-11T23:39:54.313 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.313 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "aggressiveness of optimization", 2024-09-11T23:39:54.313 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-11T23:39:54.313 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.313 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.313 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.313 INFO:teuthology.orchestra.run.smithi026.stdout: "end_time": { 2024-09-11T23:39:54.313 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "end_time", 2024-09-11T23:39:54.313 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.313 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.313 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.313 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "2359", 2024-09-11T23:39:54.313 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.314 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.314 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.314 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "ending time of day to automatically balance", 2024-09-11T23:39:54.314 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-11T23:39:54.314 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.314 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.314 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.314 INFO:teuthology.orchestra.run.smithi026.stdout: "end_weekday": { 2024-09-11T23:39:54.314 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "end_weekday", 2024-09-11T23:39:54.314 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "uint", 2024-09-11T23:39:54.314 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.314 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.314 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2024-09-11T23:39:54.314 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "0", 2024-09-11T23:39:54.315 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "6", 2024-09-11T23:39:54.315 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.315 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-11T23:39:54.315 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-11T23:39:54.315 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.315 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.315 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.315 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-09-11T23:39:54.315 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-09-11T23:39:54.315 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.315 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.315 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.315 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:54.315 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.315 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.316 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:54.316 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:54.316 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:54.316 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:54.316 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:54.316 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:54.316 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:54.316 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:54.316 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.316 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.316 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.316 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.316 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.316 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-09-11T23:39:54.317 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-09-11T23:39:54.317 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.317 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.317 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.317 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:54.317 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.317 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.317 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.317 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.317 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.317 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.317 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.317 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.317 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-09-11T23:39:54.318 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-09-11T23:39:54.318 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.318 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.318 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.318 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-09-11T23:39:54.318 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.318 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.318 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:54.318 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:54.318 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:54.318 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:54.318 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:54.318 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:54.318 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:54.319 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:54.319 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.319 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.319 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.319 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.319 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.319 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-09-11T23:39:54.319 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-09-11T23:39:54.319 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.319 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.319 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.319 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:54.319 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.319 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.320 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.320 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.320 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.320 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.320 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.320 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.320 INFO:teuthology.orchestra.run.smithi026.stdout: "min_score": { 2024-09-11T23:39:54.320 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "min_score", 2024-09-11T23:39:54.320 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "float", 2024-09-11T23:39:54.320 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.320 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.320 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2024-09-11T23:39:54.320 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.320 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.321 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.321 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-11T23:39:54.321 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.321 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.321 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.321 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.321 INFO:teuthology.orchestra.run.smithi026.stdout: "mode": { 2024-09-11T23:39:54.321 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "mode", 2024-09-11T23:39:54.321 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.321 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.321 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.321 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "upmap", 2024-09-11T23:39:54.321 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.321 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.322 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:54.322 INFO:teuthology.orchestra.run.smithi026.stdout: "crush-compat", 2024-09-11T23:39:54.322 INFO:teuthology.orchestra.run.smithi026.stdout: "none", 2024-09-11T23:39:54.322 INFO:teuthology.orchestra.run.smithi026.stdout: "read", 2024-09-11T23:39:54.322 INFO:teuthology.orchestra.run.smithi026.stdout: "upmap", 2024-09-11T23:39:54.322 INFO:teuthology.orchestra.run.smithi026.stdout: "upmap-read" 2024-09-11T23:39:54.322 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:54.322 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Balancer mode", 2024-09-11T23:39:54.322 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.322 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.322 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.322 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.322 INFO:teuthology.orchestra.run.smithi026.stdout: "pool_ids": { 2024-09-11T23:39:54.322 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "pool_ids", 2024-09-11T23:39:54.323 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.323 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.323 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.323 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:54.323 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.323 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.323 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.323 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-11T23:39:54.323 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.323 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.323 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.323 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.323 INFO:teuthology.orchestra.run.smithi026.stdout: "sleep_interval": { 2024-09-11T23:39:54.323 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sleep_interval", 2024-09-11T23:39:54.323 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-09-11T23:39:54.324 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.324 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.324 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "60", 2024-09-11T23:39:54.324 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.324 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.324 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.324 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-11T23:39:54.324 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.324 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.324 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.324 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.324 INFO:teuthology.orchestra.run.smithi026.stdout: "upmap_max_deviation": { 2024-09-11T23:39:54.324 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "upmap_max_deviation", 2024-09-11T23:39:54.325 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-09-11T23:39:54.325 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.325 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.325 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "5", 2024-09-11T23:39:54.325 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "1", 2024-09-11T23:39:54.325 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.325 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.325 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-11T23:39:54.325 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-11T23:39:54.325 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.325 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.325 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.325 INFO:teuthology.orchestra.run.smithi026.stdout: "upmap_max_optimizations": { 2024-09-11T23:39:54.325 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "upmap_max_optimizations", 2024-09-11T23:39:54.326 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "uint", 2024-09-11T23:39:54.326 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.326 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.326 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "10", 2024-09-11T23:39:54.326 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.326 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.326 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.326 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-11T23:39:54.326 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.326 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.326 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.326 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:54.326 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:54.326 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.327 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-11T23:39:54.327 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "cephadm", 2024-09-11T23:39:54.327 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-09-11T23:39:54.327 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-09-11T23:39:54.327 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-09-11T23:39:54.327 INFO:teuthology.orchestra.run.smithi026.stdout: "agent_down_multiplier": { 2024-09-11T23:39:54.327 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "agent_down_multiplier", 2024-09-11T23:39:54.327 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "float", 2024-09-11T23:39:54.327 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.327 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.327 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "3.0", 2024-09-11T23:39:54.327 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.327 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.327 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.328 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-11T23:39:54.328 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.328 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.328 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.328 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.328 INFO:teuthology.orchestra.run.smithi026.stdout: "agent_refresh_rate": { 2024-09-11T23:39:54.328 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "agent_refresh_rate", 2024-09-11T23:39:54.328 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-09-11T23:39:54.328 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.328 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.328 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "20", 2024-09-11T23:39:54.328 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.328 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.328 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.329 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-11T23:39:54.329 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.329 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.329 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.329 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.329 INFO:teuthology.orchestra.run.smithi026.stdout: "agent_starting_port": { 2024-09-11T23:39:54.329 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "agent_starting_port", 2024-09-11T23:39:54.329 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-09-11T23:39:54.329 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.329 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.329 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "4721", 2024-09-11T23:39:54.329 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.329 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.329 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.329 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-11T23:39:54.330 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.330 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.330 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.330 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.330 INFO:teuthology.orchestra.run.smithi026.stdout: "allow_ptrace": { 2024-09-11T23:39:54.330 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "allow_ptrace", 2024-09-11T23:39:54.330 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.330 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.330 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.330 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:54.330 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.330 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.330 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.330 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-11T23:39:54.331 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-11T23:39:54.331 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.331 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.331 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.331 INFO:teuthology.orchestra.run.smithi026.stdout: "autotune_interval": { 2024-09-11T23:39:54.331 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "autotune_interval", 2024-09-11T23:39:54.331 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-09-11T23:39:54.331 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.331 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.331 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "600", 2024-09-11T23:39:54.331 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.331 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.331 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.331 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-11T23:39:54.332 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.332 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.332 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.332 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.332 INFO:teuthology.orchestra.run.smithi026.stdout: "autotune_memory_target_ratio": { 2024-09-11T23:39:54.332 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "autotune_memory_target_ratio", 2024-09-11T23:39:54.332 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "float", 2024-09-11T23:39:54.332 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.332 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.332 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0.7", 2024-09-11T23:39:54.332 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.332 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.332 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.332 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-11T23:39:54.332 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.333 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.333 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.333 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.333 INFO:teuthology.orchestra.run.smithi026.stdout: "cephadm_log_destination": { 2024-09-11T23:39:54.333 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "cephadm_log_destination", 2024-09-11T23:39:54.333 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.333 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.333 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.333 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:54.333 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.333 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.333 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:54.333 INFO:teuthology.orchestra.run.smithi026.stdout: "file", 2024-09-11T23:39:54.333 INFO:teuthology.orchestra.run.smithi026.stdout: "file,syslog", 2024-09-11T23:39:54.334 INFO:teuthology.orchestra.run.smithi026.stdout: "syslog" 2024-09-11T23:39:54.334 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:54.334 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-11T23:39:54.334 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.334 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.334 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.334 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.334 INFO:teuthology.orchestra.run.smithi026.stdout: "cgroups_split": { 2024-09-11T23:39:54.334 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "cgroups_split", 2024-09-11T23:39:54.334 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.334 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.334 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.334 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-09-11T23:39:54.334 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.334 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.335 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.335 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-11T23:39:54.335 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.335 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.335 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.335 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.335 INFO:teuthology.orchestra.run.smithi026.stdout: "config_checks_enabled": { 2024-09-11T23:39:54.335 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "config_checks_enabled", 2024-09-11T23:39:54.335 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.335 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.335 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.335 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:54.335 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.335 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.336 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.336 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-11T23:39:54.336 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.336 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.336 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.336 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.336 INFO:teuthology.orchestra.run.smithi026.stdout: "config_dashboard": { 2024-09-11T23:39:54.336 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "config_dashboard", 2024-09-11T23:39:54.336 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.336 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.336 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.336 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-09-11T23:39:54.336 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.336 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.337 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.337 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-11T23:39:54.337 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.337 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.337 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.337 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.337 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_alertmanager": { 2024-09-11T23:39:54.337 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_alertmanager", 2024-09-11T23:39:54.337 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.337 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.337 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.337 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-11T23:39:54.337 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.337 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.338 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.338 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Prometheus container image", 2024-09-11T23:39:54.338 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.338 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.338 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.338 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.338 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_base": { 2024-09-11T23:39:54.338 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_base", 2024-09-11T23:39:54.338 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.338 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.338 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.338 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-11T23:39:54.338 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.338 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.339 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.339 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Container image name, without the tag", 2024-09-11T23:39:54.339 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.339 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.339 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.339 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.339 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_elasticsearch": { 2024-09-11T23:39:54.339 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_elasticsearch", 2024-09-11T23:39:54.339 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.339 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.339 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.339 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-11T23:39:54.339 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.339 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.340 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.340 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "elasticsearch container image", 2024-09-11T23:39:54.340 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.340 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.340 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.340 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.340 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_grafana": { 2024-09-11T23:39:54.340 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_grafana", 2024-09-11T23:39:54.340 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.340 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.340 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.340 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-11T23:39:54.340 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.341 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.341 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.341 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Prometheus container image", 2024-09-11T23:39:54.341 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.341 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.341 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.341 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.341 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_haproxy": { 2024-09-11T23:39:54.341 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_haproxy", 2024-09-11T23:39:54.341 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.341 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.341 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.341 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-11T23:39:54.341 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.341 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.342 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.342 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "HAproxy container image", 2024-09-11T23:39:54.342 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.342 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.342 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.342 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.342 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_jaeger_agent": { 2024-09-11T23:39:54.342 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_jaeger_agent", 2024-09-11T23:39:54.342 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.342 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.342 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.342 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-11T23:39:54.342 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.342 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.343 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.343 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Jaeger agent container image", 2024-09-11T23:39:54.343 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.343 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.343 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.343 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.343 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_jaeger_collector": { 2024-09-11T23:39:54.343 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_jaeger_collector", 2024-09-11T23:39:54.343 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.343 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.343 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.343 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-11T23:39:54.343 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.343 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.343 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.344 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Jaeger collector container image", 2024-09-11T23:39:54.344 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.344 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.344 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.344 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.344 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_jaeger_query": { 2024-09-11T23:39:54.344 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_jaeger_query", 2024-09-11T23:39:54.344 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.344 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.344 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.344 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-11T23:39:54.344 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.344 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.345 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.345 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Jaeger query container image", 2024-09-11T23:39:54.345 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.345 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.345 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.345 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.345 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_keepalived": { 2024-09-11T23:39:54.345 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_keepalived", 2024-09-11T23:39:54.345 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.345 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.345 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.345 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-11T23:39:54.345 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.345 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.345 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.346 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Keepalived container image", 2024-09-11T23:39:54.346 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.346 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.346 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.346 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.346 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_loki": { 2024-09-11T23:39:54.346 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_loki", 2024-09-11T23:39:54.346 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.346 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.346 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.346 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-11T23:39:54.346 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.346 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.346 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.347 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Loki container image", 2024-09-11T23:39:54.347 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.347 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.347 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.347 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.347 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_node_exporter": { 2024-09-11T23:39:54.347 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_node_exporter", 2024-09-11T23:39:54.347 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.347 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.347 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.347 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-11T23:39:54.347 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.347 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.347 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.348 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Prometheus container image", 2024-09-11T23:39:54.348 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.348 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.348 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.348 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.348 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_nvmeof": { 2024-09-11T23:39:54.348 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_nvmeof", 2024-09-11T23:39:54.348 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.348 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.348 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.348 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-11T23:39:54.348 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.348 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.348 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.349 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Nvme-of container image", 2024-09-11T23:39:54.349 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.349 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.349 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.349 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.349 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_prometheus": { 2024-09-11T23:39:54.349 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_prometheus", 2024-09-11T23:39:54.349 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.349 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.349 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.349 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-11T23:39:54.349 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.349 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.349 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.349 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Prometheus container image", 2024-09-11T23:39:54.350 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.350 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.350 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.350 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.350 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_promtail": { 2024-09-11T23:39:54.350 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_promtail", 2024-09-11T23:39:54.350 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.350 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.350 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.350 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-11T23:39:54.350 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.350 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.351 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.351 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Promtail container image", 2024-09-11T23:39:54.351 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.351 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.351 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.351 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.351 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_samba": { 2024-09-11T23:39:54.351 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_samba", 2024-09-11T23:39:54.351 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.351 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.351 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.351 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-11T23:39:54.351 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.351 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.352 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.352 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Samba/SMB container image", 2024-09-11T23:39:54.352 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.352 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.352 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.352 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.352 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_snmp_gateway": { 2024-09-11T23:39:54.352 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_snmp_gateway", 2024-09-11T23:39:54.352 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.352 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.352 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.352 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-11T23:39:54.352 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.352 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.353 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.353 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "SNMP Gateway container image", 2024-09-11T23:39:54.353 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.353 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.353 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.353 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.353 INFO:teuthology.orchestra.run.smithi026.stdout: "container_init": { 2024-09-11T23:39:54.353 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_init", 2024-09-11T23:39:54.353 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.353 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.353 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.353 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-09-11T23:39:54.353 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.353 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.354 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.354 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Run podman/docker with `--init`", 2024-09-11T23:39:54.354 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.354 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.354 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.354 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.354 INFO:teuthology.orchestra.run.smithi026.stdout: "daemon_cache_timeout": { 2024-09-11T23:39:54.354 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "daemon_cache_timeout", 2024-09-11T23:39:54.354 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-09-11T23:39:54.354 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.354 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.354 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "600", 2024-09-11T23:39:54.354 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.354 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.355 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.355 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-11T23:39:54.355 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.355 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.355 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.355 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.355 INFO:teuthology.orchestra.run.smithi026.stdout: "default_cephadm_command_timeout": { 2024-09-11T23:39:54.355 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "default_cephadm_command_timeout", 2024-09-11T23:39:54.355 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-09-11T23:39:54.355 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.355 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.355 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "900", 2024-09-11T23:39:54.355 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.355 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.355 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.356 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-11T23:39:54.356 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.356 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.356 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.356 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.356 INFO:teuthology.orchestra.run.smithi026.stdout: "default_registry": { 2024-09-11T23:39:54.356 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "default_registry", 2024-09-11T23:39:54.356 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.356 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.356 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.356 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "docker.io", 2024-09-11T23:39:54.356 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.356 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.357 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.357 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-11T23:39:54.357 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.357 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.357 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.357 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.357 INFO:teuthology.orchestra.run.smithi026.stdout: "device_cache_timeout": { 2024-09-11T23:39:54.357 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "device_cache_timeout", 2024-09-11T23:39:54.357 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-09-11T23:39:54.357 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.357 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.357 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "1800", 2024-09-11T23:39:54.357 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.357 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.358 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.358 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "seconds to cache device inventory", 2024-09-11T23:39:54.358 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.358 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.358 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.358 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.358 INFO:teuthology.orchestra.run.smithi026.stdout: "device_enhanced_scan": { 2024-09-11T23:39:54.358 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "device_enhanced_scan", 2024-09-11T23:39:54.358 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.358 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.358 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.358 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:54.358 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.358 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.359 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.359 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-11T23:39:54.359 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.359 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.359 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.359 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.359 INFO:teuthology.orchestra.run.smithi026.stdout: "facts_cache_timeout": { 2024-09-11T23:39:54.359 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "facts_cache_timeout", 2024-09-11T23:39:54.359 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-09-11T23:39:54.359 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.359 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.359 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "60", 2024-09-11T23:39:54.359 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.359 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.360 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.360 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "seconds to cache host facts data", 2024-09-11T23:39:54.360 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.360 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.360 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.360 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.360 INFO:teuthology.orchestra.run.smithi026.stdout: "grafana_dashboards_path": { 2024-09-11T23:39:54.360 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "grafana_dashboards_path", 2024-09-11T23:39:54.360 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.360 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.360 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.360 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-11T23:39:54.360 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.360 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.361 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.361 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-11T23:39:54.361 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.361 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.361 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.361 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.361 INFO:teuthology.orchestra.run.smithi026.stdout: "host_check_interval": { 2024-09-11T23:39:54.361 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "host_check_interval", 2024-09-11T23:39:54.361 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-09-11T23:39:54.361 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.361 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.361 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "600", 2024-09-11T23:39:54.361 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.361 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.362 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.362 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "how frequently to perform a host check", 2024-09-11T23:39:54.362 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.362 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.362 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.362 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.362 INFO:teuthology.orchestra.run.smithi026.stdout: "hw_monitoring": { 2024-09-11T23:39:54.362 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "hw_monitoring", 2024-09-11T23:39:54.362 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.362 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.362 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.362 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:54.362 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.362 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.362 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.363 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-11T23:39:54.363 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.363 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.363 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.363 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.363 INFO:teuthology.orchestra.run.smithi026.stdout: "inventory_list_all": { 2024-09-11T23:39:54.363 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "inventory_list_all", 2024-09-11T23:39:54.363 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.363 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.363 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.363 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:54.363 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.363 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.363 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.364 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-11T23:39:54.364 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.364 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.364 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.364 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.364 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-09-11T23:39:54.364 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-09-11T23:39:54.364 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.364 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.364 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.364 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:54.364 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.364 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.364 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:54.365 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:54.365 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:54.365 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:54.365 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:54.365 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:54.365 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:54.365 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:54.365 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.365 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.365 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.365 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.365 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.365 INFO:teuthology.orchestra.run.smithi026.stdout: "log_refresh_metadata": { 2024-09-11T23:39:54.365 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_refresh_metadata", 2024-09-11T23:39:54.366 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.366 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.366 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.366 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:54.366 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.366 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.366 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.366 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-11T23:39:54.366 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.366 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.366 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.366 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.366 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-09-11T23:39:54.366 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-09-11T23:39:54.366 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.367 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.367 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.367 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-09-11T23:39:54.367 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.367 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.367 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.367 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-11T23:39:54.367 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.367 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.367 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.367 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.367 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-09-11T23:39:54.367 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-09-11T23:39:54.367 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.368 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.368 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.368 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-09-11T23:39:54.368 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.368 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.368 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:54.368 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:54.368 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:54.368 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:54.368 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:54.368 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:54.368 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:54.368 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:54.369 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.369 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.369 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.369 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.369 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.369 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-09-11T23:39:54.369 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-09-11T23:39:54.369 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.369 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.369 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.369 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:54.369 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.369 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.369 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.370 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.370 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.370 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.370 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.370 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.370 INFO:teuthology.orchestra.run.smithi026.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-11T23:39:54.370 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-11T23:39:54.370 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.370 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.370 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.370 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:54.370 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.370 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.370 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.371 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-11T23:39:54.371 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.371 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.371 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.371 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.371 INFO:teuthology.orchestra.run.smithi026.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-11T23:39:54.371 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-11T23:39:54.371 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.371 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.371 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.371 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "*", 2024-09-11T23:39:54.371 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.371 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.371 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.372 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-11T23:39:54.372 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.372 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.372 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.372 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.372 INFO:teuthology.orchestra.run.smithi026.stdout: "max_count_per_host": { 2024-09-11T23:39:54.372 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "max_count_per_host", 2024-09-11T23:39:54.372 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-09-11T23:39:54.372 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.372 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.372 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "10", 2024-09-11T23:39:54.372 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.372 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.372 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.373 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "max number of daemons per service per host", 2024-09-11T23:39:54.373 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.373 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.373 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.373 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.373 INFO:teuthology.orchestra.run.smithi026.stdout: "max_osd_draining_count": { 2024-09-11T23:39:54.373 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "max_osd_draining_count", 2024-09-11T23:39:54.373 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-09-11T23:39:54.373 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.373 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.373 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "10", 2024-09-11T23:39:54.373 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.373 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.373 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.374 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-11T23:39:54.374 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.374 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.374 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.374 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.374 INFO:teuthology.orchestra.run.smithi026.stdout: "migration_current": { 2024-09-11T23:39:54.374 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "migration_current", 2024-09-11T23:39:54.374 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-09-11T23:39:54.374 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.374 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.374 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:54.374 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.374 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.375 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.375 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "internal - do not modify", 2024-09-11T23:39:54.375 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.375 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.375 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.375 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.375 INFO:teuthology.orchestra.run.smithi026.stdout: "mode": { 2024-09-11T23:39:54.375 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "mode", 2024-09-11T23:39:54.375 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.375 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.375 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.375 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "root", 2024-09-11T23:39:54.375 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.375 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.375 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:54.376 INFO:teuthology.orchestra.run.smithi026.stdout: "cephadm-package", 2024-09-11T23:39:54.376 INFO:teuthology.orchestra.run.smithi026.stdout: "root" 2024-09-11T23:39:54.376 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:54.376 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "mode for remote execution of cephadm", 2024-09-11T23:39:54.376 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.376 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.376 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.376 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.376 INFO:teuthology.orchestra.run.smithi026.stdout: "oob_default_addr": { 2024-09-11T23:39:54.376 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "oob_default_addr", 2024-09-11T23:39:54.376 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.376 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.376 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.376 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "169.254.1.1", 2024-09-11T23:39:54.377 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.377 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.377 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.377 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-11T23:39:54.377 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.377 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.377 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.377 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.377 INFO:teuthology.orchestra.run.smithi026.stdout: "prometheus_alerts_path": { 2024-09-11T23:39:54.377 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "prometheus_alerts_path", 2024-09-11T23:39:54.377 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.377 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.377 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.377 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-11T23:39:54.377 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.378 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.378 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.378 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-11T23:39:54.378 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.378 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.378 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.378 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.378 INFO:teuthology.orchestra.run.smithi026.stdout: "registry_insecure": { 2024-09-11T23:39:54.378 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "registry_insecure", 2024-09-11T23:39:54.378 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.378 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.378 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.378 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:54.378 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.379 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.379 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.379 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-11T23:39:54.379 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.379 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.379 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.379 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.379 INFO:teuthology.orchestra.run.smithi026.stdout: "registry_password": { 2024-09-11T23:39:54.379 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "registry_password", 2024-09-11T23:39:54.379 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.379 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.379 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.379 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:54.379 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.380 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.380 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.380 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-11T23:39:54.380 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.380 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.380 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.380 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.380 INFO:teuthology.orchestra.run.smithi026.stdout: "registry_url": { 2024-09-11T23:39:54.380 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "registry_url", 2024-09-11T23:39:54.380 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.380 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.380 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.380 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:54.380 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.381 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.381 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.381 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-11T23:39:54.381 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.381 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.381 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.381 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.381 INFO:teuthology.orchestra.run.smithi026.stdout: "registry_username": { 2024-09-11T23:39:54.381 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "registry_username", 2024-09-11T23:39:54.381 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.381 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.381 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.381 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:54.381 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.381 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.382 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.382 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-11T23:39:54.382 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.382 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.382 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.382 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.382 INFO:teuthology.orchestra.run.smithi026.stdout: "secure_monitoring_stack": { 2024-09-11T23:39:54.382 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "secure_monitoring_stack", 2024-09-11T23:39:54.382 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.382 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.382 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.382 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:54.382 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.382 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.383 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.383 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-11T23:39:54.383 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.383 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.383 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.383 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.383 INFO:teuthology.orchestra.run.smithi026.stdout: "service_discovery_port": { 2024-09-11T23:39:54.383 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "service_discovery_port", 2024-09-11T23:39:54.383 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-09-11T23:39:54.383 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.383 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.383 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "8765", 2024-09-11T23:39:54.383 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.383 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.384 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.384 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "cephadm service discovery port", 2024-09-11T23:39:54.384 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.384 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.384 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.384 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.384 INFO:teuthology.orchestra.run.smithi026.stdout: "ssh_config_file": { 2024-09-11T23:39:54.384 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ssh_config_file", 2024-09-11T23:39:54.384 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.384 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.384 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.384 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:54.385 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.385 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.385 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.385 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-11T23:39:54.385 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.385 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.385 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.385 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.385 INFO:teuthology.orchestra.run.smithi026.stdout: "use_agent": { 2024-09-11T23:39:54.385 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "use_agent", 2024-09-11T23:39:54.385 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.385 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.385 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.385 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:54.386 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.386 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.386 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.386 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-11T23:39:54.386 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.386 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.386 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.386 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.386 INFO:teuthology.orchestra.run.smithi026.stdout: "use_repo_digest": { 2024-09-11T23:39:54.386 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "use_repo_digest", 2024-09-11T23:39:54.386 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.386 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.386 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.386 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-09-11T23:39:54.387 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.387 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.387 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.387 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-11T23:39:54.387 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.387 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.387 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.387 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.387 INFO:teuthology.orchestra.run.smithi026.stdout: "warn_on_failed_host_check": { 2024-09-11T23:39:54.387 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "warn_on_failed_host_check", 2024-09-11T23:39:54.387 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.387 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.387 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.388 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-09-11T23:39:54.388 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.388 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.388 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.388 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "raise a health warning if the host check fails", 2024-09-11T23:39:54.388 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.388 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.388 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.388 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.388 INFO:teuthology.orchestra.run.smithi026.stdout: "warn_on_stray_daemons": { 2024-09-11T23:39:54.388 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "warn_on_stray_daemons", 2024-09-11T23:39:54.388 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.388 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.388 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.389 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-09-11T23:39:54.389 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.389 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.389 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.389 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-11T23:39:54.389 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.389 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.389 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.389 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.389 INFO:teuthology.orchestra.run.smithi026.stdout: "warn_on_stray_hosts": { 2024-09-11T23:39:54.389 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "warn_on_stray_hosts", 2024-09-11T23:39:54.389 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.389 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.390 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.390 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-09-11T23:39:54.390 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.390 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.390 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.390 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-11T23:39:54.390 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.390 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.390 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.390 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:54.390 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:54.390 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.390 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-11T23:39:54.390 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "crash", 2024-09-11T23:39:54.391 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-09-11T23:39:54.391 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-09-11T23:39:54.391 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-09-11T23:39:54.391 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-09-11T23:39:54.391 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-09-11T23:39:54.391 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.391 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.391 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.391 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:54.391 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.391 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.391 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:54.391 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:54.391 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:54.392 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:54.392 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:54.392 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:54.392 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:54.392 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:54.392 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.392 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.392 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.392 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.392 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.392 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-09-11T23:39:54.392 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-09-11T23:39:54.392 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.392 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.393 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.393 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:54.393 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.393 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.393 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.393 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.393 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.393 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.393 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.393 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.393 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-09-11T23:39:54.393 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-09-11T23:39:54.393 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.393 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.394 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.394 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-09-11T23:39:54.394 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.394 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.394 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:54.394 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:54.394 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:54.394 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:54.394 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:54.394 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:54.394 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:54.394 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:54.394 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.394 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.395 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.395 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.395 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.395 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-09-11T23:39:54.395 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-09-11T23:39:54.395 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.395 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.395 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.395 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:54.395 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.395 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.395 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.395 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.395 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.396 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.396 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.396 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.396 INFO:teuthology.orchestra.run.smithi026.stdout: "retain_interval": { 2024-09-11T23:39:54.396 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "retain_interval", 2024-09-11T23:39:54.396 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-09-11T23:39:54.396 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.396 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.396 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "31536000", 2024-09-11T23:39:54.396 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.396 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.396 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.396 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-11T23:39:54.396 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.396 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.397 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.397 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.397 INFO:teuthology.orchestra.run.smithi026.stdout: "warn_recent_interval": { 2024-09-11T23:39:54.397 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "warn_recent_interval", 2024-09-11T23:39:54.397 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-09-11T23:39:54.397 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.397 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.397 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "1209600", 2024-09-11T23:39:54.397 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.397 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.397 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.397 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-11T23:39:54.397 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.397 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.398 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.398 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:54.398 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:54.398 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.398 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-11T23:39:54.398 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "dashboard", 2024-09-11T23:39:54.398 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-09-11T23:39:54.398 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-09-11T23:39:54.398 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-09-11T23:39:54.398 INFO:teuthology.orchestra.run.smithi026.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-11T23:39:54.398 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-11T23:39:54.398 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-09-11T23:39:54.398 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.399 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.399 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "10", 2024-09-11T23:39:54.399 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.399 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.399 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.399 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.399 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.399 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.399 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.399 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.399 INFO:teuthology.orchestra.run.smithi026.stdout: "ALERTMANAGER_API_HOST": { 2024-09-11T23:39:54.399 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-11T23:39:54.399 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.399 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.400 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.400 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:54.400 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.400 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.400 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.400 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.400 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.400 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.400 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.400 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.400 INFO:teuthology.orchestra.run.smithi026.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-11T23:39:54.400 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-11T23:39:54.400 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.401 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.401 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.401 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-09-11T23:39:54.401 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.401 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.401 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.401 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.401 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.401 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.401 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.401 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.401 INFO:teuthology.orchestra.run.smithi026.stdout: "AUDIT_API_ENABLED": { 2024-09-11T23:39:54.401 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "AUDIT_API_ENABLED", 2024-09-11T23:39:54.401 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.402 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.402 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.402 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:54.402 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.402 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.402 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.402 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.402 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.402 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.402 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.402 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.402 INFO:teuthology.orchestra.run.smithi026.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-11T23:39:54.402 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-11T23:39:54.402 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.403 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.403 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.403 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-09-11T23:39:54.403 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.403 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.403 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.403 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.403 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.403 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.403 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.403 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.403 INFO:teuthology.orchestra.run.smithi026.stdout: "ENABLE_BROWSABLE_API": { 2024-09-11T23:39:54.403 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-11T23:39:54.403 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.404 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.404 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.404 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-09-11T23:39:54.404 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.404 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.404 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.404 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.404 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.404 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.404 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.404 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.404 INFO:teuthology.orchestra.run.smithi026.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-11T23:39:54.404 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-11T23:39:54.404 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.405 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.405 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.405 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-09-11T23:39:54.405 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.405 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.405 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.405 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.405 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.405 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.405 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.405 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.405 INFO:teuthology.orchestra.run.smithi026.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-11T23:39:54.405 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-11T23:39:54.405 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.406 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.406 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.406 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-09-11T23:39:54.406 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.406 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.406 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.406 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.406 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.406 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.406 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.406 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.406 INFO:teuthology.orchestra.run.smithi026.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-11T23:39:54.406 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-11T23:39:54.406 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.407 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.407 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.407 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-09-11T23:39:54.407 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.407 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.407 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.407 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.407 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.407 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.407 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.407 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.407 INFO:teuthology.orchestra.run.smithi026.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-11T23:39:54.407 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-11T23:39:54.407 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.408 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.408 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.408 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-09-11T23:39:54.408 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.408 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.408 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.408 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.408 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.408 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.408 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.408 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.408 INFO:teuthology.orchestra.run.smithi026.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-11T23:39:54.408 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-11T23:39:54.409 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.409 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.409 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.409 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-09-11T23:39:54.409 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.409 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.409 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.409 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.409 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.409 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.409 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.409 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.409 INFO:teuthology.orchestra.run.smithi026.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-11T23:39:54.409 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-11T23:39:54.410 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.410 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.410 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.410 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-09-11T23:39:54.410 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.410 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.410 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.410 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.410 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.410 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.410 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.410 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.410 INFO:teuthology.orchestra.run.smithi026.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-11T23:39:54.410 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-11T23:39:54.411 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.411 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.411 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.411 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-09-11T23:39:54.411 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.411 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.411 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.411 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.411 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.411 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.411 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.411 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.411 INFO:teuthology.orchestra.run.smithi026.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-11T23:39:54.411 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-11T23:39:54.412 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.412 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.412 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.412 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:54.412 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.412 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.412 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.412 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.412 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.412 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.412 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.412 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.412 INFO:teuthology.orchestra.run.smithi026.stdout: "GRAFANA_API_PASSWORD": { 2024-09-11T23:39:54.412 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-11T23:39:54.413 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.413 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.413 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.413 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "admin", 2024-09-11T23:39:54.413 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.413 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.413 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.413 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.413 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.413 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.413 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.413 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.413 INFO:teuthology.orchestra.run.smithi026.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-11T23:39:54.414 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-11T23:39:54.414 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.414 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.414 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.414 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-09-11T23:39:54.414 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.414 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.414 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.414 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.414 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.414 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.414 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.414 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.414 INFO:teuthology.orchestra.run.smithi026.stdout: "GRAFANA_API_URL": { 2024-09-11T23:39:54.414 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "GRAFANA_API_URL", 2024-09-11T23:39:54.415 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.415 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.415 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.415 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:54.415 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.415 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.415 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.415 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.415 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.415 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.415 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.415 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.415 INFO:teuthology.orchestra.run.smithi026.stdout: "GRAFANA_API_USERNAME": { 2024-09-11T23:39:54.416 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-11T23:39:54.416 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.416 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.416 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.416 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "admin", 2024-09-11T23:39:54.416 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.416 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.416 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.416 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.416 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.416 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.416 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.416 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.416 INFO:teuthology.orchestra.run.smithi026.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-11T23:39:54.417 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-11T23:39:54.417 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.417 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.417 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.417 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:54.417 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.417 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.417 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.417 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.417 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.417 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.417 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.417 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.417 INFO:teuthology.orchestra.run.smithi026.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-11T23:39:54.418 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-11T23:39:54.418 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.418 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.418 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.418 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:54.418 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.418 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.418 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.418 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.418 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.418 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.418 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.418 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.419 INFO:teuthology.orchestra.run.smithi026.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-11T23:39:54.419 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-11T23:39:54.419 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.419 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.419 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.419 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-09-11T23:39:54.419 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.419 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.419 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.419 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.419 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.419 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.419 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.419 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.420 INFO:teuthology.orchestra.run.smithi026.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-11T23:39:54.420 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-11T23:39:54.420 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.420 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.420 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.420 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:54.420 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.420 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.420 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.420 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.420 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.420 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.420 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.421 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.421 INFO:teuthology.orchestra.run.smithi026.stdout: "PROMETHEUS_API_HOST": { 2024-09-11T23:39:54.421 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-11T23:39:54.421 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.421 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.421 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.421 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:54.421 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.421 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.421 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.421 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.421 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.421 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.421 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.422 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.422 INFO:teuthology.orchestra.run.smithi026.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-11T23:39:54.422 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-11T23:39:54.422 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.422 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.422 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.422 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-09-11T23:39:54.422 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.422 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.422 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.422 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.422 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.422 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.422 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.423 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.423 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-11T23:39:54.423 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-11T23:39:54.423 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.423 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.423 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.423 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:54.423 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.423 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.423 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.423 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.423 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.423 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.423 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.424 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.424 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-11T23:39:54.424 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-11T23:39:54.424 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.424 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.424 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.424 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:54.424 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.424 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.424 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.424 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.424 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.424 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.424 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.425 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.425 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-11T23:39:54.425 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-11T23:39:54.425 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.425 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.425 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.425 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-09-11T23:39:54.425 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.425 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.425 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.425 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.425 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.425 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.425 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.426 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.426 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-11T23:39:54.426 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-11T23:39:54.426 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.426 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.426 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.426 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-09-11T23:39:54.426 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.426 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.426 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.426 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.426 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.426 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.426 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.427 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.427 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-11T23:39:54.427 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-11T23:39:54.427 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.427 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.427 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.427 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:54.427 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.427 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.427 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.427 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.427 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.427 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.427 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.428 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.428 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-11T23:39:54.428 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-11T23:39:54.428 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.428 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.428 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.428 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:54.428 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.428 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.428 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.428 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.428 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.428 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.429 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.429 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.429 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-11T23:39:54.429 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-11T23:39:54.429 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.429 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.429 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.429 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:54.429 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.429 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.429 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.429 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.429 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.429 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.429 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.430 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.430 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_ENABLED": { 2024-09-11T23:39:54.430 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-11T23:39:54.430 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.430 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.430 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.430 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-09-11T23:39:54.430 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.430 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.430 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.430 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.430 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.430 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.431 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.431 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.431 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-11T23:39:54.431 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-11T23:39:54.431 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.431 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.431 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.431 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-11T23:39:54.431 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.431 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.431 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.431 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.431 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.432 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.432 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.432 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.432 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-11T23:39:54.432 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-11T23:39:54.432 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-09-11T23:39:54.432 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.432 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.432 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "10", 2024-09-11T23:39:54.432 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.432 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.432 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.432 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.433 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.433 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.433 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.433 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.433 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-11T23:39:54.433 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-11T23:39:54.433 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-09-11T23:39:54.433 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.433 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.433 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "8", 2024-09-11T23:39:54.433 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.433 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.433 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.433 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.434 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.434 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.434 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.434 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.434 INFO:teuthology.orchestra.run.smithi026.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-11T23:39:54.434 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-11T23:39:54.434 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-09-11T23:39:54.434 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.434 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.434 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "45", 2024-09-11T23:39:54.434 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.434 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.434 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.435 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.435 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.435 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.435 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.435 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.435 INFO:teuthology.orchestra.run.smithi026.stdout: "RGW_API_ACCESS_KEY": { 2024-09-11T23:39:54.435 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-11T23:39:54.435 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.435 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.435 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.435 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:54.435 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.435 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.435 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.436 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.436 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.436 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.436 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.436 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.436 INFO:teuthology.orchestra.run.smithi026.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-11T23:39:54.436 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-11T23:39:54.436 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.436 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.436 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.436 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "admin", 2024-09-11T23:39:54.436 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.436 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.436 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.437 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.437 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.437 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.437 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.437 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.437 INFO:teuthology.orchestra.run.smithi026.stdout: "RGW_API_SECRET_KEY": { 2024-09-11T23:39:54.437 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-11T23:39:54.437 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.437 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.437 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.437 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:54.437 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.437 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.438 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.438 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.438 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.438 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.438 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.438 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.438 INFO:teuthology.orchestra.run.smithi026.stdout: "RGW_API_SSL_VERIFY": { 2024-09-11T23:39:54.438 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-11T23:39:54.438 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.438 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.438 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.438 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-09-11T23:39:54.438 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.438 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.439 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.439 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.439 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.439 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.439 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.439 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.439 INFO:teuthology.orchestra.run.smithi026.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-11T23:39:54.439 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-11T23:39:54.439 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-09-11T23:39:54.439 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.439 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.439 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2024-09-11T23:39:54.439 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.439 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.440 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.440 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.440 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.440 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.440 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.440 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.440 INFO:teuthology.orchestra.run.smithi026.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-11T23:39:54.440 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-11T23:39:54.440 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-09-11T23:39:54.440 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.440 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.440 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "10", 2024-09-11T23:39:54.440 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.441 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.441 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.441 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.441 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.441 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.441 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.441 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.441 INFO:teuthology.orchestra.run.smithi026.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-11T23:39:54.441 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-11T23:39:54.441 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-09-11T23:39:54.441 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.441 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.441 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "5", 2024-09-11T23:39:54.441 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.441 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.442 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.442 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.442 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.442 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.442 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.442 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.442 INFO:teuthology.orchestra.run.smithi026.stdout: "cross_origin_url": { 2024-09-11T23:39:54.442 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "cross_origin_url", 2024-09-11T23:39:54.442 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.442 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.442 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.442 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:54.442 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.442 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.443 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.443 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.443 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.443 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.443 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.443 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.443 INFO:teuthology.orchestra.run.smithi026.stdout: "crt_file": { 2024-09-11T23:39:54.443 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "crt_file", 2024-09-11T23:39:54.443 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.443 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.443 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.443 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:54.443 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.443 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.443 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.444 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.444 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.444 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.444 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.444 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.444 INFO:teuthology.orchestra.run.smithi026.stdout: "debug": { 2024-09-11T23:39:54.444 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "debug", 2024-09-11T23:39:54.444 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.444 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.444 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.444 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:54.444 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.444 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.444 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.445 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Enable/disable debug options", 2024-09-11T23:39:54.445 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.445 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.445 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.445 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.445 INFO:teuthology.orchestra.run.smithi026.stdout: "jwt_token_ttl": { 2024-09-11T23:39:54.445 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "jwt_token_ttl", 2024-09-11T23:39:54.445 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-09-11T23:39:54.445 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.445 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.445 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "28800", 2024-09-11T23:39:54.445 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.445 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.445 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.446 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.446 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.446 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.446 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.446 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.446 INFO:teuthology.orchestra.run.smithi026.stdout: "key_file": { 2024-09-11T23:39:54.446 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "key_file", 2024-09-11T23:39:54.446 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.446 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.446 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.446 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:54.446 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.446 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.446 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.447 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.447 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.447 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.447 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.447 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.447 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-09-11T23:39:54.447 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-09-11T23:39:54.447 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.447 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.447 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.447 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:54.447 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.447 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.448 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:54.448 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:54.448 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:54.448 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:54.448 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:54.448 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:54.448 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:54.448 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:54.448 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.448 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.448 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.448 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.448 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.448 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-09-11T23:39:54.449 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-09-11T23:39:54.449 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.449 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.449 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.449 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:54.449 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.449 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.449 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.449 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.449 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.449 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.449 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.449 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.449 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-09-11T23:39:54.450 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-09-11T23:39:54.450 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.450 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.450 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.450 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-09-11T23:39:54.450 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.450 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.450 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:54.450 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:54.450 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:54.450 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:54.450 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:54.450 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:54.450 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:54.451 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:54.451 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.451 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.451 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.451 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.451 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.451 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-09-11T23:39:54.451 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-09-11T23:39:54.451 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.451 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.451 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.451 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:54.451 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.451 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.452 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.452 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.452 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.452 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.452 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.452 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.452 INFO:teuthology.orchestra.run.smithi026.stdout: "motd": { 2024-09-11T23:39:54.452 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "motd", 2024-09-11T23:39:54.452 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.452 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.452 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.452 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:54.452 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.453 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.453 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.453 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "The message of the day", 2024-09-11T23:39:54.453 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.453 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.453 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.453 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.453 INFO:teuthology.orchestra.run.smithi026.stdout: "redirect_resolve_ip_addr": { 2024-09-11T23:39:54.453 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "redirect_resolve_ip_addr", 2024-09-11T23:39:54.453 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.453 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.453 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.453 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:54.454 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.454 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.454 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.454 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.454 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.454 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.454 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.454 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.454 INFO:teuthology.orchestra.run.smithi026.stdout: "server_addr": { 2024-09-11T23:39:54.454 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "server_addr", 2024-09-11T23:39:54.454 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.454 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.454 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.454 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "::", 2024-09-11T23:39:54.455 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.455 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.455 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.455 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.455 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.455 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.455 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.455 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.455 INFO:teuthology.orchestra.run.smithi026.stdout: "server_port": { 2024-09-11T23:39:54.455 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "server_port", 2024-09-11T23:39:54.455 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-09-11T23:39:54.455 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.455 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.455 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "8080", 2024-09-11T23:39:54.456 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.456 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.456 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.456 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.456 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.456 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.456 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.456 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.456 INFO:teuthology.orchestra.run.smithi026.stdout: "ssl": { 2024-09-11T23:39:54.456 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ssl", 2024-09-11T23:39:54.456 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.456 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.456 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.456 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-09-11T23:39:54.457 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.457 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.457 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.457 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.457 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.457 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.457 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.457 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.457 INFO:teuthology.orchestra.run.smithi026.stdout: "ssl_server_port": { 2024-09-11T23:39:54.457 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ssl_server_port", 2024-09-11T23:39:54.457 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-09-11T23:39:54.457 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.457 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.457 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "8443", 2024-09-11T23:39:54.458 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.458 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.458 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.458 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.458 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.458 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.458 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.458 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.458 INFO:teuthology.orchestra.run.smithi026.stdout: "standby_behaviour": { 2024-09-11T23:39:54.458 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "standby_behaviour", 2024-09-11T23:39:54.458 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.458 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.458 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.458 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "redirect", 2024-09-11T23:39:54.458 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.459 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.459 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:54.459 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:54.459 INFO:teuthology.orchestra.run.smithi026.stdout: "redirect" 2024-09-11T23:39:54.459 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:54.459 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.459 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.459 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.459 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.459 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.459 INFO:teuthology.orchestra.run.smithi026.stdout: "standby_error_status_code": { 2024-09-11T23:39:54.459 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "standby_error_status_code", 2024-09-11T23:39:54.459 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-09-11T23:39:54.459 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.460 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.460 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "500", 2024-09-11T23:39:54.460 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "400", 2024-09-11T23:39:54.460 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "599", 2024-09-11T23:39:54.460 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.460 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.460 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.460 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.460 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.460 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.460 INFO:teuthology.orchestra.run.smithi026.stdout: "url_prefix": { 2024-09-11T23:39:54.460 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "url_prefix", 2024-09-11T23:39:54.460 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.460 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.461 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.461 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:54.461 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.461 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.461 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.461 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.461 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.461 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.461 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.461 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:54.461 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:54.461 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.461 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-11T23:39:54.461 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "devicehealth", 2024-09-11T23:39:54.462 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-09-11T23:39:54.462 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-09-11T23:39:54.462 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-09-11T23:39:54.462 INFO:teuthology.orchestra.run.smithi026.stdout: "enable_monitoring": { 2024-09-11T23:39:54.462 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "enable_monitoring", 2024-09-11T23:39:54.462 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.462 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.462 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.462 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-09-11T23:39:54.462 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.462 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.462 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.462 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "monitor device health metrics", 2024-09-11T23:39:54.462 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.462 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.463 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.463 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.463 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-09-11T23:39:54.463 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-09-11T23:39:54.463 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.463 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.463 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.463 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:54.463 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.463 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.463 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:54.463 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:54.463 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:54.463 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:54.463 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:54.464 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:54.464 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:54.464 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:54.464 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.464 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.464 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.464 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.464 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.464 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-09-11T23:39:54.464 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-09-11T23:39:54.464 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.464 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.464 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.465 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:54.465 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.465 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.465 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.465 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.465 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.465 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.465 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.465 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.465 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-09-11T23:39:54.465 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-09-11T23:39:54.465 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.465 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.465 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.465 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-09-11T23:39:54.466 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.466 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.466 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:54.466 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:54.466 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:54.466 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:54.466 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:54.466 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:54.466 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:54.466 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:54.466 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.466 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.466 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.466 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.467 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.467 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-09-11T23:39:54.467 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-09-11T23:39:54.467 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.467 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.467 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.467 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:54.467 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.467 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.467 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.467 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.467 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.467 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.467 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.468 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.468 INFO:teuthology.orchestra.run.smithi026.stdout: "mark_out_threshold": { 2024-09-11T23:39:54.468 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "mark_out_threshold", 2024-09-11T23:39:54.468 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-09-11T23:39:54.468 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.468 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.468 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "2419200", 2024-09-11T23:39:54.468 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.468 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.468 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.468 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-11T23:39:54.468 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.468 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.468 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.469 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.469 INFO:teuthology.orchestra.run.smithi026.stdout: "pool_name": { 2024-09-11T23:39:54.469 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "pool_name", 2024-09-11T23:39:54.469 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.469 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.469 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.469 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "device_health_metrics", 2024-09-11T23:39:54.469 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.469 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.469 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.469 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-11T23:39:54.469 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.469 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.469 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.469 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.470 INFO:teuthology.orchestra.run.smithi026.stdout: "retention_period": { 2024-09-11T23:39:54.470 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "retention_period", 2024-09-11T23:39:54.470 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-09-11T23:39:54.470 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.470 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.470 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "15552000", 2024-09-11T23:39:54.470 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.470 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.470 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.470 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "how long to retain device health metrics", 2024-09-11T23:39:54.470 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.470 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.470 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.470 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.471 INFO:teuthology.orchestra.run.smithi026.stdout: "scrape_frequency": { 2024-09-11T23:39:54.471 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "scrape_frequency", 2024-09-11T23:39:54.471 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-09-11T23:39:54.471 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.471 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.471 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "86400", 2024-09-11T23:39:54.471 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.471 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.471 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.471 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-11T23:39:54.471 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.471 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.471 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.471 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.472 INFO:teuthology.orchestra.run.smithi026.stdout: "self_heal": { 2024-09-11T23:39:54.472 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "self_heal", 2024-09-11T23:39:54.472 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.472 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.472 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.472 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-09-11T23:39:54.472 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.472 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.472 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.472 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-11T23:39:54.472 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.472 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.472 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.472 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.473 INFO:teuthology.orchestra.run.smithi026.stdout: "sleep_interval": { 2024-09-11T23:39:54.473 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sleep_interval", 2024-09-11T23:39:54.473 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-09-11T23:39:54.473 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.473 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.473 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "600", 2024-09-11T23:39:54.473 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.473 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.473 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.473 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "how frequently to wake up and check device health", 2024-09-11T23:39:54.473 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.473 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.473 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.473 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.474 INFO:teuthology.orchestra.run.smithi026.stdout: "warn_threshold": { 2024-09-11T23:39:54.474 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "warn_threshold", 2024-09-11T23:39:54.474 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-09-11T23:39:54.474 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.474 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.474 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "7257600", 2024-09-11T23:39:54.474 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.474 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.474 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.474 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-11T23:39:54.474 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.474 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.474 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.474 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:54.475 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:54.475 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.475 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-11T23:39:54.475 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "diskprediction_local", 2024-09-11T23:39:54.475 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-09-11T23:39:54.475 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-09-11T23:39:54.475 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-09-11T23:39:54.475 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-09-11T23:39:54.475 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-09-11T23:39:54.475 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.475 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.475 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.475 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:54.475 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.476 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.476 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:54.476 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:54.476 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:54.476 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:54.476 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:54.476 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:54.476 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:54.476 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:54.476 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.476 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.476 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.476 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.476 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.477 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-09-11T23:39:54.477 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-09-11T23:39:54.477 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.477 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.477 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.477 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:54.477 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.477 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.477 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.477 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.477 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.477 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.477 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.477 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.477 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-09-11T23:39:54.478 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-09-11T23:39:54.478 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.478 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.478 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.478 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-09-11T23:39:54.478 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.478 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.478 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:54.478 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:54.478 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:54.478 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:54.478 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:54.478 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:54.478 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:54.479 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:54.479 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.479 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.479 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.479 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.479 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.479 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-09-11T23:39:54.479 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-09-11T23:39:54.479 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.479 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.479 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.479 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:54.479 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.479 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.480 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.480 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.480 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.480 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.480 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.480 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.480 INFO:teuthology.orchestra.run.smithi026.stdout: "predict_interval": { 2024-09-11T23:39:54.480 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "predict_interval", 2024-09-11T23:39:54.480 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.480 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.480 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.481 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "86400", 2024-09-11T23:39:54.481 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.481 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.481 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.481 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.481 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.481 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.481 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.481 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.481 INFO:teuthology.orchestra.run.smithi026.stdout: "predictor_model": { 2024-09-11T23:39:54.481 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "predictor_model", 2024-09-11T23:39:54.481 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.481 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.481 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.482 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "prophetstor", 2024-09-11T23:39:54.482 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.482 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.482 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.482 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.482 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.482 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.482 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.482 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.482 INFO:teuthology.orchestra.run.smithi026.stdout: "sleep_interval": { 2024-09-11T23:39:54.482 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sleep_interval", 2024-09-11T23:39:54.482 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.482 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.482 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.483 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "600", 2024-09-11T23:39:54.483 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.483 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.483 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.483 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.483 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.483 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.483 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.483 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:54.483 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:54.483 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.483 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-11T23:39:54.483 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "influx", 2024-09-11T23:39:54.483 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": false, 2024-09-11T23:39:54.483 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "influxdb python module not found", 2024-09-11T23:39:54.484 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-09-11T23:39:54.484 INFO:teuthology.orchestra.run.smithi026.stdout: "batch_size": { 2024-09-11T23:39:54.484 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "batch_size", 2024-09-11T23:39:54.484 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-09-11T23:39:54.484 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.484 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.484 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "5000", 2024-09-11T23:39:54.484 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.484 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.484 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.484 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-11T23:39:54.484 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.484 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.485 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.485 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.485 INFO:teuthology.orchestra.run.smithi026.stdout: "database": { 2024-09-11T23:39:54.485 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "database", 2024-09-11T23:39:54.485 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.485 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.485 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.485 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "ceph", 2024-09-11T23:39:54.485 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.485 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.485 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.485 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-11T23:39:54.485 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.486 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.486 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.486 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.486 INFO:teuthology.orchestra.run.smithi026.stdout: "hostname": { 2024-09-11T23:39:54.486 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "hostname", 2024-09-11T23:39:54.486 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.486 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.486 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.486 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:54.486 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.486 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.486 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.486 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "InfluxDB server hostname", 2024-09-11T23:39:54.486 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.487 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.487 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.487 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.487 INFO:teuthology.orchestra.run.smithi026.stdout: "interval": { 2024-09-11T23:39:54.487 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "interval", 2024-09-11T23:39:54.487 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-09-11T23:39:54.487 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.487 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.487 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "30", 2024-09-11T23:39:54.487 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "5", 2024-09-11T23:39:54.487 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.487 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.487 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-11T23:39:54.487 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.488 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.488 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.488 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.488 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-09-11T23:39:54.488 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-09-11T23:39:54.488 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.488 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.488 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.488 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:54.488 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.488 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.488 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:54.488 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:54.489 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:54.489 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:54.489 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:54.489 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:54.489 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:54.489 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:54.489 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.489 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.489 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.489 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.489 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.489 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-09-11T23:39:54.489 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-09-11T23:39:54.489 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.489 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.490 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.490 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:54.490 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.490 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.490 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.490 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.490 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.490 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.490 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.490 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.490 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-09-11T23:39:54.490 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-09-11T23:39:54.490 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.490 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.491 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.491 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-09-11T23:39:54.491 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.491 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.491 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:54.491 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:54.491 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:54.491 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:54.491 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:54.491 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:54.491 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:54.491 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:54.491 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.491 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.491 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.492 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.492 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.492 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-09-11T23:39:54.492 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-09-11T23:39:54.492 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.492 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.492 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.492 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:54.492 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.492 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.492 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.492 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.492 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.493 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.493 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.493 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.493 INFO:teuthology.orchestra.run.smithi026.stdout: "password": { 2024-09-11T23:39:54.493 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "password", 2024-09-11T23:39:54.493 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.493 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.493 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.493 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:54.493 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.493 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.493 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.493 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "password of InfluxDB server user", 2024-09-11T23:39:54.493 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.494 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.494 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.494 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.494 INFO:teuthology.orchestra.run.smithi026.stdout: "port": { 2024-09-11T23:39:54.494 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "port", 2024-09-11T23:39:54.494 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-09-11T23:39:54.494 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.494 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.494 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "8086", 2024-09-11T23:39:54.494 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.494 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.494 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.494 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "InfluxDB server port", 2024-09-11T23:39:54.494 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.494 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.495 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.495 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.495 INFO:teuthology.orchestra.run.smithi026.stdout: "ssl": { 2024-09-11T23:39:54.495 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ssl", 2024-09-11T23:39:54.495 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.495 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.495 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.495 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "false", 2024-09-11T23:39:54.495 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.495 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.495 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.495 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-11T23:39:54.495 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.495 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.496 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.496 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.496 INFO:teuthology.orchestra.run.smithi026.stdout: "threads": { 2024-09-11T23:39:54.496 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "threads", 2024-09-11T23:39:54.496 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-09-11T23:39:54.496 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.496 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.496 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "5", 2024-09-11T23:39:54.496 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "1", 2024-09-11T23:39:54.496 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "32", 2024-09-11T23:39:54.496 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.496 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-11T23:39:54.496 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.496 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.497 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.497 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.497 INFO:teuthology.orchestra.run.smithi026.stdout: "username": { 2024-09-11T23:39:54.497 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "username", 2024-09-11T23:39:54.497 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.497 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.497 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.497 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:54.497 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.497 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.497 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.497 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "username of InfluxDB server user", 2024-09-11T23:39:54.497 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.497 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.498 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.498 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.498 INFO:teuthology.orchestra.run.smithi026.stdout: "verify_ssl": { 2024-09-11T23:39:54.498 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "verify_ssl", 2024-09-11T23:39:54.498 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.498 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.498 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.498 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "true", 2024-09-11T23:39:54.498 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.498 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.498 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.498 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-11T23:39:54.498 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.498 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.499 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.499 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:54.499 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:54.499 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.499 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-11T23:39:54.499 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "insights", 2024-09-11T23:39:54.499 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-09-11T23:39:54.499 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-09-11T23:39:54.499 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-09-11T23:39:54.499 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-09-11T23:39:54.499 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-09-11T23:39:54.499 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.499 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.499 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.500 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:54.500 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.500 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.500 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:54.500 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:54.500 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:54.500 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:54.500 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:54.500 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:54.500 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:54.500 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:54.500 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.500 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.500 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.501 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.501 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.501 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-09-11T23:39:54.501 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-09-11T23:39:54.501 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.501 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.501 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.501 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:54.501 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.501 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.501 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.501 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.501 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.501 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.502 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.502 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.502 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-09-11T23:39:54.502 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-09-11T23:39:54.502 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.502 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.502 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.502 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-09-11T23:39:54.502 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.502 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.502 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:54.502 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:54.502 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:54.502 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:54.502 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:54.503 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:54.503 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:54.503 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:54.503 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.503 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.503 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.503 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.503 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.503 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-09-11T23:39:54.503 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-09-11T23:39:54.503 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.503 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.503 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.503 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:54.504 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.504 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.504 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.504 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.504 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.504 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.504 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.504 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:54.504 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:54.504 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.504 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-11T23:39:54.504 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "iostat", 2024-09-11T23:39:54.504 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-09-11T23:39:54.504 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-09-11T23:39:54.505 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-09-11T23:39:54.505 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-09-11T23:39:54.505 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-09-11T23:39:54.505 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.505 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.505 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.505 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:54.505 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.505 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.505 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:54.505 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:54.505 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:54.505 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:54.505 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:54.506 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:54.506 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:54.506 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:54.506 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.506 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.506 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.506 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.506 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.506 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-09-11T23:39:54.506 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-09-11T23:39:54.506 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.506 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.506 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.506 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:54.507 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.507 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.507 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.507 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.507 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.507 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.507 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.507 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.507 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-09-11T23:39:54.507 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-09-11T23:39:54.507 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.507 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.507 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.507 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-09-11T23:39:54.507 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.508 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.508 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:54.508 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:54.508 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:54.508 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:54.508 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:54.508 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:54.508 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:54.508 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:54.508 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.508 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.508 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.508 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.509 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.509 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-09-11T23:39:54.509 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-09-11T23:39:54.509 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.509 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.509 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.509 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:54.509 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.509 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.509 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.509 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.509 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.509 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.509 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.509 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:54.510 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:54.510 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.510 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-11T23:39:54.510 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "k8sevents", 2024-09-11T23:39:54.510 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-09-11T23:39:54.510 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-09-11T23:39:54.510 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-09-11T23:39:54.510 INFO:teuthology.orchestra.run.smithi026.stdout: "ceph_event_retention_days": { 2024-09-11T23:39:54.510 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ceph_event_retention_days", 2024-09-11T23:39:54.510 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-09-11T23:39:54.510 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.510 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.510 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "7", 2024-09-11T23:39:54.510 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.511 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.511 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.511 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-11T23:39:54.511 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.511 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.511 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.511 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.511 INFO:teuthology.orchestra.run.smithi026.stdout: "config_check_secs": { 2024-09-11T23:39:54.511 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "config_check_secs", 2024-09-11T23:39:54.511 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-09-11T23:39:54.511 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.511 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.511 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "10", 2024-09-11T23:39:54.511 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "10", 2024-09-11T23:39:54.512 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.512 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.512 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-11T23:39:54.512 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.512 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.512 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.512 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.512 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-09-11T23:39:54.512 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-09-11T23:39:54.512 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.512 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.512 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.512 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:54.512 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.513 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.513 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:54.513 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:54.513 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:54.513 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:54.513 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:54.513 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:54.513 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:54.513 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:54.513 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.513 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.513 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.513 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.513 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.514 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-09-11T23:39:54.514 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-09-11T23:39:54.514 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.514 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.514 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.514 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:54.514 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.514 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.514 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.514 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.514 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.514 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.514 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.514 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.515 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-09-11T23:39:54.515 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-09-11T23:39:54.515 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.515 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.515 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.515 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-09-11T23:39:54.515 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.515 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.515 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:54.515 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:54.515 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:54.515 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:54.515 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:54.515 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:54.516 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:54.516 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:54.516 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.516 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.516 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.516 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.516 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.516 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-09-11T23:39:54.516 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-09-11T23:39:54.516 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.516 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.516 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.516 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:54.517 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.517 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.517 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.517 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.517 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.517 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.517 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.517 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:54.517 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:54.517 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.517 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-11T23:39:54.517 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "localpool", 2024-09-11T23:39:54.517 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-09-11T23:39:54.517 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-09-11T23:39:54.518 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-09-11T23:39:54.518 INFO:teuthology.orchestra.run.smithi026.stdout: "failure_domain": { 2024-09-11T23:39:54.518 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "failure_domain", 2024-09-11T23:39:54.518 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.518 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.518 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.518 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "host", 2024-09-11T23:39:54.518 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.518 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.518 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.518 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "failure domain for any created local pool", 2024-09-11T23:39:54.518 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-11T23:39:54.518 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.519 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.519 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.519 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-09-11T23:39:54.519 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-09-11T23:39:54.519 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.519 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.519 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.519 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:54.519 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.519 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.519 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:54.519 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:54.519 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:54.520 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:54.520 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:54.520 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:54.520 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:54.520 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:54.520 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.520 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.520 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.520 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.520 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.520 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-09-11T23:39:54.520 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-09-11T23:39:54.520 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.521 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.521 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.521 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:54.521 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.521 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.521 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.521 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.521 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.521 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.521 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.521 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.521 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-09-11T23:39:54.521 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-09-11T23:39:54.521 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.522 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.522 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.522 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-09-11T23:39:54.522 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.522 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.522 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:54.522 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:54.522 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:54.522 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:54.522 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:54.522 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:54.522 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:54.522 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:54.522 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.523 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.523 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.523 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.523 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.523 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-09-11T23:39:54.523 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-09-11T23:39:54.523 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.523 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.523 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.523 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:54.523 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.523 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.523 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.523 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.524 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.524 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.524 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.524 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.524 INFO:teuthology.orchestra.run.smithi026.stdout: "min_size": { 2024-09-11T23:39:54.524 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "min_size", 2024-09-11T23:39:54.524 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-09-11T23:39:54.524 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.524 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.524 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:54.524 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.524 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.524 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.524 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "default min_size for any created local pool", 2024-09-11T23:39:54.525 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-11T23:39:54.525 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.525 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.525 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.525 INFO:teuthology.orchestra.run.smithi026.stdout: "num_rep": { 2024-09-11T23:39:54.525 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "num_rep", 2024-09-11T23:39:54.525 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-09-11T23:39:54.525 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.525 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.525 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "3", 2024-09-11T23:39:54.525 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.525 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.525 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.525 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "default replica count for any created local pool", 2024-09-11T23:39:54.526 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.526 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.526 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.526 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.526 INFO:teuthology.orchestra.run.smithi026.stdout: "pg_num": { 2024-09-11T23:39:54.526 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "pg_num", 2024-09-11T23:39:54.526 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-09-11T23:39:54.526 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.526 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.526 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "128", 2024-09-11T23:39:54.526 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.526 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.526 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.526 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "default pg_num for any created local pool", 2024-09-11T23:39:54.526 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.527 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.527 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.527 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.527 INFO:teuthology.orchestra.run.smithi026.stdout: "prefix": { 2024-09-11T23:39:54.527 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "prefix", 2024-09-11T23:39:54.527 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.527 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.527 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.527 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:54.527 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.527 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.527 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.527 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "name prefix for any created local pool", 2024-09-11T23:39:54.527 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.528 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.528 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.528 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.528 INFO:teuthology.orchestra.run.smithi026.stdout: "subtree": { 2024-09-11T23:39:54.528 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "subtree", 2024-09-11T23:39:54.528 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.528 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.528 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.528 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "rack", 2024-09-11T23:39:54.528 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.528 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.528 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.528 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-11T23:39:54.528 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-11T23:39:54.529 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.529 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.529 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:54.529 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:54.529 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.529 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-11T23:39:54.529 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "mds_autoscaler", 2024-09-11T23:39:54.529 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-09-11T23:39:54.529 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-09-11T23:39:54.529 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-09-11T23:39:54.529 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-09-11T23:39:54.529 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-09-11T23:39:54.529 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.529 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.529 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.530 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:54.530 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.530 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.530 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:54.530 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:54.530 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:54.530 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:54.530 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:54.530 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:54.530 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:54.530 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:54.530 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.530 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.530 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.531 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.531 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.531 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-09-11T23:39:54.531 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-09-11T23:39:54.531 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.531 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.531 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.531 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:54.531 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.531 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.531 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.531 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.531 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.531 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.532 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.532 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.532 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-09-11T23:39:54.532 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-09-11T23:39:54.532 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.532 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.532 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.532 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-09-11T23:39:54.532 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.532 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.532 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:54.532 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:54.532 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:54.532 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:54.533 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:54.533 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:54.533 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:54.533 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:54.533 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.533 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.533 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.533 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.533 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.533 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-09-11T23:39:54.533 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-09-11T23:39:54.533 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.533 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.533 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.534 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:54.534 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.534 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.534 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.534 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.534 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.534 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.534 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.534 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:54.534 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:54.534 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.534 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-11T23:39:54.534 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "mirroring", 2024-09-11T23:39:54.534 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-09-11T23:39:54.535 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-09-11T23:39:54.535 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-09-11T23:39:54.535 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-09-11T23:39:54.535 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-09-11T23:39:54.535 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.535 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.535 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.535 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:54.535 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.535 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.535 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:54.535 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:54.535 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:54.535 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:54.536 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:54.536 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:54.536 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:54.536 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:54.536 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.536 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.536 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.536 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.536 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.536 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-09-11T23:39:54.536 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-09-11T23:39:54.536 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.536 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.536 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.537 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:54.537 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.537 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.537 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.537 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.537 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.537 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.537 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.537 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.537 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-09-11T23:39:54.537 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-09-11T23:39:54.537 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.537 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.537 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.538 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-09-11T23:39:54.538 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.538 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.538 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:54.538 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:54.538 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:54.538 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:54.538 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:54.538 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:54.538 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:54.538 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:54.538 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.538 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.538 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.539 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.539 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.539 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-09-11T23:39:54.539 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-09-11T23:39:54.539 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.539 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.539 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.539 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:54.539 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.539 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.539 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.539 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.539 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.539 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.540 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.540 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:54.540 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:54.540 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.540 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-11T23:39:54.540 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "nfs", 2024-09-11T23:39:54.540 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-09-11T23:39:54.540 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-09-11T23:39:54.540 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-09-11T23:39:54.540 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-09-11T23:39:54.540 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-09-11T23:39:54.540 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.540 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.540 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.541 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:54.541 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.541 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.541 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:54.541 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:54.541 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:54.541 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:54.541 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:54.541 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:54.541 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:54.541 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:54.541 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.541 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.541 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.541 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.542 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.542 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-09-11T23:39:54.542 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-09-11T23:39:54.542 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.542 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.542 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.542 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:54.542 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.542 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.542 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.542 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.542 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.542 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.542 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.543 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.543 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-09-11T23:39:54.543 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-09-11T23:39:54.543 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.543 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.543 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.543 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-09-11T23:39:54.543 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.543 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.543 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:54.543 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:54.543 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:54.543 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:54.543 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:54.543 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:54.544 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:54.544 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:54.544 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.544 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.544 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.544 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.544 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.544 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-09-11T23:39:54.544 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-09-11T23:39:54.544 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.544 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.544 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.544 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:54.544 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.545 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.545 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.545 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.545 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.545 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.545 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.545 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:54.545 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:54.545 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.546 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-11T23:39:54.546 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "orchestrator", 2024-09-11T23:39:54.546 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-09-11T23:39:54.546 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-09-11T23:39:54.546 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-09-11T23:39:54.546 INFO:teuthology.orchestra.run.smithi026.stdout: "fail_fs": { 2024-09-11T23:39:54.546 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "fail_fs", 2024-09-11T23:39:54.546 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.546 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.547 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.547 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:54.547 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.547 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.547 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.547 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-11T23:39:54.547 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.547 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.547 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.547 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.547 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-09-11T23:39:54.547 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-09-11T23:39:54.547 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.547 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.548 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.548 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:54.548 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.548 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.548 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:54.548 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:54.548 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:54.548 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:54.548 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:54.548 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:54.548 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:54.548 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:54.548 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.549 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.549 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.549 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.549 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.549 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-09-11T23:39:54.549 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-09-11T23:39:54.549 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.549 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.549 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.549 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:54.549 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.549 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.549 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.549 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.550 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.550 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.550 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.550 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.550 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-09-11T23:39:54.550 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-09-11T23:39:54.550 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.550 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.550 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.550 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-09-11T23:39:54.550 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.550 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.550 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:54.550 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:54.550 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:54.551 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:54.551 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:54.551 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:54.551 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:54.551 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:54.551 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.551 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.551 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.551 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.551 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.551 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-09-11T23:39:54.551 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-09-11T23:39:54.551 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.552 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.552 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.552 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:54.552 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.552 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.552 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.552 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.552 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.552 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.552 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.552 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.552 INFO:teuthology.orchestra.run.smithi026.stdout: "orchestrator": { 2024-09-11T23:39:54.552 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "orchestrator", 2024-09-11T23:39:54.552 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.553 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.553 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.553 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:54.553 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.553 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.553 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:54.553 INFO:teuthology.orchestra.run.smithi026.stdout: "cephadm", 2024-09-11T23:39:54.553 INFO:teuthology.orchestra.run.smithi026.stdout: "rook", 2024-09-11T23:39:54.553 INFO:teuthology.orchestra.run.smithi026.stdout: "test_orchestrator" 2024-09-11T23:39:54.553 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:54.553 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Orchestrator backend", 2024-09-11T23:39:54.553 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.553 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.554 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.554 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:54.554 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:54.554 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.554 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-11T23:39:54.554 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "osd_perf_query", 2024-09-11T23:39:54.554 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-09-11T23:39:54.554 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-09-11T23:39:54.554 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-09-11T23:39:54.554 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-09-11T23:39:54.554 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-09-11T23:39:54.554 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.554 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.555 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.555 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:54.555 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.555 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.555 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:54.555 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:54.555 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:54.555 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:54.555 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:54.555 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:54.555 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:54.555 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:54.555 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.556 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.556 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.556 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.556 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.556 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-09-11T23:39:54.556 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-09-11T23:39:54.556 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.556 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.556 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.556 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:54.556 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.556 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.556 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.557 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.557 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.557 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.557 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.557 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.557 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-09-11T23:39:54.557 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-09-11T23:39:54.557 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.557 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.557 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.557 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-09-11T23:39:54.557 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.557 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.557 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:54.558 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:54.558 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:54.558 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:54.558 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:54.558 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:54.558 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:54.558 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:54.558 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.558 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.558 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.558 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.558 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.558 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-09-11T23:39:54.558 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-09-11T23:39:54.559 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.559 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.559 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.559 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:54.559 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.559 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.559 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.559 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.559 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.559 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.559 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.559 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:54.559 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:54.559 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.560 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-11T23:39:54.560 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "osd_support", 2024-09-11T23:39:54.560 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-09-11T23:39:54.560 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-09-11T23:39:54.560 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-09-11T23:39:54.560 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-09-11T23:39:54.560 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-09-11T23:39:54.560 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.560 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.560 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.560 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:54.560 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.560 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.561 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:54.561 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:54.561 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:54.561 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:54.561 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:54.561 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:54.561 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:54.561 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:54.561 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.561 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.561 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.561 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.561 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.561 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-09-11T23:39:54.562 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-09-11T23:39:54.562 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.562 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.562 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.562 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:54.562 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.562 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.562 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.562 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.562 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.562 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.562 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.562 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.562 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-09-11T23:39:54.563 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-09-11T23:39:54.563 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.563 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.563 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.563 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-09-11T23:39:54.563 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.563 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.563 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:54.563 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:54.563 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:54.563 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:54.563 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:54.563 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:54.563 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:54.564 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:54.564 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.564 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.564 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.564 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.564 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.564 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-09-11T23:39:54.564 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-09-11T23:39:54.564 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.564 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.564 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.564 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:54.564 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.564 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.565 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.565 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.565 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.565 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.565 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.565 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:54.565 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:54.565 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.565 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-11T23:39:54.565 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "pg_autoscaler", 2024-09-11T23:39:54.565 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-09-11T23:39:54.565 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-09-11T23:39:54.565 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-09-11T23:39:54.565 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-09-11T23:39:54.566 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-09-11T23:39:54.566 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.566 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.566 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.566 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:54.566 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.566 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.566 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:54.566 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:54.566 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:54.566 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:54.566 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:54.566 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:54.567 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:54.567 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:54.567 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.567 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.567 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.567 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.567 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.567 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-09-11T23:39:54.567 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-09-11T23:39:54.567 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.567 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.567 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.567 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:54.567 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.568 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.568 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.568 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.568 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.568 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.568 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.568 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.568 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-09-11T23:39:54.568 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-09-11T23:39:54.568 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.568 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.568 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.568 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-09-11T23:39:54.569 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.569 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.569 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:54.569 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:54.569 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:54.569 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:54.569 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:54.569 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:54.569 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:54.569 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:54.569 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.569 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.569 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.569 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.570 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.570 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-09-11T23:39:54.570 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-09-11T23:39:54.570 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.570 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.570 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.570 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:54.570 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.570 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.570 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.570 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.570 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.570 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.570 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.571 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.571 INFO:teuthology.orchestra.run.smithi026.stdout: "sleep_interval": { 2024-09-11T23:39:54.571 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sleep_interval", 2024-09-11T23:39:54.571 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-09-11T23:39:54.571 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.571 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.571 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "60", 2024-09-11T23:39:54.571 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.571 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.571 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.571 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.571 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.571 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.571 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.572 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.572 INFO:teuthology.orchestra.run.smithi026.stdout: "threshold": { 2024-09-11T23:39:54.572 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "threshold", 2024-09-11T23:39:54.572 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "float", 2024-09-11T23:39:54.572 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.572 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.572 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "3.0", 2024-09-11T23:39:54.572 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "1.0", 2024-09-11T23:39:54.572 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.572 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.572 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "scaling threshold", 2024-09-11T23:39:54.572 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-11T23:39:54.572 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.572 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.573 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:54.573 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:54.573 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.573 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-11T23:39:54.573 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "progress", 2024-09-11T23:39:54.573 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-09-11T23:39:54.573 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-09-11T23:39:54.573 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-09-11T23:39:54.573 INFO:teuthology.orchestra.run.smithi026.stdout: "allow_pg_recovery_event": { 2024-09-11T23:39:54.573 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "allow_pg_recovery_event", 2024-09-11T23:39:54.573 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.573 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.573 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.573 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:54.574 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.574 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.574 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.574 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-11T23:39:54.574 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.574 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.574 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.574 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.574 INFO:teuthology.orchestra.run.smithi026.stdout: "enabled": { 2024-09-11T23:39:54.574 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "enabled", 2024-09-11T23:39:54.574 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.574 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.574 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.574 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-09-11T23:39:54.575 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.575 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.575 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.575 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.575 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.575 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.575 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.575 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.575 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-09-11T23:39:54.575 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-09-11T23:39:54.575 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.575 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.575 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.575 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:54.576 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.576 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.576 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:54.576 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:54.576 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:54.576 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:54.576 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:54.576 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:54.576 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:54.576 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:54.576 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.576 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.576 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.576 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.577 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.577 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-09-11T23:39:54.577 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-09-11T23:39:54.577 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.577 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.577 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.577 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:54.577 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.577 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.577 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.577 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.577 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.577 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.577 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.577 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.578 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-09-11T23:39:54.578 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-09-11T23:39:54.578 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.578 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.578 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.578 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-09-11T23:39:54.578 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.578 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.578 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:54.578 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:54.578 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:54.578 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:54.578 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:54.578 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:54.579 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:54.579 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:54.579 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.579 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.579 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.579 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.579 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.579 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-09-11T23:39:54.579 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-09-11T23:39:54.579 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.579 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.579 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.579 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:54.579 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.580 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.580 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.580 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.580 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.580 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.580 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.580 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.580 INFO:teuthology.orchestra.run.smithi026.stdout: "max_completed_events": { 2024-09-11T23:39:54.580 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "max_completed_events", 2024-09-11T23:39:54.580 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-09-11T23:39:54.580 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.580 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.580 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "50", 2024-09-11T23:39:54.580 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.581 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.581 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.581 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "number of past completed events to remember", 2024-09-11T23:39:54.581 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.581 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.581 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.581 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.581 INFO:teuthology.orchestra.run.smithi026.stdout: "sleep_interval": { 2024-09-11T23:39:54.581 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sleep_interval", 2024-09-11T23:39:54.581 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-09-11T23:39:54.581 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.581 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.581 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "5", 2024-09-11T23:39:54.581 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.582 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.582 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.582 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "how long the module is going to sleep", 2024-09-11T23:39:54.582 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.582 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.582 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.582 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:54.582 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:54.582 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.582 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-11T23:39:54.582 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "prometheus", 2024-09-11T23:39:54.582 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-09-11T23:39:54.582 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-09-11T23:39:54.582 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-09-11T23:39:54.583 INFO:teuthology.orchestra.run.smithi026.stdout: "cache": { 2024-09-11T23:39:54.583 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "cache", 2024-09-11T23:39:54.583 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.583 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.583 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.583 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-09-11T23:39:54.583 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.583 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.583 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.583 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.583 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.583 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.583 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.583 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.584 INFO:teuthology.orchestra.run.smithi026.stdout: "exclude_perf_counters": { 2024-09-11T23:39:54.584 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "exclude_perf_counters", 2024-09-11T23:39:54.584 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.584 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.584 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.584 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-09-11T23:39:54.584 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.584 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.584 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.584 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-11T23:39:54.584 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-11T23:39:54.584 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.584 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.584 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.585 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-09-11T23:39:54.585 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-09-11T23:39:54.585 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.585 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.585 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.585 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:54.585 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.585 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.585 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:54.585 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:54.585 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:54.585 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:54.585 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:54.585 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:54.586 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:54.586 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:54.586 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.586 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.586 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.586 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.586 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.586 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-09-11T23:39:54.586 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-09-11T23:39:54.586 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.586 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.586 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.586 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:54.586 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.587 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.587 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.587 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.587 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.587 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.587 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.587 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.587 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-09-11T23:39:54.587 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-09-11T23:39:54.587 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.587 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.587 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.587 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-09-11T23:39:54.588 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.588 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.588 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:54.588 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:54.588 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:54.588 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:54.588 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:54.588 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:54.588 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:54.588 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:54.588 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.588 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.588 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.588 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.589 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.589 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-09-11T23:39:54.589 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-09-11T23:39:54.589 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.589 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.589 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.589 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:54.589 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.589 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.589 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.589 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.589 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.589 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.589 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.590 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.590 INFO:teuthology.orchestra.run.smithi026.stdout: "rbd_stats_pools": { 2024-09-11T23:39:54.590 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rbd_stats_pools", 2024-09-11T23:39:54.590 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.590 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.590 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.590 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:54.590 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.590 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.590 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.590 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.590 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.590 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.590 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.591 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.591 INFO:teuthology.orchestra.run.smithi026.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-11T23:39:54.591 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-11T23:39:54.591 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-09-11T23:39:54.591 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.591 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.591 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "300", 2024-09-11T23:39:54.591 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.591 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.591 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.591 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.591 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.591 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.591 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.592 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.592 INFO:teuthology.orchestra.run.smithi026.stdout: "scrape_interval": { 2024-09-11T23:39:54.592 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "scrape_interval", 2024-09-11T23:39:54.592 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "float", 2024-09-11T23:39:54.592 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.592 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.592 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "15.0", 2024-09-11T23:39:54.592 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.592 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.592 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.592 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.592 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.592 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.593 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.593 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.593 INFO:teuthology.orchestra.run.smithi026.stdout: "server_addr": { 2024-09-11T23:39:54.593 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "server_addr", 2024-09-11T23:39:54.593 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.593 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.593 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.593 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "::", 2024-09-11T23:39:54.593 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.593 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.593 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.593 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-11T23:39:54.593 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.593 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.593 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.594 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.594 INFO:teuthology.orchestra.run.smithi026.stdout: "server_port": { 2024-09-11T23:39:54.594 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "server_port", 2024-09-11T23:39:54.594 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-09-11T23:39:54.594 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.594 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.594 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "9283", 2024-09-11T23:39:54.594 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.594 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.594 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.594 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-11T23:39:54.594 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.594 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.594 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.595 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.595 INFO:teuthology.orchestra.run.smithi026.stdout: "stale_cache_strategy": { 2024-09-11T23:39:54.595 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "stale_cache_strategy", 2024-09-11T23:39:54.595 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.595 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.595 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.595 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "log", 2024-09-11T23:39:54.595 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.595 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.595 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.595 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.595 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.595 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.595 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.595 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.596 INFO:teuthology.orchestra.run.smithi026.stdout: "standby_behaviour": { 2024-09-11T23:39:54.596 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "standby_behaviour", 2024-09-11T23:39:54.596 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.596 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.596 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.596 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "default", 2024-09-11T23:39:54.596 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.596 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.596 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:54.596 INFO:teuthology.orchestra.run.smithi026.stdout: "default", 2024-09-11T23:39:54.596 INFO:teuthology.orchestra.run.smithi026.stdout: "error" 2024-09-11T23:39:54.596 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:54.596 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.596 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.597 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.597 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.597 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.597 INFO:teuthology.orchestra.run.smithi026.stdout: "standby_error_status_code": { 2024-09-11T23:39:54.597 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "standby_error_status_code", 2024-09-11T23:39:54.597 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-09-11T23:39:54.597 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.597 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.597 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "500", 2024-09-11T23:39:54.597 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "400", 2024-09-11T23:39:54.597 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "599", 2024-09-11T23:39:54.597 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.597 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.597 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.597 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.598 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.598 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:54.598 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:54.598 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.598 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-11T23:39:54.598 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rbd_support", 2024-09-11T23:39:54.598 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-09-11T23:39:54.598 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-09-11T23:39:54.598 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-09-11T23:39:54.598 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-09-11T23:39:54.598 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-09-11T23:39:54.598 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.598 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.598 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.599 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:54.599 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.599 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.599 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:54.599 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:54.599 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:54.599 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:54.599 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:54.599 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:54.599 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:54.599 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:54.599 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.600 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.600 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.600 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.600 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.600 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-09-11T23:39:54.600 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-09-11T23:39:54.600 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.600 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.600 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.600 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:54.600 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.600 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.600 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.600 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.601 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.601 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.601 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.601 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.601 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-09-11T23:39:54.601 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-09-11T23:39:54.601 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.601 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.601 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.601 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-09-11T23:39:54.601 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.601 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.601 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:54.601 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:54.602 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:54.602 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:54.602 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:54.602 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:54.602 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:54.602 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:54.602 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.602 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.602 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.602 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.602 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.602 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-09-11T23:39:54.602 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-09-11T23:39:54.602 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.603 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.603 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.603 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:54.603 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.603 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.603 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.603 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.603 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.603 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.603 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.603 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.603 INFO:teuthology.orchestra.run.smithi026.stdout: "max_concurrent_snap_create": { 2024-09-11T23:39:54.603 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "max_concurrent_snap_create", 2024-09-11T23:39:54.603 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-09-11T23:39:54.604 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.604 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.604 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "10", 2024-09-11T23:39:54.604 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.604 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.604 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.604 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.604 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.604 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.604 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.604 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.604 INFO:teuthology.orchestra.run.smithi026.stdout: "mirror_snapshot_schedule": { 2024-09-11T23:39:54.604 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "mirror_snapshot_schedule", 2024-09-11T23:39:54.604 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.605 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.605 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.605 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:54.605 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.605 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.605 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.605 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.605 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.605 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.605 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.605 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.605 INFO:teuthology.orchestra.run.smithi026.stdout: "trash_purge_schedule": { 2024-09-11T23:39:54.605 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "trash_purge_schedule", 2024-09-11T23:39:54.605 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.606 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.606 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.606 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:54.606 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.606 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.606 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.606 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.606 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.606 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.606 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.606 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:54.606 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:54.606 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.606 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-11T23:39:54.607 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "restful", 2024-09-11T23:39:54.607 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-09-11T23:39:54.607 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-09-11T23:39:54.607 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-09-11T23:39:54.607 INFO:teuthology.orchestra.run.smithi026.stdout: "enable_auth": { 2024-09-11T23:39:54.607 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "enable_auth", 2024-09-11T23:39:54.607 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.607 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.607 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.607 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-09-11T23:39:54.607 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.607 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.607 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.607 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.607 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.608 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.608 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.608 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.608 INFO:teuthology.orchestra.run.smithi026.stdout: "key_file": { 2024-09-11T23:39:54.608 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "key_file", 2024-09-11T23:39:54.608 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.608 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.608 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.608 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:54.608 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.608 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.608 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.608 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.608 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.609 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.609 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.609 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.609 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-09-11T23:39:54.609 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-09-11T23:39:54.609 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.609 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.609 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.609 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:54.609 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.609 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.609 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:54.609 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:54.609 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:54.610 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:54.610 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:54.610 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:54.610 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:54.610 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:54.610 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.610 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.610 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.610 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.610 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.610 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-09-11T23:39:54.610 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-09-11T23:39:54.610 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.610 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.610 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.611 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:54.611 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.611 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.611 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.611 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.611 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.611 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.611 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.611 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.611 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-09-11T23:39:54.611 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-09-11T23:39:54.611 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.611 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.611 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.612 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-09-11T23:39:54.612 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.612 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.612 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:54.612 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:54.612 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:54.612 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:54.612 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:54.612 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:54.612 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:54.612 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:54.612 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.612 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.612 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.612 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.613 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.613 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-09-11T23:39:54.613 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-09-11T23:39:54.613 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.613 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.613 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.613 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:54.613 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.613 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.613 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.613 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.613 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.613 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.613 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.614 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.614 INFO:teuthology.orchestra.run.smithi026.stdout: "server_addr": { 2024-09-11T23:39:54.614 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "server_addr", 2024-09-11T23:39:54.614 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.614 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.614 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.614 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:54.614 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.614 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.614 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.614 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.614 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.614 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.614 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.614 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.615 INFO:teuthology.orchestra.run.smithi026.stdout: "server_port": { 2024-09-11T23:39:54.615 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "server_port", 2024-09-11T23:39:54.615 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.615 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.615 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.615 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:54.615 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.615 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.615 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.615 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.615 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.615 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.615 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.615 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:54.615 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:54.616 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.616 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-11T23:39:54.616 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rgw", 2024-09-11T23:39:54.616 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-09-11T23:39:54.616 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-09-11T23:39:54.616 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-09-11T23:39:54.616 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-09-11T23:39:54.616 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-09-11T23:39:54.616 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.616 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.616 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.616 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:54.616 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.617 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.617 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:54.617 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:54.617 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:54.617 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:54.617 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:54.617 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:54.617 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:54.617 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:54.617 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.617 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.617 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.617 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.617 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.618 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-09-11T23:39:54.618 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-09-11T23:39:54.618 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.618 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.618 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.618 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:54.618 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.618 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.618 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.618 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.618 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.618 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.618 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.618 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.619 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-09-11T23:39:54.619 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-09-11T23:39:54.619 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.619 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.619 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.619 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-09-11T23:39:54.619 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.619 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.619 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:54.619 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:54.619 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:54.619 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:54.619 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:54.619 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:54.619 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:54.620 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:54.620 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.620 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.620 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.620 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.620 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.620 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-09-11T23:39:54.620 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-09-11T23:39:54.620 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.620 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.620 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.620 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:54.620 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.621 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.621 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.621 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.621 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.621 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.621 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.621 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:54.621 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:54.621 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.621 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-11T23:39:54.621 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rook", 2024-09-11T23:39:54.621 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-09-11T23:39:54.621 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-09-11T23:39:54.621 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-09-11T23:39:54.622 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-09-11T23:39:54.622 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-09-11T23:39:54.622 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.622 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.622 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.622 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:54.622 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.622 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.622 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:54.622 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:54.622 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:54.622 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:54.622 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:54.622 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:54.623 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:54.623 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:54.623 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.623 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.623 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.623 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.623 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.623 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-09-11T23:39:54.623 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-09-11T23:39:54.623 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.623 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.623 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.623 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:54.623 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.624 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.624 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.624 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.624 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.624 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.624 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.624 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.624 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-09-11T23:39:54.624 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-09-11T23:39:54.624 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.624 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.624 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.624 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-09-11T23:39:54.624 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.625 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.625 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:54.625 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:54.625 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:54.625 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:54.625 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:54.625 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:54.625 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:54.625 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:54.625 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.625 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.625 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.625 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.625 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.626 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-09-11T23:39:54.626 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-09-11T23:39:54.626 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.626 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.626 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.626 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:54.626 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.626 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.626 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.626 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.626 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.626 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.626 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.626 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.626 INFO:teuthology.orchestra.run.smithi026.stdout: "storage_class": { 2024-09-11T23:39:54.627 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "storage_class", 2024-09-11T23:39:54.627 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.627 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.627 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.627 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "local", 2024-09-11T23:39:54.627 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.627 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.627 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.627 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-11T23:39:54.627 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.627 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.627 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.627 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:54.627 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:54.628 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.628 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-11T23:39:54.628 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "selftest", 2024-09-11T23:39:54.628 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-09-11T23:39:54.628 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-09-11T23:39:54.628 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-09-11T23:39:54.628 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-09-11T23:39:54.628 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-09-11T23:39:54.628 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.628 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.628 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.628 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:54.628 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.628 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.628 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:54.629 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:54.629 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:54.629 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:54.629 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:54.629 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:54.629 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:54.629 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:54.629 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.629 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.629 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.629 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.629 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.629 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-09-11T23:39:54.629 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-09-11T23:39:54.630 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.630 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.630 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.630 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:54.630 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.630 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.630 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.630 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.630 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.630 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.630 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.630 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.630 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-09-11T23:39:54.630 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-09-11T23:39:54.630 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.631 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.631 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.631 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-09-11T23:39:54.631 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.631 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.631 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:54.631 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:54.631 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:54.631 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:54.631 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:54.631 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:54.631 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:54.631 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:54.631 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.632 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.632 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.632 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.632 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.632 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-09-11T23:39:54.632 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-09-11T23:39:54.632 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.632 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.632 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.632 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:54.632 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.632 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.632 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.632 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.632 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.633 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.633 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.633 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.633 INFO:teuthology.orchestra.run.smithi026.stdout: "roption1": { 2024-09-11T23:39:54.633 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "roption1", 2024-09-11T23:39:54.633 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.633 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.633 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.633 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:54.633 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.633 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.633 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.633 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.634 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.634 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.634 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.634 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.634 INFO:teuthology.orchestra.run.smithi026.stdout: "roption2": { 2024-09-11T23:39:54.634 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "roption2", 2024-09-11T23:39:54.634 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.634 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.634 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.634 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "xyz", 2024-09-11T23:39:54.634 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.634 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.634 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.634 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.635 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.635 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.635 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.635 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.635 INFO:teuthology.orchestra.run.smithi026.stdout: "rwoption1": { 2024-09-11T23:39:54.635 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rwoption1", 2024-09-11T23:39:54.635 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.635 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.635 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.635 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:54.635 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.635 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.635 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.635 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.636 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.636 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.636 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.636 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.636 INFO:teuthology.orchestra.run.smithi026.stdout: "rwoption2": { 2024-09-11T23:39:54.636 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rwoption2", 2024-09-11T23:39:54.636 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-09-11T23:39:54.636 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.636 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.636 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:54.636 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.636 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.636 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.636 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.637 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.637 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.637 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.637 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.637 INFO:teuthology.orchestra.run.smithi026.stdout: "rwoption3": { 2024-09-11T23:39:54.637 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rwoption3", 2024-09-11T23:39:54.637 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "float", 2024-09-11T23:39:54.637 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.637 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.637 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:54.637 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.637 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.637 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.637 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.638 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.638 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.638 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.638 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.638 INFO:teuthology.orchestra.run.smithi026.stdout: "rwoption4": { 2024-09-11T23:39:54.638 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rwoption4", 2024-09-11T23:39:54.638 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.638 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.638 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.638 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:54.638 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.638 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.638 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.638 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.639 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.639 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.639 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.639 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.639 INFO:teuthology.orchestra.run.smithi026.stdout: "rwoption5": { 2024-09-11T23:39:54.639 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rwoption5", 2024-09-11T23:39:54.639 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.639 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.639 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.639 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:54.639 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.639 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.639 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.639 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.640 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.640 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.640 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.640 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.640 INFO:teuthology.orchestra.run.smithi026.stdout: "rwoption6": { 2024-09-11T23:39:54.640 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rwoption6", 2024-09-11T23:39:54.640 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.640 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.640 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.640 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-09-11T23:39:54.640 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.640 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.640 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.640 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.641 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.641 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.641 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.641 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.641 INFO:teuthology.orchestra.run.smithi026.stdout: "rwoption7": { 2024-09-11T23:39:54.641 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rwoption7", 2024-09-11T23:39:54.641 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-09-11T23:39:54.641 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.641 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.641 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:54.641 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "1", 2024-09-11T23:39:54.641 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "42", 2024-09-11T23:39:54.641 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.641 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.641 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.642 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.642 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.642 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.642 INFO:teuthology.orchestra.run.smithi026.stdout: "testkey": { 2024-09-11T23:39:54.642 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "testkey", 2024-09-11T23:39:54.642 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.642 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.642 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.642 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:54.642 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.642 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.642 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.642 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.642 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.643 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.643 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.643 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.643 INFO:teuthology.orchestra.run.smithi026.stdout: "testlkey": { 2024-09-11T23:39:54.643 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "testlkey", 2024-09-11T23:39:54.643 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.643 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.643 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.643 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:54.643 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.643 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.643 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.643 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.643 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.644 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.644 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.644 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.644 INFO:teuthology.orchestra.run.smithi026.stdout: "testnewline": { 2024-09-11T23:39:54.644 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "testnewline", 2024-09-11T23:39:54.644 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.644 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.644 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.644 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:54.644 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.644 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.644 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.644 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.644 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.645 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.645 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.645 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:54.645 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:54.645 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.645 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-11T23:39:54.645 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "snap_schedule", 2024-09-11T23:39:54.645 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-09-11T23:39:54.645 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-09-11T23:39:54.645 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-09-11T23:39:54.645 INFO:teuthology.orchestra.run.smithi026.stdout: "allow_m_granularity": { 2024-09-11T23:39:54.645 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "allow_m_granularity", 2024-09-11T23:39:54.645 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.645 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.645 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.646 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:54.646 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.646 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.646 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.646 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "allow minute scheduled snapshots", 2024-09-11T23:39:54.646 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.646 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.646 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.646 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.646 INFO:teuthology.orchestra.run.smithi026.stdout: "dump_on_update": { 2024-09-11T23:39:54.646 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "dump_on_update", 2024-09-11T23:39:54.646 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.646 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.646 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.647 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:54.647 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.647 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.647 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.647 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "dump database to debug log on update", 2024-09-11T23:39:54.647 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.647 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.647 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.647 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.647 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-09-11T23:39:54.647 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-09-11T23:39:54.647 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.647 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.647 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.647 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:54.648 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.648 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.648 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:54.648 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:54.648 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:54.648 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:54.648 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:54.648 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:54.648 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:54.648 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:54.648 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.648 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.648 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.648 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.649 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.649 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-09-11T23:39:54.649 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-09-11T23:39:54.649 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.649 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.649 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.649 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:54.649 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.649 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.649 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.649 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.649 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.649 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.649 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.650 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.650 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-09-11T23:39:54.650 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-09-11T23:39:54.650 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.650 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.650 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.650 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-09-11T23:39:54.650 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.650 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.650 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:54.650 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:54.650 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:54.650 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:54.651 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:54.651 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:54.651 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:54.651 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:54.651 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.651 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.651 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.651 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.651 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.651 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-09-11T23:39:54.651 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-09-11T23:39:54.651 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.651 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.651 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.652 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:54.652 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.652 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.652 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.652 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.652 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.652 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.652 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.652 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:54.652 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:54.652 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.652 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-11T23:39:54.652 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "stats", 2024-09-11T23:39:54.652 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-09-11T23:39:54.653 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-09-11T23:39:54.653 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-09-11T23:39:54.653 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-09-11T23:39:54.653 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-09-11T23:39:54.653 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.653 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.653 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.653 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:54.653 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.653 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.653 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:54.653 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:54.653 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:54.653 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:54.654 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:54.654 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:54.654 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:54.654 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:54.654 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.654 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.654 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.654 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.654 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.654 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-09-11T23:39:54.654 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-09-11T23:39:54.654 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.654 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.655 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.655 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:54.655 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.655 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.655 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.655 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.655 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.655 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.655 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.655 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.655 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-09-11T23:39:54.655 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-09-11T23:39:54.655 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.656 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.656 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.656 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-09-11T23:39:54.656 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.656 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.656 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:54.656 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:54.656 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:54.656 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:54.656 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:54.656 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:54.656 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:54.656 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:54.656 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.657 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.657 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.657 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.657 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.657 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-09-11T23:39:54.657 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-09-11T23:39:54.657 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.657 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.657 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.657 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:54.657 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.657 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.658 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.658 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.658 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.658 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.658 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.658 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:54.658 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:54.658 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.658 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-11T23:39:54.658 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "status", 2024-09-11T23:39:54.658 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-09-11T23:39:54.658 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-09-11T23:39:54.658 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-09-11T23:39:54.658 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-09-11T23:39:54.659 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-09-11T23:39:54.659 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.659 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.659 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.659 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:54.659 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.659 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.659 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:54.659 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:54.659 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:54.659 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:54.659 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:54.659 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:54.659 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:54.659 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:54.660 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.660 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.660 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.660 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.660 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.660 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-09-11T23:39:54.660 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-09-11T23:39:54.660 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.660 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.660 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.660 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:54.660 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.660 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.660 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.660 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.661 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.661 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.661 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.661 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.661 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-09-11T23:39:54.661 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-09-11T23:39:54.661 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.661 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.661 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.661 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-09-11T23:39:54.661 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.661 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.661 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:54.661 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:54.662 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:54.662 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:54.662 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:54.662 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:54.662 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:54.662 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:54.662 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.662 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.662 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.662 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.662 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.662 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-09-11T23:39:54.662 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-09-11T23:39:54.662 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.663 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.663 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.663 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:54.663 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.663 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.663 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.663 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.663 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.663 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.663 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.663 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:54.663 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:54.663 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.663 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-11T23:39:54.664 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "telegraf", 2024-09-11T23:39:54.664 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-09-11T23:39:54.664 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-09-11T23:39:54.664 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-09-11T23:39:54.664 INFO:teuthology.orchestra.run.smithi026.stdout: "address": { 2024-09-11T23:39:54.664 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "address", 2024-09-11T23:39:54.664 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.664 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.664 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.664 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-11T23:39:54.664 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.664 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.664 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.664 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.665 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.665 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.665 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.665 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.665 INFO:teuthology.orchestra.run.smithi026.stdout: "interval": { 2024-09-11T23:39:54.665 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "interval", 2024-09-11T23:39:54.665 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-09-11T23:39:54.665 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.665 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.665 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "15", 2024-09-11T23:39:54.665 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.665 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.665 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.665 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.666 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.666 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.666 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.666 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.666 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-09-11T23:39:54.666 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-09-11T23:39:54.666 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.666 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.666 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.666 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:54.666 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.666 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.666 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:54.666 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:54.666 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:54.667 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:54.667 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:54.667 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:54.667 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:54.667 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:54.667 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.667 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.667 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.667 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.667 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.667 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-09-11T23:39:54.667 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-09-11T23:39:54.667 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.668 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.668 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.668 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:54.668 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.668 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.668 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.668 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.668 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.668 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.668 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.668 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.668 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-09-11T23:39:54.668 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-09-11T23:39:54.668 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.669 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.669 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.669 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-09-11T23:39:54.669 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.669 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.669 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:54.669 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:54.669 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:54.669 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:54.669 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:54.669 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:54.669 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:54.669 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:54.669 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.670 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.670 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.670 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.670 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.670 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-09-11T23:39:54.670 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-09-11T23:39:54.670 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.670 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.670 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.670 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:54.670 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.670 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.670 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.671 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.671 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.671 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.671 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.671 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:54.671 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:54.671 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.671 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-11T23:39:54.671 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "telemetry", 2024-09-11T23:39:54.671 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-09-11T23:39:54.671 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-09-11T23:39:54.671 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-09-11T23:39:54.671 INFO:teuthology.orchestra.run.smithi026.stdout: "channel_basic": { 2024-09-11T23:39:54.671 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "channel_basic", 2024-09-11T23:39:54.672 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.672 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.672 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.672 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-09-11T23:39:54.672 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.672 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.672 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.672 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-11T23:39:54.672 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.672 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.672 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.672 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.672 INFO:teuthology.orchestra.run.smithi026.stdout: "channel_crash": { 2024-09-11T23:39:54.672 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "channel_crash", 2024-09-11T23:39:54.673 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.673 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.673 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.673 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-09-11T23:39:54.673 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.673 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.673 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.673 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-11T23:39:54.673 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.673 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.673 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.673 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.673 INFO:teuthology.orchestra.run.smithi026.stdout: "channel_device": { 2024-09-11T23:39:54.673 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "channel_device", 2024-09-11T23:39:54.674 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.674 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.674 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.674 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-09-11T23:39:54.674 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.674 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.674 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.674 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-11T23:39:54.674 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.674 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.674 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.674 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.674 INFO:teuthology.orchestra.run.smithi026.stdout: "channel_ident": { 2024-09-11T23:39:54.674 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "channel_ident", 2024-09-11T23:39:54.675 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.675 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.675 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.675 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:54.675 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.675 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.675 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.675 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-11T23:39:54.675 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.675 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.675 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.675 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.675 INFO:teuthology.orchestra.run.smithi026.stdout: "channel_perf": { 2024-09-11T23:39:54.675 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "channel_perf", 2024-09-11T23:39:54.676 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.676 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.676 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.676 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:54.676 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.676 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.676 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.676 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-11T23:39:54.676 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.676 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.676 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.676 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.676 INFO:teuthology.orchestra.run.smithi026.stdout: "contact": { 2024-09-11T23:39:54.676 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "contact", 2024-09-11T23:39:54.677 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.677 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.677 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.677 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:54.677 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.677 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.677 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.677 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.677 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.677 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.677 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.677 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.677 INFO:teuthology.orchestra.run.smithi026.stdout: "description": { 2024-09-11T23:39:54.677 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "description", 2024-09-11T23:39:54.677 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.678 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.678 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.678 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:54.678 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.678 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.678 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.678 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.678 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.678 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.678 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.678 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.678 INFO:teuthology.orchestra.run.smithi026.stdout: "device_url": { 2024-09-11T23:39:54.678 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "device_url", 2024-09-11T23:39:54.678 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.679 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.679 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.679 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-11T23:39:54.679 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.679 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.679 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.679 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.679 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.679 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.679 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.679 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.679 INFO:teuthology.orchestra.run.smithi026.stdout: "enabled": { 2024-09-11T23:39:54.679 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "enabled", 2024-09-11T23:39:54.679 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.680 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.680 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.680 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:54.680 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.680 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.680 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.680 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.680 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.680 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.680 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.680 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.680 INFO:teuthology.orchestra.run.smithi026.stdout: "interval": { 2024-09-11T23:39:54.680 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "interval", 2024-09-11T23:39:54.680 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-09-11T23:39:54.681 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.681 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.681 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "24", 2024-09-11T23:39:54.681 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "8", 2024-09-11T23:39:54.681 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.681 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.681 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.681 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.681 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.681 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.681 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.681 INFO:teuthology.orchestra.run.smithi026.stdout: "last_opt_revision": { 2024-09-11T23:39:54.681 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "last_opt_revision", 2024-09-11T23:39:54.681 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-09-11T23:39:54.682 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.682 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.682 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "1", 2024-09-11T23:39:54.682 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.682 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.682 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.682 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.682 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.682 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.682 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.682 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.682 INFO:teuthology.orchestra.run.smithi026.stdout: "leaderboard": { 2024-09-11T23:39:54.682 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "leaderboard", 2024-09-11T23:39:54.682 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.682 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.683 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.683 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:54.683 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.683 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.683 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.683 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.683 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.683 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.683 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.683 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.683 INFO:teuthology.orchestra.run.smithi026.stdout: "leaderboard_description": { 2024-09-11T23:39:54.683 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "leaderboard_description", 2024-09-11T23:39:54.683 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.683 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.684 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.684 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:54.684 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.684 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.684 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.684 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.684 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.684 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.684 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.684 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.684 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-09-11T23:39:54.684 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-09-11T23:39:54.684 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.685 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.685 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.685 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:54.685 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.685 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.685 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:54.685 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:54.685 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:54.685 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:54.685 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:54.685 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:54.685 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:54.685 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:54.685 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.686 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.686 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.686 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.686 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.686 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-09-11T23:39:54.686 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-09-11T23:39:54.686 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.686 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.686 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.686 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:54.686 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.686 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.686 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.686 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.687 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.687 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.687 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.687 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.687 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-09-11T23:39:54.687 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-09-11T23:39:54.687 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.687 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.687 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.687 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-09-11T23:39:54.687 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.687 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.687 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:54.687 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:54.688 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:54.688 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:54.688 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:54.688 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:54.688 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:54.688 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:54.688 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.688 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.688 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.688 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.688 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.688 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-09-11T23:39:54.688 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-09-11T23:39:54.688 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.689 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.689 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.689 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:54.689 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.689 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.689 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.689 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.689 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.689 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.689 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.689 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.689 INFO:teuthology.orchestra.run.smithi026.stdout: "organization": { 2024-09-11T23:39:54.689 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "organization", 2024-09-11T23:39:54.690 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.690 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.690 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.690 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:54.690 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.690 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.690 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.690 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.690 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.690 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.690 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.690 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.690 INFO:teuthology.orchestra.run.smithi026.stdout: "proxy": { 2024-09-11T23:39:54.690 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "proxy", 2024-09-11T23:39:54.690 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.691 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.691 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.691 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:54.691 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.691 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.691 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.691 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.691 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.691 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.691 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.691 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.691 INFO:teuthology.orchestra.run.smithi026.stdout: "url": { 2024-09-11T23:39:54.691 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "url", 2024-09-11T23:39:54.691 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.691 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.692 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.692 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-11T23:39:54.692 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.692 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.692 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.692 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.692 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.692 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.692 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.692 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:54.692 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:54.692 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.692 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-11T23:39:54.692 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "test_orchestrator", 2024-09-11T23:39:54.693 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-09-11T23:39:54.693 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-09-11T23:39:54.693 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-09-11T23:39:54.693 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-09-11T23:39:54.693 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-09-11T23:39:54.693 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.693 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.693 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.693 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:54.693 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.693 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.693 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:54.693 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:54.693 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:54.694 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:54.694 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:54.694 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:54.694 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:54.694 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:54.694 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.694 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.694 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.694 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.694 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.694 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-09-11T23:39:54.694 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-09-11T23:39:54.694 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.694 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.695 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.695 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:54.695 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.695 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.695 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.695 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.695 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.695 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.695 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.695 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.695 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-09-11T23:39:54.695 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-09-11T23:39:54.695 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.695 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.696 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.696 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-09-11T23:39:54.696 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.696 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.696 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:54.696 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:54.696 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:54.696 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:54.696 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:54.696 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:54.696 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:54.696 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:54.696 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.696 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.697 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.697 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.697 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.697 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-09-11T23:39:54.697 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-09-11T23:39:54.697 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.697 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.697 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.697 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:54.697 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.697 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.697 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.697 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.697 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.698 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.698 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.698 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:54.698 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:54.698 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.698 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-11T23:39:54.698 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "volumes", 2024-09-11T23:39:54.698 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-09-11T23:39:54.698 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-09-11T23:39:54.698 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-09-11T23:39:54.698 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-09-11T23:39:54.698 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-09-11T23:39:54.698 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.698 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.699 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.699 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:54.699 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.699 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.699 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:54.699 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:54.699 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:54.699 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:54.699 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:54.699 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:54.699 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:54.699 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:54.699 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.699 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.700 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.700 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.700 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.700 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-09-11T23:39:54.700 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-09-11T23:39:54.700 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.700 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.700 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.700 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:54.700 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.700 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.700 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.700 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.701 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.701 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.701 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.701 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.701 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-09-11T23:39:54.701 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-09-11T23:39:54.701 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.701 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.701 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.701 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-09-11T23:39:54.701 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.701 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.701 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:54.701 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:54.702 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:54.702 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:54.702 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:54.702 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:54.702 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:54.702 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:54.702 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.702 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.702 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.702 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.702 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.702 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-09-11T23:39:54.702 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-09-11T23:39:54.702 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.703 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.703 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.703 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:54.703 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.703 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.703 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.703 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.703 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.703 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.703 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.703 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.703 INFO:teuthology.orchestra.run.smithi026.stdout: "max_concurrent_clones": { 2024-09-11T23:39:54.703 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "max_concurrent_clones", 2024-09-11T23:39:54.703 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-09-11T23:39:54.704 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.704 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.704 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "4", 2024-09-11T23:39:54.704 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.704 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.704 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.704 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-11T23:39:54.704 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.704 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.704 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.704 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.704 INFO:teuthology.orchestra.run.smithi026.stdout: "periodic_async_work": { 2024-09-11T23:39:54.704 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "periodic_async_work", 2024-09-11T23:39:54.704 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.705 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.705 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.705 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:54.705 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.705 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.705 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.705 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Periodically check for async work", 2024-09-11T23:39:54.705 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.705 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.705 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.705 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.705 INFO:teuthology.orchestra.run.smithi026.stdout: "snapshot_clone_delay": { 2024-09-11T23:39:54.705 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "snapshot_clone_delay", 2024-09-11T23:39:54.705 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-09-11T23:39:54.706 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.706 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.706 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2024-09-11T23:39:54.706 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.706 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.706 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.706 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-11T23:39:54.706 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.706 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.706 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.706 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.706 INFO:teuthology.orchestra.run.smithi026.stdout: "snapshot_clone_no_wait": { 2024-09-11T23:39:54.706 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "snapshot_clone_no_wait", 2024-09-11T23:39:54.707 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.707 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.707 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.707 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-09-11T23:39:54.707 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.707 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.707 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.707 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-11T23:39:54.707 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.707 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.707 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.707 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:54.707 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:54.707 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.707 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-11T23:39:54.708 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "zabbix", 2024-09-11T23:39:54.708 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-09-11T23:39:54.708 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-09-11T23:39:54.708 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-09-11T23:39:54.708 INFO:teuthology.orchestra.run.smithi026.stdout: "discovery_interval": { 2024-09-11T23:39:54.708 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "discovery_interval", 2024-09-11T23:39:54.708 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "uint", 2024-09-11T23:39:54.708 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.708 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.708 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "100", 2024-09-11T23:39:54.708 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.708 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.708 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.708 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.709 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.709 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.709 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.709 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.709 INFO:teuthology.orchestra.run.smithi026.stdout: "identifier": { 2024-09-11T23:39:54.709 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "identifier", 2024-09-11T23:39:54.709 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.709 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.709 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.709 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:54.709 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.709 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.709 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.709 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.710 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.710 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.710 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.710 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.710 INFO:teuthology.orchestra.run.smithi026.stdout: "interval": { 2024-09-11T23:39:54.710 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "interval", 2024-09-11T23:39:54.710 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-09-11T23:39:54.710 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.710 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.710 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "60", 2024-09-11T23:39:54.710 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.710 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.710 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.710 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.711 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.711 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.711 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.711 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.711 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-09-11T23:39:54.711 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-09-11T23:39:54.711 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.711 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.711 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.711 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:54.711 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.711 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.711 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:54.711 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:54.711 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:54.712 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:54.712 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:54.712 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:54.712 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:54.712 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:54.712 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.712 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.712 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.712 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.712 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.712 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-09-11T23:39:54.712 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-09-11T23:39:54.712 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.712 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.713 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.713 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:54.713 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.713 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.713 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.713 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.713 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.713 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.713 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.713 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.713 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-09-11T23:39:54.713 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-09-11T23:39:54.713 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.713 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.714 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.714 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-09-11T23:39:54.714 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.714 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.714 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-09-11T23:39:54.714 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-09-11T23:39:54.714 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-09-11T23:39:54.714 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-09-11T23:39:54.714 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-09-11T23:39:54.714 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-09-11T23:39:54.714 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-09-11T23:39:54.714 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:54.714 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.714 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.714 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.715 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.715 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.715 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-09-11T23:39:54.715 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-09-11T23:39:54.715 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-09-11T23:39:54.715 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.715 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-09-11T23:39:54.715 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-09-11T23:39:54.715 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.715 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.715 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.715 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.715 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.715 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.716 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.716 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.716 INFO:teuthology.orchestra.run.smithi026.stdout: "zabbix_host": { 2024-09-11T23:39:54.716 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "zabbix_host", 2024-09-11T23:39:54.716 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.716 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.716 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.716 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-09-11T23:39:54.716 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.716 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.716 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.716 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.716 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.716 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.717 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.717 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.717 INFO:teuthology.orchestra.run.smithi026.stdout: "zabbix_port": { 2024-09-11T23:39:54.717 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "zabbix_port", 2024-09-11T23:39:54.717 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-09-11T23:39:54.717 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.717 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.717 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "10051", 2024-09-11T23:39:54.717 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.717 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.717 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.717 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.717 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.717 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.717 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.718 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.718 INFO:teuthology.orchestra.run.smithi026.stdout: "zabbix_sender": { 2024-09-11T23:39:54.718 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "zabbix_sender", 2024-09-11T23:39:54.718 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-09-11T23:39:54.718 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-09-11T23:39:54.718 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-09-11T23:39:54.718 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-11T23:39:54.718 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-09-11T23:39:54.718 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-09-11T23:39:54.718 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-09-11T23:39:54.718 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-09-11T23:39:54.718 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-09-11T23:39:54.718 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-09-11T23:39:54.718 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-09-11T23:39:54.719 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:54.719 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:54.719 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:54.719 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:54.719 INFO:teuthology.orchestra.run.smithi026.stdout: "services": { 2024-09-11T23:39:54.719 INFO:teuthology.orchestra.run.smithi026.stdout: "dashboard": "https://172.21.15.26:8443/" 2024-09-11T23:39:54.719 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.719 INFO:teuthology.orchestra.run.smithi026.stdout: "always_on_modules": { 2024-09-11T23:39:54.719 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus": [ 2024-09-11T23:39:54.719 INFO:teuthology.orchestra.run.smithi026.stdout: "balancer", 2024-09-11T23:39:54.719 INFO:teuthology.orchestra.run.smithi026.stdout: "crash", 2024-09-11T23:39:54.719 INFO:teuthology.orchestra.run.smithi026.stdout: "devicehealth", 2024-09-11T23:39:54.719 INFO:teuthology.orchestra.run.smithi026.stdout: "orchestrator", 2024-09-11T23:39:54.719 INFO:teuthology.orchestra.run.smithi026.stdout: "pg_autoscaler", 2024-09-11T23:39:54.719 INFO:teuthology.orchestra.run.smithi026.stdout: "progress", 2024-09-11T23:39:54.720 INFO:teuthology.orchestra.run.smithi026.stdout: "rbd_support", 2024-09-11T23:39:54.720 INFO:teuthology.orchestra.run.smithi026.stdout: "status", 2024-09-11T23:39:54.720 INFO:teuthology.orchestra.run.smithi026.stdout: "telemetry", 2024-09-11T23:39:54.720 INFO:teuthology.orchestra.run.smithi026.stdout: "volumes" 2024-09-11T23:39:54.720 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:54.720 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific": [ 2024-09-11T23:39:54.720 INFO:teuthology.orchestra.run.smithi026.stdout: "balancer", 2024-09-11T23:39:54.720 INFO:teuthology.orchestra.run.smithi026.stdout: "crash", 2024-09-11T23:39:54.720 INFO:teuthology.orchestra.run.smithi026.stdout: "devicehealth", 2024-09-11T23:39:54.720 INFO:teuthology.orchestra.run.smithi026.stdout: "orchestrator", 2024-09-11T23:39:54.720 INFO:teuthology.orchestra.run.smithi026.stdout: "pg_autoscaler", 2024-09-11T23:39:54.720 INFO:teuthology.orchestra.run.smithi026.stdout: "progress", 2024-09-11T23:39:54.720 INFO:teuthology.orchestra.run.smithi026.stdout: "rbd_support", 2024-09-11T23:39:54.720 INFO:teuthology.orchestra.run.smithi026.stdout: "status", 2024-09-11T23:39:54.721 INFO:teuthology.orchestra.run.smithi026.stdout: "telemetry", 2024-09-11T23:39:54.721 INFO:teuthology.orchestra.run.smithi026.stdout: "volumes" 2024-09-11T23:39:54.721 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:54.721 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy": [ 2024-09-11T23:39:54.721 INFO:teuthology.orchestra.run.smithi026.stdout: "balancer", 2024-09-11T23:39:54.721 INFO:teuthology.orchestra.run.smithi026.stdout: "crash", 2024-09-11T23:39:54.721 INFO:teuthology.orchestra.run.smithi026.stdout: "devicehealth", 2024-09-11T23:39:54.721 INFO:teuthology.orchestra.run.smithi026.stdout: "orchestrator", 2024-09-11T23:39:54.721 INFO:teuthology.orchestra.run.smithi026.stdout: "pg_autoscaler", 2024-09-11T23:39:54.721 INFO:teuthology.orchestra.run.smithi026.stdout: "progress", 2024-09-11T23:39:54.721 INFO:teuthology.orchestra.run.smithi026.stdout: "rbd_support", 2024-09-11T23:39:54.721 INFO:teuthology.orchestra.run.smithi026.stdout: "status", 2024-09-11T23:39:54.721 INFO:teuthology.orchestra.run.smithi026.stdout: "telemetry", 2024-09-11T23:39:54.721 INFO:teuthology.orchestra.run.smithi026.stdout: "volumes" 2024-09-11T23:39:54.721 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:54.722 INFO:teuthology.orchestra.run.smithi026.stdout: "reef": [ 2024-09-11T23:39:54.722 INFO:teuthology.orchestra.run.smithi026.stdout: "balancer", 2024-09-11T23:39:54.722 INFO:teuthology.orchestra.run.smithi026.stdout: "crash", 2024-09-11T23:39:54.722 INFO:teuthology.orchestra.run.smithi026.stdout: "devicehealth", 2024-09-11T23:39:54.722 INFO:teuthology.orchestra.run.smithi026.stdout: "orchestrator", 2024-09-11T23:39:54.722 INFO:teuthology.orchestra.run.smithi026.stdout: "pg_autoscaler", 2024-09-11T23:39:54.722 INFO:teuthology.orchestra.run.smithi026.stdout: "progress", 2024-09-11T23:39:54.722 INFO:teuthology.orchestra.run.smithi026.stdout: "rbd_support", 2024-09-11T23:39:54.722 INFO:teuthology.orchestra.run.smithi026.stdout: "status", 2024-09-11T23:39:54.722 INFO:teuthology.orchestra.run.smithi026.stdout: "telemetry", 2024-09-11T23:39:54.722 INFO:teuthology.orchestra.run.smithi026.stdout: "volumes" 2024-09-11T23:39:54.722 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-11T23:39:54.722 INFO:teuthology.orchestra.run.smithi026.stdout: "squid": [ 2024-09-11T23:39:54.722 INFO:teuthology.orchestra.run.smithi026.stdout: "balancer", 2024-09-11T23:39:54.723 INFO:teuthology.orchestra.run.smithi026.stdout: "crash", 2024-09-11T23:39:54.723 INFO:teuthology.orchestra.run.smithi026.stdout: "devicehealth", 2024-09-11T23:39:54.723 INFO:teuthology.orchestra.run.smithi026.stdout: "orchestrator", 2024-09-11T23:39:54.723 INFO:teuthology.orchestra.run.smithi026.stdout: "pg_autoscaler", 2024-09-11T23:39:54.723 INFO:teuthology.orchestra.run.smithi026.stdout: "progress", 2024-09-11T23:39:54.723 INFO:teuthology.orchestra.run.smithi026.stdout: "rbd_support", 2024-09-11T23:39:54.723 INFO:teuthology.orchestra.run.smithi026.stdout: "status", 2024-09-11T23:39:54.723 INFO:teuthology.orchestra.run.smithi026.stdout: "telemetry", 2024-09-11T23:39:54.723 INFO:teuthology.orchestra.run.smithi026.stdout: "volumes" 2024-09-11T23:39:54.723 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-11T23:39:54.723 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.723 INFO:teuthology.orchestra.run.smithi026.stdout: "last_failure_osd_epoch": 86, 2024-09-11T23:39:54.723 INFO:teuthology.orchestra.run.smithi026.stdout: "active_clients": [ 2024-09-11T23:39:54.723 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-11T23:39:54.723 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "devicehealth", 2024-09-11T23:39:54.724 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-11T23:39:54.724 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-11T23:39:54.724 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v2", 2024-09-11T23:39:54.724 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:0", 2024-09-11T23:39:54.724 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 3655967317 2024-09-11T23:39:54.724 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:54.724 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-11T23:39:54.724 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.724 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-11T23:39:54.724 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "libcephsqlite", 2024-09-11T23:39:54.724 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-11T23:39:54.724 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-11T23:39:54.724 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v2", 2024-09-11T23:39:54.724 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:0", 2024-09-11T23:39:54.725 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 1298288631 2024-09-11T23:39:54.725 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:54.725 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-11T23:39:54.725 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.725 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-11T23:39:54.725 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rbd_support", 2024-09-11T23:39:54.725 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-11T23:39:54.725 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-11T23:39:54.725 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v2", 2024-09-11T23:39:54.725 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:0", 2024-09-11T23:39:54.725 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 2774844506 2024-09-11T23:39:54.725 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:54.725 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-11T23:39:54.725 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.725 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-11T23:39:54.726 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "snap_schedule", 2024-09-11T23:39:54.726 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-11T23:39:54.726 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-11T23:39:54.726 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v2", 2024-09-11T23:39:54.726 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:0", 2024-09-11T23:39:54.726 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 4158267134 2024-09-11T23:39:54.726 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:54.726 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-11T23:39:54.726 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.726 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-11T23:39:54.726 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "snap_schedule", 2024-09-11T23:39:54.726 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-11T23:39:54.726 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-11T23:39:54.726 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v2", 2024-09-11T23:39:54.727 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:0", 2024-09-11T23:39:54.727 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 3267539905 2024-09-11T23:39:54.727 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:54.727 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-11T23:39:54.727 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:54.727 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-11T23:39:54.727 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "volumes", 2024-09-11T23:39:54.727 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-11T23:39:54.727 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-11T23:39:54.727 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v2", 2024-09-11T23:39:54.727 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:0", 2024-09-11T23:39:54.727 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 3367189939 2024-09-11T23:39:54.727 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:54.727 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-11T23:39:54.727 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:54.728 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-11T23:39:54.728 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-11T23:39:54.729 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:39:53 smithi076 ceph-mon[35296]: from='client.18165 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:39:54.729 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:39:53 smithi076 ceph-mon[35296]: from='client.18171 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:39:54.729 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:39:53 smithi076 ceph-mon[35296]: from='client.? 172.21.15.26:0/723856560' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T23:39:54.730 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:39:53 smithi076 ceph-mon[35296]: from='client.? 172.21.15.26:0/3175047535' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-11T23:39:54.730 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:39:53 smithi076 ceph-mon[35296]: from='client.? 172.21.15.26:0/2213146306' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-11T23:39:54.730 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:39:53 smithi076 ceph-mon[35296]: from='client.? 172.21.15.26:0/3827593029' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-11T23:39:54.730 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:39:54 smithi076 ceph-mon[35296]: pgmap v677: 97 pgs: 4 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 88 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.4 MiB/s wr, 382 op/s 2024-09-11T23:39:54.730 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:39:54 smithi076 ceph-mon[35296]: from='client.? 172.21.15.26:0/1393308929' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-11T23:39:54.730 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:39:53 smithi136 ceph-mon[35121]: from='client.18165 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:39:54.730 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:39:53 smithi136 ceph-mon[35121]: from='client.18171 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:39:54.730 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:39:53 smithi136 ceph-mon[35121]: from='client.? 172.21.15.26:0/723856560' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T23:39:54.730 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:39:53 smithi136 ceph-mon[35121]: from='client.? 172.21.15.26:0/3175047535' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-11T23:39:54.730 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:39:53 smithi136 ceph-mon[35121]: from='client.? 172.21.15.26:0/2213146306' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-11T23:39:54.730 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:39:53 smithi136 ceph-mon[35121]: from='client.? 172.21.15.26:0/3827593029' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-11T23:39:54.730 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:39:54 smithi136 ceph-mon[35121]: pgmap v677: 97 pgs: 4 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 88 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.4 MiB/s wr, 382 op/s 2024-09-11T23:39:54.730 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:39:54 smithi136 ceph-mon[35121]: from='client.? 172.21.15.26:0/1393308929' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-11T23:39:54.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:53.421+0000 7f551a6b7640 1 -- 172.21.15.26:0/1393308929 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f54fc0a2d00 msgr2=0x7f54fc0a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:54.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:53.421+0000 7f551a6b7640 1 --2- 172.21.15.26:0/1393308929 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f54fc0a2d00 0x7f54fc0a51c0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f5514069ed0 tx=0x7f550800b390 comp rx=0 tx=0).stop 2024-09-11T23:39:54.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:53.421+0000 7f551a6b7640 1 -- 172.21.15.26:0/1393308929 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5514100980 msgr2=0x7f55141992e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:54.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:53.421+0000 7f551a6b7640 1 --2- 172.21.15.26:0/1393308929 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5514100980 0x7f55141992e0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f5504009230 tx=0x7f5504004270 comp rx=0 tx=0).stop 2024-09-11T23:39:54.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:53.422+0000 7f551a6b7640 1 -- 172.21.15.26:0/1393308929 shutdown_connections 2024-09-11T23:39:54.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:53.422+0000 7f551a6b7640 1 --2- 172.21.15.26:0/1393308929 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f54fc0a2d00 0x7f54fc0a51c0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:54.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:53.422+0000 7f551a6b7640 1 --2- 172.21.15.26:0/1393308929 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5514101320 0x7f5514199820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:54.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:53.422+0000 7f551a6b7640 1 --2- 172.21.15.26:0/1393308929 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5514106990 0x7f55141a4360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:54.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:53.422+0000 7f551a6b7640 1 --2- 172.21.15.26:0/1393308929 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5514100980 0x7f55141992e0 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:54.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:53.422+0000 7f551a6b7640 1 -- 172.21.15.26:0/1393308929 >> 172.21.15.26:0/1393308929 conn(0x7f5514077a10 msgr2=0x7f55141039e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:39:54.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:53.422+0000 7f551a6b7640 1 -- 172.21.15.26:0/1393308929 shutdown_connections 2024-09-11T23:39:54.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:39:53.422+0000 7f551a6b7640 1 -- 172.21.15.26:0/1393308929 wait complete. 2024-09-11T23:39:54.802 DEBUG:tasks.check_counter:Getting stats from a 2024-09-11T23:39:54.803 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph daemon mds.a perf dump 2024-09-11T23:39:54.803 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:39:54 smithi026 ceph-mon[31172]: pgmap v677: 97 pgs: 4 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 88 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.4 MiB/s wr, 382 op/s 2024-09-11T23:39:54.803 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:39:54 smithi026 ceph-mon[31172]: from='client.? 172.21.15.26:0/1393308929' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-11T23:39:55.074 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:39:55.574 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-11T23:39:55.575 INFO:teuthology.orchestra.run.smithi026.stdout: "AsyncMessenger::Worker-0": { 2024-09-11T23:39:55.575 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_messages": 2465, 2024-09-11T23:39:55.575 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages": 2388, 2024-09-11T23:39:55.575 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_bytes": 46457433, 2024-09-11T23:39:55.575 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_bytes": 867668, 2024-09-11T23:39:55.575 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_created_connections": 7, 2024-09-11T23:39:55.575 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_active_connections": 3, 2024-09-11T23:39:55.575 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_total_time": 0.307293384, 2024-09-11T23:39:55.575 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_send_time": 0.091804800, 2024-09-11T23:39:55.575 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_recv_time": 0.141175928, 2024-09-11T23:39:55.576 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_fast_dispatch_time": 0.072855842, 2024-09-11T23:39:55.576 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T23:39:55.576 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 2388, 2024-09-11T23:39:55.576 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.158760903, 2024-09-11T23:39:55.576 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000066482 2024-09-11T23:39:55.576 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:55.576 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_handle_ack_lat": { 2024-09-11T23:39:55.576 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-11T23:39:55.576 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-11T23:39:55.576 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-11T23:39:55.576 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:55.576 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_encrypted_bytes": 2100704, 2024-09-11T23:39:55.576 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_encrypted_bytes": 239936 2024-09-11T23:39:55.577 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:55.577 INFO:teuthology.orchestra.run.smithi026.stdout: "AsyncMessenger::Worker-1": { 2024-09-11T23:39:55.577 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_messages": 713, 2024-09-11T23:39:55.577 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages": 1000, 2024-09-11T23:39:55.577 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_bytes": 105052673, 2024-09-11T23:39:55.577 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_bytes": 990088, 2024-09-11T23:39:55.577 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_created_connections": 10, 2024-09-11T23:39:55.577 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_active_connections": 7, 2024-09-11T23:39:55.577 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_total_time": 0.164273448, 2024-09-11T23:39:55.577 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_send_time": 0.030935005, 2024-09-11T23:39:55.577 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_recv_time": 0.103536297, 2024-09-11T23:39:55.577 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_fast_dispatch_time": 0.022093976, 2024-09-11T23:39:55.577 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T23:39:55.577 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 997, 2024-09-11T23:39:55.578 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.074730229, 2024-09-11T23:39:55.578 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000074955 2024-09-11T23:39:55.578 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:55.578 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_handle_ack_lat": { 2024-09-11T23:39:55.578 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-11T23:39:55.578 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-11T23:39:55.578 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-11T23:39:55.578 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:55.578 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_encrypted_bytes": 480, 2024-09-11T23:39:55.578 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_encrypted_bytes": 764240 2024-09-11T23:39:55.578 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:55.578 INFO:teuthology.orchestra.run.smithi026.stdout: "AsyncMessenger::Worker-2": { 2024-09-11T23:39:55.578 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_messages": 1958, 2024-09-11T23:39:55.578 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages": 1960, 2024-09-11T23:39:55.579 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_bytes": 105369866, 2024-09-11T23:39:55.579 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_bytes": 622645, 2024-09-11T23:39:55.579 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_created_connections": 6, 2024-09-11T23:39:55.579 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_active_connections": 5, 2024-09-11T23:39:55.579 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_total_time": 0.219710044, 2024-09-11T23:39:55.579 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_send_time": 0.052167136, 2024-09-11T23:39:55.579 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_recv_time": 0.099302006, 2024-09-11T23:39:55.579 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_fast_dispatch_time": 0.054267132, 2024-09-11T23:39:55.579 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T23:39:55.579 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 1958, 2024-09-11T23:39:55.579 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.083833577, 2024-09-11T23:39:55.579 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000042815 2024-09-11T23:39:55.579 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:55.580 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_handle_ack_lat": { 2024-09-11T23:39:55.580 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-11T23:39:55.580 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-11T23:39:55.580 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-11T23:39:55.580 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:55.580 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-11T23:39:55.580 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_encrypted_bytes": 0 2024-09-11T23:39:55.580 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:55.580 INFO:teuthology.orchestra.run.smithi026.stdout: "cct": { 2024-09-11T23:39:55.580 INFO:teuthology.orchestra.run.smithi026.stdout: "total_workers": 1, 2024-09-11T23:39:55.580 INFO:teuthology.orchestra.run.smithi026.stdout: "unhealthy_workers": 0 2024-09-11T23:39:55.580 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:55.580 INFO:teuthology.orchestra.run.smithi026.stdout: "finisher-MDSRank": { 2024-09-11T23:39:55.581 INFO:teuthology.orchestra.run.smithi026.stdout: "queue_len": 0, 2024-09-11T23:39:55.581 INFO:teuthology.orchestra.run.smithi026.stdout: "complete_latency": { 2024-09-11T23:39:55.581 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 7598, 2024-09-11T23:39:55.581 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.218342036, 2024-09-11T23:39:55.581 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000028736 2024-09-11T23:39:55.581 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:55.581 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:55.581 INFO:teuthology.orchestra.run.smithi026.stdout: "finisher-PurgeQueue": { 2024-09-11T23:39:55.581 INFO:teuthology.orchestra.run.smithi026.stdout: "queue_len": 0, 2024-09-11T23:39:55.581 INFO:teuthology.orchestra.run.smithi026.stdout: "complete_latency": { 2024-09-11T23:39:55.581 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-11T23:39:55.581 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-11T23:39:55.581 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-11T23:39:55.582 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:55.582 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:55.582 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": { 2024-09-11T23:39:55.582 INFO:teuthology.orchestra.run.smithi026.stdout: "request": 0, 2024-09-11T23:39:55.582 INFO:teuthology.orchestra.run.smithi026.stdout: "reply": 0, 2024-09-11T23:39:55.582 INFO:teuthology.orchestra.run.smithi026.stdout: "reply_latency": { 2024-09-11T23:39:55.582 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-11T23:39:55.582 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-11T23:39:55.582 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-11T23:39:55.582 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:55.582 INFO:teuthology.orchestra.run.smithi026.stdout: "slow_reply": 0, 2024-09-11T23:39:55.582 INFO:teuthology.orchestra.run.smithi026.stdout: "forward": 0, 2024-09-11T23:39:55.582 INFO:teuthology.orchestra.run.smithi026.stdout: "dir_fetch_complete": 0, 2024-09-11T23:39:55.582 INFO:teuthology.orchestra.run.smithi026.stdout: "dir_fetch_keys": 0, 2024-09-11T23:39:55.583 INFO:teuthology.orchestra.run.smithi026.stdout: "dir_commit": 0, 2024-09-11T23:39:55.583 INFO:teuthology.orchestra.run.smithi026.stdout: "dir_split": 0, 2024-09-11T23:39:55.583 INFO:teuthology.orchestra.run.smithi026.stdout: "dir_merge": 0, 2024-09-11T23:39:55.583 INFO:teuthology.orchestra.run.smithi026.stdout: "inodes": 43912, 2024-09-11T23:39:55.583 INFO:teuthology.orchestra.run.smithi026.stdout: "inodes_top": 685, 2024-09-11T23:39:55.583 INFO:teuthology.orchestra.run.smithi026.stdout: "inodes_bottom": 43227, 2024-09-11T23:39:55.583 INFO:teuthology.orchestra.run.smithi026.stdout: "inodes_pin_tail": 0, 2024-09-11T23:39:55.583 INFO:teuthology.orchestra.run.smithi026.stdout: "inodes_pinned": 42933, 2024-09-11T23:39:55.583 INFO:teuthology.orchestra.run.smithi026.stdout: "inodes_expired": 2, 2024-09-11T23:39:55.583 INFO:teuthology.orchestra.run.smithi026.stdout: "inodes_with_caps": 0, 2024-09-11T23:39:55.583 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": 0, 2024-09-11T23:39:55.583 INFO:teuthology.orchestra.run.smithi026.stdout: "subtrees": 1, 2024-09-11T23:39:55.583 INFO:teuthology.orchestra.run.smithi026.stdout: "traverse": 0, 2024-09-11T23:39:55.583 INFO:teuthology.orchestra.run.smithi026.stdout: "traverse_hit": 0, 2024-09-11T23:39:55.584 INFO:teuthology.orchestra.run.smithi026.stdout: "traverse_forward": 0, 2024-09-11T23:39:55.584 INFO:teuthology.orchestra.run.smithi026.stdout: "traverse_discover": 0, 2024-09-11T23:39:55.584 INFO:teuthology.orchestra.run.smithi026.stdout: "traverse_dir_fetch": 0, 2024-09-11T23:39:55.584 INFO:teuthology.orchestra.run.smithi026.stdout: "traverse_remote_ino": 0, 2024-09-11T23:39:55.584 INFO:teuthology.orchestra.run.smithi026.stdout: "traverse_lock": 0, 2024-09-11T23:39:55.584 INFO:teuthology.orchestra.run.smithi026.stdout: "load_cent": 0, 2024-09-11T23:39:55.584 INFO:teuthology.orchestra.run.smithi026.stdout: "q": 0, 2024-09-11T23:39:55.584 INFO:teuthology.orchestra.run.smithi026.stdout: "exported": 0, 2024-09-11T23:39:55.584 INFO:teuthology.orchestra.run.smithi026.stdout: "exported_inodes": 0, 2024-09-11T23:39:55.584 INFO:teuthology.orchestra.run.smithi026.stdout: "imported": 0, 2024-09-11T23:39:55.584 INFO:teuthology.orchestra.run.smithi026.stdout: "imported_inodes": 0, 2024-09-11T23:39:55.584 INFO:teuthology.orchestra.run.smithi026.stdout: "openino_dir_fetch": 0, 2024-09-11T23:39:55.584 INFO:teuthology.orchestra.run.smithi026.stdout: "openino_backtrace_fetch": 0, 2024-09-11T23:39:55.585 INFO:teuthology.orchestra.run.smithi026.stdout: "openino_peer_discover": 0, 2024-09-11T23:39:55.585 INFO:teuthology.orchestra.run.smithi026.stdout: "root_rfiles": 0, 2024-09-11T23:39:55.585 INFO:teuthology.orchestra.run.smithi026.stdout: "root_rbytes": 0, 2024-09-11T23:39:55.585 INFO:teuthology.orchestra.run.smithi026.stdout: "root_rsnaps": 0, 2024-09-11T23:39:55.585 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_backtrace_fetch": 0, 2024-09-11T23:39:55.585 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_set_tag": 0, 2024-09-11T23:39:55.585 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_backtrace_repaired": 0, 2024-09-11T23:39:55.585 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_inotable_repaired": 0, 2024-09-11T23:39:55.585 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_dir_inodes": 0, 2024-09-11T23:39:55.585 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_dir_base_inodes": 0, 2024-09-11T23:39:55.585 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_dirfrag_rstats": 0, 2024-09-11T23:39:55.585 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_file_inodes": 0, 2024-09-11T23:39:55.585 INFO:teuthology.orchestra.run.smithi026.stdout: "handle_inode_file_caps": 0, 2024-09-11T23:39:55.585 INFO:teuthology.orchestra.run.smithi026.stdout: "ceph_cap_op_revoke": 0, 2024-09-11T23:39:55.585 INFO:teuthology.orchestra.run.smithi026.stdout: "ceph_cap_op_grant": 0, 2024-09-11T23:39:55.586 INFO:teuthology.orchestra.run.smithi026.stdout: "ceph_cap_op_trunc": 0, 2024-09-11T23:39:55.586 INFO:teuthology.orchestra.run.smithi026.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-11T23:39:55.586 INFO:teuthology.orchestra.run.smithi026.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-11T23:39:55.586 INFO:teuthology.orchestra.run.smithi026.stdout: "handle_client_caps": 0, 2024-09-11T23:39:55.586 INFO:teuthology.orchestra.run.smithi026.stdout: "handle_client_caps_dirty": 0, 2024-09-11T23:39:55.586 INFO:teuthology.orchestra.run.smithi026.stdout: "handle_client_cap_release": 0, 2024-09-11T23:39:55.586 INFO:teuthology.orchestra.run.smithi026.stdout: "process_request_cap_release": 0 2024-09-11T23:39:55.586 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:55.586 INFO:teuthology.orchestra.run.smithi026.stdout: "mds_cache": { 2024-09-11T23:39:55.586 INFO:teuthology.orchestra.run.smithi026.stdout: "dir_update": 0, 2024-09-11T23:39:55.586 INFO:teuthology.orchestra.run.smithi026.stdout: "dir_update_receipt": 0, 2024-09-11T23:39:55.586 INFO:teuthology.orchestra.run.smithi026.stdout: "dir_try_discover": 0, 2024-09-11T23:39:55.586 INFO:teuthology.orchestra.run.smithi026.stdout: "dir_send_discover": 0, 2024-09-11T23:39:55.586 INFO:teuthology.orchestra.run.smithi026.stdout: "dir_handle_discover": 0, 2024-09-11T23:39:55.587 INFO:teuthology.orchestra.run.smithi026.stdout: "num_strays": 0, 2024-09-11T23:39:55.587 INFO:teuthology.orchestra.run.smithi026.stdout: "num_strays_delayed": 0, 2024-09-11T23:39:55.587 INFO:teuthology.orchestra.run.smithi026.stdout: "num_strays_enqueuing": 0, 2024-09-11T23:39:55.587 INFO:teuthology.orchestra.run.smithi026.stdout: "strays_created": 0, 2024-09-11T23:39:55.587 INFO:teuthology.orchestra.run.smithi026.stdout: "strays_enqueued": 0, 2024-09-11T23:39:55.587 INFO:teuthology.orchestra.run.smithi026.stdout: "strays_reintegrated": 0, 2024-09-11T23:39:55.587 INFO:teuthology.orchestra.run.smithi026.stdout: "strays_migrated": 0, 2024-09-11T23:39:55.587 INFO:teuthology.orchestra.run.smithi026.stdout: "num_recovering_processing": 0, 2024-09-11T23:39:55.587 INFO:teuthology.orchestra.run.smithi026.stdout: "num_recovering_enqueued": 0, 2024-09-11T23:39:55.587 INFO:teuthology.orchestra.run.smithi026.stdout: "num_recovering_prioritized": 0, 2024-09-11T23:39:55.587 INFO:teuthology.orchestra.run.smithi026.stdout: "recovery_started": 0, 2024-09-11T23:39:55.587 INFO:teuthology.orchestra.run.smithi026.stdout: "recovery_completed": 0, 2024-09-11T23:39:55.587 INFO:teuthology.orchestra.run.smithi026.stdout: "ireq_quiesce_path": 0, 2024-09-11T23:39:55.587 INFO:teuthology.orchestra.run.smithi026.stdout: "ireq_quiesce_inode": 0, 2024-09-11T23:39:55.588 INFO:teuthology.orchestra.run.smithi026.stdout: "ireq_enqueue_scrub": 0, 2024-09-11T23:39:55.588 INFO:teuthology.orchestra.run.smithi026.stdout: "ireq_exportdir": 0, 2024-09-11T23:39:55.588 INFO:teuthology.orchestra.run.smithi026.stdout: "ireq_flush": 0, 2024-09-11T23:39:55.588 INFO:teuthology.orchestra.run.smithi026.stdout: "ireq_fragmentdir": 0, 2024-09-11T23:39:55.588 INFO:teuthology.orchestra.run.smithi026.stdout: "ireq_fragstats": 0, 2024-09-11T23:39:55.588 INFO:teuthology.orchestra.run.smithi026.stdout: "ireq_inodestats": 0 2024-09-11T23:39:55.588 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:55.588 INFO:teuthology.orchestra.run.smithi026.stdout: "mds_log": { 2024-09-11T23:39:55.588 INFO:teuthology.orchestra.run.smithi026.stdout: "evlrg": 0, 2024-09-11T23:39:55.588 INFO:teuthology.orchestra.run.smithi026.stdout: "evadd": 0, 2024-09-11T23:39:55.588 INFO:teuthology.orchestra.run.smithi026.stdout: "evex": 0, 2024-09-11T23:39:55.588 INFO:teuthology.orchestra.run.smithi026.stdout: "evtrm": 0, 2024-09-11T23:39:55.588 INFO:teuthology.orchestra.run.smithi026.stdout: "ev": 82347, 2024-09-11T23:39:55.588 INFO:teuthology.orchestra.run.smithi026.stdout: "evexg": 0, 2024-09-11T23:39:55.589 INFO:teuthology.orchestra.run.smithi026.stdout: "evexd": 0, 2024-09-11T23:39:55.589 INFO:teuthology.orchestra.run.smithi026.stdout: "segadd": 0, 2024-09-11T23:39:55.589 INFO:teuthology.orchestra.run.smithi026.stdout: "segex": 0, 2024-09-11T23:39:55.589 INFO:teuthology.orchestra.run.smithi026.stdout: "segtrm": 0, 2024-09-11T23:39:55.589 INFO:teuthology.orchestra.run.smithi026.stdout: "seg": 112, 2024-09-11T23:39:55.589 INFO:teuthology.orchestra.run.smithi026.stdout: "segmjr": 8, 2024-09-11T23:39:55.589 INFO:teuthology.orchestra.run.smithi026.stdout: "segexg": 0, 2024-09-11T23:39:55.589 INFO:teuthology.orchestra.run.smithi026.stdout: "segexd": 0, 2024-09-11T23:39:55.589 INFO:teuthology.orchestra.run.smithi026.stdout: "expos": 4194304, 2024-09-11T23:39:55.589 INFO:teuthology.orchestra.run.smithi026.stdout: "wrpos": 257645128, 2024-09-11T23:39:55.589 INFO:teuthology.orchestra.run.smithi026.stdout: "rdpos": 257638783, 2024-09-11T23:39:55.589 INFO:teuthology.orchestra.run.smithi026.stdout: "jlat": { 2024-09-11T23:39:55.589 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-11T23:39:55.589 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-11T23:39:55.590 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-11T23:39:55.590 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:55.590 INFO:teuthology.orchestra.run.smithi026.stdout: "replayed": 82347 2024-09-11T23:39:55.590 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:55.590 INFO:teuthology.orchestra.run.smithi026.stdout: "mds_mem": { 2024-09-11T23:39:55.590 INFO:teuthology.orchestra.run.smithi026.stdout: "ino": 12800, 2024-09-11T23:39:55.590 INFO:teuthology.orchestra.run.smithi026.stdout: "ino+": 41593, 2024-09-11T23:39:55.590 INFO:teuthology.orchestra.run.smithi026.stdout: "ino-": 28793, 2024-09-11T23:39:55.590 INFO:teuthology.orchestra.run.smithi026.stdout: "dir": 431, 2024-09-11T23:39:55.590 INFO:teuthology.orchestra.run.smithi026.stdout: "dir+": 1042, 2024-09-11T23:39:55.590 INFO:teuthology.orchestra.run.smithi026.stdout: "dir-": 611, 2024-09-11T23:39:55.590 INFO:teuthology.orchestra.run.smithi026.stdout: "dn": 43912, 2024-09-11T23:39:55.590 INFO:teuthology.orchestra.run.smithi026.stdout: "dn+": 70421, 2024-09-11T23:39:55.590 INFO:teuthology.orchestra.run.smithi026.stdout: "dn-": 26509, 2024-09-11T23:39:55.590 INFO:teuthology.orchestra.run.smithi026.stdout: "cap": 0, 2024-09-11T23:39:55.591 INFO:teuthology.orchestra.run.smithi026.stdout: "cap+": 0, 2024-09-11T23:39:55.591 INFO:teuthology.orchestra.run.smithi026.stdout: "cap-": 0, 2024-09-11T23:39:55.591 INFO:teuthology.orchestra.run.smithi026.stdout: "rss": 135568, 2024-09-11T23:39:55.591 INFO:teuthology.orchestra.run.smithi026.stdout: "heap": 215328 2024-09-11T23:39:55.591 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:55.591 INFO:teuthology.orchestra.run.smithi026.stdout: "mds_server": { 2024-09-11T23:39:55.591 INFO:teuthology.orchestra.run.smithi026.stdout: "dispatch_client_request": 0, 2024-09-11T23:39:55.591 INFO:teuthology.orchestra.run.smithi026.stdout: "dispatch_server_request": 0, 2024-09-11T23:39:55.591 INFO:teuthology.orchestra.run.smithi026.stdout: "handle_client_request": 0, 2024-09-11T23:39:55.591 INFO:teuthology.orchestra.run.smithi026.stdout: "handle_client_session": 0, 2024-09-11T23:39:55.591 INFO:teuthology.orchestra.run.smithi026.stdout: "handle_peer_request": 0, 2024-09-11T23:39:55.591 INFO:teuthology.orchestra.run.smithi026.stdout: "req_create_latency": { 2024-09-11T23:39:55.591 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-11T23:39:55.591 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-11T23:39:55.592 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-11T23:39:55.592 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:55.592 INFO:teuthology.orchestra.run.smithi026.stdout: "req_getattr_latency": { 2024-09-11T23:39:55.592 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-11T23:39:55.592 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-11T23:39:55.592 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-11T23:39:55.592 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:55.592 INFO:teuthology.orchestra.run.smithi026.stdout: "req_getfilelock_latency": { 2024-09-11T23:39:55.592 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-11T23:39:55.592 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-11T23:39:55.592 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-11T23:39:55.592 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:55.592 INFO:teuthology.orchestra.run.smithi026.stdout: "req_link_latency": { 2024-09-11T23:39:55.592 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-11T23:39:55.593 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-11T23:39:55.593 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-11T23:39:55.593 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:55.593 INFO:teuthology.orchestra.run.smithi026.stdout: "req_lookup_latency": { 2024-09-11T23:39:55.593 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-11T23:39:55.593 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-11T23:39:55.593 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-11T23:39:55.593 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:55.593 INFO:teuthology.orchestra.run.smithi026.stdout: "req_lookuphash_latency": { 2024-09-11T23:39:55.593 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-11T23:39:55.593 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-11T23:39:55.593 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-11T23:39:55.593 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:55.593 INFO:teuthology.orchestra.run.smithi026.stdout: "req_lookupino_latency": { 2024-09-11T23:39:55.594 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-11T23:39:55.594 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-11T23:39:55.594 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-11T23:39:55.594 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:55.594 INFO:teuthology.orchestra.run.smithi026.stdout: "req_lookupname_latency": { 2024-09-11T23:39:55.594 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-11T23:39:55.594 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-11T23:39:55.594 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-11T23:39:55.594 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:55.594 INFO:teuthology.orchestra.run.smithi026.stdout: "req_lookupparent_latency": { 2024-09-11T23:39:55.594 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-11T23:39:55.594 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-11T23:39:55.594 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-11T23:39:55.594 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:55.595 INFO:teuthology.orchestra.run.smithi026.stdout: "req_lookupsnap_latency": { 2024-09-11T23:39:55.595 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-11T23:39:55.595 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-11T23:39:55.595 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-11T23:39:55.595 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:55.595 INFO:teuthology.orchestra.run.smithi026.stdout: "req_lssnap_latency": { 2024-09-11T23:39:55.595 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-11T23:39:55.595 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-11T23:39:55.595 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-11T23:39:55.595 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:55.596 INFO:teuthology.orchestra.run.smithi026.stdout: "req_mkdir_latency": { 2024-09-11T23:39:55.596 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-11T23:39:55.596 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-11T23:39:55.596 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-11T23:39:55.596 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:55.596 INFO:teuthology.orchestra.run.smithi026.stdout: "req_mknod_latency": { 2024-09-11T23:39:55.596 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-11T23:39:55.596 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-11T23:39:55.596 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-11T23:39:55.597 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:55.597 INFO:teuthology.orchestra.run.smithi026.stdout: "req_mksnap_latency": { 2024-09-11T23:39:55.597 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-11T23:39:55.597 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-11T23:39:55.597 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-11T23:39:55.597 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:55.597 INFO:teuthology.orchestra.run.smithi026.stdout: "req_open_latency": { 2024-09-11T23:39:55.597 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-11T23:39:55.597 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-11T23:39:55.597 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-11T23:39:55.597 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:55.597 INFO:teuthology.orchestra.run.smithi026.stdout: "req_readdir_latency": { 2024-09-11T23:39:55.597 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-11T23:39:55.598 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-11T23:39:55.598 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-11T23:39:55.598 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:55.598 INFO:teuthology.orchestra.run.smithi026.stdout: "req_rename_latency": { 2024-09-11T23:39:55.598 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-11T23:39:55.598 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-11T23:39:55.598 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-11T23:39:55.598 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:55.598 INFO:teuthology.orchestra.run.smithi026.stdout: "req_renamesnap_latency": { 2024-09-11T23:39:55.598 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-11T23:39:55.598 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-11T23:39:55.598 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-11T23:39:55.598 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:55.598 INFO:teuthology.orchestra.run.smithi026.stdout: "req_snapdiff_latency": { 2024-09-11T23:39:55.599 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-11T23:39:55.599 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-11T23:39:55.599 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-11T23:39:55.599 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:55.599 INFO:teuthology.orchestra.run.smithi026.stdout: "req_rmdir_latency": { 2024-09-11T23:39:55.599 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-11T23:39:55.599 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-11T23:39:55.599 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-11T23:39:55.599 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:55.599 INFO:teuthology.orchestra.run.smithi026.stdout: "req_rmsnap_latency": { 2024-09-11T23:39:55.599 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-11T23:39:55.599 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-11T23:39:55.599 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-11T23:39:55.599 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:55.600 INFO:teuthology.orchestra.run.smithi026.stdout: "req_rmxattr_latency": { 2024-09-11T23:39:55.600 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-11T23:39:55.600 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-11T23:39:55.600 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-11T23:39:55.600 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:55.600 INFO:teuthology.orchestra.run.smithi026.stdout: "req_setattr_latency": { 2024-09-11T23:39:55.600 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-11T23:39:55.600 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-11T23:39:55.600 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-11T23:39:55.600 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:55.600 INFO:teuthology.orchestra.run.smithi026.stdout: "req_setdirlayout_latency": { 2024-09-11T23:39:55.600 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-11T23:39:55.600 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-11T23:39:55.601 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-11T23:39:55.601 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:55.601 INFO:teuthology.orchestra.run.smithi026.stdout: "req_setfilelock_latency": { 2024-09-11T23:39:55.601 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-11T23:39:55.601 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-11T23:39:55.601 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-11T23:39:55.601 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:55.601 INFO:teuthology.orchestra.run.smithi026.stdout: "req_setlayout_latency": { 2024-09-11T23:39:55.601 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-11T23:39:55.601 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-11T23:39:55.601 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-11T23:39:55.601 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:55.601 INFO:teuthology.orchestra.run.smithi026.stdout: "req_setxattr_latency": { 2024-09-11T23:39:55.601 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-11T23:39:55.602 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-11T23:39:55.602 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-11T23:39:55.602 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:55.602 INFO:teuthology.orchestra.run.smithi026.stdout: "req_symlink_latency": { 2024-09-11T23:39:55.602 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-11T23:39:55.602 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-11T23:39:55.602 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-11T23:39:55.602 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:55.602 INFO:teuthology.orchestra.run.smithi026.stdout: "req_unlink_latency": { 2024-09-11T23:39:55.602 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-11T23:39:55.602 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-11T23:39:55.602 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-11T23:39:55.602 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:55.602 INFO:teuthology.orchestra.run.smithi026.stdout: "cap_revoke_eviction": 0, 2024-09-11T23:39:55.603 INFO:teuthology.orchestra.run.smithi026.stdout: "cap_acquisition_throttle": 0, 2024-09-11T23:39:55.603 INFO:teuthology.orchestra.run.smithi026.stdout: "req_getvxattr_latency": { 2024-09-11T23:39:55.603 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-11T23:39:55.603 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-11T23:39:55.603 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-11T23:39:55.603 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:55.603 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:55.603 INFO:teuthology.orchestra.run.smithi026.stdout: "mds_sessions": { 2024-09-11T23:39:55.603 INFO:teuthology.orchestra.run.smithi026.stdout: "session_count": 2, 2024-09-11T23:39:55.603 INFO:teuthology.orchestra.run.smithi026.stdout: "session_add": 3, 2024-09-11T23:39:55.603 INFO:teuthology.orchestra.run.smithi026.stdout: "session_remove": 1, 2024-09-11T23:39:55.603 INFO:teuthology.orchestra.run.smithi026.stdout: "sessions_open": 2, 2024-09-11T23:39:55.603 INFO:teuthology.orchestra.run.smithi026.stdout: "sessions_stale": 0, 2024-09-11T23:39:55.603 INFO:teuthology.orchestra.run.smithi026.stdout: "total_load": 0, 2024-09-11T23:39:55.604 INFO:teuthology.orchestra.run.smithi026.stdout: "average_load": 0, 2024-09-11T23:39:55.604 INFO:teuthology.orchestra.run.smithi026.stdout: "avg_session_uptime": 2295, 2024-09-11T23:39:55.604 INFO:teuthology.orchestra.run.smithi026.stdout: "mdthresh_evicted": 0 2024-09-11T23:39:55.604 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:55.604 INFO:teuthology.orchestra.run.smithi026.stdout: "mempool": { 2024-09-11T23:39:55.604 INFO:teuthology.orchestra.run.smithi026.stdout: "bloom_filter_bytes": 0, 2024-09-11T23:39:55.604 INFO:teuthology.orchestra.run.smithi026.stdout: "bloom_filter_items": 0, 2024-09-11T23:39:55.604 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_alloc_bytes": 0, 2024-09-11T23:39:55.604 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_alloc_items": 0, 2024-09-11T23:39:55.604 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_data_bytes": 0, 2024-09-11T23:39:55.604 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_data_items": 0, 2024-09-11T23:39:55.604 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-11T23:39:55.604 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_onode_items": 0, 2024-09-11T23:39:55.604 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-11T23:39:55.605 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_meta_items": 0, 2024-09-11T23:39:55.605 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_other_bytes": 0, 2024-09-11T23:39:55.605 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_other_items": 0, 2024-09-11T23:39:55.605 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-11T23:39:55.605 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_buffer_items": 0, 2024-09-11T23:39:55.605 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_extent_bytes": 0, 2024-09-11T23:39:55.605 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_extent_items": 0, 2024-09-11T23:39:55.605 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_blob_bytes": 0, 2024-09-11T23:39:55.605 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_blob_items": 0, 2024-09-11T23:39:55.605 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-11T23:39:55.605 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_shared_blob_items": 0, 2024-09-11T23:39:55.605 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-11T23:39:55.605 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_inline_bl_items": 0, 2024-09-11T23:39:55.605 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_fsck_bytes": 0, 2024-09-11T23:39:55.606 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_fsck_items": 0, 2024-09-11T23:39:55.606 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_txc_bytes": 0, 2024-09-11T23:39:55.606 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_txc_items": 0, 2024-09-11T23:39:55.606 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-11T23:39:55.606 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_writing_deferred_items": 0, 2024-09-11T23:39:55.606 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_writing_bytes": 0, 2024-09-11T23:39:55.606 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_writing_items": 0, 2024-09-11T23:39:55.606 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_bytes": 0, 2024-09-11T23:39:55.606 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_items": 0, 2024-09-11T23:39:55.606 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_file_reader_bytes": 0, 2024-09-11T23:39:55.606 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_file_reader_items": 0, 2024-09-11T23:39:55.606 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_file_writer_bytes": 0, 2024-09-11T23:39:55.606 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_file_writer_items": 0, 2024-09-11T23:39:55.606 INFO:teuthology.orchestra.run.smithi026.stdout: "buffer_anon_bytes": 721922, 2024-09-11T23:39:55.607 INFO:teuthology.orchestra.run.smithi026.stdout: "buffer_anon_items": 13080, 2024-09-11T23:39:55.607 INFO:teuthology.orchestra.run.smithi026.stdout: "buffer_meta_bytes": 0, 2024-09-11T23:39:55.607 INFO:teuthology.orchestra.run.smithi026.stdout: "buffer_meta_items": 0, 2024-09-11T23:39:55.607 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_bytes": 0, 2024-09-11T23:39:55.607 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_items": 0, 2024-09-11T23:39:55.607 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_mapbl_bytes": 0, 2024-09-11T23:39:55.607 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_mapbl_items": 0, 2024-09-11T23:39:55.607 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_pglog_bytes": 0, 2024-09-11T23:39:55.607 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_pglog_items": 0, 2024-09-11T23:39:55.607 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_bytes": 8344, 2024-09-11T23:39:55.607 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_items": 184, 2024-09-11T23:39:55.607 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_mapping_bytes": 0, 2024-09-11T23:39:55.607 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_mapping_items": 0, 2024-09-11T23:39:55.607 INFO:teuthology.orchestra.run.smithi026.stdout: "pgmap_bytes": 0, 2024-09-11T23:39:55.608 INFO:teuthology.orchestra.run.smithi026.stdout: "pgmap_items": 0, 2024-09-11T23:39:55.608 INFO:teuthology.orchestra.run.smithi026.stdout: "mds_co_bytes": 61124716, 2024-09-11T23:39:55.608 INFO:teuthology.orchestra.run.smithi026.stdout: "mds_co_items": 609795, 2024-09-11T23:39:55.608 INFO:teuthology.orchestra.run.smithi026.stdout: "unittest_1_bytes": 0, 2024-09-11T23:39:55.608 INFO:teuthology.orchestra.run.smithi026.stdout: "unittest_1_items": 0, 2024-09-11T23:39:55.608 INFO:teuthology.orchestra.run.smithi026.stdout: "unittest_2_bytes": 0, 2024-09-11T23:39:55.608 INFO:teuthology.orchestra.run.smithi026.stdout: "unittest_2_items": 0 2024-09-11T23:39:55.608 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:55.608 INFO:teuthology.orchestra.run.smithi026.stdout: "objecter": { 2024-09-11T23:39:55.608 INFO:teuthology.orchestra.run.smithi026.stdout: "op_active": 0, 2024-09-11T23:39:55.608 INFO:teuthology.orchestra.run.smithi026.stdout: "op_laggy": 0, 2024-09-11T23:39:55.608 INFO:teuthology.orchestra.run.smithi026.stdout: "op_send": 4642, 2024-09-11T23:39:55.608 INFO:teuthology.orchestra.run.smithi026.stdout: "op_send_bytes": 21, 2024-09-11T23:39:55.608 INFO:teuthology.orchestra.run.smithi026.stdout: "op_resend": 3, 2024-09-11T23:39:55.609 INFO:teuthology.orchestra.run.smithi026.stdout: "op_reply": 4642, 2024-09-11T23:39:55.609 INFO:teuthology.orchestra.run.smithi026.stdout: "op_latency": { 2024-09-11T23:39:55.609 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 4642, 2024-09-11T23:39:55.609 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 18.049718482, 2024-09-11T23:39:55.609 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.003888349 2024-09-11T23:39:55.609 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:55.609 INFO:teuthology.orchestra.run.smithi026.stdout: "op_inflight": 0, 2024-09-11T23:39:55.609 INFO:teuthology.orchestra.run.smithi026.stdout: "oplen_avg": { 2024-09-11T23:39:55.609 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 4642, 2024-09-11T23:39:55.609 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 4643 2024-09-11T23:39:55.609 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:55.609 INFO:teuthology.orchestra.run.smithi026.stdout: "op": 4642, 2024-09-11T23:39:55.609 INFO:teuthology.orchestra.run.smithi026.stdout: "op_r": 4642, 2024-09-11T23:39:55.610 INFO:teuthology.orchestra.run.smithi026.stdout: "op_w": 0, 2024-09-11T23:39:55.610 INFO:teuthology.orchestra.run.smithi026.stdout: "op_rmw": 0, 2024-09-11T23:39:55.610 INFO:teuthology.orchestra.run.smithi026.stdout: "op_pg": 0, 2024-09-11T23:39:55.610 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_stat": 2848, 2024-09-11T23:39:55.610 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_create": 0, 2024-09-11T23:39:55.610 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_read": 1792, 2024-09-11T23:39:55.610 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_write": 0, 2024-09-11T23:39:55.610 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_writefull": 0, 2024-09-11T23:39:55.610 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_writesame": 0, 2024-09-11T23:39:55.610 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_append": 0, 2024-09-11T23:39:55.610 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_zero": 0, 2024-09-11T23:39:55.611 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_truncate": 0, 2024-09-11T23:39:55.611 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_delete": 0, 2024-09-11T23:39:55.611 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_mapext": 0, 2024-09-11T23:39:55.611 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_sparse_read": 0, 2024-09-11T23:39:55.611 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_clonerange": 0, 2024-09-11T23:39:55.611 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_getxattr": 1, 2024-09-11T23:39:55.611 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_setxattr": 0, 2024-09-11T23:39:55.611 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_cmpxattr": 0, 2024-09-11T23:39:55.611 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_rmxattr": 0, 2024-09-11T23:39:55.611 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_resetxattrs": 0, 2024-09-11T23:39:55.611 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_call": 0, 2024-09-11T23:39:55.611 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_watch": 0, 2024-09-11T23:39:55.611 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_notify": 0, 2024-09-11T23:39:55.611 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_src_cmpxattr": 0, 2024-09-11T23:39:55.612 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_pgls": 0, 2024-09-11T23:39:55.612 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_pgls_filter": 0, 2024-09-11T23:39:55.612 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_other": 0, 2024-09-11T23:39:55.612 INFO:teuthology.orchestra.run.smithi026.stdout: "linger_active": 0, 2024-09-11T23:39:55.612 INFO:teuthology.orchestra.run.smithi026.stdout: "linger_send": 0, 2024-09-11T23:39:55.612 INFO:teuthology.orchestra.run.smithi026.stdout: "linger_resend": 0, 2024-09-11T23:39:55.612 INFO:teuthology.orchestra.run.smithi026.stdout: "linger_ping": 0, 2024-09-11T23:39:55.612 INFO:teuthology.orchestra.run.smithi026.stdout: "poolop_active": 0, 2024-09-11T23:39:55.612 INFO:teuthology.orchestra.run.smithi026.stdout: "poolop_send": 0, 2024-09-11T23:39:55.612 INFO:teuthology.orchestra.run.smithi026.stdout: "poolop_resend": 0, 2024-09-11T23:39:55.612 INFO:teuthology.orchestra.run.smithi026.stdout: "poolstat_active": 0, 2024-09-11T23:39:55.612 INFO:teuthology.orchestra.run.smithi026.stdout: "poolstat_send": 0, 2024-09-11T23:39:55.612 INFO:teuthology.orchestra.run.smithi026.stdout: "poolstat_resend": 0, 2024-09-11T23:39:55.613 INFO:teuthology.orchestra.run.smithi026.stdout: "statfs_active": 0, 2024-09-11T23:39:55.613 INFO:teuthology.orchestra.run.smithi026.stdout: "statfs_send": 0, 2024-09-11T23:39:55.613 INFO:teuthology.orchestra.run.smithi026.stdout: "statfs_resend": 0, 2024-09-11T23:39:55.613 INFO:teuthology.orchestra.run.smithi026.stdout: "command_active": 0, 2024-09-11T23:39:55.613 INFO:teuthology.orchestra.run.smithi026.stdout: "command_send": 0, 2024-09-11T23:39:55.613 INFO:teuthology.orchestra.run.smithi026.stdout: "command_resend": 0, 2024-09-11T23:39:55.613 INFO:teuthology.orchestra.run.smithi026.stdout: "map_epoch": 115, 2024-09-11T23:39:55.613 INFO:teuthology.orchestra.run.smithi026.stdout: "map_full": 0, 2024-09-11T23:39:55.613 INFO:teuthology.orchestra.run.smithi026.stdout: "map_inc": 30, 2024-09-11T23:39:55.613 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_sessions": 11, 2024-09-11T23:39:55.613 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_session_open": 11, 2024-09-11T23:39:55.613 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_session_close": 0, 2024-09-11T23:39:55.613 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_laggy": 0, 2024-09-11T23:39:55.613 INFO:teuthology.orchestra.run.smithi026.stdout: "omap_wr": 0, 2024-09-11T23:39:55.613 INFO:teuthology.orchestra.run.smithi026.stdout: "omap_rd": 2, 2024-09-11T23:39:55.614 INFO:teuthology.orchestra.run.smithi026.stdout: "omap_del": 0 2024-09-11T23:39:55.614 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:55.614 INFO:teuthology.orchestra.run.smithi026.stdout: "oft": { 2024-09-11T23:39:55.614 INFO:teuthology.orchestra.run.smithi026.stdout: "omap_total_objs": 0, 2024-09-11T23:39:55.614 INFO:teuthology.orchestra.run.smithi026.stdout: "omap_total_kv_pairs": 0, 2024-09-11T23:39:55.614 INFO:teuthology.orchestra.run.smithi026.stdout: "omap_total_updates": 0, 2024-09-11T23:39:55.614 INFO:teuthology.orchestra.run.smithi026.stdout: "omap_total_removes": 0 2024-09-11T23:39:55.614 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:55.614 INFO:teuthology.orchestra.run.smithi026.stdout: "osd-slow-ops": { 2024-09-11T23:39:55.614 INFO:teuthology.orchestra.run.smithi026.stdout: "slow_ops_count": 0 2024-09-11T23:39:55.614 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:55.614 INFO:teuthology.orchestra.run.smithi026.stdout: "purge_queue": { 2024-09-11T23:39:55.614 INFO:teuthology.orchestra.run.smithi026.stdout: "pq_executing_ops": 0, 2024-09-11T23:39:55.614 INFO:teuthology.orchestra.run.smithi026.stdout: "pq_executing_ops_high_water": 0, 2024-09-11T23:39:55.615 INFO:teuthology.orchestra.run.smithi026.stdout: "pq_executing": 0, 2024-09-11T23:39:55.615 INFO:teuthology.orchestra.run.smithi026.stdout: "pq_executing_high_water": 0, 2024-09-11T23:39:55.615 INFO:teuthology.orchestra.run.smithi026.stdout: "pq_executed_ops": 0, 2024-09-11T23:39:55.615 INFO:teuthology.orchestra.run.smithi026.stdout: "pq_executed": 0, 2024-09-11T23:39:55.615 INFO:teuthology.orchestra.run.smithi026.stdout: "pq_item_in_journal": 0 2024-09-11T23:39:55.615 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:55.615 INFO:teuthology.orchestra.run.smithi026.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-11T23:39:55.615 INFO:teuthology.orchestra.run.smithi026.stdout: "val": 0, 2024-09-11T23:39:55.615 INFO:teuthology.orchestra.run.smithi026.stdout: "max": 104857600, 2024-09-11T23:39:55.615 INFO:teuthology.orchestra.run.smithi026.stdout: "get_started": 0, 2024-09-11T23:39:55.615 INFO:teuthology.orchestra.run.smithi026.stdout: "get": 5132, 2024-09-11T23:39:55.615 INFO:teuthology.orchestra.run.smithi026.stdout: "get_sum": 256390412, 2024-09-11T23:39:55.615 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_fail": 0, 2024-09-11T23:39:55.615 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_success": 5132, 2024-09-11T23:39:55.616 INFO:teuthology.orchestra.run.smithi026.stdout: "take": 0, 2024-09-11T23:39:55.616 INFO:teuthology.orchestra.run.smithi026.stdout: "take_sum": 0, 2024-09-11T23:39:55.616 INFO:teuthology.orchestra.run.smithi026.stdout: "put": 5132, 2024-09-11T23:39:55.616 INFO:teuthology.orchestra.run.smithi026.stdout: "put_sum": 256390412, 2024-09-11T23:39:55.616 INFO:teuthology.orchestra.run.smithi026.stdout: "wait": { 2024-09-11T23:39:55.616 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-11T23:39:55.616 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-11T23:39:55.616 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-11T23:39:55.616 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:55.616 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:55.616 INFO:teuthology.orchestra.run.smithi026.stdout: "throttle-objecter_bytes": { 2024-09-11T23:39:55.616 INFO:teuthology.orchestra.run.smithi026.stdout: "val": 0, 2024-09-11T23:39:55.616 INFO:teuthology.orchestra.run.smithi026.stdout: "max": 104857600, 2024-09-11T23:39:55.616 INFO:teuthology.orchestra.run.smithi026.stdout: "get_started": 0, 2024-09-11T23:39:55.617 INFO:teuthology.orchestra.run.smithi026.stdout: "get": 0, 2024-09-11T23:39:55.617 INFO:teuthology.orchestra.run.smithi026.stdout: "get_sum": 0, 2024-09-11T23:39:55.617 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_fail": 0, 2024-09-11T23:39:55.617 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_success": 0, 2024-09-11T23:39:55.617 INFO:teuthology.orchestra.run.smithi026.stdout: "take": 4642, 2024-09-11T23:39:55.617 INFO:teuthology.orchestra.run.smithi026.stdout: "take_sum": 253485332, 2024-09-11T23:39:55.617 INFO:teuthology.orchestra.run.smithi026.stdout: "put": 4642, 2024-09-11T23:39:55.617 INFO:teuthology.orchestra.run.smithi026.stdout: "put_sum": 253485332, 2024-09-11T23:39:55.617 INFO:teuthology.orchestra.run.smithi026.stdout: "wait": { 2024-09-11T23:39:55.618 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-11T23:39:55.618 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-11T23:39:55.618 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-11T23:39:55.618 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:55.618 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:55.618 INFO:teuthology.orchestra.run.smithi026.stdout: "throttle-objecter_ops": { 2024-09-11T23:39:55.618 INFO:teuthology.orchestra.run.smithi026.stdout: "val": 0, 2024-09-11T23:39:55.618 INFO:teuthology.orchestra.run.smithi026.stdout: "max": 1024, 2024-09-11T23:39:55.618 INFO:teuthology.orchestra.run.smithi026.stdout: "get_started": 0, 2024-09-11T23:39:55.618 INFO:teuthology.orchestra.run.smithi026.stdout: "get": 0, 2024-09-11T23:39:55.618 INFO:teuthology.orchestra.run.smithi026.stdout: "get_sum": 0, 2024-09-11T23:39:55.619 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_fail": 0, 2024-09-11T23:39:55.619 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_success": 0, 2024-09-11T23:39:55.619 INFO:teuthology.orchestra.run.smithi026.stdout: "take": 4642, 2024-09-11T23:39:55.620 INFO:teuthology.orchestra.run.smithi026.stdout: "take_sum": 4642, 2024-09-11T23:39:55.620 INFO:teuthology.orchestra.run.smithi026.stdout: "put": 4642, 2024-09-11T23:39:55.620 INFO:teuthology.orchestra.run.smithi026.stdout: "put_sum": 4642, 2024-09-11T23:39:55.620 INFO:teuthology.orchestra.run.smithi026.stdout: "wait": { 2024-09-11T23:39:55.620 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-11T23:39:55.620 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-11T23:39:55.620 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-11T23:39:55.620 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:55.620 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:55.620 INFO:teuthology.orchestra.run.smithi026.stdout: "throttle-write_buf_throttle": { 2024-09-11T23:39:55.620 INFO:teuthology.orchestra.run.smithi026.stdout: "val": 0, 2024-09-11T23:39:55.620 INFO:teuthology.orchestra.run.smithi026.stdout: "max": 3758096384, 2024-09-11T23:39:55.620 INFO:teuthology.orchestra.run.smithi026.stdout: "get_started": 0, 2024-09-11T23:39:55.621 INFO:teuthology.orchestra.run.smithi026.stdout: "get": 0, 2024-09-11T23:39:55.621 INFO:teuthology.orchestra.run.smithi026.stdout: "get_sum": 0, 2024-09-11T23:39:55.621 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_fail": 0, 2024-09-11T23:39:55.621 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_success": 0, 2024-09-11T23:39:55.621 INFO:teuthology.orchestra.run.smithi026.stdout: "take": 0, 2024-09-11T23:39:55.621 INFO:teuthology.orchestra.run.smithi026.stdout: "take_sum": 0, 2024-09-11T23:39:55.621 INFO:teuthology.orchestra.run.smithi026.stdout: "put": 0, 2024-09-11T23:39:55.621 INFO:teuthology.orchestra.run.smithi026.stdout: "put_sum": 0, 2024-09-11T23:39:55.621 INFO:teuthology.orchestra.run.smithi026.stdout: "wait": { 2024-09-11T23:39:55.621 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-11T23:39:55.621 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-11T23:39:55.621 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-11T23:39:55.621 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:55.621 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:55.622 INFO:teuthology.orchestra.run.smithi026.stdout: "throttle-write_buf_throttle-0x5605c9f54b40": { 2024-09-11T23:39:55.622 INFO:teuthology.orchestra.run.smithi026.stdout: "val": 0, 2024-09-11T23:39:55.622 INFO:teuthology.orchestra.run.smithi026.stdout: "max": 3758096384, 2024-09-11T23:39:55.622 INFO:teuthology.orchestra.run.smithi026.stdout: "get_started": 0, 2024-09-11T23:39:55.622 INFO:teuthology.orchestra.run.smithi026.stdout: "get": 0, 2024-09-11T23:39:55.622 INFO:teuthology.orchestra.run.smithi026.stdout: "get_sum": 0, 2024-09-11T23:39:55.622 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_fail": 0, 2024-09-11T23:39:55.622 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_success": 0, 2024-09-11T23:39:55.622 INFO:teuthology.orchestra.run.smithi026.stdout: "take": 0, 2024-09-11T23:39:55.622 INFO:teuthology.orchestra.run.smithi026.stdout: "take_sum": 0, 2024-09-11T23:39:55.622 INFO:teuthology.orchestra.run.smithi026.stdout: "put": 0, 2024-09-11T23:39:55.622 INFO:teuthology.orchestra.run.smithi026.stdout: "put_sum": 0, 2024-09-11T23:39:55.622 INFO:teuthology.orchestra.run.smithi026.stdout: "wait": { 2024-09-11T23:39:55.622 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-11T23:39:55.623 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-11T23:39:55.623 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-11T23:39:55.623 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:55.623 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:55.623 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-11T23:39:56.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:39:56 smithi026 ceph-mon[31172]: pgmap v678: 97 pgs: 2 active+clean+snaptrim, 95 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.0 MiB/s wr, 340 op/s 2024-09-11T23:39:56.362 INFO:tasks.check_counter:Daemon mds.a mds.exported=0 2024-09-11T23:39:56.362 INFO:tasks.check_counter:Daemon mds.a mds.imported=0 2024-09-11T23:39:56.362 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update=0 2024-09-11T23:39:56.362 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update_receipt=0 2024-09-11T23:39:56.362 INFO:tasks.check_counter:Daemon mds.a mds.root_rsnaps=0 2024-09-11T23:39:56.362 INFO:tasks.check_counter:Daemon mds.a mds_server.req_mksnap_latency.avgcount=0 2024-09-11T23:39:56.362 INFO:tasks.check_counter:Daemon mds.a mds_server.req_rmsnap_latency.avgcount=0 2024-09-11T23:39:56.362 INFO:tasks.check_counter:Daemon mds.a mds.dir_split=0 2024-09-11T23:39:56.362 DEBUG:tasks.check_counter:Getting stats from d 2024-09-11T23:39:56.363 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph daemon mds.d perf dump 2024-09-11T23:39:56.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:39:56 smithi136 ceph-mon[35121]: pgmap v678: 97 pgs: 2 active+clean+snaptrim, 95 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.0 MiB/s wr, 340 op/s 2024-09-11T23:39:56.552 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:39:56 smithi076 ceph-mon[35296]: pgmap v678: 97 pgs: 2 active+clean+snaptrim, 95 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.0 MiB/s wr, 340 op/s 2024-09-11T23:39:56.622 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:39:57.134 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-11T23:39:57.134 INFO:teuthology.orchestra.run.smithi026.stdout: "AsyncMessenger::Worker-0": { 2024-09-11T23:39:57.134 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_messages": 0, 2024-09-11T23:39:57.134 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages": 2, 2024-09-11T23:39:57.134 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_bytes": 0, 2024-09-11T23:39:57.134 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_bytes": 0, 2024-09-11T23:39:57.134 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_created_connections": 4, 2024-09-11T23:39:57.134 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_active_connections": 0, 2024-09-11T23:39:57.135 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_total_time": 0.001303025, 2024-09-11T23:39:57.135 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_send_time": 0.000000000, 2024-09-11T23:39:57.135 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_recv_time": 0.000243510, 2024-09-11T23:39:57.135 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-11T23:39:57.135 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T23:39:57.135 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-11T23:39:57.135 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-11T23:39:57.135 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-11T23:39:57.135 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:57.135 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_handle_ack_lat": { 2024-09-11T23:39:57.135 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-11T23:39:57.135 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-11T23:39:57.135 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-11T23:39:57.135 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:57.136 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-11T23:39:57.136 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_encrypted_bytes": 0 2024-09-11T23:39:57.136 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:57.136 INFO:teuthology.orchestra.run.smithi026.stdout: "AsyncMessenger::Worker-1": { 2024-09-11T23:39:57.136 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_messages": 398, 2024-09-11T23:39:57.136 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages": 385, 2024-09-11T23:39:57.136 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_bytes": 1633376, 2024-09-11T23:39:57.136 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_bytes": 237360, 2024-09-11T23:39:57.136 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_created_connections": 2, 2024-09-11T23:39:57.136 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_active_connections": 1, 2024-09-11T23:39:57.136 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_total_time": 0.067160142, 2024-09-11T23:39:57.136 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_send_time": 0.026933892, 2024-09-11T23:39:57.136 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_recv_time": 0.053172406, 2024-09-11T23:39:57.136 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-11T23:39:57.137 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T23:39:57.137 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 385, 2024-09-11T23:39:57.137 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.026933657, 2024-09-11T23:39:57.137 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000069957 2024-09-11T23:39:57.137 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:57.137 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_handle_ack_lat": { 2024-09-11T23:39:57.137 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-11T23:39:57.137 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-11T23:39:57.137 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-11T23:39:57.137 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:57.137 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_encrypted_bytes": 1633376, 2024-09-11T23:39:57.137 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_encrypted_bytes": 237360 2024-09-11T23:39:57.137 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:57.137 INFO:teuthology.orchestra.run.smithi026.stdout: "AsyncMessenger::Worker-2": { 2024-09-11T23:39:57.138 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_messages": 3, 2024-09-11T23:39:57.138 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages": 294, 2024-09-11T23:39:57.138 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_bytes": 480, 2024-09-11T23:39:57.138 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_bytes": 287856, 2024-09-11T23:39:57.138 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_created_connections": 6, 2024-09-11T23:39:57.138 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_active_connections": 3, 2024-09-11T23:39:57.138 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_total_time": 0.021773138, 2024-09-11T23:39:57.138 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_send_time": 0.015783976, 2024-09-11T23:39:57.138 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_recv_time": 0.002267136, 2024-09-11T23:39:57.138 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-11T23:39:57.138 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T23:39:57.138 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 291, 2024-09-11T23:39:57.138 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.026713188, 2024-09-11T23:39:57.139 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000091797 2024-09-11T23:39:57.139 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:57.139 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_handle_ack_lat": { 2024-09-11T23:39:57.139 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-11T23:39:57.139 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-11T23:39:57.139 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-11T23:39:57.139 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:57.139 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_encrypted_bytes": 480, 2024-09-11T23:39:57.139 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_encrypted_bytes": 287856 2024-09-11T23:39:57.139 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:57.139 INFO:teuthology.orchestra.run.smithi026.stdout: "cct": { 2024-09-11T23:39:57.139 INFO:teuthology.orchestra.run.smithi026.stdout: "total_workers": 0, 2024-09-11T23:39:57.139 INFO:teuthology.orchestra.run.smithi026.stdout: "unhealthy_workers": 0 2024-09-11T23:39:57.139 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:57.140 INFO:teuthology.orchestra.run.smithi026.stdout: "mempool": { 2024-09-11T23:39:57.140 INFO:teuthology.orchestra.run.smithi026.stdout: "bloom_filter_bytes": 0, 2024-09-11T23:39:57.140 INFO:teuthology.orchestra.run.smithi026.stdout: "bloom_filter_items": 0, 2024-09-11T23:39:57.140 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_alloc_bytes": 0, 2024-09-11T23:39:57.140 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_alloc_items": 0, 2024-09-11T23:39:57.140 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_data_bytes": 0, 2024-09-11T23:39:57.140 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_data_items": 0, 2024-09-11T23:39:57.140 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-11T23:39:57.140 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_onode_items": 0, 2024-09-11T23:39:57.140 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-11T23:39:57.140 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_meta_items": 0, 2024-09-11T23:39:57.140 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_other_bytes": 0, 2024-09-11T23:39:57.140 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_other_items": 0, 2024-09-11T23:39:57.140 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-11T23:39:57.140 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_buffer_items": 0, 2024-09-11T23:39:57.141 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_extent_bytes": 0, 2024-09-11T23:39:57.141 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_extent_items": 0, 2024-09-11T23:39:57.141 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_blob_bytes": 0, 2024-09-11T23:39:57.141 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_blob_items": 0, 2024-09-11T23:39:57.141 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-11T23:39:57.141 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_shared_blob_items": 0, 2024-09-11T23:39:57.141 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-11T23:39:57.141 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_inline_bl_items": 0, 2024-09-11T23:39:57.141 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_fsck_bytes": 0, 2024-09-11T23:39:57.141 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_fsck_items": 0, 2024-09-11T23:39:57.141 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_txc_bytes": 0, 2024-09-11T23:39:57.141 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_txc_items": 0, 2024-09-11T23:39:57.141 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-11T23:39:57.141 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_writing_deferred_items": 0, 2024-09-11T23:39:57.142 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_writing_bytes": 0, 2024-09-11T23:39:57.142 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_writing_items": 0, 2024-09-11T23:39:57.142 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_bytes": 0, 2024-09-11T23:39:57.142 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_items": 0, 2024-09-11T23:39:57.142 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_file_reader_bytes": 0, 2024-09-11T23:39:57.142 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_file_reader_items": 0, 2024-09-11T23:39:57.142 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_file_writer_bytes": 0, 2024-09-11T23:39:57.142 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_file_writer_items": 0, 2024-09-11T23:39:57.142 INFO:teuthology.orchestra.run.smithi026.stdout: "buffer_anon_bytes": 148714, 2024-09-11T23:39:57.142 INFO:teuthology.orchestra.run.smithi026.stdout: "buffer_anon_items": 30, 2024-09-11T23:39:57.142 INFO:teuthology.orchestra.run.smithi026.stdout: "buffer_meta_bytes": 0, 2024-09-11T23:39:57.142 INFO:teuthology.orchestra.run.smithi026.stdout: "buffer_meta_items": 0, 2024-09-11T23:39:57.142 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_bytes": 0, 2024-09-11T23:39:57.142 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_items": 0, 2024-09-11T23:39:57.143 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_mapbl_bytes": 0, 2024-09-11T23:39:57.143 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_mapbl_items": 0, 2024-09-11T23:39:57.143 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_pglog_bytes": 0, 2024-09-11T23:39:57.143 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_pglog_items": 0, 2024-09-11T23:39:57.143 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_bytes": 0, 2024-09-11T23:39:57.143 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_items": 0, 2024-09-11T23:39:57.143 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_mapping_bytes": 0, 2024-09-11T23:39:57.143 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_mapping_items": 0, 2024-09-11T23:39:57.143 INFO:teuthology.orchestra.run.smithi026.stdout: "pgmap_bytes": 0, 2024-09-11T23:39:57.143 INFO:teuthology.orchestra.run.smithi026.stdout: "pgmap_items": 0, 2024-09-11T23:39:57.143 INFO:teuthology.orchestra.run.smithi026.stdout: "mds_co_bytes": 928, 2024-09-11T23:39:57.143 INFO:teuthology.orchestra.run.smithi026.stdout: "mds_co_items": 2, 2024-09-11T23:39:57.143 INFO:teuthology.orchestra.run.smithi026.stdout: "unittest_1_bytes": 0, 2024-09-11T23:39:57.143 INFO:teuthology.orchestra.run.smithi026.stdout: "unittest_1_items": 0, 2024-09-11T23:39:57.144 INFO:teuthology.orchestra.run.smithi026.stdout: "unittest_2_bytes": 0, 2024-09-11T23:39:57.144 INFO:teuthology.orchestra.run.smithi026.stdout: "unittest_2_items": 0 2024-09-11T23:39:57.144 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:57.144 INFO:teuthology.orchestra.run.smithi026.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-11T23:39:57.144 INFO:teuthology.orchestra.run.smithi026.stdout: "val": 0, 2024-09-11T23:39:57.144 INFO:teuthology.orchestra.run.smithi026.stdout: "max": 104857600, 2024-09-11T23:39:57.144 INFO:teuthology.orchestra.run.smithi026.stdout: "get_started": 0, 2024-09-11T23:39:57.144 INFO:teuthology.orchestra.run.smithi026.stdout: "get": 397, 2024-09-11T23:39:57.144 INFO:teuthology.orchestra.run.smithi026.stdout: "get_sum": 1574729, 2024-09-11T23:39:57.144 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_fail": 0, 2024-09-11T23:39:57.144 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_success": 397, 2024-09-11T23:39:57.144 INFO:teuthology.orchestra.run.smithi026.stdout: "take": 0, 2024-09-11T23:39:57.144 INFO:teuthology.orchestra.run.smithi026.stdout: "take_sum": 0, 2024-09-11T23:39:57.144 INFO:teuthology.orchestra.run.smithi026.stdout: "put": 397, 2024-09-11T23:39:57.145 INFO:teuthology.orchestra.run.smithi026.stdout: "put_sum": 1574729, 2024-09-11T23:39:57.145 INFO:teuthology.orchestra.run.smithi026.stdout: "wait": { 2024-09-11T23:39:57.145 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-11T23:39:57.145 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-11T23:39:57.145 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-11T23:39:57.145 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:57.145 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:57.145 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-11T23:39:57.880 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-09-11T23:39:57.880 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-09-11T23:39:57.880 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.d 2024-09-11T23:39:57.881 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.d 2024-09-11T23:39:57.881 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2024-09-11T23:39:57.881 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2024-09-11T23:39:57.881 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2024-09-11T23:39:57.881 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.d 2024-09-11T23:39:57.881 DEBUG:tasks.check_counter:Getting stats from g 2024-09-11T23:39:57.881 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph daemon mds.g perf dump 2024-09-11T23:39:58.141 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:39:58.390 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:39:58 smithi026 ceph-mon[31172]: pgmap v679: 97 pgs: 97 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 711 KiB/s rd, 1.7 MiB/s wr, 332 op/s 2024-09-11T23:39:58.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:39:58 smithi136 ceph-mon[35121]: pgmap v679: 97 pgs: 97 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 711 KiB/s rd, 1.7 MiB/s wr, 332 op/s 2024-09-11T23:39:58.552 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:39:58 smithi076 ceph-mon[35296]: pgmap v679: 97 pgs: 97 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 711 KiB/s rd, 1.7 MiB/s wr, 332 op/s 2024-09-11T23:39:58.617 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-11T23:39:58.617 INFO:teuthology.orchestra.run.smithi026.stdout: "AsyncMessenger::Worker-0": { 2024-09-11T23:39:58.618 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_messages": 1872, 2024-09-11T23:39:58.618 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages": 1874, 2024-09-11T23:39:58.618 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_bytes": 130637288, 2024-09-11T23:39:58.618 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_bytes": 595297, 2024-09-11T23:39:58.618 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_created_connections": 8, 2024-09-11T23:39:58.618 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_active_connections": 4, 2024-09-11T23:39:58.618 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_total_time": 0.291428828, 2024-09-11T23:39:58.618 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_send_time": 0.075053929, 2024-09-11T23:39:58.618 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_recv_time": 0.140997625, 2024-09-11T23:39:58.618 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_fast_dispatch_time": 0.057444941, 2024-09-11T23:39:58.618 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T23:39:58.618 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 1872, 2024-09-11T23:39:58.618 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.134784697, 2024-09-11T23:39:58.619 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000072000 2024-09-11T23:39:58.619 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:58.619 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_handle_ack_lat": { 2024-09-11T23:39:58.619 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-11T23:39:58.619 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-11T23:39:58.619 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-11T23:39:58.619 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:58.619 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-11T23:39:58.619 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_encrypted_bytes": 0 2024-09-11T23:39:58.619 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:58.619 INFO:teuthology.orchestra.run.smithi026.stdout: "AsyncMessenger::Worker-1": { 2024-09-11T23:39:58.619 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_messages": 2720, 2024-09-11T23:39:58.619 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages": 2643, 2024-09-11T23:39:58.619 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_bytes": 103342268, 2024-09-11T23:39:58.619 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_bytes": 949459, 2024-09-11T23:39:58.620 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_created_connections": 6, 2024-09-11T23:39:58.620 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_active_connections": 5, 2024-09-11T23:39:58.620 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_total_time": 0.310754381, 2024-09-11T23:39:58.620 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_send_time": 0.071185519, 2024-09-11T23:39:58.620 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_recv_time": 0.178958725, 2024-09-11T23:39:58.620 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_fast_dispatch_time": 0.063378871, 2024-09-11T23:39:58.620 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T23:39:58.620 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 2643, 2024-09-11T23:39:58.620 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.104605852, 2024-09-11T23:39:58.620 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000039578 2024-09-11T23:39:58.620 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:58.620 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_handle_ack_lat": { 2024-09-11T23:39:58.620 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-11T23:39:58.621 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-11T23:39:58.621 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-11T23:39:58.621 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:58.621 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_encrypted_bytes": 2101248, 2024-09-11T23:39:58.621 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_encrypted_bytes": 241216 2024-09-11T23:39:58.621 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:58.621 INFO:teuthology.orchestra.run.smithi026.stdout: "AsyncMessenger::Worker-2": { 2024-09-11T23:39:58.621 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_messages": 569, 2024-09-11T23:39:58.621 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages": 858, 2024-09-11T23:39:58.621 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_bytes": 69464738, 2024-09-11T23:39:58.621 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_bytes": 947823, 2024-09-11T23:39:58.621 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_created_connections": 9, 2024-09-11T23:39:58.621 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_active_connections": 6, 2024-09-11T23:39:58.621 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_total_time": 0.117085926, 2024-09-11T23:39:58.622 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_send_time": 0.029416648, 2024-09-11T23:39:58.622 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_recv_time": 0.062738929, 2024-09-11T23:39:58.622 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_fast_dispatch_time": 0.016789416, 2024-09-11T23:39:58.622 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T23:39:58.622 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 855, 2024-09-11T23:39:58.622 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.067251198, 2024-09-11T23:39:58.622 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000078656 2024-09-11T23:39:58.622 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:58.622 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_handle_ack_lat": { 2024-09-11T23:39:58.622 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-11T23:39:58.622 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-11T23:39:58.622 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-11T23:39:58.622 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:58.622 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_encrypted_bytes": 480, 2024-09-11T23:39:58.623 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_encrypted_bytes": 767824 2024-09-11T23:39:58.623 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:58.623 INFO:teuthology.orchestra.run.smithi026.stdout: "cct": { 2024-09-11T23:39:58.623 INFO:teuthology.orchestra.run.smithi026.stdout: "total_workers": 1, 2024-09-11T23:39:58.623 INFO:teuthology.orchestra.run.smithi026.stdout: "unhealthy_workers": 0 2024-09-11T23:39:58.623 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:58.623 INFO:teuthology.orchestra.run.smithi026.stdout: "finisher-MDSRank": { 2024-09-11T23:39:58.623 INFO:teuthology.orchestra.run.smithi026.stdout: "queue_len": 0, 2024-09-11T23:39:58.623 INFO:teuthology.orchestra.run.smithi026.stdout: "complete_latency": { 2024-09-11T23:39:58.623 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 7642, 2024-09-11T23:39:58.623 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.210052768, 2024-09-11T23:39:58.623 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000027486 2024-09-11T23:39:58.624 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:58.624 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:58.624 INFO:teuthology.orchestra.run.smithi026.stdout: "finisher-PurgeQueue": { 2024-09-11T23:39:58.624 INFO:teuthology.orchestra.run.smithi026.stdout: "queue_len": 0, 2024-09-11T23:39:58.624 INFO:teuthology.orchestra.run.smithi026.stdout: "complete_latency": { 2024-09-11T23:39:58.624 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-11T23:39:58.624 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-11T23:39:58.624 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-11T23:39:58.624 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:58.624 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:58.624 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": { 2024-09-11T23:39:58.624 INFO:teuthology.orchestra.run.smithi026.stdout: "request": 0, 2024-09-11T23:39:58.624 INFO:teuthology.orchestra.run.smithi026.stdout: "reply": 0, 2024-09-11T23:39:58.624 INFO:teuthology.orchestra.run.smithi026.stdout: "reply_latency": { 2024-09-11T23:39:58.624 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-11T23:39:58.625 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-11T23:39:58.625 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-11T23:39:58.625 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:58.625 INFO:teuthology.orchestra.run.smithi026.stdout: "slow_reply": 0, 2024-09-11T23:39:58.625 INFO:teuthology.orchestra.run.smithi026.stdout: "forward": 0, 2024-09-11T23:39:58.625 INFO:teuthology.orchestra.run.smithi026.stdout: "dir_fetch_complete": 0, 2024-09-11T23:39:58.625 INFO:teuthology.orchestra.run.smithi026.stdout: "dir_fetch_keys": 0, 2024-09-11T23:39:58.625 INFO:teuthology.orchestra.run.smithi026.stdout: "dir_commit": 0, 2024-09-11T23:39:58.625 INFO:teuthology.orchestra.run.smithi026.stdout: "dir_split": 0, 2024-09-11T23:39:58.625 INFO:teuthology.orchestra.run.smithi026.stdout: "dir_merge": 0, 2024-09-11T23:39:58.625 INFO:teuthology.orchestra.run.smithi026.stdout: "inodes": 36287, 2024-09-11T23:39:58.625 INFO:teuthology.orchestra.run.smithi026.stdout: "inodes_top": 577, 2024-09-11T23:39:58.625 INFO:teuthology.orchestra.run.smithi026.stdout: "inodes_bottom": 35710, 2024-09-11T23:39:58.625 INFO:teuthology.orchestra.run.smithi026.stdout: "inodes_pin_tail": 0, 2024-09-11T23:39:58.626 INFO:teuthology.orchestra.run.smithi026.stdout: "inodes_pinned": 35462, 2024-09-11T23:39:58.626 INFO:teuthology.orchestra.run.smithi026.stdout: "inodes_expired": 0, 2024-09-11T23:39:58.626 INFO:teuthology.orchestra.run.smithi026.stdout: "inodes_with_caps": 0, 2024-09-11T23:39:58.626 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": 0, 2024-09-11T23:39:58.626 INFO:teuthology.orchestra.run.smithi026.stdout: "subtrees": 1, 2024-09-11T23:39:58.626 INFO:teuthology.orchestra.run.smithi026.stdout: "traverse": 0, 2024-09-11T23:39:58.626 INFO:teuthology.orchestra.run.smithi026.stdout: "traverse_hit": 0, 2024-09-11T23:39:58.626 INFO:teuthology.orchestra.run.smithi026.stdout: "traverse_forward": 0, 2024-09-11T23:39:58.626 INFO:teuthology.orchestra.run.smithi026.stdout: "traverse_discover": 0, 2024-09-11T23:39:58.626 INFO:teuthology.orchestra.run.smithi026.stdout: "traverse_dir_fetch": 0, 2024-09-11T23:39:58.626 INFO:teuthology.orchestra.run.smithi026.stdout: "traverse_remote_ino": 0, 2024-09-11T23:39:58.626 INFO:teuthology.orchestra.run.smithi026.stdout: "traverse_lock": 0, 2024-09-11T23:39:58.626 INFO:teuthology.orchestra.run.smithi026.stdout: "load_cent": 0, 2024-09-11T23:39:58.626 INFO:teuthology.orchestra.run.smithi026.stdout: "q": 0, 2024-09-11T23:39:58.627 INFO:teuthology.orchestra.run.smithi026.stdout: "exported": 0, 2024-09-11T23:39:58.627 INFO:teuthology.orchestra.run.smithi026.stdout: "exported_inodes": 0, 2024-09-11T23:39:58.627 INFO:teuthology.orchestra.run.smithi026.stdout: "imported": 0, 2024-09-11T23:39:58.627 INFO:teuthology.orchestra.run.smithi026.stdout: "imported_inodes": 0, 2024-09-11T23:39:58.627 INFO:teuthology.orchestra.run.smithi026.stdout: "openino_dir_fetch": 0, 2024-09-11T23:39:58.627 INFO:teuthology.orchestra.run.smithi026.stdout: "openino_backtrace_fetch": 0, 2024-09-11T23:39:58.627 INFO:teuthology.orchestra.run.smithi026.stdout: "openino_peer_discover": 0, 2024-09-11T23:39:58.627 INFO:teuthology.orchestra.run.smithi026.stdout: "root_rfiles": 0, 2024-09-11T23:39:58.627 INFO:teuthology.orchestra.run.smithi026.stdout: "root_rbytes": 0, 2024-09-11T23:39:58.627 INFO:teuthology.orchestra.run.smithi026.stdout: "root_rsnaps": 0, 2024-09-11T23:39:58.627 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_backtrace_fetch": 0, 2024-09-11T23:39:58.627 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_set_tag": 0, 2024-09-11T23:39:58.627 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_backtrace_repaired": 0, 2024-09-11T23:39:58.627 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_inotable_repaired": 0, 2024-09-11T23:39:58.628 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_dir_inodes": 0, 2024-09-11T23:39:58.628 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_dir_base_inodes": 0, 2024-09-11T23:39:58.628 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_dirfrag_rstats": 0, 2024-09-11T23:39:58.628 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_file_inodes": 0, 2024-09-11T23:39:58.628 INFO:teuthology.orchestra.run.smithi026.stdout: "handle_inode_file_caps": 0, 2024-09-11T23:39:58.628 INFO:teuthology.orchestra.run.smithi026.stdout: "ceph_cap_op_revoke": 0, 2024-09-11T23:39:58.628 INFO:teuthology.orchestra.run.smithi026.stdout: "ceph_cap_op_grant": 0, 2024-09-11T23:39:58.628 INFO:teuthology.orchestra.run.smithi026.stdout: "ceph_cap_op_trunc": 0, 2024-09-11T23:39:58.628 INFO:teuthology.orchestra.run.smithi026.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-11T23:39:58.628 INFO:teuthology.orchestra.run.smithi026.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-11T23:39:58.628 INFO:teuthology.orchestra.run.smithi026.stdout: "handle_client_caps": 0, 2024-09-11T23:39:58.628 INFO:teuthology.orchestra.run.smithi026.stdout: "handle_client_caps_dirty": 0, 2024-09-11T23:39:58.628 INFO:teuthology.orchestra.run.smithi026.stdout: "handle_client_cap_release": 0, 2024-09-11T23:39:58.628 INFO:teuthology.orchestra.run.smithi026.stdout: "process_request_cap_release": 0 2024-09-11T23:39:58.629 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:58.629 INFO:teuthology.orchestra.run.smithi026.stdout: "mds_cache": { 2024-09-11T23:39:58.629 INFO:teuthology.orchestra.run.smithi026.stdout: "dir_update": 0, 2024-09-11T23:39:58.629 INFO:teuthology.orchestra.run.smithi026.stdout: "dir_update_receipt": 0, 2024-09-11T23:39:58.629 INFO:teuthology.orchestra.run.smithi026.stdout: "dir_try_discover": 0, 2024-09-11T23:39:58.629 INFO:teuthology.orchestra.run.smithi026.stdout: "dir_send_discover": 0, 2024-09-11T23:39:58.629 INFO:teuthology.orchestra.run.smithi026.stdout: "dir_handle_discover": 0, 2024-09-11T23:39:58.629 INFO:teuthology.orchestra.run.smithi026.stdout: "num_strays": 0, 2024-09-11T23:39:58.629 INFO:teuthology.orchestra.run.smithi026.stdout: "num_strays_delayed": 0, 2024-09-11T23:39:58.629 INFO:teuthology.orchestra.run.smithi026.stdout: "num_strays_enqueuing": 0, 2024-09-11T23:39:58.629 INFO:teuthology.orchestra.run.smithi026.stdout: "strays_created": 0, 2024-09-11T23:39:58.629 INFO:teuthology.orchestra.run.smithi026.stdout: "strays_enqueued": 0, 2024-09-11T23:39:58.629 INFO:teuthology.orchestra.run.smithi026.stdout: "strays_reintegrated": 0, 2024-09-11T23:39:58.629 INFO:teuthology.orchestra.run.smithi026.stdout: "strays_migrated": 0, 2024-09-11T23:39:58.630 INFO:teuthology.orchestra.run.smithi026.stdout: "num_recovering_processing": 0, 2024-09-11T23:39:58.630 INFO:teuthology.orchestra.run.smithi026.stdout: "num_recovering_enqueued": 0, 2024-09-11T23:39:58.630 INFO:teuthology.orchestra.run.smithi026.stdout: "num_recovering_prioritized": 0, 2024-09-11T23:39:58.630 INFO:teuthology.orchestra.run.smithi026.stdout: "recovery_started": 0, 2024-09-11T23:39:58.630 INFO:teuthology.orchestra.run.smithi026.stdout: "recovery_completed": 0, 2024-09-11T23:39:58.630 INFO:teuthology.orchestra.run.smithi026.stdout: "ireq_quiesce_path": 0, 2024-09-11T23:39:58.630 INFO:teuthology.orchestra.run.smithi026.stdout: "ireq_quiesce_inode": 0, 2024-09-11T23:39:58.630 INFO:teuthology.orchestra.run.smithi026.stdout: "ireq_enqueue_scrub": 0, 2024-09-11T23:39:58.630 INFO:teuthology.orchestra.run.smithi026.stdout: "ireq_exportdir": 0, 2024-09-11T23:39:58.630 INFO:teuthology.orchestra.run.smithi026.stdout: "ireq_flush": 0, 2024-09-11T23:39:58.630 INFO:teuthology.orchestra.run.smithi026.stdout: "ireq_fragmentdir": 0, 2024-09-11T23:39:58.630 INFO:teuthology.orchestra.run.smithi026.stdout: "ireq_fragstats": 0, 2024-09-11T23:39:58.630 INFO:teuthology.orchestra.run.smithi026.stdout: "ireq_inodestats": 0 2024-09-11T23:39:58.630 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:58.631 INFO:teuthology.orchestra.run.smithi026.stdout: "mds_log": { 2024-09-11T23:39:58.631 INFO:teuthology.orchestra.run.smithi026.stdout: "evlrg": 0, 2024-09-11T23:39:58.631 INFO:teuthology.orchestra.run.smithi026.stdout: "evadd": 0, 2024-09-11T23:39:58.631 INFO:teuthology.orchestra.run.smithi026.stdout: "evex": 0, 2024-09-11T23:39:58.631 INFO:teuthology.orchestra.run.smithi026.stdout: "evtrm": 0, 2024-09-11T23:39:58.631 INFO:teuthology.orchestra.run.smithi026.stdout: "ev": 98369, 2024-09-11T23:39:58.631 INFO:teuthology.orchestra.run.smithi026.stdout: "evexg": 0, 2024-09-11T23:39:58.631 INFO:teuthology.orchestra.run.smithi026.stdout: "evexd": 0, 2024-09-11T23:39:58.631 INFO:teuthology.orchestra.run.smithi026.stdout: "segadd": 0, 2024-09-11T23:39:58.631 INFO:teuthology.orchestra.run.smithi026.stdout: "segex": 0, 2024-09-11T23:39:58.631 INFO:teuthology.orchestra.run.smithi026.stdout: "segtrm": 0, 2024-09-11T23:39:58.631 INFO:teuthology.orchestra.run.smithi026.stdout: "seg": 140, 2024-09-11T23:39:58.631 INFO:teuthology.orchestra.run.smithi026.stdout: "segmjr": 10, 2024-09-11T23:39:58.631 INFO:teuthology.orchestra.run.smithi026.stdout: "segexg": 0, 2024-09-11T23:39:58.631 INFO:teuthology.orchestra.run.smithi026.stdout: "segexd": 0, 2024-09-11T23:39:58.632 INFO:teuthology.orchestra.run.smithi026.stdout: "expos": 4194352, 2024-09-11T23:39:58.632 INFO:teuthology.orchestra.run.smithi026.stdout: "wrpos": 304182424, 2024-09-11T23:39:58.632 INFO:teuthology.orchestra.run.smithi026.stdout: "rdpos": 304173254, 2024-09-11T23:39:58.632 INFO:teuthology.orchestra.run.smithi026.stdout: "jlat": { 2024-09-11T23:39:58.632 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-11T23:39:58.632 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-11T23:39:58.632 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-11T23:39:58.632 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:58.632 INFO:teuthology.orchestra.run.smithi026.stdout: "replayed": 98369 2024-09-11T23:39:58.632 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:58.632 INFO:teuthology.orchestra.run.smithi026.stdout: "mds_mem": { 2024-09-11T23:39:58.632 INFO:teuthology.orchestra.run.smithi026.stdout: "ino": 13583, 2024-09-11T23:39:58.632 INFO:teuthology.orchestra.run.smithi026.stdout: "ino+": 47423, 2024-09-11T23:39:58.632 INFO:teuthology.orchestra.run.smithi026.stdout: "ino-": 33840, 2024-09-11T23:39:58.633 INFO:teuthology.orchestra.run.smithi026.stdout: "dir": 377, 2024-09-11T23:39:58.633 INFO:teuthology.orchestra.run.smithi026.stdout: "dir+": 1235, 2024-09-11T23:39:58.633 INFO:teuthology.orchestra.run.smithi026.stdout: "dir-": 858, 2024-09-11T23:39:58.633 INFO:teuthology.orchestra.run.smithi026.stdout: "dn": 36287, 2024-09-11T23:39:58.633 INFO:teuthology.orchestra.run.smithi026.stdout: "dn+": 74493, 2024-09-11T23:39:58.633 INFO:teuthology.orchestra.run.smithi026.stdout: "dn-": 38206, 2024-09-11T23:39:58.633 INFO:teuthology.orchestra.run.smithi026.stdout: "cap": 0, 2024-09-11T23:39:58.633 INFO:teuthology.orchestra.run.smithi026.stdout: "cap+": 0, 2024-09-11T23:39:58.633 INFO:teuthology.orchestra.run.smithi026.stdout: "cap-": 0, 2024-09-11T23:39:58.633 INFO:teuthology.orchestra.run.smithi026.stdout: "rss": 154768, 2024-09-11T23:39:58.633 INFO:teuthology.orchestra.run.smithi026.stdout: "heap": 215328 2024-09-11T23:39:58.633 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:58.633 INFO:teuthology.orchestra.run.smithi026.stdout: "mds_server": { 2024-09-11T23:39:58.633 INFO:teuthology.orchestra.run.smithi026.stdout: "dispatch_client_request": 0, 2024-09-11T23:39:58.633 INFO:teuthology.orchestra.run.smithi026.stdout: "dispatch_server_request": 0, 2024-09-11T23:39:58.634 INFO:teuthology.orchestra.run.smithi026.stdout: "handle_client_request": 0, 2024-09-11T23:39:58.634 INFO:teuthology.orchestra.run.smithi026.stdout: "handle_client_session": 0, 2024-09-11T23:39:58.634 INFO:teuthology.orchestra.run.smithi026.stdout: "handle_peer_request": 0, 2024-09-11T23:39:58.634 INFO:teuthology.orchestra.run.smithi026.stdout: "req_create_latency": { 2024-09-11T23:39:58.634 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-11T23:39:58.634 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-11T23:39:58.634 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-11T23:39:58.634 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:58.634 INFO:teuthology.orchestra.run.smithi026.stdout: "req_getattr_latency": { 2024-09-11T23:39:58.634 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-11T23:39:58.634 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-11T23:39:58.634 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-11T23:39:58.634 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:58.634 INFO:teuthology.orchestra.run.smithi026.stdout: "req_getfilelock_latency": { 2024-09-11T23:39:58.634 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-11T23:39:58.635 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-11T23:39:58.635 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-11T23:39:58.635 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:58.635 INFO:teuthology.orchestra.run.smithi026.stdout: "req_link_latency": { 2024-09-11T23:39:58.635 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-11T23:39:58.635 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-11T23:39:58.635 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-11T23:39:58.635 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:58.635 INFO:teuthology.orchestra.run.smithi026.stdout: "req_lookup_latency": { 2024-09-11T23:39:58.635 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-11T23:39:58.635 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-11T23:39:58.635 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-11T23:39:58.635 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:58.635 INFO:teuthology.orchestra.run.smithi026.stdout: "req_lookuphash_latency": { 2024-09-11T23:39:58.636 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-11T23:39:58.636 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-11T23:39:58.636 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-11T23:39:58.636 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:58.636 INFO:teuthology.orchestra.run.smithi026.stdout: "req_lookupino_latency": { 2024-09-11T23:39:58.636 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-11T23:39:58.636 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-11T23:39:58.636 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-11T23:39:58.636 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:58.636 INFO:teuthology.orchestra.run.smithi026.stdout: "req_lookupname_latency": { 2024-09-11T23:39:58.636 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-11T23:39:58.636 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-11T23:39:58.636 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-11T23:39:58.636 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:58.637 INFO:teuthology.orchestra.run.smithi026.stdout: "req_lookupparent_latency": { 2024-09-11T23:39:58.637 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-11T23:39:58.637 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-11T23:39:58.637 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-11T23:39:58.637 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:58.637 INFO:teuthology.orchestra.run.smithi026.stdout: "req_lookupsnap_latency": { 2024-09-11T23:39:58.637 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-11T23:39:58.637 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-11T23:39:58.637 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-11T23:39:58.637 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:58.638 INFO:teuthology.orchestra.run.smithi026.stdout: "req_lssnap_latency": { 2024-09-11T23:39:58.638 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-11T23:39:58.638 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-11T23:39:58.638 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-11T23:39:58.638 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:58.638 INFO:teuthology.orchestra.run.smithi026.stdout: "req_mkdir_latency": { 2024-09-11T23:39:58.639 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-11T23:39:58.639 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-11T23:39:58.639 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-11T23:39:58.639 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:58.639 INFO:teuthology.orchestra.run.smithi026.stdout: "req_mknod_latency": { 2024-09-11T23:39:58.639 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-11T23:39:58.639 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-11T23:39:58.639 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-11T23:39:58.639 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:58.639 INFO:teuthology.orchestra.run.smithi026.stdout: "req_mksnap_latency": { 2024-09-11T23:39:58.639 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-11T23:39:58.639 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-11T23:39:58.639 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-11T23:39:58.640 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:58.640 INFO:teuthology.orchestra.run.smithi026.stdout: "req_open_latency": { 2024-09-11T23:39:58.640 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-11T23:39:58.640 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-11T23:39:58.640 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-11T23:39:58.640 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:58.640 INFO:teuthology.orchestra.run.smithi026.stdout: "req_readdir_latency": { 2024-09-11T23:39:58.640 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-11T23:39:58.640 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-11T23:39:58.640 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-11T23:39:58.641 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:58.641 INFO:teuthology.orchestra.run.smithi026.stdout: "req_rename_latency": { 2024-09-11T23:39:58.641 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-11T23:39:58.641 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-11T23:39:58.641 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-11T23:39:58.641 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:58.641 INFO:teuthology.orchestra.run.smithi026.stdout: "req_renamesnap_latency": { 2024-09-11T23:39:58.641 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-11T23:39:58.641 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-11T23:39:58.641 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-11T23:39:58.641 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:58.641 INFO:teuthology.orchestra.run.smithi026.stdout: "req_snapdiff_latency": { 2024-09-11T23:39:58.641 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-11T23:39:58.641 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-11T23:39:58.641 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-11T23:39:58.642 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:58.642 INFO:teuthology.orchestra.run.smithi026.stdout: "req_rmdir_latency": { 2024-09-11T23:39:58.642 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-11T23:39:58.642 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-11T23:39:58.642 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-11T23:39:58.642 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:58.642 INFO:teuthology.orchestra.run.smithi026.stdout: "req_rmsnap_latency": { 2024-09-11T23:39:58.642 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-11T23:39:58.642 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-11T23:39:58.642 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-11T23:39:58.642 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:58.642 INFO:teuthology.orchestra.run.smithi026.stdout: "req_rmxattr_latency": { 2024-09-11T23:39:58.642 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-11T23:39:58.642 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-11T23:39:58.643 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-11T23:39:58.643 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:58.643 INFO:teuthology.orchestra.run.smithi026.stdout: "req_setattr_latency": { 2024-09-11T23:39:58.643 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-11T23:39:58.643 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-11T23:39:58.643 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-11T23:39:58.643 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:58.643 INFO:teuthology.orchestra.run.smithi026.stdout: "req_setdirlayout_latency": { 2024-09-11T23:39:58.643 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-11T23:39:58.643 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-11T23:39:58.643 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-11T23:39:58.643 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:58.643 INFO:teuthology.orchestra.run.smithi026.stdout: "req_setfilelock_latency": { 2024-09-11T23:39:58.643 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-11T23:39:58.644 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-11T23:39:58.644 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-11T23:39:58.644 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:58.644 INFO:teuthology.orchestra.run.smithi026.stdout: "req_setlayout_latency": { 2024-09-11T23:39:58.644 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-11T23:39:58.644 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-11T23:39:58.644 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-11T23:39:58.644 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:58.644 INFO:teuthology.orchestra.run.smithi026.stdout: "req_setxattr_latency": { 2024-09-11T23:39:58.644 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-11T23:39:58.644 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-11T23:39:58.644 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-11T23:39:58.644 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:58.645 INFO:teuthology.orchestra.run.smithi026.stdout: "req_symlink_latency": { 2024-09-11T23:39:58.645 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-11T23:39:58.645 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-11T23:39:58.645 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-11T23:39:58.645 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:58.645 INFO:teuthology.orchestra.run.smithi026.stdout: "req_unlink_latency": { 2024-09-11T23:39:58.645 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-11T23:39:58.645 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-11T23:39:58.645 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-11T23:39:58.645 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:58.645 INFO:teuthology.orchestra.run.smithi026.stdout: "cap_revoke_eviction": 0, 2024-09-11T23:39:58.645 INFO:teuthology.orchestra.run.smithi026.stdout: "cap_acquisition_throttle": 0, 2024-09-11T23:39:58.645 INFO:teuthology.orchestra.run.smithi026.stdout: "req_getvxattr_latency": { 2024-09-11T23:39:58.645 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-11T23:39:58.646 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-11T23:39:58.646 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-11T23:39:58.646 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:58.646 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:58.646 INFO:teuthology.orchestra.run.smithi026.stdout: "mds_sessions": { 2024-09-11T23:39:58.646 INFO:teuthology.orchestra.run.smithi026.stdout: "session_count": 2, 2024-09-11T23:39:58.646 INFO:teuthology.orchestra.run.smithi026.stdout: "session_add": 3, 2024-09-11T23:39:58.646 INFO:teuthology.orchestra.run.smithi026.stdout: "session_remove": 1, 2024-09-11T23:39:58.646 INFO:teuthology.orchestra.run.smithi026.stdout: "sessions_open": 2, 2024-09-11T23:39:58.646 INFO:teuthology.orchestra.run.smithi026.stdout: "sessions_stale": 0, 2024-09-11T23:39:58.646 INFO:teuthology.orchestra.run.smithi026.stdout: "total_load": 0, 2024-09-11T23:39:58.646 INFO:teuthology.orchestra.run.smithi026.stdout: "average_load": 0, 2024-09-11T23:39:58.646 INFO:teuthology.orchestra.run.smithi026.stdout: "avg_session_uptime": 3302, 2024-09-11T23:39:58.646 INFO:teuthology.orchestra.run.smithi026.stdout: "mdthresh_evicted": 0 2024-09-11T23:39:58.647 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:58.647 INFO:teuthology.orchestra.run.smithi026.stdout: "mempool": { 2024-09-11T23:39:58.647 INFO:teuthology.orchestra.run.smithi026.stdout: "bloom_filter_bytes": 0, 2024-09-11T23:39:58.647 INFO:teuthology.orchestra.run.smithi026.stdout: "bloom_filter_items": 0, 2024-09-11T23:39:58.647 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_alloc_bytes": 0, 2024-09-11T23:39:58.647 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_alloc_items": 0, 2024-09-11T23:39:58.647 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_data_bytes": 0, 2024-09-11T23:39:58.647 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_data_items": 0, 2024-09-11T23:39:58.647 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-11T23:39:58.647 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_onode_items": 0, 2024-09-11T23:39:58.647 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-11T23:39:58.647 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_meta_items": 0, 2024-09-11T23:39:58.647 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_other_bytes": 0, 2024-09-11T23:39:58.647 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_other_items": 0, 2024-09-11T23:39:58.648 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-11T23:39:58.648 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_buffer_items": 0, 2024-09-11T23:39:58.648 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_extent_bytes": 0, 2024-09-11T23:39:58.648 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_extent_items": 0, 2024-09-11T23:39:58.648 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_blob_bytes": 0, 2024-09-11T23:39:58.648 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_blob_items": 0, 2024-09-11T23:39:58.648 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-11T23:39:58.648 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_shared_blob_items": 0, 2024-09-11T23:39:58.648 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-11T23:39:58.648 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_inline_bl_items": 0, 2024-09-11T23:39:58.648 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_fsck_bytes": 0, 2024-09-11T23:39:58.648 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_fsck_items": 0, 2024-09-11T23:39:58.648 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_txc_bytes": 0, 2024-09-11T23:39:58.649 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_txc_items": 0, 2024-09-11T23:39:58.649 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-11T23:39:58.649 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_writing_deferred_items": 0, 2024-09-11T23:39:58.649 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_writing_bytes": 0, 2024-09-11T23:39:58.649 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_writing_items": 0, 2024-09-11T23:39:58.649 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_bytes": 0, 2024-09-11T23:39:58.649 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_items": 0, 2024-09-11T23:39:58.649 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_file_reader_bytes": 0, 2024-09-11T23:39:58.649 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_file_reader_items": 0, 2024-09-11T23:39:58.649 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_file_writer_bytes": 0, 2024-09-11T23:39:58.649 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_file_writer_items": 0, 2024-09-11T23:39:58.649 INFO:teuthology.orchestra.run.smithi026.stdout: "buffer_anon_bytes": 748996, 2024-09-11T23:39:58.649 INFO:teuthology.orchestra.run.smithi026.stdout: "buffer_anon_items": 13831, 2024-09-11T23:39:58.649 INFO:teuthology.orchestra.run.smithi026.stdout: "buffer_meta_bytes": 0, 2024-09-11T23:39:58.650 INFO:teuthology.orchestra.run.smithi026.stdout: "buffer_meta_items": 0, 2024-09-11T23:39:58.650 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_bytes": 0, 2024-09-11T23:39:58.650 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_items": 0, 2024-09-11T23:39:58.650 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_mapbl_bytes": 0, 2024-09-11T23:39:58.650 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_mapbl_items": 0, 2024-09-11T23:39:58.650 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_pglog_bytes": 0, 2024-09-11T23:39:58.650 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_pglog_items": 0, 2024-09-11T23:39:58.650 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_bytes": 8344, 2024-09-11T23:39:58.650 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_items": 184, 2024-09-11T23:39:58.650 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_mapping_bytes": 0, 2024-09-11T23:39:58.650 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_mapping_items": 0, 2024-09-11T23:39:58.650 INFO:teuthology.orchestra.run.smithi026.stdout: "pgmap_bytes": 0, 2024-09-11T23:39:58.650 INFO:teuthology.orchestra.run.smithi026.stdout: "pgmap_items": 0, 2024-09-11T23:39:58.650 INFO:teuthology.orchestra.run.smithi026.stdout: "mds_co_bytes": 57508386, 2024-09-11T23:39:58.651 INFO:teuthology.orchestra.run.smithi026.stdout: "mds_co_items": 572569, 2024-09-11T23:39:58.651 INFO:teuthology.orchestra.run.smithi026.stdout: "unittest_1_bytes": 0, 2024-09-11T23:39:58.651 INFO:teuthology.orchestra.run.smithi026.stdout: "unittest_1_items": 0, 2024-09-11T23:39:58.651 INFO:teuthology.orchestra.run.smithi026.stdout: "unittest_2_bytes": 0, 2024-09-11T23:39:58.651 INFO:teuthology.orchestra.run.smithi026.stdout: "unittest_2_items": 0 2024-09-11T23:39:58.651 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:58.651 INFO:teuthology.orchestra.run.smithi026.stdout: "objecter": { 2024-09-11T23:39:58.651 INFO:teuthology.orchestra.run.smithi026.stdout: "op_active": 0, 2024-09-11T23:39:58.651 INFO:teuthology.orchestra.run.smithi026.stdout: "op_laggy": 0, 2024-09-11T23:39:58.651 INFO:teuthology.orchestra.run.smithi026.stdout: "op_send": 4665, 2024-09-11T23:39:58.651 INFO:teuthology.orchestra.run.smithi026.stdout: "op_send_bytes": 21, 2024-09-11T23:39:58.651 INFO:teuthology.orchestra.run.smithi026.stdout: "op_resend": 3, 2024-09-11T23:39:58.651 INFO:teuthology.orchestra.run.smithi026.stdout: "op_reply": 4665, 2024-09-11T23:39:58.651 INFO:teuthology.orchestra.run.smithi026.stdout: "op_latency": { 2024-09-11T23:39:58.652 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 4665, 2024-09-11T23:39:58.652 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 20.440681894, 2024-09-11T23:39:58.652 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.004381711 2024-09-11T23:39:58.652 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:58.652 INFO:teuthology.orchestra.run.smithi026.stdout: "op_inflight": 0, 2024-09-11T23:39:58.652 INFO:teuthology.orchestra.run.smithi026.stdout: "oplen_avg": { 2024-09-11T23:39:58.652 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 4665, 2024-09-11T23:39:58.652 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 4666 2024-09-11T23:39:58.652 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:58.652 INFO:teuthology.orchestra.run.smithi026.stdout: "op": 4665, 2024-09-11T23:39:58.652 INFO:teuthology.orchestra.run.smithi026.stdout: "op_r": 4665, 2024-09-11T23:39:58.652 INFO:teuthology.orchestra.run.smithi026.stdout: "op_w": 0, 2024-09-11T23:39:58.652 INFO:teuthology.orchestra.run.smithi026.stdout: "op_rmw": 0, 2024-09-11T23:39:58.652 INFO:teuthology.orchestra.run.smithi026.stdout: "op_pg": 0, 2024-09-11T23:39:58.653 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_stat": 2880, 2024-09-11T23:39:58.653 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_create": 0, 2024-09-11T23:39:58.653 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_read": 1783, 2024-09-11T23:39:58.653 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_write": 0, 2024-09-11T23:39:58.653 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_writefull": 0, 2024-09-11T23:39:58.653 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_writesame": 0, 2024-09-11T23:39:58.653 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_append": 0, 2024-09-11T23:39:58.653 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_zero": 0, 2024-09-11T23:39:58.653 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_truncate": 0, 2024-09-11T23:39:58.653 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_delete": 0, 2024-09-11T23:39:58.653 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_mapext": 0, 2024-09-11T23:39:58.653 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_sparse_read": 0, 2024-09-11T23:39:58.653 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_clonerange": 0, 2024-09-11T23:39:58.653 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_getxattr": 1, 2024-09-11T23:39:58.656 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_setxattr": 0, 2024-09-11T23:39:58.656 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_cmpxattr": 0, 2024-09-11T23:39:58.656 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_rmxattr": 0, 2024-09-11T23:39:58.656 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_resetxattrs": 0, 2024-09-11T23:39:58.656 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_call": 0, 2024-09-11T23:39:58.656 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_watch": 0, 2024-09-11T23:39:58.657 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_notify": 0, 2024-09-11T23:39:58.657 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_src_cmpxattr": 0, 2024-09-11T23:39:58.657 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_pgls": 0, 2024-09-11T23:39:58.657 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_pgls_filter": 0, 2024-09-11T23:39:58.657 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_other": 0, 2024-09-11T23:39:58.657 INFO:teuthology.orchestra.run.smithi026.stdout: "linger_active": 0, 2024-09-11T23:39:58.657 INFO:teuthology.orchestra.run.smithi026.stdout: "linger_send": 0, 2024-09-11T23:39:58.657 INFO:teuthology.orchestra.run.smithi026.stdout: "linger_resend": 0, 2024-09-11T23:39:58.657 INFO:teuthology.orchestra.run.smithi026.stdout: "linger_ping": 0, 2024-09-11T23:39:58.657 INFO:teuthology.orchestra.run.smithi026.stdout: "poolop_active": 0, 2024-09-11T23:39:58.657 INFO:teuthology.orchestra.run.smithi026.stdout: "poolop_send": 0, 2024-09-11T23:39:58.657 INFO:teuthology.orchestra.run.smithi026.stdout: "poolop_resend": 0, 2024-09-11T23:39:58.657 INFO:teuthology.orchestra.run.smithi026.stdout: "poolstat_active": 0, 2024-09-11T23:39:58.657 INFO:teuthology.orchestra.run.smithi026.stdout: "poolstat_send": 0, 2024-09-11T23:39:58.658 INFO:teuthology.orchestra.run.smithi026.stdout: "poolstat_resend": 0, 2024-09-11T23:39:58.658 INFO:teuthology.orchestra.run.smithi026.stdout: "statfs_active": 0, 2024-09-11T23:39:58.658 INFO:teuthology.orchestra.run.smithi026.stdout: "statfs_send": 0, 2024-09-11T23:39:58.658 INFO:teuthology.orchestra.run.smithi026.stdout: "statfs_resend": 0, 2024-09-11T23:39:58.658 INFO:teuthology.orchestra.run.smithi026.stdout: "command_active": 0, 2024-09-11T23:39:58.658 INFO:teuthology.orchestra.run.smithi026.stdout: "command_send": 0, 2024-09-11T23:39:58.658 INFO:teuthology.orchestra.run.smithi026.stdout: "command_resend": 0, 2024-09-11T23:39:58.658 INFO:teuthology.orchestra.run.smithi026.stdout: "map_epoch": 115, 2024-09-11T23:39:58.658 INFO:teuthology.orchestra.run.smithi026.stdout: "map_full": 0, 2024-09-11T23:39:58.658 INFO:teuthology.orchestra.run.smithi026.stdout: "map_inc": 30, 2024-09-11T23:39:58.658 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_sessions": 11, 2024-09-11T23:39:58.658 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_session_open": 11, 2024-09-11T23:39:58.658 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_session_close": 0, 2024-09-11T23:39:58.658 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_laggy": 0, 2024-09-11T23:39:58.658 INFO:teuthology.orchestra.run.smithi026.stdout: "omap_wr": 0, 2024-09-11T23:39:58.659 INFO:teuthology.orchestra.run.smithi026.stdout: "omap_rd": 2, 2024-09-11T23:39:58.659 INFO:teuthology.orchestra.run.smithi026.stdout: "omap_del": 0 2024-09-11T23:39:58.659 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:58.659 INFO:teuthology.orchestra.run.smithi026.stdout: "oft": { 2024-09-11T23:39:58.659 INFO:teuthology.orchestra.run.smithi026.stdout: "omap_total_objs": 0, 2024-09-11T23:39:58.659 INFO:teuthology.orchestra.run.smithi026.stdout: "omap_total_kv_pairs": 0, 2024-09-11T23:39:58.659 INFO:teuthology.orchestra.run.smithi026.stdout: "omap_total_updates": 0, 2024-09-11T23:39:58.659 INFO:teuthology.orchestra.run.smithi026.stdout: "omap_total_removes": 0 2024-09-11T23:39:58.659 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:58.659 INFO:teuthology.orchestra.run.smithi026.stdout: "osd-slow-ops": { 2024-09-11T23:39:58.659 INFO:teuthology.orchestra.run.smithi026.stdout: "slow_ops_count": 0 2024-09-11T23:39:58.659 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:58.659 INFO:teuthology.orchestra.run.smithi026.stdout: "purge_queue": { 2024-09-11T23:39:58.659 INFO:teuthology.orchestra.run.smithi026.stdout: "pq_executing_ops": 0, 2024-09-11T23:39:58.660 INFO:teuthology.orchestra.run.smithi026.stdout: "pq_executing_ops_high_water": 0, 2024-09-11T23:39:58.660 INFO:teuthology.orchestra.run.smithi026.stdout: "pq_executing": 0, 2024-09-11T23:39:58.660 INFO:teuthology.orchestra.run.smithi026.stdout: "pq_executing_high_water": 0, 2024-09-11T23:39:58.660 INFO:teuthology.orchestra.run.smithi026.stdout: "pq_executed_ops": 0, 2024-09-11T23:39:58.660 INFO:teuthology.orchestra.run.smithi026.stdout: "pq_executed": 0, 2024-09-11T23:39:58.660 INFO:teuthology.orchestra.run.smithi026.stdout: "pq_item_in_journal": 0 2024-09-11T23:39:58.660 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:58.660 INFO:teuthology.orchestra.run.smithi026.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-11T23:39:58.660 INFO:teuthology.orchestra.run.smithi026.stdout: "val": 0, 2024-09-11T23:39:58.660 INFO:teuthology.orchestra.run.smithi026.stdout: "max": 104857600, 2024-09-11T23:39:58.660 INFO:teuthology.orchestra.run.smithi026.stdout: "get_started": 0, 2024-09-11T23:39:58.660 INFO:teuthology.orchestra.run.smithi026.stdout: "get": 5157, 2024-09-11T23:39:58.660 INFO:teuthology.orchestra.run.smithi026.stdout: "get_sum": 302952380, 2024-09-11T23:39:58.660 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_fail": 0, 2024-09-11T23:39:58.660 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_success": 5157, 2024-09-11T23:39:58.661 INFO:teuthology.orchestra.run.smithi026.stdout: "take": 0, 2024-09-11T23:39:58.661 INFO:teuthology.orchestra.run.smithi026.stdout: "take_sum": 0, 2024-09-11T23:39:58.661 INFO:teuthology.orchestra.run.smithi026.stdout: "put": 5157, 2024-09-11T23:39:58.661 INFO:teuthology.orchestra.run.smithi026.stdout: "put_sum": 302952380, 2024-09-11T23:39:58.661 INFO:teuthology.orchestra.run.smithi026.stdout: "wait": { 2024-09-11T23:39:58.661 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-11T23:39:58.661 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-11T23:39:58.661 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-11T23:39:58.661 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:58.661 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:58.661 INFO:teuthology.orchestra.run.smithi026.stdout: "throttle-objecter_bytes": { 2024-09-11T23:39:58.661 INFO:teuthology.orchestra.run.smithi026.stdout: "val": 0, 2024-09-11T23:39:58.661 INFO:teuthology.orchestra.run.smithi026.stdout: "max": 104857600, 2024-09-11T23:39:58.661 INFO:teuthology.orchestra.run.smithi026.stdout: "get_started": 0, 2024-09-11T23:39:58.662 INFO:teuthology.orchestra.run.smithi026.stdout: "get": 0, 2024-09-11T23:39:58.662 INFO:teuthology.orchestra.run.smithi026.stdout: "get_sum": 0, 2024-09-11T23:39:58.662 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_fail": 0, 2024-09-11T23:39:58.662 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_success": 0, 2024-09-11T23:39:58.662 INFO:teuthology.orchestra.run.smithi026.stdout: "take": 4665, 2024-09-11T23:39:58.662 INFO:teuthology.orchestra.run.smithi026.stdout: "take_sum": 300042506, 2024-09-11T23:39:58.662 INFO:teuthology.orchestra.run.smithi026.stdout: "put": 4665, 2024-09-11T23:39:58.662 INFO:teuthology.orchestra.run.smithi026.stdout: "put_sum": 300042506, 2024-09-11T23:39:58.662 INFO:teuthology.orchestra.run.smithi026.stdout: "wait": { 2024-09-11T23:39:58.662 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-11T23:39:58.662 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-11T23:39:58.662 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-11T23:39:58.662 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:58.662 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:58.663 INFO:teuthology.orchestra.run.smithi026.stdout: "throttle-objecter_ops": { 2024-09-11T23:39:58.663 INFO:teuthology.orchestra.run.smithi026.stdout: "val": 0, 2024-09-11T23:39:58.663 INFO:teuthology.orchestra.run.smithi026.stdout: "max": 1024, 2024-09-11T23:39:58.663 INFO:teuthology.orchestra.run.smithi026.stdout: "get_started": 0, 2024-09-11T23:39:58.663 INFO:teuthology.orchestra.run.smithi026.stdout: "get": 0, 2024-09-11T23:39:58.663 INFO:teuthology.orchestra.run.smithi026.stdout: "get_sum": 0, 2024-09-11T23:39:58.663 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_fail": 0, 2024-09-11T23:39:58.663 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_success": 0, 2024-09-11T23:39:58.663 INFO:teuthology.orchestra.run.smithi026.stdout: "take": 4665, 2024-09-11T23:39:58.663 INFO:teuthology.orchestra.run.smithi026.stdout: "take_sum": 4665, 2024-09-11T23:39:58.663 INFO:teuthology.orchestra.run.smithi026.stdout: "put": 4665, 2024-09-11T23:39:58.663 INFO:teuthology.orchestra.run.smithi026.stdout: "put_sum": 4665, 2024-09-11T23:39:58.663 INFO:teuthology.orchestra.run.smithi026.stdout: "wait": { 2024-09-11T23:39:58.663 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-11T23:39:58.664 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-11T23:39:58.664 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-11T23:39:58.664 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:58.664 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:58.664 INFO:teuthology.orchestra.run.smithi026.stdout: "throttle-write_buf_throttle": { 2024-09-11T23:39:58.664 INFO:teuthology.orchestra.run.smithi026.stdout: "val": 0, 2024-09-11T23:39:58.664 INFO:teuthology.orchestra.run.smithi026.stdout: "max": 3758096384, 2024-09-11T23:39:58.664 INFO:teuthology.orchestra.run.smithi026.stdout: "get_started": 0, 2024-09-11T23:39:58.664 INFO:teuthology.orchestra.run.smithi026.stdout: "get": 0, 2024-09-11T23:39:58.664 INFO:teuthology.orchestra.run.smithi026.stdout: "get_sum": 0, 2024-09-11T23:39:58.664 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_fail": 0, 2024-09-11T23:39:58.664 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_success": 0, 2024-09-11T23:39:58.664 INFO:teuthology.orchestra.run.smithi026.stdout: "take": 0, 2024-09-11T23:39:58.664 INFO:teuthology.orchestra.run.smithi026.stdout: "take_sum": 0, 2024-09-11T23:39:58.665 INFO:teuthology.orchestra.run.smithi026.stdout: "put": 0, 2024-09-11T23:39:58.665 INFO:teuthology.orchestra.run.smithi026.stdout: "put_sum": 0, 2024-09-11T23:39:58.665 INFO:teuthology.orchestra.run.smithi026.stdout: "wait": { 2024-09-11T23:39:58.665 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-11T23:39:58.665 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-11T23:39:58.665 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-11T23:39:58.665 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:58.665 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:39:58.665 INFO:teuthology.orchestra.run.smithi026.stdout: "throttle-write_buf_throttle-0x558ffb572b40": { 2024-09-11T23:39:58.665 INFO:teuthology.orchestra.run.smithi026.stdout: "val": 0, 2024-09-11T23:39:58.665 INFO:teuthology.orchestra.run.smithi026.stdout: "max": 3758096384, 2024-09-11T23:39:58.665 INFO:teuthology.orchestra.run.smithi026.stdout: "get_started": 0, 2024-09-11T23:39:58.665 INFO:teuthology.orchestra.run.smithi026.stdout: "get": 0, 2024-09-11T23:39:58.665 INFO:teuthology.orchestra.run.smithi026.stdout: "get_sum": 0, 2024-09-11T23:39:58.666 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_fail": 0, 2024-09-11T23:39:58.666 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_success": 0, 2024-09-11T23:39:58.666 INFO:teuthology.orchestra.run.smithi026.stdout: "take": 0, 2024-09-11T23:39:58.666 INFO:teuthology.orchestra.run.smithi026.stdout: "take_sum": 0, 2024-09-11T23:39:58.666 INFO:teuthology.orchestra.run.smithi026.stdout: "put": 0, 2024-09-11T23:39:58.666 INFO:teuthology.orchestra.run.smithi026.stdout: "put_sum": 0, 2024-09-11T23:39:58.666 INFO:teuthology.orchestra.run.smithi026.stdout: "wait": { 2024-09-11T23:39:58.666 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-11T23:39:58.666 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-11T23:39:58.666 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-11T23:39:58.666 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:58.666 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:39:58.666 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-11T23:39:59.413 INFO:tasks.check_counter:Daemon mds.g mds.exported=0 2024-09-11T23:39:59.413 INFO:tasks.check_counter:Daemon mds.g mds.imported=0 2024-09-11T23:39:59.413 INFO:tasks.check_counter:Daemon mds.g mds_cache.dir_update=0 2024-09-11T23:39:59.413 INFO:tasks.check_counter:Daemon mds.g mds_cache.dir_update_receipt=0 2024-09-11T23:39:59.413 INFO:tasks.check_counter:Daemon mds.g mds.root_rsnaps=0 2024-09-11T23:39:59.414 INFO:tasks.check_counter:Daemon mds.g mds_server.req_mksnap_latency.avgcount=0 2024-09-11T23:39:59.414 INFO:tasks.check_counter:Daemon mds.g mds_server.req_rmsnap_latency.avgcount=0 2024-09-11T23:39:59.414 INFO:tasks.check_counter:Daemon mds.g mds.dir_split=0 2024-09-11T23:39:59.414 DEBUG:tasks.check_counter:Getting stats from j 2024-09-11T23:39:59.414 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph daemon mds.j perf dump 2024-09-11T23:39:59.672 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:40:00.079 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:40:00 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mon-a[31168]: 2024-09-11T23:39:59.998+0000 7f5d93536640 -1 log_channel(cluster) log [ERR] : overall HEALTH_ERR 1 MDSs report damaged metadata 2024-09-11T23:40:00.195 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-11T23:40:00.195 INFO:teuthology.orchestra.run.smithi026.stdout: "AsyncMessenger::Worker-0": { 2024-09-11T23:40:00.195 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_messages": 0, 2024-09-11T23:40:00.195 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages": 2, 2024-09-11T23:40:00.195 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_bytes": 0, 2024-09-11T23:40:00.195 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_bytes": 0, 2024-09-11T23:40:00.195 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_created_connections": 4, 2024-09-11T23:40:00.195 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_active_connections": 0, 2024-09-11T23:40:00.195 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_total_time": 0.001639374, 2024-09-11T23:40:00.196 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_send_time": 0.000000000, 2024-09-11T23:40:00.196 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_recv_time": 0.000385831, 2024-09-11T23:40:00.196 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-11T23:40:00.196 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T23:40:00.196 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-11T23:40:00.196 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-11T23:40:00.196 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-11T23:40:00.196 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:40:00.196 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_handle_ack_lat": { 2024-09-11T23:40:00.196 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-11T23:40:00.197 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-11T23:40:00.197 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-11T23:40:00.197 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:40:00.197 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-11T23:40:00.197 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_encrypted_bytes": 0 2024-09-11T23:40:00.197 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:40:00.197 INFO:teuthology.orchestra.run.smithi026.stdout: "AsyncMessenger::Worker-1": { 2024-09-11T23:40:00.197 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_messages": 396, 2024-09-11T23:40:00.197 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages": 385, 2024-09-11T23:40:00.197 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_bytes": 1630896, 2024-09-11T23:40:00.197 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_bytes": 238192, 2024-09-11T23:40:00.197 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_created_connections": 2, 2024-09-11T23:40:00.197 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_active_connections": 1, 2024-09-11T23:40:00.198 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_total_time": 0.060755190, 2024-09-11T23:40:00.198 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_send_time": 0.025975419, 2024-09-11T23:40:00.198 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_recv_time": 0.049091726, 2024-09-11T23:40:00.198 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-11T23:40:00.198 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T23:40:00.198 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 384, 2024-09-11T23:40:00.198 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.023998469, 2024-09-11T23:40:00.198 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000062496 2024-09-11T23:40:00.198 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:40:00.198 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_handle_ack_lat": { 2024-09-11T23:40:00.198 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-11T23:40:00.198 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-11T23:40:00.198 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-11T23:40:00.199 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:40:00.199 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_encrypted_bytes": 1630896, 2024-09-11T23:40:00.199 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_encrypted_bytes": 238192 2024-09-11T23:40:00.199 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:40:00.199 INFO:teuthology.orchestra.run.smithi026.stdout: "AsyncMessenger::Worker-2": { 2024-09-11T23:40:00.199 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_messages": 7, 2024-09-11T23:40:00.199 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages": 298, 2024-09-11T23:40:00.199 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_bytes": 3504, 2024-09-11T23:40:00.199 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_bytes": 288656, 2024-09-11T23:40:00.199 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_created_connections": 6, 2024-09-11T23:40:00.199 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_active_connections": 3, 2024-09-11T23:40:00.199 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_total_time": 0.024080033, 2024-09-11T23:40:00.199 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_send_time": 0.016318054, 2024-09-11T23:40:00.200 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_recv_time": 0.004480754, 2024-09-11T23:40:00.200 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-11T23:40:00.200 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T23:40:00.200 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 296, 2024-09-11T23:40:00.200 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.048276172, 2024-09-11T23:40:00.200 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000163095 2024-09-11T23:40:00.200 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:40:00.200 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_handle_ack_lat": { 2024-09-11T23:40:00.200 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-11T23:40:00.200 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-11T23:40:00.200 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-11T23:40:00.200 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:40:00.200 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_encrypted_bytes": 3504, 2024-09-11T23:40:00.201 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_encrypted_bytes": 288656 2024-09-11T23:40:00.201 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:40:00.201 INFO:teuthology.orchestra.run.smithi026.stdout: "cct": { 2024-09-11T23:40:00.201 INFO:teuthology.orchestra.run.smithi026.stdout: "total_workers": 0, 2024-09-11T23:40:00.201 INFO:teuthology.orchestra.run.smithi026.stdout: "unhealthy_workers": 0 2024-09-11T23:40:00.201 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:40:00.201 INFO:teuthology.orchestra.run.smithi026.stdout: "mempool": { 2024-09-11T23:40:00.201 INFO:teuthology.orchestra.run.smithi026.stdout: "bloom_filter_bytes": 0, 2024-09-11T23:40:00.201 INFO:teuthology.orchestra.run.smithi026.stdout: "bloom_filter_items": 0, 2024-09-11T23:40:00.201 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_alloc_bytes": 0, 2024-09-11T23:40:00.201 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_alloc_items": 0, 2024-09-11T23:40:00.201 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_data_bytes": 0, 2024-09-11T23:40:00.201 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_data_items": 0, 2024-09-11T23:40:00.202 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-11T23:40:00.202 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_onode_items": 0, 2024-09-11T23:40:00.202 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-11T23:40:00.202 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_meta_items": 0, 2024-09-11T23:40:00.202 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_other_bytes": 0, 2024-09-11T23:40:00.202 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_other_items": 0, 2024-09-11T23:40:00.202 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-11T23:40:00.202 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_buffer_items": 0, 2024-09-11T23:40:00.202 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_extent_bytes": 0, 2024-09-11T23:40:00.202 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_extent_items": 0, 2024-09-11T23:40:00.202 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_blob_bytes": 0, 2024-09-11T23:40:00.202 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_blob_items": 0, 2024-09-11T23:40:00.202 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-11T23:40:00.202 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_shared_blob_items": 0, 2024-09-11T23:40:00.203 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-11T23:40:00.203 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_inline_bl_items": 0, 2024-09-11T23:40:00.203 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_fsck_bytes": 0, 2024-09-11T23:40:00.203 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_fsck_items": 0, 2024-09-11T23:40:00.203 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_txc_bytes": 0, 2024-09-11T23:40:00.203 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_txc_items": 0, 2024-09-11T23:40:00.203 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-11T23:40:00.203 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_writing_deferred_items": 0, 2024-09-11T23:40:00.203 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_writing_bytes": 0, 2024-09-11T23:40:00.203 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_writing_items": 0, 2024-09-11T23:40:00.203 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_bytes": 0, 2024-09-11T23:40:00.203 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_items": 0, 2024-09-11T23:40:00.203 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_file_reader_bytes": 0, 2024-09-11T23:40:00.204 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_file_reader_items": 0, 2024-09-11T23:40:00.204 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_file_writer_bytes": 0, 2024-09-11T23:40:00.204 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_file_writer_items": 0, 2024-09-11T23:40:00.204 INFO:teuthology.orchestra.run.smithi026.stdout: "buffer_anon_bytes": 148714, 2024-09-11T23:40:00.204 INFO:teuthology.orchestra.run.smithi026.stdout: "buffer_anon_items": 30, 2024-09-11T23:40:00.204 INFO:teuthology.orchestra.run.smithi026.stdout: "buffer_meta_bytes": 0, 2024-09-11T23:40:00.204 INFO:teuthology.orchestra.run.smithi026.stdout: "buffer_meta_items": 0, 2024-09-11T23:40:00.204 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_bytes": 0, 2024-09-11T23:40:00.204 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_items": 0, 2024-09-11T23:40:00.204 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_mapbl_bytes": 0, 2024-09-11T23:40:00.204 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_mapbl_items": 0, 2024-09-11T23:40:00.204 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_pglog_bytes": 0, 2024-09-11T23:40:00.204 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_pglog_items": 0, 2024-09-11T23:40:00.204 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_bytes": 0, 2024-09-11T23:40:00.205 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_items": 0, 2024-09-11T23:40:00.205 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_mapping_bytes": 0, 2024-09-11T23:40:00.205 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_mapping_items": 0, 2024-09-11T23:40:00.205 INFO:teuthology.orchestra.run.smithi026.stdout: "pgmap_bytes": 0, 2024-09-11T23:40:00.205 INFO:teuthology.orchestra.run.smithi026.stdout: "pgmap_items": 0, 2024-09-11T23:40:00.205 INFO:teuthology.orchestra.run.smithi026.stdout: "mds_co_bytes": 928, 2024-09-11T23:40:00.205 INFO:teuthology.orchestra.run.smithi026.stdout: "mds_co_items": 2, 2024-09-11T23:40:00.205 INFO:teuthology.orchestra.run.smithi026.stdout: "unittest_1_bytes": 0, 2024-09-11T23:40:00.205 INFO:teuthology.orchestra.run.smithi026.stdout: "unittest_1_items": 0, 2024-09-11T23:40:00.205 INFO:teuthology.orchestra.run.smithi026.stdout: "unittest_2_bytes": 0, 2024-09-11T23:40:00.205 INFO:teuthology.orchestra.run.smithi026.stdout: "unittest_2_items": 0 2024-09-11T23:40:00.205 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-11T23:40:00.205 INFO:teuthology.orchestra.run.smithi026.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-11T23:40:00.206 INFO:teuthology.orchestra.run.smithi026.stdout: "val": 0, 2024-09-11T23:40:00.206 INFO:teuthology.orchestra.run.smithi026.stdout: "max": 104857600, 2024-09-11T23:40:00.206 INFO:teuthology.orchestra.run.smithi026.stdout: "get_started": 0, 2024-09-11T23:40:00.206 INFO:teuthology.orchestra.run.smithi026.stdout: "get": 399, 2024-09-11T23:40:00.206 INFO:teuthology.orchestra.run.smithi026.stdout: "get_sum": 1574989, 2024-09-11T23:40:00.206 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_fail": 0, 2024-09-11T23:40:00.206 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_success": 399, 2024-09-11T23:40:00.206 INFO:teuthology.orchestra.run.smithi026.stdout: "take": 0, 2024-09-11T23:40:00.206 INFO:teuthology.orchestra.run.smithi026.stdout: "take_sum": 0, 2024-09-11T23:40:00.206 INFO:teuthology.orchestra.run.smithi026.stdout: "put": 399, 2024-09-11T23:40:00.206 INFO:teuthology.orchestra.run.smithi026.stdout: "put_sum": 1574989, 2024-09-11T23:40:00.206 INFO:teuthology.orchestra.run.smithi026.stdout: "wait": { 2024-09-11T23:40:00.206 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-11T23:40:00.206 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-11T23:40:00.207 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-11T23:40:00.207 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:40:00.207 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-11T23:40:00.207 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-11T23:40:00.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:40:00 smithi136 ceph-mon[35121]: pgmap v680: 97 pgs: 97 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 68 KiB/s wr, 55 op/s 2024-09-11T23:40:00.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:40:00 smithi136 ceph-mon[35121]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:40:00.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:40:00 smithi136 ceph-mon[35121]: overall HEALTH_ERR 1 MDSs report damaged metadata 2024-09-11T23:40:00.501 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:40:00 smithi026 ceph-mon[31172]: pgmap v680: 97 pgs: 97 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 68 KiB/s wr, 55 op/s 2024-09-11T23:40:00.501 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:40:00 smithi026 ceph-mon[31172]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:40:00.501 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:40:00 smithi026 ceph-mon[31172]: overall HEALTH_ERR 1 MDSs report damaged metadata 2024-09-11T23:40:00.552 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:40:00 smithi076 ceph-mon[35296]: pgmap v680: 97 pgs: 97 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 68 KiB/s wr, 55 op/s 2024-09-11T23:40:00.552 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:40:00 smithi076 ceph-mon[35296]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:40:00.553 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:40:00 smithi076 ceph-mon[35296]: overall HEALTH_ERR 1 MDSs report damaged metadata 2024-09-11T23:40:01.088 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-09-11T23:40:01.088 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-09-11T23:40:01.088 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.j 2024-09-11T23:40:01.088 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.j 2024-09-11T23:40:01.088 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-09-11T23:40:01.088 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-09-11T23:40:01.089 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-09-11T23:40:01.089 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.j 2024-09-11T23:40:01.089 DEBUG:tasks.check_counter:Getting stats from b 2024-09-11T23:40:01.089 DEBUG:teuthology.orchestra.run.smithi076:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph daemon mds.b perf dump 2024-09-11T23:40:01.434 INFO:teuthology.orchestra.run.smithi076.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T23:40:02.063 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-11T23:40:02.064 INFO:teuthology.orchestra.run.smithi076.stdout: "AsyncMessenger::Worker-0": { 2024-09-11T23:40:02.064 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_messages": 68332, 2024-09-11T23:40:02.064 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_messages": 83608, 2024-09-11T23:40:02.064 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_bytes": 30489023, 2024-09-11T23:40:02.064 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_bytes": 150299308, 2024-09-11T23:40:02.064 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_created_connections": 13, 2024-09-11T23:40:02.064 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_active_connections": 4, 2024-09-11T23:40:02.064 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_total_time": 5.596026829, 2024-09-11T23:40:02.064 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_send_time": 2.323643801, 2024-09-11T23:40:02.064 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_recv_time": 2.359766923, 2024-09-11T23:40:02.064 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_fast_dispatch_time": 1.016498692, 2024-09-11T23:40:02.065 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T23:40:02.065 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 83610, 2024-09-11T23:40:02.065 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 27.413327922, 2024-09-11T23:40:02.065 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000327871 2024-09-11T23:40:02.065 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:02.065 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_handle_ack_lat": { 2024-09-11T23:40:02.065 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 106206, 2024-09-11T23:40:02.065 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.022414451, 2024-09-11T23:40:02.065 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000211 2024-09-11T23:40:02.065 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:02.065 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_encrypted_bytes": 3184, 2024-09-11T23:40:02.065 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_encrypted_bytes": 559168 2024-09-11T23:40:02.065 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:02.066 INFO:teuthology.orchestra.run.smithi076.stdout: "AsyncMessenger::Worker-1": { 2024-09-11T23:40:02.066 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_messages": 51964, 2024-09-11T23:40:02.066 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_messages": 55128, 2024-09-11T23:40:02.066 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_bytes": 27552648, 2024-09-11T23:40:02.066 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_bytes": 155806222, 2024-09-11T23:40:02.066 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_created_connections": 21, 2024-09-11T23:40:02.066 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_active_connections": 7, 2024-09-11T23:40:02.066 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_total_time": 5.004687220, 2024-09-11T23:40:02.066 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_send_time": 1.933766348, 2024-09-11T23:40:02.066 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_recv_time": 1.654937375, 2024-09-11T23:40:02.066 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_fast_dispatch_time": 1.541963713, 2024-09-11T23:40:02.066 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T23:40:02.066 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 55130, 2024-09-11T23:40:02.067 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 37.329769764, 2024-09-11T23:40:02.067 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000677122 2024-09-11T23:40:02.067 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:02.067 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_handle_ack_lat": { 2024-09-11T23:40:02.067 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 51273, 2024-09-11T23:40:02.067 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.017078539, 2024-09-11T23:40:02.067 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000333 2024-09-11T23:40:02.067 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:02.067 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_encrypted_bytes": 2828368, 2024-09-11T23:40:02.067 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_encrypted_bytes": 538432 2024-09-11T23:40:02.067 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:02.067 INFO:teuthology.orchestra.run.smithi076.stdout: "AsyncMessenger::Worker-2": { 2024-09-11T23:40:02.067 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_messages": 284684, 2024-09-11T23:40:02.067 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_messages": 374832, 2024-09-11T23:40:02.067 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_bytes": 130234951, 2024-09-11T23:40:02.068 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_bytes": 275430396, 2024-09-11T23:40:02.068 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_created_connections": 286, 2024-09-11T23:40:02.068 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_active_connections": 11, 2024-09-11T23:40:02.068 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_total_time": 19.088846671, 2024-09-11T23:40:02.068 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_send_time": 8.374029174, 2024-09-11T23:40:02.068 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_recv_time": 15.991144122, 2024-09-11T23:40:02.068 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_fast_dispatch_time": 1.146951660, 2024-09-11T23:40:02.068 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T23:40:02.068 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 374823, 2024-09-11T23:40:02.068 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 68.543254442, 2024-09-11T23:40:02.068 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000182868 2024-09-11T23:40:02.068 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:02.068 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_handle_ack_lat": { 2024-09-11T23:40:02.069 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 528543, 2024-09-11T23:40:02.069 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.096489857, 2024-09-11T23:40:02.069 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000182 2024-09-11T23:40:02.069 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:02.069 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_encrypted_bytes": 86356944, 2024-09-11T23:40:02.069 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_encrypted_bytes": 168836704 2024-09-11T23:40:02.069 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:02.069 INFO:teuthology.orchestra.run.smithi076.stdout: "cct": { 2024-09-11T23:40:02.069 INFO:teuthology.orchestra.run.smithi076.stdout: "total_workers": 1, 2024-09-11T23:40:02.069 INFO:teuthology.orchestra.run.smithi076.stdout: "unhealthy_workers": 0 2024-09-11T23:40:02.069 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:02.069 INFO:teuthology.orchestra.run.smithi076.stdout: "finisher-MDSRank": { 2024-09-11T23:40:02.069 INFO:teuthology.orchestra.run.smithi076.stdout: "queue_len": 0, 2024-09-11T23:40:02.070 INFO:teuthology.orchestra.run.smithi076.stdout: "complete_latency": { 2024-09-11T23:40:02.070 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 14203, 2024-09-11T23:40:02.070 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 88.322053288, 2024-09-11T23:40:02.070 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.006218549 2024-09-11T23:40:02.070 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-11T23:40:02.070 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:02.070 INFO:teuthology.orchestra.run.smithi076.stdout: "finisher-PurgeQueue": { 2024-09-11T23:40:02.070 INFO:teuthology.orchestra.run.smithi076.stdout: "queue_len": 0, 2024-09-11T23:40:02.070 INFO:teuthology.orchestra.run.smithi076.stdout: "complete_latency": { 2024-09-11T23:40:02.070 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 29897, 2024-09-11T23:40:02.070 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 4.674554472, 2024-09-11T23:40:02.070 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000156355 2024-09-11T23:40:02.070 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-11T23:40:02.071 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:02.071 INFO:teuthology.orchestra.run.smithi076.stdout: "mds": { 2024-09-11T23:40:02.071 INFO:teuthology.orchestra.run.smithi076.stdout: "request": 115959, 2024-09-11T23:40:02.071 INFO:teuthology.orchestra.run.smithi076.stdout: "reply": 98653, 2024-09-11T23:40:02.071 INFO:teuthology.orchestra.run.smithi076.stdout: "reply_latency": { 2024-09-11T23:40:02.071 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 98653, 2024-09-11T23:40:02.071 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 1370.219830148, 2024-09-11T23:40:02.071 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.013889286 2024-09-11T23:40:02.071 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:02.071 INFO:teuthology.orchestra.run.smithi076.stdout: "slow_reply": 0, 2024-09-11T23:40:02.071 INFO:teuthology.orchestra.run.smithi076.stdout: "forward": 17280, 2024-09-11T23:40:02.071 INFO:teuthology.orchestra.run.smithi076.stdout: "dir_fetch_complete": 374, 2024-09-11T23:40:02.071 INFO:teuthology.orchestra.run.smithi076.stdout: "dir_fetch_keys": 1, 2024-09-11T23:40:02.071 INFO:teuthology.orchestra.run.smithi076.stdout: "dir_commit": 6484, 2024-09-11T23:40:02.072 INFO:teuthology.orchestra.run.smithi076.stdout: "dir_split": 85, 2024-09-11T23:40:02.072 INFO:teuthology.orchestra.run.smithi076.stdout: "dir_merge": 1, 2024-09-11T23:40:02.072 INFO:teuthology.orchestra.run.smithi076.stdout: "inodes": 19867, 2024-09-11T23:40:02.072 INFO:teuthology.orchestra.run.smithi076.stdout: "inodes_top": 13839, 2024-09-11T23:40:02.072 INFO:teuthology.orchestra.run.smithi076.stdout: "inodes_bottom": 5937, 2024-09-11T23:40:02.072 INFO:teuthology.orchestra.run.smithi076.stdout: "inodes_pin_tail": 91, 2024-09-11T23:40:02.072 INFO:teuthology.orchestra.run.smithi076.stdout: "inodes_pinned": 95, 2024-09-11T23:40:02.072 INFO:teuthology.orchestra.run.smithi076.stdout: "inodes_expired": 58011, 2024-09-11T23:40:02.072 INFO:teuthology.orchestra.run.smithi076.stdout: "inodes_with_caps": 1, 2024-09-11T23:40:02.072 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": 1, 2024-09-11T23:40:02.072 INFO:teuthology.orchestra.run.smithi076.stdout: "subtrees": 6, 2024-09-11T23:40:02.072 INFO:teuthology.orchestra.run.smithi076.stdout: "traverse": 198075, 2024-09-11T23:40:02.072 INFO:teuthology.orchestra.run.smithi076.stdout: "traverse_hit": 153557, 2024-09-11T23:40:02.072 INFO:teuthology.orchestra.run.smithi076.stdout: "traverse_forward": 17013, 2024-09-11T23:40:02.073 INFO:teuthology.orchestra.run.smithi076.stdout: "traverse_discover": 911, 2024-09-11T23:40:02.073 INFO:teuthology.orchestra.run.smithi076.stdout: "traverse_dir_fetch": 17, 2024-09-11T23:40:02.073 INFO:teuthology.orchestra.run.smithi076.stdout: "traverse_remote_ino": 0, 2024-09-11T23:40:02.073 INFO:teuthology.orchestra.run.smithi076.stdout: "traverse_lock": 417, 2024-09-11T23:40:02.073 INFO:teuthology.orchestra.run.smithi076.stdout: "load_cent": 1427, 2024-09-11T23:40:02.073 INFO:teuthology.orchestra.run.smithi076.stdout: "q": 0, 2024-09-11T23:40:02.073 INFO:teuthology.orchestra.run.smithi076.stdout: "exported": 209, 2024-09-11T23:40:02.073 INFO:teuthology.orchestra.run.smithi076.stdout: "exported_inodes": 40277, 2024-09-11T23:40:02.073 INFO:teuthology.orchestra.run.smithi076.stdout: "imported": 254, 2024-09-11T23:40:02.073 INFO:teuthology.orchestra.run.smithi076.stdout: "imported_inodes": 31069, 2024-09-11T23:40:02.073 INFO:teuthology.orchestra.run.smithi076.stdout: "openino_dir_fetch": 0, 2024-09-11T23:40:02.073 INFO:teuthology.orchestra.run.smithi076.stdout: "openino_backtrace_fetch": 570, 2024-09-11T23:40:02.073 INFO:teuthology.orchestra.run.smithi076.stdout: "openino_peer_discover": 0, 2024-09-11T23:40:02.074 INFO:teuthology.orchestra.run.smithi076.stdout: "root_rfiles": 4, 2024-09-11T23:40:02.074 INFO:teuthology.orchestra.run.smithi076.stdout: "root_rbytes": 234, 2024-09-11T23:40:02.074 INFO:teuthology.orchestra.run.smithi076.stdout: "root_rsnaps": 6, 2024-09-11T23:40:02.074 INFO:teuthology.orchestra.run.smithi076.stdout: "scrub_backtrace_fetch": 570, 2024-09-11T23:40:02.074 INFO:teuthology.orchestra.run.smithi076.stdout: "scrub_set_tag": 0, 2024-09-11T23:40:02.074 INFO:teuthology.orchestra.run.smithi076.stdout: "scrub_backtrace_repaired": 0, 2024-09-11T23:40:02.074 INFO:teuthology.orchestra.run.smithi076.stdout: "scrub_inotable_repaired": 0, 2024-09-11T23:40:02.074 INFO:teuthology.orchestra.run.smithi076.stdout: "scrub_dir_inodes": 28, 2024-09-11T23:40:02.074 INFO:teuthology.orchestra.run.smithi076.stdout: "scrub_dir_base_inodes": 3, 2024-09-11T23:40:02.074 INFO:teuthology.orchestra.run.smithi076.stdout: "scrub_dirfrag_rstats": 25, 2024-09-11T23:40:02.074 INFO:teuthology.orchestra.run.smithi076.stdout: "scrub_file_inodes": 542, 2024-09-11T23:40:02.074 INFO:teuthology.orchestra.run.smithi076.stdout: "handle_inode_file_caps": 391, 2024-09-11T23:40:02.074 INFO:teuthology.orchestra.run.smithi076.stdout: "ceph_cap_op_revoke": 51939, 2024-09-11T23:40:02.074 INFO:teuthology.orchestra.run.smithi076.stdout: "ceph_cap_op_grant": 45930, 2024-09-11T23:40:02.075 INFO:teuthology.orchestra.run.smithi076.stdout: "ceph_cap_op_trunc": 5, 2024-09-11T23:40:02.075 INFO:teuthology.orchestra.run.smithi076.stdout: "ceph_cap_op_flushsnap_ack": 9, 2024-09-11T23:40:02.075 INFO:teuthology.orchestra.run.smithi076.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-11T23:40:02.075 INFO:teuthology.orchestra.run.smithi076.stdout: "handle_client_caps": 70907, 2024-09-11T23:40:02.075 INFO:teuthology.orchestra.run.smithi076.stdout: "handle_client_caps_dirty": 18002, 2024-09-11T23:40:02.075 INFO:teuthology.orchestra.run.smithi076.stdout: "handle_client_cap_release": 1527, 2024-09-11T23:40:02.075 INFO:teuthology.orchestra.run.smithi076.stdout: "process_request_cap_release": 37097 2024-09-11T23:40:02.075 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:02.075 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_cache": { 2024-09-11T23:40:02.075 INFO:teuthology.orchestra.run.smithi076.stdout: "dir_update": 5068, 2024-09-11T23:40:02.075 INFO:teuthology.orchestra.run.smithi076.stdout: "dir_update_receipt": 10989, 2024-09-11T23:40:02.075 INFO:teuthology.orchestra.run.smithi076.stdout: "dir_try_discover": 6416, 2024-09-11T23:40:02.075 INFO:teuthology.orchestra.run.smithi076.stdout: "dir_send_discover": 7717, 2024-09-11T23:40:02.075 INFO:teuthology.orchestra.run.smithi076.stdout: "dir_handle_discover": 5449, 2024-09-11T23:40:02.076 INFO:teuthology.orchestra.run.smithi076.stdout: "num_strays": 82, 2024-09-11T23:40:02.076 INFO:teuthology.orchestra.run.smithi076.stdout: "num_strays_delayed": 0, 2024-09-11T23:40:02.076 INFO:teuthology.orchestra.run.smithi076.stdout: "num_strays_enqueuing": 0, 2024-09-11T23:40:02.076 INFO:teuthology.orchestra.run.smithi076.stdout: "strays_created": 15671, 2024-09-11T23:40:02.076 INFO:teuthology.orchestra.run.smithi076.stdout: "strays_enqueued": 15590, 2024-09-11T23:40:02.076 INFO:teuthology.orchestra.run.smithi076.stdout: "strays_reintegrated": 0, 2024-09-11T23:40:02.076 INFO:teuthology.orchestra.run.smithi076.stdout: "strays_migrated": 0, 2024-09-11T23:40:02.076 INFO:teuthology.orchestra.run.smithi076.stdout: "num_recovering_processing": 0, 2024-09-11T23:40:02.076 INFO:teuthology.orchestra.run.smithi076.stdout: "num_recovering_enqueued": 0, 2024-09-11T23:40:02.076 INFO:teuthology.orchestra.run.smithi076.stdout: "num_recovering_prioritized": 0, 2024-09-11T23:40:02.076 INFO:teuthology.orchestra.run.smithi076.stdout: "recovery_started": 0, 2024-09-11T23:40:02.076 INFO:teuthology.orchestra.run.smithi076.stdout: "recovery_completed": 0, 2024-09-11T23:40:02.076 INFO:teuthology.orchestra.run.smithi076.stdout: "ireq_quiesce_path": 18, 2024-09-11T23:40:02.077 INFO:teuthology.orchestra.run.smithi076.stdout: "ireq_quiesce_inode": 183293, 2024-09-11T23:40:02.077 INFO:teuthology.orchestra.run.smithi076.stdout: "ireq_enqueue_scrub": 3, 2024-09-11T23:40:02.077 INFO:teuthology.orchestra.run.smithi076.stdout: "ireq_exportdir": 209, 2024-09-11T23:40:02.077 INFO:teuthology.orchestra.run.smithi076.stdout: "ireq_flush": 0, 2024-09-11T23:40:02.077 INFO:teuthology.orchestra.run.smithi076.stdout: "ireq_fragmentdir": 86, 2024-09-11T23:40:02.077 INFO:teuthology.orchestra.run.smithi076.stdout: "ireq_fragstats": 0, 2024-09-11T23:40:02.077 INFO:teuthology.orchestra.run.smithi076.stdout: "ireq_inodestats": 0 2024-09-11T23:40:02.077 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:02.077 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_log": { 2024-09-11T23:40:02.077 INFO:teuthology.orchestra.run.smithi076.stdout: "evlrg": 2, 2024-09-11T23:40:02.077 INFO:teuthology.orchestra.run.smithi076.stdout: "evadd": 99783, 2024-09-11T23:40:02.077 INFO:teuthology.orchestra.run.smithi076.stdout: "evex": 99773, 2024-09-11T23:40:02.077 INFO:teuthology.orchestra.run.smithi076.stdout: "evtrm": 99773, 2024-09-11T23:40:02.077 INFO:teuthology.orchestra.run.smithi076.stdout: "ev": 10, 2024-09-11T23:40:02.078 INFO:teuthology.orchestra.run.smithi076.stdout: "evexg": 0, 2024-09-11T23:40:02.078 INFO:teuthology.orchestra.run.smithi076.stdout: "evexd": 128, 2024-09-11T23:40:02.078 INFO:teuthology.orchestra.run.smithi076.stdout: "segadd": 207, 2024-09-11T23:40:02.078 INFO:teuthology.orchestra.run.smithi076.stdout: "segex": 206, 2024-09-11T23:40:02.078 INFO:teuthology.orchestra.run.smithi076.stdout: "segtrm": 206, 2024-09-11T23:40:02.078 INFO:teuthology.orchestra.run.smithi076.stdout: "seg": 1, 2024-09-11T23:40:02.078 INFO:teuthology.orchestra.run.smithi076.stdout: "segmjr": 1, 2024-09-11T23:40:02.078 INFO:teuthology.orchestra.run.smithi076.stdout: "segexg": 0, 2024-09-11T23:40:02.078 INFO:teuthology.orchestra.run.smithi076.stdout: "segexd": 1, 2024-09-11T23:40:02.078 INFO:teuthology.orchestra.run.smithi076.stdout: "expos": 274743605, 2024-09-11T23:40:02.078 INFO:teuthology.orchestra.run.smithi076.stdout: "wrpos": 274766055, 2024-09-11T23:40:02.078 INFO:teuthology.orchestra.run.smithi076.stdout: "rdpos": 0, 2024-09-11T23:40:02.078 INFO:teuthology.orchestra.run.smithi076.stdout: "jlat": { 2024-09-11T23:40:02.078 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 10846, 2024-09-11T23:40:02.079 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 1177.614803421, 2024-09-11T23:40:02.079 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.108575954 2024-09-11T23:40:02.079 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:02.079 INFO:teuthology.orchestra.run.smithi076.stdout: "replayed": 0 2024-09-11T23:40:02.079 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:02.079 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_mem": { 2024-09-11T23:40:02.079 INFO:teuthology.orchestra.run.smithi076.stdout: "ino": 8826, 2024-09-11T23:40:02.079 INFO:teuthology.orchestra.run.smithi076.stdout: "ino+": 60266, 2024-09-11T23:40:02.079 INFO:teuthology.orchestra.run.smithi076.stdout: "ino-": 51440, 2024-09-11T23:40:02.079 INFO:teuthology.orchestra.run.smithi076.stdout: "dir": 587, 2024-09-11T23:40:02.079 INFO:teuthology.orchestra.run.smithi076.stdout: "dir+": 6283, 2024-09-11T23:40:02.079 INFO:teuthology.orchestra.run.smithi076.stdout: "dir-": 5696, 2024-09-11T23:40:02.079 INFO:teuthology.orchestra.run.smithi076.stdout: "dn": 19927, 2024-09-11T23:40:02.080 INFO:teuthology.orchestra.run.smithi076.stdout: "dn+": 98122, 2024-09-11T23:40:02.080 INFO:teuthology.orchestra.run.smithi076.stdout: "dn-": 78195, 2024-09-11T23:40:02.080 INFO:teuthology.orchestra.run.smithi076.stdout: "cap": 1, 2024-09-11T23:40:02.080 INFO:teuthology.orchestra.run.smithi076.stdout: "cap+": 34248, 2024-09-11T23:40:02.080 INFO:teuthology.orchestra.run.smithi076.stdout: "cap-": 34247, 2024-09-11T23:40:02.080 INFO:teuthology.orchestra.run.smithi076.stdout: "rss": 216848, 2024-09-11T23:40:02.080 INFO:teuthology.orchestra.run.smithi076.stdout: "heap": 231712 2024-09-11T23:40:02.080 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:02.080 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_server": { 2024-09-11T23:40:02.080 INFO:teuthology.orchestra.run.smithi076.stdout: "dispatch_client_request": 129374, 2024-09-11T23:40:02.080 INFO:teuthology.orchestra.run.smithi076.stdout: "dispatch_server_request": 2064, 2024-09-11T23:40:02.080 INFO:teuthology.orchestra.run.smithi076.stdout: "handle_client_request": 115959, 2024-09-11T23:40:02.080 INFO:teuthology.orchestra.run.smithi076.stdout: "handle_client_session": 142, 2024-09-11T23:40:02.081 INFO:teuthology.orchestra.run.smithi076.stdout: "handle_peer_request": 11148, 2024-09-11T23:40:02.081 INFO:teuthology.orchestra.run.smithi076.stdout: "req_create_latency": { 2024-09-11T23:40:02.081 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 13637, 2024-09-11T23:40:02.081 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 92.014676586, 2024-09-11T23:40:02.081 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.006747428 2024-09-11T23:40:02.081 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:02.081 INFO:teuthology.orchestra.run.smithi076.stdout: "req_getattr_latency": { 2024-09-11T23:40:02.081 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 1696, 2024-09-11T23:40:02.081 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 38.867500814, 2024-09-11T23:40:02.081 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.022917158 2024-09-11T23:40:02.081 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:02.081 INFO:teuthology.orchestra.run.smithi076.stdout: "req_getfilelock_latency": { 2024-09-11T23:40:02.081 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-09-11T23:40:02.081 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-09-11T23:40:02.082 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-09-11T23:40:02.082 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:02.082 INFO:teuthology.orchestra.run.smithi076.stdout: "req_link_latency": { 2024-09-11T23:40:02.082 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-09-11T23:40:02.082 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-09-11T23:40:02.082 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-09-11T23:40:02.082 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:02.082 INFO:teuthology.orchestra.run.smithi076.stdout: "req_lookup_latency": { 2024-09-11T23:40:02.082 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 49166, 2024-09-11T23:40:02.082 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 912.218326601, 2024-09-11T23:40:02.082 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.018553844 2024-09-11T23:40:02.082 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:02.082 INFO:teuthology.orchestra.run.smithi076.stdout: "req_lookuphash_latency": { 2024-09-11T23:40:02.082 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-09-11T23:40:02.082 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-09-11T23:40:02.083 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-09-11T23:40:02.083 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:02.083 INFO:teuthology.orchestra.run.smithi076.stdout: "req_lookupino_latency": { 2024-09-11T23:40:02.083 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-09-11T23:40:02.083 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-09-11T23:40:02.083 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-09-11T23:40:02.083 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:02.083 INFO:teuthology.orchestra.run.smithi076.stdout: "req_lookupname_latency": { 2024-09-11T23:40:02.083 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-09-11T23:40:02.083 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-09-11T23:40:02.083 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-09-11T23:40:02.083 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:02.084 INFO:teuthology.orchestra.run.smithi076.stdout: "req_lookupparent_latency": { 2024-09-11T23:40:02.084 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-09-11T23:40:02.084 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-09-11T23:40:02.084 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-09-11T23:40:02.084 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:02.084 INFO:teuthology.orchestra.run.smithi076.stdout: "req_lookupsnap_latency": { 2024-09-11T23:40:02.084 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-09-11T23:40:02.084 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-09-11T23:40:02.085 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-09-11T23:40:02.085 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:02.085 INFO:teuthology.orchestra.run.smithi076.stdout: "req_lssnap_latency": { 2024-09-11T23:40:02.085 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 22, 2024-09-11T23:40:02.085 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.008847899, 2024-09-11T23:40:02.085 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000402177 2024-09-11T23:40:02.085 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:02.085 INFO:teuthology.orchestra.run.smithi076.stdout: "req_mkdir_latency": { 2024-09-11T23:40:02.085 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 97, 2024-09-11T23:40:02.085 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 3.867937924, 2024-09-11T23:40:02.085 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.039875648 2024-09-11T23:40:02.085 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:02.085 INFO:teuthology.orchestra.run.smithi076.stdout: "req_mknod_latency": { 2024-09-11T23:40:02.086 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-09-11T23:40:02.086 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-09-11T23:40:02.086 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-09-11T23:40:02.086 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:02.086 INFO:teuthology.orchestra.run.smithi076.stdout: "req_mksnap_latency": { 2024-09-11T23:40:02.086 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 22, 2024-09-11T23:40:02.086 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 16.130359298, 2024-09-11T23:40:02.086 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.733198149 2024-09-11T23:40:02.086 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:02.086 INFO:teuthology.orchestra.run.smithi076.stdout: "req_open_latency": { 2024-09-11T23:40:02.086 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 5923, 2024-09-11T23:40:02.086 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 24.132609900, 2024-09-11T23:40:02.086 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.004074389 2024-09-11T23:40:02.086 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:02.086 INFO:teuthology.orchestra.run.smithi076.stdout: "req_readdir_latency": { 2024-09-11T23:40:02.087 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 771, 2024-09-11T23:40:02.087 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 7.433133400, 2024-09-11T23:40:02.087 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.009640899 2024-09-11T23:40:02.087 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:02.087 INFO:teuthology.orchestra.run.smithi076.stdout: "req_rename_latency": { 2024-09-11T23:40:02.087 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 13245, 2024-09-11T23:40:02.087 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 177.545221708, 2024-09-11T23:40:02.087 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.013404697 2024-09-11T23:40:02.087 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:02.087 INFO:teuthology.orchestra.run.smithi076.stdout: "req_renamesnap_latency": { 2024-09-11T23:40:02.087 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-09-11T23:40:02.087 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-09-11T23:40:02.087 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-09-11T23:40:02.087 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:02.087 INFO:teuthology.orchestra.run.smithi076.stdout: "req_snapdiff_latency": { 2024-09-11T23:40:02.088 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-09-11T23:40:02.088 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-09-11T23:40:02.088 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-09-11T23:40:02.088 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:02.088 INFO:teuthology.orchestra.run.smithi076.stdout: "req_rmdir_latency": { 2024-09-11T23:40:02.088 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 85, 2024-09-11T23:40:02.088 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 22.860026238, 2024-09-11T23:40:02.088 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.268941485 2024-09-11T23:40:02.088 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:02.088 INFO:teuthology.orchestra.run.smithi076.stdout: "req_rmsnap_latency": { 2024-09-11T23:40:02.088 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 16, 2024-09-11T23:40:02.088 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 2.888120327, 2024-09-11T23:40:02.088 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.180507520 2024-09-11T23:40:02.088 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:02.089 INFO:teuthology.orchestra.run.smithi076.stdout: "req_rmxattr_latency": { 2024-09-11T23:40:02.089 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-09-11T23:40:02.089 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-09-11T23:40:02.089 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-09-11T23:40:02.089 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:02.089 INFO:teuthology.orchestra.run.smithi076.stdout: "req_setattr_latency": { 2024-09-11T23:40:02.089 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 6, 2024-09-11T23:40:02.089 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.027903256, 2024-09-11T23:40:02.089 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.004650542 2024-09-11T23:40:02.089 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:02.089 INFO:teuthology.orchestra.run.smithi076.stdout: "req_setdirlayout_latency": { 2024-09-11T23:40:02.089 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-09-11T23:40:02.089 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-09-11T23:40:02.090 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-09-11T23:40:02.090 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:02.090 INFO:teuthology.orchestra.run.smithi076.stdout: "req_setfilelock_latency": { 2024-09-11T23:40:02.090 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-09-11T23:40:02.090 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-09-11T23:40:02.090 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-09-11T23:40:02.090 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:02.090 INFO:teuthology.orchestra.run.smithi076.stdout: "req_setlayout_latency": { 2024-09-11T23:40:02.090 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-09-11T23:40:02.090 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-09-11T23:40:02.090 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-09-11T23:40:02.090 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:02.090 INFO:teuthology.orchestra.run.smithi076.stdout: "req_setxattr_latency": { 2024-09-11T23:40:02.090 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 7, 2024-09-11T23:40:02.091 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.020179114, 2024-09-11T23:40:02.091 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.002882730 2024-09-11T23:40:02.091 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:02.091 INFO:teuthology.orchestra.run.smithi076.stdout: "req_symlink_latency": { 2024-09-11T23:40:02.091 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 3, 2024-09-11T23:40:02.091 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.002058550, 2024-09-11T23:40:02.091 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000686183 2024-09-11T23:40:02.091 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:02.091 INFO:teuthology.orchestra.run.smithi076.stdout: "req_unlink_latency": { 2024-09-11T23:40:02.091 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 13957, 2024-09-11T23:40:02.091 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 72.202928533, 2024-09-11T23:40:02.091 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.005173241 2024-09-11T23:40:02.091 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:02.091 INFO:teuthology.orchestra.run.smithi076.stdout: "cap_revoke_eviction": 0, 2024-09-11T23:40:02.092 INFO:teuthology.orchestra.run.smithi076.stdout: "cap_acquisition_throttle": 0, 2024-09-11T23:40:02.092 INFO:teuthology.orchestra.run.smithi076.stdout: "req_getvxattr_latency": { 2024-09-11T23:40:02.092 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-09-11T23:40:02.092 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-09-11T23:40:02.092 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-09-11T23:40:02.092 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-11T23:40:02.092 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:02.092 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_sessions": { 2024-09-11T23:40:02.092 INFO:teuthology.orchestra.run.smithi076.stdout: "session_count": 2, 2024-09-11T23:40:02.092 INFO:teuthology.orchestra.run.smithi076.stdout: "session_add": 5, 2024-09-11T23:40:02.092 INFO:teuthology.orchestra.run.smithi076.stdout: "session_remove": 3, 2024-09-11T23:40:02.092 INFO:teuthology.orchestra.run.smithi076.stdout: "sessions_open": 2, 2024-09-11T23:40:02.092 INFO:teuthology.orchestra.run.smithi076.stdout: "sessions_stale": 0, 2024-09-11T23:40:02.092 INFO:teuthology.orchestra.run.smithi076.stdout: "total_load": 1384, 2024-09-11T23:40:02.093 INFO:teuthology.orchestra.run.smithi076.stdout: "average_load": 692, 2024-09-11T23:40:02.093 INFO:teuthology.orchestra.run.smithi076.stdout: "avg_session_uptime": 1278, 2024-09-11T23:40:02.093 INFO:teuthology.orchestra.run.smithi076.stdout: "mdthresh_evicted": 0 2024-09-11T23:40:02.093 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:02.093 INFO:teuthology.orchestra.run.smithi076.stdout: "mempool": { 2024-09-11T23:40:02.093 INFO:teuthology.orchestra.run.smithi076.stdout: "bloom_filter_bytes": 120, 2024-09-11T23:40:02.093 INFO:teuthology.orchestra.run.smithi076.stdout: "bloom_filter_items": 120, 2024-09-11T23:40:02.093 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_alloc_bytes": 0, 2024-09-11T23:40:02.093 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_alloc_items": 0, 2024-09-11T23:40:02.093 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_data_bytes": 0, 2024-09-11T23:40:02.093 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_data_items": 0, 2024-09-11T23:40:02.093 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-11T23:40:02.093 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_onode_items": 0, 2024-09-11T23:40:02.094 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-11T23:40:02.094 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_meta_items": 0, 2024-09-11T23:40:02.094 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_other_bytes": 0, 2024-09-11T23:40:02.094 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_other_items": 0, 2024-09-11T23:40:02.094 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-11T23:40:02.094 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_buffer_items": 0, 2024-09-11T23:40:02.094 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_extent_bytes": 0, 2024-09-11T23:40:02.094 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_extent_items": 0, 2024-09-11T23:40:02.094 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_blob_bytes": 0, 2024-09-11T23:40:02.094 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_blob_items": 0, 2024-09-11T23:40:02.094 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-11T23:40:02.094 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_shared_blob_items": 0, 2024-09-11T23:40:02.094 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-11T23:40:02.094 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_inline_bl_items": 0, 2024-09-11T23:40:02.095 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_fsck_bytes": 0, 2024-09-11T23:40:02.095 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_fsck_items": 0, 2024-09-11T23:40:02.095 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_txc_bytes": 0, 2024-09-11T23:40:02.095 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_txc_items": 0, 2024-09-11T23:40:02.095 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-11T23:40:02.095 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_writing_deferred_items": 0, 2024-09-11T23:40:02.095 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_writing_bytes": 0, 2024-09-11T23:40:02.095 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_writing_items": 0, 2024-09-11T23:40:02.095 INFO:teuthology.orchestra.run.smithi076.stdout: "bluefs_bytes": 0, 2024-09-11T23:40:02.095 INFO:teuthology.orchestra.run.smithi076.stdout: "bluefs_items": 0, 2024-09-11T23:40:02.095 INFO:teuthology.orchestra.run.smithi076.stdout: "bluefs_file_reader_bytes": 0, 2024-09-11T23:40:02.095 INFO:teuthology.orchestra.run.smithi076.stdout: "bluefs_file_reader_items": 0, 2024-09-11T23:40:02.095 INFO:teuthology.orchestra.run.smithi076.stdout: "bluefs_file_writer_bytes": 0, 2024-09-11T23:40:02.095 INFO:teuthology.orchestra.run.smithi076.stdout: "bluefs_file_writer_items": 0, 2024-09-11T23:40:02.095 INFO:teuthology.orchestra.run.smithi076.stdout: "buffer_anon_bytes": 1318820, 2024-09-11T23:40:02.096 INFO:teuthology.orchestra.run.smithi076.stdout: "buffer_anon_items": 8885, 2024-09-11T23:40:02.096 INFO:teuthology.orchestra.run.smithi076.stdout: "buffer_meta_bytes": 0, 2024-09-11T23:40:02.096 INFO:teuthology.orchestra.run.smithi076.stdout: "buffer_meta_items": 0, 2024-09-11T23:40:02.096 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_bytes": 0, 2024-09-11T23:40:02.096 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_items": 0, 2024-09-11T23:40:02.096 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_mapbl_bytes": 0, 2024-09-11T23:40:02.096 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_mapbl_items": 0, 2024-09-11T23:40:02.096 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_pglog_bytes": 0, 2024-09-11T23:40:02.096 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_pglog_items": 0, 2024-09-11T23:40:02.096 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_bytes": 8224, 2024-09-11T23:40:02.096 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_items": 181, 2024-09-11T23:40:02.096 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_mapping_bytes": 0, 2024-09-11T23:40:02.096 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_mapping_items": 0, 2024-09-11T23:40:02.096 INFO:teuthology.orchestra.run.smithi076.stdout: "pgmap_bytes": 0, 2024-09-11T23:40:02.097 INFO:teuthology.orchestra.run.smithi076.stdout: "pgmap_items": 0, 2024-09-11T23:40:02.097 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_co_bytes": 35785954, 2024-09-11T23:40:02.097 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_co_items": 353182, 2024-09-11T23:40:02.097 INFO:teuthology.orchestra.run.smithi076.stdout: "unittest_1_bytes": 0, 2024-09-11T23:40:02.097 INFO:teuthology.orchestra.run.smithi076.stdout: "unittest_1_items": 0, 2024-09-11T23:40:02.097 INFO:teuthology.orchestra.run.smithi076.stdout: "unittest_2_bytes": 0, 2024-09-11T23:40:02.097 INFO:teuthology.orchestra.run.smithi076.stdout: "unittest_2_items": 0 2024-09-11T23:40:02.097 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:02.097 INFO:teuthology.orchestra.run.smithi076.stdout: "objecter": { 2024-09-11T23:40:02.097 INFO:teuthology.orchestra.run.smithi076.stdout: "op_active": 0, 2024-09-11T23:40:02.097 INFO:teuthology.orchestra.run.smithi076.stdout: "op_laggy": 0, 2024-09-11T23:40:02.097 INFO:teuthology.orchestra.run.smithi076.stdout: "op_send": 62567, 2024-09-11T23:40:02.097 INFO:teuthology.orchestra.run.smithi076.stdout: "op_send_bytes": 342289716, 2024-09-11T23:40:02.097 INFO:teuthology.orchestra.run.smithi076.stdout: "op_resend": 33, 2024-09-11T23:40:02.097 INFO:teuthology.orchestra.run.smithi076.stdout: "op_reply": 62567, 2024-09-11T23:40:02.098 INFO:teuthology.orchestra.run.smithi076.stdout: "op_latency": { 2024-09-11T23:40:02.098 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 62567, 2024-09-11T23:40:02.098 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 20268.162594922, 2024-09-11T23:40:02.098 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.323943334 2024-09-11T23:40:02.098 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:02.098 INFO:teuthology.orchestra.run.smithi076.stdout: "op_inflight": 0, 2024-09-11T23:40:02.098 INFO:teuthology.orchestra.run.smithi076.stdout: "oplen_avg": { 2024-09-11T23:40:02.098 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 62567, 2024-09-11T23:40:02.098 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 128443 2024-09-11T23:40:02.098 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:02.098 INFO:teuthology.orchestra.run.smithi076.stdout: "op": 62567, 2024-09-11T23:40:02.098 INFO:teuthology.orchestra.run.smithi076.stdout: "op_r": 1382, 2024-09-11T23:40:02.098 INFO:teuthology.orchestra.run.smithi076.stdout: "op_w": 61185, 2024-09-11T23:40:02.098 INFO:teuthology.orchestra.run.smithi076.stdout: "op_rmw": 0, 2024-09-11T23:40:02.098 INFO:teuthology.orchestra.run.smithi076.stdout: "op_pg": 0, 2024-09-11T23:40:02.099 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_stat": 5665, 2024-09-11T23:40:02.099 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_create": 25924, 2024-09-11T23:40:02.099 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_read": 434, 2024-09-11T23:40:02.099 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_write": 11279, 2024-09-11T23:40:02.099 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_writefull": 573, 2024-09-11T23:40:02.099 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_writesame": 0, 2024-09-11T23:40:02.099 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_append": 0, 2024-09-11T23:40:02.099 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_zero": 1, 2024-09-11T23:40:02.099 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_truncate": 85, 2024-09-11T23:40:02.099 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_delete": 16231, 2024-09-11T23:40:02.099 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_mapext": 0, 2024-09-11T23:40:02.099 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_sparse_read": 0, 2024-09-11T23:40:02.099 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_clonerange": 0, 2024-09-11T23:40:02.099 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_getxattr": 586, 2024-09-11T23:40:02.099 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_setxattr": 51849, 2024-09-11T23:40:02.100 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_cmpxattr": 0, 2024-09-11T23:40:02.100 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_rmxattr": 0, 2024-09-11T23:40:02.100 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_resetxattrs": 0, 2024-09-11T23:40:02.100 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_call": 0, 2024-09-11T23:40:02.100 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_watch": 0, 2024-09-11T23:40:02.100 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_notify": 0, 2024-09-11T23:40:02.100 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_src_cmpxattr": 0, 2024-09-11T23:40:02.100 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_pgls": 0, 2024-09-11T23:40:02.100 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_pgls_filter": 0, 2024-09-11T23:40:02.100 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_other": 5, 2024-09-11T23:40:02.100 INFO:teuthology.orchestra.run.smithi076.stdout: "linger_active": 0, 2024-09-11T23:40:02.100 INFO:teuthology.orchestra.run.smithi076.stdout: "linger_send": 0, 2024-09-11T23:40:02.100 INFO:teuthology.orchestra.run.smithi076.stdout: "linger_resend": 0, 2024-09-11T23:40:02.100 INFO:teuthology.orchestra.run.smithi076.stdout: "linger_ping": 0, 2024-09-11T23:40:02.101 INFO:teuthology.orchestra.run.smithi076.stdout: "poolop_active": 0, 2024-09-11T23:40:02.101 INFO:teuthology.orchestra.run.smithi076.stdout: "poolop_send": 0, 2024-09-11T23:40:02.101 INFO:teuthology.orchestra.run.smithi076.stdout: "poolop_resend": 0, 2024-09-11T23:40:02.101 INFO:teuthology.orchestra.run.smithi076.stdout: "poolstat_active": 0, 2024-09-11T23:40:02.101 INFO:teuthology.orchestra.run.smithi076.stdout: "poolstat_send": 0, 2024-09-11T23:40:02.101 INFO:teuthology.orchestra.run.smithi076.stdout: "poolstat_resend": 0, 2024-09-11T23:40:02.101 INFO:teuthology.orchestra.run.smithi076.stdout: "statfs_active": 0, 2024-09-11T23:40:02.101 INFO:teuthology.orchestra.run.smithi076.stdout: "statfs_send": 0, 2024-09-11T23:40:02.101 INFO:teuthology.orchestra.run.smithi076.stdout: "statfs_resend": 0, 2024-09-11T23:40:02.101 INFO:teuthology.orchestra.run.smithi076.stdout: "command_active": 0, 2024-09-11T23:40:02.101 INFO:teuthology.orchestra.run.smithi076.stdout: "command_send": 0, 2024-09-11T23:40:02.101 INFO:teuthology.orchestra.run.smithi076.stdout: "command_resend": 0, 2024-09-11T23:40:02.101 INFO:teuthology.orchestra.run.smithi076.stdout: "map_epoch": 116, 2024-09-11T23:40:02.102 INFO:teuthology.orchestra.run.smithi076.stdout: "map_full": 0, 2024-09-11T23:40:02.102 INFO:teuthology.orchestra.run.smithi076.stdout: "map_inc": 33, 2024-09-11T23:40:02.102 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_sessions": 12, 2024-09-11T23:40:02.102 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_session_open": 12, 2024-09-11T23:40:02.102 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_session_close": 0, 2024-09-11T23:40:02.102 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_laggy": 0, 2024-09-11T23:40:02.102 INFO:teuthology.orchestra.run.smithi076.stdout: "omap_wr": 12730, 2024-09-11T23:40:02.102 INFO:teuthology.orchestra.run.smithi076.stdout: "omap_rd": 750, 2024-09-11T23:40:02.102 INFO:teuthology.orchestra.run.smithi076.stdout: "omap_del": 2331 2024-09-11T23:40:02.102 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:02.102 INFO:teuthology.orchestra.run.smithi076.stdout: "oft": { 2024-09-11T23:40:02.102 INFO:teuthology.orchestra.run.smithi076.stdout: "omap_total_objs": 1, 2024-09-11T23:40:02.102 INFO:teuthology.orchestra.run.smithi076.stdout: "omap_total_kv_pairs": 0, 2024-09-11T23:40:02.102 INFO:teuthology.orchestra.run.smithi076.stdout: "omap_total_updates": 0, 2024-09-11T23:40:02.103 INFO:teuthology.orchestra.run.smithi076.stdout: "omap_total_removes": 0 2024-09-11T23:40:02.103 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:02.103 INFO:teuthology.orchestra.run.smithi076.stdout: "osd-slow-ops": { 2024-09-11T23:40:02.103 INFO:teuthology.orchestra.run.smithi076.stdout: "slow_ops_count": 183219 2024-09-11T23:40:02.103 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:02.103 INFO:teuthology.orchestra.run.smithi076.stdout: "purge_queue": { 2024-09-11T23:40:02.103 INFO:teuthology.orchestra.run.smithi076.stdout: "pq_executing_ops": 0, 2024-09-11T23:40:02.103 INFO:teuthology.orchestra.run.smithi076.stdout: "pq_executing_ops_high_water": 6, 2024-09-11T23:40:02.103 INFO:teuthology.orchestra.run.smithi076.stdout: "pq_executing": 0, 2024-09-11T23:40:02.103 INFO:teuthology.orchestra.run.smithi076.stdout: "pq_executing_high_water": 6, 2024-09-11T23:40:02.103 INFO:teuthology.orchestra.run.smithi076.stdout: "pq_executed_ops": 15590, 2024-09-11T23:40:02.103 INFO:teuthology.orchestra.run.smithi076.stdout: "pq_executed": 15590, 2024-09-11T23:40:02.103 INFO:teuthology.orchestra.run.smithi076.stdout: "pq_item_in_journal": 0 2024-09-11T23:40:02.104 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:02.104 INFO:teuthology.orchestra.run.smithi076.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-11T23:40:02.104 INFO:teuthology.orchestra.run.smithi076.stdout: "val": 0, 2024-09-11T23:40:02.104 INFO:teuthology.orchestra.run.smithi076.stdout: "max": 104857600, 2024-09-11T23:40:02.104 INFO:teuthology.orchestra.run.smithi076.stdout: "get_started": 0, 2024-09-11T23:40:02.104 INFO:teuthology.orchestra.run.smithi076.stdout: "get": 404976, 2024-09-11T23:40:02.104 INFO:teuthology.orchestra.run.smithi076.stdout: "get_sum": 140522910, 2024-09-11T23:40:02.104 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_fail": 0, 2024-09-11T23:40:02.104 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_success": 404976, 2024-09-11T23:40:02.104 INFO:teuthology.orchestra.run.smithi076.stdout: "take": 0, 2024-09-11T23:40:02.104 INFO:teuthology.orchestra.run.smithi076.stdout: "take_sum": 0, 2024-09-11T23:40:02.104 INFO:teuthology.orchestra.run.smithi076.stdout: "put": 404976, 2024-09-11T23:40:02.104 INFO:teuthology.orchestra.run.smithi076.stdout: "put_sum": 140522910, 2024-09-11T23:40:02.104 INFO:teuthology.orchestra.run.smithi076.stdout: "wait": { 2024-09-11T23:40:02.104 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-09-11T23:40:02.105 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-09-11T23:40:02.105 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-09-11T23:40:02.105 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-11T23:40:02.105 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:02.105 INFO:teuthology.orchestra.run.smithi076.stdout: "throttle-objecter_bytes": { 2024-09-11T23:40:02.105 INFO:teuthology.orchestra.run.smithi076.stdout: "val": 0, 2024-09-11T23:40:02.105 INFO:teuthology.orchestra.run.smithi076.stdout: "max": 104857600, 2024-09-11T23:40:02.105 INFO:teuthology.orchestra.run.smithi076.stdout: "get_started": 0, 2024-09-11T23:40:02.105 INFO:teuthology.orchestra.run.smithi076.stdout: "get": 0, 2024-09-11T23:40:02.105 INFO:teuthology.orchestra.run.smithi076.stdout: "get_sum": 0, 2024-09-11T23:40:02.105 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_fail": 0, 2024-09-11T23:40:02.105 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_success": 0, 2024-09-11T23:40:02.106 INFO:teuthology.orchestra.run.smithi076.stdout: "take": 62567, 2024-09-11T23:40:02.106 INFO:teuthology.orchestra.run.smithi076.stdout: "take_sum": 344522022, 2024-09-11T23:40:02.106 INFO:teuthology.orchestra.run.smithi076.stdout: "put": 62567, 2024-09-11T23:40:02.106 INFO:teuthology.orchestra.run.smithi076.stdout: "put_sum": 344522022, 2024-09-11T23:40:02.106 INFO:teuthology.orchestra.run.smithi076.stdout: "wait": { 2024-09-11T23:40:02.106 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-09-11T23:40:02.106 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-09-11T23:40:02.106 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-09-11T23:40:02.106 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-11T23:40:02.106 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:02.106 INFO:teuthology.orchestra.run.smithi076.stdout: "throttle-objecter_ops": { 2024-09-11T23:40:02.106 INFO:teuthology.orchestra.run.smithi076.stdout: "val": 0, 2024-09-11T23:40:02.106 INFO:teuthology.orchestra.run.smithi076.stdout: "max": 1024, 2024-09-11T23:40:02.106 INFO:teuthology.orchestra.run.smithi076.stdout: "get_started": 0, 2024-09-11T23:40:02.106 INFO:teuthology.orchestra.run.smithi076.stdout: "get": 0, 2024-09-11T23:40:02.107 INFO:teuthology.orchestra.run.smithi076.stdout: "get_sum": 0, 2024-09-11T23:40:02.107 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_fail": 0, 2024-09-11T23:40:02.107 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_success": 0, 2024-09-11T23:40:02.107 INFO:teuthology.orchestra.run.smithi076.stdout: "take": 62567, 2024-09-11T23:40:02.107 INFO:teuthology.orchestra.run.smithi076.stdout: "take_sum": 62567, 2024-09-11T23:40:02.107 INFO:teuthology.orchestra.run.smithi076.stdout: "put": 62567, 2024-09-11T23:40:02.107 INFO:teuthology.orchestra.run.smithi076.stdout: "put_sum": 62567, 2024-09-11T23:40:02.107 INFO:teuthology.orchestra.run.smithi076.stdout: "wait": { 2024-09-11T23:40:02.107 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-09-11T23:40:02.107 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-09-11T23:40:02.107 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-09-11T23:40:02.107 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-11T23:40:02.107 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:02.107 INFO:teuthology.orchestra.run.smithi076.stdout: "throttle-write_buf_throttle": { 2024-09-11T23:40:02.107 INFO:teuthology.orchestra.run.smithi076.stdout: "val": 0, 2024-09-11T23:40:02.108 INFO:teuthology.orchestra.run.smithi076.stdout: "max": 3758096384, 2024-09-11T23:40:02.108 INFO:teuthology.orchestra.run.smithi076.stdout: "get_started": 0, 2024-09-11T23:40:02.108 INFO:teuthology.orchestra.run.smithi076.stdout: "get": 15590, 2024-09-11T23:40:02.108 INFO:teuthology.orchestra.run.smithi076.stdout: "get_sum": 2238294, 2024-09-11T23:40:02.108 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_fail": 0, 2024-09-11T23:40:02.108 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_success": 15590, 2024-09-11T23:40:02.108 INFO:teuthology.orchestra.run.smithi076.stdout: "take": 0, 2024-09-11T23:40:02.108 INFO:teuthology.orchestra.run.smithi076.stdout: "take_sum": 0, 2024-09-11T23:40:02.108 INFO:teuthology.orchestra.run.smithi076.stdout: "put": 432, 2024-09-11T23:40:02.108 INFO:teuthology.orchestra.run.smithi076.stdout: "put_sum": 2238294, 2024-09-11T23:40:02.108 INFO:teuthology.orchestra.run.smithi076.stdout: "wait": { 2024-09-11T23:40:02.108 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-09-11T23:40:02.108 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-09-11T23:40:02.108 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-09-11T23:40:02.109 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-11T23:40:02.109 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:02.109 INFO:teuthology.orchestra.run.smithi076.stdout: "throttle-write_buf_throttle-0x55bd4deb0960": { 2024-09-11T23:40:02.109 INFO:teuthology.orchestra.run.smithi076.stdout: "val": 84, 2024-09-11T23:40:02.109 INFO:teuthology.orchestra.run.smithi076.stdout: "max": 3758096384, 2024-09-11T23:40:02.109 INFO:teuthology.orchestra.run.smithi076.stdout: "get_started": 0, 2024-09-11T23:40:02.109 INFO:teuthology.orchestra.run.smithi076.stdout: "get": 99783, 2024-09-11T23:40:02.109 INFO:teuthology.orchestra.run.smithi076.stdout: "get_sum": 270571751, 2024-09-11T23:40:02.109 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_fail": 0, 2024-09-11T23:40:02.109 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_success": 99783, 2024-09-11T23:40:02.109 INFO:teuthology.orchestra.run.smithi076.stdout: "take": 0, 2024-09-11T23:40:02.109 INFO:teuthology.orchestra.run.smithi076.stdout: "take_sum": 0, 2024-09-11T23:40:02.109 INFO:teuthology.orchestra.run.smithi076.stdout: "put": 10846, 2024-09-11T23:40:02.109 INFO:teuthology.orchestra.run.smithi076.stdout: "put_sum": 270571667, 2024-09-11T23:40:02.110 INFO:teuthology.orchestra.run.smithi076.stdout: "wait": { 2024-09-11T23:40:02.110 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-09-11T23:40:02.110 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-09-11T23:40:02.110 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-09-11T23:40:02.110 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-11T23:40:02.110 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-11T23:40:02.110 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-11T23:40:02.238 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:40:01 smithi136 ceph-mon[35121]: pgmap v681: 97 pgs: 97 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 41 KiB/s wr, 36 op/s 2024-09-11T23:40:02.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:40:01 smithi136 ceph-mon[35121]: osdmap e116: 12 total, 12 up, 12 in 2024-09-11T23:40:02.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:40:01 smithi026 ceph-mon[31172]: pgmap v681: 97 pgs: 97 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 41 KiB/s wr, 36 op/s 2024-09-11T23:40:02.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:40:01 smithi026 ceph-mon[31172]: osdmap e116: 12 total, 12 up, 12 in 2024-09-11T23:40:02.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:40:01 smithi076 ceph-mon[35296]: pgmap v681: 97 pgs: 97 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 41 KiB/s wr, 36 op/s 2024-09-11T23:40:02.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:40:01 smithi076 ceph-mon[35296]: osdmap e116: 12 total, 12 up, 12 in 2024-09-11T23:40:03.084 INFO:tasks.check_counter:Daemon mds.b mds.exported=209 2024-09-11T23:40:03.084 INFO:tasks.check_counter:Daemon mds.b mds.imported=254 2024-09-11T23:40:03.084 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update=5068 2024-09-11T23:40:03.085 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update_receipt=10989 2024-09-11T23:40:03.085 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2024-09-11T23:40:03.085 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=22 2024-09-11T23:40:03.085 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=16 2024-09-11T23:40:03.085 INFO:tasks.check_counter:Daemon mds.b mds.dir_split=85 2024-09-11T23:40:03.085 DEBUG:tasks.check_counter:Getting stats from e 2024-09-11T23:40:03.085 DEBUG:teuthology.orchestra.run.smithi076:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph daemon mds.e perf dump 2024-09-11T23:40:03.346 INFO:teuthology.orchestra.run.smithi076.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T23:40:03.989 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-11T23:40:03.989 INFO:teuthology.orchestra.run.smithi076.stdout: "AsyncMessenger::Worker-0": { 2024-09-11T23:40:03.989 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_messages": 35921, 2024-09-11T23:40:03.990 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_messages": 29322, 2024-09-11T23:40:03.990 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_bytes": 17424125, 2024-09-11T23:40:03.990 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_bytes": 155250666, 2024-09-11T23:40:03.990 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_created_connections": 15, 2024-09-11T23:40:03.990 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_active_connections": 4, 2024-09-11T23:40:03.990 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_total_time": 2.868582725, 2024-09-11T23:40:03.990 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_send_time": 1.332248811, 2024-09-11T23:40:03.990 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_recv_time": 1.214744379, 2024-09-11T23:40:03.990 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_fast_dispatch_time": 0.572560328, 2024-09-11T23:40:03.990 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T23:40:03.990 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 29321, 2024-09-11T23:40:03.990 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 5.138717592, 2024-09-11T23:40:03.990 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000175257 2024-09-11T23:40:03.990 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:03.991 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_handle_ack_lat": { 2024-09-11T23:40:03.991 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 46605, 2024-09-11T23:40:03.991 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.007677767, 2024-09-11T23:40:03.991 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000164 2024-09-11T23:40:03.991 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:03.991 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_encrypted_bytes": 5712, 2024-09-11T23:40:03.991 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_encrypted_bytes": 13440 2024-09-11T23:40:03.991 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:03.991 INFO:teuthology.orchestra.run.smithi076.stdout: "AsyncMessenger::Worker-1": { 2024-09-11T23:40:03.991 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_messages": 273827, 2024-09-11T23:40:03.991 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_messages": 356734, 2024-09-11T23:40:03.991 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_bytes": 103248611, 2024-09-11T23:40:03.991 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_bytes": 288107993, 2024-09-11T23:40:03.991 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_created_connections": 18, 2024-09-11T23:40:03.992 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_active_connections": 7, 2024-09-11T23:40:03.992 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_total_time": 16.772314107, 2024-09-11T23:40:03.992 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_send_time": 8.170719236, 2024-09-11T23:40:03.992 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_recv_time": 22.814341029, 2024-09-11T23:40:03.992 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_fast_dispatch_time": 0.335182950, 2024-09-11T23:40:03.992 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T23:40:03.992 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 356729, 2024-09-11T23:40:03.992 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 37.627614242, 2024-09-11T23:40:03.992 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000105479 2024-09-11T23:40:03.992 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:03.992 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_handle_ack_lat": { 2024-09-11T23:40:03.992 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 512510, 2024-09-11T23:40:03.992 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.102186570, 2024-09-11T23:40:03.993 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000199 2024-09-11T23:40:03.993 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:03.993 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_encrypted_bytes": 86701280, 2024-09-11T23:40:03.993 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_encrypted_bytes": 179316480 2024-09-11T23:40:03.993 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:03.993 INFO:teuthology.orchestra.run.smithi076.stdout: "AsyncMessenger::Worker-2": { 2024-09-11T23:40:03.993 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_messages": 72293, 2024-09-11T23:40:03.993 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_messages": 61141, 2024-09-11T23:40:03.993 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_bytes": 45707158, 2024-09-11T23:40:03.993 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_bytes": 92029571, 2024-09-11T23:40:03.993 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_created_connections": 13, 2024-09-11T23:40:03.993 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_active_connections": 5, 2024-09-11T23:40:03.994 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_total_time": 4.234871866, 2024-09-11T23:40:03.994 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_send_time": 1.582146496, 2024-09-11T23:40:03.994 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_recv_time": 2.723298461, 2024-09-11T23:40:03.994 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_fast_dispatch_time": 0.570974826, 2024-09-11T23:40:03.994 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T23:40:03.994 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 61142, 2024-09-11T23:40:03.994 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 6.211833774, 2024-09-11T23:40:03.994 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000101596 2024-09-11T23:40:03.994 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:03.994 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_handle_ack_lat": { 2024-09-11T23:40:03.994 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 119659, 2024-09-11T23:40:03.994 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.019952900, 2024-09-11T23:40:03.994 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000166 2024-09-11T23:40:03.994 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:03.995 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_encrypted_bytes": 2120032, 2024-09-11T23:40:03.995 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_encrypted_bytes": 296112 2024-09-11T23:40:03.995 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:03.995 INFO:teuthology.orchestra.run.smithi076.stdout: "cct": { 2024-09-11T23:40:03.995 INFO:teuthology.orchestra.run.smithi076.stdout: "total_workers": 1, 2024-09-11T23:40:03.995 INFO:teuthology.orchestra.run.smithi076.stdout: "unhealthy_workers": 0 2024-09-11T23:40:03.995 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:03.995 INFO:teuthology.orchestra.run.smithi076.stdout: "finisher-MDSRank": { 2024-09-11T23:40:03.995 INFO:teuthology.orchestra.run.smithi076.stdout: "queue_len": 0, 2024-09-11T23:40:03.996 INFO:teuthology.orchestra.run.smithi076.stdout: "complete_latency": { 2024-09-11T23:40:03.996 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 7862, 2024-09-11T23:40:03.996 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 79.384126106, 2024-09-11T23:40:03.996 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.010097192 2024-09-11T23:40:03.996 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-11T23:40:03.996 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:03.996 INFO:teuthology.orchestra.run.smithi076.stdout: "finisher-PurgeQueue": { 2024-09-11T23:40:03.996 INFO:teuthology.orchestra.run.smithi076.stdout: "queue_len": 0, 2024-09-11T23:40:03.996 INFO:teuthology.orchestra.run.smithi076.stdout: "complete_latency": { 2024-09-11T23:40:03.996 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 17045, 2024-09-11T23:40:03.996 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 4.080758521, 2024-09-11T23:40:03.997 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000239410 2024-09-11T23:40:03.997 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-11T23:40:03.997 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:03.997 INFO:teuthology.orchestra.run.smithi076.stdout: "mds": { 2024-09-11T23:40:03.997 INFO:teuthology.orchestra.run.smithi076.stdout: "request": 104011, 2024-09-11T23:40:03.997 INFO:teuthology.orchestra.run.smithi076.stdout: "reply": 102783, 2024-09-11T23:40:03.997 INFO:teuthology.orchestra.run.smithi076.stdout: "reply_latency": { 2024-09-11T23:40:03.997 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 102783, 2024-09-11T23:40:03.997 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 3765.903497180, 2024-09-11T23:40:03.997 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.036639361 2024-09-11T23:40:03.997 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:03.997 INFO:teuthology.orchestra.run.smithi076.stdout: "slow_reply": 0, 2024-09-11T23:40:03.997 INFO:teuthology.orchestra.run.smithi076.stdout: "forward": 1228, 2024-09-11T23:40:03.998 INFO:teuthology.orchestra.run.smithi076.stdout: "dir_fetch_complete": 39, 2024-09-11T23:40:03.998 INFO:teuthology.orchestra.run.smithi076.stdout: "dir_fetch_keys": 0, 2024-09-11T23:40:03.998 INFO:teuthology.orchestra.run.smithi076.stdout: "dir_commit": 954, 2024-09-11T23:40:03.998 INFO:teuthology.orchestra.run.smithi076.stdout: "dir_split": 91, 2024-09-11T23:40:03.998 INFO:teuthology.orchestra.run.smithi076.stdout: "dir_merge": 0, 2024-09-11T23:40:03.998 INFO:teuthology.orchestra.run.smithi076.stdout: "inodes": 18510, 2024-09-11T23:40:03.998 INFO:teuthology.orchestra.run.smithi076.stdout: "inodes_top": 739, 2024-09-11T23:40:03.998 INFO:teuthology.orchestra.run.smithi076.stdout: "inodes_bottom": 3436, 2024-09-11T23:40:03.998 INFO:teuthology.orchestra.run.smithi076.stdout: "inodes_pin_tail": 14335, 2024-09-11T23:40:03.998 INFO:teuthology.orchestra.run.smithi076.stdout: "inodes_pinned": 17451, 2024-09-11T23:40:03.998 INFO:teuthology.orchestra.run.smithi076.stdout: "inodes_expired": 49727, 2024-09-11T23:40:03.998 INFO:teuthology.orchestra.run.smithi076.stdout: "inodes_with_caps": 0, 2024-09-11T23:40:03.998 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": 0, 2024-09-11T23:40:03.998 INFO:teuthology.orchestra.run.smithi076.stdout: "subtrees": 5, 2024-09-11T23:40:03.999 INFO:teuthology.orchestra.run.smithi076.stdout: "traverse": 189600, 2024-09-11T23:40:03.999 INFO:teuthology.orchestra.run.smithi076.stdout: "traverse_hit": 154478, 2024-09-11T23:40:03.999 INFO:teuthology.orchestra.run.smithi076.stdout: "traverse_forward": 870, 2024-09-11T23:40:03.999 INFO:teuthology.orchestra.run.smithi076.stdout: "traverse_discover": 1397, 2024-09-11T23:40:03.999 INFO:teuthology.orchestra.run.smithi076.stdout: "traverse_dir_fetch": 9, 2024-09-11T23:40:03.999 INFO:teuthology.orchestra.run.smithi076.stdout: "traverse_remote_ino": 0, 2024-09-11T23:40:03.999 INFO:teuthology.orchestra.run.smithi076.stdout: "traverse_lock": 532, 2024-09-11T23:40:03.999 INFO:teuthology.orchestra.run.smithi076.stdout: "load_cent": 0, 2024-09-11T23:40:03.999 INFO:teuthology.orchestra.run.smithi076.stdout: "q": 0, 2024-09-11T23:40:03.999 INFO:teuthology.orchestra.run.smithi076.stdout: "exported": 143, 2024-09-11T23:40:03.999 INFO:teuthology.orchestra.run.smithi076.stdout: "exported_inodes": 37041, 2024-09-11T23:40:03.999 INFO:teuthology.orchestra.run.smithi076.stdout: "imported": 222, 2024-09-11T23:40:03.999 INFO:teuthology.orchestra.run.smithi076.stdout: "imported_inodes": 15960, 2024-09-11T23:40:03.999 INFO:teuthology.orchestra.run.smithi076.stdout: "openino_dir_fetch": 0, 2024-09-11T23:40:04.000 INFO:teuthology.orchestra.run.smithi076.stdout: "openino_backtrace_fetch": 338, 2024-09-11T23:40:04.000 INFO:teuthology.orchestra.run.smithi076.stdout: "openino_peer_discover": 0, 2024-09-11T23:40:04.000 INFO:teuthology.orchestra.run.smithi076.stdout: "root_rfiles": 7006, 2024-09-11T23:40:04.000 INFO:teuthology.orchestra.run.smithi076.stdout: "root_rbytes": 34252265, 2024-09-11T23:40:04.000 INFO:teuthology.orchestra.run.smithi076.stdout: "root_rsnaps": 2, 2024-09-11T23:40:04.000 INFO:teuthology.orchestra.run.smithi076.stdout: "scrub_backtrace_fetch": 338, 2024-09-11T23:40:04.000 INFO:teuthology.orchestra.run.smithi076.stdout: "scrub_set_tag": 0, 2024-09-11T23:40:04.000 INFO:teuthology.orchestra.run.smithi076.stdout: "scrub_backtrace_repaired": 0, 2024-09-11T23:40:04.000 INFO:teuthology.orchestra.run.smithi076.stdout: "scrub_inotable_repaired": 0, 2024-09-11T23:40:04.000 INFO:teuthology.orchestra.run.smithi076.stdout: "scrub_dir_inodes": 0, 2024-09-11T23:40:04.000 INFO:teuthology.orchestra.run.smithi076.stdout: "scrub_dir_base_inodes": 0, 2024-09-11T23:40:04.000 INFO:teuthology.orchestra.run.smithi076.stdout: "scrub_dirfrag_rstats": 0, 2024-09-11T23:40:04.000 INFO:teuthology.orchestra.run.smithi076.stdout: "scrub_file_inodes": 338, 2024-09-11T23:40:04.001 INFO:teuthology.orchestra.run.smithi076.stdout: "handle_inode_file_caps": 69, 2024-09-11T23:40:04.001 INFO:teuthology.orchestra.run.smithi076.stdout: "ceph_cap_op_revoke": 68567, 2024-09-11T23:40:04.001 INFO:teuthology.orchestra.run.smithi076.stdout: "ceph_cap_op_grant": 50586, 2024-09-11T23:40:04.001 INFO:teuthology.orchestra.run.smithi076.stdout: "ceph_cap_op_trunc": 0, 2024-09-11T23:40:04.001 INFO:teuthology.orchestra.run.smithi076.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-11T23:40:04.001 INFO:teuthology.orchestra.run.smithi076.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-11T23:40:04.001 INFO:teuthology.orchestra.run.smithi076.stdout: "handle_client_caps": 88819, 2024-09-11T23:40:04.001 INFO:teuthology.orchestra.run.smithi076.stdout: "handle_client_caps_dirty": 21637, 2024-09-11T23:40:04.001 INFO:teuthology.orchestra.run.smithi076.stdout: "handle_client_cap_release": 3194, 2024-09-11T23:40:04.001 INFO:teuthology.orchestra.run.smithi076.stdout: "process_request_cap_release": 27985 2024-09-11T23:40:04.001 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:04.001 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_cache": { 2024-09-11T23:40:04.001 INFO:teuthology.orchestra.run.smithi076.stdout: "dir_update": 3378, 2024-09-11T23:40:04.002 INFO:teuthology.orchestra.run.smithi076.stdout: "dir_update_receipt": 14214, 2024-09-11T23:40:04.002 INFO:teuthology.orchestra.run.smithi076.stdout: "dir_try_discover": 8591, 2024-09-11T23:40:04.002 INFO:teuthology.orchestra.run.smithi076.stdout: "dir_send_discover": 10082, 2024-09-11T23:40:04.002 INFO:teuthology.orchestra.run.smithi076.stdout: "dir_handle_discover": 4949, 2024-09-11T23:40:04.002 INFO:teuthology.orchestra.run.smithi076.stdout: "num_strays": 52, 2024-09-11T23:40:04.002 INFO:teuthology.orchestra.run.smithi076.stdout: "num_strays_delayed": 0, 2024-09-11T23:40:04.002 INFO:teuthology.orchestra.run.smithi076.stdout: "num_strays_enqueuing": 0, 2024-09-11T23:40:04.002 INFO:teuthology.orchestra.run.smithi076.stdout: "strays_created": 8845, 2024-09-11T23:40:04.002 INFO:teuthology.orchestra.run.smithi076.stdout: "strays_enqueued": 8794, 2024-09-11T23:40:04.002 INFO:teuthology.orchestra.run.smithi076.stdout: "strays_reintegrated": 0, 2024-09-11T23:40:04.002 INFO:teuthology.orchestra.run.smithi076.stdout: "strays_migrated": 0, 2024-09-11T23:40:04.002 INFO:teuthology.orchestra.run.smithi076.stdout: "num_recovering_processing": 0, 2024-09-11T23:40:04.002 INFO:teuthology.orchestra.run.smithi076.stdout: "num_recovering_enqueued": 0, 2024-09-11T23:40:04.002 INFO:teuthology.orchestra.run.smithi076.stdout: "num_recovering_prioritized": 0, 2024-09-11T23:40:04.003 INFO:teuthology.orchestra.run.smithi076.stdout: "recovery_started": 0, 2024-09-11T23:40:04.003 INFO:teuthology.orchestra.run.smithi076.stdout: "recovery_completed": 0, 2024-09-11T23:40:04.003 INFO:teuthology.orchestra.run.smithi076.stdout: "ireq_quiesce_path": 18, 2024-09-11T23:40:04.003 INFO:teuthology.orchestra.run.smithi076.stdout: "ireq_quiesce_inode": 192185, 2024-09-11T23:40:04.003 INFO:teuthology.orchestra.run.smithi076.stdout: "ireq_enqueue_scrub": 0, 2024-09-11T23:40:04.003 INFO:teuthology.orchestra.run.smithi076.stdout: "ireq_exportdir": 143, 2024-09-11T23:40:04.003 INFO:teuthology.orchestra.run.smithi076.stdout: "ireq_flush": 0, 2024-09-11T23:40:04.003 INFO:teuthology.orchestra.run.smithi076.stdout: "ireq_fragmentdir": 91, 2024-09-11T23:40:04.003 INFO:teuthology.orchestra.run.smithi076.stdout: "ireq_fragstats": 0, 2024-09-11T23:40:04.003 INFO:teuthology.orchestra.run.smithi076.stdout: "ireq_inodestats": 0 2024-09-11T23:40:04.003 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:04.003 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_log": { 2024-09-11T23:40:04.003 INFO:teuthology.orchestra.run.smithi076.stdout: "evlrg": 2, 2024-09-11T23:40:04.004 INFO:teuthology.orchestra.run.smithi076.stdout: "evadd": 98369, 2024-09-11T23:40:04.004 INFO:teuthology.orchestra.run.smithi076.stdout: "evex": 10294, 2024-09-11T23:40:04.004 INFO:teuthology.orchestra.run.smithi076.stdout: "evtrm": 1, 2024-09-11T23:40:04.004 INFO:teuthology.orchestra.run.smithi076.stdout: "ev": 98368, 2024-09-11T23:40:04.004 INFO:teuthology.orchestra.run.smithi076.stdout: "evexg": 0, 2024-09-11T23:40:04.004 INFO:teuthology.orchestra.run.smithi076.stdout: "evexd": 10293, 2024-09-11T23:40:04.004 INFO:teuthology.orchestra.run.smithi076.stdout: "segadd": 141, 2024-09-11T23:40:04.004 INFO:teuthology.orchestra.run.smithi076.stdout: "segex": 13, 2024-09-11T23:40:04.004 INFO:teuthology.orchestra.run.smithi076.stdout: "segtrm": 1, 2024-09-11T23:40:04.004 INFO:teuthology.orchestra.run.smithi076.stdout: "seg": 140, 2024-09-11T23:40:04.004 INFO:teuthology.orchestra.run.smithi076.stdout: "segmjr": 9, 2024-09-11T23:40:04.004 INFO:teuthology.orchestra.run.smithi076.stdout: "segexg": 0, 2024-09-11T23:40:04.004 INFO:teuthology.orchestra.run.smithi076.stdout: "segexd": 12, 2024-09-11T23:40:04.004 INFO:teuthology.orchestra.run.smithi076.stdout: "expos": 4194352, 2024-09-11T23:40:04.005 INFO:teuthology.orchestra.run.smithi076.stdout: "wrpos": 304182424, 2024-09-11T23:40:04.005 INFO:teuthology.orchestra.run.smithi076.stdout: "rdpos": 0, 2024-09-11T23:40:04.005 INFO:teuthology.orchestra.run.smithi076.stdout: "jlat": { 2024-09-11T23:40:04.005 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 4786, 2024-09-11T23:40:04.005 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 221.843011584, 2024-09-11T23:40:04.005 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.046352488 2024-09-11T23:40:04.005 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:04.005 INFO:teuthology.orchestra.run.smithi076.stdout: "replayed": 0 2024-09-11T23:40:04.005 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:04.005 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_mem": { 2024-09-11T23:40:04.005 INFO:teuthology.orchestra.run.smithi076.stdout: "ino": 10594, 2024-09-11T23:40:04.005 INFO:teuthology.orchestra.run.smithi076.stdout: "ino+": 48979, 2024-09-11T23:40:04.005 INFO:teuthology.orchestra.run.smithi076.stdout: "ino-": 38385, 2024-09-11T23:40:04.006 INFO:teuthology.orchestra.run.smithi076.stdout: "dir": 404, 2024-09-11T23:40:04.006 INFO:teuthology.orchestra.run.smithi076.stdout: "dir+": 6947, 2024-09-11T23:40:04.006 INFO:teuthology.orchestra.run.smithi076.stdout: "dir-": 6543, 2024-09-11T23:40:04.006 INFO:teuthology.orchestra.run.smithi076.stdout: "dn": 18510, 2024-09-11T23:40:04.006 INFO:teuthology.orchestra.run.smithi076.stdout: "dn+": 79869, 2024-09-11T23:40:04.006 INFO:teuthology.orchestra.run.smithi076.stdout: "dn-": 61359, 2024-09-11T23:40:04.006 INFO:teuthology.orchestra.run.smithi076.stdout: "cap": 0, 2024-09-11T23:40:04.006 INFO:teuthology.orchestra.run.smithi076.stdout: "cap+": 34093, 2024-09-11T23:40:04.006 INFO:teuthology.orchestra.run.smithi076.stdout: "cap-": 34093, 2024-09-11T23:40:04.006 INFO:teuthology.orchestra.run.smithi076.stdout: "rss": 229772, 2024-09-11T23:40:04.006 INFO:teuthology.orchestra.run.smithi076.stdout: "heap": 223520 2024-09-11T23:40:04.006 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:04.006 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_server": { 2024-09-11T23:40:04.007 INFO:teuthology.orchestra.run.smithi076.stdout: "dispatch_client_request": 109489, 2024-09-11T23:40:04.007 INFO:teuthology.orchestra.run.smithi076.stdout: "dispatch_server_request": 4047, 2024-09-11T23:40:04.007 INFO:teuthology.orchestra.run.smithi076.stdout: "handle_client_request": 104011, 2024-09-11T23:40:04.007 INFO:teuthology.orchestra.run.smithi076.stdout: "handle_client_session": 297, 2024-09-11T23:40:04.007 INFO:teuthology.orchestra.run.smithi076.stdout: "handle_peer_request": 8505, 2024-09-11T23:40:04.007 INFO:teuthology.orchestra.run.smithi076.stdout: "req_create_latency": { 2024-09-11T23:40:04.007 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 15962, 2024-09-11T23:40:04.007 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 134.272851387, 2024-09-11T23:40:04.007 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.008412031 2024-09-11T23:40:04.007 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:04.007 INFO:teuthology.orchestra.run.smithi076.stdout: "req_getattr_latency": { 2024-09-11T23:40:04.007 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 23, 2024-09-11T23:40:04.007 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 3.362921538, 2024-09-11T23:40:04.007 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.146213979 2024-09-11T23:40:04.008 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:04.008 INFO:teuthology.orchestra.run.smithi076.stdout: "req_getfilelock_latency": { 2024-09-11T23:40:04.008 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-09-11T23:40:04.008 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-09-11T23:40:04.008 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-09-11T23:40:04.008 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:04.008 INFO:teuthology.orchestra.run.smithi076.stdout: "req_link_latency": { 2024-09-11T23:40:04.008 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-09-11T23:40:04.008 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-09-11T23:40:04.008 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-09-11T23:40:04.008 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:04.008 INFO:teuthology.orchestra.run.smithi076.stdout: "req_lookup_latency": { 2024-09-11T23:40:04.008 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 54183, 2024-09-11T23:40:04.009 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 2826.728567890, 2024-09-11T23:40:04.009 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.052170026 2024-09-11T23:40:04.009 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:04.009 INFO:teuthology.orchestra.run.smithi076.stdout: "req_lookuphash_latency": { 2024-09-11T23:40:04.009 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-09-11T23:40:04.009 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-09-11T23:40:04.009 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-09-11T23:40:04.009 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:04.009 INFO:teuthology.orchestra.run.smithi076.stdout: "req_lookupino_latency": { 2024-09-11T23:40:04.009 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-09-11T23:40:04.009 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-09-11T23:40:04.009 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-09-11T23:40:04.009 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:04.009 INFO:teuthology.orchestra.run.smithi076.stdout: "req_lookupname_latency": { 2024-09-11T23:40:04.010 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-09-11T23:40:04.010 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-09-11T23:40:04.010 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-09-11T23:40:04.010 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:04.010 INFO:teuthology.orchestra.run.smithi076.stdout: "req_lookupparent_latency": { 2024-09-11T23:40:04.010 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-09-11T23:40:04.010 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-09-11T23:40:04.010 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-09-11T23:40:04.010 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:04.010 INFO:teuthology.orchestra.run.smithi076.stdout: "req_lookupsnap_latency": { 2024-09-11T23:40:04.011 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-09-11T23:40:04.011 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-09-11T23:40:04.011 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-09-11T23:40:04.011 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:04.011 INFO:teuthology.orchestra.run.smithi076.stdout: "req_lssnap_latency": { 2024-09-11T23:40:04.011 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-09-11T23:40:04.011 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-09-11T23:40:04.011 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-09-11T23:40:04.011 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:04.011 INFO:teuthology.orchestra.run.smithi076.stdout: "req_mkdir_latency": { 2024-09-11T23:40:04.011 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 98, 2024-09-11T23:40:04.012 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 1.064820924, 2024-09-11T23:40:04.012 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.010865519 2024-09-11T23:40:04.012 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:04.012 INFO:teuthology.orchestra.run.smithi076.stdout: "req_mknod_latency": { 2024-09-11T23:40:04.012 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-09-11T23:40:04.012 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-09-11T23:40:04.012 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-09-11T23:40:04.012 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:04.012 INFO:teuthology.orchestra.run.smithi076.stdout: "req_mksnap_latency": { 2024-09-11T23:40:04.012 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-09-11T23:40:04.012 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-09-11T23:40:04.012 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-09-11T23:40:04.012 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:04.012 INFO:teuthology.orchestra.run.smithi076.stdout: "req_open_latency": { 2024-09-11T23:40:04.013 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 8825, 2024-09-11T23:40:04.013 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 254.779609466, 2024-09-11T23:40:04.013 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.028870210 2024-09-11T23:40:04.013 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:04.013 INFO:teuthology.orchestra.run.smithi076.stdout: "req_readdir_latency": { 2024-09-11T23:40:04.013 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 345, 2024-09-11T23:40:04.013 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.600661193, 2024-09-11T23:40:04.013 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.001741046 2024-09-11T23:40:04.013 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:04.013 INFO:teuthology.orchestra.run.smithi076.stdout: "req_rename_latency": { 2024-09-11T23:40:04.013 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 15979, 2024-09-11T23:40:04.013 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 467.861120900, 2024-09-11T23:40:04.013 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.029279749 2024-09-11T23:40:04.014 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:04.014 INFO:teuthology.orchestra.run.smithi076.stdout: "req_renamesnap_latency": { 2024-09-11T23:40:04.014 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-09-11T23:40:04.014 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-09-11T23:40:04.014 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-09-11T23:40:04.014 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:04.014 INFO:teuthology.orchestra.run.smithi076.stdout: "req_snapdiff_latency": { 2024-09-11T23:40:04.014 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-09-11T23:40:04.014 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-09-11T23:40:04.014 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-09-11T23:40:04.014 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:04.014 INFO:teuthology.orchestra.run.smithi076.stdout: "req_rmdir_latency": { 2024-09-11T23:40:04.014 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 51, 2024-09-11T23:40:04.014 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 37.955798383, 2024-09-11T23:40:04.015 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.744231340 2024-09-11T23:40:04.015 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:04.015 INFO:teuthology.orchestra.run.smithi076.stdout: "req_rmsnap_latency": { 2024-09-11T23:40:04.015 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-09-11T23:40:04.015 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-09-11T23:40:04.015 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-09-11T23:40:04.015 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:04.015 INFO:teuthology.orchestra.run.smithi076.stdout: "req_rmxattr_latency": { 2024-09-11T23:40:04.015 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-09-11T23:40:04.015 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-09-11T23:40:04.015 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-09-11T23:40:04.015 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:04.015 INFO:teuthology.orchestra.run.smithi076.stdout: "req_setattr_latency": { 2024-09-11T23:40:04.015 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-09-11T23:40:04.016 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-09-11T23:40:04.016 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-09-11T23:40:04.016 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:04.016 INFO:teuthology.orchestra.run.smithi076.stdout: "req_setdirlayout_latency": { 2024-09-11T23:40:04.016 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-09-11T23:40:04.016 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-09-11T23:40:04.016 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-09-11T23:40:04.016 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:04.016 INFO:teuthology.orchestra.run.smithi076.stdout: "req_setfilelock_latency": { 2024-09-11T23:40:04.016 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-09-11T23:40:04.016 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-09-11T23:40:04.016 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-09-11T23:40:04.016 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:04.017 INFO:teuthology.orchestra.run.smithi076.stdout: "req_setlayout_latency": { 2024-09-11T23:40:04.017 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-09-11T23:40:04.017 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-09-11T23:40:04.017 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-09-11T23:40:04.017 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:04.017 INFO:teuthology.orchestra.run.smithi076.stdout: "req_setxattr_latency": { 2024-09-11T23:40:04.017 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-09-11T23:40:04.017 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-09-11T23:40:04.017 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-09-11T23:40:04.017 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:04.017 INFO:teuthology.orchestra.run.smithi076.stdout: "req_symlink_latency": { 2024-09-11T23:40:04.017 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-09-11T23:40:04.017 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-09-11T23:40:04.018 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-09-11T23:40:04.018 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:04.018 INFO:teuthology.orchestra.run.smithi076.stdout: "req_unlink_latency": { 2024-09-11T23:40:04.018 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 7317, 2024-09-11T23:40:04.018 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 39.277145499, 2024-09-11T23:40:04.018 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.005367930 2024-09-11T23:40:04.018 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:04.018 INFO:teuthology.orchestra.run.smithi076.stdout: "cap_revoke_eviction": 0, 2024-09-11T23:40:04.018 INFO:teuthology.orchestra.run.smithi076.stdout: "cap_acquisition_throttle": 0, 2024-09-11T23:40:04.018 INFO:teuthology.orchestra.run.smithi076.stdout: "req_getvxattr_latency": { 2024-09-11T23:40:04.018 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-09-11T23:40:04.018 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-09-11T23:40:04.018 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-09-11T23:40:04.018 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-11T23:40:04.019 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:04.019 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_sessions": { 2024-09-11T23:40:04.019 INFO:teuthology.orchestra.run.smithi076.stdout: "session_count": 2, 2024-09-11T23:40:04.019 INFO:teuthology.orchestra.run.smithi076.stdout: "session_add": 3, 2024-09-11T23:40:04.019 INFO:teuthology.orchestra.run.smithi076.stdout: "session_remove": 1, 2024-09-11T23:40:04.019 INFO:teuthology.orchestra.run.smithi076.stdout: "sessions_open": 2, 2024-09-11T23:40:04.019 INFO:teuthology.orchestra.run.smithi076.stdout: "sessions_stale": 0, 2024-09-11T23:40:04.019 INFO:teuthology.orchestra.run.smithi076.stdout: "total_load": 1648, 2024-09-11T23:40:04.019 INFO:teuthology.orchestra.run.smithi076.stdout: "average_load": 824, 2024-09-11T23:40:04.019 INFO:teuthology.orchestra.run.smithi076.stdout: "avg_session_uptime": 1231, 2024-09-11T23:40:04.019 INFO:teuthology.orchestra.run.smithi076.stdout: "mdthresh_evicted": 0 2024-09-11T23:40:04.019 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:04.019 INFO:teuthology.orchestra.run.smithi076.stdout: "mempool": { 2024-09-11T23:40:04.019 INFO:teuthology.orchestra.run.smithi076.stdout: "bloom_filter_bytes": 0, 2024-09-11T23:40:04.020 INFO:teuthology.orchestra.run.smithi076.stdout: "bloom_filter_items": 0, 2024-09-11T23:40:04.020 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_alloc_bytes": 0, 2024-09-11T23:40:04.020 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_alloc_items": 0, 2024-09-11T23:40:04.020 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_data_bytes": 0, 2024-09-11T23:40:04.020 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_data_items": 0, 2024-09-11T23:40:04.020 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-11T23:40:04.020 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_onode_items": 0, 2024-09-11T23:40:04.020 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-11T23:40:04.020 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_meta_items": 0, 2024-09-11T23:40:04.020 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_other_bytes": 0, 2024-09-11T23:40:04.020 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_other_items": 0, 2024-09-11T23:40:04.020 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-11T23:40:04.020 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_buffer_items": 0, 2024-09-11T23:40:04.021 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_extent_bytes": 0, 2024-09-11T23:40:04.021 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_extent_items": 0, 2024-09-11T23:40:04.021 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_blob_bytes": 0, 2024-09-11T23:40:04.021 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_blob_items": 0, 2024-09-11T23:40:04.021 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-11T23:40:04.021 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_shared_blob_items": 0, 2024-09-11T23:40:04.021 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-11T23:40:04.021 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_inline_bl_items": 0, 2024-09-11T23:40:04.021 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_fsck_bytes": 0, 2024-09-11T23:40:04.021 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_fsck_items": 0, 2024-09-11T23:40:04.021 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_txc_bytes": 0, 2024-09-11T23:40:04.021 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_txc_items": 0, 2024-09-11T23:40:04.021 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-11T23:40:04.022 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_writing_deferred_items": 0, 2024-09-11T23:40:04.022 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_writing_bytes": 0, 2024-09-11T23:40:04.022 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_writing_items": 0, 2024-09-11T23:40:04.022 INFO:teuthology.orchestra.run.smithi076.stdout: "bluefs_bytes": 0, 2024-09-11T23:40:04.022 INFO:teuthology.orchestra.run.smithi076.stdout: "bluefs_items": 0, 2024-09-11T23:40:04.022 INFO:teuthology.orchestra.run.smithi076.stdout: "bluefs_file_reader_bytes": 0, 2024-09-11T23:40:04.022 INFO:teuthology.orchestra.run.smithi076.stdout: "bluefs_file_reader_items": 0, 2024-09-11T23:40:04.022 INFO:teuthology.orchestra.run.smithi076.stdout: "bluefs_file_writer_bytes": 0, 2024-09-11T23:40:04.022 INFO:teuthology.orchestra.run.smithi076.stdout: "bluefs_file_writer_items": 0, 2024-09-11T23:40:04.022 INFO:teuthology.orchestra.run.smithi076.stdout: "buffer_anon_bytes": 1019795, 2024-09-11T23:40:04.022 INFO:teuthology.orchestra.run.smithi076.stdout: "buffer_anon_items": 7446, 2024-09-11T23:40:04.022 INFO:teuthology.orchestra.run.smithi076.stdout: "buffer_meta_bytes": 0, 2024-09-11T23:40:04.022 INFO:teuthology.orchestra.run.smithi076.stdout: "buffer_meta_items": 0, 2024-09-11T23:40:04.022 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_bytes": 0, 2024-09-11T23:40:04.023 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_items": 0, 2024-09-11T23:40:04.023 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_mapbl_bytes": 0, 2024-09-11T23:40:04.023 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_mapbl_items": 0, 2024-09-11T23:40:04.023 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_pglog_bytes": 0, 2024-09-11T23:40:04.023 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_pglog_items": 0, 2024-09-11T23:40:04.023 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_bytes": 8224, 2024-09-11T23:40:04.023 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_items": 181, 2024-09-11T23:40:04.023 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_mapping_bytes": 0, 2024-09-11T23:40:04.023 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_mapping_items": 0, 2024-09-11T23:40:04.023 INFO:teuthology.orchestra.run.smithi076.stdout: "pgmap_bytes": 0, 2024-09-11T23:40:04.023 INFO:teuthology.orchestra.run.smithi076.stdout: "pgmap_items": 0, 2024-09-11T23:40:04.023 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_co_bytes": 37939625, 2024-09-11T23:40:04.023 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_co_items": 348521, 2024-09-11T23:40:04.024 INFO:teuthology.orchestra.run.smithi076.stdout: "unittest_1_bytes": 0, 2024-09-11T23:40:04.024 INFO:teuthology.orchestra.run.smithi076.stdout: "unittest_1_items": 0, 2024-09-11T23:40:04.024 INFO:teuthology.orchestra.run.smithi076.stdout: "unittest_2_bytes": 0, 2024-09-11T23:40:04.024 INFO:teuthology.orchestra.run.smithi076.stdout: "unittest_2_items": 0 2024-09-11T23:40:04.024 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:04.024 INFO:teuthology.orchestra.run.smithi076.stdout: "objecter": { 2024-09-11T23:40:04.024 INFO:teuthology.orchestra.run.smithi076.stdout: "op_active": 0, 2024-09-11T23:40:04.024 INFO:teuthology.orchestra.run.smithi076.stdout: "op_laggy": 0, 2024-09-11T23:40:04.024 INFO:teuthology.orchestra.run.smithi076.stdout: "op_send": 18385, 2024-09-11T23:40:04.024 INFO:teuthology.orchestra.run.smithi076.stdout: "op_send_bytes": 323283075, 2024-09-11T23:40:04.024 INFO:teuthology.orchestra.run.smithi076.stdout: "op_resend": 29, 2024-09-11T23:40:04.024 INFO:teuthology.orchestra.run.smithi076.stdout: "op_reply": 18385, 2024-09-11T23:40:04.024 INFO:teuthology.orchestra.run.smithi076.stdout: "op_latency": { 2024-09-11T23:40:04.024 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 18385, 2024-09-11T23:40:04.025 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 1146.918710838, 2024-09-11T23:40:04.025 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.062383394 2024-09-11T23:40:04.025 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:04.025 INFO:teuthology.orchestra.run.smithi076.stdout: "op_inflight": 0, 2024-09-11T23:40:04.025 INFO:teuthology.orchestra.run.smithi076.stdout: "oplen_avg": { 2024-09-11T23:40:04.025 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 18385, 2024-09-11T23:40:04.025 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 24281 2024-09-11T23:40:04.025 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:04.025 INFO:teuthology.orchestra.run.smithi076.stdout: "op": 18385, 2024-09-11T23:40:04.025 INFO:teuthology.orchestra.run.smithi076.stdout: "op_r": 710, 2024-09-11T23:40:04.025 INFO:teuthology.orchestra.run.smithi076.stdout: "op_w": 17675, 2024-09-11T23:40:04.025 INFO:teuthology.orchestra.run.smithi076.stdout: "op_rmw": 0, 2024-09-11T23:40:04.025 INFO:teuthology.orchestra.run.smithi076.stdout: "op_pg": 0, 2024-09-11T23:40:04.025 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_stat": 123, 2024-09-11T23:40:04.026 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_create": 1986, 2024-09-11T23:40:04.026 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_read": 333, 2024-09-11T23:40:04.026 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_write": 5119, 2024-09-11T23:40:04.026 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_writefull": 231, 2024-09-11T23:40:04.026 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_writesame": 0, 2024-09-11T23:40:04.026 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_append": 0, 2024-09-11T23:40:04.026 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_zero": 1, 2024-09-11T23:40:04.026 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_truncate": 91, 2024-09-11T23:40:04.026 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_delete": 8876, 2024-09-11T23:40:04.026 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_mapext": 0, 2024-09-11T23:40:04.026 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_sparse_read": 0, 2024-09-11T23:40:04.026 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_clonerange": 0, 2024-09-11T23:40:04.026 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_getxattr": 339, 2024-09-11T23:40:04.027 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_setxattr": 3972, 2024-09-11T23:40:04.027 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_cmpxattr": 0, 2024-09-11T23:40:04.027 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_rmxattr": 0, 2024-09-11T23:40:04.027 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_resetxattrs": 0, 2024-09-11T23:40:04.027 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_call": 0, 2024-09-11T23:40:04.027 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_watch": 0, 2024-09-11T23:40:04.027 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_notify": 0, 2024-09-11T23:40:04.027 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_src_cmpxattr": 0, 2024-09-11T23:40:04.027 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_pgls": 0, 2024-09-11T23:40:04.027 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_pgls_filter": 0, 2024-09-11T23:40:04.027 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_other": 0, 2024-09-11T23:40:04.027 INFO:teuthology.orchestra.run.smithi076.stdout: "linger_active": 0, 2024-09-11T23:40:04.027 INFO:teuthology.orchestra.run.smithi076.stdout: "linger_send": 0, 2024-09-11T23:40:04.027 INFO:teuthology.orchestra.run.smithi076.stdout: "linger_resend": 0, 2024-09-11T23:40:04.027 INFO:teuthology.orchestra.run.smithi076.stdout: "linger_ping": 0, 2024-09-11T23:40:04.028 INFO:teuthology.orchestra.run.smithi076.stdout: "poolop_active": 0, 2024-09-11T23:40:04.028 INFO:teuthology.orchestra.run.smithi076.stdout: "poolop_send": 0, 2024-09-11T23:40:04.028 INFO:teuthology.orchestra.run.smithi076.stdout: "poolop_resend": 0, 2024-09-11T23:40:04.028 INFO:teuthology.orchestra.run.smithi076.stdout: "poolstat_active": 0, 2024-09-11T23:40:04.028 INFO:teuthology.orchestra.run.smithi076.stdout: "poolstat_send": 0, 2024-09-11T23:40:04.028 INFO:teuthology.orchestra.run.smithi076.stdout: "poolstat_resend": 0, 2024-09-11T23:40:04.028 INFO:teuthology.orchestra.run.smithi076.stdout: "statfs_active": 0, 2024-09-11T23:40:04.028 INFO:teuthology.orchestra.run.smithi076.stdout: "statfs_send": 0, 2024-09-11T23:40:04.028 INFO:teuthology.orchestra.run.smithi076.stdout: "statfs_resend": 0, 2024-09-11T23:40:04.028 INFO:teuthology.orchestra.run.smithi076.stdout: "command_active": 0, 2024-09-11T23:40:04.028 INFO:teuthology.orchestra.run.smithi076.stdout: "command_send": 0, 2024-09-11T23:40:04.028 INFO:teuthology.orchestra.run.smithi076.stdout: "command_resend": 0, 2024-09-11T23:40:04.028 INFO:teuthology.orchestra.run.smithi076.stdout: "map_epoch": 116, 2024-09-11T23:40:04.028 INFO:teuthology.orchestra.run.smithi076.stdout: "map_full": 0, 2024-09-11T23:40:04.029 INFO:teuthology.orchestra.run.smithi076.stdout: "map_inc": 31, 2024-09-11T23:40:04.029 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_sessions": 12, 2024-09-11T23:40:04.029 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_session_open": 12, 2024-09-11T23:40:04.029 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_session_close": 0, 2024-09-11T23:40:04.029 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_laggy": 0, 2024-09-11T23:40:04.029 INFO:teuthology.orchestra.run.smithi076.stdout: "omap_wr": 2653, 2024-09-11T23:40:04.029 INFO:teuthology.orchestra.run.smithi076.stdout: "omap_rd": 78, 2024-09-11T23:40:04.029 INFO:teuthology.orchestra.run.smithi076.stdout: "omap_del": 479 2024-09-11T23:40:04.029 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:04.029 INFO:teuthology.orchestra.run.smithi076.stdout: "oft": { 2024-09-11T23:40:04.029 INFO:teuthology.orchestra.run.smithi076.stdout: "omap_total_objs": 1, 2024-09-11T23:40:04.029 INFO:teuthology.orchestra.run.smithi076.stdout: "omap_total_kv_pairs": 0, 2024-09-11T23:40:04.029 INFO:teuthology.orchestra.run.smithi076.stdout: "omap_total_updates": 0, 2024-09-11T23:40:04.029 INFO:teuthology.orchestra.run.smithi076.stdout: "omap_total_removes": 0 2024-09-11T23:40:04.030 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:04.030 INFO:teuthology.orchestra.run.smithi076.stdout: "osd-slow-ops": { 2024-09-11T23:40:04.030 INFO:teuthology.orchestra.run.smithi076.stdout: "slow_ops_count": 192040 2024-09-11T23:40:04.030 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:04.030 INFO:teuthology.orchestra.run.smithi076.stdout: "purge_queue": { 2024-09-11T23:40:04.030 INFO:teuthology.orchestra.run.smithi076.stdout: "pq_executing_ops": 0, 2024-09-11T23:40:04.030 INFO:teuthology.orchestra.run.smithi076.stdout: "pq_executing_ops_high_water": 6, 2024-09-11T23:40:04.030 INFO:teuthology.orchestra.run.smithi076.stdout: "pq_executing": 0, 2024-09-11T23:40:04.030 INFO:teuthology.orchestra.run.smithi076.stdout: "pq_executing_high_water": 6, 2024-09-11T23:40:04.030 INFO:teuthology.orchestra.run.smithi076.stdout: "pq_executed_ops": 8794, 2024-09-11T23:40:04.030 INFO:teuthology.orchestra.run.smithi076.stdout: "pq_executed": 8794, 2024-09-11T23:40:04.030 INFO:teuthology.orchestra.run.smithi076.stdout: "pq_item_in_journal": 0 2024-09-11T23:40:04.030 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:04.030 INFO:teuthology.orchestra.run.smithi076.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-11T23:40:04.031 INFO:teuthology.orchestra.run.smithi076.stdout: "val": 0, 2024-09-11T23:40:04.031 INFO:teuthology.orchestra.run.smithi076.stdout: "max": 104857600, 2024-09-11T23:40:04.031 INFO:teuthology.orchestra.run.smithi076.stdout: "get_started": 0, 2024-09-11T23:40:04.031 INFO:teuthology.orchestra.run.smithi076.stdout: "get": 382037, 2024-09-11T23:40:04.031 INFO:teuthology.orchestra.run.smithi076.stdout: "get_sum": 120676714, 2024-09-11T23:40:04.031 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_fail": 0, 2024-09-11T23:40:04.031 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_success": 382037, 2024-09-11T23:40:04.031 INFO:teuthology.orchestra.run.smithi076.stdout: "take": 0, 2024-09-11T23:40:04.031 INFO:teuthology.orchestra.run.smithi076.stdout: "take_sum": 0, 2024-09-11T23:40:04.031 INFO:teuthology.orchestra.run.smithi076.stdout: "put": 382037, 2024-09-11T23:40:04.031 INFO:teuthology.orchestra.run.smithi076.stdout: "put_sum": 120676714, 2024-09-11T23:40:04.031 INFO:teuthology.orchestra.run.smithi076.stdout: "wait": { 2024-09-11T23:40:04.031 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-09-11T23:40:04.032 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-09-11T23:40:04.032 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:40:03 smithi076 ceph-mon[35296]: pgmap v683: 97 pgs: 97 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 64 KiB/s wr, 36 op/s 2024-09-11T23:40:04.032 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-09-11T23:40:04.032 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-11T23:40:04.032 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:04.032 INFO:teuthology.orchestra.run.smithi076.stdout: "throttle-objecter_bytes": { 2024-09-11T23:40:04.032 INFO:teuthology.orchestra.run.smithi076.stdout: "val": 0, 2024-09-11T23:40:04.033 INFO:teuthology.orchestra.run.smithi076.stdout: "max": 104857600, 2024-09-11T23:40:04.033 INFO:teuthology.orchestra.run.smithi076.stdout: "get_started": 0, 2024-09-11T23:40:04.033 INFO:teuthology.orchestra.run.smithi076.stdout: "get": 0, 2024-09-11T23:40:04.033 INFO:teuthology.orchestra.run.smithi076.stdout: "get_sum": 0, 2024-09-11T23:40:04.033 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_fail": 0, 2024-09-11T23:40:04.033 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_success": 0, 2024-09-11T23:40:04.033 INFO:teuthology.orchestra.run.smithi076.stdout: "take": 18385, 2024-09-11T23:40:04.033 INFO:teuthology.orchestra.run.smithi076.stdout: "take_sum": 324536205, 2024-09-11T23:40:04.033 INFO:teuthology.orchestra.run.smithi076.stdout: "put": 18385, 2024-09-11T23:40:04.033 INFO:teuthology.orchestra.run.smithi076.stdout: "put_sum": 324536205, 2024-09-11T23:40:04.033 INFO:teuthology.orchestra.run.smithi076.stdout: "wait": { 2024-09-11T23:40:04.033 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-09-11T23:40:04.033 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-09-11T23:40:04.033 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-09-11T23:40:04.034 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-11T23:40:04.034 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:04.034 INFO:teuthology.orchestra.run.smithi076.stdout: "throttle-objecter_ops": { 2024-09-11T23:40:04.034 INFO:teuthology.orchestra.run.smithi076.stdout: "val": 0, 2024-09-11T23:40:04.034 INFO:teuthology.orchestra.run.smithi076.stdout: "max": 1024, 2024-09-11T23:40:04.034 INFO:teuthology.orchestra.run.smithi076.stdout: "get_started": 0, 2024-09-11T23:40:04.034 INFO:teuthology.orchestra.run.smithi076.stdout: "get": 0, 2024-09-11T23:40:04.034 INFO:teuthology.orchestra.run.smithi076.stdout: "get_sum": 0, 2024-09-11T23:40:04.034 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_fail": 0, 2024-09-11T23:40:04.034 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_success": 0, 2024-09-11T23:40:04.034 INFO:teuthology.orchestra.run.smithi076.stdout: "take": 18385, 2024-09-11T23:40:04.034 INFO:teuthology.orchestra.run.smithi076.stdout: "take_sum": 18385, 2024-09-11T23:40:04.034 INFO:teuthology.orchestra.run.smithi076.stdout: "put": 18385, 2024-09-11T23:40:04.035 INFO:teuthology.orchestra.run.smithi076.stdout: "put_sum": 18385, 2024-09-11T23:40:04.035 INFO:teuthology.orchestra.run.smithi076.stdout: "wait": { 2024-09-11T23:40:04.035 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-09-11T23:40:04.035 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-09-11T23:40:04.035 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-09-11T23:40:04.035 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-11T23:40:04.035 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:04.035 INFO:teuthology.orchestra.run.smithi076.stdout: "throttle-write_buf_throttle": { 2024-09-11T23:40:04.035 INFO:teuthology.orchestra.run.smithi076.stdout: "val": 0, 2024-09-11T23:40:04.035 INFO:teuthology.orchestra.run.smithi076.stdout: "max": 3758096384, 2024-09-11T23:40:04.035 INFO:teuthology.orchestra.run.smithi076.stdout: "get_started": 0, 2024-09-11T23:40:04.035 INFO:teuthology.orchestra.run.smithi076.stdout: "get": 8794, 2024-09-11T23:40:04.035 INFO:teuthology.orchestra.run.smithi076.stdout: "get_sum": 1253754, 2024-09-11T23:40:04.036 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_fail": 0, 2024-09-11T23:40:04.036 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_success": 8794, 2024-09-11T23:40:04.036 INFO:teuthology.orchestra.run.smithi076.stdout: "take": 0, 2024-09-11T23:40:04.036 INFO:teuthology.orchestra.run.smithi076.stdout: "take_sum": 0, 2024-09-11T23:40:04.036 INFO:teuthology.orchestra.run.smithi076.stdout: "put": 333, 2024-09-11T23:40:04.036 INFO:teuthology.orchestra.run.smithi076.stdout: "put_sum": 1253754, 2024-09-11T23:40:04.036 INFO:teuthology.orchestra.run.smithi076.stdout: "wait": { 2024-09-11T23:40:04.036 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-09-11T23:40:04.036 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-09-11T23:40:04.036 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-09-11T23:40:04.036 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-11T23:40:04.036 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:04.036 INFO:teuthology.orchestra.run.smithi076.stdout: "throttle-write_buf_throttle-0x55e73ecb0960": { 2024-09-11T23:40:04.036 INFO:teuthology.orchestra.run.smithi076.stdout: "val": 0, 2024-09-11T23:40:04.037 INFO:teuthology.orchestra.run.smithi076.stdout: "max": 3758096384, 2024-09-11T23:40:04.037 INFO:teuthology.orchestra.run.smithi076.stdout: "get_started": 0, 2024-09-11T23:40:04.037 INFO:teuthology.orchestra.run.smithi076.stdout: "get": 98369, 2024-09-11T23:40:04.037 INFO:teuthology.orchestra.run.smithi076.stdout: "get_sum": 299988120, 2024-09-11T23:40:04.037 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_fail": 0, 2024-09-11T23:40:04.037 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_success": 98369, 2024-09-11T23:40:04.037 INFO:teuthology.orchestra.run.smithi076.stdout: "take": 0, 2024-09-11T23:40:04.037 INFO:teuthology.orchestra.run.smithi076.stdout: "take_sum": 0, 2024-09-11T23:40:04.037 INFO:teuthology.orchestra.run.smithi076.stdout: "put": 4786, 2024-09-11T23:40:04.037 INFO:teuthology.orchestra.run.smithi076.stdout: "put_sum": 299988120, 2024-09-11T23:40:04.037 INFO:teuthology.orchestra.run.smithi076.stdout: "wait": { 2024-09-11T23:40:04.037 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-09-11T23:40:04.037 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-09-11T23:40:04.037 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-09-11T23:40:04.038 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-11T23:40:04.038 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-11T23:40:04.038 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-11T23:40:04.238 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:40:03 smithi136 ceph-mon[35121]: pgmap v683: 97 pgs: 97 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 64 KiB/s wr, 36 op/s 2024-09-11T23:40:04.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:40:03 smithi026 ceph-mon[31172]: pgmap v683: 97 pgs: 97 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 64 KiB/s wr, 36 op/s 2024-09-11T23:40:04.840 INFO:tasks.check_counter:Daemon mds.e mds.exported=143 2024-09-11T23:40:04.840 INFO:tasks.check_counter:Daemon mds.e mds.imported=222 2024-09-11T23:40:04.840 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update=3378 2024-09-11T23:40:04.840 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update_receipt=14214 2024-09-11T23:40:04.840 INFO:tasks.check_counter:Daemon mds.e mds.root_rsnaps=2 2024-09-11T23:40:04.840 INFO:tasks.check_counter:Daemon mds.e mds_server.req_mksnap_latency.avgcount=0 2024-09-11T23:40:04.840 INFO:tasks.check_counter:Daemon mds.e mds_server.req_rmsnap_latency.avgcount=0 2024-09-11T23:40:04.840 INFO:tasks.check_counter:Daemon mds.e mds.dir_split=91 2024-09-11T23:40:04.840 DEBUG:tasks.check_counter:Getting stats from h 2024-09-11T23:40:04.841 DEBUG:teuthology.orchestra.run.smithi076:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph daemon mds.h perf dump 2024-09-11T23:40:05.092 INFO:teuthology.orchestra.run.smithi076.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T23:40:05.599 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-11T23:40:05.600 INFO:teuthology.orchestra.run.smithi076.stdout: "AsyncMessenger::Worker-0": { 2024-09-11T23:40:05.600 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_messages": 69560, 2024-09-11T23:40:05.600 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_messages": 71223, 2024-09-11T23:40:05.600 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_bytes": 56814298, 2024-09-11T23:40:05.600 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_bytes": 146064801, 2024-09-11T23:40:05.600 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_created_connections": 17, 2024-09-11T23:40:05.600 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_active_connections": 3, 2024-09-11T23:40:05.600 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_total_time": 4.793823099, 2024-09-11T23:40:05.600 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_send_time": 2.207917395, 2024-09-11T23:40:05.600 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_recv_time": 3.026835643, 2024-09-11T23:40:05.600 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_fast_dispatch_time": 0.362125402, 2024-09-11T23:40:05.600 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T23:40:05.601 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 71224, 2024-09-11T23:40:05.601 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 10.556008048, 2024-09-11T23:40:05.601 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000148208 2024-09-11T23:40:05.601 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:05.601 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_handle_ack_lat": { 2024-09-11T23:40:05.601 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 122957, 2024-09-11T23:40:05.601 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.023298598, 2024-09-11T23:40:05.601 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000189 2024-09-11T23:40:05.601 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:05.601 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-09-11T23:40:05.601 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_encrypted_bytes": 448 2024-09-11T23:40:05.601 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:05.602 INFO:teuthology.orchestra.run.smithi076.stdout: "AsyncMessenger::Worker-1": { 2024-09-11T23:40:05.602 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_messages": 244717, 2024-09-11T23:40:05.602 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_messages": 349765, 2024-09-11T23:40:05.602 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_bytes": 93161429, 2024-09-11T23:40:05.602 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_bytes": 333780010, 2024-09-11T23:40:05.602 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_created_connections": 16, 2024-09-11T23:40:05.602 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_active_connections": 9, 2024-09-11T23:40:05.602 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_total_time": 15.513661093, 2024-09-11T23:40:05.602 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_send_time": 7.389191048, 2024-09-11T23:40:05.602 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_recv_time": 17.559195809, 2024-09-11T23:40:05.602 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_fast_dispatch_time": 0.521020868, 2024-09-11T23:40:05.602 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T23:40:05.602 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 349758, 2024-09-11T23:40:05.602 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 42.120203096, 2024-09-11T23:40:05.603 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000120426 2024-09-11T23:40:05.603 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:05.603 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_handle_ack_lat": { 2024-09-11T23:40:05.603 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 470612, 2024-09-11T23:40:05.603 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.085540195, 2024-09-11T23:40:05.603 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000181 2024-09-11T23:40:05.603 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:05.603 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_encrypted_bytes": 90099136, 2024-09-11T23:40:05.603 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_encrypted_bytes": 181943888 2024-09-11T23:40:05.603 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:05.604 INFO:teuthology.orchestra.run.smithi076.stdout: "AsyncMessenger::Worker-2": { 2024-09-11T23:40:05.604 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_messages": 61364, 2024-09-11T23:40:05.604 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_messages": 53415, 2024-09-11T23:40:05.604 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_bytes": 27807412, 2024-09-11T23:40:05.604 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_bytes": 105514458, 2024-09-11T23:40:05.604 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_created_connections": 13, 2024-09-11T23:40:05.604 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_active_connections": 5, 2024-09-11T23:40:05.604 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_total_time": 4.059128234, 2024-09-11T23:40:05.604 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_send_time": 1.868216751, 2024-09-11T23:40:05.604 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_recv_time": 2.357115756, 2024-09-11T23:40:05.604 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_fast_dispatch_time": 0.366509822, 2024-09-11T23:40:05.604 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T23:40:05.605 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 53416, 2024-09-11T23:40:05.605 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 7.055721985, 2024-09-11T23:40:05.605 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000132090 2024-09-11T23:40:05.605 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:05.605 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_handle_ack_lat": { 2024-09-11T23:40:05.605 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 96230, 2024-09-11T23:40:05.605 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.016780057, 2024-09-11T23:40:05.605 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000174 2024-09-11T23:40:05.605 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:05.605 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_encrypted_bytes": 2135392, 2024-09-11T23:40:05.605 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_encrypted_bytes": 334976 2024-09-11T23:40:05.605 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:05.605 INFO:teuthology.orchestra.run.smithi076.stdout: "cct": { 2024-09-11T23:40:05.605 INFO:teuthology.orchestra.run.smithi076.stdout: "total_workers": 1, 2024-09-11T23:40:05.606 INFO:teuthology.orchestra.run.smithi076.stdout: "unhealthy_workers": 0 2024-09-11T23:40:05.606 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:05.606 INFO:teuthology.orchestra.run.smithi076.stdout: "finisher-MDSRank": { 2024-09-11T23:40:05.606 INFO:teuthology.orchestra.run.smithi076.stdout: "queue_len": 0, 2024-09-11T23:40:05.606 INFO:teuthology.orchestra.run.smithi076.stdout: "complete_latency": { 2024-09-11T23:40:05.606 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 8383, 2024-09-11T23:40:05.606 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 81.103561980, 2024-09-11T23:40:05.606 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.009674765 2024-09-11T23:40:05.606 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-11T23:40:05.606 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:05.606 INFO:teuthology.orchestra.run.smithi076.stdout: "finisher-PurgeQueue": { 2024-09-11T23:40:05.607 INFO:teuthology.orchestra.run.smithi076.stdout: "queue_len": 0, 2024-09-11T23:40:05.607 INFO:teuthology.orchestra.run.smithi076.stdout: "complete_latency": { 2024-09-11T23:40:05.607 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 18126, 2024-09-11T23:40:05.607 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 3.977070909, 2024-09-11T23:40:05.607 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000219412 2024-09-11T23:40:05.607 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-11T23:40:05.607 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:05.607 INFO:teuthology.orchestra.run.smithi076.stdout: "mds": { 2024-09-11T23:40:05.607 INFO:teuthology.orchestra.run.smithi076.stdout: "request": 117473, 2024-09-11T23:40:05.608 INFO:teuthology.orchestra.run.smithi076.stdout: "reply": 100273, 2024-09-11T23:40:05.608 INFO:teuthology.orchestra.run.smithi076.stdout: "reply_latency": { 2024-09-11T23:40:05.608 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 100273, 2024-09-11T23:40:05.608 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 3481.403445820, 2024-09-11T23:40:05.608 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.034719250 2024-09-11T23:40:05.608 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:05.608 INFO:teuthology.orchestra.run.smithi076.stdout: "slow_reply": 0, 2024-09-11T23:40:05.608 INFO:teuthology.orchestra.run.smithi076.stdout: "forward": 17200, 2024-09-11T23:40:05.608 INFO:teuthology.orchestra.run.smithi076.stdout: "dir_fetch_complete": 161, 2024-09-11T23:40:05.609 INFO:teuthology.orchestra.run.smithi076.stdout: "dir_fetch_keys": 0, 2024-09-11T23:40:05.609 INFO:teuthology.orchestra.run.smithi076.stdout: "dir_commit": 1142, 2024-09-11T23:40:05.609 INFO:teuthology.orchestra.run.smithi076.stdout: "dir_split": 110, 2024-09-11T23:40:05.609 INFO:teuthology.orchestra.run.smithi076.stdout: "dir_merge": 0, 2024-09-11T23:40:05.609 INFO:teuthology.orchestra.run.smithi076.stdout: "inodes": 26549, 2024-09-11T23:40:05.609 INFO:teuthology.orchestra.run.smithi076.stdout: "inodes_top": 722, 2024-09-11T23:40:05.609 INFO:teuthology.orchestra.run.smithi076.stdout: "inodes_bottom": 13480, 2024-09-11T23:40:05.610 INFO:teuthology.orchestra.run.smithi076.stdout: "inodes_pin_tail": 12347, 2024-09-11T23:40:05.610 INFO:teuthology.orchestra.run.smithi076.stdout: "inodes_pinned": 25514, 2024-09-11T23:40:05.610 INFO:teuthology.orchestra.run.smithi076.stdout: "inodes_expired": 79810, 2024-09-11T23:40:05.610 INFO:teuthology.orchestra.run.smithi076.stdout: "inodes_with_caps": 1, 2024-09-11T23:40:05.610 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": 1, 2024-09-11T23:40:05.610 INFO:teuthology.orchestra.run.smithi076.stdout: "subtrees": 7, 2024-09-11T23:40:05.610 INFO:teuthology.orchestra.run.smithi076.stdout: "traverse": 195009, 2024-09-11T23:40:05.610 INFO:teuthology.orchestra.run.smithi076.stdout: "traverse_hit": 149733, 2024-09-11T23:40:05.610 INFO:teuthology.orchestra.run.smithi076.stdout: "traverse_forward": 16696, 2024-09-11T23:40:05.610 INFO:teuthology.orchestra.run.smithi076.stdout: "traverse_discover": 1208, 2024-09-11T23:40:05.610 INFO:teuthology.orchestra.run.smithi076.stdout: "traverse_dir_fetch": 28, 2024-09-11T23:40:05.610 INFO:teuthology.orchestra.run.smithi076.stdout: "traverse_remote_ino": 0, 2024-09-11T23:40:05.610 INFO:teuthology.orchestra.run.smithi076.stdout: "traverse_lock": 219, 2024-09-11T23:40:05.611 INFO:teuthology.orchestra.run.smithi076.stdout: "load_cent": 60, 2024-09-11T23:40:05.611 INFO:teuthology.orchestra.run.smithi076.stdout: "q": 0, 2024-09-11T23:40:05.611 INFO:teuthology.orchestra.run.smithi076.stdout: "exported": 353, 2024-09-11T23:40:05.611 INFO:teuthology.orchestra.run.smithi076.stdout: "exported_inodes": 62368, 2024-09-11T23:40:05.611 INFO:teuthology.orchestra.run.smithi076.stdout: "imported": 266, 2024-09-11T23:40:05.611 INFO:teuthology.orchestra.run.smithi076.stdout: "imported_inodes": 48204, 2024-09-11T23:40:05.611 INFO:teuthology.orchestra.run.smithi076.stdout: "openino_dir_fetch": 0, 2024-09-11T23:40:05.611 INFO:teuthology.orchestra.run.smithi076.stdout: "openino_backtrace_fetch": 476, 2024-09-11T23:40:05.611 INFO:teuthology.orchestra.run.smithi076.stdout: "openino_peer_discover": 0, 2024-09-11T23:40:05.611 INFO:teuthology.orchestra.run.smithi076.stdout: "root_rfiles": 7006, 2024-09-11T23:40:05.611 INFO:teuthology.orchestra.run.smithi076.stdout: "root_rbytes": 34252265, 2024-09-11T23:40:05.611 INFO:teuthology.orchestra.run.smithi076.stdout: "root_rsnaps": 2, 2024-09-11T23:40:05.611 INFO:teuthology.orchestra.run.smithi076.stdout: "scrub_backtrace_fetch": 476, 2024-09-11T23:40:05.611 INFO:teuthology.orchestra.run.smithi076.stdout: "scrub_set_tag": 0, 2024-09-11T23:40:05.612 INFO:teuthology.orchestra.run.smithi076.stdout: "scrub_backtrace_repaired": 0, 2024-09-11T23:40:05.612 INFO:teuthology.orchestra.run.smithi076.stdout: "scrub_inotable_repaired": 0, 2024-09-11T23:40:05.612 INFO:teuthology.orchestra.run.smithi076.stdout: "scrub_dir_inodes": 0, 2024-09-11T23:40:05.612 INFO:teuthology.orchestra.run.smithi076.stdout: "scrub_dir_base_inodes": 0, 2024-09-11T23:40:05.612 INFO:teuthology.orchestra.run.smithi076.stdout: "scrub_dirfrag_rstats": 0, 2024-09-11T23:40:05.612 INFO:teuthology.orchestra.run.smithi076.stdout: "scrub_file_inodes": 476, 2024-09-11T23:40:05.612 INFO:teuthology.orchestra.run.smithi076.stdout: "handle_inode_file_caps": 213, 2024-09-11T23:40:05.612 INFO:teuthology.orchestra.run.smithi076.stdout: "ceph_cap_op_revoke": 69815, 2024-09-11T23:40:05.612 INFO:teuthology.orchestra.run.smithi076.stdout: "ceph_cap_op_grant": 51107, 2024-09-11T23:40:05.612 INFO:teuthology.orchestra.run.smithi076.stdout: "ceph_cap_op_trunc": 0, 2024-09-11T23:40:05.612 INFO:teuthology.orchestra.run.smithi076.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-11T23:40:05.612 INFO:teuthology.orchestra.run.smithi076.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-11T23:40:05.612 INFO:teuthology.orchestra.run.smithi076.stdout: "handle_client_caps": 90678, 2024-09-11T23:40:05.613 INFO:teuthology.orchestra.run.smithi076.stdout: "handle_client_caps_dirty": 22609, 2024-09-11T23:40:05.613 INFO:teuthology.orchestra.run.smithi076.stdout: "handle_client_cap_release": 3135, 2024-09-11T23:40:05.613 INFO:teuthology.orchestra.run.smithi076.stdout: "process_request_cap_release": 29925 2024-09-11T23:40:05.613 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:05.613 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_cache": { 2024-09-11T23:40:05.613 INFO:teuthology.orchestra.run.smithi076.stdout: "dir_update": 3293, 2024-09-11T23:40:05.613 INFO:teuthology.orchestra.run.smithi076.stdout: "dir_update_receipt": 14288, 2024-09-11T23:40:05.613 INFO:teuthology.orchestra.run.smithi076.stdout: "dir_try_discover": 8237, 2024-09-11T23:40:05.613 INFO:teuthology.orchestra.run.smithi076.stdout: "dir_send_discover": 9843, 2024-09-11T23:40:05.613 INFO:teuthology.orchestra.run.smithi076.stdout: "dir_handle_discover": 6878, 2024-09-11T23:40:05.613 INFO:teuthology.orchestra.run.smithi076.stdout: "num_strays": 81, 2024-09-11T23:40:05.613 INFO:teuthology.orchestra.run.smithi076.stdout: "num_strays_delayed": 0, 2024-09-11T23:40:05.614 INFO:teuthology.orchestra.run.smithi076.stdout: "num_strays_enqueuing": 0, 2024-09-11T23:40:05.614 INFO:teuthology.orchestra.run.smithi076.stdout: "strays_created": 9619, 2024-09-11T23:40:05.614 INFO:teuthology.orchestra.run.smithi076.stdout: "strays_enqueued": 9538, 2024-09-11T23:40:05.614 INFO:teuthology.orchestra.run.smithi076.stdout: "strays_reintegrated": 0, 2024-09-11T23:40:05.614 INFO:teuthology.orchestra.run.smithi076.stdout: "strays_migrated": 0, 2024-09-11T23:40:05.614 INFO:teuthology.orchestra.run.smithi076.stdout: "num_recovering_processing": 0, 2024-09-11T23:40:05.614 INFO:teuthology.orchestra.run.smithi076.stdout: "num_recovering_enqueued": 0, 2024-09-11T23:40:05.614 INFO:teuthology.orchestra.run.smithi076.stdout: "num_recovering_prioritized": 0, 2024-09-11T23:40:05.614 INFO:teuthology.orchestra.run.smithi076.stdout: "recovery_started": 0, 2024-09-11T23:40:05.614 INFO:teuthology.orchestra.run.smithi076.stdout: "recovery_completed": 0, 2024-09-11T23:40:05.614 INFO:teuthology.orchestra.run.smithi076.stdout: "ireq_quiesce_path": 18, 2024-09-11T23:40:05.614 INFO:teuthology.orchestra.run.smithi076.stdout: "ireq_quiesce_inode": 162630, 2024-09-11T23:40:05.614 INFO:teuthology.orchestra.run.smithi076.stdout: "ireq_enqueue_scrub": 0, 2024-09-11T23:40:05.614 INFO:teuthology.orchestra.run.smithi076.stdout: "ireq_exportdir": 353, 2024-09-11T23:40:05.615 INFO:teuthology.orchestra.run.smithi076.stdout: "ireq_flush": 0, 2024-09-11T23:40:05.615 INFO:teuthology.orchestra.run.smithi076.stdout: "ireq_fragmentdir": 110, 2024-09-11T23:40:05.615 INFO:teuthology.orchestra.run.smithi076.stdout: "ireq_fragstats": 0, 2024-09-11T23:40:05.615 INFO:teuthology.orchestra.run.smithi076.stdout: "ireq_inodestats": 0 2024-09-11T23:40:05.615 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:05.615 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_log": { 2024-09-11T23:40:05.615 INFO:teuthology.orchestra.run.smithi076.stdout: "evlrg": 3, 2024-09-11T23:40:05.615 INFO:teuthology.orchestra.run.smithi076.stdout: "evadd": 104390, 2024-09-11T23:40:05.615 INFO:teuthology.orchestra.run.smithi076.stdout: "evex": 16217, 2024-09-11T23:40:05.615 INFO:teuthology.orchestra.run.smithi076.stdout: "evtrm": 12291, 2024-09-11T23:40:05.616 INFO:teuthology.orchestra.run.smithi076.stdout: "ev": 92099, 2024-09-11T23:40:05.616 INFO:teuthology.orchestra.run.smithi076.stdout: "evexg": 0, 2024-09-11T23:40:05.616 INFO:teuthology.orchestra.run.smithi076.stdout: "evexd": 3926, 2024-09-11T23:40:05.616 INFO:teuthology.orchestra.run.smithi076.stdout: "segadd": 150, 2024-09-11T23:40:05.616 INFO:teuthology.orchestra.run.smithi076.stdout: "segex": 22, 2024-09-11T23:40:05.617 INFO:teuthology.orchestra.run.smithi076.stdout: "segtrm": 17, 2024-09-11T23:40:05.617 INFO:teuthology.orchestra.run.smithi076.stdout: "seg": 133, 2024-09-11T23:40:05.617 INFO:teuthology.orchestra.run.smithi076.stdout: "segmjr": 8, 2024-09-11T23:40:05.617 INFO:teuthology.orchestra.run.smithi076.stdout: "segexg": 0, 2024-09-11T23:40:05.617 INFO:teuthology.orchestra.run.smithi076.stdout: "segexd": 5, 2024-09-11T23:40:05.617 INFO:teuthology.orchestra.run.smithi076.stdout: "expos": 33882821, 2024-09-11T23:40:05.617 INFO:teuthology.orchestra.run.smithi076.stdout: "wrpos": 338401870, 2024-09-11T23:40:05.617 INFO:teuthology.orchestra.run.smithi076.stdout: "rdpos": 0, 2024-09-11T23:40:05.617 INFO:teuthology.orchestra.run.smithi076.stdout: "jlat": { 2024-09-11T23:40:05.618 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 5754, 2024-09-11T23:40:05.618 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 407.161975979, 2024-09-11T23:40:05.618 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.070761553 2024-09-11T23:40:05.618 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:05.618 INFO:teuthology.orchestra.run.smithi076.stdout: "replayed": 0 2024-09-11T23:40:05.620 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:05.621 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_mem": { 2024-09-11T23:40:05.621 INFO:teuthology.orchestra.run.smithi076.stdout: "ino": 14741, 2024-09-11T23:40:05.621 INFO:teuthology.orchestra.run.smithi076.stdout: "ino+": 72115, 2024-09-11T23:40:05.621 INFO:teuthology.orchestra.run.smithi076.stdout: "ino-": 57374, 2024-09-11T23:40:05.621 INFO:teuthology.orchestra.run.smithi076.stdout: "dir": 631, 2024-09-11T23:40:05.621 INFO:teuthology.orchestra.run.smithi076.stdout: "dir+": 8177, 2024-09-11T23:40:05.621 INFO:teuthology.orchestra.run.smithi076.stdout: "dir-": 7546, 2024-09-11T23:40:05.621 INFO:teuthology.orchestra.run.smithi076.stdout: "dn": 26549, 2024-09-11T23:40:05.622 INFO:teuthology.orchestra.run.smithi076.stdout: "dn+": 117619, 2024-09-11T23:40:05.622 INFO:teuthology.orchestra.run.smithi076.stdout: "dn-": 91070, 2024-09-11T23:40:05.622 INFO:teuthology.orchestra.run.smithi076.stdout: "cap": 1, 2024-09-11T23:40:05.622 INFO:teuthology.orchestra.run.smithi076.stdout: "cap+": 36207, 2024-09-11T23:40:05.622 INFO:teuthology.orchestra.run.smithi076.stdout: "cap-": 36206, 2024-09-11T23:40:05.622 INFO:teuthology.orchestra.run.smithi076.stdout: "rss": 254676, 2024-09-11T23:40:05.622 INFO:teuthology.orchestra.run.smithi076.stdout: "heap": 223520 2024-09-11T23:40:05.622 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:05.622 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_server": { 2024-09-11T23:40:05.622 INFO:teuthology.orchestra.run.smithi076.stdout: "dispatch_client_request": 122535, 2024-09-11T23:40:05.622 INFO:teuthology.orchestra.run.smithi076.stdout: "dispatch_server_request": 6610, 2024-09-11T23:40:05.622 INFO:teuthology.orchestra.run.smithi076.stdout: "handle_client_request": 117473, 2024-09-11T23:40:05.622 INFO:teuthology.orchestra.run.smithi076.stdout: "handle_client_session": 345, 2024-09-11T23:40:05.623 INFO:teuthology.orchestra.run.smithi076.stdout: "handle_peer_request": 14448, 2024-09-11T23:40:05.623 INFO:teuthology.orchestra.run.smithi076.stdout: "req_create_latency": { 2024-09-11T23:40:05.623 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 17256, 2024-09-11T23:40:05.623 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 125.465078058, 2024-09-11T23:40:05.623 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.007270808 2024-09-11T23:40:05.623 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:05.623 INFO:teuthology.orchestra.run.smithi076.stdout: "req_getattr_latency": { 2024-09-11T23:40:05.623 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 217, 2024-09-11T23:40:05.623 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 53.295757030, 2024-09-11T23:40:05.623 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.245602566 2024-09-11T23:40:05.623 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:05.624 INFO:teuthology.orchestra.run.smithi076.stdout: "req_getfilelock_latency": { 2024-09-11T23:40:05.624 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-09-11T23:40:05.624 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-09-11T23:40:05.624 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-09-11T23:40:05.624 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:05.624 INFO:teuthology.orchestra.run.smithi076.stdout: "req_link_latency": { 2024-09-11T23:40:05.624 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-09-11T23:40:05.624 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-09-11T23:40:05.624 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-09-11T23:40:05.624 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:05.624 INFO:teuthology.orchestra.run.smithi076.stdout: "req_lookup_latency": { 2024-09-11T23:40:05.624 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 47680, 2024-09-11T23:40:05.624 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 2881.109804951, 2024-09-11T23:40:05.625 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.060425960 2024-09-11T23:40:05.625 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:05.625 INFO:teuthology.orchestra.run.smithi076.stdout: "req_lookuphash_latency": { 2024-09-11T23:40:05.625 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-09-11T23:40:05.625 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-09-11T23:40:05.625 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-09-11T23:40:05.625 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:05.625 INFO:teuthology.orchestra.run.smithi076.stdout: "req_lookupino_latency": { 2024-09-11T23:40:05.625 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 1, 2024-09-11T23:40:05.626 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000118487, 2024-09-11T23:40:05.626 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000118487 2024-09-11T23:40:05.626 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:05.626 INFO:teuthology.orchestra.run.smithi076.stdout: "req_lookupname_latency": { 2024-09-11T23:40:05.626 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-09-11T23:40:05.626 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-09-11T23:40:05.626 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-09-11T23:40:05.626 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:05.626 INFO:teuthology.orchestra.run.smithi076.stdout: "req_lookupparent_latency": { 2024-09-11T23:40:05.626 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-09-11T23:40:05.626 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-09-11T23:40:05.626 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-09-11T23:40:05.627 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:05.627 INFO:teuthology.orchestra.run.smithi076.stdout: "req_lookupsnap_latency": { 2024-09-11T23:40:05.627 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-09-11T23:40:05.627 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-09-11T23:40:05.627 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-09-11T23:40:05.627 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:05.627 INFO:teuthology.orchestra.run.smithi076.stdout: "req_lssnap_latency": { 2024-09-11T23:40:05.627 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-09-11T23:40:05.627 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-09-11T23:40:05.627 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-09-11T23:40:05.628 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:05.628 INFO:teuthology.orchestra.run.smithi076.stdout: "req_mkdir_latency": { 2024-09-11T23:40:05.628 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 120, 2024-09-11T23:40:05.628 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 39.305250202, 2024-09-11T23:40:05.628 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.327543751 2024-09-11T23:40:05.628 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:05.628 INFO:teuthology.orchestra.run.smithi076.stdout: "req_mknod_latency": { 2024-09-11T23:40:05.628 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-09-11T23:40:05.628 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-09-11T23:40:05.628 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-09-11T23:40:05.628 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:05.629 INFO:teuthology.orchestra.run.smithi076.stdout: "req_mksnap_latency": { 2024-09-11T23:40:05.629 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-09-11T23:40:05.629 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-09-11T23:40:05.629 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-09-11T23:40:05.629 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:05.629 INFO:teuthology.orchestra.run.smithi076.stdout: "req_open_latency": { 2024-09-11T23:40:05.629 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 9549, 2024-09-11T23:40:05.629 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 121.749156386, 2024-09-11T23:40:05.629 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.012749937 2024-09-11T23:40:05.629 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:05.630 INFO:teuthology.orchestra.run.smithi076.stdout: "req_readdir_latency": { 2024-09-11T23:40:05.630 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 415, 2024-09-11T23:40:05.630 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.954621590, 2024-09-11T23:40:05.630 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.002300292 2024-09-11T23:40:05.630 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:05.630 INFO:teuthology.orchestra.run.smithi076.stdout: "req_rename_latency": { 2024-09-11T23:40:05.630 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 17258, 2024-09-11T23:40:05.630 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 238.691924815, 2024-09-11T23:40:05.630 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.013830798 2024-09-11T23:40:05.630 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:05.630 INFO:teuthology.orchestra.run.smithi076.stdout: "req_renamesnap_latency": { 2024-09-11T23:40:05.630 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-09-11T23:40:05.631 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-09-11T23:40:05.631 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-09-11T23:40:05.631 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:05.631 INFO:teuthology.orchestra.run.smithi076.stdout: "req_snapdiff_latency": { 2024-09-11T23:40:05.631 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-09-11T23:40:05.631 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-09-11T23:40:05.631 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-09-11T23:40:05.631 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:05.631 INFO:teuthology.orchestra.run.smithi076.stdout: "req_rmdir_latency": { 2024-09-11T23:40:05.631 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 82, 2024-09-11T23:40:05.631 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 7.855733311, 2024-09-11T23:40:05.631 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.095801625 2024-09-11T23:40:05.632 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:05.632 INFO:teuthology.orchestra.run.smithi076.stdout: "req_rmsnap_latency": { 2024-09-11T23:40:05.632 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-09-11T23:40:05.632 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-09-11T23:40:05.632 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-09-11T23:40:05.632 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:05.632 INFO:teuthology.orchestra.run.smithi076.stdout: "req_rmxattr_latency": { 2024-09-11T23:40:05.632 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-09-11T23:40:05.632 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-09-11T23:40:05.632 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-09-11T23:40:05.633 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:05.633 INFO:teuthology.orchestra.run.smithi076.stdout: "req_setattr_latency": { 2024-09-11T23:40:05.633 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-09-11T23:40:05.633 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-09-11T23:40:05.633 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-09-11T23:40:05.633 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:05.633 INFO:teuthology.orchestra.run.smithi076.stdout: "req_setdirlayout_latency": { 2024-09-11T23:40:05.633 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-09-11T23:40:05.633 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-09-11T23:40:05.633 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-09-11T23:40:05.633 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:05.634 INFO:teuthology.orchestra.run.smithi076.stdout: "req_setfilelock_latency": { 2024-09-11T23:40:05.634 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-09-11T23:40:05.634 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-09-11T23:40:05.634 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-09-11T23:40:05.634 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:05.634 INFO:teuthology.orchestra.run.smithi076.stdout: "req_setlayout_latency": { 2024-09-11T23:40:05.634 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-09-11T23:40:05.634 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-09-11T23:40:05.634 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-09-11T23:40:05.634 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:05.635 INFO:teuthology.orchestra.run.smithi076.stdout: "req_setxattr_latency": { 2024-09-11T23:40:05.635 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-09-11T23:40:05.635 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-09-11T23:40:05.635 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-09-11T23:40:05.635 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:05.635 INFO:teuthology.orchestra.run.smithi076.stdout: "req_symlink_latency": { 2024-09-11T23:40:05.635 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-09-11T23:40:05.635 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-09-11T23:40:05.635 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-09-11T23:40:05.635 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:05.635 INFO:teuthology.orchestra.run.smithi076.stdout: "req_unlink_latency": { 2024-09-11T23:40:05.636 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 7695, 2024-09-11T23:40:05.636 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 12.976000990, 2024-09-11T23:40:05.636 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.001686289 2024-09-11T23:40:05.636 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:05.636 INFO:teuthology.orchestra.run.smithi076.stdout: "cap_revoke_eviction": 0, 2024-09-11T23:40:05.636 INFO:teuthology.orchestra.run.smithi076.stdout: "cap_acquisition_throttle": 0, 2024-09-11T23:40:05.636 INFO:teuthology.orchestra.run.smithi076.stdout: "req_getvxattr_latency": { 2024-09-11T23:40:05.636 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-09-11T23:40:05.636 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-09-11T23:40:05.636 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-09-11T23:40:05.636 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-11T23:40:05.637 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:05.637 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_sessions": { 2024-09-11T23:40:05.637 INFO:teuthology.orchestra.run.smithi076.stdout: "session_count": 2, 2024-09-11T23:40:05.637 INFO:teuthology.orchestra.run.smithi076.stdout: "session_add": 2, 2024-09-11T23:40:05.637 INFO:teuthology.orchestra.run.smithi076.stdout: "session_remove": 0, 2024-09-11T23:40:05.637 INFO:teuthology.orchestra.run.smithi076.stdout: "sessions_open": 2, 2024-09-11T23:40:05.637 INFO:teuthology.orchestra.run.smithi076.stdout: "sessions_stale": 0, 2024-09-11T23:40:05.637 INFO:teuthology.orchestra.run.smithi076.stdout: "total_load": 170, 2024-09-11T23:40:05.637 INFO:teuthology.orchestra.run.smithi076.stdout: "average_load": 85, 2024-09-11T23:40:05.637 INFO:teuthology.orchestra.run.smithi076.stdout: "avg_session_uptime": 1270, 2024-09-11T23:40:05.637 INFO:teuthology.orchestra.run.smithi076.stdout: "mdthresh_evicted": 0 2024-09-11T23:40:05.637 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:05.638 INFO:teuthology.orchestra.run.smithi076.stdout: "mempool": { 2024-09-11T23:40:05.638 INFO:teuthology.orchestra.run.smithi076.stdout: "bloom_filter_bytes": 0, 2024-09-11T23:40:05.638 INFO:teuthology.orchestra.run.smithi076.stdout: "bloom_filter_items": 0, 2024-09-11T23:40:05.638 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_alloc_bytes": 0, 2024-09-11T23:40:05.638 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_alloc_items": 0, 2024-09-11T23:40:05.638 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_data_bytes": 0, 2024-09-11T23:40:05.638 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_data_items": 0, 2024-09-11T23:40:05.638 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-11T23:40:05.638 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_onode_items": 0, 2024-09-11T23:40:05.638 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-11T23:40:05.638 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_meta_items": 0, 2024-09-11T23:40:05.638 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_other_bytes": 0, 2024-09-11T23:40:05.639 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_other_items": 0, 2024-09-11T23:40:05.639 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-11T23:40:05.639 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_buffer_items": 0, 2024-09-11T23:40:05.639 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_extent_bytes": 0, 2024-09-11T23:40:05.639 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_extent_items": 0, 2024-09-11T23:40:05.639 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_blob_bytes": 0, 2024-09-11T23:40:05.639 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_blob_items": 0, 2024-09-11T23:40:05.639 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-11T23:40:05.639 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_shared_blob_items": 0, 2024-09-11T23:40:05.639 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-11T23:40:05.639 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_inline_bl_items": 0, 2024-09-11T23:40:05.640 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_fsck_bytes": 0, 2024-09-11T23:40:05.640 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_fsck_items": 0, 2024-09-11T23:40:05.640 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_txc_bytes": 0, 2024-09-11T23:40:05.640 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_txc_items": 0, 2024-09-11T23:40:05.640 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-11T23:40:05.640 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_writing_deferred_items": 0, 2024-09-11T23:40:05.640 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_writing_bytes": 0, 2024-09-11T23:40:05.641 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_writing_items": 0, 2024-09-11T23:40:05.641 INFO:teuthology.orchestra.run.smithi076.stdout: "bluefs_bytes": 0, 2024-09-11T23:40:05.641 INFO:teuthology.orchestra.run.smithi076.stdout: "bluefs_items": 0, 2024-09-11T23:40:05.641 INFO:teuthology.orchestra.run.smithi076.stdout: "bluefs_file_reader_bytes": 0, 2024-09-11T23:40:05.641 INFO:teuthology.orchestra.run.smithi076.stdout: "bluefs_file_reader_items": 0, 2024-09-11T23:40:05.641 INFO:teuthology.orchestra.run.smithi076.stdout: "bluefs_file_writer_bytes": 0, 2024-09-11T23:40:05.641 INFO:teuthology.orchestra.run.smithi076.stdout: "bluefs_file_writer_items": 0, 2024-09-11T23:40:05.641 INFO:teuthology.orchestra.run.smithi076.stdout: "buffer_anon_bytes": 1383187, 2024-09-11T23:40:05.641 INFO:teuthology.orchestra.run.smithi076.stdout: "buffer_anon_items": 11452, 2024-09-11T23:40:05.641 INFO:teuthology.orchestra.run.smithi076.stdout: "buffer_meta_bytes": 0, 2024-09-11T23:40:05.641 INFO:teuthology.orchestra.run.smithi076.stdout: "buffer_meta_items": 0, 2024-09-11T23:40:05.641 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_bytes": 0, 2024-09-11T23:40:05.642 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_items": 0, 2024-09-11T23:40:05.642 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_mapbl_bytes": 0, 2024-09-11T23:40:05.642 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_mapbl_items": 0, 2024-09-11T23:40:05.642 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_pglog_bytes": 0, 2024-09-11T23:40:05.642 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_pglog_items": 0, 2024-09-11T23:40:05.642 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_bytes": 8224, 2024-09-11T23:40:05.642 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_items": 181, 2024-09-11T23:40:05.642 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_mapping_bytes": 0, 2024-09-11T23:40:05.642 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_mapping_items": 0, 2024-09-11T23:40:05.642 INFO:teuthology.orchestra.run.smithi076.stdout: "pgmap_bytes": 0, 2024-09-11T23:40:05.642 INFO:teuthology.orchestra.run.smithi076.stdout: "pgmap_items": 0, 2024-09-11T23:40:05.643 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_co_bytes": 53671722, 2024-09-11T23:40:05.643 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_co_items": 501163, 2024-09-11T23:40:05.643 INFO:teuthology.orchestra.run.smithi076.stdout: "unittest_1_bytes": 0, 2024-09-11T23:40:05.643 INFO:teuthology.orchestra.run.smithi076.stdout: "unittest_1_items": 0, 2024-09-11T23:40:05.643 INFO:teuthology.orchestra.run.smithi076.stdout: "unittest_2_bytes": 0, 2024-09-11T23:40:05.643 INFO:teuthology.orchestra.run.smithi076.stdout: "unittest_2_items": 0 2024-09-11T23:40:05.643 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:05.643 INFO:teuthology.orchestra.run.smithi076.stdout: "objecter": { 2024-09-11T23:40:05.643 INFO:teuthology.orchestra.run.smithi076.stdout: "op_active": 0, 2024-09-11T23:40:05.643 INFO:teuthology.orchestra.run.smithi076.stdout: "op_laggy": 0, 2024-09-11T23:40:05.643 INFO:teuthology.orchestra.run.smithi076.stdout: "op_send": 21904, 2024-09-11T23:40:05.643 INFO:teuthology.orchestra.run.smithi076.stdout: "op_send_bytes": 361670240, 2024-09-11T23:40:05.644 INFO:teuthology.orchestra.run.smithi076.stdout: "op_resend": 33, 2024-09-11T23:40:05.644 INFO:teuthology.orchestra.run.smithi076.stdout: "op_reply": 21904, 2024-09-11T23:40:05.644 INFO:teuthology.orchestra.run.smithi076.stdout: "op_latency": { 2024-09-11T23:40:05.644 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 21904, 2024-09-11T23:40:05.644 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 1101.318173691, 2024-09-11T23:40:05.644 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.050279317 2024-09-11T23:40:05.644 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:05.644 INFO:teuthology.orchestra.run.smithi076.stdout: "op_inflight": 0, 2024-09-11T23:40:05.644 INFO:teuthology.orchestra.run.smithi076.stdout: "oplen_avg": { 2024-09-11T23:40:05.644 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 21904, 2024-09-11T23:40:05.644 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 31098 2024-09-11T23:40:05.645 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:05.645 INFO:teuthology.orchestra.run.smithi076.stdout: "op": 21904, 2024-09-11T23:40:05.645 INFO:teuthology.orchestra.run.smithi076.stdout: "op_r": 920, 2024-09-11T23:40:05.645 INFO:teuthology.orchestra.run.smithi076.stdout: "op_w": 20984, 2024-09-11T23:40:05.645 INFO:teuthology.orchestra.run.smithi076.stdout: "op_rmw": 0, 2024-09-11T23:40:05.645 INFO:teuthology.orchestra.run.smithi076.stdout: "op_pg": 0, 2024-09-11T23:40:05.645 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_stat": 141, 2024-09-11T23:40:05.645 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_create": 3482, 2024-09-11T23:40:05.645 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_read": 283, 2024-09-11T23:40:05.645 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_write": 6038, 2024-09-11T23:40:05.645 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_writefull": 197, 2024-09-11T23:40:05.646 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_writesame": 0, 2024-09-11T23:40:05.646 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_append": 0, 2024-09-11T23:40:05.646 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_zero": 0, 2024-09-11T23:40:05.646 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_truncate": 107, 2024-09-11T23:40:05.646 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_delete": 9639, 2024-09-11T23:40:05.646 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_mapext": 0, 2024-09-11T23:40:05.646 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_sparse_read": 0, 2024-09-11T23:40:05.646 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_clonerange": 0, 2024-09-11T23:40:05.646 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_getxattr": 477, 2024-09-11T23:40:05.647 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_setxattr": 6964, 2024-09-11T23:40:05.647 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_cmpxattr": 0, 2024-09-11T23:40:05.647 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_rmxattr": 0, 2024-09-11T23:40:05.647 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_resetxattrs": 0, 2024-09-11T23:40:05.647 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_call": 0, 2024-09-11T23:40:05.647 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_watch": 0, 2024-09-11T23:40:05.647 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_notify": 0, 2024-09-11T23:40:05.647 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_src_cmpxattr": 0, 2024-09-11T23:40:05.647 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_pgls": 0, 2024-09-11T23:40:05.647 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_pgls_filter": 0, 2024-09-11T23:40:05.647 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_other": 0, 2024-09-11T23:40:05.647 INFO:teuthology.orchestra.run.smithi076.stdout: "linger_active": 0, 2024-09-11T23:40:05.647 INFO:teuthology.orchestra.run.smithi076.stdout: "linger_send": 0, 2024-09-11T23:40:05.648 INFO:teuthology.orchestra.run.smithi076.stdout: "linger_resend": 0, 2024-09-11T23:40:05.648 INFO:teuthology.orchestra.run.smithi076.stdout: "linger_ping": 0, 2024-09-11T23:40:05.648 INFO:teuthology.orchestra.run.smithi076.stdout: "poolop_active": 0, 2024-09-11T23:40:05.648 INFO:teuthology.orchestra.run.smithi076.stdout: "poolop_send": 0, 2024-09-11T23:40:05.648 INFO:teuthology.orchestra.run.smithi076.stdout: "poolop_resend": 0, 2024-09-11T23:40:05.648 INFO:teuthology.orchestra.run.smithi076.stdout: "poolstat_active": 0, 2024-09-11T23:40:05.648 INFO:teuthology.orchestra.run.smithi076.stdout: "poolstat_send": 0, 2024-09-11T23:40:05.648 INFO:teuthology.orchestra.run.smithi076.stdout: "poolstat_resend": 0, 2024-09-11T23:40:05.648 INFO:teuthology.orchestra.run.smithi076.stdout: "statfs_active": 0, 2024-09-11T23:40:05.648 INFO:teuthology.orchestra.run.smithi076.stdout: "statfs_send": 0, 2024-09-11T23:40:05.648 INFO:teuthology.orchestra.run.smithi076.stdout: "statfs_resend": 0, 2024-09-11T23:40:05.649 INFO:teuthology.orchestra.run.smithi076.stdout: "command_active": 0, 2024-09-11T23:40:05.649 INFO:teuthology.orchestra.run.smithi076.stdout: "command_send": 0, 2024-09-11T23:40:05.649 INFO:teuthology.orchestra.run.smithi076.stdout: "command_resend": 0, 2024-09-11T23:40:05.649 INFO:teuthology.orchestra.run.smithi076.stdout: "map_epoch": 116, 2024-09-11T23:40:05.649 INFO:teuthology.orchestra.run.smithi076.stdout: "map_full": 0, 2024-09-11T23:40:05.649 INFO:teuthology.orchestra.run.smithi076.stdout: "map_inc": 32, 2024-09-11T23:40:05.649 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_sessions": 12, 2024-09-11T23:40:05.649 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_session_open": 12, 2024-09-11T23:40:05.649 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_session_close": 0, 2024-09-11T23:40:05.649 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_laggy": 0, 2024-09-11T23:40:05.649 INFO:teuthology.orchestra.run.smithi076.stdout: "omap_wr": 2971, 2024-09-11T23:40:05.649 INFO:teuthology.orchestra.run.smithi076.stdout: "omap_rd": 322, 2024-09-11T23:40:05.650 INFO:teuthology.orchestra.run.smithi076.stdout: "omap_del": 477 2024-09-11T23:40:05.650 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:05.650 INFO:teuthology.orchestra.run.smithi076.stdout: "oft": { 2024-09-11T23:40:05.650 INFO:teuthology.orchestra.run.smithi076.stdout: "omap_total_objs": 1, 2024-09-11T23:40:05.650 INFO:teuthology.orchestra.run.smithi076.stdout: "omap_total_kv_pairs": 0, 2024-09-11T23:40:05.650 INFO:teuthology.orchestra.run.smithi076.stdout: "omap_total_updates": 0, 2024-09-11T23:40:05.650 INFO:teuthology.orchestra.run.smithi076.stdout: "omap_total_removes": 0 2024-09-11T23:40:05.650 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:05.650 INFO:teuthology.orchestra.run.smithi076.stdout: "osd-slow-ops": { 2024-09-11T23:40:05.650 INFO:teuthology.orchestra.run.smithi076.stdout: "slow_ops_count": 162592 2024-09-11T23:40:05.650 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:05.651 INFO:teuthology.orchestra.run.smithi076.stdout: "purge_queue": { 2024-09-11T23:40:05.651 INFO:teuthology.orchestra.run.smithi076.stdout: "pq_executing_ops": 0, 2024-09-11T23:40:05.651 INFO:teuthology.orchestra.run.smithi076.stdout: "pq_executing_ops_high_water": 6, 2024-09-11T23:40:05.651 INFO:teuthology.orchestra.run.smithi076.stdout: "pq_executing": 0, 2024-09-11T23:40:05.651 INFO:teuthology.orchestra.run.smithi076.stdout: "pq_executing_high_water": 6, 2024-09-11T23:40:05.651 INFO:teuthology.orchestra.run.smithi076.stdout: "pq_executed_ops": 9538, 2024-09-11T23:40:05.651 INFO:teuthology.orchestra.run.smithi076.stdout: "pq_executed": 9538, 2024-09-11T23:40:05.651 INFO:teuthology.orchestra.run.smithi076.stdout: "pq_item_in_journal": 0 2024-09-11T23:40:05.651 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:05.651 INFO:teuthology.orchestra.run.smithi076.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-11T23:40:05.651 INFO:teuthology.orchestra.run.smithi076.stdout: "val": 0, 2024-09-11T23:40:05.652 INFO:teuthology.orchestra.run.smithi076.stdout: "max": 104857600, 2024-09-11T23:40:05.652 INFO:teuthology.orchestra.run.smithi076.stdout: "get_started": 0, 2024-09-11T23:40:05.652 INFO:teuthology.orchestra.run.smithi076.stdout: "get": 375637, 2024-09-11T23:40:05.652 INFO:teuthology.orchestra.run.smithi076.stdout: "get_sum": 132100679, 2024-09-11T23:40:05.652 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_fail": 0, 2024-09-11T23:40:05.652 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_success": 375637, 2024-09-11T23:40:05.652 INFO:teuthology.orchestra.run.smithi076.stdout: "take": 0, 2024-09-11T23:40:05.652 INFO:teuthology.orchestra.run.smithi076.stdout: "take_sum": 0, 2024-09-11T23:40:05.652 INFO:teuthology.orchestra.run.smithi076.stdout: "put": 375637, 2024-09-11T23:40:05.652 INFO:teuthology.orchestra.run.smithi076.stdout: "put_sum": 132100679, 2024-09-11T23:40:05.653 INFO:teuthology.orchestra.run.smithi076.stdout: "wait": { 2024-09-11T23:40:05.653 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-09-11T23:40:05.653 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-09-11T23:40:05.653 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-09-11T23:40:05.653 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-11T23:40:05.653 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:05.653 INFO:teuthology.orchestra.run.smithi076.stdout: "throttle-objecter_bytes": { 2024-09-11T23:40:05.653 INFO:teuthology.orchestra.run.smithi076.stdout: "val": 0, 2024-09-11T23:40:05.653 INFO:teuthology.orchestra.run.smithi076.stdout: "max": 104857600, 2024-09-11T23:40:05.653 INFO:teuthology.orchestra.run.smithi076.stdout: "get_started": 0, 2024-09-11T23:40:05.653 INFO:teuthology.orchestra.run.smithi076.stdout: "get": 0, 2024-09-11T23:40:05.654 INFO:teuthology.orchestra.run.smithi076.stdout: "get_sum": 0, 2024-09-11T23:40:05.654 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_fail": 0, 2024-09-11T23:40:05.654 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_success": 0, 2024-09-11T23:40:05.654 INFO:teuthology.orchestra.run.smithi076.stdout: "take": 21904, 2024-09-11T23:40:05.654 INFO:teuthology.orchestra.run.smithi076.stdout: "take_sum": 363017306, 2024-09-11T23:40:05.654 INFO:teuthology.orchestra.run.smithi076.stdout: "put": 21904, 2024-09-11T23:40:05.654 INFO:teuthology.orchestra.run.smithi076.stdout: "put_sum": 363017306, 2024-09-11T23:40:05.654 INFO:teuthology.orchestra.run.smithi076.stdout: "wait": { 2024-09-11T23:40:05.654 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-09-11T23:40:05.654 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-09-11T23:40:05.654 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-09-11T23:40:05.654 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-11T23:40:05.655 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:05.655 INFO:teuthology.orchestra.run.smithi076.stdout: "throttle-objecter_ops": { 2024-09-11T23:40:05.655 INFO:teuthology.orchestra.run.smithi076.stdout: "val": 0, 2024-09-11T23:40:05.655 INFO:teuthology.orchestra.run.smithi076.stdout: "max": 1024, 2024-09-11T23:40:05.655 INFO:teuthology.orchestra.run.smithi076.stdout: "get_started": 0, 2024-09-11T23:40:05.655 INFO:teuthology.orchestra.run.smithi076.stdout: "get": 0, 2024-09-11T23:40:05.655 INFO:teuthology.orchestra.run.smithi076.stdout: "get_sum": 0, 2024-09-11T23:40:05.655 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_fail": 0, 2024-09-11T23:40:05.655 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_success": 0, 2024-09-11T23:40:05.655 INFO:teuthology.orchestra.run.smithi076.stdout: "take": 21904, 2024-09-11T23:40:05.655 INFO:teuthology.orchestra.run.smithi076.stdout: "take_sum": 21904, 2024-09-11T23:40:05.656 INFO:teuthology.orchestra.run.smithi076.stdout: "put": 21904, 2024-09-11T23:40:05.656 INFO:teuthology.orchestra.run.smithi076.stdout: "put_sum": 21904, 2024-09-11T23:40:05.656 INFO:teuthology.orchestra.run.smithi076.stdout: "wait": { 2024-09-11T23:40:05.656 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-09-11T23:40:05.656 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-09-11T23:40:05.656 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-09-11T23:40:05.656 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-11T23:40:05.656 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:05.656 INFO:teuthology.orchestra.run.smithi076.stdout: "throttle-write_buf_throttle": { 2024-09-11T23:40:05.656 INFO:teuthology.orchestra.run.smithi076.stdout: "val": 0, 2024-09-11T23:40:05.656 INFO:teuthology.orchestra.run.smithi076.stdout: "max": 3758096384, 2024-09-11T23:40:05.656 INFO:teuthology.orchestra.run.smithi076.stdout: "get_started": 0, 2024-09-11T23:40:05.657 INFO:teuthology.orchestra.run.smithi076.stdout: "get": 9538, 2024-09-11T23:40:05.657 INFO:teuthology.orchestra.run.smithi076.stdout: "get_sum": 1349642, 2024-09-11T23:40:05.657 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_fail": 0, 2024-09-11T23:40:05.657 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_success": 9538, 2024-09-11T23:40:05.657 INFO:teuthology.orchestra.run.smithi076.stdout: "take": 0, 2024-09-11T23:40:05.657 INFO:teuthology.orchestra.run.smithi076.stdout: "take_sum": 0, 2024-09-11T23:40:05.657 INFO:teuthology.orchestra.run.smithi076.stdout: "put": 283, 2024-09-11T23:40:05.657 INFO:teuthology.orchestra.run.smithi076.stdout: "put_sum": 1349642, 2024-09-11T23:40:05.657 INFO:teuthology.orchestra.run.smithi076.stdout: "wait": { 2024-09-11T23:40:05.657 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-09-11T23:40:05.657 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-09-11T23:40:05.658 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-09-11T23:40:05.658 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-11T23:40:05.658 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:05.658 INFO:teuthology.orchestra.run.smithi076.stdout: "throttle-write_buf_throttle-0x559a2e6fe960": { 2024-09-11T23:40:05.658 INFO:teuthology.orchestra.run.smithi076.stdout: "val": 0, 2024-09-11T23:40:05.658 INFO:teuthology.orchestra.run.smithi076.stdout: "max": 3758096384, 2024-09-11T23:40:05.658 INFO:teuthology.orchestra.run.smithi076.stdout: "get_started": 0, 2024-09-11T23:40:05.658 INFO:teuthology.orchestra.run.smithi076.stdout: "get": 104390, 2024-09-11T23:40:05.658 INFO:teuthology.orchestra.run.smithi076.stdout: "get_sum": 334207566, 2024-09-11T23:40:05.658 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_fail": 0, 2024-09-11T23:40:05.658 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_success": 104390, 2024-09-11T23:40:05.658 INFO:teuthology.orchestra.run.smithi076.stdout: "take": 0, 2024-09-11T23:40:05.658 INFO:teuthology.orchestra.run.smithi076.stdout: "take_sum": 0, 2024-09-11T23:40:05.659 INFO:teuthology.orchestra.run.smithi076.stdout: "put": 5754, 2024-09-11T23:40:05.659 INFO:teuthology.orchestra.run.smithi076.stdout: "put_sum": 334207566, 2024-09-11T23:40:05.659 INFO:teuthology.orchestra.run.smithi076.stdout: "wait": { 2024-09-11T23:40:05.659 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-09-11T23:40:05.659 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-09-11T23:40:05.659 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-09-11T23:40:05.659 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-11T23:40:05.659 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-11T23:40:05.659 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-11T23:40:06.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:40:05 smithi136 ceph-mon[35121]: pgmap v684: 97 pgs: 97 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 64 KiB/s wr, 35 op/s 2024-09-11T23:40:06.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:40:05 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:40:06.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:40:05 smithi026 ceph-mon[31172]: pgmap v684: 97 pgs: 97 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 64 KiB/s wr, 35 op/s 2024-09-11T23:40:06.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:40:05 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:40:06.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:40:05 smithi076 ceph-mon[35296]: pgmap v684: 97 pgs: 97 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 64 KiB/s wr, 35 op/s 2024-09-11T23:40:06.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:40:05 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:40:06.472 INFO:tasks.check_counter:Daemon mds.h mds.exported=353 2024-09-11T23:40:06.473 INFO:tasks.check_counter:Daemon mds.h mds.imported=266 2024-09-11T23:40:06.473 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update=3293 2024-09-11T23:40:06.473 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update_receipt=14288 2024-09-11T23:40:06.473 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=2 2024-09-11T23:40:06.473 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2024-09-11T23:40:06.473 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2024-09-11T23:40:06.473 INFO:tasks.check_counter:Daemon mds.h mds.dir_split=110 2024-09-11T23:40:06.473 DEBUG:tasks.check_counter:Getting stats from k 2024-09-11T23:40:06.473 DEBUG:teuthology.orchestra.run.smithi076:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph daemon mds.k perf dump 2024-09-11T23:40:06.730 INFO:teuthology.orchestra.run.smithi076.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T23:40:07.258 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-11T23:40:07.258 INFO:teuthology.orchestra.run.smithi076.stdout: "AsyncMessenger::Worker-0": { 2024-09-11T23:40:07.258 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_messages": 2590, 2024-09-11T23:40:07.258 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_messages": 2590, 2024-09-11T23:40:07.258 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_bytes": 307155873, 2024-09-11T23:40:07.259 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_bytes": 822803, 2024-09-11T23:40:07.259 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_created_connections": 8, 2024-09-11T23:40:07.259 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_active_connections": 4, 2024-09-11T23:40:07.259 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_total_time": 0.412315773, 2024-09-11T23:40:07.259 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_send_time": 0.071271743, 2024-09-11T23:40:07.259 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_recv_time": 0.252809836, 2024-09-11T23:40:07.259 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_fast_dispatch_time": 0.068829104, 2024-09-11T23:40:07.259 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T23:40:07.259 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 2589, 2024-09-11T23:40:07.259 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.110897980, 2024-09-11T23:40:07.259 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000042834 2024-09-11T23:40:07.260 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:07.260 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_handle_ack_lat": { 2024-09-11T23:40:07.260 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-09-11T23:40:07.260 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-09-11T23:40:07.260 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-09-11T23:40:07.260 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:07.260 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-09-11T23:40:07.260 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_encrypted_bytes": 448 2024-09-11T23:40:07.260 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:07.260 INFO:teuthology.orchestra.run.smithi076.stdout: "AsyncMessenger::Worker-1": { 2024-09-11T23:40:07.260 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_messages": 2332, 2024-09-11T23:40:07.261 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_messages": 2256, 2024-09-11T23:40:07.261 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_bytes": 291813375, 2024-09-11T23:40:07.261 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_bytes": 829038, 2024-09-11T23:40:07.261 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_created_connections": 6, 2024-09-11T23:40:07.261 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_active_connections": 5, 2024-09-11T23:40:07.261 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_total_time": 0.347382339, 2024-09-11T23:40:07.261 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_send_time": 0.064842667, 2024-09-11T23:40:07.261 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_recv_time": 0.241678815, 2024-09-11T23:40:07.261 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_fast_dispatch_time": 0.047323018, 2024-09-11T23:40:07.261 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T23:40:07.261 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 2255, 2024-09-11T23:40:07.261 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.088476097, 2024-09-11T23:40:07.261 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000039235 2024-09-11T23:40:07.262 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:07.262 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_handle_ack_lat": { 2024-09-11T23:40:07.262 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-09-11T23:40:07.262 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-09-11T23:40:07.262 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-09-11T23:40:07.262 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:07.262 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_encrypted_bytes": 2108560, 2024-09-11T23:40:07.262 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_encrypted_bytes": 246400 2024-09-11T23:40:07.262 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:07.262 INFO:teuthology.orchestra.run.smithi076.stdout: "AsyncMessenger::Worker-2": { 2024-09-11T23:40:07.262 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_messages": 1496, 2024-09-11T23:40:07.262 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_messages": 1790, 2024-09-11T23:40:07.262 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_bytes": 340202782, 2024-09-11T23:40:07.262 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_bytes": 1243478, 2024-09-11T23:40:07.263 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_created_connections": 9, 2024-09-11T23:40:07.263 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_active_connections": 6, 2024-09-11T23:40:07.263 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_total_time": 0.305362730, 2024-09-11T23:40:07.263 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_send_time": 0.039984456, 2024-09-11T23:40:07.263 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_recv_time": 0.216297002, 2024-09-11T23:40:07.263 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_fast_dispatch_time": 0.036691459, 2024-09-11T23:40:07.263 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T23:40:07.263 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 1787, 2024-09-11T23:40:07.263 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.078101381, 2024-09-11T23:40:07.263 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000043705 2024-09-11T23:40:07.263 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:07.263 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_handle_ack_lat": { 2024-09-11T23:40:07.263 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-09-11T23:40:07.264 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-09-11T23:40:07.264 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-09-11T23:40:07.264 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:07.264 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_encrypted_bytes": 480, 2024-09-11T23:40:07.264 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_encrypted_bytes": 768704 2024-09-11T23:40:07.264 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:07.264 INFO:teuthology.orchestra.run.smithi076.stdout: "cct": { 2024-09-11T23:40:07.264 INFO:teuthology.orchestra.run.smithi076.stdout: "total_workers": 1, 2024-09-11T23:40:07.264 INFO:teuthology.orchestra.run.smithi076.stdout: "unhealthy_workers": 0 2024-09-11T23:40:07.264 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:07.264 INFO:teuthology.orchestra.run.smithi076.stdout: "finisher-MDSRank": { 2024-09-11T23:40:07.264 INFO:teuthology.orchestra.run.smithi076.stdout: "queue_len": 0, 2024-09-11T23:40:07.265 INFO:teuthology.orchestra.run.smithi076.stdout: "complete_latency": { 2024-09-11T23:40:07.265 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 9155, 2024-09-11T23:40:07.265 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.246788911, 2024-09-11T23:40:07.265 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000026956 2024-09-11T23:40:07.265 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-11T23:40:07.265 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:07.265 INFO:teuthology.orchestra.run.smithi076.stdout: "finisher-PurgeQueue": { 2024-09-11T23:40:07.265 INFO:teuthology.orchestra.run.smithi076.stdout: "queue_len": 0, 2024-09-11T23:40:07.265 INFO:teuthology.orchestra.run.smithi076.stdout: "complete_latency": { 2024-09-11T23:40:07.265 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-09-11T23:40:07.265 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-09-11T23:40:07.265 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-09-11T23:40:07.265 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-11T23:40:07.266 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:07.266 INFO:teuthology.orchestra.run.smithi076.stdout: "mds": { 2024-09-11T23:40:07.266 INFO:teuthology.orchestra.run.smithi076.stdout: "request": 0, 2024-09-11T23:40:07.266 INFO:teuthology.orchestra.run.smithi076.stdout: "reply": 0, 2024-09-11T23:40:07.266 INFO:teuthology.orchestra.run.smithi076.stdout: "reply_latency": { 2024-09-11T23:40:07.266 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-09-11T23:40:07.266 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-09-11T23:40:07.266 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-09-11T23:40:07.266 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:07.266 INFO:teuthology.orchestra.run.smithi076.stdout: "slow_reply": 0, 2024-09-11T23:40:07.266 INFO:teuthology.orchestra.run.smithi076.stdout: "forward": 0, 2024-09-11T23:40:07.267 INFO:teuthology.orchestra.run.smithi076.stdout: "dir_fetch_complete": 0, 2024-09-11T23:40:07.267 INFO:teuthology.orchestra.run.smithi076.stdout: "dir_fetch_keys": 0, 2024-09-11T23:40:07.267 INFO:teuthology.orchestra.run.smithi076.stdout: "dir_commit": 0, 2024-09-11T23:40:07.267 INFO:teuthology.orchestra.run.smithi076.stdout: "dir_split": 0, 2024-09-11T23:40:07.267 INFO:teuthology.orchestra.run.smithi076.stdout: "dir_merge": 0, 2024-09-11T23:40:07.267 INFO:teuthology.orchestra.run.smithi076.stdout: "inodes": 187419, 2024-09-11T23:40:07.268 INFO:teuthology.orchestra.run.smithi076.stdout: "inodes_top": 29183, 2024-09-11T23:40:07.268 INFO:teuthology.orchestra.run.smithi076.stdout: "inodes_bottom": 158236, 2024-09-11T23:40:07.268 INFO:teuthology.orchestra.run.smithi076.stdout: "inodes_pin_tail": 0, 2024-09-11T23:40:07.268 INFO:teuthology.orchestra.run.smithi076.stdout: "inodes_pinned": 145729, 2024-09-11T23:40:07.268 INFO:teuthology.orchestra.run.smithi076.stdout: "inodes_expired": 0, 2024-09-11T23:40:07.268 INFO:teuthology.orchestra.run.smithi076.stdout: "inodes_with_caps": 0, 2024-09-11T23:40:07.268 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": 0, 2024-09-11T23:40:07.268 INFO:teuthology.orchestra.run.smithi076.stdout: "subtrees": 3, 2024-09-11T23:40:07.268 INFO:teuthology.orchestra.run.smithi076.stdout: "traverse": 0, 2024-09-11T23:40:07.268 INFO:teuthology.orchestra.run.smithi076.stdout: "traverse_hit": 0, 2024-09-11T23:40:07.268 INFO:teuthology.orchestra.run.smithi076.stdout: "traverse_forward": 0, 2024-09-11T23:40:07.268 INFO:teuthology.orchestra.run.smithi076.stdout: "traverse_discover": 0, 2024-09-11T23:40:07.268 INFO:teuthology.orchestra.run.smithi076.stdout: "traverse_dir_fetch": 0, 2024-09-11T23:40:07.269 INFO:teuthology.orchestra.run.smithi076.stdout: "traverse_remote_ino": 0, 2024-09-11T23:40:07.269 INFO:teuthology.orchestra.run.smithi076.stdout: "traverse_lock": 0, 2024-09-11T23:40:07.269 INFO:teuthology.orchestra.run.smithi076.stdout: "load_cent": 0, 2024-09-11T23:40:07.269 INFO:teuthology.orchestra.run.smithi076.stdout: "q": 0, 2024-09-11T23:40:07.269 INFO:teuthology.orchestra.run.smithi076.stdout: "exported": 0, 2024-09-11T23:40:07.269 INFO:teuthology.orchestra.run.smithi076.stdout: "exported_inodes": 0, 2024-09-11T23:40:07.269 INFO:teuthology.orchestra.run.smithi076.stdout: "imported": 0, 2024-09-11T23:40:07.269 INFO:teuthology.orchestra.run.smithi076.stdout: "imported_inodes": 0, 2024-09-11T23:40:07.269 INFO:teuthology.orchestra.run.smithi076.stdout: "openino_dir_fetch": 0, 2024-09-11T23:40:07.269 INFO:teuthology.orchestra.run.smithi076.stdout: "openino_backtrace_fetch": 0, 2024-09-11T23:40:07.269 INFO:teuthology.orchestra.run.smithi076.stdout: "openino_peer_discover": 0, 2024-09-11T23:40:07.269 INFO:teuthology.orchestra.run.smithi076.stdout: "root_rfiles": 0, 2024-09-11T23:40:07.269 INFO:teuthology.orchestra.run.smithi076.stdout: "root_rbytes": 0, 2024-09-11T23:40:07.270 INFO:teuthology.orchestra.run.smithi076.stdout: "root_rsnaps": 0, 2024-09-11T23:40:07.270 INFO:teuthology.orchestra.run.smithi076.stdout: "scrub_backtrace_fetch": 0, 2024-09-11T23:40:07.270 INFO:teuthology.orchestra.run.smithi076.stdout: "scrub_set_tag": 0, 2024-09-11T23:40:07.270 INFO:teuthology.orchestra.run.smithi076.stdout: "scrub_backtrace_repaired": 0, 2024-09-11T23:40:07.270 INFO:teuthology.orchestra.run.smithi076.stdout: "scrub_inotable_repaired": 0, 2024-09-11T23:40:07.270 INFO:teuthology.orchestra.run.smithi076.stdout: "scrub_dir_inodes": 0, 2024-09-11T23:40:07.270 INFO:teuthology.orchestra.run.smithi076.stdout: "scrub_dir_base_inodes": 0, 2024-09-11T23:40:07.270 INFO:teuthology.orchestra.run.smithi076.stdout: "scrub_dirfrag_rstats": 0, 2024-09-11T23:40:07.270 INFO:teuthology.orchestra.run.smithi076.stdout: "scrub_file_inodes": 0, 2024-09-11T23:40:07.270 INFO:teuthology.orchestra.run.smithi076.stdout: "handle_inode_file_caps": 0, 2024-09-11T23:40:07.270 INFO:teuthology.orchestra.run.smithi076.stdout: "ceph_cap_op_revoke": 0, 2024-09-11T23:40:07.270 INFO:teuthology.orchestra.run.smithi076.stdout: "ceph_cap_op_grant": 0, 2024-09-11T23:40:07.270 INFO:teuthology.orchestra.run.smithi076.stdout: "ceph_cap_op_trunc": 0, 2024-09-11T23:40:07.271 INFO:teuthology.orchestra.run.smithi076.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-11T23:40:07.271 INFO:teuthology.orchestra.run.smithi076.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-11T23:40:07.271 INFO:teuthology.orchestra.run.smithi076.stdout: "handle_client_caps": 0, 2024-09-11T23:40:07.271 INFO:teuthology.orchestra.run.smithi076.stdout: "handle_client_caps_dirty": 0, 2024-09-11T23:40:07.271 INFO:teuthology.orchestra.run.smithi076.stdout: "handle_client_cap_release": 0, 2024-09-11T23:40:07.271 INFO:teuthology.orchestra.run.smithi076.stdout: "process_request_cap_release": 0 2024-09-11T23:40:07.271 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:07.271 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_cache": { 2024-09-11T23:40:07.271 INFO:teuthology.orchestra.run.smithi076.stdout: "dir_update": 0, 2024-09-11T23:40:07.271 INFO:teuthology.orchestra.run.smithi076.stdout: "dir_update_receipt": 0, 2024-09-11T23:40:07.271 INFO:teuthology.orchestra.run.smithi076.stdout: "dir_try_discover": 0, 2024-09-11T23:40:07.271 INFO:teuthology.orchestra.run.smithi076.stdout: "dir_send_discover": 0, 2024-09-11T23:40:07.271 INFO:teuthology.orchestra.run.smithi076.stdout: "dir_handle_discover": 0, 2024-09-11T23:40:07.271 INFO:teuthology.orchestra.run.smithi076.stdout: "num_strays": 0, 2024-09-11T23:40:07.272 INFO:teuthology.orchestra.run.smithi076.stdout: "num_strays_delayed": 0, 2024-09-11T23:40:07.272 INFO:teuthology.orchestra.run.smithi076.stdout: "num_strays_enqueuing": 0, 2024-09-11T23:40:07.272 INFO:teuthology.orchestra.run.smithi076.stdout: "strays_created": 0, 2024-09-11T23:40:07.272 INFO:teuthology.orchestra.run.smithi076.stdout: "strays_enqueued": 0, 2024-09-11T23:40:07.272 INFO:teuthology.orchestra.run.smithi076.stdout: "strays_reintegrated": 0, 2024-09-11T23:40:07.272 INFO:teuthology.orchestra.run.smithi076.stdout: "strays_migrated": 0, 2024-09-11T23:40:07.272 INFO:teuthology.orchestra.run.smithi076.stdout: "num_recovering_processing": 0, 2024-09-11T23:40:07.272 INFO:teuthology.orchestra.run.smithi076.stdout: "num_recovering_enqueued": 0, 2024-09-11T23:40:07.272 INFO:teuthology.orchestra.run.smithi076.stdout: "num_recovering_prioritized": 0, 2024-09-11T23:40:07.272 INFO:teuthology.orchestra.run.smithi076.stdout: "recovery_started": 0, 2024-09-11T23:40:07.272 INFO:teuthology.orchestra.run.smithi076.stdout: "recovery_completed": 0, 2024-09-11T23:40:07.273 INFO:teuthology.orchestra.run.smithi076.stdout: "ireq_quiesce_path": 0, 2024-09-11T23:40:07.273 INFO:teuthology.orchestra.run.smithi076.stdout: "ireq_quiesce_inode": 0, 2024-09-11T23:40:07.273 INFO:teuthology.orchestra.run.smithi076.stdout: "ireq_enqueue_scrub": 0, 2024-09-11T23:40:07.273 INFO:teuthology.orchestra.run.smithi076.stdout: "ireq_exportdir": 0, 2024-09-11T23:40:07.273 INFO:teuthology.orchestra.run.smithi076.stdout: "ireq_flush": 0, 2024-09-11T23:40:07.273 INFO:teuthology.orchestra.run.smithi076.stdout: "ireq_fragmentdir": 0, 2024-09-11T23:40:07.273 INFO:teuthology.orchestra.run.smithi076.stdout: "ireq_fragstats": 0, 2024-09-11T23:40:07.273 INFO:teuthology.orchestra.run.smithi076.stdout: "ireq_inodestats": 0 2024-09-11T23:40:07.273 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:07.273 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_log": { 2024-09-11T23:40:07.273 INFO:teuthology.orchestra.run.smithi076.stdout: "evlrg": 0, 2024-09-11T23:40:07.273 INFO:teuthology.orchestra.run.smithi076.stdout: "evadd": 0, 2024-09-11T23:40:07.274 INFO:teuthology.orchestra.run.smithi076.stdout: "evex": 0, 2024-09-11T23:40:07.274 INFO:teuthology.orchestra.run.smithi076.stdout: "evtrm": 0, 2024-09-11T23:40:07.274 INFO:teuthology.orchestra.run.smithi076.stdout: "ev": 228971, 2024-09-11T23:40:07.274 INFO:teuthology.orchestra.run.smithi076.stdout: "evexg": 0, 2024-09-11T23:40:07.274 INFO:teuthology.orchestra.run.smithi076.stdout: "evexd": 0, 2024-09-11T23:40:07.274 INFO:teuthology.orchestra.run.smithi076.stdout: "segadd": 0, 2024-09-11T23:40:07.274 INFO:teuthology.orchestra.run.smithi076.stdout: "segex": 0, 2024-09-11T23:40:07.274 INFO:teuthology.orchestra.run.smithi076.stdout: "segtrm": 0, 2024-09-11T23:40:07.274 INFO:teuthology.orchestra.run.smithi076.stdout: "seg": 145, 2024-09-11T23:40:07.274 INFO:teuthology.orchestra.run.smithi076.stdout: "segmjr": 20, 2024-09-11T23:40:07.274 INFO:teuthology.orchestra.run.smithi076.stdout: "segexg": 0, 2024-09-11T23:40:07.275 INFO:teuthology.orchestra.run.smithi076.stdout: "segexd": 0, 2024-09-11T23:40:07.275 INFO:teuthology.orchestra.run.smithi076.stdout: "expos": 369896542, 2024-09-11T23:40:07.275 INFO:teuthology.orchestra.run.smithi076.stdout: "wrpos": 939298292, 2024-09-11T23:40:07.275 INFO:teuthology.orchestra.run.smithi076.stdout: "rdpos": 939281697, 2024-09-11T23:40:07.275 INFO:teuthology.orchestra.run.smithi076.stdout: "jlat": { 2024-09-11T23:40:07.275 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-09-11T23:40:07.275 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-09-11T23:40:07.275 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-09-11T23:40:07.275 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:07.275 INFO:teuthology.orchestra.run.smithi076.stdout: "replayed": 228971 2024-09-11T23:40:07.276 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:07.276 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_mem": { 2024-09-11T23:40:07.276 INFO:teuthology.orchestra.run.smithi076.stdout: "ino": 67896, 2024-09-11T23:40:07.276 INFO:teuthology.orchestra.run.smithi076.stdout: "ino+": 171713, 2024-09-11T23:40:07.276 INFO:teuthology.orchestra.run.smithi076.stdout: "ino-": 103817, 2024-09-11T23:40:07.276 INFO:teuthology.orchestra.run.smithi076.stdout: "dir": 2209, 2024-09-11T23:40:07.276 INFO:teuthology.orchestra.run.smithi076.stdout: "dir+": 4411, 2024-09-11T23:40:07.276 INFO:teuthology.orchestra.run.smithi076.stdout: "dir-": 2202, 2024-09-11T23:40:07.276 INFO:teuthology.orchestra.run.smithi076.stdout: "dn": 187419, 2024-09-11T23:40:07.276 INFO:teuthology.orchestra.run.smithi076.stdout: "dn+": 290969, 2024-09-11T23:40:07.276 INFO:teuthology.orchestra.run.smithi076.stdout: "dn-": 103550, 2024-09-11T23:40:07.276 INFO:teuthology.orchestra.run.smithi076.stdout: "cap": 0, 2024-09-11T23:40:07.277 INFO:teuthology.orchestra.run.smithi076.stdout: "cap+": 0, 2024-09-11T23:40:07.277 INFO:teuthology.orchestra.run.smithi076.stdout: "cap-": 0, 2024-09-11T23:40:07.277 INFO:teuthology.orchestra.run.smithi076.stdout: "rss": 395704, 2024-09-11T23:40:07.277 INFO:teuthology.orchestra.run.smithi076.stdout: "heap": 215328 2024-09-11T23:40:07.277 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:07.277 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_server": { 2024-09-11T23:40:07.277 INFO:teuthology.orchestra.run.smithi076.stdout: "dispatch_client_request": 0, 2024-09-11T23:40:07.277 INFO:teuthology.orchestra.run.smithi076.stdout: "dispatch_server_request": 0, 2024-09-11T23:40:07.277 INFO:teuthology.orchestra.run.smithi076.stdout: "handle_client_request": 0, 2024-09-11T23:40:07.277 INFO:teuthology.orchestra.run.smithi076.stdout: "handle_client_session": 0, 2024-09-11T23:40:07.277 INFO:teuthology.orchestra.run.smithi076.stdout: "handle_peer_request": 0, 2024-09-11T23:40:07.278 INFO:teuthology.orchestra.run.smithi076.stdout: "req_create_latency": { 2024-09-11T23:40:07.278 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-09-11T23:40:07.278 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-09-11T23:40:07.278 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-09-11T23:40:07.278 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:07.278 INFO:teuthology.orchestra.run.smithi076.stdout: "req_getattr_latency": { 2024-09-11T23:40:07.278 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-09-11T23:40:07.278 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-09-11T23:40:07.278 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-09-11T23:40:07.278 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:07.278 INFO:teuthology.orchestra.run.smithi076.stdout: "req_getfilelock_latency": { 2024-09-11T23:40:07.278 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-09-11T23:40:07.278 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-09-11T23:40:07.279 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-09-11T23:40:07.279 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:07.279 INFO:teuthology.orchestra.run.smithi076.stdout: "req_link_latency": { 2024-09-11T23:40:07.279 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-09-11T23:40:07.279 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-09-11T23:40:07.279 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-09-11T23:40:07.279 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:07.279 INFO:teuthology.orchestra.run.smithi076.stdout: "req_lookup_latency": { 2024-09-11T23:40:07.279 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-09-11T23:40:07.279 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-09-11T23:40:07.279 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-09-11T23:40:07.279 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:07.279 INFO:teuthology.orchestra.run.smithi076.stdout: "req_lookuphash_latency": { 2024-09-11T23:40:07.279 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-09-11T23:40:07.280 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-09-11T23:40:07.280 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-09-11T23:40:07.280 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:07.280 INFO:teuthology.orchestra.run.smithi076.stdout: "req_lookupino_latency": { 2024-09-11T23:40:07.280 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-09-11T23:40:07.280 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-09-11T23:40:07.280 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-09-11T23:40:07.280 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:07.280 INFO:teuthology.orchestra.run.smithi076.stdout: "req_lookupname_latency": { 2024-09-11T23:40:07.280 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-09-11T23:40:07.280 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-09-11T23:40:07.280 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-09-11T23:40:07.280 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:07.280 INFO:teuthology.orchestra.run.smithi076.stdout: "req_lookupparent_latency": { 2024-09-11T23:40:07.281 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-09-11T23:40:07.281 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-09-11T23:40:07.281 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-09-11T23:40:07.281 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:07.281 INFO:teuthology.orchestra.run.smithi076.stdout: "req_lookupsnap_latency": { 2024-09-11T23:40:07.281 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-09-11T23:40:07.281 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-09-11T23:40:07.281 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-09-11T23:40:07.281 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:07.281 INFO:teuthology.orchestra.run.smithi076.stdout: "req_lssnap_latency": { 2024-09-11T23:40:07.281 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-09-11T23:40:07.281 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-09-11T23:40:07.282 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-09-11T23:40:07.282 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:07.282 INFO:teuthology.orchestra.run.smithi076.stdout: "req_mkdir_latency": { 2024-09-11T23:40:07.282 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-09-11T23:40:07.282 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-09-11T23:40:07.282 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-09-11T23:40:07.282 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:07.282 INFO:teuthology.orchestra.run.smithi076.stdout: "req_mknod_latency": { 2024-09-11T23:40:07.283 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-09-11T23:40:07.283 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-09-11T23:40:07.283 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-09-11T23:40:07.283 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:07.283 INFO:teuthology.orchestra.run.smithi076.stdout: "req_mksnap_latency": { 2024-09-11T23:40:07.283 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-09-11T23:40:07.283 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-09-11T23:40:07.283 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-09-11T23:40:07.283 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:07.283 INFO:teuthology.orchestra.run.smithi076.stdout: "req_open_latency": { 2024-09-11T23:40:07.283 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-09-11T23:40:07.283 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-09-11T23:40:07.283 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-09-11T23:40:07.283 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:07.284 INFO:teuthology.orchestra.run.smithi076.stdout: "req_readdir_latency": { 2024-09-11T23:40:07.284 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-09-11T23:40:07.284 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-09-11T23:40:07.284 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-09-11T23:40:07.284 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:07.284 INFO:teuthology.orchestra.run.smithi076.stdout: "req_rename_latency": { 2024-09-11T23:40:07.284 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-09-11T23:40:07.284 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-09-11T23:40:07.284 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-09-11T23:40:07.284 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:07.284 INFO:teuthology.orchestra.run.smithi076.stdout: "req_renamesnap_latency": { 2024-09-11T23:40:07.284 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-09-11T23:40:07.284 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-09-11T23:40:07.285 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-09-11T23:40:07.285 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:07.285 INFO:teuthology.orchestra.run.smithi076.stdout: "req_snapdiff_latency": { 2024-09-11T23:40:07.285 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-09-11T23:40:07.285 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-09-11T23:40:07.285 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-09-11T23:40:07.285 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:07.285 INFO:teuthology.orchestra.run.smithi076.stdout: "req_rmdir_latency": { 2024-09-11T23:40:07.285 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-09-11T23:40:07.285 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-09-11T23:40:07.285 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-09-11T23:40:07.285 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:07.285 INFO:teuthology.orchestra.run.smithi076.stdout: "req_rmsnap_latency": { 2024-09-11T23:40:07.286 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-09-11T23:40:07.286 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-09-11T23:40:07.286 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-09-11T23:40:07.286 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:07.286 INFO:teuthology.orchestra.run.smithi076.stdout: "req_rmxattr_latency": { 2024-09-11T23:40:07.286 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-09-11T23:40:07.286 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-09-11T23:40:07.286 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-09-11T23:40:07.286 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:07.286 INFO:teuthology.orchestra.run.smithi076.stdout: "req_setattr_latency": { 2024-09-11T23:40:07.286 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-09-11T23:40:07.286 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-09-11T23:40:07.286 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-09-11T23:40:07.287 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:07.287 INFO:teuthology.orchestra.run.smithi076.stdout: "req_setdirlayout_latency": { 2024-09-11T23:40:07.287 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-09-11T23:40:07.287 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-09-11T23:40:07.287 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-09-11T23:40:07.287 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:07.287 INFO:teuthology.orchestra.run.smithi076.stdout: "req_setfilelock_latency": { 2024-09-11T23:40:07.287 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-09-11T23:40:07.287 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-09-11T23:40:07.287 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-09-11T23:40:07.287 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:07.287 INFO:teuthology.orchestra.run.smithi076.stdout: "req_setlayout_latency": { 2024-09-11T23:40:07.287 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-09-11T23:40:07.287 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-09-11T23:40:07.288 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-09-11T23:40:07.288 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:07.288 INFO:teuthology.orchestra.run.smithi076.stdout: "req_setxattr_latency": { 2024-09-11T23:40:07.288 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-09-11T23:40:07.288 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-09-11T23:40:07.288 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-09-11T23:40:07.288 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:07.288 INFO:teuthology.orchestra.run.smithi076.stdout: "req_symlink_latency": { 2024-09-11T23:40:07.288 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-09-11T23:40:07.288 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-09-11T23:40:07.288 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-09-11T23:40:07.288 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:07.288 INFO:teuthology.orchestra.run.smithi076.stdout: "req_unlink_latency": { 2024-09-11T23:40:07.289 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-09-11T23:40:07.289 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-09-11T23:40:07.289 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-09-11T23:40:07.289 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:07.289 INFO:teuthology.orchestra.run.smithi076.stdout: "cap_revoke_eviction": 0, 2024-09-11T23:40:07.289 INFO:teuthology.orchestra.run.smithi076.stdout: "cap_acquisition_throttle": 0, 2024-09-11T23:40:07.289 INFO:teuthology.orchestra.run.smithi076.stdout: "req_getvxattr_latency": { 2024-09-11T23:40:07.289 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-09-11T23:40:07.289 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-09-11T23:40:07.289 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-09-11T23:40:07.289 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-11T23:40:07.289 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:07.289 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_sessions": { 2024-09-11T23:40:07.290 INFO:teuthology.orchestra.run.smithi076.stdout: "session_count": 2, 2024-09-11T23:40:07.290 INFO:teuthology.orchestra.run.smithi076.stdout: "session_add": 3, 2024-09-11T23:40:07.290 INFO:teuthology.orchestra.run.smithi076.stdout: "session_remove": 1, 2024-09-11T23:40:07.290 INFO:teuthology.orchestra.run.smithi076.stdout: "sessions_open": 3, 2024-09-11T23:40:07.290 INFO:teuthology.orchestra.run.smithi076.stdout: "sessions_stale": 0, 2024-09-11T23:40:07.290 INFO:teuthology.orchestra.run.smithi076.stdout: "total_load": 0, 2024-09-11T23:40:07.290 INFO:teuthology.orchestra.run.smithi076.stdout: "average_load": 0, 2024-09-11T23:40:07.290 INFO:teuthology.orchestra.run.smithi076.stdout: "avg_session_uptime": 3134, 2024-09-11T23:40:07.290 INFO:teuthology.orchestra.run.smithi076.stdout: "mdthresh_evicted": 0 2024-09-11T23:40:07.290 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:07.290 INFO:teuthology.orchestra.run.smithi076.stdout: "mempool": { 2024-09-11T23:40:07.290 INFO:teuthology.orchestra.run.smithi076.stdout: "bloom_filter_bytes": 0, 2024-09-11T23:40:07.290 INFO:teuthology.orchestra.run.smithi076.stdout: "bloom_filter_items": 0, 2024-09-11T23:40:07.290 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_alloc_bytes": 0, 2024-09-11T23:40:07.291 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_alloc_items": 0, 2024-09-11T23:40:07.291 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_data_bytes": 0, 2024-09-11T23:40:07.291 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_data_items": 0, 2024-09-11T23:40:07.291 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-11T23:40:07.291 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_onode_items": 0, 2024-09-11T23:40:07.291 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-11T23:40:07.291 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_meta_items": 0, 2024-09-11T23:40:07.291 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_other_bytes": 0, 2024-09-11T23:40:07.291 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_other_items": 0, 2024-09-11T23:40:07.291 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-11T23:40:07.291 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_buffer_items": 0, 2024-09-11T23:40:07.291 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_extent_bytes": 0, 2024-09-11T23:40:07.291 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_extent_items": 0, 2024-09-11T23:40:07.291 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_blob_bytes": 0, 2024-09-11T23:40:07.292 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_blob_items": 0, 2024-09-11T23:40:07.292 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-11T23:40:07.292 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_shared_blob_items": 0, 2024-09-11T23:40:07.292 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-11T23:40:07.292 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_inline_bl_items": 0, 2024-09-11T23:40:07.292 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_fsck_bytes": 0, 2024-09-11T23:40:07.292 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_fsck_items": 0, 2024-09-11T23:40:07.292 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_txc_bytes": 0, 2024-09-11T23:40:07.292 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_txc_items": 0, 2024-09-11T23:40:07.292 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-11T23:40:07.292 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_writing_deferred_items": 0, 2024-09-11T23:40:07.292 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_writing_bytes": 0, 2024-09-11T23:40:07.292 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_writing_items": 0, 2024-09-11T23:40:07.292 INFO:teuthology.orchestra.run.smithi076.stdout: "bluefs_bytes": 0, 2024-09-11T23:40:07.293 INFO:teuthology.orchestra.run.smithi076.stdout: "bluefs_items": 0, 2024-09-11T23:40:07.293 INFO:teuthology.orchestra.run.smithi076.stdout: "bluefs_file_reader_bytes": 0, 2024-09-11T23:40:07.293 INFO:teuthology.orchestra.run.smithi076.stdout: "bluefs_file_reader_items": 0, 2024-09-11T23:40:07.293 INFO:teuthology.orchestra.run.smithi076.stdout: "bluefs_file_writer_bytes": 0, 2024-09-11T23:40:07.293 INFO:teuthology.orchestra.run.smithi076.stdout: "bluefs_file_writer_items": 0, 2024-09-11T23:40:07.293 INFO:teuthology.orchestra.run.smithi076.stdout: "buffer_anon_bytes": 2792516, 2024-09-11T23:40:07.293 INFO:teuthology.orchestra.run.smithi076.stdout: "buffer_anon_items": 69081, 2024-09-11T23:40:07.293 INFO:teuthology.orchestra.run.smithi076.stdout: "buffer_meta_bytes": 0, 2024-09-11T23:40:07.293 INFO:teuthology.orchestra.run.smithi076.stdout: "buffer_meta_items": 0, 2024-09-11T23:40:07.293 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_bytes": 0, 2024-09-11T23:40:07.293 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_items": 0, 2024-09-11T23:40:07.293 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_mapbl_bytes": 0, 2024-09-11T23:40:07.293 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_mapbl_items": 0, 2024-09-11T23:40:07.293 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_pglog_bytes": 0, 2024-09-11T23:40:07.294 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_pglog_items": 0, 2024-09-11T23:40:07.294 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_bytes": 8224, 2024-09-11T23:40:07.294 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_items": 181, 2024-09-11T23:40:07.294 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_mapping_bytes": 0, 2024-09-11T23:40:07.294 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_mapping_items": 0, 2024-09-11T23:40:07.294 INFO:teuthology.orchestra.run.smithi076.stdout: "pgmap_bytes": 0, 2024-09-11T23:40:07.294 INFO:teuthology.orchestra.run.smithi076.stdout: "pgmap_items": 0, 2024-09-11T23:40:07.294 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_co_bytes": 291323176, 2024-09-11T23:40:07.294 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_co_items": 2770434, 2024-09-11T23:40:07.294 INFO:teuthology.orchestra.run.smithi076.stdout: "unittest_1_bytes": 0, 2024-09-11T23:40:07.294 INFO:teuthology.orchestra.run.smithi076.stdout: "unittest_1_items": 0, 2024-09-11T23:40:07.294 INFO:teuthology.orchestra.run.smithi076.stdout: "unittest_2_bytes": 0, 2024-09-11T23:40:07.294 INFO:teuthology.orchestra.run.smithi076.stdout: "unittest_2_items": 0 2024-09-11T23:40:07.294 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:07.294 INFO:teuthology.orchestra.run.smithi076.stdout: "objecter": { 2024-09-11T23:40:07.295 INFO:teuthology.orchestra.run.smithi076.stdout: "op_active": 0, 2024-09-11T23:40:07.295 INFO:teuthology.orchestra.run.smithi076.stdout: "op_laggy": 0, 2024-09-11T23:40:07.295 INFO:teuthology.orchestra.run.smithi076.stdout: "op_send": 5911, 2024-09-11T23:40:07.295 INFO:teuthology.orchestra.run.smithi076.stdout: "op_send_bytes": 21, 2024-09-11T23:40:07.295 INFO:teuthology.orchestra.run.smithi076.stdout: "op_resend": 3, 2024-09-11T23:40:07.295 INFO:teuthology.orchestra.run.smithi076.stdout: "op_reply": 5911, 2024-09-11T23:40:07.295 INFO:teuthology.orchestra.run.smithi076.stdout: "op_latency": { 2024-09-11T23:40:07.295 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 5911, 2024-09-11T23:40:07.295 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 23.058330987, 2024-09-11T23:40:07.295 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.003900918 2024-09-11T23:40:07.295 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:07.295 INFO:teuthology.orchestra.run.smithi076.stdout: "op_inflight": 0, 2024-09-11T23:40:07.295 INFO:teuthology.orchestra.run.smithi076.stdout: "oplen_avg": { 2024-09-11T23:40:07.296 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 5911, 2024-09-11T23:40:07.296 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 5912 2024-09-11T23:40:07.296 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:07.296 INFO:teuthology.orchestra.run.smithi076.stdout: "op": 5911, 2024-09-11T23:40:07.296 INFO:teuthology.orchestra.run.smithi076.stdout: "op_r": 5911, 2024-09-11T23:40:07.296 INFO:teuthology.orchestra.run.smithi076.stdout: "op_w": 0, 2024-09-11T23:40:07.296 INFO:teuthology.orchestra.run.smithi076.stdout: "op_rmw": 0, 2024-09-11T23:40:07.296 INFO:teuthology.orchestra.run.smithi076.stdout: "op_pg": 0, 2024-09-11T23:40:07.296 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_stat": 3709, 2024-09-11T23:40:07.296 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_create": 0, 2024-09-11T23:40:07.296 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_read": 2200, 2024-09-11T23:40:07.296 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_write": 0, 2024-09-11T23:40:07.297 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_writefull": 0, 2024-09-11T23:40:07.297 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_writesame": 0, 2024-09-11T23:40:07.297 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_append": 0, 2024-09-11T23:40:07.297 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_zero": 0, 2024-09-11T23:40:07.297 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_truncate": 0, 2024-09-11T23:40:07.297 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_delete": 0, 2024-09-11T23:40:07.297 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_mapext": 0, 2024-09-11T23:40:07.297 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_sparse_read": 0, 2024-09-11T23:40:07.297 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_clonerange": 0, 2024-09-11T23:40:07.297 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_getxattr": 1, 2024-09-11T23:40:07.297 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_setxattr": 0, 2024-09-11T23:40:07.297 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_cmpxattr": 0, 2024-09-11T23:40:07.298 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_rmxattr": 0, 2024-09-11T23:40:07.298 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_resetxattrs": 0, 2024-09-11T23:40:07.298 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_call": 0, 2024-09-11T23:40:07.298 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_watch": 0, 2024-09-11T23:40:07.298 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_notify": 0, 2024-09-11T23:40:07.298 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_src_cmpxattr": 0, 2024-09-11T23:40:07.298 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_pgls": 0, 2024-09-11T23:40:07.298 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_pgls_filter": 0, 2024-09-11T23:40:07.298 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_other": 0, 2024-09-11T23:40:07.298 INFO:teuthology.orchestra.run.smithi076.stdout: "linger_active": 0, 2024-09-11T23:40:07.298 INFO:teuthology.orchestra.run.smithi076.stdout: "linger_send": 0, 2024-09-11T23:40:07.298 INFO:teuthology.orchestra.run.smithi076.stdout: "linger_resend": 0, 2024-09-11T23:40:07.299 INFO:teuthology.orchestra.run.smithi076.stdout: "linger_ping": 0, 2024-09-11T23:40:07.299 INFO:teuthology.orchestra.run.smithi076.stdout: "poolop_active": 0, 2024-09-11T23:40:07.299 INFO:teuthology.orchestra.run.smithi076.stdout: "poolop_send": 0, 2024-09-11T23:40:07.299 INFO:teuthology.orchestra.run.smithi076.stdout: "poolop_resend": 0, 2024-09-11T23:40:07.299 INFO:teuthology.orchestra.run.smithi076.stdout: "poolstat_active": 0, 2024-09-11T23:40:07.299 INFO:teuthology.orchestra.run.smithi076.stdout: "poolstat_send": 0, 2024-09-11T23:40:07.299 INFO:teuthology.orchestra.run.smithi076.stdout: "poolstat_resend": 0, 2024-09-11T23:40:07.299 INFO:teuthology.orchestra.run.smithi076.stdout: "statfs_active": 0, 2024-09-11T23:40:07.299 INFO:teuthology.orchestra.run.smithi076.stdout: "statfs_send": 0, 2024-09-11T23:40:07.299 INFO:teuthology.orchestra.run.smithi076.stdout: "statfs_resend": 0, 2024-09-11T23:40:07.299 INFO:teuthology.orchestra.run.smithi076.stdout: "command_active": 0, 2024-09-11T23:40:07.299 INFO:teuthology.orchestra.run.smithi076.stdout: "command_send": 0, 2024-09-11T23:40:07.300 INFO:teuthology.orchestra.run.smithi076.stdout: "command_resend": 0, 2024-09-11T23:40:07.300 INFO:teuthology.orchestra.run.smithi076.stdout: "map_epoch": 116, 2024-09-11T23:40:07.300 INFO:teuthology.orchestra.run.smithi076.stdout: "map_full": 0, 2024-09-11T23:40:07.300 INFO:teuthology.orchestra.run.smithi076.stdout: "map_inc": 31, 2024-09-11T23:40:07.300 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_sessions": 11, 2024-09-11T23:40:07.300 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_session_open": 11, 2024-09-11T23:40:07.300 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_session_close": 0, 2024-09-11T23:40:07.300 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_laggy": 0, 2024-09-11T23:40:07.300 INFO:teuthology.orchestra.run.smithi076.stdout: "omap_wr": 0, 2024-09-11T23:40:07.300 INFO:teuthology.orchestra.run.smithi076.stdout: "omap_rd": 2, 2024-09-11T23:40:07.301 INFO:teuthology.orchestra.run.smithi076.stdout: "omap_del": 0 2024-09-11T23:40:07.301 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:07.301 INFO:teuthology.orchestra.run.smithi076.stdout: "oft": { 2024-09-11T23:40:07.301 INFO:teuthology.orchestra.run.smithi076.stdout: "omap_total_objs": 0, 2024-09-11T23:40:07.301 INFO:teuthology.orchestra.run.smithi076.stdout: "omap_total_kv_pairs": 0, 2024-09-11T23:40:07.301 INFO:teuthology.orchestra.run.smithi076.stdout: "omap_total_updates": 0, 2024-09-11T23:40:07.301 INFO:teuthology.orchestra.run.smithi076.stdout: "omap_total_removes": 0 2024-09-11T23:40:07.301 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:07.301 INFO:teuthology.orchestra.run.smithi076.stdout: "osd-slow-ops": { 2024-09-11T23:40:07.301 INFO:teuthology.orchestra.run.smithi076.stdout: "slow_ops_count": 0 2024-09-11T23:40:07.301 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:07.301 INFO:teuthology.orchestra.run.smithi076.stdout: "purge_queue": { 2024-09-11T23:40:07.302 INFO:teuthology.orchestra.run.smithi076.stdout: "pq_executing_ops": 0, 2024-09-11T23:40:07.302 INFO:teuthology.orchestra.run.smithi076.stdout: "pq_executing_ops_high_water": 0, 2024-09-11T23:40:07.302 INFO:teuthology.orchestra.run.smithi076.stdout: "pq_executing": 0, 2024-09-11T23:40:07.302 INFO:teuthology.orchestra.run.smithi076.stdout: "pq_executing_high_water": 0, 2024-09-11T23:40:07.302 INFO:teuthology.orchestra.run.smithi076.stdout: "pq_executed_ops": 0, 2024-09-11T23:40:07.302 INFO:teuthology.orchestra.run.smithi076.stdout: "pq_executed": 0, 2024-09-11T23:40:07.302 INFO:teuthology.orchestra.run.smithi076.stdout: "pq_item_in_journal": 0 2024-09-11T23:40:07.302 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:07.302 INFO:teuthology.orchestra.run.smithi076.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-11T23:40:07.302 INFO:teuthology.orchestra.run.smithi076.stdout: "val": 0, 2024-09-11T23:40:07.302 INFO:teuthology.orchestra.run.smithi076.stdout: "max": 104857600, 2024-09-11T23:40:07.302 INFO:teuthology.orchestra.run.smithi076.stdout: "get_started": 0, 2024-09-11T23:40:07.302 INFO:teuthology.orchestra.run.smithi076.stdout: "get": 6414, 2024-09-11T23:40:07.302 INFO:teuthology.orchestra.run.smithi076.stdout: "get_sum": 938566440, 2024-09-11T23:40:07.303 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_fail": 0, 2024-09-11T23:40:07.303 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_success": 6414, 2024-09-11T23:40:07.303 INFO:teuthology.orchestra.run.smithi076.stdout: "take": 0, 2024-09-11T23:40:07.303 INFO:teuthology.orchestra.run.smithi076.stdout: "take_sum": 0, 2024-09-11T23:40:07.303 INFO:teuthology.orchestra.run.smithi076.stdout: "put": 6414, 2024-09-11T23:40:07.303 INFO:teuthology.orchestra.run.smithi076.stdout: "put_sum": 938566440, 2024-09-11T23:40:07.303 INFO:teuthology.orchestra.run.smithi076.stdout: "wait": { 2024-09-11T23:40:07.303 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-09-11T23:40:07.303 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-09-11T23:40:07.303 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-09-11T23:40:07.303 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-11T23:40:07.303 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:07.303 INFO:teuthology.orchestra.run.smithi076.stdout: "throttle-objecter_bytes": { 2024-09-11T23:40:07.303 INFO:teuthology.orchestra.run.smithi076.stdout: "val": 0, 2024-09-11T23:40:07.304 INFO:teuthology.orchestra.run.smithi076.stdout: "max": 104857600, 2024-09-11T23:40:07.304 INFO:teuthology.orchestra.run.smithi076.stdout: "get_started": 0, 2024-09-11T23:40:07.304 INFO:teuthology.orchestra.run.smithi076.stdout: "get": 0, 2024-09-11T23:40:07.304 INFO:teuthology.orchestra.run.smithi076.stdout: "get_sum": 0, 2024-09-11T23:40:07.304 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_fail": 0, 2024-09-11T23:40:07.304 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_success": 0, 2024-09-11T23:40:07.304 INFO:teuthology.orchestra.run.smithi076.stdout: "take": 5911, 2024-09-11T23:40:07.304 INFO:teuthology.orchestra.run.smithi076.stdout: "take_sum": 935436756, 2024-09-11T23:40:07.304 INFO:teuthology.orchestra.run.smithi076.stdout: "put": 5911, 2024-09-11T23:40:07.304 INFO:teuthology.orchestra.run.smithi076.stdout: "put_sum": 935436756, 2024-09-11T23:40:07.304 INFO:teuthology.orchestra.run.smithi076.stdout: "wait": { 2024-09-11T23:40:07.304 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-09-11T23:40:07.304 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-09-11T23:40:07.304 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-09-11T23:40:07.305 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-11T23:40:07.305 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:07.305 INFO:teuthology.orchestra.run.smithi076.stdout: "throttle-objecter_ops": { 2024-09-11T23:40:07.305 INFO:teuthology.orchestra.run.smithi076.stdout: "val": 0, 2024-09-11T23:40:07.305 INFO:teuthology.orchestra.run.smithi076.stdout: "max": 1024, 2024-09-11T23:40:07.305 INFO:teuthology.orchestra.run.smithi076.stdout: "get_started": 0, 2024-09-11T23:40:07.305 INFO:teuthology.orchestra.run.smithi076.stdout: "get": 0, 2024-09-11T23:40:07.305 INFO:teuthology.orchestra.run.smithi076.stdout: "get_sum": 0, 2024-09-11T23:40:07.305 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_fail": 0, 2024-09-11T23:40:07.305 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_success": 0, 2024-09-11T23:40:07.305 INFO:teuthology.orchestra.run.smithi076.stdout: "take": 5911, 2024-09-11T23:40:07.305 INFO:teuthology.orchestra.run.smithi076.stdout: "take_sum": 5911, 2024-09-11T23:40:07.305 INFO:teuthology.orchestra.run.smithi076.stdout: "put": 5911, 2024-09-11T23:40:07.305 INFO:teuthology.orchestra.run.smithi076.stdout: "put_sum": 5911, 2024-09-11T23:40:07.306 INFO:teuthology.orchestra.run.smithi076.stdout: "wait": { 2024-09-11T23:40:07.306 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-09-11T23:40:07.306 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-09-11T23:40:07.306 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-09-11T23:40:07.306 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-11T23:40:07.306 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:07.306 INFO:teuthology.orchestra.run.smithi076.stdout: "throttle-write_buf_throttle": { 2024-09-11T23:40:07.306 INFO:teuthology.orchestra.run.smithi076.stdout: "val": 0, 2024-09-11T23:40:07.306 INFO:teuthology.orchestra.run.smithi076.stdout: "max": 3758096384, 2024-09-11T23:40:07.306 INFO:teuthology.orchestra.run.smithi076.stdout: "get_started": 0, 2024-09-11T23:40:07.306 INFO:teuthology.orchestra.run.smithi076.stdout: "get": 0, 2024-09-11T23:40:07.306 INFO:teuthology.orchestra.run.smithi076.stdout: "get_sum": 0, 2024-09-11T23:40:07.306 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_fail": 0, 2024-09-11T23:40:07.306 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_success": 0, 2024-09-11T23:40:07.307 INFO:teuthology.orchestra.run.smithi076.stdout: "take": 0, 2024-09-11T23:40:07.307 INFO:teuthology.orchestra.run.smithi076.stdout: "take_sum": 0, 2024-09-11T23:40:07.307 INFO:teuthology.orchestra.run.smithi076.stdout: "put": 0, 2024-09-11T23:40:07.307 INFO:teuthology.orchestra.run.smithi076.stdout: "put_sum": 0, 2024-09-11T23:40:07.307 INFO:teuthology.orchestra.run.smithi076.stdout: "wait": { 2024-09-11T23:40:07.307 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-09-11T23:40:07.307 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-09-11T23:40:07.307 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-09-11T23:40:07.307 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-11T23:40:07.307 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-11T23:40:07.307 INFO:teuthology.orchestra.run.smithi076.stdout: "throttle-write_buf_throttle-0x557ed8dc8b40": { 2024-09-11T23:40:07.307 INFO:teuthology.orchestra.run.smithi076.stdout: "val": 0, 2024-09-11T23:40:07.307 INFO:teuthology.orchestra.run.smithi076.stdout: "max": 3758096384, 2024-09-11T23:40:07.307 INFO:teuthology.orchestra.run.smithi076.stdout: "get_started": 0, 2024-09-11T23:40:07.308 INFO:teuthology.orchestra.run.smithi076.stdout: "get": 0, 2024-09-11T23:40:07.308 INFO:teuthology.orchestra.run.smithi076.stdout: "get_sum": 0, 2024-09-11T23:40:07.308 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_fail": 0, 2024-09-11T23:40:07.308 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_success": 0, 2024-09-11T23:40:07.308 INFO:teuthology.orchestra.run.smithi076.stdout: "take": 0, 2024-09-11T23:40:07.308 INFO:teuthology.orchestra.run.smithi076.stdout: "take_sum": 0, 2024-09-11T23:40:07.308 INFO:teuthology.orchestra.run.smithi076.stdout: "put": 0, 2024-09-11T23:40:07.308 INFO:teuthology.orchestra.run.smithi076.stdout: "put_sum": 0, 2024-09-11T23:40:07.308 INFO:teuthology.orchestra.run.smithi076.stdout: "wait": { 2024-09-11T23:40:07.308 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-09-11T23:40:07.308 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-09-11T23:40:07.308 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-09-11T23:40:07.308 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-11T23:40:07.308 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-11T23:40:07.309 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-11T23:40:07.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:40:07 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:40:07.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:40:07 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:40:07.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:40:07 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:40:07.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:40:07 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:40:07.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:40:07 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:40:07.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:40:07 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:40:07.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:40:07 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:40:07.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:40:07 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:40:07.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:40:07 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:40:07.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:40:07 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:40:07.552 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:40:07 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:40:07.552 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:40:07 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:40:07.552 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:40:07 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:40:07.552 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:40:07 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:40:07.552 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:40:07 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' 2024-09-11T23:40:07.984 INFO:tasks.check_counter:Daemon mds.k mds.exported=0 2024-09-11T23:40:07.984 INFO:tasks.check_counter:Daemon mds.k mds.imported=0 2024-09-11T23:40:07.984 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update=0 2024-09-11T23:40:07.984 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update_receipt=0 2024-09-11T23:40:07.984 INFO:tasks.check_counter:Daemon mds.k mds.root_rsnaps=0 2024-09-11T23:40:07.985 INFO:tasks.check_counter:Daemon mds.k mds_server.req_mksnap_latency.avgcount=0 2024-09-11T23:40:07.985 INFO:tasks.check_counter:Daemon mds.k mds_server.req_rmsnap_latency.avgcount=0 2024-09-11T23:40:07.985 INFO:tasks.check_counter:Daemon mds.k mds.dir_split=0 2024-09-11T23:40:07.985 DEBUG:tasks.check_counter:Getting stats from c 2024-09-11T23:40:07.985 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph daemon mds.c perf dump 2024-09-11T23:40:08.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:40:08 smithi136 ceph-mon[35121]: pgmap v685: 97 pgs: 97 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 57 KiB/s wr, 31 op/s 2024-09-11T23:40:08.493 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T23:40:08.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:40:08 smithi026 ceph-mon[31172]: pgmap v685: 97 pgs: 97 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 57 KiB/s wr, 31 op/s 2024-09-11T23:40:08.552 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:40:08 smithi076 ceph-mon[35296]: pgmap v685: 97 pgs: 97 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 57 KiB/s wr, 31 op/s 2024-09-11T23:40:09.224 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-11T23:40:09.224 INFO:teuthology.orchestra.run.smithi136.stdout: "AsyncMessenger::Worker-0": { 2024-09-11T23:40:09.224 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_messages": 45178, 2024-09-11T23:40:09.224 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages": 46378, 2024-09-11T23:40:09.224 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_bytes": 20930058, 2024-09-11T23:40:09.225 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_bytes": 104499969, 2024-09-11T23:40:09.225 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_created_connections": 15, 2024-09-11T23:40:09.225 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_active_connections": 4, 2024-09-11T23:40:09.225 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_total_time": 2.766001500, 2024-09-11T23:40:09.225 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_send_time": 1.115153500, 2024-09-11T23:40:09.225 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_recv_time": 1.620130583, 2024-09-11T23:40:09.225 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_fast_dispatch_time": 0.294904673, 2024-09-11T23:40:09.225 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T23:40:09.225 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 46376, 2024-09-11T23:40:09.225 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 4.184324312, 2024-09-11T23:40:09.225 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000090226 2024-09-11T23:40:09.225 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:09.225 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_handle_ack_lat": { 2024-09-11T23:40:09.226 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 75292, 2024-09-11T23:40:09.226 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.012574586, 2024-09-11T23:40:09.226 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000167 2024-09-11T23:40:09.226 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:09.226 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-11T23:40:09.226 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_encrypted_bytes": 0 2024-09-11T23:40:09.226 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:09.226 INFO:teuthology.orchestra.run.smithi136.stdout: "AsyncMessenger::Worker-1": { 2024-09-11T23:40:09.226 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_messages": 46313, 2024-09-11T23:40:09.226 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages": 47240, 2024-09-11T23:40:09.226 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_bytes": 26984418, 2024-09-11T23:40:09.226 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_bytes": 107925328, 2024-09-11T23:40:09.226 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_created_connections": 19, 2024-09-11T23:40:09.226 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_active_connections": 5, 2024-09-11T23:40:09.227 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_total_time": 3.381605770, 2024-09-11T23:40:09.227 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_send_time": 1.604813691, 2024-09-11T23:40:09.227 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_recv_time": 1.793005546, 2024-09-11T23:40:09.227 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_fast_dispatch_time": 0.255247869, 2024-09-11T23:40:09.227 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T23:40:09.227 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 47237, 2024-09-11T23:40:09.227 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 2.929133563, 2024-09-11T23:40:09.227 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000062009 2024-09-11T23:40:09.227 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:09.227 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_handle_ack_lat": { 2024-09-11T23:40:09.227 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 77205, 2024-09-11T23:40:09.227 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.013942815, 2024-09-11T23:40:09.227 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000180 2024-09-11T23:40:09.227 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:09.227 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_encrypted_bytes": 4752, 2024-09-11T23:40:09.228 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_encrypted_bytes": 786256 2024-09-11T23:40:09.228 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:09.228 INFO:teuthology.orchestra.run.smithi136.stdout: "AsyncMessenger::Worker-2": { 2024-09-11T23:40:09.228 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_messages": 218856, 2024-09-11T23:40:09.228 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages": 308711, 2024-09-11T23:40:09.228 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_bytes": 92817028, 2024-09-11T23:40:09.228 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_bytes": 242057806, 2024-09-11T23:40:09.228 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_created_connections": 13, 2024-09-11T23:40:09.228 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_active_connections": 6, 2024-09-11T23:40:09.228 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_total_time": 13.236788013, 2024-09-11T23:40:09.228 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_send_time": 6.110231158, 2024-09-11T23:40:09.228 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_recv_time": 11.296082599, 2024-09-11T23:40:09.228 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_fast_dispatch_time": 0.372483294, 2024-09-11T23:40:09.229 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T23:40:09.229 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 308711, 2024-09-11T23:40:09.229 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 26.283590746, 2024-09-11T23:40:09.229 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000085139 2024-09-11T23:40:09.229 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:09.229 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_handle_ack_lat": { 2024-09-11T23:40:09.229 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 416191, 2024-09-11T23:40:09.229 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.077050176, 2024-09-11T23:40:09.229 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000185 2024-09-11T23:40:09.229 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:09.229 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_encrypted_bytes": 74415696, 2024-09-11T23:40:09.229 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_encrypted_bytes": 140001968 2024-09-11T23:40:09.229 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:09.229 INFO:teuthology.orchestra.run.smithi136.stdout: "cct": { 2024-09-11T23:40:09.230 INFO:teuthology.orchestra.run.smithi136.stdout: "total_workers": 1, 2024-09-11T23:40:09.230 INFO:teuthology.orchestra.run.smithi136.stdout: "unhealthy_workers": 0 2024-09-11T23:40:09.230 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:09.230 INFO:teuthology.orchestra.run.smithi136.stdout: "finisher-MDSRank": { 2024-09-11T23:40:09.230 INFO:teuthology.orchestra.run.smithi136.stdout: "queue_len": 0, 2024-09-11T23:40:09.230 INFO:teuthology.orchestra.run.smithi136.stdout: "complete_latency": { 2024-09-11T23:40:09.230 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 7464, 2024-09-11T23:40:09.230 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 47.325632039, 2024-09-11T23:40:09.230 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.006340518 2024-09-11T23:40:09.230 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-11T23:40:09.230 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:09.230 INFO:teuthology.orchestra.run.smithi136.stdout: "finisher-PurgeQueue": { 2024-09-11T23:40:09.230 INFO:teuthology.orchestra.run.smithi136.stdout: "queue_len": 0, 2024-09-11T23:40:09.230 INFO:teuthology.orchestra.run.smithi136.stdout: "complete_latency": { 2024-09-11T23:40:09.231 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 28141, 2024-09-11T23:40:09.231 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 4.801196726, 2024-09-11T23:40:09.231 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000170612 2024-09-11T23:40:09.231 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-11T23:40:09.231 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:09.231 INFO:teuthology.orchestra.run.smithi136.stdout: "mds": { 2024-09-11T23:40:09.231 INFO:teuthology.orchestra.run.smithi136.stdout: "request": 90133, 2024-09-11T23:40:09.231 INFO:teuthology.orchestra.run.smithi136.stdout: "reply": 72188, 2024-09-11T23:40:09.231 INFO:teuthology.orchestra.run.smithi136.stdout: "reply_latency": { 2024-09-11T23:40:09.231 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 72188, 2024-09-11T23:40:09.231 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 659.591292472, 2024-09-11T23:40:09.231 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.009137132 2024-09-11T23:40:09.231 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:09.231 INFO:teuthology.orchestra.run.smithi136.stdout: "slow_reply": 0, 2024-09-11T23:40:09.231 INFO:teuthology.orchestra.run.smithi136.stdout: "forward": 17945, 2024-09-11T23:40:09.232 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_fetch_complete": 101, 2024-09-11T23:40:09.232 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_fetch_keys": 0, 2024-09-11T23:40:09.232 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_commit": 596, 2024-09-11T23:40:09.232 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_split": 65, 2024-09-11T23:40:09.232 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_merge": 0, 2024-09-11T23:40:09.232 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes": 23486, 2024-09-11T23:40:09.232 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes_top": 208, 2024-09-11T23:40:09.232 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes_bottom": 15733, 2024-09-11T23:40:09.232 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes_pin_tail": 7545, 2024-09-11T23:40:09.232 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes_pinned": 23185, 2024-09-11T23:40:09.232 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes_expired": 30962, 2024-09-11T23:40:09.232 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes_with_caps": 0, 2024-09-11T23:40:09.232 INFO:teuthology.orchestra.run.smithi136.stdout: "caps": 0, 2024-09-11T23:40:09.232 INFO:teuthology.orchestra.run.smithi136.stdout: "subtrees": 6, 2024-09-11T23:40:09.232 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse": 157565, 2024-09-11T23:40:09.233 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse_hit": 117126, 2024-09-11T23:40:09.233 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse_forward": 17807, 2024-09-11T23:40:09.233 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse_discover": 1101, 2024-09-11T23:40:09.233 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse_dir_fetch": 4, 2024-09-11T23:40:09.233 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse_remote_ino": 0, 2024-09-11T23:40:09.233 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse_lock": 306, 2024-09-11T23:40:09.233 INFO:teuthology.orchestra.run.smithi136.stdout: "load_cent": 0, 2024-09-11T23:40:09.233 INFO:teuthology.orchestra.run.smithi136.stdout: "q": 0, 2024-09-11T23:40:09.233 INFO:teuthology.orchestra.run.smithi136.stdout: "exported": 414, 2024-09-11T23:40:09.233 INFO:teuthology.orchestra.run.smithi136.stdout: "exported_inodes": 21451, 2024-09-11T23:40:09.233 INFO:teuthology.orchestra.run.smithi136.stdout: "imported": 281, 2024-09-11T23:40:09.233 INFO:teuthology.orchestra.run.smithi136.stdout: "imported_inodes": 20618, 2024-09-11T23:40:09.233 INFO:teuthology.orchestra.run.smithi136.stdout: "openino_dir_fetch": 0, 2024-09-11T23:40:09.234 INFO:teuthology.orchestra.run.smithi136.stdout: "openino_backtrace_fetch": 410, 2024-09-11T23:40:09.234 INFO:teuthology.orchestra.run.smithi136.stdout: "openino_peer_discover": 0, 2024-09-11T23:40:09.234 INFO:teuthology.orchestra.run.smithi136.stdout: "root_rfiles": 7006, 2024-09-11T23:40:09.234 INFO:teuthology.orchestra.run.smithi136.stdout: "root_rbytes": 34252265, 2024-09-11T23:40:09.234 INFO:teuthology.orchestra.run.smithi136.stdout: "root_rsnaps": 2, 2024-09-11T23:40:09.234 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_backtrace_fetch": 410, 2024-09-11T23:40:09.234 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_set_tag": 0, 2024-09-11T23:40:09.234 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_backtrace_repaired": 0, 2024-09-11T23:40:09.234 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_inotable_repaired": 0, 2024-09-11T23:40:09.234 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_dir_inodes": 0, 2024-09-11T23:40:09.234 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_dir_base_inodes": 0, 2024-09-11T23:40:09.234 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_dirfrag_rstats": 0, 2024-09-11T23:40:09.234 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_file_inodes": 410, 2024-09-11T23:40:09.234 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_inode_file_caps": 153, 2024-09-11T23:40:09.234 INFO:teuthology.orchestra.run.smithi136.stdout: "ceph_cap_op_revoke": 50814, 2024-09-11T23:40:09.234 INFO:teuthology.orchestra.run.smithi136.stdout: "ceph_cap_op_grant": 48530, 2024-09-11T23:40:09.235 INFO:teuthology.orchestra.run.smithi136.stdout: "ceph_cap_op_trunc": 0, 2024-09-11T23:40:09.235 INFO:teuthology.orchestra.run.smithi136.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-11T23:40:09.235 INFO:teuthology.orchestra.run.smithi136.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-11T23:40:09.235 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_client_caps": 64855, 2024-09-11T23:40:09.235 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_client_caps_dirty": 13671, 2024-09-11T23:40:09.235 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_client_cap_release": 1465, 2024-09-11T23:40:09.235 INFO:teuthology.orchestra.run.smithi136.stdout: "process_request_cap_release": 35462 2024-09-11T23:40:09.235 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:09.235 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_cache": { 2024-09-11T23:40:09.235 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_update": 3843, 2024-09-11T23:40:09.235 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_update_receipt": 13654, 2024-09-11T23:40:09.235 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_try_discover": 8088, 2024-09-11T23:40:09.235 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_send_discover": 9227, 2024-09-11T23:40:09.235 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_handle_discover": 4982, 2024-09-11T23:40:09.236 INFO:teuthology.orchestra.run.smithi136.stdout: "num_strays": 70, 2024-09-11T23:40:09.236 INFO:teuthology.orchestra.run.smithi136.stdout: "num_strays_delayed": 0, 2024-09-11T23:40:09.236 INFO:teuthology.orchestra.run.smithi136.stdout: "num_strays_enqueuing": 0, 2024-09-11T23:40:09.236 INFO:teuthology.orchestra.run.smithi136.stdout: "strays_created": 14959, 2024-09-11T23:40:09.236 INFO:teuthology.orchestra.run.smithi136.stdout: "strays_enqueued": 14889, 2024-09-11T23:40:09.236 INFO:teuthology.orchestra.run.smithi136.stdout: "strays_reintegrated": 0, 2024-09-11T23:40:09.236 INFO:teuthology.orchestra.run.smithi136.stdout: "strays_migrated": 0, 2024-09-11T23:40:09.236 INFO:teuthology.orchestra.run.smithi136.stdout: "num_recovering_processing": 0, 2024-09-11T23:40:09.236 INFO:teuthology.orchestra.run.smithi136.stdout: "num_recovering_enqueued": 0, 2024-09-11T23:40:09.236 INFO:teuthology.orchestra.run.smithi136.stdout: "num_recovering_prioritized": 0, 2024-09-11T23:40:09.236 INFO:teuthology.orchestra.run.smithi136.stdout: "recovery_started": 0, 2024-09-11T23:40:09.236 INFO:teuthology.orchestra.run.smithi136.stdout: "recovery_completed": 0, 2024-09-11T23:40:09.236 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_quiesce_path": 18, 2024-09-11T23:40:09.236 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_quiesce_inode": 122854, 2024-09-11T23:40:09.236 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_enqueue_scrub": 0, 2024-09-11T23:40:09.237 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_exportdir": 414, 2024-09-11T23:40:09.237 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_flush": 0, 2024-09-11T23:40:09.237 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_fragmentdir": 66, 2024-09-11T23:40:09.237 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_fragstats": 0, 2024-09-11T23:40:09.237 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_inodestats": 0 2024-09-11T23:40:09.237 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:09.237 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_log": { 2024-09-11T23:40:09.237 INFO:teuthology.orchestra.run.smithi136.stdout: "evlrg": 1, 2024-09-11T23:40:09.237 INFO:teuthology.orchestra.run.smithi136.stdout: "evadd": 82347, 2024-09-11T23:40:09.237 INFO:teuthology.orchestra.run.smithi136.stdout: "evex": 0, 2024-09-11T23:40:09.237 INFO:teuthology.orchestra.run.smithi136.stdout: "evtrm": 0, 2024-09-11T23:40:09.237 INFO:teuthology.orchestra.run.smithi136.stdout: "ev": 82347, 2024-09-11T23:40:09.237 INFO:teuthology.orchestra.run.smithi136.stdout: "evexg": 0, 2024-09-11T23:40:09.237 INFO:teuthology.orchestra.run.smithi136.stdout: "evexd": 0, 2024-09-11T23:40:09.237 INFO:teuthology.orchestra.run.smithi136.stdout: "segadd": 112, 2024-09-11T23:40:09.238 INFO:teuthology.orchestra.run.smithi136.stdout: "segex": 0, 2024-09-11T23:40:09.238 INFO:teuthology.orchestra.run.smithi136.stdout: "segtrm": 0, 2024-09-11T23:40:09.238 INFO:teuthology.orchestra.run.smithi136.stdout: "seg": 112, 2024-09-11T23:40:09.238 INFO:teuthology.orchestra.run.smithi136.stdout: "segmjr": 8, 2024-09-11T23:40:09.238 INFO:teuthology.orchestra.run.smithi136.stdout: "segexg": 0, 2024-09-11T23:40:09.238 INFO:teuthology.orchestra.run.smithi136.stdout: "segexd": 0, 2024-09-11T23:40:09.238 INFO:teuthology.orchestra.run.smithi136.stdout: "expos": 4194304, 2024-09-11T23:40:09.238 INFO:teuthology.orchestra.run.smithi136.stdout: "wrpos": 257645128, 2024-09-11T23:40:09.238 INFO:teuthology.orchestra.run.smithi136.stdout: "rdpos": 0, 2024-09-11T23:40:09.238 INFO:teuthology.orchestra.run.smithi136.stdout: "jlat": { 2024-09-11T23:40:09.238 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 4688, 2024-09-11T23:40:09.238 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 176.141874106, 2024-09-11T23:40:09.238 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.037572925 2024-09-11T23:40:09.238 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:09.239 INFO:teuthology.orchestra.run.smithi136.stdout: "replayed": 0 2024-09-11T23:40:09.239 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:09.239 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_mem": { 2024-09-11T23:40:09.239 INFO:teuthology.orchestra.run.smithi136.stdout: "ino": 11914, 2024-09-11T23:40:09.239 INFO:teuthology.orchestra.run.smithi136.stdout: "ino+": 42378, 2024-09-11T23:40:09.239 INFO:teuthology.orchestra.run.smithi136.stdout: "ino-": 30464, 2024-09-11T23:40:09.239 INFO:teuthology.orchestra.run.smithi136.stdout: "dir": 469, 2024-09-11T23:40:09.239 INFO:teuthology.orchestra.run.smithi136.stdout: "dir+": 6767, 2024-09-11T23:40:09.239 INFO:teuthology.orchestra.run.smithi136.stdout: "dir-": 6298, 2024-09-11T23:40:09.239 INFO:teuthology.orchestra.run.smithi136.stdout: "dn": 23486, 2024-09-11T23:40:09.239 INFO:teuthology.orchestra.run.smithi136.stdout: "dn+": 73237, 2024-09-11T23:40:09.239 INFO:teuthology.orchestra.run.smithi136.stdout: "dn-": 49751, 2024-09-11T23:40:09.239 INFO:teuthology.orchestra.run.smithi136.stdout: "cap": 0, 2024-09-11T23:40:09.240 INFO:teuthology.orchestra.run.smithi136.stdout: "cap+": 31581, 2024-09-11T23:40:09.240 INFO:teuthology.orchestra.run.smithi136.stdout: "cap-": 31581, 2024-09-11T23:40:09.240 INFO:teuthology.orchestra.run.smithi136.stdout: "rss": 193428, 2024-09-11T23:40:09.240 INFO:teuthology.orchestra.run.smithi136.stdout: "heap": 223520 2024-09-11T23:40:09.240 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:09.240 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_server": { 2024-09-11T23:40:09.240 INFO:teuthology.orchestra.run.smithi136.stdout: "dispatch_client_request": 104130, 2024-09-11T23:40:09.240 INFO:teuthology.orchestra.run.smithi136.stdout: "dispatch_server_request": 2885, 2024-09-11T23:40:09.240 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_client_request": 90133, 2024-09-11T23:40:09.240 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_client_session": 160, 2024-09-11T23:40:09.240 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_peer_request": 14751, 2024-09-11T23:40:09.240 INFO:teuthology.orchestra.run.smithi136.stdout: "req_create_latency": { 2024-09-11T23:40:09.240 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 9656, 2024-09-11T23:40:09.240 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 79.858963911, 2024-09-11T23:40:09.241 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.008270398 2024-09-11T23:40:09.241 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:09.241 INFO:teuthology.orchestra.run.smithi136.stdout: "req_getattr_latency": { 2024-09-11T23:40:09.241 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 59, 2024-09-11T23:40:09.241 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.745480233, 2024-09-11T23:40:09.241 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.012635258 2024-09-11T23:40:09.241 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:09.241 INFO:teuthology.orchestra.run.smithi136.stdout: "req_getfilelock_latency": { 2024-09-11T23:40:09.241 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-11T23:40:09.241 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-11T23:40:09.241 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-11T23:40:09.241 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:09.241 INFO:teuthology.orchestra.run.smithi136.stdout: "req_link_latency": { 2024-09-11T23:40:09.241 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-11T23:40:09.242 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-11T23:40:09.242 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-11T23:40:09.242 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:09.242 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lookup_latency": { 2024-09-11T23:40:09.242 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 34309, 2024-09-11T23:40:09.242 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 225.044239955, 2024-09-11T23:40:09.242 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.006559335 2024-09-11T23:40:09.242 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:09.242 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lookuphash_latency": { 2024-09-11T23:40:09.242 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-11T23:40:09.242 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-11T23:40:09.242 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-11T23:40:09.242 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:09.242 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lookupino_latency": { 2024-09-11T23:40:09.242 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-11T23:40:09.242 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-11T23:40:09.243 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-11T23:40:09.243 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:09.243 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lookupname_latency": { 2024-09-11T23:40:09.243 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-11T23:40:09.243 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-11T23:40:09.243 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-11T23:40:09.243 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:09.243 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lookupparent_latency": { 2024-09-11T23:40:09.243 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-11T23:40:09.243 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-11T23:40:09.243 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-11T23:40:09.244 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:09.244 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lookupsnap_latency": { 2024-09-11T23:40:09.244 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-11T23:40:09.244 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-11T23:40:09.244 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-11T23:40:09.245 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:09.245 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lssnap_latency": { 2024-09-11T23:40:09.245 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-11T23:40:09.245 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-11T23:40:09.245 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-11T23:40:09.245 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:09.245 INFO:teuthology.orchestra.run.smithi136.stdout: "req_mkdir_latency": { 2024-09-11T23:40:09.245 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 52, 2024-09-11T23:40:09.245 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 1.540496287, 2024-09-11T23:40:09.245 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.029624928 2024-09-11T23:40:09.245 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:09.245 INFO:teuthology.orchestra.run.smithi136.stdout: "req_mknod_latency": { 2024-09-11T23:40:09.245 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-11T23:40:09.246 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-11T23:40:09.246 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-11T23:40:09.246 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:09.246 INFO:teuthology.orchestra.run.smithi136.stdout: "req_mksnap_latency": { 2024-09-11T23:40:09.246 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-11T23:40:09.246 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-11T23:40:09.246 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-11T23:40:09.246 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:09.246 INFO:teuthology.orchestra.run.smithi136.stdout: "req_open_latency": { 2024-09-11T23:40:09.246 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 4609, 2024-09-11T23:40:09.246 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 9.510502769, 2024-09-11T23:40:09.246 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.002063463 2024-09-11T23:40:09.246 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:09.246 INFO:teuthology.orchestra.run.smithi136.stdout: "req_readdir_latency": { 2024-09-11T23:40:09.247 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 633, 2024-09-11T23:40:09.247 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 1.292457268, 2024-09-11T23:40:09.247 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.002041796 2024-09-11T23:40:09.247 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:09.247 INFO:teuthology.orchestra.run.smithi136.stdout: "req_rename_latency": { 2024-09-11T23:40:09.247 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 9644, 2024-09-11T23:40:09.247 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 279.023929239, 2024-09-11T23:40:09.247 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.028932385 2024-09-11T23:40:09.247 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:09.247 INFO:teuthology.orchestra.run.smithi136.stdout: "req_renamesnap_latency": { 2024-09-11T23:40:09.247 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-11T23:40:09.247 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-11T23:40:09.247 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-11T23:40:09.248 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:09.248 INFO:teuthology.orchestra.run.smithi136.stdout: "req_snapdiff_latency": { 2024-09-11T23:40:09.248 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-11T23:40:09.248 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-11T23:40:09.248 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-11T23:40:09.248 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:09.248 INFO:teuthology.orchestra.run.smithi136.stdout: "req_rmdir_latency": { 2024-09-11T23:40:09.248 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 70, 2024-09-11T23:40:09.248 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 12.693984117, 2024-09-11T23:40:09.248 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.181342630 2024-09-11T23:40:09.248 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:09.248 INFO:teuthology.orchestra.run.smithi136.stdout: "req_rmsnap_latency": { 2024-09-11T23:40:09.248 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-11T23:40:09.248 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-11T23:40:09.249 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-11T23:40:09.249 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:09.249 INFO:teuthology.orchestra.run.smithi136.stdout: "req_rmxattr_latency": { 2024-09-11T23:40:09.249 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-11T23:40:09.249 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-11T23:40:09.249 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-11T23:40:09.249 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:09.249 INFO:teuthology.orchestra.run.smithi136.stdout: "req_setattr_latency": { 2024-09-11T23:40:09.249 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-11T23:40:09.249 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-11T23:40:09.249 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-11T23:40:09.249 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:09.249 INFO:teuthology.orchestra.run.smithi136.stdout: "req_setdirlayout_latency": { 2024-09-11T23:40:09.250 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-11T23:40:09.250 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-11T23:40:09.250 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-11T23:40:09.250 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:09.250 INFO:teuthology.orchestra.run.smithi136.stdout: "req_setfilelock_latency": { 2024-09-11T23:40:09.250 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-11T23:40:09.250 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-11T23:40:09.250 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-11T23:40:09.250 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:09.250 INFO:teuthology.orchestra.run.smithi136.stdout: "req_setlayout_latency": { 2024-09-11T23:40:09.250 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-11T23:40:09.250 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-11T23:40:09.250 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-11T23:40:09.251 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:09.251 INFO:teuthology.orchestra.run.smithi136.stdout: "req_setxattr_latency": { 2024-09-11T23:40:09.251 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-11T23:40:09.251 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-11T23:40:09.251 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-11T23:40:09.251 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:09.251 INFO:teuthology.orchestra.run.smithi136.stdout: "req_symlink_latency": { 2024-09-11T23:40:09.251 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-11T23:40:09.251 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-11T23:40:09.251 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-11T23:40:09.251 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:09.251 INFO:teuthology.orchestra.run.smithi136.stdout: "req_unlink_latency": { 2024-09-11T23:40:09.251 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 13156, 2024-09-11T23:40:09.251 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 49.881238693, 2024-09-11T23:40:09.252 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.003791520 2024-09-11T23:40:09.252 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:09.252 INFO:teuthology.orchestra.run.smithi136.stdout: "cap_revoke_eviction": 0, 2024-09-11T23:40:09.252 INFO:teuthology.orchestra.run.smithi136.stdout: "cap_acquisition_throttle": 0, 2024-09-11T23:40:09.252 INFO:teuthology.orchestra.run.smithi136.stdout: "req_getvxattr_latency": { 2024-09-11T23:40:09.252 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-11T23:40:09.252 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-11T23:40:09.252 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-11T23:40:09.252 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-11T23:40:09.252 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:09.252 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_sessions": { 2024-09-11T23:40:09.252 INFO:teuthology.orchestra.run.smithi136.stdout: "session_count": 2, 2024-09-11T23:40:09.252 INFO:teuthology.orchestra.run.smithi136.stdout: "session_add": 3, 2024-09-11T23:40:09.252 INFO:teuthology.orchestra.run.smithi136.stdout: "session_remove": 1, 2024-09-11T23:40:09.253 INFO:teuthology.orchestra.run.smithi136.stdout: "sessions_open": 2, 2024-09-11T23:40:09.253 INFO:teuthology.orchestra.run.smithi136.stdout: "sessions_stale": 0, 2024-09-11T23:40:09.253 INFO:teuthology.orchestra.run.smithi136.stdout: "total_load": 274, 2024-09-11T23:40:09.253 INFO:teuthology.orchestra.run.smithi136.stdout: "average_load": 137, 2024-09-11T23:40:09.253 INFO:teuthology.orchestra.run.smithi136.stdout: "avg_session_uptime": 1244, 2024-09-11T23:40:09.253 INFO:teuthology.orchestra.run.smithi136.stdout: "mdthresh_evicted": 0 2024-09-11T23:40:09.253 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:09.253 INFO:teuthology.orchestra.run.smithi136.stdout: "mempool": { 2024-09-11T23:40:09.253 INFO:teuthology.orchestra.run.smithi136.stdout: "bloom_filter_bytes": 0, 2024-09-11T23:40:09.253 INFO:teuthology.orchestra.run.smithi136.stdout: "bloom_filter_items": 0, 2024-09-11T23:40:09.253 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_alloc_bytes": 0, 2024-09-11T23:40:09.253 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_alloc_items": 0, 2024-09-11T23:40:09.253 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_data_bytes": 0, 2024-09-11T23:40:09.254 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_data_items": 0, 2024-09-11T23:40:09.254 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-11T23:40:09.254 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_onode_items": 0, 2024-09-11T23:40:09.254 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-11T23:40:09.254 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_meta_items": 0, 2024-09-11T23:40:09.254 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_other_bytes": 0, 2024-09-11T23:40:09.254 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_other_items": 0, 2024-09-11T23:40:09.254 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-11T23:40:09.254 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_buffer_items": 0, 2024-09-11T23:40:09.254 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_extent_bytes": 0, 2024-09-11T23:40:09.254 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_extent_items": 0, 2024-09-11T23:40:09.255 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_blob_bytes": 0, 2024-09-11T23:40:09.255 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_blob_items": 0, 2024-09-11T23:40:09.255 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-11T23:40:09.255 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_shared_blob_items": 0, 2024-09-11T23:40:09.255 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-11T23:40:09.255 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_inline_bl_items": 0, 2024-09-11T23:40:09.255 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_fsck_bytes": 0, 2024-09-11T23:40:09.255 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_fsck_items": 0, 2024-09-11T23:40:09.255 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_txc_bytes": 0, 2024-09-11T23:40:09.255 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_txc_items": 0, 2024-09-11T23:40:09.255 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-11T23:40:09.255 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_writing_deferred_items": 0, 2024-09-11T23:40:09.255 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_writing_bytes": 0, 2024-09-11T23:40:09.255 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_writing_items": 0, 2024-09-11T23:40:09.256 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_bytes": 0, 2024-09-11T23:40:09.256 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_items": 0, 2024-09-11T23:40:09.256 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_file_reader_bytes": 0, 2024-09-11T23:40:09.256 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_file_reader_items": 0, 2024-09-11T23:40:09.256 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_file_writer_bytes": 0, 2024-09-11T23:40:09.256 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_file_writer_items": 0, 2024-09-11T23:40:09.256 INFO:teuthology.orchestra.run.smithi136.stdout: "buffer_anon_bytes": 1247295, 2024-09-11T23:40:09.256 INFO:teuthology.orchestra.run.smithi136.stdout: "buffer_anon_items": 9926, 2024-09-11T23:40:09.256 INFO:teuthology.orchestra.run.smithi136.stdout: "buffer_meta_bytes": 0, 2024-09-11T23:40:09.256 INFO:teuthology.orchestra.run.smithi136.stdout: "buffer_meta_items": 0, 2024-09-11T23:40:09.256 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_bytes": 0, 2024-09-11T23:40:09.256 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_items": 0, 2024-09-11T23:40:09.256 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_mapbl_bytes": 0, 2024-09-11T23:40:09.256 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_mapbl_items": 0, 2024-09-11T23:40:09.257 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_pglog_bytes": 0, 2024-09-11T23:40:09.257 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_pglog_items": 0, 2024-09-11T23:40:09.257 INFO:teuthology.orchestra.run.smithi136.stdout: "osdmap_bytes": 8224, 2024-09-11T23:40:09.257 INFO:teuthology.orchestra.run.smithi136.stdout: "osdmap_items": 181, 2024-09-11T23:40:09.257 INFO:teuthology.orchestra.run.smithi136.stdout: "osdmap_mapping_bytes": 0, 2024-09-11T23:40:09.257 INFO:teuthology.orchestra.run.smithi136.stdout: "osdmap_mapping_items": 0, 2024-09-11T23:40:09.257 INFO:teuthology.orchestra.run.smithi136.stdout: "pgmap_bytes": 0, 2024-09-11T23:40:09.257 INFO:teuthology.orchestra.run.smithi136.stdout: "pgmap_items": 0, 2024-09-11T23:40:09.257 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_co_bytes": 44920946, 2024-09-11T23:40:09.257 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_co_items": 462757, 2024-09-11T23:40:09.257 INFO:teuthology.orchestra.run.smithi136.stdout: "unittest_1_bytes": 0, 2024-09-11T23:40:09.257 INFO:teuthology.orchestra.run.smithi136.stdout: "unittest_1_items": 0, 2024-09-11T23:40:09.257 INFO:teuthology.orchestra.run.smithi136.stdout: "unittest_2_bytes": 0, 2024-09-11T23:40:09.258 INFO:teuthology.orchestra.run.smithi136.stdout: "unittest_2_items": 0 2024-09-11T23:40:09.258 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:09.258 INFO:teuthology.orchestra.run.smithi136.stdout: "objecter": { 2024-09-11T23:40:09.258 INFO:teuthology.orchestra.run.smithi136.stdout: "op_active": 0, 2024-09-11T23:40:09.258 INFO:teuthology.orchestra.run.smithi136.stdout: "op_laggy": 0, 2024-09-11T23:40:09.258 INFO:teuthology.orchestra.run.smithi136.stdout: "op_send": 22384, 2024-09-11T23:40:09.258 INFO:teuthology.orchestra.run.smithi136.stdout: "op_send_bytes": 267731000, 2024-09-11T23:40:09.258 INFO:teuthology.orchestra.run.smithi136.stdout: "op_resend": 28, 2024-09-11T23:40:09.258 INFO:teuthology.orchestra.run.smithi136.stdout: "op_reply": 22384, 2024-09-11T23:40:09.258 INFO:teuthology.orchestra.run.smithi136.stdout: "op_latency": { 2024-09-11T23:40:09.258 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 22384, 2024-09-11T23:40:09.258 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 411.913597671, 2024-09-11T23:40:09.258 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.018402144 2024-09-11T23:40:09.258 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:09.259 INFO:teuthology.orchestra.run.smithi136.stdout: "op_inflight": 0, 2024-09-11T23:40:09.259 INFO:teuthology.orchestra.run.smithi136.stdout: "oplen_avg": { 2024-09-11T23:40:09.259 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 22384, 2024-09-11T23:40:09.259 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 23805 2024-09-11T23:40:09.259 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:09.259 INFO:teuthology.orchestra.run.smithi136.stdout: "op": 22384, 2024-09-11T23:40:09.259 INFO:teuthology.orchestra.run.smithi136.stdout: "op_r": 937, 2024-09-11T23:40:09.259 INFO:teuthology.orchestra.run.smithi136.stdout: "op_w": 21447, 2024-09-11T23:40:09.259 INFO:teuthology.orchestra.run.smithi136.stdout: "op_rmw": 0, 2024-09-11T23:40:09.259 INFO:teuthology.orchestra.run.smithi136.stdout: "op_pg": 0, 2024-09-11T23:40:09.259 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_stat": 0, 2024-09-11T23:40:09.259 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_create": 10, 2024-09-11T23:40:09.259 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_read": 426, 2024-09-11T23:40:09.260 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_write": 5119, 2024-09-11T23:40:09.260 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_writefull": 250, 2024-09-11T23:40:09.260 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_writesame": 0, 2024-09-11T23:40:09.260 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_append": 0, 2024-09-11T23:40:09.260 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_zero": 0, 2024-09-11T23:40:09.260 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_truncate": 65, 2024-09-11T23:40:09.260 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_delete": 14961, 2024-09-11T23:40:09.260 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_mapext": 0, 2024-09-11T23:40:09.260 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_sparse_read": 0, 2024-09-11T23:40:09.260 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_clonerange": 0, 2024-09-11T23:40:09.260 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_getxattr": 411, 2024-09-11T23:40:09.260 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_setxattr": 20, 2024-09-11T23:40:09.260 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_cmpxattr": 0, 2024-09-11T23:40:09.260 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_rmxattr": 0, 2024-09-11T23:40:09.261 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_resetxattrs": 0, 2024-09-11T23:40:09.261 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_call": 0, 2024-09-11T23:40:09.261 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_watch": 0, 2024-09-11T23:40:09.261 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_notify": 0, 2024-09-11T23:40:09.261 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_src_cmpxattr": 0, 2024-09-11T23:40:09.261 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_pgls": 0, 2024-09-11T23:40:09.261 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_pgls_filter": 0, 2024-09-11T23:40:09.261 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_other": 0, 2024-09-11T23:40:09.261 INFO:teuthology.orchestra.run.smithi136.stdout: "linger_active": 0, 2024-09-11T23:40:09.261 INFO:teuthology.orchestra.run.smithi136.stdout: "linger_send": 0, 2024-09-11T23:40:09.261 INFO:teuthology.orchestra.run.smithi136.stdout: "linger_resend": 0, 2024-09-11T23:40:09.261 INFO:teuthology.orchestra.run.smithi136.stdout: "linger_ping": 0, 2024-09-11T23:40:09.261 INFO:teuthology.orchestra.run.smithi136.stdout: "poolop_active": 0, 2024-09-11T23:40:09.261 INFO:teuthology.orchestra.run.smithi136.stdout: "poolop_send": 0, 2024-09-11T23:40:09.262 INFO:teuthology.orchestra.run.smithi136.stdout: "poolop_resend": 0, 2024-09-11T23:40:09.262 INFO:teuthology.orchestra.run.smithi136.stdout: "poolstat_active": 0, 2024-09-11T23:40:09.262 INFO:teuthology.orchestra.run.smithi136.stdout: "poolstat_send": 0, 2024-09-11T23:40:09.262 INFO:teuthology.orchestra.run.smithi136.stdout: "poolstat_resend": 0, 2024-09-11T23:40:09.262 INFO:teuthology.orchestra.run.smithi136.stdout: "statfs_active": 0, 2024-09-11T23:40:09.262 INFO:teuthology.orchestra.run.smithi136.stdout: "statfs_send": 0, 2024-09-11T23:40:09.262 INFO:teuthology.orchestra.run.smithi136.stdout: "statfs_resend": 0, 2024-09-11T23:40:09.262 INFO:teuthology.orchestra.run.smithi136.stdout: "command_active": 0, 2024-09-11T23:40:09.262 INFO:teuthology.orchestra.run.smithi136.stdout: "command_send": 0, 2024-09-11T23:40:09.262 INFO:teuthology.orchestra.run.smithi136.stdout: "command_resend": 0, 2024-09-11T23:40:09.262 INFO:teuthology.orchestra.run.smithi136.stdout: "map_epoch": 116, 2024-09-11T23:40:09.262 INFO:teuthology.orchestra.run.smithi136.stdout: "map_full": 0, 2024-09-11T23:40:09.262 INFO:teuthology.orchestra.run.smithi136.stdout: "map_inc": 31, 2024-09-11T23:40:09.262 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_sessions": 12, 2024-09-11T23:40:09.263 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_session_open": 12, 2024-09-11T23:40:09.263 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_session_close": 0, 2024-09-11T23:40:09.263 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_laggy": 0, 2024-09-11T23:40:09.263 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_wr": 1999, 2024-09-11T23:40:09.263 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_rd": 202, 2024-09-11T23:40:09.263 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_del": 342 2024-09-11T23:40:09.263 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:09.263 INFO:teuthology.orchestra.run.smithi136.stdout: "oft": { 2024-09-11T23:40:09.263 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_total_objs": 1, 2024-09-11T23:40:09.263 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_total_kv_pairs": 0, 2024-09-11T23:40:09.263 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_total_updates": 0, 2024-09-11T23:40:09.263 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_total_removes": 0 2024-09-11T23:40:09.263 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:09.263 INFO:teuthology.orchestra.run.smithi136.stdout: "osd-slow-ops": { 2024-09-11T23:40:09.263 INFO:teuthology.orchestra.run.smithi136.stdout: "slow_ops_count": 122720 2024-09-11T23:40:09.263 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:09.264 INFO:teuthology.orchestra.run.smithi136.stdout: "purge_queue": { 2024-09-11T23:40:09.264 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_executing_ops": 0, 2024-09-11T23:40:09.264 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_executing_ops_high_water": 6, 2024-09-11T23:40:09.264 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_executing": 0, 2024-09-11T23:40:09.264 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_executing_high_water": 6, 2024-09-11T23:40:09.264 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_executed_ops": 14889, 2024-09-11T23:40:09.264 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_executed": 14889, 2024-09-11T23:40:09.264 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_item_in_journal": 0 2024-09-11T23:40:09.264 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:09.264 INFO:teuthology.orchestra.run.smithi136.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-11T23:40:09.264 INFO:teuthology.orchestra.run.smithi136.stdout: "val": 0, 2024-09-11T23:40:09.264 INFO:teuthology.orchestra.run.smithi136.stdout: "max": 104857600, 2024-09-11T23:40:09.264 INFO:teuthology.orchestra.run.smithi136.stdout: "get_started": 0, 2024-09-11T23:40:09.265 INFO:teuthology.orchestra.run.smithi136.stdout: "get": 310343, 2024-09-11T23:40:09.265 INFO:teuthology.orchestra.run.smithi136.stdout: "get_sum": 103489376, 2024-09-11T23:40:09.265 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_fail": 0, 2024-09-11T23:40:09.265 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_success": 310343, 2024-09-11T23:40:09.265 INFO:teuthology.orchestra.run.smithi136.stdout: "take": 0, 2024-09-11T23:40:09.265 INFO:teuthology.orchestra.run.smithi136.stdout: "take_sum": 0, 2024-09-11T23:40:09.265 INFO:teuthology.orchestra.run.smithi136.stdout: "put": 310343, 2024-09-11T23:40:09.265 INFO:teuthology.orchestra.run.smithi136.stdout: "put_sum": 103489376, 2024-09-11T23:40:09.265 INFO:teuthology.orchestra.run.smithi136.stdout: "wait": { 2024-09-11T23:40:09.265 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-11T23:40:09.265 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-11T23:40:09.265 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-11T23:40:09.266 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-11T23:40:09.266 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:09.266 INFO:teuthology.orchestra.run.smithi136.stdout: "throttle-objecter_bytes": { 2024-09-11T23:40:09.266 INFO:teuthology.orchestra.run.smithi136.stdout: "val": 0, 2024-09-11T23:40:09.266 INFO:teuthology.orchestra.run.smithi136.stdout: "max": 104857600, 2024-09-11T23:40:09.266 INFO:teuthology.orchestra.run.smithi136.stdout: "get_started": 0, 2024-09-11T23:40:09.266 INFO:teuthology.orchestra.run.smithi136.stdout: "get": 0, 2024-09-11T23:40:09.266 INFO:teuthology.orchestra.run.smithi136.stdout: "get_sum": 0, 2024-09-11T23:40:09.266 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_fail": 0, 2024-09-11T23:40:09.266 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_success": 0, 2024-09-11T23:40:09.266 INFO:teuthology.orchestra.run.smithi136.stdout: "take": 22384, 2024-09-11T23:40:09.266 INFO:teuthology.orchestra.run.smithi136.stdout: "take_sum": 269877429, 2024-09-11T23:40:09.266 INFO:teuthology.orchestra.run.smithi136.stdout: "put": 22384, 2024-09-11T23:40:09.266 INFO:teuthology.orchestra.run.smithi136.stdout: "put_sum": 269877429, 2024-09-11T23:40:09.266 INFO:teuthology.orchestra.run.smithi136.stdout: "wait": { 2024-09-11T23:40:09.266 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-11T23:40:09.267 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-11T23:40:09.267 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-11T23:40:09.267 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-11T23:40:09.267 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:09.267 INFO:teuthology.orchestra.run.smithi136.stdout: "throttle-objecter_ops": { 2024-09-11T23:40:09.267 INFO:teuthology.orchestra.run.smithi136.stdout: "val": 0, 2024-09-11T23:40:09.267 INFO:teuthology.orchestra.run.smithi136.stdout: "max": 1024, 2024-09-11T23:40:09.267 INFO:teuthology.orchestra.run.smithi136.stdout: "get_started": 0, 2024-09-11T23:40:09.267 INFO:teuthology.orchestra.run.smithi136.stdout: "get": 0, 2024-09-11T23:40:09.267 INFO:teuthology.orchestra.run.smithi136.stdout: "get_sum": 0, 2024-09-11T23:40:09.268 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_fail": 0, 2024-09-11T23:40:09.268 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_success": 0, 2024-09-11T23:40:09.268 INFO:teuthology.orchestra.run.smithi136.stdout: "take": 22384, 2024-09-11T23:40:09.268 INFO:teuthology.orchestra.run.smithi136.stdout: "take_sum": 22384, 2024-09-11T23:40:09.268 INFO:teuthology.orchestra.run.smithi136.stdout: "put": 22384, 2024-09-11T23:40:09.268 INFO:teuthology.orchestra.run.smithi136.stdout: "put_sum": 22384, 2024-09-11T23:40:09.268 INFO:teuthology.orchestra.run.smithi136.stdout: "wait": { 2024-09-11T23:40:09.268 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-11T23:40:09.268 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-11T23:40:09.268 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-11T23:40:09.268 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-11T23:40:09.268 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:09.268 INFO:teuthology.orchestra.run.smithi136.stdout: "throttle-write_buf_throttle": { 2024-09-11T23:40:09.268 INFO:teuthology.orchestra.run.smithi136.stdout: "val": 0, 2024-09-11T23:40:09.269 INFO:teuthology.orchestra.run.smithi136.stdout: "max": 3758096384, 2024-09-11T23:40:09.269 INFO:teuthology.orchestra.run.smithi136.stdout: "get_started": 0, 2024-09-11T23:40:09.269 INFO:teuthology.orchestra.run.smithi136.stdout: "get": 14889, 2024-09-11T23:40:09.269 INFO:teuthology.orchestra.run.smithi136.stdout: "get_sum": 2148045, 2024-09-11T23:40:09.269 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_fail": 0, 2024-09-11T23:40:09.269 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_success": 14889, 2024-09-11T23:40:09.269 INFO:teuthology.orchestra.run.smithi136.stdout: "take": 0, 2024-09-11T23:40:09.269 INFO:teuthology.orchestra.run.smithi136.stdout: "take_sum": 0, 2024-09-11T23:40:09.269 INFO:teuthology.orchestra.run.smithi136.stdout: "put": 431, 2024-09-11T23:40:09.269 INFO:teuthology.orchestra.run.smithi136.stdout: "put_sum": 2148045, 2024-09-11T23:40:09.269 INFO:teuthology.orchestra.run.smithi136.stdout: "wait": { 2024-09-11T23:40:09.269 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-11T23:40:09.269 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-11T23:40:09.270 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-11T23:40:09.270 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-11T23:40:09.270 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:09.270 INFO:teuthology.orchestra.run.smithi136.stdout: "throttle-write_buf_throttle-0x556780f2e960": { 2024-09-11T23:40:09.270 INFO:teuthology.orchestra.run.smithi136.stdout: "val": 0, 2024-09-11T23:40:09.270 INFO:teuthology.orchestra.run.smithi136.stdout: "max": 3758096384, 2024-09-11T23:40:09.270 INFO:teuthology.orchestra.run.smithi136.stdout: "get_started": 0, 2024-09-11T23:40:09.270 INFO:teuthology.orchestra.run.smithi136.stdout: "get": 82347, 2024-09-11T23:40:09.270 INFO:teuthology.orchestra.run.smithi136.stdout: "get_sum": 253450824, 2024-09-11T23:40:09.270 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_fail": 0, 2024-09-11T23:40:09.270 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_success": 82347, 2024-09-11T23:40:09.270 INFO:teuthology.orchestra.run.smithi136.stdout: "take": 0, 2024-09-11T23:40:09.270 INFO:teuthology.orchestra.run.smithi136.stdout: "take_sum": 0, 2024-09-11T23:40:09.270 INFO:teuthology.orchestra.run.smithi136.stdout: "put": 4688, 2024-09-11T23:40:09.271 INFO:teuthology.orchestra.run.smithi136.stdout: "put_sum": 253450824, 2024-09-11T23:40:09.271 INFO:teuthology.orchestra.run.smithi136.stdout: "wait": { 2024-09-11T23:40:09.271 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-11T23:40:09.271 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-11T23:40:09.271 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-11T23:40:09.271 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-11T23:40:09.271 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-11T23:40:09.271 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-11T23:40:10.268 INFO:tasks.check_counter:Daemon mds.c mds.exported=414 2024-09-11T23:40:10.268 INFO:tasks.check_counter:Daemon mds.c mds.imported=281 2024-09-11T23:40:10.268 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update=3843 2024-09-11T23:40:10.268 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update_receipt=13654 2024-09-11T23:40:10.268 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=2 2024-09-11T23:40:10.268 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2024-09-11T23:40:10.268 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2024-09-11T23:40:10.268 INFO:tasks.check_counter:Daemon mds.c mds.dir_split=65 2024-09-11T23:40:10.268 DEBUG:tasks.check_counter:Getting stats from f 2024-09-11T23:40:10.269 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph daemon mds.f perf dump 2024-09-11T23:40:10.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:40:10 smithi136 ceph-mon[35121]: pgmap v686: 97 pgs: 97 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 57 KiB/s wr, 31 op/s 2024-09-11T23:40:10.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:40:10 smithi136 ceph-mon[35121]: osdmap e117: 12 total, 12 up, 12 in 2024-09-11T23:40:10.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:40:10 smithi026 ceph-mon[31172]: pgmap v686: 97 pgs: 97 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 57 KiB/s wr, 31 op/s 2024-09-11T23:40:10.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:40:10 smithi026 ceph-mon[31172]: osdmap e117: 12 total, 12 up, 12 in 2024-09-11T23:40:10.552 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:40:10 smithi076 ceph-mon[35296]: pgmap v686: 97 pgs: 97 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 57 KiB/s wr, 31 op/s 2024-09-11T23:40:10.552 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:40:10 smithi076 ceph-mon[35296]: osdmap e117: 12 total, 12 up, 12 in 2024-09-11T23:40:10.760 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T23:40:11.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:40:11 smithi026 ceph-mon[31172]: pgmap v688: 97 pgs: 15 active+clean+snaptrim_wait, 9 active+clean+snaptrim, 73 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 33 KiB/s wr, 13 op/s 2024-09-11T23:40:11.527 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:40:11 smithi136 ceph-mon[35121]: pgmap v688: 97 pgs: 15 active+clean+snaptrim_wait, 9 active+clean+snaptrim, 73 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 33 KiB/s wr, 13 op/s 2024-09-11T23:40:11.528 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-11T23:40:11.528 INFO:teuthology.orchestra.run.smithi136.stdout: "AsyncMessenger::Worker-0": { 2024-09-11T23:40:11.528 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_messages": 2769, 2024-09-11T23:40:11.528 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages": 2691, 2024-09-11T23:40:11.528 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_bytes": 95079857, 2024-09-11T23:40:11.528 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_bytes": 966178, 2024-09-11T23:40:11.528 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_created_connections": 7, 2024-09-11T23:40:11.528 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_active_connections": 3, 2024-09-11T23:40:11.528 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_total_time": 0.327271419, 2024-09-11T23:40:11.528 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_send_time": 0.084255526, 2024-09-11T23:40:11.528 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_recv_time": 0.170430569, 2024-09-11T23:40:11.528 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_fast_dispatch_time": 0.072538857, 2024-09-11T23:40:11.529 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T23:40:11.529 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 2691, 2024-09-11T23:40:11.529 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.133799338, 2024-09-11T23:40:11.529 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000049721 2024-09-11T23:40:11.529 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:11.529 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_handle_ack_lat": { 2024-09-11T23:40:11.529 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-11T23:40:11.529 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-11T23:40:11.529 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-11T23:40:11.529 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:11.529 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_encrypted_bytes": 2112064, 2024-09-11T23:40:11.529 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_encrypted_bytes": 245904 2024-09-11T23:40:11.530 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:11.530 INFO:teuthology.orchestra.run.smithi136.stdout: "AsyncMessenger::Worker-1": { 2024-09-11T23:40:11.530 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_messages": 1280, 2024-09-11T23:40:11.530 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages": 1573, 2024-09-11T23:40:11.530 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_bytes": 82172525, 2024-09-11T23:40:11.530 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_bytes": 1088278, 2024-09-11T23:40:11.530 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_created_connections": 9, 2024-09-11T23:40:11.530 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_active_connections": 7, 2024-09-11T23:40:11.530 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_total_time": 0.149464442, 2024-09-11T23:40:11.530 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_send_time": 0.040749580, 2024-09-11T23:40:11.530 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_recv_time": 0.066550318, 2024-09-11T23:40:11.531 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_fast_dispatch_time": 0.032164137, 2024-09-11T23:40:11.531 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T23:40:11.531 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 1570, 2024-09-11T23:40:11.531 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.112608035, 2024-09-11T23:40:11.531 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000071724 2024-09-11T23:40:11.531 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:11.531 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_handle_ack_lat": { 2024-09-11T23:40:11.531 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-11T23:40:11.531 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-11T23:40:11.531 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-11T23:40:11.531 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:11.531 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-11T23:40:11.532 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_encrypted_bytes": 681808 2024-09-11T23:40:11.532 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:11.532 INFO:teuthology.orchestra.run.smithi136.stdout: "AsyncMessenger::Worker-2": { 2024-09-11T23:40:11.532 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_messages": 1332, 2024-09-11T23:40:11.532 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages": 1334, 2024-09-11T23:40:11.532 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_bytes": 96811793, 2024-09-11T23:40:11.532 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_bytes": 423583, 2024-09-11T23:40:11.532 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_created_connections": 6, 2024-09-11T23:40:11.532 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_active_connections": 5, 2024-09-11T23:40:11.532 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_total_time": 0.161227702, 2024-09-11T23:40:11.532 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_send_time": 0.036069392, 2024-09-11T23:40:11.533 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_recv_time": 0.078535803, 2024-09-11T23:40:11.533 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_fast_dispatch_time": 0.037299750, 2024-09-11T23:40:11.533 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T23:40:11.533 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 1332, 2024-09-11T23:40:11.533 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.053818319, 2024-09-11T23:40:11.533 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000040404 2024-09-11T23:40:11.533 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:11.533 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_handle_ack_lat": { 2024-09-11T23:40:11.533 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-11T23:40:11.533 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-11T23:40:11.533 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-11T23:40:11.534 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:11.534 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-11T23:40:11.534 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_encrypted_bytes": 0 2024-09-11T23:40:11.534 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:11.534 INFO:teuthology.orchestra.run.smithi136.stdout: "cct": { 2024-09-11T23:40:11.534 INFO:teuthology.orchestra.run.smithi136.stdout: "total_workers": 1, 2024-09-11T23:40:11.534 INFO:teuthology.orchestra.run.smithi136.stdout: "unhealthy_workers": 0 2024-09-11T23:40:11.534 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:11.534 INFO:teuthology.orchestra.run.smithi136.stdout: "finisher-MDSRank": { 2024-09-11T23:40:11.534 INFO:teuthology.orchestra.run.smithi136.stdout: "queue_len": 0, 2024-09-11T23:40:11.535 INFO:teuthology.orchestra.run.smithi136.stdout: "complete_latency": { 2024-09-11T23:40:11.535 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 8048, 2024-09-11T23:40:11.535 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.923175736, 2024-09-11T23:40:11.535 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000114708 2024-09-11T23:40:11.535 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-11T23:40:11.535 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:11.535 INFO:teuthology.orchestra.run.smithi136.stdout: "finisher-PurgeQueue": { 2024-09-11T23:40:11.535 INFO:teuthology.orchestra.run.smithi136.stdout: "queue_len": 0, 2024-09-11T23:40:11.535 INFO:teuthology.orchestra.run.smithi136.stdout: "complete_latency": { 2024-09-11T23:40:11.536 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-11T23:40:11.536 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-11T23:40:11.536 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-11T23:40:11.536 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-11T23:40:11.536 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:11.536 INFO:teuthology.orchestra.run.smithi136.stdout: "mds": { 2024-09-11T23:40:11.536 INFO:teuthology.orchestra.run.smithi136.stdout: "request": 0, 2024-09-11T23:40:11.536 INFO:teuthology.orchestra.run.smithi136.stdout: "reply": 0, 2024-09-11T23:40:11.536 INFO:teuthology.orchestra.run.smithi136.stdout: "reply_latency": { 2024-09-11T23:40:11.536 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-11T23:40:11.536 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-11T23:40:11.536 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-11T23:40:11.537 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:11.537 INFO:teuthology.orchestra.run.smithi136.stdout: "slow_reply": 0, 2024-09-11T23:40:11.537 INFO:teuthology.orchestra.run.smithi136.stdout: "forward": 0, 2024-09-11T23:40:11.537 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_fetch_complete": 0, 2024-09-11T23:40:11.537 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_fetch_keys": 0, 2024-09-11T23:40:11.537 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_commit": 0, 2024-09-11T23:40:11.537 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_split": 0, 2024-09-11T23:40:11.537 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_merge": 0, 2024-09-11T23:40:11.537 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes": 49029, 2024-09-11T23:40:11.537 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes_top": 34214, 2024-09-11T23:40:11.537 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes_bottom": 14815, 2024-09-11T23:40:11.538 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes_pin_tail": 0, 2024-09-11T23:40:11.538 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes_pinned": 92, 2024-09-11T23:40:11.538 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes_expired": 0, 2024-09-11T23:40:11.538 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes_with_caps": 0, 2024-09-11T23:40:11.538 INFO:teuthology.orchestra.run.smithi136.stdout: "caps": 0, 2024-09-11T23:40:11.538 INFO:teuthology.orchestra.run.smithi136.stdout: "subtrees": 3, 2024-09-11T23:40:11.538 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse": 0, 2024-09-11T23:40:11.538 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse_hit": 0, 2024-09-11T23:40:11.538 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse_forward": 0, 2024-09-11T23:40:11.538 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse_discover": 0, 2024-09-11T23:40:11.538 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse_dir_fetch": 0, 2024-09-11T23:40:11.538 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse_remote_ino": 0, 2024-09-11T23:40:11.538 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse_lock": 0, 2024-09-11T23:40:11.538 INFO:teuthology.orchestra.run.smithi136.stdout: "load_cent": 0, 2024-09-11T23:40:11.539 INFO:teuthology.orchestra.run.smithi136.stdout: "q": 0, 2024-09-11T23:40:11.539 INFO:teuthology.orchestra.run.smithi136.stdout: "exported": 0, 2024-09-11T23:40:11.539 INFO:teuthology.orchestra.run.smithi136.stdout: "exported_inodes": 0, 2024-09-11T23:40:11.539 INFO:teuthology.orchestra.run.smithi136.stdout: "imported": 0, 2024-09-11T23:40:11.539 INFO:teuthology.orchestra.run.smithi136.stdout: "imported_inodes": 0, 2024-09-11T23:40:11.539 INFO:teuthology.orchestra.run.smithi136.stdout: "openino_dir_fetch": 0, 2024-09-11T23:40:11.539 INFO:teuthology.orchestra.run.smithi136.stdout: "openino_backtrace_fetch": 0, 2024-09-11T23:40:11.539 INFO:teuthology.orchestra.run.smithi136.stdout: "openino_peer_discover": 0, 2024-09-11T23:40:11.539 INFO:teuthology.orchestra.run.smithi136.stdout: "root_rfiles": 4, 2024-09-11T23:40:11.539 INFO:teuthology.orchestra.run.smithi136.stdout: "root_rbytes": 234, 2024-09-11T23:40:11.539 INFO:teuthology.orchestra.run.smithi136.stdout: "root_rsnaps": 6, 2024-09-11T23:40:11.539 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_backtrace_fetch": 0, 2024-09-11T23:40:11.539 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_set_tag": 0, 2024-09-11T23:40:11.539 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_backtrace_repaired": 0, 2024-09-11T23:40:11.540 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_inotable_repaired": 0, 2024-09-11T23:40:11.540 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_dir_inodes": 0, 2024-09-11T23:40:11.540 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_dir_base_inodes": 0, 2024-09-11T23:40:11.540 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_dirfrag_rstats": 0, 2024-09-11T23:40:11.540 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_file_inodes": 0, 2024-09-11T23:40:11.540 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_inode_file_caps": 0, 2024-09-11T23:40:11.540 INFO:teuthology.orchestra.run.smithi136.stdout: "ceph_cap_op_revoke": 0, 2024-09-11T23:40:11.540 INFO:teuthology.orchestra.run.smithi136.stdout: "ceph_cap_op_grant": 0, 2024-09-11T23:40:11.540 INFO:teuthology.orchestra.run.smithi136.stdout: "ceph_cap_op_trunc": 0, 2024-09-11T23:40:11.540 INFO:teuthology.orchestra.run.smithi136.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-11T23:40:11.540 INFO:teuthology.orchestra.run.smithi136.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-11T23:40:11.540 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_client_caps": 0, 2024-09-11T23:40:11.541 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_client_caps_dirty": 0, 2024-09-11T23:40:11.541 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_client_cap_release": 0, 2024-09-11T23:40:11.541 INFO:teuthology.orchestra.run.smithi136.stdout: "process_request_cap_release": 0 2024-09-11T23:40:11.541 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:11.541 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_cache": { 2024-09-11T23:40:11.541 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_update": 0, 2024-09-11T23:40:11.541 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_update_receipt": 0, 2024-09-11T23:40:11.541 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_try_discover": 0, 2024-09-11T23:40:11.541 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_send_discover": 0, 2024-09-11T23:40:11.541 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_handle_discover": 0, 2024-09-11T23:40:11.541 INFO:teuthology.orchestra.run.smithi136.stdout: "num_strays": 0, 2024-09-11T23:40:11.541 INFO:teuthology.orchestra.run.smithi136.stdout: "num_strays_delayed": 0, 2024-09-11T23:40:11.542 INFO:teuthology.orchestra.run.smithi136.stdout: "num_strays_enqueuing": 0, 2024-09-11T23:40:11.542 INFO:teuthology.orchestra.run.smithi136.stdout: "strays_created": 0, 2024-09-11T23:40:11.542 INFO:teuthology.orchestra.run.smithi136.stdout: "strays_enqueued": 0, 2024-09-11T23:40:11.542 INFO:teuthology.orchestra.run.smithi136.stdout: "strays_reintegrated": 0, 2024-09-11T23:40:11.542 INFO:teuthology.orchestra.run.smithi136.stdout: "strays_migrated": 0, 2024-09-11T23:40:11.542 INFO:teuthology.orchestra.run.smithi136.stdout: "num_recovering_processing": 0, 2024-09-11T23:40:11.542 INFO:teuthology.orchestra.run.smithi136.stdout: "num_recovering_enqueued": 0, 2024-09-11T23:40:11.542 INFO:teuthology.orchestra.run.smithi136.stdout: "num_recovering_prioritized": 0, 2024-09-11T23:40:11.542 INFO:teuthology.orchestra.run.smithi136.stdout: "recovery_started": 0, 2024-09-11T23:40:11.543 INFO:teuthology.orchestra.run.smithi136.stdout: "recovery_completed": 0, 2024-09-11T23:40:11.543 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_quiesce_path": 0, 2024-09-11T23:40:11.543 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_quiesce_inode": 0, 2024-09-11T23:40:11.543 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_enqueue_scrub": 0, 2024-09-11T23:40:11.543 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_exportdir": 0, 2024-09-11T23:40:11.543 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_flush": 0, 2024-09-11T23:40:11.543 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_fragmentdir": 0, 2024-09-11T23:40:11.543 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_fragstats": 0, 2024-09-11T23:40:11.543 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_inodestats": 0 2024-09-11T23:40:11.544 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:11.544 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_log": { 2024-09-11T23:40:11.544 INFO:teuthology.orchestra.run.smithi136.stdout: "evlrg": 0, 2024-09-11T23:40:11.544 INFO:teuthology.orchestra.run.smithi136.stdout: "evadd": 0, 2024-09-11T23:40:11.544 INFO:teuthology.orchestra.run.smithi136.stdout: "evex": 0, 2024-09-11T23:40:11.544 INFO:teuthology.orchestra.run.smithi136.stdout: "evtrm": 0, 2024-09-11T23:40:11.544 INFO:teuthology.orchestra.run.smithi136.stdout: "ev": 99784, 2024-09-11T23:40:11.544 INFO:teuthology.orchestra.run.smithi136.stdout: "evexg": 0, 2024-09-11T23:40:11.544 INFO:teuthology.orchestra.run.smithi136.stdout: "evexd": 0, 2024-09-11T23:40:11.545 INFO:teuthology.orchestra.run.smithi136.stdout: "segadd": 0, 2024-09-11T23:40:11.545 INFO:teuthology.orchestra.run.smithi136.stdout: "segex": 0, 2024-09-11T23:40:11.545 INFO:teuthology.orchestra.run.smithi136.stdout: "segtrm": 0, 2024-09-11T23:40:11.545 INFO:teuthology.orchestra.run.smithi136.stdout: "seg": 2, 2024-09-11T23:40:11.545 INFO:teuthology.orchestra.run.smithi136.stdout: "segmjr": 92, 2024-09-11T23:40:11.545 INFO:teuthology.orchestra.run.smithi136.stdout: "segexg": 0, 2024-09-11T23:40:11.545 INFO:teuthology.orchestra.run.smithi136.stdout: "segexd": 0, 2024-09-11T23:40:11.545 INFO:teuthology.orchestra.run.smithi136.stdout: "expos": 274743605, 2024-09-11T23:40:11.545 INFO:teuthology.orchestra.run.smithi136.stdout: "wrpos": 274766115, 2024-09-11T23:40:11.545 INFO:teuthology.orchestra.run.smithi136.stdout: "rdpos": 274766055, 2024-09-11T23:40:11.545 INFO:teuthology.orchestra.run.smithi136.stdout: "jlat": { 2024-09-11T23:40:11.545 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-11T23:40:11.545 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-11T23:40:11.546 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-11T23:40:11.546 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:11.546 INFO:teuthology.orchestra.run.smithi136.stdout: "replayed": 99784 2024-09-11T23:40:11.546 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:11.546 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_mem": { 2024-09-11T23:40:11.546 INFO:teuthology.orchestra.run.smithi136.stdout: "ino": 17548, 2024-09-11T23:40:11.546 INFO:teuthology.orchestra.run.smithi136.stdout: "ino+": 56851, 2024-09-11T23:40:11.546 INFO:teuthology.orchestra.run.smithi136.stdout: "ino-": 39303, 2024-09-11T23:40:11.546 INFO:teuthology.orchestra.run.smithi136.stdout: "dir": 562, 2024-09-11T23:40:11.546 INFO:teuthology.orchestra.run.smithi136.stdout: "dir+": 1575, 2024-09-11T23:40:11.546 INFO:teuthology.orchestra.run.smithi136.stdout: "dir-": 1013, 2024-09-11T23:40:11.546 INFO:teuthology.orchestra.run.smithi136.stdout: "dn": 49029, 2024-09-11T23:40:11.546 INFO:teuthology.orchestra.run.smithi136.stdout: "dn+": 92006, 2024-09-11T23:40:11.546 INFO:teuthology.orchestra.run.smithi136.stdout: "dn-": 42977, 2024-09-11T23:40:11.547 INFO:teuthology.orchestra.run.smithi136.stdout: "cap": 0, 2024-09-11T23:40:11.547 INFO:teuthology.orchestra.run.smithi136.stdout: "cap+": 0, 2024-09-11T23:40:11.547 INFO:teuthology.orchestra.run.smithi136.stdout: "cap-": 0, 2024-09-11T23:40:11.547 INFO:teuthology.orchestra.run.smithi136.stdout: "rss": 162872, 2024-09-11T23:40:11.547 INFO:teuthology.orchestra.run.smithi136.stdout: "heap": 215328 2024-09-11T23:40:11.547 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:11.548 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_server": { 2024-09-11T23:40:11.548 INFO:teuthology.orchestra.run.smithi136.stdout: "dispatch_client_request": 0, 2024-09-11T23:40:11.548 INFO:teuthology.orchestra.run.smithi136.stdout: "dispatch_server_request": 0, 2024-09-11T23:40:11.548 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_client_request": 0, 2024-09-11T23:40:11.549 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_client_session": 0, 2024-09-11T23:40:11.549 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_peer_request": 0, 2024-09-11T23:40:11.549 INFO:teuthology.orchestra.run.smithi136.stdout: "req_create_latency": { 2024-09-11T23:40:11.549 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-11T23:40:11.549 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-11T23:40:11.549 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-11T23:40:11.549 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:11.549 INFO:teuthology.orchestra.run.smithi136.stdout: "req_getattr_latency": { 2024-09-11T23:40:11.549 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-11T23:40:11.549 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-11T23:40:11.549 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-11T23:40:11.549 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:11.549 INFO:teuthology.orchestra.run.smithi136.stdout: "req_getfilelock_latency": { 2024-09-11T23:40:11.549 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-11T23:40:11.549 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-11T23:40:11.550 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-11T23:40:11.550 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:11.550 INFO:teuthology.orchestra.run.smithi136.stdout: "req_link_latency": { 2024-09-11T23:40:11.550 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-11T23:40:11.550 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-11T23:40:11.550 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-11T23:40:11.550 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:11.550 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lookup_latency": { 2024-09-11T23:40:11.550 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-11T23:40:11.550 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-11T23:40:11.550 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-11T23:40:11.550 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:11.550 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lookuphash_latency": { 2024-09-11T23:40:11.550 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-11T23:40:11.550 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-11T23:40:11.550 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-11T23:40:11.551 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:11.551 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lookupino_latency": { 2024-09-11T23:40:11.551 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-11T23:40:11.551 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-11T23:40:11.551 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-11T23:40:11.551 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:11.551 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lookupname_latency": { 2024-09-11T23:40:11.551 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-11T23:40:11.551 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-11T23:40:11.551 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-11T23:40:11.551 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:11.551 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lookupparent_latency": { 2024-09-11T23:40:11.551 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-11T23:40:11.552 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-11T23:40:11.552 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-11T23:40:11.552 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:11.552 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lookupsnap_latency": { 2024-09-11T23:40:11.552 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-11T23:40:11.552 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-11T23:40:11.552 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-11T23:40:11.552 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:11.552 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lssnap_latency": { 2024-09-11T23:40:11.552 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-11T23:40:11.552 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-11T23:40:11.552 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-11T23:40:11.552 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:11.553 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:40:11 smithi076 ceph-mon[35296]: pgmap v688: 97 pgs: 15 active+clean+snaptrim_wait, 9 active+clean+snaptrim, 73 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 33 KiB/s wr, 13 op/s 2024-09-11T23:40:11.553 INFO:teuthology.orchestra.run.smithi136.stdout: "req_mkdir_latency": { 2024-09-11T23:40:11.553 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-11T23:40:11.554 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-11T23:40:11.554 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-11T23:40:11.554 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:11.554 INFO:teuthology.orchestra.run.smithi136.stdout: "req_mknod_latency": { 2024-09-11T23:40:11.554 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-11T23:40:11.554 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-11T23:40:11.554 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-11T23:40:11.554 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:11.554 INFO:teuthology.orchestra.run.smithi136.stdout: "req_mksnap_latency": { 2024-09-11T23:40:11.554 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-11T23:40:11.554 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-11T23:40:11.554 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-11T23:40:11.554 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:11.554 INFO:teuthology.orchestra.run.smithi136.stdout: "req_open_latency": { 2024-09-11T23:40:11.554 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-11T23:40:11.555 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-11T23:40:11.555 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-11T23:40:11.555 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:11.555 INFO:teuthology.orchestra.run.smithi136.stdout: "req_readdir_latency": { 2024-09-11T23:40:11.555 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-11T23:40:11.555 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-11T23:40:11.555 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-11T23:40:11.555 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:11.555 INFO:teuthology.orchestra.run.smithi136.stdout: "req_rename_latency": { 2024-09-11T23:40:11.555 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-11T23:40:11.555 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-11T23:40:11.555 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-11T23:40:11.555 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:11.555 INFO:teuthology.orchestra.run.smithi136.stdout: "req_renamesnap_latency": { 2024-09-11T23:40:11.556 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-11T23:40:11.556 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-11T23:40:11.556 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-11T23:40:11.556 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:11.556 INFO:teuthology.orchestra.run.smithi136.stdout: "req_snapdiff_latency": { 2024-09-11T23:40:11.556 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-11T23:40:11.556 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-11T23:40:11.556 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-11T23:40:11.556 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:11.556 INFO:teuthology.orchestra.run.smithi136.stdout: "req_rmdir_latency": { 2024-09-11T23:40:11.556 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-11T23:40:11.556 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-11T23:40:11.556 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-11T23:40:11.556 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:11.556 INFO:teuthology.orchestra.run.smithi136.stdout: "req_rmsnap_latency": { 2024-09-11T23:40:11.557 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-11T23:40:11.557 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-11T23:40:11.557 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-11T23:40:11.557 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:11.557 INFO:teuthology.orchestra.run.smithi136.stdout: "req_rmxattr_latency": { 2024-09-11T23:40:11.557 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-11T23:40:11.557 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-11T23:40:11.557 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-11T23:40:11.557 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:11.557 INFO:teuthology.orchestra.run.smithi136.stdout: "req_setattr_latency": { 2024-09-11T23:40:11.557 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-11T23:40:11.557 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-11T23:40:11.557 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-11T23:40:11.557 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:11.558 INFO:teuthology.orchestra.run.smithi136.stdout: "req_setdirlayout_latency": { 2024-09-11T23:40:11.558 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-11T23:40:11.558 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-11T23:40:11.558 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-11T23:40:11.558 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:11.558 INFO:teuthology.orchestra.run.smithi136.stdout: "req_setfilelock_latency": { 2024-09-11T23:40:11.558 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-11T23:40:11.558 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-11T23:40:11.558 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-11T23:40:11.558 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:11.558 INFO:teuthology.orchestra.run.smithi136.stdout: "req_setlayout_latency": { 2024-09-11T23:40:11.558 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-11T23:40:11.558 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-11T23:40:11.558 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-11T23:40:11.558 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:11.559 INFO:teuthology.orchestra.run.smithi136.stdout: "req_setxattr_latency": { 2024-09-11T23:40:11.559 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-11T23:40:11.559 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-11T23:40:11.559 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-11T23:40:11.559 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:11.559 INFO:teuthology.orchestra.run.smithi136.stdout: "req_symlink_latency": { 2024-09-11T23:40:11.559 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-11T23:40:11.559 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-11T23:40:11.559 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-11T23:40:11.559 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:11.559 INFO:teuthology.orchestra.run.smithi136.stdout: "req_unlink_latency": { 2024-09-11T23:40:11.559 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-11T23:40:11.559 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-11T23:40:11.560 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-11T23:40:11.560 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:11.560 INFO:teuthology.orchestra.run.smithi136.stdout: "cap_revoke_eviction": 0, 2024-09-11T23:40:11.560 INFO:teuthology.orchestra.run.smithi136.stdout: "cap_acquisition_throttle": 0, 2024-09-11T23:40:11.560 INFO:teuthology.orchestra.run.smithi136.stdout: "req_getvxattr_latency": { 2024-09-11T23:40:11.560 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-11T23:40:11.560 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-11T23:40:11.560 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-11T23:40:11.560 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-11T23:40:11.560 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:11.560 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_sessions": { 2024-09-11T23:40:11.560 INFO:teuthology.orchestra.run.smithi136.stdout: "session_count": 2, 2024-09-11T23:40:11.560 INFO:teuthology.orchestra.run.smithi136.stdout: "session_add": 5, 2024-09-11T23:40:11.560 INFO:teuthology.orchestra.run.smithi136.stdout: "session_remove": 3, 2024-09-11T23:40:11.561 INFO:teuthology.orchestra.run.smithi136.stdout: "sessions_open": 4, 2024-09-11T23:40:11.561 INFO:teuthology.orchestra.run.smithi136.stdout: "sessions_stale": 0, 2024-09-11T23:40:11.561 INFO:teuthology.orchestra.run.smithi136.stdout: "total_load": 0, 2024-09-11T23:40:11.561 INFO:teuthology.orchestra.run.smithi136.stdout: "average_load": 0, 2024-09-11T23:40:11.561 INFO:teuthology.orchestra.run.smithi136.stdout: "avg_session_uptime": 3792, 2024-09-11T23:40:11.561 INFO:teuthology.orchestra.run.smithi136.stdout: "mdthresh_evicted": 0 2024-09-11T23:40:11.561 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:11.561 INFO:teuthology.orchestra.run.smithi136.stdout: "mempool": { 2024-09-11T23:40:11.561 INFO:teuthology.orchestra.run.smithi136.stdout: "bloom_filter_bytes": 0, 2024-09-11T23:40:11.561 INFO:teuthology.orchestra.run.smithi136.stdout: "bloom_filter_items": 0, 2024-09-11T23:40:11.561 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_alloc_bytes": 0, 2024-09-11T23:40:11.561 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_alloc_items": 0, 2024-09-11T23:40:11.561 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_data_bytes": 0, 2024-09-11T23:40:11.561 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_data_items": 0, 2024-09-11T23:40:11.562 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-11T23:40:11.562 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_onode_items": 0, 2024-09-11T23:40:11.562 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-11T23:40:11.562 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_meta_items": 0, 2024-09-11T23:40:11.562 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_other_bytes": 0, 2024-09-11T23:40:11.562 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_other_items": 0, 2024-09-11T23:40:11.562 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-11T23:40:11.562 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_buffer_items": 0, 2024-09-11T23:40:11.562 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_extent_bytes": 0, 2024-09-11T23:40:11.562 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_extent_items": 0, 2024-09-11T23:40:11.562 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_blob_bytes": 0, 2024-09-11T23:40:11.562 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_blob_items": 0, 2024-09-11T23:40:11.562 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-11T23:40:11.563 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_shared_blob_items": 0, 2024-09-11T23:40:11.563 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-11T23:40:11.563 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_inline_bl_items": 0, 2024-09-11T23:40:11.563 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_fsck_bytes": 0, 2024-09-11T23:40:11.563 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_fsck_items": 0, 2024-09-11T23:40:11.563 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_txc_bytes": 0, 2024-09-11T23:40:11.563 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_txc_items": 0, 2024-09-11T23:40:11.563 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-11T23:40:11.563 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_writing_deferred_items": 0, 2024-09-11T23:40:11.563 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_writing_bytes": 0, 2024-09-11T23:40:11.563 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_writing_items": 0, 2024-09-11T23:40:11.563 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_bytes": 0, 2024-09-11T23:40:11.563 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_items": 0, 2024-09-11T23:40:11.563 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_file_reader_bytes": 0, 2024-09-11T23:40:11.564 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_file_reader_items": 0, 2024-09-11T23:40:11.564 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_file_writer_bytes": 0, 2024-09-11T23:40:11.564 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_file_writer_items": 0, 2024-09-11T23:40:11.564 INFO:teuthology.orchestra.run.smithi136.stdout: "buffer_anon_bytes": 892772, 2024-09-11T23:40:11.564 INFO:teuthology.orchestra.run.smithi136.stdout: "buffer_anon_items": 17718, 2024-09-11T23:40:11.564 INFO:teuthology.orchestra.run.smithi136.stdout: "buffer_meta_bytes": 0, 2024-09-11T23:40:11.564 INFO:teuthology.orchestra.run.smithi136.stdout: "buffer_meta_items": 0, 2024-09-11T23:40:11.564 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_bytes": 0, 2024-09-11T23:40:11.564 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_items": 0, 2024-09-11T23:40:11.564 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_mapbl_bytes": 0, 2024-09-11T23:40:11.564 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_mapbl_items": 0, 2024-09-11T23:40:11.564 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_pglog_bytes": 0, 2024-09-11T23:40:11.564 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_pglog_items": 0, 2024-09-11T23:40:11.564 INFO:teuthology.orchestra.run.smithi136.stdout: "osdmap_bytes": 8224, 2024-09-11T23:40:11.564 INFO:teuthology.orchestra.run.smithi136.stdout: "osdmap_items": 181, 2024-09-11T23:40:11.565 INFO:teuthology.orchestra.run.smithi136.stdout: "osdmap_mapping_bytes": 0, 2024-09-11T23:40:11.565 INFO:teuthology.orchestra.run.smithi136.stdout: "osdmap_mapping_items": 0, 2024-09-11T23:40:11.565 INFO:teuthology.orchestra.run.smithi136.stdout: "pgmap_bytes": 0, 2024-09-11T23:40:11.565 INFO:teuthology.orchestra.run.smithi136.stdout: "pgmap_items": 0, 2024-09-11T23:40:11.565 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_co_bytes": 75559212, 2024-09-11T23:40:11.565 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_co_items": 676077, 2024-09-11T23:40:11.565 INFO:teuthology.orchestra.run.smithi136.stdout: "unittest_1_bytes": 0, 2024-09-11T23:40:11.565 INFO:teuthology.orchestra.run.smithi136.stdout: "unittest_1_items": 0, 2024-09-11T23:40:11.565 INFO:teuthology.orchestra.run.smithi136.stdout: "unittest_2_bytes": 0, 2024-09-11T23:40:11.565 INFO:teuthology.orchestra.run.smithi136.stdout: "unittest_2_items": 0 2024-09-11T23:40:11.565 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:11.565 INFO:teuthology.orchestra.run.smithi136.stdout: "objecter": { 2024-09-11T23:40:11.565 INFO:teuthology.orchestra.run.smithi136.stdout: "op_active": 0, 2024-09-11T23:40:11.565 INFO:teuthology.orchestra.run.smithi136.stdout: "op_laggy": 0, 2024-09-11T23:40:11.565 INFO:teuthology.orchestra.run.smithi136.stdout: "op_send": 4875, 2024-09-11T23:40:11.566 INFO:teuthology.orchestra.run.smithi136.stdout: "op_send_bytes": 26, 2024-09-11T23:40:11.566 INFO:teuthology.orchestra.run.smithi136.stdout: "op_resend": 4, 2024-09-11T23:40:11.566 INFO:teuthology.orchestra.run.smithi136.stdout: "op_reply": 4875, 2024-09-11T23:40:11.566 INFO:teuthology.orchestra.run.smithi136.stdout: "op_latency": { 2024-09-11T23:40:11.566 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 4875, 2024-09-11T23:40:11.566 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 23.492919871, 2024-09-11T23:40:11.566 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.004819060 2024-09-11T23:40:11.566 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:11.566 INFO:teuthology.orchestra.run.smithi136.stdout: "op_inflight": 0, 2024-09-11T23:40:11.566 INFO:teuthology.orchestra.run.smithi136.stdout: "oplen_avg": { 2024-09-11T23:40:11.566 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 4875, 2024-09-11T23:40:11.566 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 4876 2024-09-11T23:40:11.566 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:11.567 INFO:teuthology.orchestra.run.smithi136.stdout: "op": 4875, 2024-09-11T23:40:11.567 INFO:teuthology.orchestra.run.smithi136.stdout: "op_r": 4875, 2024-09-11T23:40:11.567 INFO:teuthology.orchestra.run.smithi136.stdout: "op_w": 0, 2024-09-11T23:40:11.567 INFO:teuthology.orchestra.run.smithi136.stdout: "op_rmw": 0, 2024-09-11T23:40:11.567 INFO:teuthology.orchestra.run.smithi136.stdout: "op_pg": 0, 2024-09-11T23:40:11.567 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_stat": 2874, 2024-09-11T23:40:11.567 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_create": 0, 2024-09-11T23:40:11.567 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_read": 1998, 2024-09-11T23:40:11.567 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_write": 0, 2024-09-11T23:40:11.567 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_writefull": 0, 2024-09-11T23:40:11.567 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_writesame": 0, 2024-09-11T23:40:11.567 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_append": 0, 2024-09-11T23:40:11.567 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_zero": 0, 2024-09-11T23:40:11.567 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_truncate": 0, 2024-09-11T23:40:11.568 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_delete": 0, 2024-09-11T23:40:11.568 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_mapext": 0, 2024-09-11T23:40:11.568 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_sparse_read": 0, 2024-09-11T23:40:11.568 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_clonerange": 0, 2024-09-11T23:40:11.568 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_getxattr": 2, 2024-09-11T23:40:11.568 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_setxattr": 0, 2024-09-11T23:40:11.568 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_cmpxattr": 0, 2024-09-11T23:40:11.568 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_rmxattr": 0, 2024-09-11T23:40:11.568 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_resetxattrs": 0, 2024-09-11T23:40:11.568 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_call": 0, 2024-09-11T23:40:11.568 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_watch": 0, 2024-09-11T23:40:11.569 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_notify": 0, 2024-09-11T23:40:11.569 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_src_cmpxattr": 0, 2024-09-11T23:40:11.569 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_pgls": 0, 2024-09-11T23:40:11.569 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_pgls_filter": 0, 2024-09-11T23:40:11.569 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_other": 0, 2024-09-11T23:40:11.569 INFO:teuthology.orchestra.run.smithi136.stdout: "linger_active": 0, 2024-09-11T23:40:11.569 INFO:teuthology.orchestra.run.smithi136.stdout: "linger_send": 0, 2024-09-11T23:40:11.569 INFO:teuthology.orchestra.run.smithi136.stdout: "linger_resend": 0, 2024-09-11T23:40:11.569 INFO:teuthology.orchestra.run.smithi136.stdout: "linger_ping": 0, 2024-09-11T23:40:11.569 INFO:teuthology.orchestra.run.smithi136.stdout: "poolop_active": 0, 2024-09-11T23:40:11.569 INFO:teuthology.orchestra.run.smithi136.stdout: "poolop_send": 0, 2024-09-11T23:40:11.569 INFO:teuthology.orchestra.run.smithi136.stdout: "poolop_resend": 0, 2024-09-11T23:40:11.570 INFO:teuthology.orchestra.run.smithi136.stdout: "poolstat_active": 0, 2024-09-11T23:40:11.570 INFO:teuthology.orchestra.run.smithi136.stdout: "poolstat_send": 0, 2024-09-11T23:40:11.570 INFO:teuthology.orchestra.run.smithi136.stdout: "poolstat_resend": 0, 2024-09-11T23:40:11.570 INFO:teuthology.orchestra.run.smithi136.stdout: "statfs_active": 0, 2024-09-11T23:40:11.570 INFO:teuthology.orchestra.run.smithi136.stdout: "statfs_send": 0, 2024-09-11T23:40:11.570 INFO:teuthology.orchestra.run.smithi136.stdout: "statfs_resend": 0, 2024-09-11T23:40:11.570 INFO:teuthology.orchestra.run.smithi136.stdout: "command_active": 0, 2024-09-11T23:40:11.570 INFO:teuthology.orchestra.run.smithi136.stdout: "command_send": 0, 2024-09-11T23:40:11.570 INFO:teuthology.orchestra.run.smithi136.stdout: "command_resend": 0, 2024-09-11T23:40:11.570 INFO:teuthology.orchestra.run.smithi136.stdout: "map_epoch": 117, 2024-09-11T23:40:11.570 INFO:teuthology.orchestra.run.smithi136.stdout: "map_full": 0, 2024-09-11T23:40:11.570 INFO:teuthology.orchestra.run.smithi136.stdout: "map_inc": 32, 2024-09-11T23:40:11.570 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_sessions": 11, 2024-09-11T23:40:11.570 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_session_open": 11, 2024-09-11T23:40:11.571 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_session_close": 0, 2024-09-11T23:40:11.571 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_laggy": 0, 2024-09-11T23:40:11.571 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_wr": 0, 2024-09-11T23:40:11.571 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_rd": 2, 2024-09-11T23:40:11.571 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_del": 0 2024-09-11T23:40:11.571 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:11.571 INFO:teuthology.orchestra.run.smithi136.stdout: "oft": { 2024-09-11T23:40:11.571 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_total_objs": 0, 2024-09-11T23:40:11.571 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_total_kv_pairs": 0, 2024-09-11T23:40:11.572 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_total_updates": 0, 2024-09-11T23:40:11.572 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_total_removes": 0 2024-09-11T23:40:11.572 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:11.572 INFO:teuthology.orchestra.run.smithi136.stdout: "osd-slow-ops": { 2024-09-11T23:40:11.572 INFO:teuthology.orchestra.run.smithi136.stdout: "slow_ops_count": 0 2024-09-11T23:40:11.572 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:11.572 INFO:teuthology.orchestra.run.smithi136.stdout: "purge_queue": { 2024-09-11T23:40:11.572 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_executing_ops": 0, 2024-09-11T23:40:11.572 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_executing_ops_high_water": 0, 2024-09-11T23:40:11.572 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_executing": 0, 2024-09-11T23:40:11.572 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_executing_high_water": 0, 2024-09-11T23:40:11.572 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_executed_ops": 0, 2024-09-11T23:40:11.572 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_executed": 0, 2024-09-11T23:40:11.572 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_item_in_journal": 0 2024-09-11T23:40:11.572 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:11.573 INFO:teuthology.orchestra.run.smithi136.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-11T23:40:11.573 INFO:teuthology.orchestra.run.smithi136.stdout: "val": 0, 2024-09-11T23:40:11.573 INFO:teuthology.orchestra.run.smithi136.stdout: "max": 104857600, 2024-09-11T23:40:11.573 INFO:teuthology.orchestra.run.smithi136.stdout: "get_started": 0, 2024-09-11T23:40:11.573 INFO:teuthology.orchestra.run.smithi136.stdout: "get": 5377, 2024-09-11T23:40:11.573 INFO:teuthology.orchestra.run.smithi136.stdout: "get_sum": 273551965, 2024-09-11T23:40:11.573 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_fail": 0, 2024-09-11T23:40:11.573 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_success": 5377, 2024-09-11T23:40:11.573 INFO:teuthology.orchestra.run.smithi136.stdout: "take": 0, 2024-09-11T23:40:11.573 INFO:teuthology.orchestra.run.smithi136.stdout: "take_sum": 0, 2024-09-11T23:40:11.573 INFO:teuthology.orchestra.run.smithi136.stdout: "put": 5377, 2024-09-11T23:40:11.574 INFO:teuthology.orchestra.run.smithi136.stdout: "put_sum": 273551965, 2024-09-11T23:40:11.574 INFO:teuthology.orchestra.run.smithi136.stdout: "wait": { 2024-09-11T23:40:11.574 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-11T23:40:11.574 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-11T23:40:11.574 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-11T23:40:11.574 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-11T23:40:11.574 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:11.574 INFO:teuthology.orchestra.run.smithi136.stdout: "throttle-objecter_bytes": { 2024-09-11T23:40:11.574 INFO:teuthology.orchestra.run.smithi136.stdout: "val": 0, 2024-09-11T23:40:11.574 INFO:teuthology.orchestra.run.smithi136.stdout: "max": 104857600, 2024-09-11T23:40:11.574 INFO:teuthology.orchestra.run.smithi136.stdout: "get_started": 0, 2024-09-11T23:40:11.574 INFO:teuthology.orchestra.run.smithi136.stdout: "get": 0, 2024-09-11T23:40:11.574 INFO:teuthology.orchestra.run.smithi136.stdout: "get_sum": 0, 2024-09-11T23:40:11.574 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_fail": 0, 2024-09-11T23:40:11.575 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_success": 0, 2024-09-11T23:40:11.575 INFO:teuthology.orchestra.run.smithi136.stdout: "take": 4875, 2024-09-11T23:40:11.575 INFO:teuthology.orchestra.run.smithi136.stdout: "take_sum": 270599599, 2024-09-11T23:40:11.575 INFO:teuthology.orchestra.run.smithi136.stdout: "put": 4875, 2024-09-11T23:40:11.575 INFO:teuthology.orchestra.run.smithi136.stdout: "put_sum": 270599599, 2024-09-11T23:40:11.575 INFO:teuthology.orchestra.run.smithi136.stdout: "wait": { 2024-09-11T23:40:11.575 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-11T23:40:11.575 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-11T23:40:11.575 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-11T23:40:11.576 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-11T23:40:11.576 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:11.576 INFO:teuthology.orchestra.run.smithi136.stdout: "throttle-objecter_ops": { 2024-09-11T23:40:11.576 INFO:teuthology.orchestra.run.smithi136.stdout: "val": 0, 2024-09-11T23:40:11.576 INFO:teuthology.orchestra.run.smithi136.stdout: "max": 1024, 2024-09-11T23:40:11.576 INFO:teuthology.orchestra.run.smithi136.stdout: "get_started": 0, 2024-09-11T23:40:11.576 INFO:teuthology.orchestra.run.smithi136.stdout: "get": 0, 2024-09-11T23:40:11.576 INFO:teuthology.orchestra.run.smithi136.stdout: "get_sum": 0, 2024-09-11T23:40:11.576 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_fail": 0, 2024-09-11T23:40:11.576 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_success": 0, 2024-09-11T23:40:11.576 INFO:teuthology.orchestra.run.smithi136.stdout: "take": 4875, 2024-09-11T23:40:11.577 INFO:teuthology.orchestra.run.smithi136.stdout: "take_sum": 4875, 2024-09-11T23:40:11.577 INFO:teuthology.orchestra.run.smithi136.stdout: "put": 4875, 2024-09-11T23:40:11.577 INFO:teuthology.orchestra.run.smithi136.stdout: "put_sum": 4875, 2024-09-11T23:40:11.577 INFO:teuthology.orchestra.run.smithi136.stdout: "wait": { 2024-09-11T23:40:11.577 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-11T23:40:11.577 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-11T23:40:11.577 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-11T23:40:11.577 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-11T23:40:11.577 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:11.577 INFO:teuthology.orchestra.run.smithi136.stdout: "throttle-write_buf_throttle": { 2024-09-11T23:40:11.577 INFO:teuthology.orchestra.run.smithi136.stdout: "val": 0, 2024-09-11T23:40:11.578 INFO:teuthology.orchestra.run.smithi136.stdout: "max": 3758096384, 2024-09-11T23:40:11.578 INFO:teuthology.orchestra.run.smithi136.stdout: "get_started": 0, 2024-09-11T23:40:11.578 INFO:teuthology.orchestra.run.smithi136.stdout: "get": 0, 2024-09-11T23:40:11.578 INFO:teuthology.orchestra.run.smithi136.stdout: "get_sum": 0, 2024-09-11T23:40:11.578 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_fail": 0, 2024-09-11T23:40:11.578 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_success": 0, 2024-09-11T23:40:11.578 INFO:teuthology.orchestra.run.smithi136.stdout: "take": 0, 2024-09-11T23:40:11.578 INFO:teuthology.orchestra.run.smithi136.stdout: "take_sum": 0, 2024-09-11T23:40:11.578 INFO:teuthology.orchestra.run.smithi136.stdout: "put": 0, 2024-09-11T23:40:11.578 INFO:teuthology.orchestra.run.smithi136.stdout: "put_sum": 0, 2024-09-11T23:40:11.579 INFO:teuthology.orchestra.run.smithi136.stdout: "wait": { 2024-09-11T23:40:11.579 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-11T23:40:11.579 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-11T23:40:11.579 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-11T23:40:11.579 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-11T23:40:11.579 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:11.579 INFO:teuthology.orchestra.run.smithi136.stdout: "throttle-write_buf_throttle-0x55be04f58b40": { 2024-09-11T23:40:11.579 INFO:teuthology.orchestra.run.smithi136.stdout: "val": 0, 2024-09-11T23:40:11.579 INFO:teuthology.orchestra.run.smithi136.stdout: "max": 3758096384, 2024-09-11T23:40:11.579 INFO:teuthology.orchestra.run.smithi136.stdout: "get_started": 0, 2024-09-11T23:40:11.579 INFO:teuthology.orchestra.run.smithi136.stdout: "get": 0, 2024-09-11T23:40:11.579 INFO:teuthology.orchestra.run.smithi136.stdout: "get_sum": 0, 2024-09-11T23:40:11.579 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_fail": 0, 2024-09-11T23:40:11.580 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_success": 0, 2024-09-11T23:40:11.580 INFO:teuthology.orchestra.run.smithi136.stdout: "take": 0, 2024-09-11T23:40:11.580 INFO:teuthology.orchestra.run.smithi136.stdout: "take_sum": 0, 2024-09-11T23:40:11.580 INFO:teuthology.orchestra.run.smithi136.stdout: "put": 0, 2024-09-11T23:40:11.580 INFO:teuthology.orchestra.run.smithi136.stdout: "put_sum": 0, 2024-09-11T23:40:11.580 INFO:teuthology.orchestra.run.smithi136.stdout: "wait": { 2024-09-11T23:40:11.580 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-11T23:40:11.580 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-11T23:40:11.580 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-11T23:40:11.580 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-11T23:40:11.580 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-11T23:40:11.580 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-11T23:40:13.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:40:13 smithi136 ceph-mon[35121]: pgmap v689: 97 pgs: 36 active+clean+snaptrim_wait, 21 active+clean+snaptrim, 40 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 4.8 KiB/s wr, 11 op/s 2024-09-11T23:40:14.000 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:40:13 smithi026 ceph-mon[31172]: pgmap v689: 97 pgs: 36 active+clean+snaptrim_wait, 21 active+clean+snaptrim, 40 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 4.8 KiB/s wr, 11 op/s 2024-09-11T23:40:14.027 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:40:13 smithi076 ceph-mon[35296]: pgmap v689: 97 pgs: 36 active+clean+snaptrim_wait, 21 active+clean+snaptrim, 40 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 4.8 KiB/s wr, 11 op/s 2024-09-11T23:40:14.236 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-09-11T23:40:14.236 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-09-11T23:40:14.236 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update=0 2024-09-11T23:40:14.236 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update_receipt=0 2024-09-11T23:40:14.236 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=6 2024-09-11T23:40:14.236 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2024-09-11T23:40:14.236 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2024-09-11T23:40:14.237 INFO:tasks.check_counter:Daemon mds.f mds.dir_split=0 2024-09-11T23:40:14.237 DEBUG:tasks.check_counter:Getting stats from i 2024-09-11T23:40:14.237 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph daemon mds.i perf dump 2024-09-11T23:40:14.737 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T23:40:15.433 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-11T23:40:15.433 INFO:teuthology.orchestra.run.smithi136.stdout: "AsyncMessenger::Worker-0": { 2024-09-11T23:40:15.433 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_messages": 61893, 2024-09-11T23:40:15.434 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages": 67131, 2024-09-11T23:40:15.434 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_bytes": 51020669, 2024-09-11T23:40:15.434 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_bytes": 347838646, 2024-09-11T23:40:15.434 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_created_connections": 17, 2024-09-11T23:40:15.434 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_active_connections": 6, 2024-09-11T23:40:15.434 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_total_time": 5.936856079, 2024-09-11T23:40:15.434 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_send_time": 2.331556208, 2024-09-11T23:40:15.434 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_recv_time": 2.053442062, 2024-09-11T23:40:15.434 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_fast_dispatch_time": 1.766082133, 2024-09-11T23:40:15.434 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T23:40:15.434 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 67127, 2024-09-11T23:40:15.434 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 11.527341348, 2024-09-11T23:40:15.435 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000171724 2024-09-11T23:40:15.435 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:15.435 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_handle_ack_lat": { 2024-09-11T23:40:15.435 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 76634, 2024-09-11T23:40:15.435 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.013258066, 2024-09-11T23:40:15.435 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000173 2024-09-11T23:40:15.435 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:15.435 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-11T23:40:15.435 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_encrypted_bytes": 648096 2024-09-11T23:40:15.435 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:15.435 INFO:teuthology.orchestra.run.smithi136.stdout: "AsyncMessenger::Worker-1": { 2024-09-11T23:40:15.435 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_messages": 67889, 2024-09-11T23:40:15.435 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages": 67866, 2024-09-11T23:40:15.436 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_bytes": 42816717, 2024-09-11T23:40:15.436 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_bytes": 407694434, 2024-09-11T23:40:15.436 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_created_connections": 13, 2024-09-11T23:40:15.436 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_active_connections": 8, 2024-09-11T23:40:15.436 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_total_time": 5.959697487, 2024-09-11T23:40:15.436 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_send_time": 2.291597889, 2024-09-11T23:40:15.436 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_recv_time": 2.335148199, 2024-09-11T23:40:15.436 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_fast_dispatch_time": 1.688708699, 2024-09-11T23:40:15.436 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T23:40:15.436 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 67864, 2024-09-11T23:40:15.436 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 10.742915589, 2024-09-11T23:40:15.436 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000158300 2024-09-11T23:40:15.437 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:15.437 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_handle_ack_lat": { 2024-09-11T23:40:15.437 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 81977, 2024-09-11T23:40:15.437 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.017378775, 2024-09-11T23:40:15.437 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000211 2024-09-11T23:40:15.437 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:15.437 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_encrypted_bytes": 2151264, 2024-09-11T23:40:15.437 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_encrypted_bytes": 304704 2024-09-11T23:40:15.437 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:15.437 INFO:teuthology.orchestra.run.smithi136.stdout: "AsyncMessenger::Worker-2": { 2024-09-11T23:40:15.437 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_messages": 518892, 2024-09-11T23:40:15.437 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages": 774053, 2024-09-11T23:40:15.437 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_bytes": 218742648, 2024-09-11T23:40:15.438 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_bytes": 712630022, 2024-09-11T23:40:15.438 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_created_connections": 17, 2024-09-11T23:40:15.438 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_active_connections": 5, 2024-09-11T23:40:15.438 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_total_time": 34.919371231, 2024-09-11T23:40:15.438 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_send_time": 15.894266123, 2024-09-11T23:40:15.438 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_recv_time": 28.767234494, 2024-09-11T23:40:15.438 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_fast_dispatch_time": 1.160051177, 2024-09-11T23:40:15.438 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T23:40:15.438 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 774054, 2024-09-11T23:40:15.438 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 80.608642825, 2024-09-11T23:40:15.438 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000104138 2024-09-11T23:40:15.438 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:15.438 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_handle_ack_lat": { 2024-09-11T23:40:15.439 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 1089396, 2024-09-11T23:40:15.439 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.166596838, 2024-09-11T23:40:15.439 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000152 2024-09-11T23:40:15.439 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:15.439 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_encrypted_bytes": 155893520, 2024-09-11T23:40:15.439 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_encrypted_bytes": 391951712 2024-09-11T23:40:15.439 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:15.439 INFO:teuthology.orchestra.run.smithi136.stdout: "cct": { 2024-09-11T23:40:15.439 INFO:teuthology.orchestra.run.smithi136.stdout: "total_workers": 1, 2024-09-11T23:40:15.439 INFO:teuthology.orchestra.run.smithi136.stdout: "unhealthy_workers": 0 2024-09-11T23:40:15.439 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:15.439 INFO:teuthology.orchestra.run.smithi136.stdout: "finisher-MDSRank": { 2024-09-11T23:40:15.439 INFO:teuthology.orchestra.run.smithi136.stdout: "queue_len": 0, 2024-09-11T23:40:15.440 INFO:teuthology.orchestra.run.smithi136.stdout: "complete_latency": { 2024-09-11T23:40:15.440 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 17517, 2024-09-11T23:40:15.440 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 186.450403986, 2024-09-11T23:40:15.440 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.010643968 2024-09-11T23:40:15.440 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-11T23:40:15.440 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:15.440 INFO:teuthology.orchestra.run.smithi136.stdout: "finisher-PurgeQueue": { 2024-09-11T23:40:15.440 INFO:teuthology.orchestra.run.smithi136.stdout: "queue_len": 0, 2024-09-11T23:40:15.440 INFO:teuthology.orchestra.run.smithi136.stdout: "complete_latency": { 2024-09-11T23:40:15.440 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 81321, 2024-09-11T23:40:15.440 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 13.348869926, 2024-09-11T23:40:15.440 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000164150 2024-09-11T23:40:15.440 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-11T23:40:15.440 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:15.441 INFO:teuthology.orchestra.run.smithi136.stdout: "mds": { 2024-09-11T23:40:15.441 INFO:teuthology.orchestra.run.smithi136.stdout: "request": 271011, 2024-09-11T23:40:15.441 INFO:teuthology.orchestra.run.smithi136.stdout: "reply": 253712, 2024-09-11T23:40:15.441 INFO:teuthology.orchestra.run.smithi136.stdout: "reply_latency": { 2024-09-11T23:40:15.441 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 253712, 2024-09-11T23:40:15.441 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 6084.342547457, 2024-09-11T23:40:15.441 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.023981295 2024-09-11T23:40:15.441 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:15.441 INFO:teuthology.orchestra.run.smithi136.stdout: "slow_reply": 0, 2024-09-11T23:40:15.441 INFO:teuthology.orchestra.run.smithi136.stdout: "forward": 17254, 2024-09-11T23:40:15.441 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_fetch_complete": 302, 2024-09-11T23:40:15.441 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_fetch_keys": 0, 2024-09-11T23:40:15.441 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_commit": 2314, 2024-09-11T23:40:15.441 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_split": 168, 2024-09-11T23:40:15.442 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_merge": 0, 2024-09-11T23:40:15.442 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes": 110285, 2024-09-11T23:40:15.442 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes_top": 24081, 2024-09-11T23:40:15.442 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes_bottom": 85899, 2024-09-11T23:40:15.442 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes_pin_tail": 305, 2024-09-11T23:40:15.442 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes_pinned": 75883, 2024-09-11T23:40:15.442 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes_expired": 104667, 2024-09-11T23:40:15.442 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes_with_caps": 3, 2024-09-11T23:40:15.442 INFO:teuthology.orchestra.run.smithi136.stdout: "caps": 3, 2024-09-11T23:40:15.442 INFO:teuthology.orchestra.run.smithi136.stdout: "subtrees": 7, 2024-09-11T23:40:15.442 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse": 440492, 2024-09-11T23:40:15.442 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse_hit": 357208, 2024-09-11T23:40:15.442 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse_forward": 16777, 2024-09-11T23:40:15.442 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse_discover": 925, 2024-09-11T23:40:15.443 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse_dir_fetch": 65, 2024-09-11T23:40:15.443 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse_remote_ino": 0, 2024-09-11T23:40:15.443 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse_lock": 452, 2024-09-11T23:40:15.443 INFO:teuthology.orchestra.run.smithi136.stdout: "load_cent": 16, 2024-09-11T23:40:15.443 INFO:teuthology.orchestra.run.smithi136.stdout: "q": 0, 2024-09-11T23:40:15.443 INFO:teuthology.orchestra.run.smithi136.stdout: "exported": 333, 2024-09-11T23:40:15.443 INFO:teuthology.orchestra.run.smithi136.stdout: "exported_inodes": 79923, 2024-09-11T23:40:15.443 INFO:teuthology.orchestra.run.smithi136.stdout: "imported": 429, 2024-09-11T23:40:15.443 INFO:teuthology.orchestra.run.smithi136.stdout: "imported_inodes": 125209, 2024-09-11T23:40:15.443 INFO:teuthology.orchestra.run.smithi136.stdout: "openino_dir_fetch": 0, 2024-09-11T23:40:15.443 INFO:teuthology.orchestra.run.smithi136.stdout: "openino_backtrace_fetch": 2962, 2024-09-11T23:40:15.443 INFO:teuthology.orchestra.run.smithi136.stdout: "openino_peer_discover": 0, 2024-09-11T23:40:15.443 INFO:teuthology.orchestra.run.smithi136.stdout: "root_rfiles": 7006, 2024-09-11T23:40:15.443 INFO:teuthology.orchestra.run.smithi136.stdout: "root_rbytes": 34252265, 2024-09-11T23:40:15.444 INFO:teuthology.orchestra.run.smithi136.stdout: "root_rsnaps": 2, 2024-09-11T23:40:15.444 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_backtrace_fetch": 2962, 2024-09-11T23:40:15.444 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_set_tag": 0, 2024-09-11T23:40:15.444 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_backtrace_repaired": 0, 2024-09-11T23:40:15.444 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_inotable_repaired": 0, 2024-09-11T23:40:15.444 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_dir_inodes": 43, 2024-09-11T23:40:15.444 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_dir_base_inodes": 0, 2024-09-11T23:40:15.444 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_dirfrag_rstats": 43, 2024-09-11T23:40:15.444 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_file_inodes": 2919, 2024-09-11T23:40:15.444 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_inode_file_caps": 326, 2024-09-11T23:40:15.444 INFO:teuthology.orchestra.run.smithi136.stdout: "ceph_cap_op_revoke": 88187, 2024-09-11T23:40:15.444 INFO:teuthology.orchestra.run.smithi136.stdout: "ceph_cap_op_grant": 115109, 2024-09-11T23:40:15.444 INFO:teuthology.orchestra.run.smithi136.stdout: "ceph_cap_op_trunc": 1, 2024-09-11T23:40:15.445 INFO:teuthology.orchestra.run.smithi136.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-11T23:40:15.445 INFO:teuthology.orchestra.run.smithi136.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-11T23:40:15.445 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_client_caps": 120535, 2024-09-11T23:40:15.445 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_client_caps_dirty": 36397, 2024-09-11T23:40:15.445 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_client_cap_release": 6041, 2024-09-11T23:40:15.445 INFO:teuthology.orchestra.run.smithi136.stdout: "process_request_cap_release": 85611 2024-09-11T23:40:15.445 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:15.445 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_cache": { 2024-09-11T23:40:15.445 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_update": 9644, 2024-09-11T23:40:15.445 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_update_receipt": 7841, 2024-09-11T23:40:15.445 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_try_discover": 4428, 2024-09-11T23:40:15.445 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_send_discover": 5888, 2024-09-11T23:40:15.445 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_handle_discover": 20926, 2024-09-11T23:40:15.445 INFO:teuthology.orchestra.run.smithi136.stdout: "num_strays": 317, 2024-09-11T23:40:15.446 INFO:teuthology.orchestra.run.smithi136.stdout: "num_strays_delayed": 0, 2024-09-11T23:40:15.446 INFO:teuthology.orchestra.run.smithi136.stdout: "num_strays_enqueuing": 0, 2024-09-11T23:40:15.446 INFO:teuthology.orchestra.run.smithi136.stdout: "strays_created": 43651, 2024-09-11T23:40:15.446 INFO:teuthology.orchestra.run.smithi136.stdout: "strays_enqueued": 43334, 2024-09-11T23:40:15.446 INFO:teuthology.orchestra.run.smithi136.stdout: "strays_reintegrated": 0, 2024-09-11T23:40:15.446 INFO:teuthology.orchestra.run.smithi136.stdout: "strays_migrated": 0, 2024-09-11T23:40:15.446 INFO:teuthology.orchestra.run.smithi136.stdout: "num_recovering_processing": 0, 2024-09-11T23:40:15.446 INFO:teuthology.orchestra.run.smithi136.stdout: "num_recovering_enqueued": 0, 2024-09-11T23:40:15.446 INFO:teuthology.orchestra.run.smithi136.stdout: "num_recovering_prioritized": 0, 2024-09-11T23:40:15.446 INFO:teuthology.orchestra.run.smithi136.stdout: "recovery_started": 0, 2024-09-11T23:40:15.446 INFO:teuthology.orchestra.run.smithi136.stdout: "recovery_completed": 0, 2024-09-11T23:40:15.446 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_quiesce_path": 18, 2024-09-11T23:40:15.446 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_quiesce_inode": 198732, 2024-09-11T23:40:15.446 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_enqueue_scrub": 0, 2024-09-11T23:40:15.447 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_exportdir": 336, 2024-09-11T23:40:15.447 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_flush": 0, 2024-09-11T23:40:15.447 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_fragmentdir": 170, 2024-09-11T23:40:15.447 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_fragstats": 0, 2024-09-11T23:40:15.447 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_inodestats": 0 2024-09-11T23:40:15.447 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:15.447 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_log": { 2024-09-11T23:40:15.447 INFO:teuthology.orchestra.run.smithi136.stdout: "evlrg": 7, 2024-09-11T23:40:15.447 INFO:teuthology.orchestra.run.smithi136.stdout: "evadd": 228972, 2024-09-11T23:40:15.447 INFO:teuthology.orchestra.run.smithi136.stdout: "evex": 147211, 2024-09-11T23:40:15.447 INFO:teuthology.orchestra.run.smithi136.stdout: "evtrm": 135191, 2024-09-11T23:40:15.447 INFO:teuthology.orchestra.run.smithi136.stdout: "ev": 93781, 2024-09-11T23:40:15.447 INFO:teuthology.orchestra.run.smithi136.stdout: "evexg": 0, 2024-09-11T23:40:15.447 INFO:teuthology.orchestra.run.smithi136.stdout: "evexd": 12020, 2024-09-11T23:40:15.448 INFO:teuthology.orchestra.run.smithi136.stdout: "segadd": 329, 2024-09-11T23:40:15.448 INFO:teuthology.orchestra.run.smithi136.stdout: "segex": 201, 2024-09-11T23:40:15.448 INFO:teuthology.orchestra.run.smithi136.stdout: "segtrm": 184, 2024-09-11T23:40:15.448 INFO:teuthology.orchestra.run.smithi136.stdout: "seg": 145, 2024-09-11T23:40:15.448 INFO:teuthology.orchestra.run.smithi136.stdout: "segmjr": 8, 2024-09-11T23:40:15.448 INFO:teuthology.orchestra.run.smithi136.stdout: "segexg": 0, 2024-09-11T23:40:15.448 INFO:teuthology.orchestra.run.smithi136.stdout: "segexd": 17, 2024-09-11T23:40:15.448 INFO:teuthology.orchestra.run.smithi136.stdout: "expos": 369896542, 2024-09-11T23:40:15.448 INFO:teuthology.orchestra.run.smithi136.stdout: "wrpos": 939298866, 2024-09-11T23:40:15.448 INFO:teuthology.orchestra.run.smithi136.stdout: "rdpos": 0, 2024-09-11T23:40:15.448 INFO:teuthology.orchestra.run.smithi136.stdout: "jlat": { 2024-09-11T23:40:15.448 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 13352, 2024-09-11T23:40:15.448 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 944.740749221, 2024-09-11T23:40:15.448 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.070756497 2024-09-11T23:40:15.449 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:15.449 INFO:teuthology.orchestra.run.smithi136.stdout: "replayed": 0 2024-09-11T23:40:15.449 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:15.449 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_mem": { 2024-09-11T23:40:15.449 INFO:teuthology.orchestra.run.smithi136.stdout: "ino": 63481, 2024-09-11T23:40:15.449 INFO:teuthology.orchestra.run.smithi136.stdout: "ino+": 173255, 2024-09-11T23:40:15.449 INFO:teuthology.orchestra.run.smithi136.stdout: "ino-": 109774, 2024-09-11T23:40:15.449 INFO:teuthology.orchestra.run.smithi136.stdout: "dir": 2249, 2024-09-11T23:40:15.449 INFO:teuthology.orchestra.run.smithi136.stdout: "dir+": 8286, 2024-09-11T23:40:15.449 INFO:teuthology.orchestra.run.smithi136.stdout: "dir-": 6037, 2024-09-11T23:40:15.449 INFO:teuthology.orchestra.run.smithi136.stdout: "dn": 110285, 2024-09-11T23:40:15.449 INFO:teuthology.orchestra.run.smithi136.stdout: "dn+": 299702, 2024-09-11T23:40:15.449 INFO:teuthology.orchestra.run.smithi136.stdout: "dn-": 189417, 2024-09-11T23:40:15.449 INFO:teuthology.orchestra.run.smithi136.stdout: "cap": 3, 2024-09-11T23:40:15.450 INFO:teuthology.orchestra.run.smithi136.stdout: "cap+": 86178, 2024-09-11T23:40:15.450 INFO:teuthology.orchestra.run.smithi136.stdout: "cap-": 86175, 2024-09-11T23:40:15.450 INFO:teuthology.orchestra.run.smithi136.stdout: "rss": 427176, 2024-09-11T23:40:15.450 INFO:teuthology.orchestra.run.smithi136.stdout: "heap": 223520 2024-09-11T23:40:15.450 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:15.450 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_server": { 2024-09-11T23:40:15.450 INFO:teuthology.orchestra.run.smithi136.stdout: "dispatch_client_request": 281941, 2024-09-11T23:40:15.450 INFO:teuthology.orchestra.run.smithi136.stdout: "dispatch_server_request": 14241, 2024-09-11T23:40:15.450 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_client_request": 271011, 2024-09-11T23:40:15.450 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_client_session": 401, 2024-09-11T23:40:15.450 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_peer_request": 28970, 2024-09-11T23:40:15.450 INFO:teuthology.orchestra.run.smithi136.stdout: "req_create_latency": { 2024-09-11T23:40:15.450 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 35607, 2024-09-11T23:40:15.450 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 369.231467906, 2024-09-11T23:40:15.451 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.010369631 2024-09-11T23:40:15.451 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:15.451 INFO:teuthology.orchestra.run.smithi136.stdout: "req_getattr_latency": { 2024-09-11T23:40:15.451 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 1122, 2024-09-11T23:40:15.451 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 76.118170504, 2024-09-11T23:40:15.451 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.067841506 2024-09-11T23:40:15.451 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:15.451 INFO:teuthology.orchestra.run.smithi136.stdout: "req_getfilelock_latency": { 2024-09-11T23:40:15.451 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-11T23:40:15.451 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-11T23:40:15.451 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-11T23:40:15.451 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:15.451 INFO:teuthology.orchestra.run.smithi136.stdout: "req_link_latency": { 2024-09-11T23:40:15.451 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-11T23:40:15.452 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-11T23:40:15.452 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-11T23:40:15.452 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:15.452 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lookup_latency": { 2024-09-11T23:40:15.452 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 123795, 2024-09-11T23:40:15.452 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 4826.269334200, 2024-09-11T23:40:15.452 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.038985979 2024-09-11T23:40:15.452 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:15.452 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lookuphash_latency": { 2024-09-11T23:40:15.452 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-11T23:40:15.452 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-11T23:40:15.452 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-11T23:40:15.452 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:15.453 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lookupino_latency": { 2024-09-11T23:40:15.453 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 1, 2024-09-11T23:40:15.453 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000148523, 2024-09-11T23:40:15.453 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000148523 2024-09-11T23:40:15.453 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:15.453 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lookupname_latency": { 2024-09-11T23:40:15.453 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-11T23:40:15.453 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-11T23:40:15.453 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-11T23:40:15.453 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:15.453 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lookupparent_latency": { 2024-09-11T23:40:15.453 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-11T23:40:15.453 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-11T23:40:15.454 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-11T23:40:15.454 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:15.454 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lookupsnap_latency": { 2024-09-11T23:40:15.454 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-11T23:40:15.454 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-11T23:40:15.455 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-11T23:40:15.455 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:15.455 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lssnap_latency": { 2024-09-11T23:40:15.455 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-11T23:40:15.455 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-11T23:40:15.455 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-11T23:40:15.455 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:15.455 INFO:teuthology.orchestra.run.smithi136.stdout: "req_mkdir_latency": { 2024-09-11T23:40:15.455 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 267, 2024-09-11T23:40:15.455 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 29.271328663, 2024-09-11T23:40:15.455 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.109630444 2024-09-11T23:40:15.455 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:15.455 INFO:teuthology.orchestra.run.smithi136.stdout: "req_mknod_latency": { 2024-09-11T23:40:15.455 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-11T23:40:15.456 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-11T23:40:15.456 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-11T23:40:15.456 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:15.456 INFO:teuthology.orchestra.run.smithi136.stdout: "req_mksnap_latency": { 2024-09-11T23:40:15.456 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-11T23:40:15.456 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-11T23:40:15.456 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-11T23:40:15.456 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:15.456 INFO:teuthology.orchestra.run.smithi136.stdout: "req_open_latency": { 2024-09-11T23:40:15.456 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 14952, 2024-09-11T23:40:15.456 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 162.891060715, 2024-09-11T23:40:15.456 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.010894265 2024-09-11T23:40:15.456 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:15.456 INFO:teuthology.orchestra.run.smithi136.stdout: "req_readdir_latency": { 2024-09-11T23:40:15.457 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 2111, 2024-09-11T23:40:15.457 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 6.460599873, 2024-09-11T23:40:15.457 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.003060445 2024-09-11T23:40:15.457 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:15.457 INFO:teuthology.orchestra.run.smithi136.stdout: "req_rename_latency": { 2024-09-11T23:40:15.457 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 35614, 2024-09-11T23:40:15.457 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 440.577443946, 2024-09-11T23:40:15.457 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.012370905 2024-09-11T23:40:15.457 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:15.457 INFO:teuthology.orchestra.run.smithi136.stdout: "req_renamesnap_latency": { 2024-09-11T23:40:15.457 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-11T23:40:15.457 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-11T23:40:15.457 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-11T23:40:15.458 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:15.458 INFO:teuthology.orchestra.run.smithi136.stdout: "req_snapdiff_latency": { 2024-09-11T23:40:15.458 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-11T23:40:15.458 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-11T23:40:15.458 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-11T23:40:15.458 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:15.458 INFO:teuthology.orchestra.run.smithi136.stdout: "req_rmdir_latency": { 2024-09-11T23:40:15.458 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 340, 2024-09-11T23:40:15.458 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 52.595706736, 2024-09-11T23:40:15.458 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.154693255 2024-09-11T23:40:15.458 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:15.458 INFO:teuthology.orchestra.run.smithi136.stdout: "req_rmsnap_latency": { 2024-09-11T23:40:15.458 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-11T23:40:15.459 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-11T23:40:15.459 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-11T23:40:15.459 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:15.459 INFO:teuthology.orchestra.run.smithi136.stdout: "req_rmxattr_latency": { 2024-09-11T23:40:15.459 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-11T23:40:15.459 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-11T23:40:15.459 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-11T23:40:15.459 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:15.459 INFO:teuthology.orchestra.run.smithi136.stdout: "req_setattr_latency": { 2024-09-11T23:40:15.459 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 1, 2024-09-11T23:40:15.459 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.197102432, 2024-09-11T23:40:15.459 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.197102432 2024-09-11T23:40:15.459 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:15.459 INFO:teuthology.orchestra.run.smithi136.stdout: "req_setdirlayout_latency": { 2024-09-11T23:40:15.459 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-11T23:40:15.460 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-11T23:40:15.460 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-11T23:40:15.460 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:15.460 INFO:teuthology.orchestra.run.smithi136.stdout: "req_setfilelock_latency": { 2024-09-11T23:40:15.460 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-11T23:40:15.460 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-11T23:40:15.460 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-11T23:40:15.460 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:15.460 INFO:teuthology.orchestra.run.smithi136.stdout: "req_setlayout_latency": { 2024-09-11T23:40:15.460 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-11T23:40:15.460 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-11T23:40:15.460 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-11T23:40:15.460 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:15.461 INFO:teuthology.orchestra.run.smithi136.stdout: "req_setxattr_latency": { 2024-09-11T23:40:15.461 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-11T23:40:15.461 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-11T23:40:15.461 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-11T23:40:15.461 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:15.461 INFO:teuthology.orchestra.run.smithi136.stdout: "req_symlink_latency": { 2024-09-11T23:40:15.461 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-11T23:40:15.461 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-11T23:40:15.461 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-11T23:40:15.461 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:15.461 INFO:teuthology.orchestra.run.smithi136.stdout: "req_unlink_latency": { 2024-09-11T23:40:15.461 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 39902, 2024-09-11T23:40:15.461 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 120.730183959, 2024-09-11T23:40:15.461 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.003025667 2024-09-11T23:40:15.462 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:15.462 INFO:teuthology.orchestra.run.smithi136.stdout: "cap_revoke_eviction": 0, 2024-09-11T23:40:15.462 INFO:teuthology.orchestra.run.smithi136.stdout: "cap_acquisition_throttle": 0, 2024-09-11T23:40:15.462 INFO:teuthology.orchestra.run.smithi136.stdout: "req_getvxattr_latency": { 2024-09-11T23:40:15.462 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-11T23:40:15.462 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-11T23:40:15.462 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-11T23:40:15.462 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-11T23:40:15.462 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:15.462 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_sessions": { 2024-09-11T23:40:15.462 INFO:teuthology.orchestra.run.smithi136.stdout: "session_count": 2, 2024-09-11T23:40:15.462 INFO:teuthology.orchestra.run.smithi136.stdout: "session_add": 3, 2024-09-11T23:40:15.462 INFO:teuthology.orchestra.run.smithi136.stdout: "session_remove": 1, 2024-09-11T23:40:15.462 INFO:teuthology.orchestra.run.smithi136.stdout: "sessions_open": 2, 2024-09-11T23:40:15.462 INFO:teuthology.orchestra.run.smithi136.stdout: "sessions_stale": 0, 2024-09-11T23:40:15.463 INFO:teuthology.orchestra.run.smithi136.stdout: "total_load": 18460, 2024-09-11T23:40:15.463 INFO:teuthology.orchestra.run.smithi136.stdout: "average_load": 9230, 2024-09-11T23:40:15.463 INFO:teuthology.orchestra.run.smithi136.stdout: "avg_session_uptime": 1293, 2024-09-11T23:40:15.463 INFO:teuthology.orchestra.run.smithi136.stdout: "mdthresh_evicted": 0 2024-09-11T23:40:15.463 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:15.463 INFO:teuthology.orchestra.run.smithi136.stdout: "mempool": { 2024-09-11T23:40:15.463 INFO:teuthology.orchestra.run.smithi136.stdout: "bloom_filter_bytes": 0, 2024-09-11T23:40:15.463 INFO:teuthology.orchestra.run.smithi136.stdout: "bloom_filter_items": 0, 2024-09-11T23:40:15.463 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_alloc_bytes": 0, 2024-09-11T23:40:15.463 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_alloc_items": 0, 2024-09-11T23:40:15.463 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_data_bytes": 0, 2024-09-11T23:40:15.463 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_data_items": 0, 2024-09-11T23:40:15.463 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-11T23:40:15.463 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_onode_items": 0, 2024-09-11T23:40:15.464 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-11T23:40:15.464 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_meta_items": 0, 2024-09-11T23:40:15.464 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_other_bytes": 0, 2024-09-11T23:40:15.464 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_other_items": 0, 2024-09-11T23:40:15.464 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-11T23:40:15.464 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_buffer_items": 0, 2024-09-11T23:40:15.464 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_extent_bytes": 0, 2024-09-11T23:40:15.464 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_extent_items": 0, 2024-09-11T23:40:15.464 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_blob_bytes": 0, 2024-09-11T23:40:15.464 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_blob_items": 0, 2024-09-11T23:40:15.464 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-11T23:40:15.464 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_shared_blob_items": 0, 2024-09-11T23:40:15.464 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-11T23:40:15.464 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_inline_bl_items": 0, 2024-09-11T23:40:15.465 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_fsck_bytes": 0, 2024-09-11T23:40:15.465 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_fsck_items": 0, 2024-09-11T23:40:15.465 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_txc_bytes": 0, 2024-09-11T23:40:15.465 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_txc_items": 0, 2024-09-11T23:40:15.465 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-11T23:40:15.465 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_writing_deferred_items": 0, 2024-09-11T23:40:15.465 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_writing_bytes": 0, 2024-09-11T23:40:15.465 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_writing_items": 0, 2024-09-11T23:40:15.465 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_bytes": 0, 2024-09-11T23:40:15.465 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_items": 0, 2024-09-11T23:40:15.465 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_file_reader_bytes": 0, 2024-09-11T23:40:15.465 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_file_reader_items": 0, 2024-09-11T23:40:15.465 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_file_writer_bytes": 0, 2024-09-11T23:40:15.466 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_file_writer_items": 0, 2024-09-11T23:40:15.466 INFO:teuthology.orchestra.run.smithi136.stdout: "buffer_anon_bytes": 5232567, 2024-09-11T23:40:15.466 INFO:teuthology.orchestra.run.smithi136.stdout: "buffer_anon_items": 64188, 2024-09-11T23:40:15.466 INFO:teuthology.orchestra.run.smithi136.stdout: "buffer_meta_bytes": 0, 2024-09-11T23:40:15.466 INFO:teuthology.orchestra.run.smithi136.stdout: "buffer_meta_items": 0, 2024-09-11T23:40:15.466 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_bytes": 0, 2024-09-11T23:40:15.466 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_items": 0, 2024-09-11T23:40:15.466 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_mapbl_bytes": 0, 2024-09-11T23:40:15.466 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_mapbl_items": 0, 2024-09-11T23:40:15.466 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_pglog_bytes": 0, 2024-09-11T23:40:15.466 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_pglog_items": 0, 2024-09-11T23:40:15.466 INFO:teuthology.orchestra.run.smithi136.stdout: "osdmap_bytes": 8344, 2024-09-11T23:40:15.466 INFO:teuthology.orchestra.run.smithi136.stdout: "osdmap_items": 184, 2024-09-11T23:40:15.466 INFO:teuthology.orchestra.run.smithi136.stdout: "osdmap_mapping_bytes": 0, 2024-09-11T23:40:15.467 INFO:teuthology.orchestra.run.smithi136.stdout: "osdmap_mapping_items": 0, 2024-09-11T23:40:15.467 INFO:teuthology.orchestra.run.smithi136.stdout: "pgmap_bytes": 0, 2024-09-11T23:40:15.467 INFO:teuthology.orchestra.run.smithi136.stdout: "pgmap_items": 0, 2024-09-11T23:40:15.467 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_co_bytes": 229119883, 2024-09-11T23:40:15.467 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_co_items": 2304723, 2024-09-11T23:40:15.467 INFO:teuthology.orchestra.run.smithi136.stdout: "unittest_1_bytes": 0, 2024-09-11T23:40:15.467 INFO:teuthology.orchestra.run.smithi136.stdout: "unittest_1_items": 0, 2024-09-11T23:40:15.467 INFO:teuthology.orchestra.run.smithi136.stdout: "unittest_2_bytes": 0, 2024-09-11T23:40:15.467 INFO:teuthology.orchestra.run.smithi136.stdout: "unittest_2_items": 0 2024-09-11T23:40:15.467 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:15.467 INFO:teuthology.orchestra.run.smithi136.stdout: "objecter": { 2024-09-11T23:40:15.467 INFO:teuthology.orchestra.run.smithi136.stdout: "op_active": 0, 2024-09-11T23:40:15.467 INFO:teuthology.orchestra.run.smithi136.stdout: "op_laggy": 0, 2024-09-11T23:40:15.467 INFO:teuthology.orchestra.run.smithi136.stdout: "op_send": 68771, 2024-09-11T23:40:15.468 INFO:teuthology.orchestra.run.smithi136.stdout: "op_send_bytes": 992197144, 2024-09-11T23:40:15.468 INFO:teuthology.orchestra.run.smithi136.stdout: "op_resend": 33, 2024-09-11T23:40:15.468 INFO:teuthology.orchestra.run.smithi136.stdout: "op_reply": 68771, 2024-09-11T23:40:15.468 INFO:teuthology.orchestra.run.smithi136.stdout: "op_latency": { 2024-09-11T23:40:15.468 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 68771, 2024-09-11T23:40:15.468 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 2021.156397544, 2024-09-11T23:40:15.468 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.029389661 2024-09-11T23:40:15.468 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:15.468 INFO:teuthology.orchestra.run.smithi136.stdout: "op_inflight": 0, 2024-09-11T23:40:15.468 INFO:teuthology.orchestra.run.smithi136.stdout: "oplen_avg": { 2024-09-11T23:40:15.468 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 68771, 2024-09-11T23:40:15.468 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 80794 2024-09-11T23:40:15.468 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:15.468 INFO:teuthology.orchestra.run.smithi136.stdout: "op": 68771, 2024-09-11T23:40:15.469 INFO:teuthology.orchestra.run.smithi136.stdout: "op_r": 4023, 2024-09-11T23:40:15.469 INFO:teuthology.orchestra.run.smithi136.stdout: "op_w": 64748, 2024-09-11T23:40:15.469 INFO:teuthology.orchestra.run.smithi136.stdout: "op_rmw": 0, 2024-09-11T23:40:15.469 INFO:teuthology.orchestra.run.smithi136.stdout: "op_pg": 0, 2024-09-11T23:40:15.469 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_stat": 771, 2024-09-11T23:40:15.469 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_create": 3353, 2024-09-11T23:40:15.469 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_read": 759, 2024-09-11T23:40:15.469 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_write": 14119, 2024-09-11T23:40:15.469 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_writefull": 363, 2024-09-11T23:40:15.469 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_writesame": 0, 2024-09-11T23:40:15.469 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_append": 0, 2024-09-11T23:40:15.469 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_zero": 0, 2024-09-11T23:40:15.469 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_truncate": 168, 2024-09-11T23:40:15.469 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_delete": 43809, 2024-09-11T23:40:15.470 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_mapext": 0, 2024-09-11T23:40:15.470 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_sparse_read": 0, 2024-09-11T23:40:15.470 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_clonerange": 0, 2024-09-11T23:40:15.470 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_getxattr": 2968, 2024-09-11T23:40:15.470 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_setxattr": 6706, 2024-09-11T23:40:15.470 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_cmpxattr": 0, 2024-09-11T23:40:15.470 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_rmxattr": 0, 2024-09-11T23:40:15.470 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_resetxattrs": 0, 2024-09-11T23:40:15.470 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_call": 0, 2024-09-11T23:40:15.470 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_watch": 0, 2024-09-11T23:40:15.470 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_notify": 0, 2024-09-11T23:40:15.470 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_src_cmpxattr": 0, 2024-09-11T23:40:15.470 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_pgls": 0, 2024-09-11T23:40:15.470 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_pgls_filter": 0, 2024-09-11T23:40:15.470 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_other": 1, 2024-09-11T23:40:15.471 INFO:teuthology.orchestra.run.smithi136.stdout: "linger_active": 0, 2024-09-11T23:40:15.471 INFO:teuthology.orchestra.run.smithi136.stdout: "linger_send": 0, 2024-09-11T23:40:15.471 INFO:teuthology.orchestra.run.smithi136.stdout: "linger_resend": 0, 2024-09-11T23:40:15.471 INFO:teuthology.orchestra.run.smithi136.stdout: "linger_ping": 0, 2024-09-11T23:40:15.471 INFO:teuthology.orchestra.run.smithi136.stdout: "poolop_active": 0, 2024-09-11T23:40:15.471 INFO:teuthology.orchestra.run.smithi136.stdout: "poolop_send": 0, 2024-09-11T23:40:15.471 INFO:teuthology.orchestra.run.smithi136.stdout: "poolop_resend": 0, 2024-09-11T23:40:15.471 INFO:teuthology.orchestra.run.smithi136.stdout: "poolstat_active": 0, 2024-09-11T23:40:15.471 INFO:teuthology.orchestra.run.smithi136.stdout: "poolstat_send": 0, 2024-09-11T23:40:15.471 INFO:teuthology.orchestra.run.smithi136.stdout: "poolstat_resend": 0, 2024-09-11T23:40:15.471 INFO:teuthology.orchestra.run.smithi136.stdout: "statfs_active": 0, 2024-09-11T23:40:15.471 INFO:teuthology.orchestra.run.smithi136.stdout: "statfs_send": 0, 2024-09-11T23:40:15.471 INFO:teuthology.orchestra.run.smithi136.stdout: "statfs_resend": 0, 2024-09-11T23:40:15.471 INFO:teuthology.orchestra.run.smithi136.stdout: "command_active": 0, 2024-09-11T23:40:15.472 INFO:teuthology.orchestra.run.smithi136.stdout: "command_send": 0, 2024-09-11T23:40:15.472 INFO:teuthology.orchestra.run.smithi136.stdout: "command_resend": 0, 2024-09-11T23:40:15.472 INFO:teuthology.orchestra.run.smithi136.stdout: "map_epoch": 117, 2024-09-11T23:40:15.472 INFO:teuthology.orchestra.run.smithi136.stdout: "map_full": 0, 2024-09-11T23:40:15.472 INFO:teuthology.orchestra.run.smithi136.stdout: "map_inc": 34, 2024-09-11T23:40:15.472 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_sessions": 12, 2024-09-11T23:40:15.472 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_session_open": 12, 2024-09-11T23:40:15.472 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_session_close": 0, 2024-09-11T23:40:15.472 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_laggy": 0, 2024-09-11T23:40:15.472 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_wr": 5799, 2024-09-11T23:40:15.472 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_rd": 604, 2024-09-11T23:40:15.472 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_del": 1374 2024-09-11T23:40:15.472 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:15.472 INFO:teuthology.orchestra.run.smithi136.stdout: "oft": { 2024-09-11T23:40:15.473 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_total_objs": 1, 2024-09-11T23:40:15.473 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_total_kv_pairs": 0, 2024-09-11T23:40:15.473 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_total_updates": 0, 2024-09-11T23:40:15.473 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_total_removes": 0 2024-09-11T23:40:15.473 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:15.473 INFO:teuthology.orchestra.run.smithi136.stdout: "osd-slow-ops": { 2024-09-11T23:40:15.473 INFO:teuthology.orchestra.run.smithi136.stdout: "slow_ops_count": 191901 2024-09-11T23:40:15.473 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:15.473 INFO:teuthology.orchestra.run.smithi136.stdout: "purge_queue": { 2024-09-11T23:40:15.473 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_executing_ops": 0, 2024-09-11T23:40:15.473 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_executing_ops_high_water": 11, 2024-09-11T23:40:15.473 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_executing": 0, 2024-09-11T23:40:15.473 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_executing_high_water": 6, 2024-09-11T23:40:15.473 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_executed_ops": 43486, 2024-09-11T23:40:15.473 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_executed": 43334, 2024-09-11T23:40:15.474 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_item_in_journal": 0 2024-09-11T23:40:15.474 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:15.474 INFO:teuthology.orchestra.run.smithi136.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-11T23:40:15.474 INFO:teuthology.orchestra.run.smithi136.stdout: "val": 0, 2024-09-11T23:40:15.474 INFO:teuthology.orchestra.run.smithi136.stdout: "max": 104857600, 2024-09-11T23:40:15.474 INFO:teuthology.orchestra.run.smithi136.stdout: "get_started": 0, 2024-09-11T23:40:15.474 INFO:teuthology.orchestra.run.smithi136.stdout: "get": 648670, 2024-09-11T23:40:15.474 INFO:teuthology.orchestra.run.smithi136.stdout: "get_sum": 233285155, 2024-09-11T23:40:15.474 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_fail": 0, 2024-09-11T23:40:15.474 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_success": 648670, 2024-09-11T23:40:15.474 INFO:teuthology.orchestra.run.smithi136.stdout: "take": 0, 2024-09-11T23:40:15.474 INFO:teuthology.orchestra.run.smithi136.stdout: "take_sum": 0, 2024-09-11T23:40:15.474 INFO:teuthology.orchestra.run.smithi136.stdout: "put": 648670, 2024-09-11T23:40:15.474 INFO:teuthology.orchestra.run.smithi136.stdout: "put_sum": 233285155, 2024-09-11T23:40:15.475 INFO:teuthology.orchestra.run.smithi136.stdout: "wait": { 2024-09-11T23:40:15.475 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-11T23:40:15.475 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-11T23:40:15.475 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-11T23:40:15.475 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-11T23:40:15.475 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:15.475 INFO:teuthology.orchestra.run.smithi136.stdout: "throttle-objecter_bytes": { 2024-09-11T23:40:15.475 INFO:teuthology.orchestra.run.smithi136.stdout: "val": 0, 2024-09-11T23:40:15.475 INFO:teuthology.orchestra.run.smithi136.stdout: "max": 104857600, 2024-09-11T23:40:15.475 INFO:teuthology.orchestra.run.smithi136.stdout: "get_started": 0, 2024-09-11T23:40:15.475 INFO:teuthology.orchestra.run.smithi136.stdout: "get": 0, 2024-09-11T23:40:15.475 INFO:teuthology.orchestra.run.smithi136.stdout: "get_sum": 0, 2024-09-11T23:40:15.475 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_fail": 0, 2024-09-11T23:40:15.475 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_success": 0, 2024-09-11T23:40:15.476 INFO:teuthology.orchestra.run.smithi136.stdout: "take": 68771, 2024-09-11T23:40:15.476 INFO:teuthology.orchestra.run.smithi136.stdout: "take_sum": 998499822, 2024-09-11T23:40:15.476 INFO:teuthology.orchestra.run.smithi136.stdout: "put": 68771, 2024-09-11T23:40:15.476 INFO:teuthology.orchestra.run.smithi136.stdout: "put_sum": 998499822, 2024-09-11T23:40:15.476 INFO:teuthology.orchestra.run.smithi136.stdout: "wait": { 2024-09-11T23:40:15.476 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-11T23:40:15.476 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-11T23:40:15.476 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-11T23:40:15.476 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-11T23:40:15.476 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:15.476 INFO:teuthology.orchestra.run.smithi136.stdout: "throttle-objecter_ops": { 2024-09-11T23:40:15.476 INFO:teuthology.orchestra.run.smithi136.stdout: "val": 0, 2024-09-11T23:40:15.476 INFO:teuthology.orchestra.run.smithi136.stdout: "max": 1024, 2024-09-11T23:40:15.476 INFO:teuthology.orchestra.run.smithi136.stdout: "get_started": 0, 2024-09-11T23:40:15.477 INFO:teuthology.orchestra.run.smithi136.stdout: "get": 0, 2024-09-11T23:40:15.477 INFO:teuthology.orchestra.run.smithi136.stdout: "get_sum": 0, 2024-09-11T23:40:15.477 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_fail": 0, 2024-09-11T23:40:15.477 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_success": 0, 2024-09-11T23:40:15.477 INFO:teuthology.orchestra.run.smithi136.stdout: "take": 68771, 2024-09-11T23:40:15.477 INFO:teuthology.orchestra.run.smithi136.stdout: "take_sum": 68771, 2024-09-11T23:40:15.477 INFO:teuthology.orchestra.run.smithi136.stdout: "put": 68771, 2024-09-11T23:40:15.477 INFO:teuthology.orchestra.run.smithi136.stdout: "put_sum": 68771, 2024-09-11T23:40:15.477 INFO:teuthology.orchestra.run.smithi136.stdout: "wait": { 2024-09-11T23:40:15.477 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-11T23:40:15.477 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-11T23:40:15.477 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-11T23:40:15.477 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-11T23:40:15.477 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:15.478 INFO:teuthology.orchestra.run.smithi136.stdout: "throttle-write_buf_throttle": { 2024-09-11T23:40:15.478 INFO:teuthology.orchestra.run.smithi136.stdout: "val": 0, 2024-09-11T23:40:15.478 INFO:teuthology.orchestra.run.smithi136.stdout: "max": 3758096384, 2024-09-11T23:40:15.478 INFO:teuthology.orchestra.run.smithi136.stdout: "get_started": 0, 2024-09-11T23:40:15.478 INFO:teuthology.orchestra.run.smithi136.stdout: "get": 43334, 2024-09-11T23:40:15.478 INFO:teuthology.orchestra.run.smithi136.stdout: "get_sum": 6307510, 2024-09-11T23:40:15.478 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_fail": 0, 2024-09-11T23:40:15.478 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_success": 43334, 2024-09-11T23:40:15.478 INFO:teuthology.orchestra.run.smithi136.stdout: "take": 0, 2024-09-11T23:40:15.478 INFO:teuthology.orchestra.run.smithi136.stdout: "take_sum": 0, 2024-09-11T23:40:15.478 INFO:teuthology.orchestra.run.smithi136.stdout: "put": 762, 2024-09-11T23:40:15.478 INFO:teuthology.orchestra.run.smithi136.stdout: "put_sum": 6307510, 2024-09-11T23:40:15.478 INFO:teuthology.orchestra.run.smithi136.stdout: "wait": { 2024-09-11T23:40:15.478 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-11T23:40:15.478 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-11T23:40:15.479 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-11T23:40:15.479 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-11T23:40:15.479 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:15.479 INFO:teuthology.orchestra.run.smithi136.stdout: "throttle-write_buf_throttle-0x555c900fc960": { 2024-09-11T23:40:15.479 INFO:teuthology.orchestra.run.smithi136.stdout: "val": 574, 2024-09-11T23:40:15.479 INFO:teuthology.orchestra.run.smithi136.stdout: "max": 3758096384, 2024-09-11T23:40:15.479 INFO:teuthology.orchestra.run.smithi136.stdout: "get_started": 0, 2024-09-11T23:40:15.479 INFO:teuthology.orchestra.run.smithi136.stdout: "get": 228972, 2024-09-11T23:40:15.479 INFO:teuthology.orchestra.run.smithi136.stdout: "get_sum": 935104562, 2024-09-11T23:40:15.479 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_fail": 0, 2024-09-11T23:40:15.479 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_success": 228972, 2024-09-11T23:40:15.479 INFO:teuthology.orchestra.run.smithi136.stdout: "take": 0, 2024-09-11T23:40:15.479 INFO:teuthology.orchestra.run.smithi136.stdout: "take_sum": 0, 2024-09-11T23:40:15.479 INFO:teuthology.orchestra.run.smithi136.stdout: "put": 13352, 2024-09-11T23:40:15.480 INFO:teuthology.orchestra.run.smithi136.stdout: "put_sum": 935103988, 2024-09-11T23:40:15.480 INFO:teuthology.orchestra.run.smithi136.stdout: "wait": { 2024-09-11T23:40:15.480 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-11T23:40:15.480 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-11T23:40:15.480 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-11T23:40:15.480 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-11T23:40:15.480 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-11T23:40:15.480 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-11T23:40:15.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:40:15 smithi136 ceph-mon[35121]: pgmap v690: 97 pgs: 41 active+clean+snaptrim_wait, 23 active+clean+snaptrim, 33 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 4.8 KiB/s wr, 11 op/s 2024-09-11T23:40:16.000 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:40:15 smithi026 ceph-mon[31172]: pgmap v690: 97 pgs: 41 active+clean+snaptrim_wait, 23 active+clean+snaptrim, 33 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 4.8 KiB/s wr, 11 op/s 2024-09-11T23:40:16.052 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:40:15 smithi076 ceph-mon[35296]: pgmap v690: 97 pgs: 41 active+clean+snaptrim_wait, 23 active+clean+snaptrim, 33 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 4.8 KiB/s wr, 11 op/s 2024-09-11T23:40:16.279 INFO:tasks.check_counter:Daemon mds.i mds.exported=333 2024-09-11T23:40:16.279 INFO:tasks.check_counter:Daemon mds.i mds.imported=429 2024-09-11T23:40:16.280 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update=9644 2024-09-11T23:40:16.280 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update_receipt=7841 2024-09-11T23:40:16.280 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=2 2024-09-11T23:40:16.281 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2024-09-11T23:40:16.281 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2024-09-11T23:40:16.281 INFO:tasks.check_counter:Daemon mds.i mds.dir_split=168 2024-09-11T23:40:16.281 DEBUG:tasks.check_counter:Getting stats from l 2024-09-11T23:40:16.282 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph daemon mds.l perf dump 2024-09-11T23:40:16.739 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T23:40:17.376 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-11T23:40:17.376 INFO:teuthology.orchestra.run.smithi136.stdout: "AsyncMessenger::Worker-0": { 2024-09-11T23:40:17.376 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_messages": 2113, 2024-09-11T23:40:17.376 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages": 2115, 2024-09-11T23:40:17.377 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_bytes": 129239604, 2024-09-11T23:40:17.377 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_bytes": 671998, 2024-09-11T23:40:17.377 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_created_connections": 8, 2024-09-11T23:40:17.377 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_active_connections": 4, 2024-09-11T23:40:17.377 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_total_time": 0.230621963, 2024-09-11T23:40:17.377 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_send_time": 0.042770111, 2024-09-11T23:40:17.377 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_recv_time": 0.115829757, 2024-09-11T23:40:17.377 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_fast_dispatch_time": 0.058681271, 2024-09-11T23:40:17.377 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T23:40:17.377 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 2113, 2024-09-11T23:40:17.377 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.102918714, 2024-09-11T23:40:17.377 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000048707 2024-09-11T23:40:17.377 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:17.378 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_handle_ack_lat": { 2024-09-11T23:40:17.378 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-11T23:40:17.378 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-11T23:40:17.378 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-11T23:40:17.378 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:17.378 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-11T23:40:17.378 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_encrypted_bytes": 0 2024-09-11T23:40:17.378 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:17.378 INFO:teuthology.orchestra.run.smithi136.stdout: "AsyncMessenger::Worker-1": { 2024-09-11T23:40:17.378 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_messages": 745, 2024-09-11T23:40:17.378 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages": 1042, 2024-09-11T23:40:17.378 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_bytes": 100856529, 2024-09-11T23:40:17.378 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_bytes": 1010041, 2024-09-11T23:40:17.379 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_created_connections": 10, 2024-09-11T23:40:17.379 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_active_connections": 7, 2024-09-11T23:40:17.379 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_total_time": 0.132827229, 2024-09-11T23:40:17.379 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_send_time": 0.030983738, 2024-09-11T23:40:17.379 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_recv_time": 0.075774693, 2024-09-11T23:40:17.379 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_fast_dispatch_time": 0.018023588, 2024-09-11T23:40:17.379 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T23:40:17.380 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 1039, 2024-09-11T23:40:17.380 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.042519944, 2024-09-11T23:40:17.380 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000040923 2024-09-11T23:40:17.380 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:17.380 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_handle_ack_lat": { 2024-09-11T23:40:17.380 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-11T23:40:17.380 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-11T23:40:17.380 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-11T23:40:17.380 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:17.380 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_encrypted_bytes": 480, 2024-09-11T23:40:17.380 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_encrypted_bytes": 774080 2024-09-11T23:40:17.380 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:17.380 INFO:teuthology.orchestra.run.smithi136.stdout: "AsyncMessenger::Worker-2": { 2024-09-11T23:40:17.381 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_messages": 2388, 2024-09-11T23:40:17.381 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages": 2310, 2024-09-11T23:40:17.381 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_bytes": 107587883, 2024-09-11T23:40:17.381 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_bytes": 846784, 2024-09-11T23:40:17.381 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_created_connections": 5, 2024-09-11T23:40:17.381 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_active_connections": 4, 2024-09-11T23:40:17.381 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_total_time": 0.307228298, 2024-09-11T23:40:17.382 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_send_time": 0.078518515, 2024-09-11T23:40:17.382 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_recv_time": 0.170107168, 2024-09-11T23:40:17.382 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_fast_dispatch_time": 0.061536679, 2024-09-11T23:40:17.382 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T23:40:17.382 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 2310, 2024-09-11T23:40:17.382 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.129757308, 2024-09-11T23:40:17.382 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000056171 2024-09-11T23:40:17.382 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:17.382 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_handle_ack_lat": { 2024-09-11T23:40:17.382 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-11T23:40:17.382 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-11T23:40:17.383 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-11T23:40:17.383 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:17.383 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_encrypted_bytes": 2113264, 2024-09-11T23:40:17.383 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_encrypted_bytes": 248944 2024-09-11T23:40:17.383 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:17.383 INFO:teuthology.orchestra.run.smithi136.stdout: "cct": { 2024-09-11T23:40:17.383 INFO:teuthology.orchestra.run.smithi136.stdout: "total_workers": 1, 2024-09-11T23:40:17.383 INFO:teuthology.orchestra.run.smithi136.stdout: "unhealthy_workers": 0 2024-09-11T23:40:17.383 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:17.384 INFO:teuthology.orchestra.run.smithi136.stdout: "finisher-MDSRank": { 2024-09-11T23:40:17.384 INFO:teuthology.orchestra.run.smithi136.stdout: "queue_len": 0, 2024-09-11T23:40:17.384 INFO:teuthology.orchestra.run.smithi136.stdout: "complete_latency": { 2024-09-11T23:40:17.384 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 7694, 2024-09-11T23:40:17.384 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.213838109, 2024-09-11T23:40:17.384 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000027792 2024-09-11T23:40:17.384 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-11T23:40:17.384 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:17.384 INFO:teuthology.orchestra.run.smithi136.stdout: "finisher-PurgeQueue": { 2024-09-11T23:40:17.384 INFO:teuthology.orchestra.run.smithi136.stdout: "queue_len": 0, 2024-09-11T23:40:17.384 INFO:teuthology.orchestra.run.smithi136.stdout: "complete_latency": { 2024-09-11T23:40:17.384 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-11T23:40:17.385 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-11T23:40:17.385 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-11T23:40:17.385 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-11T23:40:17.385 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:17.385 INFO:teuthology.orchestra.run.smithi136.stdout: "mds": { 2024-09-11T23:40:17.385 INFO:teuthology.orchestra.run.smithi136.stdout: "request": 0, 2024-09-11T23:40:17.385 INFO:teuthology.orchestra.run.smithi136.stdout: "reply": 0, 2024-09-11T23:40:17.386 INFO:teuthology.orchestra.run.smithi136.stdout: "reply_latency": { 2024-09-11T23:40:17.386 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-11T23:40:17.386 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-11T23:40:17.386 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-11T23:40:17.386 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:17.386 INFO:teuthology.orchestra.run.smithi136.stdout: "slow_reply": 0, 2024-09-11T23:40:17.386 INFO:teuthology.orchestra.run.smithi136.stdout: "forward": 0, 2024-09-11T23:40:17.386 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_fetch_complete": 0, 2024-09-11T23:40:17.386 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_fetch_keys": 0, 2024-09-11T23:40:17.386 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_commit": 0, 2024-09-11T23:40:17.387 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_split": 0, 2024-09-11T23:40:17.387 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_merge": 0, 2024-09-11T23:40:17.387 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes": 44501, 2024-09-11T23:40:17.387 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes_top": 1889, 2024-09-11T23:40:17.387 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes_bottom": 42612, 2024-09-11T23:40:17.387 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes_pin_tail": 0, 2024-09-11T23:40:17.387 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes_pinned": 41802, 2024-09-11T23:40:17.387 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes_expired": 0, 2024-09-11T23:40:17.387 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes_with_caps": 0, 2024-09-11T23:40:17.387 INFO:teuthology.orchestra.run.smithi136.stdout: "caps": 0, 2024-09-11T23:40:17.387 INFO:teuthology.orchestra.run.smithi136.stdout: "subtrees": 1, 2024-09-11T23:40:17.388 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse": 0, 2024-09-11T23:40:17.388 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse_hit": 0, 2024-09-11T23:40:17.388 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse_forward": 0, 2024-09-11T23:40:17.388 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse_discover": 0, 2024-09-11T23:40:17.388 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse_dir_fetch": 0, 2024-09-11T23:40:17.388 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse_remote_ino": 0, 2024-09-11T23:40:17.388 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse_lock": 0, 2024-09-11T23:40:17.388 INFO:teuthology.orchestra.run.smithi136.stdout: "load_cent": 0, 2024-09-11T23:40:17.388 INFO:teuthology.orchestra.run.smithi136.stdout: "q": 0, 2024-09-11T23:40:17.388 INFO:teuthology.orchestra.run.smithi136.stdout: "exported": 0, 2024-09-11T23:40:17.389 INFO:teuthology.orchestra.run.smithi136.stdout: "exported_inodes": 0, 2024-09-11T23:40:17.389 INFO:teuthology.orchestra.run.smithi136.stdout: "imported": 0, 2024-09-11T23:40:17.389 INFO:teuthology.orchestra.run.smithi136.stdout: "imported_inodes": 0, 2024-09-11T23:40:17.389 INFO:teuthology.orchestra.run.smithi136.stdout: "openino_dir_fetch": 0, 2024-09-11T23:40:17.389 INFO:teuthology.orchestra.run.smithi136.stdout: "openino_backtrace_fetch": 0, 2024-09-11T23:40:17.389 INFO:teuthology.orchestra.run.smithi136.stdout: "openino_peer_discover": 0, 2024-09-11T23:40:17.389 INFO:teuthology.orchestra.run.smithi136.stdout: "root_rfiles": 0, 2024-09-11T23:40:17.390 INFO:teuthology.orchestra.run.smithi136.stdout: "root_rbytes": 0, 2024-09-11T23:40:17.390 INFO:teuthology.orchestra.run.smithi136.stdout: "root_rsnaps": 0, 2024-09-11T23:40:17.390 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_backtrace_fetch": 0, 2024-09-11T23:40:17.390 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_set_tag": 0, 2024-09-11T23:40:17.390 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_backtrace_repaired": 0, 2024-09-11T23:40:17.390 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_inotable_repaired": 0, 2024-09-11T23:40:17.390 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_dir_inodes": 0, 2024-09-11T23:40:17.390 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_dir_base_inodes": 0, 2024-09-11T23:40:17.391 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_dirfrag_rstats": 0, 2024-09-11T23:40:17.391 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_file_inodes": 0, 2024-09-11T23:40:17.391 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_inode_file_caps": 0, 2024-09-11T23:40:17.391 INFO:teuthology.orchestra.run.smithi136.stdout: "ceph_cap_op_revoke": 0, 2024-09-11T23:40:17.391 INFO:teuthology.orchestra.run.smithi136.stdout: "ceph_cap_op_grant": 0, 2024-09-11T23:40:17.391 INFO:teuthology.orchestra.run.smithi136.stdout: "ceph_cap_op_trunc": 0, 2024-09-11T23:40:17.391 INFO:teuthology.orchestra.run.smithi136.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-11T23:40:17.391 INFO:teuthology.orchestra.run.smithi136.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-11T23:40:17.391 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_client_caps": 0, 2024-09-11T23:40:17.391 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_client_caps_dirty": 0, 2024-09-11T23:40:17.391 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_client_cap_release": 0, 2024-09-11T23:40:17.392 INFO:teuthology.orchestra.run.smithi136.stdout: "process_request_cap_release": 0 2024-09-11T23:40:17.392 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:17.392 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_cache": { 2024-09-11T23:40:17.392 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_update": 0, 2024-09-11T23:40:17.392 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_update_receipt": 0, 2024-09-11T23:40:17.392 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_try_discover": 0, 2024-09-11T23:40:17.392 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_send_discover": 0, 2024-09-11T23:40:17.392 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_handle_discover": 0, 2024-09-11T23:40:17.392 INFO:teuthology.orchestra.run.smithi136.stdout: "num_strays": 0, 2024-09-11T23:40:17.392 INFO:teuthology.orchestra.run.smithi136.stdout: "num_strays_delayed": 0, 2024-09-11T23:40:17.392 INFO:teuthology.orchestra.run.smithi136.stdout: "num_strays_enqueuing": 0, 2024-09-11T23:40:17.392 INFO:teuthology.orchestra.run.smithi136.stdout: "strays_created": 0, 2024-09-11T23:40:17.393 INFO:teuthology.orchestra.run.smithi136.stdout: "strays_enqueued": 0, 2024-09-11T23:40:17.393 INFO:teuthology.orchestra.run.smithi136.stdout: "strays_reintegrated": 0, 2024-09-11T23:40:17.393 INFO:teuthology.orchestra.run.smithi136.stdout: "strays_migrated": 0, 2024-09-11T23:40:17.393 INFO:teuthology.orchestra.run.smithi136.stdout: "num_recovering_processing": 0, 2024-09-11T23:40:17.393 INFO:teuthology.orchestra.run.smithi136.stdout: "num_recovering_enqueued": 0, 2024-09-11T23:40:17.393 INFO:teuthology.orchestra.run.smithi136.stdout: "num_recovering_prioritized": 0, 2024-09-11T23:40:17.393 INFO:teuthology.orchestra.run.smithi136.stdout: "recovery_started": 0, 2024-09-11T23:40:17.393 INFO:teuthology.orchestra.run.smithi136.stdout: "recovery_completed": 0, 2024-09-11T23:40:17.393 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_quiesce_path": 0, 2024-09-11T23:40:17.393 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_quiesce_inode": 0, 2024-09-11T23:40:17.394 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_enqueue_scrub": 0, 2024-09-11T23:40:17.394 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_exportdir": 0, 2024-09-11T23:40:17.394 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_flush": 0, 2024-09-11T23:40:17.394 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_fragmentdir": 0, 2024-09-11T23:40:17.394 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_fragstats": 0, 2024-09-11T23:40:17.394 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_inodestats": 0 2024-09-11T23:40:17.394 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:17.394 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_log": { 2024-09-11T23:40:17.395 INFO:teuthology.orchestra.run.smithi136.stdout: "evlrg": 0, 2024-09-11T23:40:17.395 INFO:teuthology.orchestra.run.smithi136.stdout: "evadd": 0, 2024-09-11T23:40:17.395 INFO:teuthology.orchestra.run.smithi136.stdout: "evex": 0, 2024-09-11T23:40:17.395 INFO:teuthology.orchestra.run.smithi136.stdout: "evtrm": 0, 2024-09-11T23:40:17.395 INFO:teuthology.orchestra.run.smithi136.stdout: "ev": 104390, 2024-09-11T23:40:17.395 INFO:teuthology.orchestra.run.smithi136.stdout: "evexg": 0, 2024-09-11T23:40:17.395 INFO:teuthology.orchestra.run.smithi136.stdout: "evexd": 0, 2024-09-11T23:40:17.395 INFO:teuthology.orchestra.run.smithi136.stdout: "segadd": 0, 2024-09-11T23:40:17.395 INFO:teuthology.orchestra.run.smithi136.stdout: "segex": 0, 2024-09-11T23:40:17.395 INFO:teuthology.orchestra.run.smithi136.stdout: "segtrm": 0, 2024-09-11T23:40:17.395 INFO:teuthology.orchestra.run.smithi136.stdout: "seg": 133, 2024-09-11T23:40:17.396 INFO:teuthology.orchestra.run.smithi136.stdout: "segmjr": 10, 2024-09-11T23:40:17.396 INFO:teuthology.orchestra.run.smithi136.stdout: "segexg": 0, 2024-09-11T23:40:17.396 INFO:teuthology.orchestra.run.smithi136.stdout: "segexd": 0, 2024-09-11T23:40:17.396 INFO:teuthology.orchestra.run.smithi136.stdout: "expos": 33882821, 2024-09-11T23:40:17.396 INFO:teuthology.orchestra.run.smithi136.stdout: "wrpos": 338401870, 2024-09-11T23:40:17.396 INFO:teuthology.orchestra.run.smithi136.stdout: "rdpos": 338395998, 2024-09-11T23:40:17.396 INFO:teuthology.orchestra.run.smithi136.stdout: "jlat": { 2024-09-11T23:40:17.396 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-11T23:40:17.396 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-11T23:40:17.396 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-11T23:40:17.396 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:17.397 INFO:teuthology.orchestra.run.smithi136.stdout: "replayed": 104390 2024-09-11T23:40:17.397 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:17.397 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_mem": { 2024-09-11T23:40:17.397 INFO:teuthology.orchestra.run.smithi136.stdout: "ino": 17245, 2024-09-11T23:40:17.397 INFO:teuthology.orchestra.run.smithi136.stdout: "ino+": 70228, 2024-09-11T23:40:17.397 INFO:teuthology.orchestra.run.smithi136.stdout: "ino-": 52983, 2024-09-11T23:40:17.397 INFO:teuthology.orchestra.run.smithi136.stdout: "dir": 582, 2024-09-11T23:40:17.397 INFO:teuthology.orchestra.run.smithi136.stdout: "dir+": 2121, 2024-09-11T23:40:17.397 INFO:teuthology.orchestra.run.smithi136.stdout: "dir-": 1539, 2024-09-11T23:40:17.397 INFO:teuthology.orchestra.run.smithi136.stdout: "dn": 44501, 2024-09-11T23:40:17.397 INFO:teuthology.orchestra.run.smithi136.stdout: "dn+": 112037, 2024-09-11T23:40:17.397 INFO:teuthology.orchestra.run.smithi136.stdout: "dn-": 67536, 2024-09-11T23:40:17.398 INFO:teuthology.orchestra.run.smithi136.stdout: "cap": 0, 2024-09-11T23:40:17.398 INFO:teuthology.orchestra.run.smithi136.stdout: "cap+": 0, 2024-09-11T23:40:17.398 INFO:teuthology.orchestra.run.smithi136.stdout: "cap-": 0, 2024-09-11T23:40:17.398 INFO:teuthology.orchestra.run.smithi136.stdout: "rss": 159200, 2024-09-11T23:40:17.398 INFO:teuthology.orchestra.run.smithi136.stdout: "heap": 215328 2024-09-11T23:40:17.398 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:17.398 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_server": { 2024-09-11T23:40:17.398 INFO:teuthology.orchestra.run.smithi136.stdout: "dispatch_client_request": 0, 2024-09-11T23:40:17.398 INFO:teuthology.orchestra.run.smithi136.stdout: "dispatch_server_request": 0, 2024-09-11T23:40:17.398 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_client_request": 0, 2024-09-11T23:40:17.398 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_client_session": 0, 2024-09-11T23:40:17.398 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_peer_request": 0, 2024-09-11T23:40:17.398 INFO:teuthology.orchestra.run.smithi136.stdout: "req_create_latency": { 2024-09-11T23:40:17.399 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-11T23:40:17.399 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-11T23:40:17.399 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-11T23:40:17.399 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:17.399 INFO:teuthology.orchestra.run.smithi136.stdout: "req_getattr_latency": { 2024-09-11T23:40:17.399 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-11T23:40:17.399 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-11T23:40:17.399 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-11T23:40:17.399 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:17.399 INFO:teuthology.orchestra.run.smithi136.stdout: "req_getfilelock_latency": { 2024-09-11T23:40:17.399 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-11T23:40:17.399 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-11T23:40:17.400 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-11T23:40:17.400 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:17.400 INFO:teuthology.orchestra.run.smithi136.stdout: "req_link_latency": { 2024-09-11T23:40:17.400 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-11T23:40:17.400 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-11T23:40:17.400 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-11T23:40:17.400 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:17.400 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lookup_latency": { 2024-09-11T23:40:17.400 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-11T23:40:17.400 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-11T23:40:17.400 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-11T23:40:17.400 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:17.400 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lookuphash_latency": { 2024-09-11T23:40:17.400 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-11T23:40:17.400 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-11T23:40:17.401 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-11T23:40:17.401 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:17.401 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lookupino_latency": { 2024-09-11T23:40:17.401 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-11T23:40:17.401 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-11T23:40:17.401 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-11T23:40:17.401 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:17.401 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lookupname_latency": { 2024-09-11T23:40:17.401 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-11T23:40:17.401 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-11T23:40:17.401 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-11T23:40:17.401 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:17.401 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lookupparent_latency": { 2024-09-11T23:40:17.402 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-11T23:40:17.402 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-11T23:40:17.402 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-11T23:40:17.402 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:17.402 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lookupsnap_latency": { 2024-09-11T23:40:17.402 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-11T23:40:17.402 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-11T23:40:17.402 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-11T23:40:17.402 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:17.402 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lssnap_latency": { 2024-09-11T23:40:17.402 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-11T23:40:17.402 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-11T23:40:17.402 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-11T23:40:17.403 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:17.403 INFO:teuthology.orchestra.run.smithi136.stdout: "req_mkdir_latency": { 2024-09-11T23:40:17.403 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-11T23:40:17.403 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-11T23:40:17.403 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-11T23:40:17.403 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:17.403 INFO:teuthology.orchestra.run.smithi136.stdout: "req_mknod_latency": { 2024-09-11T23:40:17.403 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-11T23:40:17.403 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-11T23:40:17.404 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-11T23:40:17.404 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:17.404 INFO:teuthology.orchestra.run.smithi136.stdout: "req_mksnap_latency": { 2024-09-11T23:40:17.404 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-11T23:40:17.404 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-11T23:40:17.404 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-11T23:40:17.404 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:17.404 INFO:teuthology.orchestra.run.smithi136.stdout: "req_open_latency": { 2024-09-11T23:40:17.404 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-11T23:40:17.404 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-11T23:40:17.404 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-11T23:40:17.404 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:17.404 INFO:teuthology.orchestra.run.smithi136.stdout: "req_readdir_latency": { 2024-09-11T23:40:17.404 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-11T23:40:17.405 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-11T23:40:17.405 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-11T23:40:17.405 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:17.405 INFO:teuthology.orchestra.run.smithi136.stdout: "req_rename_latency": { 2024-09-11T23:40:17.405 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-11T23:40:17.405 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-11T23:40:17.405 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-11T23:40:17.405 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:17.405 INFO:teuthology.orchestra.run.smithi136.stdout: "req_renamesnap_latency": { 2024-09-11T23:40:17.405 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-11T23:40:17.405 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-11T23:40:17.405 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-11T23:40:17.405 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:17.405 INFO:teuthology.orchestra.run.smithi136.stdout: "req_snapdiff_latency": { 2024-09-11T23:40:17.406 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-11T23:40:17.406 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-11T23:40:17.406 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-11T23:40:17.406 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:17.406 INFO:teuthology.orchestra.run.smithi136.stdout: "req_rmdir_latency": { 2024-09-11T23:40:17.406 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-11T23:40:17.406 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-11T23:40:17.406 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-11T23:40:17.406 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:17.406 INFO:teuthology.orchestra.run.smithi136.stdout: "req_rmsnap_latency": { 2024-09-11T23:40:17.406 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-11T23:40:17.406 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-11T23:40:17.406 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-11T23:40:17.406 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:17.406 INFO:teuthology.orchestra.run.smithi136.stdout: "req_rmxattr_latency": { 2024-09-11T23:40:17.407 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-11T23:40:17.407 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-11T23:40:17.407 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-11T23:40:17.407 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:17.407 INFO:teuthology.orchestra.run.smithi136.stdout: "req_setattr_latency": { 2024-09-11T23:40:17.407 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-11T23:40:17.407 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-11T23:40:17.407 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-11T23:40:17.407 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:17.407 INFO:teuthology.orchestra.run.smithi136.stdout: "req_setdirlayout_latency": { 2024-09-11T23:40:17.407 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-11T23:40:17.408 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-11T23:40:17.408 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-11T23:40:17.408 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:17.408 INFO:teuthology.orchestra.run.smithi136.stdout: "req_setfilelock_latency": { 2024-09-11T23:40:17.408 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-11T23:40:17.408 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-11T23:40:17.408 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-11T23:40:17.408 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:17.408 INFO:teuthology.orchestra.run.smithi136.stdout: "req_setlayout_latency": { 2024-09-11T23:40:17.408 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-11T23:40:17.408 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-11T23:40:17.408 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-11T23:40:17.408 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:17.408 INFO:teuthology.orchestra.run.smithi136.stdout: "req_setxattr_latency": { 2024-09-11T23:40:17.409 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-11T23:40:17.409 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-11T23:40:17.409 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-11T23:40:17.409 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:17.409 INFO:teuthology.orchestra.run.smithi136.stdout: "req_symlink_latency": { 2024-09-11T23:40:17.409 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-11T23:40:17.409 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-11T23:40:17.409 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-11T23:40:17.409 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:17.409 INFO:teuthology.orchestra.run.smithi136.stdout: "req_unlink_latency": { 2024-09-11T23:40:17.409 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-11T23:40:17.409 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-11T23:40:17.410 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-11T23:40:17.410 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:17.410 INFO:teuthology.orchestra.run.smithi136.stdout: "cap_revoke_eviction": 0, 2024-09-11T23:40:17.410 INFO:teuthology.orchestra.run.smithi136.stdout: "cap_acquisition_throttle": 0, 2024-09-11T23:40:17.410 INFO:teuthology.orchestra.run.smithi136.stdout: "req_getvxattr_latency": { 2024-09-11T23:40:17.410 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-11T23:40:17.410 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-11T23:40:17.410 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-11T23:40:17.410 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-11T23:40:17.410 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:17.410 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_sessions": { 2024-09-11T23:40:17.410 INFO:teuthology.orchestra.run.smithi136.stdout: "session_count": 2, 2024-09-11T23:40:17.410 INFO:teuthology.orchestra.run.smithi136.stdout: "session_add": 2, 2024-09-11T23:40:17.410 INFO:teuthology.orchestra.run.smithi136.stdout: "session_remove": 0, 2024-09-11T23:40:17.410 INFO:teuthology.orchestra.run.smithi136.stdout: "sessions_open": 2, 2024-09-11T23:40:17.411 INFO:teuthology.orchestra.run.smithi136.stdout: "sessions_stale": 0, 2024-09-11T23:40:17.411 INFO:teuthology.orchestra.run.smithi136.stdout: "total_load": 0, 2024-09-11T23:40:17.411 INFO:teuthology.orchestra.run.smithi136.stdout: "average_load": 0, 2024-09-11T23:40:17.411 INFO:teuthology.orchestra.run.smithi136.stdout: "avg_session_uptime": 2554, 2024-09-11T23:40:17.411 INFO:teuthology.orchestra.run.smithi136.stdout: "mdthresh_evicted": 0 2024-09-11T23:40:17.411 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:17.411 INFO:teuthology.orchestra.run.smithi136.stdout: "mempool": { 2024-09-11T23:40:17.411 INFO:teuthology.orchestra.run.smithi136.stdout: "bloom_filter_bytes": 0, 2024-09-11T23:40:17.411 INFO:teuthology.orchestra.run.smithi136.stdout: "bloom_filter_items": 0, 2024-09-11T23:40:17.411 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_alloc_bytes": 0, 2024-09-11T23:40:17.411 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_alloc_items": 0, 2024-09-11T23:40:17.411 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_data_bytes": 0, 2024-09-11T23:40:17.411 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_data_items": 0, 2024-09-11T23:40:17.411 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-11T23:40:17.412 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_onode_items": 0, 2024-09-11T23:40:17.412 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-11T23:40:17.412 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_meta_items": 0, 2024-09-11T23:40:17.412 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_other_bytes": 0, 2024-09-11T23:40:17.412 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_other_items": 0, 2024-09-11T23:40:17.413 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-11T23:40:17.413 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_buffer_items": 0, 2024-09-11T23:40:17.413 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_extent_bytes": 0, 2024-09-11T23:40:17.413 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_extent_items": 0, 2024-09-11T23:40:17.413 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_blob_bytes": 0, 2024-09-11T23:40:17.414 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_blob_items": 0, 2024-09-11T23:40:17.414 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-11T23:40:17.414 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_shared_blob_items": 0, 2024-09-11T23:40:17.414 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-11T23:40:17.414 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_inline_bl_items": 0, 2024-09-11T23:40:17.414 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_fsck_bytes": 0, 2024-09-11T23:40:17.414 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_fsck_items": 0, 2024-09-11T23:40:17.414 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_txc_bytes": 0, 2024-09-11T23:40:17.414 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_txc_items": 0, 2024-09-11T23:40:17.414 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-11T23:40:17.414 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_writing_deferred_items": 0, 2024-09-11T23:40:17.414 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_writing_bytes": 0, 2024-09-11T23:40:17.414 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_writing_items": 0, 2024-09-11T23:40:17.415 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_bytes": 0, 2024-09-11T23:40:17.415 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_items": 0, 2024-09-11T23:40:17.415 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_file_reader_bytes": 0, 2024-09-11T23:40:17.416 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_file_reader_items": 0, 2024-09-11T23:40:17.416 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_file_writer_bytes": 0, 2024-09-11T23:40:17.416 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_file_writer_items": 0, 2024-09-11T23:40:17.416 INFO:teuthology.orchestra.run.smithi136.stdout: "buffer_anon_bytes": 887573, 2024-09-11T23:40:17.416 INFO:teuthology.orchestra.run.smithi136.stdout: "buffer_anon_items": 17557, 2024-09-11T23:40:17.416 INFO:teuthology.orchestra.run.smithi136.stdout: "buffer_meta_bytes": 0, 2024-09-11T23:40:17.416 INFO:teuthology.orchestra.run.smithi136.stdout: "buffer_meta_items": 0, 2024-09-11T23:40:17.416 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_bytes": 0, 2024-09-11T23:40:17.416 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_items": 0, 2024-09-11T23:40:17.416 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_mapbl_bytes": 0, 2024-09-11T23:40:17.416 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_mapbl_items": 0, 2024-09-11T23:40:17.416 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_pglog_bytes": 0, 2024-09-11T23:40:17.417 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_pglog_items": 0, 2024-09-11T23:40:17.417 INFO:teuthology.orchestra.run.smithi136.stdout: "osdmap_bytes": 8344, 2024-09-11T23:40:17.417 INFO:teuthology.orchestra.run.smithi136.stdout: "osdmap_items": 184, 2024-09-11T23:40:17.417 INFO:teuthology.orchestra.run.smithi136.stdout: "osdmap_mapping_bytes": 0, 2024-09-11T23:40:17.417 INFO:teuthology.orchestra.run.smithi136.stdout: "osdmap_mapping_items": 0, 2024-09-11T23:40:17.417 INFO:teuthology.orchestra.run.smithi136.stdout: "pgmap_bytes": 0, 2024-09-11T23:40:17.417 INFO:teuthology.orchestra.run.smithi136.stdout: "pgmap_items": 0, 2024-09-11T23:40:17.417 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_co_bytes": 72007359, 2024-09-11T23:40:17.417 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_co_items": 707865, 2024-09-11T23:40:17.417 INFO:teuthology.orchestra.run.smithi136.stdout: "unittest_1_bytes": 0, 2024-09-11T23:40:17.417 INFO:teuthology.orchestra.run.smithi136.stdout: "unittest_1_items": 0, 2024-09-11T23:40:17.418 INFO:teuthology.orchestra.run.smithi136.stdout: "unittest_2_bytes": 0, 2024-09-11T23:40:17.418 INFO:teuthology.orchestra.run.smithi136.stdout: "unittest_2_items": 0 2024-09-11T23:40:17.418 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:17.418 INFO:teuthology.orchestra.run.smithi136.stdout: "objecter": { 2024-09-11T23:40:17.418 INFO:teuthology.orchestra.run.smithi136.stdout: "op_active": 0, 2024-09-11T23:40:17.418 INFO:teuthology.orchestra.run.smithi136.stdout: "op_laggy": 0, 2024-09-11T23:40:17.418 INFO:teuthology.orchestra.run.smithi136.stdout: "op_send": 4735, 2024-09-11T23:40:17.418 INFO:teuthology.orchestra.run.smithi136.stdout: "op_send_bytes": 21, 2024-09-11T23:40:17.418 INFO:teuthology.orchestra.run.smithi136.stdout: "op_resend": 3, 2024-09-11T23:40:17.418 INFO:teuthology.orchestra.run.smithi136.stdout: "op_reply": 4735, 2024-09-11T23:40:17.419 INFO:teuthology.orchestra.run.smithi136.stdout: "op_latency": { 2024-09-11T23:40:17.419 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 4735, 2024-09-11T23:40:17.419 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 29.313899021, 2024-09-11T23:40:17.419 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.006190897 2024-09-11T23:40:17.419 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:17.419 INFO:teuthology.orchestra.run.smithi136.stdout: "op_inflight": 0, 2024-09-11T23:40:17.419 INFO:teuthology.orchestra.run.smithi136.stdout: "oplen_avg": { 2024-09-11T23:40:17.419 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 4735, 2024-09-11T23:40:17.419 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 4736 2024-09-11T23:40:17.420 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:17.420 INFO:teuthology.orchestra.run.smithi136.stdout: "op": 4735, 2024-09-11T23:40:17.420 INFO:teuthology.orchestra.run.smithi136.stdout: "op_r": 4735, 2024-09-11T23:40:17.420 INFO:teuthology.orchestra.run.smithi136.stdout: "op_w": 0, 2024-09-11T23:40:17.420 INFO:teuthology.orchestra.run.smithi136.stdout: "op_rmw": 0, 2024-09-11T23:40:17.420 INFO:teuthology.orchestra.run.smithi136.stdout: "op_pg": 0, 2024-09-11T23:40:17.420 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_stat": 2953, 2024-09-11T23:40:17.420 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_create": 0, 2024-09-11T23:40:17.420 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_read": 1780, 2024-09-11T23:40:17.420 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_write": 0, 2024-09-11T23:40:17.420 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_writefull": 0, 2024-09-11T23:40:17.420 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_writesame": 0, 2024-09-11T23:40:17.420 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_append": 0, 2024-09-11T23:40:17.421 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_zero": 0, 2024-09-11T23:40:17.421 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_truncate": 0, 2024-09-11T23:40:17.421 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_delete": 0, 2024-09-11T23:40:17.421 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_mapext": 0, 2024-09-11T23:40:17.421 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_sparse_read": 0, 2024-09-11T23:40:17.421 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_clonerange": 0, 2024-09-11T23:40:17.421 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_getxattr": 1, 2024-09-11T23:40:17.422 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_setxattr": 0, 2024-09-11T23:40:17.422 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_cmpxattr": 0, 2024-09-11T23:40:17.422 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_rmxattr": 0, 2024-09-11T23:40:17.422 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_resetxattrs": 0, 2024-09-11T23:40:17.422 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_call": 0, 2024-09-11T23:40:17.422 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_watch": 0, 2024-09-11T23:40:17.423 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_notify": 0, 2024-09-11T23:40:17.423 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_src_cmpxattr": 0, 2024-09-11T23:40:17.423 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_pgls": 0, 2024-09-11T23:40:17.423 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_pgls_filter": 0, 2024-09-11T23:40:17.423 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_other": 0, 2024-09-11T23:40:17.423 INFO:teuthology.orchestra.run.smithi136.stdout: "linger_active": 0, 2024-09-11T23:40:17.423 INFO:teuthology.orchestra.run.smithi136.stdout: "linger_send": 0, 2024-09-11T23:40:17.423 INFO:teuthology.orchestra.run.smithi136.stdout: "linger_resend": 0, 2024-09-11T23:40:17.423 INFO:teuthology.orchestra.run.smithi136.stdout: "linger_ping": 0, 2024-09-11T23:40:17.423 INFO:teuthology.orchestra.run.smithi136.stdout: "poolop_active": 0, 2024-09-11T23:40:17.423 INFO:teuthology.orchestra.run.smithi136.stdout: "poolop_send": 0, 2024-09-11T23:40:17.423 INFO:teuthology.orchestra.run.smithi136.stdout: "poolop_resend": 0, 2024-09-11T23:40:17.424 INFO:teuthology.orchestra.run.smithi136.stdout: "poolstat_active": 0, 2024-09-11T23:40:17.424 INFO:teuthology.orchestra.run.smithi136.stdout: "poolstat_send": 0, 2024-09-11T23:40:17.424 INFO:teuthology.orchestra.run.smithi136.stdout: "poolstat_resend": 0, 2024-09-11T23:40:17.424 INFO:teuthology.orchestra.run.smithi136.stdout: "statfs_active": 0, 2024-09-11T23:40:17.424 INFO:teuthology.orchestra.run.smithi136.stdout: "statfs_send": 0, 2024-09-11T23:40:17.424 INFO:teuthology.orchestra.run.smithi136.stdout: "statfs_resend": 0, 2024-09-11T23:40:17.424 INFO:teuthology.orchestra.run.smithi136.stdout: "command_active": 0, 2024-09-11T23:40:17.424 INFO:teuthology.orchestra.run.smithi136.stdout: "command_send": 0, 2024-09-11T23:40:17.424 INFO:teuthology.orchestra.run.smithi136.stdout: "command_resend": 0, 2024-09-11T23:40:17.425 INFO:teuthology.orchestra.run.smithi136.stdout: "map_epoch": 117, 2024-09-11T23:40:17.425 INFO:teuthology.orchestra.run.smithi136.stdout: "map_full": 0, 2024-09-11T23:40:17.425 INFO:teuthology.orchestra.run.smithi136.stdout: "map_inc": 32, 2024-09-11T23:40:17.425 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_sessions": 11, 2024-09-11T23:40:17.425 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_session_open": 11, 2024-09-11T23:40:17.425 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_session_close": 0, 2024-09-11T23:40:17.425 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_laggy": 0, 2024-09-11T23:40:17.425 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_wr": 0, 2024-09-11T23:40:17.425 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_rd": 2, 2024-09-11T23:40:17.425 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_del": 0 2024-09-11T23:40:17.425 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:17.425 INFO:teuthology.orchestra.run.smithi136.stdout: "oft": { 2024-09-11T23:40:17.425 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_total_objs": 0, 2024-09-11T23:40:17.426 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_total_kv_pairs": 0, 2024-09-11T23:40:17.426 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_total_updates": 0, 2024-09-11T23:40:17.426 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_total_removes": 0 2024-09-11T23:40:17.426 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:17.426 INFO:teuthology.orchestra.run.smithi136.stdout: "osd-slow-ops": { 2024-09-11T23:40:17.426 INFO:teuthology.orchestra.run.smithi136.stdout: "slow_ops_count": 0 2024-09-11T23:40:17.426 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:17.426 INFO:teuthology.orchestra.run.smithi136.stdout: "purge_queue": { 2024-09-11T23:40:17.426 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_executing_ops": 0, 2024-09-11T23:40:17.426 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_executing_ops_high_water": 0, 2024-09-11T23:40:17.426 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_executing": 0, 2024-09-11T23:40:17.426 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_executing_high_water": 0, 2024-09-11T23:40:17.426 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_executed_ops": 0, 2024-09-11T23:40:17.426 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_executed": 0, 2024-09-11T23:40:17.426 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_item_in_journal": 0 2024-09-11T23:40:17.426 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:17.427 INFO:teuthology.orchestra.run.smithi136.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-11T23:40:17.427 INFO:teuthology.orchestra.run.smithi136.stdout: "val": 0, 2024-09-11T23:40:17.427 INFO:teuthology.orchestra.run.smithi136.stdout: "max": 104857600, 2024-09-11T23:40:17.427 INFO:teuthology.orchestra.run.smithi136.stdout: "get_started": 0, 2024-09-11T23:40:17.427 INFO:teuthology.orchestra.run.smithi136.stdout: "get": 5242, 2024-09-11T23:40:17.427 INFO:teuthology.orchestra.run.smithi136.stdout: "get_sum": 337183699, 2024-09-11T23:40:17.427 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_fail": 0, 2024-09-11T23:40:17.427 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_success": 5242, 2024-09-11T23:40:17.427 INFO:teuthology.orchestra.run.smithi136.stdout: "take": 0, 2024-09-11T23:40:17.427 INFO:teuthology.orchestra.run.smithi136.stdout: "take_sum": 0, 2024-09-11T23:40:17.427 INFO:teuthology.orchestra.run.smithi136.stdout: "put": 5242, 2024-09-11T23:40:17.427 INFO:teuthology.orchestra.run.smithi136.stdout: "put_sum": 337183699, 2024-09-11T23:40:17.427 INFO:teuthology.orchestra.run.smithi136.stdout: "wait": { 2024-09-11T23:40:17.427 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-11T23:40:17.428 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-11T23:40:17.428 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-11T23:40:17.428 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-11T23:40:17.428 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:17.428 INFO:teuthology.orchestra.run.smithi136.stdout: "throttle-objecter_bytes": { 2024-09-11T23:40:17.428 INFO:teuthology.orchestra.run.smithi136.stdout: "val": 0, 2024-09-11T23:40:17.428 INFO:teuthology.orchestra.run.smithi136.stdout: "max": 104857600, 2024-09-11T23:40:17.428 INFO:teuthology.orchestra.run.smithi136.stdout: "get_started": 0, 2024-09-11T23:40:17.428 INFO:teuthology.orchestra.run.smithi136.stdout: "get": 0, 2024-09-11T23:40:17.428 INFO:teuthology.orchestra.run.smithi136.stdout: "get_sum": 0, 2024-09-11T23:40:17.428 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_fail": 0, 2024-09-11T23:40:17.428 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_success": 0, 2024-09-11T23:40:17.428 INFO:teuthology.orchestra.run.smithi136.stdout: "take": 4735, 2024-09-11T23:40:17.428 INFO:teuthology.orchestra.run.smithi136.stdout: "take_sum": 334250370, 2024-09-11T23:40:17.428 INFO:teuthology.orchestra.run.smithi136.stdout: "put": 4735, 2024-09-11T23:40:17.429 INFO:teuthology.orchestra.run.smithi136.stdout: "put_sum": 334250370, 2024-09-11T23:40:17.429 INFO:teuthology.orchestra.run.smithi136.stdout: "wait": { 2024-09-11T23:40:17.429 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-11T23:40:17.429 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-11T23:40:17.429 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-11T23:40:17.429 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-11T23:40:17.429 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:17.429 INFO:teuthology.orchestra.run.smithi136.stdout: "throttle-objecter_ops": { 2024-09-11T23:40:17.429 INFO:teuthology.orchestra.run.smithi136.stdout: "val": 0, 2024-09-11T23:40:17.429 INFO:teuthology.orchestra.run.smithi136.stdout: "max": 1024, 2024-09-11T23:40:17.429 INFO:teuthology.orchestra.run.smithi136.stdout: "get_started": 0, 2024-09-11T23:40:17.429 INFO:teuthology.orchestra.run.smithi136.stdout: "get": 0, 2024-09-11T23:40:17.429 INFO:teuthology.orchestra.run.smithi136.stdout: "get_sum": 0, 2024-09-11T23:40:17.429 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_fail": 0, 2024-09-11T23:40:17.429 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_success": 0, 2024-09-11T23:40:17.430 INFO:teuthology.orchestra.run.smithi136.stdout: "take": 4735, 2024-09-11T23:40:17.430 INFO:teuthology.orchestra.run.smithi136.stdout: "take_sum": 4735, 2024-09-11T23:40:17.430 INFO:teuthology.orchestra.run.smithi136.stdout: "put": 4735, 2024-09-11T23:40:17.430 INFO:teuthology.orchestra.run.smithi136.stdout: "put_sum": 4735, 2024-09-11T23:40:17.430 INFO:teuthology.orchestra.run.smithi136.stdout: "wait": { 2024-09-11T23:40:17.430 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-11T23:40:17.430 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-11T23:40:17.430 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-11T23:40:17.430 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-11T23:40:17.430 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:17.430 INFO:teuthology.orchestra.run.smithi136.stdout: "throttle-write_buf_throttle": { 2024-09-11T23:40:17.431 INFO:teuthology.orchestra.run.smithi136.stdout: "val": 0, 2024-09-11T23:40:17.431 INFO:teuthology.orchestra.run.smithi136.stdout: "max": 3758096384, 2024-09-11T23:40:17.431 INFO:teuthology.orchestra.run.smithi136.stdout: "get_started": 0, 2024-09-11T23:40:17.431 INFO:teuthology.orchestra.run.smithi136.stdout: "get": 0, 2024-09-11T23:40:17.431 INFO:teuthology.orchestra.run.smithi136.stdout: "get_sum": 0, 2024-09-11T23:40:17.431 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_fail": 0, 2024-09-11T23:40:17.431 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_success": 0, 2024-09-11T23:40:17.431 INFO:teuthology.orchestra.run.smithi136.stdout: "take": 0, 2024-09-11T23:40:17.431 INFO:teuthology.orchestra.run.smithi136.stdout: "take_sum": 0, 2024-09-11T23:40:17.431 INFO:teuthology.orchestra.run.smithi136.stdout: "put": 0, 2024-09-11T23:40:17.431 INFO:teuthology.orchestra.run.smithi136.stdout: "put_sum": 0, 2024-09-11T23:40:17.431 INFO:teuthology.orchestra.run.smithi136.stdout: "wait": { 2024-09-11T23:40:17.431 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-11T23:40:17.431 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-11T23:40:17.432 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-11T23:40:17.432 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-11T23:40:17.432 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-11T23:40:17.432 INFO:teuthology.orchestra.run.smithi136.stdout: "throttle-write_buf_throttle-0x559b84fe2b40": { 2024-09-11T23:40:17.432 INFO:teuthology.orchestra.run.smithi136.stdout: "val": 0, 2024-09-11T23:40:17.432 INFO:teuthology.orchestra.run.smithi136.stdout: "max": 3758096384, 2024-09-11T23:40:17.432 INFO:teuthology.orchestra.run.smithi136.stdout: "get_started": 0, 2024-09-11T23:40:17.432 INFO:teuthology.orchestra.run.smithi136.stdout: "get": 0, 2024-09-11T23:40:17.432 INFO:teuthology.orchestra.run.smithi136.stdout: "get_sum": 0, 2024-09-11T23:40:17.432 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_fail": 0, 2024-09-11T23:40:17.432 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_success": 0, 2024-09-11T23:40:17.432 INFO:teuthology.orchestra.run.smithi136.stdout: "take": 0, 2024-09-11T23:40:17.432 INFO:teuthology.orchestra.run.smithi136.stdout: "take_sum": 0, 2024-09-11T23:40:17.432 INFO:teuthology.orchestra.run.smithi136.stdout: "put": 0, 2024-09-11T23:40:17.432 INFO:teuthology.orchestra.run.smithi136.stdout: "put_sum": 0, 2024-09-11T23:40:17.432 INFO:teuthology.orchestra.run.smithi136.stdout: "wait": { 2024-09-11T23:40:17.433 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-11T23:40:17.433 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-11T23:40:17.433 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-11T23:40:17.433 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-11T23:40:17.433 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-11T23:40:17.433 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-11T23:40:17.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:40:17 smithi136 ceph-mon[35121]: pgmap v691: 97 pgs: 41 active+clean+snaptrim_wait, 23 active+clean+snaptrim, 33 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2024-09-11T23:40:18.000 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:40:17 smithi026 ceph-mon[31172]: pgmap v691: 97 pgs: 41 active+clean+snaptrim_wait, 23 active+clean+snaptrim, 33 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2024-09-11T23:40:18.052 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:40:17 smithi076 ceph-mon[35296]: pgmap v691: 97 pgs: 41 active+clean+snaptrim_wait, 23 active+clean+snaptrim, 33 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2024-09-11T23:40:19.218 INFO:tasks.check_counter:Daemon mds.l mds.exported=0 2024-09-11T23:40:19.219 INFO:tasks.check_counter:Daemon mds.l mds.imported=0 2024-09-11T23:40:19.219 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update=0 2024-09-11T23:40:19.219 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update_receipt=0 2024-09-11T23:40:19.219 INFO:tasks.check_counter:Daemon mds.l mds.root_rsnaps=0 2024-09-11T23:40:19.219 INFO:tasks.check_counter:Daemon mds.l mds_server.req_mksnap_latency.avgcount=0 2024-09-11T23:40:19.219 INFO:tasks.check_counter:Daemon mds.l mds_server.req_rmsnap_latency.avgcount=0 2024-09-11T23:40:19.219 INFO:tasks.check_counter:Daemon mds.l mds.dir_split=0 2024-09-11T23:40:19.219 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-09-11T23:40:19.231 INFO:tasks.kclient:Unmounting kernel clients... 2024-09-11T23:40:19.231 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-09-11T23:40:19.231 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T23:40:19.267 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-09-11T23:40:19.268 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T23:40:19.345 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph osd blocklist ls 2024-09-11T23:40:19.680 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:40:19 smithi026 ceph-mon[31172]: pgmap v692: 97 pgs: 41 active+clean+snaptrim_wait, 23 active+clean+snaptrim, 33 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2024-09-11T23:40:19.862 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:40:19.988 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:40:19 smithi136 ceph-mon[35121]: pgmap v692: 97 pgs: 41 active+clean+snaptrim_wait, 23 active+clean+snaptrim, 33 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2024-09-11T23:40:20.052 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:40:19 smithi076 ceph-mon[35296]: pgmap v692: 97 pgs: 41 active+clean+snaptrim_wait, 23 active+clean+snaptrim, 33 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2024-09-11T23:40:20.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:20.698+0000 7f824137a640 1 -- 172.21.15.26:0/1265085840 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f822c040860 con 0x7f823c11b140 2024-09-11T23:40:20.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:20.698+0000 7f82445e0640 1 -- 172.21.15.26:0/1265085840 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f823c11b140 msgr2=0x7f823c11d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:20.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:20.698+0000 7f82445e0640 1 --2- 172.21.15.26:0/1265085840 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f823c11b140 0x7f823c11d530 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f822c01a910 tx=0x7f822c03b470 comp rx=0 tx=0).stop 2024-09-11T23:40:20.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:20.699+0000 7f82445e0640 1 -- 172.21.15.26:0/1265085840 shutdown_connections 2024-09-11T23:40:20.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:20.699+0000 7f82445e0640 1 --2- 172.21.15.26:0/1265085840 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f823c11b140 0x7f823c11d530 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:20.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:20.699+0000 7f82445e0640 1 --2- 172.21.15.26:0/1265085840 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f823c109830 0x7f823c109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:20.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:20.699+0000 7f82445e0640 1 --2- 172.21.15.26:0/1265085840 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f823c109450 0x7f823c110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:20.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:20.699+0000 7f82445e0640 1 -- 172.21.15.26:0/1265085840 >> 172.21.15.26:0/1265085840 conn(0x7f823c06e140 msgr2=0x7f823c06e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:40:20.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:20.701+0000 7f82445e0640 1 -- 172.21.15.26:0/1265085840 shutdown_connections 2024-09-11T23:40:20.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:20.701+0000 7f82445e0640 1 -- 172.21.15.26:0/1265085840 wait complete. 2024-09-11T23:40:20.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:20.702+0000 7f82445e0640 1 Processor -- start 2024-09-11T23:40:20.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:20.702+0000 7f82445e0640 1 -- start start 2024-09-11T23:40:20.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:20.703+0000 7f82445e0640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f823c109450 0x7f823c11a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:20.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:20.703+0000 7f82445e0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f823c109830 0x7f823c1169c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:20.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:20.703+0000 7f82445e0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f823c11b140 0x7f823c116f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:20.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:20.703+0000 7f82445e0640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f823c1133e0 con 0x7f823c11b140 2024-09-11T23:40:20.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:20.703+0000 7f82445e0640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f823c113260 con 0x7f823c109830 2024-09-11T23:40:20.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:20.703+0000 7f82445e0640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f823c113560 con 0x7f823c109450 2024-09-11T23:40:20.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:20.703+0000 7f8242b7d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f823c11b140 0x7f823c116f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:20.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:20.703+0000 7f8242b7d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f823c11b140 0x7f823c116f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47296/0 (socket says 172.21.15.26:47296) 2024-09-11T23:40:20.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:20.703+0000 7f8242b7d640 1 -- 172.21.15.26:0/1010945727 learned_addr learned my addr 172.21.15.26:0/1010945727 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:40:20.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:20.704+0000 7f8241b7b640 1 --2- 172.21.15.26:0/1010945727 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f823c109830 0x7f823c1169c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:20.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:20.704+0000 7f8242b7d640 1 -- 172.21.15.26:0/1010945727 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f823c109450 msgr2=0x7f823c11a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:20.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:20.704+0000 7f8242b7d640 1 --2- 172.21.15.26:0/1010945727 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f823c109450 0x7f823c11a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:20.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:20.704+0000 7f8242b7d640 1 -- 172.21.15.26:0/1010945727 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f823c109830 msgr2=0x7f823c1169c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:20.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:20.704+0000 7f8242b7d640 1 --2- 172.21.15.26:0/1010945727 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f823c109830 0x7f823c1169c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:20.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:20.704+0000 7f8242b7d640 1 -- 172.21.15.26:0/1010945727 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f823c117680 con 0x7f823c11b140 2024-09-11T23:40:20.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:20.704+0000 7f8242b7d640 1 --2- 172.21.15.26:0/1010945727 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f823c11b140 0x7f823c116f00 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f822c004da0 tx=0x7f822c004dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:40:20.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:20.705+0000 7f82337fe640 1 -- 172.21.15.26:0/1010945727 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f822c012030 con 0x7f823c11b140 2024-09-11T23:40:20.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:20.705+0000 7f82337fe640 1 -- 172.21.15.26:0/1010945727 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f822c040820 con 0x7f823c11b140 2024-09-11T23:40:20.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:20.706+0000 7f82337fe640 1 -- 172.21.15.26:0/1010945727 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f822c046030 con 0x7f823c11b140 2024-09-11T23:40:20.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:20.706+0000 7f82445e0640 1 -- 172.21.15.26:0/1010945727 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f823c1cd180 con 0x7f823c11b140 2024-09-11T23:40:20.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:20.707+0000 7f82445e0640 1 -- 172.21.15.26:0/1010945727 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f823c1cd570 con 0x7f823c11b140 2024-09-11T23:40:20.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:20.708+0000 7f82445e0640 1 -- 172.21.15.26:0/1010945727 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f823c073b00 con 0x7f823c11b140 2024-09-11T23:40:20.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:20.714+0000 7f82337fe640 1 -- 172.21.15.26:0/1010945727 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f822c015070 con 0x7f823c11b140 2024-09-11T23:40:20.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:20.716+0000 7f82337fe640 1 --2- 172.21.15.26:0/1010945727 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f82280a2da0 0x7f82280a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:20.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:20.716+0000 7f82337fe640 1 -- 172.21.15.26:0/1010945727 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 8586+0+0 (secure 0 0 0) 0x7f822c0ef8c0 con 0x7f823c11b140 2024-09-11T23:40:20.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:20.720+0000 7f824237c640 1 --2- 172.21.15.26:0/1010945727 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f82280a2da0 0x7f82280a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:20.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:20.722+0000 7f824237c640 1 --2- 172.21.15.26:0/1010945727 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f82280a2da0 0x7f82280a5260 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f823400ab90 tx=0x7f8234009230 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:40:20.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:20.725+0000 7f82337fe640 1 -- 172.21.15.26:0/1010945727 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f822c0bc440 con 0x7f823c11b140 2024-09-11T23:40:21.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:21.090+0000 7f82445e0640 1 -- 172.21.15.26:0/1010945727 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f823c10d640 con 0x7f823c11b140 2024-09-11T23:40:21.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:21.091+0000 7f82337fe640 1 -- 172.21.15.26:0/1010945727 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v117) ==== 81+0+939 (secure 0 0 0) 0x7f822c01a100 con 0x7f823c11b140 2024-09-11T23:40:21.095 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.26:0/2626831531 2024-09-12T23:18:22.177934+0000 2024-09-11T23:40:21.095 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.26:0/3861103130 2024-09-12T23:09:09.781665+0000 2024-09-11T23:40:21.095 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.26:0/17142658 2024-09-12T23:09:09.781665+0000 2024-09-11T23:40:21.095 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.26:0/2817092653 2024-09-12T23:09:09.781665+0000 2024-09-11T23:40:21.095 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.26:6801/110833859 2024-09-12T23:18:22.177934+0000 2024-09-11T23:40:21.095 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.26:0/2107281413 2024-09-12T23:09:33.983310+0000 2024-09-11T23:40:21.095 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.26:0/883212708 2024-09-12T23:09:33.983310+0000 2024-09-11T23:40:21.095 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.26:6800/2673575521 2024-09-12T23:09:09.781665+0000 2024-09-11T23:40:21.096 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.26:0/2559066894 2024-09-12T23:18:22.177934+0000 2024-09-11T23:40:21.096 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.26:6801/2673575521 2024-09-12T23:09:09.781665+0000 2024-09-11T23:40:21.096 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.26:0/3597996003 2024-09-12T23:09:33.983310+0000 2024-09-11T23:40:21.096 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.26:6800/677148151 2024-09-12T23:09:33.983310+0000 2024-09-11T23:40:21.096 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.26:6801/677148151 2024-09-12T23:09:33.983310+0000 2024-09-11T23:40:21.096 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.26:0/1584355185 2024-09-12T23:18:22.177934+0000 2024-09-11T23:40:21.096 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.26:6800/110833859 2024-09-12T23:18:22.177934+0000 2024-09-11T23:40:21.096 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.26:0/1256290054 2024-09-12T23:18:22.177934+0000 2024-09-11T23:40:21.096 INFO:teuthology.orchestra.run.smithi026.stderr:listed 16 entries 2024-09-11T23:40:21.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:21.098+0000 7f82317fa640 1 -- 172.21.15.26:0/1010945727 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f82280a2da0 msgr2=0x7f82280a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:21.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:21.098+0000 7f82317fa640 1 --2- 172.21.15.26:0/1010945727 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f82280a2da0 0x7f82280a5260 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f823400ab90 tx=0x7f8234009230 comp rx=0 tx=0).stop 2024-09-11T23:40:21.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:21.098+0000 7f82317fa640 1 -- 172.21.15.26:0/1010945727 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f823c11b140 msgr2=0x7f823c116f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:21.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:21.098+0000 7f82317fa640 1 --2- 172.21.15.26:0/1010945727 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f823c11b140 0x7f823c116f00 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f822c004da0 tx=0x7f822c004dd0 comp rx=0 tx=0).stop 2024-09-11T23:40:21.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:21.098+0000 7f82317fa640 1 -- 172.21.15.26:0/1010945727 shutdown_connections 2024-09-11T23:40:21.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:21.098+0000 7f82317fa640 1 --2- 172.21.15.26:0/1010945727 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f82280a2da0 0x7f82280a5260 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:21.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:21.098+0000 7f82317fa640 1 --2- 172.21.15.26:0/1010945727 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f823c109830 0x7f823c1169c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:21.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:21.098+0000 7f82317fa640 1 --2- 172.21.15.26:0/1010945727 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f823c11b140 0x7f823c116f00 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:21.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:21.098+0000 7f82317fa640 1 --2- 172.21.15.26:0/1010945727 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f823c109450 0x7f823c11a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:21.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:21.098+0000 7f82317fa640 1 -- 172.21.15.26:0/1010945727 >> 172.21.15.26:0/1010945727 conn(0x7f823c06e140 msgr2=0x7f823c072290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:40:21.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:21.098+0000 7f82317fa640 1 -- 172.21.15.26:0/1010945727 shutdown_connections 2024-09-11T23:40:21.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:21.098+0000 7f82317fa640 1 -- 172.21.15.26:0/1010945727 wait complete. 2024-09-11T23:40:21.750 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:40:21 smithi026 ceph-mon[31172]: pgmap v693: 97 pgs: 41 active+clean+snaptrim_wait, 23 active+clean+snaptrim, 33 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 991 B/s wr, 10 op/s 2024-09-11T23:40:21.750 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:40:21 smithi026 ceph-mon[31172]: from='client.? 172.21.15.26:0/1010945727' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-11T23:40:21.988 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:40:21 smithi136 ceph-mon[35121]: pgmap v693: 97 pgs: 41 active+clean+snaptrim_wait, 23 active+clean+snaptrim, 33 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 991 B/s wr, 10 op/s 2024-09-11T23:40:21.988 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:40:21 smithi136 ceph-mon[35121]: from='client.? 172.21.15.26:0/1010945727' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-11T23:40:22.052 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:40:21 smithi076 ceph-mon[35296]: pgmap v693: 97 pgs: 41 active+clean+snaptrim_wait, 23 active+clean+snaptrim, 33 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 991 B/s wr, 10 op/s 2024-09-11T23:40:22.052 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:40:21 smithi076 ceph-mon[35296]: from='client.? 172.21.15.26:0/1010945727' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-11T23:40:22.222 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-11T23:40:22.222 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T23:40:22.223 DEBUG:teuthology.orchestra.run.smithi026:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-11T23:40:22.419 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi026.front.sepia.ceph.com 2024-09-11T23:40:22.419 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T23:40:22.419 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-11T23:40:22.477 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T23:40:22.478 DEBUG:teuthology.orchestra.run.smithi026:> ip netns list 2024-09-11T23:40:22.537 INFO:teuthology.orchestra.run.smithi026.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-11T23:40:22.537 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T23:40:22.538 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-11T23:40:22.622 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T23:40:22.622 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip link delete ceph-brx 2024-09-11T23:40:22.736 INFO:tasks.kclient:Unmounting kernel clients... 2024-09-11T23:40:22.737 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-09-11T23:40:22.737 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T23:40:22.781 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056795,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=174 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249348k,nr_inodes=812337,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/sda1 /var/lib/containers/storage/overlay ext4 rw,seclabel,relatime 0 0 overlay /var/lib/containers/storage/overlay/2549682ab3d1706404bd3f2b45632bbcfc18fabbfd4cf49ac6ddcfebf968e040/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/5BWN52IRDTWZ33COOOPEO6BFFZ:/var/lib/containers/storage/overlay/l/BSTYAE4MOEYMYY6VYO35NMQXF6,upperdir=/var/lib/containers/storage/overlay/2549682ab3d1706404bd3f2b45632bbcfc18fabbfd4cf49ac6ddcfebf968e040/diff,workdir=/var/lib/containers/storage/overlay/2549682ab3d1706404bd3f2b45632bbcfc18fabbfd4cf49ac6ddcfebf968e040/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/8e436ec0b516c0093aedbd98e12597e59ba10b583b870e847639a46cf7a5e526/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/5BWN52IRDTWZ33COOOPEO6BFFZ:/var/lib/containers/storage/overlay/l/BSTYAE4MOEYMYY6VYO35NMQXF6,upperdir=/var/lib/containers/storage/overlay/8e436ec0b516c0093aedbd98e12597e59ba10b583b870e847639a46cf7a5e526/diff,workdir=/var/lib/containers/storage/overlay/8e436ec0b516c0093aedbd98e12597e59ba10b583b870e847639a46cf7a5e526/work,metacopy=on,volatile 0 0 tmpfs /run/user/0 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249348k,nr_inodes=812337,mode=700,inode64 0 0 binfmt_misc /proc/sys/fs/binfmt_misc binfmt_misc rw,nosuid,nodev,noexec,relatime 0 0 overlay /var/lib/containers/storage/overlay/167773be0152f97d98eaf5f845d2bba689746e5df24e378ca5b8341d9d226473/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/5BWN52IRDTWZ33COOOPEO6BFFZ:/var/lib/containers/storage/overlay/l/BSTYAE4MOEYMYY6VYO35NMQXF6,upperdir=/var/lib/containers/storage/overlay/167773be0152f97d98eaf5f845d2bba689746e5df24e378ca5b8341d9d226473/diff,workdir=/var/lib/containers/storage/overlay/167773be0152f97d98eaf5f845d2bba689746e5df24e378ca5b8341d9d226473/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/3a1a36b114f745032795c76555642385d838303e073a9fe87bc5de66987a73aa/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/5BWN52IRDTWZ33COOOPEO6BFFZ:/var/lib/containers/storage/overlay/l/BSTYAE4MOEYMYY6VYO35NMQXF6,upperdir=/var/lib/containers/storage/overlay/3a1a36b114f745032795c76555642385d838303e073a9fe87bc5de66987a73aa/diff,workdir=/var/lib/containers/storage/overlay/3a1a36b114f745032795c76555642385d838303e073a9fe87bc5de66987a73aa/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/f0e17490801e4e2223b877b14dd5307401c8623c35ff44ec1d292d7cfb95d0dd/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/5BWN52IRDTWZ33COOOPEO6BFFZ:/var/lib/containers/storage/overlay/l/BSTYAE4MOEYMYY6VYO35NMQXF6,upperdir=/var/lib/containers/storage/overlay/f0e17490801e4e2223b877b14dd5307401c8623c35ff44ec1d292d7cfb95d0dd/diff,workdir=/var/lib/containers/storage/overlay/f0e17490801e4e2223b877b14dd5307401c8623c35ff44ec1d292d7cfb95d0dd/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/2be465475dcc6791350b407f063f86933d4d738a0b1856070684386c4d15d91c/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/5BWN52IRDTWZ33COOOPEO6BFFZ:/var/lib/containers/storage/overlay/l/BSTYAE4MOEYMYY6VYO35NMQXF6,upperdir=/var/lib/containers/storage/overlay/2be465475dcc6791350b407f063f86933d4d738a0b1856070684386c4d15d91c/diff,workdir=/var/lib/containers/storage/overlay/2be465475dcc6791350b407f063f86933d4d738a0b1856070684386c4d15d91c/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/d907ee587881fc67116cc36a3b01fe116040be6983bf769bca4d5776e165a3c0/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/5BWN52IRDTWZ33COOOPEO6BFFZ:/var/lib/containers/storage/overlay/l/BSTYAE4MOEYMYY6VYO35NMQXF6,upperdir=/var/lib/containers/storage/overlay/d907ee587881fc67116cc36a3b01fe116040be6983bf769bca4d5776e165a3c0/diff,workdir=/var/lib/containers/storage/overlay/d907ee587881fc67116cc36a3b01fe116040be6983bf769bca4d5776e165a3c0/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/822bfb5e432777f4d524300bba3308aa0c6dbb198390ed248998177c7161769b/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/5BWN52IRDTWZ33COOOPEO6BFFZ:/var/lib/containers/storage/overlay/l/BSTYAE4MOEYMYY6VYO35NMQXF6,upperdir=/var/lib/containers/storage/overlay/822bfb5e432777f4d524300bba3308aa0c6dbb198390ed248998177c7161769b/diff,workdir=/var/lib/containers/storage/overlay/822bfb5e432777f4d524300bba3308aa0c6dbb198390ed248998177c7161769b/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/a4ec8557f69d8b574425bac1eb0c32ff15f7c5d111d02274ad81ee41ef4e2cb1/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/5BWN52IRDTWZ33COOOPEO6BFFZ:/var/lib/containers/storage/overlay/l/BSTYAE4MOEYMYY6VYO35NMQXF6,upperdir=/var/lib/containers/storage/overlay/a4ec8557f69d8b574425bac1eb0c32ff15f7c5d111d02274ad81ee41ef4e2cb1/diff,workdir=/var/lib/containers/storage/overlay/a4ec8557f69d8b574425bac1eb0c32ff15f7c5d111d02274ad81ee41ef4e2cb1/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/073b36e749111da473a4f1b84d2f968bcc208f53468e8fc908ed811153bf89e5/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/5BWN52IRDTWZ33COOOPEO6BFFZ:/var/lib/containers/storage/overlay/l/BSTYAE4MOEYMYY6VYO35NMQXF6,upperdir=/var/lib/containers/storage/overlay/073b36e749111da473a4f1b84d2f968bcc208f53468e8fc908ed811153bf89e5/diff,workdir=/var/lib/containers/storage/overlay/073b36e749111da473a4f1b84d2f968bcc208f53468e8fc908ed811153bf89e5/work,metacopy=on,volatile 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 2024-09-11T23:40:22.781 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T23:40:22.781 DEBUG:teuthology.orchestra.run.smithi026:> ip netns list 2024-09-11T23:40:22.872 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T23:40:22.872 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip link delete ceph-brx 2024-09-11T23:40:22.940 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:40:22 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: 2024-09-11T23:40:22.680+0000 7f7ccf119640 -1 calc_pg_upmaps abort due to max <= 0 2024-09-11T23:40:22.957 INFO:teuthology.orchestra.run.smithi026.stderr:Cannot find device "ceph-brx" 2024-09-11T23:40:22.959 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T23:40:22.959 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-09-11T23:40:22.970 INFO:tasks.cephadm:Teardown begin 2024-09-11T23:40:22.971 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-11T23:40:22.971 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:40:23.614 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:40:23.795 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:40:23 smithi026 ceph-mon[31172]: pgmap v694: 97 pgs: 41 active+clean+snaptrim_wait, 23 active+clean+snaptrim, 33 active+clean; 2.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 1023 B/s wr, 11 op/s 2024-09-11T23:40:23.988 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:40:23 smithi136 ceph-mon[35121]: pgmap v694: 97 pgs: 41 active+clean+snaptrim_wait, 23 active+clean+snaptrim, 33 active+clean; 2.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 1023 B/s wr, 11 op/s 2024-09-11T23:40:24.052 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:40:23 smithi076 ceph-mon[35296]: pgmap v694: 97 pgs: 41 active+clean+snaptrim_wait, 23 active+clean+snaptrim, 33 active+clean; 2.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 1023 B/s wr, 11 op/s 2024-09-11T23:40:24.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:24.327+0000 7fdac759e640 1 -- 172.21.15.26:0/1504261285 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdac811b140 msgr2=0x7fdac811d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:24.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:24.327+0000 7fdac759e640 1 --2- 172.21.15.26:0/1504261285 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdac811b140 0x7fdac811d530 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7fdabc009900 tx=0x7fdabc02a440 comp rx=0 tx=0).stop 2024-09-11T23:40:24.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:24.329+0000 7fdac759e640 1 -- 172.21.15.26:0/1504261285 shutdown_connections 2024-09-11T23:40:24.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:24.329+0000 7fdac759e640 1 --2- 172.21.15.26:0/1504261285 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdac806ea30 0x7fdac806ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:24.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:24.329+0000 7fdac759e640 1 --2- 172.21.15.26:0/1504261285 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdac811b140 0x7fdac811d530 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:24.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:24.329+0000 7fdac759e640 1 --2- 172.21.15.26:0/1504261285 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdac8110190 0x7fdac806e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:24.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:24.329+0000 7fdac759e640 1 -- 172.21.15.26:0/1504261285 >> 172.21.15.26:0/1504261285 conn(0x7fdac8073180 msgr2=0x7fdac8073590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:40:24.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:24.340+0000 7fdac759e640 1 -- 172.21.15.26:0/1504261285 shutdown_connections 2024-09-11T23:40:24.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:24.340+0000 7fdac759e640 1 -- 172.21.15.26:0/1504261285 wait complete. 2024-09-11T23:40:24.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:24.341+0000 7fdac759e640 1 Processor -- start 2024-09-11T23:40:24.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:24.345+0000 7fdac759e640 1 -- start start 2024-09-11T23:40:24.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:24.345+0000 7fdac759e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdac806ea30 0x7fdac81186d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:24.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:24.345+0000 7fdac759e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdac8110190 0x7fdac8111760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:24.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:24.345+0000 7fdac759e640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdac811b140 0x7fdac8111ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:24.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:24.345+0000 7fdac759e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fdac8120ce0 con 0x7fdac8110190 2024-09-11T23:40:24.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:24.345+0000 7fdac759e640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fdac8120b60 con 0x7fdac806ea30 2024-09-11T23:40:24.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:24.345+0000 7fdac759e640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fdac8120e60 con 0x7fdac811b140 2024-09-11T23:40:24.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:24.346+0000 7fdac5d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdac8110190 0x7fdac8111760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:24.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:24.346+0000 7fdac5d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdac8110190 0x7fdac8111760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47316/0 (socket says 172.21.15.26:47316) 2024-09-11T23:40:24.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:24.346+0000 7fdac5d9b640 1 -- 172.21.15.26:0/3497611878 learned_addr learned my addr 172.21.15.26:0/3497611878 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:40:24.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:24.347+0000 7fdac5d9b640 1 -- 172.21.15.26:0/3497611878 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdac811b140 msgr2=0x7fdac8111ca0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-11T23:40:24.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:24.347+0000 7fdac5d9b640 1 --2- 172.21.15.26:0/3497611878 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdac811b140 0x7fdac8111ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:24.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:24.347+0000 7fdac5d9b640 1 -- 172.21.15.26:0/3497611878 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdac806ea30 msgr2=0x7fdac81186d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-11T23:40:24.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:24.347+0000 7fdac5d9b640 1 --2- 172.21.15.26:0/3497611878 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdac806ea30 0x7fdac81186d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:24.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:24.347+0000 7fdac5d9b640 1 -- 172.21.15.26:0/3497611878 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdac81122b0 con 0x7fdac8110190 2024-09-11T23:40:24.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:24.348+0000 7fdac5d9b640 1 --2- 172.21.15.26:0/3497611878 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdac8110190 0x7fdac8111760 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7fdab800b450 tx=0x7fdab800b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:40:24.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:24.349+0000 7fdab77fe640 1 -- 172.21.15.26:0/3497611878 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdab8015020 con 0x7fdac8110190 2024-09-11T23:40:24.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:24.349+0000 7fdac759e640 1 -- 172.21.15.26:0/3497611878 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdac81125a0 con 0x7fdac8110190 2024-09-11T23:40:24.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:24.349+0000 7fdac759e640 1 -- 172.21.15.26:0/3497611878 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdac8116570 con 0x7fdac8110190 2024-09-11T23:40:24.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:24.350+0000 7fdab77fe640 1 -- 172.21.15.26:0/3497611878 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdab8011d70 con 0x7fdac8110190 2024-09-11T23:40:24.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:24.350+0000 7fdab77fe640 1 -- 172.21.15.26:0/3497611878 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdab800f340 con 0x7fdac8110190 2024-09-11T23:40:24.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:24.353+0000 7fdac759e640 1 -- 172.21.15.26:0/3497611878 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdac81096f0 con 0x7fdac8110190 2024-09-11T23:40:24.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:24.358+0000 7fdab77fe640 1 -- 172.21.15.26:0/3497611878 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fdab8018020 con 0x7fdac8110190 2024-09-11T23:40:24.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:24.359+0000 7fdab77fe640 1 --2- 172.21.15.26:0/3497611878 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fdab00a2f00 0x7fdab00a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:24.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:24.359+0000 7fdab77fe640 1 -- 172.21.15.26:0/3497611878 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 8586+0+0 (secure 0 0 0) 0x7fdab80cf3d0 con 0x7fdac8110190 2024-09-11T23:40:24.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:24.359+0000 7fdac659c640 1 --2- 172.21.15.26:0/3497611878 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fdab00a2f00 0x7fdab00a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:24.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:24.360+0000 7fdac659c640 1 --2- 172.21.15.26:0/3497611878 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fdab00a2f00 0x7fdab00a53c0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fdac000ab90 tx=0x7fdac0009230 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:40:24.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:24.370+0000 7fdab77fe640 1 -- 172.21.15.26:0/3497611878 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fdab801f020 con 0x7fdac8110190 2024-09-11T23:40:24.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:24.787+0000 7fdac759e640 1 -- 172.21.15.26:0/3497611878 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdac8112c90 con 0x7fdac8110190 2024-09-11T23:40:24.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:24.792+0000 7fdab77fe640 1 -- 172.21.15.26:0/3497611878 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 93 v93) ==== 94+0+11999 (secure 0 0 0) 0x7fdab800d040 con 0x7fdac8110190 2024-09-11T23:40:24.791 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-11T23:40:24.792 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":93,"btime":"2024-09-11T23:39:59:126463+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14499,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/367659515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":367659515},{"type":"v1","addr":"172.21.15.26:6835","nonce":367659515}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14511,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/2041065299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":2041065299},{"type":"v1","addr":"172.21.15.26:6837","nonce":2041065299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":93,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:16:26.544203+0000","modified":"2024-09-11T23:39:58.413854+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24485,"mds_1":24463,"mds_2":24458,"mds_3":24448,"mds_4":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14550":{"gid":14550,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6839/2233068305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":2233068305},{"type":"v1","addr":"172.21.15.26:6839","nonce":2233068305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14562":{"gid":14562,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6841/2984982336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":2984982336},{"type":"v1","addr":"172.21.15.26:6841","nonce":2984982336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6833/2019841455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":2019841455},{"type":"v1","addr":"172.21.15.136:6833","nonce":2019841455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.76:6833/3438147050","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3438147050},{"type":"v1","addr":"172.21.15.76:6833","nonce":3438147050}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6835/3617665975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3617665975},{"type":"v1","addr":"172.21.15.136:6835","nonce":3617665975}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":12,"addr":"172.21.15.76:6835/393308499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":393308499},{"type":"v1","addr":"172.21.15.76:6835","nonce":393308499}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":10,"addr":"172.21.15.136:6837/1633082387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":1633082387},{"type":"v1","addr":"172.21.15.136:6837","nonce":1633082387}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.76:6837/2078978748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":2078978748},{"type":"v1","addr":"172.21.15.76:6837","nonce":2078978748}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":63,"addr":"172.21.15.136:6839/3211398630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":3211398630},{"type":"v1","addr":"172.21.15.136:6839","nonce":3211398630}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.76:6839/2077637090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6838","nonce":2077637090},{"type":"v1","addr":"172.21.15.76:6839","nonce":2077637090}]},"join_fscid":-1,"export_targets":[1,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24485,"qdb_cluster":[24443,24448,24458,24463,24485]},"id":1}]} 2024-09-11T23:40:24.792 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 93 2024-09-11T23:40:24.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:24.797+0000 7fdac759e640 1 -- 172.21.15.26:0/3497611878 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fdab00a2f00 msgr2=0x7fdab00a53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:24.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:24.797+0000 7fdac759e640 1 --2- 172.21.15.26:0/3497611878 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fdab00a2f00 0x7fdab00a53c0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fdac000ab90 tx=0x7fdac0009230 comp rx=0 tx=0).stop 2024-09-11T23:40:24.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:24.797+0000 7fdac759e640 1 -- 172.21.15.26:0/3497611878 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdac8110190 msgr2=0x7fdac8111760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:24.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:24.797+0000 7fdac759e640 1 --2- 172.21.15.26:0/3497611878 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdac8110190 0x7fdac8111760 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7fdab800b450 tx=0x7fdab800b910 comp rx=0 tx=0).stop 2024-09-11T23:40:24.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:24.798+0000 7fdac759e640 1 -- 172.21.15.26:0/3497611878 shutdown_connections 2024-09-11T23:40:24.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:24.798+0000 7fdac759e640 1 --2- 172.21.15.26:0/3497611878 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fdab00a2f00 0x7fdab00a53c0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:24.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:24.798+0000 7fdac759e640 1 --2- 172.21.15.26:0/3497611878 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdac811b140 0x7fdac8111ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:24.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:24.798+0000 7fdac759e640 1 --2- 172.21.15.26:0/3497611878 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdac8110190 0x7fdac8111760 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:24.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:24.798+0000 7fdac759e640 1 --2- 172.21.15.26:0/3497611878 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdac806ea30 0x7fdac81186d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:24.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:24.798+0000 7fdac759e640 1 -- 172.21.15.26:0/3497611878 >> 172.21.15.26:0/3497611878 conn(0x7fdac8073180 msgr2=0x7fdac81193c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:40:24.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:24.800+0000 7fdac759e640 1 -- 172.21.15.26:0/3497611878 shutdown_connections 2024-09-11T23:40:24.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:24.801+0000 7fdac759e640 1 -- 172.21.15.26:0/3497611878 wait complete. 2024-09-11T23:40:25.988 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:40:25 smithi136 ceph-mon[35121]: pgmap v695: 97 pgs: 41 active+clean+snaptrim_wait, 23 active+clean+snaptrim, 33 active+clean; 2.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1023 B/s wr, 8 op/s 2024-09-11T23:40:25.988 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:40:25 smithi136 ceph-mon[35121]: from='client.? 172.21.15.26:0/3497611878' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:40:26.000 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:40:25 smithi026 ceph-mon[31172]: pgmap v695: 97 pgs: 41 active+clean+snaptrim_wait, 23 active+clean+snaptrim, 33 active+clean; 2.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1023 B/s wr, 8 op/s 2024-09-11T23:40:26.000 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:40:25 smithi026 ceph-mon[31172]: from='client.? 172.21.15.26:0/3497611878' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:40:26.052 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:40:25 smithi076 ceph-mon[35296]: pgmap v695: 97 pgs: 41 active+clean+snaptrim_wait, 23 active+clean+snaptrim, 33 active+clean; 2.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1023 B/s wr, 8 op/s 2024-09-11T23:40:26.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:40:25 smithi076 ceph-mon[35296]: from='client.? 172.21.15.26:0/3497611878' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:40:26.204 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:40:26.725 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:40:27.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:27.334+0000 7f71240e8640 1 -- 172.21.15.26:0/759513257 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f711c101290 msgr2=0x7f711c10d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:27.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:27.334+0000 7f7120e82640 1 -- 172.21.15.26:0/759513257 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7114045020 con 0x7f711c101290 2024-09-11T23:40:27.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:27.334+0000 7f71240e8640 1 --2- 172.21.15.26:0/759513257 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f711c101290 0x7f711c10d7c0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f711401ace0 tx=0x7f711403b990 comp rx=0 tx=0).stop 2024-09-11T23:40:27.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:27.335+0000 7f71240e8640 1 -- 172.21.15.26:0/759513257 shutdown_connections 2024-09-11T23:40:27.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:27.335+0000 7f71240e8640 1 --2- 172.21.15.26:0/759513257 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f711c1008f0 0x7f711c100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:27.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:27.335+0000 7f71240e8640 1 --2- 172.21.15.26:0/759513257 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f711c101290 0x7f711c10d7c0 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:27.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:27.335+0000 7f71240e8640 1 --2- 172.21.15.26:0/759513257 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f711c1068f0 0x7f711c106cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:27.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:27.335+0000 7f71240e8640 1 -- 172.21.15.26:0/759513257 >> 172.21.15.26:0/759513257 conn(0x7f711c0fc630 msgr2=0x7f711c0fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:40:27.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:27.339+0000 7f71240e8640 1 -- 172.21.15.26:0/759513257 shutdown_connections 2024-09-11T23:40:27.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:27.342+0000 7f71240e8640 1 -- 172.21.15.26:0/759513257 wait complete. 2024-09-11T23:40:27.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:27.343+0000 7f71240e8640 1 Processor -- start 2024-09-11T23:40:27.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:27.346+0000 7f71240e8640 1 -- start start 2024-09-11T23:40:27.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:27.346+0000 7f71240e8640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f711c1008f0 0x7f711c10d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:27.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:27.346+0000 7f71240e8640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f711c101290 0x7f711c0fce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:27.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:27.346+0000 7f71240e8640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f711c1068f0 0x7f711c0fd3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:27.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:27.346+0000 7f71240e8640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f711c06e5e0 con 0x7f711c1008f0 2024-09-11T23:40:27.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:27.346+0000 7f71240e8640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f711c06e460 con 0x7f711c101290 2024-09-11T23:40:27.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:27.346+0000 7f71240e8640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f711c06e760 con 0x7f711c1068f0 2024-09-11T23:40:27.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:27.349+0000 7f7121e84640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f711c1008f0 0x7f711c10d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:27.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:27.349+0000 7f7121e84640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f711c1008f0 0x7f711c10d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47336/0 (socket says 172.21.15.26:47336) 2024-09-11T23:40:27.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:27.349+0000 7f7121e84640 1 -- 172.21.15.26:0/3722130647 learned_addr learned my addr 172.21.15.26:0/3722130647 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:40:27.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:27.351+0000 7f7122685640 1 --2- 172.21.15.26:0/3722130647 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f711c1068f0 0x7f711c0fd3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:27.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:27.353+0000 7f7121e84640 1 -- 172.21.15.26:0/3722130647 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f711c1068f0 msgr2=0x7f711c0fd3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:27.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:27.353+0000 7f7121e84640 1 --2- 172.21.15.26:0/3722130647 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f711c1068f0 0x7f711c0fd3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:27.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:27.353+0000 7f7121e84640 1 -- 172.21.15.26:0/3722130647 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f711c101290 msgr2=0x7f711c0fce90 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-11T23:40:27.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:27.353+0000 7f7121e84640 1 --2- 172.21.15.26:0/3722130647 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f711c101290 0x7f711c0fce90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:27.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:27.353+0000 7f7121e84640 1 -- 172.21.15.26:0/3722130647 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f711c0fd940 con 0x7f711c1008f0 2024-09-11T23:40:27.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:27.357+0000 7f7121e84640 1 --2- 172.21.15.26:0/3722130647 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f711c1008f0 0x7f711c10d4c0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f711000e9f0 tx=0x7f711000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:40:27.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:27.358+0000 7f710effd640 1 -- 172.21.15.26:0/3722130647 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f711000cd80 con 0x7f711c1008f0 2024-09-11T23:40:27.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:27.358+0000 7f710effd640 1 -- 172.21.15.26:0/3722130647 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7110010910 con 0x7f711c1008f0 2024-09-11T23:40:27.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:27.359+0000 7f71240e8640 1 -- 172.21.15.26:0/3722130647 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f711c0fdc30 con 0x7f711c1008f0 2024-09-11T23:40:27.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:27.359+0000 7f71240e8640 1 -- 172.21.15.26:0/3722130647 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f711c1b83d0 con 0x7f711c1008f0 2024-09-11T23:40:27.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:27.360+0000 7f710effd640 1 -- 172.21.15.26:0/3722130647 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71100188f0 con 0x7f711c1008f0 2024-09-11T23:40:27.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:27.361+0000 7f71240e8640 1 -- 172.21.15.26:0/3722130647 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7100005290 con 0x7f711c1008f0 2024-09-11T23:40:27.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:27.364+0000 7f710effd640 1 -- 172.21.15.26:0/3722130647 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f7110010430 con 0x7f711c1008f0 2024-09-11T23:40:27.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:27.365+0000 7f710effd640 1 --2- 172.21.15.26:0/3722130647 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f71080a2d30 0x7f71080a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:27.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:27.367+0000 7f710effd640 1 -- 172.21.15.26:0/3722130647 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 8586+0+0 (secure 0 0 0) 0x7f7110099f60 con 0x7f711c1008f0 2024-09-11T23:40:27.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:27.367+0000 7f7121683640 1 --2- 172.21.15.26:0/3722130647 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f71080a2d30 0x7f71080a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:27.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:27.370+0000 7f7121683640 1 --2- 172.21.15.26:0/3722130647 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f71080a2d30 0x7f71080a51f0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f711c0fe920 tx=0x7f7118006bd0 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:40:27.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:27.374+0000 7f710effd640 1 -- 172.21.15.26:0/3722130647 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f711009c5c0 con 0x7f711c1008f0 2024-09-11T23:40:27.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:27.783+0000 7f71240e8640 1 -- 172.21.15.26:0/3722130647 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7100005580 con 0x7f711c1008f0 2024-09-11T23:40:27.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:27.785+0000 7f710effd640 1 -- 172.21.15.26:0/3722130647 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 93 v93) ==== 94+0+11999 (secure 0 0 0) 0x7f711009bec0 con 0x7f711c1008f0 2024-09-11T23:40:27.784 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-11T23:40:27.785 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":93,"btime":"2024-09-11T23:39:59:126463+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14499,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/367659515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":367659515},{"type":"v1","addr":"172.21.15.26:6835","nonce":367659515}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14511,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/2041065299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":2041065299},{"type":"v1","addr":"172.21.15.26:6837","nonce":2041065299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":93,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:16:26.544203+0000","modified":"2024-09-11T23:39:58.413854+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24485,"mds_1":24463,"mds_2":24458,"mds_3":24448,"mds_4":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14550":{"gid":14550,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6839/2233068305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":2233068305},{"type":"v1","addr":"172.21.15.26:6839","nonce":2233068305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14562":{"gid":14562,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6841/2984982336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":2984982336},{"type":"v1","addr":"172.21.15.26:6841","nonce":2984982336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6833/2019841455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":2019841455},{"type":"v1","addr":"172.21.15.136:6833","nonce":2019841455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.76:6833/3438147050","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3438147050},{"type":"v1","addr":"172.21.15.76:6833","nonce":3438147050}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6835/3617665975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3617665975},{"type":"v1","addr":"172.21.15.136:6835","nonce":3617665975}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":12,"addr":"172.21.15.76:6835/393308499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":393308499},{"type":"v1","addr":"172.21.15.76:6835","nonce":393308499}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":10,"addr":"172.21.15.136:6837/1633082387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":1633082387},{"type":"v1","addr":"172.21.15.136:6837","nonce":1633082387}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.76:6837/2078978748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":2078978748},{"type":"v1","addr":"172.21.15.76:6837","nonce":2078978748}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":63,"addr":"172.21.15.136:6839/3211398630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":3211398630},{"type":"v1","addr":"172.21.15.136:6839","nonce":3211398630}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.76:6839/2077637090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6838","nonce":2077637090},{"type":"v1","addr":"172.21.15.76:6839","nonce":2077637090}]},"join_fscid":-1,"export_targets":[1,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24485,"qdb_cluster":[24443,24448,24458,24463,24485]},"id":1}]} 2024-09-11T23:40:27.785 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 93 2024-09-11T23:40:27.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:27.789+0000 7f71240e8640 1 -- 172.21.15.26:0/3722130647 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f71080a2d30 msgr2=0x7f71080a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:27.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:27.789+0000 7f71240e8640 1 --2- 172.21.15.26:0/3722130647 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f71080a2d30 0x7f71080a51f0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f711c0fe920 tx=0x7f7118006bd0 comp rx=0 tx=0).stop 2024-09-11T23:40:27.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:27.789+0000 7f71240e8640 1 -- 172.21.15.26:0/3722130647 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f711c1008f0 msgr2=0x7f711c10d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:27.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:27.789+0000 7f71240e8640 1 --2- 172.21.15.26:0/3722130647 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f711c1008f0 0x7f711c10d4c0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f711000e9f0 tx=0x7f711000eeb0 comp rx=0 tx=0).stop 2024-09-11T23:40:27.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:27.790+0000 7f71240e8640 1 -- 172.21.15.26:0/3722130647 shutdown_connections 2024-09-11T23:40:27.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:27.790+0000 7f71240e8640 1 --2- 172.21.15.26:0/3722130647 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f71080a2d30 0x7f71080a51f0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:27.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:27.790+0000 7f71240e8640 1 --2- 172.21.15.26:0/3722130647 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f711c101290 0x7f711c0fce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:27.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:27.790+0000 7f71240e8640 1 --2- 172.21.15.26:0/3722130647 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f711c1068f0 0x7f711c0fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:27.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:27.790+0000 7f71240e8640 1 --2- 172.21.15.26:0/3722130647 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f711c1008f0 0x7f711c10d4c0 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:27.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:27.790+0000 7f71240e8640 1 -- 172.21.15.26:0/3722130647 >> 172.21.15.26:0/3722130647 conn(0x7f711c0fc630 msgr2=0x7f711c06a790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:40:27.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:27.791+0000 7f71240e8640 1 -- 172.21.15.26:0/3722130647 shutdown_connections 2024-09-11T23:40:27.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:27.793+0000 7f71240e8640 1 -- 172.21.15.26:0/3722130647 wait complete. 2024-09-11T23:40:27.988 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:40:27 smithi136 ceph-mon[35121]: pgmap v696: 97 pgs: 41 active+clean+snaptrim_wait, 22 active+clean+snaptrim, 34 active+clean; 2.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 2.0 KiB/s wr, 14 op/s 2024-09-11T23:40:28.000 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:40:27 smithi026 ceph-mon[31172]: pgmap v696: 97 pgs: 41 active+clean+snaptrim_wait, 22 active+clean+snaptrim, 34 active+clean; 2.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 2.0 KiB/s wr, 14 op/s 2024-09-11T23:40:28.052 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:40:27 smithi076 ceph-mon[35296]: pgmap v696: 97 pgs: 41 active+clean+snaptrim_wait, 22 active+clean+snaptrim, 34 active+clean; 2.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 2.0 KiB/s wr, 14 op/s 2024-09-11T23:40:28.554 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-11T23:40:28.988 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:40:28 smithi136 ceph-mon[35121]: from='client.? 172.21.15.26:0/3722130647' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:40:29.000 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:40:28 smithi026 ceph-mon[31172]: from='client.? 172.21.15.26:0/3722130647' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:40:29.041 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:40:29.052 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:40:28 smithi076 ceph-mon[35296]: from='client.? 172.21.15.26:0/3722130647' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:40:29.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:29.654+0000 7f1b244ad640 1 -- 172.21.15.26:0/1481734706 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1b1c105ec0 msgr2=0x7f1b1c106340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:29.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:29.654+0000 7f1b244ad640 1 --2- 172.21.15.26:0/1481734706 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1b1c105ec0 0x7f1b1c106340 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f1b18009cd0 tx=0x7f1b1802a930 comp rx=0 tx=0).stop 2024-09-11T23:40:29.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:29.656+0000 7f1b244ad640 1 -- 172.21.15.26:0/1481734706 shutdown_connections 2024-09-11T23:40:29.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:29.656+0000 7f1b244ad640 1 --2- 172.21.15.26:0/1481734706 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1b1c105ec0 0x7f1b1c106340 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:29.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:29.656+0000 7f1b244ad640 1 --2- 172.21.15.26:0/1481734706 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1b1c1011e0 0x7f1b1c105980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:29.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:29.656+0000 7f1b244ad640 1 --2- 172.21.15.26:0/1481734706 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1b1c1008c0 0x7f1b1c100ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:29.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:29.656+0000 7f1b244ad640 1 -- 172.21.15.26:0/1481734706 >> 172.21.15.26:0/1481734706 conn(0x7f1b1c0fc7d0 msgr2=0x7f1b1c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:40:29.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:29.657+0000 7f1b244ad640 1 -- 172.21.15.26:0/1481734706 shutdown_connections 2024-09-11T23:40:29.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:29.658+0000 7f1b244ad640 1 -- 172.21.15.26:0/1481734706 wait complete. 2024-09-11T23:40:29.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:29.658+0000 7f1b244ad640 1 Processor -- start 2024-09-11T23:40:29.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:29.659+0000 7f1b244ad640 1 -- start start 2024-09-11T23:40:29.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:29.659+0000 7f1b244ad640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1b1c1008c0 0x7f1b1c114c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:29.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:29.659+0000 7f1b244ad640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1b1c1011e0 0x7f1b1c115180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:29.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:29.659+0000 7f1b244ad640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1b1c105ec0 0x7f1b1c10df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:29.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:29.659+0000 7f1b244ad640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f1b1c1186b0 con 0x7f1b1c1008c0 2024-09-11T23:40:29.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:29.659+0000 7f1b244ad640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f1b1c118530 con 0x7f1b1c105ec0 2024-09-11T23:40:29.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:29.659+0000 7f1b244ad640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f1b1c118830 con 0x7f1b1c1011e0 2024-09-11T23:40:29.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:29.659+0000 7f1b21a48640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1b1c1011e0 0x7f1b1c115180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:29.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:29.660+0000 7f1b21a48640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1b1c1011e0 0x7f1b1c115180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:55556/0 (socket says 172.21.15.26:55556) 2024-09-11T23:40:29.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:29.660+0000 7f1b21a48640 1 -- 172.21.15.26:0/3643741459 learned_addr learned my addr 172.21.15.26:0/3643741459 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:40:29.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:29.660+0000 7f1b22a4a640 1 --2- 172.21.15.26:0/3643741459 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1b1c105ec0 0x7f1b1c10df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:29.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:29.661+0000 7f1b21a48640 1 -- 172.21.15.26:0/3643741459 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1b1c105ec0 msgr2=0x7f1b1c10df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:29.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:29.661+0000 7f1b21a48640 1 --2- 172.21.15.26:0/3643741459 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1b1c105ec0 0x7f1b1c10df00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:29.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:29.661+0000 7f1b21a48640 1 -- 172.21.15.26:0/3643741459 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1b1c1008c0 msgr2=0x7f1b1c114c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:29.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:29.661+0000 7f1b21a48640 1 --2- 172.21.15.26:0/3643741459 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1b1c1008c0 0x7f1b1c114c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:29.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:29.661+0000 7f1b21a48640 1 -- 172.21.15.26:0/3643741459 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b1c10e760 con 0x7f1b1c1011e0 2024-09-11T23:40:29.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:29.661+0000 7f1b21a48640 1 --2- 172.21.15.26:0/3643741459 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1b1c1011e0 0x7f1b1c115180 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f1b0c00b450 tx=0x7f1b0c00b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:40:29.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:29.662+0000 7f1b137fe640 1 -- 172.21.15.26:0/3643741459 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b0c015020 con 0x7f1b1c1011e0 2024-09-11T23:40:29.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:29.662+0000 7f1b244ad640 1 -- 172.21.15.26:0/3643741459 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b1c10ea50 con 0x7f1b1c1011e0 2024-09-11T23:40:29.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:29.662+0000 7f1b22a4a640 1 --2- 172.21.15.26:0/3643741459 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1b1c105ec0 0x7f1b1c10df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:40:29.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:29.662+0000 7f1b244ad640 1 -- 172.21.15.26:0/3643741459 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b1c112a50 con 0x7f1b1c1011e0 2024-09-11T23:40:29.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:29.662+0000 7f1b137fe640 1 -- 172.21.15.26:0/3643741459 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b0c011d70 con 0x7f1b1c1011e0 2024-09-11T23:40:29.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:29.662+0000 7f1b137fe640 1 -- 172.21.15.26:0/3643741459 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b0c00f340 con 0x7f1b1c1011e0 2024-09-11T23:40:29.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:29.664+0000 7f1b244ad640 1 -- 172.21.15.26:0/3643741459 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b00005290 con 0x7f1b1c1011e0 2024-09-11T23:40:29.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:29.670+0000 7f1b137fe640 1 -- 172.21.15.26:0/3643741459 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f1b0c018020 con 0x7f1b1c1011e0 2024-09-11T23:40:29.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:29.671+0000 7f1b137fe640 1 --2- 172.21.15.26:0/3643741459 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f1b080a2d70 0x7f1b080a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:29.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:29.671+0000 7f1b137fe640 1 -- 172.21.15.26:0/3643741459 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 8586+0+0 (secure 0 0 0) 0x7f1b0c0cf0d0 con 0x7f1b1c1011e0 2024-09-11T23:40:29.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:29.673+0000 7f1b137fe640 1 -- 172.21.15.26:0/3643741459 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1b0c01f020 con 0x7f1b1c1011e0 2024-09-11T23:40:29.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:29.675+0000 7f1b22249640 1 --2- 172.21.15.26:0/3643741459 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f1b080a2d70 0x7f1b080a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:29.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:29.676+0000 7f1b22249640 1 --2- 172.21.15.26:0/3643741459 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f1b080a2d70 0x7f1b080a5230 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f1b1400a990 tx=0x7f1b14005c70 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:40:29.988 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:40:29 smithi136 ceph-mon[35121]: pgmap v697: 97 pgs: 41 active+clean+snaptrim_wait, 22 active+clean+snaptrim, 34 active+clean; 2.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-09-11T23:40:30.000 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:40:29 smithi026 ceph-mon[31172]: pgmap v697: 97 pgs: 41 active+clean+snaptrim_wait, 22 active+clean+snaptrim, 34 active+clean; 2.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-09-11T23:40:30.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:30.016+0000 7f1b244ad640 1 -- 172.21.15.26:0/3643741459 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1b00005850 con 0x7f1b1c1011e0 2024-09-11T23:40:30.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:30.018+0000 7f1b137fe640 1 -- 172.21.15.26:0/3643741459 <== mon.1 v2:172.21.15.136:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v117) ==== 74+0+23271 (secure 0 0 0) 0x7f1b0c00d040 con 0x7f1b1c1011e0 2024-09-11T23:40:30.018 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-11T23:40:30.018 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":117,"fsid":"af7e6fc8-7092-11ef-bcea-c7b262605968","created":"2024-09-11T23:08:40.486438+0000","modified":"2024-09-11T23:40:09.232380+0000","last_up_change":"2024-09-11T23:15:35.179446+0000","last_in_change":"2024-09-11T23:15:11.064627+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T23:11:50.494160+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-11T23:16:22.503928+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-11T23:16:24.876554+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"117","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":39,"snap_epoch":117,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a4d62fcf-c31e-41b1-932c-3d87ff640531","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":2014857905},{"type":"v1","addr":"172.21.15.26:6803","nonce":2014857905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":2014857905},{"type":"v1","addr":"172.21.15.26:6805","nonce":2014857905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":2014857905},{"type":"v1","addr":"172.21.15.26:6809","nonce":2014857905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":2014857905},{"type":"v1","addr":"172.21.15.26:6807","nonce":2014857905}]},"public_addr":"172.21.15.26:6803/2014857905","cluster_addr":"172.21.15.26:6805/2014857905","heartbeat_back_addr":"172.21.15.26:6809/2014857905","heartbeat_front_addr":"172.21.15.26:6807/2014857905","state":["exists","up"]},{"osd":1,"uuid":"e0d69112-3c0a-4fc5-8783-e41b4081e0c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":333383087},{"type":"v1","addr":"172.21.15.76:6801","nonce":333383087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":333383087},{"type":"v1","addr":"172.21.15.76:6803","nonce":333383087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":333383087},{"type":"v1","addr":"172.21.15.76:6807","nonce":333383087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":333383087},{"type":"v1","addr":"172.21.15.76:6805","nonce":333383087}]},"public_addr":"172.21.15.76:6801/333383087","cluster_addr":"172.21.15.76:6803/333383087","heartbeat_back_addr":"172.21.15.76:6807/333383087","heartbeat_front_addr":"172.21.15.76:6805/333383087","state":["exists","up"]},{"osd":2,"uuid":"d578429e-bf23-4a1e-890f-7172955c0eb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":1271538911},{"type":"v1","addr":"172.21.15.136:6801","nonce":1271538911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":1271538911},{"type":"v1","addr":"172.21.15.136:6803","nonce":1271538911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":1271538911},{"type":"v1","addr":"172.21.15.136:6807","nonce":1271538911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":1271538911},{"type":"v1","addr":"172.21.15.136:6805","nonce":1271538911}]},"public_addr":"172.21.15.136:6801/1271538911","cluster_addr":"172.21.15.136:6803/1271538911","heartbeat_back_addr":"172.21.15.136:6807/1271538911","heartbeat_front_addr":"172.21.15.136:6805/1271538911","state":["exists","up"]},{"osd":3,"uuid":"3e34623f-0ea5-4b10-b8a9-f8756c35d512","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":1740304151},{"type":"v1","addr":"172.21.15.26:6811","nonce":1740304151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":1740304151},{"type":"v1","addr":"172.21.15.26:6813","nonce":1740304151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1740304151},{"type":"v1","addr":"172.21.15.26:6817","nonce":1740304151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":1740304151},{"type":"v1","addr":"172.21.15.26:6815","nonce":1740304151}]},"public_addr":"172.21.15.26:6811/1740304151","cluster_addr":"172.21.15.26:6813/1740304151","heartbeat_back_addr":"172.21.15.26:6817/1740304151","heartbeat_front_addr":"172.21.15.26:6815/1740304151","state":["exists","up"]},{"osd":4,"uuid":"77607929-f9ed-49a9-abb5-48cd88eec4ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":2702262182},{"type":"v1","addr":"172.21.15.76:6809","nonce":2702262182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":2702262182},{"type":"v1","addr":"172.21.15.76:6811","nonce":2702262182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":2702262182},{"type":"v1","addr":"172.21.15.76:6815","nonce":2702262182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":2702262182},{"type":"v1","addr":"172.21.15.76:6813","nonce":2702262182}]},"public_addr":"172.21.15.76:6809/2702262182","cluster_addr":"172.21.15.76:6811/2702262182","heartbeat_back_addr":"172.21.15.76:6815/2702262182","heartbeat_front_addr":"172.21.15.76:6813/2702262182","state":["exists","up"]},{"osd":5,"uuid":"29e20e37-09a0-4924-8aaa-17939763a7c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":1749229277},{"type":"v1","addr":"172.21.15.136:6809","nonce":1749229277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":1749229277},{"type":"v1","addr":"172.21.15.136:6811","nonce":1749229277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":1749229277},{"type":"v1","addr":"172.21.15.136:6815","nonce":1749229277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":1749229277},{"type":"v1","addr":"172.21.15.136:6813","nonce":1749229277}]},"public_addr":"172.21.15.136:6809/1749229277","cluster_addr":"172.21.15.136:6811/1749229277","heartbeat_back_addr":"172.21.15.136:6815/1749229277","heartbeat_front_addr":"172.21.15.136:6813/1749229277","state":["exists","up"]},{"osd":6,"uuid":"e0ddd1c0-1c46-41e6-b2f4-396bdcd2bf8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1928245624},{"type":"v1","addr":"172.21.15.26:6819","nonce":1928245624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1928245624},{"type":"v1","addr":"172.21.15.26:6821","nonce":1928245624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":1928245624},{"type":"v1","addr":"172.21.15.26:6825","nonce":1928245624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1928245624},{"type":"v1","addr":"172.21.15.26:6823","nonce":1928245624}]},"public_addr":"172.21.15.26:6819/1928245624","cluster_addr":"172.21.15.26:6821/1928245624","heartbeat_back_addr":"172.21.15.26:6825/1928245624","heartbeat_front_addr":"172.21.15.26:6823/1928245624","state":["exists","up"]},{"osd":7,"uuid":"fe031f5c-d915-48fb-91a6-c76f252467fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":1409789455},{"type":"v1","addr":"172.21.15.76:6817","nonce":1409789455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":1409789455},{"type":"v1","addr":"172.21.15.76:6819","nonce":1409789455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":1409789455},{"type":"v1","addr":"172.21.15.76:6823","nonce":1409789455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":1409789455},{"type":"v1","addr":"172.21.15.76:6821","nonce":1409789455}]},"public_addr":"172.21.15.76:6817/1409789455","cluster_addr":"172.21.15.76:6819/1409789455","heartbeat_back_addr":"172.21.15.76:6823/1409789455","heartbeat_front_addr":"172.21.15.76:6821/1409789455","state":["exists","up"]},{"osd":8,"uuid":"18851bde-9e6c-4e05-b4bf-ac5c3073eee7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":543619079},{"type":"v1","addr":"172.21.15.136:6817","nonce":543619079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":543619079},{"type":"v1","addr":"172.21.15.136:6819","nonce":543619079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":543619079},{"type":"v1","addr":"172.21.15.136:6823","nonce":543619079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":543619079},{"type":"v1","addr":"172.21.15.136:6821","nonce":543619079}]},"public_addr":"172.21.15.136:6817/543619079","cluster_addr":"172.21.15.136:6819/543619079","heartbeat_back_addr":"172.21.15.136:6823/543619079","heartbeat_front_addr":"172.21.15.136:6821/543619079","state":["exists","up"]},{"osd":9,"uuid":"52a65eab-8446-47ad-bfe7-5133f7e3eb98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":2201297064},{"type":"v1","addr":"172.21.15.26:6827","nonce":2201297064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":2201297064},{"type":"v1","addr":"172.21.15.26:6829","nonce":2201297064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":2201297064},{"type":"v1","addr":"172.21.15.26:6833","nonce":2201297064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":2201297064},{"type":"v1","addr":"172.21.15.26:6831","nonce":2201297064}]},"public_addr":"172.21.15.26:6827/2201297064","cluster_addr":"172.21.15.26:6829/2201297064","heartbeat_back_addr":"172.21.15.26:6833/2201297064","heartbeat_front_addr":"172.21.15.26:6831/2201297064","state":["exists","up"]},{"osd":10,"uuid":"b758d4bb-da62-4c5f-a992-bb6a145d35d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":3184128096},{"type":"v1","addr":"172.21.15.76:6825","nonce":3184128096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":3184128096},{"type":"v1","addr":"172.21.15.76:6827","nonce":3184128096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":3184128096},{"type":"v1","addr":"172.21.15.76:6831","nonce":3184128096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":3184128096},{"type":"v1","addr":"172.21.15.76:6829","nonce":3184128096}]},"public_addr":"172.21.15.76:6825/3184128096","cluster_addr":"172.21.15.76:6827/3184128096","heartbeat_back_addr":"172.21.15.76:6831/3184128096","heartbeat_front_addr":"172.21.15.76:6829/3184128096","state":["exists","up"]},{"osd":11,"uuid":"8fddda46-e22b-4d0c-ac48-e038fb29dfec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":1960689685},{"type":"v1","addr":"172.21.15.136:6825","nonce":1960689685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":1960689685},{"type":"v1","addr":"172.21.15.136:6827","nonce":1960689685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":1960689685},{"type":"v1","addr":"172.21.15.136:6831","nonce":1960689685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":1960689685},{"type":"v1","addr":"172.21.15.136:6829","nonce":1960689685}]},"public_addr":"172.21.15.136:6825/1960689685","cluster_addr":"172.21.15.136:6827/1960689685","heartbeat_back_addr":"172.21.15.136:6831/1960689685","heartbeat_front_addr":"172.21.15.136:6829/1960689685","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:10:56.656177+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:11:22.234031+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:11:46.464102+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:12:12.353967+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:12:37.836200+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:13:01.728839+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:13:27.527230+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:13:51.981393+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:14:17.035464+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:14:42.805029+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:15:08.208999+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:15:32.800844+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.18","mappings":[{"from":3,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":6}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.26:0/2626831531":"2024-09-12T23:18:22.177934+0000","172.21.15.26:0/3861103130":"2024-09-12T23:09:09.781665+0000","172.21.15.26:0/17142658":"2024-09-12T23:09:09.781665+0000","172.21.15.26:0/2817092653":"2024-09-12T23:09:09.781665+0000","172.21.15.26:6801/110833859":"2024-09-12T23:18:22.177934+0000","172.21.15.26:0/2107281413":"2024-09-12T23:09:33.983310+0000","172.21.15.26:0/883212708":"2024-09-12T23:09:33.983310+0000","172.21.15.26:6800/2673575521":"2024-09-12T23:09:09.781665+0000","172.21.15.26:0/2559066894":"2024-09-12T23:18:22.177934+0000","172.21.15.26:6801/2673575521":"2024-09-12T23:09:09.781665+0000","172.21.15.26:0/3597996003":"2024-09-12T23:09:33.983310+0000","172.21.15.26:6800/677148151":"2024-09-12T23:09:33.983310+0000","172.21.15.26:6801/677148151":"2024-09-12T23:09:33.983310+0000","172.21.15.26:0/1584355185":"2024-09-12T23:18:22.177934+0000","172.21.15.26:6800/110833859":"2024-09-12T23:18:22.177934+0000","172.21.15.26:0/1256290054":"2024-09-12T23:18:22.177934+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":3,"snaps":[{"begin":26,"length":1},{"begin":39,"length":1}]}],"new_removed_snaps":[{"pool":3,"snaps":[{"begin":26,"length":1},{"begin":39,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T23:40:30.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:30.022+0000 7f1b244ad640 1 -- 172.21.15.26:0/3643741459 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f1b080a2d70 msgr2=0x7f1b080a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:30.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:30.022+0000 7f1b244ad640 1 --2- 172.21.15.26:0/3643741459 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f1b080a2d70 0x7f1b080a5230 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f1b1400a990 tx=0x7f1b14005c70 comp rx=0 tx=0).stop 2024-09-11T23:40:30.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:30.022+0000 7f1b244ad640 1 -- 172.21.15.26:0/3643741459 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1b1c1011e0 msgr2=0x7f1b1c115180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:30.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:30.022+0000 7f1b244ad640 1 --2- 172.21.15.26:0/3643741459 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1b1c1011e0 0x7f1b1c115180 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f1b0c00b450 tx=0x7f1b0c00b910 comp rx=0 tx=0).stop 2024-09-11T23:40:30.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:30.022+0000 7f1b244ad640 1 -- 172.21.15.26:0/3643741459 shutdown_connections 2024-09-11T23:40:30.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:30.022+0000 7f1b244ad640 1 --2- 172.21.15.26:0/3643741459 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f1b080a2d70 0x7f1b080a5230 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:30.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:30.022+0000 7f1b244ad640 1 --2- 172.21.15.26:0/3643741459 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1b1c105ec0 0x7f1b1c10df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:30.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:30.022+0000 7f1b244ad640 1 --2- 172.21.15.26:0/3643741459 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1b1c1011e0 0x7f1b1c115180 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:30.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:30.022+0000 7f1b244ad640 1 --2- 172.21.15.26:0/3643741459 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1b1c1008c0 0x7f1b1c114c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:30.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:30.022+0000 7f1b244ad640 1 -- 172.21.15.26:0/3643741459 >> 172.21.15.26:0/3643741459 conn(0x7f1b1c0fc7d0 msgr2=0x7f1b1c103950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:40:30.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:30.022+0000 7f1b244ad640 1 -- 172.21.15.26:0/3643741459 shutdown_connections 2024-09-11T23:40:30.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:30.022+0000 7f1b244ad640 1 -- 172.21.15.26:0/3643741459 wait complete. 2024-09-11T23:40:30.052 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:40:29 smithi076 ceph-mon[35296]: pgmap v697: 97 pgs: 41 active+clean+snaptrim_wait, 22 active+clean+snaptrim, 34 active+clean; 2.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-09-11T23:40:30.988 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:40:30 smithi136 ceph-mon[35121]: from='client.? 172.21.15.26:0/3643741459' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T23:40:31.000 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:40:30 smithi026 ceph-mon[31172]: from='client.? 172.21.15.26:0/3643741459' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T23:40:31.052 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:40:30 smithi076 ceph-mon[35296]: from='client.? 172.21.15.26:0/3643741459' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T23:40:31.310 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph fs fail cephfs 2024-09-11T23:40:31.748 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:40:31 smithi026 ceph-mon[31172]: pgmap v698: 97 pgs: 41 active+clean+snaptrim_wait, 22 active+clean+snaptrim, 34 active+clean; 2.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.3 KiB/s wr, 11 op/s 2024-09-11T23:40:31.778 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:40:32.052 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:40:31 smithi076 ceph-mon[35296]: pgmap v698: 97 pgs: 41 active+clean+snaptrim_wait, 22 active+clean+snaptrim, 34 active+clean; 2.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.3 KiB/s wr, 11 op/s 2024-09-11T23:40:32.238 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:40:31 smithi136 ceph-mon[35121]: pgmap v698: 97 pgs: 41 active+clean+snaptrim_wait, 22 active+clean+snaptrim, 34 active+clean; 2.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.3 KiB/s wr, 11 op/s 2024-09-11T23:40:32.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:32.371+0000 7fc222759640 1 -- 172.21.15.26:0/986033003 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc21c076720 msgr2=0x7fc21c076b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:32.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:32.371+0000 7fc222759640 1 --2- 172.21.15.26:0/986033003 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc21c076720 0x7fc21c076b00 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fc20c009900 tx=0x7fc20c02a410 comp rx=0 tx=0).stop 2024-09-11T23:40:32.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:32.372+0000 7fc222759640 1 -- 172.21.15.26:0/986033003 shutdown_connections 2024-09-11T23:40:32.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:32.372+0000 7fc222759640 1 --2- 172.21.15.26:0/986033003 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc21c112870 0x7fc21c114c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:32.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:32.372+0000 7fc222759640 1 --2- 172.21.15.26:0/986033003 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc21c077040 0x7fc21c112330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:32.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:32.372+0000 7fc222759640 1 --2- 172.21.15.26:0/986033003 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc21c076720 0x7fc21c076b00 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:32.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:32.372+0000 7fc222759640 1 -- 172.21.15.26:0/986033003 >> 172.21.15.26:0/986033003 conn(0x7fc21c0ff910 msgr2=0x7fc21c101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:40:32.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:32.373+0000 7fc222759640 1 -- 172.21.15.26:0/986033003 shutdown_connections 2024-09-11T23:40:32.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:32.373+0000 7fc222759640 1 -- 172.21.15.26:0/986033003 wait complete. 2024-09-11T23:40:32.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:32.374+0000 7fc222759640 1 Processor -- start 2024-09-11T23:40:32.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:32.374+0000 7fc222759640 1 -- start start 2024-09-11T23:40:32.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:32.375+0000 7fc222759640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc21c076720 0x7fc21c1a4570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:32.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:32.375+0000 7fc222759640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc21c077040 0x7fc21c1a4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:32.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:32.375+0000 7fc221757640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc21c076720 0x7fc21c1a4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:32.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:32.376+0000 7fc222759640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc21c112870 0x7fc21c19e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:32.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:32.376+0000 7fc220f56640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc21c077040 0x7fc21c1a4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:32.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:32.376+0000 7fc220f56640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc21c077040 0x7fc21c1a4ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:59754/0 (socket says 172.21.15.26:59754) 2024-09-11T23:40:32.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:32.376+0000 7fc221757640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc21c076720 0x7fc21c1a4570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48124/0 (socket says 172.21.15.26:48124) 2024-09-11T23:40:32.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:32.376+0000 7fc221f58640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc21c112870 0x7fc21c19e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:32.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:32.376+0000 7fc221f58640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc21c112870 0x7fc21c19e640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:37872/0 (socket says 172.21.15.26:37872) 2024-09-11T23:40:32.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:32.376+0000 7fc220f56640 1 -- 172.21.15.26:0/223084847 learned_addr learned my addr 172.21.15.26:0/223084847 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:40:32.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:32.376+0000 7fc222759640 1 -- 172.21.15.26:0/223084847 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc21c118440 con 0x7fc21c076720 2024-09-11T23:40:32.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:32.376+0000 7fc222759640 1 -- 172.21.15.26:0/223084847 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fc21c1182c0 con 0x7fc21c077040 2024-09-11T23:40:32.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:32.376+0000 7fc222759640 1 -- 172.21.15.26:0/223084847 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fc21c1185c0 con 0x7fc21c112870 2024-09-11T23:40:32.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:32.377+0000 7fc220f56640 1 -- 172.21.15.26:0/223084847 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc21c112870 msgr2=0x7fc21c19e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:32.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:32.377+0000 7fc220f56640 1 --2- 172.21.15.26:0/223084847 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc21c112870 0x7fc21c19e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:32.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:32.377+0000 7fc220f56640 1 -- 172.21.15.26:0/223084847 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc21c076720 msgr2=0x7fc21c1a4570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:32.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:32.377+0000 7fc220f56640 1 --2- 172.21.15.26:0/223084847 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc21c076720 0x7fc21c1a4570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:32.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:32.377+0000 7fc220f56640 1 -- 172.21.15.26:0/223084847 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc21c19eeb0 con 0x7fc21c077040 2024-09-11T23:40:32.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:32.377+0000 7fc220f56640 1 --2- 172.21.15.26:0/223084847 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc21c077040 0x7fc21c1a4ab0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fc21800ea60 tx=0x7fc21800ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:40:32.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:32.378+0000 7fc2127fc640 1 -- 172.21.15.26:0/223084847 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc21800cde0 con 0x7fc21c077040 2024-09-11T23:40:32.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:32.378+0000 7fc2127fc640 1 -- 172.21.15.26:0/223084847 <== mon.2 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc2180035f0 con 0x7fc21c077040 2024-09-11T23:40:32.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:32.378+0000 7fc2127fc640 1 -- 172.21.15.26:0/223084847 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc218010670 con 0x7fc21c077040 2024-09-11T23:40:32.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:32.379+0000 7fc222759640 1 -- 172.21.15.26:0/223084847 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc21c19f1a0 con 0x7fc21c077040 2024-09-11T23:40:32.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:32.379+0000 7fc222759640 1 -- 172.21.15.26:0/223084847 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc21c1ab320 con 0x7fc21c077040 2024-09-11T23:40:32.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:32.384+0000 7fc2127fc640 1 -- 172.21.15.26:0/223084847 <== mon.2 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fc2180037a0 con 0x7fc21c077040 2024-09-11T23:40:32.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:32.385+0000 7fc2127fc640 1 --2- 172.21.15.26:0/223084847 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fc2080a2ec0 0x7fc2080a5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:32.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:32.385+0000 7fc2127fc640 1 -- 172.21.15.26:0/223084847 <== mon.2 v2:172.21.15.76:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 8586+0+0 (secure 0 0 0) 0x7fc2180cf990 con 0x7fc21c077040 2024-09-11T23:40:32.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:32.384+0000 7fc222759640 1 -- 172.21.15.26:0/223084847 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc21c077e60 con 0x7fc21c077040 2024-09-11T23:40:32.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:32.386+0000 7fc221757640 1 --2- 172.21.15.26:0/223084847 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fc2080a2ec0 0x7fc2080a5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:32.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:32.390+0000 7fc221757640 1 --2- 172.21.15.26:0/223084847 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fc2080a2ec0 0x7fc2080a5380 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fc20c005ab0 tx=0x7fc20c00b040 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:40:32.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:32.394+0000 7fc2127fc640 1 -- 172.21.15.26:0/223084847 <== mon.2 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc21801d060 con 0x7fc21c077040 2024-09-11T23:40:32.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:32.814+0000 7fc222759640 1 -- 172.21.15.26:0/223084847 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fc21c1ab5f0 con 0x7fc21c077040 2024-09-11T23:40:32.835 INFO:teuthology.orchestra.run.smithi026.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-11T23:40:32.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:32.835+0000 7fc2127fc640 1 -- 172.21.15.26:0/223084847 <== mon.2 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v94) ==== 161+0+0 (secure 0 0 0) 0x7fc21809c510 con 0x7fc21c077040 2024-09-11T23:40:32.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:32.838+0000 7fc222759640 1 -- 172.21.15.26:0/223084847 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fc2080a2ec0 msgr2=0x7fc2080a5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:32.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:32.838+0000 7fc222759640 1 --2- 172.21.15.26:0/223084847 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fc2080a2ec0 0x7fc2080a5380 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fc20c005ab0 tx=0x7fc20c00b040 comp rx=0 tx=0).stop 2024-09-11T23:40:32.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:32.838+0000 7fc222759640 1 -- 172.21.15.26:0/223084847 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc21c077040 msgr2=0x7fc21c1a4ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:32.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:32.838+0000 7fc222759640 1 --2- 172.21.15.26:0/223084847 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc21c077040 0x7fc21c1a4ab0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fc21800ea60 tx=0x7fc21800ef20 comp rx=0 tx=0).stop 2024-09-11T23:40:32.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:32.838+0000 7fc222759640 1 -- 172.21.15.26:0/223084847 shutdown_connections 2024-09-11T23:40:32.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:32.838+0000 7fc222759640 1 --2- 172.21.15.26:0/223084847 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fc2080a2ec0 0x7fc2080a5380 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:32.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:32.838+0000 7fc222759640 1 --2- 172.21.15.26:0/223084847 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc21c077040 0x7fc21c1a4ab0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:32.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:32.838+0000 7fc222759640 1 --2- 172.21.15.26:0/223084847 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc21c112870 0x7fc21c19e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:32.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:32.838+0000 7fc222759640 1 --2- 172.21.15.26:0/223084847 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc21c076720 0x7fc21c1a4570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:32.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:32.838+0000 7fc222759640 1 -- 172.21.15.26:0/223084847 >> 172.21.15.26:0/223084847 conn(0x7fc21c0ff910 msgr2=0x7fc21c101d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:40:32.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:32.839+0000 7fc222759640 1 -- 172.21.15.26:0/223084847 shutdown_connections 2024-09-11T23:40:32.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:32.839+0000 7fc222759640 1 -- 172.21.15.26:0/223084847 wait complete. 2024-09-11T23:40:33.229 INFO:journalctl@ceph.mds.a.smithi026.stdout:Sep 11 23:40:32 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mds-a[72516]: ignoring --setuser ceph since I am not root 2024-09-11T23:40:33.230 INFO:journalctl@ceph.mds.a.smithi026.stdout:Sep 11 23:40:32 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mds-a[72516]: ignoring --setgroup ceph since I am not root 2024-09-11T23:40:33.230 INFO:journalctl@ceph.mds.a.smithi026.stdout:Sep 11 23:40:32 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mds-a[72516]: starting mds.a at 2024-09-11T23:40:33.231 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:40:32 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mon-a[31168]: 2024-09-11T23:40:32.818+0000 7f5d90d31640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-11T23:40:33.231 INFO:journalctl@ceph.mds.g.smithi026.stdout:Sep 11 23:40:32 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mds-g[72114]: ignoring --setuser ceph since I am not root 2024-09-11T23:40:33.231 INFO:journalctl@ceph.mds.g.smithi026.stdout:Sep 11 23:40:32 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mds-g[72114]: ignoring --setgroup ceph since I am not root 2024-09-11T23:40:33.231 INFO:journalctl@ceph.mds.g.smithi026.stdout:Sep 11 23:40:32 smithi026 ceph-mds[72119]: -- 172.21.15.26:0/895566833 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5555cf3e6680 con 0x5555cf3ba800 2024-09-11T23:40:33.231 INFO:journalctl@ceph.mds.g.smithi026.stdout:Sep 11 23:40:32 smithi026 ceph-mds[72119]: -- 172.21.15.26:0/895566833 <== mon.0 v2:172.21.15.26:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5555cf3dd860 con 0x5555cf3ba800 2024-09-11T23:40:33.231 INFO:journalctl@ceph.mds.g.smithi026.stdout:Sep 11 23:40:32 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mds-g[72114]: starting mds.g at 2024-09-11T23:40:33.303 INFO:journalctl@ceph.mds.b.smithi076.stdout:Sep 11 23:40:32 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mds-b[65334]: ignoring --setuser ceph since I am not root 2024-09-11T23:40:33.303 INFO:journalctl@ceph.mds.b.smithi076.stdout:Sep 11 23:40:32 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mds-b[65334]: ignoring --setgroup ceph since I am not root 2024-09-11T23:40:33.303 INFO:journalctl@ceph.mds.b.smithi076.stdout:Sep 11 23:40:32 smithi076 ceph-mds[65338]: -- 172.21.15.76:0/657743050 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55f21f302680 con 0x55f21f2d6800 2024-09-11T23:40:33.304 INFO:journalctl@ceph.mds.b.smithi076.stdout:Sep 11 23:40:32 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mds-b[65334]: starting mds.b at 2024-09-11T23:40:33.304 INFO:journalctl@ceph.mds.e.smithi076.stdout:Sep 11 23:40:32 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mds-e[64717]: ignoring --setuser ceph since I am not root 2024-09-11T23:40:33.304 INFO:journalctl@ceph.mds.e.smithi076.stdout:Sep 11 23:40:32 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mds-e[64717]: ignoring --setgroup ceph since I am not root 2024-09-11T23:40:33.304 INFO:journalctl@ceph.mds.e.smithi076.stdout:Sep 11 23:40:32 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mds-e[64717]: starting mds.e at 2024-09-11T23:40:33.304 INFO:journalctl@ceph.mds.h.smithi076.stdout:Sep 11 23:40:32 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mds-h[65015]: ignoring --setuser ceph since I am not root 2024-09-11T23:40:33.304 INFO:journalctl@ceph.mds.h.smithi076.stdout:Sep 11 23:40:32 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mds-h[65015]: ignoring --setgroup ceph since I am not root 2024-09-11T23:40:33.304 INFO:journalctl@ceph.mds.h.smithi076.stdout:Sep 11 23:40:32 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mds-h[65015]: starting mds.h at 2024-09-11T23:40:33.305 INFO:journalctl@ceph.mds.k.smithi076.stdout:Sep 11 23:40:32 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mds-k[64405]: ignoring --setuser ceph since I am not root 2024-09-11T23:40:33.305 INFO:journalctl@ceph.mds.k.smithi076.stdout:Sep 11 23:40:32 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mds-k[64405]: ignoring --setgroup ceph since I am not root 2024-09-11T23:40:33.305 INFO:journalctl@ceph.mds.k.smithi076.stdout:Sep 11 23:40:32 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mds-k[64405]: starting mds.k at 2024-09-11T23:40:33.488 INFO:journalctl@ceph.mds.c.smithi136.stdout:Sep 11 23:40:33 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mds-c[65041]: ignoring --setuser ceph since I am not root 2024-09-11T23:40:33.489 INFO:journalctl@ceph.mds.c.smithi136.stdout:Sep 11 23:40:33 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mds-c[65041]: ignoring --setgroup ceph since I am not root 2024-09-11T23:40:33.489 INFO:journalctl@ceph.mds.c.smithi136.stdout:Sep 11 23:40:33 smithi136 ceph-mds[65045]: -- 172.21.15.136:0/2960336516 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x562620dbe680 con 0x562620d93400 2024-09-11T23:40:33.489 INFO:journalctl@ceph.mds.c.smithi136.stdout:Sep 11 23:40:33 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mds-c[65041]: starting mds.c at 2024-09-11T23:40:33.489 INFO:journalctl@ceph.mds.f.smithi136.stdout:Sep 11 23:40:33 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mds-f[64739]: ignoring --setuser ceph since I am not root 2024-09-11T23:40:33.489 INFO:journalctl@ceph.mds.f.smithi136.stdout:Sep 11 23:40:33 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mds-f[64739]: ignoring --setgroup ceph since I am not root 2024-09-11T23:40:33.490 INFO:journalctl@ceph.mds.f.smithi136.stdout:Sep 11 23:40:33 smithi136 ceph-mds[64743]: -- 172.21.15.136:0/2614147312 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5643cba38680 con 0x5643cba0c800 2024-09-11T23:40:33.490 INFO:journalctl@ceph.mds.f.smithi136.stdout:Sep 11 23:40:33 smithi136 ceph-mds[64743]: -- 172.21.15.136:0/2614147312 <== mon.2 v2:172.21.15.76:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5643cba2f860 con 0x5643cba0c800 2024-09-11T23:40:33.490 INFO:journalctl@ceph.mds.f.smithi136.stdout:Sep 11 23:40:33 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mds-f[64739]: starting mds.f at 2024-09-11T23:40:33.490 INFO:journalctl@ceph.mds.i.smithi136.stdout:Sep 11 23:40:33 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mds-i[65341]: ignoring --setuser ceph since I am not root 2024-09-11T23:40:33.490 INFO:journalctl@ceph.mds.i.smithi136.stdout:Sep 11 23:40:33 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mds-i[65341]: ignoring --setgroup ceph since I am not root 2024-09-11T23:40:33.490 INFO:journalctl@ceph.mds.i.smithi136.stdout:Sep 11 23:40:33 smithi136 ceph-mds[65345]: -- 172.21.15.136:0/1009908542 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5614f16cc680 con 0x5614f16a1400 2024-09-11T23:40:33.490 INFO:journalctl@ceph.mds.i.smithi136.stdout:Sep 11 23:40:33 smithi136 ceph-mds[65345]: -- 172.21.15.136:0/1009908542 <== mon.1 v2:172.21.15.136:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5614f16c3860 con 0x5614f16a1400 2024-09-11T23:40:33.490 INFO:journalctl@ceph.mds.i.smithi136.stdout:Sep 11 23:40:33 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mds-i[65341]: starting mds.i at 2024-09-11T23:40:33.490 INFO:journalctl@ceph.mds.l.smithi136.stdout:Sep 11 23:40:33 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mds-l[64436]: ignoring --setuser ceph since I am not root 2024-09-11T23:40:33.491 INFO:journalctl@ceph.mds.l.smithi136.stdout:Sep 11 23:40:33 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mds-l[64436]: ignoring --setgroup ceph since I am not root 2024-09-11T23:40:33.491 INFO:journalctl@ceph.mds.l.smithi136.stdout:Sep 11 23:40:33 smithi136 ceph-mds[64440]: -- 172.21.15.136:0/2592840939 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x557965fb6680 con 0x557965f8a800 2024-09-11T23:40:33.491 INFO:journalctl@ceph.mds.l.smithi136.stdout:Sep 11 23:40:33 smithi136 ceph-mds[64440]: -- 172.21.15.136:0/2592840939 <== mon.1 v2:172.21.15.136:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557965fad860 con 0x557965f8a800 2024-09-11T23:40:33.491 INFO:journalctl@ceph.mds.l.smithi136.stdout:Sep 11 23:40:33 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mds-l[64436]: starting mds.l at 2024-09-11T23:40:33.617 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph fs rm cephfs --yes-i-really-mean-it 2024-09-11T23:40:34.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:40:33 smithi026 ceph-mon[31172]: pgmap v699: 97 pgs: 41 active+clean+snaptrim_wait, 22 active+clean+snaptrim, 34 active+clean; 2.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 1.3 KiB/s wr, 13 op/s 2024-09-11T23:40:34.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:40:33 smithi026 ceph-mon[31172]: from='client.? 172.21.15.26:0/223084847' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-11T23:40:34.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:40:33 smithi026 ceph-mon[31172]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-11T23:40:34.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:40:33 smithi026 ceph-mon[31172]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-11T23:40:34.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:40:33 smithi026 ceph-mon[31172]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-11T23:40:34.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:40:33 smithi026 ceph-mon[31172]: Health check cleared: MDS_DAMAGE (was: 1 MDSs report damaged metadata) 2024-09-11T23:40:34.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:40:33 smithi026 ceph-mon[31172]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-11T23:40:34.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:40:33 smithi026 ceph-mon[31172]: osdmap e118: 12 total, 12 up, 12 in 2024-09-11T23:40:34.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:40:33 smithi026 ceph-mon[31172]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-09-11T23:40:34.052 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:40:33 smithi076 ceph-mon[35296]: pgmap v699: 97 pgs: 41 active+clean+snaptrim_wait, 22 active+clean+snaptrim, 34 active+clean; 2.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 1.3 KiB/s wr, 13 op/s 2024-09-11T23:40:34.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:40:33 smithi076 ceph-mon[35296]: from='client.? 172.21.15.26:0/223084847' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-11T23:40:34.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:40:33 smithi076 ceph-mon[35296]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-11T23:40:34.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:40:33 smithi076 ceph-mon[35296]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-11T23:40:34.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:40:33 smithi076 ceph-mon[35296]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-11T23:40:34.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:40:33 smithi076 ceph-mon[35296]: Health check cleared: MDS_DAMAGE (was: 1 MDSs report damaged metadata) 2024-09-11T23:40:34.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:40:33 smithi076 ceph-mon[35296]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-11T23:40:34.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:40:33 smithi076 ceph-mon[35296]: osdmap e118: 12 total, 12 up, 12 in 2024-09-11T23:40:34.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:40:33 smithi076 ceph-mon[35296]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-09-11T23:40:34.104 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:40:34.238 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:40:33 smithi136 ceph-mon[35121]: pgmap v699: 97 pgs: 41 active+clean+snaptrim_wait, 22 active+clean+snaptrim, 34 active+clean; 2.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 1.3 KiB/s wr, 13 op/s 2024-09-11T23:40:34.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:40:33 smithi136 ceph-mon[35121]: from='client.? 172.21.15.26:0/223084847' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-11T23:40:34.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:40:33 smithi136 ceph-mon[35121]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-11T23:40:34.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:40:33 smithi136 ceph-mon[35121]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-11T23:40:34.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:40:33 smithi136 ceph-mon[35121]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-11T23:40:34.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:40:33 smithi136 ceph-mon[35121]: Health check cleared: MDS_DAMAGE (was: 1 MDSs report damaged metadata) 2024-09-11T23:40:34.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:40:33 smithi136 ceph-mon[35121]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-11T23:40:34.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:40:33 smithi136 ceph-mon[35121]: osdmap e118: 12 total, 12 up, 12 in 2024-09-11T23:40:34.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:40:33 smithi136 ceph-mon[35121]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-09-11T23:40:34.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:34.720+0000 7fe635081640 1 -- 172.21.15.26:0/2615248302 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe63011b140 msgr2=0x7fe63011d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:34.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:34.720+0000 7fe635081640 1 --2- 172.21.15.26:0/2615248302 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe63011b140 0x7fe63011d530 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fe628009be0 tx=0x7fe62802a710 comp rx=0 tx=0).stop 2024-09-11T23:40:34.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:34.727+0000 7fe635081640 1 -- 172.21.15.26:0/2615248302 shutdown_connections 2024-09-11T23:40:34.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:34.727+0000 7fe635081640 1 --2- 172.21.15.26:0/2615248302 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe63011b140 0x7fe63011d530 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:34.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:34.727+0000 7fe635081640 1 --2- 172.21.15.26:0/2615248302 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe630109830 0x7fe630109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:34.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:34.727+0000 7fe635081640 1 --2- 172.21.15.26:0/2615248302 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe630109450 0x7fe630110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:34.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:34.727+0000 7fe635081640 1 -- 172.21.15.26:0/2615248302 >> 172.21.15.26:0/2615248302 conn(0x7fe63006e140 msgr2=0x7fe63006e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:40:34.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:34.729+0000 7fe635081640 1 -- 172.21.15.26:0/2615248302 shutdown_connections 2024-09-11T23:40:34.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:34.729+0000 7fe635081640 1 -- 172.21.15.26:0/2615248302 wait complete. 2024-09-11T23:40:34.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:34.730+0000 7fe635081640 1 Processor -- start 2024-09-11T23:40:34.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:34.730+0000 7fe635081640 1 -- start start 2024-09-11T23:40:34.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:34.730+0000 7fe635081640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe630109450 0x7fe6301a72b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:34.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:34.730+0000 7fe635081640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe630109830 0x7fe6301a77f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:34.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:34.730+0000 7fe635081640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe63011b140 0x7fe6301abb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:34.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:34.730+0000 7fe635081640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fe630120e30 con 0x7fe630109450 2024-09-11T23:40:34.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:34.730+0000 7fe635081640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fe630120cb0 con 0x7fe63011b140 2024-09-11T23:40:34.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:34.730+0000 7fe635081640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe630120fb0 con 0x7fe630109830 2024-09-11T23:40:34.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:34.732+0000 7fe62f59e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe63011b140 0x7fe6301abb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:34.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:34.733+0000 7fe62f59e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe63011b140 0x7fe6301abb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.26:59794/0 (socket says 172.21.15.26:59794) 2024-09-11T23:40:34.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:34.733+0000 7fe62f59e640 1 -- 172.21.15.26:0/3357561275 learned_addr learned my addr 172.21.15.26:0/3357561275 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:40:34.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:34.735+0000 7fe62f59e640 1 -- 172.21.15.26:0/3357561275 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe630109830 msgr2=0x7fe6301a77f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-11T23:40:34.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:34.735+0000 7fe62f59e640 1 --2- 172.21.15.26:0/3357561275 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe630109830 0x7fe6301a77f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:34.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:34.735+0000 7fe62f59e640 1 -- 172.21.15.26:0/3357561275 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe630109450 msgr2=0x7fe6301a72b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-11T23:40:34.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:34.736+0000 7fe62f59e640 1 --2- 172.21.15.26:0/3357561275 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe630109450 0x7fe6301a72b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:34.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:34.736+0000 7fe62f59e640 1 -- 172.21.15.26:0/3357561275 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6301ac280 con 0x7fe63011b140 2024-09-11T23:40:34.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:34.736+0000 7fe62f59e640 1 --2- 172.21.15.26:0/3357561275 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe63011b140 0x7fe6301abb80 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fe628034040 tx=0x7fe628030890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:40:34.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:34.737+0000 7fe617fff640 1 -- 172.21.15.26:0/3357561275 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe628041020 con 0x7fe63011b140 2024-09-11T23:40:34.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:34.738+0000 7fe635081640 1 -- 172.21.15.26:0/3357561275 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6301ac510 con 0x7fe63011b140 2024-09-11T23:40:34.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:34.738+0000 7fe635081640 1 -- 172.21.15.26:0/3357561275 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe630110920 con 0x7fe63011b140 2024-09-11T23:40:34.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:34.738+0000 7fe617fff640 1 -- 172.21.15.26:0/3357561275 <== mon.2 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe62803cb50 con 0x7fe63011b140 2024-09-11T23:40:34.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:34.738+0000 7fe617fff640 1 -- 172.21.15.26:0/3357561275 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe628036040 con 0x7fe63011b140 2024-09-11T23:40:34.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:34.742+0000 7fe635081640 1 -- 172.21.15.26:0/3357561275 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe61c005290 con 0x7fe63011b140 2024-09-11T23:40:34.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:34.747+0000 7fe617fff640 1 -- 172.21.15.26:0/3357561275 <== mon.2 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fe628044020 con 0x7fe63011b140 2024-09-11T23:40:34.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:34.748+0000 7fe617fff640 1 --2- 172.21.15.26:0/3357561275 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fe6180a2f00 0x7fe6180a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:34.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:34.748+0000 7fe617fff640 1 -- 172.21.15.26:0/3357561275 <== mon.2 v2:172.21.15.76:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 8972+0+0 (secure 0 0 0) 0x7fe6280f0470 con 0x7fe63011b140 2024-09-11T23:40:34.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:34.751+0000 7fe62ed9d640 1 --2- 172.21.15.26:0/3357561275 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fe6180a2f00 0x7fe6180a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:34.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:34.752+0000 7fe62ed9d640 1 --2- 172.21.15.26:0/3357561275 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fe6180a2f00 0x7fe6180a53c0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fe620009670 tx=0x7fe620006bd0 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:40:34.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:34.756+0000 7fe617fff640 1 -- 172.21.15.26:0/3357561275 <== mon.2 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe62804b020 con 0x7fe63011b140 2024-09-11T23:40:35.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:35.175+0000 7fe635081640 1 -- 172.21.15.26:0/3357561275 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fe61c005850 con 0x7fe63011b140 2024-09-11T23:40:35.175 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:40:34 smithi026 ceph-mon[31172]: mds.? [v2:172.21.15.26:6839/377864103,v1:172.21.15.26:6841/377864103] up:boot 2024-09-11T23:40:35.176 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:40:34 smithi026 ceph-mon[31172]: mds.? [v2:172.21.15.26:6838/1327410980,v1:172.21.15.26:6840/1327410980] up:boot 2024-09-11T23:40:35.176 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:40:34 smithi026 ceph-mon[31172]: mds.? [v2:172.21.15.76:6832/665427879,v1:172.21.15.76:6833/665427879] up:boot 2024-09-11T23:40:35.176 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:40:34 smithi026 ceph-mon[31172]: mds.? [v2:172.21.15.76:6835/2707775727,v1:172.21.15.76:6836/2707775727] up:boot 2024-09-11T23:40:35.176 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:40:34 smithi026 ceph-mon[31172]: mds.? [v2:172.21.15.76:6834/991588372,v1:172.21.15.76:6837/991588372] up:boot 2024-09-11T23:40:35.176 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:40:34 smithi026 ceph-mon[31172]: mds.? [v2:172.21.15.76:6838/2232326203,v1:172.21.15.76:6839/2232326203] up:boot 2024-09-11T23:40:35.176 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:40:34 smithi026 ceph-mon[31172]: mds.? [v2:172.21.15.136:6832/3053397300,v1:172.21.15.136:6833/3053397300] up:boot 2024-09-11T23:40:35.176 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:40:34 smithi026 ceph-mon[31172]: mds.? [v2:172.21.15.136:6834/4195684206,v1:172.21.15.136:6835/4195684206] up:boot 2024-09-11T23:40:35.176 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:40:34 smithi026 ceph-mon[31172]: mds.? [v2:172.21.15.136:6836/3331070377,v1:172.21.15.136:6837/3331070377] up:boot 2024-09-11T23:40:35.176 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:40:34 smithi026 ceph-mon[31172]: mds.? [v2:172.21.15.136:6838/892082154,v1:172.21.15.136:6839/892082154] up:boot 2024-09-11T23:40:35.176 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:40:34 smithi026 ceph-mon[31172]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-09-11T23:40:35.176 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:40:34 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-11T23:40:35.176 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:40:34 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-11T23:40:35.176 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:40:34 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-11T23:40:35.177 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:40:34 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-11T23:40:35.177 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:40:34 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-11T23:40:35.177 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:40:34 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-11T23:40:35.177 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:40:34 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-11T23:40:35.177 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:40:34 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-11T23:40:35.177 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:40:34 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-11T23:40:35.177 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:40:34 smithi026 ceph-mon[31172]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-11T23:40:35.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:40:34 smithi136 ceph-mon[35121]: mds.? [v2:172.21.15.26:6839/377864103,v1:172.21.15.26:6841/377864103] up:boot 2024-09-11T23:40:35.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:40:34 smithi136 ceph-mon[35121]: mds.? [v2:172.21.15.26:6838/1327410980,v1:172.21.15.26:6840/1327410980] up:boot 2024-09-11T23:40:35.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:40:34 smithi136 ceph-mon[35121]: mds.? [v2:172.21.15.76:6832/665427879,v1:172.21.15.76:6833/665427879] up:boot 2024-09-11T23:40:35.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:40:34 smithi136 ceph-mon[35121]: mds.? [v2:172.21.15.76:6835/2707775727,v1:172.21.15.76:6836/2707775727] up:boot 2024-09-11T23:40:35.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:40:34 smithi136 ceph-mon[35121]: mds.? [v2:172.21.15.76:6834/991588372,v1:172.21.15.76:6837/991588372] up:boot 2024-09-11T23:40:35.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:40:34 smithi136 ceph-mon[35121]: mds.? [v2:172.21.15.76:6838/2232326203,v1:172.21.15.76:6839/2232326203] up:boot 2024-09-11T23:40:35.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:40:34 smithi136 ceph-mon[35121]: mds.? [v2:172.21.15.136:6832/3053397300,v1:172.21.15.136:6833/3053397300] up:boot 2024-09-11T23:40:35.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:40:34 smithi136 ceph-mon[35121]: mds.? [v2:172.21.15.136:6834/4195684206,v1:172.21.15.136:6835/4195684206] up:boot 2024-09-11T23:40:35.240 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:40:34 smithi136 ceph-mon[35121]: mds.? [v2:172.21.15.136:6836/3331070377,v1:172.21.15.136:6837/3331070377] up:boot 2024-09-11T23:40:35.240 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:40:34 smithi136 ceph-mon[35121]: mds.? [v2:172.21.15.136:6838/892082154,v1:172.21.15.136:6839/892082154] up:boot 2024-09-11T23:40:35.240 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:40:34 smithi136 ceph-mon[35121]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-09-11T23:40:35.240 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:40:34 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-11T23:40:35.240 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:40:34 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-11T23:40:35.240 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:40:34 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-11T23:40:35.240 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:40:34 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-11T23:40:35.240 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:40:34 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-11T23:40:35.240 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:40:34 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-11T23:40:35.240 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:40:34 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-11T23:40:35.240 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:40:34 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-11T23:40:35.240 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:40:34 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-11T23:40:35.240 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:40:34 smithi136 ceph-mon[35121]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-11T23:40:35.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:40:34 smithi076 ceph-mon[35296]: mds.? [v2:172.21.15.26:6839/377864103,v1:172.21.15.26:6841/377864103] up:boot 2024-09-11T23:40:35.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:40:34 smithi076 ceph-mon[35296]: mds.? [v2:172.21.15.26:6838/1327410980,v1:172.21.15.26:6840/1327410980] up:boot 2024-09-11T23:40:35.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:40:34 smithi076 ceph-mon[35296]: mds.? [v2:172.21.15.76:6832/665427879,v1:172.21.15.76:6833/665427879] up:boot 2024-09-11T23:40:35.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:40:34 smithi076 ceph-mon[35296]: mds.? [v2:172.21.15.76:6835/2707775727,v1:172.21.15.76:6836/2707775727] up:boot 2024-09-11T23:40:35.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:40:34 smithi076 ceph-mon[35296]: mds.? [v2:172.21.15.76:6834/991588372,v1:172.21.15.76:6837/991588372] up:boot 2024-09-11T23:40:35.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:40:34 smithi076 ceph-mon[35296]: mds.? [v2:172.21.15.76:6838/2232326203,v1:172.21.15.76:6839/2232326203] up:boot 2024-09-11T23:40:35.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:40:34 smithi076 ceph-mon[35296]: mds.? [v2:172.21.15.136:6832/3053397300,v1:172.21.15.136:6833/3053397300] up:boot 2024-09-11T23:40:35.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:40:34 smithi076 ceph-mon[35296]: mds.? [v2:172.21.15.136:6834/4195684206,v1:172.21.15.136:6835/4195684206] up:boot 2024-09-11T23:40:35.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:40:34 smithi076 ceph-mon[35296]: mds.? [v2:172.21.15.136:6836/3331070377,v1:172.21.15.136:6837/3331070377] up:boot 2024-09-11T23:40:35.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:40:34 smithi076 ceph-mon[35296]: mds.? [v2:172.21.15.136:6838/892082154,v1:172.21.15.136:6839/892082154] up:boot 2024-09-11T23:40:35.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:40:34 smithi076 ceph-mon[35296]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-09-11T23:40:35.304 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:40:34 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-11T23:40:35.304 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:40:34 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-11T23:40:35.304 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:40:34 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-11T23:40:35.304 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:40:34 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-11T23:40:35.304 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:40:34 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-11T23:40:35.304 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:40:34 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-11T23:40:35.304 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:40:34 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-11T23:40:35.304 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:40:34 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-11T23:40:35.304 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:40:34 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-11T23:40:35.304 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:40:34 smithi076 ceph-mon[35296]: from='mgr.15171 172.21.15.26:0/402391872' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-11T23:40:35.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:35.855+0000 7fe617fff640 1 -- 172.21.15.26:0/3357561275 <== mon.2 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v96) ==== 104+0+0 (secure 0 0 0) 0x7fe6280bce70 con 0x7fe63011b140 2024-09-11T23:40:35.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:35.859+0000 7fe615ffb640 1 -- 172.21.15.26:0/3357561275 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fe6180a2f00 msgr2=0x7fe6180a53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:35.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:35.859+0000 7fe615ffb640 1 --2- 172.21.15.26:0/3357561275 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fe6180a2f00 0x7fe6180a53c0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fe620009670 tx=0x7fe620006bd0 comp rx=0 tx=0).stop 2024-09-11T23:40:35.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:35.859+0000 7fe615ffb640 1 -- 172.21.15.26:0/3357561275 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe63011b140 msgr2=0x7fe6301abb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:35.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:35.859+0000 7fe615ffb640 1 --2- 172.21.15.26:0/3357561275 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe63011b140 0x7fe6301abb80 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fe628034040 tx=0x7fe628030890 comp rx=0 tx=0).stop 2024-09-11T23:40:35.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:35.859+0000 7fe615ffb640 1 -- 172.21.15.26:0/3357561275 shutdown_connections 2024-09-11T23:40:35.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:35.859+0000 7fe615ffb640 1 --2- 172.21.15.26:0/3357561275 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7fe6180a2f00 0x7fe6180a53c0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:35.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:35.859+0000 7fe615ffb640 1 --2- 172.21.15.26:0/3357561275 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe63011b140 0x7fe6301abb80 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:35.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:35.859+0000 7fe615ffb640 1 --2- 172.21.15.26:0/3357561275 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe630109830 0x7fe6301a77f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:35.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:35.859+0000 7fe615ffb640 1 --2- 172.21.15.26:0/3357561275 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe630109450 0x7fe6301a72b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:35.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:35.859+0000 7fe615ffb640 1 -- 172.21.15.26:0/3357561275 >> 172.21.15.26:0/3357561275 conn(0x7fe63006e140 msgr2=0x7fe63011bca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:40:35.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:35.859+0000 7fe615ffb640 1 -- 172.21.15.26:0/3357561275 shutdown_connections 2024-09-11T23:40:35.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:35.859+0000 7fe615ffb640 1 -- 172.21.15.26:0/3357561275 wait complete. 2024-09-11T23:40:36.238 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:40:35 smithi136 ceph-mon[35121]: pgmap v701: 97 pgs: 41 active+clean+snaptrim_wait, 22 active+clean+snaptrim, 34 active+clean; 2.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.4 KiB/s wr, 11 op/s 2024-09-11T23:40:36.238 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:40:35 smithi136 ceph-mon[35121]: from='client.? 172.21.15.26:0/3357561275' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-11T23:40:36.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:40:35 smithi136 ceph-mon[35121]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-11T23:40:36.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:40:35 smithi026 ceph-mon[31172]: pgmap v701: 97 pgs: 41 active+clean+snaptrim_wait, 22 active+clean+snaptrim, 34 active+clean; 2.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.4 KiB/s wr, 11 op/s 2024-09-11T23:40:36.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:40:35 smithi026 ceph-mon[31172]: from='client.? 172.21.15.26:0/3357561275' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-11T23:40:36.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:40:35 smithi026 ceph-mon[31172]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-11T23:40:36.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:40:35 smithi076 ceph-mon[35296]: pgmap v701: 97 pgs: 41 active+clean+snaptrim_wait, 22 active+clean+snaptrim, 34 active+clean; 2.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.4 KiB/s wr, 11 op/s 2024-09-11T23:40:36.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:40:35 smithi076 ceph-mon[35296]: from='client.? 172.21.15.26:0/3357561275' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-11T23:40:36.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:40:35 smithi076 ceph-mon[35296]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-11T23:40:36.891 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-11T23:40:37.238 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:40:36 smithi136 ceph-mon[35121]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-11T23:40:37.238 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:40:36 smithi136 ceph-mon[35121]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-11T23:40:37.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:40:36 smithi136 ceph-mon[35121]: Cluster is now healthy 2024-09-11T23:40:37.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:40:36 smithi136 ceph-mon[35121]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-11T23:40:37.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:40:36 smithi136 ceph-mon[35121]: fsmap 12 up:standby 2024-09-11T23:40:37.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:40:36 smithi026 ceph-mon[31172]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-11T23:40:37.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:40:36 smithi026 ceph-mon[31172]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-11T23:40:37.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:40:36 smithi026 ceph-mon[31172]: Cluster is now healthy 2024-09-11T23:40:37.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:40:36 smithi026 ceph-mon[31172]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-11T23:40:37.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:40:36 smithi026 ceph-mon[31172]: fsmap 12 up:standby 2024-09-11T23:40:37.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:40:36 smithi076 ceph-mon[35296]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-11T23:40:37.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:40:36 smithi076 ceph-mon[35296]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-11T23:40:37.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:40:36 smithi076 ceph-mon[35296]: Cluster is now healthy 2024-09-11T23:40:37.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:40:36 smithi076 ceph-mon[35296]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-11T23:40:37.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:40:36 smithi076 ceph-mon[35296]: fsmap 12 up:standby 2024-09-11T23:40:37.356 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:40:38.159 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:40:37 smithi026 ceph-mon[31172]: pgmap v702: 97 pgs: 39 active+clean+snaptrim_wait, 21 active+clean+snaptrim, 37 active+clean; 2.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 204 B/s wr, 7 op/s 2024-09-11T23:40:38.238 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:40:37 smithi136 ceph-mon[35121]: pgmap v702: 97 pgs: 39 active+clean+snaptrim_wait, 21 active+clean+snaptrim, 37 active+clean; 2.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 204 B/s wr, 7 op/s 2024-09-11T23:40:38.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:40:37 smithi076 ceph-mon[35296]: pgmap v702: 97 pgs: 39 active+clean+snaptrim_wait, 21 active+clean+snaptrim, 37 active+clean; 2.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 204 B/s wr, 7 op/s 2024-09-11T23:40:38.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:38.310+0000 7f6b0e77f640 1 -- 172.21.15.26:0/553028275 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6b08109450 msgr2=0x7f6b08109830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:38.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:38.310+0000 7f6b0e77f640 1 --2- 172.21.15.26:0/553028275 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6b08109450 0x7f6b08109830 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f6b00009900 tx=0x7f6b0002b6b0 comp rx=0 tx=0).stop 2024-09-11T23:40:38.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:38.313+0000 7f6b0e77f640 1 -- 172.21.15.26:0/553028275 shutdown_connections 2024-09-11T23:40:38.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:38.314+0000 7f6b0e77f640 1 --2- 172.21.15.26:0/553028275 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6b0810ea80 0x7f6b081154f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:38.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:38.314+0000 7f6b0e77f640 1 --2- 172.21.15.26:0/553028275 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b08109e00 0x7f6b0810e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:38.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:38.314+0000 7f6b0e77f640 1 --2- 172.21.15.26:0/553028275 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6b08109450 0x7f6b08109830 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:38.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:38.314+0000 7f6b0e77f640 1 -- 172.21.15.26:0/553028275 >> 172.21.15.26:0/553028275 conn(0x7f6b0806e2d0 msgr2=0x7f6b0806e6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:40:38.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:38.314+0000 7f6b0e77f640 1 -- 172.21.15.26:0/553028275 shutdown_connections 2024-09-11T23:40:38.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:38.318+0000 7f6b0e77f640 1 -- 172.21.15.26:0/553028275 wait complete. 2024-09-11T23:40:38.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:38.319+0000 7f6b0e77f640 1 Processor -- start 2024-09-11T23:40:38.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:38.319+0000 7f6b0e77f640 1 -- start start 2024-09-11T23:40:38.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:38.320+0000 7f6b0e77f640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6b08109450 0x7f6b081a45b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:38.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:38.320+0000 7f6b07fff640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6b08109450 0x7f6b081a45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:38.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:38.321+0000 7f6b07fff640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6b08109450 0x7f6b081a45b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:37930/0 (socket says 172.21.15.26:37930) 2024-09-11T23:40:38.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:38.321+0000 7f6b07fff640 1 -- 172.21.15.26:0/721262308 learned_addr learned my addr 172.21.15.26:0/721262308 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:40:38.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:38.321+0000 7f6b0e77f640 1 --2- 172.21.15.26:0/721262308 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6b08109e00 0x7f6b081a4af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:38.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:38.321+0000 7f6b0e77f640 1 --2- 172.21.15.26:0/721262308 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b0810ea80 0x7f6b0819e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:38.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:38.321+0000 7f6b077fe640 1 --2- 172.21.15.26:0/721262308 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6b08109e00 0x7f6b081a4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:38.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:38.322+0000 7f6b0e77f640 1 -- 172.21.15.26:0/721262308 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f6b081185d0 con 0x7f6b0810ea80 2024-09-11T23:40:38.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:38.322+0000 7f6b0e77f640 1 -- 172.21.15.26:0/721262308 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f6b08118450 con 0x7f6b08109e00 2024-09-11T23:40:38.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:38.322+0000 7f6b0e77f640 1 -- 172.21.15.26:0/721262308 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f6b08118750 con 0x7f6b08109450 2024-09-11T23:40:38.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:38.323+0000 7f6b077fe640 1 -- 172.21.15.26:0/721262308 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6b08109450 msgr2=0x7f6b081a45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:38.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:38.323+0000 7f6b077fe640 1 --2- 172.21.15.26:0/721262308 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6b08109450 0x7f6b081a45b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:38.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:38.323+0000 7f6b077fe640 1 -- 172.21.15.26:0/721262308 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b0810ea80 msgr2=0x7f6b0819e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:38.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:38.323+0000 7f6b077fe640 1 --2- 172.21.15.26:0/721262308 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b0810ea80 0x7f6b0819e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:38.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:38.323+0000 7f6b077fe640 1 -- 172.21.15.26:0/721262308 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b0819eef0 con 0x7f6b08109e00 2024-09-11T23:40:38.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:38.324+0000 7f6b077fe640 1 --2- 172.21.15.26:0/721262308 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6b08109e00 0x7f6b081a4af0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f6afc009830 tx=0x7f6afc009cf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:40:38.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:38.325+0000 7f6b057fa640 1 -- 172.21.15.26:0/721262308 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6afc005020 con 0x7f6b08109e00 2024-09-11T23:40:38.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:38.325+0000 7f6b057fa640 1 -- 172.21.15.26:0/721262308 <== mon.2 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6afc0047a0 con 0x7f6b08109e00 2024-09-11T23:40:38.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:38.325+0000 7f6b0e77f640 1 -- 172.21.15.26:0/721262308 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b0819f1e0 con 0x7f6b08109e00 2024-09-11T23:40:38.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:38.325+0000 7f6b0e77f640 1 -- 172.21.15.26:0/721262308 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b081ab360 con 0x7f6b08109e00 2024-09-11T23:40:38.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:38.326+0000 7f6b057fa640 1 -- 172.21.15.26:0/721262308 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6afc007a20 con 0x7f6b08109e00 2024-09-11T23:40:38.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:38.329+0000 7f6b057fa640 1 -- 172.21.15.26:0/721262308 <== mon.2 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f6afc007ca0 con 0x7f6b08109e00 2024-09-11T23:40:38.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:38.329+0000 7f6b0e77f640 1 -- 172.21.15.26:0/721262308 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6af4005290 con 0x7f6b08109e00 2024-09-11T23:40:38.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:38.330+0000 7f6b057fa640 1 --2- 172.21.15.26:0/721262308 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f6af00a2f00 0x7f6af00a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:38.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:38.330+0000 7f6b057fa640 1 -- 172.21.15.26:0/721262308 <== mon.2 v2:172.21.15.76:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 8972+0+0 (secure 0 0 0) 0x7f6afc0d0860 con 0x7f6b08109e00 2024-09-11T23:40:38.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:38.331+0000 7f6b07fff640 1 --2- 172.21.15.26:0/721262308 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f6af00a2f00 0x7f6af00a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:38.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:38.332+0000 7f6b07fff640 1 --2- 172.21.15.26:0/721262308 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f6af00a2f00 0x7f6af00a53c0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f6b00004af0 tx=0x7f6b00000f40 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:40:38.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:38.338+0000 7f6b057fa640 1 -- 172.21.15.26:0/721262308 <== mon.2 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6afc09d260 con 0x7f6b08109e00 2024-09-11T23:40:38.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:38.677+0000 7f6b0e77f640 1 -- 172.21.15.26:0/721262308 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f6af4005580 con 0x7f6b08109e00 2024-09-11T23:40:38.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:38.873+0000 7f6b057fa640 1 -- 172.21.15.26:0/721262308 <== mon.2 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v119) ==== 181+0+0 (secure 0 0 0) 0x7f6afc09cb60 con 0x7f6b08109e00 2024-09-11T23:40:38.873 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_metadata' removed 2024-09-11T23:40:38.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:38.877+0000 7f6b0e77f640 1 -- 172.21.15.26:0/721262308 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f6af00a2f00 msgr2=0x7f6af00a53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:38.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:38.877+0000 7f6b0e77f640 1 --2- 172.21.15.26:0/721262308 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f6af00a2f00 0x7f6af00a53c0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f6b00004af0 tx=0x7f6b00000f40 comp rx=0 tx=0).stop 2024-09-11T23:40:38.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:38.877+0000 7f6b0e77f640 1 -- 172.21.15.26:0/721262308 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6b08109e00 msgr2=0x7f6b081a4af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:38.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:38.877+0000 7f6b0e77f640 1 --2- 172.21.15.26:0/721262308 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6b08109e00 0x7f6b081a4af0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f6afc009830 tx=0x7f6afc009cf0 comp rx=0 tx=0).stop 2024-09-11T23:40:38.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:38.877+0000 7f6b0e77f640 1 -- 172.21.15.26:0/721262308 shutdown_connections 2024-09-11T23:40:38.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:38.877+0000 7f6b0e77f640 1 --2- 172.21.15.26:0/721262308 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f6af00a2f00 0x7f6af00a53c0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:38.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:38.877+0000 7f6b0e77f640 1 --2- 172.21.15.26:0/721262308 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6b08109e00 0x7f6b081a4af0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:38.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:38.877+0000 7f6b0e77f640 1 --2- 172.21.15.26:0/721262308 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b0810ea80 0x7f6b0819e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:38.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:38.877+0000 7f6b0e77f640 1 --2- 172.21.15.26:0/721262308 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6b08109450 0x7f6b081a45b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:38.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:38.877+0000 7f6b0e77f640 1 -- 172.21.15.26:0/721262308 >> 172.21.15.26:0/721262308 conn(0x7f6b0806e2d0 msgr2=0x7f6b080726f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:40:38.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:38.882+0000 7f6b0e77f640 1 -- 172.21.15.26:0/721262308 shutdown_connections 2024-09-11T23:40:38.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:38.882+0000 7f6b0e77f640 1 -- 172.21.15.26:0/721262308 wait complete. 2024-09-11T23:40:39.215 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:40:38 smithi026 ceph-mon[31172]: from='client.? 172.21.15.26:0/721262308' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-11T23:40:39.215 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:40:38 smithi026 ceph-mon[31172]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-11T23:40:39.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:40:38 smithi136 ceph-mon[35121]: from='client.? 172.21.15.26:0/721262308' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-11T23:40:39.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:40:38 smithi136 ceph-mon[35121]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-11T23:40:39.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:40:38 smithi076 ceph-mon[35296]: from='client.? 172.21.15.26:0/721262308' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-11T23:40:39.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:40:38 smithi076 ceph-mon[35296]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-11T23:40:39.585 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 -- ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-11T23:40:40.070 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:40:40.203 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:40:39 smithi026 ceph-mon[31172]: pgmap v703: 97 pgs: 39 active+clean+snaptrim_wait, 21 active+clean+snaptrim, 37 active+clean; 2.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 204 B/s wr, 7 op/s 2024-09-11T23:40:40.204 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:40:39 smithi026 ceph-mon[31172]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-11T23:40:40.204 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:40:39 smithi026 ceph-mon[31172]: osdmap e119: 12 total, 12 up, 12 in 2024-09-11T23:40:40.238 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:40:39 smithi136 ceph-mon[35121]: pgmap v703: 97 pgs: 39 active+clean+snaptrim_wait, 21 active+clean+snaptrim, 37 active+clean; 2.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 204 B/s wr, 7 op/s 2024-09-11T23:40:40.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:40:39 smithi136 ceph-mon[35121]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-11T23:40:40.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:40:39 smithi136 ceph-mon[35121]: osdmap e119: 12 total, 12 up, 12 in 2024-09-11T23:40:40.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:40:39 smithi076 ceph-mon[35296]: pgmap v703: 97 pgs: 39 active+clean+snaptrim_wait, 21 active+clean+snaptrim, 37 active+clean; 2.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 204 B/s wr, 7 op/s 2024-09-11T23:40:40.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:40:39 smithi076 ceph-mon[35296]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-11T23:40:40.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:40:39 smithi076 ceph-mon[35296]: osdmap e119: 12 total, 12 up, 12 in 2024-09-11T23:40:40.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:40.725+0000 7f1b22255640 1 -- 172.21.15.26:0/2030564611 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1b1c110190 msgr2=0x7f1b1c06e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:40.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:40.725+0000 7f1b22255640 1 --2- 172.21.15.26:0/2030564611 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1b1c110190 0x7f1b1c06e4f0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f1b18009900 tx=0x7f1b1802a440 comp rx=0 tx=0).stop 2024-09-11T23:40:40.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:40.725+0000 7f1b22255640 1 -- 172.21.15.26:0/2030564611 shutdown_connections 2024-09-11T23:40:40.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:40.725+0000 7f1b22255640 1 --2- 172.21.15.26:0/2030564611 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1b1c06ea30 0x7f1b1c06ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:40.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:40.725+0000 7f1b22255640 1 --2- 172.21.15.26:0/2030564611 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1b1c11b140 0x7f1b1c11d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:40.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:40.725+0000 7f1b22255640 1 --2- 172.21.15.26:0/2030564611 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1b1c110190 0x7f1b1c06e4f0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:40.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:40.725+0000 7f1b22255640 1 -- 172.21.15.26:0/2030564611 >> 172.21.15.26:0/2030564611 conn(0x7f1b1c073180 msgr2=0x7f1b1c073590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:40:40.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:40.732+0000 7f1b22255640 1 -- 172.21.15.26:0/2030564611 shutdown_connections 2024-09-11T23:40:40.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:40.735+0000 7f1b22255640 1 -- 172.21.15.26:0/2030564611 wait complete. 2024-09-11T23:40:40.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:40.735+0000 7f1b22255640 1 Processor -- start 2024-09-11T23:40:40.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:40.737+0000 7f1b22255640 1 -- start start 2024-09-11T23:40:40.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:40.737+0000 7f1b22255640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1b1c06ea30 0x7f1b1c11a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:40.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:40.737+0000 7f1b22255640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1b1c110190 0x7f1b1c114aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:40.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:40.737+0000 7f1b22255640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1b1c11b140 0x7f1b1c114fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:40.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:40.737+0000 7f1b22255640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f1b1c111370 con 0x7f1b1c11b140 2024-09-11T23:40:40.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:40.737+0000 7f1b22255640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f1b1c1111f0 con 0x7f1b1c06ea30 2024-09-11T23:40:40.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:40.737+0000 7f1b22255640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f1b1c1114f0 con 0x7f1b1c110190 2024-09-11T23:40:40.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:40.738+0000 7f1b20a52640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1b1c110190 0x7f1b1c114aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:40.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:40.739+0000 7f1b20a52640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1b1c110190 0x7f1b1c114aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:34040/0 (socket says 172.21.15.26:34040) 2024-09-11T23:40:40.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:40.739+0000 7f1b20a52640 1 -- 172.21.15.26:0/1237882790 learned_addr learned my addr 172.21.15.26:0/1237882790 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-11T23:40:40.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:40.741+0000 7f1b21253640 1 --2- 172.21.15.26:0/1237882790 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1b1c06ea30 0x7f1b1c11a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:40.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:40.742+0000 7f1b21a54640 1 --2- 172.21.15.26:0/1237882790 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1b1c11b140 0x7f1b1c114fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:40.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:40.743+0000 7f1b21a54640 1 -- 172.21.15.26:0/1237882790 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1b1c110190 msgr2=0x7f1b1c114aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:40.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:40.743+0000 7f1b21a54640 1 --2- 172.21.15.26:0/1237882790 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1b1c110190 0x7f1b1c114aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:40.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:40.743+0000 7f1b21a54640 1 -- 172.21.15.26:0/1237882790 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1b1c06ea30 msgr2=0x7f1b1c11a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:40.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:40.743+0000 7f1b21a54640 1 --2- 172.21.15.26:0/1237882790 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1b1c06ea30 0x7f1b1c11a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:40.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:40.743+0000 7f1b21a54640 1 -- 172.21.15.26:0/1237882790 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b1c1158a0 con 0x7f1b1c11b140 2024-09-11T23:40:40.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:40.744+0000 7f1b21a54640 1 --2- 172.21.15.26:0/1237882790 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1b1c11b140 0x7f1b1c114fe0 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f1b1400bdd0 tx=0x7f1b1400bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:40:40.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:40.744+0000 7f1b127fc640 1 -- 172.21.15.26:0/1237882790 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b14019070 con 0x7f1b1c11b140 2024-09-11T23:40:40.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:40.745+0000 7f1b127fc640 1 -- 172.21.15.26:0/1237882790 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b1400cd70 con 0x7f1b1c11b140 2024-09-11T23:40:40.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:40.745+0000 7f1b127fc640 1 -- 172.21.15.26:0/1237882790 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b14012880 con 0x7f1b1c11b140 2024-09-11T23:40:40.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:40.745+0000 7f1b22255640 1 -- 172.21.15.26:0/1237882790 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b1c1bc0a0 con 0x7f1b1c11b140 2024-09-11T23:40:40.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:40.746+0000 7f1b22255640 1 -- 172.21.15.26:0/1237882790 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b1c1bc560 con 0x7f1b1c11b140 2024-09-11T23:40:40.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:40.747+0000 7f1b22255640 1 -- 172.21.15.26:0/1237882790 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b1c1096f0 con 0x7f1b1c11b140 2024-09-11T23:40:40.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:40.749+0000 7f1b127fc640 1 -- 172.21.15.26:0/1237882790 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f1b14012a20 con 0x7f1b1c11b140 2024-09-11T23:40:40.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:40.750+0000 7f1b127fc640 1 --2- 172.21.15.26:0/1237882790 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f1b080a2f00 0x7f1b080a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:40.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:40.751+0000 7f1b127fc640 1 -- 172.21.15.26:0/1237882790 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 8418+0+0 (secure 0 0 0) 0x7f1b140cf310 con 0x7f1b1c11b140 2024-09-11T23:40:40.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:40.758+0000 7f1b21253640 1 --2- 172.21.15.26:0/1237882790 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f1b080a2f00 0x7f1b080a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:40.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:40.758+0000 7f1b127fc640 1 -- 172.21.15.26:0/1237882790 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1b1409ba70 con 0x7f1b1c11b140 2024-09-11T23:40:40.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:40.771+0000 7f1b21253640 1 --2- 172.21.15.26:0/1237882790 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f1b080a2f00 0x7f1b080a53c0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f1b1802aa30 tx=0x7f1b1802a940 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:40:41.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:41.103+0000 7f1b22255640 1 -- 172.21.15.26:0/1237882790 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f1b1c1bc880 con 0x7f1b1c11b140 2024-09-11T23:40:41.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:41.940+0000 7f1b127fc640 1 -- 172.21.15.26:0/1237882790 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v120) ==== 169+0+0 (secure 0 0 0) 0x7f1b1409fd70 con 0x7f1b1c11b140 2024-09-11T23:40:41.940 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_data' removed 2024-09-11T23:40:41.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:41.945+0000 7f1b07fff640 1 -- 172.21.15.26:0/1237882790 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f1b080a2f00 msgr2=0x7f1b080a53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:41.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:41.945+0000 7f1b07fff640 1 --2- 172.21.15.26:0/1237882790 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f1b080a2f00 0x7f1b080a53c0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f1b1802aa30 tx=0x7f1b1802a940 comp rx=0 tx=0).stop 2024-09-11T23:40:41.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:41.945+0000 7f1b07fff640 1 -- 172.21.15.26:0/1237882790 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1b1c11b140 msgr2=0x7f1b1c114fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:41.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:41.945+0000 7f1b07fff640 1 --2- 172.21.15.26:0/1237882790 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1b1c11b140 0x7f1b1c114fe0 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f1b1400bdd0 tx=0x7f1b1400bec0 comp rx=0 tx=0).stop 2024-09-11T23:40:41.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:41.948+0000 7f1b07fff640 1 -- 172.21.15.26:0/1237882790 shutdown_connections 2024-09-11T23:40:41.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:41.948+0000 7f1b07fff640 1 --2- 172.21.15.26:0/1237882790 >> [v2:172.21.15.26:6800/1989555790,v1:172.21.15.26:6801/1989555790] conn(0x7f1b080a2f00 0x7f1b080a53c0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:41.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:41.948+0000 7f1b07fff640 1 --2- 172.21.15.26:0/1237882790 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1b1c11b140 0x7f1b1c114fe0 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:41.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:41.948+0000 7f1b07fff640 1 --2- 172.21.15.26:0/1237882790 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1b1c110190 0x7f1b1c114aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:41.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:41.948+0000 7f1b07fff640 1 --2- 172.21.15.26:0/1237882790 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1b1c06ea30 0x7f1b1c11a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:41.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:41.948+0000 7f1b07fff640 1 -- 172.21.15.26:0/1237882790 >> 172.21.15.26:0/1237882790 conn(0x7f1b1c073180 msgr2=0x7f1b1c11b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:40:41.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:41.948+0000 7f1b07fff640 1 -- 172.21.15.26:0/1237882790 shutdown_connections 2024-09-11T23:40:41.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-11T23:40:41.949+0000 7f1b07fff640 1 -- 172.21.15.26:0/1237882790 wait complete. 2024-09-11T23:40:42.238 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:40:41 smithi136 ceph-mon[35121]: pgmap v705: 65 pgs: 37 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 9 active+clean; 707 MiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:40:42.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:40:41 smithi136 ceph-mon[35121]: from='client.? 172.21.15.26:0/1237882790' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-11T23:40:42.253 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:40:41 smithi026 ceph-mon[31172]: pgmap v705: 65 pgs: 37 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 9 active+clean; 707 MiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:40:42.253 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:40:41 smithi026 ceph-mon[31172]: from='client.? 172.21.15.26:0/1237882790' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-11T23:40:42.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:40:41 smithi076 ceph-mon[35296]: pgmap v705: 65 pgs: 37 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 9 active+clean; 707 MiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:40:42.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:40:41 smithi076 ceph-mon[35296]: from='client.? 172.21.15.26:0/1237882790' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-11T23:40:42.741 DEBUG:teuthology.orchestra.run.smithi026:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-11T23:40:42.783 DEBUG:teuthology.orchestra.run.smithi076:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-11T23:40:42.815 DEBUG:teuthology.orchestra.run.smithi136:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-11T23:40:42.872 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-09-11T23:40:42.872 DEBUG:teuthology.orchestra.run.smithi026:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-11T23:40:42.895 DEBUG:teuthology.orchestra.run.smithi076:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-11T23:40:42.911 DEBUG:teuthology.orchestra.run.smithi136:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-11T23:40:42.940 INFO:tasks.cephadm:Stopping all daemons... 2024-09-11T23:40:42.941 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-09-11T23:40:42.941 DEBUG:teuthology.orchestra.run.smithi026:> sudo systemctl stop ceph-af7e6fc8-7092-11ef-bcea-c7b262605968@mon.a 2024-09-11T23:40:42.977 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:40:42 smithi026 ceph-mon[31172]: from='client.? 172.21.15.26:0/1237882790' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-11T23:40:42.977 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:40:42 smithi026 ceph-mon[31172]: osdmap e120: 12 total, 12 up, 12 in 2024-09-11T23:40:43.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:40:42 smithi136 ceph-mon[35121]: from='client.? 172.21.15.26:0/1237882790' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-11T23:40:43.239 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:40:42 smithi136 ceph-mon[35121]: osdmap e120: 12 total, 12 up, 12 in 2024-09-11T23:40:43.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:40:43 smithi026 systemd[1]: Stopping Ceph mon.a for af7e6fc8-7092-11ef-bcea-c7b262605968... 2024-09-11T23:40:43.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:40:42 smithi076 ceph-mon[35296]: from='client.? 172.21.15.26:0/1237882790' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-11T23:40:43.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:40:42 smithi076 ceph-mon[35296]: osdmap e120: 12 total, 12 up, 12 in 2024-09-11T23:40:44.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:40:43 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mon-a[31168]: 2024-09-11T23:40:43.948+0000 7f5d9653c640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T23:40:44.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:40:43 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mon-a[31168]: 2024-09-11T23:40:43.948+0000 7f5d9653c640 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-09-11T23:40:44.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:40:44 smithi026 podman[114246]: 2024-09-11 23:40:44.153796336 +0000 UTC m=+0.908455717 container died fb7a9fd245e408acfe118ef86b660356ee76bf85b8846bb3e771b8ed7860a140 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mon-a, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2) 2024-09-11T23:40:44.750 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:40:44 smithi026 podman[114246]: 2024-09-11 23:40:44.623367009 +0000 UTC m=+1.378026383 container cleanup fb7a9fd245e408acfe118ef86b660356ee76bf85b8846bb3e771b8ed7860a140 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mon-a, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-09-11T23:40:44.750 INFO:journalctl@ceph.mon.a.smithi026.stdout:Sep 11 23:40:44 smithi026 bash[114246]: ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mon-a 2024-09-11T23:40:45.033 DEBUG:teuthology.orchestra.run.smithi026:> sudo pkill -f 'journalctl -f -n 0 -u ceph-af7e6fc8-7092-11ef-bcea-c7b262605968@mon.a.service' 2024-09-11T23:40:45.122 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T23:40:45.123 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-09-11T23:40:45.123 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-09-11T23:40:45.123 DEBUG:teuthology.orchestra.run.smithi076:> sudo systemctl stop ceph-af7e6fc8-7092-11ef-bcea-c7b262605968@mon.b 2024-09-11T23:40:45.491 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:40:45 smithi076 systemd[1]: Stopping Ceph mon.b for af7e6fc8-7092-11ef-bcea-c7b262605968... 2024-09-11T23:40:45.802 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:40:45 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mon-b[35292]: 2024-09-11T23:40:45.491+0000 7f6e95a1d640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T23:40:45.802 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:40:45 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mon-b[35292]: 2024-09-11T23:40:45.491+0000 7f6e95a1d640 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-09-11T23:40:45.802 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:40:45 smithi076 podman[73017]: 2024-09-11 23:40:45.697373016 +0000 UTC m=+0.314939015 container died 6320275d55dc8c2c86e9ff8db932ac2f1d7db268e8a28a2f59323b9ea78b54c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mon-b, GIT_CLEAN=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3) 2024-09-11T23:40:46.302 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:40:46 smithi076 podman[73017]: 2024-09-11 23:40:46.108882321 +0000 UTC m=+0.726448320 container cleanup 6320275d55dc8c2c86e9ff8db932ac2f1d7db268e8a28a2f59323b9ea78b54c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mon-b, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-09-11T23:40:46.303 INFO:journalctl@ceph.mon.b.smithi076.stdout:Sep 11 23:40:46 smithi076 bash[73017]: ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mon-b 2024-09-11T23:40:46.488 DEBUG:teuthology.orchestra.run.smithi076:> sudo pkill -f 'journalctl -f -n 0 -u ceph-af7e6fc8-7092-11ef-bcea-c7b262605968@mon.b.service' 2024-09-11T23:40:46.535 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T23:40:46.535 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-09-11T23:40:46.536 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-09-11T23:40:46.536 DEBUG:teuthology.orchestra.run.smithi136:> sudo systemctl stop ceph-af7e6fc8-7092-11ef-bcea-c7b262605968@mon.c 2024-09-11T23:40:46.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:40:46 smithi136 systemd[1]: Stopping Ceph mon.c for af7e6fc8-7092-11ef-bcea-c7b262605968... 2024-09-11T23:40:47.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:40:47 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mon-c[35117]: 2024-09-11T23:40:47.198+0000 7f4f87300640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T23:40:47.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:40:47 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mon-c[35117]: 2024-09-11T23:40:47.198+0000 7f4f87300640 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-09-11T23:40:47.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:40:47 smithi136 podman[73323]: 2024-09-11 23:40:47.431837603 +0000 UTC m=+0.351559507 container died a454789c9141578fc927a589f0159c277bd6b2fcf3e31354e4199cf1f2d31883 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mon-c, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True) 2024-09-11T23:40:47.988 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:40:47 smithi136 podman[73323]: 2024-09-11 23:40:47.789513821 +0000 UTC m=+0.709235712 container cleanup a454789c9141578fc927a589f0159c277bd6b2fcf3e31354e4199cf1f2d31883 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mon-c, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T23:40:47.989 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 11 23:40:47 smithi136 bash[73323]: ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mon-c 2024-09-11T23:40:48.225 DEBUG:teuthology.orchestra.run.smithi136:> sudo pkill -f 'journalctl -f -n 0 -u ceph-af7e6fc8-7092-11ef-bcea-c7b262605968@mon.c.service' 2024-09-11T23:40:48.277 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T23:40:48.277 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-09-11T23:40:48.277 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-09-11T23:40:48.277 DEBUG:teuthology.orchestra.run.smithi026:> sudo systemctl stop ceph-af7e6fc8-7092-11ef-bcea-c7b262605968@mgr.x 2024-09-11T23:40:48.750 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:40:48 smithi026 systemd[1]: Stopping Ceph mgr.x for af7e6fc8-7092-11ef-bcea-c7b262605968... 2024-09-11T23:40:49.137 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:40:48 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x[31398]: 2024-09-11T23:40:48.837+0000 7f7d31bbc640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T23:40:49.138 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:40:48 smithi026 podman[114368]: 2024-09-11 23:40:48.878334404 +0000 UTC m=+0.409175574 container died 865f5a94615ba4a6d0c97dcd45d8e7009aebc46cdd202cab1917878c343caac6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, io.buildah.version=1.37.2) 2024-09-11T23:40:49.500 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:40:49 smithi026 podman[114368]: 2024-09-11 23:40:49.24560786 +0000 UTC m=+0.776449029 container cleanup 865f5a94615ba4a6d0c97dcd45d8e7009aebc46cdd202cab1917878c343caac6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T23:40:49.500 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Sep 11 23:40:49 smithi026 bash[114368]: ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-x 2024-09-11T23:40:49.682 DEBUG:teuthology.orchestra.run.smithi026:> sudo pkill -f 'journalctl -f -n 0 -u ceph-af7e6fc8-7092-11ef-bcea-c7b262605968@mgr.x.service' 2024-09-11T23:40:49.720 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T23:40:49.720 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-09-11T23:40:49.720 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-09-11T23:40:49.720 DEBUG:teuthology.orchestra.run.smithi076:> sudo systemctl stop ceph-af7e6fc8-7092-11ef-bcea-c7b262605968@mgr.y 2024-09-11T23:40:50.052 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Sep 11 23:40:49 smithi076 systemd[1]: Stopping Ceph mgr.y for af7e6fc8-7092-11ef-bcea-c7b262605968... 2024-09-11T23:40:50.552 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Sep 11 23:40:50 smithi076 podman[73138]: 2024-09-11 23:40:50.261231156 +0000 UTC m=+0.341548686 container died 39fb8e0d330c81438cc8612ce0eb8d7c480ef27e87c9e38ba0338b0fc9d398ed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-y, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2) 2024-09-11T23:40:50.814 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Sep 11 23:40:50 smithi076 podman[73138]: 2024-09-11 23:40:50.813152574 +0000 UTC m=+0.893470089 container cleanup 39fb8e0d330c81438cc8612ce0eb8d7c480ef27e87c9e38ba0338b0fc9d398ed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-y, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_CLEAN=True) 2024-09-11T23:40:50.814 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Sep 11 23:40:50 smithi076 bash[73138]: ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-y 2024-09-11T23:40:51.261 DEBUG:teuthology.orchestra.run.smithi076:> sudo pkill -f 'journalctl -f -n 0 -u ceph-af7e6fc8-7092-11ef-bcea-c7b262605968@mgr.y.service' 2024-09-11T23:40:51.296 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T23:40:51.296 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-09-11T23:40:51.297 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-09-11T23:40:51.297 DEBUG:teuthology.orchestra.run.smithi136:> sudo systemctl stop ceph-af7e6fc8-7092-11ef-bcea-c7b262605968@mgr.z 2024-09-11T23:40:51.682 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Sep 11 23:40:51 smithi136 systemd[1]: Stopping Ceph mgr.z for af7e6fc8-7092-11ef-bcea-c7b262605968... 2024-09-11T23:40:51.971 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Sep 11 23:40:51 smithi136 podman[73446]: 2024-09-11 23:40:51.683270519 +0000 UTC m=+0.194791106 container died b42acef488cab28427ccf47697eaefaee6af5d6bb282cec075a039cad38038c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-z, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.schema-version=1.0) 2024-09-11T23:40:52.238 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Sep 11 23:40:52 smithi136 podman[73446]: 2024-09-11 23:40:52.071147432 +0000 UTC m=+0.582668019 container cleanup b42acef488cab28427ccf47697eaefaee6af5d6bb282cec075a039cad38038c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-z, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-09-11T23:40:52.239 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Sep 11 23:40:52 smithi136 bash[73446]: ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mgr-z 2024-09-11T23:40:52.502 DEBUG:teuthology.orchestra.run.smithi136:> sudo pkill -f 'journalctl -f -n 0 -u ceph-af7e6fc8-7092-11ef-bcea-c7b262605968@mgr.z.service' 2024-09-11T23:40:52.537 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T23:40:52.538 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-09-11T23:40:52.538 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-09-11T23:40:52.538 DEBUG:teuthology.orchestra.run.smithi026:> sudo systemctl stop ceph-af7e6fc8-7092-11ef-bcea-c7b262605968@osd.0 2024-09-11T23:40:52.835 INFO:journalctl@ceph.osd.0.smithi026.stdout:Sep 11 23:40:52 smithi026 systemd[1]: Stopping Ceph osd.0 for af7e6fc8-7092-11ef-bcea-c7b262605968... 2024-09-11T23:40:53.250 INFO:journalctl@ceph.osd.0.smithi026.stdout:Sep 11 23:40:52 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-0[44903]: 2024-09-11T23:40:52.834+0000 7f201a9dc640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T23:40:53.250 INFO:journalctl@ceph.osd.0.smithi026.stdout:Sep 11 23:40:52 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-0[44903]: 2024-09-11T23:40:52.834+0000 7f201a9dc640 -1 osd.0 120 *** Got signal Terminated *** 2024-09-11T23:40:53.250 INFO:journalctl@ceph.osd.0.smithi026.stdout:Sep 11 23:40:52 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-0[44903]: 2024-09-11T23:40:52.834+0000 7f201a9dc640 -1 osd.0 120 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T23:40:58.750 INFO:journalctl@ceph.osd.0.smithi026.stdout:Sep 11 23:40:58 smithi026 podman[114489]: 2024-09-11 23:40:58.489515396 +0000 UTC m=+5.756052491 container died 4a24d5ec0f7ed417ec4dc929c2b98a112a60964194f1cafc6addaca379f3232b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0) 2024-09-11T23:40:59.003 INFO:journalctl@ceph.osd.0.smithi026.stdout:Sep 11 23:40:58 smithi026 podman[114489]: 2024-09-11 23:40:58.927045197 +0000 UTC m=+6.193582334 container cleanup 4a24d5ec0f7ed417ec4dc929c2b98a112a60964194f1cafc6addaca379f3232b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-0, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, ceph=True, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-09-11T23:40:59.003 INFO:journalctl@ceph.osd.0.smithi026.stdout:Sep 11 23:40:58 smithi026 bash[114489]: ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-0 2024-09-11T23:40:59.500 INFO:journalctl@ceph.osd.0.smithi026.stdout:Sep 11 23:40:59 smithi026 podman[114506]: 2024-09-11 23:40:59.145274241 +0000 UTC m=+0.650070780 container remove 4a24d5ec0f7ed417ec4dc929c2b98a112a60964194f1cafc6addaca379f3232b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-0, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-09-11T23:40:59.873 INFO:journalctl@ceph.osd.0.smithi026.stdout:Sep 11 23:40:59 smithi026 podman[114594]: 2024-09-11 23:40:59.46257591 +0000 UTC m=+0.016945074 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T23:40:59.873 INFO:journalctl@ceph.osd.0.smithi026.stdout:Sep 11 23:40:59 smithi026 podman[114594]: 2024-09-11 23:40:59.571183977 +0000 UTC m=+0.125553169 container create a3d142a9857d69a4fbaa282ec2053db3a7c94d218fabfbb68d38cd3a1ca52e8d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-0-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-09-11T23:40:59.873 INFO:journalctl@ceph.osd.0.smithi026.stdout:Sep 11 23:40:59 smithi026 podman[114594]: 2024-09-11 23:40:59.790554463 +0000 UTC m=+0.344923623 container init a3d142a9857d69a4fbaa282ec2053db3a7c94d218fabfbb68d38cd3a1ca52e8d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-0-deactivate, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240731) 2024-09-11T23:40:59.873 INFO:journalctl@ceph.osd.0.smithi026.stdout:Sep 11 23:40:59 smithi026 podman[114594]: 2024-09-11 23:40:59.793917466 +0000 UTC m=+0.348286625 container start a3d142a9857d69a4fbaa282ec2053db3a7c94d218fabfbb68d38cd3a1ca52e8d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-0-deactivate, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS) 2024-09-11T23:40:59.873 INFO:journalctl@ceph.osd.0.smithi026.stdout:Sep 11 23:40:59 smithi026 podman[114594]: 2024-09-11 23:40:59.873828117 +0000 UTC m=+0.428197298 container attach a3d142a9857d69a4fbaa282ec2053db3a7c94d218fabfbb68d38cd3a1ca52e8d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-0-deactivate, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-09-11T23:41:00.250 INFO:journalctl@ceph.osd.0.smithi026.stdout:Sep 11 23:40:59 smithi026 podman[114594]: 2024-09-11 23:40:59.992025993 +0000 UTC m=+0.546395144 container died a3d142a9857d69a4fbaa282ec2053db3a7c94d218fabfbb68d38cd3a1ca52e8d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-0-deactivate, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-09-11T23:41:00.628 DEBUG:teuthology.orchestra.run.smithi026:> sudo pkill -f 'journalctl -f -n 0 -u ceph-af7e6fc8-7092-11ef-bcea-c7b262605968@osd.0.service' 2024-09-11T23:41:00.669 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T23:41:00.670 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-09-11T23:41:00.670 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-09-11T23:41:00.670 DEBUG:teuthology.orchestra.run.smithi076:> sudo systemctl stop ceph-af7e6fc8-7092-11ef-bcea-c7b262605968@osd.1 2024-09-11T23:41:01.010 INFO:journalctl@ceph.osd.1.smithi076.stdout:Sep 11 23:41:00 smithi076 systemd[1]: Stopping Ceph osd.1 for af7e6fc8-7092-11ef-bcea-c7b262605968... 2024-09-11T23:41:01.302 INFO:journalctl@ceph.osd.1.smithi076.stdout:Sep 11 23:41:01 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-1[39779]: 2024-09-11T23:41:01.010+0000 7f1083c80640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T23:41:01.302 INFO:journalctl@ceph.osd.1.smithi076.stdout:Sep 11 23:41:01 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-1[39779]: 2024-09-11T23:41:01.010+0000 7f1083c80640 -1 osd.1 120 *** Got signal Terminated *** 2024-09-11T23:41:01.303 INFO:journalctl@ceph.osd.1.smithi076.stdout:Sep 11 23:41:01 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-1[39779]: 2024-09-11T23:41:01.010+0000 7f1083c80640 -1 osd.1 120 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T23:41:06.709 INFO:journalctl@ceph.osd.1.smithi076.stdout:Sep 11 23:41:06 smithi076 podman[73261]: 2024-09-11 23:41:06.440278173 +0000 UTC m=+5.569623592 container died 4a73508cf27be892590e84277b7411ede5f7feab0f88effb02326cbdf634e83e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-1, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T23:41:07.052 INFO:journalctl@ceph.osd.1.smithi076.stdout:Sep 11 23:41:06 smithi076 podman[73261]: 2024-09-11 23:41:06.800782415 +0000 UTC m=+5.930127825 container cleanup 4a73508cf27be892590e84277b7411ede5f7feab0f88effb02326cbdf634e83e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-1, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True) 2024-09-11T23:41:07.052 INFO:journalctl@ceph.osd.1.smithi076.stdout:Sep 11 23:41:06 smithi076 bash[73261]: ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-1 2024-09-11T23:41:07.418 INFO:journalctl@ceph.osd.1.smithi076.stdout:Sep 11 23:41:07 smithi076 podman[73277]: 2024-09-11 23:41:07.118685181 +0000 UTC m=+0.676293634 container remove 4a73508cf27be892590e84277b7411ede5f7feab0f88effb02326cbdf634e83e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-1, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux ) 2024-09-11T23:41:07.762 INFO:journalctl@ceph.osd.1.smithi076.stdout:Sep 11 23:41:07 smithi076 podman[73366]: 2024-09-11 23:41:07.419241067 +0000 UTC m=+0.104549245 container create 0f3a3a8f264b203d8ea230a3f3af721210d5bdc02577336f70ca82f410084fed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-1-deactivate, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T23:41:07.762 INFO:journalctl@ceph.osd.1.smithi076.stdout:Sep 11 23:41:07 smithi076 podman[73366]: 2024-09-11 23:41:07.330395922 +0000 UTC m=+0.015704107 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T23:41:07.762 INFO:journalctl@ceph.osd.1.smithi076.stdout:Sep 11 23:41:07 smithi076 podman[73366]: 2024-09-11 23:41:07.64647124 +0000 UTC m=+0.331779418 container init 0f3a3a8f264b203d8ea230a3f3af721210d5bdc02577336f70ca82f410084fed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-1-deactivate, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True) 2024-09-11T23:41:07.762 INFO:journalctl@ceph.osd.1.smithi076.stdout:Sep 11 23:41:07 smithi076 podman[73366]: 2024-09-11 23:41:07.649696398 +0000 UTC m=+0.335004582 container start 0f3a3a8f264b203d8ea230a3f3af721210d5bdc02577336f70ca82f410084fed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-1-deactivate, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-09-11T23:41:07.762 INFO:journalctl@ceph.osd.1.smithi076.stdout:Sep 11 23:41:07 smithi076 podman[73366]: 2024-09-11 23:41:07.763125026 +0000 UTC m=+0.448433213 container attach 0f3a3a8f264b203d8ea230a3f3af721210d5bdc02577336f70ca82f410084fed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3) 2024-09-11T23:41:08.302 INFO:journalctl@ceph.osd.1.smithi076.stdout:Sep 11 23:41:08 smithi076 podman[73366]: 2024-09-11 23:41:08.031057084 +0000 UTC m=+0.716365263 container died 0f3a3a8f264b203d8ea230a3f3af721210d5bdc02577336f70ca82f410084fed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-1-deactivate, io.buildah.version=1.37.2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731) 2024-09-11T23:41:08.707 DEBUG:teuthology.orchestra.run.smithi076:> sudo pkill -f 'journalctl -f -n 0 -u ceph-af7e6fc8-7092-11ef-bcea-c7b262605968@osd.1.service' 2024-09-11T23:41:08.746 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T23:41:08.746 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-09-11T23:41:08.747 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-09-11T23:41:08.747 DEBUG:teuthology.orchestra.run.smithi136:> sudo systemctl stop ceph-af7e6fc8-7092-11ef-bcea-c7b262605968@osd.2 2024-09-11T23:41:09.050 INFO:journalctl@ceph.osd.2.smithi136.stdout:Sep 11 23:41:08 smithi136 systemd[1]: Stopping Ceph osd.2 for af7e6fc8-7092-11ef-bcea-c7b262605968... 2024-09-11T23:41:09.488 INFO:journalctl@ceph.osd.2.smithi136.stdout:Sep 11 23:41:09 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-2[40102]: 2024-09-11T23:41:09.050+0000 7f259167d640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T23:41:09.489 INFO:journalctl@ceph.osd.2.smithi136.stdout:Sep 11 23:41:09 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-2[40102]: 2024-09-11T23:41:09.050+0000 7f259167d640 -1 osd.2 120 *** Got signal Terminated *** 2024-09-11T23:41:09.489 INFO:journalctl@ceph.osd.2.smithi136.stdout:Sep 11 23:41:09 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-2[40102]: 2024-09-11T23:41:09.050+0000 7f259167d640 -1 osd.2 120 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T23:41:14.893 INFO:journalctl@ceph.osd.2.smithi136.stdout:Sep 11 23:41:14 smithi136 podman[73582]: 2024-09-11 23:41:14.596245335 +0000 UTC m=+5.650237498 container died 4e53b54442b808896de1710e1dc71ab71e79e57b4b04ae5c823e99e54f16c585 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, GIT_CLEAN=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux ) 2024-09-11T23:41:15.238 INFO:journalctl@ceph.osd.2.smithi136.stdout:Sep 11 23:41:14 smithi136 podman[73582]: 2024-09-11 23:41:14.99127482 +0000 UTC m=+6.045266983 container cleanup 4e53b54442b808896de1710e1dc71ab71e79e57b4b04ae5c823e99e54f16c585 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-2, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-09-11T23:41:15.239 INFO:journalctl@ceph.osd.2.smithi136.stdout:Sep 11 23:41:14 smithi136 bash[73582]: ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-2 2024-09-11T23:41:15.738 INFO:journalctl@ceph.osd.2.smithi136.stdout:Sep 11 23:41:15 smithi136 podman[73598]: 2024-09-11 23:41:15.352416398 +0000 UTC m=+0.741088543 container remove 4e53b54442b808896de1710e1dc71ab71e79e57b4b04ae5c823e99e54f16c585 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-09-11T23:41:16.004 INFO:journalctl@ceph.osd.2.smithi136.stdout:Sep 11 23:41:15 smithi136 podman[73683]: 2024-09-11 23:41:15.646723987 +0000 UTC m=+0.015937106 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T23:41:16.004 INFO:journalctl@ceph.osd.2.smithi136.stdout:Sep 11 23:41:15 smithi136 podman[73683]: 2024-09-11 23:41:15.752435095 +0000 UTC m=+0.121648198 container create 44c8059d492170a0709901478d06f8d20d30cff5bf511f341473442f0125e7a0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-2-deactivate, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS) 2024-09-11T23:41:16.303 INFO:journalctl@ceph.osd.2.smithi136.stdout:Sep 11 23:41:16 smithi136 podman[73683]: 2024-09-11 23:41:16.005429336 +0000 UTC m=+0.374642451 container init 44c8059d492170a0709901478d06f8d20d30cff5bf511f341473442f0125e7a0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-2-deactivate, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T23:41:16.303 INFO:journalctl@ceph.osd.2.smithi136.stdout:Sep 11 23:41:16 smithi136 podman[73683]: 2024-09-11 23:41:16.009474278 +0000 UTC m=+0.378687385 container start 44c8059d492170a0709901478d06f8d20d30cff5bf511f341473442f0125e7a0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-2-deactivate, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-09-11T23:41:16.303 INFO:journalctl@ceph.osd.2.smithi136.stdout:Sep 11 23:41:16 smithi136 podman[73683]: 2024-09-11 23:41:16.080118583 +0000 UTC m=+0.449331696 container attach 44c8059d492170a0709901478d06f8d20d30cff5bf511f341473442f0125e7a0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-2-deactivate, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, ceph=True) 2024-09-11T23:41:16.640 INFO:journalctl@ceph.osd.2.smithi136.stdout:Sep 11 23:41:16 smithi136 podman[73683]: 2024-09-11 23:41:16.304961023 +0000 UTC m=+0.674174127 container died 44c8059d492170a0709901478d06f8d20d30cff5bf511f341473442f0125e7a0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-2-deactivate, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, ceph=True) 2024-09-11T23:41:16.943 DEBUG:teuthology.orchestra.run.smithi136:> sudo pkill -f 'journalctl -f -n 0 -u ceph-af7e6fc8-7092-11ef-bcea-c7b262605968@osd.2.service' 2024-09-11T23:41:16.983 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T23:41:16.984 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-09-11T23:41:16.984 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-09-11T23:41:16.984 DEBUG:teuthology.orchestra.run.smithi026:> sudo systemctl stop ceph-af7e6fc8-7092-11ef-bcea-c7b262605968@osd.3 2024-09-11T23:41:17.305 INFO:journalctl@ceph.osd.3.smithi026.stdout:Sep 11 23:41:17 smithi026 systemd[1]: Stopping Ceph osd.3 for af7e6fc8-7092-11ef-bcea-c7b262605968... 2024-09-11T23:41:17.750 INFO:journalctl@ceph.osd.3.smithi026.stdout:Sep 11 23:41:17 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-3[51852]: 2024-09-11T23:41:17.305+0000 7f0fe7536640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T23:41:17.750 INFO:journalctl@ceph.osd.3.smithi026.stdout:Sep 11 23:41:17 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-3[51852]: 2024-09-11T23:41:17.305+0000 7f0fe7536640 -1 osd.3 120 *** Got signal Terminated *** 2024-09-11T23:41:17.750 INFO:journalctl@ceph.osd.3.smithi026.stdout:Sep 11 23:41:17 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-3[51852]: 2024-09-11T23:41:17.305+0000 7f0fe7536640 -1 osd.3 120 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T23:41:19.238 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 11 23:41:18 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-8[53569]: 2024-09-11T23:41:18.749+0000 7f99e4062640 -1 osd.8 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:53.978855+0000 front 2024-09-11T23:40:53.978923+0000 (oldest deadline 2024-09-11T23:41:18.679037+0000) 2024-09-11T23:41:20.000 INFO:journalctl@ceph.osd.6.smithi026.stdout:Sep 11 23:41:19 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-6[58789]: 2024-09-11T23:41:19.513+0000 7f8321638640 -1 osd.6 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:57.112923+0000 front 2024-09-11T23:40:57.113173+0000 (oldest deadline 2024-09-11T23:41:18.813188+0000) 2024-09-11T23:41:20.238 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 11 23:41:19 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-8[53569]: 2024-09-11T23:41:19.799+0000 7f99e4062640 -1 osd.8 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:53.978855+0000 front 2024-09-11T23:40:53.978923+0000 (oldest deadline 2024-09-11T23:41:18.679037+0000) 2024-09-11T23:41:20.500 INFO:journalctl@ceph.osd.3.smithi026.stdout:Sep 11 23:41:20 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-3[51852]: 2024-09-11T23:41:20.172+0000 7f0fdfb33640 -1 osd.3 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:55.267620+0000 front 2024-09-11T23:40:55.267669+0000 (oldest deadline 2024-09-11T23:41:19.367691+0000) 2024-09-11T23:41:20.738 INFO:journalctl@ceph.osd.5.smithi136.stdout:Sep 11 23:41:20 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-5[46773]: 2024-09-11T23:41:20.452+0000 7f5a20e77640 -1 osd.5 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:54.364889+0000 front 2024-09-11T23:40:54.364905+0000 (oldest deadline 2024-09-11T23:41:19.664838+0000) 2024-09-11T23:41:21.000 INFO:journalctl@ceph.osd.6.smithi026.stdout:Sep 11 23:41:20 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-6[58789]: 2024-09-11T23:41:20.552+0000 7f8321638640 -1 osd.6 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:57.112923+0000 front 2024-09-11T23:40:57.113173+0000 (oldest deadline 2024-09-11T23:41:18.813188+0000) 2024-09-11T23:41:21.238 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 11 23:41:20 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-8[53569]: 2024-09-11T23:41:20.772+0000 7f99e4062640 -1 osd.8 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:53.978855+0000 front 2024-09-11T23:40:53.978923+0000 (oldest deadline 2024-09-11T23:41:18.679037+0000) 2024-09-11T23:41:21.500 INFO:journalctl@ceph.osd.3.smithi026.stdout:Sep 11 23:41:21 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-3[51852]: 2024-09-11T23:41:21.192+0000 7f0fdfb33640 -1 osd.3 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:55.267620+0000 front 2024-09-11T23:40:55.267669+0000 (oldest deadline 2024-09-11T23:41:19.367691+0000) 2024-09-11T23:41:21.738 INFO:journalctl@ceph.osd.5.smithi136.stdout:Sep 11 23:41:21 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-5[46773]: 2024-09-11T23:41:21.420+0000 7f5a20e77640 -1 osd.5 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:54.364889+0000 front 2024-09-11T23:40:54.364905+0000 (oldest deadline 2024-09-11T23:41:19.664838+0000) 2024-09-11T23:41:21.739 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:41:21 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:41:21.450+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:56.932175+0000 front 2024-09-11T23:40:56.932499+0000 (oldest deadline 2024-09-11T23:41:21.032317+0000) 2024-09-11T23:41:21.827 INFO:journalctl@ceph.osd.6.smithi026.stdout:Sep 11 23:41:21 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-6[58789]: 2024-09-11T23:41:21.573+0000 7f8321638640 -1 osd.6 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:57.112923+0000 front 2024-09-11T23:40:57.113173+0000 (oldest deadline 2024-09-11T23:41:18.813188+0000) 2024-09-11T23:41:22.052 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:41:21 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:41:21.726+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:57.536934+0000 front 2024-09-11T23:40:57.537030+0000 (oldest deadline 2024-09-11T23:41:21.036908+0000) 2024-09-11T23:41:22.149 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:41:21 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9[65389]: 2024-09-11T23:41:21.827+0000 7fc7513f1640 -1 osd.9 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:57.370068+0000 front 2024-09-11T23:40:57.370191+0000 (oldest deadline 2024-09-11T23:41:21.470122+0000) 2024-09-11T23:41:22.238 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 11 23:41:21 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-8[53569]: 2024-09-11T23:41:21.805+0000 7f99e4062640 -1 osd.8 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:53.978855+0000 front 2024-09-11T23:40:53.978923+0000 (oldest deadline 2024-09-11T23:41:18.679037+0000) 2024-09-11T23:41:22.500 INFO:journalctl@ceph.osd.3.smithi026.stdout:Sep 11 23:41:22 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-3[51852]: 2024-09-11T23:41:22.149+0000 7f0fdfb33640 -1 osd.3 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:55.267620+0000 front 2024-09-11T23:40:55.267669+0000 (oldest deadline 2024-09-11T23:41:19.367691+0000) 2024-09-11T23:41:22.738 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:41:22 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:41:22.421+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:56.932175+0000 front 2024-09-11T23:40:56.932499+0000 (oldest deadline 2024-09-11T23:41:21.032317+0000) 2024-09-11T23:41:22.739 INFO:journalctl@ceph.osd.5.smithi136.stdout:Sep 11 23:41:22 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-5[46773]: 2024-09-11T23:41:22.382+0000 7f5a20e77640 -1 osd.5 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:54.364889+0000 front 2024-09-11T23:40:54.364905+0000 (oldest deadline 2024-09-11T23:41:19.664838+0000) 2024-09-11T23:41:22.866 INFO:journalctl@ceph.osd.3.smithi026.stdout:Sep 11 23:41:22 smithi026 podman[114709]: 2024-09-11 23:41:22.794732695 +0000 UTC m=+5.607771374 container died 0ac22a9bc2157a5c2e795c3dd23339a7b278304567661c9b03d3f39e61290306 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-3, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-09-11T23:41:22.866 INFO:journalctl@ceph.osd.6.smithi026.stdout:Sep 11 23:41:22 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-6[58789]: 2024-09-11T23:41:22.555+0000 7f8321638640 -1 osd.6 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:57.112923+0000 front 2024-09-11T23:40:57.113173+0000 (oldest deadline 2024-09-11T23:41:18.813188+0000) 2024-09-11T23:41:23.052 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:41:22 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:41:22.703+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:57.536934+0000 front 2024-09-11T23:40:57.537030+0000 (oldest deadline 2024-09-11T23:41:21.036908+0000) 2024-09-11T23:41:23.221 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:41:22 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9[65389]: 2024-09-11T23:41:22.866+0000 7fc7513f1640 -1 osd.9 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:57.370068+0000 front 2024-09-11T23:40:57.370191+0000 (oldest deadline 2024-09-11T23:41:21.470122+0000) 2024-09-11T23:41:23.239 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 11 23:41:22 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-8[53569]: 2024-09-11T23:41:22.800+0000 7f99e4062640 -1 osd.8 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:53.978855+0000 front 2024-09-11T23:40:53.978923+0000 (oldest deadline 2024-09-11T23:41:18.679037+0000) 2024-09-11T23:41:23.500 INFO:journalctl@ceph.osd.3.smithi026.stdout:Sep 11 23:41:23 smithi026 podman[114709]: 2024-09-11 23:41:23.221917794 +0000 UTC m=+6.034956514 container cleanup 0ac22a9bc2157a5c2e795c3dd23339a7b278304567661c9b03d3f39e61290306 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-3, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-09-11T23:41:23.501 INFO:journalctl@ceph.osd.3.smithi026.stdout:Sep 11 23:41:23 smithi026 bash[114709]: ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-3 2024-09-11T23:41:23.501 INFO:journalctl@ceph.osd.3.smithi026.stdout:Sep 11 23:41:23 smithi026 podman[114725]: 2024-09-11 23:41:23.423290568 +0000 UTC m=+0.631796434 container remove 0ac22a9bc2157a5c2e795c3dd23339a7b278304567661c9b03d3f39e61290306 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-3, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=squid-b53b6d3, ceph=True, org.label-schema.vendor=CentOS) 2024-09-11T23:41:23.738 INFO:journalctl@ceph.osd.5.smithi136.stdout:Sep 11 23:41:23 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-5[46773]: 2024-09-11T23:41:23.423+0000 7f5a20e77640 -1 osd.5 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:54.364889+0000 front 2024-09-11T23:40:54.364905+0000 (oldest deadline 2024-09-11T23:41:19.664838+0000) 2024-09-11T23:41:23.739 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:41:23 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:41:23.394+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:56.932175+0000 front 2024-09-11T23:40:56.932499+0000 (oldest deadline 2024-09-11T23:41:21.032317+0000) 2024-09-11T23:41:23.848 INFO:journalctl@ceph.osd.6.smithi026.stdout:Sep 11 23:41:23 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-6[58789]: 2024-09-11T23:41:23.540+0000 7f8321638640 -1 osd.6 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:57.112923+0000 front 2024-09-11T23:40:57.113173+0000 (oldest deadline 2024-09-11T23:41:18.813188+0000) 2024-09-11T23:41:24.052 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:41:23 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:41:23.667+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:57.536934+0000 front 2024-09-11T23:40:57.537030+0000 (oldest deadline 2024-09-11T23:41:21.036908+0000) 2024-09-11T23:41:24.100 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:41:23 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9[65389]: 2024-09-11T23:41:23.907+0000 7fc7513f1640 -1 osd.9 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:57.370068+0000 front 2024-09-11T23:40:57.370191+0000 (oldest deadline 2024-09-11T23:41:21.470122+0000) 2024-09-11T23:41:24.100 INFO:journalctl@ceph.osd.3.smithi026.stdout:Sep 11 23:41:23 smithi026 podman[114813]: 2024-09-11 23:41:23.752383005 +0000 UTC m=+0.016776263 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T23:41:24.100 INFO:journalctl@ceph.osd.3.smithi026.stdout:Sep 11 23:41:23 smithi026 podman[114813]: 2024-09-11 23:41:23.857475973 +0000 UTC m=+0.121869259 container create 1e7d60a176aafbeee08c95e1a0708e763c5afd120840b34fdca1f30c795196ff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-3-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3) 2024-09-11T23:41:24.101 INFO:journalctl@ceph.osd.3.smithi026.stdout:Sep 11 23:41:24 smithi026 podman[114813]: 2024-09-11 23:41:24.101577905 +0000 UTC m=+0.365971150 container init 1e7d60a176aafbeee08c95e1a0708e763c5afd120840b34fdca1f30c795196ff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-3-deactivate, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731) 2024-09-11T23:41:24.238 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 11 23:41:23 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-8[53569]: 2024-09-11T23:41:23.832+0000 7f99e4062640 -1 osd.8 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:53.978855+0000 front 2024-09-11T23:40:53.978923+0000 (oldest deadline 2024-09-11T23:41:18.679037+0000) 2024-09-11T23:41:24.495 INFO:journalctl@ceph.osd.3.smithi026.stdout:Sep 11 23:41:24 smithi026 podman[114813]: 2024-09-11 23:41:24.105121799 +0000 UTC m=+0.369515044 container start 1e7d60a176aafbeee08c95e1a0708e763c5afd120840b34fdca1f30c795196ff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-3-deactivate, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T23:41:24.495 INFO:journalctl@ceph.osd.3.smithi026.stdout:Sep 11 23:41:24 smithi026 podman[114813]: 2024-09-11 23:41:24.209924928 +0000 UTC m=+0.474318183 container attach 1e7d60a176aafbeee08c95e1a0708e763c5afd120840b34fdca1f30c795196ff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-3-deactivate, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-09-11T23:41:24.495 INFO:journalctl@ceph.osd.3.smithi026.stdout:Sep 11 23:41:24 smithi026 podman[114813]: 2024-09-11 23:41:24.297361832 +0000 UTC m=+0.561755094 container died 1e7d60a176aafbeee08c95e1a0708e763c5afd120840b34fdca1f30c795196ff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-3-deactivate, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3) 2024-09-11T23:41:24.738 INFO:journalctl@ceph.osd.5.smithi136.stdout:Sep 11 23:41:24 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-5[46773]: 2024-09-11T23:41:24.457+0000 7f5a20e77640 -1 osd.5 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:54.364889+0000 front 2024-09-11T23:40:54.364905+0000 (oldest deadline 2024-09-11T23:41:19.664838+0000) 2024-09-11T23:41:24.739 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:41:24 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:41:24.377+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:56.932175+0000 front 2024-09-11T23:40:56.932499+0000 (oldest deadline 2024-09-11T23:41:21.032317+0000) 2024-09-11T23:41:24.750 INFO:journalctl@ceph.osd.6.smithi026.stdout:Sep 11 23:41:24 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-6[58789]: 2024-09-11T23:41:24.495+0000 7f8321638640 -1 osd.6 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:57.112923+0000 front 2024-09-11T23:40:57.113173+0000 (oldest deadline 2024-09-11T23:41:18.813188+0000) 2024-09-11T23:41:24.923 DEBUG:teuthology.orchestra.run.smithi026:> sudo pkill -f 'journalctl -f -n 0 -u ceph-af7e6fc8-7092-11ef-bcea-c7b262605968@osd.3.service' 2024-09-11T23:41:24.965 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T23:41:24.965 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-09-11T23:41:24.965 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-09-11T23:41:24.966 DEBUG:teuthology.orchestra.run.smithi076:> sudo systemctl stop ceph-af7e6fc8-7092-11ef-bcea-c7b262605968@osd.4 2024-09-11T23:41:24.993 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:41:24 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:41:24.665+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:57.536934+0000 front 2024-09-11T23:40:57.537030+0000 (oldest deadline 2024-09-11T23:41:21.036908+0000) 2024-09-11T23:41:25.238 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 11 23:41:24 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-8[53569]: 2024-09-11T23:41:24.816+0000 7f99e4062640 -1 osd.8 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:53.978855+0000 front 2024-09-11T23:40:53.978923+0000 (oldest deadline 2024-09-11T23:41:18.679037+0000) 2024-09-11T23:41:25.250 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:41:24 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9[65389]: 2024-09-11T23:41:24.881+0000 7fc7513f1640 -1 osd.9 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:57.370068+0000 front 2024-09-11T23:40:57.370191+0000 (oldest deadline 2024-09-11T23:41:21.470122+0000) 2024-09-11T23:41:25.294 INFO:journalctl@ceph.osd.4.smithi076.stdout:Sep 11 23:41:25 smithi076 systemd[1]: Stopping Ceph osd.4 for af7e6fc8-7092-11ef-bcea-c7b262605968... 2024-09-11T23:41:25.552 INFO:journalctl@ceph.osd.4.smithi076.stdout:Sep 11 23:41:25 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-4[46247]: 2024-09-11T23:41:25.294+0000 7fa6f896b640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T23:41:25.552 INFO:journalctl@ceph.osd.4.smithi076.stdout:Sep 11 23:41:25 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-4[46247]: 2024-09-11T23:41:25.294+0000 7fa6f896b640 -1 osd.4 120 *** Got signal Terminated *** 2024-09-11T23:41:25.552 INFO:journalctl@ceph.osd.4.smithi076.stdout:Sep 11 23:41:25 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-4[46247]: 2024-09-11T23:41:25.294+0000 7fa6f896b640 -1 osd.4 120 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T23:41:25.739 INFO:journalctl@ceph.osd.5.smithi136.stdout:Sep 11 23:41:25 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-5[46773]: 2024-09-11T23:41:25.466+0000 7f5a20e77640 -1 osd.5 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:54.364889+0000 front 2024-09-11T23:40:54.364905+0000 (oldest deadline 2024-09-11T23:41:19.664838+0000) 2024-09-11T23:41:25.739 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:41:25 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:41:25.420+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:56.932175+0000 front 2024-09-11T23:40:56.932499+0000 (oldest deadline 2024-09-11T23:41:21.032317+0000) 2024-09-11T23:41:25.750 INFO:journalctl@ceph.osd.6.smithi026.stdout:Sep 11 23:41:25 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-6[58789]: 2024-09-11T23:41:25.456+0000 7f8321638640 -1 osd.6 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:57.112923+0000 front 2024-09-11T23:40:57.113173+0000 (oldest deadline 2024-09-11T23:41:18.813188+0000) 2024-09-11T23:41:26.052 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:41:25 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:41:25.676+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:57.536934+0000 front 2024-09-11T23:40:57.537030+0000 (oldest deadline 2024-09-11T23:41:21.036908+0000) 2024-09-11T23:41:26.239 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 11 23:41:25 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-8[53569]: 2024-09-11T23:41:25.808+0000 7f99e4062640 -1 osd.8 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:53.978855+0000 front 2024-09-11T23:40:53.978923+0000 (oldest deadline 2024-09-11T23:41:18.679037+0000) 2024-09-11T23:41:26.250 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:41:25 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9[65389]: 2024-09-11T23:41:25.891+0000 7fc7513f1640 -1 osd.9 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:57.370068+0000 front 2024-09-11T23:40:57.370191+0000 (oldest deadline 2024-09-11T23:41:21.470122+0000) 2024-09-11T23:41:26.738 INFO:journalctl@ceph.osd.5.smithi136.stdout:Sep 11 23:41:26 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-5[46773]: 2024-09-11T23:41:26.444+0000 7f5a20e77640 -1 osd.5 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:54.364889+0000 front 2024-09-11T23:40:54.364905+0000 (oldest deadline 2024-09-11T23:41:19.664838+0000) 2024-09-11T23:41:26.739 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:41:26 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:41:26.427+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:56.932175+0000 front 2024-09-11T23:40:56.932499+0000 (oldest deadline 2024-09-11T23:41:21.032317+0000) 2024-09-11T23:41:26.845 INFO:journalctl@ceph.osd.6.smithi026.stdout:Sep 11 23:41:26 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-6[58789]: 2024-09-11T23:41:26.506+0000 7f8321638640 -1 osd.6 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:57.112923+0000 front 2024-09-11T23:40:57.113173+0000 (oldest deadline 2024-09-11T23:41:18.813188+0000) 2024-09-11T23:41:27.052 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:41:26 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:41:26.696+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:57.536934+0000 front 2024-09-11T23:40:57.537030+0000 (oldest deadline 2024-09-11T23:41:21.036908+0000) 2024-09-11T23:41:27.239 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 11 23:41:26 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-8[53569]: 2024-09-11T23:41:26.791+0000 7f99e4062640 -1 osd.8 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:53.978855+0000 front 2024-09-11T23:40:53.978923+0000 (oldest deadline 2024-09-11T23:41:18.679037+0000) 2024-09-11T23:41:27.250 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:41:26 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9[65389]: 2024-09-11T23:41:26.845+0000 7fc7513f1640 -1 osd.9 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:57.370068+0000 front 2024-09-11T23:40:57.370191+0000 (oldest deadline 2024-09-11T23:41:21.470122+0000) 2024-09-11T23:41:27.738 INFO:journalctl@ceph.osd.5.smithi136.stdout:Sep 11 23:41:27 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-5[46773]: 2024-09-11T23:41:27.396+0000 7f5a20e77640 -1 osd.5 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:54.364889+0000 front 2024-09-11T23:40:54.364905+0000 (oldest deadline 2024-09-11T23:41:19.664838+0000) 2024-09-11T23:41:27.739 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:41:27 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:41:27.437+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:56.932175+0000 front 2024-09-11T23:40:56.932499+0000 (oldest deadline 2024-09-11T23:41:21.032317+0000) 2024-09-11T23:41:27.856 INFO:journalctl@ceph.osd.6.smithi026.stdout:Sep 11 23:41:27 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-6[58789]: 2024-09-11T23:41:27.513+0000 7f8321638640 -1 osd.6 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:57.112923+0000 front 2024-09-11T23:40:57.113173+0000 (oldest deadline 2024-09-11T23:41:18.813188+0000) 2024-09-11T23:41:28.052 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:41:27 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:41:27.659+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:57.536934+0000 front 2024-09-11T23:40:57.537030+0000 (oldest deadline 2024-09-11T23:41:21.036908+0000) 2024-09-11T23:41:28.239 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 11 23:41:27 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-8[53569]: 2024-09-11T23:41:27.833+0000 7f99e4062640 -1 osd.8 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:53.978855+0000 front 2024-09-11T23:40:53.978923+0000 (oldest deadline 2024-09-11T23:41:18.679037+0000) 2024-09-11T23:41:28.250 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:41:27 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9[65389]: 2024-09-11T23:41:27.857+0000 7fc7513f1640 -1 osd.9 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:57.370068+0000 front 2024-09-11T23:40:57.370191+0000 (oldest deadline 2024-09-11T23:41:21.470122+0000) 2024-09-11T23:41:28.250 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:41:27 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9[65389]: 2024-09-11T23:41:27.857+0000 7fc7513f1640 -1 osd.9 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:04.971207+0000 front 2024-09-11T23:41:04.971047+0000 (oldest deadline 2024-09-11T23:41:27.271258+0000) 2024-09-11T23:41:28.738 INFO:journalctl@ceph.osd.5.smithi136.stdout:Sep 11 23:41:28 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-5[46773]: 2024-09-11T23:41:28.362+0000 7f5a20e77640 -1 osd.5 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:54.364889+0000 front 2024-09-11T23:40:54.364905+0000 (oldest deadline 2024-09-11T23:41:19.664838+0000) 2024-09-11T23:41:28.739 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:41:28 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:41:28.426+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:56.932175+0000 front 2024-09-11T23:40:56.932499+0000 (oldest deadline 2024-09-11T23:41:21.032317+0000) 2024-09-11T23:41:28.750 INFO:journalctl@ceph.osd.6.smithi026.stdout:Sep 11 23:41:28 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-6[58789]: 2024-09-11T23:41:28.492+0000 7f8321638640 -1 osd.6 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:57.112923+0000 front 2024-09-11T23:40:57.113173+0000 (oldest deadline 2024-09-11T23:41:18.813188+0000) 2024-09-11T23:41:28.802 INFO:journalctl@ceph.osd.4.smithi076.stdout:Sep 11 23:41:28 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-4[46247]: 2024-09-11T23:41:28.449+0000 7fa6f0f68640 -1 osd.4 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:03.701338+0000 front 2024-09-11T23:41:03.701303+0000 (oldest deadline 2024-09-11T23:41:28.401455+0000) 2024-09-11T23:41:28.802 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:41:28 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:41:28.678+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:57.536934+0000 front 2024-09-11T23:40:57.537030+0000 (oldest deadline 2024-09-11T23:41:21.036908+0000) 2024-09-11T23:41:29.238 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 11 23:41:28 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-8[53569]: 2024-09-11T23:41:28.879+0000 7f99e4062640 -1 osd.8 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:53.978855+0000 front 2024-09-11T23:40:53.978923+0000 (oldest deadline 2024-09-11T23:41:18.679037+0000) 2024-09-11T23:41:29.250 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:41:28 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9[65389]: 2024-09-11T23:41:28.849+0000 7fc7513f1640 -1 osd.9 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:57.370068+0000 front 2024-09-11T23:40:57.370191+0000 (oldest deadline 2024-09-11T23:41:21.470122+0000) 2024-09-11T23:41:29.250 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:41:28 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9[65389]: 2024-09-11T23:41:28.849+0000 7fc7513f1640 -1 osd.9 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:04.971207+0000 front 2024-09-11T23:41:04.971047+0000 (oldest deadline 2024-09-11T23:41:27.271258+0000) 2024-09-11T23:41:29.704 INFO:journalctl@ceph.osd.4.smithi076.stdout:Sep 11 23:41:29 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-4[46247]: 2024-09-11T23:41:29.447+0000 7fa6f0f68640 -1 osd.4 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:03.701338+0000 front 2024-09-11T23:41:03.701303+0000 (oldest deadline 2024-09-11T23:41:28.401455+0000) 2024-09-11T23:41:29.704 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:41:29 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:41:29.692+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:57.536934+0000 front 2024-09-11T23:40:57.537030+0000 (oldest deadline 2024-09-11T23:41:21.036908+0000) 2024-09-11T23:41:29.704 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:41:29 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:41:29.692+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:03.838149+0000 front 2024-09-11T23:41:03.838269+0000 (oldest deadline 2024-09-11T23:41:29.138274+0000) 2024-09-11T23:41:29.739 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:41:29 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:41:29.416+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:56.932175+0000 front 2024-09-11T23:40:56.932499+0000 (oldest deadline 2024-09-11T23:41:21.032317+0000) 2024-09-11T23:41:29.739 INFO:journalctl@ceph.osd.5.smithi136.stdout:Sep 11 23:41:29 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-5[46773]: 2024-09-11T23:41:29.353+0000 7f5a20e77640 -1 osd.5 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:54.364889+0000 front 2024-09-11T23:40:54.364905+0000 (oldest deadline 2024-09-11T23:41:19.664838+0000) 2024-09-11T23:41:29.739 INFO:journalctl@ceph.osd.5.smithi136.stdout:Sep 11 23:41:29 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-5[46773]: 2024-09-11T23:41:29.353+0000 7f5a20e77640 -1 osd.5 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:04.965960+0000 front 2024-09-11T23:41:04.965997+0000 (oldest deadline 2024-09-11T23:41:28.466123+0000) 2024-09-11T23:41:29.750 INFO:journalctl@ceph.osd.6.smithi026.stdout:Sep 11 23:41:29 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-6[58789]: 2024-09-11T23:41:29.482+0000 7f8321638640 -1 osd.6 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:57.112923+0000 front 2024-09-11T23:40:57.113173+0000 (oldest deadline 2024-09-11T23:41:18.813188+0000) 2024-09-11T23:41:30.238 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 11 23:41:29 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-8[53569]: 2024-09-11T23:41:29.922+0000 7f99e4062640 -1 osd.8 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:53.978855+0000 front 2024-09-11T23:40:53.978923+0000 (oldest deadline 2024-09-11T23:41:18.679037+0000) 2024-09-11T23:41:30.250 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:41:29 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9[65389]: 2024-09-11T23:41:29.814+0000 7fc7513f1640 -1 osd.9 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:57.370068+0000 front 2024-09-11T23:40:57.370191+0000 (oldest deadline 2024-09-11T23:41:21.470122+0000) 2024-09-11T23:41:30.250 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:41:29 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9[65389]: 2024-09-11T23:41:29.814+0000 7fc7513f1640 -1 osd.9 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:04.971207+0000 front 2024-09-11T23:41:04.971047+0000 (oldest deadline 2024-09-11T23:41:27.271258+0000) 2024-09-11T23:41:30.508 INFO:journalctl@ceph.osd.5.smithi136.stdout:Sep 11 23:41:30 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-5[46773]: 2024-09-11T23:41:30.359+0000 7f5a20e77640 -1 osd.5 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:54.364889+0000 front 2024-09-11T23:40:54.364905+0000 (oldest deadline 2024-09-11T23:41:19.664838+0000) 2024-09-11T23:41:30.508 INFO:journalctl@ceph.osd.5.smithi136.stdout:Sep 11 23:41:30 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-5[46773]: 2024-09-11T23:41:30.359+0000 7f5a20e77640 -1 osd.5 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:04.965960+0000 front 2024-09-11T23:41:04.965997+0000 (oldest deadline 2024-09-11T23:41:28.466123+0000) 2024-09-11T23:41:30.509 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:41:30 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:41:30.465+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:56.932175+0000 front 2024-09-11T23:40:56.932499+0000 (oldest deadline 2024-09-11T23:41:21.032317+0000) 2024-09-11T23:41:30.509 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:41:30 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:41:30.465+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:01.032761+0000 front 2024-09-11T23:41:01.032638+0000 (oldest deadline 2024-09-11T23:41:30.433335+0000) 2024-09-11T23:41:30.750 INFO:journalctl@ceph.osd.6.smithi026.stdout:Sep 11 23:41:30 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-6[58789]: 2024-09-11T23:41:30.482+0000 7f8321638640 -1 osd.6 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:57.112923+0000 front 2024-09-11T23:40:57.113173+0000 (oldest deadline 2024-09-11T23:41:18.813188+0000) 2024-09-11T23:41:30.981 INFO:journalctl@ceph.osd.4.smithi076.stdout:Sep 11 23:41:30 smithi076 podman[73482]: 2024-09-11 23:41:30.779702367 +0000 UTC m=+5.609942762 container died c29747542f6b5550131f78cc6286728c248f595faaa733147872bce4fcfbded0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-4, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T23:41:30.981 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:41:30 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:41:30.660+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:57.536934+0000 front 2024-09-11T23:40:57.537030+0000 (oldest deadline 2024-09-11T23:41:21.036908+0000) 2024-09-11T23:41:30.982 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:41:30 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:41:30.660+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:03.838149+0000 front 2024-09-11T23:41:03.838269+0000 (oldest deadline 2024-09-11T23:41:29.138274+0000) 2024-09-11T23:41:31.239 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 11 23:41:30 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-8[53569]: 2024-09-11T23:41:30.906+0000 7f99e4062640 -1 osd.8 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:53.978855+0000 front 2024-09-11T23:40:53.978923+0000 (oldest deadline 2024-09-11T23:41:18.679037+0000) 2024-09-11T23:41:31.250 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:41:30 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9[65389]: 2024-09-11T23:41:30.838+0000 7fc7513f1640 -1 osd.9 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:57.370068+0000 front 2024-09-11T23:40:57.370191+0000 (oldest deadline 2024-09-11T23:41:21.470122+0000) 2024-09-11T23:41:31.250 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:41:30 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9[65389]: 2024-09-11T23:41:30.838+0000 7fc7513f1640 -1 osd.9 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:04.971207+0000 front 2024-09-11T23:41:04.971047+0000 (oldest deadline 2024-09-11T23:41:27.271258+0000) 2024-09-11T23:41:31.302 INFO:journalctl@ceph.osd.4.smithi076.stdout:Sep 11 23:41:31 smithi076 podman[73482]: 2024-09-11 23:41:31.185651113 +0000 UTC m=+6.015891492 container cleanup c29747542f6b5550131f78cc6286728c248f595faaa733147872bce4fcfbded0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-4, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-09-11T23:41:31.302 INFO:journalctl@ceph.osd.4.smithi076.stdout:Sep 11 23:41:31 smithi076 bash[73482]: ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-4 2024-09-11T23:41:31.738 INFO:journalctl@ceph.osd.5.smithi136.stdout:Sep 11 23:41:31 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-5[46773]: 2024-09-11T23:41:31.310+0000 7f5a20e77640 -1 osd.5 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:54.364889+0000 front 2024-09-11T23:40:54.364905+0000 (oldest deadline 2024-09-11T23:41:19.664838+0000) 2024-09-11T23:41:31.739 INFO:journalctl@ceph.osd.5.smithi136.stdout:Sep 11 23:41:31 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-5[46773]: 2024-09-11T23:41:31.310+0000 7f5a20e77640 -1 osd.5 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:04.965960+0000 front 2024-09-11T23:41:04.965997+0000 (oldest deadline 2024-09-11T23:41:28.466123+0000) 2024-09-11T23:41:31.739 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:41:31 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:41:31.511+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:56.932175+0000 front 2024-09-11T23:40:56.932499+0000 (oldest deadline 2024-09-11T23:41:21.032317+0000) 2024-09-11T23:41:31.739 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:41:31 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:41:31.511+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:01.032761+0000 front 2024-09-11T23:41:01.032638+0000 (oldest deadline 2024-09-11T23:41:30.433335+0000) 2024-09-11T23:41:31.766 INFO:journalctl@ceph.osd.4.smithi076.stdout:Sep 11 23:41:31 smithi076 podman[73499]: 2024-09-11 23:41:31.428634529 +0000 UTC m=+0.644835251 container remove c29747542f6b5550131f78cc6286728c248f595faaa733147872bce4fcfbded0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-4, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.build-date=20240731) 2024-09-11T23:41:31.766 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:41:31 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:41:31.654+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:57.536934+0000 front 2024-09-11T23:40:57.537030+0000 (oldest deadline 2024-09-11T23:41:21.036908+0000) 2024-09-11T23:41:31.767 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:41:31 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:41:31.654+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:03.838149+0000 front 2024-09-11T23:41:03.838269+0000 (oldest deadline 2024-09-11T23:41:29.138274+0000) 2024-09-11T23:41:31.866 INFO:journalctl@ceph.osd.6.smithi026.stdout:Sep 11 23:41:31 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-6[58789]: 2024-09-11T23:41:31.509+0000 7f8321638640 -1 osd.6 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:57.112923+0000 front 2024-09-11T23:40:57.113173+0000 (oldest deadline 2024-09-11T23:41:18.813188+0000) 2024-09-11T23:41:31.866 INFO:journalctl@ceph.osd.6.smithi026.stdout:Sep 11 23:41:31 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-6[58789]: 2024-09-11T23:41:31.509+0000 7f8321638640 -1 osd.6 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:05.814845+0000 front 2024-09-11T23:41:05.814472+0000 (oldest deadline 2024-09-11T23:41:31.114608+0000) 2024-09-11T23:41:32.052 INFO:journalctl@ceph.osd.4.smithi076.stdout:Sep 11 23:41:31 smithi076 podman[73597]: 2024-09-11 23:41:31.669858215 +0000 UTC m=+0.015890366 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T23:41:32.052 INFO:journalctl@ceph.osd.4.smithi076.stdout:Sep 11 23:41:31 smithi076 podman[73597]: 2024-09-11 23:41:31.771058881 +0000 UTC m=+0.117091030 container create 3ba3a7c5265cc0ed13264f3155a741c8ed7463fc332054cb46d42051838e2e9e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-4-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-09-11T23:41:32.053 INFO:journalctl@ceph.osd.4.smithi076.stdout:Sep 11 23:41:31 smithi076 podman[73597]: 2024-09-11 23:41:31.981831229 +0000 UTC m=+0.327863371 container init 3ba3a7c5265cc0ed13264f3155a741c8ed7463fc332054cb46d42051838e2e9e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-4-deactivate, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T23:41:32.053 INFO:journalctl@ceph.osd.4.smithi076.stdout:Sep 11 23:41:31 smithi076 podman[73597]: 2024-09-11 23:41:31.985284583 +0000 UTC m=+0.331316729 container start 3ba3a7c5265cc0ed13264f3155a741c8ed7463fc332054cb46d42051838e2e9e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-4-deactivate, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T23:41:32.238 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 11 23:41:31 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-8[53569]: 2024-09-11T23:41:31.942+0000 7f99e4062640 -1 osd.8 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:53.978855+0000 front 2024-09-11T23:40:53.978923+0000 (oldest deadline 2024-09-11T23:41:18.679037+0000) 2024-09-11T23:41:32.250 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:41:31 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9[65389]: 2024-09-11T23:41:31.867+0000 7fc7513f1640 -1 osd.9 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:57.370068+0000 front 2024-09-11T23:40:57.370191+0000 (oldest deadline 2024-09-11T23:41:21.470122+0000) 2024-09-11T23:41:32.250 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:41:31 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9[65389]: 2024-09-11T23:41:31.867+0000 7fc7513f1640 -1 osd.9 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:04.971207+0000 front 2024-09-11T23:41:04.971047+0000 (oldest deadline 2024-09-11T23:41:27.271258+0000) 2024-09-11T23:41:32.367 INFO:journalctl@ceph.osd.4.smithi076.stdout:Sep 11 23:41:32 smithi076 podman[73597]: 2024-09-11 23:41:32.090455541 +0000 UTC m=+0.436487689 container attach 3ba3a7c5265cc0ed13264f3155a741c8ed7463fc332054cb46d42051838e2e9e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-4-deactivate, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-09-11T23:41:32.367 INFO:journalctl@ceph.osd.4.smithi076.stdout:Sep 11 23:41:32 smithi076 podman[73597]: 2024-09-11 23:41:32.207302483 +0000 UTC m=+0.553334639 container died 3ba3a7c5265cc0ed13264f3155a741c8ed7463fc332054cb46d42051838e2e9e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T23:41:32.683 INFO:journalctl@ceph.osd.7.smithi076.stdout:Sep 11 23:41:32 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-7[52804]: 2024-09-11T23:41:32.367+0000 7f28fcc36640 -1 osd.7 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:05.616564+0000 front 2024-09-11T23:41:05.616590+0000 (oldest deadline 2024-09-11T23:41:31.516796+0000) 2024-09-11T23:41:32.739 INFO:journalctl@ceph.osd.5.smithi136.stdout:Sep 11 23:41:32 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-5[46773]: 2024-09-11T23:41:32.322+0000 7f5a20e77640 -1 osd.5 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:54.364889+0000 front 2024-09-11T23:40:54.364905+0000 (oldest deadline 2024-09-11T23:41:19.664838+0000) 2024-09-11T23:41:32.739 INFO:journalctl@ceph.osd.5.smithi136.stdout:Sep 11 23:41:32 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-5[46773]: 2024-09-11T23:41:32.322+0000 7f5a20e77640 -1 osd.5 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:04.965960+0000 front 2024-09-11T23:41:04.965997+0000 (oldest deadline 2024-09-11T23:41:28.466123+0000) 2024-09-11T23:41:32.739 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:41:32 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:41:32.477+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:56.932175+0000 front 2024-09-11T23:40:56.932499+0000 (oldest deadline 2024-09-11T23:41:21.032317+0000) 2024-09-11T23:41:32.739 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:41:32 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:41:32.477+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:01.032761+0000 front 2024-09-11T23:41:01.032638+0000 (oldest deadline 2024-09-11T23:41:30.433335+0000) 2024-09-11T23:41:32.786 DEBUG:teuthology.orchestra.run.smithi076:> sudo pkill -f 'journalctl -f -n 0 -u ceph-af7e6fc8-7092-11ef-bcea-c7b262605968@osd.4.service' 2024-09-11T23:41:32.844 INFO:journalctl@ceph.osd.6.smithi026.stdout:Sep 11 23:41:32 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-6[58789]: 2024-09-11T23:41:32.507+0000 7f8321638640 -1 osd.6 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:57.112923+0000 front 2024-09-11T23:40:57.113173+0000 (oldest deadline 2024-09-11T23:41:18.813188+0000) 2024-09-11T23:41:32.844 INFO:journalctl@ceph.osd.6.smithi026.stdout:Sep 11 23:41:32 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-6[58789]: 2024-09-11T23:41:32.507+0000 7f8321638640 -1 osd.6 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:05.814845+0000 front 2024-09-11T23:41:05.814472+0000 (oldest deadline 2024-09-11T23:41:31.114608+0000) 2024-09-11T23:41:32.865 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T23:41:32.865 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-09-11T23:41:32.865 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-09-11T23:41:32.865 DEBUG:teuthology.orchestra.run.smithi136:> sudo systemctl stop ceph-af7e6fc8-7092-11ef-bcea-c7b262605968@osd.5 2024-09-11T23:41:33.052 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:41:32 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:41:32.683+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:57.536934+0000 front 2024-09-11T23:40:57.537030+0000 (oldest deadline 2024-09-11T23:41:21.036908+0000) 2024-09-11T23:41:33.052 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:41:32 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:41:32.683+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:03.838149+0000 front 2024-09-11T23:41:03.838269+0000 (oldest deadline 2024-09-11T23:41:29.138274+0000) 2024-09-11T23:41:33.193 INFO:journalctl@ceph.osd.5.smithi136.stdout:Sep 11 23:41:32 smithi136 systemd[1]: Stopping Ceph osd.5 for af7e6fc8-7092-11ef-bcea-c7b262605968... 2024-09-11T23:41:33.194 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 11 23:41:32 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-8[53569]: 2024-09-11T23:41:32.918+0000 7f99e4062640 -1 osd.8 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:53.978855+0000 front 2024-09-11T23:40:53.978923+0000 (oldest deadline 2024-09-11T23:41:18.679037+0000) 2024-09-11T23:41:33.250 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:41:32 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9[65389]: 2024-09-11T23:41:32.845+0000 7fc7513f1640 -1 osd.9 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:57.370068+0000 front 2024-09-11T23:40:57.370191+0000 (oldest deadline 2024-09-11T23:41:21.470122+0000) 2024-09-11T23:41:33.250 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:41:32 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9[65389]: 2024-09-11T23:41:32.845+0000 7fc7513f1640 -1 osd.9 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:04.971207+0000 front 2024-09-11T23:41:04.971047+0000 (oldest deadline 2024-09-11T23:41:27.271258+0000) 2024-09-11T23:41:33.459 INFO:journalctl@ceph.osd.5.smithi136.stdout:Sep 11 23:41:33 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-5[46773]: 2024-09-11T23:41:33.193+0000 7f5a24872640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T23:41:33.459 INFO:journalctl@ceph.osd.5.smithi136.stdout:Sep 11 23:41:33 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-5[46773]: 2024-09-11T23:41:33.193+0000 7f5a24872640 -1 osd.5 120 *** Got signal Terminated *** 2024-09-11T23:41:33.459 INFO:journalctl@ceph.osd.5.smithi136.stdout:Sep 11 23:41:33 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-5[46773]: 2024-09-11T23:41:33.193+0000 7f5a24872640 -1 osd.5 120 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T23:41:33.459 INFO:journalctl@ceph.osd.5.smithi136.stdout:Sep 11 23:41:33 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-5[46773]: 2024-09-11T23:41:33.339+0000 7f5a20e77640 -1 osd.5 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:54.364889+0000 front 2024-09-11T23:40:54.364905+0000 (oldest deadline 2024-09-11T23:41:19.664838+0000) 2024-09-11T23:41:33.459 INFO:journalctl@ceph.osd.5.smithi136.stdout:Sep 11 23:41:33 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-5[46773]: 2024-09-11T23:41:33.339+0000 7f5a20e77640 -1 osd.5 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:04.965960+0000 front 2024-09-11T23:41:04.965997+0000 (oldest deadline 2024-09-11T23:41:28.466123+0000) 2024-09-11T23:41:33.738 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:41:33 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:41:33.458+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:56.932175+0000 front 2024-09-11T23:40:56.932499+0000 (oldest deadline 2024-09-11T23:41:21.032317+0000) 2024-09-11T23:41:33.738 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:41:33 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:41:33.458+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:01.032761+0000 front 2024-09-11T23:41:01.032638+0000 (oldest deadline 2024-09-11T23:41:30.433335+0000) 2024-09-11T23:41:33.802 INFO:journalctl@ceph.osd.7.smithi076.stdout:Sep 11 23:41:33 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-7[52804]: 2024-09-11T23:41:33.417+0000 7f28fcc36640 -1 osd.7 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:05.616564+0000 front 2024-09-11T23:41:05.616590+0000 (oldest deadline 2024-09-11T23:41:31.516796+0000) 2024-09-11T23:41:33.802 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:41:33 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:41:33.655+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:57.536934+0000 front 2024-09-11T23:40:57.537030+0000 (oldest deadline 2024-09-11T23:41:21.036908+0000) 2024-09-11T23:41:33.802 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:41:33 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:41:33.655+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:03.838149+0000 front 2024-09-11T23:41:03.838269+0000 (oldest deadline 2024-09-11T23:41:29.138274+0000) 2024-09-11T23:41:33.883 INFO:journalctl@ceph.osd.6.smithi026.stdout:Sep 11 23:41:33 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-6[58789]: 2024-09-11T23:41:33.510+0000 7f8321638640 -1 osd.6 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:57.112923+0000 front 2024-09-11T23:40:57.113173+0000 (oldest deadline 2024-09-11T23:41:18.813188+0000) 2024-09-11T23:41:33.883 INFO:journalctl@ceph.osd.6.smithi026.stdout:Sep 11 23:41:33 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-6[58789]: 2024-09-11T23:41:33.510+0000 7f8321638640 -1 osd.6 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:05.814845+0000 front 2024-09-11T23:41:05.814472+0000 (oldest deadline 2024-09-11T23:41:31.114608+0000) 2024-09-11T23:41:34.238 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 11 23:41:33 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-8[53569]: 2024-09-11T23:41:33.877+0000 7f99e4062640 -1 osd.8 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:53.978855+0000 front 2024-09-11T23:40:53.978923+0000 (oldest deadline 2024-09-11T23:41:18.679037+0000) 2024-09-11T23:41:34.250 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:41:33 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9[65389]: 2024-09-11T23:41:33.884+0000 7fc7513f1640 -1 osd.9 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:57.370068+0000 front 2024-09-11T23:40:57.370191+0000 (oldest deadline 2024-09-11T23:41:21.470122+0000) 2024-09-11T23:41:34.250 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:41:33 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9[65389]: 2024-09-11T23:41:33.884+0000 7fc7513f1640 -1 osd.9 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:04.971207+0000 front 2024-09-11T23:41:04.971047+0000 (oldest deadline 2024-09-11T23:41:27.271258+0000) 2024-09-11T23:41:34.738 INFO:journalctl@ceph.osd.5.smithi136.stdout:Sep 11 23:41:34 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-5[46773]: 2024-09-11T23:41:34.336+0000 7f5a20e77640 -1 osd.5 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:54.364889+0000 front 2024-09-11T23:40:54.364905+0000 (oldest deadline 2024-09-11T23:41:19.664838+0000) 2024-09-11T23:41:34.738 INFO:journalctl@ceph.osd.5.smithi136.stdout:Sep 11 23:41:34 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-5[46773]: 2024-09-11T23:41:34.336+0000 7f5a20e77640 -1 osd.5 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:04.965960+0000 front 2024-09-11T23:41:04.965997+0000 (oldest deadline 2024-09-11T23:41:28.466123+0000) 2024-09-11T23:41:34.739 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:41:34 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:41:34.435+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:56.932175+0000 front 2024-09-11T23:40:56.932499+0000 (oldest deadline 2024-09-11T23:41:21.032317+0000) 2024-09-11T23:41:34.739 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:41:34 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:41:34.435+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:01.032761+0000 front 2024-09-11T23:41:01.032638+0000 (oldest deadline 2024-09-11T23:41:30.433335+0000) 2024-09-11T23:41:34.750 INFO:journalctl@ceph.osd.6.smithi026.stdout:Sep 11 23:41:34 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-6[58789]: 2024-09-11T23:41:34.491+0000 7f8321638640 -1 osd.6 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:57.112923+0000 front 2024-09-11T23:40:57.113173+0000 (oldest deadline 2024-09-11T23:41:18.813188+0000) 2024-09-11T23:41:34.750 INFO:journalctl@ceph.osd.6.smithi026.stdout:Sep 11 23:41:34 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-6[58789]: 2024-09-11T23:41:34.491+0000 7f8321638640 -1 osd.6 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:05.814845+0000 front 2024-09-11T23:41:05.814472+0000 (oldest deadline 2024-09-11T23:41:31.114608+0000) 2024-09-11T23:41:34.802 INFO:journalctl@ceph.osd.7.smithi076.stdout:Sep 11 23:41:34 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-7[52804]: 2024-09-11T23:41:34.432+0000 7f28fcc36640 -1 osd.7 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:05.616564+0000 front 2024-09-11T23:41:05.616590+0000 (oldest deadline 2024-09-11T23:41:31.516796+0000) 2024-09-11T23:41:34.802 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:41:34 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:41:34.651+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:57.536934+0000 front 2024-09-11T23:40:57.537030+0000 (oldest deadline 2024-09-11T23:41:21.036908+0000) 2024-09-11T23:41:34.802 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:41:34 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:41:34.651+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:03.838149+0000 front 2024-09-11T23:41:03.838269+0000 (oldest deadline 2024-09-11T23:41:29.138274+0000) 2024-09-11T23:41:35.239 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 11 23:41:34 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-8[53569]: 2024-09-11T23:41:34.886+0000 7f99e4062640 -1 osd.8 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:53.978855+0000 front 2024-09-11T23:40:53.978923+0000 (oldest deadline 2024-09-11T23:41:18.679037+0000) 2024-09-11T23:41:35.250 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:41:34 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9[65389]: 2024-09-11T23:41:34.930+0000 7fc7513f1640 -1 osd.9 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:57.370068+0000 front 2024-09-11T23:40:57.370191+0000 (oldest deadline 2024-09-11T23:41:21.470122+0000) 2024-09-11T23:41:35.250 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:41:34 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9[65389]: 2024-09-11T23:41:34.930+0000 7fc7513f1640 -1 osd.9 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:04.971207+0000 front 2024-09-11T23:41:04.971047+0000 (oldest deadline 2024-09-11T23:41:27.271258+0000) 2024-09-11T23:41:35.739 INFO:journalctl@ceph.osd.5.smithi136.stdout:Sep 11 23:41:35 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-5[46773]: 2024-09-11T23:41:35.307+0000 7f5a20e77640 -1 osd.5 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:54.364889+0000 front 2024-09-11T23:40:54.364905+0000 (oldest deadline 2024-09-11T23:41:19.664838+0000) 2024-09-11T23:41:35.739 INFO:journalctl@ceph.osd.5.smithi136.stdout:Sep 11 23:41:35 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-5[46773]: 2024-09-11T23:41:35.307+0000 7f5a20e77640 -1 osd.5 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:04.965960+0000 front 2024-09-11T23:41:04.965997+0000 (oldest deadline 2024-09-11T23:41:28.466123+0000) 2024-09-11T23:41:35.739 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:41:35 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:41:35.470+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:56.932175+0000 front 2024-09-11T23:40:56.932499+0000 (oldest deadline 2024-09-11T23:41:21.032317+0000) 2024-09-11T23:41:35.739 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:41:35 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:41:35.470+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:01.032761+0000 front 2024-09-11T23:41:01.032638+0000 (oldest deadline 2024-09-11T23:41:30.433335+0000) 2024-09-11T23:41:35.802 INFO:journalctl@ceph.osd.7.smithi076.stdout:Sep 11 23:41:35 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-7[52804]: 2024-09-11T23:41:35.413+0000 7f28fcc36640 -1 osd.7 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:05.616564+0000 front 2024-09-11T23:41:05.616590+0000 (oldest deadline 2024-09-11T23:41:31.516796+0000) 2024-09-11T23:41:35.802 INFO:journalctl@ceph.osd.7.smithi076.stdout:Sep 11 23:41:35 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-7[52804]: 2024-09-11T23:41:35.413+0000 7f28fcc36640 -1 osd.7 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:11.517294+0000 front 2024-09-11T23:41:11.517280+0000 (oldest deadline 2024-09-11T23:41:35.017284+0000) 2024-09-11T23:41:35.802 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:41:35 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:41:35.616+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:57.536934+0000 front 2024-09-11T23:40:57.537030+0000 (oldest deadline 2024-09-11T23:41:21.036908+0000) 2024-09-11T23:41:35.803 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:41:35 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:41:35.616+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:03.838149+0000 front 2024-09-11T23:41:03.838269+0000 (oldest deadline 2024-09-11T23:41:29.138274+0000) 2024-09-11T23:41:35.929 INFO:journalctl@ceph.osd.6.smithi026.stdout:Sep 11 23:41:35 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-6[58789]: 2024-09-11T23:41:35.535+0000 7f8321638640 -1 osd.6 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:57.112923+0000 front 2024-09-11T23:40:57.113173+0000 (oldest deadline 2024-09-11T23:41:18.813188+0000) 2024-09-11T23:41:35.929 INFO:journalctl@ceph.osd.6.smithi026.stdout:Sep 11 23:41:35 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-6[58789]: 2024-09-11T23:41:35.535+0000 7f8321638640 -1 osd.6 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:05.814845+0000 front 2024-09-11T23:41:05.814472+0000 (oldest deadline 2024-09-11T23:41:31.114608+0000) 2024-09-11T23:41:35.929 INFO:journalctl@ceph.osd.6.smithi026.stdout:Sep 11 23:41:35 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-6[58789]: 2024-09-11T23:41:35.535+0000 7f8321638640 -1 osd.6 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:11.115165+0000 front 2024-09-11T23:41:11.115230+0000 (oldest deadline 2024-09-11T23:41:35.215334+0000) 2024-09-11T23:41:36.238 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 11 23:41:35 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-8[53569]: 2024-09-11T23:41:35.836+0000 7f99e4062640 -1 osd.8 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:53.978855+0000 front 2024-09-11T23:40:53.978923+0000 (oldest deadline 2024-09-11T23:41:18.679037+0000) 2024-09-11T23:41:36.250 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:41:35 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9[65389]: 2024-09-11T23:41:35.930+0000 7fc7513f1640 -1 osd.9 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:57.370068+0000 front 2024-09-11T23:40:57.370191+0000 (oldest deadline 2024-09-11T23:41:21.470122+0000) 2024-09-11T23:41:36.250 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:41:35 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9[65389]: 2024-09-11T23:41:35.930+0000 7fc7513f1640 -1 osd.9 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:04.971207+0000 front 2024-09-11T23:41:04.971047+0000 (oldest deadline 2024-09-11T23:41:27.271258+0000) 2024-09-11T23:41:36.738 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:41:36 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:41:36.451+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:56.932175+0000 front 2024-09-11T23:40:56.932499+0000 (oldest deadline 2024-09-11T23:41:21.032317+0000) 2024-09-11T23:41:36.739 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:41:36 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:41:36.451+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:01.032761+0000 front 2024-09-11T23:41:01.032638+0000 (oldest deadline 2024-09-11T23:41:30.433335+0000) 2024-09-11T23:41:36.739 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:41:36 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:41:36.451+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:10.934510+0000 front 2024-09-11T23:41:10.934411+0000 (oldest deadline 2024-09-11T23:41:35.634455+0000) 2024-09-11T23:41:36.739 INFO:journalctl@ceph.osd.5.smithi136.stdout:Sep 11 23:41:36 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-5[46773]: 2024-09-11T23:41:36.313+0000 7f5a20e77640 -1 osd.5 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:54.364889+0000 front 2024-09-11T23:40:54.364905+0000 (oldest deadline 2024-09-11T23:41:19.664838+0000) 2024-09-11T23:41:36.739 INFO:journalctl@ceph.osd.5.smithi136.stdout:Sep 11 23:41:36 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-5[46773]: 2024-09-11T23:41:36.313+0000 7f5a20e77640 -1 osd.5 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:04.965960+0000 front 2024-09-11T23:41:04.965997+0000 (oldest deadline 2024-09-11T23:41:28.466123+0000) 2024-09-11T23:41:36.739 INFO:journalctl@ceph.osd.5.smithi136.stdout:Sep 11 23:41:36 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-5[46773]: 2024-09-11T23:41:36.313+0000 7f5a20e77640 -1 osd.5 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:12.567008+0000 front 2024-09-11T23:41:12.567034+0000 (oldest deadline 2024-09-11T23:41:35.967615+0000) 2024-09-11T23:41:36.802 INFO:journalctl@ceph.osd.7.smithi076.stdout:Sep 11 23:41:36 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-7[52804]: 2024-09-11T23:41:36.387+0000 7f28fcc36640 -1 osd.7 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:05.616564+0000 front 2024-09-11T23:41:05.616590+0000 (oldest deadline 2024-09-11T23:41:31.516796+0000) 2024-09-11T23:41:36.802 INFO:journalctl@ceph.osd.7.smithi076.stdout:Sep 11 23:41:36 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-7[52804]: 2024-09-11T23:41:36.387+0000 7f28fcc36640 -1 osd.7 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:11.517294+0000 front 2024-09-11T23:41:11.517280+0000 (oldest deadline 2024-09-11T23:41:35.017284+0000) 2024-09-11T23:41:36.802 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:41:36 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:41:36.572+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:57.536934+0000 front 2024-09-11T23:40:57.537030+0000 (oldest deadline 2024-09-11T23:41:21.036908+0000) 2024-09-11T23:41:36.803 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:41:36 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:41:36.572+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:03.838149+0000 front 2024-09-11T23:41:03.838269+0000 (oldest deadline 2024-09-11T23:41:29.138274+0000) 2024-09-11T23:41:36.887 INFO:journalctl@ceph.osd.6.smithi026.stdout:Sep 11 23:41:36 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-6[58789]: 2024-09-11T23:41:36.514+0000 7f8321638640 -1 osd.6 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:57.112923+0000 front 2024-09-11T23:40:57.113173+0000 (oldest deadline 2024-09-11T23:41:18.813188+0000) 2024-09-11T23:41:36.887 INFO:journalctl@ceph.osd.6.smithi026.stdout:Sep 11 23:41:36 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-6[58789]: 2024-09-11T23:41:36.514+0000 7f8321638640 -1 osd.6 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:05.814845+0000 front 2024-09-11T23:41:05.814472+0000 (oldest deadline 2024-09-11T23:41:31.114608+0000) 2024-09-11T23:41:36.888 INFO:journalctl@ceph.osd.6.smithi026.stdout:Sep 11 23:41:36 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-6[58789]: 2024-09-11T23:41:36.514+0000 7f8321638640 -1 osd.6 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:11.115165+0000 front 2024-09-11T23:41:11.115230+0000 (oldest deadline 2024-09-11T23:41:35.215334+0000) 2024-09-11T23:41:37.238 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 11 23:41:36 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-8[53569]: 2024-09-11T23:41:36.850+0000 7f99e4062640 -1 osd.8 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:53.978855+0000 front 2024-09-11T23:40:53.978923+0000 (oldest deadline 2024-09-11T23:41:18.679037+0000) 2024-09-11T23:41:37.250 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:41:36 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9[65389]: 2024-09-11T23:41:36.888+0000 7fc7513f1640 -1 osd.9 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:57.370068+0000 front 2024-09-11T23:40:57.370191+0000 (oldest deadline 2024-09-11T23:41:21.470122+0000) 2024-09-11T23:41:37.250 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:41:36 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9[65389]: 2024-09-11T23:41:36.888+0000 7fc7513f1640 -1 osd.9 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:04.971207+0000 front 2024-09-11T23:41:04.971047+0000 (oldest deadline 2024-09-11T23:41:27.271258+0000) 2024-09-11T23:41:37.738 INFO:journalctl@ceph.osd.5.smithi136.stdout:Sep 11 23:41:37 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-5[46773]: 2024-09-11T23:41:37.345+0000 7f5a20e77640 -1 osd.5 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:54.364889+0000 front 2024-09-11T23:40:54.364905+0000 (oldest deadline 2024-09-11T23:41:19.664838+0000) 2024-09-11T23:41:37.739 INFO:journalctl@ceph.osd.5.smithi136.stdout:Sep 11 23:41:37 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-5[46773]: 2024-09-11T23:41:37.345+0000 7f5a20e77640 -1 osd.5 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:04.965960+0000 front 2024-09-11T23:41:04.965997+0000 (oldest deadline 2024-09-11T23:41:28.466123+0000) 2024-09-11T23:41:37.739 INFO:journalctl@ceph.osd.5.smithi136.stdout:Sep 11 23:41:37 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-5[46773]: 2024-09-11T23:41:37.345+0000 7f5a20e77640 -1 osd.5 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:12.567008+0000 front 2024-09-11T23:41:12.567034+0000 (oldest deadline 2024-09-11T23:41:35.967615+0000) 2024-09-11T23:41:37.739 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:41:37 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:41:37.451+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:56.932175+0000 front 2024-09-11T23:40:56.932499+0000 (oldest deadline 2024-09-11T23:41:21.032317+0000) 2024-09-11T23:41:37.739 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:41:37 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:41:37.451+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:01.032761+0000 front 2024-09-11T23:41:01.032638+0000 (oldest deadline 2024-09-11T23:41:30.433335+0000) 2024-09-11T23:41:37.739 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:41:37 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:41:37.451+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:10.934510+0000 front 2024-09-11T23:41:10.934411+0000 (oldest deadline 2024-09-11T23:41:35.634455+0000) 2024-09-11T23:41:37.802 INFO:journalctl@ceph.osd.7.smithi076.stdout:Sep 11 23:41:37 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-7[52804]: 2024-09-11T23:41:37.344+0000 7f28fcc36640 -1 osd.7 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:05.616564+0000 front 2024-09-11T23:41:05.616590+0000 (oldest deadline 2024-09-11T23:41:31.516796+0000) 2024-09-11T23:41:37.802 INFO:journalctl@ceph.osd.7.smithi076.stdout:Sep 11 23:41:37 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-7[52804]: 2024-09-11T23:41:37.344+0000 7f28fcc36640 -1 osd.7 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:11.517294+0000 front 2024-09-11T23:41:11.517280+0000 (oldest deadline 2024-09-11T23:41:35.017284+0000) 2024-09-11T23:41:37.803 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:41:37 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:41:37.541+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:57.536934+0000 front 2024-09-11T23:40:57.537030+0000 (oldest deadline 2024-09-11T23:41:21.036908+0000) 2024-09-11T23:41:37.803 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:41:37 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:41:37.541+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:03.838149+0000 front 2024-09-11T23:41:03.838269+0000 (oldest deadline 2024-09-11T23:41:29.138274+0000) 2024-09-11T23:41:37.803 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:41:37 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:41:37.541+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:12.539763+0000 front 2024-09-11T23:41:12.539893+0000 (oldest deadline 2024-09-11T23:41:36.639808+0000) 2024-09-11T23:41:37.865 INFO:journalctl@ceph.osd.6.smithi026.stdout:Sep 11 23:41:37 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-6[58789]: 2024-09-11T23:41:37.530+0000 7f8321638640 -1 osd.6 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:57.112923+0000 front 2024-09-11T23:40:57.113173+0000 (oldest deadline 2024-09-11T23:41:18.813188+0000) 2024-09-11T23:41:37.865 INFO:journalctl@ceph.osd.6.smithi026.stdout:Sep 11 23:41:37 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-6[58789]: 2024-09-11T23:41:37.530+0000 7f8321638640 -1 osd.6 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:05.814845+0000 front 2024-09-11T23:41:05.814472+0000 (oldest deadline 2024-09-11T23:41:31.114608+0000) 2024-09-11T23:41:37.866 INFO:journalctl@ceph.osd.6.smithi026.stdout:Sep 11 23:41:37 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-6[58789]: 2024-09-11T23:41:37.530+0000 7f8321638640 -1 osd.6 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:11.115165+0000 front 2024-09-11T23:41:11.115230+0000 (oldest deadline 2024-09-11T23:41:35.215334+0000) 2024-09-11T23:41:38.238 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 11 23:41:37 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-8[53569]: 2024-09-11T23:41:37.826+0000 7f99e4062640 -1 osd.8 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:53.978855+0000 front 2024-09-11T23:40:53.978923+0000 (oldest deadline 2024-09-11T23:41:18.679037+0000) 2024-09-11T23:41:38.250 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:41:37 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9[65389]: 2024-09-11T23:41:37.866+0000 7fc7513f1640 -1 osd.9 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:57.370068+0000 front 2024-09-11T23:40:57.370191+0000 (oldest deadline 2024-09-11T23:41:21.470122+0000) 2024-09-11T23:41:38.250 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:41:37 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9[65389]: 2024-09-11T23:41:37.866+0000 7fc7513f1640 -1 osd.9 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:04.971207+0000 front 2024-09-11T23:41:04.971047+0000 (oldest deadline 2024-09-11T23:41:27.271258+0000) 2024-09-11T23:41:38.739 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:41:38 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:41:38.446+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:56.932175+0000 front 2024-09-11T23:40:56.932499+0000 (oldest deadline 2024-09-11T23:41:21.032317+0000) 2024-09-11T23:41:38.739 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:41:38 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:41:38.446+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:01.032761+0000 front 2024-09-11T23:41:01.032638+0000 (oldest deadline 2024-09-11T23:41:30.433335+0000) 2024-09-11T23:41:38.739 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:41:38 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:41:38.446+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:10.934510+0000 front 2024-09-11T23:41:10.934411+0000 (oldest deadline 2024-09-11T23:41:35.634455+0000) 2024-09-11T23:41:38.802 INFO:journalctl@ceph.osd.7.smithi076.stdout:Sep 11 23:41:38 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-7[52804]: 2024-09-11T23:41:38.350+0000 7f28fcc36640 -1 osd.7 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:05.616564+0000 front 2024-09-11T23:41:05.616590+0000 (oldest deadline 2024-09-11T23:41:31.516796+0000) 2024-09-11T23:41:38.802 INFO:journalctl@ceph.osd.7.smithi076.stdout:Sep 11 23:41:38 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-7[52804]: 2024-09-11T23:41:38.350+0000 7f28fcc36640 -1 osd.7 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:11.517294+0000 front 2024-09-11T23:41:11.517280+0000 (oldest deadline 2024-09-11T23:41:35.017284+0000) 2024-09-11T23:41:38.803 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:41:38 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:41:38.511+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:57.536934+0000 front 2024-09-11T23:40:57.537030+0000 (oldest deadline 2024-09-11T23:41:21.036908+0000) 2024-09-11T23:41:38.803 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:41:38 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:41:38.511+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:03.838149+0000 front 2024-09-11T23:41:03.838269+0000 (oldest deadline 2024-09-11T23:41:29.138274+0000) 2024-09-11T23:41:38.803 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:41:38 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:41:38.511+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:12.539763+0000 front 2024-09-11T23:41:12.539893+0000 (oldest deadline 2024-09-11T23:41:36.639808+0000) 2024-09-11T23:41:38.865 INFO:journalctl@ceph.osd.6.smithi026.stdout:Sep 11 23:41:38 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-6[58789]: 2024-09-11T23:41:38.540+0000 7f8321638640 -1 osd.6 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:57.112923+0000 front 2024-09-11T23:40:57.113173+0000 (oldest deadline 2024-09-11T23:41:18.813188+0000) 2024-09-11T23:41:38.865 INFO:journalctl@ceph.osd.6.smithi026.stdout:Sep 11 23:41:38 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-6[58789]: 2024-09-11T23:41:38.540+0000 7f8321638640 -1 osd.6 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:05.814845+0000 front 2024-09-11T23:41:05.814472+0000 (oldest deadline 2024-09-11T23:41:31.114608+0000) 2024-09-11T23:41:38.865 INFO:journalctl@ceph.osd.6.smithi026.stdout:Sep 11 23:41:38 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-6[58789]: 2024-09-11T23:41:38.540+0000 7f8321638640 -1 osd.6 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:11.115165+0000 front 2024-09-11T23:41:11.115230+0000 (oldest deadline 2024-09-11T23:41:35.215334+0000) 2024-09-11T23:41:39.093 INFO:journalctl@ceph.osd.5.smithi136.stdout:Sep 11 23:41:38 smithi136 podman[73798]: 2024-09-11 23:41:38.818679611 +0000 UTC m=+5.750757826 container died da022b14d4c2fdb4d91fca7fe09cc49d8f7673d6ac2a593e6d5eb8788ee9e577 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-5, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-09-11T23:41:39.093 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 11 23:41:38 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-8[53569]: 2024-09-11T23:41:38.797+0000 7f99e4062640 -1 osd.8 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:53.978855+0000 front 2024-09-11T23:40:53.978923+0000 (oldest deadline 2024-09-11T23:41:18.679037+0000) 2024-09-11T23:41:39.250 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:41:38 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9[65389]: 2024-09-11T23:41:38.865+0000 7fc7513f1640 -1 osd.9 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:57.370068+0000 front 2024-09-11T23:40:57.370191+0000 (oldest deadline 2024-09-11T23:41:21.470122+0000) 2024-09-11T23:41:39.250 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:41:38 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9[65389]: 2024-09-11T23:41:38.865+0000 7fc7513f1640 -1 osd.9 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:04.971207+0000 front 2024-09-11T23:41:04.971047+0000 (oldest deadline 2024-09-11T23:41:27.271258+0000) 2024-09-11T23:41:39.250 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:41:38 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9[65389]: 2024-09-11T23:41:38.865+0000 7fc7513f1640 -1 osd.9 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:13.974640+0000 front 2024-09-11T23:41:13.974576+0000 (oldest deadline 2024-09-11T23:41:38.074690+0000) 2024-09-11T23:41:39.449 INFO:journalctl@ceph.osd.5.smithi136.stdout:Sep 11 23:41:39 smithi136 podman[73798]: 2024-09-11 23:41:39.194105443 +0000 UTC m=+6.126183704 container cleanup da022b14d4c2fdb4d91fca7fe09cc49d8f7673d6ac2a593e6d5eb8788ee9e577 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=squid-b53b6d3, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-09-11T23:41:39.449 INFO:journalctl@ceph.osd.5.smithi136.stdout:Sep 11 23:41:39 smithi136 bash[73798]: ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-5 2024-09-11T23:41:39.738 INFO:journalctl@ceph.osd.5.smithi136.stdout:Sep 11 23:41:39 smithi136 podman[73813]: 2024-09-11 23:41:39.453659304 +0000 UTC m=+0.626970646 container remove da022b14d4c2fdb4d91fca7fe09cc49d8f7673d6ac2a593e6d5eb8788ee9e577 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-5, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-09-11T23:41:39.739 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:41:39 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:41:39.449+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:56.932175+0000 front 2024-09-11T23:40:56.932499+0000 (oldest deadline 2024-09-11T23:41:21.032317+0000) 2024-09-11T23:41:39.739 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:41:39 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:41:39.449+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:01.032761+0000 front 2024-09-11T23:41:01.032638+0000 (oldest deadline 2024-09-11T23:41:30.433335+0000) 2024-09-11T23:41:39.739 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:41:39 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:41:39.449+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:10.934510+0000 front 2024-09-11T23:41:10.934411+0000 (oldest deadline 2024-09-11T23:41:35.634455+0000) 2024-09-11T23:41:39.802 INFO:journalctl@ceph.osd.7.smithi076.stdout:Sep 11 23:41:39 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-7[52804]: 2024-09-11T23:41:39.304+0000 7f28fcc36640 -1 osd.7 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:05.616564+0000 front 2024-09-11T23:41:05.616590+0000 (oldest deadline 2024-09-11T23:41:31.516796+0000) 2024-09-11T23:41:39.802 INFO:journalctl@ceph.osd.7.smithi076.stdout:Sep 11 23:41:39 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-7[52804]: 2024-09-11T23:41:39.304+0000 7f28fcc36640 -1 osd.7 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:11.517294+0000 front 2024-09-11T23:41:11.517280+0000 (oldest deadline 2024-09-11T23:41:35.017284+0000) 2024-09-11T23:41:39.802 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:41:39 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:41:39.545+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:57.536934+0000 front 2024-09-11T23:40:57.537030+0000 (oldest deadline 2024-09-11T23:41:21.036908+0000) 2024-09-11T23:41:39.802 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:41:39 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:41:39.545+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:03.838149+0000 front 2024-09-11T23:41:03.838269+0000 (oldest deadline 2024-09-11T23:41:29.138274+0000) 2024-09-11T23:41:39.803 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:41:39 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:41:39.545+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:12.539763+0000 front 2024-09-11T23:41:12.539893+0000 (oldest deadline 2024-09-11T23:41:36.639808+0000) 2024-09-11T23:41:39.882 INFO:journalctl@ceph.osd.6.smithi026.stdout:Sep 11 23:41:39 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-6[58789]: 2024-09-11T23:41:39.536+0000 7f8321638640 -1 osd.6 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:57.112923+0000 front 2024-09-11T23:40:57.113173+0000 (oldest deadline 2024-09-11T23:41:18.813188+0000) 2024-09-11T23:41:39.882 INFO:journalctl@ceph.osd.6.smithi026.stdout:Sep 11 23:41:39 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-6[58789]: 2024-09-11T23:41:39.536+0000 7f8321638640 -1 osd.6 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:05.814845+0000 front 2024-09-11T23:41:05.814472+0000 (oldest deadline 2024-09-11T23:41:31.114608+0000) 2024-09-11T23:41:39.882 INFO:journalctl@ceph.osd.6.smithi026.stdout:Sep 11 23:41:39 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-6[58789]: 2024-09-11T23:41:39.536+0000 7f8321638640 -1 osd.6 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:11.115165+0000 front 2024-09-11T23:41:11.115230+0000 (oldest deadline 2024-09-11T23:41:35.215334+0000) 2024-09-11T23:41:40.014 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 11 23:41:39 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-8[53569]: 2024-09-11T23:41:39.749+0000 7f99e4062640 -1 osd.8 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:53.978855+0000 front 2024-09-11T23:40:53.978923+0000 (oldest deadline 2024-09-11T23:41:18.679037+0000) 2024-09-11T23:41:40.014 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 11 23:41:39 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-8[53569]: 2024-09-11T23:41:39.749+0000 7f99e4062640 -1 osd.8 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:10.281739+0000 front 2024-09-11T23:41:10.281758+0000 (oldest deadline 2024-09-11T23:41:39.682327+0000) 2024-09-11T23:41:40.014 INFO:journalctl@ceph.osd.5.smithi136.stdout:Sep 11 23:41:39 smithi136 podman[73902]: 2024-09-11 23:41:39.665459477 +0000 UTC m=+0.015808850 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T23:41:40.015 INFO:journalctl@ceph.osd.5.smithi136.stdout:Sep 11 23:41:39 smithi136 podman[73902]: 2024-09-11 23:41:39.77954763 +0000 UTC m=+0.129896991 container create 912fe24df038ac3c813b848106a7b7955d2e9d5057193b689194d8721d2e153a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-5-deactivate, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3) 2024-09-11T23:41:40.250 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:41:39 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9[65389]: 2024-09-11T23:41:39.882+0000 7fc7513f1640 -1 osd.9 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:57.370068+0000 front 2024-09-11T23:40:57.370191+0000 (oldest deadline 2024-09-11T23:41:21.470122+0000) 2024-09-11T23:41:40.251 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:41:39 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9[65389]: 2024-09-11T23:41:39.882+0000 7fc7513f1640 -1 osd.9 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:04.971207+0000 front 2024-09-11T23:41:04.971047+0000 (oldest deadline 2024-09-11T23:41:27.271258+0000) 2024-09-11T23:41:40.251 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:41:39 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9[65389]: 2024-09-11T23:41:39.882+0000 7fc7513f1640 -1 osd.9 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:13.974640+0000 front 2024-09-11T23:41:13.974576+0000 (oldest deadline 2024-09-11T23:41:38.074690+0000) 2024-09-11T23:41:40.462 INFO:journalctl@ceph.osd.5.smithi136.stdout:Sep 11 23:41:40 smithi136 podman[73902]: 2024-09-11 23:41:40.015137412 +0000 UTC m=+0.365486775 container init 912fe24df038ac3c813b848106a7b7955d2e9d5057193b689194d8721d2e153a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-5-deactivate, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, ceph=True) 2024-09-11T23:41:40.462 INFO:journalctl@ceph.osd.5.smithi136.stdout:Sep 11 23:41:40 smithi136 podman[73902]: 2024-09-11 23:41:40.018408312 +0000 UTC m=+0.368757681 container start 912fe24df038ac3c813b848106a7b7955d2e9d5057193b689194d8721d2e153a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-5-deactivate, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T23:41:40.462 INFO:journalctl@ceph.osd.5.smithi136.stdout:Sep 11 23:41:40 smithi136 podman[73902]: 2024-09-11 23:41:40.098556035 +0000 UTC m=+0.448905397 container attach 912fe24df038ac3c813b848106a7b7955d2e9d5057193b689194d8721d2e153a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-5-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2) 2024-09-11T23:41:40.462 INFO:journalctl@ceph.osd.5.smithi136.stdout:Sep 11 23:41:40 smithi136 podman[73902]: 2024-09-11 23:41:40.204113337 +0000 UTC m=+0.554462710 container died 912fe24df038ac3c813b848106a7b7955d2e9d5057193b689194d8721d2e153a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-5-deactivate, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=) 2024-09-11T23:41:40.737 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:41:40 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:41:40.462+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:56.932175+0000 front 2024-09-11T23:40:56.932499+0000 (oldest deadline 2024-09-11T23:41:21.032317+0000) 2024-09-11T23:41:40.738 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:41:40 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:41:40.462+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:01.032761+0000 front 2024-09-11T23:41:01.032638+0000 (oldest deadline 2024-09-11T23:41:30.433335+0000) 2024-09-11T23:41:40.738 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:41:40 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:41:40.462+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:10.934510+0000 front 2024-09-11T23:41:10.934411+0000 (oldest deadline 2024-09-11T23:41:35.634455+0000) 2024-09-11T23:41:40.765 DEBUG:teuthology.orchestra.run.smithi136:> sudo pkill -f 'journalctl -f -n 0 -u ceph-af7e6fc8-7092-11ef-bcea-c7b262605968@osd.5.service' 2024-09-11T23:41:40.802 INFO:journalctl@ceph.osd.7.smithi076.stdout:Sep 11 23:41:40 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-7[52804]: 2024-09-11T23:41:40.338+0000 7f28fcc36640 -1 osd.7 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:05.616564+0000 front 2024-09-11T23:41:05.616590+0000 (oldest deadline 2024-09-11T23:41:31.516796+0000) 2024-09-11T23:41:40.802 INFO:journalctl@ceph.osd.7.smithi076.stdout:Sep 11 23:41:40 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-7[52804]: 2024-09-11T23:41:40.338+0000 7f28fcc36640 -1 osd.7 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:11.517294+0000 front 2024-09-11T23:41:11.517280+0000 (oldest deadline 2024-09-11T23:41:35.017284+0000) 2024-09-11T23:41:40.802 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:41:40 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:41:40.558+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:57.536934+0000 front 2024-09-11T23:40:57.537030+0000 (oldest deadline 2024-09-11T23:41:21.036908+0000) 2024-09-11T23:41:40.802 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:41:40 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:41:40.558+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:03.838149+0000 front 2024-09-11T23:41:03.838269+0000 (oldest deadline 2024-09-11T23:41:29.138274+0000) 2024-09-11T23:41:40.802 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:41:40 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:41:40.558+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:12.539763+0000 front 2024-09-11T23:41:12.539893+0000 (oldest deadline 2024-09-11T23:41:36.639808+0000) 2024-09-11T23:41:40.804 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T23:41:40.804 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-09-11T23:41:40.804 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-09-11T23:41:40.804 DEBUG:teuthology.orchestra.run.smithi026:> sudo systemctl stop ceph-af7e6fc8-7092-11ef-bcea-c7b262605968@osd.6 2024-09-11T23:41:40.833 INFO:journalctl@ceph.osd.6.smithi026.stdout:Sep 11 23:41:40 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-6[58789]: 2024-09-11T23:41:40.511+0000 7f8321638640 -1 osd.6 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:57.112923+0000 front 2024-09-11T23:40:57.113173+0000 (oldest deadline 2024-09-11T23:41:18.813188+0000) 2024-09-11T23:41:40.833 INFO:journalctl@ceph.osd.6.smithi026.stdout:Sep 11 23:41:40 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-6[58789]: 2024-09-11T23:41:40.511+0000 7f8321638640 -1 osd.6 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:05.814845+0000 front 2024-09-11T23:41:05.814472+0000 (oldest deadline 2024-09-11T23:41:31.114608+0000) 2024-09-11T23:41:40.833 INFO:journalctl@ceph.osd.6.smithi026.stdout:Sep 11 23:41:40 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-6[58789]: 2024-09-11T23:41:40.511+0000 7f8321638640 -1 osd.6 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:11.115165+0000 front 2024-09-11T23:41:11.115230+0000 (oldest deadline 2024-09-11T23:41:35.215334+0000) 2024-09-11T23:41:40.988 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 11 23:41:40 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-8[53569]: 2024-09-11T23:41:40.737+0000 7f99e4062640 -1 osd.8 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:53.978855+0000 front 2024-09-11T23:40:53.978923+0000 (oldest deadline 2024-09-11T23:41:18.679037+0000) 2024-09-11T23:41:40.989 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 11 23:41:40 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-8[53569]: 2024-09-11T23:41:40.737+0000 7f99e4062640 -1 osd.8 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:10.281739+0000 front 2024-09-11T23:41:10.281758+0000 (oldest deadline 2024-09-11T23:41:39.682327+0000) 2024-09-11T23:41:41.115 INFO:journalctl@ceph.osd.6.smithi026.stdout:Sep 11 23:41:40 smithi026 systemd[1]: Stopping Ceph osd.6 for af7e6fc8-7092-11ef-bcea-c7b262605968... 2024-09-11T23:41:41.115 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:41:40 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9[65389]: 2024-09-11T23:41:40.864+0000 7fc7513f1640 -1 osd.9 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:57.370068+0000 front 2024-09-11T23:40:57.370191+0000 (oldest deadline 2024-09-11T23:41:21.470122+0000) 2024-09-11T23:41:41.115 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:41:40 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9[65389]: 2024-09-11T23:41:40.864+0000 7fc7513f1640 -1 osd.9 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:04.971207+0000 front 2024-09-11T23:41:04.971047+0000 (oldest deadline 2024-09-11T23:41:27.271258+0000) 2024-09-11T23:41:41.115 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:41:40 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9[65389]: 2024-09-11T23:41:40.864+0000 7fc7513f1640 -1 osd.9 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:13.974640+0000 front 2024-09-11T23:41:13.974576+0000 (oldest deadline 2024-09-11T23:41:38.074690+0000) 2024-09-11T23:41:41.490 INFO:journalctl@ceph.osd.6.smithi026.stdout:Sep 11 23:41:41 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-6[58789]: 2024-09-11T23:41:41.115+0000 7f832903b640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T23:41:41.491 INFO:journalctl@ceph.osd.6.smithi026.stdout:Sep 11 23:41:41 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-6[58789]: 2024-09-11T23:41:41.115+0000 7f832903b640 -1 osd.6 120 *** Got signal Terminated *** 2024-09-11T23:41:41.491 INFO:journalctl@ceph.osd.6.smithi026.stdout:Sep 11 23:41:41 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-6[58789]: 2024-09-11T23:41:41.115+0000 7f832903b640 -1 osd.6 120 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T23:41:41.491 INFO:journalctl@ceph.osd.6.smithi026.stdout:Sep 11 23:41:41 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-6[58789]: 2024-09-11T23:41:41.491+0000 7f8321638640 -1 osd.6 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:57.112923+0000 front 2024-09-11T23:40:57.113173+0000 (oldest deadline 2024-09-11T23:41:18.813188+0000) 2024-09-11T23:41:41.491 INFO:journalctl@ceph.osd.6.smithi026.stdout:Sep 11 23:41:41 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-6[58789]: 2024-09-11T23:41:41.491+0000 7f8321638640 -1 osd.6 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:05.814845+0000 front 2024-09-11T23:41:05.814472+0000 (oldest deadline 2024-09-11T23:41:31.114608+0000) 2024-09-11T23:41:41.491 INFO:journalctl@ceph.osd.6.smithi026.stdout:Sep 11 23:41:41 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-6[58789]: 2024-09-11T23:41:41.491+0000 7f8321638640 -1 osd.6 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:11.115165+0000 front 2024-09-11T23:41:11.115230+0000 (oldest deadline 2024-09-11T23:41:35.215334+0000) 2024-09-11T23:41:41.738 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:41:41 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:41:41.481+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:56.932175+0000 front 2024-09-11T23:40:56.932499+0000 (oldest deadline 2024-09-11T23:41:21.032317+0000) 2024-09-11T23:41:41.739 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:41:41 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:41:41.481+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:01.032761+0000 front 2024-09-11T23:41:01.032638+0000 (oldest deadline 2024-09-11T23:41:30.433335+0000) 2024-09-11T23:41:41.739 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:41:41 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:41:41.481+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:10.934510+0000 front 2024-09-11T23:41:10.934411+0000 (oldest deadline 2024-09-11T23:41:35.634455+0000) 2024-09-11T23:41:41.739 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 11 23:41:41 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-8[53569]: 2024-09-11T23:41:41.720+0000 7f99e4062640 -1 osd.8 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:53.978855+0000 front 2024-09-11T23:40:53.978923+0000 (oldest deadline 2024-09-11T23:41:18.679037+0000) 2024-09-11T23:41:41.739 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 11 23:41:41 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-8[53569]: 2024-09-11T23:41:41.720+0000 7f99e4062640 -1 osd.8 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:10.281739+0000 front 2024-09-11T23:41:10.281758+0000 (oldest deadline 2024-09-11T23:41:39.682327+0000) 2024-09-11T23:41:41.802 INFO:journalctl@ceph.osd.7.smithi076.stdout:Sep 11 23:41:41 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-7[52804]: 2024-09-11T23:41:41.333+0000 7f28fcc36640 -1 osd.7 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:05.616564+0000 front 2024-09-11T23:41:05.616590+0000 (oldest deadline 2024-09-11T23:41:31.516796+0000) 2024-09-11T23:41:41.802 INFO:journalctl@ceph.osd.7.smithi076.stdout:Sep 11 23:41:41 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-7[52804]: 2024-09-11T23:41:41.333+0000 7f28fcc36640 -1 osd.7 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:11.517294+0000 front 2024-09-11T23:41:11.517280+0000 (oldest deadline 2024-09-11T23:41:35.017284+0000) 2024-09-11T23:41:41.802 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:41:41 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:41:41.546+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:57.536934+0000 front 2024-09-11T23:40:57.537030+0000 (oldest deadline 2024-09-11T23:41:21.036908+0000) 2024-09-11T23:41:41.803 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:41:41 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:41:41.546+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:03.838149+0000 front 2024-09-11T23:41:03.838269+0000 (oldest deadline 2024-09-11T23:41:29.138274+0000) 2024-09-11T23:41:41.803 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:41:41 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:41:41.546+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:12.539763+0000 front 2024-09-11T23:41:12.539893+0000 (oldest deadline 2024-09-11T23:41:36.639808+0000) 2024-09-11T23:41:42.131 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:41:41 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9[65389]: 2024-09-11T23:41:41.880+0000 7fc7513f1640 -1 osd.9 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:57.370068+0000 front 2024-09-11T23:40:57.370191+0000 (oldest deadline 2024-09-11T23:41:21.470122+0000) 2024-09-11T23:41:42.132 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:41:41 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9[65389]: 2024-09-11T23:41:41.880+0000 7fc7513f1640 -1 osd.9 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:04.971207+0000 front 2024-09-11T23:41:04.971047+0000 (oldest deadline 2024-09-11T23:41:27.271258+0000) 2024-09-11T23:41:42.132 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:41:41 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9[65389]: 2024-09-11T23:41:41.880+0000 7fc7513f1640 -1 osd.9 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:13.974640+0000 front 2024-09-11T23:41:13.974576+0000 (oldest deadline 2024-09-11T23:41:38.074690+0000) 2024-09-11T23:41:42.552 INFO:journalctl@ceph.osd.7.smithi076.stdout:Sep 11 23:41:42 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-7[52804]: 2024-09-11T23:41:42.318+0000 7f28fcc36640 -1 osd.7 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:05.616564+0000 front 2024-09-11T23:41:05.616590+0000 (oldest deadline 2024-09-11T23:41:31.516796+0000) 2024-09-11T23:41:42.552 INFO:journalctl@ceph.osd.7.smithi076.stdout:Sep 11 23:41:42 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-7[52804]: 2024-09-11T23:41:42.318+0000 7f28fcc36640 -1 osd.7 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:11.517294+0000 front 2024-09-11T23:41:11.517280+0000 (oldest deadline 2024-09-11T23:41:35.017284+0000) 2024-09-11T23:41:42.738 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 11 23:41:42 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-8[53569]: 2024-09-11T23:41:42.725+0000 7f99e4062640 -1 osd.8 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:53.978855+0000 front 2024-09-11T23:40:53.978923+0000 (oldest deadline 2024-09-11T23:41:18.679037+0000) 2024-09-11T23:41:42.739 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 11 23:41:42 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-8[53569]: 2024-09-11T23:41:42.725+0000 7f99e4062640 -1 osd.8 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:10.281739+0000 front 2024-09-11T23:41:10.281758+0000 (oldest deadline 2024-09-11T23:41:39.682327+0000) 2024-09-11T23:41:42.739 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:41:42 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:41:42.481+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:56.932175+0000 front 2024-09-11T23:40:56.932499+0000 (oldest deadline 2024-09-11T23:41:21.032317+0000) 2024-09-11T23:41:42.739 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:41:42 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:41:42.481+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:01.032761+0000 front 2024-09-11T23:41:01.032638+0000 (oldest deadline 2024-09-11T23:41:30.433335+0000) 2024-09-11T23:41:42.739 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:41:42 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:41:42.481+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:10.934510+0000 front 2024-09-11T23:41:10.934411+0000 (oldest deadline 2024-09-11T23:41:35.634455+0000) 2024-09-11T23:41:42.750 INFO:journalctl@ceph.osd.6.smithi026.stdout:Sep 11 23:41:42 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-6[58789]: 2024-09-11T23:41:42.510+0000 7f8321638640 -1 osd.6 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:57.112923+0000 front 2024-09-11T23:40:57.113173+0000 (oldest deadline 2024-09-11T23:41:18.813188+0000) 2024-09-11T23:41:42.750 INFO:journalctl@ceph.osd.6.smithi026.stdout:Sep 11 23:41:42 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-6[58789]: 2024-09-11T23:41:42.510+0000 7f8321638640 -1 osd.6 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:05.814845+0000 front 2024-09-11T23:41:05.814472+0000 (oldest deadline 2024-09-11T23:41:31.114608+0000) 2024-09-11T23:41:42.750 INFO:journalctl@ceph.osd.6.smithi026.stdout:Sep 11 23:41:42 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-6[58789]: 2024-09-11T23:41:42.510+0000 7f8321638640 -1 osd.6 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:11.115165+0000 front 2024-09-11T23:41:11.115230+0000 (oldest deadline 2024-09-11T23:41:35.215334+0000) 2024-09-11T23:41:43.052 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:41:42 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:41:42.588+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:57.536934+0000 front 2024-09-11T23:40:57.537030+0000 (oldest deadline 2024-09-11T23:41:21.036908+0000) 2024-09-11T23:41:43.052 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:41:42 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:41:42.588+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:03.838149+0000 front 2024-09-11T23:41:03.838269+0000 (oldest deadline 2024-09-11T23:41:29.138274+0000) 2024-09-11T23:41:43.052 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:41:42 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:41:42.588+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:12.539763+0000 front 2024-09-11T23:41:12.539893+0000 (oldest deadline 2024-09-11T23:41:36.639808+0000) 2024-09-11T23:41:43.250 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:41:42 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9[65389]: 2024-09-11T23:41:42.853+0000 7fc7513f1640 -1 osd.9 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:57.370068+0000 front 2024-09-11T23:40:57.370191+0000 (oldest deadline 2024-09-11T23:41:21.470122+0000) 2024-09-11T23:41:43.250 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:41:42 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9[65389]: 2024-09-11T23:41:42.853+0000 7fc7513f1640 -1 osd.9 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:04.971207+0000 front 2024-09-11T23:41:04.971047+0000 (oldest deadline 2024-09-11T23:41:27.271258+0000) 2024-09-11T23:41:43.250 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:41:42 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9[65389]: 2024-09-11T23:41:42.853+0000 7fc7513f1640 -1 osd.9 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:13.974640+0000 front 2024-09-11T23:41:13.974576+0000 (oldest deadline 2024-09-11T23:41:38.074690+0000) 2024-09-11T23:41:43.552 INFO:journalctl@ceph.osd.7.smithi076.stdout:Sep 11 23:41:43 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-7[52804]: 2024-09-11T23:41:43.288+0000 7f28fcc36640 -1 osd.7 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:05.616564+0000 front 2024-09-11T23:41:05.616590+0000 (oldest deadline 2024-09-11T23:41:31.516796+0000) 2024-09-11T23:41:43.552 INFO:journalctl@ceph.osd.7.smithi076.stdout:Sep 11 23:41:43 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-7[52804]: 2024-09-11T23:41:43.288+0000 7f28fcc36640 -1 osd.7 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:11.517294+0000 front 2024-09-11T23:41:11.517280+0000 (oldest deadline 2024-09-11T23:41:35.017284+0000) 2024-09-11T23:41:43.750 INFO:journalctl@ceph.osd.6.smithi026.stdout:Sep 11 23:41:43 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-6[58789]: 2024-09-11T23:41:43.524+0000 7f8321638640 -1 osd.6 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:57.112923+0000 front 2024-09-11T23:40:57.113173+0000 (oldest deadline 2024-09-11T23:41:18.813188+0000) 2024-09-11T23:41:43.750 INFO:journalctl@ceph.osd.6.smithi026.stdout:Sep 11 23:41:43 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-6[58789]: 2024-09-11T23:41:43.524+0000 7f8321638640 -1 osd.6 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:05.814845+0000 front 2024-09-11T23:41:05.814472+0000 (oldest deadline 2024-09-11T23:41:31.114608+0000) 2024-09-11T23:41:43.750 INFO:journalctl@ceph.osd.6.smithi026.stdout:Sep 11 23:41:43 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-6[58789]: 2024-09-11T23:41:43.524+0000 7f8321638640 -1 osd.6 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:11.115165+0000 front 2024-09-11T23:41:11.115230+0000 (oldest deadline 2024-09-11T23:41:35.215334+0000) 2024-09-11T23:41:43.989 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 11 23:41:43 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-8[53569]: 2024-09-11T23:41:43.685+0000 7f99e4062640 -1 osd.8 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:53.978855+0000 front 2024-09-11T23:40:53.978923+0000 (oldest deadline 2024-09-11T23:41:18.679037+0000) 2024-09-11T23:41:43.989 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 11 23:41:43 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-8[53569]: 2024-09-11T23:41:43.685+0000 7f99e4062640 -1 osd.8 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:10.281739+0000 front 2024-09-11T23:41:10.281758+0000 (oldest deadline 2024-09-11T23:41:39.682327+0000) 2024-09-11T23:41:43.990 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 11 23:41:43 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-8[53569]: 2024-09-11T23:41:43.685+0000 7f99e4062640 -1 osd.8 120 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-09-11T23:41:19.683087+0000 front 2024-09-11T23:41:19.683104+0000 (oldest deadline 2024-09-11T23:41:43.182872+0000) 2024-09-11T23:41:43.990 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:41:43 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:41:43.498+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:56.932175+0000 front 2024-09-11T23:40:56.932499+0000 (oldest deadline 2024-09-11T23:41:21.032317+0000) 2024-09-11T23:41:43.990 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:41:43 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:41:43.498+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:01.032761+0000 front 2024-09-11T23:41:01.032638+0000 (oldest deadline 2024-09-11T23:41:30.433335+0000) 2024-09-11T23:41:43.990 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:41:43 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:41:43.498+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:10.934510+0000 front 2024-09-11T23:41:10.934411+0000 (oldest deadline 2024-09-11T23:41:35.634455+0000) 2024-09-11T23:41:43.990 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:41:43 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:41:43.498+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-09-11T23:41:21.936325+0000 front 2024-09-11T23:41:21.936462+0000 (oldest deadline 2024-09-11T23:41:42.937021+0000) 2024-09-11T23:41:44.052 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:41:43 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:41:43.610+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:57.536934+0000 front 2024-09-11T23:40:57.537030+0000 (oldest deadline 2024-09-11T23:41:21.036908+0000) 2024-09-11T23:41:44.052 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:41:43 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:41:43.610+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:03.838149+0000 front 2024-09-11T23:41:03.838269+0000 (oldest deadline 2024-09-11T23:41:29.138274+0000) 2024-09-11T23:41:44.052 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:41:43 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:41:43.610+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:12.539763+0000 front 2024-09-11T23:41:12.539893+0000 (oldest deadline 2024-09-11T23:41:36.639808+0000) 2024-09-11T23:41:44.250 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:41:43 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9[65389]: 2024-09-11T23:41:43.849+0000 7fc7513f1640 -1 osd.9 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:57.370068+0000 front 2024-09-11T23:40:57.370191+0000 (oldest deadline 2024-09-11T23:41:21.470122+0000) 2024-09-11T23:41:44.250 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:41:43 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9[65389]: 2024-09-11T23:41:43.849+0000 7fc7513f1640 -1 osd.9 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:04.971207+0000 front 2024-09-11T23:41:04.971047+0000 (oldest deadline 2024-09-11T23:41:27.271258+0000) 2024-09-11T23:41:44.250 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:41:43 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9[65389]: 2024-09-11T23:41:43.849+0000 7fc7513f1640 -1 osd.9 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:13.974640+0000 front 2024-09-11T23:41:13.974576+0000 (oldest deadline 2024-09-11T23:41:38.074690+0000) 2024-09-11T23:41:44.251 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:41:43 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9[65389]: 2024-09-11T23:41:43.849+0000 7fc7513f1640 -1 osd.9 120 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-09-11T23:41:18.075080+0000 front 2024-09-11T23:41:18.075257+0000 (oldest deadline 2024-09-11T23:41:43.375370+0000) 2024-09-11T23:41:44.552 INFO:journalctl@ceph.osd.7.smithi076.stdout:Sep 11 23:41:44 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-7[52804]: 2024-09-11T23:41:44.249+0000 7f28fcc36640 -1 osd.7 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:05.616564+0000 front 2024-09-11T23:41:05.616590+0000 (oldest deadline 2024-09-11T23:41:31.516796+0000) 2024-09-11T23:41:44.552 INFO:journalctl@ceph.osd.7.smithi076.stdout:Sep 11 23:41:44 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-7[52804]: 2024-09-11T23:41:44.249+0000 7f28fcc36640 -1 osd.7 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:11.517294+0000 front 2024-09-11T23:41:11.517280+0000 (oldest deadline 2024-09-11T23:41:35.017284+0000) 2024-09-11T23:41:44.732 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:41:44 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:41:44.471+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:56.932175+0000 front 2024-09-11T23:40:56.932499+0000 (oldest deadline 2024-09-11T23:41:21.032317+0000) 2024-09-11T23:41:44.732 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:41:44 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:41:44.471+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:01.032761+0000 front 2024-09-11T23:41:01.032638+0000 (oldest deadline 2024-09-11T23:41:30.433335+0000) 2024-09-11T23:41:44.732 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:41:44 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:41:44.471+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:10.934510+0000 front 2024-09-11T23:41:10.934411+0000 (oldest deadline 2024-09-11T23:41:35.634455+0000) 2024-09-11T23:41:44.733 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:41:44 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:41:44.471+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-09-11T23:41:21.936325+0000 front 2024-09-11T23:41:21.936462+0000 (oldest deadline 2024-09-11T23:41:42.937021+0000) 2024-09-11T23:41:44.872 INFO:journalctl@ceph.osd.6.smithi026.stdout:Sep 11 23:41:44 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-6[58789]: 2024-09-11T23:41:44.545+0000 7f8321638640 -1 osd.6 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:57.112923+0000 front 2024-09-11T23:40:57.113173+0000 (oldest deadline 2024-09-11T23:41:18.813188+0000) 2024-09-11T23:41:44.872 INFO:journalctl@ceph.osd.6.smithi026.stdout:Sep 11 23:41:44 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-6[58789]: 2024-09-11T23:41:44.545+0000 7f8321638640 -1 osd.6 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:05.814845+0000 front 2024-09-11T23:41:05.814472+0000 (oldest deadline 2024-09-11T23:41:31.114608+0000) 2024-09-11T23:41:44.872 INFO:journalctl@ceph.osd.6.smithi026.stdout:Sep 11 23:41:44 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-6[58789]: 2024-09-11T23:41:44.545+0000 7f8321638640 -1 osd.6 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:11.115165+0000 front 2024-09-11T23:41:11.115230+0000 (oldest deadline 2024-09-11T23:41:35.215334+0000) 2024-09-11T23:41:44.988 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 11 23:41:44 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-8[53569]: 2024-09-11T23:41:44.732+0000 7f99e4062640 -1 osd.8 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:53.978855+0000 front 2024-09-11T23:40:53.978923+0000 (oldest deadline 2024-09-11T23:41:18.679037+0000) 2024-09-11T23:41:44.989 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 11 23:41:44 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-8[53569]: 2024-09-11T23:41:44.732+0000 7f99e4062640 -1 osd.8 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:10.281739+0000 front 2024-09-11T23:41:10.281758+0000 (oldest deadline 2024-09-11T23:41:39.682327+0000) 2024-09-11T23:41:44.989 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 11 23:41:44 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-8[53569]: 2024-09-11T23:41:44.732+0000 7f99e4062640 -1 osd.8 120 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-09-11T23:41:19.683087+0000 front 2024-09-11T23:41:19.683104+0000 (oldest deadline 2024-09-11T23:41:43.182872+0000) 2024-09-11T23:41:45.052 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:41:44 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:41:44.632+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:57.536934+0000 front 2024-09-11T23:40:57.537030+0000 (oldest deadline 2024-09-11T23:41:21.036908+0000) 2024-09-11T23:41:45.053 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:41:44 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:41:44.632+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:03.838149+0000 front 2024-09-11T23:41:03.838269+0000 (oldest deadline 2024-09-11T23:41:29.138274+0000) 2024-09-11T23:41:45.053 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:41:44 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:41:44.632+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:12.539763+0000 front 2024-09-11T23:41:12.539893+0000 (oldest deadline 2024-09-11T23:41:36.639808+0000) 2024-09-11T23:41:45.053 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:41:44 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:41:44.632+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-09-11T23:41:21.841541+0000 front 2024-09-11T23:41:21.841518+0000 (oldest deadline 2024-09-11T23:41:44.141586+0000) 2024-09-11T23:41:45.250 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:41:44 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9[65389]: 2024-09-11T23:41:44.873+0000 7fc7513f1640 -1 osd.9 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:57.370068+0000 front 2024-09-11T23:40:57.370191+0000 (oldest deadline 2024-09-11T23:41:21.470122+0000) 2024-09-11T23:41:45.250 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:41:44 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9[65389]: 2024-09-11T23:41:44.873+0000 7fc7513f1640 -1 osd.9 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:04.971207+0000 front 2024-09-11T23:41:04.971047+0000 (oldest deadline 2024-09-11T23:41:27.271258+0000) 2024-09-11T23:41:45.250 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:41:44 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9[65389]: 2024-09-11T23:41:44.873+0000 7fc7513f1640 -1 osd.9 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:13.974640+0000 front 2024-09-11T23:41:13.974576+0000 (oldest deadline 2024-09-11T23:41:38.074690+0000) 2024-09-11T23:41:45.250 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:41:44 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9[65389]: 2024-09-11T23:41:44.873+0000 7fc7513f1640 -1 osd.9 120 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-09-11T23:41:18.075080+0000 front 2024-09-11T23:41:18.075257+0000 (oldest deadline 2024-09-11T23:41:43.375370+0000) 2024-09-11T23:41:45.552 INFO:journalctl@ceph.osd.7.smithi076.stdout:Sep 11 23:41:45 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-7[52804]: 2024-09-11T23:41:45.217+0000 7f28fcc36640 -1 osd.7 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:05.616564+0000 front 2024-09-11T23:41:05.616590+0000 (oldest deadline 2024-09-11T23:41:31.516796+0000) 2024-09-11T23:41:45.552 INFO:journalctl@ceph.osd.7.smithi076.stdout:Sep 11 23:41:45 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-7[52804]: 2024-09-11T23:41:45.217+0000 7f28fcc36640 -1 osd.7 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:11.517294+0000 front 2024-09-11T23:41:11.517280+0000 (oldest deadline 2024-09-11T23:41:35.017284+0000) 2024-09-11T23:41:45.772 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:41:45 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:41:45.514+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:56.932175+0000 front 2024-09-11T23:40:56.932499+0000 (oldest deadline 2024-09-11T23:41:21.032317+0000) 2024-09-11T23:41:45.772 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:41:45 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:41:45.514+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:01.032761+0000 front 2024-09-11T23:41:01.032638+0000 (oldest deadline 2024-09-11T23:41:30.433335+0000) 2024-09-11T23:41:45.772 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:41:45 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:41:45.514+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:10.934510+0000 front 2024-09-11T23:41:10.934411+0000 (oldest deadline 2024-09-11T23:41:35.634455+0000) 2024-09-11T23:41:45.772 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:41:45 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:41:45.514+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-09-11T23:41:21.936325+0000 front 2024-09-11T23:41:21.936462+0000 (oldest deadline 2024-09-11T23:41:42.937021+0000) 2024-09-11T23:41:45.843 INFO:journalctl@ceph.osd.6.smithi026.stdout:Sep 11 23:41:45 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-6[58789]: 2024-09-11T23:41:45.591+0000 7f8321638640 -1 osd.6 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:57.112923+0000 front 2024-09-11T23:40:57.113173+0000 (oldest deadline 2024-09-11T23:41:18.813188+0000) 2024-09-11T23:41:45.843 INFO:journalctl@ceph.osd.6.smithi026.stdout:Sep 11 23:41:45 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-6[58789]: 2024-09-11T23:41:45.591+0000 7f8321638640 -1 osd.6 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:05.814845+0000 front 2024-09-11T23:41:05.814472+0000 (oldest deadline 2024-09-11T23:41:31.114608+0000) 2024-09-11T23:41:45.843 INFO:journalctl@ceph.osd.6.smithi026.stdout:Sep 11 23:41:45 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-6[58789]: 2024-09-11T23:41:45.591+0000 7f8321638640 -1 osd.6 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:11.115165+0000 front 2024-09-11T23:41:11.115230+0000 (oldest deadline 2024-09-11T23:41:35.215334+0000) 2024-09-11T23:41:45.844 INFO:journalctl@ceph.osd.6.smithi026.stdout:Sep 11 23:41:45 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-6[58789]: 2024-09-11T23:41:45.591+0000 7f8321638640 -1 osd.6 120 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-09-11T23:41:20.516402+0000 front 2024-09-11T23:41:20.516356+0000 (oldest deadline 2024-09-11T23:41:44.616575+0000) 2024-09-11T23:41:46.052 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:41:45 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:41:45.672+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:57.536934+0000 front 2024-09-11T23:40:57.537030+0000 (oldest deadline 2024-09-11T23:41:21.036908+0000) 2024-09-11T23:41:46.053 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:41:45 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:41:45.672+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:03.838149+0000 front 2024-09-11T23:41:03.838269+0000 (oldest deadline 2024-09-11T23:41:29.138274+0000) 2024-09-11T23:41:46.053 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:41:45 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:41:45.672+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:12.539763+0000 front 2024-09-11T23:41:12.539893+0000 (oldest deadline 2024-09-11T23:41:36.639808+0000) 2024-09-11T23:41:46.053 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:41:45 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:41:45.672+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-09-11T23:41:21.841541+0000 front 2024-09-11T23:41:21.841518+0000 (oldest deadline 2024-09-11T23:41:44.141586+0000) 2024-09-11T23:41:46.238 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 11 23:41:45 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-8[53569]: 2024-09-11T23:41:45.772+0000 7f99e4062640 -1 osd.8 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:53.978855+0000 front 2024-09-11T23:40:53.978923+0000 (oldest deadline 2024-09-11T23:41:18.679037+0000) 2024-09-11T23:41:46.239 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 11 23:41:45 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-8[53569]: 2024-09-11T23:41:45.772+0000 7f99e4062640 -1 osd.8 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:10.281739+0000 front 2024-09-11T23:41:10.281758+0000 (oldest deadline 2024-09-11T23:41:39.682327+0000) 2024-09-11T23:41:46.239 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 11 23:41:45 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-8[53569]: 2024-09-11T23:41:45.772+0000 7f99e4062640 -1 osd.8 120 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-09-11T23:41:19.683087+0000 front 2024-09-11T23:41:19.683104+0000 (oldest deadline 2024-09-11T23:41:43.182872+0000) 2024-09-11T23:41:46.250 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:41:45 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9[65389]: 2024-09-11T23:41:45.844+0000 7fc7513f1640 -1 osd.9 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:57.370068+0000 front 2024-09-11T23:40:57.370191+0000 (oldest deadline 2024-09-11T23:41:21.470122+0000) 2024-09-11T23:41:46.250 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:41:45 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9[65389]: 2024-09-11T23:41:45.844+0000 7fc7513f1640 -1 osd.9 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:04.971207+0000 front 2024-09-11T23:41:04.971047+0000 (oldest deadline 2024-09-11T23:41:27.271258+0000) 2024-09-11T23:41:46.250 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:41:45 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9[65389]: 2024-09-11T23:41:45.844+0000 7fc7513f1640 -1 osd.9 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:13.974640+0000 front 2024-09-11T23:41:13.974576+0000 (oldest deadline 2024-09-11T23:41:38.074690+0000) 2024-09-11T23:41:46.250 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:41:45 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9[65389]: 2024-09-11T23:41:45.844+0000 7fc7513f1640 -1 osd.9 120 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-09-11T23:41:18.075080+0000 front 2024-09-11T23:41:18.075257+0000 (oldest deadline 2024-09-11T23:41:43.375370+0000) 2024-09-11T23:41:46.552 INFO:journalctl@ceph.osd.7.smithi076.stdout:Sep 11 23:41:46 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-7[52804]: 2024-09-11T23:41:46.249+0000 7f28fcc36640 -1 osd.7 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:05.616564+0000 front 2024-09-11T23:41:05.616590+0000 (oldest deadline 2024-09-11T23:41:31.516796+0000) 2024-09-11T23:41:46.552 INFO:journalctl@ceph.osd.7.smithi076.stdout:Sep 11 23:41:46 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-7[52804]: 2024-09-11T23:41:46.249+0000 7f28fcc36640 -1 osd.7 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:11.517294+0000 front 2024-09-11T23:41:11.517280+0000 (oldest deadline 2024-09-11T23:41:35.017284+0000) 2024-09-11T23:41:46.552 INFO:journalctl@ceph.osd.7.smithi076.stdout:Sep 11 23:41:46 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-7[52804]: 2024-09-11T23:41:46.249+0000 7f28fcc36640 -1 osd.7 120 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-09-11T23:41:21.419184+0000 front 2024-09-11T23:41:21.419099+0000 (oldest deadline 2024-09-11T23:41:46.118940+0000) 2024-09-11T23:41:46.739 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:41:46 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:41:46.486+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:56.932175+0000 front 2024-09-11T23:40:56.932499+0000 (oldest deadline 2024-09-11T23:41:21.032317+0000) 2024-09-11T23:41:46.739 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:41:46 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:41:46.486+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:01.032761+0000 front 2024-09-11T23:41:01.032638+0000 (oldest deadline 2024-09-11T23:41:30.433335+0000) 2024-09-11T23:41:46.739 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:41:46 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:41:46.486+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:10.934510+0000 front 2024-09-11T23:41:10.934411+0000 (oldest deadline 2024-09-11T23:41:35.634455+0000) 2024-09-11T23:41:46.739 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:41:46 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:41:46.486+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-09-11T23:41:21.936325+0000 front 2024-09-11T23:41:21.936462+0000 (oldest deadline 2024-09-11T23:41:42.937021+0000) 2024-09-11T23:41:47.000 INFO:journalctl@ceph.osd.6.smithi026.stdout:Sep 11 23:41:46 smithi026 podman[114915]: 2024-09-11 23:41:46.71939519 +0000 UTC m=+5.714400079 container died b9b1c5a26366b3f3535261fac04b3e0759aa89df6a3a9460638453405cbd9e91 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-6, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_CLEAN=True, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-09-11T23:41:47.001 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:41:46 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9[65389]: 2024-09-11T23:41:46.825+0000 7fc7513f1640 -1 osd.9 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:57.370068+0000 front 2024-09-11T23:40:57.370191+0000 (oldest deadline 2024-09-11T23:41:21.470122+0000) 2024-09-11T23:41:47.001 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:41:46 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9[65389]: 2024-09-11T23:41:46.825+0000 7fc7513f1640 -1 osd.9 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:04.971207+0000 front 2024-09-11T23:41:04.971047+0000 (oldest deadline 2024-09-11T23:41:27.271258+0000) 2024-09-11T23:41:47.001 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:41:46 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9[65389]: 2024-09-11T23:41:46.825+0000 7fc7513f1640 -1 osd.9 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:13.974640+0000 front 2024-09-11T23:41:13.974576+0000 (oldest deadline 2024-09-11T23:41:38.074690+0000) 2024-09-11T23:41:47.001 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:41:46 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9[65389]: 2024-09-11T23:41:46.825+0000 7fc7513f1640 -1 osd.9 120 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-09-11T23:41:18.075080+0000 front 2024-09-11T23:41:18.075257+0000 (oldest deadline 2024-09-11T23:41:43.375370+0000) 2024-09-11T23:41:47.052 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:41:46 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:41:46.692+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:57.536934+0000 front 2024-09-11T23:40:57.537030+0000 (oldest deadline 2024-09-11T23:41:21.036908+0000) 2024-09-11T23:41:47.052 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:41:46 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:41:46.692+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:03.838149+0000 front 2024-09-11T23:41:03.838269+0000 (oldest deadline 2024-09-11T23:41:29.138274+0000) 2024-09-11T23:41:47.052 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:41:46 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:41:46.692+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:12.539763+0000 front 2024-09-11T23:41:12.539893+0000 (oldest deadline 2024-09-11T23:41:36.639808+0000) 2024-09-11T23:41:47.053 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:41:46 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:41:46.692+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-09-11T23:41:21.841541+0000 front 2024-09-11T23:41:21.841518+0000 (oldest deadline 2024-09-11T23:41:44.141586+0000) 2024-09-11T23:41:47.238 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 11 23:41:46 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-8[53569]: 2024-09-11T23:41:46.801+0000 7f99e4062640 -1 osd.8 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:53.978855+0000 front 2024-09-11T23:40:53.978923+0000 (oldest deadline 2024-09-11T23:41:18.679037+0000) 2024-09-11T23:41:47.239 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 11 23:41:46 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-8[53569]: 2024-09-11T23:41:46.801+0000 7f99e4062640 -1 osd.8 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:10.281739+0000 front 2024-09-11T23:41:10.281758+0000 (oldest deadline 2024-09-11T23:41:39.682327+0000) 2024-09-11T23:41:47.239 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 11 23:41:46 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-8[53569]: 2024-09-11T23:41:46.801+0000 7f99e4062640 -1 osd.8 120 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-09-11T23:41:19.683087+0000 front 2024-09-11T23:41:19.683104+0000 (oldest deadline 2024-09-11T23:41:43.182872+0000) 2024-09-11T23:41:47.450 INFO:journalctl@ceph.osd.6.smithi026.stdout:Sep 11 23:41:47 smithi026 podman[114915]: 2024-09-11 23:41:47.190113076 +0000 UTC m=+6.185117966 container cleanup b9b1c5a26366b3f3535261fac04b3e0759aa89df6a3a9460638453405cbd9e91 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-6, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-09-11T23:41:47.450 INFO:journalctl@ceph.osd.6.smithi026.stdout:Sep 11 23:41:47 smithi026 bash[114915]: ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-6 2024-09-11T23:41:47.552 INFO:journalctl@ceph.osd.7.smithi076.stdout:Sep 11 23:41:47 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-7[52804]: 2024-09-11T23:41:47.255+0000 7f28fcc36640 -1 osd.7 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:05.616564+0000 front 2024-09-11T23:41:05.616590+0000 (oldest deadline 2024-09-11T23:41:31.516796+0000) 2024-09-11T23:41:47.552 INFO:journalctl@ceph.osd.7.smithi076.stdout:Sep 11 23:41:47 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-7[52804]: 2024-09-11T23:41:47.255+0000 7f28fcc36640 -1 osd.7 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:11.517294+0000 front 2024-09-11T23:41:11.517280+0000 (oldest deadline 2024-09-11T23:41:35.017284+0000) 2024-09-11T23:41:47.552 INFO:journalctl@ceph.osd.7.smithi076.stdout:Sep 11 23:41:47 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-7[52804]: 2024-09-11T23:41:47.255+0000 7f28fcc36640 -1 osd.7 120 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-09-11T23:41:21.419184+0000 front 2024-09-11T23:41:21.419099+0000 (oldest deadline 2024-09-11T23:41:46.118940+0000) 2024-09-11T23:41:47.750 INFO:journalctl@ceph.osd.6.smithi026.stdout:Sep 11 23:41:47 smithi026 podman[114943]: 2024-09-11 23:41:47.450143105 +0000 UTC m=+0.731820046 container remove b9b1c5a26366b3f3535261fac04b3e0759aa89df6a3a9460638453405cbd9e91 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-6, GIT_CLEAN=True, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T23:41:47.781 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:41:47 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:41:47.505+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:56.932175+0000 front 2024-09-11T23:40:56.932499+0000 (oldest deadline 2024-09-11T23:41:21.032317+0000) 2024-09-11T23:41:47.782 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:41:47 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:41:47.505+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:01.032761+0000 front 2024-09-11T23:41:01.032638+0000 (oldest deadline 2024-09-11T23:41:30.433335+0000) 2024-09-11T23:41:47.782 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:41:47 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:41:47.505+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:10.934510+0000 front 2024-09-11T23:41:10.934411+0000 (oldest deadline 2024-09-11T23:41:35.634455+0000) 2024-09-11T23:41:47.782 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:41:47 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:41:47.505+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-09-11T23:41:21.936325+0000 front 2024-09-11T23:41:21.936462+0000 (oldest deadline 2024-09-11T23:41:42.937021+0000) 2024-09-11T23:41:48.052 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:41:47 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:41:47.710+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:57.536934+0000 front 2024-09-11T23:40:57.537030+0000 (oldest deadline 2024-09-11T23:41:21.036908+0000) 2024-09-11T23:41:48.052 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:41:47 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:41:47.710+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:03.838149+0000 front 2024-09-11T23:41:03.838269+0000 (oldest deadline 2024-09-11T23:41:29.138274+0000) 2024-09-11T23:41:48.052 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:41:47 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:41:47.710+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:12.539763+0000 front 2024-09-11T23:41:12.539893+0000 (oldest deadline 2024-09-11T23:41:36.639808+0000) 2024-09-11T23:41:48.052 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:41:47 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:41:47.710+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-09-11T23:41:21.841541+0000 front 2024-09-11T23:41:21.841518+0000 (oldest deadline 2024-09-11T23:41:44.141586+0000) 2024-09-11T23:41:48.089 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:41:47 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9[65389]: 2024-09-11T23:41:47.842+0000 7fc7513f1640 -1 osd.9 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:57.370068+0000 front 2024-09-11T23:40:57.370191+0000 (oldest deadline 2024-09-11T23:41:21.470122+0000) 2024-09-11T23:41:48.089 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:41:47 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9[65389]: 2024-09-11T23:41:47.842+0000 7fc7513f1640 -1 osd.9 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:04.971207+0000 front 2024-09-11T23:41:04.971047+0000 (oldest deadline 2024-09-11T23:41:27.271258+0000) 2024-09-11T23:41:48.089 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:41:47 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9[65389]: 2024-09-11T23:41:47.842+0000 7fc7513f1640 -1 osd.9 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:13.974640+0000 front 2024-09-11T23:41:13.974576+0000 (oldest deadline 2024-09-11T23:41:38.074690+0000) 2024-09-11T23:41:48.089 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:41:47 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9[65389]: 2024-09-11T23:41:47.842+0000 7fc7513f1640 -1 osd.9 120 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-09-11T23:41:18.075080+0000 front 2024-09-11T23:41:18.075257+0000 (oldest deadline 2024-09-11T23:41:43.375370+0000) 2024-09-11T23:41:48.089 INFO:journalctl@ceph.osd.6.smithi026.stdout:Sep 11 23:41:47 smithi026 podman[115030]: 2024-09-11 23:41:47.668594065 +0000 UTC m=+0.016262339 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T23:41:48.090 INFO:journalctl@ceph.osd.6.smithi026.stdout:Sep 11 23:41:47 smithi026 podman[115030]: 2024-09-11 23:41:47.767501006 +0000 UTC m=+0.115169263 container create 71afd4fc66f14ebaa1273864e620874f5f7205ea39bfa254f104f37038effe7e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-6-deactivate, GIT_CLEAN=True, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-09-11T23:41:48.090 INFO:journalctl@ceph.osd.6.smithi026.stdout:Sep 11 23:41:48 smithi026 podman[115030]: 2024-09-11 23:41:48.003716186 +0000 UTC m=+0.351384453 container init 71afd4fc66f14ebaa1273864e620874f5f7205ea39bfa254f104f37038effe7e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-6-deactivate, GIT_CLEAN=True, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-09-11T23:41:48.090 INFO:journalctl@ceph.osd.6.smithi026.stdout:Sep 11 23:41:48 smithi026 podman[115030]: 2024-09-11 23:41:48.007199318 +0000 UTC m=+0.354867580 container start 71afd4fc66f14ebaa1273864e620874f5f7205ea39bfa254f104f37038effe7e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-6-deactivate, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-09-11T23:41:48.090 INFO:journalctl@ceph.osd.6.smithi026.stdout:Sep 11 23:41:48 smithi026 podman[115030]: 2024-09-11 23:41:48.086838128 +0000 UTC m=+0.434506406 container attach 71afd4fc66f14ebaa1273864e620874f5f7205ea39bfa254f104f37038effe7e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-6-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240731) 2024-09-11T23:41:48.239 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 11 23:41:47 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-8[53569]: 2024-09-11T23:41:47.781+0000 7f99e4062640 -1 osd.8 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:53.978855+0000 front 2024-09-11T23:40:53.978923+0000 (oldest deadline 2024-09-11T23:41:18.679037+0000) 2024-09-11T23:41:48.239 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 11 23:41:47 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-8[53569]: 2024-09-11T23:41:47.781+0000 7f99e4062640 -1 osd.8 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:10.281739+0000 front 2024-09-11T23:41:10.281758+0000 (oldest deadline 2024-09-11T23:41:39.682327+0000) 2024-09-11T23:41:48.239 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 11 23:41:47 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-8[53569]: 2024-09-11T23:41:47.781+0000 7f99e4062640 -1 osd.8 120 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-09-11T23:41:19.683087+0000 front 2024-09-11T23:41:19.683104+0000 (oldest deadline 2024-09-11T23:41:43.182872+0000) 2024-09-11T23:41:48.500 INFO:journalctl@ceph.osd.6.smithi026.stdout:Sep 11 23:41:48 smithi026 podman[115030]: 2024-09-11 23:41:48.199103101 +0000 UTC m=+0.546771359 container died 71afd4fc66f14ebaa1273864e620874f5f7205ea39bfa254f104f37038effe7e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-6-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_CLEAN=True) 2024-09-11T23:41:48.552 INFO:journalctl@ceph.osd.7.smithi076.stdout:Sep 11 23:41:48 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-7[52804]: 2024-09-11T23:41:48.209+0000 7f28fcc36640 -1 osd.7 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:05.616564+0000 front 2024-09-11T23:41:05.616590+0000 (oldest deadline 2024-09-11T23:41:31.516796+0000) 2024-09-11T23:41:48.552 INFO:journalctl@ceph.osd.7.smithi076.stdout:Sep 11 23:41:48 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-7[52804]: 2024-09-11T23:41:48.209+0000 7f28fcc36640 -1 osd.7 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:11.517294+0000 front 2024-09-11T23:41:11.517280+0000 (oldest deadline 2024-09-11T23:41:35.017284+0000) 2024-09-11T23:41:48.552 INFO:journalctl@ceph.osd.7.smithi076.stdout:Sep 11 23:41:48 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-7[52804]: 2024-09-11T23:41:48.209+0000 7f28fcc36640 -1 osd.7 120 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-09-11T23:41:21.419184+0000 front 2024-09-11T23:41:21.419099+0000 (oldest deadline 2024-09-11T23:41:46.118940+0000) 2024-09-11T23:41:48.801 DEBUG:teuthology.orchestra.run.smithi026:> sudo pkill -f 'journalctl -f -n 0 -u ceph-af7e6fc8-7092-11ef-bcea-c7b262605968@osd.6.service' 2024-09-11T23:41:49.025 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 11 23:41:48 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-8[53569]: 2024-09-11T23:41:48.783+0000 7f99e4062640 -1 osd.8 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:53.978855+0000 front 2024-09-11T23:40:53.978923+0000 (oldest deadline 2024-09-11T23:41:18.679037+0000) 2024-09-11T23:41:49.025 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 11 23:41:48 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-8[53569]: 2024-09-11T23:41:48.783+0000 7f99e4062640 -1 osd.8 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:10.281739+0000 front 2024-09-11T23:41:10.281758+0000 (oldest deadline 2024-09-11T23:41:39.682327+0000) 2024-09-11T23:41:49.026 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 11 23:41:48 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-8[53569]: 2024-09-11T23:41:48.783+0000 7f99e4062640 -1 osd.8 120 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-09-11T23:41:19.683087+0000 front 2024-09-11T23:41:19.683104+0000 (oldest deadline 2024-09-11T23:41:43.182872+0000) 2024-09-11T23:41:49.026 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:41:48 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:41:48.537+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:56.932175+0000 front 2024-09-11T23:40:56.932499+0000 (oldest deadline 2024-09-11T23:41:21.032317+0000) 2024-09-11T23:41:49.026 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:41:48 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:41:48.537+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:01.032761+0000 front 2024-09-11T23:41:01.032638+0000 (oldest deadline 2024-09-11T23:41:30.433335+0000) 2024-09-11T23:41:49.026 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:41:48 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:41:48.537+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:10.934510+0000 front 2024-09-11T23:41:10.934411+0000 (oldest deadline 2024-09-11T23:41:35.634455+0000) 2024-09-11T23:41:49.026 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:41:48 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:41:48.537+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-09-11T23:41:21.936325+0000 front 2024-09-11T23:41:21.936462+0000 (oldest deadline 2024-09-11T23:41:42.937021+0000) 2024-09-11T23:41:49.052 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:41:48 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:41:48.660+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:57.536934+0000 front 2024-09-11T23:40:57.537030+0000 (oldest deadline 2024-09-11T23:41:21.036908+0000) 2024-09-11T23:41:49.053 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:41:48 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:41:48.660+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:03.838149+0000 front 2024-09-11T23:41:03.838269+0000 (oldest deadline 2024-09-11T23:41:29.138274+0000) 2024-09-11T23:41:49.053 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:41:48 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:41:48.660+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:12.539763+0000 front 2024-09-11T23:41:12.539893+0000 (oldest deadline 2024-09-11T23:41:36.639808+0000) 2024-09-11T23:41:49.053 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:41:48 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:41:48.660+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-09-11T23:41:21.841541+0000 front 2024-09-11T23:41:21.841518+0000 (oldest deadline 2024-09-11T23:41:44.141586+0000) 2024-09-11T23:41:49.053 INFO:journalctl@ceph.osd.6.smithi026.stdout:Sep 11 23:41:48 smithi026 podman[115052]: 2024-09-11 23:41:48.773718671 +0000 UTC m=+0.568309022 container remove 71afd4fc66f14ebaa1273864e620874f5f7205ea39bfa254f104f37038effe7e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-6-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731) 2024-09-11T23:41:49.053 INFO:journalctl@ceph.osd.6.smithi026.stdout:Sep 11 23:41:48 smithi026 systemd[1]: ceph-af7e6fc8-7092-11ef-bcea-c7b262605968@osd.6.service: Deactivated successfully. 2024-09-11T23:41:49.054 INFO:journalctl@ceph.osd.6.smithi026.stdout:Sep 11 23:41:48 smithi026 systemd[1]: Stopped Ceph osd.6 for af7e6fc8-7092-11ef-bcea-c7b262605968. 2024-09-11T23:41:49.054 INFO:journalctl@ceph.osd.6.smithi026.stdout:Sep 11 23:41:48 smithi026 systemd[1]: ceph-af7e6fc8-7092-11ef-bcea-c7b262605968@osd.6.service: Consumed 12min 34.924s CPU time. 2024-09-11T23:41:49.054 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:41:48 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9[65389]: 2024-09-11T23:41:48.848+0000 7fc7513f1640 -1 osd.9 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:57.370068+0000 front 2024-09-11T23:40:57.370191+0000 (oldest deadline 2024-09-11T23:41:21.470122+0000) 2024-09-11T23:41:49.054 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:41:48 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9[65389]: 2024-09-11T23:41:48.848+0000 7fc7513f1640 -1 osd.9 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:04.971207+0000 front 2024-09-11T23:41:04.971047+0000 (oldest deadline 2024-09-11T23:41:27.271258+0000) 2024-09-11T23:41:49.054 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:41:48 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9[65389]: 2024-09-11T23:41:48.848+0000 7fc7513f1640 -1 osd.9 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:13.974640+0000 front 2024-09-11T23:41:13.974576+0000 (oldest deadline 2024-09-11T23:41:38.074690+0000) 2024-09-11T23:41:49.054 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:41:48 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9[65389]: 2024-09-11T23:41:48.848+0000 7fc7513f1640 -1 osd.9 120 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-09-11T23:41:18.075080+0000 front 2024-09-11T23:41:18.075257+0000 (oldest deadline 2024-09-11T23:41:43.375370+0000) 2024-09-11T23:41:49.072 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T23:41:49.072 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-09-11T23:41:49.073 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-09-11T23:41:49.073 DEBUG:teuthology.orchestra.run.smithi076:> sudo systemctl stop ceph-af7e6fc8-7092-11ef-bcea-c7b262605968@osd.7 2024-09-11T23:41:49.537 INFO:journalctl@ceph.osd.7.smithi076.stdout:Sep 11 23:41:49 smithi076 systemd[1]: Stopping Ceph osd.7 for af7e6fc8-7092-11ef-bcea-c7b262605968... 2024-09-11T23:41:49.537 INFO:journalctl@ceph.osd.7.smithi076.stdout:Sep 11 23:41:49 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-7[52804]: 2024-09-11T23:41:49.254+0000 7f28fcc36640 -1 osd.7 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:05.616564+0000 front 2024-09-11T23:41:05.616590+0000 (oldest deadline 2024-09-11T23:41:31.516796+0000) 2024-09-11T23:41:49.537 INFO:journalctl@ceph.osd.7.smithi076.stdout:Sep 11 23:41:49 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-7[52804]: 2024-09-11T23:41:49.254+0000 7f28fcc36640 -1 osd.7 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:11.517294+0000 front 2024-09-11T23:41:11.517280+0000 (oldest deadline 2024-09-11T23:41:35.017284+0000) 2024-09-11T23:41:49.537 INFO:journalctl@ceph.osd.7.smithi076.stdout:Sep 11 23:41:49 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-7[52804]: 2024-09-11T23:41:49.254+0000 7f28fcc36640 -1 osd.7 120 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-09-11T23:41:21.419184+0000 front 2024-09-11T23:41:21.419099+0000 (oldest deadline 2024-09-11T23:41:46.118940+0000) 2024-09-11T23:41:49.537 INFO:journalctl@ceph.osd.7.smithi076.stdout:Sep 11 23:41:49 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-7[52804]: 2024-09-11T23:41:49.536+0000 7f2900631640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T23:41:49.537 INFO:journalctl@ceph.osd.7.smithi076.stdout:Sep 11 23:41:49 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-7[52804]: 2024-09-11T23:41:49.537+0000 7f2900631640 -1 osd.7 120 *** Got signal Terminated *** 2024-09-11T23:41:49.537 INFO:journalctl@ceph.osd.7.smithi076.stdout:Sep 11 23:41:49 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-7[52804]: 2024-09-11T23:41:49.537+0000 7f2900631640 -1 osd.7 120 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T23:41:49.763 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:41:49 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:41:49.492+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:56.932175+0000 front 2024-09-11T23:40:56.932499+0000 (oldest deadline 2024-09-11T23:41:21.032317+0000) 2024-09-11T23:41:49.763 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:41:49 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:41:49.492+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:01.032761+0000 front 2024-09-11T23:41:01.032638+0000 (oldest deadline 2024-09-11T23:41:30.433335+0000) 2024-09-11T23:41:49.763 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:41:49 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:41:49.492+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:10.934510+0000 front 2024-09-11T23:41:10.934411+0000 (oldest deadline 2024-09-11T23:41:35.634455+0000) 2024-09-11T23:41:49.763 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:41:49 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:41:49.492+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-09-11T23:41:21.936325+0000 front 2024-09-11T23:41:21.936462+0000 (oldest deadline 2024-09-11T23:41:42.937021+0000) 2024-09-11T23:41:49.802 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:41:49 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:41:49.649+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:57.536934+0000 front 2024-09-11T23:40:57.537030+0000 (oldest deadline 2024-09-11T23:41:21.036908+0000) 2024-09-11T23:41:49.802 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:41:49 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:41:49.649+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:03.838149+0000 front 2024-09-11T23:41:03.838269+0000 (oldest deadline 2024-09-11T23:41:29.138274+0000) 2024-09-11T23:41:49.802 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:41:49 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:41:49.649+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:12.539763+0000 front 2024-09-11T23:41:12.539893+0000 (oldest deadline 2024-09-11T23:41:36.639808+0000) 2024-09-11T23:41:49.802 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:41:49 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:41:49.649+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-09-11T23:41:21.841541+0000 front 2024-09-11T23:41:21.841518+0000 (oldest deadline 2024-09-11T23:41:44.141586+0000) 2024-09-11T23:41:50.238 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 11 23:41:49 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-8[53569]: 2024-09-11T23:41:49.763+0000 7f99e4062640 -1 osd.8 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:53.978855+0000 front 2024-09-11T23:40:53.978923+0000 (oldest deadline 2024-09-11T23:41:18.679037+0000) 2024-09-11T23:41:50.238 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 11 23:41:49 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-8[53569]: 2024-09-11T23:41:49.763+0000 7f99e4062640 -1 osd.8 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:10.281739+0000 front 2024-09-11T23:41:10.281758+0000 (oldest deadline 2024-09-11T23:41:39.682327+0000) 2024-09-11T23:41:50.239 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 11 23:41:49 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-8[53569]: 2024-09-11T23:41:49.763+0000 7f99e4062640 -1 osd.8 120 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-09-11T23:41:19.683087+0000 front 2024-09-11T23:41:19.683104+0000 (oldest deadline 2024-09-11T23:41:43.182872+0000) 2024-09-11T23:41:50.250 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:41:49 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9[65389]: 2024-09-11T23:41:49.861+0000 7fc7513f1640 -1 osd.9 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:57.370068+0000 front 2024-09-11T23:40:57.370191+0000 (oldest deadline 2024-09-11T23:41:21.470122+0000) 2024-09-11T23:41:50.250 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:41:49 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9[65389]: 2024-09-11T23:41:49.861+0000 7fc7513f1640 -1 osd.9 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:04.971207+0000 front 2024-09-11T23:41:04.971047+0000 (oldest deadline 2024-09-11T23:41:27.271258+0000) 2024-09-11T23:41:50.250 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:41:49 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9[65389]: 2024-09-11T23:41:49.861+0000 7fc7513f1640 -1 osd.9 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:13.974640+0000 front 2024-09-11T23:41:13.974576+0000 (oldest deadline 2024-09-11T23:41:38.074690+0000) 2024-09-11T23:41:50.250 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:41:49 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9[65389]: 2024-09-11T23:41:49.861+0000 7fc7513f1640 -1 osd.9 120 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-09-11T23:41:18.075080+0000 front 2024-09-11T23:41:18.075257+0000 (oldest deadline 2024-09-11T23:41:43.375370+0000) 2024-09-11T23:41:50.552 INFO:journalctl@ceph.osd.7.smithi076.stdout:Sep 11 23:41:50 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-7[52804]: 2024-09-11T23:41:50.272+0000 7f28fcc36640 -1 osd.7 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:05.616564+0000 front 2024-09-11T23:41:05.616590+0000 (oldest deadline 2024-09-11T23:41:31.516796+0000) 2024-09-11T23:41:50.552 INFO:journalctl@ceph.osd.7.smithi076.stdout:Sep 11 23:41:50 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-7[52804]: 2024-09-11T23:41:50.272+0000 7f28fcc36640 -1 osd.7 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:11.517294+0000 front 2024-09-11T23:41:11.517280+0000 (oldest deadline 2024-09-11T23:41:35.017284+0000) 2024-09-11T23:41:50.553 INFO:journalctl@ceph.osd.7.smithi076.stdout:Sep 11 23:41:50 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-7[52804]: 2024-09-11T23:41:50.272+0000 7f28fcc36640 -1 osd.7 120 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-09-11T23:41:21.419184+0000 front 2024-09-11T23:41:21.419099+0000 (oldest deadline 2024-09-11T23:41:46.118940+0000) 2024-09-11T23:41:50.805 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:41:50 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:41:50.493+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:56.932175+0000 front 2024-09-11T23:40:56.932499+0000 (oldest deadline 2024-09-11T23:41:21.032317+0000) 2024-09-11T23:41:50.805 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:41:50 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:41:50.493+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:01.032761+0000 front 2024-09-11T23:41:01.032638+0000 (oldest deadline 2024-09-11T23:41:30.433335+0000) 2024-09-11T23:41:50.805 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:41:50 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:41:50.493+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:10.934510+0000 front 2024-09-11T23:41:10.934411+0000 (oldest deadline 2024-09-11T23:41:35.634455+0000) 2024-09-11T23:41:50.805 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:41:50 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:41:50.493+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-09-11T23:41:21.936325+0000 front 2024-09-11T23:41:21.936462+0000 (oldest deadline 2024-09-11T23:41:42.937021+0000) 2024-09-11T23:41:51.052 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:41:50 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:41:50.614+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:57.536934+0000 front 2024-09-11T23:40:57.537030+0000 (oldest deadline 2024-09-11T23:41:21.036908+0000) 2024-09-11T23:41:51.052 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:41:50 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:41:50.614+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:03.838149+0000 front 2024-09-11T23:41:03.838269+0000 (oldest deadline 2024-09-11T23:41:29.138274+0000) 2024-09-11T23:41:51.052 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:41:50 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:41:50.614+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:12.539763+0000 front 2024-09-11T23:41:12.539893+0000 (oldest deadline 2024-09-11T23:41:36.639808+0000) 2024-09-11T23:41:51.052 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:41:50 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:41:50.614+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-09-11T23:41:21.841541+0000 front 2024-09-11T23:41:21.841518+0000 (oldest deadline 2024-09-11T23:41:44.141586+0000) 2024-09-11T23:41:51.239 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 11 23:41:50 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-8[53569]: 2024-09-11T23:41:50.805+0000 7f99e4062640 -1 osd.8 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:53.978855+0000 front 2024-09-11T23:40:53.978923+0000 (oldest deadline 2024-09-11T23:41:18.679037+0000) 2024-09-11T23:41:51.239 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 11 23:41:50 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-8[53569]: 2024-09-11T23:41:50.805+0000 7f99e4062640 -1 osd.8 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:10.281739+0000 front 2024-09-11T23:41:10.281758+0000 (oldest deadline 2024-09-11T23:41:39.682327+0000) 2024-09-11T23:41:51.239 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 11 23:41:50 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-8[53569]: 2024-09-11T23:41:50.805+0000 7f99e4062640 -1 osd.8 120 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-09-11T23:41:19.683087+0000 front 2024-09-11T23:41:19.683104+0000 (oldest deadline 2024-09-11T23:41:43.182872+0000) 2024-09-11T23:41:51.250 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:41:50 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9[65389]: 2024-09-11T23:41:50.906+0000 7fc7513f1640 -1 osd.9 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:57.370068+0000 front 2024-09-11T23:40:57.370191+0000 (oldest deadline 2024-09-11T23:41:21.470122+0000) 2024-09-11T23:41:51.250 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:41:50 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9[65389]: 2024-09-11T23:41:50.906+0000 7fc7513f1640 -1 osd.9 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:04.971207+0000 front 2024-09-11T23:41:04.971047+0000 (oldest deadline 2024-09-11T23:41:27.271258+0000) 2024-09-11T23:41:51.250 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:41:50 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9[65389]: 2024-09-11T23:41:50.906+0000 7fc7513f1640 -1 osd.9 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:13.974640+0000 front 2024-09-11T23:41:13.974576+0000 (oldest deadline 2024-09-11T23:41:38.074690+0000) 2024-09-11T23:41:51.250 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:41:50 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9[65389]: 2024-09-11T23:41:50.906+0000 7fc7513f1640 -1 osd.9 120 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-09-11T23:41:18.075080+0000 front 2024-09-11T23:41:18.075257+0000 (oldest deadline 2024-09-11T23:41:43.375370+0000) 2024-09-11T23:41:51.552 INFO:journalctl@ceph.osd.7.smithi076.stdout:Sep 11 23:41:51 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-7[52804]: 2024-09-11T23:41:51.266+0000 7f28fcc36640 -1 osd.7 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:05.616564+0000 front 2024-09-11T23:41:05.616590+0000 (oldest deadline 2024-09-11T23:41:31.516796+0000) 2024-09-11T23:41:51.552 INFO:journalctl@ceph.osd.7.smithi076.stdout:Sep 11 23:41:51 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-7[52804]: 2024-09-11T23:41:51.266+0000 7f28fcc36640 -1 osd.7 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:11.517294+0000 front 2024-09-11T23:41:11.517280+0000 (oldest deadline 2024-09-11T23:41:35.017284+0000) 2024-09-11T23:41:51.553 INFO:journalctl@ceph.osd.7.smithi076.stdout:Sep 11 23:41:51 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-7[52804]: 2024-09-11T23:41:51.266+0000 7f28fcc36640 -1 osd.7 120 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-09-11T23:41:21.419184+0000 front 2024-09-11T23:41:21.419099+0000 (oldest deadline 2024-09-11T23:41:46.118940+0000) 2024-09-11T23:41:51.769 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:41:51 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:41:51.514+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:56.932175+0000 front 2024-09-11T23:40:56.932499+0000 (oldest deadline 2024-09-11T23:41:21.032317+0000) 2024-09-11T23:41:51.769 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:41:51 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:41:51.514+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:01.032761+0000 front 2024-09-11T23:41:01.032638+0000 (oldest deadline 2024-09-11T23:41:30.433335+0000) 2024-09-11T23:41:51.769 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:41:51 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:41:51.514+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:10.934510+0000 front 2024-09-11T23:41:10.934411+0000 (oldest deadline 2024-09-11T23:41:35.634455+0000) 2024-09-11T23:41:51.769 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:41:51 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:41:51.514+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-09-11T23:41:21.936325+0000 front 2024-09-11T23:41:21.936462+0000 (oldest deadline 2024-09-11T23:41:42.937021+0000) 2024-09-11T23:41:52.052 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:41:51 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:41:51.594+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:57.536934+0000 front 2024-09-11T23:40:57.537030+0000 (oldest deadline 2024-09-11T23:41:21.036908+0000) 2024-09-11T23:41:52.052 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:41:51 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:41:51.594+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:03.838149+0000 front 2024-09-11T23:41:03.838269+0000 (oldest deadline 2024-09-11T23:41:29.138274+0000) 2024-09-11T23:41:52.052 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:41:51 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:41:51.594+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:12.539763+0000 front 2024-09-11T23:41:12.539893+0000 (oldest deadline 2024-09-11T23:41:36.639808+0000) 2024-09-11T23:41:52.052 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:41:51 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:41:51.594+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-09-11T23:41:21.841541+0000 front 2024-09-11T23:41:21.841518+0000 (oldest deadline 2024-09-11T23:41:44.141586+0000) 2024-09-11T23:41:52.238 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 11 23:41:51 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-8[53569]: 2024-09-11T23:41:51.769+0000 7f99e4062640 -1 osd.8 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:53.978855+0000 front 2024-09-11T23:40:53.978923+0000 (oldest deadline 2024-09-11T23:41:18.679037+0000) 2024-09-11T23:41:52.238 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 11 23:41:51 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-8[53569]: 2024-09-11T23:41:51.769+0000 7f99e4062640 -1 osd.8 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:10.281739+0000 front 2024-09-11T23:41:10.281758+0000 (oldest deadline 2024-09-11T23:41:39.682327+0000) 2024-09-11T23:41:52.239 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 11 23:41:51 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-8[53569]: 2024-09-11T23:41:51.769+0000 7f99e4062640 -1 osd.8 120 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-09-11T23:41:19.683087+0000 front 2024-09-11T23:41:19.683104+0000 (oldest deadline 2024-09-11T23:41:43.182872+0000) 2024-09-11T23:41:52.250 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:41:51 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9[65389]: 2024-09-11T23:41:51.881+0000 7fc7513f1640 -1 osd.9 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:57.370068+0000 front 2024-09-11T23:40:57.370191+0000 (oldest deadline 2024-09-11T23:41:21.470122+0000) 2024-09-11T23:41:52.250 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:41:51 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9[65389]: 2024-09-11T23:41:51.881+0000 7fc7513f1640 -1 osd.9 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:04.971207+0000 front 2024-09-11T23:41:04.971047+0000 (oldest deadline 2024-09-11T23:41:27.271258+0000) 2024-09-11T23:41:52.250 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:41:51 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9[65389]: 2024-09-11T23:41:51.881+0000 7fc7513f1640 -1 osd.9 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:13.974640+0000 front 2024-09-11T23:41:13.974576+0000 (oldest deadline 2024-09-11T23:41:38.074690+0000) 2024-09-11T23:41:52.250 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:41:51 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9[65389]: 2024-09-11T23:41:51.881+0000 7fc7513f1640 -1 osd.9 120 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-09-11T23:41:18.075080+0000 front 2024-09-11T23:41:18.075257+0000 (oldest deadline 2024-09-11T23:41:43.375370+0000) 2024-09-11T23:41:52.552 INFO:journalctl@ceph.osd.7.smithi076.stdout:Sep 11 23:41:52 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-7[52804]: 2024-09-11T23:41:52.291+0000 7f28fcc36640 -1 osd.7 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:05.616564+0000 front 2024-09-11T23:41:05.616590+0000 (oldest deadline 2024-09-11T23:41:31.516796+0000) 2024-09-11T23:41:52.552 INFO:journalctl@ceph.osd.7.smithi076.stdout:Sep 11 23:41:52 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-7[52804]: 2024-09-11T23:41:52.291+0000 7f28fcc36640 -1 osd.7 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:11.517294+0000 front 2024-09-11T23:41:11.517280+0000 (oldest deadline 2024-09-11T23:41:35.017284+0000) 2024-09-11T23:41:52.552 INFO:journalctl@ceph.osd.7.smithi076.stdout:Sep 11 23:41:52 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-7[52804]: 2024-09-11T23:41:52.291+0000 7f28fcc36640 -1 osd.7 120 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-09-11T23:41:21.419184+0000 front 2024-09-11T23:41:21.419099+0000 (oldest deadline 2024-09-11T23:41:46.118940+0000) 2024-09-11T23:41:52.990 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 11 23:41:52 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-8[53569]: 2024-09-11T23:41:52.747+0000 7f99e4062640 -1 osd.8 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:53.978855+0000 front 2024-09-11T23:40:53.978923+0000 (oldest deadline 2024-09-11T23:41:18.679037+0000) 2024-09-11T23:41:52.990 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 11 23:41:52 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-8[53569]: 2024-09-11T23:41:52.747+0000 7f99e4062640 -1 osd.8 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:10.281739+0000 front 2024-09-11T23:41:10.281758+0000 (oldest deadline 2024-09-11T23:41:39.682327+0000) 2024-09-11T23:41:52.990 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 11 23:41:52 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-8[53569]: 2024-09-11T23:41:52.747+0000 7f99e4062640 -1 osd.8 120 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-09-11T23:41:19.683087+0000 front 2024-09-11T23:41:19.683104+0000 (oldest deadline 2024-09-11T23:41:43.182872+0000) 2024-09-11T23:41:52.990 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:41:52 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:41:52.502+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:56.932175+0000 front 2024-09-11T23:40:56.932499+0000 (oldest deadline 2024-09-11T23:41:21.032317+0000) 2024-09-11T23:41:52.990 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:41:52 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:41:52.502+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:01.032761+0000 front 2024-09-11T23:41:01.032638+0000 (oldest deadline 2024-09-11T23:41:30.433335+0000) 2024-09-11T23:41:52.990 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:41:52 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:41:52.502+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:10.934510+0000 front 2024-09-11T23:41:10.934411+0000 (oldest deadline 2024-09-11T23:41:35.634455+0000) 2024-09-11T23:41:52.991 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:41:52 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:41:52.502+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-09-11T23:41:21.936325+0000 front 2024-09-11T23:41:21.936462+0000 (oldest deadline 2024-09-11T23:41:42.937021+0000) 2024-09-11T23:41:52.991 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:41:52 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:41:52.502+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-09-11T23:41:27.038312+0000 front 2024-09-11T23:41:27.038192+0000 (oldest deadline 2024-09-11T23:41:51.638385+0000) 2024-09-11T23:41:53.052 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:41:52 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:41:52.581+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:57.536934+0000 front 2024-09-11T23:40:57.537030+0000 (oldest deadline 2024-09-11T23:41:21.036908+0000) 2024-09-11T23:41:53.052 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:41:52 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:41:52.581+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:03.838149+0000 front 2024-09-11T23:41:03.838269+0000 (oldest deadline 2024-09-11T23:41:29.138274+0000) 2024-09-11T23:41:53.052 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:41:52 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:41:52.581+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:12.539763+0000 front 2024-09-11T23:41:12.539893+0000 (oldest deadline 2024-09-11T23:41:36.639808+0000) 2024-09-11T23:41:53.052 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:41:52 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:41:52.581+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-09-11T23:41:21.841541+0000 front 2024-09-11T23:41:21.841518+0000 (oldest deadline 2024-09-11T23:41:44.141586+0000) 2024-09-11T23:41:53.250 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:41:52 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9[65389]: 2024-09-11T23:41:52.891+0000 7fc7513f1640 -1 osd.9 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:57.370068+0000 front 2024-09-11T23:40:57.370191+0000 (oldest deadline 2024-09-11T23:41:21.470122+0000) 2024-09-11T23:41:53.250 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:41:52 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9[65389]: 2024-09-11T23:41:52.891+0000 7fc7513f1640 -1 osd.9 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:04.971207+0000 front 2024-09-11T23:41:04.971047+0000 (oldest deadline 2024-09-11T23:41:27.271258+0000) 2024-09-11T23:41:53.250 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:41:52 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9[65389]: 2024-09-11T23:41:52.891+0000 7fc7513f1640 -1 osd.9 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:13.974640+0000 front 2024-09-11T23:41:13.974576+0000 (oldest deadline 2024-09-11T23:41:38.074690+0000) 2024-09-11T23:41:53.250 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:41:52 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9[65389]: 2024-09-11T23:41:52.891+0000 7fc7513f1640 -1 osd.9 120 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-09-11T23:41:18.075080+0000 front 2024-09-11T23:41:18.075257+0000 (oldest deadline 2024-09-11T23:41:43.375370+0000) 2024-09-11T23:41:53.603 INFO:journalctl@ceph.osd.7.smithi076.stdout:Sep 11 23:41:53 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-7[52804]: 2024-09-11T23:41:53.332+0000 7f28fcc36640 -1 osd.7 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:05.616564+0000 front 2024-09-11T23:41:05.616590+0000 (oldest deadline 2024-09-11T23:41:31.516796+0000) 2024-09-11T23:41:53.603 INFO:journalctl@ceph.osd.7.smithi076.stdout:Sep 11 23:41:53 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-7[52804]: 2024-09-11T23:41:53.332+0000 7f28fcc36640 -1 osd.7 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:11.517294+0000 front 2024-09-11T23:41:11.517280+0000 (oldest deadline 2024-09-11T23:41:35.017284+0000) 2024-09-11T23:41:53.604 INFO:journalctl@ceph.osd.7.smithi076.stdout:Sep 11 23:41:53 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-7[52804]: 2024-09-11T23:41:53.332+0000 7f28fcc36640 -1 osd.7 120 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-09-11T23:41:21.419184+0000 front 2024-09-11T23:41:21.419099+0000 (oldest deadline 2024-09-11T23:41:46.118940+0000) 2024-09-11T23:41:53.767 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:41:53 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:41:53.497+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:56.932175+0000 front 2024-09-11T23:40:56.932499+0000 (oldest deadline 2024-09-11T23:41:21.032317+0000) 2024-09-11T23:41:53.767 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:41:53 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:41:53.497+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:01.032761+0000 front 2024-09-11T23:41:01.032638+0000 (oldest deadline 2024-09-11T23:41:30.433335+0000) 2024-09-11T23:41:53.767 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:41:53 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:41:53.497+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:10.934510+0000 front 2024-09-11T23:41:10.934411+0000 (oldest deadline 2024-09-11T23:41:35.634455+0000) 2024-09-11T23:41:53.767 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:41:53 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:41:53.497+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-09-11T23:41:21.936325+0000 front 2024-09-11T23:41:21.936462+0000 (oldest deadline 2024-09-11T23:41:42.937021+0000) 2024-09-11T23:41:53.767 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:41:53 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:41:53.498+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-09-11T23:41:27.038312+0000 front 2024-09-11T23:41:27.038192+0000 (oldest deadline 2024-09-11T23:41:51.638385+0000) 2024-09-11T23:41:54.052 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:41:53 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:41:53.603+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:57.536934+0000 front 2024-09-11T23:40:57.537030+0000 (oldest deadline 2024-09-11T23:41:21.036908+0000) 2024-09-11T23:41:54.052 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:41:53 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:41:53.603+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:03.838149+0000 front 2024-09-11T23:41:03.838269+0000 (oldest deadline 2024-09-11T23:41:29.138274+0000) 2024-09-11T23:41:54.052 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:41:53 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:41:53.603+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:12.539763+0000 front 2024-09-11T23:41:12.539893+0000 (oldest deadline 2024-09-11T23:41:36.639808+0000) 2024-09-11T23:41:54.052 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:41:53 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:41:53.603+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-09-11T23:41:21.841541+0000 front 2024-09-11T23:41:21.841518+0000 (oldest deadline 2024-09-11T23:41:44.141586+0000) 2024-09-11T23:41:54.238 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 11 23:41:53 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-8[53569]: 2024-09-11T23:41:53.766+0000 7f99e4062640 -1 osd.8 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:53.978855+0000 front 2024-09-11T23:40:53.978923+0000 (oldest deadline 2024-09-11T23:41:18.679037+0000) 2024-09-11T23:41:54.239 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 11 23:41:53 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-8[53569]: 2024-09-11T23:41:53.766+0000 7f99e4062640 -1 osd.8 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:10.281739+0000 front 2024-09-11T23:41:10.281758+0000 (oldest deadline 2024-09-11T23:41:39.682327+0000) 2024-09-11T23:41:54.239 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 11 23:41:53 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-8[53569]: 2024-09-11T23:41:53.766+0000 7f99e4062640 -1 osd.8 120 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-09-11T23:41:19.683087+0000 front 2024-09-11T23:41:19.683104+0000 (oldest deadline 2024-09-11T23:41:43.182872+0000) 2024-09-11T23:41:54.239 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 11 23:41:53 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-8[53569]: 2024-09-11T23:41:53.766+0000 7f99e4062640 -1 osd.8 120 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-09-11T23:41:30.184463+0000 front 2024-09-11T23:41:30.184492+0000 (oldest deadline 2024-09-11T23:41:53.684668+0000) 2024-09-11T23:41:54.250 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:41:53 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9[65389]: 2024-09-11T23:41:53.899+0000 7fc7513f1640 -1 osd.9 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:57.370068+0000 front 2024-09-11T23:40:57.370191+0000 (oldest deadline 2024-09-11T23:41:21.470122+0000) 2024-09-11T23:41:54.250 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:41:53 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9[65389]: 2024-09-11T23:41:53.899+0000 7fc7513f1640 -1 osd.9 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:04.971207+0000 front 2024-09-11T23:41:04.971047+0000 (oldest deadline 2024-09-11T23:41:27.271258+0000) 2024-09-11T23:41:54.250 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:41:53 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9[65389]: 2024-09-11T23:41:53.899+0000 7fc7513f1640 -1 osd.9 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:13.974640+0000 front 2024-09-11T23:41:13.974576+0000 (oldest deadline 2024-09-11T23:41:38.074690+0000) 2024-09-11T23:41:54.250 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:41:53 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9[65389]: 2024-09-11T23:41:53.899+0000 7fc7513f1640 -1 osd.9 120 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-09-11T23:41:18.075080+0000 front 2024-09-11T23:41:18.075257+0000 (oldest deadline 2024-09-11T23:41:43.375370+0000) 2024-09-11T23:41:54.612 INFO:journalctl@ceph.osd.7.smithi076.stdout:Sep 11 23:41:54 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-7[52804]: 2024-09-11T23:41:54.307+0000 7f28fcc36640 -1 osd.7 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:05.616564+0000 front 2024-09-11T23:41:05.616590+0000 (oldest deadline 2024-09-11T23:41:31.516796+0000) 2024-09-11T23:41:54.612 INFO:journalctl@ceph.osd.7.smithi076.stdout:Sep 11 23:41:54 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-7[52804]: 2024-09-11T23:41:54.307+0000 7f28fcc36640 -1 osd.7 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:11.517294+0000 front 2024-09-11T23:41:11.517280+0000 (oldest deadline 2024-09-11T23:41:35.017284+0000) 2024-09-11T23:41:54.612 INFO:journalctl@ceph.osd.7.smithi076.stdout:Sep 11 23:41:54 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-7[52804]: 2024-09-11T23:41:54.307+0000 7f28fcc36640 -1 osd.7 120 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-09-11T23:41:21.419184+0000 front 2024-09-11T23:41:21.419099+0000 (oldest deadline 2024-09-11T23:41:46.118940+0000) 2024-09-11T23:41:54.612 INFO:journalctl@ceph.osd.7.smithi076.stdout:Sep 11 23:41:54 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-7[52804]: 2024-09-11T23:41:54.307+0000 7f28fcc36640 -1 osd.7 120 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-09-11T23:41:28.419732+0000 front 2024-09-11T23:41:28.419755+0000 (oldest deadline 2024-09-11T23:41:53.719855+0000) 2024-09-11T23:41:54.738 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:41:54 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:41:54.480+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:56.932175+0000 front 2024-09-11T23:40:56.932499+0000 (oldest deadline 2024-09-11T23:41:21.032317+0000) 2024-09-11T23:41:54.739 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:41:54 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:41:54.480+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:01.032761+0000 front 2024-09-11T23:41:01.032638+0000 (oldest deadline 2024-09-11T23:41:30.433335+0000) 2024-09-11T23:41:54.739 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:41:54 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:41:54.480+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:10.934510+0000 front 2024-09-11T23:41:10.934411+0000 (oldest deadline 2024-09-11T23:41:35.634455+0000) 2024-09-11T23:41:54.739 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:41:54 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:41:54.480+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-09-11T23:41:21.936325+0000 front 2024-09-11T23:41:21.936462+0000 (oldest deadline 2024-09-11T23:41:42.937021+0000) 2024-09-11T23:41:54.739 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:41:54 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:41:54.480+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-09-11T23:41:27.038312+0000 front 2024-09-11T23:41:27.038192+0000 (oldest deadline 2024-09-11T23:41:51.638385+0000) 2024-09-11T23:41:55.052 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:41:54 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:41:54.612+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:57.536934+0000 front 2024-09-11T23:40:57.537030+0000 (oldest deadline 2024-09-11T23:41:21.036908+0000) 2024-09-11T23:41:55.052 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:41:54 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:41:54.612+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:03.838149+0000 front 2024-09-11T23:41:03.838269+0000 (oldest deadline 2024-09-11T23:41:29.138274+0000) 2024-09-11T23:41:55.053 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:41:54 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:41:54.612+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:12.539763+0000 front 2024-09-11T23:41:12.539893+0000 (oldest deadline 2024-09-11T23:41:36.639808+0000) 2024-09-11T23:41:55.053 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:41:54 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:41:54.612+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-09-11T23:41:21.841541+0000 front 2024-09-11T23:41:21.841518+0000 (oldest deadline 2024-09-11T23:41:44.141586+0000) 2024-09-11T23:41:55.053 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:41:54 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:41:54.612+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-09-11T23:41:28.142849+0000 front 2024-09-11T23:41:28.142868+0000 (oldest deadline 2024-09-11T23:41:54.042918+0000) 2024-09-11T23:41:55.238 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 11 23:41:54 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-8[53569]: 2024-09-11T23:41:54.793+0000 7f99e4062640 -1 osd.8 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:53.978855+0000 front 2024-09-11T23:40:53.978923+0000 (oldest deadline 2024-09-11T23:41:18.679037+0000) 2024-09-11T23:41:55.239 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 11 23:41:54 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-8[53569]: 2024-09-11T23:41:54.793+0000 7f99e4062640 -1 osd.8 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:10.281739+0000 front 2024-09-11T23:41:10.281758+0000 (oldest deadline 2024-09-11T23:41:39.682327+0000) 2024-09-11T23:41:55.239 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 11 23:41:54 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-8[53569]: 2024-09-11T23:41:54.793+0000 7f99e4062640 -1 osd.8 120 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-09-11T23:41:19.683087+0000 front 2024-09-11T23:41:19.683104+0000 (oldest deadline 2024-09-11T23:41:43.182872+0000) 2024-09-11T23:41:55.239 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 11 23:41:54 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-8[53569]: 2024-09-11T23:41:54.793+0000 7f99e4062640 -1 osd.8 120 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-09-11T23:41:30.184463+0000 front 2024-09-11T23:41:30.184492+0000 (oldest deadline 2024-09-11T23:41:53.684668+0000) 2024-09-11T23:41:55.250 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:41:54 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9[65389]: 2024-09-11T23:41:54.949+0000 7fc7513f1640 -1 osd.9 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:57.370068+0000 front 2024-09-11T23:40:57.370191+0000 (oldest deadline 2024-09-11T23:41:21.470122+0000) 2024-09-11T23:41:55.250 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:41:54 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9[65389]: 2024-09-11T23:41:54.949+0000 7fc7513f1640 -1 osd.9 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:04.971207+0000 front 2024-09-11T23:41:04.971047+0000 (oldest deadline 2024-09-11T23:41:27.271258+0000) 2024-09-11T23:41:55.250 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:41:54 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9[65389]: 2024-09-11T23:41:54.949+0000 7fc7513f1640 -1 osd.9 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:13.974640+0000 front 2024-09-11T23:41:13.974576+0000 (oldest deadline 2024-09-11T23:41:38.074690+0000) 2024-09-11T23:41:55.250 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:41:54 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9[65389]: 2024-09-11T23:41:54.949+0000 7fc7513f1640 -1 osd.9 120 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-09-11T23:41:18.075080+0000 front 2024-09-11T23:41:18.075257+0000 (oldest deadline 2024-09-11T23:41:43.375370+0000) 2024-09-11T23:41:55.503 INFO:journalctl@ceph.osd.7.smithi076.stdout:Sep 11 23:41:55 smithi076 podman[73702]: 2024-09-11 23:41:55.176990407 +0000 UTC m=+5.903027669 container died c69a833bbf62b45b8dd7339607c5cc4d61fb2de53c41d0cb97a7a760a02559df (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-7, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-09-11T23:41:55.764 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:41:55 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:41:55.510+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:56.932175+0000 front 2024-09-11T23:40:56.932499+0000 (oldest deadline 2024-09-11T23:41:21.032317+0000) 2024-09-11T23:41:55.764 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:41:55 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:41:55.510+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:01.032761+0000 front 2024-09-11T23:41:01.032638+0000 (oldest deadline 2024-09-11T23:41:30.433335+0000) 2024-09-11T23:41:55.764 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:41:55 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:41:55.510+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:10.934510+0000 front 2024-09-11T23:41:10.934411+0000 (oldest deadline 2024-09-11T23:41:35.634455+0000) 2024-09-11T23:41:55.764 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:41:55 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:41:55.510+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-09-11T23:41:21.936325+0000 front 2024-09-11T23:41:21.936462+0000 (oldest deadline 2024-09-11T23:41:42.937021+0000) 2024-09-11T23:41:55.764 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:41:55 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:41:55.510+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-09-11T23:41:27.038312+0000 front 2024-09-11T23:41:27.038192+0000 (oldest deadline 2024-09-11T23:41:51.638385+0000) 2024-09-11T23:41:55.802 INFO:journalctl@ceph.osd.7.smithi076.stdout:Sep 11 23:41:55 smithi076 podman[73702]: 2024-09-11 23:41:55.604145153 +0000 UTC m=+6.330182419 container cleanup c69a833bbf62b45b8dd7339607c5cc4d61fb2de53c41d0cb97a7a760a02559df (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-7, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T23:41:55.802 INFO:journalctl@ceph.osd.7.smithi076.stdout:Sep 11 23:41:55 smithi076 bash[73702]: ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-7 2024-09-11T23:41:55.802 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:41:55 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:41:55.565+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:57.536934+0000 front 2024-09-11T23:40:57.537030+0000 (oldest deadline 2024-09-11T23:41:21.036908+0000) 2024-09-11T23:41:55.803 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:41:55 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:41:55.565+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:03.838149+0000 front 2024-09-11T23:41:03.838269+0000 (oldest deadline 2024-09-11T23:41:29.138274+0000) 2024-09-11T23:41:55.803 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:41:55 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:41:55.565+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:12.539763+0000 front 2024-09-11T23:41:12.539893+0000 (oldest deadline 2024-09-11T23:41:36.639808+0000) 2024-09-11T23:41:55.803 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:41:55 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:41:55.565+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-09-11T23:41:21.841541+0000 front 2024-09-11T23:41:21.841518+0000 (oldest deadline 2024-09-11T23:41:44.141586+0000) 2024-09-11T23:41:55.803 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:41:55 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:41:55.565+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-09-11T23:41:28.142849+0000 front 2024-09-11T23:41:28.142868+0000 (oldest deadline 2024-09-11T23:41:54.042918+0000) 2024-09-11T23:41:56.215 INFO:journalctl@ceph.osd.7.smithi076.stdout:Sep 11 23:41:55 smithi076 podman[73717]: 2024-09-11 23:41:55.821749923 +0000 UTC m=+0.647381978 container remove c69a833bbf62b45b8dd7339607c5cc4d61fb2de53c41d0cb97a7a760a02559df (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-7, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-09-11T23:41:56.238 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 11 23:41:55 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-8[53569]: 2024-09-11T23:41:55.764+0000 7f99e4062640 -1 osd.8 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:53.978855+0000 front 2024-09-11T23:40:53.978923+0000 (oldest deadline 2024-09-11T23:41:18.679037+0000) 2024-09-11T23:41:56.238 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 11 23:41:55 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-8[53569]: 2024-09-11T23:41:55.764+0000 7f99e4062640 -1 osd.8 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:10.281739+0000 front 2024-09-11T23:41:10.281758+0000 (oldest deadline 2024-09-11T23:41:39.682327+0000) 2024-09-11T23:41:56.238 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 11 23:41:55 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-8[53569]: 2024-09-11T23:41:55.764+0000 7f99e4062640 -1 osd.8 120 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-09-11T23:41:19.683087+0000 front 2024-09-11T23:41:19.683104+0000 (oldest deadline 2024-09-11T23:41:43.182872+0000) 2024-09-11T23:41:56.239 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 11 23:41:55 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-8[53569]: 2024-09-11T23:41:55.764+0000 7f99e4062640 -1 osd.8 120 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-09-11T23:41:30.184463+0000 front 2024-09-11T23:41:30.184492+0000 (oldest deadline 2024-09-11T23:41:53.684668+0000) 2024-09-11T23:41:56.250 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:41:55 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9[65389]: 2024-09-11T23:41:55.920+0000 7fc7513f1640 -1 osd.9 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:57.370068+0000 front 2024-09-11T23:40:57.370191+0000 (oldest deadline 2024-09-11T23:41:21.470122+0000) 2024-09-11T23:41:56.250 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:41:55 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9[65389]: 2024-09-11T23:41:55.920+0000 7fc7513f1640 -1 osd.9 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:04.971207+0000 front 2024-09-11T23:41:04.971047+0000 (oldest deadline 2024-09-11T23:41:27.271258+0000) 2024-09-11T23:41:56.250 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:41:55 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9[65389]: 2024-09-11T23:41:55.920+0000 7fc7513f1640 -1 osd.9 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:13.974640+0000 front 2024-09-11T23:41:13.974576+0000 (oldest deadline 2024-09-11T23:41:38.074690+0000) 2024-09-11T23:41:56.250 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:41:55 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9[65389]: 2024-09-11T23:41:55.920+0000 7fc7513f1640 -1 osd.9 120 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-09-11T23:41:18.075080+0000 front 2024-09-11T23:41:18.075257+0000 (oldest deadline 2024-09-11T23:41:43.375370+0000) 2024-09-11T23:41:56.523 INFO:journalctl@ceph.osd.7.smithi076.stdout:Sep 11 23:41:56 smithi076 podman[73804]: 2024-09-11 23:41:56.118694422 +0000 UTC m=+0.015817984 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T23:41:56.523 INFO:journalctl@ceph.osd.7.smithi076.stdout:Sep 11 23:41:56 smithi076 podman[73804]: 2024-09-11 23:41:56.222570693 +0000 UTC m=+0.119694283 container create e004274f0bce27cc12d4d8ffb2ec54450ed32d52a00fff5d809b8a258734380b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-7-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_CLEAN=True) 2024-09-11T23:41:56.523 INFO:journalctl@ceph.osd.7.smithi076.stdout:Sep 11 23:41:56 smithi076 podman[73804]: 2024-09-11 23:41:56.441461094 +0000 UTC m=+0.338584643 container init e004274f0bce27cc12d4d8ffb2ec54450ed32d52a00fff5d809b8a258734380b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-7-deactivate, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.license=GPLv2) 2024-09-11T23:41:56.524 INFO:journalctl@ceph.osd.7.smithi076.stdout:Sep 11 23:41:56 smithi076 podman[73804]: 2024-09-11 23:41:56.444883398 +0000 UTC m=+0.342006947 container start e004274f0bce27cc12d4d8ffb2ec54450ed32d52a00fff5d809b8a258734380b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-7-deactivate, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, ceph=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T23:41:56.524 INFO:journalctl@ceph.osd.7.smithi076.stdout:Sep 11 23:41:56 smithi076 podman[73804]: 2024-09-11 23:41:56.524559263 +0000 UTC m=+0.421682825 container attach e004274f0bce27cc12d4d8ffb2ec54450ed32d52a00fff5d809b8a258734380b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-7-deactivate, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731) 2024-09-11T23:41:56.802 INFO:journalctl@ceph.osd.7.smithi076.stdout:Sep 11 23:41:56 smithi076 podman[73804]: 2024-09-11 23:41:56.632314511 +0000 UTC m=+0.529438068 container died e004274f0bce27cc12d4d8ffb2ec54450ed32d52a00fff5d809b8a258734380b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-7-deactivate, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T23:41:56.803 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:41:56 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:41:56.573+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:57.536934+0000 front 2024-09-11T23:40:57.537030+0000 (oldest deadline 2024-09-11T23:41:21.036908+0000) 2024-09-11T23:41:56.803 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:41:56 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:41:56.573+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:03.838149+0000 front 2024-09-11T23:41:03.838269+0000 (oldest deadline 2024-09-11T23:41:29.138274+0000) 2024-09-11T23:41:56.803 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:41:56 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:41:56.573+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:12.539763+0000 front 2024-09-11T23:41:12.539893+0000 (oldest deadline 2024-09-11T23:41:36.639808+0000) 2024-09-11T23:41:56.803 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:41:56 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:41:56.573+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-09-11T23:41:21.841541+0000 front 2024-09-11T23:41:21.841518+0000 (oldest deadline 2024-09-11T23:41:44.141586+0000) 2024-09-11T23:41:56.803 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:41:56 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:41:56.573+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-09-11T23:41:28.142849+0000 front 2024-09-11T23:41:28.142868+0000 (oldest deadline 2024-09-11T23:41:54.042918+0000) 2024-09-11T23:41:56.810 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:41:56 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:41:56.540+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:56.932175+0000 front 2024-09-11T23:40:56.932499+0000 (oldest deadline 2024-09-11T23:41:21.032317+0000) 2024-09-11T23:41:56.810 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:41:56 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:41:56.540+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:01.032761+0000 front 2024-09-11T23:41:01.032638+0000 (oldest deadline 2024-09-11T23:41:30.433335+0000) 2024-09-11T23:41:56.810 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:41:56 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:41:56.540+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:10.934510+0000 front 2024-09-11T23:41:10.934411+0000 (oldest deadline 2024-09-11T23:41:35.634455+0000) 2024-09-11T23:41:56.810 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:41:56 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:41:56.540+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-09-11T23:41:21.936325+0000 front 2024-09-11T23:41:21.936462+0000 (oldest deadline 2024-09-11T23:41:42.937021+0000) 2024-09-11T23:41:56.810 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:41:56 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:41:56.540+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-09-11T23:41:27.038312+0000 front 2024-09-11T23:41:27.038192+0000 (oldest deadline 2024-09-11T23:41:51.638385+0000) 2024-09-11T23:41:57.208 DEBUG:teuthology.orchestra.run.smithi076:> sudo pkill -f 'journalctl -f -n 0 -u ceph-af7e6fc8-7092-11ef-bcea-c7b262605968@osd.7.service' 2024-09-11T23:41:57.238 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 11 23:41:56 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-8[53569]: 2024-09-11T23:41:56.810+0000 7f99e4062640 -1 osd.8 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:53.978855+0000 front 2024-09-11T23:40:53.978923+0000 (oldest deadline 2024-09-11T23:41:18.679037+0000) 2024-09-11T23:41:57.238 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 11 23:41:56 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-8[53569]: 2024-09-11T23:41:56.810+0000 7f99e4062640 -1 osd.8 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:10.281739+0000 front 2024-09-11T23:41:10.281758+0000 (oldest deadline 2024-09-11T23:41:39.682327+0000) 2024-09-11T23:41:57.239 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 11 23:41:56 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-8[53569]: 2024-09-11T23:41:56.810+0000 7f99e4062640 -1 osd.8 120 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-09-11T23:41:19.683087+0000 front 2024-09-11T23:41:19.683104+0000 (oldest deadline 2024-09-11T23:41:43.182872+0000) 2024-09-11T23:41:57.239 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 11 23:41:56 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-8[53569]: 2024-09-11T23:41:56.810+0000 7f99e4062640 -1 osd.8 120 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-09-11T23:41:30.184463+0000 front 2024-09-11T23:41:30.184492+0000 (oldest deadline 2024-09-11T23:41:53.684668+0000) 2024-09-11T23:41:57.248 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T23:41:57.248 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-09-11T23:41:57.248 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-09-11T23:41:57.249 DEBUG:teuthology.orchestra.run.smithi136:> sudo systemctl stop ceph-af7e6fc8-7092-11ef-bcea-c7b262605968@osd.8 2024-09-11T23:41:57.250 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:41:56 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9[65389]: 2024-09-11T23:41:56.897+0000 7fc7513f1640 -1 osd.9 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:57.370068+0000 front 2024-09-11T23:40:57.370191+0000 (oldest deadline 2024-09-11T23:41:21.470122+0000) 2024-09-11T23:41:57.250 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:41:56 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9[65389]: 2024-09-11T23:41:56.897+0000 7fc7513f1640 -1 osd.9 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:04.971207+0000 front 2024-09-11T23:41:04.971047+0000 (oldest deadline 2024-09-11T23:41:27.271258+0000) 2024-09-11T23:41:57.250 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:41:56 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9[65389]: 2024-09-11T23:41:56.897+0000 7fc7513f1640 -1 osd.9 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:13.974640+0000 front 2024-09-11T23:41:13.974576+0000 (oldest deadline 2024-09-11T23:41:38.074690+0000) 2024-09-11T23:41:57.250 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:41:56 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9[65389]: 2024-09-11T23:41:56.897+0000 7fc7513f1640 -1 osd.9 120 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-09-11T23:41:18.075080+0000 front 2024-09-11T23:41:18.075257+0000 (oldest deadline 2024-09-11T23:41:43.375370+0000) 2024-09-11T23:41:57.250 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:41:56 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9[65389]: 2024-09-11T23:41:56.897+0000 7fc7513f1640 -1 osd.9 120 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-09-11T23:41:29.578379+0000 front 2024-09-11T23:41:29.578251+0000 (oldest deadline 2024-09-11T23:41:56.578836+0000) 2024-09-11T23:41:57.536 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 11 23:41:57 smithi136 systemd[1]: Stopping Ceph osd.8 for af7e6fc8-7092-11ef-bcea-c7b262605968... 2024-09-11T23:41:57.802 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:41:57 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:41:57.580+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:56.932175+0000 front 2024-09-11T23:40:56.932499+0000 (oldest deadline 2024-09-11T23:41:21.032317+0000) 2024-09-11T23:41:57.802 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:41:57 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:41:57.580+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:01.032761+0000 front 2024-09-11T23:41:01.032638+0000 (oldest deadline 2024-09-11T23:41:30.433335+0000) 2024-09-11T23:41:57.802 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:41:57 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:41:57.580+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:10.934510+0000 front 2024-09-11T23:41:10.934411+0000 (oldest deadline 2024-09-11T23:41:35.634455+0000) 2024-09-11T23:41:57.802 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:41:57 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:41:57.580+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-09-11T23:41:21.936325+0000 front 2024-09-11T23:41:21.936462+0000 (oldest deadline 2024-09-11T23:41:42.937021+0000) 2024-09-11T23:41:57.802 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:41:57 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:41:57.580+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-09-11T23:41:27.038312+0000 front 2024-09-11T23:41:27.038192+0000 (oldest deadline 2024-09-11T23:41:51.638385+0000) 2024-09-11T23:41:57.802 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 11 23:41:57 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-8[53569]: 2024-09-11T23:41:57.536+0000 7f99e7a5d640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T23:41:57.803 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 11 23:41:57 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-8[53569]: 2024-09-11T23:41:57.536+0000 7f99e7a5d640 -1 osd.8 120 *** Got signal Terminated *** 2024-09-11T23:41:57.803 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 11 23:41:57 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-8[53569]: 2024-09-11T23:41:57.536+0000 7f99e7a5d640 -1 osd.8 120 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T23:41:57.803 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 11 23:41:57 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-8[53569]: 2024-09-11T23:41:57.801+0000 7f99e4062640 -1 osd.8 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:53.978855+0000 front 2024-09-11T23:40:53.978923+0000 (oldest deadline 2024-09-11T23:41:18.679037+0000) 2024-09-11T23:41:57.803 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 11 23:41:57 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-8[53569]: 2024-09-11T23:41:57.801+0000 7f99e4062640 -1 osd.8 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:10.281739+0000 front 2024-09-11T23:41:10.281758+0000 (oldest deadline 2024-09-11T23:41:39.682327+0000) 2024-09-11T23:41:57.803 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 11 23:41:57 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-8[53569]: 2024-09-11T23:41:57.801+0000 7f99e4062640 -1 osd.8 120 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-09-11T23:41:19.683087+0000 front 2024-09-11T23:41:19.683104+0000 (oldest deadline 2024-09-11T23:41:43.182872+0000) 2024-09-11T23:41:57.803 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 11 23:41:57 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-8[53569]: 2024-09-11T23:41:57.801+0000 7f99e4062640 -1 osd.8 120 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-09-11T23:41:30.184463+0000 front 2024-09-11T23:41:30.184492+0000 (oldest deadline 2024-09-11T23:41:53.684668+0000) 2024-09-11T23:41:57.803 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:41:57 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:41:57.609+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:57.536934+0000 front 2024-09-11T23:40:57.537030+0000 (oldest deadline 2024-09-11T23:41:21.036908+0000) 2024-09-11T23:41:57.804 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:41:57 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:41:57.609+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:03.838149+0000 front 2024-09-11T23:41:03.838269+0000 (oldest deadline 2024-09-11T23:41:29.138274+0000) 2024-09-11T23:41:57.804 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:41:57 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:41:57.609+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:12.539763+0000 front 2024-09-11T23:41:12.539893+0000 (oldest deadline 2024-09-11T23:41:36.639808+0000) 2024-09-11T23:41:57.804 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:41:57 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:41:57.609+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-09-11T23:41:21.841541+0000 front 2024-09-11T23:41:21.841518+0000 (oldest deadline 2024-09-11T23:41:44.141586+0000) 2024-09-11T23:41:57.804 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:41:57 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:41:57.609+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-09-11T23:41:28.142849+0000 front 2024-09-11T23:41:28.142868+0000 (oldest deadline 2024-09-11T23:41:54.042918+0000) 2024-09-11T23:41:58.250 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:41:57 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9[65389]: 2024-09-11T23:41:57.878+0000 7fc7513f1640 -1 osd.9 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:57.370068+0000 front 2024-09-11T23:40:57.370191+0000 (oldest deadline 2024-09-11T23:41:21.470122+0000) 2024-09-11T23:41:58.250 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:41:57 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9[65389]: 2024-09-11T23:41:57.878+0000 7fc7513f1640 -1 osd.9 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:04.971207+0000 front 2024-09-11T23:41:04.971047+0000 (oldest deadline 2024-09-11T23:41:27.271258+0000) 2024-09-11T23:41:58.250 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:41:57 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9[65389]: 2024-09-11T23:41:57.878+0000 7fc7513f1640 -1 osd.9 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:13.974640+0000 front 2024-09-11T23:41:13.974576+0000 (oldest deadline 2024-09-11T23:41:38.074690+0000) 2024-09-11T23:41:58.251 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:41:57 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9[65389]: 2024-09-11T23:41:57.878+0000 7fc7513f1640 -1 osd.9 120 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-09-11T23:41:18.075080+0000 front 2024-09-11T23:41:18.075257+0000 (oldest deadline 2024-09-11T23:41:43.375370+0000) 2024-09-11T23:41:58.251 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:41:57 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9[65389]: 2024-09-11T23:41:57.878+0000 7fc7513f1640 -1 osd.9 120 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-09-11T23:41:29.578379+0000 front 2024-09-11T23:41:29.578251+0000 (oldest deadline 2024-09-11T23:41:56.578836+0000) 2024-09-11T23:41:58.802 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:41:58 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:41:58.571+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:57.536934+0000 front 2024-09-11T23:40:57.537030+0000 (oldest deadline 2024-09-11T23:41:21.036908+0000) 2024-09-11T23:41:58.802 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:41:58 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:41:58.571+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:03.838149+0000 front 2024-09-11T23:41:03.838269+0000 (oldest deadline 2024-09-11T23:41:29.138274+0000) 2024-09-11T23:41:58.802 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:41:58 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:41:58.571+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:12.539763+0000 front 2024-09-11T23:41:12.539893+0000 (oldest deadline 2024-09-11T23:41:36.639808+0000) 2024-09-11T23:41:58.803 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:41:58 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:41:58.571+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-09-11T23:41:21.841541+0000 front 2024-09-11T23:41:21.841518+0000 (oldest deadline 2024-09-11T23:41:44.141586+0000) 2024-09-11T23:41:58.803 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:41:58 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:41:58.571+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-09-11T23:41:28.142849+0000 front 2024-09-11T23:41:28.142868+0000 (oldest deadline 2024-09-11T23:41:54.042918+0000) 2024-09-11T23:41:58.850 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:41:58 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:41:58.539+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:56.932175+0000 front 2024-09-11T23:40:56.932499+0000 (oldest deadline 2024-09-11T23:41:21.032317+0000) 2024-09-11T23:41:58.850 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:41:58 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:41:58.539+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:01.032761+0000 front 2024-09-11T23:41:01.032638+0000 (oldest deadline 2024-09-11T23:41:30.433335+0000) 2024-09-11T23:41:58.850 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:41:58 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:41:58.539+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:10.934510+0000 front 2024-09-11T23:41:10.934411+0000 (oldest deadline 2024-09-11T23:41:35.634455+0000) 2024-09-11T23:41:58.851 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:41:58 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:41:58.539+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-09-11T23:41:21.936325+0000 front 2024-09-11T23:41:21.936462+0000 (oldest deadline 2024-09-11T23:41:42.937021+0000) 2024-09-11T23:41:58.851 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:41:58 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:41:58.539+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-09-11T23:41:27.038312+0000 front 2024-09-11T23:41:27.038192+0000 (oldest deadline 2024-09-11T23:41:51.638385+0000) 2024-09-11T23:41:59.242 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 11 23:41:58 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-8[53569]: 2024-09-11T23:41:58.850+0000 7f99e4062640 -1 osd.8 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:53.978855+0000 front 2024-09-11T23:40:53.978923+0000 (oldest deadline 2024-09-11T23:41:18.679037+0000) 2024-09-11T23:41:59.242 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 11 23:41:58 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-8[53569]: 2024-09-11T23:41:58.850+0000 7f99e4062640 -1 osd.8 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:10.281739+0000 front 2024-09-11T23:41:10.281758+0000 (oldest deadline 2024-09-11T23:41:39.682327+0000) 2024-09-11T23:41:59.242 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 11 23:41:58 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-8[53569]: 2024-09-11T23:41:58.850+0000 7f99e4062640 -1 osd.8 120 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-09-11T23:41:19.683087+0000 front 2024-09-11T23:41:19.683104+0000 (oldest deadline 2024-09-11T23:41:43.182872+0000) 2024-09-11T23:41:59.242 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 11 23:41:58 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-8[53569]: 2024-09-11T23:41:58.850+0000 7f99e4062640 -1 osd.8 120 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-09-11T23:41:30.184463+0000 front 2024-09-11T23:41:30.184492+0000 (oldest deadline 2024-09-11T23:41:53.684668+0000) 2024-09-11T23:41:59.250 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:41:58 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9[65389]: 2024-09-11T23:41:58.908+0000 7fc7513f1640 -1 osd.9 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:57.370068+0000 front 2024-09-11T23:40:57.370191+0000 (oldest deadline 2024-09-11T23:41:21.470122+0000) 2024-09-11T23:41:59.250 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:41:58 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9[65389]: 2024-09-11T23:41:58.908+0000 7fc7513f1640 -1 osd.9 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:04.971207+0000 front 2024-09-11T23:41:04.971047+0000 (oldest deadline 2024-09-11T23:41:27.271258+0000) 2024-09-11T23:41:59.250 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:41:58 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9[65389]: 2024-09-11T23:41:58.908+0000 7fc7513f1640 -1 osd.9 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:13.974640+0000 front 2024-09-11T23:41:13.974576+0000 (oldest deadline 2024-09-11T23:41:38.074690+0000) 2024-09-11T23:41:59.250 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:41:58 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9[65389]: 2024-09-11T23:41:58.908+0000 7fc7513f1640 -1 osd.9 120 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-09-11T23:41:18.075080+0000 front 2024-09-11T23:41:18.075257+0000 (oldest deadline 2024-09-11T23:41:43.375370+0000) 2024-09-11T23:41:59.250 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:41:58 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9[65389]: 2024-09-11T23:41:58.908+0000 7fc7513f1640 -1 osd.9 120 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-09-11T23:41:29.578379+0000 front 2024-09-11T23:41:29.578251+0000 (oldest deadline 2024-09-11T23:41:56.578836+0000) 2024-09-11T23:41:59.886 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:41:59 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:41:59.507+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:56.932175+0000 front 2024-09-11T23:40:56.932499+0000 (oldest deadline 2024-09-11T23:41:21.032317+0000) 2024-09-11T23:41:59.887 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:41:59 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:41:59.507+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:01.032761+0000 front 2024-09-11T23:41:01.032638+0000 (oldest deadline 2024-09-11T23:41:30.433335+0000) 2024-09-11T23:41:59.887 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:41:59 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:41:59.507+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:10.934510+0000 front 2024-09-11T23:41:10.934411+0000 (oldest deadline 2024-09-11T23:41:35.634455+0000) 2024-09-11T23:41:59.887 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:41:59 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:41:59.507+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-09-11T23:41:21.936325+0000 front 2024-09-11T23:41:21.936462+0000 (oldest deadline 2024-09-11T23:41:42.937021+0000) 2024-09-11T23:41:59.887 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:41:59 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:41:59.507+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-09-11T23:41:27.038312+0000 front 2024-09-11T23:41:27.038192+0000 (oldest deadline 2024-09-11T23:41:51.638385+0000) 2024-09-11T23:42:00.052 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:41:59 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:41:59.568+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:57.536934+0000 front 2024-09-11T23:40:57.537030+0000 (oldest deadline 2024-09-11T23:41:21.036908+0000) 2024-09-11T23:42:00.052 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:41:59 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:41:59.568+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:03.838149+0000 front 2024-09-11T23:41:03.838269+0000 (oldest deadline 2024-09-11T23:41:29.138274+0000) 2024-09-11T23:42:00.053 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:41:59 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:41:59.568+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:12.539763+0000 front 2024-09-11T23:41:12.539893+0000 (oldest deadline 2024-09-11T23:41:36.639808+0000) 2024-09-11T23:42:00.053 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:41:59 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:41:59.568+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-09-11T23:41:21.841541+0000 front 2024-09-11T23:41:21.841518+0000 (oldest deadline 2024-09-11T23:41:44.141586+0000) 2024-09-11T23:42:00.053 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:41:59 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:41:59.568+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-09-11T23:41:28.142849+0000 front 2024-09-11T23:41:28.142868+0000 (oldest deadline 2024-09-11T23:41:54.042918+0000) 2024-09-11T23:42:00.158 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 11 23:41:59 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-8[53569]: 2024-09-11T23:41:59.886+0000 7f99e4062640 -1 osd.8 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:53.978855+0000 front 2024-09-11T23:40:53.978923+0000 (oldest deadline 2024-09-11T23:41:18.679037+0000) 2024-09-11T23:42:00.158 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 11 23:41:59 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-8[53569]: 2024-09-11T23:41:59.886+0000 7f99e4062640 -1 osd.8 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:10.281739+0000 front 2024-09-11T23:41:10.281758+0000 (oldest deadline 2024-09-11T23:41:39.682327+0000) 2024-09-11T23:42:00.158 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 11 23:41:59 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-8[53569]: 2024-09-11T23:41:59.886+0000 7f99e4062640 -1 osd.8 120 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-09-11T23:41:19.683087+0000 front 2024-09-11T23:41:19.683104+0000 (oldest deadline 2024-09-11T23:41:43.182872+0000) 2024-09-11T23:42:00.158 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 11 23:41:59 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-8[53569]: 2024-09-11T23:41:59.886+0000 7f99e4062640 -1 osd.8 120 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-09-11T23:41:30.184463+0000 front 2024-09-11T23:41:30.184492+0000 (oldest deadline 2024-09-11T23:41:53.684668+0000) 2024-09-11T23:42:00.250 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:41:59 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9[65389]: 2024-09-11T23:41:59.939+0000 7fc7513f1640 -1 osd.9 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:57.370068+0000 front 2024-09-11T23:40:57.370191+0000 (oldest deadline 2024-09-11T23:41:21.470122+0000) 2024-09-11T23:42:00.250 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:41:59 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9[65389]: 2024-09-11T23:41:59.939+0000 7fc7513f1640 -1 osd.9 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:04.971207+0000 front 2024-09-11T23:41:04.971047+0000 (oldest deadline 2024-09-11T23:41:27.271258+0000) 2024-09-11T23:42:00.250 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:41:59 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9[65389]: 2024-09-11T23:41:59.939+0000 7fc7513f1640 -1 osd.9 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:13.974640+0000 front 2024-09-11T23:41:13.974576+0000 (oldest deadline 2024-09-11T23:41:38.074690+0000) 2024-09-11T23:42:00.251 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:41:59 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9[65389]: 2024-09-11T23:41:59.939+0000 7fc7513f1640 -1 osd.9 120 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-09-11T23:41:18.075080+0000 front 2024-09-11T23:41:18.075257+0000 (oldest deadline 2024-09-11T23:41:43.375370+0000) 2024-09-11T23:42:00.251 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:41:59 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9[65389]: 2024-09-11T23:41:59.939+0000 7fc7513f1640 -1 osd.9 120 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-09-11T23:41:29.578379+0000 front 2024-09-11T23:41:29.578251+0000 (oldest deadline 2024-09-11T23:41:56.578836+0000) 2024-09-11T23:42:00.802 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:42:00 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:42:00.552+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:57.536934+0000 front 2024-09-11T23:40:57.537030+0000 (oldest deadline 2024-09-11T23:41:21.036908+0000) 2024-09-11T23:42:00.802 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:42:00 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:42:00.552+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:03.838149+0000 front 2024-09-11T23:41:03.838269+0000 (oldest deadline 2024-09-11T23:41:29.138274+0000) 2024-09-11T23:42:00.802 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:42:00 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:42:00.552+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:12.539763+0000 front 2024-09-11T23:41:12.539893+0000 (oldest deadline 2024-09-11T23:41:36.639808+0000) 2024-09-11T23:42:00.803 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:42:00 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:42:00.552+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-09-11T23:41:21.841541+0000 front 2024-09-11T23:41:21.841518+0000 (oldest deadline 2024-09-11T23:41:44.141586+0000) 2024-09-11T23:42:00.803 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:42:00 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:42:00.552+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-09-11T23:41:28.142849+0000 front 2024-09-11T23:41:28.142868+0000 (oldest deadline 2024-09-11T23:41:54.042918+0000) 2024-09-11T23:42:00.931 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:00 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:42:00.554+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:56.932175+0000 front 2024-09-11T23:40:56.932499+0000 (oldest deadline 2024-09-11T23:41:21.032317+0000) 2024-09-11T23:42:00.931 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:00 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:42:00.554+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:01.032761+0000 front 2024-09-11T23:41:01.032638+0000 (oldest deadline 2024-09-11T23:41:30.433335+0000) 2024-09-11T23:42:00.931 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:00 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:42:00.554+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:10.934510+0000 front 2024-09-11T23:41:10.934411+0000 (oldest deadline 2024-09-11T23:41:35.634455+0000) 2024-09-11T23:42:00.931 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:00 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:42:00.554+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-09-11T23:41:21.936325+0000 front 2024-09-11T23:41:21.936462+0000 (oldest deadline 2024-09-11T23:41:42.937021+0000) 2024-09-11T23:42:00.932 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:00 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:42:00.554+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-09-11T23:41:27.038312+0000 front 2024-09-11T23:41:27.038192+0000 (oldest deadline 2024-09-11T23:41:51.638385+0000) 2024-09-11T23:42:01.213 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 11 23:42:00 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-8[53569]: 2024-09-11T23:42:00.930+0000 7f99e4062640 -1 osd.8 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:53.978855+0000 front 2024-09-11T23:40:53.978923+0000 (oldest deadline 2024-09-11T23:41:18.679037+0000) 2024-09-11T23:42:01.213 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 11 23:42:00 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-8[53569]: 2024-09-11T23:42:00.930+0000 7f99e4062640 -1 osd.8 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:10.281739+0000 front 2024-09-11T23:41:10.281758+0000 (oldest deadline 2024-09-11T23:41:39.682327+0000) 2024-09-11T23:42:01.213 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 11 23:42:00 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-8[53569]: 2024-09-11T23:42:00.930+0000 7f99e4062640 -1 osd.8 120 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-09-11T23:41:19.683087+0000 front 2024-09-11T23:41:19.683104+0000 (oldest deadline 2024-09-11T23:41:43.182872+0000) 2024-09-11T23:42:01.213 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 11 23:42:00 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-8[53569]: 2024-09-11T23:42:00.930+0000 7f99e4062640 -1 osd.8 120 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-09-11T23:41:30.184463+0000 front 2024-09-11T23:41:30.184492+0000 (oldest deadline 2024-09-11T23:41:53.684668+0000) 2024-09-11T23:42:01.250 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:42:00 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9[65389]: 2024-09-11T23:42:00.973+0000 7fc7513f1640 -1 osd.9 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:57.370068+0000 front 2024-09-11T23:40:57.370191+0000 (oldest deadline 2024-09-11T23:41:21.470122+0000) 2024-09-11T23:42:01.250 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:42:00 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9[65389]: 2024-09-11T23:42:00.973+0000 7fc7513f1640 -1 osd.9 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:04.971207+0000 front 2024-09-11T23:41:04.971047+0000 (oldest deadline 2024-09-11T23:41:27.271258+0000) 2024-09-11T23:42:01.250 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:42:00 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9[65389]: 2024-09-11T23:42:00.973+0000 7fc7513f1640 -1 osd.9 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:13.974640+0000 front 2024-09-11T23:41:13.974576+0000 (oldest deadline 2024-09-11T23:41:38.074690+0000) 2024-09-11T23:42:01.250 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:42:00 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9[65389]: 2024-09-11T23:42:00.973+0000 7fc7513f1640 -1 osd.9 120 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-09-11T23:41:18.075080+0000 front 2024-09-11T23:41:18.075257+0000 (oldest deadline 2024-09-11T23:41:43.375370+0000) 2024-09-11T23:42:01.250 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:42:00 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9[65389]: 2024-09-11T23:42:00.973+0000 7fc7513f1640 -1 osd.9 120 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-09-11T23:41:29.578379+0000 front 2024-09-11T23:41:29.578251+0000 (oldest deadline 2024-09-11T23:41:56.578836+0000) 2024-09-11T23:42:01.802 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:42:01 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:42:01.534+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:57.536934+0000 front 2024-09-11T23:40:57.537030+0000 (oldest deadline 2024-09-11T23:41:21.036908+0000) 2024-09-11T23:42:01.802 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:42:01 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:42:01.534+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:03.838149+0000 front 2024-09-11T23:41:03.838269+0000 (oldest deadline 2024-09-11T23:41:29.138274+0000) 2024-09-11T23:42:01.802 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:42:01 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:42:01.534+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:12.539763+0000 front 2024-09-11T23:41:12.539893+0000 (oldest deadline 2024-09-11T23:41:36.639808+0000) 2024-09-11T23:42:01.802 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:42:01 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:42:01.534+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-09-11T23:41:21.841541+0000 front 2024-09-11T23:41:21.841518+0000 (oldest deadline 2024-09-11T23:41:44.141586+0000) 2024-09-11T23:42:01.802 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:42:01 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:42:01.534+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-09-11T23:41:28.142849+0000 front 2024-09-11T23:41:28.142868+0000 (oldest deadline 2024-09-11T23:41:54.042918+0000) 2024-09-11T23:42:01.948 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:01 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:42:01.578+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:56.932175+0000 front 2024-09-11T23:40:56.932499+0000 (oldest deadline 2024-09-11T23:41:21.032317+0000) 2024-09-11T23:42:01.948 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:01 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:42:01.578+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:01.032761+0000 front 2024-09-11T23:41:01.032638+0000 (oldest deadline 2024-09-11T23:41:30.433335+0000) 2024-09-11T23:42:01.948 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:01 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:42:01.578+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:10.934510+0000 front 2024-09-11T23:41:10.934411+0000 (oldest deadline 2024-09-11T23:41:35.634455+0000) 2024-09-11T23:42:01.948 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:01 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:42:01.578+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-09-11T23:41:21.936325+0000 front 2024-09-11T23:41:21.936462+0000 (oldest deadline 2024-09-11T23:41:42.937021+0000) 2024-09-11T23:42:01.948 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:01 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:42:01.578+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-09-11T23:41:27.038312+0000 front 2024-09-11T23:41:27.038192+0000 (oldest deadline 2024-09-11T23:41:51.638385+0000) 2024-09-11T23:42:02.238 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 11 23:42:01 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-8[53569]: 2024-09-11T23:42:01.946+0000 7f99e4062640 -1 osd.8 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:53.978855+0000 front 2024-09-11T23:40:53.978923+0000 (oldest deadline 2024-09-11T23:41:18.679037+0000) 2024-09-11T23:42:02.239 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 11 23:42:01 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-8[53569]: 2024-09-11T23:42:01.946+0000 7f99e4062640 -1 osd.8 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:10.281739+0000 front 2024-09-11T23:41:10.281758+0000 (oldest deadline 2024-09-11T23:41:39.682327+0000) 2024-09-11T23:42:02.239 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 11 23:42:01 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-8[53569]: 2024-09-11T23:42:01.946+0000 7f99e4062640 -1 osd.8 120 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-09-11T23:41:19.683087+0000 front 2024-09-11T23:41:19.683104+0000 (oldest deadline 2024-09-11T23:41:43.182872+0000) 2024-09-11T23:42:02.239 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 11 23:42:01 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-8[53569]: 2024-09-11T23:42:01.946+0000 7f99e4062640 -1 osd.8 120 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-09-11T23:41:30.184463+0000 front 2024-09-11T23:41:30.184492+0000 (oldest deadline 2024-09-11T23:41:53.684668+0000) 2024-09-11T23:42:02.250 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:42:01 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9[65389]: 2024-09-11T23:42:01.944+0000 7fc7513f1640 -1 osd.9 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:57.370068+0000 front 2024-09-11T23:40:57.370191+0000 (oldest deadline 2024-09-11T23:41:21.470122+0000) 2024-09-11T23:42:02.250 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:42:01 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9[65389]: 2024-09-11T23:42:01.944+0000 7fc7513f1640 -1 osd.9 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:04.971207+0000 front 2024-09-11T23:41:04.971047+0000 (oldest deadline 2024-09-11T23:41:27.271258+0000) 2024-09-11T23:42:02.250 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:42:01 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9[65389]: 2024-09-11T23:42:01.944+0000 7fc7513f1640 -1 osd.9 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:13.974640+0000 front 2024-09-11T23:41:13.974576+0000 (oldest deadline 2024-09-11T23:41:38.074690+0000) 2024-09-11T23:42:02.250 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:42:01 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9[65389]: 2024-09-11T23:42:01.944+0000 7fc7513f1640 -1 osd.9 120 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-09-11T23:41:18.075080+0000 front 2024-09-11T23:41:18.075257+0000 (oldest deadline 2024-09-11T23:41:43.375370+0000) 2024-09-11T23:42:02.250 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:42:01 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9[65389]: 2024-09-11T23:42:01.944+0000 7fc7513f1640 -1 osd.9 120 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-09-11T23:41:29.578379+0000 front 2024-09-11T23:41:29.578251+0000 (oldest deadline 2024-09-11T23:41:56.578836+0000) 2024-09-11T23:42:02.802 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:42:02 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:42:02.516+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:57.536934+0000 front 2024-09-11T23:40:57.537030+0000 (oldest deadline 2024-09-11T23:41:21.036908+0000) 2024-09-11T23:42:02.802 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:42:02 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:42:02.516+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:03.838149+0000 front 2024-09-11T23:41:03.838269+0000 (oldest deadline 2024-09-11T23:41:29.138274+0000) 2024-09-11T23:42:02.802 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:42:02 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:42:02.516+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:12.539763+0000 front 2024-09-11T23:41:12.539893+0000 (oldest deadline 2024-09-11T23:41:36.639808+0000) 2024-09-11T23:42:02.802 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:42:02 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:42:02.516+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-09-11T23:41:21.841541+0000 front 2024-09-11T23:41:21.841518+0000 (oldest deadline 2024-09-11T23:41:44.141586+0000) 2024-09-11T23:42:02.802 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:42:02 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:42:02.516+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-09-11T23:41:28.142849+0000 front 2024-09-11T23:41:28.142868+0000 (oldest deadline 2024-09-11T23:41:54.042918+0000) 2024-09-11T23:42:02.989 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:02 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:42:02.584+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:56.932175+0000 front 2024-09-11T23:40:56.932499+0000 (oldest deadline 2024-09-11T23:41:21.032317+0000) 2024-09-11T23:42:02.989 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:02 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:42:02.584+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:01.032761+0000 front 2024-09-11T23:41:01.032638+0000 (oldest deadline 2024-09-11T23:41:30.433335+0000) 2024-09-11T23:42:02.989 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:02 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:42:02.584+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:10.934510+0000 front 2024-09-11T23:41:10.934411+0000 (oldest deadline 2024-09-11T23:41:35.634455+0000) 2024-09-11T23:42:02.989 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:02 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:42:02.584+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-09-11T23:41:21.936325+0000 front 2024-09-11T23:41:21.936462+0000 (oldest deadline 2024-09-11T23:41:42.937021+0000) 2024-09-11T23:42:02.989 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:02 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:42:02.584+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-09-11T23:41:27.038312+0000 front 2024-09-11T23:41:27.038192+0000 (oldest deadline 2024-09-11T23:41:51.638385+0000) 2024-09-11T23:42:03.250 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:42:02 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9[65389]: 2024-09-11T23:42:02.895+0000 7fc7513f1640 -1 osd.9 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:57.370068+0000 front 2024-09-11T23:40:57.370191+0000 (oldest deadline 2024-09-11T23:41:21.470122+0000) 2024-09-11T23:42:03.250 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:42:02 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9[65389]: 2024-09-11T23:42:02.895+0000 7fc7513f1640 -1 osd.9 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:04.971207+0000 front 2024-09-11T23:41:04.971047+0000 (oldest deadline 2024-09-11T23:41:27.271258+0000) 2024-09-11T23:42:03.250 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:42:02 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9[65389]: 2024-09-11T23:42:02.895+0000 7fc7513f1640 -1 osd.9 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:13.974640+0000 front 2024-09-11T23:41:13.974576+0000 (oldest deadline 2024-09-11T23:41:38.074690+0000) 2024-09-11T23:42:03.250 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:42:02 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9[65389]: 2024-09-11T23:42:02.895+0000 7fc7513f1640 -1 osd.9 120 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-09-11T23:41:18.075080+0000 front 2024-09-11T23:41:18.075257+0000 (oldest deadline 2024-09-11T23:41:43.375370+0000) 2024-09-11T23:42:03.250 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:42:02 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9[65389]: 2024-09-11T23:42:02.895+0000 7fc7513f1640 -1 osd.9 120 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-09-11T23:41:29.578379+0000 front 2024-09-11T23:41:29.578251+0000 (oldest deadline 2024-09-11T23:41:56.578836+0000) 2024-09-11T23:42:03.391 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 11 23:42:03 smithi136 podman[74006]: 2024-09-11 23:42:03.073150768 +0000 UTC m=+5.632127172 container died 6dbd11d47ec37c165da0e65124d9ca2633c4219830a47a661deaff1e024eb310 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-8, GIT_CLEAN=True, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD) 2024-09-11T23:42:03.671 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 11 23:42:03 smithi136 podman[74006]: 2024-09-11 23:42:03.478682964 +0000 UTC m=+6.037659372 container cleanup 6dbd11d47ec37c165da0e65124d9ca2633c4219830a47a661deaff1e024eb310 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-8, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-09-11T23:42:03.671 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 11 23:42:03 smithi136 bash[74006]: ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-8 2024-09-11T23:42:03.671 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:03 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:42:03.563+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:56.932175+0000 front 2024-09-11T23:40:56.932499+0000 (oldest deadline 2024-09-11T23:41:21.032317+0000) 2024-09-11T23:42:03.671 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:03 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:42:03.563+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:01.032761+0000 front 2024-09-11T23:41:01.032638+0000 (oldest deadline 2024-09-11T23:41:30.433335+0000) 2024-09-11T23:42:03.672 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:03 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:42:03.563+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:10.934510+0000 front 2024-09-11T23:41:10.934411+0000 (oldest deadline 2024-09-11T23:41:35.634455+0000) 2024-09-11T23:42:03.672 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:03 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:42:03.563+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-09-11T23:41:21.936325+0000 front 2024-09-11T23:41:21.936462+0000 (oldest deadline 2024-09-11T23:41:42.937021+0000) 2024-09-11T23:42:03.672 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:03 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:42:03.563+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-09-11T23:41:27.038312+0000 front 2024-09-11T23:41:27.038192+0000 (oldest deadline 2024-09-11T23:41:51.638385+0000) 2024-09-11T23:42:03.672 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:03 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:42:03.563+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-09-11T23:41:37.539377+0000 front 2024-09-11T23:41:37.539405+0000 (oldest deadline 2024-09-11T23:42:02.739809+0000) 2024-09-11T23:42:03.802 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:42:03 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:42:03.474+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:57.536934+0000 front 2024-09-11T23:40:57.537030+0000 (oldest deadline 2024-09-11T23:41:21.036908+0000) 2024-09-11T23:42:03.802 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:42:03 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:42:03.474+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:03.838149+0000 front 2024-09-11T23:41:03.838269+0000 (oldest deadline 2024-09-11T23:41:29.138274+0000) 2024-09-11T23:42:03.802 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:42:03 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:42:03.474+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:12.539763+0000 front 2024-09-11T23:41:12.539893+0000 (oldest deadline 2024-09-11T23:41:36.639808+0000) 2024-09-11T23:42:03.803 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:42:03 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:42:03.474+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-09-11T23:41:21.841541+0000 front 2024-09-11T23:41:21.841518+0000 (oldest deadline 2024-09-11T23:41:44.141586+0000) 2024-09-11T23:42:03.803 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:42:03 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:42:03.474+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-09-11T23:41:28.142849+0000 front 2024-09-11T23:41:28.142868+0000 (oldest deadline 2024-09-11T23:41:54.042918+0000) 2024-09-11T23:42:03.988 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 11 23:42:03 smithi136 podman[74036]: 2024-09-11 23:42:03.671890287 +0000 UTC m=+0.582483704 container remove 6dbd11d47ec37c165da0e65124d9ca2633c4219830a47a661deaff1e024eb310 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-8, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD) 2024-09-11T23:42:04.250 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:42:03 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9[65389]: 2024-09-11T23:42:03.864+0000 7fc7513f1640 -1 osd.9 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:57.370068+0000 front 2024-09-11T23:40:57.370191+0000 (oldest deadline 2024-09-11T23:41:21.470122+0000) 2024-09-11T23:42:04.250 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:42:03 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9[65389]: 2024-09-11T23:42:03.864+0000 7fc7513f1640 -1 osd.9 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:04.971207+0000 front 2024-09-11T23:41:04.971047+0000 (oldest deadline 2024-09-11T23:41:27.271258+0000) 2024-09-11T23:42:04.250 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:42:03 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9[65389]: 2024-09-11T23:42:03.864+0000 7fc7513f1640 -1 osd.9 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:13.974640+0000 front 2024-09-11T23:41:13.974576+0000 (oldest deadline 2024-09-11T23:41:38.074690+0000) 2024-09-11T23:42:04.250 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:42:03 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9[65389]: 2024-09-11T23:42:03.864+0000 7fc7513f1640 -1 osd.9 120 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-09-11T23:41:18.075080+0000 front 2024-09-11T23:41:18.075257+0000 (oldest deadline 2024-09-11T23:41:43.375370+0000) 2024-09-11T23:42:04.251 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:42:03 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9[65389]: 2024-09-11T23:42:03.864+0000 7fc7513f1640 -1 osd.9 120 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-09-11T23:41:29.578379+0000 front 2024-09-11T23:41:29.578251+0000 (oldest deadline 2024-09-11T23:41:56.578836+0000) 2024-09-11T23:42:04.457 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 11 23:42:04 smithi136 podman[74124]: 2024-09-11 23:42:04.035453953 +0000 UTC m=+0.015931756 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T23:42:04.458 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 11 23:42:04 smithi136 podman[74124]: 2024-09-11 23:42:04.13972577 +0000 UTC m=+0.120203608 container create d0152eb67a001cd8a8dd9cce73166016202a69f26cc345fb28482319172f9b6e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-8-deactivate, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux ) 2024-09-11T23:42:04.458 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 11 23:42:04 smithi136 podman[74124]: 2024-09-11 23:42:04.375243714 +0000 UTC m=+0.355721508 container init d0152eb67a001cd8a8dd9cce73166016202a69f26cc345fb28482319172f9b6e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-8-deactivate, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS) 2024-09-11T23:42:04.458 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 11 23:42:04 smithi136 podman[74124]: 2024-09-11 23:42:04.378444462 +0000 UTC m=+0.358922258 container start d0152eb67a001cd8a8dd9cce73166016202a69f26cc345fb28482319172f9b6e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-8-deactivate, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, ceph=True) 2024-09-11T23:42:04.738 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 11 23:42:04 smithi136 podman[74124]: 2024-09-11 23:42:04.458564976 +0000 UTC m=+0.439042770 container attach d0152eb67a001cd8a8dd9cce73166016202a69f26cc345fb28482319172f9b6e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-8-deactivate, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-09-11T23:42:04.739 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 11 23:42:04 smithi136 podman[74124]: 2024-09-11 23:42:04.563851158 +0000 UTC m=+0.544328953 container died d0152eb67a001cd8a8dd9cce73166016202a69f26cc345fb28482319172f9b6e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-8-deactivate, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True) 2024-09-11T23:42:04.739 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:04 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:42:04.605+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:56.932175+0000 front 2024-09-11T23:40:56.932499+0000 (oldest deadline 2024-09-11T23:41:21.032317+0000) 2024-09-11T23:42:04.739 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:04 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:42:04.605+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:01.032761+0000 front 2024-09-11T23:41:01.032638+0000 (oldest deadline 2024-09-11T23:41:30.433335+0000) 2024-09-11T23:42:04.739 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:04 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:42:04.605+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:10.934510+0000 front 2024-09-11T23:41:10.934411+0000 (oldest deadline 2024-09-11T23:41:35.634455+0000) 2024-09-11T23:42:04.739 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:04 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:42:04.605+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-09-11T23:41:21.936325+0000 front 2024-09-11T23:41:21.936462+0000 (oldest deadline 2024-09-11T23:41:42.937021+0000) 2024-09-11T23:42:04.739 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:04 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:42:04.605+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-09-11T23:41:27.038312+0000 front 2024-09-11T23:41:27.038192+0000 (oldest deadline 2024-09-11T23:41:51.638385+0000) 2024-09-11T23:42:04.739 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:04 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:42:04.605+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-09-11T23:41:37.539377+0000 front 2024-09-11T23:41:37.539405+0000 (oldest deadline 2024-09-11T23:42:02.739809+0000) 2024-09-11T23:42:04.802 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:42:04 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:42:04.470+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:57.536934+0000 front 2024-09-11T23:40:57.537030+0000 (oldest deadline 2024-09-11T23:41:21.036908+0000) 2024-09-11T23:42:04.802 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:42:04 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:42:04.470+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:03.838149+0000 front 2024-09-11T23:41:03.838269+0000 (oldest deadline 2024-09-11T23:41:29.138274+0000) 2024-09-11T23:42:04.803 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:42:04 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:42:04.470+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:12.539763+0000 front 2024-09-11T23:41:12.539893+0000 (oldest deadline 2024-09-11T23:41:36.639808+0000) 2024-09-11T23:42:04.803 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:42:04 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:42:04.470+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-09-11T23:41:21.841541+0000 front 2024-09-11T23:41:21.841518+0000 (oldest deadline 2024-09-11T23:41:44.141586+0000) 2024-09-11T23:42:04.803 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:42:04 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:42:04.470+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-09-11T23:41:28.142849+0000 front 2024-09-11T23:41:28.142868+0000 (oldest deadline 2024-09-11T23:41:54.042918+0000) 2024-09-11T23:42:04.803 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:42:04 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:42:04.470+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-09-11T23:41:38.144026+0000 front 2024-09-11T23:41:38.143908+0000 (oldest deadline 2024-09-11T23:42:04.043693+0000) 2024-09-11T23:42:05.163 DEBUG:teuthology.orchestra.run.smithi136:> sudo pkill -f 'journalctl -f -n 0 -u ceph-af7e6fc8-7092-11ef-bcea-c7b262605968@osd.8.service' 2024-09-11T23:42:05.202 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T23:42:05.203 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-09-11T23:42:05.203 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-09-11T23:42:05.203 DEBUG:teuthology.orchestra.run.smithi026:> sudo systemctl stop ceph-af7e6fc8-7092-11ef-bcea-c7b262605968@osd.9 2024-09-11T23:42:05.231 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:42:04 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9[65389]: 2024-09-11T23:42:04.906+0000 7fc7513f1640 -1 osd.9 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:57.370068+0000 front 2024-09-11T23:40:57.370191+0000 (oldest deadline 2024-09-11T23:41:21.470122+0000) 2024-09-11T23:42:05.231 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:42:04 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9[65389]: 2024-09-11T23:42:04.906+0000 7fc7513f1640 -1 osd.9 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:04.971207+0000 front 2024-09-11T23:41:04.971047+0000 (oldest deadline 2024-09-11T23:41:27.271258+0000) 2024-09-11T23:42:05.231 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:42:04 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9[65389]: 2024-09-11T23:42:04.906+0000 7fc7513f1640 -1 osd.9 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:13.974640+0000 front 2024-09-11T23:41:13.974576+0000 (oldest deadline 2024-09-11T23:41:38.074690+0000) 2024-09-11T23:42:05.231 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:42:04 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9[65389]: 2024-09-11T23:42:04.906+0000 7fc7513f1640 -1 osd.9 120 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-09-11T23:41:18.075080+0000 front 2024-09-11T23:41:18.075257+0000 (oldest deadline 2024-09-11T23:41:43.375370+0000) 2024-09-11T23:42:05.231 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:42:04 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9[65389]: 2024-09-11T23:42:04.906+0000 7fc7513f1640 -1 osd.9 120 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-09-11T23:41:29.578379+0000 front 2024-09-11T23:41:29.578251+0000 (oldest deadline 2024-09-11T23:41:56.578836+0000) 2024-09-11T23:42:05.500 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:42:05 smithi026 systemd[1]: Stopping Ceph osd.9 for af7e6fc8-7092-11ef-bcea-c7b262605968... 2024-09-11T23:42:05.802 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:42:05 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:42:05.453+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:57.536934+0000 front 2024-09-11T23:40:57.537030+0000 (oldest deadline 2024-09-11T23:41:21.036908+0000) 2024-09-11T23:42:05.803 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:42:05 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:42:05.453+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:03.838149+0000 front 2024-09-11T23:41:03.838269+0000 (oldest deadline 2024-09-11T23:41:29.138274+0000) 2024-09-11T23:42:05.803 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:42:05 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:42:05.453+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:12.539763+0000 front 2024-09-11T23:41:12.539893+0000 (oldest deadline 2024-09-11T23:41:36.639808+0000) 2024-09-11T23:42:05.803 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:42:05 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:42:05.453+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-09-11T23:41:21.841541+0000 front 2024-09-11T23:41:21.841518+0000 (oldest deadline 2024-09-11T23:41:44.141586+0000) 2024-09-11T23:42:05.803 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:42:05 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:42:05.453+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-09-11T23:41:28.142849+0000 front 2024-09-11T23:41:28.142868+0000 (oldest deadline 2024-09-11T23:41:54.042918+0000) 2024-09-11T23:42:05.803 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:42:05 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:42:05.453+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-09-11T23:41:38.144026+0000 front 2024-09-11T23:41:38.143908+0000 (oldest deadline 2024-09-11T23:42:04.043693+0000) 2024-09-11T23:42:05.936 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:42:05 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9[65389]: 2024-09-11T23:42:05.507+0000 7fc754dec640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T23:42:05.936 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:42:05 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9[65389]: 2024-09-11T23:42:05.507+0000 7fc754dec640 -1 osd.9 120 *** Got signal Terminated *** 2024-09-11T23:42:05.937 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:42:05 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9[65389]: 2024-09-11T23:42:05.507+0000 7fc754dec640 -1 osd.9 120 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T23:42:05.988 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:05 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:42:05.577+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:56.932175+0000 front 2024-09-11T23:40:56.932499+0000 (oldest deadline 2024-09-11T23:41:21.032317+0000) 2024-09-11T23:42:05.989 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:05 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:42:05.577+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:01.032761+0000 front 2024-09-11T23:41:01.032638+0000 (oldest deadline 2024-09-11T23:41:30.433335+0000) 2024-09-11T23:42:05.989 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:05 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:42:05.577+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:10.934510+0000 front 2024-09-11T23:41:10.934411+0000 (oldest deadline 2024-09-11T23:41:35.634455+0000) 2024-09-11T23:42:05.989 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:05 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:42:05.577+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-09-11T23:41:21.936325+0000 front 2024-09-11T23:41:21.936462+0000 (oldest deadline 2024-09-11T23:41:42.937021+0000) 2024-09-11T23:42:05.989 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:05 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:42:05.577+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-09-11T23:41:27.038312+0000 front 2024-09-11T23:41:27.038192+0000 (oldest deadline 2024-09-11T23:41:51.638385+0000) 2024-09-11T23:42:05.989 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:05 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:42:05.577+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-09-11T23:41:37.539377+0000 front 2024-09-11T23:41:37.539405+0000 (oldest deadline 2024-09-11T23:42:02.739809+0000) 2024-09-11T23:42:06.250 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:42:05 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9[65389]: 2024-09-11T23:42:05.936+0000 7fc7513f1640 -1 osd.9 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:57.370068+0000 front 2024-09-11T23:40:57.370191+0000 (oldest deadline 2024-09-11T23:41:21.470122+0000) 2024-09-11T23:42:06.250 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:42:05 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9[65389]: 2024-09-11T23:42:05.936+0000 7fc7513f1640 -1 osd.9 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:04.971207+0000 front 2024-09-11T23:41:04.971047+0000 (oldest deadline 2024-09-11T23:41:27.271258+0000) 2024-09-11T23:42:06.250 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:42:05 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9[65389]: 2024-09-11T23:42:05.936+0000 7fc7513f1640 -1 osd.9 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:13.974640+0000 front 2024-09-11T23:41:13.974576+0000 (oldest deadline 2024-09-11T23:41:38.074690+0000) 2024-09-11T23:42:06.250 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:42:05 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9[65389]: 2024-09-11T23:42:05.936+0000 7fc7513f1640 -1 osd.9 120 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-09-11T23:41:18.075080+0000 front 2024-09-11T23:41:18.075257+0000 (oldest deadline 2024-09-11T23:41:43.375370+0000) 2024-09-11T23:42:06.250 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:42:05 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9[65389]: 2024-09-11T23:42:05.936+0000 7fc7513f1640 -1 osd.9 120 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-09-11T23:41:29.578379+0000 front 2024-09-11T23:41:29.578251+0000 (oldest deadline 2024-09-11T23:41:56.578836+0000) 2024-09-11T23:42:06.802 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:42:06 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:42:06.453+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:57.536934+0000 front 2024-09-11T23:40:57.537030+0000 (oldest deadline 2024-09-11T23:41:21.036908+0000) 2024-09-11T23:42:06.802 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:42:06 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:42:06.453+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:03.838149+0000 front 2024-09-11T23:41:03.838269+0000 (oldest deadline 2024-09-11T23:41:29.138274+0000) 2024-09-11T23:42:06.803 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:42:06 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:42:06.453+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:12.539763+0000 front 2024-09-11T23:41:12.539893+0000 (oldest deadline 2024-09-11T23:41:36.639808+0000) 2024-09-11T23:42:06.803 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:42:06 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:42:06.453+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-09-11T23:41:21.841541+0000 front 2024-09-11T23:41:21.841518+0000 (oldest deadline 2024-09-11T23:41:44.141586+0000) 2024-09-11T23:42:06.803 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:42:06 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:42:06.453+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-09-11T23:41:28.142849+0000 front 2024-09-11T23:41:28.142868+0000 (oldest deadline 2024-09-11T23:41:54.042918+0000) 2024-09-11T23:42:06.803 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:42:06 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:42:06.453+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-09-11T23:41:38.144026+0000 front 2024-09-11T23:41:38.143908+0000 (oldest deadline 2024-09-11T23:42:04.043693+0000) 2024-09-11T23:42:06.989 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:06 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:42:06.598+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:56.932175+0000 front 2024-09-11T23:40:56.932499+0000 (oldest deadline 2024-09-11T23:41:21.032317+0000) 2024-09-11T23:42:06.989 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:06 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:42:06.598+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:01.032761+0000 front 2024-09-11T23:41:01.032638+0000 (oldest deadline 2024-09-11T23:41:30.433335+0000) 2024-09-11T23:42:06.989 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:06 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:42:06.598+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:10.934510+0000 front 2024-09-11T23:41:10.934411+0000 (oldest deadline 2024-09-11T23:41:35.634455+0000) 2024-09-11T23:42:06.989 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:06 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:42:06.598+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-09-11T23:41:21.936325+0000 front 2024-09-11T23:41:21.936462+0000 (oldest deadline 2024-09-11T23:41:42.937021+0000) 2024-09-11T23:42:06.989 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:06 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:42:06.598+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-09-11T23:41:27.038312+0000 front 2024-09-11T23:41:27.038192+0000 (oldest deadline 2024-09-11T23:41:51.638385+0000) 2024-09-11T23:42:06.989 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:06 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:42:06.598+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-09-11T23:41:37.539377+0000 front 2024-09-11T23:41:37.539405+0000 (oldest deadline 2024-09-11T23:42:02.739809+0000) 2024-09-11T23:42:07.250 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:42:06 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9[65389]: 2024-09-11T23:42:06.925+0000 7fc7513f1640 -1 osd.9 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:57.370068+0000 front 2024-09-11T23:40:57.370191+0000 (oldest deadline 2024-09-11T23:41:21.470122+0000) 2024-09-11T23:42:07.250 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:42:06 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9[65389]: 2024-09-11T23:42:06.925+0000 7fc7513f1640 -1 osd.9 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:04.971207+0000 front 2024-09-11T23:41:04.971047+0000 (oldest deadline 2024-09-11T23:41:27.271258+0000) 2024-09-11T23:42:07.250 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:42:06 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9[65389]: 2024-09-11T23:42:06.925+0000 7fc7513f1640 -1 osd.9 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:13.974640+0000 front 2024-09-11T23:41:13.974576+0000 (oldest deadline 2024-09-11T23:41:38.074690+0000) 2024-09-11T23:42:07.250 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:42:06 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9[65389]: 2024-09-11T23:42:06.925+0000 7fc7513f1640 -1 osd.9 120 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-09-11T23:41:18.075080+0000 front 2024-09-11T23:41:18.075257+0000 (oldest deadline 2024-09-11T23:41:43.375370+0000) 2024-09-11T23:42:07.250 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:42:06 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9[65389]: 2024-09-11T23:42:06.925+0000 7fc7513f1640 -1 osd.9 120 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-09-11T23:41:29.578379+0000 front 2024-09-11T23:41:29.578251+0000 (oldest deadline 2024-09-11T23:41:56.578836+0000) 2024-09-11T23:42:07.802 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:42:07 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:42:07.436+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:57.536934+0000 front 2024-09-11T23:40:57.537030+0000 (oldest deadline 2024-09-11T23:41:21.036908+0000) 2024-09-11T23:42:07.802 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:42:07 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:42:07.436+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:03.838149+0000 front 2024-09-11T23:41:03.838269+0000 (oldest deadline 2024-09-11T23:41:29.138274+0000) 2024-09-11T23:42:07.802 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:42:07 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:42:07.436+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:12.539763+0000 front 2024-09-11T23:41:12.539893+0000 (oldest deadline 2024-09-11T23:41:36.639808+0000) 2024-09-11T23:42:07.802 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:42:07 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:42:07.436+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-09-11T23:41:21.841541+0000 front 2024-09-11T23:41:21.841518+0000 (oldest deadline 2024-09-11T23:41:44.141586+0000) 2024-09-11T23:42:07.802 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:42:07 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:42:07.436+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-09-11T23:41:28.142849+0000 front 2024-09-11T23:41:28.142868+0000 (oldest deadline 2024-09-11T23:41:54.042918+0000) 2024-09-11T23:42:07.803 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:42:07 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:42:07.436+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-09-11T23:41:38.144026+0000 front 2024-09-11T23:41:38.143908+0000 (oldest deadline 2024-09-11T23:42:04.043693+0000) 2024-09-11T23:42:07.989 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:07 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:42:07.569+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:56.932175+0000 front 2024-09-11T23:40:56.932499+0000 (oldest deadline 2024-09-11T23:41:21.032317+0000) 2024-09-11T23:42:07.989 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:07 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:42:07.569+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:01.032761+0000 front 2024-09-11T23:41:01.032638+0000 (oldest deadline 2024-09-11T23:41:30.433335+0000) 2024-09-11T23:42:07.989 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:07 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:42:07.569+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:10.934510+0000 front 2024-09-11T23:41:10.934411+0000 (oldest deadline 2024-09-11T23:41:35.634455+0000) 2024-09-11T23:42:07.989 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:07 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:42:07.569+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-09-11T23:41:21.936325+0000 front 2024-09-11T23:41:21.936462+0000 (oldest deadline 2024-09-11T23:41:42.937021+0000) 2024-09-11T23:42:07.989 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:07 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:42:07.569+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-09-11T23:41:27.038312+0000 front 2024-09-11T23:41:27.038192+0000 (oldest deadline 2024-09-11T23:41:51.638385+0000) 2024-09-11T23:42:07.989 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:07 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:42:07.569+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-09-11T23:41:37.539377+0000 front 2024-09-11T23:41:37.539405+0000 (oldest deadline 2024-09-11T23:42:02.739809+0000) 2024-09-11T23:42:08.250 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:42:07 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9[65389]: 2024-09-11T23:42:07.973+0000 7fc7513f1640 -1 osd.9 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:57.370068+0000 front 2024-09-11T23:40:57.370191+0000 (oldest deadline 2024-09-11T23:41:21.470122+0000) 2024-09-11T23:42:08.250 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:42:07 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9[65389]: 2024-09-11T23:42:07.973+0000 7fc7513f1640 -1 osd.9 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:04.971207+0000 front 2024-09-11T23:41:04.971047+0000 (oldest deadline 2024-09-11T23:41:27.271258+0000) 2024-09-11T23:42:08.250 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:42:07 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9[65389]: 2024-09-11T23:42:07.973+0000 7fc7513f1640 -1 osd.9 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:13.974640+0000 front 2024-09-11T23:41:13.974576+0000 (oldest deadline 2024-09-11T23:41:38.074690+0000) 2024-09-11T23:42:08.250 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:42:07 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9[65389]: 2024-09-11T23:42:07.973+0000 7fc7513f1640 -1 osd.9 120 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-09-11T23:41:18.075080+0000 front 2024-09-11T23:41:18.075257+0000 (oldest deadline 2024-09-11T23:41:43.375370+0000) 2024-09-11T23:42:08.250 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:42:07 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9[65389]: 2024-09-11T23:42:07.973+0000 7fc7513f1640 -1 osd.9 120 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-09-11T23:41:29.578379+0000 front 2024-09-11T23:41:29.578251+0000 (oldest deadline 2024-09-11T23:41:56.578836+0000) 2024-09-11T23:42:08.802 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:42:08 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:42:08.430+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:57.536934+0000 front 2024-09-11T23:40:57.537030+0000 (oldest deadline 2024-09-11T23:41:21.036908+0000) 2024-09-11T23:42:08.802 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:42:08 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:42:08.430+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:03.838149+0000 front 2024-09-11T23:41:03.838269+0000 (oldest deadline 2024-09-11T23:41:29.138274+0000) 2024-09-11T23:42:08.802 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:42:08 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:42:08.430+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:12.539763+0000 front 2024-09-11T23:41:12.539893+0000 (oldest deadline 2024-09-11T23:41:36.639808+0000) 2024-09-11T23:42:08.803 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:42:08 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:42:08.430+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-09-11T23:41:21.841541+0000 front 2024-09-11T23:41:21.841518+0000 (oldest deadline 2024-09-11T23:41:44.141586+0000) 2024-09-11T23:42:08.803 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:42:08 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:42:08.430+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-09-11T23:41:28.142849+0000 front 2024-09-11T23:41:28.142868+0000 (oldest deadline 2024-09-11T23:41:54.042918+0000) 2024-09-11T23:42:08.803 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:42:08 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:42:08.430+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-09-11T23:41:38.144026+0000 front 2024-09-11T23:41:38.143908+0000 (oldest deadline 2024-09-11T23:42:04.043693+0000) 2024-09-11T23:42:08.803 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:42:08 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:42:08.430+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.26:6822 osd.6 since back 2024-09-11T23:41:44.044474+0000 front 2024-09-11T23:41:44.044429+0000 (oldest deadline 2024-09-11T23:42:07.544154+0000) 2024-09-11T23:42:08.988 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:08 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:42:08.606+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:56.932175+0000 front 2024-09-11T23:40:56.932499+0000 (oldest deadline 2024-09-11T23:41:21.032317+0000) 2024-09-11T23:42:08.989 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:08 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:42:08.606+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:01.032761+0000 front 2024-09-11T23:41:01.032638+0000 (oldest deadline 2024-09-11T23:41:30.433335+0000) 2024-09-11T23:42:08.989 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:08 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:42:08.606+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:10.934510+0000 front 2024-09-11T23:41:10.934411+0000 (oldest deadline 2024-09-11T23:41:35.634455+0000) 2024-09-11T23:42:08.989 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:08 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:42:08.606+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-09-11T23:41:21.936325+0000 front 2024-09-11T23:41:21.936462+0000 (oldest deadline 2024-09-11T23:41:42.937021+0000) 2024-09-11T23:42:08.989 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:08 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:42:08.606+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-09-11T23:41:27.038312+0000 front 2024-09-11T23:41:27.038192+0000 (oldest deadline 2024-09-11T23:41:51.638385+0000) 2024-09-11T23:42:08.989 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:08 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:42:08.606+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-09-11T23:41:37.539377+0000 front 2024-09-11T23:41:37.539405+0000 (oldest deadline 2024-09-11T23:42:02.739809+0000) 2024-09-11T23:42:09.250 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:42:08 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9[65389]: 2024-09-11T23:42:08.933+0000 7fc7513f1640 -1 osd.9 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:57.370068+0000 front 2024-09-11T23:40:57.370191+0000 (oldest deadline 2024-09-11T23:41:21.470122+0000) 2024-09-11T23:42:09.250 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:42:08 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9[65389]: 2024-09-11T23:42:08.933+0000 7fc7513f1640 -1 osd.9 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:04.971207+0000 front 2024-09-11T23:41:04.971047+0000 (oldest deadline 2024-09-11T23:41:27.271258+0000) 2024-09-11T23:42:09.250 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:42:08 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9[65389]: 2024-09-11T23:42:08.933+0000 7fc7513f1640 -1 osd.9 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:13.974640+0000 front 2024-09-11T23:41:13.974576+0000 (oldest deadline 2024-09-11T23:41:38.074690+0000) 2024-09-11T23:42:09.251 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:42:08 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9[65389]: 2024-09-11T23:42:08.933+0000 7fc7513f1640 -1 osd.9 120 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-09-11T23:41:18.075080+0000 front 2024-09-11T23:41:18.075257+0000 (oldest deadline 2024-09-11T23:41:43.375370+0000) 2024-09-11T23:42:09.251 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:42:08 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9[65389]: 2024-09-11T23:42:08.933+0000 7fc7513f1640 -1 osd.9 120 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-09-11T23:41:29.578379+0000 front 2024-09-11T23:41:29.578251+0000 (oldest deadline 2024-09-11T23:41:56.578836+0000) 2024-09-11T23:42:09.251 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:42:08 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9[65389]: 2024-09-11T23:42:08.933+0000 7fc7513f1640 -1 osd.9 120 heartbeat_check: no reply from 172.21.15.26:6822 osd.6 since back 2024-09-11T23:41:45.980246+0000 front 2024-09-11T23:41:45.980224+0000 (oldest deadline 2024-09-11T23:42:08.780863+0000) 2024-09-11T23:42:09.754 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:42:09 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:42:09.398+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:57.536934+0000 front 2024-09-11T23:40:57.537030+0000 (oldest deadline 2024-09-11T23:41:21.036908+0000) 2024-09-11T23:42:09.754 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:42:09 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:42:09.398+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:03.838149+0000 front 2024-09-11T23:41:03.838269+0000 (oldest deadline 2024-09-11T23:41:29.138274+0000) 2024-09-11T23:42:09.754 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:42:09 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:42:09.398+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:12.539763+0000 front 2024-09-11T23:41:12.539893+0000 (oldest deadline 2024-09-11T23:41:36.639808+0000) 2024-09-11T23:42:09.754 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:42:09 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:42:09.398+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-09-11T23:41:21.841541+0000 front 2024-09-11T23:41:21.841518+0000 (oldest deadline 2024-09-11T23:41:44.141586+0000) 2024-09-11T23:42:09.754 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:42:09 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:42:09.398+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-09-11T23:41:28.142849+0000 front 2024-09-11T23:41:28.142868+0000 (oldest deadline 2024-09-11T23:41:54.042918+0000) 2024-09-11T23:42:09.754 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:42:09 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:42:09.398+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-09-11T23:41:38.144026+0000 front 2024-09-11T23:41:38.143908+0000 (oldest deadline 2024-09-11T23:42:04.043693+0000) 2024-09-11T23:42:09.755 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:42:09 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:42:09.398+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.26:6822 osd.6 since back 2024-09-11T23:41:44.044474+0000 front 2024-09-11T23:41:44.044429+0000 (oldest deadline 2024-09-11T23:42:07.544154+0000) 2024-09-11T23:42:09.989 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:09 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:42:09.614+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:56.932175+0000 front 2024-09-11T23:40:56.932499+0000 (oldest deadline 2024-09-11T23:41:21.032317+0000) 2024-09-11T23:42:09.989 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:09 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:42:09.614+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:01.032761+0000 front 2024-09-11T23:41:01.032638+0000 (oldest deadline 2024-09-11T23:41:30.433335+0000) 2024-09-11T23:42:09.989 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:09 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:42:09.614+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:10.934510+0000 front 2024-09-11T23:41:10.934411+0000 (oldest deadline 2024-09-11T23:41:35.634455+0000) 2024-09-11T23:42:09.989 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:09 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:42:09.614+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-09-11T23:41:21.936325+0000 front 2024-09-11T23:41:21.936462+0000 (oldest deadline 2024-09-11T23:41:42.937021+0000) 2024-09-11T23:42:09.989 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:09 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:42:09.615+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-09-11T23:41:27.038312+0000 front 2024-09-11T23:41:27.038192+0000 (oldest deadline 2024-09-11T23:41:51.638385+0000) 2024-09-11T23:42:09.989 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:09 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:42:09.615+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-09-11T23:41:37.539377+0000 front 2024-09-11T23:41:37.539405+0000 (oldest deadline 2024-09-11T23:42:02.739809+0000) 2024-09-11T23:42:10.250 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:42:09 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9[65389]: 2024-09-11T23:42:09.952+0000 7fc7513f1640 -1 osd.9 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:57.370068+0000 front 2024-09-11T23:40:57.370191+0000 (oldest deadline 2024-09-11T23:41:21.470122+0000) 2024-09-11T23:42:10.250 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:42:09 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9[65389]: 2024-09-11T23:42:09.952+0000 7fc7513f1640 -1 osd.9 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:04.971207+0000 front 2024-09-11T23:41:04.971047+0000 (oldest deadline 2024-09-11T23:41:27.271258+0000) 2024-09-11T23:42:10.250 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:42:09 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9[65389]: 2024-09-11T23:42:09.952+0000 7fc7513f1640 -1 osd.9 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:13.974640+0000 front 2024-09-11T23:41:13.974576+0000 (oldest deadline 2024-09-11T23:41:38.074690+0000) 2024-09-11T23:42:10.251 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:42:09 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9[65389]: 2024-09-11T23:42:09.952+0000 7fc7513f1640 -1 osd.9 120 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-09-11T23:41:18.075080+0000 front 2024-09-11T23:41:18.075257+0000 (oldest deadline 2024-09-11T23:41:43.375370+0000) 2024-09-11T23:42:10.251 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:42:09 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9[65389]: 2024-09-11T23:42:09.952+0000 7fc7513f1640 -1 osd.9 120 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-09-11T23:41:29.578379+0000 front 2024-09-11T23:41:29.578251+0000 (oldest deadline 2024-09-11T23:41:56.578836+0000) 2024-09-11T23:42:10.251 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:42:09 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9[65389]: 2024-09-11T23:42:09.952+0000 7fc7513f1640 -1 osd.9 120 heartbeat_check: no reply from 172.21.15.26:6822 osd.6 since back 2024-09-11T23:41:45.980246+0000 front 2024-09-11T23:41:45.980224+0000 (oldest deadline 2024-09-11T23:42:08.780863+0000) 2024-09-11T23:42:10.802 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:42:10 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:42:10.390+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:57.536934+0000 front 2024-09-11T23:40:57.537030+0000 (oldest deadline 2024-09-11T23:41:21.036908+0000) 2024-09-11T23:42:10.803 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:42:10 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:42:10.390+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:03.838149+0000 front 2024-09-11T23:41:03.838269+0000 (oldest deadline 2024-09-11T23:41:29.138274+0000) 2024-09-11T23:42:10.803 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:42:10 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:42:10.390+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:12.539763+0000 front 2024-09-11T23:41:12.539893+0000 (oldest deadline 2024-09-11T23:41:36.639808+0000) 2024-09-11T23:42:10.803 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:42:10 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:42:10.390+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-09-11T23:41:21.841541+0000 front 2024-09-11T23:41:21.841518+0000 (oldest deadline 2024-09-11T23:41:44.141586+0000) 2024-09-11T23:42:10.803 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:42:10 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:42:10.390+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-09-11T23:41:28.142849+0000 front 2024-09-11T23:41:28.142868+0000 (oldest deadline 2024-09-11T23:41:54.042918+0000) 2024-09-11T23:42:10.803 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:42:10 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:42:10.390+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-09-11T23:41:38.144026+0000 front 2024-09-11T23:41:38.143908+0000 (oldest deadline 2024-09-11T23:42:04.043693+0000) 2024-09-11T23:42:10.803 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:42:10 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:42:10.390+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.26:6822 osd.6 since back 2024-09-11T23:41:44.044474+0000 front 2024-09-11T23:41:44.044429+0000 (oldest deadline 2024-09-11T23:42:07.544154+0000) 2024-09-11T23:42:10.988 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:10 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:42:10.644+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:56.932175+0000 front 2024-09-11T23:40:56.932499+0000 (oldest deadline 2024-09-11T23:41:21.032317+0000) 2024-09-11T23:42:10.989 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:10 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:42:10.644+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:01.032761+0000 front 2024-09-11T23:41:01.032638+0000 (oldest deadline 2024-09-11T23:41:30.433335+0000) 2024-09-11T23:42:10.989 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:10 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:42:10.644+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:10.934510+0000 front 2024-09-11T23:41:10.934411+0000 (oldest deadline 2024-09-11T23:41:35.634455+0000) 2024-09-11T23:42:10.989 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:10 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:42:10.644+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-09-11T23:41:21.936325+0000 front 2024-09-11T23:41:21.936462+0000 (oldest deadline 2024-09-11T23:41:42.937021+0000) 2024-09-11T23:42:10.989 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:10 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:42:10.644+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-09-11T23:41:27.038312+0000 front 2024-09-11T23:41:27.038192+0000 (oldest deadline 2024-09-11T23:41:51.638385+0000) 2024-09-11T23:42:10.989 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:10 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:42:10.644+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-09-11T23:41:37.539377+0000 front 2024-09-11T23:41:37.539405+0000 (oldest deadline 2024-09-11T23:42:02.739809+0000) 2024-09-11T23:42:11.250 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:42:10 smithi026 podman[115130]: 2024-09-11 23:42:10.997190489 +0000 UTC m=+5.588375240 container died 1749c992d30ded72738894e94f8947aa6c71944a258e869cceb996427b2e0819 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.build-date=20240731) 2024-09-11T23:42:11.633 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:42:11 smithi026 podman[115130]: 2024-09-11 23:42:11.379203371 +0000 UTC m=+5.970388122 container cleanup 1749c992d30ded72738894e94f8947aa6c71944a258e869cceb996427b2e0819 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS) 2024-09-11T23:42:11.634 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:42:11 smithi026 bash[115130]: ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9 2024-09-11T23:42:11.802 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:42:11 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:42:11.376+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:57.536934+0000 front 2024-09-11T23:40:57.537030+0000 (oldest deadline 2024-09-11T23:41:21.036908+0000) 2024-09-11T23:42:11.802 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:42:11 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:42:11.376+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:03.838149+0000 front 2024-09-11T23:41:03.838269+0000 (oldest deadline 2024-09-11T23:41:29.138274+0000) 2024-09-11T23:42:11.802 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:42:11 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:42:11.376+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:12.539763+0000 front 2024-09-11T23:41:12.539893+0000 (oldest deadline 2024-09-11T23:41:36.639808+0000) 2024-09-11T23:42:11.802 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:42:11 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:42:11.376+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-09-11T23:41:21.841541+0000 front 2024-09-11T23:41:21.841518+0000 (oldest deadline 2024-09-11T23:41:44.141586+0000) 2024-09-11T23:42:11.803 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:42:11 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:42:11.376+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-09-11T23:41:28.142849+0000 front 2024-09-11T23:41:28.142868+0000 (oldest deadline 2024-09-11T23:41:54.042918+0000) 2024-09-11T23:42:11.803 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:42:11 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:42:11.376+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-09-11T23:41:38.144026+0000 front 2024-09-11T23:41:38.143908+0000 (oldest deadline 2024-09-11T23:42:04.043693+0000) 2024-09-11T23:42:11.803 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:42:11 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:42:11.376+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.26:6822 osd.6 since back 2024-09-11T23:41:44.044474+0000 front 2024-09-11T23:41:44.044429+0000 (oldest deadline 2024-09-11T23:42:07.544154+0000) 2024-09-11T23:42:11.976 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:42:11 smithi026 podman[115146]: 2024-09-11 23:42:11.634735916 +0000 UTC m=+0.627685077 container remove 1749c992d30ded72738894e94f8947aa6c71944a258e869cceb996427b2e0819 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731) 2024-09-11T23:42:11.989 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:11 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:42:11.625+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:56.932175+0000 front 2024-09-11T23:40:56.932499+0000 (oldest deadline 2024-09-11T23:41:21.032317+0000) 2024-09-11T23:42:11.989 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:11 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:42:11.625+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:01.032761+0000 front 2024-09-11T23:41:01.032638+0000 (oldest deadline 2024-09-11T23:41:30.433335+0000) 2024-09-11T23:42:11.989 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:11 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:42:11.625+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:10.934510+0000 front 2024-09-11T23:41:10.934411+0000 (oldest deadline 2024-09-11T23:41:35.634455+0000) 2024-09-11T23:42:11.989 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:11 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:42:11.625+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-09-11T23:41:21.936325+0000 front 2024-09-11T23:41:21.936462+0000 (oldest deadline 2024-09-11T23:41:42.937021+0000) 2024-09-11T23:42:11.989 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:11 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:42:11.625+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-09-11T23:41:27.038312+0000 front 2024-09-11T23:41:27.038192+0000 (oldest deadline 2024-09-11T23:41:51.638385+0000) 2024-09-11T23:42:11.989 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:11 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:42:11.625+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-09-11T23:41:37.539377+0000 front 2024-09-11T23:41:37.539405+0000 (oldest deadline 2024-09-11T23:42:02.739809+0000) 2024-09-11T23:42:12.250 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:42:11 smithi026 podman[115242]: 2024-09-11 23:42:11.880502998 +0000 UTC m=+0.017137782 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T23:42:12.250 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:42:11 smithi026 podman[115242]: 2024-09-11 23:42:11.993809452 +0000 UTC m=+0.130444229 container create 847012a9dedf18bc3b55a8499ffaf17061c49278b859991804d57c6d7506fc5d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9-deactivate, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T23:42:12.250 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:42:12 smithi026 podman[115242]: 2024-09-11 23:42:12.204821191 +0000 UTC m=+0.341455963 container init 847012a9dedf18bc3b55a8499ffaf17061c49278b859991804d57c6d7506fc5d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9-deactivate, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3) 2024-09-11T23:42:12.250 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:42:12 smithi026 podman[115242]: 2024-09-11 23:42:12.208369355 +0000 UTC m=+0.345004144 container start 847012a9dedf18bc3b55a8499ffaf17061c49278b859991804d57c6d7506fc5d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9-deactivate, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-09-11T23:42:12.707 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:42:12 smithi026 podman[115242]: 2024-09-11 23:42:12.288246082 +0000 UTC m=+0.424880854 container attach 847012a9dedf18bc3b55a8499ffaf17061c49278b859991804d57c6d7506fc5d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9-deactivate, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-09-11T23:42:12.707 INFO:journalctl@ceph.osd.9.smithi026.stdout:Sep 11 23:42:12 smithi026 podman[115242]: 2024-09-11 23:42:12.4129642 +0000 UTC m=+0.549598981 container died 847012a9dedf18bc3b55a8499ffaf17061c49278b859991804d57c6d7506fc5d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-9-deactivate, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-09-11T23:42:12.805 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:42:12 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:42:12.349+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:57.536934+0000 front 2024-09-11T23:40:57.537030+0000 (oldest deadline 2024-09-11T23:41:21.036908+0000) 2024-09-11T23:42:12.805 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:42:12 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:42:12.349+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:03.838149+0000 front 2024-09-11T23:41:03.838269+0000 (oldest deadline 2024-09-11T23:41:29.138274+0000) 2024-09-11T23:42:12.805 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:42:12 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:42:12.349+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:12.539763+0000 front 2024-09-11T23:41:12.539893+0000 (oldest deadline 2024-09-11T23:41:36.639808+0000) 2024-09-11T23:42:12.805 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:42:12 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:42:12.349+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-09-11T23:41:21.841541+0000 front 2024-09-11T23:41:21.841518+0000 (oldest deadline 2024-09-11T23:41:44.141586+0000) 2024-09-11T23:42:12.805 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:42:12 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:42:12.349+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-09-11T23:41:28.142849+0000 front 2024-09-11T23:41:28.142868+0000 (oldest deadline 2024-09-11T23:41:54.042918+0000) 2024-09-11T23:42:12.805 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:42:12 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:42:12.349+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-09-11T23:41:38.144026+0000 front 2024-09-11T23:41:38.143908+0000 (oldest deadline 2024-09-11T23:42:04.043693+0000) 2024-09-11T23:42:12.805 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:42:12 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:42:12.349+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.26:6822 osd.6 since back 2024-09-11T23:41:44.044474+0000 front 2024-09-11T23:41:44.044429+0000 (oldest deadline 2024-09-11T23:42:07.544154+0000) 2024-09-11T23:42:12.989 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:12 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:42:12.670+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:56.932175+0000 front 2024-09-11T23:40:56.932499+0000 (oldest deadline 2024-09-11T23:41:21.032317+0000) 2024-09-11T23:42:12.989 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:12 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:42:12.670+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:01.032761+0000 front 2024-09-11T23:41:01.032638+0000 (oldest deadline 2024-09-11T23:41:30.433335+0000) 2024-09-11T23:42:12.989 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:12 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:42:12.670+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:10.934510+0000 front 2024-09-11T23:41:10.934411+0000 (oldest deadline 2024-09-11T23:41:35.634455+0000) 2024-09-11T23:42:12.989 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:12 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:42:12.670+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-09-11T23:41:21.936325+0000 front 2024-09-11T23:41:21.936462+0000 (oldest deadline 2024-09-11T23:41:42.937021+0000) 2024-09-11T23:42:12.989 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:12 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:42:12.670+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-09-11T23:41:27.038312+0000 front 2024-09-11T23:41:27.038192+0000 (oldest deadline 2024-09-11T23:41:51.638385+0000) 2024-09-11T23:42:12.990 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:12 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:42:12.670+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-09-11T23:41:37.539377+0000 front 2024-09-11T23:41:37.539405+0000 (oldest deadline 2024-09-11T23:42:02.739809+0000) 2024-09-11T23:42:13.044 DEBUG:teuthology.orchestra.run.smithi026:> sudo pkill -f 'journalctl -f -n 0 -u ceph-af7e6fc8-7092-11ef-bcea-c7b262605968@osd.9.service' 2024-09-11T23:42:13.126 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T23:42:13.127 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-09-11T23:42:13.127 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-09-11T23:42:13.127 DEBUG:teuthology.orchestra.run.smithi076:> sudo systemctl stop ceph-af7e6fc8-7092-11ef-bcea-c7b262605968@osd.10 2024-09-11T23:42:13.440 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:42:13 smithi076 systemd[1]: Stopping Ceph osd.10 for af7e6fc8-7092-11ef-bcea-c7b262605968... 2024-09-11T23:42:13.440 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:42:13 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:42:13.385+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:57.536934+0000 front 2024-09-11T23:40:57.537030+0000 (oldest deadline 2024-09-11T23:41:21.036908+0000) 2024-09-11T23:42:13.440 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:42:13 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:42:13.385+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:03.838149+0000 front 2024-09-11T23:41:03.838269+0000 (oldest deadline 2024-09-11T23:41:29.138274+0000) 2024-09-11T23:42:13.440 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:42:13 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:42:13.385+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:12.539763+0000 front 2024-09-11T23:41:12.539893+0000 (oldest deadline 2024-09-11T23:41:36.639808+0000) 2024-09-11T23:42:13.440 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:42:13 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:42:13.385+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-09-11T23:41:21.841541+0000 front 2024-09-11T23:41:21.841518+0000 (oldest deadline 2024-09-11T23:41:44.141586+0000) 2024-09-11T23:42:13.440 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:42:13 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:42:13.385+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-09-11T23:41:28.142849+0000 front 2024-09-11T23:41:28.142868+0000 (oldest deadline 2024-09-11T23:41:54.042918+0000) 2024-09-11T23:42:13.440 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:42:13 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:42:13.385+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-09-11T23:41:38.144026+0000 front 2024-09-11T23:41:38.143908+0000 (oldest deadline 2024-09-11T23:42:04.043693+0000) 2024-09-11T23:42:13.441 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:42:13 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:42:13.385+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.26:6822 osd.6 since back 2024-09-11T23:41:44.044474+0000 front 2024-09-11T23:41:44.044429+0000 (oldest deadline 2024-09-11T23:42:07.544154+0000) 2024-09-11T23:42:13.441 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:42:13 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:42:13.439+0000 7fcb75f9e640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T23:42:13.441 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:42:13 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:42:13.439+0000 7fcb75f9e640 -1 osd.10 120 *** Got signal Terminated *** 2024-09-11T23:42:13.441 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:42:13 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:42:13.440+0000 7fcb75f9e640 -1 osd.10 120 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T23:42:13.988 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:13 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:42:13.654+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:56.932175+0000 front 2024-09-11T23:40:56.932499+0000 (oldest deadline 2024-09-11T23:41:21.032317+0000) 2024-09-11T23:42:13.989 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:13 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:42:13.654+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:01.032761+0000 front 2024-09-11T23:41:01.032638+0000 (oldest deadline 2024-09-11T23:41:30.433335+0000) 2024-09-11T23:42:13.989 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:13 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:42:13.654+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:10.934510+0000 front 2024-09-11T23:41:10.934411+0000 (oldest deadline 2024-09-11T23:41:35.634455+0000) 2024-09-11T23:42:13.989 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:13 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:42:13.654+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-09-11T23:41:21.936325+0000 front 2024-09-11T23:41:21.936462+0000 (oldest deadline 2024-09-11T23:41:42.937021+0000) 2024-09-11T23:42:13.989 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:13 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:42:13.654+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-09-11T23:41:27.038312+0000 front 2024-09-11T23:41:27.038192+0000 (oldest deadline 2024-09-11T23:41:51.638385+0000) 2024-09-11T23:42:13.989 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:13 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:42:13.654+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-09-11T23:41:37.539377+0000 front 2024-09-11T23:41:37.539405+0000 (oldest deadline 2024-09-11T23:42:02.739809+0000) 2024-09-11T23:42:14.802 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:42:14 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:42:14.406+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:57.536934+0000 front 2024-09-11T23:40:57.537030+0000 (oldest deadline 2024-09-11T23:41:21.036908+0000) 2024-09-11T23:42:14.802 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:42:14 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:42:14.406+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:03.838149+0000 front 2024-09-11T23:41:03.838269+0000 (oldest deadline 2024-09-11T23:41:29.138274+0000) 2024-09-11T23:42:14.802 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:42:14 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:42:14.406+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:12.539763+0000 front 2024-09-11T23:41:12.539893+0000 (oldest deadline 2024-09-11T23:41:36.639808+0000) 2024-09-11T23:42:14.802 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:42:14 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:42:14.406+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-09-11T23:41:21.841541+0000 front 2024-09-11T23:41:21.841518+0000 (oldest deadline 2024-09-11T23:41:44.141586+0000) 2024-09-11T23:42:14.802 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:42:14 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:42:14.406+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-09-11T23:41:28.142849+0000 front 2024-09-11T23:41:28.142868+0000 (oldest deadline 2024-09-11T23:41:54.042918+0000) 2024-09-11T23:42:14.803 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:42:14 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:42:14.406+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-09-11T23:41:38.144026+0000 front 2024-09-11T23:41:38.143908+0000 (oldest deadline 2024-09-11T23:42:04.043693+0000) 2024-09-11T23:42:14.803 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:42:14 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:42:14.406+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.26:6822 osd.6 since back 2024-09-11T23:41:44.044474+0000 front 2024-09-11T23:41:44.044429+0000 (oldest deadline 2024-09-11T23:42:07.544154+0000) 2024-09-11T23:42:14.989 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:14 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:42:14.698+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:56.932175+0000 front 2024-09-11T23:40:56.932499+0000 (oldest deadline 2024-09-11T23:41:21.032317+0000) 2024-09-11T23:42:14.989 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:14 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:42:14.698+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:01.032761+0000 front 2024-09-11T23:41:01.032638+0000 (oldest deadline 2024-09-11T23:41:30.433335+0000) 2024-09-11T23:42:14.989 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:14 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:42:14.698+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:10.934510+0000 front 2024-09-11T23:41:10.934411+0000 (oldest deadline 2024-09-11T23:41:35.634455+0000) 2024-09-11T23:42:14.989 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:14 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:42:14.698+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-09-11T23:41:21.936325+0000 front 2024-09-11T23:41:21.936462+0000 (oldest deadline 2024-09-11T23:41:42.937021+0000) 2024-09-11T23:42:14.989 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:14 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:42:14.698+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-09-11T23:41:27.038312+0000 front 2024-09-11T23:41:27.038192+0000 (oldest deadline 2024-09-11T23:41:51.638385+0000) 2024-09-11T23:42:14.989 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:14 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:42:14.698+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-09-11T23:41:37.539377+0000 front 2024-09-11T23:41:37.539405+0000 (oldest deadline 2024-09-11T23:42:02.739809+0000) 2024-09-11T23:42:15.802 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:42:15 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:42:15.380+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:57.536934+0000 front 2024-09-11T23:40:57.537030+0000 (oldest deadline 2024-09-11T23:41:21.036908+0000) 2024-09-11T23:42:15.802 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:42:15 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:42:15.380+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:03.838149+0000 front 2024-09-11T23:41:03.838269+0000 (oldest deadline 2024-09-11T23:41:29.138274+0000) 2024-09-11T23:42:15.803 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:42:15 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:42:15.380+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:12.539763+0000 front 2024-09-11T23:41:12.539893+0000 (oldest deadline 2024-09-11T23:41:36.639808+0000) 2024-09-11T23:42:15.803 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:42:15 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:42:15.380+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-09-11T23:41:21.841541+0000 front 2024-09-11T23:41:21.841518+0000 (oldest deadline 2024-09-11T23:41:44.141586+0000) 2024-09-11T23:42:15.803 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:42:15 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:42:15.380+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-09-11T23:41:28.142849+0000 front 2024-09-11T23:41:28.142868+0000 (oldest deadline 2024-09-11T23:41:54.042918+0000) 2024-09-11T23:42:15.803 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:42:15 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:42:15.380+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-09-11T23:41:38.144026+0000 front 2024-09-11T23:41:38.143908+0000 (oldest deadline 2024-09-11T23:42:04.043693+0000) 2024-09-11T23:42:15.803 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:42:15 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:42:15.380+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.26:6822 osd.6 since back 2024-09-11T23:41:44.044474+0000 front 2024-09-11T23:41:44.044429+0000 (oldest deadline 2024-09-11T23:42:07.544154+0000) 2024-09-11T23:42:15.989 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:15 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:42:15.686+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:56.932175+0000 front 2024-09-11T23:40:56.932499+0000 (oldest deadline 2024-09-11T23:41:21.032317+0000) 2024-09-11T23:42:15.989 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:15 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:42:15.686+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:01.032761+0000 front 2024-09-11T23:41:01.032638+0000 (oldest deadline 2024-09-11T23:41:30.433335+0000) 2024-09-11T23:42:15.989 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:15 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:42:15.686+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:10.934510+0000 front 2024-09-11T23:41:10.934411+0000 (oldest deadline 2024-09-11T23:41:35.634455+0000) 2024-09-11T23:42:15.989 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:15 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:42:15.686+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-09-11T23:41:21.936325+0000 front 2024-09-11T23:41:21.936462+0000 (oldest deadline 2024-09-11T23:41:42.937021+0000) 2024-09-11T23:42:15.989 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:15 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:42:15.686+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-09-11T23:41:27.038312+0000 front 2024-09-11T23:41:27.038192+0000 (oldest deadline 2024-09-11T23:41:51.638385+0000) 2024-09-11T23:42:15.989 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:15 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:42:15.686+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-09-11T23:41:37.539377+0000 front 2024-09-11T23:41:37.539405+0000 (oldest deadline 2024-09-11T23:42:02.739809+0000) 2024-09-11T23:42:16.802 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:42:16 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:42:16.334+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:57.536934+0000 front 2024-09-11T23:40:57.537030+0000 (oldest deadline 2024-09-11T23:41:21.036908+0000) 2024-09-11T23:42:16.802 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:42:16 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:42:16.334+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:03.838149+0000 front 2024-09-11T23:41:03.838269+0000 (oldest deadline 2024-09-11T23:41:29.138274+0000) 2024-09-11T23:42:16.802 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:42:16 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:42:16.334+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:12.539763+0000 front 2024-09-11T23:41:12.539893+0000 (oldest deadline 2024-09-11T23:41:36.639808+0000) 2024-09-11T23:42:16.802 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:42:16 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:42:16.334+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-09-11T23:41:21.841541+0000 front 2024-09-11T23:41:21.841518+0000 (oldest deadline 2024-09-11T23:41:44.141586+0000) 2024-09-11T23:42:16.802 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:42:16 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:42:16.334+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-09-11T23:41:28.142849+0000 front 2024-09-11T23:41:28.142868+0000 (oldest deadline 2024-09-11T23:41:54.042918+0000) 2024-09-11T23:42:16.803 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:42:16 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:42:16.334+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-09-11T23:41:38.144026+0000 front 2024-09-11T23:41:38.143908+0000 (oldest deadline 2024-09-11T23:42:04.043693+0000) 2024-09-11T23:42:16.803 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:42:16 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:42:16.334+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.26:6822 osd.6 since back 2024-09-11T23:41:44.044474+0000 front 2024-09-11T23:41:44.044429+0000 (oldest deadline 2024-09-11T23:42:07.544154+0000) 2024-09-11T23:42:16.988 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:16 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:42:16.687+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:56.932175+0000 front 2024-09-11T23:40:56.932499+0000 (oldest deadline 2024-09-11T23:41:21.032317+0000) 2024-09-11T23:42:16.989 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:16 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:42:16.687+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:01.032761+0000 front 2024-09-11T23:41:01.032638+0000 (oldest deadline 2024-09-11T23:41:30.433335+0000) 2024-09-11T23:42:16.989 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:16 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:42:16.687+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:10.934510+0000 front 2024-09-11T23:41:10.934411+0000 (oldest deadline 2024-09-11T23:41:35.634455+0000) 2024-09-11T23:42:16.989 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:16 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:42:16.687+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-09-11T23:41:21.936325+0000 front 2024-09-11T23:41:21.936462+0000 (oldest deadline 2024-09-11T23:41:42.937021+0000) 2024-09-11T23:42:16.989 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:16 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:42:16.687+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-09-11T23:41:27.038312+0000 front 2024-09-11T23:41:27.038192+0000 (oldest deadline 2024-09-11T23:41:51.638385+0000) 2024-09-11T23:42:16.989 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:16 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:42:16.687+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-09-11T23:41:37.539377+0000 front 2024-09-11T23:41:37.539405+0000 (oldest deadline 2024-09-11T23:42:02.739809+0000) 2024-09-11T23:42:17.802 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:42:17 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:42:17.323+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:57.536934+0000 front 2024-09-11T23:40:57.537030+0000 (oldest deadline 2024-09-11T23:41:21.036908+0000) 2024-09-11T23:42:17.802 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:42:17 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:42:17.323+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:03.838149+0000 front 2024-09-11T23:41:03.838269+0000 (oldest deadline 2024-09-11T23:41:29.138274+0000) 2024-09-11T23:42:17.802 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:42:17 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:42:17.323+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:12.539763+0000 front 2024-09-11T23:41:12.539893+0000 (oldest deadline 2024-09-11T23:41:36.639808+0000) 2024-09-11T23:42:17.802 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:42:17 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:42:17.323+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-09-11T23:41:21.841541+0000 front 2024-09-11T23:41:21.841518+0000 (oldest deadline 2024-09-11T23:41:44.141586+0000) 2024-09-11T23:42:17.802 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:42:17 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:42:17.323+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-09-11T23:41:28.142849+0000 front 2024-09-11T23:41:28.142868+0000 (oldest deadline 2024-09-11T23:41:54.042918+0000) 2024-09-11T23:42:17.803 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:42:17 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:42:17.323+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-09-11T23:41:38.144026+0000 front 2024-09-11T23:41:38.143908+0000 (oldest deadline 2024-09-11T23:42:04.043693+0000) 2024-09-11T23:42:17.803 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:42:17 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:42:17.323+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.26:6822 osd.6 since back 2024-09-11T23:41:44.044474+0000 front 2024-09-11T23:41:44.044429+0000 (oldest deadline 2024-09-11T23:42:07.544154+0000) 2024-09-11T23:42:17.989 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:17 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:42:17.692+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:56.932175+0000 front 2024-09-11T23:40:56.932499+0000 (oldest deadline 2024-09-11T23:41:21.032317+0000) 2024-09-11T23:42:17.989 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:17 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:42:17.692+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:01.032761+0000 front 2024-09-11T23:41:01.032638+0000 (oldest deadline 2024-09-11T23:41:30.433335+0000) 2024-09-11T23:42:17.989 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:17 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:42:17.692+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:10.934510+0000 front 2024-09-11T23:41:10.934411+0000 (oldest deadline 2024-09-11T23:41:35.634455+0000) 2024-09-11T23:42:17.989 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:17 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:42:17.692+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-09-11T23:41:21.936325+0000 front 2024-09-11T23:41:21.936462+0000 (oldest deadline 2024-09-11T23:41:42.937021+0000) 2024-09-11T23:42:17.989 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:17 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:42:17.692+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-09-11T23:41:27.038312+0000 front 2024-09-11T23:41:27.038192+0000 (oldest deadline 2024-09-11T23:41:51.638385+0000) 2024-09-11T23:42:17.989 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:17 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:42:17.692+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-09-11T23:41:37.539377+0000 front 2024-09-11T23:41:37.539405+0000 (oldest deadline 2024-09-11T23:42:02.739809+0000) 2024-09-11T23:42:18.802 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:42:18 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:42:18.367+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:57.536934+0000 front 2024-09-11T23:40:57.537030+0000 (oldest deadline 2024-09-11T23:41:21.036908+0000) 2024-09-11T23:42:18.802 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:42:18 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:42:18.367+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:03.838149+0000 front 2024-09-11T23:41:03.838269+0000 (oldest deadline 2024-09-11T23:41:29.138274+0000) 2024-09-11T23:42:18.802 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:42:18 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:42:18.367+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:12.539763+0000 front 2024-09-11T23:41:12.539893+0000 (oldest deadline 2024-09-11T23:41:36.639808+0000) 2024-09-11T23:42:18.803 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:42:18 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:42:18.367+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-09-11T23:41:21.841541+0000 front 2024-09-11T23:41:21.841518+0000 (oldest deadline 2024-09-11T23:41:44.141586+0000) 2024-09-11T23:42:18.803 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:42:18 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:42:18.367+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-09-11T23:41:28.142849+0000 front 2024-09-11T23:41:28.142868+0000 (oldest deadline 2024-09-11T23:41:54.042918+0000) 2024-09-11T23:42:18.803 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:42:18 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:42:18.367+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-09-11T23:41:38.144026+0000 front 2024-09-11T23:41:38.143908+0000 (oldest deadline 2024-09-11T23:42:04.043693+0000) 2024-09-11T23:42:18.803 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:42:18 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10[59798]: 2024-09-11T23:42:18.367+0000 7fcb725a3640 -1 osd.10 120 heartbeat_check: no reply from 172.21.15.26:6822 osd.6 since back 2024-09-11T23:41:44.044474+0000 front 2024-09-11T23:41:44.044429+0000 (oldest deadline 2024-09-11T23:42:07.544154+0000) 2024-09-11T23:42:18.989 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:18 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:42:18.692+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:56.932175+0000 front 2024-09-11T23:40:56.932499+0000 (oldest deadline 2024-09-11T23:41:21.032317+0000) 2024-09-11T23:42:18.989 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:18 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:42:18.692+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:01.032761+0000 front 2024-09-11T23:41:01.032638+0000 (oldest deadline 2024-09-11T23:41:30.433335+0000) 2024-09-11T23:42:18.989 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:18 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:42:18.692+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:10.934510+0000 front 2024-09-11T23:41:10.934411+0000 (oldest deadline 2024-09-11T23:41:35.634455+0000) 2024-09-11T23:42:18.989 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:18 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:42:18.692+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-09-11T23:41:21.936325+0000 front 2024-09-11T23:41:21.936462+0000 (oldest deadline 2024-09-11T23:41:42.937021+0000) 2024-09-11T23:42:18.989 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:18 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:42:18.692+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-09-11T23:41:27.038312+0000 front 2024-09-11T23:41:27.038192+0000 (oldest deadline 2024-09-11T23:41:51.638385+0000) 2024-09-11T23:42:18.989 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:18 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:42:18.692+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-09-11T23:41:37.539377+0000 front 2024-09-11T23:41:37.539405+0000 (oldest deadline 2024-09-11T23:42:02.739809+0000) 2024-09-11T23:42:19.302 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:42:19 smithi076 podman[73920]: 2024-09-11 23:42:19.035987242 +0000 UTC m=+5.715849397 container died 1f4f299bfbb212abd0898f56c0049dec5a245729a335862e8f926418dad6df9d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731) 2024-09-11T23:42:19.623 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:42:19 smithi076 podman[73920]: 2024-09-11 23:42:19.432085063 +0000 UTC m=+6.111947210 container cleanup 1f4f299bfbb212abd0898f56c0049dec5a245729a335862e8f926418dad6df9d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-09-11T23:42:19.624 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:42:19 smithi076 bash[73920]: ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10 2024-09-11T23:42:19.988 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:19 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:42:19.659+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:56.932175+0000 front 2024-09-11T23:40:56.932499+0000 (oldest deadline 2024-09-11T23:41:21.032317+0000) 2024-09-11T23:42:19.989 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:19 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:42:19.659+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:01.032761+0000 front 2024-09-11T23:41:01.032638+0000 (oldest deadline 2024-09-11T23:41:30.433335+0000) 2024-09-11T23:42:19.989 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:19 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:42:19.659+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:10.934510+0000 front 2024-09-11T23:41:10.934411+0000 (oldest deadline 2024-09-11T23:41:35.634455+0000) 2024-09-11T23:42:19.989 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:19 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:42:19.659+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-09-11T23:41:21.936325+0000 front 2024-09-11T23:41:21.936462+0000 (oldest deadline 2024-09-11T23:41:42.937021+0000) 2024-09-11T23:42:19.989 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:19 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:42:19.659+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-09-11T23:41:27.038312+0000 front 2024-09-11T23:41:27.038192+0000 (oldest deadline 2024-09-11T23:41:51.638385+0000) 2024-09-11T23:42:19.989 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:19 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:42:19.659+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-09-11T23:41:37.539377+0000 front 2024-09-11T23:41:37.539405+0000 (oldest deadline 2024-09-11T23:42:02.739809+0000) 2024-09-11T23:42:19.989 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:19 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:42:19.659+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.76:6820 osd.7 since back 2024-09-11T23:41:53.142305+0000 front 2024-09-11T23:41:53.142362+0000 (oldest deadline 2024-09-11T23:42:19.042202+0000) 2024-09-11T23:42:20.019 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:42:19 smithi076 podman[73936]: 2024-09-11 23:42:19.624837459 +0000 UTC m=+0.591389899 container remove 1f4f299bfbb212abd0898f56c0049dec5a245729a335862e8f926418dad6df9d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-09-11T23:42:20.277 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:42:20 smithi076 podman[74026]: 2024-09-11 23:42:19.923365384 +0000 UTC m=+0.015889276 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T23:42:20.277 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:42:20 smithi076 podman[74026]: 2024-09-11 23:42:20.034197745 +0000 UTC m=+0.126721683 container create 017f05f26802f28940736ed8a1d6265264ba00b9582d409445be6b01e9908571 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, RELEASE=squid-b53b6d3) 2024-09-11T23:42:20.277 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:42:20 smithi076 podman[74026]: 2024-09-11 23:42:20.278567861 +0000 UTC m=+0.371091740 container init 017f05f26802f28940736ed8a1d6265264ba00b9582d409445be6b01e9908571 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10-deactivate, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-09-11T23:42:20.552 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:42:20 smithi076 podman[74026]: 2024-09-11 23:42:20.281882204 +0000 UTC m=+0.374406096 container start 017f05f26802f28940736ed8a1d6265264ba00b9582d409445be6b01e9908571 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10-deactivate, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-09-11T23:42:20.553 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:42:20 smithi076 podman[74026]: 2024-09-11 23:42:20.361683623 +0000 UTC m=+0.454207510 container attach 017f05f26802f28940736ed8a1d6265264ba00b9582d409445be6b01e9908571 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10-deactivate, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-09-11T23:42:20.553 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:42:20 smithi076 podman[74026]: 2024-09-11 23:42:20.472181186 +0000 UTC m=+0.564705078 container died 017f05f26802f28940736ed8a1d6265264ba00b9582d409445be6b01e9908571 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10-deactivate, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0) 2024-09-11T23:42:20.989 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:20 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:42:20.680+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:56.932175+0000 front 2024-09-11T23:40:56.932499+0000 (oldest deadline 2024-09-11T23:41:21.032317+0000) 2024-09-11T23:42:20.989 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:20 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:42:20.680+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:01.032761+0000 front 2024-09-11T23:41:01.032638+0000 (oldest deadline 2024-09-11T23:41:30.433335+0000) 2024-09-11T23:42:20.989 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:20 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:42:20.680+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:10.934510+0000 front 2024-09-11T23:41:10.934411+0000 (oldest deadline 2024-09-11T23:41:35.634455+0000) 2024-09-11T23:42:20.989 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:20 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:42:20.680+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-09-11T23:41:21.936325+0000 front 2024-09-11T23:41:21.936462+0000 (oldest deadline 2024-09-11T23:41:42.937021+0000) 2024-09-11T23:42:20.989 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:20 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:42:20.680+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-09-11T23:41:27.038312+0000 front 2024-09-11T23:41:27.038192+0000 (oldest deadline 2024-09-11T23:41:51.638385+0000) 2024-09-11T23:42:20.989 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:20 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:42:20.680+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-09-11T23:41:37.539377+0000 front 2024-09-11T23:41:37.539405+0000 (oldest deadline 2024-09-11T23:42:02.739809+0000) 2024-09-11T23:42:20.989 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:20 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:42:20.681+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.76:6820 osd.7 since back 2024-09-11T23:41:53.142305+0000 front 2024-09-11T23:41:53.142362+0000 (oldest deadline 2024-09-11T23:42:19.042202+0000) 2024-09-11T23:42:21.032 INFO:journalctl@ceph.osd.10.smithi076.stdout:Sep 11 23:42:21 smithi076 podman[74049]: 2024-09-11 23:42:21.014874949 +0000 UTC m=+0.536639079 container remove 017f05f26802f28940736ed8a1d6265264ba00b9582d409445be6b01e9908571 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-10-deactivate, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3) 2024-09-11T23:42:21.037 DEBUG:teuthology.orchestra.run.smithi076:> sudo pkill -f 'journalctl -f -n 0 -u ceph-af7e6fc8-7092-11ef-bcea-c7b262605968@osd.10.service' 2024-09-11T23:42:21.077 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T23:42:21.077 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-09-11T23:42:21.077 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-09-11T23:42:21.077 DEBUG:teuthology.orchestra.run.smithi136:> sudo systemctl stop ceph-af7e6fc8-7092-11ef-bcea-c7b262605968@osd.11 2024-09-11T23:42:21.363 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:21 smithi136 systemd[1]: Stopping Ceph osd.11 for af7e6fc8-7092-11ef-bcea-c7b262605968... 2024-09-11T23:42:21.363 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:21 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:42:21.362+0000 7f20e6832640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T23:42:21.363 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:21 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:42:21.362+0000 7f20e6832640 -1 osd.11 120 *** Got signal Terminated *** 2024-09-11T23:42:21.363 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:21 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:42:21.362+0000 7f20e6832640 -1 osd.11 120 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T23:42:21.988 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:21 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:42:21.633+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:56.932175+0000 front 2024-09-11T23:40:56.932499+0000 (oldest deadline 2024-09-11T23:41:21.032317+0000) 2024-09-11T23:42:21.989 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:21 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:42:21.633+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:01.032761+0000 front 2024-09-11T23:41:01.032638+0000 (oldest deadline 2024-09-11T23:41:30.433335+0000) 2024-09-11T23:42:21.989 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:21 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:42:21.633+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:10.934510+0000 front 2024-09-11T23:41:10.934411+0000 (oldest deadline 2024-09-11T23:41:35.634455+0000) 2024-09-11T23:42:21.993 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:21 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:42:21.633+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-09-11T23:41:21.936325+0000 front 2024-09-11T23:41:21.936462+0000 (oldest deadline 2024-09-11T23:41:42.937021+0000) 2024-09-11T23:42:21.993 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:21 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:42:21.633+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-09-11T23:41:27.038312+0000 front 2024-09-11T23:41:27.038192+0000 (oldest deadline 2024-09-11T23:41:51.638385+0000) 2024-09-11T23:42:21.993 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:21 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:42:21.633+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-09-11T23:41:37.539377+0000 front 2024-09-11T23:41:37.539405+0000 (oldest deadline 2024-09-11T23:42:02.739809+0000) 2024-09-11T23:42:21.993 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:21 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:42:21.633+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.76:6820 osd.7 since back 2024-09-11T23:41:53.142305+0000 front 2024-09-11T23:41:53.142362+0000 (oldest deadline 2024-09-11T23:42:19.042202+0000) 2024-09-11T23:42:22.988 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:22 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:42:22.597+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:56.932175+0000 front 2024-09-11T23:40:56.932499+0000 (oldest deadline 2024-09-11T23:41:21.032317+0000) 2024-09-11T23:42:22.989 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:22 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:42:22.597+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:01.032761+0000 front 2024-09-11T23:41:01.032638+0000 (oldest deadline 2024-09-11T23:41:30.433335+0000) 2024-09-11T23:42:22.989 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:22 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:42:22.597+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:10.934510+0000 front 2024-09-11T23:41:10.934411+0000 (oldest deadline 2024-09-11T23:41:35.634455+0000) 2024-09-11T23:42:22.989 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:22 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:42:22.597+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-09-11T23:41:21.936325+0000 front 2024-09-11T23:41:21.936462+0000 (oldest deadline 2024-09-11T23:41:42.937021+0000) 2024-09-11T23:42:22.989 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:22 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:42:22.597+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-09-11T23:41:27.038312+0000 front 2024-09-11T23:41:27.038192+0000 (oldest deadline 2024-09-11T23:41:51.638385+0000) 2024-09-11T23:42:22.989 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:22 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:42:22.597+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-09-11T23:41:37.539377+0000 front 2024-09-11T23:41:37.539405+0000 (oldest deadline 2024-09-11T23:42:02.739809+0000) 2024-09-11T23:42:22.989 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:22 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:42:22.597+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.76:6820 osd.7 since back 2024-09-11T23:41:53.142305+0000 front 2024-09-11T23:41:53.142362+0000 (oldest deadline 2024-09-11T23:42:19.042202+0000) 2024-09-11T23:42:23.989 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:23 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:42:23.602+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:56.932175+0000 front 2024-09-11T23:40:56.932499+0000 (oldest deadline 2024-09-11T23:41:21.032317+0000) 2024-09-11T23:42:23.989 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:23 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:42:23.602+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:01.032761+0000 front 2024-09-11T23:41:01.032638+0000 (oldest deadline 2024-09-11T23:41:30.433335+0000) 2024-09-11T23:42:23.989 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:23 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:42:23.602+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:10.934510+0000 front 2024-09-11T23:41:10.934411+0000 (oldest deadline 2024-09-11T23:41:35.634455+0000) 2024-09-11T23:42:23.989 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:23 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:42:23.602+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-09-11T23:41:21.936325+0000 front 2024-09-11T23:41:21.936462+0000 (oldest deadline 2024-09-11T23:41:42.937021+0000) 2024-09-11T23:42:23.989 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:23 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:42:23.602+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-09-11T23:41:27.038312+0000 front 2024-09-11T23:41:27.038192+0000 (oldest deadline 2024-09-11T23:41:51.638385+0000) 2024-09-11T23:42:23.989 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:23 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:42:23.602+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-09-11T23:41:37.539377+0000 front 2024-09-11T23:41:37.539405+0000 (oldest deadline 2024-09-11T23:42:02.739809+0000) 2024-09-11T23:42:23.989 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:23 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:42:23.602+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.76:6820 osd.7 since back 2024-09-11T23:41:53.142305+0000 front 2024-09-11T23:41:53.142362+0000 (oldest deadline 2024-09-11T23:42:19.042202+0000) 2024-09-11T23:42:24.988 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:24 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:42:24.642+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:56.932175+0000 front 2024-09-11T23:40:56.932499+0000 (oldest deadline 2024-09-11T23:41:21.032317+0000) 2024-09-11T23:42:24.989 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:24 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:42:24.642+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:01.032761+0000 front 2024-09-11T23:41:01.032638+0000 (oldest deadline 2024-09-11T23:41:30.433335+0000) 2024-09-11T23:42:24.989 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:24 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:42:24.642+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:10.934510+0000 front 2024-09-11T23:41:10.934411+0000 (oldest deadline 2024-09-11T23:41:35.634455+0000) 2024-09-11T23:42:24.989 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:24 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:42:24.642+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-09-11T23:41:21.936325+0000 front 2024-09-11T23:41:21.936462+0000 (oldest deadline 2024-09-11T23:41:42.937021+0000) 2024-09-11T23:42:24.989 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:24 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:42:24.642+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-09-11T23:41:27.038312+0000 front 2024-09-11T23:41:27.038192+0000 (oldest deadline 2024-09-11T23:41:51.638385+0000) 2024-09-11T23:42:24.989 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:24 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:42:24.642+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-09-11T23:41:37.539377+0000 front 2024-09-11T23:41:37.539405+0000 (oldest deadline 2024-09-11T23:42:02.739809+0000) 2024-09-11T23:42:24.989 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:24 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:42:24.642+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.76:6820 osd.7 since back 2024-09-11T23:41:53.142305+0000 front 2024-09-11T23:41:53.142362+0000 (oldest deadline 2024-09-11T23:42:19.042202+0000) 2024-09-11T23:42:25.989 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:25 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:42:25.690+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-09-11T23:40:56.932175+0000 front 2024-09-11T23:40:56.932499+0000 (oldest deadline 2024-09-11T23:41:21.032317+0000) 2024-09-11T23:42:25.989 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:25 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:42:25.690+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-09-11T23:41:01.032761+0000 front 2024-09-11T23:41:01.032638+0000 (oldest deadline 2024-09-11T23:41:30.433335+0000) 2024-09-11T23:42:25.989 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:25 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:42:25.690+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-11T23:41:10.934510+0000 front 2024-09-11T23:41:10.934411+0000 (oldest deadline 2024-09-11T23:41:35.634455+0000) 2024-09-11T23:42:25.989 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:25 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:42:25.690+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-09-11T23:41:21.936325+0000 front 2024-09-11T23:41:21.936462+0000 (oldest deadline 2024-09-11T23:41:42.937021+0000) 2024-09-11T23:42:25.989 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:25 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:42:25.690+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-09-11T23:41:27.038312+0000 front 2024-09-11T23:41:27.038192+0000 (oldest deadline 2024-09-11T23:41:51.638385+0000) 2024-09-11T23:42:25.989 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:25 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:42:25.690+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-09-11T23:41:37.539377+0000 front 2024-09-11T23:41:37.539405+0000 (oldest deadline 2024-09-11T23:42:02.739809+0000) 2024-09-11T23:42:25.989 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:25 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11[60155]: 2024-09-11T23:42:25.690+0000 7f20dee2f640 -1 osd.11 120 heartbeat_check: no reply from 172.21.15.76:6820 osd.7 since back 2024-09-11T23:41:53.142305+0000 front 2024-09-11T23:41:53.142362+0000 (oldest deadline 2024-09-11T23:42:19.042202+0000) 2024-09-11T23:42:27.238 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:26 smithi136 podman[74228]: 2024-09-11 23:42:26.961952907 +0000 UTC m=+5.692019135 container died fd25674f12e38a9cd90948296238dfe0c53ef6e7a6a1410a7089881c99a9c967 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-09-11T23:42:27.738 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:27 smithi136 podman[74228]: 2024-09-11 23:42:27.262415956 +0000 UTC m=+5.992482190 container cleanup fd25674f12e38a9cd90948296238dfe0c53ef6e7a6a1410a7089881c99a9c967 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-09-11T23:42:27.739 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:27 smithi136 bash[74228]: ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11 2024-09-11T23:42:27.739 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:27 smithi136 podman[74244]: 2024-09-11 23:42:27.463878342 +0000 UTC m=+0.497790079 container remove fd25674f12e38a9cd90948296238dfe0c53ef6e7a6a1410a7089881c99a9c967 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, ceph=True) 2024-09-11T23:42:28.166 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:27 smithi136 podman[74336]: 2024-09-11 23:42:27.773500679 +0000 UTC m=+0.015661163 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T23:42:28.166 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:27 smithi136 podman[74336]: 2024-09-11 23:42:27.881313172 +0000 UTC m=+0.123473684 container create 35b37f24a40272ae9513b39af433e91a1712d88236ae39967abd99acf274e56f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11-deactivate, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731) 2024-09-11T23:42:28.166 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:28 smithi136 podman[74336]: 2024-09-11 23:42:28.083886331 +0000 UTC m=+0.326046801 container init 35b37f24a40272ae9513b39af433e91a1712d88236ae39967abd99acf274e56f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11-deactivate, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, io.buildah.version=1.37.2) 2024-09-11T23:42:28.166 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:28 smithi136 podman[74336]: 2024-09-11 23:42:28.08715867 +0000 UTC m=+0.329319141 container start 35b37f24a40272ae9513b39af433e91a1712d88236ae39967abd99acf274e56f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11-deactivate, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, ceph=True, maintainer=Guillaume Abrioux ) 2024-09-11T23:42:28.166 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:28 smithi136 podman[74336]: 2024-09-11 23:42:28.167143382 +0000 UTC m=+0.409303855 container attach 35b37f24a40272ae9513b39af433e91a1712d88236ae39967abd99acf274e56f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11-deactivate, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2) 2024-09-11T23:42:28.488 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 11 23:42:28 smithi136 podman[74336]: 2024-09-11 23:42:28.270987236 +0000 UTC m=+0.513147720 container died 35b37f24a40272ae9513b39af433e91a1712d88236ae39967abd99acf274e56f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-osd-11-deactivate, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T23:42:28.861 DEBUG:teuthology.orchestra.run.smithi136:> sudo pkill -f 'journalctl -f -n 0 -u ceph-af7e6fc8-7092-11ef-bcea-c7b262605968@osd.11.service' 2024-09-11T23:42:28.899 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T23:42:28.900 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-09-11T23:42:28.900 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-09-11T23:42:28.900 DEBUG:teuthology.orchestra.run.smithi026:> sudo systemctl stop ceph-af7e6fc8-7092-11ef-bcea-c7b262605968@mds.a 2024-09-11T23:42:29.209 INFO:journalctl@ceph.mds.a.smithi026.stdout:Sep 11 23:42:28 smithi026 systemd[1]: Stopping Ceph mds.a for af7e6fc8-7092-11ef-bcea-c7b262605968... 2024-09-11T23:42:29.500 INFO:journalctl@ceph.mds.a.smithi026.stdout:Sep 11 23:42:29 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mds-a[72516]: 2024-09-11T23:42:29.210+0000 7fdeaeb1e640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T23:42:29.500 INFO:journalctl@ceph.mds.a.smithi026.stdout:Sep 11 23:42:29 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mds-a[72516]: 2024-09-11T23:42:29.210+0000 7fdeaeb1e640 -1 mds.a *** got signal Terminated *** 2024-09-11T23:42:33.234 INFO:journalctl@ceph.mds.a.smithi026.stdout:Sep 11 23:42:32 smithi026 podman[115349]: 2024-09-11 23:42:32.94100771 +0000 UTC m=+3.837376410 container died d6d92c598f2937c16f121752b993d7e9ce493b868f6426b714bfd8a4d3dc0750 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mds-a, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T23:42:33.500 INFO:journalctl@ceph.mds.a.smithi026.stdout:Sep 11 23:42:33 smithi026 podman[115349]: 2024-09-11 23:42:33.309698864 +0000 UTC m=+4.206067566 container cleanup d6d92c598f2937c16f121752b993d7e9ce493b868f6426b714bfd8a4d3dc0750 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mds-a, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-09-11T23:42:33.500 INFO:journalctl@ceph.mds.a.smithi026.stdout:Sep 11 23:42:33 smithi026 bash[115349]: ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mds-a 2024-09-11T23:42:33.751 DEBUG:teuthology.orchestra.run.smithi026:> sudo pkill -f 'journalctl -f -n 0 -u ceph-af7e6fc8-7092-11ef-bcea-c7b262605968@mds.a.service' 2024-09-11T23:42:33.787 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T23:42:33.787 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-09-11T23:42:33.787 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-09-11T23:42:33.787 DEBUG:teuthology.orchestra.run.smithi026:> sudo systemctl stop ceph-af7e6fc8-7092-11ef-bcea-c7b262605968@mds.d 2024-09-11T23:42:34.252 INFO:journalctl@ceph.mds.d.smithi026.stdout:Sep 11 23:42:33 smithi026 systemd[1]: Stopping Ceph mds.d for af7e6fc8-7092-11ef-bcea-c7b262605968... 2024-09-11T23:42:34.252 INFO:journalctl@ceph.mds.d.smithi026.stdout:Sep 11 23:42:34 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mds-d[71333]: 2024-09-11T23:42:34.113+0000 7fdad5857640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T23:42:34.252 INFO:journalctl@ceph.mds.d.smithi026.stdout:Sep 11 23:42:34 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mds-d[71333]: 2024-09-11T23:42:34.114+0000 7fdad5857640 -1 mds.d *** got signal Terminated *** 2024-09-11T23:42:37.904 INFO:journalctl@ceph.mds.d.smithi026.stdout:Sep 11 23:42:37 smithi026 podman[115473]: 2024-09-11 23:42:37.635069793 +0000 UTC m=+3.625148375 container died 25e20ec815ee50f0cf3aa6bd444631953c5d3e6aa8f21635da503a6ccc770f96 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mds-d, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2) 2024-09-11T23:42:38.163 INFO:journalctl@ceph.mds.d.smithi026.stdout:Sep 11 23:42:37 smithi026 podman[115473]: 2024-09-11 23:42:37.979916915 +0000 UTC m=+3.969995486 container cleanup 25e20ec815ee50f0cf3aa6bd444631953c5d3e6aa8f21635da503a6ccc770f96 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mds-d, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True) 2024-09-11T23:42:38.164 INFO:journalctl@ceph.mds.d.smithi026.stdout:Sep 11 23:42:37 smithi026 bash[115473]: ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mds-d 2024-09-11T23:42:38.408 DEBUG:teuthology.orchestra.run.smithi026:> sudo pkill -f 'journalctl -f -n 0 -u ceph-af7e6fc8-7092-11ef-bcea-c7b262605968@mds.d.service' 2024-09-11T23:42:38.430 INFO:journalctl@ceph.mds.d.smithi026.stdout:Sep 11 23:42:38 smithi026 podman[115485]: 2024-09-11 23:42:38.164577518 +0000 UTC m=+0.521069217 container remove 25e20ec815ee50f0cf3aa6bd444631953c5d3e6aa8f21635da503a6ccc770f96 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mds-d, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240731) 2024-09-11T23:42:38.430 INFO:journalctl@ceph.mds.d.smithi026.stdout:Sep 11 23:42:38 smithi026 systemd[1]: ceph-af7e6fc8-7092-11ef-bcea-c7b262605968@mds.d.service: Deactivated successfully. 2024-09-11T23:42:38.430 INFO:journalctl@ceph.mds.d.smithi026.stdout:Sep 11 23:42:38 smithi026 systemd[1]: Stopped Ceph mds.d for af7e6fc8-7092-11ef-bcea-c7b262605968. 2024-09-11T23:42:38.430 INFO:journalctl@ceph.mds.d.smithi026.stdout:Sep 11 23:42:38 smithi026 systemd[1]: ceph-af7e6fc8-7092-11ef-bcea-c7b262605968@mds.d.service: Consumed 1.203s CPU time. 2024-09-11T23:42:38.442 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T23:42:38.442 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-09-11T23:42:38.442 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-09-11T23:42:38.442 DEBUG:teuthology.orchestra.run.smithi026:> sudo systemctl stop ceph-af7e6fc8-7092-11ef-bcea-c7b262605968@mds.g 2024-09-11T23:42:38.750 INFO:journalctl@ceph.mds.g.smithi026.stdout:Sep 11 23:42:38 smithi026 systemd[1]: Stopping Ceph mds.g for af7e6fc8-7092-11ef-bcea-c7b262605968... 2024-09-11T23:42:39.250 INFO:journalctl@ceph.mds.g.smithi026.stdout:Sep 11 23:42:38 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mds-g[72114]: 2024-09-11T23:42:38.770+0000 7f5758b7d640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T23:42:39.250 INFO:journalctl@ceph.mds.g.smithi026.stdout:Sep 11 23:42:38 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mds-g[72114]: 2024-09-11T23:42:38.770+0000 7f5758b7d640 -1 mds.g *** got signal Terminated *** 2024-09-11T23:42:41.225 INFO:journalctl@ceph.mds.g.smithi026.stdout:Sep 11 23:42:40 smithi026 podman[115591]: 2024-09-11 23:42:40.945685042 +0000 UTC m=+2.278377136 container died 79a2897e16eff7cb1d8488bff9be0e3baaa01d46a3f1b87378f87e808c14c063 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mds-g, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-09-11T23:42:41.500 INFO:journalctl@ceph.mds.g.smithi026.stdout:Sep 11 23:42:41 smithi026 podman[115591]: 2024-09-11 23:42:41.325832778 +0000 UTC m=+2.658524859 container cleanup 79a2897e16eff7cb1d8488bff9be0e3baaa01d46a3f1b87378f87e808c14c063 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mds-g, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, ceph=True, GIT_CLEAN=True, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-09-11T23:42:41.500 INFO:journalctl@ceph.mds.g.smithi026.stdout:Sep 11 23:42:41 smithi026 bash[115591]: ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mds-g 2024-09-11T23:42:41.706 DEBUG:teuthology.orchestra.run.smithi026:> sudo pkill -f 'journalctl -f -n 0 -u ceph-af7e6fc8-7092-11ef-bcea-c7b262605968@mds.g.service' 2024-09-11T23:42:41.742 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T23:42:41.742 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-09-11T23:42:41.742 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-09-11T23:42:41.742 DEBUG:teuthology.orchestra.run.smithi026:> sudo systemctl stop ceph-af7e6fc8-7092-11ef-bcea-c7b262605968@mds.j 2024-09-11T23:42:42.250 INFO:journalctl@ceph.mds.j.smithi026.stdout:Sep 11 23:42:41 smithi026 systemd[1]: Stopping Ceph mds.j for af7e6fc8-7092-11ef-bcea-c7b262605968... 2024-09-11T23:42:42.250 INFO:journalctl@ceph.mds.j.smithi026.stdout:Sep 11 23:42:42 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mds-j[70934]: 2024-09-11T23:42:42.080+0000 7fa9d5835640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T23:42:42.250 INFO:journalctl@ceph.mds.j.smithi026.stdout:Sep 11 23:42:42 smithi026 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mds-j[70934]: 2024-09-11T23:42:42.080+0000 7fa9d5835640 -1 mds.j *** got signal Terminated *** 2024-09-11T23:42:46.354 INFO:journalctl@ceph.mds.j.smithi026.stdout:Sep 11 23:42:46 smithi026 podman[115722]: 2024-09-11 23:42:46.096130992 +0000 UTC m=+4.108809165 container died f4bc32015a30ec9583fe84c7986e61c4daae17b7096e52e1613908266c88eb07 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mds-j, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.37.2) 2024-09-11T23:42:46.712 INFO:journalctl@ceph.mds.j.smithi026.stdout:Sep 11 23:42:46 smithi026 podman[115722]: 2024-09-11 23:42:46.45449547 +0000 UTC m=+4.467173630 container cleanup f4bc32015a30ec9583fe84c7986e61c4daae17b7096e52e1613908266c88eb07 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mds-j, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.37.2, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-09-11T23:42:46.713 INFO:journalctl@ceph.mds.j.smithi026.stdout:Sep 11 23:42:46 smithi026 bash[115722]: ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mds-j 2024-09-11T23:42:46.894 DEBUG:teuthology.orchestra.run.smithi026:> sudo pkill -f 'journalctl -f -n 0 -u ceph-af7e6fc8-7092-11ef-bcea-c7b262605968@mds.j.service' 2024-09-11T23:42:46.933 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T23:42:46.934 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-09-11T23:42:46.934 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-09-11T23:42:46.934 DEBUG:teuthology.orchestra.run.smithi076:> sudo systemctl stop ceph-af7e6fc8-7092-11ef-bcea-c7b262605968@mds.b 2024-09-11T23:42:47.240 INFO:journalctl@ceph.mds.b.smithi076.stdout:Sep 11 23:42:46 smithi076 systemd[1]: Stopping Ceph mds.b for af7e6fc8-7092-11ef-bcea-c7b262605968... 2024-09-11T23:42:47.491 INFO:journalctl@ceph.mds.b.smithi076.stdout:Sep 11 23:42:47 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mds-b[65334]: 2024-09-11T23:42:47.240+0000 7f460ded2640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T23:42:47.491 INFO:journalctl@ceph.mds.b.smithi076.stdout:Sep 11 23:42:47 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mds-b[65334]: 2024-09-11T23:42:47.240+0000 7f460ded2640 -1 mds.b *** got signal Terminated *** 2024-09-11T23:42:49.298 INFO:journalctl@ceph.mds.b.smithi076.stdout:Sep 11 23:42:49 smithi076 podman[74128]: 2024-09-11 23:42:49.003320641 +0000 UTC m=+1.879463124 container died 74534ea440b6df92a73584ec5c812f08d4fece74c621c1f11fcfddf89e5cf65e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mds-b, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, ceph=True) 2024-09-11T23:42:49.552 INFO:journalctl@ceph.mds.b.smithi076.stdout:Sep 11 23:42:49 smithi076 podman[74128]: 2024-09-11 23:42:49.41726817 +0000 UTC m=+2.293410652 container cleanup 74534ea440b6df92a73584ec5c812f08d4fece74c621c1f11fcfddf89e5cf65e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mds-b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-09-11T23:42:49.552 INFO:journalctl@ceph.mds.b.smithi076.stdout:Sep 11 23:42:49 smithi076 bash[74128]: ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mds-b 2024-09-11T23:42:49.875 DEBUG:teuthology.orchestra.run.smithi076:> sudo pkill -f 'journalctl -f -n 0 -u ceph-af7e6fc8-7092-11ef-bcea-c7b262605968@mds.b.service' 2024-09-11T23:42:49.913 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T23:42:49.913 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-09-11T23:42:49.913 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-09-11T23:42:49.913 DEBUG:teuthology.orchestra.run.smithi076:> sudo systemctl stop ceph-af7e6fc8-7092-11ef-bcea-c7b262605968@mds.e 2024-09-11T23:42:50.245 INFO:journalctl@ceph.mds.e.smithi076.stdout:Sep 11 23:42:49 smithi076 systemd[1]: Stopping Ceph mds.e for af7e6fc8-7092-11ef-bcea-c7b262605968... 2024-09-11T23:42:50.552 INFO:journalctl@ceph.mds.e.smithi076.stdout:Sep 11 23:42:50 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mds-e[64717]: 2024-09-11T23:42:50.244+0000 7f562107e640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T23:42:50.552 INFO:journalctl@ceph.mds.e.smithi076.stdout:Sep 11 23:42:50 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mds-e[64717]: 2024-09-11T23:42:50.244+0000 7f562107e640 -1 mds.e *** got signal Terminated *** 2024-09-11T23:42:53.302 INFO:journalctl@ceph.mds.e.smithi076.stdout:Sep 11 23:42:53 smithi076 podman[74258]: 2024-09-11 23:42:53.00446253 +0000 UTC m=+2.867430207 container died 6f0b7c1d8960b84a39a890e90d7721272855c7ff693dec739914568fa7c8bb1b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mds-e, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, ceph=True) 2024-09-11T23:42:53.695 INFO:journalctl@ceph.mds.e.smithi076.stdout:Sep 11 23:42:53 smithi076 podman[74258]: 2024-09-11 23:42:53.411673625 +0000 UTC m=+3.274641298 container cleanup 6f0b7c1d8960b84a39a890e90d7721272855c7ff693dec739914568fa7c8bb1b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mds-e, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2) 2024-09-11T23:42:53.695 INFO:journalctl@ceph.mds.e.smithi076.stdout:Sep 11 23:42:53 smithi076 bash[74258]: ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mds-e 2024-09-11T23:42:53.854 DEBUG:teuthology.orchestra.run.smithi076:> sudo pkill -f 'journalctl -f -n 0 -u ceph-af7e6fc8-7092-11ef-bcea-c7b262605968@mds.e.service' 2024-09-11T23:42:53.888 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T23:42:53.889 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-09-11T23:42:53.889 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-09-11T23:42:53.889 DEBUG:teuthology.orchestra.run.smithi076:> sudo systemctl stop ceph-af7e6fc8-7092-11ef-bcea-c7b262605968@mds.h 2024-09-11T23:42:54.214 INFO:journalctl@ceph.mds.h.smithi076.stdout:Sep 11 23:42:53 smithi076 systemd[1]: Stopping Ceph mds.h for af7e6fc8-7092-11ef-bcea-c7b262605968... 2024-09-11T23:42:54.552 INFO:journalctl@ceph.mds.h.smithi076.stdout:Sep 11 23:42:54 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mds-h[65015]: 2024-09-11T23:42:54.214+0000 7fc3a50ad640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T23:42:54.552 INFO:journalctl@ceph.mds.h.smithi076.stdout:Sep 11 23:42:54 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mds-h[65015]: 2024-09-11T23:42:54.214+0000 7fc3a50ad640 -1 mds.h *** got signal Terminated *** 2024-09-11T23:42:57.298 INFO:journalctl@ceph.mds.h.smithi076.stdout:Sep 11 23:42:56 smithi076 podman[74380]: 2024-09-11 23:42:56.992811632 +0000 UTC m=+2.882787917 container died 359eac0589c22758f5dfccb41c2351b3766f952837b5989f09def23b13a7dff8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mds-h, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, ceph=True, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-09-11T23:42:57.552 INFO:journalctl@ceph.mds.h.smithi076.stdout:Sep 11 23:42:57 smithi076 podman[74380]: 2024-09-11 23:42:57.382407219 +0000 UTC m=+3.272383505 container cleanup 359eac0589c22758f5dfccb41c2351b3766f952837b5989f09def23b13a7dff8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mds-h, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-09-11T23:42:57.553 INFO:journalctl@ceph.mds.h.smithi076.stdout:Sep 11 23:42:57 smithi076 bash[74380]: ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mds-h 2024-09-11T23:42:57.812 DEBUG:teuthology.orchestra.run.smithi076:> sudo pkill -f 'journalctl -f -n 0 -u ceph-af7e6fc8-7092-11ef-bcea-c7b262605968@mds.h.service' 2024-09-11T23:42:57.844 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T23:42:57.844 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-09-11T23:42:57.844 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-09-11T23:42:57.845 DEBUG:teuthology.orchestra.run.smithi076:> sudo systemctl stop ceph-af7e6fc8-7092-11ef-bcea-c7b262605968@mds.k 2024-09-11T23:42:58.168 INFO:journalctl@ceph.mds.k.smithi076.stdout:Sep 11 23:42:57 smithi076 systemd[1]: Stopping Ceph mds.k for af7e6fc8-7092-11ef-bcea-c7b262605968... 2024-09-11T23:42:58.510 INFO:journalctl@ceph.mds.k.smithi076.stdout:Sep 11 23:42:58 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mds-k[64405]: 2024-09-11T23:42:58.169+0000 7efcd700a640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T23:42:58.510 INFO:journalctl@ceph.mds.k.smithi076.stdout:Sep 11 23:42:58 smithi076 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mds-k[64405]: 2024-09-11T23:42:58.169+0000 7efcd700a640 -1 mds.k *** got signal Terminated *** 2024-09-11T23:43:01.261 INFO:journalctl@ceph.mds.k.smithi076.stdout:Sep 11 23:43:01 smithi076 podman[74502]: 2024-09-11 23:43:01.006726224 +0000 UTC m=+2.940938489 container died 0a8b3488447032b560cf12b53ead05570c08493a5f0654814100e8fb935e0e33 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mds-k, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_CLEAN=True, io.buildah.version=1.37.2) 2024-09-11T23:43:01.552 INFO:journalctl@ceph.mds.k.smithi076.stdout:Sep 11 23:43:01 smithi076 podman[74502]: 2024-09-11 23:43:01.344614195 +0000 UTC m=+3.278826469 container cleanup 0a8b3488447032b560cf12b53ead05570c08493a5f0654814100e8fb935e0e33 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mds-k, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-09-11T23:43:01.552 INFO:journalctl@ceph.mds.k.smithi076.stdout:Sep 11 23:43:01 smithi076 bash[74502]: ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mds-k 2024-09-11T23:43:01.815 DEBUG:teuthology.orchestra.run.smithi076:> sudo pkill -f 'journalctl -f -n 0 -u ceph-af7e6fc8-7092-11ef-bcea-c7b262605968@mds.k.service' 2024-09-11T23:43:01.837 INFO:journalctl@ceph.mds.k.smithi076.stdout:Sep 11 23:43:01 smithi076 podman[74516]: 2024-09-11 23:43:01.570645439 +0000 UTC m=+0.550336838 container remove 0a8b3488447032b560cf12b53ead05570c08493a5f0654814100e8fb935e0e33 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mds-k, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T23:43:01.837 INFO:journalctl@ceph.mds.k.smithi076.stdout:Sep 11 23:43:01 smithi076 systemd[1]: ceph-af7e6fc8-7092-11ef-bcea-c7b262605968@mds.k.service: Deactivated successfully. 2024-09-11T23:43:01.837 INFO:journalctl@ceph.mds.k.smithi076.stdout:Sep 11 23:43:01 smithi076 systemd[1]: Stopped Ceph mds.k for af7e6fc8-7092-11ef-bcea-c7b262605968. 2024-09-11T23:43:01.837 INFO:journalctl@ceph.mds.k.smithi076.stdout:Sep 11 23:43:01 smithi076 systemd[1]: ceph-af7e6fc8-7092-11ef-bcea-c7b262605968@mds.k.service: Consumed 57.942s CPU time. 2024-09-11T23:43:01.848 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T23:43:01.849 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-09-11T23:43:01.849 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-09-11T23:43:01.849 DEBUG:teuthology.orchestra.run.smithi136:> sudo systemctl stop ceph-af7e6fc8-7092-11ef-bcea-c7b262605968@mds.c 2024-09-11T23:43:02.238 INFO:journalctl@ceph.mds.c.smithi136.stdout:Sep 11 23:43:01 smithi136 systemd[1]: Stopping Ceph mds.c for af7e6fc8-7092-11ef-bcea-c7b262605968... 2024-09-11T23:43:02.738 INFO:journalctl@ceph.mds.c.smithi136.stdout:Sep 11 23:43:02 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mds-c[65041]: 2024-09-11T23:43:02.375+0000 7fb4bfd26640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T23:43:02.739 INFO:journalctl@ceph.mds.c.smithi136.stdout:Sep 11 23:43:02 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mds-c[65041]: 2024-09-11T23:43:02.375+0000 7fb4bfd26640 -1 mds.c *** got signal Terminated *** 2024-09-11T23:43:05.459 INFO:journalctl@ceph.mds.c.smithi136.stdout:Sep 11 23:43:05 smithi136 podman[74452]: 2024-09-11 23:43:05.193202623 +0000 UTC m=+3.145998182 container died a8bc986f7080e19e4c4f2a05c015fdb63f204e4d1aafe91156eb8d50f14786ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mds-c, ceph=True, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_BRANCH=HEAD) 2024-09-11T23:43:05.710 INFO:journalctl@ceph.mds.c.smithi136.stdout:Sep 11 23:43:05 smithi136 podman[74452]: 2024-09-11 23:43:05.534593549 +0000 UTC m=+3.487389105 container cleanup a8bc986f7080e19e4c4f2a05c015fdb63f204e4d1aafe91156eb8d50f14786ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mds-c, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.license=GPLv2) 2024-09-11T23:43:05.710 INFO:journalctl@ceph.mds.c.smithi136.stdout:Sep 11 23:43:05 smithi136 bash[74452]: ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mds-c 2024-09-11T23:43:05.963 INFO:journalctl@ceph.mds.c.smithi136.stdout:Sep 11 23:43:05 smithi136 podman[74464]: 2024-09-11 23:43:05.710893699 +0000 UTC m=+0.512279693 container remove a8bc986f7080e19e4c4f2a05c015fdb63f204e4d1aafe91156eb8d50f14786ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mds-c, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, io.buildah.version=1.37.2) 2024-09-11T23:43:05.967 DEBUG:teuthology.orchestra.run.smithi136:> sudo pkill -f 'journalctl -f -n 0 -u ceph-af7e6fc8-7092-11ef-bcea-c7b262605968@mds.c.service' 2024-09-11T23:43:06.003 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T23:43:06.004 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-09-11T23:43:06.004 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-09-11T23:43:06.004 DEBUG:teuthology.orchestra.run.smithi136:> sudo systemctl stop ceph-af7e6fc8-7092-11ef-bcea-c7b262605968@mds.f 2024-09-11T23:43:06.238 INFO:journalctl@ceph.mds.f.smithi136.stdout:Sep 11 23:43:06 smithi136 systemd[1]: Stopping Ceph mds.f for af7e6fc8-7092-11ef-bcea-c7b262605968... 2024-09-11T23:43:06.738 INFO:journalctl@ceph.mds.f.smithi136.stdout:Sep 11 23:43:06 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mds-f[64739]: 2024-09-11T23:43:06.321+0000 7fd07c64b640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T23:43:06.739 INFO:journalctl@ceph.mds.f.smithi136.stdout:Sep 11 23:43:06 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mds-f[64739]: 2024-09-11T23:43:06.321+0000 7fd07c64b640 -1 mds.f *** got signal Terminated *** 2024-09-11T23:43:09.488 INFO:journalctl@ceph.mds.f.smithi136.stdout:Sep 11 23:43:09 smithi136 podman[74572]: 2024-09-11 23:43:09.199329978 +0000 UTC m=+2.964847616 container died 64c00ef5e0989f7f79ee650d059f8fb5502a07b80d397cac1f969db4e5ea9301 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mds-f, GIT_CLEAN=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T23:43:09.896 INFO:journalctl@ceph.mds.f.smithi136.stdout:Sep 11 23:43:09 smithi136 podman[74572]: 2024-09-11 23:43:09.530304517 +0000 UTC m=+3.295822156 container cleanup 64c00ef5e0989f7f79ee650d059f8fb5502a07b80d397cac1f969db4e5ea9301 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mds-f, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3) 2024-09-11T23:43:09.896 INFO:journalctl@ceph.mds.f.smithi136.stdout:Sep 11 23:43:09 smithi136 bash[74572]: ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mds-f 2024-09-11T23:43:09.896 INFO:journalctl@ceph.mds.f.smithi136.stdout:Sep 11 23:43:09 smithi136 podman[74585]: 2024-09-11 23:43:09.698223007 +0000 UTC m=+0.494245350 container remove 64c00ef5e0989f7f79ee650d059f8fb5502a07b80d397cac1f969db4e5ea9301 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mds-f, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2) 2024-09-11T23:43:09.924 DEBUG:teuthology.orchestra.run.smithi136:> sudo pkill -f 'journalctl -f -n 0 -u ceph-af7e6fc8-7092-11ef-bcea-c7b262605968@mds.f.service' 2024-09-11T23:43:09.956 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T23:43:09.957 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-09-11T23:43:09.957 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-09-11T23:43:09.957 DEBUG:teuthology.orchestra.run.smithi136:> sudo systemctl stop ceph-af7e6fc8-7092-11ef-bcea-c7b262605968@mds.i 2024-09-11T23:43:10.238 INFO:journalctl@ceph.mds.i.smithi136.stdout:Sep 11 23:43:10 smithi136 systemd[1]: Stopping Ceph mds.i for af7e6fc8-7092-11ef-bcea-c7b262605968... 2024-09-11T23:43:10.738 INFO:journalctl@ceph.mds.i.smithi136.stdout:Sep 11 23:43:10 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mds-i[65341]: 2024-09-11T23:43:10.274+0000 7f88a2f7f640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T23:43:10.738 INFO:journalctl@ceph.mds.i.smithi136.stdout:Sep 11 23:43:10 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mds-i[65341]: 2024-09-11T23:43:10.274+0000 7f88a2f7f640 -1 mds.i *** got signal Terminated *** 2024-09-11T23:43:13.488 INFO:journalctl@ceph.mds.i.smithi136.stdout:Sep 11 23:43:13 smithi136 podman[74692]: 2024-09-11 23:43:13.195050859 +0000 UTC m=+3.020418985 container died 796ec4a7873a26da8a0bf790952430c64f6e5978969c10d8c0c7fb07e0972029 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mds-i, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-09-11T23:43:13.936 INFO:journalctl@ceph.mds.i.smithi136.stdout:Sep 11 23:43:13 smithi136 podman[74692]: 2024-09-11 23:43:13.517320318 +0000 UTC m=+3.342688449 container cleanup 796ec4a7873a26da8a0bf790952430c64f6e5978969c10d8c0c7fb07e0972029 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mds-i, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-09-11T23:43:13.936 INFO:journalctl@ceph.mds.i.smithi136.stdout:Sep 11 23:43:13 smithi136 bash[74692]: ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mds-i 2024-09-11T23:43:13.937 INFO:journalctl@ceph.mds.i.smithi136.stdout:Sep 11 23:43:13 smithi136 podman[74706]: 2024-09-11 23:43:13.718763806 +0000 UTC m=+0.517217225 container remove 796ec4a7873a26da8a0bf790952430c64f6e5978969c10d8c0c7fb07e0972029 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mds-i, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T23:43:13.950 DEBUG:teuthology.orchestra.run.smithi136:> sudo pkill -f 'journalctl -f -n 0 -u ceph-af7e6fc8-7092-11ef-bcea-c7b262605968@mds.i.service' 2024-09-11T23:43:13.984 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T23:43:13.984 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-09-11T23:43:13.984 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-09-11T23:43:13.984 DEBUG:teuthology.orchestra.run.smithi136:> sudo systemctl stop ceph-af7e6fc8-7092-11ef-bcea-c7b262605968@mds.l 2024-09-11T23:43:14.238 INFO:journalctl@ceph.mds.l.smithi136.stdout:Sep 11 23:43:14 smithi136 systemd[1]: Stopping Ceph mds.l for af7e6fc8-7092-11ef-bcea-c7b262605968... 2024-09-11T23:43:14.738 INFO:journalctl@ceph.mds.l.smithi136.stdout:Sep 11 23:43:14 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mds-l[64436]: 2024-09-11T23:43:14.320+0000 7f5cdc6e0640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T23:43:14.739 INFO:journalctl@ceph.mds.l.smithi136.stdout:Sep 11 23:43:14 smithi136 ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mds-l[64436]: 2024-09-11T23:43:14.320+0000 7f5cdc6e0640 -1 mds.l *** got signal Terminated *** 2024-09-11T23:43:17.470 INFO:journalctl@ceph.mds.l.smithi136.stdout:Sep 11 23:43:17 smithi136 podman[74812]: 2024-09-11 23:43:17.198743393 +0000 UTC m=+2.991672570 container died 959a71ce7433140d16d8904f42ea81d20e14a5d3efd9ab0225b2ec8e8520a1d8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mds-l, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-09-11T23:43:17.738 INFO:journalctl@ceph.mds.l.smithi136.stdout:Sep 11 23:43:17 smithi136 podman[74812]: 2024-09-11 23:43:17.554470316 +0000 UTC m=+3.347399487 container cleanup 959a71ce7433140d16d8904f42ea81d20e14a5d3efd9ab0225b2ec8e8520a1d8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mds-l, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , ceph=True, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T23:43:17.738 INFO:journalctl@ceph.mds.l.smithi136.stdout:Sep 11 23:43:17 smithi136 bash[74812]: ceph-af7e6fc8-7092-11ef-bcea-c7b262605968-mds-l 2024-09-11T23:43:17.967 DEBUG:teuthology.orchestra.run.smithi136:> sudo pkill -f 'journalctl -f -n 0 -u ceph-af7e6fc8-7092-11ef-bcea-c7b262605968@mds.l.service' 2024-09-11T23:43:17.999 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T23:43:17.999 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-09-11T23:43:17.999 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 --force --keep-logs 2024-09-11T23:43:18.212 INFO:teuthology.orchestra.run.smithi026.stdout:Deleting cluster with fsid: af7e6fc8-7092-11ef-bcea-c7b262605968 2024-09-11T23:43:20.758 DEBUG:teuthology.orchestra.run.smithi076:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 --force --keep-logs 2024-09-11T23:43:20.968 INFO:teuthology.orchestra.run.smithi076.stdout:Deleting cluster with fsid: af7e6fc8-7092-11ef-bcea-c7b262605968 2024-09-11T23:43:23.715 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 --force --keep-logs 2024-09-11T23:43:23.916 INFO:teuthology.orchestra.run.smithi136.stdout:Deleting cluster with fsid: af7e6fc8-7092-11ef-bcea-c7b262605968 2024-09-11T23:43:26.378 DEBUG:teuthology.orchestra.run.smithi026:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-11T23:43:26.412 DEBUG:teuthology.orchestra.run.smithi076:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-11T23:43:26.444 DEBUG:teuthology.orchestra.run.smithi136:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-11T23:43:26.470 INFO:tasks.cephadm:Archiving crash dumps... 2024-09-11T23:43:26.472 DEBUG:teuthology.misc:Transferring archived files from smithi026:/var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901457/remote/smithi026/crash 2024-09-11T23:43:26.473 DEBUG:teuthology.orchestra.run.smithi026:> sudo tar c -f - -C /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/crash -- . 2024-09-11T23:43:26.502 INFO:teuthology.orchestra.run.smithi026.stderr:tar: /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/crash: Cannot open: No such file or directory 2024-09-11T23:43:26.503 INFO:teuthology.orchestra.run.smithi026.stderr:tar: Error is not recoverable: exiting now 2024-09-11T23:43:26.505 DEBUG:teuthology.misc:Transferring archived files from smithi076:/var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901457/remote/smithi076/crash 2024-09-11T23:43:26.506 DEBUG:teuthology.orchestra.run.smithi076:> sudo tar c -f - -C /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/crash -- . 2024-09-11T23:43:26.545 INFO:teuthology.orchestra.run.smithi076.stderr:tar: /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/crash: Cannot open: No such file or directory 2024-09-11T23:43:26.560 INFO:teuthology.orchestra.run.smithi076.stderr:tar: Error is not recoverable: exiting now 2024-09-11T23:43:26.561 DEBUG:teuthology.misc:Transferring archived files from smithi136:/var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901457/remote/smithi136/crash 2024-09-11T23:43:26.562 DEBUG:teuthology.orchestra.run.smithi136:> sudo tar c -f - -C /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/crash -- . 2024-09-11T23:43:26.595 INFO:teuthology.orchestra.run.smithi136.stderr:tar: /var/lib/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/crash: Cannot open: No such file or directory 2024-09-11T23:43:26.595 INFO:teuthology.orchestra.run.smithi136.stderr:tar: Error is not recoverable: exiting now 2024-09-11T23:43:26.596 INFO:tasks.cephadm:Checking cluster log for badness... 2024-09-11T23:43:26.597 DEBUG:teuthology.orchestra.run.smithi026:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-09-11T23:43:26.641 INFO:teuthology.orchestra.run.smithi026.stdout:2024-09-11T23:20:09.534035+0000 mds.i (mds.1) 2 : cluster [WRN] Scrub error on inode 0x1000000020b (/volumes/qa/sv_1/9fc8f023-d646-4fec-98c7-1f3d972f60a9/client.0/tmp/blogbench-1.0/src/blogbench.c) see mds.i log and `damage ls` output for details 2024-09-11T23:43:26.641 WARNING:tasks.cephadm:Found errors (ERR|WRN|SEC) in cluster log 2024-09-11T23:43:26.641 INFO:tasks.cephadm:Compressing logs... 2024-09-11T23:43:26.641 DEBUG:teuthology.orchestra.run.smithi026:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-11T23:43:26.684 DEBUG:teuthology.orchestra.run.smithi076:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-11T23:43:26.689 DEBUG:teuthology.orchestra.run.smithi136:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-11T23:43:26.710 INFO:teuthology.orchestra.run.smithi026.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/ceph-osd.0.log 2024-09-11T23:43:26.710 INFO:teuthology.orchestra.run.smithi026.stderr:: No such file or directory 2024-09-11T23:43:26.711 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/ceph-osd.9.log 2024-09-11T23:43:26.711 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/ceph-volume.log 2024-09-11T23:43:26.711 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/ceph.audit.log 2024-09-11T23:43:26.712 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/ceph-mds.a.log 2024-09-11T23:43:26.712 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/ceph.cephadm.log 2024-09-11T23:43:26.713 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/ceph-mon.a.log 2024-09-11T23:43:26.714 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/ceph-mds.g.log 2024-09-11T23:43:26.715 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/ceph-mon.a.log: 84.8% -- replaced with /var/log/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/ceph.cephadm.log.gz 2024-09-11T23:43:26.715 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5/var/log/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/ceph-mds.g.log: --verbose -- /var/log/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/ceph-osd.6.log 2024-09-11T23:43:26.715 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/ceph.log 2024-09-11T23:43:26.716 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/ceph-osd.6.log: 90.4% -- replaced with /var/log/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/ceph.audit.log.gz 2024-09-11T23:43:26.721 INFO:teuthology.orchestra.run.smithi136.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-09-11T23:43:26.722 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/ceph-volume.log 2024-09-11T23:43:26.723 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/ceph.audit.log 2024-09-11T23:43:26.723 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/ceph-mds.i.log 2024-09-11T23:43:26.723 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/ceph-volume.log: /var/log/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/ceph-mon.c.log 2024-09-11T23:43:26.723 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/ceph.cephadm.log 2024-09-11T23:43:26.727 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/ceph-osd.11.log 2024-09-11T23:43:26.727 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/ceph-osd.2.log 2024-09-11T23:43:26.727 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/ceph-osd.5.log 2024-09-11T23:43:26.727 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/ceph.log 2024-09-11T23:43:26.727 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/ceph-mds.l.log 2024-09-11T23:43:26.727 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/ceph-osd.8.log 2024-09-11T23:43:26.727 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/ceph-mds.l.log: gzip -5 --verbose -- /var/log/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/ceph-mgr.z.log 2024-09-11T23:43:26.727 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/ceph-mds.c.log 2024-09-11T23:43:26.727 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/ceph-mds.f.log 2024-09-11T23:43:26.727 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-11T23:43:26.730 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/ceph-mds.d.log 2024-09-11T23:43:26.730 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/ceph.log: 90.9% -- replaced with /var/log/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/ceph-volume.log.gz 2024-09-11T23:43:26.730 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/ceph-mgr.x.log 2024-09-11T23:43:26.731 INFO:teuthology.orchestra.run.smithi076.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-09-11T23:43:26.731 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/ceph-osd.10.log 2024-09-11T23:43:26.731 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/ceph-osd.1.log 2024-09-11T23:43:26.731 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/ceph-osd.7.log 2024-09-11T23:43:26.731 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/ceph-osd.10.log: /var/log/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/ceph-volume.log 2024-09-11T23:43:26.731 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/ceph-mds.b.log 2024-09-11T23:43:26.732 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/ceph-mds.k.log 2024-09-11T23:43:26.732 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/ceph-osd.4.log 2024-09-11T23:43:26.734 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/ceph.audit.log 2024-09-11T23:43:26.734 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/ceph.cephadm.log 2024-09-11T23:43:26.734 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/ceph-mgr.y.log 2024-09-11T23:43:26.735 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/ceph.log 2024-09-11T23:43:26.735 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/ceph-mds.h.log 2024-09-11T23:43:26.735 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/ceph-mon.b.log 2024-09-11T23:43:26.735 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/ceph-mds.e.log 2024-09-11T23:43:26.735 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-11T23:43:26.736 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/ceph-mds.d.log: 89.3% -- replaced with /var/log/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/ceph.log.gz 2024-09-11T23:43:26.737 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/ceph-mds.j.log 2024-09-11T23:43:26.737 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/ceph-mgr.x.log: 94.6% -- replaced with /var/log/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/ceph-mds.d.log.gz 2024-09-11T23:43:26.737 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/ceph-osd.3.log 2024-09-11T23:43:26.744 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/ceph-mds.j.log: 94.6% -- replaced with /var/log/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/ceph-mds.j.log.gz 2024-09-11T23:43:26.744 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-11T23:43:26.777 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/ceph-osd.3.log: /var/log/ceph/cephadm.log: 94.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-11T23:43:26.788 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/ceph-mds.e.log: /var/log/ceph/cephadm.log: 82.6% -- replaced with /var/log/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/ceph.cephadm.log.gz 2024-09-11T23:43:26.822 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/ceph-mds.f.log: /var/log/ceph/cephadm.log: 92.5% -- replaced with /var/log/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/ceph-mgr.z.log.gz 2024-09-11T23:43:26.829 INFO:teuthology.orchestra.run.smithi136.stderr: 90.6% -- replaced with /var/log/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/ceph.audit.log.gz 2024-09-11T23:43:26.846 INFO:teuthology.orchestra.run.smithi076.stderr: 92.5% -- replaced with /var/log/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/ceph-mgr.y.log.gz 2024-09-11T23:43:26.863 INFO:teuthology.orchestra.run.smithi136.stderr: 82.6% -- replaced with /var/log/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/ceph.cephadm.log.gz 2024-09-11T23:43:26.864 INFO:teuthology.orchestra.run.smithi076.stderr: 91.0% -- replaced with /var/log/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/ceph-volume.log.gz 2024-09-11T23:43:26.870 INFO:teuthology.orchestra.run.smithi076.stderr: 90.5% -- replaced with /var/log/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/ceph.audit.log.gz 2024-09-11T23:43:26.928 INFO:teuthology.orchestra.run.smithi136.stderr: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-11T23:43:26.998 INFO:teuthology.orchestra.run.smithi136.stderr: 90.9% -- replaced with /var/log/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/ceph-volume.log.gz 2024-09-11T23:43:27.068 INFO:teuthology.orchestra.run.smithi076.stderr: 91.9% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-11T23:43:27.255 INFO:teuthology.orchestra.run.smithi076.stderr: 89.3% -- replaced with /var/log/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/ceph.log.gz 2024-09-11T23:43:27.285 INFO:teuthology.orchestra.run.smithi026.stderr: 90.6% -- replaced with /var/log/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/ceph-mgr.x.log.gz 2024-09-11T23:43:27.972 INFO:teuthology.orchestra.run.smithi136.stderr: 89.3% -- replaced with /var/log/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/ceph.log.gz 2024-09-11T23:43:28.627 INFO:teuthology.orchestra.run.smithi026.stderr: 90.2% -- replaced with /var/log/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/ceph-mon.a.log.gz 2024-09-11T23:43:31.751 INFO:teuthology.orchestra.run.smithi136.stderr: 91.7% -- replaced with /var/log/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/ceph-mon.c.log.gz 2024-09-11T23:43:32.893 INFO:teuthology.orchestra.run.smithi026.stderr: 95.2% -- replaced with /var/log/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/ceph-mds.a.log.gz 2024-09-11T23:43:36.004 INFO:teuthology.orchestra.run.smithi026.stderr: 95.5% -- replaced with /var/log/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/ceph-mds.g.log.gz 2024-09-11T23:43:52.747 INFO:teuthology.orchestra.run.smithi026.stderr: 93.4% -- replaced with /var/log/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/ceph-osd.9.log.gz 2024-09-11T23:43:57.264 INFO:teuthology.orchestra.run.smithi026.stderr: 93.4% -- replaced with /var/log/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/ceph-osd.6.log.gz 2024-09-11T23:44:00.270 INFO:teuthology.orchestra.run.smithi076.stderr: 91.7% -- replaced with /var/log/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/ceph-mon.b.log.gz 2024-09-11T23:44:00.624 INFO:teuthology.orchestra.run.smithi026.stderr: 93.4% -- replaced with /var/log/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/ceph-osd.3.log.gz 2024-09-11T23:44:00.822 INFO:teuthology.orchestra.run.smithi026.stderr: 93.4% -- replaced with /var/log/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/ceph-osd.0.log.gz 2024-09-11T23:44:00.824 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-11T23:44:00.824 INFO:teuthology.orchestra.run.smithi026.stderr:real 0m34.125s 2024-09-11T23:44:00.824 INFO:teuthology.orchestra.run.smithi026.stderr:user 2m17.846s 2024-09-11T23:44:00.824 INFO:teuthology.orchestra.run.smithi026.stderr:sys 0m4.437s 2024-09-11T23:44:16.650 INFO:teuthology.orchestra.run.smithi136.stderr: 95.2% -- replaced with /var/log/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/ceph-mds.f.log.gz 2024-09-11T23:45:30.513 INFO:teuthology.orchestra.run.smithi136.stderr: 95.5% -- replaced with /var/log/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/ceph-mds.l.log.gz 2024-09-11T23:45:33.639 INFO:teuthology.orchestra.run.smithi076.stderr: 93.7% -- replaced with /var/log/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/ceph-osd.4.log.gz 2024-09-11T23:47:08.301 INFO:teuthology.orchestra.run.smithi076.stderr: 95.4% -- replaced with /var/log/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/ceph-mds.k.log.gz 2024-09-11T23:47:22.192 INFO:teuthology.orchestra.run.smithi136.stderr: 93.4% -- replaced with /var/log/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/ceph-osd.8.log.gz 2024-09-11T23:47:50.630 INFO:teuthology.orchestra.run.smithi076.stderr: 93.4% -- replaced with /var/log/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/ceph-osd.10.log.gz 2024-09-11T23:48:12.378 INFO:teuthology.orchestra.run.smithi136.stderr: 93.5% -- replaced with /var/log/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/ceph-osd.11.log.gz 2024-09-11T23:49:05.814 INFO:teuthology.orchestra.run.smithi076.stderr: 93.4% -- replaced with /var/log/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/ceph-osd.1.log.gz 2024-09-11T23:49:56.110 INFO:teuthology.orchestra.run.smithi136.stderr: 93.4% -- replaced with /var/log/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/ceph-osd.2.log.gz 2024-09-11T23:49:58.222 INFO:teuthology.orchestra.run.smithi136.stderr: 93.4% -- replaced with /var/log/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/ceph-osd.5.log.gz 2024-09-11T23:50:35.339 INFO:teuthology.orchestra.run.smithi076.stderr: 93.5% -- replaced with /var/log/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/ceph-osd.7.log.gz 2024-09-12T00:01:57.893 INFO:teuthology.orchestra.run.smithi136.stderr: 94.8% -- replaced with /var/log/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/ceph-mds.c.log.gz 2024-09-12T00:04:46.006 INFO:teuthology.orchestra.run.smithi136.stderr: 94.1% -- replaced with /var/log/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/ceph-mds.i.log.gz 2024-09-12T00:04:46.008 INFO:teuthology.orchestra.run.smithi136.stderr: 2024-09-12T00:04:46.008 INFO:teuthology.orchestra.run.smithi136.stderr:real 21m19.304s 2024-09-12T00:04:46.008 INFO:teuthology.orchestra.run.smithi136.stderr:user 8m5.884s 2024-09-12T00:04:46.008 INFO:teuthology.orchestra.run.smithi136.stderr:sys 0m23.740s 2024-09-12T00:08:27.901 INFO:teuthology.orchestra.run.smithi076.stderr: 94.7% -- replaced with /var/log/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/ceph-mds.h.log.gz 2024-09-12T00:08:29.684 INFO:teuthology.orchestra.run.smithi076.stderr: 94.8% -- replaced with /var/log/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/ceph-mds.e.log.gz 2024-09-12T00:09:04.813 INFO:teuthology.orchestra.run.smithi076.stderr: 94.8% -- replaced with /var/log/ceph/af7e6fc8-7092-11ef-bcea-c7b262605968/ceph-mds.b.log.gz 2024-09-12T00:09:04.885 INFO:teuthology.orchestra.run.smithi076.stderr: 2024-09-12T00:09:04.885 INFO:teuthology.orchestra.run.smithi076.stderr:real 25m38.185s 2024-09-12T00:09:04.886 INFO:teuthology.orchestra.run.smithi076.stderr:user 8m46.220s 2024-09-12T00:09:04.886 INFO:teuthology.orchestra.run.smithi076.stderr:sys 0m27.226s 2024-09-12T00:09:04.886 INFO:tasks.cephadm:Archiving logs... 2024-09-12T00:09:04.887 DEBUG:teuthology.misc:Transferring archived files from smithi026:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901457/remote/smithi026/log 2024-09-12T00:09:04.889 DEBUG:teuthology.orchestra.run.smithi026:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-12T00:09:24.782 DEBUG:teuthology.misc:Transferring archived files from smithi076:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901457/remote/smithi076/log 2024-09-12T00:09:24.786 DEBUG:teuthology.orchestra.run.smithi076:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-12T00:10:38.526 DEBUG:teuthology.misc:Transferring archived files from smithi136:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901457/remote/smithi136/log 2024-09-12T00:10:38.528 DEBUG:teuthology.orchestra.run.smithi136:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-12T00:11:52.088 INFO:tasks.cephadm:Removing cluster... 2024-09-12T00:11:52.088 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 --force 2024-09-12T00:11:52.302 INFO:teuthology.orchestra.run.smithi026.stdout:Deleting cluster with fsid: af7e6fc8-7092-11ef-bcea-c7b262605968 2024-09-12T00:11:52.742 DEBUG:teuthology.orchestra.run.smithi076:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 --force 2024-09-12T00:11:53.317 INFO:teuthology.orchestra.run.smithi076.stdout:Deleting cluster with fsid: af7e6fc8-7092-11ef-bcea-c7b262605968 2024-09-12T00:11:54.022 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid af7e6fc8-7092-11ef-bcea-c7b262605968 --force 2024-09-12T00:11:54.501 INFO:teuthology.orchestra.run.smithi136.stdout:Deleting cluster with fsid: af7e6fc8-7092-11ef-bcea-c7b262605968 2024-09-12T00:11:55.151 INFO:tasks.cephadm:Removing cephadm ... 2024-09-12T00:11:55.152 DEBUG:teuthology.orchestra.run.smithi026:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-12T00:11:55.171 DEBUG:teuthology.orchestra.run.smithi076:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-12T00:11:55.191 DEBUG:teuthology.orchestra.run.smithi136:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-12T00:11:55.207 INFO:tasks.cephadm:Teardown complete 2024-09-12T00:11:55.207 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-12T00:11:55.219 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-12T00:11:55.219 DEBUG:teuthology.orchestra.run.smithi026:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-12T00:11:55.222 DEBUG:teuthology.orchestra.run.smithi076:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-12T00:11:55.233 DEBUG:teuthology.orchestra.run.smithi136:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-12T00:11:55.388 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-12T00:11:55.399 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-12T00:11:55.400 DEBUG:teuthology.orchestra.run.smithi026:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-12T00:11:55.404 DEBUG:teuthology.orchestra.run.smithi076:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-12T00:11:55.406 DEBUG:teuthology.orchestra.run.smithi136:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-12T00:11:55.419 INFO:teuthology.orchestra.run.smithi026.stderr:bash: line 1: ntpq: command not found 2024-09-12T00:11:55.421 INFO:teuthology.orchestra.run.smithi136.stderr:bash: line 1: ntpq: command not found 2024-09-12T00:11:55.424 INFO:teuthology.orchestra.run.smithi026.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-12T00:11:55.424 INFO:teuthology.orchestra.run.smithi026.stdout:=============================================================================== 2024-09-12T00:11:55.424 INFO:teuthology.orchestra.run.smithi026.stdout:^* hv01.front.sepia.ceph.com 3 6 377 56 +1629us[+1630us] +/- 47ms 2024-09-12T00:11:55.424 INFO:teuthology.orchestra.run.smithi026.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 55 -1325us[-1325us] +/- 63ms 2024-09-12T00:11:55.424 INFO:teuthology.orchestra.run.smithi026.stdout:^+ hv03.front.sepia.ceph.com 3 7 377 53 -1154us[-1154us] +/- 98ms 2024-09-12T00:11:55.425 INFO:teuthology.orchestra.run.smithi026.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T00:11:55.426 INFO:teuthology.orchestra.run.smithi076.stderr:bash: line 1: ntpq: command not found 2024-09-12T00:11:55.448 INFO:teuthology.orchestra.run.smithi136.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-12T00:11:55.448 INFO:teuthology.orchestra.run.smithi136.stdout:=============================================================================== 2024-09-12T00:11:55.448 INFO:teuthology.orchestra.run.smithi136.stdout:^* hv01.front.sepia.ceph.com 3 6 377 56 +1578us[+1624us] +/- 47ms 2024-09-12T00:11:55.449 INFO:teuthology.orchestra.run.smithi136.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 58 -1359us[-1314us] +/- 63ms 2024-09-12T00:11:55.449 INFO:teuthology.orchestra.run.smithi136.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 57 -1241us[-1196us] +/- 98ms 2024-09-12T00:11:55.449 INFO:teuthology.orchestra.run.smithi136.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T00:11:55.496 INFO:teuthology.orchestra.run.smithi076.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-12T00:11:55.496 INFO:teuthology.orchestra.run.smithi076.stdout:=============================================================================== 2024-09-12T00:11:55.497 INFO:teuthology.orchestra.run.smithi076.stdout:^* hv01.front.sepia.ceph.com 3 6 377 52 +1580us[+1577us] +/- 47ms 2024-09-12T00:11:55.497 INFO:teuthology.orchestra.run.smithi076.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 55 -1371us[-1374us] +/- 63ms 2024-09-12T00:11:55.497 INFO:teuthology.orchestra.run.smithi076.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 54 -1248us[-1251us] +/- 98ms 2024-09-12T00:11:55.497 INFO:teuthology.orchestra.run.smithi076.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T00:11:55.513 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-12T00:11:55.526 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-12T00:11:55.527 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-12T00:11:55.538 DEBUG:teuthology.orchestra.run.smithi026:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-12T00:11:55.628 DEBUG:teuthology.orchestra.run.smithi076:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-12T00:11:55.766 DEBUG:teuthology.orchestra.run.smithi136:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-12T00:11:55.904 DEBUG:teuthology.orchestra.run.smithi026:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-12T00:11:55.937 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T00:11:55.938 DEBUG:teuthology.orchestra.run.smithi076:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-12T00:11:55.969 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T00:11:55.970 DEBUG:teuthology.orchestra.run.smithi136:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-12T00:11:55.998 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T00:11:55.999 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-12T00:11:56.010 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-12T00:11:56.023 INFO:teuthology.task.internal:Duration was 4192.640613 seconds 2024-09-12T00:11:56.024 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-12T00:11:56.035 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-12T00:11:56.035 DEBUG:teuthology.orchestra.run.smithi026:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-12T00:11:56.038 DEBUG:teuthology.orchestra.run.smithi076:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-12T00:11:56.041 DEBUG:teuthology.orchestra.run.smithi136:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-12T00:11:56.081 INFO:teuthology.orchestra.run.smithi026.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-12T00:11:56.098 INFO:teuthology.orchestra.run.smithi136.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-12T00:11:56.114 INFO:teuthology.orchestra.run.smithi076.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-12T00:11:56.545 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-12T00:11:56.545 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi026.front.sepia.ceph.com 2024-09-12T00:11:56.546 DEBUG:teuthology.orchestra.run.smithi026:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-12T00:11:56.570 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi076.front.sepia.ceph.com 2024-09-12T00:11:56.570 DEBUG:teuthology.orchestra.run.smithi076:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-12T00:11:56.611 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi136.front.sepia.ceph.com 2024-09-12T00:11:56.612 DEBUG:teuthology.orchestra.run.smithi136:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-12T00:11:56.639 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-12T00:11:56.639 DEBUG:teuthology.orchestra.run.smithi026:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-12T00:11:56.642 DEBUG:teuthology.orchestra.run.smithi076:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-12T00:11:56.655 DEBUG:teuthology.orchestra.run.smithi136:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-12T00:11:56.708 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-12T00:11:56.709 DEBUG:teuthology.orchestra.run.smithi026:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-12T00:11:56.712 DEBUG:teuthology.orchestra.run.smithi076:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-12T00:11:56.726 DEBUG:teuthology.orchestra.run.smithi136:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-12T00:11:57.735 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-12T00:11:57.748 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-12T00:11:57.748 DEBUG:teuthology.orchestra.run.smithi026:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-12T00:11:57.774 DEBUG:teuthology.orchestra.run.smithi076:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-12T00:11:57.833 DEBUG:teuthology.orchestra.run.smithi136:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-12T00:11:57.864 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-12T00:11:57.877 DEBUG:teuthology.orchestra.run.smithi026:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-12T00:11:57.880 DEBUG:teuthology.orchestra.run.smithi076:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-12T00:11:57.882 DEBUG:teuthology.orchestra.run.smithi136:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-12T00:11:57.905 INFO:teuthology.orchestra.run.smithi026.stdout:kernel.core_pattern = core 2024-09-12T00:11:57.911 INFO:teuthology.orchestra.run.smithi076.stdout:kernel.core_pattern = core 2024-09-12T00:11:57.943 INFO:teuthology.orchestra.run.smithi136.stdout:kernel.core_pattern = core 2024-09-12T00:11:57.966 DEBUG:teuthology.orchestra.run.smithi026:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-12T00:11:57.981 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T00:11:57.981 DEBUG:teuthology.orchestra.run.smithi076:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-12T00:11:57.997 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T00:11:57.998 DEBUG:teuthology.orchestra.run.smithi136:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-12T00:11:58.023 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T00:11:58.023 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-12T00:11:58.038 INFO:teuthology.task.internal:Transferring archived files... 2024-09-12T00:11:58.039 DEBUG:teuthology.misc:Transferring archived files from smithi026:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901457/remote/smithi026 2024-09-12T00:11:58.040 DEBUG:teuthology.orchestra.run.smithi026:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-12T00:11:58.100 DEBUG:teuthology.misc:Transferring archived files from smithi076:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901457/remote/smithi076 2024-09-12T00:11:58.100 DEBUG:teuthology.orchestra.run.smithi076:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-12T00:11:58.140 DEBUG:teuthology.misc:Transferring archived files from smithi136:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901457/remote/smithi136 2024-09-12T00:11:58.140 DEBUG:teuthology.orchestra.run.smithi136:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-12T00:11:58.182 INFO:teuthology.task.internal:Removing archive directory... 2024-09-12T00:11:58.182 DEBUG:teuthology.orchestra.run.smithi026:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-12T00:11:58.185 DEBUG:teuthology.orchestra.run.smithi076:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-12T00:11:58.188 DEBUG:teuthology.orchestra.run.smithi136:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-12T00:11:58.234 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-12T00:11:58.249 INFO:teuthology.task.internal:Not uploading archives. 2024-09-12T00:11:58.249 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-12T00:11:58.315 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-12T00:11:58.316 DEBUG:teuthology.orchestra.run.smithi026:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-12T00:11:58.318 DEBUG:teuthology.orchestra.run.smithi076:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-12T00:11:58.321 DEBUG:teuthology.orchestra.run.smithi136:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-12T00:11:58.331 INFO:teuthology.orchestra.run.smithi026.stdout: 265340 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 12 00:11 /home/ubuntu/cephtest 2024-09-12T00:11:58.334 INFO:teuthology.orchestra.run.smithi076.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 12 00:11 /home/ubuntu/cephtest 2024-09-12T00:11:58.338 INFO:teuthology.orchestra.run.smithi136.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 12 00:11 /home/ubuntu/cephtest 2024-09-12T00:11:58.339 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-12T00:11:58.352 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-12T00:11:58.382 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/blogbench}} duration: 4192.6406128406525 failure_reason: 'error during scrub thrashing: rank damage found: {''backtrace''}' flavor: default owner: scheduled_teuthology@teuthology status: fail success: false 2024-09-12T00:11:58.382 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-12T00:11:58.486 INFO:teuthology.run:FAIL